2024-04-24T08:25:02.362 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-24T08:25:02.362 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/rishabh-2024-04-24_07:32:23-fs-rishabh-main-apr17-a654945-testing-default-smithi/7671216 --name rishabh-2024-04-24_07:32:23-fs-rishabh-main-apr17-a654945-testing-default-smithi --owner scheduled_rishabh@teuthology --verbose --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/suites/dbench}} -- /home/teuthworker/archive/rishabh-2024-04-24_07:32:23-fs-rishabh-main-apr17-a654945-testing-default-smithi/7671216/orig.config.yaml 2024-04-24T08:25:02.422 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-24T08:25:02.508 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2024-04-24_07:32:23-fs-rishabh-main-apr17-a654945-testing-default-smithi/7671216 branch: rishabh-main-apr17-a654945 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/suites/dbench}} email: null first_in_suite: false job_id: '7671216' kernel: flavor: default kdb: true sha1: 806ec36b21129f113ce356c007dd59694d340b37 last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true name: rishabh-2024-04-24_07:32:23-fs-rishabh-main-apr17-a654945-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: rishabh-main-apr17-a654945 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - slow metadata IO - SLOW_OPS - slow request sha1: 1f81fb397ae98da7563d451a78c61574c8f4e6e0 subvols: create: 2 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount install: ceph: flavor: default sha1: 1f81fb397ae98da7563d451a78c61574c8f4e6e0 kclient: client.0: mount_subvol_num: 1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: rishabh-main-apr17-a654945 sha1: 1f81fb397ae98da7563d451a78c61574c8f4e6e0 owner: scheduled_rishabh@teuthology priority: 75 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 7954 sha1: 1f81fb397ae98da7563d451a78c61574c8f4e6e0 sleep_before_teardown: 0 subset: 364/512 suite: fs suite_branch: rishabh-main-apr17-a654945 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_1f81fb397ae98da7563d451a78c61574c8f4e6e0/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 1f81fb397ae98da7563d451a78c61574c8f4e6e0 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi083.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBN0CclY6xwBKkTDnYoCHmRU6US1iwRUaHtWxWScJwFdW6vbqYpL5iTPgY/IStOEZqEQytMfRDSrD6ly+IGqpoLw= smithi138.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMbippNFPXqvRGlEyFz9OxiHoGjV3HYInY/bEfMEvHclwS46aMECzZQlVddaVAGWvk3UTQxHoTotU7jYrRybgZ8= smithi189.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLDz1dff8UPWDDhhdt+mKp29SBFeAd++C1OzFH+0BSbsAuDzBolYH2YUB2p2MYiBvDqNYdtNgjrSJNL7Hj7knNk= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - ceph-fuse: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - suites/dbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-24_07:32:23 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-24T08:25:02.509 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_1f81fb397ae98da7563d451a78c61574c8f4e6e0/qa; will attempt to use it 2024-04-24T08:25:02.509 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_1f81fb397ae98da7563d451a78c61574c8f4e6e0/qa/tasks 2024-04-24T08:25:02.510 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-24T08:25:02.511 INFO:teuthology.task.internal:Checking packages... 2024-04-24T08:25:02.535 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '1f81fb397ae98da7563d451a78c61574c8f4e6e0' 2024-04-24T08:25:02.535 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-24T08:25:02.535 INFO:teuthology.packaging:ref: None 2024-04-24T08:25:02.535 INFO:teuthology.packaging:tag: None 2024-04-24T08:25:02.535 INFO:teuthology.packaging:branch: rishabh-main-apr17-a654945 2024-04-24T08:25:02.535 INFO:teuthology.packaging:sha1: 1f81fb397ae98da7563d451a78c61574c8f4e6e0 2024-04-24T08:25:02.535 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=rishabh-main-apr17-a654945 2024-04-24T08:25:02.759 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3116.g1f81fb39 2024-04-24T08:25:02.761 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-24T08:25:02.767 INFO:teuthology.task.internal:no buildpackages task found 2024-04-24T08:25:02.767 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-24T08:25:02.811 INFO:teuthology.task.internal:Saving configuration 2024-04-24T08:25:02.832 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-24T08:25:02.837 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-24T08:25:02.856 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi083.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-24_07:32:23-fs-rishabh-main-apr17-a654945-testing-default-smithi/7671216', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-24 08:19:54.065672', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBN0CclY6xwBKkTDnYoCHmRU6US1iwRUaHtWxWScJwFdW6vbqYpL5iTPgY/IStOEZqEQytMfRDSrD6ly+IGqpoLw='} 2024-04-24T08:25:02.876 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi138.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-24_07:32:23-fs-rishabh-main-apr17-a654945-testing-default-smithi/7671216', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-24 08:19:54.064423', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMbippNFPXqvRGlEyFz9OxiHoGjV3HYInY/bEfMEvHclwS46aMECzZQlVddaVAGWvk3UTQxHoTotU7jYrRybgZ8='} 2024-04-24T08:25:02.894 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi189.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-24_07:32:23-fs-rishabh-main-apr17-a654945-testing-default-smithi/7671216', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-24 08:19:54.062842', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLDz1dff8UPWDDhhdt+mKp29SBFeAd++C1OzFH+0BSbsAuDzBolYH2YUB2p2MYiBvDqNYdtNgjrSJNL7Hj7knNk='} 2024-04-24T08:25:02.894 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-24T08:25:02.900 INFO:teuthology.task.internal:roles: ubuntu@smithi083.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-04-24T08:25:02.900 INFO:teuthology.task.internal:roles: ubuntu@smithi138.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-04-24T08:25:02.900 INFO:teuthology.task.internal:roles: ubuntu@smithi189.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-04-24T08:25:02.900 INFO:teuthology.run_tasks:Running task console_log... 2024-04-24T08:25:03.002 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f27d3cde160>, signals=[15]) 2024-04-24T08:25:03.002 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-24T08:25:03.008 INFO:teuthology.task.internal:Opening connections... 2024-04-24T08:25:03.009 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi083.front.sepia.ceph.com 2024-04-24T08:25:03.010 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi083.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T08:25:03.087 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi138.front.sepia.ceph.com 2024-04-24T08:25:03.088 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T08:25:03.166 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi189.front.sepia.ceph.com 2024-04-24T08:25:03.167 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi189.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T08:25:03.240 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-24T08:25:03.247 DEBUG:teuthology.orchestra.run.smithi083:> uname -m 2024-04-24T08:25:03.265 INFO:teuthology.orchestra.run.smithi083.stdout:x86_64 2024-04-24T08:25:03.265 DEBUG:teuthology.orchestra.run.smithi083:> cat /etc/os-release 2024-04-24T08:25:03.321 INFO:teuthology.orchestra.run.smithi083.stdout:NAME="CentOS Stream" 2024-04-24T08:25:03.321 INFO:teuthology.orchestra.run.smithi083.stdout:VERSION="9" 2024-04-24T08:25:03.321 INFO:teuthology.orchestra.run.smithi083.stdout:ID="centos" 2024-04-24T08:25:03.321 INFO:teuthology.orchestra.run.smithi083.stdout:ID_LIKE="rhel fedora" 2024-04-24T08:25:03.321 INFO:teuthology.orchestra.run.smithi083.stdout:VERSION_ID="9" 2024-04-24T08:25:03.321 INFO:teuthology.orchestra.run.smithi083.stdout:PLATFORM_ID="platform:el9" 2024-04-24T08:25:03.321 INFO:teuthology.orchestra.run.smithi083.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-24T08:25:03.321 INFO:teuthology.orchestra.run.smithi083.stdout:ANSI_COLOR="0;31" 2024-04-24T08:25:03.322 INFO:teuthology.orchestra.run.smithi083.stdout:LOGO="fedora-logo-icon" 2024-04-24T08:25:03.322 INFO:teuthology.orchestra.run.smithi083.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-24T08:25:03.322 INFO:teuthology.orchestra.run.smithi083.stdout:HOME_URL="https://centos.org/" 2024-04-24T08:25:03.322 INFO:teuthology.orchestra.run.smithi083.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-24T08:25:03.322 INFO:teuthology.orchestra.run.smithi083.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-24T08:25:03.322 INFO:teuthology.orchestra.run.smithi083.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-24T08:25:03.322 INFO:teuthology.lock.ops:Updating smithi083.front.sepia.ceph.com on lock server 2024-04-24T08:25:03.343 DEBUG:teuthology.orchestra.run.smithi138:> uname -m 2024-04-24T08:25:03.361 INFO:teuthology.orchestra.run.smithi138.stdout:x86_64 2024-04-24T08:25:03.361 DEBUG:teuthology.orchestra.run.smithi138:> cat /etc/os-release 2024-04-24T08:25:03.417 INFO:teuthology.orchestra.run.smithi138.stdout:NAME="CentOS Stream" 2024-04-24T08:25:03.417 INFO:teuthology.orchestra.run.smithi138.stdout:VERSION="9" 2024-04-24T08:25:03.417 INFO:teuthology.orchestra.run.smithi138.stdout:ID="centos" 2024-04-24T08:25:03.417 INFO:teuthology.orchestra.run.smithi138.stdout:ID_LIKE="rhel fedora" 2024-04-24T08:25:03.417 INFO:teuthology.orchestra.run.smithi138.stdout:VERSION_ID="9" 2024-04-24T08:25:03.417 INFO:teuthology.orchestra.run.smithi138.stdout:PLATFORM_ID="platform:el9" 2024-04-24T08:25:03.417 INFO:teuthology.orchestra.run.smithi138.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-24T08:25:03.418 INFO:teuthology.orchestra.run.smithi138.stdout:ANSI_COLOR="0;31" 2024-04-24T08:25:03.418 INFO:teuthology.orchestra.run.smithi138.stdout:LOGO="fedora-logo-icon" 2024-04-24T08:25:03.418 INFO:teuthology.orchestra.run.smithi138.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-24T08:25:03.418 INFO:teuthology.orchestra.run.smithi138.stdout:HOME_URL="https://centos.org/" 2024-04-24T08:25:03.418 INFO:teuthology.orchestra.run.smithi138.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-24T08:25:03.418 INFO:teuthology.orchestra.run.smithi138.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-24T08:25:03.418 INFO:teuthology.orchestra.run.smithi138.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-24T08:25:03.418 INFO:teuthology.lock.ops:Updating smithi138.front.sepia.ceph.com on lock server 2024-04-24T08:25:03.443 DEBUG:teuthology.orchestra.run.smithi189:> uname -m 2024-04-24T08:25:03.462 INFO:teuthology.orchestra.run.smithi189.stdout:x86_64 2024-04-24T08:25:03.462 DEBUG:teuthology.orchestra.run.smithi189:> cat /etc/os-release 2024-04-24T08:25:03.520 INFO:teuthology.orchestra.run.smithi189.stdout:NAME="CentOS Stream" 2024-04-24T08:25:03.520 INFO:teuthology.orchestra.run.smithi189.stdout:VERSION="9" 2024-04-24T08:25:03.520 INFO:teuthology.orchestra.run.smithi189.stdout:ID="centos" 2024-04-24T08:25:03.520 INFO:teuthology.orchestra.run.smithi189.stdout:ID_LIKE="rhel fedora" 2024-04-24T08:25:03.520 INFO:teuthology.orchestra.run.smithi189.stdout:VERSION_ID="9" 2024-04-24T08:25:03.520 INFO:teuthology.orchestra.run.smithi189.stdout:PLATFORM_ID="platform:el9" 2024-04-24T08:25:03.520 INFO:teuthology.orchestra.run.smithi189.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-24T08:25:03.520 INFO:teuthology.orchestra.run.smithi189.stdout:ANSI_COLOR="0;31" 2024-04-24T08:25:03.521 INFO:teuthology.orchestra.run.smithi189.stdout:LOGO="fedora-logo-icon" 2024-04-24T08:25:03.521 INFO:teuthology.orchestra.run.smithi189.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-24T08:25:03.521 INFO:teuthology.orchestra.run.smithi189.stdout:HOME_URL="https://centos.org/" 2024-04-24T08:25:03.521 INFO:teuthology.orchestra.run.smithi189.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-24T08:25:03.521 INFO:teuthology.orchestra.run.smithi189.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-24T08:25:03.521 INFO:teuthology.orchestra.run.smithi189.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-24T08:25:03.521 INFO:teuthology.lock.ops:Updating smithi189.front.sepia.ceph.com on lock server 2024-04-24T08:25:03.543 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-24T08:25:03.552 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-24T08:25:03.559 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-24T08:25:03.559 DEBUG:teuthology.orchestra.run.smithi083:> test '!' -e /home/ubuntu/cephtest 2024-04-24T08:25:03.562 DEBUG:teuthology.orchestra.run.smithi138:> test '!' -e /home/ubuntu/cephtest 2024-04-24T08:25:03.564 DEBUG:teuthology.orchestra.run.smithi189:> test '!' -e /home/ubuntu/cephtest 2024-04-24T08:25:03.580 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-24T08:25:03.587 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-24T08:25:03.587 DEBUG:teuthology.orchestra.run.smithi083:> test -z $(ls -A /var/lib/ceph) 2024-04-24T08:25:03.619 DEBUG:teuthology.orchestra.run.smithi138:> test -z $(ls -A /var/lib/ceph) 2024-04-24T08:25:03.621 DEBUG:teuthology.orchestra.run.smithi189:> test -z $(ls -A /var/lib/ceph) 2024-04-24T08:25:03.719 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-24T08:25:03.798 INFO:teuthology.run_tasks:Running task kernel... 2024-04-24T08:25:03.813 INFO:teuthology.task.kernel:normalize config orig: {'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'} 2024-04-24T08:25:03.813 INFO:teuthology.task.kernel:config {'mon.a': {'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'}, 'mon.b': {'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'}, 'mon.c': {'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'}}, timeout 300 2024-04-24T08:25:03.813 DEBUG:teuthology.orchestra.run.smithi083:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-24T08:25:03.814 DEBUG:teuthology.orchestra.run.smithi138:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-24T08:25:03.814 DEBUG:teuthology.orchestra.run.smithi189:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-24T08:25:03.830 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T08:25:03.831 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T08:25:03.831 DEBUG:teuthology.task.kernel:sha1 for mon.b is 806ec36b21129f113ce356c007dd59694d340b37 2024-04-24T08:25:03.831 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-04-24T08:25:03.832 DEBUG:teuthology.orchestra.run.smithi138:> uname -r 2024-04-24T08:25:03.832 DEBUG:teuthology.task.kernel:sha1 for mon.a is 806ec36b21129f113ce356c007dd59694d340b37 2024-04-24T08:25:03.832 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-04-24T08:25:03.832 DEBUG:teuthology.orchestra.run.smithi083:> uname -r 2024-04-24T08:25:03.833 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T08:25:03.833 DEBUG:teuthology.task.kernel:sha1 for mon.c is 806ec36b21129f113ce356c007dd59694d340b37 2024-04-24T08:25:03.833 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-04-24T08:25:03.833 DEBUG:teuthology.orchestra.run.smithi189:> uname -r 2024-04-24T08:25:03.888 INFO:teuthology.orchestra.run.smithi083.stdout:5.14.0-437.el9.x86_64 2024-04-24T08:25:03.889 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs 806ec36b21129f113ce356c007dd59694d340b37 2024-04-24T08:25:03.889 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-24T08:25:03.889 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-24T08:25:03.894 INFO:teuthology.orchestra.run.smithi189.stdout:5.14.0-437.el9.x86_64 2024-04-24T08:25:03.894 INFO:teuthology.orchestra.run.smithi138.stdout:5.14.0-437.el9.x86_64 2024-04-24T08:25:03.894 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs 806ec36b21129f113ce356c007dd59694d340b37 2024-04-24T08:25:03.894 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-24T08:25:03.894 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-24T08:25:03.895 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs 806ec36b21129f113ce356c007dd59694d340b37 2024-04-24T08:25:03.895 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-24T08:25:03.896 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-24T08:25:04.113 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum upgrade -y linux-firmware 2024-04-24T08:25:04.128 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum upgrade -y linux-firmware 2024-04-24T08:25:04.132 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum upgrade -y linux-firmware 2024-04-24T08:25:06.837 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS Stream 9 - BaseOS 8.0 MB/s | 8.0 MB 00:01 2024-04-24T08:25:08.649 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS Stream 9 - AppStream 25 MB/s | 19 MB 00:00 2024-04-24T08:25:10.986 INFO:teuthology.orchestra.run.smithi189.stdout:CentOS Stream 9 - BaseOS 1.5 MB/s | 8.0 MB 00:05 2024-04-24T08:25:13.612 INFO:teuthology.orchestra.run.smithi189.stdout:CentOS Stream 9 - AppStream 12 MB/s | 19 MB 00:01 2024-04-24T08:25:13.688 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS Stream 9 - CRB 4.3 MB/s | 6.2 MB 00:01 2024-04-24T08:25:15.229 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - BaseOS 869 kB/s | 8.0 MB 00:09 2024-04-24T08:25:15.347 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS Stream 9 - Extras packages 48 kB/s | 16 kB 00:00 2024-04-24T08:25:17.026 INFO:teuthology.orchestra.run.smithi083.stdout:Extra Packages for Enterprise Linux 13 MB/s | 21 MB 00:01 2024-04-24T08:25:17.796 INFO:teuthology.orchestra.run.smithi189.stdout:CentOS Stream 9 - CRB 10 MB/s | 6.2 MB 00:00 2024-04-24T08:25:20.696 INFO:teuthology.orchestra.run.smithi189.stdout:CentOS Stream 9 - Extras packages 10 kB/s | 16 kB 00:01 2024-04-24T08:25:22.014 INFO:teuthology.orchestra.run.smithi189.stdout:Extra Packages for Enterprise Linux 18 MB/s | 21 MB 00:01 2024-04-24T08:25:22.140 INFO:teuthology.orchestra.run.smithi083.stdout:lab-extras 20 kB/s | 1.7 kB 00:00 2024-04-24T08:25:24.085 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2024-04-24T08:25:24.086 INFO:teuthology.orchestra.run.smithi083.stdout:Nothing to do. 2024-04-24T08:25:24.086 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2024-04-24T08:25:24.190 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on mon.a... 2024-04-24T08:25:24.193 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-24T08:25:24.408 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ 2024-04-24T08:25:24.409 DEBUG:teuthology.orchestra.run.smithi083:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ --input-file=- 2024-04-24T08:25:24.411 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi083.front.sepia.ceph.com to complete... 2024-04-24T08:25:27.072 INFO:teuthology.orchestra.run.smithi189.stdout:lab-extras 13 kB/s | 1.7 kB 00:00 2024-04-24T08:25:29.042 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2024-04-24T08:25:29.043 INFO:teuthology.orchestra.run.smithi189.stdout:Nothing to do. 2024-04-24T08:25:29.043 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2024-04-24T08:25:29.151 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on mon.c... 2024-04-24T08:25:29.151 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-24T08:25:29.372 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ 2024-04-24T08:25:29.372 DEBUG:teuthology.orchestra.run.smithi189:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ --input-file=- 2024-04-24T08:25:29.374 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi189.front.sepia.ceph.com to complete... 2024-04-24T08:25:30.920 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - AppStream 1.3 MB/s | 19 MB 00:14 2024-04-24T08:25:36.046 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - CRB 3.9 MB/s | 6.2 MB 00:01 2024-04-24T08:25:37.688 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - Extras packages 53 kB/s | 16 kB 00:00 2024-04-24T08:25:38.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24 08:25:38 URL:https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm [357367792/357367792] -> "/tmp/kernel.x86_64.rpm" [1] 2024-04-24T08:25:38.376 INFO:teuthology.orchestra.run.smithi083.stderr:FINISHED --2024-04-24 08:25:38-- 2024-04-24T08:25:38.377 INFO:teuthology.orchestra.run.smithi083.stderr:Total wall clock time: 14s 2024-04-24T08:25:38.377 INFO:teuthology.orchestra.run.smithi083.stderr:Downloaded: 1 files, 341M in 14s (24.9 MB/s) 2024-04-24T08:25:38.380 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on mon.a... 2024-04-24T08:25:38.380 DEBUG:teuthology.orchestra.run.smithi083:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-04-24T08:25:39.150 INFO:teuthology.orchestra.run.smithi138.stdout:Extra Packages for Enterprise Linux 16 MB/s | 21 MB 00:01 2024-04-24T08:25:40.559 INFO:teuthology.orchestra.run.smithi083.stdout:Verifying... ######################################## 2024-04-24T08:25:40.706 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing... ######################################## 2024-04-24T08:25:40.803 INFO:teuthology.orchestra.run.smithi083.stdout:Updating / installing... 2024-04-24T08:25:44.189 INFO:teuthology.orchestra.run.smithi138.stdout:lab-extras 26 kB/s | 1.7 kB 00:00 2024-04-24T08:25:46.149 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-24T08:25:46.150 INFO:teuthology.orchestra.run.smithi138.stdout:Nothing to do. 2024-04-24T08:25:46.150 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-04-24T08:25:46.256 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on mon.b... 2024-04-24T08:25:46.256 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-24T08:25:46.478 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ 2024-04-24T08:25:46.478 DEBUG:teuthology.orchestra.run.smithi138:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ --input-file=- 2024-04-24T08:25:46.480 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi138.front.sepia.ceph.com to complete... 2024-04-24T08:25:48.787 INFO:teuthology.orchestra.run.smithi083.stdout:kernel-6.9.0_rc3_g806ec36b2112-1 ######################################## 2024-04-24T08:26:11.131 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24 08:26:11 URL:https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm [357367792/357367792] -> "/tmp/kernel.x86_64.rpm" [1] 2024-04-24T08:26:11.132 INFO:teuthology.orchestra.run.smithi189.stderr:FINISHED --2024-04-24 08:26:11-- 2024-04-24T08:26:11.132 INFO:teuthology.orchestra.run.smithi189.stderr:Total wall clock time: 42s 2024-04-24T08:26:11.132 INFO:teuthology.orchestra.run.smithi189.stderr:Downloaded: 1 files, 341M in 41s (8.27 MB/s) 2024-04-24T08:26:11.134 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on mon.c... 2024-04-24T08:26:11.135 DEBUG:teuthology.orchestra.run.smithi189:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-04-24T08:26:13.234 INFO:teuthology.orchestra.run.smithi189.stdout:Verifying... ######################################## 2024-04-24T08:26:13.430 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing... ######################################## 2024-04-24T08:26:13.550 INFO:teuthology.orchestra.run.smithi189.stdout:Updating / installing... 2024-04-24T08:26:21.571 INFO:teuthology.orchestra.run.smithi189.stdout:kernel-6.9.0_rc3_g806ec36b2112-1 ######################################## 2024-04-24T08:26:25.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24 08:26:25 URL:https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm [357367792/357367792] -> "/tmp/kernel.x86_64.rpm" [1] 2024-04-24T08:26:25.708 INFO:teuthology.orchestra.run.smithi138.stderr:FINISHED --2024-04-24 08:26:25-- 2024-04-24T08:26:25.708 INFO:teuthology.orchestra.run.smithi138.stderr:Total wall clock time: 39s 2024-04-24T08:26:25.708 INFO:teuthology.orchestra.run.smithi138.stderr:Downloaded: 1 files, 341M in 38s (8.87 MB/s) 2024-04-24T08:26:25.711 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on mon.b... 2024-04-24T08:26:25.712 DEBUG:teuthology.orchestra.run.smithi138:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-04-24T08:26:27.799 INFO:teuthology.orchestra.run.smithi138.stdout:Verifying... ######################################## 2024-04-24T08:26:27.940 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing... ######################################## 2024-04-24T08:26:28.062 INFO:teuthology.orchestra.run.smithi138.stdout:Updating / installing... 2024-04-24T08:26:28.277 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi083.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-04-24T08:26:28.277 DEBUG:teuthology.orchestra.run.smithi083:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-04-24T08:26:28.351 INFO:teuthology.orchestra.run.smithi083.stdout:/boot/System.map-6.9.0-rc3-g806ec36b2112 2024-04-24T08:26:28.351 INFO:teuthology.orchestra.run.smithi083.stdout:/boot/config-6.9.0-rc3-g806ec36b2112 2024-04-24T08:26:28.351 INFO:teuthology.orchestra.run.smithi083.stdout:/boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-24T08:26:28.351 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/System.map 2024-04-24T08:26:28.352 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/config 2024-04-24T08:26:28.352 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel 2024-04-24T08:26:28.352 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch 2024-04-24T08:26:28.352 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86 2024-04-24T08:26:28.352 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto 2024-04-24T08:26:28.352 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-24T08:26:28.352 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-24T08:26:28.352 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-24T08:26:28.352 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-24T08:26:28.352 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-24T08:26:28.352 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-24T08:26:28.352 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-24T08:26:28.352 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32c-intel.ko 2024-04-24T08:26:28.352 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-24T08:26:28.353 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-24T08:26:28.353 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-24T08:26:28.353 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-24T08:26:28.353 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-24T08:26:28.353 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-24T08:26:28.353 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-24T08:26:28.353 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-24T08:26:28.353 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-24T08:26:28.353 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-24T08:26:28.353 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-24T08:26:28.353 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-24T08:26:28.353 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel 2024-04-24T08:26:28.353 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu 2024-04-24T08:26:28.353 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce 2024-04-24T08:26:28.353 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-24T08:26:28.353 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm 2024-04-24T08:26:28.354 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-24T08:26:28.354 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-24T08:26:28.354 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-04-24T08:26:28.354 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto 2024-04-24T08:26:28.354 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-04-24T08:26:28.354 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-04-24T08:26:28.354 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-04-24T08:26:28.354 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx 2024-04-24T08:26:28.354 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-04-24T08:26:28.354 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-04-24T08:26:28.354 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-24T08:26:28.354 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-04-24T08:26:28.354 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-04-24T08:26:28.355 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-04-24T08:26:28.355 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-04-24T08:26:28.355 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-04-24T08:26:28.355 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-04-24T08:26:28.355 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-04-24T08:26:28.355 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-04-24T08:26:28.355 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-04-24T08:26:28.355 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-04-24T08:26:28.355 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-04-24T08:26:28.355 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-04-24T08:26:28.355 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-04-24T08:26:28.355 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-04-24T08:26:28.355 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-04-24T08:26:28.355 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-04-24T08:26:28.355 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-04-24T08:26:28.356 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-04-24T08:26:28.356 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-04-24T08:26:28.356 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-04-24T08:26:28.356 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-04-24T08:26:28.356 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-04-24T08:26:28.356 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-04-24T08:26:28.356 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-04-24T08:26:28.356 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-04-24T08:26:28.356 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-04-24T08:26:28.356 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-04-24T08:26:28.356 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-04-24T08:26:28.356 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-04-24T08:26:28.356 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-04-24T08:26:28.356 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-04-24T08:26:28.357 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-04-24T08:26:28.357 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-04-24T08:26:28.357 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-04-24T08:26:28.357 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-04-24T08:26:28.357 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-04-24T08:26:28.357 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-04-24T08:26:28.357 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-04-24T08:26:28.357 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-04-24T08:26:28.357 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-04-24T08:26:28.357 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-04-24T08:26:28.357 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-04-24T08:26:28.357 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers 2024-04-24T08:26:28.357 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi 2024-04-24T08:26:28.357 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-24T08:26:28.357 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-04-24T08:26:28.357 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-04-24T08:26:28.358 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-04-24T08:26:28.358 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-04-24T08:26:28.358 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-04-24T08:26:28.358 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-04-24T08:26:28.358 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata 2024-04-24T08:26:28.358 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-04-24T08:26:28.358 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-04-24T08:26:28.358 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ata_generic.ko 2024-04-24T08:26:28.358 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-04-24T08:26:28.358 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-04-24T08:26:28.358 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-04-24T08:26:28.358 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-04-24T08:26:28.358 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-04-24T08:26:28.358 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-04-24T08:26:28.359 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-04-24T08:26:28.359 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-04-24T08:26:28.359 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-04-24T08:26:28.359 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-04-24T08:26:28.359 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-04-24T08:26:28.359 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-04-24T08:26:28.359 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-04-24T08:26:28.359 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-24T08:26:28.359 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-24T08:26:28.359 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-04-24T08:26:28.359 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-04-24T08:26:28.359 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-04-24T08:26:28.359 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-04-24T08:26:28.359 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-04-24T08:26:28.359 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-04-24T08:26:28.359 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-04-24T08:26:28.360 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-04-24T08:26:28.360 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-04-24T08:26:28.360 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-04-24T08:26:28.360 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-04-24T08:26:28.360 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-04-24T08:26:28.360 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-04-24T08:26:28.360 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-24T08:26:28.360 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-24T08:26:28.360 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-04-24T08:26:28.360 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-04-24T08:26:28.360 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-04-24T08:26:28.360 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-04-24T08:26:28.360 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-04-24T08:26:28.360 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sis.ko 2024-04-24T08:26:28.361 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-04-24T08:26:28.361 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-04-24T08:26:28.361 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-04-24T08:26:28.361 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-04-24T08:26:28.361 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-04-24T08:26:28.361 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-04-24T08:26:28.361 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-04-24T08:26:28.361 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-04-24T08:26:28.361 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-04-24T08:26:28.361 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-04-24T08:26:28.361 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-04-24T08:26:28.361 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-04-24T08:26:28.361 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-04-24T08:26:28.361 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-04-24T08:26:28.361 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-04-24T08:26:28.362 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-04-24T08:26:28.362 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-04-24T08:26:28.362 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay 2024-04-24T08:26:28.362 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-04-24T08:26:28.362 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base 2024-04-24T08:26:28.362 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap 2024-04-24T08:26:28.362 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-i2c.ko 2024-04-24T08:26:28.362 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-mmio.ko 2024-04-24T08:26:28.362 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma 2024-04-24T08:26:28.362 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-04-24T08:26:28.362 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block 2024-04-24T08:26:28.362 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe 2024-04-24T08:26:28.362 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-04-24T08:26:28.362 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/brd.ko 2024-04-24T08:26:28.362 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd 2024-04-24T08:26:28.363 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-04-24T08:26:28.363 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-04-24T08:26:28.363 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/loop.ko 2024-04-24T08:26:28.363 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx 2024-04-24T08:26:28.363 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-24T08:26:28.363 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-04-24T08:26:28.363 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk 2024-04-24T08:26:28.363 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-04-24T08:26:28.363 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-04-24T08:26:28.363 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-04-24T08:26:28.363 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/virtio_blk.ko 2024-04-24T08:26:28.363 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback 2024-04-24T08:26:28.363 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-24T08:26:28.363 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkfront.ko 2024-04-24T08:26:28.363 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram 2024-04-24T08:26:28.363 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-04-24T08:26:28.364 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char 2024-04-24T08:26:28.364 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-04-24T08:26:28.364 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random 2024-04-24T08:26:28.364 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-24T08:26:28.364 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-24T08:26:28.364 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-24T08:26:28.364 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-04-24T08:26:28.364 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-24T08:26:28.364 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi 2024-04-24T08:26:28.364 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-24T08:26:28.364 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-24T08:26:28.364 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-24T08:26:28.364 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-24T08:26:28.364 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-24T08:26:28.364 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-24T08:26:28.364 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-04-24T08:26:28.365 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave 2024-04-24T08:26:28.365 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-04-24T08:26:28.365 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-04-24T08:26:28.365 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-04-24T08:26:28.365 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm 2024-04-24T08:26:28.365 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm.ko 2024-04-24T08:26:28.365 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-24T08:26:28.365 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-24T08:26:28.365 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-24T08:26:28.365 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis.ko 2024-04-24T08:26:28.365 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-04-24T08:26:28.365 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/uv_mmtimer.ko 2024-04-24T08:26:28.365 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/virtio_console.ko 2024-04-24T08:26:28.365 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq 2024-04-24T08:26:28.365 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-04-24T08:26:28.366 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-24T08:26:28.366 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-24T08:26:28.366 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-04-24T08:26:28.366 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/powernow-k8.ko 2024-04-24T08:26:28.366 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-24T08:26:28.366 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto 2024-04-24T08:26:28.366 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp 2024-04-24T08:26:28.366 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-24T08:26:28.366 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-04-24T08:26:28.366 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel 2024-04-24T08:26:28.366 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat 2024-04-24T08:26:28.366 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common 2024-04-24T08:26:28.366 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-24T08:26:28.366 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-04-24T08:26:28.367 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-24T08:26:28.367 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-04-24T08:26:28.367 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-04-24T08:26:28.367 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio 2024-04-24T08:26:28.367 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-24T08:26:28.367 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca 2024-04-24T08:26:28.367 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-04-24T08:26:28.367 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq 2024-04-24T08:26:28.367 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-04-24T08:26:28.367 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma 2024-04-24T08:26:28.367 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw 2024-04-24T08:26:28.367 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-24T08:26:28.367 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat 2024-04-24T08:26:28.367 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-24T08:26:28.367 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac 2024-04-24T08:26:28.368 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-04-24T08:26:28.368 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-04-24T08:26:28.368 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-04-24T08:26:28.368 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-04-24T08:26:28.368 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-04-24T08:26:28.368 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-04-24T08:26:28.368 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-04-24T08:26:28.368 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-04-24T08:26:28.368 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-04-24T08:26:28.368 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-04-24T08:26:28.368 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-04-24T08:26:28.368 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-04-24T08:26:28.368 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-04-24T08:26:28.368 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware 2024-04-24T08:26:28.368 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/edd.ko 2024-04-24T08:26:28.369 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-24T08:26:28.369 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio 2024-04-24T08:26:28.369 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-04-24T08:26:28.369 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-24T08:26:28.369 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid 2024-04-24T08:26:28.369 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-04-24T08:26:28.369 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-04-24T08:26:28.369 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-04-24T08:26:28.369 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-04-24T08:26:28.369 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-04-24T08:26:28.369 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-04-24T08:26:28.369 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-04-24T08:26:28.369 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-04-24T08:26:28.369 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-04-24T08:26:28.369 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-04-24T08:26:28.369 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-24T08:26:28.370 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-24T08:26:28.370 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-04-24T08:26:28.370 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-04-24T08:26:28.370 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-04-24T08:26:28.370 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-04-24T08:26:28.370 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-04-24T08:26:28.370 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-04-24T08:26:28.370 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-04-24T08:26:28.370 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-04-24T08:26:28.370 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-24T08:26:28.370 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-24T08:26:28.370 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-04-24T08:26:28.370 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-04-24T08:26:28.370 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-04-24T08:26:28.370 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-04-24T08:26:28.371 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-04-24T08:26:28.371 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-04-24T08:26:28.371 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-04-24T08:26:28.371 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-04-24T08:26:28.371 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-04-24T08:26:28.371 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-24T08:26:28.371 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-04-24T08:26:28.371 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-24T08:26:28.371 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-24T08:26:28.371 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-24T08:26:28.371 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-24T08:26:28.371 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-24T08:26:28.371 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-24T08:26:28.371 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-24T08:26:28.372 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-24T08:26:28.372 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-24T08:26:28.372 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-04-24T08:26:28.372 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-04-24T08:26:28.372 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-04-24T08:26:28.372 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-24T08:26:28.372 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-04-24T08:26:28.372 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-04-24T08:26:28.372 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-04-24T08:26:28.372 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-04-24T08:26:28.372 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-04-24T08:26:28.372 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-24T08:26:28.372 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-04-24T08:26:28.372 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-04-24T08:26:28.372 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-04-24T08:26:28.373 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-04-24T08:26:28.373 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-04-24T08:26:28.373 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-04-24T08:26:28.373 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-04-24T08:26:28.373 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-04-24T08:26:28.373 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-04-24T08:26:28.373 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-04-24T08:26:28.373 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-04-24T08:26:28.373 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-04-24T08:26:28.373 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv 2024-04-24T08:26:28.373 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-04-24T08:26:28.373 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-04-24T08:26:28.373 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-04-24T08:26:28.373 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon 2024-04-24T08:26:28.373 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-04-24T08:26:28.373 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-04-24T08:26:28.374 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-24T08:26:28.374 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-04-24T08:26:28.374 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-04-24T08:26:28.374 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-04-24T08:26:28.374 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-04-24T08:26:28.374 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-04-24T08:26:28.374 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-04-24T08:26:28.374 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-04-24T08:26:28.374 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-04-24T08:26:28.374 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-04-24T08:26:28.374 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-04-24T08:26:28.374 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-04-24T08:26:28.374 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-04-24T08:26:28.374 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-04-24T08:26:28.374 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-04-24T08:26:28.375 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-04-24T08:26:28.375 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-04-24T08:26:28.375 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-04-24T08:26:28.375 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-04-24T08:26:28.375 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-04-24T08:26:28.375 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-24T08:26:28.375 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-04-24T08:26:28.375 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-04-24T08:26:28.375 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-24T08:26:28.375 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-04-24T08:26:28.375 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-04-24T08:26:28.375 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-04-24T08:26:28.375 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-04-24T08:26:28.375 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-04-24T08:26:28.375 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-04-24T08:26:28.376 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-04-24T08:26:28.376 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-04-24T08:26:28.376 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-04-24T08:26:28.376 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-04-24T08:26:28.376 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-04-24T08:26:28.376 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-04-24T08:26:28.376 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-04-24T08:26:28.376 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-04-24T08:26:28.376 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-24T08:26:28.376 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-04-24T08:26:28.376 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-04-24T08:26:28.376 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-04-24T08:26:28.376 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-04-24T08:26:28.376 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-04-24T08:26:28.376 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-04-24T08:26:28.377 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-04-24T08:26:28.377 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-04-24T08:26:28.377 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-04-24T08:26:28.377 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-04-24T08:26:28.377 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-04-24T08:26:28.377 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-04-24T08:26:28.377 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-04-24T08:26:28.377 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-04-24T08:26:28.377 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-04-24T08:26:28.377 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-04-24T08:26:28.377 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-04-24T08:26:28.377 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-04-24T08:26:28.377 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-04-24T08:26:28.377 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-04-24T08:26:28.377 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-04-24T08:26:28.378 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-04-24T08:26:28.378 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-04-24T08:26:28.378 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-04-24T08:26:28.378 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-04-24T08:26:28.378 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-04-24T08:26:28.378 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-04-24T08:26:28.378 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-04-24T08:26:28.378 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-04-24T08:26:28.378 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-04-24T08:26:28.378 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-04-24T08:26:28.378 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-04-24T08:26:28.378 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-04-24T08:26:28.378 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-04-24T08:26:28.378 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-04-24T08:26:28.379 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-04-24T08:26:28.379 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-04-24T08:26:28.379 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-04-24T08:26:28.379 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-04-24T08:26:28.379 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-04-24T08:26:28.379 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-04-24T08:26:28.379 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-04-24T08:26:28.379 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-04-24T08:26:28.379 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-04-24T08:26:28.379 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-04-24T08:26:28.379 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-04-24T08:26:28.379 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-04-24T08:26:28.379 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus 2024-04-24T08:26:28.379 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-24T08:26:28.379 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-24T08:26:28.380 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-24T08:26:28.380 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-24T08:26:28.380 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-24T08:26:28.380 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-24T08:26:28.380 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-24T08:26:28.380 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-24T08:26:28.380 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-24T08:26:28.380 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-24T08:26:28.380 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-24T08:26:28.380 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-24T08:26:28.380 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-04-24T08:26:28.380 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-04-24T08:26:28.380 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-04-24T08:26:28.380 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-24T08:26:28.380 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-04-24T08:26:28.381 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-04-24T08:26:28.381 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-04-24T08:26:28.381 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-04-24T08:26:28.381 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-04-24T08:26:28.381 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-04-24T08:26:28.381 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-04-24T08:26:28.381 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-04-24T08:26:28.381 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-04-24T08:26:28.381 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-04-24T08:26:28.381 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-04-24T08:26:28.381 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-04-24T08:26:28.381 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-04-24T08:26:28.381 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-04-24T08:26:28.381 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-04-24T08:26:28.381 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-04-24T08:26:28.382 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-04-24T08:26:28.382 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-04-24T08:26:28.382 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-04-24T08:26:28.382 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-04-24T08:26:28.382 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-04-24T08:26:28.382 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-04-24T08:26:28.382 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-04-24T08:26:28.382 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-04-24T08:26:28.382 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-04-24T08:26:28.382 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c 2024-04-24T08:26:28.382 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos 2024-04-24T08:26:28.382 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-24T08:26:28.382 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-24T08:26:28.382 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses 2024-04-24T08:26:28.382 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-24T08:26:28.382 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-24T08:26:28.383 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-24T08:26:28.383 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-24T08:26:28.383 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-24T08:26:28.383 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-24T08:26:28.383 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-24T08:26:28.383 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-24T08:26:28.383 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-24T08:26:28.383 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-24T08:26:28.383 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-24T08:26:28.383 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-24T08:26:28.383 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-24T08:26:28.383 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-24T08:26:28.383 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-24T08:26:28.383 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-24T08:26:28.383 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-24T08:26:28.384 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-24T08:26:28.384 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-24T08:26:28.384 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-24T08:26:28.384 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-24T08:26:28.384 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-24T08:26:28.384 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-24T08:26:28.384 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-dev.ko 2024-04-24T08:26:28.384 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-04-24T08:26:28.384 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-04-24T08:26:28.384 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-04-24T08:26:28.384 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-04-24T08:26:28.384 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband 2024-04-24T08:26:28.384 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core 2024-04-24T08:26:28.384 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-24T08:26:28.384 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-04-24T08:26:28.385 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-24T08:26:28.385 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-24T08:26:28.385 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-24T08:26:28.385 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-24T08:26:28.385 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-24T08:26:28.385 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw 2024-04-24T08:26:28.385 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4 2024-04-24T08:26:28.385 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-24T08:26:28.385 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4 2024-04-24T08:26:28.385 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-24T08:26:28.385 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5 2024-04-24T08:26:28.385 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-24T08:26:28.385 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca 2024-04-24T08:26:28.385 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-24T08:26:28.385 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp 2024-04-24T08:26:28.386 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib 2024-04-24T08:26:28.386 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-24T08:26:28.386 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser 2024-04-24T08:26:28.386 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-24T08:26:28.386 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert 2024-04-24T08:26:28.386 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-24T08:26:28.386 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp 2024-04-24T08:26:28.386 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-24T08:26:28.386 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt 2024-04-24T08:26:28.386 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-24T08:26:28.386 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input 2024-04-24T08:26:28.386 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport 2024-04-24T08:26:28.386 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-24T08:26:28.386 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-24T08:26:28.386 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-04-24T08:26:28.387 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-04-24T08:26:28.387 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard 2024-04-24T08:26:28.387 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-24T08:26:28.387 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse 2024-04-24T08:26:28.387 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-04-24T08:26:28.387 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-04-24T08:26:28.387 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-04-24T08:26:28.387 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-24T08:26:28.387 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-04-24T08:26:28.387 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-24T08:26:28.387 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-24T08:26:28.387 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-24T08:26:28.387 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4 2024-04-24T08:26:28.387 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-24T08:26:28.387 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio 2024-04-24T08:26:28.387 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-04-24T08:26:28.388 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-04-24T08:26:28.388 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-24T08:26:28.388 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-04-24T08:26:28.388 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-04-24T08:26:28.388 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds 2024-04-24T08:26:28.388 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-04-24T08:26:28.388 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-04-24T08:26:28.388 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-04-24T08:26:28.388 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-04-24T08:26:28.388 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger 2024-04-24T08:26:28.388 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-24T08:26:28.388 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-24T08:26:28.388 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-24T08:26:28.388 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-24T08:26:28.388 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-24T08:26:28.389 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-24T08:26:28.389 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-24T08:26:28.389 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-24T08:26:28.389 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-24T08:26:28.389 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md 2024-04-24T08:26:28.389 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache 2024-04-24T08:26:28.389 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-04-24T08:26:28.389 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-04-24T08:26:28.389 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-04-24T08:26:28.389 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-04-24T08:26:28.389 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-04-24T08:26:28.389 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-04-24T08:26:28.389 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-04-24T08:26:28.389 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-04-24T08:26:28.389 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-04-24T08:26:28.390 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-04-24T08:26:28.390 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-04-24T08:26:28.390 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-04-24T08:26:28.390 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-04-24T08:26:28.390 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-04-24T08:26:28.390 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-04-24T08:26:28.390 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-04-24T08:26:28.390 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data 2024-04-24T08:26:28.390 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-24T08:26:28.390 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-04-24T08:26:28.390 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-04-24T08:26:28.390 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-04-24T08:26:28.390 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-04-24T08:26:28.390 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message 2024-04-24T08:26:28.390 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion 2024-04-24T08:26:28.390 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-04-24T08:26:28.391 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-04-24T08:26:28.391 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-04-24T08:26:28.391 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-04-24T08:26:28.391 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-04-24T08:26:28.391 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-04-24T08:26:28.391 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-04-24T08:26:28.391 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd 2024-04-24T08:26:28.391 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-04-24T08:26:28.391 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-04-24T08:26:28.391 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mfd-core.ko 2024-04-24T08:26:28.391 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-04-24T08:26:28.391 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-04-24T08:26:28.391 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-04-24T08:26:28.391 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-04-24T08:26:28.391 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc 2024-04-24T08:26:28.391 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl 2024-04-24T08:26:28.392 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-24T08:26:28.392 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-04-24T08:26:28.392 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-04-24T08:26:28.392 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-04-24T08:26:28.392 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710 2024-04-24T08:26:28.392 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-04-24T08:26:28.392 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom 2024-04-24T08:26:28.392 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-04-24T08:26:28.392 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-24T08:26:28.392 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-04-24T08:26:28.392 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-04-24T08:26:28.392 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-04-24T08:26:28.392 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-04-24T08:26:28.392 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-04-24T08:26:28.392 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d 2024-04-24T08:26:28.393 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-24T08:26:28.393 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-24T08:26:28.393 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei 2024-04-24T08:26:28.393 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-04-24T08:26:28.393 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-04-24T08:26:28.393 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-04-24T08:26:28.393 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru 2024-04-24T08:26:28.393 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru/gru.ko 2024-04-24T08:26:28.393 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp 2024-04-24T08:26:28.393 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xp.ko 2024-04-24T08:26:28.393 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpc.ko 2024-04-24T08:26:28.393 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-04-24T08:26:28.393 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-04-24T08:26:28.393 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-04-24T08:26:28.393 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-04-24T08:26:28.394 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-04-24T08:26:28.394 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci 2024-04-24T08:26:28.394 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-24T08:26:28.394 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net 2024-04-24T08:26:28.394 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding 2024-04-24T08:26:28.394 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-04-24T08:26:28.394 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa 2024-04-24T08:26:28.394 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53 2024-04-24T08:26:28.394 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-24T08:26:28.394 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-24T08:26:28.394 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-24T08:26:28.394 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx 2024-04-24T08:26:28.394 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-24T08:26:28.394 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-04-24T08:26:28.394 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-04-24T08:26:28.394 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet 2024-04-24T08:26:28.395 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com 2024-04-24T08:26:28.395 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-24T08:26:28.395 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-24T08:26:28.395 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-24T08:26:28.395 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-24T08:26:28.395 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390 2024-04-24T08:26:28.395 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-24T08:26:28.395 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-24T08:26:28.395 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-24T08:26:28.395 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-24T08:26:28.395 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec 2024-04-24T08:26:28.395 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-24T08:26:28.395 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere 2024-04-24T08:26:28.395 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-24T08:26:28.396 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon 2024-04-24T08:26:28.396 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-24T08:26:28.396 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera 2024-04-24T08:26:28.396 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-24T08:26:28.396 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd 2024-04-24T08:26:28.396 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-24T08:26:28.396 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-24T08:26:28.396 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-24T08:26:28.396 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros 2024-04-24T08:26:28.396 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx 2024-04-24T08:26:28.396 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-24T08:26:28.396 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c 2024-04-24T08:26:28.396 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-24T08:26:28.396 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e 2024-04-24T08:26:28.396 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-24T08:26:28.396 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx 2024-04-24T08:26:28.397 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-24T08:26:28.397 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-24T08:26:28.397 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom 2024-04-24T08:26:28.397 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-24T08:26:28.397 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-24T08:26:28.397 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-04-24T08:26:28.397 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-24T08:26:28.397 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-24T08:26:28.397 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet 2024-04-24T08:26:28.397 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-24T08:26:28.397 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-24T08:26:28.397 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade 2024-04-24T08:26:28.397 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna 2024-04-24T08:26:28.397 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-24T08:26:28.397 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio 2024-04-24T08:26:28.397 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb 2024-04-24T08:26:28.398 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-24T08:26:28.398 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-04-24T08:26:28.398 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-24T08:26:28.398 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-04-24T08:26:28.398 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-24T08:26:28.398 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-04-24T08:26:28.398 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-24T08:26:28.398 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-04-24T08:26:28.398 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-24T08:26:28.398 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco 2024-04-24T08:26:28.398 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic 2024-04-24T08:26:28.398 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-24T08:26:28.398 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec 2024-04-24T08:26:28.398 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip 2024-04-24T08:26:28.399 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-24T08:26:28.399 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-24T08:26:28.399 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-24T08:26:28.399 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-24T08:26:28.399 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-24T08:26:28.399 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-24T08:26:28.399 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink 2024-04-24T08:26:28.399 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-24T08:26:28.399 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-24T08:26:28.399 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-04-24T08:26:28.399 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex 2024-04-24T08:26:28.399 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet 2024-04-24T08:26:28.399 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-24T08:26:28.399 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-04-24T08:26:28.400 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-04-24T08:26:28.400 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel 2024-04-24T08:26:28.400 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-24T08:26:28.400 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000 2024-04-24T08:26:28.400 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-24T08:26:28.400 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e 2024-04-24T08:26:28.400 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-24T08:26:28.400 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k 2024-04-24T08:26:28.400 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-24T08:26:28.400 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e 2024-04-24T08:26:28.400 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-24T08:26:28.400 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf 2024-04-24T08:26:28.400 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-24T08:26:28.400 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb 2024-04-24T08:26:28.401 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-24T08:26:28.401 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf 2024-04-24T08:26:28.401 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-24T08:26:28.401 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe 2024-04-24T08:26:28.401 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-24T08:26:28.401 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf 2024-04-24T08:26:28.401 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-24T08:26:28.401 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-04-24T08:26:28.401 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell 2024-04-24T08:26:28.401 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-24T08:26:28.401 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-24T08:26:28.401 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-24T08:26:28.401 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox 2024-04-24T08:26:28.401 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4 2024-04-24T08:26:28.401 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-24T08:26:28.402 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-24T08:26:28.402 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5 2024-04-24T08:26:28.402 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-04-24T08:26:28.402 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-24T08:26:28.402 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel 2024-04-24T08:26:28.402 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-24T08:26:28.402 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom 2024-04-24T08:26:28.402 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge 2024-04-24T08:26:28.402 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-24T08:26:28.402 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi 2024-04-24T08:26:28.402 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-24T08:26:28.402 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-24T08:26:28.402 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion 2024-04-24T08:26:28.402 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-24T08:26:28.402 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia 2024-04-24T08:26:28.403 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-24T08:26:28.403 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines 2024-04-24T08:26:28.403 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-24T08:26:28.403 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-24T08:26:28.403 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic 2024-04-24T08:26:28.403 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen 2024-04-24T08:26:28.403 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-24T08:26:28.403 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-24T08:26:28.403 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-04-24T08:26:28.403 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-24T08:26:28.403 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc 2024-04-24T08:26:28.403 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-24T08:26:28.403 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek 2024-04-24T08:26:28.403 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-24T08:26:28.403 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-24T08:26:28.404 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-24T08:26:28.404 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-24T08:26:28.404 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker 2024-04-24T08:26:28.404 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-04-24T08:26:28.404 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc 2024-04-24T08:26:28.404 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-24T08:26:28.404 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan 2024-04-24T08:26:28.404 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-24T08:26:28.404 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis 2024-04-24T08:26:28.404 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-24T08:26:28.404 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-24T08:26:28.404 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc 2024-04-24T08:26:28.404 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-24T08:26:28.404 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-24T08:26:28.404 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-24T08:26:28.405 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-24T08:26:28.405 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro 2024-04-24T08:26:28.405 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac 2024-04-24T08:26:28.405 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-24T08:26:28.405 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-24T08:26:28.405 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun 2024-04-24T08:26:28.405 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-24T08:26:28.405 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-24T08:26:28.405 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-24T08:26:28.405 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-24T08:26:28.405 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti 2024-04-24T08:26:28.405 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-24T08:26:28.405 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti 2024-04-24T08:26:28.405 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-24T08:26:28.405 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via 2024-04-24T08:26:28.406 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-24T08:26:28.406 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-24T08:26:28.406 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet 2024-04-24T08:26:28.406 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-24T08:26:28.406 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-24T08:26:28.406 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom 2024-04-24T08:26:28.406 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-24T08:26:28.406 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-04-24T08:26:28.406 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv 2024-04-24T08:26:28.406 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-24T08:26:28.406 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154 2024-04-24T08:26:28.406 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-24T08:26:28.406 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-04-24T08:26:28.406 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan 2024-04-24T08:26:28.407 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-24T08:26:28.407 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-04-24T08:26:28.407 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-04-24T08:26:28.407 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio 2024-04-24T08:26:28.407 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-04-24T08:26:28.407 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-24T08:26:28.407 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-24T08:26:28.407 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-24T08:26:28.407 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-04-24T08:26:28.407 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/net_failover.ko 2024-04-24T08:26:28.407 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-04-24T08:26:28.407 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-04-24T08:26:28.407 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs 2024-04-24T08:26:28.407 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-24T08:26:28.407 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-24T08:26:28.407 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy 2024-04-24T08:26:28.408 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-04-24T08:26:28.408 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-24T08:26:28.408 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-04-24T08:26:28.408 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-24T08:26:28.408 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-04-24T08:26:28.408 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-04-24T08:26:28.408 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-04-24T08:26:28.408 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-04-24T08:26:28.408 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/dp83640.ko 2024-04-24T08:26:28.408 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-04-24T08:26:28.408 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-04-24T08:26:28.408 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-04-24T08:26:28.408 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-04-24T08:26:28.408 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-04-24T08:26:28.408 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-04-24T08:26:28.408 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-04-24T08:26:28.409 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-04-24T08:26:28.409 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-04-24T08:26:28.409 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-04-24T08:26:28.409 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-04-24T08:26:28.409 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-04-24T08:26:28.409 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp 2024-04-24T08:26:28.409 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-24T08:26:28.409 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-04-24T08:26:28.409 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-24T08:26:28.409 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_generic.ko 2024-04-24T08:26:28.409 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-24T08:26:28.409 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-24T08:26:28.409 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-04-24T08:26:28.409 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-04-24T08:26:28.409 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-04-24T08:26:28.410 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip 2024-04-24T08:26:28.410 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slhc.ko 2024-04-24T08:26:28.410 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-04-24T08:26:28.410 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-04-24T08:26:28.410 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-04-24T08:26:28.410 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team 2024-04-24T08:26:28.410 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-04-24T08:26:28.410 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-24T08:26:28.410 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-24T08:26:28.410 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-24T08:26:28.410 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-04-24T08:26:28.410 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-24T08:26:28.410 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tun.ko 2024-04-24T08:26:28.410 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-04-24T08:26:28.410 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/virtio_net.ko 2024-04-24T08:26:28.411 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3 2024-04-24T08:26:28.411 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-24T08:26:28.411 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan 2024-04-24T08:26:28.411 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-04-24T08:26:28.411 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback 2024-04-24T08:26:28.411 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-24T08:26:28.411 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netfront.ko 2024-04-24T08:26:28.411 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb 2024-04-24T08:26:28.411 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-04-24T08:26:28.411 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme 2024-04-24T08:26:28.411 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host 2024-04-24T08:26:28.411 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-04-24T08:26:28.411 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-04-24T08:26:28.411 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport 2024-04-24T08:26:28.412 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-04-24T08:26:28.412 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-04-24T08:26:28.412 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-04-24T08:26:28.412 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-04-24T08:26:28.412 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci 2024-04-24T08:26:28.412 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug 2024-04-24T08:26:28.412 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-24T08:26:28.412 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie 2024-04-24T08:26:28.412 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie/aer_inject.ko 2024-04-24T08:26:28.412 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-04-24T08:26:28.412 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia 2024-04-24T08:26:28.412 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-04-24T08:26:28.412 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-04-24T08:26:28.412 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-24T08:26:28.412 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl 2024-04-24T08:26:28.412 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel 2024-04-24T08:26:28.413 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-24T08:26:28.413 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform 2024-04-24T08:26:28.413 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86 2024-04-24T08:26:28.413 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-24T08:26:28.413 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-04-24T08:26:28.413 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-24T08:26:28.413 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-24T08:26:28.413 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-24T08:26:28.413 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-24T08:26:28.413 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-24T08:26:28.413 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-24T08:26:28.413 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-24T08:26:28.413 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-24T08:26:28.413 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-24T08:26:28.413 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-04-24T08:26:28.413 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel 2024-04-24T08:26:28.414 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-24T08:26:28.414 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-04-24T08:26:28.414 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-24T08:26:28.414 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-24T08:26:28.414 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-24T08:26:28.414 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-24T08:26:28.414 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-24T08:26:28.414 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-24T08:26:28.414 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-24T08:26:28.414 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-24T08:26:28.414 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-24T08:26:28.414 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-04-24T08:26:28.414 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap 2024-04-24T08:26:28.414 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-24T08:26:28.414 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-24T08:26:28.414 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps 2024-04-24T08:26:28.415 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients 2024-04-24T08:26:28.415 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-24T08:26:28.415 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-24T08:26:28.415 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-04-24T08:26:28.415 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-04-24T08:26:28.415 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp 2024-04-24T08:26:28.415 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-04-24T08:26:28.415 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-04-24T08:26:28.415 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras 2024-04-24T08:26:28.415 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd 2024-04-24T08:26:28.415 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl 2024-04-24T08:26:28.415 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-24T08:26:28.415 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-04-24T08:26:28.415 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc 2024-04-24T08:26:28.415 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-24T08:26:28.415 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-24T08:26:28.416 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-24T08:26:28.416 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-24T08:26:28.416 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-24T08:26:28.416 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-24T08:26:28.416 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-24T08:26:28.416 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-24T08:26:28.416 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-24T08:26:28.416 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-24T08:26:28.416 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-04-24T08:26:28.416 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-24T08:26:28.416 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-24T08:26:28.416 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-24T08:26:28.416 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-24T08:26:28.416 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-24T08:26:28.416 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-24T08:26:28.417 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-04-24T08:26:28.417 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-24T08:26:28.417 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-24T08:26:28.417 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-24T08:26:28.417 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-24T08:26:28.417 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-24T08:26:28.417 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-24T08:26:28.417 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-24T08:26:28.417 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-24T08:26:28.417 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-24T08:26:28.417 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-24T08:26:28.417 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-24T08:26:28.417 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-24T08:26:28.417 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-04-24T08:26:28.417 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi 2024-04-24T08:26:28.418 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-04-24T08:26:28.418 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-04-24T08:26:28.418 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-04-24T08:26:28.418 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-04-24T08:26:28.418 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-04-24T08:26:28.418 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid 2024-04-24T08:26:28.418 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-24T08:26:28.418 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-04-24T08:26:28.418 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx 2024-04-24T08:26:28.418 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-24T08:26:28.418 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-24T08:26:28.418 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-04-24T08:26:28.418 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr 2024-04-24T08:26:28.418 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-24T08:26:28.418 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-04-24T08:26:28.419 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi 2024-04-24T08:26:28.419 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-24T08:26:28.419 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa 2024-04-24T08:26:28.419 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-04-24T08:26:28.419 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc 2024-04-24T08:26:28.419 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-24T08:26:28.419 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i 2024-04-24T08:26:28.419 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-24T08:26:28.419 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-04-24T08:26:28.419 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor 2024-04-24T08:26:28.419 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-24T08:26:28.419 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi 2024-04-24T08:26:28.419 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i 2024-04-24T08:26:28.419 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-24T08:26:28.419 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i 2024-04-24T08:26:28.420 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-24T08:26:28.420 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-24T08:26:28.420 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-04-24T08:26:28.420 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler 2024-04-24T08:26:28.420 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-04-24T08:26:28.420 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-04-24T08:26:28.420 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-04-24T08:26:28.420 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-04-24T08:26:28.420 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-04-24T08:26:28.420 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r 2024-04-24T08:26:28.420 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-24T08:26:28.420 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-04-24T08:26:28.420 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe 2024-04-24T08:26:28.420 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-24T08:26:28.421 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-24T08:26:28.421 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic 2024-04-24T08:26:28.421 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-04-24T08:26:28.421 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-04-24T08:26:28.421 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-04-24T08:26:28.421 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-04-24T08:26:28.421 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-04-24T08:26:28.421 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-04-24T08:26:28.421 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-04-24T08:26:28.421 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci 2024-04-24T08:26:28.421 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-04-24T08:26:28.421 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-24T08:26:28.421 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-24T08:26:28.421 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc 2024-04-24T08:26:28.421 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-04-24T08:26:28.422 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-04-24T08:26:28.422 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-24T08:26:28.422 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas 2024-04-24T08:26:28.422 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-04-24T08:26:28.422 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc 2024-04-24T08:26:28.422 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-24T08:26:28.422 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid 2024-04-24T08:26:28.422 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-04-24T08:26:28.422 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-24T08:26:28.422 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-24T08:26:28.422 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-24T08:26:28.422 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas 2024-04-24T08:26:28.422 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-24T08:26:28.422 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas 2024-04-24T08:26:28.423 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-24T08:26:28.423 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-04-24T08:26:28.423 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001 2024-04-24T08:26:28.423 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-24T08:26:28.423 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-04-24T08:26:28.423 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-04-24T08:26:28.423 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx 2024-04-24T08:26:28.423 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-24T08:26:28.423 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-24T08:26:28.423 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx 2024-04-24T08:26:28.423 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-24T08:26:28.423 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-04-24T08:26:28.423 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-04-24T08:26:28.423 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-24T08:26:28.423 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-24T08:26:28.424 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-24T08:26:28.424 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-24T08:26:28.424 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-24T08:26:28.424 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-04-24T08:26:28.424 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-04-24T08:26:28.424 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-04-24T08:26:28.424 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2 2024-04-24T08:26:28.424 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-24T08:26:28.424 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-04-24T08:26:28.424 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-24T08:26:28.424 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-04-24T08:26:28.424 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb 2024-04-24T08:26:28.424 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-04-24T08:26:28.424 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target 2024-04-24T08:26:28.424 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi 2024-04-24T08:26:28.424 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-24T08:26:28.425 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback 2024-04-24T08:26:28.425 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-24T08:26:28.425 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-04-24T08:26:28.425 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-04-24T08:26:28.425 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-04-24T08:26:28.425 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-04-24T08:26:28.425 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-04-24T08:26:28.425 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc 2024-04-24T08:26:28.425 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-24T08:26:28.425 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal 2024-04-24T08:26:28.425 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel 2024-04-24T08:26:28.425 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal 2024-04-24T08:26:28.425 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-24T08:26:28.425 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-24T08:26:28.425 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-24T08:26:28.426 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-24T08:26:28.426 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-24T08:26:28.426 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-24T08:26:28.426 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-24T08:26:28.426 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-24T08:26:28.426 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-24T08:26:28.426 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-24T08:26:28.426 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-24T08:26:28.426 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-24T08:26:28.426 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-24T08:26:28.426 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-24T08:26:28.426 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-24T08:26:28.426 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-24T08:26:28.427 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-24T08:26:28.427 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-24T08:26:28.427 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty 2024-04-24T08:26:28.427 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless 2024-04-24T08:26:28.427 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-24T08:26:28.427 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-04-24T08:26:28.427 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-04-24T08:26:28.427 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-04-24T08:26:28.427 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial 2024-04-24T08:26:28.427 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250 2024-04-24T08:26:28.427 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-24T08:26:28.427 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-04-24T08:26:28.427 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm 2024-04-24T08:26:28.427 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-24T08:26:28.427 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs 2024-04-24T08:26:28.427 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core 2024-04-24T08:26:28.428 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-24T08:26:28.428 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host 2024-04-24T08:26:28.428 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-24T08:26:28.428 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio 2024-04-24T08:26:28.428 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-04-24T08:26:28.428 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-04-24T08:26:28.428 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-04-24T08:26:28.428 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-04-24T08:26:28.428 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-04-24T08:26:28.428 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb 2024-04-24T08:26:28.428 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class 2024-04-24T08:26:28.428 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-04-24T08:26:28.428 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-24T08:26:28.428 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-04-24T08:26:28.428 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-04-24T08:26:28.429 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host 2024-04-24T08:26:28.429 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-24T08:26:28.429 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image 2024-04-24T08:26:28.429 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-04-24T08:26:28.429 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-04-24T08:26:28.429 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc 2024-04-24T08:26:28.429 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-04-24T08:26:28.429 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-04-24T08:26:28.429 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-04-24T08:26:28.429 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-04-24T08:26:28.429 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-04-24T08:26:28.429 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-04-24T08:26:28.429 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-04-24T08:26:28.429 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-24T08:26:28.429 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-04-24T08:26:28.430 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-04-24T08:26:28.430 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga 2024-04-24T08:26:28.430 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-24T08:26:28.430 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-24T08:26:28.430 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-04-24T08:26:28.430 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-04-24T08:26:28.430 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-24T08:26:28.430 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-04-24T08:26:28.430 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-04-24T08:26:28.430 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy 2024-04-24T08:26:28.430 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-04-24T08:26:28.430 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial 2024-04-24T08:26:28.430 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-04-24T08:26:28.430 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-04-24T08:26:28.430 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-24T08:26:28.430 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-04-24T08:26:28.431 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-04-24T08:26:28.431 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-04-24T08:26:28.431 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-24T08:26:28.431 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-24T08:26:28.431 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-04-24T08:26:28.431 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-24T08:26:28.431 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-24T08:26:28.431 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-24T08:26:28.431 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-04-24T08:26:28.431 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-04-24T08:26:28.431 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-04-24T08:26:28.431 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-04-24T08:26:28.431 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-24T08:26:28.431 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-04-24T08:26:28.431 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-24T08:26:28.432 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-24T08:26:28.432 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-24T08:26:28.432 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-04-24T08:26:28.432 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-04-24T08:26:28.432 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-04-24T08:26:28.432 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-04-24T08:26:28.432 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-04-24T08:26:28.432 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-04-24T08:26:28.432 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-04-24T08:26:28.432 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-04-24T08:26:28.432 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-04-24T08:26:28.432 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-04-24T08:26:28.432 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-04-24T08:26:28.432 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-04-24T08:26:28.433 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-04-24T08:26:28.433 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-04-24T08:26:28.433 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-24T08:26:28.433 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-04-24T08:26:28.433 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-04-24T08:26:28.433 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-24T08:26:28.433 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-24T08:26:28.433 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-04-24T08:26:28.433 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-24T08:26:28.433 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-04-24T08:26:28.433 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-04-24T08:26:28.433 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-24T08:26:28.433 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage 2024-04-24T08:26:28.433 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-04-24T08:26:28.433 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-24T08:26:28.433 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-24T08:26:28.433 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-24T08:26:28.434 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-24T08:26:28.434 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-24T08:26:28.434 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-24T08:26:28.434 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-24T08:26:28.434 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-04-24T08:26:28.434 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-24T08:26:28.434 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-24T08:26:28.434 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-24T08:26:28.434 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-24T08:26:28.434 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-24T08:26:28.434 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-04-24T08:26:28.434 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip 2024-04-24T08:26:28.434 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-24T08:26:28.434 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-24T08:26:28.434 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-24T08:26:28.435 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio 2024-04-24T08:26:28.435 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci 2024-04-24T08:26:28.435 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-24T08:26:28.435 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-24T08:26:28.435 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-04-24T08:26:28.435 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-24T08:26:28.435 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost 2024-04-24T08:26:28.435 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-04-24T08:26:28.435 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-24T08:26:28.435 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-04-24T08:26:28.435 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-04-24T08:26:28.435 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video 2024-04-24T08:26:28.435 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight 2024-04-24T08:26:28.435 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-04-24T08:26:28.435 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-04-24T08:26:28.436 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-24T08:26:28.436 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio 2024-04-24T08:26:28.436 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio.ko 2024-04-24T08:26:28.436 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_balloon.ko 2024-04-24T08:26:28.436 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_mmio.ko 2024-04-24T08:26:28.436 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci.ko 2024-04-24T08:26:28.436 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-04-24T08:26:28.436 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-04-24T08:26:28.436 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_ring.ko 2024-04-24T08:26:28.436 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1 2024-04-24T08:26:28.436 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters 2024-04-24T08:26:28.436 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-04-24T08:26:28.436 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-04-24T08:26:28.436 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves 2024-04-24T08:26:28.436 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-24T08:26:28.436 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-24T08:26:28.437 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-24T08:26:28.437 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-24T08:26:28.437 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-24T08:26:28.437 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-24T08:26:28.437 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-24T08:26:28.437 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-24T08:26:28.437 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-24T08:26:28.437 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-24T08:26:28.437 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-24T08:26:28.437 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-04-24T08:26:28.437 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog 2024-04-24T08:26:28.437 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-24T08:26:28.437 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-24T08:26:28.437 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-24T08:26:28.437 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-04-24T08:26:28.438 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-04-24T08:26:28.438 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-24T08:26:28.438 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-24T08:26:28.438 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-04-24T08:26:28.438 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-04-24T08:26:28.438 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-24T08:26:28.438 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-24T08:26:28.438 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-04-24T08:26:28.438 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-04-24T08:26:28.438 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-04-24T08:26:28.438 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-24T08:26:28.438 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-24T08:26:28.438 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-24T08:26:28.438 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-24T08:26:28.438 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-04-24T08:26:28.439 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-24T08:26:28.439 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-04-24T08:26:28.439 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-24T08:26:28.439 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-24T08:26:28.439 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-24T08:26:28.439 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-04-24T08:26:28.439 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-04-24T08:26:28.439 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen 2024-04-24T08:26:28.439 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-acpi-processor.ko 2024-04-24T08:26:28.439 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-04-24T08:26:28.439 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-04-24T08:26:28.439 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-04-24T08:26:28.439 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback 2024-04-24T08:26:28.439 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-24T08:26:28.439 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-04-24T08:26:28.439 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs 2024-04-24T08:26:28.440 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-24T08:26:28.440 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs 2024-04-24T08:26:28.440 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p 2024-04-24T08:26:28.440 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-04-24T08:26:28.440 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs 2024-04-24T08:26:28.440 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-04-24T08:26:28.440 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs 2024-04-24T08:26:28.440 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-04-24T08:26:28.440 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-04-24T08:26:28.440 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs 2024-04-24T08:26:28.440 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-04-24T08:26:28.440 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles 2024-04-24T08:26:28.440 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-04-24T08:26:28.440 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph 2024-04-24T08:26:28.440 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-04-24T08:26:28.441 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda 2024-04-24T08:26:28.441 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-04-24T08:26:28.441 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs 2024-04-24T08:26:28.441 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-04-24T08:26:28.441 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm 2024-04-24T08:26:28.441 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-04-24T08:26:28.441 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs 2024-04-24T08:26:28.441 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs/ecryptfs.ko 2024-04-24T08:26:28.441 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs 2024-04-24T08:26:28.441 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-04-24T08:26:28.441 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat 2024-04-24T08:26:28.441 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/fat.ko 2024-04-24T08:26:28.441 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-04-24T08:26:28.441 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/vfat.ko 2024-04-24T08:26:28.441 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse 2024-04-24T08:26:28.441 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-04-24T08:26:28.442 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/fuse.ko 2024-04-24T08:26:28.442 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2 2024-04-24T08:26:28.442 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-04-24T08:26:28.442 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs 2024-04-24T08:26:28.442 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-04-24T08:26:28.442 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus 2024-04-24T08:26:28.442 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-04-24T08:26:28.442 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs 2024-04-24T08:26:28.442 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-04-24T08:26:28.442 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs 2024-04-24T08:26:28.442 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-04-24T08:26:28.442 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd 2024-04-24T08:26:28.442 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-04-24T08:26:28.442 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix 2024-04-24T08:26:28.442 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-04-24T08:26:28.442 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs 2024-04-24T08:26:28.443 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-04-24T08:26:28.443 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs 2024-04-24T08:26:28.443 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout 2024-04-24T08:26:28.443 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-24T08:26:28.443 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout 2024-04-24T08:26:28.443 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-24T08:26:28.443 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout 2024-04-24T08:26:28.443 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-24T08:26:28.443 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-04-24T08:26:28.443 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-04-24T08:26:28.443 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-04-24T08:26:28.443 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common 2024-04-24T08:26:28.443 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-04-24T08:26:28.443 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-04-24T08:26:28.443 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd 2024-04-24T08:26:28.444 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-04-24T08:26:28.444 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2 2024-04-24T08:26:28.444 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-04-24T08:26:28.444 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls 2024-04-24T08:26:28.444 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-04-24T08:26:28.444 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-04-24T08:26:28.444 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-04-24T08:26:28.444 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-04-24T08:26:28.444 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-04-24T08:26:28.444 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-04-24T08:26:28.444 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-04-24T08:26:28.444 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-04-24T08:26:28.444 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-04-24T08:26:28.444 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-04-24T08:26:28.444 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-04-24T08:26:28.444 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-04-24T08:26:28.445 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-04-24T08:26:28.445 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-04-24T08:26:28.445 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-04-24T08:26:28.445 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-04-24T08:26:28.445 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-04-24T08:26:28.445 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-04-24T08:26:28.445 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-04-24T08:26:28.445 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-04-24T08:26:28.445 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-04-24T08:26:28.445 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-04-24T08:26:28.445 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-04-24T08:26:28.445 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-04-24T08:26:28.445 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-04-24T08:26:28.445 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-04-24T08:26:28.445 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-04-24T08:26:28.445 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-04-24T08:26:28.446 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-04-24T08:26:28.446 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-04-24T08:26:28.446 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-04-24T08:26:28.446 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-04-24T08:26:28.446 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-04-24T08:26:28.446 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-04-24T08:26:28.446 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-04-24T08:26:28.446 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-04-24T08:26:28.446 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-04-24T08:26:28.446 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-04-24T08:26:28.446 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-04-24T08:26:28.446 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-04-24T08:26:28.446 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-04-24T08:26:28.446 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-04-24T08:26:28.447 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-04-24T08:26:28.447 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-04-24T08:26:28.447 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-04-24T08:26:28.447 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-04-24T08:26:28.447 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-04-24T08:26:28.447 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-04-24T08:26:28.447 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-24T08:26:28.447 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-04-24T08:26:28.447 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2 2024-04-24T08:26:28.447 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster 2024-04-24T08:26:28.447 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-24T08:26:28.447 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm 2024-04-24T08:26:28.447 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-24T08:26:28.447 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs 2024-04-24T08:26:28.447 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-24T08:26:28.448 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-04-24T08:26:28.448 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-24T08:26:28.448 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-24T08:26:28.448 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-24T08:26:28.448 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs 2024-04-24T08:26:28.448 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-04-24T08:26:28.448 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore 2024-04-24T08:26:28.448 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-04-24T08:26:28.448 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs 2024-04-24T08:26:28.448 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-04-24T08:26:28.448 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs 2024-04-24T08:26:28.448 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-04-24T08:26:28.448 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb 2024-04-24T08:26:28.448 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client 2024-04-24T08:26:28.448 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-04-24T08:26:28.449 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common 2024-04-24T08:26:28.449 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-04-24T08:26:28.449 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-04-24T08:26:28.449 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs 2024-04-24T08:26:28.449 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-04-24T08:26:28.449 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv 2024-04-24T08:26:28.449 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-04-24T08:26:28.449 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf 2024-04-24T08:26:28.449 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-04-24T08:26:28.449 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs 2024-04-24T08:26:28.449 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-04-24T08:26:28.449 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs 2024-04-24T08:26:28.449 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-04-24T08:26:28.449 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel 2024-04-24T08:26:28.449 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu 2024-04-24T08:26:28.450 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu/rcutorture.ko 2024-04-24T08:26:28.450 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-04-24T08:26:28.450 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace 2024-04-24T08:26:28.450 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-04-24T08:26:28.450 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib 2024-04-24T08:26:28.450 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/asn1_encoder.ko 2024-04-24T08:26:28.450 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-04-24T08:26:28.450 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-04-24T08:26:28.450 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto 2024-04-24T08:26:28.450 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-04-24T08:26:28.450 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-04-24T08:26:28.450 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-04-24T08:26:28.450 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-04-24T08:26:28.450 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4 2024-04-24T08:26:28.451 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-04-24T08:26:28.451 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-04-24T08:26:28.451 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math 2024-04-24T08:26:28.451 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-04-24T08:26:28.451 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6 2024-04-24T08:26:28.451 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-04-24T08:26:28.451 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon 2024-04-24T08:26:28.451 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-24T08:26:28.451 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-04-24T08:26:28.451 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-04-24T08:26:28.451 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-04-24T08:26:28.451 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd 2024-04-24T08:26:28.451 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd/zstd_compress.ko 2024-04-24T08:26:28.451 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm 2024-04-24T08:26:28.451 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-04-24T08:26:28.452 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net 2024-04-24T08:26:28.452 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan 2024-04-24T08:26:28.452 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-04-24T08:26:28.452 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-04-24T08:26:28.452 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-04-24T08:26:28.452 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-04-24T08:26:28.452 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-24T08:26:28.452 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-04-24T08:26:28.452 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-04-24T08:26:28.452 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-04-24T08:26:28.452 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802 2024-04-24T08:26:28.452 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-04-24T08:26:28.452 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-04-24T08:26:28.452 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-04-24T08:26:28.452 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-04-24T08:26:28.452 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-04-24T08:26:28.453 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q 2024-04-24T08:26:28.453 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-04-24T08:26:28.453 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p 2024-04-24T08:26:28.453 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-04-24T08:26:28.453 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-04-24T08:26:28.453 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-04-24T08:26:28.453 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-04-24T08:26:28.453 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv 2024-04-24T08:26:28.453 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-04-24T08:26:28.453 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge 2024-04-24T08:26:28.453 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-04-24T08:26:28.453 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-04-24T08:26:28.453 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter 2024-04-24T08:26:28.453 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-24T08:26:28.453 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-24T08:26:28.454 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-24T08:26:28.454 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-24T08:26:28.454 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-24T08:26:28.454 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-24T08:26:28.454 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-24T08:26:28.454 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-24T08:26:28.454 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-24T08:26:28.454 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-24T08:26:28.454 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-24T08:26:28.454 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-24T08:26:28.454 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-24T08:26:28.454 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-24T08:26:28.454 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-24T08:26:28.454 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-24T08:26:28.454 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-24T08:26:28.455 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-24T08:26:28.455 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-24T08:26:28.455 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-24T08:26:28.455 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-04-24T08:26:28.455 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-04-24T08:26:28.455 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-24T08:26:28.455 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-24T08:26:28.455 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph 2024-04-24T08:26:28.455 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-04-24T08:26:28.455 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core 2024-04-24T08:26:28.455 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/failover.ko 2024-04-24T08:26:28.455 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-04-24T08:26:28.455 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp 2024-04-24T08:26:28.455 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-04-24T08:26:28.455 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-04-24T08:26:28.455 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-04-24T08:26:28.456 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-04-24T08:26:28.456 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver 2024-04-24T08:26:28.456 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver/dns_resolver.ko 2024-04-24T08:26:28.456 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa 2024-04-24T08:26:28.456 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-04-24T08:26:28.456 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-04-24T08:26:28.456 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-04-24T08:26:28.456 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-04-24T08:26:28.456 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-04-24T08:26:28.456 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154 2024-04-24T08:26:28.456 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan 2024-04-24T08:26:28.456 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-24T08:26:28.456 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-04-24T08:26:28.456 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-24T08:26:28.456 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4 2024-04-24T08:26:28.457 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-04-24T08:26:28.457 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-04-24T08:26:28.457 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-04-24T08:26:28.457 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-04-24T08:26:28.457 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-04-24T08:26:28.457 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-04-24T08:26:28.457 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-04-24T08:26:28.457 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-04-24T08:26:28.457 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-04-24T08:26:28.457 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-04-24T08:26:28.457 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter 2024-04-24T08:26:28.457 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-24T08:26:28.457 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-24T08:26:28.457 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-24T08:26:28.457 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-24T08:26:28.458 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-24T08:26:28.458 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-24T08:26:28.458 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-24T08:26:28.458 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-24T08:26:28.458 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-24T08:26:28.458 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-24T08:26:28.458 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-24T08:26:28.458 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-24T08:26:28.458 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-24T08:26:28.458 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-24T08:26:28.458 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-24T08:26:28.458 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-24T08:26:28.458 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-24T08:26:28.458 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-24T08:26:28.458 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-24T08:26:28.458 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-24T08:26:28.458 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-24T08:26:28.459 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-04-24T08:26:28.459 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-04-24T08:26:28.459 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-04-24T08:26:28.459 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-04-24T08:26:28.459 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-04-24T08:26:28.459 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-04-24T08:26:28.459 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-04-24T08:26:28.459 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-04-24T08:26:28.459 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-04-24T08:26:28.459 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-04-24T08:26:28.459 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-04-24T08:26:28.459 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-04-24T08:26:28.459 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-04-24T08:26:28.459 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-04-24T08:26:28.459 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-04-24T08:26:28.460 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-04-24T08:26:28.460 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-24T08:26:28.460 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6 2024-04-24T08:26:28.460 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-04-24T08:26:28.460 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-04-24T08:26:28.460 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-04-24T08:26:28.460 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-04-24T08:26:28.460 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-24T08:26:28.460 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-04-24T08:26:28.460 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-04-24T08:26:28.460 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter 2024-04-24T08:26:28.460 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-24T08:26:28.460 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-24T08:26:28.460 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-24T08:26:28.460 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-24T08:26:28.460 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-24T08:26:28.461 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-24T08:26:28.461 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-24T08:26:28.461 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-24T08:26:28.461 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-24T08:26:28.461 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-24T08:26:28.461 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-24T08:26:28.461 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-24T08:26:28.461 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-24T08:26:28.461 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-24T08:26:28.461 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-24T08:26:28.461 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-24T08:26:28.461 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-24T08:26:28.461 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-24T08:26:28.461 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-24T08:26:28.461 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-24T08:26:28.461 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-24T08:26:28.462 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-24T08:26:28.462 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-24T08:26:28.462 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-24T08:26:28.462 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-04-24T08:26:28.462 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-04-24T08:26:28.462 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-24T08:26:28.462 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key 2024-04-24T08:26:28.462 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-04-24T08:26:28.462 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp 2024-04-24T08:26:28.462 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-04-24T08:26:28.462 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-24T08:26:28.462 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-04-24T08:26:28.462 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-04-24T08:26:28.462 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-04-24T08:26:28.462 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-04-24T08:26:28.462 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-04-24T08:26:28.463 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc 2024-04-24T08:26:28.463 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-04-24T08:26:28.463 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154 2024-04-24T08:26:28.463 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-04-24T08:26:28.463 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls 2024-04-24T08:26:28.463 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-04-24T08:26:28.463 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter 2024-04-24T08:26:28.463 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset 2024-04-24T08:26:28.463 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-04-24T08:26:28.463 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-24T08:26:28.463 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-24T08:26:28.463 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-24T08:26:28.463 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-24T08:26:28.463 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-24T08:26:28.463 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-24T08:26:28.464 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-24T08:26:28.464 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-24T08:26:28.464 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-24T08:26:28.464 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-24T08:26:28.464 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-24T08:26:28.464 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-24T08:26:28.464 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-24T08:26:28.464 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-24T08:26:28.464 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-24T08:26:28.464 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs 2024-04-24T08:26:28.464 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-24T08:26:28.464 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-24T08:26:28.464 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-24T08:26:28.464 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-24T08:26:28.464 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-24T08:26:28.464 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-24T08:26:28.464 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-24T08:26:28.465 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-24T08:26:28.465 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-24T08:26:28.465 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-24T08:26:28.465 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-24T08:26:28.465 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-24T08:26:28.465 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-24T08:26:28.465 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-24T08:26:28.465 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-04-24T08:26:28.465 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-04-24T08:26:28.465 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-24T08:26:28.465 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-24T08:26:28.465 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-24T08:26:28.465 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-24T08:26:28.465 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-24T08:26:28.465 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-24T08:26:28.465 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-24T08:26:28.466 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-24T08:26:28.466 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-24T08:26:28.466 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-24T08:26:28.466 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-24T08:26:28.466 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-24T08:26:28.466 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-24T08:26:28.466 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-04-24T08:26:28.466 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-24T08:26:28.466 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-04-24T08:26:28.466 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-04-24T08:26:28.466 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-24T08:26:28.466 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-24T08:26:28.466 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-04-24T08:26:28.466 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-04-24T08:26:28.467 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-24T08:26:28.467 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-24T08:26:28.467 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-04-24T08:26:28.467 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-04-24T08:26:28.467 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-24T08:26:28.467 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-04-24T08:26:28.467 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-24T08:26:28.467 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-24T08:26:28.467 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-04-24T08:26:28.467 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-04-24T08:26:28.467 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-04-24T08:26:28.467 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-24T08:26:28.467 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-04-24T08:26:28.467 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-24T08:26:28.467 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-04-24T08:26:28.467 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-04-24T08:26:28.468 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-04-24T08:26:28.468 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-04-24T08:26:28.468 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-04-24T08:26:28.468 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-04-24T08:26:28.468 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-04-24T08:26:28.468 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_quota.ko 2024-04-24T08:26:28.468 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-04-24T08:26:28.468 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-04-24T08:26:28.468 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-04-24T08:26:28.468 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-04-24T08:26:28.468 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tunnel.ko 2024-04-24T08:26:28.468 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-04-24T08:26:28.468 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-24T08:26:28.468 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-24T08:26:28.469 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-24T08:26:28.469 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-04-24T08:26:28.469 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-04-24T08:26:28.469 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-04-24T08:26:28.469 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-04-24T08:26:28.469 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-24T08:26:28.469 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-04-24T08:26:28.469 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-04-24T08:26:28.469 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-24T08:26:28.469 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-04-24T08:26:28.469 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-04-24T08:26:28.469 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-24T08:26:28.469 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-04-24T08:26:28.469 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-24T08:26:28.469 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-04-24T08:26:28.470 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-24T08:26:28.470 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-24T08:26:28.470 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-04-24T08:26:28.470 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-04-24T08:26:28.470 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-04-24T08:26:28.470 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-04-24T08:26:28.470 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-04-24T08:26:28.470 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-04-24T08:26:28.470 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-04-24T08:26:28.470 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-04-24T08:26:28.470 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-04-24T08:26:28.470 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-04-24T08:26:28.470 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-04-24T08:26:28.470 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-04-24T08:26:28.471 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-04-24T08:26:28.471 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-04-24T08:26:28.471 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-04-24T08:26:28.471 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-04-24T08:26:28.471 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-04-24T08:26:28.471 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-04-24T08:26:28.471 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-04-24T08:26:28.471 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-04-24T08:26:28.471 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-04-24T08:26:28.471 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-04-24T08:26:28.471 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-04-24T08:26:28.471 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-04-24T08:26:28.471 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-04-24T08:26:28.471 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-04-24T08:26:28.471 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-04-24T08:26:28.471 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-04-24T08:26:28.472 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-04-24T08:26:28.472 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-04-24T08:26:28.472 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-04-24T08:26:28.472 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-04-24T08:26:28.472 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-04-24T08:26:28.472 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-04-24T08:26:28.472 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-04-24T08:26:28.472 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-04-24T08:26:28.472 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-04-24T08:26:28.472 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-04-24T08:26:28.472 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-04-24T08:26:28.472 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-04-24T08:26:28.472 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-04-24T08:26:28.472 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-04-24T08:26:28.472 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-04-24T08:26:28.473 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-04-24T08:26:28.473 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-04-24T08:26:28.473 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-04-24T08:26:28.473 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-04-24T08:26:28.473 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-04-24T08:26:28.473 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-04-24T08:26:28.473 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-04-24T08:26:28.473 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-04-24T08:26:28.473 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink 2024-04-24T08:26:28.473 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-04-24T08:26:28.473 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh 2024-04-24T08:26:28.473 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-04-24T08:26:28.473 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch 2024-04-24T08:26:28.473 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-04-24T08:26:28.473 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-04-24T08:26:28.474 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-04-24T08:26:28.474 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-04-24T08:26:28.474 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet 2024-04-24T08:26:28.474 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-04-24T08:26:28.474 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds 2024-04-24T08:26:28.474 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-04-24T08:26:28.474 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-04-24T08:26:28.474 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-04-24T08:26:28.474 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched 2024-04-24T08:26:28.474 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-04-24T08:26:28.474 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-04-24T08:26:28.474 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-04-24T08:26:28.474 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-04-24T08:26:28.474 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-04-24T08:26:28.474 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-04-24T08:26:28.474 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-04-24T08:26:28.475 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-04-24T08:26:28.475 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-04-24T08:26:28.475 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-04-24T08:26:28.475 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-04-24T08:26:28.475 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-04-24T08:26:28.475 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-04-24T08:26:28.475 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-04-24T08:26:28.475 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-04-24T08:26:28.475 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-04-24T08:26:28.475 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-04-24T08:26:28.475 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-04-24T08:26:28.475 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-04-24T08:26:28.475 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-04-24T08:26:28.475 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-04-24T08:26:28.475 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-04-24T08:26:28.475 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-04-24T08:26:28.476 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-04-24T08:26:28.476 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-04-24T08:26:28.476 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-04-24T08:26:28.476 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-04-24T08:26:28.476 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-04-24T08:26:28.476 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-04-24T08:26:28.476 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-04-24T08:26:28.476 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-04-24T08:26:28.476 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-04-24T08:26:28.476 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-04-24T08:26:28.476 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-04-24T08:26:28.476 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-04-24T08:26:28.476 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-04-24T08:26:28.476 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-04-24T08:26:28.477 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-04-24T08:26:28.477 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-04-24T08:26:28.477 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-04-24T08:26:28.477 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-04-24T08:26:28.477 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-04-24T08:26:28.477 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-04-24T08:26:28.477 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp 2024-04-24T08:26:28.477 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-04-24T08:26:28.477 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-04-24T08:26:28.477 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc 2024-04-24T08:26:28.477 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss 2024-04-24T08:26:28.477 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-24T08:26:28.477 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-24T08:26:28.477 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-04-24T08:26:28.477 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma 2024-04-24T08:26:28.477 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-24T08:26:28.478 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc 2024-04-24T08:26:28.478 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-04-24T08:26:28.478 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-04-24T08:26:28.478 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix 2024-04-24T08:26:28.478 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-04-24T08:26:28.478 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock 2024-04-24T08:26:28.478 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-24T08:26:28.478 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-24T08:26:28.478 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-04-24T08:26:28.478 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-24T08:26:28.478 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-24T08:26:28.478 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm 2024-04-24T08:26:28.478 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-24T08:26:28.478 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security 2024-04-24T08:26:28.479 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys 2024-04-24T08:26:28.479 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys 2024-04-24T08:26:28.479 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-04-24T08:26:28.479 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys 2024-04-24T08:26:28.479 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys/trusted.ko 2024-04-24T08:26:28.479 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-04-24T08:26:28.479 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-04-24T08:26:28.479 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-04-24T08:26:28.479 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-04-24T08:26:28.479 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-04-24T08:26:28.479 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-04-24T08:26:28.479 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-04-24T08:26:28.479 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-04-24T08:26:28.479 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-04-24T08:26:28.479 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-04-24T08:26:28.479 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-04-24T08:26:28.480 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-04-24T08:26:28.480 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-04-24T08:26:28.480 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-24T08:26:28.484 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-04-24T08:26:28.484 DEBUG:teuthology.orchestra.run.smithi083:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-04-24T08:26:28.506 INFO:teuthology.orchestra.run.smithi083.stdout:postinstall scriptlet (using /bin/sh): 2024-04-24T08:26:28.506 INFO:teuthology.orchestra.run.smithi083.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-24T08:26:28.506 INFO:teuthology.orchestra.run.smithi083.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g806ec36b2112 /lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-24T08:26:28.506 INFO:teuthology.orchestra.run.smithi083.stdout:fi 2024-04-24T08:26:28.507 INFO:teuthology.orchestra.run.smithi083.stdout:for file in vmlinuz System.map config; do 2024-04-24T08:26:28.507 INFO:teuthology.orchestra.run.smithi083.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112"; then 2024-04-24T08:26:28.507 INFO:teuthology.orchestra.run.smithi083.stdout: cp "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112" 2024-04-24T08:26:28.507 INFO:teuthology.orchestra.run.smithi083.stdout: fi 2024-04-24T08:26:28.507 INFO:teuthology.orchestra.run.smithi083.stdout:done 2024-04-24T08:26:28.507 INFO:teuthology.orchestra.run.smithi083.stdout:if [ -d "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" ] && \ 2024-04-24T08:26:28.507 INFO:teuthology.orchestra.run.smithi083.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" >/dev/null 2>&1; then 2024-04-24T08:26:28.507 INFO:teuthology.orchestra.run.smithi083.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-24T08:26:28.507 INFO:teuthology.orchestra.run.smithi083.stdout: cp -r "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-24T08:26:28.507 INFO:teuthology.orchestra.run.smithi083.stdout:fi 2024-04-24T08:26:28.507 INFO:teuthology.orchestra.run.smithi083.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep" ]; then 2024-04-24T08:26:28.507 INFO:teuthology.orchestra.run.smithi083.stdout: /usr/sbin/depmod 6.9.0-rc3-g806ec36b2112 2024-04-24T08:26:28.507 INFO:teuthology.orchestra.run.smithi083.stdout:fi 2024-04-24T08:26:28.507 INFO:teuthology.orchestra.run.smithi083.stdout:preuninstall scriptlet (using /bin/sh): 2024-04-24T08:26:28.507 INFO:teuthology.orchestra.run.smithi083.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-24T08:26:28.507 INFO:teuthology.orchestra.run.smithi083.stdout:kernel-install remove 6.9.0-rc3-g806ec36b2112 2024-04-24T08:26:28.508 INFO:teuthology.orchestra.run.smithi083.stdout:fi 2024-04-24T08:26:28.508 DEBUG:teuthology.orchestra.run.smithi083:> sudo rpm -qi grub2-tools 2024-04-24T08:26:28.583 INFO:teuthology.orchestra.run.smithi083.stdout:Name : grub2-tools 2024-04-24T08:26:28.583 INFO:teuthology.orchestra.run.smithi083.stdout:Epoch : 1 2024-04-24T08:26:28.583 INFO:teuthology.orchestra.run.smithi083.stdout:Version : 2.06 2024-04-24T08:26:28.583 INFO:teuthology.orchestra.run.smithi083.stdout:Release : 68.el9 2024-04-24T08:26:28.584 INFO:teuthology.orchestra.run.smithi083.stdout:Architecture: x86_64 2024-04-24T08:26:28.584 INFO:teuthology.orchestra.run.smithi083.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-04-24T08:26:28.584 INFO:teuthology.orchestra.run.smithi083.stdout:Group : Unspecified 2024-04-24T08:26:28.584 INFO:teuthology.orchestra.run.smithi083.stdout:Size : 8264254 2024-04-24T08:26:28.584 INFO:teuthology.orchestra.run.smithi083.stdout:License : GPLv3+ 2024-04-24T08:26:28.584 INFO:teuthology.orchestra.run.smithi083.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-24T08:26:28.584 INFO:teuthology.orchestra.run.smithi083.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-24T08:26:28.584 INFO:teuthology.orchestra.run.smithi083.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-24T08:26:28.584 INFO:teuthology.orchestra.run.smithi083.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-24T08:26:28.584 INFO:teuthology.orchestra.run.smithi083.stdout:Packager : builder@centos.org 2024-04-24T08:26:28.584 INFO:teuthology.orchestra.run.smithi083.stdout:Vendor : CentOS 2024-04-24T08:26:28.584 INFO:teuthology.orchestra.run.smithi083.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-24T08:26:28.584 INFO:teuthology.orchestra.run.smithi083.stdout:Summary : Support tools for GRUB. 2024-04-24T08:26:28.584 INFO:teuthology.orchestra.run.smithi083.stdout:Description : 2024-04-24T08:26:28.584 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:26:28.585 INFO:teuthology.orchestra.run.smithi083.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-24T08:26:28.585 INFO:teuthology.orchestra.run.smithi083.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-24T08:26:28.585 INFO:teuthology.orchestra.run.smithi083.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-24T08:26:28.585 INFO:teuthology.orchestra.run.smithi083.stdout:hardware devices. 2024-04-24T08:26:28.585 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:26:28.585 INFO:teuthology.orchestra.run.smithi083.stdout:This subpackage provides tools for support of all platforms. 2024-04-24T08:26:28.586 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-24T08:26:28.586 INFO:teuthology.task.kernel:Updating grub on smithi083 to boot 6.9.0-rc3-g806ec36b2112 2024-04-24T08:26:28.586 DEBUG:teuthology.orchestra.run.smithi083:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-24T08:26:29.165 INFO:teuthology.orchestra.run.smithi083.stderr:Generating grub configuration file ... 2024-04-24T08:26:30.558 INFO:teuthology.orchestra.run.smithi083.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-24T08:26:30.591 INFO:teuthology.orchestra.run.smithi083.stderr:done 2024-04-24T08:26:30.593 DEBUG:teuthology.orchestra.run.smithi083:> mktemp 2024-04-24T08:26:30.613 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/tmp.VmGBl9Bxij 2024-04-24T08:26:30.613 DEBUG:teuthology.orchestra.run.smithi083:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.VmGBl9Bxij 2024-04-24T08:26:30.698 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 0666 /tmp/tmp.VmGBl9Bxij 2024-04-24T08:26:30.800 DEBUG:teuthology.orchestra.remote:smithi083:/tmp/tmp.VmGBl9Bxij is 6KB 2024-04-24T08:26:30.849 DEBUG:teuthology.orchestra.run.smithi083:> rm -fr /tmp/tmp.VmGBl9Bxij 2024-04-24T08:26:30.863 DEBUG:teuthology.orchestra.run.smithi083:> sudo /bin/ls /boot/loader/entries || true 2024-04-24T08:26:30.939 INFO:teuthology.orchestra.run.smithi083.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-04-24T08:26:30.939 INFO:teuthology.orchestra.run.smithi083.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-04-24T08:26:30.939 INFO:teuthology.orchestra.run.smithi083.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-04-24T08:26:30.939 INFO:teuthology.orchestra.run.smithi083.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112.conf 2024-04-24T08:26:30.940 DEBUG:teuthology.orchestra.run.smithi083:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112 2024-04-24T08:26:31.094 DEBUG:teuthology.orchestra.run.smithi083:> sudo shutdown -r now 2024-04-24T08:26:36.021 INFO:teuthology.orchestra.run.smithi138.stdout:kernel-6.9.0_rc3_g806ec36b2112-1 ######################################## 2024-04-24T08:26:59.066 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi189.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-04-24T08:26:59.066 DEBUG:teuthology.orchestra.run.smithi189:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-04-24T08:26:59.113 INFO:teuthology.orchestra.run.smithi189.stdout:/boot/System.map-6.9.0-rc3-g806ec36b2112 2024-04-24T08:26:59.114 INFO:teuthology.orchestra.run.smithi189.stdout:/boot/config-6.9.0-rc3-g806ec36b2112 2024-04-24T08:26:59.114 INFO:teuthology.orchestra.run.smithi189.stdout:/boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-24T08:26:59.114 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/System.map 2024-04-24T08:26:59.114 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/config 2024-04-24T08:26:59.114 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel 2024-04-24T08:26:59.114 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch 2024-04-24T08:26:59.114 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86 2024-04-24T08:26:59.114 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto 2024-04-24T08:26:59.114 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-24T08:26:59.114 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-24T08:26:59.115 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-24T08:26:59.115 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-24T08:26:59.115 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-24T08:26:59.115 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-24T08:26:59.115 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-24T08:26:59.115 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32c-intel.ko 2024-04-24T08:26:59.115 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-24T08:26:59.115 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-24T08:26:59.115 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-24T08:26:59.115 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-24T08:26:59.115 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-24T08:26:59.115 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-24T08:26:59.115 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-24T08:26:59.115 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-24T08:26:59.116 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-24T08:26:59.116 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-24T08:26:59.116 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-24T08:26:59.116 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-24T08:26:59.116 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel 2024-04-24T08:26:59.116 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu 2024-04-24T08:26:59.116 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce 2024-04-24T08:26:59.116 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-24T08:26:59.116 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm 2024-04-24T08:26:59.116 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-24T08:26:59.116 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-24T08:26:59.116 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-04-24T08:26:59.116 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto 2024-04-24T08:26:59.117 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-04-24T08:26:59.117 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-04-24T08:26:59.117 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-04-24T08:26:59.117 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx 2024-04-24T08:26:59.117 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-04-24T08:26:59.117 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-04-24T08:26:59.117 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-24T08:26:59.117 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-04-24T08:26:59.117 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-04-24T08:26:59.117 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-04-24T08:26:59.117 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-04-24T08:26:59.117 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-04-24T08:26:59.117 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-04-24T08:26:59.117 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-04-24T08:26:59.118 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-04-24T08:26:59.118 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-04-24T08:26:59.118 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-04-24T08:26:59.118 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-04-24T08:26:59.118 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-04-24T08:26:59.118 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-04-24T08:26:59.118 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-04-24T08:26:59.118 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-04-24T08:26:59.118 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-04-24T08:26:59.118 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-04-24T08:26:59.118 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-04-24T08:26:59.118 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-04-24T08:26:59.119 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-04-24T08:26:59.119 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-04-24T08:26:59.119 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-04-24T08:26:59.119 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-04-24T08:26:59.119 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-04-24T08:26:59.119 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-04-24T08:26:59.119 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-04-24T08:26:59.119 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-04-24T08:26:59.119 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-04-24T08:26:59.119 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-04-24T08:26:59.119 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-04-24T08:26:59.119 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-04-24T08:26:59.120 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-04-24T08:26:59.120 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-04-24T08:26:59.120 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-04-24T08:26:59.120 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-04-24T08:26:59.120 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-04-24T08:26:59.120 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-04-24T08:26:59.120 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-04-24T08:26:59.120 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-04-24T08:26:59.120 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-04-24T08:26:59.120 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-04-24T08:26:59.120 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-04-24T08:26:59.121 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers 2024-04-24T08:26:59.121 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi 2024-04-24T08:26:59.121 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-24T08:26:59.121 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-04-24T08:26:59.121 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-04-24T08:26:59.121 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-04-24T08:26:59.121 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-04-24T08:26:59.121 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-04-24T08:26:59.121 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-04-24T08:26:59.121 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata 2024-04-24T08:26:59.121 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-04-24T08:26:59.121 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-04-24T08:26:59.122 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ata_generic.ko 2024-04-24T08:26:59.122 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-04-24T08:26:59.122 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-04-24T08:26:59.122 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-04-24T08:26:59.122 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-04-24T08:26:59.122 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-04-24T08:26:59.122 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-04-24T08:26:59.122 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-04-24T08:26:59.122 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-04-24T08:26:59.122 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-04-24T08:26:59.122 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-04-24T08:26:59.122 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-04-24T08:26:59.122 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-04-24T08:26:59.154 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-04-24T08:26:59.155 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-24T08:26:59.155 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-24T08:26:59.155 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-04-24T08:26:59.155 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-04-24T08:26:59.155 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-04-24T08:26:59.155 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-04-24T08:26:59.155 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-04-24T08:26:59.155 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-04-24T08:26:59.155 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-04-24T08:26:59.156 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-04-24T08:26:59.156 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-04-24T08:26:59.156 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-04-24T08:26:59.156 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-04-24T08:26:59.156 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-04-24T08:26:59.156 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-04-24T08:26:59.156 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-24T08:26:59.156 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-24T08:26:59.156 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-04-24T08:26:59.156 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-04-24T08:26:59.156 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-04-24T08:26:59.156 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-04-24T08:26:59.156 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-04-24T08:26:59.156 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sis.ko 2024-04-24T08:26:59.156 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-04-24T08:26:59.157 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-04-24T08:26:59.157 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-04-24T08:26:59.157 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-04-24T08:26:59.157 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-04-24T08:26:59.157 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-04-24T08:26:59.157 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-04-24T08:26:59.157 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-04-24T08:26:59.157 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-04-24T08:26:59.157 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-04-24T08:26:59.157 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-04-24T08:26:59.157 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-04-24T08:26:59.157 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-04-24T08:26:59.157 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-04-24T08:26:59.157 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-04-24T08:26:59.157 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-04-24T08:26:59.158 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-04-24T08:26:59.158 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay 2024-04-24T08:26:59.158 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-04-24T08:26:59.158 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base 2024-04-24T08:26:59.158 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap 2024-04-24T08:26:59.158 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-i2c.ko 2024-04-24T08:26:59.158 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-mmio.ko 2024-04-24T08:26:59.158 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma 2024-04-24T08:26:59.158 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-04-24T08:26:59.158 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block 2024-04-24T08:26:59.158 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe 2024-04-24T08:26:59.158 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-04-24T08:26:59.158 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/brd.ko 2024-04-24T08:26:59.158 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd 2024-04-24T08:26:59.159 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-04-24T08:26:59.159 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-04-24T08:26:59.159 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/loop.ko 2024-04-24T08:26:59.159 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx 2024-04-24T08:26:59.159 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-24T08:26:59.159 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-04-24T08:26:59.159 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk 2024-04-24T08:26:59.160 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-04-24T08:26:59.160 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-04-24T08:26:59.160 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-04-24T08:26:59.160 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/virtio_blk.ko 2024-04-24T08:26:59.160 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback 2024-04-24T08:26:59.160 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-24T08:26:59.160 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkfront.ko 2024-04-24T08:26:59.161 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram 2024-04-24T08:26:59.161 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-04-24T08:26:59.161 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char 2024-04-24T08:26:59.161 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-04-24T08:26:59.161 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random 2024-04-24T08:26:59.161 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-24T08:26:59.161 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-24T08:26:59.161 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-24T08:26:59.162 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-04-24T08:26:59.162 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-24T08:26:59.162 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi 2024-04-24T08:26:59.162 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-24T08:26:59.162 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-24T08:26:59.162 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-24T08:26:59.162 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-24T08:26:59.163 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-24T08:26:59.163 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-24T08:26:59.163 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-04-24T08:26:59.163 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave 2024-04-24T08:26:59.163 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-04-24T08:26:59.163 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-04-24T08:26:59.163 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-04-24T08:26:59.163 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm 2024-04-24T08:26:59.163 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm.ko 2024-04-24T08:26:59.163 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-24T08:26:59.163 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-24T08:26:59.163 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-24T08:26:59.164 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis.ko 2024-04-24T08:26:59.164 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-04-24T08:26:59.164 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/uv_mmtimer.ko 2024-04-24T08:26:59.164 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/virtio_console.ko 2024-04-24T08:26:59.164 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq 2024-04-24T08:26:59.164 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-04-24T08:26:59.164 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-24T08:26:59.164 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-24T08:26:59.164 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-04-24T08:26:59.164 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/powernow-k8.ko 2024-04-24T08:26:59.164 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-24T08:26:59.165 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto 2024-04-24T08:26:59.165 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp 2024-04-24T08:26:59.165 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-24T08:26:59.165 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-04-24T08:26:59.165 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel 2024-04-24T08:26:59.165 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat 2024-04-24T08:26:59.165 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common 2024-04-24T08:26:59.165 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-24T08:26:59.165 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-04-24T08:26:59.165 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-24T08:26:59.166 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-04-24T08:26:59.166 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-04-24T08:26:59.166 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio 2024-04-24T08:26:59.166 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-24T08:26:59.166 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca 2024-04-24T08:26:59.166 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-04-24T08:26:59.166 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq 2024-04-24T08:26:59.166 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-04-24T08:26:59.166 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma 2024-04-24T08:26:59.166 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw 2024-04-24T08:26:59.167 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-24T08:26:59.167 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat 2024-04-24T08:26:59.167 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-24T08:26:59.167 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac 2024-04-24T08:26:59.167 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-04-24T08:26:59.167 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-04-24T08:26:59.167 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-04-24T08:26:59.167 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-04-24T08:26:59.167 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-04-24T08:26:59.168 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-04-24T08:26:59.168 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-04-24T08:26:59.168 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-04-24T08:26:59.168 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-04-24T08:26:59.168 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-04-24T08:26:59.168 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-04-24T08:26:59.168 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-04-24T08:26:59.168 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-04-24T08:26:59.168 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware 2024-04-24T08:26:59.168 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/edd.ko 2024-04-24T08:26:59.169 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-24T08:26:59.169 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio 2024-04-24T08:26:59.169 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-04-24T08:26:59.169 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-24T08:26:59.169 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid 2024-04-24T08:26:59.169 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-04-24T08:26:59.169 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-04-24T08:26:59.169 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-04-24T08:26:59.169 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-04-24T08:26:59.170 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-04-24T08:26:59.170 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-04-24T08:26:59.170 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-04-24T08:26:59.170 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-04-24T08:26:59.170 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-04-24T08:26:59.170 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-04-24T08:26:59.170 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-24T08:26:59.170 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-24T08:26:59.170 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-04-24T08:26:59.170 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-04-24T08:26:59.171 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-04-24T08:26:59.171 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-04-24T08:26:59.171 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-04-24T08:26:59.171 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-04-24T08:26:59.171 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-04-24T08:26:59.171 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-04-24T08:26:59.171 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-24T08:26:59.171 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-24T08:26:59.171 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-04-24T08:26:59.172 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-04-24T08:26:59.172 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-04-24T08:26:59.172 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-04-24T08:26:59.172 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-04-24T08:26:59.172 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-04-24T08:26:59.172 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-04-24T08:26:59.172 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-04-24T08:26:59.172 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-04-24T08:26:59.173 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-24T08:26:59.173 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-04-24T08:26:59.173 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-24T08:26:59.173 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-24T08:26:59.173 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-24T08:26:59.173 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-24T08:26:59.173 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-24T08:26:59.173 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-24T08:26:59.174 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-24T08:26:59.174 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-24T08:26:59.174 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-24T08:26:59.174 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-04-24T08:26:59.174 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-04-24T08:26:59.174 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-04-24T08:26:59.174 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-24T08:26:59.174 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-04-24T08:26:59.174 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-04-24T08:26:59.174 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-04-24T08:26:59.174 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-04-24T08:26:59.174 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-04-24T08:26:59.175 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-24T08:26:59.175 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-04-24T08:26:59.175 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-04-24T08:26:59.175 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-04-24T08:26:59.175 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-04-24T08:26:59.175 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-04-24T08:26:59.175 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-04-24T08:26:59.175 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-04-24T08:26:59.175 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-04-24T08:26:59.175 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-04-24T08:26:59.175 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-04-24T08:26:59.175 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-04-24T08:26:59.175 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-04-24T08:26:59.176 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv 2024-04-24T08:26:59.176 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-04-24T08:26:59.176 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-04-24T08:26:59.176 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-04-24T08:26:59.176 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon 2024-04-24T08:26:59.176 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-04-24T08:26:59.176 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-04-24T08:26:59.176 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-24T08:26:59.176 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-04-24T08:26:59.176 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-04-24T08:26:59.176 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-04-24T08:26:59.176 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-04-24T08:26:59.176 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-04-24T08:26:59.176 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-04-24T08:26:59.177 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-04-24T08:26:59.177 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-04-24T08:26:59.177 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-04-24T08:26:59.177 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-04-24T08:26:59.177 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-04-24T08:26:59.177 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-04-24T08:26:59.177 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-04-24T08:26:59.177 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-04-24T08:26:59.177 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-04-24T08:26:59.177 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-04-24T08:26:59.177 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-04-24T08:26:59.177 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-04-24T08:26:59.177 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-04-24T08:26:59.178 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-24T08:26:59.178 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-04-24T08:26:59.178 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-04-24T08:26:59.178 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-24T08:26:59.178 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-04-24T08:26:59.178 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-04-24T08:26:59.178 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-04-24T08:26:59.178 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-04-24T08:26:59.178 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-04-24T08:26:59.178 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-04-24T08:26:59.178 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-04-24T08:26:59.178 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-04-24T08:26:59.178 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-04-24T08:26:59.179 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-04-24T08:26:59.179 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-04-24T08:26:59.179 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-04-24T08:26:59.179 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-04-24T08:26:59.179 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-04-24T08:26:59.179 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-24T08:26:59.179 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-04-24T08:26:59.179 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-04-24T08:26:59.179 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-04-24T08:26:59.179 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-04-24T08:26:59.179 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-04-24T08:26:59.179 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-04-24T08:26:59.179 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-04-24T08:26:59.179 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-04-24T08:26:59.179 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-04-24T08:26:59.180 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-04-24T08:26:59.180 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-04-24T08:26:59.180 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-04-24T08:26:59.180 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-04-24T08:26:59.180 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-04-24T08:26:59.180 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-04-24T08:26:59.180 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-04-24T08:26:59.180 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-04-24T08:26:59.180 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-04-24T08:26:59.180 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-04-24T08:26:59.180 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-04-24T08:26:59.180 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-04-24T08:26:59.180 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-04-24T08:26:59.181 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-04-24T08:26:59.181 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-04-24T08:26:59.181 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-04-24T08:26:59.181 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-04-24T08:26:59.181 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-04-24T08:26:59.181 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-04-24T08:26:59.181 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-04-24T08:26:59.181 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-04-24T08:26:59.181 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-04-24T08:26:59.181 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-04-24T08:26:59.181 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-04-24T08:26:59.182 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-04-24T08:26:59.182 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-04-24T08:26:59.182 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-04-24T08:26:59.182 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-04-24T08:26:59.182 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-04-24T08:26:59.182 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-04-24T08:26:59.182 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-04-24T08:26:59.182 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-04-24T08:26:59.182 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-04-24T08:26:59.182 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-04-24T08:26:59.182 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-04-24T08:26:59.182 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-04-24T08:26:59.182 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-04-24T08:26:59.182 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-04-24T08:26:59.183 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus 2024-04-24T08:26:59.183 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-24T08:26:59.183 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-24T08:26:59.183 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-24T08:26:59.183 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-24T08:26:59.183 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-24T08:26:59.183 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-24T08:26:59.183 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-24T08:26:59.183 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-24T08:26:59.183 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-24T08:26:59.183 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-24T08:26:59.183 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-24T08:26:59.184 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-24T08:26:59.184 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-04-24T08:26:59.184 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-04-24T08:26:59.184 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-04-24T08:26:59.184 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-24T08:26:59.184 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-04-24T08:26:59.184 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-04-24T08:26:59.184 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-04-24T08:26:59.184 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-04-24T08:26:59.184 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-04-24T08:26:59.184 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-04-24T08:26:59.185 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-04-24T08:26:59.185 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-04-24T08:26:59.185 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-04-24T08:26:59.185 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-04-24T08:26:59.185 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-04-24T08:26:59.185 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-04-24T08:26:59.185 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-04-24T08:26:59.185 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-04-24T08:26:59.185 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-04-24T08:26:59.186 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-04-24T08:26:59.186 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-04-24T08:26:59.186 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-04-24T08:26:59.186 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-04-24T08:26:59.186 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-04-24T08:26:59.186 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-04-24T08:26:59.186 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-04-24T08:26:59.186 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-04-24T08:26:59.186 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-04-24T08:26:59.186 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-04-24T08:26:59.187 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c 2024-04-24T08:26:59.187 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos 2024-04-24T08:26:59.187 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-24T08:26:59.187 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-24T08:26:59.187 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses 2024-04-24T08:26:59.187 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-24T08:26:59.187 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-24T08:26:59.187 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-24T08:26:59.187 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-24T08:26:59.187 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-24T08:26:59.187 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-24T08:26:59.187 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-24T08:26:59.187 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-24T08:26:59.187 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-24T08:26:59.188 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-24T08:26:59.188 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-24T08:26:59.188 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-24T08:26:59.188 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-24T08:26:59.188 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-24T08:26:59.188 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-24T08:26:59.188 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-24T08:26:59.188 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-24T08:26:59.188 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-24T08:26:59.188 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-24T08:26:59.188 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-24T08:26:59.188 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-24T08:26:59.189 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-24T08:26:59.189 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-24T08:26:59.189 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-dev.ko 2024-04-24T08:26:59.189 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-04-24T08:26:59.189 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-04-24T08:26:59.189 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-04-24T08:26:59.189 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-04-24T08:26:59.189 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband 2024-04-24T08:26:59.189 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core 2024-04-24T08:26:59.189 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-24T08:26:59.189 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-04-24T08:26:59.189 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-24T08:26:59.189 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-24T08:26:59.190 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-24T08:26:59.190 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-24T08:26:59.190 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-24T08:26:59.190 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw 2024-04-24T08:26:59.190 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4 2024-04-24T08:26:59.190 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-24T08:26:59.190 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4 2024-04-24T08:26:59.190 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-24T08:26:59.190 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5 2024-04-24T08:26:59.190 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-24T08:26:59.190 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca 2024-04-24T08:26:59.190 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-24T08:26:59.191 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp 2024-04-24T08:26:59.191 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib 2024-04-24T08:26:59.191 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-24T08:26:59.191 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser 2024-04-24T08:26:59.191 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-24T08:26:59.191 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert 2024-04-24T08:26:59.191 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-24T08:26:59.191 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp 2024-04-24T08:26:59.191 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-24T08:26:59.191 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt 2024-04-24T08:26:59.191 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-24T08:26:59.191 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input 2024-04-24T08:26:59.191 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport 2024-04-24T08:26:59.192 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-24T08:26:59.192 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-24T08:26:59.192 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-04-24T08:26:59.192 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-04-24T08:26:59.192 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard 2024-04-24T08:26:59.192 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-24T08:26:59.192 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse 2024-04-24T08:26:59.192 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-04-24T08:26:59.192 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-04-24T08:26:59.192 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-04-24T08:26:59.193 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-24T08:26:59.193 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-04-24T08:26:59.193 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-24T08:26:59.193 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-24T08:26:59.193 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-24T08:26:59.193 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4 2024-04-24T08:26:59.193 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-24T08:26:59.193 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio 2024-04-24T08:26:59.193 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-04-24T08:26:59.193 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-04-24T08:26:59.193 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-24T08:26:59.193 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-04-24T08:26:59.193 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-04-24T08:26:59.194 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds 2024-04-24T08:26:59.194 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-04-24T08:26:59.194 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-04-24T08:26:59.194 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-04-24T08:26:59.194 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-04-24T08:26:59.194 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger 2024-04-24T08:26:59.194 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-24T08:26:59.194 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-24T08:26:59.194 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-24T08:26:59.194 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-24T08:26:59.194 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-24T08:26:59.195 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-24T08:26:59.195 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-24T08:26:59.195 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-24T08:26:59.195 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-24T08:26:59.195 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md 2024-04-24T08:26:59.195 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache 2024-04-24T08:26:59.195 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-04-24T08:26:59.195 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-04-24T08:26:59.195 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-04-24T08:26:59.195 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-04-24T08:26:59.195 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-04-24T08:26:59.195 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-04-24T08:26:59.195 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-04-24T08:26:59.196 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-04-24T08:26:59.196 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-04-24T08:26:59.196 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-04-24T08:26:59.196 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-04-24T08:26:59.196 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-04-24T08:26:59.196 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-04-24T08:26:59.196 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-04-24T08:26:59.196 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-04-24T08:26:59.196 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-04-24T08:26:59.196 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data 2024-04-24T08:26:59.196 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-24T08:26:59.196 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-04-24T08:26:59.196 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-04-24T08:26:59.196 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-04-24T08:26:59.197 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-04-24T08:26:59.197 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message 2024-04-24T08:26:59.197 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion 2024-04-24T08:26:59.197 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-04-24T08:26:59.197 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-04-24T08:26:59.197 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-04-24T08:26:59.197 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-04-24T08:26:59.197 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-04-24T08:26:59.197 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-04-24T08:26:59.197 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-04-24T08:26:59.197 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd 2024-04-24T08:26:59.197 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-04-24T08:26:59.198 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-04-24T08:26:59.198 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mfd-core.ko 2024-04-24T08:26:59.198 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-04-24T08:26:59.198 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-04-24T08:26:59.198 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-04-24T08:26:59.198 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-04-24T08:26:59.198 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc 2024-04-24T08:26:59.198 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl 2024-04-24T08:26:59.198 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-24T08:26:59.198 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-04-24T08:26:59.198 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-04-24T08:26:59.198 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-04-24T08:26:59.199 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710 2024-04-24T08:26:59.199 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-04-24T08:26:59.199 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom 2024-04-24T08:26:59.199 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-04-24T08:26:59.199 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-24T08:26:59.199 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-04-24T08:26:59.199 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-04-24T08:26:59.199 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-04-24T08:26:59.199 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-04-24T08:26:59.199 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-04-24T08:26:59.199 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d 2024-04-24T08:26:59.199 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-24T08:26:59.200 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-24T08:26:59.200 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei 2024-04-24T08:26:59.200 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-04-24T08:26:59.200 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-04-24T08:26:59.200 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-04-24T08:26:59.200 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru 2024-04-24T08:26:59.200 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru/gru.ko 2024-04-24T08:26:59.200 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp 2024-04-24T08:26:59.200 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xp.ko 2024-04-24T08:26:59.200 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpc.ko 2024-04-24T08:26:59.201 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-04-24T08:26:59.201 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-04-24T08:26:59.201 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-04-24T08:26:59.201 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-04-24T08:26:59.201 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-04-24T08:26:59.201 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci 2024-04-24T08:26:59.201 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-24T08:26:59.201 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net 2024-04-24T08:26:59.201 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding 2024-04-24T08:26:59.201 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-04-24T08:26:59.201 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa 2024-04-24T08:26:59.202 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53 2024-04-24T08:26:59.202 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-24T08:26:59.202 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-24T08:26:59.202 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-24T08:26:59.202 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx 2024-04-24T08:26:59.202 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-24T08:26:59.202 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-04-24T08:26:59.202 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-04-24T08:26:59.202 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet 2024-04-24T08:26:59.203 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com 2024-04-24T08:26:59.203 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-24T08:26:59.203 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-24T08:26:59.203 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-24T08:26:59.203 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-24T08:26:59.203 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390 2024-04-24T08:26:59.203 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-24T08:26:59.203 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-24T08:26:59.203 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-24T08:26:59.203 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-24T08:26:59.203 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec 2024-04-24T08:26:59.203 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-24T08:26:59.204 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere 2024-04-24T08:26:59.204 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-24T08:26:59.204 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon 2024-04-24T08:26:59.204 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-24T08:26:59.204 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera 2024-04-24T08:26:59.204 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-24T08:26:59.204 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd 2024-04-24T08:26:59.204 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-24T08:26:59.204 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-24T08:26:59.204 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-24T08:26:59.204 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros 2024-04-24T08:26:59.204 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx 2024-04-24T08:26:59.204 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-24T08:26:59.205 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c 2024-04-24T08:26:59.205 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-24T08:26:59.205 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e 2024-04-24T08:26:59.205 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-24T08:26:59.205 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx 2024-04-24T08:26:59.205 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-24T08:26:59.205 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-24T08:26:59.205 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom 2024-04-24T08:26:59.205 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-24T08:26:59.205 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-24T08:26:59.205 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-04-24T08:26:59.205 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-24T08:26:59.205 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-24T08:26:59.206 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet 2024-04-24T08:26:59.206 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-24T08:26:59.206 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-24T08:26:59.206 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade 2024-04-24T08:26:59.206 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna 2024-04-24T08:26:59.206 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-24T08:26:59.206 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio 2024-04-24T08:26:59.206 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb 2024-04-24T08:26:59.206 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-24T08:26:59.206 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-04-24T08:26:59.206 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-24T08:26:59.207 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-04-24T08:26:59.207 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-24T08:26:59.207 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-04-24T08:26:59.207 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-24T08:26:59.207 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-04-24T08:26:59.207 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-24T08:26:59.207 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco 2024-04-24T08:26:59.207 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic 2024-04-24T08:26:59.207 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-24T08:26:59.208 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec 2024-04-24T08:26:59.208 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip 2024-04-24T08:26:59.208 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-24T08:26:59.208 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-24T08:26:59.208 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-24T08:26:59.208 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-24T08:26:59.208 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-24T08:26:59.208 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-24T08:26:59.208 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink 2024-04-24T08:26:59.208 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-24T08:26:59.209 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-24T08:26:59.209 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-04-24T08:26:59.209 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex 2024-04-24T08:26:59.209 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet 2024-04-24T08:26:59.209 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-24T08:26:59.209 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-04-24T08:26:59.209 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-04-24T08:26:59.209 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel 2024-04-24T08:26:59.209 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-24T08:26:59.209 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000 2024-04-24T08:26:59.210 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-24T08:26:59.210 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e 2024-04-24T08:26:59.210 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-24T08:26:59.210 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k 2024-04-24T08:26:59.210 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-24T08:26:59.210 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e 2024-04-24T08:26:59.210 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-24T08:26:59.210 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf 2024-04-24T08:26:59.210 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-24T08:26:59.210 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb 2024-04-24T08:26:59.210 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-24T08:26:59.210 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf 2024-04-24T08:26:59.211 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-24T08:26:59.211 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe 2024-04-24T08:26:59.211 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-24T08:26:59.211 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf 2024-04-24T08:26:59.211 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-24T08:26:59.211 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-04-24T08:26:59.211 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell 2024-04-24T08:26:59.211 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-24T08:26:59.211 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-24T08:26:59.211 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-24T08:26:59.211 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox 2024-04-24T08:26:59.211 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4 2024-04-24T08:26:59.211 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-24T08:26:59.211 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-24T08:26:59.212 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5 2024-04-24T08:26:59.212 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-04-24T08:26:59.212 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-24T08:26:59.212 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel 2024-04-24T08:26:59.212 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-24T08:26:59.212 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom 2024-04-24T08:26:59.212 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge 2024-04-24T08:26:59.212 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-24T08:26:59.212 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi 2024-04-24T08:26:59.212 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-24T08:26:59.212 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-24T08:26:59.212 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion 2024-04-24T08:26:59.213 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-24T08:26:59.213 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia 2024-04-24T08:26:59.213 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-24T08:26:59.213 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines 2024-04-24T08:26:59.213 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-24T08:26:59.213 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-24T08:26:59.213 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic 2024-04-24T08:26:59.213 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen 2024-04-24T08:26:59.213 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-24T08:26:59.213 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-24T08:26:59.213 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-04-24T08:26:59.213 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-24T08:26:59.213 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc 2024-04-24T08:26:59.214 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-24T08:26:59.214 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek 2024-04-24T08:26:59.214 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-24T08:26:59.214 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-24T08:26:59.214 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-24T08:26:59.214 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-24T08:26:59.214 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker 2024-04-24T08:26:59.214 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-04-24T08:26:59.214 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc 2024-04-24T08:26:59.214 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-24T08:26:59.214 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan 2024-04-24T08:26:59.215 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-24T08:26:59.215 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis 2024-04-24T08:26:59.215 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-24T08:26:59.215 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-24T08:26:59.215 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc 2024-04-24T08:26:59.215 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-24T08:26:59.215 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-24T08:26:59.215 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-24T08:26:59.216 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-24T08:26:59.216 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro 2024-04-24T08:26:59.216 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac 2024-04-24T08:26:59.216 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-24T08:26:59.216 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-24T08:26:59.216 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun 2024-04-24T08:26:59.216 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-24T08:26:59.216 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-24T08:26:59.216 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-24T08:26:59.216 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-24T08:26:59.216 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti 2024-04-24T08:26:59.216 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-24T08:26:59.217 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti 2024-04-24T08:26:59.217 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-24T08:26:59.217 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via 2024-04-24T08:26:59.217 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-24T08:26:59.217 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-24T08:26:59.217 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet 2024-04-24T08:26:59.217 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-24T08:26:59.217 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-24T08:26:59.217 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom 2024-04-24T08:26:59.217 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-24T08:26:59.218 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-04-24T08:26:59.218 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv 2024-04-24T08:26:59.218 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-24T08:26:59.218 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154 2024-04-24T08:26:59.218 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-24T08:26:59.218 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-04-24T08:26:59.218 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan 2024-04-24T08:26:59.218 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-24T08:26:59.218 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-04-24T08:26:59.218 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-04-24T08:26:59.218 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio 2024-04-24T08:26:59.218 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-04-24T08:26:59.218 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-24T08:26:59.219 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-24T08:26:59.219 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-24T08:26:59.219 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-04-24T08:26:59.219 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/net_failover.ko 2024-04-24T08:26:59.219 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-04-24T08:26:59.219 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-04-24T08:26:59.219 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs 2024-04-24T08:26:59.219 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-24T08:26:59.219 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-24T08:26:59.219 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy 2024-04-24T08:26:59.219 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-04-24T08:26:59.219 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-24T08:26:59.219 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-04-24T08:26:59.219 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-24T08:26:59.220 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-04-24T08:26:59.220 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-04-24T08:26:59.220 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-04-24T08:26:59.220 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-04-24T08:26:59.220 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/dp83640.ko 2024-04-24T08:26:59.220 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-04-24T08:26:59.220 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-04-24T08:26:59.220 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-04-24T08:26:59.220 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-04-24T08:26:59.220 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-04-24T08:26:59.220 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-04-24T08:26:59.220 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-04-24T08:26:59.220 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-04-24T08:26:59.220 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-04-24T08:26:59.220 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-04-24T08:26:59.220 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-04-24T08:26:59.221 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-04-24T08:26:59.221 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp 2024-04-24T08:26:59.221 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-24T08:26:59.221 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-04-24T08:26:59.221 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-24T08:26:59.221 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_generic.ko 2024-04-24T08:26:59.221 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-24T08:26:59.221 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-24T08:26:59.221 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-04-24T08:26:59.221 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-04-24T08:26:59.221 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-04-24T08:26:59.221 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip 2024-04-24T08:26:59.221 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slhc.ko 2024-04-24T08:26:59.221 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-04-24T08:26:59.221 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-04-24T08:26:59.222 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-04-24T08:26:59.222 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team 2024-04-24T08:26:59.222 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-04-24T08:26:59.222 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-24T08:26:59.222 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-24T08:26:59.222 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-24T08:26:59.222 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-04-24T08:26:59.222 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-24T08:26:59.222 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tun.ko 2024-04-24T08:26:59.222 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-04-24T08:26:59.222 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/virtio_net.ko 2024-04-24T08:26:59.222 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3 2024-04-24T08:26:59.222 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-24T08:26:59.222 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan 2024-04-24T08:26:59.223 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-04-24T08:26:59.223 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback 2024-04-24T08:26:59.223 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-24T08:26:59.223 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netfront.ko 2024-04-24T08:26:59.223 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb 2024-04-24T08:26:59.223 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-04-24T08:26:59.223 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme 2024-04-24T08:26:59.223 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host 2024-04-24T08:26:59.223 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-04-24T08:26:59.223 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-04-24T08:26:59.223 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport 2024-04-24T08:26:59.223 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-04-24T08:26:59.223 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-04-24T08:26:59.223 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-04-24T08:26:59.224 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-04-24T08:26:59.224 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci 2024-04-24T08:26:59.224 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug 2024-04-24T08:26:59.224 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-24T08:26:59.224 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie 2024-04-24T08:26:59.224 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie/aer_inject.ko 2024-04-24T08:26:59.224 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-04-24T08:26:59.224 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia 2024-04-24T08:26:59.224 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-04-24T08:26:59.225 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-04-24T08:26:59.225 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-24T08:26:59.225 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl 2024-04-24T08:26:59.225 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel 2024-04-24T08:26:59.225 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-24T08:26:59.225 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform 2024-04-24T08:26:59.225 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86 2024-04-24T08:26:59.226 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-24T08:26:59.226 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-04-24T08:26:59.226 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-24T08:26:59.226 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-24T08:26:59.226 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-24T08:26:59.226 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-24T08:26:59.226 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-24T08:26:59.226 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-24T08:26:59.226 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-24T08:26:59.226 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-24T08:26:59.226 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-24T08:26:59.227 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-04-24T08:26:59.227 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel 2024-04-24T08:26:59.227 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-24T08:26:59.227 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-04-24T08:26:59.227 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-24T08:26:59.227 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-24T08:26:59.227 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-24T08:26:59.227 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-24T08:26:59.227 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-24T08:26:59.227 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-24T08:26:59.227 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-24T08:26:59.227 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-24T08:26:59.227 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-24T08:26:59.228 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-04-24T08:26:59.228 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap 2024-04-24T08:26:59.228 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-24T08:26:59.228 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-24T08:26:59.228 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps 2024-04-24T08:26:59.228 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients 2024-04-24T08:26:59.228 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-24T08:26:59.228 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-24T08:26:59.228 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-04-24T08:26:59.228 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-04-24T08:26:59.228 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp 2024-04-24T08:26:59.228 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-04-24T08:26:59.228 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-04-24T08:26:59.228 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras 2024-04-24T08:26:59.229 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd 2024-04-24T08:26:59.229 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl 2024-04-24T08:26:59.229 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-24T08:26:59.229 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-04-24T08:26:59.229 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc 2024-04-24T08:26:59.229 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-24T08:26:59.229 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-24T08:26:59.229 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-24T08:26:59.229 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-24T08:26:59.229 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-24T08:26:59.229 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-24T08:26:59.229 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-24T08:26:59.229 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-24T08:26:59.230 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-24T08:26:59.230 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-24T08:26:59.230 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-04-24T08:26:59.230 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-24T08:26:59.230 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-24T08:26:59.230 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-24T08:26:59.230 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-24T08:26:59.230 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-24T08:26:59.230 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-24T08:26:59.230 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-04-24T08:26:59.230 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-24T08:26:59.231 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-24T08:26:59.231 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-24T08:26:59.231 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-24T08:26:59.231 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-24T08:26:59.231 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-24T08:26:59.231 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-24T08:26:59.231 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-24T08:26:59.231 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-24T08:26:59.231 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-24T08:26:59.231 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-24T08:26:59.231 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-24T08:26:59.231 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-04-24T08:26:59.231 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi 2024-04-24T08:26:59.232 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-04-24T08:26:59.232 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-04-24T08:26:59.232 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-04-24T08:26:59.232 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-04-24T08:26:59.232 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-04-24T08:26:59.232 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid 2024-04-24T08:26:59.232 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-24T08:26:59.232 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-04-24T08:26:59.232 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx 2024-04-24T08:26:59.232 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-24T08:26:59.232 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-24T08:26:59.232 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-04-24T08:26:59.232 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr 2024-04-24T08:26:59.233 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-24T08:26:59.233 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-04-24T08:26:59.233 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi 2024-04-24T08:26:59.233 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-24T08:26:59.233 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa 2024-04-24T08:26:59.233 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-04-24T08:26:59.233 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc 2024-04-24T08:26:59.233 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-24T08:26:59.233 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i 2024-04-24T08:26:59.233 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-24T08:26:59.234 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-04-24T08:26:59.234 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor 2024-04-24T08:26:59.234 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-24T08:26:59.234 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi 2024-04-24T08:26:59.234 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i 2024-04-24T08:26:59.234 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-24T08:26:59.234 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i 2024-04-24T08:26:59.234 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-24T08:26:59.234 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-24T08:26:59.234 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-04-24T08:26:59.234 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler 2024-04-24T08:26:59.234 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-04-24T08:26:59.235 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-04-24T08:26:59.235 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-04-24T08:26:59.235 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-04-24T08:26:59.235 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-04-24T08:26:59.235 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r 2024-04-24T08:26:59.235 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-24T08:26:59.235 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-04-24T08:26:59.235 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe 2024-04-24T08:26:59.235 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-24T08:26:59.236 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-24T08:26:59.236 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic 2024-04-24T08:26:59.236 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-04-24T08:26:59.236 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-04-24T08:26:59.236 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-04-24T08:26:59.236 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-04-24T08:26:59.236 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-04-24T08:26:59.236 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-04-24T08:26:59.236 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-04-24T08:26:59.236 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci 2024-04-24T08:26:59.236 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-04-24T08:26:59.237 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-24T08:26:59.237 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-24T08:26:59.237 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc 2024-04-24T08:26:59.237 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-04-24T08:26:59.237 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-04-24T08:26:59.237 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-24T08:26:59.237 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas 2024-04-24T08:26:59.237 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-04-24T08:26:59.237 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc 2024-04-24T08:26:59.237 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-24T08:26:59.237 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid 2024-04-24T08:26:59.237 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-04-24T08:26:59.238 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-24T08:26:59.238 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-24T08:26:59.238 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-24T08:26:59.238 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas 2024-04-24T08:26:59.238 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-24T08:26:59.238 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas 2024-04-24T08:26:59.238 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-24T08:26:59.238 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-04-24T08:26:59.238 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001 2024-04-24T08:26:59.238 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-24T08:26:59.238 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-04-24T08:26:59.238 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-04-24T08:26:59.238 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx 2024-04-24T08:26:59.238 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-24T08:26:59.239 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-24T08:26:59.239 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx 2024-04-24T08:26:59.239 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-24T08:26:59.239 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-04-24T08:26:59.239 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-04-24T08:26:59.239 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-24T08:26:59.239 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-24T08:26:59.239 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-24T08:26:59.239 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-24T08:26:59.239 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-24T08:26:59.239 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-04-24T08:26:59.239 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-04-24T08:26:59.240 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-04-24T08:26:59.240 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2 2024-04-24T08:26:59.240 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-24T08:26:59.240 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-04-24T08:26:59.240 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-24T08:26:59.240 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-04-24T08:26:59.240 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb 2024-04-24T08:26:59.240 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-04-24T08:26:59.240 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target 2024-04-24T08:26:59.240 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi 2024-04-24T08:26:59.241 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-24T08:26:59.241 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback 2024-04-24T08:26:59.241 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-24T08:26:59.241 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-04-24T08:26:59.241 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-04-24T08:26:59.241 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-04-24T08:26:59.241 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-04-24T08:26:59.241 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-04-24T08:26:59.241 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc 2024-04-24T08:26:59.241 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-24T08:26:59.242 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal 2024-04-24T08:26:59.242 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel 2024-04-24T08:26:59.242 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal 2024-04-24T08:26:59.242 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-24T08:26:59.242 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-24T08:26:59.242 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-24T08:26:59.242 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-24T08:26:59.242 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-24T08:26:59.243 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-24T08:26:59.243 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-24T08:26:59.243 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-24T08:26:59.243 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-24T08:26:59.243 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-24T08:26:59.243 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-24T08:26:59.243 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-24T08:26:59.243 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-24T08:26:59.243 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-24T08:26:59.243 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-24T08:26:59.243 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-24T08:26:59.244 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-24T08:26:59.244 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-24T08:26:59.244 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty 2024-04-24T08:26:59.244 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless 2024-04-24T08:26:59.244 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-24T08:26:59.244 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-04-24T08:26:59.244 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-04-24T08:26:59.244 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-04-24T08:26:59.244 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial 2024-04-24T08:26:59.244 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250 2024-04-24T08:26:59.245 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-24T08:26:59.245 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-04-24T08:26:59.245 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm 2024-04-24T08:26:59.245 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-24T08:26:59.245 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs 2024-04-24T08:26:59.245 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core 2024-04-24T08:26:59.245 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-24T08:26:59.245 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host 2024-04-24T08:26:59.245 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-24T08:26:59.245 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio 2024-04-24T08:26:59.245 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-04-24T08:26:59.245 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-04-24T08:26:59.245 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-04-24T08:26:59.246 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-04-24T08:26:59.246 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-04-24T08:26:59.246 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb 2024-04-24T08:26:59.246 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class 2024-04-24T08:26:59.246 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-04-24T08:26:59.246 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-24T08:26:59.246 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-04-24T08:26:59.246 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-04-24T08:26:59.246 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host 2024-04-24T08:26:59.246 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-24T08:26:59.247 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image 2024-04-24T08:26:59.247 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-04-24T08:26:59.247 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-04-24T08:26:59.247 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc 2024-04-24T08:26:59.247 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-04-24T08:26:59.247 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-04-24T08:26:59.247 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-04-24T08:26:59.247 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-04-24T08:26:59.247 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-04-24T08:26:59.247 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-04-24T08:26:59.247 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-04-24T08:26:59.247 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-24T08:26:59.247 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-04-24T08:26:59.248 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-04-24T08:26:59.248 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga 2024-04-24T08:26:59.248 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-24T08:26:59.248 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-24T08:26:59.248 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-04-24T08:26:59.248 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-04-24T08:26:59.248 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-24T08:26:59.248 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-04-24T08:26:59.248 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-04-24T08:26:59.248 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy 2024-04-24T08:26:59.249 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-04-24T08:26:59.249 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial 2024-04-24T08:26:59.249 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-04-24T08:26:59.249 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-04-24T08:26:59.249 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-24T08:26:59.249 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-04-24T08:26:59.249 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-04-24T08:26:59.249 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-04-24T08:26:59.249 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-24T08:26:59.249 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-24T08:26:59.250 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-04-24T08:26:59.250 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-24T08:26:59.250 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-24T08:26:59.250 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-24T08:26:59.250 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-04-24T08:26:59.250 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-04-24T08:26:59.250 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-04-24T08:26:59.250 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-04-24T08:26:59.250 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-24T08:26:59.251 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-04-24T08:26:59.251 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-24T08:26:59.251 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-24T08:26:59.251 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-24T08:26:59.251 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-04-24T08:26:59.251 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-04-24T08:26:59.251 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-04-24T08:26:59.251 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-04-24T08:26:59.251 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-04-24T08:26:59.251 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-04-24T08:26:59.252 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-04-24T08:26:59.252 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-04-24T08:26:59.252 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-04-24T08:26:59.252 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-04-24T08:26:59.252 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-04-24T08:26:59.252 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-04-24T08:26:59.252 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-04-24T08:26:59.252 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-04-24T08:26:59.252 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-24T08:26:59.252 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-04-24T08:26:59.252 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-04-24T08:26:59.252 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-24T08:26:59.253 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-24T08:26:59.253 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-04-24T08:26:59.253 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-24T08:26:59.253 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-04-24T08:26:59.253 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-04-24T08:26:59.253 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-24T08:26:59.253 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage 2024-04-24T08:26:59.253 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-04-24T08:26:59.253 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-24T08:26:59.253 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-24T08:26:59.254 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-24T08:26:59.254 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-24T08:26:59.254 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-24T08:26:59.254 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-24T08:26:59.254 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-24T08:26:59.254 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-04-24T08:26:59.254 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-24T08:26:59.254 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-24T08:26:59.254 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-24T08:26:59.254 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-24T08:26:59.254 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-24T08:26:59.254 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-04-24T08:26:59.255 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip 2024-04-24T08:26:59.255 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-24T08:26:59.255 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-24T08:26:59.255 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-24T08:26:59.255 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio 2024-04-24T08:26:59.255 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci 2024-04-24T08:26:59.255 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-24T08:26:59.255 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-24T08:26:59.255 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-04-24T08:26:59.255 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-24T08:26:59.256 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost 2024-04-24T08:26:59.256 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-04-24T08:26:59.256 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-24T08:26:59.256 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-04-24T08:26:59.256 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-04-24T08:26:59.256 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video 2024-04-24T08:26:59.256 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight 2024-04-24T08:26:59.256 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-04-24T08:26:59.256 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-04-24T08:26:59.256 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-24T08:26:59.257 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio 2024-04-24T08:26:59.257 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio.ko 2024-04-24T08:26:59.257 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_balloon.ko 2024-04-24T08:26:59.257 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_mmio.ko 2024-04-24T08:26:59.257 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci.ko 2024-04-24T08:26:59.257 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-04-24T08:26:59.257 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-04-24T08:26:59.257 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_ring.ko 2024-04-24T08:26:59.257 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1 2024-04-24T08:26:59.257 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters 2024-04-24T08:26:59.257 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-04-24T08:26:59.257 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-04-24T08:26:59.257 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves 2024-04-24T08:26:59.257 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-24T08:26:59.258 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-24T08:26:59.258 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-24T08:26:59.258 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-24T08:26:59.258 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-24T08:26:59.258 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-24T08:26:59.258 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-24T08:26:59.258 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-24T08:26:59.258 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-24T08:26:59.258 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-24T08:26:59.258 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-24T08:26:59.258 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-04-24T08:26:59.259 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog 2024-04-24T08:26:59.259 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-24T08:26:59.259 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-24T08:26:59.259 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-24T08:26:59.259 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-04-24T08:26:59.259 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-04-24T08:26:59.259 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-24T08:26:59.259 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-24T08:26:59.259 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-04-24T08:26:59.259 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-04-24T08:26:59.259 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-24T08:26:59.259 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-24T08:26:59.260 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-04-24T08:26:59.260 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-04-24T08:26:59.260 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-04-24T08:26:59.260 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-24T08:26:59.260 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-24T08:26:59.260 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-24T08:26:59.260 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-24T08:26:59.260 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-04-24T08:26:59.260 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-24T08:26:59.260 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-04-24T08:26:59.260 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-24T08:26:59.261 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-24T08:26:59.261 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-24T08:26:59.261 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-04-24T08:26:59.261 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-04-24T08:26:59.261 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen 2024-04-24T08:26:59.261 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-acpi-processor.ko 2024-04-24T08:26:59.261 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-04-24T08:26:59.261 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-04-24T08:26:59.261 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-04-24T08:26:59.261 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback 2024-04-24T08:26:59.261 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-24T08:26:59.261 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-04-24T08:26:59.261 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs 2024-04-24T08:26:59.262 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-24T08:26:59.262 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs 2024-04-24T08:26:59.262 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p 2024-04-24T08:26:59.262 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-04-24T08:26:59.262 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs 2024-04-24T08:26:59.262 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-04-24T08:26:59.262 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs 2024-04-24T08:26:59.262 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-04-24T08:26:59.262 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-04-24T08:26:59.262 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs 2024-04-24T08:26:59.262 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-04-24T08:26:59.262 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles 2024-04-24T08:26:59.262 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-04-24T08:26:59.262 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph 2024-04-24T08:26:59.263 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-04-24T08:26:59.263 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda 2024-04-24T08:26:59.263 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-04-24T08:26:59.263 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs 2024-04-24T08:26:59.263 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-04-24T08:26:59.263 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm 2024-04-24T08:26:59.263 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-04-24T08:26:59.263 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs 2024-04-24T08:26:59.263 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs/ecryptfs.ko 2024-04-24T08:26:59.263 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs 2024-04-24T08:26:59.263 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-04-24T08:26:59.263 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat 2024-04-24T08:26:59.263 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/fat.ko 2024-04-24T08:26:59.264 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-04-24T08:26:59.264 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/vfat.ko 2024-04-24T08:26:59.264 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse 2024-04-24T08:26:59.264 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-04-24T08:26:59.264 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/fuse.ko 2024-04-24T08:26:59.264 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2 2024-04-24T08:26:59.264 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-04-24T08:26:59.264 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs 2024-04-24T08:26:59.264 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-04-24T08:26:59.264 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus 2024-04-24T08:26:59.264 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-04-24T08:26:59.264 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs 2024-04-24T08:26:59.264 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-04-24T08:26:59.265 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs 2024-04-24T08:26:59.265 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-04-24T08:26:59.265 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd 2024-04-24T08:26:59.265 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-04-24T08:26:59.265 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix 2024-04-24T08:26:59.265 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-04-24T08:26:59.265 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs 2024-04-24T08:26:59.265 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-04-24T08:26:59.265 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs 2024-04-24T08:26:59.265 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout 2024-04-24T08:26:59.265 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-24T08:26:59.265 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout 2024-04-24T08:26:59.266 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-24T08:26:59.266 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout 2024-04-24T08:26:59.266 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-24T08:26:59.266 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-04-24T08:26:59.266 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-04-24T08:26:59.266 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-04-24T08:26:59.266 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common 2024-04-24T08:26:59.266 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-04-24T08:26:59.266 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-04-24T08:26:59.266 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd 2024-04-24T08:26:59.266 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-04-24T08:26:59.267 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2 2024-04-24T08:26:59.267 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-04-24T08:26:59.267 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls 2024-04-24T08:26:59.267 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-04-24T08:26:59.267 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-04-24T08:26:59.267 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-04-24T08:26:59.267 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-04-24T08:26:59.267 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-04-24T08:26:59.267 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-04-24T08:26:59.267 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-04-24T08:26:59.267 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-04-24T08:26:59.268 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-04-24T08:26:59.268 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-04-24T08:26:59.268 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-04-24T08:26:59.268 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-04-24T08:26:59.268 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-04-24T08:26:59.268 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-04-24T08:26:59.268 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-04-24T08:26:59.268 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-04-24T08:26:59.268 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-04-24T08:26:59.268 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-04-24T08:26:59.268 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-04-24T08:26:59.268 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-04-24T08:26:59.269 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-04-24T08:26:59.269 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-04-24T08:26:59.269 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-04-24T08:26:59.269 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-04-24T08:26:59.269 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-04-24T08:26:59.269 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-04-24T08:26:59.269 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-04-24T08:26:59.269 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-04-24T08:26:59.269 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-04-24T08:26:59.270 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-04-24T08:26:59.270 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-04-24T08:26:59.270 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-04-24T08:26:59.270 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-04-24T08:26:59.270 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-04-24T08:26:59.270 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-04-24T08:26:59.270 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-04-24T08:26:59.270 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-04-24T08:26:59.270 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-04-24T08:26:59.270 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-04-24T08:26:59.270 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-04-24T08:26:59.270 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-04-24T08:26:59.270 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-04-24T08:26:59.270 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-04-24T08:26:59.271 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-04-24T08:26:59.271 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-04-24T08:26:59.271 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-04-24T08:26:59.271 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-04-24T08:26:59.271 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-04-24T08:26:59.271 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-24T08:26:59.271 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-04-24T08:26:59.271 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2 2024-04-24T08:26:59.271 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster 2024-04-24T08:26:59.271 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-24T08:26:59.271 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm 2024-04-24T08:26:59.271 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-24T08:26:59.271 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs 2024-04-24T08:26:59.272 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-24T08:26:59.272 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-04-24T08:26:59.272 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-24T08:26:59.272 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-24T08:26:59.272 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-24T08:26:59.272 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs 2024-04-24T08:26:59.272 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-04-24T08:26:59.272 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore 2024-04-24T08:26:59.272 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-04-24T08:26:59.273 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs 2024-04-24T08:26:59.273 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-04-24T08:26:59.273 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs 2024-04-24T08:26:59.273 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-04-24T08:26:59.273 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb 2024-04-24T08:26:59.273 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client 2024-04-24T08:26:59.273 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-04-24T08:26:59.273 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common 2024-04-24T08:26:59.273 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-04-24T08:26:59.273 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-04-24T08:26:59.273 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs 2024-04-24T08:26:59.274 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-04-24T08:26:59.274 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv 2024-04-24T08:26:59.274 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-04-24T08:26:59.274 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf 2024-04-24T08:26:59.274 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-04-24T08:26:59.274 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs 2024-04-24T08:26:59.274 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-04-24T08:26:59.274 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs 2024-04-24T08:26:59.274 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-04-24T08:26:59.274 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel 2024-04-24T08:26:59.274 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu 2024-04-24T08:26:59.274 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu/rcutorture.ko 2024-04-24T08:26:59.274 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-04-24T08:26:59.275 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace 2024-04-24T08:26:59.275 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-04-24T08:26:59.275 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib 2024-04-24T08:26:59.275 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/asn1_encoder.ko 2024-04-24T08:26:59.275 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-04-24T08:26:59.275 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-04-24T08:26:59.275 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto 2024-04-24T08:26:59.275 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-04-24T08:26:59.275 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-04-24T08:26:59.275 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-04-24T08:26:59.275 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-04-24T08:26:59.275 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4 2024-04-24T08:26:59.275 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-04-24T08:26:59.276 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-04-24T08:26:59.276 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math 2024-04-24T08:26:59.276 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-04-24T08:26:59.276 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6 2024-04-24T08:26:59.276 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-04-24T08:26:59.276 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon 2024-04-24T08:26:59.276 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-24T08:26:59.276 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-04-24T08:26:59.276 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-04-24T08:26:59.276 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-04-24T08:26:59.276 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd 2024-04-24T08:26:59.276 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd/zstd_compress.ko 2024-04-24T08:26:59.276 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm 2024-04-24T08:26:59.277 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-04-24T08:26:59.277 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net 2024-04-24T08:26:59.277 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan 2024-04-24T08:26:59.277 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-04-24T08:26:59.277 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-04-24T08:26:59.277 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-04-24T08:26:59.277 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-04-24T08:26:59.277 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-24T08:26:59.277 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-04-24T08:26:59.277 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-04-24T08:26:59.278 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-04-24T08:26:59.278 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802 2024-04-24T08:26:59.278 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-04-24T08:26:59.278 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-04-24T08:26:59.278 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-04-24T08:26:59.278 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-04-24T08:26:59.278 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-04-24T08:26:59.279 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q 2024-04-24T08:26:59.279 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-04-24T08:26:59.279 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p 2024-04-24T08:26:59.279 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-04-24T08:26:59.279 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-04-24T08:26:59.279 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-04-24T08:26:59.279 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-04-24T08:26:59.279 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv 2024-04-24T08:26:59.279 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-04-24T08:26:59.279 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge 2024-04-24T08:26:59.279 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-04-24T08:26:59.279 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-04-24T08:26:59.280 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter 2024-04-24T08:26:59.280 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-24T08:26:59.280 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-24T08:26:59.280 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-24T08:26:59.280 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-24T08:26:59.280 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-24T08:26:59.280 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-24T08:26:59.280 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-24T08:26:59.280 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-24T08:26:59.280 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-24T08:26:59.280 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-24T08:26:59.280 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-24T08:26:59.280 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-24T08:26:59.281 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-24T08:26:59.281 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-24T08:26:59.281 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-24T08:26:59.281 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-24T08:26:59.281 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-24T08:26:59.281 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-24T08:26:59.281 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-24T08:26:59.281 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-24T08:26:59.281 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-04-24T08:26:59.281 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-04-24T08:26:59.282 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-24T08:26:59.282 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-24T08:26:59.282 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph 2024-04-24T08:26:59.282 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-04-24T08:26:59.282 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core 2024-04-24T08:26:59.282 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/failover.ko 2024-04-24T08:26:59.282 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-04-24T08:26:59.282 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp 2024-04-24T08:26:59.282 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-04-24T08:26:59.282 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-04-24T08:26:59.282 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-04-24T08:26:59.282 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-04-24T08:26:59.283 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver 2024-04-24T08:26:59.283 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver/dns_resolver.ko 2024-04-24T08:26:59.283 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa 2024-04-24T08:26:59.283 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-04-24T08:26:59.283 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-04-24T08:26:59.283 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-04-24T08:26:59.283 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-04-24T08:26:59.283 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-04-24T08:26:59.283 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154 2024-04-24T08:26:59.283 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan 2024-04-24T08:26:59.283 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-24T08:26:59.283 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-04-24T08:26:59.283 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-24T08:26:59.283 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4 2024-04-24T08:26:59.284 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-04-24T08:26:59.284 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-04-24T08:26:59.284 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-04-24T08:26:59.284 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-04-24T08:26:59.284 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-04-24T08:26:59.284 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-04-24T08:26:59.284 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-04-24T08:26:59.284 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-04-24T08:26:59.285 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-04-24T08:26:59.285 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-04-24T08:26:59.285 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter 2024-04-24T08:26:59.285 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-24T08:26:59.285 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-24T08:26:59.285 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-24T08:26:59.285 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-24T08:26:59.285 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-24T08:26:59.285 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-24T08:26:59.285 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-24T08:26:59.286 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-24T08:26:59.286 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-24T08:26:59.286 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-24T08:26:59.286 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-24T08:26:59.286 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-24T08:26:59.286 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-24T08:26:59.286 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-24T08:26:59.286 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-24T08:26:59.286 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-24T08:26:59.286 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-24T08:26:59.286 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-24T08:26:59.287 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-24T08:26:59.287 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-24T08:26:59.287 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-24T08:26:59.287 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-04-24T08:26:59.287 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-04-24T08:26:59.287 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-04-24T08:26:59.287 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-04-24T08:26:59.287 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-04-24T08:26:59.287 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-04-24T08:26:59.287 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-04-24T08:26:59.287 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-04-24T08:26:59.287 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-04-24T08:26:59.287 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-04-24T08:26:59.287 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-04-24T08:26:59.288 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-04-24T08:26:59.288 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-04-24T08:26:59.288 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-04-24T08:26:59.288 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-04-24T08:26:59.288 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-04-24T08:26:59.288 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-24T08:26:59.288 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6 2024-04-24T08:26:59.288 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-04-24T08:26:59.288 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-04-24T08:26:59.288 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-04-24T08:26:59.288 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-04-24T08:26:59.288 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-24T08:26:59.288 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-04-24T08:26:59.289 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-04-24T08:26:59.289 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter 2024-04-24T08:26:59.289 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-24T08:26:59.289 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-24T08:26:59.289 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-24T08:26:59.289 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-24T08:26:59.289 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-24T08:26:59.289 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-24T08:26:59.289 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-24T08:26:59.289 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-24T08:26:59.289 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-24T08:26:59.289 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-24T08:26:59.289 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-24T08:26:59.289 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-24T08:26:59.289 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-24T08:26:59.290 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-24T08:26:59.290 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-24T08:26:59.290 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-24T08:26:59.290 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-24T08:26:59.290 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-24T08:26:59.290 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-24T08:26:59.290 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-24T08:26:59.290 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-24T08:26:59.290 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-24T08:26:59.290 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-24T08:26:59.290 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-24T08:26:59.290 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-04-24T08:26:59.290 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-04-24T08:26:59.290 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-24T08:26:59.290 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key 2024-04-24T08:26:59.291 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-04-24T08:26:59.291 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp 2024-04-24T08:26:59.291 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-04-24T08:26:59.291 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-24T08:26:59.291 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-04-24T08:26:59.291 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-04-24T08:26:59.291 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-04-24T08:26:59.291 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-04-24T08:26:59.291 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-04-24T08:26:59.291 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc 2024-04-24T08:26:59.292 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-04-24T08:26:59.292 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154 2024-04-24T08:26:59.292 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-04-24T08:26:59.292 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls 2024-04-24T08:26:59.292 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-04-24T08:26:59.292 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter 2024-04-24T08:26:59.292 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset 2024-04-24T08:26:59.292 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-04-24T08:26:59.292 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-24T08:26:59.292 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-24T08:26:59.292 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-24T08:26:59.293 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-24T08:26:59.293 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-24T08:26:59.293 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-24T08:26:59.293 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-24T08:26:59.293 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-24T08:26:59.293 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-24T08:26:59.293 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-24T08:26:59.293 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-24T08:26:59.293 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-24T08:26:59.293 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-24T08:26:59.293 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-24T08:26:59.293 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-24T08:26:59.293 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs 2024-04-24T08:26:59.294 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-24T08:26:59.294 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-24T08:26:59.294 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-24T08:26:59.294 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-24T08:26:59.294 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-24T08:26:59.294 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-24T08:26:59.294 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-24T08:26:59.294 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-24T08:26:59.294 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-24T08:26:59.294 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-24T08:26:59.294 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-24T08:26:59.294 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-24T08:26:59.295 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-24T08:26:59.295 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-24T08:26:59.295 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-04-24T08:26:59.295 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-04-24T08:26:59.295 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-24T08:26:59.295 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-24T08:26:59.295 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-24T08:26:59.295 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-24T08:26:59.295 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-24T08:26:59.295 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-24T08:26:59.295 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-24T08:26:59.295 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-24T08:26:59.296 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-24T08:26:59.296 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-24T08:26:59.296 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-24T08:26:59.296 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-24T08:26:59.296 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-24T08:26:59.296 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-04-24T08:26:59.296 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-24T08:26:59.296 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-04-24T08:26:59.296 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-04-24T08:26:59.296 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-24T08:26:59.296 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-24T08:26:59.296 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-04-24T08:26:59.297 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-04-24T08:26:59.297 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-24T08:26:59.297 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-24T08:26:59.297 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-04-24T08:26:59.297 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-04-24T08:26:59.297 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-24T08:26:59.297 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-04-24T08:26:59.297 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-24T08:26:59.297 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-24T08:26:59.297 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-04-24T08:26:59.297 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-04-24T08:26:59.297 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-04-24T08:26:59.297 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-24T08:26:59.297 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-04-24T08:26:59.298 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-24T08:26:59.298 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-04-24T08:26:59.298 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-04-24T08:26:59.298 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-04-24T08:26:59.298 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-04-24T08:26:59.298 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-04-24T08:26:59.298 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-04-24T08:26:59.298 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-04-24T08:26:59.298 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_quota.ko 2024-04-24T08:26:59.298 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-04-24T08:26:59.298 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-04-24T08:26:59.298 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-04-24T08:26:59.298 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-04-24T08:26:59.299 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tunnel.ko 2024-04-24T08:26:59.299 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-04-24T08:26:59.299 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-24T08:26:59.299 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-24T08:26:59.299 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-24T08:26:59.299 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-04-24T08:26:59.299 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-04-24T08:26:59.299 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-04-24T08:26:59.299 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-04-24T08:26:59.299 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-24T08:26:59.299 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-04-24T08:26:59.299 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-04-24T08:26:59.299 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-24T08:26:59.299 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-04-24T08:26:59.299 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-04-24T08:26:59.300 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-24T08:26:59.300 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-04-24T08:26:59.300 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-24T08:26:59.300 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-04-24T08:26:59.300 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-24T08:26:59.300 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-24T08:26:59.300 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-04-24T08:26:59.300 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-04-24T08:26:59.300 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-04-24T08:26:59.300 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-04-24T08:26:59.300 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-04-24T08:26:59.300 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-04-24T08:26:59.300 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-04-24T08:26:59.300 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-04-24T08:26:59.301 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-04-24T08:26:59.301 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-04-24T08:26:59.301 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-04-24T08:26:59.301 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-04-24T08:26:59.301 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-04-24T08:26:59.301 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-04-24T08:26:59.301 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-04-24T08:26:59.301 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-04-24T08:26:59.301 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-04-24T08:26:59.301 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-04-24T08:26:59.301 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-04-24T08:26:59.301 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-04-24T08:26:59.301 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-04-24T08:26:59.301 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-04-24T08:26:59.302 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-04-24T08:26:59.302 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-04-24T08:26:59.302 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-04-24T08:26:59.302 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-04-24T08:26:59.302 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-04-24T08:26:59.302 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-04-24T08:26:59.302 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-04-24T08:26:59.302 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-04-24T08:26:59.302 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-04-24T08:26:59.302 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-04-24T08:26:59.302 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-04-24T08:26:59.302 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-04-24T08:26:59.303 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-04-24T08:26:59.303 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-04-24T08:26:59.303 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-04-24T08:26:59.303 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-04-24T08:26:59.303 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-04-24T08:26:59.303 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-04-24T08:26:59.303 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-04-24T08:26:59.303 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-04-24T08:26:59.303 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-04-24T08:26:59.303 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-04-24T08:26:59.303 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-04-24T08:26:59.303 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-04-24T08:26:59.304 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-04-24T08:26:59.304 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-04-24T08:26:59.304 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-04-24T08:26:59.304 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-04-24T08:26:59.304 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-04-24T08:26:59.304 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink 2024-04-24T08:26:59.304 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-04-24T08:26:59.304 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh 2024-04-24T08:26:59.304 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-04-24T08:26:59.304 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch 2024-04-24T08:26:59.304 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-04-24T08:26:59.304 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-04-24T08:26:59.304 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-04-24T08:26:59.305 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-04-24T08:26:59.305 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet 2024-04-24T08:26:59.305 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-04-24T08:26:59.305 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds 2024-04-24T08:26:59.305 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-04-24T08:26:59.305 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-04-24T08:26:59.305 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-04-24T08:26:59.305 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched 2024-04-24T08:26:59.305 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-04-24T08:26:59.305 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-04-24T08:26:59.305 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-04-24T08:26:59.306 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-04-24T08:26:59.306 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-04-24T08:26:59.306 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-04-24T08:26:59.306 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-04-24T08:26:59.306 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-04-24T08:26:59.306 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-04-24T08:26:59.306 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-04-24T08:26:59.306 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-04-24T08:26:59.306 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-04-24T08:26:59.306 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-04-24T08:26:59.306 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-04-24T08:26:59.306 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-04-24T08:26:59.306 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-04-24T08:26:59.307 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-04-24T08:26:59.307 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-04-24T08:26:59.307 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-04-24T08:26:59.307 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-04-24T08:26:59.307 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-04-24T08:26:59.307 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-04-24T08:26:59.307 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-04-24T08:26:59.307 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-04-24T08:26:59.307 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-04-24T08:26:59.307 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-04-24T08:26:59.307 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-04-24T08:26:59.308 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-04-24T08:26:59.308 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-04-24T08:26:59.308 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-04-24T08:26:59.308 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-04-24T08:26:59.308 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-04-24T08:26:59.308 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-04-24T08:26:59.308 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-04-24T08:26:59.308 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-04-24T08:26:59.308 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-04-24T08:26:59.309 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-04-24T08:26:59.309 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-04-24T08:26:59.309 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-04-24T08:26:59.309 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-04-24T08:26:59.309 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-04-24T08:26:59.309 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-04-24T08:26:59.309 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-04-24T08:26:59.309 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp 2024-04-24T08:26:59.309 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-04-24T08:26:59.309 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-04-24T08:26:59.309 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc 2024-04-24T08:26:59.309 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss 2024-04-24T08:26:59.310 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-24T08:26:59.310 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-24T08:26:59.310 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-04-24T08:26:59.310 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma 2024-04-24T08:26:59.310 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-24T08:26:59.310 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc 2024-04-24T08:26:59.310 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-04-24T08:26:59.310 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-04-24T08:26:59.310 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix 2024-04-24T08:26:59.310 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-04-24T08:26:59.310 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock 2024-04-24T08:26:59.310 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-24T08:26:59.311 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-24T08:26:59.311 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-04-24T08:26:59.311 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-24T08:26:59.311 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-24T08:26:59.311 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm 2024-04-24T08:26:59.311 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-24T08:26:59.311 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security 2024-04-24T08:26:59.311 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys 2024-04-24T08:26:59.311 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys 2024-04-24T08:26:59.311 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-04-24T08:26:59.311 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys 2024-04-24T08:26:59.311 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys/trusted.ko 2024-04-24T08:26:59.311 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-04-24T08:26:59.311 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-04-24T08:26:59.312 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-04-24T08:26:59.312 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-04-24T08:26:59.312 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-04-24T08:26:59.312 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-04-24T08:26:59.312 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-04-24T08:26:59.312 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-04-24T08:26:59.312 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-04-24T08:26:59.312 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-04-24T08:26:59.312 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-04-24T08:26:59.312 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-04-24T08:26:59.312 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-04-24T08:26:59.312 INFO:teuthology.orchestra.run.smithi189.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-24T08:26:59.316 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-04-24T08:26:59.316 DEBUG:teuthology.orchestra.run.smithi189:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-04-24T08:26:59.340 INFO:teuthology.orchestra.run.smithi189.stdout:postinstall scriptlet (using /bin/sh): 2024-04-24T08:26:59.340 INFO:teuthology.orchestra.run.smithi189.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-24T08:26:59.340 INFO:teuthology.orchestra.run.smithi189.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g806ec36b2112 /lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-24T08:26:59.340 INFO:teuthology.orchestra.run.smithi189.stdout:fi 2024-04-24T08:26:59.340 INFO:teuthology.orchestra.run.smithi189.stdout:for file in vmlinuz System.map config; do 2024-04-24T08:26:59.340 INFO:teuthology.orchestra.run.smithi189.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112"; then 2024-04-24T08:26:59.340 INFO:teuthology.orchestra.run.smithi189.stdout: cp "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112" 2024-04-24T08:26:59.340 INFO:teuthology.orchestra.run.smithi189.stdout: fi 2024-04-24T08:26:59.340 INFO:teuthology.orchestra.run.smithi189.stdout:done 2024-04-24T08:26:59.341 INFO:teuthology.orchestra.run.smithi189.stdout:if [ -d "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" ] && \ 2024-04-24T08:26:59.341 INFO:teuthology.orchestra.run.smithi189.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" >/dev/null 2>&1; then 2024-04-24T08:26:59.341 INFO:teuthology.orchestra.run.smithi189.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-24T08:26:59.341 INFO:teuthology.orchestra.run.smithi189.stdout: cp -r "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-24T08:26:59.341 INFO:teuthology.orchestra.run.smithi189.stdout:fi 2024-04-24T08:26:59.341 INFO:teuthology.orchestra.run.smithi189.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep" ]; then 2024-04-24T08:26:59.341 INFO:teuthology.orchestra.run.smithi189.stdout: /usr/sbin/depmod 6.9.0-rc3-g806ec36b2112 2024-04-24T08:26:59.341 INFO:teuthology.orchestra.run.smithi189.stdout:fi 2024-04-24T08:26:59.341 INFO:teuthology.orchestra.run.smithi189.stdout:preuninstall scriptlet (using /bin/sh): 2024-04-24T08:26:59.341 INFO:teuthology.orchestra.run.smithi189.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-24T08:26:59.341 INFO:teuthology.orchestra.run.smithi189.stdout:kernel-install remove 6.9.0-rc3-g806ec36b2112 2024-04-24T08:26:59.341 INFO:teuthology.orchestra.run.smithi189.stdout:fi 2024-04-24T08:26:59.342 DEBUG:teuthology.orchestra.run.smithi189:> sudo rpm -qi grub2-tools 2024-04-24T08:26:59.416 INFO:teuthology.orchestra.run.smithi189.stdout:Name : grub2-tools 2024-04-24T08:26:59.416 INFO:teuthology.orchestra.run.smithi189.stdout:Epoch : 1 2024-04-24T08:26:59.417 INFO:teuthology.orchestra.run.smithi189.stdout:Version : 2.06 2024-04-24T08:26:59.417 INFO:teuthology.orchestra.run.smithi189.stdout:Release : 68.el9 2024-04-24T08:26:59.417 INFO:teuthology.orchestra.run.smithi189.stdout:Architecture: x86_64 2024-04-24T08:26:59.417 INFO:teuthology.orchestra.run.smithi189.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-04-24T08:26:59.417 INFO:teuthology.orchestra.run.smithi189.stdout:Group : Unspecified 2024-04-24T08:26:59.417 INFO:teuthology.orchestra.run.smithi189.stdout:Size : 8264254 2024-04-24T08:26:59.417 INFO:teuthology.orchestra.run.smithi189.stdout:License : GPLv3+ 2024-04-24T08:26:59.417 INFO:teuthology.orchestra.run.smithi189.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-24T08:26:59.417 INFO:teuthology.orchestra.run.smithi189.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-24T08:26:59.417 INFO:teuthology.orchestra.run.smithi189.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-24T08:26:59.417 INFO:teuthology.orchestra.run.smithi189.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-24T08:26:59.417 INFO:teuthology.orchestra.run.smithi189.stdout:Packager : builder@centos.org 2024-04-24T08:26:59.417 INFO:teuthology.orchestra.run.smithi189.stdout:Vendor : CentOS 2024-04-24T08:26:59.417 INFO:teuthology.orchestra.run.smithi189.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-24T08:26:59.417 INFO:teuthology.orchestra.run.smithi189.stdout:Summary : Support tools for GRUB. 2024-04-24T08:26:59.418 INFO:teuthology.orchestra.run.smithi189.stdout:Description : 2024-04-24T08:26:59.418 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-04-24T08:26:59.418 INFO:teuthology.orchestra.run.smithi189.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-24T08:26:59.418 INFO:teuthology.orchestra.run.smithi189.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-24T08:26:59.418 INFO:teuthology.orchestra.run.smithi189.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-24T08:26:59.418 INFO:teuthology.orchestra.run.smithi189.stdout:hardware devices. 2024-04-24T08:26:59.418 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-04-24T08:26:59.418 INFO:teuthology.orchestra.run.smithi189.stdout:This subpackage provides tools for support of all platforms. 2024-04-24T08:26:59.419 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-24T08:26:59.419 INFO:teuthology.task.kernel:Updating grub on smithi189 to boot 6.9.0-rc3-g806ec36b2112 2024-04-24T08:26:59.419 DEBUG:teuthology.orchestra.run.smithi189:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-24T08:26:59.960 INFO:teuthology.orchestra.run.smithi189.stderr:Generating grub configuration file ... 2024-04-24T08:27:01.097 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-04-24T08:27:01.098 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi083.front.sepia.ceph.com' 2024-04-24T08:27:01.098 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi083.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T08:27:01.331 INFO:teuthology.orchestra.run.smithi189.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-24T08:27:01.363 INFO:teuthology.orchestra.run.smithi189.stderr:done 2024-04-24T08:27:01.366 DEBUG:teuthology.orchestra.run.smithi189:> mktemp 2024-04-24T08:27:01.397 INFO:teuthology.orchestra.run.smithi189.stdout:/tmp/tmp.6sbxklDRLs 2024-04-24T08:27:01.398 DEBUG:teuthology.orchestra.run.smithi189:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.6sbxklDRLs 2024-04-24T08:27:01.471 DEBUG:teuthology.orchestra.run.smithi189:> sudo chmod 0666 /tmp/tmp.6sbxklDRLs 2024-04-24T08:27:01.616 DEBUG:teuthology.orchestra.remote:smithi189:/tmp/tmp.6sbxklDRLs is 6KB 2024-04-24T08:27:01.664 DEBUG:teuthology.orchestra.run.smithi189:> rm -fr /tmp/tmp.6sbxklDRLs 2024-04-24T08:27:01.679 DEBUG:teuthology.orchestra.run.smithi189:> sudo /bin/ls /boot/loader/entries || true 2024-04-24T08:27:01.739 INFO:teuthology.orchestra.run.smithi189.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-04-24T08:27:01.740 INFO:teuthology.orchestra.run.smithi189.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-04-24T08:27:01.740 INFO:teuthology.orchestra.run.smithi189.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-04-24T08:27:01.740 INFO:teuthology.orchestra.run.smithi189.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112.conf 2024-04-24T08:27:01.741 DEBUG:teuthology.orchestra.run.smithi189:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112 2024-04-24T08:27:01.866 DEBUG:teuthology.orchestra.run.smithi189:> sudo shutdown -r now 2024-04-24T08:27:14.961 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi138.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-04-24T08:27:14.962 DEBUG:teuthology.orchestra.run.smithi138:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-04-24T08:27:15.051 INFO:teuthology.orchestra.run.smithi138.stdout:/boot/System.map-6.9.0-rc3-g806ec36b2112 2024-04-24T08:27:15.052 INFO:teuthology.orchestra.run.smithi138.stdout:/boot/config-6.9.0-rc3-g806ec36b2112 2024-04-24T08:27:15.052 INFO:teuthology.orchestra.run.smithi138.stdout:/boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-24T08:27:15.052 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/System.map 2024-04-24T08:27:15.052 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/config 2024-04-24T08:27:15.052 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel 2024-04-24T08:27:15.052 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch 2024-04-24T08:27:15.052 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86 2024-04-24T08:27:15.052 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto 2024-04-24T08:27:15.052 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-24T08:27:15.052 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-24T08:27:15.053 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-24T08:27:15.053 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-24T08:27:15.053 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-24T08:27:15.053 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-24T08:27:15.053 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-24T08:27:15.053 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32c-intel.ko 2024-04-24T08:27:15.053 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-24T08:27:15.053 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-24T08:27:15.053 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-24T08:27:15.053 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-24T08:27:15.053 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-24T08:27:15.054 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-24T08:27:15.054 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-24T08:27:15.054 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-24T08:27:15.054 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-24T08:27:15.054 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-24T08:27:15.054 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-24T08:27:15.054 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-24T08:27:15.054 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel 2024-04-24T08:27:15.054 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu 2024-04-24T08:27:15.055 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce 2024-04-24T08:27:15.055 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-24T08:27:15.055 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm 2024-04-24T08:27:15.055 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-24T08:27:15.055 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-24T08:27:15.055 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-04-24T08:27:15.055 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto 2024-04-24T08:27:15.055 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-04-24T08:27:15.055 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-04-24T08:27:15.055 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-04-24T08:27:15.055 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx 2024-04-24T08:27:15.056 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-04-24T08:27:15.056 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-04-24T08:27:15.056 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-24T08:27:15.056 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-04-24T08:27:15.056 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-04-24T08:27:15.056 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-04-24T08:27:15.056 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-04-24T08:27:15.056 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-04-24T08:27:15.056 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-04-24T08:27:15.056 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-04-24T08:27:15.056 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-04-24T08:27:15.056 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-04-24T08:27:15.057 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-04-24T08:27:15.057 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-04-24T08:27:15.057 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-04-24T08:27:15.057 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-04-24T08:27:15.057 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-04-24T08:27:15.057 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-04-24T08:27:15.057 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-04-24T08:27:15.057 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-04-24T08:27:15.057 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-04-24T08:27:15.057 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-04-24T08:27:15.057 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-04-24T08:27:15.057 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-04-24T08:27:15.058 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-04-24T08:27:15.058 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-04-24T08:27:15.058 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-04-24T08:27:15.058 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-04-24T08:27:15.058 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-04-24T08:27:15.058 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-04-24T08:27:15.058 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-04-24T08:27:15.058 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-04-24T08:27:15.058 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-04-24T08:27:15.058 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-04-24T08:27:15.058 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-04-24T08:27:15.059 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-04-24T08:27:15.059 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-04-24T08:27:15.059 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-04-24T08:27:15.059 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-04-24T08:27:15.059 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-04-24T08:27:15.059 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-04-24T08:27:15.059 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-04-24T08:27:15.060 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-04-24T08:27:15.060 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-04-24T08:27:15.060 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-04-24T08:27:15.060 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers 2024-04-24T08:27:15.060 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi 2024-04-24T08:27:15.060 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-24T08:27:15.060 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-04-24T08:27:15.060 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-04-24T08:27:15.061 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-04-24T08:27:15.061 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-04-24T08:27:15.061 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-04-24T08:27:15.061 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-04-24T08:27:15.061 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata 2024-04-24T08:27:15.061 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-04-24T08:27:15.061 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-04-24T08:27:15.061 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ata_generic.ko 2024-04-24T08:27:15.061 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-04-24T08:27:15.061 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-04-24T08:27:15.061 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-04-24T08:27:15.061 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-04-24T08:27:15.061 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-04-24T08:27:15.062 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-04-24T08:27:15.062 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-04-24T08:27:15.062 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-04-24T08:27:15.062 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-04-24T08:27:15.062 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-04-24T08:27:15.062 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-04-24T08:27:15.062 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-04-24T08:27:15.062 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-04-24T08:27:15.062 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-24T08:27:15.062 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-24T08:27:15.062 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-04-24T08:27:15.062 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-04-24T08:27:15.062 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-04-24T08:27:15.062 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-04-24T08:27:15.063 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-04-24T08:27:15.063 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-04-24T08:27:15.063 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-04-24T08:27:15.063 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-04-24T08:27:15.063 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-04-24T08:27:15.063 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-04-24T08:27:15.063 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-04-24T08:27:15.063 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-04-24T08:27:15.063 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-04-24T08:27:15.063 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-24T08:27:15.063 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-24T08:27:15.063 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-04-24T08:27:15.063 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-04-24T08:27:15.064 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-04-24T08:27:15.064 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-04-24T08:27:15.064 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-04-24T08:27:15.064 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sis.ko 2024-04-24T08:27:15.064 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-04-24T08:27:15.064 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-04-24T08:27:15.064 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-04-24T08:27:15.064 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-04-24T08:27:15.064 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-04-24T08:27:15.064 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-04-24T08:27:15.064 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-04-24T08:27:15.064 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-04-24T08:27:15.064 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-04-24T08:27:15.064 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-04-24T08:27:15.065 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-04-24T08:27:15.065 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-04-24T08:27:15.065 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-04-24T08:27:15.065 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-04-24T08:27:15.065 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-04-24T08:27:15.065 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-04-24T08:27:15.065 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-04-24T08:27:15.065 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay 2024-04-24T08:27:15.065 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-04-24T08:27:15.065 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base 2024-04-24T08:27:15.065 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap 2024-04-24T08:27:15.065 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-i2c.ko 2024-04-24T08:27:15.065 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-mmio.ko 2024-04-24T08:27:15.065 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma 2024-04-24T08:27:15.065 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-04-24T08:27:15.066 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block 2024-04-24T08:27:15.066 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe 2024-04-24T08:27:15.066 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-04-24T08:27:15.066 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/brd.ko 2024-04-24T08:27:15.066 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd 2024-04-24T08:27:15.066 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-04-24T08:27:15.066 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-04-24T08:27:15.066 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/loop.ko 2024-04-24T08:27:15.066 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx 2024-04-24T08:27:15.066 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-24T08:27:15.066 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-04-24T08:27:15.066 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk 2024-04-24T08:27:15.066 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-04-24T08:27:15.067 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-04-24T08:27:15.067 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-04-24T08:27:15.067 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/virtio_blk.ko 2024-04-24T08:27:15.067 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback 2024-04-24T08:27:15.067 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-24T08:27:15.067 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkfront.ko 2024-04-24T08:27:15.067 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram 2024-04-24T08:27:15.067 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-04-24T08:27:15.067 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char 2024-04-24T08:27:15.067 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-04-24T08:27:15.067 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random 2024-04-24T08:27:15.067 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-24T08:27:15.067 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-24T08:27:15.067 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-24T08:27:15.068 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-04-24T08:27:15.068 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-24T08:27:15.068 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi 2024-04-24T08:27:15.068 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-24T08:27:15.068 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-24T08:27:15.068 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-24T08:27:15.068 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-24T08:27:15.068 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-24T08:27:15.068 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-24T08:27:15.068 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-04-24T08:27:15.068 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave 2024-04-24T08:27:15.068 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-04-24T08:27:15.068 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-04-24T08:27:15.068 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-04-24T08:27:15.069 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm 2024-04-24T08:27:15.069 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm.ko 2024-04-24T08:27:15.069 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-24T08:27:15.069 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-24T08:27:15.069 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-24T08:27:15.069 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis.ko 2024-04-24T08:27:15.069 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-04-24T08:27:15.069 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/uv_mmtimer.ko 2024-04-24T08:27:15.069 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/virtio_console.ko 2024-04-24T08:27:15.069 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq 2024-04-24T08:27:15.069 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-04-24T08:27:15.069 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-24T08:27:15.069 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-24T08:27:15.069 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-04-24T08:27:15.070 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/powernow-k8.ko 2024-04-24T08:27:15.070 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-24T08:27:15.070 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto 2024-04-24T08:27:15.070 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp 2024-04-24T08:27:15.070 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-24T08:27:15.070 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-04-24T08:27:15.070 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel 2024-04-24T08:27:15.070 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat 2024-04-24T08:27:15.070 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common 2024-04-24T08:27:15.070 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-24T08:27:15.070 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-04-24T08:27:15.070 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-24T08:27:15.071 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-04-24T08:27:15.071 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-04-24T08:27:15.071 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio 2024-04-24T08:27:15.071 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-24T08:27:15.071 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca 2024-04-24T08:27:15.071 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-04-24T08:27:15.071 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq 2024-04-24T08:27:15.071 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-04-24T08:27:15.071 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma 2024-04-24T08:27:15.071 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw 2024-04-24T08:27:15.071 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-24T08:27:15.071 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat 2024-04-24T08:27:15.071 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-24T08:27:15.071 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac 2024-04-24T08:27:15.072 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-04-24T08:27:15.072 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-04-24T08:27:15.072 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-04-24T08:27:15.072 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-04-24T08:27:15.072 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-04-24T08:27:15.072 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-04-24T08:27:15.072 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-04-24T08:27:15.072 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-04-24T08:27:15.072 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-04-24T08:27:15.072 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-04-24T08:27:15.072 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-04-24T08:27:15.072 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-04-24T08:27:15.072 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-04-24T08:27:15.072 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware 2024-04-24T08:27:15.073 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/edd.ko 2024-04-24T08:27:15.073 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-24T08:27:15.073 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio 2024-04-24T08:27:15.073 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-04-24T08:27:15.073 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-24T08:27:15.073 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid 2024-04-24T08:27:15.073 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-04-24T08:27:15.073 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-04-24T08:27:15.073 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-04-24T08:27:15.073 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-04-24T08:27:15.073 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-04-24T08:27:15.073 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-04-24T08:27:15.073 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-04-24T08:27:15.074 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-04-24T08:27:15.074 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-04-24T08:27:15.074 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-04-24T08:27:15.074 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-24T08:27:15.074 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-24T08:27:15.074 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-04-24T08:27:15.074 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-04-24T08:27:15.074 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-04-24T08:27:15.074 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-04-24T08:27:15.074 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-04-24T08:27:15.074 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-04-24T08:27:15.074 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-04-24T08:27:15.074 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-04-24T08:27:15.074 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-24T08:27:15.075 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-24T08:27:15.075 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-04-24T08:27:15.075 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-04-24T08:27:15.075 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-04-24T08:27:15.075 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-04-24T08:27:15.075 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-04-24T08:27:15.075 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-04-24T08:27:15.075 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-04-24T08:27:15.075 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-04-24T08:27:15.075 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-04-24T08:27:15.075 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-24T08:27:15.075 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-04-24T08:27:15.075 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-24T08:27:15.075 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-24T08:27:15.076 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-24T08:27:15.076 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-24T08:27:15.076 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-24T08:27:15.076 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-24T08:27:15.076 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-24T08:27:15.076 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-24T08:27:15.076 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-24T08:27:15.076 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-04-24T08:27:15.076 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-04-24T08:27:15.076 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-04-24T08:27:15.076 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-24T08:27:15.076 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-04-24T08:27:15.076 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-04-24T08:27:15.076 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-04-24T08:27:15.077 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-04-24T08:27:15.077 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-04-24T08:27:15.077 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-24T08:27:15.077 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-04-24T08:27:15.077 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-04-24T08:27:15.077 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-04-24T08:27:15.077 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-04-24T08:27:15.077 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-04-24T08:27:15.077 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-04-24T08:27:15.077 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-04-24T08:27:15.077 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-04-24T08:27:15.077 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-04-24T08:27:15.077 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-04-24T08:27:15.077 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-04-24T08:27:15.078 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-04-24T08:27:15.078 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv 2024-04-24T08:27:15.078 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-04-24T08:27:15.078 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-04-24T08:27:15.078 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-04-24T08:27:15.078 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon 2024-04-24T08:27:15.078 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-04-24T08:27:15.078 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-04-24T08:27:15.078 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-24T08:27:15.078 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-04-24T08:27:15.078 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-04-24T08:27:15.078 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-04-24T08:27:15.078 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-04-24T08:27:15.079 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-04-24T08:27:15.079 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-04-24T08:27:15.079 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-04-24T08:27:15.079 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-04-24T08:27:15.079 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-04-24T08:27:15.079 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-04-24T08:27:15.079 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-04-24T08:27:15.079 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-04-24T08:27:15.079 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-04-24T08:27:15.079 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-04-24T08:27:15.079 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-04-24T08:27:15.079 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-04-24T08:27:15.079 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-04-24T08:27:15.079 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-04-24T08:27:15.080 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-04-24T08:27:15.080 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-24T08:27:15.080 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-04-24T08:27:15.080 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-04-24T08:27:15.080 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-24T08:27:15.080 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-04-24T08:27:15.080 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-04-24T08:27:15.080 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-04-24T08:27:15.080 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-04-24T08:27:15.080 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-04-24T08:27:15.080 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-04-24T08:27:15.080 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-04-24T08:27:15.080 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-04-24T08:27:15.080 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-04-24T08:27:15.081 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-04-24T08:27:15.081 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-04-24T08:27:15.081 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-04-24T08:27:15.081 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-04-24T08:27:15.081 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-04-24T08:27:15.081 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-24T08:27:15.081 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-04-24T08:27:15.081 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-04-24T08:27:15.081 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-04-24T08:27:15.081 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-04-24T08:27:15.081 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-04-24T08:27:15.081 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-04-24T08:27:15.081 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-04-24T08:27:15.082 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-04-24T08:27:15.082 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-04-24T08:27:15.082 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-04-24T08:27:15.082 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-04-24T08:27:15.082 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-04-24T08:27:15.082 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-04-24T08:27:15.082 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-04-24T08:27:15.082 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-04-24T08:27:15.082 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-04-24T08:27:15.082 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-04-24T08:27:15.082 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-04-24T08:27:15.082 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-04-24T08:27:15.082 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-04-24T08:27:15.083 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-04-24T08:27:15.083 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-04-24T08:27:15.083 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-04-24T08:27:15.083 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-04-24T08:27:15.083 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-04-24T08:27:15.083 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-04-24T08:27:15.083 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-04-24T08:27:15.083 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-04-24T08:27:15.083 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-04-24T08:27:15.083 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-04-24T08:27:15.083 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-04-24T08:27:15.083 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-04-24T08:27:15.083 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-04-24T08:27:15.083 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-04-24T08:27:15.083 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-04-24T08:27:15.084 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-04-24T08:27:15.084 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-04-24T08:27:15.084 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-04-24T08:27:15.084 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-04-24T08:27:15.084 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-04-24T08:27:15.084 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-04-24T08:27:15.084 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-04-24T08:27:15.084 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-04-24T08:27:15.084 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-04-24T08:27:15.084 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-04-24T08:27:15.084 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-04-24T08:27:15.084 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-04-24T08:27:15.084 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus 2024-04-24T08:27:15.085 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-24T08:27:15.085 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-24T08:27:15.085 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-24T08:27:15.085 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-24T08:27:15.085 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-24T08:27:15.085 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-24T08:27:15.085 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-24T08:27:15.085 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-24T08:27:15.085 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-24T08:27:15.085 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-24T08:27:15.085 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-24T08:27:15.085 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-24T08:27:15.085 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-04-24T08:27:15.085 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-04-24T08:27:15.086 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-04-24T08:27:15.086 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-24T08:27:15.086 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-04-24T08:27:15.086 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-04-24T08:27:15.086 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-04-24T08:27:15.086 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-04-24T08:27:15.086 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-04-24T08:27:15.086 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-04-24T08:27:15.086 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-04-24T08:27:15.086 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-04-24T08:27:15.086 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-04-24T08:27:15.086 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-04-24T08:27:15.086 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-04-24T08:27:15.087 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-04-24T08:27:15.087 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-04-24T08:27:15.087 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-04-24T08:27:15.087 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-04-24T08:27:15.087 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-04-24T08:27:15.087 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-04-24T08:27:15.087 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-04-24T08:27:15.087 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-04-24T08:27:15.087 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-04-24T08:27:15.087 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-04-24T08:27:15.087 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-04-24T08:27:15.087 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-04-24T08:27:15.087 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-04-24T08:27:15.087 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-04-24T08:27:15.087 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c 2024-04-24T08:27:15.088 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos 2024-04-24T08:27:15.088 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-24T08:27:15.088 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-24T08:27:15.088 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses 2024-04-24T08:27:15.088 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-24T08:27:15.088 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-24T08:27:15.088 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-24T08:27:15.088 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-24T08:27:15.088 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-24T08:27:15.088 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-24T08:27:15.088 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-24T08:27:15.088 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-24T08:27:15.088 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-24T08:27:15.089 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-24T08:27:15.089 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-24T08:27:15.089 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-24T08:27:15.089 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-24T08:27:15.089 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-24T08:27:15.089 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-24T08:27:15.089 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-24T08:27:15.089 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-24T08:27:15.089 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-24T08:27:15.089 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-24T08:27:15.089 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-24T08:27:15.089 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-24T08:27:15.089 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-24T08:27:15.090 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-24T08:27:15.090 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-dev.ko 2024-04-24T08:27:15.090 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-04-24T08:27:15.090 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-04-24T08:27:15.090 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-04-24T08:27:15.090 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-04-24T08:27:15.090 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband 2024-04-24T08:27:15.090 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core 2024-04-24T08:27:15.090 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-24T08:27:15.090 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-04-24T08:27:15.090 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-24T08:27:15.090 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-24T08:27:15.090 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-24T08:27:15.091 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-24T08:27:15.091 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-24T08:27:15.091 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw 2024-04-24T08:27:15.091 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4 2024-04-24T08:27:15.091 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-24T08:27:15.091 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4 2024-04-24T08:27:15.091 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-24T08:27:15.091 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5 2024-04-24T08:27:15.091 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-24T08:27:15.091 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca 2024-04-24T08:27:15.091 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-24T08:27:15.091 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp 2024-04-24T08:27:15.091 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib 2024-04-24T08:27:15.091 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-24T08:27:15.092 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser 2024-04-24T08:27:15.092 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-24T08:27:15.092 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert 2024-04-24T08:27:15.092 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-24T08:27:15.092 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp 2024-04-24T08:27:15.092 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-24T08:27:15.092 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt 2024-04-24T08:27:15.092 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-24T08:27:15.092 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input 2024-04-24T08:27:15.092 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport 2024-04-24T08:27:15.092 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-24T08:27:15.092 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-24T08:27:15.093 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-04-24T08:27:15.093 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-04-24T08:27:15.093 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard 2024-04-24T08:27:15.093 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-24T08:27:15.093 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse 2024-04-24T08:27:15.093 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-04-24T08:27:15.093 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-04-24T08:27:15.093 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-04-24T08:27:15.093 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-24T08:27:15.093 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-04-24T08:27:15.093 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-24T08:27:15.093 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-24T08:27:15.093 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-24T08:27:15.093 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4 2024-04-24T08:27:15.093 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-24T08:27:15.094 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio 2024-04-24T08:27:15.094 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-04-24T08:27:15.094 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-04-24T08:27:15.094 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-24T08:27:15.094 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-04-24T08:27:15.094 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-04-24T08:27:15.094 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds 2024-04-24T08:27:15.094 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-04-24T08:27:15.094 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-04-24T08:27:15.094 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-04-24T08:27:15.094 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-04-24T08:27:15.094 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger 2024-04-24T08:27:15.094 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-24T08:27:15.095 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-24T08:27:15.095 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-24T08:27:15.095 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-24T08:27:15.095 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-24T08:27:15.095 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-24T08:27:15.095 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-24T08:27:15.095 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-24T08:27:15.095 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-24T08:27:15.095 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md 2024-04-24T08:27:15.095 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache 2024-04-24T08:27:15.095 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-04-24T08:27:15.095 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-04-24T08:27:15.095 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-04-24T08:27:15.095 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-04-24T08:27:15.096 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-04-24T08:27:15.096 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-04-24T08:27:15.096 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-04-24T08:27:15.096 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-04-24T08:27:15.096 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-04-24T08:27:15.096 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-04-24T08:27:15.096 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-04-24T08:27:15.096 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-04-24T08:27:15.096 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-04-24T08:27:15.096 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-04-24T08:27:15.096 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-04-24T08:27:15.096 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-04-24T08:27:15.096 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data 2024-04-24T08:27:15.097 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-24T08:27:15.097 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-04-24T08:27:15.097 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-04-24T08:27:15.097 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-04-24T08:27:15.097 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-04-24T08:27:15.097 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message 2024-04-24T08:27:15.097 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion 2024-04-24T08:27:15.097 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-04-24T08:27:15.097 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-04-24T08:27:15.097 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-04-24T08:27:15.097 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-04-24T08:27:15.097 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-04-24T08:27:15.097 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-04-24T08:27:15.098 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-04-24T08:27:15.098 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd 2024-04-24T08:27:15.098 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-04-24T08:27:15.098 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-04-24T08:27:15.098 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mfd-core.ko 2024-04-24T08:27:15.098 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-04-24T08:27:15.099 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-04-24T08:27:15.099 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-04-24T08:27:15.099 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-04-24T08:27:15.099 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc 2024-04-24T08:27:15.099 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl 2024-04-24T08:27:15.099 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-24T08:27:15.099 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-04-24T08:27:15.099 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-04-24T08:27:15.099 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-04-24T08:27:15.099 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710 2024-04-24T08:27:15.099 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-04-24T08:27:15.099 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom 2024-04-24T08:27:15.099 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-04-24T08:27:15.099 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-24T08:27:15.100 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-04-24T08:27:15.100 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-04-24T08:27:15.100 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-04-24T08:27:15.100 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-04-24T08:27:15.100 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-04-24T08:27:15.100 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d 2024-04-24T08:27:15.100 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-24T08:27:15.100 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-24T08:27:15.100 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei 2024-04-24T08:27:15.100 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-04-24T08:27:15.100 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-04-24T08:27:15.100 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-04-24T08:27:15.100 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru 2024-04-24T08:27:15.100 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru/gru.ko 2024-04-24T08:27:15.101 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp 2024-04-24T08:27:15.101 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xp.ko 2024-04-24T08:27:15.101 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpc.ko 2024-04-24T08:27:15.101 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-04-24T08:27:15.101 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-04-24T08:27:15.101 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-04-24T08:27:15.101 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-04-24T08:27:15.101 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-04-24T08:27:15.101 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci 2024-04-24T08:27:15.101 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-24T08:27:15.101 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net 2024-04-24T08:27:15.101 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding 2024-04-24T08:27:15.101 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-04-24T08:27:15.101 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa 2024-04-24T08:27:15.101 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53 2024-04-24T08:27:15.102 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-24T08:27:15.102 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-24T08:27:15.102 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-24T08:27:15.102 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx 2024-04-24T08:27:15.102 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-24T08:27:15.102 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-04-24T08:27:15.102 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-04-24T08:27:15.102 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet 2024-04-24T08:27:15.102 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com 2024-04-24T08:27:15.102 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-24T08:27:15.102 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-24T08:27:15.102 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-24T08:27:15.102 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-24T08:27:15.102 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390 2024-04-24T08:27:15.103 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-24T08:27:15.103 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-24T08:27:15.103 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-24T08:27:15.103 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-24T08:27:15.103 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec 2024-04-24T08:27:15.103 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-24T08:27:15.103 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere 2024-04-24T08:27:15.103 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-24T08:27:15.103 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon 2024-04-24T08:27:15.103 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-24T08:27:15.103 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera 2024-04-24T08:27:15.103 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-24T08:27:15.103 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd 2024-04-24T08:27:15.103 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-24T08:27:15.103 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-24T08:27:15.104 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-24T08:27:15.104 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros 2024-04-24T08:27:15.104 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx 2024-04-24T08:27:15.104 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-24T08:27:15.104 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c 2024-04-24T08:27:15.104 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-24T08:27:15.104 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e 2024-04-24T08:27:15.104 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-24T08:27:15.104 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx 2024-04-24T08:27:15.104 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-24T08:27:15.104 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-24T08:27:15.104 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom 2024-04-24T08:27:15.104 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-24T08:27:15.104 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-24T08:27:15.105 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-04-24T08:27:15.105 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-24T08:27:15.105 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-24T08:27:15.105 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet 2024-04-24T08:27:15.105 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-24T08:27:15.105 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-24T08:27:15.105 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade 2024-04-24T08:27:15.105 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna 2024-04-24T08:27:15.105 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-24T08:27:15.105 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio 2024-04-24T08:27:15.105 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb 2024-04-24T08:27:15.105 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-24T08:27:15.105 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-04-24T08:27:15.105 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-24T08:27:15.106 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-04-24T08:27:15.106 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-24T08:27:15.106 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-04-24T08:27:15.106 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-24T08:27:15.106 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-04-24T08:27:15.106 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-24T08:27:15.106 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco 2024-04-24T08:27:15.106 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic 2024-04-24T08:27:15.107 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-24T08:27:15.107 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec 2024-04-24T08:27:15.107 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip 2024-04-24T08:27:15.107 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-24T08:27:15.107 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-24T08:27:15.107 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-24T08:27:15.107 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-24T08:27:15.107 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-24T08:27:15.107 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-24T08:27:15.107 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink 2024-04-24T08:27:15.107 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-24T08:27:15.107 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-24T08:27:15.107 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-04-24T08:27:15.107 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex 2024-04-24T08:27:15.108 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet 2024-04-24T08:27:15.108 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-24T08:27:15.108 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-04-24T08:27:15.108 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-04-24T08:27:15.108 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel 2024-04-24T08:27:15.108 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-24T08:27:15.108 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000 2024-04-24T08:27:15.108 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-24T08:27:15.108 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e 2024-04-24T08:27:15.108 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-24T08:27:15.108 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k 2024-04-24T08:27:15.108 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-24T08:27:15.108 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e 2024-04-24T08:27:15.108 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-24T08:27:15.109 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf 2024-04-24T08:27:15.109 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-24T08:27:15.109 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb 2024-04-24T08:27:15.109 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-24T08:27:15.109 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf 2024-04-24T08:27:15.109 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-24T08:27:15.109 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe 2024-04-24T08:27:15.109 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-24T08:27:15.109 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf 2024-04-24T08:27:15.109 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-24T08:27:15.109 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-04-24T08:27:15.109 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell 2024-04-24T08:27:15.109 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-24T08:27:15.109 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-24T08:27:15.110 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-24T08:27:15.110 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox 2024-04-24T08:27:15.110 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4 2024-04-24T08:27:15.110 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-24T08:27:15.110 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-24T08:27:15.110 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5 2024-04-24T08:27:15.110 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-04-24T08:27:15.110 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-24T08:27:15.110 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel 2024-04-24T08:27:15.110 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-24T08:27:15.110 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom 2024-04-24T08:27:15.110 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge 2024-04-24T08:27:15.110 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-24T08:27:15.111 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi 2024-04-24T08:27:15.111 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-24T08:27:15.111 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-24T08:27:15.111 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion 2024-04-24T08:27:15.111 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-24T08:27:15.111 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia 2024-04-24T08:27:15.111 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-24T08:27:15.111 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines 2024-04-24T08:27:15.111 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-24T08:27:15.111 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-24T08:27:15.111 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic 2024-04-24T08:27:15.111 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen 2024-04-24T08:27:15.111 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-24T08:27:15.111 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-24T08:27:15.111 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-04-24T08:27:15.112 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-24T08:27:15.112 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc 2024-04-24T08:27:15.112 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-24T08:27:15.112 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek 2024-04-24T08:27:15.112 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-24T08:27:15.112 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-24T08:27:15.112 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-24T08:27:15.112 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-24T08:27:15.112 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker 2024-04-24T08:27:15.112 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-04-24T08:27:15.112 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc 2024-04-24T08:27:15.112 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-24T08:27:15.112 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan 2024-04-24T08:27:15.112 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-24T08:27:15.113 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis 2024-04-24T08:27:15.113 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-24T08:27:15.113 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-24T08:27:15.113 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc 2024-04-24T08:27:15.113 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-24T08:27:15.113 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-24T08:27:15.113 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-24T08:27:15.113 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-24T08:27:15.113 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro 2024-04-24T08:27:15.113 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac 2024-04-24T08:27:15.113 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-24T08:27:15.113 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-24T08:27:15.113 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun 2024-04-24T08:27:15.114 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-24T08:27:15.114 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-24T08:27:15.114 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-24T08:27:15.114 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-24T08:27:15.114 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti 2024-04-24T08:27:15.114 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-24T08:27:15.114 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti 2024-04-24T08:27:15.114 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-24T08:27:15.114 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via 2024-04-24T08:27:15.114 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-24T08:27:15.114 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-24T08:27:15.114 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet 2024-04-24T08:27:15.114 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-24T08:27:15.114 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-24T08:27:15.115 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom 2024-04-24T08:27:15.115 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-24T08:27:15.115 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-04-24T08:27:15.115 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv 2024-04-24T08:27:15.115 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-24T08:27:15.115 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154 2024-04-24T08:27:15.115 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-24T08:27:15.115 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-04-24T08:27:15.115 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan 2024-04-24T08:27:15.115 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-24T08:27:15.115 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-04-24T08:27:15.115 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-04-24T08:27:15.115 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio 2024-04-24T08:27:15.115 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-04-24T08:27:15.116 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-24T08:27:15.116 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-24T08:27:15.116 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-24T08:27:15.116 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-04-24T08:27:15.116 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/net_failover.ko 2024-04-24T08:27:15.116 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-04-24T08:27:15.116 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-04-24T08:27:15.116 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs 2024-04-24T08:27:15.116 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-24T08:27:15.116 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-24T08:27:15.116 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy 2024-04-24T08:27:15.116 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-04-24T08:27:15.116 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-24T08:27:15.116 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-04-24T08:27:15.116 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-24T08:27:15.117 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-04-24T08:27:15.117 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-04-24T08:27:15.117 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-04-24T08:27:15.117 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-04-24T08:27:15.117 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/dp83640.ko 2024-04-24T08:27:15.117 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-04-24T08:27:15.117 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-04-24T08:27:15.117 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-04-24T08:27:15.117 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-04-24T08:27:15.117 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-04-24T08:27:15.117 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-04-24T08:27:15.117 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-04-24T08:27:15.117 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-04-24T08:27:15.117 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-04-24T08:27:15.118 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-04-24T08:27:15.118 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-04-24T08:27:15.118 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-04-24T08:27:15.118 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp 2024-04-24T08:27:15.118 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-24T08:27:15.118 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-04-24T08:27:15.118 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-24T08:27:15.118 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_generic.ko 2024-04-24T08:27:15.118 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-24T08:27:15.118 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-24T08:27:15.118 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-04-24T08:27:15.118 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-04-24T08:27:15.118 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-04-24T08:27:15.118 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip 2024-04-24T08:27:15.118 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slhc.ko 2024-04-24T08:27:15.119 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-04-24T08:27:15.119 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-04-24T08:27:15.119 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-04-24T08:27:15.119 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team 2024-04-24T08:27:15.119 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-04-24T08:27:15.119 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-24T08:27:15.119 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-24T08:27:15.119 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-24T08:27:15.119 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-04-24T08:27:15.119 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-24T08:27:15.119 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tun.ko 2024-04-24T08:27:15.119 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-04-24T08:27:15.119 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/virtio_net.ko 2024-04-24T08:27:15.119 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3 2024-04-24T08:27:15.120 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-24T08:27:15.120 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan 2024-04-24T08:27:15.120 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-04-24T08:27:15.120 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback 2024-04-24T08:27:15.120 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-24T08:27:15.120 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netfront.ko 2024-04-24T08:27:15.120 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb 2024-04-24T08:27:15.120 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-04-24T08:27:15.120 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme 2024-04-24T08:27:15.120 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host 2024-04-24T08:27:15.120 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-04-24T08:27:15.120 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-04-24T08:27:15.120 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport 2024-04-24T08:27:15.120 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-04-24T08:27:15.120 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-04-24T08:27:15.121 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-04-24T08:27:15.121 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-04-24T08:27:15.121 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci 2024-04-24T08:27:15.121 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug 2024-04-24T08:27:15.121 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-24T08:27:15.121 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie 2024-04-24T08:27:15.121 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie/aer_inject.ko 2024-04-24T08:27:15.121 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-04-24T08:27:15.121 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia 2024-04-24T08:27:15.121 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-04-24T08:27:15.121 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-04-24T08:27:15.121 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-24T08:27:15.121 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl 2024-04-24T08:27:15.121 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel 2024-04-24T08:27:15.121 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-24T08:27:15.122 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform 2024-04-24T08:27:15.122 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86 2024-04-24T08:27:15.122 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-24T08:27:15.122 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-04-24T08:27:15.122 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-24T08:27:15.122 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-24T08:27:15.122 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-24T08:27:15.122 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-24T08:27:15.122 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-24T08:27:15.122 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-24T08:27:15.122 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-24T08:27:15.122 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-24T08:27:15.122 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-24T08:27:15.123 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-04-24T08:27:15.123 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel 2024-04-24T08:27:15.123 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-24T08:27:15.123 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-04-24T08:27:15.123 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-24T08:27:15.123 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-24T08:27:15.123 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-24T08:27:15.123 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-24T08:27:15.123 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-24T08:27:15.123 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-24T08:27:15.123 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-24T08:27:15.123 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-24T08:27:15.123 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-24T08:27:15.123 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-04-24T08:27:15.124 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap 2024-04-24T08:27:15.124 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-24T08:27:15.124 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-24T08:27:15.124 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps 2024-04-24T08:27:15.124 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients 2024-04-24T08:27:15.124 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-24T08:27:15.124 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-24T08:27:15.124 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-04-24T08:27:15.124 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-04-24T08:27:15.124 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp 2024-04-24T08:27:15.124 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-04-24T08:27:15.124 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-04-24T08:27:15.124 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras 2024-04-24T08:27:15.124 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd 2024-04-24T08:27:15.125 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl 2024-04-24T08:27:15.125 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-24T08:27:15.125 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-04-24T08:27:15.125 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc 2024-04-24T08:27:15.125 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-24T08:27:15.125 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-24T08:27:15.125 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-24T08:27:15.125 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-24T08:27:15.125 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-24T08:27:15.125 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-24T08:27:15.125 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-24T08:27:15.125 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-24T08:27:15.125 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-24T08:27:15.125 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-24T08:27:15.126 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-04-24T08:27:15.126 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-24T08:27:15.126 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-24T08:27:15.126 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-24T08:27:15.126 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-24T08:27:15.126 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-24T08:27:15.126 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-24T08:27:15.126 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-04-24T08:27:15.126 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-24T08:27:15.126 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-24T08:27:15.126 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-24T08:27:15.126 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-24T08:27:15.126 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-24T08:27:15.127 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-24T08:27:15.127 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-24T08:27:15.127 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-24T08:27:15.127 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-24T08:27:15.127 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-24T08:27:15.127 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-24T08:27:15.127 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-24T08:27:15.127 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-04-24T08:27:15.127 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi 2024-04-24T08:27:15.127 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-04-24T08:27:15.127 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-04-24T08:27:15.127 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-04-24T08:27:15.127 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-04-24T08:27:15.127 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-04-24T08:27:15.127 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid 2024-04-24T08:27:15.128 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-24T08:27:15.128 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-04-24T08:27:15.128 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx 2024-04-24T08:27:15.128 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-24T08:27:15.128 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-24T08:27:15.128 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-04-24T08:27:15.128 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr 2024-04-24T08:27:15.128 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-24T08:27:15.128 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-04-24T08:27:15.128 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi 2024-04-24T08:27:15.128 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-24T08:27:15.128 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa 2024-04-24T08:27:15.128 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-04-24T08:27:15.128 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc 2024-04-24T08:27:15.128 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-24T08:27:15.128 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i 2024-04-24T08:27:15.129 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-24T08:27:15.129 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-04-24T08:27:15.129 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor 2024-04-24T08:27:15.129 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-24T08:27:15.129 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi 2024-04-24T08:27:15.129 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i 2024-04-24T08:27:15.129 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-24T08:27:15.129 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i 2024-04-24T08:27:15.129 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-24T08:27:15.129 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-24T08:27:15.129 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-04-24T08:27:15.129 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler 2024-04-24T08:27:15.129 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-04-24T08:27:15.129 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-04-24T08:27:15.130 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-04-24T08:27:15.130 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-04-24T08:27:15.130 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-04-24T08:27:15.130 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r 2024-04-24T08:27:15.130 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-24T08:27:15.130 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-04-24T08:27:15.130 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe 2024-04-24T08:27:15.130 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-24T08:27:15.130 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-24T08:27:15.130 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic 2024-04-24T08:27:15.130 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-04-24T08:27:15.130 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-04-24T08:27:15.130 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-04-24T08:27:15.131 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-04-24T08:27:15.131 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-04-24T08:27:15.131 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-04-24T08:27:15.131 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-04-24T08:27:15.131 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci 2024-04-24T08:27:15.131 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-04-24T08:27:15.131 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-24T08:27:15.131 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-24T08:27:15.131 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc 2024-04-24T08:27:15.131 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-04-24T08:27:15.131 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-04-24T08:27:15.131 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-24T08:27:15.131 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas 2024-04-24T08:27:15.131 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-04-24T08:27:15.132 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc 2024-04-24T08:27:15.132 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-24T08:27:15.132 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid 2024-04-24T08:27:15.132 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-04-24T08:27:15.132 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-24T08:27:15.132 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-24T08:27:15.132 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-24T08:27:15.132 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas 2024-04-24T08:27:15.132 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-24T08:27:15.132 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas 2024-04-24T08:27:15.132 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-24T08:27:15.132 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-04-24T08:27:15.132 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001 2024-04-24T08:27:15.132 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-24T08:27:15.133 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-04-24T08:27:15.133 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-04-24T08:27:15.133 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx 2024-04-24T08:27:15.133 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-24T08:27:15.133 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-24T08:27:15.133 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx 2024-04-24T08:27:15.133 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-24T08:27:15.133 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-04-24T08:27:15.133 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-04-24T08:27:15.133 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-24T08:27:15.133 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-24T08:27:15.133 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-24T08:27:15.133 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-24T08:27:15.134 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-24T08:27:15.134 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-04-24T08:27:15.134 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-04-24T08:27:15.134 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-04-24T08:27:15.134 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2 2024-04-24T08:27:15.134 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-24T08:27:15.134 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-04-24T08:27:15.134 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-24T08:27:15.134 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-04-24T08:27:15.134 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb 2024-04-24T08:27:15.134 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-04-24T08:27:15.134 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target 2024-04-24T08:27:15.134 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi 2024-04-24T08:27:15.134 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-24T08:27:15.135 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback 2024-04-24T08:27:15.135 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-24T08:27:15.135 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-04-24T08:27:15.135 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-04-24T08:27:15.135 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-04-24T08:27:15.135 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-04-24T08:27:15.135 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-04-24T08:27:15.135 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc 2024-04-24T08:27:15.135 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-24T08:27:15.135 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal 2024-04-24T08:27:15.135 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel 2024-04-24T08:27:15.135 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal 2024-04-24T08:27:15.136 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-24T08:27:15.136 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-24T08:27:15.136 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-24T08:27:15.136 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-24T08:27:15.136 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-24T08:27:15.136 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-24T08:27:15.136 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-24T08:27:15.136 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-24T08:27:15.136 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-24T08:27:15.136 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-24T08:27:15.136 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-24T08:27:15.136 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-24T08:27:15.136 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-24T08:27:15.137 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-24T08:27:15.137 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-24T08:27:15.137 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-24T08:27:15.137 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-24T08:27:15.137 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-24T08:27:15.137 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty 2024-04-24T08:27:15.137 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless 2024-04-24T08:27:15.137 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-24T08:27:15.137 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-04-24T08:27:15.137 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-04-24T08:27:15.137 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-04-24T08:27:15.137 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial 2024-04-24T08:27:15.137 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250 2024-04-24T08:27:15.138 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-24T08:27:15.138 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-04-24T08:27:15.138 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm 2024-04-24T08:27:15.138 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-24T08:27:15.138 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs 2024-04-24T08:27:15.138 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core 2024-04-24T08:27:15.138 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-24T08:27:15.138 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host 2024-04-24T08:27:15.138 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-24T08:27:15.138 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio 2024-04-24T08:27:15.138 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-04-24T08:27:15.138 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-04-24T08:27:15.138 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-04-24T08:27:15.138 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-04-24T08:27:15.139 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-04-24T08:27:15.139 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb 2024-04-24T08:27:15.139 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class 2024-04-24T08:27:15.139 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-04-24T08:27:15.139 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-24T08:27:15.139 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-04-24T08:27:15.139 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-04-24T08:27:15.139 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host 2024-04-24T08:27:15.139 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-24T08:27:15.139 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image 2024-04-24T08:27:15.139 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-04-24T08:27:15.139 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-04-24T08:27:15.139 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc 2024-04-24T08:27:15.139 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-04-24T08:27:15.140 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-04-24T08:27:15.140 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-04-24T08:27:15.140 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-04-24T08:27:15.140 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-04-24T08:27:15.140 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-04-24T08:27:15.140 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-04-24T08:27:15.140 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-24T08:27:15.140 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-04-24T08:27:15.140 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-04-24T08:27:15.140 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga 2024-04-24T08:27:15.140 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-24T08:27:15.140 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-24T08:27:15.140 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-04-24T08:27:15.140 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-04-24T08:27:15.141 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-24T08:27:15.141 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-04-24T08:27:15.141 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-04-24T08:27:15.141 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy 2024-04-24T08:27:15.141 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-04-24T08:27:15.141 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial 2024-04-24T08:27:15.141 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-04-24T08:27:15.141 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-04-24T08:27:15.141 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-24T08:27:15.141 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-04-24T08:27:15.141 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-04-24T08:27:15.141 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-04-24T08:27:15.141 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-24T08:27:15.141 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-24T08:27:15.142 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-04-24T08:27:15.142 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-24T08:27:15.142 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-24T08:27:15.142 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-24T08:27:15.142 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-04-24T08:27:15.142 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-04-24T08:27:15.142 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-04-24T08:27:15.142 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-04-24T08:27:15.142 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-24T08:27:15.142 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-04-24T08:27:15.142 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-24T08:27:15.142 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-24T08:27:15.142 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-24T08:27:15.142 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-04-24T08:27:15.143 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-04-24T08:27:15.143 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-04-24T08:27:15.143 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-04-24T08:27:15.143 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-04-24T08:27:15.143 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-04-24T08:27:15.143 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-04-24T08:27:15.143 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-04-24T08:27:15.143 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-04-24T08:27:15.143 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-04-24T08:27:15.143 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-04-24T08:27:15.143 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-04-24T08:27:15.143 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-04-24T08:27:15.143 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-04-24T08:27:15.143 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-24T08:27:15.144 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-04-24T08:27:15.144 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-04-24T08:27:15.144 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-24T08:27:15.144 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-24T08:27:15.144 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-04-24T08:27:15.144 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-24T08:27:15.144 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-04-24T08:27:15.144 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-04-24T08:27:15.144 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-24T08:27:15.144 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage 2024-04-24T08:27:15.144 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-04-24T08:27:15.144 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-24T08:27:15.144 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-24T08:27:15.144 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-24T08:27:15.144 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-24T08:27:15.145 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-24T08:27:15.145 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-24T08:27:15.145 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-24T08:27:15.145 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-04-24T08:27:15.145 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-24T08:27:15.145 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-24T08:27:15.145 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-24T08:27:15.145 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-24T08:27:15.145 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-24T08:27:15.145 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-04-24T08:27:15.145 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip 2024-04-24T08:27:15.145 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-24T08:27:15.145 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-24T08:27:15.145 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-24T08:27:15.146 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio 2024-04-24T08:27:15.146 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci 2024-04-24T08:27:15.146 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-24T08:27:15.146 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-24T08:27:15.146 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-04-24T08:27:15.146 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-24T08:27:15.146 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost 2024-04-24T08:27:15.146 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-04-24T08:27:15.146 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-24T08:27:15.146 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-04-24T08:27:15.146 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-04-24T08:27:15.146 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video 2024-04-24T08:27:15.146 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight 2024-04-24T08:27:15.147 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-04-24T08:27:15.147 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-04-24T08:27:15.147 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-24T08:27:15.147 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio 2024-04-24T08:27:15.147 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio.ko 2024-04-24T08:27:15.147 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_balloon.ko 2024-04-24T08:27:15.147 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_mmio.ko 2024-04-24T08:27:15.147 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci.ko 2024-04-24T08:27:15.147 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-04-24T08:27:15.147 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-04-24T08:27:15.147 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_ring.ko 2024-04-24T08:27:15.147 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1 2024-04-24T08:27:15.147 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters 2024-04-24T08:27:15.147 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-04-24T08:27:15.148 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-04-24T08:27:15.148 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves 2024-04-24T08:27:15.148 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-24T08:27:15.148 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-24T08:27:15.148 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-24T08:27:15.148 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-24T08:27:15.148 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-24T08:27:15.148 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-24T08:27:15.148 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-24T08:27:15.148 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-24T08:27:15.148 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-24T08:27:15.148 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-24T08:27:15.148 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-24T08:27:15.148 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-04-24T08:27:15.149 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog 2024-04-24T08:27:15.149 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-24T08:27:15.149 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-24T08:27:15.149 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-24T08:27:15.149 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-04-24T08:27:15.149 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-04-24T08:27:15.149 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-24T08:27:15.149 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-24T08:27:15.149 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-04-24T08:27:15.149 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-04-24T08:27:15.149 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-24T08:27:15.149 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-24T08:27:15.149 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-04-24T08:27:15.149 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-04-24T08:27:15.149 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-04-24T08:27:15.150 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-24T08:27:15.150 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-24T08:27:15.150 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-24T08:27:15.150 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-24T08:27:15.150 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-04-24T08:27:15.150 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-24T08:27:15.150 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-04-24T08:27:15.150 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-24T08:27:15.150 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-24T08:27:15.150 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-24T08:27:15.150 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-04-24T08:27:15.150 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-04-24T08:27:15.150 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen 2024-04-24T08:27:15.151 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-acpi-processor.ko 2024-04-24T08:27:15.151 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-04-24T08:27:15.151 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-04-24T08:27:15.151 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-04-24T08:27:15.151 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback 2024-04-24T08:27:15.151 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-24T08:27:15.151 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-04-24T08:27:15.151 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs 2024-04-24T08:27:15.151 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-24T08:27:15.151 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs 2024-04-24T08:27:15.151 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p 2024-04-24T08:27:15.151 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-04-24T08:27:15.151 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs 2024-04-24T08:27:15.151 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-04-24T08:27:15.151 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs 2024-04-24T08:27:15.152 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-04-24T08:27:15.152 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-04-24T08:27:15.152 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs 2024-04-24T08:27:15.152 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-04-24T08:27:15.152 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles 2024-04-24T08:27:15.152 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-04-24T08:27:15.152 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph 2024-04-24T08:27:15.152 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-04-24T08:27:15.152 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda 2024-04-24T08:27:15.152 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-04-24T08:27:15.152 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs 2024-04-24T08:27:15.152 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-04-24T08:27:15.152 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm 2024-04-24T08:27:15.152 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-04-24T08:27:15.153 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs 2024-04-24T08:27:15.153 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs/ecryptfs.ko 2024-04-24T08:27:15.153 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs 2024-04-24T08:27:15.153 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-04-24T08:27:15.153 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat 2024-04-24T08:27:15.153 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/fat.ko 2024-04-24T08:27:15.153 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-04-24T08:27:15.153 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/vfat.ko 2024-04-24T08:27:15.153 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse 2024-04-24T08:27:15.153 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-04-24T08:27:15.153 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/fuse.ko 2024-04-24T08:27:15.153 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2 2024-04-24T08:27:15.153 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-04-24T08:27:15.154 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs 2024-04-24T08:27:15.154 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-04-24T08:27:15.154 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus 2024-04-24T08:27:15.154 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-04-24T08:27:15.154 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs 2024-04-24T08:27:15.154 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-04-24T08:27:15.154 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs 2024-04-24T08:27:15.154 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-04-24T08:27:15.154 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd 2024-04-24T08:27:15.154 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-04-24T08:27:15.154 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix 2024-04-24T08:27:15.154 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-04-24T08:27:15.154 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs 2024-04-24T08:27:15.155 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-04-24T08:27:15.155 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs 2024-04-24T08:27:15.155 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout 2024-04-24T08:27:15.155 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-24T08:27:15.155 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout 2024-04-24T08:27:15.155 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-24T08:27:15.155 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout 2024-04-24T08:27:15.155 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-24T08:27:15.155 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-04-24T08:27:15.155 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-04-24T08:27:15.155 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-04-24T08:27:15.155 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common 2024-04-24T08:27:15.155 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-04-24T08:27:15.155 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-04-24T08:27:15.156 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd 2024-04-24T08:27:15.156 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-04-24T08:27:15.156 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2 2024-04-24T08:27:15.156 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-04-24T08:27:15.156 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls 2024-04-24T08:27:15.156 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-04-24T08:27:15.156 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-04-24T08:27:15.156 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-04-24T08:27:15.156 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-04-24T08:27:15.156 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-04-24T08:27:15.156 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-04-24T08:27:15.156 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-04-24T08:27:15.156 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-04-24T08:27:15.157 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-04-24T08:27:15.157 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-04-24T08:27:15.157 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-04-24T08:27:15.157 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-04-24T08:27:15.157 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-04-24T08:27:15.157 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-04-24T08:27:15.157 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-04-24T08:27:15.157 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-04-24T08:27:15.157 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-04-24T08:27:15.157 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-04-24T08:27:15.157 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-04-24T08:27:15.157 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-04-24T08:27:15.157 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-04-24T08:27:15.157 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-04-24T08:27:15.158 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-04-24T08:27:15.158 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-04-24T08:27:15.158 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-04-24T08:27:15.158 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-04-24T08:27:15.158 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-04-24T08:27:15.158 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-04-24T08:27:15.158 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-04-24T08:27:15.158 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-04-24T08:27:15.158 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-04-24T08:27:15.158 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-04-24T08:27:15.158 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-04-24T08:27:15.158 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-04-24T08:27:15.158 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-04-24T08:27:15.158 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-04-24T08:27:15.159 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-04-24T08:27:15.159 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-04-24T08:27:15.159 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-04-24T08:27:15.159 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-04-24T08:27:15.159 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-04-24T08:27:15.159 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-04-24T08:27:15.159 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-04-24T08:27:15.159 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-04-24T08:27:15.159 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-04-24T08:27:15.159 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-04-24T08:27:15.159 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-04-24T08:27:15.159 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-04-24T08:27:15.159 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-24T08:27:15.160 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-04-24T08:27:15.160 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2 2024-04-24T08:27:15.160 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster 2024-04-24T08:27:15.160 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-24T08:27:15.160 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm 2024-04-24T08:27:15.160 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-24T08:27:15.160 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs 2024-04-24T08:27:15.160 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-24T08:27:15.160 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-04-24T08:27:15.160 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-24T08:27:15.160 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-24T08:27:15.160 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-24T08:27:15.160 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs 2024-04-24T08:27:15.160 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-04-24T08:27:15.161 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore 2024-04-24T08:27:15.161 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-04-24T08:27:15.161 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs 2024-04-24T08:27:15.161 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-04-24T08:27:15.161 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs 2024-04-24T08:27:15.161 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-04-24T08:27:15.161 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb 2024-04-24T08:27:15.161 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client 2024-04-24T08:27:15.161 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-04-24T08:27:15.161 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common 2024-04-24T08:27:15.161 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-04-24T08:27:15.161 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-04-24T08:27:15.161 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs 2024-04-24T08:27:15.161 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-04-24T08:27:15.161 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv 2024-04-24T08:27:15.162 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-04-24T08:27:15.162 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf 2024-04-24T08:27:15.162 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-04-24T08:27:15.162 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs 2024-04-24T08:27:15.162 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-04-24T08:27:15.162 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs 2024-04-24T08:27:15.162 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-04-24T08:27:15.162 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel 2024-04-24T08:27:15.162 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu 2024-04-24T08:27:15.162 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu/rcutorture.ko 2024-04-24T08:27:15.162 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-04-24T08:27:15.162 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace 2024-04-24T08:27:15.162 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-04-24T08:27:15.162 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib 2024-04-24T08:27:15.162 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/asn1_encoder.ko 2024-04-24T08:27:15.163 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-04-24T08:27:15.163 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-04-24T08:27:15.163 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto 2024-04-24T08:27:15.163 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-04-24T08:27:15.163 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-04-24T08:27:15.163 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-04-24T08:27:15.163 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-04-24T08:27:15.163 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4 2024-04-24T08:27:15.163 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-04-24T08:27:15.163 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-04-24T08:27:15.163 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math 2024-04-24T08:27:15.163 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-04-24T08:27:15.163 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6 2024-04-24T08:27:15.164 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-04-24T08:27:15.164 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon 2024-04-24T08:27:15.164 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-24T08:27:15.164 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-04-24T08:27:15.164 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-04-24T08:27:15.164 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-04-24T08:27:15.164 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd 2024-04-24T08:27:15.164 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd/zstd_compress.ko 2024-04-24T08:27:15.164 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm 2024-04-24T08:27:15.164 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-04-24T08:27:15.164 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net 2024-04-24T08:27:15.164 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan 2024-04-24T08:27:15.164 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-04-24T08:27:15.164 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-04-24T08:27:15.164 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-04-24T08:27:15.165 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-04-24T08:27:15.165 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-24T08:27:15.165 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-04-24T08:27:15.165 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-04-24T08:27:15.165 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-04-24T08:27:15.165 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802 2024-04-24T08:27:15.165 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-04-24T08:27:15.165 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-04-24T08:27:15.165 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-04-24T08:27:15.165 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-04-24T08:27:15.165 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-04-24T08:27:15.165 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q 2024-04-24T08:27:15.165 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-04-24T08:27:15.165 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p 2024-04-24T08:27:15.165 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-04-24T08:27:15.165 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-04-24T08:27:15.166 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-04-24T08:27:15.166 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-04-24T08:27:15.166 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv 2024-04-24T08:27:15.166 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-04-24T08:27:15.166 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge 2024-04-24T08:27:15.166 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-04-24T08:27:15.166 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-04-24T08:27:15.166 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter 2024-04-24T08:27:15.166 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-24T08:27:15.166 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-24T08:27:15.166 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-24T08:27:15.166 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-24T08:27:15.166 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-24T08:27:15.166 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-24T08:27:15.167 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-24T08:27:15.167 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-24T08:27:15.167 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-24T08:27:15.167 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-24T08:27:15.167 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-24T08:27:15.167 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-24T08:27:15.167 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-24T08:27:15.167 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-24T08:27:15.167 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-24T08:27:15.167 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-24T08:27:15.167 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-24T08:27:15.167 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-24T08:27:15.167 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-24T08:27:15.168 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-24T08:27:15.168 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-04-24T08:27:15.168 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-04-24T08:27:15.168 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-24T08:27:15.168 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-24T08:27:15.168 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph 2024-04-24T08:27:15.168 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-04-24T08:27:15.168 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core 2024-04-24T08:27:15.168 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/failover.ko 2024-04-24T08:27:15.168 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-04-24T08:27:15.168 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp 2024-04-24T08:27:15.168 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-04-24T08:27:15.168 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-04-24T08:27:15.168 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-04-24T08:27:15.169 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-04-24T08:27:15.169 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver 2024-04-24T08:27:15.169 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver/dns_resolver.ko 2024-04-24T08:27:15.169 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa 2024-04-24T08:27:15.169 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-04-24T08:27:15.169 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-04-24T08:27:15.169 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-04-24T08:27:15.169 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-04-24T08:27:15.169 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-04-24T08:27:15.169 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154 2024-04-24T08:27:15.169 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan 2024-04-24T08:27:15.169 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-24T08:27:15.169 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-04-24T08:27:15.170 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-24T08:27:15.170 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4 2024-04-24T08:27:15.170 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-04-24T08:27:15.170 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-04-24T08:27:15.170 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-04-24T08:27:15.170 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-04-24T08:27:15.170 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-04-24T08:27:15.170 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-04-24T08:27:15.170 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-04-24T08:27:15.170 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-04-24T08:27:15.170 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-04-24T08:27:15.170 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-04-24T08:27:15.170 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter 2024-04-24T08:27:15.170 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-24T08:27:15.171 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-24T08:27:15.171 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-24T08:27:15.171 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-24T08:27:15.171 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-24T08:27:15.171 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-24T08:27:15.171 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-24T08:27:15.171 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-24T08:27:15.171 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-24T08:27:15.171 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-24T08:27:15.171 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-24T08:27:15.171 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-24T08:27:15.171 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-24T08:27:15.171 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-24T08:27:15.172 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-24T08:27:15.172 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-24T08:27:15.172 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-24T08:27:15.172 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-24T08:27:15.172 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-24T08:27:15.172 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-24T08:27:15.172 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-24T08:27:15.172 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-04-24T08:27:15.172 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-04-24T08:27:15.172 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-04-24T08:27:15.172 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-04-24T08:27:15.172 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-04-24T08:27:15.172 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-04-24T08:27:15.172 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-04-24T08:27:15.173 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-04-24T08:27:15.173 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-04-24T08:27:15.173 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-04-24T08:27:15.173 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-04-24T08:27:15.173 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-04-24T08:27:15.173 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-04-24T08:27:15.173 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-04-24T08:27:15.173 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-04-24T08:27:15.173 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-04-24T08:27:15.173 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-24T08:27:15.173 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6 2024-04-24T08:27:15.173 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-04-24T08:27:15.173 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-04-24T08:27:15.173 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-04-24T08:27:15.174 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-04-24T08:27:15.174 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-24T08:27:15.174 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-04-24T08:27:15.174 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-04-24T08:27:15.174 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter 2024-04-24T08:27:15.174 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-24T08:27:15.174 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-24T08:27:15.174 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-24T08:27:15.174 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-24T08:27:15.174 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-24T08:27:15.174 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-24T08:27:15.174 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-24T08:27:15.174 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-24T08:27:15.175 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-24T08:27:15.175 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-24T08:27:15.175 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-24T08:27:15.175 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-24T08:27:15.175 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-24T08:27:15.175 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-24T08:27:15.175 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-24T08:27:15.175 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-24T08:27:15.175 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-24T08:27:15.175 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-24T08:27:15.175 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-24T08:27:15.175 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-24T08:27:15.176 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-24T08:27:15.176 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-24T08:27:15.176 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-24T08:27:15.176 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-24T08:27:15.176 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-04-24T08:27:15.176 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-04-24T08:27:15.176 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-24T08:27:15.176 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key 2024-04-24T08:27:15.176 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-04-24T08:27:15.176 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp 2024-04-24T08:27:15.176 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-04-24T08:27:15.176 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-24T08:27:15.176 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-04-24T08:27:15.176 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-04-24T08:27:15.176 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-04-24T08:27:15.177 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-04-24T08:27:15.177 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-04-24T08:27:15.177 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc 2024-04-24T08:27:15.177 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-04-24T08:27:15.177 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154 2024-04-24T08:27:15.177 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-04-24T08:27:15.177 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls 2024-04-24T08:27:15.177 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-04-24T08:27:15.177 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter 2024-04-24T08:27:15.177 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset 2024-04-24T08:27:15.177 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-04-24T08:27:15.177 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-24T08:27:15.177 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-24T08:27:15.178 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-24T08:27:15.178 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-24T08:27:15.178 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-24T08:27:15.178 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-24T08:27:15.178 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-24T08:27:15.178 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-24T08:27:15.178 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-24T08:27:15.178 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-24T08:27:15.178 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-24T08:27:15.178 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-24T08:27:15.178 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-24T08:27:15.178 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-24T08:27:15.178 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-24T08:27:15.178 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs 2024-04-24T08:27:15.179 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-24T08:27:15.179 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-24T08:27:15.179 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-24T08:27:15.179 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-24T08:27:15.179 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-24T08:27:15.179 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-24T08:27:15.179 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-24T08:27:15.179 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-24T08:27:15.179 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-24T08:27:15.179 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-24T08:27:15.179 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-24T08:27:15.179 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-24T08:27:15.179 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-24T08:27:15.179 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-24T08:27:15.180 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-04-24T08:27:15.180 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-04-24T08:27:15.180 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-24T08:27:15.180 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-24T08:27:15.180 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-24T08:27:15.180 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-24T08:27:15.180 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-24T08:27:15.180 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-24T08:27:15.180 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-24T08:27:15.180 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-24T08:27:15.180 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-24T08:27:15.180 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-24T08:27:15.180 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-24T08:27:15.180 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-24T08:27:15.181 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-24T08:27:15.181 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-04-24T08:27:15.181 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-24T08:27:15.181 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-04-24T08:27:15.181 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-04-24T08:27:15.181 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-24T08:27:15.181 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-24T08:27:15.181 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-04-24T08:27:15.181 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-04-24T08:27:15.181 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-24T08:27:15.181 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-24T08:27:15.181 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-04-24T08:27:15.181 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-04-24T08:27:15.181 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-24T08:27:15.182 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-04-24T08:27:15.182 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-24T08:27:15.182 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-24T08:27:15.182 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-04-24T08:27:15.182 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-04-24T08:27:15.182 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-04-24T08:27:15.182 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-24T08:27:15.182 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-04-24T08:27:15.182 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-24T08:27:15.182 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-04-24T08:27:15.182 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-04-24T08:27:15.182 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-04-24T08:27:15.182 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-04-24T08:27:15.182 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-04-24T08:27:15.183 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-04-24T08:27:15.183 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-04-24T08:27:15.183 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_quota.ko 2024-04-24T08:27:15.183 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-04-24T08:27:15.183 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-04-24T08:27:15.183 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-04-24T08:27:15.183 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-04-24T08:27:15.183 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tunnel.ko 2024-04-24T08:27:15.183 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-04-24T08:27:15.183 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-24T08:27:15.183 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-24T08:27:15.183 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-24T08:27:15.184 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-04-24T08:27:15.184 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-04-24T08:27:15.184 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-04-24T08:27:15.184 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-04-24T08:27:15.184 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-24T08:27:15.184 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-04-24T08:27:15.184 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-04-24T08:27:15.184 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-24T08:27:15.184 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-04-24T08:27:15.184 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-04-24T08:27:15.184 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-24T08:27:15.184 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-04-24T08:27:15.184 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-24T08:27:15.184 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-04-24T08:27:15.184 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-24T08:27:15.185 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-24T08:27:15.185 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-04-24T08:27:15.185 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-04-24T08:27:15.185 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-04-24T08:27:15.185 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-04-24T08:27:15.185 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-04-24T08:27:15.185 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-04-24T08:27:15.185 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-04-24T08:27:15.185 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-04-24T08:27:15.185 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-04-24T08:27:15.185 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-04-24T08:27:15.185 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-04-24T08:27:15.185 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-04-24T08:27:15.185 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-04-24T08:27:15.185 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-04-24T08:27:15.186 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-04-24T08:27:15.186 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-04-24T08:27:15.186 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-04-24T08:27:15.186 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-04-24T08:27:15.186 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-04-24T08:27:15.186 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-04-24T08:27:15.186 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-04-24T08:27:15.186 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-04-24T08:27:15.186 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-04-24T08:27:15.186 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-04-24T08:27:15.186 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-04-24T08:27:15.186 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-04-24T08:27:15.186 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-04-24T08:27:15.186 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-04-24T08:27:15.186 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-04-24T08:27:15.187 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-04-24T08:27:15.187 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-04-24T08:27:15.187 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-04-24T08:27:15.187 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-04-24T08:27:15.187 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-04-24T08:27:15.187 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-04-24T08:27:15.187 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-04-24T08:27:15.187 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-04-24T08:27:15.187 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-04-24T08:27:15.187 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-04-24T08:27:15.187 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-04-24T08:27:15.187 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-04-24T08:27:15.187 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-04-24T08:27:15.187 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-04-24T08:27:15.188 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-04-24T08:27:15.188 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-04-24T08:27:15.188 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-04-24T08:27:15.188 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-04-24T08:27:15.188 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-04-24T08:27:15.188 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-04-24T08:27:15.188 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-04-24T08:27:15.188 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-04-24T08:27:15.188 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink 2024-04-24T08:27:15.188 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-04-24T08:27:15.188 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh 2024-04-24T08:27:15.188 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-04-24T08:27:15.188 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch 2024-04-24T08:27:15.189 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-04-24T08:27:15.189 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-04-24T08:27:15.189 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-04-24T08:27:15.189 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-04-24T08:27:15.189 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet 2024-04-24T08:27:15.189 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-04-24T08:27:15.189 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds 2024-04-24T08:27:15.189 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-04-24T08:27:15.189 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-04-24T08:27:15.189 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-04-24T08:27:15.189 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched 2024-04-24T08:27:15.189 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-04-24T08:27:15.189 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-04-24T08:27:15.189 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-04-24T08:27:15.189 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-04-24T08:27:15.190 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-04-24T08:27:15.190 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-04-24T08:27:15.190 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-04-24T08:27:15.190 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-04-24T08:27:15.190 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-04-24T08:27:15.190 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-04-24T08:27:15.190 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-04-24T08:27:15.190 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-04-24T08:27:15.190 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-04-24T08:27:15.190 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-04-24T08:27:15.190 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-04-24T08:27:15.190 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-04-24T08:27:15.190 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-04-24T08:27:15.191 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-04-24T08:27:15.191 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-04-24T08:27:15.191 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-04-24T08:27:15.191 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-04-24T08:27:15.191 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-04-24T08:27:15.191 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-04-24T08:27:15.191 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-04-24T08:27:15.191 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-04-24T08:27:15.191 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-04-24T08:27:15.191 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-04-24T08:27:15.191 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-04-24T08:27:15.191 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-04-24T08:27:15.191 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-04-24T08:27:15.191 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-04-24T08:27:15.191 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-04-24T08:27:15.192 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-04-24T08:27:15.192 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-04-24T08:27:15.192 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-04-24T08:27:15.192 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-04-24T08:27:15.192 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-04-24T08:27:15.192 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-04-24T08:27:15.192 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-04-24T08:27:15.192 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-04-24T08:27:15.192 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-04-24T08:27:15.192 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-04-24T08:27:15.192 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-04-24T08:27:15.192 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp 2024-04-24T08:27:15.192 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-04-24T08:27:15.192 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-04-24T08:27:15.193 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc 2024-04-24T08:27:15.193 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss 2024-04-24T08:27:15.193 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-24T08:27:15.193 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-24T08:27:15.193 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-04-24T08:27:15.193 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma 2024-04-24T08:27:15.193 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-24T08:27:15.193 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc 2024-04-24T08:27:15.193 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-04-24T08:27:15.193 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-04-24T08:27:15.193 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix 2024-04-24T08:27:15.193 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-04-24T08:27:15.193 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock 2024-04-24T08:27:15.193 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-24T08:27:15.194 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-24T08:27:15.194 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-04-24T08:27:15.194 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-24T08:27:15.194 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-24T08:27:15.194 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm 2024-04-24T08:27:15.194 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-24T08:27:15.194 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security 2024-04-24T08:27:15.194 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys 2024-04-24T08:27:15.194 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys 2024-04-24T08:27:15.194 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-04-24T08:27:15.194 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys 2024-04-24T08:27:15.194 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys/trusted.ko 2024-04-24T08:27:15.195 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-04-24T08:27:15.195 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-04-24T08:27:15.195 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-04-24T08:27:15.195 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-04-24T08:27:15.195 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-04-24T08:27:15.195 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-04-24T08:27:15.195 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-04-24T08:27:15.195 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-04-24T08:27:15.195 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-04-24T08:27:15.195 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-04-24T08:27:15.195 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-04-24T08:27:15.195 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-04-24T08:27:15.195 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-04-24T08:27:15.195 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-24T08:27:15.199 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-04-24T08:27:15.200 DEBUG:teuthology.orchestra.run.smithi138:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-04-24T08:27:15.222 INFO:teuthology.orchestra.run.smithi138.stdout:postinstall scriptlet (using /bin/sh): 2024-04-24T08:27:15.222 INFO:teuthology.orchestra.run.smithi138.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-24T08:27:15.222 INFO:teuthology.orchestra.run.smithi138.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g806ec36b2112 /lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-24T08:27:15.222 INFO:teuthology.orchestra.run.smithi138.stdout:fi 2024-04-24T08:27:15.222 INFO:teuthology.orchestra.run.smithi138.stdout:for file in vmlinuz System.map config; do 2024-04-24T08:27:15.222 INFO:teuthology.orchestra.run.smithi138.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112"; then 2024-04-24T08:27:15.222 INFO:teuthology.orchestra.run.smithi138.stdout: cp "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112" 2024-04-24T08:27:15.223 INFO:teuthology.orchestra.run.smithi138.stdout: fi 2024-04-24T08:27:15.223 INFO:teuthology.orchestra.run.smithi138.stdout:done 2024-04-24T08:27:15.223 INFO:teuthology.orchestra.run.smithi138.stdout:if [ -d "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" ] && \ 2024-04-24T08:27:15.223 INFO:teuthology.orchestra.run.smithi138.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" >/dev/null 2>&1; then 2024-04-24T08:27:15.223 INFO:teuthology.orchestra.run.smithi138.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-24T08:27:15.223 INFO:teuthology.orchestra.run.smithi138.stdout: cp -r "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-24T08:27:15.223 INFO:teuthology.orchestra.run.smithi138.stdout:fi 2024-04-24T08:27:15.223 INFO:teuthology.orchestra.run.smithi138.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep" ]; then 2024-04-24T08:27:15.223 INFO:teuthology.orchestra.run.smithi138.stdout: /usr/sbin/depmod 6.9.0-rc3-g806ec36b2112 2024-04-24T08:27:15.223 INFO:teuthology.orchestra.run.smithi138.stdout:fi 2024-04-24T08:27:15.223 INFO:teuthology.orchestra.run.smithi138.stdout:preuninstall scriptlet (using /bin/sh): 2024-04-24T08:27:15.223 INFO:teuthology.orchestra.run.smithi138.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-24T08:27:15.223 INFO:teuthology.orchestra.run.smithi138.stdout:kernel-install remove 6.9.0-rc3-g806ec36b2112 2024-04-24T08:27:15.223 INFO:teuthology.orchestra.run.smithi138.stdout:fi 2024-04-24T08:27:15.224 DEBUG:teuthology.orchestra.run.smithi138:> sudo rpm -qi grub2-tools 2024-04-24T08:27:15.296 INFO:teuthology.orchestra.run.smithi138.stdout:Name : grub2-tools 2024-04-24T08:27:15.296 INFO:teuthology.orchestra.run.smithi138.stdout:Epoch : 1 2024-04-24T08:27:15.296 INFO:teuthology.orchestra.run.smithi138.stdout:Version : 2.06 2024-04-24T08:27:15.296 INFO:teuthology.orchestra.run.smithi138.stdout:Release : 68.el9 2024-04-24T08:27:15.296 INFO:teuthology.orchestra.run.smithi138.stdout:Architecture: x86_64 2024-04-24T08:27:15.296 INFO:teuthology.orchestra.run.smithi138.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-04-24T08:27:15.296 INFO:teuthology.orchestra.run.smithi138.stdout:Group : Unspecified 2024-04-24T08:27:15.296 INFO:teuthology.orchestra.run.smithi138.stdout:Size : 8264254 2024-04-24T08:27:15.296 INFO:teuthology.orchestra.run.smithi138.stdout:License : GPLv3+ 2024-04-24T08:27:15.297 INFO:teuthology.orchestra.run.smithi138.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-24T08:27:15.297 INFO:teuthology.orchestra.run.smithi138.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-24T08:27:15.297 INFO:teuthology.orchestra.run.smithi138.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-24T08:27:15.297 INFO:teuthology.orchestra.run.smithi138.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-24T08:27:15.297 INFO:teuthology.orchestra.run.smithi138.stdout:Packager : builder@centos.org 2024-04-24T08:27:15.297 INFO:teuthology.orchestra.run.smithi138.stdout:Vendor : CentOS 2024-04-24T08:27:15.297 INFO:teuthology.orchestra.run.smithi138.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-24T08:27:15.297 INFO:teuthology.orchestra.run.smithi138.stdout:Summary : Support tools for GRUB. 2024-04-24T08:27:15.297 INFO:teuthology.orchestra.run.smithi138.stdout:Description : 2024-04-24T08:27:15.297 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-24T08:27:15.297 INFO:teuthology.orchestra.run.smithi138.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-24T08:27:15.297 INFO:teuthology.orchestra.run.smithi138.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-24T08:27:15.297 INFO:teuthology.orchestra.run.smithi138.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-24T08:27:15.297 INFO:teuthology.orchestra.run.smithi138.stdout:hardware devices. 2024-04-24T08:27:15.297 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-24T08:27:15.298 INFO:teuthology.orchestra.run.smithi138.stdout:This subpackage provides tools for support of all platforms. 2024-04-24T08:27:15.298 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-24T08:27:15.299 INFO:teuthology.task.kernel:Updating grub on smithi138 to boot 6.9.0-rc3-g806ec36b2112 2024-04-24T08:27:15.299 DEBUG:teuthology.orchestra.run.smithi138:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-24T08:27:15.877 INFO:teuthology.orchestra.run.smithi138.stderr:Generating grub configuration file ... 2024-04-24T08:27:17.220 INFO:teuthology.orchestra.run.smithi138.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-24T08:27:17.251 INFO:teuthology.orchestra.run.smithi138.stderr:done 2024-04-24T08:27:17.253 DEBUG:teuthology.orchestra.run.smithi138:> mktemp 2024-04-24T08:27:17.294 INFO:teuthology.orchestra.run.smithi138.stdout:/tmp/tmp.zi22FQm192 2024-04-24T08:27:17.294 DEBUG:teuthology.orchestra.run.smithi138:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.zi22FQm192 2024-04-24T08:27:17.358 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 0666 /tmp/tmp.zi22FQm192 2024-04-24T08:27:17.497 DEBUG:teuthology.orchestra.remote:smithi138:/tmp/tmp.zi22FQm192 is 6KB 2024-04-24T08:27:17.546 DEBUG:teuthology.orchestra.run.smithi138:> rm -fr /tmp/tmp.zi22FQm192 2024-04-24T08:27:17.560 DEBUG:teuthology.orchestra.run.smithi138:> sudo /bin/ls /boot/loader/entries || true 2024-04-24T08:27:17.626 INFO:teuthology.orchestra.run.smithi138.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-04-24T08:27:17.626 INFO:teuthology.orchestra.run.smithi138.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-04-24T08:27:17.626 INFO:teuthology.orchestra.run.smithi138.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-04-24T08:27:17.626 INFO:teuthology.orchestra.run.smithi138.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112.conf 2024-04-24T08:27:17.628 DEBUG:teuthology.orchestra.run.smithi138:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112 2024-04-24T08:27:17.787 DEBUG:teuthology.orchestra.run.smithi138:> sudo shutdown -r now 2024-04-24T08:27:19.454 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.83 2024-04-24T08:27:28.463 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi083.front.sepia.ceph.com' 2024-04-24T08:27:28.464 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi083.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T08:27:31.518 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.83 2024-04-24T08:27:31.869 INFO:teuthology.task.kernel:Checking client mon.c for new kernel version... 2024-04-24T08:27:31.869 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi189.front.sepia.ceph.com' 2024-04-24T08:27:31.870 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi189.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T08:27:43.530 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi083.front.sepia.ceph.com' 2024-04-24T08:27:43.531 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi083.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T08:27:47.794 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-04-24T08:27:47.795 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi138.front.sepia.ceph.com' 2024-04-24T08:27:47.795 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T08:27:50.430 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.189 2024-04-24T08:27:59.435 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi189.front.sepia.ceph.com' 2024-04-24T08:27:59.435 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi189.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T08:28:01.950 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.83 2024-04-24T08:28:02.494 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.189 2024-04-24T08:28:14.507 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi189.front.sepia.ceph.com' 2024-04-24T08:28:14.507 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi189.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T08:28:16.953 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi083.front.sepia.ceph.com' 2024-04-24T08:28:16.953 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi083.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T08:28:20.239 DEBUG:teuthology.orchestra.run.smithi083:> true 2024-04-24T08:28:20.827 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi083.front.sepia.ceph.com' 2024-04-24T08:28:20.827 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "6.9.0-rc3-g806ec36b2112"... 2024-04-24T08:28:20.827 DEBUG:teuthology.orchestra.run.smithi083:> uname -r 2024-04-24T08:28:20.869 INFO:teuthology.orchestra.run.smithi083.stdout:6.9.0-rc3-g806ec36b2112 2024-04-24T08:28:20.869 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-04-24T08:28:20.869 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-24T08:28:20.869 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-24T08:28:21.870 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-04-24T08:28:21.870 DEBUG:teuthology.orchestra.run.smithi083:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-24T08:28:22.050 INFO:teuthology.orchestra.run.smithi083.stdout:ttyS1 2024-04-24T08:28:22.093 DEBUG:teuthology.parallel:result is None 2024-04-24T08:28:32.926 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.189 2024-04-24T08:28:47.798 DEBUG:teuthology.orchestra.remote:timed out 2024-04-24T08:28:47.927 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi189.front.sepia.ceph.com' 2024-04-24T08:28:47.928 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi189.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T08:28:48.900 DEBUG:teuthology.orchestra.run.smithi189:> true 2024-04-24T08:28:49.484 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi189.front.sepia.ceph.com' 2024-04-24T08:28:49.484 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "6.9.0-rc3-g806ec36b2112"... 2024-04-24T08:28:49.484 DEBUG:teuthology.orchestra.run.smithi189:> uname -r 2024-04-24T08:28:49.528 INFO:teuthology.orchestra.run.smithi189.stdout:6.9.0-rc3-g806ec36b2112 2024-04-24T08:28:49.528 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-04-24T08:28:49.528 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-24T08:28:49.528 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-24T08:28:50.528 INFO:teuthology.task.kernel:Enabling kdb on mon.c... 2024-04-24T08:28:50.529 DEBUG:teuthology.orchestra.run.smithi189:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-24T08:28:50.675 INFO:teuthology.orchestra.run.smithi189.stdout:ttyS1 2024-04-24T08:28:50.725 DEBUG:teuthology.parallel:result is None 2024-04-24T08:28:56.799 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi138.front.sepia.ceph.com' 2024-04-24T08:28:56.800 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T08:29:00.255 DEBUG:teuthology.orchestra.run.smithi138:> true 2024-04-24T08:29:00.865 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi138.front.sepia.ceph.com' 2024-04-24T08:29:00.865 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "6.9.0-rc3-g806ec36b2112"... 2024-04-24T08:29:00.865 DEBUG:teuthology.orchestra.run.smithi138:> uname -r 2024-04-24T08:29:00.913 INFO:teuthology.orchestra.run.smithi138.stdout:6.9.0-rc3-g806ec36b2112 2024-04-24T08:29:00.913 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-04-24T08:29:00.913 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-24T08:29:00.913 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-24T08:29:01.914 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-04-24T08:29:01.915 DEBUG:teuthology.orchestra.run.smithi138:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-24T08:29:02.051 INFO:teuthology.orchestra.run.smithi138.stdout:ttyS1 2024-04-24T08:29:02.111 DEBUG:teuthology.parallel:result is None 2024-04-24T08:29:02.111 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-24T08:29:02.119 INFO:teuthology.task.internal:Creating test directory... 2024-04-24T08:29:02.119 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-24T08:29:02.128 DEBUG:teuthology.orchestra.run.smithi138:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-24T08:29:02.135 DEBUG:teuthology.orchestra.run.smithi189:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-24T08:29:02.186 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-24T08:29:02.194 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-24T08:29:02.200 INFO:teuthology.task.internal:Creating archive directory... 2024-04-24T08:29:02.200 DEBUG:teuthology.orchestra.run.smithi083:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-24T08:29:02.205 DEBUG:teuthology.orchestra.run.smithi138:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-24T08:29:02.222 DEBUG:teuthology.orchestra.run.smithi189:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-24T08:29:02.284 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-24T08:29:02.291 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-24T08:29:02.291 DEBUG:teuthology.orchestra.run.smithi083:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-24T08:29:02.296 DEBUG:teuthology.orchestra.run.smithi138:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-24T08:29:02.301 DEBUG:teuthology.orchestra.run.smithi189:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-24T08:29:02.378 INFO:teuthology.orchestra.run.smithi138.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T08:29:02.380 INFO:teuthology.orchestra.run.smithi189.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T08:29:02.385 INFO:teuthology.orchestra.run.smithi083.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T08:29:02.404 INFO:teuthology.orchestra.run.smithi138.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T08:29:02.410 INFO:teuthology.orchestra.run.smithi189.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T08:29:02.416 INFO:teuthology.orchestra.run.smithi083.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T08:29:02.420 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-24T08:29:02.427 INFO:teuthology.task.internal:Configuring sudo... 2024-04-24T08:29:02.427 DEBUG:teuthology.orchestra.run.smithi083:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-24T08:29:02.433 DEBUG:teuthology.orchestra.run.smithi138:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-24T08:29:02.437 DEBUG:teuthology.orchestra.run.smithi189:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-24T08:29:02.513 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-24T08:29:02.522 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-24T08:29:02.522 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-24T08:29:02.529 DEBUG:teuthology.orchestra.run.smithi138:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-24T08:29:02.534 DEBUG:teuthology.orchestra.run.smithi189:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-24T08:29:02.580 DEBUG:teuthology.orchestra.run.smithi083:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T08:29:02.657 DEBUG:teuthology.orchestra.run.smithi083:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T08:29:02.748 DEBUG:teuthology.orchestra.run.smithi083:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T08:29:02.830 DEBUG:teuthology.orchestra.run.smithi083:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T08:29:02.919 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-24T08:29:02.919 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-24T08:29:03.002 DEBUG:teuthology.orchestra.run.smithi138:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T08:29:03.081 DEBUG:teuthology.orchestra.run.smithi138:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T08:29:03.159 DEBUG:teuthology.orchestra.run.smithi138:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T08:29:03.227 DEBUG:teuthology.orchestra.run.smithi138:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T08:29:03.311 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-24T08:29:03.311 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-24T08:29:03.373 DEBUG:teuthology.orchestra.run.smithi189:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T08:29:03.447 DEBUG:teuthology.orchestra.run.smithi189:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T08:29:03.595 DEBUG:teuthology.orchestra.run.smithi189:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T08:29:03.715 DEBUG:teuthology.orchestra.run.smithi189:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T08:29:03.847 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-04-24T08:29:03.847 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-24T08:29:03.961 DEBUG:teuthology.orchestra.run.smithi083:> sudo service rsyslog restart 2024-04-24T08:29:03.969 DEBUG:teuthology.orchestra.run.smithi138:> sudo service rsyslog restart 2024-04-24T08:29:03.974 DEBUG:teuthology.orchestra.run.smithi189:> sudo service rsyslog restart 2024-04-24T08:29:04.075 INFO:teuthology.orchestra.run.smithi138.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-24T08:29:04.081 INFO:teuthology.orchestra.run.smithi083.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-24T08:29:04.170 INFO:teuthology.orchestra.run.smithi189.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-24T08:29:04.561 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-24T08:29:04.569 INFO:teuthology.task.internal:Starting timer... 2024-04-24T08:29:04.569 INFO:teuthology.run_tasks:Running task pcp... 2024-04-24T08:29:04.745 INFO:teuthology.run_tasks:Running task selinux... 2024-04-24T08:29:04.753 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-04-24T08:29:04.753 DEBUG:teuthology.orchestra.run.smithi083:> sudo service auditd rotate 2024-04-24T08:29:04.898 INFO:teuthology.orchestra.run.smithi083.stdout:Rotating logs: 2024-04-24T08:29:04.903 DEBUG:teuthology.orchestra.run.smithi138:> sudo service auditd rotate 2024-04-24T08:29:05.106 INFO:teuthology.orchestra.run.smithi138.stdout:Rotating logs: 2024-04-24T08:29:05.111 DEBUG:teuthology.orchestra.run.smithi189:> sudo service auditd rotate 2024-04-24T08:29:05.291 INFO:teuthology.orchestra.run.smithi189.stdout:Rotating logs: 2024-04-24T08:29:05.296 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-24T08:29:05.297 DEBUG:teuthology.orchestra.run.smithi083:> /usr/sbin/getenforce 2024-04-24T08:29:05.353 INFO:teuthology.orchestra.run.smithi083.stdout:Permissive 2024-04-24T08:29:05.354 DEBUG:teuthology.orchestra.run.smithi138:> /usr/sbin/getenforce 2024-04-24T08:29:05.414 INFO:teuthology.orchestra.run.smithi138.stdout:Permissive 2024-04-24T08:29:05.414 DEBUG:teuthology.orchestra.run.smithi189:> /usr/sbin/getenforce 2024-04-24T08:29:05.460 INFO:teuthology.orchestra.run.smithi189.stdout:Permissive 2024-04-24T08:29:05.461 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi083.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi138.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi189.front.sepia.ceph.com': 'permissive'} 2024-04-24T08:29:05.461 DEBUG:teuthology.orchestra.run.smithi083:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-24T08:29:05.529 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T08:29:05.530 DEBUG:teuthology.orchestra.run.smithi138:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-24T08:29:05.601 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T08:29:05.602 DEBUG:teuthology.orchestra.run.smithi189:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-24T08:29:05.684 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T08:29:05.685 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-24T08:29:05.685 DEBUG:teuthology.orchestra.run.smithi083:> sudo /usr/sbin/setenforce permissive 2024-04-24T08:29:05.757 DEBUG:teuthology.orchestra.run.smithi138:> sudo /usr/sbin/setenforce permissive 2024-04-24T08:29:05.836 DEBUG:teuthology.orchestra.run.smithi189:> sudo /usr/sbin/setenforce permissive 2024-04-24T08:29:05.918 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-24T08:29:05.928 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-04-24T08:29:05.934 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-04-24T08:29:06.083 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-24T08:29:06.098 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-24T08:29:06.100 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi083.front.sepia.ceph.com,smithi138.front.sepia.ceph.com,smithi189.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-24T08:34:19.476 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi083.front.sepia.ceph.com'), Remote(name='ubuntu@smithi138.front.sepia.ceph.com'), Remote(name='ubuntu@smithi189.front.sepia.ceph.com')] 2024-04-24T08:34:19.477 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi083.front.sepia.ceph.com' 2024-04-24T08:34:19.478 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi083.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T08:34:19.591 DEBUG:teuthology.orchestra.run.smithi083:> true 2024-04-24T08:34:19.737 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi083.front.sepia.ceph.com' 2024-04-24T08:34:19.738 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi138.front.sepia.ceph.com' 2024-04-24T08:34:19.739 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T08:34:19.846 DEBUG:teuthology.orchestra.run.smithi138:> true 2024-04-24T08:34:20.002 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi138.front.sepia.ceph.com' 2024-04-24T08:34:20.003 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi189.front.sepia.ceph.com' 2024-04-24T08:34:20.003 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi189.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T08:34:20.106 DEBUG:teuthology.orchestra.run.smithi189:> true 2024-04-24T08:34:20.262 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi189.front.sepia.ceph.com' 2024-04-24T08:34:20.263 INFO:teuthology.run_tasks:Running task clock... 2024-04-24T08:34:20.275 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-24T08:34:20.275 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-24T08:34:20.276 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-24T08:34:20.283 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-24T08:34:20.284 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-24T08:34:20.291 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-24T08:34:20.291 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-24T08:34:20.371 INFO:teuthology.orchestra.run.smithi083.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-24T08:34:20.378 INFO:teuthology.orchestra.run.smithi138.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-24T08:34:20.382 INFO:teuthology.orchestra.run.smithi189.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-24T08:34:20.413 INFO:teuthology.orchestra.run.smithi083.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-24T08:34:20.422 INFO:teuthology.orchestra.run.smithi138.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-24T08:34:20.424 INFO:teuthology.orchestra.run.smithi189.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-24T08:34:20.486 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: ntpd: command not found 2024-04-24T08:34:20.498 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: ntpd: command not found 2024-04-24T08:34:20.501 INFO:teuthology.orchestra.run.smithi189.stderr:sudo: ntpd: command not found 2024-04-24T08:34:20.519 INFO:teuthology.orchestra.run.smithi083.stdout:506 Cannot talk to daemon 2024-04-24T08:34:20.530 INFO:teuthology.orchestra.run.smithi138.stdout:506 Cannot talk to daemon 2024-04-24T08:34:20.534 INFO:teuthology.orchestra.run.smithi189.stdout:506 Cannot talk to daemon 2024-04-24T08:34:20.558 INFO:teuthology.orchestra.run.smithi083.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-24T08:34:20.571 INFO:teuthology.orchestra.run.smithi138.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-24T08:34:20.575 INFO:teuthology.orchestra.run.smithi189.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-24T08:34:20.597 INFO:teuthology.orchestra.run.smithi083.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-24T08:34:20.612 INFO:teuthology.orchestra.run.smithi138.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-24T08:34:20.615 INFO:teuthology.orchestra.run.smithi189.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-24T08:34:20.731 INFO:teuthology.orchestra.run.smithi138.stderr:bash: line 1: ntpq: command not found 2024-04-24T08:34:20.732 INFO:teuthology.orchestra.run.smithi083.stderr:bash: line 1: ntpq: command not found 2024-04-24T08:34:20.739 INFO:teuthology.orchestra.run.smithi189.stderr:bash: line 1: ntpq: command not found 2024-04-24T08:34:20.740 INFO:teuthology.orchestra.run.smithi138.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-24T08:34:20.740 INFO:teuthology.orchestra.run.smithi138.stdout:=============================================================================== 2024-04-24T08:34:20.740 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T08:34:20.741 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T08:34:20.741 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T08:34:20.741 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T08:34:20.741 INFO:teuthology.orchestra.run.smithi083.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-24T08:34:20.741 INFO:teuthology.orchestra.run.smithi083.stdout:=============================================================================== 2024-04-24T08:34:20.741 INFO:teuthology.orchestra.run.smithi083.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T08:34:20.741 INFO:teuthology.orchestra.run.smithi083.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T08:34:20.742 INFO:teuthology.orchestra.run.smithi083.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T08:34:20.742 INFO:teuthology.orchestra.run.smithi083.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T08:34:20.749 INFO:teuthology.orchestra.run.smithi189.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-24T08:34:20.749 INFO:teuthology.orchestra.run.smithi189.stdout:=============================================================================== 2024-04-24T08:34:20.749 INFO:teuthology.orchestra.run.smithi189.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T08:34:20.749 INFO:teuthology.orchestra.run.smithi189.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T08:34:20.749 INFO:teuthology.orchestra.run.smithi189.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T08:34:20.749 INFO:teuthology.orchestra.run.smithi189.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T08:34:20.750 INFO:teuthology.run_tasks:Running task install... 2024-04-24T08:34:20.759 DEBUG:teuthology.task.install:project ceph 2024-04-24T08:34:20.759 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '1f81fb397ae98da7563d451a78c61574c8f4e6e0'}} 2024-04-24T08:34:20.759 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '1f81fb397ae98da7563d451a78c61574c8f4e6e0'} 2024-04-24T08:34:20.759 INFO:teuthology.task.install:Using flavor: default 2024-04-24T08:34:20.765 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-24T08:34:20.765 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-24T08:34:20.765 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1f81fb397ae98da7563d451a78c61574c8f4e6e0', 'tag': None, 'wait_for_package': False} 2024-04-24T08:34:20.766 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1f81fb397ae98da7563d451a78c61574c8f4e6e0 2024-04-24T08:34:20.768 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1f81fb397ae98da7563d451a78c61574c8f4e6e0', 'tag': None, 'wait_for_package': False} 2024-04-24T08:34:20.768 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1f81fb397ae98da7563d451a78c61574c8f4e6e0 2024-04-24T08:34:20.769 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1f81fb397ae98da7563d451a78c61574c8f4e6e0', 'tag': None, 'wait_for_package': False} 2024-04-24T08:34:20.769 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1f81fb397ae98da7563d451a78c61574c8f4e6e0 2024-04-24T08:34:20.930 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/rishabh-main-apr17-a654945/1f81fb397ae98da7563d451a78c61574c8f4e6e0/centos/9/flavors/default/ 2024-04-24T08:34:20.930 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3116.g1f81fb39 2024-04-24T08:34:20.946 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/rishabh-main-apr17-a654945/1f81fb397ae98da7563d451a78c61574c8f4e6e0/centos/9/flavors/default/ 2024-04-24T08:34:20.946 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3116.g1f81fb39 2024-04-24T08:34:20.993 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/rishabh-main-apr17-a654945/1f81fb397ae98da7563d451a78c61574c8f4e6e0/centos/9/flavors/default/ 2024-04-24T08:34:20.993 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3116.g1f81fb39 2024-04-24T08:34:21.056 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/rishabh-main-apr17-a654945/1f81fb397ae98da7563d451a78c61574c8f4e6e0/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/rishabh-main-apr17-a654945/1f81fb397ae98da7563d451a78c61574c8f4e6e0/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/rishabh-main-apr17-a654945/1f81fb397ae98da7563d451a78c61574c8f4e6e0/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-24T08:34:21.056 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-24T08:34:21.056 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-24T08:34:21.100 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/rishabh-main-apr17-a654945/1f81fb397ae98da7563d451a78c61574c8f4e6e0/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/rishabh-main-apr17-a654945/1f81fb397ae98da7563d451a78c61574c8f4e6e0/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/rishabh-main-apr17-a654945/1f81fb397ae98da7563d451a78c61574c8f4e6e0/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-24T08:34:21.101 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-04-24T08:34:21.101 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-24T08:34:21.145 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-24T08:34:21.145 DEBUG:teuthology.orchestra.run.smithi138:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1f81fb397ae98da7563d451a78c61574c8f4e6e0/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-24T08:34:21.157 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/rishabh-main-apr17-a654945/1f81fb397ae98da7563d451a78c61574c8f4e6e0/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/rishabh-main-apr17-a654945/1f81fb397ae98da7563d451a78c61574c8f4e6e0/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/rishabh-main-apr17-a654945/1f81fb397ae98da7563d451a78c61574c8f4e6e0/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-24T08:34:21.157 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-24T08:34:21.157 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-24T08:34:21.192 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-24T08:34:21.193 DEBUG:teuthology.orchestra.run.smithi189:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1f81fb397ae98da7563d451a78c61574c8f4e6e0/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-24T08:34:21.244 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-24T08:34:21.244 DEBUG:teuthology.orchestra.run.smithi083:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1f81fb397ae98da7563d451a78c61574c8f4e6e0/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-24T08:34:21.276 DEBUG:teuthology.orchestra.run.smithi138:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-24T08:34:21.326 DEBUG:teuthology.orchestra.run.smithi189:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-24T08:34:21.372 DEBUG:teuthology.orchestra.run.smithi083:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-24T08:34:21.387 DEBUG:teuthology.orchestra.run.smithi138:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-24T08:34:21.437 DEBUG:teuthology.orchestra.run.smithi189:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-24T08:34:21.462 INFO:teuthology.orchestra.run.smithi138.stdout:check_obsoletes = 1 2024-04-24T08:34:21.466 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum clean all 2024-04-24T08:34:21.475 DEBUG:teuthology.orchestra.run.smithi083:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-24T08:34:21.518 INFO:teuthology.orchestra.run.smithi189.stdout:check_obsoletes = 1 2024-04-24T08:34:21.521 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum clean all 2024-04-24T08:34:21.553 INFO:teuthology.orchestra.run.smithi083.stdout:check_obsoletes = 1 2024-04-24T08:34:21.556 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum clean all 2024-04-24T08:34:21.782 INFO:teuthology.orchestra.run.smithi138.stdout:48 files removed 2024-04-24T08:34:21.821 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-24T08:34:21.850 INFO:teuthology.orchestra.run.smithi189.stdout:48 files removed 2024-04-24T08:34:21.883 INFO:teuthology.orchestra.run.smithi083.stdout:48 files removed 2024-04-24T08:34:21.890 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-24T08:34:21.922 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-24T08:34:22.833 INFO:teuthology.orchestra.run.smithi083.stdout:ceph packages for x86_64 175 kB/s | 83 kB 00:00 2024-04-24T08:34:23.189 INFO:teuthology.orchestra.run.smithi138.stdout:ceph packages for x86_64 88 kB/s | 83 kB 00:00 2024-04-24T08:34:23.392 INFO:teuthology.orchestra.run.smithi083.stdout:ceph noarch packages 25 kB/s | 12 kB 00:00 2024-04-24T08:34:23.722 INFO:teuthology.orchestra.run.smithi138.stdout:ceph noarch packages 27 kB/s | 12 kB 00:00 2024-04-24T08:34:24.001 INFO:teuthology.orchestra.run.smithi083.stdout:ceph source packages 3.6 kB/s | 1.9 kB 00:00 2024-04-24T08:34:24.021 INFO:teuthology.orchestra.run.smithi189.stdout:ceph packages for x86_64 49 kB/s | 83 kB 00:01 2024-04-24T08:34:24.256 INFO:teuthology.orchestra.run.smithi138.stdout:ceph source packages 4.2 kB/s | 1.9 kB 00:00 2024-04-24T08:34:24.471 INFO:teuthology.orchestra.run.smithi189.stdout:ceph noarch packages 33 kB/s | 12 kB 00:00 2024-04-24T08:34:25.715 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS Stream 9 - BaseOS 4.9 MB/s | 8.0 MB 00:01 2024-04-24T08:34:25.964 INFO:teuthology.orchestra.run.smithi189.stdout:ceph source packages 1.3 kB/s | 1.9 kB 00:01 2024-04-24T08:34:26.828 INFO:teuthology.orchestra.run.smithi189.stdout:CentOS Stream 9 - BaseOS 10 MB/s | 8.0 MB 00:00 2024-04-24T08:34:27.708 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS Stream 9 - AppStream 23 MB/s | 19 MB 00:00 2024-04-24T08:34:28.705 INFO:teuthology.orchestra.run.smithi189.stdout:CentOS Stream 9 - AppStream 29 MB/s | 19 MB 00:00 2024-04-24T08:34:33.251 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS Stream 9 - CRB 3.6 MB/s | 6.2 MB 00:01 2024-04-24T08:34:33.337 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - BaseOS 916 kB/s | 8.0 MB 00:08 2024-04-24T08:34:33.347 INFO:teuthology.orchestra.run.smithi189.stdout:CentOS Stream 9 - CRB 8.1 MB/s | 6.2 MB 00:00 2024-04-24T08:34:35.123 INFO:teuthology.orchestra.run.smithi189.stdout:CentOS Stream 9 - Extras packages 37 kB/s | 16 kB 00:00 2024-04-24T08:34:35.346 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - AppStream 24 MB/s | 19 MB 00:00 2024-04-24T08:34:36.624 INFO:teuthology.orchestra.run.smithi189.stdout:Extra Packages for Enterprise Linux 16 MB/s | 21 MB 00:01 2024-04-24T08:34:37.491 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS Stream 9 - Extras packages 5.6 kB/s | 16 kB 00:02 2024-04-24T08:34:39.966 INFO:teuthology.orchestra.run.smithi083.stdout:Extra Packages for Enterprise Linux 9.0 MB/s | 21 MB 00:02 2024-04-24T08:34:42.532 INFO:teuthology.orchestra.run.smithi189.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-04-24T08:34:43.792 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - CRB 1.4 MB/s | 6.2 MB 00:04 2024-04-24T08:34:44.899 INFO:teuthology.orchestra.run.smithi189.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-24T08:34:44.901 INFO:teuthology.orchestra.run.smithi189.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-24T08:34:44.920 INFO:teuthology.orchestra.run.smithi189.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-24T08:34:44.922 INFO:teuthology.orchestra.run.smithi189.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-24T08:34:44.923 INFO:teuthology.orchestra.run.smithi189.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-24T08:34:44.924 INFO:teuthology.orchestra.run.smithi189.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-24T08:34:44.926 INFO:teuthology.orchestra.run.smithi189.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-24T08:34:44.930 INFO:teuthology.orchestra.run.smithi189.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-24T08:34:44.931 INFO:teuthology.orchestra.run.smithi189.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-24T08:34:45.003 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2024-04-24T08:34:45.024 INFO:teuthology.orchestra.run.smithi189.stdout:======================================================================================= 2024-04-24T08:34:45.024 INFO:teuthology.orchestra.run.smithi189.stdout: Package Arch Version Repository Size 2024-04-24T08:34:45.024 INFO:teuthology.orchestra.run.smithi189.stdout:======================================================================================= 2024-04-24T08:34:45.024 INFO:teuthology.orchestra.run.smithi189.stdout:Installing: 2024-04-24T08:34:45.024 INFO:teuthology.orchestra.run.smithi189.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-24T08:34:45.024 INFO:teuthology.orchestra.run.smithi189.stdout: ceph x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 6.6 k 2024-04-24T08:34:45.025 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-base x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 5.4 M 2024-04-24T08:34:45.025 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-fuse x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 890 k 2024-04-24T08:34:45.025 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 145 k 2024-04-24T08:34:45.025 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 1.5 M 2024-04-24T08:34:45.025 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3116.g1f81fb39.el9 ceph-noarch 142 k 2024-04-24T08:34:45.025 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3116.g1f81fb39.el9 ceph-noarch 3.5 M 2024-04-24T08:34:45.025 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3116.g1f81fb39.el9 ceph-noarch 7.4 M 2024-04-24T08:34:45.025 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-rook noarch 2:19.0.0-3116.g1f81fb39.el9 ceph-noarch 49 k 2024-04-24T08:34:45.025 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-radosgw x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 11 M 2024-04-24T08:34:45.025 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-test x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 49 M 2024-04-24T08:34:45.025 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-volume noarch 2:19.0.0-3116.g1f81fb39.el9 ceph-noarch 273 k 2024-04-24T08:34:45.025 INFO:teuthology.orchestra.run.smithi189.stdout: cephadm noarch 2:19.0.0-3116.g1f81fb39.el9 ceph-noarch 762 k 2024-04-24T08:34:45.026 INFO:teuthology.orchestra.run.smithi189.stdout: cephfs-mirror x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 225 k 2024-04-24T08:34:45.026 INFO:teuthology.orchestra.run.smithi189.stdout: cephfs-shell x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 27 k 2024-04-24T08:34:45.026 INFO:teuthology.orchestra.run.smithi189.stdout: cephfs-top noarch 2:19.0.0-3116.g1f81fb39.el9 ceph-noarch 18 k 2024-04-24T08:34:45.026 INFO:teuthology.orchestra.run.smithi189.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-24T08:34:45.026 INFO:teuthology.orchestra.run.smithi189.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-24T08:34:45.026 INFO:teuthology.orchestra.run.smithi189.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-24T08:34:45.026 INFO:teuthology.orchestra.run.smithi189.stdout: libcephfs-devel x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 32 k 2024-04-24T08:34:45.026 INFO:teuthology.orchestra.run.smithi189.stdout: libcephfs2 x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 746 k 2024-04-24T08:34:45.026 INFO:teuthology.orchestra.run.smithi189.stdout: librados-devel x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 127 k 2024-04-24T08:34:45.026 INFO:teuthology.orchestra.run.smithi189.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-24T08:34:45.026 INFO:teuthology.orchestra.run.smithi189.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-24T08:34:45.026 INFO:teuthology.orchestra.run.smithi189.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-24T08:34:45.026 INFO:teuthology.orchestra.run.smithi189.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-24T08:34:45.026 INFO:teuthology.orchestra.run.smithi189.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-24T08:34:45.027 INFO:teuthology.orchestra.run.smithi189.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-24T08:34:45.027 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cephfs x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 161 k 2024-04-24T08:34:45.027 INFO:teuthology.orchestra.run.smithi189.stdout: python3-rados x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 320 k 2024-04-24T08:34:45.027 INFO:teuthology.orchestra.run.smithi189.stdout: python3-rbd x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 298 k 2024-04-24T08:34:45.027 INFO:teuthology.orchestra.run.smithi189.stdout: python3-rgw x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 99 k 2024-04-24T08:34:45.027 INFO:teuthology.orchestra.run.smithi189.stdout: rbd-fuse x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 85 k 2024-04-24T08:34:45.027 INFO:teuthology.orchestra.run.smithi189.stdout: rbd-mirror x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 3.1 M 2024-04-24T08:34:45.027 INFO:teuthology.orchestra.run.smithi189.stdout: rbd-nbd x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 170 k 2024-04-24T08:34:45.027 INFO:teuthology.orchestra.run.smithi189.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-24T08:34:45.027 INFO:teuthology.orchestra.run.smithi189.stdout:Upgrading: 2024-04-24T08:34:45.027 INFO:teuthology.orchestra.run.smithi189.stdout: librados2 x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 3.4 M 2024-04-24T08:34:45.027 INFO:teuthology.orchestra.run.smithi189.stdout: librbd1 x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 3.1 M 2024-04-24T08:34:45.027 INFO:teuthology.orchestra.run.smithi189.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-24T08:34:45.027 INFO:teuthology.orchestra.run.smithi189.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-24T08:34:45.027 INFO:teuthology.orchestra.run.smithi189.stdout:Installing dependencies: 2024-04-24T08:34:45.028 INFO:teuthology.orchestra.run.smithi189.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-24T08:34:45.028 INFO:teuthology.orchestra.run.smithi189.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-24T08:34:45.028 INFO:teuthology.orchestra.run.smithi189.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-24T08:34:45.028 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-common x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 21 M 2024-04-24T08:34:45.028 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3116.g1f81fb39.el9 ceph-noarch 34 k 2024-04-24T08:34:45.028 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mds x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 2.4 M 2024-04-24T08:34:45.028 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3116.g1f81fb39.el9 ceph-noarch 249 k 2024-04-24T08:34:45.028 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mon x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 4.7 M 2024-04-24T08:34:45.028 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-osd x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 17 M 2024-04-24T08:34:45.028 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3116.g1f81fb39.el9 ceph-noarch 16 k 2024-04-24T08:34:45.028 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-selinux x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 25 k 2024-04-24T08:34:45.028 INFO:teuthology.orchestra.run.smithi189.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-24T08:34:45.029 INFO:teuthology.orchestra.run.smithi189.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-24T08:34:45.029 INFO:teuthology.orchestra.run.smithi189.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-24T08:34:45.029 INFO:teuthology.orchestra.run.smithi189.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-24T08:34:45.029 INFO:teuthology.orchestra.run.smithi189.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-24T08:34:45.029 INFO:teuthology.orchestra.run.smithi189.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-24T08:34:45.029 INFO:teuthology.orchestra.run.smithi189.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-24T08:34:45.029 INFO:teuthology.orchestra.run.smithi189.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-24T08:34:45.029 INFO:teuthology.orchestra.run.smithi189.stdout: libcephsqlite x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 163 k 2024-04-24T08:34:45.029 INFO:teuthology.orchestra.run.smithi189.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-24T08:34:45.029 INFO:teuthology.orchestra.run.smithi189.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-24T08:34:45.029 INFO:teuthology.orchestra.run.smithi189.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-24T08:34:45.029 INFO:teuthology.orchestra.run.smithi189.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-24T08:34:45.029 INFO:teuthology.orchestra.run.smithi189.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-24T08:34:45.029 INFO:teuthology.orchestra.run.smithi189.stdout: libradosstriper1 x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 501 k 2024-04-24T08:34:45.030 INFO:teuthology.orchestra.run.smithi189.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-24T08:34:45.030 INFO:teuthology.orchestra.run.smithi189.stdout: librgw2 x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 5.4 M 2024-04-24T08:34:45.030 INFO:teuthology.orchestra.run.smithi189.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-24T08:34:45.030 INFO:teuthology.orchestra.run.smithi189.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-24T08:34:45.030 INFO:teuthology.orchestra.run.smithi189.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-24T08:34:45.030 INFO:teuthology.orchestra.run.smithi189.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-24T08:34:45.030 INFO:teuthology.orchestra.run.smithi189.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-24T08:34:45.030 INFO:teuthology.orchestra.run.smithi189.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-24T08:34:45.030 INFO:teuthology.orchestra.run.smithi189.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-24T08:34:45.030 INFO:teuthology.orchestra.run.smithi189.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-24T08:34:45.030 INFO:teuthology.orchestra.run.smithi189.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-24T08:34:45.030 INFO:teuthology.orchestra.run.smithi189.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-24T08:34:45.030 INFO:teuthology.orchestra.run.smithi189.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-24T08:34:45.030 INFO:teuthology.orchestra.run.smithi189.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-24T08:34:45.031 INFO:teuthology.orchestra.run.smithi189.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-24T08:34:45.031 INFO:teuthology.orchestra.run.smithi189.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-24T08:34:45.031 INFO:teuthology.orchestra.run.smithi189.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-24T08:34:45.031 INFO:teuthology.orchestra.run.smithi189.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-24T08:34:45.031 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-24T08:34:45.031 INFO:teuthology.orchestra.run.smithi189.stdout: python3-ceph-argparse x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 45 k 2024-04-24T08:34:45.031 INFO:teuthology.orchestra.run.smithi189.stdout: python3-ceph-common x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 135 k 2024-04-24T08:34:45.031 INFO:teuthology.orchestra.run.smithi189.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-24T08:34:45.031 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-24T08:34:45.031 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-24T08:34:45.031 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-24T08:34:45.031 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-24T08:34:45.032 INFO:teuthology.orchestra.run.smithi189.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-24T08:34:45.032 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-24T08:34:45.032 INFO:teuthology.orchestra.run.smithi189.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-24T08:34:45.032 INFO:teuthology.orchestra.run.smithi189.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-24T08:34:45.032 INFO:teuthology.orchestra.run.smithi189.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-24T08:34:45.032 INFO:teuthology.orchestra.run.smithi189.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-24T08:34:45.032 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-24T08:34:45.032 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-24T08:34:45.032 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-24T08:34:45.032 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-24T08:34:45.032 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-24T08:34:45.032 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-24T08:34:45.032 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-24T08:34:45.033 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-24T08:34:45.033 INFO:teuthology.orchestra.run.smithi189.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-24T08:34:45.033 INFO:teuthology.orchestra.run.smithi189.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-24T08:34:45.033 INFO:teuthology.orchestra.run.smithi189.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-24T08:34:45.033 INFO:teuthology.orchestra.run.smithi189.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-24T08:34:45.033 INFO:teuthology.orchestra.run.smithi189.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-24T08:34:45.033 INFO:teuthology.orchestra.run.smithi189.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-24T08:34:45.033 INFO:teuthology.orchestra.run.smithi189.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-24T08:34:45.033 INFO:teuthology.orchestra.run.smithi189.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-24T08:34:45.033 INFO:teuthology.orchestra.run.smithi189.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-24T08:34:45.033 INFO:teuthology.orchestra.run.smithi189.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-24T08:34:45.033 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-24T08:34:45.033 INFO:teuthology.orchestra.run.smithi189.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-24T08:34:45.033 INFO:teuthology.orchestra.run.smithi189.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-24T08:34:45.034 INFO:teuthology.orchestra.run.smithi189.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-24T08:34:45.034 INFO:teuthology.orchestra.run.smithi189.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-24T08:34:45.034 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-24T08:34:45.034 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-24T08:34:45.034 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-24T08:34:45.034 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-24T08:34:45.034 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-24T08:34:45.034 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-24T08:34:45.034 INFO:teuthology.orchestra.run.smithi189.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-24T08:34:45.034 INFO:teuthology.orchestra.run.smithi189.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-24T08:34:45.034 INFO:teuthology.orchestra.run.smithi189.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-24T08:34:45.034 INFO:teuthology.orchestra.run.smithi189.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-24T08:34:45.034 INFO:teuthology.orchestra.run.smithi189.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-24T08:34:45.035 INFO:teuthology.orchestra.run.smithi189.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-24T08:34:45.035 INFO:teuthology.orchestra.run.smithi189.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-24T08:34:45.035 INFO:teuthology.orchestra.run.smithi189.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-24T08:34:45.035 INFO:teuthology.orchestra.run.smithi189.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-24T08:34:45.035 INFO:teuthology.orchestra.run.smithi189.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-24T08:34:45.035 INFO:teuthology.orchestra.run.smithi189.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-24T08:34:45.035 INFO:teuthology.orchestra.run.smithi189.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-24T08:34:45.035 INFO:teuthology.orchestra.run.smithi189.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-24T08:34:45.035 INFO:teuthology.orchestra.run.smithi189.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-24T08:34:45.035 INFO:teuthology.orchestra.run.smithi189.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-24T08:34:45.035 INFO:teuthology.orchestra.run.smithi189.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-24T08:34:45.035 INFO:teuthology.orchestra.run.smithi189.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-24T08:34:45.035 INFO:teuthology.orchestra.run.smithi189.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-24T08:34:45.035 INFO:teuthology.orchestra.run.smithi189.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-24T08:34:45.035 INFO:teuthology.orchestra.run.smithi189.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-24T08:34:45.036 INFO:teuthology.orchestra.run.smithi189.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-24T08:34:45.036 INFO:teuthology.orchestra.run.smithi189.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-24T08:34:45.036 INFO:teuthology.orchestra.run.smithi189.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-24T08:34:45.036 INFO:teuthology.orchestra.run.smithi189.stdout:Installing weak dependencies: 2024-04-24T08:34:45.036 INFO:teuthology.orchestra.run.smithi189.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-24T08:34:45.036 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-04-24T08:34:45.036 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction Summary 2024-04-24T08:34:45.036 INFO:teuthology.orchestra.run.smithi189.stdout:======================================================================================= 2024-04-24T08:34:45.036 INFO:teuthology.orchestra.run.smithi189.stdout:Install 148 Packages 2024-04-24T08:34:45.036 INFO:teuthology.orchestra.run.smithi189.stdout:Upgrade 4 Packages 2024-04-24T08:34:45.036 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-04-24T08:34:45.037 INFO:teuthology.orchestra.run.smithi189.stdout:Total download size: 222 M 2024-04-24T08:34:45.037 INFO:teuthology.orchestra.run.smithi189.stdout:Downloading Packages: 2024-04-24T08:34:45.522 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - Extras packages 44 kB/s | 16 kB 00:00 2024-04-24T08:34:45.862 INFO:teuthology.orchestra.run.smithi083.stdout:lab-extras 26 kB/s | 1.7 kB 00:00 2024-04-24T08:34:46.059 INFO:teuthology.orchestra.run.smithi189.stdout:(1/152): ceph-19.0.0-3116.g1f81fb39.el9.x86_64. 23 kB/s | 6.6 kB 00:00 2024-04-24T08:34:46.809 INFO:teuthology.orchestra.run.smithi189.stdout:(2/152): ceph-fuse-19.0.0-3116.g1f81fb39.el9.x8 1.2 MB/s | 890 kB 00:00 2024-04-24T08:34:47.090 INFO:teuthology.orchestra.run.smithi189.stdout:(3/152): ceph-immutable-object-cache-19.0.0-311 518 kB/s | 145 kB 00:00 2024-04-24T08:34:47.326 INFO:teuthology.orchestra.run.smithi189.stdout:(4/152): ceph-common-19.0.0-3116.g1f81fb39.el9. 14 MB/s | 21 MB 00:01 2024-04-24T08:34:47.479 INFO:teuthology.orchestra.run.smithi189.stdout:(5/152): ceph-base-19.0.0-3116.g1f81fb39.el9.x8 3.1 MB/s | 5.4 MB 00:01 2024-04-24T08:34:47.543 INFO:teuthology.orchestra.run.smithi189.stdout:(6/152): ceph-mgr-19.0.0-3116.g1f81fb39.el9.x86 6.9 MB/s | 1.5 MB 00:00 2024-04-24T08:34:47.549 INFO:teuthology.orchestra.run.smithi138.stdout:Extra Packages for Enterprise Linux 11 MB/s | 21 MB 00:01 2024-04-24T08:34:48.177 INFO:teuthology.orchestra.run.smithi189.stdout:(7/152): ceph-mon-19.0.0-3116.g1f81fb39.el9.x86 6.7 MB/s | 4.7 MB 00:00 2024-04-24T08:34:48.313 INFO:teuthology.orchestra.run.smithi083.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-24T08:34:48.315 INFO:teuthology.orchestra.run.smithi083.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-24T08:34:48.336 INFO:teuthology.orchestra.run.smithi083.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-24T08:34:48.338 INFO:teuthology.orchestra.run.smithi083.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-24T08:34:48.339 INFO:teuthology.orchestra.run.smithi083.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-24T08:34:48.340 INFO:teuthology.orchestra.run.smithi083.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-24T08:34:48.343 INFO:teuthology.orchestra.run.smithi083.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-24T08:34:48.346 INFO:teuthology.orchestra.run.smithi083.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-24T08:34:48.348 INFO:teuthology.orchestra.run.smithi083.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-24T08:34:48.419 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2024-04-24T08:34:48.443 INFO:teuthology.orchestra.run.smithi083.stdout:======================================================================================= 2024-04-24T08:34:48.444 INFO:teuthology.orchestra.run.smithi083.stdout: Package Arch Version Repository Size 2024-04-24T08:34:48.444 INFO:teuthology.orchestra.run.smithi083.stdout:======================================================================================= 2024-04-24T08:34:48.444 INFO:teuthology.orchestra.run.smithi083.stdout:Installing: 2024-04-24T08:34:48.444 INFO:teuthology.orchestra.run.smithi083.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-24T08:34:48.444 INFO:teuthology.orchestra.run.smithi083.stdout: ceph x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 6.6 k 2024-04-24T08:34:48.444 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-base x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 5.4 M 2024-04-24T08:34:48.444 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-fuse x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 890 k 2024-04-24T08:34:48.444 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 145 k 2024-04-24T08:34:48.444 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 1.5 M 2024-04-24T08:34:48.444 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3116.g1f81fb39.el9 ceph-noarch 142 k 2024-04-24T08:34:48.444 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3116.g1f81fb39.el9 ceph-noarch 3.5 M 2024-04-24T08:34:48.444 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3116.g1f81fb39.el9 ceph-noarch 7.4 M 2024-04-24T08:34:48.444 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-rook noarch 2:19.0.0-3116.g1f81fb39.el9 ceph-noarch 49 k 2024-04-24T08:34:48.444 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-radosgw x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 11 M 2024-04-24T08:34:48.445 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-test x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 49 M 2024-04-24T08:34:48.445 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-volume noarch 2:19.0.0-3116.g1f81fb39.el9 ceph-noarch 273 k 2024-04-24T08:34:48.445 INFO:teuthology.orchestra.run.smithi083.stdout: cephadm noarch 2:19.0.0-3116.g1f81fb39.el9 ceph-noarch 762 k 2024-04-24T08:34:48.445 INFO:teuthology.orchestra.run.smithi083.stdout: cephfs-mirror x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 225 k 2024-04-24T08:34:48.445 INFO:teuthology.orchestra.run.smithi083.stdout: cephfs-shell x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 27 k 2024-04-24T08:34:48.445 INFO:teuthology.orchestra.run.smithi083.stdout: cephfs-top noarch 2:19.0.0-3116.g1f81fb39.el9 ceph-noarch 18 k 2024-04-24T08:34:48.445 INFO:teuthology.orchestra.run.smithi083.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-24T08:34:48.445 INFO:teuthology.orchestra.run.smithi083.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-24T08:34:48.445 INFO:teuthology.orchestra.run.smithi083.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-24T08:34:48.445 INFO:teuthology.orchestra.run.smithi083.stdout: libcephfs-devel x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 32 k 2024-04-24T08:34:48.445 INFO:teuthology.orchestra.run.smithi083.stdout: libcephfs2 x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 746 k 2024-04-24T08:34:48.445 INFO:teuthology.orchestra.run.smithi083.stdout: librados-devel x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 127 k 2024-04-24T08:34:48.445 INFO:teuthology.orchestra.run.smithi083.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-24T08:34:48.446 INFO:teuthology.orchestra.run.smithi083.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-24T08:34:48.446 INFO:teuthology.orchestra.run.smithi083.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-24T08:34:48.446 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-24T08:34:48.446 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-24T08:34:48.446 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-24T08:34:48.446 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cephfs x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 161 k 2024-04-24T08:34:48.446 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rados x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 320 k 2024-04-24T08:34:48.446 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rbd x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 298 k 2024-04-24T08:34:48.446 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rgw x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 99 k 2024-04-24T08:34:48.446 INFO:teuthology.orchestra.run.smithi083.stdout: rbd-fuse x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 85 k 2024-04-24T08:34:48.446 INFO:teuthology.orchestra.run.smithi083.stdout: rbd-mirror x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 3.1 M 2024-04-24T08:34:48.446 INFO:teuthology.orchestra.run.smithi083.stdout: rbd-nbd x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 170 k 2024-04-24T08:34:48.446 INFO:teuthology.orchestra.run.smithi083.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-24T08:34:48.446 INFO:teuthology.orchestra.run.smithi083.stdout:Upgrading: 2024-04-24T08:34:48.447 INFO:teuthology.orchestra.run.smithi083.stdout: librados2 x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 3.4 M 2024-04-24T08:34:48.447 INFO:teuthology.orchestra.run.smithi083.stdout: librbd1 x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 3.1 M 2024-04-24T08:34:48.447 INFO:teuthology.orchestra.run.smithi083.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-24T08:34:48.447 INFO:teuthology.orchestra.run.smithi083.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-24T08:34:48.447 INFO:teuthology.orchestra.run.smithi083.stdout:Installing dependencies: 2024-04-24T08:34:48.447 INFO:teuthology.orchestra.run.smithi083.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-24T08:34:48.447 INFO:teuthology.orchestra.run.smithi083.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-24T08:34:48.447 INFO:teuthology.orchestra.run.smithi083.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-24T08:34:48.447 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-common x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 21 M 2024-04-24T08:34:48.447 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3116.g1f81fb39.el9 ceph-noarch 34 k 2024-04-24T08:34:48.447 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mds x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 2.4 M 2024-04-24T08:34:48.447 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3116.g1f81fb39.el9 ceph-noarch 249 k 2024-04-24T08:34:48.447 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mon x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 4.7 M 2024-04-24T08:34:48.447 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-osd x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 17 M 2024-04-24T08:34:48.447 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3116.g1f81fb39.el9 ceph-noarch 16 k 2024-04-24T08:34:48.448 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-selinux x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 25 k 2024-04-24T08:34:48.448 INFO:teuthology.orchestra.run.smithi083.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-24T08:34:48.448 INFO:teuthology.orchestra.run.smithi083.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-24T08:34:48.448 INFO:teuthology.orchestra.run.smithi083.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-24T08:34:48.448 INFO:teuthology.orchestra.run.smithi083.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-24T08:34:48.448 INFO:teuthology.orchestra.run.smithi083.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-24T08:34:48.448 INFO:teuthology.orchestra.run.smithi083.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-24T08:34:48.448 INFO:teuthology.orchestra.run.smithi083.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-24T08:34:48.448 INFO:teuthology.orchestra.run.smithi083.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-24T08:34:48.448 INFO:teuthology.orchestra.run.smithi083.stdout: libcephsqlite x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 163 k 2024-04-24T08:34:48.448 INFO:teuthology.orchestra.run.smithi083.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-24T08:34:48.448 INFO:teuthology.orchestra.run.smithi083.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-24T08:34:48.448 INFO:teuthology.orchestra.run.smithi083.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-24T08:34:48.448 INFO:teuthology.orchestra.run.smithi083.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-24T08:34:48.448 INFO:teuthology.orchestra.run.smithi083.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-24T08:34:48.449 INFO:teuthology.orchestra.run.smithi083.stdout: libradosstriper1 x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 501 k 2024-04-24T08:34:48.449 INFO:teuthology.orchestra.run.smithi083.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-24T08:34:48.449 INFO:teuthology.orchestra.run.smithi083.stdout: librgw2 x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 5.4 M 2024-04-24T08:34:48.449 INFO:teuthology.orchestra.run.smithi083.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-24T08:34:48.449 INFO:teuthology.orchestra.run.smithi083.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-24T08:34:48.449 INFO:teuthology.orchestra.run.smithi083.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-24T08:34:48.449 INFO:teuthology.orchestra.run.smithi083.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-24T08:34:48.449 INFO:teuthology.orchestra.run.smithi083.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-24T08:34:48.449 INFO:teuthology.orchestra.run.smithi083.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-24T08:34:48.449 INFO:teuthology.orchestra.run.smithi083.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-24T08:34:48.449 INFO:teuthology.orchestra.run.smithi083.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-24T08:34:48.449 INFO:teuthology.orchestra.run.smithi083.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-24T08:34:48.449 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-24T08:34:48.450 INFO:teuthology.orchestra.run.smithi083.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-24T08:34:48.450 INFO:teuthology.orchestra.run.smithi083.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-24T08:34:48.450 INFO:teuthology.orchestra.run.smithi083.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-24T08:34:48.450 INFO:teuthology.orchestra.run.smithi083.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-24T08:34:48.450 INFO:teuthology.orchestra.run.smithi083.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-24T08:34:48.450 INFO:teuthology.orchestra.run.smithi083.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-24T08:34:48.450 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-24T08:34:48.450 INFO:teuthology.orchestra.run.smithi083.stdout: python3-ceph-argparse x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 45 k 2024-04-24T08:34:48.450 INFO:teuthology.orchestra.run.smithi083.stdout: python3-ceph-common x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 135 k 2024-04-24T08:34:48.450 INFO:teuthology.orchestra.run.smithi083.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-24T08:34:48.450 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-24T08:34:48.450 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-24T08:34:48.450 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-24T08:34:48.451 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-24T08:34:48.451 INFO:teuthology.orchestra.run.smithi083.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-24T08:34:48.451 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-24T08:34:48.451 INFO:teuthology.orchestra.run.smithi083.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-24T08:34:48.451 INFO:teuthology.orchestra.run.smithi083.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-24T08:34:48.451 INFO:teuthology.orchestra.run.smithi083.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-24T08:34:48.451 INFO:teuthology.orchestra.run.smithi083.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-24T08:34:48.451 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-24T08:34:48.451 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-24T08:34:48.451 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-24T08:34:48.451 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-24T08:34:48.451 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-24T08:34:48.451 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-24T08:34:48.452 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-24T08:34:48.452 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-24T08:34:48.452 INFO:teuthology.orchestra.run.smithi083.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-24T08:34:48.452 INFO:teuthology.orchestra.run.smithi083.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-24T08:34:48.452 INFO:teuthology.orchestra.run.smithi083.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-24T08:34:48.452 INFO:teuthology.orchestra.run.smithi083.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-24T08:34:48.452 INFO:teuthology.orchestra.run.smithi083.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-24T08:34:48.452 INFO:teuthology.orchestra.run.smithi083.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-24T08:34:48.452 INFO:teuthology.orchestra.run.smithi083.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-24T08:34:48.452 INFO:teuthology.orchestra.run.smithi083.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-24T08:34:48.452 INFO:teuthology.orchestra.run.smithi083.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-24T08:34:48.452 INFO:teuthology.orchestra.run.smithi083.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-24T08:34:48.452 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-24T08:34:48.452 INFO:teuthology.orchestra.run.smithi083.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-24T08:34:48.453 INFO:teuthology.orchestra.run.smithi083.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-24T08:34:48.453 INFO:teuthology.orchestra.run.smithi083.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-24T08:34:48.453 INFO:teuthology.orchestra.run.smithi083.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-24T08:34:48.453 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-24T08:34:48.453 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-24T08:34:48.453 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-24T08:34:48.453 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-24T08:34:48.453 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-24T08:34:48.453 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-24T08:34:48.453 INFO:teuthology.orchestra.run.smithi083.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-24T08:34:48.453 INFO:teuthology.orchestra.run.smithi083.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-24T08:34:48.453 INFO:teuthology.orchestra.run.smithi083.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-24T08:34:48.453 INFO:teuthology.orchestra.run.smithi083.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-24T08:34:48.453 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-24T08:34:48.453 INFO:teuthology.orchestra.run.smithi083.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-24T08:34:48.453 INFO:teuthology.orchestra.run.smithi083.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-24T08:34:48.454 INFO:teuthology.orchestra.run.smithi083.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-24T08:34:48.454 INFO:teuthology.orchestra.run.smithi083.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-24T08:34:48.454 INFO:teuthology.orchestra.run.smithi083.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-24T08:34:48.454 INFO:teuthology.orchestra.run.smithi083.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-24T08:34:48.454 INFO:teuthology.orchestra.run.smithi083.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-24T08:34:48.454 INFO:teuthology.orchestra.run.smithi083.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-24T08:34:48.454 INFO:teuthology.orchestra.run.smithi083.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-24T08:34:48.454 INFO:teuthology.orchestra.run.smithi083.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-24T08:34:48.454 INFO:teuthology.orchestra.run.smithi083.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-24T08:34:48.454 INFO:teuthology.orchestra.run.smithi083.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-24T08:34:48.454 INFO:teuthology.orchestra.run.smithi083.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-24T08:34:48.454 INFO:teuthology.orchestra.run.smithi083.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-24T08:34:48.454 INFO:teuthology.orchestra.run.smithi083.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-24T08:34:48.455 INFO:teuthology.orchestra.run.smithi083.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-24T08:34:48.455 INFO:teuthology.orchestra.run.smithi083.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-24T08:34:48.455 INFO:teuthology.orchestra.run.smithi083.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-24T08:34:48.455 INFO:teuthology.orchestra.run.smithi083.stdout:Installing weak dependencies: 2024-04-24T08:34:48.455 INFO:teuthology.orchestra.run.smithi083.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-24T08:34:48.455 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:34:48.455 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction Summary 2024-04-24T08:34:48.455 INFO:teuthology.orchestra.run.smithi083.stdout:======================================================================================= 2024-04-24T08:34:48.455 INFO:teuthology.orchestra.run.smithi083.stdout:Install 148 Packages 2024-04-24T08:34:48.455 INFO:teuthology.orchestra.run.smithi083.stdout:Upgrade 4 Packages 2024-04-24T08:34:48.455 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:34:48.456 INFO:teuthology.orchestra.run.smithi083.stdout:Total download size: 222 M 2024-04-24T08:34:48.456 INFO:teuthology.orchestra.run.smithi083.stdout:Downloading Packages: 2024-04-24T08:34:48.569 INFO:teuthology.orchestra.run.smithi189.stdout:(8/152): ceph-osd-19.0.0-3116.g1f81fb39.el9.x86 16 MB/s | 17 MB 00:01 2024-04-24T08:34:48.636 INFO:teuthology.orchestra.run.smithi189.stdout:(9/152): ceph-selinux-19.0.0-3116.g1f81fb39.el9 384 kB/s | 25 kB 00:00 2024-04-24T08:34:48.769 INFO:teuthology.orchestra.run.smithi189.stdout:(10/152): ceph-mds-19.0.0-3116.g1f81fb39.el9.x8 1.4 MB/s | 2.4 MB 00:01 2024-04-24T08:34:49.170 INFO:teuthology.orchestra.run.smithi189.stdout:(11/152): ceph-radosgw-19.0.0-3116.g1f81fb39.el 11 MB/s | 11 MB 00:00 2024-04-24T08:34:49.203 INFO:teuthology.orchestra.run.smithi189.stdout:(12/152): cephfs-mirror-19.0.0-3116.g1f81fb39.e 519 kB/s | 225 kB 00:00 2024-04-24T08:34:49.254 INFO:teuthology.orchestra.run.smithi189.stdout:(13/152): cephfs-shell-19.0.0-3116.g1f81fb39.el 328 kB/s | 27 kB 00:00 2024-04-24T08:34:49.279 INFO:teuthology.orchestra.run.smithi189.stdout:(14/152): libcephfs-devel-19.0.0-3116.g1f81fb39 426 kB/s | 32 kB 00:00 2024-04-24T08:34:49.396 INFO:teuthology.orchestra.run.smithi189.stdout:(15/152): libcephfs2-19.0.0-3116.g1f81fb39.el9. 5.2 MB/s | 746 kB 00:00 2024-04-24T08:34:49.429 INFO:teuthology.orchestra.run.smithi189.stdout:(16/152): libcephsqlite-19.0.0-3116.g1f81fb39.e 1.1 MB/s | 163 kB 00:00 2024-04-24T08:34:49.471 INFO:teuthology.orchestra.run.smithi189.stdout:(17/152): librados-devel-19.0.0-3116.g1f81fb39. 1.7 MB/s | 127 kB 00:00 2024-04-24T08:34:49.689 INFO:teuthology.orchestra.run.smithi083.stdout:(1/152): ceph-19.0.0-3116.g1f81fb39.el9.x86_64. 16 kB/s | 6.6 kB 00:00 2024-04-24T08:34:49.938 INFO:teuthology.orchestra.run.smithi189.stdout:(18/152): librgw2-19.0.0-3116.g1f81fb39.el9.x86 12 MB/s | 5.4 MB 00:00 2024-04-24T08:34:49.971 INFO:teuthology.orchestra.run.smithi189.stdout:(19/152): libradosstriper1-19.0.0-3116.g1f81fb3 926 kB/s | 501 kB 00:00 2024-04-24T08:34:50.030 INFO:teuthology.orchestra.run.smithi189.stdout:(20/152): python3-ceph-argparse-19.0.0-3116.g1f 495 kB/s | 45 kB 00:00 2024-04-24T08:34:50.122 INFO:teuthology.orchestra.run.smithi189.stdout:(21/152): python3-cephfs-19.0.0-3116.g1f81fb39. 1.7 MB/s | 161 kB 00:00 2024-04-24T08:34:50.139 INFO:teuthology.orchestra.run.smithi189.stdout:(22/152): python3-ceph-common-19.0.0-3116.g1f81 810 kB/s | 135 kB 00:00 2024-04-24T08:34:50.222 INFO:teuthology.orchestra.run.smithi189.stdout:(23/152): python3-rados-19.0.0-3116.g1f81fb39.e 3.1 MB/s | 320 kB 00:00 2024-04-24T08:34:50.339 INFO:teuthology.orchestra.run.smithi189.stdout:(24/152): python3-rgw-19.0.0-3116.g1f81fb39.el9 858 kB/s | 99 kB 00:00 2024-04-24T08:34:50.390 INFO:teuthology.orchestra.run.smithi189.stdout:(25/152): python3-rbd-19.0.0-3116.g1f81fb39.el9 1.2 MB/s | 298 kB 00:00 2024-04-24T08:34:50.414 INFO:teuthology.orchestra.run.smithi083.stdout:(2/152): ceph-fuse-19.0.0-3116.g1f81fb39.el9.x8 1.2 MB/s | 890 kB 00:00 2024-04-24T08:34:50.440 INFO:teuthology.orchestra.run.smithi189.stdout:(26/152): rbd-fuse-19.0.0-3116.g1f81fb39.el9.x8 856 kB/s | 85 kB 00:00 2024-04-24T08:34:50.549 INFO:teuthology.orchestra.run.smithi189.stdout:(27/152): rbd-nbd-19.0.0-3116.g1f81fb39.el9.x86 1.5 MB/s | 170 kB 00:00 2024-04-24T08:34:50.572 INFO:teuthology.orchestra.run.smithi083.stdout:(3/152): ceph-immutable-object-cache-19.0.0-311 920 kB/s | 145 kB 00:00 2024-04-24T08:34:50.641 INFO:teuthology.orchestra.run.smithi189.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3116.g 377 kB/s | 34 kB 00:00 2024-04-24T08:34:50.758 INFO:teuthology.orchestra.run.smithi189.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3116.g1f81fb3 1.2 MB/s | 142 kB 00:00 2024-04-24T08:34:51.625 INFO:teuthology.orchestra.run.smithi189.stdout:(30/152): ceph-mgr-dashboard-19.0.0-3116.g1f81f 4.1 MB/s | 3.5 MB 00:00 2024-04-24T08:34:51.756 INFO:teuthology.orchestra.run.smithi083.stdout:(4/152): ceph-base-19.0.0-3116.g1f81fb39.el9.x8 2.2 MB/s | 5.4 MB 00:02 2024-04-24T08:34:52.114 INFO:teuthology.orchestra.run.smithi083.stdout:(5/152): ceph-mds-19.0.0-3116.g1f81fb39.el9.x86 1.5 MB/s | 2.4 MB 00:01 2024-04-24T08:34:52.223 INFO:teuthology.orchestra.run.smithi083.stdout:(6/152): ceph-mgr-19.0.0-3116.g1f81fb39.el9.x86 3.2 MB/s | 1.5 MB 00:00 2024-04-24T08:34:52.875 INFO:teuthology.orchestra.run.smithi189.stdout:(31/152): ceph-test-19.0.0-3116.g1f81fb39.el9.x 12 MB/s | 49 MB 00:04 2024-04-24T08:34:52.890 INFO:teuthology.orchestra.run.smithi083.stdout:(7/152): ceph-mon-19.0.0-3116.g1f81fb39.el9.x86 6.1 MB/s | 4.7 MB 00:00 2024-04-24T08:34:53.025 INFO:teuthology.orchestra.run.smithi189.stdout:(32/152): ceph-mgr-diskprediction-local-19.0.0- 5.3 MB/s | 7.4 MB 00:01 2024-04-24T08:34:53.067 INFO:teuthology.orchestra.run.smithi189.stdout:(33/152): ceph-mgr-modules-core-19.0.0-3116.g1f 1.3 MB/s | 249 kB 00:00 2024-04-24T08:34:53.092 INFO:teuthology.orchestra.run.smithi189.stdout:(34/152): ceph-mgr-rook-19.0.0-3116.g1f81fb39.e 750 kB/s | 49 kB 00:00 2024-04-24T08:34:53.117 INFO:teuthology.orchestra.run.smithi189.stdout:(35/152): ceph-prometheus-alerts-19.0.0-3116.g1 331 kB/s | 16 kB 00:00 2024-04-24T08:34:53.157 INFO:teuthology.orchestra.run.smithi083.stdout:(8/152): ceph-common-19.0.0-3116.g1f81fb39.el9. 5.4 MB/s | 21 MB 00:03 2024-04-24T08:34:53.199 INFO:teuthology.orchestra.run.smithi083.stdout:(9/152): ceph-selinux-19.0.0-3116.g1f81fb39.el9 622 kB/s | 25 kB 00:00 2024-04-24T08:34:53.226 INFO:teuthology.orchestra.run.smithi189.stdout:(36/152): cephadm-19.0.0-3116.g1f81fb39.el9.noa 6.9 MB/s | 762 kB 00:00 2024-04-24T08:34:53.260 INFO:teuthology.orchestra.run.smithi189.stdout:(37/152): ceph-volume-19.0.0-3116.g1f81fb39.el9 1.6 MB/s | 273 kB 00:00 2024-04-24T08:34:53.285 INFO:teuthology.orchestra.run.smithi189.stdout:(38/152): cephfs-top-19.0.0-3116.g1f81fb39.el9. 316 kB/s | 18 kB 00:00 2024-04-24T08:34:53.485 INFO:teuthology.orchestra.run.smithi189.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 358 kB/s | 80 kB 00:00 2024-04-24T08:34:53.619 INFO:teuthology.orchestra.run.smithi189.stdout:(40/152): rbd-mirror-19.0.0-3116.g1f81fb39.el9. 980 kB/s | 3.1 MB 00:03 2024-04-24T08:34:53.675 INFO:teuthology.orchestra.run.smithi138.stdout:lab-extras 19 kB/s | 1.7 kB 00:00 2024-04-24T08:34:53.677 INFO:teuthology.orchestra.run.smithi189.stdout:(41/152): libquadmath-11.4.1-3.el9.x86_64.rpm 1.0 MB/s | 195 kB 00:00 2024-04-24T08:34:53.778 INFO:teuthology.orchestra.run.smithi189.stdout:(42/152): libgfortran-11.4.1-3.el9.x86_64.rpm 1.6 MB/s | 803 kB 00:00 2024-04-24T08:34:53.828 INFO:teuthology.orchestra.run.smithi189.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-04-24T08:34:53.903 INFO:teuthology.orchestra.run.smithi189.stdout:(44/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 894 kB/s | 253 kB 00:00 2024-04-24T08:34:53.907 INFO:teuthology.orchestra.run.smithi083.stdout:(10/152): ceph-radosgw-19.0.0-3116.g1f81fb39.el 10 MB/s | 11 MB 00:01 2024-04-24T08:34:53.962 INFO:teuthology.orchestra.run.smithi189.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 1.0 MB/s | 135 kB 00:00 2024-04-24T08:34:53.966 INFO:teuthology.orchestra.run.smithi083.stdout:(11/152): cephfs-mirror-19.0.0-3116.g1f81fb39.e 3.8 MB/s | 225 kB 00:00 2024-04-24T08:34:53.987 INFO:teuthology.orchestra.run.smithi189.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 1.5 MB/s | 125 kB 00:00 2024-04-24T08:34:54.016 INFO:teuthology.orchestra.run.smithi083.stdout:(12/152): cephfs-shell-19.0.0-3116.g1f81fb39.el 549 kB/s | 27 kB 00:00 2024-04-24T08:34:54.058 INFO:teuthology.orchestra.run.smithi083.stdout:(13/152): libcephfs-devel-19.0.0-3116.g1f81fb39 776 kB/s | 32 kB 00:00 2024-04-24T08:34:54.079 INFO:teuthology.orchestra.run.smithi189.stdout:(47/152): python3-cryptography-36.0.1-4.el9.x86 3.1 MB/s | 1.2 MB 00:00 2024-04-24T08:34:54.104 INFO:teuthology.orchestra.run.smithi189.stdout:(48/152): python3-urllib3-1.26.5-5.el9.noarch.r 1.5 MB/s | 215 kB 00:00 2024-04-24T08:34:54.150 INFO:teuthology.orchestra.run.smithi083.stdout:(14/152): libcephfs2-19.0.0-3116.g1f81fb39.el9. 8.1 MB/s | 746 kB 00:00 2024-04-24T08:34:54.217 INFO:teuthology.orchestra.run.smithi083.stdout:(15/152): libcephsqlite-19.0.0-3116.g1f81fb39.e 2.4 MB/s | 163 kB 00:00 2024-04-24T08:34:54.238 INFO:teuthology.orchestra.run.smithi189.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 807 kB/s | 107 kB 00:00 2024-04-24T08:34:54.284 INFO:teuthology.orchestra.run.smithi083.stdout:(16/152): librados-devel-19.0.0-3116.g1f81fb39. 1.9 MB/s | 127 kB 00:00 2024-04-24T08:34:54.288 INFO:teuthology.orchestra.run.smithi189.stdout:(50/152): automake-1.16.2-8.el9.noarch.rpm 2.2 MB/s | 689 kB 00:00 2024-04-24T08:34:54.305 INFO:teuthology.orchestra.run.smithi189.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 325 kB/s | 21 kB 00:00 2024-04-24T08:34:54.347 INFO:teuthology.orchestra.run.smithi189.stdout:(52/152): bison-3.7.4-5.el9.x86_64.rpm 3.5 MB/s | 943 kB 00:00 2024-04-24T08:34:54.372 INFO:teuthology.orchestra.run.smithi189.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 493 kB/s | 33 kB 00:00 2024-04-24T08:34:54.390 INFO:teuthology.orchestra.run.smithi083.stdout:(17/152): libradosstriper1-19.0.0-3116.g1f81fb3 4.7 MB/s | 501 kB 00:00 2024-04-24T08:34:54.414 INFO:teuthology.orchestra.run.smithi189.stdout:(54/152): flex-2.6.4-9.el9.x86_64.rpm 2.5 MB/s | 313 kB 00:00 2024-04-24T08:34:54.473 INFO:teuthology.orchestra.run.smithi189.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 183 kB/s | 18 kB 00:00 2024-04-24T08:34:54.523 INFO:teuthology.orchestra.run.smithi189.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 103 kB/s | 11 kB 00:00 2024-04-24T08:34:54.590 INFO:teuthology.orchestra.run.smithi189.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.3 MB/s | 160 kB 00:00 2024-04-24T08:34:54.682 INFO:teuthology.orchestra.run.smithi189.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 286 kB/s | 45 kB 00:00 2024-04-24T08:34:54.757 INFO:teuthology.orchestra.run.smithi189.stdout:(59/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 7.3 MB/s | 3.0 MB 00:00 2024-04-24T08:34:54.808 INFO:teuthology.orchestra.run.smithi189.stdout:(60/152): librdkafka-1.6.1-102.el9.x86_64.rpm 3.0 MB/s | 662 kB 00:00 2024-04-24T08:34:54.826 INFO:teuthology.orchestra.run.smithi083.stdout:(18/152): librgw2-19.0.0-3116.g1f81fb39.el9.x86 12 MB/s | 5.4 MB 00:00 2024-04-24T08:34:54.833 INFO:teuthology.orchestra.run.smithi189.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 342 kB/s | 25 kB 00:00 2024-04-24T08:34:54.858 INFO:teuthology.orchestra.run.smithi189.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 961 kB/s | 47 kB 00:00 2024-04-24T08:34:54.877 INFO:teuthology.orchestra.run.smithi083.stdout:(19/152): python3-ceph-argparse-19.0.0-3116.g1f 923 kB/s | 45 kB 00:00 2024-04-24T08:34:54.892 INFO:teuthology.orchestra.run.smithi189.stdout:(63/152): libtool-2.4.6-45.el9.x86_64.rpm 2.7 MB/s | 582 kB 00:00 2024-04-24T08:34:54.925 INFO:teuthology.orchestra.run.smithi189.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 2.8 MB/s | 188 kB 00:00 2024-04-24T08:34:54.951 INFO:teuthology.orchestra.run.smithi189.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 610 kB/s | 35 kB 00:00 2024-04-24T08:34:54.984 INFO:teuthology.orchestra.run.smithi189.stdout:(66/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.9 MB/s | 292 kB 00:00 2024-04-24T08:34:55.101 INFO:teuthology.orchestra.run.smithi189.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-04-24T08:34:55.102 INFO:teuthology.orchestra.run.smithi083.stdout:(20/152): ceph-osd-19.0.0-3116.g1f81fb39.el9.x8 5.8 MB/s | 17 MB 00:02 2024-04-24T08:34:55.135 INFO:teuthology.orchestra.run.smithi083.stdout:(21/152): python3-ceph-common-19.0.0-3116.g1f81 523 kB/s | 135 kB 00:00 2024-04-24T08:34:55.169 INFO:teuthology.orchestra.run.smithi083.stdout:(22/152): python3-cephfs-19.0.0-3116.g1f81fb39. 2.4 MB/s | 161 kB 00:00 2024-04-24T08:34:55.210 INFO:teuthology.orchestra.run.smithi189.stdout:(68/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 17 MB/s | 4.4 MB 00:00 2024-04-24T08:34:55.219 INFO:teuthology.orchestra.run.smithi083.stdout:(23/152): python3-rados-19.0.0-3116.g1f81fb39.e 3.8 MB/s | 320 kB 00:00 2024-04-24T08:34:55.270 INFO:teuthology.orchestra.run.smithi083.stdout:(24/152): python3-rbd-19.0.0-3116.g1f81fb39.el9 2.9 MB/s | 298 kB 00:00 2024-04-24T08:34:55.303 INFO:teuthology.orchestra.run.smithi083.stdout:(25/152): python3-rgw-19.0.0-3116.g1f81fb39.el9 1.2 MB/s | 99 kB 00:00 2024-04-24T08:34:55.310 INFO:teuthology.orchestra.run.smithi189.stdout:(69/152): openblas-openmp-0.3.21-2.el9.x86_64.r 12 MB/s | 4.7 MB 00:00 2024-04-24T08:34:55.320 INFO:teuthology.orchestra.run.smithi083.stdout:(26/152): rbd-fuse-19.0.0-3116.g1f81fb39.el9.x8 1.7 MB/s | 85 kB 00:00 2024-04-24T08:34:55.344 INFO:teuthology.orchestra.run.smithi189.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 1.0 MB/s | 133 kB 00:00 2024-04-24T08:34:55.394 INFO:teuthology.orchestra.run.smithi189.stdout:(71/152): postgresql-contrib-13.14-1.el9.x86_64 3.0 MB/s | 911 kB 00:00 2024-04-24T08:34:55.396 INFO:teuthology.orchestra.run.smithi083.stdout:(27/152): rbd-nbd-19.0.0-3116.g1f81fb39.el9.x86 2.2 MB/s | 170 kB 00:00 2024-04-24T08:34:55.479 INFO:teuthology.orchestra.run.smithi083.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3116.g 415 kB/s | 34 kB 00:00 2024-04-24T08:34:55.555 INFO:teuthology.orchestra.run.smithi083.stdout:(29/152): rbd-mirror-19.0.0-3116.g1f81fb39.el9. 12 MB/s | 3.1 MB 00:00 2024-04-24T08:34:55.613 INFO:teuthology.orchestra.run.smithi083.stdout:(30/152): ceph-mgr-cephadm-19.0.0-3116.g1f81fb3 1.0 MB/s | 142 kB 00:00 2024-04-24T08:34:55.669 INFO:teuthology.orchestra.run.smithi189.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 318 kB/s | 87 kB 00:00 2024-04-24T08:34:55.711 INFO:teuthology.orchestra.run.smithi189.stdout:(73/152): protobuf-3.14.0-13.el9.x86_64.rpm 2.8 MB/s | 1.0 MB 00:00 2024-04-24T08:34:55.797 INFO:teuthology.orchestra.run.smithi083.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3116.g1f81f 15 MB/s | 3.5 MB 00:00 2024-04-24T08:34:55.864 INFO:teuthology.orchestra.run.smithi083.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3116.g1f 3.7 MB/s | 249 kB 00:00 2024-04-24T08:34:55.922 INFO:teuthology.orchestra.run.smithi083.stdout:(33/152): ceph-mgr-rook-19.0.0-3116.g1f81fb39.e 860 kB/s | 49 kB 00:00 2024-04-24T08:34:55.936 INFO:teuthology.orchestra.run.smithi189.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 1.1 MB/s | 244 kB 00:00 2024-04-24T08:34:55.998 INFO:teuthology.orchestra.run.smithi083.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3116.g1 222 kB/s | 16 kB 00:00 2024-04-24T08:34:56.007 INFO:teuthology.orchestra.run.smithi138.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-24T08:34:56.008 INFO:teuthology.orchestra.run.smithi138.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-24T08:34:56.027 INFO:teuthology.orchestra.run.smithi138.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-24T08:34:56.028 INFO:teuthology.orchestra.run.smithi138.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-24T08:34:56.030 INFO:teuthology.orchestra.run.smithi138.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-24T08:34:56.034 INFO:teuthology.orchestra.run.smithi138.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-24T08:34:56.034 INFO:teuthology.orchestra.run.smithi138.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-24T08:34:56.036 INFO:teuthology.orchestra.run.smithi138.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-24T08:34:56.037 INFO:teuthology.orchestra.run.smithi138.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-24T08:34:56.053 INFO:teuthology.orchestra.run.smithi189.stdout:(75/152): postgresql-server-13.14-1.el9.x86_64. 7.8 MB/s | 5.8 MB 00:00 2024-04-24T08:34:56.056 INFO:teuthology.orchestra.run.smithi083.stdout:(35/152): ceph-volume-19.0.0-3116.g1f81fb39.el9 4.6 MB/s | 273 kB 00:00 2024-04-24T08:34:56.107 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-24T08:34:56.112 INFO:teuthology.orchestra.run.smithi189.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 1.4 MB/s | 248 kB 00:00 2024-04-24T08:34:56.128 INFO:teuthology.orchestra.run.smithi138.stdout:======================================================================================= 2024-04-24T08:34:56.128 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repository Size 2024-04-24T08:34:56.128 INFO:teuthology.orchestra.run.smithi138.stdout:======================================================================================= 2024-04-24T08:34:56.129 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-04-24T08:34:56.129 INFO:teuthology.orchestra.run.smithi138.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-24T08:34:56.129 INFO:teuthology.orchestra.run.smithi138.stdout: ceph x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 6.6 k 2024-04-24T08:34:56.129 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-base x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 5.4 M 2024-04-24T08:34:56.129 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-fuse x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 890 k 2024-04-24T08:34:56.129 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 145 k 2024-04-24T08:34:56.129 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 1.5 M 2024-04-24T08:34:56.129 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3116.g1f81fb39.el9 ceph-noarch 142 k 2024-04-24T08:34:56.129 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3116.g1f81fb39.el9 ceph-noarch 3.5 M 2024-04-24T08:34:56.129 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3116.g1f81fb39.el9 ceph-noarch 7.4 M 2024-04-24T08:34:56.129 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-rook noarch 2:19.0.0-3116.g1f81fb39.el9 ceph-noarch 49 k 2024-04-24T08:34:56.129 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-radosgw x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 11 M 2024-04-24T08:34:56.129 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-test x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 49 M 2024-04-24T08:34:56.129 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-volume noarch 2:19.0.0-3116.g1f81fb39.el9 ceph-noarch 273 k 2024-04-24T08:34:56.129 INFO:teuthology.orchestra.run.smithi138.stdout: cephadm noarch 2:19.0.0-3116.g1f81fb39.el9 ceph-noarch 762 k 2024-04-24T08:34:56.130 INFO:teuthology.orchestra.run.smithi138.stdout: cephfs-mirror x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 225 k 2024-04-24T08:34:56.130 INFO:teuthology.orchestra.run.smithi138.stdout: cephfs-shell x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 27 k 2024-04-24T08:34:56.130 INFO:teuthology.orchestra.run.smithi138.stdout: cephfs-top noarch 2:19.0.0-3116.g1f81fb39.el9 ceph-noarch 18 k 2024-04-24T08:34:56.130 INFO:teuthology.orchestra.run.smithi138.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-24T08:34:56.130 INFO:teuthology.orchestra.run.smithi138.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-24T08:34:56.130 INFO:teuthology.orchestra.run.smithi138.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-24T08:34:56.130 INFO:teuthology.orchestra.run.smithi138.stdout: libcephfs-devel x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 32 k 2024-04-24T08:34:56.130 INFO:teuthology.orchestra.run.smithi138.stdout: libcephfs2 x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 746 k 2024-04-24T08:34:56.130 INFO:teuthology.orchestra.run.smithi138.stdout: librados-devel x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 127 k 2024-04-24T08:34:56.130 INFO:teuthology.orchestra.run.smithi138.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-24T08:34:56.130 INFO:teuthology.orchestra.run.smithi138.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-24T08:34:56.130 INFO:teuthology.orchestra.run.smithi138.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-24T08:34:56.131 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-24T08:34:56.131 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-24T08:34:56.131 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-24T08:34:56.131 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cephfs x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 161 k 2024-04-24T08:34:56.131 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rados x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 320 k 2024-04-24T08:34:56.131 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rbd x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 298 k 2024-04-24T08:34:56.131 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rgw x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 99 k 2024-04-24T08:34:56.131 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-fuse x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 85 k 2024-04-24T08:34:56.131 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-mirror x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 3.1 M 2024-04-24T08:34:56.131 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-nbd x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 170 k 2024-04-24T08:34:56.131 INFO:teuthology.orchestra.run.smithi138.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-24T08:34:56.131 INFO:teuthology.orchestra.run.smithi138.stdout:Upgrading: 2024-04-24T08:34:56.131 INFO:teuthology.orchestra.run.smithi138.stdout: librados2 x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 3.4 M 2024-04-24T08:34:56.131 INFO:teuthology.orchestra.run.smithi138.stdout: librbd1 x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 3.1 M 2024-04-24T08:34:56.131 INFO:teuthology.orchestra.run.smithi138.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-24T08:34:56.132 INFO:teuthology.orchestra.run.smithi138.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-24T08:34:56.132 INFO:teuthology.orchestra.run.smithi138.stdout:Installing dependencies: 2024-04-24T08:34:56.132 INFO:teuthology.orchestra.run.smithi138.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-24T08:34:56.132 INFO:teuthology.orchestra.run.smithi138.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-24T08:34:56.132 INFO:teuthology.orchestra.run.smithi138.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-24T08:34:56.132 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-common x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 21 M 2024-04-24T08:34:56.132 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3116.g1f81fb39.el9 ceph-noarch 34 k 2024-04-24T08:34:56.132 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mds x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 2.4 M 2024-04-24T08:34:56.132 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3116.g1f81fb39.el9 ceph-noarch 249 k 2024-04-24T08:34:56.132 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mon x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 4.7 M 2024-04-24T08:34:56.132 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-osd x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 17 M 2024-04-24T08:34:56.132 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3116.g1f81fb39.el9 ceph-noarch 16 k 2024-04-24T08:34:56.132 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-selinux x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 25 k 2024-04-24T08:34:56.132 INFO:teuthology.orchestra.run.smithi138.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-24T08:34:56.132 INFO:teuthology.orchestra.run.smithi138.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-24T08:34:56.132 INFO:teuthology.orchestra.run.smithi138.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-24T08:34:56.133 INFO:teuthology.orchestra.run.smithi138.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-24T08:34:56.133 INFO:teuthology.orchestra.run.smithi138.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-24T08:34:56.133 INFO:teuthology.orchestra.run.smithi138.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-24T08:34:56.133 INFO:teuthology.orchestra.run.smithi138.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-24T08:34:56.133 INFO:teuthology.orchestra.run.smithi138.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-24T08:34:56.133 INFO:teuthology.orchestra.run.smithi138.stdout: libcephsqlite x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 163 k 2024-04-24T08:34:56.133 INFO:teuthology.orchestra.run.smithi138.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-24T08:34:56.133 INFO:teuthology.orchestra.run.smithi138.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-24T08:34:56.133 INFO:teuthology.orchestra.run.smithi138.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-24T08:34:56.133 INFO:teuthology.orchestra.run.smithi138.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-24T08:34:56.133 INFO:teuthology.orchestra.run.smithi138.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-24T08:34:56.133 INFO:teuthology.orchestra.run.smithi138.stdout: libradosstriper1 x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 501 k 2024-04-24T08:34:56.133 INFO:teuthology.orchestra.run.smithi138.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-24T08:34:56.133 INFO:teuthology.orchestra.run.smithi138.stdout: librgw2 x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 5.4 M 2024-04-24T08:34:56.133 INFO:teuthology.orchestra.run.smithi138.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-24T08:34:56.133 INFO:teuthology.orchestra.run.smithi138.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-24T08:34:56.134 INFO:teuthology.orchestra.run.smithi138.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-24T08:34:56.134 INFO:teuthology.orchestra.run.smithi138.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-24T08:34:56.134 INFO:teuthology.orchestra.run.smithi138.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-24T08:34:56.134 INFO:teuthology.orchestra.run.smithi138.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-24T08:34:56.134 INFO:teuthology.orchestra.run.smithi138.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-24T08:34:56.134 INFO:teuthology.orchestra.run.smithi138.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-24T08:34:56.134 INFO:teuthology.orchestra.run.smithi138.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-24T08:34:56.134 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-24T08:34:56.134 INFO:teuthology.orchestra.run.smithi138.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-24T08:34:56.134 INFO:teuthology.orchestra.run.smithi138.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-24T08:34:56.134 INFO:teuthology.orchestra.run.smithi138.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-24T08:34:56.134 INFO:teuthology.orchestra.run.smithi138.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-24T08:34:56.134 INFO:teuthology.orchestra.run.smithi138.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-24T08:34:56.134 INFO:teuthology.orchestra.run.smithi138.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-24T08:34:56.135 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-24T08:34:56.135 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ceph-argparse x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 45 k 2024-04-24T08:34:56.135 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ceph-common x86_64 2:19.0.0-3116.g1f81fb39.el9 ceph 135 k 2024-04-24T08:34:56.135 INFO:teuthology.orchestra.run.smithi138.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-24T08:34:56.135 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-24T08:34:56.135 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-24T08:34:56.135 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-24T08:34:56.135 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-24T08:34:56.135 INFO:teuthology.orchestra.run.smithi138.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-24T08:34:56.135 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-24T08:34:56.135 INFO:teuthology.orchestra.run.smithi138.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-24T08:34:56.135 INFO:teuthology.orchestra.run.smithi138.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-24T08:34:56.136 INFO:teuthology.orchestra.run.smithi138.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-24T08:34:56.136 INFO:teuthology.orchestra.run.smithi138.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-24T08:34:56.136 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-24T08:34:56.136 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-24T08:34:56.136 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-24T08:34:56.136 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-24T08:34:56.136 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-24T08:34:56.136 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-24T08:34:56.136 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-24T08:34:56.136 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-24T08:34:56.136 INFO:teuthology.orchestra.run.smithi138.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-24T08:34:56.136 INFO:teuthology.orchestra.run.smithi138.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-24T08:34:56.137 INFO:teuthology.orchestra.run.smithi138.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-24T08:34:56.137 INFO:teuthology.orchestra.run.smithi138.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-24T08:34:56.137 INFO:teuthology.orchestra.run.smithi138.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-24T08:34:56.137 INFO:teuthology.orchestra.run.smithi138.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-24T08:34:56.137 INFO:teuthology.orchestra.run.smithi138.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-24T08:34:56.137 INFO:teuthology.orchestra.run.smithi138.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-24T08:34:56.137 INFO:teuthology.orchestra.run.smithi138.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-24T08:34:56.137 INFO:teuthology.orchestra.run.smithi138.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-24T08:34:56.137 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-24T08:34:56.137 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-24T08:34:56.137 INFO:teuthology.orchestra.run.smithi138.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-24T08:34:56.137 INFO:teuthology.orchestra.run.smithi138.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-24T08:34:56.138 INFO:teuthology.orchestra.run.smithi138.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-24T08:34:56.138 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-24T08:34:56.138 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-24T08:34:56.138 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-24T08:34:56.138 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-24T08:34:56.138 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-24T08:34:56.138 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-24T08:34:56.138 INFO:teuthology.orchestra.run.smithi138.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-24T08:34:56.138 INFO:teuthology.orchestra.run.smithi138.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-24T08:34:56.138 INFO:teuthology.orchestra.run.smithi138.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-24T08:34:56.138 INFO:teuthology.orchestra.run.smithi138.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-24T08:34:56.138 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-24T08:34:56.139 INFO:teuthology.orchestra.run.smithi138.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-24T08:34:56.139 INFO:teuthology.orchestra.run.smithi138.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-24T08:34:56.139 INFO:teuthology.orchestra.run.smithi138.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-24T08:34:56.139 INFO:teuthology.orchestra.run.smithi138.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-24T08:34:56.139 INFO:teuthology.orchestra.run.smithi138.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-24T08:34:56.139 INFO:teuthology.orchestra.run.smithi138.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-24T08:34:56.139 INFO:teuthology.orchestra.run.smithi138.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-24T08:34:56.139 INFO:teuthology.orchestra.run.smithi138.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-24T08:34:56.139 INFO:teuthology.orchestra.run.smithi138.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-24T08:34:56.139 INFO:teuthology.orchestra.run.smithi138.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-24T08:34:56.139 INFO:teuthology.orchestra.run.smithi138.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-24T08:34:56.139 INFO:teuthology.orchestra.run.smithi138.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-24T08:34:56.139 INFO:teuthology.orchestra.run.smithi138.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-24T08:34:56.139 INFO:teuthology.orchestra.run.smithi138.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-24T08:34:56.140 INFO:teuthology.orchestra.run.smithi138.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-24T08:34:56.140 INFO:teuthology.orchestra.run.smithi138.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-24T08:34:56.140 INFO:teuthology.orchestra.run.smithi138.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-24T08:34:56.140 INFO:teuthology.orchestra.run.smithi138.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-24T08:34:56.140 INFO:teuthology.orchestra.run.smithi138.stdout:Installing weak dependencies: 2024-04-24T08:34:56.140 INFO:teuthology.orchestra.run.smithi138.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-24T08:34:56.140 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-24T08:34:56.140 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-04-24T08:34:56.140 INFO:teuthology.orchestra.run.smithi138.stdout:======================================================================================= 2024-04-24T08:34:56.140 INFO:teuthology.orchestra.run.smithi138.stdout:Install 148 Packages 2024-04-24T08:34:56.140 INFO:teuthology.orchestra.run.smithi138.stdout:Upgrade 4 Packages 2024-04-24T08:34:56.140 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-24T08:34:56.141 INFO:teuthology.orchestra.run.smithi189.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 311 kB/s | 26 kB 00:00 2024-04-24T08:34:56.141 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 222 M 2024-04-24T08:34:56.141 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-04-24T08:34:56.165 INFO:teuthology.orchestra.run.smithi083.stdout:(36/152): cephadm-19.0.0-3116.g1f81fb39.el9.noa 6.9 MB/s | 762 kB 00:00 2024-04-24T08:34:56.171 INFO:teuthology.orchestra.run.smithi189.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 332 kB/s | 19 kB 00:00 2024-04-24T08:34:56.204 INFO:teuthology.orchestra.run.smithi189.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 2.5 MB/s | 172 kB 00:00 2024-04-24T08:34:56.215 INFO:teuthology.orchestra.run.smithi083.stdout:(37/152): cephfs-top-19.0.0-3116.g1f81fb39.el9. 371 kB/s | 18 kB 00:00 2024-04-24T08:34:56.230 INFO:teuthology.orchestra.run.smithi189.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 602 kB/s | 35 kB 00:00 2024-04-24T08:34:56.332 INFO:teuthology.orchestra.run.smithi083.stdout:(38/152): fuse-2.9.9-15.el9.x86_64.rpm 694 kB/s | 80 kB 00:00 2024-04-24T08:34:56.380 INFO:teuthology.orchestra.run.smithi189.stdout:(81/152): python3-babel-2.9.1-2.el9.noarch.rpm 8.4 MB/s | 6.0 MB 00:00 2024-04-24T08:34:56.413 INFO:teuthology.orchestra.run.smithi189.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.3 MB/s | 250 kB 00:00 2024-04-24T08:34:56.449 INFO:teuthology.orchestra.run.smithi083.stdout:(39/152): libgfortran-11.4.1-3.el9.x86_64.rpm 6.8 MB/s | 803 kB 00:00 2024-04-24T08:34:56.455 INFO:teuthology.orchestra.run.smithi189.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 2.9 MB/s | 222 kB 00:00 2024-04-24T08:34:56.480 INFO:teuthology.orchestra.run.smithi189.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 1.1 MB/s | 77 kB 00:00 2024-04-24T08:34:56.514 INFO:teuthology.orchestra.run.smithi189.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 733 kB/s | 42 kB 00:00 2024-04-24T08:34:56.591 INFO:teuthology.orchestra.run.smithi083.stdout:(40/152): ceph-mgr-diskprediction-local-19.0.0- 7.6 MB/s | 7.4 MB 00:00 2024-04-24T08:34:56.625 INFO:teuthology.orchestra.run.smithi083.stdout:(41/152): libquadmath-11.4.1-3.el9.x86_64.rpm 1.1 MB/s | 195 kB 00:00 2024-04-24T08:34:56.631 INFO:teuthology.orchestra.run.smithi189.stdout:(86/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 12 MB/s | 5.1 MB 00:00 2024-04-24T08:34:56.665 INFO:teuthology.orchestra.run.smithi189.stdout:(87/152): python3-protobuf-3.14.0-13.el9.noarch 1.4 MB/s | 269 kB 00:00 2024-04-24T08:34:56.690 INFO:teuthology.orchestra.run.smithi189.stdout:(88/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 908 kB/s | 159 kB 00:00 2024-04-24T08:34:56.715 INFO:teuthology.orchestra.run.smithi189.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-04-24T08:34:56.717 INFO:teuthology.orchestra.run.smithi083.stdout:(42/152): python3-cryptography-36.0.1-4.el9.x86 13 MB/s | 1.2 MB 00:00 2024-04-24T08:34:56.741 INFO:teuthology.orchestra.run.smithi189.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 1.1 MB/s | 54 kB 00:00 2024-04-24T08:34:56.766 INFO:teuthology.orchestra.run.smithi189.stdout:(91/152): python3-pyasn1-modules-0.4.8-6.el9.no 2.0 MB/s | 279 kB 00:00 2024-04-24T08:34:56.775 INFO:teuthology.orchestra.run.smithi083.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-04-24T08:34:56.799 INFO:teuthology.orchestra.run.smithi189.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 721 kB/s | 42 kB 00:00 2024-04-24T08:34:56.809 INFO:teuthology.orchestra.run.smithi083.stdout:(44/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.1 MB/s | 253 kB 00:00 2024-04-24T08:34:56.835 INFO:teuthology.orchestra.run.smithi083.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-04-24T08:34:56.841 INFO:teuthology.orchestra.run.smithi189.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 592 kB/s | 44 kB 00:00 2024-04-24T08:34:56.868 INFO:teuthology.orchestra.run.smithi083.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-04-24T08:34:56.875 INFO:teuthology.orchestra.run.smithi189.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 2.9 MB/s | 221 kB 00:00 2024-04-24T08:34:56.900 INFO:teuthology.orchestra.run.smithi189.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 622 kB/s | 36 kB 00:00 2024-04-24T08:34:56.902 INFO:teuthology.orchestra.run.smithi083.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 3.2 MB/s | 215 kB 00:00 2024-04-24T08:34:56.925 INFO:teuthology.orchestra.run.smithi189.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 961 kB/s | 47 kB 00:00 2024-04-24T08:34:56.959 INFO:teuthology.orchestra.run.smithi189.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 5.2 MB/s | 305 kB 00:00 2024-04-24T08:34:56.993 INFO:teuthology.orchestra.run.smithi189.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 875 kB/s | 58 kB 00:00 2024-04-24T08:34:57.045 INFO:teuthology.orchestra.run.smithi189.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.1 MB/s | 94 kB 00:00 2024-04-24T08:34:57.085 INFO:teuthology.orchestra.run.smithi189.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 698 kB/s | 64 kB 00:00 2024-04-24T08:34:57.110 INFO:teuthology.orchestra.run.smithi189.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 700 kB/s | 45 kB 00:00 2024-04-24T08:34:57.152 INFO:teuthology.orchestra.run.smithi083.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 2.4 MB/s | 689 kB 00:00 2024-04-24T08:34:57.211 INFO:teuthology.orchestra.run.smithi083.stdout:(49/152): bison-3.7.4-5.el9.x86_64.rpm 3.0 MB/s | 943 kB 00:00 2024-04-24T08:34:57.236 INFO:teuthology.orchestra.run.smithi083.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 1.3 MB/s | 107 kB 00:00 2024-04-24T08:34:57.261 INFO:teuthology.orchestra.run.smithi083.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 434 kB/s | 21 kB 00:00 2024-04-24T08:34:57.303 INFO:teuthology.orchestra.run.smithi083.stdout:(52/152): flex-2.6.4-9.el9.x86_64.rpm 4.6 MB/s | 313 kB 00:00 2024-04-24T08:34:57.320 INFO:teuthology.orchestra.run.smithi083.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 564 kB/s | 33 kB 00:00 2024-04-24T08:34:57.370 INFO:teuthology.orchestra.run.smithi083.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 369 kB/s | 18 kB 00:00 2024-04-24T08:34:57.421 INFO:teuthology.orchestra.run.smithi083.stdout:(55/152): libaio-devel-0.3.111-13.el9.x86_64.rp 226 kB/s | 11 kB 00:00 2024-04-24T08:34:57.427 INFO:teuthology.orchestra.run.smithi189.stdout:(102/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 23 MB/s | 16 MB 00:00 2024-04-24T08:34:57.548 INFO:teuthology.orchestra.run.smithi083.stdout:(56/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.2 MB/s | 160 kB 00:00 2024-04-24T08:34:57.647 INFO:teuthology.orchestra.run.smithi083.stdout:(57/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 8.7 MB/s | 3.0 MB 00:00 2024-04-24T08:34:57.672 INFO:teuthology.orchestra.run.smithi083.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 370 kB/s | 45 kB 00:00 2024-04-24T08:34:57.730 INFO:teuthology.orchestra.run.smithi083.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 7.8 MB/s | 662 kB 00:00 2024-04-24T08:34:57.743 INFO:teuthology.orchestra.run.smithi138.stdout:(1/152): ceph-base-19.0.0-3116.g1f81fb39.el9.x8 8.0 MB/s | 5.4 MB 00:00 2024-04-24T08:34:57.761 INFO:teuthology.orchestra.run.smithi189.stdout:(103/152): abseil-cpp-20211102.0-3.el9.x86_64.r 1.6 MB/s | 548 kB 00:00 2024-04-24T08:34:57.781 INFO:teuthology.orchestra.run.smithi083.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 5.3 MB/s | 582 kB 00:00 2024-04-24T08:34:57.806 INFO:teuthology.orchestra.run.smithi083.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 342 kB/s | 25 kB 00:00 2024-04-24T08:34:57.809 INFO:teuthology.orchestra.run.smithi138.stdout:(2/152): ceph-19.0.0-3116.g1f81fb39.el9.x86_64. 8.9 kB/s | 6.6 kB 00:00 2024-04-24T08:34:57.819 INFO:teuthology.orchestra.run.smithi189.stdout:(104/152): gperftools-libs-2.9.1-2.el9.x86_64.r 5.2 MB/s | 309 kB 00:00 2024-04-24T08:34:57.840 INFO:teuthology.orchestra.run.smithi083.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 823 kB/s | 47 kB 00:00 2024-04-24T08:34:57.851 INFO:teuthology.orchestra.run.smithi138.stdout:(3/152): ceph-fuse-19.0.0-3116.g1f81fb39.el9.x8 8.1 MB/s | 890 kB 00:00 2024-04-24T08:34:57.869 INFO:teuthology.orchestra.run.smithi189.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 393 kB/s | 19 kB 00:00 2024-04-24T08:34:57.873 INFO:teuthology.orchestra.run.smithi083.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.3 MB/s | 292 kB 00:00 2024-04-24T08:34:57.895 INFO:teuthology.orchestra.run.smithi189.stdout:(106/152): lua-devel-5.4.4-4.el9.x86_64.rpm 28 kB/s | 22 kB 00:00 2024-04-24T08:34:57.907 INFO:teuthology.orchestra.run.smithi083.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 2.8 MB/s | 188 kB 00:00 2024-04-24T08:34:57.932 INFO:teuthology.orchestra.run.smithi083.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 610 kB/s | 35 kB 00:00 2024-04-24T08:34:58.035 INFO:teuthology.orchestra.run.smithi138.stdout:(4/152): ceph-immutable-object-cache-19.0.0-311 645 kB/s | 145 kB 00:00 2024-04-24T08:34:58.104 INFO:teuthology.orchestra.run.smithi189.stdout:(107/152): libarrow-9.0.0-10.el9.x86_64.rpm 19 MB/s | 4.4 MB 00:00 2024-04-24T08:34:58.128 INFO:teuthology.orchestra.run.smithi189.stdout:(108/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 112 kB/s | 26 kB 00:00 2024-04-24T08:34:58.154 INFO:teuthology.orchestra.run.smithi189.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 975 kB/s | 49 kB 00:00 2024-04-24T08:34:58.196 INFO:teuthology.orchestra.run.smithi189.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 1.0 MB/s | 67 kB 00:00 2024-04-24T08:34:58.208 INFO:teuthology.orchestra.run.smithi083.stdout:(66/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 16 MB/s | 4.4 MB 00:00 2024-04-24T08:34:58.230 INFO:teuthology.orchestra.run.smithi189.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 2.0 MB/s | 151 kB 00:00 2024-04-24T08:34:58.244 INFO:teuthology.orchestra.run.smithi138.stdout:(5/152): ceph-mgr-19.0.0-3116.g1f81fb39.el9.x86 7.2 MB/s | 1.5 MB 00:00 2024-04-24T08:34:58.305 INFO:teuthology.orchestra.run.smithi189.stdout:(112/152): python3-asyncssh-2.13.2-5.el9.noarch 7.3 MB/s | 548 kB 00:00 2024-04-24T08:34:58.324 INFO:teuthology.orchestra.run.smithi083.stdout:(67/152): openblas-openmp-0.3.21-2.el9.x86_64.r 11 MB/s | 4.7 MB 00:00 2024-04-24T08:34:58.363 INFO:teuthology.orchestra.run.smithi189.stdout:(113/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 4.9 MB/s | 838 kB 00:00 2024-04-24T08:34:58.389 INFO:teuthology.orchestra.run.smithi189.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 524 kB/s | 43 kB 00:00 2024-04-24T08:34:58.400 INFO:teuthology.orchestra.run.smithi083.stdout:(68/152): postgresql-13.14-1.el9.x86_64.rpm 8.3 MB/s | 1.6 MB 00:00 2024-04-24T08:34:58.414 INFO:teuthology.orchestra.run.smithi189.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 652 kB/s | 32 kB 00:00 2024-04-24T08:34:58.439 INFO:teuthology.orchestra.run.smithi189.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 302 kB/s | 15 kB 00:00 2024-04-24T08:34:58.467 INFO:teuthology.orchestra.run.smithi083.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 6.3 MB/s | 911 kB 00:00 2024-04-24T08:34:58.473 INFO:teuthology.orchestra.run.smithi189.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-04-24T08:34:58.492 INFO:teuthology.orchestra.run.smithi083.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 1.4 MB/s | 133 kB 00:00 2024-04-24T08:34:58.524 INFO:teuthology.orchestra.run.smithi189.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 4.2 MB/s | 358 kB 00:00 2024-04-24T08:34:58.527 INFO:teuthology.orchestra.run.smithi138.stdout:(6/152): ceph-mds-19.0.0-3116.g1f81fb39.el9.x86 3.5 MB/s | 2.4 MB 00:00 2024-04-24T08:34:58.566 INFO:teuthology.orchestra.run.smithi189.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.2 MB/s | 303 kB 00:00 2024-04-24T08:34:58.591 INFO:teuthology.orchestra.run.smithi189.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 534 kB/s | 35 kB 00:00 2024-04-24T08:34:58.593 INFO:teuthology.orchestra.run.smithi083.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 10 MB/s | 1.0 MB 00:00 2024-04-24T08:34:58.625 INFO:teuthology.orchestra.run.smithi189.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 3.5 MB/s | 206 kB 00:00 2024-04-24T08:34:58.644 INFO:teuthology.orchestra.run.smithi138.stdout:(7/152): ceph-mon-19.0.0-3116.g1f81fb39.el9.x86 12 MB/s | 4.7 MB 00:00 2024-04-24T08:34:58.668 INFO:teuthology.orchestra.run.smithi083.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.2 MB/s | 87 kB 00:00 2024-04-24T08:34:58.675 INFO:teuthology.orchestra.run.smithi189.stdout:(122/152): python3-grpcio-tools-1.46.7-10.el9.x 2.9 MB/s | 144 kB 00:00 2024-04-24T08:34:58.709 INFO:teuthology.orchestra.run.smithi189.stdout:(123/152): python3-jaraco-8.2.1-3.el9.noarch.rp 328 kB/s | 11 kB 00:00 2024-04-24T08:34:58.784 INFO:teuthology.orchestra.run.smithi189.stdout:(124/152): python3-grpcio-1.46.7-10.el9.x86_64. 11 MB/s | 2.0 MB 00:00 2024-04-24T08:34:58.809 INFO:teuthology.orchestra.run.smithi189.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 177 kB/s | 18 kB 00:00 2024-04-24T08:34:58.826 INFO:teuthology.orchestra.run.smithi189.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 563 kB/s | 23 kB 00:00 2024-04-24T08:34:58.843 INFO:teuthology.orchestra.run.smithi083.stdout:(73/152): postgresql-server-13.14-1.el9.x86_64. 15 MB/s | 5.8 MB 00:00 2024-04-24T08:34:58.860 INFO:teuthology.orchestra.run.smithi189.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 394 kB/s | 19 kB 00:00 2024-04-24T08:34:58.877 INFO:teuthology.orchestra.run.smithi189.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 395 kB/s | 20 kB 00:00 2024-04-24T08:34:58.918 INFO:teuthology.orchestra.run.smithi083.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 3.2 MB/s | 244 kB 00:00 2024-04-24T08:34:58.919 INFO:teuthology.orchestra.run.smithi189.stdout:(129/152): python3-logutils-0.3.5-21.el9.noarch 1.1 MB/s | 46 kB 00:00 2024-04-24T08:34:58.994 INFO:teuthology.orchestra.run.smithi189.stdout:(130/152): python3-kubernetes-26.1.0-2.el9.noar 7.7 MB/s | 1.0 MB 00:00 2024-04-24T08:34:59.027 INFO:teuthology.orchestra.run.smithi189.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 730 kB/s | 79 kB 00:00 2024-04-24T08:34:59.053 INFO:teuthology.orchestra.run.smithi189.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 999 kB/s | 58 kB 00:00 2024-04-24T08:34:59.069 INFO:teuthology.orchestra.run.smithi083.stdout:(75/152): python3-babel-2.9.1-2.el9.noarch.rpm 15 MB/s | 6.0 MB 00:00 2024-04-24T08:34:59.086 INFO:teuthology.orchestra.run.smithi189.stdout:(133/152): python3-pecan-1.4.2-3.el9.noarch.rpm 4.6 MB/s | 272 kB 00:00 2024-04-24T08:34:59.112 INFO:teuthology.orchestra.run.smithi189.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 283 kB/s | 16 kB 00:00 2024-04-24T08:34:59.127 INFO:teuthology.orchestra.run.smithi083.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 1.2 MB/s | 248 kB 00:00 2024-04-24T08:34:59.153 INFO:teuthology.orchestra.run.smithi083.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 310 kB/s | 26 kB 00:00 2024-04-24T08:34:59.153 INFO:teuthology.orchestra.run.smithi189.stdout:(135/152): protobuf-compiler-3.14.0-13.el9.x86_ 423 kB/s | 863 kB 00:02 2024-04-24T08:34:59.179 INFO:teuthology.orchestra.run.smithi189.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 443 kB/s | 28 kB 00:00 2024-04-24T08:34:59.194 INFO:teuthology.orchestra.run.smithi083.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 290 kB/s | 19 kB 00:00 2024-04-24T08:34:59.228 INFO:teuthology.orchestra.run.smithi083.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 2.3 MB/s | 172 kB 00:00 2024-04-24T08:34:59.270 INFO:teuthology.orchestra.run.smithi083.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 467 kB/s | 35 kB 00:00 2024-04-24T08:34:59.287 INFO:teuthology.orchestra.run.smithi189.stdout:(137/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 450 kB/s | 90 kB 00:00 2024-04-24T08:34:59.321 INFO:teuthology.orchestra.run.smithi189.stdout:(138/152): python3-routes-2.5.1-5.el9.noarch.rp 2.6 MB/s | 188 kB 00:00 2024-04-24T08:34:59.345 INFO:teuthology.orchestra.run.smithi083.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.3 MB/s | 250 kB 00:00 2024-04-24T08:34:59.346 INFO:teuthology.orchestra.run.smithi189.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 1.0 MB/s | 59 kB 00:00 2024-04-24T08:34:59.371 INFO:teuthology.orchestra.run.smithi189.stdout:(140/152): python3-repoze-lru-0.7-16.el9.noarch 142 kB/s | 31 kB 00:00 2024-04-24T08:34:59.397 INFO:teuthology.orchestra.run.smithi189.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 480 kB/s | 36 kB 00:00 2024-04-24T08:34:59.422 INFO:teuthology.orchestra.run.smithi189.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 680 kB/s | 51 kB 00:00 2024-04-24T08:34:59.429 INFO:teuthology.orchestra.run.smithi083.stdout:(82/152): python3-oauthlib-3.1.1-5.el9.noarch.r 2.6 MB/s | 222 kB 00:00 2024-04-24T08:34:59.456 INFO:teuthology.orchestra.run.smithi189.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 1.5 MB/s | 90 kB 00:00 2024-04-24T08:34:59.510 INFO:teuthology.orchestra.run.smithi189.stdout:(144/152): python3-werkzeug-2.0.3-3.el9.1.noarc 5.0 MB/s | 427 kB 00:00 2024-04-24T08:34:59.531 INFO:teuthology.orchestra.run.smithi189.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 268 kB/s | 20 kB 00:00 2024-04-24T08:34:59.554 INFO:teuthology.orchestra.run.smithi083.stdout:(83/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 16 MB/s | 5.1 MB 00:00 2024-04-24T08:34:59.565 INFO:teuthology.orchestra.run.smithi189.stdout:(146/152): python3-webob-1.8.7-6.el9.noarch.rpm 1.2 MB/s | 230 kB 00:00 2024-04-24T08:34:59.580 INFO:teuthology.orchestra.run.smithi083.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 516 kB/s | 77 kB 00:00 2024-04-24T08:34:59.599 INFO:teuthology.orchestra.run.smithi189.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 2.0 MB/s | 192 kB 00:00 2024-04-24T08:34:59.605 INFO:teuthology.orchestra.run.smithi083.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 856 kB/s | 42 kB 00:00 2024-04-24T08:34:59.636 INFO:teuthology.orchestra.run.smithi138.stdout:(8/152): ceph-osd-19.0.0-3116.g1f81fb39.el9.x86 15 MB/s | 17 MB 00:01 2024-04-24T08:34:59.647 INFO:teuthology.orchestra.run.smithi083.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 4.0 MB/s | 269 kB 00:00 2024-04-24T08:34:59.682 INFO:teuthology.orchestra.run.smithi083.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.1 MB/s | 159 kB 00:00 2024-04-24T08:34:59.690 INFO:teuthology.orchestra.run.smithi189.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 10 MB/s | 1.6 MB 00:00 2024-04-24T08:34:59.703 INFO:teuthology.orchestra.run.smithi138.stdout:(9/152): ceph-selinux-19.0.0-3116.g1f81fb39.el9 385 kB/s | 25 kB 00:00 2024-04-24T08:34:59.714 INFO:teuthology.orchestra.run.smithi083.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 279 kB 00:00 2024-04-24T08:34:59.739 INFO:teuthology.orchestra.run.smithi083.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 883 kB/s | 51 kB 00:00 2024-04-24T08:34:59.773 INFO:teuthology.orchestra.run.smithi083.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 926 kB/s | 54 kB 00:00 2024-04-24T08:34:59.832 INFO:teuthology.orchestra.run.smithi083.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 723 kB/s | 42 kB 00:00 2024-04-24T08:34:59.932 INFO:teuthology.orchestra.run.smithi083.stdout:(92/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 445 kB/s | 44 kB 00:00 2024-04-24T08:35:00.016 INFO:teuthology.orchestra.run.smithi083.stdout:(93/152): qatlib-24.02.0-1.el9.x86_64.rpm 2.6 MB/s | 221 kB 00:00 2024-04-24T08:35:00.125 INFO:teuthology.orchestra.run.smithi083.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 334 kB/s | 36 kB 00:00 2024-04-24T08:35:00.225 INFO:teuthology.orchestra.run.smithi083.stdout:(95/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 477 kB/s | 47 kB 00:00 2024-04-24T08:35:00.302 INFO:teuthology.orchestra.run.smithi083.stdout:(96/152): socat-1.7.4.1-5.el9.x86_64.rpm 3.9 MB/s | 305 kB 00:00 2024-04-24T08:35:00.332 INFO:teuthology.orchestra.run.smithi189.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 2.1 MB/s | 1.3 MB 00:00 2024-04-24T08:35:00.386 INFO:teuthology.orchestra.run.smithi083.stdout:(97/152): uuid-1.6.2-55.el9.x86_64.rpm 702 kB/s | 58 kB 00:00 2024-04-24T08:35:00.458 INFO:teuthology.orchestra.run.smithi189.stdout:(150/152): librbd1-19.0.0-3116.g1f81fb39.el9.x8 3.7 MB/s | 3.1 MB 00:00 2024-04-24T08:35:00.495 INFO:teuthology.orchestra.run.smithi138.stdout:(10/152): ceph-radosgw-19.0.0-3116.g1f81fb39.el 5.7 MB/s | 11 MB 00:01 2024-04-24T08:35:00.603 INFO:teuthology.orchestra.run.smithi083.stdout:(98/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 19 MB/s | 16 MB 00:00 2024-04-24T08:35:00.603 INFO:teuthology.orchestra.run.smithi138.stdout:(11/152): cephfs-mirror-19.0.0-3116.g1f81fb39.e 2.1 MB/s | 225 kB 00:00 2024-04-24T08:35:00.628 INFO:teuthology.orchestra.run.smithi083.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 389 kB/s | 94 kB 00:00 2024-04-24T08:35:00.704 INFO:teuthology.orchestra.run.smithi138.stdout:(12/152): cephfs-shell-19.0.0-3116.g1f81fb39.el 272 kB/s | 27 kB 00:00 2024-04-24T08:35:00.821 INFO:teuthology.orchestra.run.smithi138.stdout:(13/152): libcephfs-devel-19.0.0-3116.g1f81fb39 274 kB/s | 32 kB 00:00 2024-04-24T08:35:00.988 INFO:teuthology.orchestra.run.smithi138.stdout:(14/152): libcephfs2-19.0.0-3116.g1f81fb39.el9. 4.4 MB/s | 746 kB 00:00 2024-04-24T08:35:01.080 INFO:teuthology.orchestra.run.smithi138.stdout:(15/152): libcephsqlite-19.0.0-3116.g1f81fb39.e 1.8 MB/s | 163 kB 00:00 2024-04-24T08:35:01.163 INFO:teuthology.orchestra.run.smithi138.stdout:(16/152): librados-devel-19.0.0-3116.g1f81fb39. 1.5 MB/s | 127 kB 00:00 2024-04-24T08:35:01.170 INFO:teuthology.orchestra.run.smithi083.stdout:(100/152): ceph-test-19.0.0-3116.g1f81fb39.el9. 6.2 MB/s | 49 MB 00:07 2024-04-24T08:35:01.191 INFO:teuthology.orchestra.run.smithi189.stdout:(151/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 2.8 MB/s | 2.4 MB 00:00 2024-04-24T08:35:01.195 INFO:teuthology.orchestra.run.smithi083.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 79 kB/s | 45 kB 00:00 2024-04-24T08:35:01.220 INFO:teuthology.orchestra.run.smithi083.stdout:(102/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 103 kB/s | 64 kB 00:00 2024-04-24T08:35:01.322 INFO:teuthology.orchestra.run.smithi138.stdout:(17/152): libradosstriper1-19.0.0-3116.g1f81fb3 3.1 MB/s | 501 kB 00:00 2024-04-24T08:35:01.446 INFO:teuthology.orchestra.run.smithi083.stdout:(103/152): abseil-cpp-20211102.0-3.el9.x86_64.r 2.4 MB/s | 548 kB 00:00 2024-04-24T08:35:01.471 INFO:teuthology.orchestra.run.smithi083.stdout:(104/152): lua-devel-5.4.4-4.el9.x86_64.rpm 74 kB/s | 22 kB 00:00 2024-04-24T08:35:01.505 INFO:teuthology.orchestra.run.smithi083.stdout:(105/152): gperftools-libs-2.9.1-2.el9.x86_64.r 5.2 MB/s | 309 kB 00:00 2024-04-24T08:35:01.580 INFO:teuthology.orchestra.run.smithi083.stdout:(106/152): grpc-data-1.46.7-10.el9.noarch.rpm 180 kB/s | 19 kB 00:00 2024-04-24T08:35:01.647 INFO:teuthology.orchestra.run.smithi083.stdout:(107/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 394 kB/s | 26 kB 00:00 2024-04-24T08:35:01.748 INFO:teuthology.orchestra.run.smithi083.stdout:(108/152): libarrow-9.0.0-10.el9.x86_64.rpm 18 MB/s | 4.4 MB 00:00 2024-04-24T08:35:01.773 INFO:teuthology.orchestra.run.smithi083.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 389 kB/s | 49 kB 00:00 2024-04-24T08:35:01.798 INFO:teuthology.orchestra.run.smithi083.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-04-24T08:35:01.882 INFO:teuthology.orchestra.run.smithi083.stdout:(111/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 9.9 MB/s | 838 kB 00:00 2024-04-24T08:35:01.932 INFO:teuthology.orchestra.run.smithi083.stdout:(112/152): luarocks-3.9.2-1.el9.noarch.rpm 956 kB/s | 151 kB 00:00 2024-04-24T08:35:01.974 INFO:teuthology.orchestra.run.smithi083.stdout:(113/152): python3-asyncssh-2.13.2-5.el9.noarch 5.9 MB/s | 548 kB 00:00 2024-04-24T08:35:01.991 INFO:teuthology.orchestra.run.smithi083.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 752 kB/s | 43 kB 00:00 2024-04-24T08:35:02.025 INFO:teuthology.orchestra.run.smithi083.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 649 kB/s | 32 kB 00:00 2024-04-24T08:35:02.041 INFO:teuthology.orchestra.run.smithi083.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 301 kB/s | 15 kB 00:00 2024-04-24T08:35:02.075 INFO:teuthology.orchestra.run.smithi083.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 3.4 MB/s | 172 kB 00:00 2024-04-24T08:35:02.134 INFO:teuthology.orchestra.run.smithi083.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 3.8 MB/s | 358 kB 00:00 2024-04-24T08:35:02.159 INFO:teuthology.orchestra.run.smithi083.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.6 MB/s | 303 kB 00:00 2024-04-24T08:35:02.209 INFO:teuthology.orchestra.run.smithi083.stdout:(120/152): protobuf-compiler-3.14.0-13.el9.x86_ 852 kB/s | 863 kB 00:01 2024-04-24T08:35:02.234 INFO:teuthology.orchestra.run.smithi083.stdout:(121/152): python3-colorama-0.4.4-12.el9.noarch 476 kB/s | 35 kB 00:00 2024-04-24T08:35:02.239 INFO:teuthology.orchestra.run.smithi138.stdout:(18/152): librgw2-19.0.0-3116.g1f81fb39.el9.x86 5.9 MB/s | 5.4 MB 00:00 2024-04-24T08:35:02.268 INFO:teuthology.orchestra.run.smithi083.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 1.9 MB/s | 206 kB 00:00 2024-04-24T08:35:02.274 INFO:teuthology.orchestra.run.smithi189.stdout:(152/152): librados2-19.0.0-3116.g1f81fb39.el9. 1.3 MB/s | 3.4 MB 00:02 2024-04-24T08:35:02.287 INFO:teuthology.orchestra.run.smithi189.stdout:-------------------------------------------------------------------------------- 2024-04-24T08:35:02.287 INFO:teuthology.orchestra.run.smithi189.stdout:Total 13 MB/s | 222 MB 00:17 2024-04-24T08:35:02.322 INFO:teuthology.orchestra.run.smithi138.stdout:(19/152): python3-ceph-argparse-19.0.0-3116.g1f 549 kB/s | 45 kB 00:00 2024-04-24T08:35:02.334 INFO:teuthology.orchestra.run.smithi083.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 1.4 MB/s | 144 kB 00:00 2024-04-24T08:35:02.369 INFO:teuthology.orchestra.run.smithi083.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 107 kB/s | 11 kB 00:00 2024-04-24T08:35:02.394 INFO:teuthology.orchestra.run.smithi083.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 300 kB/s | 18 kB 00:00 2024-04-24T08:35:02.414 INFO:teuthology.orchestra.run.smithi138.stdout:(20/152): python3-ceph-common-19.0.0-3116.g1f81 1.5 MB/s | 135 kB 00:00 2024-04-24T08:35:02.419 INFO:teuthology.orchestra.run.smithi083.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 469 kB/s | 23 kB 00:00 2024-04-24T08:35:02.445 INFO:teuthology.orchestra.run.smithi083.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 393 kB/s | 19 kB 00:00 2024-04-24T08:35:02.470 INFO:teuthology.orchestra.run.smithi083.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 398 kB/s | 20 kB 00:00 2024-04-24T08:35:02.508 INFO:teuthology.orchestra.run.smithi138.stdout:(21/152): python3-cephfs-19.0.0-3116.g1f81fb39. 1.7 MB/s | 161 kB 00:00 2024-04-24T08:35:02.529 INFO:teuthology.orchestra.run.smithi083.stdout:(129/152): python3-kubernetes-26.1.0-2.el9.noar 12 MB/s | 1.0 MB 00:00 2024-04-24T08:35:02.554 INFO:teuthology.orchestra.run.smithi083.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 560 kB/s | 46 kB 00:00 2024-04-24T08:35:02.579 INFO:teuthology.orchestra.run.smithi083.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 1.6 MB/s | 79 kB 00:00 2024-04-24T08:35:02.629 INFO:teuthology.orchestra.run.smithi083.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 775 kB/s | 58 kB 00:00 2024-04-24T08:35:02.817 INFO:teuthology.orchestra.run.smithi138.stdout:(22/152): ceph-common-19.0.0-3116.g1f81fb39.el9 3.7 MB/s | 21 MB 00:05 2024-04-24T08:35:02.821 INFO:teuthology.orchestra.run.smithi083.stdout:(133/152): python3-pecan-1.4.2-3.el9.noarch.rpm 1.1 MB/s | 272 kB 00:00 2024-04-24T08:35:02.838 INFO:teuthology.orchestra.run.smithi083.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 79 kB/s | 16 kB 00:00 2024-04-24T08:35:02.850 INFO:teuthology.orchestra.run.smithi138.stdout:(23/152): python3-rados-19.0.0-3116.g1f81fb39.e 939 kB/s | 320 kB 00:00 2024-04-24T08:35:02.872 INFO:teuthology.orchestra.run.smithi083.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.8 MB/s | 90 kB 00:00 2024-04-24T08:35:02.897 INFO:teuthology.orchestra.run.smithi083.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 487 kB/s | 28 kB 00:00 2024-04-24T08:35:02.918 INFO:teuthology.orchestra.run.smithi138.stdout:(24/152): python3-rgw-19.0.0-3116.g1f81fb39.el9 1.5 MB/s | 99 kB 00:00 2024-04-24T08:35:02.922 INFO:teuthology.orchestra.run.smithi083.stdout:(137/152): python3-repoze-lru-0.7-16.el9.noarch 622 kB/s | 31 kB 00:00 2024-04-24T08:35:02.956 INFO:teuthology.orchestra.run.smithi083.stdout:(138/152): python3-routes-2.5.1-5.el9.noarch.rp 3.2 MB/s | 188 kB 00:00 2024-04-24T08:35:02.981 INFO:teuthology.orchestra.run.smithi083.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 1.0 MB/s | 59 kB 00:00 2024-04-24T08:35:03.007 INFO:teuthology.orchestra.run.smithi083.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 723 kB/s | 36 kB 00:00 2024-04-24T08:35:03.032 INFO:teuthology.orchestra.run.smithi083.stdout:(141/152): python3-typing-extensions-4.4.0-2.el 1.0 MB/s | 51 kB 00:00 2024-04-24T08:35:03.091 INFO:teuthology.orchestra.run.smithi083.stdout:(142/152): python3-grpcio-1.46.7-10.el9.x86_64. 2.3 MB/s | 2.0 MB 00:00 2024-04-24T08:35:03.141 INFO:teuthology.orchestra.run.smithi083.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 1.7 MB/s | 230 kB 00:00 2024-04-24T08:35:03.167 INFO:teuthology.orchestra.run.smithi083.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 674 kB/s | 90 kB 00:00 2024-04-24T08:35:03.192 INFO:teuthology.orchestra.run.smithi083.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 404 kB/s | 20 kB 00:00 2024-04-24T08:35:03.225 INFO:teuthology.orchestra.run.smithi083.stdout:(146/152): re2-20211101-3.el9.x86_64.rpm 3.2 MB/s | 192 kB 00:00 2024-04-24T08:35:03.259 INFO:teuthology.orchestra.run.smithi083.stdout:(147/152): python3-werkzeug-2.0.3-3.el9.1.noarc 2.5 MB/s | 427 kB 00:00 2024-04-24T08:35:03.334 INFO:teuthology.orchestra.run.smithi083.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-04-24T08:35:03.435 INFO:teuthology.orchestra.run.smithi138.stdout:(25/152): ceph-test-19.0.0-3116.g1f81fb39.el9.x 13 MB/s | 49 MB 00:03 2024-04-24T08:35:03.460 INFO:teuthology.orchestra.run.smithi138.stdout:(26/152): rbd-fuse-19.0.0-3116.g1f81fb39.el9.x8 157 kB/s | 85 kB 00:00 2024-04-24T08:35:03.526 INFO:teuthology.orchestra.run.smithi138.stdout:(27/152): python3-rbd-19.0.0-3116.g1f81fb39.el9 422 kB/s | 298 kB 00:00 2024-04-24T08:35:03.527 INFO:teuthology.orchestra.run.smithi083.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 7.0 MB/s | 1.3 MB 00:00 2024-04-24T08:35:03.602 INFO:teuthology.orchestra.run.smithi138.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3116.g 454 kB/s | 34 kB 00:00 2024-04-24T08:35:03.694 INFO:teuthology.orchestra.run.smithi138.stdout:(29/152): rbd-nbd-19.0.0-3116.g1f81fb39.el9.x86 731 kB/s | 170 kB 00:00 2024-04-24T08:35:03.703 INFO:teuthology.orchestra.run.smithi083.stdout:(150/152): librados2-19.0.0-3116.g1f81fb39.el9. 7.1 MB/s | 3.4 MB 00:00 2024-04-24T08:35:03.778 INFO:teuthology.orchestra.run.smithi138.stdout:(30/152): rbd-mirror-19.0.0-3116.g1f81fb39.el9. 9.0 MB/s | 3.1 MB 00:00 2024-04-24T08:35:03.793 INFO:teuthology.orchestra.run.smithi083.stdout:(151/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 9.0 MB/s | 2.4 MB 00:00 2024-04-24T08:35:03.803 INFO:teuthology.orchestra.run.smithi138.stdout:(31/152): ceph-mgr-cephadm-19.0.0-3116.g1f81fb3 713 kB/s | 142 kB 00:00 2024-04-24T08:35:03.860 INFO:teuthology.orchestra.run.smithi083.stdout:(152/152): librbd1-19.0.0-3116.g1f81fb39.el9.x8 5.2 MB/s | 3.1 MB 00:00 2024-04-24T08:35:03.866 INFO:teuthology.orchestra.run.smithi083.stdout:-------------------------------------------------------------------------------- 2024-04-24T08:35:03.867 INFO:teuthology.orchestra.run.smithi083.stdout:Total 14 MB/s | 222 MB 00:15 2024-04-24T08:35:03.920 INFO:teuthology.orchestra.run.smithi138.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3116.g1f 2.1 MB/s | 249 kB 00:00 2024-04-24T08:35:04.014 INFO:teuthology.orchestra.run.smithi138.stdout:(33/152): ceph-mgr-rook-19.0.0-3116.g1f81fb39.e 544 kB/s | 49 kB 00:00 2024-04-24T08:35:04.095 INFO:teuthology.orchestra.run.smithi138.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3116.g1 200 kB/s | 16 kB 00:00 2024-04-24T08:35:04.229 INFO:teuthology.orchestra.run.smithi138.stdout:(35/152): ceph-volume-19.0.0-3116.g1f81fb39.el9 2.0 MB/s | 273 kB 00:00 2024-04-24T08:35:04.355 INFO:teuthology.orchestra.run.smithi138.stdout:(36/152): ceph-mgr-diskprediction-local-19.0.0- 13 MB/s | 7.4 MB 00:00 2024-04-24T08:35:04.438 INFO:teuthology.orchestra.run.smithi138.stdout:(37/152): ceph-mgr-dashboard-19.0.0-3116.g1f81f 4.8 MB/s | 3.5 MB 00:00 2024-04-24T08:35:04.463 INFO:teuthology.orchestra.run.smithi138.stdout:(38/152): cephfs-top-19.0.0-3116.g1f81fb39.el9. 169 kB/s | 18 kB 00:00 2024-04-24T08:35:04.513 INFO:teuthology.orchestra.run.smithi138.stdout:(39/152): cephadm-19.0.0-3116.g1f81fb39.el9.noa 2.6 MB/s | 762 kB 00:00 2024-04-24T08:35:04.889 INFO:teuthology.orchestra.run.smithi138.stdout:(40/152): fuse-2.9.9-15.el9.x86_64.rpm 179 kB/s | 80 kB 00:00 2024-04-24T08:35:05.097 INFO:teuthology.orchestra.run.smithi138.stdout:(41/152): libquadmath-11.4.1-3.el9.x86_64.rpm 335 kB/s | 195 kB 00:00 2024-04-24T08:35:05.232 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction check 2024-04-24T08:35:05.239 INFO:teuthology.orchestra.run.smithi138.stdout:(42/152): libgfortran-11.4.1-3.el9.x86_64.rpm 1.0 MB/s | 803 kB 00:00 2024-04-24T08:35:05.264 INFO:teuthology.orchestra.run.smithi138.stdout:(43/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 676 kB/s | 253 kB 00:00 2024-04-24T08:35:05.356 INFO:teuthology.orchestra.run.smithi138.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 915 kB/s | 106 kB 00:00 2024-04-24T08:35:05.390 INFO:teuthology.orchestra.run.smithi138.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 1.1 MB/s | 135 kB 00:00 2024-04-24T08:35:05.466 INFO:teuthology.orchestra.run.smithi138.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 1.1 MB/s | 125 kB 00:00 2024-04-24T08:35:05.472 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction check succeeded. 2024-04-24T08:35:05.472 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction test 2024-04-24T08:35:05.499 INFO:teuthology.orchestra.run.smithi138.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 1.9 MB/s | 215 kB 00:00 2024-04-24T08:35:05.558 INFO:teuthology.orchestra.run.smithi138.stdout:(48/152): python3-cryptography-36.0.1-4.el9.x86 2.7 MB/s | 1.2 MB 00:00 2024-04-24T08:35:05.733 INFO:teuthology.orchestra.run.smithi138.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 613 kB/s | 107 kB 00:00 2024-04-24T08:35:05.783 INFO:teuthology.orchestra.run.smithi138.stdout:(50/152): automake-1.16.2-8.el9.noarch.rpm 2.1 MB/s | 689 kB 00:00 2024-04-24T08:35:05.809 INFO:teuthology.orchestra.run.smithi138.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 289 kB/s | 21 kB 00:00 2024-04-24T08:35:05.851 INFO:teuthology.orchestra.run.smithi138.stdout:(52/152): bison-3.7.4-5.el9.x86_64.rpm 2.6 MB/s | 943 kB 00:00 2024-04-24T08:35:05.876 INFO:teuthology.orchestra.run.smithi138.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 493 kB/s | 33 kB 00:00 2024-04-24T08:35:05.909 INFO:teuthology.orchestra.run.smithi138.stdout:(54/152): flex-2.6.4-9.el9.x86_64.rpm 2.4 MB/s | 313 kB 00:00 2024-04-24T08:35:05.935 INFO:teuthology.orchestra.run.smithi138.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 314 kB/s | 18 kB 00:00 2024-04-24T08:35:05.968 INFO:teuthology.orchestra.run.smithi138.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 193 kB/s | 11 kB 00:00 2024-04-24T08:35:06.002 INFO:teuthology.orchestra.run.smithi138.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.4 MB/s | 160 kB 00:00 2024-04-24T08:35:06.027 INFO:teuthology.orchestra.run.smithi138.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 785 kB/s | 45 kB 00:00 2024-04-24T08:35:06.103 INFO:teuthology.orchestra.run.smithi138.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 6.5 MB/s | 662 kB 00:00 2024-04-24T08:35:06.136 INFO:teuthology.orchestra.run.smithi138.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 5.3 MB/s | 582 kB 00:00 2024-04-24T08:35:06.161 INFO:teuthology.orchestra.run.smithi138.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 440 kB/s | 25 kB 00:00 2024-04-24T08:35:06.245 INFO:teuthology.orchestra.run.smithi138.stdout:(62/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 7.6 MB/s | 3.0 MB 00:00 2024-04-24T08:35:06.270 INFO:teuthology.orchestra.run.smithi138.stdout:(63/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 356 kB/s | 47 kB 00:00 2024-04-24T08:35:06.304 INFO:teuthology.orchestra.run.smithi138.stdout:(64/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.0 MB/s | 292 kB 00:00 2024-04-24T08:35:06.329 INFO:teuthology.orchestra.run.smithi138.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 608 kB/s | 35 kB 00:00 2024-04-24T08:35:06.365 INFO:teuthology.orchestra.run.smithi138.stdout:(66/152): lua-5.4.4-4.el9.x86_64.rpm 1.6 MB/s | 188 kB 00:00 2024-04-24T08:35:06.547 INFO:teuthology.orchestra.run.smithi138.stdout:(67/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 20 MB/s | 4.4 MB 00:00 2024-04-24T08:35:06.597 INFO:teuthology.orchestra.run.smithi138.stdout:(68/152): postgresql-13.14-1.el9.x86_64.rpm 7.0 MB/s | 1.6 MB 00:00 2024-04-24T08:35:06.697 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction check 2024-04-24T08:35:06.697 INFO:teuthology.orchestra.run.smithi138.stdout:(69/152): openblas-openmp-0.3.21-2.el9.x86_64.r 12 MB/s | 4.7 MB 00:00 2024-04-24T08:35:06.731 INFO:teuthology.orchestra.run.smithi138.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 1.0 MB/s | 133 kB 00:00 2024-04-24T08:35:06.798 INFO:teuthology.orchestra.run.smithi138.stdout:(71/152): postgresql-contrib-13.14-1.el9.x86_64 3.6 MB/s | 911 kB 00:00 2024-04-24T08:35:06.864 INFO:teuthology.orchestra.run.smithi138.stdout:(72/152): protobuf-3.14.0-13.el9.x86_64.rpm 7.6 MB/s | 1.0 MB 00:00 2024-04-24T08:35:06.890 INFO:teuthology.orchestra.run.smithi138.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 959 kB/s | 87 kB 00:00 2024-04-24T08:35:06.940 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction check succeeded. 2024-04-24T08:35:06.941 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction test 2024-04-24T08:35:06.982 INFO:teuthology.orchestra.run.smithi138.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 2.6 MB/s | 244 kB 00:00 2024-04-24T08:35:07.132 INFO:teuthology.orchestra.run.smithi138.stdout:(75/152): postgresql-server-13.14-1.el9.x86_64. 13 MB/s | 5.8 MB 00:00 2024-04-24T08:35:07.166 INFO:teuthology.orchestra.run.smithi138.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 1.3 MB/s | 248 kB 00:00 2024-04-24T08:35:07.191 INFO:teuthology.orchestra.run.smithi138.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 446 kB/s | 26 kB 00:00 2024-04-24T08:35:07.308 INFO:teuthology.orchestra.run.smithi138.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 135 kB/s | 19 kB 00:00 2024-04-24T08:35:07.342 INFO:teuthology.orchestra.run.smithi138.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 1.1 MB/s | 172 kB 00:00 2024-04-24T08:35:07.367 INFO:teuthology.orchestra.run.smithi138.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 602 kB/s | 35 kB 00:00 2024-04-24T08:35:07.501 INFO:teuthology.orchestra.run.smithi138.stdout:(81/152): python3-babel-2.9.1-2.el9.noarch.rpm 9.4 MB/s | 6.0 MB 00:00 2024-04-24T08:35:07.534 INFO:teuthology.orchestra.run.smithi138.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.5 MB/s | 250 kB 00:00 2024-04-24T08:35:07.601 INFO:teuthology.orchestra.run.smithi138.stdout:(83/152): python3-packaging-20.9-5.el9.noarch.r 1.2 MB/s | 77 kB 00:00 2024-04-24T08:35:07.635 INFO:teuthology.orchestra.run.smithi138.stdout:(84/152): python3-oauthlib-3.1.1-5.el9.noarch.r 1.6 MB/s | 222 kB 00:00 2024-04-24T08:35:07.660 INFO:teuthology.orchestra.run.smithi138.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 741 kB/s | 42 kB 00:00 2024-04-24T08:35:07.712 INFO:teuthology.orchestra.run.smithi138.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 3.5 MB/s | 269 kB 00:00 2024-04-24T08:35:07.744 INFO:teuthology.orchestra.run.smithi138.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.9 MB/s | 159 kB 00:00 2024-04-24T08:35:07.777 INFO:teuthology.orchestra.run.smithi138.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 4.2 MB/s | 279 kB 00:00 2024-04-24T08:35:07.803 INFO:teuthology.orchestra.run.smithi138.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 883 kB/s | 51 kB 00:00 2024-04-24T08:35:07.845 INFO:teuthology.orchestra.run.smithi138.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 808 kB/s | 54 kB 00:00 2024-04-24T08:35:08.002 INFO:teuthology.orchestra.run.smithi138.stdout:(91/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 7.7 MB/s | 5.1 MB 00:00 2024-04-24T08:35:08.029 INFO:teuthology.orchestra.run.smithi138.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 228 kB/s | 42 kB 00:00 2024-04-24T08:35:08.063 INFO:teuthology.orchestra.run.smithi138.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 745 kB/s | 44 kB 00:00 2024-04-24T08:35:08.096 INFO:teuthology.orchestra.run.smithi138.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 3.3 MB/s | 221 kB 00:00 2024-04-24T08:35:08.121 INFO:teuthology.orchestra.run.smithi138.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 626 kB/s | 36 kB 00:00 2024-04-24T08:35:08.155 INFO:teuthology.orchestra.run.smithi138.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 819 kB/s | 47 kB 00:00 2024-04-24T08:35:08.233 INFO:teuthology.orchestra.run.smithi138.stdout:(97/152): uuid-1.6.2-55.el9.x86_64.rpm 762 kB/s | 58 kB 00:00 2024-04-24T08:35:08.264 INFO:teuthology.orchestra.run.smithi138.stdout:(98/152): socat-1.7.4.1-5.el9.x86_64.rpm 2.1 MB/s | 305 kB 00:00 2024-04-24T08:35:08.290 INFO:teuthology.orchestra.run.smithi138.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.6 MB/s | 94 kB 00:00 2024-04-24T08:35:08.324 INFO:teuthology.orchestra.run.smithi138.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-04-24T08:35:08.348 INFO:teuthology.orchestra.run.smithi138.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 773 kB/s | 45 kB 00:00 2024-04-24T08:35:08.354 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction test succeeded. 2024-04-24T08:35:08.355 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction 2024-04-24T08:35:08.407 INFO:teuthology.orchestra.run.smithi138.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 271 kB/s | 22 kB 00:00 2024-04-24T08:35:08.558 INFO:teuthology.orchestra.run.smithi138.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 4.1 MB/s | 863 kB 00:00 2024-04-24T08:35:08.933 INFO:teuthology.orchestra.run.smithi138.stdout:(104/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 14 MB/s | 16 MB 00:01 2024-04-24T08:35:08.975 INFO:teuthology.orchestra.run.smithi138.stdout:(105/152): abseil-cpp-20211102.0-3.el9.x86_64.r 975 kB/s | 548 kB 00:00 2024-04-24T08:35:09.150 INFO:teuthology.orchestra.run.smithi138.stdout:(106/152): grpc-data-1.46.7-10.el9.noarch.rpm 90 kB/s | 19 kB 00:00 2024-04-24T08:35:09.184 INFO:teuthology.orchestra.run.smithi138.stdout:(107/152): gperftools-libs-2.9.1-2.el9.x86_64.r 496 kB/s | 309 kB 00:00 2024-04-24T08:35:09.217 INFO:teuthology.orchestra.run.smithi138.stdout:(108/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 397 kB/s | 26 kB 00:00 2024-04-24T08:35:09.259 INFO:teuthology.orchestra.run.smithi138.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 654 kB/s | 49 kB 00:00 2024-04-24T08:35:09.301 INFO:teuthology.orchestra.run.smithi138.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 815 kB/s | 67 kB 00:00 2024-04-24T08:35:09.401 INFO:teuthology.orchestra.run.smithi138.stdout:(111/152): libarrow-9.0.0-10.el9.x86_64.rpm 12 MB/s | 4.4 MB 00:00 2024-04-24T08:35:09.426 INFO:teuthology.orchestra.run.smithi138.stdout:(112/152): luarocks-3.9.2-1.el9.noarch.rpm 909 kB/s | 151 kB 00:00 2024-04-24T08:35:09.477 INFO:teuthology.orchestra.run.smithi138.stdout:(113/152): python3-asyncssh-2.13.2-5.el9.noarch 7.2 MB/s | 548 kB 00:00 2024-04-24T08:35:09.502 INFO:teuthology.orchestra.run.smithi138.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 595 kB/s | 43 kB 00:00 2024-04-24T08:35:09.528 INFO:teuthology.orchestra.run.smithi138.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 648 kB/s | 32 kB 00:00 2024-04-24T08:35:09.553 INFO:teuthology.orchestra.run.smithi138.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 306 kB/s | 15 kB 00:00 2024-04-24T08:35:09.586 INFO:teuthology.orchestra.run.smithi138.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-04-24T08:35:09.617 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction test succeeded. 2024-04-24T08:35:09.617 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction 2024-04-24T08:35:09.637 INFO:teuthology.orchestra.run.smithi138.stdout:(118/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 2.4 MB/s | 838 kB 00:00 2024-04-24T08:35:09.670 INFO:teuthology.orchestra.run.smithi138.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.6 MB/s | 303 kB 00:00 2024-04-24T08:35:09.704 INFO:teuthology.orchestra.run.smithi138.stdout:(120/152): python3-cherrypy-18.6.1-2.el9.noarch 2.3 MB/s | 358 kB 00:00 2024-04-24T08:35:09.729 INFO:teuthology.orchestra.run.smithi138.stdout:(121/152): python3-colorama-0.4.4-12.el9.noarch 387 kB/s | 35 kB 00:00 2024-04-24T08:35:09.763 INFO:teuthology.orchestra.run.smithi138.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 2.2 MB/s | 206 kB 00:00 2024-04-24T08:35:09.797 INFO:teuthology.orchestra.run.smithi138.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 2.2 MB/s | 144 kB 00:00 2024-04-24T08:35:09.844 INFO:teuthology.orchestra.run.smithi138.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 133 kB/s | 11 kB 00:00 2024-04-24T08:35:09.864 INFO:teuthology.orchestra.run.smithi138.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 268 kB/s | 18 kB 00:00 2024-04-24T08:35:09.897 INFO:teuthology.orchestra.run.smithi138.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 450 kB/s | 23 kB 00:00 2024-04-24T08:35:09.922 INFO:teuthology.orchestra.run.smithi138.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 336 kB/s | 19 kB 00:00 2024-04-24T08:35:09.948 INFO:teuthology.orchestra.run.smithi138.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 399 kB/s | 20 kB 00:00 2024-04-24T08:35:09.998 INFO:teuthology.orchestra.run.smithi138.stdout:(129/152): python3-logutils-0.3.5-21.el9.noarch 942 kB/s | 46 kB 00:00 2024-04-24T08:35:10.048 INFO:teuthology.orchestra.run.smithi138.stdout:(130/152): python3-kubernetes-26.1.0-2.el9.noar 8.2 MB/s | 1.0 MB 00:00 2024-04-24T08:35:10.074 INFO:teuthology.orchestra.run.smithi138.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 1.0 MB/s | 79 kB 00:00 2024-04-24T08:35:10.116 INFO:teuthology.orchestra.run.smithi138.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 874 kB/s | 58 kB 00:00 2024-04-24T08:35:10.149 INFO:teuthology.orchestra.run.smithi138.stdout:(133/152): python3-pecan-1.4.2-3.el9.noarch.rpm 3.6 MB/s | 272 kB 00:00 2024-04-24T08:35:10.175 INFO:teuthology.orchestra.run.smithi138.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 283 kB/s | 16 kB 00:00 2024-04-24T08:35:10.208 INFO:teuthology.orchestra.run.smithi138.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.5 MB/s | 90 kB 00:00 2024-04-24T08:35:10.234 INFO:teuthology.orchestra.run.smithi138.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 494 kB/s | 28 kB 00:00 2024-04-24T08:35:10.259 INFO:teuthology.orchestra.run.smithi138.stdout:(137/152): python3-repoze-lru-0.7-16.el9.noarch 623 kB/s | 31 kB 00:00 2024-04-24T08:35:10.318 INFO:teuthology.orchestra.run.smithi138.stdout:(138/152): python3-grpcio-1.46.7-10.el9.x86_64. 3.5 MB/s | 2.0 MB 00:00 2024-04-24T08:35:10.343 INFO:teuthology.orchestra.run.smithi138.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 714 kB/s | 59 kB 00:00 2024-04-24T08:35:10.376 INFO:teuthology.orchestra.run.smithi138.stdout:(140/152): python3-routes-2.5.1-5.el9.noarch.rp 1.3 MB/s | 188 kB 00:00 2024-04-24T08:35:10.402 INFO:teuthology.orchestra.run.smithi138.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 431 kB/s | 36 kB 00:00 2024-04-24T08:35:10.427 INFO:teuthology.orchestra.run.smithi138.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 614 kB/s | 51 kB 00:00 2024-04-24T08:35:10.469 INFO:teuthology.orchestra.run.smithi138.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 1.3 MB/s | 90 kB 00:00 2024-04-24T08:35:10.503 INFO:teuthology.orchestra.run.smithi138.stdout:(144/152): python3-webob-1.8.7-6.el9.noarch.rpm 1.8 MB/s | 230 kB 00:00 2024-04-24T08:35:10.528 INFO:teuthology.orchestra.run.smithi138.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 347 kB/s | 20 kB 00:00 2024-04-24T08:35:10.570 INFO:teuthology.orchestra.run.smithi138.stdout:(146/152): python3-werkzeug-2.0.3-3.el9.1.noarc 2.9 MB/s | 427 kB 00:00 2024-04-24T08:35:10.612 INFO:teuthology.orchestra.run.smithi138.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 1.7 MB/s | 192 kB 00:00 2024-04-24T08:35:10.879 INFO:teuthology.orchestra.run.smithi138.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 4.5 MB/s | 1.6 MB 00:00 2024-04-24T08:35:11.271 INFO:teuthology.orchestra.run.smithi138.stdout:(149/152): librados2-19.0.0-3116.g1f81fb39.el9. 4.9 MB/s | 3.4 MB 00:00 2024-04-24T08:35:11.563 INFO:teuthology.orchestra.run.smithi138.stdout:(150/152): librbd1-19.0.0-3116.g1f81fb39.el9.x8 3.3 MB/s | 3.1 MB 00:00 2024-04-24T08:35:11.771 INFO:teuthology.orchestra.run.smithi138.stdout:(151/152): openssl-3.2.1-1.el9.x86_64.rpm 1.5 MB/s | 1.3 MB 00:00 2024-04-24T08:35:12.221 INFO:teuthology.orchestra.run.smithi138.stdout:(152/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 2.5 MB/s | 2.4 MB 00:00 2024-04-24T08:35:12.234 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-04-24T08:35:12.234 INFO:teuthology.orchestra.run.smithi138.stdout:Total 14 MB/s | 222 MB 00:16 2024-04-24T08:35:14.603 INFO:teuthology.orchestra.run.smithi189.stdout: Preparing : 1/1 2024-04-24T08:35:14.723 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-24T08:35:14.794 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-24T08:35:14.938 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-24T08:35:14.997 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-04-24T08:35:15.123 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-24T08:35:15.159 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-24T08:35:15.230 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-04-24T08:35:15.230 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-04-24T08:35:15.283 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-24T08:35:15.738 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-24T08:35:15.849 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : librados2-2:19.0.0-3116.g1f81fb39.el9.x86_64 8/156 2024-04-24T08:35:16.007 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: librados2-2:19.0.0-3116.g1f81fb39.el9.x86_64 8/156 2024-04-24T08:35:16.033 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libcephfs2-2:19.0.0-3116.g1f81fb39.el9.x86_64 9/156 2024-04-24T08:35:16.054 INFO:teuthology.orchestra.run.smithi083.stdout: Preparing : 1/1 2024-04-24T08:35:16.148 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: libcephfs2-2:19.0.0-3116.g1f81fb39.el9.x86_64 9/156 2024-04-24T08:35:16.182 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-rados-2:19.0.0-3116.g1f81fb39.el9.x86_64 10/156 2024-04-24T08:35:16.184 INFO:teuthology.orchestra.run.smithi083.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-24T08:35:16.339 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-24T08:35:16.513 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-24T08:35:16.571 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-24T08:35:16.595 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libcephsqlite-2:19.0.0-3116.g1f81fb39.el9.x86_64 13/156 2024-04-24T08:35:16.642 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-24T08:35:16.707 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-24T08:35:16.749 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3116.g1f81fb39.el9.x86_64 13/156 2024-04-24T08:35:16.770 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-24T08:35:16.780 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libradosstriper1-2:19.0.0-3116.g1f81fb39.el9.x86 14/156 2024-04-24T08:35:16.866 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3116.g1f81fb39.el9.x86 14/156 2024-04-24T08:35:16.895 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-24T08:35:16.897 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-24T08:35:16.963 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-24T08:35:17.260 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-24T08:35:17.317 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-24T08:35:17.325 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-24T08:35:17.450 INFO:teuthology.orchestra.run.smithi083.stdout: Upgrading : librados2-2:19.0.0-3116.g1f81fb39.el9.x86_64 8/156 2024-04-24T08:35:17.533 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-24T08:35:17.655 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-24T08:35:17.659 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: librados2-2:19.0.0-3116.g1f81fb39.el9.x86_64 8/156 2024-04-24T08:35:17.693 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libcephfs2-2:19.0.0-3116.g1f81fb39.el9.x86_64 9/156 2024-04-24T08:35:17.698 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-24T08:35:17.747 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-24T08:35:17.828 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: libcephfs2-2:19.0.0-3116.g1f81fb39.el9.x86_64 9/156 2024-04-24T08:35:17.852 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-24T08:35:17.867 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-rados-2:19.0.0-3116.g1f81fb39.el9.x86_64 10/156 2024-04-24T08:35:17.902 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-24T08:35:17.964 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-24T08:35:18.006 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-24T08:35:18.034 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-04-24T08:35:18.035 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-04-24T08:35:18.054 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-24T08:35:18.172 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-24T08:35:18.242 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-24T08:35:18.258 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-24T08:35:18.281 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-24T08:35:18.332 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-24T08:35:18.340 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-24T08:35:18.355 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libcephsqlite-2:19.0.0-3116.g1f81fb39.el9.x86_64 13/156 2024-04-24T08:35:18.446 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-24T08:35:18.485 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-ceph-common-2:19.0.0-3116.g1f81fb39.el9. 31/156 2024-04-24T08:35:18.490 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3116.g1f81fb39.el9.x86_64 13/156 2024-04-24T08:35:18.515 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libradosstriper1-2:19.0.0-3116.g1f81fb39.el9.x86 14/156 2024-04-24T08:35:18.538 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-ceph-argparse-2:19.0.0-3116.g1f81fb39.el 32/156 2024-04-24T08:35:18.575 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-cephfs-2:19.0.0-3116.g1f81fb39.el9.x86_6 33/156 2024-04-24T08:35:18.592 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3116.g1f81fb39.el9.x86 14/156 2024-04-24T08:35:18.624 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-24T08:35:18.690 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-24T08:35:18.721 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-24T08:35:18.754 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-24T08:35:18.852 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-24T08:35:19.026 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-24T08:35:19.040 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-24T08:35:19.100 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-24T08:35:19.110 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-24T08:35:19.286 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-24T08:35:19.410 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-24T08:35:19.476 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-24T08:35:19.525 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-24T08:35:19.630 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-24T08:35:19.678 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-24T08:35:19.743 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-24T08:35:19.791 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-24T08:35:19.804 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-24T08:35:19.866 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-24T08:35:19.976 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-24T08:35:20.044 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-24T08:35:20.095 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-24T08:35:20.160 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-24T08:35:20.226 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-24T08:35:20.277 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-24T08:35:20.294 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-24T08:35:20.329 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-ceph-common-2:19.0.0-3116.g1f81fb39.el9. 31/156 2024-04-24T08:35:20.335 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-24T08:35:20.390 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-ceph-argparse-2:19.0.0-3116.g1f81fb39.el 32/156 2024-04-24T08:35:20.394 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : librados-devel-2:19.0.0-3116.g1f81fb39.el9.x86_6 43/156 2024-04-24T08:35:20.419 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cephfs-2:19.0.0-3116.g1f81fb39.el9.x86_6 33/156 2024-04-24T08:35:20.478 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-24T08:35:20.523 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-24T08:35:20.581 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-24T08:35:20.706 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-24T08:35:20.782 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-24T08:35:20.853 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-24T08:35:20.881 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-24T08:35:20.933 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-24T08:35:20.950 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-24T08:35:21.033 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-24T08:35:21.076 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-24T08:35:21.148 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-24T08:35:21.220 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-24T08:35:21.267 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-24T08:35:21.331 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-24T08:35:21.395 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-24T08:35:21.601 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-24T08:35:21.654 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-24T08:35:21.672 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-24T08:35:21.691 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-24T08:35:21.747 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-24T08:35:21.800 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-24T08:35:21.843 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-24T08:35:21.875 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-24T08:35:21.890 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-24T08:35:21.924 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-24T08:35:21.939 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : librados-devel-2:19.0.0-3116.g1f81fb39.el9.x86_6 43/156 2024-04-24T08:35:21.972 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-24T08:35:22.006 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-24T08:35:22.011 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-24T08:35:22.039 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-24T08:35:22.245 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-24T08:35:22.263 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-24T08:35:22.330 INFO:teuthology.orchestra.run.smithi083.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-24T08:35:22.409 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-24T08:35:22.432 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-24T08:35:22.479 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-24T08:35:22.501 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-24T08:35:22.516 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-24T08:35:22.554 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-24T08:35:22.571 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-24T08:35:22.594 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-24T08:35:22.633 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-24T08:35:22.663 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-24T08:35:22.692 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-24T08:35:22.718 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-24T08:35:22.732 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-24T08:35:22.780 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-24T08:35:22.781 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-24T08:35:22.828 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-24T08:35:22.848 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-24T08:35:22.912 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-24T08:35:22.959 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-24T08:35:23.044 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-24T08:35:23.103 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-24T08:35:23.107 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-24T08:35:23.147 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-24T08:35:23.163 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-24T08:35:23.203 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-24T08:35:23.208 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-24T08:35:23.277 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-24T08:35:23.332 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-24T08:35:23.379 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-24T08:35:23.425 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-24T08:35:23.426 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-24T08:35:23.460 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-24T08:35:23.489 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-24T08:35:23.493 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-24T08:35:23.682 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-24T08:35:23.858 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-24T08:35:23.900 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-24T08:35:23.938 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-24T08:35:23.976 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-24T08:35:24.032 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-24T08:35:24.103 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-24T08:35:24.145 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-24T08:35:24.192 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-24T08:35:24.242 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-24T08:35:24.307 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-04-24T08:35:24.317 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-24T08:35:24.364 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-24T08:35:24.403 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-24T08:35:24.446 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-24T08:35:24.504 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-24T08:35:24.505 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-24T08:35:24.526 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-24T08:35:24.537 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-24T08:35:24.589 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-24T08:35:24.631 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-24T08:35:24.660 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-24T08:35:24.703 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-24T08:35:24.732 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-24T08:35:24.838 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-24T08:35:24.846 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-24T08:35:24.847 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-24T08:35:24.880 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-24T08:35:24.915 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-24T08:35:24.918 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-24T08:35:24.956 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-24T08:35:25.106 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-24T08:35:25.121 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-24T08:35:25.152 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-24T08:35:25.247 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-24T08:35:25.537 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-24T08:35:25.661 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : librados2-2:19.0.0-3116.g1f81fb39.el9.x86_64 8/156 2024-04-24T08:35:25.832 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librados2-2:19.0.0-3116.g1f81fb39.el9.x86_64 8/156 2024-04-24T08:35:25.862 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libcephfs2-2:19.0.0-3116.g1f81fb39.el9.x86_64 9/156 2024-04-24T08:35:25.871 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-24T08:35:25.934 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-24T08:35:25.990 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: libcephfs2-2:19.0.0-3116.g1f81fb39.el9.x86_64 9/156 2024-04-24T08:35:25.993 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-24T08:35:26.028 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-rados-2:19.0.0-3116.g1f81fb39.el9.x86_64 10/156 2024-04-24T08:35:26.075 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-24T08:35:26.122 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-24T08:35:26.196 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-24T08:35:26.218 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-24T08:35:26.296 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-24T08:35:26.304 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-24T08:35:26.336 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-24T08:35:26.375 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-24T08:35:26.400 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libcephsqlite-2:19.0.0-3116.g1f81fb39.el9.x86_64 13/156 2024-04-24T08:35:26.547 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3116.g1f81fb39.el9.x86_64 13/156 2024-04-24T08:35:26.576 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libradosstriper1-2:19.0.0-3116.g1f81fb39.el9.x86 14/156 2024-04-24T08:35:26.596 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-24T08:35:26.633 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-24T08:35:26.662 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3116.g1f81fb39.el9.x86 14/156 2024-04-24T08:35:26.702 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-24T08:35:26.740 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-24T08:35:26.761 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-24T08:35:27.081 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-24T08:35:27.126 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-24T08:35:27.322 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-24T08:35:27.446 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-24T08:35:27.487 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-24T08:35:27.534 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-24T08:35:27.590 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-24T08:35:27.651 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-24T08:35:27.698 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-24T08:35:27.730 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-24T08:35:27.755 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-24T08:35:27.810 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-24T08:35:27.840 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-24T08:35:27.844 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-24T08:35:27.942 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-24T08:35:27.954 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-24T08:35:27.994 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-24T08:35:28.023 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-24T08:35:28.047 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-24T08:35:28.062 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-24T08:35:28.119 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-24T08:35:28.225 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-24T08:35:28.266 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-ceph-common-2:19.0.0-3116.g1f81fb39.el9. 31/156 2024-04-24T08:35:28.314 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-ceph-argparse-2:19.0.0-3116.g1f81fb39.el 32/156 2024-04-24T08:35:28.347 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cephfs-2:19.0.0-3116.g1f81fb39.el9.x86_6 33/156 2024-04-24T08:35:28.434 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-24T08:35:28.469 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-24T08:35:28.480 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : librbd1-2:19.0.0-3116.g1f81fb39.el9.x86_64 99/156 2024-04-24T08:35:28.534 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-24T08:35:28.600 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: librbd1-2:19.0.0-3116.g1f81fb39.el9.x86_64 99/156 2024-04-24T08:35:28.627 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-24T08:35:28.635 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-rbd-2:19.0.0-3116.g1f81fb39.el9.x86_64 100/156 2024-04-24T08:35:28.692 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-24T08:35:28.820 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-24T08:35:28.887 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-24T08:35:29.235 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-24T08:35:29.265 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : librgw2-2:19.0.0-3116.g1f81fb39.el9.x86_64 103/156 2024-04-24T08:35:29.288 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-24T08:35:29.349 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: librgw2-2:19.0.0-3116.g1f81fb39.el9.x86_64 103/156 2024-04-24T08:35:29.384 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-rgw-2:19.0.0-3116.g1f81fb39.el9.x86_64 104/156 2024-04-24T08:35:29.390 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-24T08:35:29.433 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-24T08:35:29.507 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-24T08:35:29.537 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-24T08:35:29.650 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-24T08:35:29.688 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-24T08:35:29.741 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-24T08:35:29.782 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : librados-devel-2:19.0.0-3116.g1f81fb39.el9.x86_6 43/156 2024-04-24T08:35:29.842 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-24T08:35:29.905 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-24T08:35:29.953 INFO:teuthology.orchestra.run.smithi083.stdout: Upgrading : librbd1-2:19.0.0-3116.g1f81fb39.el9.x86_64 99/156 2024-04-24T08:35:30.076 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: librbd1-2:19.0.0-3116.g1f81fb39.el9.x86_64 99/156 2024-04-24T08:35:30.097 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-24T08:35:30.110 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-rbd-2:19.0.0-3116.g1f81fb39.el9.x86_64 100/156 2024-04-24T08:35:30.166 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-24T08:35:30.167 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-24T08:35:30.245 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-24T08:35:30.337 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-24T08:35:30.374 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-24T08:35:30.413 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-24T08:35:30.508 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-24T08:35:30.556 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-24T08:35:30.608 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-24T08:35:30.664 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-24T08:35:30.712 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-24T08:35:30.748 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : librgw2-2:19.0.0-3116.g1f81fb39.el9.x86_64 103/156 2024-04-24T08:35:30.855 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-24T08:35:30.863 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: librgw2-2:19.0.0-3116.g1f81fb39.el9.x86_64 103/156 2024-04-24T08:35:30.891 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-rgw-2:19.0.0-3116.g1f81fb39.el9.x86_64 104/156 2024-04-24T08:35:30.910 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-24T08:35:30.947 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-24T08:35:30.993 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-24T08:35:31.118 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-24T08:35:31.166 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-24T08:35:31.213 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-24T08:35:31.256 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-24T08:35:31.289 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-24T08:35:31.473 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-24T08:35:31.662 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-24T08:35:31.704 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-24T08:35:31.742 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-24T08:35:31.789 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-24T08:35:31.847 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-24T08:35:31.900 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-24T08:35:31.949 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-24T08:35:31.995 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-24T08:35:32.047 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-24T08:35:32.048 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-common-2:19.0.0-3116.g1f81fb39.el9.x86_64 105/156 2024-04-24T08:35:32.088 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-common-2:19.0.0-3116.g1f81fb39.el9.x86_64 105/156 2024-04-24T08:35:32.121 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-24T08:35:32.176 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-24T08:35:32.318 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-24T08:35:32.396 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-24T08:35:32.445 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-24T08:35:32.526 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-24T08:35:32.679 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-24T08:35:32.739 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-24T08:35:32.741 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-common-2:19.0.0-3116.g1f81fb39.el9.x86_64 105/156 2024-04-24T08:35:32.792 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-base-2:19.0.0-3116.g1f81fb39.el9.x86_64 106/156 2024-04-24T08:35:32.947 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-base-2:19.0.0-3116.g1f81fb39.el9.x86_64 106/156 2024-04-24T08:35:32.947 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-24T08:35:32.947 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-24T08:35:32.948 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-04-24T08:35:32.985 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-selinux-2:19.0.0-3116.g1f81fb39.el9.x86_64 107/156 2024-04-24T08:35:33.687 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-24T08:35:33.747 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-24T08:35:33.773 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-common-2:19.0.0-3116.g1f81fb39.el9.x86_64 105/156 2024-04-24T08:35:33.781 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-24T08:35:33.813 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-common-2:19.0.0-3116.g1f81fb39.el9.x86_64 105/156 2024-04-24T08:35:33.920 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-24T08:35:34.000 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-24T08:35:34.127 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-24T08:35:34.166 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-24T08:35:34.398 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-24T08:35:34.421 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-24T08:35:34.452 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-common-2:19.0.0-3116.g1f81fb39.el9.x86_64 105/156 2024-04-24T08:35:34.521 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-24T08:35:34.734 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-base-2:19.0.0-3116.g1f81fb39.el9.x86_64 106/156 2024-04-24T08:35:34.932 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-base-2:19.0.0-3116.g1f81fb39.el9.x86_64 106/156 2024-04-24T08:35:34.932 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-24T08:35:34.932 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-24T08:35:34.932 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:35:34.985 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-selinux-2:19.0.0-3116.g1f81fb39.el9.x86_64 107/156 2024-04-24T08:35:35.323 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-24T08:35:35.461 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-24T08:35:37.036 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-24T08:35:37.137 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-24T08:35:37.178 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-24T08:35:37.216 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-24T08:35:37.594 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-24T08:35:37.642 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : librbd1-2:19.0.0-3116.g1f81fb39.el9.x86_64 99/156 2024-04-24T08:35:37.762 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librbd1-2:19.0.0-3116.g1f81fb39.el9.x86_64 99/156 2024-04-24T08:35:37.798 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-rbd-2:19.0.0-3116.g1f81fb39.el9.x86_64 100/156 2024-04-24T08:35:37.854 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-24T08:35:38.402 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-24T08:35:38.436 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : librgw2-2:19.0.0-3116.g1f81fb39.el9.x86_64 103/156 2024-04-24T08:35:38.552 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librgw2-2:19.0.0-3116.g1f81fb39.el9.x86_64 103/156 2024-04-24T08:35:38.588 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-rgw-2:19.0.0-3116.g1f81fb39.el9.x86_64 104/156 2024-04-24T08:35:41.260 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-common-2:19.0.0-3116.g1f81fb39.el9.x86_64 105/156 2024-04-24T08:35:41.302 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-common-2:19.0.0-3116.g1f81fb39.el9.x86_64 105/156 2024-04-24T08:35:41.960 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-common-2:19.0.0-3116.g1f81fb39.el9.x86_64 105/156 2024-04-24T08:35:42.014 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-base-2:19.0.0-3116.g1f81fb39.el9.x86_64 106/156 2024-04-24T08:35:42.202 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-base-2:19.0.0-3116.g1f81fb39.el9.x86_64 106/156 2024-04-24T08:35:42.202 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-24T08:35:42.202 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-24T08:35:42.202 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-24T08:35:42.248 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-selinux-2:19.0.0-3116.g1f81fb39.el9.x86_64 107/156 2024-04-24T08:35:46.036 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3116.g1f81fb39.el9.x86_64 107/156 2024-04-24T08:35:46.036 INFO:teuthology.orchestra.run.smithi189.stdout:skipping the directory /sys 2024-04-24T08:35:46.036 INFO:teuthology.orchestra.run.smithi189.stdout:skipping the directory /proc 2024-04-24T08:35:46.036 INFO:teuthology.orchestra.run.smithi189.stdout:skipping the directory /mnt 2024-04-24T08:35:46.036 INFO:teuthology.orchestra.run.smithi189.stdout:skipping the directory /var/tmp 2024-04-24T08:35:46.036 INFO:teuthology.orchestra.run.smithi189.stdout:skipping the directory /home 2024-04-24T08:35:46.037 INFO:teuthology.orchestra.run.smithi189.stdout:skipping the directory /root 2024-04-24T08:35:46.037 INFO:teuthology.orchestra.run.smithi189.stdout:skipping the directory /tmp 2024-04-24T08:35:46.037 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-04-24T08:35:46.265 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-volume-2:19.0.0-3116.g1f81fb39.el9.noarch 108/156 2024-04-24T08:35:46.317 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-volume-2:19.0.0-3116.g1f81fb39.el9.noarch 108/156 2024-04-24T08:35:46.317 INFO:teuthology.orchestra.run.smithi189.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T08:35:46.317 INFO:teuthology.orchestra.run.smithi189.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-24T08:35:46.317 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-04-24T08:35:47.960 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3116.g1f81fb39.el9.x86_64 107/156 2024-04-24T08:35:47.960 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /sys 2024-04-24T08:35:47.961 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /proc 2024-04-24T08:35:47.961 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /mnt 2024-04-24T08:35:47.961 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /var/tmp 2024-04-24T08:35:47.961 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /home 2024-04-24T08:35:47.961 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /root 2024-04-24T08:35:47.961 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /tmp 2024-04-24T08:35:47.961 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:35:47.975 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-osd-2:19.0.0-3116.g1f81fb39.el9.x86_64 109/156 2024-04-24T08:35:48.058 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-osd-2:19.0.0-3116.g1f81fb39.el9.x86_64 109/156 2024-04-24T08:35:48.059 INFO:teuthology.orchestra.run.smithi189.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T08:35:48.059 INFO:teuthology.orchestra.run.smithi189.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-24T08:35:48.059 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-24T08:35:48.059 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-24T08:35:48.059 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-04-24T08:35:48.205 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-volume-2:19.0.0-3116.g1f81fb39.el9.noarch 108/156 2024-04-24T08:35:48.255 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-volume-2:19.0.0-3116.g1f81fb39.el9.noarch 108/156 2024-04-24T08:35:48.255 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T08:35:48.255 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-24T08:35:48.255 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:35:48.334 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-mds-2:19.0.0-3116.g1f81fb39.el9.x86_64 110/156 2024-04-24T08:35:48.405 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-mds-2:19.0.0-3116.g1f81fb39.el9.x86_64 110/156 2024-04-24T08:35:48.405 INFO:teuthology.orchestra.run.smithi189.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T08:35:48.405 INFO:teuthology.orchestra.run.smithi189.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-24T08:35:48.405 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-24T08:35:48.405 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-24T08:35:48.405 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-04-24T08:35:48.910 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-mon-2:19.0.0-3116.g1f81fb39.el9.x86_64 111/156 2024-04-24T08:35:48.976 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-mon-2:19.0.0-3116.g1f81fb39.el9.x86_64 111/156 2024-04-24T08:35:48.977 INFO:teuthology.orchestra.run.smithi189.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T08:35:48.977 INFO:teuthology.orchestra.run.smithi189.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-24T08:35:48.977 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-24T08:35:48.977 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-24T08:35:48.977 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-04-24T08:35:49.118 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-24T08:35:49.190 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-24T08:35:49.274 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-24T08:35:49.507 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-24T08:35:49.572 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-24T08:35:49.660 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-24T08:35:49.776 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-24T08:35:49.869 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-osd-2:19.0.0-3116.g1f81fb39.el9.x86_64 109/156 2024-04-24T08:35:49.940 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-24T08:35:49.969 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-osd-2:19.0.0-3116.g1f81fb39.el9.x86_64 109/156 2024-04-24T08:35:49.969 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T08:35:49.969 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-24T08:35:49.969 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-24T08:35:49.969 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-24T08:35:49.969 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:35:49.997 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-24T08:35:50.066 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-24T08:35:50.112 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-24T08:35:50.228 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mds-2:19.0.0-3116.g1f81fb39.el9.x86_64 110/156 2024-04-24T08:35:50.323 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mds-2:19.0.0-3116.g1f81fb39.el9.x86_64 110/156 2024-04-24T08:35:50.323 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T08:35:50.323 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-24T08:35:50.323 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-24T08:35:50.323 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-24T08:35:50.324 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:35:50.703 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: cephadm-2:19.0.0-3116.g1f81fb39.el9.noarch 123/156 2024-04-24T08:35:50.740 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : cephadm-2:19.0.0-3116.g1f81fb39.el9.noarch 123/156 2024-04-24T08:35:50.803 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mon-2:19.0.0-3116.g1f81fb39.el9.x86_64 111/156 2024-04-24T08:35:50.812 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3116.g1f81fb39.e 124/156 2024-04-24T08:35:50.885 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mon-2:19.0.0-3116.g1f81fb39.el9.x86_64 111/156 2024-04-24T08:35:50.885 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T08:35:50.885 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-24T08:35:50.885 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-24T08:35:50.885 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-24T08:35:50.885 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:35:50.924 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3116.g1f81fb39. 125/156 2024-04-24T08:35:50.959 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3116.g1f81fb39.el9.noa 126/156 2024-04-24T08:35:51.020 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-24T08:35:51.093 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-24T08:35:51.168 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-24T08:35:51.399 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-24T08:35:51.466 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-24T08:35:51.550 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-24T08:35:51.652 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-24T08:35:51.832 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-24T08:35:51.891 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-24T08:35:51.933 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-24T08:35:51.974 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-24T08:35:52.274 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3116.g1f81fb39.el9.noa 126/156 2024-04-24T08:35:52.319 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3116.g1f81fb39.el9.n 127/156 2024-04-24T08:35:52.651 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: cephadm-2:19.0.0-3116.g1f81fb39.el9.noarch 123/156 2024-04-24T08:35:52.686 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : cephadm-2:19.0.0-3116.g1f81fb39.el9.noarch 123/156 2024-04-24T08:35:52.731 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3116.g1f81fb39.e 124/156 2024-04-24T08:35:52.838 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3116.g1f81fb39. 125/156 2024-04-24T08:35:52.879 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3116.g1f81fb39.el9.noa 126/156 2024-04-24T08:35:53.418 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3116.g1f81fb39.el9.n 127/156 2024-04-24T08:35:53.444 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3116.g1f8 128/156 2024-04-24T08:35:53.590 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3116.g1f8 128/156 2024-04-24T08:35:53.768 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3116.g1f81fb39.el 129/156 2024-04-24T08:35:53.813 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-mgr-2:19.0.0-3116.g1f81fb39.el9.x86_64 130/156 2024-04-24T08:35:53.905 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3116.g1f81fb39.el9.x86_64 130/156 2024-04-24T08:35:53.906 INFO:teuthology.orchestra.run.smithi189.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T08:35:53.906 INFO:teuthology.orchestra.run.smithi189.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-24T08:35:53.906 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-24T08:35:53.906 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-24T08:35:53.906 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-04-24T08:35:53.981 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-mgr-rook-2:19.0.0-3116.g1f81fb39.el9.noarch 131/156 2024-04-24T08:35:54.020 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3116.g1f81fb39.el9.noarch 131/156 2024-04-24T08:35:54.164 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-2:19.0.0-3116.g1f81fb39.el9.x86_64 132/156 2024-04-24T08:35:54.182 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3116.g1f81fb39.el9.noa 126/156 2024-04-24T08:35:54.222 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3116.g1f81fb39.el9.n 127/156 2024-04-24T08:35:54.233 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-fuse-2:19.0.0-3116.g1f81fb39.el9.x86_64 133/156 2024-04-24T08:35:55.300 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-24T08:35:55.314 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3116.g1f81fb39.el9.n 127/156 2024-04-24T08:35:55.324 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3116.g1f81fb39.el9.x86_64 107/156 2024-04-24T08:35:55.324 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /sys 2024-04-24T08:35:55.324 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /proc 2024-04-24T08:35:55.324 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /mnt 2024-04-24T08:35:55.324 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /var/tmp 2024-04-24T08:35:55.324 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /home 2024-04-24T08:35:55.324 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /root 2024-04-24T08:35:55.324 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /tmp 2024-04-24T08:35:55.324 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-24T08:35:55.333 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-radosgw-2:19.0.0-3116.g1f81fb39.el9.x86_64 135/156 2024-04-24T08:35:55.339 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3116.g1f8 128/156 2024-04-24T08:35:55.405 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3116.g1f81fb39.el9.x86_64 135/156 2024-04-24T08:35:55.406 INFO:teuthology.orchestra.run.smithi189.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T08:35:55.406 INFO:teuthology.orchestra.run.smithi189.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-24T08:35:55.406 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-24T08:35:55.406 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-24T08:35:55.406 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-04-24T08:35:55.459 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3116.g1f81f 136/156 2024-04-24T08:35:55.482 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3116.g1f8 128/156 2024-04-24T08:35:55.553 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3116.g1f81f 136/156 2024-04-24T08:35:55.553 INFO:teuthology.orchestra.run.smithi189.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T08:35:55.553 INFO:teuthology.orchestra.run.smithi189.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-24T08:35:55.553 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-04-24T08:35:55.566 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-volume-2:19.0.0-3116.g1f81fb39.el9.noarch 108/156 2024-04-24T08:35:55.613 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-volume-2:19.0.0-3116.g1f81fb39.el9.noarch 108/156 2024-04-24T08:35:55.613 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T08:35:55.613 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-24T08:35:55.613 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-24T08:35:55.626 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : cephfs-mirror-2:19.0.0-3116.g1f81fb39.el9.x86_64 137/156 2024-04-24T08:35:55.653 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3116.g1f81fb39.el 129/156 2024-04-24T08:35:55.683 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mgr-2:19.0.0-3116.g1f81fb39.el9.x86_64 130/156 2024-04-24T08:35:55.701 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3116.g1f81fb39.el9.x86_64 137/156 2024-04-24T08:35:55.702 INFO:teuthology.orchestra.run.smithi189.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T08:35:55.702 INFO:teuthology.orchestra.run.smithi189.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-24T08:35:55.702 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-24T08:35:55.702 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-24T08:35:55.702 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-04-24T08:35:55.757 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3116.g1f81fb39.el9.x86_64 130/156 2024-04-24T08:35:55.757 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T08:35:55.757 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-24T08:35:55.758 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-24T08:35:55.758 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-24T08:35:55.758 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:35:55.808 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mgr-rook-2:19.0.0-3116.g1f81fb39.el9.noarch 131/156 2024-04-24T08:35:55.845 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3116.g1f81fb39.el9.noarch 131/156 2024-04-24T08:35:55.950 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-2:19.0.0-3116.g1f81fb39.el9.x86_64 132/156 2024-04-24T08:35:56.033 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-fuse-2:19.0.0-3116.g1f81fb39.el9.x86_64 133/156 2024-04-24T08:35:56.077 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : rbd-mirror-2:19.0.0-3116.g1f81fb39.el9.x86_64 138/156 2024-04-24T08:35:56.147 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3116.g1f81fb39.el9.x86_64 138/156 2024-04-24T08:35:56.147 INFO:teuthology.orchestra.run.smithi189.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T08:35:56.147 INFO:teuthology.orchestra.run.smithi189.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-24T08:35:56.147 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-24T08:35:56.147 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-24T08:35:56.147 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-04-24T08:35:57.040 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-24T08:35:57.102 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-radosgw-2:19.0.0-3116.g1f81fb39.el9.x86_64 135/156 2024-04-24T08:35:57.193 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3116.g1f81fb39.el9.x86_64 135/156 2024-04-24T08:35:57.193 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T08:35:57.193 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-24T08:35:57.193 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-24T08:35:57.194 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-24T08:35:57.194 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:35:57.245 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3116.g1f81f 136/156 2024-04-24T08:35:57.250 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-osd-2:19.0.0-3116.g1f81fb39.el9.x86_64 109/156 2024-04-24T08:35:57.329 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-osd-2:19.0.0-3116.g1f81fb39.el9.x86_64 109/156 2024-04-24T08:35:57.329 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T08:35:57.329 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-24T08:35:57.329 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-24T08:35:57.330 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-24T08:35:57.330 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-24T08:35:57.339 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3116.g1f81f 136/156 2024-04-24T08:35:57.339 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T08:35:57.339 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-24T08:35:57.339 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:35:57.413 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : cephfs-mirror-2:19.0.0-3116.g1f81fb39.el9.x86_64 137/156 2024-04-24T08:35:57.510 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3116.g1f81fb39.el9.x86_64 137/156 2024-04-24T08:35:57.510 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T08:35:57.510 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-24T08:35:57.510 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-24T08:35:57.510 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-24T08:35:57.511 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:35:57.592 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mds-2:19.0.0-3116.g1f81fb39.el9.x86_64 110/156 2024-04-24T08:35:57.660 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mds-2:19.0.0-3116.g1f81fb39.el9.x86_64 110/156 2024-04-24T08:35:57.660 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T08:35:57.661 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-24T08:35:57.661 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-24T08:35:57.661 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-24T08:35:57.661 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-24T08:35:57.922 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : rbd-mirror-2:19.0.0-3116.g1f81fb39.el9.x86_64 138/156 2024-04-24T08:35:57.993 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3116.g1f81fb39.el9.x86_64 138/156 2024-04-24T08:35:57.993 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T08:35:57.993 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-24T08:35:57.993 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-24T08:35:57.993 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-24T08:35:57.994 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:35:58.151 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mon-2:19.0.0-3116.g1f81fb39.el9.x86_64 111/156 2024-04-24T08:35:58.219 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mon-2:19.0.0-3116.g1f81fb39.el9.x86_64 111/156 2024-04-24T08:35:58.219 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T08:35:58.219 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-24T08:35:58.219 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-24T08:35:58.219 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-24T08:35:58.219 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-24T08:35:58.351 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-24T08:35:58.414 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-24T08:35:58.498 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-24T08:35:58.741 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-24T08:35:58.806 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-24T08:35:58.890 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-24T08:35:59.009 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-24T08:35:59.173 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-24T08:35:59.228 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-24T08:35:59.272 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-24T08:35:59.313 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-24T08:35:59.935 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: cephadm-2:19.0.0-3116.g1f81fb39.el9.noarch 123/156 2024-04-24T08:35:59.981 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : cephadm-2:19.0.0-3116.g1f81fb39.el9.noarch 123/156 2024-04-24T08:36:00.040 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3116.g1f81fb39.e 124/156 2024-04-24T08:36:00.148 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3116.g1f81fb39. 125/156 2024-04-24T08:36:00.184 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3116.g1f81fb39.el9.noa 126/156 2024-04-24T08:36:01.210 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-test-2:19.0.0-3116.g1f81fb39.el9.x86_64 139/156 2024-04-24T08:36:01.265 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : rbd-fuse-2:19.0.0-3116.g1f81fb39.el9.x86_64 140/156 2024-04-24T08:36:01.351 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : rbd-nbd-2:19.0.0-3116.g1f81fb39.el9.x86_64 141/156 2024-04-24T08:36:01.396 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-24T08:36:01.444 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-24T08:36:01.492 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3116.g1f81fb39.el9.noa 126/156 2024-04-24T08:36:01.545 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3116.g1f81fb39.el9.n 127/156 2024-04-24T08:36:01.773 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : cephfs-shell-2:19.0.0-3116.g1f81fb39.el9.x86_64 144/156 2024-04-24T08:36:01.831 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-24T08:36:01.858 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libcephfs-devel-2:19.0.0-3116.g1f81fb39.el9.x86_ 146/156 2024-04-24T08:36:02.639 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3116.g1f81fb39.el9.n 127/156 2024-04-24T08:36:02.670 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3116.g1f8 128/156 2024-04-24T08:36:02.707 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T08:36:02.757 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T08:36:02.814 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3116.g1f8 128/156 2024-04-24T08:36:02.814 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T08:36:02.993 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3116.g1f81fb39.el 129/156 2024-04-24T08:36:03.016 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-test-2:19.0.0-3116.g1f81fb39.el9.x86_64 139/156 2024-04-24T08:36:03.030 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-2:19.0.0-3116.g1f81fb39.el9.x86_64 130/156 2024-04-24T08:36:03.076 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : rbd-fuse-2:19.0.0-3116.g1f81fb39.el9.x86_64 140/156 2024-04-24T08:36:03.101 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3116.g1f81fb39.el9.x86_64 130/156 2024-04-24T08:36:03.102 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T08:36:03.102 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-24T08:36:03.102 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-24T08:36:03.102 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-24T08:36:03.102 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-24T08:36:03.165 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-rook-2:19.0.0-3116.g1f81fb39.el9.noarch 131/156 2024-04-24T08:36:03.173 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : rbd-nbd-2:19.0.0-3116.g1f81fb39.el9.x86_64 141/156 2024-04-24T08:36:03.202 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3116.g1f81fb39.el9.noarch 131/156 2024-04-24T08:36:03.216 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-24T08:36:03.288 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-24T08:36:03.347 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-2:19.0.0-3116.g1f81fb39.el9.x86_64 132/156 2024-04-24T08:36:03.418 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-fuse-2:19.0.0-3116.g1f81fb39.el9.x86_64 133/156 2024-04-24T08:36:03.603 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : cephfs-shell-2:19.0.0-3116.g1f81fb39.el9.x86_64 144/156 2024-04-24T08:36:03.680 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-24T08:36:03.710 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libcephfs-devel-2:19.0.0-3116.g1f81fb39.el9.x86_ 146/156 2024-04-24T08:36:04.175 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : cephfs-top-2:19.0.0-3116.g1f81fb39.el9.noarch 148/156 2024-04-24T08:36:04.288 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-24T08:36:04.385 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-24T08:36:04.505 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-24T08:36:04.508 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T08:36:04.559 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T08:36:04.568 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-radosgw-2:19.0.0-3116.g1f81fb39.el9.x86_64 135/156 2024-04-24T08:36:04.589 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-24T08:36:04.615 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T08:36:04.636 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-24T08:36:04.636 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-24T08:36:04.636 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3116.g1f81fb39.el9.x86_64 135/156 2024-04-24T08:36:04.637 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T08:36:04.637 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-24T08:36:04.637 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-24T08:36:04.637 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-24T08:36:04.637 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-24T08:36:04.693 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3116.g1f81f 136/156 2024-04-24T08:36:04.755 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-24T08:36:04.763 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3116.g1f81f 136/156 2024-04-24T08:36:04.763 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T08:36:04.763 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-24T08:36:04.763 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-24T08:36:04.802 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-24T08:36:04.802 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-24T08:36:04.835 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : cephfs-mirror-2:19.0.0-3116.g1f81fb39.el9.x86_64 137/156 2024-04-24T08:36:04.913 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3116.g1f81fb39.el9.x86_64 137/156 2024-04-24T08:36:04.913 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T08:36:04.913 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-24T08:36:04.913 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-24T08:36:04.913 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-24T08:36:04.913 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-24T08:36:04.933 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-24T08:36:05.064 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T08:36:05.286 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : rbd-mirror-2:19.0.0-3116.g1f81fb39.el9.x86_64 138/156 2024-04-24T08:36:05.351 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T08:36:05.351 INFO:teuthology.orchestra.run.smithi189.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-04-24T08:36:05.351 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-04-24T08:36:05.371 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3116.g1f81fb39.el9.x86_64 138/156 2024-04-24T08:36:05.371 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T08:36:05.371 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-24T08:36:05.371 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-24T08:36:05.371 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-24T08:36:05.371 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-24T08:36:05.923 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : cephfs-top-2:19.0.0-3116.g1f81fb39.el9.noarch 148/156 2024-04-24T08:36:06.041 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-24T08:36:06.146 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-24T08:36:06.339 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-24T08:36:06.381 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-24T08:36:06.381 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-24T08:36:06.539 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-24T08:36:06.588 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-24T08:36:06.588 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-24T08:36:06.728 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-24T08:36:06.859 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T08:36:07.150 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T08:36:07.150 INFO:teuthology.orchestra.run.smithi083.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-04-24T08:36:07.150 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:36:10.526 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-test-2:19.0.0-3116.g1f81fb39.el9.x86_64 139/156 2024-04-24T08:36:10.586 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : rbd-fuse-2:19.0.0-3116.g1f81fb39.el9.x86_64 140/156 2024-04-24T08:36:10.712 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : rbd-nbd-2:19.0.0-3116.g1f81fb39.el9.x86_64 141/156 2024-04-24T08:36:10.760 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-24T08:36:10.807 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-24T08:36:10.915 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-2:19.0.0-3116.g1f81fb39.el9.x86_64 1/156 2024-04-24T08:36:10.915 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-base-2:19.0.0-3116.g1f81fb39.el9.x86_64 2/156 2024-04-24T08:36:10.915 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-common-2:19.0.0-3116.g1f81fb39.el9.x86_64 3/156 2024-04-24T08:36:10.915 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-fuse-2:19.0.0-3116.g1f81fb39.el9.x86_64 4/156 2024-04-24T08:36:10.915 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3116.g1f81f 5/156 2024-04-24T08:36:10.915 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-mds-2:19.0.0-3116.g1f81fb39.el9.x86_64 6/156 2024-04-24T08:36:10.915 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-mgr-2:19.0.0-3116.g1f81fb39.el9.x86_64 7/156 2024-04-24T08:36:10.915 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-mon-2:19.0.0-3116.g1f81fb39.el9.x86_64 8/156 2024-04-24T08:36:10.915 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-osd-2:19.0.0-3116.g1f81fb39.el9.x86_64 9/156 2024-04-24T08:36:10.915 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-radosgw-2:19.0.0-3116.g1f81fb39.el9.x86_64 10/156 2024-04-24T08:36:10.915 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-selinux-2:19.0.0-3116.g1f81fb39.el9.x86_64 11/156 2024-04-24T08:36:10.915 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-test-2:19.0.0-3116.g1f81fb39.el9.x86_64 12/156 2024-04-24T08:36:10.916 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : cephfs-mirror-2:19.0.0-3116.g1f81fb39.el9.x86_64 13/156 2024-04-24T08:36:10.916 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : cephfs-shell-2:19.0.0-3116.g1f81fb39.el9.x86_64 14/156 2024-04-24T08:36:10.916 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libcephfs-devel-2:19.0.0-3116.g1f81fb39.el9.x86_ 15/156 2024-04-24T08:36:10.916 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libcephfs2-2:19.0.0-3116.g1f81fb39.el9.x86_64 16/156 2024-04-24T08:36:10.916 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libcephsqlite-2:19.0.0-3116.g1f81fb39.el9.x86_64 17/156 2024-04-24T08:36:10.916 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : librados-devel-2:19.0.0-3116.g1f81fb39.el9.x86_6 18/156 2024-04-24T08:36:10.916 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libradosstriper1-2:19.0.0-3116.g1f81fb39.el9.x86 19/156 2024-04-24T08:36:10.916 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : librgw2-2:19.0.0-3116.g1f81fb39.el9.x86_64 20/156 2024-04-24T08:36:10.916 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3116.g1f81fb39.el 21/156 2024-04-24T08:36:10.916 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-ceph-common-2:19.0.0-3116.g1f81fb39.el9. 22/156 2024-04-24T08:36:10.916 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-cephfs-2:19.0.0-3116.g1f81fb39.el9.x86_6 23/156 2024-04-24T08:36:10.916 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-rados-2:19.0.0-3116.g1f81fb39.el9.x86_64 24/156 2024-04-24T08:36:10.916 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-rbd-2:19.0.0-3116.g1f81fb39.el9.x86_64 25/156 2024-04-24T08:36:10.929 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-rgw-2:19.0.0-3116.g1f81fb39.el9.x86_64 26/156 2024-04-24T08:36:10.929 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : rbd-fuse-2:19.0.0-3116.g1f81fb39.el9.x86_64 27/156 2024-04-24T08:36:10.929 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : rbd-mirror-2:19.0.0-3116.g1f81fb39.el9.x86_64 28/156 2024-04-24T08:36:10.929 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : rbd-nbd-2:19.0.0-3116.g1f81fb39.el9.x86_64 29/156 2024-04-24T08:36:10.929 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3116.g1f81fb39. 30/156 2024-04-24T08:36:10.929 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3116.g1f81fb39.el9.noa 31/156 2024-04-24T08:36:10.929 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3116.g1f81fb39.el9.n 32/156 2024-04-24T08:36:10.929 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3116.g1f8 33/156 2024-04-24T08:36:10.929 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3116.g1f81fb39.el 34/156 2024-04-24T08:36:10.929 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3116.g1f81fb39.el9.noarch 35/156 2024-04-24T08:36:10.929 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3116.g1f81fb39.e 36/156 2024-04-24T08:36:10.929 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-volume-2:19.0.0-3116.g1f81fb39.el9.noarch 37/156 2024-04-24T08:36:10.929 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : cephadm-2:19.0.0-3116.g1f81fb39.el9.noarch 38/156 2024-04-24T08:36:10.930 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : cephfs-top-2:19.0.0-3116.g1f81fb39.el9.noarch 39/156 2024-04-24T08:36:10.930 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-24T08:36:10.930 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-24T08:36:10.930 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-24T08:36:10.930 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-24T08:36:10.930 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-24T08:36:10.930 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-24T08:36:10.930 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-24T08:36:10.930 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-24T08:36:10.930 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-24T08:36:10.930 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-24T08:36:10.930 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-24T08:36:10.931 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-24T08:36:10.931 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-24T08:36:10.931 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-24T08:36:10.931 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-24T08:36:10.931 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-24T08:36:10.931 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-24T08:36:10.931 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-24T08:36:10.931 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-24T08:36:10.931 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-24T08:36:10.931 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-24T08:36:10.931 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-24T08:36:10.931 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-24T08:36:10.931 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-24T08:36:10.932 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-24T08:36:10.932 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-24T08:36:10.932 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-24T08:36:10.932 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-24T08:36:10.932 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-24T08:36:10.932 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-24T08:36:10.932 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-24T08:36:10.932 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-24T08:36:10.932 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-24T08:36:10.932 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-24T08:36:10.932 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-24T08:36:10.932 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-24T08:36:10.932 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-24T08:36:10.932 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-24T08:36:10.933 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-24T08:36:10.933 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-24T08:36:10.933 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-24T08:36:10.933 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-24T08:36:10.933 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-24T08:36:10.933 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-24T08:36:10.933 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-24T08:36:10.933 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-24T08:36:10.933 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-24T08:36:10.933 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-24T08:36:10.933 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-24T08:36:10.933 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-24T08:36:10.933 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-24T08:36:10.934 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-24T08:36:10.934 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-24T08:36:10.934 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-24T08:36:10.934 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-24T08:36:10.934 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-24T08:36:10.934 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-24T08:36:10.934 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-24T08:36:10.934 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-24T08:36:10.934 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-24T08:36:10.934 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-24T08:36:10.934 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-24T08:36:10.934 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-24T08:36:10.934 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-24T08:36:10.934 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-24T08:36:10.935 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-24T08:36:10.935 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-24T08:36:10.935 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-24T08:36:10.935 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-24T08:36:10.935 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-24T08:36:10.935 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-24T08:36:10.935 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-24T08:36:10.935 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-24T08:36:10.935 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-24T08:36:10.935 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-24T08:36:10.935 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-24T08:36:10.935 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-24T08:36:10.935 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-24T08:36:10.935 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-24T08:36:10.935 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-24T08:36:10.936 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-24T08:36:10.936 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-24T08:36:10.936 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-24T08:36:10.936 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-24T08:36:10.936 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-24T08:36:10.936 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-24T08:36:10.936 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-24T08:36:10.936 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-24T08:36:10.936 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-24T08:36:10.936 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-24T08:36:10.936 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-24T08:36:10.936 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-24T08:36:10.936 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-24T08:36:10.936 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-24T08:36:10.937 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-24T08:36:10.937 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-24T08:36:10.937 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-24T08:36:10.937 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-24T08:36:10.937 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-24T08:36:10.937 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-24T08:36:10.937 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-24T08:36:10.937 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-24T08:36:10.937 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-24T08:36:10.937 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-24T08:36:10.937 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-24T08:36:10.937 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-24T08:36:10.937 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-24T08:36:10.937 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-24T08:36:10.937 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-24T08:36:10.937 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : librados2-2:19.0.0-3116.g1f81fb39.el9.x86_64 149/156 2024-04-24T08:36:10.938 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-24T08:36:10.938 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : librbd1-2:19.0.0-3116.g1f81fb39.el9.x86_64 151/156 2024-04-24T08:36:10.938 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-24T08:36:10.938 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-24T08:36:10.938 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-24T08:36:10.938 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-24T08:36:11.122 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : cephfs-shell-2:19.0.0-3116.g1f81fb39.el9.x86_64 144/156 2024-04-24T08:36:11.183 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-24T08:36:11.220 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libcephfs-devel-2:19.0.0-3116.g1f81fb39.el9.x86_ 146/156 2024-04-24T08:36:12.144 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T08:36:12.193 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T08:36:12.253 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T08:36:13.020 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-2:19.0.0-3116.g1f81fb39.el9.x86_64 1/156 2024-04-24T08:36:13.020 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-base-2:19.0.0-3116.g1f81fb39.el9.x86_64 2/156 2024-04-24T08:36:13.020 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-common-2:19.0.0-3116.g1f81fb39.el9.x86_64 3/156 2024-04-24T08:36:13.020 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-fuse-2:19.0.0-3116.g1f81fb39.el9.x86_64 4/156 2024-04-24T08:36:13.020 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3116.g1f81f 5/156 2024-04-24T08:36:13.020 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mds-2:19.0.0-3116.g1f81fb39.el9.x86_64 6/156 2024-04-24T08:36:13.020 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mgr-2:19.0.0-3116.g1f81fb39.el9.x86_64 7/156 2024-04-24T08:36:13.020 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mon-2:19.0.0-3116.g1f81fb39.el9.x86_64 8/156 2024-04-24T08:36:13.020 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-osd-2:19.0.0-3116.g1f81fb39.el9.x86_64 9/156 2024-04-24T08:36:13.020 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-radosgw-2:19.0.0-3116.g1f81fb39.el9.x86_64 10/156 2024-04-24T08:36:13.020 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-selinux-2:19.0.0-3116.g1f81fb39.el9.x86_64 11/156 2024-04-24T08:36:13.020 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-test-2:19.0.0-3116.g1f81fb39.el9.x86_64 12/156 2024-04-24T08:36:13.021 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : cephfs-mirror-2:19.0.0-3116.g1f81fb39.el9.x86_64 13/156 2024-04-24T08:36:13.021 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : cephfs-shell-2:19.0.0-3116.g1f81fb39.el9.x86_64 14/156 2024-04-24T08:36:13.021 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libcephfs-devel-2:19.0.0-3116.g1f81fb39.el9.x86_ 15/156 2024-04-24T08:36:13.021 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libcephfs2-2:19.0.0-3116.g1f81fb39.el9.x86_64 16/156 2024-04-24T08:36:13.021 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libcephsqlite-2:19.0.0-3116.g1f81fb39.el9.x86_64 17/156 2024-04-24T08:36:13.021 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librados-devel-2:19.0.0-3116.g1f81fb39.el9.x86_6 18/156 2024-04-24T08:36:13.021 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libradosstriper1-2:19.0.0-3116.g1f81fb39.el9.x86 19/156 2024-04-24T08:36:13.021 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librgw2-2:19.0.0-3116.g1f81fb39.el9.x86_64 20/156 2024-04-24T08:36:13.021 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3116.g1f81fb39.el 21/156 2024-04-24T08:36:13.021 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-ceph-common-2:19.0.0-3116.g1f81fb39.el9. 22/156 2024-04-24T08:36:13.021 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cephfs-2:19.0.0-3116.g1f81fb39.el9.x86_6 23/156 2024-04-24T08:36:13.021 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-rados-2:19.0.0-3116.g1f81fb39.el9.x86_64 24/156 2024-04-24T08:36:13.021 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-rbd-2:19.0.0-3116.g1f81fb39.el9.x86_64 25/156 2024-04-24T08:36:13.022 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-rgw-2:19.0.0-3116.g1f81fb39.el9.x86_64 26/156 2024-04-24T08:36:13.022 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : rbd-fuse-2:19.0.0-3116.g1f81fb39.el9.x86_64 27/156 2024-04-24T08:36:13.033 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : rbd-mirror-2:19.0.0-3116.g1f81fb39.el9.x86_64 28/156 2024-04-24T08:36:13.034 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : rbd-nbd-2:19.0.0-3116.g1f81fb39.el9.x86_64 29/156 2024-04-24T08:36:13.034 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3116.g1f81fb39. 30/156 2024-04-24T08:36:13.034 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3116.g1f81fb39.el9.noa 31/156 2024-04-24T08:36:13.034 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3116.g1f81fb39.el9.n 32/156 2024-04-24T08:36:13.034 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3116.g1f8 33/156 2024-04-24T08:36:13.034 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3116.g1f81fb39.el 34/156 2024-04-24T08:36:13.034 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3116.g1f81fb39.el9.noarch 35/156 2024-04-24T08:36:13.034 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3116.g1f81fb39.e 36/156 2024-04-24T08:36:13.034 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-volume-2:19.0.0-3116.g1f81fb39.el9.noarch 37/156 2024-04-24T08:36:13.034 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : cephadm-2:19.0.0-3116.g1f81fb39.el9.noarch 38/156 2024-04-24T08:36:13.034 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : cephfs-top-2:19.0.0-3116.g1f81fb39.el9.noarch 39/156 2024-04-24T08:36:13.034 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-24T08:36:13.035 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-24T08:36:13.035 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-24T08:36:13.035 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-24T08:36:13.035 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-24T08:36:13.035 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-24T08:36:13.035 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-24T08:36:13.035 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-24T08:36:13.035 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-24T08:36:13.035 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-24T08:36:13.035 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-24T08:36:13.035 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-24T08:36:13.035 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-24T08:36:13.035 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-24T08:36:13.035 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-24T08:36:13.035 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-24T08:36:13.036 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-24T08:36:13.036 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-24T08:36:13.036 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-24T08:36:13.036 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-24T08:36:13.036 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-24T08:36:13.036 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-24T08:36:13.036 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-24T08:36:13.036 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-24T08:36:13.036 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-24T08:36:13.036 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-24T08:36:13.036 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-24T08:36:13.036 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-24T08:36:13.036 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-24T08:36:13.037 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-24T08:36:13.037 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-24T08:36:13.037 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-24T08:36:13.037 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-24T08:36:13.037 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-24T08:36:13.037 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-24T08:36:13.037 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-24T08:36:13.037 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-24T08:36:13.037 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-24T08:36:13.037 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-24T08:36:13.037 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-24T08:36:13.037 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-24T08:36:13.038 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-24T08:36:13.038 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-24T08:36:13.038 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-24T08:36:13.038 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-24T08:36:13.038 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-24T08:36:13.038 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-24T08:36:13.038 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-24T08:36:13.038 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-24T08:36:13.038 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-24T08:36:13.038 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-24T08:36:13.038 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-24T08:36:13.038 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-24T08:36:13.038 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-24T08:36:13.038 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-24T08:36:13.039 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-24T08:36:13.039 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-24T08:36:13.039 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-24T08:36:13.039 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-24T08:36:13.039 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-24T08:36:13.039 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-24T08:36:13.039 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-24T08:36:13.039 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-24T08:36:13.039 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-24T08:36:13.039 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-24T08:36:13.039 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-24T08:36:13.039 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-24T08:36:13.039 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-24T08:36:13.039 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-24T08:36:13.040 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-24T08:36:13.040 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-24T08:36:13.040 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-24T08:36:13.040 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-24T08:36:13.040 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-24T08:36:13.040 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-24T08:36:13.040 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-24T08:36:13.040 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-24T08:36:13.040 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-24T08:36:13.040 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-24T08:36:13.040 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-24T08:36:13.040 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-24T08:36:13.040 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-24T08:36:13.040 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-24T08:36:13.041 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-24T08:36:13.041 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-24T08:36:13.041 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-24T08:36:13.041 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-24T08:36:13.041 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-24T08:36:13.041 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-24T08:36:13.041 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-24T08:36:13.041 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-24T08:36:13.041 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-24T08:36:13.041 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-24T08:36:13.041 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-24T08:36:13.041 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-24T08:36:13.041 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-24T08:36:13.041 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-24T08:36:13.042 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-24T08:36:13.042 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-24T08:36:13.042 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-24T08:36:13.042 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-24T08:36:13.042 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-24T08:36:13.042 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-24T08:36:13.042 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-24T08:36:13.042 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-24T08:36:13.042 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-24T08:36:13.042 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-24T08:36:13.042 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-24T08:36:13.042 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-24T08:36:13.042 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librados2-2:19.0.0-3116.g1f81fb39.el9.x86_64 149/156 2024-04-24T08:36:13.042 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-24T08:36:13.043 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librbd1-2:19.0.0-3116.g1f81fb39.el9.x86_64 151/156 2024-04-24T08:36:13.043 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-24T08:36:13.043 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-24T08:36:13.043 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-24T08:36:13.043 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-24T08:36:13.347 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T08:36:13.347 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-04-24T08:36:13.347 INFO:teuthology.orchestra.run.smithi189.stdout:Upgraded: 2024-04-24T08:36:13.348 INFO:teuthology.orchestra.run.smithi189.stdout: librados2-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:36:13.348 INFO:teuthology.orchestra.run.smithi189.stdout: librbd1-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:36:13.348 INFO:teuthology.orchestra.run.smithi189.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-24T08:36:13.348 INFO:teuthology.orchestra.run.smithi189.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-24T08:36:13.348 INFO:teuthology.orchestra.run.smithi189.stdout:Installed: 2024-04-24T08:36:13.348 INFO:teuthology.orchestra.run.smithi189.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-24T08:36:13.348 INFO:teuthology.orchestra.run.smithi189.stdout: automake-1.16.2-8.el9.noarch 2024-04-24T08:36:13.348 INFO:teuthology.orchestra.run.smithi189.stdout: bison-3.7.4-5.el9.x86_64 2024-04-24T08:36:13.348 INFO:teuthology.orchestra.run.smithi189.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-24T08:36:13.348 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:36:13.348 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-base-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:36:13.348 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-common-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:36:13.348 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-fuse-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:36:13.348 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-grafana-dashboards-2:19.0.0-3116.g1f81fb39.el9.noarch 2024-04-24T08:36:13.349 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-immutable-object-cache-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:36:13.349 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mds-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:36:13.349 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:36:13.349 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-cephadm-2:19.0.0-3116.g1f81fb39.el9.noarch 2024-04-24T08:36:13.349 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-dashboard-2:19.0.0-3116.g1f81fb39.el9.noarch 2024-04-24T08:36:13.349 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3116.g1f81fb39.el9.noarch 2024-04-24T08:36:13.349 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-modules-core-2:19.0.0-3116.g1f81fb39.el9.noarch 2024-04-24T08:36:13.349 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-rook-2:19.0.0-3116.g1f81fb39.el9.noarch 2024-04-24T08:36:13.349 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mon-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:36:13.349 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-osd-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:36:13.349 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-prometheus-alerts-2:19.0.0-3116.g1f81fb39.el9.noarch 2024-04-24T08:36:13.349 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-radosgw-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:36:13.349 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-selinux-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:36:13.349 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-test-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:36:13.350 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-volume-2:19.0.0-3116.g1f81fb39.el9.noarch 2024-04-24T08:36:13.350 INFO:teuthology.orchestra.run.smithi189.stdout: cephadm-2:19.0.0-3116.g1f81fb39.el9.noarch 2024-04-24T08:36:13.350 INFO:teuthology.orchestra.run.smithi189.stdout: cephfs-mirror-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:36:13.350 INFO:teuthology.orchestra.run.smithi189.stdout: cephfs-shell-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:36:13.350 INFO:teuthology.orchestra.run.smithi189.stdout: cephfs-top-2:19.0.0-3116.g1f81fb39.el9.noarch 2024-04-24T08:36:13.350 INFO:teuthology.orchestra.run.smithi189.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-24T08:36:13.350 INFO:teuthology.orchestra.run.smithi189.stdout: flex-2.6.4-9.el9.x86_64 2024-04-24T08:36:13.350 INFO:teuthology.orchestra.run.smithi189.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-24T08:36:13.350 INFO:teuthology.orchestra.run.smithi189.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-24T08:36:13.350 INFO:teuthology.orchestra.run.smithi189.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-24T08:36:13.350 INFO:teuthology.orchestra.run.smithi189.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-24T08:36:13.350 INFO:teuthology.orchestra.run.smithi189.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-24T08:36:13.351 INFO:teuthology.orchestra.run.smithi189.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-24T08:36:13.351 INFO:teuthology.orchestra.run.smithi189.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-24T08:36:13.351 INFO:teuthology.orchestra.run.smithi189.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-24T08:36:13.351 INFO:teuthology.orchestra.run.smithi189.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-24T08:36:13.351 INFO:teuthology.orchestra.run.smithi189.stdout: libcephfs-devel-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:36:13.351 INFO:teuthology.orchestra.run.smithi189.stdout: libcephfs2-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:36:13.351 INFO:teuthology.orchestra.run.smithi189.stdout: libcephsqlite-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:36:13.351 INFO:teuthology.orchestra.run.smithi189.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-24T08:36:13.351 INFO:teuthology.orchestra.run.smithi189.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-24T08:36:13.351 INFO:teuthology.orchestra.run.smithi189.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-24T08:36:13.351 INFO:teuthology.orchestra.run.smithi189.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-24T08:36:13.351 INFO:teuthology.orchestra.run.smithi189.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-24T08:36:13.351 INFO:teuthology.orchestra.run.smithi189.stdout: librados-devel-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:36:13.351 INFO:teuthology.orchestra.run.smithi189.stdout: libradosstriper1-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:36:13.352 INFO:teuthology.orchestra.run.smithi189.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-24T08:36:13.352 INFO:teuthology.orchestra.run.smithi189.stdout: librgw2-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:36:13.352 INFO:teuthology.orchestra.run.smithi189.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-24T08:36:13.352 INFO:teuthology.orchestra.run.smithi189.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-24T08:36:13.352 INFO:teuthology.orchestra.run.smithi189.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-24T08:36:13.352 INFO:teuthology.orchestra.run.smithi189.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-24T08:36:13.352 INFO:teuthology.orchestra.run.smithi189.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-24T08:36:13.352 INFO:teuthology.orchestra.run.smithi189.stdout: lua-5.4.4-4.el9.x86_64 2024-04-24T08:36:13.352 INFO:teuthology.orchestra.run.smithi189.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-24T08:36:13.352 INFO:teuthology.orchestra.run.smithi189.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-24T08:36:13.352 INFO:teuthology.orchestra.run.smithi189.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-24T08:36:13.352 INFO:teuthology.orchestra.run.smithi189.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-24T08:36:13.352 INFO:teuthology.orchestra.run.smithi189.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-24T08:36:13.353 INFO:teuthology.orchestra.run.smithi189.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-24T08:36:13.353 INFO:teuthology.orchestra.run.smithi189.stdout: postgresql-13.14-1.el9.x86_64 2024-04-24T08:36:13.353 INFO:teuthology.orchestra.run.smithi189.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-24T08:36:13.353 INFO:teuthology.orchestra.run.smithi189.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-24T08:36:13.353 INFO:teuthology.orchestra.run.smithi189.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-24T08:36:13.353 INFO:teuthology.orchestra.run.smithi189.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-24T08:36:13.353 INFO:teuthology.orchestra.run.smithi189.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-24T08:36:13.353 INFO:teuthology.orchestra.run.smithi189.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-24T08:36:13.353 INFO:teuthology.orchestra.run.smithi189.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-24T08:36:13.353 INFO:teuthology.orchestra.run.smithi189.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-24T08:36:13.353 INFO:teuthology.orchestra.run.smithi189.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-24T08:36:13.353 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-24T08:36:13.353 INFO:teuthology.orchestra.run.smithi189.stdout: python3-ceph-argparse-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:36:13.353 INFO:teuthology.orchestra.run.smithi189.stdout: python3-ceph-common-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:36:13.353 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cephfs-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:36:13.354 INFO:teuthology.orchestra.run.smithi189.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-24T08:36:13.354 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-24T08:36:13.354 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-24T08:36:13.354 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-24T08:36:13.354 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-24T08:36:13.354 INFO:teuthology.orchestra.run.smithi189.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-24T08:36:13.354 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-24T08:36:13.354 INFO:teuthology.orchestra.run.smithi189.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-24T08:36:13.354 INFO:teuthology.orchestra.run.smithi189.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-24T08:36:13.354 INFO:teuthology.orchestra.run.smithi189.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-24T08:36:13.354 INFO:teuthology.orchestra.run.smithi189.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-24T08:36:13.354 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-24T08:36:13.354 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-24T08:36:13.355 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-24T08:36:13.355 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-24T08:36:13.355 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-24T08:36:13.355 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-24T08:36:13.355 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-24T08:36:13.355 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-24T08:36:13.355 INFO:teuthology.orchestra.run.smithi189.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-24T08:36:13.355 INFO:teuthology.orchestra.run.smithi189.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-24T08:36:13.355 INFO:teuthology.orchestra.run.smithi189.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-24T08:36:13.355 INFO:teuthology.orchestra.run.smithi189.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-24T08:36:13.355 INFO:teuthology.orchestra.run.smithi189.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-24T08:36:13.355 INFO:teuthology.orchestra.run.smithi189.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-24T08:36:13.355 INFO:teuthology.orchestra.run.smithi189.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-24T08:36:13.355 INFO:teuthology.orchestra.run.smithi189.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-24T08:36:13.355 INFO:teuthology.orchestra.run.smithi189.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-24T08:36:13.356 INFO:teuthology.orchestra.run.smithi189.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-24T08:36:13.356 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-24T08:36:13.356 INFO:teuthology.orchestra.run.smithi189.stdout: python3-ply-3.11-14.el9.noarch 2024-04-24T08:36:13.356 INFO:teuthology.orchestra.run.smithi189.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-24T08:36:13.356 INFO:teuthology.orchestra.run.smithi189.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-24T08:36:13.356 INFO:teuthology.orchestra.run.smithi189.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-24T08:36:13.356 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-24T08:36:13.356 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-24T08:36:13.356 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-24T08:36:13.356 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-24T08:36:13.356 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-24T08:36:13.356 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-24T08:36:13.356 INFO:teuthology.orchestra.run.smithi189.stdout: python3-rados-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:36:13.356 INFO:teuthology.orchestra.run.smithi189.stdout: python3-rbd-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:36:13.356 INFO:teuthology.orchestra.run.smithi189.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-24T08:36:13.357 INFO:teuthology.orchestra.run.smithi189.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-24T08:36:13.357 INFO:teuthology.orchestra.run.smithi189.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-24T08:36:13.357 INFO:teuthology.orchestra.run.smithi189.stdout: python3-rgw-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:36:13.357 INFO:teuthology.orchestra.run.smithi189.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-24T08:36:13.357 INFO:teuthology.orchestra.run.smithi189.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-24T08:36:13.357 INFO:teuthology.orchestra.run.smithi189.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-24T08:36:13.357 INFO:teuthology.orchestra.run.smithi189.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-24T08:36:13.357 INFO:teuthology.orchestra.run.smithi189.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-24T08:36:13.357 INFO:teuthology.orchestra.run.smithi189.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-24T08:36:13.357 INFO:teuthology.orchestra.run.smithi189.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-24T08:36:13.357 INFO:teuthology.orchestra.run.smithi189.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-24T08:36:13.357 INFO:teuthology.orchestra.run.smithi189.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-24T08:36:13.357 INFO:teuthology.orchestra.run.smithi189.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-24T08:36:13.357 INFO:teuthology.orchestra.run.smithi189.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-24T08:36:13.358 INFO:teuthology.orchestra.run.smithi189.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-24T08:36:13.358 INFO:teuthology.orchestra.run.smithi189.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-24T08:36:13.358 INFO:teuthology.orchestra.run.smithi189.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-24T08:36:13.358 INFO:teuthology.orchestra.run.smithi189.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-24T08:36:13.358 INFO:teuthology.orchestra.run.smithi189.stdout: rbd-fuse-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:36:13.358 INFO:teuthology.orchestra.run.smithi189.stdout: rbd-mirror-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:36:13.358 INFO:teuthology.orchestra.run.smithi189.stdout: rbd-nbd-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:36:13.358 INFO:teuthology.orchestra.run.smithi189.stdout: re2-1:20211101-3.el9.x86_64 2024-04-24T08:36:13.358 INFO:teuthology.orchestra.run.smithi189.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-24T08:36:13.358 INFO:teuthology.orchestra.run.smithi189.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-24T08:36:13.358 INFO:teuthology.orchestra.run.smithi189.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-24T08:36:13.358 INFO:teuthology.orchestra.run.smithi189.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-24T08:36:13.358 INFO:teuthology.orchestra.run.smithi189.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-24T08:36:13.359 INFO:teuthology.orchestra.run.smithi189.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-24T08:36:13.359 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-04-24T08:36:13.359 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2024-04-24T08:36:13.601 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : cephfs-top-2:19.0.0-3116.g1f81fb39.el9.noarch 148/156 2024-04-24T08:36:13.717 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-24T08:36:13.805 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-24T08:36:13.967 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-24T08:36:14.017 DEBUG:teuthology.parallel:result is None 2024-04-24T08:36:14.017 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-24T08:36:14.017 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-24T08:36:14.141 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-24T08:36:14.180 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-24T08:36:14.180 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-24T08:36:14.321 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-24T08:36:14.452 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T08:36:14.734 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T08:36:14.735 INFO:teuthology.orchestra.run.smithi138.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-04-24T08:36:14.735 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-24T08:36:15.644 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T08:36:15.645 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:36:15.645 INFO:teuthology.orchestra.run.smithi083.stdout:Upgraded: 2024-04-24T08:36:15.645 INFO:teuthology.orchestra.run.smithi083.stdout: librados2-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:36:15.645 INFO:teuthology.orchestra.run.smithi083.stdout: librbd1-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:36:15.645 INFO:teuthology.orchestra.run.smithi083.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-24T08:36:15.645 INFO:teuthology.orchestra.run.smithi083.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-24T08:36:15.645 INFO:teuthology.orchestra.run.smithi083.stdout:Installed: 2024-04-24T08:36:15.645 INFO:teuthology.orchestra.run.smithi083.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-24T08:36:15.645 INFO:teuthology.orchestra.run.smithi083.stdout: automake-1.16.2-8.el9.noarch 2024-04-24T08:36:15.645 INFO:teuthology.orchestra.run.smithi083.stdout: bison-3.7.4-5.el9.x86_64 2024-04-24T08:36:15.645 INFO:teuthology.orchestra.run.smithi083.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-24T08:36:15.645 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:36:15.645 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-base-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:36:15.646 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-common-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:36:15.646 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-fuse-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:36:15.646 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-grafana-dashboards-2:19.0.0-3116.g1f81fb39.el9.noarch 2024-04-24T08:36:15.646 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-immutable-object-cache-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:36:15.646 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mds-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:36:15.646 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:36:15.646 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-cephadm-2:19.0.0-3116.g1f81fb39.el9.noarch 2024-04-24T08:36:15.646 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-dashboard-2:19.0.0-3116.g1f81fb39.el9.noarch 2024-04-24T08:36:15.646 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3116.g1f81fb39.el9.noarch 2024-04-24T08:36:15.646 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-modules-core-2:19.0.0-3116.g1f81fb39.el9.noarch 2024-04-24T08:36:15.646 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-rook-2:19.0.0-3116.g1f81fb39.el9.noarch 2024-04-24T08:36:15.646 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mon-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:36:15.646 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-osd-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:36:15.647 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-prometheus-alerts-2:19.0.0-3116.g1f81fb39.el9.noarch 2024-04-24T08:36:15.647 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-radosgw-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:36:15.647 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-selinux-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:36:15.647 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-test-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:36:15.647 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-volume-2:19.0.0-3116.g1f81fb39.el9.noarch 2024-04-24T08:36:15.647 INFO:teuthology.orchestra.run.smithi083.stdout: cephadm-2:19.0.0-3116.g1f81fb39.el9.noarch 2024-04-24T08:36:15.647 INFO:teuthology.orchestra.run.smithi083.stdout: cephfs-mirror-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:36:15.647 INFO:teuthology.orchestra.run.smithi083.stdout: cephfs-shell-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:36:15.647 INFO:teuthology.orchestra.run.smithi083.stdout: cephfs-top-2:19.0.0-3116.g1f81fb39.el9.noarch 2024-04-24T08:36:15.647 INFO:teuthology.orchestra.run.smithi083.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-24T08:36:15.647 INFO:teuthology.orchestra.run.smithi083.stdout: flex-2.6.4-9.el9.x86_64 2024-04-24T08:36:15.647 INFO:teuthology.orchestra.run.smithi083.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-24T08:36:15.647 INFO:teuthology.orchestra.run.smithi083.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-24T08:36:15.647 INFO:teuthology.orchestra.run.smithi083.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-24T08:36:15.648 INFO:teuthology.orchestra.run.smithi083.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-24T08:36:15.648 INFO:teuthology.orchestra.run.smithi083.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-24T08:36:15.648 INFO:teuthology.orchestra.run.smithi083.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-24T08:36:15.648 INFO:teuthology.orchestra.run.smithi083.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-24T08:36:15.648 INFO:teuthology.orchestra.run.smithi083.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-24T08:36:15.648 INFO:teuthology.orchestra.run.smithi083.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-24T08:36:15.648 INFO:teuthology.orchestra.run.smithi083.stdout: libcephfs-devel-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:36:15.648 INFO:teuthology.orchestra.run.smithi083.stdout: libcephfs2-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:36:15.649 INFO:teuthology.orchestra.run.smithi083.stdout: libcephsqlite-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:36:15.649 INFO:teuthology.orchestra.run.smithi083.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-24T08:36:15.649 INFO:teuthology.orchestra.run.smithi083.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-24T08:36:15.649 INFO:teuthology.orchestra.run.smithi083.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-24T08:36:15.649 INFO:teuthology.orchestra.run.smithi083.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-24T08:36:15.649 INFO:teuthology.orchestra.run.smithi083.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-24T08:36:15.649 INFO:teuthology.orchestra.run.smithi083.stdout: librados-devel-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:36:15.649 INFO:teuthology.orchestra.run.smithi083.stdout: libradosstriper1-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:36:15.649 INFO:teuthology.orchestra.run.smithi083.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-24T08:36:15.649 INFO:teuthology.orchestra.run.smithi083.stdout: librgw2-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:36:15.649 INFO:teuthology.orchestra.run.smithi083.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-24T08:36:15.649 INFO:teuthology.orchestra.run.smithi083.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-24T08:36:15.649 INFO:teuthology.orchestra.run.smithi083.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-24T08:36:15.649 INFO:teuthology.orchestra.run.smithi083.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-24T08:36:15.650 INFO:teuthology.orchestra.run.smithi083.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-24T08:36:15.650 INFO:teuthology.orchestra.run.smithi083.stdout: lua-5.4.4-4.el9.x86_64 2024-04-24T08:36:15.650 INFO:teuthology.orchestra.run.smithi083.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-24T08:36:15.650 INFO:teuthology.orchestra.run.smithi083.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-24T08:36:15.650 INFO:teuthology.orchestra.run.smithi083.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-24T08:36:15.650 INFO:teuthology.orchestra.run.smithi083.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-24T08:36:15.650 INFO:teuthology.orchestra.run.smithi083.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-24T08:36:15.650 INFO:teuthology.orchestra.run.smithi083.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-24T08:36:15.650 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql-13.14-1.el9.x86_64 2024-04-24T08:36:15.650 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-24T08:36:15.650 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-24T08:36:15.650 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-24T08:36:15.650 INFO:teuthology.orchestra.run.smithi083.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-24T08:36:15.651 INFO:teuthology.orchestra.run.smithi083.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-24T08:36:15.651 INFO:teuthology.orchestra.run.smithi083.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-24T08:36:15.651 INFO:teuthology.orchestra.run.smithi083.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-24T08:36:15.651 INFO:teuthology.orchestra.run.smithi083.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-24T08:36:15.651 INFO:teuthology.orchestra.run.smithi083.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-24T08:36:15.651 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-24T08:36:15.651 INFO:teuthology.orchestra.run.smithi083.stdout: python3-ceph-argparse-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:36:15.651 INFO:teuthology.orchestra.run.smithi083.stdout: python3-ceph-common-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:36:15.651 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cephfs-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:36:15.651 INFO:teuthology.orchestra.run.smithi083.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-24T08:36:15.652 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-24T08:36:15.652 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-24T08:36:15.652 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-24T08:36:15.652 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-24T08:36:15.652 INFO:teuthology.orchestra.run.smithi083.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-24T08:36:15.653 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-24T08:36:15.653 INFO:teuthology.orchestra.run.smithi083.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-24T08:36:15.653 INFO:teuthology.orchestra.run.smithi083.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-24T08:36:15.653 INFO:teuthology.orchestra.run.smithi083.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-24T08:36:15.653 INFO:teuthology.orchestra.run.smithi083.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-24T08:36:15.653 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-24T08:36:15.653 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-24T08:36:15.653 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-24T08:36:15.653 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-24T08:36:15.653 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-24T08:36:15.653 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-24T08:36:15.653 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-24T08:36:15.653 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-24T08:36:15.653 INFO:teuthology.orchestra.run.smithi083.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-24T08:36:15.654 INFO:teuthology.orchestra.run.smithi083.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-24T08:36:15.654 INFO:teuthology.orchestra.run.smithi083.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-24T08:36:15.654 INFO:teuthology.orchestra.run.smithi083.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-24T08:36:15.654 INFO:teuthology.orchestra.run.smithi083.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-24T08:36:15.654 INFO:teuthology.orchestra.run.smithi083.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-24T08:36:15.654 INFO:teuthology.orchestra.run.smithi083.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-24T08:36:15.654 INFO:teuthology.orchestra.run.smithi083.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-24T08:36:15.654 INFO:teuthology.orchestra.run.smithi083.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-24T08:36:15.654 INFO:teuthology.orchestra.run.smithi083.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-24T08:36:15.654 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-24T08:36:15.654 INFO:teuthology.orchestra.run.smithi083.stdout: python3-ply-3.11-14.el9.noarch 2024-04-24T08:36:15.654 INFO:teuthology.orchestra.run.smithi083.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-24T08:36:15.654 INFO:teuthology.orchestra.run.smithi083.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-24T08:36:15.654 INFO:teuthology.orchestra.run.smithi083.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-24T08:36:15.655 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-24T08:36:15.655 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-24T08:36:15.655 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-24T08:36:15.655 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-24T08:36:15.655 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-24T08:36:15.655 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-24T08:36:15.655 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rados-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:36:15.655 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rbd-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:36:15.655 INFO:teuthology.orchestra.run.smithi083.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-24T08:36:15.655 INFO:teuthology.orchestra.run.smithi083.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-24T08:36:15.655 INFO:teuthology.orchestra.run.smithi083.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-24T08:36:15.655 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rgw-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:36:15.655 INFO:teuthology.orchestra.run.smithi083.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-24T08:36:15.655 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-24T08:36:15.655 INFO:teuthology.orchestra.run.smithi083.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-24T08:36:15.656 INFO:teuthology.orchestra.run.smithi083.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-24T08:36:15.656 INFO:teuthology.orchestra.run.smithi083.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-24T08:36:15.656 INFO:teuthology.orchestra.run.smithi083.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-24T08:36:15.656 INFO:teuthology.orchestra.run.smithi083.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-24T08:36:15.656 INFO:teuthology.orchestra.run.smithi083.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-24T08:36:15.656 INFO:teuthology.orchestra.run.smithi083.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-24T08:36:15.656 INFO:teuthology.orchestra.run.smithi083.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-24T08:36:15.656 INFO:teuthology.orchestra.run.smithi083.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-24T08:36:15.656 INFO:teuthology.orchestra.run.smithi083.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-24T08:36:15.656 INFO:teuthology.orchestra.run.smithi083.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-24T08:36:15.656 INFO:teuthology.orchestra.run.smithi083.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-24T08:36:15.656 INFO:teuthology.orchestra.run.smithi083.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-24T08:36:15.656 INFO:teuthology.orchestra.run.smithi083.stdout: rbd-fuse-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:36:15.656 INFO:teuthology.orchestra.run.smithi083.stdout: rbd-mirror-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:36:15.656 INFO:teuthology.orchestra.run.smithi083.stdout: rbd-nbd-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:36:15.656 INFO:teuthology.orchestra.run.smithi083.stdout: re2-1:20211101-3.el9.x86_64 2024-04-24T08:36:15.657 INFO:teuthology.orchestra.run.smithi083.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-24T08:36:15.657 INFO:teuthology.orchestra.run.smithi083.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-24T08:36:15.657 INFO:teuthology.orchestra.run.smithi083.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-24T08:36:15.657 INFO:teuthology.orchestra.run.smithi083.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-24T08:36:15.657 INFO:teuthology.orchestra.run.smithi083.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-24T08:36:15.657 INFO:teuthology.orchestra.run.smithi083.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-24T08:36:15.657 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:36:15.657 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2024-04-24T08:36:16.229 DEBUG:teuthology.parallel:result is None 2024-04-24T08:36:20.717 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-2:19.0.0-3116.g1f81fb39.el9.x86_64 1/156 2024-04-24T08:36:20.717 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-base-2:19.0.0-3116.g1f81fb39.el9.x86_64 2/156 2024-04-24T08:36:20.717 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-common-2:19.0.0-3116.g1f81fb39.el9.x86_64 3/156 2024-04-24T08:36:20.717 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-fuse-2:19.0.0-3116.g1f81fb39.el9.x86_64 4/156 2024-04-24T08:36:20.717 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3116.g1f81f 5/156 2024-04-24T08:36:20.718 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mds-2:19.0.0-3116.g1f81fb39.el9.x86_64 6/156 2024-04-24T08:36:20.718 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-2:19.0.0-3116.g1f81fb39.el9.x86_64 7/156 2024-04-24T08:36:20.718 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mon-2:19.0.0-3116.g1f81fb39.el9.x86_64 8/156 2024-04-24T08:36:20.718 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-osd-2:19.0.0-3116.g1f81fb39.el9.x86_64 9/156 2024-04-24T08:36:20.718 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-radosgw-2:19.0.0-3116.g1f81fb39.el9.x86_64 10/156 2024-04-24T08:36:20.718 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-selinux-2:19.0.0-3116.g1f81fb39.el9.x86_64 11/156 2024-04-24T08:36:20.718 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-test-2:19.0.0-3116.g1f81fb39.el9.x86_64 12/156 2024-04-24T08:36:20.718 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : cephfs-mirror-2:19.0.0-3116.g1f81fb39.el9.x86_64 13/156 2024-04-24T08:36:20.718 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : cephfs-shell-2:19.0.0-3116.g1f81fb39.el9.x86_64 14/156 2024-04-24T08:36:20.718 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libcephfs-devel-2:19.0.0-3116.g1f81fb39.el9.x86_ 15/156 2024-04-24T08:36:20.718 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libcephfs2-2:19.0.0-3116.g1f81fb39.el9.x86_64 16/156 2024-04-24T08:36:20.719 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libcephsqlite-2:19.0.0-3116.g1f81fb39.el9.x86_64 17/156 2024-04-24T08:36:20.719 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librados-devel-2:19.0.0-3116.g1f81fb39.el9.x86_6 18/156 2024-04-24T08:36:20.719 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libradosstriper1-2:19.0.0-3116.g1f81fb39.el9.x86 19/156 2024-04-24T08:36:20.719 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librgw2-2:19.0.0-3116.g1f81fb39.el9.x86_64 20/156 2024-04-24T08:36:20.719 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3116.g1f81fb39.el 21/156 2024-04-24T08:36:20.719 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-ceph-common-2:19.0.0-3116.g1f81fb39.el9. 22/156 2024-04-24T08:36:20.719 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cephfs-2:19.0.0-3116.g1f81fb39.el9.x86_6 23/156 2024-04-24T08:36:20.719 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-rados-2:19.0.0-3116.g1f81fb39.el9.x86_64 24/156 2024-04-24T08:36:20.719 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-rbd-2:19.0.0-3116.g1f81fb39.el9.x86_64 25/156 2024-04-24T08:36:20.719 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-rgw-2:19.0.0-3116.g1f81fb39.el9.x86_64 26/156 2024-04-24T08:36:20.719 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : rbd-fuse-2:19.0.0-3116.g1f81fb39.el9.x86_64 27/156 2024-04-24T08:36:20.719 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : rbd-mirror-2:19.0.0-3116.g1f81fb39.el9.x86_64 28/156 2024-04-24T08:36:20.720 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : rbd-nbd-2:19.0.0-3116.g1f81fb39.el9.x86_64 29/156 2024-04-24T08:36:20.720 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3116.g1f81fb39. 30/156 2024-04-24T08:36:20.731 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3116.g1f81fb39.el9.noa 31/156 2024-04-24T08:36:20.731 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3116.g1f81fb39.el9.n 32/156 2024-04-24T08:36:20.731 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3116.g1f8 33/156 2024-04-24T08:36:20.731 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3116.g1f81fb39.el 34/156 2024-04-24T08:36:20.732 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3116.g1f81fb39.el9.noarch 35/156 2024-04-24T08:36:20.732 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3116.g1f81fb39.e 36/156 2024-04-24T08:36:20.732 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-volume-2:19.0.0-3116.g1f81fb39.el9.noarch 37/156 2024-04-24T08:36:20.732 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : cephadm-2:19.0.0-3116.g1f81fb39.el9.noarch 38/156 2024-04-24T08:36:20.732 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : cephfs-top-2:19.0.0-3116.g1f81fb39.el9.noarch 39/156 2024-04-24T08:36:20.732 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-24T08:36:20.732 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-24T08:36:20.732 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-24T08:36:20.732 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-24T08:36:20.732 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-24T08:36:20.732 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-24T08:36:20.732 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-24T08:36:20.732 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-24T08:36:20.733 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-24T08:36:20.733 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-24T08:36:20.733 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-24T08:36:20.733 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-24T08:36:20.733 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-24T08:36:20.733 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-24T08:36:20.733 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-24T08:36:20.733 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-24T08:36:20.733 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-24T08:36:20.733 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-24T08:36:20.733 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-24T08:36:20.733 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-24T08:36:20.733 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-24T08:36:20.734 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-24T08:36:20.734 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-24T08:36:20.734 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-24T08:36:20.734 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-24T08:36:20.734 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-24T08:36:20.734 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-24T08:36:20.734 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-24T08:36:20.734 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-24T08:36:20.734 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-24T08:36:20.734 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-24T08:36:20.734 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-24T08:36:20.734 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-24T08:36:20.734 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-24T08:36:20.734 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-24T08:36:20.735 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-24T08:36:20.735 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-24T08:36:20.735 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-24T08:36:20.735 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-24T08:36:20.735 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-24T08:36:20.735 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-24T08:36:20.735 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-24T08:36:20.735 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-24T08:36:20.735 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-24T08:36:20.736 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-24T08:36:20.736 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-24T08:36:20.736 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-24T08:36:20.736 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-24T08:36:20.736 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-24T08:36:20.736 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-24T08:36:20.736 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-24T08:36:20.736 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-24T08:36:20.736 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-24T08:36:20.736 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-24T08:36:20.736 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-24T08:36:20.736 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-24T08:36:20.736 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-24T08:36:20.736 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-24T08:36:20.737 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-24T08:36:20.737 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-24T08:36:20.737 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-24T08:36:20.737 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-24T08:36:20.737 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-24T08:36:20.737 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-24T08:36:20.737 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-24T08:36:20.737 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-24T08:36:20.737 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-24T08:36:20.737 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-24T08:36:20.738 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-24T08:36:20.738 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-24T08:36:20.738 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-24T08:36:20.738 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-24T08:36:20.738 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-24T08:36:20.738 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-24T08:36:20.738 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-24T08:36:20.738 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-24T08:36:20.738 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-24T08:36:20.738 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-24T08:36:20.738 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-24T08:36:20.738 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-24T08:36:20.738 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-24T08:36:20.739 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-24T08:36:20.739 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-24T08:36:20.739 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-24T08:36:20.739 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-24T08:36:20.739 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-24T08:36:20.739 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-24T08:36:20.739 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-24T08:36:20.739 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-24T08:36:20.739 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-24T08:36:20.739 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-24T08:36:20.739 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-24T08:36:20.740 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-24T08:36:20.740 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-24T08:36:20.740 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-24T08:36:20.740 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-24T08:36:20.740 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-24T08:36:20.740 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-24T08:36:20.740 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-24T08:36:20.740 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-24T08:36:20.740 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-24T08:36:20.740 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-24T08:36:20.740 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-24T08:36:20.740 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-24T08:36:20.741 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-24T08:36:20.741 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-24T08:36:20.741 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-24T08:36:20.741 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-24T08:36:20.741 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-24T08:36:20.741 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librados2-2:19.0.0-3116.g1f81fb39.el9.x86_64 149/156 2024-04-24T08:36:20.741 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-24T08:36:20.741 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librbd1-2:19.0.0-3116.g1f81fb39.el9.x86_64 151/156 2024-04-24T08:36:20.741 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-24T08:36:20.741 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-24T08:36:20.741 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-24T08:36:20.742 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-24T08:36:23.063 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T08:36:23.063 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-24T08:36:23.063 INFO:teuthology.orchestra.run.smithi138.stdout:Upgraded: 2024-04-24T08:36:23.063 INFO:teuthology.orchestra.run.smithi138.stdout: librados2-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:36:23.063 INFO:teuthology.orchestra.run.smithi138.stdout: librbd1-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:36:23.063 INFO:teuthology.orchestra.run.smithi138.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-24T08:36:23.063 INFO:teuthology.orchestra.run.smithi138.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-24T08:36:23.063 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-04-24T08:36:23.063 INFO:teuthology.orchestra.run.smithi138.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-24T08:36:23.063 INFO:teuthology.orchestra.run.smithi138.stdout: automake-1.16.2-8.el9.noarch 2024-04-24T08:36:23.063 INFO:teuthology.orchestra.run.smithi138.stdout: bison-3.7.4-5.el9.x86_64 2024-04-24T08:36:23.064 INFO:teuthology.orchestra.run.smithi138.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-24T08:36:23.064 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:36:23.064 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-base-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:36:23.064 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-common-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:36:23.064 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-fuse-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:36:23.064 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-grafana-dashboards-2:19.0.0-3116.g1f81fb39.el9.noarch 2024-04-24T08:36:23.064 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-immutable-object-cache-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:36:23.064 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mds-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:36:23.064 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:36:23.064 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-cephadm-2:19.0.0-3116.g1f81fb39.el9.noarch 2024-04-24T08:36:23.064 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-dashboard-2:19.0.0-3116.g1f81fb39.el9.noarch 2024-04-24T08:36:23.064 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3116.g1f81fb39.el9.noarch 2024-04-24T08:36:23.064 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-modules-core-2:19.0.0-3116.g1f81fb39.el9.noarch 2024-04-24T08:36:23.064 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-rook-2:19.0.0-3116.g1f81fb39.el9.noarch 2024-04-24T08:36:23.065 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mon-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:36:23.065 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-osd-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:36:23.065 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-prometheus-alerts-2:19.0.0-3116.g1f81fb39.el9.noarch 2024-04-24T08:36:23.065 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-radosgw-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:36:23.065 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-selinux-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:36:23.065 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-test-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:36:23.065 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-volume-2:19.0.0-3116.g1f81fb39.el9.noarch 2024-04-24T08:36:23.065 INFO:teuthology.orchestra.run.smithi138.stdout: cephadm-2:19.0.0-3116.g1f81fb39.el9.noarch 2024-04-24T08:36:23.065 INFO:teuthology.orchestra.run.smithi138.stdout: cephfs-mirror-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:36:23.065 INFO:teuthology.orchestra.run.smithi138.stdout: cephfs-shell-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:36:23.065 INFO:teuthology.orchestra.run.smithi138.stdout: cephfs-top-2:19.0.0-3116.g1f81fb39.el9.noarch 2024-04-24T08:36:23.065 INFO:teuthology.orchestra.run.smithi138.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-24T08:36:23.065 INFO:teuthology.orchestra.run.smithi138.stdout: flex-2.6.4-9.el9.x86_64 2024-04-24T08:36:23.065 INFO:teuthology.orchestra.run.smithi138.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-24T08:36:23.066 INFO:teuthology.orchestra.run.smithi138.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-24T08:36:23.066 INFO:teuthology.orchestra.run.smithi138.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-24T08:36:23.066 INFO:teuthology.orchestra.run.smithi138.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-24T08:36:23.066 INFO:teuthology.orchestra.run.smithi138.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-24T08:36:23.066 INFO:teuthology.orchestra.run.smithi138.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-24T08:36:23.066 INFO:teuthology.orchestra.run.smithi138.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-24T08:36:23.066 INFO:teuthology.orchestra.run.smithi138.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-24T08:36:23.066 INFO:teuthology.orchestra.run.smithi138.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-24T08:36:23.066 INFO:teuthology.orchestra.run.smithi138.stdout: libcephfs-devel-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:36:23.066 INFO:teuthology.orchestra.run.smithi138.stdout: libcephfs2-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:36:23.066 INFO:teuthology.orchestra.run.smithi138.stdout: libcephsqlite-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:36:23.066 INFO:teuthology.orchestra.run.smithi138.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-24T08:36:23.066 INFO:teuthology.orchestra.run.smithi138.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-24T08:36:23.067 INFO:teuthology.orchestra.run.smithi138.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-24T08:36:23.067 INFO:teuthology.orchestra.run.smithi138.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-24T08:36:23.067 INFO:teuthology.orchestra.run.smithi138.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-24T08:36:23.067 INFO:teuthology.orchestra.run.smithi138.stdout: librados-devel-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:36:23.067 INFO:teuthology.orchestra.run.smithi138.stdout: libradosstriper1-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:36:23.067 INFO:teuthology.orchestra.run.smithi138.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-24T08:36:23.067 INFO:teuthology.orchestra.run.smithi138.stdout: librgw2-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:36:23.068 INFO:teuthology.orchestra.run.smithi138.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-24T08:36:23.068 INFO:teuthology.orchestra.run.smithi138.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-24T08:36:23.068 INFO:teuthology.orchestra.run.smithi138.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-24T08:36:23.068 INFO:teuthology.orchestra.run.smithi138.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-24T08:36:23.068 INFO:teuthology.orchestra.run.smithi138.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-24T08:36:23.068 INFO:teuthology.orchestra.run.smithi138.stdout: lua-5.4.4-4.el9.x86_64 2024-04-24T08:36:23.069 INFO:teuthology.orchestra.run.smithi138.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-24T08:36:23.069 INFO:teuthology.orchestra.run.smithi138.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-24T08:36:23.069 INFO:teuthology.orchestra.run.smithi138.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-24T08:36:23.069 INFO:teuthology.orchestra.run.smithi138.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-24T08:36:23.069 INFO:teuthology.orchestra.run.smithi138.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-24T08:36:23.069 INFO:teuthology.orchestra.run.smithi138.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-24T08:36:23.069 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql-13.14-1.el9.x86_64 2024-04-24T08:36:23.070 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-24T08:36:23.070 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-24T08:36:23.070 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-24T08:36:23.070 INFO:teuthology.orchestra.run.smithi138.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-24T08:36:23.070 INFO:teuthology.orchestra.run.smithi138.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-24T08:36:23.071 INFO:teuthology.orchestra.run.smithi138.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-24T08:36:23.071 INFO:teuthology.orchestra.run.smithi138.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-24T08:36:23.071 INFO:teuthology.orchestra.run.smithi138.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-24T08:36:23.071 INFO:teuthology.orchestra.run.smithi138.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-24T08:36:23.071 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-24T08:36:23.071 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ceph-argparse-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:36:23.072 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ceph-common-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:36:23.072 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cephfs-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:36:23.072 INFO:teuthology.orchestra.run.smithi138.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-24T08:36:23.072 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-24T08:36:23.072 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-24T08:36:23.072 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-24T08:36:23.072 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-24T08:36:23.072 INFO:teuthology.orchestra.run.smithi138.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-24T08:36:23.072 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-24T08:36:23.072 INFO:teuthology.orchestra.run.smithi138.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-24T08:36:23.072 INFO:teuthology.orchestra.run.smithi138.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-24T08:36:23.072 INFO:teuthology.orchestra.run.smithi138.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-24T08:36:23.073 INFO:teuthology.orchestra.run.smithi138.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-24T08:36:23.073 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-24T08:36:23.073 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-24T08:36:23.073 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-24T08:36:23.073 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-24T08:36:23.073 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-24T08:36:23.073 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-24T08:36:23.073 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-24T08:36:23.073 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-24T08:36:23.073 INFO:teuthology.orchestra.run.smithi138.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-24T08:36:23.073 INFO:teuthology.orchestra.run.smithi138.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-24T08:36:23.073 INFO:teuthology.orchestra.run.smithi138.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-24T08:36:23.073 INFO:teuthology.orchestra.run.smithi138.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-24T08:36:23.074 INFO:teuthology.orchestra.run.smithi138.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-24T08:36:23.074 INFO:teuthology.orchestra.run.smithi138.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-24T08:36:23.074 INFO:teuthology.orchestra.run.smithi138.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-24T08:36:23.074 INFO:teuthology.orchestra.run.smithi138.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-24T08:36:23.074 INFO:teuthology.orchestra.run.smithi138.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-24T08:36:23.074 INFO:teuthology.orchestra.run.smithi138.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-24T08:36:23.074 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-24T08:36:23.074 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ply-3.11-14.el9.noarch 2024-04-24T08:36:23.074 INFO:teuthology.orchestra.run.smithi138.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-24T08:36:23.075 INFO:teuthology.orchestra.run.smithi138.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-24T08:36:23.075 INFO:teuthology.orchestra.run.smithi138.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-24T08:36:23.075 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-24T08:36:23.075 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-24T08:36:23.075 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-24T08:36:23.075 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-24T08:36:23.075 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-24T08:36:23.075 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-24T08:36:23.075 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rados-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:36:23.075 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rbd-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:36:23.075 INFO:teuthology.orchestra.run.smithi138.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-24T08:36:23.075 INFO:teuthology.orchestra.run.smithi138.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-24T08:36:23.075 INFO:teuthology.orchestra.run.smithi138.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-24T08:36:23.075 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rgw-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:36:23.075 INFO:teuthology.orchestra.run.smithi138.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-24T08:36:23.076 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-24T08:36:23.076 INFO:teuthology.orchestra.run.smithi138.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-24T08:36:23.076 INFO:teuthology.orchestra.run.smithi138.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-24T08:36:23.076 INFO:teuthology.orchestra.run.smithi138.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-24T08:36:23.076 INFO:teuthology.orchestra.run.smithi138.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-24T08:36:23.076 INFO:teuthology.orchestra.run.smithi138.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-24T08:36:23.076 INFO:teuthology.orchestra.run.smithi138.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-24T08:36:23.076 INFO:teuthology.orchestra.run.smithi138.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-24T08:36:23.076 INFO:teuthology.orchestra.run.smithi138.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-24T08:36:23.076 INFO:teuthology.orchestra.run.smithi138.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-24T08:36:23.076 INFO:teuthology.orchestra.run.smithi138.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-24T08:36:23.076 INFO:teuthology.orchestra.run.smithi138.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-24T08:36:23.076 INFO:teuthology.orchestra.run.smithi138.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-24T08:36:23.076 INFO:teuthology.orchestra.run.smithi138.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-24T08:36:23.076 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-fuse-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:36:23.077 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-mirror-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:36:23.077 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-nbd-2:19.0.0-3116.g1f81fb39.el9.x86_64 2024-04-24T08:36:23.077 INFO:teuthology.orchestra.run.smithi138.stdout: re2-1:20211101-3.el9.x86_64 2024-04-24T08:36:23.077 INFO:teuthology.orchestra.run.smithi138.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-24T08:36:23.077 INFO:teuthology.orchestra.run.smithi138.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-24T08:36:23.077 INFO:teuthology.orchestra.run.smithi138.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-24T08:36:23.077 INFO:teuthology.orchestra.run.smithi138.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-24T08:36:23.077 INFO:teuthology.orchestra.run.smithi138.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-24T08:36:23.077 INFO:teuthology.orchestra.run.smithi138.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-24T08:36:23.077 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-24T08:36:23.077 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-04-24T08:36:23.649 DEBUG:teuthology.parallel:result is None 2024-04-24T08:36:23.650 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1f81fb397ae98da7563d451a78c61574c8f4e6e0 2024-04-24T08:36:23.883 DEBUG:teuthology.orchestra.run.smithi083:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-24T08:36:23.949 INFO:teuthology.orchestra.run.smithi083.stdout:19.0.0-3116.g1f81fb39.el9 2024-04-24T08:36:23.949 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3116.g1f81fb39.el9 2024-04-24T08:36:23.950 INFO:teuthology.task.install:The correct ceph version 19.0.0-3116.g1f81fb39 is installed. 2024-04-24T08:36:23.952 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1f81fb397ae98da7563d451a78c61574c8f4e6e0 2024-04-24T08:36:24.168 DEBUG:teuthology.orchestra.run.smithi138:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-24T08:36:24.241 INFO:teuthology.orchestra.run.smithi138.stdout:19.0.0-3116.g1f81fb39.el9 2024-04-24T08:36:24.241 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3116.g1f81fb39.el9 2024-04-24T08:36:24.241 INFO:teuthology.task.install:The correct ceph version 19.0.0-3116.g1f81fb39 is installed. 2024-04-24T08:36:24.243 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1f81fb397ae98da7563d451a78c61574c8f4e6e0 2024-04-24T08:36:24.385 DEBUG:teuthology.orchestra.run.smithi189:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-24T08:36:24.458 INFO:teuthology.orchestra.run.smithi189.stdout:19.0.0-3116.g1f81fb39.el9 2024-04-24T08:36:24.459 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3116.g1f81fb39.el9 2024-04-24T08:36:24.459 INFO:teuthology.task.install:The correct ceph version 19.0.0-3116.g1f81fb39 is installed. 2024-04-24T08:36:24.461 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-24T08:36:24.461 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-24T08:36:24.461 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-24T08:36:24.544 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-24T08:36:24.544 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-24T08:36:24.623 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-04-24T08:36:24.623 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-24T08:36:24.704 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-24T08:36:24.705 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-24T08:36:24.705 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/usr/bin/daemon-helper 2024-04-24T08:36:24.783 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-24T08:36:24.899 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-24T08:36:24.899 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/usr/bin/daemon-helper 2024-04-24T08:36:24.977 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-24T08:36:25.083 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-04-24T08:36:25.083 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/usr/bin/daemon-helper 2024-04-24T08:36:25.148 DEBUG:teuthology.orchestra.run.smithi189:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-24T08:36:25.260 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-24T08:36:25.260 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-24T08:36:25.260 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-24T08:36:25.330 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-24T08:36:25.430 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-24T08:36:25.430 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-24T08:36:25.499 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-24T08:36:25.604 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-04-24T08:36:25.604 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-24T08:36:25.684 DEBUG:teuthology.orchestra.run.smithi189:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-24T08:36:25.798 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-24T08:36:25.798 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-24T08:36:25.798 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/usr/bin/stdin-killer 2024-04-24T08:36:25.872 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-24T08:36:25.988 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-24T08:36:25.988 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/usr/bin/stdin-killer 2024-04-24T08:36:26.056 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-24T08:36:26.162 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-04-24T08:36:26.162 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/usr/bin/stdin-killer 2024-04-24T08:36:26.242 DEBUG:teuthology.orchestra.run.smithi189:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-24T08:36:26.319 INFO:teuthology.run_tasks:Running task cephadm... 2024-04-24T08:36:26.429 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '1f81fb397ae98da7563d451a78c61574c8f4e6e0', 'subvols': {'create': 2}} 2024-04-24T08:36:26.429 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 2024-04-24T08:36:26.430 INFO:tasks.cephadm:Cluster fsid is bd3e039e-0215-11ef-bc93-c7b262605968 2024-04-24T08:36:26.430 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-04-24T08:36:26.430 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.83', 'mon.b': '172.21.15.138', 'mon.c': '172.21.15.189'} 2024-04-24T08:36:26.430 INFO:tasks.cephadm:First mon is mon.a on smithi083 2024-04-24T08:36:26.430 INFO:tasks.cephadm:First mgr is x 2024-04-24T08:36:26.431 INFO:tasks.cephadm:Normalizing hostnames... 2024-04-24T08:36:26.431 DEBUG:teuthology.orchestra.run.smithi083:> sudo hostname $(hostname -s) 2024-04-24T08:36:26.518 DEBUG:teuthology.orchestra.run.smithi138:> sudo hostname $(hostname -s) 2024-04-24T08:36:26.587 DEBUG:teuthology.orchestra.run.smithi189:> sudo hostname $(hostname -s) 2024-04-24T08:36:26.680 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-04-24T08:36:26.680 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1f81fb397ae98da7563d451a78c61574c8f4e6e0 2024-04-24T08:36:26.824 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '1f81fb397ae98da7563d451a78c61574c8f4e6e0', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/78823/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3116-g1f81fb39', 'node_name': '172.21.2.1+braggi01', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3116.g1f81fb39'}, 'url': 'https://4.chacra.ceph.com/r/ceph/rishabh-main-apr17-a654945/1f81fb397ae98da7563d451a78c61574c8f4e6e0/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-04-24 01:00:13.634707', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'rishabh-main-apr17-a654945', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/rishabh-main-apr17-a654945/1f81fb397ae98da7563d451a78c61574c8f4e6e0/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-04-24T08:36:27.044 INFO:tasks.util.chacra:got chacra host 4.chacra.ceph.com, ref rishabh-main-apr17-a654945, sha1 1f81fb397ae98da7563d451a78c61574c8f4e6e0 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=1f81fb397ae98da7563d451a78c61574c8f4e6e0 2024-04-24T08:36:27.045 INFO:tasks.cephadm:Discovered cachra url: https://4.chacra.ceph.com/binaries/ceph/rishabh-main-apr17-a654945/1f81fb397ae98da7563d451a78c61574c8f4e6e0/centos/9/x86_64/flavors/default/cephadm 2024-04-24T08:36:27.046 DEBUG:teuthology.orchestra.run.smithi083:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/rishabh-main-apr17-a654945/1f81fb397ae98da7563d451a78c61574c8f4e6e0/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-24T08:36:27.951 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r--. 1 ubuntu ubuntu 781486 Apr 24 08:36 /home/ubuntu/cephtest/cephadm 2024-04-24T08:36:27.952 DEBUG:teuthology.orchestra.run.smithi138:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/rishabh-main-apr17-a654945/1f81fb397ae98da7563d451a78c61574c8f4e6e0/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-24T08:36:28.835 INFO:teuthology.orchestra.run.smithi138.stdout:-rw-r--r--. 1 ubuntu ubuntu 781486 Apr 24 08:36 /home/ubuntu/cephtest/cephadm 2024-04-24T08:36:28.835 DEBUG:teuthology.orchestra.run.smithi189:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/rishabh-main-apr17-a654945/1f81fb397ae98da7563d451a78c61574c8f4e6e0/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-24T08:36:29.764 INFO:teuthology.orchestra.run.smithi189.stdout:-rw-r--r--. 1 ubuntu ubuntu 781486 Apr 24 08:36 /home/ubuntu/cephtest/cephadm 2024-04-24T08:36:29.765 DEBUG:teuthology.orchestra.run.smithi083:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-24T08:36:29.825 DEBUG:teuthology.orchestra.run.smithi138:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-24T08:36:29.872 DEBUG:teuthology.orchestra.run.smithi189:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-24T08:36:29.969 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 on all hosts... 2024-04-24T08:36:29.969 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 pull 2024-04-24T08:36:29.977 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 pull 2024-04-24T08:36:29.981 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 pull 2024-04-24T08:36:31.974 INFO:teuthology.orchestra.run.smithi189.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0... 2024-04-24T08:36:32.205 INFO:teuthology.orchestra.run.smithi083.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0... 2024-04-24T08:36:32.929 INFO:teuthology.orchestra.run.smithi138.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0... 2024-04-24T08:37:08.667 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T08:37:08.667 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_version": "ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev)", 2024-04-24T08:37:08.667 INFO:teuthology.orchestra.run.smithi083.stdout: "image_id": "b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02", 2024-04-24T08:37:08.667 INFO:teuthology.orchestra.run.smithi083.stdout: "repo_digests": [ 2024-04-24T08:37:08.667 INFO:teuthology.orchestra.run.smithi083.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644" 2024-04-24T08:37:08.668 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-04-24T08:37:08.668 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T08:37:14.417 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-04-24T08:37:14.418 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_version": "ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev)", 2024-04-24T08:37:14.418 INFO:teuthology.orchestra.run.smithi138.stdout: "image_id": "b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02", 2024-04-24T08:37:14.418 INFO:teuthology.orchestra.run.smithi138.stdout: "repo_digests": [ 2024-04-24T08:37:14.418 INFO:teuthology.orchestra.run.smithi138.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644" 2024-04-24T08:37:14.418 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-04-24T08:37:14.418 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-04-24T08:37:18.242 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-04-24T08:37:18.242 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_version": "ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev)", 2024-04-24T08:37:18.242 INFO:teuthology.orchestra.run.smithi189.stdout: "image_id": "b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02", 2024-04-24T08:37:18.242 INFO:teuthology.orchestra.run.smithi189.stdout: "repo_digests": [ 2024-04-24T08:37:18.242 INFO:teuthology.orchestra.run.smithi189.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644" 2024-04-24T08:37:18.242 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-04-24T08:37:18.242 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-04-24T08:37:18.297 DEBUG:teuthology.orchestra.run.smithi083:> sudo mkdir -p /etc/ceph 2024-04-24T08:37:18.383 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /etc/ceph 2024-04-24T08:37:18.482 DEBUG:teuthology.orchestra.run.smithi189:> sudo mkdir -p /etc/ceph 2024-04-24T08:37:18.576 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 777 /etc/ceph 2024-04-24T08:37:18.661 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 777 /etc/ceph 2024-04-24T08:37:18.737 DEBUG:teuthology.orchestra.run.smithi189:> sudo chmod 777 /etc/ceph 2024-04-24T08:37:18.830 INFO:tasks.cephadm:Writing seed config... 2024-04-24T08:37:18.831 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-04-24T08:37:18.831 INFO:tasks.cephadm: override: [client] debug client = 20 2024-04-24T08:37:18.831 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-04-24T08:37:18.832 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-04-24T08:37:18.832 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-04-24T08:37:18.832 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-04-24T08:37:18.832 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-04-24T08:37:18.832 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-04-24T08:37:18.832 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-04-24T08:37:18.832 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-04-24T08:37:18.832 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-04-24T08:37:18.832 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-04-24T08:37:18.832 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-04-24T08:37:18.832 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-04-24T08:37:18.832 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-04-24T08:37:18.832 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-04-24T08:37:18.832 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-04-24T08:37:18.832 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-04-24T08:37:18.833 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-04-24T08:37:18.833 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-04-24T08:37:18.833 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-04-24T08:37:18.833 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-04-24T08:37:18.833 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-04-24T08:37:18.833 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-04-24T08:37:18.833 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-04-24T08:37:18.833 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-04-24T08:37:18.833 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-04-24T08:37:18.833 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-04-24T08:37:18.833 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-04-24T08:37:18.833 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-04-24T08:37:18.833 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-04-24T08:37:18.833 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-04-24T08:37:18.833 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-04-24T08:37:18.834 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-04-24T08:37:18.834 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-04-24T08:37:18.834 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-04-24T08:37:18.834 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-04-24T08:37:18.834 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-04-24T08:37:18.834 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-04-24T08:37:18.834 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-04-24T08:37:18.834 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-04-24T08:37:18.834 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-04-24T08:37:18.834 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-04-24T08:37:18.834 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-04-24T08:37:18.835 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-24T08:37:18.835 DEBUG:teuthology.orchestra.run.smithi083:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-04-24T08:37:18.933 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = bd3e039e-0215-11ef-bc93-c7b262605968 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-04-24T08:37:18.933 DEBUG:teuthology.orchestra.run.smithi083:mon.a> sudo journalctl -f -n 0 -u ceph-bd3e039e-0215-11ef-bc93-c7b262605968@mon.a.service 2024-04-24T08:37:18.984 DEBUG:teuthology.orchestra.run.smithi083:mgr.x> sudo journalctl -f -n 0 -u ceph-bd3e039e-0215-11ef-bc93-c7b262605968@mgr.x.service 2024-04-24T08:37:19.031 INFO:tasks.cephadm:Bootstrapping... 2024-04-24T08:37:19.031 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 -v bootstrap --fsid bd3e039e-0215-11ef-bc93-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.83 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-04-24T08:37:19.298 INFO:teuthology.orchestra.run.smithi083.stdout:-------------------------------------------------------------------------------- 2024-04-24T08:37:19.298 INFO:teuthology.orchestra.run.smithi083.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0', '-v', 'bootstrap', '--fsid', 'bd3e039e-0215-11ef-bc93-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.83', '--skip-admin-label'] 2024-04-24T08:37:19.299 INFO:teuthology.orchestra.run.smithi083.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-04-24T08:37:19.299 INFO:teuthology.orchestra.run.smithi083.stdout:Verifying podman|docker is present... 2024-04-24T08:37:19.357 INFO:teuthology.orchestra.run.smithi083.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-24T08:37:19.357 INFO:teuthology.orchestra.run.smithi083.stdout:Verifying lvm2 is present... 2024-04-24T08:37:19.358 INFO:teuthology.orchestra.run.smithi083.stdout:Verifying time synchronization is in place... 2024-04-24T08:37:19.375 INFO:teuthology.orchestra.run.smithi083.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-24T08:37:19.375 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-24T08:37:19.392 INFO:teuthology.orchestra.run.smithi083.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-24T08:37:19.392 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stdout inactive 2024-04-24T08:37:19.412 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stdout enabled 2024-04-24T08:37:19.432 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stdout active 2024-04-24T08:37:19.433 INFO:teuthology.orchestra.run.smithi083.stdout:Unit chronyd.service is enabled and running 2024-04-24T08:37:19.433 INFO:teuthology.orchestra.run.smithi083.stdout:Repeating the final host check... 2024-04-24T08:37:19.489 INFO:teuthology.orchestra.run.smithi083.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-24T08:37:19.490 INFO:teuthology.orchestra.run.smithi083.stdout:podman (/bin/podman) version 4.9.4 is present 2024-04-24T08:37:19.490 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl is present 2024-04-24T08:37:19.490 INFO:teuthology.orchestra.run.smithi083.stdout:lvcreate is present 2024-04-24T08:37:19.504 INFO:teuthology.orchestra.run.smithi083.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-24T08:37:19.504 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-24T08:37:19.519 INFO:teuthology.orchestra.run.smithi083.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-24T08:37:19.519 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stdout inactive 2024-04-24T08:37:19.540 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stdout enabled 2024-04-24T08:37:19.555 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stdout active 2024-04-24T08:37:19.556 INFO:teuthology.orchestra.run.smithi083.stdout:Unit chronyd.service is enabled and running 2024-04-24T08:37:19.556 INFO:teuthology.orchestra.run.smithi083.stdout:Host looks OK 2024-04-24T08:37:19.556 INFO:teuthology.orchestra.run.smithi083.stdout:Cluster fsid: bd3e039e-0215-11ef-bc93-c7b262605968 2024-04-24T08:37:19.556 INFO:teuthology.orchestra.run.smithi083.stdout:Acquiring lock 140028904332448 on /run/cephadm/bd3e039e-0215-11ef-bc93-c7b262605968.lock 2024-04-24T08:37:19.556 INFO:teuthology.orchestra.run.smithi083.stdout:Lock 140028904332448 acquired on /run/cephadm/bd3e039e-0215-11ef-bc93-c7b262605968.lock 2024-04-24T08:37:19.557 INFO:teuthology.orchestra.run.smithi083.stdout:Verifying IP 172.21.15.83 port 3300 ... 2024-04-24T08:37:19.557 INFO:teuthology.orchestra.run.smithi083.stdout:Verifying IP 172.21.15.83 port 6789 ... 2024-04-24T08:37:19.557 INFO:teuthology.orchestra.run.smithi083.stdout:Base mon IP(s) is [172.21.15.83:3300, 172.21.15.83:6789], mon addrv is [v2:172.21.15.83:3300,v1:172.21.15.83:6789] 2024-04-24T08:37:19.565 INFO:teuthology.orchestra.run.smithi083.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.83 metric 100 2024-04-24T08:37:19.565 INFO:teuthology.orchestra.run.smithi083.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.83 metric 100 2024-04-24T08:37:19.572 INFO:teuthology.orchestra.run.smithi083.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-04-24T08:37:19.572 INFO:teuthology.orchestra.run.smithi083.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-04-24T08:37:19.572 INFO:teuthology.orchestra.run.smithi083.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1680sec hoplimit 64 pref medium 2024-04-24T08:37:19.579 INFO:teuthology.orchestra.run.smithi083.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-04-24T08:37:19.579 INFO:teuthology.orchestra.run.smithi083.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-04-24T08:37:19.579 INFO:teuthology.orchestra.run.smithi083.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-24T08:37:19.580 INFO:teuthology.orchestra.run.smithi083.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-04-24T08:37:19.580 INFO:teuthology.orchestra.run.smithi083.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:ce91/64 scope link 2024-04-24T08:37:19.580 INFO:teuthology.orchestra.run.smithi083.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-24T08:37:19.580 INFO:teuthology.orchestra.run.smithi083.stdout:Mon IP `172.21.15.83` is in CIDR network `172.21.0.0/20` 2024-04-24T08:37:19.581 INFO:teuthology.orchestra.run.smithi083.stdout:Mon IP `172.21.15.83` is in CIDR network `172.21.0.0/20` 2024-04-24T08:37:19.581 INFO:teuthology.orchestra.run.smithi083.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-04-24T08:37:19.582 INFO:teuthology.orchestra.run.smithi083.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-04-24T08:37:19.583 INFO:teuthology.orchestra.run.smithi083.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0... 2024-04-24T08:37:21.141 INFO:teuthology.orchestra.run.smithi083.stdout:/bin/podman: stdout b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02 2024-04-24T08:37:21.142 INFO:teuthology.orchestra.run.smithi083.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0... 2024-04-24T08:37:21.142 INFO:teuthology.orchestra.run.smithi083.stdout:/bin/podman: stderr Getting image source signatures 2024-04-24T08:37:21.142 INFO:teuthology.orchestra.run.smithi083.stdout:/bin/podman: stderr Copying blob sha256:b727accf024e55f70f86e348fc139074bb9b3de1cca3400f8b76b2ed7d35ac83 2024-04-24T08:37:21.142 INFO:teuthology.orchestra.run.smithi083.stdout:/bin/podman: stderr Copying blob sha256:3d665f4c032644be9cae748af04bee69f2bfe96cdc3616dabe0ab17899633ab5 2024-04-24T08:37:21.142 INFO:teuthology.orchestra.run.smithi083.stdout:/bin/podman: stderr Copying config sha256:b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02 2024-04-24T08:37:21.142 INFO:teuthology.orchestra.run.smithi083.stdout:/bin/podman: stderr Writing manifest to image destination 2024-04-24T08:37:22.769 INFO:teuthology.orchestra.run.smithi083.stdout:ceph: stdout ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev) 2024-04-24T08:37:22.769 INFO:teuthology.orchestra.run.smithi083.stdout:Ceph version: ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev) 2024-04-24T08:37:22.769 INFO:teuthology.orchestra.run.smithi083.stdout:Extracting ceph user uid/gid from container image... 2024-04-24T08:37:24.029 INFO:teuthology.orchestra.run.smithi083.stdout:stat: stdout 167 167 2024-04-24T08:37:24.030 INFO:teuthology.orchestra.run.smithi083.stdout:Creating initial keys... 2024-04-24T08:37:25.266 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph-authtool: stdout AQDExChm1FMjIhAAauiwhG9KHOwGl6WmOa4gzQ== 2024-04-24T08:37:26.528 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph-authtool: stdout AQDFxChmAhg/MBAAgAhOvd2BRELqOkeRhjeBgA== 2024-04-24T08:37:27.631 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph-authtool: stdout AQDGxChmEwNFOhAAVpt7u5Lv6heYaK5c3mXqYA== 2024-04-24T08:37:27.634 INFO:teuthology.orchestra.run.smithi083.stdout:Creating initial monmap... 2024-04-24T08:37:28.728 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-24T08:37:28.728 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-04-24T08:37:28.728 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to bd3e039e-0215-11ef-bc93-c7b262605968 2024-04-24T08:37:28.728 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-24T08:37:28.728 INFO:teuthology.orchestra.run.smithi083.stdout:monmaptool for a [v2:172.21.15.83:3300,v1:172.21.15.83:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-24T08:37:28.728 INFO:teuthology.orchestra.run.smithi083.stdout:setting min_mon_release = quincy 2024-04-24T08:37:28.728 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/monmaptool: set fsid to bd3e039e-0215-11ef-bc93-c7b262605968 2024-04-24T08:37:28.728 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-24T08:37:28.728 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:37:28.729 INFO:teuthology.orchestra.run.smithi083.stdout:Creating mon... 2024-04-24T08:37:29.843 INFO:teuthology.orchestra.run.smithi083.stdout:create mon.a on 2024-04-24T08:37:30.241 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-04-24T08:37:30.491 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-04-24T08:37:30.771 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-bd3e039e-0215-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-bd3e039e-0215-11ef-bc93-c7b262605968.target. 2024-04-24T08:37:30.772 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-bd3e039e-0215-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-bd3e039e-0215-11ef-bc93-c7b262605968.target. 2024-04-24T08:37:31.154 INFO:teuthology.orchestra.run.smithi083.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-bd3e039e-0215-11ef-bc93-c7b262605968@mon.a 2024-04-24T08:37:31.154 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stderr Failed to reset failed state of unit ceph-bd3e039e-0215-11ef-bc93-c7b262605968@mon.a.service: Unit ceph-bd3e039e-0215-11ef-bc93-c7b262605968@mon.a.service not loaded. 2024-04-24T08:37:31.446 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-bd3e039e-0215-11ef-bc93-c7b262605968.target.wants/ceph-bd3e039e-0215-11ef-bc93-c7b262605968@mon.a.service → /etc/systemd/system/ceph-bd3e039e-0215-11ef-bc93-c7b262605968@.service. 2024-04-24T08:37:32.124 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:31 smithi083 podman[27514]: 2024-04-24 08:37:31.883714584 +0000 UTC m=+0.166002913 container create 7ad7979d238e98cb9987696d75f700bd4475f6d79cd993958a8d1afe555f29ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mon-a, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, io.buildah.version=1.33.5) 2024-04-24T08:37:32.124 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:32 smithi083 podman[27514]: 2024-04-24 08:37:32.12205324 +0000 UTC m=+0.404341603 container init 7ad7979d238e98cb9987696d75f700bd4475f6d79cd993958a8d1afe555f29ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mon-a, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True) 2024-04-24T08:37:32.284 INFO:teuthology.orchestra.run.smithi083.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-24T08:37:32.284 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stdout disabled 2024-04-24T08:37:32.305 INFO:teuthology.orchestra.run.smithi083.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-24T08:37:32.306 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stdout inactive 2024-04-24T08:37:32.306 INFO:teuthology.orchestra.run.smithi083.stdout:firewalld.service is not enabled 2024-04-24T08:37:32.306 INFO:teuthology.orchestra.run.smithi083.stdout:Not possible to enable service . firewalld.service is not available 2024-04-24T08:37:32.308 INFO:teuthology.orchestra.run.smithi083.stdout:Waiting for mon to start... 2024-04-24T08:37:32.308 INFO:teuthology.orchestra.run.smithi083.stdout:Waiting for mon... 2024-04-24T08:37:32.459 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:32 smithi083 podman[27514]: 2024-04-24 08:37:32.133854629 +0000 UTC m=+0.416142928 container start 7ad7979d238e98cb9987696d75f700bd4475f6d79cd993958a8d1afe555f29ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mon-a, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=rishabh-main-apr17-a654945-1f81fb3, ceph=True, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux ) 2024-04-24T08:37:32.460 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:32 smithi083 ceph-mon[27533]: mkfs bd3e039e-0215-11ef-bc93-c7b262605968 2024-04-24T08:37:32.460 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:32 smithi083 ceph-mon[27533]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-24T08:37:32.460 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:32 smithi083 bash[27514]: 7ad7979d238e98cb9987696d75f700bd4475f6d79cd993958a8d1afe555f29ed 2024-04-24T08:37:32.460 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:32 smithi083 systemd[1]: Started Ceph mon.a for bd3e039e-0215-11ef-bc93-c7b262605968. 2024-04-24T08:37:33.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:33 smithi083 ceph-mon[27533]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-24T08:37:33.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:33 smithi083 ceph-mon[27533]: monmap epoch 1 2024-04-24T08:37:33.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:33 smithi083 ceph-mon[27533]: fsid bd3e039e-0215-11ef-bc93-c7b262605968 2024-04-24T08:37:33.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:33 smithi083 ceph-mon[27533]: last_changed 2024-04-24T08:37:28.106852+0000 2024-04-24T08:37:33.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:33 smithi083 ceph-mon[27533]: created 2024-04-24T08:37:28.106852+0000 2024-04-24T08:37:33.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:33 smithi083 ceph-mon[27533]: min_mon_release 19 (squid) 2024-04-24T08:37:33.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:33 smithi083 ceph-mon[27533]: election_strategy: 1 2024-04-24T08:37:33.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:33 smithi083 ceph-mon[27533]: 0: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.a 2024-04-24T08:37:33.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:33 smithi083 ceph-mon[27533]: fsmap 2024-04-24T08:37:33.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:33 smithi083 ceph-mon[27533]: osdmap e1: 0 total, 0 up, 0 in 2024-04-24T08:37:33.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:33 smithi083 ceph-mon[27533]: mgrmap e1: no daemons active 2024-04-24T08:37:33.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:33 smithi083 ceph-mon[27533]: from='client.? 172.21.15.83:0/599568957' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-04-24T08:37:33.696 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout cluster: 2024-04-24T08:37:33.697 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout id: bd3e039e-0215-11ef-bc93-c7b262605968 2024-04-24T08:37:33.697 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-04-24T08:37:33.697 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 2024-04-24T08:37:33.699 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout services: 2024-04-24T08:37:33.699 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.795259s) 2024-04-24T08:37:33.699 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-04-24T08:37:33.699 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-04-24T08:37:33.699 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 2024-04-24T08:37:33.699 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout data: 2024-04-24T08:37:33.699 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-04-24T08:37:33.699 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-04-24T08:37:33.699 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-04-24T08:37:33.699 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout pgs: 2024-04-24T08:37:33.700 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 2024-04-24T08:37:33.700 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:32.912+0000 7fcf1b800700 1 Processor -- start 2024-04-24T08:37:33.704 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:32.912+0000 7fcf1b800700 1 -- start start 2024-04-24T08:37:33.704 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:32.912+0000 7fcf1b800700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf1407de30 0x7fcf1407c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:37:33.704 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:32.912+0000 7fcf1b800700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcf1407c870 con 0x7fcf1407de30 2024-04-24T08:37:33.704 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:32.913+0000 7fcf18e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf1407de30 0x7fcf1407c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:37:33.704 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:32.914+0000 7fcf18e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf1407de30 0x7fcf1407c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56758/0 (socket says 172.21.15.83:56758) 2024-04-24T08:37:33.705 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:32.914+0000 7fcf18e00700 1 -- 172.21.15.83:0/1528666416 learned_addr learned my addr 172.21.15.83:0/1528666416 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:37:33.705 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:32.914+0000 7fcf18e00700 1 -- 172.21.15.83:0/1528666416 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf1407c9f0 con 0x7fcf1407de30 2024-04-24T08:37:33.705 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:32.914+0000 7fcf18e00700 1 --2- 172.21.15.83:0/1528666416 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf1407de30 0x7fcf1407c330 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fcf040098a0 tx=0x7fcf04009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24aff95ea77a19bf server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:37:33.705 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:32.915+0000 7fcf13e00700 1 -- 172.21.15.83:0/1528666416 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcf04004160 con 0x7fcf1407de30 2024-04-24T08:37:33.705 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:32.915+0000 7fcf13e00700 1 -- 172.21.15.83:0/1528666416 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf04004300 con 0x7fcf1407de30 2024-04-24T08:37:33.705 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:32.915+0000 7fcf13e00700 1 -- 172.21.15.83:0/1528666416 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcf040036a0 con 0x7fcf1407de30 2024-04-24T08:37:33.705 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:32.915+0000 7fcf1b800700 1 -- 172.21.15.83:0/1528666416 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf1407de30 msgr2=0x7fcf1407c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:37:33.705 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:32.915+0000 7fcf1b800700 1 --2- 172.21.15.83:0/1528666416 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf1407de30 0x7fcf1407c330 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fcf040098a0 tx=0x7fcf04009bb0 comp rx=0 tx=0).stop 2024-04-24T08:37:33.705 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:32.915+0000 7fcf1b800700 1 -- 172.21.15.83:0/1528666416 shutdown_connections 2024-04-24T08:37:33.705 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:32.915+0000 7fcf1b800700 1 --2- 172.21.15.83:0/1528666416 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf1407de30 0x7fcf1407c330 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:37:33.705 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:32.915+0000 7fcf1b800700 1 -- 172.21.15.83:0/1528666416 >> 172.21.15.83:0/1528666416 conn(0x7fcf141039c0 msgr2=0x7fcf14105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:37:33.705 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:32.915+0000 7fcf1b800700 1 -- 172.21.15.83:0/1528666416 shutdown_connections 2024-04-24T08:37:33.705 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:32.916+0000 7fcf1b800700 1 -- 172.21.15.83:0/1528666416 wait complete. 2024-04-24T08:37:33.706 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:32.917+0000 7fcf1b800700 1 Processor -- start 2024-04-24T08:37:33.706 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:32.917+0000 7fcf1b800700 1 -- start start 2024-04-24T08:37:33.706 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:32.917+0000 7fcf1b800700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf1407de30 0x7fcf1410b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:37:33.706 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:32.917+0000 7fcf1b800700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcf1407cf20 con 0x7fcf1407de30 2024-04-24T08:37:33.706 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:32.917+0000 7fcf18e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf1407de30 0x7fcf1410b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:37:33.706 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:32.917+0000 7fcf18e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf1407de30 0x7fcf1410b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56766/0 (socket says 172.21.15.83:56766) 2024-04-24T08:37:33.706 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:32.917+0000 7fcf18e00700 1 -- 172.21.15.83:0/599568957 learned_addr learned my addr 172.21.15.83:0/599568957 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:37:33.706 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:32.918+0000 7fcf18e00700 1 -- 172.21.15.83:0/599568957 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf1410baf0 con 0x7fcf1407de30 2024-04-24T08:37:33.706 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:32.918+0000 7fcf18e00700 1 --2- 172.21.15.83:0/599568957 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf1407de30 0x7fcf1410b5b0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fcf04000c00 tx=0x7fcf04004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:37:33.706 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:32.918+0000 7fcf12000700 1 -- 172.21.15.83:0/599568957 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcf04003db0 con 0x7fcf1407de30 2024-04-24T08:37:33.706 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:32.918+0000 7fcf12000700 1 -- 172.21.15.83:0/599568957 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf04011630 con 0x7fcf1407de30 2024-04-24T08:37:33.706 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:32.918+0000 7fcf1b800700 1 -- 172.21.15.83:0/599568957 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf1410bd80 con 0x7fcf1407de30 2024-04-24T08:37:33.706 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:32.918+0000 7fcf12000700 1 -- 172.21.15.83:0/599568957 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcf04011950 con 0x7fcf1407de30 2024-04-24T08:37:33.707 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:32.918+0000 7fcf1b800700 1 -- 172.21.15.83:0/599568957 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf14108340 con 0x7fcf1407de30 2024-04-24T08:37:33.707 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:32.919+0000 7fcf12000700 1 -- 172.21.15.83:0/599568957 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fcf0401a460 con 0x7fcf1407de30 2024-04-24T08:37:33.707 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:32.919+0000 7fcf12000700 1 -- 172.21.15.83:0/599568957 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fcf0401b780 con 0x7fcf1407de30 2024-04-24T08:37:33.707 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:32.920+0000 7fcf1b800700 1 -- 172.21.15.83:0/599568957 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf1410bf50 con 0x7fcf1407de30 2024-04-24T08:37:33.707 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:32.923+0000 7fcf12000700 1 -- 172.21.15.83:0/599568957 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fcf1410bf50 con 0x7fcf1407de30 2024-04-24T08:37:33.707 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:33.002+0000 7fcf1b800700 1 -- 172.21.15.83:0/599568957 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7fcf14197310 con 0x7fcf1407de30 2024-04-24T08:37:33.707 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:33.002+0000 7fcf12000700 1 -- 172.21.15.83:0/599568957 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7fcf0401b980 con 0x7fcf1407de30 2024-04-24T08:37:33.707 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:33.003+0000 7fcf1b800700 1 -- 172.21.15.83:0/599568957 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf1407de30 msgr2=0x7fcf1410b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:37:33.707 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:33.003+0000 7fcf1b800700 1 --2- 172.21.15.83:0/599568957 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf1407de30 0x7fcf1410b5b0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fcf04000c00 tx=0x7fcf04004750 comp rx=0 tx=0).stop 2024-04-24T08:37:33.707 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:33.003+0000 7fcf1b800700 1 -- 172.21.15.83:0/599568957 shutdown_connections 2024-04-24T08:37:33.708 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:33.003+0000 7fcf1b800700 1 --2- 172.21.15.83:0/599568957 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf1407de30 0x7fcf1410b5b0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:37:33.708 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:33.003+0000 7fcf1b800700 1 -- 172.21.15.83:0/599568957 >> 172.21.15.83:0/599568957 conn(0x7fcf141039c0 msgr2=0x7fcf14104890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:37:33.708 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:33.004+0000 7fcf1b800700 1 -- 172.21.15.83:0/599568957 shutdown_connections 2024-04-24T08:37:33.708 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:33.004+0000 7fcf1b800700 1 -- 172.21.15.83:0/599568957 wait complete. 2024-04-24T08:37:33.708 INFO:teuthology.orchestra.run.smithi083.stdout:mon is available 2024-04-24T08:37:33.709 INFO:teuthology.orchestra.run.smithi083.stdout:Assimilating anything we can from ceph.conf... 2024-04-24T08:37:34.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:34 smithi083 ceph-mon[27533]: from='client.? 172.21.15.83:0/2114466497' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-24T08:37:34.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:34 smithi083 ceph-mon[27533]: from='client.? 172.21.15.83:0/2114466497' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-04-24T08:37:35.200 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 2024-04-24T08:37:35.200 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout [global] 2024-04-24T08:37:35.200 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout fsid = bd3e039e-0215-11ef-bc93-c7b262605968 2024-04-24T08:37:35.200 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.83:3300,v1:172.21.15.83:6789] 2024-04-24T08:37:35.200 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-24T08:37:35.200 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-24T08:37:35.200 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-24T08:37:35.200 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-24T08:37:35.200 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 2024-04-24T08:37:35.200 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-24T08:37:35.200 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-24T08:37:35.201 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 2024-04-24T08:37:35.201 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout [osd] 2024-04-24T08:37:35.201 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-24T08:37:35.201 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-24T08:37:35.201 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-24T08:37:35.201 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-24T08:37:35.201 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-24T08:37:35.201 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:34.361+0000 7f0d7ea00700 1 Processor -- start 2024-04-24T08:37:35.201 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:34.361+0000 7f0d7ea00700 1 -- start start 2024-04-24T08:37:35.201 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:34.362+0000 7f0d7ea00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d7810a540 0x7f0d7810a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:37:35.201 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:34.362+0000 7f0d7ea00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0d7810af30 con 0x7f0d7810a540 2024-04-24T08:37:35.201 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:34.362+0000 7f0d77e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d7810a540 0x7f0d7810a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:37:35.201 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:34.362+0000 7f0d77e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d7810a540 0x7f0d7810a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56770/0 (socket says 172.21.15.83:56770) 2024-04-24T08:37:35.202 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:34.362+0000 7f0d77e00700 1 -- 172.21.15.83:0/3368305631 learned_addr learned my addr 172.21.15.83:0/3368305631 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:37:35.202 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:34.363+0000 7f0d77e00700 1 -- 172.21.15.83:0/3368305631 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d7810b6f0 con 0x7f0d7810a540 2024-04-24T08:37:35.202 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:34.363+0000 7f0d77e00700 1 --2- 172.21.15.83:0/3368305631 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d7810a540 0x7f0d7810a960 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f0d680098a0 tx=0x7f0d68009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77ded02f2af23f6f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:37:35.202 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:34.363+0000 7f0d77400700 1 -- 172.21.15.83:0/3368305631 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d68004160 con 0x7f0d7810a540 2024-04-24T08:37:35.202 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:34.363+0000 7f0d77400700 1 -- 172.21.15.83:0/3368305631 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d68004300 con 0x7f0d7810a540 2024-04-24T08:37:35.202 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:34.363+0000 7f0d77400700 1 -- 172.21.15.83:0/3368305631 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d680036a0 con 0x7f0d7810a540 2024-04-24T08:37:35.202 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:34.364+0000 7f0d7ea00700 1 -- 172.21.15.83:0/3368305631 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d7810a540 msgr2=0x7f0d7810a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:37:35.202 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:34.364+0000 7f0d7ea00700 1 --2- 172.21.15.83:0/3368305631 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d7810a540 0x7f0d7810a960 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f0d680098a0 tx=0x7f0d68009bb0 comp rx=0 tx=0).stop 2024-04-24T08:37:35.202 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:34.364+0000 7f0d7ea00700 1 -- 172.21.15.83:0/3368305631 shutdown_connections 2024-04-24T08:37:35.202 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:34.364+0000 7f0d7ea00700 1 --2- 172.21.15.83:0/3368305631 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d7810a540 0x7f0d7810a960 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:37:35.202 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:34.364+0000 7f0d7ea00700 1 -- 172.21.15.83:0/3368305631 >> 172.21.15.83:0/3368305631 conn(0x7f0d7807dbe0 msgr2=0x7f0d781084c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:37:35.202 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:34.364+0000 7f0d7ea00700 1 -- 172.21.15.83:0/3368305631 shutdown_connections 2024-04-24T08:37:35.202 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:34.364+0000 7f0d7ea00700 1 -- 172.21.15.83:0/3368305631 wait complete. 2024-04-24T08:37:35.203 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:34.366+0000 7f0d7ea00700 1 Processor -- start 2024-04-24T08:37:35.203 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:34.366+0000 7f0d7ea00700 1 -- start start 2024-04-24T08:37:35.203 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:34.366+0000 7f0d7ea00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d7810a540 0x7f0d7819db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:37:35.203 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:34.366+0000 7f0d7ea00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0d7810bc20 con 0x7f0d7810a540 2024-04-24T08:37:35.203 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:34.366+0000 7f0d77e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d7810a540 0x7f0d7819db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:37:35.203 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:34.366+0000 7f0d77e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d7810a540 0x7f0d7819db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56776/0 (socket says 172.21.15.83:56776) 2024-04-24T08:37:35.203 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:34.366+0000 7f0d77e00700 1 -- 172.21.15.83:0/2114466497 learned_addr learned my addr 172.21.15.83:0/2114466497 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:37:35.203 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:34.367+0000 7f0d77e00700 1 -- 172.21.15.83:0/2114466497 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d7819e0d0 con 0x7f0d7810a540 2024-04-24T08:37:35.203 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:34.367+0000 7f0d77e00700 1 --2- 172.21.15.83:0/2114466497 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d7810a540 0x7f0d7819db90 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f0d68000c00 tx=0x7f0d68004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:37:35.203 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:34.367+0000 7f0d75600700 1 -- 172.21.15.83:0/2114466497 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d68003db0 con 0x7f0d7810a540 2024-04-24T08:37:35.203 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:34.367+0000 7f0d75600700 1 -- 172.21.15.83:0/2114466497 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d68011630 con 0x7f0d7810a540 2024-04-24T08:37:35.203 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:34.367+0000 7f0d7ea00700 1 -- 172.21.15.83:0/2114466497 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d7819e360 con 0x7f0d7810a540 2024-04-24T08:37:35.203 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:34.367+0000 7f0d75600700 1 -- 172.21.15.83:0/2114466497 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d68011950 con 0x7f0d7810a540 2024-04-24T08:37:35.204 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:34.368+0000 7f0d7ea00700 1 -- 172.21.15.83:0/2114466497 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d781a1050 con 0x7f0d7810a540 2024-04-24T08:37:35.204 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:34.368+0000 7f0d75600700 1 -- 172.21.15.83:0/2114466497 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0d68011af0 con 0x7f0d7810a540 2024-04-24T08:37:35.204 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:34.368+0000 7f0d75600700 1 -- 172.21.15.83:0/2114466497 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f0d6801c450 con 0x7f0d7810a540 2024-04-24T08:37:35.204 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:34.369+0000 7f0d7ea00700 1 -- 172.21.15.83:0/2114466497 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d78064150 con 0x7f0d7810a540 2024-04-24T08:37:35.204 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:34.372+0000 7f0d75600700 1 -- 172.21.15.83:0/2114466497 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f0d68032020 con 0x7f0d7810a540 2024-04-24T08:37:35.204 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:34.448+0000 7f0d7ea00700 1 -- 172.21.15.83:0/2114466497 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f0d7802e160 con 0x7f0d7810a540 2024-04-24T08:37:35.204 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:34.454+0000 7f0d75600700 1 -- 172.21.15.83:0/2114466497 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+429 (secure 0 0 0) 0x7f0d68004260 con 0x7f0d7810a540 2024-04-24T08:37:35.204 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:34.456+0000 7f0d7ea00700 1 -- 172.21.15.83:0/2114466497 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d7810a540 msgr2=0x7f0d7819db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:37:35.204 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:34.456+0000 7f0d7ea00700 1 --2- 172.21.15.83:0/2114466497 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d7810a540 0x7f0d7819db90 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f0d68000c00 tx=0x7f0d68004750 comp rx=0 tx=0).stop 2024-04-24T08:37:35.204 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:34.456+0000 7f0d7ea00700 1 -- 172.21.15.83:0/2114466497 shutdown_connections 2024-04-24T08:37:35.204 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:34.456+0000 7f0d7ea00700 1 --2- 172.21.15.83:0/2114466497 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d7810a540 0x7f0d7819db90 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:37:35.204 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:34.456+0000 7f0d7ea00700 1 -- 172.21.15.83:0/2114466497 >> 172.21.15.83:0/2114466497 conn(0x7f0d7807dbe0 msgr2=0x7f0d78194850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:37:35.204 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:34.457+0000 7f0d7ea00700 1 -- 172.21.15.83:0/2114466497 shutdown_connections 2024-04-24T08:37:35.204 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:34.457+0000 7f0d7ea00700 1 -- 172.21.15.83:0/2114466497 wait complete. 2024-04-24T08:37:35.205 INFO:teuthology.orchestra.run.smithi083.stdout:Generating new minimal ceph.conf... 2024-04-24T08:37:36.235 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:35 smithi083 ceph-mon[27533]: from='client.? 172.21.15.83:0/971596853' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:37:36.565 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:35.807+0000 7fd3c1800700 1 Processor -- start 2024-04-24T08:37:36.566 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:35.808+0000 7fd3c1800700 1 -- start start 2024-04-24T08:37:36.566 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:35.808+0000 7fd3c1800700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3bc10a570 0x7fd3bc10a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:37:36.566 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:35.808+0000 7fd3c1800700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd3bc10af60 con 0x7fd3bc10a570 2024-04-24T08:37:36.566 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:35.808+0000 7fd3baa00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3bc10a570 0x7fd3bc10a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:37:36.566 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:35.809+0000 7fd3baa00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3bc10a570 0x7fd3bc10a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56790/0 (socket says 172.21.15.83:56790) 2024-04-24T08:37:36.567 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:35.809+0000 7fd3baa00700 1 -- 172.21.15.83:0/4033540658 learned_addr learned my addr 172.21.15.83:0/4033540658 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:37:36.567 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:35.809+0000 7fd3baa00700 1 -- 172.21.15.83:0/4033540658 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3bc10b720 con 0x7fd3bc10a570 2024-04-24T08:37:36.569 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:35.809+0000 7fd3baa00700 1 --2- 172.21.15.83:0/4033540658 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3bc10a570 0x7fd3bc10a990 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac0098a0 tx=0x7fd3ac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97809a8a1a43b0d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:37:36.569 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:35.810+0000 7fd3ba000700 1 -- 172.21.15.83:0/4033540658 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd3ac004160 con 0x7fd3bc10a570 2024-04-24T08:37:36.569 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:35.810+0000 7fd3ba000700 1 -- 172.21.15.83:0/4033540658 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fd3ac004300 con 0x7fd3bc10a570 2024-04-24T08:37:36.569 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:35.810+0000 7fd3ba000700 1 -- 172.21.15.83:0/4033540658 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd3ac011a80 con 0x7fd3bc10a570 2024-04-24T08:37:36.569 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:35.810+0000 7fd3c1800700 1 -- 172.21.15.83:0/4033540658 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3bc10a570 msgr2=0x7fd3bc10a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:37:36.569 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:35.810+0000 7fd3c1800700 1 --2- 172.21.15.83:0/4033540658 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3bc10a570 0x7fd3bc10a990 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac0098a0 tx=0x7fd3ac009bb0 comp rx=0 tx=0).stop 2024-04-24T08:37:36.569 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:35.810+0000 7fd3c1800700 1 -- 172.21.15.83:0/4033540658 shutdown_connections 2024-04-24T08:37:36.569 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:35.811+0000 7fd3c1800700 1 --2- 172.21.15.83:0/4033540658 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3bc10a570 0x7fd3bc10a990 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:37:36.570 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:35.811+0000 7fd3c1800700 1 -- 172.21.15.83:0/4033540658 >> 172.21.15.83:0/4033540658 conn(0x7fd3bc07d9d0 msgr2=0x7fd3bc1084c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:37:36.570 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:35.811+0000 7fd3c1800700 1 -- 172.21.15.83:0/4033540658 shutdown_connections 2024-04-24T08:37:36.570 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:35.811+0000 7fd3c1800700 1 -- 172.21.15.83:0/4033540658 wait complete. 2024-04-24T08:37:36.570 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:35.812+0000 7fd3c1800700 1 Processor -- start 2024-04-24T08:37:36.570 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:35.812+0000 7fd3c1800700 1 -- start start 2024-04-24T08:37:36.570 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:35.812+0000 7fd3c1800700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3bc10a570 0x7fd3bc19e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:37:36.570 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:35.812+0000 7fd3c1800700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd3bc10ead0 con 0x7fd3bc10a570 2024-04-24T08:37:36.570 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:35.812+0000 7fd3baa00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3bc10a570 0x7fd3bc19e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:37:36.570 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:35.812+0000 7fd3baa00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3bc10a570 0x7fd3bc19e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56798/0 (socket says 172.21.15.83:56798) 2024-04-24T08:37:36.570 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:35.812+0000 7fd3baa00700 1 -- 172.21.15.83:0/971596853 learned_addr learned my addr 172.21.15.83:0/971596853 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:37:36.570 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:35.813+0000 7fd3baa00700 1 -- 172.21.15.83:0/971596853 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3bc19e9f0 con 0x7fd3bc10a570 2024-04-24T08:37:36.570 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:35.813+0000 7fd3baa00700 1 --2- 172.21.15.83:0/971596853 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3bc10a570 0x7fd3bc19e4b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac000c00 tx=0x7fd3ac004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:37:36.573 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:35.813+0000 7fd3b3400700 1 -- 172.21.15.83:0/971596853 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd3ac0039c0 con 0x7fd3bc10a570 2024-04-24T08:37:36.573 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:35.813+0000 7fd3b3400700 1 -- 172.21.15.83:0/971596853 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fd3ac003b60 con 0x7fd3bc10a570 2024-04-24T08:37:36.573 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:35.813+0000 7fd3b3400700 1 -- 172.21.15.83:0/971596853 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd3ac01a6f0 con 0x7fd3bc10a570 2024-04-24T08:37:36.573 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:35.813+0000 7fd3c1800700 1 -- 172.21.15.83:0/971596853 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3bc19ec80 con 0x7fd3bc10a570 2024-04-24T08:37:36.573 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:35.813+0000 7fd3c1800700 1 -- 172.21.15.83:0/971596853 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3bc1a1970 con 0x7fd3bc10a570 2024-04-24T08:37:36.573 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:35.814+0000 7fd3b3400700 1 -- 172.21.15.83:0/971596853 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd3ac01ac20 con 0x7fd3bc10a570 2024-04-24T08:37:36.573 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:35.814+0000 7fd3b3400700 1 -- 172.21.15.83:0/971596853 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd3ac019700 con 0x7fd3bc10a570 2024-04-24T08:37:36.573 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:35.815+0000 7fd3c1800700 1 -- 172.21.15.83:0/971596853 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3bc064150 con 0x7fd3bc10a570 2024-04-24T08:37:36.573 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:35.818+0000 7fd3b3400700 1 -- 172.21.15.83:0/971596853 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fd3ac004300 con 0x7fd3bc10a570 2024-04-24T08:37:36.573 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:35.895+0000 7fd3c1800700 1 -- 172.21.15.83:0/971596853 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fd3bc1a1b40 con 0x7fd3bc10a570 2024-04-24T08:37:36.573 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:35.896+0000 7fd3b3400700 1 -- 172.21.15.83:0/971596853 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7fd3ac024a40 con 0x7fd3bc10a570 2024-04-24T08:37:36.573 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:35.897+0000 7fd3c1800700 1 -- 172.21.15.83:0/971596853 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3bc10a570 msgr2=0x7fd3bc19e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:37:36.574 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:35.897+0000 7fd3c1800700 1 --2- 172.21.15.83:0/971596853 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3bc10a570 0x7fd3bc19e4b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac000c00 tx=0x7fd3ac004290 comp rx=0 tx=0).stop 2024-04-24T08:37:36.574 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:35.898+0000 7fd3c1800700 1 -- 172.21.15.83:0/971596853 shutdown_connections 2024-04-24T08:37:36.574 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:35.898+0000 7fd3c1800700 1 --2- 172.21.15.83:0/971596853 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3bc10a570 0x7fd3bc19e4b0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:37:36.574 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:35.898+0000 7fd3c1800700 1 -- 172.21.15.83:0/971596853 >> 172.21.15.83:0/971596853 conn(0x7fd3bc07d9d0 msgr2=0x7fd3bc107c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:37:36.574 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:35.898+0000 7fd3c1800700 1 -- 172.21.15.83:0/971596853 shutdown_connections 2024-04-24T08:37:36.574 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:35.898+0000 7fd3c1800700 1 -- 172.21.15.83:0/971596853 wait complete. 2024-04-24T08:37:36.574 INFO:teuthology.orchestra.run.smithi083.stdout:Restarting the monitor... 2024-04-24T08:37:37.016 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:36 smithi083 systemd[1]: Stopping Ceph mon.a for bd3e039e-0215-11ef-bc93-c7b262605968... 2024-04-24T08:37:37.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:37 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mon-a[27529]: 2024-04-24T08:37:37.014+0000 7fd3f6e00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T08:37:37.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:37 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mon-a[27529]: 2024-04-24T08:37:37.014+0000 7fd3f6e00700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-04-24T08:37:37.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:37 smithi083 podman[27776]: 2024-04-24 08:37:37.12811115 +0000 UTC m=+0.238440357 container died 7ad7979d238e98cb9987696d75f700bd4475f6d79cd993958a8d1afe555f29ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mon-a, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T08:37:37.690 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:37 smithi083 podman[27776]: 2024-04-24 08:37:37.585611523 +0000 UTC m=+0.695940704 container cleanup 7ad7979d238e98cb9987696d75f700bd4475f6d79cd993958a8d1afe555f29ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mon-a, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, ceph=True) 2024-04-24T08:37:37.690 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:37 smithi083 bash[27776]: ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mon-a 2024-04-24T08:37:38.013 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:37 smithi083 podman[27791]: 2024-04-24 08:37:37.811804337 +0000 UTC m=+0.669927585 container remove 7ad7979d238e98cb9987696d75f700bd4475f6d79cd993958a8d1afe555f29ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mon-a, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-24T08:37:38.263 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 systemd[1]: ceph-bd3e039e-0215-11ef-bc93-c7b262605968@mon.a.service: Deactivated successfully. 2024-04-24T08:37:38.263 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 systemd[1]: Stopped Ceph mon.a for bd3e039e-0215-11ef-bc93-c7b262605968. 2024-04-24T08:37:38.263 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 systemd[1]: ceph-bd3e039e-0215-11ef-bc93-c7b262605968@mon.a.service: Consumed 1.760s CPU time. 2024-04-24T08:37:38.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 systemd[1]: Starting Ceph mon.a for bd3e039e-0215-11ef-bc93-c7b262605968... 2024-04-24T08:37:38.920 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 podman[27887]: 2024-04-24 08:37:38.520283296 +0000 UTC m=+0.031414693 image pull b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 2024-04-24T08:37:38.920 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 podman[27887]: 2024-04-24 08:37:38.63790415 +0000 UTC m=+0.149035614 container create 2eee14394f4c6bafb289fa6aabfe4f3e96dc4d11d2c3e85ddc981d229db7d05f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mon-a, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.schema-version=1.0, ceph=True) 2024-04-24T08:37:38.920 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 podman[27887]: 2024-04-24 08:37:38.917789328 +0000 UTC m=+0.428920768 container init 2eee14394f4c6bafb289fa6aabfe4f3e96dc4d11d2c3e85ddc981d229db7d05f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mon-a, maintainer=Guillaume Abrioux , RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-24T08:37:39.018 INFO:teuthology.orchestra.run.smithi083.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-04-24T08:37:39.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 podman[27887]: 2024-04-24 08:37:38.929653652 +0000 UTC m=+0.440785093 container start 2eee14394f4c6bafb289fa6aabfe4f3e96dc4d11d2c3e85ddc981d229db7d05f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mon-a, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-04-24T08:37:39.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: set uid:gid to 167:167 (ceph:ceph) 2024-04-24T08:37:39.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev), process ceph-mon, pid 2 2024-04-24T08:37:39.290 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: pidfile_write: ignore empty --pid-file 2024-04-24T08:37:39.291 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: load: jerasure load: lrc 2024-04-24T08:37:39.291 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: RocksDB version: 7.9.2 2024-04-24T08:37:39.291 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Git sha 0 2024-04-24T08:37:39.291 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Compile date 2024-04-23 21:06:33 2024-04-24T08:37:39.291 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: DB SUMMARY 2024-04-24T08:37:39.291 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: DB Session ID: M2ASL9YJXZYY9S6LK0ZY 2024-04-24T08:37:39.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: CURRENT file: CURRENT 2024-04-24T08:37:39.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: IDENTITY file: IDENTITY 2024-04-24T08:37:39.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-04-24T08:37:39.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-04-24T08:37:39.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 103950 ; 2024-04-24T08:37:39.292 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.error_if_exists: 0 2024-04-24T08:37:39.293 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.create_if_missing: 0 2024-04-24T08:37:39.293 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.paranoid_checks: 1 2024-04-24T08:37:39.293 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-24T08:37:39.293 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-24T08:37:39.293 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-24T08:37:39.294 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.env: 0x56212e779cc0 2024-04-24T08:37:39.294 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.fs: PosixFileSystem 2024-04-24T08:37:39.294 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.info_log: 0x562150da3c60 2024-04-24T08:37:39.294 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.max_file_opening_threads: 16 2024-04-24T08:37:39.295 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.statistics: (nil) 2024-04-24T08:37:39.295 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.use_fsync: 0 2024-04-24T08:37:39.295 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.max_log_file_size: 0 2024-04-24T08:37:39.295 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-24T08:37:39.295 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-24T08:37:39.296 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.keep_log_file_num: 1000 2024-04-24T08:37:39.296 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.recycle_log_file_num: 0 2024-04-24T08:37:39.296 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.allow_fallocate: 1 2024-04-24T08:37:39.298 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.allow_mmap_reads: 0 2024-04-24T08:37:39.298 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.allow_mmap_writes: 0 2024-04-24T08:37:39.299 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.use_direct_reads: 0 2024-04-24T08:37:39.299 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-24T08:37:39.299 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.create_missing_column_families: 0 2024-04-24T08:37:39.299 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.db_log_dir: 2024-04-24T08:37:39.299 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.wal_dir: 2024-04-24T08:37:39.300 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-24T08:37:39.300 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-24T08:37:39.300 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-24T08:37:39.300 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-24T08:37:39.301 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-24T08:37:39.301 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-24T08:37:39.301 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.advise_random_on_open: 1 2024-04-24T08:37:39.301 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.db_write_buffer_size: 0 2024-04-24T08:37:39.301 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.write_buffer_manager: 0x562150d721e0 2024-04-24T08:37:39.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-24T08:37:39.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-24T08:37:39.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-24T08:37:39.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.rate_limiter: (nil) 2024-04-24T08:37:39.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-24T08:37:39.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.wal_recovery_mode: 2 2024-04-24T08:37:39.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.enable_thread_tracking: 0 2024-04-24T08:37:39.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.enable_pipelined_write: 0 2024-04-24T08:37:39.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.unordered_write: 0 2024-04-24T08:37:39.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-24T08:37:39.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-24T08:37:39.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-24T08:37:39.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-24T08:37:39.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.row_cache: None 2024-04-24T08:37:39.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.wal_filter: None 2024-04-24T08:37:39.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-24T08:37:39.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.allow_ingest_behind: 0 2024-04-24T08:37:39.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.two_write_queues: 0 2024-04-24T08:37:39.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.manual_wal_flush: 0 2024-04-24T08:37:39.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.wal_compression: 0 2024-04-24T08:37:39.306 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.atomic_flush: 0 2024-04-24T08:37:39.306 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-24T08:37:39.306 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-24T08:37:39.306 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-24T08:37:39.307 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.log_readahead_size: 0 2024-04-24T08:37:39.307 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-24T08:37:39.307 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.best_efforts_recovery: 0 2024-04-24T08:37:39.307 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-24T08:37:39.307 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-24T08:37:39.308 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.allow_data_in_errors: 0 2024-04-24T08:37:39.308 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.db_host_id: __hostname__ 2024-04-24T08:37:39.308 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-24T08:37:39.308 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.max_background_jobs: 2 2024-04-24T08:37:39.308 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.max_background_compactions: -1 2024-04-24T08:37:39.309 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.max_subcompactions: 1 2024-04-24T08:37:39.309 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-24T08:37:39.309 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-24T08:37:39.309 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-24T08:37:39.309 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.max_total_wal_size: 0 2024-04-24T08:37:39.309 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-24T08:37:39.309 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-24T08:37:39.310 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-24T08:37:39.310 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-24T08:37:39.310 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.max_open_files: -1 2024-04-24T08:37:39.310 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.bytes_per_sync: 0 2024-04-24T08:37:39.310 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-24T08:37:39.310 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-24T08:37:39.311 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.compaction_readahead_size: 0 2024-04-24T08:37:39.311 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.max_background_flushes: -1 2024-04-24T08:37:39.311 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Compression algorithms supported: 2024-04-24T08:37:39.311 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-24T08:37:39.311 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: kZSTD supported: 0 2024-04-24T08:37:39.311 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: kXpressCompression supported: 0 2024-04-24T08:37:39.311 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: kLZ4HCCompression supported: 1 2024-04-24T08:37:39.312 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: kZlibCompression supported: 1 2024-04-24T08:37:39.312 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: kSnappyCompression supported: 1 2024-04-24T08:37:39.312 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: kLZ4Compression supported: 1 2024-04-24T08:37:39.312 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: kBZip2Compression supported: 0 2024-04-24T08:37:39.312 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-24T08:37:39.312 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-24T08:37:39.313 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-04-24T08:37:39.313 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-24T08:37:39.313 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-24T08:37:39.313 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.merge_operator: 2024-04-24T08:37:39.313 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.compaction_filter: None 2024-04-24T08:37:39.314 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.compaction_filter_factory: None 2024-04-24T08:37:39.314 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.sst_partitioner_factory: None 2024-04-24T08:37:39.314 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-24T08:37:39.314 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-24T08:37:39.314 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x562150da3e20) 2024-04-24T08:37:39.315 INFO:journalctl@ceph.mon.a.smithi083.stdout: cache_index_and_filter_blocks: 1 2024-04-24T08:37:39.315 INFO:journalctl@ceph.mon.a.smithi083.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-24T08:37:39.315 INFO:journalctl@ceph.mon.a.smithi083.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-24T08:37:39.315 INFO:journalctl@ceph.mon.a.smithi083.stdout: pin_top_level_index_and_filter: 1 2024-04-24T08:37:39.315 INFO:journalctl@ceph.mon.a.smithi083.stdout: index_type: 0 2024-04-24T08:37:39.316 INFO:journalctl@ceph.mon.a.smithi083.stdout: data_block_index_type: 0 2024-04-24T08:37:39.316 INFO:journalctl@ceph.mon.a.smithi083.stdout: index_shortening: 1 2024-04-24T08:37:39.316 INFO:journalctl@ceph.mon.a.smithi083.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-24T08:37:39.316 INFO:journalctl@ceph.mon.a.smithi083.stdout: checksum: 4 2024-04-24T08:37:39.316 INFO:journalctl@ceph.mon.a.smithi083.stdout: no_block_cache: 0 2024-04-24T08:37:39.317 INFO:journalctl@ceph.mon.a.smithi083.stdout: block_cache: 0x562150da5090 2024-04-24T08:37:39.317 INFO:journalctl@ceph.mon.a.smithi083.stdout: block_cache_name: BinnedLRUCache 2024-04-24T08:37:39.317 INFO:journalctl@ceph.mon.a.smithi083.stdout: block_cache_options: 2024-04-24T08:37:39.317 INFO:journalctl@ceph.mon.a.smithi083.stdout: capacity : 536870912 2024-04-24T08:37:39.317 INFO:journalctl@ceph.mon.a.smithi083.stdout: num_shard_bits : 4 2024-04-24T08:37:39.318 INFO:journalctl@ceph.mon.a.smithi083.stdout: strict_capacity_limit : 0 2024-04-24T08:37:39.318 INFO:journalctl@ceph.mon.a.smithi083.stdout: high_pri_pool_ratio: 0.000 2024-04-24T08:37:39.318 INFO:journalctl@ceph.mon.a.smithi083.stdout: block_cache_compressed: (nil) 2024-04-24T08:37:39.318 INFO:journalctl@ceph.mon.a.smithi083.stdout: persistent_cache: (nil) 2024-04-24T08:37:39.318 INFO:journalctl@ceph.mon.a.smithi083.stdout: block_size: 4096 2024-04-24T08:37:39.318 INFO:journalctl@ceph.mon.a.smithi083.stdout: block_size_deviation: 10 2024-04-24T08:37:39.319 INFO:journalctl@ceph.mon.a.smithi083.stdout: block_restart_interval: 16 2024-04-24T08:37:39.319 INFO:journalctl@ceph.mon.a.smithi083.stdout: index_block_restart_interval: 1 2024-04-24T08:37:39.319 INFO:journalctl@ceph.mon.a.smithi083.stdout: metadata_block_size: 4096 2024-04-24T08:37:39.319 INFO:journalctl@ceph.mon.a.smithi083.stdout: partition_filters: 0 2024-04-24T08:37:39.319 INFO:journalctl@ceph.mon.a.smithi083.stdout: use_delta_encoding: 1 2024-04-24T08:37:39.319 INFO:journalctl@ceph.mon.a.smithi083.stdout: filter_policy: bloomfilter 2024-04-24T08:37:39.320 INFO:journalctl@ceph.mon.a.smithi083.stdout: whole_key_filtering: 1 2024-04-24T08:37:39.320 INFO:journalctl@ceph.mon.a.smithi083.stdout: verify_compression: 0 2024-04-24T08:37:39.320 INFO:journalctl@ceph.mon.a.smithi083.stdout: read_amp_bytes_per_bit: 0 2024-04-24T08:37:39.320 INFO:journalctl@ceph.mon.a.smithi083.stdout: format_version: 5 2024-04-24T08:37:39.320 INFO:journalctl@ceph.mon.a.smithi083.stdout: enable_index_compression: 1 2024-04-24T08:37:39.320 INFO:journalctl@ceph.mon.a.smithi083.stdout: block_align: 0 2024-04-24T08:37:39.320 INFO:journalctl@ceph.mon.a.smithi083.stdout: max_auto_readahead_size: 262144 2024-04-24T08:37:39.321 INFO:journalctl@ceph.mon.a.smithi083.stdout: prepopulate_block_cache: 0 2024-04-24T08:37:39.321 INFO:journalctl@ceph.mon.a.smithi083.stdout: initial_auto_readahead_size: 8192 2024-04-24T08:37:39.321 INFO:journalctl@ceph.mon.a.smithi083.stdout: num_file_reads_for_auto_readahead: 2 2024-04-24T08:37:39.321 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.write_buffer_size: 33554432 2024-04-24T08:37:39.321 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.max_write_buffer_number: 2 2024-04-24T08:37:39.321 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.compression: NoCompression 2024-04-24T08:37:39.322 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.bottommost_compression: Disabled 2024-04-24T08:37:39.322 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.prefix_extractor: nullptr 2024-04-24T08:37:39.322 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-24T08:37:39.322 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.num_levels: 7 2024-04-24T08:37:39.322 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-24T08:37:39.322 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-24T08:37:39.323 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-24T08:37:39.323 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-24T08:37:39.323 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-24T08:37:39.323 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-24T08:37:39.323 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-24T08:37:39.323 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-24T08:37:39.323 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-24T08:37:39.324 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-24T08:37:39.324 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-24T08:37:39.324 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-24T08:37:39.324 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-24T08:37:39.324 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.compression_opts.level: 32767 2024-04-24T08:37:39.324 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.compression_opts.strategy: 0 2024-04-24T08:37:39.325 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-24T08:37:39.325 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-24T08:37:39.325 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-24T08:37:39.325 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-24T08:37:39.325 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.compression_opts.enabled: false 2024-04-24T08:37:39.325 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-24T08:37:39.326 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-24T08:37:39.326 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-24T08:37:39.326 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-24T08:37:39.326 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.target_file_size_base: 67108864 2024-04-24T08:37:39.326 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-24T08:37:39.326 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-24T08:37:39.326 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-24T08:37:39.327 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-24T08:37:39.327 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-24T08:37:39.327 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-24T08:37:39.327 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-24T08:37:39.327 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-24T08:37:39.327 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-24T08:37:39.328 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-24T08:37:39.328 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-24T08:37:39.328 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-24T08:37:39.328 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-24T08:37:39.328 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-24T08:37:39.330 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.arena_block_size: 1048576 2024-04-24T08:37:39.330 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-24T08:37:39.330 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-24T08:37:39.330 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.disable_auto_compactions: 0 2024-04-24T08:37:39.330 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-24T08:37:39.330 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-24T08:37:39.331 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-24T08:37:39.331 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-24T08:37:39.331 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-24T08:37:39.331 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-24T08:37:39.331 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-24T08:37:39.331 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-24T08:37:39.332 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-24T08:37:39.332 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-24T08:37:39.332 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-24T08:37:39.332 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.inplace_update_support: 0 2024-04-24T08:37:39.332 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-24T08:37:39.332 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-24T08:37:39.333 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-24T08:37:39.333 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-24T08:37:39.333 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.bloom_locality: 0 2024-04-24T08:37:39.333 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.max_successive_merges: 0 2024-04-24T08:37:39.333 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-24T08:37:39.333 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.paranoid_file_checks: 0 2024-04-24T08:37:39.334 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.force_consistency_checks: 1 2024-04-24T08:37:39.334 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.report_bg_io_stats: 0 2024-04-24T08:37:39.334 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.ttl: 2592000 2024-04-24T08:37:39.334 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-24T08:37:39.334 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-24T08:37:39.334 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-24T08:37:39.335 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.enable_blob_files: false 2024-04-24T08:37:39.335 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.min_blob_size: 0 2024-04-24T08:37:39.335 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.blob_file_size: 268435456 2024-04-24T08:37:39.335 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-24T08:37:39.335 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-24T08:37:39.335 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-24T08:37:39.336 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-24T08:37:39.336 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-24T08:37:39.336 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.blob_file_starting_level: 0 2024-04-24T08:37:39.336 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-24T08:37:39.336 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-04-24T08:37:39.336 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-04-24T08:37:39.336 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: c005b8c0-0148-4934-a60c-31fe5813b509 2024-04-24T08:37:39.337 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713947858985954, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-04-24T08:37:39.337 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-04-24T08:37:39.337 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713947858987944, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 99163, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 322, "table_properties": {"data_size": 97122, "index_size": 297, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13899, "raw_average_key_size": 49, "raw_value_size": 89536, "raw_average_value_size": 318, "num_data_blocks": 12, "num_entries": 281, "num_filter_entries": 281, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1713947858, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "c005b8c0-0148-4934-a60c-31fe5813b509", "db_session_id": "M2ASL9YJXZYY9S6LK0ZY", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-04-24T08:37:39.337 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713947858988090, "job": 1, "event": "recovery_finished"} 2024-04-24T08:37:39.338 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-04-24T08:37:39.338 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-24T08:37:39.338 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x562150e92000 2024-04-24T08:37:39.338 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: DB pointer 0x562150e7c000 2024-04-24T08:37:39.338 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-24T08:37:39.338 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-24T08:37:39.338 INFO:journalctl@ceph.mon.a.smithi083.stdout: ** DB Stats ** 2024-04-24T08:37:39.338 INFO:journalctl@ceph.mon.a.smithi083.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-24T08:37:39.339 INFO:journalctl@ceph.mon.a.smithi083.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-24T08:37:39.339 INFO:journalctl@ceph.mon.a.smithi083.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-24T08:37:39.339 INFO:journalctl@ceph.mon.a.smithi083.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-24T08:37:39.339 INFO:journalctl@ceph.mon.a.smithi083.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-24T08:37:39.339 INFO:journalctl@ceph.mon.a.smithi083.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-24T08:37:39.339 INFO:journalctl@ceph.mon.a.smithi083.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-24T08:37:39.339 INFO:journalctl@ceph.mon.a.smithi083.stdout: 2024-04-24T08:37:39.339 INFO:journalctl@ceph.mon.a.smithi083.stdout: ** Compaction Stats [default] ** 2024-04-24T08:37:39.339 INFO:journalctl@ceph.mon.a.smithi083.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-24T08:37:39.339 INFO:journalctl@ceph.mon.a.smithi083.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-24T08:37:39.339 INFO:journalctl@ceph.mon.a.smithi083.stdout: L0 2/0 98.70 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 69.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-24T08:37:39.339 INFO:journalctl@ceph.mon.a.smithi083.stdout: Sum 2/0 98.70 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 69.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-24T08:37:39.339 INFO:journalctl@ceph.mon.a.smithi083.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 69.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-24T08:37:39.339 INFO:journalctl@ceph.mon.a.smithi083.stdout: 2024-04-24T08:37:39.340 INFO:journalctl@ceph.mon.a.smithi083.stdout: ** Compaction Stats [default] ** 2024-04-24T08:37:39.340 INFO:journalctl@ceph.mon.a.smithi083.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-24T08:37:39.340 INFO:journalctl@ceph.mon.a.smithi083.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-24T08:37:39.340 INFO:journalctl@ceph.mon.a.smithi083.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 69.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-24T08:37:39.340 INFO:journalctl@ceph.mon.a.smithi083.stdout: 2024-04-24T08:37:39.340 INFO:journalctl@ceph.mon.a.smithi083.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-24T08:37:39.340 INFO:journalctl@ceph.mon.a.smithi083.stdout: 2024-04-24T08:37:39.340 INFO:journalctl@ceph.mon.a.smithi083.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-24T08:37:39.340 INFO:journalctl@ceph.mon.a.smithi083.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-24T08:37:39.340 INFO:journalctl@ceph.mon.a.smithi083.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-24T08:37:39.340 INFO:journalctl@ceph.mon.a.smithi083.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-24T08:37:39.340 INFO:journalctl@ceph.mon.a.smithi083.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-24T08:37:39.340 INFO:journalctl@ceph.mon.a.smithi083.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-24T08:37:39.340 INFO:journalctl@ceph.mon.a.smithi083.stdout: Cumulative compaction: 0.00 GB write, 15.06 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-24T08:37:39.340 INFO:journalctl@ceph.mon.a.smithi083.stdout: Interval compaction: 0.00 GB write, 15.06 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-24T08:37:39.341 INFO:journalctl@ceph.mon.a.smithi083.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-24T08:37:39.341 INFO:journalctl@ceph.mon.a.smithi083.stdout: Block cache BinnedLRUCache@0x562150da5090#2 capacity: 512.00 MB usage: 1.39 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-04-24T08:37:39.341 INFO:journalctl@ceph.mon.a.smithi083.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-04-24T08:37:39.341 INFO:journalctl@ceph.mon.a.smithi083.stdout: 2024-04-24T08:37:39.341 INFO:journalctl@ceph.mon.a.smithi083.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-24T08:37:39.341 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: starting mon.a rank 0 at public addrs [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] at bind addrs [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid bd3e039e-0215-11ef-bc93-c7b262605968 2024-04-24T08:37:39.341 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: mon.a@-1(???) e1 preinit fsid bd3e039e-0215-11ef-bc93-c7b262605968 2024-04-24T08:37:39.341 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: mon.a@-1(???).mds e1 new map 2024-04-24T08:37:39.341 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: mon.a@-1(???).mds e1 print_map 2024-04-24T08:37:39.341 INFO:journalctl@ceph.mon.a.smithi083.stdout: e1 2024-04-24T08:37:39.341 INFO:journalctl@ceph.mon.a.smithi083.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-24T08:37:39.341 INFO:journalctl@ceph.mon.a.smithi083.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-24T08:37:39.341 INFO:journalctl@ceph.mon.a.smithi083.stdout: legacy client fscid: -1 2024-04-24T08:37:39.341 INFO:journalctl@ceph.mon.a.smithi083.stdout: 2024-04-24T08:37:39.342 INFO:journalctl@ceph.mon.a.smithi083.stdout: No filesystems configured 2024-04-24T08:37:39.342 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-24T08:37:39.342 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-24T08:37:39.342 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-24T08:37:39.342 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-24T08:37:39.342 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:38 smithi083 ceph-mon[27907]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-04-24T08:37:39.342 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:39 smithi083 bash[27887]: 2eee14394f4c6bafb289fa6aabfe4f3e96dc4d11d2c3e85ddc981d229db7d05f 2024-04-24T08:37:39.342 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:39 smithi083 systemd[1]: Started Ceph mon.a for bd3e039e-0215-11ef-bc93-c7b262605968. 2024-04-24T08:37:39.342 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:39 smithi083 ceph-mon[27907]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-24T08:37:39.342 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:39 smithi083 ceph-mon[27907]: monmap epoch 1 2024-04-24T08:37:39.342 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:39 smithi083 ceph-mon[27907]: fsid bd3e039e-0215-11ef-bc93-c7b262605968 2024-04-24T08:37:39.342 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:39 smithi083 ceph-mon[27907]: last_changed 2024-04-24T08:37:28.106852+0000 2024-04-24T08:37:39.342 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:39 smithi083 ceph-mon[27907]: created 2024-04-24T08:37:28.106852+0000 2024-04-24T08:37:39.342 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:39 smithi083 ceph-mon[27907]: min_mon_release 19 (squid) 2024-04-24T08:37:39.343 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:39 smithi083 ceph-mon[27907]: election_strategy: 1 2024-04-24T08:37:39.343 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:39 smithi083 ceph-mon[27907]: 0: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.a 2024-04-24T08:37:39.343 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:39 smithi083 ceph-mon[27907]: fsmap 2024-04-24T08:37:39.343 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:39 smithi083 ceph-mon[27907]: osdmap e1: 0 total, 0 up, 0 in 2024-04-24T08:37:39.343 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:39 smithi083 ceph-mon[27907]: mgrmap e1: no daemons active 2024-04-24T08:37:40.360 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:39.613+0000 7f77f3800700 1 Processor -- start 2024-04-24T08:37:40.360 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:39.613+0000 7f77f3800700 1 -- start start 2024-04-24T08:37:40.360 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:39.613+0000 7f77f3800700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77ec10ef20 0x7f77ec10f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:37:40.360 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:39.613+0000 7f77f3800700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f77ec10f880 con 0x7f77ec10ef20 2024-04-24T08:37:40.360 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:39.614+0000 7f77f0e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77ec10ef20 0x7f77ec10f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:37:40.360 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:39.614+0000 7f77f0e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77ec10ef20 0x7f77ec10f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33458/0 (socket says 172.21.15.83:33458) 2024-04-24T08:37:40.360 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:39.614+0000 7f77f0e00700 1 -- 172.21.15.83:0/1196522467 learned_addr learned my addr 172.21.15.83:0/1196522467 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:37:40.360 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:39.614+0000 7f77f0e00700 1 -- 172.21.15.83:0/1196522467 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77ec10fa00 con 0x7f77ec10ef20 2024-04-24T08:37:40.361 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:39.615+0000 7f77f0e00700 1 --2- 172.21.15.83:0/1196522467 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77ec10ef20 0x7f77ec10f340 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f77dc0098a0 tx=0x7f77dc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=864717cd31bd1d89 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:37:40.361 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:39.615+0000 7f77ebe00700 1 -- 172.21.15.83:0/1196522467 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f77dc004160 con 0x7f77ec10ef20 2024-04-24T08:37:40.361 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:39.615+0000 7f77ebe00700 1 -- 172.21.15.83:0/1196522467 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f77dc004300 con 0x7f77ec10ef20 2024-04-24T08:37:40.361 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:39.615+0000 7f77ebe00700 1 -- 172.21.15.83:0/1196522467 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f77dc011a80 con 0x7f77ec10ef20 2024-04-24T08:37:40.361 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:39.616+0000 7f77f3800700 1 -- 172.21.15.83:0/1196522467 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77ec10ef20 msgr2=0x7f77ec10f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:37:40.364 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:39.616+0000 7f77f3800700 1 --2- 172.21.15.83:0/1196522467 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77ec10ef20 0x7f77ec10f340 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f77dc0098a0 tx=0x7f77dc009bb0 comp rx=0 tx=0).stop 2024-04-24T08:37:40.364 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:39.616+0000 7f77f3800700 1 -- 172.21.15.83:0/1196522467 shutdown_connections 2024-04-24T08:37:40.364 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:39.616+0000 7f77f3800700 1 --2- 172.21.15.83:0/1196522467 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77ec10ef20 0x7f77ec10f340 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:37:40.364 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:39.616+0000 7f77f3800700 1 -- 172.21.15.83:0/1196522467 >> 172.21.15.83:0/1196522467 conn(0x7f77ec07dbb0 msgr2=0x7f77ec07ffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:37:40.364 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:39.616+0000 7f77f3800700 1 -- 172.21.15.83:0/1196522467 shutdown_connections 2024-04-24T08:37:40.364 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:39.616+0000 7f77f3800700 1 -- 172.21.15.83:0/1196522467 wait complete. 2024-04-24T08:37:40.365 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:39.617+0000 7f77f3800700 1 Processor -- start 2024-04-24T08:37:40.365 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:39.617+0000 7f77f3800700 1 -- start start 2024-04-24T08:37:40.365 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:39.617+0000 7f77f3800700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77ec10ef20 0x7f77ec1a2850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:37:40.365 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:39.617+0000 7f77f3800700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f77ec110a80 con 0x7f77ec10ef20 2024-04-24T08:37:40.365 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:39.618+0000 7f77f0e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77ec10ef20 0x7f77ec1a2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:37:40.365 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:39.618+0000 7f77f0e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77ec10ef20 0x7f77ec1a2850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33474/0 (socket says 172.21.15.83:33474) 2024-04-24T08:37:40.365 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:39.618+0000 7f77f0e00700 1 -- 172.21.15.83:0/3607601521 learned_addr learned my addr 172.21.15.83:0/3607601521 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:37:40.365 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:39.618+0000 7f77f0e00700 1 -- 172.21.15.83:0/3607601521 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77ec1a2d90 con 0x7f77ec10ef20 2024-04-24T08:37:40.365 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:39.618+0000 7f77f0e00700 1 --2- 172.21.15.83:0/3607601521 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77ec10ef20 0x7f77ec1a2850 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f77dc000c00 tx=0x7f77dc004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:37:40.365 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:39.619+0000 7f77ea000700 1 -- 172.21.15.83:0/3607601521 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f77dc0039c0 con 0x7f77ec10ef20 2024-04-24T08:37:40.365 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:39.619+0000 7f77ea000700 1 -- 172.21.15.83:0/3607601521 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f77dc003b60 con 0x7f77ec10ef20 2024-04-24T08:37:40.365 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:39.619+0000 7f77ea000700 1 -- 172.21.15.83:0/3607601521 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f77dc01a6f0 con 0x7f77ec10ef20 2024-04-24T08:37:40.365 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:39.619+0000 7f77f3800700 1 -- 172.21.15.83:0/3607601521 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77ec1a3020 con 0x7f77ec10ef20 2024-04-24T08:37:40.366 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:39.619+0000 7f77f3800700 1 -- 172.21.15.83:0/3607601521 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77ec1a5d10 con 0x7f77ec10ef20 2024-04-24T08:37:40.366 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:39.619+0000 7f77ea000700 1 -- 172.21.15.83:0/3607601521 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f77dc01ac20 con 0x7f77ec10ef20 2024-04-24T08:37:40.366 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:39.619+0000 7f77ea000700 1 -- 172.21.15.83:0/3607601521 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f77dc019700 con 0x7f77ec10ef20 2024-04-24T08:37:40.366 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:39.620+0000 7f77f3800700 1 -- 172.21.15.83:0/3607601521 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77ec064150 con 0x7f77ec10ef20 2024-04-24T08:37:40.366 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:39.623+0000 7f77ea000700 1 -- 172.21.15.83:0/3607601521 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f77dc004300 con 0x7f77ec10ef20 2024-04-24T08:37:40.366 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:39.699+0000 7f77f3800700 1 -- 172.21.15.83:0/3607601521 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f77ec1a5ee0 con 0x7f77ec10ef20 2024-04-24T08:37:40.366 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:39.702+0000 7f77ea000700 1 -- 172.21.15.83:0/3607601521 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f77dc032030 con 0x7f77ec10ef20 2024-04-24T08:37:40.366 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:39.704+0000 7f77f3800700 1 -- 172.21.15.83:0/3607601521 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77ec10ef20 msgr2=0x7f77ec1a2850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:37:40.366 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:39.704+0000 7f77f3800700 1 --2- 172.21.15.83:0/3607601521 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77ec10ef20 0x7f77ec1a2850 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f77dc000c00 tx=0x7f77dc004290 comp rx=0 tx=0).stop 2024-04-24T08:37:40.366 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:39.704+0000 7f77f3800700 1 -- 172.21.15.83:0/3607601521 shutdown_connections 2024-04-24T08:37:40.366 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:39.704+0000 7f77f3800700 1 --2- 172.21.15.83:0/3607601521 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77ec10ef20 0x7f77ec1a2850 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:37:40.366 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:39.704+0000 7f77f3800700 1 -- 172.21.15.83:0/3607601521 >> 172.21.15.83:0/3607601521 conn(0x7f77ec07dbb0 msgr2=0x7f77ec07f6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:37:40.366 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:39.704+0000 7f77f3800700 1 -- 172.21.15.83:0/3607601521 shutdown_connections 2024-04-24T08:37:40.366 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:39.704+0000 7f77f3800700 1 -- 172.21.15.83:0/3607601521 wait complete. 2024-04-24T08:37:40.367 INFO:teuthology.orchestra.run.smithi083.stdout:Wrote config to /etc/ceph/ceph.conf 2024-04-24T08:37:40.391 INFO:teuthology.orchestra.run.smithi083.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-04-24T08:37:40.391 INFO:teuthology.orchestra.run.smithi083.stdout:Creating mgr... 2024-04-24T08:37:40.393 INFO:teuthology.orchestra.run.smithi083.stdout:Verifying port 0.0.0.0:9283 ... 2024-04-24T08:37:40.394 INFO:teuthology.orchestra.run.smithi083.stdout:Verifying port 0.0.0.0:8765 ... 2024-04-24T08:37:40.851 INFO:teuthology.orchestra.run.smithi083.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-bd3e039e-0215-11ef-bc93-c7b262605968@mgr.x 2024-04-24T08:37:40.879 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stderr Failed to reset failed state of unit ceph-bd3e039e-0215-11ef-bc93-c7b262605968@mgr.x.service: Unit ceph-bd3e039e-0215-11ef-bc93-c7b262605968@mgr.x.service not loaded. 2024-04-24T08:37:41.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:40 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/3607601521' entity='client.admin' 2024-04-24T08:37:41.121 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-bd3e039e-0215-11ef-bc93-c7b262605968.target.wants/ceph-bd3e039e-0215-11ef-bc93-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-bd3e039e-0215-11ef-bc93-c7b262605968@.service. 2024-04-24T08:37:41.526 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:37:41 smithi083 systemd[1]: Starting Ceph mgr.x for bd3e039e-0215-11ef-bc93-c7b262605968... 2024-04-24T08:37:41.790 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:37:41 smithi083 podman[28123]: 2024-04-24 08:37:41.43087622 +0000 UTC m=+0.032320177 image pull b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 2024-04-24T08:37:41.790 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:37:41 smithi083 podman[28123]: 2024-04-24 08:37:41.535709537 +0000 UTC m=+0.137153498 container create c4a5d09d7dac09d1f202ebaf3029d3f218e0b0b53d61de96dd3dd22217a8a1cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=rishabh-main-apr17-a654945-1f81fb3, ceph=True) 2024-04-24T08:37:41.791 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:37:41 smithi083 podman[28123]: 2024-04-24 08:37:41.789752319 +0000 UTC m=+0.391196302 container init c4a5d09d7dac09d1f202ebaf3029d3f218e0b0b53d61de96dd3dd22217a8a1cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, RELEASE=rishabh-main-apr17-a654945-1f81fb3, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-04-24T08:37:41.898 INFO:teuthology.orchestra.run.smithi083.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-24T08:37:41.898 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stdout disabled 2024-04-24T08:37:41.915 INFO:teuthology.orchestra.run.smithi083.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-24T08:37:41.915 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stdout inactive 2024-04-24T08:37:41.915 INFO:teuthology.orchestra.run.smithi083.stdout:firewalld.service is not enabled 2024-04-24T08:37:41.915 INFO:teuthology.orchestra.run.smithi083.stdout:Not possible to enable service . firewalld.service is not available 2024-04-24T08:37:41.937 INFO:teuthology.orchestra.run.smithi083.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-24T08:37:41.938 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stdout disabled 2024-04-24T08:37:41.955 INFO:teuthology.orchestra.run.smithi083.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-24T08:37:41.955 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stdout inactive 2024-04-24T08:37:41.955 INFO:teuthology.orchestra.run.smithi083.stdout:firewalld.service is not enabled 2024-04-24T08:37:41.955 INFO:teuthology.orchestra.run.smithi083.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-04-24T08:37:41.955 INFO:teuthology.orchestra.run.smithi083.stdout:Waiting for mgr to start... 2024-04-24T08:37:41.955 INFO:teuthology.orchestra.run.smithi083.stdout:Waiting for mgr... 2024-04-24T08:37:42.054 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:37:41 smithi083 podman[28123]: 2024-04-24 08:37:41.800429106 +0000 UTC m=+0.401873052 container start c4a5d09d7dac09d1f202ebaf3029d3f218e0b0b53d61de96dd3dd22217a8a1cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, RELEASE=rishabh-main-apr17-a654945-1f81fb3, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-24T08:37:42.054 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:37:41 smithi083 bash[28123]: c4a5d09d7dac09d1f202ebaf3029d3f218e0b0b53d61de96dd3dd22217a8a1cf 2024-04-24T08:37:42.054 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:37:41 smithi083 systemd[1]: Started Ceph mgr.x for bd3e039e-0215-11ef-bc93-c7b262605968. 2024-04-24T08:37:42.324 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:37:42 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:37:42.060+0000 7f19de0fb200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-24T08:37:42.324 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:37:42 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:37:42.309+0000 7f19de0fb200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-24T08:37:42.697 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:37:42 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:37:42.439+0000 7f19de0fb200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-24T08:37:42.972 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:42 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/229638216' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-24T08:37:42.972 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:37:42 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:37:42.852+0000 7f19de0fb200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-24T08:37:43.303 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:37:42 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:37:42.971+0000 7f19de0fb200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-24T08:37:43.489 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 2024-04-24T08:37:43.489 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout { 2024-04-24T08:37:43.489 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "fsid": "bd3e039e-0215-11ef-bc93-c7b262605968", 2024-04-24T08:37:43.489 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "health": { 2024-04-24T08:37:43.489 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-24T08:37:43.489 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-24T08:37:43.489 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-24T08:37:43.489 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-04-24T08:37:43.489 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-24T08:37:43.490 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-24T08:37:43.490 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 0 2024-04-24T08:37:43.490 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout ], 2024-04-24T08:37:43.490 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-24T08:37:43.490 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "a" 2024-04-24T08:37:43.490 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout ], 2024-04-24T08:37:43.490 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-04-24T08:37:43.490 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-24T08:37:43.490 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T08:37:43.490 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-24T08:37:43.490 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-24T08:37:43.490 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-04-24T08:37:43.490 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-24T08:37:43.490 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T08:37:43.491 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-24T08:37:43.491 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-24T08:37:43.493 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-24T08:37:43.494 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-24T08:37:43.494 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-24T08:37:43.494 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-24T08:37:43.494 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-04-24T08:37:43.494 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-24T08:37:43.494 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-24T08:37:43.494 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-24T08:37:43.494 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-24T08:37:43.494 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-24T08:37:43.494 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-24T08:37:43.494 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-24T08:37:43.494 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-24T08:37:43.494 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-24T08:37:43.494 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-04-24T08:37:43.495 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-24T08:37:43.495 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T08:37:43.495 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-24T08:37:43.495 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-24T08:37:43.495 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-04-24T08:37:43.495 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-24T08:37:43.495 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-24T08:37:43.495 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-24T08:37:43.495 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-24T08:37:43.495 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-24T08:37:43.495 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-24T08:37:43.495 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "restful" 2024-04-24T08:37:43.495 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout ], 2024-04-24T08:37:43.495 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-24T08:37:43.498 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-04-24T08:37:43.498 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-24T08:37:43.498 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T08:37:43.498 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "modified": "2024-04-24T08:37:32.210505+0000", 2024-04-24T08:37:43.498 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-24T08:37:43.498 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-04-24T08:37:43.498 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-24T08:37:43.499 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout } 2024-04-24T08:37:43.499 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:42.608+0000 7f5c42e00700 1 Processor -- start 2024-04-24T08:37:43.499 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:42.609+0000 7f5c42e00700 1 -- start start 2024-04-24T08:37:43.499 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:42.609+0000 7f5c42e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5c3c07de20 0x7f5c3c07c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:37:43.499 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:42.609+0000 7f5c42e00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5c3c07c810 con 0x7f5c3c07de20 2024-04-24T08:37:43.499 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:42.609+0000 7f5c3be00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5c3c07de20 0x7f5c3c07c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:37:43.499 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:42.609+0000 7f5c3be00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5c3c07de20 0x7f5c3c07c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33510/0 (socket says 172.21.15.83:33510) 2024-04-24T08:37:43.499 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:42.609+0000 7f5c3be00700 1 -- 172.21.15.83:0/1742796136 learned_addr learned my addr 172.21.15.83:0/1742796136 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:37:43.499 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:42.610+0000 7f5c3be00700 1 -- 172.21.15.83:0/1742796136 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c3c07c990 con 0x7f5c3c07de20 2024-04-24T08:37:43.499 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:42.610+0000 7f5c3be00700 1 --2- 172.21.15.83:0/1742796136 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5c3c07de20 0x7f5c3c07c2d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c0098a0 tx=0x7f5c2c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ee59888cb6d8a21 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:37:43.499 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:42.610+0000 7f5c3aa00700 1 -- 172.21.15.83:0/1742796136 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5c2c004160 con 0x7f5c3c07de20 2024-04-24T08:37:43.499 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:42.610+0000 7f5c3aa00700 1 -- 172.21.15.83:0/1742796136 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f5c2c004300 con 0x7f5c3c07de20 2024-04-24T08:37:43.499 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:42.610+0000 7f5c3aa00700 1 -- 172.21.15.83:0/1742796136 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5c2c011a80 con 0x7f5c3c07de20 2024-04-24T08:37:43.499 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:42.611+0000 7f5c42e00700 1 -- 172.21.15.83:0/1742796136 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5c3c07de20 msgr2=0x7f5c3c07c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:37:43.500 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:42.611+0000 7f5c42e00700 1 --2- 172.21.15.83:0/1742796136 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5c3c07de20 0x7f5c3c07c2d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c0098a0 tx=0x7f5c2c009bb0 comp rx=0 tx=0).stop 2024-04-24T08:37:43.500 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:42.611+0000 7f5c42e00700 1 -- 172.21.15.83:0/1742796136 shutdown_connections 2024-04-24T08:37:43.500 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:42.611+0000 7f5c42e00700 1 --2- 172.21.15.83:0/1742796136 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5c3c07de20 0x7f5c3c07c2d0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:37:43.500 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:42.611+0000 7f5c42e00700 1 -- 172.21.15.83:0/1742796136 >> 172.21.15.83:0/1742796136 conn(0x7f5c3c103920 msgr2=0x7f5c3c105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:37:43.500 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:42.611+0000 7f5c42e00700 1 -- 172.21.15.83:0/1742796136 shutdown_connections 2024-04-24T08:37:43.500 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:42.611+0000 7f5c42e00700 1 -- 172.21.15.83:0/1742796136 wait complete. 2024-04-24T08:37:43.500 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:42.612+0000 7f5c42e00700 1 Processor -- start 2024-04-24T08:37:43.500 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:42.613+0000 7f5c42e00700 1 -- start start 2024-04-24T08:37:43.500 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:42.613+0000 7f5c42e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5c3c07de20 0x7f5c3c19e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:37:43.500 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:42.613+0000 7f5c42e00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5c3c10e8d0 con 0x7f5c3c07de20 2024-04-24T08:37:43.500 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:42.613+0000 7f5c3be00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5c3c07de20 0x7f5c3c19e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:37:43.500 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:42.613+0000 7f5c3be00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5c3c07de20 0x7f5c3c19e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33518/0 (socket says 172.21.15.83:33518) 2024-04-24T08:37:43.500 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:42.613+0000 7f5c3be00700 1 -- 172.21.15.83:0/229638216 learned_addr learned my addr 172.21.15.83:0/229638216 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:37:43.501 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:42.614+0000 7f5c3be00700 1 -- 172.21.15.83:0/229638216 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c3c19e9c0 con 0x7f5c3c07de20 2024-04-24T08:37:43.501 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:42.614+0000 7f5c3be00700 1 --2- 172.21.15.83:0/229638216 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5c3c07de20 0x7f5c3c19e480 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c011a50 tx=0x7f5c2c004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:37:43.501 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:42.614+0000 7f5c38c00700 1 -- 172.21.15.83:0/229638216 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5c2c0039f0 con 0x7f5c3c07de20 2024-04-24T08:37:43.501 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:42.614+0000 7f5c38c00700 1 -- 172.21.15.83:0/229638216 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f5c2c003b90 con 0x7f5c3c07de20 2024-04-24T08:37:43.501 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:42.614+0000 7f5c42e00700 1 -- 172.21.15.83:0/229638216 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c3c19ec50 con 0x7f5c3c07de20 2024-04-24T08:37:43.501 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:42.614+0000 7f5c38c00700 1 -- 172.21.15.83:0/229638216 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5c2c01a600 con 0x7f5c3c07de20 2024-04-24T08:37:43.501 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:42.614+0000 7f5c42e00700 1 -- 172.21.15.83:0/229638216 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c3c1a1940 con 0x7f5c3c07de20 2024-04-24T08:37:43.501 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:42.615+0000 7f5c38c00700 1 -- 172.21.15.83:0/229638216 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5c2c01acb0 con 0x7f5c3c07de20 2024-04-24T08:37:43.501 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:42.615+0000 7f5c38c00700 1 -- 172.21.15.83:0/229638216 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f5c2c019a90 con 0x7f5c3c07de20 2024-04-24T08:37:43.501 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:42.615+0000 7f5c42e00700 1 -- 172.21.15.83:0/229638216 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c3c064150 con 0x7f5c3c07de20 2024-04-24T08:37:43.501 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:42.618+0000 7f5c38c00700 1 -- 172.21.15.83:0/229638216 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f5c2c024a40 con 0x7f5c3c07de20 2024-04-24T08:37:43.501 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:42.696+0000 7f5c42e00700 1 -- 172.21.15.83:0/229638216 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f5c3c1a1e90 con 0x7f5c3c07de20 2024-04-24T08:37:43.501 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:42.697+0000 7f5c38c00700 1 -- 172.21.15.83:0/229638216 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f5c2c019c90 con 0x7f5c3c07de20 2024-04-24T08:37:43.501 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:42.698+0000 7f5c42e00700 1 -- 172.21.15.83:0/229638216 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5c3c07de20 msgr2=0x7f5c3c19e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:37:43.502 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:42.698+0000 7f5c42e00700 1 --2- 172.21.15.83:0/229638216 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5c3c07de20 0x7f5c3c19e480 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c011a50 tx=0x7f5c2c004290 comp rx=0 tx=0).stop 2024-04-24T08:37:43.502 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:42.699+0000 7f5c42e00700 1 -- 172.21.15.83:0/229638216 shutdown_connections 2024-04-24T08:37:43.502 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:42.699+0000 7f5c42e00700 1 --2- 172.21.15.83:0/229638216 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5c3c07de20 0x7f5c3c19e480 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:37:43.502 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:42.699+0000 7f5c42e00700 1 -- 172.21.15.83:0/229638216 >> 172.21.15.83:0/229638216 conn(0x7f5c3c103920 msgr2=0x7f5c3c104410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:37:43.502 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:42.699+0000 7f5c42e00700 1 -- 172.21.15.83:0/229638216 shutdown_connections 2024-04-24T08:37:43.502 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:42.699+0000 7f5c42e00700 1 -- 172.21.15.83:0/229638216 wait complete. 2024-04-24T08:37:43.502 INFO:teuthology.orchestra.run.smithi083.stdout:mgr not available, waiting (1/15)... 2024-04-24T08:37:45.009 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:37:44 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:37:44.567+0000 7f19de0fb200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-24T08:37:45.009 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:37:44 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:37:44.791+0000 7f19de0fb200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-24T08:37:45.009 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:37:45 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:37:45.008+0000 7f19de0fb200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-24T08:37:45.645 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:37:45 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:37:45.388+0000 7f19de0fb200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-24T08:37:45.646 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:37:45 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:37:45.512+0000 7f19de0fb200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-24T08:37:45.965 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:37:45 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:37:45.770+0000 7f19de0fb200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-24T08:37:46.226 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:37:46 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:37:46.022+0000 7f19de0fb200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-24T08:37:46.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:46 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/2405462249' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-24T08:37:46.904 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 2024-04-24T08:37:46.904 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout { 2024-04-24T08:37:46.904 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "fsid": "bd3e039e-0215-11ef-bc93-c7b262605968", 2024-04-24T08:37:46.904 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "health": { 2024-04-24T08:37:46.904 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-24T08:37:46.904 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-24T08:37:46.904 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-24T08:37:46.904 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-04-24T08:37:46.904 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-24T08:37:46.904 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-24T08:37:46.904 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 0 2024-04-24T08:37:46.904 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout ], 2024-04-24T08:37:46.905 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-24T08:37:46.905 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "a" 2024-04-24T08:37:46.905 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout ], 2024-04-24T08:37:46.905 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "quorum_age": 7, 2024-04-24T08:37:46.905 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-24T08:37:46.905 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T08:37:46.905 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-24T08:37:46.905 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-24T08:37:46.905 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-04-24T08:37:46.905 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-24T08:37:46.905 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T08:37:46.905 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-24T08:37:46.905 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-24T08:37:46.905 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-24T08:37:46.906 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-24T08:37:46.906 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-24T08:37:46.906 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-24T08:37:46.906 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-04-24T08:37:46.906 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-24T08:37:46.906 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-24T08:37:46.906 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-24T08:37:46.906 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-24T08:37:46.906 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-24T08:37:46.906 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-24T08:37:46.906 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-24T08:37:46.906 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-24T08:37:46.906 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-24T08:37:46.906 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-04-24T08:37:46.907 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-24T08:37:46.907 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T08:37:46.907 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-24T08:37:46.907 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-24T08:37:46.907 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-04-24T08:37:46.907 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-24T08:37:46.907 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-24T08:37:46.907 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-24T08:37:46.907 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-24T08:37:46.907 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-24T08:37:46.907 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-24T08:37:46.907 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "restful" 2024-04-24T08:37:46.907 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout ], 2024-04-24T08:37:46.907 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-24T08:37:46.907 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-04-24T08:37:46.908 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-24T08:37:46.908 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T08:37:46.908 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "modified": "2024-04-24T08:37:32.210505+0000", 2024-04-24T08:37:46.908 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-24T08:37:46.908 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-04-24T08:37:46.908 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-24T08:37:46.908 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout } 2024-04-24T08:37:46.908 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:46.119+0000 7f354b400700 1 Processor -- start 2024-04-24T08:37:46.908 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:46.119+0000 7f354b400700 1 -- start start 2024-04-24T08:37:46.908 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:46.120+0000 7f354b400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f354407de20 0x7f354407c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:37:46.908 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:46.120+0000 7f354b400700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f354407c810 con 0x7f354407de20 2024-04-24T08:37:46.908 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:46.120+0000 7f3548a00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f354407de20 0x7f354407c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:37:46.908 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:46.120+0000 7f3548a00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f354407de20 0x7f354407c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33530/0 (socket says 172.21.15.83:33530) 2024-04-24T08:37:46.908 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:46.120+0000 7f3548a00700 1 -- 172.21.15.83:0/636347781 learned_addr learned my addr 172.21.15.83:0/636347781 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:37:46.909 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:46.121+0000 7f3548a00700 1 -- 172.21.15.83:0/636347781 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f354407c990 con 0x7f354407de20 2024-04-24T08:37:46.909 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:46.121+0000 7f3548a00700 1 --2- 172.21.15.83:0/636347781 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f354407de20 0x7f354407c2d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3538009e80 tx=0x7f353800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ef153ce0fa556ff server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:37:46.909 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:46.121+0000 7f3543400700 1 -- 172.21.15.83:0/636347781 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3538017070 con 0x7f354407de20 2024-04-24T08:37:46.909 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:46.121+0000 7f3543400700 1 -- 172.21.15.83:0/636347781 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f353800b8c0 con 0x7f354407de20 2024-04-24T08:37:46.909 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:46.121+0000 7f3543400700 1 -- 172.21.15.83:0/636347781 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3538004520 con 0x7f354407de20 2024-04-24T08:37:46.909 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:46.122+0000 7f354b400700 1 -- 172.21.15.83:0/636347781 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f354407de20 msgr2=0x7f354407c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:37:46.909 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:46.122+0000 7f354b400700 1 --2- 172.21.15.83:0/636347781 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f354407de20 0x7f354407c2d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3538009e80 tx=0x7f353800b330 comp rx=0 tx=0).stop 2024-04-24T08:37:46.909 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:46.122+0000 7f354b400700 1 -- 172.21.15.83:0/636347781 shutdown_connections 2024-04-24T08:37:46.909 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:46.122+0000 7f354b400700 1 --2- 172.21.15.83:0/636347781 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f354407de20 0x7f354407c2d0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:37:46.909 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:46.122+0000 7f354b400700 1 -- 172.21.15.83:0/636347781 >> 172.21.15.83:0/636347781 conn(0x7f3544103920 msgr2=0x7f3544105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:37:46.909 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:46.122+0000 7f354b400700 1 -- 172.21.15.83:0/636347781 shutdown_connections 2024-04-24T08:37:46.909 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:46.122+0000 7f354b400700 1 -- 172.21.15.83:0/636347781 wait complete. 2024-04-24T08:37:46.909 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:46.123+0000 7f354b400700 1 Processor -- start 2024-04-24T08:37:46.909 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:46.123+0000 7f354b400700 1 -- start start 2024-04-24T08:37:46.910 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:46.124+0000 7f354b400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f354407de20 0x7f354419e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:37:46.910 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:46.124+0000 7f354b400700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f354410e8d0 con 0x7f354407de20 2024-04-24T08:37:46.910 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:46.124+0000 7f3548a00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f354407de20 0x7f354419e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:37:46.910 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:46.124+0000 7f3548a00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f354407de20 0x7f354419e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33544/0 (socket says 172.21.15.83:33544) 2024-04-24T08:37:46.910 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:46.124+0000 7f3548a00700 1 -- 172.21.15.83:0/2405462249 learned_addr learned my addr 172.21.15.83:0/2405462249 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:37:46.910 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:46.124+0000 7f3548a00700 1 -- 172.21.15.83:0/2405462249 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f354419e960 con 0x7f354407de20 2024-04-24T08:37:46.910 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:46.124+0000 7f3548a00700 1 --2- 172.21.15.83:0/2405462249 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f354407de20 0x7f354419e420 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f35380042b0 tx=0x7f3538004070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:37:46.910 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:46.125+0000 7f3541600700 1 -- 172.21.15.83:0/2405462249 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3538020020 con 0x7f354407de20 2024-04-24T08:37:46.910 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:46.125+0000 7f3541600700 1 -- 172.21.15.83:0/2405462249 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f3538012430 con 0x7f354407de20 2024-04-24T08:37:46.910 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:46.125+0000 7f3541600700 1 -- 172.21.15.83:0/2405462249 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3538017040 con 0x7f354407de20 2024-04-24T08:37:46.910 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:46.125+0000 7f354b400700 1 -- 172.21.15.83:0/2405462249 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f354419ebf0 con 0x7f354407de20 2024-04-24T08:37:46.910 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:46.125+0000 7f354b400700 1 -- 172.21.15.83:0/2405462249 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35441a18e0 con 0x7f354407de20 2024-04-24T08:37:46.910 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:46.125+0000 7f3541600700 1 -- 172.21.15.83:0/2405462249 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3538023020 con 0x7f354407de20 2024-04-24T08:37:46.911 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:46.125+0000 7f3541600700 1 -- 172.21.15.83:0/2405462249 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f353800f050 con 0x7f354407de20 2024-04-24T08:37:46.911 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:46.126+0000 7f354b400700 1 -- 172.21.15.83:0/2405462249 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3544064150 con 0x7f354407de20 2024-04-24T08:37:46.911 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:46.129+0000 7f3541600700 1 -- 172.21.15.83:0/2405462249 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f353802a020 con 0x7f354407de20 2024-04-24T08:37:46.911 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:46.208+0000 7f354b400700 1 -- 172.21.15.83:0/2405462249 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f35441a1ab0 con 0x7f354407de20 2024-04-24T08:37:46.911 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:46.209+0000 7f3541600700 1 -- 172.21.15.83:0/2405462249 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f353800b860 con 0x7f354407de20 2024-04-24T08:37:46.911 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:46.211+0000 7f354b400700 1 -- 172.21.15.83:0/2405462249 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f354407de20 msgr2=0x7f354419e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:37:46.911 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:46.211+0000 7f354b400700 1 --2- 172.21.15.83:0/2405462249 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f354407de20 0x7f354419e420 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f35380042b0 tx=0x7f3538004070 comp rx=0 tx=0).stop 2024-04-24T08:37:46.911 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:46.211+0000 7f354b400700 1 -- 172.21.15.83:0/2405462249 shutdown_connections 2024-04-24T08:37:46.911 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:46.211+0000 7f354b400700 1 --2- 172.21.15.83:0/2405462249 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f354407de20 0x7f354419e420 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:37:46.911 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:46.211+0000 7f354b400700 1 -- 172.21.15.83:0/2405462249 >> 172.21.15.83:0/2405462249 conn(0x7f3544103920 msgr2=0x7f3544104430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:37:46.911 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:46.211+0000 7f354b400700 1 -- 172.21.15.83:0/2405462249 shutdown_connections 2024-04-24T08:37:46.911 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:46.211+0000 7f354b400700 1 -- 172.21.15.83:0/2405462249 wait complete. 2024-04-24T08:37:46.911 INFO:teuthology.orchestra.run.smithi083.stdout:mgr not available, waiting (2/15)... 2024-04-24T08:37:47.303 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:37:46 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:37:46.838+0000 7f19de0fb200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-24T08:37:47.777 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:37:47 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:37:47.389+0000 7f19de0fb200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-24T08:37:47.777 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:37:47 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:37:47.514+0000 7f19de0fb200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-24T08:37:47.777 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:37:47 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:37:47.776+0000 7f19de0fb200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-24T08:37:48.770 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:37:48 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:37:48.438+0000 7f19de0fb200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-24T08:37:48.770 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:37:48 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:37:48.560+0000 7f19de0fb200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-24T08:37:49.053 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:37:48 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:37:48.884+0000 7f19de0fb200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-24T08:37:49.348 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:37:49 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:37:49.066+0000 7f19de0fb200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-24T08:37:49.348 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:37:49 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:37:49.192+0000 7f19de0fb200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-24T08:37:49.348 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:37:49 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:37:49.347+0000 7f19de0fb200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-24T08:37:50.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:49 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/1566445822' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-24T08:37:50.054 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:37:49 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:37:49.730+0000 7f19de0fb200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-24T08:37:50.405 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 2024-04-24T08:37:50.405 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout { 2024-04-24T08:37:50.405 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "fsid": "bd3e039e-0215-11ef-bc93-c7b262605968", 2024-04-24T08:37:50.405 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "health": { 2024-04-24T08:37:50.405 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-24T08:37:50.405 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-24T08:37:50.406 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-24T08:37:50.412 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-04-24T08:37:50.412 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-24T08:37:50.412 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-24T08:37:50.412 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 0 2024-04-24T08:37:50.412 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout ], 2024-04-24T08:37:50.412 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-24T08:37:50.412 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "a" 2024-04-24T08:37:50.412 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout ], 2024-04-24T08:37:50.412 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "quorum_age": 10, 2024-04-24T08:37:50.413 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-24T08:37:50.413 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T08:37:50.413 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-24T08:37:50.413 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-24T08:37:50.413 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-04-24T08:37:50.413 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-24T08:37:50.413 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T08:37:50.413 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-24T08:37:50.413 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-24T08:37:50.413 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-24T08:37:50.413 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-24T08:37:50.413 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-24T08:37:50.413 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-24T08:37:50.413 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-04-24T08:37:50.414 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-24T08:37:50.414 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-24T08:37:50.414 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-24T08:37:50.414 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-24T08:37:50.414 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-24T08:37:50.414 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-24T08:37:50.414 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-24T08:37:50.414 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-24T08:37:50.414 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-24T08:37:50.414 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-04-24T08:37:50.414 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-24T08:37:50.414 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T08:37:50.414 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-24T08:37:50.414 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-24T08:37:50.414 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-04-24T08:37:50.415 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-24T08:37:50.415 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-24T08:37:50.415 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-24T08:37:50.415 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-24T08:37:50.415 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-24T08:37:50.415 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-24T08:37:50.415 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "restful" 2024-04-24T08:37:50.415 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout ], 2024-04-24T08:37:50.415 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-24T08:37:50.415 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-04-24T08:37:50.415 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-24T08:37:50.415 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T08:37:50.415 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "modified": "2024-04-24T08:37:32.210505+0000", 2024-04-24T08:37:50.415 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-24T08:37:50.415 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-04-24T08:37:50.416 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-24T08:37:50.416 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout } 2024-04-24T08:37:50.416 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:49.549+0000 7f029fe00700 1 Processor -- start 2024-04-24T08:37:50.416 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:49.549+0000 7f029fe00700 1 -- start start 2024-04-24T08:37:50.416 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:49.550+0000 7f029fe00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f029807ddb0 0x7f029807c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:37:50.416 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:49.550+0000 7f029fe00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f029807c7f0 con 0x7f029807ddb0 2024-04-24T08:37:50.416 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:49.550+0000 7f029d400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f029807ddb0 0x7f029807c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:37:50.416 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:49.550+0000 7f029d400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f029807ddb0 0x7f029807c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49812/0 (socket says 172.21.15.83:49812) 2024-04-24T08:37:50.420 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:49.550+0000 7f029d400700 1 -- 172.21.15.83:0/1361189780 learned_addr learned my addr 172.21.15.83:0/1361189780 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:37:50.420 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:49.551+0000 7f029d400700 1 -- 172.21.15.83:0/1361189780 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f029807c970 con 0x7f029807ddb0 2024-04-24T08:37:50.420 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:49.551+0000 7f029d400700 1 --2- 172.21.15.83:0/1361189780 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f029807ddb0 0x7f029807c2b0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f028c0098a0 tx=0x7f028c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c5285b94a4aadf4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:37:50.420 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:49.551+0000 7f0297e00700 1 -- 172.21.15.83:0/1361189780 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f028c004160 con 0x7f029807ddb0 2024-04-24T08:37:50.420 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:49.551+0000 7f0297e00700 1 -- 172.21.15.83:0/1361189780 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f028c004300 con 0x7f029807ddb0 2024-04-24T08:37:50.420 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:49.551+0000 7f0297e00700 1 -- 172.21.15.83:0/1361189780 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f028c011a80 con 0x7f029807ddb0 2024-04-24T08:37:50.420 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:49.552+0000 7f029fe00700 1 -- 172.21.15.83:0/1361189780 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f029807ddb0 msgr2=0x7f029807c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:37:50.421 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:49.552+0000 7f029fe00700 1 --2- 172.21.15.83:0/1361189780 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f029807ddb0 0x7f029807c2b0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f028c0098a0 tx=0x7f028c009bb0 comp rx=0 tx=0).stop 2024-04-24T08:37:50.421 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:49.552+0000 7f029fe00700 1 -- 172.21.15.83:0/1361189780 shutdown_connections 2024-04-24T08:37:50.421 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:49.552+0000 7f029fe00700 1 --2- 172.21.15.83:0/1361189780 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f029807ddb0 0x7f029807c2b0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:37:50.421 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:49.552+0000 7f029fe00700 1 -- 172.21.15.83:0/1361189780 >> 172.21.15.83:0/1361189780 conn(0x7f02981039c0 msgr2=0x7f0298105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:37:50.421 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:49.552+0000 7f029fe00700 1 -- 172.21.15.83:0/1361189780 shutdown_connections 2024-04-24T08:37:50.421 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:49.552+0000 7f029fe00700 1 -- 172.21.15.83:0/1361189780 wait complete. 2024-04-24T08:37:50.421 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:49.554+0000 7f029fe00700 1 Processor -- start 2024-04-24T08:37:50.421 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:49.554+0000 7f029fe00700 1 -- start start 2024-04-24T08:37:50.421 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:49.554+0000 7f029fe00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f029807ddb0 0x7f029819e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:37:50.421 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:49.554+0000 7f029fe00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f029810e890 con 0x7f029807ddb0 2024-04-24T08:37:50.421 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:49.555+0000 7f029d400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f029807ddb0 0x7f029819e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:37:50.421 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:49.555+0000 7f029d400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f029807ddb0 0x7f029819e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49828/0 (socket says 172.21.15.83:49828) 2024-04-24T08:37:50.421 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:49.555+0000 7f029d400700 1 -- 172.21.15.83:0/1566445822 learned_addr learned my addr 172.21.15.83:0/1566445822 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:37:50.422 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:49.555+0000 7f029d400700 1 -- 172.21.15.83:0/1566445822 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f029819e960 con 0x7f029807ddb0 2024-04-24T08:37:50.422 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:49.555+0000 7f029d400700 1 --2- 172.21.15.83:0/1566445822 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f029807ddb0 0x7f029819e420 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f028c011a50 tx=0x7f028c004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:37:50.422 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:49.556+0000 7f0296000700 1 -- 172.21.15.83:0/1566445822 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f028c0039f0 con 0x7f029807ddb0 2024-04-24T08:37:50.422 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:49.556+0000 7f0296000700 1 -- 172.21.15.83:0/1566445822 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f028c003b90 con 0x7f029807ddb0 2024-04-24T08:37:50.422 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:49.556+0000 7f029fe00700 1 -- 172.21.15.83:0/1566445822 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f029819ebf0 con 0x7f029807ddb0 2024-04-24T08:37:50.422 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:49.556+0000 7f0296000700 1 -- 172.21.15.83:0/1566445822 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f028c01a600 con 0x7f029807ddb0 2024-04-24T08:37:50.422 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:49.556+0000 7f029fe00700 1 -- 172.21.15.83:0/1566445822 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02981a18e0 con 0x7f029807ddb0 2024-04-24T08:37:50.422 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:49.556+0000 7f0296000700 1 -- 172.21.15.83:0/1566445822 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f028c01ac60 con 0x7f029807ddb0 2024-04-24T08:37:50.422 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:49.556+0000 7f0296000700 1 -- 172.21.15.83:0/1566445822 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f028c019700 con 0x7f029807ddb0 2024-04-24T08:37:50.422 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:49.557+0000 7f029fe00700 1 -- 172.21.15.83:0/1566445822 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0298064150 con 0x7f029807ddb0 2024-04-24T08:37:50.422 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:49.560+0000 7f0296000700 1 -- 172.21.15.83:0/1566445822 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f028c024a40 con 0x7f029807ddb0 2024-04-24T08:37:50.422 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:49.641+0000 7f029fe00700 1 -- 172.21.15.83:0/1566445822 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f02981a1e30 con 0x7f029807ddb0 2024-04-24T08:37:50.422 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:49.641+0000 7f0296000700 1 -- 172.21.15.83:0/1566445822 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1240 (secure 0 0 0) 0x7f028c024440 con 0x7f029807ddb0 2024-04-24T08:37:50.422 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:49.643+0000 7f029fe00700 1 -- 172.21.15.83:0/1566445822 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f029807ddb0 msgr2=0x7f029819e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:37:50.423 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:49.643+0000 7f029fe00700 1 --2- 172.21.15.83:0/1566445822 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f029807ddb0 0x7f029819e420 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f028c011a50 tx=0x7f028c004290 comp rx=0 tx=0).stop 2024-04-24T08:37:50.423 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:49.643+0000 7f029fe00700 1 -- 172.21.15.83:0/1566445822 shutdown_connections 2024-04-24T08:37:50.423 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:49.643+0000 7f029fe00700 1 --2- 172.21.15.83:0/1566445822 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f029807ddb0 0x7f029819e420 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:37:50.423 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:49.643+0000 7f029fe00700 1 -- 172.21.15.83:0/1566445822 >> 172.21.15.83:0/1566445822 conn(0x7f02981039c0 msgr2=0x7f02981044f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:37:50.423 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:49.643+0000 7f029fe00700 1 -- 172.21.15.83:0/1566445822 shutdown_connections 2024-04-24T08:37:50.423 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:49.644+0000 7f029fe00700 1 -- 172.21.15.83:0/1566445822 wait complete. 2024-04-24T08:37:50.423 INFO:teuthology.orchestra.run.smithi083.stdout:mgr not available, waiting (3/15)... 2024-04-24T08:37:50.781 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:37:50 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:37:50.501+0000 7f19de0fb200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-24T08:37:51.055 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:37:50 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:37:50.781+0000 7f19de0fb200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-24T08:37:51.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:50 smithi083 ceph-mon[27907]: Activating manager daemon x 2024-04-24T08:37:51.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:50 smithi083 ceph-mon[27907]: mgrmap e2: x(active, starting, since 0.0111019s) 2024-04-24T08:37:51.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:50 smithi083 ceph-mon[27907]: from='mgr.14100 172.21.15.83:0/3659309998' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-24T08:37:51.056 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:50 smithi083 ceph-mon[27907]: from='mgr.14100 172.21.15.83:0/3659309998' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-24T08:37:51.056 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:50 smithi083 ceph-mon[27907]: from='mgr.14100 172.21.15.83:0/3659309998' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-24T08:37:51.056 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:50 smithi083 ceph-mon[27907]: from='mgr.14100 172.21.15.83:0/3659309998' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T08:37:51.056 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:50 smithi083 ceph-mon[27907]: from='mgr.14100 172.21.15.83:0/3659309998' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-24T08:37:51.056 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:50 smithi083 ceph-mon[27907]: Manager daemon x is now available 2024-04-24T08:37:51.056 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:50 smithi083 ceph-mon[27907]: from='mgr.14100 172.21.15.83:0/3659309998' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-24T08:37:51.056 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:50 smithi083 ceph-mon[27907]: from='mgr.14100 172.21.15.83:0/3659309998' entity='mgr.x' 2024-04-24T08:37:51.056 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:50 smithi083 ceph-mon[27907]: from='mgr.14100 172.21.15.83:0/3659309998' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-24T08:37:51.056 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:50 smithi083 ceph-mon[27907]: from='mgr.14100 172.21.15.83:0/3659309998' entity='mgr.x' 2024-04-24T08:37:51.056 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:50 smithi083 ceph-mon[27907]: from='mgr.14100 172.21.15.83:0/3659309998' entity='mgr.x' 2024-04-24T08:37:52.968 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:52 smithi083 ceph-mon[27907]: mgrmap e3: x(active, since 1.02148s) 2024-04-24T08:37:54.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:53 smithi083 ceph-mon[27907]: mgrmap e4: x(active, since 2s) 2024-04-24T08:37:54.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:53 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/1191666127' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-24T08:37:54.404 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 2024-04-24T08:37:54.407 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout { 2024-04-24T08:37:54.407 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "fsid": "bd3e039e-0215-11ef-bc93-c7b262605968", 2024-04-24T08:37:54.407 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "health": { 2024-04-24T08:37:54.407 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-24T08:37:54.407 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-24T08:37:54.407 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-24T08:37:54.407 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-04-24T08:37:54.407 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-24T08:37:54.407 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-24T08:37:54.407 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 0 2024-04-24T08:37:54.407 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout ], 2024-04-24T08:37:54.408 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-24T08:37:54.408 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "a" 2024-04-24T08:37:54.408 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout ], 2024-04-24T08:37:54.408 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "quorum_age": 14, 2024-04-24T08:37:54.408 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-24T08:37:54.408 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T08:37:54.408 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-24T08:37:54.408 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-24T08:37:54.408 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-04-24T08:37:54.408 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-24T08:37:54.408 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T08:37:54.408 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-24T08:37:54.408 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-24T08:37:54.408 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-24T08:37:54.418 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-24T08:37:54.418 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-24T08:37:54.418 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-24T08:37:54.418 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-04-24T08:37:54.418 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-24T08:37:54.418 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-24T08:37:54.419 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-24T08:37:54.419 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-24T08:37:54.419 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-24T08:37:54.419 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-24T08:37:54.419 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-24T08:37:54.419 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-24T08:37:54.419 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-24T08:37:54.419 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-04-24T08:37:54.420 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-24T08:37:54.420 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T08:37:54.420 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-24T08:37:54.420 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-24T08:37:54.420 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-04-24T08:37:54.420 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-24T08:37:54.420 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-24T08:37:54.420 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-24T08:37:54.420 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-24T08:37:54.420 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-24T08:37:54.420 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-24T08:37:54.421 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "restful" 2024-04-24T08:37:54.421 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout ], 2024-04-24T08:37:54.421 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-24T08:37:54.421 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-04-24T08:37:54.421 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-24T08:37:54.421 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T08:37:54.421 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "modified": "2024-04-24T08:37:32.210505+0000", 2024-04-24T08:37:54.421 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-24T08:37:54.421 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-04-24T08:37:54.421 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-24T08:37:54.421 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout } 2024-04-24T08:37:54.422 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:53.129+0000 7f107fe00700 1 Processor -- start 2024-04-24T08:37:54.422 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:53.129+0000 7f107fe00700 1 -- start start 2024-04-24T08:37:54.422 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:53.129+0000 7f107fe00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f108010a370 0x7f108010a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:37:54.422 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:53.130+0000 7f107fe00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f108010ad60 con 0x7f108010a370 2024-04-24T08:37:54.422 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:53.130+0000 7f107ea00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f108010a370 0x7f108010a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:37:54.422 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:53.130+0000 7f107ea00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f108010a370 0x7f108010a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49910/0 (socket says 172.21.15.83:49910) 2024-04-24T08:37:54.422 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:53.130+0000 7f107ea00700 1 -- 172.21.15.83:0/2106515995 learned_addr learned my addr 172.21.15.83:0/2106515995 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:37:54.422 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:53.131+0000 7f107ea00700 1 -- 172.21.15.83:0/2106515995 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f108010b520 con 0x7f108010a370 2024-04-24T08:37:54.423 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:53.131+0000 7f107ea00700 1 --2- 172.21.15.83:0/2106515995 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f108010a370 0x7f108010a790 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f10700098a0 tx=0x7f1070009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2db4fc26d3960f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:37:54.423 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:53.131+0000 7f107e000700 1 -- 172.21.15.83:0/2106515995 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1070004160 con 0x7f108010a370 2024-04-24T08:37:54.423 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:53.131+0000 7f107e000700 1 -- 172.21.15.83:0/2106515995 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f1070004300 con 0x7f108010a370 2024-04-24T08:37:54.423 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:53.131+0000 7f107e000700 1 -- 172.21.15.83:0/2106515995 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1070011a80 con 0x7f108010a370 2024-04-24T08:37:54.423 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:53.132+0000 7f107fe00700 1 -- 172.21.15.83:0/2106515995 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f108010a370 msgr2=0x7f108010a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:37:54.423 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:53.132+0000 7f107fe00700 1 --2- 172.21.15.83:0/2106515995 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f108010a370 0x7f108010a790 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f10700098a0 tx=0x7f1070009bb0 comp rx=0 tx=0).stop 2024-04-24T08:37:54.423 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:53.132+0000 7f107fe00700 1 -- 172.21.15.83:0/2106515995 shutdown_connections 2024-04-24T08:37:54.423 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:53.132+0000 7f107fe00700 1 --2- 172.21.15.83:0/2106515995 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f108010a370 0x7f108010a790 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:37:54.423 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:53.132+0000 7f107fe00700 1 -- 172.21.15.83:0/2106515995 >> 172.21.15.83:0/2106515995 conn(0x7f108007d8d0 msgr2=0x7f10801084c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:37:54.423 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:53.132+0000 7f107fe00700 1 -- 172.21.15.83:0/2106515995 shutdown_connections 2024-04-24T08:37:54.423 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:53.132+0000 7f107fe00700 1 -- 172.21.15.83:0/2106515995 wait complete. 2024-04-24T08:37:54.424 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:53.133+0000 7f107fe00700 1 Processor -- start 2024-04-24T08:37:54.424 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:53.133+0000 7f107fe00700 1 -- start start 2024-04-24T08:37:54.424 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:53.134+0000 7f107fe00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f108010a370 0x7f108019e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:37:54.424 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:53.134+0000 7f107fe00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f108010e8d0 con 0x7f108010a370 2024-04-24T08:37:54.424 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:53.134+0000 7f107ea00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f108010a370 0x7f108019e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:37:54.424 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:53.134+0000 7f107ea00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f108010a370 0x7f108019e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49920/0 (socket says 172.21.15.83:49920) 2024-04-24T08:37:54.424 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:53.134+0000 7f107ea00700 1 -- 172.21.15.83:0/1191666127 learned_addr learned my addr 172.21.15.83:0/1191666127 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:37:54.424 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:53.134+0000 7f107ea00700 1 -- 172.21.15.83:0/1191666127 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f108019e790 con 0x7f108010a370 2024-04-24T08:37:54.424 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:53.135+0000 7f107ea00700 1 --2- 172.21.15.83:0/1191666127 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f108010a370 0x7f108019e250 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f1070000c00 tx=0x7f1070011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:37:54.424 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:53.135+0000 7f1077400700 1 -- 172.21.15.83:0/1191666127 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f10700039f0 con 0x7f108010a370 2024-04-24T08:37:54.425 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:53.135+0000 7f1077400700 1 -- 172.21.15.83:0/1191666127 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f1070003b90 con 0x7f108010a370 2024-04-24T08:37:54.425 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:53.135+0000 7f107fe00700 1 -- 172.21.15.83:0/1191666127 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f108019ea20 con 0x7f108010a370 2024-04-24T08:37:54.425 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:53.135+0000 7f1077400700 1 -- 172.21.15.83:0/1191666127 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f107001a740 con 0x7f108010a370 2024-04-24T08:37:54.425 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:53.135+0000 7f107fe00700 1 -- 172.21.15.83:0/1191666127 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10801a1710 con 0x7f108010a370 2024-04-24T08:37:54.425 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:53.136+0000 7f1077400700 1 -- 172.21.15.83:0/1191666127 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f1070024440 con 0x7f108010a370 2024-04-24T08:37:54.425 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:53.136+0000 7f107fe00700 1 -- 172.21.15.83:0/1191666127 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1080064150 con 0x7f108010a370 2024-04-24T08:37:54.425 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:53.137+0000 7f1077400700 1 --2- 172.21.15.83:0/1191666127 >> [v2:172.21.15.83:6800/1996371091,v1:172.21.15.83:6801/1996371091] conn(0x7f10640395b0 0x7f106403ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:37:54.425 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:53.137+0000 7f1077400700 1 -- 172.21.15.83:0/1191666127 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f107004d9e0 con 0x7f108010a370 2024-04-24T08:37:54.425 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:53.137+0000 7f1077e00700 1 --2- 172.21.15.83:0/1191666127 >> [v2:172.21.15.83:6800/1996371091,v1:172.21.15.83:6801/1996371091] conn(0x7f10640395b0 0x7f106403ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:37:54.425 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:53.138+0000 7f1077e00700 1 --2- 172.21.15.83:0/1191666127 >> [v2:172.21.15.83:6800/1996371091,v1:172.21.15.83:6801/1996371091] conn(0x7f10640395b0 0x7f106403ba70 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f1068006fd0 tx=0x7f1068006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:37:54.425 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:53.143+0000 7f1077400700 1 -- 172.21.15.83:0/1191666127 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1070017ca0 con 0x7f108010a370 2024-04-24T08:37:54.426 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:53.454+0000 7f107fe00700 1 -- 172.21.15.83:0/1191666127 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f10801a18e0 con 0x7f108010a370 2024-04-24T08:37:54.426 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:53.455+0000 7f1077400700 1 -- 172.21.15.83:0/1191666127 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f10801a18e0 con 0x7f108010a370 2024-04-24T08:37:54.426 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:53.457+0000 7f107fe00700 1 -- 172.21.15.83:0/1191666127 >> [v2:172.21.15.83:6800/1996371091,v1:172.21.15.83:6801/1996371091] conn(0x7f10640395b0 msgr2=0x7f106403ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:37:54.426 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:53.457+0000 7f107fe00700 1 --2- 172.21.15.83:0/1191666127 >> [v2:172.21.15.83:6800/1996371091,v1:172.21.15.83:6801/1996371091] conn(0x7f10640395b0 0x7f106403ba70 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f1068006fd0 tx=0x7f1068006e40 comp rx=0 tx=0).stop 2024-04-24T08:37:54.426 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:53.458+0000 7f107fe00700 1 -- 172.21.15.83:0/1191666127 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f108010a370 msgr2=0x7f108019e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:37:54.426 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:53.458+0000 7f107fe00700 1 --2- 172.21.15.83:0/1191666127 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f108010a370 0x7f108019e250 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f1070000c00 tx=0x7f1070011b00 comp rx=0 tx=0).stop 2024-04-24T08:37:54.426 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:53.458+0000 7f107fe00700 1 -- 172.21.15.83:0/1191666127 shutdown_connections 2024-04-24T08:37:54.426 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:53.458+0000 7f107fe00700 1 --2- 172.21.15.83:0/1191666127 >> [v2:172.21.15.83:6800/1996371091,v1:172.21.15.83:6801/1996371091] conn(0x7f10640395b0 0x7f106403ba70 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:37:54.427 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:53.458+0000 7f107fe00700 1 --2- 172.21.15.83:0/1191666127 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f108010a370 0x7f108019e250 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:37:54.427 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:53.458+0000 7f107fe00700 1 -- 172.21.15.83:0/1191666127 >> 172.21.15.83:0/1191666127 conn(0x7f108007d8d0 msgr2=0x7f1080107c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:37:54.427 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:53.458+0000 7f107fe00700 1 -- 172.21.15.83:0/1191666127 shutdown_connections 2024-04-24T08:37:54.427 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:53.458+0000 7f107fe00700 1 -- 172.21.15.83:0/1191666127 wait complete. 2024-04-24T08:37:54.427 INFO:teuthology.orchestra.run.smithi083.stdout:mgr is available 2024-04-24T08:37:55.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:55 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/2471787592' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-24T08:37:55.935 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 2024-04-24T08:37:55.935 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout [global] 2024-04-24T08:37:55.935 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout fsid = bd3e039e-0215-11ef-bc93-c7b262605968 2024-04-24T08:37:55.935 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-24T08:37:55.935 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-24T08:37:55.936 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-24T08:37:55.936 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-24T08:37:55.936 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 2024-04-24T08:37:55.936 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-24T08:37:55.936 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-24T08:37:55.936 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 2024-04-24T08:37:55.936 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout [osd] 2024-04-24T08:37:55.936 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-24T08:37:55.936 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-24T08:37:55.936 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-24T08:37:55.936 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-24T08:37:55.936 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-24T08:37:55.936 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:55.053+0000 7ff8bd800700 1 Processor -- start 2024-04-24T08:37:55.936 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:55.053+0000 7ff8bd800700 1 -- start start 2024-04-24T08:37:55.937 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:55.054+0000 7ff8bd800700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8b81063e0 0x7ff8b8106800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:37:55.937 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:55.054+0000 7ff8bd800700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b8106dd0 con 0x7ff8b81063e0 2024-04-24T08:37:55.937 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:55.054+0000 7ff8b7e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8b81063e0 0x7ff8b8106800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:37:55.937 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:55.054+0000 7ff8b7e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8b81063e0 0x7ff8b8106800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49936/0 (socket says 172.21.15.83:49936) 2024-04-24T08:37:55.937 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:55.054+0000 7ff8b7e00700 1 -- 172.21.15.83:0/3399323054 learned_addr learned my addr 172.21.15.83:0/3399323054 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:37:55.937 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:55.054+0000 7ff8b7e00700 1 -- 172.21.15.83:0/3399323054 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8b81093c0 con 0x7ff8b81063e0 2024-04-24T08:37:55.937 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:55.055+0000 7ff8b7e00700 1 --2- 172.21.15.83:0/3399323054 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8b81063e0 0x7ff8b8106800 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff8a8009e80 tx=0x7ff8a800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bbe2d5bf630e75b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:37:55.937 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:55.055+0000 7ff8b7400700 1 -- 172.21.15.83:0/3399323054 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff8a8017070 con 0x7ff8b81063e0 2024-04-24T08:37:55.937 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:55.055+0000 7ff8b7400700 1 -- 172.21.15.83:0/3399323054 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7ff8a800b8c0 con 0x7ff8b81063e0 2024-04-24T08:37:55.937 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:55.055+0000 7ff8b7400700 1 -- 172.21.15.83:0/3399323054 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff8a8004520 con 0x7ff8b81063e0 2024-04-24T08:37:55.937 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:55.056+0000 7ff8bd800700 1 -- 172.21.15.83:0/3399323054 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8b81063e0 msgr2=0x7ff8b8106800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:37:55.937 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:55.056+0000 7ff8bd800700 1 --2- 172.21.15.83:0/3399323054 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8b81063e0 0x7ff8b8106800 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff8a8009e80 tx=0x7ff8a800b330 comp rx=0 tx=0).stop 2024-04-24T08:37:55.937 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:55.056+0000 7ff8bd800700 1 -- 172.21.15.83:0/3399323054 shutdown_connections 2024-04-24T08:37:55.937 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:55.056+0000 7ff8bd800700 1 --2- 172.21.15.83:0/3399323054 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8b81063e0 0x7ff8b8106800 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:37:55.937 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:55.056+0000 7ff8bd800700 1 -- 172.21.15.83:0/3399323054 >> 172.21.15.83:0/3399323054 conn(0x7ff8b8101d20 msgr2=0x7ff8b8104180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:37:55.938 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:55.056+0000 7ff8bd800700 1 -- 172.21.15.83:0/3399323054 shutdown_connections 2024-04-24T08:37:55.938 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:55.056+0000 7ff8bd800700 1 -- 172.21.15.83:0/3399323054 wait complete. 2024-04-24T08:37:55.938 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:55.057+0000 7ff8bd800700 1 Processor -- start 2024-04-24T08:37:55.938 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:55.057+0000 7ff8bd800700 1 -- start start 2024-04-24T08:37:55.938 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:55.058+0000 7ff8bd800700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8b81063e0 0x7ff8b819e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:37:55.938 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:55.058+0000 7ff8bd800700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b810c770 con 0x7ff8b81063e0 2024-04-24T08:37:55.938 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:55.058+0000 7ff8b7e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8b81063e0 0x7ff8b819e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:37:55.938 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:55.058+0000 7ff8b7e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8b81063e0 0x7ff8b819e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49938/0 (socket says 172.21.15.83:49938) 2024-04-24T08:37:55.938 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:55.058+0000 7ff8b7e00700 1 -- 172.21.15.83:0/2471787592 learned_addr learned my addr 172.21.15.83:0/2471787592 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:37:55.938 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:55.058+0000 7ff8b7e00700 1 -- 172.21.15.83:0/2471787592 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8b819e830 con 0x7ff8b81063e0 2024-04-24T08:37:55.938 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:55.058+0000 7ff8b7e00700 1 --2- 172.21.15.83:0/2471787592 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8b81063e0 0x7ff8b819e2f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff8a8000f80 tx=0x7ff8a8009640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:37:55.938 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:55.059+0000 7ff8b5600700 1 -- 172.21.15.83:0/2471787592 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff8a8020020 con 0x7ff8b81063e0 2024-04-24T08:37:55.938 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:55.059+0000 7ff8b5600700 1 -- 172.21.15.83:0/2471787592 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7ff8a800bba0 con 0x7ff8b81063e0 2024-04-24T08:37:55.939 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:55.059+0000 7ff8b5600700 1 -- 172.21.15.83:0/2471787592 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff8a8017040 con 0x7ff8b81063e0 2024-04-24T08:37:55.939 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:55.059+0000 7ff8bd800700 1 -- 172.21.15.83:0/2471787592 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8b819eac0 con 0x7ff8b81063e0 2024-04-24T08:37:55.939 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:55.059+0000 7ff8bd800700 1 -- 172.21.15.83:0/2471787592 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8b81a17b0 con 0x7ff8b81063e0 2024-04-24T08:37:55.939 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:55.060+0000 7ff8b5600700 1 -- 172.21.15.83:0/2471787592 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7ff8a8023020 con 0x7ff8b81063e0 2024-04-24T08:37:55.939 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:55.060+0000 7ff8bd800700 1 -- 172.21.15.83:0/2471787592 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8b8064150 con 0x7ff8b81063e0 2024-04-24T08:37:55.939 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:55.061+0000 7ff8b5600700 1 --2- 172.21.15.83:0/2471787592 >> [v2:172.21.15.83:6800/1996371091,v1:172.21.15.83:6801/1996371091] conn(0x7ff89c0395b0 0x7ff89c03ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:37:55.939 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:55.061+0000 7ff8b5600700 1 -- 172.21.15.83:0/2471787592 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff8a800f050 con 0x7ff8b81063e0 2024-04-24T08:37:55.939 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:55.061+0000 7ff8af400700 1 --2- 172.21.15.83:0/2471787592 >> [v2:172.21.15.83:6800/1996371091,v1:172.21.15.83:6801/1996371091] conn(0x7ff89c0395b0 0x7ff89c03ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:37:55.939 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:55.061+0000 7ff8af400700 1 --2- 172.21.15.83:0/2471787592 >> [v2:172.21.15.83:6800/1996371091,v1:172.21.15.83:6801/1996371091] conn(0x7ff89c0395b0 0x7ff89c03ba70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff8a0006fd0 tx=0x7ff8a0006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:37:55.939 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:55.066+0000 7ff8b5600700 1 -- 172.21.15.83:0/2471787592 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff8a802a020 con 0x7ff8b81063e0 2024-04-24T08:37:55.939 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:55.288+0000 7ff8bd800700 1 -- 172.21.15.83:0/2471787592 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7ff8b81a3020 con 0x7ff8b81063e0 2024-04-24T08:37:55.939 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:55.291+0000 7ff8b5600700 1 -- 172.21.15.83:0/2471787592 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+373 (secure 0 0 0) 0x7ff8a802c300 con 0x7ff8b81063e0 2024-04-24T08:37:55.939 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:55.294+0000 7ff8bd800700 1 -- 172.21.15.83:0/2471787592 >> [v2:172.21.15.83:6800/1996371091,v1:172.21.15.83:6801/1996371091] conn(0x7ff89c0395b0 msgr2=0x7ff89c03ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:37:55.939 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:55.294+0000 7ff8bd800700 1 --2- 172.21.15.83:0/2471787592 >> [v2:172.21.15.83:6800/1996371091,v1:172.21.15.83:6801/1996371091] conn(0x7ff89c0395b0 0x7ff89c03ba70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff8a0006fd0 tx=0x7ff8a0006e40 comp rx=0 tx=0).stop 2024-04-24T08:37:55.940 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:55.294+0000 7ff8bd800700 1 -- 172.21.15.83:0/2471787592 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8b81063e0 msgr2=0x7ff8b819e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:37:55.940 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:55.294+0000 7ff8bd800700 1 --2- 172.21.15.83:0/2471787592 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8b81063e0 0x7ff8b819e2f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff8a8000f80 tx=0x7ff8a8009640 comp rx=0 tx=0).stop 2024-04-24T08:37:55.940 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:55.294+0000 7ff8bd800700 1 -- 172.21.15.83:0/2471787592 shutdown_connections 2024-04-24T08:37:55.940 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:55.294+0000 7ff8bd800700 1 --2- 172.21.15.83:0/2471787592 >> [v2:172.21.15.83:6800/1996371091,v1:172.21.15.83:6801/1996371091] conn(0x7ff89c0395b0 0x7ff89c03ba70 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:37:55.940 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:55.294+0000 7ff8bd800700 1 --2- 172.21.15.83:0/2471787592 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8b81063e0 0x7ff8b819e2f0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:37:55.940 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:55.294+0000 7ff8bd800700 1 -- 172.21.15.83:0/2471787592 >> 172.21.15.83:0/2471787592 conn(0x7ff8b8101d20 msgr2=0x7ff8b81028f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:37:55.940 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:55.295+0000 7ff8bd800700 1 -- 172.21.15.83:0/2471787592 shutdown_connections 2024-04-24T08:37:55.940 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:55.295+0000 7ff8bd800700 1 -- 172.21.15.83:0/2471787592 wait complete. 2024-04-24T08:37:55.940 INFO:teuthology.orchestra.run.smithi083.stdout:Enabling cephadm module... 2024-04-24T08:37:57.220 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:56 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/1304226687' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-04-24T08:37:57.221 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:37:57 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: ignoring --setuser ceph since I am not root 2024-04-24T08:37:57.221 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:37:57 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: ignoring --setgroup ceph since I am not root 2024-04-24T08:37:57.221 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:37:57 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:37:57.219+0000 7fad99efd200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-24T08:37:57.553 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:37:57 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:37:57.465+0000 7fad99efd200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-24T08:37:57.701 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:56.579+0000 7fb3e2400700 1 Processor -- start 2024-04-24T08:37:57.701 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:56.579+0000 7fb3e2400700 1 -- start start 2024-04-24T08:37:57.701 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:56.579+0000 7fb3e2400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb3dc10ef20 0x7fb3dc10f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:37:57.702 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:56.579+0000 7fb3e2400700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb3dc10f880 con 0x7fb3dc10ef20 2024-04-24T08:37:57.702 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:56.579+0000 7fb3db400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb3dc10ef20 0x7fb3dc10f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:37:57.702 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:56.580+0000 7fb3db400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb3dc10ef20 0x7fb3dc10f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49950/0 (socket says 172.21.15.83:49950) 2024-04-24T08:37:57.702 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:56.580+0000 7fb3db400700 1 -- 172.21.15.83:0/362462508 learned_addr learned my addr 172.21.15.83:0/362462508 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:37:57.702 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:56.580+0000 7fb3db400700 1 -- 172.21.15.83:0/362462508 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3dc10fa00 con 0x7fb3dc10ef20 2024-04-24T08:37:57.702 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:56.580+0000 7fb3db400700 1 --2- 172.21.15.83:0/362462508 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb3dc10ef20 0x7fb3dc10f340 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fb3cc0098a0 tx=0x7fb3cc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21b849cd0d09e225 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:37:57.702 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:56.581+0000 7fb3da000700 1 -- 172.21.15.83:0/362462508 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb3cc004160 con 0x7fb3dc10ef20 2024-04-24T08:37:57.702 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:56.581+0000 7fb3da000700 1 -- 172.21.15.83:0/362462508 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fb3cc004300 con 0x7fb3dc10ef20 2024-04-24T08:37:57.702 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:56.581+0000 7fb3da000700 1 -- 172.21.15.83:0/362462508 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb3cc011a80 con 0x7fb3dc10ef20 2024-04-24T08:37:57.703 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:56.581+0000 7fb3e2400700 1 -- 172.21.15.83:0/362462508 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb3dc10ef20 msgr2=0x7fb3dc10f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:37:57.703 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:56.581+0000 7fb3e2400700 1 --2- 172.21.15.83:0/362462508 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb3dc10ef20 0x7fb3dc10f340 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fb3cc0098a0 tx=0x7fb3cc009bb0 comp rx=0 tx=0).stop 2024-04-24T08:37:57.703 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:56.581+0000 7fb3e2400700 1 -- 172.21.15.83:0/362462508 shutdown_connections 2024-04-24T08:37:57.703 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:56.581+0000 7fb3e2400700 1 --2- 172.21.15.83:0/362462508 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb3dc10ef20 0x7fb3dc10f340 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:37:57.703 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:56.581+0000 7fb3e2400700 1 -- 172.21.15.83:0/362462508 >> 172.21.15.83:0/362462508 conn(0x7fb3dc07dbb0 msgr2=0x7fb3dc07ffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:37:57.703 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:56.582+0000 7fb3e2400700 1 -- 172.21.15.83:0/362462508 shutdown_connections 2024-04-24T08:37:57.703 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:56.582+0000 7fb3e2400700 1 -- 172.21.15.83:0/362462508 wait complete. 2024-04-24T08:37:57.703 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:56.583+0000 7fb3e2400700 1 Processor -- start 2024-04-24T08:37:57.703 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:56.583+0000 7fb3e2400700 1 -- start start 2024-04-24T08:37:57.703 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:56.583+0000 7fb3e2400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb3dc10ef20 0x7fb3dc1a2850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:37:57.703 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:56.583+0000 7fb3e2400700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb3dc110a80 con 0x7fb3dc10ef20 2024-04-24T08:37:57.704 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:56.583+0000 7fb3db400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb3dc10ef20 0x7fb3dc1a2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:37:57.704 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:56.583+0000 7fb3db400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb3dc10ef20 0x7fb3dc1a2850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49956/0 (socket says 172.21.15.83:49956) 2024-04-24T08:37:57.704 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:56.583+0000 7fb3db400700 1 -- 172.21.15.83:0/1304226687 learned_addr learned my addr 172.21.15.83:0/1304226687 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:37:57.704 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:56.584+0000 7fb3db400700 1 -- 172.21.15.83:0/1304226687 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3dc1a2d90 con 0x7fb3dc10ef20 2024-04-24T08:37:57.704 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:56.584+0000 7fb3db400700 1 --2- 172.21.15.83:0/1304226687 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb3dc10ef20 0x7fb3dc1a2850 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb3cc006af0 tx=0x7fb3cc011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:37:57.704 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:56.584+0000 7fb3d3e00700 1 -- 172.21.15.83:0/1304226687 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb3cc003930 con 0x7fb3dc10ef20 2024-04-24T08:37:57.704 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:56.584+0000 7fb3d3e00700 1 -- 172.21.15.83:0/1304226687 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fb3cc003ad0 con 0x7fb3dc10ef20 2024-04-24T08:37:57.704 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:56.584+0000 7fb3d3e00700 1 -- 172.21.15.83:0/1304226687 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb3cc0195e0 con 0x7fb3dc10ef20 2024-04-24T08:37:57.704 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:56.584+0000 7fb3e2400700 1 -- 172.21.15.83:0/1304226687 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3dc1a3020 con 0x7fb3dc10ef20 2024-04-24T08:37:57.704 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:56.584+0000 7fb3e2400700 1 -- 172.21.15.83:0/1304226687 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3dc1a5d10 con 0x7fb3dc10ef20 2024-04-24T08:37:57.704 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:56.586+0000 7fb3d3e00700 1 -- 172.21.15.83:0/1304226687 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7fb3cc019890 con 0x7fb3dc10ef20 2024-04-24T08:37:57.705 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:56.586+0000 7fb3d1600700 1 -- 172.21.15.83:0/1304226687 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3b4005330 con 0x7fb3dc10ef20 2024-04-24T08:37:57.705 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:56.586+0000 7fb3d3e00700 1 --2- 172.21.15.83:0/1304226687 >> [v2:172.21.15.83:6800/1996371091,v1:172.21.15.83:6801/1996371091] conn(0x7fb3c0039560 0x7fb3c003ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:37:57.705 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:56.586+0000 7fb3d3e00700 1 -- 172.21.15.83:0/1304226687 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb3cc04db50 con 0x7fb3dc10ef20 2024-04-24T08:37:57.705 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:56.586+0000 7fb3daa00700 1 --2- 172.21.15.83:0/1304226687 >> [v2:172.21.15.83:6800/1996371091,v1:172.21.15.83:6801/1996371091] conn(0x7fb3c0039560 0x7fb3c003ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:37:57.705 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:56.587+0000 7fb3daa00700 1 --2- 172.21.15.83:0/1304226687 >> [v2:172.21.15.83:6800/1996371091,v1:172.21.15.83:6801/1996371091] conn(0x7fb3c0039560 0x7fb3c003ba20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb3c8006fd0 tx=0x7fb3c8006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:37:57.705 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:56.592+0000 7fb3d3e00700 1 -- 172.21.15.83:0/1304226687 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb3cc00b3e0 con 0x7fb3dc10ef20 2024-04-24T08:37:57.705 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:56.879+0000 7fb3d1600700 1 -- 172.21.15.83:0/1304226687 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7fb3b4005620 con 0x7fb3dc10ef20 2024-04-24T08:37:57.705 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:56.933+0000 7fb3d3e00700 1 -- 172.21.15.83:0/1304226687 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7fb3cc023e00 con 0x7fb3dc10ef20 2024-04-24T08:37:57.705 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:56.936+0000 7fb3d3e00700 1 -- 172.21.15.83:0/1304226687 <== mon.0 v2:172.21.15.83:3300/0 8 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7fb3cc01a430 con 0x7fb3dc10ef20 2024-04-24T08:37:57.705 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:56.938+0000 7fb3d1600700 1 -- 172.21.15.83:0/1304226687 >> [v2:172.21.15.83:6800/1996371091,v1:172.21.15.83:6801/1996371091] conn(0x7fb3c0039560 msgr2=0x7fb3c003ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:37:57.706 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:56.938+0000 7fb3d1600700 1 --2- 172.21.15.83:0/1304226687 >> [v2:172.21.15.83:6800/1996371091,v1:172.21.15.83:6801/1996371091] conn(0x7fb3c0039560 0x7fb3c003ba20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb3c8006fd0 tx=0x7fb3c8006e40 comp rx=0 tx=0).stop 2024-04-24T08:37:57.706 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:56.938+0000 7fb3d1600700 1 -- 172.21.15.83:0/1304226687 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb3dc10ef20 msgr2=0x7fb3dc1a2850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:37:57.706 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:56.938+0000 7fb3d1600700 1 --2- 172.21.15.83:0/1304226687 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb3dc10ef20 0x7fb3dc1a2850 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb3cc006af0 tx=0x7fb3cc011b70 comp rx=0 tx=0).stop 2024-04-24T08:37:57.706 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:56.938+0000 7fb3d1600700 1 -- 172.21.15.83:0/1304226687 shutdown_connections 2024-04-24T08:37:57.706 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:56.938+0000 7fb3d1600700 1 --2- 172.21.15.83:0/1304226687 >> [v2:172.21.15.83:6800/1996371091,v1:172.21.15.83:6801/1996371091] conn(0x7fb3c0039560 0x7fb3c003ba20 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:37:57.706 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:56.938+0000 7fb3d1600700 1 --2- 172.21.15.83:0/1304226687 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb3dc10ef20 0x7fb3dc1a2850 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:37:57.706 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:56.938+0000 7fb3d1600700 1 -- 172.21.15.83:0/1304226687 >> 172.21.15.83:0/1304226687 conn(0x7fb3dc07dbb0 msgr2=0x7fb3dc07f6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:37:57.706 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:56.939+0000 7fb3d1600700 1 -- 172.21.15.83:0/1304226687 shutdown_connections 2024-04-24T08:37:57.706 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:56.939+0000 7fb3d1600700 1 -- 172.21.15.83:0/1304226687 wait complete. 2024-04-24T08:37:57.864 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:37:57 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:37:57.597+0000 7fad99efd200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-24T08:37:58.130 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:57 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/1304226687' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-04-24T08:37:58.130 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:57 smithi083 ceph-mon[27907]: mgrmap e5: x(active, since 6s) 2024-04-24T08:37:58.131 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:37:58 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:37:58.011+0000 7fad99efd200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-24T08:37:58.496 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:37:58 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:37:58.130+0000 7fad99efd200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-24T08:37:59.276 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:37:58 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/4264922516' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-24T08:37:59.296 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout { 2024-04-24T08:37:59.296 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-04-24T08:37:59.297 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-24T08:37:59.297 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-24T08:37:59.297 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-24T08:37:59.297 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout } 2024-04-24T08:37:59.297 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:58.285+0000 7fed12e00700 1 Processor -- start 2024-04-24T08:37:59.297 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:58.285+0000 7fed12e00700 1 -- start start 2024-04-24T08:37:59.297 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:58.286+0000 7fed12e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed0c07de60 0x7fed0c07c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:37:59.297 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:58.286+0000 7fed12e00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fed0c07c8e0 con 0x7fed0c07de60 2024-04-24T08:37:59.297 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:58.286+0000 7fed11a00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed0c07de60 0x7fed0c07c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:37:59.297 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:58.286+0000 7fed11a00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed0c07de60 0x7fed0c07c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49970/0 (socket says 172.21.15.83:49970) 2024-04-24T08:37:59.298 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:58.286+0000 7fed11a00700 1 -- 172.21.15.83:0/1753477999 learned_addr learned my addr 172.21.15.83:0/1753477999 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:37:59.298 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:58.287+0000 7fed11a00700 1 -- 172.21.15.83:0/1753477999 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed0c07ca60 con 0x7fed0c07de60 2024-04-24T08:37:59.298 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:58.287+0000 7fed11a00700 1 --2- 172.21.15.83:0/1753477999 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed0c07de60 0x7fed0c07c310 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fed000098a0 tx=0x7fed00009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37133130bdb906f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:37:59.298 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:58.287+0000 7fed0be00700 1 -- 172.21.15.83:0/1753477999 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fed00004160 con 0x7fed0c07de60 2024-04-24T08:37:59.298 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:58.287+0000 7fed0be00700 1 -- 172.21.15.83:0/1753477999 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fed00004300 con 0x7fed0c07de60 2024-04-24T08:37:59.298 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:58.287+0000 7fed0be00700 1 -- 172.21.15.83:0/1753477999 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fed00011a80 con 0x7fed0c07de60 2024-04-24T08:37:59.298 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:58.288+0000 7fed12e00700 1 -- 172.21.15.83:0/1753477999 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed0c07de60 msgr2=0x7fed0c07c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:37:59.298 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:58.288+0000 7fed12e00700 1 --2- 172.21.15.83:0/1753477999 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed0c07de60 0x7fed0c07c310 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fed000098a0 tx=0x7fed00009bb0 comp rx=0 tx=0).stop 2024-04-24T08:37:59.298 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:58.288+0000 7fed12e00700 1 -- 172.21.15.83:0/1753477999 shutdown_connections 2024-04-24T08:37:59.298 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:58.288+0000 7fed12e00700 1 --2- 172.21.15.83:0/1753477999 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed0c07de60 0x7fed0c07c310 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:37:59.298 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:58.288+0000 7fed12e00700 1 -- 172.21.15.83:0/1753477999 >> 172.21.15.83:0/1753477999 conn(0x7fed0c103740 msgr2=0x7fed0c105b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:37:59.298 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:58.288+0000 7fed12e00700 1 -- 172.21.15.83:0/1753477999 shutdown_connections 2024-04-24T08:37:59.298 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:58.289+0000 7fed12e00700 1 -- 172.21.15.83:0/1753477999 wait complete. 2024-04-24T08:37:59.298 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:58.290+0000 7fed12e00700 1 Processor -- start 2024-04-24T08:37:59.299 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:58.290+0000 7fed12e00700 1 -- start start 2024-04-24T08:37:59.299 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:58.291+0000 7fed12e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed0c07de60 0x7fed0c19e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:37:59.299 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:58.291+0000 7fed12e00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fed0c10e630 con 0x7fed0c07de60 2024-04-24T08:37:59.299 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:58.291+0000 7fed11a00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed0c07de60 0x7fed0c19e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:37:59.299 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:58.291+0000 7fed11a00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed0c07de60 0x7fed0c19e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49976/0 (socket says 172.21.15.83:49976) 2024-04-24T08:37:59.299 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:58.291+0000 7fed11a00700 1 -- 172.21.15.83:0/4264922516 learned_addr learned my addr 172.21.15.83:0/4264922516 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:37:59.299 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:58.291+0000 7fed11a00700 1 -- 172.21.15.83:0/4264922516 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed0c19e7e0 con 0x7fed0c07de60 2024-04-24T08:37:59.299 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:58.292+0000 7fed11a00700 1 --2- 172.21.15.83:0/4264922516 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed0c07de60 0x7fed0c19e2a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fed00011a50 tx=0x7fed00004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:37:59.299 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:58.292+0000 7fed0a000700 1 -- 172.21.15.83:0/4264922516 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fed000039f0 con 0x7fed0c07de60 2024-04-24T08:37:59.299 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:58.292+0000 7fed0a000700 1 -- 172.21.15.83:0/4264922516 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fed00003b90 con 0x7fed0c07de60 2024-04-24T08:37:59.299 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:58.292+0000 7fed12e00700 1 -- 172.21.15.83:0/4264922516 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed0c19ea70 con 0x7fed0c07de60 2024-04-24T08:37:59.299 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:58.292+0000 7fed0a000700 1 -- 172.21.15.83:0/4264922516 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fed0001a600 con 0x7fed0c07de60 2024-04-24T08:37:59.299 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:58.292+0000 7fed12e00700 1 -- 172.21.15.83:0/4264922516 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed0c1a1760 con 0x7fed0c07de60 2024-04-24T08:37:59.299 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:58.293+0000 7fed0a000700 1 -- 172.21.15.83:0/4264922516 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7fed0001a8d0 con 0x7fed0c07de60 2024-04-24T08:37:59.300 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:58.293+0000 7fed12e00700 1 -- 172.21.15.83:0/4264922516 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed0c064150 con 0x7fed0c07de60 2024-04-24T08:37:59.300 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:58.294+0000 7fed0a000700 1 --2- 172.21.15.83:0/4264922516 >> [v2:172.21.15.83:6800/1996371091,v1:172.21.15.83:6801/1996371091] conn(0x7fecf4039600 0x7fecf403bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:37:59.300 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:58.294+0000 7fed0a000700 1 -- 172.21.15.83:0/4264922516 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fed0004dc50 con 0x7fed0c07de60 2024-04-24T08:37:59.300 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:58.294+0000 7fed11000700 1 -- 172.21.15.83:0/4264922516 >> [v2:172.21.15.83:6800/1996371091,v1:172.21.15.83:6801/1996371091] conn(0x7fecf4039600 msgr2=0x7fecf403bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.83:6800/1996371091 2024-04-24T08:37:59.300 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:58.294+0000 7fed11000700 1 --2- 172.21.15.83:0/4264922516 >> [v2:172.21.15.83:6800/1996371091,v1:172.21.15.83:6801/1996371091] conn(0x7fecf4039600 0x7fecf403bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T08:37:59.300 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:58.299+0000 7fed0a000700 1 -- 172.21.15.83:0/4264922516 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fed0000ba00 con 0x7fed0c07de60 2024-04-24T08:37:59.300 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:58.495+0000 7fed11000700 1 -- 172.21.15.83:0/4264922516 >> [v2:172.21.15.83:6800/1996371091,v1:172.21.15.83:6801/1996371091] conn(0x7fecf4039600 msgr2=0x7fecf403bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.83:6800/1996371091 2024-04-24T08:37:59.300 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:58.495+0000 7fed11000700 1 --2- 172.21.15.83:0/4264922516 >> [v2:172.21.15.83:6800/1996371091,v1:172.21.15.83:6801/1996371091] conn(0x7fecf4039600 0x7fecf403bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-24T08:37:59.300 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:58.581+0000 7fed12e00700 1 -- 172.21.15.83:0/4264922516 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fed0c1a1930 con 0x7fed0c07de60 2024-04-24T08:37:59.300 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:58.582+0000 7fed0a000700 1 -- 172.21.15.83:0/4264922516 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7fed0c1a1930 con 0x7fed0c07de60 2024-04-24T08:37:59.300 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:58.584+0000 7fed12e00700 1 -- 172.21.15.83:0/4264922516 >> [v2:172.21.15.83:6800/1996371091,v1:172.21.15.83:6801/1996371091] conn(0x7fecf4039600 msgr2=0x7fecf403bac0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T08:37:59.300 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:58.584+0000 7fed12e00700 1 --2- 172.21.15.83:0/4264922516 >> [v2:172.21.15.83:6800/1996371091,v1:172.21.15.83:6801/1996371091] conn(0x7fecf4039600 0x7fecf403bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:37:59.300 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:58.584+0000 7fed12e00700 1 -- 172.21.15.83:0/4264922516 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed0c07de60 msgr2=0x7fed0c19e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:37:59.301 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:58.584+0000 7fed12e00700 1 --2- 172.21.15.83:0/4264922516 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed0c07de60 0x7fed0c19e2a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fed00011a50 tx=0x7fed00004290 comp rx=0 tx=0).stop 2024-04-24T08:37:59.301 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:58.584+0000 7fed12e00700 1 -- 172.21.15.83:0/4264922516 shutdown_connections 2024-04-24T08:37:59.301 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:58.585+0000 7fed12e00700 1 --2- 172.21.15.83:0/4264922516 >> [v2:172.21.15.83:6800/1996371091,v1:172.21.15.83:6801/1996371091] conn(0x7fecf4039600 0x7fecf403bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:37:59.301 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:58.585+0000 7fed12e00700 1 --2- 172.21.15.83:0/4264922516 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed0c07de60 0x7fed0c19e2a0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:37:59.301 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:58.585+0000 7fed12e00700 1 -- 172.21.15.83:0/4264922516 >> 172.21.15.83:0/4264922516 conn(0x7fed0c103740 msgr2=0x7fed0c1042f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:37:59.301 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:58.585+0000 7fed12e00700 1 -- 172.21.15.83:0/4264922516 shutdown_connections 2024-04-24T08:37:59.301 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:58.585+0000 7fed12e00700 1 -- 172.21.15.83:0/4264922516 wait complete. 2024-04-24T08:37:59.301 INFO:teuthology.orchestra.run.smithi083.stdout:Waiting for the mgr to restart... 2024-04-24T08:37:59.301 INFO:teuthology.orchestra.run.smithi083.stdout:Waiting for mgr epoch 5... 2024-04-24T08:37:59.878 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:37:59 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:37:59.703+0000 7fad99efd200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-24T08:38:00.139 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:37:59 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:37:59.939+0000 7fad99efd200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-24T08:38:00.540 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:38:00 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:38:00.165+0000 7fad99efd200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-24T08:38:00.804 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:38:00 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:38:00.540+0000 7fad99efd200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-24T08:38:00.804 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:38:00 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:38:00.660+0000 7fad99efd200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-24T08:38:01.164 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:38:00 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:38:00.909+0000 7fad99efd200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-24T08:38:01.164 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:38:01 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:38:01.162+0000 7fad99efd200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-24T08:38:02.304 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:38:01 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:38:01.937+0000 7fad99efd200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-24T08:38:02.804 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:38:02 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:38:02.482+0000 7fad99efd200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-24T08:38:02.804 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:38:02 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:38:02.608+0000 7fad99efd200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-24T08:38:03.304 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:38:02 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:38:02.870+0000 7fad99efd200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-24T08:38:03.804 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:38:03 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:38:03.508+0000 7fad99efd200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-24T08:38:03.804 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:38:03 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:38:03.632+0000 7fad99efd200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-24T08:38:04.252 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:38:03 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:38:03.950+0000 7fad99efd200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-24T08:38:04.252 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:38:04 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:38:04.125+0000 7fad99efd200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-24T08:38:04.252 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:38:04 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:38:04.250+0000 7fad99efd200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-24T08:38:04.554 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:38:04 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:38:04.401+0000 7fad99efd200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-24T08:38:05.053 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:38:04 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:38:04.768+0000 7fad99efd200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-24T08:38:05.799 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:38:05 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:38:05.513+0000 7fad99efd200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-24T08:38:06.054 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:38:05 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:38:05.797+0000 7fad99efd200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-24T08:38:06.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:38:05 smithi083 ceph-mon[27907]: Active manager daemon x restarted 2024-04-24T08:38:06.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:38:05 smithi083 ceph-mon[27907]: Activating manager daemon x 2024-04-24T08:38:06.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:38:05 smithi083 ceph-mon[27907]: osdmap e2: 0 total, 0 up, 0 in 2024-04-24T08:38:06.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:38:05 smithi083 ceph-mon[27907]: mgrmap e6: x(active, starting, since 0.00766539s) 2024-04-24T08:38:06.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:38:05 smithi083 ceph-mon[27907]: from='mgr.14120 172.21.15.83:0/3660207153' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T08:38:06.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:38:05 smithi083 ceph-mon[27907]: from='mgr.14120 172.21.15.83:0/3660207153' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-24T08:38:06.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:38:05 smithi083 ceph-mon[27907]: from='mgr.14120 172.21.15.83:0/3660207153' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-24T08:38:06.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:38:05 smithi083 ceph-mon[27907]: from='mgr.14120 172.21.15.83:0/3660207153' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-24T08:38:06.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:38:05 smithi083 ceph-mon[27907]: from='mgr.14120 172.21.15.83:0/3660207153' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-24T08:38:06.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:38:05 smithi083 ceph-mon[27907]: Manager daemon x is now available 2024-04-24T08:38:06.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:38:05 smithi083 ceph-mon[27907]: from='mgr.14120 172.21.15.83:0/3660207153' entity='mgr.x' 2024-04-24T08:38:06.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:38:05 smithi083 ceph-mon[27907]: from='mgr.14120 172.21.15.83:0/3660207153' entity='mgr.x' 2024-04-24T08:38:07.238 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:38:06 smithi083 ceph-mon[27907]: Found migration_current of "None". Setting to last migration. 2024-04-24T08:38:07.238 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:38:06 smithi083 ceph-mon[27907]: from='mgr.14120 172.21.15.83:0/3660207153' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:38:07.238 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:38:06 smithi083 ceph-mon[27907]: from='mgr.14120 172.21.15.83:0/3660207153' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:38:07.238 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:38:06 smithi083 ceph-mon[27907]: from='mgr.14120 172.21.15.83:0/3660207153' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-24T08:38:07.238 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:38:06 smithi083 ceph-mon[27907]: from='mgr.14120 172.21.15.83:0/3660207153' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-24T08:38:07.238 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:38:06 smithi083 ceph-mon[27907]: from='mgr.14120 172.21.15.83:0/3660207153' entity='mgr.x' 2024-04-24T08:38:07.238 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:38:06 smithi083 ceph-mon[27907]: from='mgr.14120 172.21.15.83:0/3660207153' entity='mgr.x' 2024-04-24T08:38:07.238 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:38:06 smithi083 ceph-mon[27907]: mgrmap e7: x(active, since 1.01664s) 2024-04-24T08:38:07.566 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout { 2024-04-24T08:38:07.566 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-04-24T08:38:07.566 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-24T08:38:07.566 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout } 2024-04-24T08:38:07.566 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:59.928+0000 7f5dd5c00700 1 Processor -- start 2024-04-24T08:38:07.566 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:59.928+0000 7f5dd5c00700 1 -- start start 2024-04-24T08:38:07.566 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:59.929+0000 7f5dd5c00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dd01083c0 0x7f5dd01087e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:38:07.566 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:59.929+0000 7f5dd5c00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5dd0108db0 con 0x7f5dd01083c0 2024-04-24T08:38:07.566 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:59.929+0000 7f5dcea00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dd01083c0 0x7f5dd01087e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:38:07.566 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:59.929+0000 7f5dcea00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dd01083c0 0x7f5dd01087e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39758/0 (socket says 172.21.15.83:39758) 2024-04-24T08:38:07.567 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:59.929+0000 7f5dcea00700 1 -- 172.21.15.83:0/2531616347 learned_addr learned my addr 172.21.15.83:0/2531616347 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:38:07.567 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:59.930+0000 7f5dcea00700 1 -- 172.21.15.83:0/2531616347 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dd0109520 con 0x7f5dd01083c0 2024-04-24T08:38:07.567 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:59.930+0000 7f5dcea00700 1 --2- 172.21.15.83:0/2531616347 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dd01083c0 0x7f5dd01087e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f5dc00098a0 tx=0x7f5dc0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8d4667abdd9ad66 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:38:07.567 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:59.930+0000 7f5dcd600700 1 -- 172.21.15.83:0/2531616347 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5dc0004160 con 0x7f5dd01083c0 2024-04-24T08:38:07.567 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:59.930+0000 7f5dcd600700 1 -- 172.21.15.83:0/2531616347 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f5dc0004300 con 0x7f5dd01083c0 2024-04-24T08:38:07.567 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:59.930+0000 7f5dcd600700 1 -- 172.21.15.83:0/2531616347 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5dc0011a80 con 0x7f5dd01083c0 2024-04-24T08:38:07.567 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:59.931+0000 7f5dd5c00700 1 -- 172.21.15.83:0/2531616347 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dd01083c0 msgr2=0x7f5dd01087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:38:07.567 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:59.931+0000 7f5dd5c00700 1 --2- 172.21.15.83:0/2531616347 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dd01083c0 0x7f5dd01087e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f5dc00098a0 tx=0x7f5dc0009bb0 comp rx=0 tx=0).stop 2024-04-24T08:38:07.567 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:59.931+0000 7f5dd5c00700 1 -- 172.21.15.83:0/2531616347 shutdown_connections 2024-04-24T08:38:07.567 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:59.931+0000 7f5dd5c00700 1 --2- 172.21.15.83:0/2531616347 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dd01083c0 0x7f5dd01087e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:38:07.567 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:59.931+0000 7f5dd5c00700 1 -- 172.21.15.83:0/2531616347 >> 172.21.15.83:0/2531616347 conn(0x7f5dd0103920 msgr2=0x7f5dd0105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:38:07.567 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:59.931+0000 7f5dd5c00700 1 -- 172.21.15.83:0/2531616347 shutdown_connections 2024-04-24T08:38:07.567 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:59.931+0000 7f5dd5c00700 1 -- 172.21.15.83:0/2531616347 wait complete. 2024-04-24T08:38:07.568 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:59.933+0000 7f5dd5c00700 1 Processor -- start 2024-04-24T08:38:07.568 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:59.933+0000 7f5dd5c00700 1 -- start start 2024-04-24T08:38:07.568 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:59.933+0000 7f5dd5c00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dd01083c0 0x7f5dd019c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:38:07.568 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:59.933+0000 7f5dd5c00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5dd010c8d0 con 0x7f5dd01083c0 2024-04-24T08:38:07.568 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:59.934+0000 7f5dcea00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dd01083c0 0x7f5dd019c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:38:07.568 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:59.934+0000 7f5dcea00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dd01083c0 0x7f5dd019c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39762/0 (socket says 172.21.15.83:39762) 2024-04-24T08:38:07.568 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:59.934+0000 7f5dcea00700 1 -- 172.21.15.83:0/3823328060 learned_addr learned my addr 172.21.15.83:0/3823328060 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:38:07.568 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:59.934+0000 7f5dcea00700 1 -- 172.21.15.83:0/3823328060 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dd019c7f0 con 0x7f5dd01083c0 2024-04-24T08:38:07.568 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:59.934+0000 7f5dcea00700 1 --2- 172.21.15.83:0/3823328060 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dd01083c0 0x7f5dd019c2b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f5dc0006af0 tx=0x7f5dc0011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:38:07.568 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:59.935+0000 7f5dc7400700 1 -- 172.21.15.83:0/3823328060 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5dc0003930 con 0x7f5dd01083c0 2024-04-24T08:38:07.568 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:59.935+0000 7f5dc7400700 1 -- 172.21.15.83:0/3823328060 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f5dc0003ad0 con 0x7f5dd01083c0 2024-04-24T08:38:07.568 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:59.935+0000 7f5dd5c00700 1 -- 172.21.15.83:0/3823328060 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dd019ca80 con 0x7f5dd01083c0 2024-04-24T08:38:07.568 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:59.935+0000 7f5dd5c00700 1 -- 172.21.15.83:0/3823328060 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dd019f770 con 0x7f5dd01083c0 2024-04-24T08:38:07.568 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:59.935+0000 7f5dc7400700 1 -- 172.21.15.83:0/3823328060 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5dc00195e0 con 0x7f5dd01083c0 2024-04-24T08:38:07.569 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:59.936+0000 7f5dc7400700 1 -- 172.21.15.83:0/3823328060 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f5dc0019910 con 0x7f5dd01083c0 2024-04-24T08:38:07.569 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:59.937+0000 7f5dc7400700 1 --2- 172.21.15.83:0/3823328060 >> [v2:172.21.15.83:6800/1996371091,v1:172.21.15.83:6801/1996371091] conn(0x7f5db40395b0 0x7f5db403ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:38:07.569 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:59.937+0000 7f5dc7400700 1 -- 172.21.15.83:0/3823328060 --> [v2:172.21.15.83:6800/1996371091,v1:172.21.15.83:6801/1996371091] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f5db403c180 con 0x7f5db40395b0 2024-04-24T08:38:07.569 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:59.937+0000 7f5dc7400700 1 -- 172.21.15.83:0/3823328060 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f5dc004dba0 con 0x7f5dd01083c0 2024-04-24T08:38:07.569 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:59.937+0000 7f5dce000700 1 -- 172.21.15.83:0/3823328060 >> [v2:172.21.15.83:6800/1996371091,v1:172.21.15.83:6801/1996371091] conn(0x7f5db40395b0 msgr2=0x7f5db403ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.83:6800/1996371091 2024-04-24T08:38:07.569 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:37:59.937+0000 7f5dce000700 1 --2- 172.21.15.83:0/3823328060 >> [v2:172.21.15.83:6800/1996371091,v1:172.21.15.83:6801/1996371091] conn(0x7f5db40395b0 0x7f5db403ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T08:38:07.569 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:00.138+0000 7f5dce000700 1 -- 172.21.15.83:0/3823328060 >> [v2:172.21.15.83:6800/1996371091,v1:172.21.15.83:6801/1996371091] conn(0x7f5db40395b0 msgr2=0x7f5db403ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.83:6800/1996371091 2024-04-24T08:38:07.569 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:00.138+0000 7f5dce000700 1 --2- 172.21.15.83:0/3823328060 >> [v2:172.21.15.83:6800/1996371091,v1:172.21.15.83:6801/1996371091] conn(0x7f5db40395b0 0x7f5db403ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-24T08:38:07.569 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:00.539+0000 7f5dce000700 1 -- 172.21.15.83:0/3823328060 >> [v2:172.21.15.83:6800/1996371091,v1:172.21.15.83:6801/1996371091] conn(0x7f5db40395b0 msgr2=0x7f5db403ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.83:6800/1996371091 2024-04-24T08:38:07.569 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:00.539+0000 7f5dce000700 1 --2- 172.21.15.83:0/3823328060 >> [v2:172.21.15.83:6800/1996371091,v1:172.21.15.83:6801/1996371091] conn(0x7f5db40395b0 0x7f5db403ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-24T08:38:07.569 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:01.340+0000 7f5dce000700 1 -- 172.21.15.83:0/3823328060 >> [v2:172.21.15.83:6800/1996371091,v1:172.21.15.83:6801/1996371091] conn(0x7f5db40395b0 msgr2=0x7f5db403ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.83:6800/1996371091 2024-04-24T08:38:07.569 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:01.340+0000 7f5dce000700 1 --2- 172.21.15.83:0/3823328060 >> [v2:172.21.15.83:6800/1996371091,v1:172.21.15.83:6801/1996371091] conn(0x7f5db40395b0 0x7f5db403ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-24T08:38:07.569 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:02.942+0000 7f5dce000700 1 -- 172.21.15.83:0/3823328060 >> [v2:172.21.15.83:6800/1996371091,v1:172.21.15.83:6801/1996371091] conn(0x7f5db40395b0 msgr2=0x7f5db403ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.83:6800/1996371091 2024-04-24T08:38:07.569 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:02.942+0000 7f5dce000700 1 --2- 172.21.15.83:0/3823328060 >> [v2:172.21.15.83:6800/1996371091,v1:172.21.15.83:6801/1996371091] conn(0x7f5db40395b0 0x7f5db403ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-24T08:38:07.570 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:05.806+0000 7f5dc7400700 1 -- 172.21.15.83:0/3823328060 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mgrmap(e 6) ==== 46325+0+0 (secure 0 0 0) 0x7f5dc0019c50 con 0x7f5dd01083c0 2024-04-24T08:38:07.570 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:05.807+0000 7f5dc7400700 1 -- 172.21.15.83:0/3823328060 >> [v2:172.21.15.83:6800/1996371091,v1:172.21.15.83:6801/1996371091] conn(0x7f5db40395b0 msgr2=0x7f5db403ba70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T08:38:07.570 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:05.807+0000 7f5dc7400700 1 --2- 172.21.15.83:0/3823328060 >> [v2:172.21.15.83:6800/1996371091,v1:172.21.15.83:6801/1996371091] conn(0x7f5db40395b0 0x7f5db403ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:38:07.570 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:06.810+0000 7f5dc7400700 1 -- 172.21.15.83:0/3823328060 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mgrmap(e 7) ==== 46452+0+0 (secure 0 0 0) 0x7f5dc004f6b0 con 0x7f5dd01083c0 2024-04-24T08:38:07.570 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:06.811+0000 7f5dc7400700 1 --2- 172.21.15.83:0/3823328060 >> [v2:172.21.15.83:6800/820049883,v1:172.21.15.83:6801/820049883] conn(0x7f5db403cea0 0x7f5db403f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:38:07.570 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:06.811+0000 7f5dc7400700 1 -- 172.21.15.83:0/3823328060 --> [v2:172.21.15.83:6800/820049883,v1:172.21.15.83:6801/820049883] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f5dc000af30 con 0x7f5db403cea0 2024-04-24T08:38:07.570 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:06.813+0000 7f5dce000700 1 --2- 172.21.15.83:0/3823328060 >> [v2:172.21.15.83:6800/820049883,v1:172.21.15.83:6801/820049883] conn(0x7f5db403cea0 0x7f5db403f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:38:07.570 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:06.815+0000 7f5dce000700 1 --2- 172.21.15.83:0/3823328060 >> [v2:172.21.15.83:6800/820049883,v1:172.21.15.83:6801/820049883] conn(0x7f5db403cea0 0x7f5db403f290 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc003a10 tx=0x7f5dbc0092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:38:07.570 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:06.817+0000 7f5dc7400700 1 -- 172.21.15.83:0/3823328060 <== mgr.14120 v2:172.21.15.83:6800/820049883 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f5dc000af30 con 0x7f5db403cea0 2024-04-24T08:38:07.570 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:06.825+0000 7f5dd5c00700 1 -- 172.21.15.83:0/3823328060 --> [v2:172.21.15.83:6800/820049883,v1:172.21.15.83:6801/820049883] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f5dd0064150 con 0x7f5db403cea0 2024-04-24T08:38:07.570 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:06.825+0000 7f5dc7400700 1 -- 172.21.15.83:0/3823328060 <== mgr.14120 v2:172.21.15.83:6800/820049883 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f5dd0064150 con 0x7f5db403cea0 2024-04-24T08:38:07.570 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:06.826+0000 7f5dd5c00700 1 -- 172.21.15.83:0/3823328060 >> [v2:172.21.15.83:6800/820049883,v1:172.21.15.83:6801/820049883] conn(0x7f5db403cea0 msgr2=0x7f5db403f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:38:07.570 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:06.826+0000 7f5dd5c00700 1 --2- 172.21.15.83:0/3823328060 >> [v2:172.21.15.83:6800/820049883,v1:172.21.15.83:6801/820049883] conn(0x7f5db403cea0 0x7f5db403f290 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc003a10 tx=0x7f5dbc0092b0 comp rx=0 tx=0).stop 2024-04-24T08:38:07.571 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:06.826+0000 7f5dd5c00700 1 -- 172.21.15.83:0/3823328060 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dd01083c0 msgr2=0x7f5dd019c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:38:07.571 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:06.826+0000 7f5dd5c00700 1 --2- 172.21.15.83:0/3823328060 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dd01083c0 0x7f5dd019c2b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f5dc0006af0 tx=0x7f5dc0011b70 comp rx=0 tx=0).stop 2024-04-24T08:38:07.571 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:06.826+0000 7f5dd5c00700 1 -- 172.21.15.83:0/3823328060 shutdown_connections 2024-04-24T08:38:07.571 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:06.826+0000 7f5dd5c00700 1 --2- 172.21.15.83:0/3823328060 >> [v2:172.21.15.83:6800/820049883,v1:172.21.15.83:6801/820049883] conn(0x7f5db403cea0 0x7f5db403f290 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:38:07.571 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:06.826+0000 7f5dd5c00700 1 --2- 172.21.15.83:0/3823328060 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dd01083c0 0x7f5dd019c2b0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:38:07.571 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:06.826+0000 7f5dd5c00700 1 --2- 172.21.15.83:0/3823328060 >> [v2:172.21.15.83:6800/1996371091,v1:172.21.15.83:6801/1996371091] conn(0x7f5db40395b0 0x7f5db403ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:38:07.571 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:06.826+0000 7f5dd5c00700 1 -- 172.21.15.83:0/3823328060 >> 172.21.15.83:0/3823328060 conn(0x7f5dd0103920 msgr2=0x7f5dd01044a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:38:07.571 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:06.826+0000 7f5dd5c00700 1 -- 172.21.15.83:0/3823328060 shutdown_connections 2024-04-24T08:38:07.571 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:06.826+0000 7f5dd5c00700 1 -- 172.21.15.83:0/3823328060 wait complete. 2024-04-24T08:38:07.571 INFO:teuthology.orchestra.run.smithi083.stdout:mgr epoch 5 is available 2024-04-24T08:38:07.571 INFO:teuthology.orchestra.run.smithi083.stdout:Setting orchestrator backend to cephadm... 2024-04-24T08:38:08.168 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:38:07 smithi083 ceph-mon[27907]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-24T08:38:08.168 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:38:07 smithi083 ceph-mon[27907]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-24T08:38:08.168 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:38:07 smithi083 ceph-mon[27907]: [24/Apr/2024:08:38:06] ENGINE Bus STARTING 2024-04-24T08:38:08.168 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:38:07 smithi083 ceph-mon[27907]: [24/Apr/2024:08:38:06] ENGINE Serving on https://172.21.15.83:7150 2024-04-24T08:38:08.168 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:38:07 smithi083 ceph-mon[27907]: from='mgr.14120 172.21.15.83:0/3660207153' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:38:09.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:38:08 smithi083 ceph-mon[27907]: [24/Apr/2024:08:38:07] ENGINE Serving on http://172.21.15.83:8765 2024-04-24T08:38:09.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:38:08 smithi083 ceph-mon[27907]: [24/Apr/2024:08:38:07] ENGINE Bus STARTED 2024-04-24T08:38:09.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:38:08 smithi083 ceph-mon[27907]: mgrmap e8: x(active, since 2s) 2024-04-24T08:38:09.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:38:08 smithi083 ceph-mon[27907]: from='mgr.14120 172.21.15.83:0/3660207153' entity='mgr.x' 2024-04-24T08:38:09.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:38:08 smithi083 ceph-mon[27907]: from='mgr.14120 172.21.15.83:0/3660207153' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:38:09.191 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:08.194+0000 7f2ef9e00700 1 Processor -- start 2024-04-24T08:38:09.191 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:08.194+0000 7f2ef9e00700 1 -- start start 2024-04-24T08:38:09.191 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:08.194+0000 7f2ef9e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ef410ef20 0x7f2ef410f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:38:09.191 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:08.194+0000 7f2ef9e00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2ef410f880 con 0x7f2ef410ef20 2024-04-24T08:38:09.191 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:08.195+0000 7f2ef2a00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ef410ef20 0x7f2ef410f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:38:09.191 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:08.195+0000 7f2ef2a00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ef410ef20 0x7f2ef410f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39832/0 (socket says 172.21.15.83:39832) 2024-04-24T08:38:09.191 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:08.195+0000 7f2ef2a00700 1 -- 172.21.15.83:0/121536965 learned_addr learned my addr 172.21.15.83:0/121536965 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:38:09.191 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:08.195+0000 7f2ef2a00700 1 -- 172.21.15.83:0/121536965 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ef410fa00 con 0x7f2ef410ef20 2024-04-24T08:38:09.192 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:08.195+0000 7f2ef2a00700 1 --2- 172.21.15.83:0/121536965 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ef410ef20 0x7f2ef410f340 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2ee40098a0 tx=0x7f2ee4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4aa089d938f336df server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:38:09.192 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:08.196+0000 7f2ef1600700 1 -- 172.21.15.83:0/121536965 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2ee4004160 con 0x7f2ef410ef20 2024-04-24T08:38:09.192 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:08.196+0000 7f2ef1600700 1 -- 172.21.15.83:0/121536965 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f2ee4004300 con 0x7f2ef410ef20 2024-04-24T08:38:09.192 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:08.196+0000 7f2ef1600700 1 -- 172.21.15.83:0/121536965 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2ee4011a80 con 0x7f2ef410ef20 2024-04-24T08:38:09.192 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:08.196+0000 7f2ef9e00700 1 -- 172.21.15.83:0/121536965 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ef410ef20 msgr2=0x7f2ef410f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:38:09.192 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:08.196+0000 7f2ef9e00700 1 --2- 172.21.15.83:0/121536965 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ef410ef20 0x7f2ef410f340 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2ee40098a0 tx=0x7f2ee4009bb0 comp rx=0 tx=0).stop 2024-04-24T08:38:09.192 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:08.196+0000 7f2ef9e00700 1 -- 172.21.15.83:0/121536965 shutdown_connections 2024-04-24T08:38:09.192 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:08.196+0000 7f2ef9e00700 1 --2- 172.21.15.83:0/121536965 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ef410ef20 0x7f2ef410f340 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:38:09.192 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:08.196+0000 7f2ef9e00700 1 -- 172.21.15.83:0/121536965 >> 172.21.15.83:0/121536965 conn(0x7f2ef407dbb0 msgr2=0x7f2ef407ffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:38:09.192 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:08.197+0000 7f2ef9e00700 1 -- 172.21.15.83:0/121536965 shutdown_connections 2024-04-24T08:38:09.192 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:08.197+0000 7f2ef9e00700 1 -- 172.21.15.83:0/121536965 wait complete. 2024-04-24T08:38:09.192 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:08.198+0000 7f2ef9e00700 1 Processor -- start 2024-04-24T08:38:09.192 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:08.198+0000 7f2ef9e00700 1 -- start start 2024-04-24T08:38:09.193 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:08.198+0000 7f2ef9e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ef410ef20 0x7f2ef41a2850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:38:09.193 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:08.198+0000 7f2ef9e00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2ef4110a80 con 0x7f2ef410ef20 2024-04-24T08:38:09.193 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:08.198+0000 7f2ef2a00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ef410ef20 0x7f2ef41a2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:38:09.193 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:08.198+0000 7f2ef2a00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ef410ef20 0x7f2ef41a2850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39844/0 (socket says 172.21.15.83:39844) 2024-04-24T08:38:09.193 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:08.198+0000 7f2ef2a00700 1 -- 172.21.15.83:0/137008486 learned_addr learned my addr 172.21.15.83:0/137008486 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:38:09.193 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:08.199+0000 7f2ef2a00700 1 -- 172.21.15.83:0/137008486 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ef41a2d90 con 0x7f2ef410ef20 2024-04-24T08:38:09.193 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:08.199+0000 7f2ef2a00700 1 --2- 172.21.15.83:0/137008486 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ef410ef20 0x7f2ef41a2850 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2ee4006af0 tx=0x7f2ee4011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:38:09.193 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:08.199+0000 7f2ef8a00700 1 -- 172.21.15.83:0/137008486 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2ee4003930 con 0x7f2ef410ef20 2024-04-24T08:38:09.193 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:08.199+0000 7f2ef8a00700 1 -- 172.21.15.83:0/137008486 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f2ee4003ad0 con 0x7f2ef410ef20 2024-04-24T08:38:09.193 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:08.199+0000 7f2ef9e00700 1 -- 172.21.15.83:0/137008486 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ef41a3020 con 0x7f2ef410ef20 2024-04-24T08:38:09.193 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:08.199+0000 7f2ef8a00700 1 -- 172.21.15.83:0/137008486 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2ee40195e0 con 0x7f2ef410ef20 2024-04-24T08:38:09.193 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:08.199+0000 7f2ef9e00700 1 -- 172.21.15.83:0/137008486 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ef41a5d10 con 0x7f2ef410ef20 2024-04-24T08:38:09.193 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:08.201+0000 7f2ef9e00700 1 -- 172.21.15.83:0/137008486 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ef4064150 con 0x7f2ef410ef20 2024-04-24T08:38:09.193 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:08.201+0000 7f2ef8a00700 1 -- 172.21.15.83:0/137008486 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f2ee4019890 con 0x7f2ef410ef20 2024-04-24T08:38:09.194 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:08.201+0000 7f2ef8a00700 1 --2- 172.21.15.83:0/137008486 >> [v2:172.21.15.83:6800/820049883,v1:172.21.15.83:6801/820049883] conn(0x7f2ed80395b0 0x7f2ed803ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:38:09.194 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:08.201+0000 7f2ef8a00700 1 -- 172.21.15.83:0/137008486 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f2ee404db90 con 0x7f2ef410ef20 2024-04-24T08:38:09.194 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:08.202+0000 7f2ef2000700 1 --2- 172.21.15.83:0/137008486 >> [v2:172.21.15.83:6800/820049883,v1:172.21.15.83:6801/820049883] conn(0x7f2ed80395b0 0x7f2ed803ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:38:09.194 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:08.202+0000 7f2ef2000700 1 --2- 172.21.15.83:0/137008486 >> [v2:172.21.15.83:6800/820049883,v1:172.21.15.83:6801/820049883] conn(0x7f2ed80395b0 0x7f2ed803ba70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2ee0006fd0 tx=0x7f2ee0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:38:09.194 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:08.208+0000 7f2ef8a00700 1 -- 172.21.15.83:0/137008486 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2ee400fda0 con 0x7f2ef410ef20 2024-04-24T08:38:09.194 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:08.454+0000 7f2ef9e00700 1 -- 172.21.15.83:0/137008486 --> [v2:172.21.15.83:6800/820049883,v1:172.21.15.83:6801/820049883] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f2ef407fdd0 con 0x7f2ed80395b0 2024-04-24T08:38:09.194 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:08.461+0000 7f2ef8a00700 1 -- 172.21.15.83:0/137008486 <== mgr.14120 v2:172.21.15.83:6800/820049883 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f2ef407fdd0 con 0x7f2ed80395b0 2024-04-24T08:38:09.194 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:08.464+0000 7f2ef9e00700 1 -- 172.21.15.83:0/137008486 >> [v2:172.21.15.83:6800/820049883,v1:172.21.15.83:6801/820049883] conn(0x7f2ed80395b0 msgr2=0x7f2ed803ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:38:09.194 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:08.464+0000 7f2ef9e00700 1 --2- 172.21.15.83:0/137008486 >> [v2:172.21.15.83:6800/820049883,v1:172.21.15.83:6801/820049883] conn(0x7f2ed80395b0 0x7f2ed803ba70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2ee0006fd0 tx=0x7f2ee0006e40 comp rx=0 tx=0).stop 2024-04-24T08:38:09.194 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:08.464+0000 7f2ef9e00700 1 -- 172.21.15.83:0/137008486 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ef410ef20 msgr2=0x7f2ef41a2850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:38:09.194 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:08.464+0000 7f2ef9e00700 1 --2- 172.21.15.83:0/137008486 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ef410ef20 0x7f2ef41a2850 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2ee4006af0 tx=0x7f2ee4011b70 comp rx=0 tx=0).stop 2024-04-24T08:38:09.195 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:08.465+0000 7f2ef9e00700 1 -- 172.21.15.83:0/137008486 shutdown_connections 2024-04-24T08:38:09.195 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:08.465+0000 7f2ef9e00700 1 --2- 172.21.15.83:0/137008486 >> [v2:172.21.15.83:6800/820049883,v1:172.21.15.83:6801/820049883] conn(0x7f2ed80395b0 0x7f2ed803ba70 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:38:09.195 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:08.465+0000 7f2ef9e00700 1 --2- 172.21.15.83:0/137008486 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ef410ef20 0x7f2ef41a2850 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:38:09.195 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:08.465+0000 7f2ef9e00700 1 -- 172.21.15.83:0/137008486 >> 172.21.15.83:0/137008486 conn(0x7f2ef407dbb0 msgr2=0x7f2ef407f6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:38:09.196 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:08.465+0000 7f2ef9e00700 1 -- 172.21.15.83:0/137008486 shutdown_connections 2024-04-24T08:38:09.196 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:08.465+0000 7f2ef9e00700 1 -- 172.21.15.83:0/137008486 wait complete. 2024-04-24T08:38:10.051 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:38:09 smithi083 ceph-mon[27907]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:38:10.785 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout value unchanged 2024-04-24T08:38:10.787 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:09.805+0000 7fa2d8600700 1 Processor -- start 2024-04-24T08:38:10.787 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:09.805+0000 7fa2d8600700 1 -- start start 2024-04-24T08:38:10.787 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:09.806+0000 7fa2d8600700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2d01083c0 0x7fa2d01087e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:38:10.787 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:09.806+0000 7fa2d8600700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d0108db0 con 0x7fa2d01083c0 2024-04-24T08:38:10.787 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:09.806+0000 7fa2d5c00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2d01083c0 0x7fa2d01087e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:38:10.787 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:09.806+0000 7fa2d5c00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2d01083c0 0x7fa2d01087e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58888/0 (socket says 172.21.15.83:58888) 2024-04-24T08:38:10.787 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:09.806+0000 7fa2d5c00700 1 -- 172.21.15.83:0/472279199 learned_addr learned my addr 172.21.15.83:0/472279199 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:38:10.787 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:09.807+0000 7fa2d5c00700 1 -- 172.21.15.83:0/472279199 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2d0109520 con 0x7fa2d01083c0 2024-04-24T08:38:10.787 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:09.807+0000 7fa2d5c00700 1 --2- 172.21.15.83:0/472279199 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2d01083c0 0x7fa2d01087e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa2c40098a0 tx=0x7fa2c4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=335e9ebfc496db7e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:38:10.787 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:09.807+0000 7fa2cfe00700 1 -- 172.21.15.83:0/472279199 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa2c4004160 con 0x7fa2d01083c0 2024-04-24T08:38:10.788 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:09.807+0000 7fa2cfe00700 1 -- 172.21.15.83:0/472279199 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fa2c4004300 con 0x7fa2d01083c0 2024-04-24T08:38:10.788 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:09.807+0000 7fa2cfe00700 1 -- 172.21.15.83:0/472279199 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa2c4011a80 con 0x7fa2d01083c0 2024-04-24T08:38:10.792 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:09.808+0000 7fa2d8600700 1 -- 172.21.15.83:0/472279199 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2d01083c0 msgr2=0x7fa2d01087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:38:10.792 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:09.808+0000 7fa2d8600700 1 --2- 172.21.15.83:0/472279199 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2d01083c0 0x7fa2d01087e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa2c40098a0 tx=0x7fa2c4009bb0 comp rx=0 tx=0).stop 2024-04-24T08:38:10.793 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:09.809+0000 7fa2d8600700 1 -- 172.21.15.83:0/472279199 shutdown_connections 2024-04-24T08:38:10.793 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:09.809+0000 7fa2d8600700 1 --2- 172.21.15.83:0/472279199 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2d01083c0 0x7fa2d01087e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:38:10.793 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:09.809+0000 7fa2d8600700 1 -- 172.21.15.83:0/472279199 >> 172.21.15.83:0/472279199 conn(0x7fa2d0103920 msgr2=0x7fa2d0105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:38:10.793 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:09.809+0000 7fa2d8600700 1 -- 172.21.15.83:0/472279199 shutdown_connections 2024-04-24T08:38:10.793 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:09.809+0000 7fa2d8600700 1 -- 172.21.15.83:0/472279199 wait complete. 2024-04-24T08:38:10.793 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:09.810+0000 7fa2d8600700 1 Processor -- start 2024-04-24T08:38:10.793 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:09.810+0000 7fa2d8600700 1 -- start start 2024-04-24T08:38:10.793 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:09.811+0000 7fa2d8600700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2d01083c0 0x7fa2d019e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:38:10.793 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:09.811+0000 7fa2d8600700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d010c8d0 con 0x7fa2d01083c0 2024-04-24T08:38:10.793 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:09.811+0000 7fa2d5c00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2d01083c0 0x7fa2d019e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:38:10.793 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:09.811+0000 7fa2d5c00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2d01083c0 0x7fa2d019e4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58890/0 (socket says 172.21.15.83:58890) 2024-04-24T08:38:10.793 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:09.811+0000 7fa2d5c00700 1 -- 172.21.15.83:0/3325122036 learned_addr learned my addr 172.21.15.83:0/3325122036 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:38:10.793 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:09.811+0000 7fa2d5c00700 1 -- 172.21.15.83:0/3325122036 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2d019ea20 con 0x7fa2d01083c0 2024-04-24T08:38:10.794 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:09.812+0000 7fa2d5c00700 1 --2- 172.21.15.83:0/3325122036 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2d01083c0 0x7fa2d019e4e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa2c4000c00 tx=0x7fa2c4004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:38:10.794 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:09.812+0000 7fa2ce000700 1 -- 172.21.15.83:0/3325122036 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa2c40039c0 con 0x7fa2d01083c0 2024-04-24T08:38:10.794 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:09.812+0000 7fa2d8600700 1 -- 172.21.15.83:0/3325122036 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2d019ecb0 con 0x7fa2d01083c0 2024-04-24T08:38:10.794 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:09.812+0000 7fa2ce000700 1 -- 172.21.15.83:0/3325122036 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fa2c4003b60 con 0x7fa2d01083c0 2024-04-24T08:38:10.794 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:09.812+0000 7fa2d8600700 1 -- 172.21.15.83:0/3325122036 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2d01a19a0 con 0x7fa2d01083c0 2024-04-24T08:38:10.794 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:09.812+0000 7fa2ce000700 1 -- 172.21.15.83:0/3325122036 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa2c401a6f0 con 0x7fa2d01083c0 2024-04-24T08:38:10.794 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:09.814+0000 7fa2ce000700 1 -- 172.21.15.83:0/3325122036 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fa2c401a9a0 con 0x7fa2d01083c0 2024-04-24T08:38:10.794 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:09.814+0000 7fa2d8600700 1 -- 172.21.15.83:0/3325122036 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2b0005330 con 0x7fa2d01083c0 2024-04-24T08:38:10.794 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:09.814+0000 7fa2ce000700 1 --2- 172.21.15.83:0/3325122036 >> [v2:172.21.15.83:6800/820049883,v1:172.21.15.83:6801/820049883] conn(0x7fa2b80395b0 0x7fa2b803ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:38:10.794 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:09.814+0000 7fa2ce000700 1 -- 172.21.15.83:0/3325122036 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa2c404dac0 con 0x7fa2d01083c0 2024-04-24T08:38:10.794 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:09.814+0000 7fa2d5200700 1 --2- 172.21.15.83:0/3325122036 >> [v2:172.21.15.83:6800/820049883,v1:172.21.15.83:6801/820049883] conn(0x7fa2b80395b0 0x7fa2b803ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:38:10.794 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:09.815+0000 7fa2d5200700 1 --2- 172.21.15.83:0/3325122036 >> [v2:172.21.15.83:6800/820049883,v1:172.21.15.83:6801/820049883] conn(0x7fa2b80395b0 0x7fa2b803ba70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc006fd0 tx=0x7fa2bc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:38:10.794 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:09.825+0000 7fa2ce000700 1 -- 172.21.15.83:0/3325122036 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa2c4020240 con 0x7fa2d01083c0 2024-04-24T08:38:10.794 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:10.050+0000 7fa2d8600700 1 -- 172.21.15.83:0/3325122036 --> [v2:172.21.15.83:6800/820049883,v1:172.21.15.83:6801/820049883] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7fa2b0000bc0 con 0x7fa2b80395b0 2024-04-24T08:38:10.795 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:10.052+0000 7fa2ce000700 1 -- 172.21.15.83:0/3325122036 <== mgr.14120 v2:172.21.15.83:6800/820049883 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7fa2b0000bc0 con 0x7fa2b80395b0 2024-04-24T08:38:10.795 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:10.054+0000 7fa2d8600700 1 -- 172.21.15.83:0/3325122036 >> [v2:172.21.15.83:6800/820049883,v1:172.21.15.83:6801/820049883] conn(0x7fa2b80395b0 msgr2=0x7fa2b803ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:38:10.795 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:10.054+0000 7fa2d8600700 1 --2- 172.21.15.83:0/3325122036 >> [v2:172.21.15.83:6800/820049883,v1:172.21.15.83:6801/820049883] conn(0x7fa2b80395b0 0x7fa2b803ba70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc006fd0 tx=0x7fa2bc006e40 comp rx=0 tx=0).stop 2024-04-24T08:38:10.795 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:10.054+0000 7fa2d8600700 1 -- 172.21.15.83:0/3325122036 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2d01083c0 msgr2=0x7fa2d019e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:38:10.795 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:10.054+0000 7fa2d8600700 1 --2- 172.21.15.83:0/3325122036 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2d01083c0 0x7fa2d019e4e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa2c4000c00 tx=0x7fa2c4004290 comp rx=0 tx=0).stop 2024-04-24T08:38:10.795 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:10.054+0000 7fa2d8600700 1 -- 172.21.15.83:0/3325122036 shutdown_connections 2024-04-24T08:38:10.795 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:10.054+0000 7fa2d8600700 1 --2- 172.21.15.83:0/3325122036 >> [v2:172.21.15.83:6800/820049883,v1:172.21.15.83:6801/820049883] conn(0x7fa2b80395b0 0x7fa2b803ba70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:38:10.795 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:10.054+0000 7fa2d8600700 1 --2- 172.21.15.83:0/3325122036 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2d01083c0 0x7fa2d019e4e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:38:10.795 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:10.054+0000 7fa2d8600700 1 -- 172.21.15.83:0/3325122036 >> 172.21.15.83:0/3325122036 conn(0x7fa2d0103920 msgr2=0x7fa2d0104440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:38:10.795 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:10.055+0000 7fa2d8600700 1 -- 172.21.15.83:0/3325122036 shutdown_connections 2024-04-24T08:38:10.795 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:10.055+0000 7fa2d8600700 1 -- 172.21.15.83:0/3325122036 wait complete. 2024-04-24T08:38:10.795 INFO:teuthology.orchestra.run.smithi083.stdout:Generating ssh key... 2024-04-24T08:38:11.381 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:38:11 smithi083 ceph-mon[27907]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:38:12.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:38:12 smithi083 ceph-mon[27907]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:38:12.614 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:38:12 smithi083 ceph-mon[27907]: Generating ssh key... 2024-04-24T08:38:12.615 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:38:12 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: Generating public/private rsa key pair. 2024-04-24T08:38:12.615 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:38:12 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: Your identification has been saved in /tmp/tmpezcb1k_p/key. 2024-04-24T08:38:12.615 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:38:12 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: Your public key has been saved in /tmp/tmpezcb1k_p/key.pub. 2024-04-24T08:38:12.615 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:38:12 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: The key fingerprint is: 2024-04-24T08:38:12.615 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:38:12 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: SHA256:8CnMVH/STVFZqUD9uEvRQDgxRHdM3f2uUfnD2/fWoqM ceph-bd3e039e-0215-11ef-bc93-c7b262605968 2024-04-24T08:38:12.615 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:38:12 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: The key's randomart image is: 2024-04-24T08:38:12.616 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:38:12 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: +---[RSA 3072]----+ 2024-04-24T08:38:12.616 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:38:12 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: | . +*=o==X| 2024-04-24T08:38:12.616 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:38:12 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: | . . =o=.++| 2024-04-24T08:38:12.616 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:38:12 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: | o o = B o| 2024-04-24T08:38:12.616 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:38:12 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: | + o . o + +o| 2024-04-24T08:38:12.616 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:38:12 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: | + S +o.| 2024-04-24T08:38:12.616 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:38:12 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: | . o.oo| 2024-04-24T08:38:12.616 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:38:12 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: | . .o=| 2024-04-24T08:38:12.616 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:38:12 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: | ..o.=| 2024-04-24T08:38:12.616 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:38:12 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: | E..o o+| 2024-04-24T08:38:12.616 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:38:12 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: +----[SHA256]-----+ 2024-04-24T08:38:12.985 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:11.458+0000 7f8f99400700 1 Processor -- start 2024-04-24T08:38:12.985 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:11.459+0000 7f8f99400700 1 -- start start 2024-04-24T08:38:12.986 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:11.459+0000 7f8f99400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f9410ef20 0x7f8f94111310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:38:12.986 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:11.459+0000 7f8f99400700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8f941118e0 con 0x7f8f9410ef20 2024-04-24T08:38:12.986 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:11.459+0000 7f8f93e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f9410ef20 0x7f8f94111310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:38:12.986 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:11.460+0000 7f8f93e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f9410ef20 0x7f8f94111310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58894/0 (socket says 172.21.15.83:58894) 2024-04-24T08:38:12.986 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:11.460+0000 7f8f93e00700 1 -- 172.21.15.83:0/2269450543 learned_addr learned my addr 172.21.15.83:0/2269450543 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:38:12.986 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:11.460+0000 7f8f93e00700 1 -- 172.21.15.83:0/2269450543 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f94111a60 con 0x7f8f9410ef20 2024-04-24T08:38:12.986 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:11.460+0000 7f8f93e00700 1 --2- 172.21.15.83:0/2269450543 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f9410ef20 0x7f8f94111310 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8f840098a0 tx=0x7f8f84009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1fb072a673d958a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:38:12.986 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:11.461+0000 7f8f92a00700 1 -- 172.21.15.83:0/2269450543 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8f84004160 con 0x7f8f9410ef20 2024-04-24T08:38:12.986 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:11.461+0000 7f8f92a00700 1 -- 172.21.15.83:0/2269450543 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f8f84004300 con 0x7f8f9410ef20 2024-04-24T08:38:12.986 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:11.461+0000 7f8f92a00700 1 -- 172.21.15.83:0/2269450543 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8f84011a80 con 0x7f8f9410ef20 2024-04-24T08:38:12.986 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:11.461+0000 7f8f99400700 1 -- 172.21.15.83:0/2269450543 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f9410ef20 msgr2=0x7f8f94111310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:38:12.986 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:11.461+0000 7f8f99400700 1 --2- 172.21.15.83:0/2269450543 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f9410ef20 0x7f8f94111310 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8f840098a0 tx=0x7f8f84009bb0 comp rx=0 tx=0).stop 2024-04-24T08:38:12.987 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:11.461+0000 7f8f99400700 1 -- 172.21.15.83:0/2269450543 shutdown_connections 2024-04-24T08:38:12.987 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:11.461+0000 7f8f99400700 1 --2- 172.21.15.83:0/2269450543 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f9410ef20 0x7f8f94111310 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:38:12.987 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:11.462+0000 7f8f99400700 1 -- 172.21.15.83:0/2269450543 >> 172.21.15.83:0/2269450543 conn(0x7f8f9407d8d0 msgr2=0x7f8f9407fcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:38:12.987 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:11.462+0000 7f8f99400700 1 -- 172.21.15.83:0/2269450543 shutdown_connections 2024-04-24T08:38:12.987 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:11.462+0000 7f8f99400700 1 -- 172.21.15.83:0/2269450543 wait complete. 2024-04-24T08:38:12.987 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:11.463+0000 7f8f99400700 1 Processor -- start 2024-04-24T08:38:12.987 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:11.463+0000 7f8f99400700 1 -- start start 2024-04-24T08:38:12.987 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:11.463+0000 7f8f99400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f9410ef20 0x7f8f941a2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:38:12.987 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:11.463+0000 7f8f99400700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8f94112a90 con 0x7f8f9410ef20 2024-04-24T08:38:12.987 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:11.463+0000 7f8f93e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f9410ef20 0x7f8f941a2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:38:12.987 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:11.463+0000 7f8f93e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f9410ef20 0x7f8f941a2610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58898/0 (socket says 172.21.15.83:58898) 2024-04-24T08:38:12.987 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:11.463+0000 7f8f93e00700 1 -- 172.21.15.83:0/3087114199 learned_addr learned my addr 172.21.15.83:0/3087114199 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:38:12.988 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:11.464+0000 7f8f93e00700 1 -- 172.21.15.83:0/3087114199 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f941a2b50 con 0x7f8f9410ef20 2024-04-24T08:38:12.988 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:11.464+0000 7f8f93e00700 1 --2- 172.21.15.83:0/3087114199 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f9410ef20 0x7f8f941a2610 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8f84011a50 tx=0x7f8f84011b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:38:12.988 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:11.464+0000 7f8f90c00700 1 -- 172.21.15.83:0/3087114199 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8f84003a80 con 0x7f8f9410ef20 2024-04-24T08:38:12.988 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:11.464+0000 7f8f90c00700 1 -- 172.21.15.83:0/3087114199 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f8f84003c20 con 0x7f8f9410ef20 2024-04-24T08:38:12.988 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:11.464+0000 7f8f99400700 1 -- 172.21.15.83:0/3087114199 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f941a2de0 con 0x7f8f9410ef20 2024-04-24T08:38:12.988 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:11.464+0000 7f8f90c00700 1 -- 172.21.15.83:0/3087114199 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8f8401a650 con 0x7f8f9410ef20 2024-04-24T08:38:12.988 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:11.465+0000 7f8f99400700 1 -- 172.21.15.83:0/3087114199 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f941a5ad0 con 0x7f8f9410ef20 2024-04-24T08:38:12.988 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:11.466+0000 7f8f90c00700 1 -- 172.21.15.83:0/3087114199 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f8f84024ad0 con 0x7f8f9410ef20 2024-04-24T08:38:12.988 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:11.466+0000 7f8f99400700 1 -- 172.21.15.83:0/3087114199 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f94064150 con 0x7f8f9410ef20 2024-04-24T08:38:12.988 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:11.466+0000 7f8f90c00700 1 --2- 172.21.15.83:0/3087114199 >> [v2:172.21.15.83:6800/820049883,v1:172.21.15.83:6801/820049883] conn(0x7f8f78039600 0x7f8f7803bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:38:12.988 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:11.466+0000 7f8f90c00700 1 -- 172.21.15.83:0/3087114199 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f8f8404dc50 con 0x7f8f9410ef20 2024-04-24T08:38:12.988 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:11.466+0000 7f8f93400700 1 --2- 172.21.15.83:0/3087114199 >> [v2:172.21.15.83:6800/820049883,v1:172.21.15.83:6801/820049883] conn(0x7f8f78039600 0x7f8f7803bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:38:12.988 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:11.467+0000 7f8f93400700 1 --2- 172.21.15.83:0/3087114199 >> [v2:172.21.15.83:6800/820049883,v1:172.21.15.83:6801/820049883] conn(0x7f8f78039600 0x7f8f7803bac0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8f80006fd0 tx=0x7f8f80006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:38:12.988 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:11.471+0000 7f8f90c00700 1 -- 172.21.15.83:0/3087114199 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8f8400f2e0 con 0x7f8f9410ef20 2024-04-24T08:38:12.989 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:11.700+0000 7f8f99400700 1 -- 172.21.15.83:0/3087114199 --> [v2:172.21.15.83:6800/820049883,v1:172.21.15.83:6801/820049883] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f8f9407fac0 con 0x7f8f78039600 2024-04-24T08:38:12.989 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:12.250+0000 7f8f90c00700 1 -- 172.21.15.83:0/3087114199 <== mgr.14120 v2:172.21.15.83:6800/820049883 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8f9407fac0 con 0x7f8f78039600 2024-04-24T08:38:12.989 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:12.253+0000 7f8f99400700 1 -- 172.21.15.83:0/3087114199 >> [v2:172.21.15.83:6800/820049883,v1:172.21.15.83:6801/820049883] conn(0x7f8f78039600 msgr2=0x7f8f7803bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:38:12.989 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:12.253+0000 7f8f99400700 1 --2- 172.21.15.83:0/3087114199 >> [v2:172.21.15.83:6800/820049883,v1:172.21.15.83:6801/820049883] conn(0x7f8f78039600 0x7f8f7803bac0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8f80006fd0 tx=0x7f8f80006e40 comp rx=0 tx=0).stop 2024-04-24T08:38:12.989 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:12.253+0000 7f8f99400700 1 -- 172.21.15.83:0/3087114199 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f9410ef20 msgr2=0x7f8f941a2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:38:12.989 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:12.253+0000 7f8f99400700 1 --2- 172.21.15.83:0/3087114199 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f9410ef20 0x7f8f941a2610 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8f84011a50 tx=0x7f8f84011b30 comp rx=0 tx=0).stop 2024-04-24T08:38:12.989 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:12.253+0000 7f8f99400700 1 -- 172.21.15.83:0/3087114199 shutdown_connections 2024-04-24T08:38:12.989 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:12.253+0000 7f8f99400700 1 --2- 172.21.15.83:0/3087114199 >> [v2:172.21.15.83:6800/820049883,v1:172.21.15.83:6801/820049883] conn(0x7f8f78039600 0x7f8f7803bac0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:38:12.989 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:12.253+0000 7f8f99400700 1 --2- 172.21.15.83:0/3087114199 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f9410ef20 0x7f8f941a2610 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:38:12.989 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:12.253+0000 7f8f99400700 1 -- 172.21.15.83:0/3087114199 >> 172.21.15.83:0/3087114199 conn(0x7f8f9407d8d0 msgr2=0x7f8f9407f3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:38:12.989 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:12.254+0000 7f8f99400700 1 -- 172.21.15.83:0/3087114199 shutdown_connections 2024-04-24T08:38:12.989 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:12.254+0000 7f8f99400700 1 -- 172.21.15.83:0/3087114199 wait complete. 2024-04-24T08:38:13.521 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:38:13 smithi083 ceph-mon[27907]: from='mgr.14120 172.21.15.83:0/3660207153' entity='mgr.x' 2024-04-24T08:38:13.521 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:38:13 smithi083 ceph-mon[27907]: from='mgr.14120 172.21.15.83:0/3660207153' entity='mgr.x' 2024-04-24T08:38:14.541 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:38:14 smithi083 ceph-mon[27907]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:38:14.550 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDADFsq18W8JTifbXybABKcXFLWR3x3ActmbY2eMvg0Vlu+3KM7De9oVuGJzL+RNfLtUqh7n0kYWRUJ7hNs11wySoNCaDxyPmtUix5yphqVoEFRLBO88SsHMwLdMvWvHf7rHDQMkfThLzzqAF74JC7+hQ+j9U45gPzMCED0PnPSs8Dfigz0EkhhWG6IPDNJvWvQoyDkPkc1suGLsqVRNF6Coc8nfWU6q5GzOqV1198IA+dy/MRUAJ23vnGb9ps3Qwn/Al7ey7o2IgFoB3sqcFu7DWs4y2AAinh1Wv0XbSdgt67jNFZhoSc2pZdR2Ea7Ilr4qn1nzXNZJ4TUFSfUqinmpZGdC6nk63bly0iLmDivqOIpFrFGbMlo1JJOvLuA1XpkPyc0+MUFeVSFIoohLuEyts9xKZc3ByS3kwgf8Ym9WZbKnu2UklnpIv+KRDNCYi6udhLQ629s828wm6IMXCPIX+k78Lj2rdarmVEOcS/5M+ufM8TPHRGl61Y94KLD4ds= ceph-bd3e039e-0215-11ef-bc93-c7b262605968 2024-04-24T08:38:14.556 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:13.589+0000 7fb275000700 1 Processor -- start 2024-04-24T08:38:14.556 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:13.589+0000 7fb275000700 1 -- start start 2024-04-24T08:38:14.556 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:13.590+0000 7fb275000700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb27007de20 0x7fb27007c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:38:14.556 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:13.590+0000 7fb275000700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb27007c810 con 0x7fb27007de20 2024-04-24T08:38:14.556 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:13.590+0000 7fb26de00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb27007de20 0x7fb27007c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:38:14.556 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:13.590+0000 7fb26de00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb27007de20 0x7fb27007c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58912/0 (socket says 172.21.15.83:58912) 2024-04-24T08:38:14.556 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:13.590+0000 7fb26de00700 1 -- 172.21.15.83:0/543217734 learned_addr learned my addr 172.21.15.83:0/543217734 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:38:14.556 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:13.591+0000 7fb26de00700 1 -- 172.21.15.83:0/543217734 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb27007c990 con 0x7fb27007de20 2024-04-24T08:38:14.556 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:13.591+0000 7fb26de00700 1 --2- 172.21.15.83:0/543217734 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb27007de20 0x7fb27007c2d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb2600098a0 tx=0x7fb260009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbb5b1495c413d31 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:38:14.556 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:13.591+0000 7fb26ca00700 1 -- 172.21.15.83:0/543217734 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb260004160 con 0x7fb27007de20 2024-04-24T08:38:14.556 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:13.591+0000 7fb26ca00700 1 -- 172.21.15.83:0/543217734 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fb260004300 con 0x7fb27007de20 2024-04-24T08:38:14.556 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:13.591+0000 7fb26ca00700 1 -- 172.21.15.83:0/543217734 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb260011a80 con 0x7fb27007de20 2024-04-24T08:38:14.557 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:13.592+0000 7fb275000700 1 -- 172.21.15.83:0/543217734 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb27007de20 msgr2=0x7fb27007c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:38:14.557 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:13.592+0000 7fb275000700 1 --2- 172.21.15.83:0/543217734 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb27007de20 0x7fb27007c2d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb2600098a0 tx=0x7fb260009bb0 comp rx=0 tx=0).stop 2024-04-24T08:38:14.557 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:13.592+0000 7fb275000700 1 -- 172.21.15.83:0/543217734 shutdown_connections 2024-04-24T08:38:14.557 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:13.592+0000 7fb275000700 1 --2- 172.21.15.83:0/543217734 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb27007de20 0x7fb27007c2d0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:38:14.557 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:13.592+0000 7fb275000700 1 -- 172.21.15.83:0/543217734 >> 172.21.15.83:0/543217734 conn(0x7fb270103920 msgr2=0x7fb270105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:38:14.557 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:13.592+0000 7fb275000700 1 -- 172.21.15.83:0/543217734 shutdown_connections 2024-04-24T08:38:14.557 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:13.592+0000 7fb275000700 1 -- 172.21.15.83:0/543217734 wait complete. 2024-04-24T08:38:14.557 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:13.593+0000 7fb275000700 1 Processor -- start 2024-04-24T08:38:14.557 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:13.593+0000 7fb275000700 1 -- start start 2024-04-24T08:38:14.557 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:13.593+0000 7fb275000700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb27007de20 0x7fb27019e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:38:14.557 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:13.593+0000 7fb275000700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb27010e8d0 con 0x7fb27007de20 2024-04-24T08:38:14.557 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:13.594+0000 7fb26de00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb27007de20 0x7fb27019e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:38:14.557 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:13.594+0000 7fb26de00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb27007de20 0x7fb27019e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58920/0 (socket says 172.21.15.83:58920) 2024-04-24T08:38:14.557 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:13.594+0000 7fb26de00700 1 -- 172.21.15.83:0/3568201831 learned_addr learned my addr 172.21.15.83:0/3568201831 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:38:14.558 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:13.594+0000 7fb26de00700 1 -- 172.21.15.83:0/3568201831 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb27019e9c0 con 0x7fb27007de20 2024-04-24T08:38:14.558 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:13.594+0000 7fb26de00700 1 --2- 172.21.15.83:0/3568201831 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb27007de20 0x7fb27019e480 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb260011a50 tx=0x7fb260004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:38:14.558 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:13.595+0000 7fb266a00700 1 -- 172.21.15.83:0/3568201831 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb2600039f0 con 0x7fb27007de20 2024-04-24T08:38:14.558 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:13.595+0000 7fb266a00700 1 -- 172.21.15.83:0/3568201831 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fb260003b90 con 0x7fb27007de20 2024-04-24T08:38:14.558 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:13.595+0000 7fb266a00700 1 -- 172.21.15.83:0/3568201831 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb26001a600 con 0x7fb27007de20 2024-04-24T08:38:14.558 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:13.595+0000 7fb275000700 1 -- 172.21.15.83:0/3568201831 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb27019ec50 con 0x7fb27007de20 2024-04-24T08:38:14.558 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:13.595+0000 7fb275000700 1 -- 172.21.15.83:0/3568201831 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2701a1940 con 0x7fb27007de20 2024-04-24T08:38:14.558 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:13.596+0000 7fb275000700 1 -- 172.21.15.83:0/3568201831 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb270064150 con 0x7fb27007de20 2024-04-24T08:38:14.558 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:13.596+0000 7fb266a00700 1 -- 172.21.15.83:0/3568201831 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fb260024a40 con 0x7fb27007de20 2024-04-24T08:38:14.558 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:13.596+0000 7fb266a00700 1 --2- 172.21.15.83:0/3568201831 >> [v2:172.21.15.83:6800/820049883,v1:172.21.15.83:6801/820049883] conn(0x7fb254039600 0x7fb25403bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:38:14.558 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:13.597+0000 7fb266a00700 1 -- 172.21.15.83:0/3568201831 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fb26004dbe0 con 0x7fb27007de20 2024-04-24T08:38:14.558 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:13.597+0000 7fb26d400700 1 --2- 172.21.15.83:0/3568201831 >> [v2:172.21.15.83:6800/820049883,v1:172.21.15.83:6801/820049883] conn(0x7fb254039600 0x7fb25403bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:38:14.558 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:13.597+0000 7fb26d400700 1 --2- 172.21.15.83:0/3568201831 >> [v2:172.21.15.83:6800/820049883,v1:172.21.15.83:6801/820049883] conn(0x7fb254039600 0x7fb25403bac0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb25c006fd0 tx=0x7fb25c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:38:14.559 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:13.602+0000 7fb266a00700 1 -- 172.21.15.83:0/3568201831 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb26004d530 con 0x7fb27007de20 2024-04-24T08:38:14.559 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:13.826+0000 7fb275000700 1 -- 172.21.15.83:0/3568201831 --> [v2:172.21.15.83:6800/820049883,v1:172.21.15.83:6801/820049883] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7fb270104b20 con 0x7fb254039600 2024-04-24T08:38:14.559 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:13.827+0000 7fb266a00700 1 -- 172.21.15.83:0/3568201831 <== mgr.14120 v2:172.21.15.83:6800/820049883 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7fb270104b20 con 0x7fb254039600 2024-04-24T08:38:14.559 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:13.830+0000 7fb275000700 1 -- 172.21.15.83:0/3568201831 >> [v2:172.21.15.83:6800/820049883,v1:172.21.15.83:6801/820049883] conn(0x7fb254039600 msgr2=0x7fb25403bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:38:14.559 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:13.830+0000 7fb275000700 1 --2- 172.21.15.83:0/3568201831 >> [v2:172.21.15.83:6800/820049883,v1:172.21.15.83:6801/820049883] conn(0x7fb254039600 0x7fb25403bac0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb25c006fd0 tx=0x7fb25c006e40 comp rx=0 tx=0).stop 2024-04-24T08:38:14.559 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:13.830+0000 7fb275000700 1 -- 172.21.15.83:0/3568201831 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb27007de20 msgr2=0x7fb27019e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:38:14.559 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:13.830+0000 7fb275000700 1 --2- 172.21.15.83:0/3568201831 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb27007de20 0x7fb27019e480 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb260011a50 tx=0x7fb260004290 comp rx=0 tx=0).stop 2024-04-24T08:38:14.559 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:13.830+0000 7fb275000700 1 -- 172.21.15.83:0/3568201831 shutdown_connections 2024-04-24T08:38:14.559 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:13.830+0000 7fb275000700 1 --2- 172.21.15.83:0/3568201831 >> [v2:172.21.15.83:6800/820049883,v1:172.21.15.83:6801/820049883] conn(0x7fb254039600 0x7fb25403bac0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:38:14.559 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:13.830+0000 7fb275000700 1 --2- 172.21.15.83:0/3568201831 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb27007de20 0x7fb27019e480 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:38:14.559 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:13.830+0000 7fb275000700 1 -- 172.21.15.83:0/3568201831 >> 172.21.15.83:0/3568201831 conn(0x7fb270103920 msgr2=0x7fb270104410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:38:14.559 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:13.830+0000 7fb275000700 1 -- 172.21.15.83:0/3568201831 shutdown_connections 2024-04-24T08:38:14.559 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:13.830+0000 7fb275000700 1 -- 172.21.15.83:0/3568201831 wait complete. 2024-04-24T08:38:14.559 INFO:teuthology.orchestra.run.smithi083.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-04-24T08:38:14.560 INFO:teuthology.orchestra.run.smithi083.stdout:Adding key to root@localhost authorized_keys... 2024-04-24T08:38:14.561 INFO:teuthology.orchestra.run.smithi083.stdout:Adding host smithi083... 2024-04-24T08:38:16.366 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:38:16 smithi083 ceph-mon[27907]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi083", "addr": "172.21.15.83", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:38:17.220 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:38:17 smithi083 ceph-mon[27907]: Deploying cephadm binary to smithi083 2024-04-24T08:38:22.037 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout Added host 'smithi083' with addr '172.21.15.83' 2024-04-24T08:38:22.037 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:15.202+0000 7fc6ba600700 1 Processor -- start 2024-04-24T08:38:22.037 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:15.202+0000 7fc6ba600700 1 -- start start 2024-04-24T08:38:22.037 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:15.203+0000 7fc6ba600700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6b4108140 0x7fc6b4108560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:38:22.037 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:15.203+0000 7fc6ba600700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc6b4108b30 con 0x7fc6b4108140 2024-04-24T08:38:22.037 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:15.203+0000 7fc6b9200700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6b4108140 0x7fc6b4108560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:38:22.037 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:15.203+0000 7fc6b9200700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6b4108140 0x7fc6b4108560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58932/0 (socket says 172.21.15.83:58932) 2024-04-24T08:38:22.037 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:15.203+0000 7fc6b9200700 1 -- 172.21.15.83:0/205159105 learned_addr learned my addr 172.21.15.83:0/205159105 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:38:22.038 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:15.204+0000 7fc6b9200700 1 -- 172.21.15.83:0/205159105 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6b41092f0 con 0x7fc6b4108140 2024-04-24T08:38:22.038 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:15.204+0000 7fc6b9200700 1 --2- 172.21.15.83:0/205159105 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6b4108140 0x7fc6b4108560 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc6a80098a0 tx=0x7fc6a8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4931b16e1629e579 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:38:22.038 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:15.204+0000 7fc6b3400700 1 -- 172.21.15.83:0/205159105 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc6a8004160 con 0x7fc6b4108140 2024-04-24T08:38:22.038 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:15.204+0000 7fc6b3400700 1 -- 172.21.15.83:0/205159105 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fc6a8004300 con 0x7fc6b4108140 2024-04-24T08:38:22.038 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:15.204+0000 7fc6b3400700 1 -- 172.21.15.83:0/205159105 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc6a8011a80 con 0x7fc6b4108140 2024-04-24T08:38:22.038 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:15.205+0000 7fc6ba600700 1 -- 172.21.15.83:0/205159105 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6b4108140 msgr2=0x7fc6b4108560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:38:22.038 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:15.205+0000 7fc6ba600700 1 --2- 172.21.15.83:0/205159105 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6b4108140 0x7fc6b4108560 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc6a80098a0 tx=0x7fc6a8009bb0 comp rx=0 tx=0).stop 2024-04-24T08:38:22.038 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:15.205+0000 7fc6ba600700 1 -- 172.21.15.83:0/205159105 shutdown_connections 2024-04-24T08:38:22.038 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:15.205+0000 7fc6ba600700 1 --2- 172.21.15.83:0/205159105 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6b4108140 0x7fc6b4108560 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:38:22.038 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:15.205+0000 7fc6ba600700 1 -- 172.21.15.83:0/205159105 >> 172.21.15.83:0/205159105 conn(0x7fc6b4103720 msgr2=0x7fc6b4105b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:38:22.038 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:15.205+0000 7fc6ba600700 1 -- 172.21.15.83:0/205159105 shutdown_connections 2024-04-24T08:38:22.038 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:15.205+0000 7fc6ba600700 1 -- 172.21.15.83:0/205159105 wait complete. 2024-04-24T08:38:22.038 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:15.206+0000 7fc6ba600700 1 Processor -- start 2024-04-24T08:38:22.038 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:15.206+0000 7fc6ba600700 1 -- start start 2024-04-24T08:38:22.039 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:15.207+0000 7fc6ba600700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6b4108140 0x7fc6b407db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:38:22.039 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:15.207+0000 7fc6ba600700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc6b410c6a0 con 0x7fc6b4108140 2024-04-24T08:38:22.039 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:15.207+0000 7fc6b9200700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6b4108140 0x7fc6b407db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:38:22.039 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:15.207+0000 7fc6b9200700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6b4108140 0x7fc6b407db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58942/0 (socket says 172.21.15.83:58942) 2024-04-24T08:38:22.039 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:15.207+0000 7fc6b9200700 1 -- 172.21.15.83:0/4035474676 learned_addr learned my addr 172.21.15.83:0/4035474676 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:38:22.039 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:15.208+0000 7fc6b9200700 1 -- 172.21.15.83:0/4035474676 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6b407e0a0 con 0x7fc6b4108140 2024-04-24T08:38:22.039 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:15.208+0000 7fc6b9200700 1 --2- 172.21.15.83:0/4035474676 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6b4108140 0x7fc6b407db60 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc6a8006af0 tx=0x7fc6a8011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:38:22.039 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:15.208+0000 7fc6b1600700 1 -- 172.21.15.83:0/4035474676 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc6a80039e0 con 0x7fc6b4108140 2024-04-24T08:38:22.039 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:15.208+0000 7fc6b1600700 1 -- 172.21.15.83:0/4035474676 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fc6a8003b80 con 0x7fc6b4108140 2024-04-24T08:38:22.039 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:15.208+0000 7fc6ba600700 1 -- 172.21.15.83:0/4035474676 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6b407c290 con 0x7fc6b4108140 2024-04-24T08:38:22.039 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:15.208+0000 7fc6b1600700 1 -- 172.21.15.83:0/4035474676 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc6a80196a0 con 0x7fc6b4108140 2024-04-24T08:38:22.039 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:15.208+0000 7fc6ba600700 1 -- 172.21.15.83:0/4035474676 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6b407c7d0 con 0x7fc6b4108140 2024-04-24T08:38:22.039 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:15.209+0000 7fc6b1600700 1 -- 172.21.15.83:0/4035474676 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fc6a8019950 con 0x7fc6b4108140 2024-04-24T08:38:22.040 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:15.210+0000 7fc6ba600700 1 -- 172.21.15.83:0/4035474676 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6b4064150 con 0x7fc6b4108140 2024-04-24T08:38:22.040 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:15.210+0000 7fc6b1600700 1 --2- 172.21.15.83:0/4035474676 >> [v2:172.21.15.83:6800/820049883,v1:172.21.15.83:6801/820049883] conn(0x7fc69c039610 0x7fc69c03bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:38:22.040 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:15.210+0000 7fc6b1600700 1 -- 172.21.15.83:0/4035474676 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fc6a804dcb0 con 0x7fc6b4108140 2024-04-24T08:38:22.040 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:15.210+0000 7fc6b3e00700 1 --2- 172.21.15.83:0/4035474676 >> [v2:172.21.15.83:6800/820049883,v1:172.21.15.83:6801/820049883] conn(0x7fc69c039610 0x7fc69c03bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:38:22.040 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:15.211+0000 7fc6b3e00700 1 --2- 172.21.15.83:0/4035474676 >> [v2:172.21.15.83:6800/820049883,v1:172.21.15.83:6801/820049883] conn(0x7fc69c039610 0x7fc69c03bad0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc6a4006fd0 tx=0x7fc6a4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:38:22.040 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:15.216+0000 7fc6b1600700 1 -- 172.21.15.83:0/4035474676 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc6a8023e00 con 0x7fc6b4108140 2024-04-24T08:38:22.040 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:15.439+0000 7fc6ba600700 1 -- 172.21.15.83:0/4035474676 --> [v2:172.21.15.83:6800/820049883,v1:172.21.15.83:6801/820049883] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi083", "addr": "172.21.15.83", "target": ["mon-mgr", ""]}) -- 0x7fc6b4104990 con 0x7fc69c039610 2024-04-24T08:38:22.040 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:21.303+0000 7fc6b1600700 1 -- 172.21.15.83:0/4035474676 <== mgr.14120 v2:172.21.15.83:6800/820049883 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7fc6b4104990 con 0x7fc69c039610 2024-04-24T08:38:22.040 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:21.307+0000 7fc6ba600700 1 -- 172.21.15.83:0/4035474676 >> [v2:172.21.15.83:6800/820049883,v1:172.21.15.83:6801/820049883] conn(0x7fc69c039610 msgr2=0x7fc69c03bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:38:22.040 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:21.307+0000 7fc6ba600700 1 --2- 172.21.15.83:0/4035474676 >> [v2:172.21.15.83:6800/820049883,v1:172.21.15.83:6801/820049883] conn(0x7fc69c039610 0x7fc69c03bad0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc6a4006fd0 tx=0x7fc6a4006e40 comp rx=0 tx=0).stop 2024-04-24T08:38:22.040 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:21.308+0000 7fc6ba600700 1 -- 172.21.15.83:0/4035474676 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6b4108140 msgr2=0x7fc6b407db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:38:22.040 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:21.308+0000 7fc6ba600700 1 --2- 172.21.15.83:0/4035474676 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6b4108140 0x7fc6b407db60 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc6a8006af0 tx=0x7fc6a8011c40 comp rx=0 tx=0).stop 2024-04-24T08:38:22.040 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:21.308+0000 7fc6ba600700 1 -- 172.21.15.83:0/4035474676 shutdown_connections 2024-04-24T08:38:22.041 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:21.308+0000 7fc6ba600700 1 --2- 172.21.15.83:0/4035474676 >> [v2:172.21.15.83:6800/820049883,v1:172.21.15.83:6801/820049883] conn(0x7fc69c039610 0x7fc69c03bad0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:38:22.041 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:21.308+0000 7fc6ba600700 1 --2- 172.21.15.83:0/4035474676 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6b4108140 0x7fc6b407db60 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:38:22.041 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:21.308+0000 7fc6ba600700 1 -- 172.21.15.83:0/4035474676 >> 172.21.15.83:0/4035474676 conn(0x7fc6b4103720 msgr2=0x7fc6b4104280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:38:22.041 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:21.308+0000 7fc6ba600700 1 -- 172.21.15.83:0/4035474676 shutdown_connections 2024-04-24T08:38:22.041 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:21.308+0000 7fc6ba600700 1 -- 172.21.15.83:0/4035474676 wait complete. 2024-04-24T08:38:22.041 INFO:teuthology.orchestra.run.smithi083.stdout:Deploying unmanaged mon service... 2024-04-24T08:38:22.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:38:22 smithi083 ceph-mon[27907]: from='mgr.14120 172.21.15.83:0/3660207153' entity='mgr.x' 2024-04-24T08:38:22.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:38:22 smithi083 ceph-mon[27907]: Added host smithi083 2024-04-24T08:38:22.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:38:22 smithi083 ceph-mon[27907]: from='mgr.14120 172.21.15.83:0/3660207153' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:38:24.339 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-04-24T08:38:24.339 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:22.790+0000 7f77dc200700 1 Processor -- start 2024-04-24T08:38:24.339 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:38:24 smithi083 ceph-mon[27907]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:38:24.339 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:38:24 smithi083 ceph-mon[27907]: Saving service mon spec with placement count:5 2024-04-24T08:38:24.339 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:38:24 smithi083 ceph-mon[27907]: from='mgr.14120 172.21.15.83:0/3660207153' entity='mgr.x' 2024-04-24T08:38:24.347 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:22.790+0000 7f77dc200700 1 -- start start 2024-04-24T08:38:24.347 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:22.791+0000 7f77dc200700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77d4108dd0 0x7f77d410b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:38:24.347 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:22.791+0000 7f77dc200700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f77d407be60 con 0x7f77d4108dd0 2024-04-24T08:38:24.347 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:22.791+0000 7f77d9800700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77d4108dd0 0x7f77d410b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:38:24.347 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:22.791+0000 7f77d9800700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77d4108dd0 0x7f77d410b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35566/0 (socket says 172.21.15.83:35566) 2024-04-24T08:38:24.347 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:22.791+0000 7f77d9800700 1 -- 172.21.15.83:0/4128645237 learned_addr learned my addr 172.21.15.83:0/4128645237 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:38:24.348 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:22.791+0000 7f77d9800700 1 -- 172.21.15.83:0/4128645237 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77d410b700 con 0x7f77d4108dd0 2024-04-24T08:38:24.348 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:22.792+0000 7f77d9800700 1 --2- 172.21.15.83:0/4128645237 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77d4108dd0 0x7f77d410b1c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f77c80098a0 tx=0x7f77c8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdc59b4ff99cab91 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:38:24.348 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:22.792+0000 7f77d3e00700 1 -- 172.21.15.83:0/4128645237 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f77c8004160 con 0x7f77d4108dd0 2024-04-24T08:38:24.348 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:22.792+0000 7f77d3e00700 1 -- 172.21.15.83:0/4128645237 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f77c8004300 con 0x7f77d4108dd0 2024-04-24T08:38:24.348 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:22.792+0000 7f77d3e00700 1 -- 172.21.15.83:0/4128645237 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f77c8011a80 con 0x7f77d4108dd0 2024-04-24T08:38:24.348 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:22.793+0000 7f77dc200700 1 -- 172.21.15.83:0/4128645237 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77d4108dd0 msgr2=0x7f77d410b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:38:24.348 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:22.793+0000 7f77dc200700 1 --2- 172.21.15.83:0/4128645237 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77d4108dd0 0x7f77d410b1c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f77c80098a0 tx=0x7f77c8009bb0 comp rx=0 tx=0).stop 2024-04-24T08:38:24.348 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:22.793+0000 7f77dc200700 1 -- 172.21.15.83:0/4128645237 shutdown_connections 2024-04-24T08:38:24.348 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:22.793+0000 7f77dc200700 1 --2- 172.21.15.83:0/4128645237 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77d4108dd0 0x7f77d410b1c0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:38:24.349 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:22.793+0000 7f77dc200700 1 -- 172.21.15.83:0/4128645237 >> 172.21.15.83:0/4128645237 conn(0x7f77d41027c0 msgr2=0x7f77d4104c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:38:24.349 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:22.793+0000 7f77dc200700 1 -- 172.21.15.83:0/4128645237 shutdown_connections 2024-04-24T08:38:24.349 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:22.793+0000 7f77dc200700 1 -- 172.21.15.83:0/4128645237 wait complete. 2024-04-24T08:38:24.349 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:22.795+0000 7f77dc200700 1 Processor -- start 2024-04-24T08:38:24.349 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:22.795+0000 7f77dc200700 1 -- start start 2024-04-24T08:38:24.349 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:22.795+0000 7f77dc200700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77d4108dd0 0x7f77d419e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:38:24.349 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:22.795+0000 7f77dc200700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f77d410c6c0 con 0x7f77d4108dd0 2024-04-24T08:38:24.349 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:22.796+0000 7f77d9800700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77d4108dd0 0x7f77d419e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:38:24.349 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:22.796+0000 7f77d9800700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77d4108dd0 0x7f77d419e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35570/0 (socket says 172.21.15.83:35570) 2024-04-24T08:38:24.350 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:22.796+0000 7f77d9800700 1 -- 172.21.15.83:0/2573317826 learned_addr learned my addr 172.21.15.83:0/2573317826 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:38:24.350 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:22.796+0000 7f77d9800700 1 -- 172.21.15.83:0/2573317826 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77d419e9a0 con 0x7f77d4108dd0 2024-04-24T08:38:24.350 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:22.796+0000 7f77d9800700 1 --2- 172.21.15.83:0/2573317826 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77d4108dd0 0x7f77d419e460 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f77c8011a50 tx=0x7f77c8004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:38:24.350 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:22.796+0000 7f77d2000700 1 -- 172.21.15.83:0/2573317826 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f77c80039f0 con 0x7f77d4108dd0 2024-04-24T08:38:24.350 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:22.797+0000 7f77d2000700 1 -- 172.21.15.83:0/2573317826 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f77c8003b90 con 0x7f77d4108dd0 2024-04-24T08:38:24.350 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:22.797+0000 7f77dc200700 1 -- 172.21.15.83:0/2573317826 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77d419ec30 con 0x7f77d4108dd0 2024-04-24T08:38:24.350 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:22.797+0000 7f77d2000700 1 -- 172.21.15.83:0/2573317826 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f77c801a600 con 0x7f77d4108dd0 2024-04-24T08:38:24.350 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:22.797+0000 7f77dc200700 1 -- 172.21.15.83:0/2573317826 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77d41a1920 con 0x7f77d4108dd0 2024-04-24T08:38:24.350 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:22.798+0000 7f77d2000700 1 -- 172.21.15.83:0/2573317826 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f77c8024a40 con 0x7f77d4108dd0 2024-04-24T08:38:24.351 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:22.798+0000 7f77dc200700 1 -- 172.21.15.83:0/2573317826 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77d4064150 con 0x7f77d4108dd0 2024-04-24T08:38:24.351 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:22.798+0000 7f77d2000700 1 --2- 172.21.15.83:0/2573317826 >> [v2:172.21.15.83:6800/820049883,v1:172.21.15.83:6801/820049883] conn(0x7f77bc039600 0x7f77bc03bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:38:24.351 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:22.798+0000 7f77d2000700 1 -- 172.21.15.83:0/2573317826 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f77c804daf0 con 0x7f77d4108dd0 2024-04-24T08:38:24.351 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:22.799+0000 7f77d8e00700 1 --2- 172.21.15.83:0/2573317826 >> [v2:172.21.15.83:6800/820049883,v1:172.21.15.83:6801/820049883] conn(0x7f77bc039600 0x7f77bc03bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:38:24.351 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:22.799+0000 7f77d8e00700 1 --2- 172.21.15.83:0/2573317826 >> [v2:172.21.15.83:6800/820049883,v1:172.21.15.83:6801/820049883] conn(0x7f77bc039600 0x7f77bc03bac0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f77c0006fd0 tx=0x7f77c0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:38:24.351 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:22.804+0000 7f77d2000700 1 -- 172.21.15.83:0/2573317826 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f77c800f2e0 con 0x7f77d4108dd0 2024-04-24T08:38:24.351 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:23.074+0000 7f77dc200700 1 -- 172.21.15.83:0/2573317826 --> [v2:172.21.15.83:6800/820049883,v1:172.21.15.83:6801/820049883] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f77d41039f0 con 0x7f77bc039600 2024-04-24T08:38:24.351 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:23.084+0000 7f77d2000700 1 -- 172.21.15.83:0/2573317826 <== mgr.14120 v2:172.21.15.83:6800/820049883 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f77d41039f0 con 0x7f77bc039600 2024-04-24T08:38:24.352 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:23.087+0000 7f77dc200700 1 -- 172.21.15.83:0/2573317826 >> [v2:172.21.15.83:6800/820049883,v1:172.21.15.83:6801/820049883] conn(0x7f77bc039600 msgr2=0x7f77bc03bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:38:24.352 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:23.087+0000 7f77dc200700 1 --2- 172.21.15.83:0/2573317826 >> [v2:172.21.15.83:6800/820049883,v1:172.21.15.83:6801/820049883] conn(0x7f77bc039600 0x7f77bc03bac0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f77c0006fd0 tx=0x7f77c0006e40 comp rx=0 tx=0).stop 2024-04-24T08:38:24.352 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:23.087+0000 7f77dc200700 1 -- 172.21.15.83:0/2573317826 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77d4108dd0 msgr2=0x7f77d419e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:38:24.352 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:23.087+0000 7f77dc200700 1 --2- 172.21.15.83:0/2573317826 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77d4108dd0 0x7f77d419e460 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f77c8011a50 tx=0x7f77c8004290 comp rx=0 tx=0).stop 2024-04-24T08:38:24.352 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:23.087+0000 7f77dc200700 1 -- 172.21.15.83:0/2573317826 shutdown_connections 2024-04-24T08:38:24.352 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:23.087+0000 7f77dc200700 1 --2- 172.21.15.83:0/2573317826 >> [v2:172.21.15.83:6800/820049883,v1:172.21.15.83:6801/820049883] conn(0x7f77bc039600 0x7f77bc03bac0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:38:24.352 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:23.087+0000 7f77dc200700 1 --2- 172.21.15.83:0/2573317826 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77d4108dd0 0x7f77d419e460 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:38:24.352 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:23.087+0000 7f77dc200700 1 -- 172.21.15.83:0/2573317826 >> 172.21.15.83:0/2573317826 conn(0x7f77d41027c0 msgr2=0x7f77d41032e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:38:24.352 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:23.088+0000 7f77dc200700 1 -- 172.21.15.83:0/2573317826 shutdown_connections 2024-04-24T08:38:24.353 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:23.088+0000 7f77dc200700 1 -- 172.21.15.83:0/2573317826 wait complete. 2024-04-24T08:38:24.353 INFO:teuthology.orchestra.run.smithi083.stdout:Deploying unmanaged mgr service... 2024-04-24T08:38:26.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:38:25 smithi083 ceph-mon[27907]: from='mgr.14120 172.21.15.83:0/3660207153' entity='mgr.x' 2024-04-24T08:38:26.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:38:25 smithi083 ceph-mon[27907]: from='mgr.14120 172.21.15.83:0/3660207153' entity='mgr.x' 2024-04-24T08:38:26.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:38:25 smithi083 ceph-mon[27907]: from='mgr.14120 172.21.15.83:0/3660207153' entity='mgr.x' 2024-04-24T08:38:26.519 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-04-24T08:38:26.519 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:25.497+0000 7f61b1e00700 1 Processor -- start 2024-04-24T08:38:26.520 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:25.497+0000 7f61b1e00700 1 -- start start 2024-04-24T08:38:26.520 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:25.497+0000 7f61b1e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61ac07de00 0x7f61ac07c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:38:26.520 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:25.497+0000 7f61b1e00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f61ac07c7f0 con 0x7f61ac07de00 2024-04-24T08:38:26.520 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:25.498+0000 7f61aaa00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61ac07de00 0x7f61ac07c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:38:26.520 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:25.498+0000 7f61aaa00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61ac07de00 0x7f61ac07c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35576/0 (socket says 172.21.15.83:35576) 2024-04-24T08:38:26.520 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:25.498+0000 7f61aaa00700 1 -- 172.21.15.83:0/1361836873 learned_addr learned my addr 172.21.15.83:0/1361836873 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:38:26.524 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:25.498+0000 7f61aaa00700 1 -- 172.21.15.83:0/1361836873 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61ac07c970 con 0x7f61ac07de00 2024-04-24T08:38:26.524 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:25.499+0000 7f61aaa00700 1 --2- 172.21.15.83:0/1361836873 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61ac07de00 0x7f61ac07c2b0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f619c0098a0 tx=0x7f619c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc144fedc2c9b981 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:38:26.524 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:25.499+0000 7f61a9600700 1 -- 172.21.15.83:0/1361836873 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f619c004160 con 0x7f61ac07de00 2024-04-24T08:38:26.524 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:25.499+0000 7f61a9600700 1 -- 172.21.15.83:0/1361836873 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f619c004300 con 0x7f61ac07de00 2024-04-24T08:38:26.524 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:25.499+0000 7f61a9600700 1 -- 172.21.15.83:0/1361836873 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f619c011a50 con 0x7f61ac07de00 2024-04-24T08:38:26.524 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:25.500+0000 7f61b1e00700 1 -- 172.21.15.83:0/1361836873 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61ac07de00 msgr2=0x7f61ac07c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:38:26.524 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:25.500+0000 7f61b1e00700 1 --2- 172.21.15.83:0/1361836873 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61ac07de00 0x7f61ac07c2b0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f619c0098a0 tx=0x7f619c009bb0 comp rx=0 tx=0).stop 2024-04-24T08:38:26.525 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:25.500+0000 7f61b1e00700 1 -- 172.21.15.83:0/1361836873 shutdown_connections 2024-04-24T08:38:26.525 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:25.500+0000 7f61b1e00700 1 --2- 172.21.15.83:0/1361836873 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61ac07de00 0x7f61ac07c2b0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:38:26.525 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:25.500+0000 7f61b1e00700 1 -- 172.21.15.83:0/1361836873 >> 172.21.15.83:0/1361836873 conn(0x7f61ac103920 msgr2=0x7f61ac105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:38:26.525 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:25.500+0000 7f61b1e00700 1 -- 172.21.15.83:0/1361836873 shutdown_connections 2024-04-24T08:38:26.525 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:25.500+0000 7f61b1e00700 1 -- 172.21.15.83:0/1361836873 wait complete. 2024-04-24T08:38:26.525 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:25.501+0000 7f61b1e00700 1 Processor -- start 2024-04-24T08:38:26.525 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:25.501+0000 7f61b1e00700 1 -- start start 2024-04-24T08:38:26.525 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:25.502+0000 7f61b1e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61ac07de00 0x7f61ac19e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:38:26.525 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:25.502+0000 7f61b1e00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f61ac10e8e0 con 0x7f61ac07de00 2024-04-24T08:38:26.525 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:25.502+0000 7f61aaa00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61ac07de00 0x7f61ac19e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:38:26.525 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:25.502+0000 7f61aaa00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61ac07de00 0x7f61ac19e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35578/0 (socket says 172.21.15.83:35578) 2024-04-24T08:38:26.525 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:25.502+0000 7f61aaa00700 1 -- 172.21.15.83:0/3809201204 learned_addr learned my addr 172.21.15.83:0/3809201204 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:38:26.525 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:25.502+0000 7f61aaa00700 1 -- 172.21.15.83:0/3809201204 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61ac19e940 con 0x7f61ac07de00 2024-04-24T08:38:26.525 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:25.503+0000 7f61aaa00700 1 --2- 172.21.15.83:0/3809201204 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61ac07de00 0x7f61ac19e400 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f619c011a20 tx=0x7f619c011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:38:26.526 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:25.503+0000 7f61b0a00700 1 -- 172.21.15.83:0/3809201204 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f619c0039c0 con 0x7f61ac07de00 2024-04-24T08:38:26.526 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:25.503+0000 7f61b0a00700 1 -- 172.21.15.83:0/3809201204 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f619c003b60 con 0x7f61ac07de00 2024-04-24T08:38:26.526 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:25.503+0000 7f61b0a00700 1 -- 172.21.15.83:0/3809201204 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f619c01a6b0 con 0x7f61ac07de00 2024-04-24T08:38:26.526 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:25.503+0000 7f61b1e00700 1 -- 172.21.15.83:0/3809201204 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61ac19ebd0 con 0x7f61ac07de00 2024-04-24T08:38:26.526 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:25.503+0000 7f61b1e00700 1 -- 172.21.15.83:0/3809201204 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61ac1a18c0 con 0x7f61ac07de00 2024-04-24T08:38:26.526 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:25.505+0000 7f61b0a00700 1 -- 172.21.15.83:0/3809201204 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f619c024430 con 0x7f61ac07de00 2024-04-24T08:38:26.526 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:25.505+0000 7f61b1e00700 1 -- 172.21.15.83:0/3809201204 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61ac064150 con 0x7f61ac07de00 2024-04-24T08:38:26.526 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:25.505+0000 7f61b0a00700 1 --2- 172.21.15.83:0/3809201204 >> [v2:172.21.15.83:6800/820049883,v1:172.21.15.83:6801/820049883] conn(0x7f61900395b0 0x7f619003ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:38:26.526 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:25.505+0000 7f61b0a00700 1 -- 172.21.15.83:0/3809201204 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f619c04dd30 con 0x7f61ac07de00 2024-04-24T08:38:26.526 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:25.505+0000 7f61aa000700 1 --2- 172.21.15.83:0/3809201204 >> [v2:172.21.15.83:6800/820049883,v1:172.21.15.83:6801/820049883] conn(0x7f61900395b0 0x7f619003ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:38:26.526 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:25.506+0000 7f61aa000700 1 --2- 172.21.15.83:0/3809201204 >> [v2:172.21.15.83:6800/820049883,v1:172.21.15.83:6801/820049883] conn(0x7f61900395b0 0x7f619003ba70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f6198006fd0 tx=0x7f6198006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:38:26.526 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:25.511+0000 7f61b0a00700 1 -- 172.21.15.83:0/3809201204 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f619c00f2e0 con 0x7f61ac07de00 2024-04-24T08:38:26.526 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:25.753+0000 7f61b1e00700 1 -- 172.21.15.83:0/3809201204 --> [v2:172.21.15.83:6800/820049883,v1:172.21.15.83:6801/820049883] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f61ac104ad0 con 0x7f61900395b0 2024-04-24T08:38:26.526 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:25.762+0000 7f61b0a00700 1 -- 172.21.15.83:0/3809201204 <== mgr.14120 v2:172.21.15.83:6800/820049883 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f61ac104ad0 con 0x7f61900395b0 2024-04-24T08:38:26.528 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:25.765+0000 7f61b1e00700 1 -- 172.21.15.83:0/3809201204 >> [v2:172.21.15.83:6800/820049883,v1:172.21.15.83:6801/820049883] conn(0x7f61900395b0 msgr2=0x7f619003ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:38:26.528 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:25.765+0000 7f61b1e00700 1 --2- 172.21.15.83:0/3809201204 >> [v2:172.21.15.83:6800/820049883,v1:172.21.15.83:6801/820049883] conn(0x7f61900395b0 0x7f619003ba70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f6198006fd0 tx=0x7f6198006e40 comp rx=0 tx=0).stop 2024-04-24T08:38:26.528 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:25.765+0000 7f61b1e00700 1 -- 172.21.15.83:0/3809201204 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61ac07de00 msgr2=0x7f61ac19e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:38:26.528 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:25.765+0000 7f61b1e00700 1 --2- 172.21.15.83:0/3809201204 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61ac07de00 0x7f61ac19e400 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f619c011a20 tx=0x7f619c011b00 comp rx=0 tx=0).stop 2024-04-24T08:38:26.528 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:25.765+0000 7f61b1e00700 1 -- 172.21.15.83:0/3809201204 shutdown_connections 2024-04-24T08:38:26.528 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:25.765+0000 7f61b1e00700 1 --2- 172.21.15.83:0/3809201204 >> [v2:172.21.15.83:6800/820049883,v1:172.21.15.83:6801/820049883] conn(0x7f61900395b0 0x7f619003ba70 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:38:26.528 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:25.765+0000 7f61b1e00700 1 --2- 172.21.15.83:0/3809201204 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61ac07de00 0x7f61ac19e400 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:38:26.528 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:25.766+0000 7f61b1e00700 1 -- 172.21.15.83:0/3809201204 >> 172.21.15.83:0/3809201204 conn(0x7f61ac103920 msgr2=0x7f61ac1043c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:38:26.528 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:25.766+0000 7f61b1e00700 1 -- 172.21.15.83:0/3809201204 shutdown_connections 2024-04-24T08:38:26.528 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:25.766+0000 7f61b1e00700 1 -- 172.21.15.83:0/3809201204 wait complete. 2024-04-24T08:38:27.149 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:38:26 smithi083 ceph-mon[27907]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:38:27.149 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:38:26 smithi083 ceph-mon[27907]: Saving service mgr spec with placement count:2 2024-04-24T08:38:27.149 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:38:26 smithi083 ceph-mon[27907]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:38:28.691 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:38:28 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/3253763367' entity='client.admin' 2024-04-24T08:38:28.691 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:38:28 smithi083 ceph-mon[27907]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:38:28.706 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:27.148+0000 7f92ac800700 1 Processor -- start 2024-04-24T08:38:28.707 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:27.148+0000 7f92ac800700 1 -- start start 2024-04-24T08:38:28.707 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:27.149+0000 7f92ac800700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92a41083a0 0x7f92a41087c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:38:28.707 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:27.149+0000 7f92ac800700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f92a4108d90 con 0x7f92a41083a0 2024-04-24T08:38:28.707 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:27.149+0000 7f92a9e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92a41083a0 0x7f92a41087c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:38:28.707 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:27.149+0000 7f92a9e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92a41083a0 0x7f92a41087c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35594/0 (socket says 172.21.15.83:35594) 2024-04-24T08:38:28.707 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:27.149+0000 7f92a9e00700 1 -- 172.21.15.83:0/3107379773 learned_addr learned my addr 172.21.15.83:0/3107379773 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:38:28.707 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:27.150+0000 7f92a9e00700 1 -- 172.21.15.83:0/3107379773 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92a4109500 con 0x7f92a41083a0 2024-04-24T08:38:28.707 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:27.150+0000 7f92a9e00700 1 --2- 172.21.15.83:0/3107379773 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92a41083a0 0x7f92a41087c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f92980098a0 tx=0x7f9298009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf5ea0cfdda85153 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:38:28.707 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:27.150+0000 7f92a8a00700 1 -- 172.21.15.83:0/3107379773 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9298004160 con 0x7f92a41083a0 2024-04-24T08:38:28.707 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:27.151+0000 7f92a8a00700 1 -- 172.21.15.83:0/3107379773 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9298004300 con 0x7f92a41083a0 2024-04-24T08:38:28.707 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:27.151+0000 7f92a8a00700 1 -- 172.21.15.83:0/3107379773 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9298011a50 con 0x7f92a41083a0 2024-04-24T08:38:28.707 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:27.152+0000 7f92ac800700 1 -- 172.21.15.83:0/3107379773 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92a41083a0 msgr2=0x7f92a41087c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:38:28.707 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:27.152+0000 7f92ac800700 1 --2- 172.21.15.83:0/3107379773 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92a41083a0 0x7f92a41087c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f92980098a0 tx=0x7f9298009bb0 comp rx=0 tx=0).stop 2024-04-24T08:38:28.708 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:27.152+0000 7f92ac800700 1 -- 172.21.15.83:0/3107379773 shutdown_connections 2024-04-24T08:38:28.708 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:27.152+0000 7f92ac800700 1 --2- 172.21.15.83:0/3107379773 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92a41083a0 0x7f92a41087c0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:38:28.708 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:27.152+0000 7f92ac800700 1 -- 172.21.15.83:0/3107379773 >> 172.21.15.83:0/3107379773 conn(0x7f92a4103960 msgr2=0x7f92a4105d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:38:28.708 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:27.152+0000 7f92ac800700 1 -- 172.21.15.83:0/3107379773 shutdown_connections 2024-04-24T08:38:28.708 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:27.152+0000 7f92ac800700 1 -- 172.21.15.83:0/3107379773 wait complete. 2024-04-24T08:38:28.708 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:27.153+0000 7f92ac800700 1 Processor -- start 2024-04-24T08:38:28.708 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:27.153+0000 7f92ac800700 1 -- start start 2024-04-24T08:38:28.708 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:27.153+0000 7f92ac800700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92a41083a0 0x7f92a419e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:38:28.708 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:27.153+0000 7f92ac800700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f92a410c8b0 con 0x7f92a41083a0 2024-04-24T08:38:28.708 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:27.153+0000 7f92a9e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92a41083a0 0x7f92a419e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:38:28.708 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:27.154+0000 7f92a9e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92a41083a0 0x7f92a419e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35600/0 (socket says 172.21.15.83:35600) 2024-04-24T08:38:28.708 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:27.154+0000 7f92a9e00700 1 -- 172.21.15.83:0/3253763367 learned_addr learned my addr 172.21.15.83:0/3253763367 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:38:28.708 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:27.154+0000 7f92a9e00700 1 -- 172.21.15.83:0/3253763367 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92a419e990 con 0x7f92a41083a0 2024-04-24T08:38:28.708 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:27.154+0000 7f92a9e00700 1 --2- 172.21.15.83:0/3253763367 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92a41083a0 0x7f92a419e450 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9298011a20 tx=0x7f9298011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:38:28.709 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:27.155+0000 7f929ea00700 1 -- 172.21.15.83:0/3253763367 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9298003af0 con 0x7f92a41083a0 2024-04-24T08:38:28.709 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:27.155+0000 7f929ea00700 1 -- 172.21.15.83:0/3253763367 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f929801a8e0 con 0x7f92a41083a0 2024-04-24T08:38:28.709 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:27.155+0000 7f92ac800700 1 -- 172.21.15.83:0/3253763367 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92a419ec20 con 0x7f92a41083a0 2024-04-24T08:38:28.709 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:27.155+0000 7f929ea00700 1 -- 172.21.15.83:0/3253763367 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9298024860 con 0x7f92a41083a0 2024-04-24T08:38:28.709 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:27.155+0000 7f92ac800700 1 -- 172.21.15.83:0/3253763367 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92a41a1910 con 0x7f92a41083a0 2024-04-24T08:38:28.709 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:27.156+0000 7f929ea00700 1 -- 172.21.15.83:0/3253763367 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f9298022020 con 0x7f92a41083a0 2024-04-24T08:38:28.709 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:27.156+0000 7f92ac800700 1 -- 172.21.15.83:0/3253763367 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92a4064150 con 0x7f92a41083a0 2024-04-24T08:38:28.709 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:27.157+0000 7f929ea00700 1 --2- 172.21.15.83:0/3253763367 >> [v2:172.21.15.83:6800/820049883,v1:172.21.15.83:6801/820049883] conn(0x7f928c039600 0x7f928c03bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:38:28.709 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:27.157+0000 7f929ea00700 1 -- 172.21.15.83:0/3253763367 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f929804da40 con 0x7f92a41083a0 2024-04-24T08:38:28.709 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:27.157+0000 7f92a9400700 1 --2- 172.21.15.83:0/3253763367 >> [v2:172.21.15.83:6800/820049883,v1:172.21.15.83:6801/820049883] conn(0x7f928c039600 0x7f928c03bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:38:28.709 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:27.158+0000 7f92a9400700 1 --2- 172.21.15.83:0/3253763367 >> [v2:172.21.15.83:6800/820049883,v1:172.21.15.83:6801/820049883] conn(0x7f928c039600 0x7f928c03bac0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9294006fd0 tx=0x7f9294006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:38:28.709 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:27.162+0000 7f929ea00700 1 -- 172.21.15.83:0/3253763367 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f929804d390 con 0x7f92a41083a0 2024-04-24T08:38:28.709 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:27.392+0000 7f92ac800700 1 -- 172.21.15.83:0/3253763367 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f92a41a1ec0 con 0x7f92a41083a0 2024-04-24T08:38:28.710 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:27.396+0000 7f929ea00700 1 -- 172.21.15.83:0/3253763367 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f929801ab40 con 0x7f92a41083a0 2024-04-24T08:38:28.710 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:27.398+0000 7f92ac800700 1 -- 172.21.15.83:0/3253763367 >> [v2:172.21.15.83:6800/820049883,v1:172.21.15.83:6801/820049883] conn(0x7f928c039600 msgr2=0x7f928c03bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:38:28.710 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:27.398+0000 7f92ac800700 1 --2- 172.21.15.83:0/3253763367 >> [v2:172.21.15.83:6800/820049883,v1:172.21.15.83:6801/820049883] conn(0x7f928c039600 0x7f928c03bac0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9294006fd0 tx=0x7f9294006e40 comp rx=0 tx=0).stop 2024-04-24T08:38:28.710 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:27.398+0000 7f92ac800700 1 -- 172.21.15.83:0/3253763367 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92a41083a0 msgr2=0x7f92a419e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:38:28.710 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:27.398+0000 7f92ac800700 1 --2- 172.21.15.83:0/3253763367 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92a41083a0 0x7f92a419e450 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9298011a20 tx=0x7f9298011b00 comp rx=0 tx=0).stop 2024-04-24T08:38:28.710 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:27.399+0000 7f92ac800700 1 -- 172.21.15.83:0/3253763367 shutdown_connections 2024-04-24T08:38:28.710 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:27.399+0000 7f92ac800700 1 --2- 172.21.15.83:0/3253763367 >> [v2:172.21.15.83:6800/820049883,v1:172.21.15.83:6801/820049883] conn(0x7f928c039600 0x7f928c03bac0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:38:28.710 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:27.399+0000 7f92ac800700 1 --2- 172.21.15.83:0/3253763367 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92a41083a0 0x7f92a419e450 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:38:28.710 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:27.399+0000 7f92ac800700 1 -- 172.21.15.83:0/3253763367 >> 172.21.15.83:0/3253763367 conn(0x7f92a4103960 msgr2=0x7f92a4104450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:38:28.710 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:27.399+0000 7f92ac800700 1 -- 172.21.15.83:0/3253763367 shutdown_connections 2024-04-24T08:38:28.710 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:27.399+0000 7f92ac800700 1 -- 172.21.15.83:0/3253763367 wait complete. 2024-04-24T08:38:29.551 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:38:29 smithi083 ceph-mon[27907]: from='mgr.14120 172.21.15.83:0/3660207153' entity='mgr.x' 2024-04-24T08:38:30.286 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:29.309+0000 7fe9f8a00700 1 Processor -- start 2024-04-24T08:38:30.286 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:29.309+0000 7fe9f8a00700 1 -- start start 2024-04-24T08:38:30.286 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:29.309+0000 7fe9f8a00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9f410ef20 0x7fe9f4111310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:38:30.286 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:29.309+0000 7fe9f8a00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe9f41118e0 con 0x7fe9f410ef20 2024-04-24T08:38:30.286 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:29.310+0000 7fe9f2800700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9f410ef20 0x7fe9f4111310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:38:30.286 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:29.310+0000 7fe9f2800700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9f410ef20 0x7fe9f4111310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49674/0 (socket says 172.21.15.83:49674) 2024-04-24T08:38:30.286 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:29.310+0000 7fe9f2800700 1 -- 172.21.15.83:0/1967786190 learned_addr learned my addr 172.21.15.83:0/1967786190 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:38:30.287 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:29.311+0000 7fe9f2800700 1 -- 172.21.15.83:0/1967786190 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9f4111a60 con 0x7fe9f410ef20 2024-04-24T08:38:30.287 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:29.311+0000 7fe9f2800700 1 --2- 172.21.15.83:0/1967786190 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9f410ef20 0x7fe9f4111310 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc0098a0 tx=0x7fe9dc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ed75fbfc0415f33 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:38:30.287 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:29.311+0000 7fe9f1400700 1 -- 172.21.15.83:0/1967786190 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe9dc004160 con 0x7fe9f410ef20 2024-04-24T08:38:30.287 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:29.311+0000 7fe9f1400700 1 -- 172.21.15.83:0/1967786190 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe9dc004300 con 0x7fe9f410ef20 2024-04-24T08:38:30.287 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:29.311+0000 7fe9f1400700 1 -- 172.21.15.83:0/1967786190 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe9dc011a50 con 0x7fe9f410ef20 2024-04-24T08:38:30.287 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:29.312+0000 7fe9f8a00700 1 -- 172.21.15.83:0/1967786190 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9f410ef20 msgr2=0x7fe9f4111310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:38:30.287 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:29.312+0000 7fe9f8a00700 1 --2- 172.21.15.83:0/1967786190 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9f410ef20 0x7fe9f4111310 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc0098a0 tx=0x7fe9dc009bb0 comp rx=0 tx=0).stop 2024-04-24T08:38:30.287 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:29.312+0000 7fe9f8a00700 1 -- 172.21.15.83:0/1967786190 shutdown_connections 2024-04-24T08:38:30.287 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:29.312+0000 7fe9f8a00700 1 --2- 172.21.15.83:0/1967786190 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9f410ef20 0x7fe9f4111310 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:38:30.287 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:29.312+0000 7fe9f8a00700 1 -- 172.21.15.83:0/1967786190 >> 172.21.15.83:0/1967786190 conn(0x7fe9f407d8d0 msgr2=0x7fe9f407fcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:38:30.287 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:29.312+0000 7fe9f8a00700 1 -- 172.21.15.83:0/1967786190 shutdown_connections 2024-04-24T08:38:30.287 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:29.312+0000 7fe9f8a00700 1 -- 172.21.15.83:0/1967786190 wait complete. 2024-04-24T08:38:30.287 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:29.313+0000 7fe9f8a00700 1 Processor -- start 2024-04-24T08:38:30.287 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:29.313+0000 7fe9f8a00700 1 -- start start 2024-04-24T08:38:30.287 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:29.314+0000 7fe9f8a00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9f410ef20 0x7fe9f41a25c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:38:30.288 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:29.314+0000 7fe9f8a00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe9f4112ad0 con 0x7fe9f410ef20 2024-04-24T08:38:30.288 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:29.314+0000 7fe9f2800700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9f410ef20 0x7fe9f41a25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:38:30.288 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:29.314+0000 7fe9f2800700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9f410ef20 0x7fe9f41a25c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49682/0 (socket says 172.21.15.83:49682) 2024-04-24T08:38:30.288 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:29.314+0000 7fe9f2800700 1 -- 172.21.15.83:0/3328242432 learned_addr learned my addr 172.21.15.83:0/3328242432 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:38:30.288 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:29.314+0000 7fe9f2800700 1 -- 172.21.15.83:0/3328242432 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9f41a2b00 con 0x7fe9f410ef20 2024-04-24T08:38:30.288 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:29.315+0000 7fe9f2800700 1 --2- 172.21.15.83:0/3328242432 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9f410ef20 0x7fe9f41a25c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc006b20 tx=0x7fe9dc011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:38:30.288 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:29.315+0000 7fe9eb400700 1 -- 172.21.15.83:0/3328242432 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe9dc003b70 con 0x7fe9f410ef20 2024-04-24T08:38:30.288 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:29.315+0000 7fe9eb400700 1 -- 172.21.15.83:0/3328242432 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe9dc01a8e0 con 0x7fe9f410ef20 2024-04-24T08:38:30.288 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:29.315+0000 7fe9eb400700 1 -- 172.21.15.83:0/3328242432 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe9dc0249b0 con 0x7fe9f410ef20 2024-04-24T08:38:30.288 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:29.315+0000 7fe9f8a00700 1 -- 172.21.15.83:0/3328242432 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9f41a2d90 con 0x7fe9f410ef20 2024-04-24T08:38:30.288 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:29.315+0000 7fe9f8a00700 1 -- 172.21.15.83:0/3328242432 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9f41a5a80 con 0x7fe9f410ef20 2024-04-24T08:38:30.288 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:29.316+0000 7fe9eb400700 1 -- 172.21.15.83:0/3328242432 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fe9dc022020 con 0x7fe9f410ef20 2024-04-24T08:38:30.288 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:29.316+0000 7fe9f8a00700 1 -- 172.21.15.83:0/3328242432 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9f4064150 con 0x7fe9f410ef20 2024-04-24T08:38:30.289 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:29.317+0000 7fe9eb400700 1 --2- 172.21.15.83:0/3328242432 >> [v2:172.21.15.83:6800/820049883,v1:172.21.15.83:6801/820049883] conn(0x7fe9d8039600 0x7fe9d803bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:38:30.289 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:29.317+0000 7fe9eb400700 1 -- 172.21.15.83:0/3328242432 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fe9dc04de30 con 0x7fe9f410ef20 2024-04-24T08:38:30.289 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:29.317+0000 7fe9f1e00700 1 --2- 172.21.15.83:0/3328242432 >> [v2:172.21.15.83:6800/820049883,v1:172.21.15.83:6801/820049883] conn(0x7fe9d8039600 0x7fe9d803bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:38:30.289 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:29.318+0000 7fe9f1e00700 1 --2- 172.21.15.83:0/3328242432 >> [v2:172.21.15.83:6800/820049883,v1:172.21.15.83:6801/820049883] conn(0x7fe9d8039600 0x7fe9d803bac0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fe9e4006fd0 tx=0x7fe9e4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:38:30.289 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:29.323+0000 7fe9eb400700 1 -- 172.21.15.83:0/3328242432 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe9dc02c3a0 con 0x7fe9f410ef20 2024-04-24T08:38:30.289 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:29.550+0000 7fe9f8a00700 1 -- 172.21.15.83:0/3328242432 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7fe9f41a60a0 con 0x7fe9f410ef20 2024-04-24T08:38:30.289 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:29.556+0000 7fe9eb400700 1 -- 172.21.15.83:0/3328242432 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7fe9dc017ca0 con 0x7fe9f410ef20 2024-04-24T08:38:30.289 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:29.559+0000 7fe9f8a00700 1 -- 172.21.15.83:0/3328242432 >> [v2:172.21.15.83:6800/820049883,v1:172.21.15.83:6801/820049883] conn(0x7fe9d8039600 msgr2=0x7fe9d803bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:38:30.289 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:29.559+0000 7fe9f8a00700 1 --2- 172.21.15.83:0/3328242432 >> [v2:172.21.15.83:6800/820049883,v1:172.21.15.83:6801/820049883] conn(0x7fe9d8039600 0x7fe9d803bac0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fe9e4006fd0 tx=0x7fe9e4006e40 comp rx=0 tx=0).stop 2024-04-24T08:38:30.289 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:29.559+0000 7fe9f8a00700 1 -- 172.21.15.83:0/3328242432 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9f410ef20 msgr2=0x7fe9f41a25c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:38:30.289 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:29.559+0000 7fe9f8a00700 1 --2- 172.21.15.83:0/3328242432 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9f410ef20 0x7fe9f41a25c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc006b20 tx=0x7fe9dc011ad0 comp rx=0 tx=0).stop 2024-04-24T08:38:30.289 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:29.559+0000 7fe9f8a00700 1 -- 172.21.15.83:0/3328242432 shutdown_connections 2024-04-24T08:38:30.289 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:29.559+0000 7fe9f8a00700 1 --2- 172.21.15.83:0/3328242432 >> [v2:172.21.15.83:6800/820049883,v1:172.21.15.83:6801/820049883] conn(0x7fe9d8039600 0x7fe9d803bac0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:38:30.289 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:29.559+0000 7fe9f8a00700 1 --2- 172.21.15.83:0/3328242432 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9f410ef20 0x7fe9f41a25c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:38:30.289 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:29.559+0000 7fe9f8a00700 1 -- 172.21.15.83:0/3328242432 >> 172.21.15.83:0/3328242432 conn(0x7fe9f407d8d0 msgr2=0x7fe9f407f390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:38:30.290 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:29.559+0000 7fe9f8a00700 1 -- 172.21.15.83:0/3328242432 shutdown_connections 2024-04-24T08:38:30.290 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:29.559+0000 7fe9f8a00700 1 -- 172.21.15.83:0/3328242432 wait complete. 2024-04-24T08:38:30.290 INFO:teuthology.orchestra.run.smithi083.stdout:Enabling the dashboard module... 2024-04-24T08:38:30.891 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:38:30 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/3328242432' entity='client.admin' 2024-04-24T08:38:30.891 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:38:30 smithi083 ceph-mon[27907]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:38:32.048 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:38:31 smithi083 ceph-mon[27907]: from='mgr.14120 172.21.15.83:0/3660207153' entity='mgr.x' 2024-04-24T08:38:32.048 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:38:31 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/1824275426' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-04-24T08:38:32.048 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:38:31 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: ignoring --setuser ceph since I am not root 2024-04-24T08:38:32.049 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:38:31 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: ignoring --setgroup ceph since I am not root 2024-04-24T08:38:32.049 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:38:32 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:38:32.047+0000 7fbb19070200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-24T08:38:32.303 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:38:32 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:38:32.291+0000 7fbb19070200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-24T08:38:32.750 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:38:32 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:38:32.420+0000 7fbb19070200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-24T08:38:33.021 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:38:32 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/1824275426' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-04-24T08:38:33.021 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:38:32 smithi083 ceph-mon[27907]: mgrmap e9: x(active, since 25s) 2024-04-24T08:38:33.021 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:38:32 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:38:32.815+0000 7fbb19070200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-24T08:38:33.021 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:38:32 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:38:32.928+0000 7fbb19070200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-24T08:38:33.065 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:30.926+0000 7f4f24400700 1 Processor -- start 2024-04-24T08:38:33.065 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:30.927+0000 7f4f24400700 1 -- start start 2024-04-24T08:38:33.065 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:30.927+0000 7f4f24400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f1c07de80 0x7f4f1c07c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:38:33.065 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:30.927+0000 7f4f24400700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4f1c07c900 con 0x7f4f1c07de80 2024-04-24T08:38:33.065 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:30.927+0000 7f4f21a00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f1c07de80 0x7f4f1c07c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:38:33.066 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:30.927+0000 7f4f21a00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f1c07de80 0x7f4f1c07c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49688/0 (socket says 172.21.15.83:49688) 2024-04-24T08:38:33.066 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:30.927+0000 7f4f21a00700 1 -- 172.21.15.83:0/3398238624 learned_addr learned my addr 172.21.15.83:0/3398238624 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:38:33.066 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:30.928+0000 7f4f21a00700 1 -- 172.21.15.83:0/3398238624 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f1c07ca80 con 0x7f4f1c07de80 2024-04-24T08:38:33.066 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:30.928+0000 7f4f21a00700 1 --2- 172.21.15.83:0/3398238624 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f1c07de80 0x7f4f1c07c330 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4f100098a0 tx=0x7f4f10009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b601b3c47b7e4c2f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:38:33.066 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:30.928+0000 7f4f1be00700 1 -- 172.21.15.83:0/3398238624 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4f10004160 con 0x7f4f1c07de80 2024-04-24T08:38:33.066 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:30.929+0000 7f4f1be00700 1 -- 172.21.15.83:0/3398238624 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4f10004300 con 0x7f4f1c07de80 2024-04-24T08:38:33.066 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:30.929+0000 7f4f1be00700 1 -- 172.21.15.83:0/3398238624 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4f10011a50 con 0x7f4f1c07de80 2024-04-24T08:38:33.066 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:30.929+0000 7f4f24400700 1 -- 172.21.15.83:0/3398238624 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f1c07de80 msgr2=0x7f4f1c07c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:38:33.066 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:30.929+0000 7f4f24400700 1 --2- 172.21.15.83:0/3398238624 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f1c07de80 0x7f4f1c07c330 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4f100098a0 tx=0x7f4f10009bb0 comp rx=0 tx=0).stop 2024-04-24T08:38:33.066 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:30.929+0000 7f4f24400700 1 -- 172.21.15.83:0/3398238624 shutdown_connections 2024-04-24T08:38:33.066 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:30.929+0000 7f4f24400700 1 --2- 172.21.15.83:0/3398238624 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f1c07de80 0x7f4f1c07c330 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:38:33.066 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:30.929+0000 7f4f24400700 1 -- 172.21.15.83:0/3398238624 >> 172.21.15.83:0/3398238624 conn(0x7f4f1c103910 msgr2=0x7f4f1c105d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:38:33.067 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:30.930+0000 7f4f24400700 1 -- 172.21.15.83:0/3398238624 shutdown_connections 2024-04-24T08:38:33.067 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:30.930+0000 7f4f24400700 1 -- 172.21.15.83:0/3398238624 wait complete. 2024-04-24T08:38:33.067 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:30.931+0000 7f4f24400700 1 Processor -- start 2024-04-24T08:38:33.067 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:30.931+0000 7f4f24400700 1 -- start start 2024-04-24T08:38:33.067 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:30.931+0000 7f4f24400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f1c07de80 0x7f4f1c19e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:38:33.067 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:30.931+0000 7f4f24400700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4f1c10e8d0 con 0x7f4f1c07de80 2024-04-24T08:38:33.067 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:30.931+0000 7f4f21a00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f1c07de80 0x7f4f1c19e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:38:33.067 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:30.931+0000 7f4f21a00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f1c07de80 0x7f4f1c19e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49694/0 (socket says 172.21.15.83:49694) 2024-04-24T08:38:33.067 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:30.931+0000 7f4f21a00700 1 -- 172.21.15.83:0/1824275426 learned_addr learned my addr 172.21.15.83:0/1824275426 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:38:33.067 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:30.932+0000 7f4f21a00700 1 -- 172.21.15.83:0/1824275426 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f1c19e900 con 0x7f4f1c07de80 2024-04-24T08:38:33.067 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:30.932+0000 7f4f21a00700 1 --2- 172.21.15.83:0/1824275426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f1c07de80 0x7f4f1c19e3c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4f10011a00 tx=0x7f4f10011ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:38:33.067 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:30.932+0000 7f4f1a000700 1 -- 172.21.15.83:0/1824275426 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4f10003af0 con 0x7f4f1c07de80 2024-04-24T08:38:33.067 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:30.932+0000 7f4f1a000700 1 -- 172.21.15.83:0/1824275426 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4f1001a8e0 con 0x7f4f1c07de80 2024-04-24T08:38:33.068 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:30.932+0000 7f4f24400700 1 -- 172.21.15.83:0/1824275426 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f1c19eb90 con 0x7f4f1c07de80 2024-04-24T08:38:33.068 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:30.933+0000 7f4f1a000700 1 -- 172.21.15.83:0/1824275426 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4f10024840 con 0x7f4f1c07de80 2024-04-24T08:38:33.068 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:30.933+0000 7f4f24400700 1 -- 172.21.15.83:0/1824275426 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f1c1a1880 con 0x7f4f1c07de80 2024-04-24T08:38:33.068 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:30.934+0000 7f4f1a000700 1 -- 172.21.15.83:0/1824275426 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f4f10022020 con 0x7f4f1c07de80 2024-04-24T08:38:33.068 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:30.934+0000 7f4f24400700 1 -- 172.21.15.83:0/1824275426 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f1c064150 con 0x7f4f1c07de80 2024-04-24T08:38:33.068 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:30.934+0000 7f4f1a000700 1 --2- 172.21.15.83:0/1824275426 >> [v2:172.21.15.83:6800/820049883,v1:172.21.15.83:6801/820049883] conn(0x7f4f04039600 0x7f4f0403bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:38:33.068 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:30.934+0000 7f4f1a000700 1 -- 172.21.15.83:0/1824275426 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f4f1004daa0 con 0x7f4f1c07de80 2024-04-24T08:38:33.068 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:30.935+0000 7f4f21000700 1 --2- 172.21.15.83:0/1824275426 >> [v2:172.21.15.83:6800/820049883,v1:172.21.15.83:6801/820049883] conn(0x7f4f04039600 0x7f4f0403bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:38:33.068 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:30.935+0000 7f4f21000700 1 --2- 172.21.15.83:0/1824275426 >> [v2:172.21.15.83:6800/820049883,v1:172.21.15.83:6801/820049883] conn(0x7f4f04039600 0x7f4f0403bac0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4f08006fd0 tx=0x7f4f08006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:38:33.068 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:30.944+0000 7f4f1a000700 1 -- 172.21.15.83:0/1824275426 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4f10019c70 con 0x7f4f1c07de80 2024-04-24T08:38:33.068 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:31.236+0000 7f4f24400700 1 -- 172.21.15.83:0/1824275426 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f4f1c1a1e30 con 0x7f4f1c07de80 2024-04-24T08:38:33.068 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:31.742+0000 7f4f1a000700 1 -- 172.21.15.83:0/1824275426 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f4f1001adb0 con 0x7f4f1c07de80 2024-04-24T08:38:33.068 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:31.746+0000 7f4f24400700 1 -- 172.21.15.83:0/1824275426 >> [v2:172.21.15.83:6800/820049883,v1:172.21.15.83:6801/820049883] conn(0x7f4f04039600 msgr2=0x7f4f0403bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:38:33.068 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:31.746+0000 7f4f24400700 1 --2- 172.21.15.83:0/1824275426 >> [v2:172.21.15.83:6800/820049883,v1:172.21.15.83:6801/820049883] conn(0x7f4f04039600 0x7f4f0403bac0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4f08006fd0 tx=0x7f4f08006e40 comp rx=0 tx=0).stop 2024-04-24T08:38:33.069 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:31.747+0000 7f4f24400700 1 -- 172.21.15.83:0/1824275426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f1c07de80 msgr2=0x7f4f1c19e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:38:33.069 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:31.747+0000 7f4f24400700 1 --2- 172.21.15.83:0/1824275426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f1c07de80 0x7f4f1c19e3c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4f10011a00 tx=0x7f4f10011ae0 comp rx=0 tx=0).stop 2024-04-24T08:38:33.069 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:31.748+0000 7f4f24400700 1 -- 172.21.15.83:0/1824275426 shutdown_connections 2024-04-24T08:38:33.069 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:31.748+0000 7f4f24400700 1 --2- 172.21.15.83:0/1824275426 >> [v2:172.21.15.83:6800/820049883,v1:172.21.15.83:6801/820049883] conn(0x7f4f04039600 0x7f4f0403bac0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:38:33.069 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:31.748+0000 7f4f24400700 1 --2- 172.21.15.83:0/1824275426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f1c07de80 0x7f4f1c19e3c0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:38:33.069 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:31.748+0000 7f4f24400700 1 -- 172.21.15.83:0/1824275426 >> 172.21.15.83:0/1824275426 conn(0x7f4f1c103910 msgr2=0x7f4f1c1043d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:38:33.069 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:31.749+0000 7f4f24400700 1 -- 172.21.15.83:0/1824275426 shutdown_connections 2024-04-24T08:38:33.069 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:31.749+0000 7f4f24400700 1 -- 172.21.15.83:0/1824275426 wait complete. 2024-04-24T08:38:34.691 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:38:34 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/3592243612' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-24T08:38:34.691 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:38:34 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:38:34.497+0000 7fbb19070200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-24T08:38:35.053 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:38:34 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:38:34.726+0000 7fbb19070200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-24T08:38:35.054 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:38:34 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:38:34.939+0000 7fbb19070200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-24T08:38:35.072 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout { 2024-04-24T08:38:35.073 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-04-24T08:38:35.073 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-24T08:38:35.073 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-24T08:38:35.073 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-24T08:38:35.073 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout } 2024-04-24T08:38:35.074 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:34.036+0000 7f1063e00700 1 Processor -- start 2024-04-24T08:38:35.077 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:34.037+0000 7f1063e00700 1 -- start start 2024-04-24T08:38:35.077 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:34.037+0000 7f1063e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f106407de60 0x7f106407c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:38:35.077 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:34.037+0000 7f1063e00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f106407c8e0 con 0x7f106407de60 2024-04-24T08:38:35.077 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:34.038+0000 7f1062a00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f106407de60 0x7f106407c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:38:35.078 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:34.038+0000 7f1062a00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f106407de60 0x7f106407c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49718/0 (socket says 172.21.15.83:49718) 2024-04-24T08:38:35.078 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:34.038+0000 7f1062a00700 1 -- 172.21.15.83:0/587845956 learned_addr learned my addr 172.21.15.83:0/587845956 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:38:35.078 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:34.038+0000 7f1062a00700 1 -- 172.21.15.83:0/587845956 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f106407ca60 con 0x7f106407de60 2024-04-24T08:38:35.078 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:34.038+0000 7f1062a00700 1 --2- 172.21.15.83:0/587845956 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f106407de60 0x7f106407c310 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f10540098a0 tx=0x7f1054009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3100fcf8abbcfee5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:38:35.078 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:34.039+0000 7f1061600700 1 -- 172.21.15.83:0/587845956 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1054004160 con 0x7f106407de60 2024-04-24T08:38:35.078 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:34.039+0000 7f1061600700 1 -- 172.21.15.83:0/587845956 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1054004300 con 0x7f106407de60 2024-04-24T08:38:35.078 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:34.039+0000 7f1061600700 1 -- 172.21.15.83:0/587845956 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1054011a50 con 0x7f106407de60 2024-04-24T08:38:35.078 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:34.039+0000 7f1063e00700 1 -- 172.21.15.83:0/587845956 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f106407de60 msgr2=0x7f106407c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:38:35.078 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:34.039+0000 7f1063e00700 1 --2- 172.21.15.83:0/587845956 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f106407de60 0x7f106407c310 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f10540098a0 tx=0x7f1054009bb0 comp rx=0 tx=0).stop 2024-04-24T08:38:35.078 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:34.040+0000 7f1063e00700 1 -- 172.21.15.83:0/587845956 shutdown_connections 2024-04-24T08:38:35.078 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:34.040+0000 7f1063e00700 1 --2- 172.21.15.83:0/587845956 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f106407de60 0x7f106407c310 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:38:35.078 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:34.040+0000 7f1063e00700 1 -- 172.21.15.83:0/587845956 >> 172.21.15.83:0/587845956 conn(0x7f1064103740 msgr2=0x7f1064105b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:38:35.078 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:34.040+0000 7f1063e00700 1 -- 172.21.15.83:0/587845956 shutdown_connections 2024-04-24T08:38:35.079 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:34.040+0000 7f1063e00700 1 -- 172.21.15.83:0/587845956 wait complete. 2024-04-24T08:38:35.079 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:34.041+0000 7f1063e00700 1 Processor -- start 2024-04-24T08:38:35.079 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:34.041+0000 7f1063e00700 1 -- start start 2024-04-24T08:38:35.079 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:34.041+0000 7f1063e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f106407de60 0x7f106419e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:38:35.079 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:34.041+0000 7f1063e00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f106410e680 con 0x7f106407de60 2024-04-24T08:38:35.079 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:34.041+0000 7f1062a00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f106407de60 0x7f106419e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:38:35.079 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:34.041+0000 7f1062a00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f106407de60 0x7f106419e230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49720/0 (socket says 172.21.15.83:49720) 2024-04-24T08:38:35.079 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:34.041+0000 7f1062a00700 1 -- 172.21.15.83:0/3592243612 learned_addr learned my addr 172.21.15.83:0/3592243612 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:38:35.079 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:34.042+0000 7f1062a00700 1 -- 172.21.15.83:0/3592243612 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f106419e770 con 0x7f106407de60 2024-04-24T08:38:35.079 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:34.042+0000 7f1062a00700 1 --2- 172.21.15.83:0/3592243612 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f106407de60 0x7f106419e230 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1054011a00 tx=0x7f1054011ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:38:35.079 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:34.042+0000 7f105b400700 1 -- 172.21.15.83:0/3592243612 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1054003af0 con 0x7f106407de60 2024-04-24T08:38:35.079 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:34.042+0000 7f105b400700 1 -- 172.21.15.83:0/3592243612 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f105401a8e0 con 0x7f106407de60 2024-04-24T08:38:35.080 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:34.042+0000 7f105b400700 1 -- 172.21.15.83:0/3592243612 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1054024840 con 0x7f106407de60 2024-04-24T08:38:35.080 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:34.042+0000 7f1063e00700 1 -- 172.21.15.83:0/3592243612 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f106419ea00 con 0x7f106407de60 2024-04-24T08:38:35.080 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:34.042+0000 7f1063e00700 1 -- 172.21.15.83:0/3592243612 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10641a16f0 con 0x7f106407de60 2024-04-24T08:38:35.083 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:34.043+0000 7f1063e00700 1 -- 172.21.15.83:0/3592243612 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1064064150 con 0x7f106407de60 2024-04-24T08:38:35.083 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:34.044+0000 7f105b400700 1 -- 172.21.15.83:0/3592243612 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f1054022020 con 0x7f106407de60 2024-04-24T08:38:35.083 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:34.044+0000 7f105b400700 1 --2- 172.21.15.83:0/3592243612 >> [v2:172.21.15.83:6800/820049883,v1:172.21.15.83:6801/820049883] conn(0x7f1048039600 0x7f104803bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:38:35.083 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:34.044+0000 7f105b400700 1 -- 172.21.15.83:0/3592243612 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f105404db50 con 0x7f106407de60 2024-04-24T08:38:35.083 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:34.044+0000 7f1062000700 1 -- 172.21.15.83:0/3592243612 >> [v2:172.21.15.83:6800/820049883,v1:172.21.15.83:6801/820049883] conn(0x7f1048039600 msgr2=0x7f104803bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.83:6800/820049883 2024-04-24T08:38:35.083 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:34.044+0000 7f1062000700 1 --2- 172.21.15.83:0/3592243612 >> [v2:172.21.15.83:6800/820049883,v1:172.21.15.83:6801/820049883] conn(0x7f1048039600 0x7f104803bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T08:38:35.083 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:34.050+0000 7f105b400700 1 -- 172.21.15.83:0/3592243612 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f105400b3e0 con 0x7f106407de60 2024-04-24T08:38:35.083 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:34.245+0000 7f1062000700 1 -- 172.21.15.83:0/3592243612 >> [v2:172.21.15.83:6800/820049883,v1:172.21.15.83:6801/820049883] conn(0x7f1048039600 msgr2=0x7f104803bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.83:6800/820049883 2024-04-24T08:38:35.083 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:34.245+0000 7f1062000700 1 --2- 172.21.15.83:0/3592243612 >> [v2:172.21.15.83:6800/820049883,v1:172.21.15.83:6801/820049883] conn(0x7f1048039600 0x7f104803bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-24T08:38:35.083 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:34.332+0000 7f1063e00700 1 -- 172.21.15.83:0/3592243612 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f10641a1960 con 0x7f106407de60 2024-04-24T08:38:35.083 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:34.332+0000 7f105b400700 1 -- 172.21.15.83:0/3592243612 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f1054016260 con 0x7f106407de60 2024-04-24T08:38:35.083 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:34.335+0000 7f1063e00700 1 -- 172.21.15.83:0/3592243612 >> [v2:172.21.15.83:6800/820049883,v1:172.21.15.83:6801/820049883] conn(0x7f1048039600 msgr2=0x7f104803bac0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T08:38:35.084 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:34.335+0000 7f1063e00700 1 --2- 172.21.15.83:0/3592243612 >> [v2:172.21.15.83:6800/820049883,v1:172.21.15.83:6801/820049883] conn(0x7f1048039600 0x7f104803bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:38:35.084 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:34.335+0000 7f1063e00700 1 -- 172.21.15.83:0/3592243612 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f106407de60 msgr2=0x7f106419e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:38:35.084 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:34.335+0000 7f1063e00700 1 --2- 172.21.15.83:0/3592243612 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f106407de60 0x7f106419e230 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1054011a00 tx=0x7f1054011ae0 comp rx=0 tx=0).stop 2024-04-24T08:38:35.084 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:34.335+0000 7f1063e00700 1 -- 172.21.15.83:0/3592243612 shutdown_connections 2024-04-24T08:38:35.084 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:34.335+0000 7f1063e00700 1 --2- 172.21.15.83:0/3592243612 >> [v2:172.21.15.83:6800/820049883,v1:172.21.15.83:6801/820049883] conn(0x7f1048039600 0x7f104803bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:38:35.084 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:34.335+0000 7f1063e00700 1 --2- 172.21.15.83:0/3592243612 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f106407de60 0x7f106419e230 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:38:35.084 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:34.335+0000 7f1063e00700 1 -- 172.21.15.83:0/3592243612 >> 172.21.15.83:0/3592243612 conn(0x7f1064103740 msgr2=0x7f10641042a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:38:35.084 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:34.336+0000 7f1063e00700 1 -- 172.21.15.83:0/3592243612 shutdown_connections 2024-04-24T08:38:35.084 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:34.336+0000 7f1063e00700 1 -- 172.21.15.83:0/3592243612 wait complete. 2024-04-24T08:38:35.084 INFO:teuthology.orchestra.run.smithi083.stdout:Waiting for the mgr to restart... 2024-04-24T08:38:35.084 INFO:teuthology.orchestra.run.smithi083.stdout:Waiting for mgr epoch 9... 2024-04-24T08:38:35.603 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:38:35 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:38:35.334+0000 7fbb19070200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-24T08:38:35.603 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:38:35 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:38:35.464+0000 7fbb19070200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-24T08:38:35.888 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:38:35 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:38:35.722+0000 7fbb19070200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-24T08:38:36.289 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:38:35 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:38:35.978+0000 7fbb19070200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-24T08:38:37.053 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:38:36 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:38:36.746+0000 7fbb19070200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-24T08:38:37.410 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:38:37 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:38:37.282+0000 7fbb19070200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-24T08:38:37.410 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:38:37 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:38:37.406+0000 7fbb19070200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-24T08:38:38.053 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:38:37 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:38:37.665+0000 7fbb19070200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-24T08:38:38.420 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:38:38 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:38:38.309+0000 7fbb19070200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-24T08:38:38.692 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:38:38 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:38:38.436+0000 7fbb19070200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-24T08:38:38.955 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:38:38 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:38:38.769+0000 7fbb19070200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-24T08:38:39.241 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:38:38 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:38:38.954+0000 7fbb19070200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-24T08:38:39.241 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:38:39 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:38:39.084+0000 7fbb19070200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-24T08:38:39.241 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:38:39 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:38:39.239+0000 7fbb19070200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-24T08:38:40.054 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:38:39 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:38:39.620+0000 7fbb19070200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-24T08:38:40.658 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:38:40 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:38:40.372+0000 7fbb19070200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-24T08:38:41.053 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:38:40 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:38:40.656+0000 7fbb19070200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-24T08:38:41.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:38:40 smithi083 ceph-mon[27907]: Active manager daemon x restarted 2024-04-24T08:38:41.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:38:40 smithi083 ceph-mon[27907]: Activating manager daemon x 2024-04-24T08:38:41.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:38:40 smithi083 ceph-mon[27907]: osdmap e3: 0 total, 0 up, 0 in 2024-04-24T08:38:41.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:38:40 smithi083 ceph-mon[27907]: mgrmap e10: x(active, starting, since 0.00894271s) 2024-04-24T08:38:41.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:38:40 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T08:38:41.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:38:40 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-24T08:38:41.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:38:40 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-24T08:38:41.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:38:40 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-24T08:38:41.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:38:40 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-24T08:38:41.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:38:40 smithi083 ceph-mon[27907]: Manager daemon x is now available 2024-04-24T08:38:42.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:38:41 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:38:42.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:38:41 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-24T08:38:42.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:38:41 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-24T08:38:42.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:38:41 smithi083 ceph-mon[27907]: mgrmap e11: x(active, since 1.0099s) 2024-04-24T08:38:42.424 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout { 2024-04-24T08:38:42.424 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-04-24T08:38:42.424 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-24T08:38:42.424 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout } 2024-04-24T08:38:42.425 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:35.678+0000 7f8603800700 1 Processor -- start 2024-04-24T08:38:42.425 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:35.679+0000 7f8603800700 1 -- start start 2024-04-24T08:38:42.425 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:35.679+0000 7f8603800700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85fc107230 0x7f85fc107650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:38:42.425 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:35.679+0000 7f8603800700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f85fc107c20 con 0x7f85fc107230 2024-04-24T08:38:42.426 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:35.679+0000 7f8600e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85fc107230 0x7f85fc107650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:38:42.426 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:35.679+0000 7f8600e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85fc107230 0x7f85fc107650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49732/0 (socket says 172.21.15.83:49732) 2024-04-24T08:38:42.426 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:35.679+0000 7f8600e00700 1 -- 172.21.15.83:0/3327159639 learned_addr learned my addr 172.21.15.83:0/3327159639 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:38:42.428 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:35.680+0000 7f8600e00700 1 -- 172.21.15.83:0/3327159639 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85fc108430 con 0x7f85fc107230 2024-04-24T08:38:42.428 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:35.680+0000 7f8600e00700 1 --2- 172.21.15.83:0/3327159639 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85fc107230 0x7f85fc107650 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f85f000bb40 tx=0x7f85f000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c8024daefe76940 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:38:42.428 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:35.680+0000 7f85fb400700 1 -- 172.21.15.83:0/3327159639 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f85f000dbe0 con 0x7f85fc107230 2024-04-24T08:38:42.428 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:35.681+0000 7f85fb400700 1 -- 172.21.15.83:0/3327159639 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f85f00044e0 con 0x7f85fc107230 2024-04-24T08:38:42.428 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:35.681+0000 7f85fb400700 1 -- 172.21.15.83:0/3327159639 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f85f0010470 con 0x7f85fc107230 2024-04-24T08:38:42.428 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:35.681+0000 7f8603800700 1 -- 172.21.15.83:0/3327159639 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85fc107230 msgr2=0x7f85fc107650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:38:42.428 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:35.681+0000 7f8603800700 1 --2- 172.21.15.83:0/3327159639 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85fc107230 0x7f85fc107650 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f85f000bb40 tx=0x7f85f000be50 comp rx=0 tx=0).stop 2024-04-24T08:38:42.428 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:35.681+0000 7f8603800700 1 -- 172.21.15.83:0/3327159639 shutdown_connections 2024-04-24T08:38:42.428 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:35.681+0000 7f8603800700 1 --2- 172.21.15.83:0/3327159639 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85fc107230 0x7f85fc107650 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:38:42.428 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:35.681+0000 7f8603800700 1 -- 172.21.15.83:0/3327159639 >> 172.21.15.83:0/3327159639 conn(0x7f85fc102790 msgr2=0x7f85fc104c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:38:42.428 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:35.682+0000 7f8603800700 1 -- 172.21.15.83:0/3327159639 shutdown_connections 2024-04-24T08:38:42.428 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:35.682+0000 7f8603800700 1 -- 172.21.15.83:0/3327159639 wait complete. 2024-04-24T08:38:42.429 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:35.683+0000 7f8603800700 1 Processor -- start 2024-04-24T08:38:42.429 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:35.683+0000 7f8603800700 1 -- start start 2024-04-24T08:38:42.429 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:35.683+0000 7f8603800700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85fc107230 0x7f85fc19a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:38:42.432 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:35.683+0000 7f8603800700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f85fc07afc0 con 0x7f85fc107230 2024-04-24T08:38:42.432 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:35.684+0000 7f8600e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85fc107230 0x7f85fc19a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:38:42.432 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:35.684+0000 7f8600e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85fc107230 0x7f85fc19a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49734/0 (socket says 172.21.15.83:49734) 2024-04-24T08:38:42.432 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:35.684+0000 7f8600e00700 1 -- 172.21.15.83:0/1523096112 learned_addr learned my addr 172.21.15.83:0/1523096112 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:38:42.432 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:35.684+0000 7f8600e00700 1 -- 172.21.15.83:0/1523096112 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85fc19a5d0 con 0x7f85fc107230 2024-04-24T08:38:42.432 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:35.684+0000 7f8600e00700 1 --2- 172.21.15.83:0/1523096112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85fc107230 0x7f85fc19a090 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f85f000b340 tx=0x7f85f0003fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:38:42.432 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:35.685+0000 7f85f9600700 1 -- 172.21.15.83:0/1523096112 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f85f0027070 con 0x7f85fc107230 2024-04-24T08:38:42.432 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:35.685+0000 7f85f9600700 1 -- 172.21.15.83:0/1523096112 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f85f0010bd0 con 0x7f85fc107230 2024-04-24T08:38:42.432 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:35.685+0000 7f8603800700 1 -- 172.21.15.83:0/1523096112 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85fc19a860 con 0x7f85fc107230 2024-04-24T08:38:42.432 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:35.685+0000 7f85f9600700 1 -- 172.21.15.83:0/1523096112 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f85f0022b70 con 0x7f85fc107230 2024-04-24T08:38:42.433 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:35.685+0000 7f8603800700 1 -- 172.21.15.83:0/1523096112 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85fc19acc0 con 0x7f85fc107230 2024-04-24T08:38:42.433 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:35.686+0000 7f85f9600700 1 -- 172.21.15.83:0/1523096112 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f85f0022d10 con 0x7f85fc107230 2024-04-24T08:38:42.433 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:35.686+0000 7f85f9600700 1 --2- 172.21.15.83:0/1523096112 >> [v2:172.21.15.83:6800/820049883,v1:172.21.15.83:6801/820049883] conn(0x7f85e4039660 0x7f85e403bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:38:42.433 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:35.686+0000 7f85f9600700 1 -- 172.21.15.83:0/1523096112 --> [v2:172.21.15.83:6800/820049883,v1:172.21.15.83:6801/820049883] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f85e403c230 con 0x7f85e4039660 2024-04-24T08:38:42.433 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:35.687+0000 7f85f9600700 1 -- 172.21.15.83:0/1523096112 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f85f001f070 con 0x7f85fc107230 2024-04-24T08:38:42.433 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:35.687+0000 7f85fbe00700 1 -- 172.21.15.83:0/1523096112 >> [v2:172.21.15.83:6800/820049883,v1:172.21.15.83:6801/820049883] conn(0x7f85e4039660 msgr2=0x7f85e403bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.83:6800/820049883 2024-04-24T08:38:42.433 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:35.687+0000 7f85fbe00700 1 --2- 172.21.15.83:0/1523096112 >> [v2:172.21.15.83:6800/820049883,v1:172.21.15.83:6801/820049883] conn(0x7f85e4039660 0x7f85e403bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T08:38:42.433 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:35.888+0000 7f85fbe00700 1 -- 172.21.15.83:0/1523096112 >> [v2:172.21.15.83:6800/820049883,v1:172.21.15.83:6801/820049883] conn(0x7f85e4039660 msgr2=0x7f85e403bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.83:6800/820049883 2024-04-24T08:38:42.433 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:35.888+0000 7f85fbe00700 1 --2- 172.21.15.83:0/1523096112 >> [v2:172.21.15.83:6800/820049883,v1:172.21.15.83:6801/820049883] conn(0x7f85e4039660 0x7f85e403bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-24T08:38:42.433 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:36.289+0000 7f85fbe00700 1 -- 172.21.15.83:0/1523096112 >> [v2:172.21.15.83:6800/820049883,v1:172.21.15.83:6801/820049883] conn(0x7f85e4039660 msgr2=0x7f85e403bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.83:6800/820049883 2024-04-24T08:38:42.433 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:36.289+0000 7f85fbe00700 1 --2- 172.21.15.83:0/1523096112 >> [v2:172.21.15.83:6800/820049883,v1:172.21.15.83:6801/820049883] conn(0x7f85e4039660 0x7f85e403bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-24T08:38:42.433 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:37.089+0000 7f85fbe00700 1 -- 172.21.15.83:0/1523096112 >> [v2:172.21.15.83:6800/820049883,v1:172.21.15.83:6801/820049883] conn(0x7f85e4039660 msgr2=0x7f85e403bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.83:6800/820049883 2024-04-24T08:38:42.433 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:37.089+0000 7f85fbe00700 1 --2- 172.21.15.83:0/1523096112 >> [v2:172.21.15.83:6800/820049883,v1:172.21.15.83:6801/820049883] conn(0x7f85e4039660 0x7f85e403bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-24T08:38:42.434 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:38.691+0000 7f85fbe00700 1 -- 172.21.15.83:0/1523096112 >> [v2:172.21.15.83:6800/820049883,v1:172.21.15.83:6801/820049883] conn(0x7f85e4039660 msgr2=0x7f85e403bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.83:6800/820049883 2024-04-24T08:38:42.434 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:38.691+0000 7f85fbe00700 1 --2- 172.21.15.83:0/1523096112 >> [v2:172.21.15.83:6800/820049883,v1:172.21.15.83:6801/820049883] conn(0x7f85e4039660 0x7f85e403bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-24T08:38:42.434 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:40.666+0000 7f85f9600700 1 -- 172.21.15.83:0/1523096112 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mgrmap(e 10) ==== 46338+0+0 (secure 0 0 0) 0x7f85f002a070 con 0x7f85fc107230 2024-04-24T08:38:42.434 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:40.666+0000 7f85f9600700 1 -- 172.21.15.83:0/1523096112 >> [v2:172.21.15.83:6800/820049883,v1:172.21.15.83:6801/820049883] conn(0x7f85e4039660 msgr2=0x7f85e403bb20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T08:38:42.434 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:40.666+0000 7f85f9600700 1 --2- 172.21.15.83:0/1523096112 >> [v2:172.21.15.83:6800/820049883,v1:172.21.15.83:6801/820049883] conn(0x7f85e4039660 0x7f85e403bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:38:42.436 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:41.667+0000 7f85f9600700 1 -- 172.21.15.83:0/1523096112 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mgrmap(e 11) ==== 46465+0+0 (secure 0 0 0) 0x7f85f0051900 con 0x7f85fc107230 2024-04-24T08:38:42.436 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:41.667+0000 7f85f9600700 1 --2- 172.21.15.83:0/1523096112 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f85e403cff0 0x7f85e403f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:38:42.436 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:41.668+0000 7f85f9600700 1 -- 172.21.15.83:0/1523096112 --> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f85e403c230 con 0x7f85e403cff0 2024-04-24T08:38:42.437 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:41.668+0000 7f85fbe00700 1 --2- 172.21.15.83:0/1523096112 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f85e403cff0 0x7f85e403f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:38:42.437 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:41.669+0000 7f85fbe00700 1 --2- 172.21.15.83:0/1523096112 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f85e403cff0 0x7f85e403f3e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f85e8003a10 tx=0x7f85e8009270 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:38:42.437 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:41.671+0000 7f85f9600700 1 -- 172.21.15.83:0/1523096112 <== mgr.14152 v2:172.21.15.83:6800/2550826560 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f85e403c230 con 0x7f85e403cff0 2024-04-24T08:38:42.437 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:41.684+0000 7f8603800700 1 -- 172.21.15.83:0/1523096112 --> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f85fc064150 con 0x7f85e403cff0 2024-04-24T08:38:42.437 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:41.684+0000 7f85f9600700 1 -- 172.21.15.83:0/1523096112 <== mgr.14152 v2:172.21.15.83:6800/2550826560 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f85fc064150 con 0x7f85e403cff0 2024-04-24T08:38:42.437 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:41.685+0000 7f8603800700 1 -- 172.21.15.83:0/1523096112 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f85e403cff0 msgr2=0x7f85e403f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:38:42.437 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:41.685+0000 7f8603800700 1 --2- 172.21.15.83:0/1523096112 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f85e403cff0 0x7f85e403f3e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f85e8003a10 tx=0x7f85e8009270 comp rx=0 tx=0).stop 2024-04-24T08:38:42.437 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:41.685+0000 7f8603800700 1 -- 172.21.15.83:0/1523096112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85fc107230 msgr2=0x7f85fc19a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:38:42.437 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:41.685+0000 7f8603800700 1 --2- 172.21.15.83:0/1523096112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85fc107230 0x7f85fc19a090 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f85f000b340 tx=0x7f85f0003fb0 comp rx=0 tx=0).stop 2024-04-24T08:38:42.437 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:41.685+0000 7f8603800700 1 -- 172.21.15.83:0/1523096112 shutdown_connections 2024-04-24T08:38:42.437 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:41.685+0000 7f8603800700 1 --2- 172.21.15.83:0/1523096112 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f85e403cff0 0x7f85e403f3e0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:38:42.437 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:41.685+0000 7f8603800700 1 --2- 172.21.15.83:0/1523096112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85fc107230 0x7f85fc19a090 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:38:42.437 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:41.685+0000 7f8603800700 1 --2- 172.21.15.83:0/1523096112 >> [v2:172.21.15.83:6800/820049883,v1:172.21.15.83:6801/820049883] conn(0x7f85e4039660 0x7f85e403bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:38:42.437 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:41.685+0000 7f8603800700 1 -- 172.21.15.83:0/1523096112 >> 172.21.15.83:0/1523096112 conn(0x7f85fc102790 msgr2=0x7f85fc1906b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:38:42.438 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:41.685+0000 7f8603800700 1 -- 172.21.15.83:0/1523096112 shutdown_connections 2024-04-24T08:38:42.438 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:41.685+0000 7f8603800700 1 -- 172.21.15.83:0/1523096112 wait complete. 2024-04-24T08:38:42.438 INFO:teuthology.orchestra.run.smithi083.stdout:mgr epoch 9 is available 2024-04-24T08:38:42.438 INFO:teuthology.orchestra.run.smithi083.stdout:Generating a dashboard self-signed certificate... 2024-04-24T08:38:43.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:38:42 smithi083 ceph-mon[27907]: [24/Apr/2024:08:38:41] ENGINE Bus STARTING 2024-04-24T08:38:43.150 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:38:42 smithi083 ceph-mon[27907]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-24T08:38:43.150 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:38:42 smithi083 ceph-mon[27907]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-24T08:38:43.150 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:38:42 smithi083 ceph-mon[27907]: [24/Apr/2024:08:38:41] ENGINE Serving on https://172.21.15.83:7150 2024-04-24T08:38:43.150 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:38:42 smithi083 ceph-mon[27907]: [24/Apr/2024:08:38:41] ENGINE Serving on http://172.21.15.83:8765 2024-04-24T08:38:43.150 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:38:42 smithi083 ceph-mon[27907]: [24/Apr/2024:08:38:41] ENGINE Bus STARTED 2024-04-24T08:38:43.972 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:38:43 smithi083 ceph-mon[27907]: mgrmap e12: x(active, since 2s) 2024-04-24T08:38:43.972 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:38:43 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:38:44.946 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-04-24T08:38:44.946 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:43.722+0000 7fa8ae400700 1 Processor -- start 2024-04-24T08:38:44.946 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:43.722+0000 7fa8ae400700 1 -- start start 2024-04-24T08:38:44.950 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:43.723+0000 7fa8ae400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8a810a560 0x7fa8a810a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:38:44.950 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:43.723+0000 7fa8ae400700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa8a810af50 con 0x7fa8a810a560 2024-04-24T08:38:44.950 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:43.723+0000 7fa8a7400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8a810a560 0x7fa8a810a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:38:44.951 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:43.723+0000 7fa8a7400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8a810a560 0x7fa8a810a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54364/0 (socket says 172.21.15.83:54364) 2024-04-24T08:38:44.951 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:43.723+0000 7fa8a7400700 1 -- 172.21.15.83:0/298610044 learned_addr learned my addr 172.21.15.83:0/298610044 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:38:44.951 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:43.723+0000 7fa8a7400700 1 -- 172.21.15.83:0/298610044 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8a810b710 con 0x7fa8a810a560 2024-04-24T08:38:44.951 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:43.724+0000 7fa8a7400700 1 --2- 172.21.15.83:0/298610044 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8a810a560 0x7fa8a810a980 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa8980098a0 tx=0x7fa898009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db3e00341a0a5e0e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:38:44.951 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:43.724+0000 7fa8a6000700 1 -- 172.21.15.83:0/298610044 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa898004160 con 0x7fa8a810a560 2024-04-24T08:38:44.951 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:43.724+0000 7fa8a6000700 1 -- 172.21.15.83:0/298610044 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa898004300 con 0x7fa8a810a560 2024-04-24T08:38:44.951 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:43.724+0000 7fa8a6000700 1 -- 172.21.15.83:0/298610044 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa898011a50 con 0x7fa8a810a560 2024-04-24T08:38:44.951 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:43.725+0000 7fa8ae400700 1 -- 172.21.15.83:0/298610044 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8a810a560 msgr2=0x7fa8a810a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:38:44.951 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:43.725+0000 7fa8ae400700 1 --2- 172.21.15.83:0/298610044 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8a810a560 0x7fa8a810a980 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa8980098a0 tx=0x7fa898009bb0 comp rx=0 tx=0).stop 2024-04-24T08:38:44.951 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:43.725+0000 7fa8ae400700 1 -- 172.21.15.83:0/298610044 shutdown_connections 2024-04-24T08:38:44.951 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:43.725+0000 7fa8ae400700 1 --2- 172.21.15.83:0/298610044 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8a810a560 0x7fa8a810a980 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:38:44.951 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:43.725+0000 7fa8ae400700 1 -- 172.21.15.83:0/298610044 >> 172.21.15.83:0/298610044 conn(0x7fa8a807dbe0 msgr2=0x7fa8a81084c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:38:44.951 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:43.725+0000 7fa8ae400700 1 -- 172.21.15.83:0/298610044 shutdown_connections 2024-04-24T08:38:44.952 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:43.726+0000 7fa8ae400700 1 -- 172.21.15.83:0/298610044 wait complete. 2024-04-24T08:38:44.952 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:43.727+0000 7fa8ae400700 1 Processor -- start 2024-04-24T08:38:44.952 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:43.728+0000 7fa8ae400700 1 -- start start 2024-04-24T08:38:44.952 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:43.728+0000 7fa8ae400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8a810a560 0x7fa8a819e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:38:44.952 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:43.728+0000 7fa8ae400700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa8a810eac0 con 0x7fa8a810a560 2024-04-24T08:38:44.952 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:43.728+0000 7fa8a7400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8a810a560 0x7fa8a819e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:38:44.952 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:43.728+0000 7fa8a7400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8a810a560 0x7fa8a819e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54368/0 (socket says 172.21.15.83:54368) 2024-04-24T08:38:44.952 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:43.728+0000 7fa8a7400700 1 -- 172.21.15.83:0/1517938409 learned_addr learned my addr 172.21.15.83:0/1517938409 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:38:44.952 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:43.729+0000 7fa8a7400700 1 -- 172.21.15.83:0/1517938409 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8a819e940 con 0x7fa8a810a560 2024-04-24T08:38:44.952 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:43.729+0000 7fa8a7400700 1 --2- 172.21.15.83:0/1517938409 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8a810a560 0x7fa8a819e400 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa898011a20 tx=0x7fa898011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:38:44.952 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:43.729+0000 7fa89fe00700 1 -- 172.21.15.83:0/1517938409 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa898003af0 con 0x7fa8a810a560 2024-04-24T08:38:44.952 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:43.729+0000 7fa89fe00700 1 -- 172.21.15.83:0/1517938409 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa89801a8e0 con 0x7fa8a810a560 2024-04-24T08:38:44.952 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:43.729+0000 7fa89fe00700 1 -- 172.21.15.83:0/1517938409 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa898024860 con 0x7fa8a810a560 2024-04-24T08:38:44.952 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:43.730+0000 7fa8ae400700 1 -- 172.21.15.83:0/1517938409 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8a819ebd0 con 0x7fa8a810a560 2024-04-24T08:38:44.953 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:43.730+0000 7fa8ae400700 1 -- 172.21.15.83:0/1517938409 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8a81a18c0 con 0x7fa8a810a560 2024-04-24T08:38:44.953 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:43.731+0000 7fa8ae400700 1 -- 172.21.15.83:0/1517938409 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8a8064150 con 0x7fa8a810a560 2024-04-24T08:38:44.953 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:43.731+0000 7fa89fe00700 1 -- 172.21.15.83:0/1517938409 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7fa898022020 con 0x7fa8a810a560 2024-04-24T08:38:44.953 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:43.732+0000 7fa89fe00700 1 --2- 172.21.15.83:0/1517938409 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fa88c039660 0x7fa88c03bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:38:44.953 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:43.732+0000 7fa89fe00700 1 -- 172.21.15.83:0/1517938409 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa89804da90 con 0x7fa8a810a560 2024-04-24T08:38:44.953 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:43.732+0000 7fa8a6a00700 1 --2- 172.21.15.83:0/1517938409 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fa88c039660 0x7fa88c03bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:38:44.953 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:43.733+0000 7fa8a6a00700 1 --2- 172.21.15.83:0/1517938409 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fa88c039660 0x7fa88c03bb20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa894006fd0 tx=0x7fa894006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:38:44.953 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:43.737+0000 7fa89fe00700 1 -- 172.21.15.83:0/1517938409 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa8980212c0 con 0x7fa8a810a560 2024-04-24T08:38:44.953 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:43.971+0000 7fa8ae400700 1 -- 172.21.15.83:0/1517938409 --> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7fa8a8108280 con 0x7fa88c039660 2024-04-24T08:38:44.953 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:44.132+0000 7fa89fe00700 1 -- 172.21.15.83:0/1517938409 <== mgr.14152 v2:172.21.15.83:6800/2550826560 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7fa8a8108280 con 0x7fa88c039660 2024-04-24T08:38:44.953 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:44.135+0000 7fa8ae400700 1 -- 172.21.15.83:0/1517938409 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fa88c039660 msgr2=0x7fa88c03bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:38:44.953 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:44.135+0000 7fa8ae400700 1 --2- 172.21.15.83:0/1517938409 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fa88c039660 0x7fa88c03bb20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa894006fd0 tx=0x7fa894006e40 comp rx=0 tx=0).stop 2024-04-24T08:38:44.953 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:44.135+0000 7fa8ae400700 1 -- 172.21.15.83:0/1517938409 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8a810a560 msgr2=0x7fa8a819e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:38:44.953 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:44.135+0000 7fa8ae400700 1 --2- 172.21.15.83:0/1517938409 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8a810a560 0x7fa8a819e400 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa898011a20 tx=0x7fa898011b00 comp rx=0 tx=0).stop 2024-04-24T08:38:44.954 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:44.135+0000 7fa8ae400700 1 -- 172.21.15.83:0/1517938409 shutdown_connections 2024-04-24T08:38:44.954 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:44.135+0000 7fa8ae400700 1 --2- 172.21.15.83:0/1517938409 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fa88c039660 0x7fa88c03bb20 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:38:44.954 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:44.135+0000 7fa8ae400700 1 --2- 172.21.15.83:0/1517938409 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8a810a560 0x7fa8a819e400 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:38:44.954 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:44.135+0000 7fa8ae400700 1 -- 172.21.15.83:0/1517938409 >> 172.21.15.83:0/1517938409 conn(0x7fa8a807dbe0 msgr2=0x7fa8a8107b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:38:44.954 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:44.136+0000 7fa8ae400700 1 -- 172.21.15.83:0/1517938409 shutdown_connections 2024-04-24T08:38:44.954 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:44.136+0000 7fa8ae400700 1 -- 172.21.15.83:0/1517938409 wait complete. 2024-04-24T08:38:44.954 INFO:teuthology.orchestra.run.smithi083.stdout:Creating initial admin user... 2024-04-24T08:38:45.271 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:38:45 smithi083 ceph-mon[27907]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:38:45.271 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:38:45 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:38:45.271 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:38:45 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:38:46.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:38:46 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:38:46.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:38:46 smithi083 ceph-mon[27907]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:38:46.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:38:46 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:38:46.843 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$1TSLH3d0JCbfALWSu9sPIu1cTULXniooZNynh521rFU3vQQH1MFQ.", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1713947926, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-04-24T08:38:46.843 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:45.550+0000 7fb9b4400700 1 Processor -- start 2024-04-24T08:38:46.843 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:45.550+0000 7fb9b4400700 1 -- start start 2024-04-24T08:38:46.843 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:45.551+0000 7fb9b4400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb9ac07de60 0x7fb9ac07c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:38:46.843 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:45.551+0000 7fb9b4400700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb9ac07c8e0 con 0x7fb9ac07de60 2024-04-24T08:38:46.843 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:45.551+0000 7fb9b1a00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb9ac07de60 0x7fb9ac07c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:38:46.843 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:45.551+0000 7fb9b1a00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb9ac07de60 0x7fb9ac07c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54382/0 (socket says 172.21.15.83:54382) 2024-04-24T08:38:46.848 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:45.551+0000 7fb9b1a00700 1 -- 172.21.15.83:0/3110994345 learned_addr learned my addr 172.21.15.83:0/3110994345 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:38:46.848 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:45.552+0000 7fb9b1a00700 1 -- 172.21.15.83:0/3110994345 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9ac07ca60 con 0x7fb9ac07de60 2024-04-24T08:38:46.848 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:45.552+0000 7fb9b1a00700 1 --2- 172.21.15.83:0/3110994345 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb9ac07de60 0x7fb9ac07c310 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb9a00098a0 tx=0x7fb9a0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3808789a8854f8f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:38:46.848 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:45.552+0000 7fb9abe00700 1 -- 172.21.15.83:0/3110994345 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb9a0004160 con 0x7fb9ac07de60 2024-04-24T08:38:46.848 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:45.552+0000 7fb9abe00700 1 -- 172.21.15.83:0/3110994345 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb9a0004300 con 0x7fb9ac07de60 2024-04-24T08:38:46.849 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:45.552+0000 7fb9abe00700 1 -- 172.21.15.83:0/3110994345 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb9a0011a50 con 0x7fb9ac07de60 2024-04-24T08:38:46.849 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:45.553+0000 7fb9b4400700 1 -- 172.21.15.83:0/3110994345 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb9ac07de60 msgr2=0x7fb9ac07c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:38:46.849 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:45.553+0000 7fb9b4400700 1 --2- 172.21.15.83:0/3110994345 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb9ac07de60 0x7fb9ac07c310 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb9a00098a0 tx=0x7fb9a0009bb0 comp rx=0 tx=0).stop 2024-04-24T08:38:46.849 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:45.553+0000 7fb9b4400700 1 -- 172.21.15.83:0/3110994345 shutdown_connections 2024-04-24T08:38:46.849 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:45.553+0000 7fb9b4400700 1 --2- 172.21.15.83:0/3110994345 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb9ac07de60 0x7fb9ac07c310 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:38:46.849 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:45.553+0000 7fb9b4400700 1 -- 172.21.15.83:0/3110994345 >> 172.21.15.83:0/3110994345 conn(0x7fb9ac103920 msgr2=0x7fb9ac105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:38:46.849 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:45.554+0000 7fb9b4400700 1 -- 172.21.15.83:0/3110994345 shutdown_connections 2024-04-24T08:38:46.849 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:45.554+0000 7fb9b4400700 1 -- 172.21.15.83:0/3110994345 wait complete. 2024-04-24T08:38:46.849 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:45.555+0000 7fb9b4400700 1 Processor -- start 2024-04-24T08:38:46.849 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:45.555+0000 7fb9b4400700 1 -- start start 2024-04-24T08:38:46.849 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:45.555+0000 7fb9b4400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb9ac07de60 0x7fb9ac19e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:38:46.849 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:45.555+0000 7fb9b4400700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb9ac10e8e0 con 0x7fb9ac07de60 2024-04-24T08:38:46.849 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:45.555+0000 7fb9b1a00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb9ac07de60 0x7fb9ac19e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:38:46.850 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:45.555+0000 7fb9b1a00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb9ac07de60 0x7fb9ac19e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54392/0 (socket says 172.21.15.83:54392) 2024-04-24T08:38:46.850 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:45.555+0000 7fb9b1a00700 1 -- 172.21.15.83:0/3913094990 learned_addr learned my addr 172.21.15.83:0/3913094990 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:38:46.850 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:45.556+0000 7fb9b1a00700 1 -- 172.21.15.83:0/3913094990 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9ac19e910 con 0x7fb9ac07de60 2024-04-24T08:38:46.850 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:45.556+0000 7fb9b1a00700 1 --2- 172.21.15.83:0/3913094990 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb9ac07de60 0x7fb9ac19e3d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb9a0011a00 tx=0x7fb9a0011ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:38:46.850 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:45.556+0000 7fb9aa000700 1 -- 172.21.15.83:0/3913094990 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb9a0003af0 con 0x7fb9ac07de60 2024-04-24T08:38:46.850 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:45.556+0000 7fb9aa000700 1 -- 172.21.15.83:0/3913094990 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb9a001a8e0 con 0x7fb9ac07de60 2024-04-24T08:38:46.850 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:45.557+0000 7fb9b4400700 1 -- 172.21.15.83:0/3913094990 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9ac19eba0 con 0x7fb9ac07de60 2024-04-24T08:38:46.850 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:45.557+0000 7fb9aa000700 1 -- 172.21.15.83:0/3913094990 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb9a0024840 con 0x7fb9ac07de60 2024-04-24T08:38:46.850 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:45.557+0000 7fb9b4400700 1 -- 172.21.15.83:0/3913094990 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9ac1a1890 con 0x7fb9ac07de60 2024-04-24T08:38:46.850 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:45.558+0000 7fb9aa000700 1 -- 172.21.15.83:0/3913094990 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7fb9a0022020 con 0x7fb9ac07de60 2024-04-24T08:38:46.850 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:45.558+0000 7fb9b4400700 1 -- 172.21.15.83:0/3913094990 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9ac064150 con 0x7fb9ac07de60 2024-04-24T08:38:46.850 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:45.559+0000 7fb9aa000700 1 --2- 172.21.15.83:0/3913094990 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fb994039660 0x7fb99403bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:38:46.851 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:45.559+0000 7fb9aa000700 1 -- 172.21.15.83:0/3913094990 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb9a004db20 con 0x7fb9ac07de60 2024-04-24T08:38:46.851 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:45.559+0000 7fb9b1000700 1 --2- 172.21.15.83:0/3913094990 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fb994039660 0x7fb99403bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:38:46.851 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:45.560+0000 7fb9b1000700 1 --2- 172.21.15.83:0/3913094990 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fb994039660 0x7fb99403bb20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb998006fd0 tx=0x7fb998006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:38:46.851 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:45.565+0000 7fb9aa000700 1 -- 172.21.15.83:0/3913094990 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb9a002ddb0 con 0x7fb9ac07de60 2024-04-24T08:38:46.851 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:45.813+0000 7fb9b4400700 1 -- 172.21.15.83:0/3913094990 --> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7fb9ac19f080 con 0x7fb994039660 2024-04-24T08:38:46.851 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:46.051+0000 7fb9aa000700 1 -- 172.21.15.83:0/3913094990 <== mgr.14152 v2:172.21.15.83:6800/2550826560 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7fb9ac19f080 con 0x7fb994039660 2024-04-24T08:38:46.851 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:46.053+0000 7fb9b4400700 1 -- 172.21.15.83:0/3913094990 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fb994039660 msgr2=0x7fb99403bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:38:46.851 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:46.053+0000 7fb9b4400700 1 --2- 172.21.15.83:0/3913094990 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fb994039660 0x7fb99403bb20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb998006fd0 tx=0x7fb998006e40 comp rx=0 tx=0).stop 2024-04-24T08:38:46.851 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:46.053+0000 7fb9b4400700 1 -- 172.21.15.83:0/3913094990 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb9ac07de60 msgr2=0x7fb9ac19e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:38:46.851 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:46.053+0000 7fb9b4400700 1 --2- 172.21.15.83:0/3913094990 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb9ac07de60 0x7fb9ac19e3d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb9a0011a00 tx=0x7fb9a0011ae0 comp rx=0 tx=0).stop 2024-04-24T08:38:46.851 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:46.053+0000 7fb9b4400700 1 -- 172.21.15.83:0/3913094990 shutdown_connections 2024-04-24T08:38:46.851 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:46.053+0000 7fb9b4400700 1 --2- 172.21.15.83:0/3913094990 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fb994039660 0x7fb99403bb20 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:38:46.851 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:46.053+0000 7fb9b4400700 1 --2- 172.21.15.83:0/3913094990 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb9ac07de60 0x7fb9ac19e3d0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:38:46.851 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:46.053+0000 7fb9b4400700 1 -- 172.21.15.83:0/3913094990 >> 172.21.15.83:0/3913094990 conn(0x7fb9ac103920 msgr2=0x7fb9ac104350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:38:46.852 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:46.054+0000 7fb9b4400700 1 -- 172.21.15.83:0/3913094990 shutdown_connections 2024-04-24T08:38:46.852 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:46.054+0000 7fb9b4400700 1 -- 172.21.15.83:0/3913094990 wait complete. 2024-04-24T08:38:46.852 INFO:teuthology.orchestra.run.smithi083.stdout:Fetching dashboard port number... 2024-04-24T08:38:48.793 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:38:48 smithi083 ceph-mon[27907]: mgrmap e13: x(active, since 6s) 2024-04-24T08:38:48.793 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:38:48 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/1651394792' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-04-24T08:38:48.801 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 8443 2024-04-24T08:38:48.801 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:47.542+0000 7fb35da00700 1 Processor -- start 2024-04-24T08:38:48.802 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:47.543+0000 7fb35da00700 1 -- start start 2024-04-24T08:38:48.802 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:47.543+0000 7fb35da00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb35807de30 0x7fb35807c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:38:48.802 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:47.543+0000 7fb35da00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb35807c870 con 0x7fb35807de30 2024-04-24T08:38:48.802 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:47.543+0000 7fb357e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb35807de30 0x7fb35807c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:38:48.803 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:47.543+0000 7fb357e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb35807de30 0x7fb35807c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54398/0 (socket says 172.21.15.83:54398) 2024-04-24T08:38:48.803 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:47.543+0000 7fb357e00700 1 -- 172.21.15.83:0/494336900 learned_addr learned my addr 172.21.15.83:0/494336900 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:38:48.803 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:47.544+0000 7fb357e00700 1 -- 172.21.15.83:0/494336900 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb35807c9f0 con 0x7fb35807de30 2024-04-24T08:38:48.803 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:47.544+0000 7fb357e00700 1 --2- 172.21.15.83:0/494336900 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb35807de30 0x7fb35807c330 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb3480098a0 tx=0x7fb348009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1323b6cacedd0cf7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:38:48.803 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:47.544+0000 7fb356a00700 1 -- 172.21.15.83:0/494336900 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb348004160 con 0x7fb35807de30 2024-04-24T08:38:48.805 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:47.544+0000 7fb356a00700 1 -- 172.21.15.83:0/494336900 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb348004300 con 0x7fb35807de30 2024-04-24T08:38:48.805 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:47.544+0000 7fb356a00700 1 -- 172.21.15.83:0/494336900 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb348011a50 con 0x7fb35807de30 2024-04-24T08:38:48.805 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:47.545+0000 7fb35da00700 1 -- 172.21.15.83:0/494336900 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb35807de30 msgr2=0x7fb35807c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:38:48.805 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:47.545+0000 7fb35da00700 1 --2- 172.21.15.83:0/494336900 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb35807de30 0x7fb35807c330 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb3480098a0 tx=0x7fb348009bb0 comp rx=0 tx=0).stop 2024-04-24T08:38:48.805 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:47.545+0000 7fb35da00700 1 -- 172.21.15.83:0/494336900 shutdown_connections 2024-04-24T08:38:48.805 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:47.545+0000 7fb35da00700 1 --2- 172.21.15.83:0/494336900 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb35807de30 0x7fb35807c330 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:38:48.805 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:47.545+0000 7fb35da00700 1 -- 172.21.15.83:0/494336900 >> 172.21.15.83:0/494336900 conn(0x7fb358103740 msgr2=0x7fb358105ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:38:48.805 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:47.545+0000 7fb35da00700 1 -- 172.21.15.83:0/494336900 shutdown_connections 2024-04-24T08:38:48.805 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:47.545+0000 7fb35da00700 1 -- 172.21.15.83:0/494336900 wait complete. 2024-04-24T08:38:48.805 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:47.547+0000 7fb35da00700 1 Processor -- start 2024-04-24T08:38:48.809 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:47.547+0000 7fb35da00700 1 -- start start 2024-04-24T08:38:48.809 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:47.547+0000 7fb35da00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb35807de30 0x7fb3581a2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:38:48.809 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:47.547+0000 7fb35da00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb35810a2b0 con 0x7fb35807de30 2024-04-24T08:38:48.809 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:47.548+0000 7fb357e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb35807de30 0x7fb3581a2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:38:48.809 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:47.548+0000 7fb357e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb35807de30 0x7fb3581a2610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54400/0 (socket says 172.21.15.83:54400) 2024-04-24T08:38:48.809 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:47.548+0000 7fb357e00700 1 -- 172.21.15.83:0/1651394792 learned_addr learned my addr 172.21.15.83:0/1651394792 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:38:48.809 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:47.548+0000 7fb357e00700 1 -- 172.21.15.83:0/1651394792 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3581a2b50 con 0x7fb35807de30 2024-04-24T08:38:48.810 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:47.548+0000 7fb357e00700 1 --2- 172.21.15.83:0/1651394792 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb35807de30 0x7fb3581a2610 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb348011a20 tx=0x7fb348011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:38:48.810 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:47.548+0000 7fb354c00700 1 -- 172.21.15.83:0/1651394792 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb348003b70 con 0x7fb35807de30 2024-04-24T08:38:48.810 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:47.549+0000 7fb354c00700 1 -- 172.21.15.83:0/1651394792 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb34801a8e0 con 0x7fb35807de30 2024-04-24T08:38:48.810 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:47.549+0000 7fb35da00700 1 -- 172.21.15.83:0/1651394792 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3581a2de0 con 0x7fb35807de30 2024-04-24T08:38:48.810 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:47.549+0000 7fb354c00700 1 -- 172.21.15.83:0/1651394792 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb348024990 con 0x7fb35807de30 2024-04-24T08:38:48.810 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:47.549+0000 7fb35da00700 1 -- 172.21.15.83:0/1651394792 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3581a5ad0 con 0x7fb35807de30 2024-04-24T08:38:48.810 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:47.550+0000 7fb354c00700 1 -- 172.21.15.83:0/1651394792 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fb348022020 con 0x7fb35807de30 2024-04-24T08:38:48.810 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:47.550+0000 7fb35da00700 1 -- 172.21.15.83:0/1651394792 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb358064150 con 0x7fb35807de30 2024-04-24T08:38:48.810 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:47.550+0000 7fb354c00700 1 --2- 172.21.15.83:0/1651394792 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fb33c039650 0x7fb33c03bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:38:48.810 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:47.550+0000 7fb354c00700 1 -- 172.21.15.83:0/1651394792 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb34804dd00 con 0x7fb35807de30 2024-04-24T08:38:48.810 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:47.551+0000 7fb357400700 1 --2- 172.21.15.83:0/1651394792 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fb33c039650 0x7fb33c03bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:38:48.810 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:47.551+0000 7fb357400700 1 --2- 172.21.15.83:0/1651394792 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fb33c039650 0x7fb33c03bb10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb344006fd0 tx=0x7fb344006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:38:48.810 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:47.556+0000 7fb354c00700 1 -- 172.21.15.83:0/1651394792 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb34800b3e0 con 0x7fb35807de30 2024-04-24T08:38:48.810 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:47.792+0000 7fb35da00700 1 -- 172.21.15.83:0/1651394792 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7fb35802e230 con 0x7fb35807de30 2024-04-24T08:38:48.811 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:47.793+0000 7fb354c00700 1 -- 172.21.15.83:0/1651394792 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7fb348021d80 con 0x7fb35807de30 2024-04-24T08:38:48.811 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:47.797+0000 7fb35da00700 1 -- 172.21.15.83:0/1651394792 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fb33c039650 msgr2=0x7fb33c03bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:38:48.811 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:47.797+0000 7fb35da00700 1 --2- 172.21.15.83:0/1651394792 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fb33c039650 0x7fb33c03bb10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb344006fd0 tx=0x7fb344006e40 comp rx=0 tx=0).stop 2024-04-24T08:38:48.811 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:47.797+0000 7fb35da00700 1 -- 172.21.15.83:0/1651394792 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb35807de30 msgr2=0x7fb3581a2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:38:48.811 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:47.797+0000 7fb35da00700 1 --2- 172.21.15.83:0/1651394792 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb35807de30 0x7fb3581a2610 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb348011a20 tx=0x7fb348011b00 comp rx=0 tx=0).stop 2024-04-24T08:38:48.811 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:47.797+0000 7fb35da00700 1 -- 172.21.15.83:0/1651394792 shutdown_connections 2024-04-24T08:38:48.811 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:47.797+0000 7fb35da00700 1 --2- 172.21.15.83:0/1651394792 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fb33c039650 0x7fb33c03bb10 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:38:48.811 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:47.797+0000 7fb35da00700 1 --2- 172.21.15.83:0/1651394792 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb35807de30 0x7fb3581a2610 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:38:48.811 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:47.797+0000 7fb35da00700 1 -- 172.21.15.83:0/1651394792 >> 172.21.15.83:0/1651394792 conn(0x7fb358103740 msgr2=0x7fb3581994b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:38:48.811 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:47.797+0000 7fb35da00700 1 -- 172.21.15.83:0/1651394792 shutdown_connections 2024-04-24T08:38:48.811 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:47.798+0000 7fb35da00700 1 -- 172.21.15.83:0/1651394792 wait complete. 2024-04-24T08:38:48.837 INFO:teuthology.orchestra.run.smithi083.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-24T08:38:48.837 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stdout disabled 2024-04-24T08:38:48.856 INFO:teuthology.orchestra.run.smithi083.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-24T08:38:48.857 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stdout inactive 2024-04-24T08:38:48.857 INFO:teuthology.orchestra.run.smithi083.stdout:firewalld.service is not enabled 2024-04-24T08:38:48.857 INFO:teuthology.orchestra.run.smithi083.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-04-24T08:38:48.863 INFO:teuthology.orchestra.run.smithi083.stdout:Ceph Dashboard is now available at: 2024-04-24T08:38:48.863 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:38:48.863 INFO:teuthology.orchestra.run.smithi083.stdout: URL: https://smithi083.front.sepia.ceph.com:8443/ 2024-04-24T08:38:48.863 INFO:teuthology.orchestra.run.smithi083.stdout: User: admin 2024-04-24T08:38:48.863 INFO:teuthology.orchestra.run.smithi083.stdout: Password: g6vhctz9az 2024-04-24T08:38:48.864 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:38:48.864 INFO:teuthology.orchestra.run.smithi083.stdout:Saving cluster configuration to /var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/config directory 2024-04-24T08:38:48.866 INFO:teuthology.orchestra.run.smithi083.stdout:Enabling autotune for osd_memory_target 2024-04-24T08:38:50.858 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:49.923+0000 7ff8fa200700 1 Processor -- start 2024-04-24T08:38:50.858 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:49.923+0000 7ff8fa200700 1 -- start start 2024-04-24T08:38:50.858 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:49.923+0000 7ff8fa200700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8f410a540 0x7ff8f410a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:38:50.858 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:49.923+0000 7ff8fa200700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f410af30 con 0x7ff8f410a540 2024-04-24T08:38:50.858 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:49.923+0000 7ff8f3400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8f410a540 0x7ff8f410a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:38:50.858 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:49.924+0000 7ff8f3400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8f410a540 0x7ff8f410a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34302/0 (socket says 172.21.15.83:34302) 2024-04-24T08:38:50.858 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:49.924+0000 7ff8f3400700 1 -- 172.21.15.83:0/236945327 learned_addr learned my addr 172.21.15.83:0/236945327 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:38:50.859 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:49.924+0000 7ff8f3400700 1 -- 172.21.15.83:0/236945327 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8f410b6f0 con 0x7ff8f410a540 2024-04-24T08:38:50.859 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:49.924+0000 7ff8f3400700 1 --2- 172.21.15.83:0/236945327 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8f410a540 0x7ff8f410a960 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff8e40098a0 tx=0x7ff8e4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=850bbce77711656f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:38:50.859 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:49.925+0000 7ff8f2000700 1 -- 172.21.15.83:0/236945327 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff8e4004160 con 0x7ff8f410a540 2024-04-24T08:38:50.861 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:49.925+0000 7ff8f2000700 1 -- 172.21.15.83:0/236945327 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff8e4004300 con 0x7ff8f410a540 2024-04-24T08:38:50.861 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:49.925+0000 7ff8f2000700 1 -- 172.21.15.83:0/236945327 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff8e4011a50 con 0x7ff8f410a540 2024-04-24T08:38:50.861 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:49.925+0000 7ff8fa200700 1 -- 172.21.15.83:0/236945327 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8f410a540 msgr2=0x7ff8f410a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:38:50.861 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:49.925+0000 7ff8fa200700 1 --2- 172.21.15.83:0/236945327 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8f410a540 0x7ff8f410a960 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff8e40098a0 tx=0x7ff8e4009bb0 comp rx=0 tx=0).stop 2024-04-24T08:38:50.861 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:49.925+0000 7ff8fa200700 1 -- 172.21.15.83:0/236945327 shutdown_connections 2024-04-24T08:38:50.861 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:49.925+0000 7ff8fa200700 1 --2- 172.21.15.83:0/236945327 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8f410a540 0x7ff8f410a960 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:38:50.861 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:49.925+0000 7ff8fa200700 1 -- 172.21.15.83:0/236945327 >> 172.21.15.83:0/236945327 conn(0x7ff8f407dbe0 msgr2=0x7ff8f41084c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:38:50.861 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:49.926+0000 7ff8fa200700 1 -- 172.21.15.83:0/236945327 shutdown_connections 2024-04-24T08:38:50.861 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:49.926+0000 7ff8fa200700 1 -- 172.21.15.83:0/236945327 wait complete. 2024-04-24T08:38:50.861 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:49.927+0000 7ff8fa200700 1 Processor -- start 2024-04-24T08:38:50.861 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:49.927+0000 7ff8fa200700 1 -- start start 2024-04-24T08:38:50.861 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:49.927+0000 7ff8fa200700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8f410a540 0x7ff8f419e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:38:50.861 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:49.927+0000 7ff8fa200700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f410eaa0 con 0x7ff8f410a540 2024-04-24T08:38:50.862 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:49.927+0000 7ff8f3400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8f410a540 0x7ff8f419e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:38:50.862 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:49.927+0000 7ff8f3400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8f410a540 0x7ff8f419e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34308/0 (socket says 172.21.15.83:34308) 2024-04-24T08:38:50.865 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:49.927+0000 7ff8f3400700 1 -- 172.21.15.83:0/1330745979 learned_addr learned my addr 172.21.15.83:0/1330745979 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:38:50.865 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:49.928+0000 7ff8f3400700 1 -- 172.21.15.83:0/1330745979 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8f419e900 con 0x7ff8f410a540 2024-04-24T08:38:50.865 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:49.928+0000 7ff8f3400700 1 --2- 172.21.15.83:0/1330745979 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8f410a540 0x7ff8f419e3c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff8e4011710 tx=0x7ff8e4011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:38:50.865 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:49.928+0000 7ff8ebe00700 1 -- 172.21.15.83:0/1330745979 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff8e40039c0 con 0x7ff8f410a540 2024-04-24T08:38:50.865 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:49.928+0000 7ff8ebe00700 1 -- 172.21.15.83:0/1330745979 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff8e4003b60 con 0x7ff8f410a540 2024-04-24T08:38:50.865 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:49.928+0000 7ff8ebe00700 1 -- 172.21.15.83:0/1330745979 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff8e401a690 con 0x7ff8f410a540 2024-04-24T08:38:50.865 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:49.928+0000 7ff8fa200700 1 -- 172.21.15.83:0/1330745979 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8f419eb90 con 0x7ff8f410a540 2024-04-24T08:38:50.865 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:49.929+0000 7ff8fa200700 1 -- 172.21.15.83:0/1330745979 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8f41a1880 con 0x7ff8f410a540 2024-04-24T08:38:50.865 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:49.930+0000 7ff8ebe00700 1 -- 172.21.15.83:0/1330745979 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7ff8e4024430 con 0x7ff8f410a540 2024-04-24T08:38:50.865 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:49.930+0000 7ff8fa200700 1 -- 172.21.15.83:0/1330745979 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8f4064150 con 0x7ff8f410a540 2024-04-24T08:38:50.865 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:49.930+0000 7ff8ebe00700 1 --2- 172.21.15.83:0/1330745979 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7ff8d80396b0 0x7ff8d803bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:38:50.865 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:49.930+0000 7ff8ebe00700 1 -- 172.21.15.83:0/1330745979 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff8e404dfa0 con 0x7ff8f410a540 2024-04-24T08:38:50.866 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:49.930+0000 7ff8f2a00700 1 --2- 172.21.15.83:0/1330745979 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7ff8d80396b0 0x7ff8d803bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:38:50.866 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:49.931+0000 7ff8f2a00700 1 --2- 172.21.15.83:0/1330745979 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7ff8d80396b0 0x7ff8d803bb70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff8e0006fd0 tx=0x7ff8e0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:38:50.866 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:49.936+0000 7ff8ebe00700 1 -- 172.21.15.83:0/1330745979 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff8e40172b0 con 0x7ff8f410a540 2024-04-24T08:38:50.866 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:50.155+0000 7ff8fa200700 1 -- 172.21.15.83:0/1330745979 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7ff8f41a1ea0 con 0x7ff8f410a540 2024-04-24T08:38:50.866 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:50.155+0000 7ff8ebe00700 1 -- 172.21.15.83:0/1330745979 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7ff8e400fdd0 con 0x7ff8f410a540 2024-04-24T08:38:50.866 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:50.158+0000 7ff8fa200700 1 -- 172.21.15.83:0/1330745979 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7ff8d80396b0 msgr2=0x7ff8d803bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:38:50.866 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:50.158+0000 7ff8fa200700 1 --2- 172.21.15.83:0/1330745979 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7ff8d80396b0 0x7ff8d803bb70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff8e0006fd0 tx=0x7ff8e0006e40 comp rx=0 tx=0).stop 2024-04-24T08:38:50.866 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:50.158+0000 7ff8fa200700 1 -- 172.21.15.83:0/1330745979 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8f410a540 msgr2=0x7ff8f419e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:38:50.866 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:50.158+0000 7ff8fa200700 1 --2- 172.21.15.83:0/1330745979 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8f410a540 0x7ff8f419e3c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff8e4011710 tx=0x7ff8e4011ad0 comp rx=0 tx=0).stop 2024-04-24T08:38:50.866 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:50.158+0000 7ff8fa200700 1 -- 172.21.15.83:0/1330745979 shutdown_connections 2024-04-24T08:38:50.866 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:50.158+0000 7ff8fa200700 1 --2- 172.21.15.83:0/1330745979 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7ff8d80396b0 0x7ff8d803bb70 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:38:50.866 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:50.158+0000 7ff8fa200700 1 --2- 172.21.15.83:0/1330745979 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8f410a540 0x7ff8f419e3c0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:38:50.866 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:50.158+0000 7ff8fa200700 1 -- 172.21.15.83:0/1330745979 >> 172.21.15.83:0/1330745979 conn(0x7ff8f407dbe0 msgr2=0x7ff8f4107b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:38:50.866 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:50.158+0000 7ff8fa200700 1 -- 172.21.15.83:0/1330745979 shutdown_connections 2024-04-24T08:38:50.866 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:50.158+0000 7ff8fa200700 1 -- 172.21.15.83:0/1330745979 wait complete. 2024-04-24T08:38:52.610 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:51.539+0000 7f7e60000700 1 Processor -- start 2024-04-24T08:38:52.617 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:51.539+0000 7f7e60000700 1 -- start start 2024-04-24T08:38:52.617 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:51.540+0000 7f7e60000700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e58108360 0x7f7e58108780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:38:52.617 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:51.540+0000 7f7e60000700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7e58108d50 con 0x7f7e58108360 2024-04-24T08:38:52.617 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:51.540+0000 7f7e5d600700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e58108360 0x7f7e58108780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:38:52.617 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:51.540+0000 7f7e5d600700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e58108360 0x7f7e58108780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34324/0 (socket says 172.21.15.83:34324) 2024-04-24T08:38:52.617 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:51.540+0000 7f7e5d600700 1 -- 172.21.15.83:0/389074780 learned_addr learned my addr 172.21.15.83:0/389074780 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:38:52.617 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:51.541+0000 7f7e5d600700 1 -- 172.21.15.83:0/389074780 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e58109510 con 0x7f7e58108360 2024-04-24T08:38:52.617 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:51.541+0000 7f7e5d600700 1 --2- 172.21.15.83:0/389074780 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e58108360 0x7f7e58108780 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c009e80 tx=0x7f7e4c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71c56838e769575 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:38:52.617 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:51.541+0000 7f7e57e00700 1 -- 172.21.15.83:0/389074780 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7e4c017070 con 0x7f7e58108360 2024-04-24T08:38:52.617 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:51.541+0000 7f7e57e00700 1 -- 172.21.15.83:0/389074780 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7e4c00b8c0 con 0x7f7e58108360 2024-04-24T08:38:52.617 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:51.541+0000 7f7e57e00700 1 -- 172.21.15.83:0/389074780 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7e4c004490 con 0x7f7e58108360 2024-04-24T08:38:52.617 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:51.542+0000 7f7e60000700 1 -- 172.21.15.83:0/389074780 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e58108360 msgr2=0x7f7e58108780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:38:52.618 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:51.542+0000 7f7e60000700 1 --2- 172.21.15.83:0/389074780 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e58108360 0x7f7e58108780 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c009e80 tx=0x7f7e4c00b330 comp rx=0 tx=0).stop 2024-04-24T08:38:52.618 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:51.542+0000 7f7e60000700 1 -- 172.21.15.83:0/389074780 shutdown_connections 2024-04-24T08:38:52.618 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:51.542+0000 7f7e60000700 1 --2- 172.21.15.83:0/389074780 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e58108360 0x7f7e58108780 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:38:52.618 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:51.542+0000 7f7e60000700 1 -- 172.21.15.83:0/389074780 >> 172.21.15.83:0/389074780 conn(0x7f7e58103920 msgr2=0x7f7e58105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:38:52.618 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:51.542+0000 7f7e60000700 1 -- 172.21.15.83:0/389074780 shutdown_connections 2024-04-24T08:38:52.618 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:51.542+0000 7f7e60000700 1 -- 172.21.15.83:0/389074780 wait complete. 2024-04-24T08:38:52.618 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:51.543+0000 7f7e60000700 1 Processor -- start 2024-04-24T08:38:52.618 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:51.543+0000 7f7e60000700 1 -- start start 2024-04-24T08:38:52.618 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:51.544+0000 7f7e60000700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e58108360 0x7f7e5819e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:38:52.618 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:51.544+0000 7f7e60000700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7e5810c8c0 con 0x7f7e58108360 2024-04-24T08:38:52.618 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:51.544+0000 7f7e5d600700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e58108360 0x7f7e5819e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:38:52.618 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:51.544+0000 7f7e5d600700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e58108360 0x7f7e5819e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34328/0 (socket says 172.21.15.83:34328) 2024-04-24T08:38:52.618 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:51.544+0000 7f7e5d600700 1 -- 172.21.15.83:0/4126147519 learned_addr learned my addr 172.21.15.83:0/4126147519 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:38:52.619 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:51.544+0000 7f7e5d600700 1 -- 172.21.15.83:0/4126147519 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e5819ea10 con 0x7f7e58108360 2024-04-24T08:38:52.619 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:51.544+0000 7f7e5d600700 1 --2- 172.21.15.83:0/4126147519 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e58108360 0x7f7e5819e4d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c00b7b0 tx=0x7f7e4c003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:38:52.619 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:51.545+0000 7f7e56000700 1 -- 172.21.15.83:0/4126147519 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7e4c020020 con 0x7f7e58108360 2024-04-24T08:38:52.619 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:51.545+0000 7f7e56000700 1 -- 172.21.15.83:0/4126147519 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7e4c012430 con 0x7f7e58108360 2024-04-24T08:38:52.619 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:51.545+0000 7f7e56000700 1 -- 172.21.15.83:0/4126147519 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7e4c017040 con 0x7f7e58108360 2024-04-24T08:38:52.619 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:51.545+0000 7f7e60000700 1 -- 172.21.15.83:0/4126147519 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e5819eca0 con 0x7f7e58108360 2024-04-24T08:38:52.619 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:51.545+0000 7f7e60000700 1 -- 172.21.15.83:0/4126147519 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e581a1940 con 0x7f7e58108360 2024-04-24T08:38:52.619 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:51.546+0000 7f7e60000700 1 -- 172.21.15.83:0/4126147519 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e58064150 con 0x7f7e58108360 2024-04-24T08:38:52.619 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:51.546+0000 7f7e56000700 1 -- 172.21.15.83:0/4126147519 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f7e4c023020 con 0x7f7e58108360 2024-04-24T08:38:52.619 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:51.547+0000 7f7e56000700 1 --2- 172.21.15.83:0/4126147519 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f7e400396f0 0x7f7e4003bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:38:52.619 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:51.547+0000 7f7e56000700 1 -- 172.21.15.83:0/4126147519 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f7e4c00f050 con 0x7f7e58108360 2024-04-24T08:38:52.619 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:51.547+0000 7f7e5cc00700 1 --2- 172.21.15.83:0/4126147519 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f7e400396f0 0x7f7e4003bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:38:52.619 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:51.548+0000 7f7e5cc00700 1 --2- 172.21.15.83:0/4126147519 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f7e400396f0 0x7f7e4003bbb0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f7e44006fd0 tx=0x7f7e44006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:38:52.619 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:51.552+0000 7f7e56000700 1 -- 172.21.15.83:0/4126147519 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7e4c02a020 con 0x7f7e58108360 2024-04-24T08:38:52.620 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:51.892+0000 7f7e60000700 1 -- 172.21.15.83:0/4126147519 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f7e581a1f10 con 0x7f7e58108360 2024-04-24T08:38:52.620 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:51.896+0000 7f7e56000700 1 -- 172.21.15.83:0/4126147519 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7f7e4c025e20 con 0x7f7e58108360 2024-04-24T08:38:52.620 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:51.899+0000 7f7e60000700 1 -- 172.21.15.83:0/4126147519 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f7e400396f0 msgr2=0x7f7e4003bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:38:52.620 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:51.899+0000 7f7e60000700 1 --2- 172.21.15.83:0/4126147519 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f7e400396f0 0x7f7e4003bbb0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f7e44006fd0 tx=0x7f7e44006e40 comp rx=0 tx=0).stop 2024-04-24T08:38:52.620 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:51.899+0000 7f7e60000700 1 -- 172.21.15.83:0/4126147519 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e58108360 msgr2=0x7f7e5819e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:38:52.620 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:51.899+0000 7f7e60000700 1 --2- 172.21.15.83:0/4126147519 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e58108360 0x7f7e5819e4d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c00b7b0 tx=0x7f7e4c003ce0 comp rx=0 tx=0).stop 2024-04-24T08:38:52.620 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:51.899+0000 7f7e60000700 1 -- 172.21.15.83:0/4126147519 shutdown_connections 2024-04-24T08:38:52.620 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:51.899+0000 7f7e60000700 1 --2- 172.21.15.83:0/4126147519 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f7e400396f0 0x7f7e4003bbb0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:38:52.620 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:51.899+0000 7f7e60000700 1 --2- 172.21.15.83:0/4126147519 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e58108360 0x7f7e5819e4d0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:38:52.620 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:51.899+0000 7f7e60000700 1 -- 172.21.15.83:0/4126147519 >> 172.21.15.83:0/4126147519 conn(0x7f7e58103920 msgr2=0x7f7e58104380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:38:52.620 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:51.899+0000 7f7e60000700 1 -- 172.21.15.83:0/4126147519 shutdown_connections 2024-04-24T08:38:52.620 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-24T08:38:51.899+0000 7f7e60000700 1 -- 172.21.15.83:0/4126147519 wait complete. 2024-04-24T08:38:52.620 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-04-24T08:38:52.620 INFO:teuthology.orchestra.run.smithi083.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-04-24T08:38:52.620 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:38:52.620 INFO:teuthology.orchestra.run.smithi083.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid bd3e039e-0215-11ef-bc93-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-04-24T08:38:52.621 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:38:52.621 INFO:teuthology.orchestra.run.smithi083.stdout:Or, if you are only running a single cluster on this host: 2024-04-24T08:38:52.621 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:38:52.621 INFO:teuthology.orchestra.run.smithi083.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-04-24T08:38:52.621 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:38:52.621 INFO:teuthology.orchestra.run.smithi083.stdout:Please consider enabling telemetry to help improve Ceph: 2024-04-24T08:38:52.621 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:38:52.621 INFO:teuthology.orchestra.run.smithi083.stdout: ceph telemetry on 2024-04-24T08:38:52.621 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:38:52.621 INFO:teuthology.orchestra.run.smithi083.stdout:For more information see: 2024-04-24T08:38:52.621 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:38:52.621 INFO:teuthology.orchestra.run.smithi083.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-04-24T08:38:52.621 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:38:52.621 INFO:teuthology.orchestra.run.smithi083.stdout:Bootstrap complete. 2024-04-24T08:38:52.693 INFO:tasks.cephadm:Fetching config... 2024-04-24T08:38:52.694 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-24T08:38:52.694 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-04-24T08:38:52.740 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-04-24T08:38:52.740 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-24T08:38:52.740 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-04-24T08:38:52.821 INFO:tasks.cephadm:Fetching mon keyring... 2024-04-24T08:38:52.821 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-24T08:38:52.822 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/mon.a/keyring of=/dev/stdout 2024-04-24T08:38:52.931 INFO:tasks.cephadm:Fetching pub ssh key... 2024-04-24T08:38:52.932 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-24T08:38:52.932 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-04-24T08:38:53.022 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-04-24T08:38:53.023 DEBUG:teuthology.orchestra.run.smithi083:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDADFsq18W8JTifbXybABKcXFLWR3x3ActmbY2eMvg0Vlu+3KM7De9oVuGJzL+RNfLtUqh7n0kYWRUJ7hNs11wySoNCaDxyPmtUix5yphqVoEFRLBO88SsHMwLdMvWvHf7rHDQMkfThLzzqAF74JC7+hQ+j9U45gPzMCED0PnPSs8Dfigz0EkhhWG6IPDNJvWvQoyDkPkc1suGLsqVRNF6Coc8nfWU6q5GzOqV1198IA+dy/MRUAJ23vnGb9ps3Qwn/Al7ey7o2IgFoB3sqcFu7DWs4y2AAinh1Wv0XbSdgt67jNFZhoSc2pZdR2Ea7Ilr4qn1nzXNZJ4TUFSfUqinmpZGdC6nk63bly0iLmDivqOIpFrFGbMlo1JJOvLuA1XpkPyc0+MUFeVSFIoohLuEyts9xKZc3ByS3kwgf8Ym9WZbKnu2UklnpIv+KRDNCYi6udhLQ629s828wm6IMXCPIX+k78Lj2rdarmVEOcS/5M+ufM8TPHRGl61Y94KLD4ds= ceph-bd3e039e-0215-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-24T08:38:53.125 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:38:52 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/4126147519' entity='client.admin' 2024-04-24T08:38:53.151 INFO:teuthology.orchestra.run.smithi083.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDADFsq18W8JTifbXybABKcXFLWR3x3ActmbY2eMvg0Vlu+3KM7De9oVuGJzL+RNfLtUqh7n0kYWRUJ7hNs11wySoNCaDxyPmtUix5yphqVoEFRLBO88SsHMwLdMvWvHf7rHDQMkfThLzzqAF74JC7+hQ+j9U45gPzMCED0PnPSs8Dfigz0EkhhWG6IPDNJvWvQoyDkPkc1suGLsqVRNF6Coc8nfWU6q5GzOqV1198IA+dy/MRUAJ23vnGb9ps3Qwn/Al7ey7o2IgFoB3sqcFu7DWs4y2AAinh1Wv0XbSdgt67jNFZhoSc2pZdR2Ea7Ilr4qn1nzXNZJ4TUFSfUqinmpZGdC6nk63bly0iLmDivqOIpFrFGbMlo1JJOvLuA1XpkPyc0+MUFeVSFIoohLuEyts9xKZc3ByS3kwgf8Ym9WZbKnu2UklnpIv+KRDNCYi6udhLQ629s828wm6IMXCPIX+k78Lj2rdarmVEOcS/5M+ufM8TPHRGl61Y94KLD4ds= ceph-bd3e039e-0215-11ef-bc93-c7b262605968 2024-04-24T08:38:53.180 DEBUG:teuthology.orchestra.run.smithi138:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDADFsq18W8JTifbXybABKcXFLWR3x3ActmbY2eMvg0Vlu+3KM7De9oVuGJzL+RNfLtUqh7n0kYWRUJ7hNs11wySoNCaDxyPmtUix5yphqVoEFRLBO88SsHMwLdMvWvHf7rHDQMkfThLzzqAF74JC7+hQ+j9U45gPzMCED0PnPSs8Dfigz0EkhhWG6IPDNJvWvQoyDkPkc1suGLsqVRNF6Coc8nfWU6q5GzOqV1198IA+dy/MRUAJ23vnGb9ps3Qwn/Al7ey7o2IgFoB3sqcFu7DWs4y2AAinh1Wv0XbSdgt67jNFZhoSc2pZdR2Ea7Ilr4qn1nzXNZJ4TUFSfUqinmpZGdC6nk63bly0iLmDivqOIpFrFGbMlo1JJOvLuA1XpkPyc0+MUFeVSFIoohLuEyts9xKZc3ByS3kwgf8Ym9WZbKnu2UklnpIv+KRDNCYi6udhLQ629s828wm6IMXCPIX+k78Lj2rdarmVEOcS/5M+ufM8TPHRGl61Y94KLD4ds= ceph-bd3e039e-0215-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-24T08:38:53.304 INFO:teuthology.orchestra.run.smithi138.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDADFsq18W8JTifbXybABKcXFLWR3x3ActmbY2eMvg0Vlu+3KM7De9oVuGJzL+RNfLtUqh7n0kYWRUJ7hNs11wySoNCaDxyPmtUix5yphqVoEFRLBO88SsHMwLdMvWvHf7rHDQMkfThLzzqAF74JC7+hQ+j9U45gPzMCED0PnPSs8Dfigz0EkhhWG6IPDNJvWvQoyDkPkc1suGLsqVRNF6Coc8nfWU6q5GzOqV1198IA+dy/MRUAJ23vnGb9ps3Qwn/Al7ey7o2IgFoB3sqcFu7DWs4y2AAinh1Wv0XbSdgt67jNFZhoSc2pZdR2Ea7Ilr4qn1nzXNZJ4TUFSfUqinmpZGdC6nk63bly0iLmDivqOIpFrFGbMlo1JJOvLuA1XpkPyc0+MUFeVSFIoohLuEyts9xKZc3ByS3kwgf8Ym9WZbKnu2UklnpIv+KRDNCYi6udhLQ629s828wm6IMXCPIX+k78Lj2rdarmVEOcS/5M+ufM8TPHRGl61Y94KLD4ds= ceph-bd3e039e-0215-11ef-bc93-c7b262605968 2024-04-24T08:38:53.360 DEBUG:teuthology.orchestra.run.smithi189:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDADFsq18W8JTifbXybABKcXFLWR3x3ActmbY2eMvg0Vlu+3KM7De9oVuGJzL+RNfLtUqh7n0kYWRUJ7hNs11wySoNCaDxyPmtUix5yphqVoEFRLBO88SsHMwLdMvWvHf7rHDQMkfThLzzqAF74JC7+hQ+j9U45gPzMCED0PnPSs8Dfigz0EkhhWG6IPDNJvWvQoyDkPkc1suGLsqVRNF6Coc8nfWU6q5GzOqV1198IA+dy/MRUAJ23vnGb9ps3Qwn/Al7ey7o2IgFoB3sqcFu7DWs4y2AAinh1Wv0XbSdgt67jNFZhoSc2pZdR2Ea7Ilr4qn1nzXNZJ4TUFSfUqinmpZGdC6nk63bly0iLmDivqOIpFrFGbMlo1JJOvLuA1XpkPyc0+MUFeVSFIoohLuEyts9xKZc3ByS3kwgf8Ym9WZbKnu2UklnpIv+KRDNCYi6udhLQ629s828wm6IMXCPIX+k78Lj2rdarmVEOcS/5M+ufM8TPHRGl61Y94KLD4ds= ceph-bd3e039e-0215-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-24T08:38:53.489 INFO:teuthology.orchestra.run.smithi189.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDADFsq18W8JTifbXybABKcXFLWR3x3ActmbY2eMvg0Vlu+3KM7De9oVuGJzL+RNfLtUqh7n0kYWRUJ7hNs11wySoNCaDxyPmtUix5yphqVoEFRLBO88SsHMwLdMvWvHf7rHDQMkfThLzzqAF74JC7+hQ+j9U45gPzMCED0PnPSs8Dfigz0EkhhWG6IPDNJvWvQoyDkPkc1suGLsqVRNF6Coc8nfWU6q5GzOqV1198IA+dy/MRUAJ23vnGb9ps3Qwn/Al7ey7o2IgFoB3sqcFu7DWs4y2AAinh1Wv0XbSdgt67jNFZhoSc2pZdR2Ea7Ilr4qn1nzXNZJ4TUFSfUqinmpZGdC6nk63bly0iLmDivqOIpFrFGbMlo1JJOvLuA1XpkPyc0+MUFeVSFIoohLuEyts9xKZc3ByS3kwgf8Ym9WZbKnu2UklnpIv+KRDNCYi6udhLQ629s828wm6IMXCPIX+k78Lj2rdarmVEOcS/5M+ufM8TPHRGl61Y94KLD4ds= ceph-bd3e039e-0215-11ef-bc93-c7b262605968 2024-04-24T08:38:53.618 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd3e039e-0215-11ef-bc93-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-04-24T08:38:54.032 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T08:38:56.929 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:38:56 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:38:56.930 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:38:56 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:38:56.930 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:38:56 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi083", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:38:56.930 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:38:56 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:38:56.930 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:38:56 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:38:56.930 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:38:56 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:38:57.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:38:56.982+0000 7f3e4ea00700 1 -- 172.21.15.83:0/529053876 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e48104790 msgr2=0x7f3e48104bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:38:57.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:38:56.982+0000 7f3e4ea00700 1 --2- 172.21.15.83:0/529053876 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e48104790 0x7f3e48104bb0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3e38009910 tx=0x7f3e38009c20 comp rx=0 tx=0).stop 2024-04-24T08:38:57.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:38:56.982+0000 7f3e4ea00700 1 -- 172.21.15.83:0/529053876 shutdown_connections 2024-04-24T08:38:57.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:38:56.982+0000 7f3e4ea00700 1 --2- 172.21.15.83:0/529053876 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e48104790 0x7f3e48104bb0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:38:57.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:38:56.982+0000 7f3e4ea00700 1 -- 172.21.15.83:0/529053876 >> 172.21.15.83:0/529053876 conn(0x7f3e480ffd30 msgr2=0x7f3e48102170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:38:57.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:38:56.982+0000 7f3e4ea00700 1 -- 172.21.15.83:0/529053876 shutdown_connections 2024-04-24T08:38:57.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:38:56.982+0000 7f3e4ea00700 1 -- 172.21.15.83:0/529053876 wait complete. 2024-04-24T08:38:57.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:38:56.983+0000 7f3e4ea00700 1 Processor -- start 2024-04-24T08:38:57.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:38:56.983+0000 7f3e4ea00700 1 -- start start 2024-04-24T08:38:57.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:38:56.984+0000 7f3e4ea00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e48104790 0x7f3e48199e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:38:57.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:38:56.984+0000 7f3e4ea00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3e4810a6d0 con 0x7f3e48104790 2024-04-24T08:38:57.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:38:56.984+0000 7f3e47e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e48104790 0x7f3e48199e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:38:57.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:38:56.984+0000 7f3e47e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e48104790 0x7f3e48199e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34356/0 (socket says 172.21.15.83:34356) 2024-04-24T08:38:57.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:38:56.984+0000 7f3e47e00700 1 -- 172.21.15.83:0/1980625270 learned_addr learned my addr 172.21.15.83:0/1980625270 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:38:57.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:38:56.984+0000 7f3e47e00700 1 -- 172.21.15.83:0/1980625270 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e4819a340 con 0x7f3e48104790 2024-04-24T08:38:57.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:38:56.985+0000 7f3e47e00700 1 --2- 172.21.15.83:0/1980625270 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e48104790 0x7f3e48199e00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3e38005800 tx=0x7f3e38009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:38:57.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:38:56.985+0000 7f3e44c00700 1 -- 172.21.15.83:0/1980625270 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3e38027070 con 0x7f3e48104790 2024-04-24T08:38:57.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:38:56.985+0000 7f3e44c00700 1 -- 172.21.15.83:0/1980625270 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e3800be10 con 0x7f3e48104790 2024-04-24T08:38:57.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:38:56.985+0000 7f3e44c00700 1 -- 172.21.15.83:0/1980625270 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3e3800f460 con 0x7f3e48104790 2024-04-24T08:38:57.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:38:56.985+0000 7f3e4ea00700 1 -- 172.21.15.83:0/1980625270 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e4819a5d0 con 0x7f3e48104790 2024-04-24T08:38:57.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:38:56.985+0000 7f3e4ea00700 1 -- 172.21.15.83:0/1980625270 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e4819d2c0 con 0x7f3e48104790 2024-04-24T08:38:57.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:38:56.986+0000 7f3e4ea00700 1 -- 172.21.15.83:0/1980625270 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e4810ae50 con 0x7f3e48104790 2024-04-24T08:38:57.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:38:56.987+0000 7f3e44c00700 1 -- 172.21.15.83:0/1980625270 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f3e3802a070 con 0x7f3e48104790 2024-04-24T08:38:57.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:38:56.987+0000 7f3e44c00700 1 --2- 172.21.15.83:0/1980625270 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f3e2c0396b0 0x7f3e2c03bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:38:57.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:38:56.987+0000 7f3e44c00700 1 -- 172.21.15.83:0/1980625270 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f3e3801e030 con 0x7f3e48104790 2024-04-24T08:38:57.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:38:56.988+0000 7f3e47400700 1 --2- 172.21.15.83:0/1980625270 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f3e2c0396b0 0x7f3e2c03bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:38:57.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:38:56.988+0000 7f3e47400700 1 --2- 172.21.15.83:0/1980625270 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f3e2c0396b0 0x7f3e2c03bb70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f3e34006fd0 tx=0x7f3e34006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:38:57.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:38:56.993+0000 7f3e44c00700 1 -- 172.21.15.83:0/1980625270 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3e38013070 con 0x7f3e48104790 2024-04-24T08:38:57.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:38:57.222+0000 7f3e4ea00700 1 -- 172.21.15.83:0/1980625270 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f3e4802e0f0 con 0x7f3e48104790 2024-04-24T08:38:57.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:38:57.227+0000 7f3e44c00700 1 -- 172.21.15.83:0/1980625270 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f3e380310c0 con 0x7f3e48104790 2024-04-24T08:38:57.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:38:57.230+0000 7f3e4ea00700 1 -- 172.21.15.83:0/1980625270 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f3e2c0396b0 msgr2=0x7f3e2c03bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:38:57.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:38:57.230+0000 7f3e4ea00700 1 --2- 172.21.15.83:0/1980625270 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f3e2c0396b0 0x7f3e2c03bb70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f3e34006fd0 tx=0x7f3e34006e40 comp rx=0 tx=0).stop 2024-04-24T08:38:57.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:38:57.230+0000 7f3e4ea00700 1 -- 172.21.15.83:0/1980625270 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e48104790 msgr2=0x7f3e48199e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:38:57.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:38:57.230+0000 7f3e4ea00700 1 --2- 172.21.15.83:0/1980625270 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e48104790 0x7f3e48199e00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3e38005800 tx=0x7f3e38009f60 comp rx=0 tx=0).stop 2024-04-24T08:38:57.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:38:57.231+0000 7f3e4ea00700 1 -- 172.21.15.83:0/1980625270 shutdown_connections 2024-04-24T08:38:57.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:38:57.231+0000 7f3e4ea00700 1 --2- 172.21.15.83:0/1980625270 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f3e2c0396b0 0x7f3e2c03bb70 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:38:57.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:38:57.231+0000 7f3e4ea00700 1 --2- 172.21.15.83:0/1980625270 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e48104790 0x7f3e48199e00 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:38:57.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:38:57.231+0000 7f3e4ea00700 1 -- 172.21.15.83:0/1980625270 >> 172.21.15.83:0/1980625270 conn(0x7f3e480ffd30 msgr2=0x7f3e481008e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:38:57.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:38:57.231+0000 7f3e4ea00700 1 -- 172.21.15.83:0/1980625270 shutdown_connections 2024-04-24T08:38:57.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:38:57.231+0000 7f3e4ea00700 1 -- 172.21.15.83:0/1980625270 wait complete. 2024-04-24T08:38:58.025 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-04-24T08:38:58.025 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd3e039e-0215-11ef-bc93-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-04-24T08:38:58.500 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:38:58 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/1980625270' entity='client.admin' 2024-04-24T08:38:58.500 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:38:58 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:38:58.500 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:38:58 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:38:59.196 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T08:39:01.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:01.142+0000 7eff37c00700 1 -- 172.21.15.83:0/1582605243 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff30103c60 msgr2=0x7eff30104080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:01.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:01.142+0000 7eff37c00700 1 --2- 172.21.15.83:0/1582605243 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff30103c60 0x7eff30104080 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7eff20009910 tx=0x7eff20009c20 comp rx=0 tx=0).stop 2024-04-24T08:39:01.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:01.142+0000 7eff37c00700 1 -- 172.21.15.83:0/1582605243 shutdown_connections 2024-04-24T08:39:01.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:01.142+0000 7eff37c00700 1 --2- 172.21.15.83:0/1582605243 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff30103c60 0x7eff30104080 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:01.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:01.142+0000 7eff37c00700 1 -- 172.21.15.83:0/1582605243 >> 172.21.15.83:0/1582605243 conn(0x7eff300fdb40 msgr2=0x7eff300fff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:39:01.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:01.143+0000 7eff37c00700 1 -- 172.21.15.83:0/1582605243 shutdown_connections 2024-04-24T08:39:01.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:01.143+0000 7eff37c00700 1 -- 172.21.15.83:0/1582605243 wait complete. 2024-04-24T08:39:01.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:01.144+0000 7eff37c00700 1 Processor -- start 2024-04-24T08:39:01.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:01.144+0000 7eff37c00700 1 -- start start 2024-04-24T08:39:01.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:01.144+0000 7eff37c00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff30103c60 0x7eff30199f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:01.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:01.144+0000 7eff37c00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7eff301085d0 con 0x7eff30103c60 2024-04-24T08:39:01.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:01.145+0000 7eff35200700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff30103c60 0x7eff30199f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:01.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:01.145+0000 7eff35200700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff30103c60 0x7eff30199f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54106/0 (socket says 172.21.15.83:54106) 2024-04-24T08:39:01.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:01.145+0000 7eff35200700 1 -- 172.21.15.83:0/548309704 learned_addr learned my addr 172.21.15.83:0/548309704 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:39:01.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:01.145+0000 7eff35200700 1 -- 172.21.15.83:0/548309704 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff3019a450 con 0x7eff30103c60 2024-04-24T08:39:01.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:01.145+0000 7eff35200700 1 --2- 172.21.15.83:0/548309704 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff30103c60 0x7eff30199f10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7eff20005950 tx=0x7eff20004fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:39:01.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:01.146+0000 7eff2e000700 1 -- 172.21.15.83:0/548309704 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7eff20026070 con 0x7eff30103c60 2024-04-24T08:39:01.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:01.146+0000 7eff2e000700 1 -- 172.21.15.83:0/548309704 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7eff2000bc00 con 0x7eff30103c60 2024-04-24T08:39:01.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:01.146+0000 7eff37c00700 1 -- 172.21.15.83:0/548309704 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff3019a6e0 con 0x7eff30103c60 2024-04-24T08:39:01.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:01.146+0000 7eff2e000700 1 -- 172.21.15.83:0/548309704 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7eff200039a0 con 0x7eff30103c60 2024-04-24T08:39:01.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:01.146+0000 7eff37c00700 1 -- 172.21.15.83:0/548309704 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff3019d380 con 0x7eff30103c60 2024-04-24T08:39:01.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:01.147+0000 7eff37c00700 1 -- 172.21.15.83:0/548309704 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff30108d50 con 0x7eff30103c60 2024-04-24T08:39:01.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:01.147+0000 7eff2e000700 1 -- 172.21.15.83:0/548309704 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7eff20029070 con 0x7eff30103c60 2024-04-24T08:39:01.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:01.148+0000 7eff2e000700 1 --2- 172.21.15.83:0/548309704 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7eff14039650 0x7eff1403bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:01.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:01.148+0000 7eff2e000700 1 -- 172.21.15.83:0/548309704 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7eff2001e070 con 0x7eff30103c60 2024-04-24T08:39:01.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:01.148+0000 7eff27e00700 1 --2- 172.21.15.83:0/548309704 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7eff14039650 0x7eff1403bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:01.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:01.149+0000 7eff27e00700 1 --2- 172.21.15.83:0/548309704 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7eff14039650 0x7eff1403bb10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7eff18006fd0 tx=0x7eff18006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:39:01.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:01.153+0000 7eff2e000700 1 -- 172.21.15.83:0/548309704 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7eff20013070 con 0x7eff30103c60 2024-04-24T08:39:01.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:01.384+0000 7eff37c00700 1 -- 172.21.15.83:0/548309704 --> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7eff30062f30 con 0x7eff14039650 2024-04-24T08:39:01.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:01.391+0000 7eff2e000700 1 -- 172.21.15.83:0/548309704 <== mgr.14152 v2:172.21.15.83:6800/2550826560 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7eff30062f30 con 0x7eff14039650 2024-04-24T08:39:01.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:01.393+0000 7eff37c00700 1 -- 172.21.15.83:0/548309704 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7eff14039650 msgr2=0x7eff1403bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:01.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:01.393+0000 7eff37c00700 1 --2- 172.21.15.83:0/548309704 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7eff14039650 0x7eff1403bb10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7eff18006fd0 tx=0x7eff18006e40 comp rx=0 tx=0).stop 2024-04-24T08:39:01.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:01.393+0000 7eff37c00700 1 -- 172.21.15.83:0/548309704 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff30103c60 msgr2=0x7eff30199f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:01.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:01.393+0000 7eff37c00700 1 --2- 172.21.15.83:0/548309704 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff30103c60 0x7eff30199f10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7eff20005950 tx=0x7eff20004fa0 comp rx=0 tx=0).stop 2024-04-24T08:39:01.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:01.393+0000 7eff37c00700 1 -- 172.21.15.83:0/548309704 shutdown_connections 2024-04-24T08:39:01.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:01.393+0000 7eff37c00700 1 --2- 172.21.15.83:0/548309704 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7eff14039650 0x7eff1403bb10 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:01.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:01.394+0000 7eff37c00700 1 --2- 172.21.15.83:0/548309704 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff30103c60 0x7eff30199f10 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:01.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:01.394+0000 7eff37c00700 1 -- 172.21.15.83:0/548309704 >> 172.21.15.83:0/548309704 conn(0x7eff300fdb40 msgr2=0x7eff300fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:39:01.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:01.394+0000 7eff37c00700 1 -- 172.21.15.83:0/548309704 shutdown_connections 2024-04-24T08:39:01.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:01.394+0000 7eff37c00700 1 -- 172.21.15.83:0/548309704 wait complete. 2024-04-24T08:39:02.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:01 smithi083 ceph-mon[27907]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:39:02.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:01 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:39:02.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:01 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:39:02.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:01 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:39:02.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:01 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:39:02.183 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi138 2024-04-24T08:39:02.183 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-24T08:39:02.183 DEBUG:teuthology.orchestra.run.smithi138:> dd of=/etc/ceph/ceph.conf 2024-04-24T08:39:02.239 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-24T08:39:02.239 DEBUG:teuthology.orchestra.run.smithi138:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-24T08:39:02.333 INFO:tasks.cephadm:Adding host smithi138 to orchestrator... 2024-04-24T08:39:02.333 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd3e039e-0215-11ef-bc93-c7b262605968 -- ceph orch host add smithi138 2024-04-24T08:39:02.764 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T08:39:03.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:02 smithi083 ceph-mon[27907]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:39:03.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:02 smithi083 ceph-mon[27907]: Updating smithi083:/etc/ceph/ceph.conf 2024-04-24T08:39:04.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:03 smithi083 ceph-mon[27907]: Updating smithi083:/var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T08:39:04.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:03 smithi083 ceph-mon[27907]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:39:04.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:03 smithi083 ceph-mon[27907]: Updating smithi083:/etc/ceph/ceph.client.admin.keyring 2024-04-24T08:39:05.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:05.034+0000 7fc6dd800700 1 -- 172.21.15.83:0/1899675105 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6d81010f0 msgr2=0x7fc6d81014d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:05.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:05.034+0000 7fc6dd800700 1 --2- 172.21.15.83:0/1899675105 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6d81010f0 0x7fc6d81014d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc6c8009910 tx=0x7fc6c8009c20 comp rx=0 tx=0).stop 2024-04-24T08:39:05.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:05.034+0000 7fc6dd800700 1 -- 172.21.15.83:0/1899675105 shutdown_connections 2024-04-24T08:39:05.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:05.034+0000 7fc6dd800700 1 --2- 172.21.15.83:0/1899675105 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6d81010f0 0x7fc6d81014d0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:05.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:05.034+0000 7fc6dd800700 1 -- 172.21.15.83:0/1899675105 >> 172.21.15.83:0/1899675105 conn(0x7fc6d80fcd70 msgr2=0x7fc6d80ff190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:39:05.035 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:04 smithi083 ceph-mon[27907]: Updating smithi083:/var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T08:39:05.035 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:04 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:39:05.035 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:04 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:39:05.035 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:04 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:39:05.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:05.034+0000 7fc6dd800700 1 -- 172.21.15.83:0/1899675105 shutdown_connections 2024-04-24T08:39:05.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:05.034+0000 7fc6dd800700 1 -- 172.21.15.83:0/1899675105 wait complete. 2024-04-24T08:39:05.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:05.035+0000 7fc6dd800700 1 Processor -- start 2024-04-24T08:39:05.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:05.036+0000 7fc6dd800700 1 -- start start 2024-04-24T08:39:05.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:05.036+0000 7fc6dd800700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6d81010f0 0x7fc6d8199f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:05.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:05.036+0000 7fc6dd800700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc6d810eab0 con 0x7fc6d81010f0 2024-04-24T08:39:05.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:05.036+0000 7fc6d7e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6d81010f0 0x7fc6d8199f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:05.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:05.037+0000 7fc6d7e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6d81010f0 0x7fc6d8199f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54120/0 (socket says 172.21.15.83:54120) 2024-04-24T08:39:05.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:05.037+0000 7fc6d7e00700 1 -- 172.21.15.83:0/575950679 learned_addr learned my addr 172.21.15.83:0/575950679 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:39:05.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:05.037+0000 7fc6d7e00700 1 -- 172.21.15.83:0/575950679 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6d819a4d0 con 0x7fc6d81010f0 2024-04-24T08:39:05.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:05.037+0000 7fc6d7e00700 1 --2- 172.21.15.83:0/575950679 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6d81010f0 0x7fc6d8199f90 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc6c8006010 tx=0x7fc6c8005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:39:05.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:05.037+0000 7fc6d4c00700 1 -- 172.21.15.83:0/575950679 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc6c8026070 con 0x7fc6d81010f0 2024-04-24T08:39:05.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:05.037+0000 7fc6d4c00700 1 -- 172.21.15.83:0/575950679 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc6c800bc00 con 0x7fc6d81010f0 2024-04-24T08:39:05.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:05.038+0000 7fc6d4c00700 1 -- 172.21.15.83:0/575950679 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc6c8003930 con 0x7fc6d81010f0 2024-04-24T08:39:05.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:05.038+0000 7fc6dd800700 1 -- 172.21.15.83:0/575950679 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6d819a760 con 0x7fc6d81010f0 2024-04-24T08:39:05.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:05.038+0000 7fc6dd800700 1 -- 172.21.15.83:0/575950679 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6d819abc0 con 0x7fc6d81010f0 2024-04-24T08:39:05.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:05.039+0000 7fc6dd800700 1 -- 172.21.15.83:0/575950679 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6d806b040 con 0x7fc6d81010f0 2024-04-24T08:39:05.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:05.039+0000 7fc6d4c00700 1 -- 172.21.15.83:0/575950679 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fc6c8029070 con 0x7fc6d81010f0 2024-04-24T08:39:05.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:05.040+0000 7fc6d4c00700 1 --2- 172.21.15.83:0/575950679 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fc6bc0396b0 0x7fc6bc03bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:05.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:05.040+0000 7fc6d4c00700 1 -- 172.21.15.83:0/575950679 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc6c801e070 con 0x7fc6d81010f0 2024-04-24T08:39:05.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:05.040+0000 7fc6d7400700 1 --2- 172.21.15.83:0/575950679 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fc6bc0396b0 0x7fc6bc03bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:05.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:05.041+0000 7fc6d7400700 1 --2- 172.21.15.83:0/575950679 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fc6bc0396b0 0x7fc6bc03bb70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc6c4006fd0 tx=0x7fc6c4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:39:05.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:05.045+0000 7fc6d4c00700 1 -- 172.21.15.83:0/575950679 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc6c8013070 con 0x7fc6d81010f0 2024-04-24T08:39:05.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:05.281+0000 7fc6dd800700 1 -- 172.21.15.83:0/575950679 --> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi138", "target": ["mon-mgr", ""]}) -- 0x7fc6d819b2f0 con 0x7fc6bc0396b0 2024-04-24T08:39:06.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:05 smithi083 ceph-mon[27907]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:39:07.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:06 smithi083 ceph-mon[27907]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi138", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:39:08.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:07 smithi083 ceph-mon[27907]: Deploying cephadm binary to smithi138 2024-04-24T08:39:08.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:07 smithi083 ceph-mon[27907]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:39:09.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:09 smithi083 ceph-mon[27907]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:39:11.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:11.542+0000 7fc6d4c00700 1 -- 172.21.15.83:0/575950679 <== mgr.14152 v2:172.21.15.83:6800/2550826560 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fc6d819b2f0 con 0x7fc6bc0396b0 2024-04-24T08:39:11.545 INFO:teuthology.orchestra.run.smithi083.stdout:Added host 'smithi138' with addr '172.21.15.138' 2024-04-24T08:39:11.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:11.546+0000 7fc6dd800700 1 -- 172.21.15.83:0/575950679 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fc6bc0396b0 msgr2=0x7fc6bc03bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:11.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:11.546+0000 7fc6dd800700 1 --2- 172.21.15.83:0/575950679 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fc6bc0396b0 0x7fc6bc03bb70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc6c4006fd0 tx=0x7fc6c4006e40 comp rx=0 tx=0).stop 2024-04-24T08:39:11.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:11.546+0000 7fc6dd800700 1 -- 172.21.15.83:0/575950679 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6d81010f0 msgr2=0x7fc6d8199f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:11.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:11.546+0000 7fc6dd800700 1 --2- 172.21.15.83:0/575950679 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6d81010f0 0x7fc6d8199f90 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc6c8006010 tx=0x7fc6c8005780 comp rx=0 tx=0).stop 2024-04-24T08:39:11.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:11.547+0000 7fc6dd800700 1 -- 172.21.15.83:0/575950679 shutdown_connections 2024-04-24T08:39:11.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:11.547+0000 7fc6dd800700 1 --2- 172.21.15.83:0/575950679 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fc6bc0396b0 0x7fc6bc03bb70 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:11.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:11.547+0000 7fc6dd800700 1 --2- 172.21.15.83:0/575950679 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6d81010f0 0x7fc6d8199f90 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:11.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:11.547+0000 7fc6dd800700 1 -- 172.21.15.83:0/575950679 >> 172.21.15.83:0/575950679 conn(0x7fc6d80fcd70 msgr2=0x7fc6d80fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:39:11.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:11.547+0000 7fc6dd800700 1 -- 172.21.15.83:0/575950679 shutdown_connections 2024-04-24T08:39:11.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:11.547+0000 7fc6dd800700 1 -- 172.21.15.83:0/575950679 wait complete. 2024-04-24T08:39:11.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:11 smithi083 ceph-mon[27907]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:39:11.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:11 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:39:11.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:11 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:39:12.464 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd3e039e-0215-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-24T08:39:12.906 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T08:39:13.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:12 smithi083 ceph-mon[27907]: Added host smithi138 2024-04-24T08:39:13.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:12 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:39:14.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:13 smithi083 ceph-mon[27907]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:39:14.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:13 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:39:15.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:15.011+0000 7f2181600700 1 -- 172.21.15.83:0/3258082710 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f217c10a710 msgr2=0x7f217c10aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:15.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:15.011+0000 7f2181600700 1 --2- 172.21.15.83:0/3258082710 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f217c10a710 0x7f217c10aaf0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f216c009910 tx=0x7f216c009c20 comp rx=0 tx=0).stop 2024-04-24T08:39:15.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:15.011+0000 7f2181600700 1 -- 172.21.15.83:0/3258082710 shutdown_connections 2024-04-24T08:39:15.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:15.011+0000 7f2181600700 1 --2- 172.21.15.83:0/3258082710 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f217c10a710 0x7f217c10aaf0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:15.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:15.011+0000 7f2181600700 1 -- 172.21.15.83:0/3258082710 >> 172.21.15.83:0/3258082710 conn(0x7f217c100240 msgr2=0x7f217c102660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:39:15.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:15.011+0000 7f2181600700 1 -- 172.21.15.83:0/3258082710 shutdown_connections 2024-04-24T08:39:15.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:15.011+0000 7f2181600700 1 -- 172.21.15.83:0/3258082710 wait complete. 2024-04-24T08:39:15.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:15.013+0000 7f2181600700 1 Processor -- start 2024-04-24T08:39:15.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:15.013+0000 7f2181600700 1 -- start start 2024-04-24T08:39:15.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:15.013+0000 7f2181600700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f217c10a710 0x7f217c19a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:15.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:15.013+0000 7f2181600700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f217c10ec00 con 0x7f217c10a710 2024-04-24T08:39:15.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:15.014+0000 7f217aa00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f217c10a710 0x7f217c19a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:15.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:15.014+0000 7f217aa00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f217c10a710 0x7f217c19a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56026/0 (socket says 172.21.15.83:56026) 2024-04-24T08:39:15.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:15.014+0000 7f217aa00700 1 -- 172.21.15.83:0/869550034 learned_addr learned my addr 172.21.15.83:0/869550034 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:39:15.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:15.014+0000 7f217aa00700 1 -- 172.21.15.83:0/869550034 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f217c19a740 con 0x7f217c10a710 2024-04-24T08:39:15.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:15.014+0000 7f217aa00700 1 --2- 172.21.15.83:0/869550034 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f217c10a710 0x7f217c19a200 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f216c0056d0 tx=0x7f216c0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:39:15.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:15.015+0000 7f2173400700 1 -- 172.21.15.83:0/869550034 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f216c026070 con 0x7f217c10a710 2024-04-24T08:39:15.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:15.015+0000 7f2173400700 1 -- 172.21.15.83:0/869550034 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f216c00bba0 con 0x7f217c10a710 2024-04-24T08:39:15.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:15.015+0000 7f2181600700 1 -- 172.21.15.83:0/869550034 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f217c19aa30 con 0x7f217c10a710 2024-04-24T08:39:15.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:15.015+0000 7f2173400700 1 -- 172.21.15.83:0/869550034 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f216c0037f0 con 0x7f217c10a710 2024-04-24T08:39:15.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:15.015+0000 7f2181600700 1 -- 172.21.15.83:0/869550034 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f217c19ea80 con 0x7f217c10a710 2024-04-24T08:39:15.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:15.016+0000 7f2173400700 1 -- 172.21.15.83:0/869550034 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f216c029070 con 0x7f217c10a710 2024-04-24T08:39:15.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:15.016+0000 7f2181600700 1 -- 172.21.15.83:0/869550034 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f217c105e50 con 0x7f217c10a710 2024-04-24T08:39:15.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:15.017+0000 7f2173400700 1 --2- 172.21.15.83:0/869550034 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f21600396b0 0x7f216003bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:15.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:15.017+0000 7f2173400700 1 -- 172.21.15.83:0/869550034 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f216c01e070 con 0x7f217c10a710 2024-04-24T08:39:15.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:15.017+0000 7f217a000700 1 --2- 172.21.15.83:0/869550034 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f21600396b0 0x7f216003bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:15.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:15.018+0000 7f217a000700 1 --2- 172.21.15.83:0/869550034 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f21600396b0 0x7f216003bb70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2168006fd0 tx=0x7f2168006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:39:15.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:15.022+0000 7f2173400700 1 -- 172.21.15.83:0/869550034 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f216c013070 con 0x7f217c10a710 2024-04-24T08:39:15.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:15.255+0000 7f2181600700 1 -- 172.21.15.83:0/869550034 --> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f217c19b510 con 0x7f21600396b0 2024-04-24T08:39:15.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:15.258+0000 7f2173400700 1 -- 172.21.15.83:0/869550034 <== mgr.14152 v2:172.21.15.83:6800/2550826560 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+160 (secure 0 0 0) 0x7f217c19b510 con 0x7f21600396b0 2024-04-24T08:39:15.258 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:39:15.258 INFO:teuthology.orchestra.run.smithi083.stdout:[{"addr": "172.21.15.83", "hostname": "smithi083", "labels": [], "status": ""}, {"addr": "172.21.15.138", "hostname": "smithi138", "labels": [], "status": ""}] 2024-04-24T08:39:15.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:15.261+0000 7f2181600700 1 -- 172.21.15.83:0/869550034 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f21600396b0 msgr2=0x7f216003bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:15.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:15.261+0000 7f2181600700 1 --2- 172.21.15.83:0/869550034 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f21600396b0 0x7f216003bb70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2168006fd0 tx=0x7f2168006e40 comp rx=0 tx=0).stop 2024-04-24T08:39:15.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:15.261+0000 7f2181600700 1 -- 172.21.15.83:0/869550034 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f217c10a710 msgr2=0x7f217c19a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:15.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:15.261+0000 7f2181600700 1 --2- 172.21.15.83:0/869550034 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f217c10a710 0x7f217c19a200 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f216c0056d0 tx=0x7f216c0057b0 comp rx=0 tx=0).stop 2024-04-24T08:39:15.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:15.261+0000 7f2181600700 1 -- 172.21.15.83:0/869550034 shutdown_connections 2024-04-24T08:39:15.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:15.261+0000 7f2181600700 1 --2- 172.21.15.83:0/869550034 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f21600396b0 0x7f216003bb70 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:15.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:15.261+0000 7f2181600700 1 --2- 172.21.15.83:0/869550034 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f217c10a710 0x7f217c19a200 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:15.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:15.261+0000 7f2181600700 1 -- 172.21.15.83:0/869550034 >> 172.21.15.83:0/869550034 conn(0x7f217c100240 msgr2=0x7f217c101040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:39:15.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:15.261+0000 7f2181600700 1 -- 172.21.15.83:0/869550034 shutdown_connections 2024-04-24T08:39:15.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:15.261+0000 7f2181600700 1 -- 172.21.15.83:0/869550034 wait complete. 2024-04-24T08:39:15.988 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:15 smithi083 ceph-mon[27907]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:39:15.988 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:15 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:39:16.056 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi189 2024-04-24T08:39:16.056 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-04-24T08:39:16.056 DEBUG:teuthology.orchestra.run.smithi189:> dd of=/etc/ceph/ceph.conf 2024-04-24T08:39:16.115 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-04-24T08:39:16.115 DEBUG:teuthology.orchestra.run.smithi189:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-24T08:39:16.212 INFO:tasks.cephadm:Adding host smithi189 to orchestrator... 2024-04-24T08:39:16.212 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd3e039e-0215-11ef-bc93-c7b262605968 -- ceph orch host add smithi189 2024-04-24T08:39:16.641 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T08:39:17.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:16 smithi083 ceph-mon[27907]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T08:39:17.802 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:17 smithi083 ceph-mon[27907]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:39:18.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:18.648+0000 7f7177200700 1 -- 172.21.15.83:0/1398449643 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71701085d0 msgr2=0x7f71701089b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:18.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:18.648+0000 7f7177200700 1 --2- 172.21.15.83:0/1398449643 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71701085d0 0x7f71701089b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7160009910 tx=0x7f7160009c20 comp rx=0 tx=0).stop 2024-04-24T08:39:18.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:18.649+0000 7f7177200700 1 -- 172.21.15.83:0/1398449643 shutdown_connections 2024-04-24T08:39:18.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:18.649+0000 7f7177200700 1 --2- 172.21.15.83:0/1398449643 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71701085d0 0x7f71701089b0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:18.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:18.649+0000 7f7177200700 1 -- 172.21.15.83:0/1398449643 >> 172.21.15.83:0/1398449643 conn(0x7f71700fe030 msgr2=0x7f7170100450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:39:18.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:18.649+0000 7f7177200700 1 -- 172.21.15.83:0/1398449643 shutdown_connections 2024-04-24T08:39:18.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:18.649+0000 7f7177200700 1 -- 172.21.15.83:0/1398449643 wait complete. 2024-04-24T08:39:18.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:18.650+0000 7f7177200700 1 Processor -- start 2024-04-24T08:39:18.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:18.650+0000 7f7177200700 1 -- start start 2024-04-24T08:39:18.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:18.651+0000 7f7177200700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71701085d0 0x7f7170197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:18.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:18.651+0000 7f7177200700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f717010ca40 con 0x7f71701085d0 2024-04-24T08:39:18.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:18.651+0000 7f716fe00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71701085d0 0x7f7170197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:18.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:18.651+0000 7f716fe00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71701085d0 0x7f7170197ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56048/0 (socket says 172.21.15.83:56048) 2024-04-24T08:39:18.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:18.651+0000 7f716fe00700 1 -- 172.21.15.83:0/3304788915 learned_addr learned my addr 172.21.15.83:0/3304788915 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:39:18.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:18.651+0000 7f716fe00700 1 -- 172.21.15.83:0/3304788915 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7170198430 con 0x7f71701085d0 2024-04-24T08:39:18.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:18.652+0000 7f716fe00700 1 --2- 172.21.15.83:0/3304788915 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71701085d0 0x7f7170197ef0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7160006010 tx=0x7f7160005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:39:18.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:18.652+0000 7f716d600700 1 -- 172.21.15.83:0/3304788915 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7160026070 con 0x7f71701085d0 2024-04-24T08:39:18.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:18.652+0000 7f716d600700 1 -- 172.21.15.83:0/3304788915 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f716000bc00 con 0x7f71701085d0 2024-04-24T08:39:18.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:18.652+0000 7f7177200700 1 -- 172.21.15.83:0/3304788915 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71701986c0 con 0x7f71701085d0 2024-04-24T08:39:18.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:18.652+0000 7f716d600700 1 -- 172.21.15.83:0/3304788915 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7160003930 con 0x7f71701085d0 2024-04-24T08:39:18.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:18.652+0000 7f7177200700 1 -- 172.21.15.83:0/3304788915 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f717019c7c0 con 0x7f71701085d0 2024-04-24T08:39:18.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:18.653+0000 7f716d600700 1 -- 172.21.15.83:0/3304788915 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f7160029070 con 0x7f71701085d0 2024-04-24T08:39:18.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:18.653+0000 7f7177200700 1 -- 172.21.15.83:0/3304788915 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7170103c40 con 0x7f71701085d0 2024-04-24T08:39:18.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:18.654+0000 7f716d600700 1 --2- 172.21.15.83:0/3304788915 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f71540396b0 0x7f715403bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:18.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:18.654+0000 7f716d600700 1 -- 172.21.15.83:0/3304788915 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f716001e070 con 0x7f71701085d0 2024-04-24T08:39:18.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:18.654+0000 7f7167400700 1 --2- 172.21.15.83:0/3304788915 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f71540396b0 0x7f715403bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:18.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:18.655+0000 7f7167400700 1 --2- 172.21.15.83:0/3304788915 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f71540396b0 0x7f715403bb70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f7158006fd0 tx=0x7f7158006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:39:18.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:18.659+0000 7f716d600700 1 -- 172.21.15.83:0/3304788915 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7160013070 con 0x7f71701085d0 2024-04-24T08:39:18.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:18.891+0000 7f7177200700 1 -- 172.21.15.83:0/3304788915 --> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi189", "target": ["mon-mgr", ""]}) -- 0x7f7170199250 con 0x7f71540396b0 2024-04-24T08:39:20.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:19 smithi083 ceph-mon[27907]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:39:20.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:19 smithi083 ceph-mon[27907]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi189", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:39:21.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:21 smithi083 ceph-mon[27907]: Deploying cephadm binary to smithi189 2024-04-24T08:39:21.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:21 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:39:21.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:21 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:39:21.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:21 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:39:21.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:21 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:39:21.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:21 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi138", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:39:21.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:21 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:39:21.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:21 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:39:21.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:21 smithi083 ceph-mon[27907]: Updating smithi138:/etc/ceph/ceph.conf 2024-04-24T08:39:21.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:21 smithi083 ceph-mon[27907]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:39:24.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:23 smithi083 ceph-mon[27907]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:39:25.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:25.023+0000 7f716d600700 1 -- 172.21.15.83:0/3304788915 <== mgr.14152 v2:172.21.15.83:6800/2550826560 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f7170199250 con 0x7f71540396b0 2024-04-24T08:39:25.024 INFO:teuthology.orchestra.run.smithi083.stdout:Added host 'smithi189' with addr '172.21.15.189' 2024-04-24T08:39:25.025 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:24 smithi083 ceph-mon[27907]: Updating smithi138:/var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T08:39:25.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:25.026+0000 7f7177200700 1 -- 172.21.15.83:0/3304788915 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f71540396b0 msgr2=0x7f715403bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:25.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:25.026+0000 7f7177200700 1 --2- 172.21.15.83:0/3304788915 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f71540396b0 0x7f715403bb70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f7158006fd0 tx=0x7f7158006e40 comp rx=0 tx=0).stop 2024-04-24T08:39:25.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:25.026+0000 7f7177200700 1 -- 172.21.15.83:0/3304788915 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71701085d0 msgr2=0x7f7170197ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:25.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:25.026+0000 7f7177200700 1 --2- 172.21.15.83:0/3304788915 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71701085d0 0x7f7170197ef0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7160006010 tx=0x7f7160005780 comp rx=0 tx=0).stop 2024-04-24T08:39:25.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:25.026+0000 7f7177200700 1 -- 172.21.15.83:0/3304788915 shutdown_connections 2024-04-24T08:39:25.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:25.027+0000 7f7177200700 1 --2- 172.21.15.83:0/3304788915 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f71540396b0 0x7f715403bb70 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:25.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:25.027+0000 7f7177200700 1 --2- 172.21.15.83:0/3304788915 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71701085d0 0x7f7170197ef0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:25.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:25.027+0000 7f7177200700 1 -- 172.21.15.83:0/3304788915 >> 172.21.15.83:0/3304788915 conn(0x7f71700fe030 msgr2=0x7f71700ff8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:39:25.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:25.027+0000 7f7177200700 1 -- 172.21.15.83:0/3304788915 shutdown_connections 2024-04-24T08:39:25.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:25.027+0000 7f7177200700 1 -- 172.21.15.83:0/3304788915 wait complete. 2024-04-24T08:39:25.825 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd3e039e-0215-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-24T08:39:26.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:25 smithi083 ceph-mon[27907]: Updating smithi138:/etc/ceph/ceph.client.admin.keyring 2024-04-24T08:39:26.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:25 smithi083 ceph-mon[27907]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:39:26.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:25 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:39:26.244 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T08:39:27.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:26 smithi083 ceph-mon[27907]: Added host smithi189 2024-04-24T08:39:27.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:26 smithi083 ceph-mon[27907]: Updating smithi138:/var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T08:39:27.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:26 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:39:27.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:26 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:39:27.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:26 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:39:27.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:26 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:39:27.884 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:27 smithi083 ceph-mon[27907]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:39:27.884 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:27 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:39:28.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:28.387+0000 7f4b10800700 1 -- 172.21.15.83:0/3779088534 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b08074ec0 msgr2=0x7f4b080752a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:28.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:28.387+0000 7f4b10800700 1 --2- 172.21.15.83:0/3779088534 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b08074ec0 0x7f4b080752a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4afc009910 tx=0x7f4afc009c20 comp rx=0 tx=0).stop 2024-04-24T08:39:28.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:28.387+0000 7f4b10800700 1 -- 172.21.15.83:0/3779088534 shutdown_connections 2024-04-24T08:39:28.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:28.387+0000 7f4b10800700 1 --2- 172.21.15.83:0/3779088534 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b08074ec0 0x7f4b080752a0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:28.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:28.387+0000 7f4b10800700 1 -- 172.21.15.83:0/3779088534 >> 172.21.15.83:0/3779088534 conn(0x7f4b080fe030 msgr2=0x7f4b08100450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:39:28.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:28.387+0000 7f4b10800700 1 -- 172.21.15.83:0/3779088534 shutdown_connections 2024-04-24T08:39:28.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:28.387+0000 7f4b10800700 1 -- 172.21.15.83:0/3779088534 wait complete. 2024-04-24T08:39:28.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:28.388+0000 7f4b10800700 1 Processor -- start 2024-04-24T08:39:28.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:28.389+0000 7f4b10800700 1 -- start start 2024-04-24T08:39:28.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:28.389+0000 7f4b10800700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b08074ec0 0x7f4b0819a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:28.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:28.389+0000 7f4b10800700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4b0810ec50 con 0x7f4b08074ec0 2024-04-24T08:39:28.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:28.389+0000 7f4b0de00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b08074ec0 0x7f4b0819a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:28.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:28.389+0000 7f4b0de00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b08074ec0 0x7f4b0819a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60188/0 (socket says 172.21.15.83:60188) 2024-04-24T08:39:28.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:28.389+0000 7f4b0de00700 1 -- 172.21.15.83:0/36347794 learned_addr learned my addr 172.21.15.83:0/36347794 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:39:28.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:28.390+0000 7f4b0de00700 1 -- 172.21.15.83:0/36347794 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b0819a6e0 con 0x7f4b08074ec0 2024-04-24T08:39:28.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:28.390+0000 7f4b0de00700 1 --2- 172.21.15.83:0/36347794 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b08074ec0 0x7f4b0819a1a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4afc006010 tx=0x7f4afc005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:39:28.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:28.390+0000 7f4b02a00700 1 -- 172.21.15.83:0/36347794 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4afc026070 con 0x7f4b08074ec0 2024-04-24T08:39:28.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:28.391+0000 7f4b02a00700 1 -- 172.21.15.83:0/36347794 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4afc00bc00 con 0x7f4b08074ec0 2024-04-24T08:39:28.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:28.391+0000 7f4b10800700 1 -- 172.21.15.83:0/36347794 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b0819a970 con 0x7f4b08074ec0 2024-04-24T08:39:28.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:28.391+0000 7f4b02a00700 1 -- 172.21.15.83:0/36347794 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4afc003910 con 0x7f4b08074ec0 2024-04-24T08:39:28.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:28.391+0000 7f4b10800700 1 -- 172.21.15.83:0/36347794 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b0819add0 con 0x7f4b08074ec0 2024-04-24T08:39:28.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:28.392+0000 7f4b02a00700 1 -- 172.21.15.83:0/36347794 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f4afc029070 con 0x7f4b08074ec0 2024-04-24T08:39:28.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:28.392+0000 7f4b10800700 1 -- 172.21.15.83:0/36347794 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b08076570 con 0x7f4b08074ec0 2024-04-24T08:39:28.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:28.392+0000 7f4b02a00700 1 --2- 172.21.15.83:0/36347794 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f4af00396b0 0x7f4af003bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:28.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:28.392+0000 7f4b02a00700 1 -- 172.21.15.83:0/36347794 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f4afc01e070 con 0x7f4b08074ec0 2024-04-24T08:39:28.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:28.393+0000 7f4b0d400700 1 --2- 172.21.15.83:0/36347794 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f4af00396b0 0x7f4af003bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:28.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:28.393+0000 7f4b0d400700 1 --2- 172.21.15.83:0/36347794 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f4af00396b0 0x7f4af003bb70 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f4af8006fd0 tx=0x7f4af8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:39:28.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:28.398+0000 7f4b02a00700 1 -- 172.21.15.83:0/36347794 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4afc013070 con 0x7f4b08074ec0 2024-04-24T08:39:28.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:28.632+0000 7f4b10800700 1 -- 172.21.15.83:0/36347794 --> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4b0819b570 con 0x7f4af00396b0 2024-04-24T08:39:28.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:28.634+0000 7f4b02a00700 1 -- 172.21.15.83:0/36347794 <== mgr.14152 v2:172.21.15.83:6800/2550826560 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+240 (secure 0 0 0) 0x7f4b0819b570 con 0x7f4af00396b0 2024-04-24T08:39:28.635 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:39:28.635 INFO:teuthology.orchestra.run.smithi083.stdout:[{"addr": "172.21.15.83", "hostname": "smithi083", "labels": [], "status": ""}, {"addr": "172.21.15.138", "hostname": "smithi138", "labels": [], "status": ""}, {"addr": "172.21.15.189", "hostname": "smithi189", "labels": [], "status": ""}] 2024-04-24T08:39:28.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:28.637+0000 7f4b10800700 1 -- 172.21.15.83:0/36347794 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f4af00396b0 msgr2=0x7f4af003bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:28.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:28.637+0000 7f4b10800700 1 --2- 172.21.15.83:0/36347794 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f4af00396b0 0x7f4af003bb70 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f4af8006fd0 tx=0x7f4af8006e40 comp rx=0 tx=0).stop 2024-04-24T08:39:28.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:28.637+0000 7f4b10800700 1 -- 172.21.15.83:0/36347794 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b08074ec0 msgr2=0x7f4b0819a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:28.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:28.637+0000 7f4b10800700 1 --2- 172.21.15.83:0/36347794 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b08074ec0 0x7f4b0819a1a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4afc006010 tx=0x7f4afc005780 comp rx=0 tx=0).stop 2024-04-24T08:39:28.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:28.637+0000 7f4b10800700 1 -- 172.21.15.83:0/36347794 shutdown_connections 2024-04-24T08:39:28.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:28.637+0000 7f4b10800700 1 --2- 172.21.15.83:0/36347794 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f4af00396b0 0x7f4af003bb70 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:28.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:28.637+0000 7f4b10800700 1 --2- 172.21.15.83:0/36347794 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b08074ec0 0x7f4b0819a1a0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:28.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:28.637+0000 7f4b10800700 1 -- 172.21.15.83:0/36347794 >> 172.21.15.83:0/36347794 conn(0x7f4b080fe030 msgr2=0x7f4b08106800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:39:28.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:28.637+0000 7f4b10800700 1 -- 172.21.15.83:0/36347794 shutdown_connections 2024-04-24T08:39:28.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:28.638+0000 7f4b10800700 1 -- 172.21.15.83:0/36347794 wait complete. 2024-04-24T08:39:29.388 INFO:tasks.cephadm:Setting crush tunables to default 2024-04-24T08:39:29.389 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd3e039e-0215-11ef-bc93-c7b262605968 -- ceph osd crush tunables default 2024-04-24T08:39:29.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:29 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:39:29.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:29 smithi083 ceph-mon[27907]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T08:39:29.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:29 smithi083 ceph-mon[27907]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:39:29.835 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T08:39:31.416 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:31 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:39:31.416 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:31 smithi083 ceph-mon[27907]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:39:31.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:31.943+0000 7fb840800700 1 -- 172.21.15.83:0/1366869731 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb83810daa0 msgr2=0x7fb83810de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:31.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:31.943+0000 7fb840800700 1 --2- 172.21.15.83:0/1366869731 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb83810daa0 0x7fb83810de80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb82c009910 tx=0x7fb82c009c20 comp rx=0 tx=0).stop 2024-04-24T08:39:31.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:31.944+0000 7fb840800700 1 -- 172.21.15.83:0/1366869731 shutdown_connections 2024-04-24T08:39:31.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:31.944+0000 7fb840800700 1 --2- 172.21.15.83:0/1366869731 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb83810daa0 0x7fb83810de80 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:31.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:31.944+0000 7fb840800700 1 -- 172.21.15.83:0/1366869731 >> 172.21.15.83:0/1366869731 conn(0x7fb838078290 msgr2=0x7fb83807a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:39:31.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:31.944+0000 7fb840800700 1 -- 172.21.15.83:0/1366869731 shutdown_connections 2024-04-24T08:39:31.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:31.944+0000 7fb840800700 1 -- 172.21.15.83:0/1366869731 wait complete. 2024-04-24T08:39:31.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:31.945+0000 7fb840800700 1 Processor -- start 2024-04-24T08:39:31.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:31.945+0000 7fb840800700 1 -- start start 2024-04-24T08:39:31.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:31.946+0000 7fb840800700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb83810daa0 0x7fb83819e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:31.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:31.946+0000 7fb840800700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb838113010 con 0x7fb83810daa0 2024-04-24T08:39:31.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:31.946+0000 7fb83de00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb83810daa0 0x7fb83819e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:31.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:31.946+0000 7fb83de00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb83810daa0 0x7fb83819e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46200/0 (socket says 172.21.15.83:46200) 2024-04-24T08:39:31.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:31.946+0000 7fb83de00700 1 -- 172.21.15.83:0/1482720420 learned_addr learned my addr 172.21.15.83:0/1482720420 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:39:31.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:31.947+0000 7fb83de00700 1 -- 172.21.15.83:0/1482720420 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb83819eaf0 con 0x7fb83810daa0 2024-04-24T08:39:31.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:31.947+0000 7fb83de00700 1 --2- 172.21.15.83:0/1482720420 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb83810daa0 0x7fb83819e5b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb82c0056d0 tx=0x7fb82c0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:39:31.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:31.947+0000 7fb832a00700 1 -- 172.21.15.83:0/1482720420 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb82c026070 con 0x7fb83810daa0 2024-04-24T08:39:31.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:31.947+0000 7fb832a00700 1 -- 172.21.15.83:0/1482720420 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb82c004370 con 0x7fb83810daa0 2024-04-24T08:39:31.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:31.947+0000 7fb832a00700 1 -- 172.21.15.83:0/1482720420 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb82c0039b0 con 0x7fb83810daa0 2024-04-24T08:39:31.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:31.947+0000 7fb840800700 1 -- 172.21.15.83:0/1482720420 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb83819ede0 con 0x7fb83810daa0 2024-04-24T08:39:31.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:31.948+0000 7fb840800700 1 -- 172.21.15.83:0/1482720420 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8381a2e80 con 0x7fb83810daa0 2024-04-24T08:39:31.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:31.949+0000 7fb832a00700 1 -- 172.21.15.83:0/1482720420 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fb82c029070 con 0x7fb83810daa0 2024-04-24T08:39:31.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:31.949+0000 7fb840800700 1 -- 172.21.15.83:0/1482720420 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb83810a1f0 con 0x7fb83810daa0 2024-04-24T08:39:31.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:31.949+0000 7fb832a00700 1 --2- 172.21.15.83:0/1482720420 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fb8200396b0 0x7fb82003bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:31.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:31.949+0000 7fb832a00700 1 -- 172.21.15.83:0/1482720420 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb82c01e070 con 0x7fb83810daa0 2024-04-24T08:39:31.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:31.950+0000 7fb83d400700 1 --2- 172.21.15.83:0/1482720420 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fb8200396b0 0x7fb82003bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:31.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:31.950+0000 7fb83d400700 1 --2- 172.21.15.83:0/1482720420 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fb8200396b0 0x7fb82003bb70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb828006fd0 tx=0x7fb828006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:39:31.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:31.955+0000 7fb832a00700 1 -- 172.21.15.83:0/1482720420 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb82c013070 con 0x7fb83810daa0 2024-04-24T08:39:32.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:32.186+0000 7fb840800700 1 -- 172.21.15.83:0/1482720420 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fb83819f910 con 0x7fb83810daa0 2024-04-24T08:39:32.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:32.347+0000 7fb832a00700 1 -- 172.21.15.83:0/1482720420 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7fb82c0282c0 con 0x7fb83810daa0 2024-04-24T08:39:32.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:32.351+0000 7fb840800700 1 -- 172.21.15.83:0/1482720420 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fb8200396b0 msgr2=0x7fb82003bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:32.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:32.351+0000 7fb840800700 1 --2- 172.21.15.83:0/1482720420 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fb8200396b0 0x7fb82003bb70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb828006fd0 tx=0x7fb828006e40 comp rx=0 tx=0).stop 2024-04-24T08:39:32.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:32.351+0000 7fb840800700 1 -- 172.21.15.83:0/1482720420 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb83810daa0 msgr2=0x7fb83819e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:32.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:32.351+0000 7fb840800700 1 --2- 172.21.15.83:0/1482720420 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb83810daa0 0x7fb83819e5b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb82c0056d0 tx=0x7fb82c0057b0 comp rx=0 tx=0).stop 2024-04-24T08:39:32.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:32.352+0000 7fb840800700 1 -- 172.21.15.83:0/1482720420 shutdown_connections 2024-04-24T08:39:32.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:32.352+0000 7fb840800700 1 --2- 172.21.15.83:0/1482720420 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fb8200396b0 0x7fb82003bb70 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:32.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:32.352+0000 7fb840800700 1 --2- 172.21.15.83:0/1482720420 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb83810daa0 0x7fb83819e5b0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:32.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:32.352+0000 7fb840800700 1 -- 172.21.15.83:0/1482720420 >> 172.21.15.83:0/1482720420 conn(0x7fb838078290 msgr2=0x7fb838110630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:39:32.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:32.352+0000 7fb840800700 1 -- 172.21.15.83:0/1482720420 shutdown_connections 2024-04-24T08:39:32.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:39:32.352+0000 7fb840800700 1 -- 172.21.15.83:0/1482720420 wait complete. 2024-04-24T08:39:32.354 INFO:teuthology.orchestra.run.smithi083.stderr:adjusted tunables profile to default 2024-04-24T08:39:32.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:32 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/1482720420' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-24T08:39:33.217 INFO:tasks.cephadm:Adding mon.a on smithi083 2024-04-24T08:39:33.217 INFO:tasks.cephadm:Adding mon.b on smithi138 2024-04-24T08:39:33.217 INFO:tasks.cephadm:Adding mon.c on smithi189 2024-04-24T08:39:33.217 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd3e039e-0215-11ef-bc93-c7b262605968 -- ceph orch apply mon '3;smithi083:172.21.15.83=a;smithi138:172.21.15.138=b;smithi189:172.21.15.189=c' 2024-04-24T08:39:33.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:33 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/1482720420' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-24T08:39:33.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:33 smithi083 ceph-mon[27907]: osdmap e4: 0 total, 0 up, 0 in 2024-04-24T08:39:33.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:33 smithi083 ceph-mon[27907]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:39:33.643 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-24T08:39:33.754 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-24T08:39:36.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:35 smithi083 ceph-mon[27907]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:39:36.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:35 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:39:36.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:35 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:39:36.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:35 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:39:36.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:35 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:39:36.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:35 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi189", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:39:36.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:35 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:39:36.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:35 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:39:36.513 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:36.512+0000 7fd537000700 1 -- 172.21.15.189:0/1714796419 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd530101450 msgr2=0x7fd530101870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:36.513 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:36.512+0000 7fd537000700 1 --2- 172.21.15.189:0/1714796419 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd530101450 0x7fd530101870 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd520009910 tx=0x7fd520009c20 comp rx=0 tx=0).stop 2024-04-24T08:39:36.513 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:36.512+0000 7fd537000700 1 -- 172.21.15.189:0/1714796419 shutdown_connections 2024-04-24T08:39:36.513 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:36.512+0000 7fd537000700 1 --2- 172.21.15.189:0/1714796419 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd530101450 0x7fd530101870 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:36.513 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:36.512+0000 7fd537000700 1 -- 172.21.15.189:0/1714796419 >> 172.21.15.189:0/1714796419 conn(0x7fd5300fc9f0 msgr2=0x7fd5300fee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:39:36.514 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:36.512+0000 7fd537000700 1 -- 172.21.15.189:0/1714796419 shutdown_connections 2024-04-24T08:39:36.514 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:36.513+0000 7fd537000700 1 -- 172.21.15.189:0/1714796419 wait complete. 2024-04-24T08:39:36.514 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:36.514+0000 7fd537000700 1 Processor -- start 2024-04-24T08:39:36.514 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:36.514+0000 7fd537000700 1 -- start start 2024-04-24T08:39:36.514 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:36.514+0000 7fd537000700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd530101450 0x7fd5300fe8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:36.515 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:36.514+0000 7fd537000700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd530107390 con 0x7fd530101450 2024-04-24T08:39:36.515 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:36.515+0000 7fd52fe00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd530101450 0x7fd5300fe8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:36.515 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:36.515+0000 7fd52fe00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd530101450 0x7fd5300fe8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.189:34588/0 (socket says 172.21.15.189:34588) 2024-04-24T08:39:36.515 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:36.515+0000 7fd52fe00700 1 -- 172.21.15.189:0/3982540666 learned_addr learned my addr 172.21.15.189:0/3982540666 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-04-24T08:39:36.516 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:36.515+0000 7fd52fe00700 1 -- 172.21.15.189:0/3982540666 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5300fedf0 con 0x7fd530101450 2024-04-24T08:39:36.516 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:36.516+0000 7fd52fe00700 1 --2- 172.21.15.189:0/3982540666 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd530101450 0x7fd5300fe8b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd520005950 tx=0x7fd520004fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:39:36.516 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:36.516+0000 7fd52cc00700 1 -- 172.21.15.189:0/3982540666 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd520026070 con 0x7fd530101450 2024-04-24T08:39:36.516 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:36.516+0000 7fd537000700 1 -- 172.21.15.189:0/3982540666 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5300fcfe0 con 0x7fd530101450 2024-04-24T08:39:36.517 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:36.516+0000 7fd52cc00700 1 -- 172.21.15.189:0/3982540666 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd52000bc00 con 0x7fd530101450 2024-04-24T08:39:36.517 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:36.516+0000 7fd52cc00700 1 -- 172.21.15.189:0/3982540666 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd5200039a0 con 0x7fd530101450 2024-04-24T08:39:36.517 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:36.516+0000 7fd537000700 1 -- 172.21.15.189:0/3982540666 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5300fd520 con 0x7fd530101450 2024-04-24T08:39:36.518 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:36.517+0000 7fd537000700 1 -- 172.21.15.189:0/3982540666 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd530107b10 con 0x7fd530101450 2024-04-24T08:39:36.518 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:36.518+0000 7fd52cc00700 1 -- 172.21.15.189:0/3982540666 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fd520029070 con 0x7fd530101450 2024-04-24T08:39:36.519 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:36.518+0000 7fd52cc00700 1 --2- 172.21.15.189:0/3982540666 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fd514039650 0x7fd51403bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:36.519 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:36.519+0000 7fd52cc00700 1 -- 172.21.15.189:0/3982540666 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fd52001e070 con 0x7fd530101450 2024-04-24T08:39:36.519 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:36.519+0000 7fd52f400700 1 --2- 172.21.15.189:0/3982540666 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fd514039650 0x7fd51403bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:36.520 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:36.520+0000 7fd52f400700 1 --2- 172.21.15.189:0/3982540666 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fd514039650 0x7fd51403bb10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd51c006fd0 tx=0x7fd51c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:39:36.534 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:36.534+0000 7fd52cc00700 1 -- 172.21.15.189:0/3982540666 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd520013070 con 0x7fd530101450 2024-04-24T08:39:36.772 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:36.772+0000 7fd537000700 1 -- 172.21.15.189:0/3982540666 --> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi083:172.21.15.83=a;smithi138:172.21.15.138=b;smithi189:172.21.15.189=c", "target": ["mon-mgr", ""]}) -- 0x7fd530191180 con 0x7fd514039650 2024-04-24T08:39:36.790 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:36.790+0000 7fd52cc00700 1 -- 172.21.15.189:0/3982540666 <== mgr.14152 v2:172.21.15.83:6800/2550826560 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fd530191180 con 0x7fd514039650 2024-04-24T08:39:36.790 INFO:teuthology.orchestra.run.smithi189.stdout:Scheduled mon update... 2024-04-24T08:39:36.793 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:36.792+0000 7fd537000700 1 -- 172.21.15.189:0/3982540666 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fd514039650 msgr2=0x7fd51403bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:36.793 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:36.792+0000 7fd537000700 1 --2- 172.21.15.189:0/3982540666 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fd514039650 0x7fd51403bb10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd51c006fd0 tx=0x7fd51c006e40 comp rx=0 tx=0).stop 2024-04-24T08:39:36.793 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:36.792+0000 7fd537000700 1 -- 172.21.15.189:0/3982540666 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd530101450 msgr2=0x7fd5300fe8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:36.793 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:36.792+0000 7fd537000700 1 --2- 172.21.15.189:0/3982540666 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd530101450 0x7fd5300fe8b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd520005950 tx=0x7fd520004fa0 comp rx=0 tx=0).stop 2024-04-24T08:39:36.793 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:36.792+0000 7fd537000700 1 -- 172.21.15.189:0/3982540666 shutdown_connections 2024-04-24T08:39:36.793 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:36.792+0000 7fd537000700 1 --2- 172.21.15.189:0/3982540666 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fd514039650 0x7fd51403bb10 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:36.793 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:36.792+0000 7fd537000700 1 --2- 172.21.15.189:0/3982540666 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd530101450 0x7fd5300fe8b0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:36.793 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:36.792+0000 7fd537000700 1 -- 172.21.15.189:0/3982540666 >> 172.21.15.189:0/3982540666 conn(0x7fd5300fc9f0 msgr2=0x7fd53006a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:39:36.794 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:36.792+0000 7fd537000700 1 -- 172.21.15.189:0/3982540666 shutdown_connections 2024-04-24T08:39:36.794 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:36.793+0000 7fd537000700 1 -- 172.21.15.189:0/3982540666 wait complete. 2024-04-24T08:39:37.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:36 smithi083 ceph-mon[27907]: Updating smithi189:/etc/ceph/ceph.conf 2024-04-24T08:39:37.548 DEBUG:teuthology.orchestra.run.smithi138:mon.b> sudo journalctl -f -n 0 -u ceph-bd3e039e-0215-11ef-bc93-c7b262605968@mon.b.service 2024-04-24T08:39:37.556 DEBUG:teuthology.orchestra.run.smithi189:mon.c> sudo journalctl -f -n 0 -u ceph-bd3e039e-0215-11ef-bc93-c7b262605968@mon.c.service 2024-04-24T08:39:37.560 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-24T08:39:37.561 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd3e039e-0215-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-24T08:39:38.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:37 smithi083 ceph-mon[27907]: Updating smithi189:/var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T08:39:38.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:37 smithi083 ceph-mon[27907]: Updating smithi189:/etc/ceph/ceph.client.admin.keyring 2024-04-24T08:39:38.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:37 smithi083 ceph-mon[27907]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:39:38.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:37 smithi083 ceph-mon[27907]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi083:172.21.15.83=a;smithi138:172.21.15.138=b;smithi189:172.21.15.189=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:39:38.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:37 smithi083 ceph-mon[27907]: Saving service mon spec with placement smithi083:172.21.15.83=a;smithi138:172.21.15.138=b;smithi189:172.21.15.189=c;count:3 2024-04-24T08:39:38.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:37 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:39:38.310 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T08:39:39.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:38 smithi083 ceph-mon[27907]: Updating smithi189:/var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T08:39:39.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:38 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:39:39.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:38 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:39:39.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:38 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:39:39.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:38 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T08:39:39.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:38 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:39:40.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:39 smithi083 ceph-mon[27907]: Deploying daemon mon.c on smithi189 2024-04-24T08:39:40.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:39 smithi083 ceph-mon[27907]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:39:40.393 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:40.393+0000 7f2ee3e00700 1 -- 172.21.15.189:0/4225465072 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ee4107370 msgr2=0x7f2ee4107750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:40.394 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:40.393+0000 7f2ee3e00700 1 --2- 172.21.15.189:0/4225465072 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ee4107370 0x7f2ee4107750 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f2ed400b0a0 tx=0x7f2ed400b3b0 comp rx=0 tx=0).stop 2024-04-24T08:39:40.394 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:40.393+0000 7f2ee3e00700 1 -- 172.21.15.189:0/4225465072 shutdown_connections 2024-04-24T08:39:40.394 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:40.393+0000 7f2ee3e00700 1 --2- 172.21.15.189:0/4225465072 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ee4107370 0x7f2ee4107750 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:40.394 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:40.393+0000 7f2ee3e00700 1 -- 172.21.15.189:0/4225465072 >> 172.21.15.189:0/4225465072 conn(0x7f2ee40fc4b0 msgr2=0x7f2ee40fe8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:39:40.394 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:40.393+0000 7f2ee3e00700 1 -- 172.21.15.189:0/4225465072 shutdown_connections 2024-04-24T08:39:40.394 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:40.393+0000 7f2ee3e00700 1 -- 172.21.15.189:0/4225465072 wait complete. 2024-04-24T08:39:40.395 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:40.394+0000 7f2ee3e00700 1 Processor -- start 2024-04-24T08:39:40.395 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:40.394+0000 7f2ee3e00700 1 -- start start 2024-04-24T08:39:40.395 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:40.395+0000 7f2ee3e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ee4107370 0x7f2ee4197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:40.395 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:40.395+0000 7f2ee3e00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee410c980 con 0x7f2ee4107370 2024-04-24T08:39:40.396 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:40.395+0000 7f2ee2a00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ee4107370 0x7f2ee4197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:40.396 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:40.395+0000 7f2ee2a00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ee4107370 0x7f2ee4197e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.189:34602/0 (socket says 172.21.15.189:34602) 2024-04-24T08:39:40.396 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:40.395+0000 7f2ee2a00700 1 -- 172.21.15.189:0/182869854 learned_addr learned my addr 172.21.15.189:0/182869854 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-04-24T08:39:40.396 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:40.396+0000 7f2ee2a00700 1 -- 172.21.15.189:0/182869854 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ee41983b0 con 0x7f2ee4107370 2024-04-24T08:39:40.397 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:40.396+0000 7f2ee2a00700 1 --2- 172.21.15.189:0/182869854 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ee4107370 0x7f2ee4197e70 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2ed40062a0 tx=0x7f2ed4003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:39:40.397 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:40.397+0000 7f2ee8a00700 1 -- 172.21.15.189:0/182869854 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2ed4017040 con 0x7f2ee4107370 2024-04-24T08:39:40.397 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:40.397+0000 7f2ee8a00700 1 -- 172.21.15.189:0/182869854 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2ed4022070 con 0x7f2ee4107370 2024-04-24T08:39:40.397 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:40.397+0000 7f2ee3e00700 1 -- 172.21.15.189:0/182869854 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ee4198640 con 0x7f2ee4107370 2024-04-24T08:39:40.398 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:40.397+0000 7f2ee8a00700 1 -- 172.21.15.189:0/182869854 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2ed401d5e0 con 0x7f2ee4107370 2024-04-24T08:39:40.398 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:40.397+0000 7f2ee3e00700 1 -- 172.21.15.189:0/182869854 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ee4198aa0 con 0x7f2ee4107370 2024-04-24T08:39:40.399 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:40.399+0000 7f2ee3e00700 1 -- 172.21.15.189:0/182869854 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ec4005330 con 0x7f2ee4107370 2024-04-24T08:39:40.399 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:40.399+0000 7f2ee8a00700 1 -- 172.21.15.189:0/182869854 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f2ed401d890 con 0x7f2ee4107370 2024-04-24T08:39:40.400 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:40.400+0000 7f2ee8a00700 1 --2- 172.21.15.189:0/182869854 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f2ec80396b0 0x7f2ec803bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:40.400 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:40.400+0000 7f2ee8a00700 1 -- 172.21.15.189:0/182869854 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f2ed400f050 con 0x7f2ee4107370 2024-04-24T08:39:40.400 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:40.400+0000 7f2ee2000700 1 --2- 172.21.15.189:0/182869854 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f2ec80396b0 0x7f2ec803bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:40.401 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:40.401+0000 7f2ee2000700 1 --2- 172.21.15.189:0/182869854 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f2ec80396b0 0x7f2ec803bb70 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2ed0006fd0 tx=0x7f2ed0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:39:40.416 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:40.416+0000 7f2ee8a00700 1 -- 172.21.15.189:0/182869854 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2ed40096a0 con 0x7f2ee4107370 2024-04-24T08:39:40.730 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:40.730+0000 7f2ee3e00700 1 -- 172.21.15.189:0/182869854 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f2ec4005620 con 0x7f2ee4107370 2024-04-24T08:39:40.732 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:40.732+0000 7f2ee8a00700 1 -- 172.21.15.189:0/182869854 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7f2ed4022220 con 0x7f2ee4107370 2024-04-24T08:39:40.733 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-04-24T08:39:40.733 INFO:teuthology.orchestra.run.smithi189.stdout:{"epoch":1,"fsid":"bd3e039e-0215-11ef-bc93-c7b262605968","modified":"2024-04-24T08:37:28.106852Z","created":"2024-04-24T08:37:28.106852Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:3300","nonce":0},{"type":"v1","addr":"172.21.15.83:6789","nonce":0}]},"addr":"172.21.15.83:6789/0","public_addr":"172.21.15.83:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-24T08:39:40.736 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:40.735+0000 7f2ee3e00700 1 -- 172.21.15.189:0/182869854 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f2ec80396b0 msgr2=0x7f2ec803bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:40.736 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:40.735+0000 7f2ee3e00700 1 --2- 172.21.15.189:0/182869854 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f2ec80396b0 0x7f2ec803bb70 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2ed0006fd0 tx=0x7f2ed0006e40 comp rx=0 tx=0).stop 2024-04-24T08:39:40.736 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:40.735+0000 7f2ee3e00700 1 -- 172.21.15.189:0/182869854 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ee4107370 msgr2=0x7f2ee4197e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:40.736 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:40.735+0000 7f2ee3e00700 1 --2- 172.21.15.189:0/182869854 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ee4107370 0x7f2ee4197e70 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2ed40062a0 tx=0x7f2ed4003ce0 comp rx=0 tx=0).stop 2024-04-24T08:39:40.736 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:40.735+0000 7f2ee3e00700 1 -- 172.21.15.189:0/182869854 shutdown_connections 2024-04-24T08:39:40.736 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:40.735+0000 7f2ee3e00700 1 --2- 172.21.15.189:0/182869854 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f2ec80396b0 0x7f2ec803bb70 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:40.736 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:40.735+0000 7f2ee3e00700 1 --2- 172.21.15.189:0/182869854 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ee4107370 0x7f2ee4197e70 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:40.736 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:40.735+0000 7f2ee3e00700 1 -- 172.21.15.189:0/182869854 >> 172.21.15.189:0/182869854 conn(0x7f2ee40fc4b0 msgr2=0x7f2ee4109f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:39:40.736 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:40.735+0000 7f2ee3e00700 1 -- 172.21.15.189:0/182869854 shutdown_connections 2024-04-24T08:39:40.736 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:40.735+0000 7f2ee3e00700 1 -- 172.21.15.189:0/182869854 wait complete. 2024-04-24T08:39:40.737 INFO:teuthology.orchestra.run.smithi189.stderr:dumped monmap epoch 1 2024-04-24T08:39:41.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:40 smithi083 ceph-mon[27907]: from='client.? 172.21.15.189:0/182869854' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T08:39:42.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:41 smithi083 ceph-mon[27907]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:39:42.966 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-24T08:39:42.966 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd3e039e-0215-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-24T08:39:43.589 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T08:39:44.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:43 smithi083 ceph-mon[27907]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:39:45.181 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 podman[31750]: 2024-04-24 08:39:44.979026938 +0000 UTC m=+0.035142399 image pull b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644 2024-04-24T08:39:45.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 podman[31750]: 2024-04-24 08:39:45.554982709 +0000 UTC m=+0.611098214 container init 26c5d9f35126a527a600e689ef7420bdf2016000a6fe6bd7cc23b3fef2da291a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mon-c, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T08:39:45.709 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 podman[31750]: 2024-04-24 08:39:45.566113987 +0000 UTC m=+0.622229488 container start 26c5d9f35126a527a600e689ef7420bdf2016000a6fe6bd7cc23b3fef2da291a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mon-c, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2) 2024-04-24T08:39:45.709 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: set uid:gid to 167:167 (ceph:ceph) 2024-04-24T08:39:45.709 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev), process ceph-mon, pid 2 2024-04-24T08:39:45.709 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: pidfile_write: ignore empty --pid-file 2024-04-24T08:39:45.709 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: load: jerasure load: lrc 2024-04-24T08:39:45.709 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: RocksDB version: 7.9.2 2024-04-24T08:39:45.709 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Git sha 0 2024-04-24T08:39:45.709 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Compile date 2024-04-23 21:06:33 2024-04-24T08:39:45.709 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: DB SUMMARY 2024-04-24T08:39:45.709 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: DB Session ID: TP3R0A0MU4B3OL33NKRZ 2024-04-24T08:39:45.709 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: CURRENT file: CURRENT 2024-04-24T08:39:45.710 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: IDENTITY file: IDENTITY 2024-04-24T08:39:45.710 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-04-24T08:39:45.710 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-04-24T08:39:45.710 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-04-24T08:39:45.710 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.error_if_exists: 0 2024-04-24T08:39:45.710 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.create_if_missing: 0 2024-04-24T08:39:45.710 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.paranoid_checks: 1 2024-04-24T08:39:45.710 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-24T08:39:45.710 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-24T08:39:45.710 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-24T08:39:45.710 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.env: 0x56524b934cc0 2024-04-24T08:39:45.710 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.fs: PosixFileSystem 2024-04-24T08:39:45.710 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.info_log: 0x5652848b7c60 2024-04-24T08:39:45.710 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.max_file_opening_threads: 16 2024-04-24T08:39:45.711 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.statistics: (nil) 2024-04-24T08:39:45.712 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.use_fsync: 0 2024-04-24T08:39:45.712 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.max_log_file_size: 0 2024-04-24T08:39:45.712 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-24T08:39:45.712 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-24T08:39:45.712 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.keep_log_file_num: 1000 2024-04-24T08:39:45.712 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.recycle_log_file_num: 0 2024-04-24T08:39:45.712 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.allow_fallocate: 1 2024-04-24T08:39:45.712 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.allow_mmap_reads: 0 2024-04-24T08:39:45.712 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.allow_mmap_writes: 0 2024-04-24T08:39:45.712 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.use_direct_reads: 0 2024-04-24T08:39:45.712 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-24T08:39:45.712 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.create_missing_column_families: 0 2024-04-24T08:39:45.713 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.db_log_dir: 2024-04-24T08:39:45.713 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.wal_dir: 2024-04-24T08:39:45.713 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-24T08:39:45.713 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-24T08:39:45.713 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-24T08:39:45.713 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-24T08:39:45.713 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-24T08:39:45.713 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-24T08:39:45.713 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.advise_random_on_open: 1 2024-04-24T08:39:45.713 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.db_write_buffer_size: 0 2024-04-24T08:39:45.713 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.write_buffer_manager: 0x5652848861e0 2024-04-24T08:39:45.713 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-24T08:39:45.714 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-24T08:39:45.714 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-24T08:39:45.714 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.rate_limiter: (nil) 2024-04-24T08:39:45.714 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-24T08:39:45.714 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.wal_recovery_mode: 2 2024-04-24T08:39:45.714 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.enable_thread_tracking: 0 2024-04-24T08:39:45.714 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.enable_pipelined_write: 0 2024-04-24T08:39:45.714 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.unordered_write: 0 2024-04-24T08:39:45.714 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-24T08:39:45.714 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-24T08:39:45.714 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-24T08:39:45.714 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-24T08:39:45.714 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.row_cache: None 2024-04-24T08:39:45.714 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.wal_filter: None 2024-04-24T08:39:45.715 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-24T08:39:45.715 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.allow_ingest_behind: 0 2024-04-24T08:39:45.715 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.two_write_queues: 0 2024-04-24T08:39:45.715 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.manual_wal_flush: 0 2024-04-24T08:39:45.715 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.wal_compression: 0 2024-04-24T08:39:45.715 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.atomic_flush: 0 2024-04-24T08:39:45.715 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-24T08:39:45.715 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-24T08:39:45.715 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-24T08:39:45.715 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.log_readahead_size: 0 2024-04-24T08:39:45.716 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-24T08:39:45.716 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.best_efforts_recovery: 0 2024-04-24T08:39:45.716 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-24T08:39:45.716 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-24T08:39:45.716 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.allow_data_in_errors: 0 2024-04-24T08:39:45.716 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.db_host_id: __hostname__ 2024-04-24T08:39:45.716 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-24T08:39:45.716 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.max_background_jobs: 2 2024-04-24T08:39:45.716 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.max_background_compactions: -1 2024-04-24T08:39:45.716 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.max_subcompactions: 1 2024-04-24T08:39:45.716 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-24T08:39:45.716 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-24T08:39:45.716 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-24T08:39:45.717 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.max_total_wal_size: 0 2024-04-24T08:39:45.717 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-24T08:39:45.717 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-24T08:39:45.717 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-24T08:39:45.717 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-24T08:39:45.717 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.max_open_files: -1 2024-04-24T08:39:45.717 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.bytes_per_sync: 0 2024-04-24T08:39:45.717 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-24T08:39:45.717 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-24T08:39:45.717 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.compaction_readahead_size: 0 2024-04-24T08:39:45.717 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.max_background_flushes: -1 2024-04-24T08:39:45.717 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Compression algorithms supported: 2024-04-24T08:39:45.717 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-24T08:39:45.717 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: kZSTD supported: 0 2024-04-24T08:39:45.718 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: kXpressCompression supported: 0 2024-04-24T08:39:45.718 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: kLZ4HCCompression supported: 1 2024-04-24T08:39:45.718 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: kZlibCompression supported: 1 2024-04-24T08:39:45.718 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: kSnappyCompression supported: 1 2024-04-24T08:39:45.718 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: kLZ4Compression supported: 1 2024-04-24T08:39:45.718 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: kBZip2Compression supported: 0 2024-04-24T08:39:45.718 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-24T08:39:45.718 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-24T08:39:45.718 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-04-24T08:39:45.719 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-24T08:39:45.719 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-24T08:39:45.719 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.merge_operator: 2024-04-24T08:39:45.719 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.compaction_filter: None 2024-04-24T08:39:45.719 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.compaction_filter_factory: None 2024-04-24T08:39:45.719 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.sst_partitioner_factory: None 2024-04-24T08:39:45.719 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-24T08:39:45.719 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-24T08:39:45.719 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5652848b7e20) 2024-04-24T08:39:45.719 INFO:journalctl@ceph.mon.c.smithi189.stdout: cache_index_and_filter_blocks: 1 2024-04-24T08:39:45.719 INFO:journalctl@ceph.mon.c.smithi189.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-24T08:39:45.720 INFO:journalctl@ceph.mon.c.smithi189.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-24T08:39:45.720 INFO:journalctl@ceph.mon.c.smithi189.stdout: pin_top_level_index_and_filter: 1 2024-04-24T08:39:45.720 INFO:journalctl@ceph.mon.c.smithi189.stdout: index_type: 0 2024-04-24T08:39:45.720 INFO:journalctl@ceph.mon.c.smithi189.stdout: data_block_index_type: 0 2024-04-24T08:39:45.720 INFO:journalctl@ceph.mon.c.smithi189.stdout: index_shortening: 1 2024-04-24T08:39:45.720 INFO:journalctl@ceph.mon.c.smithi189.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-24T08:39:45.720 INFO:journalctl@ceph.mon.c.smithi189.stdout: checksum: 4 2024-04-24T08:39:45.720 INFO:journalctl@ceph.mon.c.smithi189.stdout: no_block_cache: 0 2024-04-24T08:39:45.720 INFO:journalctl@ceph.mon.c.smithi189.stdout: block_cache: 0x5652848b9090 2024-04-24T08:39:45.720 INFO:journalctl@ceph.mon.c.smithi189.stdout: block_cache_name: BinnedLRUCache 2024-04-24T08:39:45.720 INFO:journalctl@ceph.mon.c.smithi189.stdout: block_cache_options: 2024-04-24T08:39:45.721 INFO:journalctl@ceph.mon.c.smithi189.stdout: capacity : 536870912 2024-04-24T08:39:45.721 INFO:journalctl@ceph.mon.c.smithi189.stdout: num_shard_bits : 4 2024-04-24T08:39:45.721 INFO:journalctl@ceph.mon.c.smithi189.stdout: strict_capacity_limit : 0 2024-04-24T08:39:45.721 INFO:journalctl@ceph.mon.c.smithi189.stdout: high_pri_pool_ratio: 0.000 2024-04-24T08:39:45.721 INFO:journalctl@ceph.mon.c.smithi189.stdout: block_cache_compressed: (nil) 2024-04-24T08:39:45.721 INFO:journalctl@ceph.mon.c.smithi189.stdout: persistent_cache: (nil) 2024-04-24T08:39:45.721 INFO:journalctl@ceph.mon.c.smithi189.stdout: block_size: 4096 2024-04-24T08:39:45.721 INFO:journalctl@ceph.mon.c.smithi189.stdout: block_size_deviation: 10 2024-04-24T08:39:45.721 INFO:journalctl@ceph.mon.c.smithi189.stdout: block_restart_interval: 16 2024-04-24T08:39:45.721 INFO:journalctl@ceph.mon.c.smithi189.stdout: index_block_restart_interval: 1 2024-04-24T08:39:45.721 INFO:journalctl@ceph.mon.c.smithi189.stdout: metadata_block_size: 4096 2024-04-24T08:39:45.722 INFO:journalctl@ceph.mon.c.smithi189.stdout: partition_filters: 0 2024-04-24T08:39:45.722 INFO:journalctl@ceph.mon.c.smithi189.stdout: use_delta_encoding: 1 2024-04-24T08:39:45.722 INFO:journalctl@ceph.mon.c.smithi189.stdout: filter_policy: bloomfilter 2024-04-24T08:39:45.722 INFO:journalctl@ceph.mon.c.smithi189.stdout: whole_key_filtering: 1 2024-04-24T08:39:45.722 INFO:journalctl@ceph.mon.c.smithi189.stdout: verify_compression: 0 2024-04-24T08:39:45.722 INFO:journalctl@ceph.mon.c.smithi189.stdout: read_amp_bytes_per_bit: 0 2024-04-24T08:39:45.722 INFO:journalctl@ceph.mon.c.smithi189.stdout: format_version: 5 2024-04-24T08:39:45.722 INFO:journalctl@ceph.mon.c.smithi189.stdout: enable_index_compression: 1 2024-04-24T08:39:45.722 INFO:journalctl@ceph.mon.c.smithi189.stdout: block_align: 0 2024-04-24T08:39:45.722 INFO:journalctl@ceph.mon.c.smithi189.stdout: max_auto_readahead_size: 262144 2024-04-24T08:39:45.722 INFO:journalctl@ceph.mon.c.smithi189.stdout: prepopulate_block_cache: 0 2024-04-24T08:39:45.723 INFO:journalctl@ceph.mon.c.smithi189.stdout: initial_auto_readahead_size: 8192 2024-04-24T08:39:45.725 INFO:journalctl@ceph.mon.c.smithi189.stdout: num_file_reads_for_auto_readahead: 2 2024-04-24T08:39:45.725 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.write_buffer_size: 33554432 2024-04-24T08:39:45.725 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.max_write_buffer_number: 2 2024-04-24T08:39:45.725 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.compression: NoCompression 2024-04-24T08:39:45.725 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.bottommost_compression: Disabled 2024-04-24T08:39:45.725 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.prefix_extractor: nullptr 2024-04-24T08:39:45.726 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-24T08:39:45.726 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.num_levels: 7 2024-04-24T08:39:45.726 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-24T08:39:45.726 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-24T08:39:45.726 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-24T08:39:45.726 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-24T08:39:45.726 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-24T08:39:45.726 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-24T08:39:45.726 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-24T08:39:45.726 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-24T08:39:45.726 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-24T08:39:45.726 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-24T08:39:45.727 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-24T08:39:45.727 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-24T08:39:45.727 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-24T08:39:45.727 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.compression_opts.level: 32767 2024-04-24T08:39:45.727 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.compression_opts.strategy: 0 2024-04-24T08:39:45.727 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-24T08:39:45.727 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-24T08:39:45.727 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-24T08:39:45.727 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-24T08:39:45.727 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.compression_opts.enabled: false 2024-04-24T08:39:45.727 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-24T08:39:45.727 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-24T08:39:45.727 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-24T08:39:45.727 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-24T08:39:45.728 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.target_file_size_base: 67108864 2024-04-24T08:39:45.728 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-24T08:39:45.728 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-24T08:39:45.728 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-24T08:39:45.728 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-24T08:39:45.728 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-24T08:39:45.728 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-24T08:39:45.728 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-24T08:39:45.728 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-24T08:39:45.728 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-24T08:39:45.728 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-24T08:39:45.728 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-24T08:39:45.728 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-24T08:39:45.728 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-24T08:39:45.728 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-24T08:39:45.729 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.arena_block_size: 1048576 2024-04-24T08:39:45.729 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-24T08:39:45.729 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-24T08:39:45.729 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.disable_auto_compactions: 0 2024-04-24T08:39:45.729 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-24T08:39:45.729 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-24T08:39:45.729 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-24T08:39:45.729 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-24T08:39:45.729 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-24T08:39:45.729 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-24T08:39:45.729 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-24T08:39:45.729 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-24T08:39:45.729 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-24T08:39:45.730 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-24T08:39:45.730 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-24T08:39:45.730 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.inplace_update_support: 0 2024-04-24T08:39:45.730 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-24T08:39:45.730 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-24T08:39:45.730 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-24T08:39:45.730 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-24T08:39:45.730 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.bloom_locality: 0 2024-04-24T08:39:45.730 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.max_successive_merges: 0 2024-04-24T08:39:45.730 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-24T08:39:45.730 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.paranoid_file_checks: 0 2024-04-24T08:39:45.730 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.force_consistency_checks: 1 2024-04-24T08:39:45.730 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.report_bg_io_stats: 0 2024-04-24T08:39:45.730 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.ttl: 2592000 2024-04-24T08:39:45.731 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-24T08:39:45.731 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-24T08:39:45.731 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-24T08:39:45.731 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.enable_blob_files: false 2024-04-24T08:39:45.731 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.min_blob_size: 0 2024-04-24T08:39:45.731 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.blob_file_size: 268435456 2024-04-24T08:39:45.731 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-24T08:39:45.731 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-24T08:39:45.731 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-24T08:39:45.731 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-24T08:39:45.731 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-24T08:39:45.731 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.blob_file_starting_level: 0 2024-04-24T08:39:45.731 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-24T08:39:45.731 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-24T08:39:45.732 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-04-24T08:39:45.732 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 1e5b09cd-2581-4705-a7d0-52af0015d5b9 2024-04-24T08:39:45.732 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713947985624282, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-04-24T08:39:45.732 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-04-24T08:39:45.732 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713947985625350, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1713947985, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "1e5b09cd-2581-4705-a7d0-52af0015d5b9", "db_session_id": "TP3R0A0MU4B3OL33NKRZ", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-04-24T08:39:45.732 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713947985625626, "job": 1, "event": "recovery_finished"} 2024-04-24T08:39:45.732 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-04-24T08:39:45.732 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-24T08:39:45.732 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5652849a6000 2024-04-24T08:39:45.732 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: DB pointer 0x565284990000 2024-04-24T08:39:45.732 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-24T08:39:45.732 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-24T08:39:45.732 INFO:journalctl@ceph.mon.c.smithi189.stdout: ** DB Stats ** 2024-04-24T08:39:45.732 INFO:journalctl@ceph.mon.c.smithi189.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-24T08:39:45.733 INFO:journalctl@ceph.mon.c.smithi189.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-24T08:39:45.733 INFO:journalctl@ceph.mon.c.smithi189.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-24T08:39:45.733 INFO:journalctl@ceph.mon.c.smithi189.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-24T08:39:45.733 INFO:journalctl@ceph.mon.c.smithi189.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-24T08:39:45.733 INFO:journalctl@ceph.mon.c.smithi189.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-24T08:39:45.733 INFO:journalctl@ceph.mon.c.smithi189.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-24T08:39:45.733 INFO:journalctl@ceph.mon.c.smithi189.stdout: 2024-04-24T08:39:45.733 INFO:journalctl@ceph.mon.c.smithi189.stdout: ** Compaction Stats [default] ** 2024-04-24T08:39:45.733 INFO:journalctl@ceph.mon.c.smithi189.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-24T08:39:45.733 INFO:journalctl@ceph.mon.c.smithi189.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-24T08:39:45.733 INFO:journalctl@ceph.mon.c.smithi189.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-24T08:39:45.733 INFO:journalctl@ceph.mon.c.smithi189.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-24T08:39:45.733 INFO:journalctl@ceph.mon.c.smithi189.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-24T08:39:45.733 INFO:journalctl@ceph.mon.c.smithi189.stdout: 2024-04-24T08:39:45.733 INFO:journalctl@ceph.mon.c.smithi189.stdout: ** Compaction Stats [default] ** 2024-04-24T08:39:45.734 INFO:journalctl@ceph.mon.c.smithi189.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-24T08:39:45.734 INFO:journalctl@ceph.mon.c.smithi189.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-24T08:39:45.734 INFO:journalctl@ceph.mon.c.smithi189.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-24T08:39:45.734 INFO:journalctl@ceph.mon.c.smithi189.stdout: 2024-04-24T08:39:45.734 INFO:journalctl@ceph.mon.c.smithi189.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-24T08:39:45.734 INFO:journalctl@ceph.mon.c.smithi189.stdout: 2024-04-24T08:39:45.734 INFO:journalctl@ceph.mon.c.smithi189.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-24T08:39:45.734 INFO:journalctl@ceph.mon.c.smithi189.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-24T08:39:45.734 INFO:journalctl@ceph.mon.c.smithi189.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-24T08:39:45.734 INFO:journalctl@ceph.mon.c.smithi189.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-24T08:39:45.734 INFO:journalctl@ceph.mon.c.smithi189.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-24T08:39:45.734 INFO:journalctl@ceph.mon.c.smithi189.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-24T08:39:45.734 INFO:journalctl@ceph.mon.c.smithi189.stdout: Cumulative compaction: 0.00 GB write, 0.21 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-24T08:39:45.734 INFO:journalctl@ceph.mon.c.smithi189.stdout: Interval compaction: 0.00 GB write, 0.21 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-24T08:39:45.735 INFO:journalctl@ceph.mon.c.smithi189.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-24T08:39:45.735 INFO:journalctl@ceph.mon.c.smithi189.stdout: Block cache BinnedLRUCache@0x5652848b9090#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-04-24T08:39:45.735 INFO:journalctl@ceph.mon.c.smithi189.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-04-24T08:39:45.735 INFO:journalctl@ceph.mon.c.smithi189.stdout: 2024-04-24T08:39:45.735 INFO:journalctl@ceph.mon.c.smithi189.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-24T08:39:45.735 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-04-24T08:39:45.735 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: using public_addr v2:172.21.15.189:0/0 -> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] 2024-04-24T08:39:45.735 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: starting mon.c rank -1 at public addrs [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] at bind addrs [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid bd3e039e-0215-11ef-bc93-c7b262605968 2024-04-24T08:39:45.735 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: mon.c@-1(???) e0 preinit fsid bd3e039e-0215-11ef-bc93-c7b262605968 2024-04-24T08:39:45.735 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 bash[31750]: 26c5d9f35126a527a600e689ef7420bdf2016000a6fe6bd7cc23b3fef2da291a 2024-04-24T08:39:45.735 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 systemd[1]: Started Ceph mon.c for bd3e039e-0215-11ef-bc93-c7b262605968. 2024-04-24T08:39:45.735 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: mon.c@-1(synchronizing).mds e1 new map 2024-04-24T08:39:45.735 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: mon.c@-1(synchronizing).mds e1 print_map 2024-04-24T08:39:45.735 INFO:journalctl@ceph.mon.c.smithi189.stdout: e1 2024-04-24T08:39:45.736 INFO:journalctl@ceph.mon.c.smithi189.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-24T08:39:45.736 INFO:journalctl@ceph.mon.c.smithi189.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-24T08:39:45.736 INFO:journalctl@ceph.mon.c.smithi189.stdout: legacy client fscid: -1 2024-04-24T08:39:45.736 INFO:journalctl@ceph.mon.c.smithi189.stdout: 2024-04-24T08:39:45.736 INFO:journalctl@ceph.mon.c.smithi189.stdout: No filesystems configured 2024-04-24T08:39:45.736 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-04-24T08:39:45.736 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-04-24T08:39:45.736 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-04-24T08:39:45.736 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-04-24T08:39:45.736 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-04-24T08:39:45.736 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-04-24T08:39:45.736 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-24T08:39:45.736 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-24T08:39:45.736 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-24T08:39:45.736 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-24T08:39:45.737 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: Updating smithi083:/var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T08:39:45.737 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:39:45.737 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:39:45.737 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:39:45.737 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:39:45.737 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi138", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:39:45.737 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: Deploying cephadm binary to smithi138 2024-04-24T08:39:45.737 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:39:45.737 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:39:45.737 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:39:45.737 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:39:45.737 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:39:45.737 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: Added host smithi138 2024-04-24T08:39:45.737 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:39:45.738 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:39:45.738 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:39:45.738 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:39:45.738 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:39:45.738 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T08:39:45.738 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:39:45.738 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:39:45.738 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi189", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:39:45.738 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: Deploying cephadm binary to smithi189 2024-04-24T08:39:45.738 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:39:45.738 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:39:45.738 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:39:45.738 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:39:45.738 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi138", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:39:45.739 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:39:45.739 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:39:45.739 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: Updating smithi138:/etc/ceph/ceph.conf 2024-04-24T08:39:45.739 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:39:45.739 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:39:45.739 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: Updating smithi138:/var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T08:39:45.739 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: Updating smithi138:/etc/ceph/ceph.client.admin.keyring 2024-04-24T08:39:45.739 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:39:45.739 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:39:45.739 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: Added host smithi189 2024-04-24T08:39:45.739 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: Updating smithi138:/var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T08:39:45.739 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:39:45.739 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:39:45.739 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:39:45.739 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:39:45.740 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:39:45.740 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:39:45.740 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:39:45.740 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T08:39:45.740 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:39:45.740 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:39:45.740 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:39:45.740 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/1482720420' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-24T08:39:45.740 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/1482720420' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-24T08:39:45.740 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: osdmap e4: 0 total, 0 up, 0 in 2024-04-24T08:39:45.740 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:39:45.740 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:39:45.740 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:39:45.740 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:39:45.740 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:39:45.741 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:39:45.741 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi189", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:39:45.741 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:39:45.741 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:39:45.741 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: Updating smithi189:/etc/ceph/ceph.conf 2024-04-24T08:39:45.741 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: Updating smithi189:/var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T08:39:45.741 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: Updating smithi189:/etc/ceph/ceph.client.admin.keyring 2024-04-24T08:39:45.741 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:39:45.741 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi083:172.21.15.83=a;smithi138:172.21.15.138=b;smithi189:172.21.15.189=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:39:45.741 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: Saving service mon spec with placement smithi083:172.21.15.83=a;smithi138:172.21.15.138=b;smithi189:172.21.15.189=c;count:3 2024-04-24T08:39:45.741 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:39:45.741 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: Updating smithi189:/var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T08:39:45.741 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:39:45.741 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:39:45.741 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:39:45.742 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T08:39:45.742 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:39:45.742 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: Deploying daemon mon.c on smithi189 2024-04-24T08:39:45.742 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:39:45.742 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: from='client.? 172.21.15.189:0/182869854' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T08:39:45.742 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:39:45.742 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:39:45.742 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:45 smithi189 ceph-mon[31791]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-04-24T08:39:51.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:50 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T08:39:51.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:50 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T08:39:51.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:50 smithi083 ceph-mon[27907]: mon.a calling monitor election 2024-04-24T08:39:51.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:50 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T08:39:51.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:50 smithi083 ceph-mon[27907]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:39:51.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:50 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T08:39:51.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:50 smithi083 ceph-mon[27907]: mon.c calling monitor election 2024-04-24T08:39:51.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:50 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T08:39:51.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:50 smithi083 ceph-mon[27907]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:39:51.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:50 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T08:39:51.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:50 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T08:39:51.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:50 smithi083 ceph-mon[27907]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-24T08:39:51.056 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:50 smithi083 ceph-mon[27907]: monmap epoch 2 2024-04-24T08:39:51.056 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:50 smithi083 ceph-mon[27907]: fsid bd3e039e-0215-11ef-bc93-c7b262605968 2024-04-24T08:39:51.056 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:50 smithi083 ceph-mon[27907]: last_changed 2024-04-24T08:39:45.678435+0000 2024-04-24T08:39:51.056 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:50 smithi083 ceph-mon[27907]: created 2024-04-24T08:37:28.106852+0000 2024-04-24T08:39:51.056 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:50 smithi083 ceph-mon[27907]: min_mon_release 19 (squid) 2024-04-24T08:39:51.056 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:50 smithi083 ceph-mon[27907]: election_strategy: 1 2024-04-24T08:39:51.056 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:50 smithi083 ceph-mon[27907]: 0: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.a 2024-04-24T08:39:51.056 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:50 smithi083 ceph-mon[27907]: 1: [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] mon.c 2024-04-24T08:39:51.056 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:50 smithi083 ceph-mon[27907]: fsmap 2024-04-24T08:39:51.056 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:50 smithi083 ceph-mon[27907]: osdmap e4: 0 total, 0 up, 0 in 2024-04-24T08:39:51.056 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:50 smithi083 ceph-mon[27907]: mgrmap e13: x(active, since 70s) 2024-04-24T08:39:51.056 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:50 smithi083 ceph-mon[27907]: overall HEALTH_OK 2024-04-24T08:39:51.056 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:50 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:39:51.057 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:50 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:39:51.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:50 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T08:39:51.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:50 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T08:39:51.209 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:50 smithi189 ceph-mon[31791]: mon.a calling monitor election 2024-04-24T08:39:51.209 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:50 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T08:39:51.209 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:50 smithi189 ceph-mon[31791]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:39:51.209 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:50 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T08:39:51.209 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:50 smithi189 ceph-mon[31791]: mon.c calling monitor election 2024-04-24T08:39:51.209 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:50 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T08:39:51.209 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:50 smithi189 ceph-mon[31791]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:39:51.209 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:50 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T08:39:51.209 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:50 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T08:39:51.209 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:50 smithi189 ceph-mon[31791]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-24T08:39:51.209 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:50 smithi189 ceph-mon[31791]: monmap epoch 2 2024-04-24T08:39:51.209 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:50 smithi189 ceph-mon[31791]: fsid bd3e039e-0215-11ef-bc93-c7b262605968 2024-04-24T08:39:51.209 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:50 smithi189 ceph-mon[31791]: last_changed 2024-04-24T08:39:45.678435+0000 2024-04-24T08:39:51.210 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:50 smithi189 ceph-mon[31791]: created 2024-04-24T08:37:28.106852+0000 2024-04-24T08:39:51.210 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:50 smithi189 ceph-mon[31791]: min_mon_release 19 (squid) 2024-04-24T08:39:51.210 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:50 smithi189 ceph-mon[31791]: election_strategy: 1 2024-04-24T08:39:51.210 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:50 smithi189 ceph-mon[31791]: 0: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.a 2024-04-24T08:39:51.210 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:50 smithi189 ceph-mon[31791]: 1: [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] mon.c 2024-04-24T08:39:51.210 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:50 smithi189 ceph-mon[31791]: fsmap 2024-04-24T08:39:51.210 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:50 smithi189 ceph-mon[31791]: osdmap e4: 0 total, 0 up, 0 in 2024-04-24T08:39:51.210 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:50 smithi189 ceph-mon[31791]: mgrmap e13: x(active, since 70s) 2024-04-24T08:39:51.210 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:50 smithi189 ceph-mon[31791]: overall HEALTH_OK 2024-04-24T08:39:51.210 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:50 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:39:51.210 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:50 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:39:51.980 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:51.978+0000 7efed7800700 1 -- 172.21.15.189:0/826953574 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efeb4005cc0 msgr2=0x7efeb4008150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:51.981 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:51.978+0000 7efed7800700 1 --2- 172.21.15.189:0/826953574 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efeb4005cc0 0x7efeb4008150 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7efed0076320 tx=0x7efec000ff20 comp rx=0 tx=0).stop 2024-04-24T08:39:51.981 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:51.979+0000 7efed7800700 1 -- 172.21.15.189:0/826953574 shutdown_connections 2024-04-24T08:39:51.981 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:51.979+0000 7efed7800700 1 --2- 172.21.15.189:0/826953574 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efeb4005cc0 0x7efeb4008150 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:51.981 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:51.979+0000 7efed7800700 1 -- 172.21.15.189:0/826953574 >> 172.21.15.189:0/826953574 conn(0x7efed00fe0c0 msgr2=0x7efed01004e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:39:51.981 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:51.979+0000 7efed7800700 1 -- 172.21.15.189:0/826953574 shutdown_connections 2024-04-24T08:39:51.981 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:51.979+0000 7efed7800700 1 -- 172.21.15.189:0/826953574 wait complete. 2024-04-24T08:39:51.982 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:51.982+0000 7efed7800700 1 Processor -- start 2024-04-24T08:39:51.982 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:51.982+0000 7efed7800700 1 -- start start 2024-04-24T08:39:51.983 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:51.983+0000 7efed7800700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efeb4005cc0 0x7efed019e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:51.983 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:51.983+0000 7efed7800700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efed019ec30 0x7efed01a2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:51.983 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:51.983+0000 7efed7800700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7efed0110af0 con 0x7efeb4005cc0 2024-04-24T08:39:51.984 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:51.983+0000 7efed7800700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7efed0110970 con 0x7efed019ec30 2024-04-24T08:39:51.985 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:51.984+0000 7efed4e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efeb4005cc0 0x7efed019e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:51.985 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:51.984+0000 7efed4e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efeb4005cc0 0x7efed019e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.189:42266/0 (socket says 172.21.15.189:42266) 2024-04-24T08:39:51.985 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:51.984+0000 7efec7e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efed019ec30 0x7efed01a2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:51.986 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:51.984+0000 7efed4e00700 1 -- 172.21.15.189:0/2601447965 learned_addr learned my addr 172.21.15.189:0/2601447965 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-04-24T08:39:51.986 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:51.984+0000 7efec7e00700 1 -- 172.21.15.189:0/2601447965 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efed019ec30 msgr2=0x7efed01a2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-24T08:39:51.986 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:51.984+0000 7efec7e00700 1 -- 172.21.15.189:0/2601447965 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efed019ec30 msgr2=0x7efed01a2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-24T08:39:51.986 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:51.984+0000 7efec7e00700 1 --2- 172.21.15.189:0/2601447965 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efed019ec30 0x7efed01a2fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-24T08:39:51.986 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:51.984+0000 7efec7e00700 1 --2- 172.21.15.189:0/2601447965 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efed019ec30 0x7efed01a2fc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T08:39:51.986 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:51.984+0000 7efed4e00700 1 -- 172.21.15.189:0/2601447965 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efed019ec30 msgr2=0x7efed01a2fc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T08:39:51.986 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:51.984+0000 7efed4e00700 1 --2- 172.21.15.189:0/2601447965 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efed019ec30 0x7efed01a2fc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:51.986 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:51.984+0000 7efed4e00700 1 -- 172.21.15.189:0/2601447965 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efed019f270 con 0x7efeb4005cc0 2024-04-24T08:39:51.986 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:51.985+0000 7efed4e00700 1 --2- 172.21.15.189:0/2601447965 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efeb4005cc0 0x7efed019e6f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7efec0012cb0 tx=0x7efec0004780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:39:51.987 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:51.986+0000 7efece000700 1 -- 172.21.15.189:0/2601447965 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7efec0004dc0 con 0x7efeb4005cc0 2024-04-24T08:39:51.988 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:51.986+0000 7efed7800700 1 -- 172.21.15.189:0/2601447965 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efed01a3500 con 0x7efeb4005cc0 2024-04-24T08:39:51.988 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:51.986+0000 7efece000700 1 -- 172.21.15.189:0/2601447965 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efec00044d0 con 0x7efeb4005cc0 2024-04-24T08:39:51.988 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:51.986+0000 7efece000700 1 -- 172.21.15.189:0/2601447965 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7efec0027620 con 0x7efeb4005cc0 2024-04-24T08:39:51.988 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:51.986+0000 7efed7800700 1 -- 172.21.15.189:0/2601447965 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efed01a3a60 con 0x7efeb4005cc0 2024-04-24T08:39:51.989 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:51.987+0000 7efece000700 1 -- 172.21.15.189:0/2601447965 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7efec0024040 con 0x7efeb4005cc0 2024-04-24T08:39:51.989 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:51.988+0000 7efed7800700 1 -- 172.21.15.189:0/2601447965 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efed00765d0 con 0x7efeb4005cc0 2024-04-24T08:39:51.989 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:51.988+0000 7efece000700 1 --2- 172.21.15.189:0/2601447965 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7efeac039460 0x7efeac03b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:51.989 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:51.988+0000 7efece000700 1 -- 172.21.15.189:0/2601447965 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7efec0022020 con 0x7efeb4005cc0 2024-04-24T08:39:51.989 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:51.989+0000 7efec7e00700 1 --2- 172.21.15.189:0/2601447965 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7efeac039460 0x7efeac03b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:51.990 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:51.990+0000 7efec7e00700 1 --2- 172.21.15.189:0/2601447965 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7efeac039460 0x7efeac03b920 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7efeb8009850 tx=0x7efeb8008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:39:52.004 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:52.004+0000 7efece000700 1 -- 172.21.15.189:0/2601447965 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efec002b030 con 0x7efeb4005cc0 2024-04-24T08:39:52.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:51 smithi083 ceph-mon[27907]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:39:52.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:51 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:39:52.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:51 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T08:39:52.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:51 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:39:52.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:51 smithi083 ceph-mon[27907]: Deploying daemon mon.b on smithi138 2024-04-24T08:39:52.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:51 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T08:39:52.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:51 smithi189 ceph-mon[31791]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:39:52.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:51 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:39:52.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:51 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T08:39:52.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:51 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:39:52.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:51 smithi189 ceph-mon[31791]: Deploying daemon mon.b on smithi138 2024-04-24T08:39:52.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:51 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T08:39:52.332 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:52.331+0000 7efed7800700 1 -- 172.21.15.189:0/2601447965 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7efed01a3cc0 con 0x7efeb4005cc0 2024-04-24T08:39:52.333 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:52.333+0000 7efece000700 1 -- 172.21.15.189:0/2601447965 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7efec0026c70 con 0x7efeb4005cc0 2024-04-24T08:39:52.333 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-04-24T08:39:52.333 INFO:teuthology.orchestra.run.smithi189.stdout:{"epoch":2,"fsid":"bd3e039e-0215-11ef-bc93-c7b262605968","modified":"2024-04-24T08:39:45.678435Z","created":"2024-04-24T08:37:28.106852Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:3300","nonce":0},{"type":"v1","addr":"172.21.15.83:6789","nonce":0}]},"addr":"172.21.15.83:6789/0","public_addr":"172.21.15.83:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:3300","nonce":0},{"type":"v1","addr":"172.21.15.189:6789","nonce":0}]},"addr":"172.21.15.189:6789/0","public_addr":"172.21.15.189:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-24T08:39:52.337 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:52.336+0000 7efed7800700 1 -- 172.21.15.189:0/2601447965 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7efeac039460 msgr2=0x7efeac03b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:52.337 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:52.336+0000 7efed7800700 1 --2- 172.21.15.189:0/2601447965 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7efeac039460 0x7efeac03b920 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7efeb8009850 tx=0x7efeb8008040 comp rx=0 tx=0).stop 2024-04-24T08:39:52.337 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:52.336+0000 7efed7800700 1 -- 172.21.15.189:0/2601447965 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efeb4005cc0 msgr2=0x7efed019e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:52.337 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:52.336+0000 7efed7800700 1 --2- 172.21.15.189:0/2601447965 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efeb4005cc0 0x7efed019e6f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7efec0012cb0 tx=0x7efec0004780 comp rx=0 tx=0).stop 2024-04-24T08:39:52.337 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:52.336+0000 7efed7800700 1 -- 172.21.15.189:0/2601447965 shutdown_connections 2024-04-24T08:39:52.337 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:52.336+0000 7efed7800700 1 --2- 172.21.15.189:0/2601447965 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7efeac039460 0x7efeac03b920 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:52.337 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:52.336+0000 7efed7800700 1 --2- 172.21.15.189:0/2601447965 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efeb4005cc0 0x7efed019e6f0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:52.337 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:52.336+0000 7efed7800700 1 --2- 172.21.15.189:0/2601447965 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efed019ec30 0x7efed01a2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:52.338 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:52.336+0000 7efed7800700 1 -- 172.21.15.189:0/2601447965 >> 172.21.15.189:0/2601447965 conn(0x7efed00fe0c0 msgr2=0x7efed0197000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:39:52.338 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:52.336+0000 7efed7800700 1 -- 172.21.15.189:0/2601447965 shutdown_connections 2024-04-24T08:39:52.338 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:52.336+0000 7efed7800700 1 -- 172.21.15.189:0/2601447965 wait complete. 2024-04-24T08:39:52.338 INFO:teuthology.orchestra.run.smithi189.stderr:dumped monmap epoch 2 2024-04-24T08:39:52.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:52 smithi189 ceph-mon[31791]: from='client.? 172.21.15.189:0/2601447965' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T08:39:53.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:52 smithi083 ceph-mon[27907]: from='client.? 172.21.15.189:0/2601447965' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T08:39:54.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:53 smithi083 ceph-mon[27907]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:39:54.197 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-24T08:39:54.198 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd3e039e-0215-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-24T08:39:54.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:53 smithi189 ceph-mon[31791]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:39:54.631 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T08:39:56.057 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:39:55 smithi083 ceph-mon[27907]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:39:56.143 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:39:55 smithi189 ceph-mon[31791]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:39:56.374 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:56.372+0000 7fa384e00700 1 -- 172.21.15.189:0/211934613 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa38010a810 msgr2=0x7fa38010abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:56.374 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:56.372+0000 7fa384e00700 1 --2- 172.21.15.189:0/211934613 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa38010a810 0x7fa38010abf0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa370009910 tx=0x7fa370009c20 comp rx=0 tx=0).stop 2024-04-24T08:39:56.374 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:56.372+0000 7fa384e00700 1 -- 172.21.15.189:0/211934613 shutdown_connections 2024-04-24T08:39:56.374 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:56.373+0000 7fa384e00700 1 --2- 172.21.15.189:0/211934613 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa38010a810 0x7fa38010abf0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:56.374 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:56.373+0000 7fa384e00700 1 -- 172.21.15.189:0/211934613 >> 172.21.15.189:0/211934613 conn(0x7fa380100300 msgr2=0x7fa380102720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:39:56.374 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:56.373+0000 7fa384e00700 1 -- 172.21.15.189:0/211934613 shutdown_connections 2024-04-24T08:39:56.374 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:56.373+0000 7fa384e00700 1 -- 172.21.15.189:0/211934613 wait complete. 2024-04-24T08:39:56.375 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:56.374+0000 7fa384e00700 1 Processor -- start 2024-04-24T08:39:56.375 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:56.374+0000 7fa384e00700 1 -- start start 2024-04-24T08:39:56.375 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:56.375+0000 7fa384e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa38010a810 0x7fa38019a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:56.375 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:56.375+0000 7fa384e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa38019a960 0x7fa38019ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:56.375 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:56.375+0000 7fa384e00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa38010ed00 con 0x7fa38019a960 2024-04-24T08:39:56.375 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:56.375+0000 7fa384e00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa38010eb80 con 0x7fa38010a810 2024-04-24T08:39:56.376 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:56.375+0000 7fa37de00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa38010a810 0x7fa38019a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:56.376 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:56.375+0000 7fa37de00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa38010a810 0x7fa38019a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.189:42852/0 (socket says 172.21.15.189:42852) 2024-04-24T08:39:56.376 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:56.375+0000 7fa37de00700 1 -- 172.21.15.189:0/2209619017 learned_addr learned my addr 172.21.15.189:0/2209619017 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-04-24T08:39:56.376 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:56.375+0000 7fa377e00700 1 --2- 172.21.15.189:0/2209619017 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa38019a960 0x7fa38019ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:56.377 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:56.376+0000 7fa37de00700 1 -- 172.21.15.189:0/2209619017 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa38019a960 msgr2=0x7fa38019ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:56.377 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:56.376+0000 7fa37de00700 1 --2- 172.21.15.189:0/2209619017 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa38019a960 0x7fa38019ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:56.377 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:56.376+0000 7fa37de00700 1 -- 172.21.15.189:0/2209619017 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa38019afa0 con 0x7fa38010a810 2024-04-24T08:39:56.377 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:56.376+0000 7fa37de00700 1 --2- 172.21.15.189:0/2209619017 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa38010a810 0x7fa38019a420 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fa370006010 tx=0x7fa370004f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:39:56.377 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:56.376+0000 7fa376a00700 1 -- 172.21.15.189:0/2209619017 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fa370005dc0 con 0x7fa38010a810 2024-04-24T08:39:56.377 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:56.376+0000 7fa376a00700 1 -- 172.21.15.189:0/2209619017 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa37000be10 con 0x7fa38010a810 2024-04-24T08:39:56.378 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:56.376+0000 7fa376a00700 1 -- 172.21.15.189:0/2209619017 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fa370022de0 con 0x7fa38010a810 2024-04-24T08:39:56.378 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:56.377+0000 7fa384e00700 1 -- 172.21.15.189:0/2209619017 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa38019f230 con 0x7fa38010a810 2024-04-24T08:39:56.379 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:56.377+0000 7fa384e00700 1 -- 172.21.15.189:0/2209619017 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa38019f790 con 0x7fa38010a810 2024-04-24T08:39:56.379 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:56.378+0000 7fa384e00700 1 -- 172.21.15.189:0/2209619017 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa38019fb00 con 0x7fa38010a810 2024-04-24T08:39:56.379 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:56.378+0000 7fa376a00700 1 -- 172.21.15.189:0/2209619017 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fa37002a070 con 0x7fa38010a810 2024-04-24T08:39:56.380 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:56.378+0000 7fa376a00700 1 --2- 172.21.15.189:0/2209619017 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fa364039440 0x7fa36403b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:39:56.380 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:56.378+0000 7fa376a00700 1 -- 172.21.15.189:0/2209619017 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fa37001e030 con 0x7fa38010a810 2024-04-24T08:39:56.380 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:56.379+0000 7fa377e00700 1 --2- 172.21.15.189:0/2209619017 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fa364039440 0x7fa36403b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:39:56.381 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:56.380+0000 7fa377e00700 1 --2- 172.21.15.189:0/2209619017 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fa364039440 0x7fa36403b900 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fa368009ad0 tx=0x7fa368009210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:39:56.385 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:56.386+0000 7fa376a00700 1 -- 172.21.15.189:0/2209619017 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa370013070 con 0x7fa38010a810 2024-04-24T08:39:56.458 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 podman[31421]: 2024-04-24 08:39:56.308220065 +0000 UTC m=+0.166789769 container create b10d504df99335c62eca80b46119fa2e150d7165d35ad2fe83a232c680edaf09 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mon-b, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , RELEASE=rishabh-main-apr17-a654945-1f81fb3) 2024-04-24T08:39:56.700 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:56.699+0000 7fa384e00700 1 -- 172.21.15.189:0/2209619017 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fa38019b780 con 0x7fa38010a810 2024-04-24T08:39:56.700 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:56.700+0000 7fa376a00700 1 -- 172.21.15.189:0/2209619017 <== mon.1 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7fa370031090 con 0x7fa38010a810 2024-04-24T08:39:56.701 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-04-24T08:39:56.701 INFO:teuthology.orchestra.run.smithi189.stdout:{"epoch":2,"fsid":"bd3e039e-0215-11ef-bc93-c7b262605968","modified":"2024-04-24T08:39:45.678435Z","created":"2024-04-24T08:37:28.106852Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:3300","nonce":0},{"type":"v1","addr":"172.21.15.83:6789","nonce":0}]},"addr":"172.21.15.83:6789/0","public_addr":"172.21.15.83:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:3300","nonce":0},{"type":"v1","addr":"172.21.15.189:6789","nonce":0}]},"addr":"172.21.15.189:6789/0","public_addr":"172.21.15.189:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-24T08:39:56.704 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:56.703+0000 7fa384e00700 1 -- 172.21.15.189:0/2209619017 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fa364039440 msgr2=0x7fa36403b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:56.704 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:56.703+0000 7fa384e00700 1 --2- 172.21.15.189:0/2209619017 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fa364039440 0x7fa36403b900 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fa368009ad0 tx=0x7fa368009210 comp rx=0 tx=0).stop 2024-04-24T08:39:56.704 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:56.703+0000 7fa384e00700 1 -- 172.21.15.189:0/2209619017 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa38010a810 msgr2=0x7fa38019a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:39:56.704 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:56.703+0000 7fa384e00700 1 --2- 172.21.15.189:0/2209619017 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa38010a810 0x7fa38019a420 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fa370006010 tx=0x7fa370004f80 comp rx=0 tx=0).stop 2024-04-24T08:39:56.704 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:56.704+0000 7fa384e00700 1 -- 172.21.15.189:0/2209619017 shutdown_connections 2024-04-24T08:39:56.705 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:56.704+0000 7fa384e00700 1 --2- 172.21.15.189:0/2209619017 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fa364039440 0x7fa36403b900 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:56.705 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:56.704+0000 7fa384e00700 1 --2- 172.21.15.189:0/2209619017 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa38010a810 0x7fa38019a420 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:56.705 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:56.704+0000 7fa384e00700 1 --2- 172.21.15.189:0/2209619017 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa38019a960 0x7fa38019ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:39:56.705 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:56.704+0000 7fa384e00700 1 -- 172.21.15.189:0/2209619017 >> 172.21.15.189:0/2209619017 conn(0x7fa380100300 msgr2=0x7fa380106310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:39:56.705 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:56.704+0000 7fa384e00700 1 -- 172.21.15.189:0/2209619017 shutdown_connections 2024-04-24T08:39:56.705 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:39:56.704+0000 7fa384e00700 1 -- 172.21.15.189:0/2209619017 wait complete. 2024-04-24T08:39:56.705 INFO:teuthology.orchestra.run.smithi189.stderr:dumped monmap epoch 2 2024-04-24T08:39:56.957 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 podman[31421]: 2024-04-24 08:39:56.636960028 +0000 UTC m=+0.495529799 container init b10d504df99335c62eca80b46119fa2e150d7165d35ad2fe83a232c680edaf09 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mon-b, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T08:39:56.957 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 podman[31421]: 2024-04-24 08:39:56.648084132 +0000 UTC m=+0.506653819 container start b10d504df99335c62eca80b46119fa2e150d7165d35ad2fe83a232c680edaf09 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mon-b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-24T08:39:56.957 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: set uid:gid to 167:167 (ceph:ceph) 2024-04-24T08:39:56.958 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev), process ceph-mon, pid 2 2024-04-24T08:39:56.958 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: pidfile_write: ignore empty --pid-file 2024-04-24T08:39:56.958 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: load: jerasure load: lrc 2024-04-24T08:39:56.958 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: RocksDB version: 7.9.2 2024-04-24T08:39:56.958 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Git sha 0 2024-04-24T08:39:56.958 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Compile date 2024-04-23 21:06:33 2024-04-24T08:39:56.958 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: DB SUMMARY 2024-04-24T08:39:56.958 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: DB Session ID: IKKNKUSMNKFVYWC7TRPC 2024-04-24T08:39:56.958 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: CURRENT file: CURRENT 2024-04-24T08:39:56.958 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: IDENTITY file: IDENTITY 2024-04-24T08:39:56.958 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-04-24T08:39:56.958 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-04-24T08:39:56.959 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-04-24T08:39:56.959 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.error_if_exists: 0 2024-04-24T08:39:56.959 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.create_if_missing: 0 2024-04-24T08:39:56.959 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.paranoid_checks: 1 2024-04-24T08:39:56.959 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-24T08:39:56.959 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-24T08:39:56.959 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-24T08:39:56.959 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.env: 0x55b4c4eeacc0 2024-04-24T08:39:56.959 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.fs: PosixFileSystem 2024-04-24T08:39:56.959 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.info_log: 0x55b4e3d6bc60 2024-04-24T08:39:56.959 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.max_file_opening_threads: 16 2024-04-24T08:39:56.960 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.statistics: (nil) 2024-04-24T08:39:56.960 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.use_fsync: 0 2024-04-24T08:39:56.960 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.max_log_file_size: 0 2024-04-24T08:39:56.960 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-24T08:39:56.960 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-24T08:39:56.960 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.keep_log_file_num: 1000 2024-04-24T08:39:56.960 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.recycle_log_file_num: 0 2024-04-24T08:39:56.960 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.allow_fallocate: 1 2024-04-24T08:39:56.960 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.allow_mmap_reads: 0 2024-04-24T08:39:56.960 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.allow_mmap_writes: 0 2024-04-24T08:39:56.960 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.use_direct_reads: 0 2024-04-24T08:39:56.960 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-24T08:39:56.961 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.create_missing_column_families: 0 2024-04-24T08:39:56.961 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.db_log_dir: 2024-04-24T08:39:56.961 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.wal_dir: 2024-04-24T08:39:56.961 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-24T08:39:56.961 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-24T08:39:56.961 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-24T08:39:56.961 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-24T08:39:56.961 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-24T08:39:56.961 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-24T08:39:56.961 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.advise_random_on_open: 1 2024-04-24T08:39:56.961 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.db_write_buffer_size: 0 2024-04-24T08:39:56.961 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.write_buffer_manager: 0x55b4e3d3a1e0 2024-04-24T08:39:56.961 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-24T08:39:56.961 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-24T08:39:56.962 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-24T08:39:56.962 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.rate_limiter: (nil) 2024-04-24T08:39:56.962 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-24T08:39:56.962 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.wal_recovery_mode: 2 2024-04-24T08:39:56.962 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.enable_thread_tracking: 0 2024-04-24T08:39:56.962 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.enable_pipelined_write: 0 2024-04-24T08:39:56.962 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.unordered_write: 0 2024-04-24T08:39:56.962 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-24T08:39:56.962 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-24T08:39:56.962 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-24T08:39:56.962 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-24T08:39:56.962 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.row_cache: None 2024-04-24T08:39:56.962 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.wal_filter: None 2024-04-24T08:39:56.963 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-24T08:39:56.963 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.allow_ingest_behind: 0 2024-04-24T08:39:56.963 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.two_write_queues: 0 2024-04-24T08:39:56.963 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.manual_wal_flush: 0 2024-04-24T08:39:56.963 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.wal_compression: 0 2024-04-24T08:39:56.964 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.atomic_flush: 0 2024-04-24T08:39:56.964 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-24T08:39:56.964 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-24T08:39:56.964 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-24T08:39:56.964 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.log_readahead_size: 0 2024-04-24T08:39:56.964 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-24T08:39:56.964 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.best_efforts_recovery: 0 2024-04-24T08:39:56.964 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-24T08:39:56.964 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-24T08:39:56.964 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.allow_data_in_errors: 0 2024-04-24T08:39:56.964 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.db_host_id: __hostname__ 2024-04-24T08:39:56.964 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-24T08:39:56.965 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.max_background_jobs: 2 2024-04-24T08:39:56.965 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.max_background_compactions: -1 2024-04-24T08:39:56.965 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.max_subcompactions: 1 2024-04-24T08:39:56.965 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-24T08:39:56.965 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-24T08:39:56.965 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-24T08:39:56.965 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.max_total_wal_size: 0 2024-04-24T08:39:56.965 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-24T08:39:56.965 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-24T08:39:56.965 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-24T08:39:56.965 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-24T08:39:56.965 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.max_open_files: -1 2024-04-24T08:39:56.965 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.bytes_per_sync: 0 2024-04-24T08:39:56.965 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-24T08:39:56.966 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-24T08:39:56.966 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.compaction_readahead_size: 0 2024-04-24T08:39:56.966 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.max_background_flushes: -1 2024-04-24T08:39:56.966 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Compression algorithms supported: 2024-04-24T08:39:56.966 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-24T08:39:56.966 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: kZSTD supported: 0 2024-04-24T08:39:56.966 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: kXpressCompression supported: 0 2024-04-24T08:39:56.967 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: kLZ4HCCompression supported: 1 2024-04-24T08:39:56.967 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: kZlibCompression supported: 1 2024-04-24T08:39:56.967 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: kSnappyCompression supported: 1 2024-04-24T08:39:56.967 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: kLZ4Compression supported: 1 2024-04-24T08:39:56.967 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: kBZip2Compression supported: 0 2024-04-24T08:39:56.967 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-24T08:39:56.968 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-24T08:39:56.968 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-04-24T08:39:56.968 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-24T08:39:56.968 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-24T08:39:56.968 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.merge_operator: 2024-04-24T08:39:56.969 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.compaction_filter: None 2024-04-24T08:39:56.969 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.compaction_filter_factory: None 2024-04-24T08:39:56.969 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.sst_partitioner_factory: None 2024-04-24T08:39:56.969 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-24T08:39:56.969 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-24T08:39:56.969 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55b4e3d6be20) 2024-04-24T08:39:56.969 INFO:journalctl@ceph.mon.b.smithi138.stdout: cache_index_and_filter_blocks: 1 2024-04-24T08:39:56.969 INFO:journalctl@ceph.mon.b.smithi138.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-24T08:39:56.969 INFO:journalctl@ceph.mon.b.smithi138.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-24T08:39:56.970 INFO:journalctl@ceph.mon.b.smithi138.stdout: pin_top_level_index_and_filter: 1 2024-04-24T08:39:56.970 INFO:journalctl@ceph.mon.b.smithi138.stdout: index_type: 0 2024-04-24T08:39:56.970 INFO:journalctl@ceph.mon.b.smithi138.stdout: data_block_index_type: 0 2024-04-24T08:39:56.970 INFO:journalctl@ceph.mon.b.smithi138.stdout: index_shortening: 1 2024-04-24T08:39:56.970 INFO:journalctl@ceph.mon.b.smithi138.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-24T08:39:56.970 INFO:journalctl@ceph.mon.b.smithi138.stdout: checksum: 4 2024-04-24T08:39:56.970 INFO:journalctl@ceph.mon.b.smithi138.stdout: no_block_cache: 0 2024-04-24T08:39:56.970 INFO:journalctl@ceph.mon.b.smithi138.stdout: block_cache: 0x55b4e3d6d090 2024-04-24T08:39:56.970 INFO:journalctl@ceph.mon.b.smithi138.stdout: block_cache_name: BinnedLRUCache 2024-04-24T08:39:56.970 INFO:journalctl@ceph.mon.b.smithi138.stdout: block_cache_options: 2024-04-24T08:39:56.970 INFO:journalctl@ceph.mon.b.smithi138.stdout: capacity : 536870912 2024-04-24T08:39:56.970 INFO:journalctl@ceph.mon.b.smithi138.stdout: num_shard_bits : 4 2024-04-24T08:39:56.970 INFO:journalctl@ceph.mon.b.smithi138.stdout: strict_capacity_limit : 0 2024-04-24T08:39:56.970 INFO:journalctl@ceph.mon.b.smithi138.stdout: high_pri_pool_ratio: 0.000 2024-04-24T08:39:56.971 INFO:journalctl@ceph.mon.b.smithi138.stdout: block_cache_compressed: (nil) 2024-04-24T08:39:56.971 INFO:journalctl@ceph.mon.b.smithi138.stdout: persistent_cache: (nil) 2024-04-24T08:39:56.971 INFO:journalctl@ceph.mon.b.smithi138.stdout: block_size: 4096 2024-04-24T08:39:56.971 INFO:journalctl@ceph.mon.b.smithi138.stdout: block_size_deviation: 10 2024-04-24T08:39:56.971 INFO:journalctl@ceph.mon.b.smithi138.stdout: block_restart_interval: 16 2024-04-24T08:39:56.971 INFO:journalctl@ceph.mon.b.smithi138.stdout: index_block_restart_interval: 1 2024-04-24T08:39:56.971 INFO:journalctl@ceph.mon.b.smithi138.stdout: metadata_block_size: 4096 2024-04-24T08:39:56.971 INFO:journalctl@ceph.mon.b.smithi138.stdout: partition_filters: 0 2024-04-24T08:39:56.971 INFO:journalctl@ceph.mon.b.smithi138.stdout: use_delta_encoding: 1 2024-04-24T08:39:56.971 INFO:journalctl@ceph.mon.b.smithi138.stdout: filter_policy: bloomfilter 2024-04-24T08:39:56.971 INFO:journalctl@ceph.mon.b.smithi138.stdout: whole_key_filtering: 1 2024-04-24T08:39:56.971 INFO:journalctl@ceph.mon.b.smithi138.stdout: verify_compression: 0 2024-04-24T08:39:56.971 INFO:journalctl@ceph.mon.b.smithi138.stdout: read_amp_bytes_per_bit: 0 2024-04-24T08:39:56.971 INFO:journalctl@ceph.mon.b.smithi138.stdout: format_version: 5 2024-04-24T08:39:56.971 INFO:journalctl@ceph.mon.b.smithi138.stdout: enable_index_compression: 1 2024-04-24T08:39:56.971 INFO:journalctl@ceph.mon.b.smithi138.stdout: block_align: 0 2024-04-24T08:39:56.972 INFO:journalctl@ceph.mon.b.smithi138.stdout: max_auto_readahead_size: 262144 2024-04-24T08:39:56.972 INFO:journalctl@ceph.mon.b.smithi138.stdout: prepopulate_block_cache: 0 2024-04-24T08:39:56.972 INFO:journalctl@ceph.mon.b.smithi138.stdout: initial_auto_readahead_size: 8192 2024-04-24T08:39:56.972 INFO:journalctl@ceph.mon.b.smithi138.stdout: num_file_reads_for_auto_readahead: 2 2024-04-24T08:39:56.972 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.write_buffer_size: 33554432 2024-04-24T08:39:56.972 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.max_write_buffer_number: 2 2024-04-24T08:39:56.972 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.compression: NoCompression 2024-04-24T08:39:56.972 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.bottommost_compression: Disabled 2024-04-24T08:39:56.972 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.prefix_extractor: nullptr 2024-04-24T08:39:56.972 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-24T08:39:56.972 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.num_levels: 7 2024-04-24T08:39:56.972 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-24T08:39:56.972 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-24T08:39:56.973 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-24T08:39:56.973 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-24T08:39:56.973 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-24T08:39:56.973 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-24T08:39:56.973 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-24T08:39:56.973 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-24T08:39:56.973 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-24T08:39:56.973 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-24T08:39:56.973 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-24T08:39:56.973 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-24T08:39:56.973 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-24T08:39:56.973 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.compression_opts.level: 32767 2024-04-24T08:39:56.973 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.compression_opts.strategy: 0 2024-04-24T08:39:56.973 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-24T08:39:56.973 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-24T08:39:56.974 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-24T08:39:56.974 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-24T08:39:56.974 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.compression_opts.enabled: false 2024-04-24T08:39:56.974 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-24T08:39:56.974 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-24T08:39:56.974 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-24T08:39:56.974 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-24T08:39:56.974 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.target_file_size_base: 67108864 2024-04-24T08:39:56.974 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-24T08:39:56.974 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-24T08:39:56.974 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-24T08:39:56.974 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-24T08:39:56.974 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-24T08:39:56.975 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-24T08:39:56.976 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-24T08:39:56.976 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-24T08:39:56.977 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-24T08:39:56.977 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-24T08:39:56.977 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-24T08:39:56.977 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-24T08:39:56.977 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-24T08:39:56.977 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-24T08:39:56.977 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.arena_block_size: 1048576 2024-04-24T08:39:56.977 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-24T08:39:56.977 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-24T08:39:56.977 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.disable_auto_compactions: 0 2024-04-24T08:39:56.977 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-24T08:39:56.977 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-24T08:39:56.977 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-24T08:39:56.977 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-24T08:39:56.977 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-24T08:39:56.978 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-24T08:39:56.978 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-24T08:39:56.978 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-24T08:39:56.978 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-24T08:39:56.978 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-24T08:39:56.978 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-24T08:39:56.978 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.inplace_update_support: 0 2024-04-24T08:39:56.978 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-24T08:39:56.978 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-24T08:39:56.978 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-24T08:39:56.978 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-24T08:39:56.978 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.bloom_locality: 0 2024-04-24T08:39:56.978 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.max_successive_merges: 0 2024-04-24T08:39:56.978 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-24T08:39:56.979 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.paranoid_file_checks: 0 2024-04-24T08:39:56.979 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.force_consistency_checks: 1 2024-04-24T08:39:56.979 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.report_bg_io_stats: 0 2024-04-24T08:39:56.979 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.ttl: 2592000 2024-04-24T08:39:56.979 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-24T08:39:56.979 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-24T08:39:56.979 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-24T08:39:56.979 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.enable_blob_files: false 2024-04-24T08:39:56.979 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.min_blob_size: 0 2024-04-24T08:39:56.979 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.blob_file_size: 268435456 2024-04-24T08:39:56.979 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-24T08:39:56.979 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-24T08:39:56.979 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-24T08:39:56.979 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-24T08:39:56.979 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-24T08:39:56.980 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.blob_file_starting_level: 0 2024-04-24T08:39:56.980 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-24T08:39:56.980 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-24T08:39:56.980 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-04-24T08:39:56.980 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 482e2d05-8231-443e-9019-efb3bbfad472 2024-04-24T08:39:56.980 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713947996706664, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-04-24T08:39:56.980 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-04-24T08:39:56.980 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713947996708025, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1773, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1713947996, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "482e2d05-8231-443e-9019-efb3bbfad472", "db_session_id": "IKKNKUSMNKFVYWC7TRPC", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-04-24T08:39:56.980 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713947996708276, "job": 1, "event": "recovery_finished"} 2024-04-24T08:39:56.980 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-04-24T08:39:56.980 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-24T08:39:56.980 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55b4e3e5a000 2024-04-24T08:39:56.980 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: DB pointer 0x55b4e3e44000 2024-04-24T08:39:56.981 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-24T08:39:56.981 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-24T08:39:56.981 INFO:journalctl@ceph.mon.b.smithi138.stdout: ** DB Stats ** 2024-04-24T08:39:56.981 INFO:journalctl@ceph.mon.b.smithi138.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-24T08:39:56.981 INFO:journalctl@ceph.mon.b.smithi138.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-24T08:39:56.981 INFO:journalctl@ceph.mon.b.smithi138.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-24T08:39:56.981 INFO:journalctl@ceph.mon.b.smithi138.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-24T08:39:56.981 INFO:journalctl@ceph.mon.b.smithi138.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-24T08:39:56.981 INFO:journalctl@ceph.mon.b.smithi138.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-24T08:39:56.981 INFO:journalctl@ceph.mon.b.smithi138.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-24T08:39:56.981 INFO:journalctl@ceph.mon.b.smithi138.stdout: 2024-04-24T08:39:56.981 INFO:journalctl@ceph.mon.b.smithi138.stdout: ** Compaction Stats [default] ** 2024-04-24T08:39:56.981 INFO:journalctl@ceph.mon.b.smithi138.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-24T08:39:56.982 INFO:journalctl@ceph.mon.b.smithi138.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-24T08:39:56.982 INFO:journalctl@ceph.mon.b.smithi138.stdout: L0 1/0 1.73 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-24T08:39:56.982 INFO:journalctl@ceph.mon.b.smithi138.stdout: Sum 1/0 1.73 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-24T08:39:56.982 INFO:journalctl@ceph.mon.b.smithi138.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-24T08:39:56.982 INFO:journalctl@ceph.mon.b.smithi138.stdout: 2024-04-24T08:39:56.982 INFO:journalctl@ceph.mon.b.smithi138.stdout: ** Compaction Stats [default] ** 2024-04-24T08:39:56.982 INFO:journalctl@ceph.mon.b.smithi138.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-24T08:39:56.982 INFO:journalctl@ceph.mon.b.smithi138.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-24T08:39:56.982 INFO:journalctl@ceph.mon.b.smithi138.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-24T08:39:56.982 INFO:journalctl@ceph.mon.b.smithi138.stdout: 2024-04-24T08:39:56.982 INFO:journalctl@ceph.mon.b.smithi138.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-24T08:39:56.982 INFO:journalctl@ceph.mon.b.smithi138.stdout: 2024-04-24T08:39:56.982 INFO:journalctl@ceph.mon.b.smithi138.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-24T08:39:56.983 INFO:journalctl@ceph.mon.b.smithi138.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-24T08:39:56.983 INFO:journalctl@ceph.mon.b.smithi138.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-24T08:39:56.983 INFO:journalctl@ceph.mon.b.smithi138.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-24T08:39:56.983 INFO:journalctl@ceph.mon.b.smithi138.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-24T08:39:56.983 INFO:journalctl@ceph.mon.b.smithi138.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-24T08:39:56.983 INFO:journalctl@ceph.mon.b.smithi138.stdout: Cumulative compaction: 0.00 GB write, 0.28 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-24T08:39:56.983 INFO:journalctl@ceph.mon.b.smithi138.stdout: Interval compaction: 0.00 GB write, 0.28 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-24T08:39:56.983 INFO:journalctl@ceph.mon.b.smithi138.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-24T08:39:56.983 INFO:journalctl@ceph.mon.b.smithi138.stdout: Block cache BinnedLRUCache@0x55b4e3d6d090#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-04-24T08:39:56.983 INFO:journalctl@ceph.mon.b.smithi138.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-04-24T08:39:56.983 INFO:journalctl@ceph.mon.b.smithi138.stdout: 2024-04-24T08:39:56.983 INFO:journalctl@ceph.mon.b.smithi138.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-24T08:39:56.983 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-04-24T08:39:56.983 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: using public_addr v2:172.21.15.138:0/0 -> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] 2024-04-24T08:39:56.983 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: starting mon.b rank -1 at public addrs [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] at bind addrs [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid bd3e039e-0215-11ef-bc93-c7b262605968 2024-04-24T08:39:56.983 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: mon.b@-1(???) e0 preinit fsid bd3e039e-0215-11ef-bc93-c7b262605968 2024-04-24T08:39:56.984 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 bash[31421]: b10d504df99335c62eca80b46119fa2e150d7165d35ad2fe83a232c680edaf09 2024-04-24T08:39:56.984 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 systemd[1]: Started Ceph mon.b for bd3e039e-0215-11ef-bc93-c7b262605968. 2024-04-24T08:39:56.984 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: mon.b@-1(synchronizing).mds e1 new map 2024-04-24T08:39:56.984 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: mon.b@-1(synchronizing).mds e1 print_map 2024-04-24T08:39:56.984 INFO:journalctl@ceph.mon.b.smithi138.stdout: e1 2024-04-24T08:39:56.984 INFO:journalctl@ceph.mon.b.smithi138.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-24T08:39:56.984 INFO:journalctl@ceph.mon.b.smithi138.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-24T08:39:56.985 INFO:journalctl@ceph.mon.b.smithi138.stdout: legacy client fscid: -1 2024-04-24T08:39:56.985 INFO:journalctl@ceph.mon.b.smithi138.stdout: 2024-04-24T08:39:56.985 INFO:journalctl@ceph.mon.b.smithi138.stdout: No filesystems configured 2024-04-24T08:39:56.985 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-04-24T08:39:56.985 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-04-24T08:39:56.985 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-04-24T08:39:56.985 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-04-24T08:39:56.985 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-04-24T08:39:56.985 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-04-24T08:39:56.985 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-24T08:39:56.985 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-24T08:39:56.985 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-24T08:39:56.985 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-24T08:39:56.985 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: Updating smithi083:/var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T08:39:56.986 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:39:56.986 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:39:56.986 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:39:56.986 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:39:56.986 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi138", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:39:56.986 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: Deploying cephadm binary to smithi138 2024-04-24T08:39:56.986 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:39:56.986 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:39:56.986 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:39:56.986 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:39:56.986 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:39:56.986 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: Added host smithi138 2024-04-24T08:39:56.987 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:39:56.987 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:39:56.987 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:39:56.987 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:39:56.987 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:39:56.987 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T08:39:56.987 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:39:56.987 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:39:56.987 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi189", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:39:56.987 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: Deploying cephadm binary to smithi189 2024-04-24T08:39:56.987 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:39:56.987 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:39:56.987 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:39:56.987 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:39:56.987 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi138", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:39:56.988 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:39:56.988 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:39:56.988 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: Updating smithi138:/etc/ceph/ceph.conf 2024-04-24T08:39:56.988 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:39:56.988 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:39:56.988 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: Updating smithi138:/var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T08:39:56.988 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: Updating smithi138:/etc/ceph/ceph.client.admin.keyring 2024-04-24T08:39:56.988 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:39:56.988 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:39:56.988 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: Added host smithi189 2024-04-24T08:39:56.988 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: Updating smithi138:/var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T08:39:56.988 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:39:56.988 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:39:56.988 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:39:56.988 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:39:56.989 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:39:56.989 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:39:56.989 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:39:56.989 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T08:39:56.989 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:39:56.989 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:39:56.989 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:39:56.989 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/1482720420' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-24T08:39:56.989 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/1482720420' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-24T08:39:56.989 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: osdmap e4: 0 total, 0 up, 0 in 2024-04-24T08:39:56.989 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:39:56.989 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:39:56.989 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:39:56.989 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:39:56.989 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:39:56.989 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:39:56.990 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi189", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:39:56.990 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:39:56.990 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:39:56.990 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: Updating smithi189:/etc/ceph/ceph.conf 2024-04-24T08:39:56.990 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: Updating smithi189:/var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T08:39:56.990 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: Updating smithi189:/etc/ceph/ceph.client.admin.keyring 2024-04-24T08:39:56.990 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:39:56.990 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi083:172.21.15.83=a;smithi138:172.21.15.138=b;smithi189:172.21.15.189=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:39:56.990 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: Saving service mon spec with placement smithi083:172.21.15.83=a;smithi138:172.21.15.138=b;smithi189:172.21.15.189=c;count:3 2024-04-24T08:39:56.990 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:39:56.990 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: Updating smithi189:/var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T08:39:56.990 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:39:56.990 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:39:56.990 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:39:56.991 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T08:39:56.991 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:39:56.991 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: Deploying daemon mon.c on smithi189 2024-04-24T08:39:56.991 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:39:56.991 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: from='client.? 172.21.15.189:0/182869854' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T08:39:56.991 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:39:56.991 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:39:56.991 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T08:39:56.991 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T08:39:56.991 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: mon.a calling monitor election 2024-04-24T08:39:56.991 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T08:39:56.991 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:39:56.991 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T08:39:56.991 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: mon.c calling monitor election 2024-04-24T08:39:56.991 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T08:39:56.991 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:39:56.992 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T08:39:56.992 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T08:39:56.992 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-24T08:39:56.992 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: monmap epoch 2 2024-04-24T08:39:56.992 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: fsid bd3e039e-0215-11ef-bc93-c7b262605968 2024-04-24T08:39:56.992 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: last_changed 2024-04-24T08:39:45.678435+0000 2024-04-24T08:39:56.992 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: created 2024-04-24T08:37:28.106852+0000 2024-04-24T08:39:56.992 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: min_mon_release 19 (squid) 2024-04-24T08:39:56.992 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: election_strategy: 1 2024-04-24T08:39:56.992 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: 0: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.a 2024-04-24T08:39:56.992 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: 1: [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] mon.c 2024-04-24T08:39:56.992 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: fsmap 2024-04-24T08:39:56.992 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: osdmap e4: 0 total, 0 up, 0 in 2024-04-24T08:39:56.992 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: mgrmap e13: x(active, since 70s) 2024-04-24T08:39:56.992 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: overall HEALTH_OK 2024-04-24T08:39:56.993 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:39:56.993 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:39:56.993 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:39:56.993 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:39:56.993 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T08:39:56.993 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:39:56.993 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: Deploying daemon mon.b on smithi138 2024-04-24T08:39:56.993 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T08:39:56.993 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: from='client.? 172.21.15.189:0/2601447965' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T08:39:56.993 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:39:56.993 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:39:56.993 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:39:56 smithi138 ceph-mon[31441]: mon.b@-1(synchronizing).paxosservice(auth 1..4) refresh upgraded, format 0 -> 3 2024-04-24T08:39:58.348 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-24T08:39:58.348 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd3e039e-0215-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-24T08:39:58.786 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T08:40:02.129 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:40:02.126+0000 7f5deb200700 1 -- 172.21.15.189:0/1612763778 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dec10ea30 msgr2=0x7f5dec10ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:40:02.129 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:40:02.126+0000 7f5deb200700 1 --2- 172.21.15.189:0/1612763778 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dec10ea30 0x7f5dec10ee10 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc005fa0 tx=0x7f5ddc00fe80 comp rx=0 tx=0).stop 2024-04-24T08:40:02.129 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:40:02.126+0000 7f5deb200700 1 -- 172.21.15.189:0/1612763778 shutdown_connections 2024-04-24T08:40:02.129 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:40:02.126+0000 7f5deb200700 1 --2- 172.21.15.189:0/1612763778 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dec10ea30 0x7f5dec10ee10 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:40:02.129 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:40:02.126+0000 7f5deb200700 1 -- 172.21.15.189:0/1612763778 >> 172.21.15.189:0/1612763778 conn(0x7f5dec078210 msgr2=0x7f5dec07a630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:40:02.129 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:40:02.126+0000 7f5deb200700 1 -- 172.21.15.189:0/1612763778 shutdown_connections 2024-04-24T08:40:02.129 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:40:02.126+0000 7f5deb200700 1 -- 172.21.15.189:0/1612763778 wait complete. 2024-04-24T08:40:02.130 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:01 smithi189 ceph-mon[31791]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:40:02.130 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:01 smithi189 ceph-mon[31791]: mon.c calling monitor election 2024-04-24T08:40:02.130 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:01 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T08:40:02.131 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:01 smithi189 ceph-mon[31791]: mon.a calling monitor election 2024-04-24T08:40:02.131 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:01 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T08:40:02.131 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:01 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T08:40:02.131 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:01 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T08:40:02.131 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:01 smithi189 ceph-mon[31791]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:40:02.131 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:01 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T08:40:02.132 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:01 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T08:40:02.132 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:01 smithi189 ceph-mon[31791]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:40:02.132 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:01 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T08:40:02.132 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:01 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T08:40:02.132 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:01 smithi189 ceph-mon[31791]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-24T08:40:02.132 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:01 smithi189 ceph-mon[31791]: monmap epoch 3 2024-04-24T08:40:02.132 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:01 smithi189 ceph-mon[31791]: fsid bd3e039e-0215-11ef-bc93-c7b262605968 2024-04-24T08:40:02.132 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:01 smithi189 ceph-mon[31791]: last_changed 2024-04-24T08:39:56.755904+0000 2024-04-24T08:40:02.133 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:01 smithi189 ceph-mon[31791]: created 2024-04-24T08:37:28.106852+0000 2024-04-24T08:40:02.133 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:01 smithi189 ceph-mon[31791]: min_mon_release 19 (squid) 2024-04-24T08:40:02.133 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:01 smithi189 ceph-mon[31791]: election_strategy: 1 2024-04-24T08:40:02.133 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:01 smithi189 ceph-mon[31791]: 0: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.a 2024-04-24T08:40:02.133 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:01 smithi189 ceph-mon[31791]: 1: [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] mon.c 2024-04-24T08:40:02.133 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:01 smithi189 ceph-mon[31791]: 2: [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] mon.b 2024-04-24T08:40:02.133 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:01 smithi189 ceph-mon[31791]: fsmap 2024-04-24T08:40:02.133 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:01 smithi189 ceph-mon[31791]: osdmap e4: 0 total, 0 up, 0 in 2024-04-24T08:40:02.133 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:01 smithi189 ceph-mon[31791]: mgrmap e13: x(active, since 81s) 2024-04-24T08:40:02.133 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:01 smithi189 ceph-mon[31791]: overall HEALTH_OK 2024-04-24T08:40:02.133 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:01 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:02.133 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:01 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:02.135 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:40:02.130+0000 7f5deb200700 1 Processor -- start 2024-04-24T08:40:02.135 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:40:02.130+0000 7f5deb200700 1 -- start start 2024-04-24T08:40:02.136 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:40:02.131+0000 7f5deb200700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5dec10ea30 0x7f5dec19e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:40:02.136 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:40:02.131+0000 7f5deb200700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5dec19eb60 0x7f5dec1a2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:40:02.136 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:40:02.131+0000 7f5deb200700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dec1a3430 0x7f5dec1a38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:40:02.136 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:40:02.131+0000 7f5deb200700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5dec112f00 con 0x7f5dec1a3430 2024-04-24T08:40:02.136 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:40:02.131+0000 7f5deb200700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5dec112d80 con 0x7f5dec10ea30 2024-04-24T08:40:02.136 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:40:02.131+0000 7f5deb200700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5dec113080 con 0x7f5dec19eb60 2024-04-24T08:40:02.136 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:40:02.131+0000 7f5de9400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5dec19eb60 0x7f5dec1a2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:40:02.136 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:40:02.132+0000 7f5de9400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5dec19eb60 0x7f5dec1a2ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.189:56064/0 (socket says 172.21.15.189:56064) 2024-04-24T08:40:02.136 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:40:02.132+0000 7f5de9400700 1 -- 172.21.15.189:0/3792647695 learned_addr learned my addr 172.21.15.189:0/3792647695 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-04-24T08:40:02.136 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:40:02.132+0000 7f5de9e00700 1 --2- 172.21.15.189:0/3792647695 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5dec10ea30 0x7f5dec19e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:40:02.136 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:40:02.132+0000 7f5dea800700 1 --2- 172.21.15.189:0/3792647695 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dec1a3430 0x7f5dec1a38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:40:02.136 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:40:02.132+0000 7f5de9e00700 1 -- 172.21.15.189:0/3792647695 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5dec10ea30 msgr2=0x7f5dec19e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-24T08:40:02.136 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:40:02.132+0000 7f5de9e00700 1 -- 172.21.15.189:0/3792647695 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5dec10ea30 msgr2=0x7f5dec19e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-24T08:40:02.137 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:40:02.132+0000 7f5de9e00700 1 --2- 172.21.15.189:0/3792647695 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5dec10ea30 0x7f5dec19e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-24T08:40:02.137 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:40:02.132+0000 7f5de9e00700 1 --2- 172.21.15.189:0/3792647695 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5dec10ea30 0x7f5dec19e620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T08:40:02.137 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:40:02.132+0000 7f5de9400700 1 -- 172.21.15.189:0/3792647695 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5dec10ea30 msgr2=0x7f5dec19e620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T08:40:02.137 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:40:02.132+0000 7f5de9400700 1 --2- 172.21.15.189:0/3792647695 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5dec10ea30 0x7f5dec19e620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:40:02.137 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:40:02.132+0000 7f5de9400700 1 -- 172.21.15.189:0/3792647695 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dec1a3430 msgr2=0x7f5dec1a38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:40:02.137 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:40:02.132+0000 7f5de9400700 1 --2- 172.21.15.189:0/3792647695 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dec1a3430 0x7f5dec1a38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:40:02.137 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:40:02.132+0000 7f5de9400700 1 -- 172.21.15.189:0/3792647695 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dec1ab1a0 con 0x7f5dec19eb60 2024-04-24T08:40:02.137 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:40:02.133+0000 7f5de9400700 1 --2- 172.21.15.189:0/3792647695 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5dec19eb60 0x7f5dec1a2ef0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f5dd400b780 tx=0x7f5dd400ba90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:40:02.137 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:40:02.133+0000 7f5de2a00700 1 -- 172.21.15.189:0/3792647695 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd4004230 con 0x7f5dec19eb60 2024-04-24T08:40:02.137 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:40:02.133+0000 7f5de2a00700 1 -- 172.21.15.189:0/3792647695 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5dd40043d0 con 0x7f5dec19eb60 2024-04-24T08:40:02.137 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:40:02.133+0000 7f5de2a00700 1 -- 172.21.15.189:0/3792647695 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd4011b60 con 0x7f5dec19eb60 2024-04-24T08:40:02.137 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:40:02.133+0000 7f5deb200700 1 -- 172.21.15.189:0/3792647695 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dec1ab3d0 con 0x7f5dec19eb60 2024-04-24T08:40:02.137 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:40:02.133+0000 7f5deb200700 1 -- 172.21.15.189:0/3792647695 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dec1ab910 con 0x7f5dec19eb60 2024-04-24T08:40:02.138 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:40:02.134+0000 7f5de2a00700 1 -- 172.21.15.189:0/3792647695 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f5dd4011d00 con 0x7f5dec19eb60 2024-04-24T08:40:02.138 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:40:02.135+0000 7f5deb200700 1 -- 172.21.15.189:0/3792647695 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5dec10a170 con 0x7f5dec19eb60 2024-04-24T08:40:02.139 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:40:02.135+0000 7f5de2a00700 1 --2- 172.21.15.189:0/3792647695 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f5dd0039ae0 0x7f5dd003bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:40:02.139 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:40:02.135+0000 7f5de2a00700 1 -- 172.21.15.189:0/3792647695 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f5dd404c4b0 con 0x7f5dec19eb60 2024-04-24T08:40:02.139 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:40:02.136+0000 7f5de9e00700 1 --2- 172.21.15.189:0/3792647695 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f5dd0039ae0 0x7f5dd003bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:40:02.139 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:40:02.137+0000 7f5de9e00700 1 --2- 172.21.15.189:0/3792647695 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f5dd0039ae0 0x7f5dd003bfa0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc005ed0 tx=0x7f5ddc005e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:40:02.142 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:40:02.141+0000 7f5de2a00700 1 -- 172.21.15.189:0/3792647695 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5dd40204e0 con 0x7f5dec19eb60 2024-04-24T08:40:02.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:01 smithi083 ceph-mon[27907]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:40:02.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:01 smithi083 ceph-mon[27907]: mon.c calling monitor election 2024-04-24T08:40:02.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:01 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T08:40:02.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:01 smithi083 ceph-mon[27907]: mon.a calling monitor election 2024-04-24T08:40:02.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:01 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T08:40:02.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:01 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T08:40:02.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:01 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T08:40:02.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:01 smithi083 ceph-mon[27907]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:40:02.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:01 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T08:40:02.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:01 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T08:40:02.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:01 smithi083 ceph-mon[27907]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:40:02.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:01 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T08:40:02.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:01 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T08:40:02.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:01 smithi083 ceph-mon[27907]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-24T08:40:02.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:01 smithi083 ceph-mon[27907]: monmap epoch 3 2024-04-24T08:40:02.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:01 smithi083 ceph-mon[27907]: fsid bd3e039e-0215-11ef-bc93-c7b262605968 2024-04-24T08:40:02.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:01 smithi083 ceph-mon[27907]: last_changed 2024-04-24T08:39:56.755904+0000 2024-04-24T08:40:02.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:01 smithi083 ceph-mon[27907]: created 2024-04-24T08:37:28.106852+0000 2024-04-24T08:40:02.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:01 smithi083 ceph-mon[27907]: min_mon_release 19 (squid) 2024-04-24T08:40:02.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:01 smithi083 ceph-mon[27907]: election_strategy: 1 2024-04-24T08:40:02.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:01 smithi083 ceph-mon[27907]: 0: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.a 2024-04-24T08:40:02.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:01 smithi083 ceph-mon[27907]: 1: [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] mon.c 2024-04-24T08:40:02.306 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:01 smithi083 ceph-mon[27907]: 2: [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] mon.b 2024-04-24T08:40:02.306 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:01 smithi083 ceph-mon[27907]: fsmap 2024-04-24T08:40:02.306 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:01 smithi083 ceph-mon[27907]: osdmap e4: 0 total, 0 up, 0 in 2024-04-24T08:40:02.306 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:01 smithi083 ceph-mon[27907]: mgrmap e13: x(active, since 81s) 2024-04-24T08:40:02.306 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:01 smithi083 ceph-mon[27907]: overall HEALTH_OK 2024-04-24T08:40:02.306 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:01 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:02.306 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:01 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:02.471 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:40:02.470+0000 7f5deb200700 1 -- 172.21.15.189:0/3792647695 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f5dec1abbd0 con 0x7f5dec19eb60 2024-04-24T08:40:02.472 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:40:02.470+0000 7f5de2a00700 1 -- 172.21.15.189:0/3792647695 <== mon.1 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1283 (secure 0 0 0) 0x7f5dd4017d50 con 0x7f5dec19eb60 2024-04-24T08:40:02.472 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-04-24T08:40:02.472 INFO:teuthology.orchestra.run.smithi189.stdout:{"epoch":3,"fsid":"bd3e039e-0215-11ef-bc93-c7b262605968","modified":"2024-04-24T08:39:56.755904Z","created":"2024-04-24T08:37:28.106852Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:3300","nonce":0},{"type":"v1","addr":"172.21.15.83:6789","nonce":0}]},"addr":"172.21.15.83:6789/0","public_addr":"172.21.15.83:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:3300","nonce":0},{"type":"v1","addr":"172.21.15.189:6789","nonce":0}]},"addr":"172.21.15.189:6789/0","public_addr":"172.21.15.189:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:3300","nonce":0},{"type":"v1","addr":"172.21.15.138:6789","nonce":0}]},"addr":"172.21.15.138:6789/0","public_addr":"172.21.15.138:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-24T08:40:02.475 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:40:02.473+0000 7f5deb200700 1 -- 172.21.15.189:0/3792647695 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f5dd0039ae0 msgr2=0x7f5dd003bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:40:02.475 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:40:02.473+0000 7f5deb200700 1 --2- 172.21.15.189:0/3792647695 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f5dd0039ae0 0x7f5dd003bfa0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc005ed0 tx=0x7f5ddc005e40 comp rx=0 tx=0).stop 2024-04-24T08:40:02.475 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:40:02.473+0000 7f5deb200700 1 -- 172.21.15.189:0/3792647695 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5dec19eb60 msgr2=0x7f5dec1a2ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:40:02.475 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:40:02.473+0000 7f5deb200700 1 --2- 172.21.15.189:0/3792647695 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5dec19eb60 0x7f5dec1a2ef0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f5dd400b780 tx=0x7f5dd400ba90 comp rx=0 tx=0).stop 2024-04-24T08:40:02.475 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:40:02.473+0000 7f5deb200700 1 -- 172.21.15.189:0/3792647695 shutdown_connections 2024-04-24T08:40:02.476 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:40:02.473+0000 7f5deb200700 1 --2- 172.21.15.189:0/3792647695 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f5dd0039ae0 0x7f5dd003bfa0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:40:02.476 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:40:02.473+0000 7f5deb200700 1 --2- 172.21.15.189:0/3792647695 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dec1a3430 0x7f5dec1a38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:40:02.476 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:40:02.473+0000 7f5deb200700 1 --2- 172.21.15.189:0/3792647695 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5dec10ea30 0x7f5dec19e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:40:02.476 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:40:02.473+0000 7f5deb200700 1 --2- 172.21.15.189:0/3792647695 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5dec19eb60 0x7f5dec1a2ef0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:40:02.476 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:40:02.474+0000 7f5deb200700 1 -- 172.21.15.189:0/3792647695 >> 172.21.15.189:0/3792647695 conn(0x7f5dec078210 msgr2=0x7f5dec0785f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:40:02.476 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:40:02.474+0000 7f5deb200700 1 -- 172.21.15.189:0/3792647695 shutdown_connections 2024-04-24T08:40:02.476 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:40:02.474+0000 7f5deb200700 1 -- 172.21.15.189:0/3792647695 wait complete. 2024-04-24T08:40:02.476 INFO:teuthology.orchestra.run.smithi189.stderr:dumped monmap epoch 3 2024-04-24T08:40:03.103 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:02 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:03.103 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:02 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:03.103 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:02 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:40:03.103 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:02 smithi189 ceph-mon[31791]: from='client.? 172.21.15.189:0/3792647695' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T08:40:03.103 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:02 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T08:40:03.152 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-04-24T08:40:03.153 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd3e039e-0215-11ef-bc93-c7b262605968 -- ceph config generate-minimal-conf 2024-04-24T08:40:03.219 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:02 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:03.219 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:02 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:03.219 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:02 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:40:03.219 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:02 smithi083 ceph-mon[27907]: from='client.? 172.21.15.189:0/3792647695' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T08:40:03.219 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:02 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T08:40:03.585 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T08:40:03.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:03 smithi189 ceph-mon[31791]: mon.b calling monitor election 2024-04-24T08:40:03.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:03 smithi189 ceph-mon[31791]: mon.b calling monitor election 2024-04-24T08:40:03.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:03 smithi189 ceph-mon[31791]: mon.c calling monitor election 2024-04-24T08:40:03.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:03 smithi189 ceph-mon[31791]: mon.a calling monitor election 2024-04-24T08:40:03.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:03 smithi189 ceph-mon[31791]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-24T08:40:03.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:03 smithi189 ceph-mon[31791]: monmap epoch 3 2024-04-24T08:40:03.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:03 smithi189 ceph-mon[31791]: fsid bd3e039e-0215-11ef-bc93-c7b262605968 2024-04-24T08:40:03.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:03 smithi189 ceph-mon[31791]: last_changed 2024-04-24T08:39:56.755904+0000 2024-04-24T08:40:03.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:03 smithi189 ceph-mon[31791]: created 2024-04-24T08:37:28.106852+0000 2024-04-24T08:40:03.959 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:03 smithi189 ceph-mon[31791]: min_mon_release 19 (squid) 2024-04-24T08:40:03.959 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:03 smithi189 ceph-mon[31791]: election_strategy: 1 2024-04-24T08:40:03.959 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:03 smithi189 ceph-mon[31791]: 0: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.a 2024-04-24T08:40:03.959 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:03 smithi189 ceph-mon[31791]: 1: [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] mon.c 2024-04-24T08:40:03.959 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:03 smithi189 ceph-mon[31791]: 2: [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] mon.b 2024-04-24T08:40:03.959 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:03 smithi189 ceph-mon[31791]: fsmap 2024-04-24T08:40:03.959 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:03 smithi189 ceph-mon[31791]: osdmap e4: 0 total, 0 up, 0 in 2024-04-24T08:40:03.959 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:03 smithi189 ceph-mon[31791]: mgrmap e13: x(active, since 83s) 2024-04-24T08:40:03.959 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:03 smithi189 ceph-mon[31791]: overall HEALTH_OK 2024-04-24T08:40:04.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:03 smithi083 ceph-mon[27907]: mon.b calling monitor election 2024-04-24T08:40:04.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:03 smithi083 ceph-mon[27907]: mon.b calling monitor election 2024-04-24T08:40:04.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:03 smithi083 ceph-mon[27907]: mon.c calling monitor election 2024-04-24T08:40:04.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:03 smithi083 ceph-mon[27907]: mon.a calling monitor election 2024-04-24T08:40:04.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:03 smithi083 ceph-mon[27907]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-24T08:40:04.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:03 smithi083 ceph-mon[27907]: monmap epoch 3 2024-04-24T08:40:04.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:03 smithi083 ceph-mon[27907]: fsid bd3e039e-0215-11ef-bc93-c7b262605968 2024-04-24T08:40:04.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:03 smithi083 ceph-mon[27907]: last_changed 2024-04-24T08:39:56.755904+0000 2024-04-24T08:40:04.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:03 smithi083 ceph-mon[27907]: created 2024-04-24T08:37:28.106852+0000 2024-04-24T08:40:04.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:03 smithi083 ceph-mon[27907]: min_mon_release 19 (squid) 2024-04-24T08:40:04.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:03 smithi083 ceph-mon[27907]: election_strategy: 1 2024-04-24T08:40:04.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:03 smithi083 ceph-mon[27907]: 0: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.a 2024-04-24T08:40:04.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:03 smithi083 ceph-mon[27907]: 1: [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] mon.c 2024-04-24T08:40:04.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:03 smithi083 ceph-mon[27907]: 2: [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] mon.b 2024-04-24T08:40:04.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:03 smithi083 ceph-mon[27907]: fsmap 2024-04-24T08:40:04.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:03 smithi083 ceph-mon[27907]: osdmap e4: 0 total, 0 up, 0 in 2024-04-24T08:40:04.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:03 smithi083 ceph-mon[27907]: mgrmap e13: x(active, since 83s) 2024-04-24T08:40:04.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:03 smithi083 ceph-mon[27907]: overall HEALTH_OK 2024-04-24T08:40:04.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:03 smithi138 ceph-mon[31441]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:40:04.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:03 smithi138 ceph-mon[31441]: mon.c calling monitor election 2024-04-24T08:40:04.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:03 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T08:40:04.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:03 smithi138 ceph-mon[31441]: mon.a calling monitor election 2024-04-24T08:40:04.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:03 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T08:40:04.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:03 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T08:40:04.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:03 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T08:40:04.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:03 smithi138 ceph-mon[31441]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:40:04.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:03 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T08:40:04.207 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:03 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T08:40:04.207 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:03 smithi138 ceph-mon[31441]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:40:04.207 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:03 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T08:40:04.207 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:03 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T08:40:04.207 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:03 smithi138 ceph-mon[31441]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-24T08:40:04.207 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:03 smithi138 ceph-mon[31441]: monmap epoch 3 2024-04-24T08:40:04.207 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:03 smithi138 ceph-mon[31441]: fsid bd3e039e-0215-11ef-bc93-c7b262605968 2024-04-24T08:40:04.207 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:03 smithi138 ceph-mon[31441]: last_changed 2024-04-24T08:39:56.755904+0000 2024-04-24T08:40:04.207 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:03 smithi138 ceph-mon[31441]: created 2024-04-24T08:37:28.106852+0000 2024-04-24T08:40:04.207 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:03 smithi138 ceph-mon[31441]: min_mon_release 19 (squid) 2024-04-24T08:40:04.207 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:03 smithi138 ceph-mon[31441]: election_strategy: 1 2024-04-24T08:40:04.207 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:03 smithi138 ceph-mon[31441]: 0: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.a 2024-04-24T08:40:04.207 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:03 smithi138 ceph-mon[31441]: 1: [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] mon.c 2024-04-24T08:40:04.207 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:03 smithi138 ceph-mon[31441]: 2: [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] mon.b 2024-04-24T08:40:04.207 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:03 smithi138 ceph-mon[31441]: fsmap 2024-04-24T08:40:04.207 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:03 smithi138 ceph-mon[31441]: osdmap e4: 0 total, 0 up, 0 in 2024-04-24T08:40:04.208 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:03 smithi138 ceph-mon[31441]: mgrmap e13: x(active, since 81s) 2024-04-24T08:40:04.208 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:03 smithi138 ceph-mon[31441]: overall HEALTH_OK 2024-04-24T08:40:04.208 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:03 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:04.208 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:03 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:04.208 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:03 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:04.208 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:03 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:04.208 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:03 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:40:04.208 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:03 smithi138 ceph-mon[31441]: from='client.? 172.21.15.189:0/3792647695' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T08:40:04.208 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:03 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T08:40:04.208 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:03 smithi138 ceph-mon[31441]: mon.b calling monitor election 2024-04-24T08:40:04.208 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:03 smithi138 ceph-mon[31441]: mon.b calling monitor election 2024-04-24T08:40:04.208 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:03 smithi138 ceph-mon[31441]: mon.c calling monitor election 2024-04-24T08:40:04.208 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:03 smithi138 ceph-mon[31441]: mon.a calling monitor election 2024-04-24T08:40:04.208 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:03 smithi138 ceph-mon[31441]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-24T08:40:04.208 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:03 smithi138 ceph-mon[31441]: monmap epoch 3 2024-04-24T08:40:04.208 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:03 smithi138 ceph-mon[31441]: fsid bd3e039e-0215-11ef-bc93-c7b262605968 2024-04-24T08:40:04.208 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:03 smithi138 ceph-mon[31441]: last_changed 2024-04-24T08:39:56.755904+0000 2024-04-24T08:40:04.209 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:03 smithi138 ceph-mon[31441]: created 2024-04-24T08:37:28.106852+0000 2024-04-24T08:40:04.209 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:03 smithi138 ceph-mon[31441]: min_mon_release 19 (squid) 2024-04-24T08:40:04.209 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:03 smithi138 ceph-mon[31441]: election_strategy: 1 2024-04-24T08:40:04.209 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:03 smithi138 ceph-mon[31441]: 0: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.a 2024-04-24T08:40:04.209 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:03 smithi138 ceph-mon[31441]: 1: [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] mon.c 2024-04-24T08:40:04.209 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:03 smithi138 ceph-mon[31441]: 2: [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] mon.b 2024-04-24T08:40:04.209 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:03 smithi138 ceph-mon[31441]: fsmap 2024-04-24T08:40:04.209 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:03 smithi138 ceph-mon[31441]: osdmap e4: 0 total, 0 up, 0 in 2024-04-24T08:40:04.209 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:03 smithi138 ceph-mon[31441]: mgrmap e13: x(active, since 83s) 2024-04-24T08:40:04.209 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:03 smithi138 ceph-mon[31441]: overall HEALTH_OK 2024-04-24T08:40:05.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:05 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:05.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:05 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:05.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:05 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:05.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:05 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:05.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:05 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:40:05.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:05 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:40:05.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:05 smithi138 ceph-mon[31441]: Updating smithi083:/etc/ceph/ceph.conf 2024-04-24T08:40:05.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:05 smithi138 ceph-mon[31441]: Updating smithi138:/etc/ceph/ceph.conf 2024-04-24T08:40:05.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:05 smithi138 ceph-mon[31441]: Updating smithi189:/etc/ceph/ceph.conf 2024-04-24T08:40:05.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:05 smithi138 ceph-mon[31441]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:40:05.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:05 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T08:40:05.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:05 smithi138 ceph-mon[31441]: Updating smithi083:/var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T08:40:05.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:05 smithi138 ceph-mon[31441]: Updating smithi189:/var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T08:40:05.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:05 smithi138 ceph-mon[31441]: Updating smithi138:/var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T08:40:05.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:05 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:05.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:05 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:05.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:05 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:05.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:05 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:05.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:05 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:40:05.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:05 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:40:05.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:05 smithi189 ceph-mon[31791]: Updating smithi083:/etc/ceph/ceph.conf 2024-04-24T08:40:05.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:05 smithi189 ceph-mon[31791]: Updating smithi138:/etc/ceph/ceph.conf 2024-04-24T08:40:05.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:05 smithi189 ceph-mon[31791]: Updating smithi189:/etc/ceph/ceph.conf 2024-04-24T08:40:05.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:05 smithi189 ceph-mon[31791]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:40:05.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:05 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T08:40:05.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:05 smithi189 ceph-mon[31791]: Updating smithi083:/var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T08:40:05.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:05 smithi189 ceph-mon[31791]: Updating smithi189:/var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T08:40:05.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:05 smithi189 ceph-mon[31791]: Updating smithi138:/var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T08:40:05.467 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:05 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:05.467 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:05 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:05.467 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:05 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:05.467 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:05 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:05.467 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:05 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:40:05.467 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:05 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:40:05.467 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:05 smithi083 ceph-mon[27907]: Updating smithi083:/etc/ceph/ceph.conf 2024-04-24T08:40:05.467 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:05 smithi083 ceph-mon[27907]: Updating smithi138:/etc/ceph/ceph.conf 2024-04-24T08:40:05.467 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:05 smithi083 ceph-mon[27907]: Updating smithi189:/etc/ceph/ceph.conf 2024-04-24T08:40:05.467 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:05 smithi083 ceph-mon[27907]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:40:05.467 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:05 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T08:40:05.468 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:05 smithi083 ceph-mon[27907]: Updating smithi083:/var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T08:40:05.468 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:05 smithi083 ceph-mon[27907]: Updating smithi189:/var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T08:40:05.468 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:05 smithi083 ceph-mon[27907]: Updating smithi138:/var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T08:40:05.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:40:05.995+0000 7f73ff400700 1 -- 172.21.15.83:0/640129614 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f73f810a720 msgr2=0x7f73f810ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:40:05.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:40:05.995+0000 7f73ff400700 1 --2- 172.21.15.83:0/640129614 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f73f810a720 0x7f73f810ab00 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f73ec009910 tx=0x7f73ec009c20 comp rx=0 tx=0).stop 2024-04-24T08:40:05.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:40:05.995+0000 7f73ff400700 1 -- 172.21.15.83:0/640129614 shutdown_connections 2024-04-24T08:40:05.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:40:05.995+0000 7f73ff400700 1 --2- 172.21.15.83:0/640129614 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f73f810a720 0x7f73f810ab00 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:40:05.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:40:05.995+0000 7f73ff400700 1 -- 172.21.15.83:0/640129614 >> 172.21.15.83:0/640129614 conn(0x7f73f8100210 msgr2=0x7f73f8102630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:40:05.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:40:05.995+0000 7f73ff400700 1 -- 172.21.15.83:0/640129614 shutdown_connections 2024-04-24T08:40:05.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:40:05.995+0000 7f73ff400700 1 -- 172.21.15.83:0/640129614 wait complete. 2024-04-24T08:40:05.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:40:05.996+0000 7f73ff400700 1 Processor -- start 2024-04-24T08:40:05.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:40:05.996+0000 7f73ff400700 1 -- start start 2024-04-24T08:40:05.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:40:05.996+0000 7f73ff400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f73f810a720 0x7f73f819a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:40:05.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:40:05.996+0000 7f73ff400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f73f819a8c0 0x7f73f819ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:40:05.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:40:05.996+0000 7f73ff400700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f73f819f190 0x7f73f819f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:40:05.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:40:05.997+0000 7f73ff400700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f73f810ec10 con 0x7f73f819a8c0 2024-04-24T08:40:05.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:40:05.997+0000 7f73ff400700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f73f810ea90 con 0x7f73f819f190 2024-04-24T08:40:05.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:40:05.997+0000 7f73ff400700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f73f810ed90 con 0x7f73f810a720 2024-04-24T08:40:05.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:40:05.997+0000 7f73f7e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f73f819a8c0 0x7f73f819ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:40:05.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:40:05.997+0000 7f73f7e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f73f819a8c0 0x7f73f819ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53014/0 (socket says 172.21.15.83:53014) 2024-04-24T08:40:05.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:40:05.997+0000 7f73f7e00700 1 -- 172.21.15.83:0/1453844349 learned_addr learned my addr 172.21.15.83:0/1453844349 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:40:05.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:40:05.997+0000 7f73fca00700 1 --2- 172.21.15.83:0/1453844349 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f73f810a720 0x7f73f819a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:40:05.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:40:05.997+0000 7f73f7e00700 1 -- 172.21.15.83:0/1453844349 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f73f810a720 msgr2=0x7f73f819a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:40:05.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:40:05.997+0000 7f73f7e00700 1 --2- 172.21.15.83:0/1453844349 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f73f810a720 0x7f73f819a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:40:05.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:40:05.997+0000 7f73f7e00700 1 -- 172.21.15.83:0/1453844349 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f73f819f190 msgr2=0x7f73f819f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:40:05.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:40:05.997+0000 7f73f7e00700 1 --2- 172.21.15.83:0/1453844349 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f73f819f190 0x7f73f819f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:40:05.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:40:05.997+0000 7f73f7e00700 1 -- 172.21.15.83:0/1453844349 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73f81a6f00 con 0x7f73f819a8c0 2024-04-24T08:40:05.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:40:05.998+0000 7f73f7e00700 1 --2- 172.21.15.83:0/1453844349 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f73f819a8c0 0x7f73f819ec50 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f73e400b780 tx=0x7f73e400ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:40:06.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:40:05.998+0000 7f73f5600700 1 -- 172.21.15.83:0/1453844349 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e4004230 con 0x7f73f819a8c0 2024-04-24T08:40:06.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:40:05.998+0000 7f73f5600700 1 -- 172.21.15.83:0/1453844349 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f73e40043d0 con 0x7f73f819a8c0 2024-04-24T08:40:06.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:40:05.998+0000 7f73ff400700 1 -- 172.21.15.83:0/1453844349 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73f81a7130 con 0x7f73f819a8c0 2024-04-24T08:40:06.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:40:05.998+0000 7f73f5600700 1 -- 172.21.15.83:0/1453844349 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e4011b60 con 0x7f73f819a8c0 2024-04-24T08:40:06.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:40:05.998+0000 7f73ff400700 1 -- 172.21.15.83:0/1453844349 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73f81a76c0 con 0x7f73f819a8c0 2024-04-24T08:40:06.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:40:05.999+0000 7f73ff400700 1 -- 172.21.15.83:0/1453844349 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73f8064150 con 0x7f73f819a8c0 2024-04-24T08:40:06.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:40:05.999+0000 7f73f5600700 1 -- 172.21.15.83:0/1453844349 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f73e40036a0 con 0x7f73f819a8c0 2024-04-24T08:40:06.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:40:06.000+0000 7f73f5600700 1 --2- 172.21.15.83:0/1453844349 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f73e0039ae0 0x7f73e003bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:40:06.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:40:06.000+0000 7f73f5600700 1 -- 172.21.15.83:0/1453844349 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f73e404c450 con 0x7f73f819a8c0 2024-04-24T08:40:06.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:40:06.000+0000 7f73fca00700 1 --2- 172.21.15.83:0/1453844349 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f73e0039ae0 0x7f73e003bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:40:06.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:40:06.001+0000 7f73fca00700 1 --2- 172.21.15.83:0/1453844349 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f73e0039ae0 0x7f73e003bfa0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f73ec006010 tx=0x7f73ec01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:40:06.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:40:06.006+0000 7f73f5600700 1 -- 172.21.15.83:0/1453844349 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f73e4021e00 con 0x7f73f819a8c0 2024-04-24T08:40:06.053 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:40:05 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:40:05.756+0000 7fbad5600700 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-24T08:40:06.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:40:06.247+0000 7f73ff400700 1 -- 172.21.15.83:0/1453844349 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f73f81a7a70 con 0x7f73f819a8c0 2024-04-24T08:40:06.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:40:06.247+0000 7f73f5600700 1 -- 172.21.15.83:0/1453844349 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+275 (secure 0 0 0) 0x7f73e4018e20 con 0x7f73f819a8c0 2024-04-24T08:40:06.248 INFO:teuthology.orchestra.run.smithi083.stdout:# minimal ceph.conf for bd3e039e-0215-11ef-bc93-c7b262605968 2024-04-24T08:40:06.249 INFO:teuthology.orchestra.run.smithi083.stdout:[global] 2024-04-24T08:40:06.249 INFO:teuthology.orchestra.run.smithi083.stdout: fsid = bd3e039e-0215-11ef-bc93-c7b262605968 2024-04-24T08:40:06.249 INFO:teuthology.orchestra.run.smithi083.stdout: mon_host = [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] 2024-04-24T08:40:06.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:40:06.250+0000 7f73ff400700 1 -- 172.21.15.83:0/1453844349 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f73e0039ae0 msgr2=0x7f73e003bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:40:06.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:40:06.250+0000 7f73ff400700 1 --2- 172.21.15.83:0/1453844349 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f73e0039ae0 0x7f73e003bfa0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f73ec006010 tx=0x7f73ec01a040 comp rx=0 tx=0).stop 2024-04-24T08:40:06.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:40:06.250+0000 7f73ff400700 1 -- 172.21.15.83:0/1453844349 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f73f819a8c0 msgr2=0x7f73f819ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:40:06.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:40:06.250+0000 7f73ff400700 1 --2- 172.21.15.83:0/1453844349 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f73f819a8c0 0x7f73f819ec50 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f73e400b780 tx=0x7f73e400ba90 comp rx=0 tx=0).stop 2024-04-24T08:40:06.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:40:06.250+0000 7f73ff400700 1 -- 172.21.15.83:0/1453844349 shutdown_connections 2024-04-24T08:40:06.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:40:06.250+0000 7f73ff400700 1 --2- 172.21.15.83:0/1453844349 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f73e0039ae0 0x7f73e003bfa0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:40:06.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:40:06.250+0000 7f73ff400700 1 --2- 172.21.15.83:0/1453844349 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f73f819f190 0x7f73f819f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:40:06.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:40:06.250+0000 7f73ff400700 1 --2- 172.21.15.83:0/1453844349 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f73f810a720 0x7f73f819a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:40:06.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:40:06.250+0000 7f73ff400700 1 --2- 172.21.15.83:0/1453844349 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f73f819a8c0 0x7f73f819ec50 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:40:06.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:40:06.250+0000 7f73ff400700 1 -- 172.21.15.83:0/1453844349 >> 172.21.15.83:0/1453844349 conn(0x7f73f8100210 msgr2=0x7f73f8106220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:40:06.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:40:06.251+0000 7f73ff400700 1 -- 172.21.15.83:0/1453844349 shutdown_connections 2024-04-24T08:40:06.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:40:06.251+0000 7f73ff400700 1 -- 172.21.15.83:0/1453844349 wait complete. 2024-04-24T08:40:06.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:06 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:06.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:06 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:06.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:06 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:06.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:06 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:06.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:06 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:06.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:06 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:06.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:06 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:06.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:06 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:06.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:06 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:06.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:06 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:06.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:06 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:06.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:06 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:06.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:06 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T08:40:06.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:06 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T08:40:06.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:06 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:40:06.805 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:06 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/1453844349' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:40:06.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:06 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:06.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:06 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:06.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:06 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:06.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:06 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:06.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:06 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:06.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:06 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:06.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:06 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:06.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:06 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:06.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:06 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:06.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:06 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:06.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:06 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:06.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:06 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:06.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:06 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T08:40:06.957 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:06 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T08:40:06.957 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:06 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:40:06.957 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:06 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/1453844349' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:40:06.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:06 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:06.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:06 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:06.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:06 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:06.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:06 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:06.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:06 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:06.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:06 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:06.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:06 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:06.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:06 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:06.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:06 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:06.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:06 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:06.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:06 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:06.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:06 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:06.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:06 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T08:40:06.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:06 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T08:40:06.959 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:06 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:40:06.959 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:06 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/1453844349' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:40:07.084 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-04-24T08:40:07.084 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-24T08:40:07.084 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/etc/ceph/ceph.conf 2024-04-24T08:40:07.162 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-24T08:40:07.162 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-24T08:40:07.265 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-24T08:40:07.265 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/etc/ceph/ceph.conf 2024-04-24T08:40:07.357 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-24T08:40:07.357 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-24T08:40:07.486 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-04-24T08:40:07.487 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/etc/ceph/ceph.conf 2024-04-24T08:40:07.576 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-04-24T08:40:07.576 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-24T08:40:07.704 INFO:tasks.cephadm:Adding mgr.x on smithi083 2024-04-24T08:40:07.704 INFO:tasks.cephadm:Adding mgr.y on smithi138 2024-04-24T08:40:07.704 INFO:tasks.cephadm:Adding mgr.z on smithi189 2024-04-24T08:40:07.704 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd3e039e-0215-11ef-bc93-c7b262605968 -- ceph orch apply mgr '3;smithi083=x;smithi138=y;smithi189=z' 2024-04-24T08:40:07.751 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:07 smithi083 ceph-mon[27907]: Reconfiguring mon.a (unknown last config time)... 2024-04-24T08:40:07.752 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:07 smithi083 ceph-mon[27907]: Reconfiguring daemon mon.a on smithi083 2024-04-24T08:40:07.752 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:07 smithi083 ceph-mon[27907]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:40:07.820 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:07 smithi189 ceph-mon[31791]: Reconfiguring mon.a (unknown last config time)... 2024-04-24T08:40:07.820 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:07 smithi189 ceph-mon[31791]: Reconfiguring daemon mon.a on smithi083 2024-04-24T08:40:07.820 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:07 smithi189 ceph-mon[31791]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:40:08.169 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T08:40:08.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:07 smithi138 ceph-mon[31441]: Reconfiguring mon.a (unknown last config time)... 2024-04-24T08:40:08.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:07 smithi138 ceph-mon[31441]: Reconfiguring daemon mon.a on smithi083 2024-04-24T08:40:08.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:07 smithi138 ceph-mon[31441]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:40:09.804 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:09 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:09.805 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:09 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:09.805 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:09 smithi138 ceph-mon[31441]: Reconfiguring mon.b (monmap changed)... 2024-04-24T08:40:09.805 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:09 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T08:40:09.805 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:09 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T08:40:09.805 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:09 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:40:09.805 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:09 smithi138 ceph-mon[31441]: Reconfiguring daemon mon.b on smithi138 2024-04-24T08:40:09.805 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:09 smithi138 ceph-mon[31441]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:40:09.855 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:09 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:09.855 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:09 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:09.855 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:09 smithi189 ceph-mon[31791]: Reconfiguring mon.b (monmap changed)... 2024-04-24T08:40:09.855 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:09 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T08:40:09.855 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:09 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T08:40:09.855 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:09 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:40:09.855 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:09 smithi189 ceph-mon[31791]: Reconfiguring daemon mon.b on smithi138 2024-04-24T08:40:09.855 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:09 smithi189 ceph-mon[31791]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:40:10.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:09 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:10.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:09 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:10.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:09 smithi083 ceph-mon[27907]: Reconfiguring mon.b (monmap changed)... 2024-04-24T08:40:10.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:09 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T08:40:10.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:09 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T08:40:10.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:09 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:40:10.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:09 smithi083 ceph-mon[27907]: Reconfiguring daemon mon.b on smithi138 2024-04-24T08:40:10.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:09 smithi083 ceph-mon[27907]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:40:10.103 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:40:10.103+0000 7f3057200700 1 -- 172.21.15.189:0/1549024037 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3058108cb0 msgr2=0x7f30581090f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:40:10.103 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:40:10.103+0000 7f3057200700 1 --2- 172.21.15.189:0/1549024037 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3058108cb0 0x7f30581090f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3048009910 tx=0x7f3048009c20 comp rx=0 tx=0).stop 2024-04-24T08:40:10.104 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:40:10.103+0000 7f3057200700 1 -- 172.21.15.189:0/1549024037 shutdown_connections 2024-04-24T08:40:10.104 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:40:10.103+0000 7f3057200700 1 --2- 172.21.15.189:0/1549024037 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3058108cb0 0x7f30581090f0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:40:10.104 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:40:10.103+0000 7f3057200700 1 -- 172.21.15.189:0/1549024037 >> 172.21.15.189:0/1549024037 conn(0x7f3058077b70 msgr2=0x7f3058079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:40:10.104 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:40:10.103+0000 7f3057200700 1 -- 172.21.15.189:0/1549024037 shutdown_connections 2024-04-24T08:40:10.104 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:40:10.103+0000 7f3057200700 1 -- 172.21.15.189:0/1549024037 wait complete. 2024-04-24T08:40:10.104 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:40:10.104+0000 7f3057200700 1 Processor -- start 2024-04-24T08:40:10.104 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:40:10.104+0000 7f3057200700 1 -- start start 2024-04-24T08:40:10.105 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:40:10.105+0000 7f3057200700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3058108cb0 0x7f305819e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:40:10.105 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:40:10.105+0000 7f3057200700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f305819e760 0x7f30581a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:40:10.106 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:40:10.105+0000 7f3057200700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f305819ebe0 0x7f30581a5d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:40:10.106 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:40:10.105+0000 7f3057200700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f305810e970 con 0x7f305819e760 2024-04-24T08:40:10.106 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:40:10.105+0000 7f3057200700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f305810e7f0 con 0x7f3058108cb0 2024-04-24T08:40:10.106 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:40:10.105+0000 7f3057200700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f305810eaf0 con 0x7f305819ebe0 2024-04-24T08:40:10.106 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:40:10.105+0000 7f304fe00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f305819e760 0x7f30581a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:40:10.107 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:40:10.106+0000 7f3055e00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3058108cb0 0x7f305819e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:40:10.107 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:40:10.106+0000 7f304fe00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f305819e760 0x7f30581a37d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.189:42750/0 (socket says 172.21.15.189:42750) 2024-04-24T08:40:10.107 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:40:10.106+0000 7f304fe00700 1 -- 172.21.15.189:0/2167203675 learned_addr learned my addr 172.21.15.189:0/2167203675 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-04-24T08:40:10.107 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:40:10.106+0000 7f3056800700 1 --2- 172.21.15.189:0/2167203675 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f305819ebe0 0x7f30581a5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:40:10.108 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:40:10.106+0000 7f304fe00700 1 -- 172.21.15.189:0/2167203675 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f305819ebe0 msgr2=0x7f30581a5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:40:10.108 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:40:10.106+0000 7f304fe00700 1 --2- 172.21.15.189:0/2167203675 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f305819ebe0 0x7f30581a5d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:40:10.108 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:40:10.106+0000 7f304fe00700 1 -- 172.21.15.189:0/2167203675 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3058108cb0 msgr2=0x7f305819e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:40:10.108 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:40:10.106+0000 7f304fe00700 1 --2- 172.21.15.189:0/2167203675 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3058108cb0 0x7f305819e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:40:10.108 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:40:10.106+0000 7f304fe00700 1 -- 172.21.15.189:0/2167203675 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30581a6630 con 0x7f305819e760 2024-04-24T08:40:10.108 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:40:10.106+0000 7f3056800700 1 --2- 172.21.15.189:0/2167203675 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f305819ebe0 0x7f30581a5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T08:40:10.109 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:40:10.107+0000 7f304fe00700 1 --2- 172.21.15.189:0/2167203675 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f305819e760 0x7f30581a37d0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f304000b780 tx=0x7f304000ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:40:10.109 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:40:10.107+0000 7f304ea00700 1 -- 172.21.15.189:0/2167203675 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3040016070 con 0x7f305819e760 2024-04-24T08:40:10.109 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:40:10.107+0000 7f304ea00700 1 -- 172.21.15.189:0/2167203675 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f30400036a0 con 0x7f305819e760 2024-04-24T08:40:10.109 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:40:10.107+0000 7f3057200700 1 -- 172.21.15.189:0/2167203675 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30581a6920 con 0x7f305819e760 2024-04-24T08:40:10.109 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:40:10.107+0000 7f304ea00700 1 -- 172.21.15.189:0/2167203675 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30400119c0 con 0x7f305819e760 2024-04-24T08:40:10.109 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:40:10.107+0000 7f3057200700 1 -- 172.21.15.189:0/2167203675 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30581a6f30 con 0x7f305819e760 2024-04-24T08:40:10.110 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:40:10.109+0000 7f3057200700 1 -- 172.21.15.189:0/2167203675 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3058064150 con 0x7f305819e760 2024-04-24T08:40:10.110 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:40:10.109+0000 7f304ea00700 1 -- 172.21.15.189:0/2167203675 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f3040019070 con 0x7f305819e760 2024-04-24T08:40:10.110 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:40:10.109+0000 7f304ea00700 1 --2- 172.21.15.189:0/2167203675 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f303c039ae0 0x7f303c03bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:40:10.110 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:40:10.109+0000 7f304ea00700 1 -- 172.21.15.189:0/2167203675 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f304004b320 con 0x7f305819e760 2024-04-24T08:40:10.111 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:40:10.110+0000 7f3055e00700 1 --2- 172.21.15.189:0/2167203675 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f303c039ae0 0x7f303c03bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:40:10.111 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:40:10.111+0000 7f3055e00700 1 --2- 172.21.15.189:0/2167203675 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f303c039ae0 0x7f303c03bfa0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3048005190 tx=0x7f304801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:40:10.126 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:40:10.126+0000 7f304ea00700 1 -- 172.21.15.189:0/2167203675 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3040010380 con 0x7f305819e760 2024-04-24T08:40:10.373 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:40:10.373+0000 7f3057200700 1 -- 172.21.15.189:0/2167203675 --> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi083=x;smithi138=y;smithi189=z", "target": ["mon-mgr", ""]}) -- 0x7f3058062fc0 con 0x7f303c039ae0 2024-04-24T08:40:10.387 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:40:10.387+0000 7f304ea00700 1 -- 172.21.15.189:0/2167203675 <== mgr.14152 v2:172.21.15.83:6800/2550826560 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f3058062fc0 con 0x7f303c039ae0 2024-04-24T08:40:10.388 INFO:teuthology.orchestra.run.smithi189.stdout:Scheduled mgr update... 2024-04-24T08:40:10.391 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:40:10.390+0000 7f3057200700 1 -- 172.21.15.189:0/2167203675 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f303c039ae0 msgr2=0x7f303c03bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:40:10.391 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:40:10.390+0000 7f3057200700 1 --2- 172.21.15.189:0/2167203675 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f303c039ae0 0x7f303c03bfa0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3048005190 tx=0x7f304801a040 comp rx=0 tx=0).stop 2024-04-24T08:40:10.391 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:40:10.390+0000 7f3057200700 1 -- 172.21.15.189:0/2167203675 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f305819e760 msgr2=0x7f30581a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:40:10.391 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:40:10.390+0000 7f3057200700 1 --2- 172.21.15.189:0/2167203675 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f305819e760 0x7f30581a37d0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f304000b780 tx=0x7f304000ba90 comp rx=0 tx=0).stop 2024-04-24T08:40:10.391 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:40:10.390+0000 7f3057200700 1 -- 172.21.15.189:0/2167203675 shutdown_connections 2024-04-24T08:40:10.391 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:40:10.390+0000 7f3057200700 1 --2- 172.21.15.189:0/2167203675 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f303c039ae0 0x7f303c03bfa0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:40:10.392 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:40:10.390+0000 7f3057200700 1 --2- 172.21.15.189:0/2167203675 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f305819ebe0 0x7f30581a5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:40:10.392 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:40:10.390+0000 7f3057200700 1 --2- 172.21.15.189:0/2167203675 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3058108cb0 0x7f305819e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:40:10.392 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:40:10.390+0000 7f3057200700 1 --2- 172.21.15.189:0/2167203675 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f305819e760 0x7f30581a37d0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:40:10.392 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:40:10.390+0000 7f3057200700 1 -- 172.21.15.189:0/2167203675 >> 172.21.15.189:0/2167203675 conn(0x7f3058077b70 msgr2=0x7f3058078890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:40:10.392 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:40:10.391+0000 7f3057200700 1 -- 172.21.15.189:0/2167203675 shutdown_connections 2024-04-24T08:40:10.392 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:40:10.391+0000 7f3057200700 1 -- 172.21.15.189:0/2167203675 wait complete. 2024-04-24T08:40:11.092 DEBUG:teuthology.orchestra.run.smithi138:mgr.y> sudo journalctl -f -n 0 -u ceph-bd3e039e-0215-11ef-bc93-c7b262605968@mgr.y.service 2024-04-24T08:40:11.100 DEBUG:teuthology.orchestra.run.smithi189:mgr.z> sudo journalctl -f -n 0 -u ceph-bd3e039e-0215-11ef-bc93-c7b262605968@mgr.z.service 2024-04-24T08:40:11.107 INFO:tasks.cephadm:Deploying OSDs... 2024-04-24T08:40:11.107 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-24T08:40:11.108 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/scratch_devs of=/dev/stdout 2024-04-24T08:40:11.166 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-24T08:40:11.167 DEBUG:teuthology.orchestra.run.smithi083:> stat /dev/vg_nvme/lv_1 2024-04-24T08:40:11.224 INFO:teuthology.orchestra.run.smithi083.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-24T08:40:11.224 INFO:teuthology.orchestra.run.smithi083.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T08:40:11.224 INFO:teuthology.orchestra.run.smithi083.stdout:Device: 6h/6d Inode: 631 Links: 1 2024-04-24T08:40:11.224 INFO:teuthology.orchestra.run.smithi083.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T08:40:11.224 INFO:teuthology.orchestra.run.smithi083.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T08:40:11.224 INFO:teuthology.orchestra.run.smithi083.stdout:Access: 2024-04-24 08:38:57.623510421 +0000 2024-04-24T08:40:11.224 INFO:teuthology.orchestra.run.smithi083.stdout:Modify: 2024-04-24 08:33:52.316591934 +0000 2024-04-24T08:40:11.224 INFO:teuthology.orchestra.run.smithi083.stdout:Change: 2024-04-24 08:33:52.316591934 +0000 2024-04-24T08:40:11.224 INFO:teuthology.orchestra.run.smithi083.stdout: Birth: 2024-04-24 08:33:52.316591934 +0000 2024-04-24T08:40:11.225 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-24T08:40:11.308 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-24T08:40:11.308 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-24T08:40:11.308 INFO:teuthology.orchestra.run.smithi083.stderr:512 bytes copied, 0.000396244 s, 1.3 MB/s 2024-04-24T08:40:11.312 DEBUG:teuthology.orchestra.run.smithi083:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-24T08:40:11.366 DEBUG:teuthology.orchestra.run.smithi083:> stat /dev/vg_nvme/lv_2 2024-04-24T08:40:11.419 INFO:teuthology.orchestra.run.smithi083.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-24T08:40:11.419 INFO:teuthology.orchestra.run.smithi083.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T08:40:11.419 INFO:teuthology.orchestra.run.smithi083.stdout:Device: 6h/6d Inode: 650 Links: 1 2024-04-24T08:40:11.419 INFO:teuthology.orchestra.run.smithi083.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T08:40:11.419 INFO:teuthology.orchestra.run.smithi083.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T08:40:11.419 INFO:teuthology.orchestra.run.smithi083.stdout:Access: 2024-04-24 08:38:57.687508899 +0000 2024-04-24T08:40:11.419 INFO:teuthology.orchestra.run.smithi083.stdout:Modify: 2024-04-24 08:33:52.928577282 +0000 2024-04-24T08:40:11.419 INFO:teuthology.orchestra.run.smithi083.stdout:Change: 2024-04-24 08:33:52.928577282 +0000 2024-04-24T08:40:11.419 INFO:teuthology.orchestra.run.smithi083.stdout: Birth: 2024-04-24 08:33:52.928577282 +0000 2024-04-24T08:40:11.420 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-24T08:40:11.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:11 smithi138 ceph-mon[31441]: from='client.14214 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi083=x;smithi138=y;smithi189=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:40:11.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:11 smithi138 ceph-mon[31441]: Saving service mgr spec with placement smithi083=x;smithi138=y;smithi189=z;count:3 2024-04-24T08:40:11.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:11 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:11.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:11 smithi138 ceph-mon[31441]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:40:11.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:11 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:11.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:11 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:11.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:11 smithi138 ceph-mon[31441]: Reconfiguring mon.c (monmap changed)... 2024-04-24T08:40:11.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:11 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T08:40:11.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:11 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T08:40:11.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:11 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:40:11.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:11 smithi138 ceph-mon[31441]: Reconfiguring daemon mon.c on smithi189 2024-04-24T08:40:11.495 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-24T08:40:11.495 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-24T08:40:11.495 INFO:teuthology.orchestra.run.smithi083.stderr:512 bytes copied, 0.000367441 s, 1.4 MB/s 2024-04-24T08:40:11.499 DEBUG:teuthology.orchestra.run.smithi083:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-24T08:40:11.554 DEBUG:teuthology.orchestra.run.smithi083:> stat /dev/vg_nvme/lv_3 2024-04-24T08:40:11.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:11 smithi083 ceph-mon[27907]: from='client.14214 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi083=x;smithi138=y;smithi189=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:40:11.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:11 smithi083 ceph-mon[27907]: Saving service mgr spec with placement smithi083=x;smithi138=y;smithi189=z;count:3 2024-04-24T08:40:11.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:11 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:11.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:11 smithi083 ceph-mon[27907]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:40:11.555 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:11 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:11.555 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:11 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:11.555 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:11 smithi083 ceph-mon[27907]: Reconfiguring mon.c (monmap changed)... 2024-04-24T08:40:11.555 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:11 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T08:40:11.555 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:11 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T08:40:11.555 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:11 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:40:11.555 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:11 smithi083 ceph-mon[27907]: Reconfiguring daemon mon.c on smithi189 2024-04-24T08:40:11.607 INFO:teuthology.orchestra.run.smithi083.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-24T08:40:11.607 INFO:teuthology.orchestra.run.smithi083.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T08:40:11.607 INFO:teuthology.orchestra.run.smithi083.stdout:Device: 6h/6d Inode: 672 Links: 1 2024-04-24T08:40:11.607 INFO:teuthology.orchestra.run.smithi083.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T08:40:11.607 INFO:teuthology.orchestra.run.smithi083.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T08:40:11.607 INFO:teuthology.orchestra.run.smithi083.stdout:Access: 2024-04-24 08:38:57.742507591 +0000 2024-04-24T08:40:11.607 INFO:teuthology.orchestra.run.smithi083.stdout:Modify: 2024-04-24 08:33:53.510563348 +0000 2024-04-24T08:40:11.607 INFO:teuthology.orchestra.run.smithi083.stdout:Change: 2024-04-24 08:33:53.510563348 +0000 2024-04-24T08:40:11.607 INFO:teuthology.orchestra.run.smithi083.stdout: Birth: 2024-04-24 08:33:53.510563348 +0000 2024-04-24T08:40:11.607 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-24T08:40:11.686 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-24T08:40:11.686 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-24T08:40:11.686 INFO:teuthology.orchestra.run.smithi083.stderr:512 bytes copied, 0.000385684 s, 1.3 MB/s 2024-04-24T08:40:11.690 DEBUG:teuthology.orchestra.run.smithi083:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-24T08:40:11.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:11 smithi189 ceph-mon[31791]: from='client.14214 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi083=x;smithi138=y;smithi189=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:40:11.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:11 smithi189 ceph-mon[31791]: Saving service mgr spec with placement smithi083=x;smithi138=y;smithi189=z;count:3 2024-04-24T08:40:11.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:11 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:11.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:11 smithi189 ceph-mon[31791]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:40:11.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:11 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:11.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:11 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:11.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:11 smithi189 ceph-mon[31791]: Reconfiguring mon.c (monmap changed)... 2024-04-24T08:40:11.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:11 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T08:40:11.709 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:11 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T08:40:11.709 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:11 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:40:11.709 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:11 smithi189 ceph-mon[31791]: Reconfiguring daemon mon.c on smithi189 2024-04-24T08:40:11.744 DEBUG:teuthology.orchestra.run.smithi083:> stat /dev/vg_nvme/lv_4 2024-04-24T08:40:11.793 INFO:teuthology.orchestra.run.smithi083.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-24T08:40:11.793 INFO:teuthology.orchestra.run.smithi083.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T08:40:11.793 INFO:teuthology.orchestra.run.smithi083.stdout:Device: 6h/6d Inode: 690 Links: 1 2024-04-24T08:40:11.793 INFO:teuthology.orchestra.run.smithi083.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T08:40:11.793 INFO:teuthology.orchestra.run.smithi083.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T08:40:11.793 INFO:teuthology.orchestra.run.smithi083.stdout:Access: 2024-04-24 08:38:57.799506236 +0000 2024-04-24T08:40:11.793 INFO:teuthology.orchestra.run.smithi083.stdout:Modify: 2024-04-24 08:33:54.135548385 +0000 2024-04-24T08:40:11.793 INFO:teuthology.orchestra.run.smithi083.stdout:Change: 2024-04-24 08:33:54.135548385 +0000 2024-04-24T08:40:11.793 INFO:teuthology.orchestra.run.smithi083.stdout: Birth: 2024-04-24 08:33:54.135548385 +0000 2024-04-24T08:40:11.794 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-24T08:40:11.869 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-24T08:40:11.869 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-24T08:40:11.870 INFO:teuthology.orchestra.run.smithi083.stderr:512 bytes copied, 0.000363648 s, 1.4 MB/s 2024-04-24T08:40:11.873 DEBUG:teuthology.orchestra.run.smithi083:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-24T08:40:11.930 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-24T08:40:11.931 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/scratch_devs of=/dev/stdout 2024-04-24T08:40:11.989 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-24T08:40:11.989 DEBUG:teuthology.orchestra.run.smithi138:> stat /dev/vg_nvme/lv_1 2024-04-24T08:40:12.041 INFO:teuthology.orchestra.run.smithi138.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-24T08:40:12.041 INFO:teuthology.orchestra.run.smithi138.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T08:40:12.041 INFO:teuthology.orchestra.run.smithi138.stdout:Device: 6h/6d Inode: 634 Links: 1 2024-04-24T08:40:12.041 INFO:teuthology.orchestra.run.smithi138.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T08:40:12.041 INFO:teuthology.orchestra.run.smithi138.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T08:40:12.042 INFO:teuthology.orchestra.run.smithi138.stdout:Access: 2024-04-24 08:39:55.756817042 +0000 2024-04-24T08:40:12.042 INFO:teuthology.orchestra.run.smithi138.stdout:Modify: 2024-04-24 08:33:44.339331802 +0000 2024-04-24T08:40:12.042 INFO:teuthology.orchestra.run.smithi138.stdout:Change: 2024-04-24 08:33:44.339331802 +0000 2024-04-24T08:40:12.042 INFO:teuthology.orchestra.run.smithi138.stdout: Birth: 2024-04-24 08:33:44.339331802 +0000 2024-04-24T08:40:12.042 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-24T08:40:12.120 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records in 2024-04-24T08:40:12.121 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records out 2024-04-24T08:40:12.121 INFO:teuthology.orchestra.run.smithi138.stderr:512 bytes copied, 0.000363893 s, 1.4 MB/s 2024-04-24T08:40:12.124 DEBUG:teuthology.orchestra.run.smithi138:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-24T08:40:12.177 DEBUG:teuthology.orchestra.run.smithi138:> stat /dev/vg_nvme/lv_2 2024-04-24T08:40:12.225 INFO:teuthology.orchestra.run.smithi138.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-24T08:40:12.225 INFO:teuthology.orchestra.run.smithi138.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T08:40:12.226 INFO:teuthology.orchestra.run.smithi138.stdout:Device: 6h/6d Inode: 652 Links: 1 2024-04-24T08:40:12.226 INFO:teuthology.orchestra.run.smithi138.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T08:40:12.226 INFO:teuthology.orchestra.run.smithi138.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T08:40:12.226 INFO:teuthology.orchestra.run.smithi138.stdout:Access: 2024-04-24 08:39:55.757817021 +0000 2024-04-24T08:40:12.226 INFO:teuthology.orchestra.run.smithi138.stdout:Modify: 2024-04-24 08:33:44.954318947 +0000 2024-04-24T08:40:12.226 INFO:teuthology.orchestra.run.smithi138.stdout:Change: 2024-04-24 08:33:44.954318947 +0000 2024-04-24T08:40:12.226 INFO:teuthology.orchestra.run.smithi138.stdout: Birth: 2024-04-24 08:33:44.954318947 +0000 2024-04-24T08:40:12.226 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-24T08:40:12.301 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records in 2024-04-24T08:40:12.302 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records out 2024-04-24T08:40:12.302 INFO:teuthology.orchestra.run.smithi138.stderr:512 bytes copied, 0.000345499 s, 1.5 MB/s 2024-04-24T08:40:12.306 DEBUG:teuthology.orchestra.run.smithi138:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-24T08:40:12.359 DEBUG:teuthology.orchestra.run.smithi138:> stat /dev/vg_nvme/lv_3 2024-04-24T08:40:12.405 INFO:teuthology.orchestra.run.smithi138.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-24T08:40:12.405 INFO:teuthology.orchestra.run.smithi138.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T08:40:12.405 INFO:teuthology.orchestra.run.smithi138.stdout:Device: 6h/6d Inode: 669 Links: 1 2024-04-24T08:40:12.405 INFO:teuthology.orchestra.run.smithi138.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T08:40:12.406 INFO:teuthology.orchestra.run.smithi138.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T08:40:12.406 INFO:teuthology.orchestra.run.smithi138.stdout:Access: 2024-04-24 08:39:55.758817000 +0000 2024-04-24T08:40:12.406 INFO:teuthology.orchestra.run.smithi138.stdout:Modify: 2024-04-24 08:33:45.541306677 +0000 2024-04-24T08:40:12.406 INFO:teuthology.orchestra.run.smithi138.stdout:Change: 2024-04-24 08:33:45.541306677 +0000 2024-04-24T08:40:12.406 INFO:teuthology.orchestra.run.smithi138.stdout: Birth: 2024-04-24 08:33:45.541306677 +0000 2024-04-24T08:40:12.406 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-24T08:40:12.481 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records in 2024-04-24T08:40:12.481 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records out 2024-04-24T08:40:12.481 INFO:teuthology.orchestra.run.smithi138.stderr:512 bytes copied, 0.00106505 s, 481 kB/s 2024-04-24T08:40:12.485 DEBUG:teuthology.orchestra.run.smithi138:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-24T08:40:12.539 DEBUG:teuthology.orchestra.run.smithi138:> stat /dev/vg_nvme/lv_4 2024-04-24T08:40:12.587 INFO:teuthology.orchestra.run.smithi138.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-24T08:40:12.587 INFO:teuthology.orchestra.run.smithi138.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T08:40:12.588 INFO:teuthology.orchestra.run.smithi138.stdout:Device: 6h/6d Inode: 688 Links: 1 2024-04-24T08:40:12.588 INFO:teuthology.orchestra.run.smithi138.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T08:40:12.588 INFO:teuthology.orchestra.run.smithi138.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T08:40:12.588 INFO:teuthology.orchestra.run.smithi138.stdout:Access: 2024-04-24 08:39:55.758817000 +0000 2024-04-24T08:40:12.588 INFO:teuthology.orchestra.run.smithi138.stdout:Modify: 2024-04-24 08:33:46.169293551 +0000 2024-04-24T08:40:12.588 INFO:teuthology.orchestra.run.smithi138.stdout:Change: 2024-04-24 08:33:46.169293551 +0000 2024-04-24T08:40:12.588 INFO:teuthology.orchestra.run.smithi138.stdout: Birth: 2024-04-24 08:33:46.169293551 +0000 2024-04-24T08:40:12.588 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-24T08:40:12.666 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records in 2024-04-24T08:40:12.666 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records out 2024-04-24T08:40:12.666 INFO:teuthology.orchestra.run.smithi138.stderr:512 bytes copied, 0.000417067 s, 1.2 MB/s 2024-04-24T08:40:12.670 DEBUG:teuthology.orchestra.run.smithi138:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-24T08:40:12.725 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-04-24T08:40:12.725 DEBUG:teuthology.orchestra.run.smithi189:> dd if=/scratch_devs of=/dev/stdout 2024-04-24T08:40:12.772 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-24T08:40:12.772 DEBUG:teuthology.orchestra.run.smithi189:> stat /dev/vg_nvme/lv_1 2024-04-24T08:40:12.855 INFO:teuthology.orchestra.run.smithi189.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-24T08:40:12.856 INFO:teuthology.orchestra.run.smithi189.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T08:40:12.856 INFO:teuthology.orchestra.run.smithi189.stdout:Device: 6h/6d Inode: 633 Links: 1 2024-04-24T08:40:12.856 INFO:teuthology.orchestra.run.smithi189.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T08:40:12.856 INFO:teuthology.orchestra.run.smithi189.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T08:40:12.856 INFO:teuthology.orchestra.run.smithi189.stdout:Access: 2024-04-24 08:39:44.142314318 +0000 2024-04-24T08:40:12.856 INFO:teuthology.orchestra.run.smithi189.stdout:Modify: 2024-04-24 08:33:45.088620699 +0000 2024-04-24T08:40:12.856 INFO:teuthology.orchestra.run.smithi189.stdout:Change: 2024-04-24 08:33:45.088620699 +0000 2024-04-24T08:40:12.856 INFO:teuthology.orchestra.run.smithi189.stdout: Birth: 2024-04-24 08:33:45.088620699 +0000 2024-04-24T08:40:12.856 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-24T08:40:12.969 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records in 2024-04-24T08:40:12.969 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records out 2024-04-24T08:40:12.970 INFO:teuthology.orchestra.run.smithi189.stderr:512 bytes copied, 0.000374389 s, 1.4 MB/s 2024-04-24T08:40:12.973 DEBUG:teuthology.orchestra.run.smithi189:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-24T08:40:13.022 DEBUG:teuthology.orchestra.run.smithi189:> stat /dev/vg_nvme/lv_2 2024-04-24T08:40:13.069 INFO:teuthology.orchestra.run.smithi189.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-24T08:40:13.069 INFO:teuthology.orchestra.run.smithi189.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T08:40:13.069 INFO:teuthology.orchestra.run.smithi189.stdout:Device: 6h/6d Inode: 650 Links: 1 2024-04-24T08:40:13.069 INFO:teuthology.orchestra.run.smithi189.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T08:40:13.069 INFO:teuthology.orchestra.run.smithi189.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T08:40:13.069 INFO:teuthology.orchestra.run.smithi189.stdout:Access: 2024-04-24 08:39:44.143314298 +0000 2024-04-24T08:40:13.069 INFO:teuthology.orchestra.run.smithi189.stdout:Modify: 2024-04-24 08:33:45.683609142 +0000 2024-04-24T08:40:13.069 INFO:teuthology.orchestra.run.smithi189.stdout:Change: 2024-04-24 08:33:45.683609142 +0000 2024-04-24T08:40:13.070 INFO:teuthology.orchestra.run.smithi189.stdout: Birth: 2024-04-24 08:33:45.683609142 +0000 2024-04-24T08:40:13.070 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-24T08:40:13.143 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records in 2024-04-24T08:40:13.143 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records out 2024-04-24T08:40:13.143 INFO:teuthology.orchestra.run.smithi189.stderr:512 bytes copied, 0.000314824 s, 1.6 MB/s 2024-04-24T08:40:13.147 DEBUG:teuthology.orchestra.run.smithi189:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-24T08:40:13.197 DEBUG:teuthology.orchestra.run.smithi189:> stat /dev/vg_nvme/lv_3 2024-04-24T08:40:13.241 INFO:teuthology.orchestra.run.smithi189.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-24T08:40:13.241 INFO:teuthology.orchestra.run.smithi189.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T08:40:13.241 INFO:teuthology.orchestra.run.smithi189.stdout:Device: 6h/6d Inode: 670 Links: 1 2024-04-24T08:40:13.241 INFO:teuthology.orchestra.run.smithi189.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T08:40:13.241 INFO:teuthology.orchestra.run.smithi189.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T08:40:13.241 INFO:teuthology.orchestra.run.smithi189.stdout:Access: 2024-04-24 08:39:44.144314279 +0000 2024-04-24T08:40:13.241 INFO:teuthology.orchestra.run.smithi189.stdout:Modify: 2024-04-24 08:33:46.283597486 +0000 2024-04-24T08:40:13.241 INFO:teuthology.orchestra.run.smithi189.stdout:Change: 2024-04-24 08:33:46.283597486 +0000 2024-04-24T08:40:13.241 INFO:teuthology.orchestra.run.smithi189.stdout: Birth: 2024-04-24 08:33:46.283597486 +0000 2024-04-24T08:40:13.241 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-24T08:40:13.347 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records in 2024-04-24T08:40:13.347 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records out 2024-04-24T08:40:13.347 INFO:teuthology.orchestra.run.smithi189.stderr:512 bytes copied, 0.000262643 s, 1.9 MB/s 2024-04-24T08:40:13.350 DEBUG:teuthology.orchestra.run.smithi189:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-24T08:40:13.391 DEBUG:teuthology.orchestra.run.smithi189:> stat /dev/vg_nvme/lv_4 2024-04-24T08:40:13.473 INFO:teuthology.orchestra.run.smithi189.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-24T08:40:13.473 INFO:teuthology.orchestra.run.smithi189.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T08:40:13.473 INFO:teuthology.orchestra.run.smithi189.stdout:Device: 6h/6d Inode: 690 Links: 1 2024-04-24T08:40:13.473 INFO:teuthology.orchestra.run.smithi189.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T08:40:13.473 INFO:teuthology.orchestra.run.smithi189.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T08:40:13.473 INFO:teuthology.orchestra.run.smithi189.stdout:Access: 2024-04-24 08:39:44.145314260 +0000 2024-04-24T08:40:13.473 INFO:teuthology.orchestra.run.smithi189.stdout:Modify: 2024-04-24 08:33:46.874586006 +0000 2024-04-24T08:40:13.473 INFO:teuthology.orchestra.run.smithi189.stdout:Change: 2024-04-24 08:33:46.874586006 +0000 2024-04-24T08:40:13.473 INFO:teuthology.orchestra.run.smithi189.stdout: Birth: 2024-04-24 08:33:46.874586006 +0000 2024-04-24T08:40:13.474 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-24T08:40:13.570 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records in 2024-04-24T08:40:13.570 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records out 2024-04-24T08:40:13.570 INFO:teuthology.orchestra.run.smithi189.stderr:512 bytes copied, 0.000281401 s, 1.8 MB/s 2024-04-24T08:40:13.573 DEBUG:teuthology.orchestra.run.smithi189:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-24T08:40:13.614 INFO:tasks.cephadm:Deploying osd.0 on smithi083 with /dev/vg_nvme/lv_4... 2024-04-24T08:40:13.614 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd3e039e-0215-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-24T08:40:13.706 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:13 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:13.706 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:13 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:13.706 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:13 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:40:13.706 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:13 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:40:13.706 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:13 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:40:13.706 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:13 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:13.706 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:13 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T08:40:13.706 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:13 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-24T08:40:13.707 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:13 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T08:40:13.707 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:13 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:40:13.707 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:13 smithi138 ceph-mon[31441]: Deploying daemon mgr.z on smithi189 2024-04-24T08:40:13.707 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:13 smithi138 ceph-mon[31441]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:40:13.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:13 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:13.833 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:13 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:13.834 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:13 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:40:13.834 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:13 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:40:13.834 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:13 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:40:13.834 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:13 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:13.834 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:13 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T08:40:13.834 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:13 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-24T08:40:13.834 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:13 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T08:40:13.834 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:13 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:40:13.834 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:13 smithi189 ceph-mon[31791]: Deploying daemon mgr.z on smithi189 2024-04-24T08:40:13.834 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:13 smithi189 ceph-mon[31791]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:40:14.039 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T08:40:14.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:13 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:14.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:13 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:14.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:13 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:40:14.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:13 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:40:14.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:13 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:40:14.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:13 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:14.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:13 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T08:40:14.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:13 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-24T08:40:14.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:13 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T08:40:14.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:13 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:40:14.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:13 smithi083 ceph-mon[27907]: Deploying daemon mgr.z on smithi189 2024-04-24T08:40:14.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:13 smithi083 ceph-mon[27907]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:40:15.708 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 24 08:40:15 smithi189 systemd[1]: Starting Ceph mgr.z for bd3e039e-0215-11ef-bc93-c7b262605968... 2024-04-24T08:40:16.001 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:15 smithi189 ceph-mon[31791]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:40:16.001 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 24 08:40:15 smithi189 podman[33662]: 2024-04-24 08:40:15.662387626 +0000 UTC m=+0.036324449 image pull b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644 2024-04-24T08:40:16.001 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 24 08:40:15 smithi189 podman[33662]: 2024-04-24 08:40:15.772941345 +0000 UTC m=+0.146878200 container create 40d25381f5bdc0fcd785d74294ca01f0be90dbe445041a6f67099ce178449e6c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-z, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T08:40:16.002 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 24 08:40:16 smithi189 podman[33662]: 2024-04-24 08:40:16.001075289 +0000 UTC m=+0.375012151 container init 40d25381f5bdc0fcd785d74294ca01f0be90dbe445041a6f67099ce178449e6c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-z, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=rishabh-main-apr17-a654945-1f81fb3, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-24T08:40:16.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:15 smithi083 ceph-mon[27907]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:40:16.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:15 smithi138 ceph-mon[31441]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:40:16.458 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 24 08:40:16 smithi189 podman[33662]: 2024-04-24 08:40:16.011563793 +0000 UTC m=+0.385500630 container start 40d25381f5bdc0fcd785d74294ca01f0be90dbe445041a6f67099ce178449e6c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-z, RELEASE=rishabh-main-apr17-a654945-1f81fb3, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T08:40:16.458 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 24 08:40:16 smithi189 ceph-mgr[33681]: -- 172.21.15.189:0/2770520531 <== mon.1 v2:172.21.15.189:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5608b70665a0 con 0x5608b703b000 2024-04-24T08:40:16.458 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 24 08:40:16 smithi189 bash[33662]: 40d25381f5bdc0fcd785d74294ca01f0be90dbe445041a6f67099ce178449e6c 2024-04-24T08:40:16.458 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 24 08:40:16 smithi189 systemd[1]: Started Ceph mgr.z for bd3e039e-0215-11ef-bc93-c7b262605968. 2024-04-24T08:40:16.458 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 24 08:40:16 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-z[33677]: 2024-04-24T08:40:16.249+0000 7f17ebc59200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-24T08:40:16.957 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 24 08:40:16 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-z[33677]: 2024-04-24T08:40:16.504+0000 7f17ebc59200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-24T08:40:16.958 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 24 08:40:16 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-z[33677]: 2024-04-24T08:40:16.630+0000 7f17ebc59200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-24T08:40:16.982 INFO:teuthology.orchestra.run.smithi083.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-24T08:40:16.982 INFO:teuthology.orchestra.run.smithi083.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-24T08:40:16.982 INFO:teuthology.orchestra.run.smithi083.stderr: stderr: 10+0 records in 2024-04-24T08:40:16.982 INFO:teuthology.orchestra.run.smithi083.stderr:10+0 records out 2024-04-24T08:40:16.982 INFO:teuthology.orchestra.run.smithi083.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.036166 s, 290 MB/s 2024-04-24T08:40:16.982 INFO:teuthology.orchestra.run.smithi083.stderr:--> Zapping successful for: 2024-04-24T08:40:17.457 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 24 08:40:17 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-z[33677]: 2024-04-24T08:40:17.023+0000 7f17ebc59200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-24T08:40:17.458 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 24 08:40:17 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-z[33677]: 2024-04-24T08:40:17.133+0000 7f17ebc59200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-24T08:40:17.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:17 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:17.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:17 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:17.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:17 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:17.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:17 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T08:40:17.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:17 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-24T08:40:17.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:17 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T08:40:17.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:17 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:40:17.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:17 smithi189 ceph-mon[31791]: Deploying daemon mgr.y on smithi138 2024-04-24T08:40:17.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:17 smithi189 ceph-mon[31791]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:40:17.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:17 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:17.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:17 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:17.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:17 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:17.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:17 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T08:40:17.555 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:17 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-24T08:40:17.555 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:17 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T08:40:17.555 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:17 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:40:17.555 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:17 smithi083 ceph-mon[27907]: Deploying daemon mgr.y on smithi138 2024-04-24T08:40:17.555 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:17 smithi083 ceph-mon[27907]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:40:17.589 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:17 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:17.589 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:17 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:17.589 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:17 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:17.589 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:17 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T08:40:17.589 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:17 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-24T08:40:17.589 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:17 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T08:40:17.589 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:17 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:40:17.589 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:17 smithi138 ceph-mon[31441]: Deploying daemon mgr.y on smithi138 2024-04-24T08:40:17.589 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:17 smithi138 ceph-mon[31441]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:40:17.876 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd3e039e-0215-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi083:vg_nvme/lv_4 2024-04-24T08:40:18.301 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T08:40:18.957 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 24 08:40:18 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-z[33677]: 2024-04-24T08:40:18.682+0000 7f17ebc59200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-24T08:40:18.957 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 24 08:40:18 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-z[33677]: 2024-04-24T08:40:18.907+0000 7f17ebc59200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-24T08:40:19.457 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 24 08:40:19 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-z[33677]: 2024-04-24T08:40:19.118+0000 7f17ebc59200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-24T08:40:19.689 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Apr 24 08:40:19 smithi138 systemd[1]: Starting Ceph mgr.y for bd3e039e-0215-11ef-bc93-c7b262605968... 2024-04-24T08:40:19.749 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 24 08:40:19 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-z[33677]: 2024-04-24T08:40:19.491+0000 7f17ebc59200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-24T08:40:19.749 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 24 08:40:19 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-z[33677]: 2024-04-24T08:40:19.610+0000 7f17ebc59200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-24T08:40:19.953 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:19 smithi083 ceph-mon[27907]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:40:19.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:19 smithi138 ceph-mon[31441]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:40:19.956 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Apr 24 08:40:19 smithi138 podman[32795]: 2024-04-24 08:40:19.59443441 +0000 UTC m=+0.033758452 image pull b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644 2024-04-24T08:40:19.956 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Apr 24 08:40:19 smithi138 podman[32795]: 2024-04-24 08:40:19.715812156 +0000 UTC m=+0.155136222 container create 19f774befef758b542cb0f7f8f0d8b0a74c3520a2c0509a9341cf45580dd10ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-y, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-24T08:40:20.112 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:19 smithi189 ceph-mon[31791]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:40:20.112 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 24 08:40:19 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-z[33677]: 2024-04-24T08:40:19.862+0000 7f17ebc59200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-24T08:40:20.455 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Apr 24 08:40:20 smithi138 podman[32795]: 2024-04-24 08:40:20.003849913 +0000 UTC m=+0.443173986 container init 19f774befef758b542cb0f7f8f0d8b0a74c3520a2c0509a9341cf45580dd10ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-y, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=rishabh-main-apr17-a654945-1f81fb3, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD) 2024-04-24T08:40:20.456 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Apr 24 08:40:20 smithi138 podman[32795]: 2024-04-24 08:40:20.014357361 +0000 UTC m=+0.453681460 container start 19f774befef758b542cb0f7f8f0d8b0a74c3520a2c0509a9341cf45580dd10ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-y, org.label-schema.build-date=20240424, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-04-24T08:40:20.456 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Apr 24 08:40:20 smithi138 ceph-mgr[32814]: -- 172.21.15.138:0/721676622 <== mon.2 v2:172.21.15.138:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55698fa285a0 con 0x55698f9fcc00 2024-04-24T08:40:20.456 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Apr 24 08:40:20 smithi138 bash[32795]: 19f774befef758b542cb0f7f8f0d8b0a74c3520a2c0509a9341cf45580dd10ea 2024-04-24T08:40:20.456 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Apr 24 08:40:20 smithi138 systemd[1]: Started Ceph mgr.y for bd3e039e-0215-11ef-bc93-c7b262605968. 2024-04-24T08:40:20.456 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Apr 24 08:40:20 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-y[32810]: 2024-04-24T08:40:20.249+0000 7f453b80f200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-24T08:40:20.457 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 24 08:40:20 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-z[33677]: 2024-04-24T08:40:20.111+0000 7f17ebc59200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-24T08:40:20.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:40:20.492+0000 7f4bdf200700 1 -- 172.21.15.83:0/106994757 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4be0105e90 msgr2=0x7f4be01062f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:40:20.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:40:20.492+0000 7f4bdf200700 1 --2- 172.21.15.83:0/106994757 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4be0105e90 0x7f4be01062f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc0099c0 tx=0x7f4bcc009cd0 comp rx=0 tx=0).stop 2024-04-24T08:40:20.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:40:20.493+0000 7f4bdf200700 1 -- 172.21.15.83:0/106994757 shutdown_connections 2024-04-24T08:40:20.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:40:20.493+0000 7f4bdf200700 1 --2- 172.21.15.83:0/106994757 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4be0106830 0x7f4be010d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:40:20.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:40:20.493+0000 7f4bdf200700 1 --2- 172.21.15.83:0/106994757 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4be0104c90 0x7f4be01050b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:40:20.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:40:20.493+0000 7f4bdf200700 1 --2- 172.21.15.83:0/106994757 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4be0105e90 0x7f4be01062f0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:40:20.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:40:20.493+0000 7f4bdf200700 1 -- 172.21.15.83:0/106994757 >> 172.21.15.83:0/106994757 conn(0x7f4be0100230 msgr2=0x7f4be0102670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:40:20.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:40:20.493+0000 7f4bdf200700 1 -- 172.21.15.83:0/106994757 shutdown_connections 2024-04-24T08:40:20.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:40:20.493+0000 7f4bdf200700 1 -- 172.21.15.83:0/106994757 wait complete. 2024-04-24T08:40:20.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:40:20.494+0000 7f4bdf200700 1 Processor -- start 2024-04-24T08:40:20.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:40:20.494+0000 7f4bdf200700 1 -- start start 2024-04-24T08:40:20.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:40:20.494+0000 7f4bdf200700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4be0104c90 0x7f4be019ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:40:20.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:40:20.494+0000 7f4bdf200700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4be0105e90 0x7f4be019b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:40:20.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:40:20.494+0000 7f4bdf200700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4be0106830 0x7f4be01a23c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:40:20.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:40:20.494+0000 7f4bdf200700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4be0110780 con 0x7f4be0104c90 2024-04-24T08:40:20.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:40:20.494+0000 7f4bdf200700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4be0110600 con 0x7f4be0106830 2024-04-24T08:40:20.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:40:20.494+0000 7f4bdf200700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4be0110900 con 0x7f4be0105e90 2024-04-24T08:40:20.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:40:20.494+0000 7f4bdde00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4be0104c90 0x7f4be019ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:40:20.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:40:20.494+0000 7f4bdde00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4be0104c90 0x7f4be019ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38588/0 (socket says 172.21.15.83:38588) 2024-04-24T08:40:20.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:40:20.494+0000 7f4bdde00700 1 -- 172.21.15.83:0/28551012 learned_addr learned my addr 172.21.15.83:0/28551012 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:40:20.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:40:20.495+0000 7f4bde800700 1 --2- 172.21.15.83:0/28551012 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4be0106830 0x7f4be01a23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:40:20.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:40:20.495+0000 7f4bdd400700 1 --2- 172.21.15.83:0/28551012 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4be0105e90 0x7f4be019b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:40:20.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:40:20.495+0000 7f4bdde00700 1 -- 172.21.15.83:0/28551012 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4be0105e90 msgr2=0x7f4be019b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:40:20.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:40:20.495+0000 7f4bdde00700 1 --2- 172.21.15.83:0/28551012 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4be0105e90 0x7f4be019b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:40:20.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:40:20.495+0000 7f4bdde00700 1 -- 172.21.15.83:0/28551012 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4be0106830 msgr2=0x7f4be01a23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:40:20.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:40:20.495+0000 7f4bdde00700 1 --2- 172.21.15.83:0/28551012 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4be0106830 0x7f4be01a23c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:40:20.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:40:20.495+0000 7f4bdde00700 1 -- 172.21.15.83:0/28551012 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4be01a2a60 con 0x7f4be0104c90 2024-04-24T08:40:20.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:40:20.495+0000 7f4bde800700 1 --2- 172.21.15.83:0/28551012 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4be0106830 0x7f4be01a23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T08:40:20.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:40:20.495+0000 7f4bdde00700 1 --2- 172.21.15.83:0/28551012 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4be0104c90 0x7f4be019ae50 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f4bd000c9a0 tx=0x7f4bd000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:40:20.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:40:20.496+0000 7f4bd6a00700 1 -- 172.21.15.83:0/28551012 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bd0004220 con 0x7f4be0104c90 2024-04-24T08:40:20.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:40:20.496+0000 7f4bd6a00700 1 -- 172.21.15.83:0/28551012 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4bd0007d10 con 0x7f4be0104c90 2024-04-24T08:40:20.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:40:20.496+0000 7f4bd6a00700 1 -- 172.21.15.83:0/28551012 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bd0003d00 con 0x7f4be0104c90 2024-04-24T08:40:20.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:40:20.496+0000 7f4bdf200700 1 -- 172.21.15.83:0/28551012 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4be01a2d50 con 0x7f4be0104c90 2024-04-24T08:40:20.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:40:20.496+0000 7f4bdf200700 1 -- 172.21.15.83:0/28551012 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4be01a31f0 con 0x7f4be0104c90 2024-04-24T08:40:20.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:40:20.497+0000 7f4bdf200700 1 -- 172.21.15.83:0/28551012 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4be00697e0 con 0x7f4be0104c90 2024-04-24T08:40:20.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:40:20.497+0000 7f4bd6a00700 1 -- 172.21.15.83:0/28551012 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f4bd0007860 con 0x7f4be0104c90 2024-04-24T08:40:20.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:40:20.498+0000 7f4bd6a00700 1 --2- 172.21.15.83:0/28551012 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f4bc4039890 0x7f4bc403bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:40:20.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:40:20.498+0000 7f4bd6a00700 1 -- 172.21.15.83:0/28551012 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f4bd004c550 con 0x7f4be0104c90 2024-04-24T08:40:20.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:40:20.498+0000 7f4bdd400700 1 --2- 172.21.15.83:0/28551012 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f4bc4039890 0x7f4bc403bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:40:20.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:40:20.498+0000 7f4bdd400700 1 --2- 172.21.15.83:0/28551012 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f4bc4039890 0x7f4bc403bd50 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc000c00 tx=0x7f4bcc005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:40:20.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:40:20.503+0000 7f4bd6a00700 1 -- 172.21.15.83:0/28551012 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4bd0014030 con 0x7f4be0104c90 2024-04-24T08:40:20.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:40:20.745+0000 7f4bdf200700 1 -- 172.21.15.83:0/28551012 --> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f4be0062f30 con 0x7f4bc4039890 2024-04-24T08:40:20.955 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Apr 24 08:40:20 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-y[32810]: 2024-04-24T08:40:20.504+0000 7f453b80f200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-24T08:40:20.955 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Apr 24 08:40:20 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-y[32810]: 2024-04-24T08:40:20.637+0000 7f453b80f200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-24T08:40:21.207 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 24 08:40:20 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-z[33677]: 2024-04-24T08:40:20.911+0000 7f17ebc59200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-24T08:40:21.455 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Apr 24 08:40:21 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-y[32810]: 2024-04-24T08:40:21.052+0000 7f453b80f200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-24T08:40:21.455 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Apr 24 08:40:21 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-y[32810]: 2024-04-24T08:40:21.165+0000 7f453b80f200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-24T08:40:21.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:21 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:21.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:21 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:21.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:21 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:21.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:21 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:21.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:21 smithi138 ceph-mon[31441]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:40:21.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:21 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:40:21.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:21 smithi138 ceph-mon[31441]: from='client.14229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:40:21.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:21 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:21.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:21 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T08:40:21.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:21 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T08:40:21.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:21 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:40:21.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:21 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:21.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:21 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:21.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:21 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:21.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:21 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:21.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:21 smithi083 ceph-mon[27907]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:40:21.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:21 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:40:21.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:21 smithi083 ceph-mon[27907]: from='client.14229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:40:21.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:21 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:21.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:21 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T08:40:21.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:21 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T08:40:21.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:21 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:40:21.615 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 24 08:40:21 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-z[33677]: 2024-04-24T08:40:21.479+0000 7f17ebc59200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-24T08:40:21.615 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 24 08:40:21 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-z[33677]: 2024-04-24T08:40:21.614+0000 7f17ebc59200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-24T08:40:21.615 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:21 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:21.616 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:21 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:21.616 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:21 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:21.616 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:21 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:21.616 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:21 smithi189 ceph-mon[31791]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:40:21.616 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:21 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:40:21.616 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:21 smithi189 ceph-mon[31791]: from='client.14229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:40:21.616 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:21 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:21.616 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:21 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T08:40:21.616 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:21 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T08:40:21.616 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:21 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:40:22.207 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 24 08:40:21 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-z[33677]: 2024-04-24T08:40:21.890+0000 7f17ebc59200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-24T08:40:22.957 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 24 08:40:22 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-z[33677]: 2024-04-24T08:40:22.565+0000 7f17ebc59200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-24T08:40:22.958 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 24 08:40:22 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-z[33677]: 2024-04-24T08:40:22.686+0000 7f17ebc59200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-24T08:40:23.205 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Apr 24 08:40:22 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-y[32810]: 2024-04-24T08:40:22.796+0000 7f453b80f200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-24T08:40:23.205 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Apr 24 08:40:23 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-y[32810]: 2024-04-24T08:40:23.030+0000 7f453b80f200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-24T08:40:23.303 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 24 08:40:23 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-z[33677]: 2024-04-24T08:40:22.999+0000 7f17ebc59200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-24T08:40:23.303 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 24 08:40:23 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-z[33677]: 2024-04-24T08:40:23.177+0000 7f17ebc59200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-24T08:40:23.303 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 24 08:40:23 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-z[33677]: 2024-04-24T08:40:23.301+0000 7f17ebc59200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-24T08:40:23.667 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Apr 24 08:40:23 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-y[32810]: 2024-04-24T08:40:23.254+0000 7f453b80f200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-24T08:40:23.667 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:23 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:23.667 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:23 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:23.668 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:23 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:23.668 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:23 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:23.668 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:23 smithi138 ceph-mon[31441]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:40:23.707 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 24 08:40:23 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-z[33677]: 2024-04-24T08:40:23.452+0000 7f17ebc59200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-24T08:40:23.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:23 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:23.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:23 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:23.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:23 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:23.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:23 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:23.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:23 smithi189 ceph-mon[31791]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:40:23.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:23 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:23.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:23 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:23.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:23 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:23.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:23 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:23.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:23 smithi083 ceph-mon[27907]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:40:23.955 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Apr 24 08:40:23 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-y[32810]: 2024-04-24T08:40:23.666+0000 7f453b80f200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-24T08:40:23.955 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Apr 24 08:40:23 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-y[32810]: 2024-04-24T08:40:23.788+0000 7f453b80f200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-24T08:40:24.207 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 24 08:40:23 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-z[33677]: 2024-04-24T08:40:23.836+0000 7f17ebc59200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-24T08:40:24.297 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Apr 24 08:40:24 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-y[32810]: 2024-04-24T08:40:24.046+0000 7f453b80f200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-24T08:40:24.297 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Apr 24 08:40:24 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-y[32810]: 2024-04-24T08:40:24.296+0000 7f453b80f200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-24T08:40:24.896 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:24 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:40:24.896 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:24 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:40:24.896 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:24 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:24.896 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:24 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T08:40:24.896 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:24 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T08:40:24.896 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:24 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:40:24.896 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 24 08:40:24 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-z[33677]: 2024-04-24T08:40:24.619+0000 7f17ebc59200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-24T08:40:24.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:24 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:40:24.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:24 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:40:24.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:24 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:24.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:24 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T08:40:24.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:24 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T08:40:24.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:24 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:40:25.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:24 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:40:25.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:24 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:40:25.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:24 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:25.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:24 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T08:40:25.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:24 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T08:40:25.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:24 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:40:25.207 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 24 08:40:24 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-z[33677]: 2024-04-24T08:40:24.910+0000 7f17ebc59200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-24T08:40:25.455 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Apr 24 08:40:25 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-y[32810]: 2024-04-24T08:40:25.075+0000 7f453b80f200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-24T08:40:25.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:40:25.681+0000 7f4bd6a00700 1 -- 172.21.15.83:0/28551012 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mgrmap(e 14) ==== 92602+0+0 (secure 0 0 0) 0x7f4bd000f450 con 0x7f4be0104c90 2024-04-24T08:40:25.854 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:25 smithi189 ceph-mon[31791]: Reconfiguring mgr.x (unknown last config time)... 2024-04-24T08:40:25.854 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:25 smithi189 ceph-mon[31791]: Reconfiguring daemon mgr.x on smithi083 2024-04-24T08:40:25.854 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:25 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/2343547469' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2cd77f3a-19dd-4c6e-a199-1dcbd5faa679"}]: dispatch 2024-04-24T08:40:25.854 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:25 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/2343547469' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2cd77f3a-19dd-4c6e-a199-1dcbd5faa679"}]': finished 2024-04-24T08:40:25.854 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:25 smithi189 ceph-mon[31791]: osdmap e5: 1 total, 0 up, 1 in 2024-04-24T08:40:25.854 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:25 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T08:40:25.854 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:25 smithi189 ceph-mon[31791]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:40:25.854 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:25 smithi189 ceph-mon[31791]: Standby manager daemon z started 2024-04-24T08:40:25.854 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:25 smithi189 ceph-mon[31791]: from='mgr.? 172.21.15.189:0/4071059994' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-24T08:40:25.855 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:25 smithi189 ceph-mon[31791]: from='mgr.? 172.21.15.189:0/4071059994' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T08:40:25.855 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:25 smithi189 ceph-mon[31791]: from='mgr.? 172.21.15.189:0/4071059994' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-24T08:40:25.855 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:25 smithi189 ceph-mon[31791]: from='mgr.? 172.21.15.189:0/4071059994' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T08:40:25.855 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:25 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/459431637' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T08:40:25.955 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Apr 24 08:40:25 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-y[32810]: 2024-04-24T08:40:25.621+0000 7f453b80f200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-24T08:40:25.955 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Apr 24 08:40:25 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-y[32810]: 2024-04-24T08:40:25.748+0000 7f453b80f200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-24T08:40:25.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:25 smithi138 ceph-mon[31441]: Reconfiguring mgr.x (unknown last config time)... 2024-04-24T08:40:25.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:25 smithi138 ceph-mon[31441]: Reconfiguring daemon mgr.x on smithi083 2024-04-24T08:40:25.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:25 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/2343547469' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2cd77f3a-19dd-4c6e-a199-1dcbd5faa679"}]: dispatch 2024-04-24T08:40:25.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:25 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/2343547469' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2cd77f3a-19dd-4c6e-a199-1dcbd5faa679"}]': finished 2024-04-24T08:40:25.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:25 smithi138 ceph-mon[31441]: osdmap e5: 1 total, 0 up, 1 in 2024-04-24T08:40:25.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:25 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T08:40:25.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:25 smithi138 ceph-mon[31441]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:40:25.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:25 smithi138 ceph-mon[31441]: Standby manager daemon z started 2024-04-24T08:40:25.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:25 smithi138 ceph-mon[31441]: from='mgr.? 172.21.15.189:0/4071059994' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-24T08:40:25.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:25 smithi138 ceph-mon[31441]: from='mgr.? 172.21.15.189:0/4071059994' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T08:40:25.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:25 smithi138 ceph-mon[31441]: from='mgr.? 172.21.15.189:0/4071059994' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-24T08:40:25.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:25 smithi138 ceph-mon[31441]: from='mgr.? 172.21.15.189:0/4071059994' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T08:40:25.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:25 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/459431637' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T08:40:26.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:25 smithi083 ceph-mon[27907]: Reconfiguring mgr.x (unknown last config time)... 2024-04-24T08:40:26.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:25 smithi083 ceph-mon[27907]: Reconfiguring daemon mgr.x on smithi083 2024-04-24T08:40:26.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:25 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/2343547469' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2cd77f3a-19dd-4c6e-a199-1dcbd5faa679"}]: dispatch 2024-04-24T08:40:26.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:25 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/2343547469' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2cd77f3a-19dd-4c6e-a199-1dcbd5faa679"}]': finished 2024-04-24T08:40:26.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:25 smithi083 ceph-mon[27907]: osdmap e5: 1 total, 0 up, 1 in 2024-04-24T08:40:26.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:25 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T08:40:26.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:25 smithi083 ceph-mon[27907]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:40:26.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:25 smithi083 ceph-mon[27907]: Standby manager daemon z started 2024-04-24T08:40:26.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:25 smithi083 ceph-mon[27907]: from='mgr.? 172.21.15.189:0/4071059994' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-24T08:40:26.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:25 smithi083 ceph-mon[27907]: from='mgr.? 172.21.15.189:0/4071059994' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T08:40:26.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:25 smithi083 ceph-mon[27907]: from='mgr.? 172.21.15.189:0/4071059994' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-24T08:40:26.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:25 smithi083 ceph-mon[27907]: from='mgr.? 172.21.15.189:0/4071059994' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T08:40:26.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:25 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/459431637' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T08:40:26.455 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Apr 24 08:40:26 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-y[32810]: 2024-04-24T08:40:26.008+0000 7f453b80f200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-24T08:40:26.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:26 smithi138 ceph-mon[31441]: mgrmap e14: x(active, since 105s), standbys: z 2024-04-24T08:40:26.959 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:26 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-24T08:40:26.960 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Apr 24 08:40:26 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-y[32810]: 2024-04-24T08:40:26.669+0000 7f453b80f200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-24T08:40:26.960 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Apr 24 08:40:26 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-y[32810]: 2024-04-24T08:40:26.791+0000 7f453b80f200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-24T08:40:26.960 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:26 smithi189 ceph-mon[31791]: mgrmap e14: x(active, since 105s), standbys: z 2024-04-24T08:40:26.960 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:26 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-24T08:40:27.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:26 smithi083 ceph-mon[27907]: mgrmap e14: x(active, since 105s), standbys: z 2024-04-24T08:40:27.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:26 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-24T08:40:27.407 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Apr 24 08:40:27 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-y[32810]: 2024-04-24T08:40:27.105+0000 7f453b80f200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-24T08:40:27.407 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Apr 24 08:40:27 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-y[32810]: 2024-04-24T08:40:27.280+0000 7f453b80f200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-24T08:40:27.407 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Apr 24 08:40:27 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-y[32810]: 2024-04-24T08:40:27.406+0000 7f453b80f200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-24T08:40:27.692 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Apr 24 08:40:27 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-y[32810]: 2024-04-24T08:40:27.555+0000 7f453b80f200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-24T08:40:27.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:27 smithi138 ceph-mon[31441]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:40:27.955 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Apr 24 08:40:27 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-y[32810]: 2024-04-24T08:40:27.927+0000 7f453b80f200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-24T08:40:27.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:27 smithi189 ceph-mon[31791]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:40:28.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:27 smithi083 ceph-mon[27907]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:40:28.955 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Apr 24 08:40:28 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-y[32810]: 2024-04-24T08:40:28.673+0000 7f453b80f200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-24T08:40:29.455 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Apr 24 08:40:28 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-y[32810]: 2024-04-24T08:40:28.955+0000 7f453b80f200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-24T08:40:29.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:40:29.770+0000 7f4bd6a00700 1 -- 172.21.15.83:0/28551012 <== mon.0 v2:172.21.15.83:3300/0 8 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f4bd0017c30 con 0x7f4be0104c90 2024-04-24T08:40:30.009 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:29 smithi083 ceph-mon[27907]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:40:30.009 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:29 smithi083 ceph-mon[27907]: Standby manager daemon y started 2024-04-24T08:40:30.009 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:29 smithi083 ceph-mon[27907]: from='mgr.? 172.21.15.138:0/1570514678' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-24T08:40:30.009 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:29 smithi083 ceph-mon[27907]: from='mgr.? 172.21.15.138:0/1570514678' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T08:40:30.009 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:29 smithi083 ceph-mon[27907]: from='mgr.? 172.21.15.138:0/1570514678' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-24T08:40:30.009 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:29 smithi083 ceph-mon[27907]: from='mgr.? 172.21.15.138:0/1570514678' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T08:40:30.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:29 smithi138 ceph-mon[31441]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:40:30.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:29 smithi138 ceph-mon[31441]: Standby manager daemon y started 2024-04-24T08:40:30.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:29 smithi138 ceph-mon[31441]: from='mgr.? 172.21.15.138:0/1570514678' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-24T08:40:30.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:29 smithi138 ceph-mon[31441]: from='mgr.? 172.21.15.138:0/1570514678' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T08:40:30.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:29 smithi138 ceph-mon[31441]: from='mgr.? 172.21.15.138:0/1570514678' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-24T08:40:30.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:29 smithi138 ceph-mon[31441]: from='mgr.? 172.21.15.138:0/1570514678' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T08:40:30.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:29 smithi189 ceph-mon[31791]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:40:30.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:29 smithi189 ceph-mon[31791]: Standby manager daemon y started 2024-04-24T08:40:30.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:29 smithi189 ceph-mon[31791]: from='mgr.? 172.21.15.138:0/1570514678' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-24T08:40:30.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:29 smithi189 ceph-mon[31791]: from='mgr.? 172.21.15.138:0/1570514678' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T08:40:30.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:29 smithi189 ceph-mon[31791]: from='mgr.? 172.21.15.138:0/1570514678' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-24T08:40:30.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:29 smithi189 ceph-mon[31791]: from='mgr.? 172.21.15.138:0/1570514678' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T08:40:31.047 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:30 smithi083 ceph-mon[27907]: mgrmap e15: x(active, since 109s), standbys: z, y 2024-04-24T08:40:31.047 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:30 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-24T08:40:31.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:30 smithi138 ceph-mon[31441]: mgrmap e15: x(active, since 109s), standbys: z, y 2024-04-24T08:40:31.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:30 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-24T08:40:31.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:30 smithi189 ceph-mon[31791]: mgrmap e15: x(active, since 109s), standbys: z, y 2024-04-24T08:40:31.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:30 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-24T08:40:32.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:31 smithi083 ceph-mon[27907]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:40:32.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:31 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:32.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:31 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:32.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:31 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:40:32.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:31 smithi138 ceph-mon[31441]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:40:32.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:31 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:32.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:31 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:32.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:31 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:40:32.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:31 smithi189 ceph-mon[31791]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:40:32.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:31 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:32.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:31 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:32.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:31 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:40:33.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:32 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:40:33.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:32 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:40:33.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:32 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:33.107 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:32 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:40:33.107 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:32 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:40:33.107 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:32 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:33.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:32 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:40:33.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:32 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:40:33.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:32 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:34.040 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:33 smithi083 ceph-mon[27907]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:40:34.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:33 smithi138 ceph-mon[31441]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:40:34.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:33 smithi189 ceph-mon[31791]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:40:34.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:34 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-24T08:40:34.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:34 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:40:35.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:34 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-24T08:40:35.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:34 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:40:35.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:34 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-24T08:40:35.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:34 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:40:36.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:35 smithi083 ceph-mon[27907]: Deploying daemon osd.0 on smithi083 2024-04-24T08:40:36.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:35 smithi083 ceph-mon[27907]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:40:36.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:35 smithi138 ceph-mon[31441]: Deploying daemon osd.0 on smithi083 2024-04-24T08:40:36.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:35 smithi138 ceph-mon[31441]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:40:36.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:35 smithi189 ceph-mon[31791]: Deploying daemon osd.0 on smithi083 2024-04-24T08:40:36.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:35 smithi189 ceph-mon[31791]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:40:37.979 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:37 smithi083 ceph-mon[27907]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:40:38.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:37 smithi138 ceph-mon[31441]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:40:38.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:37 smithi189 ceph-mon[31791]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:40:39.976 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:39 smithi083 ceph-mon[27907]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:40:40.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:39 smithi138 ceph-mon[31441]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:40:40.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:39 smithi189 ceph-mon[31791]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:40:42.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:41 smithi083 ceph-mon[27907]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:40:42.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:41 smithi138 ceph-mon[31441]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:40:42.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:41 smithi189 ceph-mon[31791]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:40:44.094 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:43 smithi083 ceph-mon[27907]: pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:40:44.094 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:43 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:44.094 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:43 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:44.094 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:43 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:40:44.147 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:43 smithi138 ceph-mon[31441]: pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:40:44.147 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:43 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:44.147 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:43 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:44.147 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:43 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:40:44.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:43 smithi189 ceph-mon[31791]: pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:40:44.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:43 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:44.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:43 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:44.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:43 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:40:46.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:45 smithi083 ceph-mon[27907]: pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:40:46.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:45 smithi138 ceph-mon[31441]: pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:40:46.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:45 smithi189 ceph-mon[31791]: pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:40:47.235 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:47 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:47.235 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:47 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:47.235 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:47 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:40:47.235 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:47 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:40:47.235 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:47 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:47.235 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:47 smithi083 ceph-mon[27907]: pgmap v58: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:40:47.235 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:47 smithi083 ceph-mon[27907]: from='osd.0 [v2:172.21.15.83:6802/2132675555,v1:172.21.15.83:6803/2132675555]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-24T08:40:47.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:47 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:47.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:47 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:47.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:47 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:40:47.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:47 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:40:47.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:47 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:47.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:47 smithi138 ceph-mon[31441]: pgmap v58: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:40:47.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:47 smithi138 ceph-mon[31441]: from='osd.0 [v2:172.21.15.83:6802/2132675555,v1:172.21.15.83:6803/2132675555]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-24T08:40:47.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:47 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:47.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:47 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:47.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:47 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:40:47.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:47 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:40:47.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:47 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:47.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:47 smithi189 ceph-mon[31791]: pgmap v58: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:40:47.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:47 smithi189 ceph-mon[31791]: from='osd.0 [v2:172.21.15.83:6802/2132675555,v1:172.21.15.83:6803/2132675555]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-24T08:40:48.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:40:48.206+0000 7f4bd6a00700 1 -- 172.21.15.83:0/28551012 <== mgr.14152 v2:172.21.15.83:6800/2550826560 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f4be0062f30 con 0x7f4bc4039890 2024-04-24T08:40:48.209 INFO:teuthology.orchestra.run.smithi083.stdout:Created osd(s) 0 on host 'smithi083' 2024-04-24T08:40:48.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:40:48.211+0000 7f4bdf200700 1 -- 172.21.15.83:0/28551012 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f4bc4039890 msgr2=0x7f4bc403bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:40:48.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:40:48.211+0000 7f4bdf200700 1 --2- 172.21.15.83:0/28551012 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f4bc4039890 0x7f4bc403bd50 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc000c00 tx=0x7f4bcc005fb0 comp rx=0 tx=0).stop 2024-04-24T08:40:48.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:40:48.211+0000 7f4bdf200700 1 -- 172.21.15.83:0/28551012 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4be0104c90 msgr2=0x7f4be019ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:40:48.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:40:48.211+0000 7f4bdf200700 1 --2- 172.21.15.83:0/28551012 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4be0104c90 0x7f4be019ae50 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f4bd000c9a0 tx=0x7f4bd000ccb0 comp rx=0 tx=0).stop 2024-04-24T08:40:48.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:40:48.212+0000 7f4bdf200700 1 -- 172.21.15.83:0/28551012 shutdown_connections 2024-04-24T08:40:48.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:40:48.212+0000 7f4bdf200700 1 --2- 172.21.15.83:0/28551012 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f4bc4039890 0x7f4bc403bd50 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:40:48.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:40:48.212+0000 7f4bdf200700 1 --2- 172.21.15.83:0/28551012 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4be0104c90 0x7f4be019ae50 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:40:48.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:40:48.212+0000 7f4bdf200700 1 --2- 172.21.15.83:0/28551012 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4be0106830 0x7f4be01a23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:40:48.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:40:48.212+0000 7f4bdf200700 1 --2- 172.21.15.83:0/28551012 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4be0105e90 0x7f4be019b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:40:48.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:40:48.212+0000 7f4bdf200700 1 -- 172.21.15.83:0/28551012 >> 172.21.15.83:0/28551012 conn(0x7f4be0100230 msgr2=0x7f4be01091b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:40:48.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:40:48.212+0000 7f4bdf200700 1 -- 172.21.15.83:0/28551012 shutdown_connections 2024-04-24T08:40:48.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:40:48.212+0000 7f4bdf200700 1 -- 172.21.15.83:0/28551012 wait complete. 2024-04-24T08:40:48.352 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:48 smithi083 ceph-mon[27907]: from='osd.0 [v2:172.21.15.83:6802/2132675555,v1:172.21.15.83:6803/2132675555]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-24T08:40:48.352 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:48 smithi083 ceph-mon[27907]: osdmap e6: 1 total, 0 up, 1 in 2024-04-24T08:40:48.352 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:48 smithi083 ceph-mon[27907]: from='osd.0 [v2:172.21.15.83:6802/2132675555,v1:172.21.15.83:6803/2132675555]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-24T08:40:48.352 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:48 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T08:40:48.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:48 smithi138 ceph-mon[31441]: from='osd.0 [v2:172.21.15.83:6802/2132675555,v1:172.21.15.83:6803/2132675555]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-24T08:40:48.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:48 smithi138 ceph-mon[31441]: osdmap e6: 1 total, 0 up, 1 in 2024-04-24T08:40:48.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:48 smithi138 ceph-mon[31441]: from='osd.0 [v2:172.21.15.83:6802/2132675555,v1:172.21.15.83:6803/2132675555]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-24T08:40:48.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:48 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T08:40:48.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:48 smithi189 ceph-mon[31791]: from='osd.0 [v2:172.21.15.83:6802/2132675555,v1:172.21.15.83:6803/2132675555]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-24T08:40:48.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:48 smithi189 ceph-mon[31791]: osdmap e6: 1 total, 0 up, 1 in 2024-04-24T08:40:48.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:48 smithi189 ceph-mon[31791]: from='osd.0 [v2:172.21.15.83:6802/2132675555,v1:172.21.15.83:6803/2132675555]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-24T08:40:48.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:48 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T08:40:49.103 DEBUG:teuthology.orchestra.run.smithi083:osd.0> sudo journalctl -f -n 0 -u ceph-bd3e039e-0215-11ef-bc93-c7b262605968@osd.0.service 2024-04-24T08:40:49.108 INFO:tasks.cephadm:Deploying osd.1 on smithi138 with /dev/vg_nvme/lv_4... 2024-04-24T08:40:49.108 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd3e039e-0215-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-24T08:40:49.161 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:49 smithi083 ceph-mon[27907]: from='osd.0 [v2:172.21.15.83:6802/2132675555,v1:172.21.15.83:6803/2132675555]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-04-24T08:40:49.162 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:49 smithi083 ceph-mon[27907]: osdmap e7: 1 total, 0 up, 1 in 2024-04-24T08:40:49.162 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:49 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T08:40:49.162 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:49 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T08:40:49.162 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:49 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:49.162 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:49 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:40:49.162 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:49 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:49.162 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:49 smithi083 ceph-mon[27907]: pgmap v61: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:40:49.162 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:49 smithi083 ceph-mon[27907]: from='osd.0 [v2:172.21.15.83:6802/2132675555,v1:172.21.15.83:6803/2132675555]' entity='osd.0' 2024-04-24T08:40:49.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:49 smithi138 ceph-mon[31441]: from='osd.0 [v2:172.21.15.83:6802/2132675555,v1:172.21.15.83:6803/2132675555]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-04-24T08:40:49.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:49 smithi138 ceph-mon[31441]: osdmap e7: 1 total, 0 up, 1 in 2024-04-24T08:40:49.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:49 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T08:40:49.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:49 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T08:40:49.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:49 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:49.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:49 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:40:49.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:49 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:49.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:49 smithi138 ceph-mon[31441]: pgmap v61: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:40:49.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:49 smithi138 ceph-mon[31441]: from='osd.0 [v2:172.21.15.83:6802/2132675555,v1:172.21.15.83:6803/2132675555]' entity='osd.0' 2024-04-24T08:40:49.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:49 smithi189 ceph-mon[31791]: from='osd.0 [v2:172.21.15.83:6802/2132675555,v1:172.21.15.83:6803/2132675555]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-04-24T08:40:49.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:49 smithi189 ceph-mon[31791]: osdmap e7: 1 total, 0 up, 1 in 2024-04-24T08:40:49.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:49 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T08:40:49.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:49 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T08:40:49.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:49 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:49.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:49 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:40:49.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:49 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:49.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:49 smithi189 ceph-mon[31791]: pgmap v61: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T08:40:49.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:49 smithi189 ceph-mon[31791]: from='osd.0 [v2:172.21.15.83:6802/2132675555,v1:172.21.15.83:6803/2132675555]' entity='osd.0' 2024-04-24T08:40:49.539 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T08:40:50.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:50 smithi138 ceph-mon[31441]: purged_snaps scrub starts 2024-04-24T08:40:50.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:50 smithi138 ceph-mon[31441]: purged_snaps scrub ok 2024-04-24T08:40:50.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:50 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T08:40:50.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:50 smithi138 ceph-mon[31441]: osd.0 [v2:172.21.15.83:6802/2132675555,v1:172.21.15.83:6803/2132675555] boot 2024-04-24T08:40:50.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:50 smithi138 ceph-mon[31441]: osdmap e8: 1 total, 1 up, 1 in 2024-04-24T08:40:50.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:50 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T08:40:50.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:50 smithi189 ceph-mon[31791]: purged_snaps scrub starts 2024-04-24T08:40:50.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:50 smithi189 ceph-mon[31791]: purged_snaps scrub ok 2024-04-24T08:40:50.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:50 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T08:40:50.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:50 smithi189 ceph-mon[31791]: osd.0 [v2:172.21.15.83:6802/2132675555,v1:172.21.15.83:6803/2132675555] boot 2024-04-24T08:40:50.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:50 smithi189 ceph-mon[31791]: osdmap e8: 1 total, 1 up, 1 in 2024-04-24T08:40:50.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:50 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T08:40:50.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:50 smithi083 ceph-mon[27907]: purged_snaps scrub starts 2024-04-24T08:40:50.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:50 smithi083 ceph-mon[27907]: purged_snaps scrub ok 2024-04-24T08:40:50.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:50 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T08:40:50.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:50 smithi083 ceph-mon[27907]: osd.0 [v2:172.21.15.83:6802/2132675555,v1:172.21.15.83:6803/2132675555] boot 2024-04-24T08:40:50.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:50 smithi083 ceph-mon[27907]: osdmap e8: 1 total, 1 up, 1 in 2024-04-24T08:40:50.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:50 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T08:40:51.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:51 smithi083 ceph-mon[27907]: pgmap v63: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:40:51.332 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:51 smithi138 ceph-mon[31441]: pgmap v63: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:40:51.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:51 smithi189 ceph-mon[31791]: pgmap v63: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:40:52.318 INFO:teuthology.orchestra.run.smithi138.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-24T08:40:52.319 INFO:teuthology.orchestra.run.smithi138.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-24T08:40:52.319 INFO:teuthology.orchestra.run.smithi138.stderr: stderr: 10+0 records in 2024-04-24T08:40:52.319 INFO:teuthology.orchestra.run.smithi138.stderr:10+0 records out 2024-04-24T08:40:52.319 INFO:teuthology.orchestra.run.smithi138.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0351132 s, 299 MB/s 2024-04-24T08:40:52.319 INFO:teuthology.orchestra.run.smithi138.stderr:--> Zapping successful for: 2024-04-24T08:40:52.341 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:52 smithi138 ceph-mon[31441]: osdmap e9: 1 total, 1 up, 1 in 2024-04-24T08:40:52.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:52 smithi189 ceph-mon[31791]: osdmap e9: 1 total, 1 up, 1 in 2024-04-24T08:40:52.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:52 smithi083 ceph-mon[27907]: osdmap e9: 1 total, 1 up, 1 in 2024-04-24T08:40:53.142 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd3e039e-0215-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi138:vg_nvme/lv_4 2024-04-24T08:40:53.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:53 smithi138 ceph-mon[31441]: pgmap v65: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:40:53.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:53 smithi189 ceph-mon[31791]: pgmap v65: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:40:53.514 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:53 smithi083 ceph-mon[27907]: pgmap v65: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:40:53.580 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T08:40:55.559 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:55 smithi138 ceph-mon[31441]: Detected new or changed devices on smithi083 2024-04-24T08:40:55.560 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:55 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:55.560 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:55 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:55.560 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:55 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:40:55.560 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:55 smithi138 ceph-mon[31441]: Adjusting osd_memory_target on smithi083 to 17089M 2024-04-24T08:40:55.560 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:55 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:55.560 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:55 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:40:55.560 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:55 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:40:55.560 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:55 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:55.560 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:55 smithi138 ceph-mon[31441]: pgmap v66: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:40:55.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:40:55.730+0000 7f5ca2400700 1 -- 172.21.15.138:0/198690804 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c9c109190 msgr2=0x7f5c9c1095b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:40:55.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:40:55.730+0000 7f5ca2400700 1 --2- 172.21.15.138:0/198690804 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c9c109190 0x7f5c9c1095b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c0099c0 tx=0x7f5c8c009cd0 comp rx=0 tx=0).stop 2024-04-24T08:40:55.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:40:55.730+0000 7f5ca2400700 1 -- 172.21.15.138:0/198690804 shutdown_connections 2024-04-24T08:40:55.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:40:55.730+0000 7f5ca2400700 1 --2- 172.21.15.138:0/198690804 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5c9c10ad30 0x7f5c9c1115e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:40:55.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:40:55.730+0000 7f5ca2400700 1 --2- 172.21.15.138:0/198690804 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c9c109190 0x7f5c9c1095b0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:40:55.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:40:55.730+0000 7f5ca2400700 1 --2- 172.21.15.138:0/198690804 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5c9c10a390 0x7f5c9c10a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:40:55.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:40:55.730+0000 7f5ca2400700 1 -- 172.21.15.138:0/198690804 >> 172.21.15.138:0/198690804 conn(0x7f5c9c078160 msgr2=0x7f5c9c07a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:40:55.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:40:55.730+0000 7f5ca2400700 1 -- 172.21.15.138:0/198690804 shutdown_connections 2024-04-24T08:40:55.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:40:55.730+0000 7f5ca2400700 1 -- 172.21.15.138:0/198690804 wait complete. 2024-04-24T08:40:55.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:40:55.731+0000 7f5ca2400700 1 Processor -- start 2024-04-24T08:40:55.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:40:55.732+0000 7f5ca2400700 1 -- start start 2024-04-24T08:40:55.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:40:55.732+0000 7f5ca2400700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c9c109190 0x7f5c9c19f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:40:55.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:40:55.732+0000 7f5ca2400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5c9c10a390 0x7f5c9c19f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:40:55.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:40:55.732+0000 7f5ca2400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5c9c10ad30 0x7f5c9c1a6850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:40:55.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:40:55.732+0000 7f5ca2400700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5c9c114c20 con 0x7f5c9c10ad30 2024-04-24T08:40:55.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:40:55.732+0000 7f5ca2400700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5c9c114aa0 con 0x7f5c9c109190 2024-04-24T08:40:55.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:40:55.732+0000 7f5ca2400700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5c9c114da0 con 0x7f5c9c10a390 2024-04-24T08:40:55.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:40:55.732+0000 7f5c9b400700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c9c109190 0x7f5c9c19f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:40:55.734 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:40:55.732+0000 7f5c9b400700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c9c109190 0x7f5c9c19f330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:52166/0 (socket says 172.21.15.138:52166) 2024-04-24T08:40:55.734 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:40:55.733+0000 7f5c9b400700 1 -- 172.21.15.138:0/944846135 learned_addr learned my addr 172.21.15.138:0/944846135 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-24T08:40:55.734 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:40:55.733+0000 7f5c9be00700 1 --2- 172.21.15.138:0/944846135 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5c9c10ad30 0x7f5c9c1a6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:40:55.734 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:40:55.733+0000 7f5c9aa00700 1 --2- 172.21.15.138:0/944846135 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5c9c10a390 0x7f5c9c19f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:40:55.734 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:40:55.733+0000 7f5c9b400700 1 -- 172.21.15.138:0/944846135 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5c9c10a390 msgr2=0x7f5c9c19f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:40:55.734 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:40:55.733+0000 7f5c9b400700 1 --2- 172.21.15.138:0/944846135 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5c9c10a390 0x7f5c9c19f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:40:55.734 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:40:55.733+0000 7f5c9b400700 1 -- 172.21.15.138:0/944846135 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5c9c10ad30 msgr2=0x7f5c9c1a6850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:40:55.734 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:40:55.733+0000 7f5c9b400700 1 --2- 172.21.15.138:0/944846135 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5c9c10ad30 0x7f5c9c1a6850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:40:55.734 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:40:55.733+0000 7f5c9b400700 1 -- 172.21.15.138:0/944846135 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c9c1a6ef0 con 0x7f5c9c109190 2024-04-24T08:40:55.734 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:40:55.733+0000 7f5c9be00700 1 --2- 172.21.15.138:0/944846135 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5c9c10ad30 0x7f5c9c1a6850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:40:55.734 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:40:55.733+0000 7f5c9b400700 1 --2- 172.21.15.138:0/944846135 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c9c109190 0x7f5c9c19f330 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c00c010 tx=0x7f5c8c009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:40:55.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:40:55.734+0000 7f5c93e00700 1 -- 172.21.15.138:0/944846135 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c8c027070 con 0x7f5c9c109190 2024-04-24T08:40:55.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:40:55.734+0000 7f5c93e00700 1 -- 172.21.15.138:0/944846135 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5c8c0040d0 con 0x7f5c9c109190 2024-04-24T08:40:55.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:40:55.734+0000 7f5c93e00700 1 -- 172.21.15.138:0/944846135 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c8c022d40 con 0x7f5c9c109190 2024-04-24T08:40:55.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:40:55.734+0000 7f5ca2400700 1 -- 172.21.15.138:0/944846135 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c9c1a7180 con 0x7f5c9c109190 2024-04-24T08:40:55.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:40:55.734+0000 7f5ca2400700 1 -- 172.21.15.138:0/944846135 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c9c1a76b0 con 0x7f5c9c109190 2024-04-24T08:40:55.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:40:55.735+0000 7f5ca2400700 1 -- 172.21.15.138:0/944846135 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c9c0697e0 con 0x7f5c9c109190 2024-04-24T08:40:55.737 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:40:55.737+0000 7f5c93e00700 1 -- 172.21.15.138:0/944846135 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f5c8c02a070 con 0x7f5c9c109190 2024-04-24T08:40:55.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:40:55.738+0000 7f5c93e00700 1 --2- 172.21.15.138:0/944846135 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f5c800a3c30 0x7f5c800a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:40:55.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:40:55.739+0000 7f5c93e00700 1 -- 172.21.15.138:0/944846135 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7f5c8c0cf940 con 0x7f5c9c109190 2024-04-24T08:40:55.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:40:55.739+0000 7f5c9aa00700 1 --2- 172.21.15.138:0/944846135 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f5c800a3c30 0x7f5c800a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:40:55.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:40:55.740+0000 7f5c9aa00700 1 --2- 172.21.15.138:0/944846135 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f5c800a3c30 0x7f5c800a60f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5c88005950 tx=0x7f5c880058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:40:55.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:40:55.744+0000 7f5c93e00700 1 -- 172.21.15.138:0/944846135 <== mon.2 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5c8c05eff0 con 0x7f5c9c109190 2024-04-24T08:40:55.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:55 smithi083 ceph-mon[27907]: Detected new or changed devices on smithi083 2024-04-24T08:40:55.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:55 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:55.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:55 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:55.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:55 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:40:55.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:55 smithi083 ceph-mon[27907]: Adjusting osd_memory_target on smithi083 to 17089M 2024-04-24T08:40:55.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:55 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:55.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:55 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:40:55.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:55 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:40:55.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:55 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:55.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:55 smithi083 ceph-mon[27907]: pgmap v66: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:40:55.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:55 smithi189 ceph-mon[31791]: Detected new or changed devices on smithi083 2024-04-24T08:40:55.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:55 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:55.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:55 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:55.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:55 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:40:55.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:55 smithi189 ceph-mon[31791]: Adjusting osd_memory_target on smithi083 to 17089M 2024-04-24T08:40:55.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:55 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:55.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:55 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:40:55.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:55 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:40:55.959 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:55 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:40:55.959 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:55 smithi189 ceph-mon[31791]: pgmap v66: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:40:55.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:40:55.979+0000 7f5ca2400700 1 -- 172.21.15.138:0/944846135 --> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f5c9c062f30 con 0x7f5c800a3c30 2024-04-24T08:40:56.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:56 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T08:40:56.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:56 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T08:40:56.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:56 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:40:56.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:56 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T08:40:56.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:56 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T08:40:56.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:56 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:40:56.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:56 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T08:40:56.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:56 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T08:40:56.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:56 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:40:57.709 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:57 smithi138 ceph-mon[31441]: from='client.34139 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:40:57.710 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:57 smithi138 ceph-mon[31441]: pgmap v67: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:40:57.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:57 smithi083 ceph-mon[27907]: from='client.34139 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:40:57.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:57 smithi083 ceph-mon[27907]: pgmap v67: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:40:57.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:57 smithi189 ceph-mon[31791]: from='client.34139 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:40:57.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:57 smithi189 ceph-mon[31791]: pgmap v67: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:41:00.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:40:59 smithi083 ceph-mon[27907]: pgmap v68: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:41:00.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:40:59 smithi138 ceph-mon[31441]: pgmap v68: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:41:00.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:40:59 smithi189 ceph-mon[31791]: pgmap v68: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:41:01.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:00 smithi083 ceph-mon[27907]: from='client.? 172.21.15.138:0/1266034529' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aba7f5df-2a4c-4c23-b502-d193605a5b1b"}]: dispatch 2024-04-24T08:41:01.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:00 smithi083 ceph-mon[27907]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aba7f5df-2a4c-4c23-b502-d193605a5b1b"}]: dispatch 2024-04-24T08:41:01.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:00 smithi083 ceph-mon[27907]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "aba7f5df-2a4c-4c23-b502-d193605a5b1b"}]': finished 2024-04-24T08:41:01.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:00 smithi083 ceph-mon[27907]: osdmap e10: 2 total, 1 up, 2 in 2024-04-24T08:41:01.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:00 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T08:41:01.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:00 smithi083 ceph-mon[27907]: from='client.? 172.21.15.138:0/3859497468' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T08:41:01.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:00 smithi138 ceph-mon[31441]: from='client.? 172.21.15.138:0/1266034529' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aba7f5df-2a4c-4c23-b502-d193605a5b1b"}]: dispatch 2024-04-24T08:41:01.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:00 smithi138 ceph-mon[31441]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aba7f5df-2a4c-4c23-b502-d193605a5b1b"}]: dispatch 2024-04-24T08:41:01.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:00 smithi138 ceph-mon[31441]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "aba7f5df-2a4c-4c23-b502-d193605a5b1b"}]': finished 2024-04-24T08:41:01.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:00 smithi138 ceph-mon[31441]: osdmap e10: 2 total, 1 up, 2 in 2024-04-24T08:41:01.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:00 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T08:41:01.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:00 smithi138 ceph-mon[31441]: from='client.? 172.21.15.138:0/3859497468' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T08:41:01.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:00 smithi189 ceph-mon[31791]: from='client.? 172.21.15.138:0/1266034529' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aba7f5df-2a4c-4c23-b502-d193605a5b1b"}]: dispatch 2024-04-24T08:41:01.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:00 smithi189 ceph-mon[31791]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aba7f5df-2a4c-4c23-b502-d193605a5b1b"}]: dispatch 2024-04-24T08:41:01.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:00 smithi189 ceph-mon[31791]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "aba7f5df-2a4c-4c23-b502-d193605a5b1b"}]': finished 2024-04-24T08:41:01.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:00 smithi189 ceph-mon[31791]: osdmap e10: 2 total, 1 up, 2 in 2024-04-24T08:41:01.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:00 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T08:41:01.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:00 smithi189 ceph-mon[31791]: from='client.? 172.21.15.138:0/3859497468' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T08:41:02.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:01 smithi138 ceph-mon[31441]: pgmap v70: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:41:02.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:01 smithi189 ceph-mon[31791]: pgmap v70: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:41:02.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:01 smithi083 ceph-mon[27907]: pgmap v70: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:41:04.060 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:03 smithi138 ceph-mon[31441]: pgmap v71: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:41:04.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:03 smithi189 ceph-mon[31791]: pgmap v71: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:41:04.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:03 smithi083 ceph-mon[27907]: pgmap v71: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:41:05.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:05 smithi138 ceph-mon[31441]: pgmap v72: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:41:06.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:05 smithi189 ceph-mon[31791]: pgmap v72: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:41:06.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:05 smithi083 ceph-mon[27907]: pgmap v72: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:41:08.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:07 smithi138 ceph-mon[31441]: pgmap v73: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:41:08.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:07 smithi189 ceph-mon[31791]: pgmap v73: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:41:08.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:07 smithi083 ceph-mon[27907]: pgmap v73: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:41:09.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:08 smithi138 ceph-mon[31441]: pgmap v74: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:41:09.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:08 smithi189 ceph-mon[31791]: pgmap v74: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:41:09.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:08 smithi083 ceph-mon[27907]: pgmap v74: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:41:10.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:09 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-24T08:41:10.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:09 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:41:10.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:09 smithi083 ceph-mon[27907]: Deploying daemon osd.1 on smithi138 2024-04-24T08:41:10.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:09 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-24T08:41:10.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:09 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:41:10.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:09 smithi138 ceph-mon[31441]: Deploying daemon osd.1 on smithi138 2024-04-24T08:41:10.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:09 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-24T08:41:10.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:09 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:41:10.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:09 smithi189 ceph-mon[31791]: Deploying daemon osd.1 on smithi138 2024-04-24T08:41:10.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:10 smithi138 ceph-mon[31441]: pgmap v75: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:41:11.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:10 smithi189 ceph-mon[31791]: pgmap v75: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:41:11.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:10 smithi083 ceph-mon[27907]: pgmap v75: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:41:13.840 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:13 smithi138 ceph-mon[31441]: pgmap v76: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:41:14.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:13 smithi083 ceph-mon[27907]: pgmap v76: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:41:14.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:13 smithi189 ceph-mon[31791]: pgmap v76: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:41:16.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:15 smithi083 ceph-mon[27907]: pgmap v77: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:41:16.156 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:15 smithi138 ceph-mon[31441]: pgmap v77: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:41:16.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:15 smithi189 ceph-mon[31791]: pgmap v77: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:41:18.035 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:17 smithi138 ceph-mon[31441]: pgmap v78: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:41:18.035 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:17 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:41:18.035 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:17 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:41:18.035 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:17 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:41:18.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:17 smithi083 ceph-mon[27907]: pgmap v78: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:41:18.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:17 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:41:18.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:17 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:41:18.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:17 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:41:18.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:17 smithi189 ceph-mon[31791]: pgmap v78: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:41:18.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:17 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:41:18.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:17 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:41:18.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:17 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:41:19.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:19 smithi138 ceph-mon[31441]: pgmap v79: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:41:20.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:19 smithi083 ceph-mon[27907]: pgmap v79: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:41:20.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:19 smithi189 ceph-mon[31791]: pgmap v79: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:41:21.696 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:21 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:41:21.697 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:21 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:41:21.697 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:21 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:41:21.697 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:21 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:41:21.697 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:21 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:41:21.697 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:21 smithi138 ceph-mon[31441]: pgmap v80: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:41:21.697 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:21 smithi138 ceph-mon[31441]: from='osd.1 [v2:172.21.15.138:6800/3589158880,v1:172.21.15.138:6801/3589158880]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-24T08:41:21.697 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:21 smithi138 ceph-mon[31441]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-24T08:41:21.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:21 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:41:21.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:21 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:41:21.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:21 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:41:21.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:21 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:41:21.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:21 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:41:21.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:21 smithi083 ceph-mon[27907]: pgmap v80: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:41:21.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:21 smithi083 ceph-mon[27907]: from='osd.1 [v2:172.21.15.138:6800/3589158880,v1:172.21.15.138:6801/3589158880]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-24T08:41:21.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:21 smithi083 ceph-mon[27907]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-24T08:41:21.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:41:21.838+0000 7f5c93e00700 1 -- 172.21.15.138:0/944846135 <== mgr.14152 v2:172.21.15.83:6800/2550826560 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f5c9c062f30 con 0x7f5c800a3c30 2024-04-24T08:41:21.839 INFO:teuthology.orchestra.run.smithi138.stdout:Created osd(s) 1 on host 'smithi138' 2024-04-24T08:41:21.842 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:41:21.842+0000 7f5ca2400700 1 -- 172.21.15.138:0/944846135 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f5c800a3c30 msgr2=0x7f5c800a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:41:21.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:41:21.842+0000 7f5ca2400700 1 --2- 172.21.15.138:0/944846135 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f5c800a3c30 0x7f5c800a60f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5c88005950 tx=0x7f5c880058e0 comp rx=0 tx=0).stop 2024-04-24T08:41:21.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:41:21.842+0000 7f5ca2400700 1 -- 172.21.15.138:0/944846135 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c9c109190 msgr2=0x7f5c9c19f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:41:21.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:41:21.842+0000 7f5ca2400700 1 --2- 172.21.15.138:0/944846135 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c9c109190 0x7f5c9c19f330 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c00c010 tx=0x7f5c8c009580 comp rx=0 tx=0).stop 2024-04-24T08:41:21.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:41:21.843+0000 7f5ca2400700 1 -- 172.21.15.138:0/944846135 shutdown_connections 2024-04-24T08:41:21.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:41:21.843+0000 7f5ca2400700 1 --2- 172.21.15.138:0/944846135 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f5c800a3c30 0x7f5c800a60f0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:41:21.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:41:21.843+0000 7f5ca2400700 1 --2- 172.21.15.138:0/944846135 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5c9c10ad30 0x7f5c9c1a6850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:41:21.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:41:21.843+0000 7f5ca2400700 1 --2- 172.21.15.138:0/944846135 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c9c109190 0x7f5c9c19f330 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:41:21.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:41:21.843+0000 7f5ca2400700 1 --2- 172.21.15.138:0/944846135 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5c9c10a390 0x7f5c9c19f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:41:21.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:41:21.843+0000 7f5ca2400700 1 -- 172.21.15.138:0/944846135 >> 172.21.15.138:0/944846135 conn(0x7f5c9c078160 msgr2=0x7f5c9c10d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:41:21.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:41:21.843+0000 7f5ca2400700 1 -- 172.21.15.138:0/944846135 shutdown_connections 2024-04-24T08:41:21.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:41:21.843+0000 7f5ca2400700 1 -- 172.21.15.138:0/944846135 wait complete. 2024-04-24T08:41:21.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:21 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:41:21.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:21 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:41:21.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:21 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:41:21.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:21 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:41:21.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:21 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:41:21.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:21 smithi189 ceph-mon[31791]: pgmap v80: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:41:21.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:21 smithi189 ceph-mon[31791]: from='osd.1 [v2:172.21.15.138:6800/3589158880,v1:172.21.15.138:6801/3589158880]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-24T08:41:21.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:21 smithi189 ceph-mon[31791]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-24T08:41:22.629 DEBUG:teuthology.orchestra.run.smithi138:osd.1> sudo journalctl -f -n 0 -u ceph-bd3e039e-0215-11ef-bc93-c7b262605968@osd.1.service 2024-04-24T08:41:22.633 INFO:tasks.cephadm:Deploying osd.2 on smithi189 with /dev/vg_nvme/lv_4... 2024-04-24T08:41:22.633 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd3e039e-0215-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-24T08:41:22.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:22 smithi083 ceph-mon[27907]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-24T08:41:22.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:22 smithi083 ceph-mon[27907]: osdmap e11: 2 total, 1 up, 2 in 2024-04-24T08:41:22.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:22 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T08:41:22.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:22 smithi083 ceph-mon[27907]: from='osd.1 [v2:172.21.15.138:6800/3589158880,v1:172.21.15.138:6801/3589158880]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-04-24T08:41:22.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:22 smithi083 ceph-mon[27907]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-04-24T08:41:22.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:22 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:41:22.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:22 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:41:22.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:22 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:41:22.845 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:22 smithi138 ceph-mon[31441]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-24T08:41:22.845 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:22 smithi138 ceph-mon[31441]: osdmap e11: 2 total, 1 up, 2 in 2024-04-24T08:41:22.845 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:22 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T08:41:22.845 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:22 smithi138 ceph-mon[31441]: from='osd.1 [v2:172.21.15.138:6800/3589158880,v1:172.21.15.138:6801/3589158880]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-04-24T08:41:22.846 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:22 smithi138 ceph-mon[31441]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-04-24T08:41:22.846 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:22 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:41:22.846 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:22 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:41:22.846 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:22 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:41:22.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:22 smithi189 ceph-mon[31791]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-24T08:41:22.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:22 smithi189 ceph-mon[31791]: osdmap e11: 2 total, 1 up, 2 in 2024-04-24T08:41:22.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:22 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T08:41:22.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:22 smithi189 ceph-mon[31791]: from='osd.1 [v2:172.21.15.138:6800/3589158880,v1:172.21.15.138:6801/3589158880]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-04-24T08:41:22.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:22 smithi189 ceph-mon[31791]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-04-24T08:41:22.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:22 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:41:22.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:22 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:41:22.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:22 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:41:23.063 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T08:41:23.544 INFO:journalctl@ceph.osd.1.smithi138.stdout:Apr 24 08:41:23 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-1[36048]: 2024-04-24T08:41:23.451+0000 7f09cfa00700 -1 osd.1 0 waiting for initial osdmap 2024-04-24T08:41:23.544 INFO:journalctl@ceph.osd.1.smithi138.stdout:Apr 24 08:41:23 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-1[36048]: 2024-04-24T08:41:23.471+0000 7f09c5800700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T08:41:23.795 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:23 smithi138 ceph-mon[31441]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-04-24T08:41:23.795 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:23 smithi138 ceph-mon[31441]: osdmap e12: 2 total, 1 up, 2 in 2024-04-24T08:41:23.795 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:23 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T08:41:23.795 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:23 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T08:41:23.795 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:23 smithi138 ceph-mon[31441]: pgmap v83: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:41:23.795 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:23 smithi138 ceph-mon[31441]: from='osd.1 ' entity='osd.1' 2024-04-24T08:41:23.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:23 smithi083 ceph-mon[27907]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-04-24T08:41:23.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:23 smithi083 ceph-mon[27907]: osdmap e12: 2 total, 1 up, 2 in 2024-04-24T08:41:23.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:23 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T08:41:23.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:23 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T08:41:23.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:23 smithi083 ceph-mon[27907]: pgmap v83: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:41:23.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:23 smithi083 ceph-mon[27907]: from='osd.1 ' entity='osd.1' 2024-04-24T08:41:23.950 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:23 smithi189 ceph-mon[31791]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-04-24T08:41:23.951 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:23 smithi189 ceph-mon[31791]: osdmap e12: 2 total, 1 up, 2 in 2024-04-24T08:41:23.951 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:23 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T08:41:23.951 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:23 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T08:41:23.951 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:23 smithi189 ceph-mon[31791]: pgmap v83: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T08:41:23.951 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:23 smithi189 ceph-mon[31791]: from='osd.1 ' entity='osd.1' 2024-04-24T08:41:24.670 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:24 smithi138 ceph-mon[31441]: purged_snaps scrub starts 2024-04-24T08:41:24.670 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:24 smithi138 ceph-mon[31441]: purged_snaps scrub ok 2024-04-24T08:41:24.670 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:24 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T08:41:24.670 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:24 smithi138 ceph-mon[31441]: osd.1 [v2:172.21.15.138:6800/3589158880,v1:172.21.15.138:6801/3589158880] boot 2024-04-24T08:41:24.670 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:24 smithi138 ceph-mon[31441]: osdmap e13: 2 total, 2 up, 2 in 2024-04-24T08:41:24.670 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:24 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T08:41:24.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:24 smithi083 ceph-mon[27907]: purged_snaps scrub starts 2024-04-24T08:41:24.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:24 smithi083 ceph-mon[27907]: purged_snaps scrub ok 2024-04-24T08:41:24.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:24 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T08:41:24.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:24 smithi083 ceph-mon[27907]: osd.1 [v2:172.21.15.138:6800/3589158880,v1:172.21.15.138:6801/3589158880] boot 2024-04-24T08:41:24.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:24 smithi083 ceph-mon[27907]: osdmap e13: 2 total, 2 up, 2 in 2024-04-24T08:41:24.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:24 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T08:41:24.873 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:24 smithi189 ceph-mon[31791]: purged_snaps scrub starts 2024-04-24T08:41:24.873 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:24 smithi189 ceph-mon[31791]: purged_snaps scrub ok 2024-04-24T08:41:24.873 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:24 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T08:41:24.873 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:24 smithi189 ceph-mon[31791]: osd.1 [v2:172.21.15.138:6800/3589158880,v1:172.21.15.138:6801/3589158880] boot 2024-04-24T08:41:24.873 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:24 smithi189 ceph-mon[31791]: osdmap e13: 2 total, 2 up, 2 in 2024-04-24T08:41:24.873 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:24 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T08:41:25.780 INFO:teuthology.orchestra.run.smithi189.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-24T08:41:25.780 INFO:teuthology.orchestra.run.smithi189.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-24T08:41:25.780 INFO:teuthology.orchestra.run.smithi189.stderr: stderr: 10+0 records in 2024-04-24T08:41:25.780 INFO:teuthology.orchestra.run.smithi189.stderr:10+0 records out 2024-04-24T08:41:25.780 INFO:teuthology.orchestra.run.smithi189.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0346048 s, 303 MB/s 2024-04-24T08:41:25.780 INFO:teuthology.orchestra.run.smithi189.stderr:--> Zapping successful for: 2024-04-24T08:41:25.829 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:25 smithi189 ceph-mon[31791]: pgmap v85: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:41:25.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:25 smithi138 ceph-mon[31441]: pgmap v85: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:41:26.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:25 smithi083 ceph-mon[27907]: pgmap v85: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:41:26.589 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd3e039e-0215-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi189:vg_nvme/lv_4 2024-04-24T08:41:26.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:26 smithi138 ceph-mon[31441]: osdmap e14: 2 total, 2 up, 2 in 2024-04-24T08:41:26.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:26 smithi189 ceph-mon[31791]: osdmap e14: 2 total, 2 up, 2 in 2024-04-24T08:41:27.008 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T08:41:27.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:26 smithi083 ceph-mon[27907]: osdmap e14: 2 total, 2 up, 2 in 2024-04-24T08:41:27.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:27 smithi189 ceph-mon[31791]: pgmap v87: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:41:27.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:27 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:41:27.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:27 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:41:27.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:27 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:41:27.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:27 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:41:27.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:27 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:41:27.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:27 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:41:27.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:27 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:41:27.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:27 smithi138 ceph-mon[31441]: pgmap v87: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:41:27.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:27 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:41:27.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:27 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:41:27.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:27 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:41:27.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:27 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:41:27.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:27 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:41:27.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:27 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:41:27.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:27 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:41:28.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:27 smithi083 ceph-mon[27907]: pgmap v87: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:41:28.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:27 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:41:28.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:27 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:41:28.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:27 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:41:28.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:27 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:41:28.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:27 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:41:28.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:27 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:41:28.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:27 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:41:28.821 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:28 smithi189 ceph-mon[31791]: Detected new or changed devices on smithi138 2024-04-24T08:41:28.821 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:28 smithi189 ceph-mon[31791]: Adjusting osd_memory_target on smithi138 to 17089M 2024-04-24T08:41:28.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:28 smithi138 ceph-mon[31441]: Detected new or changed devices on smithi138 2024-04-24T08:41:28.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:28 smithi138 ceph-mon[31441]: Adjusting osd_memory_target on smithi138 to 17089M 2024-04-24T08:41:28.993 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:41:28.992+0000 7f1ca1c00700 1 -- 172.21.15.189:0/605255090 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1c9c0754d0 msgr2=0x7f1c9c075950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:41:28.993 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:41:28.992+0000 7f1ca1c00700 1 --2- 172.21.15.189:0/605255090 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1c9c0754d0 0x7f1c9c075950 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1c84009990 tx=0x7f1c84009ca0 comp rx=0 tx=0).stop 2024-04-24T08:41:28.993 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:41:28.992+0000 7f1ca1c00700 1 -- 172.21.15.189:0/605255090 shutdown_connections 2024-04-24T08:41:28.993 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:41:28.992+0000 7f1ca1c00700 1 --2- 172.21.15.189:0/605255090 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c9c10aa80 0x7f1c9c10cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:41:28.993 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:41:28.992+0000 7f1ca1c00700 1 --2- 172.21.15.189:0/605255090 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c9c076aa0 0x7f1c9c074f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:41:28.993 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:41:28.992+0000 7f1ca1c00700 1 --2- 172.21.15.189:0/605255090 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1c9c0754d0 0x7f1c9c075950 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:41:28.993 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:41:28.992+0000 7f1ca1c00700 1 -- 172.21.15.189:0/605255090 >> 172.21.15.189:0/605255090 conn(0x7f1c9c0fe140 msgr2=0x7f1c9c100560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:41:28.993 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:41:28.992+0000 7f1ca1c00700 1 -- 172.21.15.189:0/605255090 shutdown_connections 2024-04-24T08:41:28.993 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:41:28.992+0000 7f1ca1c00700 1 -- 172.21.15.189:0/605255090 wait complete. 2024-04-24T08:41:28.995 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:41:28.993+0000 7f1ca1c00700 1 Processor -- start 2024-04-24T08:41:28.995 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:41:28.993+0000 7f1ca1c00700 1 -- start start 2024-04-24T08:41:28.995 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:41:28.994+0000 7f1ca1c00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c9c0754d0 0x7f1c9c19b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:41:28.995 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:41:28.994+0000 7f1ca1c00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c9c076aa0 0x7f1c9c19b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:41:28.995 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:41:28.994+0000 7f1ca1c00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1c9c10aa80 0x7f1c9c1a25d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:41:28.995 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:41:28.994+0000 7f1ca1c00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1c9c110940 con 0x7f1c9c076aa0 2024-04-24T08:41:28.995 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:41:28.994+0000 7f1ca1c00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1c9c1107c0 con 0x7f1c9c0754d0 2024-04-24T08:41:28.995 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:41:28.994+0000 7f1ca1c00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1c9c110ac0 con 0x7f1c9c10aa80 2024-04-24T08:41:28.996 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:41:28.994+0000 7f1c9b400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1c9c10aa80 0x7f1c9c1a25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:41:28.996 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:41:28.994+0000 7f1c9b400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1c9c10aa80 0x7f1c9c1a25d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.189:51106/0 (socket says 172.21.15.189:51106) 2024-04-24T08:41:28.996 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:41:28.994+0000 7f1c9b400700 1 -- 172.21.15.189:0/1146817510 learned_addr learned my addr 172.21.15.189:0/1146817510 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-04-24T08:41:28.996 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:41:28.994+0000 7f1c9aa00700 1 --2- 172.21.15.189:0/1146817510 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c9c0754d0 0x7f1c9c19b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:41:28.996 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:41:28.995+0000 7f1c9b400700 1 -- 172.21.15.189:0/1146817510 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c9c0754d0 msgr2=0x7f1c9c19b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:41:28.996 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:41:28.995+0000 7f1c9b400700 1 --2- 172.21.15.189:0/1146817510 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c9c0754d0 0x7f1c9c19b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:41:28.996 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:41:28.995+0000 7f1c9b400700 1 -- 172.21.15.189:0/1146817510 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c9c076aa0 msgr2=0x7f1c9c19b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:41:28.996 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:41:28.995+0000 7f1c9b400700 1 --2- 172.21.15.189:0/1146817510 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c9c076aa0 0x7f1c9c19b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:41:28.996 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:41:28.995+0000 7f1c9b400700 1 -- 172.21.15.189:0/1146817510 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c9c1a2c70 con 0x7f1c9c10aa80 2024-04-24T08:41:28.996 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:41:28.995+0000 7f1c9b400700 1 --2- 172.21.15.189:0/1146817510 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1c9c10aa80 0x7f1c9c1a25d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1c9400b7b0 tx=0x7f1c9400bb70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:41:28.997 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:41:28.995+0000 7f1c93400700 1 -- 172.21.15.189:0/1146817510 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c9400f840 con 0x7f1c9c10aa80 2024-04-24T08:41:28.997 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:41:28.995+0000 7f1c93400700 1 -- 172.21.15.189:0/1146817510 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1c94017400 con 0x7f1c9c10aa80 2024-04-24T08:41:28.997 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:41:28.995+0000 7f1ca1c00700 1 -- 172.21.15.189:0/1146817510 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c9c1a2f60 con 0x7f1c9c10aa80 2024-04-24T08:41:28.997 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:41:28.995+0000 7f1c93400700 1 -- 172.21.15.189:0/1146817510 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c9400d610 con 0x7f1c9c10aa80 2024-04-24T08:41:28.997 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:41:28.995+0000 7f1ca1c00700 1 -- 172.21.15.189:0/1146817510 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c9c1a33e0 con 0x7f1c9c10aa80 2024-04-24T08:41:28.997 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:41:28.997+0000 7f1ca1c00700 1 -- 172.21.15.189:0/1146817510 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c9c02dfe0 con 0x7f1c9c10aa80 2024-04-24T08:41:28.999 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:41:28.999+0000 7f1c93400700 1 -- 172.21.15.189:0/1146817510 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f1c940175b0 con 0x7f1c9c10aa80 2024-04-24T08:41:29.001 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:41:29.000+0000 7f1c93400700 1 --2- 172.21.15.189:0/1146817510 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f1c7c0a3bb0 0x7f1c7c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:41:29.001 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:41:29.001+0000 7f1c93400700 1 -- 172.21.15.189:0/1146817510 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7f1c940cf020 con 0x7f1c9c10aa80 2024-04-24T08:41:29.001 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:41:29.001+0000 7f1c9aa00700 1 --2- 172.21.15.189:0/1146817510 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f1c7c0a3bb0 0x7f1c7c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:41:29.002 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:41:29.002+0000 7f1c9aa00700 1 --2- 172.21.15.189:0/1146817510 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f1c7c0a3bb0 0x7f1c7c0a6070 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1c8c006fd0 tx=0x7f1c8c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:41:29.004 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:41:29.004+0000 7f1c93400700 1 -- 172.21.15.189:0/1146817510 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1c94013030 con 0x7f1c9c10aa80 2024-04-24T08:41:29.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:28 smithi083 ceph-mon[27907]: Detected new or changed devices on smithi138 2024-04-24T08:41:29.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:28 smithi083 ceph-mon[27907]: Adjusting osd_memory_target on smithi138 to 17089M 2024-04-24T08:41:29.238 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:41:29.238+0000 7f1ca1c00700 1 -- 172.21.15.189:0/1146817510 --> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f1c9c062f30 con 0x7f1c7c0a3bb0 2024-04-24T08:41:29.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:29 smithi138 ceph-mon[31441]: pgmap v88: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:41:29.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:29 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T08:41:29.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:29 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T08:41:29.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:29 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:41:29.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:29 smithi189 ceph-mon[31791]: pgmap v88: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:41:29.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:29 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T08:41:29.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:29 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T08:41:29.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:29 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:41:30.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:29 smithi083 ceph-mon[27907]: pgmap v88: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:41:30.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:29 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T08:41:30.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:29 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T08:41:30.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:29 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:41:30.846 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:30 smithi138 ceph-mon[31441]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:41:30.946 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:30 smithi189 ceph-mon[31791]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:41:31.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:30 smithi083 ceph-mon[27907]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:41:31.910 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:31 smithi189 ceph-mon[31791]: pgmap v89: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:41:31.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:31 smithi138 ceph-mon[31441]: pgmap v89: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:41:32.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:31 smithi083 ceph-mon[27907]: pgmap v89: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:41:33.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:33 smithi189 ceph-mon[31791]: pgmap v90: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:41:33.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:33 smithi189 ceph-mon[31791]: from='client.? 172.21.15.189:0/2231046107' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "071bc167-2333-4420-bb58-bffbc10d7641"}]: dispatch 2024-04-24T08:41:33.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:33 smithi189 ceph-mon[31791]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "071bc167-2333-4420-bb58-bffbc10d7641"}]: dispatch 2024-04-24T08:41:33.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:33 smithi189 ceph-mon[31791]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "071bc167-2333-4420-bb58-bffbc10d7641"}]': finished 2024-04-24T08:41:33.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:33 smithi189 ceph-mon[31791]: osdmap e15: 3 total, 2 up, 3 in 2024-04-24T08:41:33.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:33 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T08:41:34.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:33 smithi083 ceph-mon[27907]: pgmap v90: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:41:34.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:33 smithi083 ceph-mon[27907]: from='client.? 172.21.15.189:0/2231046107' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "071bc167-2333-4420-bb58-bffbc10d7641"}]: dispatch 2024-04-24T08:41:34.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:33 smithi083 ceph-mon[27907]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "071bc167-2333-4420-bb58-bffbc10d7641"}]: dispatch 2024-04-24T08:41:34.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:33 smithi083 ceph-mon[27907]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "071bc167-2333-4420-bb58-bffbc10d7641"}]': finished 2024-04-24T08:41:34.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:33 smithi083 ceph-mon[27907]: osdmap e15: 3 total, 2 up, 3 in 2024-04-24T08:41:34.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:33 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T08:41:34.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:33 smithi138 ceph-mon[31441]: pgmap v90: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:41:34.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:33 smithi138 ceph-mon[31441]: from='client.? 172.21.15.189:0/2231046107' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "071bc167-2333-4420-bb58-bffbc10d7641"}]: dispatch 2024-04-24T08:41:34.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:33 smithi138 ceph-mon[31441]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "071bc167-2333-4420-bb58-bffbc10d7641"}]: dispatch 2024-04-24T08:41:34.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:33 smithi138 ceph-mon[31441]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "071bc167-2333-4420-bb58-bffbc10d7641"}]': finished 2024-04-24T08:41:34.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:33 smithi138 ceph-mon[31441]: osdmap e15: 3 total, 2 up, 3 in 2024-04-24T08:41:34.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:33 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T08:41:34.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:34 smithi138 ceph-mon[31441]: from='client.? 172.21.15.189:0/2805886443' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T08:41:35.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:34 smithi083 ceph-mon[27907]: from='client.? 172.21.15.189:0/2805886443' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T08:41:35.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:34 smithi189 ceph-mon[31791]: from='client.? 172.21.15.189:0/2805886443' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T08:41:36.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:35 smithi083 ceph-mon[27907]: pgmap v92: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:41:36.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:35 smithi138 ceph-mon[31441]: pgmap v92: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:41:36.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:35 smithi189 ceph-mon[31791]: pgmap v92: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:41:38.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:37 smithi083 ceph-mon[27907]: pgmap v93: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:41:38.196 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:37 smithi189 ceph-mon[31791]: pgmap v93: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:41:38.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:37 smithi138 ceph-mon[31441]: pgmap v93: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:41:40.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:39 smithi083 ceph-mon[27907]: pgmap v94: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:41:40.201 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:39 smithi189 ceph-mon[31791]: pgmap v94: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:41:40.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:39 smithi138 ceph-mon[31441]: pgmap v94: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:41:41.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:41 smithi189 ceph-mon[31791]: pgmap v95: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:41:42.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:41 smithi083 ceph-mon[27907]: pgmap v95: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:41:42.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:41 smithi138 ceph-mon[31441]: pgmap v95: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:41:43.082 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:42 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-24T08:41:43.082 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:42 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:41:43.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:42 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-24T08:41:43.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:42 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:41:43.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:42 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-24T08:41:43.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:42 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:41:44.166 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:43 smithi138 ceph-mon[31441]: Deploying daemon osd.2 on smithi189 2024-04-24T08:41:44.166 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:43 smithi138 ceph-mon[31441]: pgmap v96: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:41:44.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:43 smithi189 ceph-mon[31791]: Deploying daemon osd.2 on smithi189 2024-04-24T08:41:44.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:43 smithi189 ceph-mon[31791]: pgmap v96: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:41:44.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:43 smithi083 ceph-mon[27907]: Deploying daemon osd.2 on smithi189 2024-04-24T08:41:44.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:43 smithi083 ceph-mon[27907]: pgmap v96: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:41:45.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:45 smithi189 ceph-mon[31791]: pgmap v97: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:41:46.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:45 smithi138 ceph-mon[31441]: pgmap v97: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:41:46.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:45 smithi083 ceph-mon[27907]: pgmap v97: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:41:48.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:47 smithi138 ceph-mon[31441]: pgmap v98: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:41:48.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:47 smithi189 ceph-mon[31791]: pgmap v98: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:41:48.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:47 smithi083 ceph-mon[27907]: pgmap v98: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:41:50.002 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:49 smithi189 ceph-mon[31791]: pgmap v99: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:41:50.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:49 smithi138 ceph-mon[31441]: pgmap v99: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:41:50.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:49 smithi083 ceph-mon[27907]: pgmap v99: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:41:51.373 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:51 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:41:51.373 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:51 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:41:51.373 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:51 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:41:51.373 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:51 smithi189 ceph-mon[31791]: pgmap v100: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:41:51.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:51 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:41:51.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:51 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:41:51.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:51 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:41:51.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:51 smithi138 ceph-mon[31441]: pgmap v100: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:41:51.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:51 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:41:51.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:51 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:41:51.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:51 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:41:51.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:51 smithi083 ceph-mon[27907]: pgmap v100: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:41:54.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:53 smithi083 ceph-mon[27907]: pgmap v101: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:41:54.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:53 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:41:54.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:53 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:41:54.173 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:53 smithi189 ceph-mon[31791]: pgmap v101: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:41:54.173 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:53 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:41:54.173 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:53 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:41:54.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:53 smithi138 ceph-mon[31441]: pgmap v101: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:41:54.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:53 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:41:54.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:53 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:41:54.855 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:41:54.854+0000 7f1c93400700 1 -- 172.21.15.189:0/1146817510 <== mgr.14152 v2:172.21.15.83:6800/2550826560 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f1c9c062f30 con 0x7f1c7c0a3bb0 2024-04-24T08:41:54.855 INFO:teuthology.orchestra.run.smithi189.stdout:Created osd(s) 2 on host 'smithi189' 2024-04-24T08:41:54.859 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:41:54.858+0000 7f1ca1c00700 1 -- 172.21.15.189:0/1146817510 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f1c7c0a3bb0 msgr2=0x7f1c7c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:41:54.859 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:41:54.858+0000 7f1ca1c00700 1 --2- 172.21.15.189:0/1146817510 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f1c7c0a3bb0 0x7f1c7c0a6070 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1c8c006fd0 tx=0x7f1c8c008040 comp rx=0 tx=0).stop 2024-04-24T08:41:54.859 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:41:54.858+0000 7f1ca1c00700 1 -- 172.21.15.189:0/1146817510 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1c9c10aa80 msgr2=0x7f1c9c1a25d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:41:54.859 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:41:54.858+0000 7f1ca1c00700 1 --2- 172.21.15.189:0/1146817510 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1c9c10aa80 0x7f1c9c1a25d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1c9400b7b0 tx=0x7f1c9400bb70 comp rx=0 tx=0).stop 2024-04-24T08:41:54.859 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:41:54.858+0000 7f1ca1c00700 1 -- 172.21.15.189:0/1146817510 shutdown_connections 2024-04-24T08:41:54.859 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:41:54.858+0000 7f1ca1c00700 1 --2- 172.21.15.189:0/1146817510 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f1c7c0a3bb0 0x7f1c7c0a6070 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:41:54.860 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:41:54.858+0000 7f1ca1c00700 1 --2- 172.21.15.189:0/1146817510 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1c9c10aa80 0x7f1c9c1a25d0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:41:54.860 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:41:54.858+0000 7f1ca1c00700 1 --2- 172.21.15.189:0/1146817510 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c9c0754d0 0x7f1c9c19b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:41:54.860 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:41:54.858+0000 7f1ca1c00700 1 --2- 172.21.15.189:0/1146817510 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c9c076aa0 0x7f1c9c19b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:41:54.860 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:41:54.858+0000 7f1ca1c00700 1 -- 172.21.15.189:0/1146817510 >> 172.21.15.189:0/1146817510 conn(0x7f1c9c0fe140 msgr2=0x7f1c9c1069d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:41:54.860 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:41:54.858+0000 7f1ca1c00700 1 -- 172.21.15.189:0/1146817510 shutdown_connections 2024-04-24T08:41:54.860 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:41:54.859+0000 7f1ca1c00700 1 -- 172.21.15.189:0/1146817510 wait complete. 2024-04-24T08:41:55.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:54 smithi083 ceph-mon[27907]: from='osd.2 [v2:172.21.15.189:6800/3169072688,v1:172.21.15.189:6801/3169072688]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-24T08:41:55.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:54 smithi083 ceph-mon[27907]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-24T08:41:55.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:54 smithi138 ceph-mon[31441]: from='osd.2 [v2:172.21.15.189:6800/3169072688,v1:172.21.15.189:6801/3169072688]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-24T08:41:55.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:54 smithi138 ceph-mon[31441]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-24T08:41:55.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:54 smithi189 ceph-mon[31791]: from='osd.2 [v2:172.21.15.189:6800/3169072688,v1:172.21.15.189:6801/3169072688]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-24T08:41:55.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:54 smithi189 ceph-mon[31791]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-24T08:41:55.666 DEBUG:teuthology.orchestra.run.smithi189:osd.2> sudo journalctl -f -n 0 -u ceph-bd3e039e-0215-11ef-bc93-c7b262605968@osd.2.service 2024-04-24T08:41:55.672 INFO:tasks.cephadm:Deploying osd.3 on smithi083 with /dev/vg_nvme/lv_3... 2024-04-24T08:41:55.672 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd3e039e-0215-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-24T08:41:55.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:55 smithi189 ceph-mon[31791]: pgmap v102: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:41:55.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:55 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:41:55.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:55 smithi189 ceph-mon[31791]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-24T08:41:55.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:55 smithi189 ceph-mon[31791]: osdmap e16: 3 total, 2 up, 3 in 2024-04-24T08:41:55.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:55 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:41:55.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:55 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T08:41:55.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:55 smithi189 ceph-mon[31791]: from='osd.2 [v2:172.21.15.189:6800/3169072688,v1:172.21.15.189:6801/3169072688]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-04-24T08:41:55.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:55 smithi189 ceph-mon[31791]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-04-24T08:41:55.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:55 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:41:55.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:55 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:41:55.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:55 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:41:55.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:55 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:41:55.958 INFO:journalctl@ceph.osd.2.smithi189.stdout:Apr 24 08:41:55 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-2[36951]: 2024-04-24T08:41:55.819+0000 7fd8bb200700 -1 osd.2 0 waiting for initial osdmap 2024-04-24T08:41:55.958 INFO:journalctl@ceph.osd.2.smithi189.stdout:Apr 24 08:41:55 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-2[36951]: 2024-04-24T08:41:55.832+0000 7fd8b6000700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T08:41:56.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:55 smithi083 ceph-mon[27907]: pgmap v102: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:41:56.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:55 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:41:56.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:55 smithi083 ceph-mon[27907]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-24T08:41:56.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:55 smithi083 ceph-mon[27907]: osdmap e16: 3 total, 2 up, 3 in 2024-04-24T08:41:56.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:55 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:41:56.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:55 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T08:41:56.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:55 smithi083 ceph-mon[27907]: from='osd.2 [v2:172.21.15.189:6800/3169072688,v1:172.21.15.189:6801/3169072688]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-04-24T08:41:56.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:55 smithi083 ceph-mon[27907]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-04-24T08:41:56.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:55 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:41:56.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:55 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:41:56.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:55 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:41:56.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:55 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:41:56.119 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T08:41:56.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:55 smithi138 ceph-mon[31441]: pgmap v102: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:41:56.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:55 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:41:56.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:55 smithi138 ceph-mon[31441]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-24T08:41:56.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:55 smithi138 ceph-mon[31441]: osdmap e16: 3 total, 2 up, 3 in 2024-04-24T08:41:56.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:55 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:41:56.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:55 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T08:41:56.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:55 smithi138 ceph-mon[31441]: from='osd.2 [v2:172.21.15.189:6800/3169072688,v1:172.21.15.189:6801/3169072688]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-04-24T08:41:56.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:55 smithi138 ceph-mon[31441]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-04-24T08:41:56.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:55 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:41:56.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:55 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:41:56.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:55 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:41:56.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:55 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:41:57.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:56 smithi083 ceph-mon[27907]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-04-24T08:41:57.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:56 smithi083 ceph-mon[27907]: osdmap e17: 3 total, 2 up, 3 in 2024-04-24T08:41:57.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:56 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T08:41:57.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:56 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T08:41:57.103 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:56 smithi189 ceph-mon[31791]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-04-24T08:41:57.103 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:56 smithi189 ceph-mon[31791]: osdmap e17: 3 total, 2 up, 3 in 2024-04-24T08:41:57.103 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:56 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T08:41:57.103 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:56 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T08:41:57.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:56 smithi138 ceph-mon[31441]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-04-24T08:41:57.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:56 smithi138 ceph-mon[31441]: osdmap e17: 3 total, 2 up, 3 in 2024-04-24T08:41:57.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:56 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T08:41:57.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:56 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T08:41:57.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:57 smithi189 ceph-mon[31791]: purged_snaps scrub starts 2024-04-24T08:41:57.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:57 smithi189 ceph-mon[31791]: purged_snaps scrub ok 2024-04-24T08:41:57.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:57 smithi189 ceph-mon[31791]: pgmap v105: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:41:57.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:57 smithi189 ceph-mon[31791]: osd.2 [v2:172.21.15.189:6800/3169072688,v1:172.21.15.189:6801/3169072688] boot 2024-04-24T08:41:57.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:57 smithi189 ceph-mon[31791]: osdmap e18: 3 total, 3 up, 3 in 2024-04-24T08:41:57.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:57 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T08:41:57.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:57 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-24T08:41:58.064 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:57 smithi083 ceph-mon[27907]: purged_snaps scrub starts 2024-04-24T08:41:58.064 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:57 smithi083 ceph-mon[27907]: purged_snaps scrub ok 2024-04-24T08:41:58.064 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:57 smithi083 ceph-mon[27907]: pgmap v105: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:41:58.064 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:57 smithi083 ceph-mon[27907]: osd.2 [v2:172.21.15.189:6800/3169072688,v1:172.21.15.189:6801/3169072688] boot 2024-04-24T08:41:58.064 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:57 smithi083 ceph-mon[27907]: osdmap e18: 3 total, 3 up, 3 in 2024-04-24T08:41:58.064 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:57 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T08:41:58.064 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:57 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-24T08:41:58.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:57 smithi138 ceph-mon[31441]: purged_snaps scrub starts 2024-04-24T08:41:58.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:57 smithi138 ceph-mon[31441]: purged_snaps scrub ok 2024-04-24T08:41:58.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:57 smithi138 ceph-mon[31441]: pgmap v105: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T08:41:58.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:57 smithi138 ceph-mon[31441]: osd.2 [v2:172.21.15.189:6800/3169072688,v1:172.21.15.189:6801/3169072688] boot 2024-04-24T08:41:58.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:57 smithi138 ceph-mon[31441]: osdmap e18: 3 total, 3 up, 3 in 2024-04-24T08:41:58.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:57 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T08:41:58.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:57 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-24T08:41:59.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:58 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-24T08:41:59.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:58 smithi083 ceph-mon[27907]: osdmap e19: 3 total, 3 up, 3 in 2024-04-24T08:41:59.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:58 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-24T08:41:59.062 INFO:teuthology.orchestra.run.smithi083.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-24T08:41:59.062 INFO:teuthology.orchestra.run.smithi083.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-24T08:41:59.062 INFO:teuthology.orchestra.run.smithi083.stderr: stderr: 10+0 records in 2024-04-24T08:41:59.062 INFO:teuthology.orchestra.run.smithi083.stderr:10+0 records out 2024-04-24T08:41:59.062 INFO:teuthology.orchestra.run.smithi083.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0379765 s, 276 MB/s 2024-04-24T08:41:59.062 INFO:teuthology.orchestra.run.smithi083.stderr:--> Zapping successful for: 2024-04-24T08:41:59.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:58 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-24T08:41:59.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:58 smithi138 ceph-mon[31441]: osdmap e19: 3 total, 3 up, 3 in 2024-04-24T08:41:59.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:58 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-24T08:41:59.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:58 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-24T08:41:59.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:58 smithi189 ceph-mon[31791]: osdmap e19: 3 total, 3 up, 3 in 2024-04-24T08:41:59.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:58 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-24T08:41:59.454 INFO:journalctl@ceph.osd.0.smithi083.stdout:Apr 24 08:41:59 smithi083 sudo[46754]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-24T08:41:59.454 INFO:journalctl@ceph.osd.0.smithi083.stdout:Apr 24 08:41:59 smithi083 sudo[46754]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T08:41:59.455 INFO:journalctl@ceph.osd.0.smithi083.stdout:Apr 24 08:41:59 smithi083 sudo[46754]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T08:41:59.455 INFO:journalctl@ceph.osd.0.smithi083.stdout:Apr 24 08:41:59 smithi083 sudo[46754]: pam_unix(sudo:session): session closed for user root 2024-04-24T08:41:59.455 INFO:journalctl@ceph.osd.0.smithi083.stdout:Apr 24 08:41:59 smithi083 sudo[46766]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-24T08:41:59.455 INFO:journalctl@ceph.osd.0.smithi083.stdout:Apr 24 08:41:59 smithi083 sudo[46766]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T08:41:59.455 INFO:journalctl@ceph.osd.0.smithi083.stdout:Apr 24 08:41:59 smithi083 sudo[46766]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T08:41:59.455 INFO:journalctl@ceph.osd.0.smithi083.stdout:Apr 24 08:41:59 smithi083 sudo[46766]: pam_unix(sudo:session): session closed for user root 2024-04-24T08:41:59.706 INFO:journalctl@ceph.osd.1.smithi138.stdout:Apr 24 08:41:59 smithi138 sudo[39779]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-24T08:41:59.706 INFO:journalctl@ceph.osd.1.smithi138.stdout:Apr 24 08:41:59 smithi138 sudo[39779]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T08:41:59.706 INFO:journalctl@ceph.osd.1.smithi138.stdout:Apr 24 08:41:59 smithi138 sudo[39779]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T08:41:59.706 INFO:journalctl@ceph.osd.1.smithi138.stdout:Apr 24 08:41:59 smithi138 sudo[39779]: pam_unix(sudo:session): session closed for user root 2024-04-24T08:41:59.706 INFO:journalctl@ceph.osd.1.smithi138.stdout:Apr 24 08:41:59 smithi138 sudo[39782]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-24T08:41:59.706 INFO:journalctl@ceph.osd.1.smithi138.stdout:Apr 24 08:41:59 smithi138 sudo[39782]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T08:41:59.706 INFO:journalctl@ceph.osd.1.smithi138.stdout:Apr 24 08:41:59 smithi138 sudo[39782]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T08:41:59.706 INFO:journalctl@ceph.osd.1.smithi138.stdout:Apr 24 08:41:59 smithi138 sudo[39782]: pam_unix(sudo:session): session closed for user root 2024-04-24T08:41:59.708 INFO:journalctl@ceph.osd.2.smithi189.stdout:Apr 24 08:41:59 smithi189 sudo[40349]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-24T08:41:59.708 INFO:journalctl@ceph.osd.2.smithi189.stdout:Apr 24 08:41:59 smithi189 sudo[40349]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T08:41:59.708 INFO:journalctl@ceph.osd.2.smithi189.stdout:Apr 24 08:41:59 smithi189 sudo[40349]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T08:41:59.708 INFO:journalctl@ceph.osd.2.smithi189.stdout:Apr 24 08:41:59 smithi189 sudo[40349]: pam_unix(sudo:session): session closed for user root 2024-04-24T08:41:59.708 INFO:journalctl@ceph.osd.2.smithi189.stdout:Apr 24 08:41:59 smithi189 sudo[40601]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-24T08:41:59.708 INFO:journalctl@ceph.osd.2.smithi189.stdout:Apr 24 08:41:59 smithi189 sudo[40601]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T08:41:59.708 INFO:journalctl@ceph.osd.2.smithi189.stdout:Apr 24 08:41:59 smithi189 sudo[40601]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T08:41:59.708 INFO:journalctl@ceph.osd.2.smithi189.stdout:Apr 24 08:41:59 smithi189 sudo[40601]: pam_unix(sudo:session): session closed for user root 2024-04-24T08:41:59.717 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:59 smithi083 sudo[46773]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-24T08:41:59.717 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:59 smithi083 sudo[46773]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T08:41:59.717 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:59 smithi083 sudo[46773]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T08:41:59.717 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:59 smithi083 sudo[46773]: pam_unix(sudo:session): session closed for user root 2024-04-24T08:41:59.717 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:59 smithi083 sudo[46776]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-24T08:41:59.717 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:59 smithi083 sudo[46776]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T08:41:59.717 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:59 smithi083 sudo[46776]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T08:41:59.717 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:59 smithi083 sudo[46776]: pam_unix(sudo:session): session closed for user root 2024-04-24T08:41:59.969 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd3e039e-0215-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi083:vg_nvme/lv_3 2024-04-24T08:42:00.042 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:59 smithi083 ceph-mon[27907]: pgmap v108: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-24T08:42:00.042 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:59 smithi083 ceph-mon[27907]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-24T08:42:00.042 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:59 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-24T08:42:00.043 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:59 smithi083 ceph-mon[27907]: osdmap e20: 3 total, 3 up, 3 in 2024-04-24T08:42:00.043 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:59 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:42:00.043 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:59 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:42:00.043 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:59 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:42:00.043 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:59 smithi083 ceph-mon[27907]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-24T08:42:00.043 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:59 smithi083 ceph-mon[27907]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-24T08:42:00.043 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:59 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T08:42:00.043 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:59 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T08:42:00.043 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:59 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T08:42:00.043 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:59 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T08:42:00.044 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:59 smithi083 ceph-mon[27907]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-24T08:42:00.044 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:59 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T08:42:00.044 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:59 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T08:42:00.044 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:59 smithi083 ceph-mon[27907]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-24T08:42:00.044 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:59 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T08:42:00.044 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:59 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T08:42:00.044 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:41:59 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T08:42:00.207 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:59 smithi138 sudo[39785]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-24T08:42:00.207 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:59 smithi138 sudo[39785]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T08:42:00.207 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:59 smithi138 sudo[39785]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T08:42:00.208 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:59 smithi138 sudo[39785]: pam_unix(sudo:session): session closed for user root 2024-04-24T08:42:00.208 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:59 smithi138 sudo[39788]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-24T08:42:00.208 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:59 smithi138 sudo[39788]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T08:42:00.208 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:59 smithi138 sudo[39788]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T08:42:00.208 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:59 smithi138 sudo[39788]: pam_unix(sudo:session): session closed for user root 2024-04-24T08:42:00.208 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:59 smithi138 ceph-mon[31441]: pgmap v108: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-24T08:42:00.208 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:59 smithi138 ceph-mon[31441]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-24T08:42:00.208 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:59 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-24T08:42:00.208 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:59 smithi138 ceph-mon[31441]: osdmap e20: 3 total, 3 up, 3 in 2024-04-24T08:42:00.208 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:59 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:42:00.208 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:59 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:42:00.208 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:59 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:42:00.208 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:59 smithi138 ceph-mon[31441]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-24T08:42:00.208 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:59 smithi138 ceph-mon[31441]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-24T08:42:00.209 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:59 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T08:42:00.209 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:59 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T08:42:00.209 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:59 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T08:42:00.209 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:59 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T08:42:00.209 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:59 smithi138 ceph-mon[31441]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-24T08:42:00.209 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:59 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T08:42:00.209 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:59 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T08:42:00.209 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:59 smithi138 ceph-mon[31441]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-24T08:42:00.209 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:59 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T08:42:00.209 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:59 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T08:42:00.209 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:41:59 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T08:42:00.210 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:59 smithi189 sudo[40616]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-24T08:42:00.210 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:59 smithi189 sudo[40616]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T08:42:00.210 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:59 smithi189 sudo[40616]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T08:42:00.210 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:59 smithi189 sudo[40616]: pam_unix(sudo:session): session closed for user root 2024-04-24T08:42:00.210 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:59 smithi189 sudo[40619]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-24T08:42:00.210 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:59 smithi189 sudo[40619]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T08:42:00.210 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:59 smithi189 sudo[40619]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T08:42:00.210 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:59 smithi189 sudo[40619]: pam_unix(sudo:session): session closed for user root 2024-04-24T08:42:00.210 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:59 smithi189 ceph-mon[31791]: pgmap v108: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-24T08:42:00.210 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:59 smithi189 ceph-mon[31791]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-24T08:42:00.210 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:59 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-24T08:42:00.211 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:59 smithi189 ceph-mon[31791]: osdmap e20: 3 total, 3 up, 3 in 2024-04-24T08:42:00.211 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:59 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:42:00.211 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:59 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:42:00.211 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:59 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:42:00.211 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:59 smithi189 ceph-mon[31791]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-24T08:42:00.211 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:59 smithi189 ceph-mon[31791]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-24T08:42:00.211 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:59 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T08:42:00.211 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:59 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T08:42:00.211 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:59 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T08:42:00.211 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:59 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T08:42:00.211 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:59 smithi189 ceph-mon[31791]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-24T08:42:00.211 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:59 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T08:42:00.211 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:59 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T08:42:00.212 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:59 smithi189 ceph-mon[31791]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-24T08:42:00.212 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:59 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T08:42:00.212 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:59 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T08:42:00.212 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:41:59 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T08:42:00.390 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T08:42:01.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:42:00 smithi138 ceph-mon[31441]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-24T08:42:01.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:42:00 smithi138 ceph-mon[31441]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-24T08:42:01.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:42:00 smithi138 ceph-mon[31441]: osdmap e21: 3 total, 3 up, 3 in 2024-04-24T08:42:01.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:42:00 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:42:01.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:42:00 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:42:01.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:42:00 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:42:01.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:42:00 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:42:01.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:42:00 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:42:01.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:42:00 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:42:01.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:42:00 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:42:01.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:42:00 smithi189 ceph-mon[31791]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-24T08:42:01.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:42:00 smithi189 ceph-mon[31791]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-24T08:42:01.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:42:00 smithi189 ceph-mon[31791]: osdmap e21: 3 total, 3 up, 3 in 2024-04-24T08:42:01.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:42:00 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:42:01.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:42:00 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:42:01.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:42:00 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:42:01.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:42:00 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:42:01.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:42:00 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:42:01.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:42:00 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:42:01.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:42:00 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:42:01.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:42:00 smithi083 ceph-mon[27907]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-24T08:42:01.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:42:00 smithi083 ceph-mon[27907]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-24T08:42:01.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:42:00 smithi083 ceph-mon[27907]: osdmap e21: 3 total, 3 up, 3 in 2024-04-24T08:42:01.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:42:00 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:42:01.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:42:00 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:42:01.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:42:00 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:42:01.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:42:00 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:42:01.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:42:00 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:42:01.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:42:00 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:42:01.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:42:00 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:42:02.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:42:01 smithi083 ceph-mon[27907]: Detected new or changed devices on smithi189 2024-04-24T08:42:02.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:42:01 smithi083 ceph-mon[27907]: Adjusting osd_memory_target on smithi189 to 17089M 2024-04-24T08:42:02.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:42:01 smithi083 ceph-mon[27907]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-24T08:42:02.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:42:01 smithi083 ceph-mon[27907]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-24T08:42:02.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:42:01 smithi083 ceph-mon[27907]: Cluster is now healthy 2024-04-24T08:42:02.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:42:01 smithi083 ceph-mon[27907]: mgrmap e16: x(active, since 3m), standbys: z, y 2024-04-24T08:42:02.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:42:01 smithi138 ceph-mon[31441]: Detected new or changed devices on smithi189 2024-04-24T08:42:02.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:42:01 smithi138 ceph-mon[31441]: Adjusting osd_memory_target on smithi189 to 17089M 2024-04-24T08:42:02.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:42:01 smithi138 ceph-mon[31441]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-24T08:42:02.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:42:01 smithi138 ceph-mon[31441]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-24T08:42:02.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:42:01 smithi138 ceph-mon[31441]: Cluster is now healthy 2024-04-24T08:42:02.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:42:01 smithi138 ceph-mon[31441]: mgrmap e16: x(active, since 3m), standbys: z, y 2024-04-24T08:42:02.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:42:01 smithi189 ceph-mon[31791]: Detected new or changed devices on smithi189 2024-04-24T08:42:02.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:42:01 smithi189 ceph-mon[31791]: Adjusting osd_memory_target on smithi189 to 17089M 2024-04-24T08:42:02.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:42:01 smithi189 ceph-mon[31791]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-24T08:42:02.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:42:01 smithi189 ceph-mon[31791]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-24T08:42:02.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:42:01 smithi189 ceph-mon[31791]: Cluster is now healthy 2024-04-24T08:42:02.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:42:01 smithi189 ceph-mon[31791]: mgrmap e16: x(active, since 3m), standbys: z, y 2024-04-24T08:42:02.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:42:02.653+0000 7f136d200700 1 -- 172.21.15.83:0/4293867109 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f136810ad30 msgr2=0x7f13681115e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:42:02.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:42:02.653+0000 7f136d200700 1 --2- 172.21.15.83:0/4293867109 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f136810ad30 0x7f13681115e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f136000b0d0 tx=0x7f136000b3e0 comp rx=0 tx=0).stop 2024-04-24T08:42:02.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:42:02.653+0000 7f136d200700 1 -- 172.21.15.83:0/4293867109 shutdown_connections 2024-04-24T08:42:02.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:42:02.653+0000 7f136d200700 1 --2- 172.21.15.83:0/4293867109 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f136810ad30 0x7f13681115e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:42:02.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:42:02.653+0000 7f136d200700 1 --2- 172.21.15.83:0/4293867109 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1368109190 0x7f13681095b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:42:02.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:42:02.653+0000 7f136d200700 1 --2- 172.21.15.83:0/4293867109 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f136810a390 0x7f136810a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:42:02.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:42:02.653+0000 7f136d200700 1 -- 172.21.15.83:0/4293867109 >> 172.21.15.83:0/4293867109 conn(0x7f1368078160 msgr2=0x7f136807a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:42:02.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:42:02.653+0000 7f136d200700 1 -- 172.21.15.83:0/4293867109 shutdown_connections 2024-04-24T08:42:02.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:42:02.653+0000 7f136d200700 1 -- 172.21.15.83:0/4293867109 wait complete. 2024-04-24T08:42:02.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:42:02.654+0000 7f136d200700 1 Processor -- start 2024-04-24T08:42:02.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:42:02.655+0000 7f136d200700 1 -- start start 2024-04-24T08:42:02.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:42:02.655+0000 7f136d200700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1368109190 0x7f136819f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:42:02.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:42:02.655+0000 7f136d200700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f136810a390 0x7f136819f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:42:02.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:42:02.655+0000 7f136d200700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f136810ad30 0x7f13681a6800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:42:02.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:42:02.655+0000 7f136d200700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1368114c90 con 0x7f136810a390 2024-04-24T08:42:02.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:42:02.655+0000 7f136d200700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1368114b10 con 0x7f1368109190 2024-04-24T08:42:02.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:42:02.655+0000 7f136d200700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1368114e10 con 0x7f136810ad30 2024-04-24T08:42:02.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:42:02.655+0000 7f1365400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f136810a390 0x7f136819f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:42:02.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:42:02.655+0000 7f1365400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f136810a390 0x7f136819f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43222/0 (socket says 172.21.15.83:43222) 2024-04-24T08:42:02.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:42:02.655+0000 7f1365400700 1 -- 172.21.15.83:0/2200142148 learned_addr learned my addr 172.21.15.83:0/2200142148 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:42:02.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:42:02.656+0000 7f1365e00700 1 --2- 172.21.15.83:0/2200142148 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1368109190 0x7f136819f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:42:02.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:42:02.656+0000 7f1365400700 1 -- 172.21.15.83:0/2200142148 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f136810ad30 msgr2=0x7f13681a6800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:42:02.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:42:02.656+0000 7f1366800700 1 --2- 172.21.15.83:0/2200142148 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f136810ad30 0x7f13681a6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:42:02.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:42:02.656+0000 7f1365400700 1 --2- 172.21.15.83:0/2200142148 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f136810ad30 0x7f13681a6800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:42:02.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:42:02.656+0000 7f1365400700 1 -- 172.21.15.83:0/2200142148 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1368109190 msgr2=0x7f136819f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:42:02.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:42:02.656+0000 7f1365400700 1 --2- 172.21.15.83:0/2200142148 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1368109190 0x7f136819f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:42:02.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:42:02.656+0000 7f1365400700 1 -- 172.21.15.83:0/2200142148 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13681a6ea0 con 0x7f136810a390 2024-04-24T08:42:02.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:42:02.656+0000 7f1365e00700 1 --2- 172.21.15.83:0/2200142148 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1368109190 0x7f136819f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:42:02.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:42:02.656+0000 7f1365400700 1 --2- 172.21.15.83:0/2200142148 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f136810a390 0x7f136819f780 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f135400d980 tx=0x7f135400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:42:02.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:42:02.656+0000 7f135ea00700 1 -- 172.21.15.83:0/2200142148 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1354016070 con 0x7f136810a390 2024-04-24T08:42:02.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:42:02.657+0000 7f135ea00700 1 -- 172.21.15.83:0/2200142148 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1354010460 con 0x7f136810a390 2024-04-24T08:42:02.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:42:02.657+0000 7f136d200700 1 -- 172.21.15.83:0/2200142148 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13681a7190 con 0x7f136810a390 2024-04-24T08:42:02.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:42:02.657+0000 7f135ea00700 1 -- 172.21.15.83:0/2200142148 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f135400f620 con 0x7f136810a390 2024-04-24T08:42:02.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:42:02.657+0000 7f136d200700 1 -- 172.21.15.83:0/2200142148 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13681a76d0 con 0x7f136810a390 2024-04-24T08:42:02.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:42:02.658+0000 7f136d200700 1 -- 172.21.15.83:0/2200142148 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13680697e0 con 0x7f136810a390 2024-04-24T08:42:02.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:42:02.660+0000 7f135ea00700 1 -- 172.21.15.83:0/2200142148 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f1354019070 con 0x7f136810a390 2024-04-24T08:42:02.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:42:02.662+0000 7f135ea00700 1 --2- 172.21.15.83:0/2200142148 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f134c0a3c40 0x7f134c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:42:02.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:42:02.662+0000 7f135ea00700 1 -- 172.21.15.83:0/2200142148 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 3237+0+0 (secure 0 0 0) 0x7f13540cecb0 con 0x7f136810a390 2024-04-24T08:42:02.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:42:02.662+0000 7f1365e00700 1 --2- 172.21.15.83:0/2200142148 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f134c0a3c40 0x7f134c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:42:02.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:42:02.663+0000 7f1365e00700 1 --2- 172.21.15.83:0/2200142148 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f134c0a3c40 0x7f134c0a6100 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1358005fd0 tx=0x7f1358005f00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:42:02.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:42:02.668+0000 7f135ea00700 1 -- 172.21.15.83:0/2200142148 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f135409c9c0 con 0x7f136810a390 2024-04-24T08:42:02.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:42:02.904+0000 7f136d200700 1 -- 172.21.15.83:0/2200142148 --> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f1368062ff0 con 0x7f134c0a3c40 2024-04-24T08:42:04.091 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:42:03 smithi083 ceph-mon[27907]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-04-24T08:42:04.091 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:42:03 smithi083 ceph-mon[27907]: from='client.14310 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:42:04.091 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:42:03 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T08:42:04.091 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:42:03 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T08:42:04.091 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:42:03 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:42:04.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:42:03 smithi138 ceph-mon[31441]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-04-24T08:42:04.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:42:03 smithi138 ceph-mon[31441]: from='client.14310 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:42:04.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:42:03 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T08:42:04.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:42:03 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T08:42:04.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:42:03 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:42:04.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:42:03 smithi189 ceph-mon[31791]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-04-24T08:42:04.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:42:03 smithi189 ceph-mon[31791]: from='client.14310 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:42:04.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:42:03 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T08:42:04.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:42:03 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T08:42:04.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:42:03 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:42:06.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:42:05 smithi083 ceph-mon[27907]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 121 KiB/s wr, 7 op/s 2024-04-24T08:42:06.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:42:05 smithi138 ceph-mon[31441]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 121 KiB/s wr, 7 op/s 2024-04-24T08:42:06.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:42:05 smithi189 ceph-mon[31791]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 121 KiB/s wr, 7 op/s 2024-04-24T08:42:08.174 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:42:07 smithi189 ceph-mon[31791]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 86 KiB/s wr, 5 op/s 2024-04-24T08:42:08.174 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:42:07 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/1006902724' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f5624d78-4455-4e07-8c1a-f47c94cddb23"}]: dispatch 2024-04-24T08:42:08.174 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:42:07 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/1006902724' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f5624d78-4455-4e07-8c1a-f47c94cddb23"}]': finished 2024-04-24T08:42:08.174 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:42:07 smithi189 ceph-mon[31791]: osdmap e22: 4 total, 3 up, 4 in 2024-04-24T08:42:08.174 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:42:07 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T08:42:08.174 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:42:07 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/1991935141' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T08:42:08.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:42:07 smithi138 ceph-mon[31441]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 86 KiB/s wr, 5 op/s 2024-04-24T08:42:08.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:42:07 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/1006902724' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f5624d78-4455-4e07-8c1a-f47c94cddb23"}]: dispatch 2024-04-24T08:42:08.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:42:07 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/1006902724' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f5624d78-4455-4e07-8c1a-f47c94cddb23"}]': finished 2024-04-24T08:42:08.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:42:07 smithi138 ceph-mon[31441]: osdmap e22: 4 total, 3 up, 4 in 2024-04-24T08:42:08.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:42:07 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T08:42:08.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:42:07 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/1991935141' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T08:42:08.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:42:07 smithi083 ceph-mon[27907]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 86 KiB/s wr, 5 op/s 2024-04-24T08:42:08.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:42:07 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/1006902724' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f5624d78-4455-4e07-8c1a-f47c94cddb23"}]: dispatch 2024-04-24T08:42:08.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:42:07 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/1006902724' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f5624d78-4455-4e07-8c1a-f47c94cddb23"}]': finished 2024-04-24T08:42:08.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:42:07 smithi083 ceph-mon[27907]: osdmap e22: 4 total, 3 up, 4 in 2024-04-24T08:42:08.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:42:07 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T08:42:08.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:42:07 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/1991935141' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T08:42:10.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:42:09 smithi138 ceph-mon[31441]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail; 1.4 KiB/s rd, 66 KiB/s wr, 4 op/s 2024-04-24T08:42:10.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:42:09 smithi189 ceph-mon[31791]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail; 1.4 KiB/s rd, 66 KiB/s wr, 4 op/s 2024-04-24T08:42:10.212 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:42:09 smithi083 ceph-mon[27907]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail; 1.4 KiB/s rd, 66 KiB/s wr, 4 op/s 2024-04-24T08:42:12.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:42:11 smithi138 ceph-mon[31441]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 58 KiB/s wr, 3 op/s 2024-04-24T08:42:12.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:42:11 smithi189 ceph-mon[31791]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 58 KiB/s wr, 3 op/s 2024-04-24T08:42:12.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:42:11 smithi083 ceph-mon[27907]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 58 KiB/s wr, 3 op/s 2024-04-24T08:42:14.183 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:42:13 smithi083 ceph-mon[27907]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 58 KiB/s wr, 3 op/s 2024-04-24T08:42:14.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:42:13 smithi138 ceph-mon[31441]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 58 KiB/s wr, 3 op/s 2024-04-24T08:42:14.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:42:13 smithi189 ceph-mon[31791]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 58 KiB/s wr, 3 op/s 2024-04-24T08:42:16.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:42:15 smithi138 ceph-mon[31441]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-24T08:42:16.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:42:15 smithi189 ceph-mon[31791]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-24T08:42:16.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:42:15 smithi083 ceph-mon[27907]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-24T08:42:17.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:42:16 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-24T08:42:17.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:42:16 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:42:17.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:42:16 smithi138 ceph-mon[31441]: Deploying daemon osd.3 on smithi083 2024-04-24T08:42:17.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:42:16 smithi138 ceph-mon[31441]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-24T08:42:17.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:42:16 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-24T08:42:17.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:42:16 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:42:17.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:42:16 smithi189 ceph-mon[31791]: Deploying daemon osd.3 on smithi083 2024-04-24T08:42:17.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:42:16 smithi189 ceph-mon[31791]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-24T08:42:17.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:42:16 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-24T08:42:17.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:42:16 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:42:17.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:42:16 smithi083 ceph-mon[27907]: Deploying daemon osd.3 on smithi083 2024-04-24T08:42:17.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:42:16 smithi083 ceph-mon[27907]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-24T08:42:20.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:42:19 smithi083 ceph-mon[27907]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-24T08:42:20.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:42:19 smithi138 ceph-mon[31441]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-24T08:42:20.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:42:19 smithi189 ceph-mon[31791]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-24T08:42:22.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:42:21 smithi083 ceph-mon[27907]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-24T08:42:22.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:42:21 smithi138 ceph-mon[31441]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-24T08:42:22.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:42:21 smithi189 ceph-mon[31791]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-24T08:42:24.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:42:23 smithi083 ceph-mon[27907]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-24T08:42:24.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:42:23 smithi138 ceph-mon[31441]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-24T08:42:24.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:42:23 smithi189 ceph-mon[31791]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-24T08:42:26.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:42:25 smithi083 ceph-mon[27907]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-24T08:42:26.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:42:25 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:42:26.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:42:25 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:42:26.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:42:25 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:42:26.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:42:25 smithi138 ceph-mon[31441]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-24T08:42:26.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:42:25 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:42:26.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:42:25 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:42:26.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:42:25 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:42:26.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:42:25 smithi189 ceph-mon[31791]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-24T08:42:26.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:42:25 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:42:26.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:42:25 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:42:26.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:42:25 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:42:27.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:42:27 smithi189 ceph-mon[31791]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-24T08:42:28.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:42:27 smithi083 ceph-mon[27907]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-24T08:42:28.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:42:27 smithi138 ceph-mon[31441]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-24T08:42:30.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:42:29 smithi083 ceph-mon[27907]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-24T08:42:30.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:42:29 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:42:30.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:42:29 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:42:30.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:42:29 smithi083 ceph-mon[27907]: from='osd.3 [v2:172.21.15.83:6810/3131092450,v1:172.21.15.83:6811/3131092450]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-24T08:42:30.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:42:29 smithi138 ceph-mon[31441]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-24T08:42:30.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:42:29 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:42:30.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:42:29 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:42:30.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:42:29 smithi138 ceph-mon[31441]: from='osd.3 [v2:172.21.15.83:6810/3131092450,v1:172.21.15.83:6811/3131092450]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-24T08:42:30.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:42:29 smithi189 ceph-mon[31791]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-24T08:42:30.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:42:29 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:42:30.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:42:29 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:42:30.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:42:29 smithi189 ceph-mon[31791]: from='osd.3 [v2:172.21.15.83:6810/3131092450,v1:172.21.15.83:6811/3131092450]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-24T08:42:30.658 INFO:teuthology.orchestra.run.smithi083.stdout:Created osd(s) 3 on host 'smithi083' 2024-04-24T08:42:30.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:42:30.656+0000 7f135ea00700 1 -- 172.21.15.83:0/2200142148 <== mgr.14152 v2:172.21.15.83:6800/2550826560 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f1368062ff0 con 0x7f134c0a3c40 2024-04-24T08:42:30.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:42:30.659+0000 7f136d200700 1 -- 172.21.15.83:0/2200142148 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f134c0a3c40 msgr2=0x7f134c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:42:30.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:42:30.659+0000 7f136d200700 1 --2- 172.21.15.83:0/2200142148 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f134c0a3c40 0x7f134c0a6100 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1358005fd0 tx=0x7f1358005f00 comp rx=0 tx=0).stop 2024-04-24T08:42:30.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:42:30.659+0000 7f136d200700 1 -- 172.21.15.83:0/2200142148 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f136810a390 msgr2=0x7f136819f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:42:30.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:42:30.659+0000 7f136d200700 1 --2- 172.21.15.83:0/2200142148 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f136810a390 0x7f136819f780 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f135400d980 tx=0x7f135400dd40 comp rx=0 tx=0).stop 2024-04-24T08:42:30.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:42:30.660+0000 7f136d200700 1 -- 172.21.15.83:0/2200142148 shutdown_connections 2024-04-24T08:42:30.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:42:30.660+0000 7f136d200700 1 --2- 172.21.15.83:0/2200142148 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f134c0a3c40 0x7f134c0a6100 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:42:30.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:42:30.660+0000 7f136d200700 1 --2- 172.21.15.83:0/2200142148 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f136810ad30 0x7f13681a6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:42:30.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:42:30.660+0000 7f136d200700 1 --2- 172.21.15.83:0/2200142148 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1368109190 0x7f136819f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:42:30.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:42:30.660+0000 7f136d200700 1 --2- 172.21.15.83:0/2200142148 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f136810a390 0x7f136819f780 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:42:30.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:42:30.660+0000 7f136d200700 1 -- 172.21.15.83:0/2200142148 >> 172.21.15.83:0/2200142148 conn(0x7f1368078160 msgr2=0x7f136810d630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:42:30.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:42:30.660+0000 7f136d200700 1 -- 172.21.15.83:0/2200142148 shutdown_connections 2024-04-24T08:42:30.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:42:30.660+0000 7f136d200700 1 -- 172.21.15.83:0/2200142148 wait complete. 2024-04-24T08:42:31.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:42:31 smithi138 ceph-mon[31441]: from='osd.3 [v2:172.21.15.83:6810/3131092450,v1:172.21.15.83:6811/3131092450]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-24T08:42:31.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:42:31 smithi138 ceph-mon[31441]: osdmap e23: 4 total, 3 up, 4 in 2024-04-24T08:42:31.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:42:31 smithi138 ceph-mon[31441]: from='osd.3 [v2:172.21.15.83:6810/3131092450,v1:172.21.15.83:6811/3131092450]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-24T08:42:31.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:42:31 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T08:42:31.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:42:31 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:42:31.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:42:31 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:42:31.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:42:31 smithi138 ceph-mon[31441]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-24T08:42:31.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:42:31 smithi189 ceph-mon[31791]: from='osd.3 [v2:172.21.15.83:6810/3131092450,v1:172.21.15.83:6811/3131092450]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-24T08:42:31.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:42:31 smithi189 ceph-mon[31791]: osdmap e23: 4 total, 3 up, 4 in 2024-04-24T08:42:31.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:42:31 smithi189 ceph-mon[31791]: from='osd.3 [v2:172.21.15.83:6810/3131092450,v1:172.21.15.83:6811/3131092450]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-24T08:42:31.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:42:31 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T08:42:31.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:42:31 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:42:31.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:42:31 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:42:31.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:42:31 smithi189 ceph-mon[31791]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-24T08:42:31.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:42:31 smithi083 ceph-mon[27907]: from='osd.3 [v2:172.21.15.83:6810/3131092450,v1:172.21.15.83:6811/3131092450]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-24T08:42:31.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:42:31 smithi083 ceph-mon[27907]: osdmap e23: 4 total, 3 up, 4 in 2024-04-24T08:42:31.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:42:31 smithi083 ceph-mon[27907]: from='osd.3 [v2:172.21.15.83:6810/3131092450,v1:172.21.15.83:6811/3131092450]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-24T08:42:31.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:42:31 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T08:42:31.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:42:31 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:42:31.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:42:31 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:42:31.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:42:31 smithi083 ceph-mon[27907]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-24T08:42:31.606 DEBUG:teuthology.orchestra.run.smithi083:osd.3> sudo journalctl -f -n 0 -u ceph-bd3e039e-0215-11ef-bc93-c7b262605968@osd.3.service 2024-04-24T08:42:31.612 INFO:tasks.cephadm:Deploying osd.4 on smithi138 with /dev/vg_nvme/lv_3... 2024-04-24T08:42:31.612 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd3e039e-0215-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-24T08:42:32.041 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T08:42:32.089 INFO:journalctl@ceph.osd.3.smithi083.stdout:Apr 24 08:42:32 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-3[49218]: 2024-04-24T08:42:32.018+0000 7fbc99800700 -1 osd.3 0 waiting for initial osdmap 2024-04-24T08:42:32.089 INFO:journalctl@ceph.osd.3.smithi083.stdout:Apr 24 08:42:32 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-3[49218]: 2024-04-24T08:42:32.047+0000 7fbc8f600700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T08:42:32.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:42:32 smithi138 ceph-mon[31441]: from='osd.3 [v2:172.21.15.83:6810/3131092450,v1:172.21.15.83:6811/3131092450]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-04-24T08:42:32.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:42:32 smithi138 ceph-mon[31441]: osdmap e24: 4 total, 3 up, 4 in 2024-04-24T08:42:32.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:42:32 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T08:42:32.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:42:32 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T08:42:32.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:42:32 smithi138 ceph-mon[31441]: from='osd.3 [v2:172.21.15.83:6810/3131092450,v1:172.21.15.83:6811/3131092450]' entity='osd.3' 2024-04-24T08:42:32.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:42:32 smithi189 ceph-mon[31791]: from='osd.3 [v2:172.21.15.83:6810/3131092450,v1:172.21.15.83:6811/3131092450]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-04-24T08:42:32.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:42:32 smithi189 ceph-mon[31791]: osdmap e24: 4 total, 3 up, 4 in 2024-04-24T08:42:32.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:42:32 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T08:42:32.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:42:32 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T08:42:32.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:42:32 smithi189 ceph-mon[31791]: from='osd.3 [v2:172.21.15.83:6810/3131092450,v1:172.21.15.83:6811/3131092450]' entity='osd.3' 2024-04-24T08:42:32.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:42:32 smithi083 ceph-mon[27907]: from='osd.3 [v2:172.21.15.83:6810/3131092450,v1:172.21.15.83:6811/3131092450]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-04-24T08:42:32.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:42:32 smithi083 ceph-mon[27907]: osdmap e24: 4 total, 3 up, 4 in 2024-04-24T08:42:32.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:42:32 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T08:42:32.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:42:32 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T08:42:32.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:42:32 smithi083 ceph-mon[27907]: from='osd.3 [v2:172.21.15.83:6810/3131092450,v1:172.21.15.83:6811/3131092450]' entity='osd.3' 2024-04-24T08:42:33.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:42:33 smithi138 ceph-mon[31441]: purged_snaps scrub starts 2024-04-24T08:42:33.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:42:33 smithi138 ceph-mon[31441]: purged_snaps scrub ok 2024-04-24T08:42:33.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:42:33 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T08:42:33.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:42:33 smithi138 ceph-mon[31441]: osd.3 [v2:172.21.15.83:6810/3131092450,v1:172.21.15.83:6811/3131092450] boot 2024-04-24T08:42:33.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:42:33 smithi138 ceph-mon[31441]: osdmap e25: 4 total, 4 up, 4 in 2024-04-24T08:42:33.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:42:33 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T08:42:33.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:42:33 smithi138 ceph-mon[31441]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-24T08:42:33.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:42:33 smithi083 ceph-mon[27907]: purged_snaps scrub starts 2024-04-24T08:42:33.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:42:33 smithi083 ceph-mon[27907]: purged_snaps scrub ok 2024-04-24T08:42:33.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:42:33 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T08:42:33.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:42:33 smithi083 ceph-mon[27907]: osd.3 [v2:172.21.15.83:6810/3131092450,v1:172.21.15.83:6811/3131092450] boot 2024-04-24T08:42:33.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:42:33 smithi083 ceph-mon[27907]: osdmap e25: 4 total, 4 up, 4 in 2024-04-24T08:42:33.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:42:33 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T08:42:33.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:42:33 smithi083 ceph-mon[27907]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-24T08:42:33.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:42:33 smithi189 ceph-mon[31791]: purged_snaps scrub starts 2024-04-24T08:42:33.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:42:33 smithi189 ceph-mon[31791]: purged_snaps scrub ok 2024-04-24T08:42:33.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:42:33 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T08:42:33.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:42:33 smithi189 ceph-mon[31791]: osd.3 [v2:172.21.15.83:6810/3131092450,v1:172.21.15.83:6811/3131092450] boot 2024-04-24T08:42:33.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:42:33 smithi189 ceph-mon[31791]: osdmap e25: 4 total, 4 up, 4 in 2024-04-24T08:42:33.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:42:33 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T08:42:33.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:42:33 smithi189 ceph-mon[31791]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-24T08:42:34.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:42:34 smithi138 ceph-mon[31441]: osdmap e26: 4 total, 4 up, 4 in 2024-04-24T08:42:34.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:42:34 smithi083 ceph-mon[27907]: osdmap e26: 4 total, 4 up, 4 in 2024-04-24T08:42:34.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:42:34 smithi189 ceph-mon[31791]: osdmap e26: 4 total, 4 up, 4 in 2024-04-24T08:42:35.019 INFO:teuthology.orchestra.run.smithi138.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-24T08:42:35.019 INFO:teuthology.orchestra.run.smithi138.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-24T08:42:35.019 INFO:teuthology.orchestra.run.smithi138.stderr: stderr: 10+0 records in 2024-04-24T08:42:35.019 INFO:teuthology.orchestra.run.smithi138.stderr:10+0 records out 2024-04-24T08:42:35.020 INFO:teuthology.orchestra.run.smithi138.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0368703 s, 284 MB/s 2024-04-24T08:42:35.020 INFO:teuthology.orchestra.run.smithi138.stderr:--> Zapping successful for: 2024-04-24T08:42:35.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:42:35 smithi083 ceph-mon[27907]: osdmap e27: 4 total, 4 up, 4 in 2024-04-24T08:42:35.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:42:35 smithi083 ceph-mon[27907]: pgmap v134: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-24T08:42:35.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:42:35 smithi138 ceph-mon[31441]: osdmap e27: 4 total, 4 up, 4 in 2024-04-24T08:42:35.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:42:35 smithi138 ceph-mon[31441]: pgmap v134: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-24T08:42:35.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:42:35 smithi189 ceph-mon[31791]: osdmap e27: 4 total, 4 up, 4 in 2024-04-24T08:42:35.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:42:35 smithi189 ceph-mon[31791]: pgmap v134: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-24T08:42:35.767 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd3e039e-0215-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi138:vg_nvme/lv_3 2024-04-24T08:42:36.179 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T08:42:36.533 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:42:36 smithi138 ceph-mon[31441]: osdmap e28: 4 total, 4 up, 4 in 2024-04-24T08:42:36.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:42:36 smithi083 ceph-mon[27907]: osdmap e28: 4 total, 4 up, 4 in 2024-04-24T08:42:36.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:42:36 smithi189 ceph-mon[31791]: osdmap e28: 4 total, 4 up, 4 in 2024-04-24T08:42:37.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:42:37 smithi138 ceph-mon[31441]: osdmap e29: 4 total, 4 up, 4 in 2024-04-24T08:42:37.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:42:37 smithi138 ceph-mon[31441]: pgmap v137: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-24T08:42:37.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:42:37 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:42:37.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:42:37 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:42:37.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:42:37 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:42:37.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:42:37 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:42:37.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:42:37 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:42:37.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:42:37 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:42:37.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:42:37 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:42:37.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:42:37 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:42:37.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:42:37 smithi189 ceph-mon[31791]: osdmap e29: 4 total, 4 up, 4 in 2024-04-24T08:42:37.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:42:37 smithi189 ceph-mon[31791]: pgmap v137: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-24T08:42:37.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:42:37 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:42:37.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:42:37 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:42:37.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:42:37 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:42:37.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:42:37 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:42:37.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:42:37 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:42:37.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:42:37 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:42:37.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:42:37 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:42:37.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:42:37 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:42:37.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:42:37 smithi083 ceph-mon[27907]: osdmap e29: 4 total, 4 up, 4 in 2024-04-24T08:42:37.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:42:37 smithi083 ceph-mon[27907]: pgmap v137: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-24T08:42:37.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:42:37 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:42:37.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:42:37 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:42:37.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:42:37 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:42:37.555 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:42:37 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:42:37.555 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:42:37 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:42:37.555 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:42:37 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:42:37.555 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:42:37 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:42:37.555 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:42:37 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:42:38.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:42:38.432+0000 7fb61aa00700 1 -- 172.21.15.138:0/1206510759 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb614100d30 msgr2=0x7fb614101150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:42:38.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:42:38.432+0000 7fb61aa00700 1 --2- 172.21.15.138:0/1206510759 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb614100d30 0x7fb614101150 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb608009990 tx=0x7fb608009ca0 comp rx=0 tx=0).stop 2024-04-24T08:42:38.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:42:38.433+0000 7fb61aa00700 1 -- 172.21.15.138:0/1206510759 shutdown_connections 2024-04-24T08:42:38.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:42:38.433+0000 7fb61aa00700 1 --2- 172.21.15.138:0/1206510759 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6141046b0 0x7fb61410af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:42:38.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:42:38.433+0000 7fb61aa00700 1 --2- 172.21.15.138:0/1206510759 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb614100d30 0x7fb614101150 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:42:38.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:42:38.433+0000 7fb61aa00700 1 --2- 172.21.15.138:0/1206510759 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb614103d10 0x7fb614104170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:42:38.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:42:38.433+0000 7fb61aa00700 1 -- 172.21.15.138:0/1206510759 >> 172.21.15.138:0/1206510759 conn(0x7fb6140fc690 msgr2=0x7fb6140fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:42:38.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:42:38.433+0000 7fb61aa00700 1 -- 172.21.15.138:0/1206510759 shutdown_connections 2024-04-24T08:42:38.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:42:38.433+0000 7fb61aa00700 1 -- 172.21.15.138:0/1206510759 wait complete. 2024-04-24T08:42:38.434 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:42:38 smithi138 ceph-mon[31441]: Detected new or changed devices on smithi083 2024-04-24T08:42:38.434 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:42:38 smithi138 ceph-mon[31441]: Adjusting osd_memory_target on smithi083 to 8544M 2024-04-24T08:42:38.434 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:42:38 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:42:38.435 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:42:38 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:42:38.435 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:42:38 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:42:38.435 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:42:38 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:42:38.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:42:38.435+0000 7fb61aa00700 1 Processor -- start 2024-04-24T08:42:38.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:42:38.435+0000 7fb61aa00700 1 -- start start 2024-04-24T08:42:38.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:42:38.435+0000 7fb61aa00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb614100d30 0x7fb614103670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:42:38.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:42:38.435+0000 7fb61aa00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb614103d10 0x7fb614101cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:42:38.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:42:38.435+0000 7fb61aa00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6141046b0 0x7fb614102200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:42:38.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:42:38.436+0000 7fb61aa00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb61410e4f0 con 0x7fb614100d30 2024-04-24T08:42:38.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:42:38.436+0000 7fb61aa00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb61410e370 con 0x7fb6141046b0 2024-04-24T08:42:38.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:42:38.436+0000 7fb61aa00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb61410e670 con 0x7fb614103d10 2024-04-24T08:42:38.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:42:38.436+0000 7fb61a000700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6141046b0 0x7fb614102200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:42:38.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:42:38.436+0000 7fb61a000700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6141046b0 0x7fb614102200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:37116/0 (socket says 172.21.15.138:37116) 2024-04-24T08:42:38.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:42:38.436+0000 7fb61a000700 1 -- 172.21.15.138:0/257805144 learned_addr learned my addr 172.21.15.138:0/257805144 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-24T08:42:38.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:42:38.436+0000 7fb619600700 1 --2- 172.21.15.138:0/257805144 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb614100d30 0x7fb614103670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:42:38.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:42:38.436+0000 7fb618c00700 1 --2- 172.21.15.138:0/257805144 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb614103d10 0x7fb614101cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:42:38.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:42:38.436+0000 7fb61a000700 1 -- 172.21.15.138:0/257805144 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb614103d10 msgr2=0x7fb614101cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:42:38.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:42:38.436+0000 7fb61a000700 1 --2- 172.21.15.138:0/257805144 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb614103d10 0x7fb614101cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:42:38.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:42:38.436+0000 7fb61a000700 1 -- 172.21.15.138:0/257805144 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb614100d30 msgr2=0x7fb614103670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:42:38.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:42:38.436+0000 7fb61a000700 1 --2- 172.21.15.138:0/257805144 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb614100d30 0x7fb614103670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:42:38.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:42:38.436+0000 7fb61a000700 1 -- 172.21.15.138:0/257805144 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb614102ab0 con 0x7fb6141046b0 2024-04-24T08:42:38.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:42:38.437+0000 7fb61a000700 1 --2- 172.21.15.138:0/257805144 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6141046b0 0x7fb614102200 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb60c00ef30 tx=0x7fb60c00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:42:38.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:42:38.437+0000 7fb612000700 1 -- 172.21.15.138:0/257805144 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb60c0092e0 con 0x7fb6141046b0 2024-04-24T08:42:38.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:42:38.437+0000 7fb612000700 1 -- 172.21.15.138:0/257805144 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb60c004bb0 con 0x7fb6141046b0 2024-04-24T08:42:38.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:42:38.437+0000 7fb612000700 1 -- 172.21.15.138:0/257805144 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb60c004060 con 0x7fb6141046b0 2024-04-24T08:42:38.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:42:38.437+0000 7fb61aa00700 1 -- 172.21.15.138:0/257805144 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6141a2ca0 con 0x7fb6141046b0 2024-04-24T08:42:38.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:42:38.437+0000 7fb61aa00700 1 -- 172.21.15.138:0/257805144 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6141a31b0 con 0x7fb6141046b0 2024-04-24T08:42:38.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:42:38.439+0000 7fb61aa00700 1 -- 172.21.15.138:0/257805144 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6140697e0 con 0x7fb6141046b0 2024-04-24T08:42:38.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:42:38.441+0000 7fb612000700 1 -- 172.21.15.138:0/257805144 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb60c01b030 con 0x7fb6141046b0 2024-04-24T08:42:38.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:42:38.442+0000 7fb612000700 1 --2- 172.21.15.138:0/257805144 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fb5fc0a3cc0 0x7fb5fc0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:42:38.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:42:38.442+0000 7fb612000700 1 -- 172.21.15.138:0/257805144 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 3669+0+0 (secure 0 0 0) 0x7fb60c0ced70 con 0x7fb6141046b0 2024-04-24T08:42:38.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:42:38.443+0000 7fb619600700 1 --2- 172.21.15.138:0/257805144 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fb5fc0a3cc0 0x7fb5fc0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:42:38.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:42:38.444+0000 7fb619600700 1 --2- 172.21.15.138:0/257805144 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fb5fc0a3cc0 0x7fb5fc0a6180 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb60800b2e0 tx=0x7fb60800b7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:42:38.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:42:38.447+0000 7fb612000700 1 -- 172.21.15.138:0/257805144 <== mon.2 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb60c022020 con 0x7fb6141046b0 2024-04-24T08:42:38.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:42:38 smithi083 ceph-mon[27907]: Detected new or changed devices on smithi083 2024-04-24T08:42:38.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:42:38 smithi083 ceph-mon[27907]: Adjusting osd_memory_target on smithi083 to 8544M 2024-04-24T08:42:38.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:42:38 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:42:38.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:42:38 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:42:38.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:42:38 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:42:38.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:42:38 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:42:38.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:42:38.683+0000 7fb61aa00700 1 -- 172.21.15.138:0/257805144 --> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fb614062f30 con 0x7fb5fc0a3cc0 2024-04-24T08:42:38.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:42:38 smithi189 ceph-mon[31791]: Detected new or changed devices on smithi083 2024-04-24T08:42:38.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:42:38 smithi189 ceph-mon[31791]: Adjusting osd_memory_target on smithi083 to 8544M 2024-04-24T08:42:38.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:42:38 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:42:38.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:42:38 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:42:38.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:42:38 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:42:38.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:42:38 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:42:39.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:42:39 smithi083 ceph-mon[27907]: from='client.34211 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:42:39.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:42:39 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T08:42:39.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:42:39 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T08:42:39.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:42:39 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:42:39.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:42:39 smithi083 ceph-mon[27907]: pgmap v138: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 105 KiB/s, 0 objects/s recovering 2024-04-24T08:42:39.633 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:42:39 smithi138 ceph-mon[31441]: from='client.34211 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:42:39.633 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:42:39 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T08:42:39.633 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:42:39 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T08:42:39.633 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:42:39 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:42:39.633 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:42:39 smithi138 ceph-mon[31441]: pgmap v138: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 105 KiB/s, 0 objects/s recovering 2024-04-24T08:42:39.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:42:39 smithi189 ceph-mon[31791]: from='client.34211 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:42:39.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:42:39 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T08:42:39.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:42:39 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T08:42:39.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:42:39 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:42:39.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:42:39 smithi189 ceph-mon[31791]: pgmap v138: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 105 KiB/s, 0 objects/s recovering 2024-04-24T08:42:41.053 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:42:40 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:42:40.697+0000 7fbaa7200700 -1 calc_pg_upmaps abort due to max <= 0 2024-04-24T08:42:42.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:42:41 smithi083 ceph-mon[27907]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-04-24T08:42:42.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:42:41 smithi138 ceph-mon[31441]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-04-24T08:42:42.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:42:41 smithi189 ceph-mon[31791]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-04-24T08:42:43.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:42:42 smithi083 ceph-mon[27907]: from='client.? 172.21.15.138:0/1432072656' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7b50cd14-9385-4535-a7b0-1c1da93c8548"}]: dispatch 2024-04-24T08:42:43.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:42:42 smithi083 ceph-mon[27907]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7b50cd14-9385-4535-a7b0-1c1da93c8548"}]: dispatch 2024-04-24T08:42:43.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:42:42 smithi083 ceph-mon[27907]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7b50cd14-9385-4535-a7b0-1c1da93c8548"}]': finished 2024-04-24T08:42:43.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:42:42 smithi083 ceph-mon[27907]: osdmap e30: 5 total, 4 up, 5 in 2024-04-24T08:42:43.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:42:42 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T08:42:43.180 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:42:42 smithi138 ceph-mon[31441]: from='client.? 172.21.15.138:0/1432072656' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7b50cd14-9385-4535-a7b0-1c1da93c8548"}]: dispatch 2024-04-24T08:42:43.180 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:42:42 smithi138 ceph-mon[31441]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7b50cd14-9385-4535-a7b0-1c1da93c8548"}]: dispatch 2024-04-24T08:42:43.180 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:42:42 smithi138 ceph-mon[31441]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7b50cd14-9385-4535-a7b0-1c1da93c8548"}]': finished 2024-04-24T08:42:43.180 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:42:42 smithi138 ceph-mon[31441]: osdmap e30: 5 total, 4 up, 5 in 2024-04-24T08:42:43.180 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:42:42 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T08:42:43.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:42:42 smithi189 ceph-mon[31791]: from='client.? 172.21.15.138:0/1432072656' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7b50cd14-9385-4535-a7b0-1c1da93c8548"}]: dispatch 2024-04-24T08:42:43.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:42:42 smithi189 ceph-mon[31791]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7b50cd14-9385-4535-a7b0-1c1da93c8548"}]: dispatch 2024-04-24T08:42:43.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:42:42 smithi189 ceph-mon[31791]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7b50cd14-9385-4535-a7b0-1c1da93c8548"}]': finished 2024-04-24T08:42:43.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:42:42 smithi189 ceph-mon[31791]: osdmap e30: 5 total, 4 up, 5 in 2024-04-24T08:42:43.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:42:42 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T08:42:44.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:42:43 smithi083 ceph-mon[27907]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-24T08:42:44.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:42:43 smithi083 ceph-mon[27907]: from='client.? 172.21.15.138:0/3663757413' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T08:42:44.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:42:43 smithi138 ceph-mon[31441]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-24T08:42:44.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:42:43 smithi138 ceph-mon[31441]: from='client.? 172.21.15.138:0/3663757413' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T08:42:44.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:42:43 smithi189 ceph-mon[31791]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-24T08:42:44.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:42:43 smithi189 ceph-mon[31791]: from='client.? 172.21.15.138:0/3663757413' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T08:42:46.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:42:45 smithi083 ceph-mon[27907]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-24T08:42:46.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:42:45 smithi138 ceph-mon[31441]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-24T08:42:46.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:42:45 smithi189 ceph-mon[31791]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-24T08:42:47.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:42:47 smithi138 ceph-mon[31441]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-24T08:42:48.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:42:47 smithi083 ceph-mon[27907]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-24T08:42:48.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:42:47 smithi189 ceph-mon[31791]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-24T08:42:50.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:42:49 smithi138 ceph-mon[31441]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-24T08:42:50.196 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:42:49 smithi083 ceph-mon[27907]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-24T08:42:50.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:42:49 smithi189 ceph-mon[31791]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-24T08:42:52.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:42:51 smithi138 ceph-mon[31441]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-24T08:42:52.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:42:51 smithi189 ceph-mon[31791]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-24T08:42:52.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:42:51 smithi083 ceph-mon[27907]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-24T08:42:53.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:42:52 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-24T08:42:53.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:42:52 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:42:53.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:42:52 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-24T08:42:53.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:42:52 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:42:53.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:42:52 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-24T08:42:53.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:42:52 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:42:53.993 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:42:53 smithi138 ceph-mon[31441]: Deploying daemon osd.4 on smithi138 2024-04-24T08:42:53.993 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:42:53 smithi138 ceph-mon[31441]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-24T08:42:54.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:42:53 smithi189 ceph-mon[31791]: Deploying daemon osd.4 on smithi138 2024-04-24T08:42:54.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:42:53 smithi189 ceph-mon[31791]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-24T08:42:54.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:42:53 smithi083 ceph-mon[27907]: Deploying daemon osd.4 on smithi138 2024-04-24T08:42:54.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:42:53 smithi083 ceph-mon[27907]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-24T08:42:55.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:42:55 smithi138 ceph-mon[31441]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-24T08:42:56.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:42:55 smithi189 ceph-mon[31791]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-24T08:42:56.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:42:55 smithi083 ceph-mon[27907]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-24T08:42:58.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:42:57 smithi138 ceph-mon[31441]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-24T08:42:58.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:42:57 smithi189 ceph-mon[31791]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-24T08:42:58.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:42:57 smithi083 ceph-mon[27907]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-24T08:42:59.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:42:59 smithi138 ceph-mon[31441]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-24T08:43:00.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:42:59 smithi189 ceph-mon[31791]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-24T08:43:00.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:42:59 smithi083 ceph-mon[27907]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-24T08:43:01.951 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:01 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:43:01.952 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:01 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:43:01.952 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:01 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:43:01.952 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:01 smithi138 ceph-mon[31441]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-24T08:43:01.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:01 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:43:01.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:01 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:43:01.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:01 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:43:01.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:01 smithi189 ceph-mon[31791]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-24T08:43:02.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:01 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:43:02.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:01 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:43:02.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:01 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:43:02.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:01 smithi083 ceph-mon[27907]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-24T08:43:03.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:03 smithi138 ceph-mon[31441]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-24T08:43:03.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:03 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:43:03.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:03 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:43:04.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:03 smithi083 ceph-mon[27907]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-24T08:43:04.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:03 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:43:04.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:03 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:43:04.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:03 smithi189 ceph-mon[31791]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-24T08:43:04.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:03 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:43:04.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:03 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:43:05.053 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:04 smithi138 ceph-mon[31441]: from='osd.4 [v2:172.21.15.138:6808/823775360,v1:172.21.15.138:6809/823775360]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-24T08:43:05.053 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:04 smithi138 ceph-mon[31441]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-24T08:43:05.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:04 smithi083 ceph-mon[27907]: from='osd.4 [v2:172.21.15.138:6808/823775360,v1:172.21.15.138:6809/823775360]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-24T08:43:05.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:04 smithi083 ceph-mon[27907]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-24T08:43:05.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:04 smithi189 ceph-mon[31791]: from='osd.4 [v2:172.21.15.138:6808/823775360,v1:172.21.15.138:6809/823775360]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-24T08:43:05.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:04 smithi189 ceph-mon[31791]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-24T08:43:05.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:43:05.665+0000 7fb612000700 1 -- 172.21.15.138:0/257805144 <== mgr.14152 v2:172.21.15.83:6800/2550826560 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb614062f30 con 0x7fb5fc0a3cc0 2024-04-24T08:43:05.668 INFO:teuthology.orchestra.run.smithi138.stdout:Created osd(s) 4 on host 'smithi138' 2024-04-24T08:43:05.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:43:05.669+0000 7fb61aa00700 1 -- 172.21.15.138:0/257805144 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fb5fc0a3cc0 msgr2=0x7fb5fc0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:43:05.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:43:05.669+0000 7fb61aa00700 1 --2- 172.21.15.138:0/257805144 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fb5fc0a3cc0 0x7fb5fc0a6180 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb60800b2e0 tx=0x7fb60800b7e0 comp rx=0 tx=0).stop 2024-04-24T08:43:05.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:43:05.669+0000 7fb61aa00700 1 -- 172.21.15.138:0/257805144 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6141046b0 msgr2=0x7fb614102200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:43:05.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:43:05.669+0000 7fb61aa00700 1 --2- 172.21.15.138:0/257805144 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6141046b0 0x7fb614102200 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb60c00ef30 tx=0x7fb60c00c3a0 comp rx=0 tx=0).stop 2024-04-24T08:43:05.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:43:05.669+0000 7fb61aa00700 1 -- 172.21.15.138:0/257805144 shutdown_connections 2024-04-24T08:43:05.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:43:05.669+0000 7fb61aa00700 1 --2- 172.21.15.138:0/257805144 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fb5fc0a3cc0 0x7fb5fc0a6180 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:43:05.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:43:05.669+0000 7fb61aa00700 1 --2- 172.21.15.138:0/257805144 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb614100d30 0x7fb614103670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:43:05.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:43:05.669+0000 7fb61aa00700 1 --2- 172.21.15.138:0/257805144 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6141046b0 0x7fb614102200 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:43:05.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:43:05.669+0000 7fb61aa00700 1 --2- 172.21.15.138:0/257805144 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb614103d10 0x7fb614101cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:43:05.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:43:05.669+0000 7fb61aa00700 1 -- 172.21.15.138:0/257805144 >> 172.21.15.138:0/257805144 conn(0x7fb6140fc690 msgr2=0x7fb614107030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:43:05.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:43:05.669+0000 7fb61aa00700 1 -- 172.21.15.138:0/257805144 shutdown_connections 2024-04-24T08:43:05.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:43:05.669+0000 7fb61aa00700 1 -- 172.21.15.138:0/257805144 wait complete. 2024-04-24T08:43:06.137 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:05 smithi138 ceph-mon[31441]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-24T08:43:06.137 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:05 smithi138 ceph-mon[31441]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-24T08:43:06.138 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:05 smithi138 ceph-mon[31441]: from='osd.4 [v2:172.21.15.138:6808/823775360,v1:172.21.15.138:6809/823775360]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-04-24T08:43:06.138 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:05 smithi138 ceph-mon[31441]: osdmap e31: 5 total, 4 up, 5 in 2024-04-24T08:43:06.138 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:05 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T08:43:06.138 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:05 smithi138 ceph-mon[31441]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-04-24T08:43:06.138 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:05 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:43:06.138 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:05 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:43:06.138 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:05 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:43:06.138 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:05 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:43:06.138 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:05 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:43:06.138 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:05 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:43:06.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:05 smithi189 ceph-mon[31791]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-24T08:43:06.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:05 smithi189 ceph-mon[31791]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-24T08:43:06.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:05 smithi189 ceph-mon[31791]: from='osd.4 [v2:172.21.15.138:6808/823775360,v1:172.21.15.138:6809/823775360]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-04-24T08:43:06.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:05 smithi189 ceph-mon[31791]: osdmap e31: 5 total, 4 up, 5 in 2024-04-24T08:43:06.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:05 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T08:43:06.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:05 smithi189 ceph-mon[31791]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-04-24T08:43:06.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:05 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:43:06.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:05 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:43:06.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:05 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:43:06.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:05 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:43:06.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:05 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:43:06.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:05 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:43:06.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:05 smithi083 ceph-mon[27907]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-24T08:43:06.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:05 smithi083 ceph-mon[27907]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-24T08:43:06.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:05 smithi083 ceph-mon[27907]: from='osd.4 [v2:172.21.15.138:6808/823775360,v1:172.21.15.138:6809/823775360]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-04-24T08:43:06.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:05 smithi083 ceph-mon[27907]: osdmap e31: 5 total, 4 up, 5 in 2024-04-24T08:43:06.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:05 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T08:43:06.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:05 smithi083 ceph-mon[27907]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-04-24T08:43:06.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:05 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:43:06.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:05 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:43:06.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:05 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:43:06.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:05 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:43:06.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:05 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:43:06.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:05 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:43:06.576 DEBUG:teuthology.orchestra.run.smithi138:osd.4> sudo journalctl -f -n 0 -u ceph-bd3e039e-0215-11ef-bc93-c7b262605968@osd.4.service 2024-04-24T08:43:06.581 INFO:tasks.cephadm:Deploying osd.5 on smithi189 with /dev/vg_nvme/lv_3... 2024-04-24T08:43:06.581 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd3e039e-0215-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-24T08:43:06.820 INFO:journalctl@ceph.osd.4.smithi138.stdout:Apr 24 08:43:06 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-4[42520]: 2024-04-24T08:43:06.643+0000 7fcabf200700 -1 osd.4 0 waiting for initial osdmap 2024-04-24T08:43:06.820 INFO:journalctl@ceph.osd.4.smithi138.stdout:Apr 24 08:43:06 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-4[42520]: 2024-04-24T08:43:06.667+0000 7fcab5000700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T08:43:06.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:06 smithi189 ceph-mon[31791]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-04-24T08:43:06.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:06 smithi189 ceph-mon[31791]: osdmap e32: 5 total, 4 up, 5 in 2024-04-24T08:43:06.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:06 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T08:43:06.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:06 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T08:43:06.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:06 smithi189 ceph-mon[31791]: from='osd.4 ' entity='osd.4' 2024-04-24T08:43:07.027 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T08:43:07.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:06 smithi138 ceph-mon[31441]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-04-24T08:43:07.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:06 smithi138 ceph-mon[31441]: osdmap e32: 5 total, 4 up, 5 in 2024-04-24T08:43:07.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:06 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T08:43:07.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:06 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T08:43:07.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:06 smithi138 ceph-mon[31441]: from='osd.4 ' entity='osd.4' 2024-04-24T08:43:07.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:06 smithi083 ceph-mon[27907]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-04-24T08:43:07.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:06 smithi083 ceph-mon[27907]: osdmap e32: 5 total, 4 up, 5 in 2024-04-24T08:43:07.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:06 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T08:43:07.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:06 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T08:43:07.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:06 smithi083 ceph-mon[27907]: from='osd.4 ' entity='osd.4' 2024-04-24T08:43:08.046 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:07 smithi138 ceph-mon[31441]: purged_snaps scrub starts 2024-04-24T08:43:08.046 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:07 smithi138 ceph-mon[31441]: purged_snaps scrub ok 2024-04-24T08:43:08.046 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:07 smithi138 ceph-mon[31441]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-24T08:43:08.046 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:07 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T08:43:08.046 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:07 smithi138 ceph-mon[31441]: osd.4 [v2:172.21.15.138:6808/823775360,v1:172.21.15.138:6809/823775360] boot 2024-04-24T08:43:08.046 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:07 smithi138 ceph-mon[31441]: osdmap e33: 5 total, 5 up, 5 in 2024-04-24T08:43:08.047 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:07 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T08:43:08.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:07 smithi189 ceph-mon[31791]: purged_snaps scrub starts 2024-04-24T08:43:08.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:07 smithi189 ceph-mon[31791]: purged_snaps scrub ok 2024-04-24T08:43:08.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:07 smithi189 ceph-mon[31791]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-24T08:43:08.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:07 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T08:43:08.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:07 smithi189 ceph-mon[31791]: osd.4 [v2:172.21.15.138:6808/823775360,v1:172.21.15.138:6809/823775360] boot 2024-04-24T08:43:08.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:07 smithi189 ceph-mon[31791]: osdmap e33: 5 total, 5 up, 5 in 2024-04-24T08:43:08.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:07 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T08:43:08.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:07 smithi083 ceph-mon[27907]: purged_snaps scrub starts 2024-04-24T08:43:08.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:07 smithi083 ceph-mon[27907]: purged_snaps scrub ok 2024-04-24T08:43:08.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:07 smithi083 ceph-mon[27907]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-24T08:43:08.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:07 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T08:43:08.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:07 smithi083 ceph-mon[27907]: osd.4 [v2:172.21.15.138:6808/823775360,v1:172.21.15.138:6809/823775360] boot 2024-04-24T08:43:08.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:07 smithi083 ceph-mon[27907]: osdmap e33: 5 total, 5 up, 5 in 2024-04-24T08:43:08.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:07 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T08:43:10.029 INFO:teuthology.orchestra.run.smithi189.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-24T08:43:10.029 INFO:teuthology.orchestra.run.smithi189.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-24T08:43:10.029 INFO:teuthology.orchestra.run.smithi189.stderr: stderr: 10+0 records in 2024-04-24T08:43:10.029 INFO:teuthology.orchestra.run.smithi189.stderr:10+0 records out 2024-04-24T08:43:10.029 INFO:teuthology.orchestra.run.smithi189.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0370256 s, 283 MB/s 2024-04-24T08:43:10.029 INFO:teuthology.orchestra.run.smithi189.stderr:--> Zapping successful for: 2024-04-24T08:43:10.105 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:09 smithi083 ceph-mon[27907]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:43:10.106 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:09 smithi083 ceph-mon[27907]: osdmap e34: 5 total, 5 up, 5 in 2024-04-24T08:43:10.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:09 smithi138 ceph-mon[31441]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:43:10.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:09 smithi138 ceph-mon[31441]: osdmap e34: 5 total, 5 up, 5 in 2024-04-24T08:43:10.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:09 smithi189 ceph-mon[31791]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:43:10.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:09 smithi189 ceph-mon[31791]: osdmap e34: 5 total, 5 up, 5 in 2024-04-24T08:43:10.840 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd3e039e-0215-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi189:vg_nvme/lv_3 2024-04-24T08:43:11.267 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T08:43:11.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:11 smithi138 ceph-mon[31441]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:43:11.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:11 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:43:11.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:11 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:43:11.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:11 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:43:11.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:11 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:43:11.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:11 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:43:11.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:11 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:43:11.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:11 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:43:11.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:11 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:43:12.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:11 smithi083 ceph-mon[27907]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:43:12.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:11 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:43:12.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:11 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:43:12.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:11 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:43:12.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:11 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:43:12.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:11 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:43:12.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:11 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:43:12.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:11 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:43:12.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:11 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:43:12.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:11 smithi189 ceph-mon[31791]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:43:12.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:11 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:43:12.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:11 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:43:12.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:11 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:43:12.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:11 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:43:12.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:11 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:43:12.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:11 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:43:12.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:11 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:43:12.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:11 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:43:13.112 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:12 smithi189 ceph-mon[31791]: Detected new or changed devices on smithi138 2024-04-24T08:43:13.112 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:12 smithi189 ceph-mon[31791]: Adjusting osd_memory_target on smithi138 to 8544M 2024-04-24T08:43:13.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:12 smithi138 ceph-mon[31441]: Detected new or changed devices on smithi138 2024-04-24T08:43:13.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:12 smithi138 ceph-mon[31441]: Adjusting osd_memory_target on smithi138 to 8544M 2024-04-24T08:43:13.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:12 smithi083 ceph-mon[27907]: Detected new or changed devices on smithi138 2024-04-24T08:43:13.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:12 smithi083 ceph-mon[27907]: Adjusting osd_memory_target on smithi138 to 8544M 2024-04-24T08:43:13.401 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:43:13.400+0000 7fa0dc000700 1 -- 172.21.15.189:0/4219031991 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa0d41046e0 msgr2=0x7fa0d410af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:43:13.401 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:43:13.400+0000 7fa0dc000700 1 --2- 172.21.15.189:0/4219031991 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa0d41046e0 0x7fa0d410af90 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc00b0a0 tx=0x7fa0cc00b3b0 comp rx=0 tx=0).stop 2024-04-24T08:43:13.402 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:43:13.400+0000 7fa0dc000700 1 -- 172.21.15.189:0/4219031991 shutdown_connections 2024-04-24T08:43:13.402 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:43:13.400+0000 7fa0dc000700 1 --2- 172.21.15.189:0/4219031991 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa0d41046e0 0x7fa0d410af90 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:43:13.402 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:43:13.400+0000 7fa0dc000700 1 --2- 172.21.15.189:0/4219031991 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa0d4102b40 0x7fa0d4102f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:43:13.402 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:43:13.400+0000 7fa0dc000700 1 --2- 172.21.15.189:0/4219031991 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa0d4103d40 0x7fa0d41041a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:43:13.402 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:43:13.400+0000 7fa0dc000700 1 -- 172.21.15.189:0/4219031991 >> 172.21.15.189:0/4219031991 conn(0x7fa0d40fe0e0 msgr2=0x7fa0d4100520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:43:13.402 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:43:13.401+0000 7fa0dc000700 1 -- 172.21.15.189:0/4219031991 shutdown_connections 2024-04-24T08:43:13.402 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:43:13.401+0000 7fa0dc000700 1 -- 172.21.15.189:0/4219031991 wait complete. 2024-04-24T08:43:13.404 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:43:13.402+0000 7fa0dc000700 1 Processor -- start 2024-04-24T08:43:13.404 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:43:13.402+0000 7fa0dc000700 1 -- start start 2024-04-24T08:43:13.404 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:43:13.402+0000 7fa0dc000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa0d4102b40 0x7fa0d419ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:43:13.404 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:43:13.402+0000 7fa0dc000700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa0d4103d40 0x7fa0d419b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:43:13.404 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:43:13.402+0000 7fa0dc000700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa0d41046e0 0x7fa0d41a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:43:13.404 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:43:13.402+0000 7fa0dc000700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa0d410e630 con 0x7fa0d41046e0 2024-04-24T08:43:13.404 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:43:13.402+0000 7fa0dc000700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa0d410e4b0 con 0x7fa0d4103d40 2024-04-24T08:43:13.405 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:43:13.402+0000 7fa0dc000700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa0d410e7b0 con 0x7fa0d4102b40 2024-04-24T08:43:13.405 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:43:13.403+0000 7fa0d9600700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa0d4102b40 0x7fa0d419ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:43:13.405 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:43:13.403+0000 7fa0d9600700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa0d4102b40 0x7fa0d419ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.189:49118/0 (socket says 172.21.15.189:49118) 2024-04-24T08:43:13.405 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:43:13.403+0000 7fa0d9600700 1 -- 172.21.15.189:0/1382746599 learned_addr learned my addr 172.21.15.189:0/1382746599 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-04-24T08:43:13.405 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:43:13.403+0000 7fa0d8c00700 1 --2- 172.21.15.189:0/1382746599 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa0d4103d40 0x7fa0d419b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:43:13.405 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:43:13.403+0000 7fa0da000700 1 --2- 172.21.15.189:0/1382746599 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa0d41046e0 0x7fa0d41a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:43:13.405 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:43:13.403+0000 7fa0d9600700 1 -- 172.21.15.189:0/1382746599 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa0d4103d40 msgr2=0x7fa0d419b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:43:13.405 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:43:13.403+0000 7fa0d9600700 1 --2- 172.21.15.189:0/1382746599 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa0d4103d40 0x7fa0d419b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:43:13.405 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:43:13.403+0000 7fa0d9600700 1 -- 172.21.15.189:0/1382746599 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa0d41046e0 msgr2=0x7fa0d41a2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:43:13.405 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:43:13.403+0000 7fa0d9600700 1 --2- 172.21.15.189:0/1382746599 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa0d41046e0 0x7fa0d41a2410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:43:13.405 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:43:13.403+0000 7fa0d9600700 1 -- 172.21.15.189:0/1382746599 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0d41a2ab0 con 0x7fa0d4102b40 2024-04-24T08:43:13.405 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:43:13.404+0000 7fa0d9600700 1 --2- 172.21.15.189:0/1382746599 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa0d4102b40 0x7fa0d419ae50 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa0c800d9b0 tx=0x7fa0c800dd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:43:13.405 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:43:13.404+0000 7fa0d2000700 1 -- 172.21.15.189:0/1382746599 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c8017070 con 0x7fa0d4102b40 2024-04-24T08:43:13.406 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:43:13.404+0000 7fa0d2000700 1 -- 172.21.15.189:0/1382746599 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa0c8010460 con 0x7fa0d4102b40 2024-04-24T08:43:13.406 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:43:13.404+0000 7fa0dc000700 1 -- 172.21.15.189:0/1382746599 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0d41a2da0 con 0x7fa0d4102b40 2024-04-24T08:43:13.406 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:43:13.404+0000 7fa0d2000700 1 -- 172.21.15.189:0/1382746599 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c800f620 con 0x7fa0d4102b40 2024-04-24T08:43:13.406 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:43:13.404+0000 7fa0dc000700 1 -- 172.21.15.189:0/1382746599 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0d41a31a0 con 0x7fa0d4102b40 2024-04-24T08:43:13.406 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:43:13.405+0000 7fa0dc000700 1 -- 172.21.15.189:0/1382746599 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0d41a33d0 con 0x7fa0d4102b40 2024-04-24T08:43:13.407 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:43:13.407+0000 7fa0d2000700 1 -- 172.21.15.189:0/1382746599 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fa0c8010610 con 0x7fa0d4102b40 2024-04-24T08:43:13.408 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:43:13.408+0000 7fa0d2000700 1 --2- 172.21.15.189:0/1382746599 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fa0bc0a3c40 0x7fa0bc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:43:13.409 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:43:13.409+0000 7fa0d2000700 1 -- 172.21.15.189:0/1382746599 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 4101+0+0 (secure 0 0 0) 0x7fa0c80ceff0 con 0x7fa0d4102b40 2024-04-24T08:43:13.409 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:43:13.409+0000 7fa0d8c00700 1 --2- 172.21.15.189:0/1382746599 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fa0bc0a3c40 0x7fa0bc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:43:13.410 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:43:13.410+0000 7fa0d8c00700 1 --2- 172.21.15.189:0/1382746599 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fa0bc0a3c40 0x7fa0bc0a6100 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa0c0005950 tx=0x7fa0c00058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:43:13.414 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:43:13.414+0000 7fa0d2000700 1 -- 172.21.15.189:0/1382746599 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa0c809c9a0 con 0x7fa0d4102b40 2024-04-24T08:43:13.649 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:43:13.649+0000 7fa0dc000700 1 -- 172.21.15.189:0/1382746599 --> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fa0d4076440 con 0x7fa0bc0a3c40 2024-04-24T08:43:14.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:13 smithi138 ceph-mon[31441]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:43:14.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:13 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T08:43:14.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:13 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T08:43:14.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:13 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:43:14.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:13 smithi189 ceph-mon[31791]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:43:14.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:13 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T08:43:14.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:13 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T08:43:14.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:13 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:43:14.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:13 smithi083 ceph-mon[27907]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:43:14.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:13 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T08:43:14.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:13 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T08:43:14.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:13 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:43:14.881 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:14 smithi189 ceph-mon[31791]: from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:43:15.128 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:14 smithi138 ceph-mon[31441]: from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:43:15.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:14 smithi083 ceph-mon[27907]: from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:43:16.074 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:15 smithi189 ceph-mon[31791]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:43:16.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:15 smithi138 ceph-mon[31441]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:43:16.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:15 smithi083 ceph-mon[27907]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:43:18.164 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:17 smithi189 ceph-mon[31791]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:43:18.164 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:17 smithi189 ceph-mon[31791]: from='client.? 172.21.15.189:0/577708916' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9c647797-35bc-449f-be84-af97d2c69c2b"}]: dispatch 2024-04-24T08:43:18.164 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:17 smithi189 ceph-mon[31791]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9c647797-35bc-449f-be84-af97d2c69c2b"}]: dispatch 2024-04-24T08:43:18.164 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:17 smithi189 ceph-mon[31791]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9c647797-35bc-449f-be84-af97d2c69c2b"}]': finished 2024-04-24T08:43:18.164 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:17 smithi189 ceph-mon[31791]: osdmap e35: 6 total, 5 up, 6 in 2024-04-24T08:43:18.164 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:17 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T08:43:18.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:17 smithi138 ceph-mon[31441]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:43:18.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:17 smithi138 ceph-mon[31441]: from='client.? 172.21.15.189:0/577708916' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9c647797-35bc-449f-be84-af97d2c69c2b"}]: dispatch 2024-04-24T08:43:18.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:17 smithi138 ceph-mon[31441]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9c647797-35bc-449f-be84-af97d2c69c2b"}]: dispatch 2024-04-24T08:43:18.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:17 smithi138 ceph-mon[31441]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9c647797-35bc-449f-be84-af97d2c69c2b"}]': finished 2024-04-24T08:43:18.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:17 smithi138 ceph-mon[31441]: osdmap e35: 6 total, 5 up, 6 in 2024-04-24T08:43:18.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:17 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T08:43:18.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:17 smithi083 ceph-mon[27907]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:43:18.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:17 smithi083 ceph-mon[27907]: from='client.? 172.21.15.189:0/577708916' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9c647797-35bc-449f-be84-af97d2c69c2b"}]: dispatch 2024-04-24T08:43:18.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:17 smithi083 ceph-mon[27907]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9c647797-35bc-449f-be84-af97d2c69c2b"}]: dispatch 2024-04-24T08:43:18.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:17 smithi083 ceph-mon[27907]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9c647797-35bc-449f-be84-af97d2c69c2b"}]': finished 2024-04-24T08:43:18.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:17 smithi083 ceph-mon[27907]: osdmap e35: 6 total, 5 up, 6 in 2024-04-24T08:43:18.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:17 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T08:43:19.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:18 smithi138 ceph-mon[31441]: from='client.? 172.21.15.189:0/2862476410' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T08:43:19.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:18 smithi189 ceph-mon[31791]: from='client.? 172.21.15.189:0/2862476410' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T08:43:19.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:18 smithi083 ceph-mon[27907]: from='client.? 172.21.15.189:0/2862476410' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T08:43:20.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:19 smithi138 ceph-mon[31441]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:43:20.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:19 smithi189 ceph-mon[31791]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:43:20.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:19 smithi083 ceph-mon[27907]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:43:22.179 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:21 smithi189 ceph-mon[31791]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:43:22.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:21 smithi138 ceph-mon[31441]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:43:22.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:21 smithi083 ceph-mon[27907]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:43:24.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:23 smithi138 ceph-mon[31441]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:43:24.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:23 smithi189 ceph-mon[31791]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:43:24.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:23 smithi083 ceph-mon[27907]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:43:26.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:25 smithi138 ceph-mon[31441]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:43:26.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:25 smithi189 ceph-mon[31791]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:43:26.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:25 smithi083 ceph-mon[27907]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:43:27.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:26 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-24T08:43:27.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:26 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:43:27.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:26 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-24T08:43:27.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:26 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:43:27.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:26 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-24T08:43:27.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:26 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:43:28.075 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:27 smithi189 ceph-mon[31791]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:43:28.076 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:27 smithi189 ceph-mon[31791]: Deploying daemon osd.5 on smithi189 2024-04-24T08:43:28.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:27 smithi138 ceph-mon[31441]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:43:28.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:27 smithi138 ceph-mon[31441]: Deploying daemon osd.5 on smithi189 2024-04-24T08:43:28.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:27 smithi083 ceph-mon[27907]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:43:28.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:27 smithi083 ceph-mon[27907]: Deploying daemon osd.5 on smithi189 2024-04-24T08:43:29.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:29 smithi189 ceph-mon[31791]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:43:30.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:29 smithi138 ceph-mon[31441]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:43:30.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:29 smithi083 ceph-mon[27907]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:43:32.179 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:31 smithi189 ceph-mon[31791]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:43:32.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:31 smithi138 ceph-mon[31441]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:43:32.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:31 smithi083 ceph-mon[27907]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:43:34.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:33 smithi138 ceph-mon[31441]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:43:34.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:33 smithi189 ceph-mon[31791]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:43:34.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:33 smithi083 ceph-mon[27907]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:43:35.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:34 smithi189 ceph-mon[31791]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:43:35.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:34 smithi083 ceph-mon[27907]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:43:35.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:34 smithi138 ceph-mon[31441]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:43:36.407 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:36 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:43:36.407 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:36 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:43:36.407 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:36 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:43:36.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:36 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:43:36.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:36 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:43:36.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:36 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:43:36.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:36 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:43:36.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:36 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:43:36.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:36 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:43:37.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:37 smithi189 ceph-mon[31791]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:43:37.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:37 smithi138 ceph-mon[31441]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:43:37.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:37 smithi083 ceph-mon[27907]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:43:39.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:39 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:43:39.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:39 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:43:39.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:39 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:43:39.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:39 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:43:39.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:39 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:43:39.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:39 smithi083 ceph-mon[27907]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:43:39.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:39 smithi083 ceph-mon[27907]: from='osd.5 [v2:172.21.15.189:6808/1399817431,v1:172.21.15.189:6809/1399817431]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-24T08:43:39.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:39 smithi083 ceph-mon[27907]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-24T08:43:39.830 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:39 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:43:39.830 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:39 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:43:39.830 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:39 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:43:39.830 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:39 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:43:39.830 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:39 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:43:39.830 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:39 smithi189 ceph-mon[31791]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:43:39.830 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:39 smithi189 ceph-mon[31791]: from='osd.5 [v2:172.21.15.189:6808/1399817431,v1:172.21.15.189:6809/1399817431]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-24T08:43:39.831 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:39 smithi189 ceph-mon[31791]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-24T08:43:39.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:39 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:43:39.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:39 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:43:39.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:39 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:43:39.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:39 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:43:39.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:39 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:43:39.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:39 smithi138 ceph-mon[31441]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:43:39.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:39 smithi138 ceph-mon[31441]: from='osd.5 [v2:172.21.15.189:6808/1399817431,v1:172.21.15.189:6809/1399817431]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-24T08:43:39.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:39 smithi138 ceph-mon[31441]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-24T08:43:39.962 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:43:39.961+0000 7fa0d2000700 1 -- 172.21.15.189:0/1382746599 <== mgr.14152 v2:172.21.15.83:6800/2550826560 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa0d4076440 con 0x7fa0bc0a3c40 2024-04-24T08:43:39.962 INFO:teuthology.orchestra.run.smithi189.stdout:Created osd(s) 5 on host 'smithi189' 2024-04-24T08:43:39.965 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:43:39.965+0000 7fa0dc000700 1 -- 172.21.15.189:0/1382746599 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fa0bc0a3c40 msgr2=0x7fa0bc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:43:39.966 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:43:39.965+0000 7fa0dc000700 1 --2- 172.21.15.189:0/1382746599 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fa0bc0a3c40 0x7fa0bc0a6100 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa0c0005950 tx=0x7fa0c00058e0 comp rx=0 tx=0).stop 2024-04-24T08:43:39.966 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:43:39.965+0000 7fa0dc000700 1 -- 172.21.15.189:0/1382746599 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa0d4102b40 msgr2=0x7fa0d419ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:43:39.966 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:43:39.965+0000 7fa0dc000700 1 --2- 172.21.15.189:0/1382746599 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa0d4102b40 0x7fa0d419ae50 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa0c800d9b0 tx=0x7fa0c800dd70 comp rx=0 tx=0).stop 2024-04-24T08:43:39.966 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:43:39.965+0000 7fa0dc000700 1 -- 172.21.15.189:0/1382746599 shutdown_connections 2024-04-24T08:43:39.966 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:43:39.965+0000 7fa0dc000700 1 --2- 172.21.15.189:0/1382746599 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fa0bc0a3c40 0x7fa0bc0a6100 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:43:39.966 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:43:39.965+0000 7fa0dc000700 1 --2- 172.21.15.189:0/1382746599 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa0d41046e0 0x7fa0d41a2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:43:39.966 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:43:39.965+0000 7fa0dc000700 1 --2- 172.21.15.189:0/1382746599 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa0d4102b40 0x7fa0d419ae50 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:43:39.966 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:43:39.965+0000 7fa0dc000700 1 --2- 172.21.15.189:0/1382746599 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa0d4103d40 0x7fa0d419b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:43:39.966 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:43:39.965+0000 7fa0dc000700 1 -- 172.21.15.189:0/1382746599 >> 172.21.15.189:0/1382746599 conn(0x7fa0d40fe0e0 msgr2=0x7fa0d4074d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:43:39.966 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:43:39.965+0000 7fa0dc000700 1 -- 172.21.15.189:0/1382746599 shutdown_connections 2024-04-24T08:43:39.966 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:43:39.965+0000 7fa0dc000700 1 -- 172.21.15.189:0/1382746599 wait complete. 2024-04-24T08:43:40.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:40 smithi189 ceph-mon[31791]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-24T08:43:40.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:40 smithi189 ceph-mon[31791]: osdmap e36: 6 total, 5 up, 6 in 2024-04-24T08:43:40.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:40 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T08:43:40.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:40 smithi189 ceph-mon[31791]: from='osd.5 [v2:172.21.15.189:6808/1399817431,v1:172.21.15.189:6809/1399817431]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-04-24T08:43:40.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:40 smithi189 ceph-mon[31791]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-04-24T08:43:40.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:40 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:43:40.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:40 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:43:40.709 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:40 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:43:40.832 DEBUG:teuthology.orchestra.run.smithi189:osd.5> sudo journalctl -f -n 0 -u ceph-bd3e039e-0215-11ef-bc93-c7b262605968@osd.5.service 2024-04-24T08:43:40.835 INFO:tasks.cephadm:Deploying osd.6 on smithi083 with /dev/vg_nvme/lv_2... 2024-04-24T08:43:40.836 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd3e039e-0215-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-24T08:43:40.915 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:40 smithi083 ceph-mon[27907]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-24T08:43:40.915 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:40 smithi083 ceph-mon[27907]: osdmap e36: 6 total, 5 up, 6 in 2024-04-24T08:43:40.915 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:40 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T08:43:40.915 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:40 smithi083 ceph-mon[27907]: from='osd.5 [v2:172.21.15.189:6808/1399817431,v1:172.21.15.189:6809/1399817431]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-04-24T08:43:40.915 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:40 smithi083 ceph-mon[27907]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-04-24T08:43:40.915 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:40 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:43:40.915 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:40 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:43:40.916 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:40 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:43:40.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:40 smithi138 ceph-mon[31441]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-24T08:43:40.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:40 smithi138 ceph-mon[31441]: osdmap e36: 6 total, 5 up, 6 in 2024-04-24T08:43:40.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:40 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T08:43:40.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:40 smithi138 ceph-mon[31441]: from='osd.5 [v2:172.21.15.189:6808/1399817431,v1:172.21.15.189:6809/1399817431]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-04-24T08:43:40.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:40 smithi138 ceph-mon[31441]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-04-24T08:43:40.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:40 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:43:40.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:40 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:43:40.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:40 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:43:41.269 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T08:43:41.645 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:41 smithi189 ceph-mon[31791]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-04-24T08:43:41.645 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:41 smithi189 ceph-mon[31791]: osdmap e37: 6 total, 5 up, 6 in 2024-04-24T08:43:41.646 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:41 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T08:43:41.646 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:41 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T08:43:41.646 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:41 smithi189 ceph-mon[31791]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:43:41.646 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:41 smithi189 ceph-mon[31791]: from='osd.5 ' entity='osd.5' 2024-04-24T08:43:41.646 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:41 smithi189 ceph-mon[31791]: osd.5 [v2:172.21.15.189:6808/1399817431,v1:172.21.15.189:6809/1399817431] boot 2024-04-24T08:43:41.646 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:41 smithi189 ceph-mon[31791]: osdmap e38: 6 total, 6 up, 6 in 2024-04-24T08:43:41.646 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:41 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T08:43:41.646 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 24 08:43:41 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-5[43532]: 2024-04-24T08:43:41.384+0000 7f3191000700 -1 osd.5 0 waiting for initial osdmap 2024-04-24T08:43:41.646 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 24 08:43:41 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-5[43532]: 2024-04-24T08:43:41.415+0000 7f3186e00700 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T08:43:41.708 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:41 smithi083 ceph-mon[27907]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-04-24T08:43:41.708 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:41 smithi083 ceph-mon[27907]: osdmap e37: 6 total, 5 up, 6 in 2024-04-24T08:43:41.709 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:41 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T08:43:41.709 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:41 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T08:43:41.709 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:41 smithi083 ceph-mon[27907]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:43:41.709 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:41 smithi083 ceph-mon[27907]: from='osd.5 ' entity='osd.5' 2024-04-24T08:43:41.709 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:41 smithi083 ceph-mon[27907]: osd.5 [v2:172.21.15.189:6808/1399817431,v1:172.21.15.189:6809/1399817431] boot 2024-04-24T08:43:41.709 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:41 smithi083 ceph-mon[27907]: osdmap e38: 6 total, 6 up, 6 in 2024-04-24T08:43:41.709 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:41 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T08:43:41.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:41 smithi138 ceph-mon[31441]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-04-24T08:43:41.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:41 smithi138 ceph-mon[31441]: osdmap e37: 6 total, 5 up, 6 in 2024-04-24T08:43:41.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:41 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T08:43:41.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:41 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T08:43:41.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:41 smithi138 ceph-mon[31441]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-24T08:43:41.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:41 smithi138 ceph-mon[31441]: from='osd.5 ' entity='osd.5' 2024-04-24T08:43:41.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:41 smithi138 ceph-mon[31441]: osd.5 [v2:172.21.15.189:6808/1399817431,v1:172.21.15.189:6809/1399817431] boot 2024-04-24T08:43:41.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:41 smithi138 ceph-mon[31441]: osdmap e38: 6 total, 6 up, 6 in 2024-04-24T08:43:41.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:41 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T08:43:42.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:42 smithi083 ceph-mon[27907]: purged_snaps scrub starts 2024-04-24T08:43:42.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:42 smithi083 ceph-mon[27907]: purged_snaps scrub ok 2024-04-24T08:43:42.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:42 smithi083 ceph-mon[27907]: osdmap e39: 6 total, 6 up, 6 in 2024-04-24T08:43:42.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:42 smithi138 ceph-mon[31441]: purged_snaps scrub starts 2024-04-24T08:43:42.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:42 smithi138 ceph-mon[31441]: purged_snaps scrub ok 2024-04-24T08:43:42.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:42 smithi138 ceph-mon[31441]: osdmap e39: 6 total, 6 up, 6 in 2024-04-24T08:43:42.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:42 smithi189 ceph-mon[31791]: purged_snaps scrub starts 2024-04-24T08:43:42.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:42 smithi189 ceph-mon[31791]: purged_snaps scrub ok 2024-04-24T08:43:42.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:42 smithi189 ceph-mon[31791]: osdmap e39: 6 total, 6 up, 6 in 2024-04-24T08:43:43.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:43 smithi138 ceph-mon[31441]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 582 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:43:43.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:43 smithi138 ceph-mon[31441]: osdmap e40: 6 total, 6 up, 6 in 2024-04-24T08:43:43.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:43 smithi189 ceph-mon[31791]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 582 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:43:43.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:43 smithi189 ceph-mon[31791]: osdmap e40: 6 total, 6 up, 6 in 2024-04-24T08:43:44.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:43 smithi083 ceph-mon[27907]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 582 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:43:44.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:43 smithi083 ceph-mon[27907]: osdmap e40: 6 total, 6 up, 6 in 2024-04-24T08:43:44.496 INFO:teuthology.orchestra.run.smithi083.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-24T08:43:44.497 INFO:teuthology.orchestra.run.smithi083.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-24T08:43:44.497 INFO:teuthology.orchestra.run.smithi083.stderr: stderr: 10+0 records in 2024-04-24T08:43:44.497 INFO:teuthology.orchestra.run.smithi083.stderr:10+0 records out 2024-04-24T08:43:44.497 INFO:teuthology.orchestra.run.smithi083.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0334916 s, 313 MB/s 2024-04-24T08:43:44.497 INFO:teuthology.orchestra.run.smithi083.stderr:--> Zapping successful for: 2024-04-24T08:43:45.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:45 smithi083 ceph-mon[27907]: osdmap e41: 6 total, 6 up, 6 in 2024-04-24T08:43:45.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:45 smithi083 ceph-mon[27907]: pgmap v183: 1 pgs: 1 peering; 577 KiB data, 582 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-24T08:43:45.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:45 smithi138 ceph-mon[31441]: osdmap e41: 6 total, 6 up, 6 in 2024-04-24T08:43:45.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:45 smithi138 ceph-mon[31441]: pgmap v183: 1 pgs: 1 peering; 577 KiB data, 582 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-24T08:43:45.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:45 smithi189 ceph-mon[31791]: osdmap e41: 6 total, 6 up, 6 in 2024-04-24T08:43:45.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:45 smithi189 ceph-mon[31791]: pgmap v183: 1 pgs: 1 peering; 577 KiB data, 582 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-24T08:43:45.550 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd3e039e-0215-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi083:vg_nvme/lv_2 2024-04-24T08:43:45.967 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T08:43:46.362 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:46 smithi083 ceph-mon[27907]: osdmap e42: 6 total, 6 up, 6 in 2024-04-24T08:43:46.362 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:46 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:43:46.362 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:46 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:43:46.362 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:46 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:43:46.362 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:46 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:43:46.362 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:46 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:43:46.363 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:46 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:43:46.363 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:46 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:43:46.363 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:46 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:43:46.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:46 smithi138 ceph-mon[31441]: osdmap e42: 6 total, 6 up, 6 in 2024-04-24T08:43:46.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:46 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:43:46.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:46 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:43:46.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:46 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:43:46.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:46 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:43:46.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:46 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:43:46.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:46 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:43:46.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:46 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:43:46.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:46 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:43:46.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:46 smithi189 ceph-mon[31791]: osdmap e42: 6 total, 6 up, 6 in 2024-04-24T08:43:46.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:46 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:43:46.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:46 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:43:46.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:46 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:43:46.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:46 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:43:46.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:46 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:43:46.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:46 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:43:46.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:46 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:43:46.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:46 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:43:47.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:47 smithi138 ceph-mon[31441]: Detected new or changed devices on smithi189 2024-04-24T08:43:47.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:47 smithi138 ceph-mon[31441]: Adjusting osd_memory_target on smithi189 to 8544M 2024-04-24T08:43:47.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:47 smithi138 ceph-mon[31441]: pgmap v185: 1 pgs: 1 peering; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 139 KiB/s, 0 objects/s recovering 2024-04-24T08:43:47.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:47 smithi189 ceph-mon[31791]: Detected new or changed devices on smithi189 2024-04-24T08:43:47.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:47 smithi189 ceph-mon[31791]: Adjusting osd_memory_target on smithi189 to 8544M 2024-04-24T08:43:47.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:47 smithi189 ceph-mon[31791]: pgmap v185: 1 pgs: 1 peering; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 139 KiB/s, 0 objects/s recovering 2024-04-24T08:43:47.550 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:47 smithi083 ceph-mon[27907]: Detected new or changed devices on smithi189 2024-04-24T08:43:47.550 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:47 smithi083 ceph-mon[27907]: Adjusting osd_memory_target on smithi189 to 8544M 2024-04-24T08:43:47.550 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:47 smithi083 ceph-mon[27907]: pgmap v185: 1 pgs: 1 peering; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 139 KiB/s, 0 objects/s recovering 2024-04-24T08:43:48.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:43:48.460+0000 7f2887200700 1 -- 172.21.15.83:0/1647111565 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2880106930 msgr2=0x7f288010d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:43:48.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:43:48.460+0000 7f2887200700 1 --2- 172.21.15.83:0/1647111565 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2880106930 0x7f288010d1e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f287800b0d0 tx=0x7f287800b3e0 comp rx=0 tx=0).stop 2024-04-24T08:43:48.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:43:48.460+0000 7f2887200700 1 -- 172.21.15.83:0/1647111565 shutdown_connections 2024-04-24T08:43:48.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:43:48.460+0000 7f2887200700 1 --2- 172.21.15.83:0/1647111565 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2880106930 0x7f288010d1e0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:43:48.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:43:48.460+0000 7f2887200700 1 --2- 172.21.15.83:0/1647111565 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2880104d90 0x7f28801051b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:43:48.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:43:48.460+0000 7f2887200700 1 --2- 172.21.15.83:0/1647111565 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2880105f90 0x7f28801063f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:43:48.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:43:48.460+0000 7f2887200700 1 -- 172.21.15.83:0/1647111565 >> 172.21.15.83:0/1647111565 conn(0x7f2880100350 msgr2=0x7f2880102770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:43:48.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:43:48.460+0000 7f2887200700 1 -- 172.21.15.83:0/1647111565 shutdown_connections 2024-04-24T08:43:48.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:43:48.460+0000 7f2887200700 1 -- 172.21.15.83:0/1647111565 wait complete. 2024-04-24T08:43:48.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:43:48.461+0000 7f2887200700 1 Processor -- start 2024-04-24T08:43:48.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:43:48.461+0000 7f2887200700 1 -- start start 2024-04-24T08:43:48.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:43:48.462+0000 7f2887200700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2880104d90 0x7f288019ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:43:48.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:43:48.462+0000 7f2887200700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2880105f90 0x7f288019b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:43:48.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:43:48.462+0000 7f2887200700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2880106930 0x7f28801a23f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:43:48.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:43:48.462+0000 7f2887200700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f28801108d0 con 0x7f2880104d90 2024-04-24T08:43:48.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:43:48.462+0000 7f2887200700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2880110750 con 0x7f2880106930 2024-04-24T08:43:48.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:43:48.462+0000 7f2887200700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2880110a50 con 0x7f2880105f90 2024-04-24T08:43:48.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:43:48.462+0000 7f287fe00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2880104d90 0x7f288019ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:43:48.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:43:48.462+0000 7f287fe00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2880104d90 0x7f288019ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39926/0 (socket says 172.21.15.83:39926) 2024-04-24T08:43:48.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:43:48.462+0000 7f287fe00700 1 -- 172.21.15.83:0/3116711471 learned_addr learned my addr 172.21.15.83:0/3116711471 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:43:48.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:43:48.462+0000 7f287f400700 1 --2- 172.21.15.83:0/3116711471 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2880105f90 0x7f288019b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:43:48.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:43:48.462+0000 7f2885200700 1 --2- 172.21.15.83:0/3116711471 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2880106930 0x7f28801a23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:43:48.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:43:48.463+0000 7f287fe00700 1 -- 172.21.15.83:0/3116711471 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2880105f90 msgr2=0x7f288019b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:43:48.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:43:48.463+0000 7f287fe00700 1 --2- 172.21.15.83:0/3116711471 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2880105f90 0x7f288019b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:43:48.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:43:48.463+0000 7f287fe00700 1 -- 172.21.15.83:0/3116711471 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2880106930 msgr2=0x7f28801a23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:43:48.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:43:48.463+0000 7f287fe00700 1 --2- 172.21.15.83:0/3116711471 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2880106930 0x7f28801a23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:43:48.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:43:48.463+0000 7f287fe00700 1 -- 172.21.15.83:0/3116711471 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28801a2a90 con 0x7f2880104d90 2024-04-24T08:43:48.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:43:48.463+0000 7f287f400700 1 --2- 172.21.15.83:0/3116711471 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2880105f90 0x7f288019b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:43:48.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:43:48.463+0000 7f287fe00700 1 --2- 172.21.15.83:0/3116711471 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2880104d90 0x7f288019ae30 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f287000c9c0 tx=0x7f287000ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:43:48.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:43:48.463+0000 7f287cc00700 1 -- 172.21.15.83:0/3116711471 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2870007ac0 con 0x7f2880104d90 2024-04-24T08:43:48.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:43:48.463+0000 7f287cc00700 1 -- 172.21.15.83:0/3116711471 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2870007c60 con 0x7f2880104d90 2024-04-24T08:43:48.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:43:48.464+0000 7f2887200700 1 -- 172.21.15.83:0/3116711471 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28801a2d80 con 0x7f2880104d90 2024-04-24T08:43:48.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:43:48.464+0000 7f287cc00700 1 -- 172.21.15.83:0/3116711471 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2870018660 con 0x7f2880104d90 2024-04-24T08:43:48.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:43:48.464+0000 7f2887200700 1 -- 172.21.15.83:0/3116711471 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28801a32b0 con 0x7f2880104d90 2024-04-24T08:43:48.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:43:48.465+0000 7f2887200700 1 -- 172.21.15.83:0/3116711471 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f288002dfe0 con 0x7f2880104d90 2024-04-24T08:43:48.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:43:48.467+0000 7f287cc00700 1 -- 172.21.15.83:0/3116711471 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f287000f450 con 0x7f2880104d90 2024-04-24T08:43:48.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:43:48.468+0000 7f287cc00700 1 --2- 172.21.15.83:0/3116711471 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f28640a3cc0 0x7f28640a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:43:48.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:43:48.469+0000 7f287cc00700 1 -- 172.21.15.83:0/3116711471 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 4533+0+0 (secure 0 0 0) 0x7f28700cf660 con 0x7f2880104d90 2024-04-24T08:43:48.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:43:48.469+0000 7f287f400700 1 --2- 172.21.15.83:0/3116711471 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f28640a3cc0 0x7f28640a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:43:48.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:43:48.469+0000 7f287f400700 1 --2- 172.21.15.83:0/3116711471 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f28640a3cc0 0x7f28640a6180 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f286c0099d0 tx=0x7f286c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:43:48.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:43:48.472+0000 7f287cc00700 1 -- 172.21.15.83:0/3116711471 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2870014030 con 0x7f2880104d90 2024-04-24T08:43:48.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:43:48.709+0000 7f2887200700 1 -- 172.21.15.83:0/3116711471 --> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f2880062f30 con 0x7f28640a3cc0 2024-04-24T08:43:49.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:48 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T08:43:49.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:48 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T08:43:49.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:48 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:43:49.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:48 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T08:43:49.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:48 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T08:43:49.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:48 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:43:49.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:48 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T08:43:49.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:48 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T08:43:49.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:48 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:43:49.910 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:49 smithi083 ceph-mon[27907]: from='client.14376 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:43:49.910 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:49 smithi083 ceph-mon[27907]: pgmap v186: 1 pgs: 1 peering; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-24T08:43:50.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:49 smithi138 ceph-mon[31441]: from='client.14376 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:43:50.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:49 smithi138 ceph-mon[31441]: pgmap v186: 1 pgs: 1 peering; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-24T08:43:50.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:49 smithi189 ceph-mon[31791]: from='client.14376 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:43:50.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:49 smithi189 ceph-mon[31791]: pgmap v186: 1 pgs: 1 peering; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-24T08:43:52.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:51 smithi083 ceph-mon[27907]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-04-24T08:43:52.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:51 smithi138 ceph-mon[31441]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-04-24T08:43:52.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:51 smithi189 ceph-mon[31791]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-04-24T08:43:54.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:53 smithi138 ceph-mon[31441]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-04-24T08:43:54.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:53 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/694642481' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8f056584-16c8-4454-b445-3f780f26c196"}]: dispatch 2024-04-24T08:43:54.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:53 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/694642481' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8f056584-16c8-4454-b445-3f780f26c196"}]': finished 2024-04-24T08:43:54.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:53 smithi138 ceph-mon[31441]: osdmap e43: 7 total, 6 up, 7 in 2024-04-24T08:43:54.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:53 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T08:43:54.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:53 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/691733474' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T08:43:54.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:53 smithi189 ceph-mon[31791]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-04-24T08:43:54.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:53 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/694642481' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8f056584-16c8-4454-b445-3f780f26c196"}]: dispatch 2024-04-24T08:43:54.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:53 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/694642481' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8f056584-16c8-4454-b445-3f780f26c196"}]': finished 2024-04-24T08:43:54.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:53 smithi189 ceph-mon[31791]: osdmap e43: 7 total, 6 up, 7 in 2024-04-24T08:43:54.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:53 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T08:43:54.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:53 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/691733474' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T08:43:54.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:53 smithi083 ceph-mon[27907]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-04-24T08:43:54.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:53 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/694642481' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8f056584-16c8-4454-b445-3f780f26c196"}]: dispatch 2024-04-24T08:43:54.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:53 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/694642481' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8f056584-16c8-4454-b445-3f780f26c196"}]': finished 2024-04-24T08:43:54.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:53 smithi083 ceph-mon[27907]: osdmap e43: 7 total, 6 up, 7 in 2024-04-24T08:43:54.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:53 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T08:43:54.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:53 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/691733474' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T08:43:56.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:55 smithi138 ceph-mon[31441]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:43:56.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:55 smithi189 ceph-mon[31791]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:43:56.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:55 smithi083 ceph-mon[27907]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:43:57.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:57 smithi189 ceph-mon[31791]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:43:58.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:57 smithi083 ceph-mon[27907]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:43:58.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:57 smithi138 ceph-mon[31441]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:44:00.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:43:59 smithi083 ceph-mon[27907]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:44:00.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:43:59 smithi138 ceph-mon[31441]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:44:00.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:43:59 smithi189 ceph-mon[31791]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:44:02.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:01 smithi083 ceph-mon[27907]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:44:02.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:01 smithi138 ceph-mon[31441]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:44:02.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:01 smithi189 ceph-mon[31791]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:44:04.112 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:03 smithi083 ceph-mon[27907]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:44:04.113 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:03 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-24T08:44:04.113 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:03 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:44:04.113 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:03 smithi083 ceph-mon[27907]: Deploying daemon osd.6 on smithi083 2024-04-24T08:44:04.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:03 smithi138 ceph-mon[31441]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:44:04.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:03 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-24T08:44:04.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:03 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:44:04.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:03 smithi138 ceph-mon[31441]: Deploying daemon osd.6 on smithi083 2024-04-24T08:44:04.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:03 smithi189 ceph-mon[31791]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:44:04.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:03 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-24T08:44:04.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:03 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:44:04.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:03 smithi189 ceph-mon[31791]: Deploying daemon osd.6 on smithi083 2024-04-24T08:44:06.009 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:05 smithi083 ceph-mon[27907]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:44:06.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:05 smithi138 ceph-mon[31441]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:44:06.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:05 smithi189 ceph-mon[31791]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:44:08.025 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:07 smithi083 ceph-mon[27907]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:44:08.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:07 smithi138 ceph-mon[31441]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:44:08.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:07 smithi189 ceph-mon[31791]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:44:10.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:09 smithi138 ceph-mon[31441]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:44:10.206 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:09 smithi083 ceph-mon[27907]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:44:10.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:09 smithi189 ceph-mon[31791]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:44:12.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:11 smithi138 ceph-mon[31441]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:44:12.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:11 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:44:12.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:11 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:44:12.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:11 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:44:12.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:11 smithi189 ceph-mon[31791]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:44:12.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:11 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:44:12.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:11 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:44:12.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:11 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:44:12.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:11 smithi083 ceph-mon[27907]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:44:12.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:11 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:44:12.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:11 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:44:12.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:11 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:44:13.880 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:13 smithi083 ceph-mon[27907]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:44:14.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:13 smithi138 ceph-mon[31441]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:44:14.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:13 smithi189 ceph-mon[31791]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:44:16.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:15 smithi083 ceph-mon[27907]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:44:16.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:15 smithi083 ceph-mon[27907]: from='osd.6 [v2:172.21.15.83:6818/1611358483,v1:172.21.15.83:6819/1611358483]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-24T08:44:16.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:15 smithi138 ceph-mon[31441]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:44:16.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:15 smithi138 ceph-mon[31441]: from='osd.6 [v2:172.21.15.83:6818/1611358483,v1:172.21.15.83:6819/1611358483]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-24T08:44:16.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:15 smithi189 ceph-mon[31791]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:44:16.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:15 smithi189 ceph-mon[31791]: from='osd.6 [v2:172.21.15.83:6818/1611358483,v1:172.21.15.83:6819/1611358483]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-24T08:44:16.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:44:16.757+0000 7f287cc00700 1 -- 172.21.15.83:0/3116711471 <== mgr.14152 v2:172.21.15.83:6800/2550826560 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f2880062f30 con 0x7f28640a3cc0 2024-04-24T08:44:16.758 INFO:teuthology.orchestra.run.smithi083.stdout:Created osd(s) 6 on host 'smithi083' 2024-04-24T08:44:16.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:44:16.761+0000 7f2887200700 1 -- 172.21.15.83:0/3116711471 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f28640a3cc0 msgr2=0x7f28640a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:44:16.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:44:16.761+0000 7f2887200700 1 --2- 172.21.15.83:0/3116711471 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f28640a3cc0 0x7f28640a6180 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f286c0099d0 tx=0x7f286c008040 comp rx=0 tx=0).stop 2024-04-24T08:44:16.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:44:16.761+0000 7f2887200700 1 -- 172.21.15.83:0/3116711471 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2880104d90 msgr2=0x7f288019ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:44:16.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:44:16.761+0000 7f2887200700 1 --2- 172.21.15.83:0/3116711471 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2880104d90 0x7f288019ae30 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f287000c9c0 tx=0x7f287000ccd0 comp rx=0 tx=0).stop 2024-04-24T08:44:16.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:44:16.762+0000 7f2887200700 1 -- 172.21.15.83:0/3116711471 shutdown_connections 2024-04-24T08:44:16.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:44:16.762+0000 7f2887200700 1 --2- 172.21.15.83:0/3116711471 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f28640a3cc0 0x7f28640a6180 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:44:16.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:44:16.762+0000 7f2887200700 1 --2- 172.21.15.83:0/3116711471 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2880104d90 0x7f288019ae30 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:44:16.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:44:16.762+0000 7f2887200700 1 --2- 172.21.15.83:0/3116711471 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2880106930 0x7f28801a23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:44:16.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:44:16.762+0000 7f2887200700 1 --2- 172.21.15.83:0/3116711471 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2880105f90 0x7f288019b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:44:16.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:44:16.762+0000 7f2887200700 1 -- 172.21.15.83:0/3116711471 >> 172.21.15.83:0/3116711471 conn(0x7f2880100350 msgr2=0x7f28801091c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:44:16.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:44:16.762+0000 7f2887200700 1 -- 172.21.15.83:0/3116711471 shutdown_connections 2024-04-24T08:44:16.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:44:16.762+0000 7f2887200700 1 -- 172.21.15.83:0/3116711471 wait complete. 2024-04-24T08:44:17.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:16 smithi083 ceph-mon[27907]: from='osd.6 [v2:172.21.15.83:6818/1611358483,v1:172.21.15.83:6819/1611358483]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-24T08:44:17.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:16 smithi083 ceph-mon[27907]: osdmap e44: 7 total, 6 up, 7 in 2024-04-24T08:44:17.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:16 smithi083 ceph-mon[27907]: from='osd.6 [v2:172.21.15.83:6818/1611358483,v1:172.21.15.83:6819/1611358483]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-24T08:44:17.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:16 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T08:44:17.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:16 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:44:17.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:16 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:44:17.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:16 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:44:17.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:16 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:44:17.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:16 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:44:17.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:16 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:44:17.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:16 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:44:17.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:16 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:44:17.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:16 smithi083 ceph-mon[27907]: from='osd.6 [v2:172.21.15.83:6818/1611358483,v1:172.21.15.83:6819/1611358483]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-04-24T08:44:17.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:16 smithi083 ceph-mon[27907]: osdmap e45: 7 total, 6 up, 7 in 2024-04-24T08:44:17.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:16 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T08:44:17.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:16 smithi138 ceph-mon[31441]: from='osd.6 [v2:172.21.15.83:6818/1611358483,v1:172.21.15.83:6819/1611358483]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-24T08:44:17.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:16 smithi138 ceph-mon[31441]: osdmap e44: 7 total, 6 up, 7 in 2024-04-24T08:44:17.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:16 smithi138 ceph-mon[31441]: from='osd.6 [v2:172.21.15.83:6818/1611358483,v1:172.21.15.83:6819/1611358483]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-24T08:44:17.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:16 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T08:44:17.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:16 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:44:17.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:16 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:44:17.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:16 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:44:17.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:16 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:44:17.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:16 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:44:17.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:16 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:44:17.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:16 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:44:17.207 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:16 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:44:17.207 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:16 smithi138 ceph-mon[31441]: from='osd.6 [v2:172.21.15.83:6818/1611358483,v1:172.21.15.83:6819/1611358483]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-04-24T08:44:17.207 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:16 smithi138 ceph-mon[31441]: osdmap e45: 7 total, 6 up, 7 in 2024-04-24T08:44:17.207 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:16 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T08:44:17.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:16 smithi189 ceph-mon[31791]: from='osd.6 [v2:172.21.15.83:6818/1611358483,v1:172.21.15.83:6819/1611358483]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-24T08:44:17.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:16 smithi189 ceph-mon[31791]: osdmap e44: 7 total, 6 up, 7 in 2024-04-24T08:44:17.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:16 smithi189 ceph-mon[31791]: from='osd.6 [v2:172.21.15.83:6818/1611358483,v1:172.21.15.83:6819/1611358483]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-24T08:44:17.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:16 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T08:44:17.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:16 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:44:17.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:16 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:44:17.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:16 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:44:17.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:16 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:44:17.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:16 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:44:17.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:16 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:44:17.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:16 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:44:17.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:16 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:44:17.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:16 smithi189 ceph-mon[31791]: from='osd.6 [v2:172.21.15.83:6818/1611358483,v1:172.21.15.83:6819/1611358483]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-04-24T08:44:17.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:16 smithi189 ceph-mon[31791]: osdmap e45: 7 total, 6 up, 7 in 2024-04-24T08:44:17.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:16 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T08:44:17.752 DEBUG:teuthology.orchestra.run.smithi083:osd.6> sudo journalctl -f -n 0 -u ceph-bd3e039e-0215-11ef-bc93-c7b262605968@osd.6.service 2024-04-24T08:44:17.756 INFO:tasks.cephadm:Deploying osd.7 on smithi138 with /dev/vg_nvme/lv_2... 2024-04-24T08:44:17.756 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd3e039e-0215-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-24T08:44:17.988 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:17 smithi083 ceph-mon[27907]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:44:17.988 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:17 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T08:44:17.988 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:17 smithi083 ceph-mon[27907]: from='osd.6 [v2:172.21.15.83:6818/1611358483,v1:172.21.15.83:6819/1611358483]' entity='osd.6' 2024-04-24T08:44:17.988 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:17 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T08:44:18.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:17 smithi138 ceph-mon[31441]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:44:18.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:17 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T08:44:18.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:17 smithi138 ceph-mon[31441]: from='osd.6 [v2:172.21.15.83:6818/1611358483,v1:172.21.15.83:6819/1611358483]' entity='osd.6' 2024-04-24T08:44:18.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:17 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T08:44:18.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:17 smithi189 ceph-mon[31791]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:44:18.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:17 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T08:44:18.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:17 smithi189 ceph-mon[31791]: from='osd.6 [v2:172.21.15.83:6818/1611358483,v1:172.21.15.83:6819/1611358483]' entity='osd.6' 2024-04-24T08:44:18.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:17 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T08:44:18.208 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T08:44:19.961 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:19 smithi083 ceph-mon[27907]: purged_snaps scrub starts 2024-04-24T08:44:19.961 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:19 smithi083 ceph-mon[27907]: purged_snaps scrub ok 2024-04-24T08:44:19.962 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:19 smithi083 ceph-mon[27907]: osd.6 [v2:172.21.15.83:6818/1611358483,v1:172.21.15.83:6819/1611358483] boot 2024-04-24T08:44:19.962 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:19 smithi083 ceph-mon[27907]: osdmap e46: 7 total, 7 up, 7 in 2024-04-24T08:44:19.962 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:19 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T08:44:19.962 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:19 smithi083 ceph-mon[27907]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:44:20.045 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:19 smithi138 ceph-mon[31441]: purged_snaps scrub starts 2024-04-24T08:44:20.045 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:19 smithi138 ceph-mon[31441]: purged_snaps scrub ok 2024-04-24T08:44:20.045 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:19 smithi138 ceph-mon[31441]: osd.6 [v2:172.21.15.83:6818/1611358483,v1:172.21.15.83:6819/1611358483] boot 2024-04-24T08:44:20.045 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:19 smithi138 ceph-mon[31441]: osdmap e46: 7 total, 7 up, 7 in 2024-04-24T08:44:20.045 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:19 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T08:44:20.045 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:19 smithi138 ceph-mon[31441]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:44:20.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:19 smithi189 ceph-mon[31791]: purged_snaps scrub starts 2024-04-24T08:44:20.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:19 smithi189 ceph-mon[31791]: purged_snaps scrub ok 2024-04-24T08:44:20.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:19 smithi189 ceph-mon[31791]: osd.6 [v2:172.21.15.83:6818/1611358483,v1:172.21.15.83:6819/1611358483] boot 2024-04-24T08:44:20.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:19 smithi189 ceph-mon[31791]: osdmap e46: 7 total, 7 up, 7 in 2024-04-24T08:44:20.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:19 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T08:44:20.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:19 smithi189 ceph-mon[31791]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-24T08:44:20.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:20 smithi138 ceph-mon[31441]: osdmap e47: 7 total, 7 up, 7 in 2024-04-24T08:44:21.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:20 smithi083 ceph-mon[27907]: osdmap e47: 7 total, 7 up, 7 in 2024-04-24T08:44:21.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:20 smithi189 ceph-mon[31791]: osdmap e47: 7 total, 7 up, 7 in 2024-04-24T08:44:21.289 INFO:teuthology.orchestra.run.smithi138.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-24T08:44:21.289 INFO:teuthology.orchestra.run.smithi138.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-24T08:44:21.289 INFO:teuthology.orchestra.run.smithi138.stderr: stderr: 10+0 records in 2024-04-24T08:44:21.289 INFO:teuthology.orchestra.run.smithi138.stderr:10+0 records out 2024-04-24T08:44:21.289 INFO:teuthology.orchestra.run.smithi138.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0340145 s, 308 MB/s 2024-04-24T08:44:21.289 INFO:teuthology.orchestra.run.smithi138.stderr:--> Zapping successful for: 2024-04-24T08:44:21.923 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:21 smithi138 ceph-mon[31441]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-24T08:44:22.018 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:21 smithi083 ceph-mon[27907]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-24T08:44:22.095 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd3e039e-0215-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi138:vg_nvme/lv_2 2024-04-24T08:44:22.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:21 smithi189 ceph-mon[31791]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-24T08:44:22.530 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T08:44:24.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:23 smithi138 ceph-mon[31441]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-24T08:44:24.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:23 smithi138 ceph-mon[31441]: Detected new or changed devices on smithi083 2024-04-24T08:44:24.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:23 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:44:24.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:23 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:44:24.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:23 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:44:24.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:23 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:44:24.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:23 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:44:24.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:23 smithi138 ceph-mon[31441]: Adjusting osd_memory_target on smithi083 to 5696M 2024-04-24T08:44:24.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:23 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:44:24.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:23 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:44:24.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:23 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:44:24.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:23 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:44:24.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:23 smithi189 ceph-mon[31791]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-24T08:44:24.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:23 smithi189 ceph-mon[31791]: Detected new or changed devices on smithi083 2024-04-24T08:44:24.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:23 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:44:24.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:23 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:44:24.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:23 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:44:24.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:23 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:44:24.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:23 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:44:24.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:23 smithi189 ceph-mon[31791]: Adjusting osd_memory_target on smithi083 to 5696M 2024-04-24T08:44:24.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:23 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:44:24.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:23 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:44:24.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:23 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:44:24.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:23 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:44:24.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:23 smithi083 ceph-mon[27907]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-24T08:44:24.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:23 smithi083 ceph-mon[27907]: Detected new or changed devices on smithi083 2024-04-24T08:44:24.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:23 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:44:24.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:23 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:44:24.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:23 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:44:24.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:23 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:44:24.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:23 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:44:24.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:23 smithi083 ceph-mon[27907]: Adjusting osd_memory_target on smithi083 to 5696M 2024-04-24T08:44:24.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:23 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:44:24.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:23 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:44:24.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:23 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:44:24.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:23 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:44:24.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:44:24.804+0000 7f3d74e00700 1 -- 172.21.15.138:0/1591499885 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d701088a0 msgr2=0x7f3d7010acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:44:24.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:44:24.804+0000 7f3d74e00700 1 --2- 172.21.15.138:0/1591499885 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d701088a0 0x7f3d7010acb0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3d6800b0a0 tx=0x7f3d6800b3b0 comp rx=0 tx=0).stop 2024-04-24T08:44:24.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:44:24.804+0000 7f3d74e00700 1 -- 172.21.15.138:0/1591499885 shutdown_connections 2024-04-24T08:44:24.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:44:24.804+0000 7f3d74e00700 1 --2- 172.21.15.138:0/1591499885 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d70103640 0x7f3d70105a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:44:24.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:44:24.804+0000 7f3d74e00700 1 --2- 172.21.15.138:0/1591499885 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d701088a0 0x7f3d7010acb0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:44:24.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:44:24.804+0000 7f3d74e00700 1 --2- 172.21.15.138:0/1591499885 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3d70105f70 0x7f3d70108360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:44:24.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:44:24.804+0000 7f3d74e00700 1 -- 172.21.15.138:0/1591499885 >> 172.21.15.138:0/1591499885 conn(0x7f3d700fcff0 msgr2=0x7f3d700ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:44:24.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:44:24.804+0000 7f3d74e00700 1 -- 172.21.15.138:0/1591499885 shutdown_connections 2024-04-24T08:44:24.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:44:24.804+0000 7f3d74e00700 1 -- 172.21.15.138:0/1591499885 wait complete. 2024-04-24T08:44:24.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:44:24.805+0000 7f3d74e00700 1 Processor -- start 2024-04-24T08:44:24.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:44:24.805+0000 7f3d74e00700 1 -- start start 2024-04-24T08:44:24.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:44:24.806+0000 7f3d74e00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d70103640 0x7f3d70198d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:44:24.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:44:24.806+0000 7f3d74e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3d70105f70 0x7f3d701992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:44:24.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:44:24.806+0000 7f3d74e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d701088a0 0x7f3d701a0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:44:24.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:44:24.806+0000 7f3d74e00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3d7010e680 con 0x7f3d701088a0 2024-04-24T08:44:24.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:44:24.806+0000 7f3d74e00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3d7010e500 con 0x7f3d70103640 2024-04-24T08:44:24.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:44:24.806+0000 7f3d74e00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3d7010e800 con 0x7f3d70105f70 2024-04-24T08:44:24.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:44:24.806+0000 7f3d6de00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d70103640 0x7f3d70198d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:44:24.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:44:24.806+0000 7f3d6de00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d70103640 0x7f3d70198d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:45828/0 (socket says 172.21.15.138:45828) 2024-04-24T08:44:24.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:44:24.806+0000 7f3d6de00700 1 -- 172.21.15.138:0/4221762783 learned_addr learned my addr 172.21.15.138:0/4221762783 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-24T08:44:24.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:44:24.806+0000 7f3d6d400700 1 --2- 172.21.15.138:0/4221762783 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3d70105f70 0x7f3d701992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:44:24.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:44:24.806+0000 7f3d6e800700 1 --2- 172.21.15.138:0/4221762783 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d701088a0 0x7f3d701a0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:44:24.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:44:24.807+0000 7f3d6de00700 1 -- 172.21.15.138:0/4221762783 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3d70105f70 msgr2=0x7f3d701992d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:44:24.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:44:24.807+0000 7f3d6de00700 1 --2- 172.21.15.138:0/4221762783 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3d70105f70 0x7f3d701992d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:44:24.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:44:24.807+0000 7f3d6de00700 1 -- 172.21.15.138:0/4221762783 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d701088a0 msgr2=0x7f3d701a0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:44:24.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:44:24.807+0000 7f3d6de00700 1 --2- 172.21.15.138:0/4221762783 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d701088a0 0x7f3d701a0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:44:24.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:44:24.807+0000 7f3d6de00700 1 -- 172.21.15.138:0/4221762783 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d701a09a0 con 0x7f3d70103640 2024-04-24T08:44:24.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:44:24.807+0000 7f3d6e800700 1 --2- 172.21.15.138:0/4221762783 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d701088a0 0x7f3d701a0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:44:24.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:44:24.807+0000 7f3d6de00700 1 --2- 172.21.15.138:0/4221762783 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d70103640 0x7f3d70198d90 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3d6000c920 tx=0x7f3d6000cce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:44:24.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:44:24.807+0000 7f3d66a00700 1 -- 172.21.15.138:0/4221762783 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d60007b00 con 0x7f3d70103640 2024-04-24T08:44:24.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:44:24.807+0000 7f3d66a00700 1 -- 172.21.15.138:0/4221762783 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3d60004ce0 con 0x7f3d70103640 2024-04-24T08:44:24.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:44:24.807+0000 7f3d74e00700 1 -- 172.21.15.138:0/4221762783 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d70101490 con 0x7f3d70103640 2024-04-24T08:44:24.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:44:24.807+0000 7f3d66a00700 1 -- 172.21.15.138:0/4221762783 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d60005600 con 0x7f3d70103640 2024-04-24T08:44:24.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:44:24.808+0000 7f3d74e00700 1 -- 172.21.15.138:0/4221762783 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d70101aa0 con 0x7f3d70103640 2024-04-24T08:44:24.810 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:44:24.809+0000 7f3d74e00700 1 -- 172.21.15.138:0/4221762783 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d700697e0 con 0x7f3d70103640 2024-04-24T08:44:24.810 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:44:24.811+0000 7f3d66a00700 1 -- 172.21.15.138:0/4221762783 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f3d60004750 con 0x7f3d70103640 2024-04-24T08:44:24.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:44:24.812+0000 7f3d66a00700 1 --2- 172.21.15.138:0/4221762783 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f3d540a3cc0 0x7f3d540a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:44:24.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:44:24.812+0000 7f3d66a00700 1 -- 172.21.15.138:0/4221762783 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 4965+0+0 (secure 0 0 0) 0x7f3d600cf730 con 0x7f3d70103640 2024-04-24T08:44:24.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:44:24.813+0000 7f3d6d400700 1 --2- 172.21.15.138:0/4221762783 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f3d540a3cc0 0x7f3d540a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:44:24.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:44:24.814+0000 7f3d6d400700 1 --2- 172.21.15.138:0/4221762783 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f3d540a3cc0 0x7f3d540a6180 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c006fd0 tx=0x7f3d5c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:44:24.816 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:44:24.816+0000 7f3d66a00700 1 -- 172.21.15.138:0/4221762783 <== mon.2 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3d60014030 con 0x7f3d70103640 2024-04-24T08:44:25.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:44:25.054+0000 7f3d74e00700 1 -- 172.21.15.138:0/4221762783 --> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f3d700fe5f0 con 0x7f3d540a3cc0 2024-04-24T08:44:26.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:25 smithi138 ceph-mon[31441]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-24T08:44:26.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:25 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T08:44:26.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:25 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T08:44:26.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:25 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:44:26.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:25 smithi189 ceph-mon[31791]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-24T08:44:26.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:25 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T08:44:26.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:25 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T08:44:26.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:25 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:44:26.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:25 smithi083 ceph-mon[27907]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-24T08:44:26.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:25 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T08:44:26.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:25 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T08:44:26.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:25 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:44:27.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:26 smithi138 ceph-mon[31441]: from='client.34274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:44:27.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:26 smithi189 ceph-mon[31791]: from='client.34274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:44:27.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:26 smithi083 ceph-mon[27907]: from='client.34274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:44:28.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:27 smithi138 ceph-mon[31441]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-24T08:44:28.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:27 smithi189 ceph-mon[31791]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-24T08:44:28.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:27 smithi083 ceph-mon[27907]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-24T08:44:30.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:29 smithi138 ceph-mon[31441]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-24T08:44:30.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:29 smithi138 ceph-mon[31441]: from='client.? 172.21.15.138:0/1326068770' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7603efe3-da77-4b7f-9d03-29c4f293b6d0"}]: dispatch 2024-04-24T08:44:30.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:29 smithi138 ceph-mon[31441]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7603efe3-da77-4b7f-9d03-29c4f293b6d0"}]: dispatch 2024-04-24T08:44:30.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:29 smithi138 ceph-mon[31441]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7603efe3-da77-4b7f-9d03-29c4f293b6d0"}]': finished 2024-04-24T08:44:30.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:29 smithi138 ceph-mon[31441]: osdmap e48: 8 total, 7 up, 8 in 2024-04-24T08:44:30.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:29 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T08:44:30.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:29 smithi189 ceph-mon[31791]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-24T08:44:30.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:29 smithi189 ceph-mon[31791]: from='client.? 172.21.15.138:0/1326068770' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7603efe3-da77-4b7f-9d03-29c4f293b6d0"}]: dispatch 2024-04-24T08:44:30.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:29 smithi189 ceph-mon[31791]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7603efe3-da77-4b7f-9d03-29c4f293b6d0"}]: dispatch 2024-04-24T08:44:30.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:29 smithi189 ceph-mon[31791]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7603efe3-da77-4b7f-9d03-29c4f293b6d0"}]': finished 2024-04-24T08:44:30.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:29 smithi189 ceph-mon[31791]: osdmap e48: 8 total, 7 up, 8 in 2024-04-24T08:44:30.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:29 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T08:44:30.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:29 smithi083 ceph-mon[27907]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-24T08:44:30.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:29 smithi083 ceph-mon[27907]: from='client.? 172.21.15.138:0/1326068770' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7603efe3-da77-4b7f-9d03-29c4f293b6d0"}]: dispatch 2024-04-24T08:44:30.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:29 smithi083 ceph-mon[27907]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7603efe3-da77-4b7f-9d03-29c4f293b6d0"}]: dispatch 2024-04-24T08:44:30.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:29 smithi083 ceph-mon[27907]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7603efe3-da77-4b7f-9d03-29c4f293b6d0"}]': finished 2024-04-24T08:44:30.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:29 smithi083 ceph-mon[27907]: osdmap e48: 8 total, 7 up, 8 in 2024-04-24T08:44:30.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:29 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T08:44:31.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:30 smithi138 ceph-mon[31441]: from='client.? 172.21.15.138:0/3772264327' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T08:44:31.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:30 smithi189 ceph-mon[31791]: from='client.? 172.21.15.138:0/3772264327' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T08:44:31.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:30 smithi083 ceph-mon[27907]: from='client.? 172.21.15.138:0/3772264327' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T08:44:32.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:31 smithi138 ceph-mon[31441]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-24T08:44:32.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:31 smithi189 ceph-mon[31791]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-24T08:44:32.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:31 smithi083 ceph-mon[27907]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-24T08:44:34.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:33 smithi138 ceph-mon[31441]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-24T08:44:34.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:33 smithi083 ceph-mon[27907]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-24T08:44:34.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:33 smithi189 ceph-mon[31791]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-24T08:44:35.273 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:34 smithi138 ceph-mon[31441]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-24T08:44:35.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:34 smithi083 ceph-mon[27907]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-24T08:44:35.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:34 smithi189 ceph-mon[31791]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-24T08:44:37.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:37 smithi138 ceph-mon[31441]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-24T08:44:38.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:37 smithi189 ceph-mon[31791]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-24T08:44:38.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:37 smithi083 ceph-mon[27907]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-24T08:44:39.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:38 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-24T08:44:39.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:38 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:44:39.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:38 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-24T08:44:39.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:38 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:44:39.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:38 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-24T08:44:39.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:38 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:44:40.205 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:39 smithi083 ceph-mon[27907]: Deploying daemon osd.7 on smithi138 2024-04-24T08:44:40.205 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:39 smithi083 ceph-mon[27907]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-24T08:44:40.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:39 smithi138 ceph-mon[31441]: Deploying daemon osd.7 on smithi138 2024-04-24T08:44:40.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:39 smithi138 ceph-mon[31441]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-24T08:44:40.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:39 smithi189 ceph-mon[31791]: Deploying daemon osd.7 on smithi138 2024-04-24T08:44:40.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:39 smithi189 ceph-mon[31791]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-24T08:44:42.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:41 smithi138 ceph-mon[31441]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-24T08:44:42.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:41 smithi189 ceph-mon[31791]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-24T08:44:42.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:41 smithi083 ceph-mon[27907]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-24T08:44:44.114 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:43 smithi138 ceph-mon[31441]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-24T08:44:44.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:43 smithi189 ceph-mon[31791]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-24T08:44:44.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:43 smithi083 ceph-mon[27907]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-24T08:44:46.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:45 smithi138 ceph-mon[31441]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-24T08:44:46.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:45 smithi189 ceph-mon[31791]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-24T08:44:46.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:45 smithi083 ceph-mon[27907]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-24T08:44:48.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:47 smithi138 ceph-mon[31441]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-24T08:44:48.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:47 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:44:48.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:47 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:44:48.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:47 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:44:48.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:47 smithi189 ceph-mon[31791]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-24T08:44:48.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:47 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:44:48.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:47 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:44:48.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:47 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:44:48.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:47 smithi083 ceph-mon[27907]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-24T08:44:48.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:47 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:44:48.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:47 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:44:48.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:47 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:44:50.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:49 smithi138 ceph-mon[31441]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-24T08:44:50.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:49 smithi189 ceph-mon[31791]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-24T08:44:50.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:49 smithi083 ceph-mon[27907]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-24T08:44:52.070 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:51 smithi138 ceph-mon[31441]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-24T08:44:52.071 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:51 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:44:52.071 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:51 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:44:52.071 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:51 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:44:52.071 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:51 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:44:52.071 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:51 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:44:52.071 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:51 smithi138 ceph-mon[31441]: from='osd.7 [v2:172.21.15.138:6816/3557856706,v1:172.21.15.138:6817/3557856706]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-24T08:44:52.071 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:51 smithi138 ceph-mon[31441]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-24T08:44:52.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:51 smithi189 ceph-mon[31791]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-24T08:44:52.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:51 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:44:52.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:51 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:44:52.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:51 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:44:52.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:51 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:44:52.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:51 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:44:52.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:51 smithi189 ceph-mon[31791]: from='osd.7 [v2:172.21.15.138:6816/3557856706,v1:172.21.15.138:6817/3557856706]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-24T08:44:52.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:51 smithi189 ceph-mon[31791]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-24T08:44:52.288 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:51 smithi083 ceph-mon[27907]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-24T08:44:52.288 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:51 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:44:52.288 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:51 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:44:52.288 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:51 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:44:52.289 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:51 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:44:52.289 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:51 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:44:52.289 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:51 smithi083 ceph-mon[27907]: from='osd.7 [v2:172.21.15.138:6816/3557856706,v1:172.21.15.138:6817/3557856706]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-24T08:44:52.289 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:51 smithi083 ceph-mon[27907]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-24T08:44:52.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:44:52.315+0000 7f3d66a00700 1 -- 172.21.15.138:0/4221762783 <== mgr.14152 v2:172.21.15.83:6800/2550826560 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f3d700fe5f0 con 0x7f3d540a3cc0 2024-04-24T08:44:52.317 INFO:teuthology.orchestra.run.smithi138.stdout:Created osd(s) 7 on host 'smithi138' 2024-04-24T08:44:52.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:44:52.320+0000 7f3d74e00700 1 -- 172.21.15.138:0/4221762783 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f3d540a3cc0 msgr2=0x7f3d540a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:44:52.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:44:52.320+0000 7f3d74e00700 1 --2- 172.21.15.138:0/4221762783 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f3d540a3cc0 0x7f3d540a6180 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c006fd0 tx=0x7f3d5c008040 comp rx=0 tx=0).stop 2024-04-24T08:44:52.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:44:52.320+0000 7f3d74e00700 1 -- 172.21.15.138:0/4221762783 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d70103640 msgr2=0x7f3d70198d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:44:52.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:44:52.320+0000 7f3d74e00700 1 --2- 172.21.15.138:0/4221762783 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d70103640 0x7f3d70198d90 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3d6000c920 tx=0x7f3d6000cce0 comp rx=0 tx=0).stop 2024-04-24T08:44:52.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:44:52.320+0000 7f3d74e00700 1 -- 172.21.15.138:0/4221762783 shutdown_connections 2024-04-24T08:44:52.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:44:52.320+0000 7f3d74e00700 1 --2- 172.21.15.138:0/4221762783 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f3d540a3cc0 0x7f3d540a6180 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:44:52.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:44:52.320+0000 7f3d74e00700 1 --2- 172.21.15.138:0/4221762783 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d701088a0 0x7f3d701a0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:44:52.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:44:52.320+0000 7f3d74e00700 1 --2- 172.21.15.138:0/4221762783 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d70103640 0x7f3d70198d90 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:44:52.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:44:52.320+0000 7f3d74e00700 1 --2- 172.21.15.138:0/4221762783 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3d70105f70 0x7f3d701992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:44:52.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:44:52.320+0000 7f3d74e00700 1 -- 172.21.15.138:0/4221762783 >> 172.21.15.138:0/4221762783 conn(0x7f3d700fcff0 msgr2=0x7f3d70104060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:44:52.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:44:52.320+0000 7f3d74e00700 1 -- 172.21.15.138:0/4221762783 shutdown_connections 2024-04-24T08:44:52.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:44:52.320+0000 7f3d74e00700 1 -- 172.21.15.138:0/4221762783 wait complete. 2024-04-24T08:44:52.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:52 smithi138 ceph-mon[31441]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-24T08:44:52.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:52 smithi138 ceph-mon[31441]: osdmap e49: 8 total, 7 up, 8 in 2024-04-24T08:44:52.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:52 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T08:44:52.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:52 smithi138 ceph-mon[31441]: from='osd.7 [v2:172.21.15.138:6816/3557856706,v1:172.21.15.138:6817/3557856706]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-04-24T08:44:52.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:52 smithi138 ceph-mon[31441]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-04-24T08:44:52.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:52 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:44:52.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:52 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:44:52.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:52 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:44:53.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:52 smithi189 ceph-mon[31791]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-24T08:44:53.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:52 smithi189 ceph-mon[31791]: osdmap e49: 8 total, 7 up, 8 in 2024-04-24T08:44:53.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:52 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T08:44:53.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:52 smithi189 ceph-mon[31791]: from='osd.7 [v2:172.21.15.138:6816/3557856706,v1:172.21.15.138:6817/3557856706]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-04-24T08:44:53.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:52 smithi189 ceph-mon[31791]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-04-24T08:44:53.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:52 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:44:53.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:52 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:44:53.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:52 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:44:53.241 DEBUG:teuthology.orchestra.run.smithi138:osd.7> sudo journalctl -f -n 0 -u ceph-bd3e039e-0215-11ef-bc93-c7b262605968@osd.7.service 2024-04-24T08:44:53.247 INFO:tasks.cephadm:Deploying osd.8 on smithi189 with /dev/vg_nvme/lv_2... 2024-04-24T08:44:53.247 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd3e039e-0215-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-24T08:44:53.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:52 smithi083 ceph-mon[27907]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-24T08:44:53.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:52 smithi083 ceph-mon[27907]: osdmap e49: 8 total, 7 up, 8 in 2024-04-24T08:44:53.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:52 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T08:44:53.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:52 smithi083 ceph-mon[27907]: from='osd.7 [v2:172.21.15.138:6816/3557856706,v1:172.21.15.138:6817/3557856706]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-04-24T08:44:53.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:52 smithi083 ceph-mon[27907]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-04-24T08:44:53.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:52 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:44:53.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:52 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:44:53.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:52 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:44:53.684 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T08:44:54.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:53 smithi189 ceph-mon[31791]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-24T08:44:54.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:53 smithi189 ceph-mon[31791]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-04-24T08:44:54.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:53 smithi189 ceph-mon[31791]: osdmap e50: 8 total, 7 up, 8 in 2024-04-24T08:44:54.086 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:53 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T08:44:54.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:53 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T08:44:54.087 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:53 smithi189 ceph-mon[31791]: from='osd.7 ' entity='osd.7' 2024-04-24T08:44:54.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:53 smithi138 ceph-mon[31441]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-24T08:44:54.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:53 smithi138 ceph-mon[31441]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-04-24T08:44:54.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:53 smithi138 ceph-mon[31441]: osdmap e50: 8 total, 7 up, 8 in 2024-04-24T08:44:54.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:53 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T08:44:54.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:53 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T08:44:54.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:53 smithi138 ceph-mon[31441]: from='osd.7 ' entity='osd.7' 2024-04-24T08:44:54.206 INFO:journalctl@ceph.osd.7.smithi138.stdout:Apr 24 08:44:53 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-7[49166]: 2024-04-24T08:44:53.739+0000 7fb4a3600700 -1 osd.7 0 waiting for initial osdmap 2024-04-24T08:44:54.206 INFO:journalctl@ceph.osd.7.smithi138.stdout:Apr 24 08:44:53 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-7[49166]: 2024-04-24T08:44:53.765+0000 7fb499400700 -1 osd.7 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T08:44:54.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:53 smithi083 ceph-mon[27907]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-24T08:44:54.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:53 smithi083 ceph-mon[27907]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-04-24T08:44:54.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:53 smithi083 ceph-mon[27907]: osdmap e50: 8 total, 7 up, 8 in 2024-04-24T08:44:54.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:53 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T08:44:54.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:53 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T08:44:54.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:53 smithi083 ceph-mon[27907]: from='osd.7 ' entity='osd.7' 2024-04-24T08:44:55.041 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:54 smithi138 ceph-mon[31441]: purged_snaps scrub starts 2024-04-24T08:44:55.041 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:54 smithi138 ceph-mon[31441]: purged_snaps scrub ok 2024-04-24T08:44:55.041 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:54 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T08:44:55.041 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:54 smithi138 ceph-mon[31441]: osd.7 [v2:172.21.15.138:6816/3557856706,v1:172.21.15.138:6817/3557856706] boot 2024-04-24T08:44:55.041 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:54 smithi138 ceph-mon[31441]: osdmap e51: 8 total, 8 up, 8 in 2024-04-24T08:44:55.041 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:54 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T08:44:55.041 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:54 smithi138 ceph-mon[31441]: osdmap e52: 8 total, 8 up, 8 in 2024-04-24T08:44:55.178 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:54 smithi189 ceph-mon[31791]: purged_snaps scrub starts 2024-04-24T08:44:55.179 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:54 smithi189 ceph-mon[31791]: purged_snaps scrub ok 2024-04-24T08:44:55.179 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:54 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T08:44:55.179 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:54 smithi189 ceph-mon[31791]: osd.7 [v2:172.21.15.138:6816/3557856706,v1:172.21.15.138:6817/3557856706] boot 2024-04-24T08:44:55.179 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:54 smithi189 ceph-mon[31791]: osdmap e51: 8 total, 8 up, 8 in 2024-04-24T08:44:55.179 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:54 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T08:44:55.179 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:54 smithi189 ceph-mon[31791]: osdmap e52: 8 total, 8 up, 8 in 2024-04-24T08:44:55.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:54 smithi083 ceph-mon[27907]: purged_snaps scrub starts 2024-04-24T08:44:55.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:54 smithi083 ceph-mon[27907]: purged_snaps scrub ok 2024-04-24T08:44:55.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:54 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T08:44:55.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:54 smithi083 ceph-mon[27907]: osd.7 [v2:172.21.15.138:6816/3557856706,v1:172.21.15.138:6817/3557856706] boot 2024-04-24T08:44:55.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:54 smithi083 ceph-mon[27907]: osdmap e51: 8 total, 8 up, 8 in 2024-04-24T08:44:55.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:54 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T08:44:55.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:54 smithi083 ceph-mon[27907]: osdmap e52: 8 total, 8 up, 8 in 2024-04-24T08:44:56.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:55 smithi138 ceph-mon[31441]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-24T08:44:56.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:55 smithi138 ceph-mon[31441]: osdmap e53: 8 total, 8 up, 8 in 2024-04-24T08:44:56.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:55 smithi189 ceph-mon[31791]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-24T08:44:56.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:55 smithi189 ceph-mon[31791]: osdmap e53: 8 total, 8 up, 8 in 2024-04-24T08:44:56.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:55 smithi083 ceph-mon[27907]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-24T08:44:56.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:55 smithi083 ceph-mon[27907]: osdmap e53: 8 total, 8 up, 8 in 2024-04-24T08:44:56.758 INFO:teuthology.orchestra.run.smithi189.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-24T08:44:56.759 INFO:teuthology.orchestra.run.smithi189.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-24T08:44:56.759 INFO:teuthology.orchestra.run.smithi189.stderr: stderr: 10+0 records in 2024-04-24T08:44:56.759 INFO:teuthology.orchestra.run.smithi189.stderr:10+0 records out 2024-04-24T08:44:56.759 INFO:teuthology.orchestra.run.smithi189.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0324206 s, 323 MB/s 2024-04-24T08:44:56.759 INFO:teuthology.orchestra.run.smithi189.stderr:--> Zapping successful for: 2024-04-24T08:44:57.179 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:57 smithi189 ceph-mon[31791]: osdmap e54: 8 total, 8 up, 8 in 2024-04-24T08:44:57.179 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:57 smithi189 ceph-mon[31791]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-04-24T08:44:57.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:57 smithi138 ceph-mon[31441]: osdmap e54: 8 total, 8 up, 8 in 2024-04-24T08:44:57.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:57 smithi138 ceph-mon[31441]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-04-24T08:44:57.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:57 smithi083 ceph-mon[27907]: osdmap e54: 8 total, 8 up, 8 in 2024-04-24T08:44:57.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:57 smithi083 ceph-mon[27907]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-04-24T08:44:57.795 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd3e039e-0215-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi189:vg_nvme/lv_2 2024-04-24T08:44:58.220 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T08:44:58.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:58 smithi138 ceph-mon[31441]: osdmap e55: 8 total, 8 up, 8 in 2024-04-24T08:44:58.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:58 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:44:58.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:58 smithi189 ceph-mon[31791]: osdmap e55: 8 total, 8 up, 8 in 2024-04-24T08:44:58.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:58 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:44:58.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:58 smithi083 ceph-mon[27907]: osdmap e55: 8 total, 8 up, 8 in 2024-04-24T08:44:58.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:58 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:44:59.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:59 smithi138 ceph-mon[31441]: Detected new or changed devices on smithi138 2024-04-24T08:44:59.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:59 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:44:59.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:59 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:44:59.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:59 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:44:59.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:59 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:44:59.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:59 smithi138 ceph-mon[31441]: Adjusting osd_memory_target on smithi138 to 5696M 2024-04-24T08:44:59.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:59 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:44:59.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:59 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:44:59.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:59 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:44:59.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:59 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:44:59.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:44:59 smithi138 ceph-mon[31441]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-04-24T08:44:59.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:59 smithi189 ceph-mon[31791]: Detected new or changed devices on smithi138 2024-04-24T08:44:59.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:59 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:44:59.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:59 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:44:59.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:59 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:44:59.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:59 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:44:59.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:59 smithi189 ceph-mon[31791]: Adjusting osd_memory_target on smithi138 to 5696M 2024-04-24T08:44:59.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:59 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:44:59.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:59 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:44:59.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:59 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:44:59.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:59 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:44:59.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:44:59 smithi189 ceph-mon[31791]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-04-24T08:44:59.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:59 smithi083 ceph-mon[27907]: Detected new or changed devices on smithi138 2024-04-24T08:44:59.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:59 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:44:59.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:59 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:44:59.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:59 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:44:59.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:59 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:44:59.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:59 smithi083 ceph-mon[27907]: Adjusting osd_memory_target on smithi138 to 5696M 2024-04-24T08:44:59.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:59 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:44:59.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:59 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:44:59.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:59 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:44:59.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:59 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:44:59.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:44:59 smithi083 ceph-mon[27907]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-04-24T08:45:00.445 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:45:00.444+0000 7f3b09800700 1 -- 172.21.15.189:0/2484389092 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3b04102b50 msgr2=0x7f3b04102f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:45:00.445 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:45:00.444+0000 7f3b09800700 1 --2- 172.21.15.189:0/2484389092 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3b04102b50 0x7f3b04102f70 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f3af4009990 tx=0x7f3af4009ca0 comp rx=0 tx=0).stop 2024-04-24T08:45:00.445 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:45:00.444+0000 7f3b09800700 1 -- 172.21.15.189:0/2484389092 shutdown_connections 2024-04-24T08:45:00.445 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:45:00.444+0000 7f3b09800700 1 --2- 172.21.15.189:0/2484389092 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b041046f0 0x7f3b0410afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:45:00.445 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:45:00.444+0000 7f3b09800700 1 --2- 172.21.15.189:0/2484389092 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3b04102b50 0x7f3b04102f70 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:45:00.445 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:45:00.444+0000 7f3b09800700 1 --2- 172.21.15.189:0/2484389092 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b04103d50 0x7f3b041041b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:45:00.446 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:45:00.444+0000 7f3b09800700 1 -- 172.21.15.189:0/2484389092 >> 172.21.15.189:0/2484389092 conn(0x7f3b040fe0f0 msgr2=0x7f3b04100530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:45:00.446 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:45:00.444+0000 7f3b09800700 1 -- 172.21.15.189:0/2484389092 shutdown_connections 2024-04-24T08:45:00.446 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:45:00.444+0000 7f3b09800700 1 -- 172.21.15.189:0/2484389092 wait complete. 2024-04-24T08:45:00.446 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:45:00.445+0000 7f3b09800700 1 Processor -- start 2024-04-24T08:45:00.446 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:45:00.445+0000 7f3b09800700 1 -- start start 2024-04-24T08:45:00.447 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:45:00.445+0000 7f3b09800700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b04102b50 0x7f3b0419af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:45:00.448 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:45:00.445+0000 7f3b09800700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3b04103d50 0x7f3b0419b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:45:00.448 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:45:00.445+0000 7f3b09800700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b041046f0 0x7f3b041a24f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:45:00.448 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:45:00.445+0000 7f3b09800700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3b0410e630 con 0x7f3b04102b50 2024-04-24T08:45:00.448 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:45:00.445+0000 7f3b09800700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3b0410e4b0 con 0x7f3b041046f0 2024-04-24T08:45:00.448 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:45:00.445+0000 7f3b09800700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3b0410e7b0 con 0x7f3b04103d50 2024-04-24T08:45:00.448 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:45:00.446+0000 7f3b02000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3b04103d50 0x7f3b0419b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:45:00.448 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:45:00.446+0000 7f3b02000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3b04103d50 0x7f3b0419b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.189:45362/0 (socket says 172.21.15.189:45362) 2024-04-24T08:45:00.448 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:45:00.446+0000 7f3b02000700 1 -- 172.21.15.189:0/3161948177 learned_addr learned my addr 172.21.15.189:0/3161948177 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-04-24T08:45:00.448 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:45:00.446+0000 7f3b03400700 1 --2- 172.21.15.189:0/3161948177 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b041046f0 0x7f3b041a24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:45:00.448 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:45:00.446+0000 7f3b02a00700 1 --2- 172.21.15.189:0/3161948177 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b04102b50 0x7f3b0419af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:45:00.448 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:45:00.446+0000 7f3b02000700 1 -- 172.21.15.189:0/3161948177 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b041046f0 msgr2=0x7f3b041a24f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:45:00.448 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:45:00.446+0000 7f3b02000700 1 --2- 172.21.15.189:0/3161948177 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b041046f0 0x7f3b041a24f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:45:00.449 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:45:00.446+0000 7f3b02000700 1 -- 172.21.15.189:0/3161948177 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b04102b50 msgr2=0x7f3b0419af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:45:00.449 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:45:00.446+0000 7f3b02000700 1 --2- 172.21.15.189:0/3161948177 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b04102b50 0x7f3b0419af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:45:00.449 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:45:00.446+0000 7f3b02000700 1 -- 172.21.15.189:0/3161948177 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b041a2b90 con 0x7f3b04103d50 2024-04-24T08:45:00.449 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:45:00.446+0000 7f3b02a00700 1 --2- 172.21.15.189:0/3161948177 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b04102b50 0x7f3b0419af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:45:00.449 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:45:00.447+0000 7f3b02000700 1 --2- 172.21.15.189:0/3161948177 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3b04103d50 0x7f3b0419b4c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f3af000c950 tx=0x7f3af000cd10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:45:00.449 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:45:00.447+0000 7f3afb400700 1 -- 172.21.15.189:0/3161948177 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af0007ac0 con 0x7f3b04103d50 2024-04-24T08:45:00.449 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:45:00.447+0000 7f3afb400700 1 -- 172.21.15.189:0/3161948177 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3af0004ce0 con 0x7f3b04103d50 2024-04-24T08:45:00.449 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:45:00.447+0000 7f3afb400700 1 -- 172.21.15.189:0/3161948177 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af00055e0 con 0x7f3b04103d50 2024-04-24T08:45:00.449 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:45:00.447+0000 7f3b09800700 1 -- 172.21.15.189:0/3161948177 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b041a2e80 con 0x7f3b04103d50 2024-04-24T08:45:00.449 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:45:00.447+0000 7f3b09800700 1 -- 172.21.15.189:0/3161948177 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b041a3380 con 0x7f3b04103d50 2024-04-24T08:45:00.449 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:45:00.448+0000 7f3b09800700 1 -- 172.21.15.189:0/3161948177 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b04069810 con 0x7f3b04103d50 2024-04-24T08:45:00.450 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:45:00.450+0000 7f3afb400700 1 -- 172.21.15.189:0/3161948177 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f3af0004750 con 0x7f3b04103d50 2024-04-24T08:45:00.452 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:45:00.452+0000 7f3afb400700 1 --2- 172.21.15.189:0/3161948177 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f3ae80a3cc0 0x7f3ae80a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:45:00.452 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:45:00.452+0000 7f3afb400700 1 -- 172.21.15.189:0/3161948177 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5397+0+0 (secure 0 0 0) 0x7f3af00cf8c0 con 0x7f3b04103d50 2024-04-24T08:45:00.452 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:45:00.452+0000 7f3b02a00700 1 --2- 172.21.15.189:0/3161948177 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f3ae80a3cc0 0x7f3ae80a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:45:00.453 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:45:00.453+0000 7f3b02a00700 1 --2- 172.21.15.189:0/3161948177 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f3ae80a3cc0 0x7f3ae80a6180 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3af4006010 tx=0x7f3af4005c60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:45:00.455 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:45:00.455+0000 7f3afb400700 1 -- 172.21.15.189:0/3161948177 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3b04069810 con 0x7f3b04103d50 2024-04-24T08:45:00.689 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:45:00.689+0000 7f3b09800700 1 -- 172.21.15.189:0/3161948177 --> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f3b04076440 con 0x7f3ae80a3cc0 2024-04-24T08:45:00.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:45:00 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T08:45:00.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:45:00 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T08:45:00.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:45:00 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:45:01.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:45:00 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T08:45:01.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:45:00 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T08:45:01.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:45:00 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:45:01.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:45:00 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T08:45:01.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:45:00 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T08:45:01.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:45:00 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:45:01.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:45:01 smithi138 ceph-mon[31441]: from='client.24280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:45:01.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:45:01 smithi138 ceph-mon[31441]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-04-24T08:45:01.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:45:01 smithi189 ceph-mon[31791]: from='client.24280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:45:01.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:45:01 smithi189 ceph-mon[31791]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-04-24T08:45:02.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:45:01 smithi083 ceph-mon[27907]: from='client.24280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:45:02.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:45:01 smithi083 ceph-mon[27907]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-04-24T08:45:03.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:45:03 smithi189 ceph-mon[31791]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-04-24T08:45:04.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:45:03 smithi138 ceph-mon[31441]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-04-24T08:45:04.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:45:03 smithi083 ceph-mon[27907]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-04-24T08:45:05.914 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:45:05 smithi138 ceph-mon[31441]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-04-24T08:45:05.915 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:45:05 smithi138 ceph-mon[31441]: from='client.? 172.21.15.189:0/1767360565' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "27d7d1b8-d6a0-417e-b8aa-596325aa1a77"}]: dispatch 2024-04-24T08:45:05.915 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:45:05 smithi138 ceph-mon[31441]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "27d7d1b8-d6a0-417e-b8aa-596325aa1a77"}]: dispatch 2024-04-24T08:45:05.915 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:45:05 smithi138 ceph-mon[31441]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "27d7d1b8-d6a0-417e-b8aa-596325aa1a77"}]': finished 2024-04-24T08:45:05.915 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:45:05 smithi138 ceph-mon[31441]: osdmap e56: 9 total, 8 up, 9 in 2024-04-24T08:45:05.915 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:45:05 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T08:45:06.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:45:05 smithi189 ceph-mon[31791]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-04-24T08:45:06.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:45:05 smithi189 ceph-mon[31791]: from='client.? 172.21.15.189:0/1767360565' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "27d7d1b8-d6a0-417e-b8aa-596325aa1a77"}]: dispatch 2024-04-24T08:45:06.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:45:05 smithi189 ceph-mon[31791]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "27d7d1b8-d6a0-417e-b8aa-596325aa1a77"}]: dispatch 2024-04-24T08:45:06.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:45:05 smithi189 ceph-mon[31791]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "27d7d1b8-d6a0-417e-b8aa-596325aa1a77"}]': finished 2024-04-24T08:45:06.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:45:05 smithi189 ceph-mon[31791]: osdmap e56: 9 total, 8 up, 9 in 2024-04-24T08:45:06.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:45:05 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T08:45:06.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:45:05 smithi083 ceph-mon[27907]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-04-24T08:45:06.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:45:05 smithi083 ceph-mon[27907]: from='client.? 172.21.15.189:0/1767360565' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "27d7d1b8-d6a0-417e-b8aa-596325aa1a77"}]: dispatch 2024-04-24T08:45:06.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:45:05 smithi083 ceph-mon[27907]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "27d7d1b8-d6a0-417e-b8aa-596325aa1a77"}]: dispatch 2024-04-24T08:45:06.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:45:05 smithi083 ceph-mon[27907]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "27d7d1b8-d6a0-417e-b8aa-596325aa1a77"}]': finished 2024-04-24T08:45:06.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:45:05 smithi083 ceph-mon[27907]: osdmap e56: 9 total, 8 up, 9 in 2024-04-24T08:45:06.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:45:05 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T08:45:07.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:45:06 smithi138 ceph-mon[31441]: from='client.? 172.21.15.189:0/2478188580' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T08:45:07.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:45:06 smithi189 ceph-mon[31791]: from='client.? 172.21.15.189:0/2478188580' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T08:45:07.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:45:06 smithi083 ceph-mon[27907]: from='client.? 172.21.15.189:0/2478188580' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T08:45:08.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:45:07 smithi138 ceph-mon[31441]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-24T08:45:08.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:45:07 smithi189 ceph-mon[31791]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-24T08:45:08.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:45:07 smithi083 ceph-mon[27907]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-24T08:45:10.106 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:45:09 smithi083 ceph-mon[27907]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-24T08:45:10.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:45:09 smithi138 ceph-mon[31441]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-24T08:45:10.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:45:09 smithi189 ceph-mon[31791]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-24T08:45:12.184 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:45:11 smithi189 ceph-mon[31791]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-24T08:45:12.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:45:11 smithi138 ceph-mon[31441]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-24T08:45:12.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:45:11 smithi083 ceph-mon[27907]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-24T08:45:14.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:45:13 smithi138 ceph-mon[31441]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-24T08:45:14.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:45:13 smithi189 ceph-mon[31791]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-24T08:45:14.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:45:13 smithi083 ceph-mon[27907]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-24T08:45:15.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:45:14 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-24T08:45:15.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:45:14 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:45:15.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:45:14 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-24T08:45:15.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:45:14 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:45:15.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:45:14 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-24T08:45:15.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:45:14 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:45:15.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:45:15 smithi189 ceph-mon[31791]: Deploying daemon osd.8 on smithi189 2024-04-24T08:45:15.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:45:15 smithi189 ceph-mon[31791]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-24T08:45:16.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:45:15 smithi138 ceph-mon[31441]: Deploying daemon osd.8 on smithi189 2024-04-24T08:45:16.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:45:15 smithi138 ceph-mon[31441]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-24T08:45:16.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:45:15 smithi083 ceph-mon[27907]: Deploying daemon osd.8 on smithi189 2024-04-24T08:45:16.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:45:15 smithi083 ceph-mon[27907]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-24T08:45:17.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:45:17 smithi189 ceph-mon[31791]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-24T08:45:18.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:45:17 smithi138 ceph-mon[31441]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-24T08:45:18.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:45:17 smithi083 ceph-mon[27907]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-24T08:45:20.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:45:19 smithi138 ceph-mon[31441]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-24T08:45:20.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:45:19 smithi189 ceph-mon[31791]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-24T08:45:20.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:45:19 smithi083 ceph-mon[27907]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-24T08:45:22.176 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:45:21 smithi189 ceph-mon[31791]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-24T08:45:22.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:45:21 smithi138 ceph-mon[31441]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-24T08:45:22.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:45:21 smithi083 ceph-mon[27907]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-24T08:45:24.175 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:45:23 smithi189 ceph-mon[31791]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-24T08:45:24.175 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:45:23 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:45:24.175 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:45:23 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:45:24.175 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:45:23 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:45:24.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:45:23 smithi138 ceph-mon[31441]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-24T08:45:24.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:45:23 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:45:24.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:45:23 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:45:24.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:45:23 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:45:24.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:45:23 smithi083 ceph-mon[27907]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-24T08:45:24.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:45:23 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:45:24.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:45:23 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:45:24.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:45:23 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:45:26.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:45:25 smithi138 ceph-mon[31441]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-24T08:45:26.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:45:25 smithi189 ceph-mon[31791]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-24T08:45:26.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:45:25 smithi083 ceph-mon[27907]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-24T08:45:28.108 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:45:27 smithi189 ceph-mon[31791]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-24T08:45:28.108 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:45:27 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:45:28.108 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:45:27 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:45:28.108 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:45:27 smithi189 ceph-mon[31791]: from='osd.8 [v2:172.21.15.189:6816/207018554,v1:172.21.15.189:6817/207018554]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-24T08:45:28.108 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:45:27 smithi189 ceph-mon[31791]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-24T08:45:28.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:45:27 smithi138 ceph-mon[31441]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-24T08:45:28.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:45:27 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:45:28.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:45:27 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:45:28.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:45:27 smithi138 ceph-mon[31441]: from='osd.8 [v2:172.21.15.189:6816/207018554,v1:172.21.15.189:6817/207018554]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-24T08:45:28.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:45:27 smithi138 ceph-mon[31441]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-24T08:45:28.216 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:45:28.215+0000 7f3afb400700 1 -- 172.21.15.189:0/3161948177 <== mgr.14152 v2:172.21.15.83:6800/2550826560 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f3b04076440 con 0x7f3ae80a3cc0 2024-04-24T08:45:28.216 INFO:teuthology.orchestra.run.smithi189.stdout:Created osd(s) 8 on host 'smithi189' 2024-04-24T08:45:28.219 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:45:28.218+0000 7f3b09800700 1 -- 172.21.15.189:0/3161948177 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f3ae80a3cc0 msgr2=0x7f3ae80a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:45:28.219 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:45:28.218+0000 7f3b09800700 1 --2- 172.21.15.189:0/3161948177 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f3ae80a3cc0 0x7f3ae80a6180 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3af4006010 tx=0x7f3af4005c60 comp rx=0 tx=0).stop 2024-04-24T08:45:28.219 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:45:28.218+0000 7f3b09800700 1 -- 172.21.15.189:0/3161948177 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3b04103d50 msgr2=0x7f3b0419b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:45:28.219 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:45:28.218+0000 7f3b09800700 1 --2- 172.21.15.189:0/3161948177 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3b04103d50 0x7f3b0419b4c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f3af000c950 tx=0x7f3af000cd10 comp rx=0 tx=0).stop 2024-04-24T08:45:28.219 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:45:28.219+0000 7f3b09800700 1 -- 172.21.15.189:0/3161948177 shutdown_connections 2024-04-24T08:45:28.219 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:45:28.219+0000 7f3b09800700 1 --2- 172.21.15.189:0/3161948177 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f3ae80a3cc0 0x7f3ae80a6180 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:45:28.219 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:45:28.219+0000 7f3b09800700 1 --2- 172.21.15.189:0/3161948177 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b04102b50 0x7f3b0419af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:45:28.219 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:45:28.219+0000 7f3b09800700 1 --2- 172.21.15.189:0/3161948177 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b041046f0 0x7f3b041a24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:45:28.220 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:45:28.219+0000 7f3b09800700 1 --2- 172.21.15.189:0/3161948177 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3b04103d50 0x7f3b0419b4c0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:45:28.220 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:45:28.219+0000 7f3b09800700 1 -- 172.21.15.189:0/3161948177 >> 172.21.15.189:0/3161948177 conn(0x7f3b040fe0f0 msgr2=0x7f3b04074d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:45:28.220 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:45:28.219+0000 7f3b09800700 1 -- 172.21.15.189:0/3161948177 shutdown_connections 2024-04-24T08:45:28.220 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:45:28.219+0000 7f3b09800700 1 -- 172.21.15.189:0/3161948177 wait complete. 2024-04-24T08:45:28.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:45:27 smithi083 ceph-mon[27907]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-24T08:45:28.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:45:27 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:45:28.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:45:27 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:45:28.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:45:27 smithi083 ceph-mon[27907]: from='osd.8 [v2:172.21.15.189:6816/207018554,v1:172.21.15.189:6817/207018554]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-24T08:45:28.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:45:27 smithi083 ceph-mon[27907]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-24T08:45:28.954 DEBUG:teuthology.orchestra.run.smithi189:osd.8> sudo journalctl -f -n 0 -u ceph-bd3e039e-0215-11ef-bc93-c7b262605968@osd.8.service 2024-04-24T08:45:28.958 INFO:tasks.cephadm:Deploying osd.9 on smithi083 with /dev/vg_nvme/lv_1... 2024-04-24T08:45:28.958 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd3e039e-0215-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-24T08:45:29.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:45:29 smithi189 ceph-mon[31791]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-24T08:45:29.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:45:29 smithi189 ceph-mon[31791]: osdmap e57: 9 total, 8 up, 9 in 2024-04-24T08:45:29.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:45:29 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T08:45:29.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:45:29 smithi189 ceph-mon[31791]: from='osd.8 [v2:172.21.15.189:6816/207018554,v1:172.21.15.189:6817/207018554]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-04-24T08:45:29.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:45:29 smithi189 ceph-mon[31791]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-04-24T08:45:29.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:45:29 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:45:29.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:45:29 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:45:29.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:45:29 smithi189 ceph-mon[31791]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-24T08:45:29.208 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 08:45:29 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T08:45:29.038+0000 7efd44e00700 -1 osd.8 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T08:45:29.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:45:29 smithi083 ceph-mon[27907]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-24T08:45:29.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:45:29 smithi083 ceph-mon[27907]: osdmap e57: 9 total, 8 up, 9 in 2024-04-24T08:45:29.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:45:29 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T08:45:29.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:45:29 smithi083 ceph-mon[27907]: from='osd.8 [v2:172.21.15.189:6816/207018554,v1:172.21.15.189:6817/207018554]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-04-24T08:45:29.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:45:29 smithi083 ceph-mon[27907]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-04-24T08:45:29.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:45:29 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:45:29.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:45:29 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:45:29.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:45:29 smithi083 ceph-mon[27907]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-24T08:45:29.398 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T08:45:29.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:45:29 smithi138 ceph-mon[31441]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-24T08:45:29.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:45:29 smithi138 ceph-mon[31441]: osdmap e57: 9 total, 8 up, 9 in 2024-04-24T08:45:29.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:45:29 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T08:45:29.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:45:29 smithi138 ceph-mon[31441]: from='osd.8 [v2:172.21.15.189:6816/207018554,v1:172.21.15.189:6817/207018554]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-04-24T08:45:29.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:45:29 smithi138 ceph-mon[31441]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-04-24T08:45:29.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:45:29 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:45:29.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:45:29 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:45:29.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:45:29 smithi138 ceph-mon[31441]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-24T08:45:30.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:45:30 smithi083 ceph-mon[27907]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-04-24T08:45:30.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:45:30 smithi083 ceph-mon[27907]: osdmap e58: 9 total, 8 up, 9 in 2024-04-24T08:45:30.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:45:30 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T08:45:30.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:45:30 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T08:45:30.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:45:30 smithi083 ceph-mon[27907]: osd.8 [v2:172.21.15.189:6816/207018554,v1:172.21.15.189:6817/207018554] boot 2024-04-24T08:45:30.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:45:30 smithi083 ceph-mon[27907]: osdmap e59: 9 total, 9 up, 9 in 2024-04-24T08:45:30.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:45:30 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T08:45:30.310 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:45:30 smithi189 ceph-mon[31791]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-04-24T08:45:30.310 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:45:30 smithi189 ceph-mon[31791]: osdmap e58: 9 total, 8 up, 9 in 2024-04-24T08:45:30.310 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:45:30 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T08:45:30.310 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:45:30 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T08:45:30.310 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:45:30 smithi189 ceph-mon[31791]: osd.8 [v2:172.21.15.189:6816/207018554,v1:172.21.15.189:6817/207018554] boot 2024-04-24T08:45:30.310 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:45:30 smithi189 ceph-mon[31791]: osdmap e59: 9 total, 9 up, 9 in 2024-04-24T08:45:30.310 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:45:30 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T08:45:30.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:45:30 smithi138 ceph-mon[31441]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-04-24T08:45:30.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:45:30 smithi138 ceph-mon[31441]: osdmap e58: 9 total, 8 up, 9 in 2024-04-24T08:45:30.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:45:30 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T08:45:30.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:45:30 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T08:45:30.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:45:30 smithi138 ceph-mon[31441]: osd.8 [v2:172.21.15.189:6816/207018554,v1:172.21.15.189:6817/207018554] boot 2024-04-24T08:45:30.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:45:30 smithi138 ceph-mon[31441]: osdmap e59: 9 total, 9 up, 9 in 2024-04-24T08:45:30.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:45:30 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T08:45:31.205 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:45:31 smithi189 ceph-mon[31791]: purged_snaps scrub starts 2024-04-24T08:45:31.205 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:45:31 smithi189 ceph-mon[31791]: purged_snaps scrub ok 2024-04-24T08:45:31.205 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:45:31 smithi189 ceph-mon[31791]: osdmap e60: 9 total, 9 up, 9 in 2024-04-24T08:45:31.205 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:45:31 smithi189 ceph-mon[31791]: pgmap v255: 1 pgs: 1 remapped+peering; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-24T08:45:31.288 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:45:31 smithi083 ceph-mon[27907]: purged_snaps scrub starts 2024-04-24T08:45:31.288 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:45:31 smithi083 ceph-mon[27907]: purged_snaps scrub ok 2024-04-24T08:45:31.288 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:45:31 smithi083 ceph-mon[27907]: osdmap e60: 9 total, 9 up, 9 in 2024-04-24T08:45:31.288 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:45:31 smithi083 ceph-mon[27907]: pgmap v255: 1 pgs: 1 remapped+peering; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-24T08:45:31.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:45:31 smithi138 ceph-mon[31441]: purged_snaps scrub starts 2024-04-24T08:45:31.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:45:31 smithi138 ceph-mon[31441]: purged_snaps scrub ok 2024-04-24T08:45:31.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:45:31 smithi138 ceph-mon[31441]: osdmap e60: 9 total, 9 up, 9 in 2024-04-24T08:45:31.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:45:31 smithi138 ceph-mon[31441]: pgmap v255: 1 pgs: 1 remapped+peering; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-24T08:45:32.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:45:32 smithi138 ceph-mon[31441]: osdmap e61: 9 total, 9 up, 9 in 2024-04-24T08:45:32.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:45:32 smithi189 ceph-mon[31791]: osdmap e61: 9 total, 9 up, 9 in 2024-04-24T08:45:32.497 INFO:teuthology.orchestra.run.smithi083.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-24T08:45:32.497 INFO:teuthology.orchestra.run.smithi083.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-24T08:45:32.497 INFO:teuthology.orchestra.run.smithi083.stderr: stderr: 10+0 records in 2024-04-24T08:45:32.497 INFO:teuthology.orchestra.run.smithi083.stderr:10+0 records out 2024-04-24T08:45:32.497 INFO:teuthology.orchestra.run.smithi083.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0343539 s, 305 MB/s 2024-04-24T08:45:32.497 INFO:teuthology.orchestra.run.smithi083.stderr:--> Zapping successful for: 2024-04-24T08:45:32.498 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:45:32 smithi083 ceph-mon[27907]: osdmap e61: 9 total, 9 up, 9 in 2024-04-24T08:45:33.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:45:33 smithi083 ceph-mon[27907]: osdmap e62: 9 total, 9 up, 9 in 2024-04-24T08:45:33.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:45:33 smithi083 ceph-mon[27907]: pgmap v258: 1 pgs: 1 remapped+peering; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-24T08:45:33.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:45:33 smithi138 ceph-mon[31441]: osdmap e62: 9 total, 9 up, 9 in 2024-04-24T08:45:33.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:45:33 smithi138 ceph-mon[31441]: pgmap v258: 1 pgs: 1 remapped+peering; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-24T08:45:33.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:45:33 smithi189 ceph-mon[31791]: osdmap e62: 9 total, 9 up, 9 in 2024-04-24T08:45:33.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:45:33 smithi189 ceph-mon[31791]: pgmap v258: 1 pgs: 1 remapped+peering; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-24T08:45:33.528 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd3e039e-0215-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi083:vg_nvme/lv_1 2024-04-24T08:45:33.961 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T08:45:34.444 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:45:34 smithi083 ceph-mon[27907]: osdmap e63: 9 total, 9 up, 9 in 2024-04-24T08:45:34.444 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:45:34 smithi083 ceph-mon[27907]: Detected new or changed devices on smithi189 2024-04-24T08:45:34.444 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:45:34 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:45:34.444 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:45:34 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:45:34.444 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:45:34 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:45:34.444 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:45:34 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:45:34.444 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:45:34 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:45:34.444 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:45:34 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:45:34.444 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:45:34 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:45:34.445 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:45:34 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:45:34.445 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:45:34 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:45:34.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:45:34 smithi138 ceph-mon[31441]: osdmap e63: 9 total, 9 up, 9 in 2024-04-24T08:45:34.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:45:34 smithi138 ceph-mon[31441]: Detected new or changed devices on smithi189 2024-04-24T08:45:34.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:45:34 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:45:34.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:45:34 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:45:34.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:45:34 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:45:34.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:45:34 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:45:34.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:45:34 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:45:34.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:45:34 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:45:34.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:45:34 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:45:34.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:45:34 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:45:34.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:45:34 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:45:34.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:45:34 smithi189 ceph-mon[31791]: osdmap e63: 9 total, 9 up, 9 in 2024-04-24T08:45:34.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:45:34 smithi189 ceph-mon[31791]: Detected new or changed devices on smithi189 2024-04-24T08:45:34.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:45:34 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:45:34.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:45:34 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:45:34.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:45:34 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:45:34.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:45:34 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:45:34.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:45:34 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:45:34.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:45:34 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:45:34.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:45:34 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:45:34.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:45:34 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:45:34.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:45:34 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:45:35.280 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:45:35 smithi138 ceph-mon[31441]: Adjusting osd_memory_target on smithi189 to 5696M 2024-04-24T08:45:35.280 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:45:35 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:45:35.280 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:45:35 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:45:35.280 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:45:35 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:45:35.280 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:45:35 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:45:35.280 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:45:35 smithi138 ceph-mon[31441]: pgmap v260: 1 pgs: 1 remapped+peering; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-24T08:45:35.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:45:35 smithi189 ceph-mon[31791]: Adjusting osd_memory_target on smithi189 to 5696M 2024-04-24T08:45:35.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:45:35 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:45:35.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:45:35 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:45:35.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:45:35 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:45:35.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:45:35 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:45:35.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:45:35 smithi189 ceph-mon[31791]: pgmap v260: 1 pgs: 1 remapped+peering; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-24T08:45:35.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:45:35 smithi083 ceph-mon[27907]: Adjusting osd_memory_target on smithi189 to 5696M 2024-04-24T08:45:35.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:45:35 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:45:35.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:45:35 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:45:35.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:45:35 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:45:35.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:45:35 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:45:35.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:45:35 smithi083 ceph-mon[27907]: pgmap v260: 1 pgs: 1 remapped+peering; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-24T08:45:36.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:45:36.485+0000 7f824f000700 1 -- 172.21.15.83:0/1346776010 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f824810a280 msgr2=0x7f824810a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:45:36.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:45:36.485+0000 7f824f000700 1 --2- 172.21.15.83:0/1346776010 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f824810a280 0x7f824810a6e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f82340099c0 tx=0x7f8234009cd0 comp rx=0 tx=0).stop 2024-04-24T08:45:36.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:45:36.485+0000 7f824f000700 1 -- 172.21.15.83:0/1346776010 shutdown_connections 2024-04-24T08:45:36.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:45:36.485+0000 7f824f000700 1 --2- 172.21.15.83:0/1346776010 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f824810ac20 0x7f82481114d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:45:36.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:45:36.485+0000 7f824f000700 1 --2- 172.21.15.83:0/1346776010 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8248109080 0x7f82481094a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:45:36.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:45:36.485+0000 7f824f000700 1 --2- 172.21.15.83:0/1346776010 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f824810a280 0x7f824810a6e0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:45:36.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:45:36.485+0000 7f824f000700 1 -- 172.21.15.83:0/1346776010 >> 172.21.15.83:0/1346776010 conn(0x7f8248078050 msgr2=0x7f824807a490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:45:36.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:45:36.485+0000 7f824f000700 1 -- 172.21.15.83:0/1346776010 shutdown_connections 2024-04-24T08:45:36.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:45:36.485+0000 7f824f000700 1 -- 172.21.15.83:0/1346776010 wait complete. 2024-04-24T08:45:36.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:45:36.486+0000 7f824f000700 1 Processor -- start 2024-04-24T08:45:36.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:45:36.486+0000 7f824f000700 1 -- start start 2024-04-24T08:45:36.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:45:36.486+0000 7f824f000700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8248109080 0x7f824819f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:45:36.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:45:36.487+0000 7f824f000700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f824810a280 0x7f824819f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:45:36.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:45:36.487+0000 7f824f000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f824810ac20 0x7f82481a6710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:45:36.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:45:36.487+0000 7f824f000700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8248114bc0 con 0x7f8248109080 2024-04-24T08:45:36.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:45:36.487+0000 7f824f000700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8248114a40 con 0x7f824810a280 2024-04-24T08:45:36.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:45:36.487+0000 7f824f000700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8248114d40 con 0x7f824810ac20 2024-04-24T08:45:36.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:45:36.487+0000 7f824dc00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8248109080 0x7f824819f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:45:36.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:45:36.487+0000 7f824dc00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8248109080 0x7f824819f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41290/0 (socket says 172.21.15.83:41290) 2024-04-24T08:45:36.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:45:36.487+0000 7f824dc00700 1 -- 172.21.15.83:0/868738898 learned_addr learned my addr 172.21.15.83:0/868738898 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:45:36.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:45:36.487+0000 7f824d200700 1 --2- 172.21.15.83:0/868738898 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f824810a280 0x7f824819f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:45:36.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:45:36.487+0000 7f824dc00700 1 -- 172.21.15.83:0/868738898 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f824810ac20 msgr2=0x7f82481a6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:45:36.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:45:36.487+0000 7f824dc00700 1 --2- 172.21.15.83:0/868738898 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f824810ac20 0x7f82481a6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:45:36.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:45:36.487+0000 7f824dc00700 1 -- 172.21.15.83:0/868738898 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f824810a280 msgr2=0x7f824819f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:45:36.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:45:36.487+0000 7f824dc00700 1 --2- 172.21.15.83:0/868738898 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f824810a280 0x7f824819f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:45:36.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:45:36.487+0000 7f824dc00700 1 -- 172.21.15.83:0/868738898 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82481a6db0 con 0x7f8248109080 2024-04-24T08:45:36.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:45:36.488+0000 7f824dc00700 1 --2- 172.21.15.83:0/868738898 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8248109080 0x7f824819f150 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f823c00c9c0 tx=0x7f823c00cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:45:36.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:45:36.488+0000 7f8246000700 1 -- 172.21.15.83:0/868738898 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f823c007ac0 con 0x7f8248109080 2024-04-24T08:45:36.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:45:36.488+0000 7f8246000700 1 -- 172.21.15.83:0/868738898 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f823c007c60 con 0x7f8248109080 2024-04-24T08:45:36.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:45:36.488+0000 7f824f000700 1 -- 172.21.15.83:0/868738898 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82481a70a0 con 0x7f8248109080 2024-04-24T08:45:36.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:45:36.488+0000 7f8246000700 1 -- 172.21.15.83:0/868738898 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f823c018660 con 0x7f8248109080 2024-04-24T08:45:36.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:45:36.488+0000 7f824f000700 1 -- 172.21.15.83:0/868738898 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82481a76b0 con 0x7f8248109080 2024-04-24T08:45:36.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:45:36.489+0000 7f824f000700 1 -- 172.21.15.83:0/868738898 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82480697e0 con 0x7f8248109080 2024-04-24T08:45:36.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:45:36.492+0000 7f8246000700 1 -- 172.21.15.83:0/868738898 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f823c00f450 con 0x7f8248109080 2024-04-24T08:45:36.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:45:36.494+0000 7f8246000700 1 --2- 172.21.15.83:0/868738898 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f82300a3cc0 0x7f82300a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:45:36.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:45:36.494+0000 7f8246000700 1 -- 172.21.15.83:0/868738898 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5829+0+0 (secure 0 0 0) 0x7f823c0cfc00 con 0x7f8248109080 2024-04-24T08:45:36.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:45:36.494+0000 7f824d200700 1 --2- 172.21.15.83:0/868738898 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f82300a3cc0 0x7f82300a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:45:36.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:45:36.495+0000 7f824d200700 1 --2- 172.21.15.83:0/868738898 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f82300a3cc0 0x7f82300a6180 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f82340038f0 tx=0x7f8234000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:45:36.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:45:36.497+0000 7f8246000700 1 -- 172.21.15.83:0/868738898 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f823c014030 con 0x7f8248109080 2024-04-24T08:45:36.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:45:36.733+0000 7f824f000700 1 -- 172.21.15.83:0/868738898 --> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f8248062f30 con 0x7f82300a3cc0 2024-04-24T08:45:37.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:45:36 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T08:45:37.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:45:36 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T08:45:37.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:45:36 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:45:37.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:45:36 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T08:45:37.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:45:36 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T08:45:37.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:45:36 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:45:37.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:45:36 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T08:45:37.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:45:36 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T08:45:37.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:45:36 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:45:37.889 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:45:37 smithi083 ceph-mon[27907]: from='client.14433 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:45:37.890 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:45:37 smithi083 ceph-mon[27907]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-24T08:45:38.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:45:37 smithi138 ceph-mon[31441]: from='client.14433 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:45:38.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:45:37 smithi138 ceph-mon[31441]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-24T08:45:38.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:45:37 smithi189 ceph-mon[31791]: from='client.14433 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:45:38.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:45:37 smithi189 ceph-mon[31791]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-24T08:45:40.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:45:39 smithi083 ceph-mon[27907]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-04-24T08:45:40.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:45:39 smithi138 ceph-mon[31441]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-04-24T08:45:40.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:45:39 smithi189 ceph-mon[31791]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-04-24T08:45:42.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:45:41 smithi138 ceph-mon[31441]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-04-24T08:45:42.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:45:41 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/778705194' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "baf19b98-881b-4d3a-9f72-8aa2dd73cad6"}]: dispatch 2024-04-24T08:45:42.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:45:41 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/778705194' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "baf19b98-881b-4d3a-9f72-8aa2dd73cad6"}]': finished 2024-04-24T08:45:42.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:45:41 smithi138 ceph-mon[31441]: osdmap e64: 10 total, 9 up, 10 in 2024-04-24T08:45:42.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:45:41 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T08:45:42.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:45:41 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/12826257' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T08:45:42.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:45:41 smithi189 ceph-mon[31791]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-04-24T08:45:42.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:45:41 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/778705194' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "baf19b98-881b-4d3a-9f72-8aa2dd73cad6"}]: dispatch 2024-04-24T08:45:42.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:45:41 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/778705194' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "baf19b98-881b-4d3a-9f72-8aa2dd73cad6"}]': finished 2024-04-24T08:45:42.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:45:41 smithi189 ceph-mon[31791]: osdmap e64: 10 total, 9 up, 10 in 2024-04-24T08:45:42.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:45:41 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T08:45:42.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:45:41 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/12826257' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T08:45:42.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:45:41 smithi083 ceph-mon[27907]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-04-24T08:45:42.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:45:41 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/778705194' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "baf19b98-881b-4d3a-9f72-8aa2dd73cad6"}]: dispatch 2024-04-24T08:45:42.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:45:41 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/778705194' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "baf19b98-881b-4d3a-9f72-8aa2dd73cad6"}]': finished 2024-04-24T08:45:42.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:45:41 smithi083 ceph-mon[27907]: osdmap e64: 10 total, 9 up, 10 in 2024-04-24T08:45:42.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:45:41 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T08:45:42.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:45:41 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/12826257' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T08:45:44.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:45:43 smithi138 ceph-mon[31441]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-04-24T08:45:44.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:45:43 smithi189 ceph-mon[31791]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-04-24T08:45:44.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:45:43 smithi083 ceph-mon[27907]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-04-24T08:45:45.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:45:45 smithi138 ceph-mon[31441]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-24T08:45:46.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:45:45 smithi083 ceph-mon[27907]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-24T08:45:46.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:45:45 smithi189 ceph-mon[31791]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-24T08:45:48.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:45:47 smithi083 ceph-mon[27907]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-24T08:45:48.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:45:47 smithi138 ceph-mon[31441]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-24T08:45:48.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:45:47 smithi189 ceph-mon[31791]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-24T08:45:50.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:45:49 smithi083 ceph-mon[27907]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-24T08:45:50.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:45:49 smithi138 ceph-mon[31441]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-24T08:45:50.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:45:49 smithi189 ceph-mon[31791]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-24T08:45:52.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:45:51 smithi083 ceph-mon[27907]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-24T08:45:52.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:45:51 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-24T08:45:52.157 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:45:51 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:45:52.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:45:51 smithi138 ceph-mon[31441]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-24T08:45:52.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:45:51 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-24T08:45:52.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:45:51 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:45:52.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:45:51 smithi189 ceph-mon[31791]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-24T08:45:52.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:45:51 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-24T08:45:52.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:45:51 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:45:53.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:45:52 smithi138 ceph-mon[31441]: Deploying daemon osd.9 on smithi083 2024-04-24T08:45:53.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:45:52 smithi189 ceph-mon[31791]: Deploying daemon osd.9 on smithi083 2024-04-24T08:45:53.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:45:52 smithi083 ceph-mon[27907]: Deploying daemon osd.9 on smithi083 2024-04-24T08:45:54.089 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:45:53 smithi083 ceph-mon[27907]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-24T08:45:54.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:45:53 smithi138 ceph-mon[31441]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-24T08:45:54.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:45:53 smithi189 ceph-mon[31791]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-24T08:45:56.199 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:45:55 smithi083 ceph-mon[27907]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-24T08:45:56.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:45:55 smithi138 ceph-mon[31441]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-24T08:45:56.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:45:55 smithi189 ceph-mon[31791]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-24T08:45:58.167 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:45:57 smithi083 ceph-mon[27907]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-24T08:45:58.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:45:57 smithi138 ceph-mon[31441]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-24T08:45:58.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:45:57 smithi189 ceph-mon[31791]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-24T08:46:00.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:45:59 smithi083 ceph-mon[27907]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-24T08:46:00.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:45:59 smithi138 ceph-mon[31441]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-24T08:46:00.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:45:59 smithi189 ceph-mon[31791]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-24T08:46:01.199 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:01 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:46:01.200 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:01 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:46:01.200 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:01 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:46:01.200 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:01 smithi083 ceph-mon[27907]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-24T08:46:01.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:01 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:46:01.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:01 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:46:01.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:01 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:46:01.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:01 smithi138 ceph-mon[31441]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-24T08:46:01.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:01 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:46:01.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:01 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:46:01.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:01 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:46:01.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:01 smithi189 ceph-mon[31791]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-24T08:46:04.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:03 smithi083 ceph-mon[27907]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-24T08:46:04.159 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:03 smithi138 ceph-mon[31441]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-24T08:46:04.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:03 smithi189 ceph-mon[31791]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-24T08:46:04.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:46:04.731+0000 7f8246000700 1 -- 172.21.15.83:0/868738898 <== mgr.14152 v2:172.21.15.83:6800/2550826560 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f8248062f30 con 0x7f82300a3cc0 2024-04-24T08:46:04.732 INFO:teuthology.orchestra.run.smithi083.stdout:Created osd(s) 9 on host 'smithi083' 2024-04-24T08:46:04.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:46:04.735+0000 7f824f000700 1 -- 172.21.15.83:0/868738898 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f82300a3cc0 msgr2=0x7f82300a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:46:04.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:46:04.735+0000 7f824f000700 1 --2- 172.21.15.83:0/868738898 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f82300a3cc0 0x7f82300a6180 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f82340038f0 tx=0x7f8234000bc0 comp rx=0 tx=0).stop 2024-04-24T08:46:04.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:46:04.735+0000 7f824f000700 1 -- 172.21.15.83:0/868738898 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8248109080 msgr2=0x7f824819f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:46:04.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:46:04.735+0000 7f824f000700 1 --2- 172.21.15.83:0/868738898 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8248109080 0x7f824819f150 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f823c00c9c0 tx=0x7f823c00cd80 comp rx=0 tx=0).stop 2024-04-24T08:46:04.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:46:04.736+0000 7f824f000700 1 -- 172.21.15.83:0/868738898 shutdown_connections 2024-04-24T08:46:04.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:46:04.736+0000 7f824f000700 1 --2- 172.21.15.83:0/868738898 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f82300a3cc0 0x7f82300a6180 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:46:04.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:46:04.736+0000 7f824f000700 1 --2- 172.21.15.83:0/868738898 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8248109080 0x7f824819f150 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:46:04.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:46:04.736+0000 7f824f000700 1 --2- 172.21.15.83:0/868738898 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f824810ac20 0x7f82481a6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:46:04.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:46:04.736+0000 7f824f000700 1 --2- 172.21.15.83:0/868738898 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f824810a280 0x7f824819f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:46:04.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:46:04.736+0000 7f824f000700 1 -- 172.21.15.83:0/868738898 >> 172.21.15.83:0/868738898 conn(0x7f8248078050 msgr2=0x7f824810d580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:46:04.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:46:04.736+0000 7f824f000700 1 -- 172.21.15.83:0/868738898 shutdown_connections 2024-04-24T08:46:04.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:46:04.736+0000 7f824f000700 1 -- 172.21.15.83:0/868738898 wait complete. 2024-04-24T08:46:05.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:04 smithi083 ceph-mon[27907]: from='osd.9 [v2:172.21.15.83:6826/519636751,v1:172.21.15.83:6827/519636751]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-24T08:46:05.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:04 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:46:05.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:04 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:46:05.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:04 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:46:05.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:04 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:46:05.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:04 smithi138 ceph-mon[31441]: from='osd.9 [v2:172.21.15.83:6826/519636751,v1:172.21.15.83:6827/519636751]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-24T08:46:05.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:04 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:46:05.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:04 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:46:05.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:04 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:46:05.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:04 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:46:05.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:04 smithi189 ceph-mon[31791]: from='osd.9 [v2:172.21.15.83:6826/519636751,v1:172.21.15.83:6827/519636751]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-24T08:46:05.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:04 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:46:05.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:04 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:46:05.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:04 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:46:05.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:04 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:46:05.583 DEBUG:teuthology.orchestra.run.smithi083:osd.9> sudo journalctl -f -n 0 -u ceph-bd3e039e-0215-11ef-bc93-c7b262605968@osd.9.service 2024-04-24T08:46:05.588 INFO:tasks.cephadm:Deploying osd.10 on smithi138 with /dev/vg_nvme/lv_1... 2024-04-24T08:46:05.588 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd3e039e-0215-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-24T08:46:05.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:05 smithi138 ceph-mon[31441]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-24T08:46:05.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:05 smithi138 ceph-mon[31441]: from='osd.9 [v2:172.21.15.83:6826/519636751,v1:172.21.15.83:6827/519636751]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-24T08:46:05.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:05 smithi138 ceph-mon[31441]: osdmap e65: 10 total, 9 up, 10 in 2024-04-24T08:46:05.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:05 smithi138 ceph-mon[31441]: from='osd.9 [v2:172.21.15.83:6826/519636751,v1:172.21.15.83:6827/519636751]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-24T08:46:05.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:05 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T08:46:06.027 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T08:46:06.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:05 smithi189 ceph-mon[31791]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-24T08:46:06.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:05 smithi189 ceph-mon[31791]: from='osd.9 [v2:172.21.15.83:6826/519636751,v1:172.21.15.83:6827/519636751]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-24T08:46:06.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:05 smithi189 ceph-mon[31791]: osdmap e65: 10 total, 9 up, 10 in 2024-04-24T08:46:06.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:05 smithi189 ceph-mon[31791]: from='osd.9 [v2:172.21.15.83:6826/519636751,v1:172.21.15.83:6827/519636751]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-24T08:46:06.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:05 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T08:46:06.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:05 smithi083 ceph-mon[27907]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-24T08:46:06.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:05 smithi083 ceph-mon[27907]: from='osd.9 [v2:172.21.15.83:6826/519636751,v1:172.21.15.83:6827/519636751]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-24T08:46:06.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:05 smithi083 ceph-mon[27907]: osdmap e65: 10 total, 9 up, 10 in 2024-04-24T08:46:06.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:05 smithi083 ceph-mon[27907]: from='osd.9 [v2:172.21.15.83:6826/519636751,v1:172.21.15.83:6827/519636751]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-24T08:46:06.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:05 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T08:46:07.002 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:06 smithi083 ceph-mon[27907]: from='osd.9 [v2:172.21.15.83:6826/519636751,v1:172.21.15.83:6827/519636751]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-04-24T08:46:07.002 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:06 smithi083 ceph-mon[27907]: osdmap e66: 10 total, 9 up, 10 in 2024-04-24T08:46:07.002 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:06 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T08:46:07.003 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:06 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T08:46:07.003 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:06 smithi083 ceph-mon[27907]: from='osd.9 [v2:172.21.15.83:6826/519636751,v1:172.21.15.83:6827/519636751]' entity='osd.9' 2024-04-24T08:46:07.003 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 08:46:06 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T08:46:06.635+0000 7ff3e6600700 -1 osd.9 0 waiting for initial osdmap 2024-04-24T08:46:07.003 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 08:46:06 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T08:46:06.664+0000 7ff3dba00700 -1 osd.9 66 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T08:46:07.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:06 smithi138 ceph-mon[31441]: from='osd.9 [v2:172.21.15.83:6826/519636751,v1:172.21.15.83:6827/519636751]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-04-24T08:46:07.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:06 smithi138 ceph-mon[31441]: osdmap e66: 10 total, 9 up, 10 in 2024-04-24T08:46:07.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:06 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T08:46:07.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:06 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T08:46:07.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:06 smithi138 ceph-mon[31441]: from='osd.9 [v2:172.21.15.83:6826/519636751,v1:172.21.15.83:6827/519636751]' entity='osd.9' 2024-04-24T08:46:07.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:06 smithi189 ceph-mon[31791]: from='osd.9 [v2:172.21.15.83:6826/519636751,v1:172.21.15.83:6827/519636751]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-04-24T08:46:07.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:06 smithi189 ceph-mon[31791]: osdmap e66: 10 total, 9 up, 10 in 2024-04-24T08:46:07.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:06 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T08:46:07.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:06 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T08:46:07.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:06 smithi189 ceph-mon[31791]: from='osd.9 [v2:172.21.15.83:6826/519636751,v1:172.21.15.83:6827/519636751]' entity='osd.9' 2024-04-24T08:46:07.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:07 smithi189 ceph-mon[31791]: purged_snaps scrub starts 2024-04-24T08:46:07.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:07 smithi189 ceph-mon[31791]: purged_snaps scrub ok 2024-04-24T08:46:07.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:07 smithi189 ceph-mon[31791]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-24T08:46:07.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:07 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T08:46:07.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:07 smithi189 ceph-mon[31791]: osd.9 [v2:172.21.15.83:6826/519636751,v1:172.21.15.83:6827/519636751] boot 2024-04-24T08:46:07.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:07 smithi189 ceph-mon[31791]: osdmap e67: 10 total, 10 up, 10 in 2024-04-24T08:46:07.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:07 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T08:46:08.075 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:07 smithi138 ceph-mon[31441]: purged_snaps scrub starts 2024-04-24T08:46:08.075 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:07 smithi138 ceph-mon[31441]: purged_snaps scrub ok 2024-04-24T08:46:08.075 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:07 smithi138 ceph-mon[31441]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-24T08:46:08.075 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:07 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T08:46:08.075 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:07 smithi138 ceph-mon[31441]: osd.9 [v2:172.21.15.83:6826/519636751,v1:172.21.15.83:6827/519636751] boot 2024-04-24T08:46:08.076 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:07 smithi138 ceph-mon[31441]: osdmap e67: 10 total, 10 up, 10 in 2024-04-24T08:46:08.076 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:07 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T08:46:08.264 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:07 smithi083 ceph-mon[27907]: purged_snaps scrub starts 2024-04-24T08:46:08.264 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:07 smithi083 ceph-mon[27907]: purged_snaps scrub ok 2024-04-24T08:46:08.264 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:07 smithi083 ceph-mon[27907]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-24T08:46:08.265 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:07 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T08:46:08.265 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:07 smithi083 ceph-mon[27907]: osd.9 [v2:172.21.15.83:6826/519636751,v1:172.21.15.83:6827/519636751] boot 2024-04-24T08:46:08.265 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:07 smithi083 ceph-mon[27907]: osdmap e67: 10 total, 10 up, 10 in 2024-04-24T08:46:08.265 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:07 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T08:46:09.082 INFO:teuthology.orchestra.run.smithi138.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-24T08:46:09.082 INFO:teuthology.orchestra.run.smithi138.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-24T08:46:09.082 INFO:teuthology.orchestra.run.smithi138.stderr: stderr: 10+0 records in 2024-04-24T08:46:09.082 INFO:teuthology.orchestra.run.smithi138.stderr:10+0 records out 2024-04-24T08:46:09.082 INFO:teuthology.orchestra.run.smithi138.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.032524 s, 322 MB/s 2024-04-24T08:46:09.082 INFO:teuthology.orchestra.run.smithi138.stderr:--> Zapping successful for: 2024-04-24T08:46:10.044 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd3e039e-0215-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi138:vg_nvme/lv_1 2024-04-24T08:46:10.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:09 smithi138 ceph-mon[31441]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-24T08:46:10.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:09 smithi138 ceph-mon[31441]: osdmap e68: 10 total, 10 up, 10 in 2024-04-24T08:46:10.205 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:09 smithi083 ceph-mon[27907]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-24T08:46:10.205 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:09 smithi083 ceph-mon[27907]: osdmap e68: 10 total, 10 up, 10 in 2024-04-24T08:46:10.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:09 smithi189 ceph-mon[31791]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-24T08:46:10.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:09 smithi189 ceph-mon[31791]: osdmap e68: 10 total, 10 up, 10 in 2024-04-24T08:46:10.480 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T08:46:11.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:11 smithi138 ceph-mon[31441]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-24T08:46:12.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:11 smithi083 ceph-mon[27907]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-24T08:46:12.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:11 smithi189 ceph-mon[31791]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-24T08:46:12.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:46:12.953+0000 7f2012800700 1 -- 172.21.15.138:0/3332530717 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f200c104d90 msgr2=0x7f200c1051b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:46:12.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:46:12.953+0000 7f2012800700 1 --2- 172.21.15.138:0/3332530717 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f200c104d90 0x7f200c1051b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc0099c0 tx=0x7f1ffc009cd0 comp rx=0 tx=0).stop 2024-04-24T08:46:12.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:46:12.953+0000 7f2012800700 1 -- 172.21.15.138:0/3332530717 shutdown_connections 2024-04-24T08:46:12.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:46:12.953+0000 7f2012800700 1 --2- 172.21.15.138:0/3332530717 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f200c106930 0x7f200c10d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:46:12.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:46:12.953+0000 7f2012800700 1 --2- 172.21.15.138:0/3332530717 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f200c104d90 0x7f200c1051b0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:46:12.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:46:12.953+0000 7f2012800700 1 --2- 172.21.15.138:0/3332530717 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f200c105f90 0x7f200c1063f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:46:12.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:46:12.953+0000 7f2012800700 1 -- 172.21.15.138:0/3332530717 >> 172.21.15.138:0/3332530717 conn(0x7f200c100350 msgr2=0x7f200c102770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:46:12.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:46:12.953+0000 7f2012800700 1 -- 172.21.15.138:0/3332530717 shutdown_connections 2024-04-24T08:46:12.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:46:12.954+0000 7f2012800700 1 -- 172.21.15.138:0/3332530717 wait complete. 2024-04-24T08:46:12.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:46:12.955+0000 7f2012800700 1 Processor -- start 2024-04-24T08:46:12.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:46:12.955+0000 7f2012800700 1 -- start start 2024-04-24T08:46:12.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:46:12.955+0000 7f2012800700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f200c104d90 0x7f200c19b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:46:12.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:46:12.955+0000 7f2012800700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f200c105f90 0x7f200c19b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:46:12.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:46:12.955+0000 7f2012800700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f200c106930 0x7f200c1a2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:46:12.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:46:12.955+0000 7f2012800700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f200c110950 con 0x7f200c105f90 2024-04-24T08:46:12.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:46:12.955+0000 7f2012800700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f200c1107d0 con 0x7f200c106930 2024-04-24T08:46:12.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:46:12.955+0000 7f2012800700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f200c110ad0 con 0x7f200c104d90 2024-04-24T08:46:12.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:46:12.955+0000 7f200be00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f200c106930 0x7f200c1a2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:46:12.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:46:12.955+0000 7f200be00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f200c106930 0x7f200c1a2610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:38022/0 (socket says 172.21.15.138:38022) 2024-04-24T08:46:12.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:46:12.956+0000 7f200be00700 1 -- 172.21.15.138:0/1834794665 learned_addr learned my addr 172.21.15.138:0/1834794665 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-24T08:46:12.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:46:12.956+0000 7f200b400700 1 --2- 172.21.15.138:0/1834794665 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f200c104d90 0x7f200c19b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:46:12.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:46:12.956+0000 7f200aa00700 1 --2- 172.21.15.138:0/1834794665 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f200c105f90 0x7f200c19b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:46:12.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:46:12.956+0000 7f200be00700 1 -- 172.21.15.138:0/1834794665 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f200c104d90 msgr2=0x7f200c19b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:46:12.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:46:12.956+0000 7f200be00700 1 --2- 172.21.15.138:0/1834794665 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f200c104d90 0x7f200c19b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:46:12.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:46:12.956+0000 7f200be00700 1 -- 172.21.15.138:0/1834794665 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f200c105f90 msgr2=0x7f200c19b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:46:12.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:46:12.956+0000 7f200be00700 1 --2- 172.21.15.138:0/1834794665 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f200c105f90 0x7f200c19b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:46:12.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:46:12.956+0000 7f200be00700 1 -- 172.21.15.138:0/1834794665 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f200c1a2cb0 con 0x7f200c106930 2024-04-24T08:46:12.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:46:12.956+0000 7f200aa00700 1 --2- 172.21.15.138:0/1834794665 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f200c105f90 0x7f200c19b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:46:12.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:46:12.956+0000 7f200be00700 1 --2- 172.21.15.138:0/1834794665 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f200c106930 0x7f200c1a2610 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f200400b970 tx=0x7f200400bd30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:46:12.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:46:12.956+0000 7f2003e00700 1 -- 172.21.15.138:0/1834794665 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2004019070 con 0x7f200c106930 2024-04-24T08:46:12.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:46:12.957+0000 7f2003e00700 1 -- 172.21.15.138:0/1834794665 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2004014440 con 0x7f200c106930 2024-04-24T08:46:12.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:46:12.957+0000 7f2012800700 1 -- 172.21.15.138:0/1834794665 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f200c1a2fa0 con 0x7f200c106930 2024-04-24T08:46:12.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:46:12.957+0000 7f2003e00700 1 -- 172.21.15.138:0/1834794665 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f200400ce60 con 0x7f200c106930 2024-04-24T08:46:12.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:46:12.957+0000 7f2012800700 1 -- 172.21.15.138:0/1834794665 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f200c1a3480 con 0x7f200c106930 2024-04-24T08:46:12.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:46:12.958+0000 7f2012800700 1 -- 172.21.15.138:0/1834794665 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f200c0697e0 con 0x7f200c106930 2024-04-24T08:46:12.962 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:46:12.962+0000 7f2003e00700 1 -- 172.21.15.138:0/1834794665 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f200401c070 con 0x7f200c106930 2024-04-24T08:46:12.964 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:46:12.963+0000 7f2003e00700 1 --2- 172.21.15.138:0/1834794665 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f1ff00a3c40 0x7f1ff00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:46:12.964 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:46:12.963+0000 7f2003e00700 1 -- 172.21.15.138:0/1834794665 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6261+0+0 (secure 0 0 0) 0x7f20040cf520 con 0x7f200c106930 2024-04-24T08:46:12.964 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:46:12.964+0000 7f200b400700 1 --2- 172.21.15.138:0/1834794665 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f1ff00a3c40 0x7f1ff00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:46:12.966 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:46:12.965+0000 7f200b400700 1 --2- 172.21.15.138:0/1834794665 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f1ff00a3c40 0x7f1ff00a6100 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc000c00 tx=0x7f1ffc005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:46:12.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:46:12.967+0000 7f2003e00700 1 -- 172.21.15.138:0/1834794665 <== mon.2 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f200409c660 con 0x7f200c106930 2024-04-24T08:46:13.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:46:13.204+0000 7f2012800700 1 -- 172.21.15.138:0/1834794665 --> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f200c062f30 con 0x7f1ff00a3c40 2024-04-24T08:46:13.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:13 smithi083 ceph-mon[27907]: Detected new or changed devices on smithi083 2024-04-24T08:46:13.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:13 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:46:13.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:13 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:46:13.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:13 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:46:13.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:13 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:46:13.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:13 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:46:13.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:13 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:46:13.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:13 smithi083 ceph-mon[27907]: Adjusting osd_memory_target on smithi083 to 4272M 2024-04-24T08:46:13.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:13 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:46:13.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:13 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:46:13.555 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:13 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:46:13.555 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:13 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:46:13.555 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:13 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:46:13.555 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:13 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:46:13.555 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:13 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:46:13.555 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:13 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:46:13.555 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:13 smithi083 ceph-mon[27907]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-24T08:46:13.706 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:13 smithi138 ceph-mon[31441]: Detected new or changed devices on smithi083 2024-04-24T08:46:13.706 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:13 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:46:13.706 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:13 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:46:13.706 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:13 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:46:13.706 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:13 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:46:13.706 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:13 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:46:13.707 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:13 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:46:13.707 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:13 smithi138 ceph-mon[31441]: Adjusting osd_memory_target on smithi083 to 4272M 2024-04-24T08:46:13.707 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:13 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:46:13.707 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:13 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:46:13.707 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:13 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:46:13.707 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:13 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:46:13.707 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:13 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:46:13.707 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:13 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:46:13.707 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:13 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:46:13.707 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:13 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:46:13.707 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:13 smithi138 ceph-mon[31441]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-24T08:46:13.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:13 smithi189 ceph-mon[31791]: Detected new or changed devices on smithi083 2024-04-24T08:46:13.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:13 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:46:13.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:13 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:46:13.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:13 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:46:13.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:13 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:46:13.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:13 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:46:13.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:13 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:46:13.709 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:13 smithi189 ceph-mon[31791]: Adjusting osd_memory_target on smithi083 to 4272M 2024-04-24T08:46:13.709 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:13 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:46:13.709 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:13 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:46:13.709 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:13 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:46:13.709 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:13 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:46:13.709 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:13 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:46:13.709 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:13 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:46:13.709 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:13 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:46:13.709 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:13 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:46:13.709 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:13 smithi189 ceph-mon[31791]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-24T08:46:14.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:14 smithi138 ceph-mon[31441]: from='client.34337 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:46:14.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:14 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T08:46:14.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:14 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T08:46:14.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:14 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:46:14.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:14 smithi083 ceph-mon[27907]: from='client.34337 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:46:14.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:14 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T08:46:14.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:14 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T08:46:14.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:14 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:46:14.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:14 smithi189 ceph-mon[31791]: from='client.34337 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:46:14.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:14 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T08:46:14.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:14 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T08:46:14.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:14 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:46:15.444 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:15 smithi138 ceph-mon[31441]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-24T08:46:15.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:15 smithi083 ceph-mon[27907]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-24T08:46:15.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:15 smithi189 ceph-mon[31791]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-24T08:46:18.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:17 smithi138 ceph-mon[31441]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-24T08:46:18.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:17 smithi138 ceph-mon[31441]: from='client.? 172.21.15.138:0/2589651658' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eb346717-5c07-47c8-95be-0e68fe449fdc"}]: dispatch 2024-04-24T08:46:18.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:17 smithi138 ceph-mon[31441]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eb346717-5c07-47c8-95be-0e68fe449fdc"}]: dispatch 2024-04-24T08:46:18.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:17 smithi138 ceph-mon[31441]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "eb346717-5c07-47c8-95be-0e68fe449fdc"}]': finished 2024-04-24T08:46:18.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:17 smithi138 ceph-mon[31441]: osdmap e69: 11 total, 10 up, 11 in 2024-04-24T08:46:18.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:17 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T08:46:18.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:17 smithi189 ceph-mon[31791]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-24T08:46:18.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:17 smithi189 ceph-mon[31791]: from='client.? 172.21.15.138:0/2589651658' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eb346717-5c07-47c8-95be-0e68fe449fdc"}]: dispatch 2024-04-24T08:46:18.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:17 smithi189 ceph-mon[31791]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eb346717-5c07-47c8-95be-0e68fe449fdc"}]: dispatch 2024-04-24T08:46:18.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:17 smithi189 ceph-mon[31791]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "eb346717-5c07-47c8-95be-0e68fe449fdc"}]': finished 2024-04-24T08:46:18.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:17 smithi189 ceph-mon[31791]: osdmap e69: 11 total, 10 up, 11 in 2024-04-24T08:46:18.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:17 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T08:46:18.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:17 smithi083 ceph-mon[27907]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-24T08:46:18.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:17 smithi083 ceph-mon[27907]: from='client.? 172.21.15.138:0/2589651658' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eb346717-5c07-47c8-95be-0e68fe449fdc"}]: dispatch 2024-04-24T08:46:18.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:17 smithi083 ceph-mon[27907]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eb346717-5c07-47c8-95be-0e68fe449fdc"}]: dispatch 2024-04-24T08:46:18.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:17 smithi083 ceph-mon[27907]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "eb346717-5c07-47c8-95be-0e68fe449fdc"}]': finished 2024-04-24T08:46:18.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:17 smithi083 ceph-mon[27907]: osdmap e69: 11 total, 10 up, 11 in 2024-04-24T08:46:18.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:17 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T08:46:19.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:18 smithi138 ceph-mon[31441]: from='client.? 172.21.15.138:0/2056599393' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T08:46:19.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:18 smithi189 ceph-mon[31791]: from='client.? 172.21.15.138:0/2056599393' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T08:46:19.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:18 smithi083 ceph-mon[27907]: from='client.? 172.21.15.138:0/2056599393' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T08:46:20.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:19 smithi138 ceph-mon[31441]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-24T08:46:20.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:19 smithi189 ceph-mon[31791]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-24T08:46:20.282 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:19 smithi083 ceph-mon[27907]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-24T08:46:22.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:21 smithi138 ceph-mon[31441]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-24T08:46:22.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:21 smithi189 ceph-mon[31791]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-24T08:46:22.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:21 smithi083 ceph-mon[27907]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-24T08:46:24.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:23 smithi138 ceph-mon[31441]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-24T08:46:24.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:23 smithi189 ceph-mon[31791]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-24T08:46:24.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:23 smithi083 ceph-mon[27907]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-24T08:46:26.129 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:25 smithi138 ceph-mon[31441]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-24T08:46:26.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:25 smithi189 ceph-mon[31791]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-24T08:46:26.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:25 smithi083 ceph-mon[27907]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-24T08:46:28.148 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:27 smithi138 ceph-mon[31441]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-24T08:46:28.148 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:27 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-24T08:46:28.148 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:27 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:46:28.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:27 smithi189 ceph-mon[31791]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-24T08:46:28.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:27 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-24T08:46:28.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:27 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:46:28.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:27 smithi083 ceph-mon[27907]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-24T08:46:28.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:27 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-24T08:46:28.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:27 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:46:29.152 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:28 smithi138 ceph-mon[31441]: Deploying daemon osd.10 on smithi138 2024-04-24T08:46:29.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:28 smithi189 ceph-mon[31791]: Deploying daemon osd.10 on smithi138 2024-04-24T08:46:29.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:28 smithi083 ceph-mon[27907]: Deploying daemon osd.10 on smithi138 2024-04-24T08:46:30.173 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:29 smithi138 ceph-mon[31441]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-24T08:46:30.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:29 smithi189 ceph-mon[31791]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-24T08:46:30.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:29 smithi083 ceph-mon[27907]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-24T08:46:32.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:31 smithi138 ceph-mon[31441]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-24T08:46:32.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:31 smithi189 ceph-mon[31791]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-24T08:46:32.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:31 smithi083 ceph-mon[27907]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-24T08:46:34.179 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:33 smithi138 ceph-mon[31441]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-24T08:46:34.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:33 smithi189 ceph-mon[31791]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-24T08:46:34.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:33 smithi083 ceph-mon[27907]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-24T08:46:36.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:35 smithi138 ceph-mon[31441]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-24T08:46:36.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:35 smithi189 ceph-mon[31791]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-24T08:46:36.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:35 smithi083 ceph-mon[27907]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-24T08:46:37.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:36 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:46:37.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:36 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:46:37.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:36 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:46:37.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:36 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:46:37.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:36 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:46:37.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:36 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:46:37.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:36 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:46:37.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:36 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:46:37.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:36 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:46:38.141 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:37 smithi138 ceph-mon[31441]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-24T08:46:38.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:37 smithi083 ceph-mon[27907]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-24T08:46:38.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:37 smithi189 ceph-mon[31791]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-24T08:46:39.285 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:39 smithi138 ceph-mon[31441]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-24T08:46:39.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:39 smithi083 ceph-mon[27907]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-24T08:46:39.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:39 smithi189 ceph-mon[31791]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-24T08:46:40.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:40 smithi138 ceph-mon[31441]: from='osd.10 [v2:172.21.15.138:6824/1130872098,v1:172.21.15.138:6825/1130872098]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-24T08:46:40.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:40 smithi138 ceph-mon[31441]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-24T08:46:40.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:40 smithi083 ceph-mon[27907]: from='osd.10 [v2:172.21.15.138:6824/1130872098,v1:172.21.15.138:6825/1130872098]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-24T08:46:40.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:40 smithi083 ceph-mon[27907]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-24T08:46:40.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:40 smithi189 ceph-mon[31791]: from='osd.10 [v2:172.21.15.138:6824/1130872098,v1:172.21.15.138:6825/1130872098]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-24T08:46:40.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:40 smithi189 ceph-mon[31791]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-24T08:46:41.452 INFO:teuthology.orchestra.run.smithi138.stdout:Created osd(s) 10 on host 'smithi138' 2024-04-24T08:46:41.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:46:41.451+0000 7f2003e00700 1 -- 172.21.15.138:0/1834794665 <== mgr.14152 v2:172.21.15.83:6800/2550826560 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f200c062f30 con 0x7f1ff00a3c40 2024-04-24T08:46:41.453 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:41 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:46:41.453 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:41 smithi138 ceph-mon[31441]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-24T08:46:41.453 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:41 smithi138 ceph-mon[31441]: osdmap e70: 11 total, 10 up, 11 in 2024-04-24T08:46:41.453 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:41 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T08:46:41.453 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:41 smithi138 ceph-mon[31441]: from='osd.10 [v2:172.21.15.138:6824/1130872098,v1:172.21.15.138:6825/1130872098]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-04-24T08:46:41.453 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:41 smithi138 ceph-mon[31441]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-04-24T08:46:41.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:41 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:46:41.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:41 smithi138 ceph-mon[31441]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-24T08:46:41.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:46:41.456+0000 7f2012800700 1 -- 172.21.15.138:0/1834794665 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f1ff00a3c40 msgr2=0x7f1ff00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:46:41.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:46:41.456+0000 7f2012800700 1 --2- 172.21.15.138:0/1834794665 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f1ff00a3c40 0x7f1ff00a6100 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc000c00 tx=0x7f1ffc005fb0 comp rx=0 tx=0).stop 2024-04-24T08:46:41.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:46:41.456+0000 7f2012800700 1 -- 172.21.15.138:0/1834794665 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f200c106930 msgr2=0x7f200c1a2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:46:41.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:46:41.456+0000 7f2012800700 1 --2- 172.21.15.138:0/1834794665 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f200c106930 0x7f200c1a2610 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f200400b970 tx=0x7f200400bd30 comp rx=0 tx=0).stop 2024-04-24T08:46:41.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:46:41.456+0000 7f2012800700 1 -- 172.21.15.138:0/1834794665 shutdown_connections 2024-04-24T08:46:41.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:46:41.456+0000 7f2012800700 1 --2- 172.21.15.138:0/1834794665 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f1ff00a3c40 0x7f1ff00a6100 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:46:41.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:46:41.456+0000 7f2012800700 1 --2- 172.21.15.138:0/1834794665 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f200c106930 0x7f200c1a2610 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:46:41.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:46:41.456+0000 7f2012800700 1 --2- 172.21.15.138:0/1834794665 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f200c104d90 0x7f200c19b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:46:41.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:46:41.456+0000 7f2012800700 1 --2- 172.21.15.138:0/1834794665 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f200c105f90 0x7f200c19b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:46:41.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:46:41.456+0000 7f2012800700 1 -- 172.21.15.138:0/1834794665 >> 172.21.15.138:0/1834794665 conn(0x7f200c100350 msgr2=0x7f200c109450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:46:41.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:46:41.457+0000 7f2012800700 1 -- 172.21.15.138:0/1834794665 shutdown_connections 2024-04-24T08:46:41.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-24T08:46:41.457+0000 7f2012800700 1 -- 172.21.15.138:0/1834794665 wait complete. 2024-04-24T08:46:41.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:41 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:46:41.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:41 smithi189 ceph-mon[31791]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-24T08:46:41.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:41 smithi189 ceph-mon[31791]: osdmap e70: 11 total, 10 up, 11 in 2024-04-24T08:46:41.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:41 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T08:46:41.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:41 smithi189 ceph-mon[31791]: from='osd.10 [v2:172.21.15.138:6824/1130872098,v1:172.21.15.138:6825/1130872098]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-04-24T08:46:41.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:41 smithi189 ceph-mon[31791]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-04-24T08:46:41.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:41 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:46:41.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:41 smithi189 ceph-mon[31791]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-24T08:46:41.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:41 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:46:41.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:41 smithi083 ceph-mon[27907]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-24T08:46:41.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:41 smithi083 ceph-mon[27907]: osdmap e70: 11 total, 10 up, 11 in 2024-04-24T08:46:41.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:41 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T08:46:41.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:41 smithi083 ceph-mon[27907]: from='osd.10 [v2:172.21.15.138:6824/1130872098,v1:172.21.15.138:6825/1130872098]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-04-24T08:46:41.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:41 smithi083 ceph-mon[27907]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-04-24T08:46:41.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:41 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:46:41.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:41 smithi083 ceph-mon[27907]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-24T08:46:42.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:42 smithi138 ceph-mon[31441]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-04-24T08:46:42.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:42 smithi138 ceph-mon[31441]: osdmap e71: 11 total, 10 up, 11 in 2024-04-24T08:46:42.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:42 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T08:46:42.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:42 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T08:46:42.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:42 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:46:42.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:42 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:46:42.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:42 smithi138 ceph-mon[31441]: from='osd.10 ' entity='osd.10' 2024-04-24T08:46:42.333 DEBUG:teuthology.orchestra.run.smithi138:osd.10> sudo journalctl -f -n 0 -u ceph-bd3e039e-0215-11ef-bc93-c7b262605968@osd.10.service 2024-04-24T08:46:42.338 INFO:tasks.cephadm:Deploying osd.11 on smithi189 with /dev/vg_nvme/lv_1... 2024-04-24T08:46:42.338 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd3e039e-0215-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-24T08:46:42.417 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:42 smithi189 ceph-mon[31791]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-04-24T08:46:42.417 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:42 smithi189 ceph-mon[31791]: osdmap e71: 11 total, 10 up, 11 in 2024-04-24T08:46:42.417 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:42 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T08:46:42.417 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:42 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T08:46:42.417 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:42 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:46:42.417 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:42 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:46:42.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:42 smithi189 ceph-mon[31791]: from='osd.10 ' entity='osd.10' 2024-04-24T08:46:42.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:42 smithi083 ceph-mon[27907]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-04-24T08:46:42.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:42 smithi083 ceph-mon[27907]: osdmap e71: 11 total, 10 up, 11 in 2024-04-24T08:46:42.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:42 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T08:46:42.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:42 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T08:46:42.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:42 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:46:42.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:42 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:46:42.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:42 smithi083 ceph-mon[27907]: from='osd.10 ' entity='osd.10' 2024-04-24T08:46:42.783 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T08:46:43.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:43 smithi138 ceph-mon[31441]: purged_snaps scrub starts 2024-04-24T08:46:43.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:43 smithi138 ceph-mon[31441]: purged_snaps scrub ok 2024-04-24T08:46:43.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:43 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T08:46:43.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:43 smithi138 ceph-mon[31441]: osd.10 [v2:172.21.15.138:6824/1130872098,v1:172.21.15.138:6825/1130872098] boot 2024-04-24T08:46:43.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:43 smithi138 ceph-mon[31441]: osdmap e72: 11 total, 11 up, 11 in 2024-04-24T08:46:43.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:43 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T08:46:43.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:43 smithi138 ceph-mon[31441]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-24T08:46:43.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:43 smithi189 ceph-mon[31791]: purged_snaps scrub starts 2024-04-24T08:46:43.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:43 smithi189 ceph-mon[31791]: purged_snaps scrub ok 2024-04-24T08:46:43.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:43 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T08:46:43.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:43 smithi189 ceph-mon[31791]: osd.10 [v2:172.21.15.138:6824/1130872098,v1:172.21.15.138:6825/1130872098] boot 2024-04-24T08:46:43.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:43 smithi189 ceph-mon[31791]: osdmap e72: 11 total, 11 up, 11 in 2024-04-24T08:46:43.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:43 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T08:46:43.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:43 smithi189 ceph-mon[31791]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-24T08:46:43.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:43 smithi083 ceph-mon[27907]: purged_snaps scrub starts 2024-04-24T08:46:43.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:43 smithi083 ceph-mon[27907]: purged_snaps scrub ok 2024-04-24T08:46:43.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:43 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T08:46:43.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:43 smithi083 ceph-mon[27907]: osd.10 [v2:172.21.15.138:6824/1130872098,v1:172.21.15.138:6825/1130872098] boot 2024-04-24T08:46:43.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:43 smithi083 ceph-mon[27907]: osdmap e72: 11 total, 11 up, 11 in 2024-04-24T08:46:43.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:43 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T08:46:43.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:43 smithi083 ceph-mon[27907]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-24T08:46:44.448 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:44 smithi138 ceph-mon[31441]: osdmap e73: 11 total, 11 up, 11 in 2024-04-24T08:46:44.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:44 smithi189 ceph-mon[31791]: osdmap e73: 11 total, 11 up, 11 in 2024-04-24T08:46:44.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:44 smithi083 ceph-mon[27907]: osdmap e73: 11 total, 11 up, 11 in 2024-04-24T08:46:45.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:45 smithi189 ceph-mon[31791]: osdmap e74: 11 total, 11 up, 11 in 2024-04-24T08:46:45.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:45 smithi189 ceph-mon[31791]: osdmap e75: 11 total, 11 up, 11 in 2024-04-24T08:46:45.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:45 smithi189 ceph-mon[31791]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-04-24T08:46:45.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:45 smithi138 ceph-mon[31441]: osdmap e74: 11 total, 11 up, 11 in 2024-04-24T08:46:45.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:45 smithi138 ceph-mon[31441]: osdmap e75: 11 total, 11 up, 11 in 2024-04-24T08:46:45.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:45 smithi138 ceph-mon[31441]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-04-24T08:46:45.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:45 smithi083 ceph-mon[27907]: osdmap e74: 11 total, 11 up, 11 in 2024-04-24T08:46:45.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:45 smithi083 ceph-mon[27907]: osdmap e75: 11 total, 11 up, 11 in 2024-04-24T08:46:45.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:45 smithi083 ceph-mon[27907]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-04-24T08:46:45.860 INFO:teuthology.orchestra.run.smithi189.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-24T08:46:45.860 INFO:teuthology.orchestra.run.smithi189.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-24T08:46:45.861 INFO:teuthology.orchestra.run.smithi189.stderr: stderr: 10+0 records in 2024-04-24T08:46:45.861 INFO:teuthology.orchestra.run.smithi189.stderr:10+0 records out 2024-04-24T08:46:45.861 INFO:teuthology.orchestra.run.smithi189.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0360615 s, 291 MB/s 2024-04-24T08:46:45.861 INFO:teuthology.orchestra.run.smithi189.stderr:--> Zapping successful for: 2024-04-24T08:46:46.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:46 smithi138 ceph-mon[31441]: osdmap e76: 11 total, 11 up, 11 in 2024-04-24T08:46:46.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:46 smithi189 ceph-mon[31791]: osdmap e76: 11 total, 11 up, 11 in 2024-04-24T08:46:46.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:46 smithi083 ceph-mon[27907]: osdmap e76: 11 total, 11 up, 11 in 2024-04-24T08:46:46.834 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd3e039e-0215-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi189:vg_nvme/lv_1 2024-04-24T08:46:47.237 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T08:46:47.377 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:47 smithi138 ceph-mon[31441]: pgmap v309: 1 pgs: 1 peering; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-24T08:46:47.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:47 smithi189 ceph-mon[31791]: pgmap v309: 1 pgs: 1 peering; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-24T08:46:47.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:47 smithi083 ceph-mon[27907]: pgmap v309: 1 pgs: 1 peering; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-24T08:46:48.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:48 smithi083 ceph-mon[27907]: Detected new or changed devices on smithi138 2024-04-24T08:46:48.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:48 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:46:48.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:48 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:46:48.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:48 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:46:48.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:48 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:46:48.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:48 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:46:48.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:48 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:46:48.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:48 smithi083 ceph-mon[27907]: Adjusting osd_memory_target on smithi138 to 4272M 2024-04-24T08:46:48.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:48 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:46:48.805 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:48 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:46:48.805 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:48 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:46:48.805 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:48 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:46:48.805 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:48 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:46:48.805 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:48 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:46:48.805 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:48 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:46:48.805 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:48 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:46:48.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:48 smithi138 ceph-mon[31441]: Detected new or changed devices on smithi138 2024-04-24T08:46:48.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:48 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:46:48.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:48 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:46:48.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:48 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:46:48.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:48 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:46:48.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:48 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:46:48.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:48 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:46:48.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:48 smithi138 ceph-mon[31441]: Adjusting osd_memory_target on smithi138 to 4272M 2024-04-24T08:46:48.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:48 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:46:48.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:48 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:46:48.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:48 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:46:48.957 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:48 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:46:48.957 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:48 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:46:48.957 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:48 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:46:48.957 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:48 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:46:48.957 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:48 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:46:48.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:48 smithi189 ceph-mon[31791]: Detected new or changed devices on smithi138 2024-04-24T08:46:48.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:48 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:46:48.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:48 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:46:48.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:48 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:46:48.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:48 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:46:48.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:48 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:46:48.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:48 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:46:48.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:48 smithi189 ceph-mon[31791]: Adjusting osd_memory_target on smithi138 to 4272M 2024-04-24T08:46:48.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:48 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:46:48.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:48 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:46:48.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:48 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:46:48.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:48 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:46:48.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:48 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:46:48.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:48 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:46:48.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:48 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:46:48.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:48 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:46:49.491 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:46:49.490+0000 7fa201c00700 1 -- 172.21.15.189:0/2707340718 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa1fc104c90 msgr2=0x7fa1fc1050b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:46:49.491 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:46:49.490+0000 7fa201c00700 1 --2- 172.21.15.189:0/2707340718 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa1fc104c90 0x7fa1fc1050b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec009990 tx=0x7fa1ec009ca0 comp rx=0 tx=0).stop 2024-04-24T08:46:49.491 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:46:49.491+0000 7fa201c00700 1 -- 172.21.15.189:0/2707340718 shutdown_connections 2024-04-24T08:46:49.492 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:46:49.491+0000 7fa201c00700 1 --2- 172.21.15.189:0/2707340718 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa1fc106830 0x7fa1fc10d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:46:49.492 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:46:49.491+0000 7fa201c00700 1 --2- 172.21.15.189:0/2707340718 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa1fc104c90 0x7fa1fc1050b0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:46:49.492 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:46:49.491+0000 7fa201c00700 1 --2- 172.21.15.189:0/2707340718 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1fc105e90 0x7fa1fc1062f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:46:49.492 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:46:49.491+0000 7fa201c00700 1 -- 172.21.15.189:0/2707340718 >> 172.21.15.189:0/2707340718 conn(0x7fa1fc100230 msgr2=0x7fa1fc102670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:46:49.492 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:46:49.491+0000 7fa201c00700 1 -- 172.21.15.189:0/2707340718 shutdown_connections 2024-04-24T08:46:49.492 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:46:49.491+0000 7fa201c00700 1 -- 172.21.15.189:0/2707340718 wait complete. 2024-04-24T08:46:49.494 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:46:49.492+0000 7fa201c00700 1 Processor -- start 2024-04-24T08:46:49.494 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:46:49.492+0000 7fa201c00700 1 -- start start 2024-04-24T08:46:49.494 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:46:49.492+0000 7fa201c00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1fc104c90 0x7fa1fc19ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:46:49.494 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:46:49.492+0000 7fa201c00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa1fc105e90 0x7fa1fc19b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:46:49.494 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:46:49.492+0000 7fa201c00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa1fc106830 0x7fa1fc1a23e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:46:49.494 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:46:49.492+0000 7fa201c00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa1fc110750 con 0x7fa1fc104c90 2024-04-24T08:46:49.495 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:46:49.492+0000 7fa201c00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa1fc1105d0 con 0x7fa1fc106830 2024-04-24T08:46:49.495 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:46:49.492+0000 7fa201c00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa1fc1108d0 con 0x7fa1fc105e90 2024-04-24T08:46:49.495 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:46:49.493+0000 7fa1fb400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa1fc105e90 0x7fa1fc19b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:46:49.495 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:46:49.493+0000 7fa1fb400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa1fc105e90 0x7fa1fc19b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.189:47160/0 (socket says 172.21.15.189:47160) 2024-04-24T08:46:49.495 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:46:49.493+0000 7fa1fb400700 1 -- 172.21.15.189:0/2140620743 learned_addr learned my addr 172.21.15.189:0/2140620743 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-04-24T08:46:49.495 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:46:49.493+0000 7fa1fbe00700 1 --2- 172.21.15.189:0/2140620743 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1fc104c90 0x7fa1fc19ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:46:49.495 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:46:49.493+0000 7fa201200700 1 --2- 172.21.15.189:0/2140620743 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa1fc106830 0x7fa1fc1a23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:46:49.495 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:46:49.493+0000 7fa1fb400700 1 -- 172.21.15.189:0/2140620743 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa1fc106830 msgr2=0x7fa1fc1a23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:46:49.495 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:46:49.493+0000 7fa1fb400700 1 --2- 172.21.15.189:0/2140620743 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa1fc106830 0x7fa1fc1a23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:46:49.495 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:46:49.493+0000 7fa1fb400700 1 -- 172.21.15.189:0/2140620743 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1fc104c90 msgr2=0x7fa1fc19ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:46:49.495 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:46:49.493+0000 7fa1fb400700 1 --2- 172.21.15.189:0/2140620743 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1fc104c90 0x7fa1fc19ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:46:49.495 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:46:49.493+0000 7fa1fb400700 1 -- 172.21.15.189:0/2140620743 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1fc1a2a80 con 0x7fa1fc105e90 2024-04-24T08:46:49.496 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:46:49.493+0000 7fa1fb400700 1 --2- 172.21.15.189:0/2140620743 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa1fc105e90 0x7fa1fc19b360 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa1e800c950 tx=0x7fa1e800cd10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:46:49.496 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:46:49.494+0000 7fa1f8c00700 1 -- 172.21.15.189:0/2140620743 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e8007ac0 con 0x7fa1fc105e90 2024-04-24T08:46:49.496 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:46:49.494+0000 7fa1f8c00700 1 -- 172.21.15.189:0/2140620743 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa1e8004ce0 con 0x7fa1fc105e90 2024-04-24T08:46:49.496 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:46:49.494+0000 7fa201c00700 1 -- 172.21.15.189:0/2140620743 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1fc1a2d70 con 0x7fa1fc105e90 2024-04-24T08:46:49.496 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:46:49.494+0000 7fa1f8c00700 1 -- 172.21.15.189:0/2140620743 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e80055e0 con 0x7fa1fc105e90 2024-04-24T08:46:49.496 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:46:49.494+0000 7fa201c00700 1 -- 172.21.15.189:0/2140620743 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1fc1a3250 con 0x7fa1fc105e90 2024-04-24T08:46:49.496 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:46:49.495+0000 7fa201c00700 1 -- 172.21.15.189:0/2140620743 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1fc0697e0 con 0x7fa1fc105e90 2024-04-24T08:46:49.497 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:46:49.497+0000 7fa1f8c00700 1 -- 172.21.15.189:0/2140620743 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fa1e8004750 con 0x7fa1fc105e90 2024-04-24T08:46:49.499 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:46:49.499+0000 7fa1f8c00700 1 --2- 172.21.15.189:0/2140620743 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fa1e00a3c40 0x7fa1e00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:46:49.499 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:46:49.499+0000 7fa1f8c00700 1 -- 172.21.15.189:0/2140620743 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6694+0+0 (secure 0 0 0) 0x7fa1e80cfe70 con 0x7fa1fc105e90 2024-04-24T08:46:49.499 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:46:49.499+0000 7fa1fbe00700 1 --2- 172.21.15.189:0/2140620743 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fa1e00a3c40 0x7fa1e00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:46:49.500 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:46:49.501+0000 7fa1fbe00700 1 --2- 172.21.15.189:0/2140620743 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fa1e00a3c40 0x7fa1e00a6100 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec00b5c0 tx=0x7fa1ec0038e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:46:49.502 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:46:49.502+0000 7fa1f8c00700 1 -- 172.21.15.189:0/2140620743 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa1e8014030 con 0x7fa1fc105e90 2024-04-24T08:46:49.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:49 smithi189 ceph-mon[31791]: pgmap v310: 1 pgs: 1 peering; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-04-24T08:46:49.738 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:46:49.738+0000 7fa201c00700 1 -- 172.21.15.189:0/2140620743 --> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fa1fc062f30 con 0x7fa1e00a3c40 2024-04-24T08:46:49.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:49 smithi083 ceph-mon[27907]: pgmap v310: 1 pgs: 1 peering; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-04-24T08:46:49.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:49 smithi138 ceph-mon[31441]: pgmap v310: 1 pgs: 1 peering; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-04-24T08:46:50.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:50 smithi083 ceph-mon[27907]: from='client.24331 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:46:50.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:50 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T08:46:50.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:50 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T08:46:50.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:50 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:46:50.915 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:50 smithi189 ceph-mon[31791]: from='client.24331 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:46:50.915 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:50 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T08:46:50.915 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:50 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T08:46:50.915 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:50 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:46:50.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:50 smithi138 ceph-mon[31441]: from='client.24331 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:46:50.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:50 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T08:46:50.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:50 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T08:46:50.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:50 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:46:51.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:51 smithi083 ceph-mon[27907]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-04-24T08:46:51.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:51 smithi138 ceph-mon[31441]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-04-24T08:46:51.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:51 smithi189 ceph-mon[31791]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-04-24T08:46:54.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:53 smithi138 ceph-mon[31441]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-04-24T08:46:54.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:53 smithi138 ceph-mon[31441]: from='client.? 172.21.15.189:0/2184590045' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8e37568e-981c-45b8-b2af-2f1b73898542"}]: dispatch 2024-04-24T08:46:54.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:53 smithi138 ceph-mon[31441]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8e37568e-981c-45b8-b2af-2f1b73898542"}]: dispatch 2024-04-24T08:46:54.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:53 smithi138 ceph-mon[31441]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8e37568e-981c-45b8-b2af-2f1b73898542"}]': finished 2024-04-24T08:46:54.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:53 smithi138 ceph-mon[31441]: osdmap e77: 12 total, 11 up, 12 in 2024-04-24T08:46:54.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:53 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T08:46:54.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:53 smithi189 ceph-mon[31791]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-04-24T08:46:54.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:53 smithi189 ceph-mon[31791]: from='client.? 172.21.15.189:0/2184590045' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8e37568e-981c-45b8-b2af-2f1b73898542"}]: dispatch 2024-04-24T08:46:54.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:53 smithi189 ceph-mon[31791]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8e37568e-981c-45b8-b2af-2f1b73898542"}]: dispatch 2024-04-24T08:46:54.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:53 smithi189 ceph-mon[31791]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8e37568e-981c-45b8-b2af-2f1b73898542"}]': finished 2024-04-24T08:46:54.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:53 smithi189 ceph-mon[31791]: osdmap e77: 12 total, 11 up, 12 in 2024-04-24T08:46:54.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:53 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T08:46:54.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:53 smithi083 ceph-mon[27907]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-04-24T08:46:54.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:53 smithi083 ceph-mon[27907]: from='client.? 172.21.15.189:0/2184590045' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8e37568e-981c-45b8-b2af-2f1b73898542"}]: dispatch 2024-04-24T08:46:54.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:53 smithi083 ceph-mon[27907]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8e37568e-981c-45b8-b2af-2f1b73898542"}]: dispatch 2024-04-24T08:46:54.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:53 smithi083 ceph-mon[27907]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8e37568e-981c-45b8-b2af-2f1b73898542"}]': finished 2024-04-24T08:46:54.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:53 smithi083 ceph-mon[27907]: osdmap e77: 12 total, 11 up, 12 in 2024-04-24T08:46:54.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:53 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T08:46:55.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:54 smithi138 ceph-mon[31441]: from='client.? 172.21.15.189:0/933059920' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T08:46:55.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:54 smithi189 ceph-mon[31791]: from='client.? 172.21.15.189:0/933059920' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T08:46:55.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:54 smithi083 ceph-mon[27907]: from='client.? 172.21.15.189:0/933059920' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T08:46:56.128 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:55 smithi189 ceph-mon[31791]: pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-24T08:46:56.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:55 smithi138 ceph-mon[31441]: pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-24T08:46:56.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:55 smithi083 ceph-mon[27907]: pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-24T08:46:58.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:57 smithi138 ceph-mon[31441]: pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-24T08:46:58.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:57 smithi189 ceph-mon[31791]: pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-24T08:46:58.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:57 smithi083 ceph-mon[27907]: pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-24T08:47:00.143 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:46:59 smithi189 ceph-mon[31791]: pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-24T08:47:00.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:46:59 smithi138 ceph-mon[31441]: pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-24T08:47:00.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:46:59 smithi083 ceph-mon[27907]: pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-24T08:47:02.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:01 smithi138 ceph-mon[31441]: pgmap v317: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-24T08:47:02.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:01 smithi189 ceph-mon[31791]: pgmap v317: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-24T08:47:02.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:01 smithi083 ceph-mon[27907]: pgmap v317: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-24T08:47:04.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:03 smithi138 ceph-mon[31441]: pgmap v318: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-24T08:47:04.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:03 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-24T08:47:04.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:03 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:47:04.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:03 smithi189 ceph-mon[31791]: pgmap v318: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-24T08:47:04.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:03 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-24T08:47:04.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:03 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:47:04.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:03 smithi083 ceph-mon[27907]: pgmap v318: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-24T08:47:04.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:03 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-24T08:47:04.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:03 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:47:05.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:04 smithi138 ceph-mon[31441]: Deploying daemon osd.11 on smithi189 2024-04-24T08:47:05.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:04 smithi189 ceph-mon[31791]: Deploying daemon osd.11 on smithi189 2024-04-24T08:47:05.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:04 smithi083 ceph-mon[27907]: Deploying daemon osd.11 on smithi189 2024-04-24T08:47:06.182 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:05 smithi189 ceph-mon[31791]: pgmap v319: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-24T08:47:06.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:05 smithi138 ceph-mon[31441]: pgmap v319: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-24T08:47:06.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:05 smithi083 ceph-mon[27907]: pgmap v319: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-24T08:47:08.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:07 smithi138 ceph-mon[31441]: pgmap v320: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-24T08:47:08.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:07 smithi189 ceph-mon[31791]: pgmap v320: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-24T08:47:08.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:07 smithi083 ceph-mon[27907]: pgmap v320: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-24T08:47:10.187 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:09 smithi189 ceph-mon[31791]: pgmap v321: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-24T08:47:10.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:09 smithi138 ceph-mon[31441]: pgmap v321: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-24T08:47:10.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:09 smithi083 ceph-mon[27907]: pgmap v321: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-24T08:47:12.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:11 smithi138 ceph-mon[31441]: pgmap v322: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-24T08:47:12.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:11 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:47:12.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:11 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:47:12.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:11 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:47:12.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:11 smithi189 ceph-mon[31791]: pgmap v322: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-24T08:47:12.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:11 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:47:12.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:11 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:47:12.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:11 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:47:12.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:11 smithi083 ceph-mon[27907]: pgmap v322: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-24T08:47:12.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:11 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:47:12.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:11 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:47:12.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:11 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:47:14.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:13 smithi138 ceph-mon[31441]: pgmap v323: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-24T08:47:14.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:13 smithi189 ceph-mon[31791]: pgmap v323: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-24T08:47:14.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:13 smithi083 ceph-mon[27907]: pgmap v323: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-24T08:47:16.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:15 smithi138 ceph-mon[31441]: pgmap v324: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-24T08:47:16.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:15 smithi138 ceph-mon[31441]: from='osd.11 [v2:172.21.15.189:6824/2998782690,v1:172.21.15.189:6825/2998782690]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-24T08:47:16.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:15 smithi138 ceph-mon[31441]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-24T08:47:16.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:15 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:47:16.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:15 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:47:16.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:15 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:47:16.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:15 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:47:16.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:15 smithi189 ceph-mon[31791]: pgmap v324: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-24T08:47:16.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:15 smithi189 ceph-mon[31791]: from='osd.11 [v2:172.21.15.189:6824/2998782690,v1:172.21.15.189:6825/2998782690]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-24T08:47:16.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:15 smithi189 ceph-mon[31791]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-24T08:47:16.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:15 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:47:16.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:15 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:47:16.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:15 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:47:16.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:15 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:47:16.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:15 smithi083 ceph-mon[27907]: pgmap v324: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-24T08:47:16.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:15 smithi083 ceph-mon[27907]: from='osd.11 [v2:172.21.15.189:6824/2998782690,v1:172.21.15.189:6825/2998782690]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-24T08:47:16.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:15 smithi083 ceph-mon[27907]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-24T08:47:16.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:15 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:47:16.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:15 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:47:16.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:15 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:47:16.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:15 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:47:16.908 INFO:teuthology.orchestra.run.smithi189.stdout:Created osd(s) 11 on host 'smithi189' 2024-04-24T08:47:16.908 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:47:16.907+0000 7fa1f8c00700 1 -- 172.21.15.189:0/2140620743 <== mgr.14152 v2:172.21.15.83:6800/2550826560 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fa1fc062f30 con 0x7fa1e00a3c40 2024-04-24T08:47:16.911 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:47:16.910+0000 7fa201c00700 1 -- 172.21.15.189:0/2140620743 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fa1e00a3c40 msgr2=0x7fa1e00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:47:16.911 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:47:16.910+0000 7fa201c00700 1 --2- 172.21.15.189:0/2140620743 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fa1e00a3c40 0x7fa1e00a6100 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec00b5c0 tx=0x7fa1ec0038e0 comp rx=0 tx=0).stop 2024-04-24T08:47:16.911 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:47:16.910+0000 7fa201c00700 1 -- 172.21.15.189:0/2140620743 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa1fc105e90 msgr2=0x7fa1fc19b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:47:16.911 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:47:16.910+0000 7fa201c00700 1 --2- 172.21.15.189:0/2140620743 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa1fc105e90 0x7fa1fc19b360 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa1e800c950 tx=0x7fa1e800cd10 comp rx=0 tx=0).stop 2024-04-24T08:47:16.911 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:47:16.910+0000 7fa201c00700 1 -- 172.21.15.189:0/2140620743 shutdown_connections 2024-04-24T08:47:16.911 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:47:16.911+0000 7fa201c00700 1 --2- 172.21.15.189:0/2140620743 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fa1e00a3c40 0x7fa1e00a6100 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:16.911 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:47:16.911+0000 7fa201c00700 1 --2- 172.21.15.189:0/2140620743 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1fc104c90 0x7fa1fc19ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:16.911 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:47:16.911+0000 7fa201c00700 1 --2- 172.21.15.189:0/2140620743 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa1fc106830 0x7fa1fc1a23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:16.911 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:47:16.911+0000 7fa201c00700 1 --2- 172.21.15.189:0/2140620743 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa1fc105e90 0x7fa1fc19b360 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:16.912 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:47:16.911+0000 7fa201c00700 1 -- 172.21.15.189:0/2140620743 >> 172.21.15.189:0/2140620743 conn(0x7fa1fc100230 msgr2=0x7fa1fc1092b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:47:16.912 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:47:16.911+0000 7fa201c00700 1 -- 172.21.15.189:0/2140620743 shutdown_connections 2024-04-24T08:47:16.912 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:47:16.911+0000 7fa201c00700 1 -- 172.21.15.189:0/2140620743 wait complete. 2024-04-24T08:47:17.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:16 smithi189 ceph-mon[31791]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-24T08:47:17.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:16 smithi189 ceph-mon[31791]: from='osd.11 [v2:172.21.15.189:6824/2998782690,v1:172.21.15.189:6825/2998782690]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-04-24T08:47:17.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:16 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:47:17.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:16 smithi189 ceph-mon[31791]: osdmap e78: 12 total, 11 up, 12 in 2024-04-24T08:47:17.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:16 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T08:47:17.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:16 smithi189 ceph-mon[31791]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-04-24T08:47:17.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:16 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:47:17.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:16 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:47:17.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:16 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:47:17.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:16 smithi083 ceph-mon[27907]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-24T08:47:17.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:16 smithi083 ceph-mon[27907]: from='osd.11 [v2:172.21.15.189:6824/2998782690,v1:172.21.15.189:6825/2998782690]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-04-24T08:47:17.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:16 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:47:17.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:16 smithi083 ceph-mon[27907]: osdmap e78: 12 total, 11 up, 12 in 2024-04-24T08:47:17.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:16 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T08:47:17.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:16 smithi083 ceph-mon[27907]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-04-24T08:47:17.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:16 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:47:17.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:16 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:47:17.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:16 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:47:17.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:16 smithi138 ceph-mon[31441]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-24T08:47:17.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:16 smithi138 ceph-mon[31441]: from='osd.11 [v2:172.21.15.189:6824/2998782690,v1:172.21.15.189:6825/2998782690]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-04-24T08:47:17.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:16 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:47:17.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:16 smithi138 ceph-mon[31441]: osdmap e78: 12 total, 11 up, 12 in 2024-04-24T08:47:17.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:16 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T08:47:17.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:16 smithi138 ceph-mon[31441]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-04-24T08:47:17.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:16 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:47:17.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:16 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:47:17.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:16 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:47:17.684 DEBUG:teuthology.orchestra.run.smithi189:osd.11> sudo journalctl -f -n 0 -u ceph-bd3e039e-0215-11ef-bc93-c7b262605968@osd.11.service 2024-04-24T08:47:17.689 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-04-24T08:47:17.689 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd3e039e-0215-11ef-bc93-c7b262605968 -- ceph osd stat -f json 2024-04-24T08:47:18.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:17 smithi083 ceph-mon[27907]: pgmap v326: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-24T08:47:18.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:17 smithi083 ceph-mon[27907]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-04-24T08:47:18.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:17 smithi083 ceph-mon[27907]: osdmap e79: 12 total, 11 up, 12 in 2024-04-24T08:47:18.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:17 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T08:47:18.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:17 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T08:47:18.129 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T08:47:18.204 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:17 smithi189 ceph-mon[31791]: pgmap v326: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-24T08:47:18.204 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:17 smithi189 ceph-mon[31791]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-04-24T08:47:18.204 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:17 smithi189 ceph-mon[31791]: osdmap e79: 12 total, 11 up, 12 in 2024-04-24T08:47:18.204 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:17 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T08:47:18.204 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:17 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T08:47:18.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:17 smithi138 ceph-mon[31441]: pgmap v326: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-24T08:47:18.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:17 smithi138 ceph-mon[31441]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-04-24T08:47:18.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:17 smithi138 ceph-mon[31441]: osdmap e79: 12 total, 11 up, 12 in 2024-04-24T08:47:18.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:17 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T08:47:18.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:17 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T08:47:19.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:18 smithi083 ceph-mon[27907]: purged_snaps scrub starts 2024-04-24T08:47:19.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:18 smithi083 ceph-mon[27907]: purged_snaps scrub ok 2024-04-24T08:47:19.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:18 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T08:47:19.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:18 smithi083 ceph-mon[27907]: osd.11 [v2:172.21.15.189:6824/2998782690,v1:172.21.15.189:6825/2998782690] boot 2024-04-24T08:47:19.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:18 smithi083 ceph-mon[27907]: osdmap e80: 12 total, 12 up, 12 in 2024-04-24T08:47:19.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:18 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T08:47:19.316 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:18 smithi189 ceph-mon[31791]: purged_snaps scrub starts 2024-04-24T08:47:19.317 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:18 smithi189 ceph-mon[31791]: purged_snaps scrub ok 2024-04-24T08:47:19.317 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:18 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T08:47:19.317 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:18 smithi189 ceph-mon[31791]: osd.11 [v2:172.21.15.189:6824/2998782690,v1:172.21.15.189:6825/2998782690] boot 2024-04-24T08:47:19.317 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:18 smithi189 ceph-mon[31791]: osdmap e80: 12 total, 12 up, 12 in 2024-04-24T08:47:19.317 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:18 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T08:47:19.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:18 smithi138 ceph-mon[31441]: purged_snaps scrub starts 2024-04-24T08:47:19.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:18 smithi138 ceph-mon[31441]: purged_snaps scrub ok 2024-04-24T08:47:19.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:18 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T08:47:19.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:18 smithi138 ceph-mon[31441]: osd.11 [v2:172.21.15.189:6824/2998782690,v1:172.21.15.189:6825/2998782690] boot 2024-04-24T08:47:19.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:18 smithi138 ceph-mon[31441]: osdmap e80: 12 total, 12 up, 12 in 2024-04-24T08:47:19.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:18 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T08:47:20.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:19 smithi083 ceph-mon[27907]: pgmap v329: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-24T08:47:20.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:19 smithi083 ceph-mon[27907]: osdmap e81: 12 total, 12 up, 12 in 2024-04-24T08:47:20.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:19 smithi138 ceph-mon[31441]: pgmap v329: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-24T08:47:20.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:19 smithi138 ceph-mon[31441]: osdmap e81: 12 total, 12 up, 12 in 2024-04-24T08:47:20.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:19 smithi189 ceph-mon[31791]: pgmap v329: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-24T08:47:20.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:19 smithi189 ceph-mon[31791]: osdmap e81: 12 total, 12 up, 12 in 2024-04-24T08:47:20.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:20.578+0000 7f9cedc00700 1 -- 172.21.15.83:0/2394536694 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ce81035f0 msgr2=0x7f9ce8109ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:47:20.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:20.578+0000 7f9cedc00700 1 --2- 172.21.15.83:0/2394536694 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ce81035f0 0x7f9ce8109ea0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9ce000b0d0 tx=0x7f9ce000b3e0 comp rx=0 tx=0).stop 2024-04-24T08:47:20.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:20.579+0000 7f9cedc00700 1 -- 172.21.15.83:0/2394536694 shutdown_connections 2024-04-24T08:47:20.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:20.579+0000 7f9cedc00700 1 --2- 172.21.15.83:0/2394536694 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ce81035f0 0x7f9ce8109ea0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:20.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:20.579+0000 7f9cedc00700 1 --2- 172.21.15.83:0/2394536694 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9ce8101a50 0x7f9ce8101e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:20.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:20.579+0000 7f9cedc00700 1 --2- 172.21.15.83:0/2394536694 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9ce8102c50 0x7f9ce81030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:20.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:20.579+0000 7f9cedc00700 1 -- 172.21.15.83:0/2394536694 >> 172.21.15.83:0/2394536694 conn(0x7f9ce80fcff0 msgr2=0x7f9ce80ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:47:20.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:20.579+0000 7f9cedc00700 1 -- 172.21.15.83:0/2394536694 shutdown_connections 2024-04-24T08:47:20.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:20.579+0000 7f9cedc00700 1 -- 172.21.15.83:0/2394536694 wait complete. 2024-04-24T08:47:20.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:20.580+0000 7f9cedc00700 1 Processor -- start 2024-04-24T08:47:20.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:20.580+0000 7f9cedc00700 1 -- start start 2024-04-24T08:47:20.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:20.580+0000 7f9cedc00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9ce8101a50 0x7f9ce819ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:47:20.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:20.580+0000 7f9cedc00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9ce8102c50 0x7f9ce819b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:47:20.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:20.580+0000 7f9cedc00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ce81035f0 0x7f9ce81a2400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:47:20.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:20.580+0000 7f9cedc00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce806b440 con 0x7f9ce81035f0 2024-04-24T08:47:20.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:20.580+0000 7f9cedc00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce806b2c0 con 0x7f9ce8101a50 2024-04-24T08:47:20.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:20.580+0000 7f9cedc00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce806b5c0 con 0x7f9ce8102c50 2024-04-24T08:47:20.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:20.581+0000 7f9ce7400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ce81035f0 0x7f9ce81a2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:47:20.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:20.581+0000 7f9ce7400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ce81035f0 0x7f9ce81a2400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42676/0 (socket says 172.21.15.83:42676) 2024-04-24T08:47:20.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:20.581+0000 7f9ce7400700 1 -- 172.21.15.83:0/598963964 learned_addr learned my addr 172.21.15.83:0/598963964 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:47:20.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:20.581+0000 7f9cdfe00700 1 --2- 172.21.15.83:0/598963964 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9ce8102c50 0x7f9ce819b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:47:20.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:20.581+0000 7f9ce6a00700 1 --2- 172.21.15.83:0/598963964 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9ce8101a50 0x7f9ce819ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:47:20.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:20.581+0000 7f9ce7400700 1 -- 172.21.15.83:0/598963964 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9ce8102c50 msgr2=0x7f9ce819b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:47:20.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:20.581+0000 7f9ce7400700 1 --2- 172.21.15.83:0/598963964 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9ce8102c50 0x7f9ce819b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:20.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:20.581+0000 7f9ce7400700 1 -- 172.21.15.83:0/598963964 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9ce8101a50 msgr2=0x7f9ce819ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:47:20.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:20.581+0000 7f9ce7400700 1 --2- 172.21.15.83:0/598963964 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9ce8101a50 0x7f9ce819ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:20.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:20.581+0000 7f9ce7400700 1 -- 172.21.15.83:0/598963964 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ce81a2aa0 con 0x7f9ce81035f0 2024-04-24T08:47:20.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:20.581+0000 7f9ce7400700 1 --2- 172.21.15.83:0/598963964 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ce81035f0 0x7f9ce81a2400 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9ce00079d0 tx=0x7f9ce0007ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:47:20.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:20.582+0000 7f9cdf400700 1 -- 172.21.15.83:0/598963964 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ce0021070 con 0x7f9ce81035f0 2024-04-24T08:47:20.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:20.582+0000 7f9cdf400700 1 -- 172.21.15.83:0/598963964 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9ce001db00 con 0x7f9ce81035f0 2024-04-24T08:47:20.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:20.582+0000 7f9cedc00700 1 -- 172.21.15.83:0/598963964 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ce81a2d30 con 0x7f9ce81035f0 2024-04-24T08:47:20.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:20.582+0000 7f9cdf400700 1 -- 172.21.15.83:0/598963964 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ce001bbf0 con 0x7f9ce81035f0 2024-04-24T08:47:20.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:20.582+0000 7f9cedc00700 1 -- 172.21.15.83:0/598963964 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ce81a31d0 con 0x7f9ce81035f0 2024-04-24T08:47:20.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:20.583+0000 7f9cedc00700 1 -- 172.21.15.83:0/598963964 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ce80697e0 con 0x7f9ce81035f0 2024-04-24T08:47:20.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:20.585+0000 7f9cdf400700 1 -- 172.21.15.83:0/598963964 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9ce0024070 con 0x7f9ce81035f0 2024-04-24T08:47:20.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:20.587+0000 7f9cdf400700 1 --2- 172.21.15.83:0/598963964 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f9ccc0a3cc0 0x7f9ccc0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:47:20.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:20.588+0000 7f9cdf400700 1 -- 172.21.15.83:0/598963964 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f9ce000f020 con 0x7f9ce81035f0 2024-04-24T08:47:20.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:20.588+0000 7f9ce6a00700 1 --2- 172.21.15.83:0/598963964 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f9ccc0a3cc0 0x7f9ccc0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:47:20.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:20.588+0000 7f9ce6a00700 1 --2- 172.21.15.83:0/598963964 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f9ccc0a3cc0 0x7f9ccc0a6180 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9cd8005fd0 tx=0x7f9cd8005dd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:47:20.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:20.591+0000 7f9cdf400700 1 -- 172.21.15.83:0/598963964 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9ce002b020 con 0x7f9ce81035f0 2024-04-24T08:47:20.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:20.819+0000 7f9cedc00700 1 -- 172.21.15.83:0/598963964 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f9ce81a3640 con 0x7f9ce81035f0 2024-04-24T08:47:20.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:20.819+0000 7f9cdf400700 1 -- 172.21.15.83:0/598963964 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v81) ==== 74+0+133 (secure 0 0 0) 0x7f9ce009ecf0 con 0x7f9ce81035f0 2024-04-24T08:47:20.820 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:47:20.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:20.822+0000 7f9cedc00700 1 -- 172.21.15.83:0/598963964 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f9ccc0a3cc0 msgr2=0x7f9ccc0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:47:20.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:20.822+0000 7f9cedc00700 1 --2- 172.21.15.83:0/598963964 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f9ccc0a3cc0 0x7f9ccc0a6180 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9cd8005fd0 tx=0x7f9cd8005dd0 comp rx=0 tx=0).stop 2024-04-24T08:47:20.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:20.822+0000 7f9cedc00700 1 -- 172.21.15.83:0/598963964 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ce81035f0 msgr2=0x7f9ce81a2400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:47:20.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:20.822+0000 7f9cedc00700 1 --2- 172.21.15.83:0/598963964 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ce81035f0 0x7f9ce81a2400 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9ce00079d0 tx=0x7f9ce0007ab0 comp rx=0 tx=0).stop 2024-04-24T08:47:20.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:20.822+0000 7f9cedc00700 1 -- 172.21.15.83:0/598963964 shutdown_connections 2024-04-24T08:47:20.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:20.822+0000 7f9cedc00700 1 --2- 172.21.15.83:0/598963964 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f9ccc0a3cc0 0x7f9ccc0a6180 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:20.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:20.822+0000 7f9cedc00700 1 --2- 172.21.15.83:0/598963964 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ce81035f0 0x7f9ce81a2400 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:20.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:20.822+0000 7f9cedc00700 1 --2- 172.21.15.83:0/598963964 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9ce8101a50 0x7f9ce819ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:20.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:20.822+0000 7f9cedc00700 1 --2- 172.21.15.83:0/598963964 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9ce8102c50 0x7f9ce819b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:20.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:20.822+0000 7f9cedc00700 1 -- 172.21.15.83:0/598963964 >> 172.21.15.83:0/598963964 conn(0x7f9ce80fcff0 msgr2=0x7f9ce810c8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:47:20.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:20.822+0000 7f9cedc00700 1 -- 172.21.15.83:0/598963964 shutdown_connections 2024-04-24T08:47:20.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:20.823+0000 7f9cedc00700 1 -- 172.21.15.83:0/598963964 wait complete. 2024-04-24T08:47:21.223 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:21 smithi083 ceph-mon[27907]: pgmap v331: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:47:21.223 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:21 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/598963964' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-24T08:47:21.392 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:21 smithi189 ceph-mon[31791]: pgmap v331: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:47:21.392 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:21 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/598963964' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-24T08:47:21.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:21 smithi138 ceph-mon[31441]: pgmap v331: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:47:21.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:21 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/598963964' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-24T08:47:21.694 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":81,"num_osds":12,"num_up_osds":12,"osd_up_since":1713948437,"num_in_osds":12,"osd_in_since":1713948413,"num_remapped_pgs":0} 2024-04-24T08:47:21.695 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid bd3e039e-0215-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-24T08:47:22.130 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T08:47:23.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:23 smithi083 ceph-mon[27907]: Detected new or changed devices on smithi189 2024-04-24T08:47:23.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:23 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:47:23.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:23 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:47:23.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:23 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:47:23.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:23 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:47:23.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:23 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:47:23.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:23 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:47:23.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:23 smithi083 ceph-mon[27907]: Adjusting osd_memory_target on smithi189 to 4272M 2024-04-24T08:47:23.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:23 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:47:23.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:23 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:47:23.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:23 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:47:23.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:23 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:47:23.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:23 smithi083 ceph-mon[27907]: pgmap v332: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:47:23.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:23 smithi138 ceph-mon[31441]: Detected new or changed devices on smithi189 2024-04-24T08:47:23.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:23 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:47:23.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:23 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:47:23.706 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:23 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:47:23.706 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:23 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:47:23.706 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:23 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:47:23.706 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:23 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:47:23.706 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:23 smithi138 ceph-mon[31441]: Adjusting osd_memory_target on smithi189 to 4272M 2024-04-24T08:47:23.706 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:23 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:47:23.706 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:23 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:47:23.706 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:23 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:47:23.706 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:23 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:47:23.706 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:23 smithi138 ceph-mon[31441]: pgmap v332: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:47:23.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:23 smithi189 ceph-mon[31791]: Detected new or changed devices on smithi189 2024-04-24T08:47:23.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:23 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:47:23.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:23 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:47:23.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:23 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:47:23.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:23 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:47:23.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:23 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:47:23.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:23 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:47:23.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:23 smithi189 ceph-mon[31791]: Adjusting osd_memory_target on smithi189 to 4272M 2024-04-24T08:47:23.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:23 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:47:23.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:23 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:47:23.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:23 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:47:23.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:23 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:47:23.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:23 smithi189 ceph-mon[31791]: pgmap v332: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:47:24.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:24.492+0000 7fc194a00700 1 -- 172.21.15.83:0/3654070763 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc190105810 msgr2=0x7fc190109860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:47:24.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:24.492+0000 7fc194a00700 1 --2- 172.21.15.83:0/3654070763 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc190105810 0x7fc190109860 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc17c0099c0 tx=0x7fc17c009cd0 comp rx=0 tx=0).stop 2024-04-24T08:47:24.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:24.492+0000 7fc194a00700 1 -- 172.21.15.83:0/3654070763 shutdown_connections 2024-04-24T08:47:24.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:24.492+0000 7fc194a00700 1 --2- 172.21.15.83:0/3654070763 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc190109f40 0x7fc190111b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:24.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:24.492+0000 7fc194a00700 1 --2- 172.21.15.83:0/3654070763 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc190104e60 0x7fc190105240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:24.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:24.492+0000 7fc194a00700 1 --2- 172.21.15.83:0/3654070763 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc190105810 0x7fc190109860 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:24.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:24.492+0000 7fc194a00700 1 -- 172.21.15.83:0/3654070763 >> 172.21.15.83:0/3654070763 conn(0x7fc190100730 msgr2=0x7fc190102b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:47:24.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:24.493+0000 7fc194a00700 1 -- 172.21.15.83:0/3654070763 shutdown_connections 2024-04-24T08:47:24.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:24.493+0000 7fc194a00700 1 -- 172.21.15.83:0/3654070763 wait complete. 2024-04-24T08:47:24.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:24.494+0000 7fc194a00700 1 Processor -- start 2024-04-24T08:47:24.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:24.494+0000 7fc194a00700 1 -- start start 2024-04-24T08:47:24.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:24.495+0000 7fc194a00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc190104e60 0x7fc19019b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:47:24.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:24.495+0000 7fc194a00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc190105810 0x7fc19019b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:47:24.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:24.495+0000 7fc194a00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc190109f40 0x7fc19019fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:47:24.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:24.495+0000 7fc194a00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc190114dd0 con 0x7fc190109f40 2024-04-24T08:47:24.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:24.495+0000 7fc194a00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc190114c50 con 0x7fc190104e60 2024-04-24T08:47:24.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:24.495+0000 7fc194a00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc190114f50 con 0x7fc190105810 2024-04-24T08:47:24.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:24.495+0000 7fc18fe00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc190109f40 0x7fc19019fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:47:24.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:24.495+0000 7fc18fe00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc190109f40 0x7fc19019fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42704/0 (socket says 172.21.15.83:42704) 2024-04-24T08:47:24.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:24.495+0000 7fc18fe00700 1 -- 172.21.15.83:0/381488898 learned_addr learned my addr 172.21.15.83:0/381488898 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:47:24.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:24.495+0000 7fc18f400700 1 --2- 172.21.15.83:0/381488898 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc190104e60 0x7fc19019b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:47:24.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:24.495+0000 7fc18ea00700 1 --2- 172.21.15.83:0/381488898 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc190105810 0x7fc19019b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:47:24.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:24.496+0000 7fc18fe00700 1 -- 172.21.15.83:0/381488898 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc190105810 msgr2=0x7fc19019b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:47:24.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:24.496+0000 7fc18fe00700 1 --2- 172.21.15.83:0/381488898 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc190105810 0x7fc19019b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:24.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:24.496+0000 7fc18fe00700 1 -- 172.21.15.83:0/381488898 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc190104e60 msgr2=0x7fc19019b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:47:24.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:24.496+0000 7fc18fe00700 1 --2- 172.21.15.83:0/381488898 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc190104e60 0x7fc19019b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:24.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:24.496+0000 7fc18fe00700 1 -- 172.21.15.83:0/381488898 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1901a0340 con 0x7fc190109f40 2024-04-24T08:47:24.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:24.496+0000 7fc18ea00700 1 --2- 172.21.15.83:0/381488898 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc190105810 0x7fc19019b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:47:24.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:24.496+0000 7fc18f400700 1 --2- 172.21.15.83:0/381488898 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc190104e60 0x7fc19019b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:47:24.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:24.496+0000 7fc18fe00700 1 --2- 172.21.15.83:0/381488898 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc190109f40 0x7fc19019fc00 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc188007ee0 tx=0x7fc18800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:47:24.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:24.496+0000 7fc187e00700 1 -- 172.21.15.83:0/381488898 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc18800dbd0 con 0x7fc190109f40 2024-04-24T08:47:24.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:24.496+0000 7fc187e00700 1 -- 172.21.15.83:0/381488898 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc18800dd70 con 0x7fc190109f40 2024-04-24T08:47:24.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:24.496+0000 7fc194a00700 1 -- 172.21.15.83:0/381488898 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1901a0630 con 0x7fc190109f40 2024-04-24T08:47:24.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:24.496+0000 7fc187e00700 1 -- 172.21.15.83:0/381488898 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc188005a20 con 0x7fc190109f40 2024-04-24T08:47:24.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:24.497+0000 7fc194a00700 1 -- 172.21.15.83:0/381488898 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1901a7eb0 con 0x7fc190109f40 2024-04-24T08:47:24.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:24.498+0000 7fc194a00700 1 -- 172.21.15.83:0/381488898 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1900697e0 con 0x7fc190109f40 2024-04-24T08:47:24.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:24.500+0000 7fc187e00700 1 -- 172.21.15.83:0/381488898 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fc18801a020 con 0x7fc190109f40 2024-04-24T08:47:24.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:24.502+0000 7fc187e00700 1 --2- 172.21.15.83:0/381488898 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fc1740a3cc0 0x7fc1740a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:47:24.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:24.502+0000 7fc187e00700 1 -- 172.21.15.83:0/381488898 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fc1880cf710 con 0x7fc190109f40 2024-04-24T08:47:24.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:24.502+0000 7fc18f400700 1 --2- 172.21.15.83:0/381488898 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fc1740a3cc0 0x7fc1740a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:47:24.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:24.503+0000 7fc18f400700 1 --2- 172.21.15.83:0/381488898 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fc1740a3cc0 0x7fc1740a6180 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc18000bcf0 tx=0x7fc18000b2c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:47:24.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:24.507+0000 7fc187e00700 1 -- 172.21.15.83:0/381488898 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc188010070 con 0x7fc190109f40 2024-04-24T08:47:24.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:24.732+0000 7fc194a00700 1 -- 172.21.15.83:0/381488898 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc19002dfe0 con 0x7fc190109f40 2024-04-24T08:47:24.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:24.734+0000 7fc187e00700 1 -- 172.21.15.83:0/381488898 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+18455 (secure 0 0 0) 0x7fc18809c4f0 con 0x7fc190109f40 2024-04-24T08:47:24.735 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:47:24.736 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":81,"fsid":"bd3e039e-0215-11ef-bc93-c7b262605968","created":"2024-04-24T08:37:32.209944+0000","modified":"2024-04-24T08:47:18.981804+0000","last_up_change":"2024-04-24T08:47:17.966041+0000","last_in_change":"2024-04-24T08:46:53.835936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T08:41:56.834372+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2cd77f3a-19dd-4c6e-a199-1dcbd5faa679","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2132675555},{"type":"v1","addr":"172.21.15.83:6803","nonce":2132675555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2132675555},{"type":"v1","addr":"172.21.15.83:6805","nonce":2132675555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2132675555},{"type":"v1","addr":"172.21.15.83:6809","nonce":2132675555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2132675555},{"type":"v1","addr":"172.21.15.83:6807","nonce":2132675555}]},"public_addr":"172.21.15.83:6803/2132675555","cluster_addr":"172.21.15.83:6805/2132675555","heartbeat_back_addr":"172.21.15.83:6809/2132675555","heartbeat_front_addr":"172.21.15.83:6807/2132675555","state":["exists","up"]},{"osd":1,"uuid":"aba7f5df-2a4c-4c23-b502-d193605a5b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":3589158880},{"type":"v1","addr":"172.21.15.138:6801","nonce":3589158880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":3589158880},{"type":"v1","addr":"172.21.15.138:6803","nonce":3589158880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":3589158880},{"type":"v1","addr":"172.21.15.138:6807","nonce":3589158880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":3589158880},{"type":"v1","addr":"172.21.15.138:6805","nonce":3589158880}]},"public_addr":"172.21.15.138:6801/3589158880","cluster_addr":"172.21.15.138:6803/3589158880","heartbeat_back_addr":"172.21.15.138:6807/3589158880","heartbeat_front_addr":"172.21.15.138:6805/3589158880","state":["exists","up"]},{"osd":2,"uuid":"071bc167-2333-4420-bb58-bffbc10d7641","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":3169072688},{"type":"v1","addr":"172.21.15.189:6801","nonce":3169072688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":3169072688},{"type":"v1","addr":"172.21.15.189:6803","nonce":3169072688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":3169072688},{"type":"v1","addr":"172.21.15.189:6807","nonce":3169072688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":3169072688},{"type":"v1","addr":"172.21.15.189:6805","nonce":3169072688}]},"public_addr":"172.21.15.189:6801/3169072688","cluster_addr":"172.21.15.189:6803/3169072688","heartbeat_back_addr":"172.21.15.189:6807/3169072688","heartbeat_front_addr":"172.21.15.189:6805/3169072688","state":["exists","up"]},{"osd":3,"uuid":"f5624d78-4455-4e07-8c1a-f47c94cddb23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3131092450},{"type":"v1","addr":"172.21.15.83:6811","nonce":3131092450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3131092450},{"type":"v1","addr":"172.21.15.83:6813","nonce":3131092450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3131092450},{"type":"v1","addr":"172.21.15.83:6817","nonce":3131092450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3131092450},{"type":"v1","addr":"172.21.15.83:6815","nonce":3131092450}]},"public_addr":"172.21.15.83:6811/3131092450","cluster_addr":"172.21.15.83:6813/3131092450","heartbeat_back_addr":"172.21.15.83:6817/3131092450","heartbeat_front_addr":"172.21.15.83:6815/3131092450","state":["exists","up"]},{"osd":4,"uuid":"7b50cd14-9385-4535-a7b0-1c1da93c8548","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":823775360},{"type":"v1","addr":"172.21.15.138:6809","nonce":823775360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":823775360},{"type":"v1","addr":"172.21.15.138:6811","nonce":823775360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":823775360},{"type":"v1","addr":"172.21.15.138:6815","nonce":823775360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":823775360},{"type":"v1","addr":"172.21.15.138:6813","nonce":823775360}]},"public_addr":"172.21.15.138:6809/823775360","cluster_addr":"172.21.15.138:6811/823775360","heartbeat_back_addr":"172.21.15.138:6815/823775360","heartbeat_front_addr":"172.21.15.138:6813/823775360","state":["exists","up"]},{"osd":5,"uuid":"9c647797-35bc-449f-be84-af97d2c69c2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":1399817431},{"type":"v1","addr":"172.21.15.189:6809","nonce":1399817431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":1399817431},{"type":"v1","addr":"172.21.15.189:6811","nonce":1399817431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":1399817431},{"type":"v1","addr":"172.21.15.189:6815","nonce":1399817431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":1399817431},{"type":"v1","addr":"172.21.15.189:6813","nonce":1399817431}]},"public_addr":"172.21.15.189:6809/1399817431","cluster_addr":"172.21.15.189:6811/1399817431","heartbeat_back_addr":"172.21.15.189:6815/1399817431","heartbeat_front_addr":"172.21.15.189:6813/1399817431","state":["exists","up"]},{"osd":6,"uuid":"8f056584-16c8-4454-b445-3f780f26c196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":1611358483},{"type":"v1","addr":"172.21.15.83:6819","nonce":1611358483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1611358483},{"type":"v1","addr":"172.21.15.83:6821","nonce":1611358483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1611358483},{"type":"v1","addr":"172.21.15.83:6825","nonce":1611358483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":1611358483},{"type":"v1","addr":"172.21.15.83:6823","nonce":1611358483}]},"public_addr":"172.21.15.83:6819/1611358483","cluster_addr":"172.21.15.83:6821/1611358483","heartbeat_back_addr":"172.21.15.83:6825/1611358483","heartbeat_front_addr":"172.21.15.83:6823/1611358483","state":["exists","up"]},{"osd":7,"uuid":"7603efe3-da77-4b7f-9d03-29c4f293b6d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":3557856706},{"type":"v1","addr":"172.21.15.138:6817","nonce":3557856706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":3557856706},{"type":"v1","addr":"172.21.15.138:6819","nonce":3557856706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":3557856706},{"type":"v1","addr":"172.21.15.138:6823","nonce":3557856706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":3557856706},{"type":"v1","addr":"172.21.15.138:6821","nonce":3557856706}]},"public_addr":"172.21.15.138:6817/3557856706","cluster_addr":"172.21.15.138:6819/3557856706","heartbeat_back_addr":"172.21.15.138:6823/3557856706","heartbeat_front_addr":"172.21.15.138:6821/3557856706","state":["exists","up"]},{"osd":8,"uuid":"27d7d1b8-d6a0-417e-b8aa-596325aa1a77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":207018554},{"type":"v1","addr":"172.21.15.189:6817","nonce":207018554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":207018554},{"type":"v1","addr":"172.21.15.189:6819","nonce":207018554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":207018554},{"type":"v1","addr":"172.21.15.189:6823","nonce":207018554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":207018554},{"type":"v1","addr":"172.21.15.189:6821","nonce":207018554}]},"public_addr":"172.21.15.189:6817/207018554","cluster_addr":"172.21.15.189:6819/207018554","heartbeat_back_addr":"172.21.15.189:6823/207018554","heartbeat_front_addr":"172.21.15.189:6821/207018554","state":["exists","up"]},{"osd":9,"uuid":"baf19b98-881b-4d3a-9f72-8aa2dd73cad6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":519636751},{"type":"v1","addr":"172.21.15.83:6827","nonce":519636751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":519636751},{"type":"v1","addr":"172.21.15.83:6829","nonce":519636751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":519636751},{"type":"v1","addr":"172.21.15.83:6833","nonce":519636751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":519636751},{"type":"v1","addr":"172.21.15.83:6831","nonce":519636751}]},"public_addr":"172.21.15.83:6827/519636751","cluster_addr":"172.21.15.83:6829/519636751","heartbeat_back_addr":"172.21.15.83:6833/519636751","heartbeat_front_addr":"172.21.15.83:6831/519636751","state":["exists","up"]},{"osd":10,"uuid":"eb346717-5c07-47c8-95be-0e68fe449fdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1130872098},{"type":"v1","addr":"172.21.15.138:6825","nonce":1130872098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1130872098},{"type":"v1","addr":"172.21.15.138:6827","nonce":1130872098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1130872098},{"type":"v1","addr":"172.21.15.138:6831","nonce":1130872098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1130872098},{"type":"v1","addr":"172.21.15.138:6829","nonce":1130872098}]},"public_addr":"172.21.15.138:6825/1130872098","cluster_addr":"172.21.15.138:6827/1130872098","heartbeat_back_addr":"172.21.15.138:6831/1130872098","heartbeat_front_addr":"172.21.15.138:6829/1130872098","state":["exists","up"]},{"osd":11,"uuid":"8e37568e-981c-45b8-b2af-2f1b73898542","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":2998782690},{"type":"v1","addr":"172.21.15.189:6825","nonce":2998782690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":2998782690},{"type":"v1","addr":"172.21.15.189:6827","nonce":2998782690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":2998782690},{"type":"v1","addr":"172.21.15.189:6831","nonce":2998782690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":2998782690},{"type":"v1","addr":"172.21.15.189:6829","nonce":2998782690}]},"public_addr":"172.21.15.189:6825/2998782690","cluster_addr":"172.21.15.189:6827/2998782690","heartbeat_back_addr":"172.21.15.189:6831/2998782690","heartbeat_front_addr":"172.21.15.189:6829/2998782690","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:40:47.980103+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:41:21.970795+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:41:55.160426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:42:30.400475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:43:05.705663+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:43:40.112312+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:44:16.781289+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:44:52.151187+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:45:28.055944+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:46:04.969301+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:46:40.918306+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:47:16.886874+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:6801/820049883":"2024-04-25T08:38:40.661315+0000","172.21.15.83:0/4272874138":"2024-04-25T08:38:40.661315+0000","172.21.15.83:0/4212994335":"2024-04-25T08:38:40.661315+0000","172.21.15.83:6801/1996371091":"2024-04-25T08:38:05.800856+0000","172.21.15.83:0/1820663683":"2024-04-25T08:38:05.800856+0000","172.21.15.83:0/3802897262":"2024-04-25T08:38:05.800856+0000","172.21.15.83:6800/1996371091":"2024-04-25T08:38:05.800856+0000","172.21.15.83:6800/820049883":"2024-04-25T08:38:40.661315+0000","172.21.15.83:0/1366125049":"2024-04-25T08:38:05.800856+0000","172.21.15.83:0/340452868":"2024-04-25T08:38:40.661315+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T08:47:24.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:24.737+0000 7fc194a00700 1 -- 172.21.15.83:0/381488898 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fc1740a3cc0 msgr2=0x7fc1740a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:47:24.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:24.737+0000 7fc194a00700 1 --2- 172.21.15.83:0/381488898 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fc1740a3cc0 0x7fc1740a6180 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc18000bcf0 tx=0x7fc18000b2c0 comp rx=0 tx=0).stop 2024-04-24T08:47:24.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:24.737+0000 7fc194a00700 1 -- 172.21.15.83:0/381488898 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc190109f40 msgr2=0x7fc19019fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:47:24.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:24.737+0000 7fc194a00700 1 --2- 172.21.15.83:0/381488898 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc190109f40 0x7fc19019fc00 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc188007ee0 tx=0x7fc18800d3b0 comp rx=0 tx=0).stop 2024-04-24T08:47:24.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:24.738+0000 7fc194a00700 1 -- 172.21.15.83:0/381488898 shutdown_connections 2024-04-24T08:47:24.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:24.738+0000 7fc194a00700 1 --2- 172.21.15.83:0/381488898 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fc1740a3cc0 0x7fc1740a6180 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:24.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:24.738+0000 7fc194a00700 1 --2- 172.21.15.83:0/381488898 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc190109f40 0x7fc19019fc00 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:24.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:24.738+0000 7fc194a00700 1 --2- 172.21.15.83:0/381488898 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc190104e60 0x7fc19019b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:24.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:24.738+0000 7fc194a00700 1 --2- 172.21.15.83:0/381488898 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc190105810 0x7fc19019b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:24.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:24.738+0000 7fc194a00700 1 -- 172.21.15.83:0/381488898 >> 172.21.15.83:0/381488898 conn(0x7fc190100730 msgr2=0x7fc190102a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:47:24.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:24.738+0000 7fc194a00700 1 -- 172.21.15.83:0/381488898 shutdown_connections 2024-04-24T08:47:24.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:24.738+0000 7fc194a00700 1 -- 172.21.15.83:0/381488898 wait complete. 2024-04-24T08:47:25.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:24 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/381488898' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T08:47:25.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:24 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/381488898' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T08:47:25.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:24 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/381488898' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T08:47:25.572 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T08:41:56.834372+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T08:47:25.572 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid bd3e039e-0215-11ef-bc93-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-04-24T08:47:26.013 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T08:47:26.052 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:25 smithi083 ceph-mon[27907]: pgmap v333: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:47:26.109 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:25 smithi189 ceph-mon[31791]: pgmap v333: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:47:26.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:25 smithi138 ceph-mon[31441]: pgmap v333: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:47:28.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:27 smithi138 ceph-mon[31441]: pgmap v334: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:47:28.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:27 smithi189 ceph-mon[31791]: pgmap v334: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:47:28.217 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:27 smithi083 ceph-mon[27907]: pgmap v334: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:47:28.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:28.385+0000 7f75e2c00700 1 -- 172.21.15.83:0/3296856967 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75dc109d30 msgr2=0x7f75dc10dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:47:28.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:28.385+0000 7f75e2c00700 1 --2- 172.21.15.83:0/3296856967 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75dc109d30 0x7f75dc10dd80 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f75c80099c0 tx=0x7f75c8009cd0 comp rx=0 tx=0).stop 2024-04-24T08:47:28.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:28.385+0000 7f75e2c00700 1 -- 172.21.15.83:0/3296856967 shutdown_connections 2024-04-24T08:47:28.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:28.385+0000 7f75e2c00700 1 --2- 172.21.15.83:0/3296856967 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f75dc10e460 0x7f75dc116030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:28.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:28.385+0000 7f75e2c00700 1 --2- 172.21.15.83:0/3296856967 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f75dc109380 0x7f75dc109760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:28.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:28.385+0000 7f75e2c00700 1 --2- 172.21.15.83:0/3296856967 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75dc109d30 0x7f75dc10dd80 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:28.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:28.385+0000 7f75e2c00700 1 -- 172.21.15.83:0/3296856967 >> 172.21.15.83:0/3296856967 conn(0x7f75dc0762c0 msgr2=0x7f75dc107910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:47:28.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:28.386+0000 7f75e2c00700 1 -- 172.21.15.83:0/3296856967 shutdown_connections 2024-04-24T08:47:28.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:28.386+0000 7f75e2c00700 1 -- 172.21.15.83:0/3296856967 wait complete. 2024-04-24T08:47:28.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:28.387+0000 7f75e2c00700 1 Processor -- start 2024-04-24T08:47:28.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:28.387+0000 7f75e2c00700 1 -- start start 2024-04-24T08:47:28.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:28.387+0000 7f75e2c00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f75dc109380 0x7f75dc1a5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:47:28.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:28.387+0000 7f75e2c00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f75dc109d30 0x7f75dc1a5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:47:28.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:28.387+0000 7f75e2c00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75dc10e460 0x7f75dc19f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:47:28.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:28.387+0000 7f75e2c00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f75dc1193a0 con 0x7f75dc10e460 2024-04-24T08:47:28.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:28.387+0000 7f75e2c00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f75dc119220 con 0x7f75dc109380 2024-04-24T08:47:28.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:28.387+0000 7f75e2c00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f75dc119520 con 0x7f75dc109d30 2024-04-24T08:47:28.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:28.388+0000 7f75e0c00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75dc10e460 0x7f75dc19f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:47:28.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:28.388+0000 7f75e0c00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75dc10e460 0x7f75dc19f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42724/0 (socket says 172.21.15.83:42724) 2024-04-24T08:47:28.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:28.388+0000 7f75e0c00700 1 -- 172.21.15.83:0/959700137 learned_addr learned my addr 172.21.15.83:0/959700137 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:47:28.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:28.388+0000 7f75db400700 1 --2- 172.21.15.83:0/959700137 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f75dc109d30 0x7f75dc1a5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:47:28.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:28.388+0000 7f75dbe00700 1 --2- 172.21.15.83:0/959700137 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f75dc109380 0x7f75dc1a5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:47:28.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:28.388+0000 7f75e0c00700 1 -- 172.21.15.83:0/959700137 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f75dc109d30 msgr2=0x7f75dc1a5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:47:28.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:28.388+0000 7f75e0c00700 1 --2- 172.21.15.83:0/959700137 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f75dc109d30 0x7f75dc1a5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:28.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:28.388+0000 7f75e0c00700 1 -- 172.21.15.83:0/959700137 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f75dc109380 msgr2=0x7f75dc1a5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:47:28.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:28.388+0000 7f75e0c00700 1 --2- 172.21.15.83:0/959700137 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f75dc109380 0x7f75dc1a5600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:28.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:28.388+0000 7f75e0c00700 1 -- 172.21.15.83:0/959700137 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75dc1a0000 con 0x7f75dc10e460 2024-04-24T08:47:28.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:28.388+0000 7f75e0c00700 1 --2- 172.21.15.83:0/959700137 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75dc10e460 0x7f75dc19f720 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f75d400b970 tx=0x7f75d400bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:47:28.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:28.389+0000 7f75d8c00700 1 -- 172.21.15.83:0/959700137 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75d4019070 con 0x7f75dc10e460 2024-04-24T08:47:28.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:28.389+0000 7f75d8c00700 1 -- 172.21.15.83:0/959700137 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f75d4014440 con 0x7f75dc10e460 2024-04-24T08:47:28.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:28.389+0000 7f75e2c00700 1 -- 172.21.15.83:0/959700137 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75dc1a02f0 con 0x7f75dc10e460 2024-04-24T08:47:28.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:28.389+0000 7f75d8c00700 1 -- 172.21.15.83:0/959700137 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75d400ce60 con 0x7f75dc10e460 2024-04-24T08:47:28.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:28.389+0000 7f75e2c00700 1 -- 172.21.15.83:0/959700137 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75dc1ac580 con 0x7f75dc10e460 2024-04-24T08:47:28.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:28.390+0000 7f75e2c00700 1 -- 172.21.15.83:0/959700137 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75dc10a860 con 0x7f75dc10e460 2024-04-24T08:47:28.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:28.392+0000 7f75d8c00700 1 -- 172.21.15.83:0/959700137 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f75d401c070 con 0x7f75dc10e460 2024-04-24T08:47:28.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:28.393+0000 7f75d8c00700 1 --2- 172.21.15.83:0/959700137 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f75c00a3c40 0x7f75c00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:47:28.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:28.394+0000 7f75d8c00700 1 -- 172.21.15.83:0/959700137 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f75d40cf3e0 con 0x7f75dc10e460 2024-04-24T08:47:28.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:28.394+0000 7f75dbe00700 1 --2- 172.21.15.83:0/959700137 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f75c00a3c40 0x7f75c00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:47:28.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:28.394+0000 7f75dbe00700 1 --2- 172.21.15.83:0/959700137 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f75c00a3c40 0x7f75c00a6100 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f75cc005fd0 tx=0x7f75cc00a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:47:28.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:28.398+0000 7f75d8c00700 1 -- 172.21.15.83:0/959700137 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f75d409c1c0 con 0x7f75dc10e460 2024-04-24T08:47:28.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:28.622+0000 7f75e2c00700 1 -- 172.21.15.83:0/959700137 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f75dc1a0c00 con 0x7f75dc10e460 2024-04-24T08:47:28.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:28.623+0000 7f75d8c00700 1 -- 172.21.15.83:0/959700137 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7f75d409bac0 con 0x7f75dc10e460 2024-04-24T08:47:28.623 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-24T08:47:28.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:28.626+0000 7f75e2c00700 1 -- 172.21.15.83:0/959700137 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f75c00a3c40 msgr2=0x7f75c00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:47:28.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:28.626+0000 7f75e2c00700 1 --2- 172.21.15.83:0/959700137 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f75c00a3c40 0x7f75c00a6100 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f75cc005fd0 tx=0x7f75cc00a560 comp rx=0 tx=0).stop 2024-04-24T08:47:28.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:28.626+0000 7f75e2c00700 1 -- 172.21.15.83:0/959700137 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75dc10e460 msgr2=0x7f75dc19f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:47:28.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:28.626+0000 7f75e2c00700 1 --2- 172.21.15.83:0/959700137 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75dc10e460 0x7f75dc19f720 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f75d400b970 tx=0x7f75d400bd30 comp rx=0 tx=0).stop 2024-04-24T08:47:28.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:28.626+0000 7f75e2c00700 1 -- 172.21.15.83:0/959700137 shutdown_connections 2024-04-24T08:47:28.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:28.626+0000 7f75e2c00700 1 --2- 172.21.15.83:0/959700137 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f75c00a3c40 0x7f75c00a6100 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:28.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:28.626+0000 7f75e2c00700 1 --2- 172.21.15.83:0/959700137 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75dc10e460 0x7f75dc19f720 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:28.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:28.626+0000 7f75e2c00700 1 --2- 172.21.15.83:0/959700137 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f75dc109380 0x7f75dc1a5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:28.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:28.626+0000 7f75e2c00700 1 --2- 172.21.15.83:0/959700137 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f75dc109d30 0x7f75dc1a5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:28.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:28.627+0000 7f75e2c00700 1 -- 172.21.15.83:0/959700137 >> 172.21.15.83:0/959700137 conn(0x7f75dc0762c0 msgr2=0x7f75dc077e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:47:28.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:28.627+0000 7f75e2c00700 1 -- 172.21.15.83:0/959700137 shutdown_connections 2024-04-24T08:47:28.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:28.627+0000 7f75e2c00700 1 -- 172.21.15.83:0/959700137 wait complete. 2024-04-24T08:47:29.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:28 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/959700137' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T08:47:29.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:28 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/959700137' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T08:47:29.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:28 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/959700137' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T08:47:29.562 INFO:tasks.cephadm:Adding mds.a on smithi083 2024-04-24T08:47:29.562 INFO:tasks.cephadm:Adding mds.d on smithi083 2024-04-24T08:47:29.562 INFO:tasks.cephadm:Adding mds.g on smithi083 2024-04-24T08:47:29.562 INFO:tasks.cephadm:Adding mds.j on smithi083 2024-04-24T08:47:29.563 INFO:tasks.cephadm:Adding mds.b on smithi138 2024-04-24T08:47:29.563 INFO:tasks.cephadm:Adding mds.e on smithi138 2024-04-24T08:47:29.563 INFO:tasks.cephadm:Adding mds.h on smithi138 2024-04-24T08:47:29.563 INFO:tasks.cephadm:Adding mds.k on smithi138 2024-04-24T08:47:29.563 INFO:tasks.cephadm:Adding mds.c on smithi189 2024-04-24T08:47:29.563 INFO:tasks.cephadm:Adding mds.f on smithi189 2024-04-24T08:47:29.563 INFO:tasks.cephadm:Adding mds.i on smithi189 2024-04-24T08:47:29.563 INFO:tasks.cephadm:Adding mds.l on smithi189 2024-04-24T08:47:29.563 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd3e039e-0215-11ef-bc93-c7b262605968 -- ceph orch apply mds all '12;smithi083=a;smithi083=d;smithi083=g;smithi083=j;smithi138=b;smithi138=e;smithi138=h;smithi138=k;smithi189=c;smithi189=f;smithi189=i;smithi189=l' 2024-04-24T08:47:29.956 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:29 smithi189 ceph-mon[31791]: pgmap v335: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:47:30.009 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T08:47:30.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:29 smithi138 ceph-mon[31441]: pgmap v335: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:47:30.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:29 smithi083 ceph-mon[27907]: pgmap v335: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:47:32.070 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:31 smithi189 ceph-mon[31791]: pgmap v336: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:47:32.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:31 smithi138 ceph-mon[31441]: pgmap v336: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:47:32.243 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:47:32.243+0000 7f9771600700 1 -- 172.21.15.189:0/2547259625 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f976c1035d0 msgr2=0x7f976c1059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:47:32.244 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:47:32.243+0000 7f9771600700 1 --2- 172.21.15.189:0/2547259625 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f976c1035d0 0x7f976c1059c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f975c009990 tx=0x7f975c009ca0 comp rx=0 tx=0).stop 2024-04-24T08:47:32.244 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:47:32.243+0000 7f9771600700 1 -- 172.21.15.189:0/2547259625 shutdown_connections 2024-04-24T08:47:32.244 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:47:32.243+0000 7f9771600700 1 --2- 172.21.15.189:0/2547259625 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f976c108830 0x7f976c10aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:32.244 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:47:32.243+0000 7f9771600700 1 --2- 172.21.15.189:0/2547259625 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f976c1035d0 0x7f976c1059c0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:32.244 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:47:32.243+0000 7f9771600700 1 --2- 172.21.15.189:0/2547259625 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f976c105f00 0x7f976c1082f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:32.244 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:47:32.243+0000 7f9771600700 1 -- 172.21.15.189:0/2547259625 >> 172.21.15.189:0/2547259625 conn(0x7f976c0fcfe0 msgr2=0x7f976c0ff420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:47:32.244 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:47:32.243+0000 7f9771600700 1 -- 172.21.15.189:0/2547259625 shutdown_connections 2024-04-24T08:47:32.244 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:47:32.243+0000 7f9771600700 1 -- 172.21.15.189:0/2547259625 wait complete. 2024-04-24T08:47:32.245 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:47:32.244+0000 7f9771600700 1 Processor -- start 2024-04-24T08:47:32.245 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:47:32.245+0000 7f9771600700 1 -- start start 2024-04-24T08:47:32.246 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:47:32.245+0000 7f9771600700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f976c1035d0 0x7f976c19af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:47:32.246 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:47:32.245+0000 7f9771600700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f976c105f00 0x7f976c19b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:47:32.246 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:47:32.245+0000 7f9771600700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f976c108830 0x7f976c1a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:47:32.246 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:47:32.245+0000 7f9771600700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f976c10e630 con 0x7f976c1035d0 2024-04-24T08:47:32.246 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:47:32.245+0000 7f9771600700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f976c10e4b0 con 0x7f976c105f00 2024-04-24T08:47:32.246 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:47:32.245+0000 7f9771600700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f976c10e7b0 con 0x7f976c108830 2024-04-24T08:47:32.246 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:47:32.245+0000 7f976b400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f976c108830 0x7f976c1a24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:47:32.246 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:47:32.245+0000 7f976b400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f976c108830 0x7f976c1a24e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.189:45460/0 (socket says 172.21.15.189:45460) 2024-04-24T08:47:32.246 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:47:32.245+0000 7f976b400700 1 -- 172.21.15.189:0/2472427726 learned_addr learned my addr 172.21.15.189:0/2472427726 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-04-24T08:47:32.246 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:47:32.245+0000 7f976aa00700 1 --2- 172.21.15.189:0/2472427726 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f976c1035d0 0x7f976c19af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:47:32.246 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:47:32.246+0000 7f976a000700 1 --2- 172.21.15.189:0/2472427726 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f976c105f00 0x7f976c19b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:47:32.246 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:47:32.246+0000 7f976b400700 1 -- 172.21.15.189:0/2472427726 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f976c105f00 msgr2=0x7f976c19b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:47:32.247 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:47:32.246+0000 7f976b400700 1 --2- 172.21.15.189:0/2472427726 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f976c105f00 0x7f976c19b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:32.247 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:47:32.246+0000 7f976b400700 1 -- 172.21.15.189:0/2472427726 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f976c1035d0 msgr2=0x7f976c19af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:47:32.247 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:47:32.246+0000 7f976b400700 1 --2- 172.21.15.189:0/2472427726 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f976c1035d0 0x7f976c19af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:32.247 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:47:32.246+0000 7f976b400700 1 -- 172.21.15.189:0/2472427726 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f976c1a2b80 con 0x7f976c108830 2024-04-24T08:47:32.247 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:47:32.246+0000 7f976aa00700 1 --2- 172.21.15.189:0/2472427726 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f976c1035d0 0x7f976c19af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:47:32.248 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:47:32.246+0000 7f976b400700 1 --2- 172.21.15.189:0/2472427726 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f976c108830 0x7f976c1a24e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9764007ee0 tx=0x7f976400d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:47:32.248 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:47:32.246+0000 7f9763400700 1 -- 172.21.15.189:0/2472427726 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9764004db0 con 0x7f976c108830 2024-04-24T08:47:32.248 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:47:32.246+0000 7f9763400700 1 -- 172.21.15.189:0/2472427726 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f976400dbd0 con 0x7f976c108830 2024-04-24T08:47:32.248 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:47:32.246+0000 7f9771600700 1 -- 172.21.15.189:0/2472427726 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f976c1a2e70 con 0x7f976c108830 2024-04-24T08:47:32.248 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:47:32.247+0000 7f9763400700 1 -- 172.21.15.189:0/2472427726 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9764015400 con 0x7f976c108830 2024-04-24T08:47:32.248 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:47:32.247+0000 7f9771600700 1 -- 172.21.15.189:0/2472427726 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f976c1a3320 con 0x7f976c108830 2024-04-24T08:47:32.248 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:47:32.248+0000 7f9771600700 1 -- 172.21.15.189:0/2472427726 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f976c0697e0 con 0x7f976c108830 2024-04-24T08:47:32.249 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:47:32.250+0000 7f9763400700 1 -- 172.21.15.189:0/2472427726 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9764005020 con 0x7f976c108830 2024-04-24T08:47:32.251 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:47:32.251+0000 7f9763400700 1 --2- 172.21.15.189:0/2472427726 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f97500a3cc0 0x7f97500a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:47:32.251 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:47:32.251+0000 7f9763400700 1 -- 172.21.15.189:0/2472427726 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f97640cfa90 con 0x7f976c108830 2024-04-24T08:47:32.251 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:47:32.252+0000 7f976aa00700 1 --2- 172.21.15.189:0/2472427726 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f97500a3cc0 0x7f97500a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:47:32.252 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:47:32.253+0000 7f976aa00700 1 --2- 172.21.15.189:0/2472427726 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f97500a3cc0 0x7f97500a6180 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f975c0036e0 tx=0x7f975c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:47:32.255 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:47:32.256+0000 7f9763400700 1 -- 172.21.15.189:0/2472427726 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9764010070 con 0x7f976c108830 2024-04-24T08:47:32.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:31 smithi083 ceph-mon[27907]: pgmap v336: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:47:32.493 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:47:32.493+0000 7f9771600700 1 -- 172.21.15.189:0/2472427726 --> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi083=a;smithi083=d;smithi083=g;smithi083=j;smithi138=b;smithi138=e;smithi138=h;smithi138=k;smithi189=c;smithi189=f;smithi189=i;smithi189=l", "target": ["mon-mgr", ""]}) -- 0x7f976c0631f0 con 0x7f97500a3cc0 2024-04-24T08:47:32.511 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:47:32.511+0000 7f9763400700 1 -- 172.21.15.189:0/2472427726 <== mgr.14152 v2:172.21.15.83:6800/2550826560 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f976c0631f0 con 0x7f97500a3cc0 2024-04-24T08:47:32.511 INFO:teuthology.orchestra.run.smithi189.stdout:Scheduled mds.all update... 2024-04-24T08:47:32.514 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:47:32.514+0000 7f9771600700 1 -- 172.21.15.189:0/2472427726 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f97500a3cc0 msgr2=0x7f97500a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:47:32.515 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:47:32.514+0000 7f9771600700 1 --2- 172.21.15.189:0/2472427726 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f97500a3cc0 0x7f97500a6180 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f975c0036e0 tx=0x7f975c005fd0 comp rx=0 tx=0).stop 2024-04-24T08:47:32.515 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:47:32.514+0000 7f9771600700 1 -- 172.21.15.189:0/2472427726 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f976c108830 msgr2=0x7f976c1a24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:47:32.515 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:47:32.514+0000 7f9771600700 1 --2- 172.21.15.189:0/2472427726 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f976c108830 0x7f976c1a24e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9764007ee0 tx=0x7f976400d3b0 comp rx=0 tx=0).stop 2024-04-24T08:47:32.515 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:47:32.514+0000 7f9771600700 1 -- 172.21.15.189:0/2472427726 shutdown_connections 2024-04-24T08:47:32.515 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:47:32.514+0000 7f9771600700 1 --2- 172.21.15.189:0/2472427726 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f97500a3cc0 0x7f97500a6180 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:32.515 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:47:32.514+0000 7f9771600700 1 --2- 172.21.15.189:0/2472427726 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f976c1035d0 0x7f976c19af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:32.515 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:47:32.514+0000 7f9771600700 1 --2- 172.21.15.189:0/2472427726 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f976c108830 0x7f976c1a24e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:32.515 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:47:32.514+0000 7f9771600700 1 --2- 172.21.15.189:0/2472427726 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f976c105f00 0x7f976c19b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:32.515 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:47:32.514+0000 7f9771600700 1 -- 172.21.15.189:0/2472427726 >> 172.21.15.189:0/2472427726 conn(0x7f976c0fcfe0 msgr2=0x7f976c102e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:47:32.515 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:47:32.514+0000 7f9771600700 1 -- 172.21.15.189:0/2472427726 shutdown_connections 2024-04-24T08:47:32.515 INFO:teuthology.orchestra.run.smithi189.stderr:2024-04-24T08:47:32.515+0000 7f9771600700 1 -- 172.21.15.189:0/2472427726 wait complete. 2024-04-24T08:47:33.340 DEBUG:teuthology.orchestra.run.smithi083:mds.a> sudo journalctl -f -n 0 -u ceph-bd3e039e-0215-11ef-bc93-c7b262605968@mds.a.service 2024-04-24T08:47:33.348 DEBUG:teuthology.orchestra.run.smithi083:mds.d> sudo journalctl -f -n 0 -u ceph-bd3e039e-0215-11ef-bc93-c7b262605968@mds.d.service 2024-04-24T08:47:33.394 DEBUG:teuthology.orchestra.run.smithi083:mds.g> sudo journalctl -f -n 0 -u ceph-bd3e039e-0215-11ef-bc93-c7b262605968@mds.g.service 2024-04-24T08:47:33.440 DEBUG:teuthology.orchestra.run.smithi083:mds.j> sudo journalctl -f -n 0 -u ceph-bd3e039e-0215-11ef-bc93-c7b262605968@mds.j.service 2024-04-24T08:47:33.486 DEBUG:teuthology.orchestra.run.smithi138:mds.b> sudo journalctl -f -n 0 -u ceph-bd3e039e-0215-11ef-bc93-c7b262605968@mds.b.service 2024-04-24T08:47:33.494 DEBUG:teuthology.orchestra.run.smithi138:mds.e> sudo journalctl -f -n 0 -u ceph-bd3e039e-0215-11ef-bc93-c7b262605968@mds.e.service 2024-04-24T08:47:33.541 DEBUG:teuthology.orchestra.run.smithi138:mds.h> sudo journalctl -f -n 0 -u ceph-bd3e039e-0215-11ef-bc93-c7b262605968@mds.h.service 2024-04-24T08:47:33.588 DEBUG:teuthology.orchestra.run.smithi138:mds.k> sudo journalctl -f -n 0 -u ceph-bd3e039e-0215-11ef-bc93-c7b262605968@mds.k.service 2024-04-24T08:47:33.624 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:33 smithi189 ceph-mon[31791]: from='client.24367 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi083=a;smithi083=d;smithi083=g;smithi083=j;smithi138=b;smithi138=e;smithi138=h;smithi138=k;smithi189=c;smithi189=f;smithi189=i;smithi189=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:47:33.624 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:33 smithi189 ceph-mon[31791]: Saving service mds.all spec with placement smithi083=a;smithi083=d;smithi083=g;smithi083=j;smithi138=b;smithi138=e;smithi138=h;smithi138=k;smithi189=c;smithi189=f;smithi189=i;smithi189=l;count:12 2024-04-24T08:47:33.624 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:33 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:47:33.624 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:33 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:47:33.625 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:33 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:47:33.625 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:33 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:47:33.625 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:33 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:47:33.625 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:33 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T08:47:33.625 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:33 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T08:47:33.625 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:33 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:47:33.625 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:33 smithi189 ceph-mon[31791]: Deploying daemon mds.l on smithi189 2024-04-24T08:47:33.625 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:33 smithi189 ceph-mon[31791]: pgmap v337: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:47:33.634 DEBUG:teuthology.orchestra.run.smithi189:mds.c> sudo journalctl -f -n 0 -u ceph-bd3e039e-0215-11ef-bc93-c7b262605968@mds.c.service 2024-04-24T08:47:33.642 DEBUG:teuthology.orchestra.run.smithi189:mds.f> sudo journalctl -f -n 0 -u ceph-bd3e039e-0215-11ef-bc93-c7b262605968@mds.f.service 2024-04-24T08:47:33.688 DEBUG:teuthology.orchestra.run.smithi189:mds.i> sudo journalctl -f -n 0 -u ceph-bd3e039e-0215-11ef-bc93-c7b262605968@mds.i.service 2024-04-24T08:47:33.733 DEBUG:teuthology.orchestra.run.smithi189:mds.l> sudo journalctl -f -n 0 -u ceph-bd3e039e-0215-11ef-bc93-c7b262605968@mds.l.service 2024-04-24T08:47:33.780 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-04-24T08:47:33.781 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T08:47:33.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:33 smithi083 ceph-mon[27907]: from='client.24367 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi083=a;smithi083=d;smithi083=g;smithi083=j;smithi138=b;smithi138=e;smithi138=h;smithi138=k;smithi189=c;smithi189=f;smithi189=i;smithi189=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:47:33.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:33 smithi083 ceph-mon[27907]: Saving service mds.all spec with placement smithi083=a;smithi083=d;smithi083=g;smithi083=j;smithi138=b;smithi138=e;smithi138=h;smithi138=k;smithi189=c;smithi189=f;smithi189=i;smithi189=l;count:12 2024-04-24T08:47:33.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:33 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:47:33.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:33 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:47:33.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:33 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:47:33.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:33 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:47:33.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:33 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:47:33.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:33 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T08:47:33.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:33 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T08:47:33.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:33 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:47:33.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:33 smithi083 ceph-mon[27907]: Deploying daemon mds.l on smithi189 2024-04-24T08:47:33.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:33 smithi083 ceph-mon[27907]: pgmap v337: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:47:33.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:33 smithi138 ceph-mon[31441]: from='client.24367 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi083=a;smithi083=d;smithi083=g;smithi083=j;smithi138=b;smithi138=e;smithi138=h;smithi138=k;smithi189=c;smithi189=f;smithi189=i;smithi189=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:47:33.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:33 smithi138 ceph-mon[31441]: Saving service mds.all spec with placement smithi083=a;smithi083=d;smithi083=g;smithi083=j;smithi138=b;smithi138=e;smithi138=h;smithi138=k;smithi189=c;smithi189=f;smithi189=i;smithi189=l;count:12 2024-04-24T08:47:33.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:33 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:47:33.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:33 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:47:33.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:33 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:47:33.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:33 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:47:33.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:33 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:47:33.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:33 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T08:47:33.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:33 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T08:47:33.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:33 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:47:33.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:33 smithi138 ceph-mon[31441]: Deploying daemon mds.l on smithi189 2024-04-24T08:47:33.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:33 smithi138 ceph-mon[31441]: pgmap v337: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:47:34.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.000+0000 7fa89d600640 1 -- 172.21.15.83:0/3798459684 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa898109d60 msgr2=0x7fa89810c1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:47:34.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.000+0000 7fa89d600640 1 --2- 172.21.15.83:0/3798459684 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa898109d60 0x7fa89810c1f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa890009ca0 tx=0x7fa89002a8b0 comp rx=0 tx=0).stop 2024-04-24T08:47:34.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.000+0000 7fa89d600640 1 -- 172.21.15.83:0/3798459684 shutdown_connections 2024-04-24T08:47:34.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.000+0000 7fa89d600640 1 --2- 172.21.15.83:0/3798459684 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa898109d60 0x7fa89810c1f0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:34.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.000+0000 7fa89d600640 1 --2- 172.21.15.83:0/3798459684 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa898074a60 0x7fa898074ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:34.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.000+0000 7fa89d600640 1 --2- 172.21.15.83:0/3798459684 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa898076040 0x7fa898074490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:34.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.000+0000 7fa89d600640 1 -- 172.21.15.83:0/3798459684 >> 172.21.15.83:0/3798459684 conn(0x7fa8980fd410 msgr2=0x7fa8980ff850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:47:34.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.000+0000 7fa89d600640 1 -- 172.21.15.83:0/3798459684 shutdown_connections 2024-04-24T08:47:34.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.000+0000 7fa89d600640 1 -- 172.21.15.83:0/3798459684 wait complete. 2024-04-24T08:47:34.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.001+0000 7fa89d600640 1 Processor -- start 2024-04-24T08:47:34.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.002+0000 7fa89d600640 1 -- start start 2024-04-24T08:47:34.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.002+0000 7fa89d600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa898074a60 0x7fa89819a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:47:34.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.002+0000 7fa89d600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa898076040 0x7fa89819a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:47:34.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.002+0000 7fa89d600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa898109d60 0x7fa8981a1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:47:34.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.002+0000 7fa89d600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa89810fb90 con 0x7fa898109d60 2024-04-24T08:47:34.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.002+0000 7fa89d600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa89810fa10 con 0x7fa898074a60 2024-04-24T08:47:34.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.002+0000 7fa89d600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa89810fd10 con 0x7fa898076040 2024-04-24T08:47:34.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.002+0000 7fa89cc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa898109d60 0x7fa8981a1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:47:34.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.002+0000 7fa89cc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa898109d60 0x7fa8981a1700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56032/0 (socket says 172.21.15.83:56032) 2024-04-24T08:47:34.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.002+0000 7fa89cc00640 1 -- 172.21.15.83:0/609840148 learned_addr learned my addr 172.21.15.83:0/609840148 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:47:34.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.002+0000 7fa88fe00640 1 --2- 172.21.15.83:0/609840148 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa898076040 0x7fa89819a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:47:34.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.003+0000 7fa897e00640 1 --2- 172.21.15.83:0/609840148 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa898074a60 0x7fa89819a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:47:34.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.003+0000 7fa89cc00640 1 -- 172.21.15.83:0/609840148 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa898076040 msgr2=0x7fa89819a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:47:34.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.003+0000 7fa89cc00640 1 --2- 172.21.15.83:0/609840148 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa898076040 0x7fa89819a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:34.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.003+0000 7fa89cc00640 1 -- 172.21.15.83:0/609840148 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa898074a60 msgr2=0x7fa89819a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:47:34.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.003+0000 7fa89cc00640 1 --2- 172.21.15.83:0/609840148 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa898074a60 0x7fa89819a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:34.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.003+0000 7fa89cc00640 1 -- 172.21.15.83:0/609840148 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8981a1d70 con 0x7fa898109d60 2024-04-24T08:47:34.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.003+0000 7fa89cc00640 1 --2- 172.21.15.83:0/609840148 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa898109d60 0x7fa8981a1700 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa890006010 tx=0x7fa890003cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:47:34.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.003+0000 7fa895600640 1 -- 172.21.15.83:0/609840148 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa890004450 con 0x7fa898109d60 2024-04-24T08:47:34.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.003+0000 7fa895600640 1 -- 172.21.15.83:0/609840148 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa890040070 con 0x7fa898109d60 2024-04-24T08:47:34.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.003+0000 7fa89d600640 1 -- 172.21.15.83:0/609840148 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8981a2000 con 0x7fa898109d60 2024-04-24T08:47:34.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.003+0000 7fa895600640 1 -- 172.21.15.83:0/609840148 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa890031440 con 0x7fa898109d60 2024-04-24T08:47:34.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.004+0000 7fa89d600640 1 -- 172.21.15.83:0/609840148 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8981a2550 con 0x7fa898109d60 2024-04-24T08:47:34.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.005+0000 7fa89d600640 1 -- 172.21.15.83:0/609840148 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa898108a90 con 0x7fa898109d60 2024-04-24T08:47:34.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.007+0000 7fa895600640 1 -- 172.21.15.83:0/609840148 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fa890045470 con 0x7fa898109d60 2024-04-24T08:47:34.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.008+0000 7fa895600640 1 --2- 172.21.15.83:0/609840148 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fa87c0a3bb0 0x7fa87c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:47:34.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.008+0000 7fa895600640 1 -- 172.21.15.83:0/609840148 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fa8900f16a0 con 0x7fa898109d60 2024-04-24T08:47:34.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.008+0000 7fa897e00640 1 --2- 172.21.15.83:0/609840148 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fa87c0a3bb0 0x7fa87c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:47:34.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.009+0000 7fa897e00640 1 --2- 172.21.15.83:0/609840148 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fa87c0a3bb0 0x7fa87c0a6070 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa888005e10 tx=0x7fa888005da0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:47:34.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.012+0000 7fa895600640 1 -- 172.21.15.83:0/609840148 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa890039370 con 0x7fa898109d60 2024-04-24T08:47:34.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.219+0000 7fa89d600640 1 -- 172.21.15.83:0/609840148 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa898062c00 con 0x7fa898109d60 2024-04-24T08:47:34.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.222+0000 7fa895600640 1 -- 172.21.15.83:0/609840148 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+18455 (secure 0 0 0) 0x7fa8900be480 con 0x7fa898109d60 2024-04-24T08:47:34.223 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:47:34.223 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":81,"fsid":"bd3e039e-0215-11ef-bc93-c7b262605968","created":"2024-04-24T08:37:32.209944+0000","modified":"2024-04-24T08:47:18.981804+0000","last_up_change":"2024-04-24T08:47:17.966041+0000","last_in_change":"2024-04-24T08:46:53.835936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T08:41:56.834372+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2cd77f3a-19dd-4c6e-a199-1dcbd5faa679","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2132675555},{"type":"v1","addr":"172.21.15.83:6803","nonce":2132675555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2132675555},{"type":"v1","addr":"172.21.15.83:6805","nonce":2132675555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2132675555},{"type":"v1","addr":"172.21.15.83:6809","nonce":2132675555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2132675555},{"type":"v1","addr":"172.21.15.83:6807","nonce":2132675555}]},"public_addr":"172.21.15.83:6803/2132675555","cluster_addr":"172.21.15.83:6805/2132675555","heartbeat_back_addr":"172.21.15.83:6809/2132675555","heartbeat_front_addr":"172.21.15.83:6807/2132675555","state":["exists","up"]},{"osd":1,"uuid":"aba7f5df-2a4c-4c23-b502-d193605a5b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":3589158880},{"type":"v1","addr":"172.21.15.138:6801","nonce":3589158880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":3589158880},{"type":"v1","addr":"172.21.15.138:6803","nonce":3589158880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":3589158880},{"type":"v1","addr":"172.21.15.138:6807","nonce":3589158880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":3589158880},{"type":"v1","addr":"172.21.15.138:6805","nonce":3589158880}]},"public_addr":"172.21.15.138:6801/3589158880","cluster_addr":"172.21.15.138:6803/3589158880","heartbeat_back_addr":"172.21.15.138:6807/3589158880","heartbeat_front_addr":"172.21.15.138:6805/3589158880","state":["exists","up"]},{"osd":2,"uuid":"071bc167-2333-4420-bb58-bffbc10d7641","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":3169072688},{"type":"v1","addr":"172.21.15.189:6801","nonce":3169072688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":3169072688},{"type":"v1","addr":"172.21.15.189:6803","nonce":3169072688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":3169072688},{"type":"v1","addr":"172.21.15.189:6807","nonce":3169072688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":3169072688},{"type":"v1","addr":"172.21.15.189:6805","nonce":3169072688}]},"public_addr":"172.21.15.189:6801/3169072688","cluster_addr":"172.21.15.189:6803/3169072688","heartbeat_back_addr":"172.21.15.189:6807/3169072688","heartbeat_front_addr":"172.21.15.189:6805/3169072688","state":["exists","up"]},{"osd":3,"uuid":"f5624d78-4455-4e07-8c1a-f47c94cddb23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3131092450},{"type":"v1","addr":"172.21.15.83:6811","nonce":3131092450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3131092450},{"type":"v1","addr":"172.21.15.83:6813","nonce":3131092450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3131092450},{"type":"v1","addr":"172.21.15.83:6817","nonce":3131092450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3131092450},{"type":"v1","addr":"172.21.15.83:6815","nonce":3131092450}]},"public_addr":"172.21.15.83:6811/3131092450","cluster_addr":"172.21.15.83:6813/3131092450","heartbeat_back_addr":"172.21.15.83:6817/3131092450","heartbeat_front_addr":"172.21.15.83:6815/3131092450","state":["exists","up"]},{"osd":4,"uuid":"7b50cd14-9385-4535-a7b0-1c1da93c8548","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":823775360},{"type":"v1","addr":"172.21.15.138:6809","nonce":823775360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":823775360},{"type":"v1","addr":"172.21.15.138:6811","nonce":823775360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":823775360},{"type":"v1","addr":"172.21.15.138:6815","nonce":823775360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":823775360},{"type":"v1","addr":"172.21.15.138:6813","nonce":823775360}]},"public_addr":"172.21.15.138:6809/823775360","cluster_addr":"172.21.15.138:6811/823775360","heartbeat_back_addr":"172.21.15.138:6815/823775360","heartbeat_front_addr":"172.21.15.138:6813/823775360","state":["exists","up"]},{"osd":5,"uuid":"9c647797-35bc-449f-be84-af97d2c69c2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":1399817431},{"type":"v1","addr":"172.21.15.189:6809","nonce":1399817431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":1399817431},{"type":"v1","addr":"172.21.15.189:6811","nonce":1399817431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":1399817431},{"type":"v1","addr":"172.21.15.189:6815","nonce":1399817431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":1399817431},{"type":"v1","addr":"172.21.15.189:6813","nonce":1399817431}]},"public_addr":"172.21.15.189:6809/1399817431","cluster_addr":"172.21.15.189:6811/1399817431","heartbeat_back_addr":"172.21.15.189:6815/1399817431","heartbeat_front_addr":"172.21.15.189:6813/1399817431","state":["exists","up"]},{"osd":6,"uuid":"8f056584-16c8-4454-b445-3f780f26c196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":1611358483},{"type":"v1","addr":"172.21.15.83:6819","nonce":1611358483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1611358483},{"type":"v1","addr":"172.21.15.83:6821","nonce":1611358483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1611358483},{"type":"v1","addr":"172.21.15.83:6825","nonce":1611358483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":1611358483},{"type":"v1","addr":"172.21.15.83:6823","nonce":1611358483}]},"public_addr":"172.21.15.83:6819/1611358483","cluster_addr":"172.21.15.83:6821/1611358483","heartbeat_back_addr":"172.21.15.83:6825/1611358483","heartbeat_front_addr":"172.21.15.83:6823/1611358483","state":["exists","up"]},{"osd":7,"uuid":"7603efe3-da77-4b7f-9d03-29c4f293b6d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":3557856706},{"type":"v1","addr":"172.21.15.138:6817","nonce":3557856706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":3557856706},{"type":"v1","addr":"172.21.15.138:6819","nonce":3557856706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":3557856706},{"type":"v1","addr":"172.21.15.138:6823","nonce":3557856706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":3557856706},{"type":"v1","addr":"172.21.15.138:6821","nonce":3557856706}]},"public_addr":"172.21.15.138:6817/3557856706","cluster_addr":"172.21.15.138:6819/3557856706","heartbeat_back_addr":"172.21.15.138:6823/3557856706","heartbeat_front_addr":"172.21.15.138:6821/3557856706","state":["exists","up"]},{"osd":8,"uuid":"27d7d1b8-d6a0-417e-b8aa-596325aa1a77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":207018554},{"type":"v1","addr":"172.21.15.189:6817","nonce":207018554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":207018554},{"type":"v1","addr":"172.21.15.189:6819","nonce":207018554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":207018554},{"type":"v1","addr":"172.21.15.189:6823","nonce":207018554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":207018554},{"type":"v1","addr":"172.21.15.189:6821","nonce":207018554}]},"public_addr":"172.21.15.189:6817/207018554","cluster_addr":"172.21.15.189:6819/207018554","heartbeat_back_addr":"172.21.15.189:6823/207018554","heartbeat_front_addr":"172.21.15.189:6821/207018554","state":["exists","up"]},{"osd":9,"uuid":"baf19b98-881b-4d3a-9f72-8aa2dd73cad6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":519636751},{"type":"v1","addr":"172.21.15.83:6827","nonce":519636751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":519636751},{"type":"v1","addr":"172.21.15.83:6829","nonce":519636751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":519636751},{"type":"v1","addr":"172.21.15.83:6833","nonce":519636751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":519636751},{"type":"v1","addr":"172.21.15.83:6831","nonce":519636751}]},"public_addr":"172.21.15.83:6827/519636751","cluster_addr":"172.21.15.83:6829/519636751","heartbeat_back_addr":"172.21.15.83:6833/519636751","heartbeat_front_addr":"172.21.15.83:6831/519636751","state":["exists","up"]},{"osd":10,"uuid":"eb346717-5c07-47c8-95be-0e68fe449fdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1130872098},{"type":"v1","addr":"172.21.15.138:6825","nonce":1130872098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1130872098},{"type":"v1","addr":"172.21.15.138:6827","nonce":1130872098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1130872098},{"type":"v1","addr":"172.21.15.138:6831","nonce":1130872098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1130872098},{"type":"v1","addr":"172.21.15.138:6829","nonce":1130872098}]},"public_addr":"172.21.15.138:6825/1130872098","cluster_addr":"172.21.15.138:6827/1130872098","heartbeat_back_addr":"172.21.15.138:6831/1130872098","heartbeat_front_addr":"172.21.15.138:6829/1130872098","state":["exists","up"]},{"osd":11,"uuid":"8e37568e-981c-45b8-b2af-2f1b73898542","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":2998782690},{"type":"v1","addr":"172.21.15.189:6825","nonce":2998782690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":2998782690},{"type":"v1","addr":"172.21.15.189:6827","nonce":2998782690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":2998782690},{"type":"v1","addr":"172.21.15.189:6831","nonce":2998782690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":2998782690},{"type":"v1","addr":"172.21.15.189:6829","nonce":2998782690}]},"public_addr":"172.21.15.189:6825/2998782690","cluster_addr":"172.21.15.189:6827/2998782690","heartbeat_back_addr":"172.21.15.189:6831/2998782690","heartbeat_front_addr":"172.21.15.189:6829/2998782690","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:40:47.980103+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:41:21.970795+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:41:55.160426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:42:30.400475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:43:05.705663+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:43:40.112312+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:44:16.781289+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:44:52.151187+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:45:28.055944+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:46:04.969301+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:46:40.918306+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:47:16.886874+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:6801/820049883":"2024-04-25T08:38:40.661315+0000","172.21.15.83:0/4272874138":"2024-04-25T08:38:40.661315+0000","172.21.15.83:0/4212994335":"2024-04-25T08:38:40.661315+0000","172.21.15.83:6801/1996371091":"2024-04-25T08:38:05.800856+0000","172.21.15.83:0/1820663683":"2024-04-25T08:38:05.800856+0000","172.21.15.83:0/3802897262":"2024-04-25T08:38:05.800856+0000","172.21.15.83:6800/1996371091":"2024-04-25T08:38:05.800856+0000","172.21.15.83:6800/820049883":"2024-04-25T08:38:40.661315+0000","172.21.15.83:0/1366125049":"2024-04-25T08:38:05.800856+0000","172.21.15.83:0/340452868":"2024-04-25T08:38:40.661315+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T08:47:34.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.226+0000 7fa89d600640 1 -- 172.21.15.83:0/609840148 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fa87c0a3bb0 msgr2=0x7fa87c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:47:34.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.226+0000 7fa89d600640 1 --2- 172.21.15.83:0/609840148 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fa87c0a3bb0 0x7fa87c0a6070 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa888005e10 tx=0x7fa888005da0 comp rx=0 tx=0).stop 2024-04-24T08:47:34.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.226+0000 7fa89d600640 1 -- 172.21.15.83:0/609840148 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa898109d60 msgr2=0x7fa8981a1700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:47:34.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.226+0000 7fa89d600640 1 --2- 172.21.15.83:0/609840148 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa898109d60 0x7fa8981a1700 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa890006010 tx=0x7fa890003cb0 comp rx=0 tx=0).stop 2024-04-24T08:47:34.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.226+0000 7fa89d600640 1 -- 172.21.15.83:0/609840148 shutdown_connections 2024-04-24T08:47:34.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.226+0000 7fa89d600640 1 --2- 172.21.15.83:0/609840148 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fa87c0a3bb0 0x7fa87c0a6070 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:34.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.226+0000 7fa89d600640 1 --2- 172.21.15.83:0/609840148 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa898109d60 0x7fa8981a1700 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:34.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.226+0000 7fa89d600640 1 --2- 172.21.15.83:0/609840148 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa898076040 0x7fa89819a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:34.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.226+0000 7fa89d600640 1 --2- 172.21.15.83:0/609840148 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa898074a60 0x7fa89819a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:34.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.226+0000 7fa89d600640 1 -- 172.21.15.83:0/609840148 >> 172.21.15.83:0/609840148 conn(0x7fa8980fd410 msgr2=0x7fa898105c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:47:34.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.226+0000 7fa89d600640 1 -- 172.21.15.83:0/609840148 shutdown_connections 2024-04-24T08:47:34.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.226+0000 7fa89d600640 1 -- 172.21.15.83:0/609840148 wait complete. 2024-04-24T08:47:34.247 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T08:41:56.834372+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T08:47:34.247 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T08:47:34.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.498+0000 7f50fb200640 1 -- 172.21.15.83:0/643261351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50f4076040 msgr2=0x7f50f4074490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:47:34.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.498+0000 7f50fb200640 1 --2- 172.21.15.83:0/643261351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50f4076040 0x7f50f4074490 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f50f0009cd0 tx=0x7f50f002a8e0 comp rx=0 tx=0).stop 2024-04-24T08:47:34.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.498+0000 7f50fb200640 1 -- 172.21.15.83:0/643261351 shutdown_connections 2024-04-24T08:47:34.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.498+0000 7f50fb200640 1 --2- 172.21.15.83:0/643261351 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f50f410e560 0x7f50f4110950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:34.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.498+0000 7f50fb200640 1 --2- 172.21.15.83:0/643261351 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f50f4074a60 0x7f50f4074ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:34.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.498+0000 7f50fb200640 1 --2- 172.21.15.83:0/643261351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50f4076040 0x7f50f4074490 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:34.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.498+0000 7f50fb200640 1 -- 172.21.15.83:0/643261351 >> 172.21.15.83:0/643261351 conn(0x7f50f40fd500 msgr2=0x7f50f40ff940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:47:34.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.498+0000 7f50fb200640 1 -- 172.21.15.83:0/643261351 shutdown_connections 2024-04-24T08:47:34.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.498+0000 7f50fb200640 1 -- 172.21.15.83:0/643261351 wait complete. 2024-04-24T08:47:34.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.499+0000 7f50fb200640 1 Processor -- start 2024-04-24T08:47:34.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.499+0000 7f50fb200640 1 -- start start 2024-04-24T08:47:34.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.500+0000 7f50fb200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50f4074a60 0x7f50f419e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:47:34.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.500+0000 7f50fb200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f50f4076040 0x7f50f419ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:47:34.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.500+0000 7f50fb200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f50f410e560 0x7f50f41a5da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:47:34.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.500+0000 7f50fb200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f50f41140f0 con 0x7f50f4074a60 2024-04-24T08:47:34.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.500+0000 7f50fb200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f50f4113f70 con 0x7f50f410e560 2024-04-24T08:47:34.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.500+0000 7f50fb200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f50f4114270 con 0x7f50f4076040 2024-04-24T08:47:34.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.500+0000 7f50f9e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50f4074a60 0x7f50f419e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:47:34.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.500+0000 7f50f9e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50f4074a60 0x7f50f419e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56052/0 (socket says 172.21.15.83:56052) 2024-04-24T08:47:34.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.500+0000 7f50f9e00640 1 -- 172.21.15.83:0/1813196530 learned_addr learned my addr 172.21.15.83:0/1813196530 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:47:34.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.500+0000 7f50fa800640 1 --2- 172.21.15.83:0/1813196530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f50f410e560 0x7f50f41a5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:47:34.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.500+0000 7f50f9e00640 1 -- 172.21.15.83:0/1813196530 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f50f4076040 msgr2=0x7f50f419ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:47:34.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.500+0000 7f50f9e00640 1 --2- 172.21.15.83:0/1813196530 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f50f4076040 0x7f50f419ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:34.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.500+0000 7f50f9e00640 1 -- 172.21.15.83:0/1813196530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f50f410e560 msgr2=0x7f50f41a5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:47:34.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.500+0000 7f50f9e00640 1 --2- 172.21.15.83:0/1813196530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f50f410e560 0x7f50f41a5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:34.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.500+0000 7f50f9e00640 1 -- 172.21.15.83:0/1813196530 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50f41a6410 con 0x7f50f4074a60 2024-04-24T08:47:34.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.500+0000 7f50fa800640 1 --2- 172.21.15.83:0/1813196530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f50f410e560 0x7f50f41a5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:47:34.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.501+0000 7f50f9e00640 1 --2- 172.21.15.83:0/1813196530 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50f4074a60 0x7f50f419e7f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f50f0000f80 tx=0x7f50f0003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:47:34.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.501+0000 7f50eaa00640 1 -- 172.21.15.83:0/1813196530 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f0004450 con 0x7f50f4074a60 2024-04-24T08:47:34.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.501+0000 7f50eaa00640 1 -- 172.21.15.83:0/1813196530 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f50f0003e50 con 0x7f50f4074a60 2024-04-24T08:47:34.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.501+0000 7f50fb200640 1 -- 172.21.15.83:0/1813196530 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50f41a66a0 con 0x7f50f4074a60 2024-04-24T08:47:34.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.501+0000 7f50eaa00640 1 -- 172.21.15.83:0/1813196530 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f003a660 con 0x7f50f4074a60 2024-04-24T08:47:34.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.501+0000 7f50fb200640 1 -- 172.21.15.83:0/1813196530 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50f41a6c70 con 0x7f50f4074a60 2024-04-24T08:47:34.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.503+0000 7f50fb200640 1 -- 172.21.15.83:0/1813196530 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50f4104730 con 0x7f50f4074a60 2024-04-24T08:47:34.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.504+0000 7f50eaa00640 1 -- 172.21.15.83:0/1813196530 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f50f00305a0 con 0x7f50f4074a60 2024-04-24T08:47:34.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.506+0000 7f50eaa00640 1 --2- 172.21.15.83:0/1813196530 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f50e00a3bb0 0x7f50e00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:47:34.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.506+0000 7f50eaa00640 1 -- 172.21.15.83:0/1813196530 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f50f002e030 con 0x7f50f4074a60 2024-04-24T08:47:34.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.506+0000 7f50f9400640 1 --2- 172.21.15.83:0/1813196530 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f50e00a3bb0 0x7f50e00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:47:34.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.507+0000 7f50f9400640 1 --2- 172.21.15.83:0/1813196530 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f50e00a3bb0 0x7f50e00a6070 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f50e4005fd0 tx=0x7f50e4005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:47:34.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.510+0000 7f50eaa00640 1 -- 172.21.15.83:0/1813196530 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f50f00be050 con 0x7f50f4074a60 2024-04-24T08:47:34.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.709+0000 7f50fb200640 1 -- 172.21.15.83:0/1813196530 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f50f402d8d0 con 0x7f50f4074a60 2024-04-24T08:47:34.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.710+0000 7f50eaa00640 1 -- 172.21.15.83:0/1813196530 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7f50f00bd950 con 0x7f50f4074a60 2024-04-24T08:47:34.710 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-24T08:47:34.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.713+0000 7f50fb200640 1 -- 172.21.15.83:0/1813196530 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f50e00a3bb0 msgr2=0x7f50e00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:47:34.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.713+0000 7f50fb200640 1 --2- 172.21.15.83:0/1813196530 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f50e00a3bb0 0x7f50e00a6070 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f50e4005fd0 tx=0x7f50e4005850 comp rx=0 tx=0).stop 2024-04-24T08:47:34.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.713+0000 7f50fb200640 1 -- 172.21.15.83:0/1813196530 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50f4074a60 msgr2=0x7f50f419e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:47:34.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.713+0000 7f50fb200640 1 --2- 172.21.15.83:0/1813196530 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50f4074a60 0x7f50f419e7f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f50f0000f80 tx=0x7f50f0003ba0 comp rx=0 tx=0).stop 2024-04-24T08:47:34.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.713+0000 7f50fb200640 1 -- 172.21.15.83:0/1813196530 shutdown_connections 2024-04-24T08:47:34.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.713+0000 7f50fb200640 1 --2- 172.21.15.83:0/1813196530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f50f410e560 0x7f50f41a5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:34.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.713+0000 7f50fb200640 1 --2- 172.21.15.83:0/1813196530 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f50f4076040 0x7f50f419ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:34.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.713+0000 7f50fb200640 1 --2- 172.21.15.83:0/1813196530 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f50e00a3bb0 0x7f50e00a6070 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:34.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.713+0000 7f50fb200640 1 --2- 172.21.15.83:0/1813196530 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50f4074a60 0x7f50f419e7f0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:34.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.713+0000 7f50fb200640 1 -- 172.21.15.83:0/1813196530 >> 172.21.15.83:0/1813196530 conn(0x7f50f40fd500 msgr2=0x7f50f4101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:47:34.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.713+0000 7f50fb200640 1 -- 172.21.15.83:0/1813196530 shutdown_connections 2024-04-24T08:47:34.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.713+0000 7f50fb200640 1 -- 172.21.15.83:0/1813196530 wait complete. 2024-04-24T08:47:34.732 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T08:47:34.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.935+0000 7f8e01400640 1 -- 172.21.15.83:0/2468365575 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8dfc10a210 msgr2=0x7f8dfc110aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:47:34.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.935+0000 7f8e01400640 1 --2- 172.21.15.83:0/2468365575 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8dfc10a210 0x7f8dfc110aa0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8df4009ca0 tx=0x7f8df402a8c0 comp rx=0 tx=0).stop 2024-04-24T08:47:34.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.935+0000 7f8e01400640 1 -- 172.21.15.83:0/2468365575 shutdown_connections 2024-04-24T08:47:34.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.935+0000 7f8e01400640 1 --2- 172.21.15.83:0/2468365575 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8dfc10a210 0x7f8dfc110aa0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:34.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.935+0000 7f8e01400640 1 --2- 172.21.15.83:0/2468365575 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8dfc109850 0x7f8dfc109cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:34.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.935+0000 7f8e01400640 1 --2- 172.21.15.83:0/2468365575 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8dfc108660 0x7f8dfc108a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:34.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.935+0000 7f8e01400640 1 -- 172.21.15.83:0/2468365575 >> 172.21.15.83:0/2468365575 conn(0x7f8dfc077620 msgr2=0x7f8dfc079a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:47:34.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.935+0000 7f8e01400640 1 -- 172.21.15.83:0/2468365575 shutdown_connections 2024-04-24T08:47:34.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.935+0000 7f8e01400640 1 -- 172.21.15.83:0/2468365575 wait complete. 2024-04-24T08:47:34.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.936+0000 7f8e01400640 1 Processor -- start 2024-04-24T08:47:34.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.936+0000 7f8e01400640 1 -- start start 2024-04-24T08:47:34.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.937+0000 7f8e01400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8dfc108660 0x7f8dfc19e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:47:34.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.937+0000 7f8e01400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8dfc109850 0x7f8dfc19ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:47:34.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.937+0000 7f8e01400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8dfc10a210 0x7f8dfc1a5df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:47:34.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.937+0000 7f8e01400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8dfc114170 con 0x7f8dfc10a210 2024-04-24T08:47:34.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.937+0000 7f8e01400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8dfc113ff0 con 0x7f8dfc108660 2024-04-24T08:47:34.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.937+0000 7f8e01400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8dfc1142f0 con 0x7f8dfc109850 2024-04-24T08:47:34.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.937+0000 7f8dfb200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8dfc10a210 0x7f8dfc1a5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:47:34.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.937+0000 7f8dfb200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8dfc10a210 0x7f8dfc1a5df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56080/0 (socket says 172.21.15.83:56080) 2024-04-24T08:47:34.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.937+0000 7f8dfb200640 1 -- 172.21.15.83:0/3916177073 learned_addr learned my addr 172.21.15.83:0/3916177073 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:47:34.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.937+0000 7f8dfa800640 1 --2- 172.21.15.83:0/3916177073 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8dfc108660 0x7f8dfc19e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:47:34.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.937+0000 7f8df1e00640 1 --2- 172.21.15.83:0/3916177073 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8dfc109850 0x7f8dfc19ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:47:34.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.938+0000 7f8dfb200640 1 -- 172.21.15.83:0/3916177073 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8dfc109850 msgr2=0x7f8dfc19ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:47:34.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.938+0000 7f8dfb200640 1 --2- 172.21.15.83:0/3916177073 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8dfc109850 0x7f8dfc19ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:34.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.938+0000 7f8dfb200640 1 -- 172.21.15.83:0/3916177073 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8dfc108660 msgr2=0x7f8dfc19e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:47:34.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.938+0000 7f8dfb200640 1 --2- 172.21.15.83:0/3916177073 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8dfc108660 0x7f8dfc19e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:34.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.938+0000 7f8dfb200640 1 -- 172.21.15.83:0/3916177073 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dfc1a6460 con 0x7f8dfc10a210 2024-04-24T08:47:34.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.938+0000 7f8dfa800640 1 --2- 172.21.15.83:0/3916177073 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8dfc108660 0x7f8dfc19e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:47:34.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.938+0000 7f8dfb200640 1 --2- 172.21.15.83:0/3916177073 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8dfc10a210 0x7f8dfc1a5df0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8df4031040 tx=0x7f8df4003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:47:34.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.938+0000 7f8df3e00640 1 -- 172.21.15.83:0/3916177073 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df4004040 con 0x7f8dfc10a210 2024-04-24T08:47:34.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.938+0000 7f8df3e00640 1 -- 172.21.15.83:0/3916177073 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8df4004690 con 0x7f8dfc10a210 2024-04-24T08:47:34.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.938+0000 7f8df3e00640 1 -- 172.21.15.83:0/3916177073 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df4030cb0 con 0x7f8dfc10a210 2024-04-24T08:47:34.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.938+0000 7f8e01400640 1 -- 172.21.15.83:0/3916177073 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8dfc1a66f0 con 0x7f8dfc10a210 2024-04-24T08:47:34.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.938+0000 7f8e01400640 1 -- 172.21.15.83:0/3916177073 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8dfc1a6bf0 con 0x7f8dfc10a210 2024-04-24T08:47:34.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.939+0000 7f8e01400640 1 -- 172.21.15.83:0/3916177073 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8dfc10fa20 con 0x7f8dfc10a210 2024-04-24T08:47:34.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.941+0000 7f8df3e00640 1 -- 172.21.15.83:0/3916177073 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8df40041e0 con 0x7f8dfc10a210 2024-04-24T08:47:34.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.944+0000 7f8df3e00640 1 --2- 172.21.15.83:0/3916177073 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f8de00a3c00 0x7f8de00a60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:47:34.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.944+0000 7f8df3e00640 1 -- 172.21.15.83:0/3916177073 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f8df40f15e0 con 0x7f8dfc10a210 2024-04-24T08:47:34.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.944+0000 7f8dfa800640 1 --2- 172.21.15.83:0/3916177073 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f8de00a3c00 0x7f8de00a60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:47:34.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.944+0000 7f8dfa800640 1 --2- 172.21.15.83:0/3916177073 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f8de00a3c00 0x7f8de00a60c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8dec0058c0 tx=0x7f8dec005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:47:34.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:34.948+0000 7f8df3e00640 1 -- 172.21.15.83:0/3916177073 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8df40be3c0 con 0x7f8dfc10a210 2024-04-24T08:47:35.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:35.193+0000 7f8e01400640 1 -- 172.21.15.83:0/3916177073 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8dfc02d620 con 0x7f8dfc10a210 2024-04-24T08:47:35.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:35.195+0000 7f8df3e00640 1 -- 172.21.15.83:0/3916177073 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7f8df4039e10 con 0x7f8dfc10a210 2024-04-24T08:47:35.195 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:47:35.195 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-04-24T08:47:35.195 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 1 2024-04-24T08:47:35.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:35.198+0000 7f8e01400640 1 -- 172.21.15.83:0/3916177073 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f8de00a3c00 msgr2=0x7f8de00a60c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:47:35.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:35.198+0000 7f8e01400640 1 --2- 172.21.15.83:0/3916177073 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f8de00a3c00 0x7f8de00a60c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8dec0058c0 tx=0x7f8dec005850 comp rx=0 tx=0).stop 2024-04-24T08:47:35.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:35.198+0000 7f8e01400640 1 -- 172.21.15.83:0/3916177073 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8dfc10a210 msgr2=0x7f8dfc1a5df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:47:35.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:35.198+0000 7f8e01400640 1 --2- 172.21.15.83:0/3916177073 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8dfc10a210 0x7f8dfc1a5df0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8df4031040 tx=0x7f8df4003ba0 comp rx=0 tx=0).stop 2024-04-24T08:47:35.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:35.198+0000 7f8e01400640 1 -- 172.21.15.83:0/3916177073 shutdown_connections 2024-04-24T08:47:35.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:35.198+0000 7f8e01400640 1 --2- 172.21.15.83:0/3916177073 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f8de00a3c00 0x7f8de00a60c0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:35.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:35.198+0000 7f8e01400640 1 --2- 172.21.15.83:0/3916177073 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8dfc10a210 0x7f8dfc1a5df0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:35.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:35.198+0000 7f8e01400640 1 --2- 172.21.15.83:0/3916177073 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8dfc109850 0x7f8dfc19ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:35.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:35.198+0000 7f8e01400640 1 --2- 172.21.15.83:0/3916177073 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8dfc108660 0x7f8dfc19e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:35.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:35.198+0000 7f8e01400640 1 -- 172.21.15.83:0/3916177073 >> 172.21.15.83:0/3916177073 conn(0x7f8dfc077620 msgr2=0x7f8dfc10cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:47:35.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:35.199+0000 7f8e01400640 1 -- 172.21.15.83:0/3916177073 shutdown_connections 2024-04-24T08:47:35.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:35.199+0000 7f8e01400640 1 -- 172.21.15.83:0/3916177073 wait complete. 2024-04-24T08:47:35.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:35 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/609840148' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T08:47:35.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:35 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/1813196530' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T08:47:35.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:35 smithi138 ceph-mon[31441]: pgmap v338: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:47:35.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:35 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/609840148' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T08:47:35.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:35 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/1813196530' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T08:47:35.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:35 smithi189 ceph-mon[31791]: pgmap v338: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:47:35.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:35 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/609840148' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T08:47:35.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:35 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/1813196530' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T08:47:35.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:35 smithi083 ceph-mon[27907]: pgmap v338: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:47:35.958 INFO:journalctl@ceph.mds.l.smithi189.stdout:Apr 24 08:47:35 smithi189 systemd[1]: Starting Ceph mds.l for bd3e039e-0215-11ef-bc93-c7b262605968... 2024-04-24T08:47:36.282 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:36 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/3916177073' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:47:36.283 INFO:journalctl@ceph.mds.l.smithi189.stdout:Apr 24 08:47:35 smithi189 podman[61087]: 2024-04-24 08:47:35.871589732 +0000 UTC m=+0.035174495 image pull b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644 2024-04-24T08:47:36.283 INFO:journalctl@ceph.mds.l.smithi189.stdout:Apr 24 08:47:36 smithi189 podman[61087]: 2024-04-24 08:47:36.027012634 +0000 UTC m=+0.190597437 container create 4160a48691a987f6590b5634e2e8bdf86df71d71d9351109017c9cdb1a4738ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-l, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424) 2024-04-24T08:47:36.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:36 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/3916177073' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:47:36.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:36 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/3916177073' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:47:36.708 INFO:journalctl@ceph.mds.l.smithi189.stdout:Apr 24 08:47:36 smithi189 podman[61087]: 2024-04-24 08:47:36.331356669 +0000 UTC m=+0.494941512 container init 4160a48691a987f6590b5634e2e8bdf86df71d71d9351109017c9cdb1a4738ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-l, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T08:47:36.708 INFO:journalctl@ceph.mds.l.smithi189.stdout:Apr 24 08:47:36 smithi189 podman[61087]: 2024-04-24 08:47:36.341573062 +0000 UTC m=+0.505157846 container start 4160a48691a987f6590b5634e2e8bdf86df71d71d9351109017c9cdb1a4738ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-l, GIT_CLEAN=True, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=rishabh-main-apr17-a654945-1f81fb3, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-24T08:47:36.708 INFO:journalctl@ceph.mds.l.smithi189.stdout:Apr 24 08:47:36 smithi189 ceph-mds[61107]: -- 172.21.15.189:0/2906227087 <== mon.1 v2:172.21.15.189:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559cf6c61a40 con 0x559cf6c35000 2024-04-24T08:47:36.708 INFO:journalctl@ceph.mds.l.smithi189.stdout:Apr 24 08:47:36 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-l[61103]: starting mds.l at 2024-04-24T08:47:36.708 INFO:journalctl@ceph.mds.l.smithi189.stdout:Apr 24 08:47:36 smithi189 bash[61087]: 4160a48691a987f6590b5634e2e8bdf86df71d71d9351109017c9cdb1a4738ff 2024-04-24T08:47:36.708 INFO:journalctl@ceph.mds.l.smithi189.stdout:Apr 24 08:47:36 smithi189 systemd[1]: Started Ceph mds.l for bd3e039e-0215-11ef-bc93-c7b262605968. 2024-04-24T08:47:37.219 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T08:47:37.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:37.431+0000 7fbf5c200640 1 -- 172.21.15.83:0/2496320671 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf540769c0 msgr2=0x7fbf54076de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:47:37.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:37.431+0000 7fbf5c200640 1 --2- 172.21.15.83:0/2496320671 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf540769c0 0x7fbf54076de0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fbf48009900 tx=0x7fbf4802a410 comp rx=0 tx=0).stop 2024-04-24T08:47:37.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:37.431+0000 7fbf5c200640 1 -- 172.21.15.83:0/2496320671 shutdown_connections 2024-04-24T08:47:37.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:37.431+0000 7fbf5c200640 1 --2- 172.21.15.83:0/2496320671 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf540785a0 0x7fbf54078e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:37.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:37.431+0000 7fbf5c200640 1 --2- 172.21.15.83:0/2496320671 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbf54077c00 0x7fbf54078060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:37.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:37.431+0000 7fbf5c200640 1 --2- 172.21.15.83:0/2496320671 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf540769c0 0x7fbf54076de0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:37.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:37.431+0000 7fbf5c200640 1 -- 172.21.15.83:0/2496320671 >> 172.21.15.83:0/2496320671 conn(0x7fbf540ff7c0 msgr2=0x7fbf54101c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:47:37.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:37.431+0000 7fbf5c200640 1 -- 172.21.15.83:0/2496320671 shutdown_connections 2024-04-24T08:47:37.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:37.432+0000 7fbf5c200640 1 -- 172.21.15.83:0/2496320671 wait complete. 2024-04-24T08:47:37.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:37.432+0000 7fbf5c200640 1 Processor -- start 2024-04-24T08:47:37.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:37.433+0000 7fbf5c200640 1 -- start start 2024-04-24T08:47:37.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:37.433+0000 7fbf5c200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf540769c0 0x7fbf5419e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:47:37.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:37.433+0000 7fbf5c200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf54077c00 0x7fbf5419ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:47:37.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:37.433+0000 7fbf5c200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbf540785a0 0x7fbf541a5e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:47:37.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:37.433+0000 7fbf5c200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbf54114210 con 0x7fbf54077c00 2024-04-24T08:47:37.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:37.433+0000 7fbf5c200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbf54114090 con 0x7fbf540769c0 2024-04-24T08:47:37.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:37.433+0000 7fbf5c200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fbf54114390 con 0x7fbf540785a0 2024-04-24T08:47:37.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:37.433+0000 7fbf58e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf54077c00 0x7fbf5419ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:47:37.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:37.433+0000 7fbf58e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf54077c00 0x7fbf5419ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56108/0 (socket says 172.21.15.83:56108) 2024-04-24T08:47:37.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:37.433+0000 7fbf58e00640 1 -- 172.21.15.83:0/766701748 learned_addr learned my addr 172.21.15.83:0/766701748 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:47:37.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:37.433+0000 7fbf59800640 1 --2- 172.21.15.83:0/766701748 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf540769c0 0x7fbf5419e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:47:37.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:37.433+0000 7fbf5a200640 1 --2- 172.21.15.83:0/766701748 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbf540785a0 0x7fbf541a5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:47:37.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:37.434+0000 7fbf58e00640 1 -- 172.21.15.83:0/766701748 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbf540785a0 msgr2=0x7fbf541a5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:47:37.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:37.434+0000 7fbf58e00640 1 --2- 172.21.15.83:0/766701748 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbf540785a0 0x7fbf541a5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:37.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:37.434+0000 7fbf58e00640 1 -- 172.21.15.83:0/766701748 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf540769c0 msgr2=0x7fbf5419e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:47:37.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:37.434+0000 7fbf58e00640 1 --2- 172.21.15.83:0/766701748 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf540769c0 0x7fbf5419e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:37.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:37.434+0000 7fbf58e00640 1 -- 172.21.15.83:0/766701748 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf541a6500 con 0x7fbf54077c00 2024-04-24T08:47:37.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:37.434+0000 7fbf5a200640 1 --2- 172.21.15.83:0/766701748 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbf540785a0 0x7fbf541a5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:47:37.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:37.434+0000 7fbf58e00640 1 --2- 172.21.15.83:0/766701748 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf54077c00 0x7fbf5419ee20 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fbf4000b4a0 tx=0x7fbf4000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:47:37.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:37.434+0000 7fbf52000640 1 -- 172.21.15.83:0/766701748 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf40014070 con 0x7fbf54077c00 2024-04-24T08:47:37.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:37.434+0000 7fbf52000640 1 -- 172.21.15.83:0/766701748 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbf40010ac0 con 0x7fbf54077c00 2024-04-24T08:47:37.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:37.434+0000 7fbf5c200640 1 -- 172.21.15.83:0/766701748 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf541a67f0 con 0x7fbf54077c00 2024-04-24T08:47:37.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:37.434+0000 7fbf52000640 1 -- 172.21.15.83:0/766701748 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf40017c30 con 0x7fbf54077c00 2024-04-24T08:47:37.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:37.434+0000 7fbf5c200640 1 -- 172.21.15.83:0/766701748 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf541a6dc0 con 0x7fbf54077c00 2024-04-24T08:47:37.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:37.436+0000 7fbf5c200640 1 -- 172.21.15.83:0/766701748 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf54110bc0 con 0x7fbf54077c00 2024-04-24T08:47:37.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:37.438+0000 7fbf52000640 1 -- 172.21.15.83:0/766701748 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fbf40017070 con 0x7fbf54077c00 2024-04-24T08:47:37.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:37.440+0000 7fbf52000640 1 --2- 172.21.15.83:0/766701748 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fbf3c0a3c30 0x7fbf3c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:47:37.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:37.440+0000 7fbf52000640 1 -- 172.21.15.83:0/766701748 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fbf400cf350 con 0x7fbf54077c00 2024-04-24T08:47:37.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:37.440+0000 7fbf59800640 1 --2- 172.21.15.83:0/766701748 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fbf3c0a3c30 0x7fbf3c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:47:37.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:37.441+0000 7fbf59800640 1 --2- 172.21.15.83:0/766701748 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fbf3c0a3c30 0x7fbf3c0a60f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fbf48000c00 tx=0x7fbf480314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:47:37.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:37.444+0000 7fbf52000640 1 -- 172.21.15.83:0/766701748 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbf4009c1d0 con 0x7fbf54077c00 2024-04-24T08:47:37.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:37 smithi083 ceph-mon[27907]: mds.? [v2:172.21.15.189:6832/1859921310,v1:172.21.15.189:6833/1859921310] up:boot 2024-04-24T08:47:37.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:37 smithi083 ceph-mon[27907]: fsmap 1 up:standby 2024-04-24T08:47:37.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:37 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-24T08:47:37.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:37 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:47:37.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:37 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:47:37.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:37 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:47:37.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:37 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T08:47:37.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:37 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T08:47:37.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:37 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:47:37.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:37 smithi083 ceph-mon[27907]: Deploying daemon mds.k on smithi138 2024-04-24T08:47:37.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:37 smithi083 ceph-mon[27907]: pgmap v339: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:47:37.678 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:37 smithi138 ceph-mon[31441]: mds.? [v2:172.21.15.189:6832/1859921310,v1:172.21.15.189:6833/1859921310] up:boot 2024-04-24T08:47:37.678 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:37 smithi138 ceph-mon[31441]: fsmap 1 up:standby 2024-04-24T08:47:37.678 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:37 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-24T08:47:37.678 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:37 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:47:37.678 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:37 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:47:37.678 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:37 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:47:37.678 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:37 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T08:47:37.678 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:37 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T08:47:37.679 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:37 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:47:37.679 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:37 smithi138 ceph-mon[31441]: Deploying daemon mds.k on smithi138 2024-04-24T08:47:37.679 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:37 smithi138 ceph-mon[31441]: pgmap v339: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:47:37.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:37.687+0000 7fbf5c200640 1 -- 172.21.15.83:0/766701748 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbf5402d620 con 0x7fbf54077c00 2024-04-24T08:47:37.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:37.688+0000 7fbf52000640 1 -- 172.21.15.83:0/766701748 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7fbf4009bad0 con 0x7fbf54077c00 2024-04-24T08:47:37.687 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:47:37.687 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/1859921310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1859921310},{"type":"v1","addr":"172.21.15.189:6833","nonce":1859921310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-24T08:47:37.688 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 2 2024-04-24T08:47:37.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:37.690+0000 7fbf5c200640 1 -- 172.21.15.83:0/766701748 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fbf3c0a3c30 msgr2=0x7fbf3c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:47:37.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:37.690+0000 7fbf5c200640 1 --2- 172.21.15.83:0/766701748 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fbf3c0a3c30 0x7fbf3c0a60f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fbf48000c00 tx=0x7fbf480314a0 comp rx=0 tx=0).stop 2024-04-24T08:47:37.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:37.691+0000 7fbf5c200640 1 -- 172.21.15.83:0/766701748 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf54077c00 msgr2=0x7fbf5419ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:47:37.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:37.691+0000 7fbf5c200640 1 --2- 172.21.15.83:0/766701748 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf54077c00 0x7fbf5419ee20 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fbf4000b4a0 tx=0x7fbf4000b960 comp rx=0 tx=0).stop 2024-04-24T08:47:37.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:37.691+0000 7fbf5c200640 1 -- 172.21.15.83:0/766701748 shutdown_connections 2024-04-24T08:47:37.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:37.691+0000 7fbf5c200640 1 --2- 172.21.15.83:0/766701748 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbf540785a0 0x7fbf541a5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:37.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:37.691+0000 7fbf5c200640 1 --2- 172.21.15.83:0/766701748 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fbf3c0a3c30 0x7fbf3c0a60f0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:37.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:37.691+0000 7fbf5c200640 1 --2- 172.21.15.83:0/766701748 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf54077c00 0x7fbf5419ee20 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:37.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:37.691+0000 7fbf5c200640 1 --2- 172.21.15.83:0/766701748 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf540769c0 0x7fbf5419e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:37.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:37.691+0000 7fbf5c200640 1 -- 172.21.15.83:0/766701748 >> 172.21.15.83:0/766701748 conn(0x7fbf540ff7c0 msgr2=0x7fbf54102b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:47:37.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:37.691+0000 7fbf5c200640 1 -- 172.21.15.83:0/766701748 shutdown_connections 2024-04-24T08:47:37.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:37.691+0000 7fbf5c200640 1 -- 172.21.15.83:0/766701748 wait complete. 2024-04-24T08:47:37.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:37 smithi189 ceph-mon[31791]: mds.? [v2:172.21.15.189:6832/1859921310,v1:172.21.15.189:6833/1859921310] up:boot 2024-04-24T08:47:37.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:37 smithi189 ceph-mon[31791]: fsmap 1 up:standby 2024-04-24T08:47:37.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:37 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-24T08:47:37.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:37 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:47:37.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:37 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:47:37.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:37 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:47:37.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:37 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T08:47:37.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:37 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T08:47:37.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:37 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:47:37.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:37 smithi189 ceph-mon[31791]: Deploying daemon mds.k on smithi138 2024-04-24T08:47:37.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:37 smithi189 ceph-mon[31791]: pgmap v339: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:47:38.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:38 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/766701748' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:47:38.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:38 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/766701748' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:47:39.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:38 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/766701748' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:47:39.710 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T08:47:39.920 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:39 smithi138 ceph-mon[31441]: pgmap v340: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:47:39.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:39.921+0000 7f14ab200640 1 -- 172.21.15.83:0/610477697 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14ac101e70 msgr2=0x7f14ac102290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:47:39.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:39.921+0000 7f14ab200640 1 --2- 172.21.15.83:0/610477697 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14ac101e70 0x7f14ac102290 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f149c009900 tx=0x7f149c02a3f0 comp rx=0 tx=0).stop 2024-04-24T08:47:39.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:39.922+0000 7f14ab200640 1 -- 172.21.15.83:0/610477697 shutdown_connections 2024-04-24T08:47:39.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:39.922+0000 7f14ab200640 1 --2- 172.21.15.83:0/610477697 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f14ac103980 0x7f14ac10a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:39.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:39.922+0000 7f14ab200640 1 --2- 172.21.15.83:0/610477697 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f14ac102fc0 0x7f14ac103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:39.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:39.922+0000 7f14ab200640 1 --2- 172.21.15.83:0/610477697 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14ac101e70 0x7f14ac102290 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:39.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:39.922+0000 7f14ab200640 1 -- 172.21.15.83:0/610477697 >> 172.21.15.83:0/610477697 conn(0x7f14ac0fd3e0 msgr2=0x7f14ac0ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:47:39.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:39.922+0000 7f14ab200640 1 -- 172.21.15.83:0/610477697 shutdown_connections 2024-04-24T08:47:39.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:39.922+0000 7f14ab200640 1 -- 172.21.15.83:0/610477697 wait complete. 2024-04-24T08:47:39.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:39.923+0000 7f14ab200640 1 Processor -- start 2024-04-24T08:47:39.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:39.923+0000 7f14ab200640 1 -- start start 2024-04-24T08:47:39.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:39.923+0000 7f14ab200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f14ac101e70 0x7f14ac198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:47:39.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:39.923+0000 7f14ab200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14ac102fc0 0x7f14ac1985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:47:39.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:39.923+0000 7f14ab200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f14ac103980 0x7f14ac19f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:47:39.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:39.923+0000 7f14ab200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f14ac10d9f0 con 0x7f14ac102fc0 2024-04-24T08:47:39.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:39.923+0000 7f14ab200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f14ac10d870 con 0x7f14ac103980 2024-04-24T08:47:39.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:39.923+0000 7f14ab200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f14ac10db70 con 0x7f14ac101e70 2024-04-24T08:47:39.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:39.923+0000 7f14a9400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14ac102fc0 0x7f14ac1985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:47:39.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:39.923+0000 7f14a9400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14ac102fc0 0x7f14ac1985d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58892/0 (socket says 172.21.15.83:58892) 2024-04-24T08:47:39.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:39.923+0000 7f14a9400640 1 -- 172.21.15.83:0/3485169093 learned_addr learned my addr 172.21.15.83:0/3485169093 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:47:39.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:39.924+0000 7f14a9e00640 1 --2- 172.21.15.83:0/3485169093 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f14ac101e70 0x7f14ac198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:47:39.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:39.924+0000 7f14aa800640 1 --2- 172.21.15.83:0/3485169093 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f14ac103980 0x7f14ac19f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:47:39.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:39.924+0000 7f14a9400640 1 -- 172.21.15.83:0/3485169093 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f14ac101e70 msgr2=0x7f14ac198090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:47:39.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:39.924+0000 7f14a9400640 1 --2- 172.21.15.83:0/3485169093 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f14ac101e70 0x7f14ac198090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:39.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:39.924+0000 7f14a9400640 1 -- 172.21.15.83:0/3485169093 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f14ac103980 msgr2=0x7f14ac19f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:47:39.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:39.924+0000 7f14a9400640 1 --2- 172.21.15.83:0/3485169093 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f14ac103980 0x7f14ac19f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:39.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:39.924+0000 7f14a9400640 1 -- 172.21.15.83:0/3485169093 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14ac19fcb0 con 0x7f14ac102fc0 2024-04-24T08:47:39.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:39.924+0000 7f14aa800640 1 --2- 172.21.15.83:0/3485169093 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f14ac103980 0x7f14ac19f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:47:39.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:39.924+0000 7f14a9e00640 1 --2- 172.21.15.83:0/3485169093 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f14ac101e70 0x7f14ac198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:47:39.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:39.924+0000 7f14a9400640 1 --2- 172.21.15.83:0/3485169093 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14ac102fc0 0x7f14ac1985d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f149800ea80 tx=0x7f149800ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:47:39.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:39.924+0000 7f14a2a00640 1 -- 172.21.15.83:0/3485169093 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f149800ce40 con 0x7f14ac102fc0 2024-04-24T08:47:39.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:39.924+0000 7f14a2a00640 1 -- 172.21.15.83:0/3485169093 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1498011920 con 0x7f14ac102fc0 2024-04-24T08:47:39.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:39.924+0000 7f14ab200640 1 -- 172.21.15.83:0/3485169093 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14ac19ffa0 con 0x7f14ac102fc0 2024-04-24T08:47:39.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:39.924+0000 7f14a2a00640 1 -- 172.21.15.83:0/3485169093 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f149801aaa0 con 0x7f14ac102fc0 2024-04-24T08:47:39.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:39.924+0000 7f14ab200640 1 -- 172.21.15.83:0/3485169093 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14ac1a04b0 con 0x7f14ac102fc0 2024-04-24T08:47:39.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:39.926+0000 7f14ab200640 1 -- 172.21.15.83:0/3485169093 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14ac1091d0 con 0x7f14ac102fc0 2024-04-24T08:47:39.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:39.928+0000 7f14a2a00640 1 -- 172.21.15.83:0/3485169093 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f1498011ad0 con 0x7f14ac102fc0 2024-04-24T08:47:39.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:39.930+0000 7f14a2a00640 1 --2- 172.21.15.83:0/3485169093 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f14900a3c30 0x7f14900a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:47:39.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:39.930+0000 7f14a2a00640 1 -- 172.21.15.83:0/3485169093 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f14980d05e0 con 0x7f14ac102fc0 2024-04-24T08:47:39.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:39.930+0000 7f14a9e00640 1 --2- 172.21.15.83:0/3485169093 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f14900a3c30 0x7f14900a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:47:39.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:39.931+0000 7f14a9e00640 1 --2- 172.21.15.83:0/3485169093 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f14900a3c30 0x7f14900a60f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f149c02a910 tx=0x7f149c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:47:39.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:39.934+0000 7f14a2a00640 1 -- 172.21.15.83:0/3485169093 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1498016030 con 0x7f14ac102fc0 2024-04-24T08:47:39.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:39 smithi189 ceph-mon[31791]: pgmap v340: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:47:40.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:39 smithi083 ceph-mon[27907]: pgmap v340: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:47:40.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:40.175+0000 7f14ab200640 1 -- 172.21.15.83:0/3485169093 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f14ac02d620 con 0x7f14ac102fc0 2024-04-24T08:47:40.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:40.176+0000 7f14a2a00640 1 -- 172.21.15.83:0/3485169093 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7f149809d3c0 con 0x7f14ac102fc0 2024-04-24T08:47:40.175 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:47:40.175 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/1859921310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1859921310},{"type":"v1","addr":"172.21.15.189:6833","nonce":1859921310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-24T08:47:40.176 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 2 2024-04-24T08:47:40.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:40.178+0000 7f14ab200640 1 -- 172.21.15.83:0/3485169093 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f14900a3c30 msgr2=0x7f14900a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:47:40.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:40.178+0000 7f14ab200640 1 --2- 172.21.15.83:0/3485169093 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f14900a3c30 0x7f14900a60f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f149c02a910 tx=0x7f149c005fd0 comp rx=0 tx=0).stop 2024-04-24T08:47:40.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:40.178+0000 7f14ab200640 1 -- 172.21.15.83:0/3485169093 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14ac102fc0 msgr2=0x7f14ac1985d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:47:40.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:40.178+0000 7f14ab200640 1 --2- 172.21.15.83:0/3485169093 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14ac102fc0 0x7f14ac1985d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f149800ea80 tx=0x7f149800ef40 comp rx=0 tx=0).stop 2024-04-24T08:47:40.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:40.178+0000 7f14ab200640 1 -- 172.21.15.83:0/3485169093 shutdown_connections 2024-04-24T08:47:40.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:40.178+0000 7f14ab200640 1 --2- 172.21.15.83:0/3485169093 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f14ac103980 0x7f14ac19f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:40.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:40.178+0000 7f14ab200640 1 --2- 172.21.15.83:0/3485169093 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f14900a3c30 0x7f14900a60f0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:40.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:40.178+0000 7f14ab200640 1 --2- 172.21.15.83:0/3485169093 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14ac102fc0 0x7f14ac1985d0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:40.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:40.179+0000 7f14ab200640 1 --2- 172.21.15.83:0/3485169093 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f14ac101e70 0x7f14ac198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:40.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:40.179+0000 7f14ab200640 1 -- 172.21.15.83:0/3485169093 >> 172.21.15.83:0/3485169093 conn(0x7f14ac0fd3e0 msgr2=0x7f14ac1063a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:47:40.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:40.179+0000 7f14ab200640 1 -- 172.21.15.83:0/3485169093 shutdown_connections 2024-04-24T08:47:40.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:40.179+0000 7f14ab200640 1 -- 172.21.15.83:0/3485169093 wait complete. 2024-04-24T08:47:40.205 INFO:journalctl@ceph.mds.k.smithi138.stdout:Apr 24 08:47:39 smithi138 systemd[1]: Starting Ceph mds.k for bd3e039e-0215-11ef-bc93-c7b262605968... 2024-04-24T08:47:40.610 INFO:journalctl@ceph.mds.k.smithi138.stdout:Apr 24 08:47:40 smithi138 podman[60566]: 2024-04-24 08:47:40.195232635 +0000 UTC m=+0.038943495 image pull b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644 2024-04-24T08:47:40.610 INFO:journalctl@ceph.mds.k.smithi138.stdout:Apr 24 08:47:40 smithi138 podman[60566]: 2024-04-24 08:47:40.322303671 +0000 UTC m=+0.166014549 container create 669f8f50454579bf817dd3613dc9a816ee2c9b7037ae95669a22f9d6583024a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-k, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=rishabh-main-apr17-a654945-1f81fb3, io.buildah.version=1.33.5) 2024-04-24T08:47:40.610 INFO:journalctl@ceph.mds.k.smithi138.stdout:Apr 24 08:47:40 smithi138 podman[60566]: 2024-04-24 08:47:40.609328462 +0000 UTC m=+0.453039319 container init 669f8f50454579bf817dd3613dc9a816ee2c9b7037ae95669a22f9d6583024a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-k, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-04-24T08:47:40.864 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:40 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/3485169093' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:47:40.864 INFO:journalctl@ceph.mds.k.smithi138.stdout:Apr 24 08:47:40 smithi138 podman[60566]: 2024-04-24 08:47:40.619716779 +0000 UTC m=+0.463427628 container start 669f8f50454579bf817dd3613dc9a816ee2c9b7037ae95669a22f9d6583024a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-k, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-24T08:47:40.864 INFO:journalctl@ceph.mds.k.smithi138.stdout:Apr 24 08:47:40 smithi138 ceph-mds[60586]: -- 172.21.15.138:0/3573883011 <== mon.2 v2:172.21.15.138:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ffbe04ba40 con 0x55ffbe01e800 2024-04-24T08:47:40.864 INFO:journalctl@ceph.mds.k.smithi138.stdout:Apr 24 08:47:40 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-k[60581]: starting mds.k at 2024-04-24T08:47:40.865 INFO:journalctl@ceph.mds.k.smithi138.stdout:Apr 24 08:47:40 smithi138 bash[60566]: 669f8f50454579bf817dd3613dc9a816ee2c9b7037ae95669a22f9d6583024a6 2024-04-24T08:47:40.865 INFO:journalctl@ceph.mds.k.smithi138.stdout:Apr 24 08:47:40 smithi138 systemd[1]: Started Ceph mds.k for bd3e039e-0215-11ef-bc93-c7b262605968. 2024-04-24T08:47:40.933 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:40 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/3485169093' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:47:40.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:40 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/3485169093' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:47:41.861 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:41 smithi083 ceph-mon[27907]: pgmap v341: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:47:41.862 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:41 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:47:41.862 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:41 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:47:41.862 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:41 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:47:41.862 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:41 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T08:47:41.862 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:41 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T08:47:41.863 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:41 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:47:41.863 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:41 smithi083 ceph-mon[27907]: Deploying daemon mds.j on smithi083 2024-04-24T08:47:41.863 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:41 smithi083 ceph-mon[27907]: mds.? [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] up:boot 2024-04-24T08:47:41.863 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:41 smithi083 ceph-mon[27907]: fsmap 2 up:standby 2024-04-24T08:47:41.863 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:41 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-24T08:47:42.077 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:41 smithi138 ceph-mon[31441]: pgmap v341: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:47:42.077 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:41 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:47:42.077 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:41 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:47:42.077 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:41 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:47:42.077 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:41 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T08:47:42.077 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:41 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T08:47:42.077 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:41 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:47:42.077 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:41 smithi138 ceph-mon[31441]: Deploying daemon mds.j on smithi083 2024-04-24T08:47:42.077 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:41 smithi138 ceph-mon[31441]: mds.? [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] up:boot 2024-04-24T08:47:42.077 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:41 smithi138 ceph-mon[31441]: fsmap 2 up:standby 2024-04-24T08:47:42.077 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:41 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-24T08:47:42.198 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T08:47:42.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:41 smithi189 ceph-mon[31791]: pgmap v341: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:47:42.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:41 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:47:42.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:41 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:47:42.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:41 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:47:42.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:41 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T08:47:42.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:41 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T08:47:42.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:41 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:47:42.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:41 smithi189 ceph-mon[31791]: Deploying daemon mds.j on smithi083 2024-04-24T08:47:42.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:41 smithi189 ceph-mon[31791]: mds.? [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] up:boot 2024-04-24T08:47:42.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:41 smithi189 ceph-mon[31791]: fsmap 2 up:standby 2024-04-24T08:47:42.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:41 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-24T08:47:42.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:42.379+0000 7f2ae1e00640 1 -- 172.21.15.83:0/2158567638 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2adc101ea0 msgr2=0x7f2adc1022c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:47:42.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:42.379+0000 7f2ae1e00640 1 --2- 172.21.15.83:0/2158567638 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2adc101ea0 0x7f2adc1022c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2ad0009900 tx=0x7f2ad002a410 comp rx=0 tx=0).stop 2024-04-24T08:47:42.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:42.379+0000 7f2ae1e00640 1 -- 172.21.15.83:0/2158567638 shutdown_connections 2024-04-24T08:47:42.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:42.379+0000 7f2ae1e00640 1 --2- 172.21.15.83:0/2158567638 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2adc1039b0 0x7f2adc10a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:42.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:42.379+0000 7f2ae1e00640 1 --2- 172.21.15.83:0/2158567638 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2adc102ff0 0x7f2adc103470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:42.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:42.379+0000 7f2ae1e00640 1 --2- 172.21.15.83:0/2158567638 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2adc101ea0 0x7f2adc1022c0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:42.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:42.379+0000 7f2ae1e00640 1 -- 172.21.15.83:0/2158567638 >> 172.21.15.83:0/2158567638 conn(0x7f2adc0fd410 msgr2=0x7f2adc0ff850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:47:42.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:42.379+0000 7f2ae1e00640 1 -- 172.21.15.83:0/2158567638 shutdown_connections 2024-04-24T08:47:42.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:42.379+0000 7f2ae1e00640 1 -- 172.21.15.83:0/2158567638 wait complete. 2024-04-24T08:47:42.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:42.380+0000 7f2ae1e00640 1 Processor -- start 2024-04-24T08:47:42.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:42.380+0000 7f2ae1e00640 1 -- start start 2024-04-24T08:47:42.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:42.380+0000 7f2ae1e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2adc101ea0 0x7f2adc075db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:47:42.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:42.380+0000 7f2ae1e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2adc102ff0 0x7f2adc0743f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:47:42.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:42.380+0000 7f2ae1e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2adc1039b0 0x7f2adc074930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:47:42.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:42.380+0000 7f2ae1e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2adc10d9f0 con 0x7f2adc101ea0 2024-04-24T08:47:42.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:42.380+0000 7f2ae1e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2adc10d870 con 0x7f2adc1039b0 2024-04-24T08:47:42.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:42.381+0000 7f2ae1e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2adc10db70 con 0x7f2adc102ff0 2024-04-24T08:47:42.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:42.381+0000 7f2ae0a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2adc101ea0 0x7f2adc075db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:47:42.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:42.381+0000 7f2ae0a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2adc101ea0 0x7f2adc075db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58906/0 (socket says 172.21.15.83:58906) 2024-04-24T08:47:42.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:42.381+0000 7f2ae0a00640 1 -- 172.21.15.83:0/26386859 learned_addr learned my addr 172.21.15.83:0/26386859 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:47:42.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:42.381+0000 7f2adbe00640 1 --2- 172.21.15.83:0/26386859 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2adc102ff0 0x7f2adc0743f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:47:42.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:42.381+0000 7f2ae0a00640 1 -- 172.21.15.83:0/26386859 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2adc102ff0 msgr2=0x7f2adc0743f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:47:42.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:42.381+0000 7f2ae0a00640 1 --2- 172.21.15.83:0/26386859 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2adc102ff0 0x7f2adc0743f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:42.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:42.381+0000 7f2ae0a00640 1 -- 172.21.15.83:0/26386859 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2adc1039b0 msgr2=0x7f2adc074930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:47:42.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:42.381+0000 7f2ae1400640 1 --2- 172.21.15.83:0/26386859 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2adc1039b0 0x7f2adc074930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:47:42.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:42.381+0000 7f2ae0a00640 1 --2- 172.21.15.83:0/26386859 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2adc1039b0 0x7f2adc074930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:42.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:42.381+0000 7f2ae0a00640 1 -- 172.21.15.83:0/26386859 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2adc0751c0 con 0x7f2adc101ea0 2024-04-24T08:47:42.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:42.381+0000 7f2ae1400640 1 --2- 172.21.15.83:0/26386859 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2adc1039b0 0x7f2adc074930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:47:42.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:42.381+0000 7f2ae0a00640 1 --2- 172.21.15.83:0/26386859 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2adc101ea0 0x7f2adc075db0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2ad0004770 tx=0x7f2ad00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:47:42.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:42.382+0000 7f2ad9600640 1 -- 172.21.15.83:0/26386859 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad003f030 con 0x7f2adc101ea0 2024-04-24T08:47:42.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:42.382+0000 7f2ae1e00640 1 -- 172.21.15.83:0/26386859 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2adc1a21d0 con 0x7f2adc101ea0 2024-04-24T08:47:42.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:42.382+0000 7f2ad9600640 1 -- 172.21.15.83:0/26386859 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2ad0003cb0 con 0x7f2adc101ea0 2024-04-24T08:47:42.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:42.382+0000 7f2ad9600640 1 -- 172.21.15.83:0/26386859 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad0035070 con 0x7f2adc101ea0 2024-04-24T08:47:42.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:42.382+0000 7f2ae1e00640 1 -- 172.21.15.83:0/26386859 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2adc1a2720 con 0x7f2adc101ea0 2024-04-24T08:47:42.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:42.383+0000 7f2ae1e00640 1 -- 172.21.15.83:0/26386859 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2adc109250 con 0x7f2adc101ea0 2024-04-24T08:47:42.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:42.385+0000 7f2ad9600640 1 -- 172.21.15.83:0/26386859 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f2ad0042030 con 0x7f2adc101ea0 2024-04-24T08:47:42.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:42.386+0000 7f2ad9600640 1 --2- 172.21.15.83:0/26386859 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f2ac40a3bb0 0x7f2ac40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:47:42.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:42.387+0000 7f2ad9600640 1 -- 172.21.15.83:0/26386859 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f2ad00f0a40 con 0x7f2adc101ea0 2024-04-24T08:47:42.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:42.387+0000 7f2adbe00640 1 --2- 172.21.15.83:0/26386859 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f2ac40a3bb0 0x7f2ac40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:47:42.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:42.387+0000 7f2adbe00640 1 --2- 172.21.15.83:0/26386859 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f2ac40a3bb0 0x7f2ac40a6070 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2ac8005fd0 tx=0x7f2ac8005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:47:42.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:42.391+0000 7f2ad9600640 1 -- 172.21.15.83:0/26386859 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2ad00bd820 con 0x7f2adc101ea0 2024-04-24T08:47:42.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:42.624+0000 7f2ae1e00640 1 -- 172.21.15.83:0/26386859 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2adc02d860 con 0x7f2adc101ea0 2024-04-24T08:47:42.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:42.624+0000 7f2ad9600640 1 -- 172.21.15.83:0/26386859 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2224 (secure 0 0 0) 0x7f2ad00bd120 con 0x7f2adc101ea0 2024-04-24T08:47:42.625 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:47:42.626 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/1859921310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1859921310},{"type":"v1","addr":"172.21.15.189:6833","nonce":1859921310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":34427,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/1157967658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1157967658},{"type":"v1","addr":"172.21.15.138:6833","nonce":1157967658}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-24T08:47:42.626 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 3 2024-04-24T08:47:42.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:42.627+0000 7f2ae1e00640 1 -- 172.21.15.83:0/26386859 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f2ac40a3bb0 msgr2=0x7f2ac40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:47:42.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:42.627+0000 7f2ae1e00640 1 --2- 172.21.15.83:0/26386859 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f2ac40a3bb0 0x7f2ac40a6070 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2ac8005fd0 tx=0x7f2ac8005850 comp rx=0 tx=0).stop 2024-04-24T08:47:42.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:42.627+0000 7f2ae1e00640 1 -- 172.21.15.83:0/26386859 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2adc101ea0 msgr2=0x7f2adc075db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:47:42.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:42.627+0000 7f2ae1e00640 1 --2- 172.21.15.83:0/26386859 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2adc101ea0 0x7f2adc075db0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2ad0004770 tx=0x7f2ad00042f0 comp rx=0 tx=0).stop 2024-04-24T08:47:42.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:42.627+0000 7f2ae1e00640 1 -- 172.21.15.83:0/26386859 shutdown_connections 2024-04-24T08:47:42.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:42.627+0000 7f2ae1e00640 1 --2- 172.21.15.83:0/26386859 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2adc1039b0 0x7f2adc074930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:42.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:42.627+0000 7f2ae1e00640 1 --2- 172.21.15.83:0/26386859 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2adc102ff0 0x7f2adc0743f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:42.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:42.627+0000 7f2ae1e00640 1 --2- 172.21.15.83:0/26386859 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f2ac40a3bb0 0x7f2ac40a6070 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:42.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:42.627+0000 7f2ae1e00640 1 --2- 172.21.15.83:0/26386859 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2adc101ea0 0x7f2adc075db0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:42.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:42.627+0000 7f2ae1e00640 1 -- 172.21.15.83:0/26386859 >> 172.21.15.83:0/26386859 conn(0x7f2adc0fd410 msgr2=0x7f2adc106420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:47:42.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:42.627+0000 7f2ae1e00640 1 -- 172.21.15.83:0/26386859 shutdown_connections 2024-04-24T08:47:42.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:42.627+0000 7f2ae1e00640 1 -- 172.21.15.83:0/26386859 wait complete. 2024-04-24T08:47:43.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:42 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/26386859' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:47:43.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:42 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/26386859' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:47:43.264 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:42 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/26386859' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:47:44.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:43 smithi083 ceph-mon[27907]: pgmap v342: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:47:44.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:43 smithi138 ceph-mon[31441]: pgmap v342: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:47:44.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:43 smithi189 ceph-mon[31791]: pgmap v342: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:47:44.445 INFO:journalctl@ceph.mds.j.smithi083.stdout:Apr 24 08:47:44 smithi083 systemd[1]: Starting Ceph mds.j for bd3e039e-0215-11ef-bc93-c7b262605968... 2024-04-24T08:47:44.646 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T08:47:44.713 INFO:journalctl@ceph.mds.j.smithi083.stdout:Apr 24 08:47:44 smithi083 podman[68311]: 2024-04-24 08:47:44.34908893 +0000 UTC m=+0.033825148 image pull b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644 2024-04-24T08:47:44.713 INFO:journalctl@ceph.mds.j.smithi083.stdout:Apr 24 08:47:44 smithi083 podman[68311]: 2024-04-24 08:47:44.542331345 +0000 UTC m=+0.227067601 container create 6c5b0557d05b558e79ffd54d24031bd12589a30f2caccbc1d8d8e12c9a164e1b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-j, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True) 2024-04-24T08:47:44.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:44.851+0000 7f3854c00640 1 -- 172.21.15.83:0/1711131519 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3850104080 msgr2=0x7f38501044a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:47:44.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:44.851+0000 7f3854c00640 1 --2- 172.21.15.83:0/1711131519 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3850104080 0x7f38501044a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3840009900 tx=0x7f384002a3f0 comp rx=0 tx=0).stop 2024-04-24T08:47:44.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:44.852+0000 7f3854c00640 1 -- 172.21.15.83:0/1711131519 shutdown_connections 2024-04-24T08:47:44.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:44.852+0000 7f3854c00640 1 --2- 172.21.15.83:0/1711131519 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3850105c30 0x7f385010c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:44.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:44.852+0000 7f3854c00640 1 --2- 172.21.15.83:0/1711131519 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3850105270 0x7f38501056f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:44.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:44.852+0000 7f3854c00640 1 --2- 172.21.15.83:0/1711131519 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3850104080 0x7f38501044a0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:44.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:44.852+0000 7f3854c00640 1 -- 172.21.15.83:0/1711131519 >> 172.21.15.83:0/1711131519 conn(0x7f38500ff610 msgr2=0x7f3850101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:47:44.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:44.852+0000 7f3854c00640 1 -- 172.21.15.83:0/1711131519 shutdown_connections 2024-04-24T08:47:44.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:44.852+0000 7f3854c00640 1 -- 172.21.15.83:0/1711131519 wait complete. 2024-04-24T08:47:44.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:44.853+0000 7f3854c00640 1 Processor -- start 2024-04-24T08:47:44.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:44.853+0000 7f3854c00640 1 -- start start 2024-04-24T08:47:44.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:44.853+0000 7f3854c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3850104080 0x7f385019a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:47:44.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:44.853+0000 7f3854c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3850105270 0x7f385019a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:47:44.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:44.853+0000 7f3854c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3850105c30 0x7f38501a1820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:47:44.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:44.853+0000 7f3854c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f385010fbd0 con 0x7f3850105270 2024-04-24T08:47:44.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:44.853+0000 7f3854c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f385010fa50 con 0x7f3850104080 2024-04-24T08:47:44.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:44.853+0000 7f3854c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f385010fd50 con 0x7f3850105c30 2024-04-24T08:47:44.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:44.853+0000 7f3847e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3850105270 0x7f385019a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:47:44.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:44.854+0000 7f3847e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3850105270 0x7f385019a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58920/0 (socket says 172.21.15.83:58920) 2024-04-24T08:47:44.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:44.854+0000 7f3847e00640 1 -- 172.21.15.83:0/3146612562 learned_addr learned my addr 172.21.15.83:0/3146612562 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:47:44.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:44.854+0000 7f384fe00640 1 --2- 172.21.15.83:0/3146612562 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3850105c30 0x7f38501a1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:47:44.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:44.854+0000 7f384f400640 1 --2- 172.21.15.83:0/3146612562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3850104080 0x7f385019a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:47:44.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:44.854+0000 7f3847e00640 1 -- 172.21.15.83:0/3146612562 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3850105c30 msgr2=0x7f38501a1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:47:44.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:44.854+0000 7f3847e00640 1 --2- 172.21.15.83:0/3146612562 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3850105c30 0x7f38501a1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:44.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:44.854+0000 7f3847e00640 1 -- 172.21.15.83:0/3146612562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3850104080 msgr2=0x7f385019a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:47:44.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:44.854+0000 7f3847e00640 1 --2- 172.21.15.83:0/3146612562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3850104080 0x7f385019a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:44.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:44.854+0000 7f3847e00640 1 -- 172.21.15.83:0/3146612562 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38501a1e90 con 0x7f3850105270 2024-04-24T08:47:44.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:44.854+0000 7f384fe00640 1 --2- 172.21.15.83:0/3146612562 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3850105c30 0x7f38501a1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:47:44.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:44.854+0000 7f3847e00640 1 --2- 172.21.15.83:0/3146612562 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3850105270 0x7f385019a7b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f383800c890 tx=0x7f383800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:47:44.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:44.854+0000 7f384cc00640 1 -- 172.21.15.83:0/3146612562 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3838004220 con 0x7f3850105270 2024-04-24T08:47:44.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:44.854+0000 7f384cc00640 1 -- 172.21.15.83:0/3146612562 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f38380043c0 con 0x7f3850105270 2024-04-24T08:47:44.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:44.855+0000 7f3854c00640 1 -- 172.21.15.83:0/3146612562 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38501a2180 con 0x7f3850105270 2024-04-24T08:47:44.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:44.855+0000 7f384cc00640 1 -- 172.21.15.83:0/3146612562 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3838004a00 con 0x7f3850105270 2024-04-24T08:47:44.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:44.855+0000 7f3854c00640 1 -- 172.21.15.83:0/3146612562 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38501a26d0 con 0x7f3850105270 2024-04-24T08:47:44.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:44.856+0000 7f3854c00640 1 -- 172.21.15.83:0/3146612562 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f385010b420 con 0x7f3850105270 2024-04-24T08:47:44.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:44.859+0000 7f384cc00640 1 -- 172.21.15.83:0/3146612562 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f3838004ba0 con 0x7f3850105270 2024-04-24T08:47:44.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:44.861+0000 7f384cc00640 1 --2- 172.21.15.83:0/3146612562 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f38340a3bb0 0x7f38340a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:47:44.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:44.862+0000 7f384cc00640 1 -- 172.21.15.83:0/3146612562 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f38380cfd40 con 0x7f3850105270 2024-04-24T08:47:44.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:44.862+0000 7f384f400640 1 --2- 172.21.15.83:0/3146612562 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f38340a3bb0 0x7f38340a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:47:44.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:44.863+0000 7f384f400640 1 --2- 172.21.15.83:0/3146612562 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f38340a3bb0 0x7f38340a6070 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3840006010 tx=0x7f38400314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:47:44.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:44.866+0000 7f384cc00640 1 -- 172.21.15.83:0/3146612562 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3838014030 con 0x7f3850105270 2024-04-24T08:47:45.053 INFO:journalctl@ceph.mds.j.smithi083.stdout:Apr 24 08:47:44 smithi083 podman[68311]: 2024-04-24 08:47:44.853709478 +0000 UTC m=+0.538445708 container init 6c5b0557d05b558e79ffd54d24031bd12589a30f2caccbc1d8d8e12c9a164e1b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-j, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.schema-version=1.0) 2024-04-24T08:47:45.054 INFO:journalctl@ceph.mds.j.smithi083.stdout:Apr 24 08:47:44 smithi083 podman[68311]: 2024-04-24 08:47:44.861569596 +0000 UTC m=+0.546305824 container start 6c5b0557d05b558e79ffd54d24031bd12589a30f2caccbc1d8d8e12c9a164e1b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-j, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , RELEASE=rishabh-main-apr17-a654945-1f81fb3, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-24T08:47:45.054 INFO:journalctl@ceph.mds.j.smithi083.stdout:Apr 24 08:47:44 smithi083 ceph-mds[68369]: -- 172.21.15.83:0/1132570783 <== mon.0 v2:172.21.15.83:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d6c7fb1a40 con 0x55d6c7f84800 2024-04-24T08:47:45.054 INFO:journalctl@ceph.mds.j.smithi083.stdout:Apr 24 08:47:44 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-j[68338]: starting mds.j at 2024-04-24T08:47:45.054 INFO:journalctl@ceph.mds.j.smithi083.stdout:Apr 24 08:47:44 smithi083 bash[68311]: 6c5b0557d05b558e79ffd54d24031bd12589a30f2caccbc1d8d8e12c9a164e1b 2024-04-24T08:47:45.054 INFO:journalctl@ceph.mds.j.smithi083.stdout:Apr 24 08:47:44 smithi083 systemd[1]: Started Ceph mds.j for bd3e039e-0215-11ef-bc93-c7b262605968. 2024-04-24T08:47:45.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:45.113+0000 7f3854c00640 1 -- 172.21.15.83:0/3146612562 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f385002d620 con 0x7f3850105270 2024-04-24T08:47:45.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:45.114+0000 7f384cc00640 1 -- 172.21.15.83:0/3146612562 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2224 (secure 0 0 0) 0x7f383809cb20 con 0x7f3850105270 2024-04-24T08:47:45.115 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:47:45.115 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/1859921310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1859921310},{"type":"v1","addr":"172.21.15.189:6833","nonce":1859921310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":34427,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/1157967658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1157967658},{"type":"v1","addr":"172.21.15.138:6833","nonce":1157967658}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-24T08:47:45.115 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 3 2024-04-24T08:47:45.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:45.117+0000 7f3854c00640 1 -- 172.21.15.83:0/3146612562 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f38340a3bb0 msgr2=0x7f38340a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:47:45.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:45.117+0000 7f3854c00640 1 --2- 172.21.15.83:0/3146612562 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f38340a3bb0 0x7f38340a6070 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3840006010 tx=0x7f38400314a0 comp rx=0 tx=0).stop 2024-04-24T08:47:45.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:45.117+0000 7f3854c00640 1 -- 172.21.15.83:0/3146612562 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3850105270 msgr2=0x7f385019a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:47:45.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:45.117+0000 7f3854c00640 1 --2- 172.21.15.83:0/3146612562 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3850105270 0x7f385019a7b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f383800c890 tx=0x7f383800cd50 comp rx=0 tx=0).stop 2024-04-24T08:47:45.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:45.117+0000 7f3854c00640 1 -- 172.21.15.83:0/3146612562 shutdown_connections 2024-04-24T08:47:45.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:45.117+0000 7f3854c00640 1 --2- 172.21.15.83:0/3146612562 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3850105c30 0x7f38501a1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:45.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:45.117+0000 7f3854c00640 1 --2- 172.21.15.83:0/3146612562 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f38340a3bb0 0x7f38340a6070 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:45.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:45.117+0000 7f3854c00640 1 --2- 172.21.15.83:0/3146612562 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3850105270 0x7f385019a7b0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:45.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:45.117+0000 7f3854c00640 1 --2- 172.21.15.83:0/3146612562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3850104080 0x7f385019a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:45.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:45.117+0000 7f3854c00640 1 -- 172.21.15.83:0/3146612562 >> 172.21.15.83:0/3146612562 conn(0x7f38500ff610 msgr2=0x7f38501085f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:47:45.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:45.117+0000 7f3854c00640 1 -- 172.21.15.83:0/3146612562 shutdown_connections 2024-04-24T08:47:45.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:45.117+0000 7f3854c00640 1 -- 172.21.15.83:0/3146612562 wait complete. 2024-04-24T08:47:46.121 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:45 smithi083 ceph-mon[27907]: pgmap v343: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:47:46.121 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:45 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:47:46.122 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:45 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:47:46.122 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:45 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/3146612562' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:47:46.122 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:45 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:47:46.122 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:45 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T08:47:46.122 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:45 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T08:47:46.122 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:45 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:47:46.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:45 smithi138 ceph-mon[31441]: pgmap v343: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:47:46.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:45 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:47:46.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:45 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:47:46.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:45 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/3146612562' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:47:46.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:45 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:47:46.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:45 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T08:47:46.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:45 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T08:47:46.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:45 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:47:46.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:45 smithi189 ceph-mon[31791]: pgmap v343: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:47:46.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:45 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:47:46.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:45 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:47:46.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:45 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/3146612562' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:47:46.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:45 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:47:46.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:45 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T08:47:46.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:45 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T08:47:46.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:45 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:47:47.139 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T08:47:47.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:46 smithi083 ceph-mon[27907]: Deploying daemon mds.d on smithi083 2024-04-24T08:47:47.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:46 smithi083 ceph-mon[27907]: mds.? [v2:172.21.15.83:6834/358872257,v1:172.21.15.83:6835/358872257] up:boot 2024-04-24T08:47:47.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:46 smithi083 ceph-mon[27907]: fsmap 3 up:standby 2024-04-24T08:47:47.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:46 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-24T08:47:47.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:46 smithi138 ceph-mon[31441]: Deploying daemon mds.d on smithi083 2024-04-24T08:47:47.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:46 smithi138 ceph-mon[31441]: mds.? [v2:172.21.15.83:6834/358872257,v1:172.21.15.83:6835/358872257] up:boot 2024-04-24T08:47:47.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:46 smithi138 ceph-mon[31441]: fsmap 3 up:standby 2024-04-24T08:47:47.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:46 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-24T08:47:47.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:46 smithi189 ceph-mon[31791]: Deploying daemon mds.d on smithi083 2024-04-24T08:47:47.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:46 smithi189 ceph-mon[31791]: mds.? [v2:172.21.15.83:6834/358872257,v1:172.21.15.83:6835/358872257] up:boot 2024-04-24T08:47:47.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:46 smithi189 ceph-mon[31791]: fsmap 3 up:standby 2024-04-24T08:47:47.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:46 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-24T08:47:47.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:47.349+0000 7ff1dee00640 1 -- 172.21.15.83:0/4243018276 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff1d810a030 msgr2=0x7ff1d81108c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:47:47.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:47.349+0000 7ff1dee00640 1 --2- 172.21.15.83:0/4243018276 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff1d810a030 0x7ff1d81108c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff1d0009900 tx=0x7ff1d002a3f0 comp rx=0 tx=0).stop 2024-04-24T08:47:47.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:47.349+0000 7ff1dee00640 1 -- 172.21.15.83:0/4243018276 shutdown_connections 2024-04-24T08:47:47.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:47.349+0000 7ff1dee00640 1 --2- 172.21.15.83:0/4243018276 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff1d810a030 0x7ff1d81108c0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:47.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:47.349+0000 7ff1dee00640 1 --2- 172.21.15.83:0/4243018276 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff1d8109670 0x7ff1d8109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:47.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:47.349+0000 7ff1dee00640 1 --2- 172.21.15.83:0/4243018276 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1d8108480 0x7ff1d81088a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:47.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:47.349+0000 7ff1dee00640 1 -- 172.21.15.83:0/4243018276 >> 172.21.15.83:0/4243018276 conn(0x7ff1d8077440 msgr2=0x7ff1d8079860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:47:47.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:47.349+0000 7ff1dee00640 1 -- 172.21.15.83:0/4243018276 shutdown_connections 2024-04-24T08:47:47.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:47.349+0000 7ff1dee00640 1 -- 172.21.15.83:0/4243018276 wait complete. 2024-04-24T08:47:47.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:47.350+0000 7ff1dee00640 1 Processor -- start 2024-04-24T08:47:47.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:47.350+0000 7ff1dee00640 1 -- start start 2024-04-24T08:47:47.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:47.350+0000 7ff1dee00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff1d8108480 0x7ff1d819e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:47:47.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:47.350+0000 7ff1dee00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1d8109670 0x7ff1d819eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:47:47.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:47.350+0000 7ff1dee00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff1d810a030 0x7ff1d81a5c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:47:47.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:47.350+0000 7ff1dee00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff1d8114010 con 0x7ff1d810a030 2024-04-24T08:47:47.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:47.350+0000 7ff1dee00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff1d8113e90 con 0x7ff1d8109670 2024-04-24T08:47:47.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:47.350+0000 7ff1dee00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff1d8114190 con 0x7ff1d8108480 2024-04-24T08:47:47.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:47.351+0000 7ff1de400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff1d810a030 0x7ff1d81a5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:47:47.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:47.351+0000 7ff1de400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff1d810a030 0x7ff1d81a5c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58964/0 (socket says 172.21.15.83:58964) 2024-04-24T08:47:47.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:47.351+0000 7ff1de400640 1 -- 172.21.15.83:0/2635299945 learned_addr learned my addr 172.21.15.83:0/2635299945 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:47:47.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:47.351+0000 7ff1dda00640 1 --2- 172.21.15.83:0/2635299945 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff1d8108480 0x7ff1d819e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:47:47.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:47.351+0000 7ff1dd000640 1 --2- 172.21.15.83:0/2635299945 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1d8109670 0x7ff1d819eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:47:47.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:47.351+0000 7ff1de400640 1 -- 172.21.15.83:0/2635299945 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff1d8108480 msgr2=0x7ff1d819e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:47:47.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:47.351+0000 7ff1de400640 1 --2- 172.21.15.83:0/2635299945 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff1d8108480 0x7ff1d819e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:47.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:47.351+0000 7ff1de400640 1 -- 172.21.15.83:0/2635299945 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1d8109670 msgr2=0x7ff1d819eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:47:47.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:47.351+0000 7ff1de400640 1 --2- 172.21.15.83:0/2635299945 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1d8109670 0x7ff1d819eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:47.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:47.351+0000 7ff1de400640 1 -- 172.21.15.83:0/2635299945 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1d81a6280 con 0x7ff1d810a030 2024-04-24T08:47:47.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:47.351+0000 7ff1de400640 1 --2- 172.21.15.83:0/2635299945 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff1d810a030 0x7ff1d81a5c10 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff1d0005ab0 tx=0x7ff1d0003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:47:47.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:47.352+0000 7ff1d6000640 1 -- 172.21.15.83:0/2635299945 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1d003f070 con 0x7ff1d810a030 2024-04-24T08:47:47.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:47.352+0000 7ff1d6000640 1 -- 172.21.15.83:0/2635299945 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff1d0003e40 con 0x7ff1d810a030 2024-04-24T08:47:47.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:47.352+0000 7ff1dee00640 1 -- 172.21.15.83:0/2635299945 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1d81a6510 con 0x7ff1d810a030 2024-04-24T08:47:47.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:47.352+0000 7ff1d6000640 1 -- 172.21.15.83:0/2635299945 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1d0035030 con 0x7ff1d810a030 2024-04-24T08:47:47.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:47.352+0000 7ff1dee00640 1 -- 172.21.15.83:0/2635299945 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1d81a6a00 con 0x7ff1d810a030 2024-04-24T08:47:47.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:47.353+0000 7ff1dee00640 1 -- 172.21.15.83:0/2635299945 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1d810f7a0 con 0x7ff1d810a030 2024-04-24T08:47:47.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:47.355+0000 7ff1d6000640 1 -- 172.21.15.83:0/2635299945 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ff1d0042070 con 0x7ff1d810a030 2024-04-24T08:47:47.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:47.356+0000 7ff1d6000640 1 --2- 172.21.15.83:0/2635299945 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7ff1c00a3bb0 0x7ff1c00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:47:47.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:47.357+0000 7ff1d6000640 1 -- 172.21.15.83:0/2635299945 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7ff1d00f0920 con 0x7ff1d810a030 2024-04-24T08:47:47.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:47.357+0000 7ff1dda00640 1 --2- 172.21.15.83:0/2635299945 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7ff1c00a3bb0 0x7ff1c00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:47:47.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:47.357+0000 7ff1dda00640 1 --2- 172.21.15.83:0/2635299945 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7ff1c00a3bb0 0x7ff1c00a6070 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff1d81094d0 tx=0x7ff1cc00a590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:47:47.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:47.361+0000 7ff1d6000640 1 -- 172.21.15.83:0/2635299945 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff1d00bd700 con 0x7ff1d810a030 2024-04-24T08:47:47.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:47.641+0000 7ff1dee00640 1 -- 172.21.15.83:0/2635299945 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff1d8062c00 con 0x7ff1d810a030 2024-04-24T08:47:47.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:47.642+0000 7ff1d6000640 1 -- 172.21.15.83:0/2635299945 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3046 (secure 0 0 0) 0x7ff1d003be20 con 0x7ff1d810a030 2024-04-24T08:47:47.643 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:47:47.643 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/358872257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":358872257},{"type":"v1","addr":"172.21.15.83:6835","nonce":358872257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/1859921310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1859921310},{"type":"v1","addr":"172.21.15.189:6833","nonce":1859921310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":34427,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/1157967658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1157967658},{"type":"v1","addr":"172.21.15.138:6833","nonce":1157967658}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-24T08:47:47.643 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 4 2024-04-24T08:47:47.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:47.645+0000 7ff1dee00640 1 -- 172.21.15.83:0/2635299945 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7ff1c00a3bb0 msgr2=0x7ff1c00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:47:47.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:47.645+0000 7ff1dee00640 1 --2- 172.21.15.83:0/2635299945 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7ff1c00a3bb0 0x7ff1c00a6070 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff1d81094d0 tx=0x7ff1cc00a590 comp rx=0 tx=0).stop 2024-04-24T08:47:47.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:47.645+0000 7ff1dee00640 1 -- 172.21.15.83:0/2635299945 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff1d810a030 msgr2=0x7ff1d81a5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:47:47.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:47.645+0000 7ff1dee00640 1 --2- 172.21.15.83:0/2635299945 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff1d810a030 0x7ff1d81a5c10 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff1d0005ab0 tx=0x7ff1d0003cd0 comp rx=0 tx=0).stop 2024-04-24T08:47:47.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:47.646+0000 7ff1dee00640 1 -- 172.21.15.83:0/2635299945 shutdown_connections 2024-04-24T08:47:47.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:47.646+0000 7ff1dee00640 1 --2- 172.21.15.83:0/2635299945 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7ff1c00a3bb0 0x7ff1c00a6070 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:47.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:47.646+0000 7ff1dee00640 1 --2- 172.21.15.83:0/2635299945 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff1d810a030 0x7ff1d81a5c10 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:47.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:47.646+0000 7ff1dee00640 1 --2- 172.21.15.83:0/2635299945 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1d8109670 0x7ff1d819eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:47.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:47.646+0000 7ff1dee00640 1 --2- 172.21.15.83:0/2635299945 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff1d8108480 0x7ff1d819e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:47.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:47.646+0000 7ff1dee00640 1 -- 172.21.15.83:0/2635299945 >> 172.21.15.83:0/2635299945 conn(0x7ff1d8077440 msgr2=0x7ff1d810c970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:47:47.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:47.646+0000 7ff1dee00640 1 -- 172.21.15.83:0/2635299945 shutdown_connections 2024-04-24T08:47:47.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:47.646+0000 7ff1dee00640 1 -- 172.21.15.83:0/2635299945 wait complete. 2024-04-24T08:47:48.176 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:47 smithi083 ceph-mon[27907]: pgmap v344: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:47:48.176 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:47 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/2635299945' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:47:48.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:47 smithi138 ceph-mon[31441]: pgmap v344: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:47:48.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:47 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/2635299945' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:47:48.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:47 smithi189 ceph-mon[31791]: pgmap v344: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:47:48.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:47 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/2635299945' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:47:48.530 INFO:journalctl@ceph.mds.d.smithi083.stdout:Apr 24 08:47:48 smithi083 systemd[1]: Starting Ceph mds.d for bd3e039e-0215-11ef-bc93-c7b262605968... 2024-04-24T08:47:48.804 INFO:journalctl@ceph.mds.d.smithi083.stdout:Apr 24 08:47:48 smithi083 podman[68700]: 2024-04-24 08:47:48.43384464 +0000 UTC m=+0.035784426 image pull b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644 2024-04-24T08:47:48.804 INFO:journalctl@ceph.mds.d.smithi083.stdout:Apr 24 08:47:48 smithi083 podman[68700]: 2024-04-24 08:47:48.575162732 +0000 UTC m=+0.177102555 container create 89397ba2650e92a5175eb4fd990e102146b7f8778ba7500da76ed049fe092f0c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-d, RELEASE=rishabh-main-apr17-a654945-1f81fb3, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0) 2024-04-24T08:47:49.304 INFO:journalctl@ceph.mds.d.smithi083.stdout:Apr 24 08:47:48 smithi083 podman[68700]: 2024-04-24 08:47:48.887177445 +0000 UTC m=+0.489117275 container init 89397ba2650e92a5175eb4fd990e102146b7f8778ba7500da76ed049fe092f0c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-d, RELEASE=rishabh-main-apr17-a654945-1f81fb3, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424) 2024-04-24T08:47:49.304 INFO:journalctl@ceph.mds.d.smithi083.stdout:Apr 24 08:47:48 smithi083 podman[68700]: 2024-04-24 08:47:48.897130086 +0000 UTC m=+0.499069919 container start 89397ba2650e92a5175eb4fd990e102146b7f8778ba7500da76ed049fe092f0c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-d, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.schema-version=1.0) 2024-04-24T08:47:49.304 INFO:journalctl@ceph.mds.d.smithi083.stdout:Apr 24 08:47:48 smithi083 ceph-mds[68720]: -- 172.21.15.83:0/2792756239 <== mon.0 v2:172.21.15.83:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5653fbb1da40 con 0x5653fbaf0800 2024-04-24T08:47:49.304 INFO:journalctl@ceph.mds.d.smithi083.stdout:Apr 24 08:47:48 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-d[68716]: starting mds.d at 2024-04-24T08:47:49.304 INFO:journalctl@ceph.mds.d.smithi083.stdout:Apr 24 08:47:48 smithi083 bash[68700]: 89397ba2650e92a5175eb4fd990e102146b7f8778ba7500da76ed049fe092f0c 2024-04-24T08:47:49.304 INFO:journalctl@ceph.mds.d.smithi083.stdout:Apr 24 08:47:49 smithi083 systemd[1]: Started Ceph mds.d for bd3e039e-0215-11ef-bc93-c7b262605968. 2024-04-24T08:47:49.666 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T08:47:49.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:49.887+0000 7f6e8fc00640 1 -- 172.21.15.83:0/397443891 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e88107d30 msgr2=0x7f6e8810a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:47:49.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:49.887+0000 7f6e8fc00640 1 --2- 172.21.15.83:0/397443891 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e88107d30 0x7f6e8810a180 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6e80009ca0 tx=0x7f6e8002a8c0 comp rx=0 tx=0).stop 2024-04-24T08:47:49.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:49.887+0000 7f6e8fc00640 1 -- 172.21.15.83:0/397443891 shutdown_connections 2024-04-24T08:47:49.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:49.887+0000 7f6e8fc00640 1 --2- 172.21.15.83:0/397443891 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e88107d30 0x7f6e8810a180 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:49.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:49.887+0000 7f6e8fc00640 1 --2- 172.21.15.83:0/397443891 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e88105400 0x7f6e881077f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:49.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:49.887+0000 7f6e8fc00640 1 --2- 172.21.15.83:0/397443891 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6e88102ad0 0x7f6e88104ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:49.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:49.887+0000 7f6e8fc00640 1 -- 172.21.15.83:0/397443891 >> 172.21.15.83:0/397443891 conn(0x7f6e880fc4e0 msgr2=0x7f6e880fe900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:47:49.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:49.887+0000 7f6e8fc00640 1 -- 172.21.15.83:0/397443891 shutdown_connections 2024-04-24T08:47:49.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:49.887+0000 7f6e8fc00640 1 -- 172.21.15.83:0/397443891 wait complete. 2024-04-24T08:47:49.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:49.888+0000 7f6e8fc00640 1 Processor -- start 2024-04-24T08:47:49.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:49.888+0000 7f6e8fc00640 1 -- start start 2024-04-24T08:47:49.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:49.889+0000 7f6e8fc00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e88102ad0 0x7f6e8819a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:47:49.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:49.889+0000 7f6e8fc00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6e88105400 0x7f6e8819a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:47:49.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:49.889+0000 7f6e8fc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e88107d30 0x7f6e881a1980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:47:49.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:49.889+0000 7f6e8fc00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6e8810dbe0 con 0x7f6e88107d30 2024-04-24T08:47:49.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:49.889+0000 7f6e8fc00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6e8810da60 con 0x7f6e88102ad0 2024-04-24T08:47:49.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:49.889+0000 7f6e8fc00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6e8810dd60 con 0x7f6e88105400 2024-04-24T08:47:49.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:49.889+0000 7f6e8dc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e88107d30 0x7f6e881a1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:47:49.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:49.889+0000 7f6e8dc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e88107d30 0x7f6e881a1980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45508/0 (socket says 172.21.15.83:45508) 2024-04-24T08:47:49.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:49.889+0000 7f6e8dc00640 1 -- 172.21.15.83:0/1626610298 learned_addr learned my addr 172.21.15.83:0/1626610298 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:47:49.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:49.889+0000 7f6e8d200640 1 --2- 172.21.15.83:0/1626610298 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e88102ad0 0x7f6e8819a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:47:49.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:49.889+0000 7f6e7fe00640 1 --2- 172.21.15.83:0/1626610298 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6e88105400 0x7f6e8819a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:47:49.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:49.889+0000 7f6e8dc00640 1 -- 172.21.15.83:0/1626610298 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6e88105400 msgr2=0x7f6e8819a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:47:49.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:49.889+0000 7f6e8dc00640 1 --2- 172.21.15.83:0/1626610298 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6e88105400 0x7f6e8819a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:49.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:49.889+0000 7f6e8dc00640 1 -- 172.21.15.83:0/1626610298 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e88102ad0 msgr2=0x7f6e8819a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:47:49.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:49.889+0000 7f6e8dc00640 1 --2- 172.21.15.83:0/1626610298 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e88102ad0 0x7f6e8819a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:49.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:49.889+0000 7f6e8dc00640 1 -- 172.21.15.83:0/1626610298 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e881a1ff0 con 0x7f6e88107d30 2024-04-24T08:47:49.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:49.889+0000 7f6e7fe00640 1 --2- 172.21.15.83:0/1626610298 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6e88105400 0x7f6e8819a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:47:49.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:49.890+0000 7f6e8dc00640 1 --2- 172.21.15.83:0/1626610298 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e88107d30 0x7f6e881a1980 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6e80031040 tx=0x7f6e80003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:47:49.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:49.890+0000 7f6e86000640 1 -- 172.21.15.83:0/1626610298 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e80032ec0 con 0x7f6e88107d30 2024-04-24T08:47:49.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:49.890+0000 7f6e86000640 1 -- 172.21.15.83:0/1626610298 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6e800044f0 con 0x7f6e88107d30 2024-04-24T08:47:49.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:49.890+0000 7f6e8fc00640 1 -- 172.21.15.83:0/1626610298 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e881a2280 con 0x7f6e88107d30 2024-04-24T08:47:49.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:49.890+0000 7f6e86000640 1 -- 172.21.15.83:0/1626610298 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e80030c30 con 0x7f6e88107d30 2024-04-24T08:47:49.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:49.890+0000 7f6e8fc00640 1 -- 172.21.15.83:0/1626610298 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e881a27d0 con 0x7f6e88107d30 2024-04-24T08:47:49.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:49.891+0000 7f6e8fc00640 1 -- 172.21.15.83:0/1626610298 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e88109820 con 0x7f6e88107d30 2024-04-24T08:47:49.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:49.893+0000 7f6e86000640 1 -- 172.21.15.83:0/1626610298 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f6e80004040 con 0x7f6e88107d30 2024-04-24T08:47:49.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:49.895+0000 7f6e86000640 1 --2- 172.21.15.83:0/1626610298 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f6e6c0a3d50 0x7f6e6c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:47:49.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:49.895+0000 7f6e86000640 1 -- 172.21.15.83:0/1626610298 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f6e800f1750 con 0x7f6e88107d30 2024-04-24T08:47:49.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:49.895+0000 7f6e8d200640 1 --2- 172.21.15.83:0/1626610298 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f6e6c0a3d50 0x7f6e6c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:47:49.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:49.896+0000 7f6e8d200640 1 --2- 172.21.15.83:0/1626610298 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f6e6c0a3d50 0x7f6e6c0a6210 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f6e7800abe0 tx=0x7f6e7800a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:47:49.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:49.898+0000 7f6e86000640 1 -- 172.21.15.83:0/1626610298 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6e800be5d0 con 0x7f6e88107d30 2024-04-24T08:47:50.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:49 smithi083 ceph-mon[27907]: pgmap v345: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:47:50.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:49 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:47:50.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:49 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:47:50.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:49 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:47:50.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:49 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T08:47:50.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:49 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T08:47:50.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:49 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:47:50.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:50.145+0000 7f6e8fc00640 1 -- 172.21.15.83:0/1626610298 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6e8802d620 con 0x7f6e88107d30 2024-04-24T08:47:50.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:50.146+0000 7f6e86000640 1 -- 172.21.15.83:0/1626610298 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3871 (secure 0 0 0) 0x7f6e800bded0 con 0x7f6e88107d30 2024-04-24T08:47:50.147 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:47:50.147 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/358872257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":358872257},{"type":"v1","addr":"172.21.15.83:6835","nonce":358872257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3157070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3157070851},{"type":"v1","addr":"172.21.15.83:6837","nonce":3157070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/1859921310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1859921310},{"type":"v1","addr":"172.21.15.189:6833","nonce":1859921310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":34427,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/1157967658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1157967658},{"type":"v1","addr":"172.21.15.138:6833","nonce":1157967658}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-24T08:47:50.147 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 5 2024-04-24T08:47:50.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:50.149+0000 7f6e8fc00640 1 -- 172.21.15.83:0/1626610298 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f6e6c0a3d50 msgr2=0x7f6e6c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:47:50.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:50.149+0000 7f6e8fc00640 1 --2- 172.21.15.83:0/1626610298 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f6e6c0a3d50 0x7f6e6c0a6210 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f6e7800abe0 tx=0x7f6e7800a2d0 comp rx=0 tx=0).stop 2024-04-24T08:47:50.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:50.149+0000 7f6e8fc00640 1 -- 172.21.15.83:0/1626610298 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e88107d30 msgr2=0x7f6e881a1980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:47:50.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:50.149+0000 7f6e8fc00640 1 --2- 172.21.15.83:0/1626610298 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e88107d30 0x7f6e881a1980 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6e80031040 tx=0x7f6e80003ba0 comp rx=0 tx=0).stop 2024-04-24T08:47:50.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:50.149+0000 7f6e8fc00640 1 -- 172.21.15.83:0/1626610298 shutdown_connections 2024-04-24T08:47:50.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:50.149+0000 7f6e8fc00640 1 --2- 172.21.15.83:0/1626610298 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f6e6c0a3d50 0x7f6e6c0a6210 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:50.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:50.149+0000 7f6e8fc00640 1 --2- 172.21.15.83:0/1626610298 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e88107d30 0x7f6e881a1980 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:50.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:50.149+0000 7f6e8fc00640 1 --2- 172.21.15.83:0/1626610298 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6e88105400 0x7f6e8819a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:50.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:50.149+0000 7f6e8fc00640 1 --2- 172.21.15.83:0/1626610298 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e88102ad0 0x7f6e8819a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:50.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:50.149+0000 7f6e8fc00640 1 -- 172.21.15.83:0/1626610298 >> 172.21.15.83:0/1626610298 conn(0x7f6e880fc4e0 msgr2=0x7f6e881020e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:47:50.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:50.149+0000 7f6e8fc00640 1 -- 172.21.15.83:0/1626610298 shutdown_connections 2024-04-24T08:47:50.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:50.149+0000 7f6e8fc00640 1 -- 172.21.15.83:0/1626610298 wait complete. 2024-04-24T08:47:50.164 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:49 smithi138 ceph-mon[31441]: pgmap v345: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:47:50.164 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:49 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:47:50.164 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:49 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:47:50.164 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:49 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:47:50.164 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:49 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T08:47:50.164 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:49 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T08:47:50.165 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:49 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:47:50.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:49 smithi189 ceph-mon[31791]: pgmap v345: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:47:50.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:49 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:47:50.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:49 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:47:50.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:49 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:47:50.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:49 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T08:47:50.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:49 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T08:47:50.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:49 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:47:51.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:50 smithi138 ceph-mon[31441]: Deploying daemon mds.e on smithi138 2024-04-24T08:47:51.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:50 smithi138 ceph-mon[31441]: mds.? [v2:172.21.15.83:6836/3157070851,v1:172.21.15.83:6837/3157070851] up:boot 2024-04-24T08:47:51.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:50 smithi138 ceph-mon[31441]: fsmap 4 up:standby 2024-04-24T08:47:51.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:50 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-24T08:47:51.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:50 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/1626610298' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:47:51.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:50 smithi189 ceph-mon[31791]: Deploying daemon mds.e on smithi138 2024-04-24T08:47:51.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:50 smithi189 ceph-mon[31791]: mds.? [v2:172.21.15.83:6836/3157070851,v1:172.21.15.83:6837/3157070851] up:boot 2024-04-24T08:47:51.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:50 smithi189 ceph-mon[31791]: fsmap 4 up:standby 2024-04-24T08:47:51.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:50 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-24T08:47:51.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:50 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/1626610298' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:47:51.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:50 smithi083 ceph-mon[27907]: Deploying daemon mds.e on smithi138 2024-04-24T08:47:51.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:50 smithi083 ceph-mon[27907]: mds.? [v2:172.21.15.83:6836/3157070851,v1:172.21.15.83:6837/3157070851] up:boot 2024-04-24T08:47:51.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:50 smithi083 ceph-mon[27907]: fsmap 4 up:standby 2024-04-24T08:47:51.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:50 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-24T08:47:51.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:50 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/1626610298' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:47:52.170 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T08:47:52.173 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:51 smithi138 ceph-mon[31441]: pgmap v346: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:47:52.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:51 smithi189 ceph-mon[31791]: pgmap v346: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:47:52.245 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:51 smithi083 ceph-mon[27907]: pgmap v346: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:47:52.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:52.380+0000 7f69a5c00640 1 -- 172.21.15.83:0/1570456941 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69a0105e30 msgr2=0x7f69a010c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:47:52.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:52.380+0000 7f69a5c00640 1 --2- 172.21.15.83:0/1570456941 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69a0105e30 0x7f69a010c6c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6998009ca0 tx=0x7f699802a8c0 comp rx=0 tx=0).stop 2024-04-24T08:47:52.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:52.380+0000 7f69a5c00640 1 -- 172.21.15.83:0/1570456941 shutdown_connections 2024-04-24T08:47:52.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:52.380+0000 7f69a5c00640 1 --2- 172.21.15.83:0/1570456941 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69a0105e30 0x7f69a010c6c0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:52.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:52.380+0000 7f69a5c00640 1 --2- 172.21.15.83:0/1570456941 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69a0105470 0x7f69a01058f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:52.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:52.380+0000 7f69a5c00640 1 --2- 172.21.15.83:0/1570456941 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f69a0104280 0x7f69a01046a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:52.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:52.380+0000 7f69a5c00640 1 -- 172.21.15.83:0/1570456941 >> 172.21.15.83:0/1570456941 conn(0x7f69a00ff7f0 msgr2=0x7f69a0101c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:47:52.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:52.380+0000 7f69a5c00640 1 -- 172.21.15.83:0/1570456941 shutdown_connections 2024-04-24T08:47:52.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:52.380+0000 7f69a5c00640 1 -- 172.21.15.83:0/1570456941 wait complete. 2024-04-24T08:47:52.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:52.381+0000 7f69a5c00640 1 Processor -- start 2024-04-24T08:47:52.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:52.381+0000 7f69a5c00640 1 -- start start 2024-04-24T08:47:52.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:52.381+0000 7f69a5c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f69a0104280 0x7f69a019a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:47:52.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:52.381+0000 7f69a5c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69a0105470 0x7f69a019a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:47:52.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:52.381+0000 7f69a5c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69a0105e30 0x7f69a01a19c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:47:52.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:52.381+0000 7f69a5c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f69a010fe10 con 0x7f69a0105470 2024-04-24T08:47:52.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:52.381+0000 7f69a5c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f69a010fc90 con 0x7f69a0105e30 2024-04-24T08:47:52.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:52.381+0000 7f69a5c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f69a010ff90 con 0x7f69a0104280 2024-04-24T08:47:52.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:52.382+0000 7f6996000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69a0105470 0x7f69a019a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:47:52.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:52.382+0000 7f6996000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69a0105470 0x7f69a019a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45530/0 (socket says 172.21.15.83:45530) 2024-04-24T08:47:52.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:52.382+0000 7f6996000640 1 -- 172.21.15.83:0/2977530414 learned_addr learned my addr 172.21.15.83:0/2977530414 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:47:52.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:52.382+0000 7f699ea00640 1 --2- 172.21.15.83:0/2977530414 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f69a0104280 0x7f69a019a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:47:52.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:52.382+0000 7f6996000640 1 -- 172.21.15.83:0/2977530414 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f69a0104280 msgr2=0x7f69a019a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:47:52.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:52.382+0000 7f6996000640 1 --2- 172.21.15.83:0/2977530414 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f69a0104280 0x7f69a019a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:52.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:52.382+0000 7f699f400640 1 --2- 172.21.15.83:0/2977530414 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69a0105e30 0x7f69a01a19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:47:52.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:52.382+0000 7f6996000640 1 -- 172.21.15.83:0/2977530414 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69a0105e30 msgr2=0x7f69a01a19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:47:52.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:52.382+0000 7f6996000640 1 --2- 172.21.15.83:0/2977530414 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69a0105e30 0x7f69a01a19c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:52.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:52.382+0000 7f6996000640 1 -- 172.21.15.83:0/2977530414 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69a01a2030 con 0x7f69a0105470 2024-04-24T08:47:52.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:52.382+0000 7f699f400640 1 --2- 172.21.15.83:0/2977530414 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69a0105e30 0x7f69a01a19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:47:52.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:52.383+0000 7f6996000640 1 --2- 172.21.15.83:0/2977530414 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69a0105470 0x7f69a019a950 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f698800d950 tx=0x7f698800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:47:52.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:52.383+0000 7f6997e00640 1 -- 172.21.15.83:0/2977530414 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6988017030 con 0x7f69a0105470 2024-04-24T08:47:52.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:52.383+0000 7f69a5c00640 1 -- 172.21.15.83:0/2977530414 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69a01a2320 con 0x7f69a0105470 2024-04-24T08:47:52.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:52.383+0000 7f6997e00640 1 -- 172.21.15.83:0/2977530414 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6988004020 con 0x7f69a0105470 2024-04-24T08:47:52.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:52.383+0000 7f6997e00640 1 -- 172.21.15.83:0/2977530414 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69880047a0 con 0x7f69a0105470 2024-04-24T08:47:52.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:52.383+0000 7f69a5c00640 1 -- 172.21.15.83:0/2977530414 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69a01a28c0 con 0x7f69a0105470 2024-04-24T08:47:52.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:52.384+0000 7f69a5c00640 1 -- 172.21.15.83:0/2977530414 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69a010b5a0 con 0x7f69a0105470 2024-04-24T08:47:52.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:52.386+0000 7f6997e00640 1 -- 172.21.15.83:0/2977530414 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f69880041d0 con 0x7f69a0105470 2024-04-24T08:47:52.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:52.387+0000 7f6997e00640 1 --2- 172.21.15.83:0/2977530414 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f69840a3c30 0x7f69840a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:47:52.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:52.388+0000 7f6997e00640 1 -- 172.21.15.83:0/2977530414 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f69880cfa90 con 0x7f69a0105470 2024-04-24T08:47:52.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:52.388+0000 7f699ea00640 1 --2- 172.21.15.83:0/2977530414 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f69840a3c30 0x7f69840a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:47:52.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:52.388+0000 7f699ea00640 1 --2- 172.21.15.83:0/2977530414 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f69840a3c30 0x7f69840a60f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6990005fd0 tx=0x7f6990005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:47:52.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:52.392+0000 7f6997e00640 1 -- 172.21.15.83:0/2977530414 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f698809c870 con 0x7f69a0105470 2024-04-24T08:47:52.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:52.640+0000 7f69a5c00640 1 -- 172.21.15.83:0/2977530414 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f69a002d620 con 0x7f69a0105470 2024-04-24T08:47:52.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:52.645+0000 7f6997e00640 1 -- 172.21.15.83:0/2977530414 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3871 (secure 0 0 0) 0x7f698809c170 con 0x7f69a0105470 2024-04-24T08:47:52.645 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:47:52.646 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/358872257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":358872257},{"type":"v1","addr":"172.21.15.83:6835","nonce":358872257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3157070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3157070851},{"type":"v1","addr":"172.21.15.83:6837","nonce":3157070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/1859921310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1859921310},{"type":"v1","addr":"172.21.15.189:6833","nonce":1859921310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":34427,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/1157967658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1157967658},{"type":"v1","addr":"172.21.15.138:6833","nonce":1157967658}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-24T08:47:52.646 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 5 2024-04-24T08:47:52.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:52.648+0000 7f69a5c00640 1 -- 172.21.15.83:0/2977530414 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f69840a3c30 msgr2=0x7f69840a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:47:52.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:52.648+0000 7f69a5c00640 1 --2- 172.21.15.83:0/2977530414 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f69840a3c30 0x7f69840a60f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6990005fd0 tx=0x7f6990005850 comp rx=0 tx=0).stop 2024-04-24T08:47:52.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:52.648+0000 7f69a5c00640 1 -- 172.21.15.83:0/2977530414 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69a0105470 msgr2=0x7f69a019a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:47:52.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:52.648+0000 7f69a5c00640 1 --2- 172.21.15.83:0/2977530414 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69a0105470 0x7f69a019a950 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f698800d950 tx=0x7f698800de10 comp rx=0 tx=0).stop 2024-04-24T08:47:52.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:52.648+0000 7f69a5c00640 1 -- 172.21.15.83:0/2977530414 shutdown_connections 2024-04-24T08:47:52.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:52.648+0000 7f69a5c00640 1 --2- 172.21.15.83:0/2977530414 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69a0105e30 0x7f69a01a19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:52.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:52.648+0000 7f69a5c00640 1 --2- 172.21.15.83:0/2977530414 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f69840a3c30 0x7f69840a60f0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:52.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:52.648+0000 7f69a5c00640 1 --2- 172.21.15.83:0/2977530414 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69a0105470 0x7f69a019a950 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:52.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:52.648+0000 7f69a5c00640 1 --2- 172.21.15.83:0/2977530414 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f69a0104280 0x7f69a019a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:52.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:52.648+0000 7f69a5c00640 1 -- 172.21.15.83:0/2977530414 >> 172.21.15.83:0/2977530414 conn(0x7f69a00ff7f0 msgr2=0x7f69a0108770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:47:52.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:52.648+0000 7f69a5c00640 1 -- 172.21.15.83:0/2977530414 shutdown_connections 2024-04-24T08:47:52.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:52.648+0000 7f69a5c00640 1 -- 172.21.15.83:0/2977530414 wait complete. 2024-04-24T08:47:52.687 INFO:journalctl@ceph.mds.e.smithi138.stdout:Apr 24 08:47:52 smithi138 systemd[1]: Starting Ceph mds.e for bd3e039e-0215-11ef-bc93-c7b262605968... 2024-04-24T08:47:52.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:52 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/2977530414' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:47:52.956 INFO:journalctl@ceph.mds.e.smithi138.stdout:Apr 24 08:47:52 smithi138 podman[60882]: 2024-04-24 08:47:52.591627538 +0000 UTC m=+0.032121512 image pull b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644 2024-04-24T08:47:52.956 INFO:journalctl@ceph.mds.e.smithi138.stdout:Apr 24 08:47:52 smithi138 podman[60882]: 2024-04-24 08:47:52.703279134 +0000 UTC m=+0.143773137 container create f58a43afe68a010e282498166c3bf67976ad8fd7fbf4715e3df58c277d36d285 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=rishabh-main-apr17-a654945-1f81fb3, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-04-24T08:47:53.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:52 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/2977530414' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:47:53.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:52 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/2977530414' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:47:53.455 INFO:journalctl@ceph.mds.e.smithi138.stdout:Apr 24 08:47:52 smithi138 podman[60882]: 2024-04-24 08:47:52.991494085 +0000 UTC m=+0.431988092 container init f58a43afe68a010e282498166c3bf67976ad8fd7fbf4715e3df58c277d36d285 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-e, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True) 2024-04-24T08:47:53.455 INFO:journalctl@ceph.mds.e.smithi138.stdout:Apr 24 08:47:53 smithi138 podman[60882]: 2024-04-24 08:47:53.001743405 +0000 UTC m=+0.442237412 container start f58a43afe68a010e282498166c3bf67976ad8fd7fbf4715e3df58c277d36d285 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-e, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-24T08:47:53.456 INFO:journalctl@ceph.mds.e.smithi138.stdout:Apr 24 08:47:53 smithi138 ceph-mds[60900]: -- 172.21.15.138:0/1634176970 <== mon.2 v2:172.21.15.138:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56222a3a9a40 con 0x56222a37c800 2024-04-24T08:47:53.456 INFO:journalctl@ceph.mds.e.smithi138.stdout:Apr 24 08:47:53 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-e[60896]: starting mds.e at 2024-04-24T08:47:53.456 INFO:journalctl@ceph.mds.e.smithi138.stdout:Apr 24 08:47:53 smithi138 bash[60882]: f58a43afe68a010e282498166c3bf67976ad8fd7fbf4715e3df58c277d36d285 2024-04-24T08:47:53.456 INFO:journalctl@ceph.mds.e.smithi138.stdout:Apr 24 08:47:53 smithi138 systemd[1]: Started Ceph mds.e for bd3e039e-0215-11ef-bc93-c7b262605968. 2024-04-24T08:47:54.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:53 smithi138 ceph-mon[31441]: pgmap v347: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:47:54.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:53 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:47:54.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:53 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:47:54.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:53 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:47:54.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:53 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T08:47:54.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:53 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T08:47:54.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:53 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:47:54.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:53 smithi189 ceph-mon[31791]: pgmap v347: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:47:54.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:53 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:47:54.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:53 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:47:54.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:53 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:47:54.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:53 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T08:47:54.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:53 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T08:47:54.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:53 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:47:54.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:53 smithi083 ceph-mon[27907]: pgmap v347: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:47:54.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:53 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:47:54.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:53 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:47:54.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:53 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:47:54.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:53 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T08:47:54.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:53 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T08:47:54.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:53 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:47:54.667 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T08:47:54.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:54.878+0000 7f6440a00640 1 -- 172.21.15.83:0/3703841872 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f643c108480 msgr2=0x7f643c1088a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:47:54.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:54.878+0000 7f6440a00640 1 --2- 172.21.15.83:0/3703841872 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f643c108480 0x7f643c1088a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f642c009900 tx=0x7f642c02a400 comp rx=0 tx=0).stop 2024-04-24T08:47:54.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:54.878+0000 7f6440a00640 1 -- 172.21.15.83:0/3703841872 shutdown_connections 2024-04-24T08:47:54.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:54.878+0000 7f6440a00640 1 --2- 172.21.15.83:0/3703841872 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f643c10a030 0x7f643c1108c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:54.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:54.878+0000 7f6440a00640 1 --2- 172.21.15.83:0/3703841872 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f643c109670 0x7f643c109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:54.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:54.878+0000 7f6440a00640 1 --2- 172.21.15.83:0/3703841872 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f643c108480 0x7f643c1088a0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:54.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:54.878+0000 7f6440a00640 1 -- 172.21.15.83:0/3703841872 >> 172.21.15.83:0/3703841872 conn(0x7f643c077440 msgr2=0x7f643c079860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:47:54.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:54.878+0000 7f6440a00640 1 -- 172.21.15.83:0/3703841872 shutdown_connections 2024-04-24T08:47:54.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:54.878+0000 7f6440a00640 1 -- 172.21.15.83:0/3703841872 wait complete. 2024-04-24T08:47:54.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:54.879+0000 7f6440a00640 1 Processor -- start 2024-04-24T08:47:54.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:54.880+0000 7f6440a00640 1 -- start start 2024-04-24T08:47:54.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:54.880+0000 7f6440a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f643c108480 0x7f643c19e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:47:54.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:54.880+0000 7f6440a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f643c109670 0x7f643c19eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:47:54.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:54.880+0000 7f6440a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f643c10a030 0x7f643c1a5c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:47:54.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:54.880+0000 7f6440a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f643c114010 con 0x7f643c109670 2024-04-24T08:47:54.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:54.880+0000 7f6440a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f643c113e90 con 0x7f643c108480 2024-04-24T08:47:54.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:54.880+0000 7f6440a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f643c114190 con 0x7f643c10a030 2024-04-24T08:47:54.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:54.880+0000 7f6433e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f643c109670 0x7f643c19eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:47:54.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:54.880+0000 7f6433e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f643c109670 0x7f643c19eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45542/0 (socket says 172.21.15.83:45542) 2024-04-24T08:47:54.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:54.880+0000 7f6433e00640 1 -- 172.21.15.83:0/1638052776 learned_addr learned my addr 172.21.15.83:0/1638052776 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:47:54.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:54.880+0000 7f643b400640 1 --2- 172.21.15.83:0/1638052776 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f643c108480 0x7f643c19e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:47:54.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:54.881+0000 7f643be00640 1 --2- 172.21.15.83:0/1638052776 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f643c10a030 0x7f643c1a5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:47:54.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:54.881+0000 7f6433e00640 1 -- 172.21.15.83:0/1638052776 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f643c10a030 msgr2=0x7f643c1a5c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:47:54.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:54.881+0000 7f6433e00640 1 --2- 172.21.15.83:0/1638052776 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f643c10a030 0x7f643c1a5c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:54.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:54.881+0000 7f6433e00640 1 -- 172.21.15.83:0/1638052776 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f643c108480 msgr2=0x7f643c19e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:47:54.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:54.881+0000 7f6433e00640 1 --2- 172.21.15.83:0/1638052776 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f643c108480 0x7f643c19e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:54.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:54.881+0000 7f6433e00640 1 -- 172.21.15.83:0/1638052776 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f643c1a6270 con 0x7f643c109670 2024-04-24T08:47:54.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:54.881+0000 7f643be00640 1 --2- 172.21.15.83:0/1638052776 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f643c10a030 0x7f643c1a5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:47:54.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:54.881+0000 7f6433e00640 1 --2- 172.21.15.83:0/1638052776 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f643c109670 0x7f643c19eb90 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f642400d950 tx=0x7f642400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:47:54.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:54.881+0000 7f6438c00640 1 -- 172.21.15.83:0/1638052776 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6424017030 con 0x7f643c109670 2024-04-24T08:47:54.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:54.881+0000 7f6438c00640 1 -- 172.21.15.83:0/1638052776 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6424004020 con 0x7f643c109670 2024-04-24T08:47:54.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:54.881+0000 7f6438c00640 1 -- 172.21.15.83:0/1638052776 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64240047a0 con 0x7f643c109670 2024-04-24T08:47:54.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:54.881+0000 7f6440a00640 1 -- 172.21.15.83:0/1638052776 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f643c1a6560 con 0x7f643c109670 2024-04-24T08:47:54.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:54.881+0000 7f6440a00640 1 -- 172.21.15.83:0/1638052776 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f643c1a6ab0 con 0x7f643c109670 2024-04-24T08:47:54.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:54.882+0000 7f6440a00640 1 -- 172.21.15.83:0/1638052776 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f643c10f870 con 0x7f643c109670 2024-04-24T08:47:54.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:54.885+0000 7f6438c00640 1 -- 172.21.15.83:0/1638052776 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f64240041d0 con 0x7f643c109670 2024-04-24T08:47:54.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:54.886+0000 7f6438c00640 1 --2- 172.21.15.83:0/1638052776 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f64200a3bb0 0x7f64200a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:47:54.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:54.886+0000 7f6438c00640 1 -- 172.21.15.83:0/1638052776 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f64240d0240 con 0x7f643c109670 2024-04-24T08:47:54.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:54.886+0000 7f643b400640 1 --2- 172.21.15.83:0/1638052776 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f64200a3bb0 0x7f64200a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:47:54.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:54.887+0000 7f643b400640 1 --2- 172.21.15.83:0/1638052776 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f64200a3bb0 0x7f64200a6070 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f642c006010 tx=0x7f642c0314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:47:54.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:54.889+0000 7f6438c00640 1 -- 172.21.15.83:0/1638052776 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f642409d020 con 0x7f643c109670 2024-04-24T08:47:55.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:54 smithi083 ceph-mon[27907]: Deploying daemon mds.f on smithi189 2024-04-24T08:47:55.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:54 smithi083 ceph-mon[27907]: mds.? [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] up:boot 2024-04-24T08:47:55.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:54 smithi083 ceph-mon[27907]: fsmap 5 up:standby 2024-04-24T08:47:55.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:54 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-24T08:47:55.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:55.137+0000 7f6440a00640 1 -- 172.21.15.83:0/1638052776 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f643c02d870 con 0x7f643c109670 2024-04-24T08:47:55.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:55.138+0000 7f6438c00640 1 -- 172.21.15.83:0/1638052776 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4699 (secure 0 0 0) 0x7f642409c920 con 0x7f643c109670 2024-04-24T08:47:55.138 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:47:55.138 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/358872257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":358872257},{"type":"v1","addr":"172.21.15.83:6835","nonce":358872257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3157070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3157070851},{"type":"v1","addr":"172.21.15.83:6837","nonce":3157070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/1859921310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1859921310},{"type":"v1","addr":"172.21.15.189:6833","nonce":1859921310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":34427,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/1157967658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1157967658},{"type":"v1","addr":"172.21.15.138:6833","nonce":1157967658}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34451,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1909219758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1909219758},{"type":"v1","addr":"172.21.15.138:6835","nonce":1909219758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-24T08:47:55.139 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 6 2024-04-24T08:47:55.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:55.141+0000 7f6440a00640 1 -- 172.21.15.83:0/1638052776 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f64200a3bb0 msgr2=0x7f64200a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:47:55.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:55.141+0000 7f6440a00640 1 --2- 172.21.15.83:0/1638052776 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f64200a3bb0 0x7f64200a6070 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f642c006010 tx=0x7f642c0314a0 comp rx=0 tx=0).stop 2024-04-24T08:47:55.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:55.141+0000 7f6440a00640 1 -- 172.21.15.83:0/1638052776 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f643c109670 msgr2=0x7f643c19eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:47:55.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:55.141+0000 7f6440a00640 1 --2- 172.21.15.83:0/1638052776 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f643c109670 0x7f643c19eb90 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f642400d950 tx=0x7f642400de10 comp rx=0 tx=0).stop 2024-04-24T08:47:55.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:55.141+0000 7f6440a00640 1 -- 172.21.15.83:0/1638052776 shutdown_connections 2024-04-24T08:47:55.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:55.141+0000 7f6440a00640 1 --2- 172.21.15.83:0/1638052776 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f643c10a030 0x7f643c1a5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:55.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:55.141+0000 7f6440a00640 1 --2- 172.21.15.83:0/1638052776 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f64200a3bb0 0x7f64200a6070 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:55.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:55.141+0000 7f6440a00640 1 --2- 172.21.15.83:0/1638052776 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f643c109670 0x7f643c19eb90 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:55.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:55.141+0000 7f6440a00640 1 --2- 172.21.15.83:0/1638052776 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f643c108480 0x7f643c19e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:55.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:55.141+0000 7f6440a00640 1 -- 172.21.15.83:0/1638052776 >> 172.21.15.83:0/1638052776 conn(0x7f643c077440 msgr2=0x7f643c10ca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:47:55.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:55.141+0000 7f6440a00640 1 -- 172.21.15.83:0/1638052776 shutdown_connections 2024-04-24T08:47:55.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:55.141+0000 7f6440a00640 1 -- 172.21.15.83:0/1638052776 wait complete. 2024-04-24T08:47:55.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:54 smithi138 ceph-mon[31441]: Deploying daemon mds.f on smithi189 2024-04-24T08:47:55.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:54 smithi138 ceph-mon[31441]: mds.? [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] up:boot 2024-04-24T08:47:55.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:54 smithi138 ceph-mon[31441]: fsmap 5 up:standby 2024-04-24T08:47:55.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:54 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-24T08:47:55.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:54 smithi189 ceph-mon[31791]: Deploying daemon mds.f on smithi189 2024-04-24T08:47:55.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:54 smithi189 ceph-mon[31791]: mds.? [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] up:boot 2024-04-24T08:47:55.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:54 smithi189 ceph-mon[31791]: fsmap 5 up:standby 2024-04-24T08:47:55.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:54 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-24T08:47:56.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:55 smithi138 ceph-mon[31441]: pgmap v348: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:47:56.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:55 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/1638052776' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:47:56.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:55 smithi189 ceph-mon[31791]: pgmap v348: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:47:56.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:55 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/1638052776' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:47:56.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:55 smithi083 ceph-mon[27907]: pgmap v348: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:47:56.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:55 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/1638052776' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:47:56.639 INFO:journalctl@ceph.mds.f.smithi189.stdout:Apr 24 08:47:56 smithi189 systemd[1]: Starting Ceph mds.f for bd3e039e-0215-11ef-bc93-c7b262605968... 2024-04-24T08:47:56.893 INFO:journalctl@ceph.mds.f.smithi189.stdout:Apr 24 08:47:56 smithi189 podman[61391]: 2024-04-24 08:47:56.572786236 +0000 UTC m=+0.035112667 image pull b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644 2024-04-24T08:47:56.893 INFO:journalctl@ceph.mds.f.smithi189.stdout:Apr 24 08:47:56 smithi189 podman[61391]: 2024-04-24 08:47:56.714523823 +0000 UTC m=+0.176850317 container create 2bceb0b43cf6ee27936a72e68367666c978f2be27e8e484ca20f5fc2fdbd84c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-f, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=rishabh-main-apr17-a654945-1f81fb3, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424) 2024-04-24T08:47:57.163 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T08:47:57.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:57.378+0000 7f3d8fe00640 1 -- 172.21.15.83:0/3198688600 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d88109850 msgr2=0x7f3d88109cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:47:57.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:57.378+0000 7f3d8fe00640 1 --2- 172.21.15.83:0/3198688600 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d88109850 0x7f3d88109cd0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f3d74009860 tx=0x7f3d7402a340 comp rx=0 tx=0).stop 2024-04-24T08:47:57.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:57.378+0000 7f3d8fe00640 1 -- 172.21.15.83:0/3198688600 shutdown_connections 2024-04-24T08:47:57.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:57.378+0000 7f3d8fe00640 1 --2- 172.21.15.83:0/3198688600 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d8810a210 0x7f3d88110aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:57.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:57.378+0000 7f3d8fe00640 1 --2- 172.21.15.83:0/3198688600 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d88109850 0x7f3d88109cd0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:57.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:57.378+0000 7f3d8fe00640 1 --2- 172.21.15.83:0/3198688600 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3d88108660 0x7f3d88108a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:57.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:57.378+0000 7f3d8fe00640 1 -- 172.21.15.83:0/3198688600 >> 172.21.15.83:0/3198688600 conn(0x7f3d88077620 msgr2=0x7f3d88079a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:47:57.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:57.378+0000 7f3d8fe00640 1 -- 172.21.15.83:0/3198688600 shutdown_connections 2024-04-24T08:47:57.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:57.378+0000 7f3d8fe00640 1 -- 172.21.15.83:0/3198688600 wait complete. 2024-04-24T08:47:57.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:57.379+0000 7f3d8fe00640 1 Processor -- start 2024-04-24T08:47:57.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:57.379+0000 7f3d8fe00640 1 -- start start 2024-04-24T08:47:57.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:57.380+0000 7f3d8fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d88108660 0x7f3d8819e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:47:57.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:57.380+0000 7f3d8fe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3d88109850 0x7f3d8819ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:47:57.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:57.380+0000 7f3d8fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d8810a210 0x7f3d881a5e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:47:57.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:57.380+0000 7f3d8fe00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3d88114180 con 0x7f3d8810a210 2024-04-24T08:47:57.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:57.380+0000 7f3d8fe00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3d88114000 con 0x7f3d88108660 2024-04-24T08:47:57.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:57.380+0000 7f3d8fe00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3d88114300 con 0x7f3d88109850 2024-04-24T08:47:57.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:57.380+0000 7f3d8ca00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3d88109850 0x7f3d8819ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:47:57.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:57.380+0000 7f3d8de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d8810a210 0x7f3d881a5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:47:57.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:57.380+0000 7f3d8ca00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3d88109850 0x7f3d8819ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:37608/0 (socket says 172.21.15.83:37608) 2024-04-24T08:47:57.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:57.380+0000 7f3d8ca00640 1 -- 172.21.15.83:0/3492034448 learned_addr learned my addr 172.21.15.83:0/3492034448 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:47:57.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:57.380+0000 7f3d8d400640 1 --2- 172.21.15.83:0/3492034448 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d88108660 0x7f3d8819e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:47:57.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:57.380+0000 7f3d8ca00640 1 -- 172.21.15.83:0/3492034448 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d88108660 msgr2=0x7f3d8819e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:47:57.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:57.380+0000 7f3d8ca00640 1 --2- 172.21.15.83:0/3492034448 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d88108660 0x7f3d8819e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:57.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:57.380+0000 7f3d8ca00640 1 -- 172.21.15.83:0/3492034448 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d8810a210 msgr2=0x7f3d881a5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:47:57.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:57.380+0000 7f3d8ca00640 1 --2- 172.21.15.83:0/3492034448 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d8810a210 0x7f3d881a5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:57.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:57.381+0000 7f3d8ca00640 1 -- 172.21.15.83:0/3492034448 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d881a6500 con 0x7f3d88109850 2024-04-24T08:47:57.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:57.381+0000 7f3d8ca00640 1 --2- 172.21.15.83:0/3492034448 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3d88109850 0x7f3d8819ee20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3d740057c0 tx=0x7f3d74030e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:47:57.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:57.381+0000 7f3d86000640 1 -- 172.21.15.83:0/3492034448 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d74040030 con 0x7f3d88109850 2024-04-24T08:47:57.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:57.381+0000 7f3d8fe00640 1 -- 172.21.15.83:0/3492034448 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d881a6790 con 0x7f3d88109850 2024-04-24T08:47:57.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:57.381+0000 7f3d86000640 1 -- 172.21.15.83:0/3492034448 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3d74038040 con 0x7f3d88109850 2024-04-24T08:47:57.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:57.381+0000 7f3d86000640 1 -- 172.21.15.83:0/3492034448 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d74037070 con 0x7f3d88109850 2024-04-24T08:47:57.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:57.381+0000 7f3d8fe00640 1 -- 172.21.15.83:0/3492034448 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d881a6cd0 con 0x7f3d88109850 2024-04-24T08:47:57.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:57.382+0000 7f3d8fe00640 1 -- 172.21.15.83:0/3492034448 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d8810fa70 con 0x7f3d88109850 2024-04-24T08:47:57.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:57.384+0000 7f3d86000640 1 -- 172.21.15.83:0/3492034448 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f3d74043070 con 0x7f3d88109850 2024-04-24T08:47:57.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:57.386+0000 7f3d86000640 1 --2- 172.21.15.83:0/3492034448 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f3d700a3bb0 0x7f3d700a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:47:57.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:57.386+0000 7f3d86000640 1 -- 172.21.15.83:0/3492034448 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f3d740efb10 con 0x7f3d88109850 2024-04-24T08:47:57.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:57.386+0000 7f3d8d400640 1 --2- 172.21.15.83:0/3492034448 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f3d700a3bb0 0x7f3d700a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:47:57.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:57.387+0000 7f3d8d400640 1 --2- 172.21.15.83:0/3492034448 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f3d700a3bb0 0x7f3d700a6070 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c0096c0 tx=0x7f3d7c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:47:57.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:57.389+0000 7f3d86000640 1 -- 172.21.15.83:0/3492034448 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3d7404a020 con 0x7f3d88109850 2024-04-24T08:47:57.457 INFO:journalctl@ceph.mds.f.smithi189.stdout:Apr 24 08:47:57 smithi189 podman[61391]: 2024-04-24 08:47:57.001819279 +0000 UTC m=+0.464145753 container init 2bceb0b43cf6ee27936a72e68367666c978f2be27e8e484ca20f5fc2fdbd84c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-f, org.label-schema.build-date=20240424, RELEASE=rishabh-main-apr17-a654945-1f81fb3, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T08:47:57.457 INFO:journalctl@ceph.mds.f.smithi189.stdout:Apr 24 08:47:57 smithi189 podman[61391]: 2024-04-24 08:47:57.013790433 +0000 UTC m=+0.476116871 container start 2bceb0b43cf6ee27936a72e68367666c978f2be27e8e484ca20f5fc2fdbd84c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-f, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, ceph=True, RELEASE=rishabh-main-apr17-a654945-1f81fb3, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-04-24T08:47:57.457 INFO:journalctl@ceph.mds.f.smithi189.stdout:Apr 24 08:47:57 smithi189 ceph-mds[61418]: -- 172.21.15.189:0/2134228896 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x557e1c5dc4e0 con 0x557e1c5b3000 2024-04-24T08:47:57.457 INFO:journalctl@ceph.mds.f.smithi189.stdout:Apr 24 08:47:57 smithi189 ceph-mds[61418]: -- 172.21.15.189:0/2134228896 <== mon.1 v2:172.21.15.189:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557e1c5dfa40 con 0x557e1c5b3000 2024-04-24T08:47:57.457 INFO:journalctl@ceph.mds.f.smithi189.stdout:Apr 24 08:47:57 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-f[61409]: starting mds.f at 2024-04-24T08:47:57.457 INFO:journalctl@ceph.mds.f.smithi189.stdout:Apr 24 08:47:57 smithi189 bash[61391]: 2bceb0b43cf6ee27936a72e68367666c978f2be27e8e484ca20f5fc2fdbd84c8 2024-04-24T08:47:57.457 INFO:journalctl@ceph.mds.f.smithi189.stdout:Apr 24 08:47:57 smithi189 systemd[1]: Started Ceph mds.f for bd3e039e-0215-11ef-bc93-c7b262605968. 2024-04-24T08:47:57.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:57.640+0000 7f3d8fe00640 1 -- 172.21.15.83:0/3492034448 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3d8802d620 con 0x7f3d88109850 2024-04-24T08:47:57.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:57.641+0000 7f3d86000640 1 -- 172.21.15.83:0/3492034448 <== mon.1 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4699 (secure 0 0 0) 0x7f3d740bc8f0 con 0x7f3d88109850 2024-04-24T08:47:57.642 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:47:57.642 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/358872257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":358872257},{"type":"v1","addr":"172.21.15.83:6835","nonce":358872257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3157070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3157070851},{"type":"v1","addr":"172.21.15.83:6837","nonce":3157070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/1859921310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1859921310},{"type":"v1","addr":"172.21.15.189:6833","nonce":1859921310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":34427,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/1157967658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1157967658},{"type":"v1","addr":"172.21.15.138:6833","nonce":1157967658}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34451,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1909219758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1909219758},{"type":"v1","addr":"172.21.15.138:6835","nonce":1909219758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-24T08:47:57.643 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 6 2024-04-24T08:47:57.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:57.644+0000 7f3d8fe00640 1 -- 172.21.15.83:0/3492034448 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f3d700a3bb0 msgr2=0x7f3d700a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:47:57.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:57.644+0000 7f3d8fe00640 1 --2- 172.21.15.83:0/3492034448 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f3d700a3bb0 0x7f3d700a6070 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c0096c0 tx=0x7f3d7c009260 comp rx=0 tx=0).stop 2024-04-24T08:47:57.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:57.644+0000 7f3d8fe00640 1 -- 172.21.15.83:0/3492034448 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3d88109850 msgr2=0x7f3d8819ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:47:57.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:57.644+0000 7f3d8fe00640 1 --2- 172.21.15.83:0/3492034448 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3d88109850 0x7f3d8819ee20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3d740057c0 tx=0x7f3d74030e00 comp rx=0 tx=0).stop 2024-04-24T08:47:57.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:57.644+0000 7f3d8fe00640 1 -- 172.21.15.83:0/3492034448 shutdown_connections 2024-04-24T08:47:57.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:57.644+0000 7f3d8fe00640 1 --2- 172.21.15.83:0/3492034448 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f3d700a3bb0 0x7f3d700a6070 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:57.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:57.644+0000 7f3d8fe00640 1 --2- 172.21.15.83:0/3492034448 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d8810a210 0x7f3d881a5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:57.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:57.644+0000 7f3d8fe00640 1 --2- 172.21.15.83:0/3492034448 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3d88109850 0x7f3d8819ee20 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:57.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:57.644+0000 7f3d8fe00640 1 --2- 172.21.15.83:0/3492034448 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d88108660 0x7f3d8819e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:57.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:57.644+0000 7f3d8fe00640 1 -- 172.21.15.83:0/3492034448 >> 172.21.15.83:0/3492034448 conn(0x7f3d88077620 msgr2=0x7f3d8810cc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:47:57.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:57.644+0000 7f3d8fe00640 1 -- 172.21.15.83:0/3492034448 shutdown_connections 2024-04-24T08:47:57.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:57.644+0000 7f3d8fe00640 1 -- 172.21.15.83:0/3492034448 wait complete. 2024-04-24T08:47:58.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:57 smithi138 ceph-mon[31441]: pgmap v349: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:47:58.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:57 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:47:58.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:57 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:47:58.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:57 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:47:58.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:57 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T08:47:58.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:57 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T08:47:58.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:57 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:47:58.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:57 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/3492034448' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:47:58.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:57 smithi189 ceph-mon[31791]: pgmap v349: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:47:58.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:57 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:47:58.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:57 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:47:58.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:57 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:47:58.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:57 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T08:47:58.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:57 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T08:47:58.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:57 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:47:58.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:57 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/3492034448' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:47:58.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:57 smithi083 ceph-mon[27907]: pgmap v349: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:47:58.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:57 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:47:58.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:57 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:47:58.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:57 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:47:58.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:57 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T08:47:58.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:57 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T08:47:58.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:57 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:47:58.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:57 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/3492034448' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:47:59.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:58 smithi083 ceph-mon[27907]: Deploying daemon mds.c on smithi189 2024-04-24T08:47:59.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:58 smithi083 ceph-mon[27907]: mds.? [v2:172.21.15.189:6834/2241923197,v1:172.21.15.189:6835/2241923197] up:boot 2024-04-24T08:47:59.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:58 smithi083 ceph-mon[27907]: fsmap 6 up:standby 2024-04-24T08:47:59.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:58 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-24T08:47:59.324 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:58 smithi189 ceph-mon[31791]: Deploying daemon mds.c on smithi189 2024-04-24T08:47:59.324 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:58 smithi189 ceph-mon[31791]: mds.? [v2:172.21.15.189:6834/2241923197,v1:172.21.15.189:6835/2241923197] up:boot 2024-04-24T08:47:59.324 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:58 smithi189 ceph-mon[31791]: fsmap 6 up:standby 2024-04-24T08:47:59.324 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:58 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-24T08:47:59.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:58 smithi138 ceph-mon[31441]: Deploying daemon mds.c on smithi189 2024-04-24T08:47:59.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:58 smithi138 ceph-mon[31441]: mds.? [v2:172.21.15.189:6834/2241923197,v1:172.21.15.189:6835/2241923197] up:boot 2024-04-24T08:47:59.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:58 smithi138 ceph-mon[31441]: fsmap 6 up:standby 2024-04-24T08:47:59.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:58 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-24T08:47:59.665 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T08:47:59.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:59.879+0000 7faeefe00640 1 -- 172.21.15.83:0/2226115449 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faef0107be0 msgr2=0x7faef0109fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:47:59.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:59.879+0000 7faeefe00640 1 --2- 172.21.15.83:0/2226115449 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faef0107be0 0x7faef0109fd0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7faee8009ca0 tx=0x7faee802a8c0 comp rx=0 tx=0).stop 2024-04-24T08:47:59.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:59.879+0000 7faeefe00640 1 -- 172.21.15.83:0/2226115449 shutdown_connections 2024-04-24T08:47:59.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:59.879+0000 7faeefe00640 1 --2- 172.21.15.83:0/2226115449 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faef0107be0 0x7faef0109fd0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:59.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:59.879+0000 7faeefe00640 1 --2- 172.21.15.83:0/2226115449 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faef01052b0 0x7faef01076a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:59.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:59.879+0000 7faeefe00640 1 --2- 172.21.15.83:0/2226115449 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faef0102980 0x7faef0104d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:59.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:59.879+0000 7faeefe00640 1 -- 172.21.15.83:0/2226115449 >> 172.21.15.83:0/2226115449 conn(0x7faef00fc2d0 msgr2=0x7faef00fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:47:59.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:59.879+0000 7faeefe00640 1 -- 172.21.15.83:0/2226115449 shutdown_connections 2024-04-24T08:47:59.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:59.879+0000 7faeefe00640 1 -- 172.21.15.83:0/2226115449 wait complete. 2024-04-24T08:47:59.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:59.880+0000 7faeefe00640 1 Processor -- start 2024-04-24T08:47:59.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:59.880+0000 7faeefe00640 1 -- start start 2024-04-24T08:47:59.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:59.881+0000 7faeefe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faef0102980 0x7faef01022f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:47:59.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:59.881+0000 7faeefe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faef01052b0 0x7faef0100930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:47:59.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:59.881+0000 7faeefe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faef0107be0 0x7faef0100e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:47:59.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:59.881+0000 7faeefe00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7faef010da20 con 0x7faef0107be0 2024-04-24T08:47:59.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:59.881+0000 7faeefe00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7faef010d8a0 con 0x7faef0102980 2024-04-24T08:47:59.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:59.881+0000 7faeefe00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7faef010dba0 con 0x7faef01052b0 2024-04-24T08:47:59.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:59.881+0000 7faeef400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faef0107be0 0x7faef0100e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:47:59.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:59.881+0000 7faeef400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faef0107be0 0x7faef0100e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58538/0 (socket says 172.21.15.83:58538) 2024-04-24T08:47:59.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:59.881+0000 7faeef400640 1 -- 172.21.15.83:0/3283358271 learned_addr learned my addr 172.21.15.83:0/3283358271 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:47:59.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:59.881+0000 7faeee000640 1 --2- 172.21.15.83:0/3283358271 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faef01052b0 0x7faef0100930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:47:59.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:59.881+0000 7faeeea00640 1 --2- 172.21.15.83:0/3283358271 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faef0102980 0x7faef01022f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:47:59.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:59.881+0000 7faeef400640 1 -- 172.21.15.83:0/3283358271 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faef01052b0 msgr2=0x7faef0100930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:47:59.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:59.881+0000 7faeef400640 1 --2- 172.21.15.83:0/3283358271 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faef01052b0 0x7faef0100930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:59.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:59.881+0000 7faeef400640 1 -- 172.21.15.83:0/3283358271 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faef0102980 msgr2=0x7faef01022f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:47:59.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:59.881+0000 7faeef400640 1 --2- 172.21.15.83:0/3283358271 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faef0102980 0x7faef01022f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:47:59.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:59.881+0000 7faeef400640 1 -- 172.21.15.83:0/3283358271 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faef0101700 con 0x7faef0107be0 2024-04-24T08:47:59.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:59.882+0000 7faeeea00640 1 --2- 172.21.15.83:0/3283358271 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faef0102980 0x7faef01022f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:47:59.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:59.882+0000 7faeef400640 1 --2- 172.21.15.83:0/3283358271 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faef0107be0 0x7faef0100e70 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7faee8031040 tx=0x7faee8003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:47:59.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:59.882+0000 7faee7400640 1 -- 172.21.15.83:0/3283358271 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faee8032ec0 con 0x7faef0107be0 2024-04-24T08:47:59.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:59.882+0000 7faee7400640 1 -- 172.21.15.83:0/3283358271 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faee80044f0 con 0x7faef0107be0 2024-04-24T08:47:59.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:59.882+0000 7faee7400640 1 -- 172.21.15.83:0/3283358271 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faee8030d10 con 0x7faef0107be0 2024-04-24T08:47:59.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:59.882+0000 7faeefe00640 1 -- 172.21.15.83:0/3283358271 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faef01a21b0 con 0x7faef0107be0 2024-04-24T08:47:59.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:59.882+0000 7faeefe00640 1 -- 172.21.15.83:0/3283358271 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faef01a2750 con 0x7faef0107be0 2024-04-24T08:47:59.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:59.884+0000 7faeefe00640 1 -- 172.21.15.83:0/3283358271 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faef00fddc0 con 0x7faef0107be0 2024-04-24T08:47:59.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:59.886+0000 7faee7400640 1 -- 172.21.15.83:0/3283358271 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7faee8004040 con 0x7faef0107be0 2024-04-24T08:47:59.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:59.887+0000 7faee7400640 1 --2- 172.21.15.83:0/3283358271 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7faed40a3dd0 0x7faed40a6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:47:59.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:59.887+0000 7faee7400640 1 -- 172.21.15.83:0/3283358271 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7faee80f1690 con 0x7faef0107be0 2024-04-24T08:47:59.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:59.887+0000 7faeeea00640 1 --2- 172.21.15.83:0/3283358271 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7faed40a3dd0 0x7faed40a6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:47:59.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:59.888+0000 7faeeea00640 1 --2- 172.21.15.83:0/3283358271 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7faed40a3dd0 0x7faed40a6290 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7faee00058c0 tx=0x7faee0005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:47:59.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:47:59.892+0000 7faee7400640 1 -- 172.21.15.83:0/3283358271 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7faee80be470 con 0x7faef0107be0 2024-04-24T08:48:00.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:47:59 smithi083 ceph-mon[27907]: pgmap v350: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:48:00.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:00.136+0000 7faeefe00640 1 -- 172.21.15.83:0/3283358271 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faef002d620 con 0x7faef0107be0 2024-04-24T08:48:00.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:00.137+0000 7faee7400640 1 -- 172.21.15.83:0/3283358271 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5527 (secure 0 0 0) 0x7faee8039e10 con 0x7faef0107be0 2024-04-24T08:48:00.138 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:48:00.138 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/358872257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":358872257},{"type":"v1","addr":"172.21.15.83:6835","nonce":358872257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3157070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3157070851},{"type":"v1","addr":"172.21.15.83:6837","nonce":3157070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/1859921310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1859921310},{"type":"v1","addr":"172.21.15.189:6833","nonce":1859921310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24421,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/2241923197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2241923197},{"type":"v1","addr":"172.21.15.189:6835","nonce":2241923197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":34427,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/1157967658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1157967658},{"type":"v1","addr":"172.21.15.138:6833","nonce":1157967658}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34451,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1909219758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1909219758},{"type":"v1","addr":"172.21.15.138:6835","nonce":1909219758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-24T08:48:00.139 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 7 2024-04-24T08:48:00.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:00.140+0000 7faeefe00640 1 -- 172.21.15.83:0/3283358271 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7faed40a3dd0 msgr2=0x7faed40a6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:00.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:00.140+0000 7faeefe00640 1 --2- 172.21.15.83:0/3283358271 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7faed40a3dd0 0x7faed40a6290 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7faee00058c0 tx=0x7faee0005850 comp rx=0 tx=0).stop 2024-04-24T08:48:00.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:00.140+0000 7faeefe00640 1 -- 172.21.15.83:0/3283358271 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faef0107be0 msgr2=0x7faef0100e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:00.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:00.140+0000 7faeefe00640 1 --2- 172.21.15.83:0/3283358271 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faef0107be0 0x7faef0100e70 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7faee8031040 tx=0x7faee8003ba0 comp rx=0 tx=0).stop 2024-04-24T08:48:00.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:00.140+0000 7faeefe00640 1 -- 172.21.15.83:0/3283358271 shutdown_connections 2024-04-24T08:48:00.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:00.140+0000 7faeefe00640 1 --2- 172.21.15.83:0/3283358271 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7faed40a3dd0 0x7faed40a6290 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:00.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:00.140+0000 7faeefe00640 1 --2- 172.21.15.83:0/3283358271 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faef0107be0 0x7faef0100e70 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:00.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:00.140+0000 7faeefe00640 1 --2- 172.21.15.83:0/3283358271 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faef01052b0 0x7faef0100930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:00.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:00.140+0000 7faeefe00640 1 --2- 172.21.15.83:0/3283358271 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faef0102980 0x7faef01022f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:00.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:00.141+0000 7faeefe00640 1 -- 172.21.15.83:0/3283358271 >> 172.21.15.83:0/3283358271 conn(0x7faef00fc2d0 msgr2=0x7faef01032c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:48:00.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:00.141+0000 7faeefe00640 1 -- 172.21.15.83:0/3283358271 shutdown_connections 2024-04-24T08:48:00.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:00.141+0000 7faeefe00640 1 -- 172.21.15.83:0/3283358271 wait complete. 2024-04-24T08:48:00.286 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:47:59 smithi189 ceph-mon[31791]: pgmap v350: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:48:00.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:47:59 smithi138 ceph-mon[31441]: pgmap v350: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:48:00.669 INFO:journalctl@ceph.mds.c.smithi189.stdout:Apr 24 08:48:00 smithi189 systemd[1]: Starting Ceph mds.c for bd3e039e-0215-11ef-bc93-c7b262605968... 2024-04-24T08:48:00.958 INFO:journalctl@ceph.mds.c.smithi189.stdout:Apr 24 08:48:00 smithi189 podman[61708]: 2024-04-24 08:48:00.57235619 +0000 UTC m=+0.033811605 image pull b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644 2024-04-24T08:48:00.959 INFO:journalctl@ceph.mds.c.smithi189.stdout:Apr 24 08:48:00 smithi189 podman[61708]: 2024-04-24 08:48:00.678250889 +0000 UTC m=+0.139706328 container create f191a26fb2d2697eef67b2a1c3c0fd9d0cde7094e36f48c8133c50e7443ed4cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-c, RELEASE=rishabh-main-apr17-a654945-1f81fb3, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-04-24T08:48:01.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:00 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/3283358271' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:48:01.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:00 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/3283358271' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:48:01.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:00 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/3283358271' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:48:01.457 INFO:journalctl@ceph.mds.c.smithi189.stdout:Apr 24 08:48:00 smithi189 podman[61708]: 2024-04-24 08:48:00.99945255 +0000 UTC m=+0.460907996 container init f191a26fb2d2697eef67b2a1c3c0fd9d0cde7094e36f48c8133c50e7443ed4cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, RELEASE=rishabh-main-apr17-a654945-1f81fb3, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T08:48:01.458 INFO:journalctl@ceph.mds.c.smithi189.stdout:Apr 24 08:48:01 smithi189 podman[61708]: 2024-04-24 08:48:01.010205433 +0000 UTC m=+0.471660882 container start f191a26fb2d2697eef67b2a1c3c0fd9d0cde7094e36f48c8133c50e7443ed4cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-c, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-24T08:48:01.458 INFO:journalctl@ceph.mds.c.smithi189.stdout:Apr 24 08:48:01 smithi189 ceph-mds[61728]: -- 172.21.15.189:0/2705947229 <== mon.1 v2:172.21.15.189:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55bf71a95a40 con 0x55bf71a69000 2024-04-24T08:48:01.458 INFO:journalctl@ceph.mds.c.smithi189.stdout:Apr 24 08:48:01 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-c[61724]: starting mds.c at 2024-04-24T08:48:01.458 INFO:journalctl@ceph.mds.c.smithi189.stdout:Apr 24 08:48:01 smithi189 bash[61708]: f191a26fb2d2697eef67b2a1c3c0fd9d0cde7094e36f48c8133c50e7443ed4cd 2024-04-24T08:48:01.458 INFO:journalctl@ceph.mds.c.smithi189.stdout:Apr 24 08:48:01 smithi189 systemd[1]: Started Ceph mds.c for bd3e039e-0215-11ef-bc93-c7b262605968. 2024-04-24T08:48:02.161 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T08:48:02.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:01 smithi083 ceph-mon[27907]: pgmap v351: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:48:02.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:01 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:48:02.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:01 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:48:02.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:01 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:48:02.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:01 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T08:48:02.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:01 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T08:48:02.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:01 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:48:02.308 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:01 smithi138 ceph-mon[31441]: pgmap v351: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:48:02.308 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:01 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:48:02.309 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:01 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:48:02.309 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:01 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:48:02.309 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:01 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T08:48:02.309 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:01 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T08:48:02.309 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:01 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:48:02.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:02.372+0000 7f8988000640 1 -- 172.21.15.83:0/1734152463 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8980109850 msgr2=0x7f8980109cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:02.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:02.372+0000 7f8988000640 1 --2- 172.21.15.83:0/1734152463 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8980109850 0x7f8980109cd0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f896c009900 tx=0x7f896c02a400 comp rx=0 tx=0).stop 2024-04-24T08:48:02.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:02.372+0000 7f8988000640 1 -- 172.21.15.83:0/1734152463 shutdown_connections 2024-04-24T08:48:02.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:02.372+0000 7f8988000640 1 --2- 172.21.15.83:0/1734152463 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f898010a210 0x7f8980110aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:02.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:02.372+0000 7f8988000640 1 --2- 172.21.15.83:0/1734152463 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8980109850 0x7f8980109cd0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:02.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:02.372+0000 7f8988000640 1 --2- 172.21.15.83:0/1734152463 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8980108660 0x7f8980108a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:02.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:02.372+0000 7f8988000640 1 -- 172.21.15.83:0/1734152463 >> 172.21.15.83:0/1734152463 conn(0x7f8980077620 msgr2=0x7f8980079a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:48:02.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:02.373+0000 7f8988000640 1 -- 172.21.15.83:0/1734152463 shutdown_connections 2024-04-24T08:48:02.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:02.373+0000 7f8988000640 1 -- 172.21.15.83:0/1734152463 wait complete. 2024-04-24T08:48:02.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:02.374+0000 7f8988000640 1 Processor -- start 2024-04-24T08:48:02.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:02.374+0000 7f8988000640 1 -- start start 2024-04-24T08:48:02.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:02.374+0000 7f8988000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8980108660 0x7f898019e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:02.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:02.374+0000 7f8988000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8980109850 0x7f898019ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:02.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:02.374+0000 7f8988000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f898010a210 0x7f89801a5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:02.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:02.374+0000 7f8988000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f89801141c0 con 0x7f898010a210 2024-04-24T08:48:02.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:02.374+0000 7f8988000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8980114040 con 0x7f8980108660 2024-04-24T08:48:02.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:02.374+0000 7f8988000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8980114340 con 0x7f8980109850 2024-04-24T08:48:02.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:02.374+0000 7f8986000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f898010a210 0x7f89801a5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:02.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:02.374+0000 7f8986000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f898010a210 0x7f89801a5de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58558/0 (socket says 172.21.15.83:58558) 2024-04-24T08:48:02.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:02.374+0000 7f8986000640 1 -- 172.21.15.83:0/4261406885 learned_addr learned my addr 172.21.15.83:0/4261406885 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:48:02.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:02.375+0000 7f8984c00640 1 --2- 172.21.15.83:0/4261406885 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8980109850 0x7f898019ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:02.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:02.375+0000 7f8985600640 1 --2- 172.21.15.83:0/4261406885 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8980108660 0x7f898019e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:02.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:02.375+0000 7f8986000640 1 -- 172.21.15.83:0/4261406885 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8980109850 msgr2=0x7f898019ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:02.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:02.375+0000 7f8986000640 1 --2- 172.21.15.83:0/4261406885 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8980109850 0x7f898019ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:02.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:02.375+0000 7f8986000640 1 -- 172.21.15.83:0/4261406885 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8980108660 msgr2=0x7f898019e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:02.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:02.375+0000 7f8986000640 1 --2- 172.21.15.83:0/4261406885 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8980108660 0x7f898019e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:02.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:02.375+0000 7f8986000640 1 -- 172.21.15.83:0/4261406885 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89801a6450 con 0x7f898010a210 2024-04-24T08:48:02.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:02.375+0000 7f8986000640 1 --2- 172.21.15.83:0/4261406885 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f898010a210 0x7f89801a5de0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f897800c970 tx=0x7f897800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:48:02.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:02.375+0000 7f897e000640 1 -- 172.21.15.83:0/4261406885 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8978007bb0 con 0x7f898010a210 2024-04-24T08:48:02.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:02.375+0000 7f897e000640 1 -- 172.21.15.83:0/4261406885 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8978007d50 con 0x7f898010a210 2024-04-24T08:48:02.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:02.375+0000 7f897e000640 1 -- 172.21.15.83:0/4261406885 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8978018700 con 0x7f898010a210 2024-04-24T08:48:02.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:02.375+0000 7f8988000640 1 -- 172.21.15.83:0/4261406885 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89801a6740 con 0x7f898010a210 2024-04-24T08:48:02.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:02.376+0000 7f8988000640 1 -- 172.21.15.83:0/4261406885 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89801a6c10 con 0x7f898010a210 2024-04-24T08:48:02.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:02.377+0000 7f8988000640 1 -- 172.21.15.83:0/4261406885 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f898010f980 con 0x7f898010a210 2024-04-24T08:48:02.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:02.379+0000 7f897e000640 1 -- 172.21.15.83:0/4261406885 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f897800fa20 con 0x7f898010a210 2024-04-24T08:48:02.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:02.381+0000 7f897e000640 1 --2- 172.21.15.83:0/4261406885 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f89680a3c30 0x7f89680a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:02.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:02.381+0000 7f897e000640 1 -- 172.21.15.83:0/4261406885 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f89780d0040 con 0x7f898010a210 2024-04-24T08:48:02.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:02.381+0000 7f8985600640 1 --2- 172.21.15.83:0/4261406885 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f89680a3c30 0x7f89680a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:02.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:02.382+0000 7f8985600640 1 --2- 172.21.15.83:0/4261406885 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f89680a3c30 0x7f89680a60f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f897400a300 tx=0x7f897400a290 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:48:02.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:02.385+0000 7f897e000640 1 -- 172.21.15.83:0/4261406885 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8978014030 con 0x7f898010a210 2024-04-24T08:48:02.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:01 smithi189 ceph-mon[31791]: pgmap v351: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:48:02.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:01 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:48:02.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:01 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:48:02.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:01 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:48:02.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:01 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T08:48:02.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:01 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T08:48:02.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:01 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:48:02.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:02.627+0000 7f8988000640 1 -- 172.21.15.83:0/4261406885 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f898002d840 con 0x7f898010a210 2024-04-24T08:48:02.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:02.628+0000 7f897e000640 1 -- 172.21.15.83:0/4261406885 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6355 (secure 0 0 0) 0x7f897809cec0 con 0x7f898010a210 2024-04-24T08:48:02.628 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:48:02.629 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/358872257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":358872257},{"type":"v1","addr":"172.21.15.83:6835","nonce":358872257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3157070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3157070851},{"type":"v1","addr":"172.21.15.83:6837","nonce":3157070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/1859921310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1859921310},{"type":"v1","addr":"172.21.15.189:6833","nonce":1859921310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24421,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/2241923197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2241923197},{"type":"v1","addr":"172.21.15.189:6835","nonce":2241923197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24439,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/3172311220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":3172311220},{"type":"v1","addr":"172.21.15.189:6837","nonce":3172311220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":34427,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/1157967658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1157967658},{"type":"v1","addr":"172.21.15.138:6833","nonce":1157967658}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34451,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1909219758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1909219758},{"type":"v1","addr":"172.21.15.138:6835","nonce":1909219758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-24T08:48:02.629 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 8 2024-04-24T08:48:02.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:02.630+0000 7f8988000640 1 -- 172.21.15.83:0/4261406885 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f89680a3c30 msgr2=0x7f89680a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:02.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:02.630+0000 7f8988000640 1 --2- 172.21.15.83:0/4261406885 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f89680a3c30 0x7f89680a60f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f897400a300 tx=0x7f897400a290 comp rx=0 tx=0).stop 2024-04-24T08:48:02.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:02.630+0000 7f8988000640 1 -- 172.21.15.83:0/4261406885 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f898010a210 msgr2=0x7f89801a5de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:02.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:02.630+0000 7f8988000640 1 --2- 172.21.15.83:0/4261406885 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f898010a210 0x7f89801a5de0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f897800c970 tx=0x7f897800ce30 comp rx=0 tx=0).stop 2024-04-24T08:48:02.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:02.630+0000 7f8988000640 1 -- 172.21.15.83:0/4261406885 shutdown_connections 2024-04-24T08:48:02.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:02.630+0000 7f8988000640 1 --2- 172.21.15.83:0/4261406885 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f89680a3c30 0x7f89680a60f0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:02.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:02.630+0000 7f8988000640 1 --2- 172.21.15.83:0/4261406885 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f898010a210 0x7f89801a5de0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:02.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:02.630+0000 7f8988000640 1 --2- 172.21.15.83:0/4261406885 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8980109850 0x7f898019ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:02.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:02.630+0000 7f8988000640 1 --2- 172.21.15.83:0/4261406885 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8980108660 0x7f898019e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:02.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:02.630+0000 7f8988000640 1 -- 172.21.15.83:0/4261406885 >> 172.21.15.83:0/4261406885 conn(0x7f8980077620 msgr2=0x7f898010cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:48:02.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:02.630+0000 7f8988000640 1 -- 172.21.15.83:0/4261406885 shutdown_connections 2024-04-24T08:48:02.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:02.631+0000 7f8988000640 1 -- 172.21.15.83:0/4261406885 wait complete. 2024-04-24T08:48:03.286 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:03 smithi138 ceph-mon[31441]: Deploying daemon mds.h on smithi138 2024-04-24T08:48:03.286 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:03 smithi138 ceph-mon[31441]: mds.? [v2:172.21.15.189:6836/3172311220,v1:172.21.15.189:6837/3172311220] up:boot 2024-04-24T08:48:03.286 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:03 smithi138 ceph-mon[31441]: fsmap 7 up:standby 2024-04-24T08:48:03.286 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:03 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-24T08:48:03.287 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:03 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/4261406885' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:48:03.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:03 smithi083 ceph-mon[27907]: Deploying daemon mds.h on smithi138 2024-04-24T08:48:03.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:03 smithi083 ceph-mon[27907]: mds.? [v2:172.21.15.189:6836/3172311220,v1:172.21.15.189:6837/3172311220] up:boot 2024-04-24T08:48:03.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:03 smithi083 ceph-mon[27907]: fsmap 7 up:standby 2024-04-24T08:48:03.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:03 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-24T08:48:03.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:03 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/4261406885' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:48:03.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:03 smithi189 ceph-mon[31791]: Deploying daemon mds.h on smithi138 2024-04-24T08:48:03.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:03 smithi189 ceph-mon[31791]: mds.? [v2:172.21.15.189:6836/3172311220,v1:172.21.15.189:6837/3172311220] up:boot 2024-04-24T08:48:03.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:03 smithi189 ceph-mon[31791]: fsmap 7 up:standby 2024-04-24T08:48:03.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:03 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-24T08:48:03.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:03 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/4261406885' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:48:04.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:04 smithi138 ceph-mon[31441]: pgmap v352: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:48:04.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:04 smithi083 ceph-mon[27907]: pgmap v352: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:48:04.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:04 smithi189 ceph-mon[31791]: pgmap v352: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:48:04.652 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T08:48:04.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:04.866+0000 7f23c5c00640 1 -- 172.21.15.83:0/122637999 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23c010a030 msgr2=0x7f23c01108c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:04.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:04.866+0000 7f23c5c00640 1 --2- 172.21.15.83:0/122637999 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23c010a030 0x7f23c01108c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f23b8009ca0 tx=0x7f23b802a8c0 comp rx=0 tx=0).stop 2024-04-24T08:48:04.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:04.866+0000 7f23c5c00640 1 -- 172.21.15.83:0/122637999 shutdown_connections 2024-04-24T08:48:04.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:04.866+0000 7f23c5c00640 1 --2- 172.21.15.83:0/122637999 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23c010a030 0x7f23c01108c0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:04.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:04.866+0000 7f23c5c00640 1 --2- 172.21.15.83:0/122637999 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f23c0109670 0x7f23c0109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:04.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:04.866+0000 7f23c5c00640 1 --2- 172.21.15.83:0/122637999 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23c0108480 0x7f23c01088a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:04.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:04.866+0000 7f23c5c00640 1 -- 172.21.15.83:0/122637999 >> 172.21.15.83:0/122637999 conn(0x7f23c0077440 msgr2=0x7f23c0079860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:48:04.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:04.867+0000 7f23c5c00640 1 -- 172.21.15.83:0/122637999 shutdown_connections 2024-04-24T08:48:04.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:04.867+0000 7f23c5c00640 1 -- 172.21.15.83:0/122637999 wait complete. 2024-04-24T08:48:04.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:04.868+0000 7f23c5c00640 1 Processor -- start 2024-04-24T08:48:04.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:04.868+0000 7f23c5c00640 1 -- start start 2024-04-24T08:48:04.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:04.868+0000 7f23c5c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23c0108480 0x7f23c019e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:04.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:04.868+0000 7f23c5c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23c0109670 0x7f23c019eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:04.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:04.868+0000 7f23c5c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f23c010a030 0x7f23c01a5c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:04.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:04.868+0000 7f23c5c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f23c0113fe0 con 0x7f23c0109670 2024-04-24T08:48:04.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:04.868+0000 7f23c5c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f23c0113e60 con 0x7f23c0108480 2024-04-24T08:48:04.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:04.868+0000 7f23c5c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f23c0114160 con 0x7f23c010a030 2024-04-24T08:48:04.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:04.868+0000 7f23bf400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23c0109670 0x7f23c019eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:04.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:04.868+0000 7f23bf400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23c0109670 0x7f23c019eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58584/0 (socket says 172.21.15.83:58584) 2024-04-24T08:48:04.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:04.868+0000 7f23bf400640 1 -- 172.21.15.83:0/459813263 learned_addr learned my addr 172.21.15.83:0/459813263 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:48:04.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:04.868+0000 7f23bfe00640 1 --2- 172.21.15.83:0/459813263 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23c0108480 0x7f23c019e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:04.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:04.869+0000 7f23c5200640 1 --2- 172.21.15.83:0/459813263 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f23c010a030 0x7f23c01a5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:04.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:04.869+0000 7f23bf400640 1 -- 172.21.15.83:0/459813263 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f23c010a030 msgr2=0x7f23c01a5c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:04.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:04.869+0000 7f23bf400640 1 --2- 172.21.15.83:0/459813263 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f23c010a030 0x7f23c01a5c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:04.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:04.869+0000 7f23bf400640 1 -- 172.21.15.83:0/459813263 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23c0108480 msgr2=0x7f23c019e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:04.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:04.869+0000 7f23bf400640 1 --2- 172.21.15.83:0/459813263 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23c0108480 0x7f23c019e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:04.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:04.869+0000 7f23bf400640 1 -- 172.21.15.83:0/459813263 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23c01a6270 con 0x7f23c0109670 2024-04-24T08:48:04.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:04.869+0000 7f23bf400640 1 --2- 172.21.15.83:0/459813263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23c0109670 0x7f23c019eb90 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f23ac00c890 tx=0x7f23ac00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:48:04.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:04.869+0000 7f23bcc00640 1 -- 172.21.15.83:0/459813263 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23ac004220 con 0x7f23c0109670 2024-04-24T08:48:04.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:04.869+0000 7f23bcc00640 1 -- 172.21.15.83:0/459813263 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f23ac0043c0 con 0x7f23c0109670 2024-04-24T08:48:04.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:04.869+0000 7f23c5c00640 1 -- 172.21.15.83:0/459813263 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23c01a6560 con 0x7f23c0109670 2024-04-24T08:48:04.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:04.869+0000 7f23bcc00640 1 -- 172.21.15.83:0/459813263 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23ac004a00 con 0x7f23c0109670 2024-04-24T08:48:04.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:04.870+0000 7f23c5c00640 1 -- 172.21.15.83:0/459813263 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23c01a6ab0 con 0x7f23c0109670 2024-04-24T08:48:04.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:04.871+0000 7f23c5c00640 1 -- 172.21.15.83:0/459813263 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23c010f7a0 con 0x7f23c0109670 2024-04-24T08:48:04.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:04.873+0000 7f23bcc00640 1 -- 172.21.15.83:0/459813263 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f23ac004ba0 con 0x7f23c0109670 2024-04-24T08:48:04.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:04.875+0000 7f23bcc00640 1 --2- 172.21.15.83:0/459813263 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f23a40a3bb0 0x7f23a40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:04.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:04.876+0000 7f23bcc00640 1 -- 172.21.15.83:0/459813263 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f23ac0cfbf0 con 0x7f23c0109670 2024-04-24T08:48:04.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:04.876+0000 7f23bfe00640 1 --2- 172.21.15.83:0/459813263 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f23a40a3bb0 0x7f23a40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:04.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:04.876+0000 7f23bfe00640 1 --2- 172.21.15.83:0/459813263 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f23a40a3bb0 0x7f23a40a6070 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f23c01094d0 tx=0x7f23b000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:48:04.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:04.880+0000 7f23bcc00640 1 -- 172.21.15.83:0/459813263 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f23ac014030 con 0x7f23c0109670 2024-04-24T08:48:04.956 INFO:journalctl@ceph.mds.h.smithi138.stdout:Apr 24 08:48:04 smithi138 systemd[1]: Starting Ceph mds.h for bd3e039e-0215-11ef-bc93-c7b262605968... 2024-04-24T08:48:05.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:05.122+0000 7f23c5c00640 1 -- 172.21.15.83:0/459813263 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f23c002d880 con 0x7f23c0109670 2024-04-24T08:48:05.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:05.123+0000 7f23bcc00640 1 -- 172.21.15.83:0/459813263 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6355 (secure 0 0 0) 0x7f23ac09ca70 con 0x7f23c0109670 2024-04-24T08:48:05.124 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:48:05.124 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/358872257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":358872257},{"type":"v1","addr":"172.21.15.83:6835","nonce":358872257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3157070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3157070851},{"type":"v1","addr":"172.21.15.83:6837","nonce":3157070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/1859921310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1859921310},{"type":"v1","addr":"172.21.15.189:6833","nonce":1859921310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24421,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/2241923197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2241923197},{"type":"v1","addr":"172.21.15.189:6835","nonce":2241923197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24439,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/3172311220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":3172311220},{"type":"v1","addr":"172.21.15.189:6837","nonce":3172311220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":34427,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/1157967658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1157967658},{"type":"v1","addr":"172.21.15.138:6833","nonce":1157967658}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34451,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1909219758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1909219758},{"type":"v1","addr":"172.21.15.138:6835","nonce":1909219758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-24T08:48:05.124 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 8 2024-04-24T08:48:05.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:05.126+0000 7f23c5c00640 1 -- 172.21.15.83:0/459813263 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f23a40a3bb0 msgr2=0x7f23a40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:05.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:05.126+0000 7f23c5c00640 1 --2- 172.21.15.83:0/459813263 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f23a40a3bb0 0x7f23a40a6070 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f23c01094d0 tx=0x7f23b000b390 comp rx=0 tx=0).stop 2024-04-24T08:48:05.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:05.126+0000 7f23c5c00640 1 -- 172.21.15.83:0/459813263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23c0109670 msgr2=0x7f23c019eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:05.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:05.126+0000 7f23c5c00640 1 --2- 172.21.15.83:0/459813263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23c0109670 0x7f23c019eb90 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f23ac00c890 tx=0x7f23ac00cd50 comp rx=0 tx=0).stop 2024-04-24T08:48:05.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:05.126+0000 7f23c5c00640 1 -- 172.21.15.83:0/459813263 shutdown_connections 2024-04-24T08:48:05.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:05.126+0000 7f23c5c00640 1 --2- 172.21.15.83:0/459813263 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f23c010a030 0x7f23c01a5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:05.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:05.126+0000 7f23c5c00640 1 --2- 172.21.15.83:0/459813263 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f23a40a3bb0 0x7f23a40a6070 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:05.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:05.126+0000 7f23c5c00640 1 --2- 172.21.15.83:0/459813263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23c0109670 0x7f23c019eb90 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:05.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:05.126+0000 7f23c5c00640 1 --2- 172.21.15.83:0/459813263 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23c0108480 0x7f23c019e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:05.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:05.126+0000 7f23c5c00640 1 -- 172.21.15.83:0/459813263 >> 172.21.15.83:0/459813263 conn(0x7f23c0077440 msgr2=0x7f23c010c970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:48:05.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:05.126+0000 7f23c5c00640 1 -- 172.21.15.83:0/459813263 shutdown_connections 2024-04-24T08:48:05.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:05.126+0000 7f23c5c00640 1 -- 172.21.15.83:0/459813263 wait complete. 2024-04-24T08:48:05.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:05 smithi083 ceph-mon[27907]: pgmap v353: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:48:05.314 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:05 smithi138 ceph-mon[31441]: pgmap v353: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:48:05.315 INFO:journalctl@ceph.mds.h.smithi138.stdout:Apr 24 08:48:04 smithi138 podman[61185]: 2024-04-24 08:48:04.864354284 +0000 UTC m=+0.032370082 image pull b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644 2024-04-24T08:48:05.315 INFO:journalctl@ceph.mds.h.smithi138.stdout:Apr 24 08:48:05 smithi138 podman[61185]: 2024-04-24 08:48:05.019822152 +0000 UTC m=+0.187837984 container create 55769a4930c49093995c7798b7554eec429db4162c43c3f845aa00963123d54d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-h, RELEASE=rishabh-main-apr17-a654945-1f81fb3, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True) 2024-04-24T08:48:05.315 INFO:journalctl@ceph.mds.h.smithi138.stdout:Apr 24 08:48:05 smithi138 podman[61185]: 2024-04-24 08:48:05.31478681 +0000 UTC m=+0.482802646 container init 55769a4930c49093995c7798b7554eec429db4162c43c3f845aa00963123d54d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-h, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , RELEASE=rishabh-main-apr17-a654945-1f81fb3, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0) 2024-04-24T08:48:05.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:05 smithi189 ceph-mon[31791]: pgmap v353: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:48:05.705 INFO:journalctl@ceph.mds.h.smithi138.stdout:Apr 24 08:48:05 smithi138 podman[61185]: 2024-04-24 08:48:05.325309485 +0000 UTC m=+0.493325317 container start 55769a4930c49093995c7798b7554eec429db4162c43c3f845aa00963123d54d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-h, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T08:48:05.705 INFO:journalctl@ceph.mds.h.smithi138.stdout:Apr 24 08:48:05 smithi138 ceph-mds[61203]: -- 172.21.15.138:0/2117521543 <== mon.2 v2:172.21.15.138:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e3b87f3a40 con 0x55e3b87c6c00 2024-04-24T08:48:05.705 INFO:journalctl@ceph.mds.h.smithi138.stdout:Apr 24 08:48:05 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-h[61199]: starting mds.h at 2024-04-24T08:48:05.706 INFO:journalctl@ceph.mds.h.smithi138.stdout:Apr 24 08:48:05 smithi138 bash[61185]: 55769a4930c49093995c7798b7554eec429db4162c43c3f845aa00963123d54d 2024-04-24T08:48:05.706 INFO:journalctl@ceph.mds.h.smithi138.stdout:Apr 24 08:48:05 smithi138 systemd[1]: Started Ceph mds.h for bd3e039e-0215-11ef-bc93-c7b262605968. 2024-04-24T08:48:06.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:06 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/459813263' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:48:06.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:06 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:48:06.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:06 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:48:06.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:06 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:48:06.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:06 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T08:48:06.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:06 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T08:48:06.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:06 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:48:06.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:06 smithi083 ceph-mon[27907]: Deploying daemon mds.i on smithi189 2024-04-24T08:48:06.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:06 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/459813263' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:48:06.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:06 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:48:06.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:06 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:48:06.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:06 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:48:06.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:06 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T08:48:06.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:06 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T08:48:06.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:06 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:48:06.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:06 smithi138 ceph-mon[31441]: Deploying daemon mds.i on smithi189 2024-04-24T08:48:06.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:06 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/459813263' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:48:06.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:06 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:48:06.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:06 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:48:06.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:06 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:48:06.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:06 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T08:48:06.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:06 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T08:48:06.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:06 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:48:06.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:06 smithi189 ceph-mon[31791]: Deploying daemon mds.i on smithi189 2024-04-24T08:48:07.147 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T08:48:07.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:07 smithi138 ceph-mon[31441]: mds.? [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] up:boot 2024-04-24T08:48:07.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:07 smithi138 ceph-mon[31441]: fsmap 8 up:standby 2024-04-24T08:48:07.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:07 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-24T08:48:07.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:07 smithi138 ceph-mon[31441]: pgmap v354: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:48:07.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:07 smithi083 ceph-mon[27907]: mds.? [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] up:boot 2024-04-24T08:48:07.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:07 smithi083 ceph-mon[27907]: fsmap 8 up:standby 2024-04-24T08:48:07.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:07 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-24T08:48:07.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:07 smithi083 ceph-mon[27907]: pgmap v354: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:48:07.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:07.360+0000 7fa1d8a00640 1 -- 172.21.15.83:0/1767053109 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1d4105450 msgr2=0x7fa1d41058d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:07.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:07.360+0000 7fa1d8a00640 1 --2- 172.21.15.83:0/1767053109 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1d4105450 0x7fa1d41058d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa1bc009900 tx=0x7fa1bc02a410 comp rx=0 tx=0).stop 2024-04-24T08:48:07.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:07.360+0000 7fa1d8a00640 1 -- 172.21.15.83:0/1767053109 shutdown_connections 2024-04-24T08:48:07.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:07.360+0000 7fa1d8a00640 1 --2- 172.21.15.83:0/1767053109 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa1d4105e10 0x7fa1d410c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:07.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:07.360+0000 7fa1d8a00640 1 --2- 172.21.15.83:0/1767053109 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1d4105450 0x7fa1d41058d0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:07.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:07.360+0000 7fa1d8a00640 1 --2- 172.21.15.83:0/1767053109 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa1d4104260 0x7fa1d4104680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:07.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:07.360+0000 7fa1d8a00640 1 -- 172.21.15.83:0/1767053109 >> 172.21.15.83:0/1767053109 conn(0x7fa1d40ff7f0 msgr2=0x7fa1d4101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:48:07.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:07.361+0000 7fa1d8a00640 1 -- 172.21.15.83:0/1767053109 shutdown_connections 2024-04-24T08:48:07.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:07.361+0000 7fa1d8a00640 1 -- 172.21.15.83:0/1767053109 wait complete. 2024-04-24T08:48:07.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:07.362+0000 7fa1d8a00640 1 Processor -- start 2024-04-24T08:48:07.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:07.362+0000 7fa1d8a00640 1 -- start start 2024-04-24T08:48:07.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:07.362+0000 7fa1d8a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1d4104260 0x7fa1d419a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:07.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:07.362+0000 7fa1d8a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa1d4105450 0x7fa1d419a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:07.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:07.362+0000 7fa1d8a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa1d4105e10 0x7fa1d41a1a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:07.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:07.362+0000 7fa1d8a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d410fda0 con 0x7fa1d4104260 2024-04-24T08:48:07.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:07.362+0000 7fa1d8a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d410fc20 con 0x7fa1d4105450 2024-04-24T08:48:07.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:07.362+0000 7fa1d8a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d410ff20 con 0x7fa1d4105e10 2024-04-24T08:48:07.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:07.362+0000 7fa1d1e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1d4104260 0x7fa1d419a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:07.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:07.362+0000 7fa1d1e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1d4104260 0x7fa1d419a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58596/0 (socket says 172.21.15.83:58596) 2024-04-24T08:48:07.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:07.362+0000 7fa1d1e00640 1 -- 172.21.15.83:0/2589064159 learned_addr learned my addr 172.21.15.83:0/2589064159 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:48:07.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:07.363+0000 7fa1c9400640 1 --2- 172.21.15.83:0/2589064159 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa1d4105450 0x7fa1d419a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:07.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:07.363+0000 7fa1d2800640 1 --2- 172.21.15.83:0/2589064159 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa1d4105e10 0x7fa1d41a1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:07.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:07.363+0000 7fa1d1e00640 1 -- 172.21.15.83:0/2589064159 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa1d4105e10 msgr2=0x7fa1d41a1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:07.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:07.363+0000 7fa1d1e00640 1 --2- 172.21.15.83:0/2589064159 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa1d4105e10 0x7fa1d41a1a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:07.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:07.363+0000 7fa1d1e00640 1 -- 172.21.15.83:0/2589064159 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa1d4105450 msgr2=0x7fa1d419a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:07.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:07.363+0000 7fa1d1e00640 1 --2- 172.21.15.83:0/2589064159 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa1d4105450 0x7fa1d419a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:07.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:07.363+0000 7fa1d1e00640 1 -- 172.21.15.83:0/2589064159 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1d41a2070 con 0x7fa1d4104260 2024-04-24T08:48:07.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:07.363+0000 7fa1d1e00640 1 --2- 172.21.15.83:0/2589064159 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1d4104260 0x7fa1d419a450 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa1c400ea80 tx=0x7fa1c400ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:48:07.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:07.363+0000 7fa1cb400640 1 -- 172.21.15.83:0/2589064159 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c400ce40 con 0x7fa1d4104260 2024-04-24T08:48:07.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:07.363+0000 7fa1cb400640 1 -- 172.21.15.83:0/2589064159 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa1c4011920 con 0x7fa1d4104260 2024-04-24T08:48:07.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:07.363+0000 7fa1cb400640 1 -- 172.21.15.83:0/2589064159 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c401aaa0 con 0x7fa1d4104260 2024-04-24T08:48:07.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:07.363+0000 7fa1d8a00640 1 -- 172.21.15.83:0/2589064159 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1d41a2360 con 0x7fa1d4104260 2024-04-24T08:48:07.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:07.364+0000 7fa1d8a00640 1 -- 172.21.15.83:0/2589064159 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1d41a2900 con 0x7fa1d4104260 2024-04-24T08:48:07.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:07.365+0000 7fa1d8a00640 1 -- 172.21.15.83:0/2589064159 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1d410b600 con 0x7fa1d4104260 2024-04-24T08:48:07.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:07.367+0000 7fa1cb400640 1 -- 172.21.15.83:0/2589064159 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fa1c4011ad0 con 0x7fa1d4104260 2024-04-24T08:48:07.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:07.368+0000 7fa1cb400640 1 --2- 172.21.15.83:0/2589064159 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fa1b80a3c30 0x7fa1b80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:07.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:07.368+0000 7fa1cb400640 1 -- 172.21.15.83:0/2589064159 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fa1c40d05f0 con 0x7fa1d4104260 2024-04-24T08:48:07.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:07.368+0000 7fa1c9400640 1 --2- 172.21.15.83:0/2589064159 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fa1b80a3c30 0x7fa1b80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:07.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:07.369+0000 7fa1c9400640 1 --2- 172.21.15.83:0/2589064159 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fa1b80a3c30 0x7fa1b80a60f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa1d419b960 tx=0x7fa1bc0314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:48:07.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:07.373+0000 7fa1cb400640 1 -- 172.21.15.83:0/2589064159 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa1c4016030 con 0x7fa1d4104260 2024-04-24T08:48:07.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:07 smithi189 ceph-mon[31791]: mds.? [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] up:boot 2024-04-24T08:48:07.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:07 smithi189 ceph-mon[31791]: fsmap 8 up:standby 2024-04-24T08:48:07.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:07 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-24T08:48:07.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:07 smithi189 ceph-mon[31791]: pgmap v354: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:48:07.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:07.613+0000 7fa1d8a00640 1 -- 172.21.15.83:0/2589064159 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa1d402d620 con 0x7fa1d4104260 2024-04-24T08:48:07.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:07.614+0000 7fa1cb400640 1 -- 172.21.15.83:0/2589064159 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7180 (secure 0 0 0) 0x7fa1c409d3d0 con 0x7fa1d4104260 2024-04-24T08:48:07.615 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:48:07.615 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/358872257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":358872257},{"type":"v1","addr":"172.21.15.83:6835","nonce":358872257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3157070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3157070851},{"type":"v1","addr":"172.21.15.83:6837","nonce":3157070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/1859921310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1859921310},{"type":"v1","addr":"172.21.15.189:6833","nonce":1859921310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24421,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/2241923197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2241923197},{"type":"v1","addr":"172.21.15.189:6835","nonce":2241923197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24439,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/3172311220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":3172311220},{"type":"v1","addr":"172.21.15.189:6837","nonce":3172311220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":34427,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/1157967658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1157967658},{"type":"v1","addr":"172.21.15.138:6833","nonce":1157967658}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34451,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1909219758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1909219758},{"type":"v1","addr":"172.21.15.138:6835","nonce":1909219758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":34493,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/900566754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":900566754},{"type":"v1","addr":"172.21.15.138:6837","nonce":900566754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-24T08:48:07.615 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 9 2024-04-24T08:48:07.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:07.617+0000 7fa1d8a00640 1 -- 172.21.15.83:0/2589064159 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fa1b80a3c30 msgr2=0x7fa1b80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:07.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:07.617+0000 7fa1d8a00640 1 --2- 172.21.15.83:0/2589064159 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fa1b80a3c30 0x7fa1b80a60f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa1d419b960 tx=0x7fa1bc0314a0 comp rx=0 tx=0).stop 2024-04-24T08:48:07.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:07.617+0000 7fa1d8a00640 1 -- 172.21.15.83:0/2589064159 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1d4104260 msgr2=0x7fa1d419a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:07.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:07.617+0000 7fa1d8a00640 1 --2- 172.21.15.83:0/2589064159 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1d4104260 0x7fa1d419a450 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa1c400ea80 tx=0x7fa1c400ef40 comp rx=0 tx=0).stop 2024-04-24T08:48:07.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:07.617+0000 7fa1d8a00640 1 -- 172.21.15.83:0/2589064159 shutdown_connections 2024-04-24T08:48:07.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:07.617+0000 7fa1d8a00640 1 --2- 172.21.15.83:0/2589064159 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa1d4105e10 0x7fa1d41a1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:07.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:07.617+0000 7fa1d8a00640 1 --2- 172.21.15.83:0/2589064159 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa1d4105450 0x7fa1d419a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:07.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:07.617+0000 7fa1d8a00640 1 --2- 172.21.15.83:0/2589064159 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fa1b80a3c30 0x7fa1b80a60f0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:07.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:07.617+0000 7fa1d8a00640 1 --2- 172.21.15.83:0/2589064159 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1d4104260 0x7fa1d419a450 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:07.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:07.617+0000 7fa1d8a00640 1 -- 172.21.15.83:0/2589064159 >> 172.21.15.83:0/2589064159 conn(0x7fa1d40ff7f0 msgr2=0x7fa1d41087d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:48:07.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:07.617+0000 7fa1d8a00640 1 -- 172.21.15.83:0/2589064159 shutdown_connections 2024-04-24T08:48:07.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:07.617+0000 7fa1d8a00640 1 -- 172.21.15.83:0/2589064159 wait complete. 2024-04-24T08:48:08.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:08 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/2589064159' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:48:08.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:08 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/2589064159' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:48:08.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:08 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/2589064159' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:48:08.957 INFO:journalctl@ceph.mds.i.smithi189.stdout:Apr 24 08:48:08 smithi189 systemd[1]: Starting Ceph mds.i for bd3e039e-0215-11ef-bc93-c7b262605968... 2024-04-24T08:48:09.353 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:09 smithi189 ceph-mon[31791]: pgmap v355: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:48:09.354 INFO:journalctl@ceph.mds.i.smithi189.stdout:Apr 24 08:48:09 smithi189 podman[62015]: 2024-04-24 08:48:08.973085501 +0000 UTC m=+0.032666340 image pull b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644 2024-04-24T08:48:09.354 INFO:journalctl@ceph.mds.i.smithi189.stdout:Apr 24 08:48:09 smithi189 podman[62015]: 2024-04-24 08:48:09.092532938 +0000 UTC m=+0.152113810 container create 884f47db48cd847ef4ec1f67d55dd77d43a622e4da92dd7a8da359a9e6bec9d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-i, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, maintainer=Guillaume Abrioux , RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-24T08:48:09.354 INFO:journalctl@ceph.mds.i.smithi189.stdout:Apr 24 08:48:09 smithi189 podman[62015]: 2024-04-24 08:48:09.353779618 +0000 UTC m=+0.413360517 container init 884f47db48cd847ef4ec1f67d55dd77d43a622e4da92dd7a8da359a9e6bec9d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-i, org.label-schema.build-date=20240424, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T08:48:09.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:09 smithi138 ceph-mon[31441]: pgmap v355: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:48:09.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:09 smithi083 ceph-mon[27907]: pgmap v355: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:48:09.636 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T08:48:09.707 INFO:journalctl@ceph.mds.i.smithi189.stdout:Apr 24 08:48:09 smithi189 podman[62015]: 2024-04-24 08:48:09.363812778 +0000 UTC m=+0.423393683 container start 884f47db48cd847ef4ec1f67d55dd77d43a622e4da92dd7a8da359a9e6bec9d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-i, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, ceph=True, RELEASE=rishabh-main-apr17-a654945-1f81fb3, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T08:48:09.708 INFO:journalctl@ceph.mds.i.smithi189.stdout:Apr 24 08:48:09 smithi189 ceph-mds[62035]: -- 172.21.15.189:0/4269129007 <== mon.1 v2:172.21.15.189:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5640d24a7a40 con 0x5640d247ac00 2024-04-24T08:48:09.708 INFO:journalctl@ceph.mds.i.smithi189.stdout:Apr 24 08:48:09 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-i[62031]: starting mds.i at 2024-04-24T08:48:09.708 INFO:journalctl@ceph.mds.i.smithi189.stdout:Apr 24 08:48:09 smithi189 bash[62015]: 884f47db48cd847ef4ec1f67d55dd77d43a622e4da92dd7a8da359a9e6bec9d6 2024-04-24T08:48:09.708 INFO:journalctl@ceph.mds.i.smithi189.stdout:Apr 24 08:48:09 smithi189 systemd[1]: Started Ceph mds.i for bd3e039e-0215-11ef-bc93-c7b262605968. 2024-04-24T08:48:09.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:09.831+0000 7fab0bc00640 1 -- 172.21.15.83:0/4090152266 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab04103c00 msgr2=0x7fab0410a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:09.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:09.831+0000 7fab0bc00640 1 --2- 172.21.15.83:0/4090152266 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab04103c00 0x7fab0410a490 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7faafc009ca0 tx=0x7faafc02a8c0 comp rx=0 tx=0).stop 2024-04-24T08:48:09.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:09.832+0000 7fab0bc00640 1 -- 172.21.15.83:0/4090152266 shutdown_connections 2024-04-24T08:48:09.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:09.832+0000 7fab0bc00640 1 --2- 172.21.15.83:0/4090152266 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab04103c00 0x7fab0410a490 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:09.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:09.832+0000 7fab0bc00640 1 --2- 172.21.15.83:0/4090152266 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fab04103240 0x7fab041036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:09.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:09.832+0000 7fab0bc00640 1 --2- 172.21.15.83:0/4090152266 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fab04102050 0x7fab04102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:09.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:09.832+0000 7fab0bc00640 1 -- 172.21.15.83:0/4090152266 >> 172.21.15.83:0/4090152266 conn(0x7fab040fd5e0 msgr2=0x7fab040ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:48:09.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:09.832+0000 7fab0bc00640 1 -- 172.21.15.83:0/4090152266 shutdown_connections 2024-04-24T08:48:09.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:09.832+0000 7fab0bc00640 1 -- 172.21.15.83:0/4090152266 wait complete. 2024-04-24T08:48:09.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:09.833+0000 7fab0bc00640 1 Processor -- start 2024-04-24T08:48:09.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:09.833+0000 7fab0bc00640 1 -- start start 2024-04-24T08:48:09.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:09.833+0000 7fab0bc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab04102050 0x7fab0419a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:09.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:09.833+0000 7fab0bc00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fab04103240 0x7fab0419a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:09.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:09.833+0000 7fab0bc00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fab04103c00 0x7fab041a1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:09.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:09.833+0000 7fab0bc00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fab0410db90 con 0x7fab04102050 2024-04-24T08:48:09.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:09.833+0000 7fab0bc00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fab0410da10 con 0x7fab04103c00 2024-04-24T08:48:09.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:09.833+0000 7fab0bc00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fab0410dd10 con 0x7fab04103240 2024-04-24T08:48:09.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:09.833+0000 7fab09200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab04102050 0x7fab0419a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:09.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:09.833+0000 7fab09200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab04102050 0x7fab0419a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36602/0 (socket says 172.21.15.83:36602) 2024-04-24T08:48:09.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:09.833+0000 7fab09200640 1 -- 172.21.15.83:0/3244545614 learned_addr learned my addr 172.21.15.83:0/3244545614 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:48:09.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:09.834+0000 7fab03e00640 1 --2- 172.21.15.83:0/3244545614 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fab04103240 0x7fab0419a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:09.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:09.834+0000 7fab09c00640 1 --2- 172.21.15.83:0/3244545614 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fab04103c00 0x7fab041a1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:09.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:09.834+0000 7fab09200640 1 -- 172.21.15.83:0/3244545614 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fab04103240 msgr2=0x7fab0419a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:09.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:09.834+0000 7fab09200640 1 --2- 172.21.15.83:0/3244545614 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fab04103240 0x7fab0419a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:09.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:09.834+0000 7fab09200640 1 -- 172.21.15.83:0/3244545614 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fab04103c00 msgr2=0x7fab041a1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:09.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:09.834+0000 7fab09200640 1 --2- 172.21.15.83:0/3244545614 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fab04103c00 0x7fab041a1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:09.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:09.834+0000 7fab09200640 1 -- 172.21.15.83:0/3244545614 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab041a1fc0 con 0x7fab04102050 2024-04-24T08:48:09.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:09.834+0000 7fab09c00640 1 --2- 172.21.15.83:0/3244545614 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fab04103c00 0x7fab041a1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:48:09.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:09.834+0000 7fab03e00640 1 --2- 172.21.15.83:0/3244545614 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fab04103240 0x7fab0419a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:48:09.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:09.834+0000 7fab09200640 1 --2- 172.21.15.83:0/3244545614 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab04102050 0x7fab0419a3a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7faaf800c970 tx=0x7faaf800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:48:09.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:09.835+0000 7fab01600640 1 -- 172.21.15.83:0/3244545614 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf8004220 con 0x7fab04102050 2024-04-24T08:48:09.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:09.835+0000 7fab01600640 1 -- 172.21.15.83:0/3244545614 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faaf80043c0 con 0x7fab04102050 2024-04-24T08:48:09.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:09.835+0000 7fab0bc00640 1 -- 172.21.15.83:0/3244545614 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab041a22b0 con 0x7fab04102050 2024-04-24T08:48:09.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:09.835+0000 7fab01600640 1 -- 172.21.15.83:0/3244545614 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf800f450 con 0x7fab04102050 2024-04-24T08:48:09.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:09.835+0000 7fab0bc00640 1 -- 172.21.15.83:0/3244545614 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab041a2850 con 0x7fab04102050 2024-04-24T08:48:09.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:09.836+0000 7fab0bc00640 1 -- 172.21.15.83:0/3244545614 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab041a2b10 con 0x7fab04102050 2024-04-24T08:48:09.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:09.838+0000 7fab01600640 1 -- 172.21.15.83:0/3244545614 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7faaf80079b0 con 0x7fab04102050 2024-04-24T08:48:09.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:09.840+0000 7fab01600640 1 --2- 172.21.15.83:0/3244545614 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7faaec0a3c30 0x7faaec0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:09.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:09.840+0000 7fab01600640 1 -- 172.21.15.83:0/3244545614 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7faaf80d0120 con 0x7fab04102050 2024-04-24T08:48:09.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:09.840+0000 7fab03e00640 1 --2- 172.21.15.83:0/3244545614 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7faaec0a3c30 0x7faaec0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:09.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:09.841+0000 7fab03e00640 1 --2- 172.21.15.83:0/3244545614 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7faaec0a3c30 0x7faaec0a60f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fab0419b8b0 tx=0x7faaf000a5c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:48:09.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:09.843+0000 7fab01600640 1 -- 172.21.15.83:0/3244545614 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7faaf8014030 con 0x7fab04102050 2024-04-24T08:48:10.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:10.088+0000 7fab0bc00640 1 -- 172.21.15.83:0/3244545614 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fab041a2ce0 con 0x7fab04102050 2024-04-24T08:48:10.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:10.089+0000 7fab01600640 1 -- 172.21.15.83:0/3244545614 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8010 (secure 0 0 0) 0x7faaf809cf80 con 0x7fab04102050 2024-04-24T08:48:10.089 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:48:10.089 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/358872257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":358872257},{"type":"v1","addr":"172.21.15.83:6835","nonce":358872257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3157070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3157070851},{"type":"v1","addr":"172.21.15.83:6837","nonce":3157070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/1859921310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1859921310},{"type":"v1","addr":"172.21.15.189:6833","nonce":1859921310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24421,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/2241923197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2241923197},{"type":"v1","addr":"172.21.15.189:6835","nonce":2241923197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24439,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/3172311220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":3172311220},{"type":"v1","addr":"172.21.15.189:6837","nonce":3172311220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24454,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/2548965336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":2548965336},{"type":"v1","addr":"172.21.15.189:6839","nonce":2548965336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34427,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/1157967658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1157967658},{"type":"v1","addr":"172.21.15.138:6833","nonce":1157967658}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34451,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1909219758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1909219758},{"type":"v1","addr":"172.21.15.138:6835","nonce":1909219758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":34493,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/900566754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":900566754},{"type":"v1","addr":"172.21.15.138:6837","nonce":900566754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-24T08:48:10.089 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 10 2024-04-24T08:48:10.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:10.092+0000 7fab0bc00640 1 -- 172.21.15.83:0/3244545614 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7faaec0a3c30 msgr2=0x7faaec0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:10.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:10.092+0000 7fab0bc00640 1 --2- 172.21.15.83:0/3244545614 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7faaec0a3c30 0x7faaec0a60f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fab0419b8b0 tx=0x7faaf000a5c0 comp rx=0 tx=0).stop 2024-04-24T08:48:10.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:10.092+0000 7fab0bc00640 1 -- 172.21.15.83:0/3244545614 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab04102050 msgr2=0x7fab0419a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:10.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:10.092+0000 7fab0bc00640 1 --2- 172.21.15.83:0/3244545614 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab04102050 0x7fab0419a3a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7faaf800c970 tx=0x7faaf800ce30 comp rx=0 tx=0).stop 2024-04-24T08:48:10.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:10.092+0000 7fab0bc00640 1 -- 172.21.15.83:0/3244545614 shutdown_connections 2024-04-24T08:48:10.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:10.092+0000 7fab0bc00640 1 --2- 172.21.15.83:0/3244545614 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fab04103c00 0x7fab041a1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:10.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:10.092+0000 7fab0bc00640 1 --2- 172.21.15.83:0/3244545614 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fab04103240 0x7fab0419a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:10.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:10.092+0000 7fab0bc00640 1 --2- 172.21.15.83:0/3244545614 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7faaec0a3c30 0x7faaec0a60f0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:10.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:10.092+0000 7fab0bc00640 1 --2- 172.21.15.83:0/3244545614 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab04102050 0x7fab0419a3a0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:10.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:10.092+0000 7fab0bc00640 1 -- 172.21.15.83:0/3244545614 >> 172.21.15.83:0/3244545614 conn(0x7fab040fd5e0 msgr2=0x7fab04074230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:48:10.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:10.092+0000 7fab0bc00640 1 -- 172.21.15.83:0/3244545614 shutdown_connections 2024-04-24T08:48:10.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:10.092+0000 7fab0bc00640 1 -- 172.21.15.83:0/3244545614 wait complete. 2024-04-24T08:48:10.793 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:10 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:48:10.793 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:10 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:48:10.793 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:10 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:48:10.793 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:10 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T08:48:10.793 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:10 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T08:48:10.793 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:10 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:48:10.794 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:10 smithi083 ceph-mon[27907]: Deploying daemon mds.g on smithi083 2024-04-24T08:48:10.794 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:10 smithi083 ceph-mon[27907]: mds.? [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] up:boot 2024-04-24T08:48:10.794 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:10 smithi083 ceph-mon[27907]: fsmap 9 up:standby 2024-04-24T08:48:10.794 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:10 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-24T08:48:10.794 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:10 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/3244545614' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:48:10.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:10 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:48:10.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:10 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:48:10.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:10 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:48:10.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:10 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T08:48:10.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:10 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T08:48:10.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:10 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:48:10.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:10 smithi138 ceph-mon[31441]: Deploying daemon mds.g on smithi083 2024-04-24T08:48:10.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:10 smithi138 ceph-mon[31441]: mds.? [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] up:boot 2024-04-24T08:48:10.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:10 smithi138 ceph-mon[31441]: fsmap 9 up:standby 2024-04-24T08:48:10.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:10 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-24T08:48:10.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:10 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/3244545614' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:48:10.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:10 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:48:10.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:10 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:48:10.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:10 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:48:10.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:10 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T08:48:10.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:10 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T08:48:10.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:10 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:48:10.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:10 smithi189 ceph-mon[31791]: Deploying daemon mds.g on smithi083 2024-04-24T08:48:10.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:10 smithi189 ceph-mon[31791]: mds.? [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] up:boot 2024-04-24T08:48:10.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:10 smithi189 ceph-mon[31791]: fsmap 9 up:standby 2024-04-24T08:48:10.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:10 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-24T08:48:10.959 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:10 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/3244545614' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:48:11.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:11 smithi083 ceph-mon[27907]: pgmap v356: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:48:11.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:11 smithi138 ceph-mon[31441]: pgmap v356: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:48:11.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:11 smithi189 ceph-mon[31791]: pgmap v356: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:48:12.111 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T08:48:12.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:12.513+0000 7f8291400640 1 -- 172.21.15.83:0/503982617 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f828c0769c0 msgr2=0x7f828c076de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:12.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:12.513+0000 7f8291400640 1 --2- 172.21.15.83:0/503982617 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f828c0769c0 0x7f828c076de0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f827c009900 tx=0x7f827c02a410 comp rx=0 tx=0).stop 2024-04-24T08:48:12.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:12.514+0000 7f8291400640 1 -- 172.21.15.83:0/503982617 shutdown_connections 2024-04-24T08:48:12.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:12.514+0000 7f8291400640 1 --2- 172.21.15.83:0/503982617 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f828c0785a0 0x7f828c078e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:12.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:12.514+0000 7f8291400640 1 --2- 172.21.15.83:0/503982617 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f828c077c00 0x7f828c078060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:12.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:12.514+0000 7f8291400640 1 --2- 172.21.15.83:0/503982617 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f828c0769c0 0x7f828c076de0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:12.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:12.514+0000 7f8291400640 1 -- 172.21.15.83:0/503982617 >> 172.21.15.83:0/503982617 conn(0x7f828c0ff7f0 msgr2=0x7f828c101c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:48:12.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:12.514+0000 7f8291400640 1 -- 172.21.15.83:0/503982617 shutdown_connections 2024-04-24T08:48:12.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:12.514+0000 7f8291400640 1 -- 172.21.15.83:0/503982617 wait complete. 2024-04-24T08:48:12.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:12.516+0000 7f8291400640 1 Processor -- start 2024-04-24T08:48:12.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:12.516+0000 7f8291400640 1 -- start start 2024-04-24T08:48:12.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:12.516+0000 7f8291400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f828c077c00 0x7f828c19eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:12.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:12.516+0000 7f8291400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f828c0785a0 0x7f828c19f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:12.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:12.516+0000 7f8291400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f828c1a4080 0x7f828c1a6470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:12.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:12.516+0000 7f8291400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f828c1141c0 con 0x7f828c077c00 2024-04-24T08:48:12.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:12.516+0000 7f8291400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f828c114040 con 0x7f828c0785a0 2024-04-24T08:48:12.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:12.516+0000 7f8291400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f828c114340 con 0x7f828c1a4080 2024-04-24T08:48:12.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:12.517+0000 7f828a800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f828c077c00 0x7f828c19eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:12.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:12.517+0000 7f828a800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f828c077c00 0x7f828c19eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36634/0 (socket says 172.21.15.83:36634) 2024-04-24T08:48:12.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:12.517+0000 7f828a800640 1 -- 172.21.15.83:0/2979252368 learned_addr learned my addr 172.21.15.83:0/2979252368 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:48:12.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:12.517+0000 7f828b200640 1 --2- 172.21.15.83:0/2979252368 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f828c1a4080 0x7f828c1a6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:12.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:12.517+0000 7f8289e00640 1 --2- 172.21.15.83:0/2979252368 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f828c0785a0 0x7f828c19f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:12.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:12.517+0000 7f828a800640 1 -- 172.21.15.83:0/2979252368 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f828c1a4080 msgr2=0x7f828c1a6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:12.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:12.517+0000 7f828a800640 1 --2- 172.21.15.83:0/2979252368 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f828c1a4080 0x7f828c1a6470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:12.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:12.517+0000 7f828a800640 1 -- 172.21.15.83:0/2979252368 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f828c0785a0 msgr2=0x7f828c19f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:12.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:12.517+0000 7f828a800640 1 --2- 172.21.15.83:0/2979252368 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f828c0785a0 0x7f828c19f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:12.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:12.517+0000 7f828a800640 1 -- 172.21.15.83:0/2979252368 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f828c1a6ae0 con 0x7f828c077c00 2024-04-24T08:48:12.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:12.517+0000 7f828a800640 1 --2- 172.21.15.83:0/2979252368 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f828c077c00 0x7f828c19eae0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f827c009230 tx=0x7f827c004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:48:12.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:12.518+0000 7f8283400640 1 -- 172.21.15.83:0/2979252368 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f827c03f030 con 0x7f828c077c00 2024-04-24T08:48:12.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:12.518+0000 7f8283400640 1 -- 172.21.15.83:0/2979252368 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f827c003cb0 con 0x7f828c077c00 2024-04-24T08:48:12.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:12.518+0000 7f8291400640 1 -- 172.21.15.83:0/2979252368 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f828c1a6d70 con 0x7f828c077c00 2024-04-24T08:48:12.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:12.518+0000 7f8283400640 1 -- 172.21.15.83:0/2979252368 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f827c035070 con 0x7f828c077c00 2024-04-24T08:48:12.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:12.518+0000 7f8291400640 1 -- 172.21.15.83:0/2979252368 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f828c1a71b0 con 0x7f828c077c00 2024-04-24T08:48:12.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:12.519+0000 7f8291400640 1 -- 172.21.15.83:0/2979252368 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f828c110ec0 con 0x7f828c077c00 2024-04-24T08:48:12.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:12.521+0000 7f8283400640 1 -- 172.21.15.83:0/2979252368 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f827c042030 con 0x7f828c077c00 2024-04-24T08:48:12.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:12.522+0000 7f8283400640 1 --2- 172.21.15.83:0/2979252368 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f82700a3bb0 0x7f82700a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:12.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:12.522+0000 7f8283400640 1 -- 172.21.15.83:0/2979252368 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f827c0f0530 con 0x7f828c077c00 2024-04-24T08:48:12.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:12.523+0000 7f8289e00640 1 --2- 172.21.15.83:0/2979252368 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f82700a3bb0 0x7f82700a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:12.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:12.523+0000 7f8289e00640 1 --2- 172.21.15.83:0/2979252368 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f82700a3bb0 0x7f82700a6070 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8278005fd0 tx=0x7f8278005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:48:12.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:12.527+0000 7f8283400640 1 -- 172.21.15.83:0/2979252368 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f827c049030 con 0x7f828c077c00 2024-04-24T08:48:12.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:12.763+0000 7f8291400640 1 -- 172.21.15.83:0/2979252368 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f828c02d620 con 0x7f828c077c00 2024-04-24T08:48:12.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:12.764+0000 7f8283400640 1 -- 172.21.15.83:0/2979252368 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8010 (secure 0 0 0) 0x7f827c0bd310 con 0x7f828c077c00 2024-04-24T08:48:12.764 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:48:12.764 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/358872257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":358872257},{"type":"v1","addr":"172.21.15.83:6835","nonce":358872257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3157070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3157070851},{"type":"v1","addr":"172.21.15.83:6837","nonce":3157070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/1859921310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1859921310},{"type":"v1","addr":"172.21.15.189:6833","nonce":1859921310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24421,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/2241923197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2241923197},{"type":"v1","addr":"172.21.15.189:6835","nonce":2241923197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24439,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/3172311220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":3172311220},{"type":"v1","addr":"172.21.15.189:6837","nonce":3172311220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24454,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/2548965336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":2548965336},{"type":"v1","addr":"172.21.15.189:6839","nonce":2548965336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34427,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/1157967658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1157967658},{"type":"v1","addr":"172.21.15.138:6833","nonce":1157967658}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34451,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1909219758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1909219758},{"type":"v1","addr":"172.21.15.138:6835","nonce":1909219758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":34493,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/900566754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":900566754},{"type":"v1","addr":"172.21.15.138:6837","nonce":900566754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-24T08:48:12.764 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 10 2024-04-24T08:48:12.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:12.767+0000 7f8291400640 1 -- 172.21.15.83:0/2979252368 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f82700a3bb0 msgr2=0x7f82700a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:12.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:12.767+0000 7f8291400640 1 --2- 172.21.15.83:0/2979252368 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f82700a3bb0 0x7f82700a6070 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8278005fd0 tx=0x7f8278005850 comp rx=0 tx=0).stop 2024-04-24T08:48:12.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:12.767+0000 7f8291400640 1 -- 172.21.15.83:0/2979252368 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f828c077c00 msgr2=0x7f828c19eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:12.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:12.767+0000 7f8291400640 1 --2- 172.21.15.83:0/2979252368 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f828c077c00 0x7f828c19eae0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f827c009230 tx=0x7f827c004360 comp rx=0 tx=0).stop 2024-04-24T08:48:12.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:12.767+0000 7f8291400640 1 -- 172.21.15.83:0/2979252368 shutdown_connections 2024-04-24T08:48:12.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:12.767+0000 7f8291400640 1 --2- 172.21.15.83:0/2979252368 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f828c1a4080 0x7f828c1a6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:12.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:12.767+0000 7f8291400640 1 --2- 172.21.15.83:0/2979252368 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f828c0785a0 0x7f828c19f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:12.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:12.767+0000 7f8291400640 1 --2- 172.21.15.83:0/2979252368 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f82700a3bb0 0x7f82700a6070 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:12.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:12.767+0000 7f8291400640 1 --2- 172.21.15.83:0/2979252368 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f828c077c00 0x7f828c19eae0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:12.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:12.767+0000 7f8291400640 1 -- 172.21.15.83:0/2979252368 >> 172.21.15.83:0/2979252368 conn(0x7f828c0ff7f0 msgr2=0x7f828c1010a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:48:12.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:12.767+0000 7f8291400640 1 -- 172.21.15.83:0/2979252368 shutdown_connections 2024-04-24T08:48:12.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:12.767+0000 7f8291400640 1 -- 172.21.15.83:0/2979252368 wait complete. 2024-04-24T08:48:13.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:12 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/2979252368' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:48:13.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:12 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/2979252368' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:48:13.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:12 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/2979252368' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:48:13.313 INFO:journalctl@ceph.mds.g.smithi083.stdout:Apr 24 08:48:13 smithi083 podman[69421]: 2024-04-24 08:48:13.090505421 +0000 UTC m=+0.157441599 container create 4cd70b5367890a5992e77e603132ad8de365154ac5953b8e9c442b164a921f42 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=rishabh-main-apr17-a654945-1f81fb3, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-24T08:48:13.804 INFO:journalctl@ceph.mds.g.smithi083.stdout:Apr 24 08:48:13 smithi083 podman[69421]: 2024-04-24 08:48:13.419990817 +0000 UTC m=+0.486927038 container init 4cd70b5367890a5992e77e603132ad8de365154ac5953b8e9c442b164a921f42 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-g, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, RELEASE=rishabh-main-apr17-a654945-1f81fb3, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-04-24T08:48:13.804 INFO:journalctl@ceph.mds.g.smithi083.stdout:Apr 24 08:48:13 smithi083 podman[69421]: 2024-04-24 08:48:13.430218312 +0000 UTC m=+0.497154496 container start 4cd70b5367890a5992e77e603132ad8de365154ac5953b8e9c442b164a921f42 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-g, ceph=True, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-04-24T08:48:13.804 INFO:journalctl@ceph.mds.g.smithi083.stdout:Apr 24 08:48:13 smithi083 ceph-mds[69440]: -- 172.21.15.83:0/1890440703 <== mon.0 v2:172.21.15.83:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56449583da40 con 0x564495810800 2024-04-24T08:48:13.804 INFO:journalctl@ceph.mds.g.smithi083.stdout:Apr 24 08:48:13 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-g[69436]: starting mds.g at 2024-04-24T08:48:13.804 INFO:journalctl@ceph.mds.g.smithi083.stdout:Apr 24 08:48:13 smithi083 bash[69421]: 4cd70b5367890a5992e77e603132ad8de365154ac5953b8e9c442b164a921f42 2024-04-24T08:48:13.804 INFO:journalctl@ceph.mds.g.smithi083.stdout:Apr 24 08:48:13 smithi083 systemd[1]: Started Ceph mds.g for bd3e039e-0215-11ef-bc93-c7b262605968. 2024-04-24T08:48:14.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:13 smithi138 ceph-mon[31441]: pgmap v357: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:48:14.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:13 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:48:14.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:13 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:48:14.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:13 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:48:14.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:13 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T08:48:14.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:13 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T08:48:14.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:13 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:48:14.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:13 smithi189 ceph-mon[31791]: pgmap v357: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:48:14.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:13 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:48:14.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:13 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:48:14.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:13 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:48:14.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:13 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T08:48:14.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:13 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T08:48:14.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:13 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:48:14.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:13 smithi083 ceph-mon[27907]: pgmap v357: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:48:14.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:13 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:48:14.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:13 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:48:14.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:13 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:48:14.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:13 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T08:48:14.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:13 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T08:48:14.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:13 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:48:14.785 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T08:48:15.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:15.006+0000 7f3e42400640 1 -- 172.21.15.83:0/1996101101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e3c104260 msgr2=0x7f3e3c104680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:15.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:15.006+0000 7f3e42400640 1 --2- 172.21.15.83:0/1996101101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e3c104260 0x7f3e3c104680 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c009900 tx=0x7f3e2c02a3f0 comp rx=0 tx=0).stop 2024-04-24T08:48:15.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:15.006+0000 7f3e42400640 1 -- 172.21.15.83:0/1996101101 shutdown_connections 2024-04-24T08:48:15.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:15.006+0000 7f3e42400640 1 --2- 172.21.15.83:0/1996101101 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3e3c105e10 0x7f3e3c10c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:15.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:15.006+0000 7f3e42400640 1 --2- 172.21.15.83:0/1996101101 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3e3c105450 0x7f3e3c1058d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:15.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:15.006+0000 7f3e42400640 1 --2- 172.21.15.83:0/1996101101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e3c104260 0x7f3e3c104680 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:15.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:15.006+0000 7f3e42400640 1 -- 172.21.15.83:0/1996101101 >> 172.21.15.83:0/1996101101 conn(0x7f3e3c0ff7f0 msgr2=0x7f3e3c101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:48:15.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:15.007+0000 7f3e42400640 1 -- 172.21.15.83:0/1996101101 shutdown_connections 2024-04-24T08:48:15.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:15.007+0000 7f3e42400640 1 -- 172.21.15.83:0/1996101101 wait complete. 2024-04-24T08:48:15.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:15.008+0000 7f3e42400640 1 Processor -- start 2024-04-24T08:48:15.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:15.008+0000 7f3e42400640 1 -- start start 2024-04-24T08:48:15.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:15.008+0000 7f3e42400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3e3c104260 0x7f3e3c19a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:15.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:15.008+0000 7f3e42400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e3c105450 0x7f3e3c19aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:15.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:15.008+0000 7f3e42400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3e3c105e10 0x7f3e3c1a1a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:15.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:15.008+0000 7f3e42400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3e3c10fdf0 con 0x7f3e3c105450 2024-04-24T08:48:15.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:15.008+0000 7f3e42400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3e3c10fc70 con 0x7f3e3c105e10 2024-04-24T08:48:15.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:15.008+0000 7f3e42400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3e3c10ff70 con 0x7f3e3c104260 2024-04-24T08:48:15.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:15.009+0000 7f3e3aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e3c105450 0x7f3e3c19aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:15.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:15.009+0000 7f3e3aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e3c105450 0x7f3e3c19aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36656/0 (socket says 172.21.15.83:36656) 2024-04-24T08:48:15.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:15.009+0000 7f3e3aa00640 1 -- 172.21.15.83:0/2193297271 learned_addr learned my addr 172.21.15.83:0/2193297271 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:48:15.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:15.009+0000 7f3e3b400640 1 --2- 172.21.15.83:0/2193297271 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3e3c104260 0x7f3e3c19a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:15.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:15.009+0000 7f3e3be00640 1 --2- 172.21.15.83:0/2193297271 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3e3c105e10 0x7f3e3c1a1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:15.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:15.009+0000 7f3e3aa00640 1 -- 172.21.15.83:0/2193297271 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3e3c104260 msgr2=0x7f3e3c19a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:15.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:15.009+0000 7f3e3aa00640 1 --2- 172.21.15.83:0/2193297271 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3e3c104260 0x7f3e3c19a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:15.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:15.009+0000 7f3e3aa00640 1 -- 172.21.15.83:0/2193297271 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3e3c105e10 msgr2=0x7f3e3c1a1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:15.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:15.009+0000 7f3e3aa00640 1 --2- 172.21.15.83:0/2193297271 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3e3c105e10 0x7f3e3c1a1a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:15.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:15.009+0000 7f3e3aa00640 1 -- 172.21.15.83:0/2193297271 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e3c1a2100 con 0x7f3e3c105450 2024-04-24T08:48:15.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:15.009+0000 7f3e3be00640 1 --2- 172.21.15.83:0/2193297271 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3e3c105e10 0x7f3e3c1a1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:48:15.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:15.010+0000 7f3e3aa00640 1 --2- 172.21.15.83:0/2193297271 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e3c105450 0x7f3e3c19aa20 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f3e2800c890 tx=0x7f3e2800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:48:15.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:15.010+0000 7f3e33e00640 1 -- 172.21.15.83:0/2193297271 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e28004220 con 0x7f3e3c105450 2024-04-24T08:48:15.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:15.010+0000 7f3e33e00640 1 -- 172.21.15.83:0/2193297271 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e280043c0 con 0x7f3e3c105450 2024-04-24T08:48:15.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:15.010+0000 7f3e42400640 1 -- 172.21.15.83:0/2193297271 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e3c1a23f0 con 0x7f3e3c105450 2024-04-24T08:48:15.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:15.010+0000 7f3e33e00640 1 -- 172.21.15.83:0/2193297271 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e28004a00 con 0x7f3e3c105450 2024-04-24T08:48:15.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:15.010+0000 7f3e42400640 1 -- 172.21.15.83:0/2193297271 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e3c1a2940 con 0x7f3e3c105450 2024-04-24T08:48:15.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:15.011+0000 7f3e42400640 1 -- 172.21.15.83:0/2193297271 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e3c10b5f0 con 0x7f3e3c105450 2024-04-24T08:48:15.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:15.016+0000 7f3e33e00640 1 -- 172.21.15.83:0/2193297271 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f3e28004ba0 con 0x7f3e3c105450 2024-04-24T08:48:15.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:15.018+0000 7f3e33e00640 1 --2- 172.21.15.83:0/2193297271 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f3e200a3bb0 0x7f3e200a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:15.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:15.018+0000 7f3e33e00640 1 -- 172.21.15.83:0/2193297271 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f3e280d00d0 con 0x7f3e3c105450 2024-04-24T08:48:15.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:15.018+0000 7f3e3b400640 1 --2- 172.21.15.83:0/2193297271 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f3e200a3bb0 0x7f3e200a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:15.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:15.019+0000 7f3e3b400640 1 --2- 172.21.15.83:0/2193297271 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f3e200a3bb0 0x7f3e200a6070 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c006010 tx=0x7f3e2c0314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:48:15.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:15.020+0000 7f3e33e00640 1 -- 172.21.15.83:0/2193297271 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3e28014030 con 0x7f3e3c105450 2024-04-24T08:48:15.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:14 smithi083 ceph-mon[27907]: Deploying daemon mds.a on smithi083 2024-04-24T08:48:15.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:14 smithi083 ceph-mon[27907]: mds.? [v2:172.21.15.83:6838/4085470844,v1:172.21.15.83:6839/4085470844] up:boot 2024-04-24T08:48:15.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:14 smithi083 ceph-mon[27907]: fsmap 10 up:standby 2024-04-24T08:48:15.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:14 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-24T08:48:15.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:14 smithi138 ceph-mon[31441]: Deploying daemon mds.a on smithi083 2024-04-24T08:48:15.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:14 smithi138 ceph-mon[31441]: mds.? [v2:172.21.15.83:6838/4085470844,v1:172.21.15.83:6839/4085470844] up:boot 2024-04-24T08:48:15.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:14 smithi138 ceph-mon[31441]: fsmap 10 up:standby 2024-04-24T08:48:15.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:14 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-24T08:48:15.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:14 smithi189 ceph-mon[31791]: Deploying daemon mds.a on smithi083 2024-04-24T08:48:15.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:14 smithi189 ceph-mon[31791]: mds.? [v2:172.21.15.83:6838/4085470844,v1:172.21.15.83:6839/4085470844] up:boot 2024-04-24T08:48:15.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:14 smithi189 ceph-mon[31791]: fsmap 10 up:standby 2024-04-24T08:48:15.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:14 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-24T08:48:15.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:15.272+0000 7f3e42400640 1 -- 172.21.15.83:0/2193297271 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3e3c02d870 con 0x7f3e3c105450 2024-04-24T08:48:15.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:15.274+0000 7f3e33e00640 1 -- 172.21.15.83:0/2193297271 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8836 (secure 0 0 0) 0x7f3e2809cf50 con 0x7f3e3c105450 2024-04-24T08:48:15.275 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:48:15.275 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/358872257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":358872257},{"type":"v1","addr":"172.21.15.83:6835","nonce":358872257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3157070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3157070851},{"type":"v1","addr":"172.21.15.83:6837","nonce":3157070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14658,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6839/4085470844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":4085470844},{"type":"v1","addr":"172.21.15.83:6839","nonce":4085470844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/1859921310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1859921310},{"type":"v1","addr":"172.21.15.189:6833","nonce":1859921310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24421,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/2241923197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2241923197},{"type":"v1","addr":"172.21.15.189:6835","nonce":2241923197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24439,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/3172311220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":3172311220},{"type":"v1","addr":"172.21.15.189:6837","nonce":3172311220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24454,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/2548965336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":2548965336},{"type":"v1","addr":"172.21.15.189:6839","nonce":2548965336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34427,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/1157967658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1157967658},{"type":"v1","addr":"172.21.15.138:6833","nonce":1157967658}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34451,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1909219758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1909219758},{"type":"v1","addr":"172.21.15.138:6835","nonce":1909219758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":34493,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/900566754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":900566754},{"type":"v1","addr":"172.21.15.138:6837","nonce":900566754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-24T08:48:15.276 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 11 2024-04-24T08:48:15.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:15.276+0000 7f3e42400640 1 -- 172.21.15.83:0/2193297271 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f3e200a3bb0 msgr2=0x7f3e200a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:15.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:15.276+0000 7f3e42400640 1 --2- 172.21.15.83:0/2193297271 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f3e200a3bb0 0x7f3e200a6070 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c006010 tx=0x7f3e2c0314a0 comp rx=0 tx=0).stop 2024-04-24T08:48:15.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:15.276+0000 7f3e42400640 1 -- 172.21.15.83:0/2193297271 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e3c105450 msgr2=0x7f3e3c19aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:15.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:15.276+0000 7f3e42400640 1 --2- 172.21.15.83:0/2193297271 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e3c105450 0x7f3e3c19aa20 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f3e2800c890 tx=0x7f3e2800cd50 comp rx=0 tx=0).stop 2024-04-24T08:48:15.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:15.276+0000 7f3e42400640 1 -- 172.21.15.83:0/2193297271 shutdown_connections 2024-04-24T08:48:15.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:15.276+0000 7f3e42400640 1 --2- 172.21.15.83:0/2193297271 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3e3c105e10 0x7f3e3c1a1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:15.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:15.276+0000 7f3e42400640 1 --2- 172.21.15.83:0/2193297271 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f3e200a3bb0 0x7f3e200a6070 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:15.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:15.276+0000 7f3e42400640 1 --2- 172.21.15.83:0/2193297271 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e3c105450 0x7f3e3c19aa20 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:15.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:15.276+0000 7f3e42400640 1 --2- 172.21.15.83:0/2193297271 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3e3c104260 0x7f3e3c19a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:15.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:15.276+0000 7f3e42400640 1 -- 172.21.15.83:0/2193297271 >> 172.21.15.83:0/2193297271 conn(0x7f3e3c0ff7f0 msgr2=0x7f3e3c1087c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:48:15.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:15.277+0000 7f3e42400640 1 -- 172.21.15.83:0/2193297271 shutdown_connections 2024-04-24T08:48:15.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:15.277+0000 7f3e42400640 1 -- 172.21.15.83:0/2193297271 wait complete. 2024-04-24T08:48:16.170 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:15 smithi083 ceph-mon[27907]: pgmap v358: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:48:16.171 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:15 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/2193297271' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:48:16.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:15 smithi138 ceph-mon[31441]: pgmap v358: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:48:16.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:15 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/2193297271' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:48:16.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:15 smithi189 ceph-mon[31791]: pgmap v358: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:48:16.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:15 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/2193297271' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:48:17.297 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T08:48:17.307 INFO:journalctl@ceph.mds.a.smithi083.stdout:Apr 24 08:48:17 smithi083 systemd[1]: Starting Ceph mds.a for bd3e039e-0215-11ef-bc93-c7b262605968... 2024-04-24T08:48:17.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:17.492+0000 7ff80e000640 1 -- 172.21.15.83:0/2609494386 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff808105e10 msgr2=0x7ff80810c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:17.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:17.492+0000 7ff80e000640 1 --2- 172.21.15.83:0/2609494386 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff808105e10 0x7ff80810c6a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff800009ca0 tx=0x7ff80002a8c0 comp rx=0 tx=0).stop 2024-04-24T08:48:17.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:17.492+0000 7ff80e000640 1 -- 172.21.15.83:0/2609494386 shutdown_connections 2024-04-24T08:48:17.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:17.492+0000 7ff80e000640 1 --2- 172.21.15.83:0/2609494386 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff808105e10 0x7ff80810c6a0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:17.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:17.492+0000 7ff80e000640 1 --2- 172.21.15.83:0/2609494386 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff808105450 0x7ff8081058d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:17.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:17.492+0000 7ff80e000640 1 --2- 172.21.15.83:0/2609494386 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff808104260 0x7ff808104680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:17.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:17.492+0000 7ff80e000640 1 -- 172.21.15.83:0/2609494386 >> 172.21.15.83:0/2609494386 conn(0x7ff8080ff7f0 msgr2=0x7ff808101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:48:17.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:17.493+0000 7ff80e000640 1 -- 172.21.15.83:0/2609494386 shutdown_connections 2024-04-24T08:48:17.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:17.493+0000 7ff80e000640 1 -- 172.21.15.83:0/2609494386 wait complete. 2024-04-24T08:48:17.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:17.493+0000 7ff80e000640 1 Processor -- start 2024-04-24T08:48:17.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:17.494+0000 7ff80e000640 1 -- start start 2024-04-24T08:48:17.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:17.494+0000 7ff80e000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff808104260 0x7ff80819a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:17.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:17.494+0000 7ff80e000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff808105450 0x7ff80819a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:17.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:17.494+0000 7ff80e000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff808105e10 0x7ff8081a1a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:17.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:17.494+0000 7ff80e000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff80810fdf0 con 0x7ff808105450 2024-04-24T08:48:17.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:17.494+0000 7ff80e000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff80810fc70 con 0x7ff808105e10 2024-04-24T08:48:17.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:17.494+0000 7ff80e000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff80810ff70 con 0x7ff808104260 2024-04-24T08:48:17.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:17.494+0000 7ff806a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff808105450 0x7ff80819a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:17.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:17.494+0000 7ff806a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff808105450 0x7ff80819a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36682/0 (socket says 172.21.15.83:36682) 2024-04-24T08:48:17.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:17.494+0000 7ff806a00640 1 -- 172.21.15.83:0/3744367920 learned_addr learned my addr 172.21.15.83:0/3744367920 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:48:17.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:17.495+0000 7ff807400640 1 --2- 172.21.15.83:0/3744367920 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff808104260 0x7ff80819a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:17.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:17.495+0000 7ff806a00640 1 -- 172.21.15.83:0/3744367920 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff808104260 msgr2=0x7ff80819a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:17.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:17.495+0000 7ff806a00640 1 --2- 172.21.15.83:0/3744367920 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff808104260 0x7ff80819a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:17.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:17.495+0000 7ff806a00640 1 -- 172.21.15.83:0/3744367920 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff808105e10 msgr2=0x7ff8081a1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:17.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:17.495+0000 7ff807e00640 1 --2- 172.21.15.83:0/3744367920 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff808105e10 0x7ff8081a1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:17.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:17.495+0000 7ff806a00640 1 --2- 172.21.15.83:0/3744367920 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff808105e10 0x7ff8081a1a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:17.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:17.495+0000 7ff806a00640 1 -- 172.21.15.83:0/3744367920 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8081a20a0 con 0x7ff808105450 2024-04-24T08:48:17.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:17.495+0000 7ff806a00640 1 --2- 172.21.15.83:0/3744367920 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff808105450 0x7ff80819a9c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ff7f400c970 tx=0x7ff7f400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:48:17.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:17.495+0000 7ff7ffe00640 1 -- 172.21.15.83:0/3744367920 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7f4007bb0 con 0x7ff808105450 2024-04-24T08:48:17.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:17.495+0000 7ff7ffe00640 1 -- 172.21.15.83:0/3744367920 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff7f4007d50 con 0x7ff808105450 2024-04-24T08:48:17.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:17.495+0000 7ff7ffe00640 1 -- 172.21.15.83:0/3744367920 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7f400f450 con 0x7ff808105450 2024-04-24T08:48:17.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:17.495+0000 7ff80e000640 1 -- 172.21.15.83:0/3744367920 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8081a2390 con 0x7ff808105450 2024-04-24T08:48:17.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:17.495+0000 7ff80e000640 1 -- 172.21.15.83:0/3744367920 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8081a28e0 con 0x7ff808105450 2024-04-24T08:48:17.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:17.497+0000 7ff80e000640 1 -- 172.21.15.83:0/3744367920 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff80810b520 con 0x7ff808105450 2024-04-24T08:48:17.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:17.499+0000 7ff7ffe00640 1 -- 172.21.15.83:0/3744367920 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ff7f4004f20 con 0x7ff808105450 2024-04-24T08:48:17.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:17.500+0000 7ff7ffe00640 1 --2- 172.21.15.83:0/3744367920 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7ff7ec0a3c30 0x7ff7ec0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:17.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:17.501+0000 7ff7ffe00640 1 -- 172.21.15.83:0/3744367920 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7ff7f40cfdb0 con 0x7ff808105450 2024-04-24T08:48:17.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:17.501+0000 7ff807400640 1 --2- 172.21.15.83:0/3744367920 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7ff7ec0a3c30 0x7ff7ec0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:17.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:17.501+0000 7ff807400640 1 --2- 172.21.15.83:0/3744367920 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7ff7ec0a3c30 0x7ff7ec0a60f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff7f8005db0 tx=0x7ff7f8005d40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:48:17.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:17.505+0000 7ff7ffe00640 1 -- 172.21.15.83:0/3744367920 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff7f40d4070 con 0x7ff808105450 2024-04-24T08:48:17.711 INFO:journalctl@ceph.mds.a.smithi083.stdout:Apr 24 08:48:17 smithi083 podman[69767]: 2024-04-24 08:48:17.268977708 +0000 UTC m=+0.032655639 image pull b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644 2024-04-24T08:48:17.711 INFO:journalctl@ceph.mds.a.smithi083.stdout:Apr 24 08:48:17 smithi083 podman[69767]: 2024-04-24 08:48:17.421695664 +0000 UTC m=+0.185373631 container create 0f9ba266ef2dc6cf0a25f4b10ed3f8bbcbb7c4a3b691281151f641af9b3b4d87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=rishabh-main-apr17-a654945-1f81fb3) 2024-04-24T08:48:17.711 INFO:journalctl@ceph.mds.a.smithi083.stdout:Apr 24 08:48:17 smithi083 podman[69767]: 2024-04-24 08:48:17.710942124 +0000 UTC m=+0.474620098 container init 0f9ba266ef2dc6cf0a25f4b10ed3f8bbcbb7c4a3b691281151f641af9b3b4d87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-a, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-24T08:48:17.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:17.760+0000 7ff80e000640 1 -- 172.21.15.83:0/3744367920 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff80802d620 con 0x7ff808105450 2024-04-24T08:48:17.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:17.761+0000 7ff7ffe00640 1 -- 172.21.15.83:0/3744367920 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8836 (secure 0 0 0) 0x7ff7f4014030 con 0x7ff808105450 2024-04-24T08:48:17.761 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:48:17.761 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/358872257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":358872257},{"type":"v1","addr":"172.21.15.83:6835","nonce":358872257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3157070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3157070851},{"type":"v1","addr":"172.21.15.83:6837","nonce":3157070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14658,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6839/4085470844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":4085470844},{"type":"v1","addr":"172.21.15.83:6839","nonce":4085470844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/1859921310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1859921310},{"type":"v1","addr":"172.21.15.189:6833","nonce":1859921310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24421,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/2241923197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2241923197},{"type":"v1","addr":"172.21.15.189:6835","nonce":2241923197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24439,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/3172311220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":3172311220},{"type":"v1","addr":"172.21.15.189:6837","nonce":3172311220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24454,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/2548965336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":2548965336},{"type":"v1","addr":"172.21.15.189:6839","nonce":2548965336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34427,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/1157967658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1157967658},{"type":"v1","addr":"172.21.15.138:6833","nonce":1157967658}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34451,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1909219758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1909219758},{"type":"v1","addr":"172.21.15.138:6835","nonce":1909219758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":34493,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/900566754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":900566754},{"type":"v1","addr":"172.21.15.138:6837","nonce":900566754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-24T08:48:17.761 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 11 2024-04-24T08:48:17.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:17.764+0000 7ff80e000640 1 -- 172.21.15.83:0/3744367920 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7ff7ec0a3c30 msgr2=0x7ff7ec0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:17.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:17.764+0000 7ff80e000640 1 --2- 172.21.15.83:0/3744367920 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7ff7ec0a3c30 0x7ff7ec0a60f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff7f8005db0 tx=0x7ff7f8005d40 comp rx=0 tx=0).stop 2024-04-24T08:48:17.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:17.764+0000 7ff80e000640 1 -- 172.21.15.83:0/3744367920 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff808105450 msgr2=0x7ff80819a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:17.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:17.764+0000 7ff80e000640 1 --2- 172.21.15.83:0/3744367920 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff808105450 0x7ff80819a9c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ff7f400c970 tx=0x7ff7f400ce30 comp rx=0 tx=0).stop 2024-04-24T08:48:17.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:17.764+0000 7ff80e000640 1 -- 172.21.15.83:0/3744367920 shutdown_connections 2024-04-24T08:48:17.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:17.764+0000 7ff80e000640 1 --2- 172.21.15.83:0/3744367920 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff808105e10 0x7ff8081a1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:17.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:17.764+0000 7ff80e000640 1 --2- 172.21.15.83:0/3744367920 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7ff7ec0a3c30 0x7ff7ec0a60f0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:17.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:17.764+0000 7ff80e000640 1 --2- 172.21.15.83:0/3744367920 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff808105450 0x7ff80819a9c0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:17.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:17.764+0000 7ff80e000640 1 --2- 172.21.15.83:0/3744367920 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff808104260 0x7ff80819a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:17.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:17.764+0000 7ff80e000640 1 -- 172.21.15.83:0/3744367920 >> 172.21.15.83:0/3744367920 conn(0x7ff8080ff7f0 msgr2=0x7ff8081086f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:48:17.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:17.764+0000 7ff80e000640 1 -- 172.21.15.83:0/3744367920 shutdown_connections 2024-04-24T08:48:17.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:17.764+0000 7ff80e000640 1 -- 172.21.15.83:0/3744367920 wait complete. 2024-04-24T08:48:18.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:17 smithi083 ceph-mon[27907]: pgmap v359: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:48:18.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:17 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/3744367920' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:48:18.054 INFO:journalctl@ceph.mds.a.smithi083.stdout:Apr 24 08:48:17 smithi083 podman[69767]: 2024-04-24 08:48:17.721728932 +0000 UTC m=+0.485406877 container start 0f9ba266ef2dc6cf0a25f4b10ed3f8bbcbb7c4a3b691281151f641af9b3b4d87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=rishabh-main-apr17-a654945-1f81fb3, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, ceph=True) 2024-04-24T08:48:18.054 INFO:journalctl@ceph.mds.a.smithi083.stdout:Apr 24 08:48:17 smithi083 ceph-mds[69825]: -- 172.21.15.83:0/1846423083 <== mon.0 v2:172.21.15.83:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555dabcbfa40 con 0x555dabc93000 2024-04-24T08:48:18.054 INFO:journalctl@ceph.mds.a.smithi083.stdout:Apr 24 08:48:17 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-a[69821]: starting mds.a at 2024-04-24T08:48:18.054 INFO:journalctl@ceph.mds.a.smithi083.stdout:Apr 24 08:48:17 smithi083 bash[69767]: 0f9ba266ef2dc6cf0a25f4b10ed3f8bbcbb7c4a3b691281151f641af9b3b4d87 2024-04-24T08:48:18.054 INFO:journalctl@ceph.mds.a.smithi083.stdout:Apr 24 08:48:17 smithi083 systemd[1]: Started Ceph mds.a for bd3e039e-0215-11ef-bc93-c7b262605968. 2024-04-24T08:48:18.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:17 smithi138 ceph-mon[31441]: pgmap v359: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:48:18.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:17 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/3744367920' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:48:18.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:17 smithi189 ceph-mon[31791]: pgmap v359: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:48:18.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:17 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/3744367920' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:48:19.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:18 smithi138 ceph-mon[31441]: mds.? [v2:172.21.15.83:6840/2634754210,v1:172.21.15.83:6841/2634754210] up:boot 2024-04-24T08:48:19.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:18 smithi138 ceph-mon[31441]: fsmap 11 up:standby 2024-04-24T08:48:19.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:18 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-24T08:48:19.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:18 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:48:19.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:18 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:48:19.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:18 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:48:19.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:18 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T08:48:19.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:18 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T08:48:19.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:18 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:48:19.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:18 smithi189 ceph-mon[31791]: mds.? [v2:172.21.15.83:6840/2634754210,v1:172.21.15.83:6841/2634754210] up:boot 2024-04-24T08:48:19.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:18 smithi189 ceph-mon[31791]: fsmap 11 up:standby 2024-04-24T08:48:19.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:18 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-24T08:48:19.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:18 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:48:19.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:18 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:48:19.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:18 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:48:19.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:18 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T08:48:19.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:18 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T08:48:19.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:18 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:48:19.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:18 smithi083 ceph-mon[27907]: mds.? [v2:172.21.15.83:6840/2634754210,v1:172.21.15.83:6841/2634754210] up:boot 2024-04-24T08:48:19.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:18 smithi083 ceph-mon[27907]: fsmap 11 up:standby 2024-04-24T08:48:19.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:18 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-24T08:48:19.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:18 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:48:19.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:18 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:48:19.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:18 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:48:19.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:18 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T08:48:19.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:18 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T08:48:19.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:18 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:48:19.783 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T08:48:20.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:20.000+0000 7fb6af200640 1 -- 172.21.15.83:0/3486166399 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6b0101050 msgr2=0x7fb6b01014b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:20.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:20.000+0000 7fb6af200640 1 --2- 172.21.15.83:0/3486166399 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6b0101050 0x7fb6b01014b0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb698009900 tx=0x7fb69802a410 comp rx=0 tx=0).stop 2024-04-24T08:48:20.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:20.001+0000 7fb6af200640 1 -- 172.21.15.83:0/3486166399 shutdown_connections 2024-04-24T08:48:20.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:20.001+0000 7fb6af200640 1 --2- 172.21.15.83:0/3486166399 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6b01019f0 0x7fb6b0101e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:20.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:20.001+0000 7fb6af200640 1 --2- 172.21.15.83:0/3486166399 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6b0101050 0x7fb6b01014b0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:20.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:20.001+0000 7fb6af200640 1 --2- 172.21.15.83:0/3486166399 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb6b01006f0 0x7fb6b0100b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:20.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:20.001+0000 7fb6af200640 1 -- 172.21.15.83:0/3486166399 >> 172.21.15.83:0/3486166399 conn(0x7fb6b00fc2d0 msgr2=0x7fb6b00fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:48:20.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:20.001+0000 7fb6af200640 1 -- 172.21.15.83:0/3486166399 shutdown_connections 2024-04-24T08:48:20.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:20.001+0000 7fb6af200640 1 -- 172.21.15.83:0/3486166399 wait complete. 2024-04-24T08:48:20.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:20.002+0000 7fb6af200640 1 Processor -- start 2024-04-24T08:48:20.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:20.002+0000 7fb6af200640 1 -- start start 2024-04-24T08:48:20.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:20.002+0000 7fb6af200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6b01006f0 0x7fb6b019a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:20.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:20.002+0000 7fb6af200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb6b0101050 0x7fb6b019a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:20.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:20.002+0000 7fb6af200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6b01019f0 0x7fb6b01a1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:20.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:20.002+0000 7fb6af200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb6b010fc10 con 0x7fb6b01019f0 2024-04-24T08:48:20.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:20.002+0000 7fb6af200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb6b010fa90 con 0x7fb6b01006f0 2024-04-24T08:48:20.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:20.002+0000 7fb6af200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb6b010fd90 con 0x7fb6b0101050 2024-04-24T08:48:20.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:20.003+0000 7fb6ae800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6b01019f0 0x7fb6b01a1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:20.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:20.003+0000 7fb6ae800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6b01019f0 0x7fb6b01a1900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56630/0 (socket says 172.21.15.83:56630) 2024-04-24T08:48:20.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:20.003+0000 7fb6ae800640 1 -- 172.21.15.83:0/3882250858 learned_addr learned my addr 172.21.15.83:0/3882250858 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:48:20.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:20.003+0000 7fb6ade00640 1 --2- 172.21.15.83:0/3882250858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6b01006f0 0x7fb6b019a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:20.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:20.003+0000 7fb6ad400640 1 --2- 172.21.15.83:0/3882250858 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb6b0101050 0x7fb6b019a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:20.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:20.003+0000 7fb6ae800640 1 -- 172.21.15.83:0/3882250858 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb6b0101050 msgr2=0x7fb6b019a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:20.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:20.003+0000 7fb6ae800640 1 --2- 172.21.15.83:0/3882250858 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb6b0101050 0x7fb6b019a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:20.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:20.003+0000 7fb6ae800640 1 -- 172.21.15.83:0/3882250858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6b01006f0 msgr2=0x7fb6b019a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:20.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:20.003+0000 7fb6ae800640 1 --2- 172.21.15.83:0/3882250858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6b01006f0 0x7fb6b019a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:20.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:20.003+0000 7fb6ae800640 1 -- 172.21.15.83:0/3882250858 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6b01a1f70 con 0x7fb6b01019f0 2024-04-24T08:48:20.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:20.003+0000 7fb6ade00640 1 --2- 172.21.15.83:0/3882250858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6b01006f0 0x7fb6b019a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:48:20.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:20.004+0000 7fb6ae800640 1 --2- 172.21.15.83:0/3882250858 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6b01019f0 0x7fb6b01a1900 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb6a800b970 tx=0x7fb6a800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:48:20.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:20.004+0000 7fb6a6a00640 1 -- 172.21.15.83:0/3882250858 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6a8019070 con 0x7fb6b01019f0 2024-04-24T08:48:20.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:20.004+0000 7fb6a6a00640 1 -- 172.21.15.83:0/3882250858 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb6a8014440 con 0x7fb6b01019f0 2024-04-24T08:48:20.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:20.004+0000 7fb6a6a00640 1 -- 172.21.15.83:0/3882250858 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6a80075d0 con 0x7fb6b01019f0 2024-04-24T08:48:20.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:20.004+0000 7fb6af200640 1 -- 172.21.15.83:0/3882250858 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6b01a2260 con 0x7fb6b01019f0 2024-04-24T08:48:20.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:20.004+0000 7fb6af200640 1 -- 172.21.15.83:0/3882250858 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6b01a27e0 con 0x7fb6b01019f0 2024-04-24T08:48:20.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:20.005+0000 7fb6af200640 1 -- 172.21.15.83:0/3882250858 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6b010b440 con 0x7fb6b01019f0 2024-04-24T08:48:20.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:20.007+0000 7fb6a6a00640 1 -- 172.21.15.83:0/3882250858 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb6a801c070 con 0x7fb6b01019f0 2024-04-24T08:48:20.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:20.009+0000 7fb6a6a00640 1 --2- 172.21.15.83:0/3882250858 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fb6940a3bb0 0x7fb6940a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:20.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:20.010+0000 7fb6a6a00640 1 -- 172.21.15.83:0/3882250858 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fb6a80cf630 con 0x7fb6b01019f0 2024-04-24T08:48:20.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:20.010+0000 7fb6ade00640 1 --2- 172.21.15.83:0/3882250858 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fb6940a3bb0 0x7fb6940a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:20.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:20.010+0000 7fb6ade00640 1 --2- 172.21.15.83:0/3882250858 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fb6940a3bb0 0x7fb6940a6070 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb6a0005fd0 tx=0x7fb6a0005ea0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:48:20.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:20.015+0000 7fb6a6a00640 1 -- 172.21.15.83:0/3882250858 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb6a809c410 con 0x7fb6b01019f0 2024-04-24T08:48:20.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:19 smithi138 ceph-mon[31441]: Deploying daemon mds.b on smithi138 2024-04-24T08:48:20.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:19 smithi138 ceph-mon[31441]: pgmap v360: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:48:20.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:19 smithi189 ceph-mon[31791]: Deploying daemon mds.b on smithi138 2024-04-24T08:48:20.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:19 smithi189 ceph-mon[31791]: pgmap v360: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:48:20.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:20.256+0000 7fb6af200640 1 -- 172.21.15.83:0/3882250858 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb6b0062c00 con 0x7fb6b01019f0 2024-04-24T08:48:20.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:20.257+0000 7fb6a6a00640 1 -- 172.21.15.83:0/3882250858 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9662 (secure 0 0 0) 0x7fb6a809bd10 con 0x7fb6b01019f0 2024-04-24T08:48:20.257 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:48:20.257 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/358872257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":358872257},{"type":"v1","addr":"172.21.15.83:6835","nonce":358872257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3157070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3157070851},{"type":"v1","addr":"172.21.15.83:6837","nonce":3157070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14658,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6839/4085470844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":4085470844},{"type":"v1","addr":"172.21.15.83:6839","nonce":4085470844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14676,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6841/2634754210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":2634754210},{"type":"v1","addr":"172.21.15.83:6841","nonce":2634754210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/1859921310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1859921310},{"type":"v1","addr":"172.21.15.189:6833","nonce":1859921310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24421,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/2241923197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2241923197},{"type":"v1","addr":"172.21.15.189:6835","nonce":2241923197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24439,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/3172311220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":3172311220},{"type":"v1","addr":"172.21.15.189:6837","nonce":3172311220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24454,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/2548965336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":2548965336},{"type":"v1","addr":"172.21.15.189:6839","nonce":2548965336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34427,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/1157967658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1157967658},{"type":"v1","addr":"172.21.15.138:6833","nonce":1157967658}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34451,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1909219758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1909219758},{"type":"v1","addr":"172.21.15.138:6835","nonce":1909219758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":34493,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/900566754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":900566754},{"type":"v1","addr":"172.21.15.138:6837","nonce":900566754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-24T08:48:20.258 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 12 2024-04-24T08:48:20.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:20.260+0000 7fb6af200640 1 -- 172.21.15.83:0/3882250858 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fb6940a3bb0 msgr2=0x7fb6940a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:20.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:20.260+0000 7fb6af200640 1 --2- 172.21.15.83:0/3882250858 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fb6940a3bb0 0x7fb6940a6070 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb6a0005fd0 tx=0x7fb6a0005ea0 comp rx=0 tx=0).stop 2024-04-24T08:48:20.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:20.260+0000 7fb6af200640 1 -- 172.21.15.83:0/3882250858 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6b01019f0 msgr2=0x7fb6b01a1900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:20.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:20.260+0000 7fb6af200640 1 --2- 172.21.15.83:0/3882250858 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6b01019f0 0x7fb6b01a1900 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb6a800b970 tx=0x7fb6a800be30 comp rx=0 tx=0).stop 2024-04-24T08:48:20.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:20.261+0000 7fb6af200640 1 -- 172.21.15.83:0/3882250858 shutdown_connections 2024-04-24T08:48:20.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:20.261+0000 7fb6af200640 1 --2- 172.21.15.83:0/3882250858 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fb6940a3bb0 0x7fb6940a6070 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:20.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:20.261+0000 7fb6af200640 1 --2- 172.21.15.83:0/3882250858 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6b01019f0 0x7fb6b01a1900 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:20.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:20.261+0000 7fb6af200640 1 --2- 172.21.15.83:0/3882250858 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb6b0101050 0x7fb6b019a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:20.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:20.261+0000 7fb6af200640 1 --2- 172.21.15.83:0/3882250858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6b01006f0 0x7fb6b019a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:20.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:20.261+0000 7fb6af200640 1 -- 172.21.15.83:0/3882250858 >> 172.21.15.83:0/3882250858 conn(0x7fb6b00fc2d0 msgr2=0x7fb6b00fddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:48:20.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:20.261+0000 7fb6af200640 1 -- 172.21.15.83:0/3882250858 shutdown_connections 2024-04-24T08:48:20.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:20.261+0000 7fb6af200640 1 -- 172.21.15.83:0/3882250858 wait complete. 2024-04-24T08:48:20.275 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:19 smithi083 ceph-mon[27907]: Deploying daemon mds.b on smithi138 2024-04-24T08:48:20.276 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:19 smithi083 ceph-mon[27907]: pgmap v360: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:48:21.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:20 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/3882250858' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:48:21.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:20 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/3882250858' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:48:21.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:20 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/3882250858' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:48:21.641 INFO:journalctl@ceph.mds.b.smithi138.stdout:Apr 24 08:48:21 smithi138 systemd[1]: Starting Ceph mds.b for bd3e039e-0215-11ef-bc93-c7b262605968... 2024-04-24T08:48:21.922 INFO:journalctl@ceph.mds.b.smithi138.stdout:Apr 24 08:48:21 smithi138 podman[61501]: 2024-04-24 08:48:21.545157226 +0000 UTC m=+0.032773468 image pull b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644 2024-04-24T08:48:21.922 INFO:journalctl@ceph.mds.b.smithi138.stdout:Apr 24 08:48:21 smithi138 podman[61501]: 2024-04-24 08:48:21.675643199 +0000 UTC m=+0.163259518 container create 23d05a95c3bd84b3eba4a1e7ea52485bbb3573a70da483c32a4a3abdec7aae74 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-b, org.label-schema.build-date=20240424, GIT_CLEAN=True, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T08:48:22.187 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:21 smithi083 ceph-mon[27907]: pgmap v361: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:48:22.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:21 smithi138 ceph-mon[31441]: pgmap v361: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:48:22.206 INFO:journalctl@ceph.mds.b.smithi138.stdout:Apr 24 08:48:21 smithi138 podman[61501]: 2024-04-24 08:48:21.964340904 +0000 UTC m=+0.451957250 container init 23d05a95c3bd84b3eba4a1e7ea52485bbb3573a70da483c32a4a3abdec7aae74 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-b, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=rishabh-main-apr17-a654945-1f81fb3) 2024-04-24T08:48:22.206 INFO:journalctl@ceph.mds.b.smithi138.stdout:Apr 24 08:48:21 smithi138 podman[61501]: 2024-04-24 08:48:21.977083096 +0000 UTC m=+0.464699345 container start 23d05a95c3bd84b3eba4a1e7ea52485bbb3573a70da483c32a4a3abdec7aae74 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-b, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-04-24T08:48:22.206 INFO:journalctl@ceph.mds.b.smithi138.stdout:Apr 24 08:48:22 smithi138 ceph-mds[61519]: -- 172.21.15.138:0/105031036 <== mon.2 v2:172.21.15.138:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55cf861b1a40 con 0x55cf86184800 2024-04-24T08:48:22.206 INFO:journalctl@ceph.mds.b.smithi138.stdout:Apr 24 08:48:22 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-b[61515]: starting mds.b at 2024-04-24T08:48:22.206 INFO:journalctl@ceph.mds.b.smithi138.stdout:Apr 24 08:48:22 smithi138 bash[61501]: 23d05a95c3bd84b3eba4a1e7ea52485bbb3573a70da483c32a4a3abdec7aae74 2024-04-24T08:48:22.206 INFO:journalctl@ceph.mds.b.smithi138.stdout:Apr 24 08:48:22 smithi138 systemd[1]: Started Ceph mds.b for bd3e039e-0215-11ef-bc93-c7b262605968. 2024-04-24T08:48:22.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:21 smithi189 ceph-mon[31791]: pgmap v361: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:48:22.280 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T08:48:22.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:22.483+0000 7f9fb9c00640 1 -- 172.21.15.83:0/2039464479 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9fb41019f0 msgr2=0x7f9fb4101e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:22.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:22.483+0000 7f9fb9c00640 1 --2- 172.21.15.83:0/2039464479 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9fb41019f0 0x7f9fb4101e70 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f9fac009ca0 tx=0x7f9fac02a8c0 comp rx=0 tx=0).stop 2024-04-24T08:48:22.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:22.483+0000 7f9fb9c00640 1 -- 172.21.15.83:0/2039464479 shutdown_connections 2024-04-24T08:48:22.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:22.483+0000 7f9fb9c00640 1 --2- 172.21.15.83:0/2039464479 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9fb41019f0 0x7f9fb4101e70 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:22.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:22.483+0000 7f9fb9c00640 1 --2- 172.21.15.83:0/2039464479 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9fb4101050 0x7f9fb41014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:22.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:22.483+0000 7f9fb9c00640 1 --2- 172.21.15.83:0/2039464479 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9fb41006f0 0x7f9fb4100b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:22.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:22.483+0000 7f9fb9c00640 1 -- 172.21.15.83:0/2039464479 >> 172.21.15.83:0/2039464479 conn(0x7f9fb40fc310 msgr2=0x7f9fb40fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:48:22.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:22.483+0000 7f9fb9c00640 1 -- 172.21.15.83:0/2039464479 shutdown_connections 2024-04-24T08:48:22.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:22.483+0000 7f9fb9c00640 1 -- 172.21.15.83:0/2039464479 wait complete. 2024-04-24T08:48:22.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:22.484+0000 7f9fb9c00640 1 Processor -- start 2024-04-24T08:48:22.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:22.484+0000 7f9fb9c00640 1 -- start start 2024-04-24T08:48:22.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:22.485+0000 7f9fb9c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9fb41006f0 0x7f9fb419a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:22.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:22.485+0000 7f9fb9c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9fb4101050 0x7f9fb419a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:22.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:22.485+0000 7f9fb9c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9fb41019f0 0x7f9fb41a1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:22.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:22.485+0000 7f9fb9c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb410fbd0 con 0x7f9fb41006f0 2024-04-24T08:48:22.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:22.485+0000 7f9fb9c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb410fa50 con 0x7f9fb41019f0 2024-04-24T08:48:22.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:22.485+0000 7f9fb9c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb410fd50 con 0x7f9fb4101050 2024-04-24T08:48:22.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:22.485+0000 7f9fb3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9fb41006f0 0x7f9fb419a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:22.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:22.485+0000 7f9fb3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9fb41006f0 0x7f9fb419a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56646/0 (socket says 172.21.15.83:56646) 2024-04-24T08:48:22.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:22.485+0000 7f9fb3e00640 1 -- 172.21.15.83:0/2633864313 learned_addr learned my addr 172.21.15.83:0/2633864313 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:48:22.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:22.485+0000 7f9fb9200640 1 --2- 172.21.15.83:0/2633864313 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9fb41019f0 0x7f9fb41a1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:22.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:22.485+0000 7f9fabe00640 1 --2- 172.21.15.83:0/2633864313 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9fb4101050 0x7f9fb419a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:22.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:22.485+0000 7f9fb3e00640 1 -- 172.21.15.83:0/2633864313 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9fb4101050 msgr2=0x7f9fb419a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:22.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:22.485+0000 7f9fb3e00640 1 --2- 172.21.15.83:0/2633864313 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9fb4101050 0x7f9fb419a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:22.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:22.485+0000 7f9fb3e00640 1 -- 172.21.15.83:0/2633864313 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9fb41019f0 msgr2=0x7f9fb41a1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:22.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:22.485+0000 7f9fb3e00640 1 --2- 172.21.15.83:0/2633864313 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9fb41019f0 0x7f9fb41a1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:22.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:22.485+0000 7f9fb3e00640 1 -- 172.21.15.83:0/2633864313 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fb41a1e00 con 0x7f9fb41006f0 2024-04-24T08:48:22.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:22.485+0000 7f9fabe00640 1 --2- 172.21.15.83:0/2633864313 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9fb4101050 0x7f9fb419a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:48:22.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:22.486+0000 7f9fb3e00640 1 --2- 172.21.15.83:0/2633864313 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9fb41006f0 0x7f9fb419a1e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9fa400d950 tx=0x7f9fa400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:48:22.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:22.486+0000 7f9fb1600640 1 -- 172.21.15.83:0/2633864313 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fa4017070 con 0x7f9fb41006f0 2024-04-24T08:48:22.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:22.486+0000 7f9fb1600640 1 -- 172.21.15.83:0/2633864313 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9fa4009c30 con 0x7f9fb41006f0 2024-04-24T08:48:22.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:22.486+0000 7f9fb9c00640 1 -- 172.21.15.83:0/2633864313 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fb41a20f0 con 0x7f9fb41006f0 2024-04-24T08:48:22.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:22.486+0000 7f9fb1600640 1 -- 172.21.15.83:0/2633864313 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fa400f620 con 0x7f9fb41006f0 2024-04-24T08:48:22.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:22.486+0000 7f9fb9c00640 1 -- 172.21.15.83:0/2633864313 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fb41a2640 con 0x7f9fb41006f0 2024-04-24T08:48:22.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:22.487+0000 7f9fb9c00640 1 -- 172.21.15.83:0/2633864313 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fb410b5d0 con 0x7f9fb41006f0 2024-04-24T08:48:22.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:22.489+0000 7f9fb1600640 1 -- 172.21.15.83:0/2633864313 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9fa40109a0 con 0x7f9fb41006f0 2024-04-24T08:48:22.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:22.491+0000 7f9fb1600640 1 --2- 172.21.15.83:0/2633864313 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f9f980a3bb0 0x7f9f980a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:22.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:22.491+0000 7f9fb1600640 1 -- 172.21.15.83:0/2633864313 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f9fa40cfce0 con 0x7f9fb41006f0 2024-04-24T08:48:22.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:22.491+0000 7f9fabe00640 1 --2- 172.21.15.83:0/2633864313 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f9f980a3bb0 0x7f9f980a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:22.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:22.492+0000 7f9fabe00640 1 --2- 172.21.15.83:0/2633864313 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f9f980a3bb0 0x7f9f980a6070 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9fb419b6f0 tx=0x7f9f9c005d90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:48:22.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:22.495+0000 7f9fb1600640 1 -- 172.21.15.83:0/2633864313 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9fa4009de0 con 0x7f9fb41006f0 2024-04-24T08:48:22.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:22.737+0000 7f9fb9c00640 1 -- 172.21.15.83:0/2633864313 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9fb402d620 con 0x7f9fb41006f0 2024-04-24T08:48:22.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:22.739+0000 7f9fb1600640 1 -- 172.21.15.83:0/2633864313 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9662 (secure 0 0 0) 0x7f9fa409cac0 con 0x7f9fb41006f0 2024-04-24T08:48:22.739 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:48:22.739 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/358872257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":358872257},{"type":"v1","addr":"172.21.15.83:6835","nonce":358872257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3157070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3157070851},{"type":"v1","addr":"172.21.15.83:6837","nonce":3157070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14658,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6839/4085470844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":4085470844},{"type":"v1","addr":"172.21.15.83:6839","nonce":4085470844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14676,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6841/2634754210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":2634754210},{"type":"v1","addr":"172.21.15.83:6841","nonce":2634754210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/1859921310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1859921310},{"type":"v1","addr":"172.21.15.189:6833","nonce":1859921310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24421,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/2241923197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2241923197},{"type":"v1","addr":"172.21.15.189:6835","nonce":2241923197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24439,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/3172311220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":3172311220},{"type":"v1","addr":"172.21.15.189:6837","nonce":3172311220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24454,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/2548965336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":2548965336},{"type":"v1","addr":"172.21.15.189:6839","nonce":2548965336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34427,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/1157967658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1157967658},{"type":"v1","addr":"172.21.15.138:6833","nonce":1157967658}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34451,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1909219758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1909219758},{"type":"v1","addr":"172.21.15.138:6835","nonce":1909219758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":34493,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/900566754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":900566754},{"type":"v1","addr":"172.21.15.138:6837","nonce":900566754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-24T08:48:22.739 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 12 2024-04-24T08:48:22.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:22.742+0000 7f9fb9c00640 1 -- 172.21.15.83:0/2633864313 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f9f980a3bb0 msgr2=0x7f9f980a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:22.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:22.742+0000 7f9fb9c00640 1 --2- 172.21.15.83:0/2633864313 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f9f980a3bb0 0x7f9f980a6070 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9fb419b6f0 tx=0x7f9f9c005d90 comp rx=0 tx=0).stop 2024-04-24T08:48:22.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:22.742+0000 7f9fb9c00640 1 -- 172.21.15.83:0/2633864313 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9fb41006f0 msgr2=0x7f9fb419a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:22.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:22.742+0000 7f9fb9c00640 1 --2- 172.21.15.83:0/2633864313 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9fb41006f0 0x7f9fb419a1e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9fa400d950 tx=0x7f9fa400de10 comp rx=0 tx=0).stop 2024-04-24T08:48:22.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:22.742+0000 7f9fb9c00640 1 -- 172.21.15.83:0/2633864313 shutdown_connections 2024-04-24T08:48:22.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:22.742+0000 7f9fb9c00640 1 --2- 172.21.15.83:0/2633864313 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9fb41019f0 0x7f9fb41a1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:22.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:22.742+0000 7f9fb9c00640 1 --2- 172.21.15.83:0/2633864313 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9fb4101050 0x7f9fb419a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:22.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:22.742+0000 7f9fb9c00640 1 --2- 172.21.15.83:0/2633864313 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f9f980a3bb0 0x7f9f980a6070 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:22.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:22.742+0000 7f9fb9c00640 1 --2- 172.21.15.83:0/2633864313 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9fb41006f0 0x7f9fb419a1e0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:22.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:22.742+0000 7f9fb9c00640 1 -- 172.21.15.83:0/2633864313 >> 172.21.15.83:0/2633864313 conn(0x7f9fb40fc310 msgr2=0x7f9fb40fd9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:48:22.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:22.742+0000 7f9fb9c00640 1 -- 172.21.15.83:0/2633864313 shutdown_connections 2024-04-24T08:48:22.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:22.742+0000 7f9fb9c00640 1 -- 172.21.15.83:0/2633864313 wait complete. 2024-04-24T08:48:23.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:23 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:48:23.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:23 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:48:23.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:23 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:48:23.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:23 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:48:23.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:23 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:48:23.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:23 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/2633864313' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:48:23.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:23 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:48:23.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:23 smithi083 ceph-mon[27907]: pgmap v362: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:48:23.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:23 smithi083 ceph-mon[27907]: mds.? [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] up:boot 2024-04-24T08:48:23.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:23 smithi083 ceph-mon[27907]: fsmap 12 up:standby 2024-04-24T08:48:23.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:23 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-24T08:48:23.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:23 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:48:23.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:23 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:48:23.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:23 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:48:23.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:23 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:48:23.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:23 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:48:23.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:23 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/2633864313' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:48:23.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:23 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:48:23.706 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:23 smithi138 ceph-mon[31441]: pgmap v362: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:48:23.706 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:23 smithi138 ceph-mon[31441]: mds.? [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] up:boot 2024-04-24T08:48:23.706 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:23 smithi138 ceph-mon[31441]: fsmap 12 up:standby 2024-04-24T08:48:23.706 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:23 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-24T08:48:23.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:23 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:48:23.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:23 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:48:23.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:23 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:48:23.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:23 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:48:23.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:23 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:48:23.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:23 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/2633864313' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:48:23.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:23 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:48:23.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:23 smithi189 ceph-mon[31791]: pgmap v362: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:48:23.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:23 smithi189 ceph-mon[31791]: mds.? [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] up:boot 2024-04-24T08:48:23.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:23 smithi189 ceph-mon[31791]: fsmap 12 up:standby 2024-04-24T08:48:23.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:23 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-24T08:48:24.761 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T08:48:24.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:24.957+0000 7f1e03a00640 1 -- 172.21.15.83:0/1973985689 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1dfc105450 msgr2=0x7f1dfc1058d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:24.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:24.957+0000 7f1e03a00640 1 --2- 172.21.15.83:0/1973985689 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1dfc105450 0x7f1dfc1058d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1de8009900 tx=0x7f1de802a410 comp rx=0 tx=0).stop 2024-04-24T08:48:24.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:24.958+0000 7f1e03a00640 1 -- 172.21.15.83:0/1973985689 shutdown_connections 2024-04-24T08:48:24.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:24.958+0000 7f1e03a00640 1 --2- 172.21.15.83:0/1973985689 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1dfc105e10 0x7f1dfc10c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:24.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:24.958+0000 7f1e03a00640 1 --2- 172.21.15.83:0/1973985689 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1dfc105450 0x7f1dfc1058d0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:24.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:24.958+0000 7f1e03a00640 1 --2- 172.21.15.83:0/1973985689 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1dfc104260 0x7f1dfc104680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:24.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:24.958+0000 7f1e03a00640 1 -- 172.21.15.83:0/1973985689 >> 172.21.15.83:0/1973985689 conn(0x7f1dfc0ff7f0 msgr2=0x7f1dfc101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:48:24.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:24.958+0000 7f1e03a00640 1 -- 172.21.15.83:0/1973985689 shutdown_connections 2024-04-24T08:48:24.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:24.958+0000 7f1e03a00640 1 -- 172.21.15.83:0/1973985689 wait complete. 2024-04-24T08:48:24.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:24.959+0000 7f1e03a00640 1 Processor -- start 2024-04-24T08:48:24.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:24.959+0000 7f1e03a00640 1 -- start start 2024-04-24T08:48:24.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:24.959+0000 7f1e03a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1dfc104260 0x7f1dfc19a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:24.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:24.959+0000 7f1e03a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1dfc105450 0x7f1dfc19a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:24.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:24.959+0000 7f1e03a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1dfc105e10 0x7f1dfc1a1a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:24.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:24.959+0000 7f1e03a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1dfc10fdf0 con 0x7f1dfc104260 2024-04-24T08:48:24.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:24.959+0000 7f1e03a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1dfc10fc70 con 0x7f1dfc105e10 2024-04-24T08:48:24.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:24.959+0000 7f1e03a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1dfc10ff70 con 0x7f1dfc105450 2024-04-24T08:48:24.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:24.959+0000 7f1e01000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1dfc104260 0x7f1dfc19a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:24.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:24.959+0000 7f1e01000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1dfc104260 0x7f1dfc19a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56664/0 (socket says 172.21.15.83:56664) 2024-04-24T08:48:24.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:24.959+0000 7f1e01000640 1 -- 172.21.15.83:0/2124240226 learned_addr learned my addr 172.21.15.83:0/2124240226 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:48:24.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:24.960+0000 7f1e01a00640 1 --2- 172.21.15.83:0/2124240226 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1dfc105e10 0x7f1dfc1a1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:24.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:24.960+0000 7f1dfbe00640 1 --2- 172.21.15.83:0/2124240226 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1dfc105450 0x7f1dfc19a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:24.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:24.960+0000 7f1e01000640 1 -- 172.21.15.83:0/2124240226 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1dfc105450 msgr2=0x7f1dfc19a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:24.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:24.960+0000 7f1e01000640 1 --2- 172.21.15.83:0/2124240226 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1dfc105450 0x7f1dfc19a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:24.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:24.960+0000 7f1e01000640 1 -- 172.21.15.83:0/2124240226 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1dfc105e10 msgr2=0x7f1dfc1a1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:24.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:24.960+0000 7f1e01000640 1 --2- 172.21.15.83:0/2124240226 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1dfc105e10 0x7f1dfc1a1a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:24.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:24.960+0000 7f1e01000640 1 -- 172.21.15.83:0/2124240226 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dfc1a20a0 con 0x7f1dfc104260 2024-04-24T08:48:24.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:24.960+0000 7f1e01000640 1 --2- 172.21.15.83:0/2124240226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1dfc104260 0x7f1dfc19a480 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1df000d950 tx=0x7f1df000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:48:24.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:24.961+0000 7f1df9600640 1 -- 172.21.15.83:0/2124240226 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df0017070 con 0x7f1dfc104260 2024-04-24T08:48:24.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:24.961+0000 7f1df9600640 1 -- 172.21.15.83:0/2124240226 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1df0009c30 con 0x7f1dfc104260 2024-04-24T08:48:24.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:24.961+0000 7f1e03a00640 1 -- 172.21.15.83:0/2124240226 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dfc1a2390 con 0x7f1dfc104260 2024-04-24T08:48:24.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:24.961+0000 7f1df9600640 1 -- 172.21.15.83:0/2124240226 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df000f620 con 0x7f1dfc104260 2024-04-24T08:48:24.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:24.961+0000 7f1e03a00640 1 -- 172.21.15.83:0/2124240226 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dfc1a28b0 con 0x7f1dfc104260 2024-04-24T08:48:24.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:24.962+0000 7f1e03a00640 1 -- 172.21.15.83:0/2124240226 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1dfc10b580 con 0x7f1dfc104260 2024-04-24T08:48:24.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:24.964+0000 7f1df9600640 1 -- 172.21.15.83:0/2124240226 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f1df001a070 con 0x7f1dfc104260 2024-04-24T08:48:24.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:24.966+0000 7f1df9600640 1 --2- 172.21.15.83:0/2124240226 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f1de40a3c30 0x7f1de40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:24.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:24.966+0000 7f1df9600640 1 -- 172.21.15.83:0/2124240226 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f1df00cfa20 con 0x7f1dfc104260 2024-04-24T08:48:24.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:24.966+0000 7f1dfbe00640 1 --2- 172.21.15.83:0/2124240226 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f1de40a3c30 0x7f1de40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:24.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:24.967+0000 7f1dfbe00640 1 --2- 172.21.15.83:0/2124240226 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f1de40a3c30 0x7f1de40a60f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1de8000c00 tx=0x7f1de80314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:48:24.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:24.970+0000 7f1df9600640 1 -- 172.21.15.83:0/2124240226 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1df0009de0 con 0x7f1dfc104260 2024-04-24T08:48:25.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:25.216+0000 7f1e03a00640 1 -- 172.21.15.83:0/2124240226 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1dfc02d620 con 0x7f1dfc104260 2024-04-24T08:48:25.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:25.217+0000 7f1df9600640 1 -- 172.21.15.83:0/2124240226 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10491 (secure 0 0 0) 0x7f1df009c800 con 0x7f1dfc104260 2024-04-24T08:48:25.218 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:48:25.218 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/358872257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":358872257},{"type":"v1","addr":"172.21.15.83:6835","nonce":358872257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3157070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3157070851},{"type":"v1","addr":"172.21.15.83:6837","nonce":3157070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14658,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6839/4085470844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":4085470844},{"type":"v1","addr":"172.21.15.83:6839","nonce":4085470844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14676,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6841/2634754210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":2634754210},{"type":"v1","addr":"172.21.15.83:6841","nonce":2634754210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/1859921310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1859921310},{"type":"v1","addr":"172.21.15.189:6833","nonce":1859921310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24421,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/2241923197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2241923197},{"type":"v1","addr":"172.21.15.189:6835","nonce":2241923197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24439,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/3172311220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":3172311220},{"type":"v1","addr":"172.21.15.189:6837","nonce":3172311220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24454,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/2548965336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":2548965336},{"type":"v1","addr":"172.21.15.189:6839","nonce":2548965336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34427,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/1157967658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1157967658},{"type":"v1","addr":"172.21.15.138:6833","nonce":1157967658}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34451,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1909219758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1909219758},{"type":"v1","addr":"172.21.15.138:6835","nonce":1909219758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":34493,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/900566754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":900566754},{"type":"v1","addr":"172.21.15.138:6837","nonce":900566754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":34520,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6839/2236932393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2236932393},{"type":"v1","addr":"172.21.15.138:6839","nonce":2236932393}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-04-24T08:48:25.218 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 13 2024-04-24T08:48:25.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:25.219+0000 7f1e03a00640 1 -- 172.21.15.83:0/2124240226 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f1de40a3c30 msgr2=0x7f1de40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:25.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:25.219+0000 7f1e03a00640 1 --2- 172.21.15.83:0/2124240226 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f1de40a3c30 0x7f1de40a60f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1de8000c00 tx=0x7f1de80314a0 comp rx=0 tx=0).stop 2024-04-24T08:48:25.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:25.219+0000 7f1e03a00640 1 -- 172.21.15.83:0/2124240226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1dfc104260 msgr2=0x7f1dfc19a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:25.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:25.219+0000 7f1e03a00640 1 --2- 172.21.15.83:0/2124240226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1dfc104260 0x7f1dfc19a480 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1df000d950 tx=0x7f1df000de10 comp rx=0 tx=0).stop 2024-04-24T08:48:25.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:25.219+0000 7f1e03a00640 1 -- 172.21.15.83:0/2124240226 shutdown_connections 2024-04-24T08:48:25.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:25.219+0000 7f1e03a00640 1 --2- 172.21.15.83:0/2124240226 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1dfc105e10 0x7f1dfc1a1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:25.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:25.219+0000 7f1e03a00640 1 --2- 172.21.15.83:0/2124240226 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1dfc105450 0x7f1dfc19a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:25.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:25.219+0000 7f1e03a00640 1 --2- 172.21.15.83:0/2124240226 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f1de40a3c30 0x7f1de40a60f0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:25.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:25.219+0000 7f1e03a00640 1 --2- 172.21.15.83:0/2124240226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1dfc104260 0x7f1dfc19a480 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:25.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:25.219+0000 7f1e03a00640 1 -- 172.21.15.83:0/2124240226 >> 172.21.15.83:0/2124240226 conn(0x7f1dfc0ff7f0 msgr2=0x7f1dfc108750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:48:25.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:25.219+0000 7f1e03a00640 1 -- 172.21.15.83:0/2124240226 shutdown_connections 2024-04-24T08:48:25.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:25.220+0000 7f1e03a00640 1 -- 172.21.15.83:0/2124240226 wait complete. 2024-04-24T08:48:25.238 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T08:48:25.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:25.403+0000 7efc68200640 1 -- 172.21.15.83:0/2201099893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc60104260 msgr2=0x7efc60104680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:25.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:25.403+0000 7efc68200640 1 --2- 172.21.15.83:0/2201099893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc60104260 0x7efc60104680 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7efc54009900 tx=0x7efc5402a410 comp rx=0 tx=0).stop 2024-04-24T08:48:25.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:25.403+0000 7efc68200640 1 -- 172.21.15.83:0/2201099893 shutdown_connections 2024-04-24T08:48:25.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:25.403+0000 7efc68200640 1 --2- 172.21.15.83:0/2201099893 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc60105e10 0x7efc6010c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:25.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:25.403+0000 7efc68200640 1 --2- 172.21.15.83:0/2201099893 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efc60105450 0x7efc601058d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:25.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:25.403+0000 7efc68200640 1 --2- 172.21.15.83:0/2201099893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc60104260 0x7efc60104680 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:25.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:25.403+0000 7efc68200640 1 -- 172.21.15.83:0/2201099893 >> 172.21.15.83:0/2201099893 conn(0x7efc600ff7f0 msgr2=0x7efc60101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:48:25.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:25.403+0000 7efc68200640 1 -- 172.21.15.83:0/2201099893 shutdown_connections 2024-04-24T08:48:25.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:25.403+0000 7efc68200640 1 -- 172.21.15.83:0/2201099893 wait complete. 2024-04-24T08:48:25.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:25.404+0000 7efc68200640 1 Processor -- start 2024-04-24T08:48:25.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:25.404+0000 7efc68200640 1 -- start start 2024-04-24T08:48:25.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:25.405+0000 7efc68200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc60104260 0x7efc6019a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:25.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:25.405+0000 7efc68200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efc60105450 0x7efc6019aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:25.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:25.405+0000 7efc68200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc60105e10 0x7efc601a1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:25.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:25.405+0000 7efc68200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7efc6010fe30 con 0x7efc60105e10 2024-04-24T08:48:25.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:25.405+0000 7efc68200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efc6010fcb0 con 0x7efc60104260 2024-04-24T08:48:25.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:25.405+0000 7efc68200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7efc6010ffb0 con 0x7efc60105450 2024-04-24T08:48:25.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:25.405+0000 7efc66200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc60105e10 0x7efc601a1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:25.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:25.405+0000 7efc66200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc60105e10 0x7efc601a1ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56678/0 (socket says 172.21.15.83:56678) 2024-04-24T08:48:25.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:25.405+0000 7efc66200640 1 -- 172.21.15.83:0/1121114494 learned_addr learned my addr 172.21.15.83:0/1121114494 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:48:25.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:25.405+0000 7efc65800640 1 --2- 172.21.15.83:0/1121114494 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc60104260 0x7efc6019a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:25.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:25.405+0000 7efc64e00640 1 --2- 172.21.15.83:0/1121114494 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efc60105450 0x7efc6019aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:25.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:25.405+0000 7efc66200640 1 -- 172.21.15.83:0/1121114494 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efc60105450 msgr2=0x7efc6019aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:25.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:25.405+0000 7efc66200640 1 --2- 172.21.15.83:0/1121114494 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efc60105450 0x7efc6019aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:25.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:25.405+0000 7efc66200640 1 -- 172.21.15.83:0/1121114494 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc60104260 msgr2=0x7efc6019a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:25.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:25.405+0000 7efc66200640 1 --2- 172.21.15.83:0/1121114494 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc60104260 0x7efc6019a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:25.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:25.405+0000 7efc66200640 1 -- 172.21.15.83:0/1121114494 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc601a2150 con 0x7efc60105e10 2024-04-24T08:48:25.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:25.406+0000 7efc66200640 1 --2- 172.21.15.83:0/1121114494 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc60105e10 0x7efc601a1ae0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7efc5800eb10 tx=0x7efc5800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:48:25.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:25.406+0000 7efc5e000640 1 -- 172.21.15.83:0/1121114494 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc58004b20 con 0x7efc60105e10 2024-04-24T08:48:25.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:25.406+0000 7efc5e000640 1 -- 172.21.15.83:0/1121114494 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efc58004cc0 con 0x7efc60105e10 2024-04-24T08:48:25.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:25.406+0000 7efc68200640 1 -- 172.21.15.83:0/1121114494 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc601a2440 con 0x7efc60105e10 2024-04-24T08:48:25.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:25.406+0000 7efc5e000640 1 -- 172.21.15.83:0/1121114494 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc58010690 con 0x7efc60105e10 2024-04-24T08:48:25.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:25.406+0000 7efc68200640 1 -- 172.21.15.83:0/1121114494 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc601a29c0 con 0x7efc60105e10 2024-04-24T08:48:25.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:25.407+0000 7efc68200640 1 -- 172.21.15.83:0/1121114494 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc6010b5d0 con 0x7efc60105e10 2024-04-24T08:48:25.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:25.409+0000 7efc5e000640 1 -- 172.21.15.83:0/1121114494 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7efc58005020 con 0x7efc60105e10 2024-04-24T08:48:25.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:25.412+0000 7efc5e000640 1 --2- 172.21.15.83:0/1121114494 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7efc480a3bb0 0x7efc480a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:25.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:25.412+0000 7efc5e000640 1 -- 172.21.15.83:0/1121114494 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7efc580d0620 con 0x7efc60105e10 2024-04-24T08:48:25.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:25.412+0000 7efc65800640 1 --2- 172.21.15.83:0/1121114494 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7efc480a3bb0 0x7efc480a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:25.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:25.413+0000 7efc65800640 1 --2- 172.21.15.83:0/1121114494 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7efc480a3bb0 0x7efc480a6070 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7efc54005ab0 tx=0x7efc54005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:48:25.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:25.416+0000 7efc5e000640 1 -- 172.21.15.83:0/1121114494 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efc5809d400 con 0x7efc60105e10 2024-04-24T08:48:25.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:25.646+0000 7efc68200640 1 -- 172.21.15.83:0/1121114494 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efc60062c00 con 0x7efc60105e10 2024-04-24T08:48:25.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:25.648+0000 7efc5e000640 1 -- 172.21.15.83:0/1121114494 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+18455 (secure 0 0 0) 0x7efc5809cd00 con 0x7efc60105e10 2024-04-24T08:48:25.649 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:48:25.649 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":81,"fsid":"bd3e039e-0215-11ef-bc93-c7b262605968","created":"2024-04-24T08:37:32.209944+0000","modified":"2024-04-24T08:47:18.981804+0000","last_up_change":"2024-04-24T08:47:17.966041+0000","last_in_change":"2024-04-24T08:46:53.835936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T08:41:56.834372+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2cd77f3a-19dd-4c6e-a199-1dcbd5faa679","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2132675555},{"type":"v1","addr":"172.21.15.83:6803","nonce":2132675555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2132675555},{"type":"v1","addr":"172.21.15.83:6805","nonce":2132675555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2132675555},{"type":"v1","addr":"172.21.15.83:6809","nonce":2132675555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2132675555},{"type":"v1","addr":"172.21.15.83:6807","nonce":2132675555}]},"public_addr":"172.21.15.83:6803/2132675555","cluster_addr":"172.21.15.83:6805/2132675555","heartbeat_back_addr":"172.21.15.83:6809/2132675555","heartbeat_front_addr":"172.21.15.83:6807/2132675555","state":["exists","up"]},{"osd":1,"uuid":"aba7f5df-2a4c-4c23-b502-d193605a5b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":3589158880},{"type":"v1","addr":"172.21.15.138:6801","nonce":3589158880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":3589158880},{"type":"v1","addr":"172.21.15.138:6803","nonce":3589158880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":3589158880},{"type":"v1","addr":"172.21.15.138:6807","nonce":3589158880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":3589158880},{"type":"v1","addr":"172.21.15.138:6805","nonce":3589158880}]},"public_addr":"172.21.15.138:6801/3589158880","cluster_addr":"172.21.15.138:6803/3589158880","heartbeat_back_addr":"172.21.15.138:6807/3589158880","heartbeat_front_addr":"172.21.15.138:6805/3589158880","state":["exists","up"]},{"osd":2,"uuid":"071bc167-2333-4420-bb58-bffbc10d7641","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":3169072688},{"type":"v1","addr":"172.21.15.189:6801","nonce":3169072688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":3169072688},{"type":"v1","addr":"172.21.15.189:6803","nonce":3169072688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":3169072688},{"type":"v1","addr":"172.21.15.189:6807","nonce":3169072688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":3169072688},{"type":"v1","addr":"172.21.15.189:6805","nonce":3169072688}]},"public_addr":"172.21.15.189:6801/3169072688","cluster_addr":"172.21.15.189:6803/3169072688","heartbeat_back_addr":"172.21.15.189:6807/3169072688","heartbeat_front_addr":"172.21.15.189:6805/3169072688","state":["exists","up"]},{"osd":3,"uuid":"f5624d78-4455-4e07-8c1a-f47c94cddb23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3131092450},{"type":"v1","addr":"172.21.15.83:6811","nonce":3131092450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3131092450},{"type":"v1","addr":"172.21.15.83:6813","nonce":3131092450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3131092450},{"type":"v1","addr":"172.21.15.83:6817","nonce":3131092450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3131092450},{"type":"v1","addr":"172.21.15.83:6815","nonce":3131092450}]},"public_addr":"172.21.15.83:6811/3131092450","cluster_addr":"172.21.15.83:6813/3131092450","heartbeat_back_addr":"172.21.15.83:6817/3131092450","heartbeat_front_addr":"172.21.15.83:6815/3131092450","state":["exists","up"]},{"osd":4,"uuid":"7b50cd14-9385-4535-a7b0-1c1da93c8548","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":823775360},{"type":"v1","addr":"172.21.15.138:6809","nonce":823775360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":823775360},{"type":"v1","addr":"172.21.15.138:6811","nonce":823775360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":823775360},{"type":"v1","addr":"172.21.15.138:6815","nonce":823775360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":823775360},{"type":"v1","addr":"172.21.15.138:6813","nonce":823775360}]},"public_addr":"172.21.15.138:6809/823775360","cluster_addr":"172.21.15.138:6811/823775360","heartbeat_back_addr":"172.21.15.138:6815/823775360","heartbeat_front_addr":"172.21.15.138:6813/823775360","state":["exists","up"]},{"osd":5,"uuid":"9c647797-35bc-449f-be84-af97d2c69c2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":1399817431},{"type":"v1","addr":"172.21.15.189:6809","nonce":1399817431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":1399817431},{"type":"v1","addr":"172.21.15.189:6811","nonce":1399817431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":1399817431},{"type":"v1","addr":"172.21.15.189:6815","nonce":1399817431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":1399817431},{"type":"v1","addr":"172.21.15.189:6813","nonce":1399817431}]},"public_addr":"172.21.15.189:6809/1399817431","cluster_addr":"172.21.15.189:6811/1399817431","heartbeat_back_addr":"172.21.15.189:6815/1399817431","heartbeat_front_addr":"172.21.15.189:6813/1399817431","state":["exists","up"]},{"osd":6,"uuid":"8f056584-16c8-4454-b445-3f780f26c196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":1611358483},{"type":"v1","addr":"172.21.15.83:6819","nonce":1611358483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1611358483},{"type":"v1","addr":"172.21.15.83:6821","nonce":1611358483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1611358483},{"type":"v1","addr":"172.21.15.83:6825","nonce":1611358483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":1611358483},{"type":"v1","addr":"172.21.15.83:6823","nonce":1611358483}]},"public_addr":"172.21.15.83:6819/1611358483","cluster_addr":"172.21.15.83:6821/1611358483","heartbeat_back_addr":"172.21.15.83:6825/1611358483","heartbeat_front_addr":"172.21.15.83:6823/1611358483","state":["exists","up"]},{"osd":7,"uuid":"7603efe3-da77-4b7f-9d03-29c4f293b6d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":3557856706},{"type":"v1","addr":"172.21.15.138:6817","nonce":3557856706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":3557856706},{"type":"v1","addr":"172.21.15.138:6819","nonce":3557856706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":3557856706},{"type":"v1","addr":"172.21.15.138:6823","nonce":3557856706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":3557856706},{"type":"v1","addr":"172.21.15.138:6821","nonce":3557856706}]},"public_addr":"172.21.15.138:6817/3557856706","cluster_addr":"172.21.15.138:6819/3557856706","heartbeat_back_addr":"172.21.15.138:6823/3557856706","heartbeat_front_addr":"172.21.15.138:6821/3557856706","state":["exists","up"]},{"osd":8,"uuid":"27d7d1b8-d6a0-417e-b8aa-596325aa1a77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":207018554},{"type":"v1","addr":"172.21.15.189:6817","nonce":207018554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":207018554},{"type":"v1","addr":"172.21.15.189:6819","nonce":207018554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":207018554},{"type":"v1","addr":"172.21.15.189:6823","nonce":207018554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":207018554},{"type":"v1","addr":"172.21.15.189:6821","nonce":207018554}]},"public_addr":"172.21.15.189:6817/207018554","cluster_addr":"172.21.15.189:6819/207018554","heartbeat_back_addr":"172.21.15.189:6823/207018554","heartbeat_front_addr":"172.21.15.189:6821/207018554","state":["exists","up"]},{"osd":9,"uuid":"baf19b98-881b-4d3a-9f72-8aa2dd73cad6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":519636751},{"type":"v1","addr":"172.21.15.83:6827","nonce":519636751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":519636751},{"type":"v1","addr":"172.21.15.83:6829","nonce":519636751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":519636751},{"type":"v1","addr":"172.21.15.83:6833","nonce":519636751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":519636751},{"type":"v1","addr":"172.21.15.83:6831","nonce":519636751}]},"public_addr":"172.21.15.83:6827/519636751","cluster_addr":"172.21.15.83:6829/519636751","heartbeat_back_addr":"172.21.15.83:6833/519636751","heartbeat_front_addr":"172.21.15.83:6831/519636751","state":["exists","up"]},{"osd":10,"uuid":"eb346717-5c07-47c8-95be-0e68fe449fdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1130872098},{"type":"v1","addr":"172.21.15.138:6825","nonce":1130872098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1130872098},{"type":"v1","addr":"172.21.15.138:6827","nonce":1130872098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1130872098},{"type":"v1","addr":"172.21.15.138:6831","nonce":1130872098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1130872098},{"type":"v1","addr":"172.21.15.138:6829","nonce":1130872098}]},"public_addr":"172.21.15.138:6825/1130872098","cluster_addr":"172.21.15.138:6827/1130872098","heartbeat_back_addr":"172.21.15.138:6831/1130872098","heartbeat_front_addr":"172.21.15.138:6829/1130872098","state":["exists","up"]},{"osd":11,"uuid":"8e37568e-981c-45b8-b2af-2f1b73898542","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":2998782690},{"type":"v1","addr":"172.21.15.189:6825","nonce":2998782690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":2998782690},{"type":"v1","addr":"172.21.15.189:6827","nonce":2998782690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":2998782690},{"type":"v1","addr":"172.21.15.189:6831","nonce":2998782690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":2998782690},{"type":"v1","addr":"172.21.15.189:6829","nonce":2998782690}]},"public_addr":"172.21.15.189:6825/2998782690","cluster_addr":"172.21.15.189:6827/2998782690","heartbeat_back_addr":"172.21.15.189:6831/2998782690","heartbeat_front_addr":"172.21.15.189:6829/2998782690","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:40:47.980103+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:41:21.970795+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:41:55.160426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:42:30.400475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:43:05.705663+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:43:40.112312+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:44:16.781289+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:44:52.151187+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:45:28.055944+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:46:04.969301+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:46:40.918306+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:47:16.886874+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:6801/820049883":"2024-04-25T08:38:40.661315+0000","172.21.15.83:0/4272874138":"2024-04-25T08:38:40.661315+0000","172.21.15.83:0/4212994335":"2024-04-25T08:38:40.661315+0000","172.21.15.83:6801/1996371091":"2024-04-25T08:38:05.800856+0000","172.21.15.83:0/1820663683":"2024-04-25T08:38:05.800856+0000","172.21.15.83:0/3802897262":"2024-04-25T08:38:05.800856+0000","172.21.15.83:6800/1996371091":"2024-04-25T08:38:05.800856+0000","172.21.15.83:6800/820049883":"2024-04-25T08:38:40.661315+0000","172.21.15.83:0/1366125049":"2024-04-25T08:38:05.800856+0000","172.21.15.83:0/340452868":"2024-04-25T08:38:40.661315+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T08:48:25.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:25.652+0000 7efc68200640 1 -- 172.21.15.83:0/1121114494 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7efc480a3bb0 msgr2=0x7efc480a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:25.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:25.652+0000 7efc68200640 1 --2- 172.21.15.83:0/1121114494 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7efc480a3bb0 0x7efc480a6070 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7efc54005ab0 tx=0x7efc54005a20 comp rx=0 tx=0).stop 2024-04-24T08:48:25.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:25.652+0000 7efc68200640 1 -- 172.21.15.83:0/1121114494 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc60105e10 msgr2=0x7efc601a1ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:25.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:25.652+0000 7efc68200640 1 --2- 172.21.15.83:0/1121114494 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc60105e10 0x7efc601a1ae0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7efc5800eb10 tx=0x7efc5800c2d0 comp rx=0 tx=0).stop 2024-04-24T08:48:25.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:25.652+0000 7efc68200640 1 -- 172.21.15.83:0/1121114494 shutdown_connections 2024-04-24T08:48:25.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:25.652+0000 7efc68200640 1 --2- 172.21.15.83:0/1121114494 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7efc480a3bb0 0x7efc480a6070 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:25.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:25.652+0000 7efc68200640 1 --2- 172.21.15.83:0/1121114494 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc60105e10 0x7efc601a1ae0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:25.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:25.652+0000 7efc68200640 1 --2- 172.21.15.83:0/1121114494 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efc60105450 0x7efc6019aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:25.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:25.652+0000 7efc68200640 1 --2- 172.21.15.83:0/1121114494 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc60104260 0x7efc6019a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:25.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:25.652+0000 7efc68200640 1 -- 172.21.15.83:0/1121114494 >> 172.21.15.83:0/1121114494 conn(0x7efc600ff7f0 msgr2=0x7efc601087a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:48:25.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:25.652+0000 7efc68200640 1 -- 172.21.15.83:0/1121114494 shutdown_connections 2024-04-24T08:48:25.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:25.652+0000 7efc68200640 1 -- 172.21.15.83:0/1121114494 wait complete. 2024-04-24T08:48:25.676 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T08:41:56.834372+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T08:48:25.676 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T08:48:25.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:25.865+0000 7f58c6a00640 1 -- 172.21.15.83:0/2867107991 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58c0105470 msgr2=0x7f58c01058f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:25.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:25.865+0000 7f58c6a00640 1 --2- 172.21.15.83:0/2867107991 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58c0105470 0x7f58c01058f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f58a8009900 tx=0x7f58a802a3f0 comp rx=0 tx=0).stop 2024-04-24T08:48:25.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:25.865+0000 7f58c6a00640 1 -- 172.21.15.83:0/2867107991 shutdown_connections 2024-04-24T08:48:25.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:25.865+0000 7f58c6a00640 1 --2- 172.21.15.83:0/2867107991 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f58c0105e30 0x7f58c010c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:25.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:25.865+0000 7f58c6a00640 1 --2- 172.21.15.83:0/2867107991 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58c0105470 0x7f58c01058f0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:25.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:25.865+0000 7f58c6a00640 1 --2- 172.21.15.83:0/2867107991 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f58c0104280 0x7f58c01046a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:25.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:25.865+0000 7f58c6a00640 1 -- 172.21.15.83:0/2867107991 >> 172.21.15.83:0/2867107991 conn(0x7f58c00ff7f0 msgr2=0x7f58c0101c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:48:25.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:25.865+0000 7f58c6a00640 1 -- 172.21.15.83:0/2867107991 shutdown_connections 2024-04-24T08:48:25.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:25.865+0000 7f58c6a00640 1 -- 172.21.15.83:0/2867107991 wait complete. 2024-04-24T08:48:25.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:25.866+0000 7f58c6a00640 1 Processor -- start 2024-04-24T08:48:25.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:25.866+0000 7f58c6a00640 1 -- start start 2024-04-24T08:48:25.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:25.866+0000 7f58c6a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f58c0104280 0x7f58c019a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:25.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:25.866+0000 7f58c6a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58c0105470 0x7f58c019aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:25.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:25.866+0000 7f58c6a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f58c0105e30 0x7f58c01a1a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:25.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:25.866+0000 7f58c6a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f58c010fe10 con 0x7f58c0105470 2024-04-24T08:48:25.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:25.866+0000 7f58c6a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f58c010fc90 con 0x7f58c0104280 2024-04-24T08:48:25.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:25.866+0000 7f58c6a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f58c010ff90 con 0x7f58c0105e30 2024-04-24T08:48:25.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:25.867+0000 7f58bf400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58c0105470 0x7f58c019aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:25.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:25.867+0000 7f58bf400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58c0105470 0x7f58c019aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56704/0 (socket says 172.21.15.83:56704) 2024-04-24T08:48:25.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:25.867+0000 7f58bf400640 1 -- 172.21.15.83:0/4286057146 learned_addr learned my addr 172.21.15.83:0/4286057146 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:48:25.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:25.867+0000 7f58c4a00640 1 --2- 172.21.15.83:0/4286057146 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f58c0105e30 0x7f58c01a1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:25.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:25.867+0000 7f58bfe00640 1 --2- 172.21.15.83:0/4286057146 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f58c0104280 0x7f58c019a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:25.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:25.867+0000 7f58bf400640 1 -- 172.21.15.83:0/4286057146 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f58c0105e30 msgr2=0x7f58c01a1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:25.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:25.867+0000 7f58bf400640 1 --2- 172.21.15.83:0/4286057146 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f58c0105e30 0x7f58c01a1a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:25.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:25.867+0000 7f58bf400640 1 -- 172.21.15.83:0/4286057146 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f58c0104280 msgr2=0x7f58c019a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:25.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:25.867+0000 7f58bf400640 1 --2- 172.21.15.83:0/4286057146 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f58c0104280 0x7f58c019a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:25.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:25.867+0000 7f58bf400640 1 -- 172.21.15.83:0/4286057146 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58c01a20e0 con 0x7f58c0105470 2024-04-24T08:48:25.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:25.867+0000 7f58bfe00640 1 --2- 172.21.15.83:0/4286057146 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f58c0104280 0x7f58c019a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:48:25.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:25.867+0000 7f58bf400640 1 --2- 172.21.15.83:0/4286057146 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58c0105470 0x7f58c019aa00 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f58a80314e0 tx=0x7f58a8003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:48:25.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:25.868+0000 7f58bcc00640 1 -- 172.21.15.83:0/4286057146 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a803f070 con 0x7f58c0105470 2024-04-24T08:48:25.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:25.868+0000 7f58c6a00640 1 -- 172.21.15.83:0/4286057146 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58c01a2370 con 0x7f58c0105470 2024-04-24T08:48:25.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:25.868+0000 7f58c6a00640 1 -- 172.21.15.83:0/4286057146 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58c01a28b0 con 0x7f58c0105470 2024-04-24T08:48:25.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:25.868+0000 7f58bcc00640 1 -- 172.21.15.83:0/4286057146 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f58a803b910 con 0x7f58c0105470 2024-04-24T08:48:25.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:25.868+0000 7f58bcc00640 1 -- 172.21.15.83:0/4286057146 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a8049060 con 0x7f58c0105470 2024-04-24T08:48:25.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:25.869+0000 7f58c6a00640 1 -- 172.21.15.83:0/4286057146 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f589c0052c0 con 0x7f58c0105470 2024-04-24T08:48:25.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:25.871+0000 7f58bcc00640 1 -- 172.21.15.83:0/4286057146 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f58a8042070 con 0x7f58c0105470 2024-04-24T08:48:25.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:25.873+0000 7f58bcc00640 1 --2- 172.21.15.83:0/4286057146 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f58a40a3c30 0x7f58a40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:25.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:25.873+0000 7f58bcc00640 1 -- 172.21.15.83:0/4286057146 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f58a80f0720 con 0x7f58c0105470 2024-04-24T08:48:25.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:25.873+0000 7f58bfe00640 1 --2- 172.21.15.83:0/4286057146 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f58a40a3c30 0x7f58a40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:25.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:25.874+0000 7f58bfe00640 1 --2- 172.21.15.83:0/4286057146 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f58a40a3c30 0x7f58a40a60f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f58c01052d0 tx=0x7f58b0009540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:48:25.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:25.876+0000 7f58bcc00640 1 -- 172.21.15.83:0/4286057146 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f58a80bd5a0 con 0x7f58c0105470 2024-04-24T08:48:26.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:25 smithi083 ceph-mon[27907]: pgmap v363: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:48:26.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:25 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/2124240226' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:48:26.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:25 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/1121114494' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T08:48:26.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:25 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:48:26.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:25 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:48:26.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:25 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:48:26.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:25 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:48:26.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:25 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:48:26.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:25 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:48:26.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:25 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:48:26.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:26.082+0000 7f58c6a00640 1 -- 172.21.15.83:0/4286057146 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f589c005880 con 0x7f58c0105470 2024-04-24T08:48:26.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:26.083+0000 7f58bcc00640 1 -- 172.21.15.83:0/4286057146 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7f58a80bcea0 con 0x7f58c0105470 2024-04-24T08:48:26.084 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-24T08:48:26.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:26.086+0000 7f58c6a00640 1 -- 172.21.15.83:0/4286057146 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f58a40a3c30 msgr2=0x7f58a40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:26.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:26.086+0000 7f58c6a00640 1 --2- 172.21.15.83:0/4286057146 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f58a40a3c30 0x7f58a40a60f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f58c01052d0 tx=0x7f58b0009540 comp rx=0 tx=0).stop 2024-04-24T08:48:26.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:26.086+0000 7f58c6a00640 1 -- 172.21.15.83:0/4286057146 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58c0105470 msgr2=0x7f58c019aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:26.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:26.086+0000 7f58c6a00640 1 --2- 172.21.15.83:0/4286057146 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58c0105470 0x7f58c019aa00 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f58a80314e0 tx=0x7f58a8003c80 comp rx=0 tx=0).stop 2024-04-24T08:48:26.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:26.086+0000 7f58c6a00640 1 -- 172.21.15.83:0/4286057146 shutdown_connections 2024-04-24T08:48:26.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:26.086+0000 7f58c6a00640 1 --2- 172.21.15.83:0/4286057146 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f58c0105e30 0x7f58c01a1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:26.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:26.086+0000 7f58c6a00640 1 --2- 172.21.15.83:0/4286057146 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f58a40a3c30 0x7f58a40a60f0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:26.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:26.086+0000 7f58c6a00640 1 --2- 172.21.15.83:0/4286057146 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58c0105470 0x7f58c019aa00 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:26.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:26.086+0000 7f58c6a00640 1 --2- 172.21.15.83:0/4286057146 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f58c0104280 0x7f58c019a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:26.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:26.086+0000 7f58c6a00640 1 -- 172.21.15.83:0/4286057146 >> 172.21.15.83:0/4286057146 conn(0x7f58c00ff7f0 msgr2=0x7f58c0108790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:48:26.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:26.086+0000 7f58c6a00640 1 -- 172.21.15.83:0/4286057146 shutdown_connections 2024-04-24T08:48:26.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:26.086+0000 7f58c6a00640 1 -- 172.21.15.83:0/4286057146 wait complete. 2024-04-24T08:48:26.105 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-24T08:48:26.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:25 smithi138 ceph-mon[31441]: pgmap v363: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:48:26.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:25 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/2124240226' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:48:26.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:25 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/1121114494' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T08:48:26.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:25 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:48:26.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:25 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:48:26.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:25 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:48:26.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:25 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:48:26.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:25 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:48:26.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:25 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:48:26.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:25 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:48:26.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:25 smithi189 ceph-mon[31791]: pgmap v363: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:48:26.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:25 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/2124240226' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:48:26.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:25 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/1121114494' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T08:48:26.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:25 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:48:26.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:25 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:48:26.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:25 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:48:26.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:25 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:48:26.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:25 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:48:26.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:25 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:48:26.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:25 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:48:26.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:26.295+0000 7feb72800640 1 -- 172.21.15.83:0/2527304959 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb6c102ff0 msgr2=0x7feb6c103470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:26.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:26.295+0000 7feb72800640 1 --2- 172.21.15.83:0/2527304959 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb6c102ff0 0x7feb6c103470 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7feb58009900 tx=0x7feb5802a410 comp rx=0 tx=0).stop 2024-04-24T08:48:26.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:26.295+0000 7feb72800640 1 -- 172.21.15.83:0/2527304959 shutdown_connections 2024-04-24T08:48:26.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:26.295+0000 7feb72800640 1 --2- 172.21.15.83:0/2527304959 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb6c1039b0 0x7feb6c10a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:26.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:26.295+0000 7feb72800640 1 --2- 172.21.15.83:0/2527304959 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb6c102ff0 0x7feb6c103470 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:26.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:26.295+0000 7feb72800640 1 --2- 172.21.15.83:0/2527304959 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feb6c101ea0 0x7feb6c1022c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:26.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:26.295+0000 7feb72800640 1 -- 172.21.15.83:0/2527304959 >> 172.21.15.83:0/2527304959 conn(0x7feb6c0fd410 msgr2=0x7feb6c0ff850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:48:26.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:26.295+0000 7feb72800640 1 -- 172.21.15.83:0/2527304959 shutdown_connections 2024-04-24T08:48:26.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:26.295+0000 7feb72800640 1 -- 172.21.15.83:0/2527304959 wait complete. 2024-04-24T08:48:26.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:26.296+0000 7feb72800640 1 Processor -- start 2024-04-24T08:48:26.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:26.296+0000 7feb72800640 1 -- start start 2024-04-24T08:48:26.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:26.296+0000 7feb72800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb6c101ea0 0x7feb6c19a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:26.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:26.296+0000 7feb72800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb6c102ff0 0x7feb6c19a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:26.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:26.296+0000 7feb72800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feb6c1039b0 0x7feb6c1a18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:26.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:26.296+0000 7feb72800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7feb6c10da50 con 0x7feb6c102ff0 2024-04-24T08:48:26.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:26.296+0000 7feb72800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7feb6c10d8d0 con 0x7feb6c101ea0 2024-04-24T08:48:26.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:26.296+0000 7feb72800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7feb6c10dbd0 con 0x7feb6c1039b0 2024-04-24T08:48:26.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:26.297+0000 7feb70a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb6c102ff0 0x7feb6c19a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:26.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:26.297+0000 7feb70a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb6c102ff0 0x7feb6c19a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56718/0 (socket says 172.21.15.83:56718) 2024-04-24T08:48:26.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:26.297+0000 7feb70a00640 1 -- 172.21.15.83:0/540791650 learned_addr learned my addr 172.21.15.83:0/540791650 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:48:26.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:26.297+0000 7feb71400640 1 --2- 172.21.15.83:0/540791650 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb6c101ea0 0x7feb6c19a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:26.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:26.297+0000 7feb71e00640 1 --2- 172.21.15.83:0/540791650 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feb6c1039b0 0x7feb6c1a18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:26.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:26.297+0000 7feb70a00640 1 -- 172.21.15.83:0/540791650 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feb6c1039b0 msgr2=0x7feb6c1a18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:26.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:26.297+0000 7feb70a00640 1 --2- 172.21.15.83:0/540791650 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feb6c1039b0 0x7feb6c1a18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:26.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:26.297+0000 7feb70a00640 1 -- 172.21.15.83:0/540791650 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb6c101ea0 msgr2=0x7feb6c19a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:26.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:26.297+0000 7feb70a00640 1 --2- 172.21.15.83:0/540791650 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb6c101ea0 0x7feb6c19a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:26.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:26.297+0000 7feb70a00640 1 -- 172.21.15.83:0/540791650 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb6c1a1f30 con 0x7feb6c102ff0 2024-04-24T08:48:26.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:26.298+0000 7feb70a00640 1 --2- 172.21.15.83:0/540791650 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb6c102ff0 0x7feb6c19a850 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7feb580314e0 tx=0x7feb58003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:48:26.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:26.298+0000 7feb6a000640 1 -- 172.21.15.83:0/540791650 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb5803f070 con 0x7feb6c102ff0 2024-04-24T08:48:26.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:26.298+0000 7feb6a000640 1 -- 172.21.15.83:0/540791650 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feb58031e40 con 0x7feb6c102ff0 2024-04-24T08:48:26.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:26.298+0000 7feb72800640 1 -- 172.21.15.83:0/540791650 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb6c1a21c0 con 0x7feb6c102ff0 2024-04-24T08:48:26.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:26.298+0000 7feb6a000640 1 -- 172.21.15.83:0/540791650 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb58035030 con 0x7feb6c102ff0 2024-04-24T08:48:26.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:26.298+0000 7feb72800640 1 -- 172.21.15.83:0/540791650 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb6c1a26b0 con 0x7feb6c102ff0 2024-04-24T08:48:26.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:26.299+0000 7feb72800640 1 -- 172.21.15.83:0/540791650 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb6c1a29d0 con 0x7feb6c102ff0 2024-04-24T08:48:26.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:26.303+0000 7feb6a000640 1 -- 172.21.15.83:0/540791650 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7feb58042070 con 0x7feb6c102ff0 2024-04-24T08:48:26.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:26.305+0000 7feb6a000640 1 --2- 172.21.15.83:0/540791650 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7feb540a3bb0 0x7feb540a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:26.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:26.305+0000 7feb6a000640 1 -- 172.21.15.83:0/540791650 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7feb580f0860 con 0x7feb6c102ff0 2024-04-24T08:48:26.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:26.305+0000 7feb71400640 1 --2- 172.21.15.83:0/540791650 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7feb540a3bb0 0x7feb540a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:26.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:26.306+0000 7feb71400640 1 --2- 172.21.15.83:0/540791650 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7feb540a3bb0 0x7feb540a6070 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7feb60005fd0 tx=0x7feb6000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:48:26.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:26.310+0000 7feb6a000640 1 -- 172.21.15.83:0/540791650 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7feb580bd6e0 con 0x7feb6c102ff0 2024-04-24T08:48:26.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:26.529+0000 7feb72800640 1 -- 172.21.15.83:0/540791650 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7feb6c1a2ff0 con 0x7feb6c102ff0 2024-04-24T08:48:26.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:26.530+0000 7feb6a000640 1 -- 172.21.15.83:0/540791650 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v81) ==== 84+0+65 (secure 0 0 0) 0x7feb580bcfe0 con 0x7feb6c102ff0 2024-04-24T08:48:26.530 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:48:26.531 INFO:teuthology.orchestra.run.smithi083.stdout:[ 2024-04-24T08:48:26.531 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T08:48:26.531 INFO:teuthology.orchestra.run.smithi083.stdout: "poolnum": 1, 2024-04-24T08:48:26.531 INFO:teuthology.orchestra.run.smithi083.stdout: "poolname": ".mgr" 2024-04-24T08:48:26.531 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T08:48:26.531 INFO:teuthology.orchestra.run.smithi083.stdout:] 2024-04-24T08:48:26.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:26.532+0000 7feb72800640 1 -- 172.21.15.83:0/540791650 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7feb540a3bb0 msgr2=0x7feb540a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:26.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:26.532+0000 7feb72800640 1 --2- 172.21.15.83:0/540791650 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7feb540a3bb0 0x7feb540a6070 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7feb60005fd0 tx=0x7feb6000b390 comp rx=0 tx=0).stop 2024-04-24T08:48:26.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:26.532+0000 7feb72800640 1 -- 172.21.15.83:0/540791650 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb6c102ff0 msgr2=0x7feb6c19a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:26.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:26.532+0000 7feb72800640 1 --2- 172.21.15.83:0/540791650 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb6c102ff0 0x7feb6c19a850 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7feb580314e0 tx=0x7feb58003c80 comp rx=0 tx=0).stop 2024-04-24T08:48:26.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:26.532+0000 7feb72800640 1 -- 172.21.15.83:0/540791650 shutdown_connections 2024-04-24T08:48:26.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:26.532+0000 7feb72800640 1 --2- 172.21.15.83:0/540791650 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feb6c1039b0 0x7feb6c1a18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:26.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:26.532+0000 7feb72800640 1 --2- 172.21.15.83:0/540791650 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7feb540a3bb0 0x7feb540a6070 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:26.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:26.532+0000 7feb72800640 1 --2- 172.21.15.83:0/540791650 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb6c102ff0 0x7feb6c19a850 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:26.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:26.532+0000 7feb72800640 1 --2- 172.21.15.83:0/540791650 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb6c101ea0 0x7feb6c19a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:26.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:26.532+0000 7feb72800640 1 -- 172.21.15.83:0/540791650 >> 172.21.15.83:0/540791650 conn(0x7feb6c0fd410 msgr2=0x7feb6c074230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:48:26.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:26.532+0000 7feb72800640 1 -- 172.21.15.83:0/540791650 shutdown_connections 2024-04-24T08:48:26.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:26.532+0000 7feb72800640 1 -- 172.21.15.83:0/540791650 wait complete. 2024-04-24T08:48:26.553 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-24T08:48:26.553 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-24T08:48:26.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:26.748+0000 7fb84ea00640 1 -- 172.21.15.83:0/1709164810 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb848074a60 msgr2=0x7fb848074ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:26.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:26.748+0000 7fb84ea00640 1 --2- 172.21.15.83:0/1709164810 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb848074a60 0x7fb848074ee0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb834009900 tx=0x7fb83402a410 comp rx=0 tx=0).stop 2024-04-24T08:48:26.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:26.748+0000 7fb84ea00640 1 -- 172.21.15.83:0/1709164810 shutdown_connections 2024-04-24T08:48:26.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:26.748+0000 7fb84ea00640 1 --2- 172.21.15.83:0/1709164810 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb848109e50 0x7fb84810c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:26.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:26.748+0000 7fb84ea00640 1 --2- 172.21.15.83:0/1709164810 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb848074a60 0x7fb848074ee0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:26.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:26.748+0000 7fb84ea00640 1 --2- 172.21.15.83:0/1709164810 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb848076040 0x7fb848074490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:26.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:26.748+0000 7fb84ea00640 1 -- 172.21.15.83:0/1709164810 >> 172.21.15.83:0/1709164810 conn(0x7fb8480fd500 msgr2=0x7fb8480ff940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:48:26.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:26.748+0000 7fb84ea00640 1 -- 172.21.15.83:0/1709164810 shutdown_connections 2024-04-24T08:48:26.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:26.748+0000 7fb84ea00640 1 -- 172.21.15.83:0/1709164810 wait complete. 2024-04-24T08:48:26.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:26.749+0000 7fb84ea00640 1 Processor -- start 2024-04-24T08:48:26.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:26.749+0000 7fb84ea00640 1 -- start start 2024-04-24T08:48:26.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:26.749+0000 7fb84ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb848074a60 0x7fb84819a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:26.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:26.749+0000 7fb84ea00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb848076040 0x7fb84819a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:26.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:26.749+0000 7fb84ea00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb848109e50 0x7fb8481a1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:26.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:26.749+0000 7fb84ea00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb84810fc50 con 0x7fb848074a60 2024-04-24T08:48:26.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:26.749+0000 7fb84ea00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb84810fad0 con 0x7fb848109e50 2024-04-24T08:48:26.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:26.749+0000 7fb84ea00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb84810fdd0 con 0x7fb848076040 2024-04-24T08:48:26.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:26.750+0000 7fb84d600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb848074a60 0x7fb84819a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:26.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:26.750+0000 7fb84d600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb848074a60 0x7fb84819a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56742/0 (socket says 172.21.15.83:56742) 2024-04-24T08:48:26.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:26.750+0000 7fb84d600640 1 -- 172.21.15.83:0/785953767 learned_addr learned my addr 172.21.15.83:0/785953767 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:48:26.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:26.750+0000 7fb84cc00640 1 --2- 172.21.15.83:0/785953767 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb848076040 0x7fb84819a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:26.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:26.750+0000 7fb84e000640 1 --2- 172.21.15.83:0/785953767 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb848109e50 0x7fb8481a1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:26.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:26.750+0000 7fb84d600640 1 -- 172.21.15.83:0/785953767 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb848076040 msgr2=0x7fb84819a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:26.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:26.750+0000 7fb84d600640 1 --2- 172.21.15.83:0/785953767 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb848076040 0x7fb84819a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:26.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:26.750+0000 7fb84d600640 1 -- 172.21.15.83:0/785953767 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb848109e50 msgr2=0x7fb8481a1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:26.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:26.750+0000 7fb84d600640 1 --2- 172.21.15.83:0/785953767 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb848109e50 0x7fb8481a1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:26.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:26.750+0000 7fb84d600640 1 -- 172.21.15.83:0/785953767 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8481a1fa0 con 0x7fb848074a60 2024-04-24T08:48:26.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:26.750+0000 7fb84e000640 1 --2- 172.21.15.83:0/785953767 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb848109e50 0x7fb8481a1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:48:26.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:26.750+0000 7fb84d600640 1 --2- 172.21.15.83:0/785953767 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb848074a60 0x7fb84819a380 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb83c00d770 tx=0x7fb83c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:48:26.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:26.751+0000 7fb846000640 1 -- 172.21.15.83:0/785953767 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb83c017070 con 0x7fb848074a60 2024-04-24T08:48:26.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:26.751+0000 7fb84ea00640 1 -- 172.21.15.83:0/785953767 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8481a2290 con 0x7fb848074a60 2024-04-24T08:48:26.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:26.751+0000 7fb846000640 1 -- 172.21.15.83:0/785953767 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb83c0054d0 con 0x7fb848074a60 2024-04-24T08:48:26.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:26.751+0000 7fb846000640 1 -- 172.21.15.83:0/785953767 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb83c004cf0 con 0x7fb848074a60 2024-04-24T08:48:26.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:26.751+0000 7fb84ea00640 1 -- 172.21.15.83:0/785953767 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8481a27e0 con 0x7fb848074a60 2024-04-24T08:48:26.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:26.752+0000 7fb84ea00640 1 -- 172.21.15.83:0/785953767 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb848108b80 con 0x7fb848074a60 2024-04-24T08:48:26.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:26.754+0000 7fb846000640 1 -- 172.21.15.83:0/785953767 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb83c004e90 con 0x7fb848074a60 2024-04-24T08:48:26.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:26.755+0000 7fb846000640 1 --2- 172.21.15.83:0/785953767 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fb8300a3c30 0x7fb8300a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:26.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:26.755+0000 7fb846000640 1 -- 172.21.15.83:0/785953767 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fb83c0cfa00 con 0x7fb848074a60 2024-04-24T08:48:26.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:26.756+0000 7fb84cc00640 1 --2- 172.21.15.83:0/785953767 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fb8300a3c30 0x7fb8300a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:26.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:26.756+0000 7fb84cc00640 1 --2- 172.21.15.83:0/785953767 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fb8300a3c30 0x7fb8300a60f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb84819b890 tx=0x7fb8340314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:48:26.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:26.760+0000 7fb846000640 1 -- 172.21.15.83:0/785953767 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb83c09c880 con 0x7fb848074a60 2024-04-24T08:48:26.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:26.967+0000 7fb84ea00640 1 -- 172.21.15.83:0/785953767 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fb84802d620 con 0x7fb848074a60 2024-04-24T08:48:27.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:26 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/4286057146' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T08:48:27.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:26 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/540791650' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-24T08:48:27.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:26 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/4286057146' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T08:48:27.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:26 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/540791650' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-24T08:48:27.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:26 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/4286057146' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T08:48:27.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:26 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/540791650' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-24T08:48:27.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:27.920+0000 7fb846000640 1 -- 172.21.15.83:0/785953767 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v82) ==== 138+0+0 (secure 0 0 0) 0x7fb83c09c180 con 0x7fb848074a60 2024-04-24T08:48:27.922 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_metadata' created 2024-04-24T08:48:27.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:27.924+0000 7fb84ea00640 1 -- 172.21.15.83:0/785953767 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fb8300a3c30 msgr2=0x7fb8300a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:27.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:27.924+0000 7fb84ea00640 1 --2- 172.21.15.83:0/785953767 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fb8300a3c30 0x7fb8300a60f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb84819b890 tx=0x7fb8340314a0 comp rx=0 tx=0).stop 2024-04-24T08:48:27.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:27.924+0000 7fb84ea00640 1 -- 172.21.15.83:0/785953767 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb848074a60 msgr2=0x7fb84819a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:27.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:27.924+0000 7fb84ea00640 1 --2- 172.21.15.83:0/785953767 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb848074a60 0x7fb84819a380 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb83c00d770 tx=0x7fb83c00dc30 comp rx=0 tx=0).stop 2024-04-24T08:48:27.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:27.924+0000 7fb84ea00640 1 -- 172.21.15.83:0/785953767 shutdown_connections 2024-04-24T08:48:27.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:27.924+0000 7fb84ea00640 1 --2- 172.21.15.83:0/785953767 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb848109e50 0x7fb8481a1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:27.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:27.924+0000 7fb84ea00640 1 --2- 172.21.15.83:0/785953767 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb848076040 0x7fb84819a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:27.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:27.924+0000 7fb84ea00640 1 --2- 172.21.15.83:0/785953767 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fb8300a3c30 0x7fb8300a60f0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:27.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:27.924+0000 7fb84ea00640 1 --2- 172.21.15.83:0/785953767 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb848074a60 0x7fb84819a380 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:27.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:27.924+0000 7fb84ea00640 1 -- 172.21.15.83:0/785953767 >> 172.21.15.83:0/785953767 conn(0x7fb8480fd500 msgr2=0x7fb848105d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:48:27.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:27.924+0000 7fb84ea00640 1 -- 172.21.15.83:0/785953767 shutdown_connections 2024-04-24T08:48:27.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:27.924+0000 7fb84ea00640 1 -- 172.21.15.83:0/785953767 wait complete. 2024-04-24T08:48:27.949 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-04-24T08:48:28.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:28.129+0000 7f840ae00640 1 -- 172.21.15.83:0/207065648 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8404102040 msgr2=0x7f8404102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:28.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:28.129+0000 7f840ae00640 1 --2- 172.21.15.83:0/207065648 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8404102040 0x7f8404102460 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f83f4009900 tx=0x7f83f402a410 comp rx=0 tx=0).stop 2024-04-24T08:48:28.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:28.129+0000 7f840ae00640 1 -- 172.21.15.83:0/207065648 shutdown_connections 2024-04-24T08:48:28.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:28.129+0000 7f840ae00640 1 --2- 172.21.15.83:0/207065648 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8404103bf0 0x7f840410a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:28.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:28.129+0000 7f840ae00640 1 --2- 172.21.15.83:0/207065648 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8404103230 0x7f84041036b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:28.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:28.129+0000 7f840ae00640 1 --2- 172.21.15.83:0/207065648 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8404102040 0x7f8404102460 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:28.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:28.129+0000 7f840ae00640 1 -- 172.21.15.83:0/207065648 >> 172.21.15.83:0/207065648 conn(0x7f84040fd5b0 msgr2=0x7f84040ff9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:48:28.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:28.130+0000 7f840ae00640 1 -- 172.21.15.83:0/207065648 shutdown_connections 2024-04-24T08:48:28.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:28.130+0000 7f840ae00640 1 -- 172.21.15.83:0/207065648 wait complete. 2024-04-24T08:48:28.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:28.131+0000 7f840ae00640 1 Processor -- start 2024-04-24T08:48:28.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:28.131+0000 7f840ae00640 1 -- start start 2024-04-24T08:48:28.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:28.131+0000 7f840ae00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8404102040 0x7f840419a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:28.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:28.131+0000 7f840ae00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8404103230 0x7f840419aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:28.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:28.131+0000 7f840ae00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8404103bf0 0x7f84041a1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:28.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:28.131+0000 7f840ae00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f840410dbd0 con 0x7f8404103bf0 2024-04-24T08:48:28.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:28.131+0000 7f840ae00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f840410da50 con 0x7f8404102040 2024-04-24T08:48:28.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:28.131+0000 7f840ae00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f840410dd50 con 0x7f8404103230 2024-04-24T08:48:28.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:28.132+0000 7f8408e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8404103bf0 0x7f84041a1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:28.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:28.132+0000 7f8403e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8404102040 0x7f840419a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:28.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:28.132+0000 7f8403400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8404103230 0x7f840419aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:28.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:28.132+0000 7f8403e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8404102040 0x7f840419a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:43444/0 (socket says 172.21.15.83:43444) 2024-04-24T08:48:28.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:28.132+0000 7f8408e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8404103bf0 0x7f84041a1a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56754/0 (socket says 172.21.15.83:56754) 2024-04-24T08:48:28.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:28.132+0000 7f8403e00640 1 -- 172.21.15.83:0/4116635994 learned_addr learned my addr 172.21.15.83:0/4116635994 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:48:28.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:28.132+0000 7f8403400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8404103230 0x7f840419aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:58486/0 (socket says 172.21.15.83:58486) 2024-04-24T08:48:28.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:28.132+0000 7f8403e00640 1 -- 172.21.15.83:0/4116635994 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8404103230 msgr2=0x7f840419aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:28.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:28.132+0000 7f8403e00640 1 --2- 172.21.15.83:0/4116635994 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8404103230 0x7f840419aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:28.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:28.132+0000 7f8403e00640 1 -- 172.21.15.83:0/4116635994 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8404103bf0 msgr2=0x7f84041a1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:28.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:28.132+0000 7f8403e00640 1 --2- 172.21.15.83:0/4116635994 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8404103bf0 0x7f84041a1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:28.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:28.132+0000 7f8403e00640 1 -- 172.21.15.83:0/4116635994 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84041a20f0 con 0x7f8404102040 2024-04-24T08:48:28.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:28.132+0000 7f8408e00640 1 --2- 172.21.15.83:0/4116635994 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8404103bf0 0x7f84041a1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T08:48:28.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:28.132+0000 7f8403e00640 1 --2- 172.21.15.83:0/4116635994 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8404102040 0x7f840419a4d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f83f40314e0 tx=0x7f83f4003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:48:28.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:28.133+0000 7f8400c00640 1 -- 172.21.15.83:0/4116635994 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83f403f070 con 0x7f8404102040 2024-04-24T08:48:28.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:28.133+0000 7f840ae00640 1 -- 172.21.15.83:0/4116635994 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84041a2380 con 0x7f8404102040 2024-04-24T08:48:28.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:28.133+0000 7f8400c00640 1 -- 172.21.15.83:0/4116635994 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f83f4031c40 con 0x7f8404102040 2024-04-24T08:48:28.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:28.133+0000 7f8400c00640 1 -- 172.21.15.83:0/4116635994 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83f4035030 con 0x7f8404102040 2024-04-24T08:48:28.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:28.133+0000 7f840ae00640 1 -- 172.21.15.83:0/4116635994 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84041a2870 con 0x7f8404102040 2024-04-24T08:48:28.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:28.134+0000 7f840ae00640 1 -- 172.21.15.83:0/4116635994 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84041a2ad0 con 0x7f8404102040 2024-04-24T08:48:28.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:28.136+0000 7f8400c00640 1 -- 172.21.15.83:0/4116635994 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f83f4042070 con 0x7f8404102040 2024-04-24T08:48:28.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:28.137+0000 7f8400c00640 1 --2- 172.21.15.83:0/4116635994 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f83e80a3bb0 0x7f83e80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:28.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:28.138+0000 7f8400c00640 1 -- 172.21.15.83:0/4116635994 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7497+0+0 (secure 0 0 0) 0x7f83f40f0fa0 con 0x7f8404102040 2024-04-24T08:48:28.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:28.138+0000 7f8403400640 1 --2- 172.21.15.83:0/4116635994 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f83e80a3bb0 0x7f83e80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:28.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:28.138+0000 7f8403400640 1 --2- 172.21.15.83:0/4116635994 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f83e80a3bb0 0x7f83e80a6070 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f840419b9e0 tx=0x7f83f0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:48:28.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:28.142+0000 7f8400c00640 1 -- 172.21.15.83:0/4116635994 <== mon.2 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f83f4031df0 con 0x7f8404102040 2024-04-24T08:48:28.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:27 smithi138 ceph-mon[31441]: pgmap v364: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:48:28.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:27 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:48:28.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:27 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:48:28.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:27 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:48:28.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:27 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/785953767' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-24T08:48:28.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:27 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:48:28.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:27 smithi189 ceph-mon[31791]: pgmap v364: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:48:28.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:27 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:48:28.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:27 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:48:28.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:27 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:48:28.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:27 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/785953767' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-24T08:48:28.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:27 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:48:28.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:27 smithi083 ceph-mon[27907]: pgmap v364: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:48:28.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:27 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:48:28.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:27 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:48:28.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:27 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:48:28.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:27 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/785953767' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-24T08:48:28.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:27 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:48:28.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:28.340+0000 7f840ae00640 1 -- 172.21.15.83:0/4116635994 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7f84041a31c0 con 0x7f8404102040 2024-04-24T08:48:28.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:28.948+0000 7f8400c00640 1 -- 172.21.15.83:0/4116635994 <== mon.2 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v83) ==== 171+0+0 (secure 0 0 0) 0x7f83f40bd5c0 con 0x7f8404102040 2024-04-24T08:48:28.949 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_data' created 2024-04-24T08:48:28.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:28.952+0000 7f840ae00640 1 -- 172.21.15.83:0/4116635994 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f83e80a3bb0 msgr2=0x7f83e80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:28.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:28.952+0000 7f840ae00640 1 --2- 172.21.15.83:0/4116635994 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f83e80a3bb0 0x7f83e80a6070 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f840419b9e0 tx=0x7f83f0005f90 comp rx=0 tx=0).stop 2024-04-24T08:48:28.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:28.952+0000 7f840ae00640 1 -- 172.21.15.83:0/4116635994 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8404102040 msgr2=0x7f840419a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:28.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:28.952+0000 7f840ae00640 1 --2- 172.21.15.83:0/4116635994 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8404102040 0x7f840419a4d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f83f40314e0 tx=0x7f83f4003c10 comp rx=0 tx=0).stop 2024-04-24T08:48:28.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:28.953+0000 7f840ae00640 1 -- 172.21.15.83:0/4116635994 shutdown_connections 2024-04-24T08:48:28.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:28.953+0000 7f840ae00640 1 --2- 172.21.15.83:0/4116635994 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f83e80a3bb0 0x7f83e80a6070 secure :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f840419b9e0 tx=0x7f83f0005f90 comp rx=0 tx=0).stop 2024-04-24T08:48:28.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:28.953+0000 7f840ae00640 1 --2- 172.21.15.83:0/4116635994 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8404103bf0 0x7f84041a1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:28.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:28.953+0000 7f840ae00640 1 --2- 172.21.15.83:0/4116635994 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8404103230 0x7f840419aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:28.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:28.953+0000 7f840ae00640 1 --2- 172.21.15.83:0/4116635994 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8404102040 0x7f840419a4d0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:28.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:28.953+0000 7f840ae00640 1 -- 172.21.15.83:0/4116635994 >> 172.21.15.83:0/4116635994 conn(0x7f84040fd5b0 msgr2=0x7f8404074230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:48:28.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:28.954+0000 7f840ae00640 1 -- 172.21.15.83:0/4116635994 shutdown_connections 2024-04-24T08:48:28.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:28.954+0000 7f840ae00640 1 -- 172.21.15.83:0/4116635994 wait complete. 2024-04-24T08:48:28.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:28 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/785953767' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-24T08:48:28.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:28 smithi138 ceph-mon[31441]: osdmap e82: 12 total, 12 up, 12 in 2024-04-24T08:48:28.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:28 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:48:28.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:28 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:48:28.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:28 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/4116635994' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-04-24T08:48:28.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:28 smithi138 ceph-mon[31441]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-04-24T08:48:28.984 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-24T08:48:29.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:29.166+0000 7f31a4000640 1 -- 172.21.15.83:0/1780646012 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f319c10dc60 msgr2=0x7f319c110050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:29.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:29.166+0000 7f31a4000640 1 --2- 172.21.15.83:0/1780646012 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f319c10dc60 0x7f319c110050 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f3188009900 tx=0x7f318802a410 comp rx=0 tx=0).stop 2024-04-24T08:48:29.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:29.166+0000 7f31a4000640 1 -- 172.21.15.83:0/1780646012 shutdown_connections 2024-04-24T08:48:29.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:29.166+0000 7f31a4000640 1 --2- 172.21.15.83:0/1780646012 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f319c110590 0x7f319c112980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:29.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:29.166+0000 7f31a4000640 1 --2- 172.21.15.83:0/1780646012 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f319c10dc60 0x7f319c110050 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:29.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:29.166+0000 7f31a4000640 1 --2- 172.21.15.83:0/1780646012 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f319c10b330 0x7f319c10d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:29.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:29.166+0000 7f31a4000640 1 -- 172.21.15.83:0/1780646012 >> 172.21.15.83:0/1780646012 conn(0x7f319c06fed0 msgr2=0x7f319c072310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:48:29.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:29.166+0000 7f31a4000640 1 -- 172.21.15.83:0/1780646012 shutdown_connections 2024-04-24T08:48:29.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:29.167+0000 7f31a4000640 1 -- 172.21.15.83:0/1780646012 wait complete. 2024-04-24T08:48:29.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:29.167+0000 7f31a4000640 1 Processor -- start 2024-04-24T08:48:29.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:29.168+0000 7f31a4000640 1 -- start start 2024-04-24T08:48:29.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:29.168+0000 7f31a4000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f319c10b330 0x7f319c1a0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:29.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:29.168+0000 7f31a4000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f319c10dc60 0x7f319c1a1010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:29.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:29.168+0000 7f31a4000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f319c110590 0x7f319c1a8080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:29.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:29.168+0000 7f31a4000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f319c116350 con 0x7f319c110590 2024-04-24T08:48:29.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:29.168+0000 7f31a4000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f319c1161d0 con 0x7f319c10b330 2024-04-24T08:48:29.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:29.168+0000 7f31a4000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f319c1164d0 con 0x7f319c10dc60 2024-04-24T08:48:29.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:29.168+0000 7f31a2000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f319c110590 0x7f319c1a8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:29.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:29.168+0000 7f31a2000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f319c110590 0x7f319c1a8080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51476/0 (socket says 172.21.15.83:51476) 2024-04-24T08:48:29.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:29.168+0000 7f31a2000640 1 -- 172.21.15.83:0/742481815 learned_addr learned my addr 172.21.15.83:0/742481815 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:48:29.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:29.168+0000 7f31a0c00640 1 --2- 172.21.15.83:0/742481815 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f319c10dc60 0x7f319c1a1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:29.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:29.168+0000 7f31a1600640 1 --2- 172.21.15.83:0/742481815 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f319c10b330 0x7f319c1a0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:29.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:29.169+0000 7f31a2000640 1 -- 172.21.15.83:0/742481815 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f319c10dc60 msgr2=0x7f319c1a1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:29.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:29.169+0000 7f31a2000640 1 --2- 172.21.15.83:0/742481815 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f319c10dc60 0x7f319c1a1010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:29.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:29.169+0000 7f31a2000640 1 -- 172.21.15.83:0/742481815 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f319c10b330 msgr2=0x7f319c1a0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:29.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:29.169+0000 7f31a2000640 1 --2- 172.21.15.83:0/742481815 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f319c10b330 0x7f319c1a0ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:29.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:29.169+0000 7f31a2000640 1 -- 172.21.15.83:0/742481815 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f319c1a86f0 con 0x7f319c110590 2024-04-24T08:48:29.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:29.169+0000 7f31a2000640 1 --2- 172.21.15.83:0/742481815 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f319c110590 0x7f319c1a8080 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f319400eaa0 tx=0x7f319400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:48:29.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:29.169+0000 7f319a000640 1 -- 172.21.15.83:0/742481815 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f319400cdb0 con 0x7f319c110590 2024-04-24T08:48:29.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:29.169+0000 7f31a4000640 1 -- 172.21.15.83:0/742481815 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f319c1a89e0 con 0x7f319c110590 2024-04-24T08:48:29.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:29.169+0000 7f319a000640 1 -- 172.21.15.83:0/742481815 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3194005020 con 0x7f319c110590 2024-04-24T08:48:29.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:29.170+0000 7f319a000640 1 -- 172.21.15.83:0/742481815 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31940116d0 con 0x7f319c110590 2024-04-24T08:48:29.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:29.170+0000 7f31a4000640 1 -- 172.21.15.83:0/742481815 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f319c1a8f00 con 0x7f319c110590 2024-04-24T08:48:29.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:29.171+0000 7f31a4000640 1 -- 172.21.15.83:0/742481815 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f319c0719c0 con 0x7f319c110590 2024-04-24T08:48:29.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:29.173+0000 7f319a000640 1 -- 172.21.15.83:0/742481815 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f31940051d0 con 0x7f319c110590 2024-04-24T08:48:29.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:29.174+0000 7f319a000640 1 --2- 172.21.15.83:0/742481815 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f31840a3c30 0x7f31840a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:29.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:29.174+0000 7f319a000640 1 -- 172.21.15.83:0/742481815 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7879+0+0 (secure 0 0 0) 0x7f31940d0470 con 0x7f319c110590 2024-04-24T08:48:29.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:29.174+0000 7f31a1600640 1 --2- 172.21.15.83:0/742481815 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f31840a3c30 0x7f31840a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:29.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:29.175+0000 7f31a1600640 1 --2- 172.21.15.83:0/742481815 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f31840a3c30 0x7f31840a60f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3190009b60 tx=0x7f3190005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:48:29.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:29.179+0000 7f319a000640 1 -- 172.21.15.83:0/742481815 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3194016030 con 0x7f319c110590 2024-04-24T08:48:29.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:28 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/785953767' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-24T08:48:29.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:28 smithi189 ceph-mon[31791]: osdmap e82: 12 total, 12 up, 12 in 2024-04-24T08:48:29.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:28 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:48:29.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:28 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:48:29.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:28 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/4116635994' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-04-24T08:48:29.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:28 smithi189 ceph-mon[31791]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-04-24T08:48:29.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:28 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/785953767' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-24T08:48:29.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:28 smithi083 ceph-mon[27907]: osdmap e82: 12 total, 12 up, 12 in 2024-04-24T08:48:29.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:28 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:48:29.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:28 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:48:29.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:28 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/4116635994' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-04-24T08:48:29.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:28 smithi083 ceph-mon[27907]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-04-24T08:48:29.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:29.420+0000 7f31a4000640 1 -- 172.21.15.83:0/742481815 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f319c062c70 con 0x7f319c110590 2024-04-24T08:48:30.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:29 smithi138 ceph-mon[31441]: pgmap v366: 33 pgs: 5 creating+peering, 1 active+clean, 27 unknown; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:48:30.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:29 smithi138 ceph-mon[31441]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-04-24T08:48:30.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:29 smithi138 ceph-mon[31441]: osdmap e83: 12 total, 12 up, 12 in 2024-04-24T08:48:30.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:29 smithi138 ceph-mon[31441]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-24T08:48:30.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:29 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/742481815' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-24T08:48:30.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:29 smithi138 ceph-mon[31441]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-24T08:48:30.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:29 smithi189 ceph-mon[31791]: pgmap v366: 33 pgs: 5 creating+peering, 1 active+clean, 27 unknown; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:48:30.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:29 smithi189 ceph-mon[31791]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-04-24T08:48:30.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:29 smithi189 ceph-mon[31791]: osdmap e83: 12 total, 12 up, 12 in 2024-04-24T08:48:30.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:29 smithi189 ceph-mon[31791]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-24T08:48:30.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:29 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/742481815' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-24T08:48:30.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:29 smithi189 ceph-mon[31791]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-24T08:48:30.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:30.209+0000 7f319a000640 1 -- 172.21.15.83:0/742481815 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f319409cf60 con 0x7f319c110590 2024-04-24T08:48:30.209 INFO:teuthology.orchestra.run.smithi083.stderr:new fs with metadata pool 2 and data pool 3 2024-04-24T08:48:30.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:30.211+0000 7f31a4000640 1 -- 172.21.15.83:0/742481815 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f31840a3c30 msgr2=0x7f31840a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:30.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:30.211+0000 7f31a4000640 1 --2- 172.21.15.83:0/742481815 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f31840a3c30 0x7f31840a60f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3190009b60 tx=0x7f3190005f90 comp rx=0 tx=0).stop 2024-04-24T08:48:30.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:30.211+0000 7f31a4000640 1 -- 172.21.15.83:0/742481815 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f319c110590 msgr2=0x7f319c1a8080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:30.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:30.211+0000 7f31a4000640 1 --2- 172.21.15.83:0/742481815 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f319c110590 0x7f319c1a8080 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f319400eaa0 tx=0x7f319400ef60 comp rx=0 tx=0).stop 2024-04-24T08:48:30.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:30.212+0000 7f31a4000640 1 -- 172.21.15.83:0/742481815 shutdown_connections 2024-04-24T08:48:30.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:30.212+0000 7f31a4000640 1 --2- 172.21.15.83:0/742481815 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f31840a3c30 0x7f31840a60f0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:30.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:30.212+0000 7f31a4000640 1 --2- 172.21.15.83:0/742481815 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f319c110590 0x7f319c1a8080 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:30.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:30.212+0000 7f31a4000640 1 --2- 172.21.15.83:0/742481815 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f319c10dc60 0x7f319c1a1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:30.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:30.212+0000 7f31a4000640 1 --2- 172.21.15.83:0/742481815 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f319c10b330 0x7f319c1a0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:30.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:30.212+0000 7f31a4000640 1 -- 172.21.15.83:0/742481815 >> 172.21.15.83:0/742481815 conn(0x7f319c06fed0 msgr2=0x7f319c10d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:48:30.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:30.212+0000 7f31a4000640 1 -- 172.21.15.83:0/742481815 shutdown_connections 2024-04-24T08:48:30.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:30.212+0000 7f31a4000640 1 -- 172.21.15.83:0/742481815 wait complete. 2024-04-24T08:48:30.234 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:29 smithi083 ceph-mon[27907]: pgmap v366: 33 pgs: 5 creating+peering, 1 active+clean, 27 unknown; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:48:30.234 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:29 smithi083 ceph-mon[27907]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-04-24T08:48:30.235 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:29 smithi083 ceph-mon[27907]: osdmap e83: 12 total, 12 up, 12 in 2024-04-24T08:48:30.235 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:29 smithi083 ceph-mon[27907]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-24T08:48:30.235 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:29 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/742481815' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-24T08:48:30.235 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:29 smithi083 ceph-mon[27907]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-24T08:48:30.237 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-04-24T08:48:30.237 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-04-24T08:48:30.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:30.416+0000 7f8cf7800640 1 -- 172.21.15.83:0/951767363 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8cf0103c00 msgr2=0x7f8cf010a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:30.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:30.416+0000 7f8cf7800640 1 --2- 172.21.15.83:0/951767363 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8cf0103c00 0x7f8cf010a490 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8ce8009ca0 tx=0x7f8ce802a8c0 comp rx=0 tx=0).stop 2024-04-24T08:48:30.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:30.416+0000 7f8cf7800640 1 -- 172.21.15.83:0/951767363 shutdown_connections 2024-04-24T08:48:30.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:30.416+0000 7f8cf7800640 1 --2- 172.21.15.83:0/951767363 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8cf0103c00 0x7f8cf010a490 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:30.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:30.416+0000 7f8cf7800640 1 --2- 172.21.15.83:0/951767363 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8cf0103240 0x7f8cf01036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:30.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:30.416+0000 7f8cf7800640 1 --2- 172.21.15.83:0/951767363 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8cf0102050 0x7f8cf0102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:30.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:30.416+0000 7f8cf7800640 1 -- 172.21.15.83:0/951767363 >> 172.21.15.83:0/951767363 conn(0x7f8cf00fd5e0 msgr2=0x7f8cf00ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:48:30.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:30.416+0000 7f8cf7800640 1 -- 172.21.15.83:0/951767363 shutdown_connections 2024-04-24T08:48:30.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:30.416+0000 7f8cf7800640 1 -- 172.21.15.83:0/951767363 wait complete. 2024-04-24T08:48:30.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:30.417+0000 7f8cf7800640 1 Processor -- start 2024-04-24T08:48:30.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:30.417+0000 7f8cf7800640 1 -- start start 2024-04-24T08:48:30.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:30.417+0000 7f8cf7800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8cf0102050 0x7f8cf019a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:30.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:30.417+0000 7f8cf7800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8cf0103240 0x7f8cf019a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:30.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:30.417+0000 7f8cf7800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8cf0103c00 0x7f8cf01a19d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:30.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:30.417+0000 7f8cf7800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf010dbe0 con 0x7f8cf0103240 2024-04-24T08:48:30.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:30.417+0000 7f8cf7800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf010da60 con 0x7f8cf0103c00 2024-04-24T08:48:30.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:30.417+0000 7f8cf7800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf010dd60 con 0x7f8cf0102050 2024-04-24T08:48:30.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:30.417+0000 7f8cefe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8cf0103240 0x7f8cf019a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:30.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:30.417+0000 7f8cefe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8cf0103240 0x7f8cf019a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51490/0 (socket says 172.21.15.83:51490) 2024-04-24T08:48:30.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:30.417+0000 7f8cefe00640 1 -- 172.21.15.83:0/421562928 learned_addr learned my addr 172.21.15.83:0/421562928 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:48:30.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:30.418+0000 7f8cf4e00640 1 --2- 172.21.15.83:0/421562928 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8cf0102050 0x7f8cf019a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:30.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:30.418+0000 7f8cf5800640 1 --2- 172.21.15.83:0/421562928 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8cf0103c00 0x7f8cf01a19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:30.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:30.418+0000 7f8cefe00640 1 -- 172.21.15.83:0/421562928 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8cf0102050 msgr2=0x7f8cf019a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:30.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:30.418+0000 7f8cefe00640 1 --2- 172.21.15.83:0/421562928 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8cf0102050 0x7f8cf019a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:30.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:30.418+0000 7f8cefe00640 1 -- 172.21.15.83:0/421562928 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8cf0103c00 msgr2=0x7f8cf01a19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:30.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:30.418+0000 7f8cefe00640 1 --2- 172.21.15.83:0/421562928 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8cf0103c00 0x7f8cf01a19d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:30.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:30.418+0000 7f8cefe00640 1 -- 172.21.15.83:0/421562928 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cf01a2040 con 0x7f8cf0103240 2024-04-24T08:48:30.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:30.418+0000 7f8cefe00640 1 --2- 172.21.15.83:0/421562928 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8cf0103240 0x7f8cf019a960 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8cdc00c920 tx=0x7f8cdc00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:48:30.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:30.418+0000 7f8ced600640 1 -- 172.21.15.83:0/421562928 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cdc007c70 con 0x7f8cf0103240 2024-04-24T08:48:30.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:30.418+0000 7f8cf7800640 1 -- 172.21.15.83:0/421562928 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cf01a2330 con 0x7f8cf0103240 2024-04-24T08:48:30.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:30.418+0000 7f8ced600640 1 -- 172.21.15.83:0/421562928 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8cdc007e10 con 0x7f8cf0103240 2024-04-24T08:48:30.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:30.419+0000 7f8ced600640 1 -- 172.21.15.83:0/421562928 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cdc018820 con 0x7f8cf0103240 2024-04-24T08:48:30.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:30.419+0000 7f8cf7800640 1 -- 172.21.15.83:0/421562928 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cf01a2880 con 0x7f8cf0103240 2024-04-24T08:48:30.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:30.420+0000 7f8cf7800640 1 -- 172.21.15.83:0/421562928 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8cf01a2b40 con 0x7f8cf0103240 2024-04-24T08:48:30.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:30.422+0000 7f8ced600640 1 -- 172.21.15.83:0/421562928 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8cdc0189c0 con 0x7f8cf0103240 2024-04-24T08:48:30.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:30.424+0000 7f8ced600640 1 --2- 172.21.15.83:0/421562928 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f8cd80a3bb0 0x7f8cd80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:30.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:30.424+0000 7f8ced600640 1 -- 172.21.15.83:0/421562928 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7f8cdc0d0500 con 0x7f8cf0103240 2024-04-24T08:48:30.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:30.424+0000 7f8cf4e00640 1 --2- 172.21.15.83:0/421562928 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f8cd80a3bb0 0x7f8cd80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:30.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:30.425+0000 7f8cf4e00640 1 --2- 172.21.15.83:0/421562928 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f8cd80a3bb0 0x7f8cd80a6070 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8ce40058c0 tx=0x7f8ce400a4c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:48:30.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:30.429+0000 7f8ced600640 1 -- 172.21.15.83:0/421562928 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8cdc014030 con 0x7f8cf0103240 2024-04-24T08:48:30.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:30.662+0000 7f8cf7800640 1 -- 172.21.15.83:0/421562928 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7f8cf01a3370 con 0x7f8cf0103240 2024-04-24T08:48:30.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:30.955+0000 7f8ced600640 1 -- 172.21.15.83:0/421562928 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v85) ==== 157+0+0 (secure 0 0 0) 0x7f8cdc09cf90 con 0x7f8cf0103240 2024-04-24T08:48:30.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:30.958+0000 7f8cf7800640 1 -- 172.21.15.83:0/421562928 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f8cd80a3bb0 msgr2=0x7f8cd80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:30.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:30.958+0000 7f8cf7800640 1 --2- 172.21.15.83:0/421562928 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f8cd80a3bb0 0x7f8cd80a6070 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8ce40058c0 tx=0x7f8ce400a4c0 comp rx=0 tx=0).stop 2024-04-24T08:48:30.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:30.958+0000 7f8cf7800640 1 -- 172.21.15.83:0/421562928 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8cf0103240 msgr2=0x7f8cf019a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:30.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:30.958+0000 7f8cf7800640 1 --2- 172.21.15.83:0/421562928 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8cf0103240 0x7f8cf019a960 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8cdc00c920 tx=0x7f8cdc00cde0 comp rx=0 tx=0).stop 2024-04-24T08:48:30.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:30.959+0000 7f8cf7800640 1 -- 172.21.15.83:0/421562928 shutdown_connections 2024-04-24T08:48:30.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:30.959+0000 7f8cf7800640 1 --2- 172.21.15.83:0/421562928 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8cf0103c00 0x7f8cf01a19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:30.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:30.959+0000 7f8cf7800640 1 --2- 172.21.15.83:0/421562928 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f8cd80a3bb0 0x7f8cd80a6070 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:30.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:30.959+0000 7f8cf7800640 1 --2- 172.21.15.83:0/421562928 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8cf0103240 0x7f8cf019a960 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:30.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:30.959+0000 7f8cf7800640 1 --2- 172.21.15.83:0/421562928 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8cf0102050 0x7f8cf019a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:30.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:30.959+0000 7f8cf7800640 1 -- 172.21.15.83:0/421562928 >> 172.21.15.83:0/421562928 conn(0x7f8cf00fd5e0 msgr2=0x7f8cf0074230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:48:30.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:30.959+0000 7f8cf7800640 1 -- 172.21.15.83:0/421562928 shutdown_connections 2024-04-24T08:48:30.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:30.959+0000 7f8cf7800640 1 -- 172.21.15.83:0/421562928 wait complete. 2024-04-24T08:48:30.981 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-04-24T08:48:31.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:31.176+0000 7f939ae00640 1 -- 172.21.15.83:0/3157644789 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f93941039b0 msgr2=0x7f939410a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:31.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:31.176+0000 7f939ae00640 1 --2- 172.21.15.83:0/3157644789 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f93941039b0 0x7f939410a2d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f938c009be0 tx=0x7f938c02a6c0 comp rx=0 tx=0).stop 2024-04-24T08:48:31.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:31.176+0000 7f939ae00640 1 -- 172.21.15.83:0/3157644789 shutdown_connections 2024-04-24T08:48:31.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:31.176+0000 7f939ae00640 1 --2- 172.21.15.83:0/3157644789 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f93941039b0 0x7f939410a2d0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:31.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:31.176+0000 7f939ae00640 1 --2- 172.21.15.83:0/3157644789 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9394102ff0 0x7f9394103470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:31.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:31.176+0000 7f939ae00640 1 --2- 172.21.15.83:0/3157644789 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9394101ea0 0x7f93941022c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:31.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:31.176+0000 7f939ae00640 1 -- 172.21.15.83:0/3157644789 >> 172.21.15.83:0/3157644789 conn(0x7f93940fd410 msgr2=0x7f93940ff850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:48:31.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:31.176+0000 7f939ae00640 1 -- 172.21.15.83:0/3157644789 shutdown_connections 2024-04-24T08:48:31.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:31.176+0000 7f939ae00640 1 -- 172.21.15.83:0/3157644789 wait complete. 2024-04-24T08:48:31.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:31.177+0000 7f939ae00640 1 Processor -- start 2024-04-24T08:48:31.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:31.177+0000 7f939ae00640 1 -- start start 2024-04-24T08:48:31.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:31.178+0000 7f939ae00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9394101ea0 0x7f9394075db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:31.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:31.178+0000 7f939ae00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9394102ff0 0x7f93940743f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:31.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:31.178+0000 7f939ae00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f93941039b0 0x7f9394074930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:31.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:31.178+0000 7f939ae00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f939410da00 con 0x7f93941039b0 2024-04-24T08:48:31.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:31.178+0000 7f939ae00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f939410d880 con 0x7f9394102ff0 2024-04-24T08:48:31.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:31.178+0000 7f939ae00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f939410db80 con 0x7f9394101ea0 2024-04-24T08:48:31.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:31.178+0000 7f939a400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f93941039b0 0x7f9394074930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:31.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:31.178+0000 7f939a400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f93941039b0 0x7f9394074930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51508/0 (socket says 172.21.15.83:51508) 2024-04-24T08:48:31.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:31.178+0000 7f939a400640 1 -- 172.21.15.83:0/1891021407 learned_addr learned my addr 172.21.15.83:0/1891021407 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:48:31.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:31.178+0000 7f9399000640 1 --2- 172.21.15.83:0/1891021407 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9394102ff0 0x7f93940743f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:31.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:31.178+0000 7f9399a00640 1 --2- 172.21.15.83:0/1891021407 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9394101ea0 0x7f9394075db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:31.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:31.178+0000 7f939a400640 1 -- 172.21.15.83:0/1891021407 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9394101ea0 msgr2=0x7f9394075db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:31.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:31.178+0000 7f939a400640 1 --2- 172.21.15.83:0/1891021407 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9394101ea0 0x7f9394075db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:31.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:31.178+0000 7f939a400640 1 -- 172.21.15.83:0/1891021407 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9394102ff0 msgr2=0x7f93940743f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:31.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:31.178+0000 7f939a400640 1 --2- 172.21.15.83:0/1891021407 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9394102ff0 0x7f93940743f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:31.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:31.178+0000 7f939a400640 1 -- 172.21.15.83:0/1891021407 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93940751c0 con 0x7f93941039b0 2024-04-24T08:48:31.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:31.179+0000 7f939a400640 1 --2- 172.21.15.83:0/1891021407 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f93941039b0 0x7f9394074930 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f938c034040 tx=0x7f938c030f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:48:31.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:31.179+0000 7f9392000640 1 -- 172.21.15.83:0/1891021407 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f938c041020 con 0x7f93941039b0 2024-04-24T08:48:31.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:31.179+0000 7f939ae00640 1 -- 172.21.15.83:0/1891021407 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93941a2100 con 0x7f93941039b0 2024-04-24T08:48:31.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:31.179+0000 7f9392000640 1 -- 172.21.15.83:0/1891021407 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f938c037040 con 0x7f93941039b0 2024-04-24T08:48:31.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:31.179+0000 7f9392000640 1 -- 172.21.15.83:0/1891021407 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f938c036070 con 0x7f93941039b0 2024-04-24T08:48:31.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:31.179+0000 7f939ae00640 1 -- 172.21.15.83:0/1891021407 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93941a26a0 con 0x7f93941039b0 2024-04-24T08:48:31.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:31.180+0000 7f939ae00640 1 -- 172.21.15.83:0/1891021407 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93941091b0 con 0x7f93941039b0 2024-04-24T08:48:31.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:31.182+0000 7f9392000640 1 -- 172.21.15.83:0/1891021407 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f938c044020 con 0x7f93941039b0 2024-04-24T08:48:31.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:31.184+0000 7f9392000640 1 --2- 172.21.15.83:0/1891021407 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f937c0a3bb0 0x7f937c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:31.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:31.184+0000 7f9392000640 1 -- 172.21.15.83:0/1891021407 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8277+0+0 (secure 0 0 0) 0x7f938c0f0d30 con 0x7f93941039b0 2024-04-24T08:48:31.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:31.184+0000 7f9399a00640 1 --2- 172.21.15.83:0/1891021407 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f937c0a3bb0 0x7f937c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:31.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:31.185+0000 7f9399a00640 1 --2- 172.21.15.83:0/1891021407 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f937c0a3bb0 0x7f937c0a6070 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9388009aa0 tx=0x7f9388005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:48:31.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:31.188+0000 7f9392000640 1 -- 172.21.15.83:0/1891021407 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f938c04b020 con 0x7f93941039b0 2024-04-24T08:48:31.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:30 smithi138 ceph-mon[31441]: osdmap e84: 12 total, 12 up, 12 in 2024-04-24T08:48:31.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:30 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:48:31.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:30 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:48:31.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:30 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/742481815' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-24T08:48:31.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:30 smithi138 ceph-mon[31441]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-24T08:48:31.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:30 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:48:31.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:30 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:48:31.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:30 smithi138 ceph-mon[31441]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-24T08:48:31.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:30 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/421562928' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-04-24T08:48:31.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:30 smithi189 ceph-mon[31791]: osdmap e84: 12 total, 12 up, 12 in 2024-04-24T08:48:31.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:30 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:48:31.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:30 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:48:31.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:30 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/742481815' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-24T08:48:31.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:30 smithi189 ceph-mon[31791]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-24T08:48:31.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:30 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:48:31.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:30 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:48:31.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:30 smithi189 ceph-mon[31791]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-24T08:48:31.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:30 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/421562928' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-04-24T08:48:31.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:30 smithi083 ceph-mon[27907]: osdmap e84: 12 total, 12 up, 12 in 2024-04-24T08:48:31.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:30 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:48:31.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:30 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:48:31.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:30 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/742481815' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-24T08:48:31.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:30 smithi083 ceph-mon[27907]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-24T08:48:31.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:30 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:48:31.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:30 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:48:31.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:30 smithi083 ceph-mon[27907]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-24T08:48:31.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:30 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/421562928' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-04-24T08:48:31.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:31.399+0000 7f939ae00640 1 -- 172.21.15.83:0/1891021407 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f9394062c00 con 0x7f93941039b0 2024-04-24T08:48:32.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:31 smithi083 ceph-mon[27907]: pgmap v369: 97 pgs: 8 creating+activating, 21 creating+peering, 23 active+clean, 45 unknown; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:48:32.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:31 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/421562928' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-04-24T08:48:32.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:31 smithi083 ceph-mon[27907]: osdmap e85: 12 total, 12 up, 12 in 2024-04-24T08:48:32.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:31 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:48:32.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:31 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:48:32.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:31 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:48:32.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:31 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:48:32.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:31 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:48:32.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:31 smithi083 ceph-mon[27907]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-24T08:48:32.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:31 smithi083 ceph-mon[27907]: Cluster is now healthy 2024-04-24T08:48:32.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:31 smithi083 ceph-mon[27907]: mds.? [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] up:active 2024-04-24T08:48:32.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:31 smithi083 ceph-mon[27907]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-24T08:48:32.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:31 smithi083 ceph-mon[27907]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-24T08:48:32.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:31 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/1891021407' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-24T08:48:32.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:31 smithi138 ceph-mon[31441]: pgmap v369: 97 pgs: 8 creating+activating, 21 creating+peering, 23 active+clean, 45 unknown; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:48:32.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:31 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/421562928' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-04-24T08:48:32.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:31 smithi138 ceph-mon[31441]: osdmap e85: 12 total, 12 up, 12 in 2024-04-24T08:48:32.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:31 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:48:32.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:31 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:48:32.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:31 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:48:32.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:31 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:48:32.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:31 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:48:32.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:31 smithi138 ceph-mon[31441]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-24T08:48:32.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:31 smithi138 ceph-mon[31441]: Cluster is now healthy 2024-04-24T08:48:32.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:31 smithi138 ceph-mon[31441]: mds.? [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] up:active 2024-04-24T08:48:32.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:31 smithi138 ceph-mon[31441]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-24T08:48:32.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:31 smithi138 ceph-mon[31441]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-24T08:48:32.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:31 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/1891021407' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-24T08:48:32.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:31 smithi189 ceph-mon[31791]: pgmap v369: 97 pgs: 8 creating+activating, 21 creating+peering, 23 active+clean, 45 unknown; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:48:32.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:31 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/421562928' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-04-24T08:48:32.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:31 smithi189 ceph-mon[31791]: osdmap e85: 12 total, 12 up, 12 in 2024-04-24T08:48:32.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:31 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:48:32.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:31 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:48:32.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:31 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:48:32.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:31 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:48:32.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:31 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:48:32.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:31 smithi189 ceph-mon[31791]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-24T08:48:32.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:31 smithi189 ceph-mon[31791]: Cluster is now healthy 2024-04-24T08:48:32.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:31 smithi189 ceph-mon[31791]: mds.? [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] up:active 2024-04-24T08:48:32.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:31 smithi189 ceph-mon[31791]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-24T08:48:32.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:31 smithi189 ceph-mon[31791]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-24T08:48:32.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:31 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/1891021407' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-24T08:48:33.205 INFO:journalctl@ceph.mds.b.smithi138.stdout:Apr 24 08:48:32 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-b[61515]: 2024-04-24T08:48:32.938+0000 7f7fb8a00700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-24T08:48:33.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:33.263+0000 7f9392000640 1 -- 172.21.15.83:0/1891021407 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v87) ==== 228+0+0 (secure 0 0 0) 0x7f938c0bd680 con 0x7f93941039b0 2024-04-24T08:48:33.264 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_data_ec' created 2024-04-24T08:48:33.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:33.267+0000 7f939ae00640 1 -- 172.21.15.83:0/1891021407 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f937c0a3bb0 msgr2=0x7f937c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:33.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:33.267+0000 7f939ae00640 1 --2- 172.21.15.83:0/1891021407 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f937c0a3bb0 0x7f937c0a6070 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9388009aa0 tx=0x7f9388005f90 comp rx=0 tx=0).stop 2024-04-24T08:48:33.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:33.267+0000 7f939ae00640 1 -- 172.21.15.83:0/1891021407 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f93941039b0 msgr2=0x7f9394074930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:33.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:33.267+0000 7f939ae00640 1 --2- 172.21.15.83:0/1891021407 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f93941039b0 0x7f9394074930 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f938c034040 tx=0x7f938c030f90 comp rx=0 tx=0).stop 2024-04-24T08:48:33.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:33.267+0000 7f939ae00640 1 -- 172.21.15.83:0/1891021407 shutdown_connections 2024-04-24T08:48:33.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:33.267+0000 7f939ae00640 1 --2- 172.21.15.83:0/1891021407 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f937c0a3bb0 0x7f937c0a6070 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:33.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:33.267+0000 7f939ae00640 1 --2- 172.21.15.83:0/1891021407 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f93941039b0 0x7f9394074930 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:33.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:33.267+0000 7f939ae00640 1 --2- 172.21.15.83:0/1891021407 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9394102ff0 0x7f93940743f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:33.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:33.267+0000 7f939ae00640 1 --2- 172.21.15.83:0/1891021407 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9394101ea0 0x7f9394075db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:33.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:33.267+0000 7f939ae00640 1 -- 172.21.15.83:0/1891021407 >> 172.21.15.83:0/1891021407 conn(0x7f93940fd410 msgr2=0x7f9394106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:48:33.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:33.268+0000 7f939ae00640 1 -- 172.21.15.83:0/1891021407 shutdown_connections 2024-04-24T08:48:33.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:33.268+0000 7f939ae00640 1 -- 172.21.15.83:0/1891021407 wait complete. 2024-04-24T08:48:33.300 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-04-24T08:48:33.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:33.486+0000 7f5edb600640 1 -- 172.21.15.83:0/724787479 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ed4105e30 msgr2=0x7f5ed410c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:33.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:33.486+0000 7f5edb600640 1 --2- 172.21.15.83:0/724787479 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ed4105e30 0x7f5ed410c6c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f5ecc009ca0 tx=0x7f5ecc02a8c0 comp rx=0 tx=0).stop 2024-04-24T08:48:33.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:33.486+0000 7f5edb600640 1 -- 172.21.15.83:0/724787479 shutdown_connections 2024-04-24T08:48:33.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:33.486+0000 7f5edb600640 1 --2- 172.21.15.83:0/724787479 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ed4105e30 0x7f5ed410c6c0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:33.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:33.486+0000 7f5edb600640 1 --2- 172.21.15.83:0/724787479 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ed4105470 0x7f5ed41058f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:33.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:33.486+0000 7f5edb600640 1 --2- 172.21.15.83:0/724787479 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5ed4104280 0x7f5ed41046a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:33.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:33.486+0000 7f5edb600640 1 -- 172.21.15.83:0/724787479 >> 172.21.15.83:0/724787479 conn(0x7f5ed40ff7f0 msgr2=0x7f5ed4101c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:48:33.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:33.486+0000 7f5edb600640 1 -- 172.21.15.83:0/724787479 shutdown_connections 2024-04-24T08:48:33.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:33.486+0000 7f5edb600640 1 -- 172.21.15.83:0/724787479 wait complete. 2024-04-24T08:48:33.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:33.487+0000 7f5edb600640 1 Processor -- start 2024-04-24T08:48:33.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:33.487+0000 7f5edb600640 1 -- start start 2024-04-24T08:48:33.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:33.488+0000 7f5edb600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5ed4104280 0x7f5ed419a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:33.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:33.488+0000 7f5edb600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ed4105470 0x7f5ed419a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:33.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:33.488+0000 7f5edb600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ed4105e30 0x7f5ed41a19f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:33.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:33.488+0000 7f5edb600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed410fdc0 con 0x7f5ed4105e30 2024-04-24T08:48:33.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:33.488+0000 7f5edb600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed410fc40 con 0x7f5ed4105470 2024-04-24T08:48:33.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:33.488+0000 7f5edb600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed410ff40 con 0x7f5ed4104280 2024-04-24T08:48:33.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:33.488+0000 7f5ed9600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ed4105e30 0x7f5ed41a19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:33.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:33.488+0000 7f5ed9600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ed4105e30 0x7f5ed41a19f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51524/0 (socket says 172.21.15.83:51524) 2024-04-24T08:48:33.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:33.488+0000 7f5ed9600640 1 -- 172.21.15.83:0/4234379326 learned_addr learned my addr 172.21.15.83:0/4234379326 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:48:33.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:33.488+0000 7f5ed3e00640 1 --2- 172.21.15.83:0/4234379326 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ed4105470 0x7f5ed419a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:33.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:33.488+0000 7f5ed8c00640 1 --2- 172.21.15.83:0/4234379326 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5ed4104280 0x7f5ed419a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:33.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:33.488+0000 7f5ed9600640 1 -- 172.21.15.83:0/4234379326 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5ed4104280 msgr2=0x7f5ed419a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:33.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:33.488+0000 7f5ed9600640 1 --2- 172.21.15.83:0/4234379326 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5ed4104280 0x7f5ed419a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:33.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:33.488+0000 7f5ed9600640 1 -- 172.21.15.83:0/4234379326 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ed4105470 msgr2=0x7f5ed419a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:33.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:33.488+0000 7f5ed9600640 1 --2- 172.21.15.83:0/4234379326 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ed4105470 0x7f5ed419a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:33.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:33.488+0000 7f5ed9600640 1 -- 172.21.15.83:0/4234379326 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ed41a2060 con 0x7f5ed4105e30 2024-04-24T08:48:33.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:33.489+0000 7f5ed9600640 1 --2- 172.21.15.83:0/4234379326 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ed4105e30 0x7f5ed41a19f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5ecc009dc0 tx=0x7f5ecc003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:48:33.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:33.489+0000 7f5ed1600640 1 -- 172.21.15.83:0/4234379326 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ecc032ec0 con 0x7f5ed4105e30 2024-04-24T08:48:33.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:33.489+0000 7f5ed1600640 1 -- 172.21.15.83:0/4234379326 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5ecc0044f0 con 0x7f5ed4105e30 2024-04-24T08:48:33.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:33.489+0000 7f5ed1600640 1 -- 172.21.15.83:0/4234379326 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ecc030d40 con 0x7f5ed4105e30 2024-04-24T08:48:33.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:33.489+0000 7f5edb600640 1 -- 172.21.15.83:0/4234379326 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ed41a22f0 con 0x7f5ed4105e30 2024-04-24T08:48:33.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:33.489+0000 7f5edb600640 1 -- 172.21.15.83:0/4234379326 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ed41a27f0 con 0x7f5ed4105e30 2024-04-24T08:48:33.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:33.490+0000 7f5edb600640 1 -- 172.21.15.83:0/4234379326 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ed410b5a0 con 0x7f5ed4105e30 2024-04-24T08:48:33.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:33.493+0000 7f5ed1600640 1 -- 172.21.15.83:0/4234379326 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f5ecc004040 con 0x7f5ed4105e30 2024-04-24T08:48:33.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:33.494+0000 7f5ed1600640 1 --2- 172.21.15.83:0/4234379326 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f5eb80a3dd0 0x7f5eb80a6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:33.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:33.494+0000 7f5ed1600640 1 -- 172.21.15.83:0/4234379326 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8770+0+0 (secure 0 0 0) 0x7f5ecc0f1e60 con 0x7f5ed4105e30 2024-04-24T08:48:33.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:33.494+0000 7f5ed8c00640 1 --2- 172.21.15.83:0/4234379326 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f5eb80a3dd0 0x7f5eb80a6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:33.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:33.495+0000 7f5ed8c00640 1 --2- 172.21.15.83:0/4234379326 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f5eb80a3dd0 0x7f5eb80a6290 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f5ec4005fd0 tx=0x7f5ec4005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:48:33.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:33.497+0000 7f5ed1600640 1 -- 172.21.15.83:0/4234379326 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5ecc0be5c0 con 0x7f5ed4105e30 2024-04-24T08:48:33.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:33 smithi083 ceph-mon[27907]: osdmap e86: 12 total, 12 up, 12 in 2024-04-24T08:48:33.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:33 smithi083 ceph-mon[27907]: pgmap v372: 97 pgs: 8 creating+activating, 21 creating+peering, 40 active+clean, 28 unknown; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 1 op/s 2024-04-24T08:48:33.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:33.702+0000 7f5edb600640 1 -- 172.21.15.83:0/4234379326 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7f5ed402d620 con 0x7f5ed4105e30 2024-04-24T08:48:33.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:33 smithi138 ceph-mon[31441]: osdmap e86: 12 total, 12 up, 12 in 2024-04-24T08:48:33.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:33 smithi138 ceph-mon[31441]: pgmap v372: 97 pgs: 8 creating+activating, 21 creating+peering, 40 active+clean, 28 unknown; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 1 op/s 2024-04-24T08:48:33.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:33 smithi189 ceph-mon[31791]: osdmap e86: 12 total, 12 up, 12 in 2024-04-24T08:48:33.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:33 smithi189 ceph-mon[31791]: pgmap v372: 97 pgs: 8 creating+activating, 21 creating+peering, 40 active+clean, 28 unknown; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 1 op/s 2024-04-24T08:48:34.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:34.278+0000 7f5ed1600640 1 -- 172.21.15.83:0/4234379326 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v88) ==== 169+0+0 (secure 0 0 0) 0x7f5ecc0bdec0 con 0x7f5ed4105e30 2024-04-24T08:48:34.279 INFO:teuthology.orchestra.run.smithi083.stderr:set pool 4 allow_ec_overwrites to true 2024-04-24T08:48:34.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:34.283+0000 7f5edb600640 1 -- 172.21.15.83:0/4234379326 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f5eb80a3dd0 msgr2=0x7f5eb80a6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:34.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:34.283+0000 7f5edb600640 1 --2- 172.21.15.83:0/4234379326 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f5eb80a3dd0 0x7f5eb80a6290 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f5ec4005fd0 tx=0x7f5ec4005850 comp rx=0 tx=0).stop 2024-04-24T08:48:34.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:34.283+0000 7f5edb600640 1 -- 172.21.15.83:0/4234379326 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ed4105e30 msgr2=0x7f5ed41a19f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:34.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:34.283+0000 7f5edb600640 1 --2- 172.21.15.83:0/4234379326 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ed4105e30 0x7f5ed41a19f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5ecc009dc0 tx=0x7f5ecc003ba0 comp rx=0 tx=0).stop 2024-04-24T08:48:34.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:34.285+0000 7f5edb600640 1 -- 172.21.15.83:0/4234379326 shutdown_connections 2024-04-24T08:48:34.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:34.285+0000 7f5edb600640 1 --2- 172.21.15.83:0/4234379326 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f5eb80a3dd0 0x7f5eb80a6290 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:34.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:34.285+0000 7f5edb600640 1 --2- 172.21.15.83:0/4234379326 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ed4105e30 0x7f5ed41a19f0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:34.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:34.285+0000 7f5edb600640 1 --2- 172.21.15.83:0/4234379326 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ed4105470 0x7f5ed419a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:34.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:34.285+0000 7f5edb600640 1 --2- 172.21.15.83:0/4234379326 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5ed4104280 0x7f5ed419a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:34.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:34.285+0000 7f5edb600640 1 -- 172.21.15.83:0/4234379326 >> 172.21.15.83:0/4234379326 conn(0x7f5ed40ff7f0 msgr2=0x7f5ed4108770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:48:34.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:34.285+0000 7f5edb600640 1 -- 172.21.15.83:0/4234379326 shutdown_connections 2024-04-24T08:48:34.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:34.286+0000 7f5edb600640 1 -- 172.21.15.83:0/4234379326 wait complete. 2024-04-24T08:48:34.321 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-04-24T08:48:34.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:34.496+0000 7f1ddca00640 1 -- 172.21.15.83:0/3694205339 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dc8031880 con 0x7f1dd8077c00 2024-04-24T08:48:34.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:34.496+0000 7f1de0800640 1 -- 172.21.15.83:0/3694205339 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1dd8077c00 msgr2=0x7f1dd8078060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:34.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:34.496+0000 7f1de0800640 1 --2- 172.21.15.83:0/3694205339 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1dd8077c00 0x7f1dd8078060 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f1dc8009900 tx=0x7f1dc802a410 comp rx=0 tx=0).stop 2024-04-24T08:48:34.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:34.497+0000 7f1de0800640 1 -- 172.21.15.83:0/3694205339 shutdown_connections 2024-04-24T08:48:34.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:34.497+0000 7f1de0800640 1 --2- 172.21.15.83:0/3694205339 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1dd80785a0 0x7f1dd8078e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:34.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:34.497+0000 7f1de0800640 1 --2- 172.21.15.83:0/3694205339 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1dd8077c00 0x7f1dd8078060 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:34.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:34.497+0000 7f1de0800640 1 --2- 172.21.15.83:0/3694205339 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1dd80769c0 0x7f1dd8076de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:34.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:34.497+0000 7f1de0800640 1 -- 172.21.15.83:0/3694205339 >> 172.21.15.83:0/3694205339 conn(0x7f1dd80ff7f0 msgr2=0x7f1dd8101c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:48:34.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:34.497+0000 7f1de0800640 1 -- 172.21.15.83:0/3694205339 shutdown_connections 2024-04-24T08:48:34.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:34.497+0000 7f1de0800640 1 -- 172.21.15.83:0/3694205339 wait complete. 2024-04-24T08:48:34.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:34.498+0000 7f1de0800640 1 Processor -- start 2024-04-24T08:48:34.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:34.498+0000 7f1de0800640 1 -- start start 2024-04-24T08:48:34.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:34.498+0000 7f1de0800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1dd80769c0 0x7f1dd819ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:34.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:34.498+0000 7f1de0800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1dd8077c00 0x7f1dd819ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:34.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:34.498+0000 7f1de0800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1dd80785a0 0x7f1dd81a6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:34.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:34.498+0000 7f1de0800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd8114270 con 0x7f1dd80785a0 2024-04-24T08:48:34.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:34.498+0000 7f1de0800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd81140f0 con 0x7f1dd8077c00 2024-04-24T08:48:34.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:34.498+0000 7f1de0800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd81143f0 con 0x7f1dd80769c0 2024-04-24T08:48:34.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:34.498+0000 7f1dde800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1dd80785a0 0x7f1dd81a6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:34.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:34.498+0000 7f1dde800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1dd80785a0 0x7f1dd81a6000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51542/0 (socket says 172.21.15.83:51542) 2024-04-24T08:48:34.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:34.498+0000 7f1dde800640 1 -- 172.21.15.83:0/301656375 learned_addr learned my addr 172.21.15.83:0/301656375 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:48:34.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:34.499+0000 7f1ddde00640 1 --2- 172.21.15.83:0/301656375 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1dd80769c0 0x7f1dd819ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:34.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:34.499+0000 7f1ddd400640 1 --2- 172.21.15.83:0/301656375 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1dd8077c00 0x7f1dd819ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:34.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:34.499+0000 7f1dde800640 1 -- 172.21.15.83:0/301656375 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1dd80769c0 msgr2=0x7f1dd819ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:34.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:34.499+0000 7f1dde800640 1 --2- 172.21.15.83:0/301656375 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1dd80769c0 0x7f1dd819ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:34.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:34.499+0000 7f1dde800640 1 -- 172.21.15.83:0/301656375 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1dd8077c00 msgr2=0x7f1dd819ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:34.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:34.499+0000 7f1dde800640 1 --2- 172.21.15.83:0/301656375 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1dd8077c00 0x7f1dd819ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:34.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:34.499+0000 7f1dde800640 1 -- 172.21.15.83:0/301656375 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dd81a6670 con 0x7f1dd80785a0 2024-04-24T08:48:34.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:34.499+0000 7f1dde800640 1 --2- 172.21.15.83:0/301656375 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1dd80785a0 0x7f1dd81a6000 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f1dd0009fd0 tx=0x7f1dd000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:48:34.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:34.499+0000 7f1dcea00640 1 -- 172.21.15.83:0/301656375 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dd0011830 con 0x7f1dd80785a0 2024-04-24T08:48:34.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:34.499+0000 7f1dcea00640 1 -- 172.21.15.83:0/301656375 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1dd0016070 con 0x7f1dd80785a0 2024-04-24T08:48:34.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:34.499+0000 7f1de0800640 1 -- 172.21.15.83:0/301656375 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dd81a6960 con 0x7f1dd80785a0 2024-04-24T08:48:34.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:34.499+0000 7f1dcea00640 1 -- 172.21.15.83:0/301656375 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dd0010de0 con 0x7f1dd80785a0 2024-04-24T08:48:34.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:34.499+0000 7f1de0800640 1 -- 172.21.15.83:0/301656375 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dd81a6ee0 con 0x7f1dd80785a0 2024-04-24T08:48:34.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:34.501+0000 7f1de0800640 1 -- 172.21.15.83:0/301656375 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1dd8110bf0 con 0x7f1dd80785a0 2024-04-24T08:48:34.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:34.503+0000 7f1dcea00640 1 -- 172.21.15.83:0/301656375 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f1dd00119d0 con 0x7f1dd80785a0 2024-04-24T08:48:34.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:34.504+0000 7f1dcea00640 1 --2- 172.21.15.83:0/301656375 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f1dc40a3bb0 0x7f1dc40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:34.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:34.504+0000 7f1dcea00640 1 -- 172.21.15.83:0/301656375 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8770+0+0 (secure 0 0 0) 0x7f1dd00cfdb0 con 0x7f1dd80785a0 2024-04-24T08:48:34.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:34.505+0000 7f1ddde00640 1 --2- 172.21.15.83:0/301656375 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f1dc40a3bb0 0x7f1dc40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:34.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:34.505+0000 7f1ddde00640 1 --2- 172.21.15.83:0/301656375 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f1dc40a3bb0 0x7f1dc40a6070 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1dd4009f20 tx=0x7f1dd40094f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:48:34.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:34.510+0000 7f1dcea00640 1 -- 172.21.15.83:0/301656375 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1dd009c570 con 0x7f1dd80785a0 2024-04-24T08:48:34.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:34 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/1891021407' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-24T08:48:34.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:34 smithi083 ceph-mon[27907]: osdmap e87: 12 total, 12 up, 12 in 2024-04-24T08:48:34.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:34 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/4234379326' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-04-24T08:48:34.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:34 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/1891021407' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-24T08:48:34.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:34 smithi138 ceph-mon[31441]: osdmap e87: 12 total, 12 up, 12 in 2024-04-24T08:48:34.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:34 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/4234379326' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-04-24T08:48:34.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:34 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/1891021407' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-24T08:48:34.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:34 smithi189 ceph-mon[31791]: osdmap e87: 12 total, 12 up, 12 in 2024-04-24T08:48:34.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:34 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/4234379326' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-04-24T08:48:34.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:34.750+0000 7f1de0800640 1 -- 172.21.15.83:0/301656375 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f1dd8062c00 con 0x7f1dd80785a0 2024-04-24T08:48:35.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:35.282+0000 7f1dcea00640 1 -- 172.21.15.83:0/301656375 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) ==== 137+0+0 (secure 0 0 0) 0x7f1dd009be70 con 0x7f1dd80785a0 2024-04-24T08:48:35.284 INFO:teuthology.orchestra.run.smithi083.stderr:added data pool 4 to fsmap 2024-04-24T08:48:35.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:35.287+0000 7f1de0800640 1 -- 172.21.15.83:0/301656375 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f1dc40a3bb0 msgr2=0x7f1dc40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:35.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:35.287+0000 7f1de0800640 1 --2- 172.21.15.83:0/301656375 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f1dc40a3bb0 0x7f1dc40a6070 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1dd4009f20 tx=0x7f1dd40094f0 comp rx=0 tx=0).stop 2024-04-24T08:48:35.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:35.287+0000 7f1de0800640 1 -- 172.21.15.83:0/301656375 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1dd80785a0 msgr2=0x7f1dd81a6000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:35.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:35.287+0000 7f1de0800640 1 --2- 172.21.15.83:0/301656375 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1dd80785a0 0x7f1dd81a6000 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f1dd0009fd0 tx=0x7f1dd000b8e0 comp rx=0 tx=0).stop 2024-04-24T08:48:35.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:35.287+0000 7f1de0800640 1 -- 172.21.15.83:0/301656375 shutdown_connections 2024-04-24T08:48:35.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:35.287+0000 7f1de0800640 1 --2- 172.21.15.83:0/301656375 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f1dc40a3bb0 0x7f1dc40a6070 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:35.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:35.287+0000 7f1de0800640 1 --2- 172.21.15.83:0/301656375 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1dd80785a0 0x7f1dd81a6000 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:35.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:35.287+0000 7f1de0800640 1 --2- 172.21.15.83:0/301656375 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1dd8077c00 0x7f1dd819ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:35.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:35.287+0000 7f1de0800640 1 --2- 172.21.15.83:0/301656375 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1dd80769c0 0x7f1dd819ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:35.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:35.287+0000 7f1de0800640 1 -- 172.21.15.83:0/301656375 >> 172.21.15.83:0/301656375 conn(0x7f1dd80ff7f0 msgr2=0x7f1dd8102bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:48:35.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:35.288+0000 7f1de0800640 1 -- 172.21.15.83:0/301656375 shutdown_connections 2024-04-24T08:48:35.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:35.289+0000 7f1de0800640 1 -- 172.21.15.83:0/301656375 wait complete. 2024-04-24T08:48:35.314 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T08:48:35.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:35.510+0000 7f1968600640 1 -- 172.21.15.83:0/2926402305 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1960101a60 msgr2=0x7f1960106300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:35.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:35.510+0000 7f1968600640 1 --2- 172.21.15.83:0/2926402305 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1960101a60 0x7f1960106300 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f1958009ca0 tx=0x7f195802a8b0 comp rx=0 tx=0).stop 2024-04-24T08:48:35.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:35.511+0000 7f1968600640 1 -- 172.21.15.83:0/2926402305 shutdown_connections 2024-04-24T08:48:35.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:35.511+0000 7f1968600640 1 --2- 172.21.15.83:0/2926402305 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1960101a60 0x7f1960106300 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:35.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:35.511+0000 7f1968600640 1 --2- 172.21.15.83:0/2926402305 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19601010a0 0x7f1960101520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:35.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:35.511+0000 7f1968600640 1 --2- 172.21.15.83:0/2926402305 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1960100340 0x7f1960100760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:35.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:35.511+0000 7f1968600640 1 -- 172.21.15.83:0/2926402305 >> 172.21.15.83:0/2926402305 conn(0x7f19600fbc70 msgr2=0x7f19600fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:48:35.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:35.511+0000 7f1968600640 1 -- 172.21.15.83:0/2926402305 shutdown_connections 2024-04-24T08:48:35.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:35.511+0000 7f1968600640 1 -- 172.21.15.83:0/2926402305 wait complete. 2024-04-24T08:48:35.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:35.512+0000 7f1968600640 1 Processor -- start 2024-04-24T08:48:35.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:35.512+0000 7f1968600640 1 -- start start 2024-04-24T08:48:35.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:35.512+0000 7f1968600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1960100340 0x7f196019e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:35.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:35.512+0000 7f1968600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f19601010a0 0x7f196019ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:35.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:35.512+0000 7f1968600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1960101a60 0x7f19601a5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:35.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:35.512+0000 7f1968600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f19601141f0 con 0x7f1960101a60 2024-04-24T08:48:35.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:35.512+0000 7f1968600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1960114070 con 0x7f1960100340 2024-04-24T08:48:35.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:35.512+0000 7f1968600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1960114370 con 0x7f19601010a0 2024-04-24T08:48:35.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:35.512+0000 7f1966600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1960101a60 0x7f19601a5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:35.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:35.512+0000 7f1966600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1960101a60 0x7f19601a5ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51564/0 (socket says 172.21.15.83:51564) 2024-04-24T08:48:35.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:35.512+0000 7f1966600640 1 -- 172.21.15.83:0/3262868640 learned_addr learned my addr 172.21.15.83:0/3262868640 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:48:35.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:35.513+0000 7f1965c00640 1 --2- 172.21.15.83:0/3262868640 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1960100340 0x7f196019e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:35.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:35.513+0000 7f1965200640 1 --2- 172.21.15.83:0/3262868640 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f19601010a0 0x7f196019ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:35.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:35.513+0000 7f1966600640 1 -- 172.21.15.83:0/3262868640 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f19601010a0 msgr2=0x7f196019ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:35.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:35.513+0000 7f1966600640 1 --2- 172.21.15.83:0/3262868640 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f19601010a0 0x7f196019ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:35.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:35.513+0000 7f1966600640 1 -- 172.21.15.83:0/3262868640 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1960100340 msgr2=0x7f196019e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:35.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:35.513+0000 7f1966600640 1 --2- 172.21.15.83:0/3262868640 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1960100340 0x7f196019e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:35.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:35.513+0000 7f1966600640 1 -- 172.21.15.83:0/3262868640 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19601a6350 con 0x7f1960101a60 2024-04-24T08:48:35.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:35.513+0000 7f1966600640 1 --2- 172.21.15.83:0/3262868640 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1960101a60 0x7f19601a5ce0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1958003bd0 tx=0x7f1958003c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:48:35.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:35.513+0000 7f195e000640 1 -- 172.21.15.83:0/3262868640 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1958004450 con 0x7f1960101a60 2024-04-24T08:48:35.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:35.513+0000 7f195e000640 1 -- 172.21.15.83:0/3262868640 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1958040070 con 0x7f1960101a60 2024-04-24T08:48:35.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:35.513+0000 7f1968600640 1 -- 172.21.15.83:0/3262868640 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19601a65e0 con 0x7f1960101a60 2024-04-24T08:48:35.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:35.513+0000 7f195e000640 1 -- 172.21.15.83:0/3262868640 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f195803b430 con 0x7f1960101a60 2024-04-24T08:48:35.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:35.514+0000 7f1968600640 1 -- 172.21.15.83:0/3262868640 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19601a6b80 con 0x7f1960101a60 2024-04-24T08:48:35.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:35.515+0000 7f1968600640 1 -- 172.21.15.83:0/3262868640 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19601050f0 con 0x7f1960101a60 2024-04-24T08:48:35.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:35.517+0000 7f195e000640 1 -- 172.21.15.83:0/3262868640 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f1958031440 con 0x7f1960101a60 2024-04-24T08:48:35.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:35.518+0000 7f195e000640 1 --2- 172.21.15.83:0/3262868640 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f19480a3c30 0x7f19480a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:35.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:35.518+0000 7f195e000640 1 -- 172.21.15.83:0/3262868640 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f19580f1c70 con 0x7f1960101a60 2024-04-24T08:48:35.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:35.518+0000 7f1965c00640 1 --2- 172.21.15.83:0/3262868640 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f19480a3c30 0x7f19480a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:35.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:35.519+0000 7f1965c00640 1 --2- 172.21.15.83:0/3262868640 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f19480a3c30 0x7f19480a60f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1954005e10 tx=0x7f1954005da0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:48:35.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:35.522+0000 7f195e000640 1 -- 172.21.15.83:0/3262868640 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f19580be3b0 con 0x7f1960101a60 2024-04-24T08:48:35.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:35 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/4234379326' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-04-24T08:48:35.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:35 smithi083 ceph-mon[27907]: osdmap e88: 12 total, 12 up, 12 in 2024-04-24T08:48:35.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:35 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/301656375' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-04-24T08:48:35.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:35 smithi083 ceph-mon[27907]: pgmap v375: 129 pgs: 8 creating+activating, 6 creating+peering, 90 active+clean, 25 unknown; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 6 op/s 2024-04-24T08:48:35.607 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:35 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/4234379326' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-04-24T08:48:35.607 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:35 smithi138 ceph-mon[31441]: osdmap e88: 12 total, 12 up, 12 in 2024-04-24T08:48:35.607 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:35 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/301656375' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-04-24T08:48:35.607 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:35 smithi138 ceph-mon[31441]: pgmap v375: 129 pgs: 8 creating+activating, 6 creating+peering, 90 active+clean, 25 unknown; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 6 op/s 2024-04-24T08:48:35.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:35 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/4234379326' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-04-24T08:48:35.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:35 smithi189 ceph-mon[31791]: osdmap e88: 12 total, 12 up, 12 in 2024-04-24T08:48:35.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:35 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/301656375' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-04-24T08:48:35.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:35 smithi189 ceph-mon[31791]: pgmap v375: 129 pgs: 8 creating+activating, 6 creating+peering, 90 active+clean, 25 unknown; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 6 op/s 2024-04-24T08:48:35.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:35.762+0000 7f1968600640 1 -- 172.21.15.83:0/3262868640 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1960062c00 con 0x7f1960101a60 2024-04-24T08:48:35.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:35.764+0000 7f195e000640 1 -- 172.21.15.83:0/3262868640 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11809 (secure 0 0 0) 0x7f19580098b0 con 0x7f1960101a60 2024-04-24T08:48:35.764 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:48:35.764 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/358872257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":358872257},{"type":"v1","addr":"172.21.15.83:6835","nonce":358872257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3157070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3157070851},{"type":"v1","addr":"172.21.15.83:6837","nonce":3157070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14658,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6839/4085470844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":4085470844},{"type":"v1","addr":"172.21.15.83:6839","nonce":4085470844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14676,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6841/2634754210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":2634754210},{"type":"v1","addr":"172.21.15.83:6841","nonce":2634754210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/1859921310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1859921310},{"type":"v1","addr":"172.21.15.189:6833","nonce":1859921310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24421,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/2241923197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2241923197},{"type":"v1","addr":"172.21.15.189:6835","nonce":2241923197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24439,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/3172311220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":3172311220},{"type":"v1","addr":"172.21.15.189:6837","nonce":3172311220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24454,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/2548965336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":2548965336},{"type":"v1","addr":"172.21.15.189:6839","nonce":2548965336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34427,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/1157967658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1157967658},{"type":"v1","addr":"172.21.15.138:6833","nonce":1157967658}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34451,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1909219758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1909219758},{"type":"v1","addr":"172.21.15.138:6835","nonce":1909219758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":34493,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/900566754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":900566754},{"type":"v1","addr":"172.21.15.138:6837","nonce":900566754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T08:48:29.422576+0000","modified":"2024-04-24T08:48:34.753110+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34520":{"gid":34520,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.138:6839/2236932393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2236932393},{"type":"v1","addr":"172.21.15.138:6839","nonce":2236932393}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34520,"qdb_cluster":[34520]},"id":1}]} 2024-04-24T08:48:35.765 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 17 2024-04-24T08:48:35.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:35.766+0000 7f1968600640 1 -- 172.21.15.83:0/3262868640 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f19480a3c30 msgr2=0x7f19480a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:35.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:35.766+0000 7f1968600640 1 --2- 172.21.15.83:0/3262868640 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f19480a3c30 0x7f19480a60f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1954005e10 tx=0x7f1954005da0 comp rx=0 tx=0).stop 2024-04-24T08:48:35.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:35.766+0000 7f1968600640 1 -- 172.21.15.83:0/3262868640 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1960101a60 msgr2=0x7f19601a5ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:35.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:35.766+0000 7f1968600640 1 --2- 172.21.15.83:0/3262868640 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1960101a60 0x7f19601a5ce0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1958003bd0 tx=0x7f1958003c00 comp rx=0 tx=0).stop 2024-04-24T08:48:35.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:35.767+0000 7f1968600640 1 -- 172.21.15.83:0/3262868640 shutdown_connections 2024-04-24T08:48:35.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:35.767+0000 7f1968600640 1 --2- 172.21.15.83:0/3262868640 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f19480a3c30 0x7f19480a60f0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:35.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:35.767+0000 7f1968600640 1 --2- 172.21.15.83:0/3262868640 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1960101a60 0x7f19601a5ce0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:35.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:35.767+0000 7f1968600640 1 --2- 172.21.15.83:0/3262868640 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f19601010a0 0x7f196019ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:35.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:35.767+0000 7f1968600640 1 --2- 172.21.15.83:0/3262868640 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1960100340 0x7f196019e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:35.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:35.767+0000 7f1968600640 1 -- 172.21.15.83:0/3262868640 >> 172.21.15.83:0/3262868640 conn(0x7f19600fbc70 msgr2=0x7f19601022c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:48:35.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:35.767+0000 7f1968600640 1 -- 172.21.15.83:0/3262868640 shutdown_connections 2024-04-24T08:48:35.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:35.767+0000 7f1968600640 1 -- 172.21.15.83:0/3262868640 wait complete. 2024-04-24T08:48:35.787 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T08:48:35.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:35.989+0000 7f2ef9200640 1 -- 172.21.15.83:0/2747700777 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ef410a030 msgr2=0x7f2ef41108c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:35.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:35.989+0000 7f2ef9200640 1 --2- 172.21.15.83:0/2747700777 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ef410a030 0x7f2ef41108c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2eec009ca0 tx=0x7f2eec02a8c0 comp rx=0 tx=0).stop 2024-04-24T08:48:35.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:35.989+0000 7f2ef9200640 1 -- 172.21.15.83:0/2747700777 shutdown_connections 2024-04-24T08:48:35.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:35.989+0000 7f2ef9200640 1 --2- 172.21.15.83:0/2747700777 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ef410a030 0x7f2ef41108c0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:35.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:35.989+0000 7f2ef9200640 1 --2- 172.21.15.83:0/2747700777 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2ef4109670 0x7f2ef4109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:35.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:35.989+0000 7f2ef9200640 1 --2- 172.21.15.83:0/2747700777 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ef4108480 0x7f2ef41088a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:35.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:35.989+0000 7f2ef9200640 1 -- 172.21.15.83:0/2747700777 >> 172.21.15.83:0/2747700777 conn(0x7f2ef4077440 msgr2=0x7f2ef4079860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:48:35.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:35.989+0000 7f2ef9200640 1 -- 172.21.15.83:0/2747700777 shutdown_connections 2024-04-24T08:48:35.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:35.989+0000 7f2ef9200640 1 -- 172.21.15.83:0/2747700777 wait complete. 2024-04-24T08:48:35.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:35.990+0000 7f2ef9200640 1 Processor -- start 2024-04-24T08:48:35.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:35.990+0000 7f2ef9200640 1 -- start start 2024-04-24T08:48:35.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:35.991+0000 7f2ef9200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2ef4108480 0x7f2ef419e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:35.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:35.991+0000 7f2ef9200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ef4109670 0x7f2ef419eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:35.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:35.991+0000 7f2ef9200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ef410a030 0x7f2ef41a5c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:35.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:35.991+0000 7f2ef9200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2ef4114010 con 0x7f2ef4109670 2024-04-24T08:48:35.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:35.991+0000 7f2ef9200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2ef4113e90 con 0x7f2ef410a030 2024-04-24T08:48:35.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:35.991+0000 7f2ef9200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2ef4114190 con 0x7f2ef4108480 2024-04-24T08:48:35.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:35.991+0000 7f2ef2a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ef4109670 0x7f2ef419eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:35.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:35.991+0000 7f2ef2a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ef4109670 0x7f2ef419eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51574/0 (socket says 172.21.15.83:51574) 2024-04-24T08:48:35.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:35.991+0000 7f2ef2a00640 1 -- 172.21.15.83:0/2974429099 learned_addr learned my addr 172.21.15.83:0/2974429099 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:48:35.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:35.991+0000 7f2ef3400640 1 --2- 172.21.15.83:0/2974429099 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2ef4108480 0x7f2ef419e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:35.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:35.992+0000 7f2ef3e00640 1 --2- 172.21.15.83:0/2974429099 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ef410a030 0x7f2ef41a5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:35.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:35.992+0000 7f2ef2a00640 1 -- 172.21.15.83:0/2974429099 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2ef4108480 msgr2=0x7f2ef419e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:35.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:35.992+0000 7f2ef2a00640 1 --2- 172.21.15.83:0/2974429099 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2ef4108480 0x7f2ef419e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:35.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:35.992+0000 7f2ef2a00640 1 -- 172.21.15.83:0/2974429099 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ef410a030 msgr2=0x7f2ef41a5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:35.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:35.992+0000 7f2ef2a00640 1 --2- 172.21.15.83:0/2974429099 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ef410a030 0x7f2ef41a5c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:35.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:35.992+0000 7f2ef2a00640 1 -- 172.21.15.83:0/2974429099 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ef41a6280 con 0x7f2ef4109670 2024-04-24T08:48:35.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:35.992+0000 7f2ef2a00640 1 --2- 172.21.15.83:0/2974429099 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ef4109670 0x7f2ef419eba0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2edc00d770 tx=0x7f2edc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:48:35.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:35.992+0000 7f2eebe00640 1 -- 172.21.15.83:0/2974429099 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2edc017070 con 0x7f2ef4109670 2024-04-24T08:48:35.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:35.992+0000 7f2eebe00640 1 -- 172.21.15.83:0/2974429099 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2edc0054d0 con 0x7f2ef4109670 2024-04-24T08:48:35.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:35.992+0000 7f2eebe00640 1 -- 172.21.15.83:0/2974429099 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2edc004cf0 con 0x7f2ef4109670 2024-04-24T08:48:35.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:35.992+0000 7f2ef9200640 1 -- 172.21.15.83:0/2974429099 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ef41a6570 con 0x7f2ef4109670 2024-04-24T08:48:35.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:35.992+0000 7f2ef9200640 1 -- 172.21.15.83:0/2974429099 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ef41a6ac0 con 0x7f2ef4109670 2024-04-24T08:48:35.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:35.993+0000 7f2ef9200640 1 -- 172.21.15.83:0/2974429099 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ef410f7a0 con 0x7f2ef4109670 2024-04-24T08:48:35.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:35.995+0000 7f2eebe00640 1 -- 172.21.15.83:0/2974429099 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f2edc004e90 con 0x7f2ef4109670 2024-04-24T08:48:35.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:35.997+0000 7f2eebe00640 1 --2- 172.21.15.83:0/2974429099 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f2ed80a3c30 0x7f2ed80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:35.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:35.997+0000 7f2eebe00640 1 -- 172.21.15.83:0/2974429099 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f2edc0d0140 con 0x7f2ef4109670 2024-04-24T08:48:35.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:35.997+0000 7f2ef3400640 1 --2- 172.21.15.83:0/2974429099 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f2ed80a3c30 0x7f2ed80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:35.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:35.998+0000 7f2ef3400640 1 --2- 172.21.15.83:0/2974429099 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f2ed80a3c30 0x7f2ed80a60f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2ee4005fd0 tx=0x7f2ee4005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:48:36.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:36.001+0000 7f2eebe00640 1 -- 172.21.15.83:0/2974429099 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2edc09c880 con 0x7f2ef4109670 2024-04-24T08:48:36.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:36.203+0000 7f2ef9200640 1 -- 172.21.15.83:0/2974429099 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2ef4062c30 con 0x7f2ef4109670 2024-04-24T08:48:36.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:36.208+0000 7f2eebe00640 1 -- 172.21.15.83:0/2974429099 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24127 (secure 0 0 0) 0x7f2edc09c180 con 0x7f2ef4109670 2024-04-24T08:48:36.209 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:48:36.209 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":89,"fsid":"bd3e039e-0215-11ef-bc93-c7b262605968","created":"2024-04-24T08:37:32.209944+0000","modified":"2024-04-24T08:48:34.752923+0000","last_up_change":"2024-04-24T08:47:17.966041+0000","last_in_change":"2024-04-24T08:46:53.835936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T08:41:56.834372+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T08:48:26.978311+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T08:48:28.352731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T08:48:32.275167+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2cd77f3a-19dd-4c6e-a199-1dcbd5faa679","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2132675555},{"type":"v1","addr":"172.21.15.83:6803","nonce":2132675555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2132675555},{"type":"v1","addr":"172.21.15.83:6805","nonce":2132675555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2132675555},{"type":"v1","addr":"172.21.15.83:6809","nonce":2132675555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2132675555},{"type":"v1","addr":"172.21.15.83:6807","nonce":2132675555}]},"public_addr":"172.21.15.83:6803/2132675555","cluster_addr":"172.21.15.83:6805/2132675555","heartbeat_back_addr":"172.21.15.83:6809/2132675555","heartbeat_front_addr":"172.21.15.83:6807/2132675555","state":["exists","up"]},{"osd":1,"uuid":"aba7f5df-2a4c-4c23-b502-d193605a5b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":3589158880},{"type":"v1","addr":"172.21.15.138:6801","nonce":3589158880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":3589158880},{"type":"v1","addr":"172.21.15.138:6803","nonce":3589158880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":3589158880},{"type":"v1","addr":"172.21.15.138:6807","nonce":3589158880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":3589158880},{"type":"v1","addr":"172.21.15.138:6805","nonce":3589158880}]},"public_addr":"172.21.15.138:6801/3589158880","cluster_addr":"172.21.15.138:6803/3589158880","heartbeat_back_addr":"172.21.15.138:6807/3589158880","heartbeat_front_addr":"172.21.15.138:6805/3589158880","state":["exists","up"]},{"osd":2,"uuid":"071bc167-2333-4420-bb58-bffbc10d7641","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":3169072688},{"type":"v1","addr":"172.21.15.189:6801","nonce":3169072688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":3169072688},{"type":"v1","addr":"172.21.15.189:6803","nonce":3169072688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":3169072688},{"type":"v1","addr":"172.21.15.189:6807","nonce":3169072688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":3169072688},{"type":"v1","addr":"172.21.15.189:6805","nonce":3169072688}]},"public_addr":"172.21.15.189:6801/3169072688","cluster_addr":"172.21.15.189:6803/3169072688","heartbeat_back_addr":"172.21.15.189:6807/3169072688","heartbeat_front_addr":"172.21.15.189:6805/3169072688","state":["exists","up"]},{"osd":3,"uuid":"f5624d78-4455-4e07-8c1a-f47c94cddb23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3131092450},{"type":"v1","addr":"172.21.15.83:6811","nonce":3131092450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3131092450},{"type":"v1","addr":"172.21.15.83:6813","nonce":3131092450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3131092450},{"type":"v1","addr":"172.21.15.83:6817","nonce":3131092450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3131092450},{"type":"v1","addr":"172.21.15.83:6815","nonce":3131092450}]},"public_addr":"172.21.15.83:6811/3131092450","cluster_addr":"172.21.15.83:6813/3131092450","heartbeat_back_addr":"172.21.15.83:6817/3131092450","heartbeat_front_addr":"172.21.15.83:6815/3131092450","state":["exists","up"]},{"osd":4,"uuid":"7b50cd14-9385-4535-a7b0-1c1da93c8548","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":823775360},{"type":"v1","addr":"172.21.15.138:6809","nonce":823775360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":823775360},{"type":"v1","addr":"172.21.15.138:6811","nonce":823775360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":823775360},{"type":"v1","addr":"172.21.15.138:6815","nonce":823775360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":823775360},{"type":"v1","addr":"172.21.15.138:6813","nonce":823775360}]},"public_addr":"172.21.15.138:6809/823775360","cluster_addr":"172.21.15.138:6811/823775360","heartbeat_back_addr":"172.21.15.138:6815/823775360","heartbeat_front_addr":"172.21.15.138:6813/823775360","state":["exists","up"]},{"osd":5,"uuid":"9c647797-35bc-449f-be84-af97d2c69c2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":1399817431},{"type":"v1","addr":"172.21.15.189:6809","nonce":1399817431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":1399817431},{"type":"v1","addr":"172.21.15.189:6811","nonce":1399817431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":1399817431},{"type":"v1","addr":"172.21.15.189:6815","nonce":1399817431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":1399817431},{"type":"v1","addr":"172.21.15.189:6813","nonce":1399817431}]},"public_addr":"172.21.15.189:6809/1399817431","cluster_addr":"172.21.15.189:6811/1399817431","heartbeat_back_addr":"172.21.15.189:6815/1399817431","heartbeat_front_addr":"172.21.15.189:6813/1399817431","state":["exists","up"]},{"osd":6,"uuid":"8f056584-16c8-4454-b445-3f780f26c196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":1611358483},{"type":"v1","addr":"172.21.15.83:6819","nonce":1611358483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1611358483},{"type":"v1","addr":"172.21.15.83:6821","nonce":1611358483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1611358483},{"type":"v1","addr":"172.21.15.83:6825","nonce":1611358483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":1611358483},{"type":"v1","addr":"172.21.15.83:6823","nonce":1611358483}]},"public_addr":"172.21.15.83:6819/1611358483","cluster_addr":"172.21.15.83:6821/1611358483","heartbeat_back_addr":"172.21.15.83:6825/1611358483","heartbeat_front_addr":"172.21.15.83:6823/1611358483","state":["exists","up"]},{"osd":7,"uuid":"7603efe3-da77-4b7f-9d03-29c4f293b6d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":3557856706},{"type":"v1","addr":"172.21.15.138:6817","nonce":3557856706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":3557856706},{"type":"v1","addr":"172.21.15.138:6819","nonce":3557856706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":3557856706},{"type":"v1","addr":"172.21.15.138:6823","nonce":3557856706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":3557856706},{"type":"v1","addr":"172.21.15.138:6821","nonce":3557856706}]},"public_addr":"172.21.15.138:6817/3557856706","cluster_addr":"172.21.15.138:6819/3557856706","heartbeat_back_addr":"172.21.15.138:6823/3557856706","heartbeat_front_addr":"172.21.15.138:6821/3557856706","state":["exists","up"]},{"osd":8,"uuid":"27d7d1b8-d6a0-417e-b8aa-596325aa1a77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":207018554},{"type":"v1","addr":"172.21.15.189:6817","nonce":207018554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":207018554},{"type":"v1","addr":"172.21.15.189:6819","nonce":207018554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":207018554},{"type":"v1","addr":"172.21.15.189:6823","nonce":207018554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":207018554},{"type":"v1","addr":"172.21.15.189:6821","nonce":207018554}]},"public_addr":"172.21.15.189:6817/207018554","cluster_addr":"172.21.15.189:6819/207018554","heartbeat_back_addr":"172.21.15.189:6823/207018554","heartbeat_front_addr":"172.21.15.189:6821/207018554","state":["exists","up"]},{"osd":9,"uuid":"baf19b98-881b-4d3a-9f72-8aa2dd73cad6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":519636751},{"type":"v1","addr":"172.21.15.83:6827","nonce":519636751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":519636751},{"type":"v1","addr":"172.21.15.83:6829","nonce":519636751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":519636751},{"type":"v1","addr":"172.21.15.83:6833","nonce":519636751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":519636751},{"type":"v1","addr":"172.21.15.83:6831","nonce":519636751}]},"public_addr":"172.21.15.83:6827/519636751","cluster_addr":"172.21.15.83:6829/519636751","heartbeat_back_addr":"172.21.15.83:6833/519636751","heartbeat_front_addr":"172.21.15.83:6831/519636751","state":["exists","up"]},{"osd":10,"uuid":"eb346717-5c07-47c8-95be-0e68fe449fdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1130872098},{"type":"v1","addr":"172.21.15.138:6825","nonce":1130872098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1130872098},{"type":"v1","addr":"172.21.15.138:6827","nonce":1130872098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1130872098},{"type":"v1","addr":"172.21.15.138:6831","nonce":1130872098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1130872098},{"type":"v1","addr":"172.21.15.138:6829","nonce":1130872098}]},"public_addr":"172.21.15.138:6825/1130872098","cluster_addr":"172.21.15.138:6827/1130872098","heartbeat_back_addr":"172.21.15.138:6831/1130872098","heartbeat_front_addr":"172.21.15.138:6829/1130872098","state":["exists","up"]},{"osd":11,"uuid":"8e37568e-981c-45b8-b2af-2f1b73898542","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":2998782690},{"type":"v1","addr":"172.21.15.189:6825","nonce":2998782690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":2998782690},{"type":"v1","addr":"172.21.15.189:6827","nonce":2998782690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":2998782690},{"type":"v1","addr":"172.21.15.189:6831","nonce":2998782690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":2998782690},{"type":"v1","addr":"172.21.15.189:6829","nonce":2998782690}]},"public_addr":"172.21.15.189:6825/2998782690","cluster_addr":"172.21.15.189:6827/2998782690","heartbeat_back_addr":"172.21.15.189:6831/2998782690","heartbeat_front_addr":"172.21.15.189:6829/2998782690","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:40:47.980103+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:41:21.970795+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:41:55.160426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:42:30.400475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:43:05.705663+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:43:40.112312+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:44:16.781289+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:44:52.151187+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:45:28.055944+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:46:04.969301+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:46:40.918306+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:47:16.886874+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:6801/820049883":"2024-04-25T08:38:40.661315+0000","172.21.15.83:0/4272874138":"2024-04-25T08:38:40.661315+0000","172.21.15.83:0/4212994335":"2024-04-25T08:38:40.661315+0000","172.21.15.83:6801/1996371091":"2024-04-25T08:38:05.800856+0000","172.21.15.83:0/1820663683":"2024-04-25T08:38:05.800856+0000","172.21.15.83:0/3802897262":"2024-04-25T08:38:05.800856+0000","172.21.15.83:6800/1996371091":"2024-04-25T08:38:05.800856+0000","172.21.15.83:6800/820049883":"2024-04-25T08:38:40.661315+0000","172.21.15.83:0/1366125049":"2024-04-25T08:38:05.800856+0000","172.21.15.83:0/340452868":"2024-04-25T08:38:40.661315+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T08:48:36.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:36.210+0000 7f2ef9200640 1 -- 172.21.15.83:0/2974429099 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f2ed80a3c30 msgr2=0x7f2ed80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:36.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:36.210+0000 7f2ef9200640 1 --2- 172.21.15.83:0/2974429099 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f2ed80a3c30 0x7f2ed80a60f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2ee4005fd0 tx=0x7f2ee4005850 comp rx=0 tx=0).stop 2024-04-24T08:48:36.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:36.210+0000 7f2ef9200640 1 -- 172.21.15.83:0/2974429099 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ef4109670 msgr2=0x7f2ef419eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:36.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:36.210+0000 7f2ef9200640 1 --2- 172.21.15.83:0/2974429099 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ef4109670 0x7f2ef419eba0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2edc00d770 tx=0x7f2edc00dc30 comp rx=0 tx=0).stop 2024-04-24T08:48:36.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:36.210+0000 7f2ef9200640 1 -- 172.21.15.83:0/2974429099 shutdown_connections 2024-04-24T08:48:36.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:36.210+0000 7f2ef9200640 1 --2- 172.21.15.83:0/2974429099 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ef410a030 0x7f2ef41a5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:36.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:36.210+0000 7f2ef9200640 1 --2- 172.21.15.83:0/2974429099 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f2ed80a3c30 0x7f2ed80a60f0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:36.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:36.210+0000 7f2ef9200640 1 --2- 172.21.15.83:0/2974429099 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ef4109670 0x7f2ef419eba0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:36.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:36.210+0000 7f2ef9200640 1 --2- 172.21.15.83:0/2974429099 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2ef4108480 0x7f2ef419e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:36.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:36.210+0000 7f2ef9200640 1 -- 172.21.15.83:0/2974429099 >> 172.21.15.83:0/2974429099 conn(0x7f2ef4077440 msgr2=0x7f2ef410c970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:48:36.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:36.211+0000 7f2ef9200640 1 -- 172.21.15.83:0/2974429099 shutdown_connections 2024-04-24T08:48:36.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:36.211+0000 7f2ef9200640 1 -- 172.21.15.83:0/2974429099 wait complete. 2024-04-24T08:48:36.230 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T08:48:36.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:36.434+0000 7f4f8e800640 1 -- 172.21.15.83:0/3163045549 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f88108480 msgr2=0x7f4f881088a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:36.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:36.434+0000 7f4f8e800640 1 --2- 172.21.15.83:0/3163045549 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f88108480 0x7f4f881088a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c009900 tx=0x7f4f7c02a3f0 comp rx=0 tx=0).stop 2024-04-24T08:48:36.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:36.434+0000 7f4f8e800640 1 -- 172.21.15.83:0/3163045549 shutdown_connections 2024-04-24T08:48:36.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:36.434+0000 7f4f8e800640 1 --2- 172.21.15.83:0/3163045549 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4f8810a030 0x7f4f881108c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:36.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:36.434+0000 7f4f8e800640 1 --2- 172.21.15.83:0/3163045549 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4f88109670 0x7f4f88109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:36.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:36.434+0000 7f4f8e800640 1 --2- 172.21.15.83:0/3163045549 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f88108480 0x7f4f881088a0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:36.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:36.434+0000 7f4f8e800640 1 -- 172.21.15.83:0/3163045549 >> 172.21.15.83:0/3163045549 conn(0x7f4f88077440 msgr2=0x7f4f88079860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:48:36.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:36.434+0000 7f4f8e800640 1 -- 172.21.15.83:0/3163045549 shutdown_connections 2024-04-24T08:48:36.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:36.434+0000 7f4f8e800640 1 -- 172.21.15.83:0/3163045549 wait complete. 2024-04-24T08:48:36.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:36.435+0000 7f4f8e800640 1 Processor -- start 2024-04-24T08:48:36.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:36.436+0000 7f4f8e800640 1 -- start start 2024-04-24T08:48:36.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:36.436+0000 7f4f8e800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4f88108480 0x7f4f8819e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:36.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:36.436+0000 7f4f8e800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f88109670 0x7f4f8819ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:36.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:36.436+0000 7f4f8e800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4f8810a030 0x7f4f881a5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:36.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:36.436+0000 7f4f8e800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4f88114010 con 0x7f4f88109670 2024-04-24T08:48:36.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:36.436+0000 7f4f8e800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4f88113e90 con 0x7f4f8810a030 2024-04-24T08:48:36.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:36.436+0000 7f4f8e800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4f88114190 con 0x7f4f88108480 2024-04-24T08:48:36.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:36.436+0000 7f4f8ca00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f88109670 0x7f4f8819ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:36.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:36.436+0000 7f4f8ca00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f88109670 0x7f4f8819ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51598/0 (socket says 172.21.15.83:51598) 2024-04-24T08:48:36.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:36.436+0000 7f4f8ca00640 1 -- 172.21.15.83:0/3764506008 learned_addr learned my addr 172.21.15.83:0/3764506008 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:48:36.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:36.436+0000 7f4f8d400640 1 --2- 172.21.15.83:0/3764506008 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4f88108480 0x7f4f8819e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:36.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:36.437+0000 7f4f8de00640 1 --2- 172.21.15.83:0/3764506008 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4f8810a030 0x7f4f881a5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:36.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:36.437+0000 7f4f8ca00640 1 -- 172.21.15.83:0/3764506008 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4f88108480 msgr2=0x7f4f8819e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:36.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:36.437+0000 7f4f8ca00640 1 --2- 172.21.15.83:0/3764506008 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4f88108480 0x7f4f8819e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:36.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:36.437+0000 7f4f8ca00640 1 -- 172.21.15.83:0/3764506008 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4f8810a030 msgr2=0x7f4f881a5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:36.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:36.437+0000 7f4f8ca00640 1 --2- 172.21.15.83:0/3764506008 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4f8810a030 0x7f4f881a5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:36.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:36.437+0000 7f4f8ca00640 1 -- 172.21.15.83:0/3764506008 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f881a6320 con 0x7f4f88109670 2024-04-24T08:48:36.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:36.437+0000 7f4f8ca00640 1 --2- 172.21.15.83:0/3764506008 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f88109670 0x7f4f8819ec40 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f4f7400d950 tx=0x7f4f7400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:48:36.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:36.437+0000 7f4f86000640 1 -- 172.21.15.83:0/3764506008 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f74017070 con 0x7f4f88109670 2024-04-24T08:48:36.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:36.437+0000 7f4f86000640 1 -- 172.21.15.83:0/3764506008 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4f74009c30 con 0x7f4f88109670 2024-04-24T08:48:36.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:36.437+0000 7f4f86000640 1 -- 172.21.15.83:0/3764506008 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f7400f620 con 0x7f4f88109670 2024-04-24T08:48:36.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:36.437+0000 7f4f8e800640 1 -- 172.21.15.83:0/3764506008 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f881a6610 con 0x7f4f88109670 2024-04-24T08:48:36.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:36.437+0000 7f4f8e800640 1 -- 172.21.15.83:0/3764506008 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f881a6bb0 con 0x7f4f88109670 2024-04-24T08:48:36.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:36.438+0000 7f4f8e800640 1 -- 172.21.15.83:0/3764506008 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f8810f820 con 0x7f4f88109670 2024-04-24T08:48:36.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:36.443+0000 7f4f86000640 1 -- 172.21.15.83:0/3764506008 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f4f740109a0 con 0x7f4f88109670 2024-04-24T08:48:36.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:36.444+0000 7f4f86000640 1 --2- 172.21.15.83:0/3764506008 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f4f700a3c30 0x7f4f700a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:36.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:36.445+0000 7f4f86000640 1 -- 172.21.15.83:0/3764506008 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f4f740d0210 con 0x7f4f88109670 2024-04-24T08:48:36.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:36.445+0000 7f4f8d400640 1 --2- 172.21.15.83:0/3764506008 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f4f700a3c30 0x7f4f700a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:36.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:36.445+0000 7f4f8d400640 1 --2- 172.21.15.83:0/3764506008 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f4f700a3c30 0x7f4f700a60f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c006010 tx=0x7f4f7c0314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:48:36.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:36.446+0000 7f4f86000640 1 -- 172.21.15.83:0/3764506008 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4f74009de0 con 0x7f4f88109670 2024-04-24T08:48:36.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:36 smithi083 ceph-mon[27907]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-24T08:48:36.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:36 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/301656375' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-04-24T08:48:36.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:36 smithi083 ceph-mon[27907]: osdmap e89: 12 total, 12 up, 12 in 2024-04-24T08:48:36.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:36 smithi083 ceph-mon[27907]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-24T08:48:36.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:36 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/3262868640' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:48:36.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:36 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/2974429099' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T08:48:36.630 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:36 smithi138 ceph-mon[31441]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-24T08:48:36.630 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:36 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/301656375' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-04-24T08:48:36.630 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:36 smithi138 ceph-mon[31441]: osdmap e89: 12 total, 12 up, 12 in 2024-04-24T08:48:36.630 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:36 smithi138 ceph-mon[31441]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-24T08:48:36.630 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:36 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/3262868640' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:48:36.630 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:36 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/2974429099' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T08:48:36.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:36.654+0000 7f4f8e800640 1 -- 172.21.15.83:0/3764506008 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4f8802d870 con 0x7f4f88109670 2024-04-24T08:48:36.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:36.658+0000 7f4f86000640 1 -- 172.21.15.83:0/3764506008 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24127 (secure 0 0 0) 0x7f4f7409cad0 con 0x7f4f88109670 2024-04-24T08:48:36.659 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:48:36.659 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":89,"fsid":"bd3e039e-0215-11ef-bc93-c7b262605968","created":"2024-04-24T08:37:32.209944+0000","modified":"2024-04-24T08:48:34.752923+0000","last_up_change":"2024-04-24T08:47:17.966041+0000","last_in_change":"2024-04-24T08:46:53.835936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T08:41:56.834372+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T08:48:26.978311+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T08:48:28.352731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T08:48:32.275167+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2cd77f3a-19dd-4c6e-a199-1dcbd5faa679","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2132675555},{"type":"v1","addr":"172.21.15.83:6803","nonce":2132675555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2132675555},{"type":"v1","addr":"172.21.15.83:6805","nonce":2132675555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2132675555},{"type":"v1","addr":"172.21.15.83:6809","nonce":2132675555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2132675555},{"type":"v1","addr":"172.21.15.83:6807","nonce":2132675555}]},"public_addr":"172.21.15.83:6803/2132675555","cluster_addr":"172.21.15.83:6805/2132675555","heartbeat_back_addr":"172.21.15.83:6809/2132675555","heartbeat_front_addr":"172.21.15.83:6807/2132675555","state":["exists","up"]},{"osd":1,"uuid":"aba7f5df-2a4c-4c23-b502-d193605a5b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":3589158880},{"type":"v1","addr":"172.21.15.138:6801","nonce":3589158880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":3589158880},{"type":"v1","addr":"172.21.15.138:6803","nonce":3589158880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":3589158880},{"type":"v1","addr":"172.21.15.138:6807","nonce":3589158880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":3589158880},{"type":"v1","addr":"172.21.15.138:6805","nonce":3589158880}]},"public_addr":"172.21.15.138:6801/3589158880","cluster_addr":"172.21.15.138:6803/3589158880","heartbeat_back_addr":"172.21.15.138:6807/3589158880","heartbeat_front_addr":"172.21.15.138:6805/3589158880","state":["exists","up"]},{"osd":2,"uuid":"071bc167-2333-4420-bb58-bffbc10d7641","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":3169072688},{"type":"v1","addr":"172.21.15.189:6801","nonce":3169072688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":3169072688},{"type":"v1","addr":"172.21.15.189:6803","nonce":3169072688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":3169072688},{"type":"v1","addr":"172.21.15.189:6807","nonce":3169072688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":3169072688},{"type":"v1","addr":"172.21.15.189:6805","nonce":3169072688}]},"public_addr":"172.21.15.189:6801/3169072688","cluster_addr":"172.21.15.189:6803/3169072688","heartbeat_back_addr":"172.21.15.189:6807/3169072688","heartbeat_front_addr":"172.21.15.189:6805/3169072688","state":["exists","up"]},{"osd":3,"uuid":"f5624d78-4455-4e07-8c1a-f47c94cddb23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3131092450},{"type":"v1","addr":"172.21.15.83:6811","nonce":3131092450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3131092450},{"type":"v1","addr":"172.21.15.83:6813","nonce":3131092450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3131092450},{"type":"v1","addr":"172.21.15.83:6817","nonce":3131092450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3131092450},{"type":"v1","addr":"172.21.15.83:6815","nonce":3131092450}]},"public_addr":"172.21.15.83:6811/3131092450","cluster_addr":"172.21.15.83:6813/3131092450","heartbeat_back_addr":"172.21.15.83:6817/3131092450","heartbeat_front_addr":"172.21.15.83:6815/3131092450","state":["exists","up"]},{"osd":4,"uuid":"7b50cd14-9385-4535-a7b0-1c1da93c8548","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":823775360},{"type":"v1","addr":"172.21.15.138:6809","nonce":823775360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":823775360},{"type":"v1","addr":"172.21.15.138:6811","nonce":823775360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":823775360},{"type":"v1","addr":"172.21.15.138:6815","nonce":823775360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":823775360},{"type":"v1","addr":"172.21.15.138:6813","nonce":823775360}]},"public_addr":"172.21.15.138:6809/823775360","cluster_addr":"172.21.15.138:6811/823775360","heartbeat_back_addr":"172.21.15.138:6815/823775360","heartbeat_front_addr":"172.21.15.138:6813/823775360","state":["exists","up"]},{"osd":5,"uuid":"9c647797-35bc-449f-be84-af97d2c69c2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":1399817431},{"type":"v1","addr":"172.21.15.189:6809","nonce":1399817431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":1399817431},{"type":"v1","addr":"172.21.15.189:6811","nonce":1399817431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":1399817431},{"type":"v1","addr":"172.21.15.189:6815","nonce":1399817431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":1399817431},{"type":"v1","addr":"172.21.15.189:6813","nonce":1399817431}]},"public_addr":"172.21.15.189:6809/1399817431","cluster_addr":"172.21.15.189:6811/1399817431","heartbeat_back_addr":"172.21.15.189:6815/1399817431","heartbeat_front_addr":"172.21.15.189:6813/1399817431","state":["exists","up"]},{"osd":6,"uuid":"8f056584-16c8-4454-b445-3f780f26c196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":1611358483},{"type":"v1","addr":"172.21.15.83:6819","nonce":1611358483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1611358483},{"type":"v1","addr":"172.21.15.83:6821","nonce":1611358483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1611358483},{"type":"v1","addr":"172.21.15.83:6825","nonce":1611358483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":1611358483},{"type":"v1","addr":"172.21.15.83:6823","nonce":1611358483}]},"public_addr":"172.21.15.83:6819/1611358483","cluster_addr":"172.21.15.83:6821/1611358483","heartbeat_back_addr":"172.21.15.83:6825/1611358483","heartbeat_front_addr":"172.21.15.83:6823/1611358483","state":["exists","up"]},{"osd":7,"uuid":"7603efe3-da77-4b7f-9d03-29c4f293b6d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":3557856706},{"type":"v1","addr":"172.21.15.138:6817","nonce":3557856706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":3557856706},{"type":"v1","addr":"172.21.15.138:6819","nonce":3557856706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":3557856706},{"type":"v1","addr":"172.21.15.138:6823","nonce":3557856706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":3557856706},{"type":"v1","addr":"172.21.15.138:6821","nonce":3557856706}]},"public_addr":"172.21.15.138:6817/3557856706","cluster_addr":"172.21.15.138:6819/3557856706","heartbeat_back_addr":"172.21.15.138:6823/3557856706","heartbeat_front_addr":"172.21.15.138:6821/3557856706","state":["exists","up"]},{"osd":8,"uuid":"27d7d1b8-d6a0-417e-b8aa-596325aa1a77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":207018554},{"type":"v1","addr":"172.21.15.189:6817","nonce":207018554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":207018554},{"type":"v1","addr":"172.21.15.189:6819","nonce":207018554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":207018554},{"type":"v1","addr":"172.21.15.189:6823","nonce":207018554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":207018554},{"type":"v1","addr":"172.21.15.189:6821","nonce":207018554}]},"public_addr":"172.21.15.189:6817/207018554","cluster_addr":"172.21.15.189:6819/207018554","heartbeat_back_addr":"172.21.15.189:6823/207018554","heartbeat_front_addr":"172.21.15.189:6821/207018554","state":["exists","up"]},{"osd":9,"uuid":"baf19b98-881b-4d3a-9f72-8aa2dd73cad6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":519636751},{"type":"v1","addr":"172.21.15.83:6827","nonce":519636751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":519636751},{"type":"v1","addr":"172.21.15.83:6829","nonce":519636751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":519636751},{"type":"v1","addr":"172.21.15.83:6833","nonce":519636751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":519636751},{"type":"v1","addr":"172.21.15.83:6831","nonce":519636751}]},"public_addr":"172.21.15.83:6827/519636751","cluster_addr":"172.21.15.83:6829/519636751","heartbeat_back_addr":"172.21.15.83:6833/519636751","heartbeat_front_addr":"172.21.15.83:6831/519636751","state":["exists","up"]},{"osd":10,"uuid":"eb346717-5c07-47c8-95be-0e68fe449fdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1130872098},{"type":"v1","addr":"172.21.15.138:6825","nonce":1130872098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1130872098},{"type":"v1","addr":"172.21.15.138:6827","nonce":1130872098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1130872098},{"type":"v1","addr":"172.21.15.138:6831","nonce":1130872098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1130872098},{"type":"v1","addr":"172.21.15.138:6829","nonce":1130872098}]},"public_addr":"172.21.15.138:6825/1130872098","cluster_addr":"172.21.15.138:6827/1130872098","heartbeat_back_addr":"172.21.15.138:6831/1130872098","heartbeat_front_addr":"172.21.15.138:6829/1130872098","state":["exists","up"]},{"osd":11,"uuid":"8e37568e-981c-45b8-b2af-2f1b73898542","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":2998782690},{"type":"v1","addr":"172.21.15.189:6825","nonce":2998782690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":2998782690},{"type":"v1","addr":"172.21.15.189:6827","nonce":2998782690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":2998782690},{"type":"v1","addr":"172.21.15.189:6831","nonce":2998782690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":2998782690},{"type":"v1","addr":"172.21.15.189:6829","nonce":2998782690}]},"public_addr":"172.21.15.189:6825/2998782690","cluster_addr":"172.21.15.189:6827/2998782690","heartbeat_back_addr":"172.21.15.189:6831/2998782690","heartbeat_front_addr":"172.21.15.189:6829/2998782690","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:40:47.980103+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:41:21.970795+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:41:55.160426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:42:30.400475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:43:05.705663+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:43:40.112312+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:44:16.781289+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:44:52.151187+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:45:28.055944+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:46:04.969301+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:46:40.918306+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:47:16.886874+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:6801/820049883":"2024-04-25T08:38:40.661315+0000","172.21.15.83:0/4272874138":"2024-04-25T08:38:40.661315+0000","172.21.15.83:0/4212994335":"2024-04-25T08:38:40.661315+0000","172.21.15.83:6801/1996371091":"2024-04-25T08:38:05.800856+0000","172.21.15.83:0/1820663683":"2024-04-25T08:38:05.800856+0000","172.21.15.83:0/3802897262":"2024-04-25T08:38:05.800856+0000","172.21.15.83:6800/1996371091":"2024-04-25T08:38:05.800856+0000","172.21.15.83:6800/820049883":"2024-04-25T08:38:40.661315+0000","172.21.15.83:0/1366125049":"2024-04-25T08:38:05.800856+0000","172.21.15.83:0/340452868":"2024-04-25T08:38:40.661315+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T08:48:36.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:36.661+0000 7f4f8e800640 1 -- 172.21.15.83:0/3764506008 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f4f700a3c30 msgr2=0x7f4f700a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:36.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:36.661+0000 7f4f8e800640 1 --2- 172.21.15.83:0/3764506008 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f4f700a3c30 0x7f4f700a60f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c006010 tx=0x7f4f7c0314a0 comp rx=0 tx=0).stop 2024-04-24T08:48:36.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:36.661+0000 7f4f8e800640 1 -- 172.21.15.83:0/3764506008 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f88109670 msgr2=0x7f4f8819ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:36.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:36.661+0000 7f4f8e800640 1 --2- 172.21.15.83:0/3764506008 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f88109670 0x7f4f8819ec40 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f4f7400d950 tx=0x7f4f7400de10 comp rx=0 tx=0).stop 2024-04-24T08:48:36.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:36.661+0000 7f4f8e800640 1 -- 172.21.15.83:0/3764506008 shutdown_connections 2024-04-24T08:48:36.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:36.661+0000 7f4f8e800640 1 --2- 172.21.15.83:0/3764506008 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4f8810a030 0x7f4f881a5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:36.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:36.661+0000 7f4f8e800640 1 --2- 172.21.15.83:0/3764506008 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f4f700a3c30 0x7f4f700a60f0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:36.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:36.661+0000 7f4f8e800640 1 --2- 172.21.15.83:0/3764506008 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f88109670 0x7f4f8819ec40 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:36.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:36.661+0000 7f4f8e800640 1 --2- 172.21.15.83:0/3764506008 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4f88108480 0x7f4f8819e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:36.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:36.661+0000 7f4f8e800640 1 -- 172.21.15.83:0/3764506008 >> 172.21.15.83:0/3764506008 conn(0x7f4f88077440 msgr2=0x7f4f8810c9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:48:36.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:36.661+0000 7f4f8e800640 1 -- 172.21.15.83:0/3764506008 shutdown_connections 2024-04-24T08:48:36.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:36.661+0000 7f4f8e800640 1 -- 172.21.15.83:0/3764506008 wait complete. 2024-04-24T08:48:36.700 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T08:48:36.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:36 smithi189 ceph-mon[31791]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-24T08:48:36.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:36 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/301656375' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-04-24T08:48:36.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:36 smithi189 ceph-mon[31791]: osdmap e89: 12 total, 12 up, 12 in 2024-04-24T08:48:36.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:36 smithi189 ceph-mon[31791]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-24T08:48:36.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:36 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/3262868640' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:48:36.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:36 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/2974429099' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T08:48:36.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:36.901+0000 7f1eba000640 1 -- 172.21.15.83:0/1837660873 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1eb4108480 msgr2=0x7f1eb41088a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:36.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:36.901+0000 7f1eba000640 1 --2- 172.21.15.83:0/1837660873 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1eb4108480 0x7f1eb41088a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f1ea4009900 tx=0x7f1ea402a3f0 comp rx=0 tx=0).stop 2024-04-24T08:48:36.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:36.901+0000 7f1eba000640 1 -- 172.21.15.83:0/1837660873 shutdown_connections 2024-04-24T08:48:36.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:36.901+0000 7f1eba000640 1 --2- 172.21.15.83:0/1837660873 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1eb410a030 0x7f1eb41108c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:36.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:36.901+0000 7f1eba000640 1 --2- 172.21.15.83:0/1837660873 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1eb4109670 0x7f1eb4109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:36.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:36.901+0000 7f1eba000640 1 --2- 172.21.15.83:0/1837660873 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1eb4108480 0x7f1eb41088a0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:36.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:36.901+0000 7f1eba000640 1 -- 172.21.15.83:0/1837660873 >> 172.21.15.83:0/1837660873 conn(0x7f1eb4077440 msgr2=0x7f1eb4079860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:48:36.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:36.901+0000 7f1eba000640 1 -- 172.21.15.83:0/1837660873 shutdown_connections 2024-04-24T08:48:36.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:36.901+0000 7f1eba000640 1 -- 172.21.15.83:0/1837660873 wait complete. 2024-04-24T08:48:36.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:36.902+0000 7f1eba000640 1 Processor -- start 2024-04-24T08:48:36.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:36.902+0000 7f1eba000640 1 -- start start 2024-04-24T08:48:36.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:36.902+0000 7f1eba000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1eb4108480 0x7f1eb419e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:36.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:36.902+0000 7f1eba000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1eb4109670 0x7f1eb419ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:36.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:36.902+0000 7f1eba000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1eb410a030 0x7f1eb41a5c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:36.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:36.902+0000 7f1eba000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb4113f90 con 0x7f1eb4108480 2024-04-24T08:48:36.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:36.902+0000 7f1eba000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb4113e10 con 0x7f1eb410a030 2024-04-24T08:48:36.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:36.902+0000 7f1eba000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb4114110 con 0x7f1eb4109670 2024-04-24T08:48:36.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:36.903+0000 7f1eb8c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1eb4108480 0x7f1eb419e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:36.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:36.903+0000 7f1eb8c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1eb4108480 0x7f1eb419e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51610/0 (socket says 172.21.15.83:51610) 2024-04-24T08:48:36.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:36.903+0000 7f1eb8c00640 1 -- 172.21.15.83:0/1539128364 learned_addr learned my addr 172.21.15.83:0/1539128364 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:48:36.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:36.903+0000 7f1eabe00640 1 --2- 172.21.15.83:0/1539128364 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1eb4109670 0x7f1eb419ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:36.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:36.903+0000 7f1eb8c00640 1 -- 172.21.15.83:0/1539128364 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1eb4109670 msgr2=0x7f1eb419ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:36.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:36.903+0000 7f1eb9600640 1 --2- 172.21.15.83:0/1539128364 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1eb410a030 0x7f1eb41a5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:36.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:36.903+0000 7f1eb8c00640 1 --2- 172.21.15.83:0/1539128364 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1eb4109670 0x7f1eb419ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:36.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:36.903+0000 7f1eb8c00640 1 -- 172.21.15.83:0/1539128364 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1eb410a030 msgr2=0x7f1eb41a5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:36.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:36.903+0000 7f1eb8c00640 1 --2- 172.21.15.83:0/1539128364 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1eb410a030 0x7f1eb41a5c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:36.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:36.903+0000 7f1eb8c00640 1 -- 172.21.15.83:0/1539128364 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1eb41a6290 con 0x7f1eb4108480 2024-04-24T08:48:36.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:36.903+0000 7f1eb9600640 1 --2- 172.21.15.83:0/1539128364 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1eb410a030 0x7f1eb41a5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:48:36.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:36.903+0000 7f1eb8c00640 1 --2- 172.21.15.83:0/1539128364 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1eb4108480 0x7f1eb419e6c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1ea4009230 tx=0x7f1ea4004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:48:36.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:36.904+0000 7f1eb2000640 1 -- 172.21.15.83:0/1539128364 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea403f030 con 0x7f1eb4108480 2024-04-24T08:48:36.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:36.904+0000 7f1eb2000640 1 -- 172.21.15.83:0/1539128364 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1ea4003cb0 con 0x7f1eb4108480 2024-04-24T08:48:36.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:36.904+0000 7f1eba000640 1 -- 172.21.15.83:0/1539128364 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1eb41a6520 con 0x7f1eb4108480 2024-04-24T08:48:36.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:36.904+0000 7f1eb2000640 1 -- 172.21.15.83:0/1539128364 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea4035070 con 0x7f1eb4108480 2024-04-24T08:48:36.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:36.904+0000 7f1eba000640 1 -- 172.21.15.83:0/1539128364 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1eb41a6a60 con 0x7f1eb4108480 2024-04-24T08:48:36.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:36.905+0000 7f1eba000640 1 -- 172.21.15.83:0/1539128364 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1eb410f890 con 0x7f1eb4108480 2024-04-24T08:48:36.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:36.907+0000 7f1eb2000640 1 -- 172.21.15.83:0/1539128364 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f1ea4042030 con 0x7f1eb4108480 2024-04-24T08:48:36.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:36.908+0000 7f1eb2000640 1 --2- 172.21.15.83:0/1539128364 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f1e980a3c30 0x7f1e980a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:36.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:36.908+0000 7f1eb2000640 1 -- 172.21.15.83:0/1539128364 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f1ea40f0d10 con 0x7f1eb4108480 2024-04-24T08:48:36.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:36.908+0000 7f1eabe00640 1 --2- 172.21.15.83:0/1539128364 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f1e980a3c30 0x7f1e980a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:36.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:36.909+0000 7f1eabe00640 1 --2- 172.21.15.83:0/1539128364 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f1e980a3c30 0x7f1e980a60f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1eb419fb80 tx=0x7f1e9c005c60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:48:36.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:36.912+0000 7f1eb2000640 1 -- 172.21.15.83:0/1539128364 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1ea40bd4b0 con 0x7f1eb4108480 2024-04-24T08:48:37.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.161+0000 7f1eba000640 1 -- 172.21.15.83:0/1539128364 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1eb402d620 con 0x7f1eb4108480 2024-04-24T08:48:37.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.163+0000 7f1eb2000640 1 -- 172.21.15.83:0/1539128364 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11809 (secure 0 0 0) 0x7f1ea40bcdb0 con 0x7f1eb4108480 2024-04-24T08:48:37.164 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:48:37.164 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/358872257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":358872257},{"type":"v1","addr":"172.21.15.83:6835","nonce":358872257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3157070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3157070851},{"type":"v1","addr":"172.21.15.83:6837","nonce":3157070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14658,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6839/4085470844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":4085470844},{"type":"v1","addr":"172.21.15.83:6839","nonce":4085470844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14676,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6841/2634754210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":2634754210},{"type":"v1","addr":"172.21.15.83:6841","nonce":2634754210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/1859921310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1859921310},{"type":"v1","addr":"172.21.15.189:6833","nonce":1859921310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24421,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/2241923197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2241923197},{"type":"v1","addr":"172.21.15.189:6835","nonce":2241923197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24439,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/3172311220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":3172311220},{"type":"v1","addr":"172.21.15.189:6837","nonce":3172311220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24454,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/2548965336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":2548965336},{"type":"v1","addr":"172.21.15.189:6839","nonce":2548965336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34427,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/1157967658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1157967658},{"type":"v1","addr":"172.21.15.138:6833","nonce":1157967658}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34451,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1909219758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1909219758},{"type":"v1","addr":"172.21.15.138:6835","nonce":1909219758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":34493,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/900566754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":900566754},{"type":"v1","addr":"172.21.15.138:6837","nonce":900566754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T08:48:29.422576+0000","modified":"2024-04-24T08:48:34.753110+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34520":{"gid":34520,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.138:6839/2236932393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2236932393},{"type":"v1","addr":"172.21.15.138:6839","nonce":2236932393}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34520,"qdb_cluster":[34520]},"id":1}]} 2024-04-24T08:48:37.164 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 17 2024-04-24T08:48:37.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.166+0000 7f1eba000640 1 -- 172.21.15.83:0/1539128364 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f1e980a3c30 msgr2=0x7f1e980a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:37.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.166+0000 7f1eba000640 1 --2- 172.21.15.83:0/1539128364 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f1e980a3c30 0x7f1e980a60f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1eb419fb80 tx=0x7f1e9c005c60 comp rx=0 tx=0).stop 2024-04-24T08:48:37.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.166+0000 7f1eba000640 1 -- 172.21.15.83:0/1539128364 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1eb4108480 msgr2=0x7f1eb419e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:37.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.166+0000 7f1eba000640 1 --2- 172.21.15.83:0/1539128364 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1eb4108480 0x7f1eb419e6c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1ea4009230 tx=0x7f1ea4004340 comp rx=0 tx=0).stop 2024-04-24T08:48:37.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.166+0000 7f1eba000640 1 -- 172.21.15.83:0/1539128364 shutdown_connections 2024-04-24T08:48:37.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.166+0000 7f1eba000640 1 --2- 172.21.15.83:0/1539128364 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1eb410a030 0x7f1eb41a5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:37.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.166+0000 7f1eba000640 1 --2- 172.21.15.83:0/1539128364 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1eb4109670 0x7f1eb419ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:37.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.166+0000 7f1eba000640 1 --2- 172.21.15.83:0/1539128364 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f1e980a3c30 0x7f1e980a60f0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:37.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.166+0000 7f1eba000640 1 --2- 172.21.15.83:0/1539128364 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1eb4108480 0x7f1eb419e6c0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:37.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.166+0000 7f1eba000640 1 -- 172.21.15.83:0/1539128364 >> 172.21.15.83:0/1539128364 conn(0x7f1eb4077440 msgr2=0x7f1eb410ca60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:48:37.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.166+0000 7f1eba000640 1 -- 172.21.15.83:0/1539128364 shutdown_connections 2024-04-24T08:48:37.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.166+0000 7f1eba000640 1 -- 172.21.15.83:0/1539128364 wait complete. 2024-04-24T08:48:37.187 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T08:48:37.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.390+0000 7f451fe00640 1 -- 172.21.15.83:0/3104617842 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4520105c30 msgr2=0x7f452010c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:37.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.390+0000 7f451fe00640 1 --2- 172.21.15.83:0/3104617842 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4520105c30 0x7f452010c4c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4518009ca0 tx=0x7f451802a8c0 comp rx=0 tx=0).stop 2024-04-24T08:48:37.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.390+0000 7f451fe00640 1 -- 172.21.15.83:0/3104617842 shutdown_connections 2024-04-24T08:48:37.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.390+0000 7f451fe00640 1 --2- 172.21.15.83:0/3104617842 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4520105c30 0x7f452010c4c0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:37.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.390+0000 7f451fe00640 1 --2- 172.21.15.83:0/3104617842 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4520105270 0x7f45201056f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:37.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.390+0000 7f451fe00640 1 --2- 172.21.15.83:0/3104617842 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4520104080 0x7f45201044a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:37.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.390+0000 7f451fe00640 1 -- 172.21.15.83:0/3104617842 >> 172.21.15.83:0/3104617842 conn(0x7f45200ff610 msgr2=0x7f4520101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:48:37.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.390+0000 7f451fe00640 1 -- 172.21.15.83:0/3104617842 shutdown_connections 2024-04-24T08:48:37.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.390+0000 7f451fe00640 1 -- 172.21.15.83:0/3104617842 wait complete. 2024-04-24T08:48:37.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.391+0000 7f451fe00640 1 Processor -- start 2024-04-24T08:48:37.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.391+0000 7f451fe00640 1 -- start start 2024-04-24T08:48:37.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.392+0000 7f451fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4520104080 0x7f452019a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:37.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.392+0000 7f451fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4520105270 0x7f452019a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:37.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.392+0000 7f451fe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4520105c30 0x7f45201a1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:37.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.392+0000 7f451fe00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f452010fc10 con 0x7f4520105270 2024-04-24T08:48:37.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.392+0000 7f451fe00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f452010fa90 con 0x7f4520104080 2024-04-24T08:48:37.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.392+0000 7f451fe00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f452010fd90 con 0x7f4520105c30 2024-04-24T08:48:37.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.392+0000 7f4516000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4520105270 0x7f452019a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:37.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.392+0000 7f4516000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4520105270 0x7f452019a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51630/0 (socket says 172.21.15.83:51630) 2024-04-24T08:48:37.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.392+0000 7f4516000640 1 -- 172.21.15.83:0/1614942467 learned_addr learned my addr 172.21.15.83:0/1614942467 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:48:37.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.392+0000 7f451ea00640 1 --2- 172.21.15.83:0/1614942467 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4520104080 0x7f452019a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:37.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.392+0000 7f4516000640 1 -- 172.21.15.83:0/1614942467 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4520105c30 msgr2=0x7f45201a1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:37.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.392+0000 7f4516000640 1 --2- 172.21.15.83:0/1614942467 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4520105c30 0x7f45201a1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:37.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.392+0000 7f4516000640 1 -- 172.21.15.83:0/1614942467 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4520104080 msgr2=0x7f452019a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:37.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.392+0000 7f4516000640 1 --2- 172.21.15.83:0/1614942467 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4520104080 0x7f452019a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:37.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.392+0000 7f4516000640 1 -- 172.21.15.83:0/1614942467 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45201a1df0 con 0x7f4520105270 2024-04-24T08:48:37.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.393+0000 7f4516000640 1 --2- 172.21.15.83:0/1614942467 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4520105270 0x7f452019a710 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f450800c920 tx=0x7f450800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:48:37.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.393+0000 7f4517e00640 1 -- 172.21.15.83:0/1614942467 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4508007c70 con 0x7f4520105270 2024-04-24T08:48:37.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.393+0000 7f4517e00640 1 -- 172.21.15.83:0/1614942467 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4508007e10 con 0x7f4520105270 2024-04-24T08:48:37.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.393+0000 7f451fe00640 1 -- 172.21.15.83:0/1614942467 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45201a20e0 con 0x7f4520105270 2024-04-24T08:48:37.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.393+0000 7f4517e00640 1 -- 172.21.15.83:0/1614942467 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4508018820 con 0x7f4520105270 2024-04-24T08:48:37.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.393+0000 7f451fe00640 1 -- 172.21.15.83:0/1614942467 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45201a2630 con 0x7f4520105270 2024-04-24T08:48:37.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.394+0000 7f451fe00640 1 -- 172.21.15.83:0/1614942467 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f452010b3a0 con 0x7f4520105270 2024-04-24T08:48:37.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.396+0000 7f4517e00640 1 -- 172.21.15.83:0/1614942467 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f45080189c0 con 0x7f4520105270 2024-04-24T08:48:37.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.398+0000 7f4517e00640 1 --2- 172.21.15.83:0/1614942467 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f45040a3bb0 0x7f45040a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:37.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.398+0000 7f4517e00640 1 -- 172.21.15.83:0/1614942467 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f45080d0880 con 0x7f4520105270 2024-04-24T08:48:37.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.398+0000 7f451ea00640 1 --2- 172.21.15.83:0/1614942467 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f45040a3bb0 0x7f45040a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:37.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.398+0000 7f451ea00640 1 --2- 172.21.15.83:0/1614942467 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f45040a3bb0 0x7f45040a6070 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4510005fd0 tx=0x7f4510005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:48:37.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.401+0000 7f4517e00640 1 -- 172.21.15.83:0/1614942467 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4508014030 con 0x7f4520105270 2024-04-24T08:48:37.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:37 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/3764506008' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T08:48:37.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:37 smithi083 ceph-mon[27907]: pgmap v377: 129 pgs: 6 creating+peering, 106 active+clean, 17 unknown; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s wr, 8 op/s 2024-04-24T08:48:37.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:37 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/1539128364' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:48:37.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.590+0000 7f451fe00640 1 -- 172.21.15.83:0/1614942467 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4520062ca0 con 0x7f4520105270 2024-04-24T08:48:37.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.592+0000 7f4517e00640 1 -- 172.21.15.83:0/1614942467 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24127 (secure 0 0 0) 0x7f450809cfc0 con 0x7f4520105270 2024-04-24T08:48:37.594 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:48:37.594 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":89,"fsid":"bd3e039e-0215-11ef-bc93-c7b262605968","created":"2024-04-24T08:37:32.209944+0000","modified":"2024-04-24T08:48:34.752923+0000","last_up_change":"2024-04-24T08:47:17.966041+0000","last_in_change":"2024-04-24T08:46:53.835936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T08:41:56.834372+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T08:48:26.978311+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T08:48:28.352731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T08:48:32.275167+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2cd77f3a-19dd-4c6e-a199-1dcbd5faa679","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2132675555},{"type":"v1","addr":"172.21.15.83:6803","nonce":2132675555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2132675555},{"type":"v1","addr":"172.21.15.83:6805","nonce":2132675555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2132675555},{"type":"v1","addr":"172.21.15.83:6809","nonce":2132675555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2132675555},{"type":"v1","addr":"172.21.15.83:6807","nonce":2132675555}]},"public_addr":"172.21.15.83:6803/2132675555","cluster_addr":"172.21.15.83:6805/2132675555","heartbeat_back_addr":"172.21.15.83:6809/2132675555","heartbeat_front_addr":"172.21.15.83:6807/2132675555","state":["exists","up"]},{"osd":1,"uuid":"aba7f5df-2a4c-4c23-b502-d193605a5b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":3589158880},{"type":"v1","addr":"172.21.15.138:6801","nonce":3589158880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":3589158880},{"type":"v1","addr":"172.21.15.138:6803","nonce":3589158880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":3589158880},{"type":"v1","addr":"172.21.15.138:6807","nonce":3589158880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":3589158880},{"type":"v1","addr":"172.21.15.138:6805","nonce":3589158880}]},"public_addr":"172.21.15.138:6801/3589158880","cluster_addr":"172.21.15.138:6803/3589158880","heartbeat_back_addr":"172.21.15.138:6807/3589158880","heartbeat_front_addr":"172.21.15.138:6805/3589158880","state":["exists","up"]},{"osd":2,"uuid":"071bc167-2333-4420-bb58-bffbc10d7641","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":3169072688},{"type":"v1","addr":"172.21.15.189:6801","nonce":3169072688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":3169072688},{"type":"v1","addr":"172.21.15.189:6803","nonce":3169072688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":3169072688},{"type":"v1","addr":"172.21.15.189:6807","nonce":3169072688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":3169072688},{"type":"v1","addr":"172.21.15.189:6805","nonce":3169072688}]},"public_addr":"172.21.15.189:6801/3169072688","cluster_addr":"172.21.15.189:6803/3169072688","heartbeat_back_addr":"172.21.15.189:6807/3169072688","heartbeat_front_addr":"172.21.15.189:6805/3169072688","state":["exists","up"]},{"osd":3,"uuid":"f5624d78-4455-4e07-8c1a-f47c94cddb23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3131092450},{"type":"v1","addr":"172.21.15.83:6811","nonce":3131092450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3131092450},{"type":"v1","addr":"172.21.15.83:6813","nonce":3131092450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3131092450},{"type":"v1","addr":"172.21.15.83:6817","nonce":3131092450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3131092450},{"type":"v1","addr":"172.21.15.83:6815","nonce":3131092450}]},"public_addr":"172.21.15.83:6811/3131092450","cluster_addr":"172.21.15.83:6813/3131092450","heartbeat_back_addr":"172.21.15.83:6817/3131092450","heartbeat_front_addr":"172.21.15.83:6815/3131092450","state":["exists","up"]},{"osd":4,"uuid":"7b50cd14-9385-4535-a7b0-1c1da93c8548","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":823775360},{"type":"v1","addr":"172.21.15.138:6809","nonce":823775360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":823775360},{"type":"v1","addr":"172.21.15.138:6811","nonce":823775360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":823775360},{"type":"v1","addr":"172.21.15.138:6815","nonce":823775360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":823775360},{"type":"v1","addr":"172.21.15.138:6813","nonce":823775360}]},"public_addr":"172.21.15.138:6809/823775360","cluster_addr":"172.21.15.138:6811/823775360","heartbeat_back_addr":"172.21.15.138:6815/823775360","heartbeat_front_addr":"172.21.15.138:6813/823775360","state":["exists","up"]},{"osd":5,"uuid":"9c647797-35bc-449f-be84-af97d2c69c2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":1399817431},{"type":"v1","addr":"172.21.15.189:6809","nonce":1399817431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":1399817431},{"type":"v1","addr":"172.21.15.189:6811","nonce":1399817431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":1399817431},{"type":"v1","addr":"172.21.15.189:6815","nonce":1399817431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":1399817431},{"type":"v1","addr":"172.21.15.189:6813","nonce":1399817431}]},"public_addr":"172.21.15.189:6809/1399817431","cluster_addr":"172.21.15.189:6811/1399817431","heartbeat_back_addr":"172.21.15.189:6815/1399817431","heartbeat_front_addr":"172.21.15.189:6813/1399817431","state":["exists","up"]},{"osd":6,"uuid":"8f056584-16c8-4454-b445-3f780f26c196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":1611358483},{"type":"v1","addr":"172.21.15.83:6819","nonce":1611358483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1611358483},{"type":"v1","addr":"172.21.15.83:6821","nonce":1611358483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1611358483},{"type":"v1","addr":"172.21.15.83:6825","nonce":1611358483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":1611358483},{"type":"v1","addr":"172.21.15.83:6823","nonce":1611358483}]},"public_addr":"172.21.15.83:6819/1611358483","cluster_addr":"172.21.15.83:6821/1611358483","heartbeat_back_addr":"172.21.15.83:6825/1611358483","heartbeat_front_addr":"172.21.15.83:6823/1611358483","state":["exists","up"]},{"osd":7,"uuid":"7603efe3-da77-4b7f-9d03-29c4f293b6d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":3557856706},{"type":"v1","addr":"172.21.15.138:6817","nonce":3557856706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":3557856706},{"type":"v1","addr":"172.21.15.138:6819","nonce":3557856706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":3557856706},{"type":"v1","addr":"172.21.15.138:6823","nonce":3557856706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":3557856706},{"type":"v1","addr":"172.21.15.138:6821","nonce":3557856706}]},"public_addr":"172.21.15.138:6817/3557856706","cluster_addr":"172.21.15.138:6819/3557856706","heartbeat_back_addr":"172.21.15.138:6823/3557856706","heartbeat_front_addr":"172.21.15.138:6821/3557856706","state":["exists","up"]},{"osd":8,"uuid":"27d7d1b8-d6a0-417e-b8aa-596325aa1a77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":207018554},{"type":"v1","addr":"172.21.15.189:6817","nonce":207018554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":207018554},{"type":"v1","addr":"172.21.15.189:6819","nonce":207018554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":207018554},{"type":"v1","addr":"172.21.15.189:6823","nonce":207018554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":207018554},{"type":"v1","addr":"172.21.15.189:6821","nonce":207018554}]},"public_addr":"172.21.15.189:6817/207018554","cluster_addr":"172.21.15.189:6819/207018554","heartbeat_back_addr":"172.21.15.189:6823/207018554","heartbeat_front_addr":"172.21.15.189:6821/207018554","state":["exists","up"]},{"osd":9,"uuid":"baf19b98-881b-4d3a-9f72-8aa2dd73cad6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":519636751},{"type":"v1","addr":"172.21.15.83:6827","nonce":519636751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":519636751},{"type":"v1","addr":"172.21.15.83:6829","nonce":519636751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":519636751},{"type":"v1","addr":"172.21.15.83:6833","nonce":519636751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":519636751},{"type":"v1","addr":"172.21.15.83:6831","nonce":519636751}]},"public_addr":"172.21.15.83:6827/519636751","cluster_addr":"172.21.15.83:6829/519636751","heartbeat_back_addr":"172.21.15.83:6833/519636751","heartbeat_front_addr":"172.21.15.83:6831/519636751","state":["exists","up"]},{"osd":10,"uuid":"eb346717-5c07-47c8-95be-0e68fe449fdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1130872098},{"type":"v1","addr":"172.21.15.138:6825","nonce":1130872098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1130872098},{"type":"v1","addr":"172.21.15.138:6827","nonce":1130872098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1130872098},{"type":"v1","addr":"172.21.15.138:6831","nonce":1130872098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1130872098},{"type":"v1","addr":"172.21.15.138:6829","nonce":1130872098}]},"public_addr":"172.21.15.138:6825/1130872098","cluster_addr":"172.21.15.138:6827/1130872098","heartbeat_back_addr":"172.21.15.138:6831/1130872098","heartbeat_front_addr":"172.21.15.138:6829/1130872098","state":["exists","up"]},{"osd":11,"uuid":"8e37568e-981c-45b8-b2af-2f1b73898542","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":2998782690},{"type":"v1","addr":"172.21.15.189:6825","nonce":2998782690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":2998782690},{"type":"v1","addr":"172.21.15.189:6827","nonce":2998782690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":2998782690},{"type":"v1","addr":"172.21.15.189:6831","nonce":2998782690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":2998782690},{"type":"v1","addr":"172.21.15.189:6829","nonce":2998782690}]},"public_addr":"172.21.15.189:6825/2998782690","cluster_addr":"172.21.15.189:6827/2998782690","heartbeat_back_addr":"172.21.15.189:6831/2998782690","heartbeat_front_addr":"172.21.15.189:6829/2998782690","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:40:47.980103+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:41:21.970795+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:41:55.160426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:42:30.400475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:43:05.705663+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:43:40.112312+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:44:16.781289+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:44:52.151187+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:45:28.055944+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:46:04.969301+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:46:40.918306+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:47:16.886874+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:6801/820049883":"2024-04-25T08:38:40.661315+0000","172.21.15.83:0/4272874138":"2024-04-25T08:38:40.661315+0000","172.21.15.83:0/4212994335":"2024-04-25T08:38:40.661315+0000","172.21.15.83:6801/1996371091":"2024-04-25T08:38:05.800856+0000","172.21.15.83:0/1820663683":"2024-04-25T08:38:05.800856+0000","172.21.15.83:0/3802897262":"2024-04-25T08:38:05.800856+0000","172.21.15.83:6800/1996371091":"2024-04-25T08:38:05.800856+0000","172.21.15.83:6800/820049883":"2024-04-25T08:38:40.661315+0000","172.21.15.83:0/1366125049":"2024-04-25T08:38:05.800856+0000","172.21.15.83:0/340452868":"2024-04-25T08:38:40.661315+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T08:48:37.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.595+0000 7f451fe00640 1 -- 172.21.15.83:0/1614942467 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f45040a3bb0 msgr2=0x7f45040a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:37.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.595+0000 7f451fe00640 1 --2- 172.21.15.83:0/1614942467 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f45040a3bb0 0x7f45040a6070 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4510005fd0 tx=0x7f4510005850 comp rx=0 tx=0).stop 2024-04-24T08:48:37.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.595+0000 7f451fe00640 1 -- 172.21.15.83:0/1614942467 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4520105270 msgr2=0x7f452019a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:37.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.595+0000 7f451fe00640 1 --2- 172.21.15.83:0/1614942467 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4520105270 0x7f452019a710 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f450800c920 tx=0x7f450800cde0 comp rx=0 tx=0).stop 2024-04-24T08:48:37.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.595+0000 7f451fe00640 1 -- 172.21.15.83:0/1614942467 shutdown_connections 2024-04-24T08:48:37.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.595+0000 7f451fe00640 1 --2- 172.21.15.83:0/1614942467 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4520105c30 0x7f45201a1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:37.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.595+0000 7f451fe00640 1 --2- 172.21.15.83:0/1614942467 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f45040a3bb0 0x7f45040a6070 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:37.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.595+0000 7f451fe00640 1 --2- 172.21.15.83:0/1614942467 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4520105270 0x7f452019a710 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:37.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.595+0000 7f451fe00640 1 --2- 172.21.15.83:0/1614942467 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4520104080 0x7f452019a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:37.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.595+0000 7f451fe00640 1 -- 172.21.15.83:0/1614942467 >> 172.21.15.83:0/1614942467 conn(0x7f45200ff610 msgr2=0x7f4520108570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:48:37.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.595+0000 7f451fe00640 1 -- 172.21.15.83:0/1614942467 shutdown_connections 2024-04-24T08:48:37.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.595+0000 7f451fe00640 1 -- 172.21.15.83:0/1614942467 wait complete. 2024-04-24T08:48:37.615 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-24T08:48:29.422576+0000', 'modified': '2024-04-24T08:48:34.753110+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 34520}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_34520': {'gid': 34520, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.138:6839/2236932393', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6838', 'nonce': 2236932393}, {'type': 'v1', 'addr': '172.21.15.138:6839', 'nonce': 2236932393}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 34520, 'qdb_cluster': [34520]} 2024-04-24T08:48:37.615 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-24T08:48:37.615 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-24T08:48:37.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:37 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/3764506008' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T08:48:37.706 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:37 smithi138 ceph-mon[31441]: pgmap v377: 129 pgs: 6 creating+peering, 106 active+clean, 17 unknown; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s wr, 8 op/s 2024-04-24T08:48:37.706 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:37 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/1539128364' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:48:37.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:37 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/3764506008' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T08:48:37.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:37 smithi189 ceph-mon[31791]: pgmap v377: 129 pgs: 6 creating+peering, 106 active+clean, 17 unknown; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s wr, 8 op/s 2024-04-24T08:48:37.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:37 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/1539128364' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:48:37.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.809+0000 7f023fe00640 1 -- 172.21.15.83:0/39430323 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0240109730 msgr2=0x7f0240109bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:37.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.809+0000 7f023fe00640 1 --2- 172.21.15.83:0/39430323 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0240109730 0x7f0240109bb0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0228009900 tx=0x7f022802a410 comp rx=0 tx=0).stop 2024-04-24T08:48:37.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.809+0000 7f023fe00640 1 -- 172.21.15.83:0/39430323 shutdown_connections 2024-04-24T08:48:37.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.809+0000 7f023fe00640 1 --2- 172.21.15.83:0/39430323 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f024010a0f0 0x7f0240110980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:37.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.809+0000 7f023fe00640 1 --2- 172.21.15.83:0/39430323 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0240109730 0x7f0240109bb0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:37.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.809+0000 7f023fe00640 1 --2- 172.21.15.83:0/39430323 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0240108540 0x7f0240108960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:37.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.809+0000 7f023fe00640 1 -- 172.21.15.83:0/39430323 >> 172.21.15.83:0/39430323 conn(0x7f02400774d0 msgr2=0x7f0240079910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:48:37.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.809+0000 7f023fe00640 1 -- 172.21.15.83:0/39430323 shutdown_connections 2024-04-24T08:48:37.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.809+0000 7f023fe00640 1 -- 172.21.15.83:0/39430323 wait complete. 2024-04-24T08:48:37.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.810+0000 7f023fe00640 1 Processor -- start 2024-04-24T08:48:37.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.810+0000 7f023fe00640 1 -- start start 2024-04-24T08:48:37.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.810+0000 7f023fe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0240108540 0x7f024019e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:37.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.810+0000 7f023fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0240109730 0x7f024019ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:37.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.810+0000 7f023fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f024010a0f0 0x7f02401a5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:37.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.810+0000 7f023fe00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f02401140d0 con 0x7f024010a0f0 2024-04-24T08:48:37.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.810+0000 7f023fe00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0240113f50 con 0x7f0240109730 2024-04-24T08:48:37.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.810+0000 7f023fe00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0240114250 con 0x7f0240108540 2024-04-24T08:48:37.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.810+0000 7f023f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f024010a0f0 0x7f02401a5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:37.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.810+0000 7f023f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f024010a0f0 0x7f02401a5cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51648/0 (socket says 172.21.15.83:51648) 2024-04-24T08:48:37.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.810+0000 7f023f400640 1 -- 172.21.15.83:0/2489070789 learned_addr learned my addr 172.21.15.83:0/2489070789 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:48:37.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.811+0000 7f023ea00640 1 --2- 172.21.15.83:0/2489070789 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0240108540 0x7f024019e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:37.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.811+0000 7f0236000640 1 --2- 172.21.15.83:0/2489070789 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0240109730 0x7f024019ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:37.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.811+0000 7f023f400640 1 -- 172.21.15.83:0/2489070789 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0240108540 msgr2=0x7f024019e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:37.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.811+0000 7f023f400640 1 --2- 172.21.15.83:0/2489070789 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0240108540 0x7f024019e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:37.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.811+0000 7f023f400640 1 -- 172.21.15.83:0/2489070789 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0240109730 msgr2=0x7f024019ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:37.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.811+0000 7f023f400640 1 --2- 172.21.15.83:0/2489070789 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0240109730 0x7f024019ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:37.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.811+0000 7f023f400640 1 -- 172.21.15.83:0/2489070789 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02401a6320 con 0x7f024010a0f0 2024-04-24T08:48:37.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.811+0000 7f023ea00640 1 --2- 172.21.15.83:0/2489070789 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0240108540 0x7f024019e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:48:37.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.811+0000 7f023f400640 1 --2- 172.21.15.83:0/2489070789 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f024010a0f0 0x7f02401a5cb0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f023800eaa0 tx=0x7f023800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:48:37.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.811+0000 7f0237e00640 1 -- 172.21.15.83:0/2489070789 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f023800cdb0 con 0x7f024010a0f0 2024-04-24T08:48:37.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.811+0000 7f0237e00640 1 -- 172.21.15.83:0/2489070789 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0238005020 con 0x7f024010a0f0 2024-04-24T08:48:37.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.811+0000 7f0237e00640 1 -- 172.21.15.83:0/2489070789 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02380116d0 con 0x7f024010a0f0 2024-04-24T08:48:37.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.811+0000 7f023fe00640 1 -- 172.21.15.83:0/2489070789 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02401a6610 con 0x7f024010a0f0 2024-04-24T08:48:37.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.811+0000 7f023fe00640 1 -- 172.21.15.83:0/2489070789 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02401a6b10 con 0x7f024010a0f0 2024-04-24T08:48:37.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.815+0000 7f02483cd740 1 Processor -- start 2024-04-24T08:48:37.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.815+0000 7f0237e00640 1 -- 172.21.15.83:0/2489070789 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f02380051d0 con 0x7f024010a0f0 2024-04-24T08:48:37.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.815+0000 7f02483cd740 1 -- start start 2024-04-24T08:48:37.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.815+0000 7f02483cd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56155fb87260 0x56155fb87680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:37.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.815+0000 7f02483cd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56155fb87c50 0x56155fb8c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:37.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.815+0000 7f02483cd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56155fb8cdc0 0x56155fb8f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:37.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.815+0000 7f02483cd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56155faecb10 con 0x56155fb87c50 2024-04-24T08:48:37.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.815+0000 7f02483cd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56155fa932b0 con 0x56155fb87260 2024-04-24T08:48:37.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.815+0000 7f02483cd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56155fae8ad0 con 0x56155fb8cdc0 2024-04-24T08:48:37.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.815+0000 7f023ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56155fb87c50 0x56155fb8c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:37.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.816+0000 7f023ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56155fb87c50 0x56155fb8c750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51658/0 (socket says 172.21.15.83:51658) 2024-04-24T08:48:37.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.816+0000 7f023ea00640 1 -- 172.21.15.83:0/202604036 learned_addr learned my addr 172.21.15.83:0/202604036 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:48:37.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.816+0000 7f0236000640 1 --2- 172.21.15.83:0/202604036 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56155fb87260 0x56155fb87680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:37.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.816+0000 7f0236000640 1 --2- 172.21.15.83:0/202604036 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56155fb8cdc0 0x56155fb8f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:37.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.816+0000 7f023ea00640 1 -- 172.21.15.83:0/202604036 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56155fb8cdc0 msgr2=0x56155fb8f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:48:37.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.816+0000 7f023ea00640 1 --2- 172.21.15.83:0/202604036 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56155fb8cdc0 0x56155fb8f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:37.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.816+0000 7f023ea00640 1 -- 172.21.15.83:0/202604036 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56155fb87260 msgr2=0x56155fb87680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:48:37.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.816+0000 7f023ea00640 1 --2- 172.21.15.83:0/202604036 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56155fb87260 0x56155fb87680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:37.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.816+0000 7f023ea00640 1 -- 172.21.15.83:0/202604036 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56155f7ff2d0 con 0x56155fb87c50 2024-04-24T08:48:37.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.816+0000 7f0237e00640 1 --2- 172.21.15.83:0/2489070789 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f02240a3c30 0x7f02240a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:37.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.816+0000 7f023ea00640 1 --2- 172.21.15.83:0/202604036 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56155fb87c50 0x56155fb8c750 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f0230009c30 tx=0x7f023000e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e64c0ff5fe2b3d79 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:48:37.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.816+0000 7f0237e00640 1 -- 172.21.15.83:0/2489070789 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f02380d0870 con 0x7f024010a0f0 2024-04-24T08:48:37.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.817+0000 7f0236000640 1 --2- 172.21.15.83:0/2489070789 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f02240a3c30 0x7f02240a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:37.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.817+0000 7f022f400640 1 -- 172.21.15.83:0/202604036 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0230016070 con 0x56155fb87c50 2024-04-24T08:48:37.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.817+0000 7f022f400640 1 -- 172.21.15.83:0/202604036 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f02300054d0 con 0x56155fb87c50 2024-04-24T08:48:37.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.817+0000 7f022f400640 1 -- 172.21.15.83:0/202604036 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0230011470 con 0x56155fb87c50 2024-04-24T08:48:37.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.817+0000 7f02483cd740 1 -- 172.21.15.83:0/202604036 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56155fb87c50 msgr2=0x56155fb8c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:37.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.817+0000 7f0236000640 1 --2- 172.21.15.83:0/2489070789 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f02240a3c30 0x7f02240a60f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x56155fb884b0 tx=0x7f02280314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:48:37.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.817+0000 7f02483cd740 1 --2- 172.21.15.83:0/202604036 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56155fb87c50 0x56155fb8c750 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f0230009c30 tx=0x7f023000e6c0 comp rx=0 tx=0).stop 2024-04-24T08:48:37.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.818+0000 7f02483cd740 1 -- 172.21.15.83:0/202604036 shutdown_connections 2024-04-24T08:48:37.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.818+0000 7f02483cd740 1 --2- 172.21.15.83:0/202604036 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56155fb8cdc0 0x56155fb8f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:37.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.818+0000 7f02483cd740 1 --2- 172.21.15.83:0/202604036 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56155fb87c50 0x56155fb8c750 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:37.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.818+0000 7f02483cd740 1 --2- 172.21.15.83:0/202604036 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56155fb87260 0x56155fb87680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:37.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.818+0000 7f02483cd740 1 -- 172.21.15.83:0/202604036 >> 172.21.15.83:0/202604036 conn(0x56155fb828d0 msgr2=0x56155fb82cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:48:37.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.818+0000 7f02483cd740 1 -- 172.21.15.83:0/202604036 shutdown_connections 2024-04-24T08:48:37.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.818+0000 7f02483cd740 1 -- 172.21.15.83:0/202604036 wait complete. 2024-04-24T08:48:37.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.818+0000 7f02483cd740 1 -- start start 2024-04-24T08:48:37.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.818+0000 7f02483cd740 1 Processor -- start 2024-04-24T08:48:37.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.819+0000 7f02483cd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56155fb87260 0x56155fb91680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:37.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.819+0000 7f02483cd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56155fb87c50 0x56155fb91bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:37.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.819+0000 7f02483cd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56155fb8cdc0 0x56155fb98c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:37.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.819+0000 7f02483cd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56155fb8fd70 con 0x56155fb87c50 2024-04-24T08:48:37.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.819+0000 7f02483cd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56155faa6590 con 0x56155fb8cdc0 2024-04-24T08:48:37.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.819+0000 7f02483cd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56155fb8fef0 con 0x56155fb87260 2024-04-24T08:48:37.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.819+0000 7f02483cd740 10 client.0 _ll_register_callbacks cb 0x56155fa893a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:48:37.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.819+0000 7f0236000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56155fb87c50 0x56155fb91bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:37.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.819+0000 7f0236000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56155fb87c50 0x56155fb91bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51660/0 (socket says 172.21.15.83:51660) 2024-04-24T08:48:37.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.819+0000 7f0236000640 1 -- 172.21.15.83:0/2360828022 learned_addr learned my addr 172.21.15.83:0/2360828022 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:48:37.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.819+0000 7f023f400640 1 --2- 172.21.15.83:0/2360828022 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56155fb8cdc0 0x56155fb98c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:37.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.819+0000 7f023ea00640 1 --2- 172.21.15.83:0/2360828022 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56155fb87260 0x56155fb91680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:37.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.819+0000 7f0236000640 1 -- 172.21.15.83:0/2360828022 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56155fb87260 msgr2=0x56155fb91680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:48:37.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.819+0000 7f0236000640 1 --2- 172.21.15.83:0/2360828022 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56155fb87260 0x56155fb91680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:37.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.819+0000 7f0236000640 1 -- 172.21.15.83:0/2360828022 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56155fb8cdc0 msgr2=0x56155fb98c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:48:37.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.819+0000 7f0236000640 1 --2- 172.21.15.83:0/2360828022 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56155fb8cdc0 0x56155fb98c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:37.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.819+0000 7f0236000640 1 -- 172.21.15.83:0/2360828022 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56155f8c96b0 con 0x56155fb87c50 2024-04-24T08:48:37.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.819+0000 7f023f400640 1 --2- 172.21.15.83:0/2360828022 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56155fb8cdc0 0x56155fb98c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:48:37.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.819+0000 7f023ea00640 1 --2- 172.21.15.83:0/2360828022 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56155fb87260 0x56155fb91680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:48:37.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.820+0000 7f0236000640 1 --2- 172.21.15.83:0/2360828022 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56155fb87c50 0x56155fb91bc0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f0228009900 tx=0x7f022802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b39cce95290c384 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:48:37.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.820+0000 7f0223e00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:48:37.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.820+0000 7f0223e00640 1 -- 172.21.15.83:0/2360828022 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f022804f020 con 0x56155fb87c50 2024-04-24T08:48:37.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.820+0000 7f0223e00640 1 -- 172.21.15.83:0/2360828022 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f02280035f0 con 0x56155fb87c50 2024-04-24T08:48:37.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.820+0000 7f02483cd740 1 -- 172.21.15.83:0/2360828022 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02300054d0 con 0x56155fb87c50 2024-04-24T08:48:37.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.820+0000 7f0223e00640 1 -- 172.21.15.83:0/2360828022 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f0228035040 con 0x56155fb87c50 2024-04-24T08:48:37.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.820+0000 7f02483cd740 10 client.14799 fetch_fsmap learned FSMap version 17 2024-04-24T08:48:37.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.820+0000 7f02483cd740 1 -- 172.21.15.83:0/2360828022 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x56155fb9bfe0 con 0x56155fb87c50 2024-04-24T08:48:37.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.821+0000 7f0223e00640 1 -- 172.21.15.83:0/2360828022 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f02300054d0 con 0x56155fb87c50 2024-04-24T08:48:37.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.821+0000 7f0223e00640 1 -- 172.21.15.83:0/2360828022 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 17) ==== 8342+0+0 (secure 0 0 0) 0x7f022804b740 con 0x56155fb87c50 2024-04-24T08:48:37.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.821+0000 7f02483cd740 10 client.14799 fetch_fsmap finished waiting for FSMap version 17 2024-04-24T08:48:37.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.821+0000 7f02483cd740 10 client.14799 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-24T08:48:37.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.821+0000 7f02483cd740 20 client.14799 populate_metadata read hostname 'smithi083' 2024-04-24T08:48:37.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.821+0000 7f02483cd740 1 --2- 172.21.15.83:0/2360828022 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56155fb9e730 0x56155fb9fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:37.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.821+0000 7f02483cd740 4 client.14799 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:48:37.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.821+0000 7f02483cd740 1 -- 172.21.15.83:0/2360828022 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56155fafa530 con 0x56155fb9e730 2024-04-24T08:48:37.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.822+0000 7f023ea00640 1 --2- 172.21.15.83:0/2360828022 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56155fb9e730 0x56155fb9fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:37.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.823+0000 7f023ea00640 1 --2- 172.21.15.83:0/2360828022 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56155fb9e730 0x56155fb9fba0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29e241fce32c3865 server_cookie=4514a8658a25eb4c in_seq=0 out_seq=0 2024-04-24T08:48:37.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.823+0000 7f0223e00640 10 client.14799 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:48:37.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.825+0000 7f0223e00640 1 -- 172.21.15.83:0/2360828022 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56155fafa530 con 0x56155fb9e730 2024-04-24T08:48:37.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.825+0000 7f0223e00640 10 client.14799 handle_command_reply: tid=0 2024-04-24T08:48:37.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.825+0000 7f02483cd740 1 client.14799 shutdown 2024-04-24T08:48:37.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.826+0000 7f02483cd740 1 -- 172.21.15.83:0/2360828022 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56155fb87c50 msgr2=0x56155fb91bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:37.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.826+0000 7f02483cd740 1 --2- 172.21.15.83:0/2360828022 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56155fb87c50 0x56155fb91bc0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f0228009900 tx=0x7f022802a410 comp rx=0 tx=0).stop 2024-04-24T08:48:37.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.826+0000 7f02483cd740 1 -- 172.21.15.83:0/2360828022 shutdown_connections 2024-04-24T08:48:37.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.826+0000 7f02483cd740 1 --2- 172.21.15.83:0/2360828022 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56155fb9e730 0x56155fb9fba0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:37.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.826+0000 7f02483cd740 1 --2- 172.21.15.83:0/2360828022 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56155fb8cdc0 0x56155fb98c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:37.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.826+0000 7f02483cd740 1 --2- 172.21.15.83:0/2360828022 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56155fb87c50 0x56155fb91bc0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:37.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.826+0000 7f0223e00640 0 client.14799 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:48:37.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.826+0000 7f02483cd740 1 --2- 172.21.15.83:0/2360828022 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56155fb87260 0x56155fb91680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:37.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.826+0000 7f02483cd740 1 -- 172.21.15.83:0/2360828022 >> 172.21.15.83:0/2360828022 conn(0x56155fb828d0 msgr2=0x56155fb81970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:48:37.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.826+0000 7f02483cd740 1 -- 172.21.15.83:0/2360828022 shutdown_connections 2024-04-24T08:48:37.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.826+0000 7f02483cd740 1 -- 172.21.15.83:0/2360828022 wait complete. 2024-04-24T08:48:37.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.826+0000 7f02483cd740 20 client.14799 trim_cache size 0 max 16384 2024-04-24T08:48:37.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.846+0000 7f02483cd740 1 Processor -- start 2024-04-24T08:48:37.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.846+0000 7f02483cd740 1 -- start start 2024-04-24T08:48:37.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.846+0000 7f02483cd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56155fb87260 0x56155fb89050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:37.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.846+0000 7f02483cd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56155fb87c50 0x56155fb89590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:37.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.846+0000 7f02483cd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56155fb8cdc0 0x56155fb98bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:37.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.846+0000 7f02483cd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56155fb9dee0 con 0x56155fb87c50 2024-04-24T08:48:37.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.846+0000 7f02483cd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56155fb9e5b0 con 0x56155fb8cdc0 2024-04-24T08:48:37.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.846+0000 7f02483cd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56155fb9ec80 con 0x56155fb87260 2024-04-24T08:48:37.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.847+0000 7f0236000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56155fb87c50 0x56155fb89590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:37.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.847+0000 7f0236000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56155fb87c50 0x56155fb89590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51664/0 (socket says 172.21.15.83:51664) 2024-04-24T08:48:37.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.847+0000 7f0236000640 1 -- 172.21.15.83:0/1929974583 learned_addr learned my addr 172.21.15.83:0/1929974583 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:48:37.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.847+0000 7f023f400640 1 --2- 172.21.15.83:0/1929974583 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56155fb8cdc0 0x56155fb98bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:37.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.847+0000 7f023ea00640 1 --2- 172.21.15.83:0/1929974583 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56155fb87260 0x56155fb89050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:37.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.847+0000 7f0236000640 1 -- 172.21.15.83:0/1929974583 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56155fb87260 msgr2=0x56155fb89050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:48:37.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.847+0000 7f0236000640 1 --2- 172.21.15.83:0/1929974583 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56155fb87260 0x56155fb89050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:37.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.847+0000 7f0236000640 1 -- 172.21.15.83:0/1929974583 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56155fb8cdc0 msgr2=0x56155fb98bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:48:37.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.847+0000 7f0236000640 1 --2- 172.21.15.83:0/1929974583 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56155fb8cdc0 0x56155fb98bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:37.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.847+0000 7f0236000640 1 -- 172.21.15.83:0/1929974583 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56155fb884e0 con 0x56155fb87c50 2024-04-24T08:48:37.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.847+0000 7f023ea00640 1 --2- 172.21.15.83:0/1929974583 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56155fb87260 0x56155fb89050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:48:37.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.847+0000 7f0236000640 1 --2- 172.21.15.83:0/1929974583 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56155fb87c50 0x56155fb89590 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f022804bc80 tx=0x7f0228031d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17952c23a6b58840 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:48:37.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.848+0000 7f022ea00640 1 -- 172.21.15.83:0/1929974583 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f022804f070 con 0x56155fb87c50 2024-04-24T08:48:37.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.848+0000 7f022ea00640 1 -- 172.21.15.83:0/1929974583 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0228044030 con 0x56155fb87c50 2024-04-24T08:48:37.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.848+0000 7f022ea00640 1 -- 172.21.15.83:0/1929974583 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f022805ee60 con 0x56155fb87c50 2024-04-24T08:48:37.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.848+0000 7f02483cd740 1 -- 172.21.15.83:0/1929974583 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56155fb87c50 msgr2=0x56155fb89590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:37.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.848+0000 7f02483cd740 1 --2- 172.21.15.83:0/1929974583 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56155fb87c50 0x56155fb89590 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f022804bc80 tx=0x7f0228031d10 comp rx=0 tx=0).stop 2024-04-24T08:48:37.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.848+0000 7f02483cd740 1 -- 172.21.15.83:0/1929974583 shutdown_connections 2024-04-24T08:48:37.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.848+0000 7f02483cd740 1 --2- 172.21.15.83:0/1929974583 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56155fb8cdc0 0x56155fb98bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:37.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.848+0000 7f02483cd740 1 --2- 172.21.15.83:0/1929974583 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56155fb87c50 0x56155fb89590 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:37.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.848+0000 7f02483cd740 1 --2- 172.21.15.83:0/1929974583 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56155fb87260 0x56155fb89050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:37.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.848+0000 7f02483cd740 1 -- 172.21.15.83:0/1929974583 >> 172.21.15.83:0/1929974583 conn(0x56155fb828d0 msgr2=0x56155fb91680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:48:37.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.848+0000 7f02483cd740 1 -- 172.21.15.83:0/1929974583 shutdown_connections 2024-04-24T08:48:37.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.848+0000 7f02483cd740 1 -- 172.21.15.83:0/1929974583 wait complete. 2024-04-24T08:48:37.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.849+0000 7f02483cd740 1 -- start start 2024-04-24T08:48:37.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.849+0000 7f02483cd740 1 Processor -- start 2024-04-24T08:48:37.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.849+0000 7f02483cd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56155fb87260 0x56155fb94100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:37.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.849+0000 7f02483cd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56155fb87c50 0x56155fb94640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:37.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.849+0000 7f02483cd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56155fb8cdc0 0x56155fb97660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:37.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.850+0000 7f02483cd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56155fb9fa20 con 0x56155fb8cdc0 2024-04-24T08:48:37.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.850+0000 7f02483cd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56155fb9f350 con 0x56155fb87c50 2024-04-24T08:48:37.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.850+0000 7f02483cd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56155fba00f0 con 0x56155fb87260 2024-04-24T08:48:37.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.850+0000 7f02483cd740 10 client.0 _ll_register_callbacks cb 0x7f0228031580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:48:37.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.850+0000 7f023f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56155fb8cdc0 0x56155fb97660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:37.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.850+0000 7f023f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56155fb8cdc0 0x56155fb97660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51676/0 (socket says 172.21.15.83:51676) 2024-04-24T08:48:37.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.850+0000 7f023f400640 1 -- 172.21.15.83:0/749737942 learned_addr learned my addr 172.21.15.83:0/749737942 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:48:37.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.850+0000 7f0236000640 1 --2- 172.21.15.83:0/749737942 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56155fb87c50 0x56155fb94640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:37.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.850+0000 7f023ea00640 1 --2- 172.21.15.83:0/749737942 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56155fb87260 0x56155fb94100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:37.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.850+0000 7f023f400640 1 -- 172.21.15.83:0/749737942 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56155fb87260 msgr2=0x56155fb94100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:48:37.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.850+0000 7f023f400640 1 --2- 172.21.15.83:0/749737942 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56155fb87260 0x56155fb94100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:37.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.850+0000 7f023f400640 1 -- 172.21.15.83:0/749737942 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56155fb87c50 msgr2=0x56155fb94640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:48:37.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.850+0000 7f023f400640 1 --2- 172.21.15.83:0/749737942 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56155fb87c50 0x56155fb94640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:37.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.850+0000 7f023f400640 1 -- 172.21.15.83:0/749737942 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56155fb97c50 con 0x56155fb8cdc0 2024-04-24T08:48:37.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.850+0000 7f023f400640 1 --2- 172.21.15.83:0/749737942 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56155fb8cdc0 0x56155fb97660 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f023809c840 tx=0x7f02380a0c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=facc6aca99c09313 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:48:37.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.850+0000 7f0223e00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:48:37.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.851+0000 7f0223e00640 1 -- 172.21.15.83:0/749737942 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f023801f040 con 0x56155fb8cdc0 2024-04-24T08:48:37.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.851+0000 7f0223e00640 1 -- 172.21.15.83:0/749737942 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f023809dbe0 con 0x56155fb8cdc0 2024-04-24T08:48:37.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.851+0000 7f02483cd740 1 -- 172.21.15.83:0/749737942 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0228044030 con 0x56155fb8cdc0 2024-04-24T08:48:37.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.851+0000 7f0223e00640 1 -- 172.21.15.83:0/749737942 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f023809ac70 con 0x56155fb8cdc0 2024-04-24T08:48:37.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.851+0000 7f02483cd740 10 client.14805 fetch_fsmap learned FSMap version 17 2024-04-24T08:48:37.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.851+0000 7f02483cd740 1 -- 172.21.15.83:0/749737942 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x56155fb98f30 con 0x56155fb8cdc0 2024-04-24T08:48:37.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.851+0000 7f0223e00640 1 -- 172.21.15.83:0/749737942 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0228044030 con 0x56155fb8cdc0 2024-04-24T08:48:37.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.851+0000 7f0223e00640 1 -- 172.21.15.83:0/749737942 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 17) ==== 8342+0+0 (secure 0 0 0) 0x7f023800f040 con 0x56155fb8cdc0 2024-04-24T08:48:37.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.851+0000 7f02483cd740 10 client.14805 fetch_fsmap finished waiting for FSMap version 17 2024-04-24T08:48:37.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.851+0000 7f02483cd740 10 client.14805 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-24T08:48:37.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.851+0000 7f02483cd740 20 client.14805 populate_metadata read hostname 'smithi083' 2024-04-24T08:48:37.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.851+0000 7f02483cd740 1 --2- 172.21.15.83:0/749737942 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56155fc09090 0x56155fc07da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:37.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.851+0000 7f02483cd740 4 client.14805 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T08:48:37.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.851+0000 7f02483cd740 1 -- 172.21.15.83:0/749737942 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "status"}) -- 0x56155fb8f6d0 con 0x56155fc09090 2024-04-24T08:48:37.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.852+0000 7f023ea00640 1 --2- 172.21.15.83:0/749737942 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56155fc09090 0x56155fc07da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:37.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.853+0000 7f023ea00640 1 --2- 172.21.15.83:0/749737942 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56155fc09090 0x56155fc07da0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=115863d53ea36fa9 server_cookie=f77ccf70e83e99a9 in_seq=0 out_seq=0 2024-04-24T08:48:37.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.853+0000 7f0223e00640 10 client.14805 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:48:37.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.853+0000 7f0223e00640 1 -- 172.21.15.83:0/749737942 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x56155fb8f6d0 con 0x56155fc09090 2024-04-24T08:48:37.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.854+0000 7f0223e00640 10 client.14805 handle_command_reply: tid=0 2024-04-24T08:48:37.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.854+0000 7f02483cd740 1 client.14805 shutdown 2024-04-24T08:48:37.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.854+0000 7f02483cd740 1 -- 172.21.15.83:0/749737942 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56155fb8cdc0 msgr2=0x56155fb97660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:37.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.854+0000 7f02483cd740 1 --2- 172.21.15.83:0/749737942 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56155fb8cdc0 0x56155fb97660 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f023809c840 tx=0x7f02380a0c60 comp rx=0 tx=0).stop 2024-04-24T08:48:37.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.854+0000 7f02483cd740 1 -- 172.21.15.83:0/749737942 shutdown_connections 2024-04-24T08:48:37.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.854+0000 7f02483cd740 1 --2- 172.21.15.83:0/749737942 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56155fc09090 0x56155fc07da0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:37.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.854+0000 7f02483cd740 1 --2- 172.21.15.83:0/749737942 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56155fb8cdc0 0x56155fb97660 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:37.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.854+0000 7f02483cd740 1 --2- 172.21.15.83:0/749737942 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56155fb87c50 0x56155fb94640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:37.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.854+0000 7f0223e00640 0 client.14805 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:48:37.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.854+0000 7f02483cd740 1 --2- 172.21.15.83:0/749737942 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56155fb87260 0x56155fb94100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:37.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.854+0000 7f02483cd740 1 -- 172.21.15.83:0/749737942 >> 172.21.15.83:0/749737942 conn(0x56155fb828d0 msgr2=0x56155fb89050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:48:37.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.854+0000 7f02483cd740 1 -- 172.21.15.83:0/749737942 shutdown_connections 2024-04-24T08:48:37.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.854+0000 7f02483cd740 1 -- 172.21.15.83:0/749737942 wait complete. 2024-04-24T08:48:37.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.854+0000 7f02483cd740 20 client.14805 trim_cache size 0 max 16384 2024-04-24T08:48:37.858 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T08:48:37.858 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_fsid": "bd3e039e-0215-11ef-bc93-c7b262605968", 2024-04-24T08:48:37.858 INFO:teuthology.orchestra.run.smithi083.stdout: "whoami": 0, 2024-04-24T08:48:37.858 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 34520, 2024-04-24T08:48:37.858 INFO:teuthology.orchestra.run.smithi083.stdout: "want_state": "up:active", 2024-04-24T08:48:37.858 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "up:active", 2024-04-24T08:48:37.859 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs", 2024-04-24T08:48:37.859 INFO:teuthology.orchestra.run.smithi083.stdout: "rank_uptime": 7.6407250050000002, 2024-04-24T08:48:37.859 INFO:teuthology.orchestra.run.smithi083.stdout: "mdsmap_epoch": 17, 2024-04-24T08:48:37.859 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch": 89, 2024-04-24T08:48:37.859 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch_barrier": 88, 2024-04-24T08:48:37.859 INFO:teuthology.orchestra.run.smithi083.stdout: "uptime": 15.825851850999999 2024-04-24T08:48:37.859 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T08:48:37.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.855+0000 7f023fe00640 1 -- 172.21.15.83:0/2489070789 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f02240a3c30 msgr2=0x7f02240a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:37.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.855+0000 7f023fe00640 1 --2- 172.21.15.83:0/2489070789 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f02240a3c30 0x7f02240a60f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x56155fb884b0 tx=0x7f02280314a0 comp rx=0 tx=0).stop 2024-04-24T08:48:37.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.855+0000 7f023fe00640 1 -- 172.21.15.83:0/2489070789 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f024010a0f0 msgr2=0x7f02401a5cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:37.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.855+0000 7f023fe00640 1 --2- 172.21.15.83:0/2489070789 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f024010a0f0 0x7f02401a5cb0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f023800eaa0 tx=0x7f023800ef60 comp rx=0 tx=0).stop 2024-04-24T08:48:37.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.855+0000 7f023fe00640 1 -- 172.21.15.83:0/2489070789 shutdown_connections 2024-04-24T08:48:37.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.855+0000 7f023fe00640 1 --2- 172.21.15.83:0/2489070789 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f02240a3c30 0x7f02240a60f0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:37.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.855+0000 7f023fe00640 1 --2- 172.21.15.83:0/2489070789 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f024010a0f0 0x7f02401a5cb0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:37.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.855+0000 7f023fe00640 1 --2- 172.21.15.83:0/2489070789 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0240109730 0x7f024019ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:37.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.855+0000 7f023fe00640 1 --2- 172.21.15.83:0/2489070789 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0240108540 0x7f024019e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:37.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.855+0000 7f023fe00640 1 -- 172.21.15.83:0/2489070789 >> 172.21.15.83:0/2489070789 conn(0x7f02400774d0 msgr2=0x7f024010ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:48:37.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.855+0000 7f023fe00640 1 -- 172.21.15.83:0/2489070789 shutdown_connections 2024-04-24T08:48:37.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:37.855+0000 7f023fe00640 1 -- 172.21.15.83:0/2489070789 wait complete. 2024-04-24T08:48:37.874 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-24T08:48:37.874 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-24T08:48:37.875 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid bd3e039e-0215-11ef-bc93-c7b262605968 -- ceph fs ls 2024-04-24T08:48:38.315 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T08:48:38.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:38 smithi138 ceph-mon[31441]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-24T08:48:38.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:38 smithi138 ceph-mon[31441]: Cluster is now healthy 2024-04-24T08:48:38.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:38 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/1614942467' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T08:48:38.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:38 smithi083 ceph-mon[27907]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-24T08:48:38.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:38 smithi083 ceph-mon[27907]: Cluster is now healthy 2024-04-24T08:48:38.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:38 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/1614942467' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T08:48:38.621 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:38 smithi189 ceph-mon[31791]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-24T08:48:38.621 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:38 smithi189 ceph-mon[31791]: Cluster is now healthy 2024-04-24T08:48:38.621 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:38 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/1614942467' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T08:48:39.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:39 smithi138 ceph-mon[31441]: pgmap v378: 129 pgs: 6 creating+peering, 123 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s wr, 6 op/s 2024-04-24T08:48:39.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:39 smithi189 ceph-mon[31791]: pgmap v378: 129 pgs: 6 creating+peering, 123 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s wr, 6 op/s 2024-04-24T08:48:39.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:39 smithi083 ceph-mon[27907]: pgmap v378: 129 pgs: 6 creating+peering, 123 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s wr, 6 op/s 2024-04-24T08:48:40.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:40.621+0000 7faff5e00700 1 -- 172.21.15.83:0/1765061672 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faff0108b90 msgr2=0x7faff0108f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:40.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:40.621+0000 7faff5e00700 1 --2- 172.21.15.83:0/1765061672 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faff0108b90 0x7faff0108f70 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fafe0009990 tx=0x7fafe0009ca0 comp rx=0 tx=0).stop 2024-04-24T08:48:40.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:40.621+0000 7faff5e00700 1 -- 172.21.15.83:0/1765061672 shutdown_connections 2024-04-24T08:48:40.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:40.621+0000 7faff5e00700 1 --2- 172.21.15.83:0/1765061672 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faff0108b90 0x7faff0108f70 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:40.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:40.621+0000 7faff5e00700 1 --2- 172.21.15.83:0/1765061672 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faff0103500 0x7faff010f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:40.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:40.621+0000 7faff5e00700 1 --2- 172.21.15.83:0/1765061672 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faff0102b40 0x7faff0102fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:40.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:40.621+0000 7faff5e00700 1 -- 172.21.15.83:0/1765061672 >> 172.21.15.83:0/1765061672 conn(0x7faff0079c50 msgr2=0x7faff007a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:48:40.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:40.621+0000 7faff5e00700 1 -- 172.21.15.83:0/1765061672 shutdown_connections 2024-04-24T08:48:40.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:40.621+0000 7faff5e00700 1 -- 172.21.15.83:0/1765061672 wait complete. 2024-04-24T08:48:40.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:40.622+0000 7faff5e00700 1 Processor -- start 2024-04-24T08:48:40.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:40.623+0000 7faff5e00700 1 -- start start 2024-04-24T08:48:40.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:40.623+0000 7faff5e00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faff0102b40 0x7faff0074df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:40.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:40.623+0000 7faff5e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faff0103500 0x7faff0075330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:40.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:40.623+0000 7faff5e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faff0108b90 0x7faff0075870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:40.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:40.623+0000 7faff5e00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7faff0112d00 con 0x7faff0103500 2024-04-24T08:48:40.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:40.623+0000 7faff5e00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7faff0112b80 con 0x7faff0102b40 2024-04-24T08:48:40.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:40.623+0000 7faff5e00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7faff0112e80 con 0x7faff0108b90 2024-04-24T08:48:40.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:40.623+0000 7fafee000700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faff0103500 0x7faff0075330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:40.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:40.623+0000 7fafee000700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faff0103500 0x7faff0075330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33164/0 (socket says 172.21.15.83:33164) 2024-04-24T08:48:40.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:40.624+0000 7fafee000700 1 -- 172.21.15.83:0/1692238343 learned_addr learned my addr 172.21.15.83:0/1692238343 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:48:40.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:40.624+0000 7fafeea00700 1 --2- 172.21.15.83:0/1692238343 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faff0102b40 0x7faff0074df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:40.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:40.624+0000 7fafef400700 1 --2- 172.21.15.83:0/1692238343 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faff0108b90 0x7faff0075870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:40.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:40.624+0000 7fafee000700 1 -- 172.21.15.83:0/1692238343 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faff0108b90 msgr2=0x7faff0075870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:40.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:40.624+0000 7fafee000700 1 --2- 172.21.15.83:0/1692238343 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faff0108b90 0x7faff0075870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:40.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:40.624+0000 7fafee000700 1 -- 172.21.15.83:0/1692238343 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faff0102b40 msgr2=0x7faff0074df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:40.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:40.624+0000 7fafee000700 1 --2- 172.21.15.83:0/1692238343 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faff0102b40 0x7faff0074df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:40.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:40.624+0000 7fafee000700 1 -- 172.21.15.83:0/1692238343 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faff01a7760 con 0x7faff0103500 2024-04-24T08:48:40.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:40.624+0000 7fafeea00700 1 --2- 172.21.15.83:0/1692238343 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faff0102b40 0x7faff0074df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:48:40.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:40.624+0000 7fafef400700 1 --2- 172.21.15.83:0/1692238343 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faff0108b90 0x7faff0075870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:48:40.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:40.624+0000 7fafee000700 1 --2- 172.21.15.83:0/1692238343 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faff0103500 0x7faff0075330 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fafdc00d980 tx=0x7fafdc00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:48:40.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:40.625+0000 7faff4a00700 1 -- 172.21.15.83:0/1692238343 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafdc016070 con 0x7faff0103500 2024-04-24T08:48:40.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:40.625+0000 7faff5e00700 1 -- 172.21.15.83:0/1692238343 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faff01a7a50 con 0x7faff0103500 2024-04-24T08:48:40.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:40.625+0000 7faff5e00700 1 -- 172.21.15.83:0/1692238343 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faff01a7fe0 con 0x7faff0103500 2024-04-24T08:48:40.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:40.625+0000 7faff4a00700 1 -- 172.21.15.83:0/1692238343 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fafdc0044d0 con 0x7faff0103500 2024-04-24T08:48:40.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:40.625+0000 7faff4a00700 1 -- 172.21.15.83:0/1692238343 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafdc004cf0 con 0x7faff0103500 2024-04-24T08:48:40.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:40.626+0000 7faff5e00700 1 -- 172.21.15.83:0/1692238343 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafd0005330 con 0x7faff0103500 2024-04-24T08:48:40.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:40.628+0000 7faff4a00700 1 -- 172.21.15.83:0/1692238343 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fafdc00bb60 con 0x7faff0103500 2024-04-24T08:48:40.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:40.630+0000 7faff4a00700 1 --2- 172.21.15.83:0/1692238343 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fafd40a3cc0 0x7fafd40a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:40.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:40.631+0000 7faff4a00700 1 -- 172.21.15.83:0/1692238343 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fafdc0d0380 con 0x7faff0103500 2024-04-24T08:48:40.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:40.631+0000 7fafeea00700 1 --2- 172.21.15.83:0/1692238343 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fafd40a3cc0 0x7fafd40a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:40.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:40.631+0000 7fafeea00700 1 --2- 172.21.15.83:0/1692238343 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fafd40a3cc0 0x7fafd40a6180 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fafe000b5c0 tx=0x7fafe0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:48:40.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:40.635+0000 7faff4a00700 1 -- 172.21.15.83:0/1692238343 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fafdc01f020 con 0x7faff0103500 2024-04-24T08:48:40.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:40.933+0000 7faff5e00700 1 -- 172.21.15.83:0/1692238343 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fafd0005e30 con 0x7faff0103500 2024-04-24T08:48:40.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:40.934+0000 7faff4a00700 1 -- 172.21.15.83:0/1692238343 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+88 (secure 0 0 0) 0x7fafdc09cac0 con 0x7faff0103500 2024-04-24T08:48:40.935 INFO:teuthology.orchestra.run.smithi083.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-24T08:48:40.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:40.937+0000 7faff5e00700 1 -- 172.21.15.83:0/1692238343 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fafd40a3cc0 msgr2=0x7fafd40a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:40.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:40.937+0000 7faff5e00700 1 --2- 172.21.15.83:0/1692238343 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fafd40a3cc0 0x7fafd40a6180 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fafe000b5c0 tx=0x7fafe0005fb0 comp rx=0 tx=0).stop 2024-04-24T08:48:40.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:40.937+0000 7faff5e00700 1 -- 172.21.15.83:0/1692238343 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faff0103500 msgr2=0x7faff0075330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:40.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:40.937+0000 7faff5e00700 1 --2- 172.21.15.83:0/1692238343 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faff0103500 0x7faff0075330 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fafdc00d980 tx=0x7fafdc00dd40 comp rx=0 tx=0).stop 2024-04-24T08:48:40.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:40.937+0000 7faff5e00700 1 -- 172.21.15.83:0/1692238343 shutdown_connections 2024-04-24T08:48:40.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:40.937+0000 7faff5e00700 1 --2- 172.21.15.83:0/1692238343 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fafd40a3cc0 0x7fafd40a6180 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:40.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:40.937+0000 7faff5e00700 1 --2- 172.21.15.83:0/1692238343 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faff0108b90 0x7faff0075870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:40.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:40.937+0000 7faff5e00700 1 --2- 172.21.15.83:0/1692238343 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faff0102b40 0x7faff0074df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:40.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:40.937+0000 7faff5e00700 1 --2- 172.21.15.83:0/1692238343 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faff0103500 0x7faff0075330 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:40.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:40.937+0000 7faff5e00700 1 -- 172.21.15.83:0/1692238343 >> 172.21.15.83:0/1692238343 conn(0x7faff0079c50 msgr2=0x7faff01019e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:48:40.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:40.937+0000 7faff5e00700 1 -- 172.21.15.83:0/1692238343 shutdown_connections 2024-04-24T08:48:40.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:40.937+0000 7faff5e00700 1 -- 172.21.15.83:0/1692238343 wait complete. 2024-04-24T08:48:41.839 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-24T08:48:41.839 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-24T08:48:41.839 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi083.front.sepia.ceph.com 2024-04-24T08:48:41.839 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-04-24T08:48:41.839 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-24T08:48:41.839 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-24T08:48:41.840 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-24T08:48:41.840 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-24T08:48:41.840 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-04-24T08:48:41.840 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T08:48:41.840 DEBUG:teuthology.orchestra.run.smithi083:> ip addr 2024-04-24T08:48:41.893 INFO:teuthology.orchestra.run.smithi083.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-24T08:48:41.893 INFO:teuthology.orchestra.run.smithi083.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-24T08:48:41.893 INFO:teuthology.orchestra.run.smithi083.stdout: inet 127.0.0.1/8 scope host lo 2024-04-24T08:48:41.893 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-24T08:48:41.894 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 ::1/128 scope host 2024-04-24T08:48:41.894 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-24T08:48:41.894 INFO:teuthology.orchestra.run.smithi083.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T08:48:41.894 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1e brd ff:ff:ff:ff:ff:ff 2024-04-24T08:48:41.894 INFO:teuthology.orchestra.run.smithi083.stdout: altname enp7s0f0 2024-04-24T08:48:41.894 INFO:teuthology.orchestra.run.smithi083.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T08:48:41.894 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1f brd ff:ff:ff:ff:ff:ff 2024-04-24T08:48:41.894 INFO:teuthology.orchestra.run.smithi083.stdout: altname enp7s0f1 2024-04-24T08:48:41.894 INFO:teuthology.orchestra.run.smithi083.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T08:48:41.894 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:90 brd ff:ff:ff:ff:ff:ff 2024-04-24T08:48:41.894 INFO:teuthology.orchestra.run.smithi083.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-24T08:48:41.894 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:91 brd ff:ff:ff:ff:ff:ff 2024-04-24T08:48:41.894 INFO:teuthology.orchestra.run.smithi083.stdout: inet 172.21.15.83/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-24T08:48:41.894 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft 41978sec preferred_lft 41978sec 2024-04-24T08:48:41.895 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::ec4:7aff:fe8f:ce91/64 scope link 2024-04-24T08:48:41.895 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-24T08:48:41.896 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-24T08:48:41.896 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T08:48:41.896 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-24T08:48:41.896 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link add name ceph-brx type bridge 2024-04-24T08:48:41.896 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip addr flush dev ceph-brx 2024-04-24T08:48:41.896 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link set ceph-brx up 2024-04-24T08:48:41.896 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-24T08:48:41.896 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-24T08:48:42.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T08:48:42.027 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:41 smithi083 ceph-mon[27907]: pgmap v379: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 135 B/s wr, 3 op/s 2024-04-24T08:48:42.027 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:41 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/1692238343' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-24T08:48:42.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:41 smithi138 ceph-mon[31441]: pgmap v379: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 135 B/s wr, 3 op/s 2024-04-24T08:48:42.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:41 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/1692238343' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-24T08:48:42.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:41 smithi189 ceph-mon[31791]: pgmap v379: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 135 B/s wr, 3 op/s 2024-04-24T08:48:42.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:41 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/1692238343' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-24T08:48:42.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T08:48:42.391 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T08:48:42.391 DEBUG:teuthology.orchestra.run.smithi083:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-24T08:48:42.455 INFO:teuthology.orchestra.run.smithi083.stdout:1 2024-04-24T08:48:42.459 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T08:48:42.459 DEBUG:teuthology.orchestra.run.smithi083:> route 2024-04-24T08:48:42.513 INFO:teuthology.orchestra.run.smithi083.stdout:Kernel IP routing table 2024-04-24T08:48:42.513 INFO:teuthology.orchestra.run.smithi083.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-24T08:48:42.513 INFO:teuthology.orchestra.run.smithi083.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-24T08:48:42.513 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-24T08:48:42.513 INFO:teuthology.orchestra.run.smithi083.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-24T08:48:42.514 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T08:48:42.514 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-24T08:48:42.514 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-24T08:48:42.514 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-24T08:48:42.514 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-24T08:48:42.514 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-24T08:48:42.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T08:48:43.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T08:48:43.317 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T08:48:43.317 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list 2024-04-24T08:48:43.370 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T08:48:43.370 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list-id 2024-04-24T08:48:43.420 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T08:48:43.420 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-24T08:48:43.420 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-24T08:48:43.420 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-04-24T08:48:43.420 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-24T08:48:43.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:43 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T08:48:43.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T08:48:43.610 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-04-24T08:48:43.610 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T08:48:43.610 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-24T08:48:43.610 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-04-24T08:48:43.610 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-24T08:48:43.610 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-04-24T08:48:43.610 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-04-24T08:48:43.610 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-04-24T08:48:43.610 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-24T08:48:43.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:43 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T08:48:43.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T08:48:43.897 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T08:48:43.897 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-24T08:48:43.897 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link set brx.0 up 2024-04-24T08:48:43.898 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link set dev brx.0 master ceph-brx 2024-04-24T08:48:43.898 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-24T08:48:43.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:43 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T08:48:44.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:43 smithi083 ceph-mon[27907]: pgmap v380: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 119 B/s wr, 2 op/s 2024-04-24T08:48:44.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T08:48:44.094 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-04-24T08:48:44.095 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T08:48:44.095 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-04-24T08:48:44.143 INFO:teuthology.orchestra.run.smithi083.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-04-24T08:48:44.144 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T08:48:44.144 DEBUG:teuthology.orchestra.run.smithi083:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-04-24T08:48:44.191 DEBUG:teuthology.orchestra.run.smithi083:> sudo modprobe fuse 2024-04-24T08:48:44.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:43 smithi138 ceph-mon[31441]: pgmap v380: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 119 B/s wr, 2 op/s 2024-04-24T08:48:44.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:43 smithi189 ceph-mon[31791]: pgmap v380: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 119 B/s wr, 2 op/s 2024-04-24T08:48:44.275 DEBUG:teuthology.orchestra.run.smithi083:> cat /proc/self/mounts | awk '{print $2}' 2024-04-24T08:48:44.324 INFO:teuthology.orchestra.run.smithi083.stdout:/proc 2024-04-24T08:48:44.324 INFO:teuthology.orchestra.run.smithi083.stdout:/sys 2024-04-24T08:48:44.324 INFO:teuthology.orchestra.run.smithi083.stdout:/dev 2024-04-24T08:48:44.324 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/security 2024-04-24T08:48:44.324 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/shm 2024-04-24T08:48:44.324 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/pts 2024-04-24T08:48:44.324 INFO:teuthology.orchestra.run.smithi083.stdout:/run 2024-04-24T08:48:44.324 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/cgroup 2024-04-24T08:48:44.324 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/pstore 2024-04-24T08:48:44.324 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/bpf 2024-04-24T08:48:44.324 INFO:teuthology.orchestra.run.smithi083.stdout:/ 2024-04-24T08:48:44.325 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/selinux 2024-04-24T08:48:44.325 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T08:48:44.325 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/hugepages 2024-04-24T08:48:44.325 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/mqueue 2024-04-24T08:48:44.325 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/debug 2024-04-24T08:48:44.325 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/tracing 2024-04-24T08:48:44.325 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/fuse/connections 2024-04-24T08:48:44.325 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/config 2024-04-24T08:48:44.325 INFO:teuthology.orchestra.run.smithi083.stdout:/run/credentials/systemd-sysctl.service 2024-04-24T08:48:44.325 INFO:teuthology.orchestra.run.smithi083.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-24T08:48:44.325 INFO:teuthology.orchestra.run.smithi083.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-24T08:48:44.325 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/nfs/rpc_pipefs 2024-04-24T08:48:44.325 INFO:teuthology.orchestra.run.smithi083.stdout:/run/user/1000 2024-04-24T08:48:44.325 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph 2024-04-24T08:48:44.326 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/containers/storage/overlay 2024-04-24T08:48:44.326 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/containers/storage/overlay/49049192a6c660dfa299f98afedd81704406f70e80d6c99e3017cc3798469627/merged 2024-04-24T08:48:44.326 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/containers/storage/overlay/6a0c2bc8270c7dfd42477801d69ee13df03cea164cd5ecacf24b5fdc496904a4/merged 2024-04-24T08:48:44.326 INFO:teuthology.orchestra.run.smithi083.stdout:/run/user/0 2024-04-24T08:48:44.326 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T08:48:44.326 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/containers/storage/overlay/c92130de88cd4e5f8b30940a442652d0f71b4f10030df7685cf09f54ee334ca9/merged 2024-04-24T08:48:44.326 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/containers/storage/overlay/91a1ec8e5a8b56746793a65be525d445eb58a998d01ab8a7be11fefc80543fc1/merged 2024-04-24T08:48:44.326 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/containers/storage/overlay/59c7644b789b5c770c3accf58b941cb591889e0fcd9d79c001122751ebd34006/merged 2024-04-24T08:48:44.326 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/containers/storage/overlay/c39d5dae117cb52392575e92167a37143623a99da0c2d54cdd6ba44559ddd9b1/merged 2024-04-24T08:48:44.326 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/containers/storage/overlay/2aaa102c1272f2328e73534c89583df21950659a9917506415c62d4dc7cfd384/merged 2024-04-24T08:48:44.326 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/containers/storage/overlay/d997c279ce46cf62112b51cf6a3efea5734bbd6c9e363b3d45dcf7418695073a/merged 2024-04-24T08:48:44.326 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/containers/storage/overlay/01054164ae515e45a02a7aae9331e96774dfdccb9a503b4ee1bc6532a01ca330/merged 2024-04-24T08:48:44.326 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/containers/storage/overlay/f3868c721fea77ab7c659487533247b0fbc23b8ef9840112767a07b6744f8a12/merged 2024-04-24T08:48:44.326 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns 2024-04-24T08:48:44.327 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-24T08:48:44.327 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-24T08:48:44.327 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T08:48:44.328 DEBUG:teuthology.orchestra.run.smithi083:> ls /sys/fs/fuse/connections 2024-04-24T08:48:44.375 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-24T08:48:44.375 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-04-24T08:48:44.421 DEBUG:teuthology.orchestra.run.smithi083:> sudo modprobe fuse 2024-04-24T08:48:44.490 DEBUG:teuthology.orchestra.run.smithi083:> cat /proc/self/mounts | awk '{print $2}' 2024-04-24T08:48:44.530 INFO:teuthology.orchestra.run.smithi083.stdout:/proc 2024-04-24T08:48:44.530 INFO:teuthology.orchestra.run.smithi083.stdout:/sys 2024-04-24T08:48:44.530 INFO:teuthology.orchestra.run.smithi083.stdout:/dev 2024-04-24T08:48:44.530 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/security 2024-04-24T08:48:44.530 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/shm 2024-04-24T08:48:44.531 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/pts 2024-04-24T08:48:44.531 INFO:teuthology.orchestra.run.smithi083.stdout:/run 2024-04-24T08:48:44.531 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/cgroup 2024-04-24T08:48:44.531 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/pstore 2024-04-24T08:48:44.531 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/bpf 2024-04-24T08:48:44.531 INFO:teuthology.orchestra.run.smithi083.stdout:/ 2024-04-24T08:48:44.531 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/selinux 2024-04-24T08:48:44.531 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T08:48:44.531 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/hugepages 2024-04-24T08:48:44.531 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/mqueue 2024-04-24T08:48:44.531 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/debug 2024-04-24T08:48:44.531 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/tracing 2024-04-24T08:48:44.531 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/fuse/connections 2024-04-24T08:48:44.532 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/config 2024-04-24T08:48:44.532 INFO:teuthology.orchestra.run.smithi083.stdout:/run/credentials/systemd-sysctl.service 2024-04-24T08:48:44.532 INFO:teuthology.orchestra.run.smithi083.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-24T08:48:44.532 INFO:teuthology.orchestra.run.smithi083.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-24T08:48:44.532 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/nfs/rpc_pipefs 2024-04-24T08:48:44.532 INFO:teuthology.orchestra.run.smithi083.stdout:/run/user/1000 2024-04-24T08:48:44.532 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph 2024-04-24T08:48:44.533 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/containers/storage/overlay 2024-04-24T08:48:44.533 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/containers/storage/overlay/49049192a6c660dfa299f98afedd81704406f70e80d6c99e3017cc3798469627/merged 2024-04-24T08:48:44.533 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/containers/storage/overlay/6a0c2bc8270c7dfd42477801d69ee13df03cea164cd5ecacf24b5fdc496904a4/merged 2024-04-24T08:48:44.533 INFO:teuthology.orchestra.run.smithi083.stdout:/run/user/0 2024-04-24T08:48:44.533 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T08:48:44.533 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/containers/storage/overlay/c92130de88cd4e5f8b30940a442652d0f71b4f10030df7685cf09f54ee334ca9/merged 2024-04-24T08:48:44.533 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/containers/storage/overlay/91a1ec8e5a8b56746793a65be525d445eb58a998d01ab8a7be11fefc80543fc1/merged 2024-04-24T08:48:44.533 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/containers/storage/overlay/59c7644b789b5c770c3accf58b941cb591889e0fcd9d79c001122751ebd34006/merged 2024-04-24T08:48:44.533 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/containers/storage/overlay/c39d5dae117cb52392575e92167a37143623a99da0c2d54cdd6ba44559ddd9b1/merged 2024-04-24T08:48:44.533 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/containers/storage/overlay/2aaa102c1272f2328e73534c89583df21950659a9917506415c62d4dc7cfd384/merged 2024-04-24T08:48:44.533 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/containers/storage/overlay/d997c279ce46cf62112b51cf6a3efea5734bbd6c9e363b3d45dcf7418695073a/merged 2024-04-24T08:48:44.533 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/containers/storage/overlay/01054164ae515e45a02a7aae9331e96774dfdccb9a503b4ee1bc6532a01ca330/merged 2024-04-24T08:48:44.533 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/containers/storage/overlay/f3868c721fea77ab7c659487533247b0fbc23b8ef9840112767a07b6744f8a12/merged 2024-04-24T08:48:44.533 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns 2024-04-24T08:48:44.533 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-24T08:48:44.534 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-24T08:48:44.534 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T08:48:44.535 DEBUG:teuthology.orchestra.run.smithi083:> ls /sys/fs/fuse/connections 2024-04-24T08:48:44.639 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi083.stderr:2024-04-24T08:48:44.638+0000 7fe0d5c2d480 -1 init, newargv = 0x561c365424f0 newargc=15 2024-04-24T08:48:44.639 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi083.stderr:ceph-fuse[72178]: starting ceph client 2024-04-24T08:48:44.653 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi083.stderr:ceph-fuse[72178]: starting fuse 2024-04-24T08:48:45.721 DEBUG:teuthology.orchestra.run.smithi083:> sudo modprobe fuse 2024-04-24T08:48:45.812 DEBUG:teuthology.orchestra.run.smithi083:> cat /proc/self/mounts | awk '{print $2}' 2024-04-24T08:48:45.864 INFO:teuthology.orchestra.run.smithi083.stdout:/proc 2024-04-24T08:48:45.864 INFO:teuthology.orchestra.run.smithi083.stdout:/sys 2024-04-24T08:48:45.864 INFO:teuthology.orchestra.run.smithi083.stdout:/dev 2024-04-24T08:48:45.864 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/security 2024-04-24T08:48:45.864 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/shm 2024-04-24T08:48:45.864 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/pts 2024-04-24T08:48:45.864 INFO:teuthology.orchestra.run.smithi083.stdout:/run 2024-04-24T08:48:45.864 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/cgroup 2024-04-24T08:48:45.865 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/pstore 2024-04-24T08:48:45.865 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/bpf 2024-04-24T08:48:45.865 INFO:teuthology.orchestra.run.smithi083.stdout:/ 2024-04-24T08:48:45.865 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/selinux 2024-04-24T08:48:45.865 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T08:48:45.865 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/hugepages 2024-04-24T08:48:45.865 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/mqueue 2024-04-24T08:48:45.865 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/debug 2024-04-24T08:48:45.865 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/tracing 2024-04-24T08:48:45.865 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/fuse/connections 2024-04-24T08:48:45.865 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/config 2024-04-24T08:48:45.865 INFO:teuthology.orchestra.run.smithi083.stdout:/run/credentials/systemd-sysctl.service 2024-04-24T08:48:45.865 INFO:teuthology.orchestra.run.smithi083.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-24T08:48:45.866 INFO:teuthology.orchestra.run.smithi083.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-24T08:48:45.866 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/nfs/rpc_pipefs 2024-04-24T08:48:45.866 INFO:teuthology.orchestra.run.smithi083.stdout:/run/user/1000 2024-04-24T08:48:45.866 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph 2024-04-24T08:48:45.866 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/containers/storage/overlay 2024-04-24T08:48:45.866 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/containers/storage/overlay/49049192a6c660dfa299f98afedd81704406f70e80d6c99e3017cc3798469627/merged 2024-04-24T08:48:45.866 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/containers/storage/overlay/6a0c2bc8270c7dfd42477801d69ee13df03cea164cd5ecacf24b5fdc496904a4/merged 2024-04-24T08:48:45.866 INFO:teuthology.orchestra.run.smithi083.stdout:/run/user/0 2024-04-24T08:48:45.866 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T08:48:45.866 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/containers/storage/overlay/c92130de88cd4e5f8b30940a442652d0f71b4f10030df7685cf09f54ee334ca9/merged 2024-04-24T08:48:45.866 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/containers/storage/overlay/91a1ec8e5a8b56746793a65be525d445eb58a998d01ab8a7be11fefc80543fc1/merged 2024-04-24T08:48:45.866 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/containers/storage/overlay/59c7644b789b5c770c3accf58b941cb591889e0fcd9d79c001122751ebd34006/merged 2024-04-24T08:48:45.866 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/containers/storage/overlay/c39d5dae117cb52392575e92167a37143623a99da0c2d54cdd6ba44559ddd9b1/merged 2024-04-24T08:48:45.866 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/containers/storage/overlay/2aaa102c1272f2328e73534c89583df21950659a9917506415c62d4dc7cfd384/merged 2024-04-24T08:48:45.867 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/containers/storage/overlay/d997c279ce46cf62112b51cf6a3efea5734bbd6c9e363b3d45dcf7418695073a/merged 2024-04-24T08:48:45.867 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/containers/storage/overlay/01054164ae515e45a02a7aae9331e96774dfdccb9a503b4ee1bc6532a01ca330/merged 2024-04-24T08:48:45.867 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/containers/storage/overlay/f3868c721fea77ab7c659487533247b0fbc23b8ef9840112767a07b6744f8a12/merged 2024-04-24T08:48:45.867 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns 2024-04-24T08:48:45.867 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-24T08:48:45.867 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-24T08:48:45.867 INFO:teuthology.orchestra.run.smithi083.stdout:/home/ubuntu/cephtest/mnt.admin 2024-04-24T08:48:45.868 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T08:48:45.868 DEBUG:teuthology.orchestra.run.smithi083:> ls /sys/fs/fuse/connections 2024-04-24T08:48:45.913 INFO:teuthology.orchestra.run.smithi083.stdout:114 2024-04-24T08:48:45.913 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [114] 2024-04-24T08:48:45.913 DEBUG:teuthology.orchestra.run.smithi083:> sudo stdin-killer -- python3 -c ' 2024-04-24T08:48:45.914 DEBUG:teuthology.orchestra.run.smithi083:> import glob 2024-04-24T08:48:45.914 DEBUG:teuthology.orchestra.run.smithi083:> import re 2024-04-24T08:48:45.914 DEBUG:teuthology.orchestra.run.smithi083:> import os 2024-04-24T08:48:45.914 DEBUG:teuthology.orchestra.run.smithi083:> import subprocess 2024-04-24T08:48:45.914 DEBUG:teuthology.orchestra.run.smithi083:> 2024-04-24T08:48:45.914 DEBUG:teuthology.orchestra.run.smithi083:> def _find_admin_socket(client_name): 2024-04-24T08:48:45.914 DEBUG:teuthology.orchestra.run.smithi083:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-04-24T08:48:45.914 DEBUG:teuthology.orchestra.run.smithi083:> files = glob.glob(asok_path) 2024-04-24T08:48:45.914 DEBUG:teuthology.orchestra.run.smithi083:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-04-24T08:48:45.914 DEBUG:teuthology.orchestra.run.smithi083:> 2024-04-24T08:48:45.914 DEBUG:teuthology.orchestra.run.smithi083:> # Given a non-glob path, it better be there 2024-04-24T08:48:45.914 DEBUG:teuthology.orchestra.run.smithi083:> if "*" not in asok_path: 2024-04-24T08:48:45.914 DEBUG:teuthology.orchestra.run.smithi083:> assert(len(files) == 1) 2024-04-24T08:48:45.914 DEBUG:teuthology.orchestra.run.smithi083:> return files[0] 2024-04-24T08:48:45.914 DEBUG:teuthology.orchestra.run.smithi083:> 2024-04-24T08:48:45.915 DEBUG:teuthology.orchestra.run.smithi083:> for f in files: 2024-04-24T08:48:45.915 DEBUG:teuthology.orchestra.run.smithi083:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-24T08:48:45.915 DEBUG:teuthology.orchestra.run.smithi083:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-24T08:48:45.915 DEBUG:teuthology.orchestra.run.smithi083:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-24T08:48:45.915 DEBUG:teuthology.orchestra.run.smithi083:> contents = proc_f.read() 2024-04-24T08:48:45.915 DEBUG:teuthology.orchestra.run.smithi083:> if mountpoint in contents: 2024-04-24T08:48:45.915 DEBUG:teuthology.orchestra.run.smithi083:> return f 2024-04-24T08:48:45.915 DEBUG:teuthology.orchestra.run.smithi083:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-24T08:48:45.915 DEBUG:teuthology.orchestra.run.smithi083:> 2024-04-24T08:48:45.915 DEBUG:teuthology.orchestra.run.smithi083:> print(_find_admin_socket("client.admin")) 2024-04-24T08:48:45.915 DEBUG:teuthology.orchestra.run.smithi083:> ' 2024-04-24T08:48:46.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:45 smithi083 ceph-mon[27907]: pgmap v381: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 102 B/s wr, 1 op/s 2024-04-24T08:48:46.119 INFO:teuthology.orchestra.run.smithi083.stdout:/var/run/ceph/ceph-client.admin.72178.asok 2024-04-24T08:48:46.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T08:48:46.137 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.72178.asok 2024-04-24T08:48:46.137 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T08:48:46.137 DEBUG:teuthology.orchestra.run.smithi083:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.72178.asok status 2024-04-24T08:48:46.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:45 smithi138 ceph-mon[31441]: pgmap v381: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 102 B/s wr, 1 op/s 2024-04-24T08:48:46.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:45 smithi189 ceph-mon[31791]: pgmap v381: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 102 B/s wr, 1 op/s 2024-04-24T08:48:46.367 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T08:48:46.367 INFO:teuthology.orchestra.run.smithi083.stdout: "metadata": { 2024-04-24T08:48:46.367 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_sha1": "1f81fb397ae98da7563d451a78c61574c8f4e6e0", 2024-04-24T08:48:46.367 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_version": "ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev)", 2024-04-24T08:48:46.367 INFO:teuthology.orchestra.run.smithi083.stdout: "entity_id": "admin", 2024-04-24T08:48:46.367 INFO:teuthology.orchestra.run.smithi083.stdout: "hostname": "smithi083", 2024-04-24T08:48:46.368 INFO:teuthology.orchestra.run.smithi083.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-04-24T08:48:46.368 INFO:teuthology.orchestra.run.smithi083.stdout: "pid": "72178", 2024-04-24T08:48:46.368 INFO:teuthology.orchestra.run.smithi083.stdout: "root": "/" 2024-04-24T08:48:46.368 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T08:48:46.368 INFO:teuthology.orchestra.run.smithi083.stdout: "dentry_count": 0, 2024-04-24T08:48:46.368 INFO:teuthology.orchestra.run.smithi083.stdout: "dentry_pinned_count": 0, 2024-04-24T08:48:46.368 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 14817, 2024-04-24T08:48:46.368 INFO:teuthology.orchestra.run.smithi083.stdout: "inst": { 2024-04-24T08:48:46.368 INFO:teuthology.orchestra.run.smithi083.stdout: "name": { 2024-04-24T08:48:46.368 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "client", 2024-04-24T08:48:46.368 INFO:teuthology.orchestra.run.smithi083.stdout: "num": 14817 2024-04-24T08:48:46.368 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T08:48:46.368 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": { 2024-04-24T08:48:46.368 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-04-24T08:48:46.369 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "192.168.0.1:0", 2024-04-24T08:48:46.369 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1872671006 2024-04-24T08:48:46.369 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T08:48:46.369 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T08:48:46.369 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": { 2024-04-24T08:48:46.369 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-04-24T08:48:46.369 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "192.168.0.1:0", 2024-04-24T08:48:46.369 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1872671006 2024-04-24T08:48:46.369 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T08:48:46.369 INFO:teuthology.orchestra.run.smithi083.stdout: "inst_str": "client.14817 192.168.0.1:0/1872671006", 2024-04-24T08:48:46.369 INFO:teuthology.orchestra.run.smithi083.stdout: "addr_str": "192.168.0.1:0/1872671006", 2024-04-24T08:48:46.369 INFO:teuthology.orchestra.run.smithi083.stdout: "inode_count": 1, 2024-04-24T08:48:46.369 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_epoch": 17, 2024-04-24T08:48:46.369 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_epoch": 89, 2024-04-24T08:48:46.370 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_epoch_barrier": 0, 2024-04-24T08:48:46.370 INFO:teuthology.orchestra.run.smithi083.stdout: "blocklisted": false, 2024-04-24T08:48:46.370 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs" 2024-04-24T08:48:46.370 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T08:48:46.382 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T08:48:46.382 DEBUG:teuthology.orchestra.run.smithi083:> stat --file-system '--printf=%T 2024-04-24T08:48:46.382 DEBUG:teuthology.orchestra.run.smithi083:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-04-24T08:48:46.431 INFO:teuthology.orchestra.run.smithi083.stdout:fuseblk 2024-04-24T08:48:46.432 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-04-24T08:48:46.432 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T08:48:46.432 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-04-24T08:48:46.522 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-04-24T08:48:46.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:46 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-24T08:48:46.676 INFO:teuthology.orchestra.run.smithi083.stdout:# file: . 2024-04-24T08:48:46.676 INFO:teuthology.orchestra.run.smithi083.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-04-24T08:48:46.676 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:48:46.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T08:48:46.686 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-24T08:48:46.686 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T08:48:46.736 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T08:48:46.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:46.986+0000 7f4d50000640 1 -- 172.21.15.83:0/1670131770 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d48105e30 msgr2=0x7f4d4810c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:46.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:46.986+0000 7f4d50000640 1 --2- 172.21.15.83:0/1670131770 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d48105e30 0x7f4d4810c6c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f4d40009ca0 tx=0x7f4d4002a8c0 comp rx=0 tx=0).stop 2024-04-24T08:48:46.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:46.986+0000 7f4d50000640 1 -- 172.21.15.83:0/1670131770 shutdown_connections 2024-04-24T08:48:46.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:46.986+0000 7f4d50000640 1 --2- 172.21.15.83:0/1670131770 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d48105e30 0x7f4d4810c6c0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:46.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:46.986+0000 7f4d50000640 1 --2- 172.21.15.83:0/1670131770 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4d48105470 0x7f4d481058f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:46.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:46.986+0000 7f4d50000640 1 --2- 172.21.15.83:0/1670131770 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d48104280 0x7f4d481046a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:46.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:46.986+0000 7f4d50000640 1 -- 172.21.15.83:0/1670131770 >> 172.21.15.83:0/1670131770 conn(0x7f4d480ff7f0 msgr2=0x7f4d48101c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:48:46.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:46.986+0000 7f4d50000640 1 -- 172.21.15.83:0/1670131770 shutdown_connections 2024-04-24T08:48:46.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:46.986+0000 7f4d50000640 1 -- 172.21.15.83:0/1670131770 wait complete. 2024-04-24T08:48:46.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:46.987+0000 7f4d50000640 1 Processor -- start 2024-04-24T08:48:46.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:46.987+0000 7f4d50000640 1 -- start start 2024-04-24T08:48:46.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:46.987+0000 7f4d50000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4d48104280 0x7f4d4819a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:46.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:46.987+0000 7f4d50000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d48105470 0x7f4d4819a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:46.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:46.987+0000 7f4d50000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d48105e30 0x7f4d481a1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:46.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:46.988+0000 7f4d50000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4d4810fdc0 con 0x7f4d48105e30 2024-04-24T08:48:46.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:46.988+0000 7f4d50000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4d4810fc40 con 0x7f4d48105470 2024-04-24T08:48:46.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:46.988+0000 7f4d50000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4d4810ff40 con 0x7f4d48104280 2024-04-24T08:48:46.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:46.988+0000 7f4d4e000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d48105e30 0x7f4d481a1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:46.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:46.988+0000 7f4d4e000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d48105e30 0x7f4d481a1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33180/0 (socket says 172.21.15.83:33180) 2024-04-24T08:48:46.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:46.988+0000 7f4d4e000640 1 -- 172.21.15.83:0/3437126217 learned_addr learned my addr 172.21.15.83:0/3437126217 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:48:46.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:46.988+0000 7f4d4cc00640 1 --2- 172.21.15.83:0/3437126217 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d48105470 0x7f4d4819a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:46.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:46.988+0000 7f4d4e000640 1 -- 172.21.15.83:0/3437126217 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4d48104280 msgr2=0x7f4d4819a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:46.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:46.988+0000 7f4d4e000640 1 --2- 172.21.15.83:0/3437126217 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4d48104280 0x7f4d4819a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:46.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:46.988+0000 7f4d4e000640 1 -- 172.21.15.83:0/3437126217 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d48105470 msgr2=0x7f4d4819a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:46.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:46.988+0000 7f4d4e000640 1 --2- 172.21.15.83:0/3437126217 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d48105470 0x7f4d4819a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:46.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:46.988+0000 7f4d4e000640 1 -- 172.21.15.83:0/3437126217 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d481a1fc0 con 0x7f4d48105e30 2024-04-24T08:48:46.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:46.988+0000 7f4d4cc00640 1 --2- 172.21.15.83:0/3437126217 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d48105470 0x7f4d4819a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:48:46.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:46.989+0000 7f4d4e000640 1 --2- 172.21.15.83:0/3437126217 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d48105e30 0x7f4d481a1950 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f4d40009dc0 tx=0x7f4d40003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:48:46.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:46.989+0000 7f4d46000640 1 -- 172.21.15.83:0/3437126217 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d40032ec0 con 0x7f4d48105e30 2024-04-24T08:48:46.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:46.989+0000 7f4d46000640 1 -- 172.21.15.83:0/3437126217 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4d400044f0 con 0x7f4d48105e30 2024-04-24T08:48:46.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:46.989+0000 7f4d46000640 1 -- 172.21.15.83:0/3437126217 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d40030d40 con 0x7f4d48105e30 2024-04-24T08:48:46.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:46.989+0000 7f4d50000640 1 -- 172.21.15.83:0/3437126217 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d481a2250 con 0x7f4d48105e30 2024-04-24T08:48:46.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:46.989+0000 7f4d50000640 1 -- 172.21.15.83:0/3437126217 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d481a2750 con 0x7f4d48105e30 2024-04-24T08:48:46.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:46.990+0000 7f4d50000640 1 -- 172.21.15.83:0/3437126217 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d4810b5a0 con 0x7f4d48105e30 2024-04-24T08:48:46.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:46.992+0000 7f4d46000640 1 -- 172.21.15.83:0/3437126217 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f4d40004040 con 0x7f4d48105e30 2024-04-24T08:48:46.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:46.994+0000 7f4d46000640 1 --2- 172.21.15.83:0/3437126217 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f4d300a3dd0 0x7f4d300a6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:46.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:46.995+0000 7f4d46000640 1 -- 172.21.15.83:0/3437126217 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f4d400f1ce0 con 0x7f4d48105e30 2024-04-24T08:48:46.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:46.995+0000 7f4d4d600640 1 --2- 172.21.15.83:0/3437126217 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f4d300a3dd0 0x7f4d300a6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:46.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:46.995+0000 7f4d4d600640 1 --2- 172.21.15.83:0/3437126217 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f4d300a3dd0 0x7f4d300a6290 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c005fd0 tx=0x7f4d3c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:48:46.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:46.999+0000 7f4d46000640 1 -- 172.21.15.83:0/3437126217 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4d400be420 con 0x7f4d48105e30 2024-04-24T08:48:47.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:47.205+0000 7f4d50000640 1 -- 172.21.15.83:0/3437126217 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4d48062d00 con 0x7f4d48105e30 2024-04-24T08:48:47.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:47.207+0000 7f4d46000640 1 -- 172.21.15.83:0/3437126217 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24127 (secure 0 0 0) 0x7f4d400bdd20 con 0x7f4d48105e30 2024-04-24T08:48:47.208 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:48:47.208 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":89,"fsid":"bd3e039e-0215-11ef-bc93-c7b262605968","created":"2024-04-24T08:37:32.209944+0000","modified":"2024-04-24T08:48:34.752923+0000","last_up_change":"2024-04-24T08:47:17.966041+0000","last_in_change":"2024-04-24T08:46:53.835936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T08:41:56.834372+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T08:48:26.978311+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T08:48:28.352731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T08:48:32.275167+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2cd77f3a-19dd-4c6e-a199-1dcbd5faa679","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2132675555},{"type":"v1","addr":"172.21.15.83:6803","nonce":2132675555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2132675555},{"type":"v1","addr":"172.21.15.83:6805","nonce":2132675555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2132675555},{"type":"v1","addr":"172.21.15.83:6809","nonce":2132675555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2132675555},{"type":"v1","addr":"172.21.15.83:6807","nonce":2132675555}]},"public_addr":"172.21.15.83:6803/2132675555","cluster_addr":"172.21.15.83:6805/2132675555","heartbeat_back_addr":"172.21.15.83:6809/2132675555","heartbeat_front_addr":"172.21.15.83:6807/2132675555","state":["exists","up"]},{"osd":1,"uuid":"aba7f5df-2a4c-4c23-b502-d193605a5b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":3589158880},{"type":"v1","addr":"172.21.15.138:6801","nonce":3589158880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":3589158880},{"type":"v1","addr":"172.21.15.138:6803","nonce":3589158880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":3589158880},{"type":"v1","addr":"172.21.15.138:6807","nonce":3589158880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":3589158880},{"type":"v1","addr":"172.21.15.138:6805","nonce":3589158880}]},"public_addr":"172.21.15.138:6801/3589158880","cluster_addr":"172.21.15.138:6803/3589158880","heartbeat_back_addr":"172.21.15.138:6807/3589158880","heartbeat_front_addr":"172.21.15.138:6805/3589158880","state":["exists","up"]},{"osd":2,"uuid":"071bc167-2333-4420-bb58-bffbc10d7641","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":3169072688},{"type":"v1","addr":"172.21.15.189:6801","nonce":3169072688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":3169072688},{"type":"v1","addr":"172.21.15.189:6803","nonce":3169072688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":3169072688},{"type":"v1","addr":"172.21.15.189:6807","nonce":3169072688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":3169072688},{"type":"v1","addr":"172.21.15.189:6805","nonce":3169072688}]},"public_addr":"172.21.15.189:6801/3169072688","cluster_addr":"172.21.15.189:6803/3169072688","heartbeat_back_addr":"172.21.15.189:6807/3169072688","heartbeat_front_addr":"172.21.15.189:6805/3169072688","state":["exists","up"]},{"osd":3,"uuid":"f5624d78-4455-4e07-8c1a-f47c94cddb23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3131092450},{"type":"v1","addr":"172.21.15.83:6811","nonce":3131092450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3131092450},{"type":"v1","addr":"172.21.15.83:6813","nonce":3131092450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3131092450},{"type":"v1","addr":"172.21.15.83:6817","nonce":3131092450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3131092450},{"type":"v1","addr":"172.21.15.83:6815","nonce":3131092450}]},"public_addr":"172.21.15.83:6811/3131092450","cluster_addr":"172.21.15.83:6813/3131092450","heartbeat_back_addr":"172.21.15.83:6817/3131092450","heartbeat_front_addr":"172.21.15.83:6815/3131092450","state":["exists","up"]},{"osd":4,"uuid":"7b50cd14-9385-4535-a7b0-1c1da93c8548","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":823775360},{"type":"v1","addr":"172.21.15.138:6809","nonce":823775360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":823775360},{"type":"v1","addr":"172.21.15.138:6811","nonce":823775360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":823775360},{"type":"v1","addr":"172.21.15.138:6815","nonce":823775360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":823775360},{"type":"v1","addr":"172.21.15.138:6813","nonce":823775360}]},"public_addr":"172.21.15.138:6809/823775360","cluster_addr":"172.21.15.138:6811/823775360","heartbeat_back_addr":"172.21.15.138:6815/823775360","heartbeat_front_addr":"172.21.15.138:6813/823775360","state":["exists","up"]},{"osd":5,"uuid":"9c647797-35bc-449f-be84-af97d2c69c2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":1399817431},{"type":"v1","addr":"172.21.15.189:6809","nonce":1399817431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":1399817431},{"type":"v1","addr":"172.21.15.189:6811","nonce":1399817431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":1399817431},{"type":"v1","addr":"172.21.15.189:6815","nonce":1399817431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":1399817431},{"type":"v1","addr":"172.21.15.189:6813","nonce":1399817431}]},"public_addr":"172.21.15.189:6809/1399817431","cluster_addr":"172.21.15.189:6811/1399817431","heartbeat_back_addr":"172.21.15.189:6815/1399817431","heartbeat_front_addr":"172.21.15.189:6813/1399817431","state":["exists","up"]},{"osd":6,"uuid":"8f056584-16c8-4454-b445-3f780f26c196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":1611358483},{"type":"v1","addr":"172.21.15.83:6819","nonce":1611358483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1611358483},{"type":"v1","addr":"172.21.15.83:6821","nonce":1611358483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1611358483},{"type":"v1","addr":"172.21.15.83:6825","nonce":1611358483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":1611358483},{"type":"v1","addr":"172.21.15.83:6823","nonce":1611358483}]},"public_addr":"172.21.15.83:6819/1611358483","cluster_addr":"172.21.15.83:6821/1611358483","heartbeat_back_addr":"172.21.15.83:6825/1611358483","heartbeat_front_addr":"172.21.15.83:6823/1611358483","state":["exists","up"]},{"osd":7,"uuid":"7603efe3-da77-4b7f-9d03-29c4f293b6d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":3557856706},{"type":"v1","addr":"172.21.15.138:6817","nonce":3557856706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":3557856706},{"type":"v1","addr":"172.21.15.138:6819","nonce":3557856706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":3557856706},{"type":"v1","addr":"172.21.15.138:6823","nonce":3557856706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":3557856706},{"type":"v1","addr":"172.21.15.138:6821","nonce":3557856706}]},"public_addr":"172.21.15.138:6817/3557856706","cluster_addr":"172.21.15.138:6819/3557856706","heartbeat_back_addr":"172.21.15.138:6823/3557856706","heartbeat_front_addr":"172.21.15.138:6821/3557856706","state":["exists","up"]},{"osd":8,"uuid":"27d7d1b8-d6a0-417e-b8aa-596325aa1a77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":207018554},{"type":"v1","addr":"172.21.15.189:6817","nonce":207018554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":207018554},{"type":"v1","addr":"172.21.15.189:6819","nonce":207018554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":207018554},{"type":"v1","addr":"172.21.15.189:6823","nonce":207018554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":207018554},{"type":"v1","addr":"172.21.15.189:6821","nonce":207018554}]},"public_addr":"172.21.15.189:6817/207018554","cluster_addr":"172.21.15.189:6819/207018554","heartbeat_back_addr":"172.21.15.189:6823/207018554","heartbeat_front_addr":"172.21.15.189:6821/207018554","state":["exists","up"]},{"osd":9,"uuid":"baf19b98-881b-4d3a-9f72-8aa2dd73cad6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":519636751},{"type":"v1","addr":"172.21.15.83:6827","nonce":519636751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":519636751},{"type":"v1","addr":"172.21.15.83:6829","nonce":519636751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":519636751},{"type":"v1","addr":"172.21.15.83:6833","nonce":519636751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":519636751},{"type":"v1","addr":"172.21.15.83:6831","nonce":519636751}]},"public_addr":"172.21.15.83:6827/519636751","cluster_addr":"172.21.15.83:6829/519636751","heartbeat_back_addr":"172.21.15.83:6833/519636751","heartbeat_front_addr":"172.21.15.83:6831/519636751","state":["exists","up"]},{"osd":10,"uuid":"eb346717-5c07-47c8-95be-0e68fe449fdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1130872098},{"type":"v1","addr":"172.21.15.138:6825","nonce":1130872098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1130872098},{"type":"v1","addr":"172.21.15.138:6827","nonce":1130872098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1130872098},{"type":"v1","addr":"172.21.15.138:6831","nonce":1130872098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1130872098},{"type":"v1","addr":"172.21.15.138:6829","nonce":1130872098}]},"public_addr":"172.21.15.138:6825/1130872098","cluster_addr":"172.21.15.138:6827/1130872098","heartbeat_back_addr":"172.21.15.138:6831/1130872098","heartbeat_front_addr":"172.21.15.138:6829/1130872098","state":["exists","up"]},{"osd":11,"uuid":"8e37568e-981c-45b8-b2af-2f1b73898542","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":2998782690},{"type":"v1","addr":"172.21.15.189:6825","nonce":2998782690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":2998782690},{"type":"v1","addr":"172.21.15.189:6827","nonce":2998782690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":2998782690},{"type":"v1","addr":"172.21.15.189:6831","nonce":2998782690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":2998782690},{"type":"v1","addr":"172.21.15.189:6829","nonce":2998782690}]},"public_addr":"172.21.15.189:6825/2998782690","cluster_addr":"172.21.15.189:6827/2998782690","heartbeat_back_addr":"172.21.15.189:6831/2998782690","heartbeat_front_addr":"172.21.15.189:6829/2998782690","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:40:47.980103+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:41:21.970795+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:41:55.160426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:42:30.400475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:43:05.705663+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:43:40.112312+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:44:16.781289+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:44:52.151187+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:45:28.055944+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:46:04.969301+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:46:40.918306+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:47:16.886874+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:6801/820049883":"2024-04-25T08:38:40.661315+0000","172.21.15.83:0/4272874138":"2024-04-25T08:38:40.661315+0000","172.21.15.83:0/4212994335":"2024-04-25T08:38:40.661315+0000","172.21.15.83:6801/1996371091":"2024-04-25T08:38:05.800856+0000","172.21.15.83:0/1820663683":"2024-04-25T08:38:05.800856+0000","172.21.15.83:0/3802897262":"2024-04-25T08:38:05.800856+0000","172.21.15.83:6800/1996371091":"2024-04-25T08:38:05.800856+0000","172.21.15.83:6800/820049883":"2024-04-25T08:38:40.661315+0000","172.21.15.83:0/1366125049":"2024-04-25T08:38:05.800856+0000","172.21.15.83:0/340452868":"2024-04-25T08:38:40.661315+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T08:48:47.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:47.211+0000 7f4d50000640 1 -- 172.21.15.83:0/3437126217 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f4d300a3dd0 msgr2=0x7f4d300a6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:47.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:47.211+0000 7f4d50000640 1 --2- 172.21.15.83:0/3437126217 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f4d300a3dd0 0x7f4d300a6290 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c005fd0 tx=0x7f4d3c00b410 comp rx=0 tx=0).stop 2024-04-24T08:48:47.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:47.211+0000 7f4d50000640 1 -- 172.21.15.83:0/3437126217 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d48105e30 msgr2=0x7f4d481a1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:47.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:47.211+0000 7f4d50000640 1 --2- 172.21.15.83:0/3437126217 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d48105e30 0x7f4d481a1950 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f4d40009dc0 tx=0x7f4d40003ba0 comp rx=0 tx=0).stop 2024-04-24T08:48:47.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:47.211+0000 7f4d50000640 1 -- 172.21.15.83:0/3437126217 shutdown_connections 2024-04-24T08:48:47.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:47.211+0000 7f4d50000640 1 --2- 172.21.15.83:0/3437126217 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f4d300a3dd0 0x7f4d300a6290 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:47.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:47.211+0000 7f4d50000640 1 --2- 172.21.15.83:0/3437126217 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d48105e30 0x7f4d481a1950 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:47.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:47.211+0000 7f4d50000640 1 --2- 172.21.15.83:0/3437126217 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d48105470 0x7f4d4819a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:47.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:47.211+0000 7f4d50000640 1 --2- 172.21.15.83:0/3437126217 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4d48104280 0x7f4d4819a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:47.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:47.211+0000 7f4d50000640 1 -- 172.21.15.83:0/3437126217 >> 172.21.15.83:0/3437126217 conn(0x7f4d480ff7f0 msgr2=0x7f4d48108770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:48:47.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:47.211+0000 7f4d50000640 1 -- 172.21.15.83:0/3437126217 shutdown_connections 2024-04-24T08:48:47.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:47.211+0000 7f4d50000640 1 -- 172.21.15.83:0/3437126217 wait complete. 2024-04-24T08:48:47.232 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T08:41:56.834372+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T08:48:26.978311+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T08:48:28.352731+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-24T08:48:32.275167+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-24T08:48:47.232 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T08:48:47.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:47.430+0000 7f7c1e400640 1 -- 172.21.15.83:0/560747882 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c18107dc0 msgr2=0x7f7c1810a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:47.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:47.430+0000 7f7c1e400640 1 --2- 172.21.15.83:0/560747882 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c18107dc0 0x7f7c1810a1b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f7c10009ca0 tx=0x7f7c1002a8c0 comp rx=0 tx=0).stop 2024-04-24T08:48:47.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:47.430+0000 7f7c1e400640 1 -- 172.21.15.83:0/560747882 shutdown_connections 2024-04-24T08:48:47.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:47.430+0000 7f7c1e400640 1 --2- 172.21.15.83:0/560747882 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c18107dc0 0x7f7c1810a1b0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:47.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:47.430+0000 7f7c1e400640 1 --2- 172.21.15.83:0/560747882 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7c18105490 0x7f7c18107880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:47.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:47.430+0000 7f7c1e400640 1 --2- 172.21.15.83:0/560747882 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c18102b60 0x7f7c18104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:47.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:47.430+0000 7f7c1e400640 1 -- 172.21.15.83:0/560747882 >> 172.21.15.83:0/560747882 conn(0x7f7c180fc4d0 msgr2=0x7f7c180fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:48:47.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:47.430+0000 7f7c1e400640 1 -- 172.21.15.83:0/560747882 shutdown_connections 2024-04-24T08:48:47.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:47.430+0000 7f7c1e400640 1 -- 172.21.15.83:0/560747882 wait complete. 2024-04-24T08:48:47.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:47.431+0000 7f7c1e400640 1 Processor -- start 2024-04-24T08:48:47.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:47.431+0000 7f7c1e400640 1 -- start start 2024-04-24T08:48:47.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:47.431+0000 7f7c1e400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7c18102b60 0x7f7c1819a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:47.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:47.431+0000 7f7c1e400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c18105490 0x7f7c1819a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:47.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:47.431+0000 7f7c1e400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c18107dc0 0x7f7c181a1a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:47.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:47.432+0000 7f7c1e400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7c1810dbf0 con 0x7f7c18107dc0 2024-04-24T08:48:47.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:47.432+0000 7f7c1e400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7c1810da70 con 0x7f7c18105490 2024-04-24T08:48:47.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:47.432+0000 7f7c1e400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7c1810dd70 con 0x7f7c18102b60 2024-04-24T08:48:47.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:47.432+0000 7f7c17e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c18107dc0 0x7f7c181a1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:47.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:47.432+0000 7f7c17e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c18107dc0 0x7f7c181a1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33206/0 (socket says 172.21.15.83:33206) 2024-04-24T08:48:47.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:47.432+0000 7f7c17e00640 1 -- 172.21.15.83:0/1106298845 learned_addr learned my addr 172.21.15.83:0/1106298845 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:48:47.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:47.432+0000 7f7c17400640 1 --2- 172.21.15.83:0/1106298845 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7c18102b60 0x7f7c1819a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:47.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:47.432+0000 7f7c0ea00640 1 --2- 172.21.15.83:0/1106298845 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c18105490 0x7f7c1819a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:47.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:47.432+0000 7f7c17e00640 1 -- 172.21.15.83:0/1106298845 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7c18102b60 msgr2=0x7f7c1819a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:47.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:47.432+0000 7f7c17e00640 1 --2- 172.21.15.83:0/1106298845 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7c18102b60 0x7f7c1819a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:47.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:47.432+0000 7f7c17e00640 1 -- 172.21.15.83:0/1106298845 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c18105490 msgr2=0x7f7c1819a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:47.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:47.432+0000 7f7c17e00640 1 --2- 172.21.15.83:0/1106298845 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c18105490 0x7f7c1819a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:47.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:47.432+0000 7f7c17e00640 1 -- 172.21.15.83:0/1106298845 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c181a2080 con 0x7f7c18107dc0 2024-04-24T08:48:47.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:47.433+0000 7f7c17e00640 1 --2- 172.21.15.83:0/1106298845 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c18107dc0 0x7f7c181a1a10 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f7c10031040 tx=0x7f7c10003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:48:47.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:47.433+0000 7f7c14c00640 1 -- 172.21.15.83:0/1106298845 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c10032ec0 con 0x7f7c18107dc0 2024-04-24T08:48:47.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:47.433+0000 7f7c14c00640 1 -- 172.21.15.83:0/1106298845 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7c100044f0 con 0x7f7c18107dc0 2024-04-24T08:48:47.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:47.433+0000 7f7c14c00640 1 -- 172.21.15.83:0/1106298845 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c10030d10 con 0x7f7c18107dc0 2024-04-24T08:48:47.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:47.433+0000 7f7c1e400640 1 -- 172.21.15.83:0/1106298845 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c181a2310 con 0x7f7c18107dc0 2024-04-24T08:48:47.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:47.433+0000 7f7c1e400640 1 -- 172.21.15.83:0/1106298845 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c181a28b0 con 0x7f7c18107dc0 2024-04-24T08:48:47.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:47.434+0000 7f7c1e400640 1 -- 172.21.15.83:0/1106298845 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c181098b0 con 0x7f7c18107dc0 2024-04-24T08:48:47.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:47.436+0000 7f7c14c00640 1 -- 172.21.15.83:0/1106298845 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f7c10004040 con 0x7f7c18107dc0 2024-04-24T08:48:47.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:47.439+0000 7f7c14c00640 1 --2- 172.21.15.83:0/1106298845 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f7bfc0a3dd0 0x7f7bfc0a6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:47.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:47.439+0000 7f7c14c00640 1 -- 172.21.15.83:0/1106298845 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f7c100f1ef0 con 0x7f7c18107dc0 2024-04-24T08:48:47.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:47.439+0000 7f7c17400640 1 --2- 172.21.15.83:0/1106298845 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f7bfc0a3dd0 0x7f7bfc0a6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:47.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:47.440+0000 7f7c17400640 1 --2- 172.21.15.83:0/1106298845 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f7bfc0a3dd0 0x7f7bfc0a6290 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7c08005fd0 tx=0x7f7c08005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:48:47.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:47.443+0000 7f7c14c00640 1 -- 172.21.15.83:0/1106298845 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7c100be630 con 0x7f7c18107dc0 2024-04-24T08:48:47.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:47.646+0000 7f7c1e400640 1 -- 172.21.15.83:0/1106298845 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7c18062c00 con 0x7f7c18107dc0 2024-04-24T08:48:47.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:47.646+0000 7f7c14c00640 1 -- 172.21.15.83:0/1106298845 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) ==== 93+0+10 (secure 0 0 0) 0x7f7c100bdf30 con 0x7f7c18107dc0 2024-04-24T08:48:47.647 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-24T08:48:47.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:47.649+0000 7f7c1e400640 1 -- 172.21.15.83:0/1106298845 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f7bfc0a3dd0 msgr2=0x7f7bfc0a6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:47.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:47.649+0000 7f7c1e400640 1 --2- 172.21.15.83:0/1106298845 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f7bfc0a3dd0 0x7f7bfc0a6290 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7c08005fd0 tx=0x7f7c08005850 comp rx=0 tx=0).stop 2024-04-24T08:48:47.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:47.649+0000 7f7c1e400640 1 -- 172.21.15.83:0/1106298845 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c18107dc0 msgr2=0x7f7c181a1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:47.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:47.649+0000 7f7c1e400640 1 --2- 172.21.15.83:0/1106298845 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c18107dc0 0x7f7c181a1a10 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f7c10031040 tx=0x7f7c10003ba0 comp rx=0 tx=0).stop 2024-04-24T08:48:47.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:47.649+0000 7f7c1e400640 1 -- 172.21.15.83:0/1106298845 shutdown_connections 2024-04-24T08:48:47.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:47.649+0000 7f7c1e400640 1 --2- 172.21.15.83:0/1106298845 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f7bfc0a3dd0 0x7f7bfc0a6290 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:47.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:47.649+0000 7f7c1e400640 1 --2- 172.21.15.83:0/1106298845 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c18107dc0 0x7f7c181a1a10 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:47.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:47.649+0000 7f7c1e400640 1 --2- 172.21.15.83:0/1106298845 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c18105490 0x7f7c1819a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:47.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:47.649+0000 7f7c1e400640 1 --2- 172.21.15.83:0/1106298845 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7c18102b60 0x7f7c1819a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:47.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:47.649+0000 7f7c1e400640 1 -- 172.21.15.83:0/1106298845 >> 172.21.15.83:0/1106298845 conn(0x7f7c180fc4d0 msgr2=0x7f7c18102270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:48:47.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:47.649+0000 7f7c1e400640 1 -- 172.21.15.83:0/1106298845 shutdown_connections 2024-04-24T08:48:47.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:47.649+0000 7f7c1e400640 1 -- 172.21.15.83:0/1106298845 wait complete. 2024-04-24T08:48:47.668 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T08:48:47.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:47.874+0000 7ff292600640 1 -- 172.21.15.83:0/1721057504 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff28c109850 msgr2=0x7ff28c109cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:47.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:47.874+0000 7ff292600640 1 --2- 172.21.15.83:0/1721057504 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff28c109850 0x7ff28c109cd0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7ff278009900 tx=0x7ff27802a410 comp rx=0 tx=0).stop 2024-04-24T08:48:47.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:47.875+0000 7ff292600640 1 -- 172.21.15.83:0/1721057504 shutdown_connections 2024-04-24T08:48:47.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:47.875+0000 7ff292600640 1 --2- 172.21.15.83:0/1721057504 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff28c10a210 0x7ff28c110aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:47.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:47.875+0000 7ff292600640 1 --2- 172.21.15.83:0/1721057504 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff28c109850 0x7ff28c109cd0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:47.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:47.875+0000 7ff292600640 1 --2- 172.21.15.83:0/1721057504 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff28c108660 0x7ff28c108a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:47.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:47.875+0000 7ff292600640 1 -- 172.21.15.83:0/1721057504 >> 172.21.15.83:0/1721057504 conn(0x7ff28c077600 msgr2=0x7ff28c079a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:48:47.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:47.875+0000 7ff292600640 1 -- 172.21.15.83:0/1721057504 shutdown_connections 2024-04-24T08:48:47.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:47.875+0000 7ff292600640 1 -- 172.21.15.83:0/1721057504 wait complete. 2024-04-24T08:48:47.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:47.876+0000 7ff292600640 1 Processor -- start 2024-04-24T08:48:47.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:47.876+0000 7ff292600640 1 -- start start 2024-04-24T08:48:47.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:47.876+0000 7ff292600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff28c108660 0x7ff28c19e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:47.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:47.876+0000 7ff292600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff28c109850 0x7ff28c19ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:47.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:47.876+0000 7ff292600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff28c10a210 0x7ff28c1a5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:47.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:47.876+0000 7ff292600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff28c114240 con 0x7ff28c109850 2024-04-24T08:48:47.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:47.876+0000 7ff292600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff28c1140c0 con 0x7ff28c108660 2024-04-24T08:48:47.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:47.876+0000 7ff292600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff28c1143c0 con 0x7ff28c10a210 2024-04-24T08:48:47.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:47.876+0000 7ff28aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff28c109850 0x7ff28c19ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:47.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:47.876+0000 7ff28aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff28c109850 0x7ff28c19ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33232/0 (socket says 172.21.15.83:33232) 2024-04-24T08:48:47.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:47.876+0000 7ff28aa00640 1 -- 172.21.15.83:0/628069724 learned_addr learned my addr 172.21.15.83:0/628069724 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:48:47.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:47.877+0000 7ff28b400640 1 --2- 172.21.15.83:0/628069724 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff28c108660 0x7ff28c19e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:47.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:47.877+0000 7ff28be00640 1 --2- 172.21.15.83:0/628069724 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff28c10a210 0x7ff28c1a5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:47.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:47.877+0000 7ff28aa00640 1 -- 172.21.15.83:0/628069724 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff28c10a210 msgr2=0x7ff28c1a5ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:47.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:47.877+0000 7ff28aa00640 1 --2- 172.21.15.83:0/628069724 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff28c10a210 0x7ff28c1a5ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:47.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:47.877+0000 7ff28aa00640 1 -- 172.21.15.83:0/628069724 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff28c108660 msgr2=0x7ff28c19e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:47.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:47.877+0000 7ff28aa00640 1 --2- 172.21.15.83:0/628069724 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff28c108660 0x7ff28c19e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:47.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:47.877+0000 7ff28aa00640 1 -- 172.21.15.83:0/628069724 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff28c1a6560 con 0x7ff28c109850 2024-04-24T08:48:47.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:47.877+0000 7ff28aa00640 1 --2- 172.21.15.83:0/628069724 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff28c109850 0x7ff28c19ee80 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff2780314e0 tx=0x7ff278003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:48:47.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:47.878+0000 7ff283e00640 1 -- 172.21.15.83:0/628069724 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff27803f070 con 0x7ff28c109850 2024-04-24T08:48:47.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:47.878+0000 7ff283e00640 1 -- 172.21.15.83:0/628069724 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff278031e40 con 0x7ff28c109850 2024-04-24T08:48:47.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:47.878+0000 7ff292600640 1 -- 172.21.15.83:0/628069724 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff28c1a67f0 con 0x7ff28c109850 2024-04-24T08:48:47.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:47.878+0000 7ff283e00640 1 -- 172.21.15.83:0/628069724 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff278035030 con 0x7ff28c109850 2024-04-24T08:48:47.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:47.878+0000 7ff292600640 1 -- 172.21.15.83:0/628069724 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff28c1a6d60 con 0x7ff28c109850 2024-04-24T08:48:47.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:47.879+0000 7ff292600640 1 -- 172.21.15.83:0/628069724 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff28c10f9a0 con 0x7ff28c109850 2024-04-24T08:48:47.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:47.881+0000 7ff283e00640 1 -- 172.21.15.83:0/628069724 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ff278042070 con 0x7ff28c109850 2024-04-24T08:48:47.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:47.883+0000 7ff283e00640 1 --2- 172.21.15.83:0/628069724 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7ff2700a3bb0 0x7ff2700a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:47.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:47.883+0000 7ff283e00640 1 -- 172.21.15.83:0/628069724 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7ff2780f1190 con 0x7ff28c109850 2024-04-24T08:48:47.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:47.884+0000 7ff28b400640 1 --2- 172.21.15.83:0/628069724 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7ff2700a3bb0 0x7ff2700a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:47.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:47.884+0000 7ff28b400640 1 --2- 172.21.15.83:0/628069724 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7ff2700a3bb0 0x7ff2700a6070 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff27c005db0 tx=0x7ff27c005d40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:48:47.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:47.888+0000 7ff283e00640 1 -- 172.21.15.83:0/628069724 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff2780bd8d0 con 0x7ff28c109850 2024-04-24T08:48:47.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:47 smithi138 ceph-mon[31441]: pgmap v382: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:48:47.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:47 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/3437126217' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T08:48:47.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:47 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/1106298845' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T08:48:48.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:47 smithi083 ceph-mon[27907]: pgmap v382: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:48:48.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:47 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/3437126217' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T08:48:48.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:47 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/1106298845' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T08:48:48.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.090+0000 7ff292600640 1 -- 172.21.15.83:0/628069724 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff28c062c00 con 0x7ff28c109850 2024-04-24T08:48:48.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.091+0000 7ff283e00640 1 -- 172.21.15.83:0/628069724 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) ==== 104+0+11 (secure 0 0 0) 0x7ff2780bd1d0 con 0x7ff28c109850 2024-04-24T08:48:48.092 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-24T08:48:48.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.093+0000 7ff292600640 1 -- 172.21.15.83:0/628069724 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7ff2700a3bb0 msgr2=0x7ff2700a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:48.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.093+0000 7ff292600640 1 --2- 172.21.15.83:0/628069724 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7ff2700a3bb0 0x7ff2700a6070 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff27c005db0 tx=0x7ff27c005d40 comp rx=0 tx=0).stop 2024-04-24T08:48:48.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.093+0000 7ff292600640 1 -- 172.21.15.83:0/628069724 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff28c109850 msgr2=0x7ff28c19ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:48.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.093+0000 7ff292600640 1 --2- 172.21.15.83:0/628069724 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff28c109850 0x7ff28c19ee80 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff2780314e0 tx=0x7ff278003d50 comp rx=0 tx=0).stop 2024-04-24T08:48:48.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.094+0000 7ff292600640 1 -- 172.21.15.83:0/628069724 shutdown_connections 2024-04-24T08:48:48.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.094+0000 7ff292600640 1 --2- 172.21.15.83:0/628069724 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff28c10a210 0x7ff28c1a5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:48.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.094+0000 7ff292600640 1 --2- 172.21.15.83:0/628069724 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7ff2700a3bb0 0x7ff2700a6070 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:48.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.094+0000 7ff292600640 1 --2- 172.21.15.83:0/628069724 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff28c109850 0x7ff28c19ee80 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:48.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.094+0000 7ff292600640 1 --2- 172.21.15.83:0/628069724 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff28c108660 0x7ff28c19e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:48.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.094+0000 7ff292600640 1 -- 172.21.15.83:0/628069724 >> 172.21.15.83:0/628069724 conn(0x7ff28c077600 msgr2=0x7ff28c10cb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:48:48.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.094+0000 7ff292600640 1 -- 172.21.15.83:0/628069724 shutdown_connections 2024-04-24T08:48:48.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.094+0000 7ff292600640 1 -- 172.21.15.83:0/628069724 wait complete. 2024-04-24T08:48:48.114 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T08:48:48.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:47 smithi189 ceph-mon[31791]: pgmap v382: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:48:48.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:47 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/3437126217' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T08:48:48.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:47 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/1106298845' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T08:48:48.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.313+0000 7f6209200640 1 -- 172.21.15.83:0/144921963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6204105330 msgr2=0x7f62041057b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:48.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.313+0000 7f6209200640 1 --2- 172.21.15.83:0/144921963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6204105330 0x7f62041057b0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f61f0009900 tx=0x7f61f002a410 comp rx=0 tx=0).stop 2024-04-24T08:48:48.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.313+0000 7f6209200640 1 -- 172.21.15.83:0/144921963 shutdown_connections 2024-04-24T08:48:48.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.313+0000 7f6209200640 1 --2- 172.21.15.83:0/144921963 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6204105cf0 0x7f620410c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:48.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.313+0000 7f6209200640 1 --2- 172.21.15.83:0/144921963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6204105330 0x7f62041057b0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:48.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.313+0000 7f6209200640 1 --2- 172.21.15.83:0/144921963 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6204104140 0x7f6204104560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:48.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.313+0000 7f6209200640 1 -- 172.21.15.83:0/144921963 >> 172.21.15.83:0/144921963 conn(0x7f62040ff6b0 msgr2=0x7f6204101af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:48:48.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.314+0000 7f6209200640 1 -- 172.21.15.83:0/144921963 shutdown_connections 2024-04-24T08:48:48.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.314+0000 7f6209200640 1 -- 172.21.15.83:0/144921963 wait complete. 2024-04-24T08:48:48.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.315+0000 7f6209200640 1 Processor -- start 2024-04-24T08:48:48.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.315+0000 7f6209200640 1 -- start start 2024-04-24T08:48:48.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.315+0000 7f6209200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6204104140 0x7f620419a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:48.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.315+0000 7f6209200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6204105330 0x7f620419a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:48.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.315+0000 7f6209200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6204105cf0 0x7f62041a18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:48.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.315+0000 7f6209200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f620410fcd0 con 0x7f6204105cf0 2024-04-24T08:48:48.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.315+0000 7f6209200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f620410fb50 con 0x7f6204105330 2024-04-24T08:48:48.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.315+0000 7f6209200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f620410fe50 con 0x7f6204104140 2024-04-24T08:48:48.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.316+0000 7f6203e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6204105cf0 0x7f62041a18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:48.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.316+0000 7f6203e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6204105cf0 0x7f62041a18f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33252/0 (socket says 172.21.15.83:33252) 2024-04-24T08:48:48.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.316+0000 7f6203e00640 1 -- 172.21.15.83:0/26026976 learned_addr learned my addr 172.21.15.83:0/26026976 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:48:48.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.316+0000 7f6202a00640 1 --2- 172.21.15.83:0/26026976 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6204105330 0x7f620419a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:48.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.316+0000 7f6203400640 1 --2- 172.21.15.83:0/26026976 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6204104140 0x7f620419a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:48.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.316+0000 7f6203e00640 1 -- 172.21.15.83:0/26026976 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6204104140 msgr2=0x7f620419a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:48.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.316+0000 7f6203e00640 1 --2- 172.21.15.83:0/26026976 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6204104140 0x7f620419a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:48.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.316+0000 7f6203e00640 1 -- 172.21.15.83:0/26026976 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6204105330 msgr2=0x7f620419a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:48.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.316+0000 7f6203e00640 1 --2- 172.21.15.83:0/26026976 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6204105330 0x7f620419a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:48.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.316+0000 7f6203e00640 1 -- 172.21.15.83:0/26026976 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62041a1f60 con 0x7f6204105cf0 2024-04-24T08:48:48.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.316+0000 7f6203400640 1 --2- 172.21.15.83:0/26026976 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6204104140 0x7f620419a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:48:48.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.317+0000 7f6203e00640 1 --2- 172.21.15.83:0/26026976 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6204105cf0 0x7f62041a18f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f61fc00efd0 tx=0x7f61fc00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:48:48.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.317+0000 7f61fbe00640 1 -- 172.21.15.83:0/26026976 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61fc003b70 con 0x7f6204105cf0 2024-04-24T08:48:48.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.317+0000 7f61fbe00640 1 -- 172.21.15.83:0/26026976 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f61fc003d10 con 0x7f6204105cf0 2024-04-24T08:48:48.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.317+0000 7f6209200640 1 -- 172.21.15.83:0/26026976 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62041a2250 con 0x7f6204105cf0 2024-04-24T08:48:48.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.317+0000 7f61fbe00640 1 -- 172.21.15.83:0/26026976 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61fc010980 con 0x7f6204105cf0 2024-04-24T08:48:48.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.317+0000 7f6209200640 1 -- 172.21.15.83:0/26026976 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62041a2770 con 0x7f6204105cf0 2024-04-24T08:48:48.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.318+0000 7f6209200640 1 -- 172.21.15.83:0/26026976 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f620410b4e0 con 0x7f6204105cf0 2024-04-24T08:48:48.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.320+0000 7f61fbe00640 1 -- 172.21.15.83:0/26026976 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f61fc004020 con 0x7f6204105cf0 2024-04-24T08:48:48.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.322+0000 7f61fbe00640 1 --2- 172.21.15.83:0/26026976 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f61e80a3c30 0x7f61e80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:48.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.322+0000 7f61fbe00640 1 -- 172.21.15.83:0/26026976 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f61fc0d0ad0 con 0x7f6204105cf0 2024-04-24T08:48:48.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.322+0000 7f6203400640 1 --2- 172.21.15.83:0/26026976 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f61e80a3c30 0x7f61e80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:48.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.323+0000 7f6203400640 1 --2- 172.21.15.83:0/26026976 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f61e80a3c30 0x7f61e80a60f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f61f40058c0 tx=0x7f61f4005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:48:48.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.326+0000 7f61fbe00640 1 -- 172.21.15.83:0/26026976 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f61fc09d210 con 0x7f6204105cf0 2024-04-24T08:48:48.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.525+0000 7f6209200640 1 -- 172.21.15.83:0/26026976 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f6204062c00 con 0x7f6204105cf0 2024-04-24T08:48:48.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.526+0000 7f61fbe00640 1 -- 172.21.15.83:0/26026976 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) ==== 100+0+11 (secure 0 0 0) 0x7f61fc09cb10 con 0x7f6204105cf0 2024-04-24T08:48:48.527 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2024-04-24T08:48:48.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.529+0000 7f6209200640 1 -- 172.21.15.83:0/26026976 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f61e80a3c30 msgr2=0x7f61e80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:48.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.529+0000 7f6209200640 1 --2- 172.21.15.83:0/26026976 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f61e80a3c30 0x7f61e80a60f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f61f40058c0 tx=0x7f61f4005850 comp rx=0 tx=0).stop 2024-04-24T08:48:48.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.529+0000 7f6209200640 1 -- 172.21.15.83:0/26026976 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6204105cf0 msgr2=0x7f62041a18f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:48.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.529+0000 7f6209200640 1 --2- 172.21.15.83:0/26026976 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6204105cf0 0x7f62041a18f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f61fc00efd0 tx=0x7f61fc00c6a0 comp rx=0 tx=0).stop 2024-04-24T08:48:48.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.529+0000 7f6209200640 1 -- 172.21.15.83:0/26026976 shutdown_connections 2024-04-24T08:48:48.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.529+0000 7f6209200640 1 --2- 172.21.15.83:0/26026976 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f61e80a3c30 0x7f61e80a60f0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:48.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.529+0000 7f6209200640 1 --2- 172.21.15.83:0/26026976 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6204105cf0 0x7f62041a18f0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:48.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.529+0000 7f6209200640 1 --2- 172.21.15.83:0/26026976 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6204105330 0x7f620419a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:48.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.529+0000 7f6209200640 1 --2- 172.21.15.83:0/26026976 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6204104140 0x7f620419a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:48.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.529+0000 7f6209200640 1 -- 172.21.15.83:0/26026976 >> 172.21.15.83:0/26026976 conn(0x7f62040ff6b0 msgr2=0x7f62041086b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:48:48.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.530+0000 7f6209200640 1 -- 172.21.15.83:0/26026976 shutdown_connections 2024-04-24T08:48:48.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.530+0000 7f6209200640 1 -- 172.21.15.83:0/26026976 wait complete. 2024-04-24T08:48:48.549 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-24T08:48:48.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.755+0000 7f48f8c00640 1 -- 172.21.15.83:0/2441733905 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f48f41052c0 msgr2=0x7f48f41076b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:48.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.755+0000 7f48f8c00640 1 --2- 172.21.15.83:0/2441733905 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f48f41052c0 0x7f48f41076b0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f48dc009900 tx=0x7f48dc02a410 comp rx=0 tx=0).stop 2024-04-24T08:48:48.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.755+0000 7f48f8c00640 1 -- 172.21.15.83:0/2441733905 shutdown_connections 2024-04-24T08:48:48.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.755+0000 7f48f8c00640 1 --2- 172.21.15.83:0/2441733905 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f48f4107bf0 0x7f48f410a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:48.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.755+0000 7f48f8c00640 1 --2- 172.21.15.83:0/2441733905 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f48f41052c0 0x7f48f41076b0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:48.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.755+0000 7f48f8c00640 1 --2- 172.21.15.83:0/2441733905 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f48f4102990 0x7f48f4104d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:48.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.755+0000 7f48f8c00640 1 -- 172.21.15.83:0/2441733905 >> 172.21.15.83:0/2441733905 conn(0x7f48f40fc3a0 msgr2=0x7f48f40fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:48:48.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.755+0000 7f48f8c00640 1 -- 172.21.15.83:0/2441733905 shutdown_connections 2024-04-24T08:48:48.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.755+0000 7f48f8c00640 1 -- 172.21.15.83:0/2441733905 wait complete. 2024-04-24T08:48:48.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.756+0000 7f48f8c00640 1 Processor -- start 2024-04-24T08:48:48.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.756+0000 7f48f8c00640 1 -- start start 2024-04-24T08:48:48.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.757+0000 7f48f8c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f48f4102990 0x7f48f419a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:48.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.757+0000 7f48f8c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f48f41052c0 0x7f48f419a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:48.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.757+0000 7f48f8c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f48f4107bf0 0x7f48f41a1920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:48.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.757+0000 7f48f8c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f48f410da20 con 0x7f48f4102990 2024-04-24T08:48:48.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.757+0000 7f48f8c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f48f410d8a0 con 0x7f48f41052c0 2024-04-24T08:48:48.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.757+0000 7f48f8c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f48f410dba0 con 0x7f48f4107bf0 2024-04-24T08:48:48.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.757+0000 7f48f3400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f48f4102990 0x7f48f419a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:48.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.757+0000 7f48f3400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f48f4102990 0x7f48f419a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33272/0 (socket says 172.21.15.83:33272) 2024-04-24T08:48:48.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.757+0000 7f48f3400640 1 -- 172.21.15.83:0/996811087 learned_addr learned my addr 172.21.15.83:0/996811087 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:48:48.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.757+0000 7f48eaa00640 1 --2- 172.21.15.83:0/996811087 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f48f41052c0 0x7f48f419a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:48.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.757+0000 7f48f3e00640 1 --2- 172.21.15.83:0/996811087 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f48f4107bf0 0x7f48f41a1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:48.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.757+0000 7f48f3400640 1 -- 172.21.15.83:0/996811087 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f48f4107bf0 msgr2=0x7f48f41a1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:48.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.757+0000 7f48f3400640 1 --2- 172.21.15.83:0/996811087 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f48f4107bf0 0x7f48f41a1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:48.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.757+0000 7f48f3400640 1 -- 172.21.15.83:0/996811087 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f48f41052c0 msgr2=0x7f48f419a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:48.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.757+0000 7f48f3400640 1 --2- 172.21.15.83:0/996811087 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f48f41052c0 0x7f48f419a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:48.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.757+0000 7f48f3400640 1 -- 172.21.15.83:0/996811087 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48f41a1f90 con 0x7f48f4102990 2024-04-24T08:48:48.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.757+0000 7f48f3e00640 1 --2- 172.21.15.83:0/996811087 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f48f4107bf0 0x7f48f41a1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:48:48.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.758+0000 7f48f3400640 1 --2- 172.21.15.83:0/996811087 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f48f4102990 0x7f48f419a370 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f48e400eab0 tx=0x7f48e400ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:48:48.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.758+0000 7f48f0c00640 1 -- 172.21.15.83:0/996811087 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48e400ce40 con 0x7f48f4102990 2024-04-24T08:48:48.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.758+0000 7f48f0c00640 1 -- 172.21.15.83:0/996811087 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f48e4011920 con 0x7f48f4102990 2024-04-24T08:48:48.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.758+0000 7f48f8c00640 1 -- 172.21.15.83:0/996811087 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48f41a2280 con 0x7f48f4102990 2024-04-24T08:48:48.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.758+0000 7f48f0c00640 1 -- 172.21.15.83:0/996811087 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48e4010a00 con 0x7f48f4102990 2024-04-24T08:48:48.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.758+0000 7f48f8c00640 1 -- 172.21.15.83:0/996811087 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48f41a27a0 con 0x7f48f4102990 2024-04-24T08:48:48.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.759+0000 7f48f8c00640 1 -- 172.21.15.83:0/996811087 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48f4109740 con 0x7f48f4102990 2024-04-24T08:48:48.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.761+0000 7f48f0c00640 1 -- 172.21.15.83:0/996811087 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f48e4011ad0 con 0x7f48f4102990 2024-04-24T08:48:48.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.763+0000 7f48f0c00640 1 --2- 172.21.15.83:0/996811087 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f48d80a3bb0 0x7f48d80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:48.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.763+0000 7f48f0c00640 1 -- 172.21.15.83:0/996811087 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f48e40d0d90 con 0x7f48f4102990 2024-04-24T08:48:48.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.763+0000 7f48eaa00640 1 --2- 172.21.15.83:0/996811087 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f48d80a3bb0 0x7f48d80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:48.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.764+0000 7f48eaa00640 1 --2- 172.21.15.83:0/996811087 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f48d80a3bb0 0x7f48d80a6070 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f48dc000c00 tx=0x7f48dc0314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:48:48.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.767+0000 7f48f0c00640 1 -- 172.21.15.83:0/996811087 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f48e4016030 con 0x7f48f4102990 2024-04-24T08:48:48.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.972+0000 7f48f8c00640 1 -- 172.21.15.83:0/996811087 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f48f4062c00 con 0x7f48f4102990 2024-04-24T08:48:48.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.973+0000 7f48f0c00640 1 -- 172.21.15.83:0/996811087 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v89) ==== 103+0+11 (secure 0 0 0) 0x7f48e409d4d0 con 0x7f48f4102990 2024-04-24T08:48:48.973 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-24T08:48:48.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.975+0000 7f48f8c00640 1 -- 172.21.15.83:0/996811087 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f48d80a3bb0 msgr2=0x7f48d80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:48.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.975+0000 7f48f8c00640 1 --2- 172.21.15.83:0/996811087 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f48d80a3bb0 0x7f48d80a6070 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f48dc000c00 tx=0x7f48dc0314a0 comp rx=0 tx=0).stop 2024-04-24T08:48:48.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.975+0000 7f48f8c00640 1 -- 172.21.15.83:0/996811087 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f48f4102990 msgr2=0x7f48f419a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:48.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.975+0000 7f48f8c00640 1 --2- 172.21.15.83:0/996811087 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f48f4102990 0x7f48f419a370 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f48e400eab0 tx=0x7f48e400ef70 comp rx=0 tx=0).stop 2024-04-24T08:48:48.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.976+0000 7f48f8c00640 1 -- 172.21.15.83:0/996811087 shutdown_connections 2024-04-24T08:48:48.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.976+0000 7f48f8c00640 1 --2- 172.21.15.83:0/996811087 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f48f4107bf0 0x7f48f41a1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:48.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.976+0000 7f48f8c00640 1 --2- 172.21.15.83:0/996811087 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f48f41052c0 0x7f48f419a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:48.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.976+0000 7f48f8c00640 1 --2- 172.21.15.83:0/996811087 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f48d80a3bb0 0x7f48d80a6070 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:48.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.976+0000 7f48f8c00640 1 --2- 172.21.15.83:0/996811087 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f48f4102990 0x7f48f419a370 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:48.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.976+0000 7f48f8c00640 1 -- 172.21.15.83:0/996811087 >> 172.21.15.83:0/996811087 conn(0x7f48f40fc3a0 msgr2=0x7f48f4102200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:48:48.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.976+0000 7f48f8c00640 1 -- 172.21.15.83:0/996811087 shutdown_connections 2024-04-24T08:48:48.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:48.976+0000 7f48f8c00640 1 -- 172.21.15.83:0/996811087 wait complete. 2024-04-24T08:48:48.996 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-24T08:48:49.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:49.189+0000 7f9ce2400640 1 -- 172.21.15.83:0/3772011134 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9cdc104280 msgr2=0x7f9cdc1046a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:49.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:49.189+0000 7f9ce2400640 1 --2- 172.21.15.83:0/3772011134 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9cdc104280 0x7f9cdc1046a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f9ccc009ca0 tx=0x7f9ccc02a8c0 comp rx=0 tx=0).stop 2024-04-24T08:48:49.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:49.189+0000 7f9ce2400640 1 -- 172.21.15.83:0/3772011134 shutdown_connections 2024-04-24T08:48:49.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:49.189+0000 7f9ce2400640 1 --2- 172.21.15.83:0/3772011134 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9cdc105e30 0x7f9cdc10c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:49.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:49.189+0000 7f9ce2400640 1 --2- 172.21.15.83:0/3772011134 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9cdc105470 0x7f9cdc1058f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:49.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:49.189+0000 7f9ce2400640 1 --2- 172.21.15.83:0/3772011134 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9cdc104280 0x7f9cdc1046a0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:49.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:49.189+0000 7f9ce2400640 1 -- 172.21.15.83:0/3772011134 >> 172.21.15.83:0/3772011134 conn(0x7f9cdc0ff7f0 msgr2=0x7f9cdc101c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:48:49.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:49.189+0000 7f9ce2400640 1 -- 172.21.15.83:0/3772011134 shutdown_connections 2024-04-24T08:48:49.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:49.189+0000 7f9ce2400640 1 -- 172.21.15.83:0/3772011134 wait complete. 2024-04-24T08:48:49.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:49.191+0000 7f9ce2400640 1 Processor -- start 2024-04-24T08:48:49.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:49.191+0000 7f9ce2400640 1 -- start start 2024-04-24T08:48:49.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:49.191+0000 7f9ce2400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9cdc104280 0x7f9cdc19a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:49.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:49.191+0000 7f9ce2400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9cdc105470 0x7f9cdc19aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:49.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:49.191+0000 7f9ce2400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9cdc105e30 0x7f9cdc1a1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:49.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:49.191+0000 7f9ce2400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9cdc10fe10 con 0x7f9cdc105470 2024-04-24T08:48:49.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:49.191+0000 7f9ce2400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9cdc10fc90 con 0x7f9cdc104280 2024-04-24T08:48:49.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:49.191+0000 7f9ce2400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9cdc10ff90 con 0x7f9cdc105e30 2024-04-24T08:48:49.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:49.191+0000 7f9cd3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9cdc105470 0x7f9cdc19aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:49.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:49.191+0000 7f9cd3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9cdc105470 0x7f9cdc19aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35328/0 (socket says 172.21.15.83:35328) 2024-04-24T08:48:49.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:49.191+0000 7f9cd3e00640 1 -- 172.21.15.83:0/2018818020 learned_addr learned my addr 172.21.15.83:0/2018818020 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:48:49.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:49.192+0000 7f9cdb400640 1 --2- 172.21.15.83:0/2018818020 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9cdc104280 0x7f9cdc19a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:49.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:49.192+0000 7f9cdbe00640 1 --2- 172.21.15.83:0/2018818020 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9cdc105e30 0x7f9cdc1a1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:49.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:49.192+0000 7f9cd3e00640 1 -- 172.21.15.83:0/2018818020 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9cdc105e30 msgr2=0x7f9cdc1a1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:49.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:49.192+0000 7f9cd3e00640 1 --2- 172.21.15.83:0/2018818020 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9cdc105e30 0x7f9cdc1a1ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:49.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:49.192+0000 7f9cd3e00640 1 -- 172.21.15.83:0/2018818020 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9cdc104280 msgr2=0x7f9cdc19a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:49.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:49.192+0000 7f9cd3e00640 1 --2- 172.21.15.83:0/2018818020 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9cdc104280 0x7f9cdc19a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:49.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:49.192+0000 7f9cd3e00640 1 -- 172.21.15.83:0/2018818020 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cdc1a2140 con 0x7f9cdc105470 2024-04-24T08:48:49.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:49.192+0000 7f9cd3e00640 1 --2- 172.21.15.83:0/2018818020 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9cdc105470 0x7f9cdc19aa60 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f9cc400c970 tx=0x7f9cc400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:48:49.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:49.192+0000 7f9cd8c00640 1 -- 172.21.15.83:0/2018818020 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc4007bb0 con 0x7f9cdc105470 2024-04-24T08:48:49.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:49.192+0000 7f9cd8c00640 1 -- 172.21.15.83:0/2018818020 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9cc4007d50 con 0x7f9cdc105470 2024-04-24T08:48:49.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:49.192+0000 7f9ce2400640 1 -- 172.21.15.83:0/2018818020 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cdc1a2430 con 0x7f9cdc105470 2024-04-24T08:48:49.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:49.192+0000 7f9cd8c00640 1 -- 172.21.15.83:0/2018818020 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc400f450 con 0x7f9cdc105470 2024-04-24T08:48:49.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:49.193+0000 7f9ce2400640 1 -- 172.21.15.83:0/2018818020 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cdc1a2930 con 0x7f9cdc105470 2024-04-24T08:48:49.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:49.194+0000 7f9ce2400640 1 -- 172.21.15.83:0/2018818020 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cdc10b6c0 con 0x7f9cdc105470 2024-04-24T08:48:49.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:49.196+0000 7f9cd8c00640 1 -- 172.21.15.83:0/2018818020 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9cc4004f20 con 0x7f9cdc105470 2024-04-24T08:48:49.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:49.197+0000 7f9cd8c00640 1 --2- 172.21.15.83:0/2018818020 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f9cc00a3bb0 0x7f9cc00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:49.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:49.198+0000 7f9cd8c00640 1 -- 172.21.15.83:0/2018818020 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f9cc40d04d0 con 0x7f9cdc105470 2024-04-24T08:48:49.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:49.198+0000 7f9cdb400640 1 --2- 172.21.15.83:0/2018818020 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f9cc00a3bb0 0x7f9cc00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:49.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:49.198+0000 7f9cdb400640 1 --2- 172.21.15.83:0/2018818020 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f9cc00a3bb0 0x7f9cc00a6070 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9ccc006010 tx=0x7f9ccc0093f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:48:49.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:49.203+0000 7f9cd8c00640 1 -- 172.21.15.83:0/2018818020 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9cc4014030 con 0x7f9cdc105470 2024-04-24T08:48:49.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:48 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/628069724' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T08:48:49.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:48 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/26026976' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T08:48:49.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:48 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/628069724' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T08:48:49.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:48 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/26026976' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T08:48:49.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:48 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/628069724' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T08:48:49.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:48 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/26026976' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T08:48:49.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:49.414+0000 7f9ce2400640 1 -- 172.21.15.83:0/2018818020 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f9cdc02d620 con 0x7f9cdc105470 2024-04-24T08:48:49.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:49.415+0000 7f9cd8c00640 1 -- 172.21.15.83:0/2018818020 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v89) ==== 81+0+589 (secure 0 0 0) 0x7f9cc409cc10 con 0x7f9cdc105470 2024-04-24T08:48:49.415 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:6801/820049883 2024-04-25T08:38:40.661315+0000 2024-04-24T08:48:49.415 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:0/4272874138 2024-04-25T08:38:40.661315+0000 2024-04-24T08:48:49.416 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:0/4212994335 2024-04-25T08:38:40.661315+0000 2024-04-24T08:48:49.416 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:6801/1996371091 2024-04-25T08:38:05.800856+0000 2024-04-24T08:48:49.416 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:0/1820663683 2024-04-25T08:38:05.800856+0000 2024-04-24T08:48:49.416 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:0/3802897262 2024-04-25T08:38:05.800856+0000 2024-04-24T08:48:49.416 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:6800/1996371091 2024-04-25T08:38:05.800856+0000 2024-04-24T08:48:49.416 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:6800/820049883 2024-04-25T08:38:40.661315+0000 2024-04-24T08:48:49.416 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:0/1366125049 2024-04-25T08:38:05.800856+0000 2024-04-24T08:48:49.416 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:0/340452868 2024-04-25T08:38:40.661315+0000 2024-04-24T08:48:49.416 INFO:teuthology.orchestra.run.smithi083.stderr:listed 10 entries 2024-04-24T08:48:49.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:49.417+0000 7f9ce2400640 1 -- 172.21.15.83:0/2018818020 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f9cc00a3bb0 msgr2=0x7f9cc00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:49.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:49.417+0000 7f9ce2400640 1 --2- 172.21.15.83:0/2018818020 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f9cc00a3bb0 0x7f9cc00a6070 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9ccc006010 tx=0x7f9ccc0093f0 comp rx=0 tx=0).stop 2024-04-24T08:48:49.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:49.417+0000 7f9ce2400640 1 -- 172.21.15.83:0/2018818020 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9cdc105470 msgr2=0x7f9cdc19aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:49.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:49.417+0000 7f9ce2400640 1 --2- 172.21.15.83:0/2018818020 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9cdc105470 0x7f9cdc19aa60 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f9cc400c970 tx=0x7f9cc400ce30 comp rx=0 tx=0).stop 2024-04-24T08:48:49.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:49.417+0000 7f9ce2400640 1 -- 172.21.15.83:0/2018818020 shutdown_connections 2024-04-24T08:48:49.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:49.417+0000 7f9ce2400640 1 --2- 172.21.15.83:0/2018818020 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9cdc105e30 0x7f9cdc1a1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:49.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:49.417+0000 7f9ce2400640 1 --2- 172.21.15.83:0/2018818020 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f9cc00a3bb0 0x7f9cc00a6070 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:49.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:49.417+0000 7f9ce2400640 1 --2- 172.21.15.83:0/2018818020 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9cdc105470 0x7f9cdc19aa60 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:49.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:49.417+0000 7f9ce2400640 1 --2- 172.21.15.83:0/2018818020 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9cdc104280 0x7f9cdc19a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:49.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:49.417+0000 7f9ce2400640 1 -- 172.21.15.83:0/2018818020 >> 172.21.15.83:0/2018818020 conn(0x7f9cdc0ff7f0 msgr2=0x7f9cdc108890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:48:49.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:49.418+0000 7f9ce2400640 1 -- 172.21.15.83:0/2018818020 shutdown_connections 2024-04-24T08:48:49.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:49.418+0000 7f9ce2400640 1 -- 172.21.15.83:0/2018818020 wait complete. 2024-04-24T08:48:49.436 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-24T08:48:49.436 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T08:48:49.485 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T08:48:49.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:49.729+0000 7f629ec00640 1 -- 172.21.15.83:0/1149498162 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6298077c00 msgr2=0x7f6298078060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:49.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:49.729+0000 7f629ec00640 1 --2- 172.21.15.83:0/1149498162 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6298077c00 0x7f6298078060 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f6284009900 tx=0x7f628402a3f0 comp rx=0 tx=0).stop 2024-04-24T08:48:49.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:49.729+0000 7f629ec00640 1 -- 172.21.15.83:0/1149498162 shutdown_connections 2024-04-24T08:48:49.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:49.729+0000 7f629ec00640 1 --2- 172.21.15.83:0/1149498162 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f62980785a0 0x7f6298078e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:49.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:49.729+0000 7f629ec00640 1 --2- 172.21.15.83:0/1149498162 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6298077c00 0x7f6298078060 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:49.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:49.729+0000 7f629ec00640 1 --2- 172.21.15.83:0/1149498162 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f62980769c0 0x7f6298076de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:49.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:49.729+0000 7f629ec00640 1 -- 172.21.15.83:0/1149498162 >> 172.21.15.83:0/1149498162 conn(0x7f62980ff610 msgr2=0x7f6298101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:48:49.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:49.730+0000 7f629ec00640 1 -- 172.21.15.83:0/1149498162 shutdown_connections 2024-04-24T08:48:49.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:49.730+0000 7f629ec00640 1 -- 172.21.15.83:0/1149498162 wait complete. 2024-04-24T08:48:49.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:49.731+0000 7f629ec00640 1 Processor -- start 2024-04-24T08:48:49.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:49.731+0000 7f629ec00640 1 -- start start 2024-04-24T08:48:49.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:49.731+0000 7f629ec00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f62980769c0 0x7f629819e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:49.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:49.731+0000 7f629ec00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6298077c00 0x7f629819ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:49.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:49.731+0000 7f629ec00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62980785a0 0x7f62981a5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:49.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:49.731+0000 7f629ec00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6298114040 con 0x7f62980785a0 2024-04-24T08:48:49.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:49.731+0000 7f629ec00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6298113ec0 con 0x7f62980769c0 2024-04-24T08:48:49.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:49.731+0000 7f629ec00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f62981141c0 con 0x7f6298077c00 2024-04-24T08:48:49.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:49.731+0000 7f629e200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62980785a0 0x7f62981a5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:49.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:49.731+0000 7f629e200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62980785a0 0x7f62981a5cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35346/0 (socket says 172.21.15.83:35346) 2024-04-24T08:48:49.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:49.731+0000 7f629e200640 1 -- 172.21.15.83:0/3327306564 learned_addr learned my addr 172.21.15.83:0/3327306564 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:48:49.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:49.732+0000 7f629d800640 1 --2- 172.21.15.83:0/3327306564 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f62980769c0 0x7f629819e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:49.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:49.732+0000 7f629ce00640 1 --2- 172.21.15.83:0/3327306564 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6298077c00 0x7f629819ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:49.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:49.732+0000 7f629e200640 1 -- 172.21.15.83:0/3327306564 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6298077c00 msgr2=0x7f629819ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:49.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:49.732+0000 7f629e200640 1 --2- 172.21.15.83:0/3327306564 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6298077c00 0x7f629819ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:49.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:49.732+0000 7f629e200640 1 -- 172.21.15.83:0/3327306564 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f62980769c0 msgr2=0x7f629819e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:49.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:49.732+0000 7f629e200640 1 --2- 172.21.15.83:0/3327306564 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f62980769c0 0x7f629819e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:49.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:49.732+0000 7f629e200640 1 -- 172.21.15.83:0/3327306564 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62981a6330 con 0x7f62980785a0 2024-04-24T08:48:49.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:49.732+0000 7f629e200640 1 --2- 172.21.15.83:0/3327306564 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62980785a0 0x7f62981a5cc0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f629000cad0 tx=0x7f629000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:48:49.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:49.732+0000 7f6296000640 1 -- 172.21.15.83:0/3327306564 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f629000ae20 con 0x7f62980785a0 2024-04-24T08:48:49.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:49.732+0000 7f6296000640 1 -- 172.21.15.83:0/3327306564 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6290005020 con 0x7f62980785a0 2024-04-24T08:48:49.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:49.732+0000 7f629ec00640 1 -- 172.21.15.83:0/3327306564 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62981a6620 con 0x7f62980785a0 2024-04-24T08:48:49.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:49.732+0000 7f6296000640 1 -- 172.21.15.83:0/3327306564 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f629000f6b0 con 0x7f62980785a0 2024-04-24T08:48:49.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:49.733+0000 7f629ec00640 1 -- 172.21.15.83:0/3327306564 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62981a6b70 con 0x7f62980785a0 2024-04-24T08:48:49.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:49.734+0000 7f629ec00640 1 -- 172.21.15.83:0/3327306564 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6298110be0 con 0x7f62980785a0 2024-04-24T08:48:49.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:49.736+0000 7f6296000640 1 -- 172.21.15.83:0/3327306564 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f62900051d0 con 0x7f62980785a0 2024-04-24T08:48:49.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:49.737+0000 7f6296000640 1 --2- 172.21.15.83:0/3327306564 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f62800a3bb0 0x7f62800a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:49.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:49.737+0000 7f6296000640 1 -- 172.21.15.83:0/3327306564 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f62900d07d0 con 0x7f62980785a0 2024-04-24T08:48:49.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:49.737+0000 7f629d800640 1 --2- 172.21.15.83:0/3327306564 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f62800a3bb0 0x7f62800a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:49.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:49.738+0000 7f629d800640 1 --2- 172.21.15.83:0/3327306564 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f62800a3bb0 0x7f62800a6070 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f628c009890 tx=0x7f628c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:48:49.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:49.742+0000 7f6296000640 1 -- 172.21.15.83:0/3327306564 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6290014030 con 0x7f62980785a0 2024-04-24T08:48:49.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:49.945+0000 7f629ec00640 1 -- 172.21.15.83:0/3327306564 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6298062c00 con 0x7f62980785a0 2024-04-24T08:48:49.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:49.949+0000 7f6296000640 1 -- 172.21.15.83:0/3327306564 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24127 (secure 0 0 0) 0x7f629009cf10 con 0x7f62980785a0 2024-04-24T08:48:49.949 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:48:49.949 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":89,"fsid":"bd3e039e-0215-11ef-bc93-c7b262605968","created":"2024-04-24T08:37:32.209944+0000","modified":"2024-04-24T08:48:34.752923+0000","last_up_change":"2024-04-24T08:47:17.966041+0000","last_in_change":"2024-04-24T08:46:53.835936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T08:41:56.834372+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T08:48:26.978311+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T08:48:28.352731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T08:48:32.275167+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2cd77f3a-19dd-4c6e-a199-1dcbd5faa679","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2132675555},{"type":"v1","addr":"172.21.15.83:6803","nonce":2132675555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2132675555},{"type":"v1","addr":"172.21.15.83:6805","nonce":2132675555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2132675555},{"type":"v1","addr":"172.21.15.83:6809","nonce":2132675555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2132675555},{"type":"v1","addr":"172.21.15.83:6807","nonce":2132675555}]},"public_addr":"172.21.15.83:6803/2132675555","cluster_addr":"172.21.15.83:6805/2132675555","heartbeat_back_addr":"172.21.15.83:6809/2132675555","heartbeat_front_addr":"172.21.15.83:6807/2132675555","state":["exists","up"]},{"osd":1,"uuid":"aba7f5df-2a4c-4c23-b502-d193605a5b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":3589158880},{"type":"v1","addr":"172.21.15.138:6801","nonce":3589158880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":3589158880},{"type":"v1","addr":"172.21.15.138:6803","nonce":3589158880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":3589158880},{"type":"v1","addr":"172.21.15.138:6807","nonce":3589158880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":3589158880},{"type":"v1","addr":"172.21.15.138:6805","nonce":3589158880}]},"public_addr":"172.21.15.138:6801/3589158880","cluster_addr":"172.21.15.138:6803/3589158880","heartbeat_back_addr":"172.21.15.138:6807/3589158880","heartbeat_front_addr":"172.21.15.138:6805/3589158880","state":["exists","up"]},{"osd":2,"uuid":"071bc167-2333-4420-bb58-bffbc10d7641","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":3169072688},{"type":"v1","addr":"172.21.15.189:6801","nonce":3169072688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":3169072688},{"type":"v1","addr":"172.21.15.189:6803","nonce":3169072688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":3169072688},{"type":"v1","addr":"172.21.15.189:6807","nonce":3169072688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":3169072688},{"type":"v1","addr":"172.21.15.189:6805","nonce":3169072688}]},"public_addr":"172.21.15.189:6801/3169072688","cluster_addr":"172.21.15.189:6803/3169072688","heartbeat_back_addr":"172.21.15.189:6807/3169072688","heartbeat_front_addr":"172.21.15.189:6805/3169072688","state":["exists","up"]},{"osd":3,"uuid":"f5624d78-4455-4e07-8c1a-f47c94cddb23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3131092450},{"type":"v1","addr":"172.21.15.83:6811","nonce":3131092450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3131092450},{"type":"v1","addr":"172.21.15.83:6813","nonce":3131092450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3131092450},{"type":"v1","addr":"172.21.15.83:6817","nonce":3131092450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3131092450},{"type":"v1","addr":"172.21.15.83:6815","nonce":3131092450}]},"public_addr":"172.21.15.83:6811/3131092450","cluster_addr":"172.21.15.83:6813/3131092450","heartbeat_back_addr":"172.21.15.83:6817/3131092450","heartbeat_front_addr":"172.21.15.83:6815/3131092450","state":["exists","up"]},{"osd":4,"uuid":"7b50cd14-9385-4535-a7b0-1c1da93c8548","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":823775360},{"type":"v1","addr":"172.21.15.138:6809","nonce":823775360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":823775360},{"type":"v1","addr":"172.21.15.138:6811","nonce":823775360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":823775360},{"type":"v1","addr":"172.21.15.138:6815","nonce":823775360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":823775360},{"type":"v1","addr":"172.21.15.138:6813","nonce":823775360}]},"public_addr":"172.21.15.138:6809/823775360","cluster_addr":"172.21.15.138:6811/823775360","heartbeat_back_addr":"172.21.15.138:6815/823775360","heartbeat_front_addr":"172.21.15.138:6813/823775360","state":["exists","up"]},{"osd":5,"uuid":"9c647797-35bc-449f-be84-af97d2c69c2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":1399817431},{"type":"v1","addr":"172.21.15.189:6809","nonce":1399817431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":1399817431},{"type":"v1","addr":"172.21.15.189:6811","nonce":1399817431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":1399817431},{"type":"v1","addr":"172.21.15.189:6815","nonce":1399817431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":1399817431},{"type":"v1","addr":"172.21.15.189:6813","nonce":1399817431}]},"public_addr":"172.21.15.189:6809/1399817431","cluster_addr":"172.21.15.189:6811/1399817431","heartbeat_back_addr":"172.21.15.189:6815/1399817431","heartbeat_front_addr":"172.21.15.189:6813/1399817431","state":["exists","up"]},{"osd":6,"uuid":"8f056584-16c8-4454-b445-3f780f26c196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":1611358483},{"type":"v1","addr":"172.21.15.83:6819","nonce":1611358483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1611358483},{"type":"v1","addr":"172.21.15.83:6821","nonce":1611358483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1611358483},{"type":"v1","addr":"172.21.15.83:6825","nonce":1611358483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":1611358483},{"type":"v1","addr":"172.21.15.83:6823","nonce":1611358483}]},"public_addr":"172.21.15.83:6819/1611358483","cluster_addr":"172.21.15.83:6821/1611358483","heartbeat_back_addr":"172.21.15.83:6825/1611358483","heartbeat_front_addr":"172.21.15.83:6823/1611358483","state":["exists","up"]},{"osd":7,"uuid":"7603efe3-da77-4b7f-9d03-29c4f293b6d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":3557856706},{"type":"v1","addr":"172.21.15.138:6817","nonce":3557856706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":3557856706},{"type":"v1","addr":"172.21.15.138:6819","nonce":3557856706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":3557856706},{"type":"v1","addr":"172.21.15.138:6823","nonce":3557856706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":3557856706},{"type":"v1","addr":"172.21.15.138:6821","nonce":3557856706}]},"public_addr":"172.21.15.138:6817/3557856706","cluster_addr":"172.21.15.138:6819/3557856706","heartbeat_back_addr":"172.21.15.138:6823/3557856706","heartbeat_front_addr":"172.21.15.138:6821/3557856706","state":["exists","up"]},{"osd":8,"uuid":"27d7d1b8-d6a0-417e-b8aa-596325aa1a77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":207018554},{"type":"v1","addr":"172.21.15.189:6817","nonce":207018554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":207018554},{"type":"v1","addr":"172.21.15.189:6819","nonce":207018554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":207018554},{"type":"v1","addr":"172.21.15.189:6823","nonce":207018554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":207018554},{"type":"v1","addr":"172.21.15.189:6821","nonce":207018554}]},"public_addr":"172.21.15.189:6817/207018554","cluster_addr":"172.21.15.189:6819/207018554","heartbeat_back_addr":"172.21.15.189:6823/207018554","heartbeat_front_addr":"172.21.15.189:6821/207018554","state":["exists","up"]},{"osd":9,"uuid":"baf19b98-881b-4d3a-9f72-8aa2dd73cad6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":519636751},{"type":"v1","addr":"172.21.15.83:6827","nonce":519636751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":519636751},{"type":"v1","addr":"172.21.15.83:6829","nonce":519636751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":519636751},{"type":"v1","addr":"172.21.15.83:6833","nonce":519636751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":519636751},{"type":"v1","addr":"172.21.15.83:6831","nonce":519636751}]},"public_addr":"172.21.15.83:6827/519636751","cluster_addr":"172.21.15.83:6829/519636751","heartbeat_back_addr":"172.21.15.83:6833/519636751","heartbeat_front_addr":"172.21.15.83:6831/519636751","state":["exists","up"]},{"osd":10,"uuid":"eb346717-5c07-47c8-95be-0e68fe449fdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1130872098},{"type":"v1","addr":"172.21.15.138:6825","nonce":1130872098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1130872098},{"type":"v1","addr":"172.21.15.138:6827","nonce":1130872098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1130872098},{"type":"v1","addr":"172.21.15.138:6831","nonce":1130872098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1130872098},{"type":"v1","addr":"172.21.15.138:6829","nonce":1130872098}]},"public_addr":"172.21.15.138:6825/1130872098","cluster_addr":"172.21.15.138:6827/1130872098","heartbeat_back_addr":"172.21.15.138:6831/1130872098","heartbeat_front_addr":"172.21.15.138:6829/1130872098","state":["exists","up"]},{"osd":11,"uuid":"8e37568e-981c-45b8-b2af-2f1b73898542","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":2998782690},{"type":"v1","addr":"172.21.15.189:6825","nonce":2998782690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":2998782690},{"type":"v1","addr":"172.21.15.189:6827","nonce":2998782690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":2998782690},{"type":"v1","addr":"172.21.15.189:6831","nonce":2998782690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":2998782690},{"type":"v1","addr":"172.21.15.189:6829","nonce":2998782690}]},"public_addr":"172.21.15.189:6825/2998782690","cluster_addr":"172.21.15.189:6827/2998782690","heartbeat_back_addr":"172.21.15.189:6831/2998782690","heartbeat_front_addr":"172.21.15.189:6829/2998782690","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:40:47.980103+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:41:21.970795+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:41:55.160426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:42:30.400475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:43:05.705663+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:43:40.112312+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:44:16.781289+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:44:52.151187+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:45:28.055944+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:46:04.969301+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:46:40.918306+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:47:16.886874+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:6801/820049883":"2024-04-25T08:38:40.661315+0000","172.21.15.83:0/4272874138":"2024-04-25T08:38:40.661315+0000","172.21.15.83:0/4212994335":"2024-04-25T08:38:40.661315+0000","172.21.15.83:6801/1996371091":"2024-04-25T08:38:05.800856+0000","172.21.15.83:0/1820663683":"2024-04-25T08:38:05.800856+0000","172.21.15.83:0/3802897262":"2024-04-25T08:38:05.800856+0000","172.21.15.83:6800/1996371091":"2024-04-25T08:38:05.800856+0000","172.21.15.83:6800/820049883":"2024-04-25T08:38:40.661315+0000","172.21.15.83:0/1366125049":"2024-04-25T08:38:05.800856+0000","172.21.15.83:0/340452868":"2024-04-25T08:38:40.661315+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T08:48:49.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:49.951+0000 7f629ec00640 1 -- 172.21.15.83:0/3327306564 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f62800a3bb0 msgr2=0x7f62800a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:49.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:49.951+0000 7f629ec00640 1 --2- 172.21.15.83:0/3327306564 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f62800a3bb0 0x7f62800a6070 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f628c009890 tx=0x7f628c008040 comp rx=0 tx=0).stop 2024-04-24T08:48:49.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:49.951+0000 7f629ec00640 1 -- 172.21.15.83:0/3327306564 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62980785a0 msgr2=0x7f62981a5cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:49.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:49.951+0000 7f629ec00640 1 --2- 172.21.15.83:0/3327306564 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62980785a0 0x7f62981a5cc0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f629000cad0 tx=0x7f629000cf90 comp rx=0 tx=0).stop 2024-04-24T08:48:49.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:49.951+0000 7f629ec00640 1 -- 172.21.15.83:0/3327306564 shutdown_connections 2024-04-24T08:48:49.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:49.951+0000 7f629ec00640 1 --2- 172.21.15.83:0/3327306564 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f62800a3bb0 0x7f62800a6070 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:49.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:49.951+0000 7f629ec00640 1 --2- 172.21.15.83:0/3327306564 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62980785a0 0x7f62981a5cc0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:49.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:49.951+0000 7f629ec00640 1 --2- 172.21.15.83:0/3327306564 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6298077c00 0x7f629819ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:49.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:49.951+0000 7f629ec00640 1 --2- 172.21.15.83:0/3327306564 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f62980769c0 0x7f629819e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:49.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:49.951+0000 7f629ec00640 1 -- 172.21.15.83:0/3327306564 >> 172.21.15.83:0/3327306564 conn(0x7f62980ff610 msgr2=0x7f6298100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:48:49.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:49.951+0000 7f629ec00640 1 -- 172.21.15.83:0/3327306564 shutdown_connections 2024-04-24T08:48:49.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:49.951+0000 7f629ec00640 1 -- 172.21.15.83:0/3327306564 wait complete. 2024-04-24T08:48:49.972 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T08:41:56.834372+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T08:48:26.978311+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T08:48:28.352731+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-24T08:48:32.275167+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-24T08:48:49.972 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T08:48:50.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:50.176+0000 7fa4bd200640 1 -- 172.21.15.83:0/1249264996 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4b8105330 msgr2=0x7fa4b81057b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:50.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:50.176+0000 7fa4bd200640 1 --2- 172.21.15.83:0/1249264996 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4b8105330 0x7fa4b81057b0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa4a0009900 tx=0x7fa4a002a410 comp rx=0 tx=0).stop 2024-04-24T08:48:50.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:50.176+0000 7fa4bd200640 1 -- 172.21.15.83:0/1249264996 shutdown_connections 2024-04-24T08:48:50.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:50.176+0000 7fa4bd200640 1 --2- 172.21.15.83:0/1249264996 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa4b8105cf0 0x7fa4b810c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:50.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:50.176+0000 7fa4bd200640 1 --2- 172.21.15.83:0/1249264996 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4b8105330 0x7fa4b81057b0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:50.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:50.176+0000 7fa4bd200640 1 --2- 172.21.15.83:0/1249264996 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4b8104140 0x7fa4b8104560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:50.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:50.176+0000 7fa4bd200640 1 -- 172.21.15.83:0/1249264996 >> 172.21.15.83:0/1249264996 conn(0x7fa4b80ff6b0 msgr2=0x7fa4b8101af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:48:50.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:50.176+0000 7fa4bd200640 1 -- 172.21.15.83:0/1249264996 shutdown_connections 2024-04-24T08:48:50.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:50.176+0000 7fa4bd200640 1 -- 172.21.15.83:0/1249264996 wait complete. 2024-04-24T08:48:50.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:50.177+0000 7fa4bd200640 1 Processor -- start 2024-04-24T08:48:50.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:50.178+0000 7fa4bd200640 1 -- start start 2024-04-24T08:48:50.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:50.178+0000 7fa4bd200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4b8104140 0x7fa4b819a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:50.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:50.178+0000 7fa4bd200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa4b8105330 0x7fa4b819a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:50.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:50.178+0000 7fa4bd200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4b8105cf0 0x7fa4b81a18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:50.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:50.178+0000 7fa4bd200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b810fc80 con 0x7fa4b8105cf0 2024-04-24T08:48:50.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:50.178+0000 7fa4bd200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b810fb00 con 0x7fa4b8104140 2024-04-24T08:48:50.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:50.178+0000 7fa4bd200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b810fe00 con 0x7fa4b8105330 2024-04-24T08:48:50.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:50.178+0000 7fa4b7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4b8105cf0 0x7fa4b81a18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:50.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:50.178+0000 7fa4b7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4b8105cf0 0x7fa4b81a18e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35372/0 (socket says 172.21.15.83:35372) 2024-04-24T08:48:50.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:50.178+0000 7fa4b7e00640 1 -- 172.21.15.83:0/3264861577 learned_addr learned my addr 172.21.15.83:0/3264861577 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:48:50.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:50.179+0000 7fa4b7400640 1 --2- 172.21.15.83:0/3264861577 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4b8104140 0x7fa4b819a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:50.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:50.179+0000 7fa4afe00640 1 --2- 172.21.15.83:0/3264861577 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa4b8105330 0x7fa4b819a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:50.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:50.179+0000 7fa4b7e00640 1 -- 172.21.15.83:0/3264861577 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa4b8105330 msgr2=0x7fa4b819a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:50.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:50.179+0000 7fa4b7e00640 1 --2- 172.21.15.83:0/3264861577 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa4b8105330 0x7fa4b819a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:50.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:50.179+0000 7fa4b7e00640 1 -- 172.21.15.83:0/3264861577 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4b8104140 msgr2=0x7fa4b819a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:50.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:50.179+0000 7fa4b7e00640 1 --2- 172.21.15.83:0/3264861577 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4b8104140 0x7fa4b819a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:50.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:50.179+0000 7fa4b7e00640 1 -- 172.21.15.83:0/3264861577 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4b81a1f50 con 0x7fa4b8105cf0 2024-04-24T08:48:50.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:50.179+0000 7fa4b7e00640 1 --2- 172.21.15.83:0/3264861577 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4b8105cf0 0x7fa4b81a18e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fa4b000eaa0 tx=0x7fa4b000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:48:50.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:50.179+0000 7fa4b4c00640 1 -- 172.21.15.83:0/3264861577 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4b000cdb0 con 0x7fa4b8105cf0 2024-04-24T08:48:50.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:50.179+0000 7fa4b4c00640 1 -- 172.21.15.83:0/3264861577 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa4b0005020 con 0x7fa4b8105cf0 2024-04-24T08:48:50.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:50.179+0000 7fa4bd200640 1 -- 172.21.15.83:0/3264861577 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4b81a2240 con 0x7fa4b8105cf0 2024-04-24T08:48:50.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:50.179+0000 7fa4b4c00640 1 -- 172.21.15.83:0/3264861577 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4b00116d0 con 0x7fa4b8105cf0 2024-04-24T08:48:50.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:50.180+0000 7fa4bd200640 1 -- 172.21.15.83:0/3264861577 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4b81a2790 con 0x7fa4b8105cf0 2024-04-24T08:48:50.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:50.181+0000 7fa4bd200640 1 -- 172.21.15.83:0/3264861577 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4b810b4b0 con 0x7fa4b8105cf0 2024-04-24T08:48:50.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:50.183+0000 7fa4b4c00640 1 -- 172.21.15.83:0/3264861577 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fa4b00051d0 con 0x7fa4b8105cf0 2024-04-24T08:48:50.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:50.184+0000 7fa4b4c00640 1 --2- 172.21.15.83:0/3264861577 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fa49c0a3bb0 0x7fa49c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:50.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:50.184+0000 7fa4b4c00640 1 -- 172.21.15.83:0/3264861577 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fa4b00d0bf0 con 0x7fa4b8105cf0 2024-04-24T08:48:50.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:50.185+0000 7fa4b7400640 1 --2- 172.21.15.83:0/3264861577 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fa49c0a3bb0 0x7fa49c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:50.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:50.185+0000 7fa4b7400640 1 --2- 172.21.15.83:0/3264861577 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fa49c0a3bb0 0x7fa49c0a6070 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa4a8005d00 tx=0x7fa4a8005c50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:48:50.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:50.189+0000 7fa4b4c00640 1 -- 172.21.15.83:0/3264861577 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa4b0016030 con 0x7fa4b8105cf0 2024-04-24T08:48:50.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:49 smithi138 ceph-mon[31441]: pgmap v383: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-24T08:48:50.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:49 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/996811087' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T08:48:50.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:49 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/2018818020' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-24T08:48:50.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:49 smithi189 ceph-mon[31791]: pgmap v383: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-24T08:48:50.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:49 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/996811087' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T08:48:50.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:49 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/2018818020' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-24T08:48:50.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:49 smithi083 ceph-mon[27907]: pgmap v383: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-24T08:48:50.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:49 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/996811087' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T08:48:50.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:49 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/2018818020' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-24T08:48:50.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:50.397+0000 7fa4bd200640 1 -- 172.21.15.83:0/3264861577 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa4b8062c00 con 0x7fa4b8105cf0 2024-04-24T08:48:50.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:50.398+0000 7fa4b4c00640 1 -- 172.21.15.83:0/3264861577 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) ==== 93+0+10 (secure 0 0 0) 0x7fa4b009d390 con 0x7fa4b8105cf0 2024-04-24T08:48:50.398 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-24T08:48:50.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:50.402+0000 7fa4bd200640 1 -- 172.21.15.83:0/3264861577 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fa49c0a3bb0 msgr2=0x7fa49c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:50.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:50.402+0000 7fa4bd200640 1 --2- 172.21.15.83:0/3264861577 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fa49c0a3bb0 0x7fa49c0a6070 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa4a8005d00 tx=0x7fa4a8005c50 comp rx=0 tx=0).stop 2024-04-24T08:48:50.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:50.402+0000 7fa4bd200640 1 -- 172.21.15.83:0/3264861577 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4b8105cf0 msgr2=0x7fa4b81a18e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:50.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:50.402+0000 7fa4bd200640 1 --2- 172.21.15.83:0/3264861577 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4b8105cf0 0x7fa4b81a18e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fa4b000eaa0 tx=0x7fa4b000ef60 comp rx=0 tx=0).stop 2024-04-24T08:48:50.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:50.402+0000 7fa4bd200640 1 -- 172.21.15.83:0/3264861577 shutdown_connections 2024-04-24T08:48:50.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:50.402+0000 7fa4bd200640 1 --2- 172.21.15.83:0/3264861577 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fa49c0a3bb0 0x7fa49c0a6070 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:50.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:50.402+0000 7fa4bd200640 1 --2- 172.21.15.83:0/3264861577 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4b8105cf0 0x7fa4b81a18e0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:50.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:50.402+0000 7fa4bd200640 1 --2- 172.21.15.83:0/3264861577 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa4b8105330 0x7fa4b819a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:50.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:50.402+0000 7fa4bd200640 1 --2- 172.21.15.83:0/3264861577 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4b8104140 0x7fa4b819a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:50.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:50.402+0000 7fa4bd200640 1 -- 172.21.15.83:0/3264861577 >> 172.21.15.83:0/3264861577 conn(0x7fa4b80ff6b0 msgr2=0x7fa4b8108680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:48:50.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:50.402+0000 7fa4bd200640 1 -- 172.21.15.83:0/3264861577 shutdown_connections 2024-04-24T08:48:50.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:50.402+0000 7fa4bd200640 1 -- 172.21.15.83:0/3264861577 wait complete. 2024-04-24T08:48:50.421 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T08:48:50.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:50.630+0000 7fa01de00640 1 -- 172.21.15.83:0/3985337951 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa018105cd0 msgr2=0x7fa01810c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:50.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:50.630+0000 7fa01de00640 1 --2- 172.21.15.83:0/3985337951 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa018105cd0 0x7fa01810c560 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fa010009ca0 tx=0x7fa01002a8c0 comp rx=0 tx=0).stop 2024-04-24T08:48:50.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:50.630+0000 7fa01de00640 1 -- 172.21.15.83:0/3985337951 shutdown_connections 2024-04-24T08:48:50.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:50.630+0000 7fa01de00640 1 --2- 172.21.15.83:0/3985337951 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa018105cd0 0x7fa01810c560 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:50.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:50.630+0000 7fa01de00640 1 --2- 172.21.15.83:0/3985337951 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa018105310 0x7fa018105790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:50.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:50.630+0000 7fa01de00640 1 --2- 172.21.15.83:0/3985337951 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa018104120 0x7fa018104540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:50.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:50.630+0000 7fa01de00640 1 -- 172.21.15.83:0/3985337951 >> 172.21.15.83:0/3985337951 conn(0x7fa0180ff6b0 msgr2=0x7fa018101ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:48:50.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:50.630+0000 7fa01de00640 1 -- 172.21.15.83:0/3985337951 shutdown_connections 2024-04-24T08:48:50.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:50.630+0000 7fa01de00640 1 -- 172.21.15.83:0/3985337951 wait complete. 2024-04-24T08:48:50.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:50.631+0000 7fa01de00640 1 Processor -- start 2024-04-24T08:48:50.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:50.631+0000 7fa01de00640 1 -- start start 2024-04-24T08:48:50.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:50.631+0000 7fa01de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa018104120 0x7fa01819a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:50.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:50.631+0000 7fa01de00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa018105310 0x7fa01819a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:50.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:50.631+0000 7fa01de00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa018105cd0 0x7fa0181a18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:50.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:50.631+0000 7fa01de00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa01810fc30 con 0x7fa018104120 2024-04-24T08:48:50.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:50.631+0000 7fa01de00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa01810fab0 con 0x7fa018105310 2024-04-24T08:48:50.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:50.631+0000 7fa01de00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa01810fdb0 con 0x7fa018105cd0 2024-04-24T08:48:50.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:50.632+0000 7fa01ca00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa018104120 0x7fa01819a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:50.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:50.632+0000 7fa01ca00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa018104120 0x7fa01819a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35386/0 (socket says 172.21.15.83:35386) 2024-04-24T08:48:50.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:50.632+0000 7fa01ca00640 1 -- 172.21.15.83:0/1683346346 learned_addr learned my addr 172.21.15.83:0/1683346346 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:48:50.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:50.632+0000 7fa01d400640 1 --2- 172.21.15.83:0/1683346346 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa018105cd0 0x7fa0181a18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:50.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:50.632+0000 7fa017e00640 1 --2- 172.21.15.83:0/1683346346 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa018105310 0x7fa01819a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:50.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:50.632+0000 7fa01ca00640 1 -- 172.21.15.83:0/1683346346 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa018105cd0 msgr2=0x7fa0181a18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:50.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:50.632+0000 7fa01ca00640 1 --2- 172.21.15.83:0/1683346346 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa018105cd0 0x7fa0181a18b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:50.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:50.632+0000 7fa01ca00640 1 -- 172.21.15.83:0/1683346346 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa018105310 msgr2=0x7fa01819a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:50.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:50.632+0000 7fa01ca00640 1 --2- 172.21.15.83:0/1683346346 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa018105310 0x7fa01819a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:50.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:50.632+0000 7fa01ca00640 1 -- 172.21.15.83:0/1683346346 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0181a1f20 con 0x7fa018104120 2024-04-24T08:48:50.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:50.632+0000 7fa01ca00640 1 --2- 172.21.15.83:0/1683346346 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa018104120 0x7fa01819a300 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa00400d770 tx=0x7fa00400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:48:50.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:50.633+0000 7fa015600640 1 -- 172.21.15.83:0/1683346346 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa004017070 con 0x7fa018104120 2024-04-24T08:48:50.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:50.633+0000 7fa015600640 1 -- 172.21.15.83:0/1683346346 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa004009c30 con 0x7fa018104120 2024-04-24T08:48:50.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:50.633+0000 7fa01de00640 1 -- 172.21.15.83:0/1683346346 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0181a2210 con 0x7fa018104120 2024-04-24T08:48:50.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:50.633+0000 7fa015600640 1 -- 172.21.15.83:0/1683346346 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa004003d00 con 0x7fa018104120 2024-04-24T08:48:50.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:50.633+0000 7fa01de00640 1 -- 172.21.15.83:0/1683346346 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0181a2760 con 0x7fa018104120 2024-04-24T08:48:50.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:50.634+0000 7fa01de00640 1 -- 172.21.15.83:0/1683346346 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa01810b560 con 0x7fa018104120 2024-04-24T08:48:50.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:50.637+0000 7fa015600640 1 -- 172.21.15.83:0/1683346346 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fa004003ea0 con 0x7fa018104120 2024-04-24T08:48:50.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:50.638+0000 7fa015600640 1 --2- 172.21.15.83:0/1683346346 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fa0000a3bb0 0x7fa0000a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:50.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:50.638+0000 7fa015600640 1 -- 172.21.15.83:0/1683346346 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fa0040d0400 con 0x7fa018104120 2024-04-24T08:48:50.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:50.639+0000 7fa017e00640 1 --2- 172.21.15.83:0/1683346346 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fa0000a3bb0 0x7fa0000a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:50.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:50.639+0000 7fa017e00640 1 --2- 172.21.15.83:0/1683346346 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fa0000a3bb0 0x7fa0000a6070 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa01819b810 tx=0x7fa00c00b590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:48:50.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:50.643+0000 7fa015600640 1 -- 172.21.15.83:0/1683346346 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa00409cba0 con 0x7fa018104120 2024-04-24T08:48:50.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:50.847+0000 7fa01de00640 1 -- 172.21.15.83:0/1683346346 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa018062c00 con 0x7fa018104120 2024-04-24T08:48:50.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:50.848+0000 7fa015600640 1 -- 172.21.15.83:0/1683346346 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) ==== 104+0+11 (secure 0 0 0) 0x7fa004009de0 con 0x7fa018104120 2024-04-24T08:48:50.848 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-24T08:48:50.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:50.851+0000 7fa01de00640 1 -- 172.21.15.83:0/1683346346 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fa0000a3bb0 msgr2=0x7fa0000a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:50.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:50.851+0000 7fa01de00640 1 --2- 172.21.15.83:0/1683346346 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fa0000a3bb0 0x7fa0000a6070 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa01819b810 tx=0x7fa00c00b590 comp rx=0 tx=0).stop 2024-04-24T08:48:50.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:50.851+0000 7fa01de00640 1 -- 172.21.15.83:0/1683346346 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa018104120 msgr2=0x7fa01819a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:50.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:50.851+0000 7fa01de00640 1 --2- 172.21.15.83:0/1683346346 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa018104120 0x7fa01819a300 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa00400d770 tx=0x7fa00400dc30 comp rx=0 tx=0).stop 2024-04-24T08:48:50.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:50.851+0000 7fa01de00640 1 -- 172.21.15.83:0/1683346346 shutdown_connections 2024-04-24T08:48:50.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:50.851+0000 7fa01de00640 1 --2- 172.21.15.83:0/1683346346 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa018105cd0 0x7fa0181a18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:50.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:50.851+0000 7fa01de00640 1 --2- 172.21.15.83:0/1683346346 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa018105310 0x7fa01819a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:50.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:50.851+0000 7fa01de00640 1 --2- 172.21.15.83:0/1683346346 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fa0000a3bb0 0x7fa0000a6070 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:50.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:50.851+0000 7fa01de00640 1 --2- 172.21.15.83:0/1683346346 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa018104120 0x7fa01819a300 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:50.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:50.851+0000 7fa01de00640 1 -- 172.21.15.83:0/1683346346 >> 172.21.15.83:0/1683346346 conn(0x7fa0180ff6b0 msgr2=0x7fa018108730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:48:50.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:50.851+0000 7fa01de00640 1 -- 172.21.15.83:0/1683346346 shutdown_connections 2024-04-24T08:48:50.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:50.851+0000 7fa01de00640 1 -- 172.21.15.83:0/1683346346 wait complete. 2024-04-24T08:48:50.869 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T08:48:51.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.067+0000 7fa087200640 1 -- 172.21.15.83:0/360725466 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa080101f30 msgr2=0x7fa080102350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:51.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.067+0000 7fa087200640 1 --2- 172.21.15.83:0/360725466 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa080101f30 0x7fa080102350 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa074009900 tx=0x7fa07402a410 comp rx=0 tx=0).stop 2024-04-24T08:48:51.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.067+0000 7fa087200640 1 -- 172.21.15.83:0/360725466 shutdown_connections 2024-04-24T08:48:51.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.067+0000 7fa087200640 1 --2- 172.21.15.83:0/360725466 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa080103a40 0x7fa08010a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:51.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.067+0000 7fa087200640 1 --2- 172.21.15.83:0/360725466 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa080103080 0x7fa080103500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:51.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.067+0000 7fa087200640 1 --2- 172.21.15.83:0/360725466 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa080101f30 0x7fa080102350 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:51.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.067+0000 7fa087200640 1 -- 172.21.15.83:0/360725466 >> 172.21.15.83:0/360725466 conn(0x7fa0800fd4a0 msgr2=0x7fa0800ff8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:48:51.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.068+0000 7fa087200640 1 -- 172.21.15.83:0/360725466 shutdown_connections 2024-04-24T08:48:51.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.068+0000 7fa087200640 1 -- 172.21.15.83:0/360725466 wait complete. 2024-04-24T08:48:51.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.068+0000 7fa087200640 1 Processor -- start 2024-04-24T08:48:51.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.069+0000 7fa087200640 1 -- start start 2024-04-24T08:48:51.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.069+0000 7fa087200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa080101f30 0x7fa0801981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:51.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.069+0000 7fa087200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa080103080 0x7fa0801986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:51.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.069+0000 7fa087200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa080103a40 0x7fa08019f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:51.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.069+0000 7fa087200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa08010da70 con 0x7fa080103a40 2024-04-24T08:48:51.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.069+0000 7fa087200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa08010d8f0 con 0x7fa080101f30 2024-04-24T08:48:51.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.069+0000 7fa087200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa08010dbf0 con 0x7fa080103080 2024-04-24T08:48:51.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.069+0000 7fa086800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa080103a40 0x7fa08019f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:51.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.069+0000 7fa086800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa080103a40 0x7fa08019f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35414/0 (socket says 172.21.15.83:35414) 2024-04-24T08:48:51.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.069+0000 7fa086800640 1 -- 172.21.15.83:0/4019098517 learned_addr learned my addr 172.21.15.83:0/4019098517 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:48:51.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.069+0000 7fa085e00640 1 --2- 172.21.15.83:0/4019098517 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa080101f30 0x7fa0801981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:51.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.070+0000 7fa085400640 1 --2- 172.21.15.83:0/4019098517 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa080103080 0x7fa0801986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:51.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.070+0000 7fa086800640 1 -- 172.21.15.83:0/4019098517 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa080103080 msgr2=0x7fa0801986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:51.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.070+0000 7fa086800640 1 --2- 172.21.15.83:0/4019098517 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa080103080 0x7fa0801986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:51.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.070+0000 7fa086800640 1 -- 172.21.15.83:0/4019098517 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa080101f30 msgr2=0x7fa0801981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:51.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.070+0000 7fa086800640 1 --2- 172.21.15.83:0/4019098517 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa080101f30 0x7fa0801981a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:51.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.070+0000 7fa086800640 1 -- 172.21.15.83:0/4019098517 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa08019fd70 con 0x7fa080103a40 2024-04-24T08:48:51.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.070+0000 7fa086800640 1 --2- 172.21.15.83:0/4019098517 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa080103a40 0x7fa08019f700 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fa07c00eb10 tx=0x7fa07c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:48:51.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.070+0000 7fa07aa00640 1 -- 172.21.15.83:0/4019098517 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa07c004b20 con 0x7fa080103a40 2024-04-24T08:48:51.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.070+0000 7fa07aa00640 1 -- 172.21.15.83:0/4019098517 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa07c004cc0 con 0x7fa080103a40 2024-04-24T08:48:51.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.070+0000 7fa087200640 1 -- 172.21.15.83:0/4019098517 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0801a0060 con 0x7fa080103a40 2024-04-24T08:48:51.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.070+0000 7fa07aa00640 1 -- 172.21.15.83:0/4019098517 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa07c010690 con 0x7fa080103a40 2024-04-24T08:48:51.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.070+0000 7fa087200640 1 -- 172.21.15.83:0/4019098517 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0801a0560 con 0x7fa080103a40 2024-04-24T08:48:51.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.072+0000 7fa087200640 1 -- 172.21.15.83:0/4019098517 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa080109330 con 0x7fa080103a40 2024-04-24T08:48:51.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.074+0000 7fa07aa00640 1 -- 172.21.15.83:0/4019098517 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fa07c005020 con 0x7fa080103a40 2024-04-24T08:48:51.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.075+0000 7fa07aa00640 1 --2- 172.21.15.83:0/4019098517 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fa0680a3bb0 0x7fa0680a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:51.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.075+0000 7fa07aa00640 1 -- 172.21.15.83:0/4019098517 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fa07c0d0d00 con 0x7fa080103a40 2024-04-24T08:48:51.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.075+0000 7fa085e00640 1 --2- 172.21.15.83:0/4019098517 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fa0680a3bb0 0x7fa0680a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:51.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.076+0000 7fa085e00640 1 --2- 172.21.15.83:0/4019098517 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fa0680a3bb0 0x7fa0680a6070 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa074000c00 tx=0x7fa0740314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:48:51.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.080+0000 7fa07aa00640 1 -- 172.21.15.83:0/4019098517 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa07c09d440 con 0x7fa080103a40 2024-04-24T08:48:51.145 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:50 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/3327306564' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T08:48:51.145 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:50 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/3264861577' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T08:48:51.145 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:50 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/1683346346' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T08:48:51.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:50 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/3327306564' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T08:48:51.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:50 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/3264861577' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T08:48:51.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:50 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/1683346346' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T08:48:51.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:50 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/3327306564' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T08:48:51.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:50 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/3264861577' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T08:48:51.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:50 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/1683346346' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T08:48:51.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.282+0000 7fa087200640 1 -- 172.21.15.83:0/4019098517 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa080062c00 con 0x7fa080103a40 2024-04-24T08:48:51.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.283+0000 7fa07aa00640 1 -- 172.21.15.83:0/4019098517 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) ==== 100+0+11 (secure 0 0 0) 0x7fa07c09cd40 con 0x7fa080103a40 2024-04-24T08:48:51.282 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2024-04-24T08:48:51.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.286+0000 7fa087200640 1 -- 172.21.15.83:0/4019098517 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fa0680a3bb0 msgr2=0x7fa0680a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:51.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.286+0000 7fa087200640 1 --2- 172.21.15.83:0/4019098517 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fa0680a3bb0 0x7fa0680a6070 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa074000c00 tx=0x7fa0740314a0 comp rx=0 tx=0).stop 2024-04-24T08:48:51.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.286+0000 7fa087200640 1 -- 172.21.15.83:0/4019098517 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa080103a40 msgr2=0x7fa08019f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:51.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.286+0000 7fa087200640 1 --2- 172.21.15.83:0/4019098517 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa080103a40 0x7fa08019f700 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fa07c00eb10 tx=0x7fa07c00c2d0 comp rx=0 tx=0).stop 2024-04-24T08:48:51.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.286+0000 7fa087200640 1 -- 172.21.15.83:0/4019098517 shutdown_connections 2024-04-24T08:48:51.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.286+0000 7fa087200640 1 --2- 172.21.15.83:0/4019098517 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fa0680a3bb0 0x7fa0680a6070 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:51.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.286+0000 7fa087200640 1 --2- 172.21.15.83:0/4019098517 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa080103a40 0x7fa08019f700 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:51.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.286+0000 7fa087200640 1 --2- 172.21.15.83:0/4019098517 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa080103080 0x7fa0801986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:51.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.286+0000 7fa087200640 1 --2- 172.21.15.83:0/4019098517 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa080101f30 0x7fa0801981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:51.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.286+0000 7fa087200640 1 -- 172.21.15.83:0/4019098517 >> 172.21.15.83:0/4019098517 conn(0x7fa0800fd4a0 msgr2=0x7fa080106500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:48:51.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.286+0000 7fa087200640 1 -- 172.21.15.83:0/4019098517 shutdown_connections 2024-04-24T08:48:51.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.286+0000 7fa087200640 1 -- 172.21.15.83:0/4019098517 wait complete. 2024-04-24T08:48:51.306 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-24T08:48:51.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.525+0000 7f8b37200640 1 -- 172.21.15.83:0/1483372889 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b38105cf0 msgr2=0x7f8b3810c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:51.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.525+0000 7f8b37200640 1 --2- 172.21.15.83:0/1483372889 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b38105cf0 0x7f8b3810c580 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f8b30009ca0 tx=0x7f8b3002a8c0 comp rx=0 tx=0).stop 2024-04-24T08:48:51.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.525+0000 7f8b37200640 1 -- 172.21.15.83:0/1483372889 shutdown_connections 2024-04-24T08:48:51.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.525+0000 7f8b37200640 1 --2- 172.21.15.83:0/1483372889 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b38105cf0 0x7f8b3810c580 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:51.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.525+0000 7f8b37200640 1 --2- 172.21.15.83:0/1483372889 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8b38105330 0x7f8b381057b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:51.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.525+0000 7f8b37200640 1 --2- 172.21.15.83:0/1483372889 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b38104140 0x7f8b38104560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:51.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.525+0000 7f8b37200640 1 -- 172.21.15.83:0/1483372889 >> 172.21.15.83:0/1483372889 conn(0x7f8b380ff6b0 msgr2=0x7f8b38101af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:48:51.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.525+0000 7f8b37200640 1 -- 172.21.15.83:0/1483372889 shutdown_connections 2024-04-24T08:48:51.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.525+0000 7f8b37200640 1 -- 172.21.15.83:0/1483372889 wait complete. 2024-04-24T08:48:51.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.526+0000 7f8b37200640 1 Processor -- start 2024-04-24T08:48:51.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.526+0000 7f8b37200640 1 -- start start 2024-04-24T08:48:51.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.526+0000 7f8b37200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b38104140 0x7f8b3819a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:51.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.526+0000 7f8b37200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8b38105330 0x7f8b3819a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:51.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.526+0000 7f8b37200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b38105cf0 0x7f8b381a1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:51.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.526+0000 7f8b37200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8b3810fc80 con 0x7f8b38105cf0 2024-04-24T08:48:51.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.526+0000 7f8b37200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8b3810fb00 con 0x7f8b38104140 2024-04-24T08:48:51.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.526+0000 7f8b37200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8b3810fe00 con 0x7f8b38105330 2024-04-24T08:48:51.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.527+0000 7f8b36800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b38105cf0 0x7f8b381a1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:51.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.527+0000 7f8b36800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b38105cf0 0x7f8b381a1890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35424/0 (socket says 172.21.15.83:35424) 2024-04-24T08:48:51.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.527+0000 7f8b36800640 1 -- 172.21.15.83:0/3138346124 learned_addr learned my addr 172.21.15.83:0/3138346124 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:48:51.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.527+0000 7f8b35e00640 1 --2- 172.21.15.83:0/3138346124 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b38104140 0x7f8b3819a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:51.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.527+0000 7f8b2fe00640 1 --2- 172.21.15.83:0/3138346124 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8b38105330 0x7f8b3819a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:51.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.527+0000 7f8b36800640 1 -- 172.21.15.83:0/3138346124 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8b38105330 msgr2=0x7f8b3819a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:51.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.527+0000 7f8b36800640 1 --2- 172.21.15.83:0/3138346124 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8b38105330 0x7f8b3819a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:51.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.527+0000 7f8b36800640 1 -- 172.21.15.83:0/3138346124 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b38104140 msgr2=0x7f8b3819a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:51.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.527+0000 7f8b36800640 1 --2- 172.21.15.83:0/3138346124 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b38104140 0x7f8b3819a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:51.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.527+0000 7f8b36800640 1 -- 172.21.15.83:0/3138346124 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b381a1f00 con 0x7f8b38105cf0 2024-04-24T08:48:51.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.527+0000 7f8b36800640 1 --2- 172.21.15.83:0/3138346124 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b38105cf0 0x7f8b381a1890 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8b30031970 tx=0x7f8b30003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:48:51.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.528+0000 7f8b2ea00640 1 -- 172.21.15.83:0/3138346124 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b30003d90 con 0x7f8b38105cf0 2024-04-24T08:48:51.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.528+0000 7f8b2ea00640 1 -- 172.21.15.83:0/3138346124 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8b300044f0 con 0x7f8b38105cf0 2024-04-24T08:48:51.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.528+0000 7f8b37200640 1 -- 172.21.15.83:0/3138346124 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b381a2190 con 0x7f8b38105cf0 2024-04-24T08:48:51.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.528+0000 7f8b2ea00640 1 -- 172.21.15.83:0/3138346124 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b30030bd0 con 0x7f8b38105cf0 2024-04-24T08:48:51.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.528+0000 7f8b37200640 1 -- 172.21.15.83:0/3138346124 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b381a26e0 con 0x7f8b38105cf0 2024-04-24T08:48:51.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.529+0000 7f8b37200640 1 -- 172.21.15.83:0/3138346124 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b3810b460 con 0x7f8b38105cf0 2024-04-24T08:48:51.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.531+0000 7f8b2ea00640 1 -- 172.21.15.83:0/3138346124 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8b30004040 con 0x7f8b38105cf0 2024-04-24T08:48:51.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.532+0000 7f8b2ea00640 1 --2- 172.21.15.83:0/3138346124 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f8b1c0a3bb0 0x7f8b1c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:51.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.533+0000 7f8b2ea00640 1 -- 172.21.15.83:0/3138346124 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f8b300f1b30 con 0x7f8b38105cf0 2024-04-24T08:48:51.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.533+0000 7f8b35e00640 1 --2- 172.21.15.83:0/3138346124 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f8b1c0a3bb0 0x7f8b1c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:51.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.534+0000 7f8b35e00640 1 --2- 172.21.15.83:0/3138346124 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f8b1c0a3bb0 0x7f8b1c0a6070 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f8b38105190 tx=0x7f8b28005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:48:51.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.537+0000 7f8b2ea00640 1 -- 172.21.15.83:0/3138346124 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8b300be2d0 con 0x7f8b38105cf0 2024-04-24T08:48:51.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.741+0000 7f8b37200640 1 -- 172.21.15.83:0/3138346124 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f8b38062c00 con 0x7f8b38105cf0 2024-04-24T08:48:51.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.742+0000 7f8b2ea00640 1 -- 172.21.15.83:0/3138346124 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v89) ==== 103+0+11 (secure 0 0 0) 0x7f8b30030d70 con 0x7f8b38105cf0 2024-04-24T08:48:51.742 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-24T08:48:51.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.745+0000 7f8b37200640 1 -- 172.21.15.83:0/3138346124 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f8b1c0a3bb0 msgr2=0x7f8b1c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:51.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.745+0000 7f8b37200640 1 --2- 172.21.15.83:0/3138346124 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f8b1c0a3bb0 0x7f8b1c0a6070 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f8b38105190 tx=0x7f8b28005f90 comp rx=0 tx=0).stop 2024-04-24T08:48:51.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.745+0000 7f8b37200640 1 -- 172.21.15.83:0/3138346124 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b38105cf0 msgr2=0x7f8b381a1890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:51.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.745+0000 7f8b37200640 1 --2- 172.21.15.83:0/3138346124 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b38105cf0 0x7f8b381a1890 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8b30031970 tx=0x7f8b30003ba0 comp rx=0 tx=0).stop 2024-04-24T08:48:51.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.745+0000 7f8b37200640 1 -- 172.21.15.83:0/3138346124 shutdown_connections 2024-04-24T08:48:51.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.745+0000 7f8b37200640 1 --2- 172.21.15.83:0/3138346124 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f8b1c0a3bb0 0x7f8b1c0a6070 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:51.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.745+0000 7f8b37200640 1 --2- 172.21.15.83:0/3138346124 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b38105cf0 0x7f8b381a1890 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:51.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.745+0000 7f8b37200640 1 --2- 172.21.15.83:0/3138346124 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8b38105330 0x7f8b3819a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:51.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.745+0000 7f8b37200640 1 --2- 172.21.15.83:0/3138346124 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b38104140 0x7f8b3819a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:51.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.745+0000 7f8b37200640 1 -- 172.21.15.83:0/3138346124 >> 172.21.15.83:0/3138346124 conn(0x7f8b380ff6b0 msgr2=0x7f8b38108630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:48:51.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.745+0000 7f8b37200640 1 -- 172.21.15.83:0/3138346124 shutdown_connections 2024-04-24T08:48:51.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.745+0000 7f8b37200640 1 -- 172.21.15.83:0/3138346124 wait complete. 2024-04-24T08:48:51.762 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-24T08:48:51.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.961+0000 7f3c4a600640 1 -- 172.21.15.83:0/3115476546 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c44104260 msgr2=0x7f3c44104680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:51.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.961+0000 7f3c4a600640 1 --2- 172.21.15.83:0/3115476546 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c44104260 0x7f3c44104680 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f3c34009ca0 tx=0x7f3c3402a8c0 comp rx=0 tx=0).stop 2024-04-24T08:48:51.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.961+0000 7f3c4a600640 1 -- 172.21.15.83:0/3115476546 shutdown_connections 2024-04-24T08:48:51.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.961+0000 7f3c4a600640 1 --2- 172.21.15.83:0/3115476546 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3c44105e10 0x7f3c4410c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:51.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.961+0000 7f3c4a600640 1 --2- 172.21.15.83:0/3115476546 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c44105450 0x7f3c441058d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:51.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.961+0000 7f3c4a600640 1 --2- 172.21.15.83:0/3115476546 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c44104260 0x7f3c44104680 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:51.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.961+0000 7f3c4a600640 1 -- 172.21.15.83:0/3115476546 >> 172.21.15.83:0/3115476546 conn(0x7f3c440ff7b0 msgr2=0x7f3c44101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:48:51.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.961+0000 7f3c4a600640 1 -- 172.21.15.83:0/3115476546 shutdown_connections 2024-04-24T08:48:51.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.961+0000 7f3c4a600640 1 -- 172.21.15.83:0/3115476546 wait complete. 2024-04-24T08:48:51.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.962+0000 7f3c4a600640 1 Processor -- start 2024-04-24T08:48:51.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.962+0000 7f3c4a600640 1 -- start start 2024-04-24T08:48:51.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.962+0000 7f3c4a600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3c44104260 0x7f3c4419a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:51.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.962+0000 7f3c4a600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c44105450 0x7f3c4419a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:51.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.962+0000 7f3c4a600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c44105e10 0x7f3c441a1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:51.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.962+0000 7f3c4a600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3c4410fdf0 con 0x7f3c44105e10 2024-04-24T08:48:51.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.962+0000 7f3c4a600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3c4410fc70 con 0x7f3c44105450 2024-04-24T08:48:51.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.963+0000 7f3c4a600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3c4410ff70 con 0x7f3c44104260 2024-04-24T08:48:51.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.963+0000 7f3c43e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c44105e10 0x7f3c441a1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:51.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.963+0000 7f3c43e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c44105e10 0x7f3c441a1a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35432/0 (socket says 172.21.15.83:35432) 2024-04-24T08:48:51.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.963+0000 7f3c43e00640 1 -- 172.21.15.83:0/1944452560 learned_addr learned my addr 172.21.15.83:0/1944452560 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:48:51.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.963+0000 7f3c43400640 1 --2- 172.21.15.83:0/1944452560 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3c44104260 0x7f3c4419a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:51.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.963+0000 7f3c42a00640 1 --2- 172.21.15.83:0/1944452560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c44105450 0x7f3c4419a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:51.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.963+0000 7f3c43e00640 1 -- 172.21.15.83:0/1944452560 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3c44104260 msgr2=0x7f3c4419a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:51.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.963+0000 7f3c43e00640 1 --2- 172.21.15.83:0/1944452560 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3c44104260 0x7f3c4419a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:51.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.963+0000 7f3c43e00640 1 -- 172.21.15.83:0/1944452560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c44105450 msgr2=0x7f3c4419a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:51.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.963+0000 7f3c43e00640 1 --2- 172.21.15.83:0/1944452560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c44105450 0x7f3c4419a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:51.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.963+0000 7f3c43e00640 1 -- 172.21.15.83:0/1944452560 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c441a20c0 con 0x7f3c44105e10 2024-04-24T08:48:51.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.964+0000 7f3c43e00640 1 --2- 172.21.15.83:0/1944452560 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c44105e10 0x7f3c441a1a50 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c00d750 tx=0x7f3c3c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:48:51.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.964+0000 7f3c3be00640 1 -- 172.21.15.83:0/1944452560 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c3c017070 con 0x7f3c44105e10 2024-04-24T08:48:51.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.964+0000 7f3c3be00640 1 -- 172.21.15.83:0/1944452560 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3c3c0054d0 con 0x7f3c44105e10 2024-04-24T08:48:51.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.964+0000 7f3c4a600640 1 -- 172.21.15.83:0/1944452560 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c441a23b0 con 0x7f3c44105e10 2024-04-24T08:48:51.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.964+0000 7f3c3be00640 1 -- 172.21.15.83:0/1944452560 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c3c004c60 con 0x7f3c44105e10 2024-04-24T08:48:51.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.964+0000 7f3c4a600640 1 -- 172.21.15.83:0/1944452560 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c441a2950 con 0x7f3c44105e10 2024-04-24T08:48:51.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.965+0000 7f3c4a600640 1 -- 172.21.15.83:0/1944452560 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c4410b600 con 0x7f3c44105e10 2024-04-24T08:48:51.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.967+0000 7f3c3be00640 1 -- 172.21.15.83:0/1944452560 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f3c3c004e00 con 0x7f3c44105e10 2024-04-24T08:48:51.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.968+0000 7f3c3be00640 1 --2- 172.21.15.83:0/1944452560 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f3c280a3c30 0x7f3c280a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:51.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.969+0000 7f3c3be00640 1 -- 172.21.15.83:0/1944452560 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f3c3c0d0060 con 0x7f3c44105e10 2024-04-24T08:48:51.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.969+0000 7f3c43400640 1 --2- 172.21.15.83:0/1944452560 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f3c280a3c30 0x7f3c280a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:51.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.970+0000 7f3c43400640 1 --2- 172.21.15.83:0/1944452560 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f3c280a3c30 0x7f3c280a60f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3c34006010 tx=0x7f3c340093f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:48:51.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:51.973+0000 7f3c3be00640 1 -- 172.21.15.83:0/1944452560 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3c3c09c7a0 con 0x7f3c44105e10 2024-04-24T08:48:52.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:51 smithi083 ceph-mon[27907]: pgmap v384: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-24T08:48:52.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:51 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/4019098517' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T08:48:52.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:51 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/3138346124' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T08:48:52.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:52.189+0000 7f3c4a600640 1 -- 172.21.15.83:0/1944452560 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f3c44062c00 con 0x7f3c44105e10 2024-04-24T08:48:52.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:52.190+0000 7f3c3be00640 1 -- 172.21.15.83:0/1944452560 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v89) ==== 81+0+589 (secure 0 0 0) 0x7f3c3c09c0a0 con 0x7f3c44105e10 2024-04-24T08:48:52.190 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:6801/820049883 2024-04-25T08:38:40.661315+0000 2024-04-24T08:48:52.190 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:0/4272874138 2024-04-25T08:38:40.661315+0000 2024-04-24T08:48:52.190 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:0/4212994335 2024-04-25T08:38:40.661315+0000 2024-04-24T08:48:52.190 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:6801/1996371091 2024-04-25T08:38:05.800856+0000 2024-04-24T08:48:52.190 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:0/1820663683 2024-04-25T08:38:05.800856+0000 2024-04-24T08:48:52.191 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:0/3802897262 2024-04-25T08:38:05.800856+0000 2024-04-24T08:48:52.191 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:6800/1996371091 2024-04-25T08:38:05.800856+0000 2024-04-24T08:48:52.191 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:6800/820049883 2024-04-25T08:38:40.661315+0000 2024-04-24T08:48:52.191 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:0/1366125049 2024-04-25T08:38:05.800856+0000 2024-04-24T08:48:52.191 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:0/340452868 2024-04-25T08:38:40.661315+0000 2024-04-24T08:48:52.191 INFO:teuthology.orchestra.run.smithi083.stderr:listed 10 entries 2024-04-24T08:48:52.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:52.193+0000 7f3c4a600640 1 -- 172.21.15.83:0/1944452560 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f3c280a3c30 msgr2=0x7f3c280a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:52.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:52.193+0000 7f3c4a600640 1 --2- 172.21.15.83:0/1944452560 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f3c280a3c30 0x7f3c280a60f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3c34006010 tx=0x7f3c340093f0 comp rx=0 tx=0).stop 2024-04-24T08:48:52.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:52.193+0000 7f3c4a600640 1 -- 172.21.15.83:0/1944452560 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c44105e10 msgr2=0x7f3c441a1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:52.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:52.193+0000 7f3c4a600640 1 --2- 172.21.15.83:0/1944452560 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c44105e10 0x7f3c441a1a50 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c00d750 tx=0x7f3c3c00dc10 comp rx=0 tx=0).stop 2024-04-24T08:48:52.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:52.193+0000 7f3c4a600640 1 -- 172.21.15.83:0/1944452560 shutdown_connections 2024-04-24T08:48:52.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:52.193+0000 7f3c4a600640 1 --2- 172.21.15.83:0/1944452560 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f3c280a3c30 0x7f3c280a60f0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:52.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:52.193+0000 7f3c4a600640 1 --2- 172.21.15.83:0/1944452560 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c44105e10 0x7f3c441a1a50 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:52.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:52.193+0000 7f3c4a600640 1 --2- 172.21.15.83:0/1944452560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c44105450 0x7f3c4419a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:52.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:52.193+0000 7f3c4a600640 1 --2- 172.21.15.83:0/1944452560 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3c44104260 0x7f3c4419a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:52.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:52.193+0000 7f3c4a600640 1 -- 172.21.15.83:0/1944452560 >> 172.21.15.83:0/1944452560 conn(0x7f3c440ff7b0 msgr2=0x7f3c441087d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:48:52.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:52.193+0000 7f3c4a600640 1 -- 172.21.15.83:0/1944452560 shutdown_connections 2024-04-24T08:48:52.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:52.193+0000 7f3c4a600640 1 -- 172.21.15.83:0/1944452560 wait complete. 2024-04-24T08:48:52.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:51 smithi138 ceph-mon[31441]: pgmap v384: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-24T08:48:52.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:51 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/4019098517' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T08:48:52.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:51 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/3138346124' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T08:48:52.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:51 smithi189 ceph-mon[31791]: pgmap v384: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-24T08:48:52.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:51 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/4019098517' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T08:48:52.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:51 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/3138346124' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T08:48:52.212 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi083.front.sepia.ceph.com... 2024-04-24T08:48:52.212 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T08:48:52.212 DEBUG:teuthology.orchestra.run.smithi083:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-04-24T08:48:52.290 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi083.stderr:ceph-fuse[72178]: fuse finished with error 0 and tester_r 0 2024-04-24T08:48:52.295 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T08:48:53.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:52 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/1944452560' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-24T08:48:53.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:52 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/1944452560' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-24T08:48:53.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:52 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/1944452560' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-24T08:48:54.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:53 smithi083 ceph-mon[27907]: pgmap v385: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-24T08:48:54.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:53 smithi138 ceph-mon[31441]: pgmap v385: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-24T08:48:54.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:53 smithi189 ceph-mon[31791]: pgmap v385: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-24T08:48:56.261 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:55 smithi189 ceph-mon[31791]: pgmap v386: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-24T08:48:56.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:55 smithi083 ceph-mon[27907]: pgmap v386: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-24T08:48:56.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:55 smithi138 ceph-mon[31441]: pgmap v386: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-24T08:48:58.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:57 smithi189 ceph-mon[31791]: pgmap v387: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-24T08:48:58.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:57 smithi083 ceph-mon[27907]: pgmap v387: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-24T08:48:58.397 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi083.front.sepia.ceph.com 2024-04-24T08:48:58.397 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T08:48:58.397 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-04-24T08:48:58.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:57 smithi138 ceph-mon[31441]: pgmap v387: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-24T08:48:58.465 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T08:48:58.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:58.668+0000 7fe38ae00640 1 -- 172.21.15.83:0/1008358624 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe384104260 msgr2=0x7fe384104680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:58.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:58.668+0000 7fe38ae00640 1 --2- 172.21.15.83:0/1008358624 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe384104260 0x7fe384104680 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe36c009900 tx=0x7fe36c02a410 comp rx=0 tx=0).stop 2024-04-24T08:48:58.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:58.668+0000 7fe38ae00640 1 -- 172.21.15.83:0/1008358624 shutdown_connections 2024-04-24T08:48:58.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:58.668+0000 7fe38ae00640 1 --2- 172.21.15.83:0/1008358624 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe384105e10 0x7fe38410c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:58.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:58.668+0000 7fe38ae00640 1 --2- 172.21.15.83:0/1008358624 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe384105450 0x7fe3841058d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:58.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:58.668+0000 7fe38ae00640 1 --2- 172.21.15.83:0/1008358624 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe384104260 0x7fe384104680 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:58.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:58.668+0000 7fe38ae00640 1 -- 172.21.15.83:0/1008358624 >> 172.21.15.83:0/1008358624 conn(0x7fe3840ff7f0 msgr2=0x7fe384101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:48:58.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:58.668+0000 7fe38ae00640 1 -- 172.21.15.83:0/1008358624 shutdown_connections 2024-04-24T08:48:58.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:58.668+0000 7fe38ae00640 1 -- 172.21.15.83:0/1008358624 wait complete. 2024-04-24T08:48:58.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:58.669+0000 7fe38ae00640 1 Processor -- start 2024-04-24T08:48:58.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:58.670+0000 7fe38ae00640 1 -- start start 2024-04-24T08:48:58.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:58.670+0000 7fe38ae00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe384104260 0x7fe38419a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:58.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:58.670+0000 7fe38ae00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe384105450 0x7fe38419aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:58.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:58.670+0000 7fe38ae00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe384105e10 0x7fe3841a1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:58.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:58.670+0000 7fe38ae00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe38410fda0 con 0x7fe384104260 2024-04-24T08:48:58.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:58.670+0000 7fe38ae00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe38410fc20 con 0x7fe384105450 2024-04-24T08:48:58.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:58.670+0000 7fe38ae00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe38410ff20 con 0x7fe384105e10 2024-04-24T08:48:58.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:58.670+0000 7fe383e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe384104260 0x7fe38419a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:58.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:58.670+0000 7fe383e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe384104260 0x7fe38419a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35458/0 (socket says 172.21.15.83:35458) 2024-04-24T08:48:58.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:58.670+0000 7fe383e00640 1 -- 172.21.15.83:0/2571898907 learned_addr learned my addr 172.21.15.83:0/2571898907 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:48:58.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:58.670+0000 7fe383400640 1 --2- 172.21.15.83:0/2571898907 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe384105450 0x7fe38419aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:58.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:58.671+0000 7fe388e00640 1 --2- 172.21.15.83:0/2571898907 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe384105e10 0x7fe3841a1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:58.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:58.671+0000 7fe383e00640 1 -- 172.21.15.83:0/2571898907 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe384105e10 msgr2=0x7fe3841a1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:58.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:58.671+0000 7fe383e00640 1 --2- 172.21.15.83:0/2571898907 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe384105e10 0x7fe3841a1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:58.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:58.671+0000 7fe383e00640 1 -- 172.21.15.83:0/2571898907 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe384105450 msgr2=0x7fe38419aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:58.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:58.671+0000 7fe383e00640 1 --2- 172.21.15.83:0/2571898907 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe384105450 0x7fe38419aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:58.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:58.671+0000 7fe383e00640 1 -- 172.21.15.83:0/2571898907 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3841a2130 con 0x7fe384104260 2024-04-24T08:48:58.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:58.671+0000 7fe388e00640 1 --2- 172.21.15.83:0/2571898907 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe384105e10 0x7fe3841a1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:48:58.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:58.671+0000 7fe383e00640 1 --2- 172.21.15.83:0/2571898907 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe384104260 0x7fe38419a510 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe36c0314e0 tx=0x7fe36c0042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:48:58.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:58.671+0000 7fe380c00640 1 -- 172.21.15.83:0/2571898907 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe36c03f030 con 0x7fe384104260 2024-04-24T08:48:58.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:58.671+0000 7fe380c00640 1 -- 172.21.15.83:0/2571898907 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe36c003cb0 con 0x7fe384104260 2024-04-24T08:48:58.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:58.671+0000 7fe38ae00640 1 -- 172.21.15.83:0/2571898907 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3841a23c0 con 0x7fe384104260 2024-04-24T08:48:58.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:58.671+0000 7fe380c00640 1 -- 172.21.15.83:0/2571898907 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe36c035070 con 0x7fe384104260 2024-04-24T08:48:58.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:58.672+0000 7fe38ae00640 1 -- 172.21.15.83:0/2571898907 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3841a2900 con 0x7fe384104260 2024-04-24T08:48:58.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:58.673+0000 7fe38ae00640 1 -- 172.21.15.83:0/2571898907 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe38410b6f0 con 0x7fe384104260 2024-04-24T08:48:58.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:58.675+0000 7fe380c00640 1 -- 172.21.15.83:0/2571898907 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe36c042030 con 0x7fe384104260 2024-04-24T08:48:58.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:58.676+0000 7fe380c00640 1 --2- 172.21.15.83:0/2571898907 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fe3680a3c30 0x7fe3680a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:58.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:58.676+0000 7fe380c00640 1 -- 172.21.15.83:0/2571898907 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fe36c0f0ce0 con 0x7fe384104260 2024-04-24T08:48:58.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:58.677+0000 7fe383400640 1 --2- 172.21.15.83:0/2571898907 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fe3680a3c30 0x7fe3680a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:58.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:58.678+0000 7fe383400640 1 --2- 172.21.15.83:0/2571898907 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fe3680a3c30 0x7fe3680a60f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe38419ba20 tx=0x7fe374009870 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:48:58.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:58.682+0000 7fe380c00640 1 -- 172.21.15.83:0/2571898907 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe36c0bd420 con 0x7fe384104260 2024-04-24T08:48:58.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:58.886+0000 7fe38ae00640 1 -- 172.21.15.83:0/2571898907 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe384062c00 con 0x7fe384104260 2024-04-24T08:48:58.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:58.889+0000 7fe380c00640 1 -- 172.21.15.83:0/2571898907 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24127 (secure 0 0 0) 0x7fe36c0bcd20 con 0x7fe384104260 2024-04-24T08:48:58.890 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:48:58.890 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":89,"fsid":"bd3e039e-0215-11ef-bc93-c7b262605968","created":"2024-04-24T08:37:32.209944+0000","modified":"2024-04-24T08:48:34.752923+0000","last_up_change":"2024-04-24T08:47:17.966041+0000","last_in_change":"2024-04-24T08:46:53.835936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T08:41:56.834372+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T08:48:26.978311+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T08:48:28.352731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T08:48:32.275167+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2cd77f3a-19dd-4c6e-a199-1dcbd5faa679","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2132675555},{"type":"v1","addr":"172.21.15.83:6803","nonce":2132675555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2132675555},{"type":"v1","addr":"172.21.15.83:6805","nonce":2132675555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2132675555},{"type":"v1","addr":"172.21.15.83:6809","nonce":2132675555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2132675555},{"type":"v1","addr":"172.21.15.83:6807","nonce":2132675555}]},"public_addr":"172.21.15.83:6803/2132675555","cluster_addr":"172.21.15.83:6805/2132675555","heartbeat_back_addr":"172.21.15.83:6809/2132675555","heartbeat_front_addr":"172.21.15.83:6807/2132675555","state":["exists","up"]},{"osd":1,"uuid":"aba7f5df-2a4c-4c23-b502-d193605a5b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":3589158880},{"type":"v1","addr":"172.21.15.138:6801","nonce":3589158880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":3589158880},{"type":"v1","addr":"172.21.15.138:6803","nonce":3589158880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":3589158880},{"type":"v1","addr":"172.21.15.138:6807","nonce":3589158880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":3589158880},{"type":"v1","addr":"172.21.15.138:6805","nonce":3589158880}]},"public_addr":"172.21.15.138:6801/3589158880","cluster_addr":"172.21.15.138:6803/3589158880","heartbeat_back_addr":"172.21.15.138:6807/3589158880","heartbeat_front_addr":"172.21.15.138:6805/3589158880","state":["exists","up"]},{"osd":2,"uuid":"071bc167-2333-4420-bb58-bffbc10d7641","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":3169072688},{"type":"v1","addr":"172.21.15.189:6801","nonce":3169072688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":3169072688},{"type":"v1","addr":"172.21.15.189:6803","nonce":3169072688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":3169072688},{"type":"v1","addr":"172.21.15.189:6807","nonce":3169072688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":3169072688},{"type":"v1","addr":"172.21.15.189:6805","nonce":3169072688}]},"public_addr":"172.21.15.189:6801/3169072688","cluster_addr":"172.21.15.189:6803/3169072688","heartbeat_back_addr":"172.21.15.189:6807/3169072688","heartbeat_front_addr":"172.21.15.189:6805/3169072688","state":["exists","up"]},{"osd":3,"uuid":"f5624d78-4455-4e07-8c1a-f47c94cddb23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3131092450},{"type":"v1","addr":"172.21.15.83:6811","nonce":3131092450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3131092450},{"type":"v1","addr":"172.21.15.83:6813","nonce":3131092450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3131092450},{"type":"v1","addr":"172.21.15.83:6817","nonce":3131092450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3131092450},{"type":"v1","addr":"172.21.15.83:6815","nonce":3131092450}]},"public_addr":"172.21.15.83:6811/3131092450","cluster_addr":"172.21.15.83:6813/3131092450","heartbeat_back_addr":"172.21.15.83:6817/3131092450","heartbeat_front_addr":"172.21.15.83:6815/3131092450","state":["exists","up"]},{"osd":4,"uuid":"7b50cd14-9385-4535-a7b0-1c1da93c8548","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":823775360},{"type":"v1","addr":"172.21.15.138:6809","nonce":823775360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":823775360},{"type":"v1","addr":"172.21.15.138:6811","nonce":823775360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":823775360},{"type":"v1","addr":"172.21.15.138:6815","nonce":823775360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":823775360},{"type":"v1","addr":"172.21.15.138:6813","nonce":823775360}]},"public_addr":"172.21.15.138:6809/823775360","cluster_addr":"172.21.15.138:6811/823775360","heartbeat_back_addr":"172.21.15.138:6815/823775360","heartbeat_front_addr":"172.21.15.138:6813/823775360","state":["exists","up"]},{"osd":5,"uuid":"9c647797-35bc-449f-be84-af97d2c69c2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":1399817431},{"type":"v1","addr":"172.21.15.189:6809","nonce":1399817431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":1399817431},{"type":"v1","addr":"172.21.15.189:6811","nonce":1399817431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":1399817431},{"type":"v1","addr":"172.21.15.189:6815","nonce":1399817431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":1399817431},{"type":"v1","addr":"172.21.15.189:6813","nonce":1399817431}]},"public_addr":"172.21.15.189:6809/1399817431","cluster_addr":"172.21.15.189:6811/1399817431","heartbeat_back_addr":"172.21.15.189:6815/1399817431","heartbeat_front_addr":"172.21.15.189:6813/1399817431","state":["exists","up"]},{"osd":6,"uuid":"8f056584-16c8-4454-b445-3f780f26c196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":1611358483},{"type":"v1","addr":"172.21.15.83:6819","nonce":1611358483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1611358483},{"type":"v1","addr":"172.21.15.83:6821","nonce":1611358483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1611358483},{"type":"v1","addr":"172.21.15.83:6825","nonce":1611358483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":1611358483},{"type":"v1","addr":"172.21.15.83:6823","nonce":1611358483}]},"public_addr":"172.21.15.83:6819/1611358483","cluster_addr":"172.21.15.83:6821/1611358483","heartbeat_back_addr":"172.21.15.83:6825/1611358483","heartbeat_front_addr":"172.21.15.83:6823/1611358483","state":["exists","up"]},{"osd":7,"uuid":"7603efe3-da77-4b7f-9d03-29c4f293b6d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":3557856706},{"type":"v1","addr":"172.21.15.138:6817","nonce":3557856706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":3557856706},{"type":"v1","addr":"172.21.15.138:6819","nonce":3557856706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":3557856706},{"type":"v1","addr":"172.21.15.138:6823","nonce":3557856706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":3557856706},{"type":"v1","addr":"172.21.15.138:6821","nonce":3557856706}]},"public_addr":"172.21.15.138:6817/3557856706","cluster_addr":"172.21.15.138:6819/3557856706","heartbeat_back_addr":"172.21.15.138:6823/3557856706","heartbeat_front_addr":"172.21.15.138:6821/3557856706","state":["exists","up"]},{"osd":8,"uuid":"27d7d1b8-d6a0-417e-b8aa-596325aa1a77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":207018554},{"type":"v1","addr":"172.21.15.189:6817","nonce":207018554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":207018554},{"type":"v1","addr":"172.21.15.189:6819","nonce":207018554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":207018554},{"type":"v1","addr":"172.21.15.189:6823","nonce":207018554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":207018554},{"type":"v1","addr":"172.21.15.189:6821","nonce":207018554}]},"public_addr":"172.21.15.189:6817/207018554","cluster_addr":"172.21.15.189:6819/207018554","heartbeat_back_addr":"172.21.15.189:6823/207018554","heartbeat_front_addr":"172.21.15.189:6821/207018554","state":["exists","up"]},{"osd":9,"uuid":"baf19b98-881b-4d3a-9f72-8aa2dd73cad6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":519636751},{"type":"v1","addr":"172.21.15.83:6827","nonce":519636751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":519636751},{"type":"v1","addr":"172.21.15.83:6829","nonce":519636751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":519636751},{"type":"v1","addr":"172.21.15.83:6833","nonce":519636751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":519636751},{"type":"v1","addr":"172.21.15.83:6831","nonce":519636751}]},"public_addr":"172.21.15.83:6827/519636751","cluster_addr":"172.21.15.83:6829/519636751","heartbeat_back_addr":"172.21.15.83:6833/519636751","heartbeat_front_addr":"172.21.15.83:6831/519636751","state":["exists","up"]},{"osd":10,"uuid":"eb346717-5c07-47c8-95be-0e68fe449fdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1130872098},{"type":"v1","addr":"172.21.15.138:6825","nonce":1130872098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1130872098},{"type":"v1","addr":"172.21.15.138:6827","nonce":1130872098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1130872098},{"type":"v1","addr":"172.21.15.138:6831","nonce":1130872098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1130872098},{"type":"v1","addr":"172.21.15.138:6829","nonce":1130872098}]},"public_addr":"172.21.15.138:6825/1130872098","cluster_addr":"172.21.15.138:6827/1130872098","heartbeat_back_addr":"172.21.15.138:6831/1130872098","heartbeat_front_addr":"172.21.15.138:6829/1130872098","state":["exists","up"]},{"osd":11,"uuid":"8e37568e-981c-45b8-b2af-2f1b73898542","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":2998782690},{"type":"v1","addr":"172.21.15.189:6825","nonce":2998782690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":2998782690},{"type":"v1","addr":"172.21.15.189:6827","nonce":2998782690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":2998782690},{"type":"v1","addr":"172.21.15.189:6831","nonce":2998782690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":2998782690},{"type":"v1","addr":"172.21.15.189:6829","nonce":2998782690}]},"public_addr":"172.21.15.189:6825/2998782690","cluster_addr":"172.21.15.189:6827/2998782690","heartbeat_back_addr":"172.21.15.189:6831/2998782690","heartbeat_front_addr":"172.21.15.189:6829/2998782690","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:40:47.980103+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:41:21.970795+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:41:55.160426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:42:30.400475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:43:05.705663+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:43:40.112312+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:44:16.781289+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:44:52.151187+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:45:28.055944+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:46:04.969301+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:46:40.918306+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:47:16.886874+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:6801/820049883":"2024-04-25T08:38:40.661315+0000","172.21.15.83:0/4272874138":"2024-04-25T08:38:40.661315+0000","172.21.15.83:0/4212994335":"2024-04-25T08:38:40.661315+0000","172.21.15.83:6801/1996371091":"2024-04-25T08:38:05.800856+0000","172.21.15.83:0/1820663683":"2024-04-25T08:38:05.800856+0000","172.21.15.83:0/3802897262":"2024-04-25T08:38:05.800856+0000","172.21.15.83:6800/1996371091":"2024-04-25T08:38:05.800856+0000","172.21.15.83:6800/820049883":"2024-04-25T08:38:40.661315+0000","172.21.15.83:0/1366125049":"2024-04-25T08:38:05.800856+0000","172.21.15.83:0/340452868":"2024-04-25T08:38:40.661315+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T08:48:58.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:58.892+0000 7fe38ae00640 1 -- 172.21.15.83:0/2571898907 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fe3680a3c30 msgr2=0x7fe3680a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:58.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:58.892+0000 7fe38ae00640 1 --2- 172.21.15.83:0/2571898907 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fe3680a3c30 0x7fe3680a60f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe38419ba20 tx=0x7fe374009870 comp rx=0 tx=0).stop 2024-04-24T08:48:58.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:58.892+0000 7fe38ae00640 1 -- 172.21.15.83:0/2571898907 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe384104260 msgr2=0x7fe38419a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:58.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:58.892+0000 7fe38ae00640 1 --2- 172.21.15.83:0/2571898907 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe384104260 0x7fe38419a510 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe36c0314e0 tx=0x7fe36c0042e0 comp rx=0 tx=0).stop 2024-04-24T08:48:58.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:58.892+0000 7fe38ae00640 1 -- 172.21.15.83:0/2571898907 shutdown_connections 2024-04-24T08:48:58.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:58.892+0000 7fe38ae00640 1 --2- 172.21.15.83:0/2571898907 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe384105e10 0x7fe3841a1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:58.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:58.892+0000 7fe38ae00640 1 --2- 172.21.15.83:0/2571898907 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe384105450 0x7fe38419aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:58.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:58.892+0000 7fe38ae00640 1 --2- 172.21.15.83:0/2571898907 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fe3680a3c30 0x7fe3680a60f0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:58.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:58.892+0000 7fe38ae00640 1 --2- 172.21.15.83:0/2571898907 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe384104260 0x7fe38419a510 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:58.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:58.892+0000 7fe38ae00640 1 -- 172.21.15.83:0/2571898907 >> 172.21.15.83:0/2571898907 conn(0x7fe3840ff7f0 msgr2=0x7fe3841088c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:48:58.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:58.892+0000 7fe38ae00640 1 -- 172.21.15.83:0/2571898907 shutdown_connections 2024-04-24T08:48:58.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:58.892+0000 7fe38ae00640 1 -- 172.21.15.83:0/2571898907 wait complete. 2024-04-24T08:48:58.912 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T08:48:59.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:59.107+0000 7fe680200640 1 -- 172.21.15.83:0/1040419869 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6781008b0 msgr2=0x7fe678100cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:59.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:59.107+0000 7fe680200640 1 --2- 172.21.15.83:0/1040419869 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6781008b0 0x7fe678100cd0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fe668009900 tx=0x7fe66802a3f0 comp rx=0 tx=0).stop 2024-04-24T08:48:59.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:59.107+0000 7fe680200640 1 -- 172.21.15.83:0/1040419869 shutdown_connections 2024-04-24T08:48:59.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:59.107+0000 7fe680200640 1 --2- 172.21.15.83:0/1040419869 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe678101bb0 0x7fe678102030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:59.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:59.107+0000 7fe680200640 1 --2- 172.21.15.83:0/1040419869 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe678101210 0x7fe678101670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:59.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:59.107+0000 7fe680200640 1 --2- 172.21.15.83:0/1040419869 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6781008b0 0x7fe678100cd0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:59.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:59.107+0000 7fe680200640 1 -- 172.21.15.83:0/1040419869 >> 172.21.15.83:0/1040419869 conn(0x7fe6780fc4b0 msgr2=0x7fe6780fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:48:59.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:59.107+0000 7fe680200640 1 -- 172.21.15.83:0/1040419869 shutdown_connections 2024-04-24T08:48:59.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:59.108+0000 7fe680200640 1 -- 172.21.15.83:0/1040419869 wait complete. 2024-04-24T08:48:59.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:59.109+0000 7fe680200640 1 Processor -- start 2024-04-24T08:48:59.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:59.109+0000 7fe680200640 1 -- start start 2024-04-24T08:48:59.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:59.109+0000 7fe680200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe6781008b0 0x7fe67819a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:59.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:59.109+0000 7fe680200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe678101210 0x7fe67819a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:59.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:59.109+0000 7fe680200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe678101bb0 0x7fe6781a1a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:59.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:59.109+0000 7fe680200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe67810fd90 con 0x7fe678101210 2024-04-24T08:48:59.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:59.109+0000 7fe680200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe67810fc10 con 0x7fe678101bb0 2024-04-24T08:48:59.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:59.109+0000 7fe680200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe67810ff10 con 0x7fe6781008b0 2024-04-24T08:48:59.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:59.109+0000 7fe67ce00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe678101210 0x7fe67819a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:59.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:59.109+0000 7fe67ce00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe678101210 0x7fe67819a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57588/0 (socket says 172.21.15.83:57588) 2024-04-24T08:48:59.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:59.109+0000 7fe67ce00640 1 -- 172.21.15.83:0/3292975829 learned_addr learned my addr 172.21.15.83:0/3292975829 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:48:59.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:59.110+0000 7fe67d800640 1 --2- 172.21.15.83:0/3292975829 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe6781008b0 0x7fe67819a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:59.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:59.110+0000 7fe67e200640 1 --2- 172.21.15.83:0/3292975829 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe678101bb0 0x7fe6781a1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:59.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:59.110+0000 7fe67ce00640 1 -- 172.21.15.83:0/3292975829 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe6781008b0 msgr2=0x7fe67819a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:59.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:59.110+0000 7fe67ce00640 1 --2- 172.21.15.83:0/3292975829 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe6781008b0 0x7fe67819a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:59.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:59.110+0000 7fe67ce00640 1 -- 172.21.15.83:0/3292975829 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe678101bb0 msgr2=0x7fe6781a1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:59.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:59.110+0000 7fe67ce00640 1 --2- 172.21.15.83:0/3292975829 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe678101bb0 0x7fe6781a1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:59.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:59.110+0000 7fe67ce00640 1 -- 172.21.15.83:0/3292975829 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6781a2080 con 0x7fe678101210 2024-04-24T08:48:59.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:59.110+0000 7fe67e200640 1 --2- 172.21.15.83:0/3292975829 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe678101bb0 0x7fe6781a1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:48:59.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:59.110+0000 7fe67ce00640 1 --2- 172.21.15.83:0/3292975829 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe678101210 0x7fe67819a9a0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fe66400ea80 tx=0x7fe66400ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:48:59.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:59.110+0000 7fe676000640 1 -- 172.21.15.83:0/3292975829 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe66400ce40 con 0x7fe678101210 2024-04-24T08:48:59.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:59.110+0000 7fe676000640 1 -- 172.21.15.83:0/3292975829 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe664005020 con 0x7fe678101210 2024-04-24T08:48:59.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:59.111+0000 7fe676000640 1 -- 172.21.15.83:0/3292975829 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6640116d0 con 0x7fe678101210 2024-04-24T08:48:59.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:59.111+0000 7fe680200640 1 -- 172.21.15.83:0/3292975829 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6781a2370 con 0x7fe678101210 2024-04-24T08:48:59.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:59.111+0000 7fe680200640 1 -- 172.21.15.83:0/3292975829 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6781a28c0 con 0x7fe678101210 2024-04-24T08:48:59.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:59.112+0000 7fe680200640 1 -- 172.21.15.83:0/3292975829 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe67810b840 con 0x7fe678101210 2024-04-24T08:48:59.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:59.114+0000 7fe676000640 1 -- 172.21.15.83:0/3292975829 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe6640051d0 con 0x7fe678101210 2024-04-24T08:48:59.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:59.116+0000 7fe676000640 1 --2- 172.21.15.83:0/3292975829 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fe65c0a3c30 0x7fe65c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:59.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:59.116+0000 7fe676000640 1 -- 172.21.15.83:0/3292975829 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fe6640d09a0 con 0x7fe678101210 2024-04-24T08:48:59.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:59.116+0000 7fe67d800640 1 --2- 172.21.15.83:0/3292975829 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fe65c0a3c30 0x7fe65c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:59.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:59.117+0000 7fe67d800640 1 --2- 172.21.15.83:0/3292975829 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fe65c0a3c30 0x7fe65c0a60f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe668000c00 tx=0x7fe6680314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:48:59.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:59.120+0000 7fe676000640 1 -- 172.21.15.83:0/3292975829 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe664016030 con 0x7fe678101210 2024-04-24T08:48:59.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:58 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/2571898907' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T08:48:59.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:59.326+0000 7fe680200640 1 -- 172.21.15.83:0/3292975829 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe678062c00 con 0x7fe678101210 2024-04-24T08:48:59.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:59.328+0000 7fe676000640 1 -- 172.21.15.83:0/3292975829 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24127 (secure 0 0 0) 0x7fe66409d0e0 con 0x7fe678101210 2024-04-24T08:48:59.328 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:48:59.329 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":89,"fsid":"bd3e039e-0215-11ef-bc93-c7b262605968","created":"2024-04-24T08:37:32.209944+0000","modified":"2024-04-24T08:48:34.752923+0000","last_up_change":"2024-04-24T08:47:17.966041+0000","last_in_change":"2024-04-24T08:46:53.835936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T08:41:56.834372+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T08:48:26.978311+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T08:48:28.352731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T08:48:32.275167+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2cd77f3a-19dd-4c6e-a199-1dcbd5faa679","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2132675555},{"type":"v1","addr":"172.21.15.83:6803","nonce":2132675555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2132675555},{"type":"v1","addr":"172.21.15.83:6805","nonce":2132675555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2132675555},{"type":"v1","addr":"172.21.15.83:6809","nonce":2132675555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2132675555},{"type":"v1","addr":"172.21.15.83:6807","nonce":2132675555}]},"public_addr":"172.21.15.83:6803/2132675555","cluster_addr":"172.21.15.83:6805/2132675555","heartbeat_back_addr":"172.21.15.83:6809/2132675555","heartbeat_front_addr":"172.21.15.83:6807/2132675555","state":["exists","up"]},{"osd":1,"uuid":"aba7f5df-2a4c-4c23-b502-d193605a5b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":3589158880},{"type":"v1","addr":"172.21.15.138:6801","nonce":3589158880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":3589158880},{"type":"v1","addr":"172.21.15.138:6803","nonce":3589158880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":3589158880},{"type":"v1","addr":"172.21.15.138:6807","nonce":3589158880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":3589158880},{"type":"v1","addr":"172.21.15.138:6805","nonce":3589158880}]},"public_addr":"172.21.15.138:6801/3589158880","cluster_addr":"172.21.15.138:6803/3589158880","heartbeat_back_addr":"172.21.15.138:6807/3589158880","heartbeat_front_addr":"172.21.15.138:6805/3589158880","state":["exists","up"]},{"osd":2,"uuid":"071bc167-2333-4420-bb58-bffbc10d7641","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":3169072688},{"type":"v1","addr":"172.21.15.189:6801","nonce":3169072688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":3169072688},{"type":"v1","addr":"172.21.15.189:6803","nonce":3169072688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":3169072688},{"type":"v1","addr":"172.21.15.189:6807","nonce":3169072688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":3169072688},{"type":"v1","addr":"172.21.15.189:6805","nonce":3169072688}]},"public_addr":"172.21.15.189:6801/3169072688","cluster_addr":"172.21.15.189:6803/3169072688","heartbeat_back_addr":"172.21.15.189:6807/3169072688","heartbeat_front_addr":"172.21.15.189:6805/3169072688","state":["exists","up"]},{"osd":3,"uuid":"f5624d78-4455-4e07-8c1a-f47c94cddb23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3131092450},{"type":"v1","addr":"172.21.15.83:6811","nonce":3131092450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3131092450},{"type":"v1","addr":"172.21.15.83:6813","nonce":3131092450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3131092450},{"type":"v1","addr":"172.21.15.83:6817","nonce":3131092450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3131092450},{"type":"v1","addr":"172.21.15.83:6815","nonce":3131092450}]},"public_addr":"172.21.15.83:6811/3131092450","cluster_addr":"172.21.15.83:6813/3131092450","heartbeat_back_addr":"172.21.15.83:6817/3131092450","heartbeat_front_addr":"172.21.15.83:6815/3131092450","state":["exists","up"]},{"osd":4,"uuid":"7b50cd14-9385-4535-a7b0-1c1da93c8548","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":823775360},{"type":"v1","addr":"172.21.15.138:6809","nonce":823775360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":823775360},{"type":"v1","addr":"172.21.15.138:6811","nonce":823775360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":823775360},{"type":"v1","addr":"172.21.15.138:6815","nonce":823775360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":823775360},{"type":"v1","addr":"172.21.15.138:6813","nonce":823775360}]},"public_addr":"172.21.15.138:6809/823775360","cluster_addr":"172.21.15.138:6811/823775360","heartbeat_back_addr":"172.21.15.138:6815/823775360","heartbeat_front_addr":"172.21.15.138:6813/823775360","state":["exists","up"]},{"osd":5,"uuid":"9c647797-35bc-449f-be84-af97d2c69c2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":1399817431},{"type":"v1","addr":"172.21.15.189:6809","nonce":1399817431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":1399817431},{"type":"v1","addr":"172.21.15.189:6811","nonce":1399817431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":1399817431},{"type":"v1","addr":"172.21.15.189:6815","nonce":1399817431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":1399817431},{"type":"v1","addr":"172.21.15.189:6813","nonce":1399817431}]},"public_addr":"172.21.15.189:6809/1399817431","cluster_addr":"172.21.15.189:6811/1399817431","heartbeat_back_addr":"172.21.15.189:6815/1399817431","heartbeat_front_addr":"172.21.15.189:6813/1399817431","state":["exists","up"]},{"osd":6,"uuid":"8f056584-16c8-4454-b445-3f780f26c196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":1611358483},{"type":"v1","addr":"172.21.15.83:6819","nonce":1611358483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1611358483},{"type":"v1","addr":"172.21.15.83:6821","nonce":1611358483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1611358483},{"type":"v1","addr":"172.21.15.83:6825","nonce":1611358483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":1611358483},{"type":"v1","addr":"172.21.15.83:6823","nonce":1611358483}]},"public_addr":"172.21.15.83:6819/1611358483","cluster_addr":"172.21.15.83:6821/1611358483","heartbeat_back_addr":"172.21.15.83:6825/1611358483","heartbeat_front_addr":"172.21.15.83:6823/1611358483","state":["exists","up"]},{"osd":7,"uuid":"7603efe3-da77-4b7f-9d03-29c4f293b6d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":3557856706},{"type":"v1","addr":"172.21.15.138:6817","nonce":3557856706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":3557856706},{"type":"v1","addr":"172.21.15.138:6819","nonce":3557856706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":3557856706},{"type":"v1","addr":"172.21.15.138:6823","nonce":3557856706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":3557856706},{"type":"v1","addr":"172.21.15.138:6821","nonce":3557856706}]},"public_addr":"172.21.15.138:6817/3557856706","cluster_addr":"172.21.15.138:6819/3557856706","heartbeat_back_addr":"172.21.15.138:6823/3557856706","heartbeat_front_addr":"172.21.15.138:6821/3557856706","state":["exists","up"]},{"osd":8,"uuid":"27d7d1b8-d6a0-417e-b8aa-596325aa1a77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":207018554},{"type":"v1","addr":"172.21.15.189:6817","nonce":207018554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":207018554},{"type":"v1","addr":"172.21.15.189:6819","nonce":207018554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":207018554},{"type":"v1","addr":"172.21.15.189:6823","nonce":207018554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":207018554},{"type":"v1","addr":"172.21.15.189:6821","nonce":207018554}]},"public_addr":"172.21.15.189:6817/207018554","cluster_addr":"172.21.15.189:6819/207018554","heartbeat_back_addr":"172.21.15.189:6823/207018554","heartbeat_front_addr":"172.21.15.189:6821/207018554","state":["exists","up"]},{"osd":9,"uuid":"baf19b98-881b-4d3a-9f72-8aa2dd73cad6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":519636751},{"type":"v1","addr":"172.21.15.83:6827","nonce":519636751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":519636751},{"type":"v1","addr":"172.21.15.83:6829","nonce":519636751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":519636751},{"type":"v1","addr":"172.21.15.83:6833","nonce":519636751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":519636751},{"type":"v1","addr":"172.21.15.83:6831","nonce":519636751}]},"public_addr":"172.21.15.83:6827/519636751","cluster_addr":"172.21.15.83:6829/519636751","heartbeat_back_addr":"172.21.15.83:6833/519636751","heartbeat_front_addr":"172.21.15.83:6831/519636751","state":["exists","up"]},{"osd":10,"uuid":"eb346717-5c07-47c8-95be-0e68fe449fdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1130872098},{"type":"v1","addr":"172.21.15.138:6825","nonce":1130872098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1130872098},{"type":"v1","addr":"172.21.15.138:6827","nonce":1130872098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1130872098},{"type":"v1","addr":"172.21.15.138:6831","nonce":1130872098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1130872098},{"type":"v1","addr":"172.21.15.138:6829","nonce":1130872098}]},"public_addr":"172.21.15.138:6825/1130872098","cluster_addr":"172.21.15.138:6827/1130872098","heartbeat_back_addr":"172.21.15.138:6831/1130872098","heartbeat_front_addr":"172.21.15.138:6829/1130872098","state":["exists","up"]},{"osd":11,"uuid":"8e37568e-981c-45b8-b2af-2f1b73898542","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":2998782690},{"type":"v1","addr":"172.21.15.189:6825","nonce":2998782690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":2998782690},{"type":"v1","addr":"172.21.15.189:6827","nonce":2998782690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":2998782690},{"type":"v1","addr":"172.21.15.189:6831","nonce":2998782690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":2998782690},{"type":"v1","addr":"172.21.15.189:6829","nonce":2998782690}]},"public_addr":"172.21.15.189:6825/2998782690","cluster_addr":"172.21.15.189:6827/2998782690","heartbeat_back_addr":"172.21.15.189:6831/2998782690","heartbeat_front_addr":"172.21.15.189:6829/2998782690","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:40:47.980103+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:41:21.970795+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:41:55.160426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:42:30.400475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:43:05.705663+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:43:40.112312+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:44:16.781289+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:44:52.151187+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:45:28.055944+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:46:04.969301+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:46:40.918306+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:47:16.886874+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:6801/820049883":"2024-04-25T08:38:40.661315+0000","172.21.15.83:0/4272874138":"2024-04-25T08:38:40.661315+0000","172.21.15.83:0/4212994335":"2024-04-25T08:38:40.661315+0000","172.21.15.83:6801/1996371091":"2024-04-25T08:38:05.800856+0000","172.21.15.83:0/1820663683":"2024-04-25T08:38:05.800856+0000","172.21.15.83:0/3802897262":"2024-04-25T08:38:05.800856+0000","172.21.15.83:6800/1996371091":"2024-04-25T08:38:05.800856+0000","172.21.15.83:6800/820049883":"2024-04-25T08:38:40.661315+0000","172.21.15.83:0/1366125049":"2024-04-25T08:38:05.800856+0000","172.21.15.83:0/340452868":"2024-04-25T08:38:40.661315+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T08:48:59.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:59.331+0000 7fe680200640 1 -- 172.21.15.83:0/3292975829 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fe65c0a3c30 msgr2=0x7fe65c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:59.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:59.331+0000 7fe680200640 1 --2- 172.21.15.83:0/3292975829 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fe65c0a3c30 0x7fe65c0a60f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe668000c00 tx=0x7fe6680314a0 comp rx=0 tx=0).stop 2024-04-24T08:48:59.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:59.331+0000 7fe680200640 1 -- 172.21.15.83:0/3292975829 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe678101210 msgr2=0x7fe67819a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:59.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:59.331+0000 7fe680200640 1 --2- 172.21.15.83:0/3292975829 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe678101210 0x7fe67819a9a0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fe66400ea80 tx=0x7fe66400ef40 comp rx=0 tx=0).stop 2024-04-24T08:48:59.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:59.331+0000 7fe680200640 1 -- 172.21.15.83:0/3292975829 shutdown_connections 2024-04-24T08:48:59.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:59.331+0000 7fe680200640 1 --2- 172.21.15.83:0/3292975829 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe678101bb0 0x7fe6781a1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:59.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:59.331+0000 7fe680200640 1 --2- 172.21.15.83:0/3292975829 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fe65c0a3c30 0x7fe65c0a60f0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:59.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:59.331+0000 7fe680200640 1 --2- 172.21.15.83:0/3292975829 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe678101210 0x7fe67819a9a0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:59.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:59.331+0000 7fe680200640 1 --2- 172.21.15.83:0/3292975829 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe6781008b0 0x7fe67819a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:59.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:59.331+0000 7fe680200640 1 -- 172.21.15.83:0/3292975829 >> 172.21.15.83:0/3292975829 conn(0x7fe6780fc4b0 msgr2=0x7fe6780fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:48:59.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:59.331+0000 7fe680200640 1 -- 172.21.15.83:0/3292975829 shutdown_connections 2024-04-24T08:48:59.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:59.331+0000 7fe680200640 1 -- 172.21.15.83:0/3292975829 wait complete. 2024-04-24T08:48:59.350 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-24T08:48:59.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:58 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/2571898907' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T08:48:59.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:58 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/2571898907' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T08:48:59.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:59.563+0000 7f640b200640 1 -- 172.21.15.83:0/1610612528 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f640c105330 msgr2=0x7f640c1057b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:59.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:59.563+0000 7f640b200640 1 --2- 172.21.15.83:0/1610612528 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f640c105330 0x7f640c1057b0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f63f4009ca0 tx=0x7f63f402a8c0 comp rx=0 tx=0).stop 2024-04-24T08:48:59.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:59.564+0000 7f640b200640 1 -- 172.21.15.83:0/1610612528 shutdown_connections 2024-04-24T08:48:59.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:59.564+0000 7f640b200640 1 --2- 172.21.15.83:0/1610612528 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f640c105cf0 0x7f640c10c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:59.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:59.564+0000 7f640b200640 1 --2- 172.21.15.83:0/1610612528 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f640c105330 0x7f640c1057b0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:59.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:59.564+0000 7f640b200640 1 --2- 172.21.15.83:0/1610612528 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f640c104140 0x7f640c104560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:59.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:59.564+0000 7f640b200640 1 -- 172.21.15.83:0/1610612528 >> 172.21.15.83:0/1610612528 conn(0x7f640c0ff6b0 msgr2=0x7f640c101af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:48:59.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:59.564+0000 7f640b200640 1 -- 172.21.15.83:0/1610612528 shutdown_connections 2024-04-24T08:48:59.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:59.564+0000 7f640b200640 1 -- 172.21.15.83:0/1610612528 wait complete. 2024-04-24T08:48:59.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:59.565+0000 7f640b200640 1 Processor -- start 2024-04-24T08:48:59.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:59.565+0000 7f640b200640 1 -- start start 2024-04-24T08:48:59.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:59.565+0000 7f640b200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f640c104140 0x7f640c19a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:59.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:59.565+0000 7f640b200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f640c105330 0x7f640c19a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:59.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:59.565+0000 7f640b200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f640c105cf0 0x7f640c1a18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:59.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:59.565+0000 7f640b200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f640c10fc80 con 0x7f640c105cf0 2024-04-24T08:48:59.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:59.565+0000 7f640b200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f640c10fb00 con 0x7f640c104140 2024-04-24T08:48:59.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:59.565+0000 7f640b200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f640c10fe00 con 0x7f640c105330 2024-04-24T08:48:59.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:59.566+0000 7f640a800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f640c105cf0 0x7f640c1a18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:59.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:59.566+0000 7f640a800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f640c105cf0 0x7f640c1a18a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57618/0 (socket says 172.21.15.83:57618) 2024-04-24T08:48:59.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:59.566+0000 7f640a800640 1 -- 172.21.15.83:0/1229053139 learned_addr learned my addr 172.21.15.83:0/1229053139 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:48:59.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:59.566+0000 7f6409400640 1 --2- 172.21.15.83:0/1229053139 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f640c105330 0x7f640c19a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:59.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:59.566+0000 7f6409e00640 1 --2- 172.21.15.83:0/1229053139 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f640c104140 0x7f640c19a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:59.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:59.566+0000 7f640a800640 1 -- 172.21.15.83:0/1229053139 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f640c105330 msgr2=0x7f640c19a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:59.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:59.566+0000 7f640a800640 1 --2- 172.21.15.83:0/1229053139 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f640c105330 0x7f640c19a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:59.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:59.566+0000 7f640a800640 1 -- 172.21.15.83:0/1229053139 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f640c104140 msgr2=0x7f640c19a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:59.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:59.566+0000 7f640a800640 1 --2- 172.21.15.83:0/1229053139 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f640c104140 0x7f640c19a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:59.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:59.566+0000 7f640a800640 1 -- 172.21.15.83:0/1229053139 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f640c1a1f10 con 0x7f640c105cf0 2024-04-24T08:48:59.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:59.567+0000 7f640a800640 1 --2- 172.21.15.83:0/1229053139 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f640c105cf0 0x7f640c1a18a0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f640400c970 tx=0x7f640400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:48:59.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:59.567+0000 7f6402a00640 1 -- 172.21.15.83:0/1229053139 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6404007bb0 con 0x7f640c105cf0 2024-04-24T08:48:59.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:59.567+0000 7f6402a00640 1 -- 172.21.15.83:0/1229053139 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6404007d50 con 0x7f640c105cf0 2024-04-24T08:48:59.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:59.567+0000 7f640b200640 1 -- 172.21.15.83:0/1229053139 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f640c1a2200 con 0x7f640c105cf0 2024-04-24T08:48:59.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:59.567+0000 7f6402a00640 1 -- 172.21.15.83:0/1229053139 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6404018700 con 0x7f640c105cf0 2024-04-24T08:48:59.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:59.567+0000 7f640b200640 1 -- 172.21.15.83:0/1229053139 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f640c1a2700 con 0x7f640c105cf0 2024-04-24T08:48:59.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:59.568+0000 7f640b200640 1 -- 172.21.15.83:0/1229053139 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f640c10b4e0 con 0x7f640c105cf0 2024-04-24T08:48:59.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:59.570+0000 7f6402a00640 1 -- 172.21.15.83:0/1229053139 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f640400fa20 con 0x7f640c105cf0 2024-04-24T08:48:59.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:59.571+0000 7f6402a00640 1 --2- 172.21.15.83:0/1229053139 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f63ec0a3bb0 0x7f63ec0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:48:59.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:59.572+0000 7f6402a00640 1 -- 172.21.15.83:0/1229053139 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f64040d0840 con 0x7f640c105cf0 2024-04-24T08:48:59.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:59.572+0000 7f6409e00640 1 --2- 172.21.15.83:0/1229053139 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f63ec0a3bb0 0x7f63ec0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:48:59.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:59.573+0000 7f6409e00640 1 --2- 172.21.15.83:0/1229053139 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f63ec0a3bb0 0x7f63ec0a6070 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f63fc005fd0 tx=0x7f63fc005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:48:59.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:59.577+0000 7f6402a00640 1 -- 172.21.15.83:0/1229053139 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6404014030 con 0x7f640c105cf0 2024-04-24T08:48:59.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:59.825+0000 7f640b200640 1 -- 172.21.15.83:0/1229053139 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f640c062c00 con 0x7f640c105cf0 2024-04-24T08:48:59.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:59.984+0000 7f6402a00640 1 -- 172.21.15.83:0/1229053139 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) ==== 118+0+0 (secure 0 0 0) 0x7f640409cf80 con 0x7f640c105cf0 2024-04-24T08:48:59.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:59.989+0000 7f640b200640 1 -- 172.21.15.83:0/1229053139 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f63ec0a3bb0 msgr2=0x7f63ec0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:59.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:59.989+0000 7f640b200640 1 --2- 172.21.15.83:0/1229053139 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f63ec0a3bb0 0x7f63ec0a6070 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f63fc005fd0 tx=0x7f63fc005850 comp rx=0 tx=0).stop 2024-04-24T08:48:59.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:59.989+0000 7f640b200640 1 -- 172.21.15.83:0/1229053139 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f640c105cf0 msgr2=0x7f640c1a18a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:48:59.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:59.989+0000 7f640b200640 1 --2- 172.21.15.83:0/1229053139 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f640c105cf0 0x7f640c1a18a0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f640400c970 tx=0x7f640400ce30 comp rx=0 tx=0).stop 2024-04-24T08:48:59.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:59.990+0000 7f640b200640 1 -- 172.21.15.83:0/1229053139 shutdown_connections 2024-04-24T08:48:59.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:59.990+0000 7f640b200640 1 --2- 172.21.15.83:0/1229053139 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f63ec0a3bb0 0x7f63ec0a6070 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:59.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:59.990+0000 7f640b200640 1 --2- 172.21.15.83:0/1229053139 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f640c105cf0 0x7f640c1a18a0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:59.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:59.990+0000 7f640b200640 1 --2- 172.21.15.83:0/1229053139 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f640c105330 0x7f640c19a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:59.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:59.990+0000 7f640b200640 1 --2- 172.21.15.83:0/1229053139 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f640c104140 0x7f640c19a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:48:59.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:59.990+0000 7f640b200640 1 -- 172.21.15.83:0/1229053139 >> 172.21.15.83:0/1229053139 conn(0x7f640c0ff6b0 msgr2=0x7f640c1086b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:48:59.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:59.990+0000 7f640b200640 1 -- 172.21.15.83:0/1229053139 shutdown_connections 2024-04-24T08:48:59.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:48:59.990+0000 7f640b200640 1 -- 172.21.15.83:0/1229053139 wait complete. 2024-04-24T08:49:00.014 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2}} 2024-04-24T08:49:00.014 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5 2024-04-24T08:49:00.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:00.222+0000 7ff605600640 1 -- 172.21.15.83:0/3821741078 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff600101340 msgr2=0x7ff6001017c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:00.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:00.222+0000 7ff605600640 1 --2- 172.21.15.83:0/3821741078 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff600101340 0x7ff6001017c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7ff5e8009370 tx=0x7ff5e802a410 comp rx=0 tx=0).stop 2024-04-24T08:49:00.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:00.223+0000 7ff605600640 1 -- 172.21.15.83:0/3821741078 shutdown_connections 2024-04-24T08:49:00.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:00.223+0000 7ff605600640 1 --2- 172.21.15.83:0/3821741078 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff600101d00 0x7ff600108590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:00.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:00.223+0000 7ff605600640 1 --2- 172.21.15.83:0/3821741078 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff600101340 0x7ff6001017c0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:00.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:00.223+0000 7ff605600640 1 --2- 172.21.15.83:0/3821741078 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff600100150 0x7ff600100570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:00.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:00.223+0000 7ff605600640 1 -- 172.21.15.83:0/3821741078 >> 172.21.15.83:0/3821741078 conn(0x7ff6000fb6c0 msgr2=0x7ff6000fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:49:00.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:00.223+0000 7ff605600640 1 -- 172.21.15.83:0/3821741078 shutdown_connections 2024-04-24T08:49:00.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:00.223+0000 7ff605600640 1 -- 172.21.15.83:0/3821741078 wait complete. 2024-04-24T08:49:00.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:00.224+0000 7ff605600640 1 Processor -- start 2024-04-24T08:49:00.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:00.224+0000 7ff605600640 1 -- start start 2024-04-24T08:49:00.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:00.224+0000 7ff605600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff600100150 0x7ff60019a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:00.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:00.224+0000 7ff605600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff600101340 0x7ff60019a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:00.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:00.224+0000 7ff605600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff600101d00 0x7ff6001a1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:00.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:00.224+0000 7ff605600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff60010b850 con 0x7ff600100150 2024-04-24T08:49:00.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:00.224+0000 7ff605600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff60010b6d0 con 0x7ff600101340 2024-04-24T08:49:00.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:00.224+0000 7ff605600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff60010b9d0 con 0x7ff600101d00 2024-04-24T08:49:00.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:00.225+0000 7ff5ffe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff600100150 0x7ff60019a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:00.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:00.225+0000 7ff5ffe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff600100150 0x7ff60019a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57648/0 (socket says 172.21.15.83:57648) 2024-04-24T08:49:00.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:00.225+0000 7ff5ffe00640 1 -- 172.21.15.83:0/2693167363 learned_addr learned my addr 172.21.15.83:0/2693167363 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:49:00.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:00.225+0000 7ff5ffe00640 1 -- 172.21.15.83:0/2693167363 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff600101d00 msgr2=0x7ff6001a1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:00.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:00.225+0000 7ff5ffe00640 1 --2- 172.21.15.83:0/2693167363 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff600101d00 0x7ff6001a1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:00.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:00.225+0000 7ff5ffe00640 1 -- 172.21.15.83:0/2693167363 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff600101340 msgr2=0x7ff60019a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:00.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:00.225+0000 7ff5ffe00640 1 --2- 172.21.15.83:0/2693167363 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff600101340 0x7ff60019a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:00.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:00.225+0000 7ff5ffe00640 1 -- 172.21.15.83:0/2693167363 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6001a1fa0 con 0x7ff600100150 2024-04-24T08:49:00.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:00.225+0000 7ff5ffe00640 1 --2- 172.21.15.83:0/2693167363 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff600100150 0x7ff60019a380 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7ff5f000dbc0 tx=0x7ff5f000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:49:00.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:00.225+0000 7ff5fd600640 1 -- 172.21.15.83:0/2693167363 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f0004220 con 0x7ff600100150 2024-04-24T08:49:00.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:00.226+0000 7ff5fd600640 1 -- 172.21.15.83:0/2693167363 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff5f000ae50 con 0x7ff600100150 2024-04-24T08:49:00.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:00.226+0000 7ff5fd600640 1 -- 172.21.15.83:0/2693167363 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f000e7d0 con 0x7ff600100150 2024-04-24T08:49:00.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:00.226+0000 7ff605600640 1 -- 172.21.15.83:0/2693167363 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6001a2290 con 0x7ff600100150 2024-04-24T08:49:00.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:00.226+0000 7ff605600640 1 -- 172.21.15.83:0/2693167363 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6001a27e0 con 0x7ff600100150 2024-04-24T08:49:00.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:00.227+0000 7ff605600640 1 -- 172.21.15.83:0/2693167363 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6001076f0 con 0x7ff600100150 2024-04-24T08:49:00.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:00.229+0000 7ff5fd600640 1 -- 172.21.15.83:0/2693167363 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ff5f000e970 con 0x7ff600100150 2024-04-24T08:49:00.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:00.230+0000 7ff5fd600640 1 --2- 172.21.15.83:0/2693167363 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7ff5e40a3bb0 0x7ff5e40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:00.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:00.231+0000 7ff5fd600640 1 -- 172.21.15.83:0/2693167363 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7ff5f00d0b10 con 0x7ff600100150 2024-04-24T08:49:00.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:00.231+0000 7ff5f7e00640 1 --2- 172.21.15.83:0/2693167363 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7ff5e40a3bb0 0x7ff5e40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:00.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:00.231+0000 7ff5f7e00640 1 --2- 172.21.15.83:0/2693167363 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7ff5e40a3bb0 0x7ff5e40a6070 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff5e8009f50 tx=0x7ff5e80314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:49:00.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:00.233+0000 7ff5fd600640 1 -- 172.21.15.83:0/2693167363 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff5f009d2b0 con 0x7ff600100150 2024-04-24T08:49:00.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:59 smithi083 ceph-mon[27907]: pgmap v388: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-24T08:49:00.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:59 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/3292975829' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T08:49:00.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:48:59 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/1229053139' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-24T08:49:00.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:59 smithi138 ceph-mon[31441]: pgmap v388: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-24T08:49:00.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:59 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/3292975829' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T08:49:00.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:48:59 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/1229053139' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-24T08:49:00.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:59 smithi189 ceph-mon[31791]: pgmap v388: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-24T08:49:00.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:59 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/3292975829' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T08:49:00.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:48:59 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/1229053139' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-24T08:49:00.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:00.478+0000 7ff605600640 1 -- 172.21.15.83:0/2693167363 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7ff60002d620 con 0x7ff600100150 2024-04-24T08:49:01.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:01.008+0000 7ff5fd600640 1 -- 172.21.15.83:0/2693167363 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v19) ==== 105+0+0 (secure 0 0 0) 0x7ff5f009cbb0 con 0x7ff600100150 2024-04-24T08:49:01.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:01.013+0000 7ff605600640 1 -- 172.21.15.83:0/2693167363 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7ff5e40a3bb0 msgr2=0x7ff5e40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:01.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:01.013+0000 7ff605600640 1 --2- 172.21.15.83:0/2693167363 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7ff5e40a3bb0 0x7ff5e40a6070 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff5e8009f50 tx=0x7ff5e80314a0 comp rx=0 tx=0).stop 2024-04-24T08:49:01.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:01.013+0000 7ff605600640 1 -- 172.21.15.83:0/2693167363 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff600100150 msgr2=0x7ff60019a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:01.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:01.013+0000 7ff605600640 1 --2- 172.21.15.83:0/2693167363 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff600100150 0x7ff60019a380 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7ff5f000dbc0 tx=0x7ff5f000a620 comp rx=0 tx=0).stop 2024-04-24T08:49:01.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:01.013+0000 7ff605600640 1 -- 172.21.15.83:0/2693167363 shutdown_connections 2024-04-24T08:49:01.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:01.013+0000 7ff605600640 1 --2- 172.21.15.83:0/2693167363 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff600101d00 0x7ff6001a1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:01.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:01.013+0000 7ff605600640 1 --2- 172.21.15.83:0/2693167363 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff600101340 0x7ff60019a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:01.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:01.013+0000 7ff605600640 1 --2- 172.21.15.83:0/2693167363 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7ff5e40a3bb0 0x7ff5e40a6070 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:01.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:01.013+0000 7ff605600640 1 --2- 172.21.15.83:0/2693167363 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff600100150 0x7ff60019a380 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:01.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:01.013+0000 7ff605600640 1 -- 172.21.15.83:0/2693167363 >> 172.21.15.83:0/2693167363 conn(0x7ff6000fb6c0 msgr2=0x7ff6001048c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:49:01.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:01.014+0000 7ff605600640 1 -- 172.21.15.83:0/2693167363 shutdown_connections 2024-04-24T08:49:01.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:01.014+0000 7ff605600640 1 -- 172.21.15.83:0/2693167363 wait complete. 2024-04-24T08:49:01.036 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-04-24T08:49:01.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:01.227+0000 7fe200e00640 1 -- 172.21.15.83:0/394771634 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe1fc107bf0 msgr2=0x7fe1fc10a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:01.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:01.227+0000 7fe200e00640 1 --2- 172.21.15.83:0/394771634 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe1fc107bf0 0x7fe1fc10a040 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe1f4009be0 tx=0x7fe1f402a6c0 comp rx=0 tx=0).stop 2024-04-24T08:49:01.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:01.228+0000 7fe200e00640 1 -- 172.21.15.83:0/394771634 shutdown_connections 2024-04-24T08:49:01.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:01.228+0000 7fe200e00640 1 --2- 172.21.15.83:0/394771634 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe1fc107bf0 0x7fe1fc10a040 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:01.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:01.228+0000 7fe200e00640 1 --2- 172.21.15.83:0/394771634 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1fc1052c0 0x7fe1fc1076b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:01.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:01.228+0000 7fe200e00640 1 --2- 172.21.15.83:0/394771634 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1fc102990 0x7fe1fc104d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:01.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:01.228+0000 7fe200e00640 1 -- 172.21.15.83:0/394771634 >> 172.21.15.83:0/394771634 conn(0x7fe1fc0fc3a0 msgr2=0x7fe1fc0fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:49:01.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:01.228+0000 7fe200e00640 1 -- 172.21.15.83:0/394771634 shutdown_connections 2024-04-24T08:49:01.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:01.228+0000 7fe200e00640 1 -- 172.21.15.83:0/394771634 wait complete. 2024-04-24T08:49:01.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:01.229+0000 7fe200e00640 1 Processor -- start 2024-04-24T08:49:01.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:01.229+0000 7fe200e00640 1 -- start start 2024-04-24T08:49:01.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:01.229+0000 7fe200e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe1fc102990 0x7fe1fc19a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:01.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:01.229+0000 7fe200e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1fc1052c0 0x7fe1fc19a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:01.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:01.229+0000 7fe200e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1fc107bf0 0x7fe1fc1a18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:01.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:01.229+0000 7fe200e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe1fc10da50 con 0x7fe1fc1052c0 2024-04-24T08:49:01.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:01.229+0000 7fe200e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe1fc10d8d0 con 0x7fe1fc107bf0 2024-04-24T08:49:01.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:01.229+0000 7fe200e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe1fc10dbd0 con 0x7fe1fc102990 2024-04-24T08:49:01.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:01.229+0000 7fe1faa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1fc1052c0 0x7fe1fc19a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:01.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:01.230+0000 7fe1faa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1fc1052c0 0x7fe1fc19a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57656/0 (socket says 172.21.15.83:57656) 2024-04-24T08:49:01.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:01.230+0000 7fe1faa00640 1 -- 172.21.15.83:0/2412816474 learned_addr learned my addr 172.21.15.83:0/2412816474 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:49:01.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:01.230+0000 7fe1fbe00640 1 --2- 172.21.15.83:0/2412816474 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1fc107bf0 0x7fe1fc1a18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:01.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:01.230+0000 7fe1faa00640 1 -- 172.21.15.83:0/2412816474 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe1fc102990 msgr2=0x7fe1fc19a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:01.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:01.230+0000 7fe1faa00640 1 --2- 172.21.15.83:0/2412816474 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe1fc102990 0x7fe1fc19a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:01.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:01.230+0000 7fe1faa00640 1 -- 172.21.15.83:0/2412816474 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1fc107bf0 msgr2=0x7fe1fc1a18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:01.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:01.230+0000 7fe1faa00640 1 --2- 172.21.15.83:0/2412816474 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1fc107bf0 0x7fe1fc1a18b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:01.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:01.230+0000 7fe1faa00640 1 -- 172.21.15.83:0/2412816474 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1fc1a1f20 con 0x7fe1fc1052c0 2024-04-24T08:49:01.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:01.230+0000 7fe1fbe00640 1 --2- 172.21.15.83:0/2412816474 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1fc107bf0 0x7fe1fc1a18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:49:01.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:01.230+0000 7fe1faa00640 1 --2- 172.21.15.83:0/2412816474 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1fc1052c0 0x7fe1fc19a840 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fe1ec00e9a0 tx=0x7fe1ec00ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:49:01.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:01.231+0000 7fe1f3e00640 1 -- 172.21.15.83:0/2412816474 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1ec00cd80 con 0x7fe1fc1052c0 2024-04-24T08:49:01.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:01.231+0000 7fe1f3e00640 1 -- 172.21.15.83:0/2412816474 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe1ec0035f0 con 0x7fe1fc1052c0 2024-04-24T08:49:01.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:01.231+0000 7fe1f3e00640 1 -- 172.21.15.83:0/2412816474 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1ec010690 con 0x7fe1fc1052c0 2024-04-24T08:49:01.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:01.231+0000 7fe200e00640 1 -- 172.21.15.83:0/2412816474 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1fc1a2210 con 0x7fe1fc1052c0 2024-04-24T08:49:01.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:01.231+0000 7fe200e00640 1 -- 172.21.15.83:0/2412816474 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1fc1a2710 con 0x7fe1fc1052c0 2024-04-24T08:49:01.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:01.232+0000 7fe200e00640 1 -- 172.21.15.83:0/2412816474 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1fc1096e0 con 0x7fe1fc1052c0 2024-04-24T08:49:01.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:01.236+0000 7fe1f3e00640 1 -- 172.21.15.83:0/2412816474 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe1ec0037a0 con 0x7fe1fc1052c0 2024-04-24T08:49:01.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:01.237+0000 7fe1f3e00640 1 --2- 172.21.15.83:0/2412816474 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fe1e00a3bb0 0x7fe1e00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:01.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:01.237+0000 7fe1f3e00640 1 -- 172.21.15.83:0/2412816474 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fe1ec0d0be0 con 0x7fe1fc1052c0 2024-04-24T08:49:01.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:01.238+0000 7fe1fb400640 1 --2- 172.21.15.83:0/2412816474 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fe1e00a3bb0 0x7fe1e00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:01.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:01.239+0000 7fe1fb400640 1 --2- 172.21.15.83:0/2412816474 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fe1e00a3bb0 0x7fe1e00a6070 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe1e80058c0 tx=0x7fe1e8005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:49:01.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:01.240+0000 7fe1f3e00640 1 -- 172.21.15.83:0/2412816474 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe1ec09d3b0 con 0x7fe1fc1052c0 2024-04-24T08:49:01.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:49:00 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/1229053139' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-24T08:49:01.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:49:00 smithi083 ceph-mon[27907]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-24T08:49:01.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:49:00 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/2693167363' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-04-24T08:49:01.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:49:00 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/1229053139' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-24T08:49:01.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:49:00 smithi138 ceph-mon[31441]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-24T08:49:01.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:49:00 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/2693167363' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-04-24T08:49:01.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:49:00 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/1229053139' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-24T08:49:01.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:49:00 smithi189 ceph-mon[31791]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-24T08:49:01.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:49:00 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/2693167363' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-04-24T08:49:01.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:01.485+0000 7fe200e00640 1 -- 172.21.15.83:0/2412816474 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7fe1fc02d620 con 0x7fe1fc1052c0 2024-04-24T08:49:02.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:02.020+0000 7fe1f3e00640 1 -- 172.21.15.83:0/2412816474 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v21) ==== 121+0+0 (secure 0 0 0) 0x7fe1ec09ccb0 con 0x7fe1fc1052c0 2024-04-24T08:49:02.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:02.025+0000 7fe200e00640 1 -- 172.21.15.83:0/2412816474 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fe1e00a3bb0 msgr2=0x7fe1e00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:02.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:02.025+0000 7fe200e00640 1 --2- 172.21.15.83:0/2412816474 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fe1e00a3bb0 0x7fe1e00a6070 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe1e80058c0 tx=0x7fe1e8005850 comp rx=0 tx=0).stop 2024-04-24T08:49:02.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:02.025+0000 7fe200e00640 1 -- 172.21.15.83:0/2412816474 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1fc1052c0 msgr2=0x7fe1fc19a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:02.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:02.025+0000 7fe200e00640 1 --2- 172.21.15.83:0/2412816474 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1fc1052c0 0x7fe1fc19a840 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fe1ec00e9a0 tx=0x7fe1ec00ee60 comp rx=0 tx=0).stop 2024-04-24T08:49:02.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:02.025+0000 7fe200e00640 1 -- 172.21.15.83:0/2412816474 shutdown_connections 2024-04-24T08:49:02.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:02.025+0000 7fe200e00640 1 --2- 172.21.15.83:0/2412816474 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1fc107bf0 0x7fe1fc1a18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:02.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:02.025+0000 7fe200e00640 1 --2- 172.21.15.83:0/2412816474 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fe1e00a3bb0 0x7fe1e00a6070 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:02.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:02.025+0000 7fe200e00640 1 --2- 172.21.15.83:0/2412816474 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1fc1052c0 0x7fe1fc19a840 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:02.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:02.025+0000 7fe200e00640 1 --2- 172.21.15.83:0/2412816474 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe1fc102990 0x7fe1fc19a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:02.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:02.025+0000 7fe200e00640 1 -- 172.21.15.83:0/2412816474 >> 172.21.15.83:0/2412816474 conn(0x7fe1fc0fc3a0 msgr2=0x7fe1fc102100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:49:02.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:02.026+0000 7fe200e00640 1 -- 172.21.15.83:0/2412816474 shutdown_connections 2024-04-24T08:49:02.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:02.026+0000 7fe200e00640 1 -- 172.21.15.83:0/2412816474 wait complete. 2024-04-24T08:49:02.053 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-04-24T08:49:02.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:02.242+0000 7fc7d2e00640 1 -- 172.21.15.83:0/2300540863 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7cc105450 msgr2=0x7fc7cc1058d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:02.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:02.242+0000 7fc7d2e00640 1 --2- 172.21.15.83:0/2300540863 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7cc105450 0x7fc7cc1058d0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fc7b4009900 tx=0x7fc7b402a3f0 comp rx=0 tx=0).stop 2024-04-24T08:49:02.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:02.243+0000 7fc7d2e00640 1 -- 172.21.15.83:0/2300540863 shutdown_connections 2024-04-24T08:49:02.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:02.243+0000 7fc7d2e00640 1 --2- 172.21.15.83:0/2300540863 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc7cc105e10 0x7fc7cc10c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:02.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:02.243+0000 7fc7d2e00640 1 --2- 172.21.15.83:0/2300540863 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7cc105450 0x7fc7cc1058d0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:02.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:02.243+0000 7fc7d2e00640 1 --2- 172.21.15.83:0/2300540863 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7cc104260 0x7fc7cc104680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:02.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:02.243+0000 7fc7d2e00640 1 -- 172.21.15.83:0/2300540863 >> 172.21.15.83:0/2300540863 conn(0x7fc7cc0ff7b0 msgr2=0x7fc7cc101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:49:02.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:02.243+0000 7fc7d2e00640 1 -- 172.21.15.83:0/2300540863 shutdown_connections 2024-04-24T08:49:02.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:02.243+0000 7fc7d2e00640 1 -- 172.21.15.83:0/2300540863 wait complete. 2024-04-24T08:49:02.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:02.244+0000 7fc7d2e00640 1 Processor -- start 2024-04-24T08:49:02.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:02.244+0000 7fc7d2e00640 1 -- start start 2024-04-24T08:49:02.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:02.244+0000 7fc7d2e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7cc104260 0x7fc7cc19a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:02.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:02.244+0000 7fc7d2e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc7cc105450 0x7fc7cc19a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:02.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:02.244+0000 7fc7d2e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7cc105e10 0x7fc7cc1a1a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:02.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:02.244+0000 7fc7d2e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc7cc10fda0 con 0x7fc7cc104260 2024-04-24T08:49:02.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:02.244+0000 7fc7d2e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc7cc10fc20 con 0x7fc7cc105e10 2024-04-24T08:49:02.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:02.244+0000 7fc7d2e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc7cc10ff20 con 0x7fc7cc105450 2024-04-24T08:49:02.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:02.244+0000 7fc7cbe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7cc104260 0x7fc7cc19a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:02.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:02.244+0000 7fc7cbe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7cc104260 0x7fc7cc19a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57670/0 (socket says 172.21.15.83:57670) 2024-04-24T08:49:02.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:02.244+0000 7fc7cbe00640 1 -- 172.21.15.83:0/3122229860 learned_addr learned my addr 172.21.15.83:0/3122229860 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:49:02.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:02.245+0000 7fc7d0e00640 1 --2- 172.21.15.83:0/3122229860 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7cc105e10 0x7fc7cc1a1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:02.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:02.245+0000 7fc7c3400640 1 --2- 172.21.15.83:0/3122229860 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc7cc105450 0x7fc7cc19a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:02.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:02.245+0000 7fc7cbe00640 1 -- 172.21.15.83:0/3122229860 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc7cc105450 msgr2=0x7fc7cc19a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:02.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:02.245+0000 7fc7cbe00640 1 --2- 172.21.15.83:0/3122229860 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc7cc105450 0x7fc7cc19a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:02.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:02.245+0000 7fc7cbe00640 1 -- 172.21.15.83:0/3122229860 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7cc105e10 msgr2=0x7fc7cc1a1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:02.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:02.245+0000 7fc7cbe00640 1 --2- 172.21.15.83:0/3122229860 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7cc105e10 0x7fc7cc1a1a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:02.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:02.245+0000 7fc7cbe00640 1 -- 172.21.15.83:0/3122229860 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7cc1a20a0 con 0x7fc7cc104260 2024-04-24T08:49:02.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:02.245+0000 7fc7cbe00640 1 --2- 172.21.15.83:0/3122229860 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7cc104260 0x7fc7cc19a480 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc7bc00ea80 tx=0x7fc7bc00ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:49:02.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:02.245+0000 7fc7c9600640 1 -- 172.21.15.83:0/3122229860 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7bc00ce40 con 0x7fc7cc104260 2024-04-24T08:49:02.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:02.245+0000 7fc7d2e00640 1 -- 172.21.15.83:0/3122229860 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7cc1a2390 con 0x7fc7cc104260 2024-04-24T08:49:02.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:02.245+0000 7fc7c9600640 1 -- 172.21.15.83:0/3122229860 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc7bc011920 con 0x7fc7cc104260 2024-04-24T08:49:02.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:02.245+0000 7fc7c9600640 1 -- 172.21.15.83:0/3122229860 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7bc01aa40 con 0x7fc7cc104260 2024-04-24T08:49:02.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:02.245+0000 7fc7d2e00640 1 -- 172.21.15.83:0/3122229860 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7cc1a2930 con 0x7fc7cc104260 2024-04-24T08:49:02.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:02.247+0000 7fc7d2e00640 1 -- 172.21.15.83:0/3122229860 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7cc10b600 con 0x7fc7cc104260 2024-04-24T08:49:02.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:02.249+0000 7fc7c9600640 1 -- 172.21.15.83:0/3122229860 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fc7bc011ad0 con 0x7fc7cc104260 2024-04-24T08:49:02.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:02.250+0000 7fc7c9600640 1 --2- 172.21.15.83:0/3122229860 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fc7b00a3bb0 0x7fc7b00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:02.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:02.250+0000 7fc7c9600640 1 -- 172.21.15.83:0/3122229860 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fc7bc0d0d40 con 0x7fc7cc104260 2024-04-24T08:49:02.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:02.250+0000 7fc7c3400640 1 --2- 172.21.15.83:0/3122229860 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fc7b00a3bb0 0x7fc7b00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:02.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:02.251+0000 7fc7c3400640 1 --2- 172.21.15.83:0/3122229860 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fc7b00a3bb0 0x7fc7b00a6070 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc7cc19b990 tx=0x7fc7b40314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:49:02.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:02.254+0000 7fc7c9600640 1 -- 172.21.15.83:0/3122229860 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc7bc016030 con 0x7fc7cc104260 2024-04-24T08:49:02.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:49:02 smithi083 ceph-mon[27907]: pgmap v389: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-04-24T08:49:02.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:49:02 smithi083 ceph-mon[27907]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-24T08:49:02.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:49:02 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/2693167363' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-04-24T08:49:02.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:49:02 smithi083 ceph-mon[27907]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-24T08:49:02.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:49:02 smithi083 ceph-mon[27907]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-24T08:49:02.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:49:02 smithi083 ceph-mon[27907]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-04-24T08:49:02.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:49:02 smithi083 ceph-mon[27907]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-04-24T08:49:02.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:49:02 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/2412816474' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-24T08:49:02.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:49:02 smithi138 ceph-mon[31441]: pgmap v389: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-04-24T08:49:02.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:49:02 smithi138 ceph-mon[31441]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-24T08:49:02.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:49:02 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/2693167363' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-04-24T08:49:02.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:49:02 smithi138 ceph-mon[31441]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-24T08:49:02.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:49:02 smithi138 ceph-mon[31441]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-24T08:49:02.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:49:02 smithi138 ceph-mon[31441]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-04-24T08:49:02.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:49:02 smithi138 ceph-mon[31441]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-04-24T08:49:02.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:49:02 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/2412816474' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-24T08:49:02.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:49:02 smithi189 ceph-mon[31791]: pgmap v389: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-04-24T08:49:02.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:49:02 smithi189 ceph-mon[31791]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-24T08:49:02.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:49:02 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/2693167363' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-04-24T08:49:02.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:49:02 smithi189 ceph-mon[31791]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-24T08:49:02.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:49:02 smithi189 ceph-mon[31791]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-24T08:49:02.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:49:02 smithi189 ceph-mon[31791]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-04-24T08:49:02.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:49:02 smithi189 ceph-mon[31791]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-04-24T08:49:02.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:49:02 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/2412816474' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-24T08:49:02.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:02.500+0000 7fc7d2e00640 1 -- 172.21.15.83:0/3122229860 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7fc7cc02d620 con 0x7fc7cc104260 2024-04-24T08:49:03.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:03.044+0000 7fc7c9600640 1 -- 172.21.15.83:0/3122229860 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v23) ==== 115+0+0 (secure 0 0 0) 0x7fc7bc09d480 con 0x7fc7cc104260 2024-04-24T08:49:03.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:03.048+0000 7fc7d2e00640 1 -- 172.21.15.83:0/3122229860 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fc7b00a3bb0 msgr2=0x7fc7b00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:03.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:03.048+0000 7fc7d2e00640 1 --2- 172.21.15.83:0/3122229860 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fc7b00a3bb0 0x7fc7b00a6070 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc7cc19b990 tx=0x7fc7b40314a0 comp rx=0 tx=0).stop 2024-04-24T08:49:03.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:03.048+0000 7fc7d2e00640 1 -- 172.21.15.83:0/3122229860 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7cc104260 msgr2=0x7fc7cc19a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:03.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:03.048+0000 7fc7d2e00640 1 --2- 172.21.15.83:0/3122229860 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7cc104260 0x7fc7cc19a480 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc7bc00ea80 tx=0x7fc7bc00ef40 comp rx=0 tx=0).stop 2024-04-24T08:49:03.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:03.048+0000 7fc7d2e00640 1 -- 172.21.15.83:0/3122229860 shutdown_connections 2024-04-24T08:49:03.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:03.048+0000 7fc7d2e00640 1 --2- 172.21.15.83:0/3122229860 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7cc105e10 0x7fc7cc1a1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:03.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:03.048+0000 7fc7d2e00640 1 --2- 172.21.15.83:0/3122229860 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc7cc105450 0x7fc7cc19a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:03.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:03.048+0000 7fc7d2e00640 1 --2- 172.21.15.83:0/3122229860 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fc7b00a3bb0 0x7fc7b00a6070 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:03.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:03.048+0000 7fc7d2e00640 1 --2- 172.21.15.83:0/3122229860 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7cc104260 0x7fc7cc19a480 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:03.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:03.048+0000 7fc7d2e00640 1 -- 172.21.15.83:0/3122229860 >> 172.21.15.83:0/3122229860 conn(0x7fc7cc0ff7b0 msgr2=0x7fc7cc1087d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:49:03.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:03.049+0000 7fc7d2e00640 1 -- 172.21.15.83:0/3122229860 shutdown_connections 2024-04-24T08:49:03.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:03.049+0000 7fc7d2e00640 1 -- 172.21.15.83:0/3122229860 wait complete. 2024-04-24T08:49:03.077 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2} subvols for filesystem 'cephfs' 2024-04-24T08:49:03.078 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-04-24T08:49:03.205 INFO:journalctl@ceph.mds.b.smithi138.stdout:Apr 24 08:49:02 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-b[61515]: 2024-04-24T08:49:02.942+0000 7f7fb8a00700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-04-24T08:49:03.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:49:03 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/2412816474' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-24T08:49:03.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:49:03 smithi138 ceph-mon[31441]: mds.? [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] up:active 2024-04-24T08:49:03.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:49:03 smithi138 ceph-mon[31441]: daemon mds.e assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-24T08:49:03.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:49:03 smithi138 ceph-mon[31441]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-04-24T08:49:03.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:49:03 smithi138 ceph-mon[31441]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:creating} 9 up:standby 2024-04-24T08:49:03.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:49:03 smithi138 ceph-mon[31441]: daemon mds.e is now active in filesystem cephfs as rank 2 2024-04-24T08:49:03.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:49:03 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/3122229860' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-24T08:49:03.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:49:03 smithi138 ceph-mon[31441]: pgmap v390: 129 pgs: 129 active+clean; 582 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-24T08:49:03.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:03.269+0000 7fb6f4200640 1 -- 172.21.15.83:0/1946711810 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6ec104260 msgr2=0x7fb6ec104680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:03.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:03.269+0000 7fb6f4200640 1 --2- 172.21.15.83:0/1946711810 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6ec104260 0x7fb6ec104680 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fb6e0009900 tx=0x7fb6e002a3f0 comp rx=0 tx=0).stop 2024-04-24T08:49:03.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:03.269+0000 7fb6f4200640 1 -- 172.21.15.83:0/1946711810 shutdown_connections 2024-04-24T08:49:03.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:03.269+0000 7fb6f4200640 1 --2- 172.21.15.83:0/1946711810 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb6ec105e10 0x7fb6ec10c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:03.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:03.269+0000 7fb6f4200640 1 --2- 172.21.15.83:0/1946711810 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6ec105450 0x7fb6ec1058d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:03.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:03.269+0000 7fb6f4200640 1 --2- 172.21.15.83:0/1946711810 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6ec104260 0x7fb6ec104680 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:03.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:03.269+0000 7fb6f4200640 1 -- 172.21.15.83:0/1946711810 >> 172.21.15.83:0/1946711810 conn(0x7fb6ec0ff7f0 msgr2=0x7fb6ec101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:49:03.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:03.269+0000 7fb6f4200640 1 -- 172.21.15.83:0/1946711810 shutdown_connections 2024-04-24T08:49:03.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:03.269+0000 7fb6f4200640 1 -- 172.21.15.83:0/1946711810 wait complete. 2024-04-24T08:49:03.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:03.270+0000 7fb6f4200640 1 Processor -- start 2024-04-24T08:49:03.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:03.270+0000 7fb6f4200640 1 -- start start 2024-04-24T08:49:03.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:03.270+0000 7fb6f4200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6ec104260 0x7fb6ec19a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:03.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:03.270+0000 7fb6f4200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6ec105450 0x7fb6ec19aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:03.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:03.270+0000 7fb6f4200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb6ec105e10 0x7fb6ec1a1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:03.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:03.270+0000 7fb6f4200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb6ec10fdf0 con 0x7fb6ec104260 2024-04-24T08:49:03.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:03.270+0000 7fb6f4200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb6ec10fc70 con 0x7fb6ec105450 2024-04-24T08:49:03.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:03.270+0000 7fb6f4200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb6ec10ff70 con 0x7fb6ec105e10 2024-04-24T08:49:03.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:03.271+0000 7fb6f1800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6ec104260 0x7fb6ec19a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:03.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:03.271+0000 7fb6f1800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6ec104260 0x7fb6ec19a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57694/0 (socket says 172.21.15.83:57694) 2024-04-24T08:49:03.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:03.271+0000 7fb6f1800640 1 -- 172.21.15.83:0/1835540683 learned_addr learned my addr 172.21.15.83:0/1835540683 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:49:03.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:03.271+0000 7fb6f0e00640 1 --2- 172.21.15.83:0/1835540683 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6ec105450 0x7fb6ec19aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:03.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:03.271+0000 7fb6f1800640 1 -- 172.21.15.83:0/1835540683 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb6ec105e10 msgr2=0x7fb6ec1a1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:03.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:03.271+0000 7fb6f2200640 1 --2- 172.21.15.83:0/1835540683 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb6ec105e10 0x7fb6ec1a1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:03.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:03.271+0000 7fb6f1800640 1 --2- 172.21.15.83:0/1835540683 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb6ec105e10 0x7fb6ec1a1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:03.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:03.271+0000 7fb6f1800640 1 -- 172.21.15.83:0/1835540683 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6ec105450 msgr2=0x7fb6ec19aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:03.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:03.271+0000 7fb6f1800640 1 --2- 172.21.15.83:0/1835540683 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6ec105450 0x7fb6ec19aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:03.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:03.271+0000 7fb6f1800640 1 -- 172.21.15.83:0/1835540683 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6ec1a20f0 con 0x7fb6ec104260 2024-04-24T08:49:03.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:03.271+0000 7fb6f2200640 1 --2- 172.21.15.83:0/1835540683 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb6ec105e10 0x7fb6ec1a1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:49:03.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:03.271+0000 7fb6f1800640 1 --2- 172.21.15.83:0/1835540683 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6ec104260 0x7fb6ec19a4d0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fb6e00314e0 tx=0x7fb6e0003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:49:03.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:03.272+0000 7fb6ea000640 1 -- 172.21.15.83:0/1835540683 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e003f070 con 0x7fb6ec104260 2024-04-24T08:49:03.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:03.272+0000 7fb6ea000640 1 -- 172.21.15.83:0/1835540683 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb6e0031e40 con 0x7fb6ec104260 2024-04-24T08:49:03.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:03.272+0000 7fb6ea000640 1 -- 172.21.15.83:0/1835540683 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e0035030 con 0x7fb6ec104260 2024-04-24T08:49:03.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:03.272+0000 7fb6f4200640 1 -- 172.21.15.83:0/1835540683 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6ec1a2380 con 0x7fb6ec104260 2024-04-24T08:49:03.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:03.272+0000 7fb6f4200640 1 -- 172.21.15.83:0/1835540683 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6ec1a2820 con 0x7fb6ec104260 2024-04-24T08:49:03.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:03.273+0000 7fb6f4200640 1 -- 172.21.15.83:0/1835540683 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6ec10b620 con 0x7fb6ec104260 2024-04-24T08:49:03.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:03.275+0000 7fb6ea000640 1 -- 172.21.15.83:0/1835540683 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb6e0042070 con 0x7fb6ec104260 2024-04-24T08:49:03.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:03.276+0000 7fb6ea000640 1 --2- 172.21.15.83:0/1835540683 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fb6d40a3c30 0x7fb6d40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:03.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:03.277+0000 7fb6ea000640 1 -- 172.21.15.83:0/1835540683 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fb6e00f0fd0 con 0x7fb6ec104260 2024-04-24T08:49:03.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:03.277+0000 7fb6f0e00640 1 --2- 172.21.15.83:0/1835540683 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fb6d40a3c30 0x7fb6d40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:03.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:03.277+0000 7fb6f0e00640 1 --2- 172.21.15.83:0/1835540683 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fb6d40a3c30 0x7fb6d40a60f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb6d8005fd0 tx=0x7fb6d8005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:49:03.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:03.287+0000 7fb6ea000640 1 -- 172.21.15.83:0/1835540683 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb6e00bd710 con 0x7fb6ec104260 2024-04-24T08:49:03.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:49:03 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/2412816474' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-24T08:49:03.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:49:03 smithi083 ceph-mon[27907]: mds.? [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] up:active 2024-04-24T08:49:03.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:49:03 smithi083 ceph-mon[27907]: daemon mds.e assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-24T08:49:03.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:49:03 smithi083 ceph-mon[27907]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-04-24T08:49:03.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:49:03 smithi083 ceph-mon[27907]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:creating} 9 up:standby 2024-04-24T08:49:03.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:49:03 smithi083 ceph-mon[27907]: daemon mds.e is now active in filesystem cephfs as rank 2 2024-04-24T08:49:03.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:49:03 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/3122229860' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-24T08:49:03.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:49:03 smithi083 ceph-mon[27907]: pgmap v390: 129 pgs: 129 active+clean; 582 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-24T08:49:03.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:49:03 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/2412816474' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-24T08:49:03.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:49:03 smithi189 ceph-mon[31791]: mds.? [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] up:active 2024-04-24T08:49:03.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:49:03 smithi189 ceph-mon[31791]: daemon mds.e assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-24T08:49:03.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:49:03 smithi189 ceph-mon[31791]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-04-24T08:49:03.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:49:03 smithi189 ceph-mon[31791]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:creating} 9 up:standby 2024-04-24T08:49:03.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:49:03 smithi189 ceph-mon[31791]: daemon mds.e is now active in filesystem cephfs as rank 2 2024-04-24T08:49:03.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:49:03 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/3122229860' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-24T08:49:03.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:49:03 smithi189 ceph-mon[31791]: pgmap v390: 129 pgs: 129 active+clean; 582 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-24T08:49:03.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:03.515+0000 7fb6f4200640 1 -- 172.21.15.83:0/1835540683 --> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fb6ec062440 con 0x7fb6d40a3c30 2024-04-24T08:49:03.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:03.574+0000 7fb6ea000640 1 -- 172.21.15.83:0/1835540683 <== mgr.14152 v2:172.21.15.83:6800/2550826560 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb6ec062440 con 0x7fb6d40a3c30 2024-04-24T08:49:03.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:03.577+0000 7fb6f4200640 1 -- 172.21.15.83:0/1835540683 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fb6d40a3c30 msgr2=0x7fb6d40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:03.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:03.577+0000 7fb6f4200640 1 --2- 172.21.15.83:0/1835540683 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fb6d40a3c30 0x7fb6d40a60f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb6d8005fd0 tx=0x7fb6d8005850 comp rx=0 tx=0).stop 2024-04-24T08:49:03.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:03.577+0000 7fb6f4200640 1 -- 172.21.15.83:0/1835540683 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6ec104260 msgr2=0x7fb6ec19a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:03.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:03.577+0000 7fb6f4200640 1 --2- 172.21.15.83:0/1835540683 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6ec104260 0x7fb6ec19a4d0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fb6e00314e0 tx=0x7fb6e0003c80 comp rx=0 tx=0).stop 2024-04-24T08:49:03.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:03.577+0000 7fb6f4200640 1 -- 172.21.15.83:0/1835540683 shutdown_connections 2024-04-24T08:49:03.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:03.577+0000 7fb6f4200640 1 --2- 172.21.15.83:0/1835540683 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb6ec105e10 0x7fb6ec1a1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:03.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:03.577+0000 7fb6f4200640 1 --2- 172.21.15.83:0/1835540683 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6ec105450 0x7fb6ec19aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:03.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:03.577+0000 7fb6f4200640 1 --2- 172.21.15.83:0/1835540683 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fb6d40a3c30 0x7fb6d40a60f0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:03.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:03.577+0000 7fb6f4200640 1 --2- 172.21.15.83:0/1835540683 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6ec104260 0x7fb6ec19a4d0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:03.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:03.577+0000 7fb6f4200640 1 -- 172.21.15.83:0/1835540683 >> 172.21.15.83:0/1835540683 conn(0x7fb6ec0ff7f0 msgr2=0x7fb6ec1087f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:49:03.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:03.578+0000 7fb6f4200640 1 -- 172.21.15.83:0/1835540683 shutdown_connections 2024-04-24T08:49:03.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:03.578+0000 7fb6f4200640 1 -- 172.21.15.83:0/1835540683 wait complete. 2024-04-24T08:49:03.597 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-04-24T08:49:03.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:03.796+0000 7f1f67200640 1 -- 172.21.15.83:0/507271199 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f68105320 msgr2=0x7f1f68107710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:03.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:03.796+0000 7f1f67200640 1 --2- 172.21.15.83:0/507271199 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f68105320 0x7f1f68107710 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f1f54009900 tx=0x7f1f5402a410 comp rx=0 tx=0).stop 2024-04-24T08:49:03.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:03.796+0000 7f1f67200640 1 -- 172.21.15.83:0/507271199 shutdown_connections 2024-04-24T08:49:03.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:03.796+0000 7f1f67200640 1 --2- 172.21.15.83:0/507271199 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1f68107c50 0x7f1f6810a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:03.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:03.796+0000 7f1f67200640 1 --2- 172.21.15.83:0/507271199 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f68105320 0x7f1f68107710 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:03.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:03.796+0000 7f1f67200640 1 --2- 172.21.15.83:0/507271199 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f681029f0 0x7f1f68104de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:03.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:03.796+0000 7f1f67200640 1 -- 172.21.15.83:0/507271199 >> 172.21.15.83:0/507271199 conn(0x7f1f680fc3a0 msgr2=0x7f1f680fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:49:03.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:03.796+0000 7f1f67200640 1 -- 172.21.15.83:0/507271199 shutdown_connections 2024-04-24T08:49:03.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:03.796+0000 7f1f67200640 1 -- 172.21.15.83:0/507271199 wait complete. 2024-04-24T08:49:03.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:03.797+0000 7f1f67200640 1 Processor -- start 2024-04-24T08:49:03.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:03.797+0000 7f1f67200640 1 -- start start 2024-04-24T08:49:03.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:03.798+0000 7f1f67200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f681029f0 0x7f1f6819a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:03.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:03.798+0000 7f1f67200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1f68105320 0x7f1f6819a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:03.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:03.798+0000 7f1f67200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f68107c50 0x7f1f681a1920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:03.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:03.798+0000 7f1f67200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1f6810da20 con 0x7f1f681029f0 2024-04-24T08:49:03.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:03.798+0000 7f1f67200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1f6810d8a0 con 0x7f1f68107c50 2024-04-24T08:49:03.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:03.798+0000 7f1f67200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1f6810dba0 con 0x7f1f68105320 2024-04-24T08:49:03.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:03.798+0000 7f1f65e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f681029f0 0x7f1f6819a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:03.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:03.798+0000 7f1f65e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f681029f0 0x7f1f6819a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57736/0 (socket says 172.21.15.83:57736) 2024-04-24T08:49:03.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:03.798+0000 7f1f65e00640 1 -- 172.21.15.83:0/2586864381 learned_addr learned my addr 172.21.15.83:0/2586864381 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:49:03.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:03.798+0000 7f1f65400640 1 --2- 172.21.15.83:0/2586864381 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1f68105320 0x7f1f6819a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:03.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:03.798+0000 7f1f66800640 1 --2- 172.21.15.83:0/2586864381 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f68107c50 0x7f1f681a1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:03.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:03.798+0000 7f1f65e00640 1 -- 172.21.15.83:0/2586864381 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1f68105320 msgr2=0x7f1f6819a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:03.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:03.799+0000 7f1f65e00640 1 --2- 172.21.15.83:0/2586864381 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1f68105320 0x7f1f6819a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:03.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:03.799+0000 7f1f65e00640 1 -- 172.21.15.83:0/2586864381 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f68107c50 msgr2=0x7f1f681a1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:03.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:03.799+0000 7f1f65e00640 1 --2- 172.21.15.83:0/2586864381 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f68107c50 0x7f1f681a1920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:03.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:03.799+0000 7f1f65e00640 1 -- 172.21.15.83:0/2586864381 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f681a1f90 con 0x7f1f681029f0 2024-04-24T08:49:03.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:03.799+0000 7f1f65400640 1 --2- 172.21.15.83:0/2586864381 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1f68105320 0x7f1f6819a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:49:03.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:03.799+0000 7f1f65e00640 1 --2- 172.21.15.83:0/2586864381 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f681029f0 0x7f1f6819a370 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f1f5800e9f0 tx=0x7f1f5800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:49:03.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:03.799+0000 7f1f5ea00640 1 -- 172.21.15.83:0/2586864381 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f5800cd80 con 0x7f1f681029f0 2024-04-24T08:49:03.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:03.799+0000 7f1f5ea00640 1 -- 172.21.15.83:0/2586864381 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1f580108e0 con 0x7f1f681029f0 2024-04-24T08:49:03.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:03.799+0000 7f1f67200640 1 -- 172.21.15.83:0/2586864381 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f681a2280 con 0x7f1f681029f0 2024-04-24T08:49:03.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:03.799+0000 7f1f5ea00640 1 -- 172.21.15.83:0/2586864381 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f58018920 con 0x7f1f681029f0 2024-04-24T08:49:03.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:03.799+0000 7f1f67200640 1 -- 172.21.15.83:0/2586864381 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f681a2820 con 0x7f1f681029f0 2024-04-24T08:49:03.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:03.800+0000 7f1f67200640 1 -- 172.21.15.83:0/2586864381 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f68109740 con 0x7f1f681029f0 2024-04-24T08:49:03.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:03.802+0000 7f1f5ea00640 1 -- 172.21.15.83:0/2586864381 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f1f58010a90 con 0x7f1f681029f0 2024-04-24T08:49:03.803 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:49:03 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:49:03.528+0000 7fba6ea00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T08:49:03.803 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:49:03 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:49:03.528+0000 7fba6ea00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T08:49:03.803 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:49:03 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:49:03.528+0000 7fba6ea00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T08:49:03.803 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:49:03 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:49:03.528+0000 7fba6ea00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T08:49:03.803 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:49:03 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:49:03.528+0000 7fba6ea00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T08:49:03.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:03.804+0000 7f1f5ea00640 1 --2- 172.21.15.83:0/2586864381 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f1f4c0a3c30 0x7f1f4c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:03.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:03.804+0000 7f1f5ea00640 1 -- 172.21.15.83:0/2586864381 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f1f580d0a00 con 0x7f1f681029f0 2024-04-24T08:49:03.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:03.804+0000 7f1f65400640 1 --2- 172.21.15.83:0/2586864381 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f1f4c0a3c30 0x7f1f4c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:03.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:03.805+0000 7f1f65400640 1 --2- 172.21.15.83:0/2586864381 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f1f4c0a3c30 0x7f1f4c0a60f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1f54006010 tx=0x7f1f54000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:49:03.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:03.809+0000 7f1f5ea00640 1 -- 172.21.15.83:0/2586864381 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1f5809d1a0 con 0x7f1f681029f0 2024-04-24T08:49:04.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:04.041+0000 7f1f67200640 1 -- 172.21.15.83:0/2586864381 --> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f1f680624d0 con 0x7f1f4c0a3c30 2024-04-24T08:49:04.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:04.104+0000 7f1f5ea00640 1 -- 172.21.15.83:0/2586864381 <== mgr.14152 v2:172.21.15.83:6800/2550826560 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f1f680624d0 con 0x7f1f4c0a3c30 2024-04-24T08:49:04.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:04.107+0000 7f1f67200640 1 -- 172.21.15.83:0/2586864381 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f1f4c0a3c30 msgr2=0x7f1f4c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:04.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:04.107+0000 7f1f67200640 1 --2- 172.21.15.83:0/2586864381 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f1f4c0a3c30 0x7f1f4c0a60f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1f54006010 tx=0x7f1f54000bc0 comp rx=0 tx=0).stop 2024-04-24T08:49:04.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:04.107+0000 7f1f67200640 1 -- 172.21.15.83:0/2586864381 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f681029f0 msgr2=0x7f1f6819a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:04.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:04.107+0000 7f1f67200640 1 --2- 172.21.15.83:0/2586864381 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f681029f0 0x7f1f6819a370 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f1f5800e9f0 tx=0x7f1f5800eeb0 comp rx=0 tx=0).stop 2024-04-24T08:49:04.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:04.107+0000 7f1f67200640 1 -- 172.21.15.83:0/2586864381 shutdown_connections 2024-04-24T08:49:04.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:04.107+0000 7f1f67200640 1 --2- 172.21.15.83:0/2586864381 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f68107c50 0x7f1f681a1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:04.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:04.107+0000 7f1f67200640 1 --2- 172.21.15.83:0/2586864381 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1f68105320 0x7f1f6819a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:04.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:04.107+0000 7f1f67200640 1 --2- 172.21.15.83:0/2586864381 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f1f4c0a3c30 0x7f1f4c0a60f0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:04.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:04.107+0000 7f1f67200640 1 --2- 172.21.15.83:0/2586864381 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f681029f0 0x7f1f6819a370 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:04.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:04.107+0000 7f1f67200640 1 -- 172.21.15.83:0/2586864381 >> 172.21.15.83:0/2586864381 conn(0x7f1f680fc3a0 msgr2=0x7f1f68102250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:49:04.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:04.108+0000 7f1f67200640 1 -- 172.21.15.83:0/2586864381 shutdown_connections 2024-04-24T08:49:04.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:04.108+0000 7f1f67200640 1 -- 172.21.15.83:0/2586864381 wait complete. 2024-04-24T08:49:04.127 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-04-24T08:49:04.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:49:04 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/3122229860' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-24T08:49:04.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:49:04 smithi083 ceph-mon[27907]: mds.? [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] up:active 2024-04-24T08:49:04.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:49:04 smithi083 ceph-mon[27907]: daemon mds.k assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-04-24T08:49:04.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:49:04 smithi083 ceph-mon[27907]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 9 up:standby 2024-04-24T08:49:04.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:49:04 smithi083 ceph-mon[27907]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=e=up:active,3=k=up:creating} 8 up:standby 2024-04-24T08:49:04.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:49:04 smithi083 ceph-mon[27907]: daemon mds.k is now active in filesystem cephfs as rank 3 2024-04-24T08:49:04.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:49:04 smithi083 ceph-mon[27907]: from='client.14931 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:49:04.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:04.321+0000 7f9291a00640 1 -- 172.21.15.83:0/4247153434 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f928c109730 msgr2=0x7f928c109bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:04.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:04.321+0000 7f9291a00640 1 --2- 172.21.15.83:0/4247153434 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f928c109730 0x7f928c109bb0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f9274009900 tx=0x7f927402a410 comp rx=0 tx=0).stop 2024-04-24T08:49:04.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:04.321+0000 7f9291a00640 1 -- 172.21.15.83:0/4247153434 shutdown_connections 2024-04-24T08:49:04.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:04.321+0000 7f9291a00640 1 --2- 172.21.15.83:0/4247153434 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f928c10a0f0 0x7f928c110980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:04.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:04.321+0000 7f9291a00640 1 --2- 172.21.15.83:0/4247153434 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f928c109730 0x7f928c109bb0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:04.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:04.321+0000 7f9291a00640 1 --2- 172.21.15.83:0/4247153434 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f928c108540 0x7f928c108960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:04.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:04.321+0000 7f9291a00640 1 -- 172.21.15.83:0/4247153434 >> 172.21.15.83:0/4247153434 conn(0x7f928c0774d0 msgr2=0x7f928c079910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:49:04.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:04.321+0000 7f9291a00640 1 -- 172.21.15.83:0/4247153434 shutdown_connections 2024-04-24T08:49:04.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:04.321+0000 7f9291a00640 1 -- 172.21.15.83:0/4247153434 wait complete. 2024-04-24T08:49:04.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:04.322+0000 7f9291a00640 1 Processor -- start 2024-04-24T08:49:04.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:04.322+0000 7f9291a00640 1 -- start start 2024-04-24T08:49:04.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:04.322+0000 7f9291a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f928c108540 0x7f928c19e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:04.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:04.322+0000 7f9291a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f928c109730 0x7f928c19ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:04.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:04.322+0000 7f9291a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f928c10a0f0 0x7f928c1a5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:04.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:04.322+0000 7f9291a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f928c114080 con 0x7f928c108540 2024-04-24T08:49:04.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:04.322+0000 7f9291a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f928c113f00 con 0x7f928c109730 2024-04-24T08:49:04.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:04.322+0000 7f9291a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f928c114200 con 0x7f928c10a0f0 2024-04-24T08:49:04.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:04.323+0000 7f928be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f928c108540 0x7f928c19e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:04.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:04.323+0000 7f928be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f928c108540 0x7f928c19e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57750/0 (socket says 172.21.15.83:57750) 2024-04-24T08:49:04.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:04.323+0000 7f928be00640 1 -- 172.21.15.83:0/3907618450 learned_addr learned my addr 172.21.15.83:0/3907618450 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:49:04.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:04.323+0000 7f9291000640 1 --2- 172.21.15.83:0/3907618450 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f928c10a0f0 0x7f928c1a5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:04.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:04.323+0000 7f928be00640 1 -- 172.21.15.83:0/3907618450 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f928c10a0f0 msgr2=0x7f928c1a5cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:04.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:04.323+0000 7f928be00640 1 --2- 172.21.15.83:0/3907618450 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f928c10a0f0 0x7f928c1a5cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:04.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:04.323+0000 7f9283400640 1 --2- 172.21.15.83:0/3907618450 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f928c109730 0x7f928c19ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:04.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:04.323+0000 7f928be00640 1 -- 172.21.15.83:0/3907618450 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f928c109730 msgr2=0x7f928c19ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:04.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:04.323+0000 7f928be00640 1 --2- 172.21.15.83:0/3907618450 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f928c109730 0x7f928c19ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:04.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:04.323+0000 7f928be00640 1 -- 172.21.15.83:0/3907618450 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f928c1a6340 con 0x7f928c108540 2024-04-24T08:49:04.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:04.323+0000 7f928be00640 1 --2- 172.21.15.83:0/3907618450 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f928c108540 0x7f928c19e720 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f927c00d770 tx=0x7f927c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:49:04.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:04.323+0000 7f9289600640 1 -- 172.21.15.83:0/3907618450 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f927c017070 con 0x7f928c108540 2024-04-24T08:49:04.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:04.324+0000 7f9291a00640 1 -- 172.21.15.83:0/3907618450 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f928c1a6630 con 0x7f928c108540 2024-04-24T08:49:04.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:04.324+0000 7f9289600640 1 -- 172.21.15.83:0/3907618450 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f927c009c30 con 0x7f928c108540 2024-04-24T08:49:04.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:04.324+0000 7f9289600640 1 -- 172.21.15.83:0/3907618450 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f927c003d00 con 0x7f928c108540 2024-04-24T08:49:04.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:04.324+0000 7f9291a00640 1 -- 172.21.15.83:0/3907618450 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f928c1a6bd0 con 0x7f928c108540 2024-04-24T08:49:04.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:04.325+0000 7f9291a00640 1 -- 172.21.15.83:0/3907618450 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f928c10f860 con 0x7f928c108540 2024-04-24T08:49:04.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:04.327+0000 7f9289600640 1 -- 172.21.15.83:0/3907618450 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f927c003ea0 con 0x7f928c108540 2024-04-24T08:49:04.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:04.328+0000 7f9289600640 1 --2- 172.21.15.83:0/3907618450 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f92700a3c30 0x7f92700a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:04.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:04.328+0000 7f9289600640 1 -- 172.21.15.83:0/3907618450 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f927c0d03a0 con 0x7f928c108540 2024-04-24T08:49:04.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:04.328+0000 7f9283400640 1 --2- 172.21.15.83:0/3907618450 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f92700a3c30 0x7f92700a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:04.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:04.329+0000 7f9283400640 1 --2- 172.21.15.83:0/3907618450 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f92700a3c30 0x7f92700a60f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f927402a930 tx=0x7f9274000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:49:04.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:04.332+0000 7f9289600640 1 -- 172.21.15.83:0/3907618450 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f927c09cb40 con 0x7f928c108540 2024-04-24T08:49:04.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:49:04 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/3122229860' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-24T08:49:04.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:49:04 smithi138 ceph-mon[31441]: mds.? [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] up:active 2024-04-24T08:49:04.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:49:04 smithi138 ceph-mon[31441]: daemon mds.k assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-04-24T08:49:04.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:49:04 smithi138 ceph-mon[31441]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 9 up:standby 2024-04-24T08:49:04.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:49:04 smithi138 ceph-mon[31441]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=e=up:active,3=k=up:creating} 8 up:standby 2024-04-24T08:49:04.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:49:04 smithi138 ceph-mon[31441]: daemon mds.k is now active in filesystem cephfs as rank 3 2024-04-24T08:49:04.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:49:04 smithi138 ceph-mon[31441]: from='client.14931 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:49:04.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:49:04 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/3122229860' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-24T08:49:04.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:49:04 smithi189 ceph-mon[31791]: mds.? [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] up:active 2024-04-24T08:49:04.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:49:04 smithi189 ceph-mon[31791]: daemon mds.k assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-04-24T08:49:04.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:49:04 smithi189 ceph-mon[31791]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 9 up:standby 2024-04-24T08:49:04.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:49:04 smithi189 ceph-mon[31791]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=e=up:active,3=k=up:creating} 8 up:standby 2024-04-24T08:49:04.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:49:04 smithi189 ceph-mon[31791]: daemon mds.k is now active in filesystem cephfs as rank 3 2024-04-24T08:49:04.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:49:04 smithi189 ceph-mon[31791]: from='client.14931 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:49:04.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:04.564+0000 7f9291a00640 1 -- 172.21.15.83:0/3907618450 --> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f928c0624d0 con 0x7f92700a3c30 2024-04-24T08:49:04.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:04.578+0000 7f9289600640 1 -- 172.21.15.83:0/3907618450 <== mgr.14152 v2:172.21.15.83:6800/2550826560 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f928c0624d0 con 0x7f92700a3c30 2024-04-24T08:49:04.578 INFO:teuthology.orchestra.run.smithi083.stdout:/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2 2024-04-24T08:49:04.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:04.581+0000 7f9291a00640 1 -- 172.21.15.83:0/3907618450 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f92700a3c30 msgr2=0x7f92700a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:04.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:04.581+0000 7f9291a00640 1 --2- 172.21.15.83:0/3907618450 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f92700a3c30 0x7f92700a60f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f927402a930 tx=0x7f9274000bc0 comp rx=0 tx=0).stop 2024-04-24T08:49:04.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:04.581+0000 7f9291a00640 1 -- 172.21.15.83:0/3907618450 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f928c108540 msgr2=0x7f928c19e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:04.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:04.581+0000 7f9291a00640 1 --2- 172.21.15.83:0/3907618450 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f928c108540 0x7f928c19e720 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f927c00d770 tx=0x7f927c00dc30 comp rx=0 tx=0).stop 2024-04-24T08:49:04.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:04.581+0000 7f9291a00640 1 -- 172.21.15.83:0/3907618450 shutdown_connections 2024-04-24T08:49:04.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:04.581+0000 7f9291a00640 1 --2- 172.21.15.83:0/3907618450 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f928c10a0f0 0x7f928c1a5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:04.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:04.581+0000 7f9291a00640 1 --2- 172.21.15.83:0/3907618450 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f928c109730 0x7f928c19ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:04.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:04.581+0000 7f9291a00640 1 --2- 172.21.15.83:0/3907618450 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f92700a3c30 0x7f92700a60f0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:04.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:04.581+0000 7f9291a00640 1 --2- 172.21.15.83:0/3907618450 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f928c108540 0x7f928c19e720 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:04.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:04.581+0000 7f9291a00640 1 -- 172.21.15.83:0/3907618450 >> 172.21.15.83:0/3907618450 conn(0x7f928c0774d0 msgr2=0x7f928c10ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:49:04.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:04.581+0000 7f9291a00640 1 -- 172.21.15.83:0/3907618450 shutdown_connections 2024-04-24T08:49:04.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:04.581+0000 7f9291a00640 1 -- 172.21.15.83:0/3907618450 wait complete. 2024-04-24T08:49:04.599 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-04-24T08:49:04.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:04.804+0000 7f9ef7c00640 1 -- 172.21.15.83:0/2861487252 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ef0105e10 msgr2=0x7f9ef010c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:04.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:04.804+0000 7f9ef7c00640 1 --2- 172.21.15.83:0/2861487252 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ef0105e10 0x7f9ef010c6a0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f9ee8009900 tx=0x7f9ee802a410 comp rx=0 tx=0).stop 2024-04-24T08:49:04.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:04.804+0000 7f9ef7c00640 1 -- 172.21.15.83:0/2861487252 shutdown_connections 2024-04-24T08:49:04.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:04.804+0000 7f9ef7c00640 1 --2- 172.21.15.83:0/2861487252 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ef0105e10 0x7f9ef010c6a0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:04.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:04.804+0000 7f9ef7c00640 1 --2- 172.21.15.83:0/2861487252 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9ef0105450 0x7f9ef01058d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:04.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:04.804+0000 7f9ef7c00640 1 --2- 172.21.15.83:0/2861487252 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9ef0104260 0x7f9ef0104680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:04.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:04.804+0000 7f9ef7c00640 1 -- 172.21.15.83:0/2861487252 >> 172.21.15.83:0/2861487252 conn(0x7f9ef00ff7f0 msgr2=0x7f9ef0101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:49:04.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:04.804+0000 7f9ef7c00640 1 -- 172.21.15.83:0/2861487252 shutdown_connections 2024-04-24T08:49:04.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:04.805+0000 7f9ef7c00640 1 -- 172.21.15.83:0/2861487252 wait complete. 2024-04-24T08:49:04.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:04.805+0000 7f9ef7c00640 1 Processor -- start 2024-04-24T08:49:04.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:04.806+0000 7f9ef7c00640 1 -- start start 2024-04-24T08:49:04.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:04.806+0000 7f9ef7c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9ef0104260 0x7f9ef019a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:04.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:04.806+0000 7f9ef7c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9ef0105450 0x7f9ef019a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:04.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:04.806+0000 7f9ef7c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ef0105e10 0x7f9ef01a19f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:04.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:04.806+0000 7f9ef7c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef010fdc0 con 0x7f9ef0105e10 2024-04-24T08:49:04.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:04.806+0000 7f9ef7c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef010fc40 con 0x7f9ef0104260 2024-04-24T08:49:04.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:04.806+0000 7f9ef7c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef010ff40 con 0x7f9ef0105450 2024-04-24T08:49:04.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:04.806+0000 7f9ef5c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ef0105e10 0x7f9ef01a19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:04.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:04.806+0000 7f9ef5c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ef0105e10 0x7f9ef01a19f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57760/0 (socket says 172.21.15.83:57760) 2024-04-24T08:49:04.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:04.806+0000 7f9ef5c00640 1 -- 172.21.15.83:0/1584036074 learned_addr learned my addr 172.21.15.83:0/1584036074 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:49:04.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:04.806+0000 7f9ef5200640 1 --2- 172.21.15.83:0/1584036074 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9ef0104260 0x7f9ef019a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:04.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:04.807+0000 7f9eefe00640 1 --2- 172.21.15.83:0/1584036074 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9ef0105450 0x7f9ef019a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:04.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:04.807+0000 7f9ef5c00640 1 -- 172.21.15.83:0/1584036074 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9ef0105450 msgr2=0x7f9ef019a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:04.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:04.807+0000 7f9ef5c00640 1 --2- 172.21.15.83:0/1584036074 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9ef0105450 0x7f9ef019a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:04.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:04.807+0000 7f9ef5c00640 1 -- 172.21.15.83:0/1584036074 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9ef0104260 msgr2=0x7f9ef019a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:04.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:04.807+0000 7f9ef5c00640 1 --2- 172.21.15.83:0/1584036074 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9ef0104260 0x7f9ef019a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:04.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:04.807+0000 7f9ef5c00640 1 -- 172.21.15.83:0/1584036074 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ef01a2060 con 0x7f9ef0105e10 2024-04-24T08:49:04.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:04.807+0000 7f9ef5c00640 1 --2- 172.21.15.83:0/1584036074 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ef0105e10 0x7f9ef01a19f0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f9ee8005ab0 tx=0x7f9ee8003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:49:04.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:04.807+0000 7f9eed600640 1 -- 172.21.15.83:0/1584036074 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ee803f070 con 0x7f9ef0105e10 2024-04-24T08:49:04.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:04.807+0000 7f9eed600640 1 -- 172.21.15.83:0/1584036074 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9ee8031c30 con 0x7f9ef0105e10 2024-04-24T08:49:04.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:04.807+0000 7f9ef7c00640 1 -- 172.21.15.83:0/1584036074 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ef01a22f0 con 0x7f9ef0105e10 2024-04-24T08:49:04.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:04.807+0000 7f9eed600640 1 -- 172.21.15.83:0/1584036074 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ee8035030 con 0x7f9ef0105e10 2024-04-24T08:49:04.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:04.808+0000 7f9ef7c00640 1 -- 172.21.15.83:0/1584036074 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ef01a2830 con 0x7f9ef0105e10 2024-04-24T08:49:04.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:04.809+0000 7f9ef7c00640 1 -- 172.21.15.83:0/1584036074 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ef010b650 con 0x7f9ef0105e10 2024-04-24T08:49:04.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:04.811+0000 7f9eed600640 1 -- 172.21.15.83:0/1584036074 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9ee8042070 con 0x7f9ef0105e10 2024-04-24T08:49:04.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:04.812+0000 7f9eed600640 1 --2- 172.21.15.83:0/1584036074 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f9ed40a3c30 0x7f9ed40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:04.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:04.812+0000 7f9eed600640 1 -- 172.21.15.83:0/1584036074 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f9ee80f0e60 con 0x7f9ef0105e10 2024-04-24T08:49:04.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:04.813+0000 7f9ef5200640 1 --2- 172.21.15.83:0/1584036074 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f9ed40a3c30 0x7f9ed40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:04.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:04.813+0000 7f9ef5200640 1 --2- 172.21.15.83:0/1584036074 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f9ed40a3c30 0x7f9ed40a60f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9ef01052b0 tx=0x7f9ee0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:49:04.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:04.816+0000 7f9eed600640 1 -- 172.21.15.83:0/1584036074 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9ee8031de0 con 0x7f9ef0105e10 2024-04-24T08:49:05.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:05.050+0000 7f9ef7c00640 1 -- 172.21.15.83:0/1584036074 --> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f9ef00624d0 con 0x7f9ed40a3c30 2024-04-24T08:49:05.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:05.113+0000 7f9eed600640 1 -- 172.21.15.83:0/1584036074 <== mgr.14152 v2:172.21.15.83:6800/2550826560 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f9ef00624d0 con 0x7f9ed40a3c30 2024-04-24T08:49:05.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:05.117+0000 7f9ef7c00640 1 -- 172.21.15.83:0/1584036074 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f9ed40a3c30 msgr2=0x7f9ed40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:05.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:05.117+0000 7f9ef7c00640 1 --2- 172.21.15.83:0/1584036074 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f9ed40a3c30 0x7f9ed40a60f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9ef01052b0 tx=0x7f9ee0005f90 comp rx=0 tx=0).stop 2024-04-24T08:49:05.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:05.117+0000 7f9ef7c00640 1 -- 172.21.15.83:0/1584036074 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ef0105e10 msgr2=0x7f9ef01a19f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:05.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:05.117+0000 7f9ef7c00640 1 --2- 172.21.15.83:0/1584036074 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ef0105e10 0x7f9ef01a19f0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f9ee8005ab0 tx=0x7f9ee8003b90 comp rx=0 tx=0).stop 2024-04-24T08:49:05.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:05.117+0000 7f9ef7c00640 1 -- 172.21.15.83:0/1584036074 shutdown_connections 2024-04-24T08:49:05.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:05.117+0000 7f9ef7c00640 1 --2- 172.21.15.83:0/1584036074 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f9ed40a3c30 0x7f9ed40a60f0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:05.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:05.117+0000 7f9ef7c00640 1 --2- 172.21.15.83:0/1584036074 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ef0105e10 0x7f9ef01a19f0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:05.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:05.117+0000 7f9ef7c00640 1 --2- 172.21.15.83:0/1584036074 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9ef0105450 0x7f9ef019a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:05.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:05.117+0000 7f9ef7c00640 1 --2- 172.21.15.83:0/1584036074 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9ef0104260 0x7f9ef019a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:05.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:05.117+0000 7f9ef7c00640 1 -- 172.21.15.83:0/1584036074 >> 172.21.15.83:0/1584036074 conn(0x7f9ef00ff7f0 msgr2=0x7f9ef0108820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:49:05.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:05.117+0000 7f9ef7c00640 1 -- 172.21.15.83:0/1584036074 shutdown_connections 2024-04-24T08:49:05.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:05.118+0000 7f9ef7c00640 1 -- 172.21.15.83:0/1584036074 wait complete. 2024-04-24T08:49:05.137 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-04-24T08:49:05.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:05.348+0000 7f7c73200640 1 -- 172.21.15.83:0/1509559672 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c6c077c00 msgr2=0x7f7c6c078060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:05.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:05.348+0000 7f7c73200640 1 --2- 172.21.15.83:0/1509559672 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c6c077c00 0x7f7c6c078060 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f7c54009900 tx=0x7f7c5402a410 comp rx=0 tx=0).stop 2024-04-24T08:49:05.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:05.348+0000 7f7c73200640 1 -- 172.21.15.83:0/1509559672 shutdown_connections 2024-04-24T08:49:05.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:05.348+0000 7f7c73200640 1 --2- 172.21.15.83:0/1509559672 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c6c0785a0 0x7f7c6c078e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:05.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:05.348+0000 7f7c73200640 1 --2- 172.21.15.83:0/1509559672 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c6c077c00 0x7f7c6c078060 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:05.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:05.348+0000 7f7c73200640 1 --2- 172.21.15.83:0/1509559672 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7c6c0769c0 0x7f7c6c076de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:05.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:05.348+0000 7f7c73200640 1 -- 172.21.15.83:0/1509559672 >> 172.21.15.83:0/1509559672 conn(0x7f7c6c0ff7f0 msgr2=0x7f7c6c101c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:49:05.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:05.348+0000 7f7c73200640 1 -- 172.21.15.83:0/1509559672 shutdown_connections 2024-04-24T08:49:05.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:05.348+0000 7f7c73200640 1 -- 172.21.15.83:0/1509559672 wait complete. 2024-04-24T08:49:05.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:05.349+0000 7f7c73200640 1 Processor -- start 2024-04-24T08:49:05.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:05.349+0000 7f7c73200640 1 -- start start 2024-04-24T08:49:05.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:05.349+0000 7f7c73200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7c6c0769c0 0x7f7c6c19e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:05.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:05.350+0000 7f7c73200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c6c077c00 0x7f7c6c19edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:05.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:05.350+0000 7f7c73200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c6c0785a0 0x7f7c6c1a5e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:05.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:05.350+0000 7f7c73200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7c6c1141c0 con 0x7f7c6c077c00 2024-04-24T08:49:05.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:05.350+0000 7f7c73200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7c6c114040 con 0x7f7c6c0785a0 2024-04-24T08:49:05.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:05.350+0000 7f7c73200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7c6c114340 con 0x7f7c6c0769c0 2024-04-24T08:49:05.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:05.350+0000 7f7c63400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c6c077c00 0x7f7c6c19edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:05.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:05.350+0000 7f7c63400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c6c077c00 0x7f7c6c19edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57778/0 (socket says 172.21.15.83:57778) 2024-04-24T08:49:05.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:05.350+0000 7f7c63400640 1 -- 172.21.15.83:0/3770367475 learned_addr learned my addr 172.21.15.83:0/3770367475 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:49:05.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:05.350+0000 7f7c6be00640 1 --2- 172.21.15.83:0/3770367475 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7c6c0769c0 0x7f7c6c19e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:05.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:05.350+0000 7f7c71200640 1 --2- 172.21.15.83:0/3770367475 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c6c0785a0 0x7f7c6c1a5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:05.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:05.350+0000 7f7c63400640 1 -- 172.21.15.83:0/3770367475 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7c6c0769c0 msgr2=0x7f7c6c19e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:05.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:05.350+0000 7f7c63400640 1 --2- 172.21.15.83:0/3770367475 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7c6c0769c0 0x7f7c6c19e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:05.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:05.350+0000 7f7c63400640 1 -- 172.21.15.83:0/3770367475 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c6c0785a0 msgr2=0x7f7c6c1a5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:05.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:05.350+0000 7f7c63400640 1 --2- 172.21.15.83:0/3770367475 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c6c0785a0 0x7f7c6c1a5e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:05.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:05.350+0000 7f7c63400640 1 -- 172.21.15.83:0/3770367475 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c6c1a64d0 con 0x7f7c6c077c00 2024-04-24T08:49:05.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:05.350+0000 7f7c71200640 1 --2- 172.21.15.83:0/3770367475 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c6c0785a0 0x7f7c6c1a5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:49:05.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:05.351+0000 7f7c63400640 1 --2- 172.21.15.83:0/3770367475 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c6c077c00 0x7f7c6c19edf0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f7c54004770 tx=0x7f7c540042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:49:05.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:05.351+0000 7f7c69600640 1 -- 172.21.15.83:0/3770367475 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c5403f030 con 0x7f7c6c077c00 2024-04-24T08:49:05.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:05.351+0000 7f7c69600640 1 -- 172.21.15.83:0/3770367475 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7c54003cb0 con 0x7f7c6c077c00 2024-04-24T08:49:05.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:05.351+0000 7f7c69600640 1 -- 172.21.15.83:0/3770367475 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c54035070 con 0x7f7c6c077c00 2024-04-24T08:49:05.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:05.351+0000 7f7c73200640 1 -- 172.21.15.83:0/3770367475 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c6c1a6760 con 0x7f7c6c077c00 2024-04-24T08:49:05.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:05.351+0000 7f7c73200640 1 -- 172.21.15.83:0/3770367475 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c6c1a6ca0 con 0x7f7c6c077c00 2024-04-24T08:49:05.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:05.352+0000 7f7c73200640 1 -- 172.21.15.83:0/3770367475 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c6c110bf0 con 0x7f7c6c077c00 2024-04-24T08:49:05.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:05.354+0000 7f7c69600640 1 -- 172.21.15.83:0/3770367475 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f7c54042030 con 0x7f7c6c077c00 2024-04-24T08:49:05.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:05.355+0000 7f7c69600640 1 --2- 172.21.15.83:0/3770367475 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f7c500a3cc0 0x7f7c500a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:05.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:05.356+0000 7f7c69600640 1 -- 172.21.15.83:0/3770367475 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f7c540f11a0 con 0x7f7c6c077c00 2024-04-24T08:49:05.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:05.356+0000 7f7c6be00640 1 --2- 172.21.15.83:0/3770367475 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f7c500a3cc0 0x7f7c500a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:05.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:05.357+0000 7f7c6be00640 1 --2- 172.21.15.83:0/3770367475 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f7c500a3cc0 0x7f7c500a6180 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7c5c005fd0 tx=0x7f7c5c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:49:05.357 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:49:05 smithi138 ceph-mon[31441]: from='client.14943 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:49:05.357 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:49:05 smithi138 ceph-mon[31441]: mds.? [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] up:active 2024-04-24T08:49:05.357 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:49:05 smithi138 ceph-mon[31441]: daemon mds.i assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-04-24T08:49:05.357 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:49:05 smithi138 ceph-mon[31441]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-24T08:49:05.357 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:49:05 smithi138 ceph-mon[31441]: Cluster is now healthy 2024-04-24T08:49:05.357 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:49:05 smithi138 ceph-mon[31441]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=e=up:active,3=k=up:active} 8 up:standby 2024-04-24T08:49:05.357 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:49:05 smithi138 ceph-mon[31441]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-04-24T08:49:05.357 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:49:05 smithi138 ceph-mon[31441]: daemon mds.i is now active in filesystem cephfs as rank 4 2024-04-24T08:49:05.357 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:49:05 smithi138 ceph-mon[31441]: mds.? [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] up:active 2024-04-24T08:49:05.357 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:49:05 smithi138 ceph-mon[31441]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T08:49:05.358 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:49:05 smithi138 ceph-mon[31441]: from='client.14949 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:49:05.358 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:49:05 smithi138 ceph-mon[31441]: pgmap v391: 129 pgs: 129 active+clean; 583 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 5 op/s 2024-04-24T08:49:05.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:05.360+0000 7f7c69600640 1 -- 172.21.15.83:0/3770367475 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7c540bd7e0 con 0x7f7c6c077c00 2024-04-24T08:49:05.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:49:05 smithi189 ceph-mon[31791]: from='client.14943 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:49:05.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:49:05 smithi189 ceph-mon[31791]: mds.? [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] up:active 2024-04-24T08:49:05.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:49:05 smithi189 ceph-mon[31791]: daemon mds.i assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-04-24T08:49:05.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:49:05 smithi189 ceph-mon[31791]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-24T08:49:05.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:49:05 smithi189 ceph-mon[31791]: Cluster is now healthy 2024-04-24T08:49:05.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:49:05 smithi189 ceph-mon[31791]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=e=up:active,3=k=up:active} 8 up:standby 2024-04-24T08:49:05.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:49:05 smithi189 ceph-mon[31791]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-04-24T08:49:05.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:49:05 smithi189 ceph-mon[31791]: daemon mds.i is now active in filesystem cephfs as rank 4 2024-04-24T08:49:05.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:49:05 smithi189 ceph-mon[31791]: mds.? [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] up:active 2024-04-24T08:49:05.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:49:05 smithi189 ceph-mon[31791]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T08:49:05.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:49:05 smithi189 ceph-mon[31791]: from='client.14949 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:49:05.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:49:05 smithi189 ceph-mon[31791]: pgmap v391: 129 pgs: 129 active+clean; 583 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 5 op/s 2024-04-24T08:49:05.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:49:05 smithi083 ceph-mon[27907]: from='client.14943 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:49:05.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:49:05 smithi083 ceph-mon[27907]: mds.? [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] up:active 2024-04-24T08:49:05.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:49:05 smithi083 ceph-mon[27907]: daemon mds.i assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-04-24T08:49:05.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:49:05 smithi083 ceph-mon[27907]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-24T08:49:05.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:49:05 smithi083 ceph-mon[27907]: Cluster is now healthy 2024-04-24T08:49:05.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:49:05 smithi083 ceph-mon[27907]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=e=up:active,3=k=up:active} 8 up:standby 2024-04-24T08:49:05.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:49:05 smithi083 ceph-mon[27907]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-04-24T08:49:05.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:49:05 smithi083 ceph-mon[27907]: daemon mds.i is now active in filesystem cephfs as rank 4 2024-04-24T08:49:05.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:49:05 smithi083 ceph-mon[27907]: mds.? [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] up:active 2024-04-24T08:49:05.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:49:05 smithi083 ceph-mon[27907]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T08:49:05.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:49:05 smithi083 ceph-mon[27907]: from='client.14949 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:49:05.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:49:05 smithi083 ceph-mon[27907]: pgmap v391: 129 pgs: 129 active+clean; 583 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 5 op/s 2024-04-24T08:49:05.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:05.598+0000 7f7c73200640 1 -- 172.21.15.83:0/3770367475 --> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f7c6c0624d0 con 0x7f7c500a3cc0 2024-04-24T08:49:05.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:05.606+0000 7f7c69600640 1 -- 172.21.15.83:0/3770367475 <== mgr.14152 v2:172.21.15.83:6800/2550826560 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f7c6c0624d0 con 0x7f7c500a3cc0 2024-04-24T08:49:05.606 INFO:teuthology.orchestra.run.smithi083.stdout:/volumes/qa/sv_1/3ca7f423-8ca9-4be3-88d6-c3eb5e51bedc 2024-04-24T08:49:05.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:05.609+0000 7f7c73200640 1 -- 172.21.15.83:0/3770367475 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f7c500a3cc0 msgr2=0x7f7c500a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:05.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:05.609+0000 7f7c73200640 1 --2- 172.21.15.83:0/3770367475 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f7c500a3cc0 0x7f7c500a6180 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7c5c005fd0 tx=0x7f7c5c005850 comp rx=0 tx=0).stop 2024-04-24T08:49:05.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:05.609+0000 7f7c73200640 1 -- 172.21.15.83:0/3770367475 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c6c077c00 msgr2=0x7f7c6c19edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:05.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:05.609+0000 7f7c73200640 1 --2- 172.21.15.83:0/3770367475 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c6c077c00 0x7f7c6c19edf0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f7c54004770 tx=0x7f7c540042f0 comp rx=0 tx=0).stop 2024-04-24T08:49:05.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:05.609+0000 7f7c73200640 1 -- 172.21.15.83:0/3770367475 shutdown_connections 2024-04-24T08:49:05.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:05.609+0000 7f7c73200640 1 --2- 172.21.15.83:0/3770367475 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c6c0785a0 0x7f7c6c1a5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:05.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:05.609+0000 7f7c73200640 1 --2- 172.21.15.83:0/3770367475 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f7c500a3cc0 0x7f7c500a6180 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:05.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:05.609+0000 7f7c73200640 1 --2- 172.21.15.83:0/3770367475 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c6c077c00 0x7f7c6c19edf0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:05.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:05.609+0000 7f7c73200640 1 --2- 172.21.15.83:0/3770367475 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7c6c0769c0 0x7f7c6c19e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:05.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:05.609+0000 7f7c73200640 1 -- 172.21.15.83:0/3770367475 >> 172.21.15.83:0/3770367475 conn(0x7f7c6c0ff7f0 msgr2=0x7f7c6c102b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:49:05.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:05.610+0000 7f7c73200640 1 -- 172.21.15.83:0/3770367475 shutdown_connections 2024-04-24T08:49:05.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:05.610+0000 7f7c73200640 1 -- 172.21.15.83:0/3770367475 wait complete. 2024-04-24T08:49:05.626 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T08:49:05.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:05.827+0000 7f90a5800640 1 -- 172.21.15.83:0/118612896 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f90a0102900 msgr2=0x7f90a0104cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:05.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:05.827+0000 7f90a5800640 1 --2- 172.21.15.83:0/118612896 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f90a0102900 0x7f90a0104cf0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f9090009ca0 tx=0x7f909002a8c0 comp rx=0 tx=0).stop 2024-04-24T08:49:05.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:05.827+0000 7f90a5800640 1 -- 172.21.15.83:0/118612896 shutdown_connections 2024-04-24T08:49:05.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:05.827+0000 7f90a5800640 1 --2- 172.21.15.83:0/118612896 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f90a0107b60 0x7f90a0109fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:05.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:05.827+0000 7f90a5800640 1 --2- 172.21.15.83:0/118612896 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90a0105230 0x7f90a0107620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:05.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:05.827+0000 7f90a5800640 1 --2- 172.21.15.83:0/118612896 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f90a0102900 0x7f90a0104cf0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:05.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:05.827+0000 7f90a5800640 1 -- 172.21.15.83:0/118612896 >> 172.21.15.83:0/118612896 conn(0x7f90a00fc2d0 msgr2=0x7f90a00fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:49:05.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:05.827+0000 7f90a5800640 1 -- 172.21.15.83:0/118612896 shutdown_connections 2024-04-24T08:49:05.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:05.827+0000 7f90a5800640 1 -- 172.21.15.83:0/118612896 wait complete. 2024-04-24T08:49:05.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:05.828+0000 7f90a5800640 1 Processor -- start 2024-04-24T08:49:05.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:05.828+0000 7f90a5800640 1 -- start start 2024-04-24T08:49:05.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:05.828+0000 7f90a5800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90a0102900 0x7f90a0102270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:05.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:05.828+0000 7f90a5800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f90a0105230 0x7f90a01008b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:05.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:05.828+0000 7f90a5800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f90a0107b60 0x7f90a0100df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:05.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:05.828+0000 7f90a5800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f90a010da20 con 0x7f90a0107b60 2024-04-24T08:49:05.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:05.828+0000 7f90a5800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f90a010d8a0 con 0x7f90a0102900 2024-04-24T08:49:05.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:05.828+0000 7f90a5800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f90a010dba0 con 0x7f90a0105230 2024-04-24T08:49:05.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:05.829+0000 7f90a4e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f90a0107b60 0x7f90a0100df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:05.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:05.829+0000 7f90a4e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f90a0107b60 0x7f90a0100df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57790/0 (socket says 172.21.15.83:57790) 2024-04-24T08:49:05.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:05.829+0000 7f90a4e00640 1 -- 172.21.15.83:0/2024030723 learned_addr learned my addr 172.21.15.83:0/2024030723 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:49:05.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:05.829+0000 7f909fe00640 1 --2- 172.21.15.83:0/2024030723 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90a0102900 0x7f90a0102270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:05.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:05.829+0000 7f909f400640 1 --2- 172.21.15.83:0/2024030723 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f90a0105230 0x7f90a01008b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:05.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:05.829+0000 7f90a4e00640 1 -- 172.21.15.83:0/2024030723 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f90a0105230 msgr2=0x7f90a01008b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:05.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:05.829+0000 7f90a4e00640 1 --2- 172.21.15.83:0/2024030723 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f90a0105230 0x7f90a01008b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:05.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:05.829+0000 7f90a4e00640 1 -- 172.21.15.83:0/2024030723 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90a0102900 msgr2=0x7f90a0102270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:05.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:05.829+0000 7f90a4e00640 1 --2- 172.21.15.83:0/2024030723 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90a0102900 0x7f90a0102270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:05.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:05.829+0000 7f90a4e00640 1 -- 172.21.15.83:0/2024030723 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90a0101680 con 0x7f90a0107b60 2024-04-24T08:49:05.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:05.829+0000 7f90a4e00640 1 --2- 172.21.15.83:0/2024030723 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f90a0107b60 0x7f90a0100df0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f909800e970 tx=0x7f909800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:49:05.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:05.830+0000 7f909cc00640 1 -- 172.21.15.83:0/2024030723 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f909800ccc0 con 0x7f90a0107b60 2024-04-24T08:49:05.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:05.830+0000 7f909cc00640 1 -- 172.21.15.83:0/2024030723 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f909800ce60 con 0x7f90a0107b60 2024-04-24T08:49:05.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:05.830+0000 7f909cc00640 1 -- 172.21.15.83:0/2024030723 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9098010430 con 0x7f90a0107b60 2024-04-24T08:49:05.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:05.830+0000 7f90a5800640 1 -- 172.21.15.83:0/2024030723 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90a01a21e0 con 0x7f90a0107b60 2024-04-24T08:49:05.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:05.830+0000 7f90a5800640 1 -- 172.21.15.83:0/2024030723 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90a01a26e0 con 0x7f90a0107b60 2024-04-24T08:49:05.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:05.831+0000 7f90a5800640 1 -- 172.21.15.83:0/2024030723 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90a00fddc0 con 0x7f90a0107b60 2024-04-24T08:49:05.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:05.833+0000 7f909cc00640 1 -- 172.21.15.83:0/2024030723 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f9098004f90 con 0x7f90a0107b60 2024-04-24T08:49:05.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:05.835+0000 7f909cc00640 1 --2- 172.21.15.83:0/2024030723 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f90840a3cc0 0x7f90840a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:05.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:05.835+0000 7f909cc00640 1 -- 172.21.15.83:0/2024030723 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f90980d0c90 con 0x7f90a0107b60 2024-04-24T08:49:05.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:05.835+0000 7f909fe00640 1 --2- 172.21.15.83:0/2024030723 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f90840a3cc0 0x7f90840a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:05.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:05.836+0000 7f909fe00640 1 --2- 172.21.15.83:0/2024030723 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f90840a3cc0 0x7f90840a6180 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9090000f80 tx=0x7f90900093f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:49:05.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:05.840+0000 7f909cc00640 1 -- 172.21.15.83:0/2024030723 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f909809d270 con 0x7f90a0107b60 2024-04-24T08:49:06.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:06.085+0000 7f90a5800640 1 -- 172.21.15.83:0/2024030723 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f90a002d620 con 0x7f90a0107b60 2024-04-24T08:49:06.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:06.086+0000 7f909cc00640 1 -- 172.21.15.83:0/2024030723 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11944 (secure 0 0 0) 0x7f909809cb70 con 0x7f90a0107b60 2024-04-24T08:49:06.086 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:49:06.086 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/358872257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":358872257},{"type":"v1","addr":"172.21.15.83:6835","nonce":358872257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3157070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3157070851},{"type":"v1","addr":"172.21.15.83:6837","nonce":3157070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T08:48:29.422576+0000","modified":"2024-04-24T08:49:04.217098+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34520,"mds_1":34493,"mds_2":34451,"mds_3":34427,"mds_4":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14658":{"gid":14658,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6839/4085470844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":4085470844},{"type":"v1","addr":"172.21.15.83:6839","nonce":4085470844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14676":{"gid":14676,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6841/2634754210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":2634754210},{"type":"v1","addr":"172.21.15.83:6841","nonce":2634754210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24382":{"gid":24382,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/1859921310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1859921310},{"type":"v1","addr":"172.21.15.189:6833","nonce":1859921310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/2241923197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2241923197},{"type":"v1","addr":"172.21.15.189:6835","nonce":2241923197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6837/3172311220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":3172311220},{"type":"v1","addr":"172.21.15.189:6837","nonce":3172311220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.189:6839/2548965336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":2548965336},{"type":"v1","addr":"172.21.15.189:6839","nonce":2548965336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.138:6833/1157967658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1157967658},{"type":"v1","addr":"172.21.15.138:6833","nonce":1157967658}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.138:6835/1909219758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1909219758},{"type":"v1","addr":"172.21.15.138:6835","nonce":1909219758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34493":{"gid":34493,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.138:6837/900566754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":900566754},{"type":"v1","addr":"172.21.15.138:6837","nonce":900566754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34520":{"gid":34520,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.138:6839/2236932393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2236932393},{"type":"v1","addr":"172.21.15.138:6839","nonce":2236932393}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34520,"qdb_cluster":[34427,24454,34520,34493,34451]},"id":1}]} 2024-04-24T08:49:06.086 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 27 2024-04-24T08:49:06.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:06.089+0000 7f90a5800640 1 -- 172.21.15.83:0/2024030723 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f90840a3cc0 msgr2=0x7f90840a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:06.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:06.089+0000 7f90a5800640 1 --2- 172.21.15.83:0/2024030723 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f90840a3cc0 0x7f90840a6180 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9090000f80 tx=0x7f90900093f0 comp rx=0 tx=0).stop 2024-04-24T08:49:06.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:06.089+0000 7f90a5800640 1 -- 172.21.15.83:0/2024030723 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f90a0107b60 msgr2=0x7f90a0100df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:06.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:06.089+0000 7f90a5800640 1 --2- 172.21.15.83:0/2024030723 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f90a0107b60 0x7f90a0100df0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f909800e970 tx=0x7f909800ed70 comp rx=0 tx=0).stop 2024-04-24T08:49:06.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:06.090+0000 7f90a5800640 1 -- 172.21.15.83:0/2024030723 shutdown_connections 2024-04-24T08:49:06.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:06.090+0000 7f90a5800640 1 --2- 172.21.15.83:0/2024030723 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f90840a3cc0 0x7f90840a6180 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:06.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:06.090+0000 7f90a5800640 1 --2- 172.21.15.83:0/2024030723 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f90a0107b60 0x7f90a0100df0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:06.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:06.090+0000 7f90a5800640 1 --2- 172.21.15.83:0/2024030723 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f90a0105230 0x7f90a01008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:06.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:06.090+0000 7f90a5800640 1 --2- 172.21.15.83:0/2024030723 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90a0102900 0x7f90a0102270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:06.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:06.090+0000 7f90a5800640 1 -- 172.21.15.83:0/2024030723 >> 172.21.15.83:0/2024030723 conn(0x7f90a00fc2d0 msgr2=0x7f90a0103380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:49:06.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:06.090+0000 7f90a5800640 1 -- 172.21.15.83:0/2024030723 shutdown_connections 2024-04-24T08:49:06.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:06.090+0000 7f90a5800640 1 -- 172.21.15.83:0/2024030723 wait complete. 2024-04-24T08:49:06.110 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T08:49:06.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:06.320+0000 7fb972a00640 1 -- 172.21.15.83:0/2368164899 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb96c105450 msgr2=0x7fb96c1058d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:06.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:06.320+0000 7fb972a00640 1 --2- 172.21.15.83:0/2368164899 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb96c105450 0x7fb96c1058d0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb958009900 tx=0x7fb95802a3f0 comp rx=0 tx=0).stop 2024-04-24T08:49:06.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:06.320+0000 7fb972a00640 1 -- 172.21.15.83:0/2368164899 shutdown_connections 2024-04-24T08:49:06.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:06.320+0000 7fb972a00640 1 --2- 172.21.15.83:0/2368164899 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb96c105e10 0x7fb96c10c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:06.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:06.320+0000 7fb972a00640 1 --2- 172.21.15.83:0/2368164899 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb96c105450 0x7fb96c1058d0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:06.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:06.320+0000 7fb972a00640 1 --2- 172.21.15.83:0/2368164899 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb96c104260 0x7fb96c104680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:06.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:06.320+0000 7fb972a00640 1 -- 172.21.15.83:0/2368164899 >> 172.21.15.83:0/2368164899 conn(0x7fb96c0ff7f0 msgr2=0x7fb96c101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:49:06.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:06.320+0000 7fb972a00640 1 -- 172.21.15.83:0/2368164899 shutdown_connections 2024-04-24T08:49:06.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:06.320+0000 7fb972a00640 1 -- 172.21.15.83:0/2368164899 wait complete. 2024-04-24T08:49:06.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:06.321+0000 7fb972a00640 1 Processor -- start 2024-04-24T08:49:06.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:06.321+0000 7fb972a00640 1 -- start start 2024-04-24T08:49:06.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:06.321+0000 7fb972a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb96c104260 0x7fb96c19a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:06.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:06.321+0000 7fb972a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb96c105450 0x7fb96c19a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:06.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:06.321+0000 7fb972a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb96c105e10 0x7fb96c1a19a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:06.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:06.321+0000 7fb972a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb96c10fdf0 con 0x7fb96c105450 2024-04-24T08:49:06.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:06.321+0000 7fb972a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb96c10fc70 con 0x7fb96c104260 2024-04-24T08:49:06.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:06.321+0000 7fb972a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb96c10ff70 con 0x7fb96c105e10 2024-04-24T08:49:06.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:06.321+0000 7fb96b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb96c105450 0x7fb96c19a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:06.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:06.322+0000 7fb96b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb96c105450 0x7fb96c19a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57808/0 (socket says 172.21.15.83:57808) 2024-04-24T08:49:06.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:06.322+0000 7fb96b400640 1 -- 172.21.15.83:0/4265399431 learned_addr learned my addr 172.21.15.83:0/4265399431 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:49:06.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:06.322+0000 7fb96be00640 1 --2- 172.21.15.83:0/4265399431 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb96c104260 0x7fb96c19a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:06.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:06.322+0000 7fb970a00640 1 --2- 172.21.15.83:0/4265399431 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb96c105e10 0x7fb96c1a19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:06.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:06.322+0000 7fb96b400640 1 -- 172.21.15.83:0/4265399431 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb96c105e10 msgr2=0x7fb96c1a19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:06.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:06.322+0000 7fb96b400640 1 --2- 172.21.15.83:0/4265399431 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb96c105e10 0x7fb96c1a19a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:06.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:06.322+0000 7fb96b400640 1 -- 172.21.15.83:0/4265399431 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb96c104260 msgr2=0x7fb96c19a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:06.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:06.322+0000 7fb96b400640 1 --2- 172.21.15.83:0/4265399431 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb96c104260 0x7fb96c19a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:06.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:06.322+0000 7fb96b400640 1 -- 172.21.15.83:0/4265399431 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb96c1a2010 con 0x7fb96c105450 2024-04-24T08:49:06.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:06.323+0000 7fb96b400640 1 --2- 172.21.15.83:0/4265399431 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb96c105450 0x7fb96c19a930 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fb9580314e0 tx=0x7fb958003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:49:06.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:06.323+0000 7fb968c00640 1 -- 172.21.15.83:0/4265399431 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb95803f070 con 0x7fb96c105450 2024-04-24T08:49:06.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:06.323+0000 7fb968c00640 1 -- 172.21.15.83:0/4265399431 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb958031e40 con 0x7fb96c105450 2024-04-24T08:49:06.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:06.323+0000 7fb968c00640 1 -- 172.21.15.83:0/4265399431 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb958035030 con 0x7fb96c105450 2024-04-24T08:49:06.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:06.323+0000 7fb972a00640 1 -- 172.21.15.83:0/4265399431 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb96c1a22a0 con 0x7fb96c105450 2024-04-24T08:49:06.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:06.323+0000 7fb972a00640 1 -- 172.21.15.83:0/4265399431 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb96c1a2740 con 0x7fb96c105450 2024-04-24T08:49:06.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:06.325+0000 7fb972a00640 1 -- 172.21.15.83:0/4265399431 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb96c10b520 con 0x7fb96c105450 2024-04-24T08:49:06.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:06.327+0000 7fb968c00640 1 -- 172.21.15.83:0/4265399431 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fb958042070 con 0x7fb96c105450 2024-04-24T08:49:06.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:06.328+0000 7fb968c00640 1 --2- 172.21.15.83:0/4265399431 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fb9500a3cc0 0x7fb9500a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:06.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:06.328+0000 7fb968c00640 1 -- 172.21.15.83:0/4265399431 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fb9580f1750 con 0x7fb96c105450 2024-04-24T08:49:06.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:06.329+0000 7fb96be00640 1 --2- 172.21.15.83:0/4265399431 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fb9500a3cc0 0x7fb9500a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:06.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:06.329+0000 7fb96be00640 1 --2- 172.21.15.83:0/4265399431 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fb9500a3cc0 0x7fb9500a6180 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb95c005fd0 tx=0x7fb95c005ce0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:49:06.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:06.331+0000 7fb968c00640 1 -- 172.21.15.83:0/4265399431 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb9580bdd30 con 0x7fb96c105450 2024-04-24T08:49:06.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:06.530+0000 7fb972a00640 1 -- 172.21.15.83:0/4265399431 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb96c02d870 con 0x7fb96c105450 2024-04-24T08:49:06.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:06.533+0000 7fb968c00640 1 -- 172.21.15.83:0/4265399431 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24127 (secure 0 0 0) 0x7fb9580bd630 con 0x7fb96c105450 2024-04-24T08:49:06.534 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:49:06.534 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":89,"fsid":"bd3e039e-0215-11ef-bc93-c7b262605968","created":"2024-04-24T08:37:32.209944+0000","modified":"2024-04-24T08:48:34.752923+0000","last_up_change":"2024-04-24T08:47:17.966041+0000","last_in_change":"2024-04-24T08:46:53.835936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T08:41:56.834372+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T08:48:26.978311+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T08:48:28.352731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T08:48:32.275167+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2cd77f3a-19dd-4c6e-a199-1dcbd5faa679","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2132675555},{"type":"v1","addr":"172.21.15.83:6803","nonce":2132675555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2132675555},{"type":"v1","addr":"172.21.15.83:6805","nonce":2132675555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2132675555},{"type":"v1","addr":"172.21.15.83:6809","nonce":2132675555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2132675555},{"type":"v1","addr":"172.21.15.83:6807","nonce":2132675555}]},"public_addr":"172.21.15.83:6803/2132675555","cluster_addr":"172.21.15.83:6805/2132675555","heartbeat_back_addr":"172.21.15.83:6809/2132675555","heartbeat_front_addr":"172.21.15.83:6807/2132675555","state":["exists","up"]},{"osd":1,"uuid":"aba7f5df-2a4c-4c23-b502-d193605a5b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":3589158880},{"type":"v1","addr":"172.21.15.138:6801","nonce":3589158880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":3589158880},{"type":"v1","addr":"172.21.15.138:6803","nonce":3589158880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":3589158880},{"type":"v1","addr":"172.21.15.138:6807","nonce":3589158880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":3589158880},{"type":"v1","addr":"172.21.15.138:6805","nonce":3589158880}]},"public_addr":"172.21.15.138:6801/3589158880","cluster_addr":"172.21.15.138:6803/3589158880","heartbeat_back_addr":"172.21.15.138:6807/3589158880","heartbeat_front_addr":"172.21.15.138:6805/3589158880","state":["exists","up"]},{"osd":2,"uuid":"071bc167-2333-4420-bb58-bffbc10d7641","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":3169072688},{"type":"v1","addr":"172.21.15.189:6801","nonce":3169072688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":3169072688},{"type":"v1","addr":"172.21.15.189:6803","nonce":3169072688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":3169072688},{"type":"v1","addr":"172.21.15.189:6807","nonce":3169072688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":3169072688},{"type":"v1","addr":"172.21.15.189:6805","nonce":3169072688}]},"public_addr":"172.21.15.189:6801/3169072688","cluster_addr":"172.21.15.189:6803/3169072688","heartbeat_back_addr":"172.21.15.189:6807/3169072688","heartbeat_front_addr":"172.21.15.189:6805/3169072688","state":["exists","up"]},{"osd":3,"uuid":"f5624d78-4455-4e07-8c1a-f47c94cddb23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3131092450},{"type":"v1","addr":"172.21.15.83:6811","nonce":3131092450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3131092450},{"type":"v1","addr":"172.21.15.83:6813","nonce":3131092450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3131092450},{"type":"v1","addr":"172.21.15.83:6817","nonce":3131092450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3131092450},{"type":"v1","addr":"172.21.15.83:6815","nonce":3131092450}]},"public_addr":"172.21.15.83:6811/3131092450","cluster_addr":"172.21.15.83:6813/3131092450","heartbeat_back_addr":"172.21.15.83:6817/3131092450","heartbeat_front_addr":"172.21.15.83:6815/3131092450","state":["exists","up"]},{"osd":4,"uuid":"7b50cd14-9385-4535-a7b0-1c1da93c8548","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":823775360},{"type":"v1","addr":"172.21.15.138:6809","nonce":823775360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":823775360},{"type":"v1","addr":"172.21.15.138:6811","nonce":823775360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":823775360},{"type":"v1","addr":"172.21.15.138:6815","nonce":823775360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":823775360},{"type":"v1","addr":"172.21.15.138:6813","nonce":823775360}]},"public_addr":"172.21.15.138:6809/823775360","cluster_addr":"172.21.15.138:6811/823775360","heartbeat_back_addr":"172.21.15.138:6815/823775360","heartbeat_front_addr":"172.21.15.138:6813/823775360","state":["exists","up"]},{"osd":5,"uuid":"9c647797-35bc-449f-be84-af97d2c69c2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":1399817431},{"type":"v1","addr":"172.21.15.189:6809","nonce":1399817431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":1399817431},{"type":"v1","addr":"172.21.15.189:6811","nonce":1399817431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":1399817431},{"type":"v1","addr":"172.21.15.189:6815","nonce":1399817431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":1399817431},{"type":"v1","addr":"172.21.15.189:6813","nonce":1399817431}]},"public_addr":"172.21.15.189:6809/1399817431","cluster_addr":"172.21.15.189:6811/1399817431","heartbeat_back_addr":"172.21.15.189:6815/1399817431","heartbeat_front_addr":"172.21.15.189:6813/1399817431","state":["exists","up"]},{"osd":6,"uuid":"8f056584-16c8-4454-b445-3f780f26c196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":1611358483},{"type":"v1","addr":"172.21.15.83:6819","nonce":1611358483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1611358483},{"type":"v1","addr":"172.21.15.83:6821","nonce":1611358483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1611358483},{"type":"v1","addr":"172.21.15.83:6825","nonce":1611358483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":1611358483},{"type":"v1","addr":"172.21.15.83:6823","nonce":1611358483}]},"public_addr":"172.21.15.83:6819/1611358483","cluster_addr":"172.21.15.83:6821/1611358483","heartbeat_back_addr":"172.21.15.83:6825/1611358483","heartbeat_front_addr":"172.21.15.83:6823/1611358483","state":["exists","up"]},{"osd":7,"uuid":"7603efe3-da77-4b7f-9d03-29c4f293b6d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":3557856706},{"type":"v1","addr":"172.21.15.138:6817","nonce":3557856706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":3557856706},{"type":"v1","addr":"172.21.15.138:6819","nonce":3557856706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":3557856706},{"type":"v1","addr":"172.21.15.138:6823","nonce":3557856706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":3557856706},{"type":"v1","addr":"172.21.15.138:6821","nonce":3557856706}]},"public_addr":"172.21.15.138:6817/3557856706","cluster_addr":"172.21.15.138:6819/3557856706","heartbeat_back_addr":"172.21.15.138:6823/3557856706","heartbeat_front_addr":"172.21.15.138:6821/3557856706","state":["exists","up"]},{"osd":8,"uuid":"27d7d1b8-d6a0-417e-b8aa-596325aa1a77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":207018554},{"type":"v1","addr":"172.21.15.189:6817","nonce":207018554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":207018554},{"type":"v1","addr":"172.21.15.189:6819","nonce":207018554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":207018554},{"type":"v1","addr":"172.21.15.189:6823","nonce":207018554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":207018554},{"type":"v1","addr":"172.21.15.189:6821","nonce":207018554}]},"public_addr":"172.21.15.189:6817/207018554","cluster_addr":"172.21.15.189:6819/207018554","heartbeat_back_addr":"172.21.15.189:6823/207018554","heartbeat_front_addr":"172.21.15.189:6821/207018554","state":["exists","up"]},{"osd":9,"uuid":"baf19b98-881b-4d3a-9f72-8aa2dd73cad6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":519636751},{"type":"v1","addr":"172.21.15.83:6827","nonce":519636751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":519636751},{"type":"v1","addr":"172.21.15.83:6829","nonce":519636751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":519636751},{"type":"v1","addr":"172.21.15.83:6833","nonce":519636751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":519636751},{"type":"v1","addr":"172.21.15.83:6831","nonce":519636751}]},"public_addr":"172.21.15.83:6827/519636751","cluster_addr":"172.21.15.83:6829/519636751","heartbeat_back_addr":"172.21.15.83:6833/519636751","heartbeat_front_addr":"172.21.15.83:6831/519636751","state":["exists","up"]},{"osd":10,"uuid":"eb346717-5c07-47c8-95be-0e68fe449fdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1130872098},{"type":"v1","addr":"172.21.15.138:6825","nonce":1130872098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1130872098},{"type":"v1","addr":"172.21.15.138:6827","nonce":1130872098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1130872098},{"type":"v1","addr":"172.21.15.138:6831","nonce":1130872098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1130872098},{"type":"v1","addr":"172.21.15.138:6829","nonce":1130872098}]},"public_addr":"172.21.15.138:6825/1130872098","cluster_addr":"172.21.15.138:6827/1130872098","heartbeat_back_addr":"172.21.15.138:6831/1130872098","heartbeat_front_addr":"172.21.15.138:6829/1130872098","state":["exists","up"]},{"osd":11,"uuid":"8e37568e-981c-45b8-b2af-2f1b73898542","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":2998782690},{"type":"v1","addr":"172.21.15.189:6825","nonce":2998782690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":2998782690},{"type":"v1","addr":"172.21.15.189:6827","nonce":2998782690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":2998782690},{"type":"v1","addr":"172.21.15.189:6831","nonce":2998782690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":2998782690},{"type":"v1","addr":"172.21.15.189:6829","nonce":2998782690}]},"public_addr":"172.21.15.189:6825/2998782690","cluster_addr":"172.21.15.189:6827/2998782690","heartbeat_back_addr":"172.21.15.189:6831/2998782690","heartbeat_front_addr":"172.21.15.189:6829/2998782690","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:40:47.980103+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:41:21.970795+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:41:55.160426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:42:30.400475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:43:05.705663+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:43:40.112312+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:44:16.781289+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:44:52.151187+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:45:28.055944+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:46:04.969301+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:46:40.918306+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:47:16.886874+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:6801/820049883":"2024-04-25T08:38:40.661315+0000","172.21.15.83:0/4272874138":"2024-04-25T08:38:40.661315+0000","172.21.15.83:0/4212994335":"2024-04-25T08:38:40.661315+0000","172.21.15.83:6801/1996371091":"2024-04-25T08:38:05.800856+0000","172.21.15.83:0/1820663683":"2024-04-25T08:38:05.800856+0000","172.21.15.83:0/3802897262":"2024-04-25T08:38:05.800856+0000","172.21.15.83:6800/1996371091":"2024-04-25T08:38:05.800856+0000","172.21.15.83:6800/820049883":"2024-04-25T08:38:40.661315+0000","172.21.15.83:0/1366125049":"2024-04-25T08:38:05.800856+0000","172.21.15.83:0/340452868":"2024-04-25T08:38:40.661315+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T08:49:06.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:06.535+0000 7fb972a00640 1 -- 172.21.15.83:0/4265399431 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fb9500a3cc0 msgr2=0x7fb9500a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:06.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:06.535+0000 7fb972a00640 1 --2- 172.21.15.83:0/4265399431 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fb9500a3cc0 0x7fb9500a6180 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb95c005fd0 tx=0x7fb95c005ce0 comp rx=0 tx=0).stop 2024-04-24T08:49:06.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:06.536+0000 7fb972a00640 1 -- 172.21.15.83:0/4265399431 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb96c105450 msgr2=0x7fb96c19a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:06.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:06.536+0000 7fb972a00640 1 --2- 172.21.15.83:0/4265399431 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb96c105450 0x7fb96c19a930 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fb9580314e0 tx=0x7fb958003c80 comp rx=0 tx=0).stop 2024-04-24T08:49:06.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:06.536+0000 7fb972a00640 1 -- 172.21.15.83:0/4265399431 shutdown_connections 2024-04-24T08:49:06.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:06.536+0000 7fb972a00640 1 --2- 172.21.15.83:0/4265399431 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb96c105e10 0x7fb96c1a19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:06.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:06.536+0000 7fb972a00640 1 --2- 172.21.15.83:0/4265399431 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fb9500a3cc0 0x7fb9500a6180 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:06.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:06.536+0000 7fb972a00640 1 --2- 172.21.15.83:0/4265399431 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb96c105450 0x7fb96c19a930 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:06.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:06.536+0000 7fb972a00640 1 --2- 172.21.15.83:0/4265399431 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb96c104260 0x7fb96c19a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:06.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:06.536+0000 7fb972a00640 1 -- 172.21.15.83:0/4265399431 >> 172.21.15.83:0/4265399431 conn(0x7fb96c0ff7f0 msgr2=0x7fb96c1086f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:49:06.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:06.536+0000 7fb972a00640 1 -- 172.21.15.83:0/4265399431 shutdown_connections 2024-04-24T08:49:06.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:06.536+0000 7fb972a00640 1 -- 172.21.15.83:0/4265399431 wait complete. 2024-04-24T08:49:06.552 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:49:06 smithi083 ceph-mon[27907]: from='client.14955 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:49:06.552 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:49:06 smithi083 ceph-mon[27907]: mgrmap e17: x(active, since 10m), standbys: z, y 2024-04-24T08:49:06.552 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:49:06 smithi083 ceph-mon[27907]: from='client.14961 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:49:06.552 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:49:06 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/2024030723' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:49:06.556 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-24T08:49:06.557 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T08:49:06.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:49:06 smithi138 ceph-mon[31441]: from='client.14955 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:49:06.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:49:06 smithi138 ceph-mon[31441]: mgrmap e17: x(active, since 10m), standbys: z, y 2024-04-24T08:49:06.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:49:06 smithi138 ceph-mon[31441]: from='client.14961 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:49:06.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:49:06 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/2024030723' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:49:06.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:49:06 smithi189 ceph-mon[31791]: from='client.14955 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:49:06.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:49:06 smithi189 ceph-mon[31791]: mgrmap e17: x(active, since 10m), standbys: z, y 2024-04-24T08:49:06.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:49:06 smithi189 ceph-mon[31791]: from='client.14961 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:49:06.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:49:06 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/2024030723' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:49:06.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:06.757+0000 7f739ba00640 1 -- 172.21.15.83:0/3419344147 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7394108660 msgr2=0x7f7394108a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:06.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:06.757+0000 7f739ba00640 1 --2- 172.21.15.83:0/3419344147 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7394108660 0x7f7394108a80 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f7384009900 tx=0x7f738402a3f0 comp rx=0 tx=0).stop 2024-04-24T08:49:06.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:06.757+0000 7f739ba00640 1 -- 172.21.15.83:0/3419344147 shutdown_connections 2024-04-24T08:49:06.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:06.758+0000 7f739ba00640 1 --2- 172.21.15.83:0/3419344147 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f739410a210 0x7f7394110aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:06.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:06.758+0000 7f739ba00640 1 --2- 172.21.15.83:0/3419344147 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7394109850 0x7f7394109cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:06.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:06.758+0000 7f739ba00640 1 --2- 172.21.15.83:0/3419344147 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7394108660 0x7f7394108a80 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:06.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:06.758+0000 7f739ba00640 1 -- 172.21.15.83:0/3419344147 >> 172.21.15.83:0/3419344147 conn(0x7f7394077620 msgr2=0x7f7394079a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:49:06.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:06.758+0000 7f739ba00640 1 -- 172.21.15.83:0/3419344147 shutdown_connections 2024-04-24T08:49:06.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:06.758+0000 7f739ba00640 1 -- 172.21.15.83:0/3419344147 wait complete. 2024-04-24T08:49:06.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:06.759+0000 7f739ba00640 1 Processor -- start 2024-04-24T08:49:06.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:06.759+0000 7f739ba00640 1 -- start start 2024-04-24T08:49:06.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:06.759+0000 7f739ba00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7394108660 0x7f739419e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:06.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:06.759+0000 7f739ba00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7394109850 0x7f739419edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:06.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:06.759+0000 7f739ba00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f739410a210 0x7f73941a5e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:06.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:06.759+0000 7f739ba00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f73941141e0 con 0x7f739410a210 2024-04-24T08:49:06.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:06.759+0000 7f739ba00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7394114060 con 0x7f7394108660 2024-04-24T08:49:06.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:06.759+0000 7f739ba00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7394114360 con 0x7f7394109850 2024-04-24T08:49:06.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:06.759+0000 7f7399a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f739410a210 0x7f73941a5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:06.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:06.759+0000 7f7399a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f739410a210 0x7f73941a5e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57818/0 (socket says 172.21.15.83:57818) 2024-04-24T08:49:06.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:06.759+0000 7f7399a00640 1 -- 172.21.15.83:0/1073567724 learned_addr learned my addr 172.21.15.83:0/1073567724 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:49:06.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:06.760+0000 7f738be00640 1 --2- 172.21.15.83:0/1073567724 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7394109850 0x7f739419edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:06.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:06.760+0000 7f7399000640 1 --2- 172.21.15.83:0/1073567724 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7394108660 0x7f739419e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:06.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:06.760+0000 7f7399a00640 1 -- 172.21.15.83:0/1073567724 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7394109850 msgr2=0x7f739419edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:06.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:06.760+0000 7f7399a00640 1 --2- 172.21.15.83:0/1073567724 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7394109850 0x7f739419edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:06.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:06.760+0000 7f7399a00640 1 -- 172.21.15.83:0/1073567724 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7394108660 msgr2=0x7f739419e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:06.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:06.760+0000 7f7399a00640 1 --2- 172.21.15.83:0/1073567724 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7394108660 0x7f739419e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:06.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:06.760+0000 7f7399a00640 1 -- 172.21.15.83:0/1073567724 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73941a64d0 con 0x7f739410a210 2024-04-24T08:49:06.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:06.760+0000 7f7399a00640 1 --2- 172.21.15.83:0/1073567724 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f739410a210 0x7f73941a5e60 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f738c00eaa0 tx=0x7f738c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:49:06.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:06.760+0000 7f7392000640 1 -- 172.21.15.83:0/1073567724 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f738c00cdb0 con 0x7f739410a210 2024-04-24T08:49:06.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:06.760+0000 7f7392000640 1 -- 172.21.15.83:0/1073567724 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f738c005020 con 0x7f739410a210 2024-04-24T08:49:06.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:06.760+0000 7f739ba00640 1 -- 172.21.15.83:0/1073567724 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73941a67c0 con 0x7f739410a210 2024-04-24T08:49:06.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:06.760+0000 7f7392000640 1 -- 172.21.15.83:0/1073567724 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f738c0116d0 con 0x7f739410a210 2024-04-24T08:49:06.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:06.760+0000 7f739ba00640 1 -- 172.21.15.83:0/1073567724 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73941a6d60 con 0x7f739410a210 2024-04-24T08:49:06.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:06.762+0000 7f739ba00640 1 -- 172.21.15.83:0/1073567724 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f739410f980 con 0x7f739410a210 2024-04-24T08:49:06.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:06.763+0000 7f7392000640 1 -- 172.21.15.83:0/1073567724 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f738c0051d0 con 0x7f739410a210 2024-04-24T08:49:06.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:06.765+0000 7f7392000640 1 --2- 172.21.15.83:0/1073567724 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f73780a3cc0 0x7f73780a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:06.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:06.765+0000 7f7392000640 1 -- 172.21.15.83:0/1073567724 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f738c0d0a20 con 0x7f739410a210 2024-04-24T08:49:06.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:06.765+0000 7f7399000640 1 --2- 172.21.15.83:0/1073567724 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f73780a3cc0 0x7f73780a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:06.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:06.766+0000 7f7399000640 1 --2- 172.21.15.83:0/1073567724 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f73780a3cc0 0x7f73780a6180 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f7384006010 tx=0x7f73840314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:49:06.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:06.769+0000 7f7392000640 1 -- 172.21.15.83:0/1073567724 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f738c016030 con 0x7f739410a210 2024-04-24T08:49:06.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:06.969+0000 7f739ba00640 1 -- 172.21.15.83:0/1073567724 --> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7394062440 con 0x7f73780a3cc0 2024-04-24T08:49:07.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:07.004+0000 7f7392000640 1 -- 172.21.15.83:0/1073567724 <== mgr.14152 v2:172.21.15.83:6800/2550826560 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340459 (secure 0 0 0) 0x7f7394062440 con 0x7f73780a3cc0 2024-04-24T08:49:07.008 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:49:07.012 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-04-24T08:49:07.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:07.012+0000 7f739ba00640 1 -- 172.21.15.83:0/1073567724 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f73780a3cc0 msgr2=0x7f73780a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:07.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:07.012+0000 7f739ba00640 1 --2- 172.21.15.83:0/1073567724 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f73780a3cc0 0x7f73780a6180 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f7384006010 tx=0x7f73840314a0 comp rx=0 tx=0).stop 2024-04-24T08:49:07.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:07.012+0000 7f739ba00640 1 -- 172.21.15.83:0/1073567724 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f739410a210 msgr2=0x7f73941a5e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:07.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:07.012+0000 7f739ba00640 1 --2- 172.21.15.83:0/1073567724 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f739410a210 0x7f73941a5e60 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f738c00eaa0 tx=0x7f738c00ef60 comp rx=0 tx=0).stop 2024-04-24T08:49:07.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:07.012+0000 7f739ba00640 1 -- 172.21.15.83:0/1073567724 shutdown_connections 2024-04-24T08:49:07.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:07.012+0000 7f739ba00640 1 --2- 172.21.15.83:0/1073567724 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f73780a3cc0 0x7f73780a6180 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:07.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:07.012+0000 7f739ba00640 1 --2- 172.21.15.83:0/1073567724 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f739410a210 0x7f73941a5e60 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:07.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:07.012+0000 7f739ba00640 1 --2- 172.21.15.83:0/1073567724 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7394109850 0x7f739419edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:07.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:07.012+0000 7f739ba00640 1 --2- 172.21.15.83:0/1073567724 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7394108660 0x7f739419e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:07.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:07.012+0000 7f739ba00640 1 -- 172.21.15.83:0/1073567724 >> 172.21.15.83:0/1073567724 conn(0x7f7394077620 msgr2=0x7f739410cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:49:07.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:07.012+0000 7f739ba00640 1 -- 172.21.15.83:0/1073567724 shutdown_connections 2024-04-24T08:49:07.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:07.012+0000 7f739ba00640 1 -- 172.21.15.83:0/1073567724 wait complete. 2024-04-24T08:49:07.035 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":392,"stamp":"2024-04-24T08:49:06.847852+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":599768,"num_objects":78,"num_object_clones":0,"num_object_copies":235,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":0,"num_read":156,"num_read_kb":135,"num_write":344,"num_write_kb":2994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":50,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":283,"ondisk_log_size":283,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":379364,"kb_used_data":13572,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124562460,"statfs":{"total":1151940427776,"available":1151551959040,"internally_reserved":0,"allocated":13897728,"data_stored":7539652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":374453561},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4760,"num_objects":54,"num_object_clones":0,"num_object_copies":163,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":20,"num_read_kb":16,"num_write":114,"num_write_kb":37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006378"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:35.326280+0000","last_change":"2024-04-24T08:48:29.993009+0000","last_active":"2024-04-24T08:48:35.326280+0000","last_peered":"2024-04-24T08:48:35.326280+0000","last_clean":"2024-04-24T08:48:35.326280+0000","last_became_active":"2024-04-24T08:48:29.990613+0000","last_became_peered":"2024-04-24T08:48:29.990613+0000","last_unstale":"2024-04-24T08:48:35.326280+0000","last_undegraded":"2024-04-24T08:48:35.326280+0000","last_fullsized":"2024-04-24T08:48:35.326280+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:13:43.871742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.498605+0000","last_change":"2024-04-24T08:48:29.986957+0000","last_active":"2024-04-24T08:48:36.498605+0000","last_peered":"2024-04-24T08:48:36.498605+0000","last_clean":"2024-04-24T08:48:36.498605+0000","last_became_active":"2024-04-24T08:48:29.984691+0000","last_became_peered":"2024-04-24T08:48:29.984691+0000","last_unstale":"2024-04-24T08:48:36.498605+0000","last_undegraded":"2024-04-24T08:48:36.498605+0000","last_fullsized":"2024-04-24T08:48:36.498605+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:07:38.031213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.498428+0000","last_change":"2024-04-24T08:48:29.991478+0000","last_active":"2024-04-24T08:48:36.498428+0000","last_peered":"2024-04-24T08:48:36.498428+0000","last_clean":"2024-04-24T08:48:36.498428+0000","last_became_active":"2024-04-24T08:48:29.991013+0000","last_became_peered":"2024-04-24T08:48:29.991013+0000","last_unstale":"2024-04-24T08:48:36.498428+0000","last_undegraded":"2024-04-24T08:48:36.498428+0000","last_fullsized":"2024-04-24T08:48:36.498428+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:10:39.381353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.501016+0000","last_change":"2024-04-24T08:48:29.983664+0000","last_active":"2024-04-24T08:48:36.501016+0000","last_peered":"2024-04-24T08:48:36.501016+0000","last_clean":"2024-04-24T08:48:36.501016+0000","last_became_active":"2024-04-24T08:48:29.978835+0000","last_became_peered":"2024-04-24T08:48:29.978835+0000","last_unstale":"2024-04-24T08:48:36.501016+0000","last_undegraded":"2024-04-24T08:48:36.501016+0000","last_fullsized":"2024-04-24T08:48:36.501016+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:29:20.501652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.498099+0000","last_change":"2024-04-24T08:48:29.989886+0000","last_active":"2024-04-24T08:48:36.498099+0000","last_peered":"2024-04-24T08:48:36.498099+0000","last_clean":"2024-04-24T08:48:36.498099+0000","last_became_active":"2024-04-24T08:48:29.989574+0000","last_became_peered":"2024-04-24T08:48:29.989574+0000","last_unstale":"2024-04-24T08:48:36.498099+0000","last_undegraded":"2024-04-24T08:48:36.498099+0000","last_fullsized":"2024-04-24T08:48:36.498099+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:14:01.085543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.501305+0000","last_change":"2024-04-24T08:48:29.978045+0000","last_active":"2024-04-24T08:48:36.501305+0000","last_peered":"2024-04-24T08:48:36.501305+0000","last_clean":"2024-04-24T08:48:36.501305+0000","last_became_active":"2024-04-24T08:48:29.977729+0000","last_became_peered":"2024-04-24T08:48:29.977729+0000","last_unstale":"2024-04-24T08:48:36.501305+0000","last_undegraded":"2024-04-24T08:48:36.501305+0000","last_fullsized":"2024-04-24T08:48:36.501305+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:03:33.382473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:35.290755+0000","last_change":"2024-04-24T08:48:29.976355+0000","last_active":"2024-04-24T08:48:35.290755+0000","last_peered":"2024-04-24T08:48:35.290755+0000","last_clean":"2024-04-24T08:48:35.290755+0000","last_became_active":"2024-04-24T08:48:29.975849+0000","last_became_peered":"2024-04-24T08:48:29.975849+0000","last_unstale":"2024-04-24T08:48:35.290755+0000","last_undegraded":"2024-04-24T08:48:35.290755+0000","last_fullsized":"2024-04-24T08:48:35.290755+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:13:51.302995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.498709+0000","last_change":"2024-04-24T08:48:29.983149+0000","last_active":"2024-04-24T08:48:36.498709+0000","last_peered":"2024-04-24T08:48:36.498709+0000","last_clean":"2024-04-24T08:48:36.498709+0000","last_became_active":"2024-04-24T08:48:29.982692+0000","last_became_peered":"2024-04-24T08:48:29.982692+0000","last_unstale":"2024-04-24T08:48:36.498709+0000","last_undegraded":"2024-04-24T08:48:36.498709+0000","last_fullsized":"2024-04-24T08:48:36.498709+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:59:06.151441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.503665+0000","last_change":"2024-04-24T08:48:29.985531+0000","last_active":"2024-04-24T08:48:36.503665+0000","last_peered":"2024-04-24T08:48:36.503665+0000","last_clean":"2024-04-24T08:48:36.503665+0000","last_became_active":"2024-04-24T08:48:29.982484+0000","last_became_peered":"2024-04-24T08:48:29.982484+0000","last_unstale":"2024-04-24T08:48:36.503665+0000","last_undegraded":"2024-04-24T08:48:36.503665+0000","last_fullsized":"2024-04-24T08:48:36.503665+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:03:50.114096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.502970+0000","last_change":"2024-04-24T08:48:29.982016+0000","last_active":"2024-04-24T08:48:36.502970+0000","last_peered":"2024-04-24T08:48:36.502970+0000","last_clean":"2024-04-24T08:48:36.502970+0000","last_became_active":"2024-04-24T08:48:29.977639+0000","last_became_peered":"2024-04-24T08:48:29.977639+0000","last_unstale":"2024-04-24T08:48:36.502970+0000","last_undegraded":"2024-04-24T08:48:36.502970+0000","last_fullsized":"2024-04-24T08:48:36.502970+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:35:48.676822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:35.292353+0000","last_change":"2024-04-24T08:48:29.977982+0000","last_active":"2024-04-24T08:48:35.292353+0000","last_peered":"2024-04-24T08:48:35.292353+0000","last_clean":"2024-04-24T08:48:35.292353+0000","last_became_active":"2024-04-24T08:48:29.977664+0000","last_became_peered":"2024-04-24T08:48:29.977664+0000","last_unstale":"2024-04-24T08:48:35.292353+0000","last_undegraded":"2024-04-24T08:48:35.292353+0000","last_fullsized":"2024-04-24T08:48:35.292353+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:45:17.597291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.500944+0000","last_change":"2024-04-24T08:48:29.981615+0000","last_active":"2024-04-24T08:48:36.500944+0000","last_peered":"2024-04-24T08:48:36.500944+0000","last_clean":"2024-04-24T08:48:36.500944+0000","last_became_active":"2024-04-24T08:48:29.979292+0000","last_became_peered":"2024-04-24T08:48:29.979292+0000","last_unstale":"2024-04-24T08:48:36.500944+0000","last_undegraded":"2024-04-24T08:48:36.500944+0000","last_fullsized":"2024-04-24T08:48:36.500944+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:48:44.195626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.499761+0000","last_change":"2024-04-24T08:48:29.952632+0000","last_active":"2024-04-24T08:48:36.499761+0000","last_peered":"2024-04-24T08:48:36.499761+0000","last_clean":"2024-04-24T08:48:36.499761+0000","last_became_active":"2024-04-24T08:48:29.951862+0000","last_became_peered":"2024-04-24T08:48:29.951862+0000","last_unstale":"2024-04-24T08:48:36.499761+0000","last_undegraded":"2024-04-24T08:48:36.499761+0000","last_fullsized":"2024-04-24T08:48:36.499761+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:23:40.038517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.497179+0000","last_change":"2024-04-24T08:48:29.984727+0000","last_active":"2024-04-24T08:48:36.497179+0000","last_peered":"2024-04-24T08:48:36.497179+0000","last_clean":"2024-04-24T08:48:36.497179+0000","last_became_active":"2024-04-24T08:48:29.983530+0000","last_became_peered":"2024-04-24T08:48:29.983530+0000","last_unstale":"2024-04-24T08:48:36.497179+0000","last_undegraded":"2024-04-24T08:48:36.497179+0000","last_fullsized":"2024-04-24T08:48:36.497179+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:00:12.729972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.497226+0000","last_change":"2024-04-24T08:48:29.981023+0000","last_active":"2024-04-24T08:48:36.497226+0000","last_peered":"2024-04-24T08:48:36.497226+0000","last_clean":"2024-04-24T08:48:36.497226+0000","last_became_active":"2024-04-24T08:48:29.979986+0000","last_became_peered":"2024-04-24T08:48:29.979986+0000","last_unstale":"2024-04-24T08:48:36.497226+0000","last_undegraded":"2024-04-24T08:48:36.497226+0000","last_fullsized":"2024-04-24T08:48:36.497226+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:05:36.163209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.499105+0000","last_change":"2024-04-24T08:48:29.989036+0000","last_active":"2024-04-24T08:48:36.499105+0000","last_peered":"2024-04-24T08:48:36.499105+0000","last_clean":"2024-04-24T08:48:36.499105+0000","last_became_active":"2024-04-24T08:48:29.988676+0000","last_became_peered":"2024-04-24T08:48:29.988676+0000","last_unstale":"2024-04-24T08:48:36.499105+0000","last_undegraded":"2024-04-24T08:48:36.499105+0000","last_fullsized":"2024-04-24T08:48:36.499105+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:03:38.544596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:35.326627+0000","last_change":"2024-04-24T08:48:29.986424+0000","last_active":"2024-04-24T08:48:35.326627+0000","last_peered":"2024-04-24T08:48:35.326627+0000","last_clean":"2024-04-24T08:48:35.326627+0000","last_became_active":"2024-04-24T08:48:29.985973+0000","last_became_peered":"2024-04-24T08:48:29.985973+0000","last_unstale":"2024-04-24T08:48:35.326627+0000","last_undegraded":"2024-04-24T08:48:35.326627+0000","last_fullsized":"2024-04-24T08:48:35.326627+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:48:31.533282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:35.290609+0000","last_change":"2024-04-24T08:48:29.975470+0000","last_active":"2024-04-24T08:48:35.290609+0000","last_peered":"2024-04-24T08:48:35.290609+0000","last_clean":"2024-04-24T08:48:35.290609+0000","last_became_active":"2024-04-24T08:48:29.974777+0000","last_became_peered":"2024-04-24T08:48:29.974777+0000","last_unstale":"2024-04-24T08:48:35.290609+0000","last_undegraded":"2024-04-24T08:48:35.290609+0000","last_fullsized":"2024-04-24T08:48:35.290609+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:04:26.423840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:35.291868+0000","last_change":"2024-04-24T08:48:29.988001+0000","last_active":"2024-04-24T08:48:35.291868+0000","last_peered":"2024-04-24T08:48:35.291868+0000","last_clean":"2024-04-24T08:48:35.291868+0000","last_became_active":"2024-04-24T08:48:29.987669+0000","last_became_peered":"2024-04-24T08:48:29.987669+0000","last_unstale":"2024-04-24T08:48:35.291868+0000","last_undegraded":"2024-04-24T08:48:35.291868+0000","last_fullsized":"2024-04-24T08:48:35.291868+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:29:24.457369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:35.292296+0000","last_change":"2024-04-24T08:48:34.308732+0000","last_active":"2024-04-24T08:48:35.292296+0000","last_peered":"2024-04-24T08:48:35.292296+0000","last_clean":"2024-04-24T08:48:35.292296+0000","last_became_active":"2024-04-24T08:48:34.307598+0000","last_became_peered":"2024-04-24T08:48:34.307598+0000","last_unstale":"2024-04-24T08:48:35.292296+0000","last_undegraded":"2024-04-24T08:48:35.292296+0000","last_fullsized":"2024-04-24T08:48:35.292296+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:28:57.260836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'9","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:49:02.055766+0000","last_change":"2024-04-24T08:48:28.970230+0000","last_active":"2024-04-24T08:49:02.055766+0000","last_peered":"2024-04-24T08:49:02.055766+0000","last_clean":"2024-04-24T08:49:02.055766+0000","last_became_active":"2024-04-24T08:48:28.969878+0000","last_became_peered":"2024-04-24T08:48:28.969878+0000","last_unstale":"2024-04-24T08:49:02.055766+0000","last_undegraded":"2024-04-24T08:49:02.055766+0000","last_fullsized":"2024-04-24T08:49:02.055766+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:05:19.206557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.500894+0000","last_change":"2024-04-24T08:48:29.986350+0000","last_active":"2024-04-24T08:48:36.500894+0000","last_peered":"2024-04-24T08:48:36.500894+0000","last_clean":"2024-04-24T08:48:36.500894+0000","last_became_active":"2024-04-24T08:48:29.985167+0000","last_became_peered":"2024-04-24T08:48:29.985167+0000","last_unstale":"2024-04-24T08:48:36.500894+0000","last_undegraded":"2024-04-24T08:48:36.500894+0000","last_fullsized":"2024-04-24T08:48:36.500894+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:43:33.326808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.498889+0000","last_change":"2024-04-24T08:48:34.315159+0000","last_active":"2024-04-24T08:48:36.498889+0000","last_peered":"2024-04-24T08:48:36.498889+0000","last_clean":"2024-04-24T08:48:36.498889+0000","last_became_active":"2024-04-24T08:48:34.312119+0000","last_became_peered":"2024-04-24T08:48:34.312119+0000","last_unstale":"2024-04-24T08:48:36.498889+0000","last_undegraded":"2024-04-24T08:48:36.498889+0000","last_fullsized":"2024-04-24T08:48:36.498889+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:56:08.024123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.501331+0000","last_change":"2024-04-24T08:48:28.971787+0000","last_active":"2024-04-24T08:48:36.501331+0000","last_peered":"2024-04-24T08:48:36.501331+0000","last_clean":"2024-04-24T08:48:36.501331+0000","last_became_active":"2024-04-24T08:48:28.971479+0000","last_became_peered":"2024-04-24T08:48:28.971479+0000","last_unstale":"2024-04-24T08:48:36.501331+0000","last_undegraded":"2024-04-24T08:48:36.501331+0000","last_fullsized":"2024-04-24T08:48:36.501331+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:09:52.498833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.498297+0000","last_change":"2024-04-24T08:48:29.989737+0000","last_active":"2024-04-24T08:48:36.498297+0000","last_peered":"2024-04-24T08:48:36.498297+0000","last_clean":"2024-04-24T08:48:36.498297+0000","last_became_active":"2024-04-24T08:48:29.989341+0000","last_became_peered":"2024-04-24T08:48:29.989341+0000","last_unstale":"2024-04-24T08:48:36.498297+0000","last_undegraded":"2024-04-24T08:48:36.498297+0000","last_fullsized":"2024-04-24T08:48:36.498297+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:02:32.714848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.499895+0000","last_change":"2024-04-24T08:48:34.322666+0000","last_active":"2024-04-24T08:48:36.499895+0000","last_peered":"2024-04-24T08:48:36.499895+0000","last_clean":"2024-04-24T08:48:36.499895+0000","last_became_active":"2024-04-24T08:48:34.321546+0000","last_became_peered":"2024-04-24T08:48:34.321546+0000","last_unstale":"2024-04-24T08:48:36.499895+0000","last_undegraded":"2024-04-24T08:48:36.499895+0000","last_fullsized":"2024-04-24T08:48:36.499895+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:42:45.066043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'11","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:49:04.102263+0000","last_change":"2024-04-24T08:48:28.978738+0000","last_active":"2024-04-24T08:49:04.102263+0000","last_peered":"2024-04-24T08:49:04.102263+0000","last_clean":"2024-04-24T08:49:04.102263+0000","last_became_active":"2024-04-24T08:48:28.978378+0000","last_became_peered":"2024-04-24T08:48:28.978378+0000","last_unstale":"2024-04-24T08:49:04.102263+0000","last_undegraded":"2024-04-24T08:49:04.102263+0000","last_fullsized":"2024-04-24T08:49:04.102263+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:46:22.668234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.501908+0000","last_change":"2024-04-24T08:48:29.974588+0000","last_active":"2024-04-24T08:48:36.501908+0000","last_peered":"2024-04-24T08:48:36.501908+0000","last_clean":"2024-04-24T08:48:36.501908+0000","last_became_active":"2024-04-24T08:48:29.974190+0000","last_became_peered":"2024-04-24T08:48:29.974190+0000","last_unstale":"2024-04-24T08:48:36.501908+0000","last_undegraded":"2024-04-24T08:48:36.501908+0000","last_fullsized":"2024-04-24T08:48:36.501908+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:07:12.948660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.499360+0000","last_change":"2024-04-24T08:48:34.321954+0000","last_active":"2024-04-24T08:48:36.499360+0000","last_peered":"2024-04-24T08:48:36.499360+0000","last_clean":"2024-04-24T08:48:36.499360+0000","last_became_active":"2024-04-24T08:48:34.321417+0000","last_became_peered":"2024-04-24T08:48:34.321417+0000","last_unstale":"2024-04-24T08:48:36.499360+0000","last_undegraded":"2024-04-24T08:48:36.499360+0000","last_fullsized":"2024-04-24T08:48:36.499360+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:39:07.205709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:49:04.097612+0000","last_change":"2024-04-24T08:48:28.974464+0000","last_active":"2024-04-24T08:49:04.097612+0000","last_peered":"2024-04-24T08:49:04.097612+0000","last_clean":"2024-04-24T08:49:04.097612+0000","last_became_active":"2024-04-24T08:48:28.974001+0000","last_became_peered":"2024-04-24T08:48:28.974001+0000","last_unstale":"2024-04-24T08:49:04.097612+0000","last_undegraded":"2024-04-24T08:49:04.097612+0000","last_fullsized":"2024-04-24T08:49:04.097612+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:38:55.193427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.063853+0000","last_change":"2024-04-24T08:48:29.981336+0000","last_active":"2024-04-24T08:48:36.063853+0000","last_peered":"2024-04-24T08:48:36.063853+0000","last_clean":"2024-04-24T08:48:36.063853+0000","last_became_active":"2024-04-24T08:48:29.980333+0000","last_became_peered":"2024-04-24T08:48:29.980333+0000","last_unstale":"2024-04-24T08:48:36.063853+0000","last_undegraded":"2024-04-24T08:48:36.063853+0000","last_fullsized":"2024-04-24T08:48:36.063853+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:46:03.640332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:35.291679+0000","last_change":"2024-04-24T08:48:34.323099+0000","last_active":"2024-04-24T08:48:35.291679+0000","last_peered":"2024-04-24T08:48:35.291679+0000","last_clean":"2024-04-24T08:48:35.291679+0000","last_became_active":"2024-04-24T08:48:34.322557+0000","last_became_peered":"2024-04-24T08:48:34.322557+0000","last_unstale":"2024-04-24T08:48:35.291679+0000","last_undegraded":"2024-04-24T08:48:35.291679+0000","last_fullsized":"2024-04-24T08:48:35.291679+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:59:59.887087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'6","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:49:05.241898+0000","last_change":"2024-04-24T08:48:28.978427+0000","last_active":"2024-04-24T08:49:05.241898+0000","last_peered":"2024-04-24T08:49:05.241898+0000","last_clean":"2024-04-24T08:49:05.241898+0000","last_became_active":"2024-04-24T08:48:28.977291+0000","last_became_peered":"2024-04-24T08:48:28.977291+0000","last_unstale":"2024-04-24T08:49:05.241898+0000","last_undegraded":"2024-04-24T08:49:05.241898+0000","last_fullsized":"2024-04-24T08:49:05.241898+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:03:03.085265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":2,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.498491+0000","last_change":"2024-04-24T08:48:29.978008+0000","last_active":"2024-04-24T08:48:36.498491+0000","last_peered":"2024-04-24T08:48:36.498491+0000","last_clean":"2024-04-24T08:48:36.498491+0000","last_became_active":"2024-04-24T08:48:29.977389+0000","last_became_peered":"2024-04-24T08:48:29.977389+0000","last_unstale":"2024-04-24T08:48:36.498491+0000","last_undegraded":"2024-04-24T08:48:36.498491+0000","last_fullsized":"2024-04-24T08:48:36.498491+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:35:28.035378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:35.291612+0000","last_change":"2024-04-24T08:48:34.321477+0000","last_active":"2024-04-24T08:48:35.291612+0000","last_peered":"2024-04-24T08:48:35.291612+0000","last_clean":"2024-04-24T08:48:35.291612+0000","last_became_active":"2024-04-24T08:48:34.320860+0000","last_became_peered":"2024-04-24T08:48:34.320860+0000","last_unstale":"2024-04-24T08:48:35.291612+0000","last_undegraded":"2024-04-24T08:48:35.291612+0000","last_fullsized":"2024-04-24T08:48:35.291612+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:26:07.862590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:49:03.074625+0000","last_change":"2024-04-24T08:48:28.978147+0000","last_active":"2024-04-24T08:49:03.074625+0000","last_peered":"2024-04-24T08:49:03.074625+0000","last_clean":"2024-04-24T08:49:03.074625+0000","last_became_active":"2024-04-24T08:48:28.977646+0000","last_became_peered":"2024-04-24T08:48:28.977646+0000","last_unstale":"2024-04-24T08:49:03.074625+0000","last_undegraded":"2024-04-24T08:49:03.074625+0000","last_fullsized":"2024-04-24T08:49:03.074625+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:31:35.557886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.064424+0000","last_change":"2024-04-24T08:48:29.990788+0000","last_active":"2024-04-24T08:48:36.064424+0000","last_peered":"2024-04-24T08:48:36.064424+0000","last_clean":"2024-04-24T08:48:36.064424+0000","last_became_active":"2024-04-24T08:48:29.988476+0000","last_became_peered":"2024-04-24T08:48:29.988476+0000","last_unstale":"2024-04-24T08:48:36.064424+0000","last_undegraded":"2024-04-24T08:48:36.064424+0000","last_fullsized":"2024-04-24T08:48:36.064424+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:08:48.389877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:35.326254+0000","last_change":"2024-04-24T08:48:34.311020+0000","last_active":"2024-04-24T08:48:35.326254+0000","last_peered":"2024-04-24T08:48:35.326254+0000","last_clean":"2024-04-24T08:48:35.326254+0000","last_became_active":"2024-04-24T08:48:34.310395+0000","last_became_peered":"2024-04-24T08:48:34.310395+0000","last_unstale":"2024-04-24T08:48:35.326254+0000","last_undegraded":"2024-04-24T08:48:35.326254+0000","last_fullsized":"2024-04-24T08:48:35.326254+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:55:38.704954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:49:02.051829+0000","last_change":"2024-04-24T08:48:28.969991+0000","last_active":"2024-04-24T08:49:02.051829+0000","last_peered":"2024-04-24T08:49:02.051829+0000","last_clean":"2024-04-24T08:49:02.051829+0000","last_became_active":"2024-04-24T08:48:28.967937+0000","last_became_peered":"2024-04-24T08:48:28.967937+0000","last_unstale":"2024-04-24T08:49:02.051829+0000","last_undegraded":"2024-04-24T08:49:02.051829+0000","last_fullsized":"2024-04-24T08:49:02.051829+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:09:49.736108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.498485+0000","last_change":"2024-04-24T08:48:29.992814+0000","last_active":"2024-04-24T08:48:36.498485+0000","last_peered":"2024-04-24T08:48:36.498485+0000","last_clean":"2024-04-24T08:48:36.498485+0000","last_became_active":"2024-04-24T08:48:29.992188+0000","last_became_peered":"2024-04-24T08:48:29.992188+0000","last_unstale":"2024-04-24T08:48:36.498485+0000","last_undegraded":"2024-04-24T08:48:36.498485+0000","last_fullsized":"2024-04-24T08:48:36.498485+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:29:33.974958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.498447+0000","last_change":"2024-04-24T08:48:34.311196+0000","last_active":"2024-04-24T08:48:36.498447+0000","last_peered":"2024-04-24T08:48:36.498447+0000","last_clean":"2024-04-24T08:48:36.498447+0000","last_became_active":"2024-04-24T08:48:34.310723+0000","last_became_peered":"2024-04-24T08:48:34.310723+0000","last_unstale":"2024-04-24T08:48:36.498447+0000","last_undegraded":"2024-04-24T08:48:36.498447+0000","last_fullsized":"2024-04-24T08:48:36.498447+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:04:30.149510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:49:04.102749+0000","last_change":"2024-04-24T08:48:28.978534+0000","last_active":"2024-04-24T08:49:04.102749+0000","last_peered":"2024-04-24T08:49:04.102749+0000","last_clean":"2024-04-24T08:49:04.102749+0000","last_became_active":"2024-04-24T08:48:28.975407+0000","last_became_peered":"2024-04-24T08:48:28.975407+0000","last_unstale":"2024-04-24T08:49:04.102749+0000","last_undegraded":"2024-04-24T08:49:04.102749+0000","last_fullsized":"2024-04-24T08:49:04.102749+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:10:45.355039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:35.288846+0000","last_change":"2024-04-24T08:48:29.997682+0000","last_active":"2024-04-24T08:48:35.288846+0000","last_peered":"2024-04-24T08:48:35.288846+0000","last_clean":"2024-04-24T08:48:35.288846+0000","last_became_active":"2024-04-24T08:48:29.997223+0000","last_became_peered":"2024-04-24T08:48:29.997223+0000","last_unstale":"2024-04-24T08:48:35.288846+0000","last_undegraded":"2024-04-24T08:48:35.288846+0000","last_fullsized":"2024-04-24T08:48:35.288846+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:03:00.807249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.500029+0000","last_change":"2024-04-24T08:48:34.321477+0000","last_active":"2024-04-24T08:48:36.500029+0000","last_peered":"2024-04-24T08:48:36.500029+0000","last_clean":"2024-04-24T08:48:36.500029+0000","last_became_active":"2024-04-24T08:48:34.321018+0000","last_became_peered":"2024-04-24T08:48:34.321018+0000","last_unstale":"2024-04-24T08:48:36.500029+0000","last_undegraded":"2024-04-24T08:48:36.500029+0000","last_fullsized":"2024-04-24T08:48:36.500029+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:56:50.657298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'4","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:49:01.046781+0000","last_change":"2024-04-24T08:48:28.983722+0000","last_active":"2024-04-24T08:49:01.046781+0000","last_peered":"2024-04-24T08:49:01.046781+0000","last_clean":"2024-04-24T08:49:01.046781+0000","last_became_active":"2024-04-24T08:48:28.983394+0000","last_became_peered":"2024-04-24T08:48:28.983394+0000","last_unstale":"2024-04-24T08:49:01.046781+0000","last_undegraded":"2024-04-24T08:49:01.046781+0000","last_fullsized":"2024-04-24T08:49:01.046781+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:53:14.288742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.503838+0000","last_change":"2024-04-24T08:48:29.975009+0000","last_active":"2024-04-24T08:48:36.503838+0000","last_peered":"2024-04-24T08:48:36.503838+0000","last_clean":"2024-04-24T08:48:36.503838+0000","last_became_active":"2024-04-24T08:48:29.974137+0000","last_became_peered":"2024-04-24T08:48:29.974137+0000","last_unstale":"2024-04-24T08:48:36.503838+0000","last_undegraded":"2024-04-24T08:48:36.503838+0000","last_fullsized":"2024-04-24T08:48:36.503838+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:13:16.381923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:35.288980+0000","last_change":"2024-04-24T08:48:34.312126+0000","last_active":"2024-04-24T08:48:35.288980+0000","last_peered":"2024-04-24T08:48:35.288980+0000","last_clean":"2024-04-24T08:48:35.288980+0000","last_became_active":"2024-04-24T08:48:34.311689+0000","last_became_peered":"2024-04-24T08:48:34.311689+0000","last_unstale":"2024-04-24T08:48:35.288980+0000","last_undegraded":"2024-04-24T08:48:35.288980+0000","last_fullsized":"2024-04-24T08:48:35.288980+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:11:07.942412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:49:04.098187+0000","last_change":"2024-04-24T08:48:28.978355+0000","last_active":"2024-04-24T08:49:04.098187+0000","last_peered":"2024-04-24T08:49:04.098187+0000","last_clean":"2024-04-24T08:49:04.098187+0000","last_became_active":"2024-04-24T08:48:28.977351+0000","last_became_peered":"2024-04-24T08:48:28.977351+0000","last_unstale":"2024-04-24T08:49:04.098187+0000","last_undegraded":"2024-04-24T08:49:04.098187+0000","last_fullsized":"2024-04-24T08:49:04.098187+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:00:12.836569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.499015+0000","last_change":"2024-04-24T08:48:29.989015+0000","last_active":"2024-04-24T08:48:36.499015+0000","last_peered":"2024-04-24T08:48:36.499015+0000","last_clean":"2024-04-24T08:48:36.499015+0000","last_became_active":"2024-04-24T08:48:29.988648+0000","last_became_peered":"2024-04-24T08:48:29.988648+0000","last_unstale":"2024-04-24T08:48:36.499015+0000","last_undegraded":"2024-04-24T08:48:36.499015+0000","last_fullsized":"2024-04-24T08:48:36.499015+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:04:52.824894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.500746+0000","last_change":"2024-04-24T08:48:34.314230+0000","last_active":"2024-04-24T08:48:36.500746+0000","last_peered":"2024-04-24T08:48:36.500746+0000","last_clean":"2024-04-24T08:48:36.500746+0000","last_became_active":"2024-04-24T08:48:34.313617+0000","last_became_peered":"2024-04-24T08:48:34.313617+0000","last_unstale":"2024-04-24T08:48:36.500746+0000","last_undegraded":"2024-04-24T08:48:36.500746+0000","last_fullsized":"2024-04-24T08:48:36.500746+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:06:02.307575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:49:03.071387+0000","last_change":"2024-04-24T08:48:28.974516+0000","last_active":"2024-04-24T08:49:03.071387+0000","last_peered":"2024-04-24T08:49:03.071387+0000","last_clean":"2024-04-24T08:49:03.071387+0000","last_became_active":"2024-04-24T08:48:28.974194+0000","last_became_peered":"2024-04-24T08:48:28.974194+0000","last_unstale":"2024-04-24T08:49:03.071387+0000","last_undegraded":"2024-04-24T08:49:03.071387+0000","last_fullsized":"2024-04-24T08:49:03.071387+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:03:38.582610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.502357+0000","last_change":"2024-04-24T08:48:29.981132+0000","last_active":"2024-04-24T08:48:36.502357+0000","last_peered":"2024-04-24T08:48:36.502357+0000","last_clean":"2024-04-24T08:48:36.502357+0000","last_became_active":"2024-04-24T08:48:29.980252+0000","last_became_peered":"2024-04-24T08:48:29.980252+0000","last_unstale":"2024-04-24T08:48:36.502357+0000","last_undegraded":"2024-04-24T08:48:36.502357+0000","last_fullsized":"2024-04-24T08:48:36.502357+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:43:28.632117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:35.289424+0000","last_change":"2024-04-24T08:48:34.312585+0000","last_active":"2024-04-24T08:48:35.289424+0000","last_peered":"2024-04-24T08:48:35.289424+0000","last_clean":"2024-04-24T08:48:35.289424+0000","last_became_active":"2024-04-24T08:48:34.311289+0000","last_became_peered":"2024-04-24T08:48:34.311289+0000","last_unstale":"2024-04-24T08:48:35.289424+0000","last_undegraded":"2024-04-24T08:48:35.289424+0000","last_fullsized":"2024-04-24T08:48:35.289424+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:01:20.080310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"89'12","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:49:01.044855+0000","last_change":"2024-04-24T08:48:28.981675+0000","last_active":"2024-04-24T08:49:01.044855+0000","last_peered":"2024-04-24T08:49:01.044855+0000","last_clean":"2024-04-24T08:49:01.044855+0000","last_became_active":"2024-04-24T08:48:28.981082+0000","last_became_peered":"2024-04-24T08:48:28.981082+0000","last_unstale":"2024-04-24T08:49:01.044855+0000","last_undegraded":"2024-04-24T08:49:01.044855+0000","last_fullsized":"2024-04-24T08:49:01.044855+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:22:07.826011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.498801+0000","last_change":"2024-04-24T08:48:29.986932+0000","last_active":"2024-04-24T08:48:36.498801+0000","last_peered":"2024-04-24T08:48:36.498801+0000","last_clean":"2024-04-24T08:48:36.498801+0000","last_became_active":"2024-04-24T08:48:29.986505+0000","last_became_peered":"2024-04-24T08:48:29.986505+0000","last_unstale":"2024-04-24T08:48:36.498801+0000","last_undegraded":"2024-04-24T08:48:36.498801+0000","last_fullsized":"2024-04-24T08:48:36.498801+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:53:36.050294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.064820+0000","last_change":"2024-04-24T08:48:34.327127+0000","last_active":"2024-04-24T08:48:36.064820+0000","last_peered":"2024-04-24T08:48:36.064820+0000","last_clean":"2024-04-24T08:48:36.064820+0000","last_became_active":"2024-04-24T08:48:34.323942+0000","last_became_peered":"2024-04-24T08:48:34.323942+0000","last_unstale":"2024-04-24T08:48:36.064820+0000","last_undegraded":"2024-04-24T08:48:36.064820+0000","last_fullsized":"2024-04-24T08:48:36.064820+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:22:49.637513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'4","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:49:01.044674+0000","last_change":"2024-04-24T08:48:28.978618+0000","last_active":"2024-04-24T08:49:01.044674+0000","last_peered":"2024-04-24T08:49:01.044674+0000","last_clean":"2024-04-24T08:49:01.044674+0000","last_became_active":"2024-04-24T08:48:28.978309+0000","last_became_peered":"2024-04-24T08:48:28.978309+0000","last_unstale":"2024-04-24T08:49:01.044674+0000","last_undegraded":"2024-04-24T08:49:01.044674+0000","last_fullsized":"2024-04-24T08:49:01.044674+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:35:50.091398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.499248+0000","last_change":"2024-04-24T08:48:29.985629+0000","last_active":"2024-04-24T08:48:36.499248+0000","last_peered":"2024-04-24T08:48:36.499248+0000","last_clean":"2024-04-24T08:48:36.499248+0000","last_became_active":"2024-04-24T08:48:29.985258+0000","last_became_peered":"2024-04-24T08:48:29.985258+0000","last_unstale":"2024-04-24T08:48:36.499248+0000","last_undegraded":"2024-04-24T08:48:36.499248+0000","last_fullsized":"2024-04-24T08:48:36.499248+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:57:29.302175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:35.325632+0000","last_change":"2024-04-24T08:48:34.314516+0000","last_active":"2024-04-24T08:48:35.325632+0000","last_peered":"2024-04-24T08:48:35.325632+0000","last_clean":"2024-04-24T08:48:35.325632+0000","last_became_active":"2024-04-24T08:48:34.313775+0000","last_became_peered":"2024-04-24T08:48:34.313775+0000","last_unstale":"2024-04-24T08:48:35.325632+0000","last_undegraded":"2024-04-24T08:48:35.325632+0000","last_fullsized":"2024-04-24T08:48:35.325632+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:06:44.003226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:49:04.102859+0000","last_change":"2024-04-24T08:48:28.977840+0000","last_active":"2024-04-24T08:49:04.102859+0000","last_peered":"2024-04-24T08:49:04.102859+0000","last_clean":"2024-04-24T08:49:04.102859+0000","last_became_active":"2024-04-24T08:48:28.975482+0000","last_became_peered":"2024-04-24T08:48:28.975482+0000","last_unstale":"2024-04-24T08:49:04.102859+0000","last_undegraded":"2024-04-24T08:49:04.102859+0000","last_fullsized":"2024-04-24T08:49:04.102859+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:51:52.468627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:35.292371+0000","last_change":"2024-04-24T08:48:29.976802+0000","last_active":"2024-04-24T08:48:35.292371+0000","last_peered":"2024-04-24T08:48:35.292371+0000","last_clean":"2024-04-24T08:48:35.292371+0000","last_became_active":"2024-04-24T08:48:29.976457+0000","last_became_peered":"2024-04-24T08:48:29.976457+0000","last_unstale":"2024-04-24T08:48:35.292371+0000","last_undegraded":"2024-04-24T08:48:35.292371+0000","last_fullsized":"2024-04-24T08:48:35.292371+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:55:31.465512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:35.290931+0000","last_change":"2024-04-24T08:48:34.314956+0000","last_active":"2024-04-24T08:48:35.290931+0000","last_peered":"2024-04-24T08:48:35.290931+0000","last_clean":"2024-04-24T08:48:35.290931+0000","last_became_active":"2024-04-24T08:48:34.314541+0000","last_became_peered":"2024-04-24T08:48:34.314541+0000","last_unstale":"2024-04-24T08:48:35.290931+0000","last_undegraded":"2024-04-24T08:48:35.290931+0000","last_fullsized":"2024-04-24T08:48:35.290931+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:42:48.614000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"89'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:49:02.049535+0000","last_change":"2024-04-24T08:48:28.974529+0000","last_active":"2024-04-24T08:49:02.049535+0000","last_peered":"2024-04-24T08:49:02.049535+0000","last_clean":"2024-04-24T08:49:02.049535+0000","last_became_active":"2024-04-24T08:48:28.973957+0000","last_became_peered":"2024-04-24T08:48:28.973957+0000","last_unstale":"2024-04-24T08:49:02.049535+0000","last_undegraded":"2024-04-24T08:49:02.049535+0000","last_fullsized":"2024-04-24T08:49:02.049535+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:44:46.216389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:35.290729+0000","last_change":"2024-04-24T08:48:29.981995+0000","last_active":"2024-04-24T08:48:35.290729+0000","last_peered":"2024-04-24T08:48:35.290729+0000","last_clean":"2024-04-24T08:48:35.290729+0000","last_became_active":"2024-04-24T08:48:29.980756+0000","last_became_peered":"2024-04-24T08:48:29.980756+0000","last_unstale":"2024-04-24T08:48:35.290729+0000","last_undegraded":"2024-04-24T08:48:35.290729+0000","last_fullsized":"2024-04-24T08:48:35.290729+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:42:52.770069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.501786+0000","last_change":"2024-04-24T08:48:34.308836+0000","last_active":"2024-04-24T08:48:36.501786+0000","last_peered":"2024-04-24T08:48:36.501786+0000","last_clean":"2024-04-24T08:48:36.501786+0000","last_became_active":"2024-04-24T08:48:34.308470+0000","last_became_peered":"2024-04-24T08:48:34.308470+0000","last_unstale":"2024-04-24T08:48:36.501786+0000","last_undegraded":"2024-04-24T08:48:36.501786+0000","last_fullsized":"2024-04-24T08:48:36.501786+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:38:14.906117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:49:04.091588+0000","last_change":"2024-04-24T08:48:28.971818+0000","last_active":"2024-04-24T08:49:04.091588+0000","last_peered":"2024-04-24T08:49:04.091588+0000","last_clean":"2024-04-24T08:49:04.091588+0000","last_became_active":"2024-04-24T08:48:28.971463+0000","last_became_peered":"2024-04-24T08:48:28.971463+0000","last_unstale":"2024-04-24T08:49:04.091588+0000","last_undegraded":"2024-04-24T08:49:04.091588+0000","last_fullsized":"2024-04-24T08:49:04.091588+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:10:22.829722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.063542+0000","last_change":"2024-04-24T08:48:29.980632+0000","last_active":"2024-04-24T08:48:36.063542+0000","last_peered":"2024-04-24T08:48:36.063542+0000","last_clean":"2024-04-24T08:48:36.063542+0000","last_became_active":"2024-04-24T08:48:29.980174+0000","last_became_peered":"2024-04-24T08:48:29.980174+0000","last_unstale":"2024-04-24T08:48:36.063542+0000","last_undegraded":"2024-04-24T08:48:36.063542+0000","last_fullsized":"2024-04-24T08:48:36.063542+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:45:32.709091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.063394+0000","last_change":"2024-04-24T08:48:34.316302+0000","last_active":"2024-04-24T08:48:36.063394+0000","last_peered":"2024-04-24T08:48:36.063394+0000","last_clean":"2024-04-24T08:48:36.063394+0000","last_became_active":"2024-04-24T08:48:34.315783+0000","last_became_peered":"2024-04-24T08:48:34.315783+0000","last_unstale":"2024-04-24T08:48:36.063394+0000","last_undegraded":"2024-04-24T08:48:36.063394+0000","last_fullsized":"2024-04-24T08:48:36.063394+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:29:01.733931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:49:04.042315+0000","last_change":"2024-04-24T08:48:28.972033+0000","last_active":"2024-04-24T08:49:04.042315+0000","last_peered":"2024-04-24T08:49:04.042315+0000","last_clean":"2024-04-24T08:49:04.042315+0000","last_became_active":"2024-04-24T08:48:28.971597+0000","last_became_peered":"2024-04-24T08:48:28.971597+0000","last_unstale":"2024-04-24T08:49:04.042315+0000","last_undegraded":"2024-04-24T08:49:04.042315+0000","last_fullsized":"2024-04-24T08:49:04.042315+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:11:27.827599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:35.325942+0000","last_change":"2024-04-24T08:48:29.984679+0000","last_active":"2024-04-24T08:48:35.325942+0000","last_peered":"2024-04-24T08:48:35.325942+0000","last_clean":"2024-04-24T08:48:35.325942+0000","last_became_active":"2024-04-24T08:48:29.984323+0000","last_became_peered":"2024-04-24T08:48:29.984323+0000","last_unstale":"2024-04-24T08:48:35.325942+0000","last_undegraded":"2024-04-24T08:48:35.325942+0000","last_fullsized":"2024-04-24T08:48:35.325942+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:57:46.885873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.498574+0000","last_change":"2024-04-24T08:48:34.319324+0000","last_active":"2024-04-24T08:48:36.498574+0000","last_peered":"2024-04-24T08:48:36.498574+0000","last_clean":"2024-04-24T08:48:36.498574+0000","last_became_active":"2024-04-24T08:48:34.318828+0000","last_became_peered":"2024-04-24T08:48:34.318828+0000","last_unstale":"2024-04-24T08:48:36.498574+0000","last_undegraded":"2024-04-24T08:48:36.498574+0000","last_fullsized":"2024-04-24T08:48:36.498574+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:31:52.097032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:35.289683+0000","last_change":"2024-04-24T08:48:28.970726+0000","last_active":"2024-04-24T08:48:35.289683+0000","last_peered":"2024-04-24T08:48:35.289683+0000","last_clean":"2024-04-24T08:48:35.289683+0000","last_became_active":"2024-04-24T08:48:28.969911+0000","last_became_peered":"2024-04-24T08:48:28.969911+0000","last_unstale":"2024-04-24T08:48:35.289683+0000","last_undegraded":"2024-04-24T08:48:35.289683+0000","last_fullsized":"2024-04-24T08:48:35.289683+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:55:26.357256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.502488+0000","last_change":"2024-04-24T08:48:29.967804+0000","last_active":"2024-04-24T08:48:36.502488+0000","last_peered":"2024-04-24T08:48:36.502488+0000","last_clean":"2024-04-24T08:48:36.502488+0000","last_became_active":"2024-04-24T08:48:29.967431+0000","last_became_peered":"2024-04-24T08:48:29.967431+0000","last_unstale":"2024-04-24T08:48:36.502488+0000","last_undegraded":"2024-04-24T08:48:36.502488+0000","last_fullsized":"2024-04-24T08:48:36.502488+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:00:02.779543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.499507+0000","last_change":"2024-04-24T08:48:34.316844+0000","last_active":"2024-04-24T08:48:36.499507+0000","last_peered":"2024-04-24T08:48:36.499507+0000","last_clean":"2024-04-24T08:48:36.499507+0000","last_became_active":"2024-04-24T08:48:34.316426+0000","last_became_peered":"2024-04-24T08:48:34.316426+0000","last_unstale":"2024-04-24T08:48:36.499507+0000","last_undegraded":"2024-04-24T08:48:36.499507+0000","last_fullsized":"2024-04-24T08:48:36.499507+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:10:13.378443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'5","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:49:04.232354+0000","last_change":"2024-04-24T08:48:28.971544+0000","last_active":"2024-04-24T08:49:04.232354+0000","last_peered":"2024-04-24T08:49:04.232354+0000","last_clean":"2024-04-24T08:49:04.232354+0000","last_became_active":"2024-04-24T08:48:28.970417+0000","last_became_peered":"2024-04-24T08:48:28.970417+0000","last_unstale":"2024-04-24T08:49:04.232354+0000","last_undegraded":"2024-04-24T08:49:04.232354+0000","last_fullsized":"2024-04-24T08:49:04.232354+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:35:04.506126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.501666+0000","last_change":"2024-04-24T08:48:29.987542+0000","last_active":"2024-04-24T08:48:36.501666+0000","last_peered":"2024-04-24T08:48:36.501666+0000","last_clean":"2024-04-24T08:48:36.501666+0000","last_became_active":"2024-04-24T08:48:29.987258+0000","last_became_peered":"2024-04-24T08:48:29.987258+0000","last_unstale":"2024-04-24T08:48:36.501666+0000","last_undegraded":"2024-04-24T08:48:36.501666+0000","last_fullsized":"2024-04-24T08:48:36.501666+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:37:23.341132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.501684+0000","last_change":"2024-04-24T08:48:34.322636+0000","last_active":"2024-04-24T08:48:36.501684+0000","last_peered":"2024-04-24T08:48:36.501684+0000","last_clean":"2024-04-24T08:48:36.501684+0000","last_became_active":"2024-04-24T08:48:34.321764+0000","last_became_peered":"2024-04-24T08:48:34.321764+0000","last_unstale":"2024-04-24T08:48:36.501684+0000","last_undegraded":"2024-04-24T08:48:36.501684+0000","last_fullsized":"2024-04-24T08:48:36.501684+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:56:10.563907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.064050+0000","last_change":"2024-04-24T08:48:29.981734+0000","last_active":"2024-04-24T08:48:36.064050+0000","last_peered":"2024-04-24T08:48:36.064050+0000","last_clean":"2024-04-24T08:48:36.064050+0000","last_became_active":"2024-04-24T08:48:29.980491+0000","last_became_peered":"2024-04-24T08:48:29.980491+0000","last_unstale":"2024-04-24T08:48:36.064050+0000","last_undegraded":"2024-04-24T08:48:36.064050+0000","last_fullsized":"2024-04-24T08:48:36.064050+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:08:45.189026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:49:02.053122+0000","last_change":"2024-04-24T08:48:28.954621+0000","last_active":"2024-04-24T08:49:02.053122+0000","last_peered":"2024-04-24T08:49:02.053122+0000","last_clean":"2024-04-24T08:49:02.053122+0000","last_became_active":"2024-04-24T08:48:28.954139+0000","last_became_peered":"2024-04-24T08:48:28.954139+0000","last_unstale":"2024-04-24T08:49:02.053122+0000","last_undegraded":"2024-04-24T08:49:02.053122+0000","last_fullsized":"2024-04-24T08:49:02.053122+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:59:19.360414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:35.291923+0000","last_change":"2024-04-24T08:48:34.323774+0000","last_active":"2024-04-24T08:48:35.291923+0000","last_peered":"2024-04-24T08:48:35.291923+0000","last_clean":"2024-04-24T08:48:35.291923+0000","last_became_active":"2024-04-24T08:48:34.322214+0000","last_became_peered":"2024-04-24T08:48:34.322214+0000","last_unstale":"2024-04-24T08:48:35.291923+0000","last_undegraded":"2024-04-24T08:48:35.291923+0000","last_fullsized":"2024-04-24T08:48:35.291923+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:51:03.971107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.499633+0000","last_change":"2024-04-24T08:48:29.995372+0000","last_active":"2024-04-24T08:48:36.499633+0000","last_peered":"2024-04-24T08:48:36.499633+0000","last_clean":"2024-04-24T08:48:36.499633+0000","last_became_active":"2024-04-24T08:48:29.994744+0000","last_became_peered":"2024-04-24T08:48:29.994744+0000","last_unstale":"2024-04-24T08:48:36.499633+0000","last_undegraded":"2024-04-24T08:48:36.499633+0000","last_fullsized":"2024-04-24T08:48:36.499633+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:28:55.356390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:49:01.042628+0000","last_change":"2024-04-24T08:48:28.974921+0000","last_active":"2024-04-24T08:49:01.042628+0000","last_peered":"2024-04-24T08:49:01.042628+0000","last_clean":"2024-04-24T08:49:01.042628+0000","last_became_active":"2024-04-24T08:48:28.974113+0000","last_became_peered":"2024-04-24T08:48:28.974113+0000","last_unstale":"2024-04-24T08:49:01.042628+0000","last_undegraded":"2024-04-24T08:49:01.042628+0000","last_fullsized":"2024-04-24T08:49:01.042628+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:35:12.635253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":356,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:35.290533+0000","last_change":"2024-04-24T08:46:45.190615+0000","last_active":"2024-04-24T08:48:35.290533+0000","last_peered":"2024-04-24T08:48:35.290533+0000","last_clean":"2024-04-24T08:48:35.290533+0000","last_became_active":"2024-04-24T08:46:45.190011+0000","last_became_peered":"2024-04-24T08:46:45.190011+0000","last_unstale":"2024-04-24T08:48:35.290533+0000","last_undegraded":"2024-04-24T08:48:35.290533+0000","last_fullsized":"2024-04-24T08:48:35.290533+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:41:57.806569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:41:57.806569+0000","last_clean_scrub_stamp":"2024-04-24T08:41:57.806569+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:13:08.283236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.063697+0000","last_change":"2024-04-24T08:48:34.324588+0000","last_active":"2024-04-24T08:48:36.063697+0000","last_peered":"2024-04-24T08:48:36.063697+0000","last_clean":"2024-04-24T08:48:36.063697+0000","last_became_active":"2024-04-24T08:48:34.321969+0000","last_became_peered":"2024-04-24T08:48:34.321969+0000","last_unstale":"2024-04-24T08:48:36.063697+0000","last_undegraded":"2024-04-24T08:48:36.063697+0000","last_fullsized":"2024-04-24T08:48:36.063697+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:19:56.921198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.498162+0000","last_change":"2024-04-24T08:48:29.981573+0000","last_active":"2024-04-24T08:48:36.498162+0000","last_peered":"2024-04-24T08:48:36.498162+0000","last_clean":"2024-04-24T08:48:36.498162+0000","last_became_active":"2024-04-24T08:48:29.979216+0000","last_became_peered":"2024-04-24T08:48:29.979216+0000","last_unstale":"2024-04-24T08:48:36.498162+0000","last_undegraded":"2024-04-24T08:48:36.498162+0000","last_fullsized":"2024-04-24T08:48:36.498162+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:38:09.164883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:49:02.055998+0000","last_change":"2024-04-24T08:48:28.981492+0000","last_active":"2024-04-24T08:49:02.055998+0000","last_peered":"2024-04-24T08:49:02.055998+0000","last_clean":"2024-04-24T08:49:02.055998+0000","last_became_active":"2024-04-24T08:48:28.981050+0000","last_became_peered":"2024-04-24T08:48:28.981050+0000","last_unstale":"2024-04-24T08:49:02.055998+0000","last_undegraded":"2024-04-24T08:49:02.055998+0000","last_fullsized":"2024-04-24T08:49:02.055998+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:22:09.829566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:35.325781+0000","last_change":"2024-04-24T08:48:34.317502+0000","last_active":"2024-04-24T08:48:35.325781+0000","last_peered":"2024-04-24T08:48:35.325781+0000","last_clean":"2024-04-24T08:48:35.325781+0000","last_became_active":"2024-04-24T08:48:34.316614+0000","last_became_peered":"2024-04-24T08:48:34.316614+0000","last_unstale":"2024-04-24T08:48:35.325781+0000","last_undegraded":"2024-04-24T08:48:35.325781+0000","last_fullsized":"2024-04-24T08:48:35.325781+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:44:12.469926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:49:04.092938+0000","last_change":"2024-04-24T08:48:28.970261+0000","last_active":"2024-04-24T08:49:04.092938+0000","last_peered":"2024-04-24T08:49:04.092938+0000","last_clean":"2024-04-24T08:49:04.092938+0000","last_became_active":"2024-04-24T08:48:28.969813+0000","last_became_peered":"2024-04-24T08:48:28.969813+0000","last_unstale":"2024-04-24T08:49:04.092938+0000","last_undegraded":"2024-04-24T08:49:04.092938+0000","last_fullsized":"2024-04-24T08:49:04.092938+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:46:12.311756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.501189+0000","last_change":"2024-04-24T08:48:29.976712+0000","last_active":"2024-04-24T08:48:36.501189+0000","last_peered":"2024-04-24T08:48:36.501189+0000","last_clean":"2024-04-24T08:48:36.501189+0000","last_became_active":"2024-04-24T08:48:29.976166+0000","last_became_peered":"2024-04-24T08:48:29.976166+0000","last_unstale":"2024-04-24T08:48:36.501189+0000","last_undegraded":"2024-04-24T08:48:36.501189+0000","last_fullsized":"2024-04-24T08:48:36.501189+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:52:53.757829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.498338+0000","last_change":"2024-04-24T08:48:34.310173+0000","last_active":"2024-04-24T08:48:36.498338+0000","last_peered":"2024-04-24T08:48:36.498338+0000","last_clean":"2024-04-24T08:48:36.498338+0000","last_became_active":"2024-04-24T08:48:34.309772+0000","last_became_peered":"2024-04-24T08:48:34.309772+0000","last_unstale":"2024-04-24T08:48:36.498338+0000","last_undegraded":"2024-04-24T08:48:36.498338+0000","last_fullsized":"2024-04-24T08:48:36.498338+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:35:50.302769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'3","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:49:02.053206+0000","last_change":"2024-04-24T08:48:28.970127+0000","last_active":"2024-04-24T08:49:02.053206+0000","last_peered":"2024-04-24T08:49:02.053206+0000","last_clean":"2024-04-24T08:49:02.053206+0000","last_became_active":"2024-04-24T08:48:28.968803+0000","last_became_peered":"2024-04-24T08:48:28.968803+0000","last_unstale":"2024-04-24T08:49:02.053206+0000","last_undegraded":"2024-04-24T08:49:02.053206+0000","last_fullsized":"2024-04-24T08:49:02.053206+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:28:47.782276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.500645+0000","last_change":"2024-04-24T08:48:29.992964+0000","last_active":"2024-04-24T08:48:36.500645+0000","last_peered":"2024-04-24T08:48:36.500645+0000","last_clean":"2024-04-24T08:48:36.500645+0000","last_became_active":"2024-04-24T08:48:29.987501+0000","last_became_peered":"2024-04-24T08:48:29.987501+0000","last_unstale":"2024-04-24T08:48:36.500645+0000","last_undegraded":"2024-04-24T08:48:36.500645+0000","last_fullsized":"2024-04-24T08:48:36.500645+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:49:19.157052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:35.290875+0000","last_change":"2024-04-24T08:48:34.315393+0000","last_active":"2024-04-24T08:48:35.290875+0000","last_peered":"2024-04-24T08:48:35.290875+0000","last_clean":"2024-04-24T08:48:35.290875+0000","last_became_active":"2024-04-24T08:48:34.314943+0000","last_became_peered":"2024-04-24T08:48:34.314943+0000","last_unstale":"2024-04-24T08:48:35.290875+0000","last_undegraded":"2024-04-24T08:48:35.290875+0000","last_fullsized":"2024-04-24T08:48:35.290875+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:03:13.312223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:49:01.039089+0000","last_change":"2024-04-24T08:48:28.969813+0000","last_active":"2024-04-24T08:49:01.039089+0000","last_peered":"2024-04-24T08:49:01.039089+0000","last_clean":"2024-04-24T08:49:01.039089+0000","last_became_active":"2024-04-24T08:48:28.968485+0000","last_became_peered":"2024-04-24T08:48:28.968485+0000","last_unstale":"2024-04-24T08:49:01.039089+0000","last_undegraded":"2024-04-24T08:49:01.039089+0000","last_fullsized":"2024-04-24T08:49:01.039089+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:32:10.095723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.498659+0000","last_change":"2024-04-24T08:48:29.997731+0000","last_active":"2024-04-24T08:48:36.498659+0000","last_peered":"2024-04-24T08:48:36.498659+0000","last_clean":"2024-04-24T08:48:36.498659+0000","last_became_active":"2024-04-24T08:48:29.997373+0000","last_became_peered":"2024-04-24T08:48:29.997373+0000","last_unstale":"2024-04-24T08:48:36.498659+0000","last_undegraded":"2024-04-24T08:48:36.498659+0000","last_fullsized":"2024-04-24T08:48:36.498659+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:57:19.531173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.500827+0000","last_change":"2024-04-24T08:48:34.326170+0000","last_active":"2024-04-24T08:48:36.500827+0000","last_peered":"2024-04-24T08:48:36.500827+0000","last_clean":"2024-04-24T08:48:36.500827+0000","last_became_active":"2024-04-24T08:48:34.325678+0000","last_became_peered":"2024-04-24T08:48:34.325678+0000","last_unstale":"2024-04-24T08:48:36.500827+0000","last_undegraded":"2024-04-24T08:48:36.500827+0000","last_fullsized":"2024-04-24T08:48:36.500827+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:12:47.003864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:49:01.047700+0000","last_change":"2024-04-24T08:48:28.975522+0000","last_active":"2024-04-24T08:49:01.047700+0000","last_peered":"2024-04-24T08:49:01.047700+0000","last_clean":"2024-04-24T08:49:01.047700+0000","last_became_active":"2024-04-24T08:48:28.975093+0000","last_became_peered":"2024-04-24T08:48:28.975093+0000","last_unstale":"2024-04-24T08:49:01.047700+0000","last_undegraded":"2024-04-24T08:49:01.047700+0000","last_fullsized":"2024-04-24T08:49:01.047700+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:37:31.809694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.497877+0000","last_change":"2024-04-24T08:48:29.982502+0000","last_active":"2024-04-24T08:48:36.497877+0000","last_peered":"2024-04-24T08:48:36.497877+0000","last_clean":"2024-04-24T08:48:36.497877+0000","last_became_active":"2024-04-24T08:48:29.982141+0000","last_became_peered":"2024-04-24T08:48:29.982141+0000","last_unstale":"2024-04-24T08:48:36.497877+0000","last_undegraded":"2024-04-24T08:48:36.497877+0000","last_fullsized":"2024-04-24T08:48:36.497877+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:46:15.851359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:35.291126+0000","last_change":"2024-04-24T08:48:34.322717+0000","last_active":"2024-04-24T08:48:35.291126+0000","last_peered":"2024-04-24T08:48:35.291126+0000","last_clean":"2024-04-24T08:48:35.291126+0000","last_became_active":"2024-04-24T08:48:34.321837+0000","last_became_peered":"2024-04-24T08:48:34.321837+0000","last_unstale":"2024-04-24T08:48:35.291126+0000","last_undegraded":"2024-04-24T08:48:35.291126+0000","last_fullsized":"2024-04-24T08:48:35.291126+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:40:11.557436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'5","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:49:03.070283+0000","last_change":"2024-04-24T08:48:28.973951+0000","last_active":"2024-04-24T08:49:03.070283+0000","last_peered":"2024-04-24T08:49:03.070283+0000","last_clean":"2024-04-24T08:49:03.070283+0000","last_became_active":"2024-04-24T08:48:28.973131+0000","last_became_peered":"2024-04-24T08:48:28.973131+0000","last_unstale":"2024-04-24T08:49:03.070283+0000","last_undegraded":"2024-04-24T08:49:03.070283+0000","last_fullsized":"2024-04-24T08:49:03.070283+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:45:45.197337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:35.325745+0000","last_change":"2024-04-24T08:48:29.992689+0000","last_active":"2024-04-24T08:48:35.325745+0000","last_peered":"2024-04-24T08:48:35.325745+0000","last_clean":"2024-04-24T08:48:35.325745+0000","last_became_active":"2024-04-24T08:48:29.988874+0000","last_became_peered":"2024-04-24T08:48:29.988874+0000","last_unstale":"2024-04-24T08:48:35.325745+0000","last_undegraded":"2024-04-24T08:48:35.325745+0000","last_fullsized":"2024-04-24T08:48:35.325745+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:42:41.578424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.500481+0000","last_change":"2024-04-24T08:48:34.318640+0000","last_active":"2024-04-24T08:48:36.500481+0000","last_peered":"2024-04-24T08:48:36.500481+0000","last_clean":"2024-04-24T08:48:36.500481+0000","last_became_active":"2024-04-24T08:48:34.318039+0000","last_became_peered":"2024-04-24T08:48:34.318039+0000","last_unstale":"2024-04-24T08:48:36.500481+0000","last_undegraded":"2024-04-24T08:48:36.500481+0000","last_fullsized":"2024-04-24T08:48:36.500481+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:17:27.110930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:35.291081+0000","last_change":"2024-04-24T08:48:28.980828+0000","last_active":"2024-04-24T08:48:35.291081+0000","last_peered":"2024-04-24T08:48:35.291081+0000","last_clean":"2024-04-24T08:48:35.291081+0000","last_became_active":"2024-04-24T08:48:28.979814+0000","last_became_peered":"2024-04-24T08:48:28.979814+0000","last_unstale":"2024-04-24T08:48:35.291081+0000","last_undegraded":"2024-04-24T08:48:35.291081+0000","last_fullsized":"2024-04-24T08:48:35.291081+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:02:28.145847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.500112+0000","last_change":"2024-04-24T08:48:29.995003+0000","last_active":"2024-04-24T08:48:36.500112+0000","last_peered":"2024-04-24T08:48:36.500112+0000","last_clean":"2024-04-24T08:48:36.500112+0000","last_became_active":"2024-04-24T08:48:29.994516+0000","last_became_peered":"2024-04-24T08:48:29.994516+0000","last_unstale":"2024-04-24T08:48:36.500112+0000","last_undegraded":"2024-04-24T08:48:36.500112+0000","last_fullsized":"2024-04-24T08:48:36.500112+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:15:46.094534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.501593+0000","last_change":"2024-04-24T08:48:34.311721+0000","last_active":"2024-04-24T08:48:36.501593+0000","last_peered":"2024-04-24T08:48:36.501593+0000","last_clean":"2024-04-24T08:48:36.501593+0000","last_became_active":"2024-04-24T08:48:34.311241+0000","last_became_peered":"2024-04-24T08:48:34.311241+0000","last_unstale":"2024-04-24T08:48:36.501593+0000","last_undegraded":"2024-04-24T08:48:36.501593+0000","last_fullsized":"2024-04-24T08:48:36.501593+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:16:17.679237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:49:01.047134+0000","last_change":"2024-04-24T08:48:28.973336+0000","last_active":"2024-04-24T08:49:01.047134+0000","last_peered":"2024-04-24T08:49:01.047134+0000","last_clean":"2024-04-24T08:49:01.047134+0000","last_became_active":"2024-04-24T08:48:28.972666+0000","last_became_peered":"2024-04-24T08:48:28.972666+0000","last_unstale":"2024-04-24T08:49:01.047134+0000","last_undegraded":"2024-04-24T08:49:01.047134+0000","last_fullsized":"2024-04-24T08:49:01.047134+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:53:28.358291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:35.291248+0000","last_change":"2024-04-24T08:48:29.983498+0000","last_active":"2024-04-24T08:48:35.291248+0000","last_peered":"2024-04-24T08:48:35.291248+0000","last_clean":"2024-04-24T08:48:35.291248+0000","last_became_active":"2024-04-24T08:48:29.983167+0000","last_became_peered":"2024-04-24T08:48:29.983167+0000","last_unstale":"2024-04-24T08:48:35.291248+0000","last_undegraded":"2024-04-24T08:48:35.291248+0000","last_fullsized":"2024-04-24T08:48:35.291248+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:09:56.648047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.501528+0000","last_change":"2024-04-24T08:48:34.317151+0000","last_active":"2024-04-24T08:48:36.501528+0000","last_peered":"2024-04-24T08:48:36.501528+0000","last_clean":"2024-04-24T08:48:36.501528+0000","last_became_active":"2024-04-24T08:48:34.316362+0000","last_became_peered":"2024-04-24T08:48:34.316362+0000","last_unstale":"2024-04-24T08:48:36.501528+0000","last_undegraded":"2024-04-24T08:48:36.501528+0000","last_fullsized":"2024-04-24T08:48:36.501528+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:35:38.196763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:35.291937+0000","last_change":"2024-04-24T08:48:28.974528+0000","last_active":"2024-04-24T08:48:35.291937+0000","last_peered":"2024-04-24T08:48:35.291937+0000","last_clean":"2024-04-24T08:48:35.291937+0000","last_became_active":"2024-04-24T08:48:28.974203+0000","last_became_peered":"2024-04-24T08:48:28.974203+0000","last_unstale":"2024-04-24T08:48:35.291937+0000","last_undegraded":"2024-04-24T08:48:35.291937+0000","last_fullsized":"2024-04-24T08:48:35.291937+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:26:15.701204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:35.292350+0000","last_change":"2024-04-24T08:48:29.984356+0000","last_active":"2024-04-24T08:48:35.292350+0000","last_peered":"2024-04-24T08:48:35.292350+0000","last_clean":"2024-04-24T08:48:35.292350+0000","last_became_active":"2024-04-24T08:48:29.983949+0000","last_became_peered":"2024-04-24T08:48:29.983949+0000","last_unstale":"2024-04-24T08:48:35.292350+0000","last_undegraded":"2024-04-24T08:48:35.292350+0000","last_fullsized":"2024-04-24T08:48:35.292350+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:51:59.409835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.498254+0000","last_change":"2024-04-24T08:48:34.312418+0000","last_active":"2024-04-24T08:48:36.498254+0000","last_peered":"2024-04-24T08:48:36.498254+0000","last_clean":"2024-04-24T08:48:36.498254+0000","last_became_active":"2024-04-24T08:48:34.312035+0000","last_became_peered":"2024-04-24T08:48:34.312035+0000","last_unstale":"2024-04-24T08:48:36.498254+0000","last_undegraded":"2024-04-24T08:48:36.498254+0000","last_fullsized":"2024-04-24T08:48:36.498254+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:27:33.461732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"89'3","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:49:04.099932+0000","last_change":"2024-04-24T08:48:28.976070+0000","last_active":"2024-04-24T08:49:04.099932+0000","last_peered":"2024-04-24T08:49:04.099932+0000","last_clean":"2024-04-24T08:49:04.099932+0000","last_became_active":"2024-04-24T08:48:28.975703+0000","last_became_peered":"2024-04-24T08:48:28.975703+0000","last_unstale":"2024-04-24T08:49:04.099932+0000","last_undegraded":"2024-04-24T08:49:04.099932+0000","last_fullsized":"2024-04-24T08:49:04.099932+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:46:13.227529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:35.291223+0000","last_change":"2024-04-24T08:48:29.986423+0000","last_active":"2024-04-24T08:48:35.291223+0000","last_peered":"2024-04-24T08:48:35.291223+0000","last_clean":"2024-04-24T08:48:35.291223+0000","last_became_active":"2024-04-24T08:48:29.985452+0000","last_became_peered":"2024-04-24T08:48:29.985452+0000","last_unstale":"2024-04-24T08:48:35.291223+0000","last_undegraded":"2024-04-24T08:48:35.291223+0000","last_fullsized":"2024-04-24T08:48:35.291223+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:05:42.686370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.062764+0000","last_change":"2024-04-24T08:48:34.320191+0000","last_active":"2024-04-24T08:48:36.062764+0000","last_peered":"2024-04-24T08:48:36.062764+0000","last_clean":"2024-04-24T08:48:36.062764+0000","last_became_active":"2024-04-24T08:48:34.315415+0000","last_became_peered":"2024-04-24T08:48:34.315415+0000","last_unstale":"2024-04-24T08:48:36.062764+0000","last_undegraded":"2024-04-24T08:48:36.062764+0000","last_fullsized":"2024-04-24T08:48:36.062764+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:55:11.372789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"89'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.501691+0000","last_change":"2024-04-24T08:48:28.968550+0000","last_active":"2024-04-24T08:48:36.501691+0000","last_peered":"2024-04-24T08:48:36.501691+0000","last_clean":"2024-04-24T08:48:36.501691+0000","last_became_active":"2024-04-24T08:48:28.968120+0000","last_became_peered":"2024-04-24T08:48:28.968120+0000","last_unstale":"2024-04-24T08:48:36.501691+0000","last_undegraded":"2024-04-24T08:48:36.501691+0000","last_fullsized":"2024-04-24T08:48:36.501691+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:46:00.870285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:35.292109+0000","last_change":"2024-04-24T08:48:29.984755+0000","last_active":"2024-04-24T08:48:35.292109+0000","last_peered":"2024-04-24T08:48:35.292109+0000","last_clean":"2024-04-24T08:48:35.292109+0000","last_became_active":"2024-04-24T08:48:29.982093+0000","last_became_peered":"2024-04-24T08:48:29.982093+0000","last_unstale":"2024-04-24T08:48:35.292109+0000","last_undegraded":"2024-04-24T08:48:35.292109+0000","last_fullsized":"2024-04-24T08:48:35.292109+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:25:22.620152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:35.325401+0000","last_change":"2024-04-24T08:48:29.983101+0000","last_active":"2024-04-24T08:48:35.325401+0000","last_peered":"2024-04-24T08:48:35.325401+0000","last_clean":"2024-04-24T08:48:35.325401+0000","last_became_active":"2024-04-24T08:48:29.981871+0000","last_became_peered":"2024-04-24T08:48:29.981871+0000","last_unstale":"2024-04-24T08:48:35.325401+0000","last_undegraded":"2024-04-24T08:48:35.325401+0000","last_fullsized":"2024-04-24T08:48:35.325401+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:02:44.582111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:35.292306+0000","last_change":"2024-04-24T08:48:29.978469+0000","last_active":"2024-04-24T08:48:35.292306+0000","last_peered":"2024-04-24T08:48:35.292306+0000","last_clean":"2024-04-24T08:48:35.292306+0000","last_became_active":"2024-04-24T08:48:29.977382+0000","last_became_peered":"2024-04-24T08:48:29.977382+0000","last_unstale":"2024-04-24T08:48:35.292306+0000","last_undegraded":"2024-04-24T08:48:35.292306+0000","last_fullsized":"2024-04-24T08:48:35.292306+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:00:33.863236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:35.290715+0000","last_change":"2024-04-24T08:48:29.988241+0000","last_active":"2024-04-24T08:48:35.290715+0000","last_peered":"2024-04-24T08:48:35.290715+0000","last_clean":"2024-04-24T08:48:35.290715+0000","last_became_active":"2024-04-24T08:48:29.987930+0000","last_became_peered":"2024-04-24T08:48:29.987930+0000","last_unstale":"2024-04-24T08:48:35.290715+0000","last_undegraded":"2024-04-24T08:48:35.290715+0000","last_fullsized":"2024-04-24T08:48:35.290715+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:42:24.687624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.497713+0000","last_change":"2024-04-24T08:48:29.992441+0000","last_active":"2024-04-24T08:48:36.497713+0000","last_peered":"2024-04-24T08:48:36.497713+0000","last_clean":"2024-04-24T08:48:36.497713+0000","last_became_active":"2024-04-24T08:48:29.988788+0000","last_became_peered":"2024-04-24T08:48:29.988788+0000","last_unstale":"2024-04-24T08:48:36.497713+0000","last_undegraded":"2024-04-24T08:48:36.497713+0000","last_fullsized":"2024-04-24T08:48:36.497713+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:08:31.549509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.501904+0000","last_change":"2024-04-24T08:48:29.982116+0000","last_active":"2024-04-24T08:48:36.501904+0000","last_peered":"2024-04-24T08:48:36.501904+0000","last_clean":"2024-04-24T08:48:36.501904+0000","last_became_active":"2024-04-24T08:48:29.981636+0000","last_became_peered":"2024-04-24T08:48:29.981636+0000","last_unstale":"2024-04-24T08:48:36.501904+0000","last_undegraded":"2024-04-24T08:48:36.501904+0000","last_fullsized":"2024-04-24T08:48:36.501904+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:11:01.943425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:35.326069+0000","last_change":"2024-04-24T08:48:29.982244+0000","last_active":"2024-04-24T08:48:35.326069+0000","last_peered":"2024-04-24T08:48:35.326069+0000","last_clean":"2024-04-24T08:48:35.326069+0000","last_became_active":"2024-04-24T08:48:29.981622+0000","last_became_peered":"2024-04-24T08:48:29.981622+0000","last_unstale":"2024-04-24T08:48:35.326069+0000","last_undegraded":"2024-04-24T08:48:35.326069+0000","last_fullsized":"2024-04-24T08:48:35.326069+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:02:57.723949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:35.292425+0000","last_change":"2024-04-24T08:48:29.985867+0000","last_active":"2024-04-24T08:48:35.292425+0000","last_peered":"2024-04-24T08:48:35.292425+0000","last_clean":"2024-04-24T08:48:35.292425+0000","last_became_active":"2024-04-24T08:48:29.985430+0000","last_became_peered":"2024-04-24T08:48:29.985430+0000","last_unstale":"2024-04-24T08:48:35.292425+0000","last_undegraded":"2024-04-24T08:48:35.292425+0000","last_fullsized":"2024-04-24T08:48:35.292425+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:09:40.467623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.500900+0000","last_change":"2024-04-24T08:48:29.996474+0000","last_active":"2024-04-24T08:48:36.500900+0000","last_peered":"2024-04-24T08:48:36.500900+0000","last_clean":"2024-04-24T08:48:36.500900+0000","last_became_active":"2024-04-24T08:48:29.995958+0000","last_became_peered":"2024-04-24T08:48:29.995958+0000","last_unstale":"2024-04-24T08:48:36.500900+0000","last_undegraded":"2024-04-24T08:48:36.500900+0000","last_fullsized":"2024-04-24T08:48:36.500900+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:50:19.865125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.063623+0000","last_change":"2024-04-24T08:48:29.978285+0000","last_active":"2024-04-24T08:48:36.063623+0000","last_peered":"2024-04-24T08:48:36.063623+0000","last_clean":"2024-04-24T08:48:36.063623+0000","last_became_active":"2024-04-24T08:48:29.976445+0000","last_became_peered":"2024-04-24T08:48:29.976445+0000","last_unstale":"2024-04-24T08:48:36.063623+0000","last_undegraded":"2024-04-24T08:48:36.063623+0000","last_fullsized":"2024-04-24T08:48:36.063623+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:30:29.598733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.499261+0000","last_change":"2024-04-24T08:48:29.994442+0000","last_active":"2024-04-24T08:48:36.499261+0000","last_peered":"2024-04-24T08:48:36.499261+0000","last_clean":"2024-04-24T08:48:36.499261+0000","last_became_active":"2024-04-24T08:48:29.993896+0000","last_became_peered":"2024-04-24T08:48:29.993896+0000","last_unstale":"2024-04-24T08:48:36.499261+0000","last_undegraded":"2024-04-24T08:48:36.499261+0000","last_fullsized":"2024-04-24T08:48:36.499261+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:37:14.023511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.502312+0000","last_change":"2024-04-24T08:48:29.997254+0000","last_active":"2024-04-24T08:48:36.502312+0000","last_peered":"2024-04-24T08:48:36.502312+0000","last_clean":"2024-04-24T08:48:36.502312+0000","last_became_active":"2024-04-24T08:48:29.996871+0000","last_became_peered":"2024-04-24T08:48:29.996871+0000","last_unstale":"2024-04-24T08:48:36.502312+0000","last_undegraded":"2024-04-24T08:48:36.502312+0000","last_fullsized":"2024-04-24T08:48:36.502312+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:53:37.288858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.499571+0000","last_change":"2024-04-24T08:48:29.986021+0000","last_active":"2024-04-24T08:48:36.499571+0000","last_peered":"2024-04-24T08:48:36.499571+0000","last_clean":"2024-04-24T08:48:36.499571+0000","last_became_active":"2024-04-24T08:48:29.985182+0000","last_became_peered":"2024-04-24T08:48:29.985182+0000","last_unstale":"2024-04-24T08:48:36.499571+0000","last_undegraded":"2024-04-24T08:48:36.499571+0000","last_fullsized":"2024-04-24T08:48:36.499571+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:10:57.143065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:35.325368+0000","last_change":"2024-04-24T08:48:29.979886+0000","last_active":"2024-04-24T08:48:35.325368+0000","last_peered":"2024-04-24T08:48:35.325368+0000","last_clean":"2024-04-24T08:48:35.325368+0000","last_became_active":"2024-04-24T08:48:29.978779+0000","last_became_peered":"2024-04-24T08:48:29.978779+0000","last_unstale":"2024-04-24T08:48:35.325368+0000","last_undegraded":"2024-04-24T08:48:35.325368+0000","last_fullsized":"2024-04-24T08:48:35.325368+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:51:51.303990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":9283,"num_objects":75,"num_object_clones":0,"num_object_copies":225,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":0,"num_read":20,"num_read_kb":16,"num_write":162,"num_write_kb":55,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":50,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":352256,"data_stored":48531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":145,"ondisk_log_size":145,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383703,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":995328,"data_stored":478407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645342,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32760,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712392,"statfs":{"total":95995035648,"available":95961489408,"internally_reserved":0,"allocated":1622016,"data_stored":1073003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808869,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":1003520,"data_stored":478033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070508,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":1003520,"data_stored":477953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332148,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32752,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712400,"statfs":{"total":95995035648,"available":95961497600,"internally_reserved":0,"allocated":1613824,"data_stored":1075212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495675,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":991232,"data_stored":477327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574943,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36320,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708832,"statfs":{"total":95995035648,"available":95957843968,"internally_reserved":0,"allocated":1007616,"data_stored":488239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738470,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1024000,"data_stored":479393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411414,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1007616,"data_stored":481713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182481,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32160,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712992,"statfs":{"total":95995035648,"available":95962103808,"internally_reserved":0,"allocated":1007616,"data_stored":477623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920841,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":995328,"data_stored":477859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757315,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32764,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712388,"statfs":{"total":95995035648,"available":95961485312,"internally_reserved":0,"allocated":1626112,"data_stored":1074890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":24893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":4498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":5544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T08:49:07.040 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T08:49:07.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:07.286+0000 7f38d5200640 1 -- 172.21.15.83:0/2284065289 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38d0077c00 msgr2=0x7f38d0078060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:07.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:07.286+0000 7f38d5200640 1 --2- 172.21.15.83:0/2284065289 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38d0077c00 0x7f38d0078060 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f38b8009900 tx=0x7f38b802a410 comp rx=0 tx=0).stop 2024-04-24T08:49:07.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:07.286+0000 7f38d5200640 1 -- 172.21.15.83:0/2284065289 shutdown_connections 2024-04-24T08:49:07.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:07.286+0000 7f38d5200640 1 --2- 172.21.15.83:0/2284065289 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f38d00785a0 0x7f38d0078e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:07.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:07.286+0000 7f38d5200640 1 --2- 172.21.15.83:0/2284065289 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38d0077c00 0x7f38d0078060 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:07.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:07.286+0000 7f38d5200640 1 --2- 172.21.15.83:0/2284065289 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f38d00769c0 0x7f38d0076de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:07.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:07.286+0000 7f38d5200640 1 -- 172.21.15.83:0/2284065289 >> 172.21.15.83:0/2284065289 conn(0x7f38d00ff7f0 msgr2=0x7f38d0101c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:49:07.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:07.286+0000 7f38d5200640 1 -- 172.21.15.83:0/2284065289 shutdown_connections 2024-04-24T08:49:07.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:07.286+0000 7f38d5200640 1 -- 172.21.15.83:0/2284065289 wait complete. 2024-04-24T08:49:07.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:07.287+0000 7f38d5200640 1 Processor -- start 2024-04-24T08:49:07.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:07.287+0000 7f38d5200640 1 -- start start 2024-04-24T08:49:07.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:07.288+0000 7f38d5200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f38d00769c0 0x7f38d019e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:07.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:07.288+0000 7f38d5200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f38d0077c00 0x7f38d019ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:07.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:07.288+0000 7f38d5200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38d00785a0 0x7f38d01a5e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:07.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:07.288+0000 7f38d5200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f38d01141c0 con 0x7f38d00785a0 2024-04-24T08:49:07.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:07.288+0000 7f38d5200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f38d0114040 con 0x7f38d00769c0 2024-04-24T08:49:07.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:07.288+0000 7f38d5200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f38d0114340 con 0x7f38d0077c00 2024-04-24T08:49:07.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:07.288+0000 7f38ce800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38d00785a0 0x7f38d01a5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:07.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:07.288+0000 7f38ce800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38d00785a0 0x7f38d01a5e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57834/0 (socket says 172.21.15.83:57834) 2024-04-24T08:49:07.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:07.288+0000 7f38ce800640 1 -- 172.21.15.83:0/1354668291 learned_addr learned my addr 172.21.15.83:0/1354668291 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:49:07.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:07.288+0000 7f38cde00640 1 --2- 172.21.15.83:0/1354668291 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f38d00769c0 0x7f38d019e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:07.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:07.288+0000 7f38c7e00640 1 --2- 172.21.15.83:0/1354668291 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f38d0077c00 0x7f38d019ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:07.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:07.288+0000 7f38ce800640 1 -- 172.21.15.83:0/1354668291 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f38d0077c00 msgr2=0x7f38d019ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:07.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:07.288+0000 7f38ce800640 1 --2- 172.21.15.83:0/1354668291 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f38d0077c00 0x7f38d019ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:07.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:07.288+0000 7f38ce800640 1 -- 172.21.15.83:0/1354668291 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f38d00769c0 msgr2=0x7f38d019e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:07.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:07.288+0000 7f38ce800640 1 --2- 172.21.15.83:0/1354668291 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f38d00769c0 0x7f38d019e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:07.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:07.289+0000 7f38ce800640 1 -- 172.21.15.83:0/1354668291 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38d01a64c0 con 0x7f38d00785a0 2024-04-24T08:49:07.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:07.289+0000 7f38ce800640 1 --2- 172.21.15.83:0/1354668291 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38d00785a0 0x7f38d01a5e50 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f38c800eaa0 tx=0x7f38c800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:49:07.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:07.289+0000 7f38c6a00640 1 -- 172.21.15.83:0/1354668291 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c800cdb0 con 0x7f38d00785a0 2024-04-24T08:49:07.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:07.289+0000 7f38d5200640 1 -- 172.21.15.83:0/1354668291 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38d01a67b0 con 0x7f38d00785a0 2024-04-24T08:49:07.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:07.289+0000 7f38c6a00640 1 -- 172.21.15.83:0/1354668291 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f38c8005020 con 0x7f38d00785a0 2024-04-24T08:49:07.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:07.289+0000 7f38c6a00640 1 -- 172.21.15.83:0/1354668291 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c801f060 con 0x7f38d00785a0 2024-04-24T08:49:07.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:07.289+0000 7f38d5200640 1 -- 172.21.15.83:0/1354668291 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38d01a6cb0 con 0x7f38d00785a0 2024-04-24T08:49:07.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:07.291+0000 7f38d5200640 1 -- 172.21.15.83:0/1354668291 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38d0110e20 con 0x7f38d00785a0 2024-04-24T08:49:07.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:07.292+0000 7f38c6a00640 1 -- 172.21.15.83:0/1354668291 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f38c80051d0 con 0x7f38d00785a0 2024-04-24T08:49:07.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:07.294+0000 7f38c6a00640 1 --2- 172.21.15.83:0/1354668291 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f38b40a3c40 0x7f38b40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:07.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:07.294+0000 7f38c6a00640 1 -- 172.21.15.83:0/1354668291 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f38c80d0a70 con 0x7f38d00785a0 2024-04-24T08:49:07.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:07.294+0000 7f38cde00640 1 --2- 172.21.15.83:0/1354668291 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f38b40a3c40 0x7f38b40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:07.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:07.295+0000 7f38cde00640 1 --2- 172.21.15.83:0/1354668291 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f38b40a3c40 0x7f38b40a6100 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f38c0005fd0 tx=0x7f38c0005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:49:07.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:07.298+0000 7f38c6a00640 1 -- 172.21.15.83:0/1354668291 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f38c8016030 con 0x7f38d00785a0 2024-04-24T08:49:07.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:49:07 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/4265399431' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T08:49:07.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:49:07 smithi083 ceph-mon[27907]: pgmap v392: 129 pgs: 129 active+clean; 586 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 3.1 KiB/s wr, 11 op/s 2024-04-24T08:49:07.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:49:07 smithi083 ceph-mon[27907]: from='client.14979 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T08:49:07.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:07.498+0000 7f38d5200640 1 -- 172.21.15.83:0/1354668291 --> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f38d0062440 con 0x7f38b40a3c40 2024-04-24T08:49:07.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:07.512+0000 7f38c6a00640 1 -- 172.21.15.83:0/1354668291 <== mgr.14152 v2:172.21.15.83:6800/2550826560 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340459 (secure 0 0 0) 0x7f38d0062440 con 0x7f38b40a3c40 2024-04-24T08:49:07.516 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:49:07.519 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-04-24T08:49:07.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:07.519+0000 7f38d5200640 1 -- 172.21.15.83:0/1354668291 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f38b40a3c40 msgr2=0x7f38b40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:07.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:07.519+0000 7f38d5200640 1 --2- 172.21.15.83:0/1354668291 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f38b40a3c40 0x7f38b40a6100 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f38c0005fd0 tx=0x7f38c0005850 comp rx=0 tx=0).stop 2024-04-24T08:49:07.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:07.519+0000 7f38d5200640 1 -- 172.21.15.83:0/1354668291 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38d00785a0 msgr2=0x7f38d01a5e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:07.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:07.519+0000 7f38d5200640 1 --2- 172.21.15.83:0/1354668291 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38d00785a0 0x7f38d01a5e50 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f38c800eaa0 tx=0x7f38c800ef60 comp rx=0 tx=0).stop 2024-04-24T08:49:07.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:07.520+0000 7f38d5200640 1 -- 172.21.15.83:0/1354668291 shutdown_connections 2024-04-24T08:49:07.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:07.520+0000 7f38d5200640 1 --2- 172.21.15.83:0/1354668291 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f38b40a3c40 0x7f38b40a6100 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:07.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:07.520+0000 7f38d5200640 1 --2- 172.21.15.83:0/1354668291 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38d00785a0 0x7f38d01a5e50 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:07.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:07.520+0000 7f38d5200640 1 --2- 172.21.15.83:0/1354668291 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f38d0077c00 0x7f38d019ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:07.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:07.520+0000 7f38d5200640 1 --2- 172.21.15.83:0/1354668291 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f38d00769c0 0x7f38d019e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:07.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:07.520+0000 7f38d5200640 1 -- 172.21.15.83:0/1354668291 >> 172.21.15.83:0/1354668291 conn(0x7f38d00ff7f0 msgr2=0x7f38d0100f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:49:07.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:07.520+0000 7f38d5200640 1 -- 172.21.15.83:0/1354668291 shutdown_connections 2024-04-24T08:49:07.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:07.520+0000 7f38d5200640 1 -- 172.21.15.83:0/1354668291 wait complete. 2024-04-24T08:49:07.539 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":392,"stamp":"2024-04-24T08:49:06.847852+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":599768,"num_objects":78,"num_object_clones":0,"num_object_copies":235,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":0,"num_read":156,"num_read_kb":135,"num_write":344,"num_write_kb":2994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":50,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":283,"ondisk_log_size":283,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":379364,"kb_used_data":13572,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124562460,"statfs":{"total":1151940427776,"available":1151551959040,"internally_reserved":0,"allocated":13897728,"data_stored":7539652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":374453561},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4760,"num_objects":54,"num_object_clones":0,"num_object_copies":163,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":20,"num_read_kb":16,"num_write":114,"num_write_kb":37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006378"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:35.326280+0000","last_change":"2024-04-24T08:48:29.993009+0000","last_active":"2024-04-24T08:48:35.326280+0000","last_peered":"2024-04-24T08:48:35.326280+0000","last_clean":"2024-04-24T08:48:35.326280+0000","last_became_active":"2024-04-24T08:48:29.990613+0000","last_became_peered":"2024-04-24T08:48:29.990613+0000","last_unstale":"2024-04-24T08:48:35.326280+0000","last_undegraded":"2024-04-24T08:48:35.326280+0000","last_fullsized":"2024-04-24T08:48:35.326280+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:13:43.871742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.498605+0000","last_change":"2024-04-24T08:48:29.986957+0000","last_active":"2024-04-24T08:48:36.498605+0000","last_peered":"2024-04-24T08:48:36.498605+0000","last_clean":"2024-04-24T08:48:36.498605+0000","last_became_active":"2024-04-24T08:48:29.984691+0000","last_became_peered":"2024-04-24T08:48:29.984691+0000","last_unstale":"2024-04-24T08:48:36.498605+0000","last_undegraded":"2024-04-24T08:48:36.498605+0000","last_fullsized":"2024-04-24T08:48:36.498605+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:07:38.031213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.498428+0000","last_change":"2024-04-24T08:48:29.991478+0000","last_active":"2024-04-24T08:48:36.498428+0000","last_peered":"2024-04-24T08:48:36.498428+0000","last_clean":"2024-04-24T08:48:36.498428+0000","last_became_active":"2024-04-24T08:48:29.991013+0000","last_became_peered":"2024-04-24T08:48:29.991013+0000","last_unstale":"2024-04-24T08:48:36.498428+0000","last_undegraded":"2024-04-24T08:48:36.498428+0000","last_fullsized":"2024-04-24T08:48:36.498428+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:10:39.381353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.501016+0000","last_change":"2024-04-24T08:48:29.983664+0000","last_active":"2024-04-24T08:48:36.501016+0000","last_peered":"2024-04-24T08:48:36.501016+0000","last_clean":"2024-04-24T08:48:36.501016+0000","last_became_active":"2024-04-24T08:48:29.978835+0000","last_became_peered":"2024-04-24T08:48:29.978835+0000","last_unstale":"2024-04-24T08:48:36.501016+0000","last_undegraded":"2024-04-24T08:48:36.501016+0000","last_fullsized":"2024-04-24T08:48:36.501016+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:29:20.501652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.498099+0000","last_change":"2024-04-24T08:48:29.989886+0000","last_active":"2024-04-24T08:48:36.498099+0000","last_peered":"2024-04-24T08:48:36.498099+0000","last_clean":"2024-04-24T08:48:36.498099+0000","last_became_active":"2024-04-24T08:48:29.989574+0000","last_became_peered":"2024-04-24T08:48:29.989574+0000","last_unstale":"2024-04-24T08:48:36.498099+0000","last_undegraded":"2024-04-24T08:48:36.498099+0000","last_fullsized":"2024-04-24T08:48:36.498099+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:14:01.085543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.501305+0000","last_change":"2024-04-24T08:48:29.978045+0000","last_active":"2024-04-24T08:48:36.501305+0000","last_peered":"2024-04-24T08:48:36.501305+0000","last_clean":"2024-04-24T08:48:36.501305+0000","last_became_active":"2024-04-24T08:48:29.977729+0000","last_became_peered":"2024-04-24T08:48:29.977729+0000","last_unstale":"2024-04-24T08:48:36.501305+0000","last_undegraded":"2024-04-24T08:48:36.501305+0000","last_fullsized":"2024-04-24T08:48:36.501305+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:03:33.382473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:35.290755+0000","last_change":"2024-04-24T08:48:29.976355+0000","last_active":"2024-04-24T08:48:35.290755+0000","last_peered":"2024-04-24T08:48:35.290755+0000","last_clean":"2024-04-24T08:48:35.290755+0000","last_became_active":"2024-04-24T08:48:29.975849+0000","last_became_peered":"2024-04-24T08:48:29.975849+0000","last_unstale":"2024-04-24T08:48:35.290755+0000","last_undegraded":"2024-04-24T08:48:35.290755+0000","last_fullsized":"2024-04-24T08:48:35.290755+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:13:51.302995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.498709+0000","last_change":"2024-04-24T08:48:29.983149+0000","last_active":"2024-04-24T08:48:36.498709+0000","last_peered":"2024-04-24T08:48:36.498709+0000","last_clean":"2024-04-24T08:48:36.498709+0000","last_became_active":"2024-04-24T08:48:29.982692+0000","last_became_peered":"2024-04-24T08:48:29.982692+0000","last_unstale":"2024-04-24T08:48:36.498709+0000","last_undegraded":"2024-04-24T08:48:36.498709+0000","last_fullsized":"2024-04-24T08:48:36.498709+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:59:06.151441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.503665+0000","last_change":"2024-04-24T08:48:29.985531+0000","last_active":"2024-04-24T08:48:36.503665+0000","last_peered":"2024-04-24T08:48:36.503665+0000","last_clean":"2024-04-24T08:48:36.503665+0000","last_became_active":"2024-04-24T08:48:29.982484+0000","last_became_peered":"2024-04-24T08:48:29.982484+0000","last_unstale":"2024-04-24T08:48:36.503665+0000","last_undegraded":"2024-04-24T08:48:36.503665+0000","last_fullsized":"2024-04-24T08:48:36.503665+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:03:50.114096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.502970+0000","last_change":"2024-04-24T08:48:29.982016+0000","last_active":"2024-04-24T08:48:36.502970+0000","last_peered":"2024-04-24T08:48:36.502970+0000","last_clean":"2024-04-24T08:48:36.502970+0000","last_became_active":"2024-04-24T08:48:29.977639+0000","last_became_peered":"2024-04-24T08:48:29.977639+0000","last_unstale":"2024-04-24T08:48:36.502970+0000","last_undegraded":"2024-04-24T08:48:36.502970+0000","last_fullsized":"2024-04-24T08:48:36.502970+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:35:48.676822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:35.292353+0000","last_change":"2024-04-24T08:48:29.977982+0000","last_active":"2024-04-24T08:48:35.292353+0000","last_peered":"2024-04-24T08:48:35.292353+0000","last_clean":"2024-04-24T08:48:35.292353+0000","last_became_active":"2024-04-24T08:48:29.977664+0000","last_became_peered":"2024-04-24T08:48:29.977664+0000","last_unstale":"2024-04-24T08:48:35.292353+0000","last_undegraded":"2024-04-24T08:48:35.292353+0000","last_fullsized":"2024-04-24T08:48:35.292353+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:45:17.597291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.500944+0000","last_change":"2024-04-24T08:48:29.981615+0000","last_active":"2024-04-24T08:48:36.500944+0000","last_peered":"2024-04-24T08:48:36.500944+0000","last_clean":"2024-04-24T08:48:36.500944+0000","last_became_active":"2024-04-24T08:48:29.979292+0000","last_became_peered":"2024-04-24T08:48:29.979292+0000","last_unstale":"2024-04-24T08:48:36.500944+0000","last_undegraded":"2024-04-24T08:48:36.500944+0000","last_fullsized":"2024-04-24T08:48:36.500944+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:48:44.195626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.499761+0000","last_change":"2024-04-24T08:48:29.952632+0000","last_active":"2024-04-24T08:48:36.499761+0000","last_peered":"2024-04-24T08:48:36.499761+0000","last_clean":"2024-04-24T08:48:36.499761+0000","last_became_active":"2024-04-24T08:48:29.951862+0000","last_became_peered":"2024-04-24T08:48:29.951862+0000","last_unstale":"2024-04-24T08:48:36.499761+0000","last_undegraded":"2024-04-24T08:48:36.499761+0000","last_fullsized":"2024-04-24T08:48:36.499761+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:23:40.038517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.497179+0000","last_change":"2024-04-24T08:48:29.984727+0000","last_active":"2024-04-24T08:48:36.497179+0000","last_peered":"2024-04-24T08:48:36.497179+0000","last_clean":"2024-04-24T08:48:36.497179+0000","last_became_active":"2024-04-24T08:48:29.983530+0000","last_became_peered":"2024-04-24T08:48:29.983530+0000","last_unstale":"2024-04-24T08:48:36.497179+0000","last_undegraded":"2024-04-24T08:48:36.497179+0000","last_fullsized":"2024-04-24T08:48:36.497179+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:00:12.729972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.497226+0000","last_change":"2024-04-24T08:48:29.981023+0000","last_active":"2024-04-24T08:48:36.497226+0000","last_peered":"2024-04-24T08:48:36.497226+0000","last_clean":"2024-04-24T08:48:36.497226+0000","last_became_active":"2024-04-24T08:48:29.979986+0000","last_became_peered":"2024-04-24T08:48:29.979986+0000","last_unstale":"2024-04-24T08:48:36.497226+0000","last_undegraded":"2024-04-24T08:48:36.497226+0000","last_fullsized":"2024-04-24T08:48:36.497226+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:05:36.163209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.499105+0000","last_change":"2024-04-24T08:48:29.989036+0000","last_active":"2024-04-24T08:48:36.499105+0000","last_peered":"2024-04-24T08:48:36.499105+0000","last_clean":"2024-04-24T08:48:36.499105+0000","last_became_active":"2024-04-24T08:48:29.988676+0000","last_became_peered":"2024-04-24T08:48:29.988676+0000","last_unstale":"2024-04-24T08:48:36.499105+0000","last_undegraded":"2024-04-24T08:48:36.499105+0000","last_fullsized":"2024-04-24T08:48:36.499105+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:03:38.544596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:35.326627+0000","last_change":"2024-04-24T08:48:29.986424+0000","last_active":"2024-04-24T08:48:35.326627+0000","last_peered":"2024-04-24T08:48:35.326627+0000","last_clean":"2024-04-24T08:48:35.326627+0000","last_became_active":"2024-04-24T08:48:29.985973+0000","last_became_peered":"2024-04-24T08:48:29.985973+0000","last_unstale":"2024-04-24T08:48:35.326627+0000","last_undegraded":"2024-04-24T08:48:35.326627+0000","last_fullsized":"2024-04-24T08:48:35.326627+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:48:31.533282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:35.290609+0000","last_change":"2024-04-24T08:48:29.975470+0000","last_active":"2024-04-24T08:48:35.290609+0000","last_peered":"2024-04-24T08:48:35.290609+0000","last_clean":"2024-04-24T08:48:35.290609+0000","last_became_active":"2024-04-24T08:48:29.974777+0000","last_became_peered":"2024-04-24T08:48:29.974777+0000","last_unstale":"2024-04-24T08:48:35.290609+0000","last_undegraded":"2024-04-24T08:48:35.290609+0000","last_fullsized":"2024-04-24T08:48:35.290609+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:04:26.423840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:35.291868+0000","last_change":"2024-04-24T08:48:29.988001+0000","last_active":"2024-04-24T08:48:35.291868+0000","last_peered":"2024-04-24T08:48:35.291868+0000","last_clean":"2024-04-24T08:48:35.291868+0000","last_became_active":"2024-04-24T08:48:29.987669+0000","last_became_peered":"2024-04-24T08:48:29.987669+0000","last_unstale":"2024-04-24T08:48:35.291868+0000","last_undegraded":"2024-04-24T08:48:35.291868+0000","last_fullsized":"2024-04-24T08:48:35.291868+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:29:24.457369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:35.292296+0000","last_change":"2024-04-24T08:48:34.308732+0000","last_active":"2024-04-24T08:48:35.292296+0000","last_peered":"2024-04-24T08:48:35.292296+0000","last_clean":"2024-04-24T08:48:35.292296+0000","last_became_active":"2024-04-24T08:48:34.307598+0000","last_became_peered":"2024-04-24T08:48:34.307598+0000","last_unstale":"2024-04-24T08:48:35.292296+0000","last_undegraded":"2024-04-24T08:48:35.292296+0000","last_fullsized":"2024-04-24T08:48:35.292296+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:28:57.260836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'9","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:49:02.055766+0000","last_change":"2024-04-24T08:48:28.970230+0000","last_active":"2024-04-24T08:49:02.055766+0000","last_peered":"2024-04-24T08:49:02.055766+0000","last_clean":"2024-04-24T08:49:02.055766+0000","last_became_active":"2024-04-24T08:48:28.969878+0000","last_became_peered":"2024-04-24T08:48:28.969878+0000","last_unstale":"2024-04-24T08:49:02.055766+0000","last_undegraded":"2024-04-24T08:49:02.055766+0000","last_fullsized":"2024-04-24T08:49:02.055766+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:05:19.206557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.500894+0000","last_change":"2024-04-24T08:48:29.986350+0000","last_active":"2024-04-24T08:48:36.500894+0000","last_peered":"2024-04-24T08:48:36.500894+0000","last_clean":"2024-04-24T08:48:36.500894+0000","last_became_active":"2024-04-24T08:48:29.985167+0000","last_became_peered":"2024-04-24T08:48:29.985167+0000","last_unstale":"2024-04-24T08:48:36.500894+0000","last_undegraded":"2024-04-24T08:48:36.500894+0000","last_fullsized":"2024-04-24T08:48:36.500894+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:43:33.326808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.498889+0000","last_change":"2024-04-24T08:48:34.315159+0000","last_active":"2024-04-24T08:48:36.498889+0000","last_peered":"2024-04-24T08:48:36.498889+0000","last_clean":"2024-04-24T08:48:36.498889+0000","last_became_active":"2024-04-24T08:48:34.312119+0000","last_became_peered":"2024-04-24T08:48:34.312119+0000","last_unstale":"2024-04-24T08:48:36.498889+0000","last_undegraded":"2024-04-24T08:48:36.498889+0000","last_fullsized":"2024-04-24T08:48:36.498889+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:56:08.024123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.501331+0000","last_change":"2024-04-24T08:48:28.971787+0000","last_active":"2024-04-24T08:48:36.501331+0000","last_peered":"2024-04-24T08:48:36.501331+0000","last_clean":"2024-04-24T08:48:36.501331+0000","last_became_active":"2024-04-24T08:48:28.971479+0000","last_became_peered":"2024-04-24T08:48:28.971479+0000","last_unstale":"2024-04-24T08:48:36.501331+0000","last_undegraded":"2024-04-24T08:48:36.501331+0000","last_fullsized":"2024-04-24T08:48:36.501331+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:09:52.498833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.498297+0000","last_change":"2024-04-24T08:48:29.989737+0000","last_active":"2024-04-24T08:48:36.498297+0000","last_peered":"2024-04-24T08:48:36.498297+0000","last_clean":"2024-04-24T08:48:36.498297+0000","last_became_active":"2024-04-24T08:48:29.989341+0000","last_became_peered":"2024-04-24T08:48:29.989341+0000","last_unstale":"2024-04-24T08:48:36.498297+0000","last_undegraded":"2024-04-24T08:48:36.498297+0000","last_fullsized":"2024-04-24T08:48:36.498297+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:02:32.714848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.499895+0000","last_change":"2024-04-24T08:48:34.322666+0000","last_active":"2024-04-24T08:48:36.499895+0000","last_peered":"2024-04-24T08:48:36.499895+0000","last_clean":"2024-04-24T08:48:36.499895+0000","last_became_active":"2024-04-24T08:48:34.321546+0000","last_became_peered":"2024-04-24T08:48:34.321546+0000","last_unstale":"2024-04-24T08:48:36.499895+0000","last_undegraded":"2024-04-24T08:48:36.499895+0000","last_fullsized":"2024-04-24T08:48:36.499895+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:42:45.066043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'11","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:49:04.102263+0000","last_change":"2024-04-24T08:48:28.978738+0000","last_active":"2024-04-24T08:49:04.102263+0000","last_peered":"2024-04-24T08:49:04.102263+0000","last_clean":"2024-04-24T08:49:04.102263+0000","last_became_active":"2024-04-24T08:48:28.978378+0000","last_became_peered":"2024-04-24T08:48:28.978378+0000","last_unstale":"2024-04-24T08:49:04.102263+0000","last_undegraded":"2024-04-24T08:49:04.102263+0000","last_fullsized":"2024-04-24T08:49:04.102263+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:46:22.668234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.501908+0000","last_change":"2024-04-24T08:48:29.974588+0000","last_active":"2024-04-24T08:48:36.501908+0000","last_peered":"2024-04-24T08:48:36.501908+0000","last_clean":"2024-04-24T08:48:36.501908+0000","last_became_active":"2024-04-24T08:48:29.974190+0000","last_became_peered":"2024-04-24T08:48:29.974190+0000","last_unstale":"2024-04-24T08:48:36.501908+0000","last_undegraded":"2024-04-24T08:48:36.501908+0000","last_fullsized":"2024-04-24T08:48:36.501908+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:07:12.948660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.499360+0000","last_change":"2024-04-24T08:48:34.321954+0000","last_active":"2024-04-24T08:48:36.499360+0000","last_peered":"2024-04-24T08:48:36.499360+0000","last_clean":"2024-04-24T08:48:36.499360+0000","last_became_active":"2024-04-24T08:48:34.321417+0000","last_became_peered":"2024-04-24T08:48:34.321417+0000","last_unstale":"2024-04-24T08:48:36.499360+0000","last_undegraded":"2024-04-24T08:48:36.499360+0000","last_fullsized":"2024-04-24T08:48:36.499360+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:39:07.205709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:49:04.097612+0000","last_change":"2024-04-24T08:48:28.974464+0000","last_active":"2024-04-24T08:49:04.097612+0000","last_peered":"2024-04-24T08:49:04.097612+0000","last_clean":"2024-04-24T08:49:04.097612+0000","last_became_active":"2024-04-24T08:48:28.974001+0000","last_became_peered":"2024-04-24T08:48:28.974001+0000","last_unstale":"2024-04-24T08:49:04.097612+0000","last_undegraded":"2024-04-24T08:49:04.097612+0000","last_fullsized":"2024-04-24T08:49:04.097612+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:38:55.193427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.063853+0000","last_change":"2024-04-24T08:48:29.981336+0000","last_active":"2024-04-24T08:48:36.063853+0000","last_peered":"2024-04-24T08:48:36.063853+0000","last_clean":"2024-04-24T08:48:36.063853+0000","last_became_active":"2024-04-24T08:48:29.980333+0000","last_became_peered":"2024-04-24T08:48:29.980333+0000","last_unstale":"2024-04-24T08:48:36.063853+0000","last_undegraded":"2024-04-24T08:48:36.063853+0000","last_fullsized":"2024-04-24T08:48:36.063853+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:46:03.640332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:35.291679+0000","last_change":"2024-04-24T08:48:34.323099+0000","last_active":"2024-04-24T08:48:35.291679+0000","last_peered":"2024-04-24T08:48:35.291679+0000","last_clean":"2024-04-24T08:48:35.291679+0000","last_became_active":"2024-04-24T08:48:34.322557+0000","last_became_peered":"2024-04-24T08:48:34.322557+0000","last_unstale":"2024-04-24T08:48:35.291679+0000","last_undegraded":"2024-04-24T08:48:35.291679+0000","last_fullsized":"2024-04-24T08:48:35.291679+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:59:59.887087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'6","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:49:05.241898+0000","last_change":"2024-04-24T08:48:28.978427+0000","last_active":"2024-04-24T08:49:05.241898+0000","last_peered":"2024-04-24T08:49:05.241898+0000","last_clean":"2024-04-24T08:49:05.241898+0000","last_became_active":"2024-04-24T08:48:28.977291+0000","last_became_peered":"2024-04-24T08:48:28.977291+0000","last_unstale":"2024-04-24T08:49:05.241898+0000","last_undegraded":"2024-04-24T08:49:05.241898+0000","last_fullsized":"2024-04-24T08:49:05.241898+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:03:03.085265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":2,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.498491+0000","last_change":"2024-04-24T08:48:29.978008+0000","last_active":"2024-04-24T08:48:36.498491+0000","last_peered":"2024-04-24T08:48:36.498491+0000","last_clean":"2024-04-24T08:48:36.498491+0000","last_became_active":"2024-04-24T08:48:29.977389+0000","last_became_peered":"2024-04-24T08:48:29.977389+0000","last_unstale":"2024-04-24T08:48:36.498491+0000","last_undegraded":"2024-04-24T08:48:36.498491+0000","last_fullsized":"2024-04-24T08:48:36.498491+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:35:28.035378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:35.291612+0000","last_change":"2024-04-24T08:48:34.321477+0000","last_active":"2024-04-24T08:48:35.291612+0000","last_peered":"2024-04-24T08:48:35.291612+0000","last_clean":"2024-04-24T08:48:35.291612+0000","last_became_active":"2024-04-24T08:48:34.320860+0000","last_became_peered":"2024-04-24T08:48:34.320860+0000","last_unstale":"2024-04-24T08:48:35.291612+0000","last_undegraded":"2024-04-24T08:48:35.291612+0000","last_fullsized":"2024-04-24T08:48:35.291612+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:26:07.862590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:49:03.074625+0000","last_change":"2024-04-24T08:48:28.978147+0000","last_active":"2024-04-24T08:49:03.074625+0000","last_peered":"2024-04-24T08:49:03.074625+0000","last_clean":"2024-04-24T08:49:03.074625+0000","last_became_active":"2024-04-24T08:48:28.977646+0000","last_became_peered":"2024-04-24T08:48:28.977646+0000","last_unstale":"2024-04-24T08:49:03.074625+0000","last_undegraded":"2024-04-24T08:49:03.074625+0000","last_fullsized":"2024-04-24T08:49:03.074625+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:31:35.557886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.064424+0000","last_change":"2024-04-24T08:48:29.990788+0000","last_active":"2024-04-24T08:48:36.064424+0000","last_peered":"2024-04-24T08:48:36.064424+0000","last_clean":"2024-04-24T08:48:36.064424+0000","last_became_active":"2024-04-24T08:48:29.988476+0000","last_became_peered":"2024-04-24T08:48:29.988476+0000","last_unstale":"2024-04-24T08:48:36.064424+0000","last_undegraded":"2024-04-24T08:48:36.064424+0000","last_fullsized":"2024-04-24T08:48:36.064424+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:08:48.389877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:35.326254+0000","last_change":"2024-04-24T08:48:34.311020+0000","last_active":"2024-04-24T08:48:35.326254+0000","last_peered":"2024-04-24T08:48:35.326254+0000","last_clean":"2024-04-24T08:48:35.326254+0000","last_became_active":"2024-04-24T08:48:34.310395+0000","last_became_peered":"2024-04-24T08:48:34.310395+0000","last_unstale":"2024-04-24T08:48:35.326254+0000","last_undegraded":"2024-04-24T08:48:35.326254+0000","last_fullsized":"2024-04-24T08:48:35.326254+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:55:38.704954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:49:02.051829+0000","last_change":"2024-04-24T08:48:28.969991+0000","last_active":"2024-04-24T08:49:02.051829+0000","last_peered":"2024-04-24T08:49:02.051829+0000","last_clean":"2024-04-24T08:49:02.051829+0000","last_became_active":"2024-04-24T08:48:28.967937+0000","last_became_peered":"2024-04-24T08:48:28.967937+0000","last_unstale":"2024-04-24T08:49:02.051829+0000","last_undegraded":"2024-04-24T08:49:02.051829+0000","last_fullsized":"2024-04-24T08:49:02.051829+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:09:49.736108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.498485+0000","last_change":"2024-04-24T08:48:29.992814+0000","last_active":"2024-04-24T08:48:36.498485+0000","last_peered":"2024-04-24T08:48:36.498485+0000","last_clean":"2024-04-24T08:48:36.498485+0000","last_became_active":"2024-04-24T08:48:29.992188+0000","last_became_peered":"2024-04-24T08:48:29.992188+0000","last_unstale":"2024-04-24T08:48:36.498485+0000","last_undegraded":"2024-04-24T08:48:36.498485+0000","last_fullsized":"2024-04-24T08:48:36.498485+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:29:33.974958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.498447+0000","last_change":"2024-04-24T08:48:34.311196+0000","last_active":"2024-04-24T08:48:36.498447+0000","last_peered":"2024-04-24T08:48:36.498447+0000","last_clean":"2024-04-24T08:48:36.498447+0000","last_became_active":"2024-04-24T08:48:34.310723+0000","last_became_peered":"2024-04-24T08:48:34.310723+0000","last_unstale":"2024-04-24T08:48:36.498447+0000","last_undegraded":"2024-04-24T08:48:36.498447+0000","last_fullsized":"2024-04-24T08:48:36.498447+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:04:30.149510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:49:04.102749+0000","last_change":"2024-04-24T08:48:28.978534+0000","last_active":"2024-04-24T08:49:04.102749+0000","last_peered":"2024-04-24T08:49:04.102749+0000","last_clean":"2024-04-24T08:49:04.102749+0000","last_became_active":"2024-04-24T08:48:28.975407+0000","last_became_peered":"2024-04-24T08:48:28.975407+0000","last_unstale":"2024-04-24T08:49:04.102749+0000","last_undegraded":"2024-04-24T08:49:04.102749+0000","last_fullsized":"2024-04-24T08:49:04.102749+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:10:45.355039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:35.288846+0000","last_change":"2024-04-24T08:48:29.997682+0000","last_active":"2024-04-24T08:48:35.288846+0000","last_peered":"2024-04-24T08:48:35.288846+0000","last_clean":"2024-04-24T08:48:35.288846+0000","last_became_active":"2024-04-24T08:48:29.997223+0000","last_became_peered":"2024-04-24T08:48:29.997223+0000","last_unstale":"2024-04-24T08:48:35.288846+0000","last_undegraded":"2024-04-24T08:48:35.288846+0000","last_fullsized":"2024-04-24T08:48:35.288846+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:03:00.807249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.500029+0000","last_change":"2024-04-24T08:48:34.321477+0000","last_active":"2024-04-24T08:48:36.500029+0000","last_peered":"2024-04-24T08:48:36.500029+0000","last_clean":"2024-04-24T08:48:36.500029+0000","last_became_active":"2024-04-24T08:48:34.321018+0000","last_became_peered":"2024-04-24T08:48:34.321018+0000","last_unstale":"2024-04-24T08:48:36.500029+0000","last_undegraded":"2024-04-24T08:48:36.500029+0000","last_fullsized":"2024-04-24T08:48:36.500029+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:56:50.657298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'4","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:49:01.046781+0000","last_change":"2024-04-24T08:48:28.983722+0000","last_active":"2024-04-24T08:49:01.046781+0000","last_peered":"2024-04-24T08:49:01.046781+0000","last_clean":"2024-04-24T08:49:01.046781+0000","last_became_active":"2024-04-24T08:48:28.983394+0000","last_became_peered":"2024-04-24T08:48:28.983394+0000","last_unstale":"2024-04-24T08:49:01.046781+0000","last_undegraded":"2024-04-24T08:49:01.046781+0000","last_fullsized":"2024-04-24T08:49:01.046781+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:53:14.288742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.503838+0000","last_change":"2024-04-24T08:48:29.975009+0000","last_active":"2024-04-24T08:48:36.503838+0000","last_peered":"2024-04-24T08:48:36.503838+0000","last_clean":"2024-04-24T08:48:36.503838+0000","last_became_active":"2024-04-24T08:48:29.974137+0000","last_became_peered":"2024-04-24T08:48:29.974137+0000","last_unstale":"2024-04-24T08:48:36.503838+0000","last_undegraded":"2024-04-24T08:48:36.503838+0000","last_fullsized":"2024-04-24T08:48:36.503838+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:13:16.381923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:35.288980+0000","last_change":"2024-04-24T08:48:34.312126+0000","last_active":"2024-04-24T08:48:35.288980+0000","last_peered":"2024-04-24T08:48:35.288980+0000","last_clean":"2024-04-24T08:48:35.288980+0000","last_became_active":"2024-04-24T08:48:34.311689+0000","last_became_peered":"2024-04-24T08:48:34.311689+0000","last_unstale":"2024-04-24T08:48:35.288980+0000","last_undegraded":"2024-04-24T08:48:35.288980+0000","last_fullsized":"2024-04-24T08:48:35.288980+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:11:07.942412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:49:04.098187+0000","last_change":"2024-04-24T08:48:28.978355+0000","last_active":"2024-04-24T08:49:04.098187+0000","last_peered":"2024-04-24T08:49:04.098187+0000","last_clean":"2024-04-24T08:49:04.098187+0000","last_became_active":"2024-04-24T08:48:28.977351+0000","last_became_peered":"2024-04-24T08:48:28.977351+0000","last_unstale":"2024-04-24T08:49:04.098187+0000","last_undegraded":"2024-04-24T08:49:04.098187+0000","last_fullsized":"2024-04-24T08:49:04.098187+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:00:12.836569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.499015+0000","last_change":"2024-04-24T08:48:29.989015+0000","last_active":"2024-04-24T08:48:36.499015+0000","last_peered":"2024-04-24T08:48:36.499015+0000","last_clean":"2024-04-24T08:48:36.499015+0000","last_became_active":"2024-04-24T08:48:29.988648+0000","last_became_peered":"2024-04-24T08:48:29.988648+0000","last_unstale":"2024-04-24T08:48:36.499015+0000","last_undegraded":"2024-04-24T08:48:36.499015+0000","last_fullsized":"2024-04-24T08:48:36.499015+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:04:52.824894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.500746+0000","last_change":"2024-04-24T08:48:34.314230+0000","last_active":"2024-04-24T08:48:36.500746+0000","last_peered":"2024-04-24T08:48:36.500746+0000","last_clean":"2024-04-24T08:48:36.500746+0000","last_became_active":"2024-04-24T08:48:34.313617+0000","last_became_peered":"2024-04-24T08:48:34.313617+0000","last_unstale":"2024-04-24T08:48:36.500746+0000","last_undegraded":"2024-04-24T08:48:36.500746+0000","last_fullsized":"2024-04-24T08:48:36.500746+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:06:02.307575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:49:03.071387+0000","last_change":"2024-04-24T08:48:28.974516+0000","last_active":"2024-04-24T08:49:03.071387+0000","last_peered":"2024-04-24T08:49:03.071387+0000","last_clean":"2024-04-24T08:49:03.071387+0000","last_became_active":"2024-04-24T08:48:28.974194+0000","last_became_peered":"2024-04-24T08:48:28.974194+0000","last_unstale":"2024-04-24T08:49:03.071387+0000","last_undegraded":"2024-04-24T08:49:03.071387+0000","last_fullsized":"2024-04-24T08:49:03.071387+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:03:38.582610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.502357+0000","last_change":"2024-04-24T08:48:29.981132+0000","last_active":"2024-04-24T08:48:36.502357+0000","last_peered":"2024-04-24T08:48:36.502357+0000","last_clean":"2024-04-24T08:48:36.502357+0000","last_became_active":"2024-04-24T08:48:29.980252+0000","last_became_peered":"2024-04-24T08:48:29.980252+0000","last_unstale":"2024-04-24T08:48:36.502357+0000","last_undegraded":"2024-04-24T08:48:36.502357+0000","last_fullsized":"2024-04-24T08:48:36.502357+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:43:28.632117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:35.289424+0000","last_change":"2024-04-24T08:48:34.312585+0000","last_active":"2024-04-24T08:48:35.289424+0000","last_peered":"2024-04-24T08:48:35.289424+0000","last_clean":"2024-04-24T08:48:35.289424+0000","last_became_active":"2024-04-24T08:48:34.311289+0000","last_became_peered":"2024-04-24T08:48:34.311289+0000","last_unstale":"2024-04-24T08:48:35.289424+0000","last_undegraded":"2024-04-24T08:48:35.289424+0000","last_fullsized":"2024-04-24T08:48:35.289424+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:01:20.080310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"89'12","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:49:01.044855+0000","last_change":"2024-04-24T08:48:28.981675+0000","last_active":"2024-04-24T08:49:01.044855+0000","last_peered":"2024-04-24T08:49:01.044855+0000","last_clean":"2024-04-24T08:49:01.044855+0000","last_became_active":"2024-04-24T08:48:28.981082+0000","last_became_peered":"2024-04-24T08:48:28.981082+0000","last_unstale":"2024-04-24T08:49:01.044855+0000","last_undegraded":"2024-04-24T08:49:01.044855+0000","last_fullsized":"2024-04-24T08:49:01.044855+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:22:07.826011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.498801+0000","last_change":"2024-04-24T08:48:29.986932+0000","last_active":"2024-04-24T08:48:36.498801+0000","last_peered":"2024-04-24T08:48:36.498801+0000","last_clean":"2024-04-24T08:48:36.498801+0000","last_became_active":"2024-04-24T08:48:29.986505+0000","last_became_peered":"2024-04-24T08:48:29.986505+0000","last_unstale":"2024-04-24T08:48:36.498801+0000","last_undegraded":"2024-04-24T08:48:36.498801+0000","last_fullsized":"2024-04-24T08:48:36.498801+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:53:36.050294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.064820+0000","last_change":"2024-04-24T08:48:34.327127+0000","last_active":"2024-04-24T08:48:36.064820+0000","last_peered":"2024-04-24T08:48:36.064820+0000","last_clean":"2024-04-24T08:48:36.064820+0000","last_became_active":"2024-04-24T08:48:34.323942+0000","last_became_peered":"2024-04-24T08:48:34.323942+0000","last_unstale":"2024-04-24T08:48:36.064820+0000","last_undegraded":"2024-04-24T08:48:36.064820+0000","last_fullsized":"2024-04-24T08:48:36.064820+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:22:49.637513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'4","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:49:01.044674+0000","last_change":"2024-04-24T08:48:28.978618+0000","last_active":"2024-04-24T08:49:01.044674+0000","last_peered":"2024-04-24T08:49:01.044674+0000","last_clean":"2024-04-24T08:49:01.044674+0000","last_became_active":"2024-04-24T08:48:28.978309+0000","last_became_peered":"2024-04-24T08:48:28.978309+0000","last_unstale":"2024-04-24T08:49:01.044674+0000","last_undegraded":"2024-04-24T08:49:01.044674+0000","last_fullsized":"2024-04-24T08:49:01.044674+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:35:50.091398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.499248+0000","last_change":"2024-04-24T08:48:29.985629+0000","last_active":"2024-04-24T08:48:36.499248+0000","last_peered":"2024-04-24T08:48:36.499248+0000","last_clean":"2024-04-24T08:48:36.499248+0000","last_became_active":"2024-04-24T08:48:29.985258+0000","last_became_peered":"2024-04-24T08:48:29.985258+0000","last_unstale":"2024-04-24T08:48:36.499248+0000","last_undegraded":"2024-04-24T08:48:36.499248+0000","last_fullsized":"2024-04-24T08:48:36.499248+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:57:29.302175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:35.325632+0000","last_change":"2024-04-24T08:48:34.314516+0000","last_active":"2024-04-24T08:48:35.325632+0000","last_peered":"2024-04-24T08:48:35.325632+0000","last_clean":"2024-04-24T08:48:35.325632+0000","last_became_active":"2024-04-24T08:48:34.313775+0000","last_became_peered":"2024-04-24T08:48:34.313775+0000","last_unstale":"2024-04-24T08:48:35.325632+0000","last_undegraded":"2024-04-24T08:48:35.325632+0000","last_fullsized":"2024-04-24T08:48:35.325632+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:06:44.003226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:49:04.102859+0000","last_change":"2024-04-24T08:48:28.977840+0000","last_active":"2024-04-24T08:49:04.102859+0000","last_peered":"2024-04-24T08:49:04.102859+0000","last_clean":"2024-04-24T08:49:04.102859+0000","last_became_active":"2024-04-24T08:48:28.975482+0000","last_became_peered":"2024-04-24T08:48:28.975482+0000","last_unstale":"2024-04-24T08:49:04.102859+0000","last_undegraded":"2024-04-24T08:49:04.102859+0000","last_fullsized":"2024-04-24T08:49:04.102859+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:51:52.468627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:35.292371+0000","last_change":"2024-04-24T08:48:29.976802+0000","last_active":"2024-04-24T08:48:35.292371+0000","last_peered":"2024-04-24T08:48:35.292371+0000","last_clean":"2024-04-24T08:48:35.292371+0000","last_became_active":"2024-04-24T08:48:29.976457+0000","last_became_peered":"2024-04-24T08:48:29.976457+0000","last_unstale":"2024-04-24T08:48:35.292371+0000","last_undegraded":"2024-04-24T08:48:35.292371+0000","last_fullsized":"2024-04-24T08:48:35.292371+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:55:31.465512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:35.290931+0000","last_change":"2024-04-24T08:48:34.314956+0000","last_active":"2024-04-24T08:48:35.290931+0000","last_peered":"2024-04-24T08:48:35.290931+0000","last_clean":"2024-04-24T08:48:35.290931+0000","last_became_active":"2024-04-24T08:48:34.314541+0000","last_became_peered":"2024-04-24T08:48:34.314541+0000","last_unstale":"2024-04-24T08:48:35.290931+0000","last_undegraded":"2024-04-24T08:48:35.290931+0000","last_fullsized":"2024-04-24T08:48:35.290931+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:42:48.614000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"89'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:49:02.049535+0000","last_change":"2024-04-24T08:48:28.974529+0000","last_active":"2024-04-24T08:49:02.049535+0000","last_peered":"2024-04-24T08:49:02.049535+0000","last_clean":"2024-04-24T08:49:02.049535+0000","last_became_active":"2024-04-24T08:48:28.973957+0000","last_became_peered":"2024-04-24T08:48:28.973957+0000","last_unstale":"2024-04-24T08:49:02.049535+0000","last_undegraded":"2024-04-24T08:49:02.049535+0000","last_fullsized":"2024-04-24T08:49:02.049535+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:44:46.216389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:35.290729+0000","last_change":"2024-04-24T08:48:29.981995+0000","last_active":"2024-04-24T08:48:35.290729+0000","last_peered":"2024-04-24T08:48:35.290729+0000","last_clean":"2024-04-24T08:48:35.290729+0000","last_became_active":"2024-04-24T08:48:29.980756+0000","last_became_peered":"2024-04-24T08:48:29.980756+0000","last_unstale":"2024-04-24T08:48:35.290729+0000","last_undegraded":"2024-04-24T08:48:35.290729+0000","last_fullsized":"2024-04-24T08:48:35.290729+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:42:52.770069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.501786+0000","last_change":"2024-04-24T08:48:34.308836+0000","last_active":"2024-04-24T08:48:36.501786+0000","last_peered":"2024-04-24T08:48:36.501786+0000","last_clean":"2024-04-24T08:48:36.501786+0000","last_became_active":"2024-04-24T08:48:34.308470+0000","last_became_peered":"2024-04-24T08:48:34.308470+0000","last_unstale":"2024-04-24T08:48:36.501786+0000","last_undegraded":"2024-04-24T08:48:36.501786+0000","last_fullsized":"2024-04-24T08:48:36.501786+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:38:14.906117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:49:04.091588+0000","last_change":"2024-04-24T08:48:28.971818+0000","last_active":"2024-04-24T08:49:04.091588+0000","last_peered":"2024-04-24T08:49:04.091588+0000","last_clean":"2024-04-24T08:49:04.091588+0000","last_became_active":"2024-04-24T08:48:28.971463+0000","last_became_peered":"2024-04-24T08:48:28.971463+0000","last_unstale":"2024-04-24T08:49:04.091588+0000","last_undegraded":"2024-04-24T08:49:04.091588+0000","last_fullsized":"2024-04-24T08:49:04.091588+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:10:22.829722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.063542+0000","last_change":"2024-04-24T08:48:29.980632+0000","last_active":"2024-04-24T08:48:36.063542+0000","last_peered":"2024-04-24T08:48:36.063542+0000","last_clean":"2024-04-24T08:48:36.063542+0000","last_became_active":"2024-04-24T08:48:29.980174+0000","last_became_peered":"2024-04-24T08:48:29.980174+0000","last_unstale":"2024-04-24T08:48:36.063542+0000","last_undegraded":"2024-04-24T08:48:36.063542+0000","last_fullsized":"2024-04-24T08:48:36.063542+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:45:32.709091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.063394+0000","last_change":"2024-04-24T08:48:34.316302+0000","last_active":"2024-04-24T08:48:36.063394+0000","last_peered":"2024-04-24T08:48:36.063394+0000","last_clean":"2024-04-24T08:48:36.063394+0000","last_became_active":"2024-04-24T08:48:34.315783+0000","last_became_peered":"2024-04-24T08:48:34.315783+0000","last_unstale":"2024-04-24T08:48:36.063394+0000","last_undegraded":"2024-04-24T08:48:36.063394+0000","last_fullsized":"2024-04-24T08:48:36.063394+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:29:01.733931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:49:04.042315+0000","last_change":"2024-04-24T08:48:28.972033+0000","last_active":"2024-04-24T08:49:04.042315+0000","last_peered":"2024-04-24T08:49:04.042315+0000","last_clean":"2024-04-24T08:49:04.042315+0000","last_became_active":"2024-04-24T08:48:28.971597+0000","last_became_peered":"2024-04-24T08:48:28.971597+0000","last_unstale":"2024-04-24T08:49:04.042315+0000","last_undegraded":"2024-04-24T08:49:04.042315+0000","last_fullsized":"2024-04-24T08:49:04.042315+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:11:27.827599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:35.325942+0000","last_change":"2024-04-24T08:48:29.984679+0000","last_active":"2024-04-24T08:48:35.325942+0000","last_peered":"2024-04-24T08:48:35.325942+0000","last_clean":"2024-04-24T08:48:35.325942+0000","last_became_active":"2024-04-24T08:48:29.984323+0000","last_became_peered":"2024-04-24T08:48:29.984323+0000","last_unstale":"2024-04-24T08:48:35.325942+0000","last_undegraded":"2024-04-24T08:48:35.325942+0000","last_fullsized":"2024-04-24T08:48:35.325942+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:57:46.885873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.498574+0000","last_change":"2024-04-24T08:48:34.319324+0000","last_active":"2024-04-24T08:48:36.498574+0000","last_peered":"2024-04-24T08:48:36.498574+0000","last_clean":"2024-04-24T08:48:36.498574+0000","last_became_active":"2024-04-24T08:48:34.318828+0000","last_became_peered":"2024-04-24T08:48:34.318828+0000","last_unstale":"2024-04-24T08:48:36.498574+0000","last_undegraded":"2024-04-24T08:48:36.498574+0000","last_fullsized":"2024-04-24T08:48:36.498574+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:31:52.097032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:35.289683+0000","last_change":"2024-04-24T08:48:28.970726+0000","last_active":"2024-04-24T08:48:35.289683+0000","last_peered":"2024-04-24T08:48:35.289683+0000","last_clean":"2024-04-24T08:48:35.289683+0000","last_became_active":"2024-04-24T08:48:28.969911+0000","last_became_peered":"2024-04-24T08:48:28.969911+0000","last_unstale":"2024-04-24T08:48:35.289683+0000","last_undegraded":"2024-04-24T08:48:35.289683+0000","last_fullsized":"2024-04-24T08:48:35.289683+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:55:26.357256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.502488+0000","last_change":"2024-04-24T08:48:29.967804+0000","last_active":"2024-04-24T08:48:36.502488+0000","last_peered":"2024-04-24T08:48:36.502488+0000","last_clean":"2024-04-24T08:48:36.502488+0000","last_became_active":"2024-04-24T08:48:29.967431+0000","last_became_peered":"2024-04-24T08:48:29.967431+0000","last_unstale":"2024-04-24T08:48:36.502488+0000","last_undegraded":"2024-04-24T08:48:36.502488+0000","last_fullsized":"2024-04-24T08:48:36.502488+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:00:02.779543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.499507+0000","last_change":"2024-04-24T08:48:34.316844+0000","last_active":"2024-04-24T08:48:36.499507+0000","last_peered":"2024-04-24T08:48:36.499507+0000","last_clean":"2024-04-24T08:48:36.499507+0000","last_became_active":"2024-04-24T08:48:34.316426+0000","last_became_peered":"2024-04-24T08:48:34.316426+0000","last_unstale":"2024-04-24T08:48:36.499507+0000","last_undegraded":"2024-04-24T08:48:36.499507+0000","last_fullsized":"2024-04-24T08:48:36.499507+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:10:13.378443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'5","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:49:04.232354+0000","last_change":"2024-04-24T08:48:28.971544+0000","last_active":"2024-04-24T08:49:04.232354+0000","last_peered":"2024-04-24T08:49:04.232354+0000","last_clean":"2024-04-24T08:49:04.232354+0000","last_became_active":"2024-04-24T08:48:28.970417+0000","last_became_peered":"2024-04-24T08:48:28.970417+0000","last_unstale":"2024-04-24T08:49:04.232354+0000","last_undegraded":"2024-04-24T08:49:04.232354+0000","last_fullsized":"2024-04-24T08:49:04.232354+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:35:04.506126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.501666+0000","last_change":"2024-04-24T08:48:29.987542+0000","last_active":"2024-04-24T08:48:36.501666+0000","last_peered":"2024-04-24T08:48:36.501666+0000","last_clean":"2024-04-24T08:48:36.501666+0000","last_became_active":"2024-04-24T08:48:29.987258+0000","last_became_peered":"2024-04-24T08:48:29.987258+0000","last_unstale":"2024-04-24T08:48:36.501666+0000","last_undegraded":"2024-04-24T08:48:36.501666+0000","last_fullsized":"2024-04-24T08:48:36.501666+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:37:23.341132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.501684+0000","last_change":"2024-04-24T08:48:34.322636+0000","last_active":"2024-04-24T08:48:36.501684+0000","last_peered":"2024-04-24T08:48:36.501684+0000","last_clean":"2024-04-24T08:48:36.501684+0000","last_became_active":"2024-04-24T08:48:34.321764+0000","last_became_peered":"2024-04-24T08:48:34.321764+0000","last_unstale":"2024-04-24T08:48:36.501684+0000","last_undegraded":"2024-04-24T08:48:36.501684+0000","last_fullsized":"2024-04-24T08:48:36.501684+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:56:10.563907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.064050+0000","last_change":"2024-04-24T08:48:29.981734+0000","last_active":"2024-04-24T08:48:36.064050+0000","last_peered":"2024-04-24T08:48:36.064050+0000","last_clean":"2024-04-24T08:48:36.064050+0000","last_became_active":"2024-04-24T08:48:29.980491+0000","last_became_peered":"2024-04-24T08:48:29.980491+0000","last_unstale":"2024-04-24T08:48:36.064050+0000","last_undegraded":"2024-04-24T08:48:36.064050+0000","last_fullsized":"2024-04-24T08:48:36.064050+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:08:45.189026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:49:02.053122+0000","last_change":"2024-04-24T08:48:28.954621+0000","last_active":"2024-04-24T08:49:02.053122+0000","last_peered":"2024-04-24T08:49:02.053122+0000","last_clean":"2024-04-24T08:49:02.053122+0000","last_became_active":"2024-04-24T08:48:28.954139+0000","last_became_peered":"2024-04-24T08:48:28.954139+0000","last_unstale":"2024-04-24T08:49:02.053122+0000","last_undegraded":"2024-04-24T08:49:02.053122+0000","last_fullsized":"2024-04-24T08:49:02.053122+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:59:19.360414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:35.291923+0000","last_change":"2024-04-24T08:48:34.323774+0000","last_active":"2024-04-24T08:48:35.291923+0000","last_peered":"2024-04-24T08:48:35.291923+0000","last_clean":"2024-04-24T08:48:35.291923+0000","last_became_active":"2024-04-24T08:48:34.322214+0000","last_became_peered":"2024-04-24T08:48:34.322214+0000","last_unstale":"2024-04-24T08:48:35.291923+0000","last_undegraded":"2024-04-24T08:48:35.291923+0000","last_fullsized":"2024-04-24T08:48:35.291923+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:51:03.971107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.499633+0000","last_change":"2024-04-24T08:48:29.995372+0000","last_active":"2024-04-24T08:48:36.499633+0000","last_peered":"2024-04-24T08:48:36.499633+0000","last_clean":"2024-04-24T08:48:36.499633+0000","last_became_active":"2024-04-24T08:48:29.994744+0000","last_became_peered":"2024-04-24T08:48:29.994744+0000","last_unstale":"2024-04-24T08:48:36.499633+0000","last_undegraded":"2024-04-24T08:48:36.499633+0000","last_fullsized":"2024-04-24T08:48:36.499633+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:28:55.356390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:49:01.042628+0000","last_change":"2024-04-24T08:48:28.974921+0000","last_active":"2024-04-24T08:49:01.042628+0000","last_peered":"2024-04-24T08:49:01.042628+0000","last_clean":"2024-04-24T08:49:01.042628+0000","last_became_active":"2024-04-24T08:48:28.974113+0000","last_became_peered":"2024-04-24T08:48:28.974113+0000","last_unstale":"2024-04-24T08:49:01.042628+0000","last_undegraded":"2024-04-24T08:49:01.042628+0000","last_fullsized":"2024-04-24T08:49:01.042628+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:35:12.635253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":356,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:35.290533+0000","last_change":"2024-04-24T08:46:45.190615+0000","last_active":"2024-04-24T08:48:35.290533+0000","last_peered":"2024-04-24T08:48:35.290533+0000","last_clean":"2024-04-24T08:48:35.290533+0000","last_became_active":"2024-04-24T08:46:45.190011+0000","last_became_peered":"2024-04-24T08:46:45.190011+0000","last_unstale":"2024-04-24T08:48:35.290533+0000","last_undegraded":"2024-04-24T08:48:35.290533+0000","last_fullsized":"2024-04-24T08:48:35.290533+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:41:57.806569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:41:57.806569+0000","last_clean_scrub_stamp":"2024-04-24T08:41:57.806569+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:13:08.283236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.063697+0000","last_change":"2024-04-24T08:48:34.324588+0000","last_active":"2024-04-24T08:48:36.063697+0000","last_peered":"2024-04-24T08:48:36.063697+0000","last_clean":"2024-04-24T08:48:36.063697+0000","last_became_active":"2024-04-24T08:48:34.321969+0000","last_became_peered":"2024-04-24T08:48:34.321969+0000","last_unstale":"2024-04-24T08:48:36.063697+0000","last_undegraded":"2024-04-24T08:48:36.063697+0000","last_fullsized":"2024-04-24T08:48:36.063697+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:19:56.921198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.498162+0000","last_change":"2024-04-24T08:48:29.981573+0000","last_active":"2024-04-24T08:48:36.498162+0000","last_peered":"2024-04-24T08:48:36.498162+0000","last_clean":"2024-04-24T08:48:36.498162+0000","last_became_active":"2024-04-24T08:48:29.979216+0000","last_became_peered":"2024-04-24T08:48:29.979216+0000","last_unstale":"2024-04-24T08:48:36.498162+0000","last_undegraded":"2024-04-24T08:48:36.498162+0000","last_fullsized":"2024-04-24T08:48:36.498162+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:38:09.164883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:49:02.055998+0000","last_change":"2024-04-24T08:48:28.981492+0000","last_active":"2024-04-24T08:49:02.055998+0000","last_peered":"2024-04-24T08:49:02.055998+0000","last_clean":"2024-04-24T08:49:02.055998+0000","last_became_active":"2024-04-24T08:48:28.981050+0000","last_became_peered":"2024-04-24T08:48:28.981050+0000","last_unstale":"2024-04-24T08:49:02.055998+0000","last_undegraded":"2024-04-24T08:49:02.055998+0000","last_fullsized":"2024-04-24T08:49:02.055998+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:22:09.829566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:35.325781+0000","last_change":"2024-04-24T08:48:34.317502+0000","last_active":"2024-04-24T08:48:35.325781+0000","last_peered":"2024-04-24T08:48:35.325781+0000","last_clean":"2024-04-24T08:48:35.325781+0000","last_became_active":"2024-04-24T08:48:34.316614+0000","last_became_peered":"2024-04-24T08:48:34.316614+0000","last_unstale":"2024-04-24T08:48:35.325781+0000","last_undegraded":"2024-04-24T08:48:35.325781+0000","last_fullsized":"2024-04-24T08:48:35.325781+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:44:12.469926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:49:04.092938+0000","last_change":"2024-04-24T08:48:28.970261+0000","last_active":"2024-04-24T08:49:04.092938+0000","last_peered":"2024-04-24T08:49:04.092938+0000","last_clean":"2024-04-24T08:49:04.092938+0000","last_became_active":"2024-04-24T08:48:28.969813+0000","last_became_peered":"2024-04-24T08:48:28.969813+0000","last_unstale":"2024-04-24T08:49:04.092938+0000","last_undegraded":"2024-04-24T08:49:04.092938+0000","last_fullsized":"2024-04-24T08:49:04.092938+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:46:12.311756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.501189+0000","last_change":"2024-04-24T08:48:29.976712+0000","last_active":"2024-04-24T08:48:36.501189+0000","last_peered":"2024-04-24T08:48:36.501189+0000","last_clean":"2024-04-24T08:48:36.501189+0000","last_became_active":"2024-04-24T08:48:29.976166+0000","last_became_peered":"2024-04-24T08:48:29.976166+0000","last_unstale":"2024-04-24T08:48:36.501189+0000","last_undegraded":"2024-04-24T08:48:36.501189+0000","last_fullsized":"2024-04-24T08:48:36.501189+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:52:53.757829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.498338+0000","last_change":"2024-04-24T08:48:34.310173+0000","last_active":"2024-04-24T08:48:36.498338+0000","last_peered":"2024-04-24T08:48:36.498338+0000","last_clean":"2024-04-24T08:48:36.498338+0000","last_became_active":"2024-04-24T08:48:34.309772+0000","last_became_peered":"2024-04-24T08:48:34.309772+0000","last_unstale":"2024-04-24T08:48:36.498338+0000","last_undegraded":"2024-04-24T08:48:36.498338+0000","last_fullsized":"2024-04-24T08:48:36.498338+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:35:50.302769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'3","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:49:02.053206+0000","last_change":"2024-04-24T08:48:28.970127+0000","last_active":"2024-04-24T08:49:02.053206+0000","last_peered":"2024-04-24T08:49:02.053206+0000","last_clean":"2024-04-24T08:49:02.053206+0000","last_became_active":"2024-04-24T08:48:28.968803+0000","last_became_peered":"2024-04-24T08:48:28.968803+0000","last_unstale":"2024-04-24T08:49:02.053206+0000","last_undegraded":"2024-04-24T08:49:02.053206+0000","last_fullsized":"2024-04-24T08:49:02.053206+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:28:47.782276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.500645+0000","last_change":"2024-04-24T08:48:29.992964+0000","last_active":"2024-04-24T08:48:36.500645+0000","last_peered":"2024-04-24T08:48:36.500645+0000","last_clean":"2024-04-24T08:48:36.500645+0000","last_became_active":"2024-04-24T08:48:29.987501+0000","last_became_peered":"2024-04-24T08:48:29.987501+0000","last_unstale":"2024-04-24T08:48:36.500645+0000","last_undegraded":"2024-04-24T08:48:36.500645+0000","last_fullsized":"2024-04-24T08:48:36.500645+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:49:19.157052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:35.290875+0000","last_change":"2024-04-24T08:48:34.315393+0000","last_active":"2024-04-24T08:48:35.290875+0000","last_peered":"2024-04-24T08:48:35.290875+0000","last_clean":"2024-04-24T08:48:35.290875+0000","last_became_active":"2024-04-24T08:48:34.314943+0000","last_became_peered":"2024-04-24T08:48:34.314943+0000","last_unstale":"2024-04-24T08:48:35.290875+0000","last_undegraded":"2024-04-24T08:48:35.290875+0000","last_fullsized":"2024-04-24T08:48:35.290875+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:03:13.312223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:49:01.039089+0000","last_change":"2024-04-24T08:48:28.969813+0000","last_active":"2024-04-24T08:49:01.039089+0000","last_peered":"2024-04-24T08:49:01.039089+0000","last_clean":"2024-04-24T08:49:01.039089+0000","last_became_active":"2024-04-24T08:48:28.968485+0000","last_became_peered":"2024-04-24T08:48:28.968485+0000","last_unstale":"2024-04-24T08:49:01.039089+0000","last_undegraded":"2024-04-24T08:49:01.039089+0000","last_fullsized":"2024-04-24T08:49:01.039089+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:32:10.095723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.498659+0000","last_change":"2024-04-24T08:48:29.997731+0000","last_active":"2024-04-24T08:48:36.498659+0000","last_peered":"2024-04-24T08:48:36.498659+0000","last_clean":"2024-04-24T08:48:36.498659+0000","last_became_active":"2024-04-24T08:48:29.997373+0000","last_became_peered":"2024-04-24T08:48:29.997373+0000","last_unstale":"2024-04-24T08:48:36.498659+0000","last_undegraded":"2024-04-24T08:48:36.498659+0000","last_fullsized":"2024-04-24T08:48:36.498659+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:57:19.531173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.500827+0000","last_change":"2024-04-24T08:48:34.326170+0000","last_active":"2024-04-24T08:48:36.500827+0000","last_peered":"2024-04-24T08:48:36.500827+0000","last_clean":"2024-04-24T08:48:36.500827+0000","last_became_active":"2024-04-24T08:48:34.325678+0000","last_became_peered":"2024-04-24T08:48:34.325678+0000","last_unstale":"2024-04-24T08:48:36.500827+0000","last_undegraded":"2024-04-24T08:48:36.500827+0000","last_fullsized":"2024-04-24T08:48:36.500827+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:12:47.003864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:49:01.047700+0000","last_change":"2024-04-24T08:48:28.975522+0000","last_active":"2024-04-24T08:49:01.047700+0000","last_peered":"2024-04-24T08:49:01.047700+0000","last_clean":"2024-04-24T08:49:01.047700+0000","last_became_active":"2024-04-24T08:48:28.975093+0000","last_became_peered":"2024-04-24T08:48:28.975093+0000","last_unstale":"2024-04-24T08:49:01.047700+0000","last_undegraded":"2024-04-24T08:49:01.047700+0000","last_fullsized":"2024-04-24T08:49:01.047700+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:37:31.809694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.497877+0000","last_change":"2024-04-24T08:48:29.982502+0000","last_active":"2024-04-24T08:48:36.497877+0000","last_peered":"2024-04-24T08:48:36.497877+0000","last_clean":"2024-04-24T08:48:36.497877+0000","last_became_active":"2024-04-24T08:48:29.982141+0000","last_became_peered":"2024-04-24T08:48:29.982141+0000","last_unstale":"2024-04-24T08:48:36.497877+0000","last_undegraded":"2024-04-24T08:48:36.497877+0000","last_fullsized":"2024-04-24T08:48:36.497877+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:46:15.851359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:35.291126+0000","last_change":"2024-04-24T08:48:34.322717+0000","last_active":"2024-04-24T08:48:35.291126+0000","last_peered":"2024-04-24T08:48:35.291126+0000","last_clean":"2024-04-24T08:48:35.291126+0000","last_became_active":"2024-04-24T08:48:34.321837+0000","last_became_peered":"2024-04-24T08:48:34.321837+0000","last_unstale":"2024-04-24T08:48:35.291126+0000","last_undegraded":"2024-04-24T08:48:35.291126+0000","last_fullsized":"2024-04-24T08:48:35.291126+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:40:11.557436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'5","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:49:03.070283+0000","last_change":"2024-04-24T08:48:28.973951+0000","last_active":"2024-04-24T08:49:03.070283+0000","last_peered":"2024-04-24T08:49:03.070283+0000","last_clean":"2024-04-24T08:49:03.070283+0000","last_became_active":"2024-04-24T08:48:28.973131+0000","last_became_peered":"2024-04-24T08:48:28.973131+0000","last_unstale":"2024-04-24T08:49:03.070283+0000","last_undegraded":"2024-04-24T08:49:03.070283+0000","last_fullsized":"2024-04-24T08:49:03.070283+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:45:45.197337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:35.325745+0000","last_change":"2024-04-24T08:48:29.992689+0000","last_active":"2024-04-24T08:48:35.325745+0000","last_peered":"2024-04-24T08:48:35.325745+0000","last_clean":"2024-04-24T08:48:35.325745+0000","last_became_active":"2024-04-24T08:48:29.988874+0000","last_became_peered":"2024-04-24T08:48:29.988874+0000","last_unstale":"2024-04-24T08:48:35.325745+0000","last_undegraded":"2024-04-24T08:48:35.325745+0000","last_fullsized":"2024-04-24T08:48:35.325745+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:42:41.578424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.500481+0000","last_change":"2024-04-24T08:48:34.318640+0000","last_active":"2024-04-24T08:48:36.500481+0000","last_peered":"2024-04-24T08:48:36.500481+0000","last_clean":"2024-04-24T08:48:36.500481+0000","last_became_active":"2024-04-24T08:48:34.318039+0000","last_became_peered":"2024-04-24T08:48:34.318039+0000","last_unstale":"2024-04-24T08:48:36.500481+0000","last_undegraded":"2024-04-24T08:48:36.500481+0000","last_fullsized":"2024-04-24T08:48:36.500481+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:17:27.110930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:35.291081+0000","last_change":"2024-04-24T08:48:28.980828+0000","last_active":"2024-04-24T08:48:35.291081+0000","last_peered":"2024-04-24T08:48:35.291081+0000","last_clean":"2024-04-24T08:48:35.291081+0000","last_became_active":"2024-04-24T08:48:28.979814+0000","last_became_peered":"2024-04-24T08:48:28.979814+0000","last_unstale":"2024-04-24T08:48:35.291081+0000","last_undegraded":"2024-04-24T08:48:35.291081+0000","last_fullsized":"2024-04-24T08:48:35.291081+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:02:28.145847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.500112+0000","last_change":"2024-04-24T08:48:29.995003+0000","last_active":"2024-04-24T08:48:36.500112+0000","last_peered":"2024-04-24T08:48:36.500112+0000","last_clean":"2024-04-24T08:48:36.500112+0000","last_became_active":"2024-04-24T08:48:29.994516+0000","last_became_peered":"2024-04-24T08:48:29.994516+0000","last_unstale":"2024-04-24T08:48:36.500112+0000","last_undegraded":"2024-04-24T08:48:36.500112+0000","last_fullsized":"2024-04-24T08:48:36.500112+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:15:46.094534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.501593+0000","last_change":"2024-04-24T08:48:34.311721+0000","last_active":"2024-04-24T08:48:36.501593+0000","last_peered":"2024-04-24T08:48:36.501593+0000","last_clean":"2024-04-24T08:48:36.501593+0000","last_became_active":"2024-04-24T08:48:34.311241+0000","last_became_peered":"2024-04-24T08:48:34.311241+0000","last_unstale":"2024-04-24T08:48:36.501593+0000","last_undegraded":"2024-04-24T08:48:36.501593+0000","last_fullsized":"2024-04-24T08:48:36.501593+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:16:17.679237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:49:01.047134+0000","last_change":"2024-04-24T08:48:28.973336+0000","last_active":"2024-04-24T08:49:01.047134+0000","last_peered":"2024-04-24T08:49:01.047134+0000","last_clean":"2024-04-24T08:49:01.047134+0000","last_became_active":"2024-04-24T08:48:28.972666+0000","last_became_peered":"2024-04-24T08:48:28.972666+0000","last_unstale":"2024-04-24T08:49:01.047134+0000","last_undegraded":"2024-04-24T08:49:01.047134+0000","last_fullsized":"2024-04-24T08:49:01.047134+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:53:28.358291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:35.291248+0000","last_change":"2024-04-24T08:48:29.983498+0000","last_active":"2024-04-24T08:48:35.291248+0000","last_peered":"2024-04-24T08:48:35.291248+0000","last_clean":"2024-04-24T08:48:35.291248+0000","last_became_active":"2024-04-24T08:48:29.983167+0000","last_became_peered":"2024-04-24T08:48:29.983167+0000","last_unstale":"2024-04-24T08:48:35.291248+0000","last_undegraded":"2024-04-24T08:48:35.291248+0000","last_fullsized":"2024-04-24T08:48:35.291248+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:09:56.648047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.501528+0000","last_change":"2024-04-24T08:48:34.317151+0000","last_active":"2024-04-24T08:48:36.501528+0000","last_peered":"2024-04-24T08:48:36.501528+0000","last_clean":"2024-04-24T08:48:36.501528+0000","last_became_active":"2024-04-24T08:48:34.316362+0000","last_became_peered":"2024-04-24T08:48:34.316362+0000","last_unstale":"2024-04-24T08:48:36.501528+0000","last_undegraded":"2024-04-24T08:48:36.501528+0000","last_fullsized":"2024-04-24T08:48:36.501528+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:35:38.196763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:35.291937+0000","last_change":"2024-04-24T08:48:28.974528+0000","last_active":"2024-04-24T08:48:35.291937+0000","last_peered":"2024-04-24T08:48:35.291937+0000","last_clean":"2024-04-24T08:48:35.291937+0000","last_became_active":"2024-04-24T08:48:28.974203+0000","last_became_peered":"2024-04-24T08:48:28.974203+0000","last_unstale":"2024-04-24T08:48:35.291937+0000","last_undegraded":"2024-04-24T08:48:35.291937+0000","last_fullsized":"2024-04-24T08:48:35.291937+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:26:15.701204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:35.292350+0000","last_change":"2024-04-24T08:48:29.984356+0000","last_active":"2024-04-24T08:48:35.292350+0000","last_peered":"2024-04-24T08:48:35.292350+0000","last_clean":"2024-04-24T08:48:35.292350+0000","last_became_active":"2024-04-24T08:48:29.983949+0000","last_became_peered":"2024-04-24T08:48:29.983949+0000","last_unstale":"2024-04-24T08:48:35.292350+0000","last_undegraded":"2024-04-24T08:48:35.292350+0000","last_fullsized":"2024-04-24T08:48:35.292350+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:51:59.409835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.498254+0000","last_change":"2024-04-24T08:48:34.312418+0000","last_active":"2024-04-24T08:48:36.498254+0000","last_peered":"2024-04-24T08:48:36.498254+0000","last_clean":"2024-04-24T08:48:36.498254+0000","last_became_active":"2024-04-24T08:48:34.312035+0000","last_became_peered":"2024-04-24T08:48:34.312035+0000","last_unstale":"2024-04-24T08:48:36.498254+0000","last_undegraded":"2024-04-24T08:48:36.498254+0000","last_fullsized":"2024-04-24T08:48:36.498254+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:27:33.461732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"89'3","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:49:04.099932+0000","last_change":"2024-04-24T08:48:28.976070+0000","last_active":"2024-04-24T08:49:04.099932+0000","last_peered":"2024-04-24T08:49:04.099932+0000","last_clean":"2024-04-24T08:49:04.099932+0000","last_became_active":"2024-04-24T08:48:28.975703+0000","last_became_peered":"2024-04-24T08:48:28.975703+0000","last_unstale":"2024-04-24T08:49:04.099932+0000","last_undegraded":"2024-04-24T08:49:04.099932+0000","last_fullsized":"2024-04-24T08:49:04.099932+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:46:13.227529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:35.291223+0000","last_change":"2024-04-24T08:48:29.986423+0000","last_active":"2024-04-24T08:48:35.291223+0000","last_peered":"2024-04-24T08:48:35.291223+0000","last_clean":"2024-04-24T08:48:35.291223+0000","last_became_active":"2024-04-24T08:48:29.985452+0000","last_became_peered":"2024-04-24T08:48:29.985452+0000","last_unstale":"2024-04-24T08:48:35.291223+0000","last_undegraded":"2024-04-24T08:48:35.291223+0000","last_fullsized":"2024-04-24T08:48:35.291223+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:05:42.686370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.062764+0000","last_change":"2024-04-24T08:48:34.320191+0000","last_active":"2024-04-24T08:48:36.062764+0000","last_peered":"2024-04-24T08:48:36.062764+0000","last_clean":"2024-04-24T08:48:36.062764+0000","last_became_active":"2024-04-24T08:48:34.315415+0000","last_became_peered":"2024-04-24T08:48:34.315415+0000","last_unstale":"2024-04-24T08:48:36.062764+0000","last_undegraded":"2024-04-24T08:48:36.062764+0000","last_fullsized":"2024-04-24T08:48:36.062764+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:55:11.372789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"89'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.501691+0000","last_change":"2024-04-24T08:48:28.968550+0000","last_active":"2024-04-24T08:48:36.501691+0000","last_peered":"2024-04-24T08:48:36.501691+0000","last_clean":"2024-04-24T08:48:36.501691+0000","last_became_active":"2024-04-24T08:48:28.968120+0000","last_became_peered":"2024-04-24T08:48:28.968120+0000","last_unstale":"2024-04-24T08:48:36.501691+0000","last_undegraded":"2024-04-24T08:48:36.501691+0000","last_fullsized":"2024-04-24T08:48:36.501691+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:46:00.870285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:35.292109+0000","last_change":"2024-04-24T08:48:29.984755+0000","last_active":"2024-04-24T08:48:35.292109+0000","last_peered":"2024-04-24T08:48:35.292109+0000","last_clean":"2024-04-24T08:48:35.292109+0000","last_became_active":"2024-04-24T08:48:29.982093+0000","last_became_peered":"2024-04-24T08:48:29.982093+0000","last_unstale":"2024-04-24T08:48:35.292109+0000","last_undegraded":"2024-04-24T08:48:35.292109+0000","last_fullsized":"2024-04-24T08:48:35.292109+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:25:22.620152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:35.325401+0000","last_change":"2024-04-24T08:48:29.983101+0000","last_active":"2024-04-24T08:48:35.325401+0000","last_peered":"2024-04-24T08:48:35.325401+0000","last_clean":"2024-04-24T08:48:35.325401+0000","last_became_active":"2024-04-24T08:48:29.981871+0000","last_became_peered":"2024-04-24T08:48:29.981871+0000","last_unstale":"2024-04-24T08:48:35.325401+0000","last_undegraded":"2024-04-24T08:48:35.325401+0000","last_fullsized":"2024-04-24T08:48:35.325401+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:02:44.582111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:35.292306+0000","last_change":"2024-04-24T08:48:29.978469+0000","last_active":"2024-04-24T08:48:35.292306+0000","last_peered":"2024-04-24T08:48:35.292306+0000","last_clean":"2024-04-24T08:48:35.292306+0000","last_became_active":"2024-04-24T08:48:29.977382+0000","last_became_peered":"2024-04-24T08:48:29.977382+0000","last_unstale":"2024-04-24T08:48:35.292306+0000","last_undegraded":"2024-04-24T08:48:35.292306+0000","last_fullsized":"2024-04-24T08:48:35.292306+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:00:33.863236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:35.290715+0000","last_change":"2024-04-24T08:48:29.988241+0000","last_active":"2024-04-24T08:48:35.290715+0000","last_peered":"2024-04-24T08:48:35.290715+0000","last_clean":"2024-04-24T08:48:35.290715+0000","last_became_active":"2024-04-24T08:48:29.987930+0000","last_became_peered":"2024-04-24T08:48:29.987930+0000","last_unstale":"2024-04-24T08:48:35.290715+0000","last_undegraded":"2024-04-24T08:48:35.290715+0000","last_fullsized":"2024-04-24T08:48:35.290715+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:42:24.687624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.497713+0000","last_change":"2024-04-24T08:48:29.992441+0000","last_active":"2024-04-24T08:48:36.497713+0000","last_peered":"2024-04-24T08:48:36.497713+0000","last_clean":"2024-04-24T08:48:36.497713+0000","last_became_active":"2024-04-24T08:48:29.988788+0000","last_became_peered":"2024-04-24T08:48:29.988788+0000","last_unstale":"2024-04-24T08:48:36.497713+0000","last_undegraded":"2024-04-24T08:48:36.497713+0000","last_fullsized":"2024-04-24T08:48:36.497713+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:08:31.549509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.501904+0000","last_change":"2024-04-24T08:48:29.982116+0000","last_active":"2024-04-24T08:48:36.501904+0000","last_peered":"2024-04-24T08:48:36.501904+0000","last_clean":"2024-04-24T08:48:36.501904+0000","last_became_active":"2024-04-24T08:48:29.981636+0000","last_became_peered":"2024-04-24T08:48:29.981636+0000","last_unstale":"2024-04-24T08:48:36.501904+0000","last_undegraded":"2024-04-24T08:48:36.501904+0000","last_fullsized":"2024-04-24T08:48:36.501904+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:11:01.943425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:35.326069+0000","last_change":"2024-04-24T08:48:29.982244+0000","last_active":"2024-04-24T08:48:35.326069+0000","last_peered":"2024-04-24T08:48:35.326069+0000","last_clean":"2024-04-24T08:48:35.326069+0000","last_became_active":"2024-04-24T08:48:29.981622+0000","last_became_peered":"2024-04-24T08:48:29.981622+0000","last_unstale":"2024-04-24T08:48:35.326069+0000","last_undegraded":"2024-04-24T08:48:35.326069+0000","last_fullsized":"2024-04-24T08:48:35.326069+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:02:57.723949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:35.292425+0000","last_change":"2024-04-24T08:48:29.985867+0000","last_active":"2024-04-24T08:48:35.292425+0000","last_peered":"2024-04-24T08:48:35.292425+0000","last_clean":"2024-04-24T08:48:35.292425+0000","last_became_active":"2024-04-24T08:48:29.985430+0000","last_became_peered":"2024-04-24T08:48:29.985430+0000","last_unstale":"2024-04-24T08:48:35.292425+0000","last_undegraded":"2024-04-24T08:48:35.292425+0000","last_fullsized":"2024-04-24T08:48:35.292425+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:09:40.467623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.500900+0000","last_change":"2024-04-24T08:48:29.996474+0000","last_active":"2024-04-24T08:48:36.500900+0000","last_peered":"2024-04-24T08:48:36.500900+0000","last_clean":"2024-04-24T08:48:36.500900+0000","last_became_active":"2024-04-24T08:48:29.995958+0000","last_became_peered":"2024-04-24T08:48:29.995958+0000","last_unstale":"2024-04-24T08:48:36.500900+0000","last_undegraded":"2024-04-24T08:48:36.500900+0000","last_fullsized":"2024-04-24T08:48:36.500900+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:50:19.865125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.063623+0000","last_change":"2024-04-24T08:48:29.978285+0000","last_active":"2024-04-24T08:48:36.063623+0000","last_peered":"2024-04-24T08:48:36.063623+0000","last_clean":"2024-04-24T08:48:36.063623+0000","last_became_active":"2024-04-24T08:48:29.976445+0000","last_became_peered":"2024-04-24T08:48:29.976445+0000","last_unstale":"2024-04-24T08:48:36.063623+0000","last_undegraded":"2024-04-24T08:48:36.063623+0000","last_fullsized":"2024-04-24T08:48:36.063623+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:30:29.598733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.499261+0000","last_change":"2024-04-24T08:48:29.994442+0000","last_active":"2024-04-24T08:48:36.499261+0000","last_peered":"2024-04-24T08:48:36.499261+0000","last_clean":"2024-04-24T08:48:36.499261+0000","last_became_active":"2024-04-24T08:48:29.993896+0000","last_became_peered":"2024-04-24T08:48:29.993896+0000","last_unstale":"2024-04-24T08:48:36.499261+0000","last_undegraded":"2024-04-24T08:48:36.499261+0000","last_fullsized":"2024-04-24T08:48:36.499261+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:37:14.023511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.502312+0000","last_change":"2024-04-24T08:48:29.997254+0000","last_active":"2024-04-24T08:48:36.502312+0000","last_peered":"2024-04-24T08:48:36.502312+0000","last_clean":"2024-04-24T08:48:36.502312+0000","last_became_active":"2024-04-24T08:48:29.996871+0000","last_became_peered":"2024-04-24T08:48:29.996871+0000","last_unstale":"2024-04-24T08:48:36.502312+0000","last_undegraded":"2024-04-24T08:48:36.502312+0000","last_fullsized":"2024-04-24T08:48:36.502312+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:53:37.288858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:36.499571+0000","last_change":"2024-04-24T08:48:29.986021+0000","last_active":"2024-04-24T08:48:36.499571+0000","last_peered":"2024-04-24T08:48:36.499571+0000","last_clean":"2024-04-24T08:48:36.499571+0000","last_became_active":"2024-04-24T08:48:29.985182+0000","last_became_peered":"2024-04-24T08:48:29.985182+0000","last_unstale":"2024-04-24T08:48:36.499571+0000","last_undegraded":"2024-04-24T08:48:36.499571+0000","last_fullsized":"2024-04-24T08:48:36.499571+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:10:57.143065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-24T08:48:35.325368+0000","last_change":"2024-04-24T08:48:29.979886+0000","last_active":"2024-04-24T08:48:35.325368+0000","last_peered":"2024-04-24T08:48:35.325368+0000","last_clean":"2024-04-24T08:48:35.325368+0000","last_became_active":"2024-04-24T08:48:29.978779+0000","last_became_peered":"2024-04-24T08:48:29.978779+0000","last_unstale":"2024-04-24T08:48:35.325368+0000","last_undegraded":"2024-04-24T08:48:35.325368+0000","last_fullsized":"2024-04-24T08:48:35.325368+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:51:51.303990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":9283,"num_objects":75,"num_object_clones":0,"num_object_copies":225,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":0,"num_read":20,"num_read_kb":16,"num_write":162,"num_write_kb":55,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":50,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":352256,"data_stored":48531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":145,"ondisk_log_size":145,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383703,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":995328,"data_stored":478407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645342,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32760,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712392,"statfs":{"total":95995035648,"available":95961489408,"internally_reserved":0,"allocated":1622016,"data_stored":1073003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808869,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":1003520,"data_stored":478033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070508,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":1003520,"data_stored":477953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332148,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32752,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712400,"statfs":{"total":95995035648,"available":95961497600,"internally_reserved":0,"allocated":1613824,"data_stored":1075212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495675,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":991232,"data_stored":477327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574943,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36320,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708832,"statfs":{"total":95995035648,"available":95957843968,"internally_reserved":0,"allocated":1007616,"data_stored":488239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738470,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1024000,"data_stored":479393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411414,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1007616,"data_stored":481713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182481,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32160,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712992,"statfs":{"total":95995035648,"available":95962103808,"internally_reserved":0,"allocated":1007616,"data_stored":477623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920841,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":995328,"data_stored":477859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757315,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32764,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712388,"statfs":{"total":95995035648,"available":95961485312,"internally_reserved":0,"allocated":1626112,"data_stored":1074890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":24893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":4498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":5544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T08:49:07.544 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-24T08:49:07.544 INFO:tasks.cephadm:Setting up client nodes... 2024-04-24T08:49:07.545 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd3e039e-0215-11ef-bc93-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-24T08:49:07.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:49:07 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/4265399431' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T08:49:07.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:49:07 smithi138 ceph-mon[31441]: pgmap v392: 129 pgs: 129 active+clean; 586 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 3.1 KiB/s wr, 11 op/s 2024-04-24T08:49:07.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:49:07 smithi138 ceph-mon[31441]: from='client.14979 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T08:49:07.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:49:07 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/4265399431' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T08:49:07.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:49:07 smithi189 ceph-mon[31791]: pgmap v392: 129 pgs: 129 active+clean; 586 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 3.1 KiB/s wr, 11 op/s 2024-04-24T08:49:07.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:49:07 smithi189 ceph-mon[31791]: from='client.14979 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T08:49:08.019 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T08:49:08.443 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:49:08 smithi083 ceph-mon[27907]: from='client.14985 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T08:49:08.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:49:08 smithi138 ceph-mon[31441]: from='client.14985 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T08:49:08.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:49:08 smithi189 ceph-mon[31791]: from='client.14985 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T08:49:09.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:49:09 smithi083 ceph-mon[27907]: pgmap v393: 129 pgs: 129 active+clean; 628 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 7.8 KiB/s wr, 19 op/s 2024-04-24T08:49:09.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:49:09 smithi138 ceph-mon[31441]: pgmap v393: 129 pgs: 129 active+clean; 628 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 7.8 KiB/s wr, 19 op/s 2024-04-24T08:49:09.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:49:09 smithi189 ceph-mon[31791]: pgmap v393: 129 pgs: 129 active+clean; 628 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 7.8 KiB/s wr, 19 op/s 2024-04-24T08:49:10.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:10.529+0000 7f8bd2200700 1 -- 172.21.15.83:0/3134023675 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bcc103600 msgr2=0x7f8bcc1059f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:10.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:10.529+0000 7f8bd2200700 1 --2- 172.21.15.83:0/3134023675 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bcc103600 0x7f8bcc1059f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc009990 tx=0x7f8bbc009ca0 comp rx=0 tx=0).stop 2024-04-24T08:49:10.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:10.529+0000 7f8bd2200700 1 -- 172.21.15.83:0/3134023675 shutdown_connections 2024-04-24T08:49:10.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:10.529+0000 7f8bd2200700 1 --2- 172.21.15.83:0/3134023675 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bcc103600 0x7f8bcc1059f0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:10.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:10.529+0000 7f8bd2200700 1 --2- 172.21.15.83:0/3134023675 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8bcc108860 0x7f8bcc10acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:10.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:10.529+0000 7f8bd2200700 1 --2- 172.21.15.83:0/3134023675 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8bcc105f30 0x7f8bcc108320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:10.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:10.529+0000 7f8bd2200700 1 -- 172.21.15.83:0/3134023675 >> 172.21.15.83:0/3134023675 conn(0x7f8bcc0fd010 msgr2=0x7f8bcc0ff450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:49:10.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:10.529+0000 7f8bd2200700 1 -- 172.21.15.83:0/3134023675 shutdown_connections 2024-04-24T08:49:10.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:10.529+0000 7f8bd2200700 1 -- 172.21.15.83:0/3134023675 wait complete. 2024-04-24T08:49:10.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:10.530+0000 7f8bd2200700 1 Processor -- start 2024-04-24T08:49:10.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:10.530+0000 7f8bd2200700 1 -- start start 2024-04-24T08:49:10.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:10.531+0000 7f8bd2200700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bcc103600 0x7f8bcc198ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:10.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:10.531+0000 7f8bd2200700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8bcc105f30 0x7f8bcc1990e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:10.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:10.531+0000 7f8bd2200700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8bcc108860 0x7f8bcc1a0160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:10.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:10.531+0000 7f8bd2200700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8bcc10e620 con 0x7f8bcc103600 2024-04-24T08:49:10.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:10.531+0000 7f8bd2200700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8bcc10e4a0 con 0x7f8bcc105f30 2024-04-24T08:49:10.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:10.531+0000 7f8bd2200700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8bcc10e7a0 con 0x7f8bcc108860 2024-04-24T08:49:10.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:10.531+0000 7f8bcb400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bcc103600 0x7f8bcc198ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:10.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:10.531+0000 7f8bcb400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bcc103600 0x7f8bcc198ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54220/0 (socket says 172.21.15.83:54220) 2024-04-24T08:49:10.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:10.531+0000 7f8bcb400700 1 -- 172.21.15.83:0/3635294282 learned_addr learned my addr 172.21.15.83:0/3635294282 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:49:10.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:10.531+0000 7f8bcaa00700 1 --2- 172.21.15.83:0/3635294282 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8bcc105f30 0x7f8bcc1990e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:10.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:10.532+0000 7f8bcbe00700 1 --2- 172.21.15.83:0/3635294282 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8bcc108860 0x7f8bcc1a0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:10.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:10.532+0000 7f8bcb400700 1 -- 172.21.15.83:0/3635294282 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8bcc108860 msgr2=0x7f8bcc1a0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:10.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:10.532+0000 7f8bcb400700 1 --2- 172.21.15.83:0/3635294282 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8bcc108860 0x7f8bcc1a0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:10.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:10.532+0000 7f8bcb400700 1 -- 172.21.15.83:0/3635294282 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8bcc105f30 msgr2=0x7f8bcc1990e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:10.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:10.532+0000 7f8bcb400700 1 --2- 172.21.15.83:0/3635294282 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8bcc105f30 0x7f8bcc1990e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:10.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:10.532+0000 7f8bcb400700 1 -- 172.21.15.83:0/3635294282 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bcc1a0800 con 0x7f8bcc103600 2024-04-24T08:49:10.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:10.532+0000 7f8bcbe00700 1 --2- 172.21.15.83:0/3635294282 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8bcc108860 0x7f8bcc1a0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:49:10.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:10.532+0000 7f8bcaa00700 1 --2- 172.21.15.83:0/3635294282 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8bcc105f30 0x7f8bcc1990e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:49:10.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:10.532+0000 7f8bcb400700 1 --2- 172.21.15.83:0/3635294282 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bcc103600 0x7f8bcc198ba0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc0098e0 tx=0x7f8bbc005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:49:10.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:10.532+0000 7f8bc3e00700 1 -- 172.21.15.83:0/3635294282 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc027070 con 0x7f8bcc103600 2024-04-24T08:49:10.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:10.532+0000 7f8bc3e00700 1 -- 172.21.15.83:0/3635294282 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8bbc022470 con 0x7f8bcc103600 2024-04-24T08:49:10.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:10.532+0000 7f8bc3e00700 1 -- 172.21.15.83:0/3635294282 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc00f6c0 con 0x7f8bcc103600 2024-04-24T08:49:10.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:10.532+0000 7f8bd2200700 1 -- 172.21.15.83:0/3635294282 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bcc1a0a90 con 0x7f8bcc103600 2024-04-24T08:49:10.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:10.533+0000 7f8bd2200700 1 -- 172.21.15.83:0/3635294282 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bcc1a0ee0 con 0x7f8bcc103600 2024-04-24T08:49:10.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:10.534+0000 7f8bd2200700 1 -- 172.21.15.83:0/3635294282 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bcc051030 con 0x7f8bcc103600 2024-04-24T08:49:10.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:10.537+0000 7f8bc3e00700 1 -- 172.21.15.83:0/3635294282 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f8bbc0052a0 con 0x7f8bcc103600 2024-04-24T08:49:10.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:10.539+0000 7f8bc3e00700 1 --2- 172.21.15.83:0/3635294282 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f8bb00a3d80 0x7f8bb00a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:10.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:10.539+0000 7f8bc3e00700 1 -- 172.21.15.83:0/3635294282 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f8bbc0d1f80 con 0x7f8bcc103600 2024-04-24T08:49:10.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:10.540+0000 7f8bcaa00700 1 --2- 172.21.15.83:0/3635294282 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f8bb00a3d80 0x7f8bb00a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:10.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:10.540+0000 7f8bc3e00700 1 -- 172.21.15.83:0/3635294282 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8bbc030020 con 0x7f8bcc103600 2024-04-24T08:49:10.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:10.540+0000 7f8bcaa00700 1 --2- 172.21.15.83:0/3635294282 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f8bb00a3d80 0x7f8bb00a6240 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8bb8005950 tx=0x7f8bb8005f50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:49:10.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:10.856+0000 7f8bd2200700 1 -- 172.21.15.83:0/3635294282 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f8bcc02e0b0 con 0x7f8bcc103600 2024-04-24T08:49:10.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:10.860+0000 7f8bc3e00700 1 -- 172.21.15.83:0/3635294282 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v33) ==== 170+0+59 (secure 0 0 0) 0x7f8bcc02e0b0 con 0x7f8bcc103600 2024-04-24T08:49:10.861 INFO:teuthology.orchestra.run.smithi083.stdout:[client.0] 2024-04-24T08:49:10.861 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQCGxyhm4g4xMxAAz33lXkp/huI8E+8PgOpvLg== 2024-04-24T08:49:10.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:10.863+0000 7f8bd2200700 1 -- 172.21.15.83:0/3635294282 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f8bb00a3d80 msgr2=0x7f8bb00a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:10.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:10.863+0000 7f8bd2200700 1 --2- 172.21.15.83:0/3635294282 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f8bb00a3d80 0x7f8bb00a6240 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8bb8005950 tx=0x7f8bb8005f50 comp rx=0 tx=0).stop 2024-04-24T08:49:10.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:10.863+0000 7f8bd2200700 1 -- 172.21.15.83:0/3635294282 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bcc103600 msgr2=0x7f8bcc198ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:10.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:10.863+0000 7f8bd2200700 1 --2- 172.21.15.83:0/3635294282 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bcc103600 0x7f8bcc198ba0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc0098e0 tx=0x7f8bbc005ef0 comp rx=0 tx=0).stop 2024-04-24T08:49:10.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:10.863+0000 7f8bd2200700 1 -- 172.21.15.83:0/3635294282 shutdown_connections 2024-04-24T08:49:10.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:10.863+0000 7f8bd2200700 1 --2- 172.21.15.83:0/3635294282 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f8bb00a3d80 0x7f8bb00a6240 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:10.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:10.863+0000 7f8bd2200700 1 --2- 172.21.15.83:0/3635294282 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bcc103600 0x7f8bcc198ba0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:10.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:10.863+0000 7f8bd2200700 1 --2- 172.21.15.83:0/3635294282 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8bcc108860 0x7f8bcc1a0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:10.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:10.863+0000 7f8bd2200700 1 --2- 172.21.15.83:0/3635294282 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8bcc105f30 0x7f8bcc1990e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:10.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:10.863+0000 7f8bd2200700 1 -- 172.21.15.83:0/3635294282 >> 172.21.15.83:0/3635294282 conn(0x7f8bcc0fd010 msgr2=0x7f8bcc103f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:49:10.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:10.864+0000 7f8bd2200700 1 -- 172.21.15.83:0/3635294282 shutdown_connections 2024-04-24T08:49:10.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:10.864+0000 7f8bd2200700 1 -- 172.21.15.83:0/3635294282 wait complete. 2024-04-24T08:49:11.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:49:10 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/3635294282' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-24T08:49:11.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:49:10 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/3635294282' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-24T08:49:11.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:49:10 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/3635294282' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-24T08:49:11.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:49:10 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/3635294282' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-24T08:49:11.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:49:10 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/3635294282' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-24T08:49:11.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:49:10 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/3635294282' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-24T08:49:11.977 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-24T08:49:12.089 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-04-24T08:49:12.090 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-24T08:49:12.091 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:49:11 smithi083 ceph-mon[27907]: pgmap v394: 129 pgs: 129 active+clean; 628 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 8.4 KiB/s wr, 21 op/s 2024-04-24T08:49:12.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:49:11 smithi138 ceph-mon[31441]: pgmap v394: 129 pgs: 129 active+clean; 628 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 8.4 KiB/s wr, 21 op/s 2024-04-24T08:49:12.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:49:11 smithi189 ceph-mon[31791]: pgmap v394: 129 pgs: 129 active+clean; 628 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 8.4 KiB/s wr, 21 op/s 2024-04-24T08:49:12.219 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-24T08:49:12.220 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-04-24T08:49:12.220 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid bd3e039e-0215-11ef-bc93-c7b262605968 -- ceph mgr dump --format=json 2024-04-24T08:49:12.674 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T08:49:14.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:49:13 smithi083 ceph-mon[27907]: pgmap v395: 129 pgs: 129 active+clean; 628 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 8.4 KiB/s wr, 22 op/s 2024-04-24T08:49:14.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:49:13 smithi138 ceph-mon[31441]: pgmap v395: 129 pgs: 129 active+clean; 628 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 8.4 KiB/s wr, 22 op/s 2024-04-24T08:49:14.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:49:13 smithi189 ceph-mon[31791]: pgmap v395: 129 pgs: 129 active+clean; 628 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 8.4 KiB/s wr, 22 op/s 2024-04-24T08:49:15.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:15.156+0000 7f6112600700 1 -- 172.21.15.83:0/2900463423 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f610c103540 msgr2=0x7f610c10fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:15.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:15.156+0000 7f6112600700 1 --2- 172.21.15.83:0/2900463423 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f610c103540 0x7f610c10fa20 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f610400b0a0 tx=0x7f610400b3b0 comp rx=0 tx=0).stop 2024-04-24T08:49:15.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:15.156+0000 7f6112600700 1 -- 172.21.15.83:0/2900463423 shutdown_connections 2024-04-24T08:49:15.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:15.156+0000 7f6112600700 1 --2- 172.21.15.83:0/2900463423 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f610c103540 0x7f610c10fa20 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:15.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:15.156+0000 7f6112600700 1 --2- 172.21.15.83:0/2900463423 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f610c107ab0 0x7f610c107e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:15.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:15.156+0000 7f6112600700 1 --2- 172.21.15.83:0/2900463423 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f610c102ba0 0x7f610c103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:15.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:15.156+0000 7f6112600700 1 -- 172.21.15.83:0/2900463423 >> 172.21.15.83:0/2900463423 conn(0x7f610c079c50 msgr2=0x7f610c07a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:49:15.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:15.156+0000 7f6112600700 1 -- 172.21.15.83:0/2900463423 shutdown_connections 2024-04-24T08:49:15.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:15.156+0000 7f6112600700 1 -- 172.21.15.83:0/2900463423 wait complete. 2024-04-24T08:49:15.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:15.157+0000 7f6112600700 1 Processor -- start 2024-04-24T08:49:15.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:15.157+0000 7f6112600700 1 -- start start 2024-04-24T08:49:15.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:15.158+0000 7f6112600700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f610c102ba0 0x7f610c19b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:15.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:15.158+0000 7f6112600700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f610c103540 0x7f610c19b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:15.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:15.158+0000 7f6112600700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f610c107ab0 0x7f610c19fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:15.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:15.158+0000 7f6112600700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f610c112cf0 con 0x7f610c102ba0 2024-04-24T08:49:15.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:15.158+0000 7f6112600700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f610c112b70 con 0x7f610c103540 2024-04-24T08:49:15.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:15.158+0000 7f6112600700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f610c112e70 con 0x7f610c107ab0 2024-04-24T08:49:15.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:15.158+0000 7f610b400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f610c102ba0 0x7f610c19b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:15.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:15.158+0000 7f610b400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f610c102ba0 0x7f610c19b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54230/0 (socket says 172.21.15.83:54230) 2024-04-24T08:49:15.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:15.158+0000 7f610b400700 1 -- 172.21.15.83:0/2352021682 learned_addr learned my addr 172.21.15.83:0/2352021682 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:49:15.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:15.158+0000 7f6102a00700 1 --2- 172.21.15.83:0/2352021682 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f610c103540 0x7f610c19b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:15.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:15.158+0000 7f610b400700 1 -- 172.21.15.83:0/2352021682 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f610c107ab0 msgr2=0x7f610c19fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:15.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:15.158+0000 7f610be00700 1 --2- 172.21.15.83:0/2352021682 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f610c107ab0 0x7f610c19fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:15.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:15.158+0000 7f610b400700 1 --2- 172.21.15.83:0/2352021682 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f610c107ab0 0x7f610c19fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:15.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:15.158+0000 7f610b400700 1 -- 172.21.15.83:0/2352021682 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f610c103540 msgr2=0x7f610c19b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:15.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:15.158+0000 7f610b400700 1 --2- 172.21.15.83:0/2352021682 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f610c103540 0x7f610c19b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:15.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:15.158+0000 7f610b400700 1 -- 172.21.15.83:0/2352021682 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f610c1a03a0 con 0x7f610c102ba0 2024-04-24T08:49:15.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:15.159+0000 7f610b400700 1 --2- 172.21.15.83:0/2352021682 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f610c102ba0 0x7f610c19b390 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f60fc00d980 tx=0x7f60fc00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:49:15.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:15.159+0000 7f6108c00700 1 -- 172.21.15.83:0/2352021682 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60fc017030 con 0x7f610c102ba0 2024-04-24T08:49:15.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:15.159+0000 7f6108c00700 1 -- 172.21.15.83:0/2352021682 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f60fc0044d0 con 0x7f610c102ba0 2024-04-24T08:49:15.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:15.159+0000 7f6108c00700 1 -- 172.21.15.83:0/2352021682 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60fc003aa0 con 0x7f610c102ba0 2024-04-24T08:49:15.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:15.159+0000 7f6112600700 1 -- 172.21.15.83:0/2352021682 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f610c1a0690 con 0x7f610c102ba0 2024-04-24T08:49:15.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:15.159+0000 7f6112600700 1 -- 172.21.15.83:0/2352021682 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f610c1a7f10 con 0x7f610c102ba0 2024-04-24T08:49:15.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:15.160+0000 7f6112600700 1 -- 172.21.15.83:0/2352021682 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f610c0697e0 con 0x7f610c102ba0 2024-04-24T08:49:15.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:15.162+0000 7f6108c00700 1 -- 172.21.15.83:0/2352021682 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f60fc01a070 con 0x7f610c102ba0 2024-04-24T08:49:15.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:15.164+0000 7f6108c00700 1 --2- 172.21.15.83:0/2352021682 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f60f00a3d50 0x7f60f00a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:15.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:15.164+0000 7f6108c00700 1 -- 172.21.15.83:0/2352021682 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f60fc0d00e0 con 0x7f610c102ba0 2024-04-24T08:49:15.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:15.164+0000 7f6102a00700 1 --2- 172.21.15.83:0/2352021682 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f60f00a3d50 0x7f60f00a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:15.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:15.165+0000 7f6102a00700 1 --2- 172.21.15.83:0/2352021682 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f60f00a3d50 0x7f60f00a6210 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f60f4005fd0 tx=0x7f60f400a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:49:15.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:15.169+0000 7f6108c00700 1 -- 172.21.15.83:0/2352021682 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f60fc09c6c0 con 0x7f610c102ba0 2024-04-24T08:49:15.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:15.457+0000 7f6112600700 1 -- 172.21.15.83:0/2352021682 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f610c19c6f0 con 0x7f610c102ba0 2024-04-24T08:49:15.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:15.464+0000 7f6108c00700 1 -- 172.21.15.83:0/2352021682 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+264997 (secure 0 0 0) 0x7f60fc09bfc0 con 0x7f610c102ba0 2024-04-24T08:49:15.467 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:49:15.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:15.472+0000 7f6112600700 1 -- 172.21.15.83:0/2352021682 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f60f00a3d50 msgr2=0x7f60f00a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:15.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:15.472+0000 7f6112600700 1 --2- 172.21.15.83:0/2352021682 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f60f00a3d50 0x7f60f00a6210 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f60f4005fd0 tx=0x7f60f400a560 comp rx=0 tx=0).stop 2024-04-24T08:49:15.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:15.472+0000 7f6112600700 1 -- 172.21.15.83:0/2352021682 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f610c102ba0 msgr2=0x7f610c19b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:15.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:15.472+0000 7f6112600700 1 --2- 172.21.15.83:0/2352021682 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f610c102ba0 0x7f610c19b390 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f60fc00d980 tx=0x7f60fc00dd40 comp rx=0 tx=0).stop 2024-04-24T08:49:15.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:15.473+0000 7f6112600700 1 -- 172.21.15.83:0/2352021682 shutdown_connections 2024-04-24T08:49:15.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:15.473+0000 7f6112600700 1 --2- 172.21.15.83:0/2352021682 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f60f00a3d50 0x7f60f00a6210 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:15.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:15.473+0000 7f6112600700 1 --2- 172.21.15.83:0/2352021682 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f610c102ba0 0x7f610c19b390 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:15.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:15.473+0000 7f6112600700 1 --2- 172.21.15.83:0/2352021682 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f610c107ab0 0x7f610c19fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:15.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:15.473+0000 7f6112600700 1 --2- 172.21.15.83:0/2352021682 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f610c103540 0x7f610c19b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:15.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:15.473+0000 7f6112600700 1 -- 172.21.15.83:0/2352021682 >> 172.21.15.83:0/2352021682 conn(0x7f610c079c50 msgr2=0x7f610c100a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:49:15.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:15.473+0000 7f6112600700 1 -- 172.21.15.83:0/2352021682 shutdown_connections 2024-04-24T08:49:15.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:15.474+0000 7f6112600700 1 -- 172.21.15.83:0/2352021682 wait complete. 2024-04-24T08:49:16.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:49:15 smithi138 ceph-mon[31441]: pgmap v396: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.4 KiB/s wr, 25 op/s 2024-04-24T08:49:16.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:49:15 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/2352021682' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-24T08:49:16.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:49:15 smithi189 ceph-mon[31791]: pgmap v396: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.4 KiB/s wr, 25 op/s 2024-04-24T08:49:16.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:49:15 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/2352021682' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-24T08:49:16.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:49:15 smithi083 ceph-mon[27907]: pgmap v396: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.4 KiB/s wr, 25 op/s 2024-04-24T08:49:16.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:49:15 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/2352021682' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-24T08:49:16.689 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":2550826560},{"type":"v1","addr":"172.21.15.83:6801","nonce":2550826560}]},"active_addr":"172.21.15.83:6801/2550826560","active_change":"2024-04-24T08:38:40.661573+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24115,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":34112,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.83:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.83:0","nonce":1096290901}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.83:0","nonce":3306681366}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.83:0","nonce":4159820288}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.83:0","nonce":1743153945}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.83:0","nonce":271553014}]}]} 2024-04-24T08:49:16.695 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-04-24T08:49:16.695 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-04-24T08:49:16.695 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid bd3e039e-0215-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-24T08:49:17.195 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T08:49:18.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:49:17 smithi138 ceph-mon[31441]: pgmap v397: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.4 KiB/s wr, 21 op/s 2024-04-24T08:49:18.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:49:17 smithi189 ceph-mon[31791]: pgmap v397: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.4 KiB/s wr, 21 op/s 2024-04-24T08:49:18.211 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:49:17 smithi083 ceph-mon[27907]: pgmap v397: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.4 KiB/s wr, 21 op/s 2024-04-24T08:49:19.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:19.573+0000 7fee6c200700 1 -- 172.21.15.83:0/2783314760 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee64105920 msgr2=0x7fee64109970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:19.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:19.573+0000 7fee6c200700 1 --2- 172.21.15.83:0/2783314760 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee64105920 0x7fee64109970 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fee500099c0 tx=0x7fee50009cd0 comp rx=0 tx=0).stop 2024-04-24T08:49:19.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:19.574+0000 7fee6c200700 1 -- 172.21.15.83:0/2783314760 shutdown_connections 2024-04-24T08:49:19.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:19.574+0000 7fee6c200700 1 --2- 172.21.15.83:0/2783314760 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee6410a050 0x7fee64111c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:19.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:19.574+0000 7fee6c200700 1 --2- 172.21.15.83:0/2783314760 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fee64104f70 0x7fee64105350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:19.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:19.574+0000 7fee6c200700 1 --2- 172.21.15.83:0/2783314760 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee64105920 0x7fee64109970 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:19.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:19.574+0000 7fee6c200700 1 -- 172.21.15.83:0/2783314760 >> 172.21.15.83:0/2783314760 conn(0x7fee64100800 msgr2=0x7fee64102c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:49:19.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:19.574+0000 7fee6c200700 1 -- 172.21.15.83:0/2783314760 shutdown_connections 2024-04-24T08:49:19.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:19.574+0000 7fee6c200700 1 -- 172.21.15.83:0/2783314760 wait complete. 2024-04-24T08:49:19.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:19.575+0000 7fee6c200700 1 Processor -- start 2024-04-24T08:49:19.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:19.575+0000 7fee6c200700 1 -- start start 2024-04-24T08:49:19.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:19.576+0000 7fee6c200700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee64104f70 0x7fee6419b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:19.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:19.576+0000 7fee6c200700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee64105920 0x7fee6419b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:19.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:19.576+0000 7fee6c200700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fee6410a050 0x7fee6419fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:19.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:19.576+0000 7fee6c200700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fee64114f30 con 0x7fee64105920 2024-04-24T08:49:19.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:19.576+0000 7fee6c200700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fee64114db0 con 0x7fee64104f70 2024-04-24T08:49:19.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:19.576+0000 7fee6c200700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fee641150b0 con 0x7fee6410a050 2024-04-24T08:49:19.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:19.576+0000 7fee68e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee64105920 0x7fee6419b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:19.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:19.576+0000 7fee68e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee64105920 0x7fee6419b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54232/0 (socket says 172.21.15.83:54232) 2024-04-24T08:49:19.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:19.576+0000 7fee68e00700 1 -- 172.21.15.83:0/3874799527 learned_addr learned my addr 172.21.15.83:0/3874799527 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:49:19.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:19.576+0000 7fee69800700 1 --2- 172.21.15.83:0/3874799527 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee64104f70 0x7fee6419b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:19.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:19.577+0000 7fee6a200700 1 --2- 172.21.15.83:0/3874799527 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fee6410a050 0x7fee6419fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:19.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:19.577+0000 7fee68e00700 1 -- 172.21.15.83:0/3874799527 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fee6410a050 msgr2=0x7fee6419fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:19.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:19.577+0000 7fee68e00700 1 --2- 172.21.15.83:0/3874799527 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fee6410a050 0x7fee6419fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:19.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:19.577+0000 7fee68e00700 1 -- 172.21.15.83:0/3874799527 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee64104f70 msgr2=0x7fee6419b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:19.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:19.577+0000 7fee68e00700 1 --2- 172.21.15.83:0/3874799527 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee64104f70 0x7fee6419b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:19.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:19.577+0000 7fee68e00700 1 -- 172.21.15.83:0/3874799527 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee641a0460 con 0x7fee64105920 2024-04-24T08:49:19.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:19.577+0000 7fee68e00700 1 --2- 172.21.15.83:0/3874799527 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee64105920 0x7fee6419b9e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fee5000c010 tx=0x7fee500047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:49:19.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:19.577+0000 7fee62000700 1 -- 172.21.15.83:0/3874799527 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee50026070 con 0x7fee64105920 2024-04-24T08:49:19.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:19.577+0000 7fee62000700 1 -- 172.21.15.83:0/3874799527 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fee50003f50 con 0x7fee64105920 2024-04-24T08:49:19.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:19.578+0000 7fee6c200700 1 -- 172.21.15.83:0/3874799527 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee641a06f0 con 0x7fee64105920 2024-04-24T08:49:19.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:19.578+0000 7fee62000700 1 -- 172.21.15.83:0/3874799527 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee5000fc20 con 0x7fee64105920 2024-04-24T08:49:19.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:19.578+0000 7fee6c200700 1 -- 172.21.15.83:0/3874799527 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee641a7fd0 con 0x7fee64105920 2024-04-24T08:49:19.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:19.579+0000 7fee6c200700 1 -- 172.21.15.83:0/3874799527 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee64106450 con 0x7fee64105920 2024-04-24T08:49:19.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:19.581+0000 7fee62000700 1 -- 172.21.15.83:0/3874799527 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fee50029070 con 0x7fee64105920 2024-04-24T08:49:19.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:19.582+0000 7fee62000700 1 --2- 172.21.15.83:0/3874799527 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fee4c0a3cd0 0x7fee4c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:19.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:19.582+0000 7fee62000700 1 -- 172.21.15.83:0/3874799527 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fee500d1240 con 0x7fee64105920 2024-04-24T08:49:19.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:19.582+0000 7fee69800700 1 --2- 172.21.15.83:0/3874799527 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fee4c0a3cd0 0x7fee4c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:19.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:19.583+0000 7fee69800700 1 --2- 172.21.15.83:0/3874799527 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fee4c0a3cd0 0x7fee4c0a6190 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fee58004630 tx=0x7fee5800b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:49:19.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:19.587+0000 7fee62000700 1 -- 172.21.15.83:0/3874799527 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fee5005dff0 con 0x7fee64105920 2024-04-24T08:49:19.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:19.812+0000 7fee6c200700 1 -- 172.21.15.83:0/3874799527 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fee6402dfe0 con 0x7fee64105920 2024-04-24T08:49:19.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:19.815+0000 7fee62000700 1 -- 172.21.15.83:0/3874799527 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24127 (secure 0 0 0) 0x7fee50013070 con 0x7fee64105920 2024-04-24T08:49:19.816 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:49:19.816 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":89,"fsid":"bd3e039e-0215-11ef-bc93-c7b262605968","created":"2024-04-24T08:37:32.209944+0000","modified":"2024-04-24T08:48:34.752923+0000","last_up_change":"2024-04-24T08:47:17.966041+0000","last_in_change":"2024-04-24T08:46:53.835936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T08:41:56.834372+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T08:48:26.978311+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T08:48:28.352731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T08:48:32.275167+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2cd77f3a-19dd-4c6e-a199-1dcbd5faa679","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2132675555},{"type":"v1","addr":"172.21.15.83:6803","nonce":2132675555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2132675555},{"type":"v1","addr":"172.21.15.83:6805","nonce":2132675555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2132675555},{"type":"v1","addr":"172.21.15.83:6809","nonce":2132675555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2132675555},{"type":"v1","addr":"172.21.15.83:6807","nonce":2132675555}]},"public_addr":"172.21.15.83:6803/2132675555","cluster_addr":"172.21.15.83:6805/2132675555","heartbeat_back_addr":"172.21.15.83:6809/2132675555","heartbeat_front_addr":"172.21.15.83:6807/2132675555","state":["exists","up"]},{"osd":1,"uuid":"aba7f5df-2a4c-4c23-b502-d193605a5b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":3589158880},{"type":"v1","addr":"172.21.15.138:6801","nonce":3589158880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":3589158880},{"type":"v1","addr":"172.21.15.138:6803","nonce":3589158880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":3589158880},{"type":"v1","addr":"172.21.15.138:6807","nonce":3589158880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":3589158880},{"type":"v1","addr":"172.21.15.138:6805","nonce":3589158880}]},"public_addr":"172.21.15.138:6801/3589158880","cluster_addr":"172.21.15.138:6803/3589158880","heartbeat_back_addr":"172.21.15.138:6807/3589158880","heartbeat_front_addr":"172.21.15.138:6805/3589158880","state":["exists","up"]},{"osd":2,"uuid":"071bc167-2333-4420-bb58-bffbc10d7641","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":3169072688},{"type":"v1","addr":"172.21.15.189:6801","nonce":3169072688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":3169072688},{"type":"v1","addr":"172.21.15.189:6803","nonce":3169072688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":3169072688},{"type":"v1","addr":"172.21.15.189:6807","nonce":3169072688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":3169072688},{"type":"v1","addr":"172.21.15.189:6805","nonce":3169072688}]},"public_addr":"172.21.15.189:6801/3169072688","cluster_addr":"172.21.15.189:6803/3169072688","heartbeat_back_addr":"172.21.15.189:6807/3169072688","heartbeat_front_addr":"172.21.15.189:6805/3169072688","state":["exists","up"]},{"osd":3,"uuid":"f5624d78-4455-4e07-8c1a-f47c94cddb23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3131092450},{"type":"v1","addr":"172.21.15.83:6811","nonce":3131092450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3131092450},{"type":"v1","addr":"172.21.15.83:6813","nonce":3131092450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3131092450},{"type":"v1","addr":"172.21.15.83:6817","nonce":3131092450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3131092450},{"type":"v1","addr":"172.21.15.83:6815","nonce":3131092450}]},"public_addr":"172.21.15.83:6811/3131092450","cluster_addr":"172.21.15.83:6813/3131092450","heartbeat_back_addr":"172.21.15.83:6817/3131092450","heartbeat_front_addr":"172.21.15.83:6815/3131092450","state":["exists","up"]},{"osd":4,"uuid":"7b50cd14-9385-4535-a7b0-1c1da93c8548","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":823775360},{"type":"v1","addr":"172.21.15.138:6809","nonce":823775360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":823775360},{"type":"v1","addr":"172.21.15.138:6811","nonce":823775360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":823775360},{"type":"v1","addr":"172.21.15.138:6815","nonce":823775360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":823775360},{"type":"v1","addr":"172.21.15.138:6813","nonce":823775360}]},"public_addr":"172.21.15.138:6809/823775360","cluster_addr":"172.21.15.138:6811/823775360","heartbeat_back_addr":"172.21.15.138:6815/823775360","heartbeat_front_addr":"172.21.15.138:6813/823775360","state":["exists","up"]},{"osd":5,"uuid":"9c647797-35bc-449f-be84-af97d2c69c2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":1399817431},{"type":"v1","addr":"172.21.15.189:6809","nonce":1399817431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":1399817431},{"type":"v1","addr":"172.21.15.189:6811","nonce":1399817431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":1399817431},{"type":"v1","addr":"172.21.15.189:6815","nonce":1399817431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":1399817431},{"type":"v1","addr":"172.21.15.189:6813","nonce":1399817431}]},"public_addr":"172.21.15.189:6809/1399817431","cluster_addr":"172.21.15.189:6811/1399817431","heartbeat_back_addr":"172.21.15.189:6815/1399817431","heartbeat_front_addr":"172.21.15.189:6813/1399817431","state":["exists","up"]},{"osd":6,"uuid":"8f056584-16c8-4454-b445-3f780f26c196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":1611358483},{"type":"v1","addr":"172.21.15.83:6819","nonce":1611358483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1611358483},{"type":"v1","addr":"172.21.15.83:6821","nonce":1611358483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1611358483},{"type":"v1","addr":"172.21.15.83:6825","nonce":1611358483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":1611358483},{"type":"v1","addr":"172.21.15.83:6823","nonce":1611358483}]},"public_addr":"172.21.15.83:6819/1611358483","cluster_addr":"172.21.15.83:6821/1611358483","heartbeat_back_addr":"172.21.15.83:6825/1611358483","heartbeat_front_addr":"172.21.15.83:6823/1611358483","state":["exists","up"]},{"osd":7,"uuid":"7603efe3-da77-4b7f-9d03-29c4f293b6d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":3557856706},{"type":"v1","addr":"172.21.15.138:6817","nonce":3557856706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":3557856706},{"type":"v1","addr":"172.21.15.138:6819","nonce":3557856706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":3557856706},{"type":"v1","addr":"172.21.15.138:6823","nonce":3557856706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":3557856706},{"type":"v1","addr":"172.21.15.138:6821","nonce":3557856706}]},"public_addr":"172.21.15.138:6817/3557856706","cluster_addr":"172.21.15.138:6819/3557856706","heartbeat_back_addr":"172.21.15.138:6823/3557856706","heartbeat_front_addr":"172.21.15.138:6821/3557856706","state":["exists","up"]},{"osd":8,"uuid":"27d7d1b8-d6a0-417e-b8aa-596325aa1a77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":207018554},{"type":"v1","addr":"172.21.15.189:6817","nonce":207018554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":207018554},{"type":"v1","addr":"172.21.15.189:6819","nonce":207018554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":207018554},{"type":"v1","addr":"172.21.15.189:6823","nonce":207018554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":207018554},{"type":"v1","addr":"172.21.15.189:6821","nonce":207018554}]},"public_addr":"172.21.15.189:6817/207018554","cluster_addr":"172.21.15.189:6819/207018554","heartbeat_back_addr":"172.21.15.189:6823/207018554","heartbeat_front_addr":"172.21.15.189:6821/207018554","state":["exists","up"]},{"osd":9,"uuid":"baf19b98-881b-4d3a-9f72-8aa2dd73cad6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":519636751},{"type":"v1","addr":"172.21.15.83:6827","nonce":519636751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":519636751},{"type":"v1","addr":"172.21.15.83:6829","nonce":519636751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":519636751},{"type":"v1","addr":"172.21.15.83:6833","nonce":519636751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":519636751},{"type":"v1","addr":"172.21.15.83:6831","nonce":519636751}]},"public_addr":"172.21.15.83:6827/519636751","cluster_addr":"172.21.15.83:6829/519636751","heartbeat_back_addr":"172.21.15.83:6833/519636751","heartbeat_front_addr":"172.21.15.83:6831/519636751","state":["exists","up"]},{"osd":10,"uuid":"eb346717-5c07-47c8-95be-0e68fe449fdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1130872098},{"type":"v1","addr":"172.21.15.138:6825","nonce":1130872098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1130872098},{"type":"v1","addr":"172.21.15.138:6827","nonce":1130872098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1130872098},{"type":"v1","addr":"172.21.15.138:6831","nonce":1130872098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1130872098},{"type":"v1","addr":"172.21.15.138:6829","nonce":1130872098}]},"public_addr":"172.21.15.138:6825/1130872098","cluster_addr":"172.21.15.138:6827/1130872098","heartbeat_back_addr":"172.21.15.138:6831/1130872098","heartbeat_front_addr":"172.21.15.138:6829/1130872098","state":["exists","up"]},{"osd":11,"uuid":"8e37568e-981c-45b8-b2af-2f1b73898542","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":2998782690},{"type":"v1","addr":"172.21.15.189:6825","nonce":2998782690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":2998782690},{"type":"v1","addr":"172.21.15.189:6827","nonce":2998782690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":2998782690},{"type":"v1","addr":"172.21.15.189:6831","nonce":2998782690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":2998782690},{"type":"v1","addr":"172.21.15.189:6829","nonce":2998782690}]},"public_addr":"172.21.15.189:6825/2998782690","cluster_addr":"172.21.15.189:6827/2998782690","heartbeat_back_addr":"172.21.15.189:6831/2998782690","heartbeat_front_addr":"172.21.15.189:6829/2998782690","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:40:47.980103+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:41:21.970795+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:41:55.160426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:42:30.400475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:43:05.705663+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:43:40.112312+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:44:16.781289+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:44:52.151187+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:45:28.055944+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:46:04.969301+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:46:40.918306+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:47:16.886874+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:6801/820049883":"2024-04-25T08:38:40.661315+0000","172.21.15.83:0/4272874138":"2024-04-25T08:38:40.661315+0000","172.21.15.83:0/4212994335":"2024-04-25T08:38:40.661315+0000","172.21.15.83:6801/1996371091":"2024-04-25T08:38:05.800856+0000","172.21.15.83:0/1820663683":"2024-04-25T08:38:05.800856+0000","172.21.15.83:0/3802897262":"2024-04-25T08:38:05.800856+0000","172.21.15.83:6800/1996371091":"2024-04-25T08:38:05.800856+0000","172.21.15.83:6800/820049883":"2024-04-25T08:38:40.661315+0000","172.21.15.83:0/1366125049":"2024-04-25T08:38:05.800856+0000","172.21.15.83:0/340452868":"2024-04-25T08:38:40.661315+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T08:49:19.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:19.818+0000 7fee6c200700 1 -- 172.21.15.83:0/3874799527 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fee4c0a3cd0 msgr2=0x7fee4c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:19.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:19.818+0000 7fee6c200700 1 --2- 172.21.15.83:0/3874799527 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fee4c0a3cd0 0x7fee4c0a6190 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fee58004630 tx=0x7fee5800b3c0 comp rx=0 tx=0).stop 2024-04-24T08:49:19.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:19.818+0000 7fee6c200700 1 -- 172.21.15.83:0/3874799527 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee64105920 msgr2=0x7fee6419b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:19.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:19.818+0000 7fee6c200700 1 --2- 172.21.15.83:0/3874799527 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee64105920 0x7fee6419b9e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fee5000c010 tx=0x7fee500047b0 comp rx=0 tx=0).stop 2024-04-24T08:49:19.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:19.818+0000 7fee6c200700 1 -- 172.21.15.83:0/3874799527 shutdown_connections 2024-04-24T08:49:19.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:19.818+0000 7fee6c200700 1 --2- 172.21.15.83:0/3874799527 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fee4c0a3cd0 0x7fee4c0a6190 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:19.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:19.818+0000 7fee6c200700 1 --2- 172.21.15.83:0/3874799527 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fee6410a050 0x7fee6419fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:19.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:19.818+0000 7fee6c200700 1 --2- 172.21.15.83:0/3874799527 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee64104f70 0x7fee6419b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:19.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:19.818+0000 7fee6c200700 1 --2- 172.21.15.83:0/3874799527 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee64105920 0x7fee6419b9e0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:19.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:19.818+0000 7fee6c200700 1 -- 172.21.15.83:0/3874799527 >> 172.21.15.83:0/3874799527 conn(0x7fee64100800 msgr2=0x7fee64102ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:49:19.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:19.818+0000 7fee6c200700 1 -- 172.21.15.83:0/3874799527 shutdown_connections 2024-04-24T08:49:19.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:19.818+0000 7fee6c200700 1 -- 172.21.15.83:0/3874799527 wait complete. 2024-04-24T08:49:20.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:49:19 smithi138 ceph-mon[31441]: pgmap v398: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.6 KiB/s wr, 18 op/s 2024-04-24T08:49:20.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:49:19 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/3874799527' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T08:49:20.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:49:19 smithi189 ceph-mon[31791]: pgmap v398: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.6 KiB/s wr, 18 op/s 2024-04-24T08:49:20.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:49:19 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/3874799527' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T08:49:20.226 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:49:19 smithi083 ceph-mon[27907]: pgmap v398: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.6 KiB/s wr, 18 op/s 2024-04-24T08:49:20.226 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:49:19 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/3874799527' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T08:49:20.668 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-04-24T08:49:20.668 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid bd3e039e-0215-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-24T08:49:21.104 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T08:49:22.170 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:49:21 smithi083 ceph-mon[27907]: pgmap v399: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.8 KiB/s wr, 10 op/s 2024-04-24T08:49:22.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:49:21 smithi138 ceph-mon[31441]: pgmap v399: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.8 KiB/s wr, 10 op/s 2024-04-24T08:49:22.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:49:21 smithi189 ceph-mon[31791]: pgmap v399: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.8 KiB/s wr, 10 op/s 2024-04-24T08:49:23.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:23.675+0000 7f1cae000700 1 -- 172.21.15.83:0/2388883818 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ca8109d30 msgr2=0x7f1ca810dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:23.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:23.675+0000 7f1cae000700 1 --2- 172.21.15.83:0/2388883818 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ca8109d30 0x7f1ca810dd80 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f1c900099c0 tx=0x7f1c90009cd0 comp rx=0 tx=0).stop 2024-04-24T08:49:23.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:23.675+0000 7f1cae000700 1 -- 172.21.15.83:0/2388883818 shutdown_connections 2024-04-24T08:49:23.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:23.675+0000 7f1cae000700 1 --2- 172.21.15.83:0/2388883818 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ca810e460 0x7f1ca8116030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:23.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:23.675+0000 7f1cae000700 1 --2- 172.21.15.83:0/2388883818 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1ca8109380 0x7f1ca8109760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:23.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:23.675+0000 7f1cae000700 1 --2- 172.21.15.83:0/2388883818 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ca8109d30 0x7f1ca810dd80 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:23.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:23.675+0000 7f1cae000700 1 -- 172.21.15.83:0/2388883818 >> 172.21.15.83:0/2388883818 conn(0x7f1ca80762c0 msgr2=0x7f1ca8107910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:49:23.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:23.675+0000 7f1cae000700 1 -- 172.21.15.83:0/2388883818 shutdown_connections 2024-04-24T08:49:23.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:23.675+0000 7f1cae000700 1 -- 172.21.15.83:0/2388883818 wait complete. 2024-04-24T08:49:23.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:23.676+0000 7f1cae000700 1 Processor -- start 2024-04-24T08:49:23.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:23.676+0000 7f1cae000700 1 -- start start 2024-04-24T08:49:23.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:23.677+0000 7f1cae000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1ca8109380 0x7f1ca81a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:23.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:23.677+0000 7f1cae000700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ca8109d30 0x7f1ca81a5b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:23.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:23.677+0000 7f1cae000700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ca810e460 0x7f1ca819f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:23.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:23.677+0000 7f1cae000700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1ca8119350 con 0x7f1ca810e460 2024-04-24T08:49:23.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:23.677+0000 7f1cae000700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1ca81191d0 con 0x7f1ca8109d30 2024-04-24T08:49:23.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:23.677+0000 7f1cae000700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1ca81194d0 con 0x7f1ca8109380 2024-04-24T08:49:23.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:23.677+0000 7f1ca7e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ca810e460 0x7f1ca819f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:23.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:23.677+0000 7f1ca7e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ca810e460 0x7f1ca819f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54244/0 (socket says 172.21.15.83:54244) 2024-04-24T08:49:23.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:23.677+0000 7f1ca7e00700 1 -- 172.21.15.83:0/3692759529 learned_addr learned my addr 172.21.15.83:0/3692759529 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:49:23.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:23.677+0000 7f1c9ea00700 1 --2- 172.21.15.83:0/3692759529 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ca8109d30 0x7f1ca81a5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:23.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:23.677+0000 7f1ca7400700 1 --2- 172.21.15.83:0/3692759529 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1ca8109380 0x7f1ca81a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:23.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:23.678+0000 7f1ca7e00700 1 -- 172.21.15.83:0/3692759529 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1ca8109380 msgr2=0x7f1ca81a55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:23.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:23.678+0000 7f1ca7e00700 1 --2- 172.21.15.83:0/3692759529 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1ca8109380 0x7f1ca81a55d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:23.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:23.678+0000 7f1ca7e00700 1 -- 172.21.15.83:0/3692759529 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ca8109d30 msgr2=0x7f1ca81a5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:23.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:23.678+0000 7f1ca7e00700 1 --2- 172.21.15.83:0/3692759529 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ca8109d30 0x7f1ca81a5b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:23.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:23.678+0000 7f1ca7e00700 1 -- 172.21.15.83:0/3692759529 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ca819ffd0 con 0x7f1ca810e460 2024-04-24T08:49:23.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:23.678+0000 7f1c9ea00700 1 --2- 172.21.15.83:0/3692759529 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ca8109d30 0x7f1ca81a5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:49:23.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:23.678+0000 7f1ca7e00700 1 --2- 172.21.15.83:0/3692759529 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ca810e460 0x7f1ca819f6f0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f1ca000ada0 tx=0x7f1ca000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:49:23.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:23.678+0000 7f1ca4c00700 1 -- 172.21.15.83:0/3692759529 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca0017020 con 0x7f1ca810e460 2024-04-24T08:49:23.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:23.678+0000 7f1cae000700 1 -- 172.21.15.83:0/3692759529 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ca81a02c0 con 0x7f1ca810e460 2024-04-24T08:49:23.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:23.678+0000 7f1ca4c00700 1 -- 172.21.15.83:0/3692759529 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1ca000ce30 con 0x7f1ca810e460 2024-04-24T08:49:23.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:23.678+0000 7f1ca4c00700 1 -- 172.21.15.83:0/3692759529 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca0005460 con 0x7f1ca810e460 2024-04-24T08:49:23.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:23.679+0000 7f1cae000700 1 -- 172.21.15.83:0/3692759529 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ca81ac550 con 0x7f1ca810e460 2024-04-24T08:49:23.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:23.680+0000 7f1cae000700 1 -- 172.21.15.83:0/3692759529 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ca810a860 con 0x7f1ca810e460 2024-04-24T08:49:23.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:23.682+0000 7f1ca4c00700 1 -- 172.21.15.83:0/3692759529 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f1ca000c980 con 0x7f1ca810e460 2024-04-24T08:49:23.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:23.683+0000 7f1ca4c00700 1 --2- 172.21.15.83:0/3692759529 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f1c8c0a3cd0 0x7f1c8c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:23.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:23.683+0000 7f1ca4c00700 1 -- 172.21.15.83:0/3692759529 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f1ca00d00c0 con 0x7f1ca810e460 2024-04-24T08:49:23.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:23.683+0000 7f1ca7400700 1 --2- 172.21.15.83:0/3692759529 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f1c8c0a3cd0 0x7f1c8c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:23.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:23.685+0000 7f1ca7400700 1 --2- 172.21.15.83:0/3692759529 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f1c8c0a3cd0 0x7f1c8c0a6190 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1c98006fd0 tx=0x7f1c98008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:49:23.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:23.688+0000 7f1ca4c00700 1 -- 172.21.15.83:0/3692759529 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1ca0021020 con 0x7f1ca810e460 2024-04-24T08:49:23.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:23.917+0000 7f1cae000700 1 -- 172.21.15.83:0/3692759529 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1ca802dfe0 con 0x7f1ca810e460 2024-04-24T08:49:23.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:23.921+0000 7f1ca4c00700 1 -- 172.21.15.83:0/3692759529 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24127 (secure 0 0 0) 0x7f1ca009c730 con 0x7f1ca810e460 2024-04-24T08:49:23.922 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:49:23.923 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":89,"fsid":"bd3e039e-0215-11ef-bc93-c7b262605968","created":"2024-04-24T08:37:32.209944+0000","modified":"2024-04-24T08:48:34.752923+0000","last_up_change":"2024-04-24T08:47:17.966041+0000","last_in_change":"2024-04-24T08:46:53.835936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T08:41:56.834372+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T08:48:26.978311+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T08:48:28.352731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T08:48:32.275167+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2cd77f3a-19dd-4c6e-a199-1dcbd5faa679","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2132675555},{"type":"v1","addr":"172.21.15.83:6803","nonce":2132675555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2132675555},{"type":"v1","addr":"172.21.15.83:6805","nonce":2132675555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2132675555},{"type":"v1","addr":"172.21.15.83:6809","nonce":2132675555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2132675555},{"type":"v1","addr":"172.21.15.83:6807","nonce":2132675555}]},"public_addr":"172.21.15.83:6803/2132675555","cluster_addr":"172.21.15.83:6805/2132675555","heartbeat_back_addr":"172.21.15.83:6809/2132675555","heartbeat_front_addr":"172.21.15.83:6807/2132675555","state":["exists","up"]},{"osd":1,"uuid":"aba7f5df-2a4c-4c23-b502-d193605a5b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":3589158880},{"type":"v1","addr":"172.21.15.138:6801","nonce":3589158880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":3589158880},{"type":"v1","addr":"172.21.15.138:6803","nonce":3589158880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":3589158880},{"type":"v1","addr":"172.21.15.138:6807","nonce":3589158880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":3589158880},{"type":"v1","addr":"172.21.15.138:6805","nonce":3589158880}]},"public_addr":"172.21.15.138:6801/3589158880","cluster_addr":"172.21.15.138:6803/3589158880","heartbeat_back_addr":"172.21.15.138:6807/3589158880","heartbeat_front_addr":"172.21.15.138:6805/3589158880","state":["exists","up"]},{"osd":2,"uuid":"071bc167-2333-4420-bb58-bffbc10d7641","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":3169072688},{"type":"v1","addr":"172.21.15.189:6801","nonce":3169072688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":3169072688},{"type":"v1","addr":"172.21.15.189:6803","nonce":3169072688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":3169072688},{"type":"v1","addr":"172.21.15.189:6807","nonce":3169072688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":3169072688},{"type":"v1","addr":"172.21.15.189:6805","nonce":3169072688}]},"public_addr":"172.21.15.189:6801/3169072688","cluster_addr":"172.21.15.189:6803/3169072688","heartbeat_back_addr":"172.21.15.189:6807/3169072688","heartbeat_front_addr":"172.21.15.189:6805/3169072688","state":["exists","up"]},{"osd":3,"uuid":"f5624d78-4455-4e07-8c1a-f47c94cddb23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3131092450},{"type":"v1","addr":"172.21.15.83:6811","nonce":3131092450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3131092450},{"type":"v1","addr":"172.21.15.83:6813","nonce":3131092450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3131092450},{"type":"v1","addr":"172.21.15.83:6817","nonce":3131092450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3131092450},{"type":"v1","addr":"172.21.15.83:6815","nonce":3131092450}]},"public_addr":"172.21.15.83:6811/3131092450","cluster_addr":"172.21.15.83:6813/3131092450","heartbeat_back_addr":"172.21.15.83:6817/3131092450","heartbeat_front_addr":"172.21.15.83:6815/3131092450","state":["exists","up"]},{"osd":4,"uuid":"7b50cd14-9385-4535-a7b0-1c1da93c8548","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":823775360},{"type":"v1","addr":"172.21.15.138:6809","nonce":823775360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":823775360},{"type":"v1","addr":"172.21.15.138:6811","nonce":823775360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":823775360},{"type":"v1","addr":"172.21.15.138:6815","nonce":823775360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":823775360},{"type":"v1","addr":"172.21.15.138:6813","nonce":823775360}]},"public_addr":"172.21.15.138:6809/823775360","cluster_addr":"172.21.15.138:6811/823775360","heartbeat_back_addr":"172.21.15.138:6815/823775360","heartbeat_front_addr":"172.21.15.138:6813/823775360","state":["exists","up"]},{"osd":5,"uuid":"9c647797-35bc-449f-be84-af97d2c69c2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":1399817431},{"type":"v1","addr":"172.21.15.189:6809","nonce":1399817431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":1399817431},{"type":"v1","addr":"172.21.15.189:6811","nonce":1399817431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":1399817431},{"type":"v1","addr":"172.21.15.189:6815","nonce":1399817431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":1399817431},{"type":"v1","addr":"172.21.15.189:6813","nonce":1399817431}]},"public_addr":"172.21.15.189:6809/1399817431","cluster_addr":"172.21.15.189:6811/1399817431","heartbeat_back_addr":"172.21.15.189:6815/1399817431","heartbeat_front_addr":"172.21.15.189:6813/1399817431","state":["exists","up"]},{"osd":6,"uuid":"8f056584-16c8-4454-b445-3f780f26c196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":1611358483},{"type":"v1","addr":"172.21.15.83:6819","nonce":1611358483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1611358483},{"type":"v1","addr":"172.21.15.83:6821","nonce":1611358483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1611358483},{"type":"v1","addr":"172.21.15.83:6825","nonce":1611358483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":1611358483},{"type":"v1","addr":"172.21.15.83:6823","nonce":1611358483}]},"public_addr":"172.21.15.83:6819/1611358483","cluster_addr":"172.21.15.83:6821/1611358483","heartbeat_back_addr":"172.21.15.83:6825/1611358483","heartbeat_front_addr":"172.21.15.83:6823/1611358483","state":["exists","up"]},{"osd":7,"uuid":"7603efe3-da77-4b7f-9d03-29c4f293b6d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":3557856706},{"type":"v1","addr":"172.21.15.138:6817","nonce":3557856706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":3557856706},{"type":"v1","addr":"172.21.15.138:6819","nonce":3557856706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":3557856706},{"type":"v1","addr":"172.21.15.138:6823","nonce":3557856706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":3557856706},{"type":"v1","addr":"172.21.15.138:6821","nonce":3557856706}]},"public_addr":"172.21.15.138:6817/3557856706","cluster_addr":"172.21.15.138:6819/3557856706","heartbeat_back_addr":"172.21.15.138:6823/3557856706","heartbeat_front_addr":"172.21.15.138:6821/3557856706","state":["exists","up"]},{"osd":8,"uuid":"27d7d1b8-d6a0-417e-b8aa-596325aa1a77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":207018554},{"type":"v1","addr":"172.21.15.189:6817","nonce":207018554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":207018554},{"type":"v1","addr":"172.21.15.189:6819","nonce":207018554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":207018554},{"type":"v1","addr":"172.21.15.189:6823","nonce":207018554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":207018554},{"type":"v1","addr":"172.21.15.189:6821","nonce":207018554}]},"public_addr":"172.21.15.189:6817/207018554","cluster_addr":"172.21.15.189:6819/207018554","heartbeat_back_addr":"172.21.15.189:6823/207018554","heartbeat_front_addr":"172.21.15.189:6821/207018554","state":["exists","up"]},{"osd":9,"uuid":"baf19b98-881b-4d3a-9f72-8aa2dd73cad6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":519636751},{"type":"v1","addr":"172.21.15.83:6827","nonce":519636751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":519636751},{"type":"v1","addr":"172.21.15.83:6829","nonce":519636751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":519636751},{"type":"v1","addr":"172.21.15.83:6833","nonce":519636751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":519636751},{"type":"v1","addr":"172.21.15.83:6831","nonce":519636751}]},"public_addr":"172.21.15.83:6827/519636751","cluster_addr":"172.21.15.83:6829/519636751","heartbeat_back_addr":"172.21.15.83:6833/519636751","heartbeat_front_addr":"172.21.15.83:6831/519636751","state":["exists","up"]},{"osd":10,"uuid":"eb346717-5c07-47c8-95be-0e68fe449fdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1130872098},{"type":"v1","addr":"172.21.15.138:6825","nonce":1130872098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1130872098},{"type":"v1","addr":"172.21.15.138:6827","nonce":1130872098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1130872098},{"type":"v1","addr":"172.21.15.138:6831","nonce":1130872098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1130872098},{"type":"v1","addr":"172.21.15.138:6829","nonce":1130872098}]},"public_addr":"172.21.15.138:6825/1130872098","cluster_addr":"172.21.15.138:6827/1130872098","heartbeat_back_addr":"172.21.15.138:6831/1130872098","heartbeat_front_addr":"172.21.15.138:6829/1130872098","state":["exists","up"]},{"osd":11,"uuid":"8e37568e-981c-45b8-b2af-2f1b73898542","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":2998782690},{"type":"v1","addr":"172.21.15.189:6825","nonce":2998782690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":2998782690},{"type":"v1","addr":"172.21.15.189:6827","nonce":2998782690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":2998782690},{"type":"v1","addr":"172.21.15.189:6831","nonce":2998782690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":2998782690},{"type":"v1","addr":"172.21.15.189:6829","nonce":2998782690}]},"public_addr":"172.21.15.189:6825/2998782690","cluster_addr":"172.21.15.189:6827/2998782690","heartbeat_back_addr":"172.21.15.189:6831/2998782690","heartbeat_front_addr":"172.21.15.189:6829/2998782690","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:40:47.980103+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:41:21.970795+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:41:55.160426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:42:30.400475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:43:05.705663+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:43:40.112312+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:44:16.781289+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:44:52.151187+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:45:28.055944+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:46:04.969301+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:46:40.918306+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:47:16.886874+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:6801/820049883":"2024-04-25T08:38:40.661315+0000","172.21.15.83:0/4272874138":"2024-04-25T08:38:40.661315+0000","172.21.15.83:0/4212994335":"2024-04-25T08:38:40.661315+0000","172.21.15.83:6801/1996371091":"2024-04-25T08:38:05.800856+0000","172.21.15.83:0/1820663683":"2024-04-25T08:38:05.800856+0000","172.21.15.83:0/3802897262":"2024-04-25T08:38:05.800856+0000","172.21.15.83:6800/1996371091":"2024-04-25T08:38:05.800856+0000","172.21.15.83:6800/820049883":"2024-04-25T08:38:40.661315+0000","172.21.15.83:0/1366125049":"2024-04-25T08:38:05.800856+0000","172.21.15.83:0/340452868":"2024-04-25T08:38:40.661315+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T08:49:23.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:23.924+0000 7f1cae000700 1 -- 172.21.15.83:0/3692759529 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f1c8c0a3cd0 msgr2=0x7f1c8c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:23.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:23.924+0000 7f1cae000700 1 --2- 172.21.15.83:0/3692759529 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f1c8c0a3cd0 0x7f1c8c0a6190 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1c98006fd0 tx=0x7f1c98008040 comp rx=0 tx=0).stop 2024-04-24T08:49:23.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:23.924+0000 7f1cae000700 1 -- 172.21.15.83:0/3692759529 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ca810e460 msgr2=0x7f1ca819f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:23.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:23.924+0000 7f1cae000700 1 --2- 172.21.15.83:0/3692759529 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ca810e460 0x7f1ca819f6f0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f1ca000ada0 tx=0x7f1ca000c330 comp rx=0 tx=0).stop 2024-04-24T08:49:23.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:23.924+0000 7f1cae000700 1 -- 172.21.15.83:0/3692759529 shutdown_connections 2024-04-24T08:49:23.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:23.924+0000 7f1cae000700 1 --2- 172.21.15.83:0/3692759529 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f1c8c0a3cd0 0x7f1c8c0a6190 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:23.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:23.925+0000 7f1cae000700 1 --2- 172.21.15.83:0/3692759529 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ca810e460 0x7f1ca819f6f0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:23.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:23.925+0000 7f1cae000700 1 --2- 172.21.15.83:0/3692759529 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1ca8109380 0x7f1ca81a55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:23.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:23.925+0000 7f1cae000700 1 --2- 172.21.15.83:0/3692759529 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ca8109d30 0x7f1ca81a5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:23.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:23.925+0000 7f1cae000700 1 -- 172.21.15.83:0/3692759529 >> 172.21.15.83:0/3692759529 conn(0x7f1ca80762c0 msgr2=0x7f1ca810f630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:49:23.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:23.925+0000 7f1cae000700 1 -- 172.21.15.83:0/3692759529 shutdown_connections 2024-04-24T08:49:23.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:23.925+0000 7f1cae000700 1 -- 172.21.15.83:0/3692759529 wait complete. 2024-04-24T08:49:24.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:49:23 smithi138 ceph-mon[31441]: pgmap v400: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-04-24T08:49:24.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:49:23 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/3692759529' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T08:49:24.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:49:23 smithi189 ceph-mon[31791]: pgmap v400: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-04-24T08:49:24.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:49:23 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/3692759529' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T08:49:24.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:49:23 smithi083 ceph-mon[27907]: pgmap v400: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-04-24T08:49:24.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:49:23 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/3692759529' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T08:49:24.792 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid bd3e039e-0215-11ef-bc93-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-04-24T08:49:24.793 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid bd3e039e-0215-11ef-bc93-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-04-24T08:49:24.793 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid bd3e039e-0215-11ef-bc93-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-04-24T08:49:24.793 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid bd3e039e-0215-11ef-bc93-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-04-24T08:49:24.794 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid bd3e039e-0215-11ef-bc93-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-04-24T08:49:24.794 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid bd3e039e-0215-11ef-bc93-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-04-24T08:49:24.795 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid bd3e039e-0215-11ef-bc93-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-04-24T08:49:24.795 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid bd3e039e-0215-11ef-bc93-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-04-24T08:49:24.796 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid bd3e039e-0215-11ef-bc93-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-04-24T08:49:24.796 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid bd3e039e-0215-11ef-bc93-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-04-24T08:49:24.796 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid bd3e039e-0215-11ef-bc93-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-04-24T08:49:24.797 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid bd3e039e-0215-11ef-bc93-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-04-24T08:49:25.758 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T08:49:25.769 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T08:49:26.222 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:49:25 smithi083 ceph-mon[27907]: pgmap v401: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2024-04-24T08:49:26.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:49:25 smithi138 ceph-mon[31441]: pgmap v401: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2024-04-24T08:49:26.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:49:25 smithi189 ceph-mon[31791]: pgmap v401: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2024-04-24T08:49:27.195 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T08:49:27.201 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T08:49:27.202 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T08:49:27.203 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T08:49:27.206 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T08:49:27.209 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T08:49:27.209 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T08:49:28.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:49:27 smithi083 ceph-mon[27907]: pgmap v402: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 341 B/s wr, 9 op/s 2024-04-24T08:49:28.389 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T08:49:28.390 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T08:49:28.390 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T08:49:28.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:49:27 smithi138 ceph-mon[31441]: pgmap v402: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 341 B/s wr, 9 op/s 2024-04-24T08:49:28.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:49:27 smithi189 ceph-mon[31791]: pgmap v402: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 341 B/s wr, 9 op/s 2024-04-24T08:49:30.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:49:29 smithi083 ceph-mon[27907]: pgmap v403: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 341 B/s wr, 11 op/s 2024-04-24T08:49:30.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:49:29 smithi138 ceph-mon[31441]: pgmap v403: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 341 B/s wr, 11 op/s 2024-04-24T08:49:30.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:49:29 smithi189 ceph-mon[31791]: pgmap v403: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 341 B/s wr, 11 op/s 2024-04-24T08:49:32.243 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:49:31 smithi083 ceph-mon[27907]: pgmap v404: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 341 B/s wr, 9 op/s 2024-04-24T08:49:32.243 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:49:31 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:49:32.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:49:31 smithi138 ceph-mon[31441]: pgmap v404: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 341 B/s wr, 9 op/s 2024-04-24T08:49:32.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:49:31 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:49:32.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:49:31 smithi189 ceph-mon[31791]: pgmap v404: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 341 B/s wr, 9 op/s 2024-04-24T08:49:32.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:49:31 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:49:33.468 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:49:33 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:49:33.468 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:49:33 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:49:33.468 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:49:33 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:49:33.468 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:49:33 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:49:33.468 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:49:33 smithi083 ceph-mon[27907]: pgmap v405: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 341 B/s wr, 9 op/s 2024-04-24T08:49:33.562 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:49:33 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:49:33.562 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:49:33 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:49:33.562 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:49:33 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:49:33.562 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:49:33 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:49:33.562 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:49:33 smithi138 ceph-mon[31441]: pgmap v405: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 341 B/s wr, 9 op/s 2024-04-24T08:49:33.639 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:49:33 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:49:33.639 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:49:33 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:49:33.639 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:49:33 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:49:33.640 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:49:33 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:49:33.640 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:49:33 smithi189 ceph-mon[31791]: pgmap v405: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 341 B/s wr, 9 op/s 2024-04-24T08:49:34.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:32.939+0000 7f7a75800700 1 -- 172.21.15.83:0/2765161488 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a70109030 msgr2=0x7f7a701094b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:34.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:32.939+0000 7f7a75800700 1 --2- 172.21.15.83:0/2765161488 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a70109030 0x7f7a701094b0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c009990 tx=0x7f7a5c009ca0 comp rx=0 tx=0).stop 2024-04-24T08:49:34.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:32.939+0000 7f7a75800700 1 -- 172.21.15.83:0/2765161488 shutdown_connections 2024-04-24T08:49:34.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:32.939+0000 7f7a75800700 1 --2- 172.21.15.83:0/2765161488 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7a701099f0 0x7f7a70115e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:34.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:32.939+0000 7f7a75800700 1 --2- 172.21.15.83:0/2765161488 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a7010f0e0 0x7f7a7010f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:34.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:32.939+0000 7f7a75800700 1 --2- 172.21.15.83:0/2765161488 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a70109030 0x7f7a701094b0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:34.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:32.939+0000 7f7a75800700 1 -- 172.21.15.83:0/2765161488 >> 172.21.15.83:0/2765161488 conn(0x7f7a70078880 msgr2=0x7f7a70107910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:49:34.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:32.940+0000 7f7a75800700 1 -- 172.21.15.83:0/2765161488 shutdown_connections 2024-04-24T08:49:34.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:32.940+0000 7f7a75800700 1 -- 172.21.15.83:0/2765161488 wait complete. 2024-04-24T08:49:34.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:32.941+0000 7f7a75800700 1 Processor -- start 2024-04-24T08:49:34.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:32.941+0000 7f7a75800700 1 -- start start 2024-04-24T08:49:34.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:32.941+0000 7f7a75800700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a70109030 0x7f7a7019f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:34.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:32.941+0000 7f7a75800700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7a701099f0 0x7f7a7019fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:34.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:32.941+0000 7f7a75800700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a7010f0e0 0x7f7a701a3f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:34.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:32.941+0000 7f7a75800700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7a701191f0 con 0x7f7a7010f0e0 2024-04-24T08:49:34.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:32.941+0000 7f7a75800700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7a70119070 con 0x7f7a70109030 2024-04-24T08:49:34.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:32.941+0000 7f7a75800700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7a70119370 con 0x7f7a701099f0 2024-04-24T08:49:34.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:32.942+0000 7f7a74e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a7010f0e0 0x7f7a701a3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:34.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:32.942+0000 7f7a74e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a7010f0e0 0x7f7a701a3f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60448/0 (socket says 172.21.15.83:60448) 2024-04-24T08:49:34.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:32.942+0000 7f7a74e00700 1 -- 172.21.15.83:0/3506004874 learned_addr learned my addr 172.21.15.83:0/3506004874 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:49:34.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:32.942+0000 7f7a6f400700 1 --2- 172.21.15.83:0/3506004874 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7a701099f0 0x7f7a7019fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:34.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:32.942+0000 7f7a6fe00700 1 --2- 172.21.15.83:0/3506004874 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a70109030 0x7f7a7019f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:34.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:32.942+0000 7f7a74e00700 1 -- 172.21.15.83:0/3506004874 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7a701099f0 msgr2=0x7f7a7019fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:34.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:32.942+0000 7f7a74e00700 1 --2- 172.21.15.83:0/3506004874 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7a701099f0 0x7f7a7019fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:34.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:32.942+0000 7f7a74e00700 1 -- 172.21.15.83:0/3506004874 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a70109030 msgr2=0x7f7a7019f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:34.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:32.942+0000 7f7a74e00700 1 --2- 172.21.15.83:0/3506004874 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a70109030 0x7f7a7019f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:34.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:32.942+0000 7f7a74e00700 1 -- 172.21.15.83:0/3506004874 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a701a4670 con 0x7f7a7010f0e0 2024-04-24T08:49:34.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:32.942+0000 7f7a6f400700 1 --2- 172.21.15.83:0/3506004874 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7a701099f0 0x7f7a7019fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:49:34.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:32.942+0000 7f7a74e00700 1 --2- 172.21.15.83:0/3506004874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a7010f0e0 0x7f7a701a3f30 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f7a68007ee0 tx=0x7f7a6800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:49:34.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:32.943+0000 7f7a6cc00700 1 -- 172.21.15.83:0/3506004874 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a6800dbd0 con 0x7f7a7010f0e0 2024-04-24T08:49:34.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:32.943+0000 7f7a75800700 1 -- 172.21.15.83:0/3506004874 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a701a4960 con 0x7f7a7010f0e0 2024-04-24T08:49:34.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:32.943+0000 7f7a6cc00700 1 -- 172.21.15.83:0/3506004874 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7a6800dd70 con 0x7f7a7010f0e0 2024-04-24T08:49:34.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:32.943+0000 7f7a6cc00700 1 -- 172.21.15.83:0/3506004874 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a68005a20 con 0x7f7a7010f0e0 2024-04-24T08:49:34.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:32.943+0000 7f7a75800700 1 -- 172.21.15.83:0/3506004874 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a701ac1e0 con 0x7f7a7010f0e0 2024-04-24T08:49:34.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:32.944+0000 7f7a75800700 1 -- 172.21.15.83:0/3506004874 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7a70063280 con 0x7f7a7010f0e0 2024-04-24T08:49:34.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:32.946+0000 7f7a6cc00700 1 -- 172.21.15.83:0/3506004874 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f7a6801a020 con 0x7f7a7010f0e0 2024-04-24T08:49:34.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:32.947+0000 7f7a6cc00700 1 --2- 172.21.15.83:0/3506004874 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f7a540a3d50 0x7f7a540a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:34.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:32.947+0000 7f7a6cc00700 1 -- 172.21.15.83:0/3506004874 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f7a680d0690 con 0x7f7a7010f0e0 2024-04-24T08:49:34.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:32.948+0000 7f7a6fe00700 1 --2- 172.21.15.83:0/3506004874 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f7a540a3d50 0x7f7a540a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:34.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:32.948+0000 7f7a6cc00700 1 --2- 172.21.15.83:0/3506004874 >> [v2:172.21.15.189:6824/2998782690,v1:172.21.15.189:6825/2998782690] conn(0x7f7a540aea10 0x7f7a540b0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:34.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:32.948+0000 7f7a6cc00700 1 -- 172.21.15.83:0/3506004874 --> [v2:172.21.15.189:6824/2998782690,v1:172.21.15.189:6825/2998782690] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7a68009390 con 0x7f7a540aea10 2024-04-24T08:49:34.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:32.948+0000 7f7a6cc00700 1 -- 172.21.15.83:0/3506004874 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f7a680d0a40 con 0x7f7a7010f0e0 2024-04-24T08:49:34.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:32.948+0000 7f7a6fe00700 1 --2- 172.21.15.83:0/3506004874 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f7a540a3d50 0x7f7a540a6210 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7a60005fd0 tx=0x7f7a6000af20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:49:34.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:32.948+0000 7f7a6f400700 1 --2- 172.21.15.83:0/3506004874 >> [v2:172.21.15.189:6824/2998782690,v1:172.21.15.189:6825/2998782690] conn(0x7f7a540aea10 0x7f7a540b0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:34.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:32.949+0000 7f7a6f400700 1 --2- 172.21.15.83:0/3506004874 >> [v2:172.21.15.189:6824/2998782690,v1:172.21.15.189:6825/2998782690] conn(0x7f7a540aea10 0x7f7a540b0e50 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:49:34.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:32.952+0000 7f7a6cc00700 1 -- 172.21.15.83:0/3506004874 <== osd.11 v2:172.21.15.189:6824/2998782690 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7a68009390 con 0x7f7a540aea10 2024-04-24T08:49:34.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:32.970+0000 7f7a75800700 1 -- 172.21.15.83:0/3506004874 --> [v2:172.21.15.189:6824/2998782690,v1:172.21.15.189:6825/2998782690] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7a70063700 con 0x7f7a540aea10 2024-04-24T08:49:34.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:32.971+0000 7f7a6cc00700 1 -- 172.21.15.83:0/3506004874 <== osd.11 v2:172.21.15.189:6824/2998782690 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f7a70063700 con 0x7f7a540aea10 2024-04-24T08:49:34.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:32.972+0000 7f7a75800700 1 -- 172.21.15.83:0/3506004874 >> [v2:172.21.15.189:6824/2998782690,v1:172.21.15.189:6825/2998782690] conn(0x7f7a540aea10 msgr2=0x7f7a540b0e50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:34.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:32.972+0000 7f7a75800700 1 --2- 172.21.15.83:0/3506004874 >> [v2:172.21.15.189:6824/2998782690,v1:172.21.15.189:6825/2998782690] conn(0x7f7a540aea10 0x7f7a540b0e50 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:34.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:32.972+0000 7f7a75800700 1 -- 172.21.15.83:0/3506004874 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f7a540a3d50 msgr2=0x7f7a540a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:34.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:32.972+0000 7f7a75800700 1 --2- 172.21.15.83:0/3506004874 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f7a540a3d50 0x7f7a540a6210 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7a60005fd0 tx=0x7f7a6000af20 comp rx=0 tx=0).stop 2024-04-24T08:49:34.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:32.972+0000 7f7a75800700 1 -- 172.21.15.83:0/3506004874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a7010f0e0 msgr2=0x7f7a701a3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:34.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:32.972+0000 7f7a75800700 1 --2- 172.21.15.83:0/3506004874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a7010f0e0 0x7f7a701a3f30 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f7a68007ee0 tx=0x7f7a6800d3b0 comp rx=0 tx=0).stop 2024-04-24T08:49:34.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:32.972+0000 7f7a74e00700 1 -- 172.21.15.83:0/3506004874 reap_dead start 2024-04-24T08:49:34.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:32.973+0000 7f7a75800700 1 -- 172.21.15.83:0/3506004874 shutdown_connections 2024-04-24T08:49:34.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:32.973+0000 7f7a75800700 1 -- 172.21.15.83:0/3506004874 >> 172.21.15.83:0/3506004874 conn(0x7f7a70078880 msgr2=0x7f7a70079aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:49:34.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:32.973+0000 7f7a75800700 1 -- 172.21.15.83:0/3506004874 shutdown_connections 2024-04-24T08:49:34.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:32.973+0000 7f7a75800700 1 -- 172.21.15.83:0/3506004874 wait complete. 2024-04-24T08:49:35.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:49:35 smithi083 ceph-mon[27907]: pgmap v406: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 341 B/s wr, 11 op/s 2024-04-24T08:49:35.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:49:35 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:49:35.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:49:35 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:49:35.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:49:35 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:49:35.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:49:35 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:49:35.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:49:35 smithi138 ceph-mon[31441]: pgmap v406: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 341 B/s wr, 11 op/s 2024-04-24T08:49:35.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:49:35 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:49:35.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:49:35 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:49:35.706 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:49:35 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:49:35.706 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:49:35 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:49:35.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:49:35 smithi189 ceph-mon[31791]: pgmap v406: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 341 B/s wr, 11 op/s 2024-04-24T08:49:35.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:49:35 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:49:35.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:49:35 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:49:35.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:49:35 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:49:35.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:49:35 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:49:38.150 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:49:37 smithi083 ceph-mon[27907]: pgmap v407: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-24T08:49:38.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:49:37 smithi138 ceph-mon[31441]: pgmap v407: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-24T08:49:38.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:49:37 smithi189 ceph-mon[31791]: pgmap v407: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-24T08:49:38.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:38.526+0000 7fb865200700 1 -- 172.21.15.83:0/370627546 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb860106e30 msgr2=0x7fb8601072e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:38.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:38.526+0000 7fb865200700 1 --2- 172.21.15.83:0/370627546 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb860106e30 0x7fb8601072e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fb85800b0a0 tx=0x7fb85800b3b0 comp rx=0 tx=0).stop 2024-04-24T08:49:38.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:38.526+0000 7fb865200700 1 -- 172.21.15.83:0/370627546 shutdown_connections 2024-04-24T08:49:38.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:38.526+0000 7fb865200700 1 --2- 172.21.15.83:0/370627546 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb860106e30 0x7fb8601072e0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:38.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:38.526+0000 7fb865200700 1 --2- 172.21.15.83:0/370627546 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb860101a20 0x7fb860101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:38.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:38.526+0000 7fb865200700 1 --2- 172.21.15.83:0/370627546 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb860102340 0x7fb860106810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:38.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:38.526+0000 7fb865200700 1 -- 172.21.15.83:0/370627546 >> 172.21.15.83:0/370627546 conn(0x7fb8600fd480 msgr2=0x7fb8600ff8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:49:38.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:38.526+0000 7fb865200700 1 -- 172.21.15.83:0/370627546 shutdown_connections 2024-04-24T08:49:38.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:38.526+0000 7fb865200700 1 -- 172.21.15.83:0/370627546 wait complete. 2024-04-24T08:49:38.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:38.527+0000 7fb865200700 1 Processor -- start 2024-04-24T08:49:38.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:38.527+0000 7fb865200700 1 -- start start 2024-04-24T08:49:38.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:38.528+0000 7fb865200700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb860101a20 0x7fb86019f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:38.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:38.528+0000 7fb865200700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb860102340 0x7fb86019fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:38.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:38.528+0000 7fb865200700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb860106e30 0x7fb8601a3f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:38.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:38.528+0000 7fb865200700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb8601192b0 con 0x7fb860106e30 2024-04-24T08:49:38.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:38.528+0000 7fb865200700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb860119130 con 0x7fb860101a20 2024-04-24T08:49:38.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:38.528+0000 7fb865200700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb860119430 con 0x7fb860102340 2024-04-24T08:49:38.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:38.528+0000 7fb85fe00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb860106e30 0x7fb8601a3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:38.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:38.528+0000 7fb85fe00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb860106e30 0x7fb8601a3f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60480/0 (socket says 172.21.15.83:60480) 2024-04-24T08:49:38.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:38.528+0000 7fb85fe00700 1 -- 172.21.15.83:0/3717745153 learned_addr learned my addr 172.21.15.83:0/3717745153 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:49:38.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:38.528+0000 7fb85f400700 1 --2- 172.21.15.83:0/3717745153 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb860101a20 0x7fb86019f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:38.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:38.529+0000 7fb85ea00700 1 --2- 172.21.15.83:0/3717745153 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb860102340 0x7fb86019fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:38.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:38.529+0000 7fb85fe00700 1 -- 172.21.15.83:0/3717745153 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb860102340 msgr2=0x7fb86019fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:38.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:38.529+0000 7fb85fe00700 1 --2- 172.21.15.83:0/3717745153 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb860102340 0x7fb86019fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:38.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:38.529+0000 7fb85fe00700 1 -- 172.21.15.83:0/3717745153 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb860101a20 msgr2=0x7fb86019f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:38.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:38.529+0000 7fb85fe00700 1 --2- 172.21.15.83:0/3717745153 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb860101a20 0x7fb86019f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:38.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:38.529+0000 7fb85fe00700 1 -- 172.21.15.83:0/3717745153 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8601a4650 con 0x7fb860106e30 2024-04-24T08:49:38.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:38.529+0000 7fb85ea00700 1 --2- 172.21.15.83:0/3717745153 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb860102340 0x7fb86019fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:49:38.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:38.529+0000 7fb85fe00700 1 --2- 172.21.15.83:0/3717745153 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb860106e30 0x7fb8601a3f10 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fb85800b070 tx=0x7fb85800cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:49:38.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:38.529+0000 7fb857e00700 1 -- 172.21.15.83:0/3717745153 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb858021070 con 0x7fb860106e30 2024-04-24T08:49:38.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:38.529+0000 7fb857e00700 1 -- 172.21.15.83:0/3717745153 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb858007cf0 con 0x7fb860106e30 2024-04-24T08:49:38.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:38.529+0000 7fb857e00700 1 -- 172.21.15.83:0/3717745153 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb858005440 con 0x7fb860106e30 2024-04-24T08:49:38.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:38.529+0000 7fb865200700 1 -- 172.21.15.83:0/3717745153 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8601a48e0 con 0x7fb860106e30 2024-04-24T08:49:38.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:38.530+0000 7fb865200700 1 -- 172.21.15.83:0/3717745153 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8601ac1c0 con 0x7fb860106e30 2024-04-24T08:49:38.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:38.530+0000 7fb865200700 1 -- 172.21.15.83:0/3717745153 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb860106190 con 0x7fb860106e30 2024-04-24T08:49:38.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:38.533+0000 7fb857e00700 1 -- 172.21.15.83:0/3717745153 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fb858024070 con 0x7fb860106e30 2024-04-24T08:49:38.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:38.534+0000 7fb857e00700 1 --2- 172.21.15.83:0/3717745153 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fb8440a3d50 0x7fb8440a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:38.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:38.534+0000 7fb857e00700 1 -- 172.21.15.83:0/3717745153 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fb85800f020 con 0x7fb860106e30 2024-04-24T08:49:38.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:38.534+0000 7fb85f400700 1 --2- 172.21.15.83:0/3717745153 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fb8440a3d50 0x7fb8440a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:38.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:38.535+0000 7fb857e00700 1 --2- 172.21.15.83:0/3717745153 >> [v2:172.21.15.83:6802/2132675555,v1:172.21.15.83:6803/2132675555] conn(0x7fb8440ae8e0 0x7fb8440b0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:38.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:38.535+0000 7fb857e00700 1 -- 172.21.15.83:0/3717745153 --> [v2:172.21.15.83:6802/2132675555,v1:172.21.15.83:6803/2132675555] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb85800cbd0 con 0x7fb8440ae8e0 2024-04-24T08:49:38.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:38.535+0000 7fb857e00700 1 -- 172.21.15.83:0/3717745153 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7fb8580d2570 con 0x7fb860106e30 2024-04-24T08:49:38.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:38.535+0000 7fb85ea00700 1 --2- 172.21.15.83:0/3717745153 >> [v2:172.21.15.83:6802/2132675555,v1:172.21.15.83:6803/2132675555] conn(0x7fb8440ae8e0 0x7fb8440b0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:38.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:38.535+0000 7fb85f400700 1 --2- 172.21.15.83:0/3717745153 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fb8440a3d50 0x7fb8440a6210 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb850005ea0 tx=0x7fb850005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:49:38.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:38.536+0000 7fb85ea00700 1 --2- 172.21.15.83:0/3717745153 >> [v2:172.21.15.83:6802/2132675555,v1:172.21.15.83:6803/2132675555] conn(0x7fb8440ae8e0 0x7fb8440b0d20 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:49:38.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:38.537+0000 7fb857e00700 1 -- 172.21.15.83:0/3717745153 <== osd.0 v2:172.21.15.83:6802/2132675555 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb85800cbd0 con 0x7fb8440ae8e0 2024-04-24T08:49:38.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:38.555+0000 7fb865200700 1 -- 172.21.15.83:0/3717745153 --> [v2:172.21.15.83:6802/2132675555,v1:172.21.15.83:6803/2132675555] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb86006b940 con 0x7fb8440ae8e0 2024-04-24T08:49:38.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:38.556+0000 7fb857e00700 1 -- 172.21.15.83:0/3717745153 <== osd.0 v2:172.21.15.83:6802/2132675555 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fb86006b940 con 0x7fb8440ae8e0 2024-04-24T08:49:38.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:38.556+0000 7fb865200700 1 -- 172.21.15.83:0/3717745153 >> [v2:172.21.15.83:6802/2132675555,v1:172.21.15.83:6803/2132675555] conn(0x7fb8440ae8e0 msgr2=0x7fb8440b0d20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:38.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:38.556+0000 7fb865200700 1 --2- 172.21.15.83:0/3717745153 >> [v2:172.21.15.83:6802/2132675555,v1:172.21.15.83:6803/2132675555] conn(0x7fb8440ae8e0 0x7fb8440b0d20 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:38.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:38.557+0000 7fb865200700 1 -- 172.21.15.83:0/3717745153 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fb8440a3d50 msgr2=0x7fb8440a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:38.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:38.557+0000 7fb865200700 1 --2- 172.21.15.83:0/3717745153 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fb8440a3d50 0x7fb8440a6210 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb850005ea0 tx=0x7fb850005e30 comp rx=0 tx=0).stop 2024-04-24T08:49:38.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:38.557+0000 7fb865200700 1 -- 172.21.15.83:0/3717745153 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb860106e30 msgr2=0x7fb8601a3f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:38.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:38.557+0000 7fb865200700 1 --2- 172.21.15.83:0/3717745153 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb860106e30 0x7fb8601a3f10 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fb85800b070 tx=0x7fb85800cf00 comp rx=0 tx=0).stop 2024-04-24T08:49:38.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:38.557+0000 7fb85fe00700 1 -- 172.21.15.83:0/3717745153 reap_dead start 2024-04-24T08:49:38.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:38.557+0000 7fb865200700 1 -- 172.21.15.83:0/3717745153 shutdown_connections 2024-04-24T08:49:38.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:38.557+0000 7fb865200700 1 -- 172.21.15.83:0/3717745153 >> 172.21.15.83:0/3717745153 conn(0x7fb8600fd480 msgr2=0x7fb8601047e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:49:38.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:38.557+0000 7fb865200700 1 -- 172.21.15.83:0/3717745153 shutdown_connections 2024-04-24T08:49:38.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:38.557+0000 7fb865200700 1 -- 172.21.15.83:0/3717745153 wait complete. 2024-04-24T08:49:39.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:39.204+0000 7f4f0a400700 1 -- 172.21.15.83:0/4219434628 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f04109030 msgr2=0x7f4f041094b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:39.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:39.204+0000 7f4f0a400700 1 --2- 172.21.15.83:0/4219434628 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f04109030 0x7f4f041094b0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f4ef4009990 tx=0x7f4ef4009ca0 comp rx=0 tx=0).stop 2024-04-24T08:49:39.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:39.204+0000 7f4f0a400700 1 -- 172.21.15.83:0/4219434628 shutdown_connections 2024-04-24T08:49:39.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:39.204+0000 7f4f0a400700 1 --2- 172.21.15.83:0/4219434628 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4f041099f0 0x7f4f04115e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:39.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:39.204+0000 7f4f0a400700 1 --2- 172.21.15.83:0/4219434628 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4f0410f0e0 0x7f4f0410f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:39.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:39.204+0000 7f4f0a400700 1 --2- 172.21.15.83:0/4219434628 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f04109030 0x7f4f041094b0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:39.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:39.204+0000 7f4f0a400700 1 -- 172.21.15.83:0/4219434628 >> 172.21.15.83:0/4219434628 conn(0x7f4f04078880 msgr2=0x7f4f04107910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:49:39.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:39.204+0000 7f4f0a400700 1 -- 172.21.15.83:0/4219434628 shutdown_connections 2024-04-24T08:49:39.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:39.204+0000 7f4f0a400700 1 -- 172.21.15.83:0/4219434628 wait complete. 2024-04-24T08:49:39.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:39.205+0000 7f4f0a400700 1 Processor -- start 2024-04-24T08:49:39.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:39.205+0000 7f4f0a400700 1 -- start start 2024-04-24T08:49:39.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:39.206+0000 7f4f0a400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4f04109030 0x7f4f0419f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:39.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:39.206+0000 7f4f0a400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f041099f0 0x7f4f0419fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:39.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:39.206+0000 7f4f0a400700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4f0410f0e0 0x7f4f041a3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:39.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:39.206+0000 7f4f0a400700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4f041191f0 con 0x7f4f041099f0 2024-04-24T08:49:39.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:39.206+0000 7f4f0a400700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4f04119070 con 0x7f4f0410f0e0 2024-04-24T08:49:39.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:39.206+0000 7f4f0a400700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4f04119370 con 0x7f4f04109030 2024-04-24T08:49:39.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:39.206+0000 7f4f03e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f041099f0 0x7f4f0419fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:39.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:39.206+0000 7f4f03e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f041099f0 0x7f4f0419fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55312/0 (socket says 172.21.15.83:55312) 2024-04-24T08:49:39.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:39.206+0000 7f4f03e00700 1 -- 172.21.15.83:0/3228862582 learned_addr learned my addr 172.21.15.83:0/3228862582 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:49:39.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:39.206+0000 7f4f09000700 1 --2- 172.21.15.83:0/3228862582 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4f04109030 0x7f4f0419f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:39.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:39.206+0000 7f4f09a00700 1 --2- 172.21.15.83:0/3228862582 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4f0410f0e0 0x7f4f041a3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:39.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:39.207+0000 7f4f03e00700 1 -- 172.21.15.83:0/3228862582 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4f04109030 msgr2=0x7f4f0419f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:39.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:39.207+0000 7f4f03e00700 1 --2- 172.21.15.83:0/3228862582 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4f04109030 0x7f4f0419f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:39.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:39.207+0000 7f4f03e00700 1 -- 172.21.15.83:0/3228862582 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4f0410f0e0 msgr2=0x7f4f041a3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:39.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:39.207+0000 7f4f03e00700 1 --2- 172.21.15.83:0/3228862582 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4f0410f0e0 0x7f4f041a3ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:39.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:39.207+0000 7f4f03e00700 1 -- 172.21.15.83:0/3228862582 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f041a4730 con 0x7f4f041099f0 2024-04-24T08:49:39.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:39.207+0000 7f4f09000700 1 --2- 172.21.15.83:0/3228862582 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4f04109030 0x7f4f0419f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:49:39.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:39.207+0000 7f4f09a00700 1 --2- 172.21.15.83:0/3228862582 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4f0410f0e0 0x7f4f041a3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:49:39.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:39.207+0000 7f4f03e00700 1 --2- 172.21.15.83:0/3228862582 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f041099f0 0x7f4f0419fc60 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f4ef4000c00 tx=0x7f4ef40036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:49:39.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:39.207+0000 7f4f01600700 1 -- 172.21.15.83:0/3228862582 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef4027070 con 0x7f4f041099f0 2024-04-24T08:49:39.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:39.207+0000 7f4f0a400700 1 -- 172.21.15.83:0/3228862582 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f041a49c0 con 0x7f4f041099f0 2024-04-24T08:49:39.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:39.207+0000 7f4f01600700 1 -- 172.21.15.83:0/3228862582 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4ef4004090 con 0x7f4f041099f0 2024-04-24T08:49:39.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:39.207+0000 7f4f01600700 1 -- 172.21.15.83:0/3228862582 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef4022de0 con 0x7f4f041099f0 2024-04-24T08:49:39.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:39.207+0000 7f4f0a400700 1 -- 172.21.15.83:0/3228862582 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f041ac2a0 con 0x7f4f041099f0 2024-04-24T08:49:39.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:39.208+0000 7f4f0a400700 1 -- 172.21.15.83:0/3228862582 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4f04063200 con 0x7f4f041099f0 2024-04-24T08:49:39.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:39.211+0000 7f4f01600700 1 -- 172.21.15.83:0/3228862582 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f4ef402a070 con 0x7f4f041099f0 2024-04-24T08:49:39.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:39.212+0000 7f4f01600700 1 --2- 172.21.15.83:0/3228862582 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f4eec0a3d50 0x7f4eec0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:39.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:39.212+0000 7f4f01600700 1 -- 172.21.15.83:0/3228862582 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f4ef40d2140 con 0x7f4f041099f0 2024-04-24T08:49:39.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:39.212+0000 7f4f09000700 1 --2- 172.21.15.83:0/3228862582 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f4eec0a3d50 0x7f4eec0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:39.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:39.212+0000 7f4f01600700 1 --2- 172.21.15.83:0/3228862582 >> [v2:172.21.15.138:6824/1130872098,v1:172.21.15.138:6825/1130872098] conn(0x7f4eec0aea10 0x7f4eec0b0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:39.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:39.212+0000 7f4f01600700 1 -- 172.21.15.83:0/3228862582 --> [v2:172.21.15.138:6824/1130872098,v1:172.21.15.138:6825/1130872098] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4eec0b1520 con 0x7f4eec0aea10 2024-04-24T08:49:39.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:39.212+0000 7f4f01600700 1 -- 172.21.15.83:0/3228862582 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f4ef40d24f0 con 0x7f4f041099f0 2024-04-24T08:49:39.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:39.213+0000 7f4f09000700 1 --2- 172.21.15.83:0/3228862582 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f4eec0a3d50 0x7f4eec0a6210 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4ef8005fd0 tx=0x7f4ef8005d40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:49:39.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:39.213+0000 7f4f09a00700 1 --2- 172.21.15.83:0/3228862582 >> [v2:172.21.15.138:6824/1130872098,v1:172.21.15.138:6825/1130872098] conn(0x7f4eec0aea10 0x7f4eec0b0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:39.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:39.214+0000 7f4f09a00700 1 --2- 172.21.15.83:0/3228862582 >> [v2:172.21.15.138:6824/1130872098,v1:172.21.15.138:6825/1130872098] conn(0x7f4eec0aea10 0x7f4eec0b0e50 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:49:39.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:39.217+0000 7f4f01600700 1 -- 172.21.15.83:0/3228862582 <== osd.10 v2:172.21.15.138:6824/1130872098 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4eec0b1520 con 0x7f4eec0aea10 2024-04-24T08:49:39.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:39.236+0000 7f4f0a400700 1 -- 172.21.15.83:0/3228862582 --> [v2:172.21.15.138:6824/1130872098,v1:172.21.15.138:6825/1130872098] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4f04063600 con 0x7f4eec0aea10 2024-04-24T08:49:39.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:39.237+0000 7f4f01600700 1 -- 172.21.15.83:0/3228862582 <== osd.10 v2:172.21.15.138:6824/1130872098 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f4f04063600 con 0x7f4eec0aea10 2024-04-24T08:49:39.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:39.238+0000 7f4f0a400700 1 -- 172.21.15.83:0/3228862582 >> [v2:172.21.15.138:6824/1130872098,v1:172.21.15.138:6825/1130872098] conn(0x7f4eec0aea10 msgr2=0x7f4eec0b0e50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:39.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:39.238+0000 7f4f0a400700 1 --2- 172.21.15.83:0/3228862582 >> [v2:172.21.15.138:6824/1130872098,v1:172.21.15.138:6825/1130872098] conn(0x7f4eec0aea10 0x7f4eec0b0e50 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:39.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:39.238+0000 7f4f0a400700 1 -- 172.21.15.83:0/3228862582 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f4eec0a3d50 msgr2=0x7f4eec0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:39.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:39.238+0000 7f4f0a400700 1 --2- 172.21.15.83:0/3228862582 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f4eec0a3d50 0x7f4eec0a6210 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4ef8005fd0 tx=0x7f4ef8005d40 comp rx=0 tx=0).stop 2024-04-24T08:49:39.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:39.238+0000 7f4f0a400700 1 -- 172.21.15.83:0/3228862582 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f041099f0 msgr2=0x7f4f0419fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:39.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:39.238+0000 7f4f0a400700 1 --2- 172.21.15.83:0/3228862582 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f041099f0 0x7f4f0419fc60 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f4ef4000c00 tx=0x7f4ef40036f0 comp rx=0 tx=0).stop 2024-04-24T08:49:39.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:39.238+0000 7f4f09a00700 1 -- 172.21.15.83:0/3228862582 reap_dead start 2024-04-24T08:49:39.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:39.238+0000 7f4f0a400700 1 -- 172.21.15.83:0/3228862582 shutdown_connections 2024-04-24T08:49:39.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:39.238+0000 7f4f0a400700 1 -- 172.21.15.83:0/3228862582 >> 172.21.15.83:0/3228862582 conn(0x7f4f04078880 msgr2=0x7f4f04079aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:49:39.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:39.238+0000 7f4f0a400700 1 -- 172.21.15.83:0/3228862582 shutdown_connections 2024-04-24T08:49:39.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:39.238+0000 7f4f0a400700 1 -- 172.21.15.83:0/3228862582 wait complete. 2024-04-24T08:49:39.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:35.985+0000 7f2c2b200700 1 -- 172.21.15.83:0/2426018438 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c2c109f40 msgr2=0x7f2c2c111b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:39.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:35.985+0000 7f2c2b200700 1 --2- 172.21.15.83:0/2426018438 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c2c109f40 0x7f2c2c111b10 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f2c2400b0d0 tx=0x7f2c2400b3e0 comp rx=0 tx=0).stop 2024-04-24T08:49:39.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:35.985+0000 7f2c2b200700 1 -- 172.21.15.83:0/2426018438 shutdown_connections 2024-04-24T08:49:39.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:35.985+0000 7f2c2b200700 1 --2- 172.21.15.83:0/2426018438 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c2c109f40 0x7f2c2c111b10 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:39.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:35.985+0000 7f2c2b200700 1 --2- 172.21.15.83:0/2426018438 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2c2c104e60 0x7f2c2c105240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:39.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:35.985+0000 7f2c2b200700 1 --2- 172.21.15.83:0/2426018438 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c2c105810 0x7f2c2c109860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:39.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:35.985+0000 7f2c2b200700 1 -- 172.21.15.83:0/2426018438 >> 172.21.15.83:0/2426018438 conn(0x7f2c2c1006d0 msgr2=0x7f2c2c102af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:49:39.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:35.985+0000 7f2c2b200700 1 -- 172.21.15.83:0/2426018438 shutdown_connections 2024-04-24T08:49:39.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:35.985+0000 7f2c2b200700 1 -- 172.21.15.83:0/2426018438 wait complete. 2024-04-24T08:49:39.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:35.987+0000 7f2c2b200700 1 Processor -- start 2024-04-24T08:49:39.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:35.987+0000 7f2c2b200700 1 -- start start 2024-04-24T08:49:39.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:35.987+0000 7f2c2b200700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2c2c104e60 0x7f2c2c19b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:39.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:35.987+0000 7f2c2b200700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c2c105810 0x7f2c2c19b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:39.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:35.987+0000 7f2c2b200700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c2c109f40 0x7f2c2c19fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:39.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:35.987+0000 7f2c2b200700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2c2c114dd0 con 0x7f2c2c109f40 2024-04-24T08:49:39.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:35.987+0000 7f2c2b200700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2c2c114c50 con 0x7f2c2c105810 2024-04-24T08:49:39.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:35.987+0000 7f2c2b200700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2c2c114f50 con 0x7f2c2c104e60 2024-04-24T08:49:39.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:35.988+0000 7f2c2a800700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c2c109f40 0x7f2c2c19fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:39.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:35.988+0000 7f2c2a800700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c2c109f40 0x7f2c2c19fb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60470/0 (socket says 172.21.15.83:60470) 2024-04-24T08:49:39.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:35.988+0000 7f2c2a800700 1 -- 172.21.15.83:0/1970545458 learned_addr learned my addr 172.21.15.83:0/1970545458 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:49:39.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:35.988+0000 7f2c29e00700 1 --2- 172.21.15.83:0/1970545458 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2c2c104e60 0x7f2c2c19b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:39.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:35.988+0000 7f2c29400700 1 --2- 172.21.15.83:0/1970545458 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c2c105810 0x7f2c2c19b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:39.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:35.988+0000 7f2c2a800700 1 -- 172.21.15.83:0/1970545458 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2c2c104e60 msgr2=0x7f2c2c19b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:39.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:35.988+0000 7f2c2a800700 1 --2- 172.21.15.83:0/1970545458 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2c2c104e60 0x7f2c2c19b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:39.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:35.988+0000 7f2c2a800700 1 -- 172.21.15.83:0/1970545458 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c2c105810 msgr2=0x7f2c2c19b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:39.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:35.988+0000 7f2c2a800700 1 --2- 172.21.15.83:0/1970545458 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c2c105810 0x7f2c2c19b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:39.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:35.988+0000 7f2c2a800700 1 -- 172.21.15.83:0/1970545458 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c2c1a0250 con 0x7f2c2c109f40 2024-04-24T08:49:39.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:35.988+0000 7f2c2a800700 1 --2- 172.21.15.83:0/1970545458 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c2c109f40 0x7f2c2c19fb10 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f2c24009850 tx=0x7f2c24007ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:49:39.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:35.989+0000 7f2c22a00700 1 -- 172.21.15.83:0/1970545458 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c2401f070 con 0x7f2c2c109f40 2024-04-24T08:49:39.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:35.989+0000 7f2c22a00700 1 -- 172.21.15.83:0/1970545458 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2c24005520 con 0x7f2c2c109f40 2024-04-24T08:49:39.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:35.989+0000 7f2c22a00700 1 -- 172.21.15.83:0/1970545458 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c2401b7d0 con 0x7f2c2c109f40 2024-04-24T08:49:39.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:35.989+0000 7f2c2b200700 1 -- 172.21.15.83:0/1970545458 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c2c1a04e0 con 0x7f2c2c109f40 2024-04-24T08:49:39.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:35.989+0000 7f2c2b200700 1 -- 172.21.15.83:0/1970545458 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c2c1a7dc0 con 0x7f2c2c109f40 2024-04-24T08:49:39.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:35.990+0000 7f2c2b200700 1 -- 172.21.15.83:0/1970545458 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2c2c1a0970 con 0x7f2c2c109f40 2024-04-24T08:49:39.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:35.992+0000 7f2c22a00700 1 -- 172.21.15.83:0/1970545458 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f2c24022070 con 0x7f2c2c109f40 2024-04-24T08:49:39.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:35.993+0000 7f2c22a00700 1 --2- 172.21.15.83:0/1970545458 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f2c100a3dd0 0x7f2c100a6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:39.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:35.994+0000 7f2c22a00700 1 -- 172.21.15.83:0/1970545458 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f2c2400f020 con 0x7f2c2c109f40 2024-04-24T08:49:39.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:35.994+0000 7f2c29e00700 1 --2- 172.21.15.83:0/1970545458 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f2c100a3dd0 0x7f2c100a6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:39.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:35.994+0000 7f2c22a00700 1 --2- 172.21.15.83:0/1970545458 >> [v2:172.21.15.138:6800/3589158880,v1:172.21.15.138:6801/3589158880] conn(0x7f2c100aea90 0x7f2c100b0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:39.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:35.994+0000 7f2c22a00700 1 -- 172.21.15.83:0/1970545458 --> [v2:172.21.15.138:6800/3589158880,v1:172.21.15.138:6801/3589158880] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2c24010ec0 con 0x7f2c100aea90 2024-04-24T08:49:39.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:35.994+0000 7f2c22a00700 1 -- 172.21.15.83:0/1970545458 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f2c240d2030 con 0x7f2c2c109f40 2024-04-24T08:49:39.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:35.994+0000 7f2c29e00700 1 --2- 172.21.15.83:0/1970545458 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f2c100a3dd0 0x7f2c100a6290 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2c1c005950 tx=0x7f2c1c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:49:39.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:35.995+0000 7f2c29400700 1 --2- 172.21.15.83:0/1970545458 >> [v2:172.21.15.138:6800/3589158880,v1:172.21.15.138:6801/3589158880] conn(0x7f2c100aea90 0x7f2c100b0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:39.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:35.996+0000 7f2c29400700 1 --2- 172.21.15.83:0/1970545458 >> [v2:172.21.15.138:6800/3589158880,v1:172.21.15.138:6801/3589158880] conn(0x7f2c100aea90 0x7f2c100b0ed0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:49:39.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:35.998+0000 7f2c22a00700 1 -- 172.21.15.83:0/1970545458 <== osd.1 v2:172.21.15.138:6800/3589158880 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2c24010ec0 con 0x7f2c100aea90 2024-04-24T08:49:39.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:36.017+0000 7f2c2b200700 1 -- 172.21.15.83:0/1970545458 --> [v2:172.21.15.138:6800/3589158880,v1:172.21.15.138:6801/3589158880] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2c2c0635a0 con 0x7f2c100aea90 2024-04-24T08:49:39.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:36.018+0000 7f2c22a00700 1 -- 172.21.15.83:0/1970545458 <== osd.1 v2:172.21.15.138:6800/3589158880 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f2c2c0635a0 con 0x7f2c100aea90 2024-04-24T08:49:39.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:36.019+0000 7f2c2b200700 1 -- 172.21.15.83:0/1970545458 >> [v2:172.21.15.138:6800/3589158880,v1:172.21.15.138:6801/3589158880] conn(0x7f2c100aea90 msgr2=0x7f2c100b0ed0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:39.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:36.019+0000 7f2c2b200700 1 --2- 172.21.15.83:0/1970545458 >> [v2:172.21.15.138:6800/3589158880,v1:172.21.15.138:6801/3589158880] conn(0x7f2c100aea90 0x7f2c100b0ed0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:39.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:36.019+0000 7f2c2b200700 1 -- 172.21.15.83:0/1970545458 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f2c100a3dd0 msgr2=0x7f2c100a6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:39.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:36.019+0000 7f2c2b200700 1 --2- 172.21.15.83:0/1970545458 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f2c100a3dd0 0x7f2c100a6290 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2c1c005950 tx=0x7f2c1c0058e0 comp rx=0 tx=0).stop 2024-04-24T08:49:39.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:36.019+0000 7f2c2b200700 1 -- 172.21.15.83:0/1970545458 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c2c109f40 msgr2=0x7f2c2c19fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:39.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:36.019+0000 7f2c2b200700 1 --2- 172.21.15.83:0/1970545458 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c2c109f40 0x7f2c2c19fb10 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f2c24009850 tx=0x7f2c24007ad0 comp rx=0 tx=0).stop 2024-04-24T08:49:39.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:36.019+0000 7f2c2a800700 1 -- 172.21.15.83:0/1970545458 reap_dead start 2024-04-24T08:49:39.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:36.019+0000 7f2c2b200700 1 -- 172.21.15.83:0/1970545458 shutdown_connections 2024-04-24T08:49:39.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:36.019+0000 7f2c2b200700 1 -- 172.21.15.83:0/1970545458 >> 172.21.15.83:0/1970545458 conn(0x7f2c2c1006d0 msgr2=0x7f2c2c1029a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:49:39.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:36.019+0000 7f2c2b200700 1 -- 172.21.15.83:0/1970545458 shutdown_connections 2024-04-24T08:49:39.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:36.019+0000 7f2c2b200700 1 -- 172.21.15.83:0/1970545458 wait complete. 2024-04-24T08:49:40.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:49:39 smithi083 ceph-mon[27907]: pgmap v408: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-24T08:49:40.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:49:39 smithi138 ceph-mon[31441]: pgmap v408: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-24T08:49:40.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:49:39 smithi189 ceph-mon[31791]: pgmap v408: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-24T08:49:41.141 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:49:40 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2024-04-24T08:49:41.141 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:49:40 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]: dispatch 2024-04-24T08:49:41.142 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:49:40 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-04-24T08:49:41.142 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:49:40 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]: dispatch 2024-04-24T08:49:41.142 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:49:40 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]: dispatch 2024-04-24T08:49:41.142 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:49:40 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2a", "id": [3, 8]}]: dispatch 2024-04-24T08:49:41.142 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:49:40 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]: dispatch 2024-04-24T08:49:41.142 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:49:40 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [5, 8]}]: dispatch 2024-04-24T08:49:41.142 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:49:40 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]: dispatch 2024-04-24T08:49:41.142 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:49:40 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]: dispatch 2024-04-24T08:49:41.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:49:40 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2024-04-24T08:49:41.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:49:40 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]: dispatch 2024-04-24T08:49:41.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:49:40 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-04-24T08:49:41.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:49:40 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]: dispatch 2024-04-24T08:49:41.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:49:40 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]: dispatch 2024-04-24T08:49:41.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:49:40 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2a", "id": [3, 8]}]: dispatch 2024-04-24T08:49:41.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:49:40 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]: dispatch 2024-04-24T08:49:41.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:49:40 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [5, 8]}]: dispatch 2024-04-24T08:49:41.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:49:40 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]: dispatch 2024-04-24T08:49:41.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:49:40 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]: dispatch 2024-04-24T08:49:41.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:49:40 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2024-04-24T08:49:41.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:49:40 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]: dispatch 2024-04-24T08:49:41.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:49:40 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-04-24T08:49:41.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:49:40 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]: dispatch 2024-04-24T08:49:41.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:49:40 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]: dispatch 2024-04-24T08:49:41.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:49:40 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2a", "id": [3, 8]}]: dispatch 2024-04-24T08:49:41.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:49:40 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]: dispatch 2024-04-24T08:49:41.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:49:40 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [5, 8]}]: dispatch 2024-04-24T08:49:41.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:49:40 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]: dispatch 2024-04-24T08:49:41.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:49:40 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]: dispatch 2024-04-24T08:49:42.266 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:49:41 smithi083 ceph-mon[27907]: pgmap v409: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-04-24T08:49:42.266 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:49:41 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2024-04-24T08:49:42.266 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:49:41 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]': finished 2024-04-24T08:49:42.266 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:49:41 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-04-24T08:49:42.266 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:49:41 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]': finished 2024-04-24T08:49:42.266 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:49:41 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]': finished 2024-04-24T08:49:42.267 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:49:41 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2a", "id": [3, 8]}]': finished 2024-04-24T08:49:42.267 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:49:41 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]': finished 2024-04-24T08:49:42.267 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:49:41 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [5, 8]}]': finished 2024-04-24T08:49:42.267 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:49:41 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]': finished 2024-04-24T08:49:42.267 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:49:41 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]': finished 2024-04-24T08:49:42.267 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:49:41 smithi083 ceph-mon[27907]: osdmap e90: 12 total, 12 up, 12 in 2024-04-24T08:49:42.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:49:41 smithi138 ceph-mon[31441]: pgmap v409: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-04-24T08:49:42.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:49:41 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2024-04-24T08:49:42.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:49:41 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]': finished 2024-04-24T08:49:42.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:49:41 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-04-24T08:49:42.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:49:41 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]': finished 2024-04-24T08:49:42.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:49:41 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]': finished 2024-04-24T08:49:42.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:49:41 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2a", "id": [3, 8]}]': finished 2024-04-24T08:49:42.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:49:41 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]': finished 2024-04-24T08:49:42.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:49:41 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [5, 8]}]': finished 2024-04-24T08:49:42.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:49:41 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]': finished 2024-04-24T08:49:42.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:49:41 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]': finished 2024-04-24T08:49:42.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:49:41 smithi138 ceph-mon[31441]: osdmap e90: 12 total, 12 up, 12 in 2024-04-24T08:49:42.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:49:41 smithi189 ceph-mon[31791]: pgmap v409: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-04-24T08:49:42.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:49:41 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2024-04-24T08:49:42.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:49:41 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]': finished 2024-04-24T08:49:42.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:49:41 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-04-24T08:49:42.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:49:41 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]': finished 2024-04-24T08:49:42.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:49:41 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 8]}]': finished 2024-04-24T08:49:42.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:49:41 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2a", "id": [3, 8]}]': finished 2024-04-24T08:49:42.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:49:41 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]': finished 2024-04-24T08:49:42.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:49:41 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [5, 8]}]': finished 2024-04-24T08:49:42.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:49:41 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]': finished 2024-04-24T08:49:42.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:49:41 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]': finished 2024-04-24T08:49:42.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:49:41 smithi189 ceph-mon[31791]: osdmap e90: 12 total, 12 up, 12 in 2024-04-24T08:49:43.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:49:42 smithi083 ceph-mon[27907]: osdmap e91: 12 total, 12 up, 12 in 2024-04-24T08:49:43.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:49:42 smithi138 ceph-mon[31441]: osdmap e91: 12 total, 12 up, 12 in 2024-04-24T08:49:43.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:49:42 smithi189 ceph-mon[31791]: osdmap e91: 12 total, 12 up, 12 in 2024-04-24T08:49:44.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:49:43 smithi083 ceph-mon[27907]: pgmap v412: 129 pgs: 1 peering, 128 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 8 op/s 2024-04-24T08:49:44.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:49:43 smithi083 ceph-mon[27907]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-04-24T08:49:44.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:49:43 smithi138 ceph-mon[31441]: pgmap v412: 129 pgs: 1 peering, 128 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 8 op/s 2024-04-24T08:49:44.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:49:43 smithi138 ceph-mon[31441]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-04-24T08:49:44.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:49:43 smithi189 ceph-mon[31791]: pgmap v412: 129 pgs: 1 peering, 128 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 8 op/s 2024-04-24T08:49:44.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:49:43 smithi189 ceph-mon[31791]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-04-24T08:49:44.858 INFO:teuthology.orchestra.run.smithi083.stdout:55834574950 2024-04-24T08:49:44.859 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid bd3e039e-0215-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-04-24T08:49:45.704 INFO:teuthology.orchestra.run.smithi083.stdout:309237645350 2024-04-24T08:49:45.704 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid bd3e039e-0215-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-04-24T08:49:46.270 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:49:45 smithi083 ceph-mon[27907]: pgmap v413: 129 pgs: 1 peering, 128 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-24T08:49:46.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:49:45 smithi138 ceph-mon[31441]: pgmap v413: 129 pgs: 1 peering, 128 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-24T08:49:46.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:49:45 smithi189 ceph-mon[31791]: pgmap v413: 129 pgs: 1 peering, 128 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-24T08:49:46.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.319+0000 7f5872000700 1 -- 172.21.15.83:0/287492457 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f586c102b40 msgr2=0x7f586c102fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:46.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.319+0000 7f5872000700 1 --2- 172.21.15.83:0/287492457 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f586c102b40 0x7f586c102fc0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f5854009990 tx=0x7f5854009ca0 comp rx=0 tx=0).stop 2024-04-24T08:49:46.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.319+0000 7f5872000700 1 -- 172.21.15.83:0/287492457 shutdown_connections 2024-04-24T08:49:46.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.319+0000 7f5872000700 1 --2- 172.21.15.83:0/287492457 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f586c103500 0x7f586c10f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:46.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.319+0000 7f5872000700 1 --2- 172.21.15.83:0/287492457 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f586c108b90 0x7f586c108f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:46.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.319+0000 7f5872000700 1 --2- 172.21.15.83:0/287492457 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f586c102b40 0x7f586c102fc0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:46.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.319+0000 7f5872000700 1 -- 172.21.15.83:0/287492457 >> 172.21.15.83:0/287492457 conn(0x7f586c079c50 msgr2=0x7f586c07a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:49:46.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.319+0000 7f5872000700 1 -- 172.21.15.83:0/287492457 shutdown_connections 2024-04-24T08:49:46.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.320+0000 7f5872000700 1 -- 172.21.15.83:0/287492457 wait complete. 2024-04-24T08:49:46.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.321+0000 7f5872000700 1 Processor -- start 2024-04-24T08:49:46.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.321+0000 7f5872000700 1 -- start start 2024-04-24T08:49:46.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.322+0000 7f5872000700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f586c102b40 0x7f586c074df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:46.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.322+0000 7f5872000700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f586c103500 0x7f586c075330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:46.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.322+0000 7f5872000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f586c108b90 0x7f586c075870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:46.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.322+0000 7f5872000700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f586c112d10 con 0x7f586c102b40 2024-04-24T08:49:46.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.322+0000 7f5872000700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f586c112b90 con 0x7f586c103500 2024-04-24T08:49:46.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.322+0000 7f5872000700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f586c112e90 con 0x7f586c108b90 2024-04-24T08:49:46.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.322+0000 7f586b400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f586c102b40 0x7f586c074df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:46.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.322+0000 7f586b400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f586c102b40 0x7f586c074df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55380/0 (socket says 172.21.15.83:55380) 2024-04-24T08:49:46.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.322+0000 7f586b400700 1 -- 172.21.15.83:0/2242130800 learned_addr learned my addr 172.21.15.83:0/2242130800 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:49:46.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.322+0000 7f5863e00700 1 --2- 172.21.15.83:0/2242130800 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f586c103500 0x7f586c075330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:46.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.322+0000 7f586b400700 1 -- 172.21.15.83:0/2242130800 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f586c108b90 msgr2=0x7f586c075870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:46.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.322+0000 7f586be00700 1 --2- 172.21.15.83:0/2242130800 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f586c108b90 0x7f586c075870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:46.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.322+0000 7f586b400700 1 --2- 172.21.15.83:0/2242130800 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f586c108b90 0x7f586c075870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:46.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.322+0000 7f586b400700 1 -- 172.21.15.83:0/2242130800 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f586c103500 msgr2=0x7f586c075330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:46.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.322+0000 7f586b400700 1 --2- 172.21.15.83:0/2242130800 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f586c103500 0x7f586c075330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:46.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.322+0000 7f586b400700 1 -- 172.21.15.83:0/2242130800 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f586c1a77d0 con 0x7f586c102b40 2024-04-24T08:49:46.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.323+0000 7f586b400700 1 --2- 172.21.15.83:0/2242130800 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f586c102b40 0x7f586c074df0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f585c00e9c0 tx=0x7f585c00ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:49:46.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.323+0000 7f5868c00700 1 -- 172.21.15.83:0/2242130800 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f585c00cbd0 con 0x7f586c102b40 2024-04-24T08:49:46.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.323+0000 7f5872000700 1 -- 172.21.15.83:0/2242130800 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f586c1a7ac0 con 0x7f586c102b40 2024-04-24T08:49:46.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.323+0000 7f5868c00700 1 -- 172.21.15.83:0/2242130800 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f585c00cd70 con 0x7f586c102b40 2024-04-24T08:49:46.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.323+0000 7f5868c00700 1 -- 172.21.15.83:0/2242130800 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f585c010430 con 0x7f586c102b40 2024-04-24T08:49:46.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.323+0000 7f5872000700 1 -- 172.21.15.83:0/2242130800 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f586c1a8000 con 0x7f586c102b40 2024-04-24T08:49:46.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.325+0000 7f5872000700 1 -- 172.21.15.83:0/2242130800 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f586c063180 con 0x7f586c102b40 2024-04-24T08:49:46.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.327+0000 7f5868c00700 1 -- 172.21.15.83:0/2242130800 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f585c004750 con 0x7f586c102b40 2024-04-24T08:49:46.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.329+0000 7f5868c00700 1 --2- 172.21.15.83:0/2242130800 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f58500a3d50 0x7f58500a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:46.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.329+0000 7f5868c00700 1 -- 172.21.15.83:0/2242130800 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f585c0d0f00 con 0x7f586c102b40 2024-04-24T08:49:46.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.329+0000 7f5863e00700 1 --2- 172.21.15.83:0/2242130800 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f58500a3d50 0x7f58500a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:46.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.329+0000 7f5868c00700 1 --2- 172.21.15.83:0/2242130800 >> [v2:172.21.15.189:6816/207018554,v1:172.21.15.189:6817/207018554] conn(0x7f58500aeeb0 0x7f58500b1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:46.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.329+0000 7f5868c00700 1 -- 172.21.15.83:0/2242130800 --> [v2:172.21.15.189:6816/207018554,v1:172.21.15.189:6817/207018554] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f58500b1a20 con 0x7f58500aeeb0 2024-04-24T08:49:46.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.329+0000 7f5868c00700 1 -- 172.21.15.83:0/2242130800 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f585c0d12b0 con 0x7f586c102b40 2024-04-24T08:49:46.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.330+0000 7f5863e00700 1 --2- 172.21.15.83:0/2242130800 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f58500a3d50 0x7f58500a6210 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f586c07a390 tx=0x7f585401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:49:46.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.336+0000 7f586be00700 1 --2- 172.21.15.83:0/2242130800 >> [v2:172.21.15.189:6816/207018554,v1:172.21.15.189:6817/207018554] conn(0x7f58500aeeb0 0x7f58500b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:46.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.337+0000 7f586be00700 1 --2- 172.21.15.83:0/2242130800 >> [v2:172.21.15.189:6816/207018554,v1:172.21.15.189:6817/207018554] conn(0x7f58500aeeb0 0x7f58500b1350 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:49:46.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.340+0000 7f5868c00700 1 -- 172.21.15.83:0/2242130800 <== osd.8 v2:172.21.15.189:6816/207018554 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f58500b1a20 con 0x7f58500aeeb0 2024-04-24T08:49:46.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.358+0000 7f5872000700 1 -- 172.21.15.83:0/2242130800 --> [v2:172.21.15.189:6816/207018554,v1:172.21.15.189:6817/207018554] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f586c063600 con 0x7f58500aeeb0 2024-04-24T08:49:46.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.359+0000 7f5868c00700 1 -- 172.21.15.83:0/2242130800 <== osd.8 v2:172.21.15.189:6816/207018554 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f586c063600 con 0x7f58500aeeb0 2024-04-24T08:49:46.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.360+0000 7f5872000700 1 -- 172.21.15.83:0/2242130800 >> [v2:172.21.15.189:6816/207018554,v1:172.21.15.189:6817/207018554] conn(0x7f58500aeeb0 msgr2=0x7f58500b1350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:46.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.360+0000 7f5872000700 1 --2- 172.21.15.83:0/2242130800 >> [v2:172.21.15.189:6816/207018554,v1:172.21.15.189:6817/207018554] conn(0x7f58500aeeb0 0x7f58500b1350 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:46.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.360+0000 7f5872000700 1 -- 172.21.15.83:0/2242130800 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f58500a3d50 msgr2=0x7f58500a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:46.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.360+0000 7f5872000700 1 --2- 172.21.15.83:0/2242130800 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f58500a3d50 0x7f58500a6210 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f586c07a390 tx=0x7f585401a040 comp rx=0 tx=0).stop 2024-04-24T08:49:46.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.360+0000 7f5872000700 1 -- 172.21.15.83:0/2242130800 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f586c102b40 msgr2=0x7f586c074df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:46.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.360+0000 7f5872000700 1 --2- 172.21.15.83:0/2242130800 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f586c102b40 0x7f586c074df0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f585c00e9c0 tx=0x7f585c00ed80 comp rx=0 tx=0).stop 2024-04-24T08:49:46.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.360+0000 7f586be00700 1 -- 172.21.15.83:0/2242130800 reap_dead start 2024-04-24T08:49:46.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.360+0000 7f5872000700 1 -- 172.21.15.83:0/2242130800 shutdown_connections 2024-04-24T08:49:46.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.360+0000 7f5872000700 1 -- 172.21.15.83:0/2242130800 >> 172.21.15.83:0/2242130800 conn(0x7f586c079c50 msgr2=0x7f586c1019c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:49:46.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.360+0000 7f5872000700 1 -- 172.21.15.83:0/2242130800 shutdown_connections 2024-04-24T08:49:46.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.360+0000 7f5872000700 1 -- 172.21.15.83:0/2242130800 wait complete. 2024-04-24T08:49:47.050 INFO:teuthology.orchestra.run.smithi083.stdout:343597383709 2024-04-24T08:49:47.050 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid bd3e039e-0215-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-04-24T08:49:47.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:47.517+0000 7f4971c00700 1 -- 172.21.15.83:0/654610978 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f496c075640 msgr2=0x7f496c10efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:47.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:47.517+0000 7f4971c00700 1 --2- 172.21.15.83:0/654610978 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f496c075640 0x7f496c10efb0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f4958009990 tx=0x7f4958009ca0 comp rx=0 tx=0).stop 2024-04-24T08:49:47.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:47.517+0000 7f4971c00700 1 -- 172.21.15.83:0/654610978 shutdown_connections 2024-04-24T08:49:47.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:47.517+0000 7f4971c00700 1 --2- 172.21.15.83:0/654610978 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f496c10f4f0 0x7f496c1118e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:47.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:47.517+0000 7f4971c00700 1 --2- 172.21.15.83:0/654610978 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f496c074d20 0x7f496c075100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:47.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:47.517+0000 7f4971c00700 1 --2- 172.21.15.83:0/654610978 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f496c075640 0x7f496c10efb0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:47.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:47.517+0000 7f4971c00700 1 -- 172.21.15.83:0/654610978 >> 172.21.15.83:0/654610978 conn(0x7f496c079c50 msgr2=0x7f496c07a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:49:47.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:47.517+0000 7f4971c00700 1 -- 172.21.15.83:0/654610978 shutdown_connections 2024-04-24T08:49:47.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:47.517+0000 7f4971c00700 1 -- 172.21.15.83:0/654610978 wait complete. 2024-04-24T08:49:47.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:47.519+0000 7f4971c00700 1 Processor -- start 2024-04-24T08:49:47.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:47.519+0000 7f4971c00700 1 -- start start 2024-04-24T08:49:47.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:47.519+0000 7f4971c00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f496c074d20 0x7f496c19b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:47.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:47.519+0000 7f4971c00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f496c075640 0x7f496c19b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:47.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:47.519+0000 7f4971c00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f496c10f4f0 0x7f496c19fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:47.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:47.519+0000 7f4971c00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f496c114ee0 con 0x7f496c075640 2024-04-24T08:49:47.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:47.519+0000 7f4971c00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f496c114d60 con 0x7f496c10f4f0 2024-04-24T08:49:47.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:47.519+0000 7f4971c00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f496c115060 con 0x7f496c074d20 2024-04-24T08:49:47.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:47.520+0000 7f496a000700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f496c075640 0x7f496c19b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:47.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:47.520+0000 7f496a000700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f496c075640 0x7f496c19b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55422/0 (socket says 172.21.15.83:55422) 2024-04-24T08:49:47.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:47.520+0000 7f496a000700 1 -- 172.21.15.83:0/565225888 learned_addr learned my addr 172.21.15.83:0/565225888 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:49:47.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:47.520+0000 7f496aa00700 1 --2- 172.21.15.83:0/565225888 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f496c074d20 0x7f496c19b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:47.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:47.520+0000 7f496a000700 1 -- 172.21.15.83:0/565225888 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f496c074d20 msgr2=0x7f496c19b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:47.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:47.520+0000 7f496a000700 1 --2- 172.21.15.83:0/565225888 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f496c074d20 0x7f496c19b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:47.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:47.520+0000 7f496a000700 1 -- 172.21.15.83:0/565225888 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f496c10f4f0 msgr2=0x7f496c19fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:47.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:47.520+0000 7f496a000700 1 --2- 172.21.15.83:0/565225888 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f496c10f4f0 0x7f496c19fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:47.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:47.520+0000 7f496a000700 1 -- 172.21.15.83:0/565225888 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f496c1a03c0 con 0x7f496c075640 2024-04-24T08:49:47.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:47.520+0000 7f496aa00700 1 --2- 172.21.15.83:0/565225888 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f496c074d20 0x7f496c19b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:49:47.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:47.520+0000 7f496a000700 1 --2- 172.21.15.83:0/565225888 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f496c075640 0x7f496c19b8f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f4958005910 tx=0x7f4958009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:49:47.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:47.521+0000 7f4963400700 1 -- 172.21.15.83:0/565225888 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4958027070 con 0x7f496c075640 2024-04-24T08:49:47.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:47.521+0000 7f4971c00700 1 -- 172.21.15.83:0/565225888 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f496c1a0650 con 0x7f496c075640 2024-04-24T08:49:47.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:47.521+0000 7f4963400700 1 -- 172.21.15.83:0/565225888 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4958003f90 con 0x7f496c075640 2024-04-24T08:49:47.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:47.521+0000 7f4963400700 1 -- 172.21.15.83:0/565225888 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4958022d70 con 0x7f496c075640 2024-04-24T08:49:47.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:47.521+0000 7f4971c00700 1 -- 172.21.15.83:0/565225888 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f496c1a7f30 con 0x7f496c075640 2024-04-24T08:49:47.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:47.522+0000 7f4971c00700 1 -- 172.21.15.83:0/565225888 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f496c063280 con 0x7f496c075640 2024-04-24T08:49:47.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:47.524+0000 7f4963400700 1 -- 172.21.15.83:0/565225888 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f495802a070 con 0x7f496c075640 2024-04-24T08:49:47.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:47.526+0000 7f4963400700 1 --2- 172.21.15.83:0/565225888 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f49500a3d50 0x7f49500a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:47.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:47.526+0000 7f4963400700 1 -- 172.21.15.83:0/565225888 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f49580d19b0 con 0x7f496c075640 2024-04-24T08:49:47.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:47.526+0000 7f496aa00700 1 --2- 172.21.15.83:0/565225888 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f49500a3d50 0x7f49500a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:47.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:47.526+0000 7f4963400700 1 --2- 172.21.15.83:0/565225888 >> [v2:172.21.15.189:6800/3169072688,v1:172.21.15.189:6801/3169072688] conn(0x7f49500aeeb0 0x7f49500b1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:47.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:47.526+0000 7f4963400700 1 -- 172.21.15.83:0/565225888 --> [v2:172.21.15.189:6800/3169072688,v1:172.21.15.189:6801/3169072688] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f49500b1a20 con 0x7f49500aeeb0 2024-04-24T08:49:47.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:47.526+0000 7f4963400700 1 -- 172.21.15.83:0/565225888 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f49580d1d60 con 0x7f496c075640 2024-04-24T08:49:47.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:47.527+0000 7f496aa00700 1 --2- 172.21.15.83:0/565225888 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f49500a3d50 0x7f49500a6210 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f495c005fd0 tx=0x7f495c005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:49:47.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:47.527+0000 7f496b400700 1 --2- 172.21.15.83:0/565225888 >> [v2:172.21.15.189:6800/3169072688,v1:172.21.15.189:6801/3169072688] conn(0x7f49500aeeb0 0x7f49500b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:47.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:47.528+0000 7f496b400700 1 --2- 172.21.15.83:0/565225888 >> [v2:172.21.15.189:6800/3169072688,v1:172.21.15.189:6801/3169072688] conn(0x7f49500aeeb0 0x7f49500b1350 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:49:47.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:47.531+0000 7f4963400700 1 -- 172.21.15.83:0/565225888 <== osd.2 v2:172.21.15.189:6800/3169072688 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f49500b1a20 con 0x7f49500aeeb0 2024-04-24T08:49:47.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:47.548+0000 7f4971c00700 1 -- 172.21.15.83:0/565225888 --> [v2:172.21.15.189:6800/3169072688,v1:172.21.15.189:6801/3169072688] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f496c063680 con 0x7f49500aeeb0 2024-04-24T08:49:47.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:47.550+0000 7f4963400700 1 -- 172.21.15.83:0/565225888 <== osd.2 v2:172.21.15.189:6800/3169072688 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f496c063680 con 0x7f49500aeeb0 2024-04-24T08:49:47.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:47.550+0000 7f4971c00700 1 -- 172.21.15.83:0/565225888 >> [v2:172.21.15.189:6800/3169072688,v1:172.21.15.189:6801/3169072688] conn(0x7f49500aeeb0 msgr2=0x7f49500b1350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:47.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:47.550+0000 7f4971c00700 1 --2- 172.21.15.83:0/565225888 >> [v2:172.21.15.189:6800/3169072688,v1:172.21.15.189:6801/3169072688] conn(0x7f49500aeeb0 0x7f49500b1350 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:47.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:47.550+0000 7f4971c00700 1 -- 172.21.15.83:0/565225888 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f49500a3d50 msgr2=0x7f49500a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:47.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:47.550+0000 7f4971c00700 1 --2- 172.21.15.83:0/565225888 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f49500a3d50 0x7f49500a6210 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f495c005fd0 tx=0x7f495c005ee0 comp rx=0 tx=0).stop 2024-04-24T08:49:47.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:47.550+0000 7f4971c00700 1 -- 172.21.15.83:0/565225888 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f496c075640 msgr2=0x7f496c19b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:47.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:47.550+0000 7f4971c00700 1 --2- 172.21.15.83:0/565225888 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f496c075640 0x7f496c19b8f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f4958005910 tx=0x7f4958009f90 comp rx=0 tx=0).stop 2024-04-24T08:49:47.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:47.550+0000 7f496b400700 1 -- 172.21.15.83:0/565225888 reap_dead start 2024-04-24T08:49:47.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:47.551+0000 7f4971c00700 1 -- 172.21.15.83:0/565225888 shutdown_connections 2024-04-24T08:49:47.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:47.551+0000 7f4971c00700 1 -- 172.21.15.83:0/565225888 >> 172.21.15.83:0/565225888 conn(0x7f496c079c50 msgr2=0x7f496c0ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:49:47.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:47.551+0000 7f4971c00700 1 -- 172.21.15.83:0/565225888 shutdown_connections 2024-04-24T08:49:47.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:47.551+0000 7f4971c00700 1 -- 172.21.15.83:0/565225888 wait complete. 2024-04-24T08:49:48.165 INFO:teuthology.orchestra.run.smithi083.stdout:253403070517 2024-04-24T08:49:48.166 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid bd3e039e-0215-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-04-24T08:49:48.242 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:49:47 smithi083 ceph-mon[27907]: pgmap v414: 129 pgs: 1 peering, 128 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-24T08:49:48.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:49:47 smithi138 ceph-mon[31441]: pgmap v414: 129 pgs: 1 peering, 128 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-24T08:49:48.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:49:47 smithi189 ceph-mon[31791]: pgmap v414: 129 pgs: 1 peering, 128 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-24T08:49:48.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:44.654+0000 7f91dca00700 1 -- 172.21.15.83:0/1761197054 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f91d81033e0 msgr2=0x7f91d810f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:48.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:44.654+0000 7f91dca00700 1 --2- 172.21.15.83:0/1761197054 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f91d81033e0 0x7f91d810f8f0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f91d0009990 tx=0x7f91d0009ca0 comp rx=0 tx=0).stop 2024-04-24T08:49:48.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:44.654+0000 7f91dca00700 1 -- 172.21.15.83:0/1761197054 shutdown_connections 2024-04-24T08:49:48.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:44.654+0000 7f91dca00700 1 --2- 172.21.15.83:0/1761197054 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f91d81033e0 0x7f91d810f8f0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:48.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:44.654+0000 7f91dca00700 1 --2- 172.21.15.83:0/1761197054 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f91d8108a20 0x7f91d8108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:48.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:44.654+0000 7f91dca00700 1 --2- 172.21.15.83:0/1761197054 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91d8102a20 0x7f91d8102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:48.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:44.654+0000 7f91dca00700 1 -- 172.21.15.83:0/1761197054 >> 172.21.15.83:0/1761197054 conn(0x7f91d80fe570 msgr2=0x7f91d8100990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:49:48.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:44.654+0000 7f91dca00700 1 -- 172.21.15.83:0/1761197054 shutdown_connections 2024-04-24T08:49:48.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:44.654+0000 7f91dca00700 1 -- 172.21.15.83:0/1761197054 wait complete. 2024-04-24T08:49:48.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:44.655+0000 7f91dca00700 1 Processor -- start 2024-04-24T08:49:48.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:44.655+0000 7f91dca00700 1 -- start start 2024-04-24T08:49:48.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:44.656+0000 7f91dca00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f91d8102a20 0x7f91d819b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:48.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:44.656+0000 7f91dca00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f91d81033e0 0x7f91d819b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:48.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:44.656+0000 7f91dca00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91d8108a20 0x7f91d819fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:48.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:44.656+0000 7f91dca00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f91d8112c50 con 0x7f91d8102a20 2024-04-24T08:49:48.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:44.656+0000 7f91dca00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f91d8112ad0 con 0x7f91d8108a20 2024-04-24T08:49:48.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:44.656+0000 7f91dca00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f91d8112dd0 con 0x7f91d81033e0 2024-04-24T08:49:48.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:44.656+0000 7f91d7400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f91d8102a20 0x7f91d819b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:48.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:44.656+0000 7f91d7400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f91d8102a20 0x7f91d819b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55354/0 (socket says 172.21.15.83:55354) 2024-04-24T08:49:48.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:44.656+0000 7f91d7400700 1 -- 172.21.15.83:0/3513638526 learned_addr learned my addr 172.21.15.83:0/3513638526 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:49:48.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:44.656+0000 7f91d7e00700 1 --2- 172.21.15.83:0/3513638526 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91d8108a20 0x7f91d819fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:48.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:44.656+0000 7f91d6a00700 1 --2- 172.21.15.83:0/3513638526 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f91d81033e0 0x7f91d819b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:48.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:44.656+0000 7f91d7400700 1 -- 172.21.15.83:0/3513638526 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f91d81033e0 msgr2=0x7f91d819b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:48.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:44.656+0000 7f91d7400700 1 --2- 172.21.15.83:0/3513638526 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f91d81033e0 0x7f91d819b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:48.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:44.656+0000 7f91d7400700 1 -- 172.21.15.83:0/3513638526 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91d8108a20 msgr2=0x7f91d819fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:48.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:44.656+0000 7f91d7400700 1 --2- 172.21.15.83:0/3513638526 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91d8108a20 0x7f91d819fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:48.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:44.656+0000 7f91d7400700 1 -- 172.21.15.83:0/3513638526 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91d81a02b0 con 0x7f91d8102a20 2024-04-24T08:49:48.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:44.657+0000 7f91d7400700 1 --2- 172.21.15.83:0/3513638526 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f91d8102a20 0x7f91d819b2a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f91c80060b0 tx=0x7f91c8007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:49:48.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:44.657+0000 7f91cfe00700 1 -- 172.21.15.83:0/3513638526 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c801a070 con 0x7f91d8102a20 2024-04-24T08:49:48.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:44.657+0000 7f91cfe00700 1 -- 172.21.15.83:0/3513638526 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f91c800de20 con 0x7f91d8102a20 2024-04-24T08:49:48.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:44.657+0000 7f91cfe00700 1 -- 172.21.15.83:0/3513638526 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c8014720 con 0x7f91d8102a20 2024-04-24T08:49:48.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:44.657+0000 7f91dca00700 1 -- 172.21.15.83:0/3513638526 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91d81a05a0 con 0x7f91d8102a20 2024-04-24T08:49:48.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:44.657+0000 7f91dca00700 1 -- 172.21.15.83:0/3513638526 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91d81a7e20 con 0x7f91d8102a20 2024-04-24T08:49:48.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:44.658+0000 7f91dca00700 1 -- 172.21.15.83:0/3513638526 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f91d80ff350 con 0x7f91d8102a20 2024-04-24T08:49:48.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:44.660+0000 7f91cfe00700 1 -- 172.21.15.83:0/3513638526 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f91c801d070 con 0x7f91d8102a20 2024-04-24T08:49:48.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:44.662+0000 7f91cfe00700 1 --2- 172.21.15.83:0/3513638526 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f91bc0a3d50 0x7f91bc0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:48.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:44.662+0000 7f91cfe00700 1 -- 172.21.15.83:0/3513638526 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f91c80d09b0 con 0x7f91d8102a20 2024-04-24T08:49:48.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:44.662+0000 7f91d6a00700 1 --2- 172.21.15.83:0/3513638526 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f91bc0a3d50 0x7f91bc0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:48.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:44.662+0000 7f91cfe00700 1 --2- 172.21.15.83:0/3513638526 >> [v2:172.21.15.138:6816/3557856706,v1:172.21.15.138:6817/3557856706] conn(0x7f91bc0aeeb0 0x7f91bc0b1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:48.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:44.662+0000 7f91cfe00700 1 -- 172.21.15.83:0/3513638526 --> [v2:172.21.15.138:6816/3557856706,v1:172.21.15.138:6817/3557856706] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f91bc0b1a20 con 0x7f91bc0aeeb0 2024-04-24T08:49:48.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:44.662+0000 7f91cfe00700 1 -- 172.21.15.83:0/3513638526 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f91c80d0d60 con 0x7f91d8102a20 2024-04-24T08:49:48.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:44.663+0000 7f91d6a00700 1 --2- 172.21.15.83:0/3513638526 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f91bc0a3d50 0x7f91bc0a6210 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f91c4005950 tx=0x7f91c40058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:49:48.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:44.663+0000 7f91d7e00700 1 --2- 172.21.15.83:0/3513638526 >> [v2:172.21.15.138:6816/3557856706,v1:172.21.15.138:6817/3557856706] conn(0x7f91bc0aeeb0 0x7f91bc0b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:48.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:44.664+0000 7f91d7e00700 1 --2- 172.21.15.83:0/3513638526 >> [v2:172.21.15.138:6816/3557856706,v1:172.21.15.138:6817/3557856706] conn(0x7f91bc0aeeb0 0x7f91bc0b1350 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:49:48.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:44.666+0000 7f91cfe00700 1 -- 172.21.15.83:0/3513638526 <== osd.7 v2:172.21.15.138:6816/3557856706 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f91bc0b1a20 con 0x7f91bc0aeeb0 2024-04-24T08:49:48.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:44.686+0000 7f91dca00700 1 -- 172.21.15.83:0/3513638526 --> [v2:172.21.15.138:6816/3557856706,v1:172.21.15.138:6817/3557856706] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f91d8104160 con 0x7f91bc0aeeb0 2024-04-24T08:49:48.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:44.688+0000 7f91cfe00700 1 -- 172.21.15.83:0/3513638526 <== osd.7 v2:172.21.15.138:6816/3557856706 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f91d8104160 con 0x7f91bc0aeeb0 2024-04-24T08:49:48.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:44.688+0000 7f91dca00700 1 -- 172.21.15.83:0/3513638526 >> [v2:172.21.15.138:6816/3557856706,v1:172.21.15.138:6817/3557856706] conn(0x7f91bc0aeeb0 msgr2=0x7f91bc0b1350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:48.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:44.688+0000 7f91dca00700 1 --2- 172.21.15.83:0/3513638526 >> [v2:172.21.15.138:6816/3557856706,v1:172.21.15.138:6817/3557856706] conn(0x7f91bc0aeeb0 0x7f91bc0b1350 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:48.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:44.688+0000 7f91dca00700 1 -- 172.21.15.83:0/3513638526 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f91bc0a3d50 msgr2=0x7f91bc0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:48.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:44.688+0000 7f91dca00700 1 --2- 172.21.15.83:0/3513638526 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f91bc0a3d50 0x7f91bc0a6210 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f91c4005950 tx=0x7f91c40058e0 comp rx=0 tx=0).stop 2024-04-24T08:49:48.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:44.688+0000 7f91dca00700 1 -- 172.21.15.83:0/3513638526 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f91d8102a20 msgr2=0x7f91d819b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:48.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:44.688+0000 7f91dca00700 1 --2- 172.21.15.83:0/3513638526 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f91d8102a20 0x7f91d819b2a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f91c80060b0 tx=0x7f91c8007f90 comp rx=0 tx=0).stop 2024-04-24T08:49:48.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:44.688+0000 7f91d7e00700 1 -- 172.21.15.83:0/3513638526 reap_dead start 2024-04-24T08:49:48.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:44.688+0000 7f91dca00700 1 -- 172.21.15.83:0/3513638526 shutdown_connections 2024-04-24T08:49:48.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:44.688+0000 7f91dca00700 1 -- 172.21.15.83:0/3513638526 >> 172.21.15.83:0/3513638526 conn(0x7f91d80fe570 msgr2=0x7f91d80fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:49:48.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:44.689+0000 7f91dca00700 1 -- 172.21.15.83:0/3513638526 shutdown_connections 2024-04-24T08:49:48.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:44.689+0000 7f91dca00700 1 -- 172.21.15.83:0/3513638526 wait complete. 2024-04-24T08:49:49.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:43.732+0000 7f53ffe00700 1 -- 172.21.15.83:0/884303461 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5400108a20 msgr2=0x7f5400108e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:49.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:43.732+0000 7f53ffe00700 1 --2- 172.21.15.83:0/884303461 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5400108a20 0x7f5400108e00 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f53f0009990 tx=0x7f53f0009ca0 comp rx=0 tx=0).stop 2024-04-24T08:49:49.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:43.732+0000 7f53ffe00700 1 -- 172.21.15.83:0/884303461 shutdown_connections 2024-04-24T08:49:49.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:43.732+0000 7f53ffe00700 1 --2- 172.21.15.83:0/884303461 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5400108a20 0x7f5400108e00 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:49.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:43.732+0000 7f53ffe00700 1 --2- 172.21.15.83:0/884303461 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f54001033e0 0x7f540010f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:49.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:43.732+0000 7f53ffe00700 1 --2- 172.21.15.83:0/884303461 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5400102a20 0x7f5400102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:49.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:43.732+0000 7f53ffe00700 1 -- 172.21.15.83:0/884303461 >> 172.21.15.83:0/884303461 conn(0x7f54000fe570 msgr2=0x7f5400100990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:49:49.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:43.732+0000 7f53ffe00700 1 -- 172.21.15.83:0/884303461 shutdown_connections 2024-04-24T08:49:49.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:43.732+0000 7f53ffe00700 1 -- 172.21.15.83:0/884303461 wait complete. 2024-04-24T08:49:49.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:43.733+0000 7f53ffe00700 1 Processor -- start 2024-04-24T08:49:49.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:43.733+0000 7f53ffe00700 1 -- start start 2024-04-24T08:49:49.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:43.734+0000 7f53ffe00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5400102a20 0x7f540019b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:49.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:43.734+0000 7f53ffe00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54001033e0 0x7f540019b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:49.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:43.734+0000 7f53ffe00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5400108a20 0x7f540019fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:49.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:43.734+0000 7f53ffe00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5400112c70 con 0x7f54001033e0 2024-04-24T08:49:49.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:43.734+0000 7f53ffe00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5400112af0 con 0x7f5400108a20 2024-04-24T08:49:49.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:43.734+0000 7f53ffe00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5400112df0 con 0x7f5400102a20 2024-04-24T08:49:49.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:43.734+0000 7f53fe000700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54001033e0 0x7f540019b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:49.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:43.734+0000 7f53fe000700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54001033e0 0x7f540019b7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55334/0 (socket says 172.21.15.83:55334) 2024-04-24T08:49:49.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:43.734+0000 7f53fe000700 1 -- 172.21.15.83:0/2792312938 learned_addr learned my addr 172.21.15.83:0/2792312938 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:49:49.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:43.735+0000 7f53fea00700 1 --2- 172.21.15.83:0/2792312938 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5400102a20 0x7f540019b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:49.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:43.735+0000 7f53ff400700 1 --2- 172.21.15.83:0/2792312938 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5400108a20 0x7f540019fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:49.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:43.735+0000 7f53fe000700 1 -- 172.21.15.83:0/2792312938 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5400102a20 msgr2=0x7f540019b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:49.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:43.735+0000 7f53fe000700 1 --2- 172.21.15.83:0/2792312938 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5400102a20 0x7f540019b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:49.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:43.735+0000 7f53fe000700 1 -- 172.21.15.83:0/2792312938 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5400108a20 msgr2=0x7f540019fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:49.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:43.735+0000 7f53fe000700 1 --2- 172.21.15.83:0/2792312938 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5400108a20 0x7f540019fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:49.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:43.735+0000 7f53fe000700 1 -- 172.21.15.83:0/2792312938 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54001a02b0 con 0x7f54001033e0 2024-04-24T08:49:49.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:43.735+0000 7f53fe000700 1 --2- 172.21.15.83:0/2792312938 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54001033e0 0x7f540019b7e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f53ec00c970 tx=0x7f53ec00cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:49:49.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:43.736+0000 7f53f7400700 1 -- 172.21.15.83:0/2792312938 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53ec007aa0 con 0x7f54001033e0 2024-04-24T08:49:49.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:43.736+0000 7f53f7400700 1 -- 172.21.15.83:0/2792312938 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f53ec004ce0 con 0x7f54001033e0 2024-04-24T08:49:49.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:43.736+0000 7f53ffe00700 1 -- 172.21.15.83:0/2792312938 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54001a05a0 con 0x7f54001033e0 2024-04-24T08:49:49.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:43.736+0000 7f53f7400700 1 -- 172.21.15.83:0/2792312938 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53ec005620 con 0x7f54001033e0 2024-04-24T08:49:49.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:43.736+0000 7f53ffe00700 1 -- 172.21.15.83:0/2792312938 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54001a7e20 con 0x7f54001033e0 2024-04-24T08:49:49.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:43.737+0000 7f53ffe00700 1 -- 172.21.15.83:0/2792312938 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f540010f910 con 0x7f54001033e0 2024-04-24T08:49:49.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:43.739+0000 7f53f7400700 1 -- 172.21.15.83:0/2792312938 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f53ec004750 con 0x7f54001033e0 2024-04-24T08:49:49.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:43.740+0000 7f53f7400700 1 --2- 172.21.15.83:0/2792312938 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f53e40a3cd0 0x7f53e40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:49.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:43.741+0000 7f53f7400700 1 -- 172.21.15.83:0/2792312938 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f53ec0d0be0 con 0x7f54001033e0 2024-04-24T08:49:49.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:43.741+0000 7f53fea00700 1 --2- 172.21.15.83:0/2792312938 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f53e40a3cd0 0x7f53e40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:49.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:43.741+0000 7f53f7400700 1 --2- 172.21.15.83:0/2792312938 >> [v2:172.21.15.83:6818/1611358483,v1:172.21.15.83:6819/1611358483] conn(0x7f53e40aee30 0x7f53e40b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:49.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:43.741+0000 7f53f7400700 1 -- 172.21.15.83:0/2792312938 --> [v2:172.21.15.83:6818/1611358483,v1:172.21.15.83:6819/1611358483] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f53e40b19a0 con 0x7f53e40aee30 2024-04-24T08:49:49.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:43.741+0000 7f53f7400700 1 -- 172.21.15.83:0/2792312938 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f53ec0d0f90 con 0x7f54001033e0 2024-04-24T08:49:49.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:43.741+0000 7f53ff400700 1 --2- 172.21.15.83:0/2792312938 >> [v2:172.21.15.83:6818/1611358483,v1:172.21.15.83:6819/1611358483] conn(0x7f53e40aee30 0x7f53e40b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:49.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:43.741+0000 7f53fea00700 1 --2- 172.21.15.83:0/2792312938 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f53e40a3cd0 0x7f53e40a6190 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f53f000b5c0 tx=0x7f53f001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:49:49.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:43.742+0000 7f53ff400700 1 --2- 172.21.15.83:0/2792312938 >> [v2:172.21.15.83:6818/1611358483,v1:172.21.15.83:6819/1611358483] conn(0x7f53e40aee30 0x7f53e40b12d0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:49:49.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:43.743+0000 7f53f7400700 1 -- 172.21.15.83:0/2792312938 <== osd.6 v2:172.21.15.83:6818/1611358483 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f53e40b19a0 con 0x7f53e40aee30 2024-04-24T08:49:49.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:43.761+0000 7f53ffe00700 1 -- 172.21.15.83:0/2792312938 --> [v2:172.21.15.83:6818/1611358483,v1:172.21.15.83:6819/1611358483] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5400104160 con 0x7f53e40aee30 2024-04-24T08:49:49.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:43.761+0000 7f53f7400700 1 -- 172.21.15.83:0/2792312938 <== osd.6 v2:172.21.15.83:6818/1611358483 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f5400104160 con 0x7f53e40aee30 2024-04-24T08:49:49.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:43.762+0000 7f53ffe00700 1 -- 172.21.15.83:0/2792312938 >> [v2:172.21.15.83:6818/1611358483,v1:172.21.15.83:6819/1611358483] conn(0x7f53e40aee30 msgr2=0x7f53e40b12d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:49.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:43.762+0000 7f53ffe00700 1 --2- 172.21.15.83:0/2792312938 >> [v2:172.21.15.83:6818/1611358483,v1:172.21.15.83:6819/1611358483] conn(0x7f53e40aee30 0x7f53e40b12d0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:49.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:43.762+0000 7f53ffe00700 1 -- 172.21.15.83:0/2792312938 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f53e40a3cd0 msgr2=0x7f53e40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:49.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:43.762+0000 7f53ffe00700 1 --2- 172.21.15.83:0/2792312938 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f53e40a3cd0 0x7f53e40a6190 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f53f000b5c0 tx=0x7f53f001a040 comp rx=0 tx=0).stop 2024-04-24T08:49:49.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:43.762+0000 7f53ffe00700 1 -- 172.21.15.83:0/2792312938 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54001033e0 msgr2=0x7f540019b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:49.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:43.762+0000 7f53ffe00700 1 --2- 172.21.15.83:0/2792312938 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54001033e0 0x7f540019b7e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f53ec00c970 tx=0x7f53ec00cd30 comp rx=0 tx=0).stop 2024-04-24T08:49:49.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:43.762+0000 7f53ff400700 1 -- 172.21.15.83:0/2792312938 reap_dead start 2024-04-24T08:49:49.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:43.762+0000 7f53ffe00700 1 -- 172.21.15.83:0/2792312938 shutdown_connections 2024-04-24T08:49:49.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:43.762+0000 7f53ffe00700 1 -- 172.21.15.83:0/2792312938 >> 172.21.15.83:0/2792312938 conn(0x7f54000fe570 msgr2=0x7f5400107ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:49:49.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:43.762+0000 7f53ffe00700 1 -- 172.21.15.83:0/2792312938 shutdown_connections 2024-04-24T08:49:49.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:43.763+0000 7f53ffe00700 1 -- 172.21.15.83:0/2792312938 wait complete. 2024-04-24T08:49:49.215 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:49:49 smithi083 ceph-mon[27907]: pgmap v415: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-04-24T08:49:49.215 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:49:49 smithi083 ceph-mon[27907]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-04-24T08:49:49.215 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:49:49 smithi083 ceph-mon[27907]: Cluster is now healthy 2024-04-24T08:49:49.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:49:49 smithi138 ceph-mon[31441]: pgmap v415: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-04-24T08:49:49.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:49:49 smithi138 ceph-mon[31441]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-04-24T08:49:49.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:49:49 smithi138 ceph-mon[31441]: Cluster is now healthy 2024-04-24T08:49:49.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:49:49 smithi189 ceph-mon[31791]: pgmap v415: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-04-24T08:49:49.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:49:49 smithi189 ceph-mon[31791]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-04-24T08:49:49.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:49:49 smithi189 ceph-mon[31791]: Cluster is now healthy 2024-04-24T08:49:49.949 INFO:teuthology.orchestra.run.smithi083.stdout:219043332156 2024-04-24T08:49:49.949 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid bd3e039e-0215-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-04-24T08:49:51.115 INFO:teuthology.orchestra.run.smithi083.stdout:197568495683 2024-04-24T08:49:51.116 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid bd3e039e-0215-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-04-24T08:49:51.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.441+0000 7f2994800700 1 -- 172.21.15.83:0/363317137 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f298c074d20 msgr2=0x7f298c075100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:51.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.441+0000 7f2994800700 1 --2- 172.21.15.83:0/363317137 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f298c074d20 0x7f298c075100 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f2980009990 tx=0x7f2980009ca0 comp rx=0 tx=0).stop 2024-04-24T08:49:51.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.442+0000 7f2994800700 1 -- 172.21.15.83:0/363317137 shutdown_connections 2024-04-24T08:49:51.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.442+0000 7f2994800700 1 --2- 172.21.15.83:0/363317137 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f298c074d20 0x7f298c075100 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:51.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.442+0000 7f2994800700 1 --2- 172.21.15.83:0/363317137 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f298c10f4f0 0x7f298c1118e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:51.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.442+0000 7f2994800700 1 --2- 172.21.15.83:0/363317137 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f298c075640 0x7f298c10efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:51.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.442+0000 7f2994800700 1 -- 172.21.15.83:0/363317137 >> 172.21.15.83:0/363317137 conn(0x7f298c079c50 msgr2=0x7f298c07a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:49:51.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.442+0000 7f2994800700 1 -- 172.21.15.83:0/363317137 shutdown_connections 2024-04-24T08:49:51.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.442+0000 7f2994800700 1 -- 172.21.15.83:0/363317137 wait complete. 2024-04-24T08:49:51.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.443+0000 7f2994800700 1 Processor -- start 2024-04-24T08:49:51.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.443+0000 7f2994800700 1 -- start start 2024-04-24T08:49:51.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.443+0000 7f2994800700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f298c074d20 0x7f298c19b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:51.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.443+0000 7f2994800700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f298c075640 0x7f298c19b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:51.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.443+0000 7f2994800700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f298c10f4f0 0x7f298c19fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:51.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.443+0000 7f2994800700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f298c114f10 con 0x7f298c10f4f0 2024-04-24T08:49:51.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.443+0000 7f2994800700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f298c114d90 con 0x7f298c074d20 2024-04-24T08:49:51.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.443+0000 7f2994800700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f298c115090 con 0x7f298c075640 2024-04-24T08:49:51.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.444+0000 7f2992800700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f298c10f4f0 0x7f298c19fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:51.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.444+0000 7f2992800700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f298c10f4f0 0x7f298c19fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55404/0 (socket says 172.21.15.83:55404) 2024-04-24T08:49:51.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.444+0000 7f2992800700 1 -- 172.21.15.83:0/3974243588 learned_addr learned my addr 172.21.15.83:0/3974243588 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:49:51.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.444+0000 7f2991e00700 1 --2- 172.21.15.83:0/3974243588 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f298c074d20 0x7f298c19b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:51.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.444+0000 7f2991400700 1 --2- 172.21.15.83:0/3974243588 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f298c075640 0x7f298c19b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:51.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.444+0000 7f2992800700 1 -- 172.21.15.83:0/3974243588 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f298c075640 msgr2=0x7f298c19b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:51.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.444+0000 7f2992800700 1 --2- 172.21.15.83:0/3974243588 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f298c075640 0x7f298c19b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:51.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.444+0000 7f2992800700 1 -- 172.21.15.83:0/3974243588 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f298c074d20 msgr2=0x7f298c19b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:51.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.444+0000 7f2992800700 1 --2- 172.21.15.83:0/3974243588 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f298c074d20 0x7f298c19b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:51.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.444+0000 7f2992800700 1 -- 172.21.15.83:0/3974243588 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f298c1a0370 con 0x7f298c10f4f0 2024-04-24T08:49:51.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.445+0000 7f2992800700 1 --2- 172.21.15.83:0/3974243588 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f298c10f4f0 0x7f298c19fc30 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f298800ada0 tx=0x7f298800c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:49:51.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.445+0000 7f2986a00700 1 -- 172.21.15.83:0/3974243588 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2988017020 con 0x7f298c10f4f0 2024-04-24T08:49:51.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.445+0000 7f2994800700 1 -- 172.21.15.83:0/3974243588 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f298c1a0660 con 0x7f298c10f4f0 2024-04-24T08:49:51.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.445+0000 7f2986a00700 1 -- 172.21.15.83:0/3974243588 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2988007e30 con 0x7f298c10f4f0 2024-04-24T08:49:51.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.445+0000 7f2986a00700 1 -- 172.21.15.83:0/3974243588 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2988005420 con 0x7f298c10f4f0 2024-04-24T08:49:51.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.445+0000 7f2994800700 1 -- 172.21.15.83:0/3974243588 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f298c1a7ee0 con 0x7f298c10f4f0 2024-04-24T08:49:51.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.446+0000 7f2994800700 1 -- 172.21.15.83:0/3974243588 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f298c063240 con 0x7f298c10f4f0 2024-04-24T08:49:51.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.448+0000 7f2986a00700 1 -- 172.21.15.83:0/3974243588 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f298800c980 con 0x7f298c10f4f0 2024-04-24T08:49:51.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.449+0000 7f2986a00700 1 --2- 172.21.15.83:0/3974243588 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f29740a3cd0 0x7f29740a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:51.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.450+0000 7f2986a00700 1 -- 172.21.15.83:0/3974243588 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f29880d0ad0 con 0x7f298c10f4f0 2024-04-24T08:49:51.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.450+0000 7f2991e00700 1 --2- 172.21.15.83:0/3974243588 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f29740a3cd0 0x7f29740a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:51.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.450+0000 7f2986a00700 1 --2- 172.21.15.83:0/3974243588 >> [v2:172.21.15.83:6826/519636751,v1:172.21.15.83:6827/519636751] conn(0x7f29740aee30 0x7f29740b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:51.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.450+0000 7f2986a00700 1 -- 172.21.15.83:0/3974243588 --> [v2:172.21.15.83:6826/519636751,v1:172.21.15.83:6827/519636751] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2988019e50 con 0x7f29740aee30 2024-04-24T08:49:51.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.450+0000 7f2986a00700 1 -- 172.21.15.83:0/3974243588 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f29880d0e80 con 0x7f298c10f4f0 2024-04-24T08:49:51.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.450+0000 7f2991400700 1 --2- 172.21.15.83:0/3974243588 >> [v2:172.21.15.83:6826/519636751,v1:172.21.15.83:6827/519636751] conn(0x7f29740aee30 0x7f29740b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:51.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.451+0000 7f2991e00700 1 --2- 172.21.15.83:0/3974243588 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f29740a3cd0 0x7f29740a6190 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f298000b5c0 tx=0x7f298001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:49:51.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.451+0000 7f2991400700 1 --2- 172.21.15.83:0/3974243588 >> [v2:172.21.15.83:6826/519636751,v1:172.21.15.83:6827/519636751] conn(0x7f29740aee30 0x7f29740b12d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:49:51.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.452+0000 7f2986a00700 1 -- 172.21.15.83:0/3974243588 <== osd.9 v2:172.21.15.83:6826/519636751 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2988019e50 con 0x7f29740aee30 2024-04-24T08:49:51.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.470+0000 7f2994800700 1 -- 172.21.15.83:0/3974243588 --> [v2:172.21.15.83:6826/519636751,v1:172.21.15.83:6827/519636751] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f298c0636c0 con 0x7f29740aee30 2024-04-24T08:49:51.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.471+0000 7f2986a00700 1 -- 172.21.15.83:0/3974243588 <== osd.9 v2:172.21.15.83:6826/519636751 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f298c0636c0 con 0x7f29740aee30 2024-04-24T08:49:51.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.471+0000 7f2994800700 1 -- 172.21.15.83:0/3974243588 >> [v2:172.21.15.83:6826/519636751,v1:172.21.15.83:6827/519636751] conn(0x7f29740aee30 msgr2=0x7f29740b12d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:51.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.471+0000 7f2994800700 1 --2- 172.21.15.83:0/3974243588 >> [v2:172.21.15.83:6826/519636751,v1:172.21.15.83:6827/519636751] conn(0x7f29740aee30 0x7f29740b12d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:51.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.471+0000 7f2994800700 1 -- 172.21.15.83:0/3974243588 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f29740a3cd0 msgr2=0x7f29740a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:51.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.471+0000 7f2994800700 1 --2- 172.21.15.83:0/3974243588 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f29740a3cd0 0x7f29740a6190 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f298000b5c0 tx=0x7f298001a040 comp rx=0 tx=0).stop 2024-04-24T08:49:51.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.471+0000 7f2994800700 1 -- 172.21.15.83:0/3974243588 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f298c10f4f0 msgr2=0x7f298c19fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:51.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.471+0000 7f2994800700 1 --2- 172.21.15.83:0/3974243588 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f298c10f4f0 0x7f298c19fc30 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f298800ada0 tx=0x7f298800c330 comp rx=0 tx=0).stop 2024-04-24T08:49:51.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.471+0000 7f2992800700 1 -- 172.21.15.83:0/3974243588 reap_dead start 2024-04-24T08:49:51.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.472+0000 7f2994800700 1 -- 172.21.15.83:0/3974243588 shutdown_connections 2024-04-24T08:49:51.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.472+0000 7f2994800700 1 -- 172.21.15.83:0/3974243588 >> 172.21.15.83:0/3974243588 conn(0x7f298c079c50 msgr2=0x7f298c0ffc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:49:51.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.472+0000 7f2994800700 1 -- 172.21.15.83:0/3974243588 shutdown_connections 2024-04-24T08:49:51.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:46.472+0000 7f2994800700 1 -- 172.21.15.83:0/3974243588 wait complete. 2024-04-24T08:49:52.015 INFO:teuthology.orchestra.run.smithi083.stdout:34359738477 2024-04-24T08:49:52.016 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid bd3e039e-0215-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-04-24T08:49:52.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:49.272+0000 7f675fe00700 1 -- 172.21.15.83:0/78845733 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6760108a00 msgr2=0x7f6760108de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:52.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:49.272+0000 7f675fe00700 1 --2- 172.21.15.83:0/78845733 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6760108a00 0x7f6760108de0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f675000b0a0 tx=0x7f675000b3b0 comp rx=0 tx=0).stop 2024-04-24T08:49:52.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:49.272+0000 7f675fe00700 1 -- 172.21.15.83:0/78845733 shutdown_connections 2024-04-24T08:49:52.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:49.272+0000 7f675fe00700 1 --2- 172.21.15.83:0/78845733 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6760108a00 0x7f6760108de0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:52.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:49.272+0000 7f675fe00700 1 --2- 172.21.15.83:0/78845733 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f67601033c0 0x7f676010f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:52.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:49.272+0000 7f675fe00700 1 --2- 172.21.15.83:0/78845733 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6760102a00 0x7f6760102e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:52.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:49.272+0000 7f675fe00700 1 -- 172.21.15.83:0/78845733 >> 172.21.15.83:0/78845733 conn(0x7f67600fe4d0 msgr2=0x7f67601008f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:49:52.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:49.272+0000 7f675fe00700 1 -- 172.21.15.83:0/78845733 shutdown_connections 2024-04-24T08:49:52.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:49.272+0000 7f675fe00700 1 -- 172.21.15.83:0/78845733 wait complete. 2024-04-24T08:49:52.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:49.273+0000 7f675fe00700 1 Processor -- start 2024-04-24T08:49:52.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:49.273+0000 7f675fe00700 1 -- start start 2024-04-24T08:49:52.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:49.273+0000 7f675fe00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6760102a00 0x7f6760198ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:52.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:49.273+0000 7f675fe00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67601033c0 0x7f6760199530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:52.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:49.273+0000 7f675fe00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6760108a00 0x7f676019d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:52.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:49.273+0000 7f675fe00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6760112c00 con 0x7f67601033c0 2024-04-24T08:49:52.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:49.273+0000 7f675fe00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6760112a80 con 0x7f6760108a00 2024-04-24T08:49:52.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:49.273+0000 7f675fe00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6760112d80 con 0x7f6760102a00 2024-04-24T08:49:52.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:49.274+0000 7f6756000700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67601033c0 0x7f6760199530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:52.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:49.274+0000 7f6756000700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67601033c0 0x7f6760199530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:59634/0 (socket says 172.21.15.83:59634) 2024-04-24T08:49:52.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:49.274+0000 7f6756000700 1 -- 172.21.15.83:0/473203600 learned_addr learned my addr 172.21.15.83:0/473203600 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:49:52.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:49.274+0000 7f675ea00700 1 --2- 172.21.15.83:0/473203600 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6760102a00 0x7f6760198ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:52.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:49.274+0000 7f675f400700 1 --2- 172.21.15.83:0/473203600 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6760108a00 0x7f676019d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:52.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:49.274+0000 7f6756000700 1 -- 172.21.15.83:0/473203600 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6760102a00 msgr2=0x7f6760198ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:52.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:49.274+0000 7f6756000700 1 --2- 172.21.15.83:0/473203600 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6760102a00 0x7f6760198ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:52.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:49.274+0000 7f6756000700 1 -- 172.21.15.83:0/473203600 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6760108a00 msgr2=0x7f676019d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:52.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:49.274+0000 7f6756000700 1 --2- 172.21.15.83:0/473203600 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6760108a00 0x7f676019d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:52.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:49.274+0000 7f6756000700 1 -- 172.21.15.83:0/473203600 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f676019e000 con 0x7f67601033c0 2024-04-24T08:49:52.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:49.275+0000 7f6756000700 1 --2- 172.21.15.83:0/473203600 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67601033c0 0x7f6760199530 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f674800c9b0 tx=0x7f674800cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:49:52.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:49.275+0000 7f6757e00700 1 -- 172.21.15.83:0/473203600 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6748007b00 con 0x7f67601033c0 2024-04-24T08:49:52.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:49.275+0000 7f6757e00700 1 -- 172.21.15.83:0/473203600 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6748007ca0 con 0x7f67601033c0 2024-04-24T08:49:52.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:49.275+0000 7f6757e00700 1 -- 172.21.15.83:0/473203600 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6748018730 con 0x7f67601033c0 2024-04-24T08:49:52.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:49.275+0000 7f675fe00700 1 -- 172.21.15.83:0/473203600 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f676019e2f0 con 0x7f67601033c0 2024-04-24T08:49:52.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:49.275+0000 7f675fe00700 1 -- 172.21.15.83:0/473203600 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67601a5b70 con 0x7f67601033c0 2024-04-24T08:49:52.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:49.276+0000 7f675fe00700 1 -- 172.21.15.83:0/473203600 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6760063280 con 0x7f67601033c0 2024-04-24T08:49:52.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:49.278+0000 7f6757e00700 1 -- 172.21.15.83:0/473203600 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f674800f450 con 0x7f67601033c0 2024-04-24T08:49:52.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:49.279+0000 7f6757e00700 1 --2- 172.21.15.83:0/473203600 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f67440a3cd0 0x7f67440a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:52.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:49.280+0000 7f6757e00700 1 -- 172.21.15.83:0/473203600 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f67480d0e50 con 0x7f67601033c0 2024-04-24T08:49:52.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:49.280+0000 7f675ea00700 1 --2- 172.21.15.83:0/473203600 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f67440a3cd0 0x7f67440a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:52.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:49.280+0000 7f6757e00700 1 --2- 172.21.15.83:0/473203600 >> [v2:172.21.15.83:6810/3131092450,v1:172.21.15.83:6811/3131092450] conn(0x7f67440aee30 0x7f67440b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:52.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:49.280+0000 7f6757e00700 1 -- 172.21.15.83:0/473203600 --> [v2:172.21.15.83:6810/3131092450,v1:172.21.15.83:6811/3131092450] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f67440b19a0 con 0x7f67440aee30 2024-04-24T08:49:52.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:49.280+0000 7f6757e00700 1 -- 172.21.15.83:0/473203600 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f67480d1200 con 0x7f67601033c0 2024-04-24T08:49:52.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:49.280+0000 7f675f400700 1 --2- 172.21.15.83:0/473203600 >> [v2:172.21.15.83:6810/3131092450,v1:172.21.15.83:6811/3131092450] conn(0x7f67440aee30 0x7f67440b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:52.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:49.280+0000 7f675ea00700 1 --2- 172.21.15.83:0/473203600 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f67440a3cd0 0x7f67440a6190 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f6750006130 tx=0x7f67500060a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:49:52.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:49.281+0000 7f675f400700 1 --2- 172.21.15.83:0/473203600 >> [v2:172.21.15.83:6810/3131092450,v1:172.21.15.83:6811/3131092450] conn(0x7f67440aee30 0x7f67440b12d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:49:52.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:49.282+0000 7f6757e00700 1 -- 172.21.15.83:0/473203600 <== osd.3 v2:172.21.15.83:6810/3131092450 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f67440b19a0 con 0x7f67440aee30 2024-04-24T08:49:52.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:49.300+0000 7f675fe00700 1 -- 172.21.15.83:0/473203600 --> [v2:172.21.15.83:6810/3131092450,v1:172.21.15.83:6811/3131092450] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6760063700 con 0x7f67440aee30 2024-04-24T08:49:52.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:49.301+0000 7f6757e00700 1 -- 172.21.15.83:0/473203600 <== osd.3 v2:172.21.15.83:6810/3131092450 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f6760063700 con 0x7f67440aee30 2024-04-24T08:49:52.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:49.301+0000 7f675fe00700 1 -- 172.21.15.83:0/473203600 >> [v2:172.21.15.83:6810/3131092450,v1:172.21.15.83:6811/3131092450] conn(0x7f67440aee30 msgr2=0x7f67440b12d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:52.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:49.301+0000 7f675fe00700 1 --2- 172.21.15.83:0/473203600 >> [v2:172.21.15.83:6810/3131092450,v1:172.21.15.83:6811/3131092450] conn(0x7f67440aee30 0x7f67440b12d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:52.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:49.302+0000 7f675fe00700 1 -- 172.21.15.83:0/473203600 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f67440a3cd0 msgr2=0x7f67440a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:52.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:49.302+0000 7f675fe00700 1 --2- 172.21.15.83:0/473203600 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f67440a3cd0 0x7f67440a6190 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f6750006130 tx=0x7f67500060a0 comp rx=0 tx=0).stop 2024-04-24T08:49:52.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:49.302+0000 7f675fe00700 1 -- 172.21.15.83:0/473203600 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67601033c0 msgr2=0x7f6760199530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:52.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:49.302+0000 7f675fe00700 1 --2- 172.21.15.83:0/473203600 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67601033c0 0x7f6760199530 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f674800c9b0 tx=0x7f674800cd70 comp rx=0 tx=0).stop 2024-04-24T08:49:52.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:49.302+0000 7f675f400700 1 -- 172.21.15.83:0/473203600 reap_dead start 2024-04-24T08:49:52.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:49.302+0000 7f675fe00700 1 -- 172.21.15.83:0/473203600 shutdown_connections 2024-04-24T08:49:52.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:49.302+0000 7f675fe00700 1 -- 172.21.15.83:0/473203600 >> 172.21.15.83:0/473203600 conn(0x7f67600fe4d0 msgr2=0x7f67600feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:49:52.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:49.302+0000 7f675fe00700 1 -- 172.21.15.83:0/473203600 shutdown_connections 2024-04-24T08:49:52.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:49.302+0000 7f675fe00700 1 -- 172.21.15.83:0/473203600 wait complete. 2024-04-24T08:49:52.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:49:51 smithi138 ceph-mon[31441]: pgmap v416: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-04-24T08:49:52.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:49:51 smithi189 ceph-mon[31791]: pgmap v416: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-04-24T08:49:52.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:49:51 smithi083 ceph-mon[27907]: pgmap v416: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-04-24T08:49:52.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:52.695+0000 7f62a7200700 1 -- 172.21.15.83:0/2556351037 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62a810f0e0 msgr2=0x7f62a810f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:52.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:52.695+0000 7f62a7200700 1 --2- 172.21.15.83:0/2556351037 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62a810f0e0 0x7f62a810f4c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f6298009990 tx=0x7f6298009ca0 comp rx=0 tx=0).stop 2024-04-24T08:49:52.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:52.695+0000 7f62a7200700 1 -- 172.21.15.83:0/2556351037 shutdown_connections 2024-04-24T08:49:52.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:52.695+0000 7f62a7200700 1 --2- 172.21.15.83:0/2556351037 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62a810f0e0 0x7f62a810f4c0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:52.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:52.695+0000 7f62a7200700 1 --2- 172.21.15.83:0/2556351037 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f62a81099f0 0x7f62a8115e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:52.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:52.695+0000 7f62a7200700 1 --2- 172.21.15.83:0/2556351037 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f62a8109030 0x7f62a81094b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:52.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:52.695+0000 7f62a7200700 1 -- 172.21.15.83:0/2556351037 >> 172.21.15.83:0/2556351037 conn(0x7f62a8078880 msgr2=0x7f62a8107910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:49:52.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:52.695+0000 7f62a7200700 1 -- 172.21.15.83:0/2556351037 shutdown_connections 2024-04-24T08:49:52.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:52.695+0000 7f62a7200700 1 -- 172.21.15.83:0/2556351037 wait complete. 2024-04-24T08:49:52.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:52.696+0000 7f62a7200700 1 Processor -- start 2024-04-24T08:49:52.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:52.696+0000 7f62a7200700 1 -- start start 2024-04-24T08:49:52.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:52.697+0000 7f62a7200700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f62a8109030 0x7f62a819f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:52.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:52.697+0000 7f62a7200700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f62a81099f0 0x7f62a819fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:52.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:52.697+0000 7f62a7200700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62a810f0e0 0x7f62a81a3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:52.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:52.697+0000 7f62a7200700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f62a81191d0 con 0x7f62a810f0e0 2024-04-24T08:49:52.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:52.697+0000 7f62a7200700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f62a8119050 con 0x7f62a8109030 2024-04-24T08:49:52.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:52.697+0000 7f62a7200700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f62a8119350 con 0x7f62a81099f0 2024-04-24T08:49:52.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:52.697+0000 7f62a6800700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62a810f0e0 0x7f62a81a3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:52.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:52.697+0000 7f62a6800700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62a810f0e0 0x7f62a81a3eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:59656/0 (socket says 172.21.15.83:59656) 2024-04-24T08:49:52.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:52.697+0000 7f62a6800700 1 -- 172.21.15.83:0/3735546473 learned_addr learned my addr 172.21.15.83:0/3735546473 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:49:52.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:52.697+0000 7f62a5e00700 1 --2- 172.21.15.83:0/3735546473 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f62a81099f0 0x7f62a819fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:52.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:52.697+0000 7f629de00700 1 --2- 172.21.15.83:0/3735546473 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f62a8109030 0x7f62a819f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:52.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:52.698+0000 7f62a6800700 1 -- 172.21.15.83:0/3735546473 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f62a81099f0 msgr2=0x7f62a819fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:52.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:52.698+0000 7f62a6800700 1 --2- 172.21.15.83:0/3735546473 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f62a81099f0 0x7f62a819fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:52.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:52.698+0000 7f62a6800700 1 -- 172.21.15.83:0/3735546473 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f62a8109030 msgr2=0x7f62a819f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:52.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:52.698+0000 7f62a6800700 1 --2- 172.21.15.83:0/3735546473 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f62a8109030 0x7f62a819f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:52.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:52.698+0000 7f62a6800700 1 -- 172.21.15.83:0/3735546473 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62a81a45f0 con 0x7f62a810f0e0 2024-04-24T08:49:52.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:52.698+0000 7f62a6800700 1 --2- 172.21.15.83:0/3735546473 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62a810f0e0 0x7f62a81a3eb0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f62a0007d70 tx=0x7f62a000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:49:52.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:52.698+0000 7f629f400700 1 -- 172.21.15.83:0/3735546473 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a0016020 con 0x7f62a810f0e0 2024-04-24T08:49:52.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:52.699+0000 7f629f400700 1 -- 172.21.15.83:0/3735546473 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f62a000ac90 con 0x7f62a810f0e0 2024-04-24T08:49:52.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:52.699+0000 7f62a7200700 1 -- 172.21.15.83:0/3735546473 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62a81a48e0 con 0x7f62a810f0e0 2024-04-24T08:49:52.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:52.699+0000 7f629f400700 1 -- 172.21.15.83:0/3735546473 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a00115e0 con 0x7f62a810f0e0 2024-04-24T08:49:52.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:52.699+0000 7f62a7200700 1 -- 172.21.15.83:0/3735546473 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62a81ac160 con 0x7f62a810f0e0 2024-04-24T08:49:52.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:52.700+0000 7f62a7200700 1 -- 172.21.15.83:0/3735546473 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f62a8063240 con 0x7f62a810f0e0 2024-04-24T08:49:52.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:52.702+0000 7f629f400700 1 -- 172.21.15.83:0/3735546473 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f62a0019020 con 0x7f62a810f0e0 2024-04-24T08:49:52.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:52.703+0000 7f629f400700 1 --2- 172.21.15.83:0/3735546473 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f62880a3d50 0x7f62880a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:52.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:52.703+0000 7f629f400700 1 -- 172.21.15.83:0/3735546473 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f62a00d0c30 con 0x7f62a810f0e0 2024-04-24T08:49:52.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:52.704+0000 7f629de00700 1 --2- 172.21.15.83:0/3735546473 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f62880a3d50 0x7f62880a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:52.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:52.704+0000 7f629f400700 1 --2- 172.21.15.83:0/3735546473 >> [v2:172.21.15.138:6808/823775360,v1:172.21.15.138:6809/823775360] conn(0x7f62880aeeb0 0x7f62880b1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:52.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:52.704+0000 7f629f400700 1 -- 172.21.15.83:0/3735546473 --> [v2:172.21.15.138:6808/823775360,v1:172.21.15.138:6809/823775360] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f62a0024e50 con 0x7f62880aeeb0 2024-04-24T08:49:52.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:52.704+0000 7f629f400700 1 -- 172.21.15.83:0/3735546473 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f62a00d0fe0 con 0x7f62a810f0e0 2024-04-24T08:49:52.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:52.704+0000 7f629de00700 1 --2- 172.21.15.83:0/3735546473 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f62880a3d50 0x7f62880a6210 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f62980098e0 tx=0x7f6298005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:49:52.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:52.710+0000 7f62a5e00700 1 --2- 172.21.15.83:0/3735546473 >> [v2:172.21.15.138:6808/823775360,v1:172.21.15.138:6809/823775360] conn(0x7f62880aeeb0 0x7f62880b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:52.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:52.711+0000 7f62a5e00700 1 --2- 172.21.15.83:0/3735546473 >> [v2:172.21.15.138:6808/823775360,v1:172.21.15.138:6809/823775360] conn(0x7f62880aeeb0 0x7f62880b1350 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:49:52.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:52.713+0000 7f629f400700 1 -- 172.21.15.83:0/3735546473 <== osd.4 v2:172.21.15.138:6808/823775360 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f62a0024e50 con 0x7f62880aeeb0 2024-04-24T08:49:52.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:52.731+0000 7f62a7200700 1 -- 172.21.15.83:0/3735546473 --> [v2:172.21.15.138:6808/823775360,v1:172.21.15.138:6809/823775360] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f62a80636c0 con 0x7f62880aeeb0 2024-04-24T08:49:52.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:52.732+0000 7f629f400700 1 -- 172.21.15.83:0/3735546473 <== osd.4 v2:172.21.15.138:6808/823775360 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f62a80636c0 con 0x7f62880aeeb0 2024-04-24T08:49:52.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:52.733+0000 7f62a7200700 1 -- 172.21.15.83:0/3735546473 >> [v2:172.21.15.138:6808/823775360,v1:172.21.15.138:6809/823775360] conn(0x7f62880aeeb0 msgr2=0x7f62880b1350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:52.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:52.733+0000 7f62a7200700 1 --2- 172.21.15.83:0/3735546473 >> [v2:172.21.15.138:6808/823775360,v1:172.21.15.138:6809/823775360] conn(0x7f62880aeeb0 0x7f62880b1350 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:52.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:52.733+0000 7f62a7200700 1 -- 172.21.15.83:0/3735546473 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f62880a3d50 msgr2=0x7f62880a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:52.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:52.733+0000 7f62a7200700 1 --2- 172.21.15.83:0/3735546473 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f62880a3d50 0x7f62880a6210 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f62980098e0 tx=0x7f6298005960 comp rx=0 tx=0).stop 2024-04-24T08:49:52.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:52.733+0000 7f62a7200700 1 -- 172.21.15.83:0/3735546473 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62a810f0e0 msgr2=0x7f62a81a3eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:52.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:52.733+0000 7f62a7200700 1 --2- 172.21.15.83:0/3735546473 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62a810f0e0 0x7f62a81a3eb0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f62a0007d70 tx=0x7f62a000a340 comp rx=0 tx=0).stop 2024-04-24T08:49:52.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:52.733+0000 7f62a6800700 1 -- 172.21.15.83:0/3735546473 reap_dead start 2024-04-24T08:49:52.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:52.733+0000 7f62a7200700 1 -- 172.21.15.83:0/3735546473 shutdown_connections 2024-04-24T08:49:52.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:52.733+0000 7f62a7200700 1 -- 172.21.15.83:0/3735546473 >> 172.21.15.83:0/3735546473 conn(0x7f62a8078880 msgr2=0x7f62a8079950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:49:52.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:52.733+0000 7f62a7200700 1 -- 172.21.15.83:0/3735546473 shutdown_connections 2024-04-24T08:49:52.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:52.733+0000 7f62a7200700 1 -- 172.21.15.83:0/3735546473 wait complete. 2024-04-24T08:49:53.572 INFO:teuthology.orchestra.run.smithi083.stdout:77309411424 2024-04-24T08:49:53.572 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid bd3e039e-0215-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-04-24T08:49:53.660 INFO:teuthology.orchestra.run.smithi083.stdout:107374182491 2024-04-24T08:49:53.660 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid bd3e039e-0215-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-04-24T08:49:54.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:49:53 smithi083 ceph-mon[27907]: pgmap v417: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s 2024-04-24T08:49:54.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:49:53 smithi138 ceph-mon[31441]: pgmap v417: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s 2024-04-24T08:49:54.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:49:53 smithi189 ceph-mon[31791]: pgmap v417: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s 2024-04-24T08:49:54.999 INFO:teuthology.orchestra.run.smithi083.stdout:287762808878 2024-04-24T08:49:55.000 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid bd3e039e-0215-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-04-24T08:49:55.528 INFO:teuthology.orchestra.run.smithi083.stdout:141733920851 2024-04-24T08:49:55.528 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid bd3e039e-0215-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-04-24T08:49:55.894 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T08:49:55.923 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T08:49:56.183 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:49:55 smithi083 ceph-mon[27907]: pgmap v418: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s 2024-04-24T08:49:56.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:49:55 smithi138 ceph-mon[31441]: pgmap v418: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s 2024-04-24T08:49:56.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:56.206+0000 7ff70a400700 1 -- 172.21.15.83:0/172356793 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff70410f4f0 msgr2=0x7ff7041118e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:56.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:56.206+0000 7ff70a400700 1 --2- 172.21.15.83:0/172356793 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff70410f4f0 0x7ff7041118e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc00b0a0 tx=0x7ff6fc00b3b0 comp rx=0 tx=0).stop 2024-04-24T08:49:56.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:56.206+0000 7ff70a400700 1 -- 172.21.15.83:0/172356793 shutdown_connections 2024-04-24T08:49:56.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:56.206+0000 7ff70a400700 1 --2- 172.21.15.83:0/172356793 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff70410f4f0 0x7ff7041118e0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:56.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:56.206+0000 7ff70a400700 1 --2- 172.21.15.83:0/172356793 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff704074d20 0x7ff704075100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:56.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:56.206+0000 7ff70a400700 1 --2- 172.21.15.83:0/172356793 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff704075640 0x7ff70410efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:56.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:56.206+0000 7ff70a400700 1 -- 172.21.15.83:0/172356793 >> 172.21.15.83:0/172356793 conn(0x7ff704079ca0 msgr2=0x7ff70407a0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:49:56.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:56.206+0000 7ff70a400700 1 -- 172.21.15.83:0/172356793 shutdown_connections 2024-04-24T08:49:56.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:56.206+0000 7ff70a400700 1 -- 172.21.15.83:0/172356793 wait complete. 2024-04-24T08:49:56.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:49:55 smithi189 ceph-mon[31791]: pgmap v418: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s 2024-04-24T08:49:56.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:56.207+0000 7ff70a400700 1 Processor -- start 2024-04-24T08:49:56.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:56.208+0000 7ff70a400700 1 -- start start 2024-04-24T08:49:56.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:56.208+0000 7ff70a400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff704074d20 0x7ff70419b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:56.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:56.208+0000 7ff70a400700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff704075640 0x7ff70419b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:56.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:56.208+0000 7ff70a400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff70410f4f0 0x7ff70419fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:56.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:56.208+0000 7ff70a400700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff704114ef0 con 0x7ff704074d20 2024-04-24T08:49:56.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:56.208+0000 7ff70a400700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff704114d70 con 0x7ff704075640 2024-04-24T08:49:56.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:56.208+0000 7ff70a400700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff704115070 con 0x7ff70410f4f0 2024-04-24T08:49:56.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:56.208+0000 7ff703400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff704074d20 0x7ff70419b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:56.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:56.209+0000 7ff703400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff704074d20 0x7ff70419b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:59682/0 (socket says 172.21.15.83:59682) 2024-04-24T08:49:56.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:56.209+0000 7ff703400700 1 -- 172.21.15.83:0/2854523393 learned_addr learned my addr 172.21.15.83:0/2854523393 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:49:56.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:56.209+0000 7ff702a00700 1 --2- 172.21.15.83:0/2854523393 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff704075640 0x7ff70419b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:56.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:56.209+0000 7ff703e00700 1 --2- 172.21.15.83:0/2854523393 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff70410f4f0 0x7ff70419fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:56.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:56.209+0000 7ff703400700 1 -- 172.21.15.83:0/2854523393 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff70410f4f0 msgr2=0x7ff70419fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:56.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:56.209+0000 7ff703400700 1 --2- 172.21.15.83:0/2854523393 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff70410f4f0 0x7ff70419fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:56.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:56.209+0000 7ff703400700 1 -- 172.21.15.83:0/2854523393 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff704075640 msgr2=0x7ff70419b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:56.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:56.209+0000 7ff703400700 1 --2- 172.21.15.83:0/2854523393 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff704075640 0x7ff70419b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:56.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:56.209+0000 7ff703400700 1 -- 172.21.15.83:0/2854523393 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7041a0340 con 0x7ff704074d20 2024-04-24T08:49:56.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:56.209+0000 7ff703e00700 1 --2- 172.21.15.83:0/2854523393 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff70410f4f0 0x7ff70419fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:49:56.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:56.209+0000 7ff703400700 1 --2- 172.21.15.83:0/2854523393 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff704074d20 0x7ff70419b380 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7ff6f400c9b0 tx=0x7ff6f400cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:49:56.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:56.210+0000 7ff6fbe00700 1 -- 172.21.15.83:0/2854523393 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6f4007b00 con 0x7ff704074d20 2024-04-24T08:49:56.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:56.210+0000 7ff6fbe00700 1 -- 172.21.15.83:0/2854523393 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff6f4007ca0 con 0x7ff704074d20 2024-04-24T08:49:56.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:56.210+0000 7ff6fbe00700 1 -- 172.21.15.83:0/2854523393 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6f4018730 con 0x7ff704074d20 2024-04-24T08:49:56.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:56.210+0000 7ff70a400700 1 -- 172.21.15.83:0/2854523393 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7041a0630 con 0x7ff704074d20 2024-04-24T08:49:56.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:56.210+0000 7ff70a400700 1 -- 172.21.15.83:0/2854523393 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7041a7eb0 con 0x7ff704074d20 2024-04-24T08:49:56.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:56.211+0000 7ff70a400700 1 -- 172.21.15.83:0/2854523393 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff704063200 con 0x7ff704074d20 2024-04-24T08:49:56.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:56.214+0000 7ff6fbe00700 1 -- 172.21.15.83:0/2854523393 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ff6f400f450 con 0x7ff704074d20 2024-04-24T08:49:56.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:56.215+0000 7ff6fbe00700 1 --2- 172.21.15.83:0/2854523393 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7ff6e80a3d50 0x7ff6e80a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:56.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:56.215+0000 7ff6fbe00700 1 -- 172.21.15.83:0/2854523393 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7ff6f40d0d80 con 0x7ff704074d20 2024-04-24T08:49:56.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:56.215+0000 7ff702a00700 1 --2- 172.21.15.83:0/2854523393 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7ff6e80a3d50 0x7ff6e80a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:56.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:56.216+0000 7ff6fbe00700 1 --2- 172.21.15.83:0/2854523393 >> [v2:172.21.15.189:6808/1399817431,v1:172.21.15.189:6809/1399817431] conn(0x7ff6e80aeeb0 0x7ff6e80b1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:49:56.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:56.216+0000 7ff6fbe00700 1 -- 172.21.15.83:0/2854523393 --> [v2:172.21.15.189:6808/1399817431,v1:172.21.15.189:6809/1399817431] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff6e80b1a20 con 0x7ff6e80aeeb0 2024-04-24T08:49:56.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:56.216+0000 7ff6fbe00700 1 -- 172.21.15.83:0/2854523393 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7ff6f40d1130 con 0x7ff704074d20 2024-04-24T08:49:56.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:56.216+0000 7ff702a00700 1 --2- 172.21.15.83:0/2854523393 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7ff6e80a3d50 0x7ff6e80a6210 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff6f0005fd0 tx=0x7ff6f0005d90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:49:56.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:56.216+0000 7ff703e00700 1 --2- 172.21.15.83:0/2854523393 >> [v2:172.21.15.189:6808/1399817431,v1:172.21.15.189:6809/1399817431] conn(0x7ff6e80aeeb0 0x7ff6e80b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:49:56.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:56.218+0000 7ff703e00700 1 --2- 172.21.15.83:0/2854523393 >> [v2:172.21.15.189:6808/1399817431,v1:172.21.15.189:6809/1399817431] conn(0x7ff6e80aeeb0 0x7ff6e80b1350 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:49:56.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:56.220+0000 7ff6fbe00700 1 -- 172.21.15.83:0/2854523393 <== osd.5 v2:172.21.15.189:6808/1399817431 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff6e80b1a20 con 0x7ff6e80aeeb0 2024-04-24T08:49:56.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:56.240+0000 7ff70a400700 1 -- 172.21.15.83:0/2854523393 --> [v2:172.21.15.189:6808/1399817431,v1:172.21.15.189:6809/1399817431] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff704063680 con 0x7ff6e80aeeb0 2024-04-24T08:49:56.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:56.241+0000 7ff6fbe00700 1 -- 172.21.15.83:0/2854523393 <== osd.5 v2:172.21.15.189:6808/1399817431 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ff704063680 con 0x7ff6e80aeeb0 2024-04-24T08:49:56.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:56.242+0000 7ff70a400700 1 -- 172.21.15.83:0/2854523393 >> [v2:172.21.15.189:6808/1399817431,v1:172.21.15.189:6809/1399817431] conn(0x7ff6e80aeeb0 msgr2=0x7ff6e80b1350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:56.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:56.242+0000 7ff70a400700 1 --2- 172.21.15.83:0/2854523393 >> [v2:172.21.15.189:6808/1399817431,v1:172.21.15.189:6809/1399817431] conn(0x7ff6e80aeeb0 0x7ff6e80b1350 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:49:56.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:56.242+0000 7ff70a400700 1 -- 172.21.15.83:0/2854523393 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7ff6e80a3d50 msgr2=0x7ff6e80a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:56.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:56.242+0000 7ff70a400700 1 --2- 172.21.15.83:0/2854523393 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7ff6e80a3d50 0x7ff6e80a6210 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff6f0005fd0 tx=0x7ff6f0005d90 comp rx=0 tx=0).stop 2024-04-24T08:49:56.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:56.242+0000 7ff70a400700 1 -- 172.21.15.83:0/2854523393 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff704074d20 msgr2=0x7ff70419b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:49:56.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:56.242+0000 7ff70a400700 1 --2- 172.21.15.83:0/2854523393 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff704074d20 0x7ff70419b380 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7ff6f400c9b0 tx=0x7ff6f400cd70 comp rx=0 tx=0).stop 2024-04-24T08:49:56.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:56.242+0000 7ff703e00700 1 -- 172.21.15.83:0/2854523393 reap_dead start 2024-04-24T08:49:56.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:56.242+0000 7ff70a400700 1 -- 172.21.15.83:0/2854523393 shutdown_connections 2024-04-24T08:49:56.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:56.242+0000 7ff70a400700 1 -- 172.21.15.83:0/2854523393 >> 172.21.15.83:0/2854523393 conn(0x7ff704079ca0 msgr2=0x7ff7040ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:49:56.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:56.243+0000 7ff70a400700 1 -- 172.21.15.83:0/2854523393 shutdown_connections 2024-04-24T08:49:56.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:49:56.243+0000 7ff70a400700 1 -- 172.21.15.83:0/2854523393 wait complete. 2024-04-24T08:49:58.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:49:57 smithi083 ceph-mon[27907]: pgmap v419: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-24T08:49:58.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:49:57 smithi138 ceph-mon[31441]: pgmap v419: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-24T08:49:58.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:49:57 smithi189 ceph-mon[31791]: pgmap v419: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-24T08:49:58.615 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T08:49:58.617 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T08:49:58.618 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T08:49:58.621 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T08:49:58.630 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T08:49:58.641 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T08:49:58.643 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T08:49:58.647 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T08:49:58.681 INFO:teuthology.orchestra.run.smithi083.stdout:163208757325 2024-04-24T08:49:58.683 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid bd3e039e-0215-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-04-24T08:49:59.203 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T08:50:00.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:49:59 smithi083 ceph-mon[27907]: pgmap v420: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 10 op/s 2024-04-24T08:50:00.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:49:59 smithi189 ceph-mon[31791]: pgmap v420: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 10 op/s 2024-04-24T08:50:00.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:49:59 smithi138 ceph-mon[31441]: pgmap v420: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 10 op/s 2024-04-24T08:50:01.096 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:50:00 smithi083 ceph-mon[27907]: overall HEALTH_OK 2024-04-24T08:50:01.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:50:00 smithi138 ceph-mon[31441]: overall HEALTH_OK 2024-04-24T08:50:01.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:50:00 smithi189 ceph-mon[31791]: overall HEALTH_OK 2024-04-24T08:50:02.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:50:01 smithi083 ceph-mon[27907]: pgmap v421: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-24T08:50:02.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:50:01 smithi138 ceph-mon[31441]: pgmap v421: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-24T08:50:02.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:50:01 smithi189 ceph-mon[31791]: pgmap v421: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-24T08:50:04.094 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:50:03 smithi083 ceph-mon[27907]: pgmap v422: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-24T08:50:04.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:50:03 smithi138 ceph-mon[31441]: pgmap v422: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-24T08:50:04.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:50:03 smithi189 ceph-mon[31791]: pgmap v422: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-24T08:50:05.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:05.317+0000 7f94d1600700 1 -- 172.21.15.83:0/2653827719 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f94cc1099f0 msgr2=0x7f94cc115e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:05.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:05.317+0000 7f94d1600700 1 --2- 172.21.15.83:0/2653827719 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f94cc1099f0 0x7f94cc115e90 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f94c400b0a0 tx=0x7f94c400b3b0 comp rx=0 tx=0).stop 2024-04-24T08:50:05.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:05.317+0000 7f94d1600700 1 -- 172.21.15.83:0/2653827719 shutdown_connections 2024-04-24T08:50:05.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:05.317+0000 7f94d1600700 1 --2- 172.21.15.83:0/2653827719 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f94cc1099f0 0x7f94cc115e90 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:05.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:05.317+0000 7f94d1600700 1 --2- 172.21.15.83:0/2653827719 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f94cc10f0e0 0x7f94cc10f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:05.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:05.317+0000 7f94d1600700 1 --2- 172.21.15.83:0/2653827719 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f94cc109030 0x7f94cc1094b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:05.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:05.317+0000 7f94d1600700 1 -- 172.21.15.83:0/2653827719 >> 172.21.15.83:0/2653827719 conn(0x7f94cc078880 msgr2=0x7f94cc107910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:50:05.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:05.317+0000 7f94d1600700 1 -- 172.21.15.83:0/2653827719 shutdown_connections 2024-04-24T08:50:05.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:05.317+0000 7f94d1600700 1 -- 172.21.15.83:0/2653827719 wait complete. 2024-04-24T08:50:05.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:05.318+0000 7f94d1600700 1 Processor -- start 2024-04-24T08:50:05.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:05.318+0000 7f94d1600700 1 -- start start 2024-04-24T08:50:05.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:05.319+0000 7f94d1600700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f94cc109030 0x7f94cc19f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:05.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:05.319+0000 7f94d1600700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f94cc1099f0 0x7f94cc19faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:05.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:05.319+0000 7f94d1600700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f94cc10f0e0 0x7f94cc1a3e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:05.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:05.319+0000 7f94d1600700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f94cc1191c0 con 0x7f94cc1099f0 2024-04-24T08:50:05.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:05.319+0000 7f94d1600700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f94cc119040 con 0x7f94cc10f0e0 2024-04-24T08:50:05.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:05.319+0000 7f94d1600700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f94cc119340 con 0x7f94cc109030 2024-04-24T08:50:05.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:05.319+0000 7f94cb400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f94cc1099f0 0x7f94cc19faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:05.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:05.319+0000 7f94cb400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f94cc1099f0 0x7f94cc19faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41172/0 (socket says 172.21.15.83:41172) 2024-04-24T08:50:05.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:05.319+0000 7f94cb400700 1 -- 172.21.15.83:0/2498303457 learned_addr learned my addr 172.21.15.83:0/2498303457 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:50:05.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:05.319+0000 7f94d0c00700 1 --2- 172.21.15.83:0/2498303457 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f94cc10f0e0 0x7f94cc1a3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:05.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:05.319+0000 7f94cbe00700 1 --2- 172.21.15.83:0/2498303457 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f94cc109030 0x7f94cc19f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:05.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:05.319+0000 7f94cb400700 1 -- 172.21.15.83:0/2498303457 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f94cc109030 msgr2=0x7f94cc19f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:05.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:05.319+0000 7f94cb400700 1 --2- 172.21.15.83:0/2498303457 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f94cc109030 0x7f94cc19f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:05.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:05.319+0000 7f94cb400700 1 -- 172.21.15.83:0/2498303457 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f94cc10f0e0 msgr2=0x7f94cc1a3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:05.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:05.319+0000 7f94cb400700 1 --2- 172.21.15.83:0/2498303457 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f94cc10f0e0 0x7f94cc1a3e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:05.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:05.320+0000 7f94cb400700 1 -- 172.21.15.83:0/2498303457 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94cc1a45c0 con 0x7f94cc1099f0 2024-04-24T08:50:05.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:05.320+0000 7f94cb400700 1 --2- 172.21.15.83:0/2498303457 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f94cc1099f0 0x7f94cc19faf0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f94b800d950 tx=0x7f94b800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:50:05.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:05.320+0000 7f94c8c00700 1 -- 172.21.15.83:0/2498303457 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94b8016070 con 0x7f94cc1099f0 2024-04-24T08:50:05.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:05.320+0000 7f94c8c00700 1 -- 172.21.15.83:0/2498303457 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f94b8010460 con 0x7f94cc1099f0 2024-04-24T08:50:05.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:05.320+0000 7f94d1600700 1 -- 172.21.15.83:0/2498303457 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94cc1a48b0 con 0x7f94cc1099f0 2024-04-24T08:50:05.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:05.320+0000 7f94c8c00700 1 -- 172.21.15.83:0/2498303457 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94b800f620 con 0x7f94cc1099f0 2024-04-24T08:50:05.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:05.320+0000 7f94d1600700 1 -- 172.21.15.83:0/2498303457 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94cc1ac130 con 0x7f94cc1099f0 2024-04-24T08:50:05.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:05.321+0000 7f94d1600700 1 -- 172.21.15.83:0/2498303457 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94cc0697e0 con 0x7f94cc1099f0 2024-04-24T08:50:05.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:05.323+0000 7f94c8c00700 1 -- 172.21.15.83:0/2498303457 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f94b8019070 con 0x7f94cc1099f0 2024-04-24T08:50:05.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:05.326+0000 7f94c8c00700 1 --2- 172.21.15.83:0/2498303457 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f94b00a3cd0 0x7f94b00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:05.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:05.326+0000 7f94c8c00700 1 -- 172.21.15.83:0/2498303457 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f94b80d05c0 con 0x7f94cc1099f0 2024-04-24T08:50:05.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:05.326+0000 7f94cbe00700 1 --2- 172.21.15.83:0/2498303457 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f94b00a3cd0 0x7f94b00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:05.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:05.327+0000 7f94cbe00700 1 --2- 172.21.15.83:0/2498303457 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f94b00a3cd0 0x7f94b00a6190 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f94bc005950 tx=0x7f94bc0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:50:05.984 INFO:teuthology.orchestra.run.smithi083.stdout:253403070520 2024-04-24T08:50:05.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:05.331+0000 7f94c8c00700 1 -- 172.21.15.83:0/2498303457 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f94b809ca80 con 0x7f94cc1099f0 2024-04-24T08:50:05.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:05.562+0000 7f94d1600700 1 -- 172.21.15.83:0/2498303457 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f94cc02dfe0 con 0x7f94cc1099f0 2024-04-24T08:50:05.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:05.562+0000 7f94c8c00700 1 -- 172.21.15.83:0/2498303457 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f94b809c380 con 0x7f94cc1099f0 2024-04-24T08:50:05.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:05.565+0000 7f94d1600700 1 -- 172.21.15.83:0/2498303457 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f94b00a3cd0 msgr2=0x7f94b00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:05.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:05.565+0000 7f94d1600700 1 --2- 172.21.15.83:0/2498303457 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f94b00a3cd0 0x7f94b00a6190 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f94bc005950 tx=0x7f94bc0058e0 comp rx=0 tx=0).stop 2024-04-24T08:50:05.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:05.565+0000 7f94d1600700 1 -- 172.21.15.83:0/2498303457 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f94cc1099f0 msgr2=0x7f94cc19faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:05.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:05.565+0000 7f94d1600700 1 --2- 172.21.15.83:0/2498303457 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f94cc1099f0 0x7f94cc19faf0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f94b800d950 tx=0x7f94b800dd10 comp rx=0 tx=0).stop 2024-04-24T08:50:05.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:05.565+0000 7f94d1600700 1 -- 172.21.15.83:0/2498303457 shutdown_connections 2024-04-24T08:50:05.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:05.565+0000 7f94d1600700 1 --2- 172.21.15.83:0/2498303457 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f94b00a3cd0 0x7f94b00a6190 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:05.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:05.565+0000 7f94d1600700 1 --2- 172.21.15.83:0/2498303457 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f94cc10f0e0 0x7f94cc1a3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:05.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:05.565+0000 7f94d1600700 1 --2- 172.21.15.83:0/2498303457 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f94cc109030 0x7f94cc19f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:05.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:05.565+0000 7f94d1600700 1 --2- 172.21.15.83:0/2498303457 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f94cc1099f0 0x7f94cc19faf0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:05.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:05.565+0000 7f94d1600700 1 -- 172.21.15.83:0/2498303457 >> 172.21.15.83:0/2498303457 conn(0x7f94cc078880 msgr2=0x7f94cc079ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:50:05.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:05.566+0000 7f94d1600700 1 -- 172.21.15.83:0/2498303457 shutdown_connections 2024-04-24T08:50:05.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:05.566+0000 7f94d1600700 1 -- 172.21.15.83:0/2498303457 wait complete. 2024-04-24T08:50:06.258 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:50:05 smithi138 ceph-mon[31441]: pgmap v423: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-04-24T08:50:06.258 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:50:05 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/2498303457' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-24T08:50:06.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:50:05 smithi083 ceph-mon[27907]: pgmap v423: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-04-24T08:50:06.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:50:05 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/2498303457' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-24T08:50:06.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:50:05 smithi189 ceph-mon[31791]: pgmap v423: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-04-24T08:50:06.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:50:05 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/2498303457' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-24T08:50:07.124 INFO:tasks.cephadm.ceph_manager.ceph:need seq 253403070517 got 253403070520 for osd.8 2024-04-24T08:50:07.124 DEBUG:teuthology.parallel:result is None 2024-04-24T08:50:08.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:50:07 smithi083 ceph-mon[27907]: pgmap v424: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-24T08:50:08.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:50:07 smithi138 ceph-mon[31441]: pgmap v424: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-24T08:50:08.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:50:07 smithi189 ceph-mon[31791]: pgmap v424: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-24T08:50:10.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:50:09 smithi083 ceph-mon[27907]: pgmap v425: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-24T08:50:10.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:50:09 smithi138 ceph-mon[31441]: pgmap v425: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-24T08:50:10.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:50:09 smithi189 ceph-mon[31791]: pgmap v425: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-24T08:50:11.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:11.794+0000 7f2f90a00700 1 -- 172.21.15.83:0/2301890011 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f8c1033e0 msgr2=0x7f2f8c10f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:11.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:11.794+0000 7f2f90a00700 1 --2- 172.21.15.83:0/2301890011 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f8c1033e0 0x7f2f8c10f8f0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f2f8400b0a0 tx=0x7f2f8400b3b0 comp rx=0 tx=0).stop 2024-04-24T08:50:11.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:11.794+0000 7f2f90a00700 1 -- 172.21.15.83:0/2301890011 shutdown_connections 2024-04-24T08:50:11.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:11.794+0000 7f2f90a00700 1 --2- 172.21.15.83:0/2301890011 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f8c1033e0 0x7f2f8c10f8f0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:11.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:11.794+0000 7f2f90a00700 1 --2- 172.21.15.83:0/2301890011 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2f8c108a20 0x7f2f8c108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:11.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:11.794+0000 7f2f90a00700 1 --2- 172.21.15.83:0/2301890011 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f8c102a20 0x7f2f8c102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:11.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:11.794+0000 7f2f90a00700 1 -- 172.21.15.83:0/2301890011 >> 172.21.15.83:0/2301890011 conn(0x7f2f8c0fe570 msgr2=0x7f2f8c100990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:50:11.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:11.795+0000 7f2f90a00700 1 -- 172.21.15.83:0/2301890011 shutdown_connections 2024-04-24T08:50:11.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:11.795+0000 7f2f90a00700 1 -- 172.21.15.83:0/2301890011 wait complete. 2024-04-24T08:50:11.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:11.796+0000 7f2f90a00700 1 Processor -- start 2024-04-24T08:50:11.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:11.796+0000 7f2f90a00700 1 -- start start 2024-04-24T08:50:11.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:11.796+0000 7f2f90a00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f8c102a20 0x7f2f8c19b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:11.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:11.796+0000 7f2f90a00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2f8c1033e0 0x7f2f8c19b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:11.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:11.796+0000 7f2f90a00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f8c108a20 0x7f2f8c19fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:11.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:11.796+0000 7f2f90a00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2f8c112c50 con 0x7f2f8c108a20 2024-04-24T08:50:11.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:11.796+0000 7f2f90a00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2f8c112ad0 con 0x7f2f8c102a20 2024-04-24T08:50:11.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:11.796+0000 7f2f90a00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2f8c112dd0 con 0x7f2f8c1033e0 2024-04-24T08:50:11.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:11.797+0000 7f2f8be00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f8c108a20 0x7f2f8c19fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:11.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:11.797+0000 7f2f8be00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f8c108a20 0x7f2f8c19fb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58262/0 (socket says 172.21.15.83:58262) 2024-04-24T08:50:11.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:11.797+0000 7f2f8be00700 1 -- 172.21.15.83:0/1901707693 learned_addr learned my addr 172.21.15.83:0/1901707693 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:50:11.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:11.797+0000 7f2f8aa00700 1 --2- 172.21.15.83:0/1901707693 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2f8c1033e0 0x7f2f8c19b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:11.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:11.797+0000 7f2f8b400700 1 --2- 172.21.15.83:0/1901707693 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f8c102a20 0x7f2f8c19b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:11.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:11.797+0000 7f2f8be00700 1 -- 172.21.15.83:0/1901707693 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2f8c1033e0 msgr2=0x7f2f8c19b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:11.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:11.797+0000 7f2f8be00700 1 --2- 172.21.15.83:0/1901707693 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2f8c1033e0 0x7f2f8c19b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:11.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:11.797+0000 7f2f8be00700 1 -- 172.21.15.83:0/1901707693 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f8c102a20 msgr2=0x7f2f8c19b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:11.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:11.797+0000 7f2f8be00700 1 --2- 172.21.15.83:0/1901707693 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f8c102a20 0x7f2f8c19b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:11.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:11.797+0000 7f2f8be00700 1 -- 172.21.15.83:0/1901707693 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f8c1a0270 con 0x7f2f8c108a20 2024-04-24T08:50:11.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:11.797+0000 7f2f8b400700 1 --2- 172.21.15.83:0/1901707693 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f8c102a20 0x7f2f8c19b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:50:11.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:11.797+0000 7f2f8be00700 1 --2- 172.21.15.83:0/1901707693 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f8c108a20 0x7f2f8c19fb30 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f2f84009fd0 tx=0x7f2f84004e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:50:11.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:11.798+0000 7f2f83e00700 1 -- 172.21.15.83:0/1901707693 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f84021070 con 0x7f2f8c108a20 2024-04-24T08:50:11.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:11.798+0000 7f2f90a00700 1 -- 172.21.15.83:0/1901707693 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f8c1a0500 con 0x7f2f8c108a20 2024-04-24T08:50:11.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:11.798+0000 7f2f83e00700 1 -- 172.21.15.83:0/1901707693 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2f84007c80 con 0x7f2f8c108a20 2024-04-24T08:50:11.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:11.798+0000 7f2f83e00700 1 -- 172.21.15.83:0/1901707693 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f84005280 con 0x7f2f8c108a20 2024-04-24T08:50:11.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:11.798+0000 7f2f90a00700 1 -- 172.21.15.83:0/1901707693 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f8c1a7de0 con 0x7f2f8c108a20 2024-04-24T08:50:11.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:11.799+0000 7f2f90a00700 1 -- 172.21.15.83:0/1901707693 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f8c0697e0 con 0x7f2f8c108a20 2024-04-24T08:50:11.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:11.801+0000 7f2f83e00700 1 -- 172.21.15.83:0/1901707693 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f2f84024070 con 0x7f2f8c108a20 2024-04-24T08:50:11.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:11.802+0000 7f2f83e00700 1 --2- 172.21.15.83:0/1901707693 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f2f700a3cd0 0x7f2f700a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:11.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:11.802+0000 7f2f83e00700 1 -- 172.21.15.83:0/1901707693 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f2f840d1ab0 con 0x7f2f8c108a20 2024-04-24T08:50:11.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:11.803+0000 7f2f8b400700 1 --2- 172.21.15.83:0/1901707693 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f2f700a3cd0 0x7f2f700a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:11.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:11.803+0000 7f2f8b400700 1 --2- 172.21.15.83:0/1901707693 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f2f700a3cd0 0x7f2f700a6190 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c005e90 tx=0x7f2f7c005d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:50:11.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:11.807+0000 7f2f83e00700 1 -- 172.21.15.83:0/1901707693 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2f8409e000 con 0x7f2f8c108a20 2024-04-24T08:50:12.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:12.028+0000 7f2f83e00700 1 -- 172.21.15.83:0/1901707693 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f2f84099f70 con 0x7f2f8c108a20 2024-04-24T08:50:12.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:12.049+0000 7f2f90a00700 1 -- 172.21.15.83:0/1901707693 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f2f8c19c5c0 con 0x7f2f8c108a20 2024-04-24T08:50:12.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:12.050+0000 7f2f83e00700 1 -- 172.21.15.83:0/1901707693 <== mon.0 v2:172.21.15.83:3300/0 8 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f2f8c19c5c0 con 0x7f2f8c108a20 2024-04-24T08:50:12.050 INFO:teuthology.orchestra.run.smithi083.stdout:219043332162 2024-04-24T08:50:12.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:12.052+0000 7f2f90a00700 1 -- 172.21.15.83:0/1901707693 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f2f700a3cd0 msgr2=0x7f2f700a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:12.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:12.052+0000 7f2f90a00700 1 --2- 172.21.15.83:0/1901707693 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f2f700a3cd0 0x7f2f700a6190 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c005e90 tx=0x7f2f7c005d20 comp rx=0 tx=0).stop 2024-04-24T08:50:12.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:12.052+0000 7f2f90a00700 1 -- 172.21.15.83:0/1901707693 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f8c108a20 msgr2=0x7f2f8c19fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:12.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:12.052+0000 7f2f90a00700 1 --2- 172.21.15.83:0/1901707693 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f8c108a20 0x7f2f8c19fb30 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f2f84009fd0 tx=0x7f2f84004e70 comp rx=0 tx=0).stop 2024-04-24T08:50:12.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:12.052+0000 7f2f90a00700 1 -- 172.21.15.83:0/1901707693 shutdown_connections 2024-04-24T08:50:12.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:12.053+0000 7f2f90a00700 1 --2- 172.21.15.83:0/1901707693 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f2f700a3cd0 0x7f2f700a6190 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:12.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:12.053+0000 7f2f90a00700 1 --2- 172.21.15.83:0/1901707693 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f8c108a20 0x7f2f8c19fb30 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:12.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:12.053+0000 7f2f90a00700 1 --2- 172.21.15.83:0/1901707693 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f8c102a20 0x7f2f8c19b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:12.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:12.053+0000 7f2f90a00700 1 --2- 172.21.15.83:0/1901707693 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2f8c1033e0 0x7f2f8c19b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:12.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:12.053+0000 7f2f90a00700 1 -- 172.21.15.83:0/1901707693 >> 172.21.15.83:0/1901707693 conn(0x7f2f8c0fe570 msgr2=0x7f2f8c10dce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:50:12.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:12.053+0000 7f2f90a00700 1 -- 172.21.15.83:0/1901707693 shutdown_connections 2024-04-24T08:50:12.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:12.053+0000 7f2f90a00700 1 -- 172.21.15.83:0/1901707693 wait complete. 2024-04-24T08:50:12.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:50:12 smithi083 ceph-mon[27907]: pgmap v426: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-24T08:50:12.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:50:12 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/3000411394' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-24T08:50:12.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:50:12 smithi138 ceph-mon[31441]: pgmap v426: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-24T08:50:12.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:50:12 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/3000411394' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-24T08:50:12.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:50:12 smithi189 ceph-mon[31791]: pgmap v426: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-24T08:50:12.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:50:12 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/3000411394' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-24T08:50:13.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:50:13 smithi083 ceph-mon[27907]: mgrmap e18: x(active, since 11m), standbys: z, y 2024-04-24T08:50:13.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:50:13 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/1901707693' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-24T08:50:13.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:50:13 smithi083 ceph-mon[27907]: pgmap v427: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-24T08:50:13.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:10.943+0000 7fbd5da00700 1 -- 172.21.15.83:0/2881746560 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd58108b90 msgr2=0x7fbd58108f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:13.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:10.943+0000 7fbd5da00700 1 --2- 172.21.15.83:0/2881746560 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd58108b90 0x7fbd58108f70 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fbd48009990 tx=0x7fbd48009ca0 comp rx=0 tx=0).stop 2024-04-24T08:50:13.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:10.943+0000 7fbd5da00700 1 -- 172.21.15.83:0/2881746560 shutdown_connections 2024-04-24T08:50:13.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:10.943+0000 7fbd5da00700 1 --2- 172.21.15.83:0/2881746560 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd58108b90 0x7fbd58108f70 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:13.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:10.943+0000 7fbd5da00700 1 --2- 172.21.15.83:0/2881746560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd58103500 0x7fbd5810f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:13.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:10.943+0000 7fbd5da00700 1 --2- 172.21.15.83:0/2881746560 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbd58102b40 0x7fbd58102fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:13.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:10.943+0000 7fbd5da00700 1 -- 172.21.15.83:0/2881746560 >> 172.21.15.83:0/2881746560 conn(0x7fbd58079c50 msgr2=0x7fbd5807a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:50:13.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:10.943+0000 7fbd5da00700 1 -- 172.21.15.83:0/2881746560 shutdown_connections 2024-04-24T08:50:13.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:10.943+0000 7fbd5da00700 1 -- 172.21.15.83:0/2881746560 wait complete. 2024-04-24T08:50:13.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:10.944+0000 7fbd5da00700 1 Processor -- start 2024-04-24T08:50:13.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:10.945+0000 7fbd5da00700 1 -- start start 2024-04-24T08:50:13.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:10.945+0000 7fbd5da00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbd58102b40 0x7fbd5819b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:13.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:10.945+0000 7fbd5da00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd58103500 0x7fbd5819b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:13.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:10.945+0000 7fbd5da00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd58108b90 0x7fbd5819fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:13.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:10.945+0000 7fbd5da00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbd58112d10 con 0x7fbd58108b90 2024-04-24T08:50:13.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:10.945+0000 7fbd5da00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbd58112b90 con 0x7fbd58103500 2024-04-24T08:50:13.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:10.945+0000 7fbd5da00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fbd58112e90 con 0x7fbd58102b40 2024-04-24T08:50:13.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:10.945+0000 7fbd57400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd58108b90 0x7fbd5819fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:13.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:10.945+0000 7fbd57400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd58108b90 0x7fbd5819fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58242/0 (socket says 172.21.15.83:58242) 2024-04-24T08:50:13.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:10.945+0000 7fbd57400700 1 -- 172.21.15.83:0/3000411394 learned_addr learned my addr 172.21.15.83:0/3000411394 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:50:13.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:10.945+0000 7fbd4fe00700 1 --2- 172.21.15.83:0/3000411394 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd58103500 0x7fbd5819b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:13.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:10.946+0000 7fbd56a00700 1 --2- 172.21.15.83:0/3000411394 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbd58102b40 0x7fbd5819b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:13.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:10.946+0000 7fbd57400700 1 -- 172.21.15.83:0/3000411394 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbd58102b40 msgr2=0x7fbd5819b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:13.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:10.946+0000 7fbd57400700 1 --2- 172.21.15.83:0/3000411394 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbd58102b40 0x7fbd5819b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:13.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:10.946+0000 7fbd57400700 1 -- 172.21.15.83:0/3000411394 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd58103500 msgr2=0x7fbd5819b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:13.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:10.946+0000 7fbd57400700 1 --2- 172.21.15.83:0/3000411394 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd58103500 0x7fbd5819b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:13.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:10.946+0000 7fbd57400700 1 -- 172.21.15.83:0/3000411394 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd581a0450 con 0x7fbd58108b90 2024-04-24T08:50:13.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:10.946+0000 7fbd57400700 1 --2- 172.21.15.83:0/3000411394 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd58108b90 0x7fbd5819fd10 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fbd50007ee0 tx=0x7fbd5000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:50:13.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:10.947+0000 7fbd4f400700 1 -- 172.21.15.83:0/3000411394 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd50004bb0 con 0x7fbd58108b90 2024-04-24T08:50:13.386 INFO:teuthology.orchestra.run.smithi083.stdout:77309411428 2024-04-24T08:50:13.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:10.947+0000 7fbd4f400700 1 -- 172.21.15.83:0/3000411394 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbd50004d50 con 0x7fbd58108b90 2024-04-24T08:50:13.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:10.947+0000 7fbd5da00700 1 -- 172.21.15.83:0/3000411394 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd581a0740 con 0x7fbd58108b90 2024-04-24T08:50:13.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:10.947+0000 7fbd4f400700 1 -- 172.21.15.83:0/3000411394 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd500058e0 con 0x7fbd58108b90 2024-04-24T08:50:13.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:10.947+0000 7fbd5da00700 1 -- 172.21.15.83:0/3000411394 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd581a7fc0 con 0x7fbd58108b90 2024-04-24T08:50:13.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:10.948+0000 7fbd5da00700 1 -- 172.21.15.83:0/3000411394 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd580697e0 con 0x7fbd58108b90 2024-04-24T08:50:13.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:10.950+0000 7fbd4f400700 1 -- 172.21.15.83:0/3000411394 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fbd5001a020 con 0x7fbd58108b90 2024-04-24T08:50:13.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:10.951+0000 7fbd4f400700 1 --2- 172.21.15.83:0/3000411394 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fbd3c0a3d50 0x7fbd3c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:13.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:10.952+0000 7fbd4f400700 1 -- 172.21.15.83:0/3000411394 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fbd500d0520 con 0x7fbd58108b90 2024-04-24T08:50:13.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:10.952+0000 7fbd56a00700 1 --2- 172.21.15.83:0/3000411394 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fbd3c0a3d50 0x7fbd3c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:13.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:10.953+0000 7fbd56a00700 1 --2- 172.21.15.83:0/3000411394 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fbd3c0a3d50 0x7fbd3c0a6210 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fbd48006010 tx=0x7fbd4800b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:50:13.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:10.955+0000 7fbd4f400700 1 -- 172.21.15.83:0/3000411394 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbd50010070 con 0x7fbd58108b90 2024-04-24T08:50:13.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:11.190+0000 7fbd5da00700 1 -- 172.21.15.83:0/3000411394 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fbd5802dfe0 con 0x7fbd58108b90 2024-04-24T08:50:13.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:11.191+0000 7fbd4f400700 1 -- 172.21.15.83:0/3000411394 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fbd5009ca40 con 0x7fbd58108b90 2024-04-24T08:50:13.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:11.194+0000 7fbd5da00700 1 -- 172.21.15.83:0/3000411394 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fbd3c0a3d50 msgr2=0x7fbd3c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:13.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:11.194+0000 7fbd5da00700 1 --2- 172.21.15.83:0/3000411394 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fbd3c0a3d50 0x7fbd3c0a6210 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fbd48006010 tx=0x7fbd4800b540 comp rx=0 tx=0).stop 2024-04-24T08:50:13.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:11.194+0000 7fbd5da00700 1 -- 172.21.15.83:0/3000411394 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd58108b90 msgr2=0x7fbd5819fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:13.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:11.194+0000 7fbd5da00700 1 --2- 172.21.15.83:0/3000411394 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd58108b90 0x7fbd5819fd10 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fbd50007ee0 tx=0x7fbd5000d3b0 comp rx=0 tx=0).stop 2024-04-24T08:50:13.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:11.194+0000 7fbd5da00700 1 -- 172.21.15.83:0/3000411394 shutdown_connections 2024-04-24T08:50:13.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:11.194+0000 7fbd5da00700 1 --2- 172.21.15.83:0/3000411394 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fbd3c0a3d50 0x7fbd3c0a6210 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:13.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:11.194+0000 7fbd5da00700 1 --2- 172.21.15.83:0/3000411394 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd58108b90 0x7fbd5819fd10 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:13.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:11.194+0000 7fbd5da00700 1 --2- 172.21.15.83:0/3000411394 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbd58102b40 0x7fbd5819b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:13.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:11.194+0000 7fbd5da00700 1 --2- 172.21.15.83:0/3000411394 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd58103500 0x7fbd5819b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:13.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:11.194+0000 7fbd5da00700 1 -- 172.21.15.83:0/3000411394 >> 172.21.15.83:0/3000411394 conn(0x7fbd58079c50 msgr2=0x7fbd5810e030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:50:13.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:11.195+0000 7fbd5da00700 1 -- 172.21.15.83:0/3000411394 shutdown_connections 2024-04-24T08:50:13.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:11.195+0000 7fbd5da00700 1 -- 172.21.15.83:0/3000411394 wait complete. 2024-04-24T08:50:13.389 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:50:13 smithi138 ceph-mon[31441]: mgrmap e18: x(active, since 11m), standbys: z, y 2024-04-24T08:50:13.389 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:50:13 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/1901707693' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-24T08:50:13.389 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:50:13 smithi138 ceph-mon[31441]: pgmap v427: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-24T08:50:13.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:50:13 smithi189 ceph-mon[31791]: mgrmap e18: x(active, since 11m), standbys: z, y 2024-04-24T08:50:13.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:50:13 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/1901707693' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-24T08:50:13.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:50:13 smithi189 ceph-mon[31791]: pgmap v427: 129 pgs: 129 active+clean; 643 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-24T08:50:14.613 INFO:tasks.cephadm.ceph_manager.ceph:need seq 219043332156 got 219043332162 for osd.7 2024-04-24T08:50:14.613 DEBUG:teuthology.parallel:result is None 2024-04-24T08:50:15.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:15.637+0000 7f22aa000700 1 -- 172.21.15.83:0/501865343 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22a4105920 msgr2=0x7f22a4109970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:15.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:15.637+0000 7f22aa000700 1 --2- 172.21.15.83:0/501865343 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22a4105920 0x7f22a4109970 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f22900099c0 tx=0x7f2290009cd0 comp rx=0 tx=0).stop 2024-04-24T08:50:15.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:15.637+0000 7f22aa000700 1 -- 172.21.15.83:0/501865343 shutdown_connections 2024-04-24T08:50:15.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:15.637+0000 7f22aa000700 1 --2- 172.21.15.83:0/501865343 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f22a410a050 0x7f22a4111c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:15.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:15.637+0000 7f22aa000700 1 --2- 172.21.15.83:0/501865343 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f22a4104f70 0x7f22a4105350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:15.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:15.637+0000 7f22aa000700 1 --2- 172.21.15.83:0/501865343 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22a4105920 0x7f22a4109970 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:15.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:15.637+0000 7f22aa000700 1 -- 172.21.15.83:0/501865343 >> 172.21.15.83:0/501865343 conn(0x7f22a4100820 msgr2=0x7f22a4102c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:50:15.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:15.637+0000 7f22aa000700 1 -- 172.21.15.83:0/501865343 shutdown_connections 2024-04-24T08:50:15.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:15.637+0000 7f22aa000700 1 -- 172.21.15.83:0/501865343 wait complete. 2024-04-24T08:50:15.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:15.638+0000 7f22aa000700 1 Processor -- start 2024-04-24T08:50:15.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:15.639+0000 7f22aa000700 1 -- start start 2024-04-24T08:50:15.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:15.639+0000 7f22aa000700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f22a4104f70 0x7f22a419b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:15.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:15.639+0000 7f22aa000700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f22a4105920 0x7f22a419ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:15.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:15.639+0000 7f22aa000700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22a410a050 0x7f22a419fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:15.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:15.639+0000 7f22aa000700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f22a4114f30 con 0x7f22a410a050 2024-04-24T08:50:15.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:15.639+0000 7f22aa000700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f22a4114db0 con 0x7f22a4104f70 2024-04-24T08:50:15.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:15.639+0000 7f22aa000700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f22a41150b0 con 0x7f22a4105920 2024-04-24T08:50:15.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:15.640+0000 7f22a3e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22a410a050 0x7f22a419fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:15.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:15.640+0000 7f22a3e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22a410a050 0x7f22a419fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58312/0 (socket says 172.21.15.83:58312) 2024-04-24T08:50:15.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:15.640+0000 7f22a3e00700 1 -- 172.21.15.83:0/945817399 learned_addr learned my addr 172.21.15.83:0/945817399 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:50:15.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:15.640+0000 7f22a2a00700 1 --2- 172.21.15.83:0/945817399 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f22a4105920 0x7f22a419ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:15.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:15.640+0000 7f22a3400700 1 --2- 172.21.15.83:0/945817399 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f22a4104f70 0x7f22a419b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:15.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:15.640+0000 7f22a3e00700 1 -- 172.21.15.83:0/945817399 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f22a4105920 msgr2=0x7f22a419ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:15.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:15.640+0000 7f22a3e00700 1 --2- 172.21.15.83:0/945817399 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f22a4105920 0x7f22a419ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:15.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:15.640+0000 7f22a3e00700 1 -- 172.21.15.83:0/945817399 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f22a4104f70 msgr2=0x7f22a419b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:15.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:15.640+0000 7f22a3e00700 1 --2- 172.21.15.83:0/945817399 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f22a4104f70 0x7f22a419b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:15.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:15.640+0000 7f22a3e00700 1 -- 172.21.15.83:0/945817399 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22a41a0480 con 0x7f22a410a050 2024-04-24T08:50:15.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:15.641+0000 7f22a3e00700 1 --2- 172.21.15.83:0/945817399 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22a410a050 0x7f22a419fd40 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f229c007ee0 tx=0x7f229c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:50:15.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:15.641+0000 7f229be00700 1 -- 172.21.15.83:0/945817399 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f229c00dd20 con 0x7f22a410a050 2024-04-24T08:50:15.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:15.641+0000 7f229be00700 1 -- 172.21.15.83:0/945817399 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f229c004020 con 0x7f22a410a050 2024-04-24T08:50:15.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:15.641+0000 7f229be00700 1 -- 172.21.15.83:0/945817399 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f229c015660 con 0x7f22a410a050 2024-04-24T08:50:15.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:15.641+0000 7f22aa000700 1 -- 172.21.15.83:0/945817399 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22a41a0770 con 0x7f22a410a050 2024-04-24T08:50:15.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:15.641+0000 7f22aa000700 1 -- 172.21.15.83:0/945817399 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22a41a7ff0 con 0x7f22a410a050 2024-04-24T08:50:15.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:15.643+0000 7f22aa000700 1 -- 172.21.15.83:0/945817399 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22a40697e0 con 0x7f22a410a050 2024-04-24T08:50:15.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:15.644+0000 7f229be00700 1 -- 172.21.15.83:0/945817399 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f229c0041d0 con 0x7f22a410a050 2024-04-24T08:50:15.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:15.646+0000 7f229be00700 1 --2- 172.21.15.83:0/945817399 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f22880a3cc0 0x7f22880a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:15.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:15.646+0000 7f229be00700 1 -- 172.21.15.83:0/945817399 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f229c0cffd0 con 0x7f22a410a050 2024-04-24T08:50:15.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:15.646+0000 7f22a3400700 1 --2- 172.21.15.83:0/945817399 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f22880a3cc0 0x7f22880a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:15.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:15.647+0000 7f22a3400700 1 --2- 172.21.15.83:0/945817399 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f22880a3cc0 0x7f22880a6180 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f2294004630 tx=0x7f229400b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:50:15.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:15.651+0000 7f229be00700 1 -- 172.21.15.83:0/945817399 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f229c010070 con 0x7f22a410a050 2024-04-24T08:50:15.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:15.912+0000 7f22aa000700 1 -- 172.21.15.83:0/945817399 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f22a402dfe0 con 0x7f22a410a050 2024-04-24T08:50:15.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:15.912+0000 7f229be00700 1 -- 172.21.15.83:0/945817399 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f229c09c5f0 con 0x7f22a410a050 2024-04-24T08:50:15.912 INFO:teuthology.orchestra.run.smithi083.stdout:287762808884 2024-04-24T08:50:15.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:15.916+0000 7f22aa000700 1 -- 172.21.15.83:0/945817399 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f22880a3cc0 msgr2=0x7f22880a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:15.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:15.916+0000 7f22aa000700 1 --2- 172.21.15.83:0/945817399 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f22880a3cc0 0x7f22880a6180 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f2294004630 tx=0x7f229400b3c0 comp rx=0 tx=0).stop 2024-04-24T08:50:15.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:15.916+0000 7f22aa000700 1 -- 172.21.15.83:0/945817399 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22a410a050 msgr2=0x7f22a419fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:15.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:15.916+0000 7f22aa000700 1 --2- 172.21.15.83:0/945817399 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22a410a050 0x7f22a419fd40 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f229c007ee0 tx=0x7f229c00d3b0 comp rx=0 tx=0).stop 2024-04-24T08:50:15.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:15.916+0000 7f22aa000700 1 -- 172.21.15.83:0/945817399 shutdown_connections 2024-04-24T08:50:15.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:15.916+0000 7f22aa000700 1 --2- 172.21.15.83:0/945817399 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f22880a3cc0 0x7f22880a6180 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:15.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:15.916+0000 7f22aa000700 1 --2- 172.21.15.83:0/945817399 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22a410a050 0x7f22a419fd40 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:15.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:15.916+0000 7f22aa000700 1 --2- 172.21.15.83:0/945817399 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f22a4104f70 0x7f22a419b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:15.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:15.916+0000 7f22aa000700 1 --2- 172.21.15.83:0/945817399 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f22a4105920 0x7f22a419ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:15.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:15.916+0000 7f22aa000700 1 -- 172.21.15.83:0/945817399 >> 172.21.15.83:0/945817399 conn(0x7f22a4100820 msgr2=0x7f22a4102bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:50:15.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:15.916+0000 7f22aa000700 1 -- 172.21.15.83:0/945817399 shutdown_connections 2024-04-24T08:50:15.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:15.916+0000 7f22aa000700 1 -- 172.21.15.83:0/945817399 wait complete. 2024-04-24T08:50:16.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:50:15 smithi138 ceph-mon[31441]: pgmap v428: 129 pgs: 129 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 426 B/s wr, 11 op/s 2024-04-24T08:50:16.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:50:15 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/599051359' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-24T08:50:16.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:50:15 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/945817399' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-24T08:50:16.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:50:15 smithi189 ceph-mon[31791]: pgmap v428: 129 pgs: 129 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 426 B/s wr, 11 op/s 2024-04-24T08:50:16.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:50:15 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/599051359' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-24T08:50:16.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:50:15 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/945817399' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-24T08:50:16.242 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:50:15 smithi083 ceph-mon[27907]: pgmap v428: 129 pgs: 129 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 426 B/s wr, 11 op/s 2024-04-24T08:50:16.243 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:50:15 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/599051359' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-24T08:50:16.243 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:50:15 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/945817399' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-24T08:50:16.305 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411424 got 77309411428 for osd.2 2024-04-24T08:50:16.305 DEBUG:teuthology.parallel:result is None 2024-04-24T08:50:16.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:16.831+0000 7fb205600700 1 -- 172.21.15.83:0/3193823507 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb200105940 msgr2=0x7fb200109990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:16.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:16.831+0000 7fb205600700 1 --2- 172.21.15.83:0/3193823507 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb200105940 0x7fb200109990 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fb1ec0099c0 tx=0x7fb1ec009cd0 comp rx=0 tx=0).stop 2024-04-24T08:50:16.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:16.831+0000 7fb205600700 1 -- 172.21.15.83:0/3193823507 shutdown_connections 2024-04-24T08:50:16.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:16.831+0000 7fb205600700 1 --2- 172.21.15.83:0/3193823507 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb20010a070 0x7fb200111c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:16.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:16.831+0000 7fb205600700 1 --2- 172.21.15.83:0/3193823507 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb200104f90 0x7fb200105370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:16.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:16.831+0000 7fb205600700 1 --2- 172.21.15.83:0/3193823507 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb200105940 0x7fb200109990 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:16.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:16.831+0000 7fb205600700 1 -- 172.21.15.83:0/3193823507 >> 172.21.15.83:0/3193823507 conn(0x7fb200100820 msgr2=0x7fb200102c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:50:16.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:16.831+0000 7fb205600700 1 -- 172.21.15.83:0/3193823507 shutdown_connections 2024-04-24T08:50:16.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:16.831+0000 7fb205600700 1 -- 172.21.15.83:0/3193823507 wait complete. 2024-04-24T08:50:16.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:16.832+0000 7fb205600700 1 Processor -- start 2024-04-24T08:50:16.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:16.833+0000 7fb205600700 1 -- start start 2024-04-24T08:50:16.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:16.833+0000 7fb205600700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb200104f90 0x7fb20019b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:16.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:16.833+0000 7fb205600700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb200105940 0x7fb20019b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:16.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:16.833+0000 7fb205600700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb20010a070 0x7fb20019fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:16.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:16.833+0000 7fb205600700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb200114f50 con 0x7fb20010a070 2024-04-24T08:50:16.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:16.833+0000 7fb205600700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb200114dd0 con 0x7fb200105940 2024-04-24T08:50:16.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:16.833+0000 7fb205600700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb2001150d0 con 0x7fb200104f90 2024-04-24T08:50:16.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:16.833+0000 7fb1ff400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb20010a070 0x7fb20019fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:16.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:16.833+0000 7fb1ff400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb20010a070 0x7fb20019fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58340/0 (socket says 172.21.15.83:58340) 2024-04-24T08:50:16.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:16.833+0000 7fb1ff400700 1 -- 172.21.15.83:0/1517682806 learned_addr learned my addr 172.21.15.83:0/1517682806 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:50:16.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:16.834+0000 7fb1fe000700 1 --2- 172.21.15.83:0/1517682806 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb200105940 0x7fb20019b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:16.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:16.834+0000 7fb1fea00700 1 --2- 172.21.15.83:0/1517682806 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb200104f90 0x7fb20019b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:16.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:16.834+0000 7fb1ff400700 1 -- 172.21.15.83:0/1517682806 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb200104f90 msgr2=0x7fb20019b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:16.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:16.834+0000 7fb1ff400700 1 --2- 172.21.15.83:0/1517682806 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb200104f90 0x7fb20019b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:16.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:16.834+0000 7fb1ff400700 1 -- 172.21.15.83:0/1517682806 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb200105940 msgr2=0x7fb20019b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:16.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:16.834+0000 7fb1ff400700 1 --2- 172.21.15.83:0/1517682806 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb200105940 0x7fb20019b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:16.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:16.834+0000 7fb1ff400700 1 -- 172.21.15.83:0/1517682806 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2001a0430 con 0x7fb20010a070 2024-04-24T08:50:16.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:16.834+0000 7fb1ff400700 1 --2- 172.21.15.83:0/1517682806 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb20010a070 0x7fb20019fcf0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fb1f8007d70 tx=0x7fb1f800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:50:16.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:16.835+0000 7fb1f7400700 1 -- 172.21.15.83:0/1517682806 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1f8016020 con 0x7fb20010a070 2024-04-24T08:50:16.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:16.835+0000 7fb1f7400700 1 -- 172.21.15.83:0/1517682806 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb1f800ad40 con 0x7fb20010a070 2024-04-24T08:50:16.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:16.835+0000 7fb1f7400700 1 -- 172.21.15.83:0/1517682806 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1f8011620 con 0x7fb20010a070 2024-04-24T08:50:16.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:16.835+0000 7fb205600700 1 -- 172.21.15.83:0/1517682806 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2001a0720 con 0x7fb20010a070 2024-04-24T08:50:16.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:16.835+0000 7fb205600700 1 -- 172.21.15.83:0/1517682806 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2001a7fa0 con 0x7fb20010a070 2024-04-24T08:50:16.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:16.836+0000 7fb205600700 1 -- 172.21.15.83:0/1517682806 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2000697e0 con 0x7fb20010a070 2024-04-24T08:50:16.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:16.838+0000 7fb1f7400700 1 -- 172.21.15.83:0/1517682806 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fb1f8019020 con 0x7fb20010a070 2024-04-24T08:50:16.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:16.840+0000 7fb1f7400700 1 --2- 172.21.15.83:0/1517682806 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fb1e40a3c40 0x7fb1e40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:16.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:16.840+0000 7fb1f7400700 1 -- 172.21.15.83:0/1517682806 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fb1f80d0740 con 0x7fb20010a070 2024-04-24T08:50:16.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:16.840+0000 7fb1fea00700 1 --2- 172.21.15.83:0/1517682806 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fb1e40a3c40 0x7fb1e40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:16.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:16.841+0000 7fb1fea00700 1 --2- 172.21.15.83:0/1517682806 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fb1e40a3c40 0x7fb1e40a6100 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb1f0005950 tx=0x7fb1f00058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:50:16.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:16.845+0000 7fb1f7400700 1 -- 172.21.15.83:0/1517682806 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb1f8020020 con 0x7fb20010a070 2024-04-24T08:50:17.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:17.082+0000 7fb205600700 1 -- 172.21.15.83:0/1517682806 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7fb20002dfe0 con 0x7fb20010a070 2024-04-24T08:50:17.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:17.083+0000 7fb1f7400700 1 -- 172.21.15.83:0/1517682806 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fb1f8019310 con 0x7fb20010a070 2024-04-24T08:50:17.082 INFO:teuthology.orchestra.run.smithi083.stdout:309237645357 2024-04-24T08:50:17.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:17.085+0000 7fb205600700 1 -- 172.21.15.83:0/1517682806 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fb1e40a3c40 msgr2=0x7fb1e40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:17.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:17.085+0000 7fb205600700 1 --2- 172.21.15.83:0/1517682806 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fb1e40a3c40 0x7fb1e40a6100 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb1f0005950 tx=0x7fb1f00058e0 comp rx=0 tx=0).stop 2024-04-24T08:50:17.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:17.085+0000 7fb205600700 1 -- 172.21.15.83:0/1517682806 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb20010a070 msgr2=0x7fb20019fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:17.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:17.085+0000 7fb205600700 1 --2- 172.21.15.83:0/1517682806 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb20010a070 0x7fb20019fcf0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fb1f8007d70 tx=0x7fb1f800a340 comp rx=0 tx=0).stop 2024-04-24T08:50:17.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:17.086+0000 7fb205600700 1 -- 172.21.15.83:0/1517682806 shutdown_connections 2024-04-24T08:50:17.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:17.086+0000 7fb205600700 1 --2- 172.21.15.83:0/1517682806 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fb1e40a3c40 0x7fb1e40a6100 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:17.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:17.086+0000 7fb205600700 1 --2- 172.21.15.83:0/1517682806 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb20010a070 0x7fb20019fcf0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:17.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:17.086+0000 7fb205600700 1 --2- 172.21.15.83:0/1517682806 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb200104f90 0x7fb20019b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:17.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:17.086+0000 7fb205600700 1 --2- 172.21.15.83:0/1517682806 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb200105940 0x7fb20019b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:17.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:17.086+0000 7fb205600700 1 -- 172.21.15.83:0/1517682806 >> 172.21.15.83:0/1517682806 conn(0x7fb200100820 msgr2=0x7fb200102af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:50:17.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:17.086+0000 7fb205600700 1 -- 172.21.15.83:0/1517682806 shutdown_connections 2024-04-24T08:50:17.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:17.086+0000 7fb205600700 1 -- 172.21.15.83:0/1517682806 wait complete. 2024-04-24T08:50:17.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:17.098+0000 7f4b4c600700 1 -- 172.21.15.83:0/1990164446 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b44102b40 msgr2=0x7f4b44102fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:17.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:17.098+0000 7f4b4c600700 1 --2- 172.21.15.83:0/1990164446 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b44102b40 0x7f4b44102fc0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f4b30009990 tx=0x7f4b30009ca0 comp rx=0 tx=0).stop 2024-04-24T08:50:17.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:17.098+0000 7f4b4c600700 1 -- 172.21.15.83:0/1990164446 shutdown_connections 2024-04-24T08:50:17.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:17.098+0000 7f4b4c600700 1 --2- 172.21.15.83:0/1990164446 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4b44103500 0x7f4b4410f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:17.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:17.098+0000 7f4b4c600700 1 --2- 172.21.15.83:0/1990164446 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b44108b90 0x7f4b44108f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:17.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:17.098+0000 7f4b4c600700 1 --2- 172.21.15.83:0/1990164446 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b44102b40 0x7f4b44102fc0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:17.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:17.098+0000 7f4b4c600700 1 -- 172.21.15.83:0/1990164446 >> 172.21.15.83:0/1990164446 conn(0x7f4b44079c50 msgr2=0x7f4b4407a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:50:17.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:17.098+0000 7f4b4c600700 1 -- 172.21.15.83:0/1990164446 shutdown_connections 2024-04-24T08:50:17.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:17.098+0000 7f4b4c600700 1 -- 172.21.15.83:0/1990164446 wait complete. 2024-04-24T08:50:17.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:17.099+0000 7f4b4c600700 1 Processor -- start 2024-04-24T08:50:17.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:17.099+0000 7f4b4c600700 1 -- start start 2024-04-24T08:50:17.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:17.099+0000 7f4b4c600700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b44102b40 0x7f4b44199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:17.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:17.100+0000 7f4b4c600700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b44103500 0x7f4b44199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:17.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:17.100+0000 7f4b4c600700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4b44108b90 0x7f4b4419da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:17.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:17.100+0000 7f4b4c600700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4b44112cb0 con 0x7f4b44103500 2024-04-24T08:50:17.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:17.100+0000 7f4b4c600700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4b44112b30 con 0x7f4b44102b40 2024-04-24T08:50:17.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:17.100+0000 7f4b4c600700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4b44112e30 con 0x7f4b44108b90 2024-04-24T08:50:17.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:17.100+0000 7f4b49200700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b44103500 0x7f4b44199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:17.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:17.100+0000 7f4b49200700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b44103500 0x7f4b44199740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58350/0 (socket says 172.21.15.83:58350) 2024-04-24T08:50:17.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:17.100+0000 7f4b49200700 1 -- 172.21.15.83:0/2425091583 learned_addr learned my addr 172.21.15.83:0/2425091583 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:50:17.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:17.100+0000 7f4b49c00700 1 --2- 172.21.15.83:0/2425091583 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b44102b40 0x7f4b44199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:17.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:17.100+0000 7f4b4a600700 1 --2- 172.21.15.83:0/2425091583 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4b44108b90 0x7f4b4419da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:17.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:17.100+0000 7f4b49200700 1 -- 172.21.15.83:0/2425091583 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4b44108b90 msgr2=0x7f4b4419da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:17.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:17.100+0000 7f4b49200700 1 --2- 172.21.15.83:0/2425091583 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4b44108b90 0x7f4b4419da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:17.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:17.100+0000 7f4b49200700 1 -- 172.21.15.83:0/2425091583 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b44102b40 msgr2=0x7f4b44199200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:17.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:17.100+0000 7f4b49200700 1 --2- 172.21.15.83:0/2425091583 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b44102b40 0x7f4b44199200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:17.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:17.100+0000 7f4b49200700 1 -- 172.21.15.83:0/2425091583 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b4419e1c0 con 0x7f4b44103500 2024-04-24T08:50:17.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:17.100+0000 7f4b49c00700 1 --2- 172.21.15.83:0/2425091583 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b44102b40 0x7f4b44199200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:50:17.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:17.101+0000 7f4b49200700 1 --2- 172.21.15.83:0/2425091583 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b44103500 0x7f4b44199740 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f4b30003ab0 tx=0x7f4b30003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:50:17.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:17.101+0000 7f4b42000700 1 -- 172.21.15.83:0/2425091583 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b30027070 con 0x7f4b44103500 2024-04-24T08:50:17.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:17.101+0000 7f4b4c600700 1 -- 172.21.15.83:0/2425091583 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b4419e450 con 0x7f4b44103500 2024-04-24T08:50:17.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:17.101+0000 7f4b42000700 1 -- 172.21.15.83:0/2425091583 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4b3000bdc0 con 0x7f4b44103500 2024-04-24T08:50:17.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:17.101+0000 7f4b42000700 1 -- 172.21.15.83:0/2425091583 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b30022db0 con 0x7f4b44103500 2024-04-24T08:50:17.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:17.101+0000 7f4b4c600700 1 -- 172.21.15.83:0/2425091583 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b441a5d30 con 0x7f4b44103500 2024-04-24T08:50:17.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:17.102+0000 7f4b4c600700 1 -- 172.21.15.83:0/2425091583 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b440697e0 con 0x7f4b44103500 2024-04-24T08:50:17.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:17.104+0000 7f4b42000700 1 -- 172.21.15.83:0/2425091583 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f4b3002a070 con 0x7f4b44103500 2024-04-24T08:50:17.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:17.106+0000 7f4b42000700 1 --2- 172.21.15.83:0/2425091583 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f4b2c0a3c40 0x7f4b2c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:17.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:17.106+0000 7f4b42000700 1 -- 172.21.15.83:0/2425091583 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f4b300d1d60 con 0x7f4b44103500 2024-04-24T08:50:17.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:17.106+0000 7f4b49c00700 1 --2- 172.21.15.83:0/2425091583 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f4b2c0a3c40 0x7f4b2c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:17.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:17.107+0000 7f4b49c00700 1 --2- 172.21.15.83:0/2425091583 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f4b2c0a3c40 0x7f4b2c0a6100 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4b38009b50 tx=0x7f4b38009550 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:50:17.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:17.108+0000 7f4b42000700 1 -- 172.21.15.83:0/2425091583 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4b30031020 con 0x7f4b44103500 2024-04-24T08:50:17.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:17.356+0000 7f4b4c600700 1 -- 172.21.15.83:0/2425091583 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f4b4402dfe0 con 0x7f4b44103500 2024-04-24T08:50:17.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:17.357+0000 7f4b42000700 1 -- 172.21.15.83:0/2425091583 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f4b30013070 con 0x7f4b44103500 2024-04-24T08:50:17.356 INFO:teuthology.orchestra.run.smithi083.stdout:55834574958 2024-04-24T08:50:17.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:17.360+0000 7f4b4c600700 1 -- 172.21.15.83:0/2425091583 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f4b2c0a3c40 msgr2=0x7f4b2c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:17.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:17.360+0000 7f4b4c600700 1 --2- 172.21.15.83:0/2425091583 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f4b2c0a3c40 0x7f4b2c0a6100 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4b38009b50 tx=0x7f4b38009550 comp rx=0 tx=0).stop 2024-04-24T08:50:17.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:17.360+0000 7f4b4c600700 1 -- 172.21.15.83:0/2425091583 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b44103500 msgr2=0x7f4b44199740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:17.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:17.360+0000 7f4b4c600700 1 --2- 172.21.15.83:0/2425091583 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b44103500 0x7f4b44199740 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f4b30003ab0 tx=0x7f4b30003b90 comp rx=0 tx=0).stop 2024-04-24T08:50:17.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:17.360+0000 7f4b4c600700 1 -- 172.21.15.83:0/2425091583 shutdown_connections 2024-04-24T08:50:17.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:17.360+0000 7f4b4c600700 1 --2- 172.21.15.83:0/2425091583 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f4b2c0a3c40 0x7f4b2c0a6100 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:17.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:17.360+0000 7f4b4c600700 1 --2- 172.21.15.83:0/2425091583 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4b44108b90 0x7f4b4419da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:17.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:17.360+0000 7f4b4c600700 1 --2- 172.21.15.83:0/2425091583 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b44102b40 0x7f4b44199200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:17.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:17.360+0000 7f4b4c600700 1 --2- 172.21.15.83:0/2425091583 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b44103500 0x7f4b44199740 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:17.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:17.360+0000 7f4b4c600700 1 -- 172.21.15.83:0/2425091583 >> 172.21.15.83:0/2425091583 conn(0x7f4b44079c50 msgr2=0x7f4b441018a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:50:17.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:17.360+0000 7f4b4c600700 1 -- 172.21.15.83:0/2425091583 shutdown_connections 2024-04-24T08:50:17.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:17.360+0000 7f4b4c600700 1 -- 172.21.15.83:0/2425091583 wait complete. 2024-04-24T08:50:17.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:15.421+0000 7f8441600700 1 -- 172.21.15.83:0/1317502271 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f843c075640 msgr2=0x7f843c113780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:17.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:15.421+0000 7f8441600700 1 --2- 172.21.15.83:0/1317502271 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f843c075640 0x7f843c113780 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f8428009990 tx=0x7f8428009ca0 comp rx=0 tx=0).stop 2024-04-24T08:50:17.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:15.421+0000 7f8441600700 1 -- 172.21.15.83:0/1317502271 shutdown_connections 2024-04-24T08:50:17.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:15.421+0000 7f8441600700 1 --2- 172.21.15.83:0/1317502271 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f843c113cc0 0x7f843c1160b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:17.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:15.421+0000 7f8441600700 1 --2- 172.21.15.83:0/1317502271 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f843c074d20 0x7f843c075100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:17.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:15.421+0000 7f8441600700 1 --2- 172.21.15.83:0/1317502271 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f843c075640 0x7f843c113780 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:17.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:15.421+0000 7f8441600700 1 -- 172.21.15.83:0/1317502271 >> 172.21.15.83:0/1317502271 conn(0x7f843c079c50 msgr2=0x7f843c07a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:50:17.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:15.422+0000 7f8441600700 1 -- 172.21.15.83:0/1317502271 shutdown_connections 2024-04-24T08:50:17.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:15.422+0000 7f8441600700 1 -- 172.21.15.83:0/1317502271 wait complete. 2024-04-24T08:50:17.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:15.423+0000 7f8441600700 1 Processor -- start 2024-04-24T08:50:17.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:15.423+0000 7f8441600700 1 -- start start 2024-04-24T08:50:17.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:15.423+0000 7f8441600700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f843c074d20 0x7f843c1a53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:17.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:15.423+0000 7f8441600700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f843c075640 0x7f843c1a5930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:17.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:15.423+0000 7f8441600700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f843c113cc0 0x7f843c19f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:17.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:15.423+0000 7f8441600700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f843c1192b0 con 0x7f843c074d20 2024-04-24T08:50:17.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:15.423+0000 7f8441600700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f843c119130 con 0x7f843c075640 2024-04-24T08:50:17.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:15.423+0000 7f8441600700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f843c119430 con 0x7f843c113cc0 2024-04-24T08:50:17.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:15.423+0000 7f843aa00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f843c074d20 0x7f843c1a53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:17.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:15.424+0000 7f843aa00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f843c074d20 0x7f843c1a53f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58286/0 (socket says 172.21.15.83:58286) 2024-04-24T08:50:17.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:15.424+0000 7f843aa00700 1 -- 172.21.15.83:0/599051359 learned_addr learned my addr 172.21.15.83:0/599051359 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:50:17.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:15.424+0000 7f843b400700 1 --2- 172.21.15.83:0/599051359 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f843c113cc0 0x7f843c19f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:17.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:15.424+0000 7f843a000700 1 --2- 172.21.15.83:0/599051359 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f843c075640 0x7f843c1a5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:17.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:15.424+0000 7f843aa00700 1 -- 172.21.15.83:0/599051359 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f843c113cc0 msgr2=0x7f843c19f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:17.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:15.424+0000 7f843aa00700 1 --2- 172.21.15.83:0/599051359 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f843c113cc0 0x7f843c19f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:17.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:15.424+0000 7f843aa00700 1 -- 172.21.15.83:0/599051359 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f843c075640 msgr2=0x7f843c1a5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:17.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:15.424+0000 7f843aa00700 1 --2- 172.21.15.83:0/599051359 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f843c075640 0x7f843c1a5930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:17.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:15.424+0000 7f843aa00700 1 -- 172.21.15.83:0/599051359 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f843c19fd10 con 0x7f843c074d20 2024-04-24T08:50:17.905 INFO:teuthology.orchestra.run.smithi083.stdout:107374182496 2024-04-24T08:50:17.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:15.424+0000 7f843b400700 1 --2- 172.21.15.83:0/599051359 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f843c113cc0 0x7f843c19f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:50:17.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:15.424+0000 7f843aa00700 1 --2- 172.21.15.83:0/599051359 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f843c074d20 0x7f843c1a53f0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f842c00c920 tx=0x7f842c00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:50:17.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:15.425+0000 7f8433400700 1 -- 172.21.15.83:0/599051359 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f842c007b00 con 0x7f843c074d20 2024-04-24T08:50:17.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:15.425+0000 7f8433400700 1 -- 172.21.15.83:0/599051359 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f842c004ce0 con 0x7f843c074d20 2024-04-24T08:50:17.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:15.425+0000 7f8441600700 1 -- 172.21.15.83:0/599051359 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f843c1a0000 con 0x7f843c074d20 2024-04-24T08:50:17.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:15.425+0000 7f8433400700 1 -- 172.21.15.83:0/599051359 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f842c005600 con 0x7f843c074d20 2024-04-24T08:50:17.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:15.425+0000 7f8441600700 1 -- 172.21.15.83:0/599051359 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f843c1ac2c0 con 0x7f843c074d20 2024-04-24T08:50:17.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:15.426+0000 7f8441600700 1 -- 172.21.15.83:0/599051359 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f843c0697e0 con 0x7f843c074d20 2024-04-24T08:50:17.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:15.428+0000 7f8433400700 1 -- 172.21.15.83:0/599051359 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f842c007ca0 con 0x7f843c074d20 2024-04-24T08:50:17.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:15.429+0000 7f8433400700 1 --2- 172.21.15.83:0/599051359 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f84200a3cc0 0x7f84200a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:17.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:15.430+0000 7f8433400700 1 -- 172.21.15.83:0/599051359 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f842c0d0370 con 0x7f843c074d20 2024-04-24T08:50:17.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:15.430+0000 7f843a000700 1 --2- 172.21.15.83:0/599051359 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f84200a3cc0 0x7f84200a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:17.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:15.431+0000 7f843a000700 1 --2- 172.21.15.83:0/599051359 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f84200a3cc0 0x7f84200a6180 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f8428006010 tx=0x7f842801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:50:17.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:15.434+0000 7f8433400700 1 -- 172.21.15.83:0/599051359 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f842c014030 con 0x7f843c074d20 2024-04-24T08:50:17.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:15.698+0000 7f8441600700 1 -- 172.21.15.83:0/599051359 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f843c1ac550 con 0x7f843c074d20 2024-04-24T08:50:17.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:15.698+0000 7f8433400700 1 -- 172.21.15.83:0/599051359 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f842c09c990 con 0x7f843c074d20 2024-04-24T08:50:17.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:15.701+0000 7f8441600700 1 -- 172.21.15.83:0/599051359 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f84200a3cc0 msgr2=0x7f84200a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:17.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:15.701+0000 7f8441600700 1 --2- 172.21.15.83:0/599051359 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f84200a3cc0 0x7f84200a6180 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f8428006010 tx=0x7f842801a040 comp rx=0 tx=0).stop 2024-04-24T08:50:17.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:15.701+0000 7f8441600700 1 -- 172.21.15.83:0/599051359 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f843c074d20 msgr2=0x7f843c1a53f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:17.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:15.701+0000 7f8441600700 1 --2- 172.21.15.83:0/599051359 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f843c074d20 0x7f843c1a53f0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f842c00c920 tx=0x7f842c00cce0 comp rx=0 tx=0).stop 2024-04-24T08:50:17.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:15.701+0000 7f8441600700 1 -- 172.21.15.83:0/599051359 shutdown_connections 2024-04-24T08:50:17.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:15.701+0000 7f8441600700 1 --2- 172.21.15.83:0/599051359 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f84200a3cc0 0x7f84200a6180 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:17.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:15.701+0000 7f8441600700 1 --2- 172.21.15.83:0/599051359 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f843c074d20 0x7f843c1a53f0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:17.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:15.701+0000 7f8441600700 1 --2- 172.21.15.83:0/599051359 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f843c113cc0 0x7f843c19f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:17.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:15.701+0000 7f8441600700 1 --2- 172.21.15.83:0/599051359 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f843c075640 0x7f843c1a5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:17.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:15.701+0000 7f8441600700 1 -- 172.21.15.83:0/599051359 >> 172.21.15.83:0/599051359 conn(0x7f843c079c50 msgr2=0x7f843c112a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:50:17.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:15.701+0000 7f8441600700 1 -- 172.21.15.83:0/599051359 shutdown_connections 2024-04-24T08:50:17.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:15.701+0000 7f8441600700 1 -- 172.21.15.83:0/599051359 wait complete. 2024-04-24T08:50:18.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:50:17 smithi138 ceph-mon[31441]: pgmap v429: 129 pgs: 129 active+clean; 644 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 10 op/s 2024-04-24T08:50:18.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:50:17 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/1517682806' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-24T08:50:18.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:50:17 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/2425091583' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-24T08:50:18.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:50:17 smithi189 ceph-mon[31791]: pgmap v429: 129 pgs: 129 active+clean; 644 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 10 op/s 2024-04-24T08:50:18.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:50:17 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/1517682806' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-24T08:50:18.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:50:17 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/2425091583' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-24T08:50:18.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:50:17 smithi083 ceph-mon[27907]: pgmap v429: 129 pgs: 129 active+clean; 644 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 10 op/s 2024-04-24T08:50:18.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:50:17 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/1517682806' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-24T08:50:18.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:50:17 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/2425091583' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-24T08:50:19.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:19.585+0000 7f241f400700 1 -- 172.21.15.83:0/874100967 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2418109d30 msgr2=0x7f241810dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:19.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:19.585+0000 7f241f400700 1 --2- 172.21.15.83:0/874100967 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2418109d30 0x7f241810dd80 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f24040099c0 tx=0x7f2404009cd0 comp rx=0 tx=0).stop 2024-04-24T08:50:19.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:19.585+0000 7f241f400700 1 -- 172.21.15.83:0/874100967 shutdown_connections 2024-04-24T08:50:19.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:19.585+0000 7f241f400700 1 --2- 172.21.15.83:0/874100967 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f241810e460 0x7f2418116030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:19.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:19.585+0000 7f241f400700 1 --2- 172.21.15.83:0/874100967 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2418109380 0x7f2418109760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:19.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:19.585+0000 7f241f400700 1 --2- 172.21.15.83:0/874100967 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2418109d30 0x7f241810dd80 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:19.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:19.585+0000 7f241f400700 1 -- 172.21.15.83:0/874100967 >> 172.21.15.83:0/874100967 conn(0x7f24180762c0 msgr2=0x7f2418107910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:50:19.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:19.585+0000 7f241f400700 1 -- 172.21.15.83:0/874100967 shutdown_connections 2024-04-24T08:50:19.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:19.585+0000 7f241f400700 1 -- 172.21.15.83:0/874100967 wait complete. 2024-04-24T08:50:19.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:19.586+0000 7f241f400700 1 Processor -- start 2024-04-24T08:50:19.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:19.586+0000 7f241f400700 1 -- start start 2024-04-24T08:50:19.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:19.587+0000 7f241f400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2418109380 0x7f241819f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:19.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:19.587+0000 7f241f400700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2418109d30 0x7f241819fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:19.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:19.587+0000 7f241f400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f241810e460 0x7f24181a4020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:19.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:19.587+0000 7f241f400700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f24181192f0 con 0x7f2418109380 2024-04-24T08:50:19.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:19.587+0000 7f241f400700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2418119170 con 0x7f2418109d30 2024-04-24T08:50:19.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:19.587+0000 7f241f400700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2418119470 con 0x7f241810e460 2024-04-24T08:50:19.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:19.587+0000 7f241ca00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2418109380 0x7f241819f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:19.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:19.587+0000 7f241ca00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2418109380 0x7f241819f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58288/0 (socket says 172.21.15.83:58288) 2024-04-24T08:50:19.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:19.587+0000 7f241ca00700 1 -- 172.21.15.83:0/3246539695 learned_addr learned my addr 172.21.15.83:0/3246539695 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:50:19.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:19.587+0000 7f2417e00700 1 --2- 172.21.15.83:0/3246539695 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2418109d30 0x7f241819fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:19.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:19.587+0000 7f241ca00700 1 -- 172.21.15.83:0/3246539695 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f241810e460 msgr2=0x7f24181a4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:19.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:19.587+0000 7f241ca00700 1 --2- 172.21.15.83:0/3246539695 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f241810e460 0x7f24181a4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:19.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:19.587+0000 7f241ca00700 1 -- 172.21.15.83:0/3246539695 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2418109d30 msgr2=0x7f241819fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:19.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:19.587+0000 7f241ca00700 1 --2- 172.21.15.83:0/3246539695 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2418109d30 0x7f241819fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:19.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:19.587+0000 7f241ca00700 1 -- 172.21.15.83:0/3246539695 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24181a4760 con 0x7f2418109380 2024-04-24T08:50:19.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:19.588+0000 7f241ca00700 1 --2- 172.21.15.83:0/3246539695 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2418109380 0x7f241819f7a0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f240c00c970 tx=0x7f240c00cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:50:19.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:19.588+0000 7f2415600700 1 -- 172.21.15.83:0/3246539695 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f240c007aa0 con 0x7f2418109380 2024-04-24T08:50:19.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:19.588+0000 7f2415600700 1 -- 172.21.15.83:0/3246539695 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f240c004ce0 con 0x7f2418109380 2024-04-24T08:50:19.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:19.588+0000 7f241f400700 1 -- 172.21.15.83:0/3246539695 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24181a4a50 con 0x7f2418109380 2024-04-24T08:50:19.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:19.588+0000 7f2415600700 1 -- 172.21.15.83:0/3246539695 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f240c005620 con 0x7f2418109380 2024-04-24T08:50:19.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:19.588+0000 7f241f400700 1 -- 172.21.15.83:0/3246539695 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24181ac2d0 con 0x7f2418109380 2024-04-24T08:50:19.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:19.589+0000 7f241f400700 1 -- 172.21.15.83:0/3246539695 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24180697e0 con 0x7f2418109380 2024-04-24T08:50:19.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:19.591+0000 7f2415600700 1 -- 172.21.15.83:0/3246539695 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f240c004750 con 0x7f2418109380 2024-04-24T08:50:19.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:19.593+0000 7f2415600700 1 --2- 172.21.15.83:0/3246539695 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f24000a3c40 0x7f24000a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:19.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:19.593+0000 7f2415600700 1 -- 172.21.15.83:0/3246539695 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f240c0d0600 con 0x7f2418109380 2024-04-24T08:50:19.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:19.593+0000 7f2417e00700 1 --2- 172.21.15.83:0/3246539695 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f24000a3c40 0x7f24000a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:19.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:19.594+0000 7f2417e00700 1 --2- 172.21.15.83:0/3246539695 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f24000a3c40 0x7f24000a6100 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2404000c00 tx=0x7f2404005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:50:19.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:19.597+0000 7f2415600700 1 -- 172.21.15.83:0/3246539695 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f240c09ccb0 con 0x7f2418109380 2024-04-24T08:50:19.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:19.828+0000 7f241f400700 1 -- 172.21.15.83:0/3246539695 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f241802dfe0 con 0x7f2418109380 2024-04-24T08:50:19.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:19.828+0000 7f2415600700 1 -- 172.21.15.83:0/3246539695 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f240c09c5b0 con 0x7f2418109380 2024-04-24T08:50:19.830 INFO:teuthology.orchestra.run.smithi083.stdout:141733920857 2024-04-24T08:50:19.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:19.831+0000 7f241f400700 1 -- 172.21.15.83:0/3246539695 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f24000a3c40 msgr2=0x7f24000a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:19.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:19.831+0000 7f241f400700 1 --2- 172.21.15.83:0/3246539695 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f24000a3c40 0x7f24000a6100 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2404000c00 tx=0x7f2404005fd0 comp rx=0 tx=0).stop 2024-04-24T08:50:19.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:19.831+0000 7f241f400700 1 -- 172.21.15.83:0/3246539695 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2418109380 msgr2=0x7f241819f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:19.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:19.831+0000 7f241f400700 1 --2- 172.21.15.83:0/3246539695 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2418109380 0x7f241819f7a0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f240c00c970 tx=0x7f240c00cd30 comp rx=0 tx=0).stop 2024-04-24T08:50:19.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:19.832+0000 7f241f400700 1 -- 172.21.15.83:0/3246539695 shutdown_connections 2024-04-24T08:50:19.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:19.832+0000 7f241f400700 1 --2- 172.21.15.83:0/3246539695 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f24000a3c40 0x7f24000a6100 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:19.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:19.832+0000 7f241f400700 1 --2- 172.21.15.83:0/3246539695 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2418109380 0x7f241819f7a0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:19.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:19.832+0000 7f241f400700 1 --2- 172.21.15.83:0/3246539695 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f241810e460 0x7f24181a4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:19.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:19.832+0000 7f241f400700 1 --2- 172.21.15.83:0/3246539695 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2418109d30 0x7f241819fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:19.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:19.832+0000 7f241f400700 1 -- 172.21.15.83:0/3246539695 >> 172.21.15.83:0/3246539695 conn(0x7f24180762c0 msgr2=0x7f2418076880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:50:19.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:19.832+0000 7f241f400700 1 -- 172.21.15.83:0/3246539695 shutdown_connections 2024-04-24T08:50:19.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:19.832+0000 7f241f400700 1 -- 172.21.15.83:0/3246539695 wait complete. 2024-04-24T08:50:20.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:50:19 smithi083 ceph-mon[27907]: pgmap v430: 129 pgs: 129 active+clean; 644 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-04-24T08:50:20.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:50:19 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/3246539695' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-24T08:50:20.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:50:19 smithi189 ceph-mon[31791]: pgmap v430: 129 pgs: 129 active+clean; 644 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-04-24T08:50:20.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:50:19 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/3246539695' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-24T08:50:20.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:50:19 smithi138 ceph-mon[31441]: pgmap v430: 129 pgs: 129 active+clean; 644 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-04-24T08:50:20.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:50:19 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/3246539695' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-24T08:50:20.806 INFO:tasks.cephadm.ceph_manager.ceph:need seq 309237645350 got 309237645357 for osd.10 2024-04-24T08:50:20.806 DEBUG:teuthology.parallel:result is None 2024-04-24T08:50:21.224 INFO:tasks.cephadm.ceph_manager.ceph:need seq 287762808878 got 287762808884 for osd.9 2024-04-24T08:50:21.224 DEBUG:teuthology.parallel:result is None 2024-04-24T08:50:22.277 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:50:21 smithi083 ceph-mon[27907]: pgmap v431: 129 pgs: 129 active+clean; 644 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-24T08:50:22.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:22.281+0000 7f18e0c00700 1 -- 172.21.15.83:0/1981929186 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18dc104f90 msgr2=0x7f18dc105370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:22.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:22.281+0000 7f18e0c00700 1 --2- 172.21.15.83:0/1981929186 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18dc104f90 0x7f18dc105370 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f18cc0099c0 tx=0x7f18cc009cd0 comp rx=0 tx=0).stop 2024-04-24T08:50:22.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:22.282+0000 7f18e0c00700 1 -- 172.21.15.83:0/1981929186 shutdown_connections 2024-04-24T08:50:22.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:22.282+0000 7f18e0c00700 1 --2- 172.21.15.83:0/1981929186 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18dc104f90 0x7f18dc105370 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:22.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:22.282+0000 7f18e0c00700 1 --2- 172.21.15.83:0/1981929186 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18dc10a070 0x7f18dc111c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:22.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:22.282+0000 7f18e0c00700 1 --2- 172.21.15.83:0/1981929186 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f18dc105940 0x7f18dc109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:22.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:22.282+0000 7f18e0c00700 1 -- 172.21.15.83:0/1981929186 >> 172.21.15.83:0/1981929186 conn(0x7f18dc100820 msgr2=0x7f18dc102c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:50:22.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:22.282+0000 7f18e0c00700 1 -- 172.21.15.83:0/1981929186 shutdown_connections 2024-04-24T08:50:22.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:22.282+0000 7f18e0c00700 1 -- 172.21.15.83:0/1981929186 wait complete. 2024-04-24T08:50:22.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:22.283+0000 7f18e0c00700 1 Processor -- start 2024-04-24T08:50:22.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:22.283+0000 7f18e0c00700 1 -- start start 2024-04-24T08:50:22.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:22.284+0000 7f18e0c00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18dc104f90 0x7f18dc19b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:22.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:22.284+0000 7f18e0c00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f18dc105940 0x7f18dc19b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:22.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:22.284+0000 7f18e0c00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18dc10a070 0x7f18dc19fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:22.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:22.284+0000 7f18e0c00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f18dc114f80 con 0x7f18dc104f90 2024-04-24T08:50:22.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:22.284+0000 7f18e0c00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f18dc114e00 con 0x7f18dc10a070 2024-04-24T08:50:22.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:22.284+0000 7f18e0c00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f18dc115100 con 0x7f18dc105940 2024-04-24T08:50:22.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:22.284+0000 7f18d9e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18dc104f90 0x7f18dc19b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:22.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:22.284+0000 7f18d9e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18dc104f90 0x7f18dc19b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58310/0 (socket says 172.21.15.83:58310) 2024-04-24T08:50:22.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:22.284+0000 7f18d9e00700 1 -- 172.21.15.83:0/2094152751 learned_addr learned my addr 172.21.15.83:0/2094152751 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:50:22.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:22.284+0000 7f18d9400700 1 --2- 172.21.15.83:0/2094152751 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f18dc105940 0x7f18dc19b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:22.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:22.285+0000 7f18da800700 1 --2- 172.21.15.83:0/2094152751 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18dc10a070 0x7f18dc19fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:22.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:22.285+0000 7f18d9e00700 1 -- 172.21.15.83:0/2094152751 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f18dc105940 msgr2=0x7f18dc19b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:22.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:22.285+0000 7f18d9e00700 1 --2- 172.21.15.83:0/2094152751 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f18dc105940 0x7f18dc19b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:22.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:22.285+0000 7f18d9e00700 1 -- 172.21.15.83:0/2094152751 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18dc10a070 msgr2=0x7f18dc19fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:22.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:22.285+0000 7f18d9e00700 1 --2- 172.21.15.83:0/2094152751 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18dc10a070 0x7f18dc19fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:22.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:22.285+0000 7f18d9e00700 1 -- 172.21.15.83:0/2094152751 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18dc1a0430 con 0x7f18dc104f90 2024-04-24T08:50:22.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:22.285+0000 7f18d9e00700 1 --2- 172.21.15.83:0/2094152751 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18dc104f90 0x7f18dc19b470 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f18cc006010 tx=0x7f18cc0057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:50:22.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:22.286+0000 7f18d2a00700 1 -- 172.21.15.83:0/2094152751 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18cc027070 con 0x7f18dc104f90 2024-04-24T08:50:22.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:22.286+0000 7f18d2a00700 1 -- 172.21.15.83:0/2094152751 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f18cc003f50 con 0x7f18dc104f90 2024-04-24T08:50:22.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:22.286+0000 7f18d2a00700 1 -- 172.21.15.83:0/2094152751 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18cc022d10 con 0x7f18dc104f90 2024-04-24T08:50:22.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:22.286+0000 7f18e0c00700 1 -- 172.21.15.83:0/2094152751 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18dc1a06c0 con 0x7f18dc104f90 2024-04-24T08:50:22.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:22.286+0000 7f18e0c00700 1 -- 172.21.15.83:0/2094152751 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18dc1a7fa0 con 0x7f18dc104f90 2024-04-24T08:50:22.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:22.287+0000 7f18e0c00700 1 -- 172.21.15.83:0/2094152751 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18dc106470 con 0x7f18dc104f90 2024-04-24T08:50:22.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:22.289+0000 7f18d2a00700 1 -- 172.21.15.83:0/2094152751 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f18cc02a070 con 0x7f18dc104f90 2024-04-24T08:50:22.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:22.291+0000 7f18d2a00700 1 --2- 172.21.15.83:0/2094152751 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f18c00a3cc0 0x7f18c00a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:22.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:22.291+0000 7f18d2a00700 1 -- 172.21.15.83:0/2094152751 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f18cc0d1d40 con 0x7f18dc104f90 2024-04-24T08:50:22.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:22.291+0000 7f18d9400700 1 --2- 172.21.15.83:0/2094152751 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f18c00a3cc0 0x7f18c00a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:22.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:22.292+0000 7f18d9400700 1 --2- 172.21.15.83:0/2094152751 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f18c00a3cc0 0x7f18c00a6180 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f18dc10b610 tx=0x7f18c800a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:50:22.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:22.293+0000 7f18d2a00700 1 -- 172.21.15.83:0/2094152751 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f18cc031020 con 0x7f18dc104f90 2024-04-24T08:50:22.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:50:21 smithi138 ceph-mon[31441]: pgmap v431: 129 pgs: 129 active+clean; 644 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-24T08:50:22.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:50:21 smithi189 ceph-mon[31791]: pgmap v431: 129 pgs: 129 active+clean; 644 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-24T08:50:22.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:22.539+0000 7f18e0c00700 1 -- 172.21.15.83:0/2094152751 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f18dc02dfe0 con 0x7f18dc104f90 2024-04-24T08:50:22.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:22.540+0000 7f18d2a00700 1 -- 172.21.15.83:0/2094152751 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f18cc0bce20 con 0x7f18dc104f90 2024-04-24T08:50:22.540 INFO:teuthology.orchestra.run.smithi083.stdout:34359738486 2024-04-24T08:50:22.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:22.543+0000 7f18e0c00700 1 -- 172.21.15.83:0/2094152751 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f18c00a3cc0 msgr2=0x7f18c00a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:22.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:22.543+0000 7f18e0c00700 1 --2- 172.21.15.83:0/2094152751 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f18c00a3cc0 0x7f18c00a6180 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f18dc10b610 tx=0x7f18c800a300 comp rx=0 tx=0).stop 2024-04-24T08:50:22.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:22.543+0000 7f18e0c00700 1 -- 172.21.15.83:0/2094152751 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18dc104f90 msgr2=0x7f18dc19b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:22.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:22.543+0000 7f18e0c00700 1 --2- 172.21.15.83:0/2094152751 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18dc104f90 0x7f18dc19b470 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f18cc006010 tx=0x7f18cc0057c0 comp rx=0 tx=0).stop 2024-04-24T08:50:22.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:22.543+0000 7f18e0c00700 1 -- 172.21.15.83:0/2094152751 shutdown_connections 2024-04-24T08:50:22.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:22.543+0000 7f18e0c00700 1 --2- 172.21.15.83:0/2094152751 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f18c00a3cc0 0x7f18c00a6180 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:22.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:22.543+0000 7f18e0c00700 1 --2- 172.21.15.83:0/2094152751 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18dc104f90 0x7f18dc19b470 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:22.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:22.543+0000 7f18e0c00700 1 --2- 172.21.15.83:0/2094152751 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18dc10a070 0x7f18dc19fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:22.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:22.543+0000 7f18e0c00700 1 --2- 172.21.15.83:0/2094152751 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f18dc105940 0x7f18dc19b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:22.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:22.543+0000 7f18e0c00700 1 -- 172.21.15.83:0/2094152751 >> 172.21.15.83:0/2094152751 conn(0x7f18dc100820 msgr2=0x7f18dc102b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:50:22.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:22.543+0000 7f18e0c00700 1 -- 172.21.15.83:0/2094152751 shutdown_connections 2024-04-24T08:50:22.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:22.543+0000 7f18e0c00700 1 -- 172.21.15.83:0/2094152751 wait complete. 2024-04-24T08:50:22.622 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T08:50:23.099 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:50:22 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/2094152751' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-24T08:50:23.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:50:22 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/2094152751' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-24T08:50:23.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:50:22 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/2094152751' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-24T08:50:23.667 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574950 got 55834574958 for osd.1 2024-04-24T08:50:23.667 DEBUG:teuthology.parallel:result is None 2024-04-24T08:50:24.068 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738477 got 34359738486 for osd.0 2024-04-24T08:50:24.069 DEBUG:teuthology.parallel:result is None 2024-04-24T08:50:24.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:50:23 smithi083 ceph-mon[27907]: pgmap v432: 129 pgs: 129 active+clean; 644 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 10 op/s 2024-04-24T08:50:24.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:50:23 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/1457796891' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-24T08:50:24.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:24.328+0000 7f89caa00700 1 -- 172.21.15.83:0/2118663705 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89c4102a20 msgr2=0x7f89c4102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:24.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:24.328+0000 7f89caa00700 1 --2- 172.21.15.83:0/2118663705 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89c4102a20 0x7f89c4102ea0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f89b0009990 tx=0x7f89b0009ca0 comp rx=0 tx=0).stop 2024-04-24T08:50:24.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:24.329+0000 7f89caa00700 1 -- 172.21.15.83:0/2118663705 shutdown_connections 2024-04-24T08:50:24.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:24.329+0000 7f89caa00700 1 --2- 172.21.15.83:0/2118663705 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f89c41033e0 0x7f89c410f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:24.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:24.329+0000 7f89caa00700 1 --2- 172.21.15.83:0/2118663705 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f89c4108a20 0x7f89c4108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:24.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:24.329+0000 7f89caa00700 1 --2- 172.21.15.83:0/2118663705 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89c4102a20 0x7f89c4102ea0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:24.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:24.329+0000 7f89caa00700 1 -- 172.21.15.83:0/2118663705 >> 172.21.15.83:0/2118663705 conn(0x7f89c40fe570 msgr2=0x7f89c4100990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:50:24.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:24.329+0000 7f89caa00700 1 -- 172.21.15.83:0/2118663705 shutdown_connections 2024-04-24T08:50:24.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:24.329+0000 7f89caa00700 1 -- 172.21.15.83:0/2118663705 wait complete. 2024-04-24T08:50:24.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:24.330+0000 7f89caa00700 1 Processor -- start 2024-04-24T08:50:24.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:24.330+0000 7f89caa00700 1 -- start start 2024-04-24T08:50:24.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:24.330+0000 7f89caa00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f89c4102a20 0x7f89c4074df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:24.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:24.330+0000 7f89caa00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89c41033e0 0x7f89c4075330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:24.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:24.330+0000 7f89caa00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f89c4108a20 0x7f89c4075870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:24.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:24.331+0000 7f89caa00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f89c4112c60 con 0x7f89c41033e0 2024-04-24T08:50:24.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:24.331+0000 7f89caa00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f89c4112ae0 con 0x7f89c4108a20 2024-04-24T08:50:24.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:24.331+0000 7f89caa00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f89c4112de0 con 0x7f89c4102a20 2024-04-24T08:50:24.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:24.331+0000 7f89c8c00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89c41033e0 0x7f89c4075330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:24.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:24.331+0000 7f89c8c00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89c41033e0 0x7f89c4075330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58360/0 (socket says 172.21.15.83:58360) 2024-04-24T08:50:24.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:24.331+0000 7f89c8c00700 1 -- 172.21.15.83:0/2421909050 learned_addr learned my addr 172.21.15.83:0/2421909050 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:50:24.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:24.331+0000 7f89c9600700 1 --2- 172.21.15.83:0/2421909050 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f89c4102a20 0x7f89c4074df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:24.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:24.331+0000 7f89ca000700 1 --2- 172.21.15.83:0/2421909050 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f89c4108a20 0x7f89c4075870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:24.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:24.331+0000 7f89c8c00700 1 -- 172.21.15.83:0/2421909050 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f89c4102a20 msgr2=0x7f89c4074df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:24.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:24.331+0000 7f89c8c00700 1 --2- 172.21.15.83:0/2421909050 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f89c4102a20 0x7f89c4074df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:24.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:24.331+0000 7f89c8c00700 1 -- 172.21.15.83:0/2421909050 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f89c4108a20 msgr2=0x7f89c4075870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:24.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:24.331+0000 7f89c8c00700 1 --2- 172.21.15.83:0/2421909050 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f89c4108a20 0x7f89c4075870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:24.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:24.331+0000 7f89c8c00700 1 -- 172.21.15.83:0/2421909050 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89c41a75b0 con 0x7f89c41033e0 2024-04-24T08:50:24.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:24.331+0000 7f89ca000700 1 --2- 172.21.15.83:0/2421909050 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f89c4108a20 0x7f89c4075870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:50:24.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:24.332+0000 7f89c8c00700 1 --2- 172.21.15.83:0/2421909050 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89c41033e0 0x7f89c4075330 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f89b0003ab0 tx=0x7f89b0003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:50:24.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:24.332+0000 7f89c2000700 1 -- 172.21.15.83:0/2421909050 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b0027070 con 0x7f89c41033e0 2024-04-24T08:50:24.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:24.332+0000 7f89caa00700 1 -- 172.21.15.83:0/2421909050 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89c41a7840 con 0x7f89c41033e0 2024-04-24T08:50:24.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:24.332+0000 7f89c2000700 1 -- 172.21.15.83:0/2421909050 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f89b000bdc0 con 0x7f89c41033e0 2024-04-24T08:50:24.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:24.332+0000 7f89c2000700 1 -- 172.21.15.83:0/2421909050 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b0022d40 con 0x7f89c41033e0 2024-04-24T08:50:24.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:24.332+0000 7f89caa00700 1 -- 172.21.15.83:0/2421909050 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89c41a7d20 con 0x7f89c41033e0 2024-04-24T08:50:24.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:24.333+0000 7f89caa00700 1 -- 172.21.15.83:0/2421909050 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89c410afb0 con 0x7f89c41033e0 2024-04-24T08:50:24.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:24.335+0000 7f89c2000700 1 -- 172.21.15.83:0/2421909050 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f89b002a070 con 0x7f89c41033e0 2024-04-24T08:50:24.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:24.337+0000 7f89c2000700 1 --2- 172.21.15.83:0/2421909050 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f89ac0a3c40 0x7f89ac0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:24.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:24.337+0000 7f89c2000700 1 -- 172.21.15.83:0/2421909050 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f89b00d14c0 con 0x7f89c41033e0 2024-04-24T08:50:24.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:24.338+0000 7f89c9600700 1 --2- 172.21.15.83:0/2421909050 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f89ac0a3c40 0x7f89ac0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:24.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:24.338+0000 7f89c9600700 1 --2- 172.21.15.83:0/2421909050 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f89ac0a3c40 0x7f89ac0a6100 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f89b800bbb0 tx=0x7f89b800b590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:50:24.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:24.342+0000 7f89c2000700 1 -- 172.21.15.83:0/2421909050 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f89b005eff0 con 0x7f89c41033e0 2024-04-24T08:50:24.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:50:23 smithi138 ceph-mon[31441]: pgmap v432: 129 pgs: 129 active+clean; 644 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 10 op/s 2024-04-24T08:50:24.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:50:23 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/1457796891' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-24T08:50:24.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:50:23 smithi189 ceph-mon[31791]: pgmap v432: 129 pgs: 129 active+clean; 644 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 10 op/s 2024-04-24T08:50:24.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:50:23 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/1457796891' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-24T08:50:24.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:24.570+0000 7f89caa00700 1 -- 172.21.15.83:0/2421909050 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f89c40762d0 con 0x7f89c41033e0 2024-04-24T08:50:24.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:24.571+0000 7f89c2000700 1 -- 172.21.15.83:0/2421909050 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f89b0013070 con 0x7f89c41033e0 2024-04-24T08:50:24.571 INFO:teuthology.orchestra.run.smithi083.stdout:343597383719 2024-04-24T08:50:24.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:24.574+0000 7f89caa00700 1 -- 172.21.15.83:0/2421909050 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f89ac0a3c40 msgr2=0x7f89ac0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:24.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:24.574+0000 7f89caa00700 1 --2- 172.21.15.83:0/2421909050 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f89ac0a3c40 0x7f89ac0a6100 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f89b800bbb0 tx=0x7f89b800b590 comp rx=0 tx=0).stop 2024-04-24T08:50:24.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:24.574+0000 7f89caa00700 1 -- 172.21.15.83:0/2421909050 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89c41033e0 msgr2=0x7f89c4075330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:24.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:24.574+0000 7f89caa00700 1 --2- 172.21.15.83:0/2421909050 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89c41033e0 0x7f89c4075330 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f89b0003ab0 tx=0x7f89b0003b90 comp rx=0 tx=0).stop 2024-04-24T08:50:24.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:24.576+0000 7f89caa00700 1 -- 172.21.15.83:0/2421909050 shutdown_connections 2024-04-24T08:50:24.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:24.576+0000 7f89caa00700 1 --2- 172.21.15.83:0/2421909050 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f89ac0a3c40 0x7f89ac0a6100 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:24.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:24.576+0000 7f89caa00700 1 --2- 172.21.15.83:0/2421909050 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f89c4108a20 0x7f89c4075870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:24.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:24.576+0000 7f89caa00700 1 --2- 172.21.15.83:0/2421909050 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f89c4102a20 0x7f89c4074df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:24.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:24.576+0000 7f89caa00700 1 --2- 172.21.15.83:0/2421909050 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89c41033e0 0x7f89c4075330 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:24.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:24.576+0000 7f89caa00700 1 -- 172.21.15.83:0/2421909050 >> 172.21.15.83:0/2421909050 conn(0x7f89c40fe570 msgr2=0x7f89c40fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:50:24.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:24.576+0000 7f89caa00700 1 -- 172.21.15.83:0/2421909050 shutdown_connections 2024-04-24T08:50:24.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:24.576+0000 7f89caa00700 1 -- 172.21.15.83:0/2421909050 wait complete. 2024-04-24T08:50:24.609 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920851 got 141733920857 for osd.4 2024-04-24T08:50:24.609 DEBUG:teuthology.parallel:result is None 2024-04-24T08:50:24.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:23.410+0000 7f956f200700 1 -- 172.21.15.83:0/830509296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95701099f0 msgr2=0x7f9570115e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:24.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:23.410+0000 7f956f200700 1 --2- 172.21.15.83:0/830509296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95701099f0 0x7f9570115e90 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f956800b0a0 tx=0x7f956800b3b0 comp rx=0 tx=0).stop 2024-04-24T08:50:24.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:23.410+0000 7f956f200700 1 -- 172.21.15.83:0/830509296 shutdown_connections 2024-04-24T08:50:24.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:23.410+0000 7f956f200700 1 --2- 172.21.15.83:0/830509296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95701099f0 0x7f9570115e90 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:24.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:23.410+0000 7f956f200700 1 --2- 172.21.15.83:0/830509296 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f957010f0e0 0x7f957010f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:24.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:23.410+0000 7f956f200700 1 --2- 172.21.15.83:0/830509296 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9570109030 0x7f95701094b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:24.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:23.410+0000 7f956f200700 1 -- 172.21.15.83:0/830509296 >> 172.21.15.83:0/830509296 conn(0x7f9570078880 msgr2=0x7f9570107910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:50:24.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:23.410+0000 7f956f200700 1 -- 172.21.15.83:0/830509296 shutdown_connections 2024-04-24T08:50:24.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:23.410+0000 7f956f200700 1 -- 172.21.15.83:0/830509296 wait complete. 2024-04-24T08:50:24.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:23.411+0000 7f956f200700 1 Processor -- start 2024-04-24T08:50:24.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:23.411+0000 7f956f200700 1 -- start start 2024-04-24T08:50:24.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:23.412+0000 7f956f200700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9570109030 0x7f957019f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:24.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:23.412+0000 7f956f200700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95701099f0 0x7f957019fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:24.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:23.412+0000 7f956f200700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f957010f0e0 0x7f95701a3d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:24.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:23.412+0000 7f956f200700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f95701191f0 con 0x7f9570109030 2024-04-24T08:50:24.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:23.412+0000 7f956f200700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9570119070 con 0x7f95701099f0 2024-04-24T08:50:24.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:23.412+0000 7f956f200700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9570119370 con 0x7f957010f0e0 2024-04-24T08:50:24.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:23.412+0000 7f956de00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9570109030 0x7f957019f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:24.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:23.412+0000 7f956de00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9570109030 0x7f957019f500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58334/0 (socket says 172.21.15.83:58334) 2024-04-24T08:50:24.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:23.412+0000 7f956de00700 1 -- 172.21.15.83:0/1457796891 learned_addr learned my addr 172.21.15.83:0/1457796891 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:50:24.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:23.412+0000 7f956d400700 1 --2- 172.21.15.83:0/1457796891 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95701099f0 0x7f957019fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:24.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:23.412+0000 7f956e800700 1 --2- 172.21.15.83:0/1457796891 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f957010f0e0 0x7f95701a3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:24.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:23.412+0000 7f956de00700 1 -- 172.21.15.83:0/1457796891 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f957010f0e0 msgr2=0x7f95701a3d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:24.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:23.412+0000 7f956de00700 1 --2- 172.21.15.83:0/1457796891 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f957010f0e0 0x7f95701a3d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:24.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:23.412+0000 7f956de00700 1 -- 172.21.15.83:0/1457796891 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95701099f0 msgr2=0x7f957019fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:24.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:23.412+0000 7f956de00700 1 --2- 172.21.15.83:0/1457796891 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95701099f0 0x7f957019fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:24.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:23.413+0000 7f956de00700 1 -- 172.21.15.83:0/1457796891 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95701a44c0 con 0x7f9570109030 2024-04-24T08:50:24.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:23.413+0000 7f956e800700 1 --2- 172.21.15.83:0/1457796891 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f957010f0e0 0x7f95701a3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:50:24.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:23.413+0000 7f956de00700 1 --2- 172.21.15.83:0/1457796891 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9570109030 0x7f957019f500 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f956000c9b0 tx=0x7f956000cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:50:24.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:23.413+0000 7f9566a00700 1 -- 172.21.15.83:0/1457796891 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9560007b00 con 0x7f9570109030 2024-04-24T08:50:24.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:23.413+0000 7f9566a00700 1 -- 172.21.15.83:0/1457796891 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9560007ca0 con 0x7f9570109030 2024-04-24T08:50:24.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:23.413+0000 7f9566a00700 1 -- 172.21.15.83:0/1457796891 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9560018730 con 0x7f9570109030 2024-04-24T08:50:24.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:23.413+0000 7f956f200700 1 -- 172.21.15.83:0/1457796891 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95701a47b0 con 0x7f9570109030 2024-04-24T08:50:24.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:23.413+0000 7f956f200700 1 -- 172.21.15.83:0/1457796891 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95701ac030 con 0x7f9570109030 2024-04-24T08:50:24.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:23.415+0000 7f956f200700 1 -- 172.21.15.83:0/1457796891 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95700697e0 con 0x7f9570109030 2024-04-24T08:50:24.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:23.417+0000 7f9566a00700 1 -- 172.21.15.83:0/1457796891 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f956000f450 con 0x7f9570109030 2024-04-24T08:50:24.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:23.418+0000 7f9566a00700 1 --2- 172.21.15.83:0/1457796891 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f95540a3cc0 0x7f95540a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:24.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:23.418+0000 7f9566a00700 1 -- 172.21.15.83:0/1457796891 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f95600d07a0 con 0x7f9570109030 2024-04-24T08:50:24.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:23.418+0000 7f956d400700 1 --2- 172.21.15.83:0/1457796891 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f95540a3cc0 0x7f95540a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:24.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:23.419+0000 7f956d400700 1 --2- 172.21.15.83:0/1457796891 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f95540a3cc0 0x7f95540a6180 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f955c005ea0 tx=0x7f955c005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:50:24.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:23.422+0000 7f9566a00700 1 -- 172.21.15.83:0/1457796891 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9560014030 con 0x7f9570109030 2024-04-24T08:50:24.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:23.654+0000 7f956f200700 1 -- 172.21.15.83:0/1457796891 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f957002dfe0 con 0x7f9570109030 2024-04-24T08:50:24.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:23.654+0000 7f9566a00700 1 -- 172.21.15.83:0/1457796891 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f956009cdc0 con 0x7f9570109030 2024-04-24T08:50:24.793 INFO:teuthology.orchestra.run.smithi083.stdout:197568495691 2024-04-24T08:50:24.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:23.657+0000 7f956f200700 1 -- 172.21.15.83:0/1457796891 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f95540a3cc0 msgr2=0x7f95540a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:24.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:23.657+0000 7f956f200700 1 --2- 172.21.15.83:0/1457796891 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f95540a3cc0 0x7f95540a6180 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f955c005ea0 tx=0x7f955c005e30 comp rx=0 tx=0).stop 2024-04-24T08:50:24.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:23.657+0000 7f956f200700 1 -- 172.21.15.83:0/1457796891 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9570109030 msgr2=0x7f957019f500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:24.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:23.657+0000 7f956f200700 1 --2- 172.21.15.83:0/1457796891 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9570109030 0x7f957019f500 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f956000c9b0 tx=0x7f956000cd70 comp rx=0 tx=0).stop 2024-04-24T08:50:24.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:23.657+0000 7f956f200700 1 -- 172.21.15.83:0/1457796891 shutdown_connections 2024-04-24T08:50:24.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:23.657+0000 7f956f200700 1 --2- 172.21.15.83:0/1457796891 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f95540a3cc0 0x7f95540a6180 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:24.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:23.657+0000 7f956f200700 1 --2- 172.21.15.83:0/1457796891 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9570109030 0x7f957019f500 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:24.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:23.657+0000 7f956f200700 1 --2- 172.21.15.83:0/1457796891 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f957010f0e0 0x7f95701a3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:24.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:23.657+0000 7f956f200700 1 --2- 172.21.15.83:0/1457796891 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95701099f0 0x7f957019fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:24.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:23.657+0000 7f956f200700 1 -- 172.21.15.83:0/1457796891 >> 172.21.15.83:0/1457796891 conn(0x7f9570078880 msgr2=0x7f95700798a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:50:24.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:23.658+0000 7f956f200700 1 -- 172.21.15.83:0/1457796891 shutdown_connections 2024-04-24T08:50:24.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:23.658+0000 7f956f200700 1 -- 172.21.15.83:0/1457796891 wait complete. 2024-04-24T08:50:25.263 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:50:24 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/2421909050' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-24T08:50:25.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:50:24 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/2421909050' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-24T08:50:25.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:50:24 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/2421909050' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-24T08:50:25.926 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495683 got 197568495691 for osd.6 2024-04-24T08:50:25.926 DEBUG:teuthology.parallel:result is None 2024-04-24T08:50:26.267 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:50:25 smithi083 ceph-mon[27907]: pgmap v433: 129 pgs: 129 active+clean; 644 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 341 B/s wr, 11 op/s 2024-04-24T08:50:26.267 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:50:25 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:50:26.267 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:50:25 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:50:26.311 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182491 got 107374182496 for osd.3 2024-04-24T08:50:26.312 DEBUG:teuthology.parallel:result is None 2024-04-24T08:50:26.359 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:50:25 smithi189 ceph-mon[31791]: pgmap v433: 129 pgs: 129 active+clean; 644 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 341 B/s wr, 11 op/s 2024-04-24T08:50:26.359 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:50:25 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:50:26.359 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:50:25 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:50:26.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:50:25 smithi138 ceph-mon[31441]: pgmap v433: 129 pgs: 129 active+clean; 644 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 341 B/s wr, 11 op/s 2024-04-24T08:50:26.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:50:25 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:50:26.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:50:25 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:50:27.151 INFO:tasks.cephadm.ceph_manager.ceph:need seq 343597383709 got 343597383719 for osd.11 2024-04-24T08:50:27.152 DEBUG:teuthology.parallel:result is None 2024-04-24T08:50:27.282 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:50:26 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:50:27.282 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:50:26 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:50:27.282 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:50:26 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:50:27.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:50:26 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:50:27.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:50:26 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:50:27.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:50:26 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:50:27.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:50:26 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:50:27.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:50:26 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:50:27.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:50:26 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' 2024-04-24T08:50:28.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:50:27 smithi083 ceph-mon[27907]: pgmap v434: 129 pgs: 129 active+clean; 644 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-24T08:50:28.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:50:27 smithi138 ceph-mon[31441]: pgmap v434: 129 pgs: 129 active+clean; 644 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-24T08:50:28.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:50:27 smithi189 ceph-mon[31791]: pgmap v434: 129 pgs: 129 active+clean; 644 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-24T08:50:29.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:29.094+0000 7f9312c00700 1 -- 172.21.15.83:0/1710122450 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f930c104f70 msgr2=0x7f930c105350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:29.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:29.094+0000 7f9312c00700 1 --2- 172.21.15.83:0/1710122450 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f930c104f70 0x7f930c105350 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f92fc0099c0 tx=0x7f92fc009cd0 comp rx=0 tx=0).stop 2024-04-24T08:50:29.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:29.094+0000 7f9312c00700 1 -- 172.21.15.83:0/1710122450 shutdown_connections 2024-04-24T08:50:29.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:29.094+0000 7f9312c00700 1 --2- 172.21.15.83:0/1710122450 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f930c104f70 0x7f930c105350 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:29.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:29.094+0000 7f9312c00700 1 --2- 172.21.15.83:0/1710122450 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f930c10a050 0x7f930c111c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:29.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:29.094+0000 7f9312c00700 1 --2- 172.21.15.83:0/1710122450 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f930c105920 0x7f930c109970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:29.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:29.094+0000 7f9312c00700 1 -- 172.21.15.83:0/1710122450 >> 172.21.15.83:0/1710122450 conn(0x7f930c100800 msgr2=0x7f930c102c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:50:29.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:29.094+0000 7f9312c00700 1 -- 172.21.15.83:0/1710122450 shutdown_connections 2024-04-24T08:50:29.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:29.094+0000 7f9312c00700 1 -- 172.21.15.83:0/1710122450 wait complete. 2024-04-24T08:50:29.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:29.095+0000 7f9312c00700 1 Processor -- start 2024-04-24T08:50:29.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:29.096+0000 7f9312c00700 1 -- start start 2024-04-24T08:50:29.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:29.096+0000 7f9312c00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f930c104f70 0x7f930c19b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:29.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:29.096+0000 7f9312c00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f930c105920 0x7f930c19bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:29.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:29.096+0000 7f9312c00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f930c10a050 0x7f930c19fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:29.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:29.096+0000 7f9312c00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f930c115030 con 0x7f930c105920 2024-04-24T08:50:29.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:29.096+0000 7f9312c00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f930c114eb0 con 0x7f930c10a050 2024-04-24T08:50:29.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:29.096+0000 7f9312c00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f930c1151b0 con 0x7f930c104f70 2024-04-24T08:50:29.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:29.096+0000 7f930b400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f930c105920 0x7f930c19bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:29.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:29.096+0000 7f930b400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f930c105920 0x7f930c19bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48274/0 (socket says 172.21.15.83:48274) 2024-04-24T08:50:29.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:29.097+0000 7f930b400700 1 -- 172.21.15.83:0/2725492575 learned_addr learned my addr 172.21.15.83:0/2725492575 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:50:29.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:29.097+0000 7f930be00700 1 --2- 172.21.15.83:0/2725492575 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f930c104f70 0x7f930c19b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:29.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:29.097+0000 7f9310c00700 1 --2- 172.21.15.83:0/2725492575 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f930c10a050 0x7f930c19fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:29.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:29.097+0000 7f930b400700 1 -- 172.21.15.83:0/2725492575 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f930c104f70 msgr2=0x7f930c19b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:29.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:29.097+0000 7f930b400700 1 --2- 172.21.15.83:0/2725492575 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f930c104f70 0x7f930c19b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:29.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:29.097+0000 7f930b400700 1 -- 172.21.15.83:0/2725492575 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f930c10a050 msgr2=0x7f930c19fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:29.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:29.097+0000 7f930b400700 1 --2- 172.21.15.83:0/2725492575 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f930c10a050 0x7f930c19fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:29.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:29.097+0000 7f930b400700 1 -- 172.21.15.83:0/2725492575 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f930c1a0560 con 0x7f930c105920 2024-04-24T08:50:29.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:29.097+0000 7f930b400700 1 --2- 172.21.15.83:0/2725492575 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f930c105920 0x7f930c19bae0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f92f800b500 tx=0x7f92f800b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:50:29.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:29.098+0000 7f9308c00700 1 -- 172.21.15.83:0/2725492575 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92f8015020 con 0x7f930c105920 2024-04-24T08:50:29.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:29.098+0000 7f9308c00700 1 -- 172.21.15.83:0/2725492575 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f92f8011ce0 con 0x7f930c105920 2024-04-24T08:50:29.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:29.098+0000 7f9312c00700 1 -- 172.21.15.83:0/2725492575 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f930c1a0850 con 0x7f930c105920 2024-04-24T08:50:29.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:29.098+0000 7f9308c00700 1 -- 172.21.15.83:0/2725492575 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92f8010d40 con 0x7f930c105920 2024-04-24T08:50:29.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:29.098+0000 7f9312c00700 1 -- 172.21.15.83:0/2725492575 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f930c1a80d0 con 0x7f930c105920 2024-04-24T08:50:29.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:29.099+0000 7f9312c00700 1 -- 172.21.15.83:0/2725492575 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f930c0697e0 con 0x7f930c105920 2024-04-24T08:50:29.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:29.101+0000 7f9308c00700 1 -- 172.21.15.83:0/2725492575 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f92f8018020 con 0x7f930c105920 2024-04-24T08:50:29.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:29.102+0000 7f9308c00700 1 --2- 172.21.15.83:0/2725492575 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f92f00a3cc0 0x7f92f00a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:29.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:29.102+0000 7f9308c00700 1 -- 172.21.15.83:0/2725492575 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f92f80cff50 con 0x7f930c105920 2024-04-24T08:50:29.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:29.103+0000 7f930be00700 1 --2- 172.21.15.83:0/2725492575 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f92f00a3cc0 0x7f92f00a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:29.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:29.103+0000 7f930be00700 1 --2- 172.21.15.83:0/2725492575 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f92f00a3cc0 0x7f92f00a6180 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f92fc009fd0 tx=0x7f92fc005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:50:29.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:29.107+0000 7f9308c00700 1 -- 172.21.15.83:0/2725492575 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f92f801f020 con 0x7f930c105920 2024-04-24T08:50:29.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:29.334+0000 7f9312c00700 1 -- 172.21.15.83:0/2725492575 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f930c02dfe0 con 0x7f930c105920 2024-04-24T08:50:29.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:29.335+0000 7f9308c00700 1 -- 172.21.15.83:0/2725492575 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f92f809c570 con 0x7f930c105920 2024-04-24T08:50:29.334 INFO:teuthology.orchestra.run.smithi083.stdout:163208757332 2024-04-24T08:50:29.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:29.338+0000 7f9312c00700 1 -- 172.21.15.83:0/2725492575 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f92f00a3cc0 msgr2=0x7f92f00a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:29.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:29.338+0000 7f9312c00700 1 --2- 172.21.15.83:0/2725492575 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f92f00a3cc0 0x7f92f00a6180 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f92fc009fd0 tx=0x7f92fc005fb0 comp rx=0 tx=0).stop 2024-04-24T08:50:29.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:29.338+0000 7f9312c00700 1 -- 172.21.15.83:0/2725492575 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f930c105920 msgr2=0x7f930c19bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:29.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:29.338+0000 7f9312c00700 1 --2- 172.21.15.83:0/2725492575 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f930c105920 0x7f930c19bae0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f92f800b500 tx=0x7f92f800b8c0 comp rx=0 tx=0).stop 2024-04-24T08:50:29.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:29.338+0000 7f9312c00700 1 -- 172.21.15.83:0/2725492575 shutdown_connections 2024-04-24T08:50:29.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:29.338+0000 7f9312c00700 1 --2- 172.21.15.83:0/2725492575 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f92f00a3cc0 0x7f92f00a6180 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:29.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:29.338+0000 7f9312c00700 1 --2- 172.21.15.83:0/2725492575 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f930c10a050 0x7f930c19fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:29.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:29.338+0000 7f9312c00700 1 --2- 172.21.15.83:0/2725492575 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f930c104f70 0x7f930c19b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:29.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:29.338+0000 7f9312c00700 1 --2- 172.21.15.83:0/2725492575 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f930c105920 0x7f930c19bae0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:29.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:29.338+0000 7f9312c00700 1 -- 172.21.15.83:0/2725492575 >> 172.21.15.83:0/2725492575 conn(0x7f930c100800 msgr2=0x7f930c101bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:50:29.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:29.339+0000 7f9312c00700 1 -- 172.21.15.83:0/2725492575 shutdown_connections 2024-04-24T08:50:29.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:29.339+0000 7f9312c00700 1 -- 172.21.15.83:0/2725492575 wait complete. 2024-04-24T08:50:30.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:50:30 smithi083 ceph-mon[27907]: pgmap v435: 129 pgs: 129 active+clean; 644 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-24T08:50:30.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:50:30 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/2725492575' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-24T08:50:30.149 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757325 got 163208757332 for osd.5 2024-04-24T08:50:30.149 DEBUG:teuthology.parallel:result is None 2024-04-24T08:50:30.149 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-24T08:50:30.149 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid bd3e039e-0215-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-24T08:50:30.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:50:30 smithi138 ceph-mon[31441]: pgmap v435: 129 pgs: 129 active+clean; 644 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-24T08:50:30.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:50:30 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/2725492575' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-24T08:50:30.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:50:30 smithi189 ceph-mon[31791]: pgmap v435: 129 pgs: 129 active+clean; 644 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-24T08:50:30.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:50:30 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/2725492575' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-24T08:50:30.597 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T08:50:32.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:50:32 smithi083 ceph-mon[27907]: pgmap v436: 129 pgs: 129 active+clean; 644 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-24T08:50:32.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:50:32 smithi138 ceph-mon[31441]: pgmap v436: 129 pgs: 129 active+clean; 644 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-24T08:50:32.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:50:32 smithi189 ceph-mon[31791]: pgmap v436: 129 pgs: 129 active+clean; 644 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-24T08:50:33.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:33.040+0000 7f5b42600700 1 -- 172.21.15.83:0/2610559140 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b3c109d30 msgr2=0x7f5b3c10dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:33.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:33.040+0000 7f5b42600700 1 --2- 172.21.15.83:0/2610559140 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b3c109d30 0x7f5b3c10dd80 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f5b240099c0 tx=0x7f5b24009cd0 comp rx=0 tx=0).stop 2024-04-24T08:50:33.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:33.040+0000 7f5b42600700 1 -- 172.21.15.83:0/2610559140 shutdown_connections 2024-04-24T08:50:33.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:33.040+0000 7f5b42600700 1 --2- 172.21.15.83:0/2610559140 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b3c10e460 0x7f5b3c116030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:33.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:33.040+0000 7f5b42600700 1 --2- 172.21.15.83:0/2610559140 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5b3c109380 0x7f5b3c109760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:33.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:33.040+0000 7f5b42600700 1 --2- 172.21.15.83:0/2610559140 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b3c109d30 0x7f5b3c10dd80 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:33.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:33.040+0000 7f5b42600700 1 -- 172.21.15.83:0/2610559140 >> 172.21.15.83:0/2610559140 conn(0x7f5b3c0762c0 msgr2=0x7f5b3c107910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:50:33.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:33.040+0000 7f5b42600700 1 -- 172.21.15.83:0/2610559140 shutdown_connections 2024-04-24T08:50:33.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:33.040+0000 7f5b42600700 1 -- 172.21.15.83:0/2610559140 wait complete. 2024-04-24T08:50:33.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:33.041+0000 7f5b42600700 1 Processor -- start 2024-04-24T08:50:33.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:33.042+0000 7f5b42600700 1 -- start start 2024-04-24T08:50:33.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:33.042+0000 7f5b42600700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b3c109380 0x7f5b3c1a55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:33.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:33.042+0000 7f5b42600700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5b3c109d30 0x7f5b3c1a5b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:33.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:33.042+0000 7f5b42600700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b3c10e460 0x7f5b3c19f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:33.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:33.042+0000 7f5b42600700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5b3c119340 con 0x7f5b3c109380 2024-04-24T08:50:33.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:33.042+0000 7f5b42600700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5b3c1191c0 con 0x7f5b3c10e460 2024-04-24T08:50:33.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:33.042+0000 7f5b42600700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5b3c1194c0 con 0x7f5b3c109d30 2024-04-24T08:50:33.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:33.042+0000 7f5b3b400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b3c109380 0x7f5b3c1a55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:33.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:33.042+0000 7f5b3b400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b3c109380 0x7f5b3c1a55f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48298/0 (socket says 172.21.15.83:48298) 2024-04-24T08:50:33.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:33.042+0000 7f5b3b400700 1 -- 172.21.15.83:0/1620940031 learned_addr learned my addr 172.21.15.83:0/1620940031 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:50:33.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:33.043+0000 7f5b33e00700 1 --2- 172.21.15.83:0/1620940031 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5b3c109d30 0x7f5b3c1a5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:33.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:33.043+0000 7f5b3be00700 1 --2- 172.21.15.83:0/1620940031 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b3c10e460 0x7f5b3c19f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:33.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:33.043+0000 7f5b3b400700 1 -- 172.21.15.83:0/1620940031 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5b3c109d30 msgr2=0x7f5b3c1a5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:33.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:33.043+0000 7f5b3b400700 1 --2- 172.21.15.83:0/1620940031 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5b3c109d30 0x7f5b3c1a5b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:33.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:33.043+0000 7f5b3b400700 1 -- 172.21.15.83:0/1620940031 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b3c10e460 msgr2=0x7f5b3c19f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:33.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:33.043+0000 7f5b3b400700 1 --2- 172.21.15.83:0/1620940031 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b3c10e460 0x7f5b3c19f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:33.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:33.043+0000 7f5b3b400700 1 -- 172.21.15.83:0/1620940031 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b3c19ffa0 con 0x7f5b3c109380 2024-04-24T08:50:33.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:33.043+0000 7f5b33e00700 1 --2- 172.21.15.83:0/1620940031 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5b3c109d30 0x7f5b3c1a5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:50:33.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:33.043+0000 7f5b3be00700 1 --2- 172.21.15.83:0/1620940031 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b3c10e460 0x7f5b3c19f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:50:33.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:33.043+0000 7f5b3b400700 1 --2- 172.21.15.83:0/1620940031 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b3c109380 0x7f5b3c1a55f0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c00d980 tx=0x7f5b2c00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:50:33.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:33.043+0000 7f5b38c00700 1 -- 172.21.15.83:0/1620940031 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b2c017070 con 0x7f5b3c109380 2024-04-24T08:50:33.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:33.043+0000 7f5b38c00700 1 -- 172.21.15.83:0/1620940031 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5b2c004830 con 0x7f5b3c109380 2024-04-24T08:50:33.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:33.043+0000 7f5b42600700 1 -- 172.21.15.83:0/1620940031 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b3c1a0290 con 0x7f5b3c109380 2024-04-24T08:50:33.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:33.044+0000 7f5b38c00700 1 -- 172.21.15.83:0/1620940031 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b2c005440 con 0x7f5b3c109380 2024-04-24T08:50:33.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:33.044+0000 7f5b42600700 1 -- 172.21.15.83:0/1620940031 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b3c1ac570 con 0x7f5b3c109380 2024-04-24T08:50:33.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:33.045+0000 7f5b42600700 1 -- 172.21.15.83:0/1620940031 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b3c10a860 con 0x7f5b3c109380 2024-04-24T08:50:33.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:33.047+0000 7f5b38c00700 1 -- 172.21.15.83:0/1620940031 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f5b2c0055e0 con 0x7f5b3c109380 2024-04-24T08:50:33.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:33.048+0000 7f5b38c00700 1 --2- 172.21.15.83:0/1620940031 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f5b1c0a3c40 0x7f5b1c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:33.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:33.048+0000 7f5b38c00700 1 -- 172.21.15.83:0/1620940031 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f5b2c0d0710 con 0x7f5b3c109380 2024-04-24T08:50:33.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:33.049+0000 7f5b33e00700 1 --2- 172.21.15.83:0/1620940031 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f5b1c0a3c40 0x7f5b1c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:33.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:33.049+0000 7f5b33e00700 1 --2- 172.21.15.83:0/1620940031 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f5b1c0a3c40 0x7f5b1c0a6100 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5b24006010 tx=0x7f5b24000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:50:33.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:33.053+0000 7f5b38c00700 1 -- 172.21.15.83:0/1620940031 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5b2c020020 con 0x7f5b3c109380 2024-04-24T08:50:33.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:33.272+0000 7f5b42600700 1 -- 172.21.15.83:0/1620940031 --> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5b3c1a04c0 con 0x7f5b1c0a3c40 2024-04-24T08:50:33.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:33.304+0000 7f5b38c00700 1 -- 172.21.15.83:0/1620940031 <== mgr.14152 v2:172.21.15.83:6800/2550826560 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340513 (secure 0 0 0) 0x7f5b3c1a04c0 con 0x7f5b1c0a3c40 2024-04-24T08:50:33.310 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:50:33.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:33.310+0000 7f5b42600700 1 -- 172.21.15.83:0/1620940031 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f5b1c0a3c40 msgr2=0x7f5b1c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:33.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:33.310+0000 7f5b42600700 1 --2- 172.21.15.83:0/1620940031 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f5b1c0a3c40 0x7f5b1c0a6100 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5b24006010 tx=0x7f5b24000bc0 comp rx=0 tx=0).stop 2024-04-24T08:50:33.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:33.310+0000 7f5b42600700 1 -- 172.21.15.83:0/1620940031 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b3c109380 msgr2=0x7f5b3c1a55f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:33.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:33.310+0000 7f5b42600700 1 --2- 172.21.15.83:0/1620940031 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b3c109380 0x7f5b3c1a55f0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c00d980 tx=0x7f5b2c00dd40 comp rx=0 tx=0).stop 2024-04-24T08:50:33.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:33.310+0000 7f5b42600700 1 -- 172.21.15.83:0/1620940031 shutdown_connections 2024-04-24T08:50:33.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:33.310+0000 7f5b42600700 1 --2- 172.21.15.83:0/1620940031 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f5b1c0a3c40 0x7f5b1c0a6100 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:33.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:33.310+0000 7f5b42600700 1 --2- 172.21.15.83:0/1620940031 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b3c109380 0x7f5b3c1a55f0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:33.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:33.310+0000 7f5b42600700 1 --2- 172.21.15.83:0/1620940031 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b3c10e460 0x7f5b3c19f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:33.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:33.310+0000 7f5b42600700 1 --2- 172.21.15.83:0/1620940031 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5b3c109d30 0x7f5b3c1a5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:33.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:33.310+0000 7f5b42600700 1 -- 172.21.15.83:0/1620940031 >> 172.21.15.83:0/1620940031 conn(0x7f5b3c0762c0 msgr2=0x7f5b3c077ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:50:33.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:33.311+0000 7f5b42600700 1 -- 172.21.15.83:0/1620940031 shutdown_connections 2024-04-24T08:50:33.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:33.311+0000 7f5b42600700 1 -- 172.21.15.83:0/1620940031 wait complete. 2024-04-24T08:50:33.313 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-04-24T08:50:34.267 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:50:34 smithi083 ceph-mon[27907]: pgmap v437: 129 pgs: 129 active+clean; 644 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-24T08:50:34.346 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":437,"stamp":"2024-04-24T08:50:32.892433+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658956,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1030,"num_read_kb":634,"num_write":431,"num_write_kb":3082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":854,"ondisk_log_size":854,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":384180,"kb_used_data":14228,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124557644,"statfs":{"total":1151940427776,"available":1151547027456,"internally_reserved":0,"allocated":14569472,"data_stored":7962578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":378713401},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":115,"num_read_kb":50,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006421"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.864690+0000","last_change":"2024-04-24T08:48:29.993009+0000","last_active":"2024-04-24T08:50:10.864690+0000","last_peered":"2024-04-24T08:50:10.864690+0000","last_clean":"2024-04-24T08:50:10.864690+0000","last_became_active":"2024-04-24T08:48:29.990613+0000","last_became_peered":"2024-04-24T08:48:29.990613+0000","last_unstale":"2024-04-24T08:50:10.864690+0000","last_undegraded":"2024-04-24T08:50:10.864690+0000","last_fullsized":"2024-04-24T08:50:10.864690+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:13:43.871742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.857731+0000","last_change":"2024-04-24T08:48:29.986957+0000","last_active":"2024-04-24T08:50:10.857731+0000","last_peered":"2024-04-24T08:50:10.857731+0000","last_clean":"2024-04-24T08:50:10.857731+0000","last_became_active":"2024-04-24T08:48:29.984691+0000","last_became_peered":"2024-04-24T08:48:29.984691+0000","last_unstale":"2024-04-24T08:50:10.857731+0000","last_undegraded":"2024-04-24T08:50:10.857731+0000","last_fullsized":"2024-04-24T08:50:10.857731+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:07:38.031213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.862237+0000","last_change":"2024-04-24T08:48:29.991478+0000","last_active":"2024-04-24T08:50:10.862237+0000","last_peered":"2024-04-24T08:50:10.862237+0000","last_clean":"2024-04-24T08:50:10.862237+0000","last_became_active":"2024-04-24T08:48:29.991013+0000","last_became_peered":"2024-04-24T08:48:29.991013+0000","last_unstale":"2024-04-24T08:50:10.862237+0000","last_undegraded":"2024-04-24T08:50:10.862237+0000","last_fullsized":"2024-04-24T08:50:10.862237+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:10:39.381353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.837896+0000","last_change":"2024-04-24T08:48:29.983664+0000","last_active":"2024-04-24T08:50:10.837896+0000","last_peered":"2024-04-24T08:50:10.837896+0000","last_clean":"2024-04-24T08:50:10.837896+0000","last_became_active":"2024-04-24T08:48:29.978835+0000","last_became_peered":"2024-04-24T08:48:29.978835+0000","last_unstale":"2024-04-24T08:50:10.837896+0000","last_undegraded":"2024-04-24T08:50:10.837896+0000","last_fullsized":"2024-04-24T08:50:10.837896+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:29:20.501652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.841727+0000","last_change":"2024-04-24T08:48:29.989886+0000","last_active":"2024-04-24T08:50:10.841727+0000","last_peered":"2024-04-24T08:50:10.841727+0000","last_clean":"2024-04-24T08:50:10.841727+0000","last_became_active":"2024-04-24T08:48:29.989574+0000","last_became_peered":"2024-04-24T08:48:29.989574+0000","last_unstale":"2024-04-24T08:50:10.841727+0000","last_undegraded":"2024-04-24T08:50:10.841727+0000","last_fullsized":"2024-04-24T08:50:10.841727+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:14:01.085543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.830854+0000","last_change":"2024-04-24T08:48:29.978045+0000","last_active":"2024-04-24T08:50:10.830854+0000","last_peered":"2024-04-24T08:50:10.830854+0000","last_clean":"2024-04-24T08:50:10.830854+0000","last_became_active":"2024-04-24T08:48:29.977729+0000","last_became_peered":"2024-04-24T08:48:29.977729+0000","last_unstale":"2024-04-24T08:50:10.830854+0000","last_undegraded":"2024-04-24T08:50:10.830854+0000","last_fullsized":"2024-04-24T08:50:10.830854+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:03:33.382473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.846631+0000","last_change":"2024-04-24T08:48:29.976355+0000","last_active":"2024-04-24T08:50:10.846631+0000","last_peered":"2024-04-24T08:50:10.846631+0000","last_clean":"2024-04-24T08:50:10.846631+0000","last_became_active":"2024-04-24T08:48:29.975849+0000","last_became_peered":"2024-04-24T08:48:29.975849+0000","last_unstale":"2024-04-24T08:50:10.846631+0000","last_undegraded":"2024-04-24T08:50:10.846631+0000","last_fullsized":"2024-04-24T08:50:10.846631+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:13:51.302995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.861785+0000","last_change":"2024-04-24T08:48:29.983149+0000","last_active":"2024-04-24T08:50:10.861785+0000","last_peered":"2024-04-24T08:50:10.861785+0000","last_clean":"2024-04-24T08:50:10.861785+0000","last_became_active":"2024-04-24T08:48:29.982692+0000","last_became_peered":"2024-04-24T08:48:29.982692+0000","last_unstale":"2024-04-24T08:50:10.861785+0000","last_undegraded":"2024-04-24T08:50:10.861785+0000","last_fullsized":"2024-04-24T08:50:10.861785+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:59:06.151441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.859122+0000","last_change":"2024-04-24T08:48:29.985531+0000","last_active":"2024-04-24T08:50:10.859122+0000","last_peered":"2024-04-24T08:50:10.859122+0000","last_clean":"2024-04-24T08:50:10.859122+0000","last_became_active":"2024-04-24T08:48:29.982484+0000","last_became_peered":"2024-04-24T08:48:29.982484+0000","last_unstale":"2024-04-24T08:50:10.859122+0000","last_undegraded":"2024-04-24T08:50:10.859122+0000","last_fullsized":"2024-04-24T08:50:10.859122+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:03:50.114096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.842142+0000","last_change":"2024-04-24T08:48:29.982016+0000","last_active":"2024-04-24T08:50:10.842142+0000","last_peered":"2024-04-24T08:50:10.842142+0000","last_clean":"2024-04-24T08:50:10.842142+0000","last_became_active":"2024-04-24T08:48:29.977639+0000","last_became_peered":"2024-04-24T08:48:29.977639+0000","last_unstale":"2024-04-24T08:50:10.842142+0000","last_undegraded":"2024-04-24T08:50:10.842142+0000","last_fullsized":"2024-04-24T08:50:10.842142+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:35:48.676822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.862192+0000","last_change":"2024-04-24T08:48:29.977982+0000","last_active":"2024-04-24T08:50:10.862192+0000","last_peered":"2024-04-24T08:50:10.862192+0000","last_clean":"2024-04-24T08:50:10.862192+0000","last_became_active":"2024-04-24T08:48:29.977664+0000","last_became_peered":"2024-04-24T08:48:29.977664+0000","last_unstale":"2024-04-24T08:50:10.862192+0000","last_undegraded":"2024-04-24T08:50:10.862192+0000","last_fullsized":"2024-04-24T08:50:10.862192+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:45:17.597291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.832716+0000","last_change":"2024-04-24T08:48:29.981615+0000","last_active":"2024-04-24T08:50:10.832716+0000","last_peered":"2024-04-24T08:50:10.832716+0000","last_clean":"2024-04-24T08:50:10.832716+0000","last_became_active":"2024-04-24T08:48:29.979292+0000","last_became_peered":"2024-04-24T08:48:29.979292+0000","last_unstale":"2024-04-24T08:50:10.832716+0000","last_undegraded":"2024-04-24T08:50:10.832716+0000","last_fullsized":"2024-04-24T08:50:10.832716+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:48:44.195626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.839868+0000","last_change":"2024-04-24T08:48:29.952632+0000","last_active":"2024-04-24T08:50:10.839868+0000","last_peered":"2024-04-24T08:50:10.839868+0000","last_clean":"2024-04-24T08:50:10.839868+0000","last_became_active":"2024-04-24T08:48:29.951862+0000","last_became_peered":"2024-04-24T08:48:29.951862+0000","last_unstale":"2024-04-24T08:50:10.839868+0000","last_undegraded":"2024-04-24T08:50:10.839868+0000","last_fullsized":"2024-04-24T08:50:10.839868+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:23:40.038517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.841463+0000","last_change":"2024-04-24T08:48:29.984727+0000","last_active":"2024-04-24T08:50:10.841463+0000","last_peered":"2024-04-24T08:50:10.841463+0000","last_clean":"2024-04-24T08:50:10.841463+0000","last_became_active":"2024-04-24T08:48:29.983530+0000","last_became_peered":"2024-04-24T08:48:29.983530+0000","last_unstale":"2024-04-24T08:50:10.841463+0000","last_undegraded":"2024-04-24T08:50:10.841463+0000","last_fullsized":"2024-04-24T08:50:10.841463+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:00:12.729972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.860004+0000","last_change":"2024-04-24T08:48:29.981023+0000","last_active":"2024-04-24T08:50:10.860004+0000","last_peered":"2024-04-24T08:50:10.860004+0000","last_clean":"2024-04-24T08:50:10.860004+0000","last_became_active":"2024-04-24T08:48:29.979986+0000","last_became_peered":"2024-04-24T08:48:29.979986+0000","last_unstale":"2024-04-24T08:50:10.860004+0000","last_undegraded":"2024-04-24T08:50:10.860004+0000","last_fullsized":"2024-04-24T08:50:10.860004+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:05:36.163209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.862412+0000","last_change":"2024-04-24T08:48:29.989036+0000","last_active":"2024-04-24T08:50:10.862412+0000","last_peered":"2024-04-24T08:50:10.862412+0000","last_clean":"2024-04-24T08:50:10.862412+0000","last_became_active":"2024-04-24T08:48:29.988676+0000","last_became_peered":"2024-04-24T08:48:29.988676+0000","last_unstale":"2024-04-24T08:50:10.862412+0000","last_undegraded":"2024-04-24T08:50:10.862412+0000","last_fullsized":"2024-04-24T08:50:10.862412+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:03:38.544596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.864828+0000","last_change":"2024-04-24T08:48:29.986424+0000","last_active":"2024-04-24T08:50:10.864828+0000","last_peered":"2024-04-24T08:50:10.864828+0000","last_clean":"2024-04-24T08:50:10.864828+0000","last_became_active":"2024-04-24T08:48:29.985973+0000","last_became_peered":"2024-04-24T08:48:29.985973+0000","last_unstale":"2024-04-24T08:50:10.864828+0000","last_undegraded":"2024-04-24T08:50:10.864828+0000","last_fullsized":"2024-04-24T08:50:10.864828+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:48:31.533282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.851059+0000","last_change":"2024-04-24T08:48:29.975470+0000","last_active":"2024-04-24T08:50:10.851059+0000","last_peered":"2024-04-24T08:50:10.851059+0000","last_clean":"2024-04-24T08:50:10.851059+0000","last_became_active":"2024-04-24T08:48:29.974777+0000","last_became_peered":"2024-04-24T08:48:29.974777+0000","last_unstale":"2024-04-24T08:50:10.851059+0000","last_undegraded":"2024-04-24T08:50:10.851059+0000","last_fullsized":"2024-04-24T08:50:10.851059+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:04:26.423840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.853453+0000","last_change":"2024-04-24T08:48:29.988001+0000","last_active":"2024-04-24T08:50:10.853453+0000","last_peered":"2024-04-24T08:50:10.853453+0000","last_clean":"2024-04-24T08:50:10.853453+0000","last_became_active":"2024-04-24T08:48:29.987669+0000","last_became_peered":"2024-04-24T08:48:29.987669+0000","last_unstale":"2024-04-24T08:50:10.853453+0000","last_undegraded":"2024-04-24T08:50:10.853453+0000","last_fullsized":"2024-04-24T08:50:10.853453+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:29:24.457369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:49:41.967679+0000","last_change":"2024-04-24T08:48:34.308732+0000","last_active":"2024-04-24T08:49:41.967679+0000","last_peered":"2024-04-24T08:49:41.967679+0000","last_clean":"2024-04-24T08:49:41.967679+0000","last_became_active":"2024-04-24T08:48:34.307598+0000","last_became_peered":"2024-04-24T08:48:34.307598+0000","last_unstale":"2024-04-24T08:49:41.967679+0000","last_undegraded":"2024-04-24T08:49:41.967679+0000","last_fullsized":"2024-04-24T08:49:41.967679+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:28:57.260836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'13","reported_seq":124,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:26.465217+0000","last_change":"2024-04-24T08:48:28.970230+0000","last_active":"2024-04-24T08:50:26.465217+0000","last_peered":"2024-04-24T08:50:26.465217+0000","last_clean":"2024-04-24T08:50:26.465217+0000","last_became_active":"2024-04-24T08:48:28.969878+0000","last_became_peered":"2024-04-24T08:48:28.969878+0000","last_unstale":"2024-04-24T08:50:26.465217+0000","last_undegraded":"2024-04-24T08:50:26.465217+0000","last_fullsized":"2024-04-24T08:50:26.465217+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:05:19.206557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":85,"num_read_kb":85,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.856676+0000","last_change":"2024-04-24T08:48:29.986350+0000","last_active":"2024-04-24T08:50:10.856676+0000","last_peered":"2024-04-24T08:50:10.856676+0000","last_clean":"2024-04-24T08:50:10.856676+0000","last_became_active":"2024-04-24T08:48:29.985167+0000","last_became_peered":"2024-04-24T08:48:29.985167+0000","last_unstale":"2024-04-24T08:50:10.856676+0000","last_undegraded":"2024-04-24T08:50:10.856676+0000","last_fullsized":"2024-04-24T08:50:10.856676+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:43:33.326808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:49:41.959522+0000","last_change":"2024-04-24T08:48:34.315159+0000","last_active":"2024-04-24T08:49:41.959522+0000","last_peered":"2024-04-24T08:49:41.959522+0000","last_clean":"2024-04-24T08:49:41.959522+0000","last_became_active":"2024-04-24T08:48:34.312119+0000","last_became_peered":"2024-04-24T08:48:34.312119+0000","last_unstale":"2024-04-24T08:49:41.959522+0000","last_undegraded":"2024-04-24T08:49:41.959522+0000","last_fullsized":"2024-04-24T08:49:41.959522+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:56:08.024123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:49:42.105245+0000","last_change":"2024-04-24T08:48:28.971787+0000","last_active":"2024-04-24T08:49:42.105245+0000","last_peered":"2024-04-24T08:49:42.105245+0000","last_clean":"2024-04-24T08:49:42.105245+0000","last_became_active":"2024-04-24T08:48:28.971479+0000","last_became_peered":"2024-04-24T08:48:28.971479+0000","last_unstale":"2024-04-24T08:49:42.105245+0000","last_undegraded":"2024-04-24T08:49:42.105245+0000","last_fullsized":"2024-04-24T08:49:42.105245+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:09:52.498833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.864004+0000","last_change":"2024-04-24T08:48:29.989737+0000","last_active":"2024-04-24T08:50:10.864004+0000","last_peered":"2024-04-24T08:50:10.864004+0000","last_clean":"2024-04-24T08:50:10.864004+0000","last_became_active":"2024-04-24T08:48:29.989341+0000","last_became_peered":"2024-04-24T08:48:29.989341+0000","last_unstale":"2024-04-24T08:50:10.864004+0000","last_undegraded":"2024-04-24T08:50:10.864004+0000","last_fullsized":"2024-04-24T08:50:10.864004+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:02:32.714848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:49:41.960130+0000","last_change":"2024-04-24T08:48:34.322666+0000","last_active":"2024-04-24T08:49:41.960130+0000","last_peered":"2024-04-24T08:49:41.960130+0000","last_clean":"2024-04-24T08:49:41.960130+0000","last_became_active":"2024-04-24T08:48:34.321546+0000","last_became_peered":"2024-04-24T08:48:34.321546+0000","last_unstale":"2024-04-24T08:49:41.960130+0000","last_undegraded":"2024-04-24T08:49:41.960130+0000","last_fullsized":"2024-04-24T08:49:41.960130+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:42:45.066043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:49:41.966819+0000","last_change":"2024-04-24T08:48:28.978738+0000","last_active":"2024-04-24T08:49:41.966819+0000","last_peered":"2024-04-24T08:49:41.966819+0000","last_clean":"2024-04-24T08:49:41.966819+0000","last_became_active":"2024-04-24T08:48:28.978378+0000","last_became_peered":"2024-04-24T08:48:28.978378+0000","last_unstale":"2024-04-24T08:49:41.966819+0000","last_undegraded":"2024-04-24T08:49:41.966819+0000","last_fullsized":"2024-04-24T08:49:41.966819+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:46:22.668234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.862689+0000","last_change":"2024-04-24T08:48:29.974588+0000","last_active":"2024-04-24T08:50:10.862689+0000","last_peered":"2024-04-24T08:50:10.862689+0000","last_clean":"2024-04-24T08:50:10.862689+0000","last_became_active":"2024-04-24T08:48:29.974190+0000","last_became_peered":"2024-04-24T08:48:29.974190+0000","last_unstale":"2024-04-24T08:50:10.862689+0000","last_undegraded":"2024-04-24T08:50:10.862689+0000","last_fullsized":"2024-04-24T08:50:10.862689+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:07:12.948660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:49:41.960046+0000","last_change":"2024-04-24T08:48:34.321954+0000","last_active":"2024-04-24T08:49:41.960046+0000","last_peered":"2024-04-24T08:49:41.960046+0000","last_clean":"2024-04-24T08:49:41.960046+0000","last_became_active":"2024-04-24T08:48:34.321417+0000","last_became_peered":"2024-04-24T08:48:34.321417+0000","last_unstale":"2024-04-24T08:49:41.960046+0000","last_undegraded":"2024-04-24T08:49:41.960046+0000","last_fullsized":"2024-04-24T08:49:41.960046+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:39:07.205709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:49:41.967697+0000","last_change":"2024-04-24T08:48:28.974464+0000","last_active":"2024-04-24T08:49:41.967697+0000","last_peered":"2024-04-24T08:49:41.967697+0000","last_clean":"2024-04-24T08:49:41.967697+0000","last_became_active":"2024-04-24T08:48:28.974001+0000","last_became_peered":"2024-04-24T08:48:28.974001+0000","last_unstale":"2024-04-24T08:49:41.967697+0000","last_undegraded":"2024-04-24T08:49:41.967697+0000","last_fullsized":"2024-04-24T08:49:41.967697+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:38:55.193427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.860498+0000","last_change":"2024-04-24T08:48:29.981336+0000","last_active":"2024-04-24T08:50:10.860498+0000","last_peered":"2024-04-24T08:50:10.860498+0000","last_clean":"2024-04-24T08:50:10.860498+0000","last_became_active":"2024-04-24T08:48:29.980333+0000","last_became_peered":"2024-04-24T08:48:29.980333+0000","last_unstale":"2024-04-24T08:50:10.860498+0000","last_undegraded":"2024-04-24T08:50:10.860498+0000","last_fullsized":"2024-04-24T08:50:10.860498+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:46:03.640332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:49:41.964607+0000","last_change":"2024-04-24T08:48:34.323099+0000","last_active":"2024-04-24T08:49:41.964607+0000","last_peered":"2024-04-24T08:49:41.964607+0000","last_clean":"2024-04-24T08:49:41.964607+0000","last_became_active":"2024-04-24T08:48:34.322557+0000","last_became_peered":"2024-04-24T08:48:34.322557+0000","last_unstale":"2024-04-24T08:49:41.964607+0000","last_undegraded":"2024-04-24T08:49:41.964607+0000","last_fullsized":"2024-04-24T08:49:41.964607+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:59:59.887087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"91'7","reported_seq":125,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:30.495264+0000","last_change":"2024-04-24T08:48:28.978427+0000","last_active":"2024-04-24T08:50:30.495264+0000","last_peered":"2024-04-24T08:50:30.495264+0000","last_clean":"2024-04-24T08:50:30.495264+0000","last_became_active":"2024-04-24T08:48:28.977291+0000","last_became_peered":"2024-04-24T08:48:28.977291+0000","last_unstale":"2024-04-24T08:50:30.495264+0000","last_undegraded":"2024-04-24T08:50:30.495264+0000","last_fullsized":"2024-04-24T08:50:30.495264+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:03:03.085265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":91,"num_read_kb":3,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.828288+0000","last_change":"2024-04-24T08:48:29.978008+0000","last_active":"2024-04-24T08:50:10.828288+0000","last_peered":"2024-04-24T08:50:10.828288+0000","last_clean":"2024-04-24T08:50:10.828288+0000","last_became_active":"2024-04-24T08:48:29.977389+0000","last_became_peered":"2024-04-24T08:48:29.977389+0000","last_unstale":"2024-04-24T08:50:10.828288+0000","last_undegraded":"2024-04-24T08:50:10.828288+0000","last_fullsized":"2024-04-24T08:50:10.828288+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:35:28.035378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:49:41.968546+0000","last_change":"2024-04-24T08:48:34.321477+0000","last_active":"2024-04-24T08:49:41.968546+0000","last_peered":"2024-04-24T08:49:41.968546+0000","last_clean":"2024-04-24T08:49:41.968546+0000","last_became_active":"2024-04-24T08:48:34.320860+0000","last_became_peered":"2024-04-24T08:48:34.320860+0000","last_unstale":"2024-04-24T08:49:41.968546+0000","last_undegraded":"2024-04-24T08:49:41.968546+0000","last_fullsized":"2024-04-24T08:49:41.968546+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:26:07.862590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:49:41.971416+0000","last_change":"2024-04-24T08:48:28.978147+0000","last_active":"2024-04-24T08:49:41.971416+0000","last_peered":"2024-04-24T08:49:41.971416+0000","last_clean":"2024-04-24T08:49:41.971416+0000","last_became_active":"2024-04-24T08:48:28.977646+0000","last_became_peered":"2024-04-24T08:48:28.977646+0000","last_unstale":"2024-04-24T08:49:41.971416+0000","last_undegraded":"2024-04-24T08:49:41.971416+0000","last_fullsized":"2024-04-24T08:49:41.971416+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:31:35.557886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.856313+0000","last_change":"2024-04-24T08:48:29.990788+0000","last_active":"2024-04-24T08:50:10.856313+0000","last_peered":"2024-04-24T08:50:10.856313+0000","last_clean":"2024-04-24T08:50:10.856313+0000","last_became_active":"2024-04-24T08:48:29.988476+0000","last_became_peered":"2024-04-24T08:48:29.988476+0000","last_unstale":"2024-04-24T08:50:10.856313+0000","last_undegraded":"2024-04-24T08:50:10.856313+0000","last_fullsized":"2024-04-24T08:50:10.856313+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:08:48.389877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:49:41.973582+0000","last_change":"2024-04-24T08:48:34.311020+0000","last_active":"2024-04-24T08:49:41.973582+0000","last_peered":"2024-04-24T08:49:41.973582+0000","last_clean":"2024-04-24T08:49:41.973582+0000","last_became_active":"2024-04-24T08:48:34.310395+0000","last_became_peered":"2024-04-24T08:48:34.310395+0000","last_unstale":"2024-04-24T08:49:41.973582+0000","last_undegraded":"2024-04-24T08:49:41.973582+0000","last_fullsized":"2024-04-24T08:49:41.973582+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:55:38.704954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:49:42.124068+0000","last_change":"2024-04-24T08:48:28.969991+0000","last_active":"2024-04-24T08:49:42.124068+0000","last_peered":"2024-04-24T08:49:42.124068+0000","last_clean":"2024-04-24T08:49:42.124068+0000","last_became_active":"2024-04-24T08:48:28.967937+0000","last_became_peered":"2024-04-24T08:48:28.967937+0000","last_unstale":"2024-04-24T08:49:42.124068+0000","last_undegraded":"2024-04-24T08:49:42.124068+0000","last_fullsized":"2024-04-24T08:49:42.124068+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:09:49.736108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.854053+0000","last_change":"2024-04-24T08:48:29.992814+0000","last_active":"2024-04-24T08:50:10.854053+0000","last_peered":"2024-04-24T08:50:10.854053+0000","last_clean":"2024-04-24T08:50:10.854053+0000","last_became_active":"2024-04-24T08:48:29.992188+0000","last_became_peered":"2024-04-24T08:48:29.992188+0000","last_unstale":"2024-04-24T08:50:10.854053+0000","last_undegraded":"2024-04-24T08:50:10.854053+0000","last_fullsized":"2024-04-24T08:50:10.854053+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:29:33.974958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:49:42.129127+0000","last_change":"2024-04-24T08:48:34.311196+0000","last_active":"2024-04-24T08:49:42.129127+0000","last_peered":"2024-04-24T08:49:42.129127+0000","last_clean":"2024-04-24T08:49:42.129127+0000","last_became_active":"2024-04-24T08:48:34.310723+0000","last_became_peered":"2024-04-24T08:48:34.310723+0000","last_unstale":"2024-04-24T08:49:42.129127+0000","last_undegraded":"2024-04-24T08:49:42.129127+0000","last_fullsized":"2024-04-24T08:49:42.129127+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:04:30.149510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:49:42.124430+0000","last_change":"2024-04-24T08:48:28.978534+0000","last_active":"2024-04-24T08:49:42.124430+0000","last_peered":"2024-04-24T08:49:42.124430+0000","last_clean":"2024-04-24T08:49:42.124430+0000","last_became_active":"2024-04-24T08:48:28.975407+0000","last_became_peered":"2024-04-24T08:48:28.975407+0000","last_unstale":"2024-04-24T08:49:42.124430+0000","last_undegraded":"2024-04-24T08:49:42.124430+0000","last_fullsized":"2024-04-24T08:49:42.124430+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:10:45.355039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.844780+0000","last_change":"2024-04-24T08:48:29.997682+0000","last_active":"2024-04-24T08:50:10.844780+0000","last_peered":"2024-04-24T08:50:10.844780+0000","last_clean":"2024-04-24T08:50:10.844780+0000","last_became_active":"2024-04-24T08:48:29.997223+0000","last_became_peered":"2024-04-24T08:48:29.997223+0000","last_unstale":"2024-04-24T08:50:10.844780+0000","last_undegraded":"2024-04-24T08:50:10.844780+0000","last_fullsized":"2024-04-24T08:50:10.844780+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:03:00.807249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:49:42.104177+0000","last_change":"2024-04-24T08:48:34.321477+0000","last_active":"2024-04-24T08:49:42.104177+0000","last_peered":"2024-04-24T08:49:42.104177+0000","last_clean":"2024-04-24T08:49:42.104177+0000","last_became_active":"2024-04-24T08:48:34.321018+0000","last_became_peered":"2024-04-24T08:48:34.321018+0000","last_unstale":"2024-04-24T08:49:42.104177+0000","last_undegraded":"2024-04-24T08:49:42.104177+0000","last_fullsized":"2024-04-24T08:49:42.104177+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:56:50.657298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"91'5","reported_seq":199,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:27.479876+0000","last_change":"2024-04-24T08:48:28.983722+0000","last_active":"2024-04-24T08:50:27.479876+0000","last_peered":"2024-04-24T08:50:27.479876+0000","last_clean":"2024-04-24T08:50:27.479876+0000","last_became_active":"2024-04-24T08:48:28.983394+0000","last_became_peered":"2024-04-24T08:48:28.983394+0000","last_unstale":"2024-04-24T08:50:27.479876+0000","last_undegraded":"2024-04-24T08:50:27.479876+0000","last_fullsized":"2024-04-24T08:50:27.479876+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:53:14.288742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":168,"num_read_kb":168,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.859762+0000","last_change":"2024-04-24T08:48:29.975009+0000","last_active":"2024-04-24T08:50:10.859762+0000","last_peered":"2024-04-24T08:50:10.859762+0000","last_clean":"2024-04-24T08:50:10.859762+0000","last_became_active":"2024-04-24T08:48:29.974137+0000","last_became_peered":"2024-04-24T08:48:29.974137+0000","last_unstale":"2024-04-24T08:50:10.859762+0000","last_undegraded":"2024-04-24T08:50:10.859762+0000","last_fullsized":"2024-04-24T08:50:10.859762+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:13:16.381923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:49:41.957314+0000","last_change":"2024-04-24T08:48:34.312126+0000","last_active":"2024-04-24T08:49:41.957314+0000","last_peered":"2024-04-24T08:49:41.957314+0000","last_clean":"2024-04-24T08:49:41.957314+0000","last_became_active":"2024-04-24T08:48:34.311689+0000","last_became_peered":"2024-04-24T08:48:34.311689+0000","last_unstale":"2024-04-24T08:49:41.957314+0000","last_undegraded":"2024-04-24T08:49:41.957314+0000","last_fullsized":"2024-04-24T08:49:41.957314+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:11:07.942412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:49:41.972481+0000","last_change":"2024-04-24T08:48:28.978355+0000","last_active":"2024-04-24T08:49:41.972481+0000","last_peered":"2024-04-24T08:49:41.972481+0000","last_clean":"2024-04-24T08:49:41.972481+0000","last_became_active":"2024-04-24T08:48:28.977351+0000","last_became_peered":"2024-04-24T08:48:28.977351+0000","last_unstale":"2024-04-24T08:49:41.972481+0000","last_undegraded":"2024-04-24T08:49:41.972481+0000","last_fullsized":"2024-04-24T08:49:41.972481+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:00:12.836569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.859435+0000","last_change":"2024-04-24T08:48:29.989015+0000","last_active":"2024-04-24T08:50:10.859435+0000","last_peered":"2024-04-24T08:50:10.859435+0000","last_clean":"2024-04-24T08:50:10.859435+0000","last_became_active":"2024-04-24T08:48:29.988648+0000","last_became_peered":"2024-04-24T08:48:29.988648+0000","last_unstale":"2024-04-24T08:50:10.859435+0000","last_undegraded":"2024-04-24T08:50:10.859435+0000","last_fullsized":"2024-04-24T08:50:10.859435+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:04:52.824894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:49:41.961094+0000","last_change":"2024-04-24T08:48:34.314230+0000","last_active":"2024-04-24T08:49:41.961094+0000","last_peered":"2024-04-24T08:49:41.961094+0000","last_clean":"2024-04-24T08:49:41.961094+0000","last_became_active":"2024-04-24T08:48:34.313617+0000","last_became_peered":"2024-04-24T08:48:34.313617+0000","last_unstale":"2024-04-24T08:49:41.961094+0000","last_undegraded":"2024-04-24T08:49:41.961094+0000","last_fullsized":"2024-04-24T08:49:41.961094+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:06:02.307575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:49:42.125704+0000","last_change":"2024-04-24T08:48:28.974516+0000","last_active":"2024-04-24T08:49:42.125704+0000","last_peered":"2024-04-24T08:49:42.125704+0000","last_clean":"2024-04-24T08:49:42.125704+0000","last_became_active":"2024-04-24T08:48:28.974194+0000","last_became_peered":"2024-04-24T08:48:28.974194+0000","last_unstale":"2024-04-24T08:49:42.125704+0000","last_undegraded":"2024-04-24T08:49:42.125704+0000","last_fullsized":"2024-04-24T08:49:42.125704+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:03:38.582610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.862776+0000","last_change":"2024-04-24T08:49:42.128553+0000","last_active":"2024-04-24T08:50:10.862776+0000","last_peered":"2024-04-24T08:50:10.862776+0000","last_clean":"2024-04-24T08:50:10.862776+0000","last_became_active":"2024-04-24T08:49:42.127061+0000","last_became_peered":"2024-04-24T08:49:42.127061+0000","last_unstale":"2024-04-24T08:50:10.862776+0000","last_undegraded":"2024-04-24T08:50:10.862776+0000","last_fullsized":"2024-04-24T08:50:10.862776+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:07:06.798284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:49:42.133052+0000","last_change":"2024-04-24T08:49:42.131305+0000","last_active":"2024-04-24T08:49:42.133052+0000","last_peered":"2024-04-24T08:49:42.133052+0000","last_clean":"2024-04-24T08:49:42.133052+0000","last_became_active":"2024-04-24T08:49:42.129680+0000","last_became_peered":"2024-04-24T08:49:42.129680+0000","last_unstale":"2024-04-24T08:49:42.133052+0000","last_undegraded":"2024-04-24T08:49:42.133052+0000","last_fullsized":"2024-04-24T08:49:42.133052+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:14:09.579649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"91'36","reported_seq":237,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:27.492271+0000","last_change":"2024-04-24T08:48:28.981675+0000","last_active":"2024-04-24T08:50:27.492271+0000","last_peered":"2024-04-24T08:50:27.492271+0000","last_clean":"2024-04-24T08:50:27.492271+0000","last_became_active":"2024-04-24T08:48:28.981082+0000","last_became_peered":"2024-04-24T08:48:28.981082+0000","last_unstale":"2024-04-24T08:50:27.492271+0000","last_undegraded":"2024-04-24T08:50:27.492271+0000","last_fullsized":"2024-04-24T08:50:27.492271+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:22:07.826011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61400,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":175,"num_read_kb":147,"num_write":38,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.845675+0000","last_change":"2024-04-24T08:48:29.986932+0000","last_active":"2024-04-24T08:50:10.845675+0000","last_peered":"2024-04-24T08:50:10.845675+0000","last_clean":"2024-04-24T08:50:10.845675+0000","last_became_active":"2024-04-24T08:48:29.986505+0000","last_became_peered":"2024-04-24T08:48:29.986505+0000","last_unstale":"2024-04-24T08:50:10.845675+0000","last_undegraded":"2024-04-24T08:50:10.845675+0000","last_fullsized":"2024-04-24T08:50:10.845675+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:53:36.050294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:49:42.126219+0000","last_change":"2024-04-24T08:48:34.327127+0000","last_active":"2024-04-24T08:49:42.126219+0000","last_peered":"2024-04-24T08:49:42.126219+0000","last_clean":"2024-04-24T08:49:42.126219+0000","last_became_active":"2024-04-24T08:48:34.323942+0000","last_became_peered":"2024-04-24T08:48:34.323942+0000","last_unstale":"2024-04-24T08:49:42.126219+0000","last_undegraded":"2024-04-24T08:49:42.126219+0000","last_fullsized":"2024-04-24T08:49:42.126219+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:22:49.637513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:49:41.965201+0000","last_change":"2024-04-24T08:48:28.978618+0000","last_active":"2024-04-24T08:49:41.965201+0000","last_peered":"2024-04-24T08:49:41.965201+0000","last_clean":"2024-04-24T08:49:41.965201+0000","last_became_active":"2024-04-24T08:48:28.978309+0000","last_became_peered":"2024-04-24T08:48:28.978309+0000","last_unstale":"2024-04-24T08:49:41.965201+0000","last_undegraded":"2024-04-24T08:49:41.965201+0000","last_fullsized":"2024-04-24T08:49:41.965201+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:35:50.091398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.852209+0000","last_change":"2024-04-24T08:48:29.985629+0000","last_active":"2024-04-24T08:50:10.852209+0000","last_peered":"2024-04-24T08:50:10.852209+0000","last_clean":"2024-04-24T08:50:10.852209+0000","last_became_active":"2024-04-24T08:48:29.985258+0000","last_became_peered":"2024-04-24T08:48:29.985258+0000","last_unstale":"2024-04-24T08:50:10.852209+0000","last_undegraded":"2024-04-24T08:50:10.852209+0000","last_fullsized":"2024-04-24T08:50:10.852209+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:57:29.302175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:49:42.129242+0000","last_change":"2024-04-24T08:49:42.129242+0000","last_active":"2024-04-24T08:49:42.129242+0000","last_peered":"2024-04-24T08:49:42.129242+0000","last_clean":"2024-04-24T08:49:42.129242+0000","last_became_active":"2024-04-24T08:49:42.127967+0000","last_became_peered":"2024-04-24T08:49:42.127967+0000","last_unstale":"2024-04-24T08:49:42.129242+0000","last_undegraded":"2024-04-24T08:49:42.129242+0000","last_fullsized":"2024-04-24T08:49:42.129242+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:42:11.206123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:49:42.124700+0000","last_change":"2024-04-24T08:48:28.977840+0000","last_active":"2024-04-24T08:49:42.124700+0000","last_peered":"2024-04-24T08:49:42.124700+0000","last_clean":"2024-04-24T08:49:42.124700+0000","last_became_active":"2024-04-24T08:48:28.975482+0000","last_became_peered":"2024-04-24T08:48:28.975482+0000","last_unstale":"2024-04-24T08:49:42.124700+0000","last_undegraded":"2024-04-24T08:49:42.124700+0000","last_fullsized":"2024-04-24T08:49:42.124700+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:51:52.468627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.860821+0000","last_change":"2024-04-24T08:49:42.129823+0000","last_active":"2024-04-24T08:50:10.860821+0000","last_peered":"2024-04-24T08:50:10.860821+0000","last_clean":"2024-04-24T08:50:10.860821+0000","last_became_active":"2024-04-24T08:49:42.127254+0000","last_became_peered":"2024-04-24T08:49:42.127254+0000","last_unstale":"2024-04-24T08:50:10.860821+0000","last_undegraded":"2024-04-24T08:50:10.860821+0000","last_fullsized":"2024-04-24T08:50:10.860821+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:51:10.804703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:49:41.958200+0000","last_change":"2024-04-24T08:48:34.314956+0000","last_active":"2024-04-24T08:49:41.958200+0000","last_peered":"2024-04-24T08:49:41.958200+0000","last_clean":"2024-04-24T08:49:41.958200+0000","last_became_active":"2024-04-24T08:48:34.314541+0000","last_became_peered":"2024-04-24T08:48:34.314541+0000","last_unstale":"2024-04-24T08:49:41.958200+0000","last_undegraded":"2024-04-24T08:49:41.958200+0000","last_fullsized":"2024-04-24T08:49:41.958200+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:42:48.614000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"89'5","reported_seq":123,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:31.534363+0000","last_change":"2024-04-24T08:48:28.974529+0000","last_active":"2024-04-24T08:50:31.534363+0000","last_peered":"2024-04-24T08:50:31.534363+0000","last_clean":"2024-04-24T08:50:31.534363+0000","last_became_active":"2024-04-24T08:48:28.973957+0000","last_became_peered":"2024-04-24T08:48:28.973957+0000","last_unstale":"2024-04-24T08:50:31.534363+0000","last_undegraded":"2024-04-24T08:50:31.534363+0000","last_fullsized":"2024-04-24T08:50:31.534363+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:44:46.216389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":92,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.847638+0000","last_change":"2024-04-24T08:48:29.981995+0000","last_active":"2024-04-24T08:50:10.847638+0000","last_peered":"2024-04-24T08:50:10.847638+0000","last_clean":"2024-04-24T08:50:10.847638+0000","last_became_active":"2024-04-24T08:48:29.980756+0000","last_became_peered":"2024-04-24T08:48:29.980756+0000","last_unstale":"2024-04-24T08:50:10.847638+0000","last_undegraded":"2024-04-24T08:50:10.847638+0000","last_fullsized":"2024-04-24T08:50:10.847638+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:42:52.770069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:49:42.122633+0000","last_change":"2024-04-24T08:48:34.308836+0000","last_active":"2024-04-24T08:49:42.122633+0000","last_peered":"2024-04-24T08:49:42.122633+0000","last_clean":"2024-04-24T08:49:42.122633+0000","last_became_active":"2024-04-24T08:48:34.308470+0000","last_became_peered":"2024-04-24T08:48:34.308470+0000","last_unstale":"2024-04-24T08:49:42.122633+0000","last_undegraded":"2024-04-24T08:49:42.122633+0000","last_fullsized":"2024-04-24T08:49:42.122633+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:38:14.906117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:49:41.966163+0000","last_change":"2024-04-24T08:48:28.971818+0000","last_active":"2024-04-24T08:49:41.966163+0000","last_peered":"2024-04-24T08:49:41.966163+0000","last_clean":"2024-04-24T08:49:41.966163+0000","last_became_active":"2024-04-24T08:48:28.971463+0000","last_became_peered":"2024-04-24T08:48:28.971463+0000","last_unstale":"2024-04-24T08:49:41.966163+0000","last_undegraded":"2024-04-24T08:49:41.966163+0000","last_fullsized":"2024-04-24T08:49:41.966163+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:10:22.829722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.860330+0000","last_change":"2024-04-24T08:48:29.980632+0000","last_active":"2024-04-24T08:50:10.860330+0000","last_peered":"2024-04-24T08:50:10.860330+0000","last_clean":"2024-04-24T08:50:10.860330+0000","last_became_active":"2024-04-24T08:48:29.980174+0000","last_became_peered":"2024-04-24T08:48:29.980174+0000","last_unstale":"2024-04-24T08:50:10.860330+0000","last_undegraded":"2024-04-24T08:50:10.860330+0000","last_fullsized":"2024-04-24T08:50:10.860330+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:45:32.709091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:49:42.122941+0000","last_change":"2024-04-24T08:48:34.316302+0000","last_active":"2024-04-24T08:49:42.122941+0000","last_peered":"2024-04-24T08:49:42.122941+0000","last_clean":"2024-04-24T08:49:42.122941+0000","last_became_active":"2024-04-24T08:48:34.315783+0000","last_became_peered":"2024-04-24T08:48:34.315783+0000","last_unstale":"2024-04-24T08:49:42.122941+0000","last_undegraded":"2024-04-24T08:49:42.122941+0000","last_fullsized":"2024-04-24T08:49:42.122941+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:29:01.733931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:49:41.966235+0000","last_change":"2024-04-24T08:48:28.972033+0000","last_active":"2024-04-24T08:49:41.966235+0000","last_peered":"2024-04-24T08:49:41.966235+0000","last_clean":"2024-04-24T08:49:41.966235+0000","last_became_active":"2024-04-24T08:48:28.971597+0000","last_became_peered":"2024-04-24T08:48:28.971597+0000","last_unstale":"2024-04-24T08:49:41.966235+0000","last_undegraded":"2024-04-24T08:49:41.966235+0000","last_fullsized":"2024-04-24T08:49:41.966235+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:11:27.827599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.837485+0000","last_change":"2024-04-24T08:49:41.975577+0000","last_active":"2024-04-24T08:50:10.837485+0000","last_peered":"2024-04-24T08:50:10.837485+0000","last_clean":"2024-04-24T08:50:10.837485+0000","last_became_active":"2024-04-24T08:49:41.975018+0000","last_became_peered":"2024-04-24T08:49:41.975018+0000","last_unstale":"2024-04-24T08:50:10.837485+0000","last_undegraded":"2024-04-24T08:50:10.837485+0000","last_fullsized":"2024-04-24T08:50:10.837485+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:07:53.051847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:49:41.978827+0000","last_change":"2024-04-24T08:49:41.978827+0000","last_active":"2024-04-24T08:49:41.978827+0000","last_peered":"2024-04-24T08:49:41.978827+0000","last_clean":"2024-04-24T08:49:41.978827+0000","last_became_active":"2024-04-24T08:49:41.977741+0000","last_became_peered":"2024-04-24T08:49:41.977741+0000","last_unstale":"2024-04-24T08:49:41.978827+0000","last_undegraded":"2024-04-24T08:49:41.978827+0000","last_fullsized":"2024-04-24T08:49:41.978827+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:16:54.754341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:49:41.958636+0000","last_change":"2024-04-24T08:48:28.970726+0000","last_active":"2024-04-24T08:49:41.958636+0000","last_peered":"2024-04-24T08:49:41.958636+0000","last_clean":"2024-04-24T08:49:41.958636+0000","last_became_active":"2024-04-24T08:48:28.969911+0000","last_became_peered":"2024-04-24T08:48:28.969911+0000","last_unstale":"2024-04-24T08:49:41.958636+0000","last_undegraded":"2024-04-24T08:49:41.958636+0000","last_fullsized":"2024-04-24T08:49:41.958636+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:55:26.357256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'4","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.851182+0000","last_change":"2024-04-24T08:48:29.967804+0000","last_active":"2024-04-24T08:50:10.851182+0000","last_peered":"2024-04-24T08:50:10.851182+0000","last_clean":"2024-04-24T08:50:10.851182+0000","last_became_active":"2024-04-24T08:48:29.967431+0000","last_became_peered":"2024-04-24T08:48:29.967431+0000","last_unstale":"2024-04-24T08:50:10.851182+0000","last_undegraded":"2024-04-24T08:50:10.851182+0000","last_fullsized":"2024-04-24T08:50:10.851182+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:00:02.779543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:49:42.124119+0000","last_change":"2024-04-24T08:48:34.316844+0000","last_active":"2024-04-24T08:49:42.124119+0000","last_peered":"2024-04-24T08:49:42.124119+0000","last_clean":"2024-04-24T08:49:42.124119+0000","last_became_active":"2024-04-24T08:48:34.316426+0000","last_became_peered":"2024-04-24T08:48:34.316426+0000","last_unstale":"2024-04-24T08:49:42.124119+0000","last_undegraded":"2024-04-24T08:49:42.124119+0000","last_fullsized":"2024-04-24T08:49:42.124119+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:10:13.378443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'5","reported_seq":118,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:28.473072+0000","last_change":"2024-04-24T08:48:28.971544+0000","last_active":"2024-04-24T08:50:28.473072+0000","last_peered":"2024-04-24T08:50:28.473072+0000","last_clean":"2024-04-24T08:50:28.473072+0000","last_became_active":"2024-04-24T08:48:28.970417+0000","last_became_peered":"2024-04-24T08:48:28.970417+0000","last_unstale":"2024-04-24T08:50:28.473072+0000","last_undegraded":"2024-04-24T08:50:28.473072+0000","last_fullsized":"2024-04-24T08:50:28.473072+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:35:04.506126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":87,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.847509+0000","last_change":"2024-04-24T08:48:29.987542+0000","last_active":"2024-04-24T08:50:10.847509+0000","last_peered":"2024-04-24T08:50:10.847509+0000","last_clean":"2024-04-24T08:50:10.847509+0000","last_became_active":"2024-04-24T08:48:29.987258+0000","last_became_peered":"2024-04-24T08:48:29.987258+0000","last_unstale":"2024-04-24T08:50:10.847509+0000","last_undegraded":"2024-04-24T08:50:10.847509+0000","last_fullsized":"2024-04-24T08:50:10.847509+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:37:23.341132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:49:42.122302+0000","last_change":"2024-04-24T08:48:34.322636+0000","last_active":"2024-04-24T08:49:42.122302+0000","last_peered":"2024-04-24T08:49:42.122302+0000","last_clean":"2024-04-24T08:49:42.122302+0000","last_became_active":"2024-04-24T08:48:34.321764+0000","last_became_peered":"2024-04-24T08:48:34.321764+0000","last_unstale":"2024-04-24T08:49:42.122302+0000","last_undegraded":"2024-04-24T08:49:42.122302+0000","last_fullsized":"2024-04-24T08:49:42.122302+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:56:10.563907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.836543+0000","last_change":"2024-04-24T08:48:29.981734+0000","last_active":"2024-04-24T08:50:10.836543+0000","last_peered":"2024-04-24T08:50:10.836543+0000","last_clean":"2024-04-24T08:50:10.836543+0000","last_became_active":"2024-04-24T08:48:29.980491+0000","last_became_peered":"2024-04-24T08:48:29.980491+0000","last_unstale":"2024-04-24T08:50:10.836543+0000","last_undegraded":"2024-04-24T08:50:10.836543+0000","last_fullsized":"2024-04-24T08:50:10.836543+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:08:45.189026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'5","reported_seq":118,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:27.519974+0000","last_change":"2024-04-24T08:48:28.954621+0000","last_active":"2024-04-24T08:50:27.519974+0000","last_peered":"2024-04-24T08:50:27.519974+0000","last_clean":"2024-04-24T08:50:27.519974+0000","last_became_active":"2024-04-24T08:48:28.954139+0000","last_became_peered":"2024-04-24T08:48:28.954139+0000","last_unstale":"2024-04-24T08:50:27.519974+0000","last_undegraded":"2024-04-24T08:50:27.519974+0000","last_fullsized":"2024-04-24T08:50:27.519974+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:59:19.360414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":86,"num_read_kb":86,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:49:41.970520+0000","last_change":"2024-04-24T08:48:34.323774+0000","last_active":"2024-04-24T08:49:41.970520+0000","last_peered":"2024-04-24T08:49:41.970520+0000","last_clean":"2024-04-24T08:49:41.970520+0000","last_became_active":"2024-04-24T08:48:34.322214+0000","last_became_peered":"2024-04-24T08:48:34.322214+0000","last_unstale":"2024-04-24T08:49:41.970520+0000","last_undegraded":"2024-04-24T08:49:41.970520+0000","last_fullsized":"2024-04-24T08:49:41.970520+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:51:03.971107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.860028+0000","last_change":"2024-04-24T08:48:29.995372+0000","last_active":"2024-04-24T08:50:10.860028+0000","last_peered":"2024-04-24T08:50:10.860028+0000","last_clean":"2024-04-24T08:50:10.860028+0000","last_became_active":"2024-04-24T08:48:29.994744+0000","last_became_peered":"2024-04-24T08:48:29.994744+0000","last_unstale":"2024-04-24T08:50:10.860028+0000","last_undegraded":"2024-04-24T08:50:10.860028+0000","last_fullsized":"2024-04-24T08:50:10.860028+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:28:55.356390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:49:41.960779+0000","last_change":"2024-04-24T08:48:28.974921+0000","last_active":"2024-04-24T08:49:41.960779+0000","last_peered":"2024-04-24T08:49:41.960779+0000","last_clean":"2024-04-24T08:49:41.960779+0000","last_became_active":"2024-04-24T08:48:28.974113+0000","last_became_peered":"2024-04-24T08:48:28.974113+0000","last_unstale":"2024-04-24T08:49:41.960779+0000","last_undegraded":"2024-04-24T08:49:41.960779+0000","last_fullsized":"2024-04-24T08:49:41.960779+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:35:12.635253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":360,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:49:41.971050+0000","last_change":"2024-04-24T08:46:45.190615+0000","last_active":"2024-04-24T08:49:41.971050+0000","last_peered":"2024-04-24T08:49:41.971050+0000","last_clean":"2024-04-24T08:49:41.971050+0000","last_became_active":"2024-04-24T08:46:45.190011+0000","last_became_peered":"2024-04-24T08:46:45.190011+0000","last_unstale":"2024-04-24T08:49:41.971050+0000","last_undegraded":"2024-04-24T08:49:41.971050+0000","last_fullsized":"2024-04-24T08:49:41.971050+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:41:57.806569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:41:57.806569+0000","last_clean_scrub_stamp":"2024-04-24T08:41:57.806569+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:13:08.283236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:49:42.126082+0000","last_change":"2024-04-24T08:48:34.324588+0000","last_active":"2024-04-24T08:49:42.126082+0000","last_peered":"2024-04-24T08:49:42.126082+0000","last_clean":"2024-04-24T08:49:42.126082+0000","last_became_active":"2024-04-24T08:48:34.321969+0000","last_became_peered":"2024-04-24T08:48:34.321969+0000","last_unstale":"2024-04-24T08:49:42.126082+0000","last_undegraded":"2024-04-24T08:49:42.126082+0000","last_fullsized":"2024-04-24T08:49:42.126082+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:19:56.921198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.840077+0000","last_change":"2024-04-24T08:48:29.981573+0000","last_active":"2024-04-24T08:50:10.840077+0000","last_peered":"2024-04-24T08:50:10.840077+0000","last_clean":"2024-04-24T08:50:10.840077+0000","last_became_active":"2024-04-24T08:48:29.979216+0000","last_became_peered":"2024-04-24T08:48:29.979216+0000","last_unstale":"2024-04-24T08:50:10.840077+0000","last_undegraded":"2024-04-24T08:50:10.840077+0000","last_fullsized":"2024-04-24T08:50:10.840077+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:38:09.164883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:49:41.971476+0000","last_change":"2024-04-24T08:48:28.981492+0000","last_active":"2024-04-24T08:49:41.971476+0000","last_peered":"2024-04-24T08:49:41.971476+0000","last_clean":"2024-04-24T08:49:41.971476+0000","last_became_active":"2024-04-24T08:48:28.981050+0000","last_became_peered":"2024-04-24T08:48:28.981050+0000","last_unstale":"2024-04-24T08:49:41.971476+0000","last_undegraded":"2024-04-24T08:49:41.971476+0000","last_fullsized":"2024-04-24T08:49:41.971476+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:22:09.829566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:49:41.969209+0000","last_change":"2024-04-24T08:48:34.317502+0000","last_active":"2024-04-24T08:49:41.969209+0000","last_peered":"2024-04-24T08:49:41.969209+0000","last_clean":"2024-04-24T08:49:41.969209+0000","last_became_active":"2024-04-24T08:48:34.316614+0000","last_became_peered":"2024-04-24T08:48:34.316614+0000","last_unstale":"2024-04-24T08:49:41.969209+0000","last_undegraded":"2024-04-24T08:49:41.969209+0000","last_fullsized":"2024-04-24T08:49:41.969209+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:44:12.469926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:49:41.965412+0000","last_change":"2024-04-24T08:48:28.970261+0000","last_active":"2024-04-24T08:49:41.965412+0000","last_peered":"2024-04-24T08:49:41.965412+0000","last_clean":"2024-04-24T08:49:41.965412+0000","last_became_active":"2024-04-24T08:48:28.969813+0000","last_became_peered":"2024-04-24T08:48:28.969813+0000","last_unstale":"2024-04-24T08:49:41.965412+0000","last_undegraded":"2024-04-24T08:49:41.965412+0000","last_fullsized":"2024-04-24T08:49:41.965412+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:46:12.311756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.838518+0000","last_change":"2024-04-24T08:48:29.976712+0000","last_active":"2024-04-24T08:50:10.838518+0000","last_peered":"2024-04-24T08:50:10.838518+0000","last_clean":"2024-04-24T08:50:10.838518+0000","last_became_active":"2024-04-24T08:48:29.976166+0000","last_became_peered":"2024-04-24T08:48:29.976166+0000","last_unstale":"2024-04-24T08:50:10.838518+0000","last_undegraded":"2024-04-24T08:50:10.838518+0000","last_fullsized":"2024-04-24T08:50:10.838518+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:52:53.757829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:49:42.103682+0000","last_change":"2024-04-24T08:48:34.310173+0000","last_active":"2024-04-24T08:49:42.103682+0000","last_peered":"2024-04-24T08:49:42.103682+0000","last_clean":"2024-04-24T08:49:42.103682+0000","last_became_active":"2024-04-24T08:48:34.309772+0000","last_became_peered":"2024-04-24T08:48:34.309772+0000","last_unstale":"2024-04-24T08:49:42.103682+0000","last_undegraded":"2024-04-24T08:49:42.103682+0000","last_fullsized":"2024-04-24T08:49:42.103682+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:35:50.302769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'14","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:49:41.959148+0000","last_change":"2024-04-24T08:48:28.970127+0000","last_active":"2024-04-24T08:49:41.959148+0000","last_peered":"2024-04-24T08:49:41.959148+0000","last_clean":"2024-04-24T08:49:41.959148+0000","last_became_active":"2024-04-24T08:48:28.968803+0000","last_became_peered":"2024-04-24T08:48:28.968803+0000","last_unstale":"2024-04-24T08:49:41.959148+0000","last_undegraded":"2024-04-24T08:49:41.959148+0000","last_fullsized":"2024-04-24T08:49:41.959148+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:28:47.782276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.853054+0000","last_change":"2024-04-24T08:48:29.992964+0000","last_active":"2024-04-24T08:50:10.853054+0000","last_peered":"2024-04-24T08:50:10.853054+0000","last_clean":"2024-04-24T08:50:10.853054+0000","last_became_active":"2024-04-24T08:48:29.987501+0000","last_became_peered":"2024-04-24T08:48:29.987501+0000","last_unstale":"2024-04-24T08:50:10.853054+0000","last_undegraded":"2024-04-24T08:50:10.853054+0000","last_fullsized":"2024-04-24T08:50:10.853054+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:49:19.157052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:49:41.960206+0000","last_change":"2024-04-24T08:48:34.315393+0000","last_active":"2024-04-24T08:49:41.960206+0000","last_peered":"2024-04-24T08:49:41.960206+0000","last_clean":"2024-04-24T08:49:41.960206+0000","last_became_active":"2024-04-24T08:48:34.314943+0000","last_became_peered":"2024-04-24T08:48:34.314943+0000","last_unstale":"2024-04-24T08:49:41.960206+0000","last_undegraded":"2024-04-24T08:49:41.960206+0000","last_fullsized":"2024-04-24T08:49:41.960206+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:03:13.312223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:49:41.963669+0000","last_change":"2024-04-24T08:48:28.969813+0000","last_active":"2024-04-24T08:49:41.963669+0000","last_peered":"2024-04-24T08:49:41.963669+0000","last_clean":"2024-04-24T08:49:41.963669+0000","last_became_active":"2024-04-24T08:48:28.968485+0000","last_became_peered":"2024-04-24T08:48:28.968485+0000","last_unstale":"2024-04-24T08:49:41.963669+0000","last_undegraded":"2024-04-24T08:49:41.963669+0000","last_fullsized":"2024-04-24T08:49:41.963669+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:32:10.095723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.862968+0000","last_change":"2024-04-24T08:48:29.997731+0000","last_active":"2024-04-24T08:50:10.862968+0000","last_peered":"2024-04-24T08:50:10.862968+0000","last_clean":"2024-04-24T08:50:10.862968+0000","last_became_active":"2024-04-24T08:48:29.997373+0000","last_became_peered":"2024-04-24T08:48:29.997373+0000","last_unstale":"2024-04-24T08:50:10.862968+0000","last_undegraded":"2024-04-24T08:50:10.862968+0000","last_fullsized":"2024-04-24T08:50:10.862968+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:57:19.531173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:49:41.961100+0000","last_change":"2024-04-24T08:48:34.326170+0000","last_active":"2024-04-24T08:49:41.961100+0000","last_peered":"2024-04-24T08:49:41.961100+0000","last_clean":"2024-04-24T08:49:41.961100+0000","last_became_active":"2024-04-24T08:48:34.325678+0000","last_became_peered":"2024-04-24T08:48:34.325678+0000","last_unstale":"2024-04-24T08:49:41.961100+0000","last_undegraded":"2024-04-24T08:49:41.961100+0000","last_fullsized":"2024-04-24T08:49:41.961100+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:12:47.003864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:49:41.968205+0000","last_change":"2024-04-24T08:48:28.975522+0000","last_active":"2024-04-24T08:49:41.968205+0000","last_peered":"2024-04-24T08:49:41.968205+0000","last_clean":"2024-04-24T08:49:41.968205+0000","last_became_active":"2024-04-24T08:48:28.975093+0000","last_became_peered":"2024-04-24T08:48:28.975093+0000","last_unstale":"2024-04-24T08:49:41.968205+0000","last_undegraded":"2024-04-24T08:49:41.968205+0000","last_fullsized":"2024-04-24T08:49:41.968205+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:37:31.809694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.795450+0000","last_change":"2024-04-24T08:48:29.982502+0000","last_active":"2024-04-24T08:50:10.795450+0000","last_peered":"2024-04-24T08:50:10.795450+0000","last_clean":"2024-04-24T08:50:10.795450+0000","last_became_active":"2024-04-24T08:48:29.982141+0000","last_became_peered":"2024-04-24T08:48:29.982141+0000","last_unstale":"2024-04-24T08:50:10.795450+0000","last_undegraded":"2024-04-24T08:50:10.795450+0000","last_fullsized":"2024-04-24T08:50:10.795450+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:46:15.851359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:49:41.958317+0000","last_change":"2024-04-24T08:48:34.322717+0000","last_active":"2024-04-24T08:49:41.958317+0000","last_peered":"2024-04-24T08:49:41.958317+0000","last_clean":"2024-04-24T08:49:41.958317+0000","last_became_active":"2024-04-24T08:48:34.321837+0000","last_became_peered":"2024-04-24T08:48:34.321837+0000","last_unstale":"2024-04-24T08:49:41.958317+0000","last_undegraded":"2024-04-24T08:49:41.958317+0000","last_fullsized":"2024-04-24T08:49:41.958317+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:40:11.557436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:49:42.103699+0000","last_change":"2024-04-24T08:48:28.973951+0000","last_active":"2024-04-24T08:49:42.103699+0000","last_peered":"2024-04-24T08:49:42.103699+0000","last_clean":"2024-04-24T08:49:42.103699+0000","last_became_active":"2024-04-24T08:48:28.973131+0000","last_became_peered":"2024-04-24T08:48:28.973131+0000","last_unstale":"2024-04-24T08:49:42.103699+0000","last_undegraded":"2024-04-24T08:49:42.103699+0000","last_fullsized":"2024-04-24T08:49:42.103699+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:45:45.197337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.856618+0000","last_change":"2024-04-24T08:48:29.992689+0000","last_active":"2024-04-24T08:50:10.856618+0000","last_peered":"2024-04-24T08:50:10.856618+0000","last_clean":"2024-04-24T08:50:10.856618+0000","last_became_active":"2024-04-24T08:48:29.988874+0000","last_became_peered":"2024-04-24T08:48:29.988874+0000","last_unstale":"2024-04-24T08:50:10.856618+0000","last_undegraded":"2024-04-24T08:50:10.856618+0000","last_fullsized":"2024-04-24T08:50:10.856618+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:42:41.578424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:49:41.961021+0000","last_change":"2024-04-24T08:48:34.318640+0000","last_active":"2024-04-24T08:49:41.961021+0000","last_peered":"2024-04-24T08:49:41.961021+0000","last_clean":"2024-04-24T08:49:41.961021+0000","last_became_active":"2024-04-24T08:48:34.318039+0000","last_became_peered":"2024-04-24T08:48:34.318039+0000","last_unstale":"2024-04-24T08:49:41.961021+0000","last_undegraded":"2024-04-24T08:49:41.961021+0000","last_fullsized":"2024-04-24T08:49:41.961021+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:17:27.110930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'3","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:49:41.964478+0000","last_change":"2024-04-24T08:48:28.980828+0000","last_active":"2024-04-24T08:49:41.964478+0000","last_peered":"2024-04-24T08:49:41.964478+0000","last_clean":"2024-04-24T08:49:41.964478+0000","last_became_active":"2024-04-24T08:48:28.979814+0000","last_became_peered":"2024-04-24T08:48:28.979814+0000","last_unstale":"2024-04-24T08:49:41.964478+0000","last_undegraded":"2024-04-24T08:49:41.964478+0000","last_fullsized":"2024-04-24T08:49:41.964478+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:02:28.145847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.861479+0000","last_change":"2024-04-24T08:48:29.995003+0000","last_active":"2024-04-24T08:50:10.861479+0000","last_peered":"2024-04-24T08:50:10.861479+0000","last_clean":"2024-04-24T08:50:10.861479+0000","last_became_active":"2024-04-24T08:48:29.994516+0000","last_became_peered":"2024-04-24T08:48:29.994516+0000","last_unstale":"2024-04-24T08:50:10.861479+0000","last_undegraded":"2024-04-24T08:50:10.861479+0000","last_fullsized":"2024-04-24T08:50:10.861479+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:15:46.094534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:49:41.960975+0000","last_change":"2024-04-24T08:48:34.311721+0000","last_active":"2024-04-24T08:49:41.960975+0000","last_peered":"2024-04-24T08:49:41.960975+0000","last_clean":"2024-04-24T08:49:41.960975+0000","last_became_active":"2024-04-24T08:48:34.311241+0000","last_became_peered":"2024-04-24T08:48:34.311241+0000","last_unstale":"2024-04-24T08:49:41.960975+0000","last_undegraded":"2024-04-24T08:49:41.960975+0000","last_fullsized":"2024-04-24T08:49:41.960975+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:16:17.679237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:49:41.964980+0000","last_change":"2024-04-24T08:48:28.973336+0000","last_active":"2024-04-24T08:49:41.964980+0000","last_peered":"2024-04-24T08:49:41.964980+0000","last_clean":"2024-04-24T08:49:41.964980+0000","last_became_active":"2024-04-24T08:48:28.972666+0000","last_became_peered":"2024-04-24T08:48:28.972666+0000","last_unstale":"2024-04-24T08:49:41.964980+0000","last_undegraded":"2024-04-24T08:49:41.964980+0000","last_fullsized":"2024-04-24T08:49:41.964980+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:53:28.358291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.860099+0000","last_change":"2024-04-24T08:48:29.983498+0000","last_active":"2024-04-24T08:50:10.860099+0000","last_peered":"2024-04-24T08:50:10.860099+0000","last_clean":"2024-04-24T08:50:10.860099+0000","last_became_active":"2024-04-24T08:48:29.983167+0000","last_became_peered":"2024-04-24T08:48:29.983167+0000","last_unstale":"2024-04-24T08:50:10.860099+0000","last_undegraded":"2024-04-24T08:50:10.860099+0000","last_fullsized":"2024-04-24T08:50:10.860099+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:09:56.648047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:49:42.129192+0000","last_change":"2024-04-24T08:49:42.129192+0000","last_active":"2024-04-24T08:49:42.129192+0000","last_peered":"2024-04-24T08:49:42.129192+0000","last_clean":"2024-04-24T08:49:42.129192+0000","last_became_active":"2024-04-24T08:49:42.124814+0000","last_became_peered":"2024-04-24T08:49:42.124814+0000","last_unstale":"2024-04-24T08:49:42.129192+0000","last_undegraded":"2024-04-24T08:49:42.129192+0000","last_fullsized":"2024-04-24T08:49:42.129192+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:52:46.273836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6,7],"acting":[1,8,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'5","reported_seq":121,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:30.490282+0000","last_change":"2024-04-24T08:48:28.974528+0000","last_active":"2024-04-24T08:50:30.490282+0000","last_peered":"2024-04-24T08:50:30.490282+0000","last_clean":"2024-04-24T08:50:30.490282+0000","last_became_active":"2024-04-24T08:48:28.974203+0000","last_became_peered":"2024-04-24T08:48:28.974203+0000","last_unstale":"2024-04-24T08:50:30.490282+0000","last_undegraded":"2024-04-24T08:50:30.490282+0000","last_fullsized":"2024-04-24T08:50:30.490282+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:26:15.701204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":90,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.862198+0000","last_change":"2024-04-24T08:48:29.984356+0000","last_active":"2024-04-24T08:50:10.862198+0000","last_peered":"2024-04-24T08:50:10.862198+0000","last_clean":"2024-04-24T08:50:10.862198+0000","last_became_active":"2024-04-24T08:48:29.983949+0000","last_became_peered":"2024-04-24T08:48:29.983949+0000","last_unstale":"2024-04-24T08:50:10.862198+0000","last_undegraded":"2024-04-24T08:50:10.862198+0000","last_fullsized":"2024-04-24T08:50:10.862198+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:51:59.409835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:49:42.126598+0000","last_change":"2024-04-24T08:48:34.312418+0000","last_active":"2024-04-24T08:49:42.126598+0000","last_peered":"2024-04-24T08:49:42.126598+0000","last_clean":"2024-04-24T08:49:42.126598+0000","last_became_active":"2024-04-24T08:48:34.312035+0000","last_became_peered":"2024-04-24T08:48:34.312035+0000","last_unstale":"2024-04-24T08:49:42.126598+0000","last_undegraded":"2024-04-24T08:49:42.126598+0000","last_fullsized":"2024-04-24T08:49:42.126598+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:27:33.461732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"89'3","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:49:42.125750+0000","last_change":"2024-04-24T08:48:28.976070+0000","last_active":"2024-04-24T08:49:42.125750+0000","last_peered":"2024-04-24T08:49:42.125750+0000","last_clean":"2024-04-24T08:49:42.125750+0000","last_became_active":"2024-04-24T08:48:28.975703+0000","last_became_peered":"2024-04-24T08:48:28.975703+0000","last_unstale":"2024-04-24T08:49:42.125750+0000","last_undegraded":"2024-04-24T08:49:42.125750+0000","last_fullsized":"2024-04-24T08:49:42.125750+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:46:13.227529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'6","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.843654+0000","last_change":"2024-04-24T08:49:42.128814+0000","last_active":"2024-04-24T08:50:10.843654+0000","last_peered":"2024-04-24T08:50:10.843654+0000","last_clean":"2024-04-24T08:50:10.843654+0000","last_became_active":"2024-04-24T08:49:42.127422+0000","last_became_peered":"2024-04-24T08:49:42.127422+0000","last_unstale":"2024-04-24T08:50:10.843654+0000","last_undegraded":"2024-04-24T08:50:10.843654+0000","last_fullsized":"2024-04-24T08:50:10.843654+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:58:25.181798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:49:42.123553+0000","last_change":"2024-04-24T08:48:34.320191+0000","last_active":"2024-04-24T08:49:42.123553+0000","last_peered":"2024-04-24T08:49:42.123553+0000","last_clean":"2024-04-24T08:49:42.123553+0000","last_became_active":"2024-04-24T08:48:34.315415+0000","last_became_peered":"2024-04-24T08:48:34.315415+0000","last_unstale":"2024-04-24T08:49:42.123553+0000","last_undegraded":"2024-04-24T08:49:42.123553+0000","last_fullsized":"2024-04-24T08:49:42.123553+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:55:11.372789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:49:42.122289+0000","last_change":"2024-04-24T08:48:28.968550+0000","last_active":"2024-04-24T08:49:42.122289+0000","last_peered":"2024-04-24T08:49:42.122289+0000","last_clean":"2024-04-24T08:49:42.122289+0000","last_became_active":"2024-04-24T08:48:28.968120+0000","last_became_peered":"2024-04-24T08:48:28.968120+0000","last_unstale":"2024-04-24T08:49:42.122289+0000","last_undegraded":"2024-04-24T08:49:42.122289+0000","last_fullsized":"2024-04-24T08:49:42.122289+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:46:00.870285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.850343+0000","last_change":"2024-04-24T08:48:29.984755+0000","last_active":"2024-04-24T08:50:10.850343+0000","last_peered":"2024-04-24T08:50:10.850343+0000","last_clean":"2024-04-24T08:50:10.850343+0000","last_became_active":"2024-04-24T08:48:29.982093+0000","last_became_peered":"2024-04-24T08:48:29.982093+0000","last_unstale":"2024-04-24T08:50:10.850343+0000","last_undegraded":"2024-04-24T08:50:10.850343+0000","last_fullsized":"2024-04-24T08:50:10.850343+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:25:22.620152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.838322+0000","last_change":"2024-04-24T08:48:29.983101+0000","last_active":"2024-04-24T08:50:10.838322+0000","last_peered":"2024-04-24T08:50:10.838322+0000","last_clean":"2024-04-24T08:50:10.838322+0000","last_became_active":"2024-04-24T08:48:29.981871+0000","last_became_peered":"2024-04-24T08:48:29.981871+0000","last_unstale":"2024-04-24T08:50:10.838322+0000","last_undegraded":"2024-04-24T08:50:10.838322+0000","last_fullsized":"2024-04-24T08:50:10.838322+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:02:44.582111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.833294+0000","last_change":"2024-04-24T08:49:42.128974+0000","last_active":"2024-04-24T08:50:10.833294+0000","last_peered":"2024-04-24T08:50:10.833294+0000","last_clean":"2024-04-24T08:50:10.833294+0000","last_became_active":"2024-04-24T08:49:42.127710+0000","last_became_peered":"2024-04-24T08:49:42.127710+0000","last_unstale":"2024-04-24T08:50:10.833294+0000","last_undegraded":"2024-04-24T08:50:10.833294+0000","last_fullsized":"2024-04-24T08:50:10.833294+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:09:20.279852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.856373+0000","last_change":"2024-04-24T08:48:29.988241+0000","last_active":"2024-04-24T08:50:10.856373+0000","last_peered":"2024-04-24T08:50:10.856373+0000","last_clean":"2024-04-24T08:50:10.856373+0000","last_became_active":"2024-04-24T08:48:29.987930+0000","last_became_peered":"2024-04-24T08:48:29.987930+0000","last_unstale":"2024-04-24T08:50:10.856373+0000","last_undegraded":"2024-04-24T08:50:10.856373+0000","last_fullsized":"2024-04-24T08:50:10.856373+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:42:24.687624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.864733+0000","last_change":"2024-04-24T08:48:29.992441+0000","last_active":"2024-04-24T08:50:10.864733+0000","last_peered":"2024-04-24T08:50:10.864733+0000","last_clean":"2024-04-24T08:50:10.864733+0000","last_became_active":"2024-04-24T08:48:29.988788+0000","last_became_peered":"2024-04-24T08:48:29.988788+0000","last_unstale":"2024-04-24T08:50:10.864733+0000","last_undegraded":"2024-04-24T08:50:10.864733+0000","last_fullsized":"2024-04-24T08:50:10.864733+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:08:31.549509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.863506+0000","last_change":"2024-04-24T08:48:29.982116+0000","last_active":"2024-04-24T08:50:10.863506+0000","last_peered":"2024-04-24T08:50:10.863506+0000","last_clean":"2024-04-24T08:50:10.863506+0000","last_became_active":"2024-04-24T08:48:29.981636+0000","last_became_peered":"2024-04-24T08:48:29.981636+0000","last_unstale":"2024-04-24T08:50:10.863506+0000","last_undegraded":"2024-04-24T08:50:10.863506+0000","last_fullsized":"2024-04-24T08:50:10.863506+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:11:01.943425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.831656+0000","last_change":"2024-04-24T08:48:29.982244+0000","last_active":"2024-04-24T08:50:10.831656+0000","last_peered":"2024-04-24T08:50:10.831656+0000","last_clean":"2024-04-24T08:50:10.831656+0000","last_became_active":"2024-04-24T08:48:29.981622+0000","last_became_peered":"2024-04-24T08:48:29.981622+0000","last_unstale":"2024-04-24T08:50:10.831656+0000","last_undegraded":"2024-04-24T08:50:10.831656+0000","last_fullsized":"2024-04-24T08:50:10.831656+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:02:57.723949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.859981+0000","last_change":"2024-04-24T08:48:29.985867+0000","last_active":"2024-04-24T08:50:10.859981+0000","last_peered":"2024-04-24T08:50:10.859981+0000","last_clean":"2024-04-24T08:50:10.859981+0000","last_became_active":"2024-04-24T08:48:29.985430+0000","last_became_peered":"2024-04-24T08:48:29.985430+0000","last_unstale":"2024-04-24T08:50:10.859981+0000","last_undegraded":"2024-04-24T08:50:10.859981+0000","last_fullsized":"2024-04-24T08:50:10.859981+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:09:40.467623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.848111+0000","last_change":"2024-04-24T08:48:29.996474+0000","last_active":"2024-04-24T08:50:10.848111+0000","last_peered":"2024-04-24T08:50:10.848111+0000","last_clean":"2024-04-24T08:50:10.848111+0000","last_became_active":"2024-04-24T08:48:29.995958+0000","last_became_peered":"2024-04-24T08:48:29.995958+0000","last_unstale":"2024-04-24T08:50:10.848111+0000","last_undegraded":"2024-04-24T08:50:10.848111+0000","last_fullsized":"2024-04-24T08:50:10.848111+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:50:19.865125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.856229+0000","last_change":"2024-04-24T08:48:29.978285+0000","last_active":"2024-04-24T08:50:10.856229+0000","last_peered":"2024-04-24T08:50:10.856229+0000","last_clean":"2024-04-24T08:50:10.856229+0000","last_became_active":"2024-04-24T08:48:29.976445+0000","last_became_peered":"2024-04-24T08:48:29.976445+0000","last_unstale":"2024-04-24T08:50:10.856229+0000","last_undegraded":"2024-04-24T08:50:10.856229+0000","last_fullsized":"2024-04-24T08:50:10.856229+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:30:29.598733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.852463+0000","last_change":"2024-04-24T08:48:29.994442+0000","last_active":"2024-04-24T08:50:10.852463+0000","last_peered":"2024-04-24T08:50:10.852463+0000","last_clean":"2024-04-24T08:50:10.852463+0000","last_became_active":"2024-04-24T08:48:29.993896+0000","last_became_peered":"2024-04-24T08:48:29.993896+0000","last_unstale":"2024-04-24T08:50:10.852463+0000","last_undegraded":"2024-04-24T08:50:10.852463+0000","last_fullsized":"2024-04-24T08:50:10.852463+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:37:14.023511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.861403+0000","last_change":"2024-04-24T08:49:41.981252+0000","last_active":"2024-04-24T08:50:10.861403+0000","last_peered":"2024-04-24T08:50:10.861403+0000","last_clean":"2024-04-24T08:50:10.861403+0000","last_became_active":"2024-04-24T08:49:41.980495+0000","last_became_peered":"2024-04-24T08:49:41.980495+0000","last_unstale":"2024-04-24T08:50:10.861403+0000","last_undegraded":"2024-04-24T08:50:10.861403+0000","last_fullsized":"2024-04-24T08:50:10.861403+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:47:33.579204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,8],"acting":[1,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.859268+0000","last_change":"2024-04-24T08:48:29.986021+0000","last_active":"2024-04-24T08:50:10.859268+0000","last_peered":"2024-04-24T08:50:10.859268+0000","last_clean":"2024-04-24T08:50:10.859268+0000","last_became_active":"2024-04-24T08:48:29.985182+0000","last_became_peered":"2024-04-24T08:48:29.985182+0000","last_unstale":"2024-04-24T08:50:10.859268+0000","last_undegraded":"2024-04-24T08:50:10.859268+0000","last_fullsized":"2024-04-24T08:50:10.859268+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:10:57.143065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.833371+0000","last_change":"2024-04-24T08:48:29.979886+0000","last_active":"2024-04-24T08:50:10.833371+0000","last_peered":"2024-04-24T08:50:10.833371+0000","last_clean":"2024-04-24T08:50:10.833371+0000","last_became_active":"2024-04-24T08:48:29.978779+0000","last_became_peered":"2024-04-24T08:48:29.978779+0000","last_unstale":"2024-04-24T08:50:10.833371+0000","last_undegraded":"2024-04-24T08:50:10.833371+0000","last_fullsized":"2024-04-24T08:50:10.833371+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:51:51.303990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68354,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":894,"num_read_kb":515,"num_write":248,"num_write_kb":142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":205062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":215,"ondisk_log_size":215,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383721,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":501425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645360,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32860,"kb_used_data":1684,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712292,"statfs":{"total":95995035648,"available":95961387008,"internally_reserved":0,"allocated":1724416,"data_stored":1149641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808887,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32212,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712940,"statfs":{"total":95995035648,"available":95962050560,"internally_reserved":0,"allocated":1060864,"data_stored":505451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070526,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1052672,"data_stored":501733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332166,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32788,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712364,"statfs":{"total":95995035648,"available":95961460736,"internally_reserved":0,"allocated":1650688,"data_stored":1098371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495693,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1032192,"data_stored":496859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574961,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36408,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708744,"statfs":{"total":95995035648,"available":95957753856,"internally_reserved":0,"allocated":1097728,"data_stored":559289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738488,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36368,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708784,"statfs":{"total":95995035648,"available":95957794816,"internally_reserved":0,"allocated":1056768,"data_stored":498315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411433,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28108,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717044,"statfs":{"total":95995035648,"available":95966253056,"internally_reserved":0,"allocated":1118208,"data_stored":560041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182499,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1044480,"data_stored":496567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920860,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1040384,"data_stored":500933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757333,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32796,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712356,"statfs":{"total":95995035648,"available":95961452544,"internally_reserved":0,"allocated":1658880,"data_stored":1093953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":63152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":63904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":63136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T08:50:34.352 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid bd3e039e-0215-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-24T08:50:34.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:50:34 smithi138 ceph-mon[31441]: pgmap v437: 129 pgs: 129 active+clean; 644 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-24T08:50:34.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:50:34 smithi189 ceph-mon[31791]: pgmap v437: 129 pgs: 129 active+clean; 644 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-24T08:50:34.832 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T08:50:35.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:50:35 smithi083 ceph-mon[27907]: from='client.15159 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T08:50:35.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:50:35 smithi138 ceph-mon[31441]: from='client.15159 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T08:50:35.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:50:35 smithi189 ceph-mon[31791]: from='client.15159 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T08:50:36.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:50:36 smithi083 ceph-mon[27907]: pgmap v438: 129 pgs: 129 active+clean; 644 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-04-24T08:50:36.356 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:50:36 smithi138 ceph-mon[31441]: pgmap v438: 129 pgs: 129 active+clean; 644 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-04-24T08:50:36.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:50:36 smithi189 ceph-mon[31791]: pgmap v438: 129 pgs: 129 active+clean; 644 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-04-24T08:50:37.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:37.263+0000 7fd391e00700 1 -- 172.21.15.83:0/272309970 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd38c105940 msgr2=0x7fd38c109990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:37.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:37.263+0000 7fd391e00700 1 --2- 172.21.15.83:0/272309970 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd38c105940 0x7fd38c109990 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fd3780099c0 tx=0x7fd378009cd0 comp rx=0 tx=0).stop 2024-04-24T08:50:37.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:37.263+0000 7fd391e00700 1 -- 172.21.15.83:0/272309970 shutdown_connections 2024-04-24T08:50:37.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:37.263+0000 7fd391e00700 1 --2- 172.21.15.83:0/272309970 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd38c10a070 0x7fd38c111c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:37.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:37.263+0000 7fd391e00700 1 --2- 172.21.15.83:0/272309970 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd38c104f90 0x7fd38c105370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:37.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:37.263+0000 7fd391e00700 1 --2- 172.21.15.83:0/272309970 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd38c105940 0x7fd38c109990 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:37.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:37.263+0000 7fd391e00700 1 -- 172.21.15.83:0/272309970 >> 172.21.15.83:0/272309970 conn(0x7fd38c100820 msgr2=0x7fd38c102c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:50:37.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:37.263+0000 7fd391e00700 1 -- 172.21.15.83:0/272309970 shutdown_connections 2024-04-24T08:50:37.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:37.263+0000 7fd391e00700 1 -- 172.21.15.83:0/272309970 wait complete. 2024-04-24T08:50:37.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:37.265+0000 7fd391e00700 1 Processor -- start 2024-04-24T08:50:37.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:37.265+0000 7fd391e00700 1 -- start start 2024-04-24T08:50:37.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:37.265+0000 7fd391e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd38c104f90 0x7fd38c19b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:37.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:37.265+0000 7fd391e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd38c105940 0x7fd38c19b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:37.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:37.265+0000 7fd391e00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd38c10a070 0x7fd38c19fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:37.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:37.266+0000 7fd391e00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd38c114f50 con 0x7fd38c104f90 2024-04-24T08:50:37.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:37.266+0000 7fd391e00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd38c114dd0 con 0x7fd38c10a070 2024-04-24T08:50:37.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:37.266+0000 7fd391e00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd38c1150d0 con 0x7fd38c105940 2024-04-24T08:50:37.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:37.266+0000 7fd38aa00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd38c104f90 0x7fd38c19b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:37.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:37.266+0000 7fd38aa00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd38c104f90 0x7fd38c19b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48310/0 (socket says 172.21.15.83:48310) 2024-04-24T08:50:37.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:37.266+0000 7fd38aa00700 1 -- 172.21.15.83:0/2378465258 learned_addr learned my addr 172.21.15.83:0/2378465258 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:50:37.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:37.266+0000 7fd38b400700 1 --2- 172.21.15.83:0/2378465258 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd38c10a070 0x7fd38c19fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:37.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:37.266+0000 7fd38a000700 1 --2- 172.21.15.83:0/2378465258 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd38c105940 0x7fd38c19b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:37.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:37.266+0000 7fd38aa00700 1 -- 172.21.15.83:0/2378465258 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd38c105940 msgr2=0x7fd38c19b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:37.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:37.266+0000 7fd38aa00700 1 --2- 172.21.15.83:0/2378465258 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd38c105940 0x7fd38c19b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:37.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:37.266+0000 7fd38aa00700 1 -- 172.21.15.83:0/2378465258 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd38c10a070 msgr2=0x7fd38c19fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:37.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:37.266+0000 7fd38aa00700 1 --2- 172.21.15.83:0/2378465258 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd38c10a070 0x7fd38c19fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:37.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:37.266+0000 7fd38aa00700 1 -- 172.21.15.83:0/2378465258 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd38c1a0470 con 0x7fd38c104f90 2024-04-24T08:50:37.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:37.266+0000 7fd38a000700 1 --2- 172.21.15.83:0/2378465258 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd38c105940 0x7fd38c19b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:50:37.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:37.267+0000 7fd38aa00700 1 --2- 172.21.15.83:0/2378465258 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd38c104f90 0x7fd38c19b460 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fd37c00d980 tx=0x7fd37c00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:50:37.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:37.267+0000 7fd390a00700 1 -- 172.21.15.83:0/2378465258 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd37c017070 con 0x7fd38c104f90 2024-04-24T08:50:37.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:37.267+0000 7fd390a00700 1 -- 172.21.15.83:0/2378465258 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd37c004830 con 0x7fd38c104f90 2024-04-24T08:50:37.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:37.267+0000 7fd391e00700 1 -- 172.21.15.83:0/2378465258 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd38c1a0760 con 0x7fd38c104f90 2024-04-24T08:50:37.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:37.267+0000 7fd390a00700 1 -- 172.21.15.83:0/2378465258 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd37c005440 con 0x7fd38c104f90 2024-04-24T08:50:37.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:37.267+0000 7fd391e00700 1 -- 172.21.15.83:0/2378465258 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd38c1a7fe0 con 0x7fd38c104f90 2024-04-24T08:50:37.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:37.268+0000 7fd391e00700 1 -- 172.21.15.83:0/2378465258 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd38c0697e0 con 0x7fd38c104f90 2024-04-24T08:50:37.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:37.270+0000 7fd390a00700 1 -- 172.21.15.83:0/2378465258 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fd37c0055e0 con 0x7fd38c104f90 2024-04-24T08:50:37.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:37.271+0000 7fd390a00700 1 --2- 172.21.15.83:0/2378465258 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fd3700a3cc0 0x7fd3700a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:37.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:37.272+0000 7fd390a00700 1 -- 172.21.15.83:0/2378465258 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fd37c0d06a0 con 0x7fd38c104f90 2024-04-24T08:50:37.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:37.272+0000 7fd38a000700 1 --2- 172.21.15.83:0/2378465258 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fd3700a3cc0 0x7fd3700a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:37.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:37.273+0000 7fd38a000700 1 --2- 172.21.15.83:0/2378465258 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fd3700a3cc0 0x7fd3700a6180 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd378006010 tx=0x7fd378005e80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:50:37.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:37.276+0000 7fd390a00700 1 -- 172.21.15.83:0/2378465258 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd37c020020 con 0x7fd38c104f90 2024-04-24T08:50:37.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:37.498+0000 7fd391e00700 1 -- 172.21.15.83:0/2378465258 --> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd38c062f30 con 0x7fd3700a3cc0 2024-04-24T08:50:37.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:37.509+0000 7fd390a00700 1 -- 172.21.15.83:0/2378465258 <== mgr.14152 v2:172.21.15.83:6800/2550826560 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340513 (secure 0 0 0) 0x7fd38c062f30 con 0x7fd3700a3cc0 2024-04-24T08:50:37.511 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:50:37.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:37.517+0000 7fd391e00700 1 -- 172.21.15.83:0/2378465258 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fd3700a3cc0 msgr2=0x7fd3700a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:37.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:37.517+0000 7fd391e00700 1 --2- 172.21.15.83:0/2378465258 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fd3700a3cc0 0x7fd3700a6180 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd378006010 tx=0x7fd378005e80 comp rx=0 tx=0).stop 2024-04-24T08:50:37.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:37.517+0000 7fd391e00700 1 -- 172.21.15.83:0/2378465258 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd38c104f90 msgr2=0x7fd38c19b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:37.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:37.517+0000 7fd391e00700 1 --2- 172.21.15.83:0/2378465258 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd38c104f90 0x7fd38c19b460 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fd37c00d980 tx=0x7fd37c00dd40 comp rx=0 tx=0).stop 2024-04-24T08:50:37.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:37.518+0000 7fd391e00700 1 -- 172.21.15.83:0/2378465258 shutdown_connections 2024-04-24T08:50:37.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:37.518+0000 7fd391e00700 1 --2- 172.21.15.83:0/2378465258 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fd3700a3cc0 0x7fd3700a6180 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:37.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:37.518+0000 7fd391e00700 1 --2- 172.21.15.83:0/2378465258 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd38c104f90 0x7fd38c19b460 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:37.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:37.518+0000 7fd391e00700 1 --2- 172.21.15.83:0/2378465258 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd38c10a070 0x7fd38c19fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:37.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:37.518+0000 7fd391e00700 1 --2- 172.21.15.83:0/2378465258 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd38c105940 0x7fd38c19b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:37.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:37.518+0000 7fd391e00700 1 -- 172.21.15.83:0/2378465258 >> 172.21.15.83:0/2378465258 conn(0x7fd38c100820 msgr2=0x7fd38c102af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:50:37.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:37.518+0000 7fd391e00700 1 -- 172.21.15.83:0/2378465258 shutdown_connections 2024-04-24T08:50:37.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:37.518+0000 7fd391e00700 1 -- 172.21.15.83:0/2378465258 wait complete. 2024-04-24T08:50:37.520 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-04-24T08:50:38.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:50:38 smithi083 ceph-mon[27907]: pgmap v439: 129 pgs: 129 active+clean; 644 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-24T08:50:38.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:50:38 smithi138 ceph-mon[31441]: pgmap v439: 129 pgs: 129 active+clean; 644 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-24T08:50:38.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:50:38 smithi189 ceph-mon[31791]: pgmap v439: 129 pgs: 129 active+clean; 644 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-24T08:50:38.497 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":439,"stamp":"2024-04-24T08:50:36.894207+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658956,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1075,"num_read_kb":659,"num_write":431,"num_write_kb":3082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":854,"ondisk_log_size":854,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":384180,"kb_used_data":14228,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124557644,"statfs":{"total":1151940427776,"available":1151547027456,"internally_reserved":0,"allocated":14569472,"data_stored":7962578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":378713401},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":115,"num_read_kb":50,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006235"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.864789+0000","last_change":"2024-04-24T08:48:29.993009+0000","last_active":"2024-04-24T08:50:10.864789+0000","last_peered":"2024-04-24T08:50:10.864789+0000","last_clean":"2024-04-24T08:50:10.864789+0000","last_became_active":"2024-04-24T08:48:29.990613+0000","last_became_peered":"2024-04-24T08:48:29.990613+0000","last_unstale":"2024-04-24T08:50:10.864789+0000","last_undegraded":"2024-04-24T08:50:10.864789+0000","last_fullsized":"2024-04-24T08:50:10.864789+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:13:43.871742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.857859+0000","last_change":"2024-04-24T08:48:29.986957+0000","last_active":"2024-04-24T08:50:10.857859+0000","last_peered":"2024-04-24T08:50:10.857859+0000","last_clean":"2024-04-24T08:50:10.857859+0000","last_became_active":"2024-04-24T08:48:29.984691+0000","last_became_peered":"2024-04-24T08:48:29.984691+0000","last_unstale":"2024-04-24T08:50:10.857859+0000","last_undegraded":"2024-04-24T08:50:10.857859+0000","last_fullsized":"2024-04-24T08:50:10.857859+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:07:38.031213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.862715+0000","last_change":"2024-04-24T08:48:29.991478+0000","last_active":"2024-04-24T08:50:10.862715+0000","last_peered":"2024-04-24T08:50:10.862715+0000","last_clean":"2024-04-24T08:50:10.862715+0000","last_became_active":"2024-04-24T08:48:29.991013+0000","last_became_peered":"2024-04-24T08:48:29.991013+0000","last_unstale":"2024-04-24T08:50:10.862715+0000","last_undegraded":"2024-04-24T08:50:10.862715+0000","last_fullsized":"2024-04-24T08:50:10.862715+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:10:39.381353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.837896+0000","last_change":"2024-04-24T08:48:29.983664+0000","last_active":"2024-04-24T08:50:10.837896+0000","last_peered":"2024-04-24T08:50:10.837896+0000","last_clean":"2024-04-24T08:50:10.837896+0000","last_became_active":"2024-04-24T08:48:29.978835+0000","last_became_peered":"2024-04-24T08:48:29.978835+0000","last_unstale":"2024-04-24T08:50:10.837896+0000","last_undegraded":"2024-04-24T08:50:10.837896+0000","last_fullsized":"2024-04-24T08:50:10.837896+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:29:20.501652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.841727+0000","last_change":"2024-04-24T08:48:29.989886+0000","last_active":"2024-04-24T08:50:10.841727+0000","last_peered":"2024-04-24T08:50:10.841727+0000","last_clean":"2024-04-24T08:50:10.841727+0000","last_became_active":"2024-04-24T08:48:29.989574+0000","last_became_peered":"2024-04-24T08:48:29.989574+0000","last_unstale":"2024-04-24T08:50:10.841727+0000","last_undegraded":"2024-04-24T08:50:10.841727+0000","last_fullsized":"2024-04-24T08:50:10.841727+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:14:01.085543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.830854+0000","last_change":"2024-04-24T08:48:29.978045+0000","last_active":"2024-04-24T08:50:10.830854+0000","last_peered":"2024-04-24T08:50:10.830854+0000","last_clean":"2024-04-24T08:50:10.830854+0000","last_became_active":"2024-04-24T08:48:29.977729+0000","last_became_peered":"2024-04-24T08:48:29.977729+0000","last_unstale":"2024-04-24T08:50:10.830854+0000","last_undegraded":"2024-04-24T08:50:10.830854+0000","last_fullsized":"2024-04-24T08:50:10.830854+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:03:33.382473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.846631+0000","last_change":"2024-04-24T08:48:29.976355+0000","last_active":"2024-04-24T08:50:10.846631+0000","last_peered":"2024-04-24T08:50:10.846631+0000","last_clean":"2024-04-24T08:50:10.846631+0000","last_became_active":"2024-04-24T08:48:29.975849+0000","last_became_peered":"2024-04-24T08:48:29.975849+0000","last_unstale":"2024-04-24T08:50:10.846631+0000","last_undegraded":"2024-04-24T08:50:10.846631+0000","last_fullsized":"2024-04-24T08:50:10.846631+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:13:51.302995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.861785+0000","last_change":"2024-04-24T08:48:29.983149+0000","last_active":"2024-04-24T08:50:10.861785+0000","last_peered":"2024-04-24T08:50:10.861785+0000","last_clean":"2024-04-24T08:50:10.861785+0000","last_became_active":"2024-04-24T08:48:29.982692+0000","last_became_peered":"2024-04-24T08:48:29.982692+0000","last_unstale":"2024-04-24T08:50:10.861785+0000","last_undegraded":"2024-04-24T08:50:10.861785+0000","last_fullsized":"2024-04-24T08:50:10.861785+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:59:06.151441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.859122+0000","last_change":"2024-04-24T08:48:29.985531+0000","last_active":"2024-04-24T08:50:10.859122+0000","last_peered":"2024-04-24T08:50:10.859122+0000","last_clean":"2024-04-24T08:50:10.859122+0000","last_became_active":"2024-04-24T08:48:29.982484+0000","last_became_peered":"2024-04-24T08:48:29.982484+0000","last_unstale":"2024-04-24T08:50:10.859122+0000","last_undegraded":"2024-04-24T08:50:10.859122+0000","last_fullsized":"2024-04-24T08:50:10.859122+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:03:50.114096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.842142+0000","last_change":"2024-04-24T08:48:29.982016+0000","last_active":"2024-04-24T08:50:10.842142+0000","last_peered":"2024-04-24T08:50:10.842142+0000","last_clean":"2024-04-24T08:50:10.842142+0000","last_became_active":"2024-04-24T08:48:29.977639+0000","last_became_peered":"2024-04-24T08:48:29.977639+0000","last_unstale":"2024-04-24T08:50:10.842142+0000","last_undegraded":"2024-04-24T08:50:10.842142+0000","last_fullsized":"2024-04-24T08:50:10.842142+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:35:48.676822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.862192+0000","last_change":"2024-04-24T08:48:29.977982+0000","last_active":"2024-04-24T08:50:10.862192+0000","last_peered":"2024-04-24T08:50:10.862192+0000","last_clean":"2024-04-24T08:50:10.862192+0000","last_became_active":"2024-04-24T08:48:29.977664+0000","last_became_peered":"2024-04-24T08:48:29.977664+0000","last_unstale":"2024-04-24T08:50:10.862192+0000","last_undegraded":"2024-04-24T08:50:10.862192+0000","last_fullsized":"2024-04-24T08:50:10.862192+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:45:17.597291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.832716+0000","last_change":"2024-04-24T08:48:29.981615+0000","last_active":"2024-04-24T08:50:10.832716+0000","last_peered":"2024-04-24T08:50:10.832716+0000","last_clean":"2024-04-24T08:50:10.832716+0000","last_became_active":"2024-04-24T08:48:29.979292+0000","last_became_peered":"2024-04-24T08:48:29.979292+0000","last_unstale":"2024-04-24T08:50:10.832716+0000","last_undegraded":"2024-04-24T08:50:10.832716+0000","last_fullsized":"2024-04-24T08:50:10.832716+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:48:44.195626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.839868+0000","last_change":"2024-04-24T08:48:29.952632+0000","last_active":"2024-04-24T08:50:10.839868+0000","last_peered":"2024-04-24T08:50:10.839868+0000","last_clean":"2024-04-24T08:50:10.839868+0000","last_became_active":"2024-04-24T08:48:29.951862+0000","last_became_peered":"2024-04-24T08:48:29.951862+0000","last_unstale":"2024-04-24T08:50:10.839868+0000","last_undegraded":"2024-04-24T08:50:10.839868+0000","last_fullsized":"2024-04-24T08:50:10.839868+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:23:40.038517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.841463+0000","last_change":"2024-04-24T08:48:29.984727+0000","last_active":"2024-04-24T08:50:10.841463+0000","last_peered":"2024-04-24T08:50:10.841463+0000","last_clean":"2024-04-24T08:50:10.841463+0000","last_became_active":"2024-04-24T08:48:29.983530+0000","last_became_peered":"2024-04-24T08:48:29.983530+0000","last_unstale":"2024-04-24T08:50:10.841463+0000","last_undegraded":"2024-04-24T08:50:10.841463+0000","last_fullsized":"2024-04-24T08:50:10.841463+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:00:12.729972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.860004+0000","last_change":"2024-04-24T08:48:29.981023+0000","last_active":"2024-04-24T08:50:10.860004+0000","last_peered":"2024-04-24T08:50:10.860004+0000","last_clean":"2024-04-24T08:50:10.860004+0000","last_became_active":"2024-04-24T08:48:29.979986+0000","last_became_peered":"2024-04-24T08:48:29.979986+0000","last_unstale":"2024-04-24T08:50:10.860004+0000","last_undegraded":"2024-04-24T08:50:10.860004+0000","last_fullsized":"2024-04-24T08:50:10.860004+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:05:36.163209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.862412+0000","last_change":"2024-04-24T08:48:29.989036+0000","last_active":"2024-04-24T08:50:10.862412+0000","last_peered":"2024-04-24T08:50:10.862412+0000","last_clean":"2024-04-24T08:50:10.862412+0000","last_became_active":"2024-04-24T08:48:29.988676+0000","last_became_peered":"2024-04-24T08:48:29.988676+0000","last_unstale":"2024-04-24T08:50:10.862412+0000","last_undegraded":"2024-04-24T08:50:10.862412+0000","last_fullsized":"2024-04-24T08:50:10.862412+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:03:38.544596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.864828+0000","last_change":"2024-04-24T08:48:29.986424+0000","last_active":"2024-04-24T08:50:10.864828+0000","last_peered":"2024-04-24T08:50:10.864828+0000","last_clean":"2024-04-24T08:50:10.864828+0000","last_became_active":"2024-04-24T08:48:29.985973+0000","last_became_peered":"2024-04-24T08:48:29.985973+0000","last_unstale":"2024-04-24T08:50:10.864828+0000","last_undegraded":"2024-04-24T08:50:10.864828+0000","last_fullsized":"2024-04-24T08:50:10.864828+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:48:31.533282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.851059+0000","last_change":"2024-04-24T08:48:29.975470+0000","last_active":"2024-04-24T08:50:10.851059+0000","last_peered":"2024-04-24T08:50:10.851059+0000","last_clean":"2024-04-24T08:50:10.851059+0000","last_became_active":"2024-04-24T08:48:29.974777+0000","last_became_peered":"2024-04-24T08:48:29.974777+0000","last_unstale":"2024-04-24T08:50:10.851059+0000","last_undegraded":"2024-04-24T08:50:10.851059+0000","last_fullsized":"2024-04-24T08:50:10.851059+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:04:26.423840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.853453+0000","last_change":"2024-04-24T08:48:29.988001+0000","last_active":"2024-04-24T08:50:10.853453+0000","last_peered":"2024-04-24T08:50:10.853453+0000","last_clean":"2024-04-24T08:50:10.853453+0000","last_became_active":"2024-04-24T08:48:29.987669+0000","last_became_peered":"2024-04-24T08:48:29.987669+0000","last_unstale":"2024-04-24T08:50:10.853453+0000","last_undegraded":"2024-04-24T08:50:10.853453+0000","last_fullsized":"2024-04-24T08:50:10.853453+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:29:24.457369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:49:41.967679+0000","last_change":"2024-04-24T08:48:34.308732+0000","last_active":"2024-04-24T08:49:41.967679+0000","last_peered":"2024-04-24T08:49:41.967679+0000","last_clean":"2024-04-24T08:49:41.967679+0000","last_became_active":"2024-04-24T08:48:34.307598+0000","last_became_peered":"2024-04-24T08:48:34.307598+0000","last_unstale":"2024-04-24T08:49:41.967679+0000","last_undegraded":"2024-04-24T08:49:41.967679+0000","last_fullsized":"2024-04-24T08:49:41.967679+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:28:57.260836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'13","reported_seq":129,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:31.480157+0000","last_change":"2024-04-24T08:48:28.970230+0000","last_active":"2024-04-24T08:50:31.480157+0000","last_peered":"2024-04-24T08:50:31.480157+0000","last_clean":"2024-04-24T08:50:31.480157+0000","last_became_active":"2024-04-24T08:48:28.969878+0000","last_became_peered":"2024-04-24T08:48:28.969878+0000","last_unstale":"2024-04-24T08:50:31.480157+0000","last_undegraded":"2024-04-24T08:50:31.480157+0000","last_fullsized":"2024-04-24T08:50:31.480157+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:05:19.206557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":90,"num_read_kb":90,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.859573+0000","last_change":"2024-04-24T08:48:29.986350+0000","last_active":"2024-04-24T08:50:10.859573+0000","last_peered":"2024-04-24T08:50:10.859573+0000","last_clean":"2024-04-24T08:50:10.859573+0000","last_became_active":"2024-04-24T08:48:29.985167+0000","last_became_peered":"2024-04-24T08:48:29.985167+0000","last_unstale":"2024-04-24T08:50:10.859573+0000","last_undegraded":"2024-04-24T08:50:10.859573+0000","last_fullsized":"2024-04-24T08:50:10.859573+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:43:33.326808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:49:41.959522+0000","last_change":"2024-04-24T08:48:34.315159+0000","last_active":"2024-04-24T08:49:41.959522+0000","last_peered":"2024-04-24T08:49:41.959522+0000","last_clean":"2024-04-24T08:49:41.959522+0000","last_became_active":"2024-04-24T08:48:34.312119+0000","last_became_peered":"2024-04-24T08:48:34.312119+0000","last_unstale":"2024-04-24T08:49:41.959522+0000","last_undegraded":"2024-04-24T08:49:41.959522+0000","last_fullsized":"2024-04-24T08:49:41.959522+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:56:08.024123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:49:42.105245+0000","last_change":"2024-04-24T08:48:28.971787+0000","last_active":"2024-04-24T08:49:42.105245+0000","last_peered":"2024-04-24T08:49:42.105245+0000","last_clean":"2024-04-24T08:49:42.105245+0000","last_became_active":"2024-04-24T08:48:28.971479+0000","last_became_peered":"2024-04-24T08:48:28.971479+0000","last_unstale":"2024-04-24T08:49:42.105245+0000","last_undegraded":"2024-04-24T08:49:42.105245+0000","last_fullsized":"2024-04-24T08:49:42.105245+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:09:52.498833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.865106+0000","last_change":"2024-04-24T08:48:29.989737+0000","last_active":"2024-04-24T08:50:10.865106+0000","last_peered":"2024-04-24T08:50:10.865106+0000","last_clean":"2024-04-24T08:50:10.865106+0000","last_became_active":"2024-04-24T08:48:29.989341+0000","last_became_peered":"2024-04-24T08:48:29.989341+0000","last_unstale":"2024-04-24T08:50:10.865106+0000","last_undegraded":"2024-04-24T08:50:10.865106+0000","last_fullsized":"2024-04-24T08:50:10.865106+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:02:32.714848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:49:41.960130+0000","last_change":"2024-04-24T08:48:34.322666+0000","last_active":"2024-04-24T08:49:41.960130+0000","last_peered":"2024-04-24T08:49:41.960130+0000","last_clean":"2024-04-24T08:49:41.960130+0000","last_became_active":"2024-04-24T08:48:34.321546+0000","last_became_peered":"2024-04-24T08:48:34.321546+0000","last_unstale":"2024-04-24T08:49:41.960130+0000","last_undegraded":"2024-04-24T08:49:41.960130+0000","last_fullsized":"2024-04-24T08:49:41.960130+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:42:45.066043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:49:41.966819+0000","last_change":"2024-04-24T08:48:28.978738+0000","last_active":"2024-04-24T08:49:41.966819+0000","last_peered":"2024-04-24T08:49:41.966819+0000","last_clean":"2024-04-24T08:49:41.966819+0000","last_became_active":"2024-04-24T08:48:28.978378+0000","last_became_peered":"2024-04-24T08:48:28.978378+0000","last_unstale":"2024-04-24T08:49:41.966819+0000","last_undegraded":"2024-04-24T08:49:41.966819+0000","last_fullsized":"2024-04-24T08:49:41.966819+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:46:22.668234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.862689+0000","last_change":"2024-04-24T08:48:29.974588+0000","last_active":"2024-04-24T08:50:10.862689+0000","last_peered":"2024-04-24T08:50:10.862689+0000","last_clean":"2024-04-24T08:50:10.862689+0000","last_became_active":"2024-04-24T08:48:29.974190+0000","last_became_peered":"2024-04-24T08:48:29.974190+0000","last_unstale":"2024-04-24T08:50:10.862689+0000","last_undegraded":"2024-04-24T08:50:10.862689+0000","last_fullsized":"2024-04-24T08:50:10.862689+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:07:12.948660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:49:41.960046+0000","last_change":"2024-04-24T08:48:34.321954+0000","last_active":"2024-04-24T08:49:41.960046+0000","last_peered":"2024-04-24T08:49:41.960046+0000","last_clean":"2024-04-24T08:49:41.960046+0000","last_became_active":"2024-04-24T08:48:34.321417+0000","last_became_peered":"2024-04-24T08:48:34.321417+0000","last_unstale":"2024-04-24T08:49:41.960046+0000","last_undegraded":"2024-04-24T08:49:41.960046+0000","last_fullsized":"2024-04-24T08:49:41.960046+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:39:07.205709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:49:41.967697+0000","last_change":"2024-04-24T08:48:28.974464+0000","last_active":"2024-04-24T08:49:41.967697+0000","last_peered":"2024-04-24T08:49:41.967697+0000","last_clean":"2024-04-24T08:49:41.967697+0000","last_became_active":"2024-04-24T08:48:28.974001+0000","last_became_peered":"2024-04-24T08:48:28.974001+0000","last_unstale":"2024-04-24T08:49:41.967697+0000","last_undegraded":"2024-04-24T08:49:41.967697+0000","last_fullsized":"2024-04-24T08:49:41.967697+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:38:55.193427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.860498+0000","last_change":"2024-04-24T08:48:29.981336+0000","last_active":"2024-04-24T08:50:10.860498+0000","last_peered":"2024-04-24T08:50:10.860498+0000","last_clean":"2024-04-24T08:50:10.860498+0000","last_became_active":"2024-04-24T08:48:29.980333+0000","last_became_peered":"2024-04-24T08:48:29.980333+0000","last_unstale":"2024-04-24T08:50:10.860498+0000","last_undegraded":"2024-04-24T08:50:10.860498+0000","last_fullsized":"2024-04-24T08:50:10.860498+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:46:03.640332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:49:41.964607+0000","last_change":"2024-04-24T08:48:34.323099+0000","last_active":"2024-04-24T08:49:41.964607+0000","last_peered":"2024-04-24T08:49:41.964607+0000","last_clean":"2024-04-24T08:49:41.964607+0000","last_became_active":"2024-04-24T08:48:34.322557+0000","last_became_peered":"2024-04-24T08:48:34.322557+0000","last_unstale":"2024-04-24T08:49:41.964607+0000","last_undegraded":"2024-04-24T08:49:41.964607+0000","last_fullsized":"2024-04-24T08:49:41.964607+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:59:59.887087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"91'7","reported_seq":130,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:35.511958+0000","last_change":"2024-04-24T08:48:28.978427+0000","last_active":"2024-04-24T08:50:35.511958+0000","last_peered":"2024-04-24T08:50:35.511958+0000","last_clean":"2024-04-24T08:50:35.511958+0000","last_became_active":"2024-04-24T08:48:28.977291+0000","last_became_peered":"2024-04-24T08:48:28.977291+0000","last_unstale":"2024-04-24T08:50:35.511958+0000","last_undegraded":"2024-04-24T08:50:35.511958+0000","last_fullsized":"2024-04-24T08:50:35.511958+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:03:03.085265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":3,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.828288+0000","last_change":"2024-04-24T08:48:29.978008+0000","last_active":"2024-04-24T08:50:10.828288+0000","last_peered":"2024-04-24T08:50:10.828288+0000","last_clean":"2024-04-24T08:50:10.828288+0000","last_became_active":"2024-04-24T08:48:29.977389+0000","last_became_peered":"2024-04-24T08:48:29.977389+0000","last_unstale":"2024-04-24T08:50:10.828288+0000","last_undegraded":"2024-04-24T08:50:10.828288+0000","last_fullsized":"2024-04-24T08:50:10.828288+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:35:28.035378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:49:41.968546+0000","last_change":"2024-04-24T08:48:34.321477+0000","last_active":"2024-04-24T08:49:41.968546+0000","last_peered":"2024-04-24T08:49:41.968546+0000","last_clean":"2024-04-24T08:49:41.968546+0000","last_became_active":"2024-04-24T08:48:34.320860+0000","last_became_peered":"2024-04-24T08:48:34.320860+0000","last_unstale":"2024-04-24T08:49:41.968546+0000","last_undegraded":"2024-04-24T08:49:41.968546+0000","last_fullsized":"2024-04-24T08:49:41.968546+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:26:07.862590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:49:41.971416+0000","last_change":"2024-04-24T08:48:28.978147+0000","last_active":"2024-04-24T08:49:41.971416+0000","last_peered":"2024-04-24T08:49:41.971416+0000","last_clean":"2024-04-24T08:49:41.971416+0000","last_became_active":"2024-04-24T08:48:28.977646+0000","last_became_peered":"2024-04-24T08:48:28.977646+0000","last_unstale":"2024-04-24T08:49:41.971416+0000","last_undegraded":"2024-04-24T08:49:41.971416+0000","last_fullsized":"2024-04-24T08:49:41.971416+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:31:35.557886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.856313+0000","last_change":"2024-04-24T08:48:29.990788+0000","last_active":"2024-04-24T08:50:10.856313+0000","last_peered":"2024-04-24T08:50:10.856313+0000","last_clean":"2024-04-24T08:50:10.856313+0000","last_became_active":"2024-04-24T08:48:29.988476+0000","last_became_peered":"2024-04-24T08:48:29.988476+0000","last_unstale":"2024-04-24T08:50:10.856313+0000","last_undegraded":"2024-04-24T08:50:10.856313+0000","last_fullsized":"2024-04-24T08:50:10.856313+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:08:48.389877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:49:41.973582+0000","last_change":"2024-04-24T08:48:34.311020+0000","last_active":"2024-04-24T08:49:41.973582+0000","last_peered":"2024-04-24T08:49:41.973582+0000","last_clean":"2024-04-24T08:49:41.973582+0000","last_became_active":"2024-04-24T08:48:34.310395+0000","last_became_peered":"2024-04-24T08:48:34.310395+0000","last_unstale":"2024-04-24T08:49:41.973582+0000","last_undegraded":"2024-04-24T08:49:41.973582+0000","last_fullsized":"2024-04-24T08:49:41.973582+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:55:38.704954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:49:42.124068+0000","last_change":"2024-04-24T08:48:28.969991+0000","last_active":"2024-04-24T08:49:42.124068+0000","last_peered":"2024-04-24T08:49:42.124068+0000","last_clean":"2024-04-24T08:49:42.124068+0000","last_became_active":"2024-04-24T08:48:28.967937+0000","last_became_peered":"2024-04-24T08:48:28.967937+0000","last_unstale":"2024-04-24T08:49:42.124068+0000","last_undegraded":"2024-04-24T08:49:42.124068+0000","last_fullsized":"2024-04-24T08:49:42.124068+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:09:49.736108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.854053+0000","last_change":"2024-04-24T08:48:29.992814+0000","last_active":"2024-04-24T08:50:10.854053+0000","last_peered":"2024-04-24T08:50:10.854053+0000","last_clean":"2024-04-24T08:50:10.854053+0000","last_became_active":"2024-04-24T08:48:29.992188+0000","last_became_peered":"2024-04-24T08:48:29.992188+0000","last_unstale":"2024-04-24T08:50:10.854053+0000","last_undegraded":"2024-04-24T08:50:10.854053+0000","last_fullsized":"2024-04-24T08:50:10.854053+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:29:33.974958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:49:42.129184+0000","last_change":"2024-04-24T08:48:34.311196+0000","last_active":"2024-04-24T08:49:42.129184+0000","last_peered":"2024-04-24T08:49:42.129184+0000","last_clean":"2024-04-24T08:49:42.129184+0000","last_became_active":"2024-04-24T08:48:34.310723+0000","last_became_peered":"2024-04-24T08:48:34.310723+0000","last_unstale":"2024-04-24T08:49:42.129184+0000","last_undegraded":"2024-04-24T08:49:42.129184+0000","last_fullsized":"2024-04-24T08:49:42.129184+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:04:30.149510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:49:42.124430+0000","last_change":"2024-04-24T08:48:28.978534+0000","last_active":"2024-04-24T08:49:42.124430+0000","last_peered":"2024-04-24T08:49:42.124430+0000","last_clean":"2024-04-24T08:49:42.124430+0000","last_became_active":"2024-04-24T08:48:28.975407+0000","last_became_peered":"2024-04-24T08:48:28.975407+0000","last_unstale":"2024-04-24T08:49:42.124430+0000","last_undegraded":"2024-04-24T08:49:42.124430+0000","last_fullsized":"2024-04-24T08:49:42.124430+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:10:45.355039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.844780+0000","last_change":"2024-04-24T08:48:29.997682+0000","last_active":"2024-04-24T08:50:10.844780+0000","last_peered":"2024-04-24T08:50:10.844780+0000","last_clean":"2024-04-24T08:50:10.844780+0000","last_became_active":"2024-04-24T08:48:29.997223+0000","last_became_peered":"2024-04-24T08:48:29.997223+0000","last_unstale":"2024-04-24T08:50:10.844780+0000","last_undegraded":"2024-04-24T08:50:10.844780+0000","last_fullsized":"2024-04-24T08:50:10.844780+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:03:00.807249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:49:42.104177+0000","last_change":"2024-04-24T08:48:34.321477+0000","last_active":"2024-04-24T08:49:42.104177+0000","last_peered":"2024-04-24T08:49:42.104177+0000","last_clean":"2024-04-24T08:49:42.104177+0000","last_became_active":"2024-04-24T08:48:34.321018+0000","last_became_peered":"2024-04-24T08:48:34.321018+0000","last_unstale":"2024-04-24T08:49:42.104177+0000","last_undegraded":"2024-04-24T08:49:42.104177+0000","last_fullsized":"2024-04-24T08:49:42.104177+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:56:50.657298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"91'5","reported_seq":209,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:32.494946+0000","last_change":"2024-04-24T08:48:28.983722+0000","last_active":"2024-04-24T08:50:32.494946+0000","last_peered":"2024-04-24T08:50:32.494946+0000","last_clean":"2024-04-24T08:50:32.494946+0000","last_became_active":"2024-04-24T08:48:28.983394+0000","last_became_peered":"2024-04-24T08:48:28.983394+0000","last_unstale":"2024-04-24T08:50:32.494946+0000","last_undegraded":"2024-04-24T08:50:32.494946+0000","last_fullsized":"2024-04-24T08:50:32.494946+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:53:14.288742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":178,"num_read_kb":178,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.859762+0000","last_change":"2024-04-24T08:48:29.975009+0000","last_active":"2024-04-24T08:50:10.859762+0000","last_peered":"2024-04-24T08:50:10.859762+0000","last_clean":"2024-04-24T08:50:10.859762+0000","last_became_active":"2024-04-24T08:48:29.974137+0000","last_became_peered":"2024-04-24T08:48:29.974137+0000","last_unstale":"2024-04-24T08:50:10.859762+0000","last_undegraded":"2024-04-24T08:50:10.859762+0000","last_fullsized":"2024-04-24T08:50:10.859762+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:13:16.381923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:49:41.957314+0000","last_change":"2024-04-24T08:48:34.312126+0000","last_active":"2024-04-24T08:49:41.957314+0000","last_peered":"2024-04-24T08:49:41.957314+0000","last_clean":"2024-04-24T08:49:41.957314+0000","last_became_active":"2024-04-24T08:48:34.311689+0000","last_became_peered":"2024-04-24T08:48:34.311689+0000","last_unstale":"2024-04-24T08:49:41.957314+0000","last_undegraded":"2024-04-24T08:49:41.957314+0000","last_fullsized":"2024-04-24T08:49:41.957314+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:11:07.942412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:49:41.972481+0000","last_change":"2024-04-24T08:48:28.978355+0000","last_active":"2024-04-24T08:49:41.972481+0000","last_peered":"2024-04-24T08:49:41.972481+0000","last_clean":"2024-04-24T08:49:41.972481+0000","last_became_active":"2024-04-24T08:48:28.977351+0000","last_became_peered":"2024-04-24T08:48:28.977351+0000","last_unstale":"2024-04-24T08:49:41.972481+0000","last_undegraded":"2024-04-24T08:49:41.972481+0000","last_fullsized":"2024-04-24T08:49:41.972481+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:00:12.836569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.859435+0000","last_change":"2024-04-24T08:48:29.989015+0000","last_active":"2024-04-24T08:50:10.859435+0000","last_peered":"2024-04-24T08:50:10.859435+0000","last_clean":"2024-04-24T08:50:10.859435+0000","last_became_active":"2024-04-24T08:48:29.988648+0000","last_became_peered":"2024-04-24T08:48:29.988648+0000","last_unstale":"2024-04-24T08:50:10.859435+0000","last_undegraded":"2024-04-24T08:50:10.859435+0000","last_fullsized":"2024-04-24T08:50:10.859435+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:04:52.824894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:49:41.961094+0000","last_change":"2024-04-24T08:48:34.314230+0000","last_active":"2024-04-24T08:49:41.961094+0000","last_peered":"2024-04-24T08:49:41.961094+0000","last_clean":"2024-04-24T08:49:41.961094+0000","last_became_active":"2024-04-24T08:48:34.313617+0000","last_became_peered":"2024-04-24T08:48:34.313617+0000","last_unstale":"2024-04-24T08:49:41.961094+0000","last_undegraded":"2024-04-24T08:49:41.961094+0000","last_fullsized":"2024-04-24T08:49:41.961094+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:06:02.307575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:49:42.125704+0000","last_change":"2024-04-24T08:48:28.974516+0000","last_active":"2024-04-24T08:49:42.125704+0000","last_peered":"2024-04-24T08:49:42.125704+0000","last_clean":"2024-04-24T08:49:42.125704+0000","last_became_active":"2024-04-24T08:48:28.974194+0000","last_became_peered":"2024-04-24T08:48:28.974194+0000","last_unstale":"2024-04-24T08:49:42.125704+0000","last_undegraded":"2024-04-24T08:49:42.125704+0000","last_fullsized":"2024-04-24T08:49:42.125704+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:03:38.582610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.862776+0000","last_change":"2024-04-24T08:49:42.128553+0000","last_active":"2024-04-24T08:50:10.862776+0000","last_peered":"2024-04-24T08:50:10.862776+0000","last_clean":"2024-04-24T08:50:10.862776+0000","last_became_active":"2024-04-24T08:49:42.127061+0000","last_became_peered":"2024-04-24T08:49:42.127061+0000","last_unstale":"2024-04-24T08:50:10.862776+0000","last_undegraded":"2024-04-24T08:50:10.862776+0000","last_fullsized":"2024-04-24T08:50:10.862776+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:07:06.798284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:49:42.133052+0000","last_change":"2024-04-24T08:49:42.131305+0000","last_active":"2024-04-24T08:49:42.133052+0000","last_peered":"2024-04-24T08:49:42.133052+0000","last_clean":"2024-04-24T08:49:42.133052+0000","last_became_active":"2024-04-24T08:49:42.129680+0000","last_became_peered":"2024-04-24T08:49:42.129680+0000","last_unstale":"2024-04-24T08:49:42.133052+0000","last_undegraded":"2024-04-24T08:49:42.133052+0000","last_fullsized":"2024-04-24T08:49:42.133052+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:14:09.579649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"91'36","reported_seq":247,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:32.506639+0000","last_change":"2024-04-24T08:48:28.981675+0000","last_active":"2024-04-24T08:50:32.506639+0000","last_peered":"2024-04-24T08:50:32.506639+0000","last_clean":"2024-04-24T08:50:32.506639+0000","last_became_active":"2024-04-24T08:48:28.981082+0000","last_became_peered":"2024-04-24T08:48:28.981082+0000","last_unstale":"2024-04-24T08:50:32.506639+0000","last_undegraded":"2024-04-24T08:50:32.506639+0000","last_fullsized":"2024-04-24T08:50:32.506639+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:22:07.826011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61400,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":185,"num_read_kb":152,"num_write":38,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.845675+0000","last_change":"2024-04-24T08:48:29.986932+0000","last_active":"2024-04-24T08:50:10.845675+0000","last_peered":"2024-04-24T08:50:10.845675+0000","last_clean":"2024-04-24T08:50:10.845675+0000","last_became_active":"2024-04-24T08:48:29.986505+0000","last_became_peered":"2024-04-24T08:48:29.986505+0000","last_unstale":"2024-04-24T08:50:10.845675+0000","last_undegraded":"2024-04-24T08:50:10.845675+0000","last_fullsized":"2024-04-24T08:50:10.845675+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:53:36.050294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:49:42.126219+0000","last_change":"2024-04-24T08:48:34.327127+0000","last_active":"2024-04-24T08:49:42.126219+0000","last_peered":"2024-04-24T08:49:42.126219+0000","last_clean":"2024-04-24T08:49:42.126219+0000","last_became_active":"2024-04-24T08:48:34.323942+0000","last_became_peered":"2024-04-24T08:48:34.323942+0000","last_unstale":"2024-04-24T08:49:42.126219+0000","last_undegraded":"2024-04-24T08:49:42.126219+0000","last_fullsized":"2024-04-24T08:49:42.126219+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:22:49.637513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:49:41.965201+0000","last_change":"2024-04-24T08:48:28.978618+0000","last_active":"2024-04-24T08:49:41.965201+0000","last_peered":"2024-04-24T08:49:41.965201+0000","last_clean":"2024-04-24T08:49:41.965201+0000","last_became_active":"2024-04-24T08:48:28.978309+0000","last_became_peered":"2024-04-24T08:48:28.978309+0000","last_unstale":"2024-04-24T08:49:41.965201+0000","last_undegraded":"2024-04-24T08:49:41.965201+0000","last_fullsized":"2024-04-24T08:49:41.965201+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:35:50.091398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.852209+0000","last_change":"2024-04-24T08:48:29.985629+0000","last_active":"2024-04-24T08:50:10.852209+0000","last_peered":"2024-04-24T08:50:10.852209+0000","last_clean":"2024-04-24T08:50:10.852209+0000","last_became_active":"2024-04-24T08:48:29.985258+0000","last_became_peered":"2024-04-24T08:48:29.985258+0000","last_unstale":"2024-04-24T08:50:10.852209+0000","last_undegraded":"2024-04-24T08:50:10.852209+0000","last_fullsized":"2024-04-24T08:50:10.852209+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:57:29.302175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:49:42.129242+0000","last_change":"2024-04-24T08:49:42.129242+0000","last_active":"2024-04-24T08:49:42.129242+0000","last_peered":"2024-04-24T08:49:42.129242+0000","last_clean":"2024-04-24T08:49:42.129242+0000","last_became_active":"2024-04-24T08:49:42.127967+0000","last_became_peered":"2024-04-24T08:49:42.127967+0000","last_unstale":"2024-04-24T08:49:42.129242+0000","last_undegraded":"2024-04-24T08:49:42.129242+0000","last_fullsized":"2024-04-24T08:49:42.129242+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:42:11.206123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:49:42.124700+0000","last_change":"2024-04-24T08:48:28.977840+0000","last_active":"2024-04-24T08:49:42.124700+0000","last_peered":"2024-04-24T08:49:42.124700+0000","last_clean":"2024-04-24T08:49:42.124700+0000","last_became_active":"2024-04-24T08:48:28.975482+0000","last_became_peered":"2024-04-24T08:48:28.975482+0000","last_unstale":"2024-04-24T08:49:42.124700+0000","last_undegraded":"2024-04-24T08:49:42.124700+0000","last_fullsized":"2024-04-24T08:49:42.124700+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:51:52.468627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.860821+0000","last_change":"2024-04-24T08:49:42.129823+0000","last_active":"2024-04-24T08:50:10.860821+0000","last_peered":"2024-04-24T08:50:10.860821+0000","last_clean":"2024-04-24T08:50:10.860821+0000","last_became_active":"2024-04-24T08:49:42.127254+0000","last_became_peered":"2024-04-24T08:49:42.127254+0000","last_unstale":"2024-04-24T08:50:10.860821+0000","last_undegraded":"2024-04-24T08:50:10.860821+0000","last_fullsized":"2024-04-24T08:50:10.860821+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:51:10.804703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:49:41.958200+0000","last_change":"2024-04-24T08:48:34.314956+0000","last_active":"2024-04-24T08:49:41.958200+0000","last_peered":"2024-04-24T08:49:41.958200+0000","last_clean":"2024-04-24T08:49:41.958200+0000","last_became_active":"2024-04-24T08:48:34.314541+0000","last_became_peered":"2024-04-24T08:48:34.314541+0000","last_unstale":"2024-04-24T08:49:41.958200+0000","last_undegraded":"2024-04-24T08:49:41.958200+0000","last_fullsized":"2024-04-24T08:49:41.958200+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:42:48.614000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"89'5","reported_seq":123,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:31.534363+0000","last_change":"2024-04-24T08:48:28.974529+0000","last_active":"2024-04-24T08:50:31.534363+0000","last_peered":"2024-04-24T08:50:31.534363+0000","last_clean":"2024-04-24T08:50:31.534363+0000","last_became_active":"2024-04-24T08:48:28.973957+0000","last_became_peered":"2024-04-24T08:48:28.973957+0000","last_unstale":"2024-04-24T08:50:31.534363+0000","last_undegraded":"2024-04-24T08:50:31.534363+0000","last_fullsized":"2024-04-24T08:50:31.534363+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:44:46.216389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":92,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.847638+0000","last_change":"2024-04-24T08:48:29.981995+0000","last_active":"2024-04-24T08:50:10.847638+0000","last_peered":"2024-04-24T08:50:10.847638+0000","last_clean":"2024-04-24T08:50:10.847638+0000","last_became_active":"2024-04-24T08:48:29.980756+0000","last_became_peered":"2024-04-24T08:48:29.980756+0000","last_unstale":"2024-04-24T08:50:10.847638+0000","last_undegraded":"2024-04-24T08:50:10.847638+0000","last_fullsized":"2024-04-24T08:50:10.847638+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:42:52.770069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:49:42.122633+0000","last_change":"2024-04-24T08:48:34.308836+0000","last_active":"2024-04-24T08:49:42.122633+0000","last_peered":"2024-04-24T08:49:42.122633+0000","last_clean":"2024-04-24T08:49:42.122633+0000","last_became_active":"2024-04-24T08:48:34.308470+0000","last_became_peered":"2024-04-24T08:48:34.308470+0000","last_unstale":"2024-04-24T08:49:42.122633+0000","last_undegraded":"2024-04-24T08:49:42.122633+0000","last_fullsized":"2024-04-24T08:49:42.122633+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:38:14.906117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:49:41.966163+0000","last_change":"2024-04-24T08:48:28.971818+0000","last_active":"2024-04-24T08:49:41.966163+0000","last_peered":"2024-04-24T08:49:41.966163+0000","last_clean":"2024-04-24T08:49:41.966163+0000","last_became_active":"2024-04-24T08:48:28.971463+0000","last_became_peered":"2024-04-24T08:48:28.971463+0000","last_unstale":"2024-04-24T08:49:41.966163+0000","last_undegraded":"2024-04-24T08:49:41.966163+0000","last_fullsized":"2024-04-24T08:49:41.966163+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:10:22.829722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.860330+0000","last_change":"2024-04-24T08:48:29.980632+0000","last_active":"2024-04-24T08:50:10.860330+0000","last_peered":"2024-04-24T08:50:10.860330+0000","last_clean":"2024-04-24T08:50:10.860330+0000","last_became_active":"2024-04-24T08:48:29.980174+0000","last_became_peered":"2024-04-24T08:48:29.980174+0000","last_unstale":"2024-04-24T08:50:10.860330+0000","last_undegraded":"2024-04-24T08:50:10.860330+0000","last_fullsized":"2024-04-24T08:50:10.860330+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:45:32.709091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:49:42.122941+0000","last_change":"2024-04-24T08:48:34.316302+0000","last_active":"2024-04-24T08:49:42.122941+0000","last_peered":"2024-04-24T08:49:42.122941+0000","last_clean":"2024-04-24T08:49:42.122941+0000","last_became_active":"2024-04-24T08:48:34.315783+0000","last_became_peered":"2024-04-24T08:48:34.315783+0000","last_unstale":"2024-04-24T08:49:42.122941+0000","last_undegraded":"2024-04-24T08:49:42.122941+0000","last_fullsized":"2024-04-24T08:49:42.122941+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:29:01.733931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:49:41.966235+0000","last_change":"2024-04-24T08:48:28.972033+0000","last_active":"2024-04-24T08:49:41.966235+0000","last_peered":"2024-04-24T08:49:41.966235+0000","last_clean":"2024-04-24T08:49:41.966235+0000","last_became_active":"2024-04-24T08:48:28.971597+0000","last_became_peered":"2024-04-24T08:48:28.971597+0000","last_unstale":"2024-04-24T08:49:41.966235+0000","last_undegraded":"2024-04-24T08:49:41.966235+0000","last_fullsized":"2024-04-24T08:49:41.966235+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:11:27.827599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.837485+0000","last_change":"2024-04-24T08:49:41.975577+0000","last_active":"2024-04-24T08:50:10.837485+0000","last_peered":"2024-04-24T08:50:10.837485+0000","last_clean":"2024-04-24T08:50:10.837485+0000","last_became_active":"2024-04-24T08:49:41.975018+0000","last_became_peered":"2024-04-24T08:49:41.975018+0000","last_unstale":"2024-04-24T08:50:10.837485+0000","last_undegraded":"2024-04-24T08:50:10.837485+0000","last_fullsized":"2024-04-24T08:50:10.837485+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:07:53.051847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:49:41.978827+0000","last_change":"2024-04-24T08:49:41.978827+0000","last_active":"2024-04-24T08:49:41.978827+0000","last_peered":"2024-04-24T08:49:41.978827+0000","last_clean":"2024-04-24T08:49:41.978827+0000","last_became_active":"2024-04-24T08:49:41.977741+0000","last_became_peered":"2024-04-24T08:49:41.977741+0000","last_unstale":"2024-04-24T08:49:41.978827+0000","last_undegraded":"2024-04-24T08:49:41.978827+0000","last_fullsized":"2024-04-24T08:49:41.978827+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:16:54.754341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:49:41.958636+0000","last_change":"2024-04-24T08:48:28.970726+0000","last_active":"2024-04-24T08:49:41.958636+0000","last_peered":"2024-04-24T08:49:41.958636+0000","last_clean":"2024-04-24T08:49:41.958636+0000","last_became_active":"2024-04-24T08:48:28.969911+0000","last_became_peered":"2024-04-24T08:48:28.969911+0000","last_unstale":"2024-04-24T08:49:41.958636+0000","last_undegraded":"2024-04-24T08:49:41.958636+0000","last_fullsized":"2024-04-24T08:49:41.958636+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:55:26.357256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'4","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.851182+0000","last_change":"2024-04-24T08:48:29.967804+0000","last_active":"2024-04-24T08:50:10.851182+0000","last_peered":"2024-04-24T08:50:10.851182+0000","last_clean":"2024-04-24T08:50:10.851182+0000","last_became_active":"2024-04-24T08:48:29.967431+0000","last_became_peered":"2024-04-24T08:48:29.967431+0000","last_unstale":"2024-04-24T08:50:10.851182+0000","last_undegraded":"2024-04-24T08:50:10.851182+0000","last_fullsized":"2024-04-24T08:50:10.851182+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:00:02.779543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:49:42.124119+0000","last_change":"2024-04-24T08:48:34.316844+0000","last_active":"2024-04-24T08:49:42.124119+0000","last_peered":"2024-04-24T08:49:42.124119+0000","last_clean":"2024-04-24T08:49:42.124119+0000","last_became_active":"2024-04-24T08:48:34.316426+0000","last_became_peered":"2024-04-24T08:48:34.316426+0000","last_unstale":"2024-04-24T08:49:42.124119+0000","last_undegraded":"2024-04-24T08:49:42.124119+0000","last_fullsized":"2024-04-24T08:49:42.124119+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:10:13.378443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'5","reported_seq":124,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:34.490255+0000","last_change":"2024-04-24T08:48:28.971544+0000","last_active":"2024-04-24T08:50:34.490255+0000","last_peered":"2024-04-24T08:50:34.490255+0000","last_clean":"2024-04-24T08:50:34.490255+0000","last_became_active":"2024-04-24T08:48:28.970417+0000","last_became_peered":"2024-04-24T08:48:28.970417+0000","last_unstale":"2024-04-24T08:50:34.490255+0000","last_undegraded":"2024-04-24T08:50:34.490255+0000","last_fullsized":"2024-04-24T08:50:34.490255+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:35:04.506126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":92,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.847509+0000","last_change":"2024-04-24T08:48:29.987542+0000","last_active":"2024-04-24T08:50:10.847509+0000","last_peered":"2024-04-24T08:50:10.847509+0000","last_clean":"2024-04-24T08:50:10.847509+0000","last_became_active":"2024-04-24T08:48:29.987258+0000","last_became_peered":"2024-04-24T08:48:29.987258+0000","last_unstale":"2024-04-24T08:50:10.847509+0000","last_undegraded":"2024-04-24T08:50:10.847509+0000","last_fullsized":"2024-04-24T08:50:10.847509+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:37:23.341132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:49:42.122302+0000","last_change":"2024-04-24T08:48:34.322636+0000","last_active":"2024-04-24T08:49:42.122302+0000","last_peered":"2024-04-24T08:49:42.122302+0000","last_clean":"2024-04-24T08:49:42.122302+0000","last_became_active":"2024-04-24T08:48:34.321764+0000","last_became_peered":"2024-04-24T08:48:34.321764+0000","last_unstale":"2024-04-24T08:49:42.122302+0000","last_undegraded":"2024-04-24T08:49:42.122302+0000","last_fullsized":"2024-04-24T08:49:42.122302+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:56:10.563907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.836543+0000","last_change":"2024-04-24T08:48:29.981734+0000","last_active":"2024-04-24T08:50:10.836543+0000","last_peered":"2024-04-24T08:50:10.836543+0000","last_clean":"2024-04-24T08:50:10.836543+0000","last_became_active":"2024-04-24T08:48:29.980491+0000","last_became_peered":"2024-04-24T08:48:29.980491+0000","last_unstale":"2024-04-24T08:50:10.836543+0000","last_undegraded":"2024-04-24T08:50:10.836543+0000","last_fullsized":"2024-04-24T08:50:10.836543+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:08:45.189026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'5","reported_seq":124,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:33.539111+0000","last_change":"2024-04-24T08:48:28.954621+0000","last_active":"2024-04-24T08:50:33.539111+0000","last_peered":"2024-04-24T08:50:33.539111+0000","last_clean":"2024-04-24T08:50:33.539111+0000","last_became_active":"2024-04-24T08:48:28.954139+0000","last_became_peered":"2024-04-24T08:48:28.954139+0000","last_unstale":"2024-04-24T08:50:33.539111+0000","last_undegraded":"2024-04-24T08:50:33.539111+0000","last_fullsized":"2024-04-24T08:50:33.539111+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:59:19.360414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":91,"num_read_kb":91,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:49:41.970520+0000","last_change":"2024-04-24T08:48:34.323774+0000","last_active":"2024-04-24T08:49:41.970520+0000","last_peered":"2024-04-24T08:49:41.970520+0000","last_clean":"2024-04-24T08:49:41.970520+0000","last_became_active":"2024-04-24T08:48:34.322214+0000","last_became_peered":"2024-04-24T08:48:34.322214+0000","last_unstale":"2024-04-24T08:49:41.970520+0000","last_undegraded":"2024-04-24T08:49:41.970520+0000","last_fullsized":"2024-04-24T08:49:41.970520+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:51:03.971107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.860028+0000","last_change":"2024-04-24T08:48:29.995372+0000","last_active":"2024-04-24T08:50:10.860028+0000","last_peered":"2024-04-24T08:50:10.860028+0000","last_clean":"2024-04-24T08:50:10.860028+0000","last_became_active":"2024-04-24T08:48:29.994744+0000","last_became_peered":"2024-04-24T08:48:29.994744+0000","last_unstale":"2024-04-24T08:50:10.860028+0000","last_undegraded":"2024-04-24T08:50:10.860028+0000","last_fullsized":"2024-04-24T08:50:10.860028+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:28:55.356390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:49:41.960779+0000","last_change":"2024-04-24T08:48:28.974921+0000","last_active":"2024-04-24T08:49:41.960779+0000","last_peered":"2024-04-24T08:49:41.960779+0000","last_clean":"2024-04-24T08:49:41.960779+0000","last_became_active":"2024-04-24T08:48:28.974113+0000","last_became_peered":"2024-04-24T08:48:28.974113+0000","last_unstale":"2024-04-24T08:49:41.960779+0000","last_undegraded":"2024-04-24T08:49:41.960779+0000","last_fullsized":"2024-04-24T08:49:41.960779+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:35:12.635253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":360,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:49:41.971050+0000","last_change":"2024-04-24T08:46:45.190615+0000","last_active":"2024-04-24T08:49:41.971050+0000","last_peered":"2024-04-24T08:49:41.971050+0000","last_clean":"2024-04-24T08:49:41.971050+0000","last_became_active":"2024-04-24T08:46:45.190011+0000","last_became_peered":"2024-04-24T08:46:45.190011+0000","last_unstale":"2024-04-24T08:49:41.971050+0000","last_undegraded":"2024-04-24T08:49:41.971050+0000","last_fullsized":"2024-04-24T08:49:41.971050+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:41:57.806569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:41:57.806569+0000","last_clean_scrub_stamp":"2024-04-24T08:41:57.806569+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:13:08.283236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:49:42.126082+0000","last_change":"2024-04-24T08:48:34.324588+0000","last_active":"2024-04-24T08:49:42.126082+0000","last_peered":"2024-04-24T08:49:42.126082+0000","last_clean":"2024-04-24T08:49:42.126082+0000","last_became_active":"2024-04-24T08:48:34.321969+0000","last_became_peered":"2024-04-24T08:48:34.321969+0000","last_unstale":"2024-04-24T08:49:42.126082+0000","last_undegraded":"2024-04-24T08:49:42.126082+0000","last_fullsized":"2024-04-24T08:49:42.126082+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:19:56.921198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.840077+0000","last_change":"2024-04-24T08:48:29.981573+0000","last_active":"2024-04-24T08:50:10.840077+0000","last_peered":"2024-04-24T08:50:10.840077+0000","last_clean":"2024-04-24T08:50:10.840077+0000","last_became_active":"2024-04-24T08:48:29.979216+0000","last_became_peered":"2024-04-24T08:48:29.979216+0000","last_unstale":"2024-04-24T08:50:10.840077+0000","last_undegraded":"2024-04-24T08:50:10.840077+0000","last_fullsized":"2024-04-24T08:50:10.840077+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:38:09.164883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:49:41.971476+0000","last_change":"2024-04-24T08:48:28.981492+0000","last_active":"2024-04-24T08:49:41.971476+0000","last_peered":"2024-04-24T08:49:41.971476+0000","last_clean":"2024-04-24T08:49:41.971476+0000","last_became_active":"2024-04-24T08:48:28.981050+0000","last_became_peered":"2024-04-24T08:48:28.981050+0000","last_unstale":"2024-04-24T08:49:41.971476+0000","last_undegraded":"2024-04-24T08:49:41.971476+0000","last_fullsized":"2024-04-24T08:49:41.971476+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:22:09.829566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:49:41.969209+0000","last_change":"2024-04-24T08:48:34.317502+0000","last_active":"2024-04-24T08:49:41.969209+0000","last_peered":"2024-04-24T08:49:41.969209+0000","last_clean":"2024-04-24T08:49:41.969209+0000","last_became_active":"2024-04-24T08:48:34.316614+0000","last_became_peered":"2024-04-24T08:48:34.316614+0000","last_unstale":"2024-04-24T08:49:41.969209+0000","last_undegraded":"2024-04-24T08:49:41.969209+0000","last_fullsized":"2024-04-24T08:49:41.969209+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:44:12.469926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:49:41.965569+0000","last_change":"2024-04-24T08:48:28.970261+0000","last_active":"2024-04-24T08:49:41.965569+0000","last_peered":"2024-04-24T08:49:41.965569+0000","last_clean":"2024-04-24T08:49:41.965569+0000","last_became_active":"2024-04-24T08:48:28.969813+0000","last_became_peered":"2024-04-24T08:48:28.969813+0000","last_unstale":"2024-04-24T08:49:41.965569+0000","last_undegraded":"2024-04-24T08:49:41.965569+0000","last_fullsized":"2024-04-24T08:49:41.965569+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:46:12.311756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.838518+0000","last_change":"2024-04-24T08:48:29.976712+0000","last_active":"2024-04-24T08:50:10.838518+0000","last_peered":"2024-04-24T08:50:10.838518+0000","last_clean":"2024-04-24T08:50:10.838518+0000","last_became_active":"2024-04-24T08:48:29.976166+0000","last_became_peered":"2024-04-24T08:48:29.976166+0000","last_unstale":"2024-04-24T08:50:10.838518+0000","last_undegraded":"2024-04-24T08:50:10.838518+0000","last_fullsized":"2024-04-24T08:50:10.838518+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:52:53.757829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:49:42.103682+0000","last_change":"2024-04-24T08:48:34.310173+0000","last_active":"2024-04-24T08:49:42.103682+0000","last_peered":"2024-04-24T08:49:42.103682+0000","last_clean":"2024-04-24T08:49:42.103682+0000","last_became_active":"2024-04-24T08:48:34.309772+0000","last_became_peered":"2024-04-24T08:48:34.309772+0000","last_unstale":"2024-04-24T08:49:42.103682+0000","last_undegraded":"2024-04-24T08:49:42.103682+0000","last_fullsized":"2024-04-24T08:49:42.103682+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:35:50.302769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'14","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:49:41.959148+0000","last_change":"2024-04-24T08:48:28.970127+0000","last_active":"2024-04-24T08:49:41.959148+0000","last_peered":"2024-04-24T08:49:41.959148+0000","last_clean":"2024-04-24T08:49:41.959148+0000","last_became_active":"2024-04-24T08:48:28.968803+0000","last_became_peered":"2024-04-24T08:48:28.968803+0000","last_unstale":"2024-04-24T08:49:41.959148+0000","last_undegraded":"2024-04-24T08:49:41.959148+0000","last_fullsized":"2024-04-24T08:49:41.959148+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:28:47.782276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.853054+0000","last_change":"2024-04-24T08:48:29.992964+0000","last_active":"2024-04-24T08:50:10.853054+0000","last_peered":"2024-04-24T08:50:10.853054+0000","last_clean":"2024-04-24T08:50:10.853054+0000","last_became_active":"2024-04-24T08:48:29.987501+0000","last_became_peered":"2024-04-24T08:48:29.987501+0000","last_unstale":"2024-04-24T08:50:10.853054+0000","last_undegraded":"2024-04-24T08:50:10.853054+0000","last_fullsized":"2024-04-24T08:50:10.853054+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:49:19.157052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:49:41.960206+0000","last_change":"2024-04-24T08:48:34.315393+0000","last_active":"2024-04-24T08:49:41.960206+0000","last_peered":"2024-04-24T08:49:41.960206+0000","last_clean":"2024-04-24T08:49:41.960206+0000","last_became_active":"2024-04-24T08:48:34.314943+0000","last_became_peered":"2024-04-24T08:48:34.314943+0000","last_unstale":"2024-04-24T08:49:41.960206+0000","last_undegraded":"2024-04-24T08:49:41.960206+0000","last_fullsized":"2024-04-24T08:49:41.960206+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:03:13.312223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:49:41.963669+0000","last_change":"2024-04-24T08:48:28.969813+0000","last_active":"2024-04-24T08:49:41.963669+0000","last_peered":"2024-04-24T08:49:41.963669+0000","last_clean":"2024-04-24T08:49:41.963669+0000","last_became_active":"2024-04-24T08:48:28.968485+0000","last_became_peered":"2024-04-24T08:48:28.968485+0000","last_unstale":"2024-04-24T08:49:41.963669+0000","last_undegraded":"2024-04-24T08:49:41.963669+0000","last_fullsized":"2024-04-24T08:49:41.963669+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:32:10.095723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.862968+0000","last_change":"2024-04-24T08:48:29.997731+0000","last_active":"2024-04-24T08:50:10.862968+0000","last_peered":"2024-04-24T08:50:10.862968+0000","last_clean":"2024-04-24T08:50:10.862968+0000","last_became_active":"2024-04-24T08:48:29.997373+0000","last_became_peered":"2024-04-24T08:48:29.997373+0000","last_unstale":"2024-04-24T08:50:10.862968+0000","last_undegraded":"2024-04-24T08:50:10.862968+0000","last_fullsized":"2024-04-24T08:50:10.862968+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:57:19.531173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:49:41.961100+0000","last_change":"2024-04-24T08:48:34.326170+0000","last_active":"2024-04-24T08:49:41.961100+0000","last_peered":"2024-04-24T08:49:41.961100+0000","last_clean":"2024-04-24T08:49:41.961100+0000","last_became_active":"2024-04-24T08:48:34.325678+0000","last_became_peered":"2024-04-24T08:48:34.325678+0000","last_unstale":"2024-04-24T08:49:41.961100+0000","last_undegraded":"2024-04-24T08:49:41.961100+0000","last_fullsized":"2024-04-24T08:49:41.961100+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:12:47.003864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:49:41.968205+0000","last_change":"2024-04-24T08:48:28.975522+0000","last_active":"2024-04-24T08:49:41.968205+0000","last_peered":"2024-04-24T08:49:41.968205+0000","last_clean":"2024-04-24T08:49:41.968205+0000","last_became_active":"2024-04-24T08:48:28.975093+0000","last_became_peered":"2024-04-24T08:48:28.975093+0000","last_unstale":"2024-04-24T08:49:41.968205+0000","last_undegraded":"2024-04-24T08:49:41.968205+0000","last_fullsized":"2024-04-24T08:49:41.968205+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:37:31.809694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.795450+0000","last_change":"2024-04-24T08:48:29.982502+0000","last_active":"2024-04-24T08:50:10.795450+0000","last_peered":"2024-04-24T08:50:10.795450+0000","last_clean":"2024-04-24T08:50:10.795450+0000","last_became_active":"2024-04-24T08:48:29.982141+0000","last_became_peered":"2024-04-24T08:48:29.982141+0000","last_unstale":"2024-04-24T08:50:10.795450+0000","last_undegraded":"2024-04-24T08:50:10.795450+0000","last_fullsized":"2024-04-24T08:50:10.795450+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:46:15.851359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:49:41.958317+0000","last_change":"2024-04-24T08:48:34.322717+0000","last_active":"2024-04-24T08:49:41.958317+0000","last_peered":"2024-04-24T08:49:41.958317+0000","last_clean":"2024-04-24T08:49:41.958317+0000","last_became_active":"2024-04-24T08:48:34.321837+0000","last_became_peered":"2024-04-24T08:48:34.321837+0000","last_unstale":"2024-04-24T08:49:41.958317+0000","last_undegraded":"2024-04-24T08:49:41.958317+0000","last_fullsized":"2024-04-24T08:49:41.958317+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:40:11.557436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:49:42.103699+0000","last_change":"2024-04-24T08:48:28.973951+0000","last_active":"2024-04-24T08:49:42.103699+0000","last_peered":"2024-04-24T08:49:42.103699+0000","last_clean":"2024-04-24T08:49:42.103699+0000","last_became_active":"2024-04-24T08:48:28.973131+0000","last_became_peered":"2024-04-24T08:48:28.973131+0000","last_unstale":"2024-04-24T08:49:42.103699+0000","last_undegraded":"2024-04-24T08:49:42.103699+0000","last_fullsized":"2024-04-24T08:49:42.103699+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:45:45.197337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.856618+0000","last_change":"2024-04-24T08:48:29.992689+0000","last_active":"2024-04-24T08:50:10.856618+0000","last_peered":"2024-04-24T08:50:10.856618+0000","last_clean":"2024-04-24T08:50:10.856618+0000","last_became_active":"2024-04-24T08:48:29.988874+0000","last_became_peered":"2024-04-24T08:48:29.988874+0000","last_unstale":"2024-04-24T08:50:10.856618+0000","last_undegraded":"2024-04-24T08:50:10.856618+0000","last_fullsized":"2024-04-24T08:50:10.856618+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:42:41.578424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:49:41.961021+0000","last_change":"2024-04-24T08:48:34.318640+0000","last_active":"2024-04-24T08:49:41.961021+0000","last_peered":"2024-04-24T08:49:41.961021+0000","last_clean":"2024-04-24T08:49:41.961021+0000","last_became_active":"2024-04-24T08:48:34.318039+0000","last_became_peered":"2024-04-24T08:48:34.318039+0000","last_unstale":"2024-04-24T08:49:41.961021+0000","last_undegraded":"2024-04-24T08:49:41.961021+0000","last_fullsized":"2024-04-24T08:49:41.961021+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:17:27.110930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'3","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:49:41.964478+0000","last_change":"2024-04-24T08:48:28.980828+0000","last_active":"2024-04-24T08:49:41.964478+0000","last_peered":"2024-04-24T08:49:41.964478+0000","last_clean":"2024-04-24T08:49:41.964478+0000","last_became_active":"2024-04-24T08:48:28.979814+0000","last_became_peered":"2024-04-24T08:48:28.979814+0000","last_unstale":"2024-04-24T08:49:41.964478+0000","last_undegraded":"2024-04-24T08:49:41.964478+0000","last_fullsized":"2024-04-24T08:49:41.964478+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:02:28.145847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.861479+0000","last_change":"2024-04-24T08:48:29.995003+0000","last_active":"2024-04-24T08:50:10.861479+0000","last_peered":"2024-04-24T08:50:10.861479+0000","last_clean":"2024-04-24T08:50:10.861479+0000","last_became_active":"2024-04-24T08:48:29.994516+0000","last_became_peered":"2024-04-24T08:48:29.994516+0000","last_unstale":"2024-04-24T08:50:10.861479+0000","last_undegraded":"2024-04-24T08:50:10.861479+0000","last_fullsized":"2024-04-24T08:50:10.861479+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:15:46.094534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:49:41.960975+0000","last_change":"2024-04-24T08:48:34.311721+0000","last_active":"2024-04-24T08:49:41.960975+0000","last_peered":"2024-04-24T08:49:41.960975+0000","last_clean":"2024-04-24T08:49:41.960975+0000","last_became_active":"2024-04-24T08:48:34.311241+0000","last_became_peered":"2024-04-24T08:48:34.311241+0000","last_unstale":"2024-04-24T08:49:41.960975+0000","last_undegraded":"2024-04-24T08:49:41.960975+0000","last_fullsized":"2024-04-24T08:49:41.960975+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:16:17.679237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:49:41.964980+0000","last_change":"2024-04-24T08:48:28.973336+0000","last_active":"2024-04-24T08:49:41.964980+0000","last_peered":"2024-04-24T08:49:41.964980+0000","last_clean":"2024-04-24T08:49:41.964980+0000","last_became_active":"2024-04-24T08:48:28.972666+0000","last_became_peered":"2024-04-24T08:48:28.972666+0000","last_unstale":"2024-04-24T08:49:41.964980+0000","last_undegraded":"2024-04-24T08:49:41.964980+0000","last_fullsized":"2024-04-24T08:49:41.964980+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:53:28.358291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.860099+0000","last_change":"2024-04-24T08:48:29.983498+0000","last_active":"2024-04-24T08:50:10.860099+0000","last_peered":"2024-04-24T08:50:10.860099+0000","last_clean":"2024-04-24T08:50:10.860099+0000","last_became_active":"2024-04-24T08:48:29.983167+0000","last_became_peered":"2024-04-24T08:48:29.983167+0000","last_unstale":"2024-04-24T08:50:10.860099+0000","last_undegraded":"2024-04-24T08:50:10.860099+0000","last_fullsized":"2024-04-24T08:50:10.860099+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:09:56.648047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:49:42.129192+0000","last_change":"2024-04-24T08:49:42.129192+0000","last_active":"2024-04-24T08:49:42.129192+0000","last_peered":"2024-04-24T08:49:42.129192+0000","last_clean":"2024-04-24T08:49:42.129192+0000","last_became_active":"2024-04-24T08:49:42.124814+0000","last_became_peered":"2024-04-24T08:49:42.124814+0000","last_unstale":"2024-04-24T08:49:42.129192+0000","last_undegraded":"2024-04-24T08:49:42.129192+0000","last_fullsized":"2024-04-24T08:49:42.129192+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:52:46.273836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6,7],"acting":[1,8,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'5","reported_seq":126,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:35.506060+0000","last_change":"2024-04-24T08:48:28.974528+0000","last_active":"2024-04-24T08:50:35.506060+0000","last_peered":"2024-04-24T08:50:35.506060+0000","last_clean":"2024-04-24T08:50:35.506060+0000","last_became_active":"2024-04-24T08:48:28.974203+0000","last_became_peered":"2024-04-24T08:48:28.974203+0000","last_unstale":"2024-04-24T08:50:35.506060+0000","last_undegraded":"2024-04-24T08:50:35.506060+0000","last_fullsized":"2024-04-24T08:50:35.506060+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:26:15.701204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":95,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.862198+0000","last_change":"2024-04-24T08:48:29.984356+0000","last_active":"2024-04-24T08:50:10.862198+0000","last_peered":"2024-04-24T08:50:10.862198+0000","last_clean":"2024-04-24T08:50:10.862198+0000","last_became_active":"2024-04-24T08:48:29.983949+0000","last_became_peered":"2024-04-24T08:48:29.983949+0000","last_unstale":"2024-04-24T08:50:10.862198+0000","last_undegraded":"2024-04-24T08:50:10.862198+0000","last_fullsized":"2024-04-24T08:50:10.862198+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:51:59.409835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:49:42.126598+0000","last_change":"2024-04-24T08:48:34.312418+0000","last_active":"2024-04-24T08:49:42.126598+0000","last_peered":"2024-04-24T08:49:42.126598+0000","last_clean":"2024-04-24T08:49:42.126598+0000","last_became_active":"2024-04-24T08:48:34.312035+0000","last_became_peered":"2024-04-24T08:48:34.312035+0000","last_unstale":"2024-04-24T08:49:42.126598+0000","last_undegraded":"2024-04-24T08:49:42.126598+0000","last_fullsized":"2024-04-24T08:49:42.126598+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:27:33.461732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"89'3","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:49:42.125750+0000","last_change":"2024-04-24T08:48:28.976070+0000","last_active":"2024-04-24T08:49:42.125750+0000","last_peered":"2024-04-24T08:49:42.125750+0000","last_clean":"2024-04-24T08:49:42.125750+0000","last_became_active":"2024-04-24T08:48:28.975703+0000","last_became_peered":"2024-04-24T08:48:28.975703+0000","last_unstale":"2024-04-24T08:49:42.125750+0000","last_undegraded":"2024-04-24T08:49:42.125750+0000","last_fullsized":"2024-04-24T08:49:42.125750+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:46:13.227529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'6","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.843654+0000","last_change":"2024-04-24T08:49:42.128814+0000","last_active":"2024-04-24T08:50:10.843654+0000","last_peered":"2024-04-24T08:50:10.843654+0000","last_clean":"2024-04-24T08:50:10.843654+0000","last_became_active":"2024-04-24T08:49:42.127422+0000","last_became_peered":"2024-04-24T08:49:42.127422+0000","last_unstale":"2024-04-24T08:50:10.843654+0000","last_undegraded":"2024-04-24T08:50:10.843654+0000","last_fullsized":"2024-04-24T08:50:10.843654+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:58:25.181798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:49:42.123553+0000","last_change":"2024-04-24T08:48:34.320191+0000","last_active":"2024-04-24T08:49:42.123553+0000","last_peered":"2024-04-24T08:49:42.123553+0000","last_clean":"2024-04-24T08:49:42.123553+0000","last_became_active":"2024-04-24T08:48:34.315415+0000","last_became_peered":"2024-04-24T08:48:34.315415+0000","last_unstale":"2024-04-24T08:49:42.123553+0000","last_undegraded":"2024-04-24T08:49:42.123553+0000","last_fullsized":"2024-04-24T08:49:42.123553+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:55:11.372789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:49:42.122289+0000","last_change":"2024-04-24T08:48:28.968550+0000","last_active":"2024-04-24T08:49:42.122289+0000","last_peered":"2024-04-24T08:49:42.122289+0000","last_clean":"2024-04-24T08:49:42.122289+0000","last_became_active":"2024-04-24T08:48:28.968120+0000","last_became_peered":"2024-04-24T08:48:28.968120+0000","last_unstale":"2024-04-24T08:49:42.122289+0000","last_undegraded":"2024-04-24T08:49:42.122289+0000","last_fullsized":"2024-04-24T08:49:42.122289+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:46:00.870285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.850343+0000","last_change":"2024-04-24T08:48:29.984755+0000","last_active":"2024-04-24T08:50:10.850343+0000","last_peered":"2024-04-24T08:50:10.850343+0000","last_clean":"2024-04-24T08:50:10.850343+0000","last_became_active":"2024-04-24T08:48:29.982093+0000","last_became_peered":"2024-04-24T08:48:29.982093+0000","last_unstale":"2024-04-24T08:50:10.850343+0000","last_undegraded":"2024-04-24T08:50:10.850343+0000","last_fullsized":"2024-04-24T08:50:10.850343+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:25:22.620152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.838530+0000","last_change":"2024-04-24T08:48:29.983101+0000","last_active":"2024-04-24T08:50:10.838530+0000","last_peered":"2024-04-24T08:50:10.838530+0000","last_clean":"2024-04-24T08:50:10.838530+0000","last_became_active":"2024-04-24T08:48:29.981871+0000","last_became_peered":"2024-04-24T08:48:29.981871+0000","last_unstale":"2024-04-24T08:50:10.838530+0000","last_undegraded":"2024-04-24T08:50:10.838530+0000","last_fullsized":"2024-04-24T08:50:10.838530+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:02:44.582111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.833695+0000","last_change":"2024-04-24T08:49:42.128974+0000","last_active":"2024-04-24T08:50:10.833695+0000","last_peered":"2024-04-24T08:50:10.833695+0000","last_clean":"2024-04-24T08:50:10.833695+0000","last_became_active":"2024-04-24T08:49:42.127710+0000","last_became_peered":"2024-04-24T08:49:42.127710+0000","last_unstale":"2024-04-24T08:50:10.833695+0000","last_undegraded":"2024-04-24T08:50:10.833695+0000","last_fullsized":"2024-04-24T08:50:10.833695+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:09:20.279852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.856499+0000","last_change":"2024-04-24T08:48:29.988241+0000","last_active":"2024-04-24T08:50:10.856499+0000","last_peered":"2024-04-24T08:50:10.856499+0000","last_clean":"2024-04-24T08:50:10.856499+0000","last_became_active":"2024-04-24T08:48:29.987930+0000","last_became_peered":"2024-04-24T08:48:29.987930+0000","last_unstale":"2024-04-24T08:50:10.856499+0000","last_undegraded":"2024-04-24T08:50:10.856499+0000","last_fullsized":"2024-04-24T08:50:10.856499+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:42:24.687624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.864848+0000","last_change":"2024-04-24T08:48:29.992441+0000","last_active":"2024-04-24T08:50:10.864848+0000","last_peered":"2024-04-24T08:50:10.864848+0000","last_clean":"2024-04-24T08:50:10.864848+0000","last_became_active":"2024-04-24T08:48:29.988788+0000","last_became_peered":"2024-04-24T08:48:29.988788+0000","last_unstale":"2024-04-24T08:50:10.864848+0000","last_undegraded":"2024-04-24T08:50:10.864848+0000","last_fullsized":"2024-04-24T08:50:10.864848+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:08:31.549509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.863506+0000","last_change":"2024-04-24T08:48:29.982116+0000","last_active":"2024-04-24T08:50:10.863506+0000","last_peered":"2024-04-24T08:50:10.863506+0000","last_clean":"2024-04-24T08:50:10.863506+0000","last_became_active":"2024-04-24T08:48:29.981636+0000","last_became_peered":"2024-04-24T08:48:29.981636+0000","last_unstale":"2024-04-24T08:50:10.863506+0000","last_undegraded":"2024-04-24T08:50:10.863506+0000","last_fullsized":"2024-04-24T08:50:10.863506+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:11:01.943425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.831656+0000","last_change":"2024-04-24T08:48:29.982244+0000","last_active":"2024-04-24T08:50:10.831656+0000","last_peered":"2024-04-24T08:50:10.831656+0000","last_clean":"2024-04-24T08:50:10.831656+0000","last_became_active":"2024-04-24T08:48:29.981622+0000","last_became_peered":"2024-04-24T08:48:29.981622+0000","last_unstale":"2024-04-24T08:50:10.831656+0000","last_undegraded":"2024-04-24T08:50:10.831656+0000","last_fullsized":"2024-04-24T08:50:10.831656+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:02:57.723949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.859981+0000","last_change":"2024-04-24T08:48:29.985867+0000","last_active":"2024-04-24T08:50:10.859981+0000","last_peered":"2024-04-24T08:50:10.859981+0000","last_clean":"2024-04-24T08:50:10.859981+0000","last_became_active":"2024-04-24T08:48:29.985430+0000","last_became_peered":"2024-04-24T08:48:29.985430+0000","last_unstale":"2024-04-24T08:50:10.859981+0000","last_undegraded":"2024-04-24T08:50:10.859981+0000","last_fullsized":"2024-04-24T08:50:10.859981+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:09:40.467623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.848111+0000","last_change":"2024-04-24T08:48:29.996474+0000","last_active":"2024-04-24T08:50:10.848111+0000","last_peered":"2024-04-24T08:50:10.848111+0000","last_clean":"2024-04-24T08:50:10.848111+0000","last_became_active":"2024-04-24T08:48:29.995958+0000","last_became_peered":"2024-04-24T08:48:29.995958+0000","last_unstale":"2024-04-24T08:50:10.848111+0000","last_undegraded":"2024-04-24T08:50:10.848111+0000","last_fullsized":"2024-04-24T08:50:10.848111+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:50:19.865125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.856229+0000","last_change":"2024-04-24T08:48:29.978285+0000","last_active":"2024-04-24T08:50:10.856229+0000","last_peered":"2024-04-24T08:50:10.856229+0000","last_clean":"2024-04-24T08:50:10.856229+0000","last_became_active":"2024-04-24T08:48:29.976445+0000","last_became_peered":"2024-04-24T08:48:29.976445+0000","last_unstale":"2024-04-24T08:50:10.856229+0000","last_undegraded":"2024-04-24T08:50:10.856229+0000","last_fullsized":"2024-04-24T08:50:10.856229+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:30:29.598733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.852463+0000","last_change":"2024-04-24T08:48:29.994442+0000","last_active":"2024-04-24T08:50:10.852463+0000","last_peered":"2024-04-24T08:50:10.852463+0000","last_clean":"2024-04-24T08:50:10.852463+0000","last_became_active":"2024-04-24T08:48:29.993896+0000","last_became_peered":"2024-04-24T08:48:29.993896+0000","last_unstale":"2024-04-24T08:50:10.852463+0000","last_undegraded":"2024-04-24T08:50:10.852463+0000","last_fullsized":"2024-04-24T08:50:10.852463+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:37:14.023511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.861403+0000","last_change":"2024-04-24T08:49:41.981252+0000","last_active":"2024-04-24T08:50:10.861403+0000","last_peered":"2024-04-24T08:50:10.861403+0000","last_clean":"2024-04-24T08:50:10.861403+0000","last_became_active":"2024-04-24T08:49:41.980495+0000","last_became_peered":"2024-04-24T08:49:41.980495+0000","last_unstale":"2024-04-24T08:50:10.861403+0000","last_undegraded":"2024-04-24T08:50:10.861403+0000","last_fullsized":"2024-04-24T08:50:10.861403+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:47:33.579204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,8],"acting":[1,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.859268+0000","last_change":"2024-04-24T08:48:29.986021+0000","last_active":"2024-04-24T08:50:10.859268+0000","last_peered":"2024-04-24T08:50:10.859268+0000","last_clean":"2024-04-24T08:50:10.859268+0000","last_became_active":"2024-04-24T08:48:29.985182+0000","last_became_peered":"2024-04-24T08:48:29.985182+0000","last_unstale":"2024-04-24T08:50:10.859268+0000","last_undegraded":"2024-04-24T08:50:10.859268+0000","last_fullsized":"2024-04-24T08:50:10.859268+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:10:57.143065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-24T08:50:10.833371+0000","last_change":"2024-04-24T08:48:29.979886+0000","last_active":"2024-04-24T08:50:10.833371+0000","last_peered":"2024-04-24T08:50:10.833371+0000","last_clean":"2024-04-24T08:50:10.833371+0000","last_became_active":"2024-04-24T08:48:29.978779+0000","last_became_peered":"2024-04-24T08:48:29.978779+0000","last_unstale":"2024-04-24T08:50:10.833371+0000","last_undegraded":"2024-04-24T08:50:10.833371+0000","last_fullsized":"2024-04-24T08:50:10.833371+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:51:51.303990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68354,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":939,"num_read_kb":540,"num_write":248,"num_write_kb":142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":205062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":215,"ondisk_log_size":215,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383722,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":501425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645361,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32860,"kb_used_data":1684,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712292,"statfs":{"total":95995035648,"available":95961387008,"internally_reserved":0,"allocated":1724416,"data_stored":1149641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808888,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32212,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712940,"statfs":{"total":95995035648,"available":95962050560,"internally_reserved":0,"allocated":1060864,"data_stored":505451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070527,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1052672,"data_stored":501733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332167,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32788,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712364,"statfs":{"total":95995035648,"available":95961460736,"internally_reserved":0,"allocated":1650688,"data_stored":1098371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495694,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1032192,"data_stored":496859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574962,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36408,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708744,"statfs":{"total":95995035648,"available":95957753856,"internally_reserved":0,"allocated":1097728,"data_stored":559289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738489,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36368,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708784,"statfs":{"total":95995035648,"available":95957794816,"internally_reserved":0,"allocated":1056768,"data_stored":498315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411434,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28108,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717044,"statfs":{"total":95995035648,"available":95966253056,"internally_reserved":0,"allocated":1118208,"data_stored":560041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182500,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1044480,"data_stored":496567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920860,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1040384,"data_stored":500933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757334,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32796,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712356,"statfs":{"total":95995035648,"available":95961452544,"internally_reserved":0,"allocated":1658880,"data_stored":1093953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":63152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":63904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":63136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T08:50:38.502 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-24T08:50:38.502 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-24T08:50:38.502 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-04-24T08:50:38.503 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid bd3e039e-0215-11ef-bc93-c7b262605968 -- ceph health --format=json 2024-04-24T08:50:38.986 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T08:50:39.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:50:39 smithi083 ceph-mon[27907]: from='client.15165 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T08:50:39.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:50:39 smithi083 ceph-mon[27907]: pgmap v440: 129 pgs: 129 active+clean; 644 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-24T08:50:39.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:50:39 smithi138 ceph-mon[31441]: from='client.15165 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T08:50:39.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:50:39 smithi138 ceph-mon[31441]: pgmap v440: 129 pgs: 129 active+clean; 644 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-24T08:50:39.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:50:39 smithi189 ceph-mon[31791]: from='client.15165 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T08:50:39.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:50:39 smithi189 ceph-mon[31791]: pgmap v440: 129 pgs: 129 active+clean; 644 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-24T08:50:41.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:41.274+0000 7fe9ffc00700 1 -- 172.21.15.83:0/3863749086 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9f8103500 msgr2=0x7fe9f810f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:41.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:41.274+0000 7fe9ffc00700 1 --2- 172.21.15.83:0/3863749086 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9f8103500 0x7fe9f810f960 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fe9f000b0a0 tx=0x7fe9f000b3b0 comp rx=0 tx=0).stop 2024-04-24T08:50:41.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:41.274+0000 7fe9ffc00700 1 -- 172.21.15.83:0/3863749086 shutdown_connections 2024-04-24T08:50:41.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:41.274+0000 7fe9ffc00700 1 --2- 172.21.15.83:0/3863749086 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9f8103500 0x7fe9f810f960 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:41.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:41.274+0000 7fe9ffc00700 1 --2- 172.21.15.83:0/3863749086 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe9f8108b90 0x7fe9f8108f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:41.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:41.274+0000 7fe9ffc00700 1 --2- 172.21.15.83:0/3863749086 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe9f8102b40 0x7fe9f8102fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:41.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:41.274+0000 7fe9ffc00700 1 -- 172.21.15.83:0/3863749086 >> 172.21.15.83:0/3863749086 conn(0x7fe9f8079c50 msgr2=0x7fe9f807a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:50:41.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:41.274+0000 7fe9ffc00700 1 -- 172.21.15.83:0/3863749086 shutdown_connections 2024-04-24T08:50:41.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:41.274+0000 7fe9ffc00700 1 -- 172.21.15.83:0/3863749086 wait complete. 2024-04-24T08:50:41.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:41.275+0000 7fe9ffc00700 1 Processor -- start 2024-04-24T08:50:41.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:41.276+0000 7fe9ffc00700 1 -- start start 2024-04-24T08:50:41.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:41.276+0000 7fe9ffc00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9f8102b40 0x7fe9f819fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:41.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:41.276+0000 7fe9ffc00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe9f8103500 0x7fe9f81a03c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:41.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:41.276+0000 7fe9ffc00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe9f8108b90 0x7fe9f810ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:41.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:41.276+0000 7fe9ffc00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe9f8112d00 con 0x7fe9f8102b40 2024-04-24T08:50:41.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:41.276+0000 7fe9ffc00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe9f8112b80 con 0x7fe9f8108b90 2024-04-24T08:50:41.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:41.276+0000 7fe9ffc00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe9f8112e80 con 0x7fe9f8103500 2024-04-24T08:50:41.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:41.276+0000 7fe9fd200700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9f8102b40 0x7fe9f819fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:41.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:41.276+0000 7fe9fd200700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9f8102b40 0x7fe9f819fe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48444/0 (socket says 172.21.15.83:48444) 2024-04-24T08:50:41.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:41.276+0000 7fe9fd200700 1 -- 172.21.15.83:0/1494941982 learned_addr learned my addr 172.21.15.83:0/1494941982 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:50:41.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:41.277+0000 7fe9fdc00700 1 --2- 172.21.15.83:0/1494941982 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe9f8108b90 0x7fe9f810ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:41.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:41.277+0000 7fe9fd200700 1 -- 172.21.15.83:0/1494941982 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe9f8103500 msgr2=0x7fe9f81a03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:41.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:41.277+0000 7fe9fd200700 1 --2- 172.21.15.83:0/1494941982 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe9f8103500 0x7fe9f81a03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:41.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:41.277+0000 7fe9fd200700 1 -- 172.21.15.83:0/1494941982 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe9f8108b90 msgr2=0x7fe9f810ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:41.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:41.277+0000 7fe9fd200700 1 --2- 172.21.15.83:0/1494941982 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe9f8108b90 0x7fe9f810ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:41.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:41.277+0000 7fe9fd200700 1 -- 172.21.15.83:0/1494941982 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9f810d310 con 0x7fe9f8102b40 2024-04-24T08:50:41.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:41.277+0000 7fe9fdc00700 1 --2- 172.21.15.83:0/1494941982 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe9f8108b90 0x7fe9f810ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:50:41.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:41.277+0000 7fe9fd200700 1 --2- 172.21.15.83:0/1494941982 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9f8102b40 0x7fe9f819fe80 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec00d950 tx=0x7fe9ec00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:50:41.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:41.278+0000 7fe9f5600700 1 -- 172.21.15.83:0/1494941982 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9ec016070 con 0x7fe9f8102b40 2024-04-24T08:50:41.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:41.278+0000 7fe9f5600700 1 -- 172.21.15.83:0/1494941982 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe9ec010460 con 0x7fe9f8102b40 2024-04-24T08:50:41.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:41.278+0000 7fe9ffc00700 1 -- 172.21.15.83:0/1494941982 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9f810d600 con 0x7fe9f8102b40 2024-04-24T08:50:41.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:41.278+0000 7fe9f5600700 1 -- 172.21.15.83:0/1494941982 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9ec00f600 con 0x7fe9f8102b40 2024-04-24T08:50:41.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:41.278+0000 7fe9ffc00700 1 -- 172.21.15.83:0/1494941982 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9f819bb00 con 0x7fe9f8102b40 2024-04-24T08:50:41.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:41.280+0000 7fe9ffc00700 1 -- 172.21.15.83:0/1494941982 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9f80697e0 con 0x7fe9f8102b40 2024-04-24T08:50:41.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:41.281+0000 7fe9f5600700 1 -- 172.21.15.83:0/1494941982 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fe9ec019070 con 0x7fe9f8102b40 2024-04-24T08:50:41.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:41.283+0000 7fe9f5600700 1 --2- 172.21.15.83:0/1494941982 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fe9e00a3c40 0x7fe9e00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:41.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:41.283+0000 7fe9f5600700 1 -- 172.21.15.83:0/1494941982 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fe9ec0d0310 con 0x7fe9f8102b40 2024-04-24T08:50:41.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:41.283+0000 7fe9f7e00700 1 --2- 172.21.15.83:0/1494941982 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fe9e00a3c40 0x7fe9e00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:41.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:41.284+0000 7fe9f7e00700 1 --2- 172.21.15.83:0/1494941982 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fe9e00a3c40 0x7fe9e00a6100 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe9e4005950 tx=0x7fe9e40058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:50:41.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:41.288+0000 7fe9f5600700 1 -- 172.21.15.83:0/1494941982 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe9ec09c930 con 0x7fe9f8102b40 2024-04-24T08:50:41.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:41.571+0000 7fe9ffc00700 1 -- 172.21.15.83:0/1494941982 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fe9f819bd60 con 0x7fe9f8102b40 2024-04-24T08:50:41.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:41.572+0000 7fe9f5600700 1 -- 172.21.15.83:0/1494941982 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fe9ec09c230 con 0x7fe9f8102b40 2024-04-24T08:50:41.573 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:50:41.573 INFO:teuthology.orchestra.run.smithi083.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-24T08:50:41.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:41.575+0000 7fe9ffc00700 1 -- 172.21.15.83:0/1494941982 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fe9e00a3c40 msgr2=0x7fe9e00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:41.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:41.575+0000 7fe9ffc00700 1 --2- 172.21.15.83:0/1494941982 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fe9e00a3c40 0x7fe9e00a6100 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe9e4005950 tx=0x7fe9e40058e0 comp rx=0 tx=0).stop 2024-04-24T08:50:41.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:41.575+0000 7fe9ffc00700 1 -- 172.21.15.83:0/1494941982 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9f8102b40 msgr2=0x7fe9f819fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:41.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:41.575+0000 7fe9ffc00700 1 --2- 172.21.15.83:0/1494941982 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9f8102b40 0x7fe9f819fe80 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec00d950 tx=0x7fe9ec00dd10 comp rx=0 tx=0).stop 2024-04-24T08:50:41.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:41.575+0000 7fe9ffc00700 1 -- 172.21.15.83:0/1494941982 shutdown_connections 2024-04-24T08:50:41.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:41.575+0000 7fe9ffc00700 1 --2- 172.21.15.83:0/1494941982 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fe9e00a3c40 0x7fe9e00a6100 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:41.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:41.575+0000 7fe9ffc00700 1 --2- 172.21.15.83:0/1494941982 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9f8102b40 0x7fe9f819fe80 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:41.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:41.575+0000 7fe9ffc00700 1 --2- 172.21.15.83:0/1494941982 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe9f8108b90 0x7fe9f810ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:41.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:41.575+0000 7fe9ffc00700 1 --2- 172.21.15.83:0/1494941982 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe9f8103500 0x7fe9f81a03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:41.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:41.575+0000 7fe9ffc00700 1 -- 172.21.15.83:0/1494941982 >> 172.21.15.83:0/1494941982 conn(0x7fe9f8079c50 msgr2=0x7fe9f810dfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:50:41.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:41.575+0000 7fe9ffc00700 1 -- 172.21.15.83:0/1494941982 shutdown_connections 2024-04-24T08:50:41.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:41.575+0000 7fe9ffc00700 1 -- 172.21.15.83:0/1494941982 wait complete. 2024-04-24T08:50:42.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:50:41 smithi189 ceph-mon[31791]: pgmap v441: 129 pgs: 129 active+clean; 644 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-24T08:50:42.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:50:41 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]: dispatch 2024-04-24T08:50:42.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:50:41 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-04-24T08:50:42.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:50:41 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]: dispatch 2024-04-24T08:50:42.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:50:41 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]: dispatch 2024-04-24T08:50:42.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:50:41 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 1]}]: dispatch 2024-04-24T08:50:42.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:50:41 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/1494941982' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-24T08:50:42.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:50:41 smithi083 ceph-mon[27907]: pgmap v441: 129 pgs: 129 active+clean; 644 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-24T08:50:42.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:50:41 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]: dispatch 2024-04-24T08:50:42.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:50:41 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-04-24T08:50:42.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:50:41 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]: dispatch 2024-04-24T08:50:42.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:50:41 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]: dispatch 2024-04-24T08:50:42.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:50:41 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 1]}]: dispatch 2024-04-24T08:50:42.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:50:41 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/1494941982' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-24T08:50:42.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:50:41 smithi138 ceph-mon[31441]: pgmap v441: 129 pgs: 129 active+clean; 644 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-24T08:50:42.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:50:41 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]: dispatch 2024-04-24T08:50:42.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:50:41 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-04-24T08:50:42.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:50:41 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]: dispatch 2024-04-24T08:50:42.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:50:41 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]: dispatch 2024-04-24T08:50:42.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:50:41 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 1]}]: dispatch 2024-04-24T08:50:42.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:50:41 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/1494941982' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-24T08:50:42.479 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-04-24T08:50:42.479 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T08:50:42.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:42.679+0000 7f4566e00640 1 -- 172.21.15.83:0/1953322174 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45601019f0 msgr2=0x7f4560101e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:42.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:42.679+0000 7f4566e00640 1 --2- 172.21.15.83:0/1953322174 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45601019f0 0x7f4560101e70 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f4558009ca0 tx=0x7f455802a8c0 comp rx=0 tx=0).stop 2024-04-24T08:50:42.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:42.680+0000 7f4566e00640 1 -- 172.21.15.83:0/1953322174 shutdown_connections 2024-04-24T08:50:42.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:42.680+0000 7f4566e00640 1 --2- 172.21.15.83:0/1953322174 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45601019f0 0x7f4560101e70 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:42.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:42.680+0000 7f4566e00640 1 --2- 172.21.15.83:0/1953322174 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4560101050 0x7f45601014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:42.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:42.680+0000 7f4566e00640 1 --2- 172.21.15.83:0/1953322174 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f45601006f0 0x7f4560100b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:42.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:42.680+0000 7f4566e00640 1 -- 172.21.15.83:0/1953322174 >> 172.21.15.83:0/1953322174 conn(0x7f45600fc310 msgr2=0x7f45600fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:50:42.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:42.680+0000 7f4566e00640 1 -- 172.21.15.83:0/1953322174 shutdown_connections 2024-04-24T08:50:42.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:42.680+0000 7f4566e00640 1 -- 172.21.15.83:0/1953322174 wait complete. 2024-04-24T08:50:42.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:42.681+0000 7f4566e00640 1 Processor -- start 2024-04-24T08:50:42.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:42.681+0000 7f4566e00640 1 -- start start 2024-04-24T08:50:42.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:42.681+0000 7f4566e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45601006f0 0x7f456019a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:42.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:42.681+0000 7f4566e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4560101050 0x7f456019a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:42.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:42.681+0000 7f4566e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f45601019f0 0x7f45601a1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:42.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:42.681+0000 7f4566e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f456010fbf0 con 0x7f45601006f0 2024-04-24T08:50:42.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:42.681+0000 7f4566e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f456010fa70 con 0x7f4560101050 2024-04-24T08:50:42.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:42.681+0000 7f4566e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f456010fd70 con 0x7f45601019f0 2024-04-24T08:50:42.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:42.681+0000 7f4565a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45601006f0 0x7f456019a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:42.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:42.681+0000 7f4565a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45601006f0 0x7f456019a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48456/0 (socket says 172.21.15.83:48456) 2024-04-24T08:50:42.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:42.682+0000 7f4565a00640 1 -- 172.21.15.83:0/2892092471 learned_addr learned my addr 172.21.15.83:0/2892092471 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:50:42.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:42.682+0000 7f4566400640 1 --2- 172.21.15.83:0/2892092471 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f45601019f0 0x7f45601a1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:42.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:42.682+0000 7f4565000640 1 --2- 172.21.15.83:0/2892092471 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4560101050 0x7f456019a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:42.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:42.682+0000 7f4565a00640 1 -- 172.21.15.83:0/2892092471 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f45601019f0 msgr2=0x7f45601a1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:42.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:42.682+0000 7f4565a00640 1 --2- 172.21.15.83:0/2892092471 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f45601019f0 0x7f45601a1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:42.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:42.682+0000 7f4565a00640 1 -- 172.21.15.83:0/2892092471 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4560101050 msgr2=0x7f456019a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:42.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:42.682+0000 7f4565a00640 1 --2- 172.21.15.83:0/2892092471 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4560101050 0x7f456019a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:42.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:42.682+0000 7f4565a00640 1 -- 172.21.15.83:0/2892092471 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45601a1ec0 con 0x7f45601006f0 2024-04-24T08:50:42.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:42.682+0000 7f4566400640 1 --2- 172.21.15.83:0/2892092471 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f45601019f0 0x7f45601a1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:50:42.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:42.682+0000 7f4565000640 1 --2- 172.21.15.83:0/2892092471 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4560101050 0x7f456019a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:50:42.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:42.682+0000 7f4565a00640 1 --2- 172.21.15.83:0/2892092471 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45601006f0 0x7f456019a2a0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f455400b4a0 tx=0x7f455400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:50:42.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:42.683+0000 7f455e000640 1 -- 172.21.15.83:0/2892092471 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4554014070 con 0x7f45601006f0 2024-04-24T08:50:42.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:42.683+0000 7f455e000640 1 -- 172.21.15.83:0/2892092471 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4554010ac0 con 0x7f45601006f0 2024-04-24T08:50:42.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:42.683+0000 7f455e000640 1 -- 172.21.15.83:0/2892092471 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4554017c30 con 0x7f45601006f0 2024-04-24T08:50:42.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:42.683+0000 7f4566e00640 1 -- 172.21.15.83:0/2892092471 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45601a21b0 con 0x7f45601006f0 2024-04-24T08:50:42.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:42.683+0000 7f4566e00640 1 -- 172.21.15.83:0/2892092471 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45601a26b0 con 0x7f45601006f0 2024-04-24T08:50:42.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:42.684+0000 7f4566e00640 1 -- 172.21.15.83:0/2892092471 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f456010b620 con 0x7f45601006f0 2024-04-24T08:50:42.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:42.686+0000 7f455e000640 1 -- 172.21.15.83:0/2892092471 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f4554017070 con 0x7f45601006f0 2024-04-24T08:50:42.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:42.688+0000 7f455e000640 1 --2- 172.21.15.83:0/2892092471 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f45480a3c30 0x7f45480a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:42.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:42.688+0000 7f455e000640 1 -- 172.21.15.83:0/2892092471 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9237+0+0 (secure 0 0 0) 0x7f45540cff20 con 0x7f45601006f0 2024-04-24T08:50:42.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:42.688+0000 7f4565000640 1 --2- 172.21.15.83:0/2892092471 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f45480a3c30 0x7f45480a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:42.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:42.689+0000 7f4565000640 1 --2- 172.21.15.83:0/2892092471 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f45480a3c30 0x7f45480a60f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f456019b7b0 tx=0x7f454c005d30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:50:42.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:42.692+0000 7f455e000640 1 -- 172.21.15.83:0/2892092471 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f455409c4b0 con 0x7f45601006f0 2024-04-24T08:50:42.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:42.897+0000 7f4566e00640 1 -- 172.21.15.83:0/2892092471 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4560062c30 con 0x7f45601006f0 2024-04-24T08:50:42.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:42.901+0000 7f455e000640 1 -- 172.21.15.83:0/2892092471 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24887 (secure 0 0 0) 0x7f455409bdb0 con 0x7f45601006f0 2024-04-24T08:50:42.902 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:50:42.902 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":92,"fsid":"bd3e039e-0215-11ef-bc93-c7b262605968","created":"2024-04-24T08:37:32.209944+0000","modified":"2024-04-24T08:50:41.952973+0000","last_up_change":"2024-04-24T08:47:17.966041+0000","last_in_change":"2024-04-24T08:46:53.835936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T08:41:56.834372+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T08:48:26.978311+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T08:48:28.352731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T08:48:32.275167+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2cd77f3a-19dd-4c6e-a199-1dcbd5faa679","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2132675555},{"type":"v1","addr":"172.21.15.83:6803","nonce":2132675555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2132675555},{"type":"v1","addr":"172.21.15.83:6805","nonce":2132675555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2132675555},{"type":"v1","addr":"172.21.15.83:6809","nonce":2132675555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2132675555},{"type":"v1","addr":"172.21.15.83:6807","nonce":2132675555}]},"public_addr":"172.21.15.83:6803/2132675555","cluster_addr":"172.21.15.83:6805/2132675555","heartbeat_back_addr":"172.21.15.83:6809/2132675555","heartbeat_front_addr":"172.21.15.83:6807/2132675555","state":["exists","up"]},{"osd":1,"uuid":"aba7f5df-2a4c-4c23-b502-d193605a5b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":3589158880},{"type":"v1","addr":"172.21.15.138:6801","nonce":3589158880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":3589158880},{"type":"v1","addr":"172.21.15.138:6803","nonce":3589158880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":3589158880},{"type":"v1","addr":"172.21.15.138:6807","nonce":3589158880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":3589158880},{"type":"v1","addr":"172.21.15.138:6805","nonce":3589158880}]},"public_addr":"172.21.15.138:6801/3589158880","cluster_addr":"172.21.15.138:6803/3589158880","heartbeat_back_addr":"172.21.15.138:6807/3589158880","heartbeat_front_addr":"172.21.15.138:6805/3589158880","state":["exists","up"]},{"osd":2,"uuid":"071bc167-2333-4420-bb58-bffbc10d7641","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":3169072688},{"type":"v1","addr":"172.21.15.189:6801","nonce":3169072688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":3169072688},{"type":"v1","addr":"172.21.15.189:6803","nonce":3169072688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":3169072688},{"type":"v1","addr":"172.21.15.189:6807","nonce":3169072688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":3169072688},{"type":"v1","addr":"172.21.15.189:6805","nonce":3169072688}]},"public_addr":"172.21.15.189:6801/3169072688","cluster_addr":"172.21.15.189:6803/3169072688","heartbeat_back_addr":"172.21.15.189:6807/3169072688","heartbeat_front_addr":"172.21.15.189:6805/3169072688","state":["exists","up"]},{"osd":3,"uuid":"f5624d78-4455-4e07-8c1a-f47c94cddb23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3131092450},{"type":"v1","addr":"172.21.15.83:6811","nonce":3131092450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3131092450},{"type":"v1","addr":"172.21.15.83:6813","nonce":3131092450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3131092450},{"type":"v1","addr":"172.21.15.83:6817","nonce":3131092450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3131092450},{"type":"v1","addr":"172.21.15.83:6815","nonce":3131092450}]},"public_addr":"172.21.15.83:6811/3131092450","cluster_addr":"172.21.15.83:6813/3131092450","heartbeat_back_addr":"172.21.15.83:6817/3131092450","heartbeat_front_addr":"172.21.15.83:6815/3131092450","state":["exists","up"]},{"osd":4,"uuid":"7b50cd14-9385-4535-a7b0-1c1da93c8548","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":823775360},{"type":"v1","addr":"172.21.15.138:6809","nonce":823775360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":823775360},{"type":"v1","addr":"172.21.15.138:6811","nonce":823775360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":823775360},{"type":"v1","addr":"172.21.15.138:6815","nonce":823775360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":823775360},{"type":"v1","addr":"172.21.15.138:6813","nonce":823775360}]},"public_addr":"172.21.15.138:6809/823775360","cluster_addr":"172.21.15.138:6811/823775360","heartbeat_back_addr":"172.21.15.138:6815/823775360","heartbeat_front_addr":"172.21.15.138:6813/823775360","state":["exists","up"]},{"osd":5,"uuid":"9c647797-35bc-449f-be84-af97d2c69c2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":1399817431},{"type":"v1","addr":"172.21.15.189:6809","nonce":1399817431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":1399817431},{"type":"v1","addr":"172.21.15.189:6811","nonce":1399817431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":1399817431},{"type":"v1","addr":"172.21.15.189:6815","nonce":1399817431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":1399817431},{"type":"v1","addr":"172.21.15.189:6813","nonce":1399817431}]},"public_addr":"172.21.15.189:6809/1399817431","cluster_addr":"172.21.15.189:6811/1399817431","heartbeat_back_addr":"172.21.15.189:6815/1399817431","heartbeat_front_addr":"172.21.15.189:6813/1399817431","state":["exists","up"]},{"osd":6,"uuid":"8f056584-16c8-4454-b445-3f780f26c196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":1611358483},{"type":"v1","addr":"172.21.15.83:6819","nonce":1611358483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1611358483},{"type":"v1","addr":"172.21.15.83:6821","nonce":1611358483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1611358483},{"type":"v1","addr":"172.21.15.83:6825","nonce":1611358483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":1611358483},{"type":"v1","addr":"172.21.15.83:6823","nonce":1611358483}]},"public_addr":"172.21.15.83:6819/1611358483","cluster_addr":"172.21.15.83:6821/1611358483","heartbeat_back_addr":"172.21.15.83:6825/1611358483","heartbeat_front_addr":"172.21.15.83:6823/1611358483","state":["exists","up"]},{"osd":7,"uuid":"7603efe3-da77-4b7f-9d03-29c4f293b6d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":3557856706},{"type":"v1","addr":"172.21.15.138:6817","nonce":3557856706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":3557856706},{"type":"v1","addr":"172.21.15.138:6819","nonce":3557856706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":3557856706},{"type":"v1","addr":"172.21.15.138:6823","nonce":3557856706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":3557856706},{"type":"v1","addr":"172.21.15.138:6821","nonce":3557856706}]},"public_addr":"172.21.15.138:6817/3557856706","cluster_addr":"172.21.15.138:6819/3557856706","heartbeat_back_addr":"172.21.15.138:6823/3557856706","heartbeat_front_addr":"172.21.15.138:6821/3557856706","state":["exists","up"]},{"osd":8,"uuid":"27d7d1b8-d6a0-417e-b8aa-596325aa1a77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":207018554},{"type":"v1","addr":"172.21.15.189:6817","nonce":207018554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":207018554},{"type":"v1","addr":"172.21.15.189:6819","nonce":207018554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":207018554},{"type":"v1","addr":"172.21.15.189:6823","nonce":207018554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":207018554},{"type":"v1","addr":"172.21.15.189:6821","nonce":207018554}]},"public_addr":"172.21.15.189:6817/207018554","cluster_addr":"172.21.15.189:6819/207018554","heartbeat_back_addr":"172.21.15.189:6823/207018554","heartbeat_front_addr":"172.21.15.189:6821/207018554","state":["exists","up"]},{"osd":9,"uuid":"baf19b98-881b-4d3a-9f72-8aa2dd73cad6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":519636751},{"type":"v1","addr":"172.21.15.83:6827","nonce":519636751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":519636751},{"type":"v1","addr":"172.21.15.83:6829","nonce":519636751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":519636751},{"type":"v1","addr":"172.21.15.83:6833","nonce":519636751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":519636751},{"type":"v1","addr":"172.21.15.83:6831","nonce":519636751}]},"public_addr":"172.21.15.83:6827/519636751","cluster_addr":"172.21.15.83:6829/519636751","heartbeat_back_addr":"172.21.15.83:6833/519636751","heartbeat_front_addr":"172.21.15.83:6831/519636751","state":["exists","up"]},{"osd":10,"uuid":"eb346717-5c07-47c8-95be-0e68fe449fdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1130872098},{"type":"v1","addr":"172.21.15.138:6825","nonce":1130872098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1130872098},{"type":"v1","addr":"172.21.15.138:6827","nonce":1130872098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1130872098},{"type":"v1","addr":"172.21.15.138:6831","nonce":1130872098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1130872098},{"type":"v1","addr":"172.21.15.138:6829","nonce":1130872098}]},"public_addr":"172.21.15.138:6825/1130872098","cluster_addr":"172.21.15.138:6827/1130872098","heartbeat_back_addr":"172.21.15.138:6831/1130872098","heartbeat_front_addr":"172.21.15.138:6829/1130872098","state":["exists","up"]},{"osd":11,"uuid":"8e37568e-981c-45b8-b2af-2f1b73898542","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":2998782690},{"type":"v1","addr":"172.21.15.189:6825","nonce":2998782690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":2998782690},{"type":"v1","addr":"172.21.15.189:6827","nonce":2998782690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":2998782690},{"type":"v1","addr":"172.21.15.189:6831","nonce":2998782690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":2998782690},{"type":"v1","addr":"172.21.15.189:6829","nonce":2998782690}]},"public_addr":"172.21.15.189:6825/2998782690","cluster_addr":"172.21.15.189:6827/2998782690","heartbeat_back_addr":"172.21.15.189:6831/2998782690","heartbeat_front_addr":"172.21.15.189:6829/2998782690","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:40:47.980103+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:41:21.970795+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:41:55.160426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:42:30.400475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:43:05.705663+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:43:40.112312+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:44:16.781289+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:44:52.151187+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:45:28.055944+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:46:04.969301+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:46:40.918306+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:47:16.886874+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"2.1d","mappings":[{"from":2,"to":1}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.2a","mappings":[{"from":3,"to":8}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.a","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:6801/820049883":"2024-04-25T08:38:40.661315+0000","172.21.15.83:0/4272874138":"2024-04-25T08:38:40.661315+0000","172.21.15.83:0/4212994335":"2024-04-25T08:38:40.661315+0000","172.21.15.83:6801/1996371091":"2024-04-25T08:38:05.800856+0000","172.21.15.83:0/1820663683":"2024-04-25T08:38:05.800856+0000","172.21.15.83:0/3802897262":"2024-04-25T08:38:05.800856+0000","172.21.15.83:6800/1996371091":"2024-04-25T08:38:05.800856+0000","172.21.15.83:6800/820049883":"2024-04-25T08:38:40.661315+0000","172.21.15.83:0/1366125049":"2024-04-25T08:38:05.800856+0000","172.21.15.83:0/340452868":"2024-04-25T08:38:40.661315+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T08:50:42.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:42.903+0000 7f4566e00640 1 -- 172.21.15.83:0/2892092471 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f45480a3c30 msgr2=0x7f45480a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:42.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:42.903+0000 7f4566e00640 1 --2- 172.21.15.83:0/2892092471 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f45480a3c30 0x7f45480a60f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f456019b7b0 tx=0x7f454c005d30 comp rx=0 tx=0).stop 2024-04-24T08:50:42.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:42.904+0000 7f4566e00640 1 -- 172.21.15.83:0/2892092471 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45601006f0 msgr2=0x7f456019a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:42.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:42.904+0000 7f4566e00640 1 --2- 172.21.15.83:0/2892092471 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45601006f0 0x7f456019a2a0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f455400b4a0 tx=0x7f455400b960 comp rx=0 tx=0).stop 2024-04-24T08:50:42.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:42.904+0000 7f4566e00640 1 -- 172.21.15.83:0/2892092471 shutdown_connections 2024-04-24T08:50:42.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:42.904+0000 7f4566e00640 1 --2- 172.21.15.83:0/2892092471 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f45601019f0 0x7f45601a1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:42.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:42.904+0000 7f4566e00640 1 --2- 172.21.15.83:0/2892092471 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4560101050 0x7f456019a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:42.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:42.904+0000 7f4566e00640 1 --2- 172.21.15.83:0/2892092471 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f45480a3c30 0x7f45480a60f0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:42.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:42.904+0000 7f4566e00640 1 --2- 172.21.15.83:0/2892092471 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45601006f0 0x7f456019a2a0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:42.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:42.904+0000 7f4566e00640 1 -- 172.21.15.83:0/2892092471 >> 172.21.15.83:0/2892092471 conn(0x7f45600fc310 msgr2=0x7f45600fda60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:50:42.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:42.904+0000 7f4566e00640 1 -- 172.21.15.83:0/2892092471 shutdown_connections 2024-04-24T08:50:42.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:42.904+0000 7f4566e00640 1 -- 172.21.15.83:0/2892092471 wait complete. 2024-04-24T08:50:42.923 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T08:41:56.834372+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T08:48:26.978311+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T08:48:28.352731+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-24T08:48:32.275167+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-24T08:50:42.923 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T08:50:43.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:43.131+0000 7ff541200640 1 -- 172.21.15.83:0/924026282 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff53c105cf0 msgr2=0x7ff53c10c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:43.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:43.131+0000 7ff541200640 1 --2- 172.21.15.83:0/924026282 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff53c105cf0 0x7ff53c10c580 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7ff534009ca0 tx=0x7ff53402a8c0 comp rx=0 tx=0).stop 2024-04-24T08:50:43.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:43.131+0000 7ff541200640 1 -- 172.21.15.83:0/924026282 shutdown_connections 2024-04-24T08:50:43.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:43.131+0000 7ff541200640 1 --2- 172.21.15.83:0/924026282 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff53c105cf0 0x7ff53c10c580 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:43.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:43.131+0000 7ff541200640 1 --2- 172.21.15.83:0/924026282 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff53c105330 0x7ff53c1057b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:43.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:43.131+0000 7ff541200640 1 --2- 172.21.15.83:0/924026282 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff53c104140 0x7ff53c104560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:43.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:43.131+0000 7ff541200640 1 -- 172.21.15.83:0/924026282 >> 172.21.15.83:0/924026282 conn(0x7ff53c0ff6b0 msgr2=0x7ff53c101af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:50:43.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:43.132+0000 7ff541200640 1 -- 172.21.15.83:0/924026282 shutdown_connections 2024-04-24T08:50:43.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:43.132+0000 7ff541200640 1 -- 172.21.15.83:0/924026282 wait complete. 2024-04-24T08:50:43.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:43.132+0000 7ff541200640 1 Processor -- start 2024-04-24T08:50:43.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:43.133+0000 7ff541200640 1 -- start start 2024-04-24T08:50:43.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:43.133+0000 7ff541200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff53c104140 0x7ff53c19a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:43.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:43.133+0000 7ff541200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff53c105330 0x7ff53c19a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:43.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:43.133+0000 7ff541200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff53c105cf0 0x7ff53c1a1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:43.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:43.133+0000 7ff541200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff53c10fd30 con 0x7ff53c104140 2024-04-24T08:50:43.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:43.133+0000 7ff541200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff53c10fbb0 con 0x7ff53c105330 2024-04-24T08:50:43.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:43.133+0000 7ff541200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff53c10feb0 con 0x7ff53c105cf0 2024-04-24T08:50:43.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:43.133+0000 7ff53b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff53c104140 0x7ff53c19a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:43.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:43.133+0000 7ff53b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff53c104140 0x7ff53c19a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48472/0 (socket says 172.21.15.83:48472) 2024-04-24T08:50:43.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:43.133+0000 7ff53b400640 1 -- 172.21.15.83:0/3708362236 learned_addr learned my addr 172.21.15.83:0/3708362236 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:50:43.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:43.133+0000 7ff53aa00640 1 --2- 172.21.15.83:0/3708362236 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff53c105330 0x7ff53c19a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:43.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:43.133+0000 7ff53be00640 1 --2- 172.21.15.83:0/3708362236 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff53c105cf0 0x7ff53c1a1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:43.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:43.134+0000 7ff53b400640 1 -- 172.21.15.83:0/3708362236 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff53c105cf0 msgr2=0x7ff53c1a1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:43.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:43.134+0000 7ff53b400640 1 --2- 172.21.15.83:0/3708362236 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff53c105cf0 0x7ff53c1a1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:43.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:43.134+0000 7ff53b400640 1 -- 172.21.15.83:0/3708362236 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff53c105330 msgr2=0x7ff53c19a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:43.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:43.134+0000 7ff53b400640 1 --2- 172.21.15.83:0/3708362236 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff53c105330 0x7ff53c19a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:43.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:43.134+0000 7ff53b400640 1 -- 172.21.15.83:0/3708362236 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff53c1a1fc0 con 0x7ff53c104140 2024-04-24T08:50:43.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:43.134+0000 7ff53b400640 1 --2- 172.21.15.83:0/3708362236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff53c104140 0x7ff53c19a3a0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7ff52c00b450 tx=0x7ff52c00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:50:43.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:43.134+0000 7ff533e00640 1 -- 172.21.15.83:0/3708362236 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff52c015020 con 0x7ff53c104140 2024-04-24T08:50:43.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:43.134+0000 7ff533e00640 1 -- 172.21.15.83:0/3708362236 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff52c011d40 con 0x7ff53c104140 2024-04-24T08:50:43.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:43.134+0000 7ff533e00640 1 -- 172.21.15.83:0/3708362236 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff52c00f340 con 0x7ff53c104140 2024-04-24T08:50:43.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:43.134+0000 7ff541200640 1 -- 172.21.15.83:0/3708362236 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff53c1a22b0 con 0x7ff53c104140 2024-04-24T08:50:43.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:43.134+0000 7ff541200640 1 -- 172.21.15.83:0/3708362236 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff53c1a2800 con 0x7ff53c104140 2024-04-24T08:50:43.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:43.135+0000 7ff541200640 1 -- 172.21.15.83:0/3708362236 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff53c10b460 con 0x7ff53c104140 2024-04-24T08:50:43.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:43.137+0000 7ff533e00640 1 -- 172.21.15.83:0/3708362236 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7ff52c018020 con 0x7ff53c104140 2024-04-24T08:50:43.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:43.139+0000 7ff533e00640 1 --2- 172.21.15.83:0/3708362236 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7ff5200a3bb0 0x7ff5200a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:43.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:43.139+0000 7ff533e00640 1 -- 172.21.15.83:0/3708362236 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9237+0+0 (secure 0 0 0) 0x7ff52c0d0380 con 0x7ff53c104140 2024-04-24T08:50:43.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:43.139+0000 7ff53aa00640 1 --2- 172.21.15.83:0/3708362236 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7ff5200a3bb0 0x7ff5200a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:43.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:43.140+0000 7ff53aa00640 1 --2- 172.21.15.83:0/3708362236 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7ff5200a3bb0 0x7ff5200a6070 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff53c19b8b0 tx=0x7ff52800a4c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:50:43.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:43.143+0000 7ff533e00640 1 -- 172.21.15.83:0/3708362236 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff52c01f020 con 0x7ff53c104140 2024-04-24T08:50:43.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:50:42 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]': finished 2024-04-24T08:50:43.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:50:42 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-04-24T08:50:43.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:50:42 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]': finished 2024-04-24T08:50:43.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:50:42 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]': finished 2024-04-24T08:50:43.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:50:42 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 1]}]': finished 2024-04-24T08:50:43.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:50:42 smithi083 ceph-mon[27907]: osdmap e92: 12 total, 12 up, 12 in 2024-04-24T08:50:43.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:50:42 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/2892092471' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T08:50:43.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:43.343+0000 7ff541200640 1 -- 172.21.15.83:0/3708362236 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff53c062c00 con 0x7ff53c104140 2024-04-24T08:50:43.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:43.344+0000 7ff533e00640 1 -- 172.21.15.83:0/3708362236 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) ==== 93+0+10 (secure 0 0 0) 0x7ff52c00d040 con 0x7ff53c104140 2024-04-24T08:50:43.345 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-24T08:50:43.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:43.347+0000 7ff541200640 1 -- 172.21.15.83:0/3708362236 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7ff5200a3bb0 msgr2=0x7ff5200a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:43.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:43.347+0000 7ff541200640 1 --2- 172.21.15.83:0/3708362236 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7ff5200a3bb0 0x7ff5200a6070 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff53c19b8b0 tx=0x7ff52800a4c0 comp rx=0 tx=0).stop 2024-04-24T08:50:43.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:43.347+0000 7ff541200640 1 -- 172.21.15.83:0/3708362236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff53c104140 msgr2=0x7ff53c19a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:43.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:43.347+0000 7ff541200640 1 --2- 172.21.15.83:0/3708362236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff53c104140 0x7ff53c19a3a0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7ff52c00b450 tx=0x7ff52c00b910 comp rx=0 tx=0).stop 2024-04-24T08:50:43.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:43.347+0000 7ff541200640 1 -- 172.21.15.83:0/3708362236 shutdown_connections 2024-04-24T08:50:43.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:43.347+0000 7ff541200640 1 --2- 172.21.15.83:0/3708362236 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff53c105cf0 0x7ff53c1a1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:43.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:43.347+0000 7ff541200640 1 --2- 172.21.15.83:0/3708362236 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff53c105330 0x7ff53c19a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:43.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:43.347+0000 7ff541200640 1 --2- 172.21.15.83:0/3708362236 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7ff5200a3bb0 0x7ff5200a6070 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:43.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:43.347+0000 7ff541200640 1 --2- 172.21.15.83:0/3708362236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff53c104140 0x7ff53c19a3a0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:43.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:43.347+0000 7ff541200640 1 -- 172.21.15.83:0/3708362236 >> 172.21.15.83:0/3708362236 conn(0x7ff53c0ff6b0 msgr2=0x7ff53c108630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:50:43.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:43.347+0000 7ff541200640 1 -- 172.21.15.83:0/3708362236 shutdown_connections 2024-04-24T08:50:43.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:43.347+0000 7ff541200640 1 -- 172.21.15.83:0/3708362236 wait complete. 2024-04-24T08:50:43.368 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T08:50:43.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:50:42 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]': finished 2024-04-24T08:50:43.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:50:42 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-04-24T08:50:43.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:50:42 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]': finished 2024-04-24T08:50:43.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:50:42 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]': finished 2024-04-24T08:50:43.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:50:42 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 1]}]': finished 2024-04-24T08:50:43.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:50:42 smithi138 ceph-mon[31441]: osdmap e92: 12 total, 12 up, 12 in 2024-04-24T08:50:43.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:50:42 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/2892092471' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T08:50:43.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:50:42 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]': finished 2024-04-24T08:50:43.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:50:42 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-04-24T08:50:43.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:50:42 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]': finished 2024-04-24T08:50:43.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:50:42 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]': finished 2024-04-24T08:50:43.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:50:42 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 1]}]': finished 2024-04-24T08:50:43.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:50:42 smithi189 ceph-mon[31791]: osdmap e92: 12 total, 12 up, 12 in 2024-04-24T08:50:43.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:50:42 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/2892092471' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T08:50:43.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:43.576+0000 7f7e29000640 1 -- 172.21.15.83:0/3003721620 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e24105cf0 msgr2=0x7f7e2410c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:43.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:43.576+0000 7f7e29000640 1 --2- 172.21.15.83:0/3003721620 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e24105cf0 0x7f7e2410c580 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c009ca0 tx=0x7f7e1c02a8c0 comp rx=0 tx=0).stop 2024-04-24T08:50:43.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:43.576+0000 7f7e29000640 1 -- 172.21.15.83:0/3003721620 shutdown_connections 2024-04-24T08:50:43.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:43.576+0000 7f7e29000640 1 --2- 172.21.15.83:0/3003721620 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e24105cf0 0x7f7e2410c580 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:43.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:43.576+0000 7f7e29000640 1 --2- 172.21.15.83:0/3003721620 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7e24105330 0x7f7e241057b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:43.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:43.576+0000 7f7e29000640 1 --2- 172.21.15.83:0/3003721620 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e24104140 0x7f7e24104560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:43.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:43.576+0000 7f7e29000640 1 -- 172.21.15.83:0/3003721620 >> 172.21.15.83:0/3003721620 conn(0x7f7e240ff6b0 msgr2=0x7f7e24101af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:50:43.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:43.576+0000 7f7e29000640 1 -- 172.21.15.83:0/3003721620 shutdown_connections 2024-04-24T08:50:43.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:43.576+0000 7f7e29000640 1 -- 172.21.15.83:0/3003721620 wait complete. 2024-04-24T08:50:43.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:43.577+0000 7f7e29000640 1 Processor -- start 2024-04-24T08:50:43.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:43.577+0000 7f7e29000640 1 -- start start 2024-04-24T08:50:43.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:43.577+0000 7f7e29000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e24104140 0x7f7e2419a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:43.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:43.578+0000 7f7e29000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e24105330 0x7f7e2419a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:43.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:43.578+0000 7f7e29000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7e24105cf0 0x7f7e241a1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:43.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:43.578+0000 7f7e29000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7e2410fc80 con 0x7f7e24104140 2024-04-24T08:50:43.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:43.578+0000 7f7e29000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7e2410fb00 con 0x7f7e24105330 2024-04-24T08:50:43.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:43.578+0000 7f7e29000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7e2410fe00 con 0x7f7e24105cf0 2024-04-24T08:50:43.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:43.578+0000 7f7e23400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e24104140 0x7f7e2419a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:43.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:43.578+0000 7f7e23400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e24104140 0x7f7e2419a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48496/0 (socket says 172.21.15.83:48496) 2024-04-24T08:50:43.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:43.578+0000 7f7e23400640 1 -- 172.21.15.83:0/2571549815 learned_addr learned my addr 172.21.15.83:0/2571549815 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:50:43.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:43.578+0000 7f7e1aa00640 1 --2- 172.21.15.83:0/2571549815 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e24105330 0x7f7e2419a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:43.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:43.578+0000 7f7e23400640 1 -- 172.21.15.83:0/2571549815 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7e24105cf0 msgr2=0x7f7e241a1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:43.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:43.578+0000 7f7e23400640 1 --2- 172.21.15.83:0/2571549815 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7e24105cf0 0x7f7e241a1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:43.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:43.578+0000 7f7e23400640 1 -- 172.21.15.83:0/2571549815 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e24105330 msgr2=0x7f7e2419a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:43.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:43.578+0000 7f7e23400640 1 --2- 172.21.15.83:0/2571549815 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e24105330 0x7f7e2419a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:43.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:43.578+0000 7f7e23400640 1 -- 172.21.15.83:0/2571549815 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e241a1ef0 con 0x7f7e24104140 2024-04-24T08:50:43.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:43.579+0000 7f7e23400640 1 --2- 172.21.15.83:0/2571549815 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e24104140 0x7f7e2419a2d0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f7e1400d770 tx=0x7f7e1400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:50:43.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:43.579+0000 7f7e20c00640 1 -- 172.21.15.83:0/2571549815 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e14017070 con 0x7f7e24104140 2024-04-24T08:50:43.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:43.579+0000 7f7e29000640 1 -- 172.21.15.83:0/2571549815 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e241a21e0 con 0x7f7e24104140 2024-04-24T08:50:43.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:43.579+0000 7f7e20c00640 1 -- 172.21.15.83:0/2571549815 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7e14009c30 con 0x7f7e24104140 2024-04-24T08:50:43.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:43.579+0000 7f7e20c00640 1 -- 172.21.15.83:0/2571549815 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e14020060 con 0x7f7e24104140 2024-04-24T08:50:43.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:43.579+0000 7f7e29000640 1 -- 172.21.15.83:0/2571549815 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e241a2730 con 0x7f7e24104140 2024-04-24T08:50:43.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:43.580+0000 7f7e29000640 1 -- 172.21.15.83:0/2571549815 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e2410b460 con 0x7f7e24104140 2024-04-24T08:50:43.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:43.582+0000 7f7e20c00640 1 -- 172.21.15.83:0/2571549815 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f7e14003ca0 con 0x7f7e24104140 2024-04-24T08:50:43.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:43.583+0000 7f7e20c00640 1 --2- 172.21.15.83:0/2571549815 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f7e080a3bb0 0x7f7e080a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:43.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:43.584+0000 7f7e20c00640 1 -- 172.21.15.83:0/2571549815 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9237+0+0 (secure 0 0 0) 0x7f7e140d0360 con 0x7f7e24104140 2024-04-24T08:50:43.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:43.584+0000 7f7e1aa00640 1 --2- 172.21.15.83:0/2571549815 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f7e080a3bb0 0x7f7e080a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:43.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:43.584+0000 7f7e1aa00640 1 --2- 172.21.15.83:0/2571549815 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f7e080a3bb0 0x7f7e080a6070 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c005fd0 tx=0x7f7e0c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:50:43.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:43.588+0000 7f7e20c00640 1 -- 172.21.15.83:0/2571549815 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7e1409c8f0 con 0x7f7e24104140 2024-04-24T08:50:43.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:43.789+0000 7f7e29000640 1 -- 172.21.15.83:0/2571549815 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f7e24062c00 con 0x7f7e24104140 2024-04-24T08:50:43.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:43.790+0000 7f7e20c00640 1 -- 172.21.15.83:0/2571549815 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) ==== 104+0+11 (secure 0 0 0) 0x7f7e14009de0 con 0x7f7e24104140 2024-04-24T08:50:43.790 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-24T08:50:43.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:43.793+0000 7f7e29000640 1 -- 172.21.15.83:0/2571549815 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f7e080a3bb0 msgr2=0x7f7e080a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:43.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:43.793+0000 7f7e29000640 1 --2- 172.21.15.83:0/2571549815 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f7e080a3bb0 0x7f7e080a6070 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c005fd0 tx=0x7f7e0c005850 comp rx=0 tx=0).stop 2024-04-24T08:50:43.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:43.793+0000 7f7e29000640 1 -- 172.21.15.83:0/2571549815 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e24104140 msgr2=0x7f7e2419a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:43.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:43.793+0000 7f7e29000640 1 --2- 172.21.15.83:0/2571549815 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e24104140 0x7f7e2419a2d0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f7e1400d770 tx=0x7f7e1400dc30 comp rx=0 tx=0).stop 2024-04-24T08:50:43.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:43.793+0000 7f7e29000640 1 -- 172.21.15.83:0/2571549815 shutdown_connections 2024-04-24T08:50:43.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:43.793+0000 7f7e29000640 1 --2- 172.21.15.83:0/2571549815 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7e24105cf0 0x7f7e241a1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:43.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:43.793+0000 7f7e29000640 1 --2- 172.21.15.83:0/2571549815 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e24105330 0x7f7e2419a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:43.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:43.793+0000 7f7e29000640 1 --2- 172.21.15.83:0/2571549815 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f7e080a3bb0 0x7f7e080a6070 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:43.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:43.793+0000 7f7e29000640 1 --2- 172.21.15.83:0/2571549815 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e24104140 0x7f7e2419a2d0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:43.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:43.793+0000 7f7e29000640 1 -- 172.21.15.83:0/2571549815 >> 172.21.15.83:0/2571549815 conn(0x7f7e240ff6b0 msgr2=0x7f7e24108630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:50:43.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:43.793+0000 7f7e29000640 1 -- 172.21.15.83:0/2571549815 shutdown_connections 2024-04-24T08:50:43.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:43.793+0000 7f7e29000640 1 -- 172.21.15.83:0/2571549815 wait complete. 2024-04-24T08:50:43.814 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T08:50:44.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.023+0000 7faed9200640 1 -- 172.21.15.83:0/512027365 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faed4109df0 msgr2=0x7faed410c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:44.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.023+0000 7faed9200640 1 --2- 172.21.15.83:0/512027365 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faed4109df0 0x7faed410c280 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7faecc009ca0 tx=0x7faecc02a950 comp rx=0 tx=0).stop 2024-04-24T08:50:44.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.023+0000 7faed9200640 1 -- 172.21.15.83:0/512027365 shutdown_connections 2024-04-24T08:50:44.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.023+0000 7faed9200640 1 --2- 172.21.15.83:0/512027365 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faed4109df0 0x7faed410c280 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:44.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.023+0000 7faed9200640 1 --2- 172.21.15.83:0/512027365 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faed40749e0 0x7faed4074e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:44.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.023+0000 7faed9200640 1 --2- 172.21.15.83:0/512027365 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faed4075fc0 0x7faed4074410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:44.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.023+0000 7faed9200640 1 -- 172.21.15.83:0/512027365 >> 172.21.15.83:0/512027365 conn(0x7faed40fd4d0 msgr2=0x7faed40ff8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:50:44.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.023+0000 7faed9200640 1 -- 172.21.15.83:0/512027365 shutdown_connections 2024-04-24T08:50:44.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.023+0000 7faed9200640 1 -- 172.21.15.83:0/512027365 wait complete. 2024-04-24T08:50:44.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.024+0000 7faed9200640 1 Processor -- start 2024-04-24T08:50:44.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.024+0000 7faed9200640 1 -- start start 2024-04-24T08:50:44.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.025+0000 7faed9200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faed40749e0 0x7faed419a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:44.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.025+0000 7faed9200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faed4075fc0 0x7faed419a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:44.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.025+0000 7faed9200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faed4109df0 0x7faed41a1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:44.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.025+0000 7faed9200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7faed410fce0 con 0x7faed40749e0 2024-04-24T08:50:44.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.025+0000 7faed9200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7faed410fb60 con 0x7faed4109df0 2024-04-24T08:50:44.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.025+0000 7faed9200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7faed410fe60 con 0x7faed4075fc0 2024-04-24T08:50:44.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.025+0000 7faed3400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faed40749e0 0x7faed419a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:44.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.025+0000 7faed3400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faed40749e0 0x7faed419a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48512/0 (socket says 172.21.15.83:48512) 2024-04-24T08:50:44.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.025+0000 7faed3400640 1 -- 172.21.15.83:0/650797029 learned_addr learned my addr 172.21.15.83:0/650797029 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:50:44.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.025+0000 7faed3e00640 1 --2- 172.21.15.83:0/650797029 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faed4109df0 0x7faed41a1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:44.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.025+0000 7faed3400640 1 -- 172.21.15.83:0/650797029 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faed4075fc0 msgr2=0x7faed419a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:44.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.025+0000 7faed3400640 1 --2- 172.21.15.83:0/650797029 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faed4075fc0 0x7faed419a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:44.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.025+0000 7faed3400640 1 -- 172.21.15.83:0/650797029 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faed4109df0 msgr2=0x7faed41a1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:44.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.025+0000 7faed3400640 1 --2- 172.21.15.83:0/650797029 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faed4109df0 0x7faed41a1860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:44.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.025+0000 7faed3400640 1 -- 172.21.15.83:0/650797029 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faed41a1ed0 con 0x7faed40749e0 2024-04-24T08:50:44.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.026+0000 7faed3400640 1 --2- 172.21.15.83:0/650797029 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faed40749e0 0x7faed419a2b0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7faec400d950 tx=0x7faec400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:50:44.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.026+0000 7faed0c00640 1 -- 172.21.15.83:0/650797029 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec4017070 con 0x7faed40749e0 2024-04-24T08:50:44.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.026+0000 7faed0c00640 1 -- 172.21.15.83:0/650797029 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faec4009c30 con 0x7faed40749e0 2024-04-24T08:50:44.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.026+0000 7faed9200640 1 -- 172.21.15.83:0/650797029 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faed41a21c0 con 0x7faed40749e0 2024-04-24T08:50:44.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.026+0000 7faed0c00640 1 -- 172.21.15.83:0/650797029 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec400f620 con 0x7faed40749e0 2024-04-24T08:50:44.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.026+0000 7faed9200640 1 -- 172.21.15.83:0/650797029 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faed41a2760 con 0x7faed40749e0 2024-04-24T08:50:44.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.027+0000 7faed9200640 1 -- 172.21.15.83:0/650797029 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faed4108bb0 con 0x7faed40749e0 2024-04-24T08:50:44.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.029+0000 7faed0c00640 1 -- 172.21.15.83:0/650797029 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7faec40109a0 con 0x7faed40749e0 2024-04-24T08:50:44.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.031+0000 7faed0c00640 1 --2- 172.21.15.83:0/650797029 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7faeb80a3c30 0x7faeb80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:44.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.032+0000 7faed0c00640 1 -- 172.21.15.83:0/650797029 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9237+0+0 (secure 0 0 0) 0x7faec40d0450 con 0x7faed40749e0 2024-04-24T08:50:44.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.032+0000 7faecaa00640 1 --2- 172.21.15.83:0/650797029 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7faeb80a3c30 0x7faeb80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:44.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.032+0000 7faecaa00640 1 --2- 172.21.15.83:0/650797029 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7faeb80a3c30 0x7faeb80a60f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7faed419b7c0 tx=0x7faebc005bf0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:50:44.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.036+0000 7faed0c00640 1 -- 172.21.15.83:0/650797029 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7faec4009de0 con 0x7faed40749e0 2024-04-24T08:50:44.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.238+0000 7faed9200640 1 -- 172.21.15.83:0/650797029 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7faed4062c00 con 0x7faed40749e0 2024-04-24T08:50:44.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.239+0000 7faed0c00640 1 -- 172.21.15.83:0/650797029 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) ==== 100+0+11 (secure 0 0 0) 0x7faec409ca40 con 0x7faed40749e0 2024-04-24T08:50:44.240 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2024-04-24T08:50:44.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.242+0000 7faed9200640 1 -- 172.21.15.83:0/650797029 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7faeb80a3c30 msgr2=0x7faeb80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:44.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.242+0000 7faed9200640 1 --2- 172.21.15.83:0/650797029 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7faeb80a3c30 0x7faeb80a60f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7faed419b7c0 tx=0x7faebc005bf0 comp rx=0 tx=0).stop 2024-04-24T08:50:44.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.242+0000 7faed9200640 1 -- 172.21.15.83:0/650797029 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faed40749e0 msgr2=0x7faed419a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:44.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.242+0000 7faed9200640 1 --2- 172.21.15.83:0/650797029 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faed40749e0 0x7faed419a2b0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7faec400d950 tx=0x7faec400de10 comp rx=0 tx=0).stop 2024-04-24T08:50:44.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.242+0000 7faed9200640 1 -- 172.21.15.83:0/650797029 shutdown_connections 2024-04-24T08:50:44.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.242+0000 7faed9200640 1 --2- 172.21.15.83:0/650797029 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faed4109df0 0x7faed41a1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:44.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.242+0000 7faed9200640 1 --2- 172.21.15.83:0/650797029 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faed4075fc0 0x7faed419a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:44.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.242+0000 7faed9200640 1 --2- 172.21.15.83:0/650797029 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7faeb80a3c30 0x7faeb80a60f0 secure :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7faed419b7c0 tx=0x7faebc005bf0 comp rx=0 tx=0).stop 2024-04-24T08:50:44.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.242+0000 7faed9200640 1 --2- 172.21.15.83:0/650797029 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faed40749e0 0x7faed419a2b0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:44.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.242+0000 7faed9200640 1 -- 172.21.15.83:0/650797029 >> 172.21.15.83:0/650797029 conn(0x7faed40fd4d0 msgr2=0x7faed4105d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:50:44.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.242+0000 7faed9200640 1 -- 172.21.15.83:0/650797029 shutdown_connections 2024-04-24T08:50:44.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.242+0000 7faed9200640 1 -- 172.21.15.83:0/650797029 wait complete. 2024-04-24T08:50:44.259 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:50:43 smithi083 ceph-mon[27907]: pgmap v443: 129 pgs: 129 active+clean; 644 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-24T08:50:44.259 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:50:43 smithi083 ceph-mon[27907]: osdmap e93: 12 total, 12 up, 12 in 2024-04-24T08:50:44.259 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:50:43 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/3708362236' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T08:50:44.259 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:50:43 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/2571549815' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T08:50:44.261 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-24T08:50:44.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:50:43 smithi138 ceph-mon[31441]: pgmap v443: 129 pgs: 129 active+clean; 644 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-24T08:50:44.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:50:43 smithi138 ceph-mon[31441]: osdmap e93: 12 total, 12 up, 12 in 2024-04-24T08:50:44.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:50:43 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/3708362236' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T08:50:44.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:50:43 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/2571549815' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T08:50:44.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:50:43 smithi189 ceph-mon[31791]: pgmap v443: 129 pgs: 129 active+clean; 644 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-24T08:50:44.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:50:43 smithi189 ceph-mon[31791]: osdmap e93: 12 total, 12 up, 12 in 2024-04-24T08:50:44.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:50:43 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/3708362236' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T08:50:44.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:50:43 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/2571549815' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T08:50:44.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.460+0000 7fd1f4e00640 1 -- 172.21.15.83:0/2981591758 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd1f0100e10 msgr2=0x7fd1f0101230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:44.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.460+0000 7fd1f4e00640 1 --2- 172.21.15.83:0/2981591758 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd1f0100e10 0x7fd1f0101230 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fd1e0009900 tx=0x7fd1e002a410 comp rx=0 tx=0).stop 2024-04-24T08:50:44.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.460+0000 7fd1f4e00640 1 -- 172.21.15.83:0/2981591758 shutdown_connections 2024-04-24T08:50:44.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.460+0000 7fd1f4e00640 1 --2- 172.21.15.83:0/2981591758 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd1f0102920 0x7fd1f0109240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:44.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.460+0000 7fd1f4e00640 1 --2- 172.21.15.83:0/2981591758 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd1f0101f60 0x7fd1f01023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:44.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.460+0000 7fd1f4e00640 1 --2- 172.21.15.83:0/2981591758 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd1f0100e10 0x7fd1f0101230 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:44.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.460+0000 7fd1f4e00640 1 -- 172.21.15.83:0/2981591758 >> 172.21.15.83:0/2981591758 conn(0x7fd1f00fc3a0 msgr2=0x7fd1f00fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:50:44.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.460+0000 7fd1f4e00640 1 -- 172.21.15.83:0/2981591758 shutdown_connections 2024-04-24T08:50:44.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.460+0000 7fd1f4e00640 1 -- 172.21.15.83:0/2981591758 wait complete. 2024-04-24T08:50:44.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.461+0000 7fd1f4e00640 1 Processor -- start 2024-04-24T08:50:44.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.461+0000 7fd1f4e00640 1 -- start start 2024-04-24T08:50:44.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.461+0000 7fd1f4e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd1f0100e10 0x7fd1f010b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:44.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.462+0000 7fd1f4e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd1f0101f60 0x7fd1f0109df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:44.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.462+0000 7fd1f4e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd1f0102920 0x7fd1f010a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:44.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.462+0000 7fd1f4e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f010db10 con 0x7fd1f0101f60 2024-04-24T08:50:44.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.462+0000 7fd1f4e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f010d990 con 0x7fd1f0102920 2024-04-24T08:50:44.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.462+0000 7fd1f4e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f010dc90 con 0x7fd1f0100e10 2024-04-24T08:50:44.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.462+0000 7fd1eea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd1f0101f60 0x7fd1f0109df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:44.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.462+0000 7fd1eea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd1f0101f60 0x7fd1f0109df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48536/0 (socket says 172.21.15.83:48536) 2024-04-24T08:50:44.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.462+0000 7fd1eea00640 1 -- 172.21.15.83:0/2007936991 learned_addr learned my addr 172.21.15.83:0/2007936991 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:50:44.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.462+0000 7fd1efe00640 1 --2- 172.21.15.83:0/2007936991 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd1f0102920 0x7fd1f010a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:44.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.462+0000 7fd1eea00640 1 -- 172.21.15.83:0/2007936991 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd1f0100e10 msgr2=0x7fd1f010b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:44.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.462+0000 7fd1eea00640 1 --2- 172.21.15.83:0/2007936991 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd1f0100e10 0x7fd1f010b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:44.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.462+0000 7fd1eea00640 1 -- 172.21.15.83:0/2007936991 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd1f0102920 msgr2=0x7fd1f010a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:44.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.462+0000 7fd1eea00640 1 --2- 172.21.15.83:0/2007936991 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd1f0102920 0x7fd1f010a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:44.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.462+0000 7fd1eea00640 1 -- 172.21.15.83:0/2007936991 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1f010abc0 con 0x7fd1f0101f60 2024-04-24T08:50:44.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.463+0000 7fd1eea00640 1 --2- 172.21.15.83:0/2007936991 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd1f0101f60 0x7fd1f0109df0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc00d770 tx=0x7fd1dc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:50:44.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.463+0000 7fd1e7e00640 1 -- 172.21.15.83:0/2007936991 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1dc017070 con 0x7fd1f0101f60 2024-04-24T08:50:44.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.463+0000 7fd1e7e00640 1 -- 172.21.15.83:0/2007936991 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd1dc009c30 con 0x7fd1f0101f60 2024-04-24T08:50:44.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.463+0000 7fd1f4e00640 1 -- 172.21.15.83:0/2007936991 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1f01a2220 con 0x7fd1f0101f60 2024-04-24T08:50:44.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.463+0000 7fd1f4e00640 1 -- 172.21.15.83:0/2007936991 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1f01a2770 con 0x7fd1f0101f60 2024-04-24T08:50:44.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.463+0000 7fd1e7e00640 1 -- 172.21.15.83:0/2007936991 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1dc020060 con 0x7fd1f0101f60 2024-04-24T08:50:44.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.464+0000 7fd1f4e00640 1 -- 172.21.15.83:0/2007936991 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1f0108120 con 0x7fd1f0101f60 2024-04-24T08:50:44.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.467+0000 7fd1e7e00640 1 -- 172.21.15.83:0/2007936991 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fd1dc003ca0 con 0x7fd1f0101f60 2024-04-24T08:50:44.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.468+0000 7fd1e7e00640 1 --2- 172.21.15.83:0/2007936991 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fd1d40a3bb0 0x7fd1d40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:44.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.468+0000 7fd1e7e00640 1 -- 172.21.15.83:0/2007936991 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9237+0+0 (secure 0 0 0) 0x7fd1dc0d08d0 con 0x7fd1f0101f60 2024-04-24T08:50:44.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.469+0000 7fd1ef400640 1 --2- 172.21.15.83:0/2007936991 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fd1d40a3bb0 0x7fd1d40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:44.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.469+0000 7fd1ef400640 1 --2- 172.21.15.83:0/2007936991 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fd1d40a3bb0 0x7fd1d40a6070 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd1e0009230 tx=0x7fd1e00314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:50:44.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.471+0000 7fd1e7e00640 1 -- 172.21.15.83:0/2007936991 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd1dc009de0 con 0x7fd1f0101f60 2024-04-24T08:50:44.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.671+0000 7fd1f4e00640 1 -- 172.21.15.83:0/2007936991 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fd1f002d890 con 0x7fd1f0101f60 2024-04-24T08:50:44.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.672+0000 7fd1e7e00640 1 -- 172.21.15.83:0/2007936991 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) ==== 103+0+11 (secure 0 0 0) 0x7fd1dc09ce60 con 0x7fd1f0101f60 2024-04-24T08:50:44.673 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-24T08:50:44.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.675+0000 7fd1f4e00640 1 -- 172.21.15.83:0/2007936991 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fd1d40a3bb0 msgr2=0x7fd1d40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:44.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.675+0000 7fd1f4e00640 1 --2- 172.21.15.83:0/2007936991 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fd1d40a3bb0 0x7fd1d40a6070 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd1e0009230 tx=0x7fd1e00314a0 comp rx=0 tx=0).stop 2024-04-24T08:50:44.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.675+0000 7fd1f4e00640 1 -- 172.21.15.83:0/2007936991 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd1f0101f60 msgr2=0x7fd1f0109df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:44.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.675+0000 7fd1f4e00640 1 --2- 172.21.15.83:0/2007936991 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd1f0101f60 0x7fd1f0109df0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc00d770 tx=0x7fd1dc00dc30 comp rx=0 tx=0).stop 2024-04-24T08:50:44.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.675+0000 7fd1f4e00640 1 -- 172.21.15.83:0/2007936991 shutdown_connections 2024-04-24T08:50:44.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.675+0000 7fd1f4e00640 1 --2- 172.21.15.83:0/2007936991 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd1f0102920 0x7fd1f010a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:44.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.675+0000 7fd1f4e00640 1 --2- 172.21.15.83:0/2007936991 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fd1d40a3bb0 0x7fd1d40a6070 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:44.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.675+0000 7fd1f4e00640 1 --2- 172.21.15.83:0/2007936991 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd1f0101f60 0x7fd1f0109df0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:44.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.675+0000 7fd1f4e00640 1 --2- 172.21.15.83:0/2007936991 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd1f0100e10 0x7fd1f010b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:44.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.675+0000 7fd1f4e00640 1 -- 172.21.15.83:0/2007936991 >> 172.21.15.83:0/2007936991 conn(0x7fd1f00fc3a0 msgr2=0x7fd1f01052f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:50:44.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.675+0000 7fd1f4e00640 1 -- 172.21.15.83:0/2007936991 shutdown_connections 2024-04-24T08:50:44.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.675+0000 7fd1f4e00640 1 -- 172.21.15.83:0/2007936991 wait complete. 2024-04-24T08:50:44.694 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T08:50:44.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.891+0000 7f5da9c00640 1 -- 172.21.15.83:0/2922566017 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5da4105270 msgr2=0x7f5da41056f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:44.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.891+0000 7f5da9c00640 1 --2- 172.21.15.83:0/2922566017 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5da4105270 0x7f5da41056f0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c009900 tx=0x7f5d8c02a3f0 comp rx=0 tx=0).stop 2024-04-24T08:50:44.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.892+0000 7f5da9c00640 1 -- 172.21.15.83:0/2922566017 shutdown_connections 2024-04-24T08:50:44.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.892+0000 7f5da9c00640 1 --2- 172.21.15.83:0/2922566017 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5da4105c30 0x7f5da410c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:44.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.892+0000 7f5da9c00640 1 --2- 172.21.15.83:0/2922566017 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5da4105270 0x7f5da41056f0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:44.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.892+0000 7f5da9c00640 1 --2- 172.21.15.83:0/2922566017 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5da4104080 0x7f5da41044a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:44.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.892+0000 7f5da9c00640 1 -- 172.21.15.83:0/2922566017 >> 172.21.15.83:0/2922566017 conn(0x7f5da40ff610 msgr2=0x7f5da4101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:50:44.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.892+0000 7f5da9c00640 1 -- 172.21.15.83:0/2922566017 shutdown_connections 2024-04-24T08:50:44.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.892+0000 7f5da9c00640 1 -- 172.21.15.83:0/2922566017 wait complete. 2024-04-24T08:50:44.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.893+0000 7f5da9c00640 1 Processor -- start 2024-04-24T08:50:44.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.893+0000 7f5da9c00640 1 -- start start 2024-04-24T08:50:44.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.893+0000 7f5da9c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5da4104080 0x7f5da419a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:44.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.893+0000 7f5da9c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5da4105270 0x7f5da419a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:44.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.893+0000 7f5da9c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5da4105c30 0x7f5da41a18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:44.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.893+0000 7f5da9c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5da410fc50 con 0x7f5da4105c30 2024-04-24T08:50:44.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.893+0000 7f5da9c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5da410fad0 con 0x7f5da4105270 2024-04-24T08:50:44.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.893+0000 7f5da9c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5da410fdd0 con 0x7f5da4104080 2024-04-24T08:50:44.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.893+0000 7f5da9200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5da4105c30 0x7f5da41a18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:44.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.893+0000 7f5da9200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5da4105c30 0x7f5da41a18a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48560/0 (socket says 172.21.15.83:48560) 2024-04-24T08:50:44.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.893+0000 7f5da9200640 1 -- 172.21.15.83:0/1813069110 learned_addr learned my addr 172.21.15.83:0/1813069110 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:50:44.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.894+0000 7f5da3400640 1 --2- 172.21.15.83:0/1813069110 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5da4105270 0x7f5da419a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:44.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.894+0000 7f5da3e00640 1 --2- 172.21.15.83:0/1813069110 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5da4104080 0x7f5da419a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:44.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.894+0000 7f5da9200640 1 -- 172.21.15.83:0/1813069110 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5da4104080 msgr2=0x7f5da419a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:44.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.894+0000 7f5da9200640 1 --2- 172.21.15.83:0/1813069110 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5da4104080 0x7f5da419a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:44.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.894+0000 7f5da9200640 1 -- 172.21.15.83:0/1813069110 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5da4105270 msgr2=0x7f5da419a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:44.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.894+0000 7f5da9200640 1 --2- 172.21.15.83:0/1813069110 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5da4105270 0x7f5da419a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:44.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.894+0000 7f5da9200640 1 -- 172.21.15.83:0/1813069110 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5da41a1f10 con 0x7f5da4105c30 2024-04-24T08:50:44.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.894+0000 7f5da9200640 1 --2- 172.21.15.83:0/1813069110 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5da4105c30 0x7f5da41a18a0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c009fd0 tx=0x7f5d9c00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:50:44.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.895+0000 7f5da0c00640 1 -- 172.21.15.83:0/1813069110 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d9c011830 con 0x7f5da4105c30 2024-04-24T08:50:44.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.895+0000 7f5da9c00640 1 -- 172.21.15.83:0/1813069110 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5da41a2200 con 0x7f5da4105c30 2024-04-24T08:50:44.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.895+0000 7f5da0c00640 1 -- 172.21.15.83:0/1813069110 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5d9c016070 con 0x7f5da4105c30 2024-04-24T08:50:44.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.895+0000 7f5da0c00640 1 -- 172.21.15.83:0/1813069110 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d9c010de0 con 0x7f5da4105c30 2024-04-24T08:50:44.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.895+0000 7f5da9c00640 1 -- 172.21.15.83:0/1813069110 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5da41a2750 con 0x7f5da4105c30 2024-04-24T08:50:44.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.896+0000 7f5da9c00640 1 -- 172.21.15.83:0/1813069110 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5da410b420 con 0x7f5da4105c30 2024-04-24T08:50:44.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.898+0000 7f5da0c00640 1 -- 172.21.15.83:0/1813069110 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f5d9c0119d0 con 0x7f5da4105c30 2024-04-24T08:50:44.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.899+0000 7f5da0c00640 1 --2- 172.21.15.83:0/1813069110 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f5d880a3bb0 0x7f5d880a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:44.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.899+0000 7f5da0c00640 1 -- 172.21.15.83:0/1813069110 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9237+0+0 (secure 0 0 0) 0x7f5d9c0cfff0 con 0x7f5da4105c30 2024-04-24T08:50:44.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.900+0000 7f5da3e00640 1 --2- 172.21.15.83:0/1813069110 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f5d880a3bb0 0x7f5d880a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:44.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.900+0000 7f5da3e00640 1 --2- 172.21.15.83:0/1813069110 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f5d880a3bb0 0x7f5d880a6070 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5d9400a300 tx=0x7f5d9400a290 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:50:44.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:44.904+0000 7f5da0c00640 1 -- 172.21.15.83:0/1813069110 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5d9c09c580 con 0x7f5da4105c30 2024-04-24T08:50:45.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:45.153+0000 7f5da9c00640 1 -- 172.21.15.83:0/1813069110 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5da402d620 con 0x7f5da4105c30 2024-04-24T08:50:45.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:45.155+0000 7f5da0c00640 1 -- 172.21.15.83:0/1813069110 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11944 (secure 0 0 0) 0x7f5d9c09be80 con 0x7f5da4105c30 2024-04-24T08:50:45.156 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:50:45.156 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/358872257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":358872257},{"type":"v1","addr":"172.21.15.83:6835","nonce":358872257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3157070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3157070851},{"type":"v1","addr":"172.21.15.83:6837","nonce":3157070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T08:48:29.422576+0000","modified":"2024-04-24T08:49:04.217098+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34520,"mds_1":34493,"mds_2":34451,"mds_3":34427,"mds_4":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14658":{"gid":14658,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6839/4085470844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":4085470844},{"type":"v1","addr":"172.21.15.83:6839","nonce":4085470844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14676":{"gid":14676,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6841/2634754210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":2634754210},{"type":"v1","addr":"172.21.15.83:6841","nonce":2634754210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24382":{"gid":24382,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/1859921310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1859921310},{"type":"v1","addr":"172.21.15.189:6833","nonce":1859921310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/2241923197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2241923197},{"type":"v1","addr":"172.21.15.189:6835","nonce":2241923197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6837/3172311220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":3172311220},{"type":"v1","addr":"172.21.15.189:6837","nonce":3172311220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.189:6839/2548965336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":2548965336},{"type":"v1","addr":"172.21.15.189:6839","nonce":2548965336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.138:6833/1157967658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1157967658},{"type":"v1","addr":"172.21.15.138:6833","nonce":1157967658}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.138:6835/1909219758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1909219758},{"type":"v1","addr":"172.21.15.138:6835","nonce":1909219758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34493":{"gid":34493,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.138:6837/900566754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":900566754},{"type":"v1","addr":"172.21.15.138:6837","nonce":900566754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34520":{"gid":34520,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.138:6839/2236932393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2236932393},{"type":"v1","addr":"172.21.15.138:6839","nonce":2236932393}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34520,"qdb_cluster":[34427,24454,34520,34493,34451]},"id":1}]} 2024-04-24T08:50:45.157 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 27 2024-04-24T08:50:45.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:45.158+0000 7f5da9c00640 1 -- 172.21.15.83:0/1813069110 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f5d880a3bb0 msgr2=0x7f5d880a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:45.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:45.158+0000 7f5da9c00640 1 --2- 172.21.15.83:0/1813069110 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f5d880a3bb0 0x7f5d880a6070 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5d9400a300 tx=0x7f5d9400a290 comp rx=0 tx=0).stop 2024-04-24T08:50:45.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:45.158+0000 7f5da9c00640 1 -- 172.21.15.83:0/1813069110 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5da4105c30 msgr2=0x7f5da41a18a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:45.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:45.158+0000 7f5da9c00640 1 --2- 172.21.15.83:0/1813069110 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5da4105c30 0x7f5da41a18a0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c009fd0 tx=0x7f5d9c00b8e0 comp rx=0 tx=0).stop 2024-04-24T08:50:45.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:45.158+0000 7f5da9c00640 1 -- 172.21.15.83:0/1813069110 shutdown_connections 2024-04-24T08:50:45.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:45.158+0000 7f5da9c00640 1 --2- 172.21.15.83:0/1813069110 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f5d880a3bb0 0x7f5d880a6070 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:45.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:45.158+0000 7f5da9c00640 1 --2- 172.21.15.83:0/1813069110 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5da4105c30 0x7f5da41a18a0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:45.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:45.158+0000 7f5da9c00640 1 --2- 172.21.15.83:0/1813069110 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5da4105270 0x7f5da419a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:45.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:45.158+0000 7f5da9c00640 1 --2- 172.21.15.83:0/1813069110 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5da4104080 0x7f5da419a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:45.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:45.158+0000 7f5da9c00640 1 -- 172.21.15.83:0/1813069110 >> 172.21.15.83:0/1813069110 conn(0x7f5da40ff610 msgr2=0x7f5da41085f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:50:45.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:45.159+0000 7f5da9c00640 1 -- 172.21.15.83:0/1813069110 shutdown_connections 2024-04-24T08:50:45.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:45.159+0000 7f5da9c00640 1 -- 172.21.15.83:0/1813069110 wait complete. 2024-04-24T08:50:45.178 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T08:50:45.247 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:50:44 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/650797029' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T08:50:45.248 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:50:44 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/2007936991' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T08:50:45.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:45.381+0000 7f3251800640 1 -- 172.21.15.83:0/954518235 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f324c105270 msgr2=0x7f324c1056f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:45.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:45.381+0000 7f3251800640 1 --2- 172.21.15.83:0/954518235 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f324c105270 0x7f324c1056f0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f3234009900 tx=0x7f323402a410 comp rx=0 tx=0).stop 2024-04-24T08:50:45.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:45.381+0000 7f3251800640 1 -- 172.21.15.83:0/954518235 shutdown_connections 2024-04-24T08:50:45.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:45.381+0000 7f3251800640 1 --2- 172.21.15.83:0/954518235 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f324c105c30 0x7f324c10c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:45.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:45.381+0000 7f3251800640 1 --2- 172.21.15.83:0/954518235 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f324c105270 0x7f324c1056f0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:45.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:45.381+0000 7f3251800640 1 --2- 172.21.15.83:0/954518235 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f324c104080 0x7f324c1044a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:45.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:45.381+0000 7f3251800640 1 -- 172.21.15.83:0/954518235 >> 172.21.15.83:0/954518235 conn(0x7f324c0ff610 msgr2=0x7f324c101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:50:45.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:45.381+0000 7f3251800640 1 -- 172.21.15.83:0/954518235 shutdown_connections 2024-04-24T08:50:45.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:45.381+0000 7f3251800640 1 -- 172.21.15.83:0/954518235 wait complete. 2024-04-24T08:50:45.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:45.382+0000 7f3251800640 1 Processor -- start 2024-04-24T08:50:45.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:45.382+0000 7f3251800640 1 -- start start 2024-04-24T08:50:45.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:45.383+0000 7f3251800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f324c104080 0x7f324c19a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:45.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:45.383+0000 7f3251800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f324c105270 0x7f324c19a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:45.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:45.383+0000 7f3251800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f324c105c30 0x7f324c1a1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:45.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:45.383+0000 7f3251800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f324c10fc10 con 0x7f324c104080 2024-04-24T08:50:45.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:45.383+0000 7f3251800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f324c10fa90 con 0x7f324c105270 2024-04-24T08:50:45.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:45.383+0000 7f3251800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f324c10fd90 con 0x7f324c105c30 2024-04-24T08:50:45.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:45.383+0000 7f324be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f324c104080 0x7f324c19a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:45.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:45.383+0000 7f324be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f324c104080 0x7f324c19a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48586/0 (socket says 172.21.15.83:48586) 2024-04-24T08:50:45.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:45.383+0000 7f324be00640 1 -- 172.21.15.83:0/4170748725 learned_addr learned my addr 172.21.15.83:0/4170748725 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:50:45.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:45.383+0000 7f3243400640 1 --2- 172.21.15.83:0/4170748725 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f324c105270 0x7f324c19a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:45.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:45.383+0000 7f3250e00640 1 --2- 172.21.15.83:0/4170748725 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f324c105c30 0x7f324c1a1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:45.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:45.383+0000 7f324be00640 1 -- 172.21.15.83:0/4170748725 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f324c105c30 msgr2=0x7f324c1a1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:45.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:45.383+0000 7f324be00640 1 --2- 172.21.15.83:0/4170748725 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f324c105c30 0x7f324c1a1850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:45.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:45.383+0000 7f324be00640 1 -- 172.21.15.83:0/4170748725 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f324c105270 msgr2=0x7f324c19a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:45.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:45.383+0000 7f324be00640 1 --2- 172.21.15.83:0/4170748725 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f324c105270 0x7f324c19a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:45.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:45.383+0000 7f324be00640 1 -- 172.21.15.83:0/4170748725 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f324c1a1ec0 con 0x7f324c104080 2024-04-24T08:50:45.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:45.383+0000 7f3243400640 1 --2- 172.21.15.83:0/4170748725 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f324c105270 0x7f324c19a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:50:45.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:45.384+0000 7f324be00640 1 --2- 172.21.15.83:0/4170748725 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f324c104080 0x7f324c19a2a0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f323c00d950 tx=0x7f323c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:50:45.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:45.384+0000 7f3249600640 1 -- 172.21.15.83:0/4170748725 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f323c017070 con 0x7f324c104080 2024-04-24T08:50:45.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:45.384+0000 7f3249600640 1 -- 172.21.15.83:0/4170748725 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f323c009c30 con 0x7f324c104080 2024-04-24T08:50:45.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:45.384+0000 7f3249600640 1 -- 172.21.15.83:0/4170748725 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f323c00f620 con 0x7f324c104080 2024-04-24T08:50:45.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:45.384+0000 7f3251800640 1 -- 172.21.15.83:0/4170748725 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f324c1a21b0 con 0x7f324c104080 2024-04-24T08:50:45.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:45.384+0000 7f3251800640 1 -- 172.21.15.83:0/4170748725 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f324c1a26b0 con 0x7f324c104080 2024-04-24T08:50:45.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:45.386+0000 7f3251800640 1 -- 172.21.15.83:0/4170748725 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f324c10b420 con 0x7f324c104080 2024-04-24T08:50:45.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:45.387+0000 7f3249600640 1 -- 172.21.15.83:0/4170748725 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f323c01a070 con 0x7f324c104080 2024-04-24T08:50:45.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:45.390+0000 7f3249600640 1 --2- 172.21.15.83:0/4170748725 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f32300a3bb0 0x7f32300a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:45.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:45.390+0000 7f3249600640 1 -- 172.21.15.83:0/4170748725 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9237+0+0 (secure 0 0 0) 0x7f323c0d0370 con 0x7f324c104080 2024-04-24T08:50:45.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:45.390+0000 7f3243400640 1 --2- 172.21.15.83:0/4170748725 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f32300a3bb0 0x7f32300a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:45.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:45.391+0000 7f3243400640 1 --2- 172.21.15.83:0/4170748725 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f32300a3bb0 0x7f32300a6070 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f323402a930 tx=0x7f3234005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:50:45.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:45.394+0000 7f3249600640 1 -- 172.21.15.83:0/4170748725 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f323c009de0 con 0x7f324c104080 2024-04-24T08:50:45.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:50:44 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/650797029' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T08:50:45.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:50:44 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/2007936991' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T08:50:45.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:50:44 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/650797029' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T08:50:45.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:50:44 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/2007936991' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T08:50:45.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:45.600+0000 7f3251800640 1 -- 172.21.15.83:0/4170748725 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f324c062c30 con 0x7f324c104080 2024-04-24T08:50:45.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:45.603+0000 7f3249600640 1 -- 172.21.15.83:0/4170748725 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+24887 (secure 0 0 0) 0x7f323c09c900 con 0x7f324c104080 2024-04-24T08:50:45.604 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:50:45.604 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":93,"fsid":"bd3e039e-0215-11ef-bc93-c7b262605968","created":"2024-04-24T08:37:32.209944+0000","modified":"2024-04-24T08:50:42.968900+0000","last_up_change":"2024-04-24T08:47:17.966041+0000","last_in_change":"2024-04-24T08:46:53.835936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T08:41:56.834372+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T08:48:26.978311+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T08:48:28.352731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T08:48:32.275167+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2cd77f3a-19dd-4c6e-a199-1dcbd5faa679","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2132675555},{"type":"v1","addr":"172.21.15.83:6803","nonce":2132675555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2132675555},{"type":"v1","addr":"172.21.15.83:6805","nonce":2132675555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2132675555},{"type":"v1","addr":"172.21.15.83:6809","nonce":2132675555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2132675555},{"type":"v1","addr":"172.21.15.83:6807","nonce":2132675555}]},"public_addr":"172.21.15.83:6803/2132675555","cluster_addr":"172.21.15.83:6805/2132675555","heartbeat_back_addr":"172.21.15.83:6809/2132675555","heartbeat_front_addr":"172.21.15.83:6807/2132675555","state":["exists","up"]},{"osd":1,"uuid":"aba7f5df-2a4c-4c23-b502-d193605a5b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":3589158880},{"type":"v1","addr":"172.21.15.138:6801","nonce":3589158880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":3589158880},{"type":"v1","addr":"172.21.15.138:6803","nonce":3589158880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":3589158880},{"type":"v1","addr":"172.21.15.138:6807","nonce":3589158880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":3589158880},{"type":"v1","addr":"172.21.15.138:6805","nonce":3589158880}]},"public_addr":"172.21.15.138:6801/3589158880","cluster_addr":"172.21.15.138:6803/3589158880","heartbeat_back_addr":"172.21.15.138:6807/3589158880","heartbeat_front_addr":"172.21.15.138:6805/3589158880","state":["exists","up"]},{"osd":2,"uuid":"071bc167-2333-4420-bb58-bffbc10d7641","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":3169072688},{"type":"v1","addr":"172.21.15.189:6801","nonce":3169072688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":3169072688},{"type":"v1","addr":"172.21.15.189:6803","nonce":3169072688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":3169072688},{"type":"v1","addr":"172.21.15.189:6807","nonce":3169072688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":3169072688},{"type":"v1","addr":"172.21.15.189:6805","nonce":3169072688}]},"public_addr":"172.21.15.189:6801/3169072688","cluster_addr":"172.21.15.189:6803/3169072688","heartbeat_back_addr":"172.21.15.189:6807/3169072688","heartbeat_front_addr":"172.21.15.189:6805/3169072688","state":["exists","up"]},{"osd":3,"uuid":"f5624d78-4455-4e07-8c1a-f47c94cddb23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3131092450},{"type":"v1","addr":"172.21.15.83:6811","nonce":3131092450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3131092450},{"type":"v1","addr":"172.21.15.83:6813","nonce":3131092450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3131092450},{"type":"v1","addr":"172.21.15.83:6817","nonce":3131092450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3131092450},{"type":"v1","addr":"172.21.15.83:6815","nonce":3131092450}]},"public_addr":"172.21.15.83:6811/3131092450","cluster_addr":"172.21.15.83:6813/3131092450","heartbeat_back_addr":"172.21.15.83:6817/3131092450","heartbeat_front_addr":"172.21.15.83:6815/3131092450","state":["exists","up"]},{"osd":4,"uuid":"7b50cd14-9385-4535-a7b0-1c1da93c8548","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":823775360},{"type":"v1","addr":"172.21.15.138:6809","nonce":823775360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":823775360},{"type":"v1","addr":"172.21.15.138:6811","nonce":823775360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":823775360},{"type":"v1","addr":"172.21.15.138:6815","nonce":823775360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":823775360},{"type":"v1","addr":"172.21.15.138:6813","nonce":823775360}]},"public_addr":"172.21.15.138:6809/823775360","cluster_addr":"172.21.15.138:6811/823775360","heartbeat_back_addr":"172.21.15.138:6815/823775360","heartbeat_front_addr":"172.21.15.138:6813/823775360","state":["exists","up"]},{"osd":5,"uuid":"9c647797-35bc-449f-be84-af97d2c69c2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":1399817431},{"type":"v1","addr":"172.21.15.189:6809","nonce":1399817431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":1399817431},{"type":"v1","addr":"172.21.15.189:6811","nonce":1399817431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":1399817431},{"type":"v1","addr":"172.21.15.189:6815","nonce":1399817431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":1399817431},{"type":"v1","addr":"172.21.15.189:6813","nonce":1399817431}]},"public_addr":"172.21.15.189:6809/1399817431","cluster_addr":"172.21.15.189:6811/1399817431","heartbeat_back_addr":"172.21.15.189:6815/1399817431","heartbeat_front_addr":"172.21.15.189:6813/1399817431","state":["exists","up"]},{"osd":6,"uuid":"8f056584-16c8-4454-b445-3f780f26c196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":1611358483},{"type":"v1","addr":"172.21.15.83:6819","nonce":1611358483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1611358483},{"type":"v1","addr":"172.21.15.83:6821","nonce":1611358483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1611358483},{"type":"v1","addr":"172.21.15.83:6825","nonce":1611358483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":1611358483},{"type":"v1","addr":"172.21.15.83:6823","nonce":1611358483}]},"public_addr":"172.21.15.83:6819/1611358483","cluster_addr":"172.21.15.83:6821/1611358483","heartbeat_back_addr":"172.21.15.83:6825/1611358483","heartbeat_front_addr":"172.21.15.83:6823/1611358483","state":["exists","up"]},{"osd":7,"uuid":"7603efe3-da77-4b7f-9d03-29c4f293b6d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":3557856706},{"type":"v1","addr":"172.21.15.138:6817","nonce":3557856706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":3557856706},{"type":"v1","addr":"172.21.15.138:6819","nonce":3557856706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":3557856706},{"type":"v1","addr":"172.21.15.138:6823","nonce":3557856706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":3557856706},{"type":"v1","addr":"172.21.15.138:6821","nonce":3557856706}]},"public_addr":"172.21.15.138:6817/3557856706","cluster_addr":"172.21.15.138:6819/3557856706","heartbeat_back_addr":"172.21.15.138:6823/3557856706","heartbeat_front_addr":"172.21.15.138:6821/3557856706","state":["exists","up"]},{"osd":8,"uuid":"27d7d1b8-d6a0-417e-b8aa-596325aa1a77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":207018554},{"type":"v1","addr":"172.21.15.189:6817","nonce":207018554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":207018554},{"type":"v1","addr":"172.21.15.189:6819","nonce":207018554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":207018554},{"type":"v1","addr":"172.21.15.189:6823","nonce":207018554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":207018554},{"type":"v1","addr":"172.21.15.189:6821","nonce":207018554}]},"public_addr":"172.21.15.189:6817/207018554","cluster_addr":"172.21.15.189:6819/207018554","heartbeat_back_addr":"172.21.15.189:6823/207018554","heartbeat_front_addr":"172.21.15.189:6821/207018554","state":["exists","up"]},{"osd":9,"uuid":"baf19b98-881b-4d3a-9f72-8aa2dd73cad6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":519636751},{"type":"v1","addr":"172.21.15.83:6827","nonce":519636751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":519636751},{"type":"v1","addr":"172.21.15.83:6829","nonce":519636751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":519636751},{"type":"v1","addr":"172.21.15.83:6833","nonce":519636751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":519636751},{"type":"v1","addr":"172.21.15.83:6831","nonce":519636751}]},"public_addr":"172.21.15.83:6827/519636751","cluster_addr":"172.21.15.83:6829/519636751","heartbeat_back_addr":"172.21.15.83:6833/519636751","heartbeat_front_addr":"172.21.15.83:6831/519636751","state":["exists","up"]},{"osd":10,"uuid":"eb346717-5c07-47c8-95be-0e68fe449fdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1130872098},{"type":"v1","addr":"172.21.15.138:6825","nonce":1130872098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1130872098},{"type":"v1","addr":"172.21.15.138:6827","nonce":1130872098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1130872098},{"type":"v1","addr":"172.21.15.138:6831","nonce":1130872098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1130872098},{"type":"v1","addr":"172.21.15.138:6829","nonce":1130872098}]},"public_addr":"172.21.15.138:6825/1130872098","cluster_addr":"172.21.15.138:6827/1130872098","heartbeat_back_addr":"172.21.15.138:6831/1130872098","heartbeat_front_addr":"172.21.15.138:6829/1130872098","state":["exists","up"]},{"osd":11,"uuid":"8e37568e-981c-45b8-b2af-2f1b73898542","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":2998782690},{"type":"v1","addr":"172.21.15.189:6825","nonce":2998782690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":2998782690},{"type":"v1","addr":"172.21.15.189:6827","nonce":2998782690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":2998782690},{"type":"v1","addr":"172.21.15.189:6831","nonce":2998782690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":2998782690},{"type":"v1","addr":"172.21.15.189:6829","nonce":2998782690}]},"public_addr":"172.21.15.189:6825/2998782690","cluster_addr":"172.21.15.189:6827/2998782690","heartbeat_back_addr":"172.21.15.189:6831/2998782690","heartbeat_front_addr":"172.21.15.189:6829/2998782690","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:40:47.980103+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:41:21.970795+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:41:55.160426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:42:30.400475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:43:05.705663+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:43:40.112312+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:44:16.781289+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:44:52.151187+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:45:28.055944+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:46:04.969301+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:46:40.918306+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:47:16.886874+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"2.1d","mappings":[{"from":2,"to":1}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.2a","mappings":[{"from":3,"to":8}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.a","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:6801/820049883":"2024-04-25T08:38:40.661315+0000","172.21.15.83:0/4272874138":"2024-04-25T08:38:40.661315+0000","172.21.15.83:0/4212994335":"2024-04-25T08:38:40.661315+0000","172.21.15.83:6801/1996371091":"2024-04-25T08:38:05.800856+0000","172.21.15.83:0/1820663683":"2024-04-25T08:38:05.800856+0000","172.21.15.83:0/3802897262":"2024-04-25T08:38:05.800856+0000","172.21.15.83:6800/1996371091":"2024-04-25T08:38:05.800856+0000","172.21.15.83:6800/820049883":"2024-04-25T08:38:40.661315+0000","172.21.15.83:0/1366125049":"2024-04-25T08:38:05.800856+0000","172.21.15.83:0/340452868":"2024-04-25T08:38:40.661315+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T08:50:45.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:45.605+0000 7f3251800640 1 -- 172.21.15.83:0/4170748725 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f32300a3bb0 msgr2=0x7f32300a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:45.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:45.605+0000 7f3251800640 1 --2- 172.21.15.83:0/4170748725 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f32300a3bb0 0x7f32300a6070 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f323402a930 tx=0x7f3234005fd0 comp rx=0 tx=0).stop 2024-04-24T08:50:45.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:45.605+0000 7f3251800640 1 -- 172.21.15.83:0/4170748725 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f324c104080 msgr2=0x7f324c19a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:45.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:45.605+0000 7f3251800640 1 --2- 172.21.15.83:0/4170748725 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f324c104080 0x7f324c19a2a0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f323c00d950 tx=0x7f323c00de10 comp rx=0 tx=0).stop 2024-04-24T08:50:45.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:45.605+0000 7f3251800640 1 -- 172.21.15.83:0/4170748725 shutdown_connections 2024-04-24T08:50:45.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:45.605+0000 7f3251800640 1 --2- 172.21.15.83:0/4170748725 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f324c105c30 0x7f324c1a1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:45.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:45.605+0000 7f3251800640 1 --2- 172.21.15.83:0/4170748725 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f324c105270 0x7f324c19a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:45.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:45.605+0000 7f3251800640 1 --2- 172.21.15.83:0/4170748725 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f32300a3bb0 0x7f32300a6070 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:45.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:45.605+0000 7f3251800640 1 --2- 172.21.15.83:0/4170748725 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f324c104080 0x7f324c19a2a0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:45.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:45.605+0000 7f3251800640 1 -- 172.21.15.83:0/4170748725 >> 172.21.15.83:0/4170748725 conn(0x7f324c0ff610 msgr2=0x7f324c1085f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:50:45.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:45.606+0000 7f3251800640 1 -- 172.21.15.83:0/4170748725 shutdown_connections 2024-04-24T08:50:45.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:45.606+0000 7f3251800640 1 -- 172.21.15.83:0/4170748725 wait complete. 2024-04-24T08:50:45.625 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T08:41:56.834372+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T08:48:26.978311+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T08:48:28.352731+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-24T08:48:32.275167+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-24T08:50:45.625 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T08:50:45.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:45.834+0000 7ff32ea00640 1 -- 172.21.15.83:0/4193297495 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff328105330 msgr2=0x7ff3281057b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:45.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:45.834+0000 7ff32ea00640 1 --2- 172.21.15.83:0/4193297495 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff328105330 0x7ff3281057b0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7ff314009900 tx=0x7ff31402a3f0 comp rx=0 tx=0).stop 2024-04-24T08:50:45.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:45.834+0000 7ff32ea00640 1 -- 172.21.15.83:0/4193297495 shutdown_connections 2024-04-24T08:50:45.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:45.834+0000 7ff32ea00640 1 --2- 172.21.15.83:0/4193297495 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff328105cf0 0x7ff32810c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:45.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:45.834+0000 7ff32ea00640 1 --2- 172.21.15.83:0/4193297495 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff328105330 0x7ff3281057b0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:45.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:45.834+0000 7ff32ea00640 1 --2- 172.21.15.83:0/4193297495 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff328104140 0x7ff328104560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:45.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:45.834+0000 7ff32ea00640 1 -- 172.21.15.83:0/4193297495 >> 172.21.15.83:0/4193297495 conn(0x7ff3280ff6b0 msgr2=0x7ff328101af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:50:45.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:45.834+0000 7ff32ea00640 1 -- 172.21.15.83:0/4193297495 shutdown_connections 2024-04-24T08:50:45.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:45.834+0000 7ff32ea00640 1 -- 172.21.15.83:0/4193297495 wait complete. 2024-04-24T08:50:45.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:45.835+0000 7ff32ea00640 1 Processor -- start 2024-04-24T08:50:45.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:45.835+0000 7ff32ea00640 1 -- start start 2024-04-24T08:50:45.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:45.836+0000 7ff32ea00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff328104140 0x7ff32819a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:45.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:45.836+0000 7ff32ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff328105330 0x7ff32819a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:45.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:45.836+0000 7ff32ea00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff328105cf0 0x7ff3281a18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:45.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:45.836+0000 7ff32ea00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff32810fc80 con 0x7ff328105330 2024-04-24T08:50:45.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:45.836+0000 7ff32ea00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff32810fb00 con 0x7ff328104140 2024-04-24T08:50:45.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:45.836+0000 7ff32ea00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff32810fe00 con 0x7ff328105cf0 2024-04-24T08:50:45.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:45.836+0000 7ff32cc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff328105330 0x7ff32819a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:45.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:45.836+0000 7ff32cc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff328105330 0x7ff32819a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48602/0 (socket says 172.21.15.83:48602) 2024-04-24T08:50:45.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:45.836+0000 7ff32cc00640 1 -- 172.21.15.83:0/1603481902 learned_addr learned my addr 172.21.15.83:0/1603481902 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:50:45.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:45.836+0000 7ff32d600640 1 --2- 172.21.15.83:0/1603481902 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff328104140 0x7ff32819a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:45.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:45.836+0000 7ff32e000640 1 --2- 172.21.15.83:0/1603481902 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff328105cf0 0x7ff3281a18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:45.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:45.836+0000 7ff32cc00640 1 -- 172.21.15.83:0/1603481902 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff328105cf0 msgr2=0x7ff3281a18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:45.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:45.836+0000 7ff32cc00640 1 --2- 172.21.15.83:0/1603481902 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff328105cf0 0x7ff3281a18b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:45.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:45.836+0000 7ff32cc00640 1 -- 172.21.15.83:0/1603481902 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff328104140 msgr2=0x7ff32819a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:45.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:45.836+0000 7ff32cc00640 1 --2- 172.21.15.83:0/1603481902 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff328104140 0x7ff32819a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:45.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:45.836+0000 7ff32cc00640 1 -- 172.21.15.83:0/1603481902 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3281a1f20 con 0x7ff328105330 2024-04-24T08:50:45.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:45.837+0000 7ff32e000640 1 --2- 172.21.15.83:0/1603481902 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff328105cf0 0x7ff3281a18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:50:45.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:45.837+0000 7ff32cc00640 1 --2- 172.21.15.83:0/1603481902 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff328105330 0x7ff32819a840 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7ff314004770 tx=0x7ff3140042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:50:45.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:45.837+0000 7ff326000640 1 -- 172.21.15.83:0/1603481902 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff31403f030 con 0x7ff328105330 2024-04-24T08:50:45.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:45.837+0000 7ff326000640 1 -- 172.21.15.83:0/1603481902 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff314003cb0 con 0x7ff328105330 2024-04-24T08:50:45.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:45.837+0000 7ff32ea00640 1 -- 172.21.15.83:0/1603481902 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3281a21b0 con 0x7ff328105330 2024-04-24T08:50:45.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:45.837+0000 7ff326000640 1 -- 172.21.15.83:0/1603481902 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff314035070 con 0x7ff328105330 2024-04-24T08:50:45.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:45.837+0000 7ff32ea00640 1 -- 172.21.15.83:0/1603481902 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3281a26a0 con 0x7ff328105330 2024-04-24T08:50:45.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:45.838+0000 7ff32ea00640 1 -- 172.21.15.83:0/1603481902 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff32810b460 con 0x7ff328105330 2024-04-24T08:50:45.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:45.840+0000 7ff326000640 1 -- 172.21.15.83:0/1603481902 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7ff314042030 con 0x7ff328105330 2024-04-24T08:50:45.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:45.842+0000 7ff326000640 1 --2- 172.21.15.83:0/1603481902 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7ff3100a3bb0 0x7ff3100a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:45.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:45.842+0000 7ff326000640 1 -- 172.21.15.83:0/1603481902 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9237+0+0 (secure 0 0 0) 0x7ff3140f1490 con 0x7ff328105330 2024-04-24T08:50:45.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:45.842+0000 7ff32d600640 1 --2- 172.21.15.83:0/1603481902 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7ff3100a3bb0 0x7ff3100a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:45.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:45.843+0000 7ff32d600640 1 --2- 172.21.15.83:0/1603481902 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7ff3100a3bb0 0x7ff3100a6070 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff31c005fd0 tx=0x7ff31c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:50:45.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:45.844+0000 7ff326000640 1 -- 172.21.15.83:0/1603481902 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff3140bda80 con 0x7ff328105330 2024-04-24T08:50:46.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.057+0000 7ff32ea00640 1 -- 172.21.15.83:0/1603481902 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff32802d7b0 con 0x7ff328105330 2024-04-24T08:50:46.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.058+0000 7ff326000640 1 -- 172.21.15.83:0/1603481902 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) ==== 93+0+10 (secure 0 0 0) 0x7ff3140bd380 con 0x7ff328105330 2024-04-24T08:50:46.058 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-24T08:50:46.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.061+0000 7ff32ea00640 1 -- 172.21.15.83:0/1603481902 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7ff3100a3bb0 msgr2=0x7ff3100a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:46.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.061+0000 7ff32ea00640 1 --2- 172.21.15.83:0/1603481902 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7ff3100a3bb0 0x7ff3100a6070 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff31c005fd0 tx=0x7ff31c005850 comp rx=0 tx=0).stop 2024-04-24T08:50:46.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.061+0000 7ff32ea00640 1 -- 172.21.15.83:0/1603481902 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff328105330 msgr2=0x7ff32819a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:46.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.061+0000 7ff32ea00640 1 --2- 172.21.15.83:0/1603481902 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff328105330 0x7ff32819a840 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7ff314004770 tx=0x7ff3140042f0 comp rx=0 tx=0).stop 2024-04-24T08:50:46.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.061+0000 7ff32ea00640 1 -- 172.21.15.83:0/1603481902 shutdown_connections 2024-04-24T08:50:46.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.061+0000 7ff32ea00640 1 --2- 172.21.15.83:0/1603481902 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff328105cf0 0x7ff3281a18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:46.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.061+0000 7ff32ea00640 1 --2- 172.21.15.83:0/1603481902 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7ff3100a3bb0 0x7ff3100a6070 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:46.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.061+0000 7ff32ea00640 1 --2- 172.21.15.83:0/1603481902 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff328105330 0x7ff32819a840 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:46.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.061+0000 7ff32ea00640 1 --2- 172.21.15.83:0/1603481902 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff328104140 0x7ff32819a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:46.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.061+0000 7ff32ea00640 1 -- 172.21.15.83:0/1603481902 >> 172.21.15.83:0/1603481902 conn(0x7ff3280ff6b0 msgr2=0x7ff328108630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:50:46.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.061+0000 7ff32ea00640 1 -- 172.21.15.83:0/1603481902 shutdown_connections 2024-04-24T08:50:46.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.061+0000 7ff32ea00640 1 -- 172.21.15.83:0/1603481902 wait complete. 2024-04-24T08:50:46.080 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T08:50:46.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.273+0000 7f8f09600640 1 -- 172.21.15.83:0/1447942324 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f04104140 msgr2=0x7f8f04104560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:46.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.273+0000 7f8f09600640 1 --2- 172.21.15.83:0/1447942324 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f04104140 0x7f8f04104560 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f8ef4009900 tx=0x7f8ef402a3f0 comp rx=0 tx=0).stop 2024-04-24T08:50:46.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.273+0000 7f8f09600640 1 -- 172.21.15.83:0/1447942324 shutdown_connections 2024-04-24T08:50:46.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.273+0000 7f8f09600640 1 --2- 172.21.15.83:0/1447942324 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f04105cf0 0x7f8f0410c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:46.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.273+0000 7f8f09600640 1 --2- 172.21.15.83:0/1447942324 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8f04105330 0x7f8f041057b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:46.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.273+0000 7f8f09600640 1 --2- 172.21.15.83:0/1447942324 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f04104140 0x7f8f04104560 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:46.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.273+0000 7f8f09600640 1 -- 172.21.15.83:0/1447942324 >> 172.21.15.83:0/1447942324 conn(0x7f8f040ff6b0 msgr2=0x7f8f04101af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:50:46.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.273+0000 7f8f09600640 1 -- 172.21.15.83:0/1447942324 shutdown_connections 2024-04-24T08:50:46.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.273+0000 7f8f09600640 1 -- 172.21.15.83:0/1447942324 wait complete. 2024-04-24T08:50:46.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.274+0000 7f8f09600640 1 Processor -- start 2024-04-24T08:50:46.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.274+0000 7f8f09600640 1 -- start start 2024-04-24T08:50:46.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.275+0000 7f8f09600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f04104140 0x7f8f0419a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:46.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.275+0000 7f8f09600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8f04105330 0x7f8f0419a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:46.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.275+0000 7f8f09600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f04105cf0 0x7f8f041a18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:46.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.275+0000 7f8f09600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8f0410fc80 con 0x7f8f04104140 2024-04-24T08:50:46.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.275+0000 7f8f09600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8f0410fb00 con 0x7f8f04105cf0 2024-04-24T08:50:46.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.275+0000 7f8f09600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8f0410fe00 con 0x7f8f04105330 2024-04-24T08:50:46.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.275+0000 7f8f03e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f04104140 0x7f8f0419a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:46.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.275+0000 7f8f03e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f04104140 0x7f8f0419a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48632/0 (socket says 172.21.15.83:48632) 2024-04-24T08:50:46.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.275+0000 7f8f03e00640 1 -- 172.21.15.83:0/1433167636 learned_addr learned my addr 172.21.15.83:0/1433167636 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:50:46.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.275+0000 7f8f03400640 1 --2- 172.21.15.83:0/1433167636 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8f04105330 0x7f8f0419a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:46.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.275+0000 7f8f08c00640 1 --2- 172.21.15.83:0/1433167636 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f04105cf0 0x7f8f041a18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:46.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.275+0000 7f8f03e00640 1 -- 172.21.15.83:0/1433167636 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8f04105330 msgr2=0x7f8f0419a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:46.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.275+0000 7f8f03e00640 1 --2- 172.21.15.83:0/1433167636 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8f04105330 0x7f8f0419a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:46.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.275+0000 7f8f03e00640 1 -- 172.21.15.83:0/1433167636 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f04105cf0 msgr2=0x7f8f041a18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:46.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.275+0000 7f8f03e00640 1 --2- 172.21.15.83:0/1433167636 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f04105cf0 0x7f8f041a18a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:46.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.275+0000 7f8f03e00640 1 -- 172.21.15.83:0/1433167636 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f041a1f10 con 0x7f8f04104140 2024-04-24T08:50:46.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.276+0000 7f8f03e00640 1 --2- 172.21.15.83:0/1433167636 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f04104140 0x7f8f0419a2f0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f8ef4009230 tx=0x7f8ef4004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:50:46.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.276+0000 7f8f00c00640 1 -- 172.21.15.83:0/1433167636 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef403f030 con 0x7f8f04104140 2024-04-24T08:50:46.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.276+0000 7f8f00c00640 1 -- 172.21.15.83:0/1433167636 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8ef4003cb0 con 0x7f8f04104140 2024-04-24T08:50:46.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.276+0000 7f8f09600640 1 -- 172.21.15.83:0/1433167636 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f041a21a0 con 0x7f8f04104140 2024-04-24T08:50:46.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.276+0000 7f8f00c00640 1 -- 172.21.15.83:0/1433167636 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef4035070 con 0x7f8f04104140 2024-04-24T08:50:46.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.276+0000 7f8f09600640 1 -- 172.21.15.83:0/1433167636 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f041a2690 con 0x7f8f04104140 2024-04-24T08:50:46.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.278+0000 7f8f09600640 1 -- 172.21.15.83:0/1433167636 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f0410b460 con 0x7f8f04104140 2024-04-24T08:50:46.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.279+0000 7f8f00c00640 1 -- 172.21.15.83:0/1433167636 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f8ef4042030 con 0x7f8f04104140 2024-04-24T08:50:46.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.281+0000 7f8f00c00640 1 --2- 172.21.15.83:0/1433167636 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f8ee80a3bb0 0x7f8ee80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:46.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.281+0000 7f8f00c00640 1 -- 172.21.15.83:0/1433167636 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9237+0+0 (secure 0 0 0) 0x7f8ef40f0f10 con 0x7f8f04104140 2024-04-24T08:50:46.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.281+0000 7f8f03400640 1 --2- 172.21.15.83:0/1433167636 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f8ee80a3bb0 0x7f8ee80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:46.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.282+0000 7f8f03400640 1 --2- 172.21.15.83:0/1433167636 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f8ee80a3bb0 0x7f8ee80a6070 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f8f0419b800 tx=0x7f8ef0005cc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:50:46.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.285+0000 7f8f00c00640 1 -- 172.21.15.83:0/1433167636 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8ef40bd500 con 0x7f8f04104140 2024-04-24T08:50:46.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:50:46 smithi083 ceph-mon[27907]: pgmap v445: 129 pgs: 129 active+clean; 644 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 10 op/s; 2 B/s, 0 objects/s recovering 2024-04-24T08:50:46.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:50:46 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/1813069110' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:50:46.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:50:46 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/4170748725' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T08:50:46.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:50:46 smithi138 ceph-mon[31441]: pgmap v445: 129 pgs: 129 active+clean; 644 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 10 op/s; 2 B/s, 0 objects/s recovering 2024-04-24T08:50:46.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:50:46 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/1813069110' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:50:46.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:50:46 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/4170748725' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T08:50:46.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:50:46 smithi189 ceph-mon[31791]: pgmap v445: 129 pgs: 129 active+clean; 644 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 10 op/s; 2 B/s, 0 objects/s recovering 2024-04-24T08:50:46.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:50:46 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/1813069110' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:50:46.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:50:46 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/4170748725' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T08:50:46.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.487+0000 7f8f09600640 1 -- 172.21.15.83:0/1433167636 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f8f04062c00 con 0x7f8f04104140 2024-04-24T08:50:46.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.488+0000 7f8f00c00640 1 -- 172.21.15.83:0/1433167636 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) ==== 104+0+11 (secure 0 0 0) 0x7f8ef40bce00 con 0x7f8f04104140 2024-04-24T08:50:46.488 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-24T08:50:46.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.491+0000 7f8f09600640 1 -- 172.21.15.83:0/1433167636 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f8ee80a3bb0 msgr2=0x7f8ee80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:46.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.491+0000 7f8f09600640 1 --2- 172.21.15.83:0/1433167636 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f8ee80a3bb0 0x7f8ee80a6070 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f8f0419b800 tx=0x7f8ef0005cc0 comp rx=0 tx=0).stop 2024-04-24T08:50:46.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.491+0000 7f8f09600640 1 -- 172.21.15.83:0/1433167636 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f04104140 msgr2=0x7f8f0419a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:46.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.491+0000 7f8f09600640 1 --2- 172.21.15.83:0/1433167636 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f04104140 0x7f8f0419a2f0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f8ef4009230 tx=0x7f8ef4004340 comp rx=0 tx=0).stop 2024-04-24T08:50:46.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.491+0000 7f8f09600640 1 -- 172.21.15.83:0/1433167636 shutdown_connections 2024-04-24T08:50:46.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.491+0000 7f8f09600640 1 --2- 172.21.15.83:0/1433167636 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f04105cf0 0x7f8f041a18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:46.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.491+0000 7f8f09600640 1 --2- 172.21.15.83:0/1433167636 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8f04105330 0x7f8f0419a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:46.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.491+0000 7f8f09600640 1 --2- 172.21.15.83:0/1433167636 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f8ee80a3bb0 0x7f8ee80a6070 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:46.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.491+0000 7f8f09600640 1 --2- 172.21.15.83:0/1433167636 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f04104140 0x7f8f0419a2f0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:46.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.491+0000 7f8f09600640 1 -- 172.21.15.83:0/1433167636 >> 172.21.15.83:0/1433167636 conn(0x7f8f040ff6b0 msgr2=0x7f8f04108630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:50:46.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.491+0000 7f8f09600640 1 -- 172.21.15.83:0/1433167636 shutdown_connections 2024-04-24T08:50:46.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.491+0000 7f8f09600640 1 -- 172.21.15.83:0/1433167636 wait complete. 2024-04-24T08:50:46.510 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T08:50:46.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.716+0000 7f8eafa00640 1 -- 172.21.15.83:0/831897814 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ea810a210 msgr2=0x7f8ea8110aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:46.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.716+0000 7f8eafa00640 1 --2- 172.21.15.83:0/831897814 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ea810a210 0x7f8ea8110aa0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f8ea0009ca0 tx=0x7f8ea002a8c0 comp rx=0 tx=0).stop 2024-04-24T08:50:46.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.716+0000 7f8eafa00640 1 -- 172.21.15.83:0/831897814 shutdown_connections 2024-04-24T08:50:46.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.716+0000 7f8eafa00640 1 --2- 172.21.15.83:0/831897814 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ea810a210 0x7f8ea8110aa0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:46.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.716+0000 7f8eafa00640 1 --2- 172.21.15.83:0/831897814 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ea8109850 0x7f8ea8109cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:46.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.716+0000 7f8eafa00640 1 --2- 172.21.15.83:0/831897814 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8ea8108660 0x7f8ea8108a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:46.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.717+0000 7f8eafa00640 1 -- 172.21.15.83:0/831897814 >> 172.21.15.83:0/831897814 conn(0x7f8ea8077600 msgr2=0x7f8ea8079a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:50:46.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.717+0000 7f8eafa00640 1 -- 172.21.15.83:0/831897814 shutdown_connections 2024-04-24T08:50:46.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.717+0000 7f8eafa00640 1 -- 172.21.15.83:0/831897814 wait complete. 2024-04-24T08:50:46.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.718+0000 7f8eafa00640 1 Processor -- start 2024-04-24T08:50:46.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.718+0000 7f8eafa00640 1 -- start start 2024-04-24T08:50:46.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.718+0000 7f8eafa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ea8108660 0x7f8ea819e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:46.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.718+0000 7f8eafa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ea8109850 0x7f8ea819ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:46.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.718+0000 7f8eafa00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8ea810a210 0x7f8ea81a5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:46.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.718+0000 7f8eafa00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8ea81141e0 con 0x7f8ea8109850 2024-04-24T08:50:46.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.718+0000 7f8eafa00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8ea8114060 con 0x7f8ea8108660 2024-04-24T08:50:46.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.718+0000 7f8eafa00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8ea8114360 con 0x7f8ea810a210 2024-04-24T08:50:46.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.718+0000 7f8ea7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ea8109850 0x7f8ea819ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:46.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.718+0000 7f8ea7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ea8109850 0x7f8ea819ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48652/0 (socket says 172.21.15.83:48652) 2024-04-24T08:50:46.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.718+0000 7f8ea7e00640 1 -- 172.21.15.83:0/2105319804 learned_addr learned my addr 172.21.15.83:0/2105319804 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:50:46.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.719+0000 7f8eada00640 1 --2- 172.21.15.83:0/2105319804 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8ea810a210 0x7f8ea81a5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:46.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.719+0000 7f8ead000640 1 --2- 172.21.15.83:0/2105319804 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ea8108660 0x7f8ea819e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:46.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.719+0000 7f8ea7e00640 1 -- 172.21.15.83:0/2105319804 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8ea810a210 msgr2=0x7f8ea81a5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:46.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.719+0000 7f8ea7e00640 1 --2- 172.21.15.83:0/2105319804 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8ea810a210 0x7f8ea81a5de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:46.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.719+0000 7f8ea7e00640 1 -- 172.21.15.83:0/2105319804 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ea8108660 msgr2=0x7f8ea819e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:46.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.719+0000 7f8ea7e00640 1 --2- 172.21.15.83:0/2105319804 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ea8108660 0x7f8ea819e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:46.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.719+0000 7f8ea7e00640 1 -- 172.21.15.83:0/2105319804 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ea81a6450 con 0x7f8ea8109850 2024-04-24T08:50:46.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.719+0000 7f8ea7e00640 1 --2- 172.21.15.83:0/2105319804 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ea8109850 0x7f8ea819ed70 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f8e9800d950 tx=0x7f8e9800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:50:46.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.719+0000 7f8ea5600640 1 -- 172.21.15.83:0/2105319804 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e98017070 con 0x7f8ea8109850 2024-04-24T08:50:46.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.720+0000 7f8ea5600640 1 -- 172.21.15.83:0/2105319804 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8e98009c30 con 0x7f8ea8109850 2024-04-24T08:50:46.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.720+0000 7f8ea5600640 1 -- 172.21.15.83:0/2105319804 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e9800f620 con 0x7f8ea8109850 2024-04-24T08:50:46.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.720+0000 7f8eafa00640 1 -- 172.21.15.83:0/2105319804 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ea81a6740 con 0x7f8ea8109850 2024-04-24T08:50:46.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.720+0000 7f8eafa00640 1 -- 172.21.15.83:0/2105319804 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ea81a6c40 con 0x7f8ea8109850 2024-04-24T08:50:46.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.721+0000 7f8eafa00640 1 -- 172.21.15.83:0/2105319804 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ea810f980 con 0x7f8ea8109850 2024-04-24T08:50:46.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.723+0000 7f8ea5600640 1 -- 172.21.15.83:0/2105319804 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f8e980109a0 con 0x7f8ea8109850 2024-04-24T08:50:46.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.725+0000 7f8ea5600640 1 --2- 172.21.15.83:0/2105319804 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f8e900a3c30 0x7f8e900a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:46.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.725+0000 7f8ea5600640 1 -- 172.21.15.83:0/2105319804 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9237+0+0 (secure 0 0 0) 0x7f8e980d0450 con 0x7f8ea8109850 2024-04-24T08:50:46.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.725+0000 7f8ead000640 1 --2- 172.21.15.83:0/2105319804 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f8e900a3c30 0x7f8e900a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:46.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.726+0000 7f8ead000640 1 --2- 172.21.15.83:0/2105319804 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f8e900a3c30 0x7f8e900a60f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c005fd0 tx=0x7f8e9c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:50:46.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.730+0000 7f8ea5600640 1 -- 172.21.15.83:0/2105319804 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8e98009de0 con 0x7f8ea8109850 2024-04-24T08:50:46.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.933+0000 7f8eafa00640 1 -- 172.21.15.83:0/2105319804 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f8ea8062c30 con 0x7f8ea8109850 2024-04-24T08:50:46.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.934+0000 7f8ea5600640 1 -- 172.21.15.83:0/2105319804 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) ==== 100+0+11 (secure 0 0 0) 0x7f8e9809ca40 con 0x7f8ea8109850 2024-04-24T08:50:46.935 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2024-04-24T08:50:46.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.937+0000 7f8eafa00640 1 -- 172.21.15.83:0/2105319804 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f8e900a3c30 msgr2=0x7f8e900a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:46.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.937+0000 7f8eafa00640 1 --2- 172.21.15.83:0/2105319804 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f8e900a3c30 0x7f8e900a60f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c005fd0 tx=0x7f8e9c005850 comp rx=0 tx=0).stop 2024-04-24T08:50:46.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.937+0000 7f8eafa00640 1 -- 172.21.15.83:0/2105319804 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ea8109850 msgr2=0x7f8ea819ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:46.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.937+0000 7f8eafa00640 1 --2- 172.21.15.83:0/2105319804 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ea8109850 0x7f8ea819ed70 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f8e9800d950 tx=0x7f8e9800de10 comp rx=0 tx=0).stop 2024-04-24T08:50:46.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.937+0000 7f8eafa00640 1 -- 172.21.15.83:0/2105319804 shutdown_connections 2024-04-24T08:50:46.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.937+0000 7f8eafa00640 1 --2- 172.21.15.83:0/2105319804 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8ea810a210 0x7f8ea81a5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:46.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.937+0000 7f8eafa00640 1 --2- 172.21.15.83:0/2105319804 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f8e900a3c30 0x7f8e900a60f0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:46.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.937+0000 7f8eafa00640 1 --2- 172.21.15.83:0/2105319804 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ea8109850 0x7f8ea819ed70 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:46.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.937+0000 7f8eafa00640 1 --2- 172.21.15.83:0/2105319804 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ea8108660 0x7f8ea819e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:46.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.937+0000 7f8eafa00640 1 -- 172.21.15.83:0/2105319804 >> 172.21.15.83:0/2105319804 conn(0x7f8ea8077600 msgr2=0x7f8ea810cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:50:46.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.937+0000 7f8eafa00640 1 -- 172.21.15.83:0/2105319804 shutdown_connections 2024-04-24T08:50:46.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:46.938+0000 7f8eafa00640 1 -- 172.21.15.83:0/2105319804 wait complete. 2024-04-24T08:50:46.958 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-24T08:50:47.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:47.175+0000 7f087be00640 1 -- 172.21.15.83:0/189507933 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f087c103180 msgr2=0x7f087c103600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:47.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:47.175+0000 7f087be00640 1 --2- 172.21.15.83:0/189507933 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f087c103180 0x7f087c103600 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f0864009900 tx=0x7f086402a410 comp rx=0 tx=0).stop 2024-04-24T08:50:47.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:47.175+0000 7f087be00640 1 -- 172.21.15.83:0/189507933 shutdown_connections 2024-04-24T08:50:47.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:47.175+0000 7f087be00640 1 --2- 172.21.15.83:0/189507933 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f087c103b40 0x7f087c10a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:47.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:47.175+0000 7f087be00640 1 --2- 172.21.15.83:0/189507933 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f087c103180 0x7f087c103600 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:47.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:47.175+0000 7f087be00640 1 --2- 172.21.15.83:0/189507933 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f087c101f90 0x7f087c1023b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:47.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:47.175+0000 7f087be00640 1 -- 172.21.15.83:0/189507933 >> 172.21.15.83:0/189507933 conn(0x7f087c0fd500 msgr2=0x7f087c0ff940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:50:47.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:47.175+0000 7f087be00640 1 -- 172.21.15.83:0/189507933 shutdown_connections 2024-04-24T08:50:47.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:47.175+0000 7f087be00640 1 -- 172.21.15.83:0/189507933 wait complete. 2024-04-24T08:50:47.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:47.176+0000 7f087be00640 1 Processor -- start 2024-04-24T08:50:47.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:47.176+0000 7f087be00640 1 -- start start 2024-04-24T08:50:47.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:47.176+0000 7f087be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f087c101f90 0x7f087c075db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:47.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:47.177+0000 7f087be00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f087c103180 0x7f087c0743f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:47.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:47.177+0000 7f087be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f087c103b40 0x7f087c074930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:47.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:47.177+0000 7f087be00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f087c10dab0 con 0x7f087c103b40 2024-04-24T08:50:47.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:47.177+0000 7f087be00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f087c10d930 con 0x7f087c101f90 2024-04-24T08:50:47.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:47.177+0000 7f087be00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f087c10dc30 con 0x7f087c103180 2024-04-24T08:50:47.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:47.177+0000 7f087b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f087c103b40 0x7f087c074930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:47.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:47.177+0000 7f087b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f087c103b40 0x7f087c074930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48678/0 (socket says 172.21.15.83:48678) 2024-04-24T08:50:47.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:47.177+0000 7f087b400640 1 -- 172.21.15.83:0/4284129300 learned_addr learned my addr 172.21.15.83:0/4284129300 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:50:47.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:47.177+0000 7f087aa00640 1 --2- 172.21.15.83:0/4284129300 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f087c101f90 0x7f087c075db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:47.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:47.177+0000 7f087b400640 1 -- 172.21.15.83:0/4284129300 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f087c103180 msgr2=0x7f087c0743f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:47.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:47.177+0000 7f087b400640 1 --2- 172.21.15.83:0/4284129300 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f087c103180 0x7f087c0743f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:47.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:47.177+0000 7f087b400640 1 -- 172.21.15.83:0/4284129300 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f087c101f90 msgr2=0x7f087c075db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:47.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:47.177+0000 7f087b400640 1 --2- 172.21.15.83:0/4284129300 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f087c101f90 0x7f087c075db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:47.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:47.177+0000 7f087b400640 1 -- 172.21.15.83:0/4284129300 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f087c0751c0 con 0x7f087c103b40 2024-04-24T08:50:47.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:47.178+0000 7f087b400640 1 --2- 172.21.15.83:0/4284129300 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f087c103b40 0x7f087c074930 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f087400eaa0 tx=0x7f087400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:50:47.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:47.178+0000 7f0880a00640 1 -- 172.21.15.83:0/4284129300 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f087400cdb0 con 0x7f087c103b40 2024-04-24T08:50:47.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:47.178+0000 7f0880a00640 1 -- 172.21.15.83:0/4284129300 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0874005020 con 0x7f087c103b40 2024-04-24T08:50:47.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:47.178+0000 7f087be00640 1 -- 172.21.15.83:0/4284129300 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f087c1a2230 con 0x7f087c103b40 2024-04-24T08:50:47.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:47.178+0000 7f0880a00640 1 -- 172.21.15.83:0/4284129300 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08740116d0 con 0x7f087c103b40 2024-04-24T08:50:47.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:47.178+0000 7f087be00640 1 -- 172.21.15.83:0/4284129300 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f087c1a2780 con 0x7f087c103b40 2024-04-24T08:50:47.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:47.179+0000 7f087be00640 1 -- 172.21.15.83:0/4284129300 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f087c1092b0 con 0x7f087c103b40 2024-04-24T08:50:47.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:47.181+0000 7f0880a00640 1 -- 172.21.15.83:0/4284129300 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f08740051d0 con 0x7f087c103b40 2024-04-24T08:50:47.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:47.182+0000 7f0880a00640 1 --2- 172.21.15.83:0/4284129300 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f08600a3c30 0x7f08600a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:47.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:47.183+0000 7f0880a00640 1 -- 172.21.15.83:0/4284129300 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9237+0+0 (secure 0 0 0) 0x7f08740d0aa0 con 0x7f087c103b40 2024-04-24T08:50:47.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:47.183+0000 7f087aa00640 1 --2- 172.21.15.83:0/4284129300 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f08600a3c30 0x7f08600a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:47.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:47.183+0000 7f087aa00640 1 --2- 172.21.15.83:0/4284129300 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f08600a3c30 0x7f08600a60f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f086c005fd0 tx=0x7f086c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:50:47.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:47.187+0000 7f0880a00640 1 -- 172.21.15.83:0/4284129300 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0874016030 con 0x7f087c103b40 2024-04-24T08:50:47.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:50:47 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/1603481902' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T08:50:47.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:50:47 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/1433167636' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T08:50:47.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:50:47 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/2105319804' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T08:50:47.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:47.391+0000 7f087be00640 1 -- 172.21.15.83:0/4284129300 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f087c062c00 con 0x7f087c103b40 2024-04-24T08:50:47.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:47.392+0000 7f0880a00640 1 -- 172.21.15.83:0/4284129300 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) ==== 103+0+11 (secure 0 0 0) 0x7f087409d030 con 0x7f087c103b40 2024-04-24T08:50:47.392 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-24T08:50:47.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:47.394+0000 7f087be00640 1 -- 172.21.15.83:0/4284129300 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f08600a3c30 msgr2=0x7f08600a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:47.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:47.394+0000 7f087be00640 1 --2- 172.21.15.83:0/4284129300 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f08600a3c30 0x7f08600a60f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f086c005fd0 tx=0x7f086c005850 comp rx=0 tx=0).stop 2024-04-24T08:50:47.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:47.394+0000 7f087be00640 1 -- 172.21.15.83:0/4284129300 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f087c103b40 msgr2=0x7f087c074930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:47.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:47.394+0000 7f087be00640 1 --2- 172.21.15.83:0/4284129300 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f087c103b40 0x7f087c074930 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f087400eaa0 tx=0x7f087400ef60 comp rx=0 tx=0).stop 2024-04-24T08:50:47.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:47.394+0000 7f087be00640 1 -- 172.21.15.83:0/4284129300 shutdown_connections 2024-04-24T08:50:47.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:47.394+0000 7f087be00640 1 --2- 172.21.15.83:0/4284129300 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f08600a3c30 0x7f08600a60f0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:47.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:47.394+0000 7f087be00640 1 --2- 172.21.15.83:0/4284129300 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f087c103b40 0x7f087c074930 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:47.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:47.394+0000 7f087be00640 1 --2- 172.21.15.83:0/4284129300 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f087c103180 0x7f087c0743f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:47.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:47.394+0000 7f087be00640 1 --2- 172.21.15.83:0/4284129300 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f087c101f90 0x7f087c075db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:47.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:47.394+0000 7f087be00640 1 -- 172.21.15.83:0/4284129300 >> 172.21.15.83:0/4284129300 conn(0x7f087c0fd500 msgr2=0x7f087c106480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:50:47.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:47.395+0000 7f087be00640 1 -- 172.21.15.83:0/4284129300 shutdown_connections 2024-04-24T08:50:47.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:47.395+0000 7f087be00640 1 -- 172.21.15.83:0/4284129300 wait complete. 2024-04-24T08:50:47.419 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T08:50:47.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:50:47 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/1603481902' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T08:50:47.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:50:47 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/1433167636' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T08:50:47.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:50:47 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/2105319804' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T08:50:47.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:50:47 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/1603481902' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T08:50:47.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:50:47 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/1433167636' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T08:50:47.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:50:47 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/2105319804' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T08:50:47.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:47.623+0000 7f26e1000640 1 -- 172.21.15.83:0/370914252 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26dc105450 msgr2=0x7f26dc1058d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:47.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:47.623+0000 7f26e1000640 1 --2- 172.21.15.83:0/370914252 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26dc105450 0x7f26dc1058d0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f26c4009900 tx=0x7f26c402a410 comp rx=0 tx=0).stop 2024-04-24T08:50:47.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:47.623+0000 7f26e1000640 1 -- 172.21.15.83:0/370914252 shutdown_connections 2024-04-24T08:50:47.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:47.623+0000 7f26e1000640 1 --2- 172.21.15.83:0/370914252 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f26dc105e10 0x7f26dc10c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:47.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:47.623+0000 7f26e1000640 1 --2- 172.21.15.83:0/370914252 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26dc105450 0x7f26dc1058d0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:47.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:47.623+0000 7f26e1000640 1 --2- 172.21.15.83:0/370914252 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26dc104260 0x7f26dc104680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:47.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:47.623+0000 7f26e1000640 1 -- 172.21.15.83:0/370914252 >> 172.21.15.83:0/370914252 conn(0x7f26dc0ff7f0 msgr2=0x7f26dc101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:50:47.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:47.623+0000 7f26e1000640 1 -- 172.21.15.83:0/370914252 shutdown_connections 2024-04-24T08:50:47.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:47.623+0000 7f26e1000640 1 -- 172.21.15.83:0/370914252 wait complete. 2024-04-24T08:50:47.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:47.624+0000 7f26e1000640 1 Processor -- start 2024-04-24T08:50:47.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:47.624+0000 7f26e1000640 1 -- start start 2024-04-24T08:50:47.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:47.625+0000 7f26e1000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f26dc104260 0x7f26dc19a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:47.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:47.625+0000 7f26e1000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26dc105450 0x7f26dc19a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:47.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:47.625+0000 7f26e1000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26dc105e10 0x7f26dc1a19a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:47.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:47.625+0000 7f26e1000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f26dc10fdf0 con 0x7f26dc105450 2024-04-24T08:50:47.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:47.625+0000 7f26e1000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f26dc10fc70 con 0x7f26dc105e10 2024-04-24T08:50:47.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:47.625+0000 7f26e1000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f26dc10ff70 con 0x7f26dc104260 2024-04-24T08:50:47.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:47.625+0000 7f26d3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26dc105450 0x7f26dc19a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:47.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:47.625+0000 7f26d3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26dc105450 0x7f26dc19a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48704/0 (socket says 172.21.15.83:48704) 2024-04-24T08:50:47.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:47.625+0000 7f26d3e00640 1 -- 172.21.15.83:0/956903910 learned_addr learned my addr 172.21.15.83:0/956903910 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:50:47.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:47.625+0000 7f26da800640 1 --2- 172.21.15.83:0/956903910 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26dc105e10 0x7f26dc1a19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:47.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:47.625+0000 7f26d9e00640 1 --2- 172.21.15.83:0/956903910 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f26dc104260 0x7f26dc19a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:47.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:47.625+0000 7f26d3e00640 1 -- 172.21.15.83:0/956903910 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f26dc104260 msgr2=0x7f26dc19a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:47.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:47.625+0000 7f26d3e00640 1 --2- 172.21.15.83:0/956903910 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f26dc104260 0x7f26dc19a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:47.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:47.625+0000 7f26d3e00640 1 -- 172.21.15.83:0/956903910 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26dc105e10 msgr2=0x7f26dc1a19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:47.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:47.626+0000 7f26d3e00640 1 --2- 172.21.15.83:0/956903910 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26dc105e10 0x7f26dc1a19a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:47.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:47.626+0000 7f26d3e00640 1 -- 172.21.15.83:0/956903910 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26dc1a2010 con 0x7f26dc105450 2024-04-24T08:50:47.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:47.626+0000 7f26d3e00640 1 --2- 172.21.15.83:0/956903910 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26dc105450 0x7f26dc19a930 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f26c40314e0 tx=0x7f26c4003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:50:47.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:47.626+0000 7f26d2a00640 1 -- 172.21.15.83:0/956903910 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c403f070 con 0x7f26dc105450 2024-04-24T08:50:47.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:47.626+0000 7f26d2a00640 1 -- 172.21.15.83:0/956903910 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f26c4031e40 con 0x7f26dc105450 2024-04-24T08:50:47.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:47.626+0000 7f26e1000640 1 -- 172.21.15.83:0/956903910 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26dc1a22a0 con 0x7f26dc105450 2024-04-24T08:50:47.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:47.626+0000 7f26d2a00640 1 -- 172.21.15.83:0/956903910 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c4035030 con 0x7f26dc105450 2024-04-24T08:50:47.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:47.626+0000 7f26e1000640 1 -- 172.21.15.83:0/956903910 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26dc1a2790 con 0x7f26dc105450 2024-04-24T08:50:47.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:47.628+0000 7f26e1000640 1 -- 172.21.15.83:0/956903910 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26dc10b520 con 0x7f26dc105450 2024-04-24T08:50:47.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:47.629+0000 7f26d2a00640 1 -- 172.21.15.83:0/956903910 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f26c4042070 con 0x7f26dc105450 2024-04-24T08:50:47.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:47.631+0000 7f26d2a00640 1 --2- 172.21.15.83:0/956903910 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f26c00a3bb0 0x7f26c00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:47.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:47.631+0000 7f26d2a00640 1 -- 172.21.15.83:0/956903910 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9237+0+0 (secure 0 0 0) 0x7f26c40f1230 con 0x7f26dc105450 2024-04-24T08:50:47.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:47.631+0000 7f26d9e00640 1 --2- 172.21.15.83:0/956903910 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f26c00a3bb0 0x7f26c00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:47.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:47.632+0000 7f26d9e00640 1 --2- 172.21.15.83:0/956903910 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f26c00a3bb0 0x7f26c00a6070 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f26cc005fd0 tx=0x7f26cc005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:50:47.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:47.635+0000 7f26d2a00640 1 -- 172.21.15.83:0/956903910 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f26c40bd7c0 con 0x7f26dc105450 2024-04-24T08:50:47.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:47.873+0000 7f26e1000640 1 -- 172.21.15.83:0/956903910 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f26dc062c00 con 0x7f26dc105450 2024-04-24T08:50:47.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:47.875+0000 7f26d2a00640 1 -- 172.21.15.83:0/956903910 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11944 (secure 0 0 0) 0x7f26c40bd0c0 con 0x7f26dc105450 2024-04-24T08:50:47.875 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:50:47.875 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/358872257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":358872257},{"type":"v1","addr":"172.21.15.83:6835","nonce":358872257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3157070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3157070851},{"type":"v1","addr":"172.21.15.83:6837","nonce":3157070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T08:48:29.422576+0000","modified":"2024-04-24T08:49:04.217098+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34520,"mds_1":34493,"mds_2":34451,"mds_3":34427,"mds_4":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14658":{"gid":14658,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6839/4085470844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":4085470844},{"type":"v1","addr":"172.21.15.83:6839","nonce":4085470844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14676":{"gid":14676,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6841/2634754210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":2634754210},{"type":"v1","addr":"172.21.15.83:6841","nonce":2634754210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24382":{"gid":24382,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/1859921310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1859921310},{"type":"v1","addr":"172.21.15.189:6833","nonce":1859921310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/2241923197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2241923197},{"type":"v1","addr":"172.21.15.189:6835","nonce":2241923197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6837/3172311220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":3172311220},{"type":"v1","addr":"172.21.15.189:6837","nonce":3172311220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.189:6839/2548965336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":2548965336},{"type":"v1","addr":"172.21.15.189:6839","nonce":2548965336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.138:6833/1157967658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1157967658},{"type":"v1","addr":"172.21.15.138:6833","nonce":1157967658}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.138:6835/1909219758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1909219758},{"type":"v1","addr":"172.21.15.138:6835","nonce":1909219758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34493":{"gid":34493,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.138:6837/900566754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":900566754},{"type":"v1","addr":"172.21.15.138:6837","nonce":900566754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34520":{"gid":34520,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.138:6839/2236932393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2236932393},{"type":"v1","addr":"172.21.15.138:6839","nonce":2236932393}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34520,"qdb_cluster":[34427,24454,34520,34493,34451]},"id":1}]} 2024-04-24T08:50:47.876 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 27 2024-04-24T08:50:47.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:47.877+0000 7f26e1000640 1 -- 172.21.15.83:0/956903910 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f26c00a3bb0 msgr2=0x7f26c00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:47.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:47.877+0000 7f26e1000640 1 --2- 172.21.15.83:0/956903910 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f26c00a3bb0 0x7f26c00a6070 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f26cc005fd0 tx=0x7f26cc005850 comp rx=0 tx=0).stop 2024-04-24T08:50:47.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:47.877+0000 7f26e1000640 1 -- 172.21.15.83:0/956903910 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26dc105450 msgr2=0x7f26dc19a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:47.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:47.877+0000 7f26e1000640 1 --2- 172.21.15.83:0/956903910 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26dc105450 0x7f26dc19a930 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f26c40314e0 tx=0x7f26c4003c80 comp rx=0 tx=0).stop 2024-04-24T08:50:47.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:47.877+0000 7f26e1000640 1 -- 172.21.15.83:0/956903910 shutdown_connections 2024-04-24T08:50:47.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:47.877+0000 7f26e1000640 1 --2- 172.21.15.83:0/956903910 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26dc105e10 0x7f26dc1a19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:47.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:47.877+0000 7f26e1000640 1 --2- 172.21.15.83:0/956903910 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f26c00a3bb0 0x7f26c00a6070 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:47.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:47.877+0000 7f26e1000640 1 --2- 172.21.15.83:0/956903910 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26dc105450 0x7f26dc19a930 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:47.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:47.877+0000 7f26e1000640 1 --2- 172.21.15.83:0/956903910 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f26dc104260 0x7f26dc19a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:47.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:47.877+0000 7f26e1000640 1 -- 172.21.15.83:0/956903910 >> 172.21.15.83:0/956903910 conn(0x7f26dc0ff7f0 msgr2=0x7f26dc1086f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:50:47.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:47.878+0000 7f26e1000640 1 -- 172.21.15.83:0/956903910 shutdown_connections 2024-04-24T08:50:47.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:47.878+0000 7f26e1000640 1 -- 172.21.15.83:0/956903910 wait complete. 2024-04-24T08:50:47.898 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T08:50:48.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.097+0000 7f6fe3200640 1 -- 172.21.15.83:0/2945741822 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6fe4101ea0 msgr2=0x7f6fe41022c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:48.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.097+0000 7f6fe3200640 1 --2- 172.21.15.83:0/2945741822 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6fe4101ea0 0x7f6fe41022c0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f6fd4009900 tx=0x7f6fd402a3f0 comp rx=0 tx=0).stop 2024-04-24T08:50:48.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.097+0000 7f6fe3200640 1 -- 172.21.15.83:0/2945741822 shutdown_connections 2024-04-24T08:50:48.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.097+0000 7f6fe3200640 1 --2- 172.21.15.83:0/2945741822 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6fe41039b0 0x7f6fe410a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:48.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.097+0000 7f6fe3200640 1 --2- 172.21.15.83:0/2945741822 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6fe4102ff0 0x7f6fe4103470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:48.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.097+0000 7f6fe3200640 1 --2- 172.21.15.83:0/2945741822 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6fe4101ea0 0x7f6fe41022c0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:48.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.097+0000 7f6fe3200640 1 -- 172.21.15.83:0/2945741822 >> 172.21.15.83:0/2945741822 conn(0x7f6fe40fd410 msgr2=0x7f6fe40ff850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:50:48.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.097+0000 7f6fe3200640 1 -- 172.21.15.83:0/2945741822 shutdown_connections 2024-04-24T08:50:48.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.097+0000 7f6fe3200640 1 -- 172.21.15.83:0/2945741822 wait complete. 2024-04-24T08:50:48.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.098+0000 7f6fe3200640 1 Processor -- start 2024-04-24T08:50:48.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.098+0000 7f6fe3200640 1 -- start start 2024-04-24T08:50:48.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.098+0000 7f6fe3200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6fe4101ea0 0x7f6fe4075db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:48.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.098+0000 7f6fe3200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6fe4102ff0 0x7f6fe40743f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:48.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.098+0000 7f6fe3200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6fe41039b0 0x7f6fe4074930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:48.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.099+0000 7f6fe3200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6fe410da40 con 0x7f6fe4102ff0 2024-04-24T08:50:48.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.099+0000 7f6fe3200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6fe410d8c0 con 0x7f6fe4101ea0 2024-04-24T08:50:48.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.099+0000 7f6fe3200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6fe410dbc0 con 0x7f6fe41039b0 2024-04-24T08:50:48.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.099+0000 7f6fd9400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6fe4102ff0 0x7f6fe40743f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:48.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.099+0000 7f6fd9400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6fe4102ff0 0x7f6fe40743f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48708/0 (socket says 172.21.15.83:48708) 2024-04-24T08:50:48.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.099+0000 7f6fd9400640 1 -- 172.21.15.83:0/390013927 learned_addr learned my addr 172.21.15.83:0/390013927 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:50:48.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.099+0000 7f6fe1e00640 1 --2- 172.21.15.83:0/390013927 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6fe4101ea0 0x7f6fe4075db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:48.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.099+0000 7f6fd9400640 1 -- 172.21.15.83:0/390013927 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6fe41039b0 msgr2=0x7f6fe4074930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:48.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.099+0000 7f6fd9400640 1 --2- 172.21.15.83:0/390013927 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6fe41039b0 0x7f6fe4074930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:48.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.099+0000 7f6fd9400640 1 -- 172.21.15.83:0/390013927 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6fe4101ea0 msgr2=0x7f6fe4075db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:48.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.099+0000 7f6fd9400640 1 --2- 172.21.15.83:0/390013927 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6fe4101ea0 0x7f6fe4075db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:48.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.099+0000 7f6fd9400640 1 -- 172.21.15.83:0/390013927 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fe40751c0 con 0x7f6fe4102ff0 2024-04-24T08:50:48.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.100+0000 7f6fd9400640 1 --2- 172.21.15.83:0/390013927 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6fe4102ff0 0x7f6fe40743f0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc00c970 tx=0x7f6fcc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:50:48.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.100+0000 7f6fdb400640 1 -- 172.21.15.83:0/390013927 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fcc007c10 con 0x7f6fe4102ff0 2024-04-24T08:50:48.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.100+0000 7f6fdb400640 1 -- 172.21.15.83:0/390013927 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6fcc007db0 con 0x7f6fe4102ff0 2024-04-24T08:50:48.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.100+0000 7f6fdb400640 1 -- 172.21.15.83:0/390013927 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fcc00f870 con 0x7f6fe4102ff0 2024-04-24T08:50:48.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.100+0000 7f6fe3200640 1 -- 172.21.15.83:0/390013927 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fe41a2170 con 0x7f6fe4102ff0 2024-04-24T08:50:48.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.100+0000 7f6fe3200640 1 -- 172.21.15.83:0/390013927 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fe41a2670 con 0x7f6fe4102ff0 2024-04-24T08:50:48.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.101+0000 7f6fe3200640 1 -- 172.21.15.83:0/390013927 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fe41091b0 con 0x7f6fe4102ff0 2024-04-24T08:50:48.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.103+0000 7f6fdb400640 1 -- 172.21.15.83:0/390013927 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f6fcc00fa10 con 0x7f6fe4102ff0 2024-04-24T08:50:48.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.104+0000 7f6fdb400640 1 --2- 172.21.15.83:0/390013927 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f6fc80a3bb0 0x7f6fc80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:48.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.105+0000 7f6fdb400640 1 -- 172.21.15.83:0/390013927 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9237+0+0 (secure 0 0 0) 0x7f6fcc0d0d10 con 0x7f6fe4102ff0 2024-04-24T08:50:48.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.105+0000 7f6fe1e00640 1 --2- 172.21.15.83:0/390013927 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f6fc80a3bb0 0x7f6fc80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:48.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.105+0000 7f6fe1e00640 1 --2- 172.21.15.83:0/390013927 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f6fc80a3bb0 0x7f6fc80a6070 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6fd4000c00 tx=0x7f6fd40314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:50:48.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.108+0000 7f6fdb400640 1 -- 172.21.15.83:0/390013927 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6fcc0d6020 con 0x7f6fe4102ff0 2024-04-24T08:50:48.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:50:48 smithi083 ceph-mon[27907]: pgmap v446: 129 pgs: 129 active+clean; 644 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 8 op/s; 16 B/s, 1 keys/s, 2 objects/s recovering 2024-04-24T08:50:48.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:50:48 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/4284129300' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T08:50:48.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:50:48 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/956903910' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:50:48.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.309+0000 7f6fe3200640 1 -- 172.21.15.83:0/390013927 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6fe4062c30 con 0x7f6fe4102ff0 2024-04-24T08:50:48.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.314+0000 7f6fdb400640 1 -- 172.21.15.83:0/390013927 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+24887 (secure 0 0 0) 0x7f6fcc09d330 con 0x7f6fe4102ff0 2024-04-24T08:50:48.315 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:50:48.315 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":93,"fsid":"bd3e039e-0215-11ef-bc93-c7b262605968","created":"2024-04-24T08:37:32.209944+0000","modified":"2024-04-24T08:50:42.968900+0000","last_up_change":"2024-04-24T08:47:17.966041+0000","last_in_change":"2024-04-24T08:46:53.835936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T08:41:56.834372+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T08:48:26.978311+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T08:48:28.352731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T08:48:32.275167+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2cd77f3a-19dd-4c6e-a199-1dcbd5faa679","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2132675555},{"type":"v1","addr":"172.21.15.83:6803","nonce":2132675555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2132675555},{"type":"v1","addr":"172.21.15.83:6805","nonce":2132675555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2132675555},{"type":"v1","addr":"172.21.15.83:6809","nonce":2132675555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2132675555},{"type":"v1","addr":"172.21.15.83:6807","nonce":2132675555}]},"public_addr":"172.21.15.83:6803/2132675555","cluster_addr":"172.21.15.83:6805/2132675555","heartbeat_back_addr":"172.21.15.83:6809/2132675555","heartbeat_front_addr":"172.21.15.83:6807/2132675555","state":["exists","up"]},{"osd":1,"uuid":"aba7f5df-2a4c-4c23-b502-d193605a5b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":3589158880},{"type":"v1","addr":"172.21.15.138:6801","nonce":3589158880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":3589158880},{"type":"v1","addr":"172.21.15.138:6803","nonce":3589158880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":3589158880},{"type":"v1","addr":"172.21.15.138:6807","nonce":3589158880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":3589158880},{"type":"v1","addr":"172.21.15.138:6805","nonce":3589158880}]},"public_addr":"172.21.15.138:6801/3589158880","cluster_addr":"172.21.15.138:6803/3589158880","heartbeat_back_addr":"172.21.15.138:6807/3589158880","heartbeat_front_addr":"172.21.15.138:6805/3589158880","state":["exists","up"]},{"osd":2,"uuid":"071bc167-2333-4420-bb58-bffbc10d7641","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":3169072688},{"type":"v1","addr":"172.21.15.189:6801","nonce":3169072688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":3169072688},{"type":"v1","addr":"172.21.15.189:6803","nonce":3169072688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":3169072688},{"type":"v1","addr":"172.21.15.189:6807","nonce":3169072688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":3169072688},{"type":"v1","addr":"172.21.15.189:6805","nonce":3169072688}]},"public_addr":"172.21.15.189:6801/3169072688","cluster_addr":"172.21.15.189:6803/3169072688","heartbeat_back_addr":"172.21.15.189:6807/3169072688","heartbeat_front_addr":"172.21.15.189:6805/3169072688","state":["exists","up"]},{"osd":3,"uuid":"f5624d78-4455-4e07-8c1a-f47c94cddb23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3131092450},{"type":"v1","addr":"172.21.15.83:6811","nonce":3131092450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3131092450},{"type":"v1","addr":"172.21.15.83:6813","nonce":3131092450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3131092450},{"type":"v1","addr":"172.21.15.83:6817","nonce":3131092450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3131092450},{"type":"v1","addr":"172.21.15.83:6815","nonce":3131092450}]},"public_addr":"172.21.15.83:6811/3131092450","cluster_addr":"172.21.15.83:6813/3131092450","heartbeat_back_addr":"172.21.15.83:6817/3131092450","heartbeat_front_addr":"172.21.15.83:6815/3131092450","state":["exists","up"]},{"osd":4,"uuid":"7b50cd14-9385-4535-a7b0-1c1da93c8548","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":823775360},{"type":"v1","addr":"172.21.15.138:6809","nonce":823775360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":823775360},{"type":"v1","addr":"172.21.15.138:6811","nonce":823775360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":823775360},{"type":"v1","addr":"172.21.15.138:6815","nonce":823775360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":823775360},{"type":"v1","addr":"172.21.15.138:6813","nonce":823775360}]},"public_addr":"172.21.15.138:6809/823775360","cluster_addr":"172.21.15.138:6811/823775360","heartbeat_back_addr":"172.21.15.138:6815/823775360","heartbeat_front_addr":"172.21.15.138:6813/823775360","state":["exists","up"]},{"osd":5,"uuid":"9c647797-35bc-449f-be84-af97d2c69c2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":1399817431},{"type":"v1","addr":"172.21.15.189:6809","nonce":1399817431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":1399817431},{"type":"v1","addr":"172.21.15.189:6811","nonce":1399817431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":1399817431},{"type":"v1","addr":"172.21.15.189:6815","nonce":1399817431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":1399817431},{"type":"v1","addr":"172.21.15.189:6813","nonce":1399817431}]},"public_addr":"172.21.15.189:6809/1399817431","cluster_addr":"172.21.15.189:6811/1399817431","heartbeat_back_addr":"172.21.15.189:6815/1399817431","heartbeat_front_addr":"172.21.15.189:6813/1399817431","state":["exists","up"]},{"osd":6,"uuid":"8f056584-16c8-4454-b445-3f780f26c196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":1611358483},{"type":"v1","addr":"172.21.15.83:6819","nonce":1611358483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1611358483},{"type":"v1","addr":"172.21.15.83:6821","nonce":1611358483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1611358483},{"type":"v1","addr":"172.21.15.83:6825","nonce":1611358483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":1611358483},{"type":"v1","addr":"172.21.15.83:6823","nonce":1611358483}]},"public_addr":"172.21.15.83:6819/1611358483","cluster_addr":"172.21.15.83:6821/1611358483","heartbeat_back_addr":"172.21.15.83:6825/1611358483","heartbeat_front_addr":"172.21.15.83:6823/1611358483","state":["exists","up"]},{"osd":7,"uuid":"7603efe3-da77-4b7f-9d03-29c4f293b6d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":3557856706},{"type":"v1","addr":"172.21.15.138:6817","nonce":3557856706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":3557856706},{"type":"v1","addr":"172.21.15.138:6819","nonce":3557856706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":3557856706},{"type":"v1","addr":"172.21.15.138:6823","nonce":3557856706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":3557856706},{"type":"v1","addr":"172.21.15.138:6821","nonce":3557856706}]},"public_addr":"172.21.15.138:6817/3557856706","cluster_addr":"172.21.15.138:6819/3557856706","heartbeat_back_addr":"172.21.15.138:6823/3557856706","heartbeat_front_addr":"172.21.15.138:6821/3557856706","state":["exists","up"]},{"osd":8,"uuid":"27d7d1b8-d6a0-417e-b8aa-596325aa1a77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":207018554},{"type":"v1","addr":"172.21.15.189:6817","nonce":207018554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":207018554},{"type":"v1","addr":"172.21.15.189:6819","nonce":207018554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":207018554},{"type":"v1","addr":"172.21.15.189:6823","nonce":207018554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":207018554},{"type":"v1","addr":"172.21.15.189:6821","nonce":207018554}]},"public_addr":"172.21.15.189:6817/207018554","cluster_addr":"172.21.15.189:6819/207018554","heartbeat_back_addr":"172.21.15.189:6823/207018554","heartbeat_front_addr":"172.21.15.189:6821/207018554","state":["exists","up"]},{"osd":9,"uuid":"baf19b98-881b-4d3a-9f72-8aa2dd73cad6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":519636751},{"type":"v1","addr":"172.21.15.83:6827","nonce":519636751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":519636751},{"type":"v1","addr":"172.21.15.83:6829","nonce":519636751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":519636751},{"type":"v1","addr":"172.21.15.83:6833","nonce":519636751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":519636751},{"type":"v1","addr":"172.21.15.83:6831","nonce":519636751}]},"public_addr":"172.21.15.83:6827/519636751","cluster_addr":"172.21.15.83:6829/519636751","heartbeat_back_addr":"172.21.15.83:6833/519636751","heartbeat_front_addr":"172.21.15.83:6831/519636751","state":["exists","up"]},{"osd":10,"uuid":"eb346717-5c07-47c8-95be-0e68fe449fdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1130872098},{"type":"v1","addr":"172.21.15.138:6825","nonce":1130872098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1130872098},{"type":"v1","addr":"172.21.15.138:6827","nonce":1130872098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1130872098},{"type":"v1","addr":"172.21.15.138:6831","nonce":1130872098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1130872098},{"type":"v1","addr":"172.21.15.138:6829","nonce":1130872098}]},"public_addr":"172.21.15.138:6825/1130872098","cluster_addr":"172.21.15.138:6827/1130872098","heartbeat_back_addr":"172.21.15.138:6831/1130872098","heartbeat_front_addr":"172.21.15.138:6829/1130872098","state":["exists","up"]},{"osd":11,"uuid":"8e37568e-981c-45b8-b2af-2f1b73898542","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":2998782690},{"type":"v1","addr":"172.21.15.189:6825","nonce":2998782690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":2998782690},{"type":"v1","addr":"172.21.15.189:6827","nonce":2998782690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":2998782690},{"type":"v1","addr":"172.21.15.189:6831","nonce":2998782690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":2998782690},{"type":"v1","addr":"172.21.15.189:6829","nonce":2998782690}]},"public_addr":"172.21.15.189:6825/2998782690","cluster_addr":"172.21.15.189:6827/2998782690","heartbeat_back_addr":"172.21.15.189:6831/2998782690","heartbeat_front_addr":"172.21.15.189:6829/2998782690","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:40:47.980103+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:41:21.970795+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:41:55.160426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:42:30.400475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:43:05.705663+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:43:40.112312+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:44:16.781289+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:44:52.151187+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:45:28.055944+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:46:04.969301+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:46:40.918306+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:47:16.886874+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"2.1d","mappings":[{"from":2,"to":1}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.2a","mappings":[{"from":3,"to":8}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.a","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:6801/820049883":"2024-04-25T08:38:40.661315+0000","172.21.15.83:0/4272874138":"2024-04-25T08:38:40.661315+0000","172.21.15.83:0/4212994335":"2024-04-25T08:38:40.661315+0000","172.21.15.83:6801/1996371091":"2024-04-25T08:38:05.800856+0000","172.21.15.83:0/1820663683":"2024-04-25T08:38:05.800856+0000","172.21.15.83:0/3802897262":"2024-04-25T08:38:05.800856+0000","172.21.15.83:6800/1996371091":"2024-04-25T08:38:05.800856+0000","172.21.15.83:6800/820049883":"2024-04-25T08:38:40.661315+0000","172.21.15.83:0/1366125049":"2024-04-25T08:38:05.800856+0000","172.21.15.83:0/340452868":"2024-04-25T08:38:40.661315+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T08:50:48.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.316+0000 7f6fe3200640 1 -- 172.21.15.83:0/390013927 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f6fc80a3bb0 msgr2=0x7f6fc80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:48.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.316+0000 7f6fe3200640 1 --2- 172.21.15.83:0/390013927 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f6fc80a3bb0 0x7f6fc80a6070 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6fd4000c00 tx=0x7f6fd40314a0 comp rx=0 tx=0).stop 2024-04-24T08:50:48.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.316+0000 7f6fe3200640 1 -- 172.21.15.83:0/390013927 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6fe4102ff0 msgr2=0x7f6fe40743f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:48.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.316+0000 7f6fe3200640 1 --2- 172.21.15.83:0/390013927 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6fe4102ff0 0x7f6fe40743f0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc00c970 tx=0x7f6fcc00ce30 comp rx=0 tx=0).stop 2024-04-24T08:50:48.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.317+0000 7f6fe3200640 1 -- 172.21.15.83:0/390013927 shutdown_connections 2024-04-24T08:50:48.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.317+0000 7f6fe3200640 1 --2- 172.21.15.83:0/390013927 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6fe41039b0 0x7f6fe4074930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:48.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.317+0000 7f6fe3200640 1 --2- 172.21.15.83:0/390013927 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f6fc80a3bb0 0x7f6fc80a6070 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:48.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.317+0000 7f6fe3200640 1 --2- 172.21.15.83:0/390013927 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6fe4102ff0 0x7f6fe40743f0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:48.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.317+0000 7f6fe3200640 1 --2- 172.21.15.83:0/390013927 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6fe4101ea0 0x7f6fe4075db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:48.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.317+0000 7f6fe3200640 1 -- 172.21.15.83:0/390013927 >> 172.21.15.83:0/390013927 conn(0x7f6fe40fd410 msgr2=0x7f6fe4106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:50:48.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.317+0000 7f6fe3200640 1 -- 172.21.15.83:0/390013927 shutdown_connections 2024-04-24T08:50:48.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.317+0000 7f6fe3200640 1 -- 172.21.15.83:0/390013927 wait complete. 2024-04-24T08:50:48.337 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T08:50:48.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:50:48 smithi138 ceph-mon[31441]: pgmap v446: 129 pgs: 129 active+clean; 644 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 8 op/s; 16 B/s, 1 keys/s, 2 objects/s recovering 2024-04-24T08:50:48.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:50:48 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/4284129300' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T08:50:48.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:50:48 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/956903910' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:50:48.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:50:48 smithi189 ceph-mon[31791]: pgmap v446: 129 pgs: 129 active+clean; 644 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 8 op/s; 16 B/s, 1 keys/s, 2 objects/s recovering 2024-04-24T08:50:48.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:50:48 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/4284129300' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T08:50:48.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:50:48 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/956903910' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:50:48.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.538+0000 7fb7c2e00640 1 -- 172.21.15.83:0/1394320854 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7bc105c30 msgr2=0x7fb7bc10c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:48.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.538+0000 7fb7c2e00640 1 --2- 172.21.15.83:0/1394320854 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7bc105c30 0x7fb7bc10c4c0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fb7b4009ca0 tx=0x7fb7b402a8c0 comp rx=0 tx=0).stop 2024-04-24T08:50:48.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.538+0000 7fb7c2e00640 1 -- 172.21.15.83:0/1394320854 shutdown_connections 2024-04-24T08:50:48.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.538+0000 7fb7c2e00640 1 --2- 172.21.15.83:0/1394320854 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7bc105c30 0x7fb7bc10c4c0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:48.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.538+0000 7fb7c2e00640 1 --2- 172.21.15.83:0/1394320854 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb7bc105270 0x7fb7bc1056f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:48.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.538+0000 7fb7c2e00640 1 --2- 172.21.15.83:0/1394320854 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb7bc104080 0x7fb7bc1044a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:48.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.538+0000 7fb7c2e00640 1 -- 172.21.15.83:0/1394320854 >> 172.21.15.83:0/1394320854 conn(0x7fb7bc0ff610 msgr2=0x7fb7bc101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:50:48.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.538+0000 7fb7c2e00640 1 -- 172.21.15.83:0/1394320854 shutdown_connections 2024-04-24T08:50:48.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.538+0000 7fb7c2e00640 1 -- 172.21.15.83:0/1394320854 wait complete. 2024-04-24T08:50:48.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.539+0000 7fb7c2e00640 1 Processor -- start 2024-04-24T08:50:48.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.539+0000 7fb7c2e00640 1 -- start start 2024-04-24T08:50:48.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.540+0000 7fb7c2e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb7bc104080 0x7fb7bc19a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:48.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.540+0000 7fb7c2e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb7bc105270 0x7fb7bc19a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:48.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.540+0000 7fb7c2e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7bc105c30 0x7fb7bc1a1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:48.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.540+0000 7fb7c2e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb7bc10fc20 con 0x7fb7bc105c30 2024-04-24T08:50:48.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.540+0000 7fb7c2e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb7bc10faa0 con 0x7fb7bc104080 2024-04-24T08:50:48.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.540+0000 7fb7c2e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb7bc10fda0 con 0x7fb7bc105270 2024-04-24T08:50:48.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.540+0000 7fb7c2400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7bc105c30 0x7fb7bc1a1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:48.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.540+0000 7fb7c2400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7bc105c30 0x7fb7bc1a1880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48726/0 (socket says 172.21.15.83:48726) 2024-04-24T08:50:48.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.540+0000 7fb7c2400640 1 -- 172.21.15.83:0/1757211451 learned_addr learned my addr 172.21.15.83:0/1757211451 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:50:48.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.540+0000 7fb7c1a00640 1 --2- 172.21.15.83:0/1757211451 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb7bc104080 0x7fb7bc19a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:48.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.541+0000 7fb7c1000640 1 --2- 172.21.15.83:0/1757211451 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb7bc105270 0x7fb7bc19a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:48.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.541+0000 7fb7c2400640 1 -- 172.21.15.83:0/1757211451 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb7bc105270 msgr2=0x7fb7bc19a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:48.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.541+0000 7fb7c2400640 1 --2- 172.21.15.83:0/1757211451 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb7bc105270 0x7fb7bc19a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:48.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.541+0000 7fb7c2400640 1 -- 172.21.15.83:0/1757211451 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb7bc104080 msgr2=0x7fb7bc19a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:48.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.541+0000 7fb7c2400640 1 --2- 172.21.15.83:0/1757211451 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb7bc104080 0x7fb7bc19a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:48.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.541+0000 7fb7c2400640 1 -- 172.21.15.83:0/1757211451 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7bc1a1ef0 con 0x7fb7bc105c30 2024-04-24T08:50:48.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.541+0000 7fb7c2400640 1 --2- 172.21.15.83:0/1757211451 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7bc105c30 0x7fb7bc1a1880 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fb7b4009dc0 tx=0x7fb7b4003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:50:48.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.541+0000 7fb7ba000640 1 -- 172.21.15.83:0/1757211451 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7b4004040 con 0x7fb7bc105c30 2024-04-24T08:50:48.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.541+0000 7fb7ba000640 1 -- 172.21.15.83:0/1757211451 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb7b4004690 con 0x7fb7bc105c30 2024-04-24T08:50:48.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.541+0000 7fb7c2e00640 1 -- 172.21.15.83:0/1757211451 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7bc1a2180 con 0x7fb7bc105c30 2024-04-24T08:50:48.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.541+0000 7fb7ba000640 1 -- 172.21.15.83:0/1757211451 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7b402f4c0 con 0x7fb7bc105c30 2024-04-24T08:50:48.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.542+0000 7fb7c2e00640 1 -- 172.21.15.83:0/1757211451 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7bc1a26d0 con 0x7fb7bc105c30 2024-04-24T08:50:48.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.543+0000 7fb7c2e00640 1 -- 172.21.15.83:0/1757211451 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7bc10b420 con 0x7fb7bc105c30 2024-04-24T08:50:48.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.545+0000 7fb7ba000640 1 -- 172.21.15.83:0/1757211451 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fb7b40305c0 con 0x7fb7bc105c30 2024-04-24T08:50:48.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.546+0000 7fb7ba000640 1 --2- 172.21.15.83:0/1757211451 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fb7a40a3bb0 0x7fb7a40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:48.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.546+0000 7fb7ba000640 1 -- 172.21.15.83:0/1757211451 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9237+0+0 (secure 0 0 0) 0x7fb7b40f2020 con 0x7fb7bc105c30 2024-04-24T08:50:48.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.546+0000 7fb7c1a00640 1 --2- 172.21.15.83:0/1757211451 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fb7a40a3bb0 0x7fb7a40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:48.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.547+0000 7fb7c1a00640 1 --2- 172.21.15.83:0/1757211451 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fb7a40a3bb0 0x7fb7a40a6070 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc1050d0 tx=0x7fb7b0009110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:50:48.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.550+0000 7fb7ba000640 1 -- 172.21.15.83:0/1757211451 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb7b40be610 con 0x7fb7bc105c30 2024-04-24T08:50:48.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.773+0000 7fb7c2e00640 1 -- 172.21.15.83:0/1757211451 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb7bc062c00 con 0x7fb7bc105c30 2024-04-24T08:50:48.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.775+0000 7fb7ba000640 1 -- 172.21.15.83:0/1757211451 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11944 (secure 0 0 0) 0x7fb7b40bdf10 con 0x7fb7bc105c30 2024-04-24T08:50:48.776 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:50:48.776 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/358872257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":358872257},{"type":"v1","addr":"172.21.15.83:6835","nonce":358872257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3157070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3157070851},{"type":"v1","addr":"172.21.15.83:6837","nonce":3157070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T08:48:29.422576+0000","modified":"2024-04-24T08:49:04.217098+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34520,"mds_1":34493,"mds_2":34451,"mds_3":34427,"mds_4":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14658":{"gid":14658,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6839/4085470844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":4085470844},{"type":"v1","addr":"172.21.15.83:6839","nonce":4085470844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14676":{"gid":14676,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6841/2634754210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":2634754210},{"type":"v1","addr":"172.21.15.83:6841","nonce":2634754210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24382":{"gid":24382,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/1859921310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1859921310},{"type":"v1","addr":"172.21.15.189:6833","nonce":1859921310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/2241923197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2241923197},{"type":"v1","addr":"172.21.15.189:6835","nonce":2241923197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6837/3172311220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":3172311220},{"type":"v1","addr":"172.21.15.189:6837","nonce":3172311220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.189:6839/2548965336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":2548965336},{"type":"v1","addr":"172.21.15.189:6839","nonce":2548965336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.138:6833/1157967658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1157967658},{"type":"v1","addr":"172.21.15.138:6833","nonce":1157967658}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.138:6835/1909219758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1909219758},{"type":"v1","addr":"172.21.15.138:6835","nonce":1909219758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34493":{"gid":34493,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.138:6837/900566754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":900566754},{"type":"v1","addr":"172.21.15.138:6837","nonce":900566754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34520":{"gid":34520,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.138:6839/2236932393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2236932393},{"type":"v1","addr":"172.21.15.138:6839","nonce":2236932393}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34520,"qdb_cluster":[34427,24454,34520,34493,34451]},"id":1}]} 2024-04-24T08:50:48.776 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 27 2024-04-24T08:50:48.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.778+0000 7fb7c2e00640 1 -- 172.21.15.83:0/1757211451 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fb7a40a3bb0 msgr2=0x7fb7a40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:48.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.778+0000 7fb7c2e00640 1 --2- 172.21.15.83:0/1757211451 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fb7a40a3bb0 0x7fb7a40a6070 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc1050d0 tx=0x7fb7b0009110 comp rx=0 tx=0).stop 2024-04-24T08:50:48.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.778+0000 7fb7c2e00640 1 -- 172.21.15.83:0/1757211451 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7bc105c30 msgr2=0x7fb7bc1a1880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:48.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.778+0000 7fb7c2e00640 1 --2- 172.21.15.83:0/1757211451 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7bc105c30 0x7fb7bc1a1880 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fb7b4009dc0 tx=0x7fb7b4003ba0 comp rx=0 tx=0).stop 2024-04-24T08:50:48.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.778+0000 7fb7c2e00640 1 -- 172.21.15.83:0/1757211451 shutdown_connections 2024-04-24T08:50:48.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.778+0000 7fb7c2e00640 1 --2- 172.21.15.83:0/1757211451 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fb7a40a3bb0 0x7fb7a40a6070 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:48.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.778+0000 7fb7c2e00640 1 --2- 172.21.15.83:0/1757211451 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7bc105c30 0x7fb7bc1a1880 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:48.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.778+0000 7fb7c2e00640 1 --2- 172.21.15.83:0/1757211451 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb7bc105270 0x7fb7bc19a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:48.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.778+0000 7fb7c2e00640 1 --2- 172.21.15.83:0/1757211451 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb7bc104080 0x7fb7bc19a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:48.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.778+0000 7fb7c2e00640 1 -- 172.21.15.83:0/1757211451 >> 172.21.15.83:0/1757211451 conn(0x7fb7bc0ff610 msgr2=0x7fb7bc1085f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:50:48.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.778+0000 7fb7c2e00640 1 -- 172.21.15.83:0/1757211451 shutdown_connections 2024-04-24T08:50:48.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.778+0000 7fb7c2e00640 1 -- 172.21.15.83:0/1757211451 wait complete. 2024-04-24T08:50:48.797 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-24T08:48:29.422576+0000', 'modified': '2024-04-24T08:49:04.217098+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 34520, 'mds_1': 34493, 'mds_2': 34451, 'mds_3': 34427, 'mds_4': 24454}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14658': {'gid': 14658, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.83:6839/4085470844', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6838', 'nonce': 4085470844}, {'type': 'v1', 'addr': '172.21.15.83:6839', 'nonce': 4085470844}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14676': {'gid': 14676, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.83:6841/2634754210', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6840', 'nonce': 2634754210}, {'type': 'v1', 'addr': '172.21.15.83:6841', 'nonce': 2634754210}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24382': {'gid': 24382, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.189:6833/1859921310', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6832', 'nonce': 1859921310}, {'type': 'v1', 'addr': '172.21.15.189:6833', 'nonce': 1859921310}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24421': {'gid': 24421, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.189:6835/2241923197', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6834', 'nonce': 2241923197}, {'type': 'v1', 'addr': '172.21.15.189:6835', 'nonce': 2241923197}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24439': {'gid': 24439, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.189:6837/3172311220', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6836', 'nonce': 3172311220}, {'type': 'v1', 'addr': '172.21.15.189:6837', 'nonce': 3172311220}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24454': {'gid': 24454, 'name': 'i', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 15, 'addr': '172.21.15.189:6839/2548965336', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6838', 'nonce': 2548965336}, {'type': 'v1', 'addr': '172.21.15.189:6839', 'nonce': 2548965336}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34427': {'gid': 34427, 'name': 'k', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 22, 'addr': '172.21.15.138:6833/1157967658', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6832', 'nonce': 1157967658}, {'type': 'v1', 'addr': '172.21.15.138:6833', 'nonce': 1157967658}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34451': {'gid': 34451, 'name': 'e', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 19, 'addr': '172.21.15.138:6835/1909219758', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6834', 'nonce': 1909219758}, {'type': 'v1', 'addr': '172.21.15.138:6835', 'nonce': 1909219758}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34493': {'gid': 34493, 'name': 'h', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 15, 'addr': '172.21.15.138:6837/900566754', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6836', 'nonce': 900566754}, {'type': 'v1', 'addr': '172.21.15.138:6837', 'nonce': 900566754}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34520': {'gid': 34520, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.138:6839/2236932393', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6838', 'nonce': 2236932393}, {'type': 'v1', 'addr': '172.21.15.138:6839', 'nonce': 2236932393}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 34520, 'qdb_cluster': [34427, 24454, 34520, 34493, 34451]} 2024-04-24T08:50:48.797 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-04-24T08:50:48.797 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-24T08:50:48.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.986+0000 7f59ebc00640 1 -- 172.21.15.83:0/3930458133 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59e4109ee0 msgr2=0x7f59e410c380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:48.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.986+0000 7f59ebc00640 1 --2- 172.21.15.83:0/3930458133 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59e4109ee0 0x7f59e410c380 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f59dc009ca0 tx=0x7f59dc02a950 comp rx=0 tx=0).stop 2024-04-24T08:50:48.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.987+0000 7f59ebc00640 1 -- 172.21.15.83:0/3930458133 shutdown_connections 2024-04-24T08:50:48.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.987+0000 7f59ebc00640 1 --2- 172.21.15.83:0/3930458133 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59e4109ee0 0x7f59e410c380 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:48.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.987+0000 7f59ebc00640 1 --2- 172.21.15.83:0/3930458133 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59e4074a00 0x7f59e4074e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:48.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.987+0000 7f59ebc00640 1 --2- 172.21.15.83:0/3930458133 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f59e4075fe0 0x7f59e4074430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:48.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.987+0000 7f59ebc00640 1 -- 172.21.15.83:0/3930458133 >> 172.21.15.83:0/3930458133 conn(0x7f59e40fd5e0 msgr2=0x7f59e40ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:50:48.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.987+0000 7f59ebc00640 1 -- 172.21.15.83:0/3930458133 shutdown_connections 2024-04-24T08:50:48.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.987+0000 7f59ebc00640 1 -- 172.21.15.83:0/3930458133 wait complete. 2024-04-24T08:50:48.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.988+0000 7f59ebc00640 1 Processor -- start 2024-04-24T08:50:48.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.988+0000 7f59ebc00640 1 -- start start 2024-04-24T08:50:48.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.988+0000 7f59ebc00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f59e4074a00 0x7f59e419a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:48.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.988+0000 7f59ebc00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59e4075fe0 0x7f59e419a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:48.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.988+0000 7f59ebc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59e4109ee0 0x7f59e41a19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:48.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.988+0000 7f59ebc00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f59e410fd90 con 0x7f59e4109ee0 2024-04-24T08:50:48.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.988+0000 7f59ebc00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f59e410fc10 con 0x7f59e4075fe0 2024-04-24T08:50:48.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.988+0000 7f59ebc00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f59e410ff10 con 0x7f59e4074a00 2024-04-24T08:50:48.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.989+0000 7f59e9c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59e4109ee0 0x7f59e41a19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:48.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.989+0000 7f59e9c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59e4109ee0 0x7f59e41a19b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56356/0 (socket says 172.21.15.83:56356) 2024-04-24T08:50:48.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.989+0000 7f59e9c00640 1 -- 172.21.15.83:0/4111810969 learned_addr learned my addr 172.21.15.83:0/4111810969 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:50:48.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.989+0000 7f59e9200640 1 --2- 172.21.15.83:0/4111810969 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f59e4074a00 0x7f59e419a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:48.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.989+0000 7f59dbe00640 1 --2- 172.21.15.83:0/4111810969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59e4075fe0 0x7f59e419a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:48.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.989+0000 7f59e9c00640 1 -- 172.21.15.83:0/4111810969 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f59e4074a00 msgr2=0x7f59e419a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:48.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.989+0000 7f59e9c00640 1 --2- 172.21.15.83:0/4111810969 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f59e4074a00 0x7f59e419a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:48.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.989+0000 7f59e9c00640 1 -- 172.21.15.83:0/4111810969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59e4075fe0 msgr2=0x7f59e419a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:48.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.989+0000 7f59e9c00640 1 --2- 172.21.15.83:0/4111810969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59e4075fe0 0x7f59e419a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:48.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.989+0000 7f59e9c00640 1 -- 172.21.15.83:0/4111810969 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59e41a2020 con 0x7f59e4109ee0 2024-04-24T08:50:48.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.989+0000 7f59e9200640 1 --2- 172.21.15.83:0/4111810969 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f59e4074a00 0x7f59e419a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:50:48.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.990+0000 7f59e9c00640 1 --2- 172.21.15.83:0/4111810969 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59e4109ee0 0x7f59e41a19b0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f59dc0098d0 tx=0x7f59dc02fba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:50:48.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.990+0000 7f59e2000640 1 -- 172.21.15.83:0/4111810969 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59dc034040 con 0x7f59e4109ee0 2024-04-24T08:50:48.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.990+0000 7f59e2000640 1 -- 172.21.15.83:0/4111810969 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f59dc0308f0 con 0x7f59e4109ee0 2024-04-24T08:50:48.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.990+0000 7f59e2000640 1 -- 172.21.15.83:0/4111810969 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59dc045480 con 0x7f59e4109ee0 2024-04-24T08:50:48.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.990+0000 7f59ebc00640 1 -- 172.21.15.83:0/4111810969 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59e41a22b0 con 0x7f59e4109ee0 2024-04-24T08:50:48.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.990+0000 7f59ebc00640 1 -- 172.21.15.83:0/4111810969 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59e41a2750 con 0x7f59e4109ee0 2024-04-24T08:50:48.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.993+0000 7f59ee992740 1 Processor -- start 2024-04-24T08:50:48.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.993+0000 7f59e2000640 1 -- 172.21.15.83:0/4111810969 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f59dc037020 con 0x7f59e4109ee0 2024-04-24T08:50:48.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.993+0000 7f59ee992740 1 -- start start 2024-04-24T08:50:48.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.994+0000 7f59ee992740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5623de47e090 0x5623de47e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:48.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.994+0000 7f59ee992740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5623de47ea80 0x5623de483580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:48.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.994+0000 7f59ee992740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5623de483bf0 0x5623de486080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:48.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.994+0000 7f59ee992740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5623de3e3b10 con 0x5623de483bf0 2024-04-24T08:50:48.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.994+0000 7f59ee992740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5623de38a2b0 con 0x5623de47ea80 2024-04-24T08:50:48.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.994+0000 7f59ee992740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5623de3dfad0 con 0x5623de47e090 2024-04-24T08:50:48.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.994+0000 7f59dbe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5623de483bf0 0x5623de486080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:48.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.994+0000 7f59dbe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5623de483bf0 0x5623de486080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56358/0 (socket says 172.21.15.83:56358) 2024-04-24T08:50:48.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.994+0000 7f59dbe00640 1 -- 172.21.15.83:0/2440957653 learned_addr learned my addr 172.21.15.83:0/2440957653 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:50:48.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.994+0000 7f59e9200640 1 --2- 172.21.15.83:0/2440957653 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5623de47ea80 0x5623de483580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:48.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.994+0000 7f59dbe00640 1 --2- 172.21.15.83:0/2440957653 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5623de47e090 0x5623de47e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:48.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.994+0000 7f59dbe00640 1 -- 172.21.15.83:0/2440957653 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5623de47e090 msgr2=0x5623de47e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:50:48.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.994+0000 7f59dbe00640 1 --2- 172.21.15.83:0/2440957653 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5623de47e090 0x5623de47e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:48.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.994+0000 7f59dbe00640 1 -- 172.21.15.83:0/2440957653 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5623de47ea80 msgr2=0x5623de483580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:50:48.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.994+0000 7f59dbe00640 1 --2- 172.21.15.83:0/2440957653 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5623de47ea80 0x5623de483580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:48.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.994+0000 7f59dbe00640 1 -- 172.21.15.83:0/2440957653 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623de0f62d0 con 0x5623de483bf0 2024-04-24T08:50:48.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.994+0000 7f59e9200640 1 --2- 172.21.15.83:0/2440957653 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5623de47ea80 0x5623de483580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:50:48.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.995+0000 7f59e2000640 1 --2- 172.21.15.83:0/4111810969 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f59c80a3c30 0x7f59c80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:48.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.995+0000 7f59dbe00640 1 --2- 172.21.15.83:0/2440957653 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5623de483bf0 0x5623de486080 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f59cc0058c0 tx=0x7f59cc00c6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5e566e737fba150 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:50:48.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.995+0000 7f59e2000640 1 -- 172.21.15.83:0/4111810969 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9237+0+0 (secure 0 0 0) 0x7f59dc0f19c0 con 0x7f59e4109ee0 2024-04-24T08:50:48.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.995+0000 7f59e9200640 1 --2- 172.21.15.83:0/4111810969 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f59c80a3c30 0x7f59c80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:48.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.995+0000 7f59da000640 1 -- 172.21.15.83:0/2440957653 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59cc012650 con 0x5623de483bf0 2024-04-24T08:50:48.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.995+0000 7f59da000640 1 -- 172.21.15.83:0/2440957653 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f59cc017030 con 0x5623de483bf0 2024-04-24T08:50:48.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.995+0000 7f59da000640 1 -- 172.21.15.83:0/2440957653 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59cc00faa0 con 0x5623de483bf0 2024-04-24T08:50:49.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.996+0000 7f59ee992740 1 -- 172.21.15.83:0/2440957653 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5623de483bf0 msgr2=0x5623de486080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:49.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.996+0000 7f59e9200640 1 --2- 172.21.15.83:0/4111810969 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f59c80a3c30 0x7f59c80a60f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x5623de47ee80 tx=0x7f59d40093f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:50:49.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.996+0000 7f59ee992740 1 --2- 172.21.15.83:0/2440957653 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5623de483bf0 0x5623de486080 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f59cc0058c0 tx=0x7f59cc00c6b0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.996+0000 7f59ee992740 1 -- 172.21.15.83:0/2440957653 shutdown_connections 2024-04-24T08:50:49.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.996+0000 7f59ee992740 1 --2- 172.21.15.83:0/2440957653 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5623de483bf0 0x5623de486080 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.996+0000 7f59ee992740 1 --2- 172.21.15.83:0/2440957653 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5623de47ea80 0x5623de483580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.996+0000 7f59ee992740 1 --2- 172.21.15.83:0/2440957653 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5623de47e090 0x5623de47e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.996+0000 7f59ee992740 1 -- 172.21.15.83:0/2440957653 >> 172.21.15.83:0/2440957653 conn(0x5623de479700 msgr2=0x5623de479b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:50:49.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.996+0000 7f59ee992740 1 -- 172.21.15.83:0/2440957653 shutdown_connections 2024-04-24T08:50:49.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.996+0000 7f59ee992740 1 -- 172.21.15.83:0/2440957653 wait complete. 2024-04-24T08:50:49.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.997+0000 7f59ee992740 1 -- start start 2024-04-24T08:50:49.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.997+0000 7f59ee992740 1 Processor -- start 2024-04-24T08:50:49.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.997+0000 7f59ee992740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5623de47e090 0x5623de488600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:49.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.997+0000 7f59ee992740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5623de47ea80 0x5623de488b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:49.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.997+0000 7f59ee992740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5623de483bf0 0x5623de48fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:49.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.997+0000 7f59ee992740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5623de486ba0 con 0x5623de47e090 2024-04-24T08:50:49.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.997+0000 7f59ee992740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5623de39d590 con 0x5623de47ea80 2024-04-24T08:50:49.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.997+0000 7f59ee992740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5623de486d20 con 0x5623de483bf0 2024-04-24T08:50:49.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.997+0000 7f59ee992740 10 client.0 _ll_register_callbacks cb 0x5623de3803a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:50:49.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.998+0000 7f59e9200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5623de47e090 0x5623de488600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:49.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.998+0000 7f59e9200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5623de47e090 0x5623de488600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56372/0 (socket says 172.21.15.83:56372) 2024-04-24T08:50:49.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.998+0000 7f59e9200640 1 -- 172.21.15.83:0/603491456 learned_addr learned my addr 172.21.15.83:0/603491456 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:50:49.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.998+0000 7f59dbe00640 1 --2- 172.21.15.83:0/603491456 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5623de47ea80 0x5623de488b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:49.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.998+0000 7f59e9200640 1 -- 172.21.15.83:0/603491456 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5623de483bf0 msgr2=0x5623de48fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:50:49.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.998+0000 7f59e9200640 1 --2- 172.21.15.83:0/603491456 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5623de483bf0 0x5623de48fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.998+0000 7f59e9200640 1 -- 172.21.15.83:0/603491456 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5623de47ea80 msgr2=0x5623de488b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:50:49.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.998+0000 7f59e9200640 1 --2- 172.21.15.83:0/603491456 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5623de47ea80 0x5623de488b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.998+0000 7f59e9200640 1 -- 172.21.15.83:0/603491456 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623de1c06b0 con 0x5623de47e090 2024-04-24T08:50:49.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.998+0000 7f59dbe00640 1 --2- 172.21.15.83:0/603491456 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5623de47ea80 0x5623de488b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:50:49.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.998+0000 7f59e9200640 1 --2- 172.21.15.83:0/603491456 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5623de47e090 0x5623de488600 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f59d40126d0 tx=0x7f59d4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f52357867204edf7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:50:49.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.998+0000 7f59d2a00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:50:49.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.998+0000 7f59d2a00640 1 -- 172.21.15.83:0/603491456 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d4023070 con 0x5623de47e090 2024-04-24T08:50:49.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.999+0000 7f59d2a00640 1 -- 172.21.15.83:0/603491456 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9237+0+0 (secure 0 0 0) 0x7f59d4011a60 con 0x5623de47e090 2024-04-24T08:50:49.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.999+0000 7f59ee992740 1 -- 172.21.15.83:0/603491456 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59cc017030 con 0x5623de47e090 2024-04-24T08:50:49.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.999+0000 7f59d2a00640 1 -- 172.21.15.83:0/603491456 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f59d402f050 con 0x5623de47e090 2024-04-24T08:50:49.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.999+0000 7f59ee992740 10 client.15267 fetch_fsmap learned FSMap version 27 2024-04-24T08:50:49.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.999+0000 7f59ee992740 1 -- 172.21.15.83:0/603491456 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5623de492fc0 con 0x5623de47e090 2024-04-24T08:50:49.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.999+0000 7f59d2a00640 1 -- 172.21.15.83:0/603491456 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f59cc017030 con 0x5623de47e090 2024-04-24T08:50:49.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.999+0000 7f59d2a00640 1 -- 172.21.15.83:0/603491456 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f59d400f040 con 0x5623de47e090 2024-04-24T08:50:49.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.999+0000 7f59ee992740 10 client.15267 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T08:50:49.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.999+0000 7f59ee992740 10 client.15267 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-24T08:50:49.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:48.999+0000 7f59ee992740 20 client.15267 populate_metadata read hostname 'smithi083' 2024-04-24T08:50:49.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.000+0000 7f59ee992740 1 --2- 172.21.15.83:0/603491456 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x5623de495720 0x5623de496b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:49.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.000+0000 7f59ee992740 4 client.15267 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:50:49.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.000+0000 7f59ee992740 1 -- 172.21.15.83:0/603491456 --> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5623de3f1530 con 0x5623de495720 2024-04-24T08:50:49.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.000+0000 7f59dbe00640 1 --2- 172.21.15.83:0/603491456 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x5623de495720 0x5623de496b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:49.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.001+0000 7f59dbe00640 1 --2- 172.21.15.83:0/603491456 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x5623de495720 0x5623de496b90 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1549a07a54676a66 server_cookie=6e29c47fb7a1c499 in_seq=0 out_seq=0 2024-04-24T08:50:49.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.001+0000 7f59d2a00640 10 client.15267 ms_handle_connect on v2:172.21.15.189:6838/2548965336 2024-04-24T08:50:49.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.003+0000 7f59d2a00640 1 -- 172.21.15.83:0/603491456 <== mds.4 v2:172.21.15.189:6838/2548965336 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5623de3f1530 con 0x5623de495720 2024-04-24T08:50:49.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.003+0000 7f59d2a00640 10 client.15267 handle_command_reply: tid=0 2024-04-24T08:50:49.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.004+0000 7f59ee992740 1 client.15267 shutdown 2024-04-24T08:50:49.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.004+0000 7f59ee992740 1 -- 172.21.15.83:0/603491456 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5623de47e090 msgr2=0x5623de488600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:49.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.004+0000 7f59ee992740 1 --2- 172.21.15.83:0/603491456 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5623de47e090 0x5623de488600 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f59d40126d0 tx=0x7f59d4004730 comp rx=0 tx=0).stop 2024-04-24T08:50:49.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.004+0000 7f59ee992740 1 -- 172.21.15.83:0/603491456 shutdown_connections 2024-04-24T08:50:49.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.004+0000 7f59ee992740 1 --2- 172.21.15.83:0/603491456 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x5623de495720 0x5623de496b90 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.004+0000 7f59ee992740 1 --2- 172.21.15.83:0/603491456 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5623de483bf0 0x5623de48fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.004+0000 7f59ee992740 1 --2- 172.21.15.83:0/603491456 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5623de47ea80 0x5623de488b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.004+0000 7f59d2a00640 0 client.15267 ms_handle_reset on v2:172.21.15.189:6838/2548965336 2024-04-24T08:50:49.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.004+0000 7f59ee992740 1 --2- 172.21.15.83:0/603491456 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5623de47e090 0x5623de488600 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.004+0000 7f59ee992740 1 -- 172.21.15.83:0/603491456 >> 172.21.15.83:0/603491456 conn(0x5623de479700 msgr2=0x5623de4788c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:50:49.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.004+0000 7f59ee992740 1 -- 172.21.15.83:0/603491456 shutdown_connections 2024-04-24T08:50:49.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.004+0000 7f59ee992740 1 -- 172.21.15.83:0/603491456 wait complete. 2024-04-24T08:50:49.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.004+0000 7f59ee992740 20 client.15267 trim_cache size 0 max 16384 2024-04-24T08:50:49.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.025+0000 7f59ee992740 1 Processor -- start 2024-04-24T08:50:49.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.025+0000 7f59ee992740 1 -- start start 2024-04-24T08:50:49.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.025+0000 7f59ee992740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5623de47e090 0x5623de474750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:49.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.025+0000 7f59ee992740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5623de47ea80 0x5623de48fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:49.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.025+0000 7f59ee992740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5623de483bf0 0x5623de488210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:49.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.026+0000 7f59ee992740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5623de494ed0 con 0x5623de47ea80 2024-04-24T08:50:49.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.026+0000 7f59ee992740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5623de4955a0 con 0x5623de47e090 2024-04-24T08:50:49.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.026+0000 7f59ee992740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5623de495c70 con 0x5623de483bf0 2024-04-24T08:50:49.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.026+0000 7f59e9c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5623de483bf0 0x5623de488210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:49.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.026+0000 7f59dbe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5623de47ea80 0x5623de48fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:49.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.026+0000 7f59e9200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5623de47e090 0x5623de474750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:49.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.026+0000 7f59e9c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5623de483bf0 0x5623de488210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:48806/0 (socket says 172.21.15.83:48806) 2024-04-24T08:50:49.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.026+0000 7f59e9c00640 1 -- 172.21.15.83:0/3193942340 learned_addr learned my addr 172.21.15.83:0/3193942340 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:50:49.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.027+0000 7f59e9c00640 1 -- 172.21.15.83:0/3193942340 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5623de47e090 msgr2=0x5623de474750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:50:49.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.027+0000 7f59e9c00640 1 --2- 172.21.15.83:0/3193942340 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5623de47e090 0x5623de474750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.027+0000 7f59e9c00640 1 -- 172.21.15.83:0/3193942340 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5623de47ea80 msgr2=0x5623de48fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:50:49.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.027+0000 7f59e9c00640 1 --2- 172.21.15.83:0/3193942340 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5623de47ea80 0x5623de48fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.027+0000 7f59e9c00640 1 -- 172.21.15.83:0/3193942340 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623de46ae20 con 0x5623de483bf0 2024-04-24T08:50:49.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.027+0000 7f59e9c00640 1 --2- 172.21.15.83:0/3193942340 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5623de483bf0 0x5623de488210 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f59dc0b9d40 tx=0x7f59dc0bd770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d24a189cb40bfd9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:50:49.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.031+0000 7f59d9600640 1 -- 172.21.15.83:0/3193942340 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59dc0bd8e0 con 0x5623de483bf0 2024-04-24T08:50:49.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.031+0000 7f59d9600640 1 -- 172.21.15.83:0/3193942340 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f59dc0c1c90 con 0x5623de483bf0 2024-04-24T08:50:49.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.031+0000 7f59d9600640 1 -- 172.21.15.83:0/3193942340 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59dc004040 con 0x5623de483bf0 2024-04-24T08:50:49.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.032+0000 7f59ee992740 1 -- 172.21.15.83:0/3193942340 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5623de483bf0 msgr2=0x5623de488210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:49.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.032+0000 7f59ee992740 1 --2- 172.21.15.83:0/3193942340 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5623de483bf0 0x5623de488210 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f59dc0b9d40 tx=0x7f59dc0bd770 comp rx=0 tx=0).stop 2024-04-24T08:50:49.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.032+0000 7f59ee992740 1 -- 172.21.15.83:0/3193942340 shutdown_connections 2024-04-24T08:50:49.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.032+0000 7f59ee992740 1 --2- 172.21.15.83:0/3193942340 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5623de483bf0 0x5623de488210 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.032+0000 7f59ee992740 1 --2- 172.21.15.83:0/3193942340 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5623de47ea80 0x5623de48fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.032+0000 7f59ee992740 1 --2- 172.21.15.83:0/3193942340 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5623de47e090 0x5623de474750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.032+0000 7f59ee992740 1 -- 172.21.15.83:0/3193942340 >> 172.21.15.83:0/3193942340 conn(0x5623de479700 msgr2=0x5623de47f4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:50:49.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.032+0000 7f59ee992740 1 -- 172.21.15.83:0/3193942340 shutdown_connections 2024-04-24T08:50:49.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.032+0000 7f59ee992740 1 -- 172.21.15.83:0/3193942340 wait complete. 2024-04-24T08:50:49.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.032+0000 7f59ee992740 1 -- start start 2024-04-24T08:50:49.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.033+0000 7f59ee992740 1 Processor -- start 2024-04-24T08:50:49.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.033+0000 7f59ee992740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5623de47e090 0x5623de481de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:49.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.033+0000 7f59ee992740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5623de47ea80 0x5623de482320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:49.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.033+0000 7f59ee992740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5623de483bf0 0x5623de4f96c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:49.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.033+0000 7f59ee992740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5623de496a10 con 0x5623de47e090 2024-04-24T08:50:49.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.033+0000 7f59ee992740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5623de496340 con 0x5623de483bf0 2024-04-24T08:50:49.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.033+0000 7f59ee992740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5623de4970e0 con 0x5623de47ea80 2024-04-24T08:50:49.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.033+0000 7f59ee992740 10 client.0 _ll_register_callbacks cb 0x7f59d4009da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:50:49.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.033+0000 7f59e9200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5623de47e090 0x5623de481de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:49.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.033+0000 7f59e9200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5623de47e090 0x5623de481de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56396/0 (socket says 172.21.15.83:56396) 2024-04-24T08:50:49.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.033+0000 7f59e9200640 1 -- 172.21.15.83:0/3481591538 learned_addr learned my addr 172.21.15.83:0/3481591538 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:50:49.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.034+0000 7f59e9c00640 1 --2- 172.21.15.83:0/3481591538 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5623de483bf0 0x5623de4f96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:49.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.034+0000 7f59e9200640 1 -- 172.21.15.83:0/3481591538 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5623de47ea80 msgr2=0x5623de482320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:50:49.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.034+0000 7f59e9200640 1 --2- 172.21.15.83:0/3481591538 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5623de47ea80 0x5623de482320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.034+0000 7f59e9200640 1 -- 172.21.15.83:0/3481591538 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5623de483bf0 msgr2=0x5623de4f96c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:50:49.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.034+0000 7f59e9200640 1 --2- 172.21.15.83:0/3481591538 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5623de483bf0 0x5623de4f96c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.034+0000 7f59e9200640 1 -- 172.21.15.83:0/3481591538 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623de4f9c00 con 0x5623de47e090 2024-04-24T08:50:49.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.034+0000 7f59e9200640 1 --2- 172.21.15.83:0/3481591538 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5623de47e090 0x5623de481de0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f59d401fcc0 tx=0x7f59d40275e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd0b3c3d25165581 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:50:49.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.034+0000 7f59d2a00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:50:49.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.034+0000 7f59d2a00640 1 -- 172.21.15.83:0/3481591538 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d4023040 con 0x5623de47e090 2024-04-24T08:50:49.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.034+0000 7f59d2a00640 1 -- 172.21.15.83:0/3481591538 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9237+0+0 (secure 0 0 0) 0x7f59d401f450 con 0x5623de47e090 2024-04-24T08:50:49.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.034+0000 7f59ee992740 1 -- 172.21.15.83:0/3481591538 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59dc0c1c90 con 0x5623de47e090 2024-04-24T08:50:49.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.035+0000 7f59d2a00640 1 -- 172.21.15.83:0/3481591538 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f59d401f650 con 0x5623de47e090 2024-04-24T08:50:49.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.035+0000 7f59ee992740 10 client.15273 fetch_fsmap learned FSMap version 27 2024-04-24T08:50:49.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.035+0000 7f59ee992740 1 -- 172.21.15.83:0/3481591538 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5623de4fadb0 con 0x5623de47e090 2024-04-24T08:50:49.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.035+0000 7f59d2a00640 1 -- 172.21.15.83:0/3481591538 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f59dc0c1c90 con 0x5623de47e090 2024-04-24T08:50:49.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.035+0000 7f59d2a00640 1 -- 172.21.15.83:0/3481591538 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f59d4027650 con 0x5623de47e090 2024-04-24T08:50:49.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.035+0000 7f59ee992740 10 client.15273 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T08:50:49.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.035+0000 7f59ee992740 10 client.15273 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-24T08:50:49.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.035+0000 7f59ee992740 20 client.15273 populate_metadata read hostname 'smithi083' 2024-04-24T08:50:49.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.035+0000 7f59ee992740 1 --2- 172.21.15.83:0/3481591538 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x5623de4ffac0 0x5623de4fe7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:49.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.035+0000 7f59ee992740 4 client.15273 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T08:50:49.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.035+0000 7f59ee992740 1 -- 172.21.15.83:0/3481591538 --> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] -- command(tid 0: {"prefix": "status"}) -- 0x5623de3eb3a0 con 0x5623de4ffac0 2024-04-24T08:50:49.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.036+0000 7f59dbe00640 1 --2- 172.21.15.83:0/3481591538 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x5623de4ffac0 0x5623de4fe7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:49.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.037+0000 7f59dbe00640 1 --2- 172.21.15.83:0/3481591538 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x5623de4ffac0 0x5623de4fe7d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8fb95d0466308fc5 server_cookie=bd9a52084698dfa2 in_seq=0 out_seq=0 2024-04-24T08:50:49.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.037+0000 7f59d2a00640 10 client.15273 ms_handle_connect on v2:172.21.15.189:6838/2548965336 2024-04-24T08:50:49.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.037+0000 7f59d2a00640 1 -- 172.21.15.83:0/3481591538 <== mds.4 v2:172.21.15.189:6838/2548965336 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x5623de3eb3a0 con 0x5623de4ffac0 2024-04-24T08:50:49.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.038+0000 7f59d2a00640 10 client.15273 handle_command_reply: tid=0 2024-04-24T08:50:49.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.038+0000 7f59ee992740 1 client.15273 shutdown 2024-04-24T08:50:49.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.038+0000 7f59ee992740 1 -- 172.21.15.83:0/3481591538 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5623de47e090 msgr2=0x5623de481de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:49.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.038+0000 7f59ee992740 1 --2- 172.21.15.83:0/3481591538 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5623de47e090 0x5623de481de0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f59d401fcc0 tx=0x7f59d40275e0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.038+0000 7f59ee992740 1 -- 172.21.15.83:0/3481591538 shutdown_connections 2024-04-24T08:50:49.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.038+0000 7f59ee992740 1 --2- 172.21.15.83:0/3481591538 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x5623de4ffac0 0x5623de4fe7d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.038+0000 7f59ee992740 1 --2- 172.21.15.83:0/3481591538 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5623de483bf0 0x5623de4f96c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.038+0000 7f59ee992740 1 --2- 172.21.15.83:0/3481591538 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5623de47ea80 0x5623de482320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.038+0000 7f59d2a00640 0 client.15273 ms_handle_reset on v2:172.21.15.189:6838/2548965336 2024-04-24T08:50:49.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.038+0000 7f59ee992740 1 --2- 172.21.15.83:0/3481591538 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5623de47e090 0x5623de481de0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.038+0000 7f59ee992740 1 -- 172.21.15.83:0/3481591538 >> 172.21.15.83:0/3481591538 conn(0x5623de479700 msgr2=0x5623de47f480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:50:49.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.038+0000 7f59ee992740 1 -- 172.21.15.83:0/3481591538 shutdown_connections 2024-04-24T08:50:49.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.038+0000 7f59ee992740 1 -- 172.21.15.83:0/3481591538 wait complete. 2024-04-24T08:50:49.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.038+0000 7f59ee992740 20 client.15273 trim_cache size 0 max 16384 2024-04-24T08:50:49.041 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T08:50:49.041 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_fsid": "bd3e039e-0215-11ef-bc93-c7b262605968", 2024-04-24T08:50:49.041 INFO:teuthology.orchestra.run.smithi083.stdout: "whoami": 4, 2024-04-24T08:50:49.041 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 24454, 2024-04-24T08:50:49.041 INFO:teuthology.orchestra.run.smithi083.stdout: "want_state": "up:active", 2024-04-24T08:50:49.041 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "up:active", 2024-04-24T08:50:49.042 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs", 2024-04-24T08:50:49.042 INFO:teuthology.orchestra.run.smithi083.stdout: "rank_uptime": 104.967151667, 2024-04-24T08:50:49.042 INFO:teuthology.orchestra.run.smithi083.stdout: "mdsmap_epoch": 27, 2024-04-24T08:50:49.042 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch": 93, 2024-04-24T08:50:49.042 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch_barrier": 89, 2024-04-24T08:50:49.042 INFO:teuthology.orchestra.run.smithi083.stdout: "uptime": 159.621064077 2024-04-24T08:50:49.042 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T08:50:49.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.039+0000 7f59ebc00640 1 -- 172.21.15.83:0/4111810969 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f59c80a3c30 msgr2=0x7f59c80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:49.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.039+0000 7f59ebc00640 1 --2- 172.21.15.83:0/4111810969 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f59c80a3c30 0x7f59c80a60f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x5623de47ee80 tx=0x7f59d40093f0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.039+0000 7f59ebc00640 1 -- 172.21.15.83:0/4111810969 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59e4109ee0 msgr2=0x7f59e41a19b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:49.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.039+0000 7f59ebc00640 1 --2- 172.21.15.83:0/4111810969 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59e4109ee0 0x7f59e41a19b0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f59dc0098d0 tx=0x7f59dc02fba0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.039+0000 7f59ebc00640 1 -- 172.21.15.83:0/4111810969 shutdown_connections 2024-04-24T08:50:49.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.039+0000 7f59ebc00640 1 --2- 172.21.15.83:0/4111810969 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f59c80a3c30 0x7f59c80a60f0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.039+0000 7f59ebc00640 1 --2- 172.21.15.83:0/4111810969 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59e4109ee0 0x7f59e41a19b0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.039+0000 7f59ebc00640 1 --2- 172.21.15.83:0/4111810969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59e4075fe0 0x7f59e419a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.039+0000 7f59ebc00640 1 --2- 172.21.15.83:0/4111810969 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f59e4074a00 0x7f59e419a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.039+0000 7f59ebc00640 1 -- 172.21.15.83:0/4111810969 >> 172.21.15.83:0/4111810969 conn(0x7f59e40fd5e0 msgr2=0x7f59e4105e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:50:49.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.039+0000 7f59ebc00640 1 -- 172.21.15.83:0/4111810969 shutdown_connections 2024-04-24T08:50:49.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.039+0000 7f59ebc00640 1 -- 172.21.15.83:0/4111810969 wait complete. 2024-04-24T08:50:49.052 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:50:49 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/390013927' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T08:50:49.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:50:49 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/1757211451' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:50:49.059 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.k status 2024-04-24T08:50:49.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.259+0000 7f7b51a00640 1 -- 172.21.15.83:0/539349329 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b4c105470 msgr2=0x7f7b4c1058f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:49.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.259+0000 7f7b51a00640 1 --2- 172.21.15.83:0/539349329 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b4c105470 0x7f7b4c1058f0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f7b34009900 tx=0x7f7b3402a410 comp rx=0 tx=0).stop 2024-04-24T08:50:49.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.259+0000 7f7b51a00640 1 -- 172.21.15.83:0/539349329 shutdown_connections 2024-04-24T08:50:49.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.259+0000 7f7b51a00640 1 --2- 172.21.15.83:0/539349329 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7b4c105e30 0x7f7b4c10c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.259+0000 7f7b51a00640 1 --2- 172.21.15.83:0/539349329 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b4c105470 0x7f7b4c1058f0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.259+0000 7f7b51a00640 1 --2- 172.21.15.83:0/539349329 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7b4c104280 0x7f7b4c1046a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.259+0000 7f7b51a00640 1 -- 172.21.15.83:0/539349329 >> 172.21.15.83:0/539349329 conn(0x7f7b4c0ff7f0 msgr2=0x7f7b4c101c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:50:49.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.259+0000 7f7b51a00640 1 -- 172.21.15.83:0/539349329 shutdown_connections 2024-04-24T08:50:49.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.259+0000 7f7b51a00640 1 -- 172.21.15.83:0/539349329 wait complete. 2024-04-24T08:50:49.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.260+0000 7f7b51a00640 1 Processor -- start 2024-04-24T08:50:49.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.260+0000 7f7b51a00640 1 -- start start 2024-04-24T08:50:49.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.261+0000 7f7b51a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7b4c104280 0x7f7b4c19a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:49.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.261+0000 7f7b51a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7b4c105470 0x7f7b4c19aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:49.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.261+0000 7f7b51a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b4c105e30 0x7f7b4c1a1a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:49.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.261+0000 7f7b51a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7b4c10fe10 con 0x7f7b4c105e30 2024-04-24T08:50:49.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.261+0000 7f7b51a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7b4c10fc90 con 0x7f7b4c104280 2024-04-24T08:50:49.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.261+0000 7f7b51a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7b4c10ff90 con 0x7f7b4c105470 2024-04-24T08:50:49.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.261+0000 7f7b4b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b4c105e30 0x7f7b4c1a1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:49.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.261+0000 7f7b4b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b4c105e30 0x7f7b4c1a1a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56416/0 (socket says 172.21.15.83:56416) 2024-04-24T08:50:49.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.261+0000 7f7b4b400640 1 -- 172.21.15.83:0/2612482645 learned_addr learned my addr 172.21.15.83:0/2612482645 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:50:49.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.261+0000 7f7b4aa00640 1 --2- 172.21.15.83:0/2612482645 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7b4c104280 0x7f7b4c19a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:49.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.261+0000 7f7b4a000640 1 --2- 172.21.15.83:0/2612482645 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7b4c105470 0x7f7b4c19aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:49.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.261+0000 7f7b4b400640 1 -- 172.21.15.83:0/2612482645 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7b4c105470 msgr2=0x7f7b4c19aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:49.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.261+0000 7f7b4b400640 1 --2- 172.21.15.83:0/2612482645 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7b4c105470 0x7f7b4c19aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.261+0000 7f7b4b400640 1 -- 172.21.15.83:0/2612482645 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7b4c104280 msgr2=0x7f7b4c19a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:49.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.261+0000 7f7b4b400640 1 --2- 172.21.15.83:0/2612482645 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7b4c104280 0x7f7b4c19a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.261+0000 7f7b4b400640 1 -- 172.21.15.83:0/2612482645 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b4c1a20e0 con 0x7f7b4c105e30 2024-04-24T08:50:49.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.261+0000 7f7b4a000640 1 --2- 172.21.15.83:0/2612482645 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7b4c105470 0x7f7b4c19aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:50:49.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.262+0000 7f7b4b400640 1 --2- 172.21.15.83:0/2612482645 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b4c105e30 0x7f7b4c1a1a70 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f7b3c00d750 tx=0x7f7b3c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:50:49.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.262+0000 7f7b43400640 1 -- 172.21.15.83:0/2612482645 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b3c017070 con 0x7f7b4c105e30 2024-04-24T08:50:49.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.262+0000 7f7b43400640 1 -- 172.21.15.83:0/2612482645 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7b3c0054d0 con 0x7f7b4c105e30 2024-04-24T08:50:49.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.262+0000 7f7b51a00640 1 -- 172.21.15.83:0/2612482645 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b4c1a23d0 con 0x7f7b4c105e30 2024-04-24T08:50:49.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.262+0000 7f7b43400640 1 -- 172.21.15.83:0/2612482645 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b3c004c60 con 0x7f7b4c105e30 2024-04-24T08:50:49.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.262+0000 7f7b51a00640 1 -- 172.21.15.83:0/2612482645 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b4c1a2970 con 0x7f7b4c105e30 2024-04-24T08:50:49.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.266+0000 7f7b547bf740 1 Processor -- start 2024-04-24T08:50:49.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.266+0000 7f7b43400640 1 -- 172.21.15.83:0/2612482645 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f7b3c004e00 con 0x7f7b4c105e30 2024-04-24T08:50:49.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.266+0000 7f7b547bf740 1 -- start start 2024-04-24T08:50:49.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.266+0000 7f7b547bf740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56438d281090 0x56438d2814b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:49.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.266+0000 7f7b547bf740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56438d281a80 0x56438d286580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:49.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.266+0000 7f7b547bf740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56438d286bf0 0x56438d289080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:49.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.266+0000 7f7b547bf740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56438d1e6b10 con 0x56438d286bf0 2024-04-24T08:50:49.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.266+0000 7f7b547bf740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56438d18d2b0 con 0x56438d281a80 2024-04-24T08:50:49.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.266+0000 7f7b547bf740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56438d1e2ad0 con 0x56438d281090 2024-04-24T08:50:49.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.267+0000 7f7b4a000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56438d286bf0 0x56438d289080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:49.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.267+0000 7f7b4a000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56438d286bf0 0x56438d289080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56424/0 (socket says 172.21.15.83:56424) 2024-04-24T08:50:49.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.267+0000 7f7b4a000640 1 -- 172.21.15.83:0/3224298235 learned_addr learned my addr 172.21.15.83:0/3224298235 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:50:49.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.267+0000 7f7b4a000640 1 --2- 172.21.15.83:0/3224298235 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56438d281090 0x56438d2814b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:49.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.267+0000 7f7b4aa00640 1 --2- 172.21.15.83:0/3224298235 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56438d281a80 0x56438d286580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:49.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.267+0000 7f7b4a000640 1 -- 172.21.15.83:0/3224298235 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56438d281090 msgr2=0x56438d2814b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:50:49.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.267+0000 7f7b4a000640 1 --2- 172.21.15.83:0/3224298235 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56438d281090 0x56438d2814b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.267+0000 7f7b4a000640 1 -- 172.21.15.83:0/3224298235 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56438d281a80 msgr2=0x56438d286580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:50:49.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.267+0000 7f7b4a000640 1 --2- 172.21.15.83:0/3224298235 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56438d281a80 0x56438d286580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.267+0000 7f7b4a000640 1 -- 172.21.15.83:0/3224298235 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56438cef92d0 con 0x56438d286bf0 2024-04-24T08:50:49.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.267+0000 7f7b43400640 1 --2- 172.21.15.83:0/2612482645 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f7b300a3c30 0x7f7b300a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:49.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.267+0000 7f7b4a000640 1 --2- 172.21.15.83:0/3224298235 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56438d286bf0 0x56438d289080 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f7b34005ab0 tx=0x7f7b340314a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6914832ddf46453 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:50:49.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.268+0000 7f7b43400640 1 -- 172.21.15.83:0/2612482645 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9237+0+0 (secure 0 0 0) 0x7f7b3c0d01b0 con 0x7f7b4c105e30 2024-04-24T08:50:49.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.268+0000 7f7b4aa00640 1 --2- 172.21.15.83:0/2612482645 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f7b300a3c30 0x7f7b300a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:49.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.268+0000 7f7b3be00640 1 -- 172.21.15.83:0/3224298235 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b34042020 con 0x56438d286bf0 2024-04-24T08:50:49.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.268+0000 7f7b3be00640 1 -- 172.21.15.83:0/3224298235 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7b34003ca0 con 0x56438d286bf0 2024-04-24T08:50:49.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.268+0000 7f7b3be00640 1 -- 172.21.15.83:0/3224298235 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b34035040 con 0x56438d286bf0 2024-04-24T08:50:49.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.269+0000 7f7b547bf740 1 -- 172.21.15.83:0/3224298235 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56438d286bf0 msgr2=0x56438d289080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:49.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.269+0000 7f7b4aa00640 1 --2- 172.21.15.83:0/2612482645 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f7b300a3c30 0x7f7b300a60f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x56438d281e80 tx=0x7f7b44009580 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:50:49.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.269+0000 7f7b547bf740 1 --2- 172.21.15.83:0/3224298235 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56438d286bf0 0x56438d289080 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f7b34005ab0 tx=0x7f7b340314a0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.269+0000 7f7b547bf740 1 -- 172.21.15.83:0/3224298235 shutdown_connections 2024-04-24T08:50:49.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.269+0000 7f7b547bf740 1 --2- 172.21.15.83:0/3224298235 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56438d286bf0 0x56438d289080 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.269+0000 7f7b547bf740 1 --2- 172.21.15.83:0/3224298235 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56438d281a80 0x56438d286580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.269+0000 7f7b547bf740 1 --2- 172.21.15.83:0/3224298235 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56438d281090 0x56438d2814b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.269+0000 7f7b547bf740 1 -- 172.21.15.83:0/3224298235 >> 172.21.15.83:0/3224298235 conn(0x56438d27c700 msgr2=0x56438d27cb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:50:49.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.269+0000 7f7b547bf740 1 -- 172.21.15.83:0/3224298235 shutdown_connections 2024-04-24T08:50:49.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.269+0000 7f7b547bf740 1 -- 172.21.15.83:0/3224298235 wait complete. 2024-04-24T08:50:49.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.269+0000 7f7b547bf740 1 -- start start 2024-04-24T08:50:49.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.270+0000 7f7b547bf740 1 Processor -- start 2024-04-24T08:50:49.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.270+0000 7f7b547bf740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56438d281090 0x56438d28b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:49.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.270+0000 7f7b547bf740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56438d281a80 0x56438d28b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:49.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.270+0000 7f7b547bf740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56438d286bf0 0x56438d292a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:49.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.270+0000 7f7b547bf740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56438d289b40 con 0x56438d281a80 2024-04-24T08:50:49.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.270+0000 7f7b547bf740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56438d1a0590 con 0x56438d281090 2024-04-24T08:50:49.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.270+0000 7f7b547bf740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56438d289cc0 con 0x56438d286bf0 2024-04-24T08:50:49.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.270+0000 7f7b547bf740 10 client.0 _ll_register_callbacks cb 0x56438d1833a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:50:49.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.270+0000 7f7b4a000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56438d281a80 0x56438d28b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:49.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.270+0000 7f7b4a000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56438d281a80 0x56438d28b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56426/0 (socket says 172.21.15.83:56426) 2024-04-24T08:50:49.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.271+0000 7f7b4a000640 1 -- 172.21.15.83:0/427008475 learned_addr learned my addr 172.21.15.83:0/427008475 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:50:49.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.271+0000 7f7b4aa00640 1 --2- 172.21.15.83:0/427008475 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56438d281090 0x56438d28b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:49.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.271+0000 7f7b4b400640 1 --2- 172.21.15.83:0/427008475 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56438d286bf0 0x56438d292a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:49.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.271+0000 7f7b4a000640 1 -- 172.21.15.83:0/427008475 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56438d286bf0 msgr2=0x56438d292a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:50:49.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.271+0000 7f7b4a000640 1 --2- 172.21.15.83:0/427008475 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56438d286bf0 0x56438d292a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.271+0000 7f7b4a000640 1 -- 172.21.15.83:0/427008475 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56438d281090 msgr2=0x56438d28b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:50:49.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.271+0000 7f7b4a000640 1 --2- 172.21.15.83:0/427008475 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56438d281090 0x56438d28b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.271+0000 7f7b4a000640 1 -- 172.21.15.83:0/427008475 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56438cfc36b0 con 0x56438d281a80 2024-04-24T08:50:49.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.271+0000 7f7b4a000640 1 --2- 172.21.15.83:0/427008475 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56438d281a80 0x56438d28b9d0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f7b34031cd0 tx=0x7f7b34031d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb712cbc189e3173 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:50:49.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.271+0000 7f7b38c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:50:49.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.272+0000 7f7b38c00640 1 -- 172.21.15.83:0/427008475 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b34042030 con 0x56438d281a80 2024-04-24T08:50:49.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.272+0000 7f7b38c00640 1 -- 172.21.15.83:0/427008475 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9237+0+0 (secure 0 0 0) 0x7f7b34003a00 con 0x56438d281a80 2024-04-24T08:50:49.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.272+0000 7f7b547bf740 1 -- 172.21.15.83:0/427008475 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b34003ca0 con 0x56438d281a80 2024-04-24T08:50:49.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.272+0000 7f7b38c00640 1 -- 172.21.15.83:0/427008475 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7b34058050 con 0x56438d281a80 2024-04-24T08:50:49.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.272+0000 7f7b547bf740 10 client.15285 fetch_fsmap learned FSMap version 27 2024-04-24T08:50:49.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.272+0000 7f7b547bf740 1 -- 172.21.15.83:0/427008475 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56438d295d80 con 0x56438d281a80 2024-04-24T08:50:49.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.272+0000 7f7b38c00640 1 -- 172.21.15.83:0/427008475 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7b34003ca0 con 0x56438d281a80 2024-04-24T08:50:49.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.272+0000 7f7b38c00640 1 -- 172.21.15.83:0/427008475 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f7b3403a040 con 0x56438d281a80 2024-04-24T08:50:49.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.273+0000 7f7b547bf740 10 client.15285 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T08:50:49.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.273+0000 7f7b547bf740 10 client.15285 resolve_mds: resolved name 'k' to daemon mds.k 2024-04-24T08:50:49.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.273+0000 7f7b547bf740 20 client.15285 populate_metadata read hostname 'smithi083' 2024-04-24T08:50:49.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.273+0000 7f7b547bf740 1 --2- 172.21.15.83:0/427008475 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x56438d297fd0 0x56438d299440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:49.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.273+0000 7f7b547bf740 4 client.15285 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:50:49.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.273+0000 7f7b547bf740 1 -- 172.21.15.83:0/427008475 --> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56438d1f4530 con 0x56438d297fd0 2024-04-24T08:50:49.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.273+0000 7f7b4aa00640 1 --2- 172.21.15.83:0/427008475 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x56438d297fd0 0x56438d299440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:49.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.274+0000 7f7b4aa00640 1 --2- 172.21.15.83:0/427008475 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x56438d297fd0 0x56438d299440 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3f6a92e3ddf9e641 server_cookie=6c9786bdedc1e48e in_seq=0 out_seq=0 2024-04-24T08:50:49.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.274+0000 7f7b38c00640 10 client.15285 ms_handle_connect on v2:172.21.15.138:6832/1157967658 2024-04-24T08:50:49.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.277+0000 7f7b38c00640 1 -- 172.21.15.83:0/427008475 <== mds.3 v2:172.21.15.138:6832/1157967658 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56438d1f4530 con 0x56438d297fd0 2024-04-24T08:50:49.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.277+0000 7f7b38c00640 10 client.15285 handle_command_reply: tid=0 2024-04-24T08:50:49.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.277+0000 7f7b547bf740 1 client.15285 shutdown 2024-04-24T08:50:49.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.277+0000 7f7b547bf740 1 -- 172.21.15.83:0/427008475 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56438d281a80 msgr2=0x56438d28b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:49.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.277+0000 7f7b547bf740 1 --2- 172.21.15.83:0/427008475 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56438d281a80 0x56438d28b9d0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f7b34031cd0 tx=0x7f7b34031d00 comp rx=0 tx=0).stop 2024-04-24T08:50:49.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.277+0000 7f7b547bf740 1 -- 172.21.15.83:0/427008475 shutdown_connections 2024-04-24T08:50:49.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.277+0000 7f7b547bf740 1 --2- 172.21.15.83:0/427008475 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x56438d297fd0 0x56438d299440 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.277+0000 7f7b547bf740 1 --2- 172.21.15.83:0/427008475 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56438d286bf0 0x56438d292a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.277+0000 7f7b547bf740 1 --2- 172.21.15.83:0/427008475 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56438d281a80 0x56438d28b9d0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.277+0000 7f7b547bf740 1 --2- 172.21.15.83:0/427008475 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56438d281090 0x56438d28b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.277+0000 7f7b547bf740 1 -- 172.21.15.83:0/427008475 >> 172.21.15.83:0/427008475 conn(0x56438d27c700 msgr2=0x56438d27b8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:50:49.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.277+0000 7f7b38c00640 0 client.15285 ms_handle_reset on v2:172.21.15.138:6832/1157967658 2024-04-24T08:50:49.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.277+0000 7f7b547bf740 1 -- 172.21.15.83:0/427008475 shutdown_connections 2024-04-24T08:50:49.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.277+0000 7f7b547bf740 1 -- 172.21.15.83:0/427008475 wait complete. 2024-04-24T08:50:49.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.277+0000 7f7b547bf740 20 client.15285 trim_cache size 0 max 16384 2024-04-24T08:50:49.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.297+0000 7f7b547bf740 1 Processor -- start 2024-04-24T08:50:49.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.297+0000 7f7b547bf740 1 -- start start 2024-04-24T08:50:49.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.297+0000 7f7b547bf740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56438d281090 0x56438d2864b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:49.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.297+0000 7f7b547bf740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56438d281a80 0x56438d2929d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:49.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.297+0000 7f7b547bf740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56438d286bf0 0x56438d289020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:49.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.297+0000 7f7b547bf740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56438d297780 con 0x56438d281090 2024-04-24T08:50:49.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.297+0000 7f7b547bf740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56438d297e50 con 0x56438d281a80 2024-04-24T08:50:49.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.297+0000 7f7b547bf740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56438d298520 con 0x56438d286bf0 2024-04-24T08:50:49.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.298+0000 7f7b4aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56438d281090 0x56438d2864b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:49.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.298+0000 7f7b4aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56438d281090 0x56438d2864b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56428/0 (socket says 172.21.15.83:56428) 2024-04-24T08:50:49.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.298+0000 7f7b4aa00640 1 -- 172.21.15.83:0/3386108139 learned_addr learned my addr 172.21.15.83:0/3386108139 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:50:49.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.298+0000 7f7b4b400640 1 --2- 172.21.15.83:0/3386108139 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56438d286bf0 0x56438d289020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:49.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.298+0000 7f7b4aa00640 1 -- 172.21.15.83:0/3386108139 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56438d286bf0 msgr2=0x56438d289020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:50:49.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.298+0000 7f7b4aa00640 1 --2- 172.21.15.83:0/3386108139 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56438d286bf0 0x56438d289020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.298+0000 7f7b4a000640 1 --2- 172.21.15.83:0/3386108139 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56438d281a80 0x56438d2929d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:49.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.298+0000 7f7b4aa00640 1 -- 172.21.15.83:0/3386108139 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56438d281a80 msgr2=0x56438d2929d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:50:49.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.298+0000 7f7b4aa00640 1 --2- 172.21.15.83:0/3386108139 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56438d281a80 0x56438d2929d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.298+0000 7f7b4aa00640 1 -- 172.21.15.83:0/3386108139 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56438d26de20 con 0x56438d281090 2024-04-24T08:50:49.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.298+0000 7f7b4aa00640 1 --2- 172.21.15.83:0/3386108139 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56438d281090 0x56438d2864b0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x56438d2824f0 tx=0x7f7b44015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aacde0813967a4ae server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:50:49.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.299+0000 7f7b3b400640 1 -- 172.21.15.83:0/3386108139 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b440293a0 con 0x56438d281090 2024-04-24T08:50:49.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.299+0000 7f7b3b400640 1 -- 172.21.15.83:0/3386108139 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7b4402f070 con 0x56438d281090 2024-04-24T08:50:49.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.299+0000 7f7b3b400640 1 -- 172.21.15.83:0/3386108139 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b44020df0 con 0x56438d281090 2024-04-24T08:50:49.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.299+0000 7f7b547bf740 1 -- 172.21.15.83:0/3386108139 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56438d281090 msgr2=0x56438d2864b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:49.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.299+0000 7f7b547bf740 1 --2- 172.21.15.83:0/3386108139 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56438d281090 0x56438d2864b0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x56438d2824f0 tx=0x7f7b44015000 comp rx=0 tx=0).stop 2024-04-24T08:50:49.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.299+0000 7f7b547bf740 1 -- 172.21.15.83:0/3386108139 shutdown_connections 2024-04-24T08:50:49.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.299+0000 7f7b547bf740 1 --2- 172.21.15.83:0/3386108139 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56438d286bf0 0x56438d289020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.299+0000 7f7b547bf740 1 --2- 172.21.15.83:0/3386108139 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56438d281a80 0x56438d2929d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.299+0000 7f7b547bf740 1 --2- 172.21.15.83:0/3386108139 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56438d281090 0x56438d2864b0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.299+0000 7f7b547bf740 1 -- 172.21.15.83:0/3386108139 >> 172.21.15.83:0/3386108139 conn(0x56438d27c700 msgr2=0x56438d27cae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:50:49.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.300+0000 7f7b547bf740 1 -- 172.21.15.83:0/3386108139 shutdown_connections 2024-04-24T08:50:49.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.300+0000 7f7b547bf740 1 -- 172.21.15.83:0/3386108139 wait complete. 2024-04-24T08:50:49.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.300+0000 7f7b547bf740 1 -- start start 2024-04-24T08:50:49.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.300+0000 7f7b547bf740 1 Processor -- start 2024-04-24T08:50:49.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.300+0000 7f7b547bf740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56438d281090 0x56438d28ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:49.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.300+0000 7f7b547bf740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56438d281a80 0x56438d28f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:49.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.301+0000 7f7b547bf740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56438d286bf0 0x56438d291d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:49.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.301+0000 7f7b547bf740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56438d2992c0 con 0x56438d281a80 2024-04-24T08:50:49.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.301+0000 7f7b547bf740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56438d298bf0 con 0x56438d286bf0 2024-04-24T08:50:49.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.301+0000 7f7b547bf740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56438d299990 con 0x56438d281090 2024-04-24T08:50:49.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.301+0000 7f7b547bf740 10 client.0 _ll_register_callbacks cb 0x7f7b3402aa90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:50:49.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.301+0000 7f7b4a000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56438d281a80 0x56438d28f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:49.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.301+0000 7f7b4a000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56438d281a80 0x56438d28f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56438/0 (socket says 172.21.15.83:56438) 2024-04-24T08:50:49.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.301+0000 7f7b4a000640 1 -- 172.21.15.83:0/2195242673 learned_addr learned my addr 172.21.15.83:0/2195242673 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:50:49.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.301+0000 7f7b4b400640 1 --2- 172.21.15.83:0/2195242673 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56438d286bf0 0x56438d291d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:49.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.301+0000 7f7b4a000640 1 -- 172.21.15.83:0/2195242673 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56438d281090 msgr2=0x56438d28ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:50:49.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.301+0000 7f7b4aa00640 1 --2- 172.21.15.83:0/2195242673 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56438d281090 0x56438d28ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:49.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.301+0000 7f7b4a000640 1 --2- 172.21.15.83:0/2195242673 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56438d281090 0x56438d28ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.301+0000 7f7b4a000640 1 -- 172.21.15.83:0/2195242673 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56438d286bf0 msgr2=0x56438d291d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:50:49.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.301+0000 7f7b4a000640 1 --2- 172.21.15.83:0/2195242673 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56438d286bf0 0x56438d291d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.301+0000 7f7b4a000640 1 -- 172.21.15.83:0/2195242673 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56438d278060 con 0x56438d281a80 2024-04-24T08:50:49.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.302+0000 7f7b4a000640 1 --2- 172.21.15.83:0/2195242673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56438d281a80 0x56438d28f140 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f7b3402a930 tx=0x7f7b340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d414aed763763ad3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:50:49.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.302+0000 7f7b38c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:50:49.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.302+0000 7f7b38c00640 1 -- 172.21.15.83:0/2195242673 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b34042070 con 0x56438d281a80 2024-04-24T08:50:49.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.302+0000 7f7b38c00640 1 -- 172.21.15.83:0/2195242673 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9237+0+0 (secure 0 0 0) 0x7f7b34003c10 con 0x56438d281a80 2024-04-24T08:50:49.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.302+0000 7f7b547bf740 1 -- 172.21.15.83:0/2195242673 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b4402f070 con 0x56438d281a80 2024-04-24T08:50:49.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.302+0000 7f7b38c00640 1 -- 172.21.15.83:0/2195242673 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7b34062050 con 0x56438d281a80 2024-04-24T08:50:49.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.302+0000 7f7b547bf740 10 client.15291 fetch_fsmap learned FSMap version 27 2024-04-24T08:50:49.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.302+0000 7f7b547bf740 1 -- 172.21.15.83:0/2195242673 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56438d2fe4e0 con 0x56438d281a80 2024-04-24T08:50:49.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.302+0000 7f7b38c00640 1 -- 172.21.15.83:0/2195242673 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7b4402f070 con 0x56438d281a80 2024-04-24T08:50:49.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.303+0000 7f7b38c00640 1 -- 172.21.15.83:0/2195242673 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f7b3403a030 con 0x56438d281a80 2024-04-24T08:50:49.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.303+0000 7f7b547bf740 10 client.15291 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T08:50:49.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.303+0000 7f7b547bf740 10 client.15291 resolve_mds: resolved name 'k' to daemon mds.k 2024-04-24T08:50:49.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.303+0000 7f7b547bf740 20 client.15291 populate_metadata read hostname 'smithi083' 2024-04-24T08:50:49.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.303+0000 7f7b547bf740 1 --2- 172.21.15.83:0/2195242673 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x56438d303110 0x56438d301e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:49.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.303+0000 7f7b547bf740 4 client.15291 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T08:50:49.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.303+0000 7f7b547bf740 1 -- 172.21.15.83:0/2195242673 --> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] -- command(tid 0: {"prefix": "status"}) -- 0x56438d1ee3a0 con 0x56438d303110 2024-04-24T08:50:49.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.303+0000 7f7b4aa00640 1 --2- 172.21.15.83:0/2195242673 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x56438d303110 0x56438d301e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:49.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.304+0000 7f7b4aa00640 1 --2- 172.21.15.83:0/2195242673 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x56438d303110 0x56438d301e20 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3b7f2e1d475d2d5b server_cookie=30a387823d9da86a in_seq=0 out_seq=0 2024-04-24T08:50:49.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.304+0000 7f7b38c00640 10 client.15291 ms_handle_connect on v2:172.21.15.138:6832/1157967658 2024-04-24T08:50:49.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.305+0000 7f7b38c00640 1 -- 172.21.15.83:0/2195242673 <== mds.3 v2:172.21.15.138:6832/1157967658 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x56438d1ee3a0 con 0x56438d303110 2024-04-24T08:50:49.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.305+0000 7f7b38c00640 10 client.15291 handle_command_reply: tid=0 2024-04-24T08:50:49.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.305+0000 7f7b547bf740 1 client.15291 shutdown 2024-04-24T08:50:49.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.305+0000 7f7b547bf740 1 -- 172.21.15.83:0/2195242673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56438d281a80 msgr2=0x56438d28f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:49.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.305+0000 7f7b547bf740 1 --2- 172.21.15.83:0/2195242673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56438d281a80 0x56438d28f140 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f7b3402a930 tx=0x7f7b340046c0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.305+0000 7f7b547bf740 1 -- 172.21.15.83:0/2195242673 shutdown_connections 2024-04-24T08:50:49.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.305+0000 7f7b547bf740 1 --2- 172.21.15.83:0/2195242673 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56438d286bf0 0x56438d291d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.305+0000 7f7b547bf740 1 --2- 172.21.15.83:0/2195242673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56438d281a80 0x56438d28f140 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.305+0000 7f7b547bf740 1 --2- 172.21.15.83:0/2195242673 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x56438d303110 0x56438d301e20 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.305+0000 7f7b547bf740 1 --2- 172.21.15.83:0/2195242673 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56438d281090 0x56438d28ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.305+0000 7f7b547bf740 1 -- 172.21.15.83:0/2195242673 >> 172.21.15.83:0/2195242673 conn(0x56438d27c700 msgr2=0x56438d277750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:50:49.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.305+0000 7f7b38c00640 0 client.15291 ms_handle_reset on v2:172.21.15.138:6832/1157967658 2024-04-24T08:50:49.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.305+0000 7f7b547bf740 1 -- 172.21.15.83:0/2195242673 shutdown_connections 2024-04-24T08:50:49.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.305+0000 7f7b547bf740 1 -- 172.21.15.83:0/2195242673 wait complete. 2024-04-24T08:50:49.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.305+0000 7f7b547bf740 20 client.15291 trim_cache size 0 max 16384 2024-04-24T08:50:49.440 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T08:50:49.440 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_fsid": "bd3e039e-0215-11ef-bc93-c7b262605968", 2024-04-24T08:50:49.440 INFO:teuthology.orchestra.run.smithi083.stdout: "whoami": 3, 2024-04-24T08:50:49.440 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 34427, 2024-04-24T08:50:49.440 INFO:teuthology.orchestra.run.smithi083.stdout: "want_state": "up:active", 2024-04-24T08:50:49.440 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "up:active", 2024-04-24T08:50:49.440 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs", 2024-04-24T08:50:49.441 INFO:teuthology.orchestra.run.smithi083.stdout: "rank_uptime": 106.25130994600001, 2024-04-24T08:50:49.441 INFO:teuthology.orchestra.run.smithi083.stdout: "mdsmap_epoch": 27, 2024-04-24T08:50:49.441 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch": 93, 2024-04-24T08:50:49.441 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch_barrier": 89, 2024-04-24T08:50:49.441 INFO:teuthology.orchestra.run.smithi083.stdout: "uptime": 188.636759692 2024-04-24T08:50:49.441 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T08:50:49.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.306+0000 7f7b51a00640 1 -- 172.21.15.83:0/2612482645 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f7b300a3c30 msgr2=0x7f7b300a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:49.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.306+0000 7f7b51a00640 1 --2- 172.21.15.83:0/2612482645 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f7b300a3c30 0x7f7b300a60f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x56438d281e80 tx=0x7f7b44009580 comp rx=0 tx=0).stop 2024-04-24T08:50:49.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.306+0000 7f7b51a00640 1 -- 172.21.15.83:0/2612482645 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b4c105e30 msgr2=0x7f7b4c1a1a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:49.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.306+0000 7f7b51a00640 1 --2- 172.21.15.83:0/2612482645 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b4c105e30 0x7f7b4c1a1a70 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f7b3c00d750 tx=0x7f7b3c00dc10 comp rx=0 tx=0).stop 2024-04-24T08:50:49.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.306+0000 7f7b51a00640 1 -- 172.21.15.83:0/2612482645 shutdown_connections 2024-04-24T08:50:49.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.306+0000 7f7b51a00640 1 --2- 172.21.15.83:0/2612482645 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f7b300a3c30 0x7f7b300a60f0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.306+0000 7f7b51a00640 1 --2- 172.21.15.83:0/2612482645 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b4c105e30 0x7f7b4c1a1a70 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.306+0000 7f7b51a00640 1 --2- 172.21.15.83:0/2612482645 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7b4c105470 0x7f7b4c19aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.306+0000 7f7b51a00640 1 --2- 172.21.15.83:0/2612482645 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7b4c104280 0x7f7b4c19a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.306+0000 7f7b51a00640 1 -- 172.21.15.83:0/2612482645 >> 172.21.15.83:0/2612482645 conn(0x7f7b4c0ff7f0 msgr2=0x7f7b4c108790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:50:49.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.306+0000 7f7b51a00640 1 -- 172.21.15.83:0/2612482645 shutdown_connections 2024-04-24T08:50:49.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.306+0000 7f7b51a00640 1 -- 172.21.15.83:0/2612482645 wait complete. 2024-04-24T08:50:49.444 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2024-04-24T08:50:49.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:50:49 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/390013927' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T08:50:49.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:50:49 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/1757211451' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:50:49.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:50:49 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/390013927' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T08:50:49.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:50:49 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/1757211451' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:50:49.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.656+0000 7f8861a00640 1 -- 172.21.15.83:0/1891984017 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f885c109850 msgr2=0x7f885c109cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:49.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.656+0000 7f8861a00640 1 --2- 172.21.15.83:0/1891984017 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f885c109850 0x7f885c109cd0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f8848009900 tx=0x7f884802a410 comp rx=0 tx=0).stop 2024-04-24T08:50:49.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.656+0000 7f8861a00640 1 -- 172.21.15.83:0/1891984017 shutdown_connections 2024-04-24T08:50:49.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.656+0000 7f8861a00640 1 --2- 172.21.15.83:0/1891984017 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f885c10a210 0x7f885c110aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.656+0000 7f8861a00640 1 --2- 172.21.15.83:0/1891984017 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f885c109850 0x7f885c109cd0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.656+0000 7f8861a00640 1 --2- 172.21.15.83:0/1891984017 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f885c108660 0x7f885c108a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.656+0000 7f8861a00640 1 -- 172.21.15.83:0/1891984017 >> 172.21.15.83:0/1891984017 conn(0x7f885c077600 msgr2=0x7f885c079a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:50:49.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.656+0000 7f8861a00640 1 -- 172.21.15.83:0/1891984017 shutdown_connections 2024-04-24T08:50:49.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.656+0000 7f8861a00640 1 -- 172.21.15.83:0/1891984017 wait complete. 2024-04-24T08:50:49.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.657+0000 7f8861a00640 1 Processor -- start 2024-04-24T08:50:49.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.658+0000 7f8861a00640 1 -- start start 2024-04-24T08:50:49.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.658+0000 7f8861a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f885c108660 0x7f885c19e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:49.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.658+0000 7f8861a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f885c109850 0x7f885c19ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:49.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.658+0000 7f8861a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f885c10a210 0x7f885c1a5e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:49.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.658+0000 7f8861a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f885c1141c0 con 0x7f885c109850 2024-04-24T08:50:49.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.658+0000 7f8861a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f885c114040 con 0x7f885c108660 2024-04-24T08:50:49.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.658+0000 7f8861a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f885c114340 con 0x7f885c10a210 2024-04-24T08:50:49.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.658+0000 7f885a000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f885c109850 0x7f885c19ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:49.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.659+0000 7f885a000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f885c109850 0x7f885c19ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56468/0 (socket says 172.21.15.83:56468) 2024-04-24T08:50:49.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.659+0000 7f885a000640 1 -- 172.21.15.83:0/471386168 learned_addr learned my addr 172.21.15.83:0/471386168 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:50:49.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.659+0000 7f885aa00640 1 --2- 172.21.15.83:0/471386168 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f885c108660 0x7f885c19e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:49.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.659+0000 7f885b400640 1 --2- 172.21.15.83:0/471386168 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f885c10a210 0x7f885c1a5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:49.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.659+0000 7f885a000640 1 -- 172.21.15.83:0/471386168 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f885c10a210 msgr2=0x7f885c1a5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:49.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.659+0000 7f885a000640 1 --2- 172.21.15.83:0/471386168 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f885c10a210 0x7f885c1a5e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.659+0000 7f885a000640 1 -- 172.21.15.83:0/471386168 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f885c108660 msgr2=0x7f885c19e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:49.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.659+0000 7f885a000640 1 --2- 172.21.15.83:0/471386168 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f885c108660 0x7f885c19e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.659+0000 7f885a000640 1 -- 172.21.15.83:0/471386168 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f885c1a64f0 con 0x7f885c109850 2024-04-24T08:50:49.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.659+0000 7f885aa00640 1 --2- 172.21.15.83:0/471386168 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f885c108660 0x7f885c19e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:50:49.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.659+0000 7f885a000640 1 --2- 172.21.15.83:0/471386168 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f885c109850 0x7f885c19ee60 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f88480314e0 tx=0x7f8848003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:50:49.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.660+0000 7f8853400640 1 -- 172.21.15.83:0/471386168 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f884803f070 con 0x7f885c109850 2024-04-24T08:50:49.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.660+0000 7f8853400640 1 -- 172.21.15.83:0/471386168 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8848031e40 con 0x7f885c109850 2024-04-24T08:50:49.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.660+0000 7f8861a00640 1 -- 172.21.15.83:0/471386168 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f885c1a6780 con 0x7f885c109850 2024-04-24T08:50:49.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.660+0000 7f8853400640 1 -- 172.21.15.83:0/471386168 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8848035030 con 0x7f885c109850 2024-04-24T08:50:49.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.660+0000 7f8861a00640 1 -- 172.21.15.83:0/471386168 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f885c1a6c70 con 0x7f885c109850 2024-04-24T08:50:49.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.663+0000 7f8864813740 1 Processor -- start 2024-04-24T08:50:49.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.663+0000 7f8864813740 1 -- start start 2024-04-24T08:50:49.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.664+0000 7f8864813740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d9f8134090 0x55d9f81344b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:49.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.664+0000 7f8864813740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9f8134a80 0x55d9f8139580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:49.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.664+0000 7f8853400640 1 -- 172.21.15.83:0/471386168 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f8848042070 con 0x7f885c109850 2024-04-24T08:50:49.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.664+0000 7f8864813740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d9f8139bf0 0x55d9f813c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:49.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.664+0000 7f8864813740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d9f8099b10 con 0x55d9f8134a80 2024-04-24T08:50:49.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.664+0000 7f8864813740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d9f80402b0 con 0x55d9f8139bf0 2024-04-24T08:50:49.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.664+0000 7f8864813740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d9f8095ad0 con 0x55d9f8134090 2024-04-24T08:50:49.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.664+0000 7f885aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9f8134a80 0x55d9f8139580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:49.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.664+0000 7f885aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9f8134a80 0x55d9f8139580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56482/0 (socket says 172.21.15.83:56482) 2024-04-24T08:50:49.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.664+0000 7f885aa00640 1 -- 172.21.15.83:0/2573036310 learned_addr learned my addr 172.21.15.83:0/2573036310 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:50:49.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.664+0000 7f885a000640 1 --2- 172.21.15.83:0/2573036310 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d9f8139bf0 0x55d9f813c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:49.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.664+0000 7f885b400640 1 --2- 172.21.15.83:0/2573036310 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d9f8134090 0x55d9f81344b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:49.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.665+0000 7f885aa00640 1 -- 172.21.15.83:0/2573036310 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d9f8134090 msgr2=0x55d9f81344b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:50:49.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.665+0000 7f885aa00640 1 --2- 172.21.15.83:0/2573036310 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d9f8134090 0x55d9f81344b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.665+0000 7f885aa00640 1 -- 172.21.15.83:0/2573036310 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d9f8139bf0 msgr2=0x55d9f813c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:50:49.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.665+0000 7f885aa00640 1 --2- 172.21.15.83:0/2573036310 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d9f8139bf0 0x55d9f813c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.665+0000 7f885aa00640 1 -- 172.21.15.83:0/2573036310 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9f7dac2d0 con 0x55d9f8134a80 2024-04-24T08:50:49.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.665+0000 7f885aa00640 1 --2- 172.21.15.83:0/2573036310 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9f8134a80 0x55d9f8139580 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f884c00aed0 tx=0x7f884c00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6ba1a6c6d54e7ab server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:50:49.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.665+0000 7f8847e00640 1 -- 172.21.15.83:0/2573036310 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f884c004d20 con 0x55d9f8134a80 2024-04-24T08:50:49.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.665+0000 7f8847e00640 1 -- 172.21.15.83:0/2573036310 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f884c011b00 con 0x55d9f8134a80 2024-04-24T08:50:49.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.665+0000 7f8847e00640 1 -- 172.21.15.83:0/2573036310 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f884c005540 con 0x55d9f8134a80 2024-04-24T08:50:49.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.666+0000 7f8853400640 1 --2- 172.21.15.83:0/471386168 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f88400a3bb0 0x7f88400a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:49.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.666+0000 7f8853400640 1 -- 172.21.15.83:0/471386168 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9237+0+0 (secure 0 0 0) 0x7f88480f1340 con 0x7f885c109850 2024-04-24T08:50:49.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.666+0000 7f885b400640 1 --2- 172.21.15.83:0/471386168 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f88400a3bb0 0x7f88400a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:49.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.666+0000 7f8864813740 1 -- 172.21.15.83:0/2573036310 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9f8134a80 msgr2=0x55d9f8139580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:49.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.666+0000 7f8864813740 1 --2- 172.21.15.83:0/2573036310 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9f8134a80 0x55d9f8139580 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f884c00aed0 tx=0x7f884c00f730 comp rx=0 tx=0).stop 2024-04-24T08:50:49.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.666+0000 7f8864813740 1 -- 172.21.15.83:0/2573036310 shutdown_connections 2024-04-24T08:50:49.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.666+0000 7f8864813740 1 --2- 172.21.15.83:0/2573036310 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d9f8139bf0 0x55d9f813c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.666+0000 7f8864813740 1 --2- 172.21.15.83:0/2573036310 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9f8134a80 0x55d9f8139580 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.666+0000 7f8864813740 1 --2- 172.21.15.83:0/2573036310 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d9f8134090 0x55d9f81344b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.666+0000 7f8864813740 1 -- 172.21.15.83:0/2573036310 >> 172.21.15.83:0/2573036310 conn(0x55d9f812f700 msgr2=0x55d9f812fb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:50:49.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.667+0000 7f8864813740 1 -- 172.21.15.83:0/2573036310 shutdown_connections 2024-04-24T08:50:49.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.667+0000 7f885b400640 1 --2- 172.21.15.83:0/471386168 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f88400a3bb0 0x7f88400a6070 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f8854005fd0 tx=0x7f8854005e20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:50:49.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.667+0000 7f8864813740 1 -- 172.21.15.83:0/2573036310 wait complete. 2024-04-24T08:50:49.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.667+0000 7f8864813740 1 -- start start 2024-04-24T08:50:49.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.667+0000 7f8864813740 1 Processor -- start 2024-04-24T08:50:49.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.668+0000 7f8864813740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d9f8134090 0x55d9f813e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:49.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.668+0000 7f8864813740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d9f8134a80 0x55d9f813ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:49.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.668+0000 7f8864813740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9f8139bf0 0x55d9f8145a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:49.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.668+0000 7f8864813740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d9f813cba0 con 0x55d9f8139bf0 2024-04-24T08:50:49.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.668+0000 7f8864813740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d9f8053590 con 0x55d9f8134a80 2024-04-24T08:50:49.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.668+0000 7f8864813740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d9f813cd20 con 0x55d9f8134090 2024-04-24T08:50:49.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.668+0000 7f8864813740 10 client.0 _ll_register_callbacks cb 0x55d9f80363a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:50:49.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.668+0000 7f885b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9f8139bf0 0x55d9f8145a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:49.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.668+0000 7f885b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9f8139bf0 0x55d9f8145a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56488/0 (socket says 172.21.15.83:56488) 2024-04-24T08:50:49.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.668+0000 7f885b400640 1 -- 172.21.15.83:0/1605957298 learned_addr learned my addr 172.21.15.83:0/1605957298 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:50:49.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.668+0000 7f885a000640 1 --2- 172.21.15.83:0/1605957298 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d9f8134a80 0x55d9f813ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:49.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.668+0000 7f885aa00640 1 --2- 172.21.15.83:0/1605957298 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d9f8134090 0x55d9f813e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:49.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.669+0000 7f885b400640 1 -- 172.21.15.83:0/1605957298 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d9f8134090 msgr2=0x55d9f813e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:50:49.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.669+0000 7f885b400640 1 --2- 172.21.15.83:0/1605957298 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d9f8134090 0x55d9f813e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.669+0000 7f885b400640 1 -- 172.21.15.83:0/1605957298 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d9f8134a80 msgr2=0x55d9f813ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:50:49.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.669+0000 7f885b400640 1 --2- 172.21.15.83:0/1605957298 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d9f8134a80 0x55d9f813ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.669+0000 7f885b400640 1 -- 172.21.15.83:0/1605957298 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9f7e766b0 con 0x55d9f8139bf0 2024-04-24T08:50:49.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.669+0000 7f885b400640 1 --2- 172.21.15.83:0/1605957298 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9f8139bf0 0x55d9f8145a90 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f8854011a90 tx=0x7f88540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c2d8370ef6c36d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:50:49.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.669+0000 7f8844c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:50:49.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.669+0000 7f8844c00640 1 -- 172.21.15.83:0/1605957298 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f885400ee10 con 0x55d9f8139bf0 2024-04-24T08:50:49.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.669+0000 7f8844c00640 1 -- 172.21.15.83:0/1605957298 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9237+0+0 (secure 0 0 0) 0x7f885401e420 con 0x55d9f8139bf0 2024-04-24T08:50:49.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.669+0000 7f8864813740 1 -- 172.21.15.83:0/1605957298 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f884c011b00 con 0x55d9f8139bf0 2024-04-24T08:50:49.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.669+0000 7f8844c00640 1 -- 172.21.15.83:0/1605957298 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f885402f050 con 0x55d9f8139bf0 2024-04-24T08:50:49.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.669+0000 7f8864813740 10 client.15303 fetch_fsmap learned FSMap version 27 2024-04-24T08:50:49.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.669+0000 7f8864813740 1 -- 172.21.15.83:0/1605957298 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d9f8148ed0 con 0x55d9f8139bf0 2024-04-24T08:50:49.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.670+0000 7f8844c00640 1 -- 172.21.15.83:0/1605957298 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f884c011b00 con 0x55d9f8139bf0 2024-04-24T08:50:49.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.670+0000 7f8844c00640 1 -- 172.21.15.83:0/1605957298 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f885401c590 con 0x55d9f8139bf0 2024-04-24T08:50:49.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.670+0000 7f8864813740 10 client.15303 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T08:50:49.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.670+0000 7f8864813740 10 client.15303 resolve_mds: resolved name 'e' to daemon mds.e 2024-04-24T08:50:49.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.670+0000 7f8864813740 20 client.15303 populate_metadata read hostname 'smithi083' 2024-04-24T08:50:49.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.670+0000 7f8864813740 1 --2- 172.21.15.83:0/1605957298 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55d9f814b5f0 0x55d9f814ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:49.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.670+0000 7f8864813740 4 client.15303 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:50:49.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.670+0000 7f8864813740 1 -- 172.21.15.83:0/1605957298 --> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d9f80a7530 con 0x55d9f814b5f0 2024-04-24T08:50:49.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.671+0000 7f885aa00640 1 --2- 172.21.15.83:0/1605957298 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55d9f814b5f0 0x55d9f814ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:49.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.672+0000 7f885aa00640 1 --2- 172.21.15.83:0/1605957298 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55d9f814b5f0 0x55d9f814ca60 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=62767f77197fc02 server_cookie=6f02738657d889a in_seq=0 out_seq=0 2024-04-24T08:50:49.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.672+0000 7f8844c00640 10 client.15303 ms_handle_connect on v2:172.21.15.138:6834/1909219758 2024-04-24T08:50:49.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.674+0000 7f8844c00640 1 -- 172.21.15.83:0/1605957298 <== mds.2 v2:172.21.15.138:6834/1909219758 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d9f80a7530 con 0x55d9f814b5f0 2024-04-24T08:50:49.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.674+0000 7f8844c00640 10 client.15303 handle_command_reply: tid=0 2024-04-24T08:50:49.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.674+0000 7f8864813740 1 client.15303 shutdown 2024-04-24T08:50:49.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.675+0000 7f8864813740 1 -- 172.21.15.83:0/1605957298 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9f8139bf0 msgr2=0x55d9f8145a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:49.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.675+0000 7f8864813740 1 --2- 172.21.15.83:0/1605957298 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9f8139bf0 0x55d9f8145a90 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f8854011a90 tx=0x7f88540046c0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.675+0000 7f8864813740 1 -- 172.21.15.83:0/1605957298 shutdown_connections 2024-04-24T08:50:49.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.675+0000 7f8864813740 1 --2- 172.21.15.83:0/1605957298 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55d9f814b5f0 0x55d9f814ca60 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.675+0000 7f8864813740 1 --2- 172.21.15.83:0/1605957298 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9f8139bf0 0x55d9f8145a90 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.675+0000 7f8864813740 1 --2- 172.21.15.83:0/1605957298 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d9f8134a80 0x55d9f813ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.675+0000 7f8844c00640 0 client.15303 ms_handle_reset on v2:172.21.15.138:6834/1909219758 2024-04-24T08:50:49.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.675+0000 7f8864813740 1 --2- 172.21.15.83:0/1605957298 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d9f8134090 0x55d9f813e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.675+0000 7f8864813740 1 -- 172.21.15.83:0/1605957298 >> 172.21.15.83:0/1605957298 conn(0x55d9f812f700 msgr2=0x55d9f812e8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:50:49.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.675+0000 7f8864813740 1 -- 172.21.15.83:0/1605957298 shutdown_connections 2024-04-24T08:50:49.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.675+0000 7f8864813740 1 -- 172.21.15.83:0/1605957298 wait complete. 2024-04-24T08:50:49.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.675+0000 7f8864813740 20 client.15303 trim_cache size 0 max 16384 2024-04-24T08:50:49.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.694+0000 7f8864813740 1 Processor -- start 2024-04-24T08:50:49.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.694+0000 7f8864813740 1 -- start start 2024-04-24T08:50:49.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.695+0000 7f8864813740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d9f8134090 0x55d9f8135520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:49.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.695+0000 7f8864813740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9f8134a80 0x55d9f8145a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:49.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.695+0000 7f8864813740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d9f8139bf0 0x55d9f81394b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:49.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.695+0000 7f8864813740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d9f814ada0 con 0x55d9f8134a80 2024-04-24T08:50:49.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.695+0000 7f8864813740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d9f814b470 con 0x55d9f8139bf0 2024-04-24T08:50:49.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.695+0000 7f8864813740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d9f814bb40 con 0x55d9f8134090 2024-04-24T08:50:49.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.695+0000 7f885a000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9f8134a80 0x55d9f8145a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:49.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.695+0000 7f885a000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9f8134a80 0x55d9f8145a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56500/0 (socket says 172.21.15.83:56500) 2024-04-24T08:50:49.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.695+0000 7f885a000640 1 -- 172.21.15.83:0/2851860720 learned_addr learned my addr 172.21.15.83:0/2851860720 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:50:49.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.695+0000 7f885aa00640 1 --2- 172.21.15.83:0/2851860720 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d9f8134090 0x55d9f8135520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:49.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.695+0000 7f885b400640 1 --2- 172.21.15.83:0/2851860720 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d9f8139bf0 0x55d9f81394b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:49.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.696+0000 7f885a000640 1 -- 172.21.15.83:0/2851860720 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d9f8134090 msgr2=0x55d9f8135520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:50:49.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.696+0000 7f885a000640 1 --2- 172.21.15.83:0/2851860720 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d9f8134090 0x55d9f8135520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.696+0000 7f885a000640 1 -- 172.21.15.83:0/2851860720 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d9f8139bf0 msgr2=0x55d9f81394b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:50:49.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.696+0000 7f885a000640 1 --2- 172.21.15.83:0/2851860720 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d9f8139bf0 0x55d9f81394b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.696+0000 7f885a000640 1 -- 172.21.15.83:0/2851860720 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9f8133bf0 con 0x55d9f8134a80 2024-04-24T08:50:49.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.696+0000 7f885a000640 1 --2- 172.21.15.83:0/2851860720 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9f8134a80 0x55d9f8145a50 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f88480b9ba0 tx=0x7f8848005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=820119c9a485e1a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:50:49.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.696+0000 7f8847400640 1 -- 172.21.15.83:0/2851860720 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8848049040 con 0x55d9f8134a80 2024-04-24T08:50:49.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.697+0000 7f8847400640 1 -- 172.21.15.83:0/2851860720 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f88480bd8d0 con 0x55d9f8134a80 2024-04-24T08:50:49.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.697+0000 7f8847400640 1 -- 172.21.15.83:0/2851860720 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88480bb560 con 0x55d9f8134a80 2024-04-24T08:50:49.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.697+0000 7f8864813740 1 -- 172.21.15.83:0/2851860720 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9f8134a80 msgr2=0x55d9f8145a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:49.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.697+0000 7f8864813740 1 --2- 172.21.15.83:0/2851860720 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9f8134a80 0x55d9f8145a50 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f88480b9ba0 tx=0x7f8848005fd0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.697+0000 7f8864813740 1 -- 172.21.15.83:0/2851860720 shutdown_connections 2024-04-24T08:50:49.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.697+0000 7f8864813740 1 --2- 172.21.15.83:0/2851860720 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d9f8139bf0 0x55d9f81394b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.697+0000 7f8864813740 1 --2- 172.21.15.83:0/2851860720 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9f8134a80 0x55d9f8145a50 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.697+0000 7f8864813740 1 --2- 172.21.15.83:0/2851860720 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d9f8134090 0x55d9f8135520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.697+0000 7f8864813740 1 -- 172.21.15.83:0/2851860720 >> 172.21.15.83:0/2851860720 conn(0x55d9f812f700 msgr2=0x55d9f812fae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:50:49.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.697+0000 7f8864813740 1 -- 172.21.15.83:0/2851860720 shutdown_connections 2024-04-24T08:50:49.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.697+0000 7f8864813740 1 -- 172.21.15.83:0/2851860720 wait complete. 2024-04-24T08:50:49.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.698+0000 7f8864813740 1 -- start start 2024-04-24T08:50:49.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.698+0000 7f8864813740 1 Processor -- start 2024-04-24T08:50:49.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.698+0000 7f8864813740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d9f8134090 0x55d9f8143a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:49.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.698+0000 7f8864813740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9f8134a80 0x55d9f81ad580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:49.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.698+0000 7f8864813740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d9f8139bf0 0x55d9f81afad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:49.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.698+0000 7f8864813740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d9f814c8e0 con 0x55d9f8134a80 2024-04-24T08:50:49.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.698+0000 7f8864813740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d9f814c210 con 0x55d9f8139bf0 2024-04-24T08:50:49.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.698+0000 7f8864813740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d9f814cfb0 con 0x55d9f8134090 2024-04-24T08:50:49.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.698+0000 7f8864813740 10 client.0 _ll_register_callbacks cb 0x7f8854011740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:50:49.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.699+0000 7f885a000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9f8134a80 0x55d9f81ad580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:49.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.699+0000 7f885a000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9f8134a80 0x55d9f81ad580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56516/0 (socket says 172.21.15.83:56516) 2024-04-24T08:50:49.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.699+0000 7f885a000640 1 -- 172.21.15.83:0/1411600073 learned_addr learned my addr 172.21.15.83:0/1411600073 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:50:49.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.699+0000 7f885a000640 1 -- 172.21.15.83:0/1411600073 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d9f8134090 msgr2=0x55d9f8143a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:50:49.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.699+0000 7f885a000640 1 --2- 172.21.15.83:0/1411600073 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d9f8134090 0x55d9f8143a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.699+0000 7f885a000640 1 -- 172.21.15.83:0/1411600073 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d9f8139bf0 msgr2=0x55d9f81afad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:50:49.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.699+0000 7f885a000640 1 --2- 172.21.15.83:0/1411600073 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d9f8139bf0 0x55d9f81afad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.699+0000 7f885a000640 1 -- 172.21.15.83:0/1411600073 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9f81b00c0 con 0x55d9f8134a80 2024-04-24T08:50:49.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.699+0000 7f885a000640 1 --2- 172.21.15.83:0/1411600073 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9f8134a80 0x55d9f81ad580 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f8848000c00 tx=0x7f8848005e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91a22bd615518ec9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:50:49.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.699+0000 7f8844c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:50:49.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.700+0000 7f8844c00640 1 -- 172.21.15.83:0/1411600073 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8848049050 con 0x55d9f8134a80 2024-04-24T08:50:49.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.700+0000 7f8844c00640 1 -- 172.21.15.83:0/1411600073 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9237+0+0 (secure 0 0 0) 0x7f88480b9840 con 0x55d9f8134a80 2024-04-24T08:50:49.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.700+0000 7f8864813740 1 -- 172.21.15.83:0/1411600073 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88480bd8d0 con 0x55d9f8134a80 2024-04-24T08:50:49.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.700+0000 7f8844c00640 1 -- 172.21.15.83:0/1411600073 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f88480bd1d0 con 0x55d9f8134a80 2024-04-24T08:50:49.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.700+0000 7f8864813740 10 client.15309 fetch_fsmap learned FSMap version 27 2024-04-24T08:50:49.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.700+0000 7f8864813740 1 -- 172.21.15.83:0/1411600073 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d9f81b13c0 con 0x55d9f8134a80 2024-04-24T08:50:49.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.700+0000 7f8844c00640 1 -- 172.21.15.83:0/1411600073 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f88480bd8d0 con 0x55d9f8134a80 2024-04-24T08:50:49.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.700+0000 7f8844c00640 1 -- 172.21.15.83:0/1411600073 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f88480c11f0 con 0x55d9f8134a80 2024-04-24T08:50:49.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.700+0000 7f8864813740 10 client.15309 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T08:50:49.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.700+0000 7f8864813740 10 client.15309 resolve_mds: resolved name 'e' to daemon mds.e 2024-04-24T08:50:49.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.700+0000 7f8864813740 20 client.15309 populate_metadata read hostname 'smithi083' 2024-04-24T08:50:49.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.700+0000 7f8864813740 1 --2- 172.21.15.83:0/1411600073 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55d9f81b5e90 0x55d9f81b4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:49.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.700+0000 7f8864813740 4 client.15309 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T08:50:49.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.700+0000 7f8864813740 1 -- 172.21.15.83:0/1411600073 --> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] -- command(tid 0: {"prefix": "status"}) -- 0x55d9f80a13a0 con 0x55d9f81b5e90 2024-04-24T08:50:49.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.701+0000 7f885aa00640 1 --2- 172.21.15.83:0/1411600073 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55d9f81b5e90 0x55d9f81b4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:49.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.702+0000 7f885aa00640 1 --2- 172.21.15.83:0/1411600073 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55d9f81b5e90 0x55d9f81b4ba0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6e7b5cd8902d59a7 server_cookie=2ae89d70efcf04fa in_seq=0 out_seq=0 2024-04-24T08:50:49.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.702+0000 7f8844c00640 10 client.15309 ms_handle_connect on v2:172.21.15.138:6834/1909219758 2024-04-24T08:50:49.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.703+0000 7f8844c00640 1 -- 172.21.15.83:0/1411600073 <== mds.2 v2:172.21.15.138:6834/1909219758 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55d9f80a13a0 con 0x55d9f81b5e90 2024-04-24T08:50:49.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.703+0000 7f8844c00640 10 client.15309 handle_command_reply: tid=0 2024-04-24T08:50:49.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.703+0000 7f8864813740 1 client.15309 shutdown 2024-04-24T08:50:49.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.703+0000 7f8864813740 1 -- 172.21.15.83:0/1411600073 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9f8134a80 msgr2=0x55d9f81ad580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:49.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.703+0000 7f8864813740 1 --2- 172.21.15.83:0/1411600073 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9f8134a80 0x55d9f81ad580 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f8848000c00 tx=0x7f8848005e20 comp rx=0 tx=0).stop 2024-04-24T08:50:49.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.703+0000 7f8864813740 1 -- 172.21.15.83:0/1411600073 shutdown_connections 2024-04-24T08:50:49.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.703+0000 7f8864813740 1 --2- 172.21.15.83:0/1411600073 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55d9f81b5e90 0x55d9f81b4ba0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.703+0000 7f8864813740 1 --2- 172.21.15.83:0/1411600073 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d9f8139bf0 0x55d9f81afad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.703+0000 7f8864813740 1 --2- 172.21.15.83:0/1411600073 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9f8134a80 0x55d9f81ad580 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.703+0000 7f8844c00640 0 client.15309 ms_handle_reset on v2:172.21.15.138:6834/1909219758 2024-04-24T08:50:49.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.703+0000 7f8864813740 1 --2- 172.21.15.83:0/1411600073 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d9f8134090 0x55d9f8143a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.703+0000 7f8864813740 1 -- 172.21.15.83:0/1411600073 >> 172.21.15.83:0/1411600073 conn(0x55d9f812f700 msgr2=0x55d9f812a750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:50:49.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.703+0000 7f8864813740 1 -- 172.21.15.83:0/1411600073 shutdown_connections 2024-04-24T08:50:49.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.703+0000 7f8864813740 1 -- 172.21.15.83:0/1411600073 wait complete. 2024-04-24T08:50:49.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.703+0000 7f8864813740 20 client.15309 trim_cache size 0 max 16384 2024-04-24T08:50:49.707 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T08:50:49.707 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_fsid": "bd3e039e-0215-11ef-bc93-c7b262605968", 2024-04-24T08:50:49.707 INFO:teuthology.orchestra.run.smithi083.stdout: "whoami": 2, 2024-04-24T08:50:49.708 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 34451, 2024-04-24T08:50:49.708 INFO:teuthology.orchestra.run.smithi083.stdout: "want_state": "up:active", 2024-04-24T08:50:49.708 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "up:active", 2024-04-24T08:50:49.708 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs", 2024-04-24T08:50:49.708 INFO:teuthology.orchestra.run.smithi083.stdout: "rank_uptime": 107.671038123, 2024-04-24T08:50:49.708 INFO:teuthology.orchestra.run.smithi083.stdout: "mdsmap_epoch": 27, 2024-04-24T08:50:49.708 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch": 93, 2024-04-24T08:50:49.708 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch_barrier": 89, 2024-04-24T08:50:49.708 INFO:teuthology.orchestra.run.smithi083.stdout: "uptime": 176.64976704899999 2024-04-24T08:50:49.708 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T08:50:49.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.704+0000 7f8861a00640 1 -- 172.21.15.83:0/471386168 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f88400a3bb0 msgr2=0x7f88400a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:49.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.704+0000 7f8861a00640 1 --2- 172.21.15.83:0/471386168 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f88400a3bb0 0x7f88400a6070 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f8854005fd0 tx=0x7f8854005e20 comp rx=0 tx=0).stop 2024-04-24T08:50:49.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.704+0000 7f8861a00640 1 -- 172.21.15.83:0/471386168 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f885c109850 msgr2=0x7f885c19ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:49.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.704+0000 7f8861a00640 1 --2- 172.21.15.83:0/471386168 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f885c109850 0x7f885c19ee60 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f88480314e0 tx=0x7f8848003d50 comp rx=0 tx=0).stop 2024-04-24T08:50:49.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.704+0000 7f8861a00640 1 -- 172.21.15.83:0/471386168 shutdown_connections 2024-04-24T08:50:49.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.704+0000 7f8861a00640 1 --2- 172.21.15.83:0/471386168 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f885c10a210 0x7f885c1a5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.704+0000 7f8861a00640 1 --2- 172.21.15.83:0/471386168 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f88400a3bb0 0x7f88400a6070 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.704+0000 7f8861a00640 1 --2- 172.21.15.83:0/471386168 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f885c109850 0x7f885c19ee60 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.704+0000 7f8861a00640 1 --2- 172.21.15.83:0/471386168 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f885c108660 0x7f885c19e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.704+0000 7f8861a00640 1 -- 172.21.15.83:0/471386168 >> 172.21.15.83:0/471386168 conn(0x7f885c077600 msgr2=0x7f885c10cc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:50:49.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.704+0000 7f8861a00640 1 -- 172.21.15.83:0/471386168 shutdown_connections 2024-04-24T08:50:49.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.704+0000 7f8861a00640 1 -- 172.21.15.83:0/471386168 wait complete. 2024-04-24T08:50:49.721 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-04-24T08:50:49.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.919+0000 7f4ec4e00640 1 -- 172.21.15.83:0/3027940705 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ec0105e10 msgr2=0x7f4ec010c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:49.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.919+0000 7f4ec4e00640 1 --2- 172.21.15.83:0/3027940705 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ec0105e10 0x7f4ec010c6a0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f4eb8009ca0 tx=0x7f4eb802a8c0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.919+0000 7f4ec4e00640 1 -- 172.21.15.83:0/3027940705 shutdown_connections 2024-04-24T08:50:49.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.919+0000 7f4ec4e00640 1 --2- 172.21.15.83:0/3027940705 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ec0105e10 0x7f4ec010c6a0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.919+0000 7f4ec4e00640 1 --2- 172.21.15.83:0/3027940705 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4ec0105450 0x7f4ec01058d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.919+0000 7f4ec4e00640 1 --2- 172.21.15.83:0/3027940705 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ec0104260 0x7f4ec0104680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.919+0000 7f4ec4e00640 1 -- 172.21.15.83:0/3027940705 >> 172.21.15.83:0/3027940705 conn(0x7f4ec00ff7f0 msgr2=0x7f4ec0101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:50:49.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.919+0000 7f4ec4e00640 1 -- 172.21.15.83:0/3027940705 shutdown_connections 2024-04-24T08:50:49.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.919+0000 7f4ec4e00640 1 -- 172.21.15.83:0/3027940705 wait complete. 2024-04-24T08:50:49.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.920+0000 7f4ec4e00640 1 Processor -- start 2024-04-24T08:50:49.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.920+0000 7f4ec4e00640 1 -- start start 2024-04-24T08:50:49.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.920+0000 7f4ec4e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ec0104260 0x7f4ec019a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:49.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.920+0000 7f4ec4e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ec0105450 0x7f4ec019a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:49.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.920+0000 7f4ec4e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4ec0105e10 0x7f4ec01a1a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:49.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.920+0000 7f4ec4e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec010fdf0 con 0x7f4ec0105450 2024-04-24T08:50:49.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.920+0000 7f4ec4e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec010fc70 con 0x7f4ec0104260 2024-04-24T08:50:49.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.920+0000 7f4ec4e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec010ff70 con 0x7f4ec0105e10 2024-04-24T08:50:49.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.921+0000 7f4eb5400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ec0105450 0x7f4ec019a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:49.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.921+0000 7f4eb5400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ec0105450 0x7f4ec019a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56538/0 (socket says 172.21.15.83:56538) 2024-04-24T08:50:49.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.921+0000 7f4eb5400640 1 -- 172.21.15.83:0/1406389671 learned_addr learned my addr 172.21.15.83:0/1406389671 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:50:49.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.921+0000 7f4ebde00640 1 --2- 172.21.15.83:0/1406389671 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ec0104260 0x7f4ec019a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:49.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.921+0000 7f4ebe800640 1 --2- 172.21.15.83:0/1406389671 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4ec0105e10 0x7f4ec01a1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:49.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.921+0000 7f4eb5400640 1 -- 172.21.15.83:0/1406389671 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4ec0105e10 msgr2=0x7f4ec01a1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:49.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.921+0000 7f4eb5400640 1 --2- 172.21.15.83:0/1406389671 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4ec0105e10 0x7f4ec01a1a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.921+0000 7f4eb5400640 1 -- 172.21.15.83:0/1406389671 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ec0104260 msgr2=0x7f4ec019a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:49.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.921+0000 7f4eb5400640 1 --2- 172.21.15.83:0/1406389671 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ec0104260 0x7f4ec019a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.921+0000 7f4eb5400640 1 -- 172.21.15.83:0/1406389671 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ec01a2070 con 0x7f4ec0105450 2024-04-24T08:50:49.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.921+0000 7f4ebde00640 1 --2- 172.21.15.83:0/1406389671 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ec0104260 0x7f4ec019a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:50:49.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.922+0000 7f4eb5400640 1 --2- 172.21.15.83:0/1406389671 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ec0105450 0x7f4ec019a990 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f4eb000d770 tx=0x7f4eb000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:50:49.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.922+0000 7f4eb7400640 1 -- 172.21.15.83:0/1406389671 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb0017070 con 0x7f4ec0105450 2024-04-24T08:50:49.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.922+0000 7f4eb7400640 1 -- 172.21.15.83:0/1406389671 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4eb00054d0 con 0x7f4ec0105450 2024-04-24T08:50:49.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.922+0000 7f4ec4e00640 1 -- 172.21.15.83:0/1406389671 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ec01a2360 con 0x7f4ec0105450 2024-04-24T08:50:49.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.922+0000 7f4eb7400640 1 -- 172.21.15.83:0/1406389671 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb0004cf0 con 0x7f4ec0105450 2024-04-24T08:50:49.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.922+0000 7f4ec4e00640 1 -- 172.21.15.83:0/1406389671 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ec01a2860 con 0x7f4ec0105450 2024-04-24T08:50:49.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.925+0000 7f4ec7dc6740 1 Processor -- start 2024-04-24T08:50:49.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.925+0000 7f4eb7400640 1 -- 172.21.15.83:0/1406389671 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f4eb0004e90 con 0x7f4ec0105450 2024-04-24T08:50:49.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.925+0000 7f4ec7dc6740 1 -- start start 2024-04-24T08:50:49.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.926+0000 7f4ec7dc6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b175f14090 0x55b175f144b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:49.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.926+0000 7f4ec7dc6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b175f14a80 0x55b175f19580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:49.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.926+0000 7f4ec7dc6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b175f19bf0 0x55b175f1c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:49.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.926+0000 7f4ec7dc6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55b175e79b10 con 0x55b175f19bf0 2024-04-24T08:50:49.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.926+0000 7f4ec7dc6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b175e202b0 con 0x55b175f14a80 2024-04-24T08:50:49.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.926+0000 7f4ec7dc6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55b175e75ad0 con 0x55b175f14090 2024-04-24T08:50:49.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.926+0000 7f4eb5400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b175f19bf0 0x55b175f1c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:49.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.926+0000 7f4eb5400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b175f19bf0 0x55b175f1c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56554/0 (socket says 172.21.15.83:56554) 2024-04-24T08:50:49.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.926+0000 7f4eb5400640 1 -- 172.21.15.83:0/3025330288 learned_addr learned my addr 172.21.15.83:0/3025330288 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:50:49.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.926+0000 7f4ebde00640 1 --2- 172.21.15.83:0/3025330288 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b175f14a80 0x55b175f19580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:49.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.926+0000 7f4ebe800640 1 --2- 172.21.15.83:0/3025330288 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b175f14090 0x55b175f144b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:49.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.926+0000 7f4eb5400640 1 -- 172.21.15.83:0/3025330288 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b175f14090 msgr2=0x55b175f144b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:50:49.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.926+0000 7f4eb5400640 1 --2- 172.21.15.83:0/3025330288 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b175f14090 0x55b175f144b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.927+0000 7f4eb5400640 1 -- 172.21.15.83:0/3025330288 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b175f14a80 msgr2=0x55b175f19580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:50:49.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.927+0000 7f4eb5400640 1 --2- 172.21.15.83:0/3025330288 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b175f14a80 0x55b175f19580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.927+0000 7f4eb5400640 1 -- 172.21.15.83:0/3025330288 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b175b8c2d0 con 0x55b175f19bf0 2024-04-24T08:50:49.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.927+0000 7f4eb7400640 1 --2- 172.21.15.83:0/1406389671 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f4ea40a3bb0 0x7f4ea40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:49.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.927+0000 7f4eb5400640 1 --2- 172.21.15.83:0/3025330288 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b175f19bf0 0x55b175f1c080 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f4eb00d85d0 tx=0x7f4eb00d8a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4359e5877871fb5a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:50:49.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.927+0000 7f4ebe800640 1 --2- 172.21.15.83:0/1406389671 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f4ea40a3bb0 0x7f4ea40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:49.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.927+0000 7f4eb7400640 1 -- 172.21.15.83:0/1406389671 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9237+0+0 (secure 0 0 0) 0x7f4eb00d0370 con 0x7f4ec0105450 2024-04-24T08:50:49.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.927+0000 7f4eaf400640 1 -- 172.21.15.83:0/3025330288 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb00984a0 con 0x55b175f19bf0 2024-04-24T08:50:49.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.927+0000 7f4eaf400640 1 -- 172.21.15.83:0/3025330288 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4eb0009c30 con 0x55b175f19bf0 2024-04-24T08:50:49.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.927+0000 7f4eaf400640 1 -- 172.21.15.83:0/3025330288 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb0005180 con 0x55b175f19bf0 2024-04-24T08:50:49.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.928+0000 7f4ec7dc6740 1 -- 172.21.15.83:0/3025330288 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b175f19bf0 msgr2=0x55b175f1c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:49.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.928+0000 7f4ebe800640 1 --2- 172.21.15.83:0/1406389671 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f4ea40a3bb0 0x7f4ea40a6070 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4eb8006010 tx=0x7f4eb8031a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:50:49.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.928+0000 7f4ec7dc6740 1 --2- 172.21.15.83:0/3025330288 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b175f19bf0 0x55b175f1c080 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f4eb00d85d0 tx=0x7f4eb00d8a90 comp rx=0 tx=0).stop 2024-04-24T08:50:49.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.928+0000 7f4ec7dc6740 1 -- 172.21.15.83:0/3025330288 shutdown_connections 2024-04-24T08:50:49.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.928+0000 7f4ec7dc6740 1 --2- 172.21.15.83:0/3025330288 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b175f19bf0 0x55b175f1c080 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.928+0000 7f4ec7dc6740 1 --2- 172.21.15.83:0/3025330288 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b175f14a80 0x55b175f19580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.928+0000 7f4ec7dc6740 1 --2- 172.21.15.83:0/3025330288 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b175f14090 0x55b175f144b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.928+0000 7f4ec7dc6740 1 -- 172.21.15.83:0/3025330288 >> 172.21.15.83:0/3025330288 conn(0x55b175f0f700 msgr2=0x55b175f0fb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:50:49.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.928+0000 7f4ec7dc6740 1 -- 172.21.15.83:0/3025330288 shutdown_connections 2024-04-24T08:50:49.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.929+0000 7f4ec7dc6740 1 -- 172.21.15.83:0/3025330288 wait complete. 2024-04-24T08:50:49.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.929+0000 7f4ec7dc6740 1 -- start start 2024-04-24T08:50:49.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.929+0000 7f4ec7dc6740 1 Processor -- start 2024-04-24T08:50:49.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.929+0000 7f4ec7dc6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b175f14090 0x55b175f1e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:49.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.929+0000 7f4ec7dc6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b175f14a80 0x55b175f1ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:49.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.930+0000 7f4ec7dc6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b175f19bf0 0x55b175f25a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:49.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.930+0000 7f4ec7dc6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55b175f1cba0 con 0x55b175f19bf0 2024-04-24T08:50:49.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.930+0000 7f4ec7dc6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b175e33590 con 0x55b175f14090 2024-04-24T08:50:49.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.930+0000 7f4ec7dc6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55b175f1cd20 con 0x55b175f14a80 2024-04-24T08:50:49.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.930+0000 7f4ec7dc6740 10 client.0 _ll_register_callbacks cb 0x55b175e163a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:50:49.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.930+0000 7f4ebe800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b175f19bf0 0x55b175f25a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:49.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.930+0000 7f4ebe800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b175f19bf0 0x55b175f25a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56568/0 (socket says 172.21.15.83:56568) 2024-04-24T08:50:49.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.930+0000 7f4ebe800640 1 -- 172.21.15.83:0/928052572 learned_addr learned my addr 172.21.15.83:0/928052572 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:50:49.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.930+0000 7f4ebde00640 1 --2- 172.21.15.83:0/928052572 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b175f14090 0x55b175f1e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:49.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.930+0000 7f4ebe800640 1 -- 172.21.15.83:0/928052572 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b175f14a80 msgr2=0x55b175f1ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:50:49.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.930+0000 7f4ebe800640 1 --2- 172.21.15.83:0/928052572 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b175f14a80 0x55b175f1ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.930+0000 7f4ebe800640 1 -- 172.21.15.83:0/928052572 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b175f14090 msgr2=0x55b175f1e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:50:49.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.930+0000 7f4ebe800640 1 --2- 172.21.15.83:0/928052572 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b175f14090 0x55b175f1e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.930+0000 7f4ebe800640 1 -- 172.21.15.83:0/928052572 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b175c566b0 con 0x55b175f19bf0 2024-04-24T08:50:49.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.930+0000 7f4ebde00640 1 --2- 172.21.15.83:0/928052572 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b175f14090 0x55b175f1e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:50:49.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.931+0000 7f4ebe800640 1 --2- 172.21.15.83:0/928052572 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b175f19bf0 0x55b175f25a60 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f4eb8009ca0 tx=0x7f4eb80095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=447e88378c491366 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:50:49.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.931+0000 7f4ea3e00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:50:49.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.931+0000 7f4ea3e00640 1 -- 172.21.15.83:0/928052572 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb804e070 con 0x55b175f19bf0 2024-04-24T08:50:49.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.931+0000 7f4ea3e00640 1 -- 172.21.15.83:0/928052572 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9237+0+0 (secure 0 0 0) 0x7f4eb8004790 con 0x55b175f19bf0 2024-04-24T08:50:49.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.931+0000 7f4ec7dc6740 1 -- 172.21.15.83:0/928052572 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b175e87530 con 0x55b175f19bf0 2024-04-24T08:50:49.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.931+0000 7f4ea3e00640 1 -- 172.21.15.83:0/928052572 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4eb805a050 con 0x55b175f19bf0 2024-04-24T08:50:49.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.931+0000 7f4ec7dc6740 10 client.15321 fetch_fsmap learned FSMap version 27 2024-04-24T08:50:49.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.931+0000 7f4ec7dc6740 1 -- 172.21.15.83:0/928052572 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b175f28dd0 con 0x55b175f19bf0 2024-04-24T08:50:49.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.931+0000 7f4ea3e00640 1 -- 172.21.15.83:0/928052572 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55b175e87530 con 0x55b175f19bf0 2024-04-24T08:50:49.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.932+0000 7f4ea3e00640 1 -- 172.21.15.83:0/928052572 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f4eb8043040 con 0x55b175f19bf0 2024-04-24T08:50:49.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.932+0000 7f4ec7dc6740 10 client.15321 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T08:50:49.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.932+0000 7f4ec7dc6740 10 client.15321 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-24T08:50:49.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.932+0000 7f4ec7dc6740 20 client.15321 populate_metadata read hostname 'smithi083' 2024-04-24T08:50:49.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.932+0000 7f4ec7dc6740 1 --2- 172.21.15.83:0/928052572 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x55b175f2b600 0x55b175f2ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:49.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.932+0000 7f4ec7dc6740 4 client.15321 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:50:49.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.932+0000 7f4ec7dc6740 1 -- 172.21.15.83:0/928052572 --> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b175e813a0 con 0x55b175f2b600 2024-04-24T08:50:49.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.933+0000 7f4ebde00640 1 --2- 172.21.15.83:0/928052572 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x55b175f2b600 0x55b175f2ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:49.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.934+0000 7f4ebde00640 1 --2- 172.21.15.83:0/928052572 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x55b175f2b600 0x55b175f2ca70 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=97ae9d1132dbc2b1 server_cookie=51bbac1c29536e87 in_seq=0 out_seq=0 2024-04-24T08:50:49.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.934+0000 7f4ea3e00640 10 client.15321 ms_handle_connect on v2:172.21.15.138:6836/900566754 2024-04-24T08:50:49.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.936+0000 7f4ea3e00640 1 -- 172.21.15.83:0/928052572 <== mds.1 v2:172.21.15.138:6836/900566754 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b175e813a0 con 0x55b175f2b600 2024-04-24T08:50:49.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.936+0000 7f4ea3e00640 10 client.15321 handle_command_reply: tid=0 2024-04-24T08:50:49.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.936+0000 7f4ec7dc6740 1 client.15321 shutdown 2024-04-24T08:50:49.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.936+0000 7f4ec7dc6740 1 -- 172.21.15.83:0/928052572 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b175f19bf0 msgr2=0x55b175f25a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:49.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.936+0000 7f4ec7dc6740 1 --2- 172.21.15.83:0/928052572 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b175f19bf0 0x55b175f25a60 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f4eb8009ca0 tx=0x7f4eb80095c0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.937+0000 7f4ec7dc6740 1 -- 172.21.15.83:0/928052572 shutdown_connections 2024-04-24T08:50:49.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.937+0000 7f4ec7dc6740 1 --2- 172.21.15.83:0/928052572 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b175f19bf0 0x55b175f25a60 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.937+0000 7f4ec7dc6740 1 --2- 172.21.15.83:0/928052572 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x55b175f2b600 0x55b175f2ca70 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.937+0000 7f4ec7dc6740 1 --2- 172.21.15.83:0/928052572 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b175f14a80 0x55b175f1ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.937+0000 7f4ec7dc6740 1 --2- 172.21.15.83:0/928052572 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b175f14090 0x55b175f1e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.937+0000 7f4ec7dc6740 1 -- 172.21.15.83:0/928052572 >> 172.21.15.83:0/928052572 conn(0x55b175f0f700 msgr2=0x55b175f0e8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:50:49.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.937+0000 7f4ea3e00640 0 client.15321 ms_handle_reset on v2:172.21.15.138:6836/900566754 2024-04-24T08:50:49.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.937+0000 7f4ec7dc6740 1 -- 172.21.15.83:0/928052572 shutdown_connections 2024-04-24T08:50:49.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.937+0000 7f4ec7dc6740 1 -- 172.21.15.83:0/928052572 wait complete. 2024-04-24T08:50:49.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.937+0000 7f4ec7dc6740 20 client.15321 trim_cache size 0 max 16384 2024-04-24T08:50:49.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.957+0000 7f4ec7dc6740 1 Processor -- start 2024-04-24T08:50:49.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.957+0000 7f4ec7dc6740 1 -- start start 2024-04-24T08:50:49.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.957+0000 7f4ec7dc6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b175f14090 0x55b175f0a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:49.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.957+0000 7f4ec7dc6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b175f14a80 0x55b175f25a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:49.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.957+0000 7f4ec7dc6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b175f19bf0 0x55b175f1e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:49.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.957+0000 7f4ec7dc6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55b175f2adb0 con 0x55b175f19bf0 2024-04-24T08:50:49.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.957+0000 7f4ec7dc6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b175f2b480 con 0x55b175f14090 2024-04-24T08:50:49.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.957+0000 7f4ec7dc6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55b175f2bb50 con 0x55b175f14a80 2024-04-24T08:50:49.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.957+0000 7f4ebe800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b175f19bf0 0x55b175f1e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:49.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.958+0000 7f4ebe800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b175f19bf0 0x55b175f1e4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56578/0 (socket says 172.21.15.83:56578) 2024-04-24T08:50:49.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.958+0000 7f4ebe800640 1 -- 172.21.15.83:0/362449349 learned_addr learned my addr 172.21.15.83:0/362449349 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:50:49.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.958+0000 7f4eb5400640 1 --2- 172.21.15.83:0/362449349 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b175f14a80 0x55b175f25a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:49.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.958+0000 7f4ebe800640 1 -- 172.21.15.83:0/362449349 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b175f14a80 msgr2=0x55b175f25a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:50:49.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.958+0000 7f4ebe800640 1 --2- 172.21.15.83:0/362449349 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b175f14a80 0x55b175f25a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.958+0000 7f4ebe800640 1 -- 172.21.15.83:0/362449349 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b175f14090 msgr2=0x55b175f0a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:50:49.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.958+0000 7f4ebde00640 1 --2- 172.21.15.83:0/362449349 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b175f14090 0x55b175f0a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:49.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.958+0000 7f4ebe800640 1 --2- 172.21.15.83:0/362449349 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b175f14090 0x55b175f0a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.958+0000 7f4ebe800640 1 -- 172.21.15.83:0/362449349 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b175f15270 con 0x55b175f19bf0 2024-04-24T08:50:49.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.958+0000 7f4ebde00640 1 --2- 172.21.15.83:0/362449349 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b175f14090 0x55b175f0a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:50:49.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.958+0000 7f4ebe800640 1 --2- 172.21.15.83:0/362449349 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b175f19bf0 0x55b175f1e4c0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f4eb802ade0 tx=0x7f4eb8031890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=985e6e2d5daf897d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:50:49.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.958+0000 7f4eaea00640 1 -- 172.21.15.83:0/362449349 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb804e040 con 0x55b175f19bf0 2024-04-24T08:50:49.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.959+0000 7f4eaea00640 1 -- 172.21.15.83:0/362449349 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4eb8036040 con 0x55b175f19bf0 2024-04-24T08:50:49.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.959+0000 7f4eaea00640 1 -- 172.21.15.83:0/362449349 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb8030ca0 con 0x55b175f19bf0 2024-04-24T08:50:49.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.959+0000 7f4ec7dc6740 1 -- 172.21.15.83:0/362449349 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b175f19bf0 msgr2=0x55b175f1e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:49.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.959+0000 7f4ec7dc6740 1 --2- 172.21.15.83:0/362449349 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b175f19bf0 0x55b175f1e4c0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f4eb802ade0 tx=0x7f4eb8031890 comp rx=0 tx=0).stop 2024-04-24T08:50:49.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.959+0000 7f4ec7dc6740 1 -- 172.21.15.83:0/362449349 shutdown_connections 2024-04-24T08:50:49.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.959+0000 7f4ec7dc6740 1 --2- 172.21.15.83:0/362449349 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b175f19bf0 0x55b175f1e4c0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.959+0000 7f4ec7dc6740 1 --2- 172.21.15.83:0/362449349 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b175f14a80 0x55b175f25a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.959+0000 7f4ec7dc6740 1 --2- 172.21.15.83:0/362449349 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b175f14090 0x55b175f0a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.959+0000 7f4ec7dc6740 1 -- 172.21.15.83:0/362449349 >> 172.21.15.83:0/362449349 conn(0x55b175f0f700 msgr2=0x55b175f0fae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:50:49.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.959+0000 7f4ec7dc6740 1 -- 172.21.15.83:0/362449349 shutdown_connections 2024-04-24T08:50:49.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.959+0000 7f4ec7dc6740 1 -- 172.21.15.83:0/362449349 wait complete. 2024-04-24T08:50:49.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.960+0000 7f4ec7dc6740 1 -- start start 2024-04-24T08:50:49.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.960+0000 7f4ec7dc6740 1 Processor -- start 2024-04-24T08:50:49.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.960+0000 7f4ec7dc6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b175f14090 0x55b175f23a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:49.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.960+0000 7f4ec7dc6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b175f14a80 0x55b175f8d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:49.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.960+0000 7f4ec7dc6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b175f19bf0 0x55b175f8fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:49.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.960+0000 7f4ec7dc6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55b175f2c8f0 con 0x55b175f19bf0 2024-04-24T08:50:49.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.960+0000 7f4ec7dc6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b175f2c220 con 0x55b175f14a80 2024-04-24T08:50:49.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.960+0000 7f4ec7dc6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55b175f2cfc0 con 0x55b175f14090 2024-04-24T08:50:49.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.960+0000 7f4ec7dc6740 10 client.0 _ll_register_callbacks cb 0x7f4eb8009820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:50:49.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.961+0000 7f4ebe800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b175f19bf0 0x55b175f8fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:49.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.961+0000 7f4ebe800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b175f19bf0 0x55b175f8fae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56594/0 (socket says 172.21.15.83:56594) 2024-04-24T08:50:49.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.961+0000 7f4ebe800640 1 -- 172.21.15.83:0/1281818201 learned_addr learned my addr 172.21.15.83:0/1281818201 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:50:49.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.961+0000 7f4ebde00640 1 --2- 172.21.15.83:0/1281818201 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b175f14090 0x55b175f23a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:49.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.961+0000 7f4eb5400640 1 --2- 172.21.15.83:0/1281818201 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b175f14a80 0x55b175f8d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:49.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.961+0000 7f4ebe800640 1 -- 172.21.15.83:0/1281818201 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b175f14090 msgr2=0x55b175f23a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:50:49.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.961+0000 7f4ebe800640 1 --2- 172.21.15.83:0/1281818201 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b175f14090 0x55b175f23a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.961+0000 7f4ebe800640 1 -- 172.21.15.83:0/1281818201 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b175f14a80 msgr2=0x55b175f8d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:50:49.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.961+0000 7f4ebe800640 1 --2- 172.21.15.83:0/1281818201 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b175f14a80 0x55b175f8d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.961+0000 7f4ebe800640 1 -- 172.21.15.83:0/1281818201 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b175f90120 con 0x55b175f19bf0 2024-04-24T08:50:49.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.961+0000 7f4eb5400640 1 --2- 172.21.15.83:0/1281818201 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b175f14a80 0x55b175f8d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:50:49.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.961+0000 7f4ebe800640 1 --2- 172.21.15.83:0/1281818201 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b175f19bf0 0x55b175f8fae0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f4eb80455c0 tx=0x7f4eb8045e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=461ea13c715ddf83 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:50:49.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.961+0000 7f4ea3e00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:50:49.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.962+0000 7f4ea3e00640 1 -- 172.21.15.83:0/1281818201 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb8065020 con 0x55b175f19bf0 2024-04-24T08:50:49.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.962+0000 7f4ea3e00640 1 -- 172.21.15.83:0/1281818201 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9237+0+0 (secure 0 0 0) 0x7f4eb8004610 con 0x55b175f19bf0 2024-04-24T08:50:49.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.962+0000 7f4ec7dc6740 1 -- 172.21.15.83:0/1281818201 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4eb8036040 con 0x55b175f19bf0 2024-04-24T08:50:49.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.962+0000 7f4ea3e00640 1 -- 172.21.15.83:0/1281818201 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4eb8070050 con 0x55b175f19bf0 2024-04-24T08:50:49.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.962+0000 7f4ec7dc6740 10 client.15327 fetch_fsmap learned FSMap version 27 2024-04-24T08:50:49.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.962+0000 7f4ec7dc6740 1 -- 172.21.15.83:0/1281818201 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b175f91370 con 0x55b175f19bf0 2024-04-24T08:50:49.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.962+0000 7f4ea3e00640 1 -- 172.21.15.83:0/1281818201 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4eb8036040 con 0x55b175f19bf0 2024-04-24T08:50:49.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.962+0000 7f4ea3e00640 1 -- 172.21.15.83:0/1281818201 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f4eb8043070 con 0x55b175f19bf0 2024-04-24T08:50:49.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.962+0000 7f4ec7dc6740 10 client.15327 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T08:50:49.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.963+0000 7f4ec7dc6740 10 client.15327 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-24T08:50:49.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.963+0000 7f4ec7dc6740 20 client.15327 populate_metadata read hostname 'smithi083' 2024-04-24T08:50:49.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.963+0000 7f4ec7dc6740 1 --2- 172.21.15.83:0/1281818201 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x55b175f95bf0 0x55b175f94900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:49.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.963+0000 7f4ec7dc6740 4 client.15327 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T08:50:49.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.963+0000 7f4ec7dc6740 1 -- 172.21.15.83:0/1281818201 --> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] -- command(tid 0: {"prefix": "status"}) -- 0x55b175dfd860 con 0x55b175f95bf0 2024-04-24T08:50:49.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.963+0000 7f4ebde00640 1 --2- 172.21.15.83:0/1281818201 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x55b175f95bf0 0x55b175f94900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:49.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.964+0000 7f4ebde00640 1 --2- 172.21.15.83:0/1281818201 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x55b175f95bf0 0x55b175f94900 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=412ce113ce275396 server_cookie=26ed173293683753 in_seq=0 out_seq=0 2024-04-24T08:50:49.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.964+0000 7f4ea3e00640 10 client.15327 ms_handle_connect on v2:172.21.15.138:6836/900566754 2024-04-24T08:50:49.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.965+0000 7f4ea3e00640 1 -- 172.21.15.83:0/1281818201 <== mds.1 v2:172.21.15.138:6836/900566754 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x55b175dfd860 con 0x55b175f95bf0 2024-04-24T08:50:49.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.965+0000 7f4ea3e00640 10 client.15327 handle_command_reply: tid=0 2024-04-24T08:50:49.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.965+0000 7f4ec7dc6740 1 client.15327 shutdown 2024-04-24T08:50:49.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.965+0000 7f4ec7dc6740 1 -- 172.21.15.83:0/1281818201 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b175f19bf0 msgr2=0x55b175f8fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:49.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.965+0000 7f4ec7dc6740 1 --2- 172.21.15.83:0/1281818201 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b175f19bf0 0x55b175f8fae0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f4eb80455c0 tx=0x7f4eb8045e20 comp rx=0 tx=0).stop 2024-04-24T08:50:49.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.965+0000 7f4ec7dc6740 1 -- 172.21.15.83:0/1281818201 shutdown_connections 2024-04-24T08:50:49.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.965+0000 7f4ec7dc6740 1 --2- 172.21.15.83:0/1281818201 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b175f19bf0 0x55b175f8fae0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.965+0000 7f4ec7dc6740 1 --2- 172.21.15.83:0/1281818201 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b175f14a80 0x55b175f8d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.965+0000 7f4ec7dc6740 1 --2- 172.21.15.83:0/1281818201 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x55b175f95bf0 0x55b175f94900 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.965+0000 7f4ec7dc6740 1 --2- 172.21.15.83:0/1281818201 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b175f14090 0x55b175f23a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.965+0000 7f4ec7dc6740 1 -- 172.21.15.83:0/1281818201 >> 172.21.15.83:0/1281818201 conn(0x55b175f0f700 msgr2=0x55b175f0a750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:50:49.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.965+0000 7f4ea3e00640 0 client.15327 ms_handle_reset on v2:172.21.15.138:6836/900566754 2024-04-24T08:50:49.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.965+0000 7f4ec7dc6740 1 -- 172.21.15.83:0/1281818201 shutdown_connections 2024-04-24T08:50:49.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.965+0000 7f4ec7dc6740 1 -- 172.21.15.83:0/1281818201 wait complete. 2024-04-24T08:50:49.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.965+0000 7f4ec7dc6740 20 client.15327 trim_cache size 0 max 16384 2024-04-24T08:50:49.970 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T08:50:49.971 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_fsid": "bd3e039e-0215-11ef-bc93-c7b262605968", 2024-04-24T08:50:49.971 INFO:teuthology.orchestra.run.smithi083.stdout: "whoami": 1, 2024-04-24T08:50:49.971 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 34493, 2024-04-24T08:50:49.971 INFO:teuthology.orchestra.run.smithi083.stdout: "want_state": "up:active", 2024-04-24T08:50:49.971 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "up:active", 2024-04-24T08:50:49.971 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs", 2024-04-24T08:50:49.971 INFO:teuthology.orchestra.run.smithi083.stdout: "rank_uptime": 108.946232063, 2024-04-24T08:50:49.971 INFO:teuthology.orchestra.run.smithi083.stdout: "mdsmap_epoch": 27, 2024-04-24T08:50:49.971 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch": 93, 2024-04-24T08:50:49.971 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch_barrier": 89, 2024-04-24T08:50:49.972 INFO:teuthology.orchestra.run.smithi083.stdout: "uptime": 164.588696443 2024-04-24T08:50:49.972 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T08:50:49.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.966+0000 7f4ec4e00640 1 -- 172.21.15.83:0/1406389671 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f4ea40a3bb0 msgr2=0x7f4ea40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:49.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.966+0000 7f4ec4e00640 1 --2- 172.21.15.83:0/1406389671 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f4ea40a3bb0 0x7f4ea40a6070 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4eb8006010 tx=0x7f4eb8031a40 comp rx=0 tx=0).stop 2024-04-24T08:50:49.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.966+0000 7f4ec4e00640 1 -- 172.21.15.83:0/1406389671 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ec0105450 msgr2=0x7f4ec019a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:49.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.966+0000 7f4ec4e00640 1 --2- 172.21.15.83:0/1406389671 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ec0105450 0x7f4ec019a990 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f4eb000d770 tx=0x7f4eb000dc30 comp rx=0 tx=0).stop 2024-04-24T08:50:49.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.966+0000 7f4ec4e00640 1 -- 172.21.15.83:0/1406389671 shutdown_connections 2024-04-24T08:50:49.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.966+0000 7f4ec4e00640 1 --2- 172.21.15.83:0/1406389671 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4ec0105e10 0x7f4ec01a1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.966+0000 7f4ec4e00640 1 --2- 172.21.15.83:0/1406389671 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f4ea40a3bb0 0x7f4ea40a6070 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.966+0000 7f4ec4e00640 1 --2- 172.21.15.83:0/1406389671 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ec0105450 0x7f4ec019a990 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.966+0000 7f4ec4e00640 1 --2- 172.21.15.83:0/1406389671 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ec0104260 0x7f4ec019a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:49.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.966+0000 7f4ec4e00640 1 -- 172.21.15.83:0/1406389671 >> 172.21.15.83:0/1406389671 conn(0x7f4ec00ff7f0 msgr2=0x7f4ec0108820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:50:49.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.966+0000 7f4ec4e00640 1 -- 172.21.15.83:0/1406389671 shutdown_connections 2024-04-24T08:50:49.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:49.966+0000 7f4ec4e00640 1 -- 172.21.15.83:0/1406389671 wait complete. 2024-04-24T08:50:49.984 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-24T08:50:50.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.188+0000 7fb6cc000640 1 -- 172.21.15.83:0/1667861923 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6c4105e10 msgr2=0x7fb6c410c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:50.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.188+0000 7fb6cc000640 1 --2- 172.21.15.83:0/1667861923 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6c4105e10 0x7fb6c410c6a0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fb6bc009ca0 tx=0x7fb6bc02a8c0 comp rx=0 tx=0).stop 2024-04-24T08:50:50.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.188+0000 7fb6cc000640 1 -- 172.21.15.83:0/1667861923 shutdown_connections 2024-04-24T08:50:50.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.188+0000 7fb6cc000640 1 --2- 172.21.15.83:0/1667861923 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6c4105e10 0x7fb6c410c6a0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:50.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.188+0000 7fb6cc000640 1 --2- 172.21.15.83:0/1667861923 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6c4105450 0x7fb6c41058d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:50.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.188+0000 7fb6cc000640 1 --2- 172.21.15.83:0/1667861923 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb6c4104260 0x7fb6c4104680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:50.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.188+0000 7fb6cc000640 1 -- 172.21.15.83:0/1667861923 >> 172.21.15.83:0/1667861923 conn(0x7fb6c40ff7f0 msgr2=0x7fb6c4101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:50:50.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.188+0000 7fb6cc000640 1 -- 172.21.15.83:0/1667861923 shutdown_connections 2024-04-24T08:50:50.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.188+0000 7fb6cc000640 1 -- 172.21.15.83:0/1667861923 wait complete. 2024-04-24T08:50:50.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.189+0000 7fb6cc000640 1 Processor -- start 2024-04-24T08:50:50.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.189+0000 7fb6cc000640 1 -- start start 2024-04-24T08:50:50.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.189+0000 7fb6cc000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6c4104260 0x7fb6c419a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:50.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.189+0000 7fb6cc000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb6c4105450 0x7fb6c419a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:50.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.189+0000 7fb6cc000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6c4105e10 0x7fb6c41a19d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:50.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.189+0000 7fb6cc000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb6c410fdf0 con 0x7fb6c4105e10 2024-04-24T08:50:50.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.189+0000 7fb6cc000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb6c410fc70 con 0x7fb6c4104260 2024-04-24T08:50:50.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.189+0000 7fb6cc000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb6c410ff70 con 0x7fb6c4105450 2024-04-24T08:50:50.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.190+0000 7fb6ca000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6c4105e10 0x7fb6c41a19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:50.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.190+0000 7fb6ca000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6c4105e10 0x7fb6c41a19d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56612/0 (socket says 172.21.15.83:56612) 2024-04-24T08:50:50.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.190+0000 7fb6ca000640 1 -- 172.21.15.83:0/893364895 learned_addr learned my addr 172.21.15.83:0/893364895 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:50:50.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.190+0000 7fb6c8c00640 1 --2- 172.21.15.83:0/893364895 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb6c4105450 0x7fb6c419a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:50.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.190+0000 7fb6c9600640 1 --2- 172.21.15.83:0/893364895 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6c4104260 0x7fb6c419a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:50.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.190+0000 7fb6ca000640 1 -- 172.21.15.83:0/893364895 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb6c4105450 msgr2=0x7fb6c419a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:50.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.190+0000 7fb6ca000640 1 --2- 172.21.15.83:0/893364895 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb6c4105450 0x7fb6c419a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:50.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.190+0000 7fb6ca000640 1 -- 172.21.15.83:0/893364895 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6c4104260 msgr2=0x7fb6c419a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:50.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.190+0000 7fb6ca000640 1 --2- 172.21.15.83:0/893364895 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6c4104260 0x7fb6c419a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:50.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.190+0000 7fb6ca000640 1 -- 172.21.15.83:0/893364895 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6c41a2040 con 0x7fb6c4105e10 2024-04-24T08:50:50.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.190+0000 7fb6ca000640 1 --2- 172.21.15.83:0/893364895 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6c4105e10 0x7fb6c41a19d0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fb6bc031040 tx=0x7fb6bc003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:50:50.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.191+0000 7fb6c2000640 1 -- 172.21.15.83:0/893364895 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6bc032ec0 con 0x7fb6c4105e10 2024-04-24T08:50:50.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.191+0000 7fb6c2000640 1 -- 172.21.15.83:0/893364895 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb6bc0044f0 con 0x7fb6c4105e10 2024-04-24T08:50:50.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.191+0000 7fb6c2000640 1 -- 172.21.15.83:0/893364895 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6bc030d10 con 0x7fb6c4105e10 2024-04-24T08:50:50.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.191+0000 7fb6cc000640 1 -- 172.21.15.83:0/893364895 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6c41a22d0 con 0x7fb6c4105e10 2024-04-24T08:50:50.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.191+0000 7fb6cc000640 1 -- 172.21.15.83:0/893364895 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6c41a27d0 con 0x7fb6c4105e10 2024-04-24T08:50:50.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.194+0000 7fb6cefb1740 1 Processor -- start 2024-04-24T08:50:50.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.194+0000 7fb6c2000640 1 -- 172.21.15.83:0/893364895 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fb6bc004040 con 0x7fb6c4105e10 2024-04-24T08:50:50.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.195+0000 7fb6cefb1740 1 -- start start 2024-04-24T08:50:50.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.195+0000 7fb6cefb1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e134a91090 0x55e134a914b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:50.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.195+0000 7fb6cefb1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e134a91a80 0x55e134a96580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:50.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.195+0000 7fb6cefb1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e134a96bf0 0x55e134a99080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:50.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.195+0000 7fb6cefb1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e1349f6b10 con 0x55e134a91090 2024-04-24T08:50:50.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.195+0000 7fb6cefb1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e13499d2b0 con 0x55e134a96bf0 2024-04-24T08:50:50.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.195+0000 7fb6cefb1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55e1349f2ad0 con 0x55e134a91a80 2024-04-24T08:50:50.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.196+0000 7fb6c8c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e134a91090 0x55e134a914b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:50.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.196+0000 7fb6c8c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e134a91090 0x55e134a914b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56628/0 (socket says 172.21.15.83:56628) 2024-04-24T08:50:50.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.196+0000 7fb6c8c00640 1 -- 172.21.15.83:0/287151002 learned_addr learned my addr 172.21.15.83:0/287151002 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:50:50.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.196+0000 7fb6c9600640 1 --2- 172.21.15.83:0/287151002 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e134a91a80 0x55e134a96580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:50.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.196+0000 7fb6c8c00640 1 -- 172.21.15.83:0/287151002 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e134a91a80 msgr2=0x55e134a96580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:50:50.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.196+0000 7fb6c8c00640 1 --2- 172.21.15.83:0/287151002 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e134a91a80 0x55e134a96580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:50.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.196+0000 7fb6c8c00640 1 -- 172.21.15.83:0/287151002 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e134a96bf0 msgr2=0x55e134a99080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:50:50.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.196+0000 7fb6c8c00640 1 --2- 172.21.15.83:0/287151002 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e134a96bf0 0x55e134a99080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:50.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.196+0000 7fb6c8c00640 1 -- 172.21.15.83:0/287151002 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1347092d0 con 0x55e134a91090 2024-04-24T08:50:50.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.196+0000 7fb6c9600640 1 --2- 172.21.15.83:0/287151002 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e134a91a80 0x55e134a96580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:50:50.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.196+0000 7fb6c2000640 1 --2- 172.21.15.83:0/893364895 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fb6ac0a3d50 0x7fb6ac0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:50.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.196+0000 7fb6c8c00640 1 --2- 172.21.15.83:0/287151002 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e134a91090 0x55e134a914b0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fb6b0009bf0 tx=0x7fb6b00106e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=906234a4fd560657 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:50:50.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.197+0000 7fb6c2000640 1 -- 172.21.15.83:0/893364895 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9237+0+0 (secure 0 0 0) 0x7fb6bc0f1ef0 con 0x7fb6c4105e10 2024-04-24T08:50:50.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.197+0000 7fb6c9600640 1 --2- 172.21.15.83:0/893364895 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fb6ac0a3d50 0x7fb6ac0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:50.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.197+0000 7fb6b6600640 1 -- 172.21.15.83:0/287151002 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6b0016070 con 0x55e134a91090 2024-04-24T08:50:50.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.197+0000 7fb6b6600640 1 -- 172.21.15.83:0/287151002 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb6b00054d0 con 0x55e134a91090 2024-04-24T08:50:50.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.197+0000 7fb6b6600640 1 -- 172.21.15.83:0/287151002 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6b0012420 con 0x55e134a91090 2024-04-24T08:50:50.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.198+0000 7fb6cefb1740 1 -- 172.21.15.83:0/287151002 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e134a91090 msgr2=0x55e134a914b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:50.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.198+0000 7fb6cefb1740 1 --2- 172.21.15.83:0/287151002 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e134a91090 0x55e134a914b0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fb6b0009bf0 tx=0x7fb6b00106e0 comp rx=0 tx=0).stop 2024-04-24T08:50:50.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.198+0000 7fb6c9600640 1 --2- 172.21.15.83:0/893364895 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fb6ac0a3d50 0x7fb6ac0a6210 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x55e134a91e80 tx=0x7fb6b800b710 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:50:50.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.198+0000 7fb6cefb1740 1 -- 172.21.15.83:0/287151002 shutdown_connections 2024-04-24T08:50:50.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.198+0000 7fb6cefb1740 1 --2- 172.21.15.83:0/287151002 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e134a96bf0 0x55e134a99080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:50.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.198+0000 7fb6cefb1740 1 --2- 172.21.15.83:0/287151002 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e134a91a80 0x55e134a96580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:50.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.198+0000 7fb6cefb1740 1 --2- 172.21.15.83:0/287151002 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e134a91090 0x55e134a914b0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:50.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.198+0000 7fb6cefb1740 1 -- 172.21.15.83:0/287151002 >> 172.21.15.83:0/287151002 conn(0x55e134a8c700 msgr2=0x55e134a8cb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:50:50.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.198+0000 7fb6cefb1740 1 -- 172.21.15.83:0/287151002 shutdown_connections 2024-04-24T08:50:50.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.198+0000 7fb6cefb1740 1 -- 172.21.15.83:0/287151002 wait complete. 2024-04-24T08:50:50.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.198+0000 7fb6cefb1740 1 -- start start 2024-04-24T08:50:50.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.198+0000 7fb6cefb1740 1 Processor -- start 2024-04-24T08:50:50.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.199+0000 7fb6cefb1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e134a91090 0x55e134a9b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:50.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.199+0000 7fb6cefb1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e134a91a80 0x55e134a9b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:50.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.199+0000 7fb6cefb1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e134a96bf0 0x55e134aa2a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:50.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.199+0000 7fb6cefb1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e134a99ba0 con 0x55e134a96bf0 2024-04-24T08:50:50.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.199+0000 7fb6cefb1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e1349b0590 con 0x55e134a91090 2024-04-24T08:50:50.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.199+0000 7fb6cefb1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55e134a99d20 con 0x55e134a91a80 2024-04-24T08:50:50.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.199+0000 7fb6cefb1740 10 client.0 _ll_register_callbacks cb 0x55e1349933a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:50:50.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.199+0000 7fb6ca000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e134a96bf0 0x55e134aa2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:50.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.199+0000 7fb6ca000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e134a96bf0 0x55e134aa2a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56644/0 (socket says 172.21.15.83:56644) 2024-04-24T08:50:50.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.199+0000 7fb6ca000640 1 -- 172.21.15.83:0/278533351 learned_addr learned my addr 172.21.15.83:0/278533351 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:50:50.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.199+0000 7fb6c9600640 1 --2- 172.21.15.83:0/278533351 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e134a91090 0x55e134a9b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:50.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.200+0000 7fb6c8c00640 1 --2- 172.21.15.83:0/278533351 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e134a91a80 0x55e134a9b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:50.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.200+0000 7fb6ca000640 1 -- 172.21.15.83:0/278533351 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e134a91a80 msgr2=0x55e134a9b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:50:50.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.200+0000 7fb6ca000640 1 --2- 172.21.15.83:0/278533351 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e134a91a80 0x55e134a9b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:50.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.200+0000 7fb6ca000640 1 -- 172.21.15.83:0/278533351 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e134a91090 msgr2=0x55e134a9b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:50:50.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.200+0000 7fb6ca000640 1 --2- 172.21.15.83:0/278533351 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e134a91090 0x55e134a9b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:50.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.200+0000 7fb6ca000640 1 -- 172.21.15.83:0/278533351 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1347d36b0 con 0x55e134a96bf0 2024-04-24T08:50:50.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.200+0000 7fb6ca000640 1 --2- 172.21.15.83:0/278533351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e134a96bf0 0x55e134aa2a30 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fb6bc0bdae0 tx=0x7fb6bc0bd9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9c1679f09f39cd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:50:50.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.200+0000 7fb6aaa00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:50:50.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.200+0000 7fb6aaa00640 1 -- 172.21.15.83:0/278533351 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6bc0d7060 con 0x55e134a96bf0 2024-04-24T08:50:50.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.200+0000 7fb6aaa00640 1 -- 172.21.15.83:0/278533351 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9237+0+0 (secure 0 0 0) 0x7fb6bc0d7200 con 0x55e134a96bf0 2024-04-24T08:50:50.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.200+0000 7fb6cefb1740 1 -- 172.21.15.83:0/278533351 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6b00054d0 con 0x55e134a96bf0 2024-04-24T08:50:50.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.200+0000 7fb6aaa00640 1 -- 172.21.15.83:0/278533351 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb6bc0be4e0 con 0x55e134a96bf0 2024-04-24T08:50:50.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.200+0000 7fb6cefb1740 10 client.15339 fetch_fsmap learned FSMap version 27 2024-04-24T08:50:50.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.200+0000 7fb6cefb1740 1 -- 172.21.15.83:0/278533351 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e134a99ba0 con 0x55e134a96bf0 2024-04-24T08:50:50.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.201+0000 7fb6aaa00640 1 -- 172.21.15.83:0/278533351 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb6b00054d0 con 0x55e134a96bf0 2024-04-24T08:50:50.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.201+0000 7fb6aaa00640 1 -- 172.21.15.83:0/278533351 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fb6bc0be040 con 0x55e134a96bf0 2024-04-24T08:50:50.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.201+0000 7fb6cefb1740 10 client.15339 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T08:50:50.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.201+0000 7fb6cefb1740 10 client.15339 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-24T08:50:50.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.201+0000 7fb6cefb1740 20 client.15339 populate_metadata read hostname 'smithi083' 2024-04-24T08:50:50.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.201+0000 7fb6cefb1740 1 --2- 172.21.15.83:0/278533351 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55e134aa84e0 0x55e134aa9950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:50.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.201+0000 7fb6cefb1740 4 client.15339 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:50:50.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.201+0000 7fb6cefb1740 1 -- 172.21.15.83:0/278533351 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e134a04530 con 0x55e134aa84e0 2024-04-24T08:50:50.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.202+0000 7fb6c9600640 1 --2- 172.21.15.83:0/278533351 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55e134aa84e0 0x55e134aa9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:50.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.203+0000 7fb6c9600640 1 --2- 172.21.15.83:0/278533351 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55e134aa84e0 0x55e134aa9950 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d79b2016344637a7 server_cookie=39bc0452e6907783 in_seq=0 out_seq=0 2024-04-24T08:50:50.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.203+0000 7fb6aaa00640 10 client.15339 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:50:50.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.205+0000 7fb6aaa00640 1 -- 172.21.15.83:0/278533351 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e134a04530 con 0x55e134aa84e0 2024-04-24T08:50:50.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.205+0000 7fb6aaa00640 10 client.15339 handle_command_reply: tid=0 2024-04-24T08:50:50.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.205+0000 7fb6cefb1740 1 client.15339 shutdown 2024-04-24T08:50:50.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.205+0000 7fb6cefb1740 1 -- 172.21.15.83:0/278533351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e134a96bf0 msgr2=0x55e134aa2a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:50.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.205+0000 7fb6cefb1740 1 --2- 172.21.15.83:0/278533351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e134a96bf0 0x55e134aa2a30 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fb6bc0bdae0 tx=0x7fb6bc0bd9e0 comp rx=0 tx=0).stop 2024-04-24T08:50:50.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.206+0000 7fb6cefb1740 1 -- 172.21.15.83:0/278533351 shutdown_connections 2024-04-24T08:50:50.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.206+0000 7fb6cefb1740 1 --2- 172.21.15.83:0/278533351 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55e134aa84e0 0x55e134aa9950 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:50.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.206+0000 7fb6cefb1740 1 --2- 172.21.15.83:0/278533351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e134a96bf0 0x55e134aa2a30 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:50.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.206+0000 7fb6cefb1740 1 --2- 172.21.15.83:0/278533351 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e134a91a80 0x55e134a9b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:50.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.206+0000 7fb6cefb1740 1 --2- 172.21.15.83:0/278533351 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e134a91090 0x55e134a9b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:50.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.206+0000 7fb6aaa00640 0 client.15339 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:50:50.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.206+0000 7fb6cefb1740 1 -- 172.21.15.83:0/278533351 >> 172.21.15.83:0/278533351 conn(0x55e134a8c700 msgr2=0x55e134a8b8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:50:50.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.206+0000 7fb6cefb1740 1 -- 172.21.15.83:0/278533351 shutdown_connections 2024-04-24T08:50:50.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.206+0000 7fb6cefb1740 1 -- 172.21.15.83:0/278533351 wait complete. 2024-04-24T08:50:50.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.206+0000 7fb6cefb1740 20 client.15339 trim_cache size 0 max 16384 2024-04-24T08:50:50.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.225+0000 7fb6cefb1740 1 Processor -- start 2024-04-24T08:50:50.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.225+0000 7fb6cefb1740 1 -- start start 2024-04-24T08:50:50.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.226+0000 7fb6cefb1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e134a91090 0x55e134a98fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:50.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.226+0000 7fb6cefb1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e134a91a80 0x55e134aa29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:50.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.226+0000 7fb6cefb1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e134a96bf0 0x55e134a964b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:50.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.226+0000 7fb6cefb1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e134aa7c90 con 0x55e134a96bf0 2024-04-24T08:50:50.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.226+0000 7fb6cefb1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e134aa8360 con 0x55e134a91a80 2024-04-24T08:50:50.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.226+0000 7fb6cefb1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55e134aa8a30 con 0x55e134a91090 2024-04-24T08:50:50.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.226+0000 7fb6ca000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e134a96bf0 0x55e134a964b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:50.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.226+0000 7fb6ca000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e134a96bf0 0x55e134a964b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56646/0 (socket says 172.21.15.83:56646) 2024-04-24T08:50:50.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.226+0000 7fb6ca000640 1 -- 172.21.15.83:0/3964131919 learned_addr learned my addr 172.21.15.83:0/3964131919 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:50:50.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.226+0000 7fb6c8c00640 1 --2- 172.21.15.83:0/3964131919 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e134a91a80 0x55e134aa29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:50.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.226+0000 7fb6ca000640 1 -- 172.21.15.83:0/3964131919 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e134a91090 msgr2=0x55e134a98fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:50:50.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.226+0000 7fb6ca000640 1 --2- 172.21.15.83:0/3964131919 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e134a91090 0x55e134a98fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:50.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.226+0000 7fb6ca000640 1 -- 172.21.15.83:0/3964131919 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e134a91a80 msgr2=0x55e134aa29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:50:50.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.226+0000 7fb6ca000640 1 --2- 172.21.15.83:0/3964131919 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e134a91a80 0x55e134aa29f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:50.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.226+0000 7fb6ca000640 1 -- 172.21.15.83:0/3964131919 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e134aa8140 con 0x55e134a96bf0 2024-04-24T08:50:50.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.227+0000 7fb6ca000640 1 --2- 172.21.15.83:0/3964131919 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e134a96bf0 0x55e134a964b0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fb6bc031aa0 tx=0x7fb6bc0bc2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6447970f1f000671 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:50:50.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.227+0000 7fb6b5c00640 1 -- 172.21.15.83:0/3964131919 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6bc039e10 con 0x55e134a96bf0 2024-04-24T08:50:50.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.227+0000 7fb6b5c00640 1 -- 172.21.15.83:0/3964131919 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb6bc036040 con 0x55e134a96bf0 2024-04-24T08:50:50.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.227+0000 7fb6b5c00640 1 -- 172.21.15.83:0/3964131919 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6bc0c2190 con 0x55e134a96bf0 2024-04-24T08:50:50.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.227+0000 7fb6cefb1740 1 -- 172.21.15.83:0/3964131919 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e134a96bf0 msgr2=0x55e134a964b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:50.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.227+0000 7fb6cefb1740 1 --2- 172.21.15.83:0/3964131919 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e134a96bf0 0x55e134a964b0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fb6bc031aa0 tx=0x7fb6bc0bc2f0 comp rx=0 tx=0).stop 2024-04-24T08:50:50.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.228+0000 7fb6cefb1740 1 -- 172.21.15.83:0/3964131919 shutdown_connections 2024-04-24T08:50:50.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.228+0000 7fb6cefb1740 1 --2- 172.21.15.83:0/3964131919 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e134a96bf0 0x55e134a964b0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:50.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.228+0000 7fb6cefb1740 1 --2- 172.21.15.83:0/3964131919 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e134a91a80 0x55e134aa29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:50.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.228+0000 7fb6cefb1740 1 --2- 172.21.15.83:0/3964131919 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e134a91090 0x55e134a98fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:50.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.228+0000 7fb6cefb1740 1 -- 172.21.15.83:0/3964131919 >> 172.21.15.83:0/3964131919 conn(0x55e134a8c700 msgr2=0x55e134a8cae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:50:50.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.228+0000 7fb6cefb1740 1 -- 172.21.15.83:0/3964131919 shutdown_connections 2024-04-24T08:50:50.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.228+0000 7fb6cefb1740 1 -- 172.21.15.83:0/3964131919 wait complete. 2024-04-24T08:50:50.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.228+0000 7fb6cefb1740 1 -- start start 2024-04-24T08:50:50.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.228+0000 7fb6cefb1740 1 Processor -- start 2024-04-24T08:50:50.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.229+0000 7fb6cefb1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e134a91090 0x55e134aa0c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:50.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.229+0000 7fb6cefb1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e134a91a80 0x55e134aa1170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:50.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.229+0000 7fb6cefb1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e134a96bf0 0x55e134aa17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:50.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.229+0000 7fb6cefb1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e134aa97d0 con 0x55e134a91090 2024-04-24T08:50:50.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.229+0000 7fb6cefb1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e134aa9100 con 0x55e134a96bf0 2024-04-24T08:50:50.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.229+0000 7fb6cefb1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55e134aa9ea0 con 0x55e134a91a80 2024-04-24T08:50:50.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.229+0000 7fb6cefb1740 10 client.0 _ll_register_callbacks cb 0x7fb6bc0bdbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:50:50.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.229+0000 7fb6c9600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e134a91090 0x55e134aa0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:50.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.229+0000 7fb6c9600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e134a91090 0x55e134aa0c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56654/0 (socket says 172.21.15.83:56654) 2024-04-24T08:50:50.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.229+0000 7fb6c9600640 1 -- 172.21.15.83:0/2063502154 learned_addr learned my addr 172.21.15.83:0/2063502154 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:50:50.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.229+0000 7fb6c9600640 1 -- 172.21.15.83:0/2063502154 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e134a91a80 msgr2=0x55e134aa1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:50:50.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.229+0000 7fb6ca000640 1 --2- 172.21.15.83:0/2063502154 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e134a96bf0 0x55e134aa17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:50.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.229+0000 7fb6c9600640 1 --2- 172.21.15.83:0/2063502154 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e134a91a80 0x55e134aa1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:50.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.229+0000 7fb6c9600640 1 -- 172.21.15.83:0/2063502154 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e134a96bf0 msgr2=0x55e134aa17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:50:50.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.229+0000 7fb6c9600640 1 --2- 172.21.15.83:0/2063502154 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e134a96bf0 0x55e134aa17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:50.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.229+0000 7fb6c9600640 1 -- 172.21.15.83:0/2063502154 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e134aa1fd0 con 0x55e134a91090 2024-04-24T08:50:50.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.229+0000 7fb6ca000640 1 --2- 172.21.15.83:0/2063502154 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e134a96bf0 0x55e134aa17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:50:50.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.230+0000 7fb6c9600640 1 --2- 172.21.15.83:0/2063502154 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e134a91090 0x55e134aa0c30 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fb6b8005fd0 tx=0x7fb6b8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c62fb4b1b350791 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:50:50.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.230+0000 7fb6aaa00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:50:50.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.230+0000 7fb6aaa00640 1 -- 172.21.15.83:0/2063502154 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6b802d020 con 0x55e134a91090 2024-04-24T08:50:50.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.230+0000 7fb6aaa00640 1 -- 172.21.15.83:0/2063502154 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9237+0+0 (secure 0 0 0) 0x7fb6b8004440 con 0x55e134a91090 2024-04-24T08:50:50.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.230+0000 7fb6cefb1740 1 -- 172.21.15.83:0/2063502154 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6bc036040 con 0x55e134a91090 2024-04-24T08:50:50.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.230+0000 7fb6aaa00640 1 -- 172.21.15.83:0/2063502154 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb6b8038050 con 0x55e134a91090 2024-04-24T08:50:50.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.230+0000 7fb6cefb1740 10 client.15345 fetch_fsmap learned FSMap version 27 2024-04-24T08:50:50.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.230+0000 7fb6cefb1740 1 -- 172.21.15.83:0/2063502154 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e134b0e8a0 con 0x55e134a91090 2024-04-24T08:50:50.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.230+0000 7fb6aaa00640 1 -- 172.21.15.83:0/2063502154 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb6bc036040 con 0x55e134a91090 2024-04-24T08:50:50.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.231+0000 7fb6aaa00640 1 -- 172.21.15.83:0/2063502154 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fb6b80039b0 con 0x55e134a91090 2024-04-24T08:50:50.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.231+0000 7fb6cefb1740 10 client.15345 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T08:50:50.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.231+0000 7fb6cefb1740 10 client.15345 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-24T08:50:50.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.231+0000 7fb6cefb1740 20 client.15345 populate_metadata read hostname 'smithi083' 2024-04-24T08:50:50.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.231+0000 7fb6cefb1740 1 --2- 172.21.15.83:0/2063502154 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55e134b13510 0x55e134b12220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:50.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.231+0000 7fb6cefb1740 4 client.15345 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T08:50:50.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.231+0000 7fb6cefb1740 1 -- 172.21.15.83:0/2063502154 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "status"}) -- 0x55e1349fe3a0 con 0x55e134b13510 2024-04-24T08:50:50.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.231+0000 7fb6c8c00640 1 --2- 172.21.15.83:0/2063502154 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55e134b13510 0x55e134b12220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:50.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.232+0000 7fb6c8c00640 1 --2- 172.21.15.83:0/2063502154 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55e134b13510 0x55e134b12220 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=202c2fb2e92ec565 server_cookie=46f3f735da072557 in_seq=0 out_seq=0 2024-04-24T08:50:50.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.232+0000 7fb6aaa00640 10 client.15345 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:50:50.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.233+0000 7fb6aaa00640 1 -- 172.21.15.83:0/2063502154 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+321 (crc 0 0 0) 0x55e1349fe3a0 con 0x55e134b13510 2024-04-24T08:50:50.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.233+0000 7fb6aaa00640 10 client.15345 handle_command_reply: tid=0 2024-04-24T08:50:50.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.233+0000 7fb6cefb1740 1 client.15345 shutdown 2024-04-24T08:50:50.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.233+0000 7fb6cefb1740 1 -- 172.21.15.83:0/2063502154 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e134a91090 msgr2=0x55e134aa0c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:50.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.233+0000 7fb6cefb1740 1 --2- 172.21.15.83:0/2063502154 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e134a91090 0x55e134aa0c30 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fb6b8005fd0 tx=0x7fb6b8004730 comp rx=0 tx=0).stop 2024-04-24T08:50:50.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.233+0000 7fb6cefb1740 1 -- 172.21.15.83:0/2063502154 shutdown_connections 2024-04-24T08:50:50.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.233+0000 7fb6cefb1740 1 --2- 172.21.15.83:0/2063502154 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55e134b13510 0x55e134b12220 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:50.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.233+0000 7fb6cefb1740 1 --2- 172.21.15.83:0/2063502154 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e134a96bf0 0x55e134aa17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:50.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.233+0000 7fb6cefb1740 1 --2- 172.21.15.83:0/2063502154 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e134a91a80 0x55e134aa1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:50.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.233+0000 7fb6aaa00640 0 client.15345 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:50:50.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.233+0000 7fb6cefb1740 1 --2- 172.21.15.83:0/2063502154 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e134a91090 0x55e134aa0c30 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:50.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.233+0000 7fb6cefb1740 1 -- 172.21.15.83:0/2063502154 >> 172.21.15.83:0/2063502154 conn(0x55e134a8c700 msgr2=0x55e134a8cae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:50:50.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.234+0000 7fb6cefb1740 1 -- 172.21.15.83:0/2063502154 shutdown_connections 2024-04-24T08:50:50.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.234+0000 7fb6cefb1740 1 -- 172.21.15.83:0/2063502154 wait complete. 2024-04-24T08:50:50.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.234+0000 7fb6cefb1740 20 client.15345 trim_cache size 0 max 16384 2024-04-24T08:50:50.237 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T08:50:50.237 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_fsid": "bd3e039e-0215-11ef-bc93-c7b262605968", 2024-04-24T08:50:50.237 INFO:teuthology.orchestra.run.smithi083.stdout: "whoami": 0, 2024-04-24T08:50:50.237 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 34520, 2024-04-24T08:50:50.237 INFO:teuthology.orchestra.run.smithi083.stdout: "want_state": "up:active", 2024-04-24T08:50:50.237 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "up:active", 2024-04-24T08:50:50.237 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs", 2024-04-24T08:50:50.237 INFO:teuthology.orchestra.run.smithi083.stdout: "rank_uptime": 140.02016695, 2024-04-24T08:50:50.238 INFO:teuthology.orchestra.run.smithi083.stdout: "mdsmap_epoch": 27, 2024-04-24T08:50:50.238 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch": 93, 2024-04-24T08:50:50.238 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch_barrier": 89, 2024-04-24T08:50:50.238 INFO:teuthology.orchestra.run.smithi083.stdout: "uptime": 148.205296409 2024-04-24T08:50:50.238 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T08:50:50.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.234+0000 7fb6cc000640 1 -- 172.21.15.83:0/893364895 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fb6ac0a3d50 msgr2=0x7fb6ac0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:50.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.234+0000 7fb6cc000640 1 --2- 172.21.15.83:0/893364895 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fb6ac0a3d50 0x7fb6ac0a6210 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x55e134a91e80 tx=0x7fb6b800b710 comp rx=0 tx=0).stop 2024-04-24T08:50:50.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.234+0000 7fb6cc000640 1 -- 172.21.15.83:0/893364895 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6c4105e10 msgr2=0x7fb6c41a19d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:50.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.234+0000 7fb6cc000640 1 --2- 172.21.15.83:0/893364895 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6c4105e10 0x7fb6c41a19d0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fb6bc031040 tx=0x7fb6bc003ba0 comp rx=0 tx=0).stop 2024-04-24T08:50:50.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.234+0000 7fb6cc000640 1 -- 172.21.15.83:0/893364895 shutdown_connections 2024-04-24T08:50:50.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.234+0000 7fb6cc000640 1 --2- 172.21.15.83:0/893364895 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fb6ac0a3d50 0x7fb6ac0a6210 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:50.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.234+0000 7fb6cc000640 1 --2- 172.21.15.83:0/893364895 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6c4105e10 0x7fb6c41a19d0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:50.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.234+0000 7fb6cc000640 1 --2- 172.21.15.83:0/893364895 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb6c4105450 0x7fb6c419a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:50.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.234+0000 7fb6cc000640 1 --2- 172.21.15.83:0/893364895 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6c4104260 0x7fb6c419a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:50.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.234+0000 7fb6cc000640 1 -- 172.21.15.83:0/893364895 >> 172.21.15.83:0/893364895 conn(0x7fb6c40ff7f0 msgr2=0x7fb6c4108750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:50:50.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.235+0000 7fb6cc000640 1 -- 172.21.15.83:0/893364895 shutdown_connections 2024-04-24T08:50:50.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:50.235+0000 7fb6cc000640 1 -- 172.21.15.83:0/893364895 wait complete. 2024-04-24T08:50:50.249 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:50:50 smithi083 ceph-mon[27907]: pgmap v447: 129 pgs: 129 active+clean; 644 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 11 op/s; 16 B/s, 1 keys/s, 2 objects/s recovering 2024-04-24T08:50:50.251 INFO:tasks.cephadm:Setup complete, yielding 2024-04-24T08:50:50.252 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-24T08:50:50.265 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi083.front.sepia.ceph.com 2024-04-24T08:50:50.265 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd3e039e-0215-11ef-bc93-c7b262605968 -- bash -c 'ceph orch status' 2024-04-24T08:50:50.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:50:50 smithi138 ceph-mon[31441]: pgmap v447: 129 pgs: 129 active+clean; 644 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 11 op/s; 16 B/s, 1 keys/s, 2 objects/s recovering 2024-04-24T08:50:50.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:50:50 smithi189 ceph-mon[31791]: pgmap v447: 129 pgs: 129 active+clean; 644 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 11 op/s; 16 B/s, 1 keys/s, 2 objects/s recovering 2024-04-24T08:50:50.706 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T08:50:52.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:50:52 smithi083 ceph-mon[27907]: pgmap v448: 129 pgs: 129 active+clean; 644 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 9 op/s; 15 B/s, 1 keys/s, 2 objects/s recovering 2024-04-24T08:50:52.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:50:52 smithi138 ceph-mon[31441]: pgmap v448: 129 pgs: 129 active+clean; 644 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 9 op/s; 15 B/s, 1 keys/s, 2 objects/s recovering 2024-04-24T08:50:52.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:50:52 smithi189 ceph-mon[31791]: pgmap v448: 129 pgs: 129 active+clean; 644 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 9 op/s; 15 B/s, 1 keys/s, 2 objects/s recovering 2024-04-24T08:50:53.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:53.143+0000 7f39ec400700 1 -- 172.21.15.83:0/1010605200 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39e4102b60 msgr2=0x7f39e4102f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:53.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:53.143+0000 7f39ec400700 1 --2- 172.21.15.83:0/1010605200 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39e4102b60 0x7f39e4102f80 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f39d800b0d0 tx=0x7f39d800b3e0 comp rx=0 tx=0).stop 2024-04-24T08:50:53.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:53.143+0000 7f39ec400700 1 -- 172.21.15.83:0/1010605200 shutdown_connections 2024-04-24T08:50:53.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:53.143+0000 7f39ec400700 1 --2- 172.21.15.83:0/1010605200 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39e4102b60 0x7f39e4102f80 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:53.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:53.143+0000 7f39ec400700 1 --2- 172.21.15.83:0/1010605200 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39e4104700 0x7f39e410afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:53.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:53.143+0000 7f39ec400700 1 --2- 172.21.15.83:0/1010605200 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f39e4103d60 0x7f39e41041c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:53.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:53.143+0000 7f39ec400700 1 -- 172.21.15.83:0/1010605200 >> 172.21.15.83:0/1010605200 conn(0x7f39e40fe0c0 msgr2=0x7f39e4100540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:50:53.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:53.143+0000 7f39ec400700 1 -- 172.21.15.83:0/1010605200 shutdown_connections 2024-04-24T08:50:53.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:53.143+0000 7f39ec400700 1 -- 172.21.15.83:0/1010605200 wait complete. 2024-04-24T08:50:53.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:53.144+0000 7f39ec400700 1 Processor -- start 2024-04-24T08:50:53.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:53.145+0000 7f39ec400700 1 -- start start 2024-04-24T08:50:53.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:53.145+0000 7f39ec400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f39e4102b60 0x7f39e419af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:53.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:53.145+0000 7f39ec400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39e4103d60 0x7f39e419b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:53.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:53.145+0000 7f39ec400700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39e4104700 0x7f39e41a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:53.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:53.145+0000 7f39ec400700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f39e410e640 con 0x7f39e4103d60 2024-04-24T08:50:53.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:53.145+0000 7f39ec400700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f39e410e4c0 con 0x7f39e4104700 2024-04-24T08:50:53.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:53.145+0000 7f39ec400700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f39e410e7c0 con 0x7f39e4102b60 2024-04-24T08:50:53.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:53.145+0000 7f39e9000700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39e4103d60 0x7f39e419b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:53.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:53.145+0000 7f39e9000700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39e4103d60 0x7f39e419b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56670/0 (socket says 172.21.15.83:56670) 2024-04-24T08:50:53.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:53.145+0000 7f39e9000700 1 -- 172.21.15.83:0/910310083 learned_addr learned my addr 172.21.15.83:0/910310083 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:50:53.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:53.146+0000 7f39e9a00700 1 --2- 172.21.15.83:0/910310083 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f39e4102b60 0x7f39e419af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:53.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:53.146+0000 7f39ea400700 1 --2- 172.21.15.83:0/910310083 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39e4104700 0x7f39e41a24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:53.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:53.146+0000 7f39e9000700 1 -- 172.21.15.83:0/910310083 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f39e4102b60 msgr2=0x7f39e419af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:53.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:53.146+0000 7f39e9000700 1 --2- 172.21.15.83:0/910310083 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f39e4102b60 0x7f39e419af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:53.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:53.146+0000 7f39e9000700 1 -- 172.21.15.83:0/910310083 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39e4104700 msgr2=0x7f39e41a24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:53.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:53.146+0000 7f39e9000700 1 --2- 172.21.15.83:0/910310083 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39e4104700 0x7f39e41a24e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:53.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:53.146+0000 7f39e9000700 1 -- 172.21.15.83:0/910310083 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39e41a2b80 con 0x7f39e4103d60 2024-04-24T08:50:53.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:53.146+0000 7f39e9a00700 1 --2- 172.21.15.83:0/910310083 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f39e4102b60 0x7f39e419af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:50:53.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:53.146+0000 7f39e9000700 1 --2- 172.21.15.83:0/910310083 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39e4103d60 0x7f39e419b460 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f39d000d9b0 tx=0x7f39d000dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:50:53.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:53.146+0000 7f39e2000700 1 -- 172.21.15.83:0/910310083 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d0017070 con 0x7f39e4103d60 2024-04-24T08:50:53.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:53.146+0000 7f39e2000700 1 -- 172.21.15.83:0/910310083 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f39d0004830 con 0x7f39e4103d60 2024-04-24T08:50:53.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:53.146+0000 7f39e2000700 1 -- 172.21.15.83:0/910310083 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d00053a0 con 0x7f39e4103d60 2024-04-24T08:50:53.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:53.147+0000 7f39ec400700 1 -- 172.21.15.83:0/910310083 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39e41a2e70 con 0x7f39e4103d60 2024-04-24T08:50:53.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:53.147+0000 7f39ec400700 1 -- 172.21.15.83:0/910310083 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39e41a32d0 con 0x7f39e4103d60 2024-04-24T08:50:53.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:53.148+0000 7f39ec400700 1 -- 172.21.15.83:0/910310083 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39e4069810 con 0x7f39e4103d60 2024-04-24T08:50:53.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:53.150+0000 7f39e2000700 1 -- 172.21.15.83:0/910310083 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f39d00049e0 con 0x7f39e4103d60 2024-04-24T08:50:53.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:53.151+0000 7f39e2000700 1 --2- 172.21.15.83:0/910310083 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f39cc0a3c40 0x7f39cc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:53.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:53.151+0000 7f39e2000700 1 -- 172.21.15.83:0/910310083 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9237+0+0 (secure 0 0 0) 0x7f39d00d0780 con 0x7f39e4103d60 2024-04-24T08:50:53.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:53.151+0000 7f39e9a00700 1 --2- 172.21.15.83:0/910310083 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f39cc0a3c40 0x7f39cc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:53.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:53.152+0000 7f39e9a00700 1 --2- 172.21.15.83:0/910310083 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f39cc0a3c40 0x7f39cc0a6100 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f39d8009fd0 tx=0x7f39d800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:50:53.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:53.156+0000 7f39e2000700 1 -- 172.21.15.83:0/910310083 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f39e4069810 con 0x7f39e4103d60 2024-04-24T08:50:53.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:53.409+0000 7f39ec400700 1 -- 172.21.15.83:0/910310083 --> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f39e4076440 con 0x7f39cc0a3c40 2024-04-24T08:50:53.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:53.412+0000 7f39e2000700 1 -- 172.21.15.83:0/910310083 <== mgr.14152 v2:172.21.15.83:6800/2550826560 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f39e4076440 con 0x7f39cc0a3c40 2024-04-24T08:50:53.413 INFO:teuthology.orchestra.run.smithi083.stdout:Backend: cephadm 2024-04-24T08:50:53.413 INFO:teuthology.orchestra.run.smithi083.stdout:Available: Yes 2024-04-24T08:50:53.413 INFO:teuthology.orchestra.run.smithi083.stdout:Paused: No 2024-04-24T08:50:53.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:53.415+0000 7f39ec400700 1 -- 172.21.15.83:0/910310083 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f39cc0a3c40 msgr2=0x7f39cc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:53.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:53.415+0000 7f39ec400700 1 --2- 172.21.15.83:0/910310083 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f39cc0a3c40 0x7f39cc0a6100 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f39d8009fd0 tx=0x7f39d800b040 comp rx=0 tx=0).stop 2024-04-24T08:50:53.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:53.415+0000 7f39ec400700 1 -- 172.21.15.83:0/910310083 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39e4103d60 msgr2=0x7f39e419b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:53.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:53.416+0000 7f39ec400700 1 --2- 172.21.15.83:0/910310083 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39e4103d60 0x7f39e419b460 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f39d000d9b0 tx=0x7f39d000dd70 comp rx=0 tx=0).stop 2024-04-24T08:50:53.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:53.416+0000 7f39ec400700 1 -- 172.21.15.83:0/910310083 shutdown_connections 2024-04-24T08:50:53.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:53.416+0000 7f39ec400700 1 --2- 172.21.15.83:0/910310083 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f39cc0a3c40 0x7f39cc0a6100 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:53.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:53.416+0000 7f39ec400700 1 --2- 172.21.15.83:0/910310083 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39e4104700 0x7f39e41a24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:53.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:53.416+0000 7f39ec400700 1 --2- 172.21.15.83:0/910310083 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f39e4102b60 0x7f39e419af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:53.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:53.416+0000 7f39ec400700 1 --2- 172.21.15.83:0/910310083 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39e4103d60 0x7f39e419b460 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:53.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:53.416+0000 7f39ec400700 1 -- 172.21.15.83:0/910310083 >> 172.21.15.83:0/910310083 conn(0x7f39e40fe0c0 msgr2=0x7f39e4074d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:50:53.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:53.416+0000 7f39ec400700 1 -- 172.21.15.83:0/910310083 shutdown_connections 2024-04-24T08:50:53.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:53.416+0000 7f39ec400700 1 -- 172.21.15.83:0/910310083 wait complete. 2024-04-24T08:50:54.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:50:54 smithi083 ceph-mon[27907]: pgmap v449: 129 pgs: 129 active+clean; 644 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s; 13 B/s, 0 keys/s, 2 objects/s recovering 2024-04-24T08:50:54.340 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd3e039e-0215-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ps' 2024-04-24T08:50:54.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:50:54 smithi138 ceph-mon[31441]: pgmap v449: 129 pgs: 129 active+clean; 644 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s; 13 B/s, 0 keys/s, 2 objects/s recovering 2024-04-24T08:50:54.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:50:54 smithi189 ceph-mon[31791]: pgmap v449: 129 pgs: 129 active+clean; 644 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s; 13 B/s, 0 keys/s, 2 objects/s recovering 2024-04-24T08:50:54.775 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T08:50:55.268 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:50:55 smithi083 ceph-mon[27907]: from='client.15351 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:50:55.268 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:50:55 smithi083 ceph-mon[27907]: pgmap v450: 129 pgs: 129 active+clean; 644 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s; 11 B/s, 0 keys/s, 1 objects/s recovering 2024-04-24T08:50:55.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:50:55 smithi138 ceph-mon[31441]: from='client.15351 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:50:55.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:50:55 smithi138 ceph-mon[31441]: pgmap v450: 129 pgs: 129 active+clean; 644 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s; 11 B/s, 0 keys/s, 1 objects/s recovering 2024-04-24T08:50:55.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:50:55 smithi189 ceph-mon[31791]: from='client.15351 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:50:55.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:50:55 smithi189 ceph-mon[31791]: pgmap v450: 129 pgs: 129 active+clean; 644 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s; 11 B/s, 0 keys/s, 1 objects/s recovering 2024-04-24T08:50:57.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:57.167+0000 7f7a15200700 1 -- 172.21.15.83:0/4211508625 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a10103680 msgr2=0x7f7a10105a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:57.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:57.167+0000 7f7a15200700 1 --2- 172.21.15.83:0/4211508625 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a10103680 0x7f7a10105a70 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f7a00009990 tx=0x7f7a00009ca0 comp rx=0 tx=0).stop 2024-04-24T08:50:57.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:57.167+0000 7f7a15200700 1 -- 172.21.15.83:0/4211508625 shutdown_connections 2024-04-24T08:50:57.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:57.167+0000 7f7a15200700 1 --2- 172.21.15.83:0/4211508625 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a10103680 0x7f7a10105a70 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:57.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:57.167+0000 7f7a15200700 1 --2- 172.21.15.83:0/4211508625 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a101088e0 0x7f7a1010acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:57.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:57.167+0000 7f7a15200700 1 --2- 172.21.15.83:0/4211508625 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7a10105fb0 0x7f7a101083a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:57.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:57.167+0000 7f7a15200700 1 -- 172.21.15.83:0/4211508625 >> 172.21.15.83:0/4211508625 conn(0x7f7a100fcff0 msgr2=0x7f7a100ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:50:57.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:57.167+0000 7f7a15200700 1 -- 172.21.15.83:0/4211508625 shutdown_connections 2024-04-24T08:50:57.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:57.167+0000 7f7a15200700 1 -- 172.21.15.83:0/4211508625 wait complete. 2024-04-24T08:50:57.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:57.168+0000 7f7a15200700 1 Processor -- start 2024-04-24T08:50:57.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:57.168+0000 7f7a15200700 1 -- start start 2024-04-24T08:50:57.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:57.169+0000 7f7a15200700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a10103680 0x7f7a10198d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:57.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:57.169+0000 7f7a15200700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a10105fb0 0x7f7a101992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:57.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:57.169+0000 7f7a15200700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7a101088e0 0x7f7a101a0290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:57.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:57.169+0000 7f7a15200700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7a1010e5d0 con 0x7f7a10105fb0 2024-04-24T08:50:57.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:57.169+0000 7f7a15200700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7a1010e450 con 0x7f7a10103680 2024-04-24T08:50:57.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:57.169+0000 7f7a15200700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7a1010e750 con 0x7f7a101088e0 2024-04-24T08:50:57.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:57.169+0000 7f7a05400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a10105fb0 0x7f7a101992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:57.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:57.169+0000 7f7a05400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a10105fb0 0x7f7a101992a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56674/0 (socket says 172.21.15.83:56674) 2024-04-24T08:50:57.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:57.169+0000 7f7a05400700 1 -- 172.21.15.83:0/2595508494 learned_addr learned my addr 172.21.15.83:0/2595508494 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:50:57.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:57.169+0000 7f7a0de00700 1 --2- 172.21.15.83:0/2595508494 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a10103680 0x7f7a10198d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:57.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:57.169+0000 7f7a0e800700 1 --2- 172.21.15.83:0/2595508494 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7a101088e0 0x7f7a101a0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:57.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:57.170+0000 7f7a05400700 1 -- 172.21.15.83:0/2595508494 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7a101088e0 msgr2=0x7f7a101a0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:57.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:57.170+0000 7f7a05400700 1 --2- 172.21.15.83:0/2595508494 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7a101088e0 0x7f7a101a0290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:57.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:57.170+0000 7f7a05400700 1 -- 172.21.15.83:0/2595508494 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a10103680 msgr2=0x7f7a10198d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:57.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:57.170+0000 7f7a05400700 1 --2- 172.21.15.83:0/2595508494 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a10103680 0x7f7a10198d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:57.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:57.170+0000 7f7a05400700 1 -- 172.21.15.83:0/2595508494 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a101a0960 con 0x7f7a10105fb0 2024-04-24T08:50:57.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:57.170+0000 7f7a05400700 1 --2- 172.21.15.83:0/2595508494 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a10105fb0 0x7f7a101992a0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f79f800c970 tx=0x7f79f800cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:50:57.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:57.170+0000 7f7a07400700 1 -- 172.21.15.83:0/2595508494 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79f8007aa0 con 0x7f7a10105fb0 2024-04-24T08:50:57.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:57.170+0000 7f7a07400700 1 -- 172.21.15.83:0/2595508494 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f79f8004ce0 con 0x7f7a10105fb0 2024-04-24T08:50:57.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:57.170+0000 7f7a07400700 1 -- 172.21.15.83:0/2595508494 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79f8005620 con 0x7f7a10105fb0 2024-04-24T08:50:57.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:57.170+0000 7f7a15200700 1 -- 172.21.15.83:0/2595508494 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a101014d0 con 0x7f7a10105fb0 2024-04-24T08:50:57.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:57.170+0000 7f7a15200700 1 -- 172.21.15.83:0/2595508494 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a101019c0 con 0x7f7a10105fb0 2024-04-24T08:50:57.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:57.172+0000 7f7a15200700 1 -- 172.21.15.83:0/2595508494 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a100697e0 con 0x7f7a10105fb0 2024-04-24T08:50:57.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:57.174+0000 7f7a07400700 1 -- 172.21.15.83:0/2595508494 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f79f8004750 con 0x7f7a10105fb0 2024-04-24T08:50:57.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:57.175+0000 7f7a07400700 1 --2- 172.21.15.83:0/2595508494 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f79f40a3cc0 0x7f79f40a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:50:57.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:57.175+0000 7f7a07400700 1 -- 172.21.15.83:0/2595508494 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9237+0+0 (secure 0 0 0) 0x7f79f80d0630 con 0x7f7a10105fb0 2024-04-24T08:50:57.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:57.176+0000 7f7a0de00700 1 --2- 172.21.15.83:0/2595508494 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f79f40a3cc0 0x7f79f40a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:50:57.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:57.176+0000 7f7a0de00700 1 --2- 172.21.15.83:0/2595508494 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f79f40a3cc0 0x7f79f40a6180 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7a0000b5c0 tx=0x7f7a00005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:50:57.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:57.180+0000 7f7a07400700 1 -- 172.21.15.83:0/2595508494 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f79f809cc50 con 0x7f7a10105fb0 2024-04-24T08:50:57.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:57.419+0000 7f7a15200700 1 -- 172.21.15.83:0/2595508494 --> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f7a10062f30 con 0x7f79f40a3cc0 2024-04-24T08:50:57.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:57.439+0000 7f7a07400700 1 -- 172.21.15.83:0/2595508494 <== mgr.14152 v2:172.21.15.83:6800/2550826560 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4123 (secure 0 0 0) 0x7f7a10062f30 con 0x7f79f40a3cc0 2024-04-24T08:50:57.439 INFO:teuthology.orchestra.run.smithi083.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-24T08:50:57.439 INFO:teuthology.orchestra.run.smithi083.stdout:mds.a smithi083 running (2m) 32s ago 2m 15.3M - 19.0.0-3116-g1f81fb39 b98aea5aa06f 0f9ba266ef2d 2024-04-24T08:50:57.439 INFO:teuthology.orchestra.run.smithi083.stdout:mds.b smithi138 running (2m) 82s ago 2m 25.1M - 19.0.0-3116-g1f81fb39 b98aea5aa06f 23d05a95c3bd 2024-04-24T08:50:57.439 INFO:teuthology.orchestra.run.smithi083.stdout:mds.c smithi189 running (2m) 82s ago 2m 15.4M - 19.0.0-3116-g1f81fb39 b98aea5aa06f f191a26fb2d2 2024-04-24T08:50:57.439 INFO:teuthology.orchestra.run.smithi083.stdout:mds.d smithi083 running (3m) 32s ago 3m 12.6M - 19.0.0-3116-g1f81fb39 b98aea5aa06f 89397ba2650e 2024-04-24T08:50:57.439 INFO:teuthology.orchestra.run.smithi083.stdout:mds.e smithi138 running (3m) 82s ago 3m 15.5M - 19.0.0-3116-g1f81fb39 b98aea5aa06f f58a43afe68a 2024-04-24T08:50:57.439 INFO:teuthology.orchestra.run.smithi083.stdout:mds.f smithi189 running (3m) 82s ago 3m 14.4M - 19.0.0-3116-g1f81fb39 b98aea5aa06f 2bceb0b43cf6 2024-04-24T08:50:57.439 INFO:teuthology.orchestra.run.smithi083.stdout:mds.g smithi083 running (2m) 32s ago 2m 15.2M - 19.0.0-3116-g1f81fb39 b98aea5aa06f 4cd70b536789 2024-04-24T08:50:57.439 INFO:teuthology.orchestra.run.smithi083.stdout:mds.h smithi138 running (2m) 82s ago 2m 15.8M - 19.0.0-3116-g1f81fb39 b98aea5aa06f 55769a4930c4 2024-04-24T08:50:57.440 INFO:teuthology.orchestra.run.smithi083.stdout:mds.i smithi189 running (2m) 82s ago 2m 15.1M - 19.0.0-3116-g1f81fb39 b98aea5aa06f 884f47db48cd 2024-04-24T08:50:57.440 INFO:teuthology.orchestra.run.smithi083.stdout:mds.j smithi083 running (3m) 32s ago 3m 12.5M - 19.0.0-3116-g1f81fb39 b98aea5aa06f 6c5b0557d05b 2024-04-24T08:50:57.440 INFO:teuthology.orchestra.run.smithi083.stdout:mds.k smithi138 running (3m) 82s ago 3m 15.5M - 19.0.0-3116-g1f81fb39 b98aea5aa06f 669f8f504545 2024-04-24T08:50:57.440 INFO:teuthology.orchestra.run.smithi083.stdout:mds.l smithi189 running (3m) 82s ago 3m 14.5M - 19.0.0-3116-g1f81fb39 b98aea5aa06f 4160a48691a9 2024-04-24T08:50:57.440 INFO:teuthology.orchestra.run.smithi083.stdout:mgr.x smithi083 *:9283,8765 running (13m) 32s ago 13m 516M - 19.0.0-3116-g1f81fb39 b98aea5aa06f c4a5d09d7dac 2024-04-24T08:50:57.440 INFO:teuthology.orchestra.run.smithi083.stdout:mgr.y smithi138 *:8443,8765 running (10m) 82s ago 10m 452M - 19.0.0-3116-g1f81fb39 b98aea5aa06f 19f774befef7 2024-04-24T08:50:57.440 INFO:teuthology.orchestra.run.smithi083.stdout:mgr.z smithi189 *:8443,8765 running (10m) 82s ago 10m 452M - 19.0.0-3116-g1f81fb39 b98aea5aa06f 40d25381f5bd 2024-04-24T08:50:57.440 INFO:teuthology.orchestra.run.smithi083.stdout:mon.a smithi083 running (13m) 32s ago 13m 50.3M 2048M 19.0.0-3116-g1f81fb39 b98aea5aa06f 2eee14394f4c 2024-04-24T08:50:57.440 INFO:teuthology.orchestra.run.smithi083.stdout:mon.b smithi138 running (11m) 82s ago 11m 40.8M 2048M 19.0.0-3116-g1f81fb39 b98aea5aa06f b10d504df993 2024-04-24T08:50:57.440 INFO:teuthology.orchestra.run.smithi083.stdout:mon.c smithi189 running (11m) 82s ago 11m 41.5M 2048M 19.0.0-3116-g1f81fb39 b98aea5aa06f 26c5d9f35126 2024-04-24T08:50:57.440 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0 smithi083 running (10m) 32s ago 10m 53.4M 4272M 19.0.0-3116-g1f81fb39 b98aea5aa06f 78641b700829 2024-04-24T08:50:57.440 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1 smithi138 running (9m) 82s ago 9m 53.4M 4272M 19.0.0-3116-g1f81fb39 b98aea5aa06f 0371890d4f5a 2024-04-24T08:50:57.440 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2 smithi189 running (9m) 82s ago 9m 46.6M 4272M 19.0.0-3116-g1f81fb39 b98aea5aa06f b3aabdbe9519 2024-04-24T08:50:57.440 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3 smithi083 running (8m) 32s ago 8m 51.3M 4272M 19.0.0-3116-g1f81fb39 b98aea5aa06f 045e0dfeee7a 2024-04-24T08:50:57.440 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4 smithi138 running (7m) 82s ago 7m 49.5M 4272M 19.0.0-3116-g1f81fb39 b98aea5aa06f d786b86efb78 2024-04-24T08:50:57.440 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5 smithi189 running (7m) 82s ago 7m 50.4M 4272M 19.0.0-3116-g1f81fb39 b98aea5aa06f 208bb941707e 2024-04-24T08:50:57.440 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6 smithi083 running (6m) 32s ago 6m 49.8M 4272M 19.0.0-3116-g1f81fb39 b98aea5aa06f 55e96ecb5117 2024-04-24T08:50:57.441 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7 smithi138 running (6m) 82s ago 6m 51.1M 4272M 19.0.0-3116-g1f81fb39 b98aea5aa06f 0de391613a5a 2024-04-24T08:50:57.441 INFO:teuthology.orchestra.run.smithi083.stdout:osd.8 smithi189 running (5m) 82s ago 5m 42.1M 4272M 19.0.0-3116-g1f81fb39 b98aea5aa06f 544e3f92c74d 2024-04-24T08:50:57.441 INFO:teuthology.orchestra.run.smithi083.stdout:osd.9 smithi083 running (4m) 32s ago 4m 49.9M 4272M 19.0.0-3116-g1f81fb39 b98aea5aa06f e394d0129690 2024-04-24T08:50:57.441 INFO:teuthology.orchestra.run.smithi083.stdout:osd.10 smithi138 running (4m) 82s ago 4m 50.6M 4272M 19.0.0-3116-g1f81fb39 b98aea5aa06f bb82991e68c9 2024-04-24T08:50:57.441 INFO:teuthology.orchestra.run.smithi083.stdout:osd.11 smithi189 running (3m) 82s ago 3m 40.2M 4272M 19.0.0-3116-g1f81fb39 b98aea5aa06f 82b9ae1f76d1 2024-04-24T08:50:57.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:57.442+0000 7f7a15200700 1 -- 172.21.15.83:0/2595508494 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f79f40a3cc0 msgr2=0x7f79f40a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:57.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:57.442+0000 7f7a15200700 1 --2- 172.21.15.83:0/2595508494 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f79f40a3cc0 0x7f79f40a6180 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7a0000b5c0 tx=0x7f7a00005fb0 comp rx=0 tx=0).stop 2024-04-24T08:50:57.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:57.442+0000 7f7a15200700 1 -- 172.21.15.83:0/2595508494 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a10105fb0 msgr2=0x7f7a101992a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:50:57.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:57.442+0000 7f7a15200700 1 --2- 172.21.15.83:0/2595508494 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a10105fb0 0x7f7a101992a0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f79f800c970 tx=0x7f79f800cd30 comp rx=0 tx=0).stop 2024-04-24T08:50:57.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:57.442+0000 7f7a15200700 1 -- 172.21.15.83:0/2595508494 shutdown_connections 2024-04-24T08:50:57.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:57.442+0000 7f7a15200700 1 --2- 172.21.15.83:0/2595508494 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f79f40a3cc0 0x7f79f40a6180 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:57.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:57.442+0000 7f7a15200700 1 --2- 172.21.15.83:0/2595508494 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7a101088e0 0x7f7a101a0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:57.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:57.442+0000 7f7a15200700 1 --2- 172.21.15.83:0/2595508494 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a10103680 0x7f7a10198d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:57.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:57.442+0000 7f7a15200700 1 --2- 172.21.15.83:0/2595508494 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a10105fb0 0x7f7a101992a0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:50:57.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:57.442+0000 7f7a15200700 1 -- 172.21.15.83:0/2595508494 >> 172.21.15.83:0/2595508494 conn(0x7f7a100fcff0 msgr2=0x7f7a10105530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:50:57.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:57.442+0000 7f7a15200700 1 -- 172.21.15.83:0/2595508494 shutdown_connections 2024-04-24T08:50:57.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:50:57.442+0000 7f7a15200700 1 -- 172.21.15.83:0/2595508494 wait complete. 2024-04-24T08:50:58.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:50:57 smithi083 ceph-mon[27907]: pgmap v451: 129 pgs: 129 active+clean; 644 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s; 9 B/s, 0 keys/s, 1 objects/s recovering 2024-04-24T08:50:58.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:50:57 smithi138 ceph-mon[31441]: pgmap v451: 129 pgs: 129 active+clean; 644 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s; 9 B/s, 0 keys/s, 1 objects/s recovering 2024-04-24T08:50:58.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:50:57 smithi189 ceph-mon[31791]: pgmap v451: 129 pgs: 129 active+clean; 644 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s; 9 B/s, 0 keys/s, 1 objects/s recovering 2024-04-24T08:50:58.493 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd3e039e-0215-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ls' 2024-04-24T08:50:58.938 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T08:50:59.279 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:50:58 smithi083 ceph-mon[27907]: from='client.15357 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:50:59.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:50:58 smithi138 ceph-mon[31441]: from='client.15357 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:50:59.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:50:58 smithi189 ceph-mon[31791]: from='client.15357 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:51:00.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:50:59 smithi083 ceph-mon[27907]: pgmap v452: 129 pgs: 129 active+clean; 644 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-24T08:51:00.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:50:59 smithi138 ceph-mon[31441]: pgmap v452: 129 pgs: 129 active+clean; 644 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-24T08:51:00.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:50:59 smithi189 ceph-mon[31791]: pgmap v452: 129 pgs: 129 active+clean; 644 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-24T08:51:01.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:01.359+0000 7f14b1c00700 1 -- 172.21.15.83:0/1803095647 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14ac102ba0 msgr2=0x7f14ac102fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:01.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:01.359+0000 7f14b1c00700 1 --2- 172.21.15.83:0/1803095647 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14ac102ba0 0x7f14ac102fc0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f149c0099c0 tx=0x7f149c009cd0 comp rx=0 tx=0).stop 2024-04-24T08:51:01.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:01.359+0000 7f14b1c00700 1 -- 172.21.15.83:0/1803095647 shutdown_connections 2024-04-24T08:51:01.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:01.359+0000 7f14b1c00700 1 --2- 172.21.15.83:0/1803095647 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14ac102ba0 0x7f14ac102fc0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:01.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:01.359+0000 7f14b1c00700 1 --2- 172.21.15.83:0/1803095647 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f14ac104740 0x7f14ac10aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:01.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:01.359+0000 7f14b1c00700 1 --2- 172.21.15.83:0/1803095647 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f14ac103da0 0x7f14ac104200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:01.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:01.359+0000 7f14b1c00700 1 -- 172.21.15.83:0/1803095647 >> 172.21.15.83:0/1803095647 conn(0x7f14ac0fe140 msgr2=0x7f14ac100580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:01.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:01.359+0000 7f14b1c00700 1 -- 172.21.15.83:0/1803095647 shutdown_connections 2024-04-24T08:51:01.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:01.360+0000 7f14b1c00700 1 -- 172.21.15.83:0/1803095647 wait complete. 2024-04-24T08:51:01.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:01.361+0000 7f14b1c00700 1 Processor -- start 2024-04-24T08:51:01.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:01.361+0000 7f14b1c00700 1 -- start start 2024-04-24T08:51:01.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:01.361+0000 7f14b1c00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f14ac102ba0 0x7f14ac198d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:01.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:01.361+0000 7f14b1c00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14ac103da0 0x7f14ac199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:01.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:01.361+0000 7f14b1c00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f14ac104740 0x7f14ac1a0260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:01.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:01.361+0000 7f14b1c00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f14ac10e630 con 0x7f14ac103da0 2024-04-24T08:51:01.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:01.361+0000 7f14b1c00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f14ac10e4b0 con 0x7f14ac104740 2024-04-24T08:51:01.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:01.361+0000 7f14b1c00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f14ac10e7b0 con 0x7f14ac102ba0 2024-04-24T08:51:01.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:01.361+0000 7f14a3e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14ac103da0 0x7f14ac199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:01.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:01.362+0000 7f14a3e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14ac103da0 0x7f14ac199240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33102/0 (socket says 172.21.15.83:33102) 2024-04-24T08:51:01.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:01.362+0000 7f14a3e00700 1 -- 172.21.15.83:0/3696726435 learned_addr learned my addr 172.21.15.83:0/3696726435 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:51:01.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:01.362+0000 7f14aaa00700 1 --2- 172.21.15.83:0/3696726435 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f14ac102ba0 0x7f14ac198d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:01.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:01.362+0000 7f14ab400700 1 --2- 172.21.15.83:0/3696726435 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f14ac104740 0x7f14ac1a0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:01.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:01.362+0000 7f14a3e00700 1 -- 172.21.15.83:0/3696726435 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f14ac102ba0 msgr2=0x7f14ac198d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:01.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:01.362+0000 7f14a3e00700 1 --2- 172.21.15.83:0/3696726435 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f14ac102ba0 0x7f14ac198d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:01.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:01.362+0000 7f14a3e00700 1 -- 172.21.15.83:0/3696726435 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f14ac104740 msgr2=0x7f14ac1a0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:01.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:01.362+0000 7f14a3e00700 1 --2- 172.21.15.83:0/3696726435 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f14ac104740 0x7f14ac1a0260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:01.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:01.362+0000 7f14a3e00700 1 -- 172.21.15.83:0/3696726435 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14ac1a0930 con 0x7f14ac103da0 2024-04-24T08:51:01.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:01.362+0000 7f14aaa00700 1 --2- 172.21.15.83:0/3696726435 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f14ac102ba0 0x7f14ac198d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:51:01.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:01.362+0000 7f14a3e00700 1 --2- 172.21.15.83:0/3696726435 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14ac103da0 0x7f14ac199240 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f149400e9c0 tx=0x7f149400ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:01.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:01.363+0000 7f14a3400700 1 -- 172.21.15.83:0/3696726435 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f149400cbd0 con 0x7f14ac103da0 2024-04-24T08:51:01.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:01.363+0000 7f14a3400700 1 -- 172.21.15.83:0/3696726435 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f149400cd70 con 0x7f14ac103da0 2024-04-24T08:51:01.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:01.363+0000 7f14a3400700 1 -- 172.21.15.83:0/3696726435 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1494010430 con 0x7f14ac103da0 2024-04-24T08:51:01.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:01.363+0000 7f14b1c00700 1 -- 172.21.15.83:0/3696726435 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14ac1a0c20 con 0x7f14ac103da0 2024-04-24T08:51:01.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:01.363+0000 7f14b1c00700 1 -- 172.21.15.83:0/3696726435 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14ac1a1110 con 0x7f14ac103da0 2024-04-24T08:51:01.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:01.364+0000 7f14b1c00700 1 -- 172.21.15.83:0/3696726435 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14ac0697e0 con 0x7f14ac103da0 2024-04-24T08:51:01.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:01.366+0000 7f14a3400700 1 -- 172.21.15.83:0/3696726435 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f1494004750 con 0x7f14ac103da0 2024-04-24T08:51:01.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:01.368+0000 7f14a3400700 1 --2- 172.21.15.83:0/3696726435 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f14900a3cc0 0x7f14900a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:01.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:01.368+0000 7f14a3400700 1 -- 172.21.15.83:0/3696726435 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9237+0+0 (secure 0 0 0) 0x7f14940d0e10 con 0x7f14ac103da0 2024-04-24T08:51:01.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:01.368+0000 7f14aaa00700 1 --2- 172.21.15.83:0/3696726435 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f14900a3cc0 0x7f14900a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:01.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:01.369+0000 7f14aaa00700 1 --2- 172.21.15.83:0/3696726435 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f14900a3cc0 0x7f14900a6180 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f149c009990 tx=0x7f149c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:01.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:01.371+0000 7f14a3400700 1 -- 172.21.15.83:0/3696726435 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1494053ff0 con 0x7f14ac103da0 2024-04-24T08:51:01.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:01.608+0000 7f14b1c00700 1 -- 172.21.15.83:0/3696726435 --> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f14ac062f30 con 0x7f14900a3cc0 2024-04-24T08:51:01.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:01.625+0000 7f14a3400700 1 -- 172.21.15.83:0/3696726435 <== mgr.14152 v2:172.21.15.83:6800/2550826560 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f14ac062f30 con 0x7f14900a3cc0 2024-04-24T08:51:01.624 INFO:teuthology.orchestra.run.smithi083.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-04-24T08:51:01.624 INFO:teuthology.orchestra.run.smithi083.stdout:mds.all 12/12 86s ago 3m smithi083=a;smithi083=d;smithi083=g;smithi083=j;smithi138=b;smithi138=e;smithi138=h;smithi138=k;smithi189=c;smithi189=f;smithi189=i;smithi189=l;count:12 2024-04-24T08:51:01.624 INFO:teuthology.orchestra.run.smithi083.stdout:mgr 3/3 86s ago 10m smithi083=x;smithi138=y;smithi189=z;count:3 2024-04-24T08:51:01.624 INFO:teuthology.orchestra.run.smithi083.stdout:mon 3/3 86s ago 11m smithi083:172.21.15.83=a;smithi138:172.21.15.138=b;smithi189:172.21.15.189=c;count:3 2024-04-24T08:51:01.624 INFO:teuthology.orchestra.run.smithi083.stdout:osd 12 86s ago - 2024-04-24T08:51:01.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:01.628+0000 7f14b1c00700 1 -- 172.21.15.83:0/3696726435 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f14900a3cc0 msgr2=0x7f14900a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:01.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:01.628+0000 7f14b1c00700 1 --2- 172.21.15.83:0/3696726435 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f14900a3cc0 0x7f14900a6180 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f149c009990 tx=0x7f149c005fb0 comp rx=0 tx=0).stop 2024-04-24T08:51:01.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:01.628+0000 7f14b1c00700 1 -- 172.21.15.83:0/3696726435 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14ac103da0 msgr2=0x7f14ac199240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:01.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:01.628+0000 7f14b1c00700 1 --2- 172.21.15.83:0/3696726435 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14ac103da0 0x7f14ac199240 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f149400e9c0 tx=0x7f149400ed80 comp rx=0 tx=0).stop 2024-04-24T08:51:01.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:01.628+0000 7f14b1c00700 1 -- 172.21.15.83:0/3696726435 shutdown_connections 2024-04-24T08:51:01.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:01.628+0000 7f14b1c00700 1 --2- 172.21.15.83:0/3696726435 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f14900a3cc0 0x7f14900a6180 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:01.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:01.628+0000 7f14b1c00700 1 --2- 172.21.15.83:0/3696726435 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f14ac104740 0x7f14ac1a0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:01.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:01.628+0000 7f14b1c00700 1 --2- 172.21.15.83:0/3696726435 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f14ac102ba0 0x7f14ac198d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:01.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:01.628+0000 7f14b1c00700 1 --2- 172.21.15.83:0/3696726435 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14ac103da0 0x7f14ac199240 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:01.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:01.628+0000 7f14b1c00700 1 -- 172.21.15.83:0/3696726435 >> 172.21.15.83:0/3696726435 conn(0x7f14ac0fe140 msgr2=0x7f14ac1070c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:01.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:01.628+0000 7f14b1c00700 1 -- 172.21.15.83:0/3696726435 shutdown_connections 2024-04-24T08:51:01.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:01.628+0000 7f14b1c00700 1 -- 172.21.15.83:0/3696726435 wait complete. 2024-04-24T08:51:02.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:01 smithi083 ceph-mon[27907]: pgmap v453: 129 pgs: 129 active+clean; 644 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-24T08:51:02.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:01 smithi138 ceph-mon[31441]: pgmap v453: 129 pgs: 129 active+clean; 644 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-24T08:51:02.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:01 smithi189 ceph-mon[31791]: pgmap v453: 129 pgs: 129 active+clean; 644 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-24T08:51:02.553 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd3e039e-0215-11ef-bc93-c7b262605968 -- bash -c 'ceph orch host ls' 2024-04-24T08:51:03.001 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T08:51:03.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:02 smithi083 ceph-mon[27907]: from='client.15363 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:51:03.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:02 smithi138 ceph-mon[31441]: from='client.15363 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:51:03.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:02 smithi189 ceph-mon[31791]: from='client.15363 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:51:04.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:03 smithi083 ceph-mon[27907]: pgmap v454: 129 pgs: 129 active+clean; 644 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-24T08:51:04.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:03 smithi138 ceph-mon[31441]: pgmap v454: 129 pgs: 129 active+clean; 644 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-24T08:51:04.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:03 smithi189 ceph-mon[31791]: pgmap v454: 129 pgs: 129 active+clean; 644 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-24T08:51:05.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:05.435+0000 7f7d0d400700 1 -- 172.21.15.83:0/4034619812 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d08105f90 msgr2=0x7f7d081063f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:05.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:05.435+0000 7f7d0d400700 1 --2- 172.21.15.83:0/4034619812 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d08105f90 0x7f7d081063f0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f7cf40099c0 tx=0x7f7cf4009cd0 comp rx=0 tx=0).stop 2024-04-24T08:51:05.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:05.436+0000 7f7d0d400700 1 -- 172.21.15.83:0/4034619812 shutdown_connections 2024-04-24T08:51:05.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:05.436+0000 7f7d0d400700 1 --2- 172.21.15.83:0/4034619812 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d08106930 0x7f7d0810d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:05.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:05.436+0000 7f7d0d400700 1 --2- 172.21.15.83:0/4034619812 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7d08104d90 0x7f7d081051b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:05.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:05.436+0000 7f7d0d400700 1 --2- 172.21.15.83:0/4034619812 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d08105f90 0x7f7d081063f0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:05.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:05.436+0000 7f7d0d400700 1 -- 172.21.15.83:0/4034619812 >> 172.21.15.83:0/4034619812 conn(0x7f7d08100350 msgr2=0x7f7d08102770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:05.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:05.436+0000 7f7d0d400700 1 -- 172.21.15.83:0/4034619812 shutdown_connections 2024-04-24T08:51:05.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:05.436+0000 7f7d0d400700 1 -- 172.21.15.83:0/4034619812 wait complete. 2024-04-24T08:51:05.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:05.437+0000 7f7d0d400700 1 Processor -- start 2024-04-24T08:51:05.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:05.437+0000 7f7d0d400700 1 -- start start 2024-04-24T08:51:05.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:05.437+0000 7f7d0d400700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d08104d90 0x7f7d0819ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:05.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:05.437+0000 7f7d0d400700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7d08105f90 0x7f7d0819b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:05.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:05.437+0000 7f7d0d400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d08106930 0x7f7d081a22c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:05.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:05.437+0000 7f7d0d400700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7d08110800 con 0x7f7d08106930 2024-04-24T08:51:05.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:05.437+0000 7f7d0d400700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7d08110680 con 0x7f7d08104d90 2024-04-24T08:51:05.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:05.437+0000 7f7d0d400700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7d08110980 con 0x7f7d08105f90 2024-04-24T08:51:05.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:05.438+0000 7f7d07200700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d08106930 0x7f7d081a22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:05.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:05.438+0000 7f7d07200700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d08106930 0x7f7d081a22c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33126/0 (socket says 172.21.15.83:33126) 2024-04-24T08:51:05.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:05.438+0000 7f7d07200700 1 -- 172.21.15.83:0/1436807958 learned_addr learned my addr 172.21.15.83:0/1436807958 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:51:05.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:05.438+0000 7f7d06800700 1 --2- 172.21.15.83:0/1436807958 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d08104d90 0x7f7d0819ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:05.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:05.438+0000 7f7d05e00700 1 --2- 172.21.15.83:0/1436807958 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7d08105f90 0x7f7d0819b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:05.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:05.438+0000 7f7d07200700 1 -- 172.21.15.83:0/1436807958 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7d08105f90 msgr2=0x7f7d0819b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:05.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:05.438+0000 7f7d07200700 1 --2- 172.21.15.83:0/1436807958 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7d08105f90 0x7f7d0819b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:05.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:05.438+0000 7f7d07200700 1 -- 172.21.15.83:0/1436807958 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d08104d90 msgr2=0x7f7d0819ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:05.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:05.438+0000 7f7d07200700 1 --2- 172.21.15.83:0/1436807958 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d08104d90 0x7f7d0819ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:05.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:05.438+0000 7f7d07200700 1 -- 172.21.15.83:0/1436807958 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d081a2990 con 0x7f7d08106930 2024-04-24T08:51:05.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:05.438+0000 7f7d07200700 1 --2- 172.21.15.83:0/1436807958 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d08106930 0x7f7d081a22c0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f7d00007d70 tx=0x7f7d0000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:05.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:05.439+0000 7f7cff400700 1 -- 172.21.15.83:0/1436807958 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d00016020 con 0x7f7d08106930 2024-04-24T08:51:05.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:05.439+0000 7f7cff400700 1 -- 172.21.15.83:0/1436807958 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7d0000ad40 con 0x7f7d08106930 2024-04-24T08:51:05.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:05.439+0000 7f7cff400700 1 -- 172.21.15.83:0/1436807958 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d00011620 con 0x7f7d08106930 2024-04-24T08:51:05.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:05.439+0000 7f7d0d400700 1 -- 172.21.15.83:0/1436807958 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d081a2c80 con 0x7f7d08106930 2024-04-24T08:51:05.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:05.439+0000 7f7d0d400700 1 -- 172.21.15.83:0/1436807958 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d081a3170 con 0x7f7d08106930 2024-04-24T08:51:05.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:05.440+0000 7f7d0d400700 1 -- 172.21.15.83:0/1436807958 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d080697e0 con 0x7f7d08106930 2024-04-24T08:51:05.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:05.442+0000 7f7cff400700 1 -- 172.21.15.83:0/1436807958 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f7d00019020 con 0x7f7d08106930 2024-04-24T08:51:05.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:05.444+0000 7f7cff400700 1 --2- 172.21.15.83:0/1436807958 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f7cec0a3cc0 0x7f7cec0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:05.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:05.444+0000 7f7cff400700 1 -- 172.21.15.83:0/1436807958 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9237+0+0 (secure 0 0 0) 0x7f7d000d0830 con 0x7f7d08106930 2024-04-24T08:51:05.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:05.444+0000 7f7d06800700 1 --2- 172.21.15.83:0/1436807958 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f7cec0a3cc0 0x7f7cec0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:05.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:05.445+0000 7f7d06800700 1 --2- 172.21.15.83:0/1436807958 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f7cec0a3cc0 0x7f7cec0a6180 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f7d0819c220 tx=0x7f7cf800a5e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:05.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:05.448+0000 7f7cff400700 1 -- 172.21.15.83:0/1436807958 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7d00020020 con 0x7f7d08106930 2024-04-24T08:51:05.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:05.679+0000 7f7d0d400700 1 -- 172.21.15.83:0/1436807958 --> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f7d08062f30 con 0x7f7cec0a3cc0 2024-04-24T08:51:05.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:05.682+0000 7f7cff400700 1 -- 172.21.15.83:0/1436807958 <== mgr.14152 v2:172.21.15.83:6800/2550826560 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f7d08062f30 con 0x7f7cec0a3cc0 2024-04-24T08:51:05.681 INFO:teuthology.orchestra.run.smithi083.stdout:HOST ADDR LABELS STATUS 2024-04-24T08:51:05.682 INFO:teuthology.orchestra.run.smithi083.stdout:smithi083 172.21.15.83 2024-04-24T08:51:05.682 INFO:teuthology.orchestra.run.smithi083.stdout:smithi138 172.21.15.138 2024-04-24T08:51:05.682 INFO:teuthology.orchestra.run.smithi083.stdout:smithi189 172.21.15.189 2024-04-24T08:51:05.682 INFO:teuthology.orchestra.run.smithi083.stdout:3 hosts in cluster 2024-04-24T08:51:05.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:05.685+0000 7f7d0d400700 1 -- 172.21.15.83:0/1436807958 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f7cec0a3cc0 msgr2=0x7f7cec0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:05.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:05.685+0000 7f7d0d400700 1 --2- 172.21.15.83:0/1436807958 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f7cec0a3cc0 0x7f7cec0a6180 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f7d0819c220 tx=0x7f7cf800a5e0 comp rx=0 tx=0).stop 2024-04-24T08:51:05.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:05.685+0000 7f7d0d400700 1 -- 172.21.15.83:0/1436807958 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d08106930 msgr2=0x7f7d081a22c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:05.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:05.685+0000 7f7d0d400700 1 --2- 172.21.15.83:0/1436807958 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d08106930 0x7f7d081a22c0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f7d00007d70 tx=0x7f7d0000a340 comp rx=0 tx=0).stop 2024-04-24T08:51:05.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:05.685+0000 7f7d0d400700 1 -- 172.21.15.83:0/1436807958 shutdown_connections 2024-04-24T08:51:05.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:05.685+0000 7f7d0d400700 1 --2- 172.21.15.83:0/1436807958 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f7cec0a3cc0 0x7f7cec0a6180 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:05.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:05.685+0000 7f7d0d400700 1 --2- 172.21.15.83:0/1436807958 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d08106930 0x7f7d081a22c0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:05.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:05.685+0000 7f7d0d400700 1 --2- 172.21.15.83:0/1436807958 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d08104d90 0x7f7d0819ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:05.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:05.685+0000 7f7d0d400700 1 --2- 172.21.15.83:0/1436807958 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7d08105f90 0x7f7d0819b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:05.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:05.685+0000 7f7d0d400700 1 -- 172.21.15.83:0/1436807958 >> 172.21.15.83:0/1436807958 conn(0x7f7d08100350 msgr2=0x7f7d081091c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:05.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:05.685+0000 7f7d0d400700 1 -- 172.21.15.83:0/1436807958 shutdown_connections 2024-04-24T08:51:05.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:05.685+0000 7f7d0d400700 1 -- 172.21.15.83:0/1436807958 wait complete. 2024-04-24T08:51:06.257 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:06 smithi138 ceph-mon[31441]: pgmap v455: 129 pgs: 129 active+clean; 644 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-04-24T08:51:06.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:06 smithi083 ceph-mon[27907]: pgmap v455: 129 pgs: 129 active+clean; 644 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-04-24T08:51:06.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:06 smithi189 ceph-mon[31791]: pgmap v455: 129 pgs: 129 active+clean; 644 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-04-24T08:51:06.614 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd3e039e-0215-11ef-bc93-c7b262605968 -- bash -c 'ceph orch device ls' 2024-04-24T08:51:07.056 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T08:51:07.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:07 smithi083 ceph-mon[27907]: from='client.15369 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:51:07.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:07 smithi138 ceph-mon[31441]: from='client.15369 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:51:07.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:07 smithi189 ceph-mon[31791]: from='client.15369 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:51:08.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:08 smithi083 ceph-mon[27907]: pgmap v456: 129 pgs: 129 active+clean; 644 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-24T08:51:08.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:08 smithi138 ceph-mon[31441]: pgmap v456: 129 pgs: 129 active+clean; 644 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-24T08:51:08.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:08 smithi189 ceph-mon[31791]: pgmap v456: 129 pgs: 129 active+clean; 644 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-24T08:51:09.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:09.518+0000 7feeace00700 1 -- 172.21.15.83:0/2723148385 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feea8106830 msgr2=0x7feea810d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:09.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:09.518+0000 7feeace00700 1 --2- 172.21.15.83:0/2723148385 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feea8106830 0x7feea810d0e0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7feea000b0d0 tx=0x7feea000b3e0 comp rx=0 tx=0).stop 2024-04-24T08:51:09.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:09.518+0000 7feeace00700 1 -- 172.21.15.83:0/2723148385 shutdown_connections 2024-04-24T08:51:09.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:09.518+0000 7feeace00700 1 --2- 172.21.15.83:0/2723148385 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feea8106830 0x7feea810d0e0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:09.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:09.518+0000 7feeace00700 1 --2- 172.21.15.83:0/2723148385 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feea8104c90 0x7feea81050b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:09.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:09.518+0000 7feeace00700 1 --2- 172.21.15.83:0/2723148385 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feea8105e90 0x7feea81062f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:09.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:09.518+0000 7feeace00700 1 -- 172.21.15.83:0/2723148385 >> 172.21.15.83:0/2723148385 conn(0x7feea8100230 msgr2=0x7feea8102670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:09.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:09.519+0000 7feeace00700 1 -- 172.21.15.83:0/2723148385 shutdown_connections 2024-04-24T08:51:09.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:09.519+0000 7feeace00700 1 -- 172.21.15.83:0/2723148385 wait complete. 2024-04-24T08:51:09.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:09.520+0000 7feeace00700 1 Processor -- start 2024-04-24T08:51:09.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:09.520+0000 7feeace00700 1 -- start start 2024-04-24T08:51:09.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:09.520+0000 7feeace00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feea8104c90 0x7feea819ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:09.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:09.520+0000 7feeace00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feea8105e90 0x7feea819b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:09.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:09.520+0000 7feeace00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feea8106830 0x7feea81a2290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:09.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:09.520+0000 7feeace00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7feea8110760 con 0x7feea8105e90 2024-04-24T08:51:09.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:09.520+0000 7feeace00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7feea81105e0 con 0x7feea8106830 2024-04-24T08:51:09.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:09.520+0000 7feeace00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7feea81108e0 con 0x7feea8104c90 2024-04-24T08:51:09.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:09.521+0000 7fee9fe00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feea8105e90 0x7feea819b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:09.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:09.521+0000 7fee9fe00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feea8105e90 0x7feea819b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43318/0 (socket says 172.21.15.83:43318) 2024-04-24T08:51:09.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:09.521+0000 7fee9fe00700 1 -- 172.21.15.83:0/3029003215 learned_addr learned my addr 172.21.15.83:0/3029003215 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:51:09.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:09.521+0000 7feea7400700 1 --2- 172.21.15.83:0/3029003215 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feea8104c90 0x7feea819ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:09.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:09.521+0000 7feea7e00700 1 --2- 172.21.15.83:0/3029003215 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feea8106830 0x7feea81a2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:09.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:09.521+0000 7fee9fe00700 1 -- 172.21.15.83:0/3029003215 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feea8104c90 msgr2=0x7feea819ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:09.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:09.521+0000 7fee9fe00700 1 --2- 172.21.15.83:0/3029003215 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feea8104c90 0x7feea819ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:09.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:09.521+0000 7fee9fe00700 1 -- 172.21.15.83:0/3029003215 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feea8106830 msgr2=0x7feea81a2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:09.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:09.521+0000 7fee9fe00700 1 --2- 172.21.15.83:0/3029003215 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feea8106830 0x7feea81a2290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:09.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:09.521+0000 7fee9fe00700 1 -- 172.21.15.83:0/3029003215 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feea81a2960 con 0x7feea8105e90 2024-04-24T08:51:09.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:09.521+0000 7feea7e00700 1 --2- 172.21.15.83:0/3029003215 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feea8106830 0x7feea81a2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:51:09.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:09.522+0000 7fee9fe00700 1 --2- 172.21.15.83:0/3029003215 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feea8105e90 0x7feea819b250 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fee9000d950 tx=0x7fee9000dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:09.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:09.522+0000 7feea4c00700 1 -- 172.21.15.83:0/3029003215 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee90016070 con 0x7feea8105e90 2024-04-24T08:51:09.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:09.522+0000 7feea4c00700 1 -- 172.21.15.83:0/3029003215 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fee90010460 con 0x7feea8105e90 2024-04-24T08:51:09.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:09.522+0000 7feea4c00700 1 -- 172.21.15.83:0/3029003215 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee9000f620 con 0x7feea8105e90 2024-04-24T08:51:09.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:09.522+0000 7feeace00700 1 -- 172.21.15.83:0/3029003215 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feea81a2c50 con 0x7feea8105e90 2024-04-24T08:51:09.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:09.522+0000 7feeace00700 1 -- 172.21.15.83:0/3029003215 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feea81a3190 con 0x7feea8105e90 2024-04-24T08:51:09.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:09.523+0000 7feeace00700 1 -- 172.21.15.83:0/3029003215 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feea80697e0 con 0x7feea8105e90 2024-04-24T08:51:09.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:09.525+0000 7feea4c00700 1 -- 172.21.15.83:0/3029003215 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fee90019070 con 0x7feea8105e90 2024-04-24T08:51:09.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:09.527+0000 7feea4c00700 1 --2- 172.21.15.83:0/3029003215 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fee8c0a3cc0 0x7fee8c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:09.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:09.527+0000 7feea4c00700 1 -- 172.21.15.83:0/3029003215 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9237+0+0 (secure 0 0 0) 0x7fee900d0410 con 0x7feea8105e90 2024-04-24T08:51:09.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:09.527+0000 7feea7400700 1 --2- 172.21.15.83:0/3029003215 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fee8c0a3cc0 0x7fee8c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:09.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:09.528+0000 7feea7400700 1 --2- 172.21.15.83:0/3029003215 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fee8c0a3cc0 0x7fee8c0a6180 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fee98005fd0 tx=0x7fee98005dd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:09.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:09.530+0000 7feea4c00700 1 -- 172.21.15.83:0/3029003215 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fee9009c9a0 con 0x7feea8105e90 2024-04-24T08:51:09.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:09.766+0000 7feeace00700 1 -- 172.21.15.83:0/3029003215 --> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7feea8062ff0 con 0x7fee8c0a3cc0 2024-04-24T08:51:09.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:09.769+0000 7feea4c00700 1 -- 172.21.15.83:0/3029003215 <== mgr.14152 v2:172.21.15.83:6800/2550826560 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7feea8062ff0 con 0x7fee8c0a3cc0 2024-04-24T08:51:09.768 INFO:teuthology.orchestra.run.smithi083.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-04-24T08:51:09.769 INFO:teuthology.orchestra.run.smithi083.stdout:smithi083 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT62040148400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-04-24T08:51:09.769 INFO:teuthology.orchestra.run.smithi083.stdout:smithi138 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6161003C400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-04-24T08:51:09.769 INFO:teuthology.orchestra.run.smithi083.stdout:smithi189 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6232004L400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-04-24T08:51:09.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:09.772+0000 7feeace00700 1 -- 172.21.15.83:0/3029003215 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fee8c0a3cc0 msgr2=0x7fee8c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:09.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:09.772+0000 7feeace00700 1 --2- 172.21.15.83:0/3029003215 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fee8c0a3cc0 0x7fee8c0a6180 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fee98005fd0 tx=0x7fee98005dd0 comp rx=0 tx=0).stop 2024-04-24T08:51:09.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:09.772+0000 7feeace00700 1 -- 172.21.15.83:0/3029003215 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feea8105e90 msgr2=0x7feea819b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:09.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:09.772+0000 7feeace00700 1 --2- 172.21.15.83:0/3029003215 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feea8105e90 0x7feea819b250 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fee9000d950 tx=0x7fee9000dd10 comp rx=0 tx=0).stop 2024-04-24T08:51:09.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:09.773+0000 7feeace00700 1 -- 172.21.15.83:0/3029003215 shutdown_connections 2024-04-24T08:51:09.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:09.773+0000 7feeace00700 1 --2- 172.21.15.83:0/3029003215 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fee8c0a3cc0 0x7fee8c0a6180 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:09.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:09.773+0000 7feeace00700 1 --2- 172.21.15.83:0/3029003215 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feea8106830 0x7feea81a2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:09.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:09.773+0000 7feeace00700 1 --2- 172.21.15.83:0/3029003215 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feea8104c90 0x7feea819ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:09.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:09.773+0000 7feeace00700 1 --2- 172.21.15.83:0/3029003215 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feea8105e90 0x7feea819b250 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:09.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:09.773+0000 7feeace00700 1 -- 172.21.15.83:0/3029003215 >> 172.21.15.83:0/3029003215 conn(0x7feea8100230 msgr2=0x7feea8109130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:09.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:09.773+0000 7feeace00700 1 -- 172.21.15.83:0/3029003215 shutdown_connections 2024-04-24T08:51:09.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:09.774+0000 7feeace00700 1 -- 172.21.15.83:0/3029003215 wait complete. 2024-04-24T08:51:10.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:10 smithi083 ceph-mon[27907]: pgmap v457: 129 pgs: 129 active+clean; 644 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-24T08:51:10.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:10 smithi138 ceph-mon[31441]: pgmap v457: 129 pgs: 129 active+clean; 644 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-24T08:51:10.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:10 smithi189 ceph-mon[31791]: pgmap v457: 129 pgs: 129 active+clean; 644 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-24T08:51:10.751 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-24T08:51:10.760 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi083.front.sepia.ceph.com 2024-04-24T08:51:10.760 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd3e039e-0215-11ef-bc93-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-24T08:51:11.197 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T08:51:11.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:11 smithi083 ceph-mon[27907]: from='client.15375 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:51:11.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:11 smithi138 ceph-mon[31441]: from='client.15375 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:51:11.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:11 smithi189 ceph-mon[31791]: from='client.15375 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:51:12.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:12 smithi083 ceph-mon[27907]: pgmap v458: 129 pgs: 129 active+clean; 644 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-24T08:51:12.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:12 smithi138 ceph-mon[31441]: pgmap v458: 129 pgs: 129 active+clean; 644 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-24T08:51:12.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:12 smithi189 ceph-mon[31791]: pgmap v458: 129 pgs: 129 active+clean; 644 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-24T08:51:13.237 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:13 smithi083 ceph-mon[27907]: pgmap v459: 129 pgs: 129 active+clean; 644 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-24T08:51:13.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:13 smithi138 ceph-mon[31441]: pgmap v459: 129 pgs: 129 active+clean; 644 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-24T08:51:13.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:13.454+0000 7f2adcc00700 1 -- 172.21.15.83:0/687455517 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ad8105f90 msgr2=0x7f2ad81063f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:13.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:13.454+0000 7f2adcc00700 1 --2- 172.21.15.83:0/687455517 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ad8105f90 0x7f2ad81063f0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f2ac40099c0 tx=0x7f2ac4009cd0 comp rx=0 tx=0).stop 2024-04-24T08:51:13.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:13.454+0000 7f2adcc00700 1 -- 172.21.15.83:0/687455517 shutdown_connections 2024-04-24T08:51:13.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:13.454+0000 7f2adcc00700 1 --2- 172.21.15.83:0/687455517 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ad8106930 0x7f2ad810d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:13.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:13.454+0000 7f2adcc00700 1 --2- 172.21.15.83:0/687455517 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2ad8104d90 0x7f2ad81051b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:13.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:13.454+0000 7f2adcc00700 1 --2- 172.21.15.83:0/687455517 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ad8105f90 0x7f2ad81063f0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:13.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:13.454+0000 7f2adcc00700 1 -- 172.21.15.83:0/687455517 >> 172.21.15.83:0/687455517 conn(0x7f2ad8100350 msgr2=0x7f2ad8102770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:13.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:13.454+0000 7f2adcc00700 1 -- 172.21.15.83:0/687455517 shutdown_connections 2024-04-24T08:51:13.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:13.454+0000 7f2adcc00700 1 -- 172.21.15.83:0/687455517 wait complete. 2024-04-24T08:51:13.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:13.456+0000 7f2adcc00700 1 Processor -- start 2024-04-24T08:51:13.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:13.456+0000 7f2adcc00700 1 -- start start 2024-04-24T08:51:13.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:13.456+0000 7f2adcc00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2ad8104d90 0x7f2ad819ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:13.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:13.456+0000 7f2adcc00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ad8105f90 0x7f2ad819b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:13.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:13.456+0000 7f2adcc00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ad8106930 0x7f2ad81a23a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:13.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:13.456+0000 7f2adcc00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2ad8110850 con 0x7f2ad8105f90 2024-04-24T08:51:13.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:13.456+0000 7f2adcc00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2ad81106d0 con 0x7f2ad8106930 2024-04-24T08:51:13.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:13.456+0000 7f2adcc00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2ad81109d0 con 0x7f2ad8104d90 2024-04-24T08:51:13.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:13.457+0000 7f2ad5400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ad8105f90 0x7f2ad819b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:13.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:13.457+0000 7f2ad5400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ad8105f90 0x7f2ad819b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43330/0 (socket says 172.21.15.83:43330) 2024-04-24T08:51:13.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:13.457+0000 7f2ad5400700 1 -- 172.21.15.83:0/1946323494 learned_addr learned my addr 172.21.15.83:0/1946323494 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:51:13.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:13.457+0000 7f2ad5e00700 1 --2- 172.21.15.83:0/1946323494 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2ad8104d90 0x7f2ad819ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:13.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:13 smithi189 ceph-mon[31791]: pgmap v459: 129 pgs: 129 active+clean; 644 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-24T08:51:13.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:13.457+0000 7f2ad6800700 1 --2- 172.21.15.83:0/1946323494 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ad8106930 0x7f2ad81a23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:13.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:13.457+0000 7f2ad5400700 1 -- 172.21.15.83:0/1946323494 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2ad8104d90 msgr2=0x7f2ad819ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:13.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:13.457+0000 7f2ad5400700 1 --2- 172.21.15.83:0/1946323494 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2ad8104d90 0x7f2ad819ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:13.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:13.457+0000 7f2ad5400700 1 -- 172.21.15.83:0/1946323494 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ad8106930 msgr2=0x7f2ad81a23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:13.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:13.457+0000 7f2ad5400700 1 --2- 172.21.15.83:0/1946323494 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ad8106930 0x7f2ad81a23a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:13.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:13.457+0000 7f2ad5400700 1 -- 172.21.15.83:0/1946323494 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ad81a2a40 con 0x7f2ad8105f90 2024-04-24T08:51:13.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:13.457+0000 7f2ad6800700 1 --2- 172.21.15.83:0/1946323494 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ad8106930 0x7f2ad81a23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:51:13.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:13.458+0000 7f2ad5400700 1 --2- 172.21.15.83:0/1946323494 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ad8105f90 0x7f2ad819b320 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f2ac4005910 tx=0x7f2ac40047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:13.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:13.458+0000 7f2acea00700 1 -- 172.21.15.83:0/1946323494 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac4027070 con 0x7f2ad8105f90 2024-04-24T08:51:13.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:13.458+0000 7f2acea00700 1 -- 172.21.15.83:0/1946323494 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2ac40040d0 con 0x7f2ad8105f90 2024-04-24T08:51:13.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:13.458+0000 7f2acea00700 1 -- 172.21.15.83:0/1946323494 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac400f460 con 0x7f2ad8105f90 2024-04-24T08:51:13.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:13.458+0000 7f2adcc00700 1 -- 172.21.15.83:0/1946323494 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ad81a2cd0 con 0x7f2ad8105f90 2024-04-24T08:51:13.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:13.458+0000 7f2adcc00700 1 -- 172.21.15.83:0/1946323494 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ad81a30d0 con 0x7f2ad8105f90 2024-04-24T08:51:13.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:13.459+0000 7f2adcc00700 1 -- 172.21.15.83:0/1946323494 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ad80697e0 con 0x7f2ad8105f90 2024-04-24T08:51:13.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:13.463+0000 7f2acea00700 1 -- 172.21.15.83:0/1946323494 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f2ac402a070 con 0x7f2ad8105f90 2024-04-24T08:51:13.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:13.464+0000 7f2acea00700 1 --2- 172.21.15.83:0/1946323494 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f2abc0a3c40 0x7f2abc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:13.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:13.464+0000 7f2acea00700 1 -- 172.21.15.83:0/1946323494 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9237+0+0 (secure 0 0 0) 0x7f2ac40d1fe0 con 0x7f2ad8105f90 2024-04-24T08:51:13.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:13.464+0000 7f2ad5e00700 1 --2- 172.21.15.83:0/1946323494 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f2abc0a3c40 0x7f2abc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:13.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:13.465+0000 7f2ad5e00700 1 --2- 172.21.15.83:0/1946323494 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f2abc0a3c40 0x7f2abc0a6100 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f2ac8005fd0 tx=0x7f2ac8005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:13.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:13.467+0000 7f2acea00700 1 -- 172.21.15.83:0/1946323494 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2ac4031070 con 0x7f2ad8105f90 2024-04-24T08:51:13.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:13.742+0000 7f2adcc00700 1 -- 172.21.15.83:0/1946323494 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f2ad802e0b0 con 0x7f2ad8105f90 2024-04-24T08:51:13.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:13.744+0000 7f2acea00700 1 -- 172.21.15.83:0/1946323494 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 27 v27) ==== 76+0+3100 (secure 0 0 0) 0x7f2ac4013070 con 0x7f2ad8105f90 2024-04-24T08:51:13.745 INFO:teuthology.orchestra.run.smithi083.stdout:e27 2024-04-24T08:51:13.745 INFO:teuthology.orchestra.run.smithi083.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-24T08:51:13.745 INFO:teuthology.orchestra.run.smithi083.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-24T08:51:13.745 INFO:teuthology.orchestra.run.smithi083.stdout:legacy client fscid: 1 2024-04-24T08:51:13.745 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:51:13.745 INFO:teuthology.orchestra.run.smithi083.stdout:Filesystem 'cephfs' (1) 2024-04-24T08:51:13.745 INFO:teuthology.orchestra.run.smithi083.stdout:fs_name cephfs 2024-04-24T08:51:13.745 INFO:teuthology.orchestra.run.smithi083.stdout:epoch 27 2024-04-24T08:51:13.745 INFO:teuthology.orchestra.run.smithi083.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-24T08:51:13.745 INFO:teuthology.orchestra.run.smithi083.stdout:created 2024-04-24T08:48:29.422576+0000 2024-04-24T08:51:13.745 INFO:teuthology.orchestra.run.smithi083.stdout:modified 2024-04-24T08:49:04.217098+0000 2024-04-24T08:51:13.745 INFO:teuthology.orchestra.run.smithi083.stdout:tableserver 0 2024-04-24T08:51:13.746 INFO:teuthology.orchestra.run.smithi083.stdout:root 0 2024-04-24T08:51:13.746 INFO:teuthology.orchestra.run.smithi083.stdout:session_timeout 300 2024-04-24T08:51:13.746 INFO:teuthology.orchestra.run.smithi083.stdout:session_autoclose 300 2024-04-24T08:51:13.746 INFO:teuthology.orchestra.run.smithi083.stdout:max_file_size 1099511627776 2024-04-24T08:51:13.746 INFO:teuthology.orchestra.run.smithi083.stdout:max_xattr_size 65536 2024-04-24T08:51:13.746 INFO:teuthology.orchestra.run.smithi083.stdout:required_client_features {} 2024-04-24T08:51:13.746 INFO:teuthology.orchestra.run.smithi083.stdout:last_failure 0 2024-04-24T08:51:13.746 INFO:teuthology.orchestra.run.smithi083.stdout:last_failure_osd_epoch 0 2024-04-24T08:51:13.746 INFO:teuthology.orchestra.run.smithi083.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-24T08:51:13.746 INFO:teuthology.orchestra.run.smithi083.stdout:max_mds 5 2024-04-24T08:51:13.746 INFO:teuthology.orchestra.run.smithi083.stdout:in 0,1,2,3,4 2024-04-24T08:51:13.746 INFO:teuthology.orchestra.run.smithi083.stdout:up {0=34520,1=34493,2=34451,3=34427,4=24454} 2024-04-24T08:51:13.746 INFO:teuthology.orchestra.run.smithi083.stdout:failed 2024-04-24T08:51:13.746 INFO:teuthology.orchestra.run.smithi083.stdout:damaged 2024-04-24T08:51:13.747 INFO:teuthology.orchestra.run.smithi083.stdout:stopped 2024-04-24T08:51:13.747 INFO:teuthology.orchestra.run.smithi083.stdout:data_pools [3,4] 2024-04-24T08:51:13.747 INFO:teuthology.orchestra.run.smithi083.stdout:metadata_pool 2 2024-04-24T08:51:13.747 INFO:teuthology.orchestra.run.smithi083.stdout:inline_data disabled 2024-04-24T08:51:13.747 INFO:teuthology.orchestra.run.smithi083.stdout:balancer 2024-04-24T08:51:13.747 INFO:teuthology.orchestra.run.smithi083.stdout:bal_rank_mask -1 2024-04-24T08:51:13.747 INFO:teuthology.orchestra.run.smithi083.stdout:standby_count_wanted 0 2024-04-24T08:51:13.747 INFO:teuthology.orchestra.run.smithi083.stdout:qdb_cluster leader: 34520 members: 34427,24454,34520,34493,34451 2024-04-24T08:51:13.747 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.b{0:34520} state up:active seq 4 addr [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T08:51:13.747 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.c{0:24439} state up:standby-replay seq 1 addr [v2:172.21.15.189:6836/3172311220,v1:172.21.15.189:6837/3172311220] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T08:51:13.747 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.h{1:34493} state up:active seq 15 addr [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T08:51:13.747 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.f{1:24421} state up:standby-replay seq 1 addr [v2:172.21.15.189:6834/2241923197,v1:172.21.15.189:6835/2241923197] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T08:51:13.747 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.e{2:34451} state up:active seq 19 addr [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T08:51:13.747 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.l{2:24382} state up:standby-replay seq 1 addr [v2:172.21.15.189:6832/1859921310,v1:172.21.15.189:6833/1859921310] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T08:51:13.748 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.k{3:34427} state up:active seq 22 addr [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T08:51:13.748 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.a{3:14676} state up:standby-replay seq 1 addr [v2:172.21.15.83:6840/2634754210,v1:172.21.15.83:6841/2634754210] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T08:51:13.748 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.i{4:24454} state up:active seq 15 addr [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T08:51:13.748 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.g{4:14658} state up:standby-replay seq 1 addr [v2:172.21.15.83:6838/4085470844,v1:172.21.15.83:6839/4085470844] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T08:51:13.748 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:51:13.748 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:51:13.748 INFO:teuthology.orchestra.run.smithi083.stdout:Standby daemons: 2024-04-24T08:51:13.748 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:51:13.748 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.j{-1:14559} state up:standby seq 1 addr [v2:172.21.15.83:6834/358872257,v1:172.21.15.83:6835/358872257] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T08:51:13.748 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.d{-1:14571} state up:standby seq 1 addr [v2:172.21.15.83:6836/3157070851,v1:172.21.15.83:6837/3157070851] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T08:51:13.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:13.747+0000 7f2adcc00700 1 -- 172.21.15.83:0/1946323494 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f2abc0a3c40 msgr2=0x7f2abc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:13.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:13.747+0000 7f2adcc00700 1 --2- 172.21.15.83:0/1946323494 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f2abc0a3c40 0x7f2abc0a6100 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f2ac8005fd0 tx=0x7f2ac8005e50 comp rx=0 tx=0).stop 2024-04-24T08:51:13.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:13.747+0000 7f2adcc00700 1 -- 172.21.15.83:0/1946323494 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ad8105f90 msgr2=0x7f2ad819b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:13.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:13.747+0000 7f2adcc00700 1 --2- 172.21.15.83:0/1946323494 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ad8105f90 0x7f2ad819b320 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f2ac4005910 tx=0x7f2ac40047b0 comp rx=0 tx=0).stop 2024-04-24T08:51:13.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:13.747+0000 7f2adcc00700 1 -- 172.21.15.83:0/1946323494 shutdown_connections 2024-04-24T08:51:13.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:13.747+0000 7f2adcc00700 1 --2- 172.21.15.83:0/1946323494 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f2abc0a3c40 0x7f2abc0a6100 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:13.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:13.747+0000 7f2adcc00700 1 --2- 172.21.15.83:0/1946323494 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ad8106930 0x7f2ad81a23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:13.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:13.747+0000 7f2adcc00700 1 --2- 172.21.15.83:0/1946323494 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2ad8104d90 0x7f2ad819ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:13.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:13.747+0000 7f2adcc00700 1 --2- 172.21.15.83:0/1946323494 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ad8105f90 0x7f2ad819b320 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:13.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:13.747+0000 7f2adcc00700 1 -- 172.21.15.83:0/1946323494 >> 172.21.15.83:0/1946323494 conn(0x7f2ad8100350 msgr2=0x7f2ad8109290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:13.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:13.747+0000 7f2adcc00700 1 -- 172.21.15.83:0/1946323494 shutdown_connections 2024-04-24T08:51:13.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:13.747+0000 7f2adcc00700 1 -- 172.21.15.83:0/1946323494 wait complete. 2024-04-24T08:51:13.750 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 27 2024-04-24T08:51:14.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:14 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/1946323494' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T08:51:14.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:14 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/1946323494' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T08:51:14.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:14 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/1946323494' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T08:51:14.681 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bd3e039e-0215-11ef-bc93-c7b262605968 -- bash -c 'ceph osd dump' 2024-04-24T08:51:15.127 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T08:51:15.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:15 smithi138 ceph-mon[31441]: pgmap v460: 129 pgs: 129 active+clean; 644 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-04-24T08:51:15.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:15 smithi189 ceph-mon[31791]: pgmap v460: 129 pgs: 129 active+clean; 644 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-04-24T08:51:15.459 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:15 smithi083 ceph-mon[27907]: pgmap v460: 129 pgs: 129 active+clean; 644 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-04-24T08:51:17.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:17.412+0000 7f4739e00700 1 -- 172.21.15.83:0/3583900902 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47340790b0 msgr2=0x7f4734079930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:17.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:17.412+0000 7f4739e00700 1 --2- 172.21.15.83:0/3583900902 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47340790b0 0x7f4734079930 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f472c00b0a0 tx=0x7f472c00b3b0 comp rx=0 tx=0).stop 2024-04-24T08:51:17.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:17.412+0000 7f4739e00700 1 -- 172.21.15.83:0/3583900902 shutdown_connections 2024-04-24T08:51:17.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:17.412+0000 7f4739e00700 1 --2- 172.21.15.83:0/3583900902 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47340790b0 0x7f4734079930 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:17.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:17.412+0000 7f4739e00700 1 --2- 172.21.15.83:0/3583900902 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f47340774a0 0x7f47340778c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:17.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:17.412+0000 7f4739e00700 1 --2- 172.21.15.83:0/3583900902 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47340786f0 0x7f4734078b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:17.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:17.412+0000 7f4739e00700 1 -- 172.21.15.83:0/3583900902 >> 172.21.15.83:0/3583900902 conn(0x7f4734100340 msgr2=0x7f4734102780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:17.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:17.413+0000 7f4739e00700 1 -- 172.21.15.83:0/3583900902 shutdown_connections 2024-04-24T08:51:17.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:17.413+0000 7f4739e00700 1 -- 172.21.15.83:0/3583900902 wait complete. 2024-04-24T08:51:17.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:17.414+0000 7f4739e00700 1 Processor -- start 2024-04-24T08:51:17.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:17.414+0000 7f4739e00700 1 -- start start 2024-04-24T08:51:17.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:17.415+0000 7f4739e00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f47340774a0 0x7f473419f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:17.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:17.415+0000 7f4739e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47340786f0 0x7f473419f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:17.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:17.415+0000 7f4739e00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47340790b0 0x7f47341a67e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:17.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:17.415+0000 7f4739e00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4734114cd0 con 0x7f47340786f0 2024-04-24T08:51:17.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:17.415+0000 7f4739e00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4734114b50 con 0x7f47340790b0 2024-04-24T08:51:17.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:17.415+0000 7f4739e00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4734114e50 con 0x7f47340774a0 2024-04-24T08:51:17.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:17.415+0000 7f472a000700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47340786f0 0x7f473419f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:17.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:17.415+0000 7f472a000700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47340786f0 0x7f473419f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43354/0 (socket says 172.21.15.83:43354) 2024-04-24T08:51:17.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:17.415+0000 7f472a000700 1 -- 172.21.15.83:0/256022369 learned_addr learned my addr 172.21.15.83:0/256022369 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:51:17.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:17.415+0000 7f4732a00700 1 --2- 172.21.15.83:0/256022369 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f47340774a0 0x7f473419f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:17.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:17.415+0000 7f4733400700 1 --2- 172.21.15.83:0/256022369 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47340790b0 0x7f47341a67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:17.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:17.416+0000 7f472a000700 1 -- 172.21.15.83:0/256022369 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f47340774a0 msgr2=0x7f473419f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:17.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:17.416+0000 7f472a000700 1 --2- 172.21.15.83:0/256022369 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f47340774a0 0x7f473419f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:17.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:17.416+0000 7f472a000700 1 -- 172.21.15.83:0/256022369 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47340790b0 msgr2=0x7f47341a67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:17.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:17.416+0000 7f472a000700 1 --2- 172.21.15.83:0/256022369 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47340790b0 0x7f47341a67e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:17.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:17.416+0000 7f472a000700 1 -- 172.21.15.83:0/256022369 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47341a6e80 con 0x7f47340786f0 2024-04-24T08:51:17.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:17.416+0000 7f4732a00700 1 --2- 172.21.15.83:0/256022369 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f47340774a0 0x7f473419f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:51:17.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:17.416+0000 7f472a000700 1 --2- 172.21.15.83:0/256022369 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47340786f0 0x7f473419f760 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f471c00c950 tx=0x7f471c00cd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:17.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:17.416+0000 7f4738a00700 1 -- 172.21.15.83:0/256022369 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f471c007ac0 con 0x7f47340786f0 2024-04-24T08:51:17.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:17.416+0000 7f4738a00700 1 -- 172.21.15.83:0/256022369 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f471c004ce0 con 0x7f47340786f0 2024-04-24T08:51:17.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:17.416+0000 7f4738a00700 1 -- 172.21.15.83:0/256022369 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f471c0055e0 con 0x7f47340786f0 2024-04-24T08:51:17.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:17.416+0000 7f4739e00700 1 -- 172.21.15.83:0/256022369 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47341a7170 con 0x7f47340786f0 2024-04-24T08:51:17.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:17.416+0000 7f4739e00700 1 -- 172.21.15.83:0/256022369 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47341a7660 con 0x7f47340786f0 2024-04-24T08:51:17.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:17.417+0000 7f4739e00700 1 -- 172.21.15.83:0/256022369 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47340697e0 con 0x7f47340786f0 2024-04-24T08:51:17.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:17.420+0000 7f4738a00700 1 -- 172.21.15.83:0/256022369 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f471c004750 con 0x7f47340786f0 2024-04-24T08:51:17.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:17.421+0000 7f4738a00700 1 --2- 172.21.15.83:0/256022369 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f47180a3c40 0x7f47180a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:17.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:17.421+0000 7f4738a00700 1 -- 172.21.15.83:0/256022369 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9237+0+0 (secure 0 0 0) 0x7f471c0d0440 con 0x7f47340786f0 2024-04-24T08:51:17.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:17.421+0000 7f4732a00700 1 --2- 172.21.15.83:0/256022369 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f47180a3c40 0x7f47180a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:17.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:17.422+0000 7f4732a00700 1 --2- 172.21.15.83:0/256022369 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f47180a3c40 0x7f47180a6100 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4724005950 tx=0x7f47240058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:17.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:17.426+0000 7f4738a00700 1 -- 172.21.15.83:0/256022369 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f471c014030 con 0x7f47340786f0 2024-04-24T08:51:17.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:17.650+0000 7f4739e00700 1 -- 172.21.15.83:0/256022369 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f4734063700 con 0x7f47340786f0 2024-04-24T08:51:17.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:17.651+0000 7f4738a00700 1 -- 172.21.15.83:0/256022369 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v93) ==== 56+0+5729 (secure 0 0 0) 0x7f471c09ca30 con 0x7f47340786f0 2024-04-24T08:51:17.653 INFO:teuthology.orchestra.run.smithi083.stdout:epoch 93 2024-04-24T08:51:17.653 INFO:teuthology.orchestra.run.smithi083.stdout:fsid bd3e039e-0215-11ef-bc93-c7b262605968 2024-04-24T08:51:17.653 INFO:teuthology.orchestra.run.smithi083.stdout:created 2024-04-24T08:37:32.209944+0000 2024-04-24T08:51:17.653 INFO:teuthology.orchestra.run.smithi083.stdout:modified 2024-04-24T08:50:42.968900+0000 2024-04-24T08:51:17.653 INFO:teuthology.orchestra.run.smithi083.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-04-24T08:51:17.654 INFO:teuthology.orchestra.run.smithi083.stdout:crush_version 27 2024-04-24T08:51:17.654 INFO:teuthology.orchestra.run.smithi083.stdout:full_ratio 0.95 2024-04-24T08:51:17.654 INFO:teuthology.orchestra.run.smithi083.stdout:backfillfull_ratio 0.9 2024-04-24T08:51:17.654 INFO:teuthology.orchestra.run.smithi083.stdout:nearfull_ratio 0.85 2024-04-24T08:51:17.654 INFO:teuthology.orchestra.run.smithi083.stdout:require_min_compat_client luminous 2024-04-24T08:51:17.654 INFO:teuthology.orchestra.run.smithi083.stdout:min_compat_client luminous 2024-04-24T08:51:17.654 INFO:teuthology.orchestra.run.smithi083.stdout:require_osd_release squid 2024-04-24T08:51:17.654 INFO:teuthology.orchestra.run.smithi083.stdout:stretch_mode_enabled false 2024-04-24T08:51:17.654 INFO:teuthology.orchestra.run.smithi083.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-04-24T08:51:17.654 INFO:teuthology.orchestra.run.smithi083.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 84 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-04-24T08:51:17.654 INFO:teuthology.orchestra.run.smithi083.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-04-24T08:51:17.654 INFO:teuthology.orchestra.run.smithi083.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 89 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-04-24T08:51:17.655 INFO:teuthology.orchestra.run.smithi083.stdout:max_osd 12 2024-04-24T08:51:17.655 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0 up in weight 1 up_from 8 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.83:6802/2132675555,v1:172.21.15.83:6803/2132675555] [v2:172.21.15.83:6804/2132675555,v1:172.21.15.83:6805/2132675555] exists,up 2cd77f3a-19dd-4c6e-a199-1dcbd5faa679 2024-04-24T08:51:17.655 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1 up in weight 1 up_from 13 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.138:6800/3589158880,v1:172.21.15.138:6801/3589158880] [v2:172.21.15.138:6802/3589158880,v1:172.21.15.138:6803/3589158880] exists,up aba7f5df-2a4c-4c23-b502-d193605a5b1b 2024-04-24T08:51:17.655 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2 up in weight 1 up_from 18 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.189:6800/3169072688,v1:172.21.15.189:6801/3169072688] [v2:172.21.15.189:6802/3169072688,v1:172.21.15.189:6803/3169072688] exists,up 071bc167-2333-4420-bb58-bffbc10d7641 2024-04-24T08:51:17.655 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3 up in weight 1 up_from 25 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.83:6810/3131092450,v1:172.21.15.83:6811/3131092450] [v2:172.21.15.83:6812/3131092450,v1:172.21.15.83:6813/3131092450] exists,up f5624d78-4455-4e07-8c1a-f47c94cddb23 2024-04-24T08:51:17.655 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4 up in weight 1 up_from 33 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.138:6808/823775360,v1:172.21.15.138:6809/823775360] [v2:172.21.15.138:6810/823775360,v1:172.21.15.138:6811/823775360] exists,up 7b50cd14-9385-4535-a7b0-1c1da93c8548 2024-04-24T08:51:17.655 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5 up in weight 1 up_from 38 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.189:6808/1399817431,v1:172.21.15.189:6809/1399817431] [v2:172.21.15.189:6810/1399817431,v1:172.21.15.189:6811/1399817431] exists,up 9c647797-35bc-449f-be84-af97d2c69c2b 2024-04-24T08:51:17.655 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6 up in weight 1 up_from 46 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.83:6818/1611358483,v1:172.21.15.83:6819/1611358483] [v2:172.21.15.83:6820/1611358483,v1:172.21.15.83:6821/1611358483] exists,up 8f056584-16c8-4454-b445-3f780f26c196 2024-04-24T08:51:17.655 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7 up in weight 1 up_from 51 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.138:6816/3557856706,v1:172.21.15.138:6817/3557856706] [v2:172.21.15.138:6818/3557856706,v1:172.21.15.138:6819/3557856706] exists,up 7603efe3-da77-4b7f-9d03-29c4f293b6d0 2024-04-24T08:51:17.655 INFO:teuthology.orchestra.run.smithi083.stdout:osd.8 up in weight 1 up_from 59 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.189:6816/207018554,v1:172.21.15.189:6817/207018554] [v2:172.21.15.189:6818/207018554,v1:172.21.15.189:6819/207018554] exists,up 27d7d1b8-d6a0-417e-b8aa-596325aa1a77 2024-04-24T08:51:17.655 INFO:teuthology.orchestra.run.smithi083.stdout:osd.9 up in weight 1 up_from 67 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.83:6826/519636751,v1:172.21.15.83:6827/519636751] [v2:172.21.15.83:6828/519636751,v1:172.21.15.83:6829/519636751] exists,up baf19b98-881b-4d3a-9f72-8aa2dd73cad6 2024-04-24T08:51:17.655 INFO:teuthology.orchestra.run.smithi083.stdout:osd.10 up in weight 1 up_from 72 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.138:6824/1130872098,v1:172.21.15.138:6825/1130872098] [v2:172.21.15.138:6826/1130872098,v1:172.21.15.138:6827/1130872098] exists,up eb346717-5c07-47c8-95be-0e68fe449fdc 2024-04-24T08:51:17.655 INFO:teuthology.orchestra.run.smithi083.stdout:osd.11 up in weight 1 up_from 80 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.189:6824/2998782690,v1:172.21.15.189:6825/2998782690] [v2:172.21.15.189:6826/2998782690,v1:172.21.15.189:6827/2998782690] exists,up 8e37568e-981c-45b8-b2af-2f1b73898542 2024-04-24T08:51:17.655 INFO:teuthology.orchestra.run.smithi083.stdout:pg_upmap_items 2.0 [7,11] 2024-04-24T08:51:17.656 INFO:teuthology.orchestra.run.smithi083.stdout:pg_upmap_items 2.f [8,11] 2024-04-24T08:51:17.656 INFO:teuthology.orchestra.run.smithi083.stdout:pg_upmap_items 2.10 [2,4] 2024-04-24T08:51:17.656 INFO:teuthology.orchestra.run.smithi083.stdout:pg_upmap_items 2.1a [8,4] 2024-04-24T08:51:17.656 INFO:teuthology.orchestra.run.smithi083.stdout:pg_upmap_items 2.1d [2,1] 2024-04-24T08:51:17.656 INFO:teuthology.orchestra.run.smithi083.stdout:pg_upmap_items 3.c [3,9] 2024-04-24T08:51:17.656 INFO:teuthology.orchestra.run.smithi083.stdout:pg_upmap_items 3.e [5,9] 2024-04-24T08:51:17.656 INFO:teuthology.orchestra.run.smithi083.stdout:pg_upmap_items 3.13 [4,9] 2024-04-24T08:51:17.656 INFO:teuthology.orchestra.run.smithi083.stdout:pg_upmap_items 3.14 [4,6] 2024-04-24T08:51:17.656 INFO:teuthology.orchestra.run.smithi083.stdout:pg_upmap_items 3.21 [4,8] 2024-04-24T08:51:17.656 INFO:teuthology.orchestra.run.smithi083.stdout:pg_upmap_items 3.2a [3,8] 2024-04-24T08:51:17.656 INFO:teuthology.orchestra.run.smithi083.stdout:pg_upmap_items 4.2 [9,1] 2024-04-24T08:51:17.656 INFO:teuthology.orchestra.run.smithi083.stdout:pg_upmap_items 4.a [5,8] 2024-04-24T08:51:17.656 INFO:teuthology.orchestra.run.smithi083.stdout:pg_upmap_items 4.12 [9,10] 2024-04-24T08:51:17.656 INFO:teuthology.orchestra.run.smithi083.stdout:pg_upmap_items 4.14 [9,8] 2024-04-24T08:51:17.657 INFO:teuthology.orchestra.run.smithi083.stdout:blocklist 172.21.15.83:6801/820049883 expires 2024-04-25T08:38:40.661315+0000 2024-04-24T08:51:17.657 INFO:teuthology.orchestra.run.smithi083.stdout:blocklist 172.21.15.83:0/4272874138 expires 2024-04-25T08:38:40.661315+0000 2024-04-24T08:51:17.657 INFO:teuthology.orchestra.run.smithi083.stdout:blocklist 172.21.15.83:0/4212994335 expires 2024-04-25T08:38:40.661315+0000 2024-04-24T08:51:17.657 INFO:teuthology.orchestra.run.smithi083.stdout:blocklist 172.21.15.83:6801/1996371091 expires 2024-04-25T08:38:05.800856+0000 2024-04-24T08:51:17.657 INFO:teuthology.orchestra.run.smithi083.stdout:blocklist 172.21.15.83:0/1820663683 expires 2024-04-25T08:38:05.800856+0000 2024-04-24T08:51:17.657 INFO:teuthology.orchestra.run.smithi083.stdout:blocklist 172.21.15.83:0/3802897262 expires 2024-04-25T08:38:05.800856+0000 2024-04-24T08:51:17.657 INFO:teuthology.orchestra.run.smithi083.stdout:blocklist 172.21.15.83:6800/1996371091 expires 2024-04-25T08:38:05.800856+0000 2024-04-24T08:51:17.657 INFO:teuthology.orchestra.run.smithi083.stdout:blocklist 172.21.15.83:6800/820049883 expires 2024-04-25T08:38:40.661315+0000 2024-04-24T08:51:17.657 INFO:teuthology.orchestra.run.smithi083.stdout:blocklist 172.21.15.83:0/1366125049 expires 2024-04-25T08:38:05.800856+0000 2024-04-24T08:51:17.657 INFO:teuthology.orchestra.run.smithi083.stdout:blocklist 172.21.15.83:0/340452868 expires 2024-04-25T08:38:40.661315+0000 2024-04-24T08:51:17.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:17.656+0000 7f4739e00700 1 -- 172.21.15.83:0/256022369 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f47180a3c40 msgr2=0x7f47180a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:17.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:17.656+0000 7f4739e00700 1 --2- 172.21.15.83:0/256022369 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f47180a3c40 0x7f47180a6100 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4724005950 tx=0x7f47240058e0 comp rx=0 tx=0).stop 2024-04-24T08:51:17.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:17.656+0000 7f4739e00700 1 -- 172.21.15.83:0/256022369 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47340786f0 msgr2=0x7f473419f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:17.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:17.656+0000 7f4739e00700 1 --2- 172.21.15.83:0/256022369 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47340786f0 0x7f473419f760 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f471c00c950 tx=0x7f471c00cd10 comp rx=0 tx=0).stop 2024-04-24T08:51:17.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:17.656+0000 7f4739e00700 1 -- 172.21.15.83:0/256022369 shutdown_connections 2024-04-24T08:51:17.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:17.656+0000 7f4739e00700 1 --2- 172.21.15.83:0/256022369 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f47180a3c40 0x7f47180a6100 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:17.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:17.656+0000 7f4739e00700 1 --2- 172.21.15.83:0/256022369 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47340790b0 0x7f47341a67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:17.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:17.656+0000 7f4739e00700 1 --2- 172.21.15.83:0/256022369 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f47340774a0 0x7f473419f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:17.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:17.656+0000 7f4739e00700 1 --2- 172.21.15.83:0/256022369 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47340786f0 0x7f473419f760 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:17.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:17.656+0000 7f4739e00700 1 -- 172.21.15.83:0/256022369 >> 172.21.15.83:0/256022369 conn(0x7f4734100340 msgr2=0x7f4734101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:17.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:17.656+0000 7f4739e00700 1 -- 172.21.15.83:0/256022369 shutdown_connections 2024-04-24T08:51:17.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:17.656+0000 7f4739e00700 1 -- 172.21.15.83:0/256022369 wait complete. 2024-04-24T08:51:18.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:17 smithi083 ceph-mon[27907]: pgmap v461: 129 pgs: 129 active+clean; 644 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-24T08:51:18.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:17 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/256022369' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-24T08:51:18.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:17 smithi138 ceph-mon[31441]: pgmap v461: 129 pgs: 129 active+clean; 644 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-24T08:51:18.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:17 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/256022369' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-24T08:51:18.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:17 smithi189 ceph-mon[31791]: pgmap v461: 129 pgs: 129 active+clean; 644 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-24T08:51:18.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:17 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/256022369' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-24T08:51:18.680 INFO:teuthology.run_tasks:Running task fs.ready... 2024-04-24T08:51:18.692 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T08:51:18.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:18.905+0000 7fd053200640 1 -- 172.21.15.83:0/2301483675 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd054102ff0 msgr2=0x7fd054103470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:18.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:18.905+0000 7fd053200640 1 --2- 172.21.15.83:0/2301483675 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd054102ff0 0x7fd054103470 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fd040009ca0 tx=0x7fd04002a8b0 comp rx=0 tx=0).stop 2024-04-24T08:51:18.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:18.905+0000 7fd053200640 1 -- 172.21.15.83:0/2301483675 shutdown_connections 2024-04-24T08:51:18.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:18.905+0000 7fd053200640 1 --2- 172.21.15.83:0/2301483675 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd0541039b0 0x7fd05410a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:18.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:18.905+0000 7fd053200640 1 --2- 172.21.15.83:0/2301483675 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd054102ff0 0x7fd054103470 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:18.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:18.905+0000 7fd053200640 1 --2- 172.21.15.83:0/2301483675 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd054101ea0 0x7fd0541022c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:18.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:18.905+0000 7fd053200640 1 -- 172.21.15.83:0/2301483675 >> 172.21.15.83:0/2301483675 conn(0x7fd0540fd410 msgr2=0x7fd0540ff850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:18.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:18.905+0000 7fd053200640 1 -- 172.21.15.83:0/2301483675 shutdown_connections 2024-04-24T08:51:18.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:18.905+0000 7fd053200640 1 -- 172.21.15.83:0/2301483675 wait complete. 2024-04-24T08:51:18.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:18.906+0000 7fd053200640 1 Processor -- start 2024-04-24T08:51:18.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:18.906+0000 7fd053200640 1 -- start start 2024-04-24T08:51:18.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:18.907+0000 7fd053200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd054101ea0 0x7fd05419a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:18.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:18.907+0000 7fd053200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd054102ff0 0x7fd05419a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:18.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:18.907+0000 7fd053200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd0541039b0 0x7fd0541a1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:18.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:18.907+0000 7fd053200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd05410da30 con 0x7fd054101ea0 2024-04-24T08:51:18.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:18.907+0000 7fd053200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd05410d8b0 con 0x7fd0541039b0 2024-04-24T08:51:18.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:18.907+0000 7fd053200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd05410dbb0 con 0x7fd054102ff0 2024-04-24T08:51:18.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:18.907+0000 7fd051e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd054101ea0 0x7fd05419a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:18.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:18.907+0000 7fd051e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd054101ea0 0x7fd05419a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41090/0 (socket says 172.21.15.83:41090) 2024-04-24T08:51:18.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:18.907+0000 7fd051e00640 1 -- 172.21.15.83:0/1043133502 learned_addr learned my addr 172.21.15.83:0/1043133502 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:51:18.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:18.907+0000 7fd051400640 1 --2- 172.21.15.83:0/1043133502 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd054102ff0 0x7fd05419a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:18.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:18.907+0000 7fd052800640 1 --2- 172.21.15.83:0/1043133502 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd0541039b0 0x7fd0541a1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:18.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:18.907+0000 7fd051e00640 1 -- 172.21.15.83:0/1043133502 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd054102ff0 msgr2=0x7fd05419a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:18.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:18.908+0000 7fd051e00640 1 --2- 172.21.15.83:0/1043133502 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd054102ff0 0x7fd05419a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:18.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:18.908+0000 7fd051e00640 1 -- 172.21.15.83:0/1043133502 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd0541039b0 msgr2=0x7fd0541a1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:18.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:18.908+0000 7fd051e00640 1 --2- 172.21.15.83:0/1043133502 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd0541039b0 0x7fd0541a1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:18.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:18.908+0000 7fd051e00640 1 -- 172.21.15.83:0/1043133502 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0541a1df0 con 0x7fd054101ea0 2024-04-24T08:51:18.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:18.908+0000 7fd052800640 1 --2- 172.21.15.83:0/1043133502 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd0541039b0 0x7fd0541a1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:51:18.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:18.908+0000 7fd051e00640 1 --2- 172.21.15.83:0/1043133502 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd054101ea0 0x7fd05419a1d0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fd04400c920 tx=0x7fd04400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:18.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:18.908+0000 7fd04aa00640 1 -- 172.21.15.83:0/1043133502 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd044007c70 con 0x7fd054101ea0 2024-04-24T08:51:18.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:18.908+0000 7fd04aa00640 1 -- 172.21.15.83:0/1043133502 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd044007e10 con 0x7fd054101ea0 2024-04-24T08:51:18.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:18.908+0000 7fd04aa00640 1 -- 172.21.15.83:0/1043133502 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd044018820 con 0x7fd054101ea0 2024-04-24T08:51:18.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:18.908+0000 7fd053200640 1 -- 172.21.15.83:0/1043133502 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0541a20e0 con 0x7fd054101ea0 2024-04-24T08:51:18.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:18.908+0000 7fd053200640 1 -- 172.21.15.83:0/1043133502 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0541a25e0 con 0x7fd054101ea0 2024-04-24T08:51:18.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:18.910+0000 7fd053200640 1 -- 172.21.15.83:0/1043133502 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0541a28a0 con 0x7fd054101ea0 2024-04-24T08:51:18.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:18.911+0000 7fd04aa00640 1 -- 172.21.15.83:0/1043133502 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fd0440189c0 con 0x7fd054101ea0 2024-04-24T08:51:18.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:18.913+0000 7fd04aa00640 1 --2- 172.21.15.83:0/1043133502 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fd0380a3bb0 0x7fd0380a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:18.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:18.914+0000 7fd04aa00640 1 -- 172.21.15.83:0/1043133502 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9237+0+0 (secure 0 0 0) 0x7fd0440d0a30 con 0x7fd054101ea0 2024-04-24T08:51:18.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:18.914+0000 7fd051400640 1 --2- 172.21.15.83:0/1043133502 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fd0380a3bb0 0x7fd0380a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:18.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:18.914+0000 7fd051400640 1 --2- 172.21.15.83:0/1043133502 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fd0380a3bb0 0x7fd0380a6070 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd05419b6e0 tx=0x7fd040030880 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:18.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:18.918+0000 7fd04aa00640 1 -- 172.21.15.83:0/1043133502 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd044014030 con 0x7fd054101ea0 2024-04-24T08:51:19.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:19.124+0000 7fd053200640 1 -- 172.21.15.83:0/1043133502 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd0541a2a70 con 0x7fd054101ea0 2024-04-24T08:51:19.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:19.128+0000 7fd04aa00640 1 -- 172.21.15.83:0/1043133502 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+24887 (secure 0 0 0) 0x7fd04409cfc0 con 0x7fd054101ea0 2024-04-24T08:51:19.129 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:51:19.129 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":93,"fsid":"bd3e039e-0215-11ef-bc93-c7b262605968","created":"2024-04-24T08:37:32.209944+0000","modified":"2024-04-24T08:50:42.968900+0000","last_up_change":"2024-04-24T08:47:17.966041+0000","last_in_change":"2024-04-24T08:46:53.835936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T08:41:56.834372+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T08:48:26.978311+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T08:48:28.352731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T08:48:32.275167+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2cd77f3a-19dd-4c6e-a199-1dcbd5faa679","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2132675555},{"type":"v1","addr":"172.21.15.83:6803","nonce":2132675555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2132675555},{"type":"v1","addr":"172.21.15.83:6805","nonce":2132675555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2132675555},{"type":"v1","addr":"172.21.15.83:6809","nonce":2132675555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2132675555},{"type":"v1","addr":"172.21.15.83:6807","nonce":2132675555}]},"public_addr":"172.21.15.83:6803/2132675555","cluster_addr":"172.21.15.83:6805/2132675555","heartbeat_back_addr":"172.21.15.83:6809/2132675555","heartbeat_front_addr":"172.21.15.83:6807/2132675555","state":["exists","up"]},{"osd":1,"uuid":"aba7f5df-2a4c-4c23-b502-d193605a5b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":3589158880},{"type":"v1","addr":"172.21.15.138:6801","nonce":3589158880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":3589158880},{"type":"v1","addr":"172.21.15.138:6803","nonce":3589158880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":3589158880},{"type":"v1","addr":"172.21.15.138:6807","nonce":3589158880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":3589158880},{"type":"v1","addr":"172.21.15.138:6805","nonce":3589158880}]},"public_addr":"172.21.15.138:6801/3589158880","cluster_addr":"172.21.15.138:6803/3589158880","heartbeat_back_addr":"172.21.15.138:6807/3589158880","heartbeat_front_addr":"172.21.15.138:6805/3589158880","state":["exists","up"]},{"osd":2,"uuid":"071bc167-2333-4420-bb58-bffbc10d7641","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":3169072688},{"type":"v1","addr":"172.21.15.189:6801","nonce":3169072688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":3169072688},{"type":"v1","addr":"172.21.15.189:6803","nonce":3169072688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":3169072688},{"type":"v1","addr":"172.21.15.189:6807","nonce":3169072688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":3169072688},{"type":"v1","addr":"172.21.15.189:6805","nonce":3169072688}]},"public_addr":"172.21.15.189:6801/3169072688","cluster_addr":"172.21.15.189:6803/3169072688","heartbeat_back_addr":"172.21.15.189:6807/3169072688","heartbeat_front_addr":"172.21.15.189:6805/3169072688","state":["exists","up"]},{"osd":3,"uuid":"f5624d78-4455-4e07-8c1a-f47c94cddb23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3131092450},{"type":"v1","addr":"172.21.15.83:6811","nonce":3131092450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3131092450},{"type":"v1","addr":"172.21.15.83:6813","nonce":3131092450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3131092450},{"type":"v1","addr":"172.21.15.83:6817","nonce":3131092450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3131092450},{"type":"v1","addr":"172.21.15.83:6815","nonce":3131092450}]},"public_addr":"172.21.15.83:6811/3131092450","cluster_addr":"172.21.15.83:6813/3131092450","heartbeat_back_addr":"172.21.15.83:6817/3131092450","heartbeat_front_addr":"172.21.15.83:6815/3131092450","state":["exists","up"]},{"osd":4,"uuid":"7b50cd14-9385-4535-a7b0-1c1da93c8548","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":823775360},{"type":"v1","addr":"172.21.15.138:6809","nonce":823775360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":823775360},{"type":"v1","addr":"172.21.15.138:6811","nonce":823775360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":823775360},{"type":"v1","addr":"172.21.15.138:6815","nonce":823775360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":823775360},{"type":"v1","addr":"172.21.15.138:6813","nonce":823775360}]},"public_addr":"172.21.15.138:6809/823775360","cluster_addr":"172.21.15.138:6811/823775360","heartbeat_back_addr":"172.21.15.138:6815/823775360","heartbeat_front_addr":"172.21.15.138:6813/823775360","state":["exists","up"]},{"osd":5,"uuid":"9c647797-35bc-449f-be84-af97d2c69c2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":1399817431},{"type":"v1","addr":"172.21.15.189:6809","nonce":1399817431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":1399817431},{"type":"v1","addr":"172.21.15.189:6811","nonce":1399817431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":1399817431},{"type":"v1","addr":"172.21.15.189:6815","nonce":1399817431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":1399817431},{"type":"v1","addr":"172.21.15.189:6813","nonce":1399817431}]},"public_addr":"172.21.15.189:6809/1399817431","cluster_addr":"172.21.15.189:6811/1399817431","heartbeat_back_addr":"172.21.15.189:6815/1399817431","heartbeat_front_addr":"172.21.15.189:6813/1399817431","state":["exists","up"]},{"osd":6,"uuid":"8f056584-16c8-4454-b445-3f780f26c196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":1611358483},{"type":"v1","addr":"172.21.15.83:6819","nonce":1611358483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1611358483},{"type":"v1","addr":"172.21.15.83:6821","nonce":1611358483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1611358483},{"type":"v1","addr":"172.21.15.83:6825","nonce":1611358483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":1611358483},{"type":"v1","addr":"172.21.15.83:6823","nonce":1611358483}]},"public_addr":"172.21.15.83:6819/1611358483","cluster_addr":"172.21.15.83:6821/1611358483","heartbeat_back_addr":"172.21.15.83:6825/1611358483","heartbeat_front_addr":"172.21.15.83:6823/1611358483","state":["exists","up"]},{"osd":7,"uuid":"7603efe3-da77-4b7f-9d03-29c4f293b6d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":3557856706},{"type":"v1","addr":"172.21.15.138:6817","nonce":3557856706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":3557856706},{"type":"v1","addr":"172.21.15.138:6819","nonce":3557856706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":3557856706},{"type":"v1","addr":"172.21.15.138:6823","nonce":3557856706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":3557856706},{"type":"v1","addr":"172.21.15.138:6821","nonce":3557856706}]},"public_addr":"172.21.15.138:6817/3557856706","cluster_addr":"172.21.15.138:6819/3557856706","heartbeat_back_addr":"172.21.15.138:6823/3557856706","heartbeat_front_addr":"172.21.15.138:6821/3557856706","state":["exists","up"]},{"osd":8,"uuid":"27d7d1b8-d6a0-417e-b8aa-596325aa1a77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":207018554},{"type":"v1","addr":"172.21.15.189:6817","nonce":207018554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":207018554},{"type":"v1","addr":"172.21.15.189:6819","nonce":207018554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":207018554},{"type":"v1","addr":"172.21.15.189:6823","nonce":207018554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":207018554},{"type":"v1","addr":"172.21.15.189:6821","nonce":207018554}]},"public_addr":"172.21.15.189:6817/207018554","cluster_addr":"172.21.15.189:6819/207018554","heartbeat_back_addr":"172.21.15.189:6823/207018554","heartbeat_front_addr":"172.21.15.189:6821/207018554","state":["exists","up"]},{"osd":9,"uuid":"baf19b98-881b-4d3a-9f72-8aa2dd73cad6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":519636751},{"type":"v1","addr":"172.21.15.83:6827","nonce":519636751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":519636751},{"type":"v1","addr":"172.21.15.83:6829","nonce":519636751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":519636751},{"type":"v1","addr":"172.21.15.83:6833","nonce":519636751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":519636751},{"type":"v1","addr":"172.21.15.83:6831","nonce":519636751}]},"public_addr":"172.21.15.83:6827/519636751","cluster_addr":"172.21.15.83:6829/519636751","heartbeat_back_addr":"172.21.15.83:6833/519636751","heartbeat_front_addr":"172.21.15.83:6831/519636751","state":["exists","up"]},{"osd":10,"uuid":"eb346717-5c07-47c8-95be-0e68fe449fdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1130872098},{"type":"v1","addr":"172.21.15.138:6825","nonce":1130872098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1130872098},{"type":"v1","addr":"172.21.15.138:6827","nonce":1130872098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1130872098},{"type":"v1","addr":"172.21.15.138:6831","nonce":1130872098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1130872098},{"type":"v1","addr":"172.21.15.138:6829","nonce":1130872098}]},"public_addr":"172.21.15.138:6825/1130872098","cluster_addr":"172.21.15.138:6827/1130872098","heartbeat_back_addr":"172.21.15.138:6831/1130872098","heartbeat_front_addr":"172.21.15.138:6829/1130872098","state":["exists","up"]},{"osd":11,"uuid":"8e37568e-981c-45b8-b2af-2f1b73898542","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":2998782690},{"type":"v1","addr":"172.21.15.189:6825","nonce":2998782690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":2998782690},{"type":"v1","addr":"172.21.15.189:6827","nonce":2998782690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":2998782690},{"type":"v1","addr":"172.21.15.189:6831","nonce":2998782690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":2998782690},{"type":"v1","addr":"172.21.15.189:6829","nonce":2998782690}]},"public_addr":"172.21.15.189:6825/2998782690","cluster_addr":"172.21.15.189:6827/2998782690","heartbeat_back_addr":"172.21.15.189:6831/2998782690","heartbeat_front_addr":"172.21.15.189:6829/2998782690","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:40:47.980103+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:41:21.970795+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:41:55.160426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:42:30.400475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:43:05.705663+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:43:40.112312+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:44:16.781289+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:44:52.151187+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:45:28.055944+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:46:04.969301+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:46:40.918306+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:47:16.886874+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"2.1d","mappings":[{"from":2,"to":1}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.2a","mappings":[{"from":3,"to":8}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.a","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:6801/820049883":"2024-04-25T08:38:40.661315+0000","172.21.15.83:0/4272874138":"2024-04-25T08:38:40.661315+0000","172.21.15.83:0/4212994335":"2024-04-25T08:38:40.661315+0000","172.21.15.83:6801/1996371091":"2024-04-25T08:38:05.800856+0000","172.21.15.83:0/1820663683":"2024-04-25T08:38:05.800856+0000","172.21.15.83:0/3802897262":"2024-04-25T08:38:05.800856+0000","172.21.15.83:6800/1996371091":"2024-04-25T08:38:05.800856+0000","172.21.15.83:6800/820049883":"2024-04-25T08:38:40.661315+0000","172.21.15.83:0/1366125049":"2024-04-25T08:38:05.800856+0000","172.21.15.83:0/340452868":"2024-04-25T08:38:40.661315+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T08:51:19.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:19.132+0000 7fd053200640 1 -- 172.21.15.83:0/1043133502 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fd0380a3bb0 msgr2=0x7fd0380a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:19.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:19.132+0000 7fd053200640 1 --2- 172.21.15.83:0/1043133502 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fd0380a3bb0 0x7fd0380a6070 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd05419b6e0 tx=0x7fd040030880 comp rx=0 tx=0).stop 2024-04-24T08:51:19.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:19.132+0000 7fd053200640 1 -- 172.21.15.83:0/1043133502 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd054101ea0 msgr2=0x7fd05419a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:19.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:19.132+0000 7fd053200640 1 --2- 172.21.15.83:0/1043133502 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd054101ea0 0x7fd05419a1d0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fd04400c920 tx=0x7fd04400cde0 comp rx=0 tx=0).stop 2024-04-24T08:51:19.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:19.132+0000 7fd053200640 1 -- 172.21.15.83:0/1043133502 shutdown_connections 2024-04-24T08:51:19.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:19.132+0000 7fd053200640 1 --2- 172.21.15.83:0/1043133502 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd0541039b0 0x7fd0541a1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:19.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:19.132+0000 7fd053200640 1 --2- 172.21.15.83:0/1043133502 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd054102ff0 0x7fd05419a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:19.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:19.132+0000 7fd053200640 1 --2- 172.21.15.83:0/1043133502 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fd0380a3bb0 0x7fd0380a6070 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:19.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:19.132+0000 7fd053200640 1 --2- 172.21.15.83:0/1043133502 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd054101ea0 0x7fd05419a1d0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:19.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:19.132+0000 7fd053200640 1 -- 172.21.15.83:0/1043133502 >> 172.21.15.83:0/1043133502 conn(0x7fd0540fd410 msgr2=0x7fd054074230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:19.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:19.132+0000 7fd053200640 1 -- 172.21.15.83:0/1043133502 shutdown_connections 2024-04-24T08:51:19.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:19.132+0000 7fd053200640 1 -- 172.21.15.83:0/1043133502 wait complete. 2024-04-24T08:51:19.153 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T08:41:56.834372+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T08:48:26.978311+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T08:48:28.352731+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-24T08:48:32.275167+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-24T08:51:19.154 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T08:51:19.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:19.363+0000 7f6e6e400640 1 -- 172.21.15.83:0/2646480325 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e681010e0 msgr2=0x7f6e68101540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:19.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:19.363+0000 7f6e6e400640 1 --2- 172.21.15.83:0/2646480325 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e681010e0 0x7f6e68101540 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f6e54009900 tx=0x7f6e5402a3f0 comp rx=0 tx=0).stop 2024-04-24T08:51:19.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:19.363+0000 7f6e6e400640 1 -- 172.21.15.83:0/2646480325 shutdown_connections 2024-04-24T08:51:19.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:19.363+0000 7f6e6e400640 1 --2- 172.21.15.83:0/2646480325 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6e68101a80 0x7f6e68101f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:19.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:19.363+0000 7f6e6e400640 1 --2- 172.21.15.83:0/2646480325 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e681010e0 0x7f6e68101540 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:19.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:19.363+0000 7f6e6e400640 1 --2- 172.21.15.83:0/2646480325 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e68100780 0x7f6e68100ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:19.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:19.363+0000 7f6e6e400640 1 -- 172.21.15.83:0/2646480325 >> 172.21.15.83:0/2646480325 conn(0x7f6e680fc3a0 msgr2=0x7f6e680fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:19.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:19.363+0000 7f6e6e400640 1 -- 172.21.15.83:0/2646480325 shutdown_connections 2024-04-24T08:51:19.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:19.364+0000 7f6e6e400640 1 -- 172.21.15.83:0/2646480325 wait complete. 2024-04-24T08:51:19.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:19.364+0000 7f6e6e400640 1 Processor -- start 2024-04-24T08:51:19.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:19.365+0000 7f6e6e400640 1 -- start start 2024-04-24T08:51:19.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:19.365+0000 7f6e6e400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6e68100780 0x7f6e6819a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:19.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:19.365+0000 7f6e6e400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e681010e0 0x7f6e6819a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:19.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:19.365+0000 7f6e6e400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e68101a80 0x7f6e681a1820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:19.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:19.365+0000 7f6e6e400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6e6810fcb0 con 0x7f6e681010e0 2024-04-24T08:51:19.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:19.365+0000 7f6e6e400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6e6810fb30 con 0x7f6e68101a80 2024-04-24T08:51:19.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:19.365+0000 7f6e6e400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6e6810fe30 con 0x7f6e68100780 2024-04-24T08:51:19.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:19.365+0000 7f6e67e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e681010e0 0x7f6e6819a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:19.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:19.365+0000 7f6e67e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e681010e0 0x7f6e6819a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41112/0 (socket says 172.21.15.83:41112) 2024-04-24T08:51:19.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:19.365+0000 7f6e67e00640 1 -- 172.21.15.83:0/2883603913 learned_addr learned my addr 172.21.15.83:0/2883603913 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:51:19.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:19.366+0000 7f6e6d000640 1 --2- 172.21.15.83:0/2883603913 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6e68100780 0x7f6e6819a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:19.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:19.366+0000 7f6e6da00640 1 --2- 172.21.15.83:0/2883603913 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e68101a80 0x7f6e681a1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:19.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:19.366+0000 7f6e67e00640 1 -- 172.21.15.83:0/2883603913 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6e68100780 msgr2=0x7f6e6819a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:19.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:19.366+0000 7f6e67e00640 1 --2- 172.21.15.83:0/2883603913 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6e68100780 0x7f6e6819a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:19.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:19.366+0000 7f6e67e00640 1 -- 172.21.15.83:0/2883603913 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e68101a80 msgr2=0x7f6e681a1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:19.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:19.366+0000 7f6e67e00640 1 --2- 172.21.15.83:0/2883603913 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e68101a80 0x7f6e681a1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:19.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:19.366+0000 7f6e67e00640 1 -- 172.21.15.83:0/2883603913 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e681a1e90 con 0x7f6e681010e0 2024-04-24T08:51:19.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:19.366+0000 7f6e6d000640 1 --2- 172.21.15.83:0/2883603913 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6e68100780 0x7f6e6819a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:51:19.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:19.366+0000 7f6e67e00640 1 --2- 172.21.15.83:0/2883603913 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e681010e0 0x7f6e6819a800 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f6e540314e0 tx=0x7f6e54003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:19.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:19.367+0000 7f6e65600640 1 -- 172.21.15.83:0/2883603913 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e5403f070 con 0x7f6e681010e0 2024-04-24T08:51:19.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:19.367+0000 7f6e65600640 1 -- 172.21.15.83:0/2883603913 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6e54031e40 con 0x7f6e681010e0 2024-04-24T08:51:19.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:19.367+0000 7f6e65600640 1 -- 172.21.15.83:0/2883603913 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e54035030 con 0x7f6e681010e0 2024-04-24T08:51:19.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:19.367+0000 7f6e6e400640 1 -- 172.21.15.83:0/2883603913 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e681a2120 con 0x7f6e681010e0 2024-04-24T08:51:19.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:19.367+0000 7f6e6e400640 1 -- 172.21.15.83:0/2883603913 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e681a2610 con 0x7f6e681010e0 2024-04-24T08:51:19.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:19.368+0000 7f6e6e400640 1 -- 172.21.15.83:0/2883603913 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e6810b4d0 con 0x7f6e681010e0 2024-04-24T08:51:19.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:19.370+0000 7f6e65600640 1 -- 172.21.15.83:0/2883603913 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f6e54042070 con 0x7f6e681010e0 2024-04-24T08:51:19.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:19.374+0000 7f6e65600640 1 --2- 172.21.15.83:0/2883603913 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f6e500a3bb0 0x7f6e500a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:19.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:19.374+0000 7f6e65600640 1 -- 172.21.15.83:0/2883603913 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9237+0+0 (secure 0 0 0) 0x7f6e540f1230 con 0x7f6e681010e0 2024-04-24T08:51:19.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:19.374+0000 7f6e6d000640 1 --2- 172.21.15.83:0/2883603913 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f6e500a3bb0 0x7f6e500a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:19.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:19.375+0000 7f6e6d000640 1 --2- 172.21.15.83:0/2883603913 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f6e500a3bb0 0x7f6e500a6070 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c0058c0 tx=0x7f6e5c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:19.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:19.378+0000 7f6e65600640 1 -- 172.21.15.83:0/2883603913 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6e540bd7c0 con 0x7f6e681010e0 2024-04-24T08:51:19.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:19.580+0000 7f6e6e400640 1 -- 172.21.15.83:0/2883603913 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6e68062c00 con 0x7f6e681010e0 2024-04-24T08:51:19.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:19.581+0000 7f6e65600640 1 -- 172.21.15.83:0/2883603913 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) ==== 93+0+10 (secure 0 0 0) 0x7f6e540bd0c0 con 0x7f6e681010e0 2024-04-24T08:51:19.581 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-24T08:51:19.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:19.583+0000 7f6e6e400640 1 -- 172.21.15.83:0/2883603913 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f6e500a3bb0 msgr2=0x7f6e500a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:19.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:19.583+0000 7f6e6e400640 1 --2- 172.21.15.83:0/2883603913 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f6e500a3bb0 0x7f6e500a6070 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c0058c0 tx=0x7f6e5c005850 comp rx=0 tx=0).stop 2024-04-24T08:51:19.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:19.583+0000 7f6e6e400640 1 -- 172.21.15.83:0/2883603913 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e681010e0 msgr2=0x7f6e6819a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:19.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:19.583+0000 7f6e6e400640 1 --2- 172.21.15.83:0/2883603913 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e681010e0 0x7f6e6819a800 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f6e540314e0 tx=0x7f6e54003c80 comp rx=0 tx=0).stop 2024-04-24T08:51:19.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:19.583+0000 7f6e6e400640 1 -- 172.21.15.83:0/2883603913 shutdown_connections 2024-04-24T08:51:19.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:19.583+0000 7f6e6e400640 1 --2- 172.21.15.83:0/2883603913 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e68101a80 0x7f6e681a1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:19.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:19.583+0000 7f6e6e400640 1 --2- 172.21.15.83:0/2883603913 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f6e500a3bb0 0x7f6e500a6070 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:19.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:19.583+0000 7f6e6e400640 1 --2- 172.21.15.83:0/2883603913 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e681010e0 0x7f6e6819a800 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:19.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:19.583+0000 7f6e6e400640 1 --2- 172.21.15.83:0/2883603913 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6e68100780 0x7f6e6819a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:19.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:19.583+0000 7f6e6e400640 1 -- 172.21.15.83:0/2883603913 >> 172.21.15.83:0/2883603913 conn(0x7f6e680fc3a0 msgr2=0x7f6e680fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:19.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:19.583+0000 7f6e6e400640 1 -- 172.21.15.83:0/2883603913 shutdown_connections 2024-04-24T08:51:19.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:19.584+0000 7f6e6e400640 1 -- 172.21.15.83:0/2883603913 wait complete. 2024-04-24T08:51:19.603 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T08:51:19.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:19.810+0000 7fd292400640 1 -- 172.21.15.83:0/2955861113 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd28c0769c0 msgr2=0x7fd28c076de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:19.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:19.810+0000 7fd292400640 1 --2- 172.21.15.83:0/2955861113 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd28c0769c0 0x7fd28c076de0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fd27c009900 tx=0x7fd27c02a3f0 comp rx=0 tx=0).stop 2024-04-24T08:51:19.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:19.810+0000 7fd292400640 1 -- 172.21.15.83:0/2955861113 shutdown_connections 2024-04-24T08:51:19.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:19.810+0000 7fd292400640 1 --2- 172.21.15.83:0/2955861113 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd28c0785a0 0x7fd28c078e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:19.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:19.810+0000 7fd292400640 1 --2- 172.21.15.83:0/2955861113 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd28c077c00 0x7fd28c078060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:19.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:19.810+0000 7fd292400640 1 --2- 172.21.15.83:0/2955861113 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd28c0769c0 0x7fd28c076de0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:19.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:19.810+0000 7fd292400640 1 -- 172.21.15.83:0/2955861113 >> 172.21.15.83:0/2955861113 conn(0x7fd28c0ff7f0 msgr2=0x7fd28c101c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:19.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:19.810+0000 7fd292400640 1 -- 172.21.15.83:0/2955861113 shutdown_connections 2024-04-24T08:51:19.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:19.810+0000 7fd292400640 1 -- 172.21.15.83:0/2955861113 wait complete. 2024-04-24T08:51:19.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:19.811+0000 7fd292400640 1 Processor -- start 2024-04-24T08:51:19.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:19.811+0000 7fd292400640 1 -- start start 2024-04-24T08:51:19.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:19.812+0000 7fd292400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd28c0769c0 0x7fd28c19e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:19.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:19.812+0000 7fd292400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd28c077c00 0x7fd28c19ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:19.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:19.812+0000 7fd292400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd28c0785a0 0x7fd28c1a5e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:19.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:19.812+0000 7fd292400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd28c1141c0 con 0x7fd28c077c00 2024-04-24T08:51:19.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:19.812+0000 7fd292400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd28c114040 con 0x7fd28c0785a0 2024-04-24T08:51:19.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:19.812+0000 7fd292400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd28c114340 con 0x7fd28c0769c0 2024-04-24T08:51:19.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:19.812+0000 7fd282a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd28c077c00 0x7fd28c19ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:19.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:19.812+0000 7fd282a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd28c077c00 0x7fd28c19ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41122/0 (socket says 172.21.15.83:41122) 2024-04-24T08:51:19.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:19.812+0000 7fd282a00640 1 -- 172.21.15.83:0/1207850812 learned_addr learned my addr 172.21.15.83:0/1207850812 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:51:19.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:19.812+0000 7fd28b400640 1 --2- 172.21.15.83:0/1207850812 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd28c0769c0 0x7fd28c19e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:19.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:19.812+0000 7fd28be00640 1 --2- 172.21.15.83:0/1207850812 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd28c0785a0 0x7fd28c1a5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:19.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:19.812+0000 7fd282a00640 1 -- 172.21.15.83:0/1207850812 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd28c0769c0 msgr2=0x7fd28c19e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:19.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:19.812+0000 7fd282a00640 1 --2- 172.21.15.83:0/1207850812 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd28c0769c0 0x7fd28c19e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:19.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:19.813+0000 7fd282a00640 1 -- 172.21.15.83:0/1207850812 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd28c0785a0 msgr2=0x7fd28c1a5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:19.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:19.813+0000 7fd282a00640 1 --2- 172.21.15.83:0/1207850812 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd28c0785a0 0x7fd28c1a5e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:19.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:19.813+0000 7fd282a00640 1 -- 172.21.15.83:0/1207850812 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd28c1a64f0 con 0x7fd28c077c00 2024-04-24T08:51:19.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:19.813+0000 7fd28be00640 1 --2- 172.21.15.83:0/1207850812 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd28c0785a0 0x7fd28c1a5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:51:19.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:19.813+0000 7fd282a00640 1 --2- 172.21.15.83:0/1207850812 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd28c077c00 0x7fd28c19ee10 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fd27400d770 tx=0x7fd27400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:19.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:19.813+0000 7fd288c00640 1 -- 172.21.15.83:0/1207850812 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd274017070 con 0x7fd28c077c00 2024-04-24T08:51:19.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:19.813+0000 7fd288c00640 1 -- 172.21.15.83:0/1207850812 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd2740054d0 con 0x7fd28c077c00 2024-04-24T08:51:19.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:19.813+0000 7fd288c00640 1 -- 172.21.15.83:0/1207850812 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd274004cf0 con 0x7fd28c077c00 2024-04-24T08:51:19.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:19.813+0000 7fd292400640 1 -- 172.21.15.83:0/1207850812 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd28c1a67e0 con 0x7fd28c077c00 2024-04-24T08:51:19.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:19.813+0000 7fd292400640 1 -- 172.21.15.83:0/1207850812 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd28c1a6ce0 con 0x7fd28c077c00 2024-04-24T08:51:19.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:19.814+0000 7fd292400640 1 -- 172.21.15.83:0/1207850812 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd28c110bf0 con 0x7fd28c077c00 2024-04-24T08:51:19.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:19.816+0000 7fd288c00640 1 -- 172.21.15.83:0/1207850812 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fd274004e90 con 0x7fd28c077c00 2024-04-24T08:51:19.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:19.818+0000 7fd288c00640 1 --2- 172.21.15.83:0/1207850812 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fd2700a3bb0 0x7fd2700a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:19.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:19.818+0000 7fd288c00640 1 -- 172.21.15.83:0/1207850812 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9237+0+0 (secure 0 0 0) 0x7fd2740d0370 con 0x7fd28c077c00 2024-04-24T08:51:19.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:19.818+0000 7fd28b400640 1 --2- 172.21.15.83:0/1207850812 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fd2700a3bb0 0x7fd2700a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:19.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:19.819+0000 7fd28b400640 1 --2- 172.21.15.83:0/1207850812 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fd2700a3bb0 0x7fd2700a6070 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd27c000c00 tx=0x7fd27c0314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:19.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:19.822+0000 7fd288c00640 1 -- 172.21.15.83:0/1207850812 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd27409c900 con 0x7fd28c077c00 2024-04-24T08:51:20.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.027+0000 7fd292400640 1 -- 172.21.15.83:0/1207850812 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd28c062c00 con 0x7fd28c077c00 2024-04-24T08:51:20.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.028+0000 7fd288c00640 1 -- 172.21.15.83:0/1207850812 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) ==== 104+0+11 (secure 0 0 0) 0x7fd27409c200 con 0x7fd28c077c00 2024-04-24T08:51:20.028 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-24T08:51:20.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.030+0000 7fd292400640 1 -- 172.21.15.83:0/1207850812 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fd2700a3bb0 msgr2=0x7fd2700a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:20.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.030+0000 7fd292400640 1 --2- 172.21.15.83:0/1207850812 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fd2700a3bb0 0x7fd2700a6070 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd27c000c00 tx=0x7fd27c0314a0 comp rx=0 tx=0).stop 2024-04-24T08:51:20.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.030+0000 7fd292400640 1 -- 172.21.15.83:0/1207850812 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd28c077c00 msgr2=0x7fd28c19ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:20.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.030+0000 7fd292400640 1 --2- 172.21.15.83:0/1207850812 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd28c077c00 0x7fd28c19ee10 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fd27400d770 tx=0x7fd27400dc30 comp rx=0 tx=0).stop 2024-04-24T08:51:20.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.030+0000 7fd292400640 1 -- 172.21.15.83:0/1207850812 shutdown_connections 2024-04-24T08:51:20.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.030+0000 7fd292400640 1 --2- 172.21.15.83:0/1207850812 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd28c0785a0 0x7fd28c1a5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:20.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.030+0000 7fd292400640 1 --2- 172.21.15.83:0/1207850812 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fd2700a3bb0 0x7fd2700a6070 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:20.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.030+0000 7fd292400640 1 --2- 172.21.15.83:0/1207850812 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd28c077c00 0x7fd28c19ee10 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:20.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.030+0000 7fd292400640 1 --2- 172.21.15.83:0/1207850812 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd28c0769c0 0x7fd28c19e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:20.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.030+0000 7fd292400640 1 -- 172.21.15.83:0/1207850812 >> 172.21.15.83:0/1207850812 conn(0x7fd28c0ff7f0 msgr2=0x7fd28c102b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:20.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.030+0000 7fd292400640 1 -- 172.21.15.83:0/1207850812 shutdown_connections 2024-04-24T08:51:20.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.030+0000 7fd292400640 1 -- 172.21.15.83:0/1207850812 wait complete. 2024-04-24T08:51:20.051 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T08:51:20.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.254+0000 7fa759200640 1 -- 172.21.15.83:0/3525324085 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa75410a0f0 msgr2=0x7fa754110980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:20.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.254+0000 7fa759200640 1 --2- 172.21.15.83:0/3525324085 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa75410a0f0 0x7fa754110980 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fa74c009ca0 tx=0x7fa74c02a8c0 comp rx=0 tx=0).stop 2024-04-24T08:51:20.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.254+0000 7fa759200640 1 -- 172.21.15.83:0/3525324085 shutdown_connections 2024-04-24T08:51:20.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.254+0000 7fa759200640 1 --2- 172.21.15.83:0/3525324085 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa75410a0f0 0x7fa754110980 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:20.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.254+0000 7fa759200640 1 --2- 172.21.15.83:0/3525324085 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa754109730 0x7fa754109bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:20.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.254+0000 7fa759200640 1 --2- 172.21.15.83:0/3525324085 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa754108540 0x7fa754108960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:20.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.254+0000 7fa759200640 1 -- 172.21.15.83:0/3525324085 >> 172.21.15.83:0/3525324085 conn(0x7fa7540774d0 msgr2=0x7fa754079910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:20.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.254+0000 7fa759200640 1 -- 172.21.15.83:0/3525324085 shutdown_connections 2024-04-24T08:51:20.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.254+0000 7fa759200640 1 -- 172.21.15.83:0/3525324085 wait complete. 2024-04-24T08:51:20.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.255+0000 7fa759200640 1 Processor -- start 2024-04-24T08:51:20.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.255+0000 7fa759200640 1 -- start start 2024-04-24T08:51:20.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.255+0000 7fa759200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa754108540 0x7fa75419e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:20.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.255+0000 7fa759200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa754109730 0x7fa75419ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:20.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.255+0000 7fa759200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa75410a0f0 0x7fa7541a5c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:20.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.255+0000 7fa759200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa754114080 con 0x7fa75410a0f0 2024-04-24T08:51:20.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.255+0000 7fa759200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa754113f00 con 0x7fa754109730 2024-04-24T08:51:20.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.255+0000 7fa759200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa754114200 con 0x7fa754108540 2024-04-24T08:51:20.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.255+0000 7fa753e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa75410a0f0 0x7fa7541a5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:20.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.255+0000 7fa753e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa75410a0f0 0x7fa7541a5c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41134/0 (socket says 172.21.15.83:41134) 2024-04-24T08:51:20.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.255+0000 7fa753e00640 1 -- 172.21.15.83:0/3767235805 learned_addr learned my addr 172.21.15.83:0/3767235805 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:51:20.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.256+0000 7fa753400640 1 --2- 172.21.15.83:0/3767235805 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa754108540 0x7fa75419e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:20.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.256+0000 7fa753e00640 1 -- 172.21.15.83:0/3767235805 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa754108540 msgr2=0x7fa75419e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:20.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.256+0000 7fa753e00640 1 --2- 172.21.15.83:0/3767235805 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa754108540 0x7fa75419e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:20.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.256+0000 7fa74be00640 1 --2- 172.21.15.83:0/3767235805 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa754109730 0x7fa75419ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:20.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.256+0000 7fa753e00640 1 -- 172.21.15.83:0/3767235805 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa754109730 msgr2=0x7fa75419ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:20.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.256+0000 7fa753e00640 1 --2- 172.21.15.83:0/3767235805 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa754109730 0x7fa75419ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:20.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.256+0000 7fa753e00640 1 -- 172.21.15.83:0/3767235805 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7541a62f0 con 0x7fa75410a0f0 2024-04-24T08:51:20.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.256+0000 7fa753e00640 1 --2- 172.21.15.83:0/3767235805 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa75410a0f0 0x7fa7541a5c80 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fa74c009dc0 tx=0x7fa74c003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:20.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.257+0000 7fa750c00640 1 -- 172.21.15.83:0/3767235805 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa74c032ec0 con 0x7fa75410a0f0 2024-04-24T08:51:20.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.257+0000 7fa750c00640 1 -- 172.21.15.83:0/3767235805 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa74c0044f0 con 0x7fa75410a0f0 2024-04-24T08:51:20.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.257+0000 7fa759200640 1 -- 172.21.15.83:0/3767235805 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7541a6580 con 0x7fa75410a0f0 2024-04-24T08:51:20.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.257+0000 7fa750c00640 1 -- 172.21.15.83:0/3767235805 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa74c030d40 con 0x7fa75410a0f0 2024-04-24T08:51:20.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.257+0000 7fa759200640 1 -- 172.21.15.83:0/3767235805 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7541a6ad0 con 0x7fa75410a0f0 2024-04-24T08:51:20.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.258+0000 7fa759200640 1 -- 172.21.15.83:0/3767235805 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa75410f860 con 0x7fa75410a0f0 2024-04-24T08:51:20.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.260+0000 7fa750c00640 1 -- 172.21.15.83:0/3767235805 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fa74c004040 con 0x7fa75410a0f0 2024-04-24T08:51:20.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.261+0000 7fa750c00640 1 --2- 172.21.15.83:0/3767235805 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fa7380a3d50 0x7fa7380a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:20.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.261+0000 7fa750c00640 1 -- 172.21.15.83:0/3767235805 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9237+0+0 (secure 0 0 0) 0x7fa74c0f1f30 con 0x7fa75410a0f0 2024-04-24T08:51:20.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.262+0000 7fa753400640 1 --2- 172.21.15.83:0/3767235805 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fa7380a3d50 0x7fa7380a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:20.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.262+0000 7fa753400640 1 --2- 172.21.15.83:0/3767235805 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fa7380a3d50 0x7fa7380a6210 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fa744005fd0 tx=0x7fa744005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:20.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.265+0000 7fa750c00640 1 -- 172.21.15.83:0/3767235805 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa74c0be4c0 con 0x7fa75410a0f0 2024-04-24T08:51:20.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:19 smithi083 ceph-mon[27907]: pgmap v462: 129 pgs: 129 active+clean; 644 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-24T08:51:20.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:19 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/1043133502' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T08:51:20.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:19 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/2883603913' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T08:51:20.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:19 smithi138 ceph-mon[31441]: pgmap v462: 129 pgs: 129 active+clean; 644 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-24T08:51:20.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:19 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/1043133502' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T08:51:20.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:19 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/2883603913' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T08:51:20.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:19 smithi189 ceph-mon[31791]: pgmap v462: 129 pgs: 129 active+clean; 644 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-24T08:51:20.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:19 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/1043133502' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T08:51:20.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:19 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/2883603913' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T08:51:20.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.469+0000 7fa759200640 1 -- 172.21.15.83:0/3767235805 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa754062c00 con 0x7fa75410a0f0 2024-04-24T08:51:20.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.470+0000 7fa750c00640 1 -- 172.21.15.83:0/3767235805 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) ==== 100+0+11 (secure 0 0 0) 0x7fa74c0bddc0 con 0x7fa75410a0f0 2024-04-24T08:51:20.470 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2024-04-24T08:51:20.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.472+0000 7fa759200640 1 -- 172.21.15.83:0/3767235805 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fa7380a3d50 msgr2=0x7fa7380a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:20.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.472+0000 7fa759200640 1 --2- 172.21.15.83:0/3767235805 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fa7380a3d50 0x7fa7380a6210 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fa744005fd0 tx=0x7fa744005850 comp rx=0 tx=0).stop 2024-04-24T08:51:20.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.472+0000 7fa759200640 1 -- 172.21.15.83:0/3767235805 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa75410a0f0 msgr2=0x7fa7541a5c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:20.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.472+0000 7fa759200640 1 --2- 172.21.15.83:0/3767235805 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa75410a0f0 0x7fa7541a5c80 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fa74c009dc0 tx=0x7fa74c003ba0 comp rx=0 tx=0).stop 2024-04-24T08:51:20.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.472+0000 7fa759200640 1 -- 172.21.15.83:0/3767235805 shutdown_connections 2024-04-24T08:51:20.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.472+0000 7fa759200640 1 --2- 172.21.15.83:0/3767235805 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fa7380a3d50 0x7fa7380a6210 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:20.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.472+0000 7fa759200640 1 --2- 172.21.15.83:0/3767235805 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa75410a0f0 0x7fa7541a5c80 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:20.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.472+0000 7fa759200640 1 --2- 172.21.15.83:0/3767235805 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa754109730 0x7fa75419ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:20.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.472+0000 7fa759200640 1 --2- 172.21.15.83:0/3767235805 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa754108540 0x7fa75419e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:20.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.472+0000 7fa759200640 1 -- 172.21.15.83:0/3767235805 >> 172.21.15.83:0/3767235805 conn(0x7fa7540774d0 msgr2=0x7fa75410ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:20.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.473+0000 7fa759200640 1 -- 172.21.15.83:0/3767235805 shutdown_connections 2024-04-24T08:51:20.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.473+0000 7fa759200640 1 -- 172.21.15.83:0/3767235805 wait complete. 2024-04-24T08:51:20.491 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-24T08:51:20.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.693+0000 7f161a000640 1 -- 172.21.15.83:0/3129782145 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1614105450 msgr2=0x7f16141058d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:20.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.693+0000 7f161a000640 1 --2- 172.21.15.83:0/3129782145 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1614105450 0x7f16141058d0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f15fc009900 tx=0x7f15fc02a410 comp rx=0 tx=0).stop 2024-04-24T08:51:20.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.693+0000 7f161a000640 1 -- 172.21.15.83:0/3129782145 shutdown_connections 2024-04-24T08:51:20.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.693+0000 7f161a000640 1 --2- 172.21.15.83:0/3129782145 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1614105e10 0x7f161410c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:20.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.693+0000 7f161a000640 1 --2- 172.21.15.83:0/3129782145 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1614105450 0x7f16141058d0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:20.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.693+0000 7f161a000640 1 --2- 172.21.15.83:0/3129782145 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1614104260 0x7f1614104680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:20.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.693+0000 7f161a000640 1 -- 172.21.15.83:0/3129782145 >> 172.21.15.83:0/3129782145 conn(0x7f16140ff7f0 msgr2=0x7f1614101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:20.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.693+0000 7f161a000640 1 -- 172.21.15.83:0/3129782145 shutdown_connections 2024-04-24T08:51:20.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.693+0000 7f161a000640 1 -- 172.21.15.83:0/3129782145 wait complete. 2024-04-24T08:51:20.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.694+0000 7f161a000640 1 Processor -- start 2024-04-24T08:51:20.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.694+0000 7f161a000640 1 -- start start 2024-04-24T08:51:20.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.694+0000 7f161a000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1614104260 0x7f161419a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:20.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.694+0000 7f161a000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1614105450 0x7f161419a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:20.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.695+0000 7f161a000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1614105e10 0x7f16141a1a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:20.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.695+0000 7f161a000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f161410fda0 con 0x7f1614105450 2024-04-24T08:51:20.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.695+0000 7f161a000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f161410fc20 con 0x7f1614104260 2024-04-24T08:51:20.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.695+0000 7f161a000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f161410ff20 con 0x7f1614105e10 2024-04-24T08:51:20.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.695+0000 7f160be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1614105450 0x7f161419a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:20.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.695+0000 7f160be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1614105450 0x7f161419a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41162/0 (socket says 172.21.15.83:41162) 2024-04-24T08:51:20.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.695+0000 7f160be00640 1 -- 172.21.15.83:0/1110600921 learned_addr learned my addr 172.21.15.83:0/1110600921 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:51:20.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.695+0000 7f1613400640 1 --2- 172.21.15.83:0/1110600921 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1614104260 0x7f161419a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:20.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.695+0000 7f1613e00640 1 --2- 172.21.15.83:0/1110600921 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1614105e10 0x7f16141a1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:20.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.695+0000 7f160be00640 1 -- 172.21.15.83:0/1110600921 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1614105e10 msgr2=0x7f16141a1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:20.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.695+0000 7f160be00640 1 --2- 172.21.15.83:0/1110600921 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1614105e10 0x7f16141a1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:20.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.695+0000 7f160be00640 1 -- 172.21.15.83:0/1110600921 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1614104260 msgr2=0x7f161419a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:20.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.695+0000 7f160be00640 1 --2- 172.21.15.83:0/1110600921 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1614104260 0x7f161419a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:20.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.695+0000 7f160be00640 1 -- 172.21.15.83:0/1110600921 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16141a2080 con 0x7f1614105450 2024-04-24T08:51:20.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.695+0000 7f1613400640 1 --2- 172.21.15.83:0/1110600921 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1614104260 0x7f161419a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:51:20.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.696+0000 7f160be00640 1 --2- 172.21.15.83:0/1110600921 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1614105450 0x7f161419a9a0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f15fc004770 tx=0x7f15fc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:20.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.696+0000 7f1610c00640 1 -- 172.21.15.83:0/1110600921 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15fc03f030 con 0x7f1614105450 2024-04-24T08:51:20.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.696+0000 7f1610c00640 1 -- 172.21.15.83:0/1110600921 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f15fc003cb0 con 0x7f1614105450 2024-04-24T08:51:20.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.696+0000 7f161a000640 1 -- 172.21.15.83:0/1110600921 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16141a2310 con 0x7f1614105450 2024-04-24T08:51:20.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.696+0000 7f1610c00640 1 -- 172.21.15.83:0/1110600921 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15fc035070 con 0x7f1614105450 2024-04-24T08:51:20.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.696+0000 7f161a000640 1 -- 172.21.15.83:0/1110600921 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16141a2800 con 0x7f1614105450 2024-04-24T08:51:20.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.697+0000 7f161a000640 1 -- 172.21.15.83:0/1110600921 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f161410b600 con 0x7f1614105450 2024-04-24T08:51:20.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.699+0000 7f1610c00640 1 -- 172.21.15.83:0/1110600921 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f15fc042030 con 0x7f1614105450 2024-04-24T08:51:20.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.701+0000 7f1610c00640 1 --2- 172.21.15.83:0/1110600921 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f15f80a3bb0 0x7f15f80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:20.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.701+0000 7f1610c00640 1 -- 172.21.15.83:0/1110600921 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9237+0+0 (secure 0 0 0) 0x7f15fc0f1230 con 0x7f1614105450 2024-04-24T08:51:20.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.701+0000 7f1613400640 1 --2- 172.21.15.83:0/1110600921 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f15f80a3bb0 0x7f15f80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:20.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.702+0000 7f1613400640 1 --2- 172.21.15.83:0/1110600921 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f15f80a3bb0 0x7f15f80a6070 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f16141052b0 tx=0x7f1604005e70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:20.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.705+0000 7f1610c00640 1 -- 172.21.15.83:0/1110600921 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f15fc0bd820 con 0x7f1614105450 2024-04-24T08:51:20.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.909+0000 7f161a000640 1 -- 172.21.15.83:0/1110600921 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f1614062c00 con 0x7f1614105450 2024-04-24T08:51:20.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.909+0000 7f1610c00640 1 -- 172.21.15.83:0/1110600921 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) ==== 103+0+11 (secure 0 0 0) 0x7f15fc0bd120 con 0x7f1614105450 2024-04-24T08:51:20.910 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-24T08:51:20.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.912+0000 7f161a000640 1 -- 172.21.15.83:0/1110600921 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f15f80a3bb0 msgr2=0x7f15f80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:20.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.912+0000 7f161a000640 1 --2- 172.21.15.83:0/1110600921 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f15f80a3bb0 0x7f15f80a6070 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f16141052b0 tx=0x7f1604005e70 comp rx=0 tx=0).stop 2024-04-24T08:51:20.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.912+0000 7f161a000640 1 -- 172.21.15.83:0/1110600921 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1614105450 msgr2=0x7f161419a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:20.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.912+0000 7f161a000640 1 --2- 172.21.15.83:0/1110600921 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1614105450 0x7f161419a9a0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f15fc004770 tx=0x7f15fc0042f0 comp rx=0 tx=0).stop 2024-04-24T08:51:20.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.912+0000 7f161a000640 1 -- 172.21.15.83:0/1110600921 shutdown_connections 2024-04-24T08:51:20.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.912+0000 7f161a000640 1 --2- 172.21.15.83:0/1110600921 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1614105e10 0x7f16141a1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:20.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.912+0000 7f161a000640 1 --2- 172.21.15.83:0/1110600921 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f15f80a3bb0 0x7f15f80a6070 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:20.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.912+0000 7f161a000640 1 --2- 172.21.15.83:0/1110600921 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1614105450 0x7f161419a9a0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:20.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.912+0000 7f161a000640 1 --2- 172.21.15.83:0/1110600921 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1614104260 0x7f161419a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:20.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.912+0000 7f161a000640 1 -- 172.21.15.83:0/1110600921 >> 172.21.15.83:0/1110600921 conn(0x7f16140ff7f0 msgr2=0x7f16141087d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:20.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.913+0000 7f161a000640 1 -- 172.21.15.83:0/1110600921 shutdown_connections 2024-04-24T08:51:20.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:20.913+0000 7f161a000640 1 -- 172.21.15.83:0/1110600921 wait complete. 2024-04-24T08:51:20.932 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T08:51:21.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:21.127+0000 7f2d8dc00640 1 -- 172.21.15.83:0/4094030673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d88105e10 msgr2=0x7f2d8810c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:21.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:21.127+0000 7f2d8dc00640 1 --2- 172.21.15.83:0/4094030673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d88105e10 0x7f2d8810c6a0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f2d80009ca0 tx=0x7f2d8002a8c0 comp rx=0 tx=0).stop 2024-04-24T08:51:21.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:21.127+0000 7f2d8dc00640 1 -- 172.21.15.83:0/4094030673 shutdown_connections 2024-04-24T08:51:21.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:21.127+0000 7f2d8dc00640 1 --2- 172.21.15.83:0/4094030673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d88105e10 0x7f2d8810c6a0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:21.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:21.128+0000 7f2d8dc00640 1 --2- 172.21.15.83:0/4094030673 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2d88105450 0x7f2d881058d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:21.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:21.128+0000 7f2d8dc00640 1 --2- 172.21.15.83:0/4094030673 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d88104260 0x7f2d88104680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:21.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:21.128+0000 7f2d8dc00640 1 -- 172.21.15.83:0/4094030673 >> 172.21.15.83:0/4094030673 conn(0x7f2d880ff7f0 msgr2=0x7f2d88101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:21.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:21.128+0000 7f2d8dc00640 1 -- 172.21.15.83:0/4094030673 shutdown_connections 2024-04-24T08:51:21.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:21.128+0000 7f2d8dc00640 1 -- 172.21.15.83:0/4094030673 wait complete. 2024-04-24T08:51:21.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:21.129+0000 7f2d8dc00640 1 Processor -- start 2024-04-24T08:51:21.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:21.129+0000 7f2d8dc00640 1 -- start start 2024-04-24T08:51:21.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:21.129+0000 7f2d8dc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d88104260 0x7f2d8819a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:21.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:21.129+0000 7f2d8dc00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2d88105450 0x7f2d8819a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:21.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:21.129+0000 7f2d8dc00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d88105e10 0x7f2d881a1a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:21.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:21.129+0000 7f2d8dc00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2d8810fdf0 con 0x7f2d88104260 2024-04-24T08:51:21.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:21.129+0000 7f2d8dc00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2d8810fc70 con 0x7f2d88105e10 2024-04-24T08:51:21.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:21.129+0000 7f2d8dc00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2d8810ff70 con 0x7f2d88105450 2024-04-24T08:51:21.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:21.129+0000 7f2d86a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d88104260 0x7f2d8819a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:21.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:21.129+0000 7f2d86a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d88104260 0x7f2d8819a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41180/0 (socket says 172.21.15.83:41180) 2024-04-24T08:51:21.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:21.129+0000 7f2d86a00640 1 -- 172.21.15.83:0/1166911209 learned_addr learned my addr 172.21.15.83:0/1166911209 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:51:21.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:21.130+0000 7f2d87400640 1 --2- 172.21.15.83:0/1166911209 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d88105e10 0x7f2d881a1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:21.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:21.130+0000 7f2d86000640 1 --2- 172.21.15.83:0/1166911209 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2d88105450 0x7f2d8819a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:21.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:21.130+0000 7f2d86a00640 1 -- 172.21.15.83:0/1166911209 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2d88105450 msgr2=0x7f2d8819a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:21.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:21.130+0000 7f2d86a00640 1 --2- 172.21.15.83:0/1166911209 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2d88105450 0x7f2d8819a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:21.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:21.130+0000 7f2d86a00640 1 -- 172.21.15.83:0/1166911209 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d88105e10 msgr2=0x7f2d881a1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:21.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:21.130+0000 7f2d86a00640 1 --2- 172.21.15.83:0/1166911209 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d88105e10 0x7f2d881a1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:21.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:21.130+0000 7f2d86a00640 1 -- 172.21.15.83:0/1166911209 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d881a2080 con 0x7f2d88104260 2024-04-24T08:51:21.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:21.130+0000 7f2d86000640 1 --2- 172.21.15.83:0/1166911209 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2d88105450 0x7f2d8819a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:51:21.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:21.130+0000 7f2d86a00640 1 --2- 172.21.15.83:0/1166911209 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d88104260 0x7f2d8819a460 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f2d7800d950 tx=0x7f2d7800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:21.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:21.130+0000 7f2d7f400640 1 -- 172.21.15.83:0/1166911209 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d78017070 con 0x7f2d88104260 2024-04-24T08:51:21.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:21.130+0000 7f2d7f400640 1 -- 172.21.15.83:0/1166911209 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2d78009c30 con 0x7f2d88104260 2024-04-24T08:51:21.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:21.130+0000 7f2d7f400640 1 -- 172.21.15.83:0/1166911209 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d7800f620 con 0x7f2d88104260 2024-04-24T08:51:21.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:21.130+0000 7f2d8dc00640 1 -- 172.21.15.83:0/1166911209 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d881a2370 con 0x7f2d88104260 2024-04-24T08:51:21.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:21.131+0000 7f2d8dc00640 1 -- 172.21.15.83:0/1166911209 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d881a2870 con 0x7f2d88104260 2024-04-24T08:51:21.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:21.132+0000 7f2d8dc00640 1 -- 172.21.15.83:0/1166911209 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d8810b580 con 0x7f2d88104260 2024-04-24T08:51:21.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:21.134+0000 7f2d7f400640 1 -- 172.21.15.83:0/1166911209 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f2d7801a070 con 0x7f2d88104260 2024-04-24T08:51:21.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:21.136+0000 7f2d7f400640 1 --2- 172.21.15.83:0/1166911209 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f2d6c0a3bb0 0x7f2d6c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:21.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:21.136+0000 7f2d7f400640 1 -- 172.21.15.83:0/1166911209 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9237+0+0 (secure 0 0 0) 0x7f2d780d0310 con 0x7f2d88104260 2024-04-24T08:51:21.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:21.136+0000 7f2d86000640 1 --2- 172.21.15.83:0/1166911209 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f2d6c0a3bb0 0x7f2d6c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:21.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:21.137+0000 7f2d86000640 1 --2- 172.21.15.83:0/1166911209 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f2d6c0a3bb0 0x7f2d6c0a6070 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f2d8819b970 tx=0x7f2d74005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:21.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:21.141+0000 7f2d7f400640 1 -- 172.21.15.83:0/1166911209 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2d78009de0 con 0x7f2d88104260 2024-04-24T08:51:21.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:20 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/1207850812' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T08:51:21.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:20 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/3767235805' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T08:51:21.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:20 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/1110600921' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T08:51:21.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:21.390+0000 7f2d8dc00640 1 -- 172.21.15.83:0/1166911209 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2d8802d620 con 0x7f2d88104260 2024-04-24T08:51:21.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:21.392+0000 7f2d7f400640 1 -- 172.21.15.83:0/1166911209 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11944 (secure 0 0 0) 0x7f2d7809c900 con 0x7f2d88104260 2024-04-24T08:51:21.393 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:51:21.393 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/358872257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":358872257},{"type":"v1","addr":"172.21.15.83:6835","nonce":358872257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3157070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3157070851},{"type":"v1","addr":"172.21.15.83:6837","nonce":3157070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T08:48:29.422576+0000","modified":"2024-04-24T08:49:04.217098+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34520,"mds_1":34493,"mds_2":34451,"mds_3":34427,"mds_4":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14658":{"gid":14658,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6839/4085470844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":4085470844},{"type":"v1","addr":"172.21.15.83:6839","nonce":4085470844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14676":{"gid":14676,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6841/2634754210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":2634754210},{"type":"v1","addr":"172.21.15.83:6841","nonce":2634754210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24382":{"gid":24382,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/1859921310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1859921310},{"type":"v1","addr":"172.21.15.189:6833","nonce":1859921310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/2241923197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2241923197},{"type":"v1","addr":"172.21.15.189:6835","nonce":2241923197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6837/3172311220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":3172311220},{"type":"v1","addr":"172.21.15.189:6837","nonce":3172311220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.189:6839/2548965336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":2548965336},{"type":"v1","addr":"172.21.15.189:6839","nonce":2548965336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.138:6833/1157967658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1157967658},{"type":"v1","addr":"172.21.15.138:6833","nonce":1157967658}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.138:6835/1909219758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1909219758},{"type":"v1","addr":"172.21.15.138:6835","nonce":1909219758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34493":{"gid":34493,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.138:6837/900566754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":900566754},{"type":"v1","addr":"172.21.15.138:6837","nonce":900566754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34520":{"gid":34520,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.138:6839/2236932393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2236932393},{"type":"v1","addr":"172.21.15.138:6839","nonce":2236932393}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34520,"qdb_cluster":[34427,24454,34520,34493,34451]},"id":1}]} 2024-04-24T08:51:21.393 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 27 2024-04-24T08:51:21.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:21.396+0000 7f2d8dc00640 1 -- 172.21.15.83:0/1166911209 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f2d6c0a3bb0 msgr2=0x7f2d6c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:21.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:21.396+0000 7f2d8dc00640 1 --2- 172.21.15.83:0/1166911209 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f2d6c0a3bb0 0x7f2d6c0a6070 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f2d8819b970 tx=0x7f2d74005e50 comp rx=0 tx=0).stop 2024-04-24T08:51:21.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:21.396+0000 7f2d8dc00640 1 -- 172.21.15.83:0/1166911209 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d88104260 msgr2=0x7f2d8819a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:21.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:21.396+0000 7f2d8dc00640 1 --2- 172.21.15.83:0/1166911209 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d88104260 0x7f2d8819a460 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f2d7800d950 tx=0x7f2d7800de10 comp rx=0 tx=0).stop 2024-04-24T08:51:21.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:21.397+0000 7f2d8dc00640 1 -- 172.21.15.83:0/1166911209 shutdown_connections 2024-04-24T08:51:21.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:21.397+0000 7f2d8dc00640 1 --2- 172.21.15.83:0/1166911209 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d88105e10 0x7f2d881a1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:21.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:21.397+0000 7f2d8dc00640 1 --2- 172.21.15.83:0/1166911209 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2d88105450 0x7f2d8819a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:21.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:21.397+0000 7f2d8dc00640 1 --2- 172.21.15.83:0/1166911209 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f2d6c0a3bb0 0x7f2d6c0a6070 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:21.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:21.397+0000 7f2d8dc00640 1 --2- 172.21.15.83:0/1166911209 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d88104260 0x7f2d8819a460 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:21.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:21.397+0000 7f2d8dc00640 1 -- 172.21.15.83:0/1166911209 >> 172.21.15.83:0/1166911209 conn(0x7f2d880ff7f0 msgr2=0x7f2d88108750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:21.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:21.397+0000 7f2d8dc00640 1 -- 172.21.15.83:0/1166911209 shutdown_connections 2024-04-24T08:51:21.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:21.397+0000 7f2d8dc00640 1 -- 172.21.15.83:0/1166911209 wait complete. 2024-04-24T08:51:21.420 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T08:51:21.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:20 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/1207850812' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T08:51:21.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:20 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/3767235805' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T08:51:21.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:20 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/1110600921' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T08:51:21.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:20 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/1207850812' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T08:51:21.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:20 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/3767235805' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T08:51:21.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:20 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/1110600921' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T08:51:21.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:21.616+0000 7f51e5600640 1 -- 172.21.15.83:0/2367271405 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51e01040a0 msgr2=0x7f51e01044c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:21.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:21.616+0000 7f51e5600640 1 --2- 172.21.15.83:0/2367271405 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51e01040a0 0x7f51e01044c0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f51d0009900 tx=0x7f51d002a3f0 comp rx=0 tx=0).stop 2024-04-24T08:51:21.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:21.616+0000 7f51e5600640 1 -- 172.21.15.83:0/2367271405 shutdown_connections 2024-04-24T08:51:21.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:21.616+0000 7f51e5600640 1 --2- 172.21.15.83:0/2367271405 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f51e0105c50 0x7f51e010c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:21.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:21.616+0000 7f51e5600640 1 --2- 172.21.15.83:0/2367271405 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f51e0105290 0x7f51e0105710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:21.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:21.616+0000 7f51e5600640 1 --2- 172.21.15.83:0/2367271405 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51e01040a0 0x7f51e01044c0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:21.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:21.616+0000 7f51e5600640 1 -- 172.21.15.83:0/2367271405 >> 172.21.15.83:0/2367271405 conn(0x7f51e00ff610 msgr2=0x7f51e0101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:21.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:21.616+0000 7f51e5600640 1 -- 172.21.15.83:0/2367271405 shutdown_connections 2024-04-24T08:51:21.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:21.616+0000 7f51e5600640 1 -- 172.21.15.83:0/2367271405 wait complete. 2024-04-24T08:51:21.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:21.617+0000 7f51e5600640 1 Processor -- start 2024-04-24T08:51:21.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:21.617+0000 7f51e5600640 1 -- start start 2024-04-24T08:51:21.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:21.617+0000 7f51e5600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f51e0105290 0x7f51e019a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:21.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:21.617+0000 7f51e5600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f51e0105c50 0x7f51e019a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:21.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:21.618+0000 7f51e5600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51e019fa50 0x7f51e01a1e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:21.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:21.618+0000 7f51e5600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f51e010fbf0 con 0x7f51e019fa50 2024-04-24T08:51:21.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:21.618+0000 7f51e5600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f51e010fa70 con 0x7f51e0105c50 2024-04-24T08:51:21.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:21.618+0000 7f51e5600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f51e010fd70 con 0x7f51e0105290 2024-04-24T08:51:21.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:21.618+0000 7f51e4c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51e019fa50 0x7f51e01a1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:21.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:21.618+0000 7f51e4c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51e019fa50 0x7f51e01a1e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41200/0 (socket says 172.21.15.83:41200) 2024-04-24T08:51:21.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:21.618+0000 7f51e4c00640 1 -- 172.21.15.83:0/2500706185 learned_addr learned my addr 172.21.15.83:0/2500706185 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:51:21.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:21.618+0000 7f51df400640 1 --2- 172.21.15.83:0/2500706185 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f51e0105c50 0x7f51e019a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:21.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:21.618+0000 7f51dfe00640 1 --2- 172.21.15.83:0/2500706185 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f51e0105290 0x7f51e019a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:21.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:21.618+0000 7f51e4c00640 1 -- 172.21.15.83:0/2500706185 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f51e0105290 msgr2=0x7f51e019a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:21.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:21.618+0000 7f51e4c00640 1 --2- 172.21.15.83:0/2500706185 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f51e0105290 0x7f51e019a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:21.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:21.618+0000 7f51e4c00640 1 -- 172.21.15.83:0/2500706185 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f51e0105c50 msgr2=0x7f51e019a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:21.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:21.618+0000 7f51e4c00640 1 --2- 172.21.15.83:0/2500706185 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f51e0105c50 0x7f51e019a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:21.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:21.618+0000 7f51e4c00640 1 -- 172.21.15.83:0/2500706185 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51e01a24b0 con 0x7f51e019fa50 2024-04-24T08:51:21.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:21.619+0000 7f51e4c00640 1 --2- 172.21.15.83:0/2500706185 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51e019fa50 0x7f51e01a1e40 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f51d800c970 tx=0x7f51d800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:21.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:21.619+0000 7f51dcc00640 1 -- 172.21.15.83:0/2500706185 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51d8007bb0 con 0x7f51e019fa50 2024-04-24T08:51:21.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:21.619+0000 7f51dcc00640 1 -- 172.21.15.83:0/2500706185 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f51d8007d50 con 0x7f51e019fa50 2024-04-24T08:51:21.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:21.619+0000 7f51e5600640 1 -- 172.21.15.83:0/2500706185 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51e01a27a0 con 0x7f51e019fa50 2024-04-24T08:51:21.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:21.619+0000 7f51dcc00640 1 -- 172.21.15.83:0/2500706185 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51d8004e10 con 0x7f51e019fa50 2024-04-24T08:51:21.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:21.619+0000 7f51e5600640 1 -- 172.21.15.83:0/2500706185 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51e01a2ca0 con 0x7f51e019fa50 2024-04-24T08:51:21.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:21.620+0000 7f51e5600640 1 -- 172.21.15.83:0/2500706185 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51e010b4e0 con 0x7f51e019fa50 2024-04-24T08:51:21.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:21.622+0000 7f51dcc00640 1 -- 172.21.15.83:0/2500706185 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f51d8005020 con 0x7f51e019fa50 2024-04-24T08:51:21.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:21.624+0000 7f51dcc00640 1 --2- 172.21.15.83:0/2500706185 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f51c40a3bb0 0x7f51c40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:21.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:21.624+0000 7f51dcc00640 1 -- 172.21.15.83:0/2500706185 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9237+0+0 (secure 0 0 0) 0x7f51d80d05c0 con 0x7f51e019fa50 2024-04-24T08:51:21.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:21.624+0000 7f51dfe00640 1 --2- 172.21.15.83:0/2500706185 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f51c40a3bb0 0x7f51c40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:21.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:21.625+0000 7f51dfe00640 1 --2- 172.21.15.83:0/2500706185 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f51c40a3bb0 0x7f51c40a6070 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f51d002a9e0 tx=0x7f51d00314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:21.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:21.628+0000 7f51dcc00640 1 -- 172.21.15.83:0/2500706185 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f51d8014030 con 0x7f51e019fa50 2024-04-24T08:51:21.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:21.831+0000 7f51e5600640 1 -- 172.21.15.83:0/2500706185 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f51e0062c00 con 0x7f51e019fa50 2024-04-24T08:51:21.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:21.833+0000 7f51dcc00640 1 -- 172.21.15.83:0/2500706185 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+24887 (secure 0 0 0) 0x7f51d809cb50 con 0x7f51e019fa50 2024-04-24T08:51:21.834 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:51:21.834 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":93,"fsid":"bd3e039e-0215-11ef-bc93-c7b262605968","created":"2024-04-24T08:37:32.209944+0000","modified":"2024-04-24T08:50:42.968900+0000","last_up_change":"2024-04-24T08:47:17.966041+0000","last_in_change":"2024-04-24T08:46:53.835936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T08:41:56.834372+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T08:48:26.978311+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T08:48:28.352731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T08:48:32.275167+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2cd77f3a-19dd-4c6e-a199-1dcbd5faa679","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2132675555},{"type":"v1","addr":"172.21.15.83:6803","nonce":2132675555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2132675555},{"type":"v1","addr":"172.21.15.83:6805","nonce":2132675555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2132675555},{"type":"v1","addr":"172.21.15.83:6809","nonce":2132675555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2132675555},{"type":"v1","addr":"172.21.15.83:6807","nonce":2132675555}]},"public_addr":"172.21.15.83:6803/2132675555","cluster_addr":"172.21.15.83:6805/2132675555","heartbeat_back_addr":"172.21.15.83:6809/2132675555","heartbeat_front_addr":"172.21.15.83:6807/2132675555","state":["exists","up"]},{"osd":1,"uuid":"aba7f5df-2a4c-4c23-b502-d193605a5b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":3589158880},{"type":"v1","addr":"172.21.15.138:6801","nonce":3589158880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":3589158880},{"type":"v1","addr":"172.21.15.138:6803","nonce":3589158880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":3589158880},{"type":"v1","addr":"172.21.15.138:6807","nonce":3589158880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":3589158880},{"type":"v1","addr":"172.21.15.138:6805","nonce":3589158880}]},"public_addr":"172.21.15.138:6801/3589158880","cluster_addr":"172.21.15.138:6803/3589158880","heartbeat_back_addr":"172.21.15.138:6807/3589158880","heartbeat_front_addr":"172.21.15.138:6805/3589158880","state":["exists","up"]},{"osd":2,"uuid":"071bc167-2333-4420-bb58-bffbc10d7641","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":3169072688},{"type":"v1","addr":"172.21.15.189:6801","nonce":3169072688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":3169072688},{"type":"v1","addr":"172.21.15.189:6803","nonce":3169072688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":3169072688},{"type":"v1","addr":"172.21.15.189:6807","nonce":3169072688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":3169072688},{"type":"v1","addr":"172.21.15.189:6805","nonce":3169072688}]},"public_addr":"172.21.15.189:6801/3169072688","cluster_addr":"172.21.15.189:6803/3169072688","heartbeat_back_addr":"172.21.15.189:6807/3169072688","heartbeat_front_addr":"172.21.15.189:6805/3169072688","state":["exists","up"]},{"osd":3,"uuid":"f5624d78-4455-4e07-8c1a-f47c94cddb23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3131092450},{"type":"v1","addr":"172.21.15.83:6811","nonce":3131092450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3131092450},{"type":"v1","addr":"172.21.15.83:6813","nonce":3131092450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3131092450},{"type":"v1","addr":"172.21.15.83:6817","nonce":3131092450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3131092450},{"type":"v1","addr":"172.21.15.83:6815","nonce":3131092450}]},"public_addr":"172.21.15.83:6811/3131092450","cluster_addr":"172.21.15.83:6813/3131092450","heartbeat_back_addr":"172.21.15.83:6817/3131092450","heartbeat_front_addr":"172.21.15.83:6815/3131092450","state":["exists","up"]},{"osd":4,"uuid":"7b50cd14-9385-4535-a7b0-1c1da93c8548","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":823775360},{"type":"v1","addr":"172.21.15.138:6809","nonce":823775360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":823775360},{"type":"v1","addr":"172.21.15.138:6811","nonce":823775360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":823775360},{"type":"v1","addr":"172.21.15.138:6815","nonce":823775360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":823775360},{"type":"v1","addr":"172.21.15.138:6813","nonce":823775360}]},"public_addr":"172.21.15.138:6809/823775360","cluster_addr":"172.21.15.138:6811/823775360","heartbeat_back_addr":"172.21.15.138:6815/823775360","heartbeat_front_addr":"172.21.15.138:6813/823775360","state":["exists","up"]},{"osd":5,"uuid":"9c647797-35bc-449f-be84-af97d2c69c2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":1399817431},{"type":"v1","addr":"172.21.15.189:6809","nonce":1399817431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":1399817431},{"type":"v1","addr":"172.21.15.189:6811","nonce":1399817431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":1399817431},{"type":"v1","addr":"172.21.15.189:6815","nonce":1399817431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":1399817431},{"type":"v1","addr":"172.21.15.189:6813","nonce":1399817431}]},"public_addr":"172.21.15.189:6809/1399817431","cluster_addr":"172.21.15.189:6811/1399817431","heartbeat_back_addr":"172.21.15.189:6815/1399817431","heartbeat_front_addr":"172.21.15.189:6813/1399817431","state":["exists","up"]},{"osd":6,"uuid":"8f056584-16c8-4454-b445-3f780f26c196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":1611358483},{"type":"v1","addr":"172.21.15.83:6819","nonce":1611358483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1611358483},{"type":"v1","addr":"172.21.15.83:6821","nonce":1611358483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1611358483},{"type":"v1","addr":"172.21.15.83:6825","nonce":1611358483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":1611358483},{"type":"v1","addr":"172.21.15.83:6823","nonce":1611358483}]},"public_addr":"172.21.15.83:6819/1611358483","cluster_addr":"172.21.15.83:6821/1611358483","heartbeat_back_addr":"172.21.15.83:6825/1611358483","heartbeat_front_addr":"172.21.15.83:6823/1611358483","state":["exists","up"]},{"osd":7,"uuid":"7603efe3-da77-4b7f-9d03-29c4f293b6d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":3557856706},{"type":"v1","addr":"172.21.15.138:6817","nonce":3557856706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":3557856706},{"type":"v1","addr":"172.21.15.138:6819","nonce":3557856706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":3557856706},{"type":"v1","addr":"172.21.15.138:6823","nonce":3557856706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":3557856706},{"type":"v1","addr":"172.21.15.138:6821","nonce":3557856706}]},"public_addr":"172.21.15.138:6817/3557856706","cluster_addr":"172.21.15.138:6819/3557856706","heartbeat_back_addr":"172.21.15.138:6823/3557856706","heartbeat_front_addr":"172.21.15.138:6821/3557856706","state":["exists","up"]},{"osd":8,"uuid":"27d7d1b8-d6a0-417e-b8aa-596325aa1a77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":207018554},{"type":"v1","addr":"172.21.15.189:6817","nonce":207018554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":207018554},{"type":"v1","addr":"172.21.15.189:6819","nonce":207018554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":207018554},{"type":"v1","addr":"172.21.15.189:6823","nonce":207018554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":207018554},{"type":"v1","addr":"172.21.15.189:6821","nonce":207018554}]},"public_addr":"172.21.15.189:6817/207018554","cluster_addr":"172.21.15.189:6819/207018554","heartbeat_back_addr":"172.21.15.189:6823/207018554","heartbeat_front_addr":"172.21.15.189:6821/207018554","state":["exists","up"]},{"osd":9,"uuid":"baf19b98-881b-4d3a-9f72-8aa2dd73cad6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":519636751},{"type":"v1","addr":"172.21.15.83:6827","nonce":519636751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":519636751},{"type":"v1","addr":"172.21.15.83:6829","nonce":519636751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":519636751},{"type":"v1","addr":"172.21.15.83:6833","nonce":519636751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":519636751},{"type":"v1","addr":"172.21.15.83:6831","nonce":519636751}]},"public_addr":"172.21.15.83:6827/519636751","cluster_addr":"172.21.15.83:6829/519636751","heartbeat_back_addr":"172.21.15.83:6833/519636751","heartbeat_front_addr":"172.21.15.83:6831/519636751","state":["exists","up"]},{"osd":10,"uuid":"eb346717-5c07-47c8-95be-0e68fe449fdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1130872098},{"type":"v1","addr":"172.21.15.138:6825","nonce":1130872098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1130872098},{"type":"v1","addr":"172.21.15.138:6827","nonce":1130872098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1130872098},{"type":"v1","addr":"172.21.15.138:6831","nonce":1130872098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1130872098},{"type":"v1","addr":"172.21.15.138:6829","nonce":1130872098}]},"public_addr":"172.21.15.138:6825/1130872098","cluster_addr":"172.21.15.138:6827/1130872098","heartbeat_back_addr":"172.21.15.138:6831/1130872098","heartbeat_front_addr":"172.21.15.138:6829/1130872098","state":["exists","up"]},{"osd":11,"uuid":"8e37568e-981c-45b8-b2af-2f1b73898542","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":2998782690},{"type":"v1","addr":"172.21.15.189:6825","nonce":2998782690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":2998782690},{"type":"v1","addr":"172.21.15.189:6827","nonce":2998782690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":2998782690},{"type":"v1","addr":"172.21.15.189:6831","nonce":2998782690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":2998782690},{"type":"v1","addr":"172.21.15.189:6829","nonce":2998782690}]},"public_addr":"172.21.15.189:6825/2998782690","cluster_addr":"172.21.15.189:6827/2998782690","heartbeat_back_addr":"172.21.15.189:6831/2998782690","heartbeat_front_addr":"172.21.15.189:6829/2998782690","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:40:47.980103+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:41:21.970795+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:41:55.160426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:42:30.400475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:43:05.705663+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:43:40.112312+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:44:16.781289+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:44:52.151187+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:45:28.055944+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:46:04.969301+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:46:40.918306+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:47:16.886874+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"2.1d","mappings":[{"from":2,"to":1}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.2a","mappings":[{"from":3,"to":8}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.a","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:6801/820049883":"2024-04-25T08:38:40.661315+0000","172.21.15.83:0/4272874138":"2024-04-25T08:38:40.661315+0000","172.21.15.83:0/4212994335":"2024-04-25T08:38:40.661315+0000","172.21.15.83:6801/1996371091":"2024-04-25T08:38:05.800856+0000","172.21.15.83:0/1820663683":"2024-04-25T08:38:05.800856+0000","172.21.15.83:0/3802897262":"2024-04-25T08:38:05.800856+0000","172.21.15.83:6800/1996371091":"2024-04-25T08:38:05.800856+0000","172.21.15.83:6800/820049883":"2024-04-25T08:38:40.661315+0000","172.21.15.83:0/1366125049":"2024-04-25T08:38:05.800856+0000","172.21.15.83:0/340452868":"2024-04-25T08:38:40.661315+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T08:51:21.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:21.836+0000 7f51e5600640 1 -- 172.21.15.83:0/2500706185 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f51c40a3bb0 msgr2=0x7f51c40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:21.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:21.836+0000 7f51e5600640 1 --2- 172.21.15.83:0/2500706185 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f51c40a3bb0 0x7f51c40a6070 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f51d002a9e0 tx=0x7f51d00314a0 comp rx=0 tx=0).stop 2024-04-24T08:51:21.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:21.836+0000 7f51e5600640 1 -- 172.21.15.83:0/2500706185 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51e019fa50 msgr2=0x7f51e01a1e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:21.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:21.836+0000 7f51e5600640 1 --2- 172.21.15.83:0/2500706185 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51e019fa50 0x7f51e01a1e40 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f51d800c970 tx=0x7f51d800ce30 comp rx=0 tx=0).stop 2024-04-24T08:51:21.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:21.836+0000 7f51e5600640 1 -- 172.21.15.83:0/2500706185 shutdown_connections 2024-04-24T08:51:21.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:21.836+0000 7f51e5600640 1 --2- 172.21.15.83:0/2500706185 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f51c40a3bb0 0x7f51c40a6070 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:21.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:21.836+0000 7f51e5600640 1 --2- 172.21.15.83:0/2500706185 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51e019fa50 0x7f51e01a1e40 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:21.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:21.836+0000 7f51e5600640 1 --2- 172.21.15.83:0/2500706185 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f51e0105c50 0x7f51e019a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:21.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:21.836+0000 7f51e5600640 1 --2- 172.21.15.83:0/2500706185 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f51e0105290 0x7f51e019a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:21.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:21.836+0000 7f51e5600640 1 -- 172.21.15.83:0/2500706185 >> 172.21.15.83:0/2500706185 conn(0x7f51e00ff610 msgr2=0x7f51e01086b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:21.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:21.836+0000 7f51e5600640 1 -- 172.21.15.83:0/2500706185 shutdown_connections 2024-04-24T08:51:21.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:21.836+0000 7f51e5600640 1 -- 172.21.15.83:0/2500706185 wait complete. 2024-04-24T08:51:21.855 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T08:41:56.834372+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T08:48:26.978311+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T08:48:28.352731+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-24T08:48:32.275167+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-24T08:51:21.856 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T08:51:22.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.066+0000 7f7ff7e00640 1 -- 172.21.15.83:0/3604669243 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ff0109850 msgr2=0x7f7ff0109cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:22.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.066+0000 7f7ff7e00640 1 --2- 172.21.15.83:0/3604669243 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ff0109850 0x7f7ff0109cd0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc009900 tx=0x7f7fdc02a400 comp rx=0 tx=0).stop 2024-04-24T08:51:22.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.066+0000 7f7ff7e00640 1 -- 172.21.15.83:0/3604669243 shutdown_connections 2024-04-24T08:51:22.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.066+0000 7f7ff7e00640 1 --2- 172.21.15.83:0/3604669243 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7ff010a210 0x7f7ff0110aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:22.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.066+0000 7f7ff7e00640 1 --2- 172.21.15.83:0/3604669243 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ff0109850 0x7f7ff0109cd0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:22.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.066+0000 7f7ff7e00640 1 --2- 172.21.15.83:0/3604669243 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ff0108660 0x7f7ff0108a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:22.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.066+0000 7f7ff7e00640 1 -- 172.21.15.83:0/3604669243 >> 172.21.15.83:0/3604669243 conn(0x7f7ff0077600 msgr2=0x7f7ff0079a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:22.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.067+0000 7f7ff7e00640 1 -- 172.21.15.83:0/3604669243 shutdown_connections 2024-04-24T08:51:22.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.067+0000 7f7ff7e00640 1 -- 172.21.15.83:0/3604669243 wait complete. 2024-04-24T08:51:22.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.068+0000 7f7ff7e00640 1 Processor -- start 2024-04-24T08:51:22.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.068+0000 7f7ff7e00640 1 -- start start 2024-04-24T08:51:22.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.068+0000 7f7ff7e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7ff0108660 0x7f7ff019e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:22.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.068+0000 7f7ff7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ff0109850 0x7f7ff019ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:22.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.068+0000 7f7ff7e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ff010a210 0x7f7ff01a5d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:22.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.068+0000 7f7ff7e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7ff0114180 con 0x7f7ff0109850 2024-04-24T08:51:22.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.068+0000 7f7ff7e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7ff0114000 con 0x7f7ff010a210 2024-04-24T08:51:22.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.068+0000 7f7ff7e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7ff0114300 con 0x7f7ff0108660 2024-04-24T08:51:22.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.068+0000 7f7ff4a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ff0109850 0x7f7ff019ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:22.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.069+0000 7f7ff4a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ff0109850 0x7f7ff019ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41216/0 (socket says 172.21.15.83:41216) 2024-04-24T08:51:22.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.069+0000 7f7ff4a00640 1 -- 172.21.15.83:0/3739027243 learned_addr learned my addr 172.21.15.83:0/3739027243 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:51:22.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.069+0000 7f7ff5400640 1 --2- 172.21.15.83:0/3739027243 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7ff0108660 0x7f7ff019e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:22.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.069+0000 7f7ff5e00640 1 --2- 172.21.15.83:0/3739027243 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ff010a210 0x7f7ff01a5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:22.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.069+0000 7f7ff4a00640 1 -- 172.21.15.83:0/3739027243 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7ff0108660 msgr2=0x7f7ff019e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:22.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.069+0000 7f7ff4a00640 1 --2- 172.21.15.83:0/3739027243 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7ff0108660 0x7f7ff019e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:22.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.069+0000 7f7ff4a00640 1 -- 172.21.15.83:0/3739027243 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ff010a210 msgr2=0x7f7ff01a5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:22.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.069+0000 7f7ff4a00640 1 --2- 172.21.15.83:0/3739027243 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ff010a210 0x7f7ff01a5d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:22.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.069+0000 7f7ff4a00640 1 -- 172.21.15.83:0/3739027243 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ff01a63e0 con 0x7f7ff0109850 2024-04-24T08:51:22.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.069+0000 7f7ff5e00640 1 --2- 172.21.15.83:0/3739027243 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ff010a210 0x7f7ff01a5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:51:22.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.069+0000 7f7ff4a00640 1 --2- 172.21.15.83:0/3739027243 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ff0109850 0x7f7ff019ed00 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc005940 tx=0x7f7fdc004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:22.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.070+0000 7f7fee000640 1 -- 172.21.15.83:0/3739027243 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fdc03f030 con 0x7f7ff0109850 2024-04-24T08:51:22.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.070+0000 7f7fee000640 1 -- 172.21.15.83:0/3739027243 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7fdc003cb0 con 0x7f7ff0109850 2024-04-24T08:51:22.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.070+0000 7f7ff7e00640 1 -- 172.21.15.83:0/3739027243 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ff01a6670 con 0x7f7ff0109850 2024-04-24T08:51:22.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.070+0000 7f7fee000640 1 -- 172.21.15.83:0/3739027243 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fdc035070 con 0x7f7ff0109850 2024-04-24T08:51:22.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.070+0000 7f7ff7e00640 1 -- 172.21.15.83:0/3739027243 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ff01a6b60 con 0x7f7ff0109850 2024-04-24T08:51:22.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.071+0000 7f7ff7e00640 1 -- 172.21.15.83:0/3739027243 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ff010f980 con 0x7f7ff0109850 2024-04-24T08:51:22.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.073+0000 7f7fee000640 1 -- 172.21.15.83:0/3739027243 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f7fdc042030 con 0x7f7ff0109850 2024-04-24T08:51:22.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.074+0000 7f7fee000640 1 --2- 172.21.15.83:0/3739027243 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f7fd80a3bb0 0x7f7fd80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:22.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.074+0000 7f7fee000640 1 -- 172.21.15.83:0/3739027243 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9237+0+0 (secure 0 0 0) 0x7f7fdc0f13b0 con 0x7f7ff0109850 2024-04-24T08:51:22.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.074+0000 7f7ff5400640 1 --2- 172.21.15.83:0/3739027243 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f7fd80a3bb0 0x7f7fd80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:22.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.075+0000 7f7ff5400640 1 --2- 172.21.15.83:0/3739027243 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f7fd80a3bb0 0x7f7fd80a6070 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f7fe40058c0 tx=0x7f7fe4005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:22.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.078+0000 7f7fee000640 1 -- 172.21.15.83:0/3739027243 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7fdc0bd940 con 0x7f7ff0109850 2024-04-24T08:51:22.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.279+0000 7f7ff7e00640 1 -- 172.21.15.83:0/3739027243 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7ff0062c00 con 0x7f7ff0109850 2024-04-24T08:51:22.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.280+0000 7f7fee000640 1 -- 172.21.15.83:0/3739027243 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) ==== 93+0+10 (secure 0 0 0) 0x7f7fdc0bd240 con 0x7f7ff0109850 2024-04-24T08:51:22.281 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-24T08:51:22.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.283+0000 7f7ff7e00640 1 -- 172.21.15.83:0/3739027243 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f7fd80a3bb0 msgr2=0x7f7fd80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:22.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.283+0000 7f7ff7e00640 1 --2- 172.21.15.83:0/3739027243 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f7fd80a3bb0 0x7f7fd80a6070 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f7fe40058c0 tx=0x7f7fe4005850 comp rx=0 tx=0).stop 2024-04-24T08:51:22.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.283+0000 7f7ff7e00640 1 -- 172.21.15.83:0/3739027243 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ff0109850 msgr2=0x7f7ff019ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:22.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.283+0000 7f7ff7e00640 1 --2- 172.21.15.83:0/3739027243 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ff0109850 0x7f7ff019ed00 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc005940 tx=0x7f7fdc004430 comp rx=0 tx=0).stop 2024-04-24T08:51:22.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.283+0000 7f7ff7e00640 1 -- 172.21.15.83:0/3739027243 shutdown_connections 2024-04-24T08:51:22.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.283+0000 7f7ff7e00640 1 --2- 172.21.15.83:0/3739027243 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ff010a210 0x7f7ff01a5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:22.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.283+0000 7f7ff7e00640 1 --2- 172.21.15.83:0/3739027243 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f7fd80a3bb0 0x7f7fd80a6070 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:22.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.283+0000 7f7ff7e00640 1 --2- 172.21.15.83:0/3739027243 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ff0109850 0x7f7ff019ed00 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:22.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.283+0000 7f7ff7e00640 1 --2- 172.21.15.83:0/3739027243 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7ff0108660 0x7f7ff019e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:22.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.283+0000 7f7ff7e00640 1 -- 172.21.15.83:0/3739027243 >> 172.21.15.83:0/3739027243 conn(0x7f7ff0077600 msgr2=0x7f7ff010cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:22.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.283+0000 7f7ff7e00640 1 -- 172.21.15.83:0/3739027243 shutdown_connections 2024-04-24T08:51:22.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.283+0000 7f7ff7e00640 1 -- 172.21.15.83:0/3739027243 wait complete. 2024-04-24T08:51:22.298 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:21 smithi083 ceph-mon[27907]: pgmap v463: 129 pgs: 129 active+clean; 644 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-24T08:51:22.298 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:21 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/1166911209' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:51:22.298 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:21 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/2500706185' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T08:51:22.301 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T08:51:22.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:21 smithi138 ceph-mon[31441]: pgmap v463: 129 pgs: 129 active+clean; 644 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-24T08:51:22.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:21 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/1166911209' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:51:22.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:21 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/2500706185' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T08:51:22.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:21 smithi189 ceph-mon[31791]: pgmap v463: 129 pgs: 129 active+clean; 644 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-24T08:51:22.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:21 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/1166911209' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:51:22.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:21 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/2500706185' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T08:51:22.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.504+0000 7f5a51a00640 1 -- 172.21.15.83:0/3001489959 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a4c0769c0 msgr2=0x7f5a4c076de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:22.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.504+0000 7f5a51a00640 1 --2- 172.21.15.83:0/3001489959 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a4c0769c0 0x7f5a4c076de0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c009900 tx=0x7f5a3c02a410 comp rx=0 tx=0).stop 2024-04-24T08:51:22.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.504+0000 7f5a51a00640 1 -- 172.21.15.83:0/3001489959 shutdown_connections 2024-04-24T08:51:22.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.504+0000 7f5a51a00640 1 --2- 172.21.15.83:0/3001489959 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5a4c0785a0 0x7f5a4c078e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:22.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.504+0000 7f5a51a00640 1 --2- 172.21.15.83:0/3001489959 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5a4c077c00 0x7f5a4c078060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:22.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.504+0000 7f5a51a00640 1 --2- 172.21.15.83:0/3001489959 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a4c0769c0 0x7f5a4c076de0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:22.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.504+0000 7f5a51a00640 1 -- 172.21.15.83:0/3001489959 >> 172.21.15.83:0/3001489959 conn(0x7f5a4c0ff6b0 msgr2=0x7f5a4c101af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:22.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.504+0000 7f5a51a00640 1 -- 172.21.15.83:0/3001489959 shutdown_connections 2024-04-24T08:51:22.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.504+0000 7f5a51a00640 1 -- 172.21.15.83:0/3001489959 wait complete. 2024-04-24T08:51:22.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.505+0000 7f5a51a00640 1 Processor -- start 2024-04-24T08:51:22.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.505+0000 7f5a51a00640 1 -- start start 2024-04-24T08:51:22.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.506+0000 7f5a51a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5a4c0769c0 0x7f5a4c19e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:22.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.506+0000 7f5a51a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5a4c077c00 0x7f5a4c19ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:22.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.506+0000 7f5a51a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a4c0785a0 0x7f5a4c1a5d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:22.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.506+0000 7f5a51a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5a4c1140c0 con 0x7f5a4c0785a0 2024-04-24T08:51:22.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.506+0000 7f5a51a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5a4c113f40 con 0x7f5a4c0769c0 2024-04-24T08:51:22.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.506+0000 7f5a51a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5a4c114240 con 0x7f5a4c077c00 2024-04-24T08:51:22.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.506+0000 7f5a51000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a4c0785a0 0x7f5a4c1a5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:22.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.506+0000 7f5a51000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a4c0785a0 0x7f5a4c1a5d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41232/0 (socket says 172.21.15.83:41232) 2024-04-24T08:51:22.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.506+0000 7f5a51000640 1 -- 172.21.15.83:0/432228842 learned_addr learned my addr 172.21.15.83:0/432228842 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:51:22.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.506+0000 7f5a4be00640 1 --2- 172.21.15.83:0/432228842 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5a4c0769c0 0x7f5a4c19e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:22.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.507+0000 7f5a4b400640 1 --2- 172.21.15.83:0/432228842 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5a4c077c00 0x7f5a4c19ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:22.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.507+0000 7f5a51000640 1 -- 172.21.15.83:0/432228842 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5a4c077c00 msgr2=0x7f5a4c19ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:22.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.507+0000 7f5a51000640 1 --2- 172.21.15.83:0/432228842 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5a4c077c00 0x7f5a4c19ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:22.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.507+0000 7f5a51000640 1 -- 172.21.15.83:0/432228842 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5a4c0769c0 msgr2=0x7f5a4c19e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:22.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.507+0000 7f5a51000640 1 --2- 172.21.15.83:0/432228842 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5a4c0769c0 0x7f5a4c19e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:22.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.507+0000 7f5a51000640 1 -- 172.21.15.83:0/432228842 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a4c1a6370 con 0x7f5a4c0785a0 2024-04-24T08:51:22.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.507+0000 7f5a4be00640 1 --2- 172.21.15.83:0/432228842 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5a4c0769c0 0x7f5a4c19e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:51:22.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.507+0000 7f5a4b400640 1 --2- 172.21.15.83:0/432228842 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5a4c077c00 0x7f5a4c19ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:51:22.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.507+0000 7f5a51000640 1 --2- 172.21.15.83:0/432228842 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a4c0785a0 0x7f5a4c1a5d00 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f5a4400eaa0 tx=0x7f5a4400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:22.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.507+0000 7f5a48c00640 1 -- 172.21.15.83:0/432228842 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a4400cdb0 con 0x7f5a4c0785a0 2024-04-24T08:51:22.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.507+0000 7f5a48c00640 1 -- 172.21.15.83:0/432228842 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5a44005020 con 0x7f5a4c0785a0 2024-04-24T08:51:22.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.507+0000 7f5a51a00640 1 -- 172.21.15.83:0/432228842 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a4c1a6660 con 0x7f5a4c0785a0 2024-04-24T08:51:22.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.507+0000 7f5a48c00640 1 -- 172.21.15.83:0/432228842 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a440116d0 con 0x7f5a4c0785a0 2024-04-24T08:51:22.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.507+0000 7f5a51a00640 1 -- 172.21.15.83:0/432228842 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a4c1a6b60 con 0x7f5a4c0785a0 2024-04-24T08:51:22.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.508+0000 7f5a51a00640 1 -- 172.21.15.83:0/432228842 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a4c110cf0 con 0x7f5a4c0785a0 2024-04-24T08:51:22.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.510+0000 7f5a48c00640 1 -- 172.21.15.83:0/432228842 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f5a440051d0 con 0x7f5a4c0785a0 2024-04-24T08:51:22.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.512+0000 7f5a48c00640 1 --2- 172.21.15.83:0/432228842 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f5a300a3bb0 0x7f5a300a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:22.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.512+0000 7f5a48c00640 1 -- 172.21.15.83:0/432228842 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9237+0+0 (secure 0 0 0) 0x7f5a440d0af0 con 0x7f5a4c0785a0 2024-04-24T08:51:22.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.512+0000 7f5a4be00640 1 --2- 172.21.15.83:0/432228842 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f5a300a3bb0 0x7f5a300a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:22.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.513+0000 7f5a4be00640 1 --2- 172.21.15.83:0/432228842 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f5a300a3bb0 0x7f5a300a6070 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c02a930 tx=0x7f5a3c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:22.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.516+0000 7f5a48c00640 1 -- 172.21.15.83:0/432228842 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5a44016030 con 0x7f5a4c0785a0 2024-04-24T08:51:22.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.728+0000 7f5a51a00640 1 -- 172.21.15.83:0/432228842 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f5a4c062c00 con 0x7f5a4c0785a0 2024-04-24T08:51:22.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.729+0000 7f5a48c00640 1 -- 172.21.15.83:0/432228842 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) ==== 104+0+11 (secure 0 0 0) 0x7f5a4409d080 con 0x7f5a4c0785a0 2024-04-24T08:51:22.730 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-24T08:51:22.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.732+0000 7f5a51a00640 1 -- 172.21.15.83:0/432228842 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f5a300a3bb0 msgr2=0x7f5a300a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:22.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.732+0000 7f5a51a00640 1 --2- 172.21.15.83:0/432228842 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f5a300a3bb0 0x7f5a300a6070 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c02a930 tx=0x7f5a3c005fd0 comp rx=0 tx=0).stop 2024-04-24T08:51:22.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.732+0000 7f5a51a00640 1 -- 172.21.15.83:0/432228842 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a4c0785a0 msgr2=0x7f5a4c1a5d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:22.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.732+0000 7f5a51a00640 1 --2- 172.21.15.83:0/432228842 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a4c0785a0 0x7f5a4c1a5d00 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f5a4400eaa0 tx=0x7f5a4400ef60 comp rx=0 tx=0).stop 2024-04-24T08:51:22.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.732+0000 7f5a51a00640 1 -- 172.21.15.83:0/432228842 shutdown_connections 2024-04-24T08:51:22.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.732+0000 7f5a51a00640 1 --2- 172.21.15.83:0/432228842 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f5a300a3bb0 0x7f5a300a6070 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:22.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.732+0000 7f5a51a00640 1 --2- 172.21.15.83:0/432228842 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a4c0785a0 0x7f5a4c1a5d00 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:22.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.732+0000 7f5a51a00640 1 --2- 172.21.15.83:0/432228842 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5a4c077c00 0x7f5a4c19ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:22.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.732+0000 7f5a51a00640 1 --2- 172.21.15.83:0/432228842 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5a4c0769c0 0x7f5a4c19e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:22.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.732+0000 7f5a51a00640 1 -- 172.21.15.83:0/432228842 >> 172.21.15.83:0/432228842 conn(0x7f5a4c0ff6b0 msgr2=0x7f5a4c100e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:22.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.732+0000 7f5a51a00640 1 -- 172.21.15.83:0/432228842 shutdown_connections 2024-04-24T08:51:22.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.732+0000 7f5a51a00640 1 -- 172.21.15.83:0/432228842 wait complete. 2024-04-24T08:51:22.752 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T08:51:22.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.953+0000 7fa277200640 1 -- 172.21.15.83:0/3833883094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa278104140 msgr2=0x7fa278104560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:22.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.953+0000 7fa277200640 1 --2- 172.21.15.83:0/3833883094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa278104140 0x7fa278104560 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fa268009900 tx=0x7fa26802a410 comp rx=0 tx=0).stop 2024-04-24T08:51:22.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.953+0000 7fa277200640 1 -- 172.21.15.83:0/3833883094 shutdown_connections 2024-04-24T08:51:22.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.953+0000 7fa277200640 1 --2- 172.21.15.83:0/3833883094 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa278105cf0 0x7fa27810c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:22.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.953+0000 7fa277200640 1 --2- 172.21.15.83:0/3833883094 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa278105330 0x7fa2781057b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:22.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.953+0000 7fa277200640 1 --2- 172.21.15.83:0/3833883094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa278104140 0x7fa278104560 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:22.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.953+0000 7fa277200640 1 -- 172.21.15.83:0/3833883094 >> 172.21.15.83:0/3833883094 conn(0x7fa2780ff6b0 msgr2=0x7fa278101af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:22.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.953+0000 7fa277200640 1 -- 172.21.15.83:0/3833883094 shutdown_connections 2024-04-24T08:51:22.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.953+0000 7fa277200640 1 -- 172.21.15.83:0/3833883094 wait complete. 2024-04-24T08:51:22.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.954+0000 7fa277200640 1 Processor -- start 2024-04-24T08:51:22.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.954+0000 7fa277200640 1 -- start start 2024-04-24T08:51:22.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.955+0000 7fa277200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa278104140 0x7fa27819a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:22.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.955+0000 7fa277200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa278105330 0x7fa27819a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:22.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.955+0000 7fa277200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa278105cf0 0x7fa2781a1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:22.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.955+0000 7fa277200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa27810fcd0 con 0x7fa278105330 2024-04-24T08:51:22.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.955+0000 7fa277200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa27810fb50 con 0x7fa278105cf0 2024-04-24T08:51:22.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.955+0000 7fa277200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa27810fe50 con 0x7fa278104140 2024-04-24T08:51:22.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.955+0000 7fa26fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa278105330 0x7fa27819a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:22.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.955+0000 7fa26fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa278105330 0x7fa27819a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41254/0 (socket says 172.21.15.83:41254) 2024-04-24T08:51:22.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.955+0000 7fa26fe00640 1 -- 172.21.15.83:0/3076693377 learned_addr learned my addr 172.21.15.83:0/3076693377 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:51:22.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.956+0000 7fa276800640 1 --2- 172.21.15.83:0/3076693377 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa278105cf0 0x7fa2781a1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:22.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.956+0000 7fa275e00640 1 --2- 172.21.15.83:0/3076693377 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa278104140 0x7fa27819a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:22.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.956+0000 7fa26fe00640 1 -- 172.21.15.83:0/3076693377 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa278104140 msgr2=0x7fa27819a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:22.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.956+0000 7fa26fe00640 1 --2- 172.21.15.83:0/3076693377 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa278104140 0x7fa27819a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:22.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.956+0000 7fa26fe00640 1 -- 172.21.15.83:0/3076693377 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa278105cf0 msgr2=0x7fa2781a1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:22.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.956+0000 7fa26fe00640 1 --2- 172.21.15.83:0/3076693377 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa278105cf0 0x7fa2781a1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:22.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.956+0000 7fa26fe00640 1 -- 172.21.15.83:0/3076693377 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2781a1f70 con 0x7fa278105330 2024-04-24T08:51:22.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.956+0000 7fa276800640 1 --2- 172.21.15.83:0/3076693377 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa278105cf0 0x7fa2781a1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:51:22.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.956+0000 7fa26fe00640 1 --2- 172.21.15.83:0/3076693377 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa278105330 0x7fa27819a890 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fa26000d950 tx=0x7fa26000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:22.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.956+0000 7fa26ea00640 1 -- 172.21.15.83:0/3076693377 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa260017030 con 0x7fa278105330 2024-04-24T08:51:22.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.956+0000 7fa26ea00640 1 -- 172.21.15.83:0/3076693377 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa260004020 con 0x7fa278105330 2024-04-24T08:51:22.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.956+0000 7fa277200640 1 -- 172.21.15.83:0/3076693377 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2781a2260 con 0x7fa278105330 2024-04-24T08:51:22.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.956+0000 7fa26ea00640 1 -- 172.21.15.83:0/3076693377 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2600047a0 con 0x7fa278105330 2024-04-24T08:51:22.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.956+0000 7fa277200640 1 -- 172.21.15.83:0/3076693377 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2781a27b0 con 0x7fa278105330 2024-04-24T08:51:22.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.957+0000 7fa277200640 1 -- 172.21.15.83:0/3076693377 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa27810b4e0 con 0x7fa278105330 2024-04-24T08:51:22.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.959+0000 7fa26ea00640 1 -- 172.21.15.83:0/3076693377 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fa2600041d0 con 0x7fa278105330 2024-04-24T08:51:22.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.961+0000 7fa26ea00640 1 --2- 172.21.15.83:0/3076693377 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fa25c0a3bb0 0x7fa25c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:22.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.961+0000 7fa26ea00640 1 -- 172.21.15.83:0/3076693377 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9237+0+0 (secure 0 0 0) 0x7fa2600d0430 con 0x7fa278105330 2024-04-24T08:51:22.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.961+0000 7fa275e00640 1 --2- 172.21.15.83:0/3076693377 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fa25c0a3bb0 0x7fa25c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:22.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.962+0000 7fa275e00640 1 --2- 172.21.15.83:0/3076693377 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fa25c0a3bb0 0x7fa25c0a6070 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa26802a930 tx=0x7fa268005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:22.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:22.965+0000 7fa26ea00640 1 -- 172.21.15.83:0/3076693377 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa26009c9c0 con 0x7fa278105330 2024-04-24T08:51:23.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:23.169+0000 7fa277200640 1 -- 172.21.15.83:0/3076693377 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa278062c00 con 0x7fa278105330 2024-04-24T08:51:23.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:23.170+0000 7fa26ea00640 1 -- 172.21.15.83:0/3076693377 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) ==== 100+0+11 (secure 0 0 0) 0x7fa26009c2c0 con 0x7fa278105330 2024-04-24T08:51:23.171 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2024-04-24T08:51:23.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:23.173+0000 7fa277200640 1 -- 172.21.15.83:0/3076693377 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fa25c0a3bb0 msgr2=0x7fa25c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:23.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:23.173+0000 7fa277200640 1 --2- 172.21.15.83:0/3076693377 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fa25c0a3bb0 0x7fa25c0a6070 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa26802a930 tx=0x7fa268005fd0 comp rx=0 tx=0).stop 2024-04-24T08:51:23.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:23.173+0000 7fa277200640 1 -- 172.21.15.83:0/3076693377 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa278105330 msgr2=0x7fa27819a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:23.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:23.173+0000 7fa277200640 1 --2- 172.21.15.83:0/3076693377 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa278105330 0x7fa27819a890 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fa26000d950 tx=0x7fa26000de10 comp rx=0 tx=0).stop 2024-04-24T08:51:23.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:23.173+0000 7fa277200640 1 -- 172.21.15.83:0/3076693377 shutdown_connections 2024-04-24T08:51:23.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:23.173+0000 7fa277200640 1 --2- 172.21.15.83:0/3076693377 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa278105cf0 0x7fa2781a1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:23.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:23.173+0000 7fa277200640 1 --2- 172.21.15.83:0/3076693377 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fa25c0a3bb0 0x7fa25c0a6070 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:23.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:23.173+0000 7fa277200640 1 --2- 172.21.15.83:0/3076693377 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa278105330 0x7fa27819a890 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:23.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:23.173+0000 7fa277200640 1 --2- 172.21.15.83:0/3076693377 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa278104140 0x7fa27819a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:23.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:23.173+0000 7fa277200640 1 -- 172.21.15.83:0/3076693377 >> 172.21.15.83:0/3076693377 conn(0x7fa2780ff6b0 msgr2=0x7fa2781086b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:23.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:23.173+0000 7fa277200640 1 -- 172.21.15.83:0/3076693377 shutdown_connections 2024-04-24T08:51:23.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:23.173+0000 7fa277200640 1 -- 172.21.15.83:0/3076693377 wait complete. 2024-04-24T08:51:23.192 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-24T08:51:23.251 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:23 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/3739027243' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T08:51:23.251 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:23 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/432228842' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T08:51:23.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:23.386+0000 7f6802a00640 1 -- 172.21.15.83:0/497538746 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67fc109730 msgr2=0x7f67fc109bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:23.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:23.386+0000 7f6802a00640 1 --2- 172.21.15.83:0/497538746 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67fc109730 0x7f67fc109bb0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f67e8009900 tx=0x7f67e802a410 comp rx=0 tx=0).stop 2024-04-24T08:51:23.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:23.386+0000 7f6802a00640 1 -- 172.21.15.83:0/497538746 shutdown_connections 2024-04-24T08:51:23.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:23.386+0000 7f6802a00640 1 --2- 172.21.15.83:0/497538746 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f67fc10a0f0 0x7f67fc110980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:23.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:23.386+0000 7f6802a00640 1 --2- 172.21.15.83:0/497538746 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67fc109730 0x7f67fc109bb0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:23.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:23.386+0000 7f6802a00640 1 --2- 172.21.15.83:0/497538746 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f67fc108540 0x7f67fc108960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:23.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:23.386+0000 7f6802a00640 1 -- 172.21.15.83:0/497538746 >> 172.21.15.83:0/497538746 conn(0x7f67fc0774d0 msgr2=0x7f67fc079910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:23.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:23.386+0000 7f6802a00640 1 -- 172.21.15.83:0/497538746 shutdown_connections 2024-04-24T08:51:23.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:23.386+0000 7f6802a00640 1 -- 172.21.15.83:0/497538746 wait complete. 2024-04-24T08:51:23.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:23.387+0000 7f6802a00640 1 Processor -- start 2024-04-24T08:51:23.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:23.387+0000 7f6802a00640 1 -- start start 2024-04-24T08:51:23.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:23.388+0000 7f6802a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67fc108540 0x7f67fc19e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:23.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:23.388+0000 7f6802a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f67fc109730 0x7f67fc19ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:23.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:23.388+0000 7f6802a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f67fc10a0f0 0x7f67fc1a5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:23.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:23.388+0000 7f6802a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f67fc114160 con 0x7f67fc108540 2024-04-24T08:51:23.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:23.388+0000 7f6802a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f67fc113fe0 con 0x7f67fc109730 2024-04-24T08:51:23.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:23.388+0000 7f6802a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f67fc1142e0 con 0x7f67fc10a0f0 2024-04-24T08:51:23.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:23.388+0000 7f6801600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67fc108540 0x7f67fc19e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:23.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:23.388+0000 7f6801600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67fc108540 0x7f67fc19e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41272/0 (socket says 172.21.15.83:41272) 2024-04-24T08:51:23.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:23.388+0000 7f6801600640 1 -- 172.21.15.83:0/1927083727 learned_addr learned my addr 172.21.15.83:0/1927083727 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:51:23.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:23.388+0000 7f6800c00640 1 --2- 172.21.15.83:0/1927083727 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f67fc109730 0x7f67fc19ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:23.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:23.388+0000 7f6801600640 1 -- 172.21.15.83:0/1927083727 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f67fc10a0f0 msgr2=0x7f67fc1a5ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:23.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:23.388+0000 7f6801600640 1 --2- 172.21.15.83:0/1927083727 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f67fc10a0f0 0x7f67fc1a5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:23.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:23.388+0000 7f6801600640 1 -- 172.21.15.83:0/1927083727 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f67fc109730 msgr2=0x7f67fc19ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:23.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:23.388+0000 7f6801600640 1 --2- 172.21.15.83:0/1927083727 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f67fc109730 0x7f67fc19ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:23.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:23.389+0000 7f6801600640 1 -- 172.21.15.83:0/1927083727 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67fc1a6510 con 0x7f67fc108540 2024-04-24T08:51:23.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:23.389+0000 7f6801600640 1 --2- 172.21.15.83:0/1927083727 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67fc108540 0x7f67fc19e8f0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f67f000b450 tx=0x7f67f000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:23.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:23.389+0000 7f67fa000640 1 -- 172.21.15.83:0/1927083727 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67f0015020 con 0x7f67fc108540 2024-04-24T08:51:23.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:23.389+0000 7f6802a00640 1 -- 172.21.15.83:0/1927083727 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67fc1a6800 con 0x7f67fc108540 2024-04-24T08:51:23.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:23.389+0000 7f67fa000640 1 -- 172.21.15.83:0/1927083727 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f67f0011d40 con 0x7f67fc108540 2024-04-24T08:51:23.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:23.389+0000 7f67fa000640 1 -- 172.21.15.83:0/1927083727 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67f000f340 con 0x7f67fc108540 2024-04-24T08:51:23.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:23.389+0000 7f6802a00640 1 -- 172.21.15.83:0/1927083727 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67fc1a6dd0 con 0x7f67fc108540 2024-04-24T08:51:23.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:23.390+0000 7f6802a00640 1 -- 172.21.15.83:0/1927083727 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67fc10f920 con 0x7f67fc108540 2024-04-24T08:51:23.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:23.392+0000 7f67fa000640 1 -- 172.21.15.83:0/1927083727 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f67f0018020 con 0x7f67fc108540 2024-04-24T08:51:23.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:23.394+0000 7f67fa000640 1 --2- 172.21.15.83:0/1927083727 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f67e40a3bb0 0x7f67e40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:23.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:23.394+0000 7f67fa000640 1 -- 172.21.15.83:0/1927083727 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9237+0+0 (secure 0 0 0) 0x7f67f00d0380 con 0x7f67fc108540 2024-04-24T08:51:23.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:23.394+0000 7f6800c00640 1 --2- 172.21.15.83:0/1927083727 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f67e40a3bb0 0x7f67e40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:23.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:23.395+0000 7f6800c00640 1 --2- 172.21.15.83:0/1927083727 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f67e40a3bb0 0x7f67e40a6070 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f67e802a930 tx=0x7f67e8000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:23.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:23.398+0000 7f67fa000640 1 -- 172.21.15.83:0/1927083727 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f67f001f020 con 0x7f67fc108540 2024-04-24T08:51:23.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:23 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/3739027243' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T08:51:23.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:23 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/432228842' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T08:51:23.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:23 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/3739027243' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T08:51:23.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:23 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/432228842' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T08:51:23.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:23.586+0000 7f6802a00640 1 -- 172.21.15.83:0/1927083727 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f67fc062c00 con 0x7f67fc108540 2024-04-24T08:51:23.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:23.587+0000 7f67fa000640 1 -- 172.21.15.83:0/1927083727 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) ==== 103+0+11 (secure 0 0 0) 0x7f67f000d040 con 0x7f67fc108540 2024-04-24T08:51:23.587 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-24T08:51:23.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:23.589+0000 7f6802a00640 1 -- 172.21.15.83:0/1927083727 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f67e40a3bb0 msgr2=0x7f67e40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:23.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:23.589+0000 7f6802a00640 1 --2- 172.21.15.83:0/1927083727 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f67e40a3bb0 0x7f67e40a6070 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f67e802a930 tx=0x7f67e8000bc0 comp rx=0 tx=0).stop 2024-04-24T08:51:23.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:23.589+0000 7f6802a00640 1 -- 172.21.15.83:0/1927083727 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67fc108540 msgr2=0x7f67fc19e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:23.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:23.589+0000 7f6802a00640 1 --2- 172.21.15.83:0/1927083727 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67fc108540 0x7f67fc19e8f0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f67f000b450 tx=0x7f67f000b910 comp rx=0 tx=0).stop 2024-04-24T08:51:23.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:23.589+0000 7f6802a00640 1 -- 172.21.15.83:0/1927083727 shutdown_connections 2024-04-24T08:51:23.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:23.589+0000 7f6802a00640 1 --2- 172.21.15.83:0/1927083727 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f67fc10a0f0 0x7f67fc1a5ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:23.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:23.589+0000 7f6802a00640 1 --2- 172.21.15.83:0/1927083727 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f67fc109730 0x7f67fc19ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:23.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:23.589+0000 7f6802a00640 1 --2- 172.21.15.83:0/1927083727 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f67e40a3bb0 0x7f67e40a6070 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:23.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:23.589+0000 7f6802a00640 1 --2- 172.21.15.83:0/1927083727 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67fc108540 0x7f67fc19e8f0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:23.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:23.589+0000 7f6802a00640 1 -- 172.21.15.83:0/1927083727 >> 172.21.15.83:0/1927083727 conn(0x7f67fc0774d0 msgr2=0x7f67fc10caf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:23.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:23.589+0000 7f6802a00640 1 -- 172.21.15.83:0/1927083727 shutdown_connections 2024-04-24T08:51:23.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:23.589+0000 7f6802a00640 1 -- 172.21.15.83:0/1927083727 wait complete. 2024-04-24T08:51:23.606 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T08:51:23.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:23.798+0000 7fa0bf200640 1 -- 172.21.15.83:0/3173512641 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa0c0100d80 msgr2=0x7fa0c01011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:23.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:23.798+0000 7fa0bf200640 1 --2- 172.21.15.83:0/3173512641 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa0c0100d80 0x7fa0c01011a0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fa0b0009ca0 tx=0x7fa0b002a8b0 comp rx=0 tx=0).stop 2024-04-24T08:51:23.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:23.798+0000 7fa0bf200640 1 -- 172.21.15.83:0/3173512641 shutdown_connections 2024-04-24T08:51:23.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:23.798+0000 7fa0bf200640 1 --2- 172.21.15.83:0/3173512641 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa0c0102930 0x7fa0c01091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:23.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:23.798+0000 7fa0bf200640 1 --2- 172.21.15.83:0/3173512641 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa0c0101f70 0x7fa0c01023f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:23.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:23.798+0000 7fa0bf200640 1 --2- 172.21.15.83:0/3173512641 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa0c0100d80 0x7fa0c01011a0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:23.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:23.798+0000 7fa0bf200640 1 -- 172.21.15.83:0/3173512641 >> 172.21.15.83:0/3173512641 conn(0x7fa0c00fc2d0 msgr2=0x7fa0c00fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:23.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:23.798+0000 7fa0bf200640 1 -- 172.21.15.83:0/3173512641 shutdown_connections 2024-04-24T08:51:23.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:23.798+0000 7fa0bf200640 1 -- 172.21.15.83:0/3173512641 wait complete. 2024-04-24T08:51:23.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:23.799+0000 7fa0bf200640 1 Processor -- start 2024-04-24T08:51:23.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:23.799+0000 7fa0bf200640 1 -- start start 2024-04-24T08:51:23.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:23.799+0000 7fa0bf200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa0c0100d80 0x7fa0c019a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:23.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:23.799+0000 7fa0bf200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa0c0101f70 0x7fa0c019a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:23.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:23.799+0000 7fa0bf200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa0c0102930 0x7fa0c01a17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:23.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:23.799+0000 7fa0bf200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c010c8d0 con 0x7fa0c0101f70 2024-04-24T08:51:23.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:23.799+0000 7fa0bf200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c010c750 con 0x7fa0c0100d80 2024-04-24T08:51:23.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:23.799+0000 7fa0bf200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c010ca50 con 0x7fa0c0102930 2024-04-24T08:51:23.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:23.800+0000 7fa0b5400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa0c0101f70 0x7fa0c019a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:23.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:23.800+0000 7fa0b5400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa0c0101f70 0x7fa0c019a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41278/0 (socket says 172.21.15.83:41278) 2024-04-24T08:51:23.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:23.800+0000 7fa0b5400640 1 -- 172.21.15.83:0/1176763574 learned_addr learned my addr 172.21.15.83:0/1176763574 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:51:23.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:23.800+0000 7fa0bde00640 1 --2- 172.21.15.83:0/1176763574 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa0c0100d80 0x7fa0c019a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:23.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:23.800+0000 7fa0be800640 1 --2- 172.21.15.83:0/1176763574 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa0c0102930 0x7fa0c01a17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:23.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:23.800+0000 7fa0b5400640 1 -- 172.21.15.83:0/1176763574 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa0c0102930 msgr2=0x7fa0c01a17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:23.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:23.800+0000 7fa0b5400640 1 --2- 172.21.15.83:0/1176763574 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa0c0102930 0x7fa0c01a17a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:23.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:23.800+0000 7fa0b5400640 1 -- 172.21.15.83:0/1176763574 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa0c0100d80 msgr2=0x7fa0c019a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:23.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:23.800+0000 7fa0b5400640 1 --2- 172.21.15.83:0/1176763574 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa0c0100d80 0x7fa0c019a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:23.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:23.800+0000 7fa0b5400640 1 -- 172.21.15.83:0/1176763574 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0c01a1e10 con 0x7fa0c0101f70 2024-04-24T08:51:23.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:23.801+0000 7fa0b5400640 1 --2- 172.21.15.83:0/1176763574 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa0c0101f70 0x7fa0c019a730 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fa0a800c920 tx=0x7fa0a800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:23.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:23.801+0000 7fa0b7400640 1 -- 172.21.15.83:0/1176763574 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a8007c70 con 0x7fa0c0101f70 2024-04-24T08:51:23.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:23.801+0000 7fa0b7400640 1 -- 172.21.15.83:0/1176763574 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa0a8007e10 con 0x7fa0c0101f70 2024-04-24T08:51:23.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:23.801+0000 7fa0bf200640 1 -- 172.21.15.83:0/1176763574 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0c01a2100 con 0x7fa0c0101f70 2024-04-24T08:51:23.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:23.801+0000 7fa0b7400640 1 -- 172.21.15.83:0/1176763574 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a8018820 con 0x7fa0c0101f70 2024-04-24T08:51:23.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:23.801+0000 7fa0bf200640 1 -- 172.21.15.83:0/1176763574 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0c01a26a0 con 0x7fa0c0101f70 2024-04-24T08:51:23.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:23.802+0000 7fa0bf200640 1 -- 172.21.15.83:0/1176763574 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0c01a2960 con 0x7fa0c0101f70 2024-04-24T08:51:23.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:23.804+0000 7fa0b7400640 1 -- 172.21.15.83:0/1176763574 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fa0a80189c0 con 0x7fa0c0101f70 2024-04-24T08:51:23.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:23.806+0000 7fa0b7400640 1 --2- 172.21.15.83:0/1176763574 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fa0a40a3c30 0x7fa0a40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:23.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:23.807+0000 7fa0b7400640 1 -- 172.21.15.83:0/1176763574 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9237+0+0 (secure 0 0 0) 0x7fa0a80d0990 con 0x7fa0c0101f70 2024-04-24T08:51:23.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:23.807+0000 7fa0bde00640 1 --2- 172.21.15.83:0/1176763574 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fa0a40a3c30 0x7fa0a40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:23.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:23.807+0000 7fa0bde00640 1 --2- 172.21.15.83:0/1176763574 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fa0a40a3c30 0x7fa0a40a60f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa0b0000f80 tx=0x7fa0b0005fa0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:23.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:23.811+0000 7fa0b7400640 1 -- 172.21.15.83:0/1176763574 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa0a8014030 con 0x7fa0c0101f70 2024-04-24T08:51:24.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.054+0000 7fa0bf200640 1 -- 172.21.15.83:0/1176763574 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa0c01a2b30 con 0x7fa0c0101f70 2024-04-24T08:51:24.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.056+0000 7fa0b7400640 1 -- 172.21.15.83:0/1176763574 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11944 (secure 0 0 0) 0x7fa0a809cf80 con 0x7fa0c0101f70 2024-04-24T08:51:24.057 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:51:24.057 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/358872257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":358872257},{"type":"v1","addr":"172.21.15.83:6835","nonce":358872257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3157070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3157070851},{"type":"v1","addr":"172.21.15.83:6837","nonce":3157070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T08:48:29.422576+0000","modified":"2024-04-24T08:49:04.217098+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34520,"mds_1":34493,"mds_2":34451,"mds_3":34427,"mds_4":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14658":{"gid":14658,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6839/4085470844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":4085470844},{"type":"v1","addr":"172.21.15.83:6839","nonce":4085470844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14676":{"gid":14676,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6841/2634754210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":2634754210},{"type":"v1","addr":"172.21.15.83:6841","nonce":2634754210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24382":{"gid":24382,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/1859921310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1859921310},{"type":"v1","addr":"172.21.15.189:6833","nonce":1859921310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/2241923197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2241923197},{"type":"v1","addr":"172.21.15.189:6835","nonce":2241923197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6837/3172311220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":3172311220},{"type":"v1","addr":"172.21.15.189:6837","nonce":3172311220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.189:6839/2548965336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":2548965336},{"type":"v1","addr":"172.21.15.189:6839","nonce":2548965336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.138:6833/1157967658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1157967658},{"type":"v1","addr":"172.21.15.138:6833","nonce":1157967658}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.138:6835/1909219758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1909219758},{"type":"v1","addr":"172.21.15.138:6835","nonce":1909219758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34493":{"gid":34493,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.138:6837/900566754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":900566754},{"type":"v1","addr":"172.21.15.138:6837","nonce":900566754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34520":{"gid":34520,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.138:6839/2236932393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2236932393},{"type":"v1","addr":"172.21.15.138:6839","nonce":2236932393}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34520,"qdb_cluster":[34427,24454,34520,34493,34451]},"id":1}]} 2024-04-24T08:51:24.057 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 27 2024-04-24T08:51:24.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.059+0000 7fa0bf200640 1 -- 172.21.15.83:0/1176763574 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fa0a40a3c30 msgr2=0x7fa0a40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:24.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.059+0000 7fa0bf200640 1 --2- 172.21.15.83:0/1176763574 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fa0a40a3c30 0x7fa0a40a60f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa0b0000f80 tx=0x7fa0b0005fa0 comp rx=0 tx=0).stop 2024-04-24T08:51:24.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.059+0000 7fa0bf200640 1 -- 172.21.15.83:0/1176763574 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa0c0101f70 msgr2=0x7fa0c019a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:24.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.059+0000 7fa0bf200640 1 --2- 172.21.15.83:0/1176763574 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa0c0101f70 0x7fa0c019a730 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fa0a800c920 tx=0x7fa0a800cde0 comp rx=0 tx=0).stop 2024-04-24T08:51:24.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.059+0000 7fa0bf200640 1 -- 172.21.15.83:0/1176763574 shutdown_connections 2024-04-24T08:51:24.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.059+0000 7fa0bf200640 1 --2- 172.21.15.83:0/1176763574 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa0c0102930 0x7fa0c01a17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:24.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.059+0000 7fa0bf200640 1 --2- 172.21.15.83:0/1176763574 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fa0a40a3c30 0x7fa0a40a60f0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:24.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.059+0000 7fa0bf200640 1 --2- 172.21.15.83:0/1176763574 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa0c0101f70 0x7fa0c019a730 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:24.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.059+0000 7fa0bf200640 1 --2- 172.21.15.83:0/1176763574 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa0c0100d80 0x7fa0c019a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:24.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.059+0000 7fa0bf200640 1 -- 172.21.15.83:0/1176763574 >> 172.21.15.83:0/1176763574 conn(0x7fa0c00fc2d0 msgr2=0x7fa0c00fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:24.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.059+0000 7fa0bf200640 1 -- 172.21.15.83:0/1176763574 shutdown_connections 2024-04-24T08:51:24.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.059+0000 7fa0bf200640 1 -- 172.21.15.83:0/1176763574 wait complete. 2024-04-24T08:51:24.079 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T08:51:24.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.280+0000 7fe76d000640 1 -- 172.21.15.83:0/1541063907 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe768102900 msgr2=0x7fe768104cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:24.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.280+0000 7fe76d000640 1 --2- 172.21.15.83:0/1541063907 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe768102900 0x7fe768104cf0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fe758009900 tx=0x7fe75802a3f0 comp rx=0 tx=0).stop 2024-04-24T08:51:24.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.280+0000 7fe76d000640 1 -- 172.21.15.83:0/1541063907 shutdown_connections 2024-04-24T08:51:24.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.280+0000 7fe76d000640 1 --2- 172.21.15.83:0/1541063907 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe768107b60 0x7fe768109fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:24.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.280+0000 7fe76d000640 1 --2- 172.21.15.83:0/1541063907 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe768105230 0x7fe768107620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:24.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.280+0000 7fe76d000640 1 --2- 172.21.15.83:0/1541063907 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe768102900 0x7fe768104cf0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:24.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.280+0000 7fe76d000640 1 -- 172.21.15.83:0/1541063907 >> 172.21.15.83:0/1541063907 conn(0x7fe7680fc2d0 msgr2=0x7fe7680fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:24.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.280+0000 7fe76d000640 1 -- 172.21.15.83:0/1541063907 shutdown_connections 2024-04-24T08:51:24.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.280+0000 7fe76d000640 1 -- 172.21.15.83:0/1541063907 wait complete. 2024-04-24T08:51:24.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.281+0000 7fe76d000640 1 Processor -- start 2024-04-24T08:51:24.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.281+0000 7fe76d000640 1 -- start start 2024-04-24T08:51:24.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.282+0000 7fe76d000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe768102900 0x7fe768102270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:24.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.282+0000 7fe76d000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe768105230 0x7fe7681008b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:24.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.282+0000 7fe76d000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe768107b60 0x7fe768100df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:24.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.282+0000 7fe76d000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe76810da70 con 0x7fe768107b60 2024-04-24T08:51:24.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.282+0000 7fe76d000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe76810d8f0 con 0x7fe768105230 2024-04-24T08:51:24.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.282+0000 7fe76d000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe76810dbf0 con 0x7fe768102900 2024-04-24T08:51:24.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.282+0000 7fe767e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe768107b60 0x7fe768100df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:24.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.282+0000 7fe767e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe768107b60 0x7fe768100df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41290/0 (socket says 172.21.15.83:41290) 2024-04-24T08:51:24.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.282+0000 7fe767e00640 1 -- 172.21.15.83:0/2189932798 learned_addr learned my addr 172.21.15.83:0/2189932798 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:51:24.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.282+0000 7fe766a00640 1 --2- 172.21.15.83:0/2189932798 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe768105230 0x7fe7681008b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:24.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.282+0000 7fe767400640 1 --2- 172.21.15.83:0/2189932798 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe768102900 0x7fe768102270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:24.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.283+0000 7fe767e00640 1 -- 172.21.15.83:0/2189932798 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe768102900 msgr2=0x7fe768102270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:24.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.283+0000 7fe767e00640 1 --2- 172.21.15.83:0/2189932798 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe768102900 0x7fe768102270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:24.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.283+0000 7fe767e00640 1 -- 172.21.15.83:0/2189932798 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe768105230 msgr2=0x7fe7681008b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:24.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.283+0000 7fe767e00640 1 --2- 172.21.15.83:0/2189932798 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe768105230 0x7fe7681008b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:24.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.283+0000 7fe767e00640 1 -- 172.21.15.83:0/2189932798 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe768101680 con 0x7fe768107b60 2024-04-24T08:51:24.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.283+0000 7fe767400640 1 --2- 172.21.15.83:0/2189932798 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe768102900 0x7fe768102270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:51:24.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.283+0000 7fe767e00640 1 --2- 172.21.15.83:0/2189932798 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe768107b60 0x7fe768100df0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fe76000b970 tx=0x7fe76000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:24.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.283+0000 7fe75fe00640 1 -- 172.21.15.83:0/2189932798 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe760019070 con 0x7fe768107b60 2024-04-24T08:51:24.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.283+0000 7fe75fe00640 1 -- 172.21.15.83:0/2189932798 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe760014440 con 0x7fe768107b60 2024-04-24T08:51:24.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.283+0000 7fe75fe00640 1 -- 172.21.15.83:0/2189932798 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7600075d0 con 0x7fe768107b60 2024-04-24T08:51:24.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.283+0000 7fe76d000640 1 -- 172.21.15.83:0/2189932798 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7681a21b0 con 0x7fe768107b60 2024-04-24T08:51:24.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.283+0000 7fe76d000640 1 -- 172.21.15.83:0/2189932798 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7681a2780 con 0x7fe768107b60 2024-04-24T08:51:24.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.284+0000 7fe76d000640 1 -- 172.21.15.83:0/2189932798 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7680fddc0 con 0x7fe768107b60 2024-04-24T08:51:24.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.286+0000 7fe75fe00640 1 -- 172.21.15.83:0/2189932798 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fe76001c070 con 0x7fe768107b60 2024-04-24T08:51:24.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.288+0000 7fe75fe00640 1 --2- 172.21.15.83:0/2189932798 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fe74c0a3c30 0x7fe74c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:24.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.288+0000 7fe75fe00640 1 -- 172.21.15.83:0/2189932798 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9237+0+0 (secure 0 0 0) 0x7fe7600cfe40 con 0x7fe768107b60 2024-04-24T08:51:24.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.288+0000 7fe767400640 1 --2- 172.21.15.83:0/2189932798 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fe74c0a3c30 0x7fe74c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:24.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.289+0000 7fe767400640 1 --2- 172.21.15.83:0/2189932798 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fe74c0a3c30 0x7fe74c0a60f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe75802a910 tx=0x7fe758005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:24.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.293+0000 7fe75fe00640 1 -- 172.21.15.83:0/2189932798 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe76009c3d0 con 0x7fe768107b60 2024-04-24T08:51:24.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:24 smithi083 ceph-mon[27907]: pgmap v464: 129 pgs: 129 active+clean; 644 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-24T08:51:24.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:24 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/3076693377' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T08:51:24.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:24 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/1927083727' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T08:51:24.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:24 smithi138 ceph-mon[31441]: pgmap v464: 129 pgs: 129 active+clean; 644 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-24T08:51:24.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:24 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/3076693377' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T08:51:24.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:24 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/1927083727' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T08:51:24.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:24 smithi189 ceph-mon[31791]: pgmap v464: 129 pgs: 129 active+clean; 644 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-24T08:51:24.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:24 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/3076693377' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T08:51:24.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:24 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/1927083727' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T08:51:24.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.498+0000 7fe76d000640 1 -- 172.21.15.83:0/2189932798 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe768062c00 con 0x7fe768107b60 2024-04-24T08:51:24.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.502+0000 7fe75fe00640 1 -- 172.21.15.83:0/2189932798 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+24887 (secure 0 0 0) 0x7fe76009bcd0 con 0x7fe768107b60 2024-04-24T08:51:24.502 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:51:24.503 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":93,"fsid":"bd3e039e-0215-11ef-bc93-c7b262605968","created":"2024-04-24T08:37:32.209944+0000","modified":"2024-04-24T08:50:42.968900+0000","last_up_change":"2024-04-24T08:47:17.966041+0000","last_in_change":"2024-04-24T08:46:53.835936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T08:41:56.834372+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T08:48:26.978311+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T08:48:28.352731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T08:48:32.275167+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2cd77f3a-19dd-4c6e-a199-1dcbd5faa679","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2132675555},{"type":"v1","addr":"172.21.15.83:6803","nonce":2132675555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2132675555},{"type":"v1","addr":"172.21.15.83:6805","nonce":2132675555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2132675555},{"type":"v1","addr":"172.21.15.83:6809","nonce":2132675555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2132675555},{"type":"v1","addr":"172.21.15.83:6807","nonce":2132675555}]},"public_addr":"172.21.15.83:6803/2132675555","cluster_addr":"172.21.15.83:6805/2132675555","heartbeat_back_addr":"172.21.15.83:6809/2132675555","heartbeat_front_addr":"172.21.15.83:6807/2132675555","state":["exists","up"]},{"osd":1,"uuid":"aba7f5df-2a4c-4c23-b502-d193605a5b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":3589158880},{"type":"v1","addr":"172.21.15.138:6801","nonce":3589158880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":3589158880},{"type":"v1","addr":"172.21.15.138:6803","nonce":3589158880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":3589158880},{"type":"v1","addr":"172.21.15.138:6807","nonce":3589158880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":3589158880},{"type":"v1","addr":"172.21.15.138:6805","nonce":3589158880}]},"public_addr":"172.21.15.138:6801/3589158880","cluster_addr":"172.21.15.138:6803/3589158880","heartbeat_back_addr":"172.21.15.138:6807/3589158880","heartbeat_front_addr":"172.21.15.138:6805/3589158880","state":["exists","up"]},{"osd":2,"uuid":"071bc167-2333-4420-bb58-bffbc10d7641","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":3169072688},{"type":"v1","addr":"172.21.15.189:6801","nonce":3169072688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":3169072688},{"type":"v1","addr":"172.21.15.189:6803","nonce":3169072688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":3169072688},{"type":"v1","addr":"172.21.15.189:6807","nonce":3169072688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":3169072688},{"type":"v1","addr":"172.21.15.189:6805","nonce":3169072688}]},"public_addr":"172.21.15.189:6801/3169072688","cluster_addr":"172.21.15.189:6803/3169072688","heartbeat_back_addr":"172.21.15.189:6807/3169072688","heartbeat_front_addr":"172.21.15.189:6805/3169072688","state":["exists","up"]},{"osd":3,"uuid":"f5624d78-4455-4e07-8c1a-f47c94cddb23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3131092450},{"type":"v1","addr":"172.21.15.83:6811","nonce":3131092450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3131092450},{"type":"v1","addr":"172.21.15.83:6813","nonce":3131092450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3131092450},{"type":"v1","addr":"172.21.15.83:6817","nonce":3131092450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3131092450},{"type":"v1","addr":"172.21.15.83:6815","nonce":3131092450}]},"public_addr":"172.21.15.83:6811/3131092450","cluster_addr":"172.21.15.83:6813/3131092450","heartbeat_back_addr":"172.21.15.83:6817/3131092450","heartbeat_front_addr":"172.21.15.83:6815/3131092450","state":["exists","up"]},{"osd":4,"uuid":"7b50cd14-9385-4535-a7b0-1c1da93c8548","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":823775360},{"type":"v1","addr":"172.21.15.138:6809","nonce":823775360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":823775360},{"type":"v1","addr":"172.21.15.138:6811","nonce":823775360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":823775360},{"type":"v1","addr":"172.21.15.138:6815","nonce":823775360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":823775360},{"type":"v1","addr":"172.21.15.138:6813","nonce":823775360}]},"public_addr":"172.21.15.138:6809/823775360","cluster_addr":"172.21.15.138:6811/823775360","heartbeat_back_addr":"172.21.15.138:6815/823775360","heartbeat_front_addr":"172.21.15.138:6813/823775360","state":["exists","up"]},{"osd":5,"uuid":"9c647797-35bc-449f-be84-af97d2c69c2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":1399817431},{"type":"v1","addr":"172.21.15.189:6809","nonce":1399817431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":1399817431},{"type":"v1","addr":"172.21.15.189:6811","nonce":1399817431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":1399817431},{"type":"v1","addr":"172.21.15.189:6815","nonce":1399817431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":1399817431},{"type":"v1","addr":"172.21.15.189:6813","nonce":1399817431}]},"public_addr":"172.21.15.189:6809/1399817431","cluster_addr":"172.21.15.189:6811/1399817431","heartbeat_back_addr":"172.21.15.189:6815/1399817431","heartbeat_front_addr":"172.21.15.189:6813/1399817431","state":["exists","up"]},{"osd":6,"uuid":"8f056584-16c8-4454-b445-3f780f26c196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":1611358483},{"type":"v1","addr":"172.21.15.83:6819","nonce":1611358483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1611358483},{"type":"v1","addr":"172.21.15.83:6821","nonce":1611358483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1611358483},{"type":"v1","addr":"172.21.15.83:6825","nonce":1611358483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":1611358483},{"type":"v1","addr":"172.21.15.83:6823","nonce":1611358483}]},"public_addr":"172.21.15.83:6819/1611358483","cluster_addr":"172.21.15.83:6821/1611358483","heartbeat_back_addr":"172.21.15.83:6825/1611358483","heartbeat_front_addr":"172.21.15.83:6823/1611358483","state":["exists","up"]},{"osd":7,"uuid":"7603efe3-da77-4b7f-9d03-29c4f293b6d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":3557856706},{"type":"v1","addr":"172.21.15.138:6817","nonce":3557856706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":3557856706},{"type":"v1","addr":"172.21.15.138:6819","nonce":3557856706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":3557856706},{"type":"v1","addr":"172.21.15.138:6823","nonce":3557856706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":3557856706},{"type":"v1","addr":"172.21.15.138:6821","nonce":3557856706}]},"public_addr":"172.21.15.138:6817/3557856706","cluster_addr":"172.21.15.138:6819/3557856706","heartbeat_back_addr":"172.21.15.138:6823/3557856706","heartbeat_front_addr":"172.21.15.138:6821/3557856706","state":["exists","up"]},{"osd":8,"uuid":"27d7d1b8-d6a0-417e-b8aa-596325aa1a77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":207018554},{"type":"v1","addr":"172.21.15.189:6817","nonce":207018554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":207018554},{"type":"v1","addr":"172.21.15.189:6819","nonce":207018554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":207018554},{"type":"v1","addr":"172.21.15.189:6823","nonce":207018554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":207018554},{"type":"v1","addr":"172.21.15.189:6821","nonce":207018554}]},"public_addr":"172.21.15.189:6817/207018554","cluster_addr":"172.21.15.189:6819/207018554","heartbeat_back_addr":"172.21.15.189:6823/207018554","heartbeat_front_addr":"172.21.15.189:6821/207018554","state":["exists","up"]},{"osd":9,"uuid":"baf19b98-881b-4d3a-9f72-8aa2dd73cad6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":519636751},{"type":"v1","addr":"172.21.15.83:6827","nonce":519636751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":519636751},{"type":"v1","addr":"172.21.15.83:6829","nonce":519636751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":519636751},{"type":"v1","addr":"172.21.15.83:6833","nonce":519636751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":519636751},{"type":"v1","addr":"172.21.15.83:6831","nonce":519636751}]},"public_addr":"172.21.15.83:6827/519636751","cluster_addr":"172.21.15.83:6829/519636751","heartbeat_back_addr":"172.21.15.83:6833/519636751","heartbeat_front_addr":"172.21.15.83:6831/519636751","state":["exists","up"]},{"osd":10,"uuid":"eb346717-5c07-47c8-95be-0e68fe449fdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1130872098},{"type":"v1","addr":"172.21.15.138:6825","nonce":1130872098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1130872098},{"type":"v1","addr":"172.21.15.138:6827","nonce":1130872098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1130872098},{"type":"v1","addr":"172.21.15.138:6831","nonce":1130872098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1130872098},{"type":"v1","addr":"172.21.15.138:6829","nonce":1130872098}]},"public_addr":"172.21.15.138:6825/1130872098","cluster_addr":"172.21.15.138:6827/1130872098","heartbeat_back_addr":"172.21.15.138:6831/1130872098","heartbeat_front_addr":"172.21.15.138:6829/1130872098","state":["exists","up"]},{"osd":11,"uuid":"8e37568e-981c-45b8-b2af-2f1b73898542","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":2998782690},{"type":"v1","addr":"172.21.15.189:6825","nonce":2998782690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":2998782690},{"type":"v1","addr":"172.21.15.189:6827","nonce":2998782690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":2998782690},{"type":"v1","addr":"172.21.15.189:6831","nonce":2998782690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":2998782690},{"type":"v1","addr":"172.21.15.189:6829","nonce":2998782690}]},"public_addr":"172.21.15.189:6825/2998782690","cluster_addr":"172.21.15.189:6827/2998782690","heartbeat_back_addr":"172.21.15.189:6831/2998782690","heartbeat_front_addr":"172.21.15.189:6829/2998782690","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:40:47.980103+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:41:21.970795+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:41:55.160426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:42:30.400475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:43:05.705663+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:43:40.112312+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:44:16.781289+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:44:52.151187+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:45:28.055944+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:46:04.969301+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:46:40.918306+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:47:16.886874+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"2.1d","mappings":[{"from":2,"to":1}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.2a","mappings":[{"from":3,"to":8}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.a","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:6801/820049883":"2024-04-25T08:38:40.661315+0000","172.21.15.83:0/4272874138":"2024-04-25T08:38:40.661315+0000","172.21.15.83:0/4212994335":"2024-04-25T08:38:40.661315+0000","172.21.15.83:6801/1996371091":"2024-04-25T08:38:05.800856+0000","172.21.15.83:0/1820663683":"2024-04-25T08:38:05.800856+0000","172.21.15.83:0/3802897262":"2024-04-25T08:38:05.800856+0000","172.21.15.83:6800/1996371091":"2024-04-25T08:38:05.800856+0000","172.21.15.83:6800/820049883":"2024-04-25T08:38:40.661315+0000","172.21.15.83:0/1366125049":"2024-04-25T08:38:05.800856+0000","172.21.15.83:0/340452868":"2024-04-25T08:38:40.661315+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T08:51:24.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.505+0000 7fe76d000640 1 -- 172.21.15.83:0/2189932798 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fe74c0a3c30 msgr2=0x7fe74c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:24.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.505+0000 7fe76d000640 1 --2- 172.21.15.83:0/2189932798 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fe74c0a3c30 0x7fe74c0a60f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe75802a910 tx=0x7fe758005fd0 comp rx=0 tx=0).stop 2024-04-24T08:51:24.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.505+0000 7fe76d000640 1 -- 172.21.15.83:0/2189932798 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe768107b60 msgr2=0x7fe768100df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:24.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.505+0000 7fe76d000640 1 --2- 172.21.15.83:0/2189932798 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe768107b60 0x7fe768100df0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fe76000b970 tx=0x7fe76000be30 comp rx=0 tx=0).stop 2024-04-24T08:51:24.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.505+0000 7fe76d000640 1 -- 172.21.15.83:0/2189932798 shutdown_connections 2024-04-24T08:51:24.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.505+0000 7fe76d000640 1 --2- 172.21.15.83:0/2189932798 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fe74c0a3c30 0x7fe74c0a60f0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:24.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.505+0000 7fe76d000640 1 --2- 172.21.15.83:0/2189932798 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe768107b60 0x7fe768100df0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:24.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.505+0000 7fe76d000640 1 --2- 172.21.15.83:0/2189932798 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe768105230 0x7fe7681008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:24.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.505+0000 7fe76d000640 1 --2- 172.21.15.83:0/2189932798 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe768102900 0x7fe768102270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:24.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.505+0000 7fe76d000640 1 -- 172.21.15.83:0/2189932798 >> 172.21.15.83:0/2189932798 conn(0x7fe7680fc2d0 msgr2=0x7fe768103260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:24.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.505+0000 7fe76d000640 1 -- 172.21.15.83:0/2189932798 shutdown_connections 2024-04-24T08:51:24.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.505+0000 7fe76d000640 1 -- 172.21.15.83:0/2189932798 wait complete. 2024-04-24T08:51:24.526 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-24T08:48:29.422576+0000', 'modified': '2024-04-24T08:49:04.217098+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 34520, 'mds_1': 34493, 'mds_2': 34451, 'mds_3': 34427, 'mds_4': 24454}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14658': {'gid': 14658, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.83:6839/4085470844', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6838', 'nonce': 4085470844}, {'type': 'v1', 'addr': '172.21.15.83:6839', 'nonce': 4085470844}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14676': {'gid': 14676, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.83:6841/2634754210', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6840', 'nonce': 2634754210}, {'type': 'v1', 'addr': '172.21.15.83:6841', 'nonce': 2634754210}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24382': {'gid': 24382, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.189:6833/1859921310', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6832', 'nonce': 1859921310}, {'type': 'v1', 'addr': '172.21.15.189:6833', 'nonce': 1859921310}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24421': {'gid': 24421, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.189:6835/2241923197', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6834', 'nonce': 2241923197}, {'type': 'v1', 'addr': '172.21.15.189:6835', 'nonce': 2241923197}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24439': {'gid': 24439, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.189:6837/3172311220', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6836', 'nonce': 3172311220}, {'type': 'v1', 'addr': '172.21.15.189:6837', 'nonce': 3172311220}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24454': {'gid': 24454, 'name': 'i', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 15, 'addr': '172.21.15.189:6839/2548965336', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6838', 'nonce': 2548965336}, {'type': 'v1', 'addr': '172.21.15.189:6839', 'nonce': 2548965336}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34427': {'gid': 34427, 'name': 'k', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 22, 'addr': '172.21.15.138:6833/1157967658', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6832', 'nonce': 1157967658}, {'type': 'v1', 'addr': '172.21.15.138:6833', 'nonce': 1157967658}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34451': {'gid': 34451, 'name': 'e', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 19, 'addr': '172.21.15.138:6835/1909219758', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6834', 'nonce': 1909219758}, {'type': 'v1', 'addr': '172.21.15.138:6835', 'nonce': 1909219758}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34493': {'gid': 34493, 'name': 'h', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 15, 'addr': '172.21.15.138:6837/900566754', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6836', 'nonce': 900566754}, {'type': 'v1', 'addr': '172.21.15.138:6837', 'nonce': 900566754}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34520': {'gid': 34520, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.138:6839/2236932393', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6838', 'nonce': 2236932393}, {'type': 'v1', 'addr': '172.21.15.138:6839', 'nonce': 2236932393}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 34520, 'qdb_cluster': [34427, 24454, 34520, 34493, 34451]} 2024-04-24T08:51:24.526 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-04-24T08:51:24.526 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-24T08:51:24.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.728+0000 7f302b200640 1 -- 172.21.15.83:0/2098212488 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f302c107bf0 msgr2=0x7f302c10a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:24.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.728+0000 7f302b200640 1 --2- 172.21.15.83:0/2098212488 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f302c107bf0 0x7f302c10a040 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f3024009ca0 tx=0x7f302402a8c0 comp rx=0 tx=0).stop 2024-04-24T08:51:24.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.728+0000 7f302b200640 1 -- 172.21.15.83:0/2098212488 shutdown_connections 2024-04-24T08:51:24.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.728+0000 7f302b200640 1 --2- 172.21.15.83:0/2098212488 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f302c107bf0 0x7f302c10a040 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:24.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.728+0000 7f302b200640 1 --2- 172.21.15.83:0/2098212488 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f302c1052c0 0x7f302c1076b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:24.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.728+0000 7f302b200640 1 --2- 172.21.15.83:0/2098212488 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f302c102990 0x7f302c104d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:24.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.728+0000 7f302b200640 1 -- 172.21.15.83:0/2098212488 >> 172.21.15.83:0/2098212488 conn(0x7f302c0fc3a0 msgr2=0x7f302c0fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:24.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.728+0000 7f302b200640 1 -- 172.21.15.83:0/2098212488 shutdown_connections 2024-04-24T08:51:24.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.728+0000 7f302b200640 1 -- 172.21.15.83:0/2098212488 wait complete. 2024-04-24T08:51:24.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.729+0000 7f302b200640 1 Processor -- start 2024-04-24T08:51:24.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.729+0000 7f302b200640 1 -- start start 2024-04-24T08:51:24.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.730+0000 7f302b200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f302c102990 0x7f302c19a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:24.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.730+0000 7f302b200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f302c1052c0 0x7f302c19a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:24.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.730+0000 7f302b200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f302c107bf0 0x7f302c1a18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:24.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.730+0000 7f302b200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f302c10daa0 con 0x7f302c1052c0 2024-04-24T08:51:24.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.730+0000 7f302b200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f302c10d920 con 0x7f302c102990 2024-04-24T08:51:24.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.730+0000 7f302b200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f302c10dc20 con 0x7f302c107bf0 2024-04-24T08:51:24.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.730+0000 7f3029400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f302c1052c0 0x7f302c19a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:24.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.730+0000 7f3029400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f302c1052c0 0x7f302c19a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41306/0 (socket says 172.21.15.83:41306) 2024-04-24T08:51:24.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.730+0000 7f3029400640 1 -- 172.21.15.83:0/3211025694 learned_addr learned my addr 172.21.15.83:0/3211025694 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:51:24.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.730+0000 7f3029e00640 1 --2- 172.21.15.83:0/3211025694 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f302c102990 0x7f302c19a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:24.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.730+0000 7f3029400640 1 -- 172.21.15.83:0/3211025694 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f302c107bf0 msgr2=0x7f302c1a18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:24.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.730+0000 7f3029400640 1 --2- 172.21.15.83:0/3211025694 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f302c107bf0 0x7f302c1a18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:24.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.730+0000 7f3029400640 1 -- 172.21.15.83:0/3211025694 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f302c102990 msgr2=0x7f302c19a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:24.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.730+0000 7f3029400640 1 --2- 172.21.15.83:0/3211025694 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f302c102990 0x7f302c19a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:24.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.730+0000 7f3029400640 1 -- 172.21.15.83:0/3211025694 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f302c1a1f10 con 0x7f302c1052c0 2024-04-24T08:51:24.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.730+0000 7f3029e00640 1 --2- 172.21.15.83:0/3211025694 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f302c102990 0x7f302c19a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:51:24.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.731+0000 7f3029400640 1 --2- 172.21.15.83:0/3211025694 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f302c1052c0 0x7f302c19a830 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f301800d770 tx=0x7f301800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:24.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.731+0000 7f3022a00640 1 -- 172.21.15.83:0/3211025694 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3018017070 con 0x7f302c1052c0 2024-04-24T08:51:24.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.731+0000 7f3022a00640 1 -- 172.21.15.83:0/3211025694 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3018009c30 con 0x7f302c1052c0 2024-04-24T08:51:24.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.731+0000 7f3022a00640 1 -- 172.21.15.83:0/3211025694 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f301800f620 con 0x7f302c1052c0 2024-04-24T08:51:24.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.731+0000 7f302b200640 1 -- 172.21.15.83:0/3211025694 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f302c1a2200 con 0x7f302c1052c0 2024-04-24T08:51:24.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.731+0000 7f302b200640 1 -- 172.21.15.83:0/3211025694 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f302c1a2750 con 0x7f302c1052c0 2024-04-24T08:51:24.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.734+0000 7f3033e30740 1 Processor -- start 2024-04-24T08:51:24.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.735+0000 7f3033e30740 1 -- start start 2024-04-24T08:51:24.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.735+0000 7f3033e30740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7a8a11260 0x55a7a8a11680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:24.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.735+0000 7f3022a00640 1 -- 172.21.15.83:0/3211025694 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f301801a070 con 0x7f302c1052c0 2024-04-24T08:51:24.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.735+0000 7f3033e30740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a7a8a11c50 0x55a7a8a16750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:24.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.735+0000 7f3033e30740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7a8a16dc0 0x55a7a8a19250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:24.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.735+0000 7f3033e30740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a7a8976b10 con 0x55a7a8a16dc0 2024-04-24T08:51:24.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.735+0000 7f3033e30740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a7a891d2b0 con 0x55a7a8a11260 2024-04-24T08:51:24.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.735+0000 7f3033e30740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a7a8972ad0 con 0x55a7a8a11c50 2024-04-24T08:51:24.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.735+0000 7f3029400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7a8a16dc0 0x55a7a8a19250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:24.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.735+0000 7f3029400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7a8a16dc0 0x55a7a8a19250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41310/0 (socket says 172.21.15.83:41310) 2024-04-24T08:51:24.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.735+0000 7f3029400640 1 -- 172.21.15.83:0/4225701781 learned_addr learned my addr 172.21.15.83:0/4225701781 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:51:24.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.736+0000 7f3029e00640 1 --2- 172.21.15.83:0/4225701781 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a7a8a11c50 0x55a7a8a16750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:24.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.736+0000 7f302a800640 1 --2- 172.21.15.83:0/4225701781 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7a8a11260 0x55a7a8a11680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:24.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.736+0000 7f3029400640 1 -- 172.21.15.83:0/4225701781 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a7a8a11c50 msgr2=0x55a7a8a16750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:51:24.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.736+0000 7f3029400640 1 --2- 172.21.15.83:0/4225701781 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a7a8a11c50 0x55a7a8a16750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:24.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.736+0000 7f3029400640 1 -- 172.21.15.83:0/4225701781 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7a8a11260 msgr2=0x55a7a8a11680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:51:24.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.736+0000 7f3029400640 1 --2- 172.21.15.83:0/4225701781 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7a8a11260 0x55a7a8a11680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:24.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.736+0000 7f3029400640 1 -- 172.21.15.83:0/4225701781 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7a86892d0 con 0x55a7a8a16dc0 2024-04-24T08:51:24.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.736+0000 7f3029400640 1 --2- 172.21.15.83:0/4225701781 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7a8a16dc0 0x55a7a8a19250 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f30180d65f0 tx=0x7f30180d6ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9333dc4b16ba7137 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:24.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.736+0000 7f3017400640 1 -- 172.21.15.83:0/4225701781 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30180e3070 con 0x55a7a8a16dc0 2024-04-24T08:51:24.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.737+0000 7f3017400640 1 -- 172.21.15.83:0/4225701781 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f30180d0cb0 con 0x55a7a8a16dc0 2024-04-24T08:51:24.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.737+0000 7f3017400640 1 -- 172.21.15.83:0/4225701781 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30180dea00 con 0x55a7a8a16dc0 2024-04-24T08:51:24.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.737+0000 7f3022a00640 1 --2- 172.21.15.83:0/3211025694 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f30100a3bb0 0x7f30100a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:24.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.737+0000 7f3022a00640 1 -- 172.21.15.83:0/3211025694 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9237+0+0 (secure 0 0 0) 0x7f30180d0410 con 0x7f302c1052c0 2024-04-24T08:51:24.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.737+0000 7f302a800640 1 --2- 172.21.15.83:0/3211025694 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f30100a3bb0 0x7f30100a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:24.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.737+0000 7f3033e30740 1 -- 172.21.15.83:0/4225701781 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7a8a16dc0 msgr2=0x55a7a8a19250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:24.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.737+0000 7f3033e30740 1 --2- 172.21.15.83:0/4225701781 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7a8a16dc0 0x55a7a8a19250 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f30180d65f0 tx=0x7f30180d6ab0 comp rx=0 tx=0).stop 2024-04-24T08:51:24.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.737+0000 7f3033e30740 1 -- 172.21.15.83:0/4225701781 shutdown_connections 2024-04-24T08:51:24.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.737+0000 7f3033e30740 1 --2- 172.21.15.83:0/4225701781 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7a8a16dc0 0x55a7a8a19250 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:24.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.737+0000 7f3033e30740 1 --2- 172.21.15.83:0/4225701781 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a7a8a11c50 0x55a7a8a16750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:24.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.737+0000 7f3033e30740 1 --2- 172.21.15.83:0/4225701781 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7a8a11260 0x55a7a8a11680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:24.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.737+0000 7f3033e30740 1 -- 172.21.15.83:0/4225701781 >> 172.21.15.83:0/4225701781 conn(0x55a7a8a0c8d0 msgr2=0x55a7a8a0ccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:24.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.737+0000 7f3033e30740 1 -- 172.21.15.83:0/4225701781 shutdown_connections 2024-04-24T08:51:24.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.738+0000 7f3033e30740 1 -- 172.21.15.83:0/4225701781 wait complete. 2024-04-24T08:51:24.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.738+0000 7f302a800640 1 --2- 172.21.15.83:0/3211025694 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f30100a3bb0 0x7f30100a6070 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f3024031aa0 tx=0x7f3024005d80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:24.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.738+0000 7f3033e30740 1 -- start start 2024-04-24T08:51:24.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.738+0000 7f3033e30740 1 Processor -- start 2024-04-24T08:51:24.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.739+0000 7f3033e30740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7a8a11260 0x55a7a8a1b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:24.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.739+0000 7f3033e30740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a7a8a11c50 0x55a7a8a1bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:24.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.739+0000 7f3033e30740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7a8a16dc0 0x55a7a8a22c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:24.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.739+0000 7f3033e30740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a7a8a19d70 con 0x55a7a8a11260 2024-04-24T08:51:24.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.739+0000 7f3033e30740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a7a8930590 con 0x55a7a8a16dc0 2024-04-24T08:51:24.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.739+0000 7f3033e30740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a7a8a19ef0 con 0x55a7a8a11c50 2024-04-24T08:51:24.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.739+0000 7f3033e30740 10 client.0 _ll_register_callbacks cb 0x55a7a89133a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:51:24.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.739+0000 7f3029e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7a8a11260 0x55a7a8a1b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:24.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.739+0000 7f3029e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7a8a11260 0x55a7a8a1b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41320/0 (socket says 172.21.15.83:41320) 2024-04-24T08:51:24.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.739+0000 7f3029e00640 1 -- 172.21.15.83:0/3147813260 learned_addr learned my addr 172.21.15.83:0/3147813260 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:51:24.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.739+0000 7f302a800640 1 --2- 172.21.15.83:0/3147813260 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7a8a16dc0 0x55a7a8a22c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:24.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.739+0000 7f3029e00640 1 -- 172.21.15.83:0/3147813260 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a7a8a11c50 msgr2=0x55a7a8a1bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:51:24.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.739+0000 7f3029e00640 1 --2- 172.21.15.83:0/3147813260 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a7a8a11c50 0x55a7a8a1bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:24.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.739+0000 7f3029e00640 1 -- 172.21.15.83:0/3147813260 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7a8a16dc0 msgr2=0x55a7a8a22c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:51:24.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.739+0000 7f3029e00640 1 --2- 172.21.15.83:0/3147813260 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7a8a16dc0 0x55a7a8a22c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:24.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.739+0000 7f3029e00640 1 -- 172.21.15.83:0/3147813260 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7a87536b0 con 0x55a7a8a11260 2024-04-24T08:51:24.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.739+0000 7f302a800640 1 --2- 172.21.15.83:0/3147813260 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7a8a16dc0 0x55a7a8a22c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:51:24.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.740+0000 7f3029e00640 1 --2- 172.21.15.83:0/3147813260 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7a8a11260 0x55a7a8a1b6e0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f301c010670 tx=0x7f301c010b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cbaf4fe796a72c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:24.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.740+0000 7f300be00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:51:24.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.740+0000 7f300be00640 1 -- 172.21.15.83:0/3147813260 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f301c00ea70 con 0x55a7a8a11260 2024-04-24T08:51:24.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.740+0000 7f300be00640 1 -- 172.21.15.83:0/3147813260 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9237+0+0 (secure 0 0 0) 0x7f301c00ec10 con 0x55a7a8a11260 2024-04-24T08:51:24.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.740+0000 7f3033e30740 1 -- 172.21.15.83:0/3147813260 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30180d0cb0 con 0x55a7a8a11260 2024-04-24T08:51:24.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.740+0000 7f300be00640 1 -- 172.21.15.83:0/3147813260 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f301c025050 con 0x55a7a8a11260 2024-04-24T08:51:24.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.740+0000 7f3033e30740 10 client.15477 fetch_fsmap learned FSMap version 27 2024-04-24T08:51:24.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.740+0000 7f3033e30740 1 -- 172.21.15.83:0/3147813260 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a7a8a260a0 con 0x55a7a8a11260 2024-04-24T08:51:24.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.740+0000 7f300be00640 1 -- 172.21.15.83:0/3147813260 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f30180d0cb0 con 0x55a7a8a11260 2024-04-24T08:51:24.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.741+0000 7f300be00640 1 -- 172.21.15.83:0/3147813260 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f301c00f040 con 0x55a7a8a11260 2024-04-24T08:51:24.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.741+0000 7f3033e30740 10 client.15477 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T08:51:24.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.741+0000 7f3033e30740 10 client.15477 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-24T08:51:24.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.741+0000 7f3033e30740 20 client.15477 populate_metadata read hostname 'smithi083' 2024-04-24T08:51:24.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.741+0000 7f3033e30740 1 --2- 172.21.15.83:0/3147813260 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55a7a8a287c0 0x55a7a8a29c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:24.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.741+0000 7f3033e30740 4 client.15477 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:51:24.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.741+0000 7f3033e30740 1 -- 172.21.15.83:0/3147813260 --> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a7a8984530 con 0x55a7a8a287c0 2024-04-24T08:51:24.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.741+0000 7f3029400640 1 --2- 172.21.15.83:0/3147813260 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55a7a8a287c0 0x55a7a8a29c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:24.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.742+0000 7f3029400640 1 --2- 172.21.15.83:0/3147813260 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55a7a8a287c0 0x55a7a8a29c30 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a6453745fb4d83fa server_cookie=cd2342c6576043ca in_seq=0 out_seq=0 2024-04-24T08:51:24.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.742+0000 7f300be00640 10 client.15477 ms_handle_connect on v2:172.21.15.189:6838/2548965336 2024-04-24T08:51:24.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.744+0000 7f300be00640 1 -- 172.21.15.83:0/3147813260 <== mds.4 v2:172.21.15.189:6838/2548965336 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a7a8984530 con 0x55a7a8a287c0 2024-04-24T08:51:24.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.744+0000 7f300be00640 10 client.15477 handle_command_reply: tid=0 2024-04-24T08:51:24.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.744+0000 7f3033e30740 1 client.15477 shutdown 2024-04-24T08:51:24.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.745+0000 7f3033e30740 1 -- 172.21.15.83:0/3147813260 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7a8a11260 msgr2=0x55a7a8a1b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:24.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.745+0000 7f3033e30740 1 --2- 172.21.15.83:0/3147813260 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7a8a11260 0x55a7a8a1b6e0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f301c010670 tx=0x7f301c010b30 comp rx=0 tx=0).stop 2024-04-24T08:51:24.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.745+0000 7f3033e30740 1 -- 172.21.15.83:0/3147813260 shutdown_connections 2024-04-24T08:51:24.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.745+0000 7f3033e30740 1 --2- 172.21.15.83:0/3147813260 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55a7a8a287c0 0x55a7a8a29c30 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:24.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.745+0000 7f3033e30740 1 --2- 172.21.15.83:0/3147813260 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7a8a16dc0 0x55a7a8a22c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:24.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.745+0000 7f3033e30740 1 --2- 172.21.15.83:0/3147813260 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a7a8a11c50 0x55a7a8a1bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:24.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.745+0000 7f3033e30740 1 --2- 172.21.15.83:0/3147813260 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7a8a11260 0x55a7a8a1b6e0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:24.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.745+0000 7f300be00640 0 client.15477 ms_handle_reset on v2:172.21.15.189:6838/2548965336 2024-04-24T08:51:24.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.745+0000 7f3033e30740 1 -- 172.21.15.83:0/3147813260 >> 172.21.15.83:0/3147813260 conn(0x55a7a8a0c8d0 msgr2=0x55a7a8a0b970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:24.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.745+0000 7f3033e30740 1 -- 172.21.15.83:0/3147813260 shutdown_connections 2024-04-24T08:51:24.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.745+0000 7f3033e30740 1 -- 172.21.15.83:0/3147813260 wait complete. 2024-04-24T08:51:24.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.745+0000 7f3033e30740 20 client.15477 trim_cache size 0 max 16384 2024-04-24T08:51:24.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.765+0000 7f3033e30740 1 Processor -- start 2024-04-24T08:51:24.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.765+0000 7f3033e30740 1 -- start start 2024-04-24T08:51:24.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.765+0000 7f3033e30740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7a8a11260 0x55a7a8a0b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:24.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.765+0000 7f3033e30740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7a8a11c50 0x55a7a8a22c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:24.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.765+0000 7f3033e30740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a7a8a16dc0 0x55a7a8a0b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:24.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.765+0000 7f3033e30740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a7a8a27f70 con 0x55a7a8a11c50 2024-04-24T08:51:24.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.765+0000 7f3033e30740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a7a8a28640 con 0x55a7a8a11260 2024-04-24T08:51:24.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.765+0000 7f3033e30740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a7a8a28d10 con 0x55a7a8a16dc0 2024-04-24T08:51:24.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.765+0000 7f3029400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7a8a11c50 0x55a7a8a22c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:24.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.765+0000 7f3029400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7a8a11c50 0x55a7a8a22c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41330/0 (socket says 172.21.15.83:41330) 2024-04-24T08:51:24.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.765+0000 7f3029400640 1 -- 172.21.15.83:0/2096039997 learned_addr learned my addr 172.21.15.83:0/2096039997 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:51:24.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.766+0000 7f3029e00640 1 --2- 172.21.15.83:0/2096039997 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7a8a11260 0x55a7a8a0b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:24.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.766+0000 7f302a800640 1 --2- 172.21.15.83:0/2096039997 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a7a8a16dc0 0x55a7a8a0b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:24.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.766+0000 7f3029400640 1 -- 172.21.15.83:0/2096039997 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a7a8a16dc0 msgr2=0x55a7a8a0b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:51:24.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.766+0000 7f3029400640 1 --2- 172.21.15.83:0/2096039997 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a7a8a16dc0 0x55a7a8a0b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:24.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.766+0000 7f3029400640 1 -- 172.21.15.83:0/2096039997 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7a8a11260 msgr2=0x55a7a8a0b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:51:24.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.766+0000 7f3029400640 1 --2- 172.21.15.83:0/2096039997 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7a8a11260 0x55a7a8a0b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:24.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.766+0000 7f3029400640 1 -- 172.21.15.83:0/2096039997 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7a8a0d5a0 con 0x55a7a8a11c50 2024-04-24T08:51:24.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.766+0000 7f3029400640 1 --2- 172.21.15.83:0/2096039997 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7a8a11c50 0x55a7a8a22c20 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f301800d740 tx=0x7f3018010c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29b07b60dc26fb1a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:24.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.767+0000 7f3016a00640 1 -- 172.21.15.83:0/2096039997 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3018021040 con 0x55a7a8a11c50 2024-04-24T08:51:24.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.767+0000 7f3016a00640 1 -- 172.21.15.83:0/2096039997 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3018009de0 con 0x55a7a8a11c50 2024-04-24T08:51:24.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.767+0000 7f3016a00640 1 -- 172.21.15.83:0/2096039997 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f301809c2a0 con 0x55a7a8a11c50 2024-04-24T08:51:24.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.767+0000 7f3033e30740 1 -- 172.21.15.83:0/2096039997 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7a8a11c50 msgr2=0x55a7a8a22c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:24.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.767+0000 7f3033e30740 1 --2- 172.21.15.83:0/2096039997 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7a8a11c50 0x55a7a8a22c20 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f301800d740 tx=0x7f3018010c10 comp rx=0 tx=0).stop 2024-04-24T08:51:24.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.767+0000 7f3033e30740 1 -- 172.21.15.83:0/2096039997 shutdown_connections 2024-04-24T08:51:24.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.767+0000 7f3033e30740 1 --2- 172.21.15.83:0/2096039997 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a7a8a16dc0 0x55a7a8a0b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:24.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.767+0000 7f3033e30740 1 --2- 172.21.15.83:0/2096039997 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7a8a11c50 0x55a7a8a22c20 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:24.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.767+0000 7f3033e30740 1 --2- 172.21.15.83:0/2096039997 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7a8a11260 0x55a7a8a0b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:24.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.767+0000 7f3033e30740 1 -- 172.21.15.83:0/2096039997 >> 172.21.15.83:0/2096039997 conn(0x55a7a8a0c8d0 msgr2=0x55a7a8a130b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:24.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.767+0000 7f3033e30740 1 -- 172.21.15.83:0/2096039997 shutdown_connections 2024-04-24T08:51:24.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.767+0000 7f3033e30740 1 -- 172.21.15.83:0/2096039997 wait complete. 2024-04-24T08:51:24.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.768+0000 7f3033e30740 1 -- start start 2024-04-24T08:51:24.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.768+0000 7f3033e30740 1 Processor -- start 2024-04-24T08:51:24.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.768+0000 7f3033e30740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7a8a11260 0x55a7a8a20c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:24.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.768+0000 7f3033e30740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7a8a11c50 0x55a7a8a92f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:24.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.768+0000 7f3033e30740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a7a8a16dc0 0x55a7a8a95510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:24.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.768+0000 7f3033e30740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a7a8a29ab0 con 0x55a7a8a11260 2024-04-24T08:51:24.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.768+0000 7f3033e30740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a7a8a293e0 con 0x55a7a8a11c50 2024-04-24T08:51:24.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.768+0000 7f3033e30740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a7a8a2a180 con 0x55a7a8a16dc0 2024-04-24T08:51:24.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.768+0000 7f3033e30740 10 client.0 _ll_register_callbacks cb 0x7f301c009d20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:51:24.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.769+0000 7f3029e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7a8a11260 0x55a7a8a20c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:24.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.769+0000 7f3029e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7a8a11260 0x55a7a8a20c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41344/0 (socket says 172.21.15.83:41344) 2024-04-24T08:51:24.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.769+0000 7f3029e00640 1 -- 172.21.15.83:0/4150616985 learned_addr learned my addr 172.21.15.83:0/4150616985 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:51:24.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.769+0000 7f3029400640 1 --2- 172.21.15.83:0/4150616985 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7a8a11c50 0x55a7a8a92f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:24.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.769+0000 7f3029e00640 1 -- 172.21.15.83:0/4150616985 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a7a8a16dc0 msgr2=0x55a7a8a95510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:51:24.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.769+0000 7f3029e00640 1 --2- 172.21.15.83:0/4150616985 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a7a8a16dc0 0x55a7a8a95510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:24.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.769+0000 7f3029e00640 1 -- 172.21.15.83:0/4150616985 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7a8a11c50 msgr2=0x55a7a8a92f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:51:24.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.769+0000 7f3029e00640 1 --2- 172.21.15.83:0/4150616985 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7a8a11c50 0x55a7a8a92f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:24.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.769+0000 7f3029e00640 1 -- 172.21.15.83:0/4150616985 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7a8a95d40 con 0x55a7a8a11260 2024-04-24T08:51:24.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.769+0000 7f3029e00640 1 --2- 172.21.15.83:0/4150616985 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7a8a11260 0x55a7a8a20c10 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f301c01c3e0 tx=0x7f301c005660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2782255204b86b87 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:24.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.769+0000 7f300be00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:51:24.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.770+0000 7f300be00640 1 -- 172.21.15.83:0/4150616985 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f301c010e70 con 0x55a7a8a11260 2024-04-24T08:51:24.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.770+0000 7f300be00640 1 -- 172.21.15.83:0/4150616985 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9237+0+0 (secure 0 0 0) 0x7f301c00e8d0 con 0x55a7a8a11260 2024-04-24T08:51:24.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.770+0000 7f3033e30740 1 -- 172.21.15.83:0/4150616985 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3018009de0 con 0x55a7a8a11260 2024-04-24T08:51:24.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.770+0000 7f300be00640 1 -- 172.21.15.83:0/4150616985 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f301c00eaf0 con 0x55a7a8a11260 2024-04-24T08:51:24.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.770+0000 7f3033e30740 10 client.15483 fetch_fsmap learned FSMap version 27 2024-04-24T08:51:24.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.770+0000 7f3033e30740 1 -- 172.21.15.83:0/4150616985 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a7a8a97110 con 0x55a7a8a11260 2024-04-24T08:51:24.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.770+0000 7f300be00640 1 -- 172.21.15.83:0/4150616985 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3018009de0 con 0x55a7a8a11260 2024-04-24T08:51:24.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.770+0000 7f300be00640 1 -- 172.21.15.83:0/4150616985 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f301c01c4e0 con 0x55a7a8a11260 2024-04-24T08:51:24.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.770+0000 7f3033e30740 10 client.15483 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T08:51:24.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.770+0000 7f3033e30740 10 client.15483 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-24T08:51:24.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.770+0000 7f3033e30740 20 client.15483 populate_metadata read hostname 'smithi083' 2024-04-24T08:51:24.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.771+0000 7f3033e30740 1 --2- 172.21.15.83:0/4150616985 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55a7a8a9bdc0 0x55a7a8a9aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:24.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.771+0000 7f3033e30740 4 client.15483 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T08:51:24.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.771+0000 7f3033e30740 1 -- 172.21.15.83:0/4150616985 --> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] -- command(tid 0: {"prefix": "status"}) -- 0x55a7a897e3a0 con 0x55a7a8a9bdc0 2024-04-24T08:51:24.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.771+0000 7f3029400640 1 --2- 172.21.15.83:0/4150616985 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55a7a8a9bdc0 0x55a7a8a9aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:24.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.772+0000 7f3029400640 1 --2- 172.21.15.83:0/4150616985 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55a7a8a9bdc0 0x55a7a8a9aad0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3c0c9d3650191626 server_cookie=3e3da8040acda1f2 in_seq=0 out_seq=0 2024-04-24T08:51:24.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.772+0000 7f300be00640 10 client.15483 ms_handle_connect on v2:172.21.15.189:6838/2548965336 2024-04-24T08:51:24.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.773+0000 7f300be00640 1 -- 172.21.15.83:0/4150616985 <== mds.4 v2:172.21.15.189:6838/2548965336 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55a7a897e3a0 con 0x55a7a8a9bdc0 2024-04-24T08:51:24.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.773+0000 7f300be00640 10 client.15483 handle_command_reply: tid=0 2024-04-24T08:51:24.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.773+0000 7f3033e30740 1 client.15483 shutdown 2024-04-24T08:51:24.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.773+0000 7f3033e30740 1 -- 172.21.15.83:0/4150616985 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7a8a11260 msgr2=0x55a7a8a20c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:24.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.773+0000 7f3033e30740 1 --2- 172.21.15.83:0/4150616985 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7a8a11260 0x55a7a8a20c10 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f301c01c3e0 tx=0x7f301c005660 comp rx=0 tx=0).stop 2024-04-24T08:51:24.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.773+0000 7f3033e30740 1 -- 172.21.15.83:0/4150616985 shutdown_connections 2024-04-24T08:51:24.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.773+0000 7f3033e30740 1 --2- 172.21.15.83:0/4150616985 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55a7a8a9bdc0 0x55a7a8a9aad0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:24.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.773+0000 7f3033e30740 1 --2- 172.21.15.83:0/4150616985 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a7a8a16dc0 0x55a7a8a95510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:24.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.773+0000 7f3033e30740 1 --2- 172.21.15.83:0/4150616985 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7a8a11c50 0x55a7a8a92f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:24.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.773+0000 7f3033e30740 1 --2- 172.21.15.83:0/4150616985 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7a8a11260 0x55a7a8a20c10 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:24.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.773+0000 7f300be00640 0 client.15483 ms_handle_reset on v2:172.21.15.189:6838/2548965336 2024-04-24T08:51:24.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.773+0000 7f3033e30740 1 -- 172.21.15.83:0/4150616985 >> 172.21.15.83:0/4150616985 conn(0x55a7a8a0c8d0 msgr2=0x55a7a8a27560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:24.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.773+0000 7f3033e30740 1 -- 172.21.15.83:0/4150616985 shutdown_connections 2024-04-24T08:51:24.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.773+0000 7f3033e30740 1 -- 172.21.15.83:0/4150616985 wait complete. 2024-04-24T08:51:24.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.773+0000 7f3033e30740 20 client.15483 trim_cache size 0 max 16384 2024-04-24T08:51:24.780 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T08:51:24.780 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_fsid": "bd3e039e-0215-11ef-bc93-c7b262605968", 2024-04-24T08:51:24.780 INFO:teuthology.orchestra.run.smithi083.stdout: "whoami": 4, 2024-04-24T08:51:24.780 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 24454, 2024-04-24T08:51:24.780 INFO:teuthology.orchestra.run.smithi083.stdout: "want_state": "up:active", 2024-04-24T08:51:24.780 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "up:active", 2024-04-24T08:51:24.780 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs", 2024-04-24T08:51:24.780 INFO:teuthology.orchestra.run.smithi083.stdout: "rank_uptime": 140.70237185799999, 2024-04-24T08:51:24.780 INFO:teuthology.orchestra.run.smithi083.stdout: "mdsmap_epoch": 27, 2024-04-24T08:51:24.780 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch": 93, 2024-04-24T08:51:24.780 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch_barrier": 89, 2024-04-24T08:51:24.780 INFO:teuthology.orchestra.run.smithi083.stdout: "uptime": 195.35628284399999 2024-04-24T08:51:24.780 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T08:51:24.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.774+0000 7f302b200640 1 -- 172.21.15.83:0/3211025694 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f30100a3bb0 msgr2=0x7f30100a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:24.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.774+0000 7f302b200640 1 --2- 172.21.15.83:0/3211025694 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f30100a3bb0 0x7f30100a6070 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f3024031aa0 tx=0x7f3024005d80 comp rx=0 tx=0).stop 2024-04-24T08:51:24.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.774+0000 7f302b200640 1 -- 172.21.15.83:0/3211025694 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f302c1052c0 msgr2=0x7f302c19a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:24.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.774+0000 7f302b200640 1 --2- 172.21.15.83:0/3211025694 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f302c1052c0 0x7f302c19a830 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f301800d770 tx=0x7f301800dc30 comp rx=0 tx=0).stop 2024-04-24T08:51:24.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.774+0000 7f302b200640 1 -- 172.21.15.83:0/3211025694 shutdown_connections 2024-04-24T08:51:24.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.774+0000 7f302b200640 1 --2- 172.21.15.83:0/3211025694 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f302c107bf0 0x7f302c1a18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:24.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.774+0000 7f302b200640 1 --2- 172.21.15.83:0/3211025694 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f30100a3bb0 0x7f30100a6070 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:24.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.774+0000 7f302b200640 1 --2- 172.21.15.83:0/3211025694 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f302c1052c0 0x7f302c19a830 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:24.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.774+0000 7f302b200640 1 --2- 172.21.15.83:0/3211025694 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f302c102990 0x7f302c19a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:24.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.774+0000 7f302b200640 1 -- 172.21.15.83:0/3211025694 >> 172.21.15.83:0/3211025694 conn(0x7f302c0fc3a0 msgr2=0x7f302c102000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:24.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.774+0000 7f302b200640 1 -- 172.21.15.83:0/3211025694 shutdown_connections 2024-04-24T08:51:24.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.774+0000 7f302b200640 1 -- 172.21.15.83:0/3211025694 wait complete. 2024-04-24T08:51:24.793 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.k status 2024-04-24T08:51:24.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.990+0000 7f13a5400640 1 -- 172.21.15.83:0/843231660 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13a0109e50 msgr2=0x7f13a010c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:24.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.990+0000 7f13a5400640 1 --2- 172.21.15.83:0/843231660 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13a0109e50 0x7f13a010c2e0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f1398009ca0 tx=0x7f139802a8c0 comp rx=0 tx=0).stop 2024-04-24T08:51:24.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.990+0000 7f13a5400640 1 -- 172.21.15.83:0/843231660 shutdown_connections 2024-04-24T08:51:24.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.990+0000 7f13a5400640 1 --2- 172.21.15.83:0/843231660 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13a0109e50 0x7f13a010c2e0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:24.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.990+0000 7f13a5400640 1 --2- 172.21.15.83:0/843231660 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f13a0074a60 0x7f13a0074ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:24.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.990+0000 7f13a5400640 1 --2- 172.21.15.83:0/843231660 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13a0076040 0x7f13a0074490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:24.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.990+0000 7f13a5400640 1 -- 172.21.15.83:0/843231660 >> 172.21.15.83:0/843231660 conn(0x7f13a00fd500 msgr2=0x7f13a00ff940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:24.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.990+0000 7f13a5400640 1 -- 172.21.15.83:0/843231660 shutdown_connections 2024-04-24T08:51:24.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.990+0000 7f13a5400640 1 -- 172.21.15.83:0/843231660 wait complete. 2024-04-24T08:51:24.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.991+0000 7f13a5400640 1 Processor -- start 2024-04-24T08:51:24.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.992+0000 7f13a5400640 1 -- start start 2024-04-24T08:51:24.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.992+0000 7f13a5400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f13a0074a60 0x7f13a019a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:24.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.992+0000 7f13a5400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13a0076040 0x7f13a019a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:24.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.992+0000 7f13a5400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13a0109e50 0x7f13a01a1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:24.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.992+0000 7f13a5400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f13a010fc50 con 0x7f13a0109e50 2024-04-24T08:51:24.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.992+0000 7f13a5400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f13a010fad0 con 0x7f13a0076040 2024-04-24T08:51:24.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.992+0000 7f13a5400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f13a010fdd0 con 0x7f13a0074a60 2024-04-24T08:51:24.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.992+0000 7f13a4a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13a0109e50 0x7f13a01a1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:24.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.992+0000 7f13a4a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13a0109e50 0x7f13a01a1840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41356/0 (socket says 172.21.15.83:41356) 2024-04-24T08:51:24.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.992+0000 7f13a4a00640 1 -- 172.21.15.83:0/1825609044 learned_addr learned my addr 172.21.15.83:0/1825609044 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:51:24.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.993+0000 7f139fe00640 1 --2- 172.21.15.83:0/1825609044 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f13a0074a60 0x7f13a019a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:24.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.993+0000 7f139f400640 1 --2- 172.21.15.83:0/1825609044 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13a0076040 0x7f13a019a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:24.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.993+0000 7f13a4a00640 1 -- 172.21.15.83:0/1825609044 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f13a0074a60 msgr2=0x7f13a019a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:24.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.993+0000 7f13a4a00640 1 --2- 172.21.15.83:0/1825609044 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f13a0074a60 0x7f13a019a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:24.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.993+0000 7f13a4a00640 1 -- 172.21.15.83:0/1825609044 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13a0076040 msgr2=0x7f13a019a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:24.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.993+0000 7f13a4a00640 1 --2- 172.21.15.83:0/1825609044 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13a0076040 0x7f13a019a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:24.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.993+0000 7f13a4a00640 1 -- 172.21.15.83:0/1825609044 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13a01a1eb0 con 0x7f13a0109e50 2024-04-24T08:51:24.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.993+0000 7f139f400640 1 --2- 172.21.15.83:0/1825609044 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13a0076040 0x7f13a019a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:51:24.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.993+0000 7f13a4a00640 1 --2- 172.21.15.83:0/1825609044 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13a0109e50 0x7f13a01a1840 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f1398009dc0 tx=0x7f1398003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:24.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.993+0000 7f139cc00640 1 -- 172.21.15.83:0/1825609044 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1398004040 con 0x7f13a0109e50 2024-04-24T08:51:24.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.994+0000 7f139cc00640 1 -- 172.21.15.83:0/1825609044 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1398004690 con 0x7f13a0109e50 2024-04-24T08:51:24.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.994+0000 7f139cc00640 1 -- 172.21.15.83:0/1825609044 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f139802f4c0 con 0x7f13a0109e50 2024-04-24T08:51:24.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.994+0000 7f13a5400640 1 -- 172.21.15.83:0/1825609044 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13a01a2140 con 0x7f13a0109e50 2024-04-24T08:51:24.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.994+0000 7f13a5400640 1 -- 172.21.15.83:0/1825609044 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13a01a26e0 con 0x7f13a0109e50 2024-04-24T08:51:24.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.997+0000 7f13a9779740 1 Processor -- start 2024-04-24T08:51:24.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.997+0000 7f139cc00640 1 -- 172.21.15.83:0/1825609044 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f1398030650 con 0x7f13a0109e50 2024-04-24T08:51:24.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.997+0000 7f13a9779740 1 -- start start 2024-04-24T08:51:24.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.997+0000 7f13a9779740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55de713ca260 0x55de713ca680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:24.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.997+0000 7f13a9779740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55de713cac50 0x55de713cf750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:24.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.997+0000 7f13a9779740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de713cfdc0 0x55de713d2250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:24.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.997+0000 7f13a9779740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55de7132fb10 con 0x55de713ca260 2024-04-24T08:51:24.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.997+0000 7f13a9779740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55de712d62b0 con 0x55de713cfdc0 2024-04-24T08:51:24.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.997+0000 7f13a9779740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55de7132bad0 con 0x55de713cac50 2024-04-24T08:51:24.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.998+0000 7f139f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55de713ca260 0x55de713ca680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:24.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.998+0000 7f139f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55de713ca260 0x55de713ca680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41370/0 (socket says 172.21.15.83:41370) 2024-04-24T08:51:24.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.998+0000 7f139f400640 1 -- 172.21.15.83:0/2540515242 learned_addr learned my addr 172.21.15.83:0/2540515242 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:51:24.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.998+0000 7f139fe00640 1 --2- 172.21.15.83:0/2540515242 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55de713cac50 0x55de713cf750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:25.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.998+0000 7f139f400640 1 -- 172.21.15.83:0/2540515242 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55de713cac50 msgr2=0x55de713cf750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:51:25.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.998+0000 7f139f400640 1 --2- 172.21.15.83:0/2540515242 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55de713cac50 0x55de713cf750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.998+0000 7f139f400640 1 -- 172.21.15.83:0/2540515242 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de713cfdc0 msgr2=0x55de713d2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:51:25.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.998+0000 7f139f400640 1 --2- 172.21.15.83:0/2540515242 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de713cfdc0 0x55de713d2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.998+0000 7f139f400640 1 -- 172.21.15.83:0/2540515242 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de710422d0 con 0x55de713ca260 2024-04-24T08:51:25.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.998+0000 7f139fe00640 1 --2- 172.21.15.83:0/2540515242 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55de713cac50 0x55de713cf750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:51:25.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.998+0000 7f139cc00640 1 --2- 172.21.15.83:0/1825609044 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f13840a3c30 0x7f13840a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:25.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.999+0000 7f139cc00640 1 -- 172.21.15.83:0/1825609044 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9237+0+0 (secure 0 0 0) 0x7f13980f1fc0 con 0x7f13a0109e50 2024-04-24T08:51:25.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.999+0000 7f139f400640 1 --2- 172.21.15.83:0/2540515242 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55de713ca260 0x55de713ca680 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f138c00bbe0 tx=0x7f138c00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d53abe773275a041 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:25.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.999+0000 7f139fe00640 1 --2- 172.21.15.83:0/1825609044 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f13840a3c30 0x7f13840a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:25.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.999+0000 7f1395200640 1 -- 172.21.15.83:0/2540515242 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f138c011670 con 0x55de713ca260 2024-04-24T08:51:25.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.999+0000 7f1395200640 1 -- 172.21.15.83:0/2540515242 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f138c011cc0 con 0x55de713ca260 2024-04-24T08:51:25.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:24.999+0000 7f1395200640 1 -- 172.21.15.83:0/2540515242 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f138c019bd0 con 0x55de713ca260 2024-04-24T08:51:25.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.000+0000 7f13a9779740 1 -- 172.21.15.83:0/2540515242 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55de713ca260 msgr2=0x55de713ca680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:25.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.000+0000 7f139fe00640 1 --2- 172.21.15.83:0/1825609044 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f13840a3c30 0x7f13840a60f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f13900058c0 tx=0x7f1390005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:25.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.000+0000 7f13a9779740 1 --2- 172.21.15.83:0/2540515242 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55de713ca260 0x55de713ca680 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f138c00bbe0 tx=0x7f138c00f730 comp rx=0 tx=0).stop 2024-04-24T08:51:25.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.000+0000 7f13a9779740 1 -- 172.21.15.83:0/2540515242 shutdown_connections 2024-04-24T08:51:25.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.000+0000 7f13a9779740 1 --2- 172.21.15.83:0/2540515242 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de713cfdc0 0x55de713d2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.000+0000 7f13a9779740 1 --2- 172.21.15.83:0/2540515242 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55de713cac50 0x55de713cf750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.000+0000 7f13a9779740 1 --2- 172.21.15.83:0/2540515242 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55de713ca260 0x55de713ca680 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.000+0000 7f13a9779740 1 -- 172.21.15.83:0/2540515242 >> 172.21.15.83:0/2540515242 conn(0x55de713c58d0 msgr2=0x55de713c5cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:25.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.000+0000 7f13a9779740 1 -- 172.21.15.83:0/2540515242 shutdown_connections 2024-04-24T08:51:25.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.000+0000 7f13a9779740 1 -- 172.21.15.83:0/2540515242 wait complete. 2024-04-24T08:51:25.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.000+0000 7f13a9779740 1 -- start start 2024-04-24T08:51:25.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.001+0000 7f13a9779740 1 Processor -- start 2024-04-24T08:51:25.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.001+0000 7f13a9779740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55de713ca260 0x55de713d4720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:25.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.001+0000 7f13a9779740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de713cac50 0x55de713d4c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:25.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.001+0000 7f13a9779740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55de713cfdc0 0x55de713dbca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:25.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.001+0000 7f13a9779740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55de713d2d70 con 0x55de713ca260 2024-04-24T08:51:25.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.001+0000 7f13a9779740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55de712e9590 con 0x55de713cac50 2024-04-24T08:51:25.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.001+0000 7f13a9779740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55de713d2ef0 con 0x55de713cfdc0 2024-04-24T08:51:25.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.001+0000 7f13a9779740 10 client.0 _ll_register_callbacks cb 0x55de712cc3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:51:25.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.001+0000 7f139fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55de713ca260 0x55de713d4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:25.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.001+0000 7f139fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55de713ca260 0x55de713d4720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41386/0 (socket says 172.21.15.83:41386) 2024-04-24T08:51:25.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.001+0000 7f139fe00640 1 -- 172.21.15.83:0/4188412426 learned_addr learned my addr 172.21.15.83:0/4188412426 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:51:25.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.002+0000 7f139f400640 1 --2- 172.21.15.83:0/4188412426 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de713cac50 0x55de713d4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:25.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.002+0000 7f13a4a00640 1 --2- 172.21.15.83:0/4188412426 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55de713cfdc0 0x55de713dbca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:25.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.002+0000 7f139fe00640 1 -- 172.21.15.83:0/4188412426 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55de713cfdc0 msgr2=0x55de713dbca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:51:25.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.002+0000 7f139fe00640 1 --2- 172.21.15.83:0/4188412426 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55de713cfdc0 0x55de713dbca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.002+0000 7f139fe00640 1 -- 172.21.15.83:0/4188412426 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de713cac50 msgr2=0x55de713d4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:51:25.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.002+0000 7f139fe00640 1 --2- 172.21.15.83:0/4188412426 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de713cac50 0x55de713d4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.002+0000 7f139fe00640 1 -- 172.21.15.83:0/4188412426 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de7110c6b0 con 0x55de713ca260 2024-04-24T08:51:25.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.002+0000 7f139f400640 1 --2- 172.21.15.83:0/4188412426 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de713cac50 0x55de713d4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:51:25.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.002+0000 7f13a4a00640 1 --2- 172.21.15.83:0/4188412426 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55de713cfdc0 0x55de713dbca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:51:25.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.002+0000 7f139fe00640 1 --2- 172.21.15.83:0/4188412426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55de713ca260 0x55de713d4720 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f1390004650 tx=0x7f13900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=193f9682d669355 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:25.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.002+0000 7f1389600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:51:25.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.002+0000 7f1389600640 1 -- 172.21.15.83:0/4188412426 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f139001e420 con 0x55de713ca260 2024-04-24T08:51:25.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.003+0000 7f13a9779740 1 -- 172.21.15.83:0/4188412426 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f138c011cc0 con 0x55de713ca260 2024-04-24T08:51:25.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.003+0000 7f1389600640 1 -- 172.21.15.83:0/4188412426 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9237+0+0 (secure 0 0 0) 0x7f139001e5c0 con 0x55de713ca260 2024-04-24T08:51:25.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.003+0000 7f1389600640 1 -- 172.21.15.83:0/4188412426 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f139002f050 con 0x55de713ca260 2024-04-24T08:51:25.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.003+0000 7f13a9779740 10 client.15495 fetch_fsmap learned FSMap version 27 2024-04-24T08:51:25.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.003+0000 7f13a9779740 1 -- 172.21.15.83:0/4188412426 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55de713df130 con 0x55de713ca260 2024-04-24T08:51:25.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.003+0000 7f1389600640 1 -- 172.21.15.83:0/4188412426 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f138c011cc0 con 0x55de713ca260 2024-04-24T08:51:25.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.003+0000 7f1389600640 1 -- 172.21.15.83:0/4188412426 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f1390026690 con 0x55de713ca260 2024-04-24T08:51:25.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.003+0000 7f13a9779740 10 client.15495 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T08:51:25.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.004+0000 7f13a9779740 10 client.15495 resolve_mds: resolved name 'k' to daemon mds.k 2024-04-24T08:51:25.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.004+0000 7f13a9779740 20 client.15495 populate_metadata read hostname 'smithi083' 2024-04-24T08:51:25.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.004+0000 7f13a9779740 1 --2- 172.21.15.83:0/4188412426 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55de713e17c0 0x55de713e2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:25.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.004+0000 7f13a9779740 4 client.15495 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:51:25.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.004+0000 7f13a9779740 1 -- 172.21.15.83:0/4188412426 --> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55de7133d530 con 0x55de713e17c0 2024-04-24T08:51:25.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.004+0000 7f139f400640 1 --2- 172.21.15.83:0/4188412426 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55de713e17c0 0x55de713e2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:25.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.005+0000 7f139f400640 1 --2- 172.21.15.83:0/4188412426 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55de713e17c0 0x55de713e2c30 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f2e4a64c373502e6 server_cookie=ad6b298ef740ef69 in_seq=0 out_seq=0 2024-04-24T08:51:25.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.005+0000 7f1389600640 10 client.15495 ms_handle_connect on v2:172.21.15.138:6832/1157967658 2024-04-24T08:51:25.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.007+0000 7f1389600640 1 -- 172.21.15.83:0/4188412426 <== mds.3 v2:172.21.15.138:6832/1157967658 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55de7133d530 con 0x55de713e17c0 2024-04-24T08:51:25.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.007+0000 7f1389600640 10 client.15495 handle_command_reply: tid=0 2024-04-24T08:51:25.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.008+0000 7f13a9779740 1 client.15495 shutdown 2024-04-24T08:51:25.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.008+0000 7f13a9779740 1 -- 172.21.15.83:0/4188412426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55de713ca260 msgr2=0x55de713d4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:25.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.008+0000 7f13a9779740 1 --2- 172.21.15.83:0/4188412426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55de713ca260 0x55de713d4720 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f1390004650 tx=0x7f13900046c0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.008+0000 7f13a9779740 1 -- 172.21.15.83:0/4188412426 shutdown_connections 2024-04-24T08:51:25.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.008+0000 7f13a9779740 1 --2- 172.21.15.83:0/4188412426 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55de713e17c0 0x55de713e2c30 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.008+0000 7f13a9779740 1 --2- 172.21.15.83:0/4188412426 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55de713cfdc0 0x55de713dbca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.008+0000 7f13a9779740 1 --2- 172.21.15.83:0/4188412426 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de713cac50 0x55de713d4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.008+0000 7f1389600640 0 client.15495 ms_handle_reset on v2:172.21.15.138:6832/1157967658 2024-04-24T08:51:25.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.008+0000 7f13a9779740 1 --2- 172.21.15.83:0/4188412426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55de713ca260 0x55de713d4720 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.008+0000 7f13a9779740 1 -- 172.21.15.83:0/4188412426 >> 172.21.15.83:0/4188412426 conn(0x55de713c58d0 msgr2=0x55de713c4970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:25.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.008+0000 7f13a9779740 1 -- 172.21.15.83:0/4188412426 shutdown_connections 2024-04-24T08:51:25.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.008+0000 7f13a9779740 1 -- 172.21.15.83:0/4188412426 wait complete. 2024-04-24T08:51:25.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.008+0000 7f13a9779740 20 client.15495 trim_cache size 0 max 16384 2024-04-24T08:51:25.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.028+0000 7f13a9779740 1 Processor -- start 2024-04-24T08:51:25.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.029+0000 7f13a9779740 1 -- start start 2024-04-24T08:51:25.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.029+0000 7f13a9779740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de713ca260 0x55de713d21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:25.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.029+0000 7f13a9779740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55de713cac50 0x55de713dbc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:25.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.029+0000 7f13a9779740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55de713cfdc0 0x55de713cc150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:25.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.029+0000 7f13a9779740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55de713e0f70 con 0x55de713cfdc0 2024-04-24T08:51:25.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.029+0000 7f13a9779740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55de713e1640 con 0x55de713ca260 2024-04-24T08:51:25.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.029+0000 7f13a9779740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55de713e1d10 con 0x55de713cac50 2024-04-24T08:51:25.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.029+0000 7f13a4a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55de713cfdc0 0x55de713cc150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:25.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.029+0000 7f13a4a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55de713cfdc0 0x55de713cc150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41402/0 (socket says 172.21.15.83:41402) 2024-04-24T08:51:25.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.029+0000 7f13a4a00640 1 -- 172.21.15.83:0/1598844400 learned_addr learned my addr 172.21.15.83:0/1598844400 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:51:25.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.030+0000 7f139fe00640 1 --2- 172.21.15.83:0/1598844400 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de713ca260 0x55de713d21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:25.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.030+0000 7f139f400640 1 --2- 172.21.15.83:0/1598844400 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55de713cac50 0x55de713dbc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:25.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.030+0000 7f13a4a00640 1 -- 172.21.15.83:0/1598844400 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55de713cac50 msgr2=0x55de713dbc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:51:25.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.030+0000 7f13a4a00640 1 --2- 172.21.15.83:0/1598844400 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55de713cac50 0x55de713dbc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.030+0000 7f13a4a00640 1 -- 172.21.15.83:0/1598844400 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de713ca260 msgr2=0x55de713d21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:51:25.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.030+0000 7f13a4a00640 1 --2- 172.21.15.83:0/1598844400 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de713ca260 0x55de713d21b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.030+0000 7f13a4a00640 1 -- 172.21.15.83:0/1598844400 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de713c5700 con 0x55de713cfdc0 2024-04-24T08:51:25.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.030+0000 7f139f400640 1 --2- 172.21.15.83:0/1598844400 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55de713cac50 0x55de713dbc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:51:25.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.030+0000 7f13a4a00640 1 --2- 172.21.15.83:0/1598844400 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55de713cfdc0 0x55de713cc150 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f1398031040 tx=0x7f1398003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0a588b9a4d32418 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:25.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.030+0000 7f138be00640 1 -- 172.21.15.83:0/1598844400 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1398039e10 con 0x55de713cfdc0 2024-04-24T08:51:25.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.030+0000 7f138be00640 1 -- 172.21.15.83:0/1598844400 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f13980c0070 con 0x55de713cfdc0 2024-04-24T08:51:25.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.030+0000 7f138be00640 1 -- 172.21.15.83:0/1598844400 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13980d7270 con 0x55de713cfdc0 2024-04-24T08:51:25.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.031+0000 7f13a9779740 1 -- 172.21.15.83:0/1598844400 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55de713cfdc0 msgr2=0x55de713cc150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:25.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.031+0000 7f13a9779740 1 --2- 172.21.15.83:0/1598844400 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55de713cfdc0 0x55de713cc150 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f1398031040 tx=0x7f1398003f90 comp rx=0 tx=0).stop 2024-04-24T08:51:25.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.031+0000 7f13a9779740 1 -- 172.21.15.83:0/1598844400 shutdown_connections 2024-04-24T08:51:25.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.031+0000 7f13a9779740 1 --2- 172.21.15.83:0/1598844400 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55de713cfdc0 0x55de713cc150 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.031+0000 7f13a9779740 1 --2- 172.21.15.83:0/1598844400 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55de713cac50 0x55de713dbc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.031+0000 7f13a9779740 1 --2- 172.21.15.83:0/1598844400 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de713ca260 0x55de713d21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.031+0000 7f13a9779740 1 -- 172.21.15.83:0/1598844400 >> 172.21.15.83:0/1598844400 conn(0x55de713c58d0 msgr2=0x55de713c5cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:25.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.031+0000 7f13a9779740 1 -- 172.21.15.83:0/1598844400 shutdown_connections 2024-04-24T08:51:25.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.031+0000 7f13a9779740 1 -- 172.21.15.83:0/1598844400 wait complete. 2024-04-24T08:51:25.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.032+0000 7f13a9779740 1 -- start start 2024-04-24T08:51:25.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.032+0000 7f13a9779740 1 Processor -- start 2024-04-24T08:51:25.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.032+0000 7f13a9779740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de713ca260 0x55de713d9c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:25.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.032+0000 7f13a9779740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55de713cac50 0x55de71443750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:25.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.032+0000 7f13a9779740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55de713cfdc0 0x55de71445ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:25.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.032+0000 7f13a9779740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55de713e2ab0 con 0x55de713cfdc0 2024-04-24T08:51:25.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.032+0000 7f13a9779740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55de713e23e0 con 0x55de713ca260 2024-04-24T08:51:25.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.032+0000 7f13a9779740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55de713e3180 con 0x55de713cac50 2024-04-24T08:51:25.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.032+0000 7f13a9779740 10 client.0 _ll_register_callbacks cb 0x7f1390011ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:51:25.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.033+0000 7f13a4a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55de713cfdc0 0x55de71445ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:25.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.033+0000 7f13a4a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55de713cfdc0 0x55de71445ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41408/0 (socket says 172.21.15.83:41408) 2024-04-24T08:51:25.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.033+0000 7f13a4a00640 1 -- 172.21.15.83:0/1433756190 learned_addr learned my addr 172.21.15.83:0/1433756190 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:51:25.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.033+0000 7f13a4a00640 1 -- 172.21.15.83:0/1433756190 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55de713cac50 msgr2=0x55de71443750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:51:25.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.033+0000 7f139f400640 1 --2- 172.21.15.83:0/1433756190 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55de713cac50 0x55de71443750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:25.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.033+0000 7f13a4a00640 1 --2- 172.21.15.83:0/1433756190 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55de713cac50 0x55de71443750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.033+0000 7f139fe00640 1 --2- 172.21.15.83:0/1433756190 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de713ca260 0x55de713d9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:25.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.033+0000 7f13a4a00640 1 -- 172.21.15.83:0/1433756190 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de713ca260 msgr2=0x55de713d9c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:51:25.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.033+0000 7f13a4a00640 1 --2- 172.21.15.83:0/1433756190 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de713ca260 0x55de713d9c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.033+0000 7f13a4a00640 1 -- 172.21.15.83:0/1433756190 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de714461e0 con 0x55de713cfdc0 2024-04-24T08:51:25.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.033+0000 7f13a4a00640 1 --2- 172.21.15.83:0/1433756190 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55de713cfdc0 0x55de71445ca0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f1398004980 tx=0x7f13980b9ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d9e82152c62fac0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:25.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.033+0000 7f1389600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:51:25.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.033+0000 7f1389600640 1 -- 172.21.15.83:0/1433756190 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13980db070 con 0x55de713cfdc0 2024-04-24T08:51:25.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.034+0000 7f1389600640 1 -- 172.21.15.83:0/1433756190 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9237+0+0 (secure 0 0 0) 0x7f13980ba4c0 con 0x55de713cfdc0 2024-04-24T08:51:25.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.034+0000 7f13a9779740 1 -- 172.21.15.83:0/1433756190 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13980c0070 con 0x55de713cfdc0 2024-04-24T08:51:25.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.034+0000 7f1389600640 1 -- 172.21.15.83:0/1433756190 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1398039e10 con 0x55de713cfdc0 2024-04-24T08:51:25.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.034+0000 7f13a9779740 10 client.15501 fetch_fsmap learned FSMap version 27 2024-04-24T08:51:25.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.034+0000 7f13a9779740 1 -- 172.21.15.83:0/1433756190 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55de71447400 con 0x55de713cfdc0 2024-04-24T08:51:25.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.034+0000 7f1389600640 1 -- 172.21.15.83:0/1433756190 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f13980c0070 con 0x55de713cfdc0 2024-04-24T08:51:25.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.034+0000 7f1389600640 1 -- 172.21.15.83:0/1433756190 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f13980be140 con 0x55de713cfdc0 2024-04-24T08:51:25.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.034+0000 7f13a9779740 10 client.15501 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T08:51:25.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.034+0000 7f13a9779740 10 client.15501 resolve_mds: resolved name 'k' to daemon mds.k 2024-04-24T08:51:25.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.034+0000 7f13a9779740 20 client.15501 populate_metadata read hostname 'smithi083' 2024-04-24T08:51:25.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.034+0000 7f13a9779740 1 --2- 172.21.15.83:0/1433756190 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55de7144c000 0x55de7144ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:25.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.034+0000 7f13a9779740 4 client.15501 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T08:51:25.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.034+0000 7f13a9779740 1 -- 172.21.15.83:0/1433756190 --> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] -- command(tid 0: {"prefix": "status"}) -- 0x55de713373a0 con 0x55de7144c000 2024-04-24T08:51:25.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.035+0000 7f139fe00640 1 --2- 172.21.15.83:0/1433756190 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55de7144c000 0x55de7144ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:25.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.036+0000 7f1389600640 10 client.15501 ms_handle_connect on v2:172.21.15.138:6832/1157967658 2024-04-24T08:51:25.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.036+0000 7f139fe00640 1 --2- 172.21.15.83:0/1433756190 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55de7144c000 0x55de7144ad10 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3ce55e517991b9d2 server_cookie=5ae33a2b0c743541 in_seq=0 out_seq=0 2024-04-24T08:51:25.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.037+0000 7f1389600640 1 -- 172.21.15.83:0/1433756190 <== mds.3 v2:172.21.15.138:6832/1157967658 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55de713373a0 con 0x55de7144c000 2024-04-24T08:51:25.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.037+0000 7f1389600640 10 client.15501 handle_command_reply: tid=0 2024-04-24T08:51:25.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.037+0000 7f13a9779740 1 client.15501 shutdown 2024-04-24T08:51:25.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.037+0000 7f13a9779740 1 -- 172.21.15.83:0/1433756190 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55de713cfdc0 msgr2=0x55de71445ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:25.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.037+0000 7f13a9779740 1 --2- 172.21.15.83:0/1433756190 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55de713cfdc0 0x55de71445ca0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f1398004980 tx=0x7f13980b9ac0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.037+0000 7f13a9779740 1 -- 172.21.15.83:0/1433756190 shutdown_connections 2024-04-24T08:51:25.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.037+0000 7f13a9779740 1 --2- 172.21.15.83:0/1433756190 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55de713cfdc0 0x55de71445ca0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.037+0000 7f13a9779740 1 --2- 172.21.15.83:0/1433756190 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55de7144c000 0x55de7144ad10 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.037+0000 7f13a9779740 1 --2- 172.21.15.83:0/1433756190 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55de713cac50 0x55de71443750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.037+0000 7f13a9779740 1 --2- 172.21.15.83:0/1433756190 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de713ca260 0x55de713d9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.037+0000 7f1389600640 0 client.15501 ms_handle_reset on v2:172.21.15.138:6832/1157967658 2024-04-24T08:51:25.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.037+0000 7f13a9779740 1 -- 172.21.15.83:0/1433756190 >> 172.21.15.83:0/1433756190 conn(0x55de713c58d0 msgr2=0x55de713e05a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:25.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.037+0000 7f13a9779740 1 -- 172.21.15.83:0/1433756190 shutdown_connections 2024-04-24T08:51:25.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.037+0000 7f13a9779740 1 -- 172.21.15.83:0/1433756190 wait complete. 2024-04-24T08:51:25.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.037+0000 7f13a9779740 20 client.15501 trim_cache size 0 max 16384 2024-04-24T08:51:25.044 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T08:51:25.044 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_fsid": "bd3e039e-0215-11ef-bc93-c7b262605968", 2024-04-24T08:51:25.044 INFO:teuthology.orchestra.run.smithi083.stdout: "whoami": 3, 2024-04-24T08:51:25.045 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 34427, 2024-04-24T08:51:25.045 INFO:teuthology.orchestra.run.smithi083.stdout: "want_state": "up:active", 2024-04-24T08:51:25.045 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "up:active", 2024-04-24T08:51:25.045 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs", 2024-04-24T08:51:25.045 INFO:teuthology.orchestra.run.smithi083.stdout: "rank_uptime": 141.98307295500001, 2024-04-24T08:51:25.045 INFO:teuthology.orchestra.run.smithi083.stdout: "mdsmap_epoch": 27, 2024-04-24T08:51:25.045 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch": 93, 2024-04-24T08:51:25.045 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch_barrier": 89, 2024-04-24T08:51:25.045 INFO:teuthology.orchestra.run.smithi083.stdout: "uptime": 224.36852528899999 2024-04-24T08:51:25.045 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T08:51:25.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.038+0000 7f13a5400640 1 -- 172.21.15.83:0/1825609044 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f13840a3c30 msgr2=0x7f13840a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:25.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.038+0000 7f13a5400640 1 --2- 172.21.15.83:0/1825609044 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f13840a3c30 0x7f13840a60f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f13900058c0 tx=0x7f1390005850 comp rx=0 tx=0).stop 2024-04-24T08:51:25.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.038+0000 7f13a5400640 1 -- 172.21.15.83:0/1825609044 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13a0109e50 msgr2=0x7f13a01a1840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:25.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.038+0000 7f13a5400640 1 --2- 172.21.15.83:0/1825609044 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13a0109e50 0x7f13a01a1840 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f1398009dc0 tx=0x7f1398003ba0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.038+0000 7f13a5400640 1 -- 172.21.15.83:0/1825609044 shutdown_connections 2024-04-24T08:51:25.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.038+0000 7f13a5400640 1 --2- 172.21.15.83:0/1825609044 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f13840a3c30 0x7f13840a60f0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.038+0000 7f13a5400640 1 --2- 172.21.15.83:0/1825609044 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13a0109e50 0x7f13a01a1840 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.038+0000 7f13a5400640 1 --2- 172.21.15.83:0/1825609044 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13a0076040 0x7f13a019a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.038+0000 7f13a5400640 1 --2- 172.21.15.83:0/1825609044 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f13a0074a60 0x7f13a019a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.038+0000 7f13a5400640 1 -- 172.21.15.83:0/1825609044 >> 172.21.15.83:0/1825609044 conn(0x7f13a00fd500 msgr2=0x7f13a0105d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:25.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.038+0000 7f13a5400640 1 -- 172.21.15.83:0/1825609044 shutdown_connections 2024-04-24T08:51:25.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.038+0000 7f13a5400640 1 -- 172.21.15.83:0/1825609044 wait complete. 2024-04-24T08:51:25.052 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:25 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/1176763574' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:51:25.052 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:25 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/2189932798' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T08:51:25.056 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2024-04-24T08:51:25.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.249+0000 7fb53b000640 1 -- 172.21.15.83:0/3712667971 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb534105470 msgr2=0x7fb5341058f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:25.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.249+0000 7fb53b000640 1 --2- 172.21.15.83:0/3712667971 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb534105470 0x7fb5341058f0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fb51c009900 tx=0x7fb51c02a3f0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.249+0000 7fb53b000640 1 -- 172.21.15.83:0/3712667971 shutdown_connections 2024-04-24T08:51:25.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.249+0000 7fb53b000640 1 --2- 172.21.15.83:0/3712667971 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb534105e30 0x7fb53410c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.249+0000 7fb53b000640 1 --2- 172.21.15.83:0/3712667971 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb534105470 0x7fb5341058f0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.249+0000 7fb53b000640 1 --2- 172.21.15.83:0/3712667971 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb534104280 0x7fb5341046a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.249+0000 7fb53b000640 1 -- 172.21.15.83:0/3712667971 >> 172.21.15.83:0/3712667971 conn(0x7fb5340ff7f0 msgr2=0x7fb534101c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:25.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.249+0000 7fb53b000640 1 -- 172.21.15.83:0/3712667971 shutdown_connections 2024-04-24T08:51:25.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.249+0000 7fb53b000640 1 -- 172.21.15.83:0/3712667971 wait complete. 2024-04-24T08:51:25.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.250+0000 7fb53b000640 1 Processor -- start 2024-04-24T08:51:25.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.250+0000 7fb53b000640 1 -- start start 2024-04-24T08:51:25.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.251+0000 7fb53b000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb534104280 0x7fb53419a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:25.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.251+0000 7fb53b000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb534105470 0x7fb53419aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:25.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.251+0000 7fb53b000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb534105e30 0x7fb5341a1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:25.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.251+0000 7fb53b000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb53410fe10 con 0x7fb534104280 2024-04-24T08:51:25.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.251+0000 7fb53b000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb53410fc90 con 0x7fb534105e30 2024-04-24T08:51:25.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.251+0000 7fb53b000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb53410ff90 con 0x7fb534105470 2024-04-24T08:51:25.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.251+0000 7fb533e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb534104280 0x7fb53419a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:25.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.251+0000 7fb533e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb534104280 0x7fb53419a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41424/0 (socket says 172.21.15.83:41424) 2024-04-24T08:51:25.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.251+0000 7fb533e00640 1 -- 172.21.15.83:0/303649950 learned_addr learned my addr 172.21.15.83:0/303649950 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:51:25.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.251+0000 7fb533400640 1 --2- 172.21.15.83:0/303649950 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb534105470 0x7fb53419aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:25.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.251+0000 7fb539000640 1 --2- 172.21.15.83:0/303649950 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb534105e30 0x7fb5341a1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:25.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.251+0000 7fb533e00640 1 -- 172.21.15.83:0/303649950 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb534105470 msgr2=0x7fb53419aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:25.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.251+0000 7fb533e00640 1 --2- 172.21.15.83:0/303649950 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb534105470 0x7fb53419aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.251+0000 7fb533e00640 1 -- 172.21.15.83:0/303649950 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb534105e30 msgr2=0x7fb5341a1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:25.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.251+0000 7fb533e00640 1 --2- 172.21.15.83:0/303649950 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb534105e30 0x7fb5341a1a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.251+0000 7fb533e00640 1 -- 172.21.15.83:0/303649950 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5341a20f0 con 0x7fb534104280 2024-04-24T08:51:25.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.251+0000 7fb533400640 1 --2- 172.21.15.83:0/303649950 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb534105470 0x7fb53419aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:51:25.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.252+0000 7fb533e00640 1 --2- 172.21.15.83:0/303649950 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb534104280 0x7fb53419a4d0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fb52400d770 tx=0x7fb52400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:25.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.252+0000 7fb530c00640 1 -- 172.21.15.83:0/303649950 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb524017070 con 0x7fb534104280 2024-04-24T08:51:25.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.252+0000 7fb530c00640 1 -- 172.21.15.83:0/303649950 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb524009c30 con 0x7fb534104280 2024-04-24T08:51:25.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.252+0000 7fb53b000640 1 -- 172.21.15.83:0/303649950 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5341a23e0 con 0x7fb534104280 2024-04-24T08:51:25.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.252+0000 7fb530c00640 1 -- 172.21.15.83:0/303649950 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb524003d00 con 0x7fb534104280 2024-04-24T08:51:25.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.252+0000 7fb53b000640 1 -- 172.21.15.83:0/303649950 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5341a2930 con 0x7fb534104280 2024-04-24T08:51:25.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.255+0000 7fb53de0f740 1 Processor -- start 2024-04-24T08:51:25.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.255+0000 7fb53de0f740 1 -- start start 2024-04-24T08:51:25.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.255+0000 7fb530c00640 1 -- 172.21.15.83:0/303649950 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fb524003ea0 con 0x7fb534104280 2024-04-24T08:51:25.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.256+0000 7fb53de0f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6f394f090 0x55f6f394f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:25.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.256+0000 7fb53de0f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6f394fa80 0x55f6f3954580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:25.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.256+0000 7fb53de0f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f6f3954bf0 0x55f6f3957080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:25.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.256+0000 7fb53de0f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55f6f38b4b10 con 0x55f6f394f090 2024-04-24T08:51:25.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.256+0000 7fb53de0f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f6f385b2b0 con 0x55f6f394fa80 2024-04-24T08:51:25.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.256+0000 7fb53de0f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55f6f38b0ad0 con 0x55f6f3954bf0 2024-04-24T08:51:25.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.256+0000 7fb539000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6f394f090 0x55f6f394f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:25.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.256+0000 7fb539000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6f394f090 0x55f6f394f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41430/0 (socket says 172.21.15.83:41430) 2024-04-24T08:51:25.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.256+0000 7fb539000640 1 -- 172.21.15.83:0/3764619508 learned_addr learned my addr 172.21.15.83:0/3764619508 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:51:25.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.256+0000 7fb533e00640 1 --2- 172.21.15.83:0/3764619508 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6f394fa80 0x55f6f3954580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:25.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.256+0000 7fb539000640 1 -- 172.21.15.83:0/3764619508 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f6f3954bf0 msgr2=0x55f6f3957080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:51:25.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.256+0000 7fb539000640 1 --2- 172.21.15.83:0/3764619508 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f6f3954bf0 0x55f6f3957080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.256+0000 7fb539000640 1 -- 172.21.15.83:0/3764619508 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6f394fa80 msgr2=0x55f6f3954580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:51:25.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.256+0000 7fb539000640 1 --2- 172.21.15.83:0/3764619508 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6f394fa80 0x55f6f3954580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.256+0000 7fb539000640 1 -- 172.21.15.83:0/3764619508 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6f35c72d0 con 0x55f6f394f090 2024-04-24T08:51:25.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.256+0000 7fb533e00640 1 --2- 172.21.15.83:0/3764619508 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6f394fa80 0x55f6f3954580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:51:25.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.257+0000 7fb539000640 1 --2- 172.21.15.83:0/3764619508 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6f394f090 0x55f6f394f4b0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fb52c008d00 tx=0x7fb52c00d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95875496f96a1d92 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:25.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.257+0000 7fb529600640 1 -- 172.21.15.83:0/3764619508 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb52c018680 con 0x55f6f394f090 2024-04-24T08:51:25.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.257+0000 7fb529600640 1 -- 172.21.15.83:0/3764619508 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb52c018cd0 con 0x55f6f394f090 2024-04-24T08:51:25.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.257+0000 7fb529600640 1 -- 172.21.15.83:0/3764619508 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb52c00fde0 con 0x55f6f394f090 2024-04-24T08:51:25.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.257+0000 7fb530c00640 1 --2- 172.21.15.83:0/303649950 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fb5180a3bb0 0x7fb5180a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:25.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.257+0000 7fb530c00640 1 -- 172.21.15.83:0/303649950 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9237+0+0 (secure 0 0 0) 0x7fb5240d0610 con 0x7fb534104280 2024-04-24T08:51:25.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.257+0000 7fb533e00640 1 --2- 172.21.15.83:0/303649950 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fb5180a3bb0 0x7fb5180a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:25.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.258+0000 7fb53de0f740 1 -- 172.21.15.83:0/3764619508 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6f394f090 msgr2=0x55f6f394f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:25.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.258+0000 7fb53de0f740 1 --2- 172.21.15.83:0/3764619508 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6f394f090 0x55f6f394f4b0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fb52c008d00 tx=0x7fb52c00d710 comp rx=0 tx=0).stop 2024-04-24T08:51:25.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.258+0000 7fb53de0f740 1 -- 172.21.15.83:0/3764619508 shutdown_connections 2024-04-24T08:51:25.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.258+0000 7fb53de0f740 1 --2- 172.21.15.83:0/3764619508 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f6f3954bf0 0x55f6f3957080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.258+0000 7fb53de0f740 1 --2- 172.21.15.83:0/3764619508 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6f394fa80 0x55f6f3954580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.258+0000 7fb53de0f740 1 --2- 172.21.15.83:0/3764619508 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6f394f090 0x55f6f394f4b0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.258+0000 7fb53de0f740 1 -- 172.21.15.83:0/3764619508 >> 172.21.15.83:0/3764619508 conn(0x55f6f394a700 msgr2=0x55f6f394ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:25.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.258+0000 7fb533e00640 1 --2- 172.21.15.83:0/303649950 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fb5180a3bb0 0x7fb5180a6070 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fb524098e70 tx=0x7fb524005f30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:25.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.258+0000 7fb53de0f740 1 -- 172.21.15.83:0/3764619508 shutdown_connections 2024-04-24T08:51:25.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.258+0000 7fb53de0f740 1 -- 172.21.15.83:0/3764619508 wait complete. 2024-04-24T08:51:25.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.259+0000 7fb53de0f740 1 -- start start 2024-04-24T08:51:25.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.259+0000 7fb53de0f740 1 Processor -- start 2024-04-24T08:51:25.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.259+0000 7fb53de0f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6f394f090 0x55f6f3959540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:25.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.259+0000 7fb53de0f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6f394fa80 0x55f6f3959a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:25.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.259+0000 7fb53de0f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f6f3954bf0 0x55f6f3960ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:25.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.259+0000 7fb53de0f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55f6f3957ba0 con 0x55f6f394f090 2024-04-24T08:51:25.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.259+0000 7fb53de0f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f6f386e590 con 0x55f6f394fa80 2024-04-24T08:51:25.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.259+0000 7fb53de0f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55f6f3957d20 con 0x55f6f3954bf0 2024-04-24T08:51:25.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.259+0000 7fb53de0f740 10 client.0 _ll_register_callbacks cb 0x55f6f38513a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:51:25.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.259+0000 7fb539000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6f394f090 0x55f6f3959540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:25.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.260+0000 7fb539000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6f394f090 0x55f6f3959540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41432/0 (socket says 172.21.15.83:41432) 2024-04-24T08:51:25.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.260+0000 7fb539000640 1 -- 172.21.15.83:0/3407665653 learned_addr learned my addr 172.21.15.83:0/3407665653 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:51:25.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.260+0000 7fb533400640 1 --2- 172.21.15.83:0/3407665653 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6f394fa80 0x55f6f3959a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:25.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.260+0000 7fb539000640 1 -- 172.21.15.83:0/3407665653 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f6f3954bf0 msgr2=0x55f6f3960ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:51:25.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.260+0000 7fb539000640 1 --2- 172.21.15.83:0/3407665653 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f6f3954bf0 0x55f6f3960ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.260+0000 7fb539000640 1 -- 172.21.15.83:0/3407665653 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6f394fa80 msgr2=0x55f6f3959a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:51:25.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.260+0000 7fb539000640 1 --2- 172.21.15.83:0/3407665653 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6f394fa80 0x55f6f3959a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.260+0000 7fb539000640 1 -- 172.21.15.83:0/3407665653 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6f36916b0 con 0x55f6f394f090 2024-04-24T08:51:25.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.260+0000 7fb539000640 1 --2- 172.21.15.83:0/3407665653 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6f394f090 0x55f6f3959540 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fb52c005f30 tx=0x7fb52c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18f65c6c4882981f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:25.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.260+0000 7fb522000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:51:25.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.260+0000 7fb522000640 1 -- 172.21.15.83:0/3407665653 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb52c029060 con 0x55f6f394f090 2024-04-24T08:51:25.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.261+0000 7fb522000640 1 -- 172.21.15.83:0/3407665653 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9237+0+0 (secure 0 0 0) 0x7fb52c018680 con 0x55f6f394f090 2024-04-24T08:51:25.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.261+0000 7fb53de0f740 1 -- 172.21.15.83:0/3407665653 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb52c018cd0 con 0x55f6f394f090 2024-04-24T08:51:25.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.261+0000 7fb522000640 1 -- 172.21.15.83:0/3407665653 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb52c030050 con 0x55f6f394f090 2024-04-24T08:51:25.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.261+0000 7fb53de0f740 10 client.15513 fetch_fsmap learned FSMap version 27 2024-04-24T08:51:25.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.261+0000 7fb53de0f740 1 -- 172.21.15.83:0/3407665653 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f6f3963da0 con 0x55f6f394f090 2024-04-24T08:51:25.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.261+0000 7fb522000640 1 -- 172.21.15.83:0/3407665653 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb52c018cd0 con 0x55f6f394f090 2024-04-24T08:51:25.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.261+0000 7fb522000640 1 -- 172.21.15.83:0/3407665653 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fb52c0056d0 con 0x55f6f394f090 2024-04-24T08:51:25.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.261+0000 7fb53de0f740 10 client.15513 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T08:51:25.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.262+0000 7fb53de0f740 10 client.15513 resolve_mds: resolved name 'e' to daemon mds.e 2024-04-24T08:51:25.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.262+0000 7fb53de0f740 20 client.15513 populate_metadata read hostname 'smithi083' 2024-04-24T08:51:25.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.262+0000 7fb53de0f740 1 --2- 172.21.15.83:0/3407665653 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55f6f3965fd0 0x55f6f3967440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:25.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.262+0000 7fb53de0f740 4 client.15513 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:51:25.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.262+0000 7fb53de0f740 1 -- 172.21.15.83:0/3407665653 --> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f6f38c2530 con 0x55f6f3965fd0 2024-04-24T08:51:25.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.262+0000 7fb533400640 1 --2- 172.21.15.83:0/3407665653 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55f6f3965fd0 0x55f6f3967440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:25.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.263+0000 7fb533400640 1 --2- 172.21.15.83:0/3407665653 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55f6f3965fd0 0x55f6f3967440 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bbfc4e32f2f581c2 server_cookie=95e016357e26313e in_seq=0 out_seq=0 2024-04-24T08:51:25.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.263+0000 7fb522000640 10 client.15513 ms_handle_connect on v2:172.21.15.138:6834/1909219758 2024-04-24T08:51:25.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.265+0000 7fb522000640 1 -- 172.21.15.83:0/3407665653 <== mds.2 v2:172.21.15.138:6834/1909219758 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f6f38c2530 con 0x55f6f3965fd0 2024-04-24T08:51:25.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.265+0000 7fb522000640 10 client.15513 handle_command_reply: tid=0 2024-04-24T08:51:25.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.265+0000 7fb53de0f740 1 client.15513 shutdown 2024-04-24T08:51:25.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.265+0000 7fb53de0f740 1 -- 172.21.15.83:0/3407665653 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6f394f090 msgr2=0x55f6f3959540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:25.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.265+0000 7fb53de0f740 1 --2- 172.21.15.83:0/3407665653 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6f394f090 0x55f6f3959540 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fb52c005f30 tx=0x7fb52c004a40 comp rx=0 tx=0).stop 2024-04-24T08:51:25.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.266+0000 7fb53de0f740 1 -- 172.21.15.83:0/3407665653 shutdown_connections 2024-04-24T08:51:25.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.266+0000 7fb53de0f740 1 --2- 172.21.15.83:0/3407665653 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55f6f3965fd0 0x55f6f3967440 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.266+0000 7fb53de0f740 1 --2- 172.21.15.83:0/3407665653 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f6f3954bf0 0x55f6f3960ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.266+0000 7fb53de0f740 1 --2- 172.21.15.83:0/3407665653 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6f394fa80 0x55f6f3959a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.266+0000 7fb53de0f740 1 --2- 172.21.15.83:0/3407665653 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6f394f090 0x55f6f3959540 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.266+0000 7fb522000640 0 client.15513 ms_handle_reset on v2:172.21.15.138:6834/1909219758 2024-04-24T08:51:25.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.266+0000 7fb53de0f740 1 -- 172.21.15.83:0/3407665653 >> 172.21.15.83:0/3407665653 conn(0x55f6f394a700 msgr2=0x55f6f39498c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:25.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.266+0000 7fb53de0f740 1 -- 172.21.15.83:0/3407665653 shutdown_connections 2024-04-24T08:51:25.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.266+0000 7fb53de0f740 1 -- 172.21.15.83:0/3407665653 wait complete. 2024-04-24T08:51:25.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.266+0000 7fb53de0f740 20 client.15513 trim_cache size 0 max 16384 2024-04-24T08:51:25.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.285+0000 7fb53de0f740 1 Processor -- start 2024-04-24T08:51:25.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.286+0000 7fb53de0f740 1 -- start start 2024-04-24T08:51:25.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.286+0000 7fb53de0f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6f394f090 0x55f6f39599e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:25.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.286+0000 7fb53de0f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f6f394fa80 0x55f6f39544e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:25.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.286+0000 7fb53de0f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6f3954bf0 0x55f6f3957a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:25.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.286+0000 7fb53de0f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55f6f3965780 con 0x55f6f3954bf0 2024-04-24T08:51:25.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.286+0000 7fb53de0f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f6f3965e50 con 0x55f6f394f090 2024-04-24T08:51:25.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.286+0000 7fb53de0f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55f6f3966520 con 0x55f6f394fa80 2024-04-24T08:51:25.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.286+0000 7fb539000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6f3954bf0 0x55f6f3957a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:25.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.286+0000 7fb539000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6f3954bf0 0x55f6f3957a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41448/0 (socket says 172.21.15.83:41448) 2024-04-24T08:51:25.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.286+0000 7fb539000640 1 -- 172.21.15.83:0/813415358 learned_addr learned my addr 172.21.15.83:0/813415358 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:51:25.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.286+0000 7fb539000640 1 -- 172.21.15.83:0/813415358 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f6f394fa80 msgr2=0x55f6f39544e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:51:25.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.286+0000 7fb539000640 1 --2- 172.21.15.83:0/813415358 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f6f394fa80 0x55f6f39544e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.286+0000 7fb539000640 1 -- 172.21.15.83:0/813415358 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6f394f090 msgr2=0x55f6f39599e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:51:25.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.286+0000 7fb539000640 1 --2- 172.21.15.83:0/813415358 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6f394f090 0x55f6f39599e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.287+0000 7fb539000640 1 -- 172.21.15.83:0/813415358 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6f395a530 con 0x55f6f3954bf0 2024-04-24T08:51:25.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.287+0000 7fb539000640 1 --2- 172.21.15.83:0/813415358 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6f3954bf0 0x55f6f3957a80 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fb52c011020 tx=0x7fb52c005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a22b9f13331654e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:25.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.287+0000 7fb528c00640 1 -- 172.21.15.83:0/813415358 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb52c029040 con 0x55f6f3954bf0 2024-04-24T08:51:25.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.287+0000 7fb528c00640 1 -- 172.21.15.83:0/813415358 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb52c020050 con 0x55f6f3954bf0 2024-04-24T08:51:25.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.287+0000 7fb528c00640 1 -- 172.21.15.83:0/813415358 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb52c00b9e0 con 0x55f6f3954bf0 2024-04-24T08:51:25.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.288+0000 7fb53de0f740 1 -- 172.21.15.83:0/813415358 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6f3954bf0 msgr2=0x55f6f3957a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:25.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.288+0000 7fb53de0f740 1 --2- 172.21.15.83:0/813415358 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6f3954bf0 0x55f6f3957a80 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fb52c011020 tx=0x7fb52c005dd0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.288+0000 7fb53de0f740 1 -- 172.21.15.83:0/813415358 shutdown_connections 2024-04-24T08:51:25.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.288+0000 7fb53de0f740 1 --2- 172.21.15.83:0/813415358 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6f3954bf0 0x55f6f3957a80 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.288+0000 7fb53de0f740 1 --2- 172.21.15.83:0/813415358 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f6f394fa80 0x55f6f39544e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.288+0000 7fb53de0f740 1 --2- 172.21.15.83:0/813415358 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6f394f090 0x55f6f39599e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.288+0000 7fb53de0f740 1 -- 172.21.15.83:0/813415358 >> 172.21.15.83:0/813415358 conn(0x55f6f394a700 msgr2=0x55f6f394aae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:25.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.288+0000 7fb53de0f740 1 -- 172.21.15.83:0/813415358 shutdown_connections 2024-04-24T08:51:25.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.288+0000 7fb53de0f740 1 -- 172.21.15.83:0/813415358 wait complete. 2024-04-24T08:51:25.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.288+0000 7fb53de0f740 1 -- start start 2024-04-24T08:51:25.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.289+0000 7fb53de0f740 1 Processor -- start 2024-04-24T08:51:25.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.289+0000 7fb53de0f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6f394f090 0x55f6f395cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:25.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.289+0000 7fb53de0f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f6f394fa80 0x55f6f395d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:25.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.289+0000 7fb53de0f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6f3954bf0 0x55f6f395fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:25.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.289+0000 7fb53de0f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55f6f39672c0 con 0x55f6f3954bf0 2024-04-24T08:51:25.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.289+0000 7fb53de0f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f6f3966bf0 con 0x55f6f394f090 2024-04-24T08:51:25.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.289+0000 7fb53de0f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55f6f3967990 con 0x55f6f394fa80 2024-04-24T08:51:25.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.289+0000 7fb53de0f740 10 client.0 _ll_register_callbacks cb 0x7fb52c0089b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:51:25.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.289+0000 7fb539000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6f3954bf0 0x55f6f395fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:25.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.289+0000 7fb539000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6f3954bf0 0x55f6f395fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41458/0 (socket says 172.21.15.83:41458) 2024-04-24T08:51:25.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.289+0000 7fb539000640 1 -- 172.21.15.83:0/2418810402 learned_addr learned my addr 172.21.15.83:0/2418810402 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:51:25.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.290+0000 7fb539000640 1 --2- 172.21.15.83:0/2418810402 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6f394f090 0x55f6f395cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:25.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.290+0000 7fb533400640 1 --2- 172.21.15.83:0/2418810402 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f6f394fa80 0x55f6f395d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:25.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.290+0000 7fb539000640 1 -- 172.21.15.83:0/2418810402 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f6f394fa80 msgr2=0x55f6f395d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:51:25.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.290+0000 7fb539000640 1 --2- 172.21.15.83:0/2418810402 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f6f394fa80 0x55f6f395d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.290+0000 7fb539000640 1 -- 172.21.15.83:0/2418810402 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6f394f090 msgr2=0x55f6f395cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:51:25.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.290+0000 7fb539000640 1 --2- 172.21.15.83:0/2418810402 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6f394f090 0x55f6f395cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.290+0000 7fb539000640 1 -- 172.21.15.83:0/2418810402 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6f3960060 con 0x55f6f3954bf0 2024-04-24T08:51:25.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.290+0000 7fb539000640 1 --2- 172.21.15.83:0/2418810402 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6f3954bf0 0x55f6f395fa20 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fb52c008530 tx=0x7fb52c0088d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d67c4db59bbfc3c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:25.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.290+0000 7fb522000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:51:25.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.290+0000 7fb522000640 1 -- 172.21.15.83:0/2418810402 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb52c029070 con 0x55f6f3954bf0 2024-04-24T08:51:25.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.290+0000 7fb53de0f740 1 -- 172.21.15.83:0/2418810402 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb52c020050 con 0x55f6f3954bf0 2024-04-24T08:51:25.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.290+0000 7fb522000640 1 -- 172.21.15.83:0/2418810402 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9237+0+0 (secure 0 0 0) 0x7fb52c00b9e0 con 0x55f6f3954bf0 2024-04-24T08:51:25.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.291+0000 7fb522000640 1 -- 172.21.15.83:0/2418810402 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb52c00bbe0 con 0x55f6f3954bf0 2024-04-24T08:51:25.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.291+0000 7fb53de0f740 10 client.15519 fetch_fsmap learned FSMap version 27 2024-04-24T08:51:25.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.291+0000 7fb53de0f740 1 -- 172.21.15.83:0/2418810402 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f6f39d49e0 con 0x55f6f3954bf0 2024-04-24T08:51:25.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.291+0000 7fb522000640 1 -- 172.21.15.83:0/2418810402 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb52c020050 con 0x55f6f3954bf0 2024-04-24T08:51:25.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.291+0000 7fb522000640 1 -- 172.21.15.83:0/2418810402 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fb52c00f9b0 con 0x55f6f3954bf0 2024-04-24T08:51:25.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.291+0000 7fb53de0f740 10 client.15519 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T08:51:25.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.291+0000 7fb53de0f740 10 client.15519 resolve_mds: resolved name 'e' to daemon mds.e 2024-04-24T08:51:25.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.291+0000 7fb53de0f740 20 client.15519 populate_metadata read hostname 'smithi083' 2024-04-24T08:51:25.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.291+0000 7fb53de0f740 1 --2- 172.21.15.83:0/2418810402 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55f6f39d9380 0x55f6f39d8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:25.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.291+0000 7fb53de0f740 4 client.15519 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T08:51:25.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.291+0000 7fb53de0f740 1 -- 172.21.15.83:0/2418810402 --> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] -- command(tid 0: {"prefix": "status"}) -- 0x55f6f38bc3a0 con 0x55f6f39d9380 2024-04-24T08:51:25.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.292+0000 7fb533400640 1 --2- 172.21.15.83:0/2418810402 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55f6f39d9380 0x55f6f39d8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:25.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.293+0000 7fb533400640 1 --2- 172.21.15.83:0/2418810402 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55f6f39d9380 0x55f6f39d8090 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=74c73bd7353852d3 server_cookie=4ddf43bd4234a21b in_seq=0 out_seq=0 2024-04-24T08:51:25.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.293+0000 7fb522000640 10 client.15519 ms_handle_connect on v2:172.21.15.138:6834/1909219758 2024-04-24T08:51:25.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.294+0000 7fb522000640 1 -- 172.21.15.83:0/2418810402 <== mds.2 v2:172.21.15.138:6834/1909219758 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55f6f38bc3a0 con 0x55f6f39d9380 2024-04-24T08:51:25.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.294+0000 7fb522000640 10 client.15519 handle_command_reply: tid=0 2024-04-24T08:51:25.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.294+0000 7fb53de0f740 1 client.15519 shutdown 2024-04-24T08:51:25.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.294+0000 7fb53de0f740 1 -- 172.21.15.83:0/2418810402 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6f3954bf0 msgr2=0x55f6f395fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:25.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.294+0000 7fb53de0f740 1 --2- 172.21.15.83:0/2418810402 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6f3954bf0 0x55f6f395fa20 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fb52c008530 tx=0x7fb52c0088d0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.294+0000 7fb53de0f740 1 -- 172.21.15.83:0/2418810402 shutdown_connections 2024-04-24T08:51:25.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.294+0000 7fb53de0f740 1 --2- 172.21.15.83:0/2418810402 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55f6f39d9380 0x55f6f39d8090 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.294+0000 7fb53de0f740 1 --2- 172.21.15.83:0/2418810402 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6f3954bf0 0x55f6f395fa20 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.294+0000 7fb53de0f740 1 --2- 172.21.15.83:0/2418810402 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f6f394fa80 0x55f6f395d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.294+0000 7fb53de0f740 1 --2- 172.21.15.83:0/2418810402 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6f394f090 0x55f6f395cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.294+0000 7fb522000640 0 client.15519 ms_handle_reset on v2:172.21.15.138:6834/1909219758 2024-04-24T08:51:25.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.294+0000 7fb53de0f740 1 -- 172.21.15.83:0/2418810402 >> 172.21.15.83:0/2418810402 conn(0x55f6f394a700 msgr2=0x55f6f3945750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:25.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.294+0000 7fb53de0f740 1 -- 172.21.15.83:0/2418810402 shutdown_connections 2024-04-24T08:51:25.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.294+0000 7fb53de0f740 1 -- 172.21.15.83:0/2418810402 wait complete. 2024-04-24T08:51:25.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.294+0000 7fb53de0f740 20 client.15519 trim_cache size 0 max 16384 2024-04-24T08:51:25.297 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T08:51:25.297 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_fsid": "bd3e039e-0215-11ef-bc93-c7b262605968", 2024-04-24T08:51:25.297 INFO:teuthology.orchestra.run.smithi083.stdout: "whoami": 2, 2024-04-24T08:51:25.297 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 34451, 2024-04-24T08:51:25.297 INFO:teuthology.orchestra.run.smithi083.stdout: "want_state": "up:active", 2024-04-24T08:51:25.297 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "up:active", 2024-04-24T08:51:25.297 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs", 2024-04-24T08:51:25.297 INFO:teuthology.orchestra.run.smithi083.stdout: "rank_uptime": 143.26191901999999, 2024-04-24T08:51:25.297 INFO:teuthology.orchestra.run.smithi083.stdout: "mdsmap_epoch": 27, 2024-04-24T08:51:25.297 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch": 93, 2024-04-24T08:51:25.297 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch_barrier": 89, 2024-04-24T08:51:25.297 INFO:teuthology.orchestra.run.smithi083.stdout: "uptime": 212.240651074 2024-04-24T08:51:25.297 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T08:51:25.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.295+0000 7fb53b000640 1 -- 172.21.15.83:0/303649950 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fb5180a3bb0 msgr2=0x7fb5180a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:25.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.295+0000 7fb53b000640 1 --2- 172.21.15.83:0/303649950 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fb5180a3bb0 0x7fb5180a6070 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fb524098e70 tx=0x7fb524005f30 comp rx=0 tx=0).stop 2024-04-24T08:51:25.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.295+0000 7fb53b000640 1 -- 172.21.15.83:0/303649950 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb534104280 msgr2=0x7fb53419a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:25.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.295+0000 7fb53b000640 1 --2- 172.21.15.83:0/303649950 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb534104280 0x7fb53419a4d0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fb52400d770 tx=0x7fb52400dc30 comp rx=0 tx=0).stop 2024-04-24T08:51:25.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.295+0000 7fb53b000640 1 -- 172.21.15.83:0/303649950 shutdown_connections 2024-04-24T08:51:25.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.295+0000 7fb53b000640 1 --2- 172.21.15.83:0/303649950 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb534105e30 0x7fb5341a1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.295+0000 7fb53b000640 1 --2- 172.21.15.83:0/303649950 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb534105470 0x7fb53419aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.295+0000 7fb53b000640 1 --2- 172.21.15.83:0/303649950 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7fb5180a3bb0 0x7fb5180a6070 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.295+0000 7fb53b000640 1 --2- 172.21.15.83:0/303649950 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb534104280 0x7fb53419a4d0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.295+0000 7fb53b000640 1 -- 172.21.15.83:0/303649950 >> 172.21.15.83:0/303649950 conn(0x7fb5340ff7f0 msgr2=0x7fb5341087e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:25.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.295+0000 7fb53b000640 1 -- 172.21.15.83:0/303649950 shutdown_connections 2024-04-24T08:51:25.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.295+0000 7fb53b000640 1 -- 172.21.15.83:0/303649950 wait complete. 2024-04-24T08:51:25.313 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-04-24T08:51:25.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:25 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/1176763574' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:51:25.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:25 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/2189932798' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T08:51:25.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:25 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/1176763574' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:51:25.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:25 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/2189932798' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T08:51:25.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.516+0000 7f2ae8c00640 1 -- 172.21.15.83:0/2904110850 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ae4105cf0 msgr2=0x7f2ae410c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:25.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.516+0000 7f2ae8c00640 1 --2- 172.21.15.83:0/2904110850 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ae4105cf0 0x7f2ae410c580 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f2adc009ca0 tx=0x7f2adc02a8c0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.516+0000 7f2ae8c00640 1 -- 172.21.15.83:0/2904110850 shutdown_connections 2024-04-24T08:51:25.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.516+0000 7f2ae8c00640 1 --2- 172.21.15.83:0/2904110850 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ae4105cf0 0x7f2ae410c580 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.516+0000 7f2ae8c00640 1 --2- 172.21.15.83:0/2904110850 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2ae4105330 0x7f2ae41057b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.516+0000 7f2ae8c00640 1 --2- 172.21.15.83:0/2904110850 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ae4104140 0x7f2ae4104560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.516+0000 7f2ae8c00640 1 -- 172.21.15.83:0/2904110850 >> 172.21.15.83:0/2904110850 conn(0x7f2ae40ff6b0 msgr2=0x7f2ae4101af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:25.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.516+0000 7f2ae8c00640 1 -- 172.21.15.83:0/2904110850 shutdown_connections 2024-04-24T08:51:25.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.516+0000 7f2ae8c00640 1 -- 172.21.15.83:0/2904110850 wait complete. 2024-04-24T08:51:25.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.517+0000 7f2ae8c00640 1 Processor -- start 2024-04-24T08:51:25.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.518+0000 7f2ae8c00640 1 -- start start 2024-04-24T08:51:25.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.518+0000 7f2ae8c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ae4104140 0x7f2ae419a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:25.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.518+0000 7f2ae8c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ae4105330 0x7f2ae419a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:25.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.518+0000 7f2ae8c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2ae4105cf0 0x7f2ae41a1810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:25.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.518+0000 7f2ae8c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2ae410fcd0 con 0x7f2ae4105330 2024-04-24T08:51:25.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.518+0000 7f2ae8c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2ae410fb50 con 0x7f2ae4104140 2024-04-24T08:51:25.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.518+0000 7f2ae8c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2ae410fe50 con 0x7f2ae4105cf0 2024-04-24T08:51:25.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.518+0000 7f2ae2a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ae4105330 0x7f2ae419a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:25.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.518+0000 7f2ae2a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ae4105330 0x7f2ae419a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41484/0 (socket says 172.21.15.83:41484) 2024-04-24T08:51:25.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.518+0000 7f2ae2a00640 1 -- 172.21.15.83:0/2609417169 learned_addr learned my addr 172.21.15.83:0/2609417169 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:51:25.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.518+0000 7f2ae3400640 1 --2- 172.21.15.83:0/2609417169 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ae4104140 0x7f2ae419a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:25.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.518+0000 7f2ae3e00640 1 --2- 172.21.15.83:0/2609417169 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2ae4105cf0 0x7f2ae41a1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:25.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.519+0000 7f2ae2a00640 1 -- 172.21.15.83:0/2609417169 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2ae4105cf0 msgr2=0x7f2ae41a1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:25.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.519+0000 7f2ae2a00640 1 --2- 172.21.15.83:0/2609417169 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2ae4105cf0 0x7f2ae41a1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.519+0000 7f2ae2a00640 1 -- 172.21.15.83:0/2609417169 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ae4104140 msgr2=0x7f2ae419a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:25.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.519+0000 7f2ae2a00640 1 --2- 172.21.15.83:0/2609417169 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ae4104140 0x7f2ae419a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.519+0000 7f2ae2a00640 1 -- 172.21.15.83:0/2609417169 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ae41a1e80 con 0x7f2ae4105330 2024-04-24T08:51:25.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.519+0000 7f2ae2a00640 1 --2- 172.21.15.83:0/2609417169 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ae4105330 0x7f2ae419a7a0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f2ad000d770 tx=0x7f2ad000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:25.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.519+0000 7f2adbe00640 1 -- 172.21.15.83:0/2609417169 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad0017070 con 0x7f2ae4105330 2024-04-24T08:51:25.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.519+0000 7f2adbe00640 1 -- 172.21.15.83:0/2609417169 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2ad00054d0 con 0x7f2ae4105330 2024-04-24T08:51:25.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.519+0000 7f2ae8c00640 1 -- 172.21.15.83:0/2609417169 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ae41a2170 con 0x7f2ae4105330 2024-04-24T08:51:25.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.519+0000 7f2adbe00640 1 -- 172.21.15.83:0/2609417169 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad0004cf0 con 0x7f2ae4105330 2024-04-24T08:51:25.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.519+0000 7f2ae8c00640 1 -- 172.21.15.83:0/2609417169 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ae41a26c0 con 0x7f2ae4105330 2024-04-24T08:51:25.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.522+0000 7f2aed03d740 1 Processor -- start 2024-04-24T08:51:25.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.523+0000 7f2aed03d740 1 -- start start 2024-04-24T08:51:25.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.523+0000 7f2aed03d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5602cd371260 0x5602cd371680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:25.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.523+0000 7f2aed03d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602cd371c50 0x5602cd376750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:25.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.523+0000 7f2aed03d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602cd376dc0 0x5602cd379250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:25.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.523+0000 7f2aed03d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5602cd2d6b10 con 0x5602cd371c50 2024-04-24T08:51:25.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.523+0000 7f2aed03d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5602cd27d2b0 con 0x5602cd371260 2024-04-24T08:51:25.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.523+0000 7f2aed03d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5602cd2d2ad0 con 0x5602cd376dc0 2024-04-24T08:51:25.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.523+0000 7f2ae3400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602cd371c50 0x5602cd376750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:25.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.523+0000 7f2ae3400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602cd371c50 0x5602cd376750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41488/0 (socket says 172.21.15.83:41488) 2024-04-24T08:51:25.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.523+0000 7f2ae3400640 1 -- 172.21.15.83:0/1095528336 learned_addr learned my addr 172.21.15.83:0/1095528336 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:51:25.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.524+0000 7f2ae3e00640 1 --2- 172.21.15.83:0/1095528336 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5602cd371260 0x5602cd371680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:25.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.524+0000 7f2adbe00640 1 -- 172.21.15.83:0/2609417169 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f2ad0004e90 con 0x7f2ae4105330 2024-04-24T08:51:25.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.524+0000 7f2ae3400640 1 -- 172.21.15.83:0/1095528336 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602cd376dc0 msgr2=0x5602cd379250 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T08:51:25.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.524+0000 7f2ae3400640 1 --2- 172.21.15.83:0/1095528336 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602cd376dc0 0x5602cd379250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.524+0000 7f2ae3400640 1 -- 172.21.15.83:0/1095528336 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5602cd371260 msgr2=0x5602cd371680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:51:25.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.524+0000 7f2ae3400640 1 --2- 172.21.15.83:0/1095528336 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5602cd371260 0x5602cd371680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.524+0000 7f2ae3400640 1 -- 172.21.15.83:0/1095528336 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602ccfe92d0 con 0x5602cd371c50 2024-04-24T08:51:25.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.524+0000 7f2ae3e00640 1 --2- 172.21.15.83:0/1095528336 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5602cd371260 0x5602cd371680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:51:25.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.524+0000 7f2ae3400640 1 --2- 172.21.15.83:0/1095528336 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602cd371c50 0x5602cd376750 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f2ad400cc40 tx=0x7f2ad400e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85c949cfacf6d941 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:25.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.525+0000 7f2acfe00640 1 -- 172.21.15.83:0/1095528336 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad4014070 con 0x5602cd371c50 2024-04-24T08:51:25.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.525+0000 7f2acfe00640 1 -- 172.21.15.83:0/1095528336 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2ad40054d0 con 0x5602cd371c50 2024-04-24T08:51:25.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.525+0000 7f2acfe00640 1 -- 172.21.15.83:0/1095528336 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad4004cf0 con 0x5602cd371c50 2024-04-24T08:51:25.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.525+0000 7f2adbe00640 1 --2- 172.21.15.83:0/2609417169 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f2ac80a3c30 0x7f2ac80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:25.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.525+0000 7f2aed03d740 1 -- 172.21.15.83:0/1095528336 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602cd371c50 msgr2=0x5602cd376750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:25.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.525+0000 7f2aed03d740 1 --2- 172.21.15.83:0/1095528336 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602cd371c50 0x5602cd376750 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f2ad400cc40 tx=0x7f2ad400e720 comp rx=0 tx=0).stop 2024-04-24T08:51:25.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.526+0000 7f2adbe00640 1 -- 172.21.15.83:0/2609417169 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9237+0+0 (secure 0 0 0) 0x7f2ad00d0290 con 0x7f2ae4105330 2024-04-24T08:51:25.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.526+0000 7f2aed03d740 1 -- 172.21.15.83:0/1095528336 shutdown_connections 2024-04-24T08:51:25.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.526+0000 7f2aed03d740 1 --2- 172.21.15.83:0/1095528336 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602cd376dc0 0x5602cd379250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.526+0000 7f2aed03d740 1 --2- 172.21.15.83:0/1095528336 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602cd371c50 0x5602cd376750 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.526+0000 7f2aed03d740 1 --2- 172.21.15.83:0/1095528336 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5602cd371260 0x5602cd371680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.526+0000 7f2aed03d740 1 -- 172.21.15.83:0/1095528336 >> 172.21.15.83:0/1095528336 conn(0x5602cd36c8d0 msgr2=0x5602cd36ccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:25.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.526+0000 7f2ae3e00640 1 --2- 172.21.15.83:0/2609417169 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f2ac80a3c30 0x7f2ac80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:25.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.526+0000 7f2aed03d740 1 -- 172.21.15.83:0/1095528336 shutdown_connections 2024-04-24T08:51:25.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.526+0000 7f2aed03d740 1 -- 172.21.15.83:0/1095528336 wait complete. 2024-04-24T08:51:25.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.526+0000 7f2aed03d740 1 -- start start 2024-04-24T08:51:25.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.527+0000 7f2aed03d740 1 Processor -- start 2024-04-24T08:51:25.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.527+0000 7f2ae3e00640 1 --2- 172.21.15.83:0/2609417169 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f2ac80a3c30 0x7f2ac80a60f0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x5602cd3724b0 tx=0x7f2adc0093f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:25.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.527+0000 7f2aed03d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5602cd371260 0x5602cd37b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:25.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.527+0000 7f2aed03d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602cd371c50 0x5602cd37bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:25.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.527+0000 7f2aed03d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602cd376dc0 0x5602cd382c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:25.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.527+0000 7f2aed03d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5602cd379d70 con 0x5602cd376dc0 2024-04-24T08:51:25.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.527+0000 7f2aed03d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5602cd290590 con 0x5602cd371260 2024-04-24T08:51:25.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.527+0000 7f2aed03d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5602cd379ef0 con 0x5602cd371c50 2024-04-24T08:51:25.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.527+0000 7f2aed03d740 10 client.0 _ll_register_callbacks cb 0x5602cd2733a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:51:25.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.527+0000 7f2ae3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602cd376dc0 0x5602cd382c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:25.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.527+0000 7f2ae3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602cd376dc0 0x5602cd382c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41498/0 (socket says 172.21.15.83:41498) 2024-04-24T08:51:25.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.527+0000 7f2ae3e00640 1 -- 172.21.15.83:0/1066484882 learned_addr learned my addr 172.21.15.83:0/1066484882 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:51:25.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.528+0000 7f2ae2a00640 1 --2- 172.21.15.83:0/1066484882 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602cd371c50 0x5602cd37bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:25.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.528+0000 7f2ae3400640 1 --2- 172.21.15.83:0/1066484882 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5602cd371260 0x5602cd37b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:25.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.528+0000 7f2ae3e00640 1 -- 172.21.15.83:0/1066484882 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602cd371c50 msgr2=0x5602cd37bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:51:25.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.528+0000 7f2ae3e00640 1 --2- 172.21.15.83:0/1066484882 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602cd371c50 0x5602cd37bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.528+0000 7f2ae3e00640 1 -- 172.21.15.83:0/1066484882 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5602cd371260 msgr2=0x5602cd37b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:51:25.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.528+0000 7f2ae3e00640 1 --2- 172.21.15.83:0/1066484882 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5602cd371260 0x5602cd37b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.528+0000 7f2ae3e00640 1 -- 172.21.15.83:0/1066484882 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602cd0b36b0 con 0x5602cd376dc0 2024-04-24T08:51:25.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.528+0000 7f2ae3e00640 1 --2- 172.21.15.83:0/1066484882 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602cd376dc0 0x5602cd382c80 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f2adc035000 tx=0x7f2adc0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24acd4328a8bd704 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:25.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.528+0000 7f2accc00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:51:25.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.528+0000 7f2accc00640 1 -- 172.21.15.83:0/1066484882 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2adc04e070 con 0x5602cd376dc0 2024-04-24T08:51:25.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.529+0000 7f2accc00640 1 -- 172.21.15.83:0/1066484882 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9237+0+0 (secure 0 0 0) 0x7f2adc003e00 con 0x5602cd376dc0 2024-04-24T08:51:25.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.529+0000 7f2aed03d740 1 -- 172.21.15.83:0/1066484882 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ad40054d0 con 0x5602cd376dc0 2024-04-24T08:51:25.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.529+0000 7f2accc00640 1 -- 172.21.15.83:0/1066484882 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2adc05a050 con 0x5602cd376dc0 2024-04-24T08:51:25.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.529+0000 7f2aed03d740 10 client.15531 fetch_fsmap learned FSMap version 27 2024-04-24T08:51:25.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.529+0000 7f2aed03d740 1 -- 172.21.15.83:0/1066484882 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5602cd385f60 con 0x5602cd376dc0 2024-04-24T08:51:25.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.529+0000 7f2accc00640 1 -- 172.21.15.83:0/1066484882 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2ad40054d0 con 0x5602cd376dc0 2024-04-24T08:51:25.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.529+0000 7f2accc00640 1 -- 172.21.15.83:0/1066484882 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f2adc043040 con 0x5602cd376dc0 2024-04-24T08:51:25.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.529+0000 7f2aed03d740 10 client.15531 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T08:51:25.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.529+0000 7f2aed03d740 10 client.15531 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-24T08:51:25.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.530+0000 7f2aed03d740 20 client.15531 populate_metadata read hostname 'smithi083' 2024-04-24T08:51:25.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.530+0000 7f2aed03d740 1 --2- 172.21.15.83:0/1066484882 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x5602cd388640 0x5602cd389ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:25.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.530+0000 7f2aed03d740 4 client.15531 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:51:25.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.530+0000 7f2aed03d740 1 -- 172.21.15.83:0/1066484882 --> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5602cd2e4530 con 0x5602cd388640 2024-04-24T08:51:25.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.530+0000 7f2ae3400640 1 --2- 172.21.15.83:0/1066484882 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x5602cd388640 0x5602cd389ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:25.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.531+0000 7f2ae3400640 1 --2- 172.21.15.83:0/1066484882 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x5602cd388640 0x5602cd389ab0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=302549a97f79bc2e server_cookie=b3942b863b1d2dc6 in_seq=0 out_seq=0 2024-04-24T08:51:25.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.531+0000 7f2accc00640 10 client.15531 ms_handle_connect on v2:172.21.15.138:6836/900566754 2024-04-24T08:51:25.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.533+0000 7f2accc00640 1 -- 172.21.15.83:0/1066484882 <== mds.1 v2:172.21.15.138:6836/900566754 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5602cd2e4530 con 0x5602cd388640 2024-04-24T08:51:25.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.533+0000 7f2accc00640 10 client.15531 handle_command_reply: tid=0 2024-04-24T08:51:25.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.533+0000 7f2aed03d740 1 client.15531 shutdown 2024-04-24T08:51:25.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.534+0000 7f2aed03d740 1 -- 172.21.15.83:0/1066484882 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602cd376dc0 msgr2=0x5602cd382c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:25.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.534+0000 7f2aed03d740 1 --2- 172.21.15.83:0/1066484882 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602cd376dc0 0x5602cd382c80 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f2adc035000 tx=0x7f2adc0095c0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.534+0000 7f2aed03d740 1 -- 172.21.15.83:0/1066484882 shutdown_connections 2024-04-24T08:51:25.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.534+0000 7f2aed03d740 1 --2- 172.21.15.83:0/1066484882 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602cd376dc0 0x5602cd382c80 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.534+0000 7f2aed03d740 1 --2- 172.21.15.83:0/1066484882 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x5602cd388640 0x5602cd389ab0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.534+0000 7f2aed03d740 1 --2- 172.21.15.83:0/1066484882 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602cd371c50 0x5602cd37bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.534+0000 7f2aed03d740 1 --2- 172.21.15.83:0/1066484882 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5602cd371260 0x5602cd37b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.534+0000 7f2accc00640 0 client.15531 ms_handle_reset on v2:172.21.15.138:6836/900566754 2024-04-24T08:51:25.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.534+0000 7f2aed03d740 1 -- 172.21.15.83:0/1066484882 >> 172.21.15.83:0/1066484882 conn(0x5602cd36c8d0 msgr2=0x5602cd36b970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:25.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.534+0000 7f2aed03d740 1 -- 172.21.15.83:0/1066484882 shutdown_connections 2024-04-24T08:51:25.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.534+0000 7f2aed03d740 1 -- 172.21.15.83:0/1066484882 wait complete. 2024-04-24T08:51:25.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.534+0000 7f2aed03d740 20 client.15531 trim_cache size 0 max 16384 2024-04-24T08:51:25.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.554+0000 7f2aed03d740 1 Processor -- start 2024-04-24T08:51:25.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.554+0000 7f2aed03d740 1 -- start start 2024-04-24T08:51:25.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.554+0000 7f2aed03d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602cd371260 0x5602cd36c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:25.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.554+0000 7f2aed03d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602cd371c50 0x5602cd37ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:25.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.554+0000 7f2aed03d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5602cd376dc0 0x5602cd382ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:25.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.554+0000 7f2aed03d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5602cd387df0 con 0x5602cd371c50 2024-04-24T08:51:25.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.554+0000 7f2aed03d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5602cd3884c0 con 0x5602cd376dc0 2024-04-24T08:51:25.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.554+0000 7f2aed03d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5602cd388b90 con 0x5602cd371260 2024-04-24T08:51:25.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.554+0000 7f2ae2a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602cd371c50 0x5602cd37ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:25.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.554+0000 7f2ae2a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602cd371c50 0x5602cd37ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41506/0 (socket says 172.21.15.83:41506) 2024-04-24T08:51:25.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.554+0000 7f2ae2a00640 1 -- 172.21.15.83:0/3326108534 learned_addr learned my addr 172.21.15.83:0/3326108534 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:51:25.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.555+0000 7f2ae3400640 1 --2- 172.21.15.83:0/3326108534 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602cd371260 0x5602cd36c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:25.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.555+0000 7f2ae2a00640 1 -- 172.21.15.83:0/3326108534 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602cd371260 msgr2=0x5602cd36c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:51:25.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.555+0000 7f2ae2a00640 1 --2- 172.21.15.83:0/3326108534 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602cd371260 0x5602cd36c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.555+0000 7f2ae2a00640 1 -- 172.21.15.83:0/3326108534 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5602cd376dc0 msgr2=0x5602cd382ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:51:25.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.555+0000 7f2ae2a00640 1 --2- 172.21.15.83:0/3326108534 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5602cd376dc0 0x5602cd382ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.555+0000 7f2ae2a00640 1 -- 172.21.15.83:0/3326108534 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602cd36c700 con 0x5602cd371c50 2024-04-24T08:51:25.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.555+0000 7f2ae3400640 1 --2- 172.21.15.83:0/3326108534 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602cd371260 0x5602cd36c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:51:25.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.555+0000 7f2ae2a00640 1 --2- 172.21.15.83:0/3326108534 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602cd371c50 0x5602cd37ad20 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f2ad00985e0 tx=0x7f2ad009bcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c83a7395ca28b16 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:25.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.555+0000 7f2acf400640 1 -- 172.21.15.83:0/3326108534 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad00cde60 con 0x5602cd371c50 2024-04-24T08:51:25.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.555+0000 7f2acf400640 1 -- 172.21.15.83:0/3326108534 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2ad0009c30 con 0x5602cd371c50 2024-04-24T08:51:25.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.555+0000 7f2acf400640 1 -- 172.21.15.83:0/3326108534 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad0020040 con 0x5602cd371c50 2024-04-24T08:51:25.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.556+0000 7f2aed03d740 1 -- 172.21.15.83:0/3326108534 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602cd371c50 msgr2=0x5602cd37ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:25.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.556+0000 7f2aed03d740 1 --2- 172.21.15.83:0/3326108534 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602cd371c50 0x5602cd37ad20 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f2ad00985e0 tx=0x7f2ad009bcc0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.556+0000 7f2aed03d740 1 -- 172.21.15.83:0/3326108534 shutdown_connections 2024-04-24T08:51:25.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.556+0000 7f2aed03d740 1 --2- 172.21.15.83:0/3326108534 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5602cd376dc0 0x5602cd382ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.556+0000 7f2aed03d740 1 --2- 172.21.15.83:0/3326108534 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602cd371c50 0x5602cd37ad20 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.556+0000 7f2aed03d740 1 --2- 172.21.15.83:0/3326108534 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602cd371260 0x5602cd36c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.556+0000 7f2aed03d740 1 -- 172.21.15.83:0/3326108534 >> 172.21.15.83:0/3326108534 conn(0x5602cd36c8d0 msgr2=0x5602cd36ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:25.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.556+0000 7f2aed03d740 1 -- 172.21.15.83:0/3326108534 shutdown_connections 2024-04-24T08:51:25.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.556+0000 7f2aed03d740 1 -- 172.21.15.83:0/3326108534 wait complete. 2024-04-24T08:51:25.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.556+0000 7f2aed03d740 1 -- start start 2024-04-24T08:51:25.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.557+0000 7f2aed03d740 1 Processor -- start 2024-04-24T08:51:25.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.557+0000 7f2aed03d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602cd371260 0x5602cd37edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:25.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.557+0000 7f2aed03d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602cd376dc0 0x5602cd37f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:25.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.557+0000 7f2aed03d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5602cd381fb0 0x5602cd382430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:25.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.557+0000 7f2aed03d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5602cd389930 con 0x5602cd376dc0 2024-04-24T08:51:25.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.557+0000 7f2aed03d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5602cd389260 con 0x5602cd381fb0 2024-04-24T08:51:25.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.557+0000 7f2aed03d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5602cd38a000 con 0x5602cd371260 2024-04-24T08:51:25.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.557+0000 7f2aed03d740 10 client.0 _ll_register_callbacks cb 0x7f2adc031c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:51:25.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.557+0000 7f2ae2a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602cd376dc0 0x5602cd37f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:25.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.557+0000 7f2ae2a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602cd376dc0 0x5602cd37f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41512/0 (socket says 172.21.15.83:41512) 2024-04-24T08:51:25.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.557+0000 7f2ae2a00640 1 -- 172.21.15.83:0/3052553014 learned_addr learned my addr 172.21.15.83:0/3052553014 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:51:25.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.558+0000 7f2ae3400640 1 --2- 172.21.15.83:0/3052553014 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602cd371260 0x5602cd37edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:25.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.558+0000 7f2ae2a00640 1 -- 172.21.15.83:0/3052553014 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602cd371260 msgr2=0x5602cd37edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:51:25.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.558+0000 7f2ae2a00640 1 --2- 172.21.15.83:0/3052553014 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602cd371260 0x5602cd37edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.558+0000 7f2ae2a00640 1 -- 172.21.15.83:0/3052553014 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5602cd381fb0 msgr2=0x5602cd382430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:51:25.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.558+0000 7f2ae3e00640 1 --2- 172.21.15.83:0/3052553014 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5602cd381fb0 0x5602cd382430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:25.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.558+0000 7f2ae2a00640 1 --2- 172.21.15.83:0/3052553014 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5602cd381fb0 0x5602cd382430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.558+0000 7f2ae2a00640 1 -- 172.21.15.83:0/3052553014 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602cd3829a0 con 0x5602cd376dc0 2024-04-24T08:51:25.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.558+0000 7f2ae3e00640 1 --2- 172.21.15.83:0/3052553014 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5602cd381fb0 0x5602cd382430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:51:25.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.558+0000 7f2ae2a00640 1 --2- 172.21.15.83:0/3052553014 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602cd376dc0 0x5602cd37f310 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f2ad0009ed0 tx=0x7f2ad009c180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39ef367228031353 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:25.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.558+0000 7f2accc00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:51:25.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.558+0000 7f2accc00640 1 -- 172.21.15.83:0/3052553014 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad00bb020 con 0x5602cd376dc0 2024-04-24T08:51:25.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.558+0000 7f2accc00640 1 -- 172.21.15.83:0/3052553014 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9237+0+0 (secure 0 0 0) 0x7f2ad00a0120 con 0x5602cd376dc0 2024-04-24T08:51:25.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.558+0000 7f2aed03d740 1 -- 172.21.15.83:0/3052553014 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5602cd2de3a0 con 0x5602cd376dc0 2024-04-24T08:51:25.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.559+0000 7f2accc00640 1 -- 172.21.15.83:0/3052553014 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2ad00cde60 con 0x5602cd376dc0 2024-04-24T08:51:25.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.559+0000 7f2aed03d740 10 client.15537 fetch_fsmap learned FSMap version 27 2024-04-24T08:51:25.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.559+0000 7f2aed03d740 1 -- 172.21.15.83:0/3052553014 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5602cd3f77b0 con 0x5602cd376dc0 2024-04-24T08:51:25.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.559+0000 7f2accc00640 1 -- 172.21.15.83:0/3052553014 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2ad007f020 con 0x5602cd376dc0 2024-04-24T08:51:25.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.559+0000 7f2accc00640 1 -- 172.21.15.83:0/3052553014 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f2ad009b040 con 0x5602cd376dc0 2024-04-24T08:51:25.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.559+0000 7f2aed03d740 10 client.15537 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T08:51:25.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.559+0000 7f2aed03d740 10 client.15537 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-24T08:51:25.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.559+0000 7f2aed03d740 20 client.15537 populate_metadata read hostname 'smithi083' 2024-04-24T08:51:25.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.559+0000 7f2aed03d740 1 --2- 172.21.15.83:0/3052553014 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x5602cd3fbf80 0x5602cd3f8a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:25.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.559+0000 7f2aed03d740 4 client.15537 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T08:51:25.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.559+0000 7f2aed03d740 1 -- 172.21.15.83:0/3052553014 --> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] -- command(tid 0: {"prefix": "status"}) -- 0x5602cd25a860 con 0x5602cd3fbf80 2024-04-24T08:51:25.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.560+0000 7f2ae3400640 1 --2- 172.21.15.83:0/3052553014 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x5602cd3fbf80 0x5602cd3f8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:25.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.561+0000 7f2ae3400640 1 --2- 172.21.15.83:0/3052553014 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x5602cd3fbf80 0x5602cd3f8a90 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1a5c8d18faf4f0eb server_cookie=1d8c476453157083 in_seq=0 out_seq=0 2024-04-24T08:51:25.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.561+0000 7f2accc00640 10 client.15537 ms_handle_connect on v2:172.21.15.138:6836/900566754 2024-04-24T08:51:25.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.562+0000 7f2accc00640 1 -- 172.21.15.83:0/3052553014 <== mds.1 v2:172.21.15.138:6836/900566754 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x5602cd25a860 con 0x5602cd3fbf80 2024-04-24T08:51:25.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.562+0000 7f2accc00640 10 client.15537 handle_command_reply: tid=0 2024-04-24T08:51:25.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.562+0000 7f2aed03d740 1 client.15537 shutdown 2024-04-24T08:51:25.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.562+0000 7f2aed03d740 1 -- 172.21.15.83:0/3052553014 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602cd376dc0 msgr2=0x5602cd37f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:25.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.562+0000 7f2aed03d740 1 --2- 172.21.15.83:0/3052553014 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602cd376dc0 0x5602cd37f310 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f2ad0009ed0 tx=0x7f2ad009c180 comp rx=0 tx=0).stop 2024-04-24T08:51:25.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.562+0000 7f2aed03d740 1 -- 172.21.15.83:0/3052553014 shutdown_connections 2024-04-24T08:51:25.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.562+0000 7f2aed03d740 1 --2- 172.21.15.83:0/3052553014 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5602cd381fb0 0x5602cd382430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.562+0000 7f2aed03d740 1 --2- 172.21.15.83:0/3052553014 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602cd376dc0 0x5602cd37f310 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.562+0000 7f2aed03d740 1 --2- 172.21.15.83:0/3052553014 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x5602cd3fbf80 0x5602cd3f8a90 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.562+0000 7f2aed03d740 1 --2- 172.21.15.83:0/3052553014 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602cd371260 0x5602cd37edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.562+0000 7f2aed03d740 1 -- 172.21.15.83:0/3052553014 >> 172.21.15.83:0/3052553014 conn(0x5602cd36c8d0 msgr2=0x5602cd37b670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:25.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.562+0000 7f2accc00640 0 client.15537 ms_handle_reset on v2:172.21.15.138:6836/900566754 2024-04-24T08:51:25.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.562+0000 7f2aed03d740 1 -- 172.21.15.83:0/3052553014 shutdown_connections 2024-04-24T08:51:25.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.562+0000 7f2aed03d740 1 -- 172.21.15.83:0/3052553014 wait complete. 2024-04-24T08:51:25.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.562+0000 7f2aed03d740 20 client.15537 trim_cache size 0 max 16384 2024-04-24T08:51:25.565 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T08:51:25.565 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_fsid": "bd3e039e-0215-11ef-bc93-c7b262605968", 2024-04-24T08:51:25.566 INFO:teuthology.orchestra.run.smithi083.stdout: "whoami": 1, 2024-04-24T08:51:25.566 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 34493, 2024-04-24T08:51:25.566 INFO:teuthology.orchestra.run.smithi083.stdout: "want_state": "up:active", 2024-04-24T08:51:25.566 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "up:active", 2024-04-24T08:51:25.566 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs", 2024-04-24T08:51:25.566 INFO:teuthology.orchestra.run.smithi083.stdout: "rank_uptime": 144.543075071, 2024-04-24T08:51:25.566 INFO:teuthology.orchestra.run.smithi083.stdout: "mdsmap_epoch": 27, 2024-04-24T08:51:25.566 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch": 93, 2024-04-24T08:51:25.566 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch_barrier": 89, 2024-04-24T08:51:25.566 INFO:teuthology.orchestra.run.smithi083.stdout: "uptime": 200.185539097 2024-04-24T08:51:25.566 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T08:51:25.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.563+0000 7f2ae8c00640 1 -- 172.21.15.83:0/2609417169 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f2ac80a3c30 msgr2=0x7f2ac80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:25.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.563+0000 7f2ae8c00640 1 --2- 172.21.15.83:0/2609417169 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f2ac80a3c30 0x7f2ac80a60f0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x5602cd3724b0 tx=0x7f2adc0093f0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.563+0000 7f2ae8c00640 1 -- 172.21.15.83:0/2609417169 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ae4105330 msgr2=0x7f2ae419a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:25.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.563+0000 7f2ae8c00640 1 --2- 172.21.15.83:0/2609417169 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ae4105330 0x7f2ae419a7a0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f2ad000d770 tx=0x7f2ad000dc30 comp rx=0 tx=0).stop 2024-04-24T08:51:25.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.563+0000 7f2ae8c00640 1 -- 172.21.15.83:0/2609417169 shutdown_connections 2024-04-24T08:51:25.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.563+0000 7f2ae8c00640 1 --2- 172.21.15.83:0/2609417169 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2ae4105cf0 0x7f2ae41a1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.563+0000 7f2ae8c00640 1 --2- 172.21.15.83:0/2609417169 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f2ac80a3c30 0x7f2ac80a60f0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.563+0000 7f2ae8c00640 1 --2- 172.21.15.83:0/2609417169 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ae4105330 0x7f2ae419a7a0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.563+0000 7f2ae8c00640 1 --2- 172.21.15.83:0/2609417169 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ae4104140 0x7f2ae419a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.563+0000 7f2ae8c00640 1 -- 172.21.15.83:0/2609417169 >> 172.21.15.83:0/2609417169 conn(0x7f2ae40ff6b0 msgr2=0x7f2ae41085d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:25.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.563+0000 7f2ae8c00640 1 -- 172.21.15.83:0/2609417169 shutdown_connections 2024-04-24T08:51:25.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.563+0000 7f2ae8c00640 1 -- 172.21.15.83:0/2609417169 wait complete. 2024-04-24T08:51:25.582 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-24T08:51:25.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.783+0000 7f4b43000640 1 -- 172.21.15.83:0/4192553679 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b3c103a40 msgr2=0x7f4b3c10a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:25.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.783+0000 7f4b43000640 1 --2- 172.21.15.83:0/4192553679 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b3c103a40 0x7f4b3c10a360 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f4b34009ca0 tx=0x7f4b3402a8c0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.783+0000 7f4b43000640 1 -- 172.21.15.83:0/4192553679 shutdown_connections 2024-04-24T08:51:25.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.783+0000 7f4b43000640 1 --2- 172.21.15.83:0/4192553679 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b3c103a40 0x7f4b3c10a360 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.783+0000 7f4b43000640 1 --2- 172.21.15.83:0/4192553679 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4b3c103080 0x7f4b3c103500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.783+0000 7f4b43000640 1 --2- 172.21.15.83:0/4192553679 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b3c101f30 0x7f4b3c102350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.783+0000 7f4b43000640 1 -- 172.21.15.83:0/4192553679 >> 172.21.15.83:0/4192553679 conn(0x7f4b3c0fd4a0 msgr2=0x7f4b3c0ff8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:25.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.783+0000 7f4b43000640 1 -- 172.21.15.83:0/4192553679 shutdown_connections 2024-04-24T08:51:25.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.783+0000 7f4b43000640 1 -- 172.21.15.83:0/4192553679 wait complete. 2024-04-24T08:51:25.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.784+0000 7f4b43000640 1 Processor -- start 2024-04-24T08:51:25.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.784+0000 7f4b43000640 1 -- start start 2024-04-24T08:51:25.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.785+0000 7f4b43000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4b3c101f30 0x7f4b3c1980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:25.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.785+0000 7f4b43000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b3c103080 0x7f4b3c198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:25.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.785+0000 7f4b43000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b3c103a40 0x7f4b3c19f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:25.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.785+0000 7f4b43000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4b3c10dae0 con 0x7f4b3c103080 2024-04-24T08:51:25.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.785+0000 7f4b43000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4b3c10d960 con 0x7f4b3c103a40 2024-04-24T08:51:25.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.785+0000 7f4b43000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4b3c10dc60 con 0x7f4b3c101f30 2024-04-24T08:51:25.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.785+0000 7f4b41200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b3c103080 0x7f4b3c198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:25.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.785+0000 7f4b41200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b3c103080 0x7f4b3c198600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41524/0 (socket says 172.21.15.83:41524) 2024-04-24T08:51:25.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.785+0000 7f4b41200640 1 -- 172.21.15.83:0/3045874040 learned_addr learned my addr 172.21.15.83:0/3045874040 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:51:25.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.785+0000 7f4b41c00640 1 --2- 172.21.15.83:0/3045874040 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4b3c101f30 0x7f4b3c1980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:25.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.785+0000 7f4b42600640 1 --2- 172.21.15.83:0/3045874040 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b3c103a40 0x7f4b3c19f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:25.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.785+0000 7f4b41200640 1 -- 172.21.15.83:0/3045874040 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4b3c101f30 msgr2=0x7f4b3c1980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:25.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.785+0000 7f4b41200640 1 --2- 172.21.15.83:0/3045874040 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4b3c101f30 0x7f4b3c1980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.786+0000 7f4b41200640 1 -- 172.21.15.83:0/3045874040 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b3c103a40 msgr2=0x7f4b3c19f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:25.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.786+0000 7f4b41200640 1 --2- 172.21.15.83:0/3045874040 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b3c103a40 0x7f4b3c19f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.786+0000 7f4b41200640 1 -- 172.21.15.83:0/3045874040 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b3c19fce0 con 0x7f4b3c103080 2024-04-24T08:51:25.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.786+0000 7f4b41200640 1 --2- 172.21.15.83:0/3045874040 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b3c103080 0x7f4b3c198600 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f4b2800e9c0 tx=0x7f4b2800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:25.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.786+0000 7f4b3a000640 1 -- 172.21.15.83:0/3045874040 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b2800cd80 con 0x7f4b3c103080 2024-04-24T08:51:25.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.786+0000 7f4b3a000640 1 -- 172.21.15.83:0/3045874040 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4b280035f0 con 0x7f4b3c103080 2024-04-24T08:51:25.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.786+0000 7f4b43000640 1 -- 172.21.15.83:0/3045874040 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b3c19ffd0 con 0x7f4b3c103080 2024-04-24T08:51:25.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.786+0000 7f4b3a000640 1 -- 172.21.15.83:0/3045874040 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b28010690 con 0x7f4b3c103080 2024-04-24T08:51:25.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.786+0000 7f4b43000640 1 -- 172.21.15.83:0/3045874040 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b3c1a0440 con 0x7f4b3c103080 2024-04-24T08:51:25.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.790+0000 7f4b4742b740 1 Processor -- start 2024-04-24T08:51:25.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.790+0000 7f4b4742b740 1 -- start start 2024-04-24T08:51:25.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.790+0000 7f4b4742b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e700b58260 0x55e700b58680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:25.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.790+0000 7f4b4742b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e700b58c50 0x55e700b5d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:25.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.790+0000 7f4b4742b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e700b5ddc0 0x55e700b60250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:25.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.790+0000 7f4b4742b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e700abdb10 con 0x55e700b58c50 2024-04-24T08:51:25.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.790+0000 7f4b4742b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e700a642b0 con 0x55e700b58260 2024-04-24T08:51:25.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.790+0000 7f4b4742b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55e700ab9ad0 con 0x55e700b5ddc0 2024-04-24T08:51:25.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.790+0000 7f4b3a000640 1 -- 172.21.15.83:0/3045874040 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f4b280037a0 con 0x7f4b3c103080 2024-04-24T08:51:25.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.790+0000 7f4b41c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e700b58c50 0x55e700b5d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:25.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.790+0000 7f4b41c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e700b58c50 0x55e700b5d750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41540/0 (socket says 172.21.15.83:41540) 2024-04-24T08:51:25.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.790+0000 7f4b41c00640 1 -- 172.21.15.83:0/2990969769 learned_addr learned my addr 172.21.15.83:0/2990969769 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:51:25.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.791+0000 7f4b42600640 1 --2- 172.21.15.83:0/2990969769 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e700b58260 0x55e700b58680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:25.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.791+0000 7f4b41c00640 1 -- 172.21.15.83:0/2990969769 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e700b5ddc0 msgr2=0x55e700b60250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:51:25.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.791+0000 7f4b41200640 1 --2- 172.21.15.83:0/2990969769 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e700b5ddc0 0x55e700b60250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:25.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.791+0000 7f4b41c00640 1 --2- 172.21.15.83:0/2990969769 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e700b5ddc0 0x55e700b60250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.791+0000 7f4b41c00640 1 -- 172.21.15.83:0/2990969769 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e700b58260 msgr2=0x55e700b58680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:51:25.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.791+0000 7f4b41c00640 1 --2- 172.21.15.83:0/2990969769 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e700b58260 0x55e700b58680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.791+0000 7f4b41c00640 1 -- 172.21.15.83:0/2990969769 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7007d02d0 con 0x55e700b58c50 2024-04-24T08:51:25.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.791+0000 7f4b41200640 1 --2- 172.21.15.83:0/2990969769 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e700b5ddc0 0x55e700b60250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:51:25.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.791+0000 7f4b42600640 1 --2- 172.21.15.83:0/2990969769 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e700b58260 0x55e700b58680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:51:25.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.791+0000 7f4b41c00640 1 --2- 172.21.15.83:0/2990969769 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e700b58c50 0x55e700b5d750 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c009c80 tx=0x7f4b2c0117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3e10e4c8963037f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:25.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.792+0000 7f4b32a00640 1 -- 172.21.15.83:0/2990969769 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b2c012450 con 0x55e700b58c50 2024-04-24T08:51:25.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.792+0000 7f4b32a00640 1 -- 172.21.15.83:0/2990969769 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4b2c012aa0 con 0x55e700b58c50 2024-04-24T08:51:25.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.792+0000 7f4b32a00640 1 -- 172.21.15.83:0/2990969769 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b2c00fb70 con 0x55e700b58c50 2024-04-24T08:51:25.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.792+0000 7f4b3a000640 1 --2- 172.21.15.83:0/3045874040 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f4b200a3bb0 0x7f4b200a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:25.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.792+0000 7f4b3a000640 1 -- 172.21.15.83:0/3045874040 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9237+0+0 (secure 0 0 0) 0x7f4b280d0e40 con 0x7f4b3c103080 2024-04-24T08:51:25.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.792+0000 7f4b42600640 1 --2- 172.21.15.83:0/3045874040 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f4b200a3bb0 0x7f4b200a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:25.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.792+0000 7f4b4742b740 1 -- 172.21.15.83:0/2990969769 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e700b58c50 msgr2=0x55e700b5d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:25.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.792+0000 7f4b4742b740 1 --2- 172.21.15.83:0/2990969769 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e700b58c50 0x55e700b5d750 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c009c80 tx=0x7f4b2c0117b0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.793+0000 7f4b4742b740 1 -- 172.21.15.83:0/2990969769 shutdown_connections 2024-04-24T08:51:25.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.793+0000 7f4b4742b740 1 --2- 172.21.15.83:0/2990969769 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e700b5ddc0 0x55e700b60250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.793+0000 7f4b4742b740 1 --2- 172.21.15.83:0/2990969769 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e700b58c50 0x55e700b5d750 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.793+0000 7f4b4742b740 1 --2- 172.21.15.83:0/2990969769 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e700b58260 0x55e700b58680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.793+0000 7f4b4742b740 1 -- 172.21.15.83:0/2990969769 >> 172.21.15.83:0/2990969769 conn(0x55e700b538d0 msgr2=0x55e700b53cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:25.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.793+0000 7f4b4742b740 1 -- 172.21.15.83:0/2990969769 shutdown_connections 2024-04-24T08:51:25.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.793+0000 7f4b4742b740 1 -- 172.21.15.83:0/2990969769 wait complete. 2024-04-24T08:51:25.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.793+0000 7f4b42600640 1 --2- 172.21.15.83:0/3045874040 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f4b200a3bb0 0x7f4b200a6070 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x55e700b594b0 tx=0x7f4b34031a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:25.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.793+0000 7f4b4742b740 1 -- start start 2024-04-24T08:51:25.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.793+0000 7f4b4742b740 1 Processor -- start 2024-04-24T08:51:25.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.794+0000 7f4b4742b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e700b58260 0x55e700b62640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:25.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.794+0000 7f4b4742b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e700b58c50 0x55e700b62b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:25.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.794+0000 7f4b4742b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e700b5ddc0 0x55e700b69bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:25.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.794+0000 7f4b4742b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e700b60d70 con 0x55e700b58c50 2024-04-24T08:51:25.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.794+0000 7f4b4742b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e700a77590 con 0x55e700b58260 2024-04-24T08:51:25.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.794+0000 7f4b4742b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55e700b60ef0 con 0x55e700b5ddc0 2024-04-24T08:51:25.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.794+0000 7f4b4742b740 10 client.0 _ll_register_callbacks cb 0x55e700a5a3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:51:25.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.794+0000 7f4b41200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e700b58c50 0x55e700b62b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:25.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.794+0000 7f4b41200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e700b58c50 0x55e700b62b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41550/0 (socket says 172.21.15.83:41550) 2024-04-24T08:51:25.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.794+0000 7f4b41200640 1 -- 172.21.15.83:0/3584559440 learned_addr learned my addr 172.21.15.83:0/3584559440 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:51:25.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.794+0000 7f4b42600640 1 --2- 172.21.15.83:0/3584559440 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e700b5ddc0 0x55e700b69bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:25.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.794+0000 7f4b41c00640 1 --2- 172.21.15.83:0/3584559440 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e700b58260 0x55e700b62640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:25.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.795+0000 7f4b41200640 1 -- 172.21.15.83:0/3584559440 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e700b5ddc0 msgr2=0x55e700b69bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:51:25.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.795+0000 7f4b41200640 1 --2- 172.21.15.83:0/3584559440 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e700b5ddc0 0x55e700b69bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.795+0000 7f4b41200640 1 -- 172.21.15.83:0/3584559440 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e700b58260 msgr2=0x55e700b62640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:51:25.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.795+0000 7f4b41200640 1 --2- 172.21.15.83:0/3584559440 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e700b58260 0x55e700b62640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.795+0000 7f4b41200640 1 -- 172.21.15.83:0/3584559440 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e70089a6b0 con 0x55e700b58c50 2024-04-24T08:51:25.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.795+0000 7f4b41200640 1 --2- 172.21.15.83:0/3584559440 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e700b58c50 0x55e700b62b80 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f4b28009230 tx=0x7f4b28003bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac8da58a5caa77b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:25.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.795+0000 7f4b27400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:51:25.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.795+0000 7f4b27400640 1 -- 172.21.15.83:0/3584559440 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b280a5710 con 0x55e700b58c50 2024-04-24T08:51:25.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.795+0000 7f4b27400640 1 -- 172.21.15.83:0/3584559440 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9237+0+0 (secure 0 0 0) 0x7f4b280bd0f0 con 0x55e700b58c50 2024-04-24T08:51:25.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.795+0000 7f4b4742b740 1 -- 172.21.15.83:0/3584559440 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b2c012aa0 con 0x55e700b58c50 2024-04-24T08:51:25.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.795+0000 7f4b27400640 1 -- 172.21.15.83:0/3584559440 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4b280a10e0 con 0x55e700b58c50 2024-04-24T08:51:25.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.795+0000 7f4b4742b740 10 client.15549 fetch_fsmap learned FSMap version 27 2024-04-24T08:51:25.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.795+0000 7f4b4742b740 1 -- 172.21.15.83:0/3584559440 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e700b6cf10 con 0x55e700b58c50 2024-04-24T08:51:25.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.796+0000 7f4b27400640 1 -- 172.21.15.83:0/3584559440 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4b2c012aa0 con 0x55e700b58c50 2024-04-24T08:51:25.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.796+0000 7f4b27400640 1 -- 172.21.15.83:0/3584559440 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f4b280d2070 con 0x55e700b58c50 2024-04-24T08:51:25.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.796+0000 7f4b4742b740 10 client.15549 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T08:51:25.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.796+0000 7f4b4742b740 10 client.15549 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-24T08:51:25.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.796+0000 7f4b4742b740 20 client.15549 populate_metadata read hostname 'smithi083' 2024-04-24T08:51:25.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.796+0000 7f4b4742b740 1 --2- 172.21.15.83:0/3584559440 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55e700b6f540 0x55e700b709b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:25.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.796+0000 7f4b4742b740 4 client.15549 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:51:25.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.796+0000 7f4b4742b740 1 -- 172.21.15.83:0/3584559440 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e700acb530 con 0x55e700b6f540 2024-04-24T08:51:25.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.797+0000 7f4b41c00640 1 --2- 172.21.15.83:0/3584559440 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55e700b6f540 0x55e700b709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:25.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.798+0000 7f4b41c00640 1 --2- 172.21.15.83:0/3584559440 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55e700b6f540 0x55e700b709b0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=189ff61a792d0e12 server_cookie=6308e11f2bc59dca in_seq=0 out_seq=0 2024-04-24T08:51:25.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.798+0000 7f4b27400640 10 client.15549 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:51:25.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.800+0000 7f4b27400640 1 -- 172.21.15.83:0/3584559440 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e700acb530 con 0x55e700b6f540 2024-04-24T08:51:25.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.800+0000 7f4b27400640 10 client.15549 handle_command_reply: tid=0 2024-04-24T08:51:25.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.800+0000 7f4b4742b740 1 client.15549 shutdown 2024-04-24T08:51:25.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.800+0000 7f4b4742b740 1 -- 172.21.15.83:0/3584559440 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e700b58c50 msgr2=0x55e700b62b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:25.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.800+0000 7f4b4742b740 1 --2- 172.21.15.83:0/3584559440 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e700b58c50 0x55e700b62b80 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f4b28009230 tx=0x7f4b28003bd0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.800+0000 7f4b4742b740 1 -- 172.21.15.83:0/3584559440 shutdown_connections 2024-04-24T08:51:25.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.800+0000 7f4b4742b740 1 --2- 172.21.15.83:0/3584559440 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55e700b6f540 0x55e700b709b0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.800+0000 7f4b4742b740 1 --2- 172.21.15.83:0/3584559440 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e700b5ddc0 0x55e700b69bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.800+0000 7f4b4742b740 1 --2- 172.21.15.83:0/3584559440 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e700b58c50 0x55e700b62b80 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.800+0000 7f4b4742b740 1 --2- 172.21.15.83:0/3584559440 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e700b58260 0x55e700b62640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.800+0000 7f4b27400640 0 client.15549 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:51:25.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.800+0000 7f4b4742b740 1 -- 172.21.15.83:0/3584559440 >> 172.21.15.83:0/3584559440 conn(0x55e700b538d0 msgr2=0x55e700b52970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:25.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.800+0000 7f4b4742b740 1 -- 172.21.15.83:0/3584559440 shutdown_connections 2024-04-24T08:51:25.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.800+0000 7f4b4742b740 1 -- 172.21.15.83:0/3584559440 wait complete. 2024-04-24T08:51:25.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.800+0000 7f4b4742b740 20 client.15549 trim_cache size 0 max 16384 2024-04-24T08:51:25.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.820+0000 7f4b4742b740 1 Processor -- start 2024-04-24T08:51:25.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.820+0000 7f4b4742b740 1 -- start start 2024-04-24T08:51:25.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.820+0000 7f4b4742b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e700b58260 0x55e700b5d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:25.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.820+0000 7f4b4742b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e700b58c50 0x55e700b69b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:25.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.820+0000 7f4b4742b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e700b5ddc0 0x55e700b5a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:25.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.820+0000 7f4b4742b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e700b6ecf0 con 0x55e700b5ddc0 2024-04-24T08:51:25.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.820+0000 7f4b4742b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e700b6f3c0 con 0x55e700b58260 2024-04-24T08:51:25.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.820+0000 7f4b4742b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55e700b6fa90 con 0x55e700b58c50 2024-04-24T08:51:25.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.821+0000 7f4b42600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e700b5ddc0 0x55e700b5a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:25.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.821+0000 7f4b42600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e700b5ddc0 0x55e700b5a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41560/0 (socket says 172.21.15.83:41560) 2024-04-24T08:51:25.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.821+0000 7f4b42600640 1 -- 172.21.15.83:0/3314867142 learned_addr learned my addr 172.21.15.83:0/3314867142 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:51:25.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.821+0000 7f4b41c00640 1 --2- 172.21.15.83:0/3314867142 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e700b58260 0x55e700b5d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:25.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.821+0000 7f4b41200640 1 --2- 172.21.15.83:0/3314867142 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e700b58c50 0x55e700b69b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:25.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.821+0000 7f4b42600640 1 -- 172.21.15.83:0/3314867142 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e700b58c50 msgr2=0x55e700b69b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:51:25.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.821+0000 7f4b42600640 1 --2- 172.21.15.83:0/3314867142 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e700b58c50 0x55e700b69b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.821+0000 7f4b42600640 1 -- 172.21.15.83:0/3314867142 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e700b58260 msgr2=0x55e700b5d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:51:25.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.821+0000 7f4b42600640 1 --2- 172.21.15.83:0/3314867142 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e700b58260 0x55e700b5d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.821+0000 7f4b42600640 1 -- 172.21.15.83:0/3314867142 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e700b594e0 con 0x55e700b5ddc0 2024-04-24T08:51:25.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.822+0000 7f4b42600640 1 --2- 172.21.15.83:0/3314867142 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e700b5ddc0 0x55e700b5a0b0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f4b34035000 tx=0x7f4b34040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b7941847d941ba2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:25.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.822+0000 7f4b32000640 1 -- 172.21.15.83:0/3314867142 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b34003970 con 0x55e700b5ddc0 2024-04-24T08:51:25.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.822+0000 7f4b32000640 1 -- 172.21.15.83:0/3314867142 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4b34036040 con 0x55e700b5ddc0 2024-04-24T08:51:25.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.822+0000 7f4b32000640 1 -- 172.21.15.83:0/3314867142 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b340305c0 con 0x55e700b5ddc0 2024-04-24T08:51:25.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.823+0000 7f4b4742b740 1 -- 172.21.15.83:0/3314867142 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e700b5ddc0 msgr2=0x55e700b5a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:25.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.823+0000 7f4b4742b740 1 --2- 172.21.15.83:0/3314867142 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e700b5ddc0 0x55e700b5a0b0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f4b34035000 tx=0x7f4b34040000 comp rx=0 tx=0).stop 2024-04-24T08:51:25.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.823+0000 7f4b4742b740 1 -- 172.21.15.83:0/3314867142 shutdown_connections 2024-04-24T08:51:25.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.823+0000 7f4b4742b740 1 --2- 172.21.15.83:0/3314867142 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e700b5ddc0 0x55e700b5a0b0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.823+0000 7f4b4742b740 1 --2- 172.21.15.83:0/3314867142 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e700b58c50 0x55e700b69b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.823+0000 7f4b4742b740 1 --2- 172.21.15.83:0/3314867142 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e700b58260 0x55e700b5d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.823+0000 7f4b4742b740 1 -- 172.21.15.83:0/3314867142 >> 172.21.15.83:0/3314867142 conn(0x55e700b538d0 msgr2=0x55e700b53cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:25.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.823+0000 7f4b4742b740 1 -- 172.21.15.83:0/3314867142 shutdown_connections 2024-04-24T08:51:25.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.823+0000 7f4b4742b740 1 -- 172.21.15.83:0/3314867142 wait complete. 2024-04-24T08:51:25.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.823+0000 7f4b4742b740 1 -- start start 2024-04-24T08:51:25.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.824+0000 7f4b4742b740 1 Processor -- start 2024-04-24T08:51:25.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.824+0000 7f4b4742b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e700b58260 0x55e700b67dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:25.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.824+0000 7f4b4742b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e700b58c50 0x55e700b68300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:25.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.824+0000 7f4b4742b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e700b5ddc0 0x55e700b68840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:25.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.824+0000 7f4b4742b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e700b70830 con 0x55e700b58c50 2024-04-24T08:51:25.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.824+0000 7f4b4742b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e700b70160 con 0x55e700b58260 2024-04-24T08:51:25.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.824+0000 7f4b4742b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55e700b70f00 con 0x55e700b5ddc0 2024-04-24T08:51:25.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.824+0000 7f4b4742b740 10 client.0 _ll_register_callbacks cb 0x7f4b2809c900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:51:25.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.824+0000 7f4b41200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e700b58c50 0x55e700b68300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:25.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.824+0000 7f4b41200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e700b58c50 0x55e700b68300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41576/0 (socket says 172.21.15.83:41576) 2024-04-24T08:51:25.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.824+0000 7f4b41200640 1 -- 172.21.15.83:0/3383149414 learned_addr learned my addr 172.21.15.83:0/3383149414 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:51:25.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.825+0000 7f4b42600640 1 --2- 172.21.15.83:0/3383149414 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e700b5ddc0 0x55e700b68840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:25.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.825+0000 7f4b41200640 1 -- 172.21.15.83:0/3383149414 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e700b5ddc0 msgr2=0x55e700b68840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:51:25.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.825+0000 7f4b41200640 1 --2- 172.21.15.83:0/3383149414 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e700b5ddc0 0x55e700b68840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.825+0000 7f4b41200640 1 -- 172.21.15.83:0/3383149414 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e700b58260 msgr2=0x55e700b67dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:51:25.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.825+0000 7f4b41c00640 1 --2- 172.21.15.83:0/3383149414 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e700b58260 0x55e700b67dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:25.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.825+0000 7f4b41200640 1 --2- 172.21.15.83:0/3383149414 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e700b58260 0x55e700b67dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.825+0000 7f4b41200640 1 -- 172.21.15.83:0/3383149414 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e700b68fd0 con 0x55e700b58c50 2024-04-24T08:51:25.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.825+0000 7f4b41200640 1 --2- 172.21.15.83:0/3383149414 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e700b58c50 0x55e700b68300 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f4b280996b0 tx=0x7f4b28009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4f1fda182bfab91 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:25.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.825+0000 7f4b27400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:51:25.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.825+0000 7f4b27400640 1 -- 172.21.15.83:0/3383149414 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b28098fd0 con 0x55e700b58c50 2024-04-24T08:51:25.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.825+0000 7f4b27400640 1 -- 172.21.15.83:0/3383149414 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9237+0+0 (secure 0 0 0) 0x7f4b280a5710 con 0x55e700b58c50 2024-04-24T08:51:25.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.825+0000 7f4b4742b740 1 -- 172.21.15.83:0/3383149414 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b34036040 con 0x55e700b58c50 2024-04-24T08:51:25.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.826+0000 7f4b27400640 1 -- 172.21.15.83:0/3383149414 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4b280a59a0 con 0x55e700b58c50 2024-04-24T08:51:25.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.826+0000 7f4b4742b740 10 client.15555 fetch_fsmap learned FSMap version 27 2024-04-24T08:51:25.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.826+0000 7f4b4742b740 1 -- 172.21.15.83:0/3383149414 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e700bd5720 con 0x55e700b58c50 2024-04-24T08:51:25.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.826+0000 7f4b27400640 1 -- 172.21.15.83:0/3383149414 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4b34036040 con 0x55e700b58c50 2024-04-24T08:51:25.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.826+0000 7f4b27400640 1 -- 172.21.15.83:0/3383149414 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f4b280d3030 con 0x55e700b58c50 2024-04-24T08:51:25.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.826+0000 7f4b4742b740 10 client.15555 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T08:51:25.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.826+0000 7f4b4742b740 10 client.15555 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-24T08:51:25.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.826+0000 7f4b4742b740 20 client.15555 populate_metadata read hostname 'smithi083' 2024-04-24T08:51:25.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.826+0000 7f4b4742b740 1 --2- 172.21.15.83:0/3383149414 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55e700bda380 0x55e700bd67c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:25.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.826+0000 7f4b4742b740 4 client.15555 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T08:51:25.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.826+0000 7f4b4742b740 1 -- 172.21.15.83:0/3383149414 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "status"}) -- 0x55e700ac53a0 con 0x55e700bda380 2024-04-24T08:51:25.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.827+0000 7f4b41c00640 1 --2- 172.21.15.83:0/3383149414 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55e700bda380 0x55e700bd67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:25.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.828+0000 7f4b41c00640 1 --2- 172.21.15.83:0/3383149414 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55e700bda380 0x55e700bd67c0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f291927d1c0e1a15 server_cookie=73909ca967f9c448 in_seq=0 out_seq=0 2024-04-24T08:51:25.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.828+0000 7f4b27400640 10 client.15555 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:51:25.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.828+0000 7f4b27400640 1 -- 172.21.15.83:0/3383149414 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55e700ac53a0 con 0x55e700bda380 2024-04-24T08:51:25.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.828+0000 7f4b27400640 10 client.15555 handle_command_reply: tid=0 2024-04-24T08:51:25.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.828+0000 7f4b4742b740 1 client.15555 shutdown 2024-04-24T08:51:25.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.828+0000 7f4b4742b740 1 -- 172.21.15.83:0/3383149414 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e700b58c50 msgr2=0x55e700b68300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:25.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.828+0000 7f4b4742b740 1 --2- 172.21.15.83:0/3383149414 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e700b58c50 0x55e700b68300 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f4b280996b0 tx=0x7f4b28009310 comp rx=0 tx=0).stop 2024-04-24T08:51:25.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.828+0000 7f4b4742b740 1 -- 172.21.15.83:0/3383149414 shutdown_connections 2024-04-24T08:51:25.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.828+0000 7f4b4742b740 1 --2- 172.21.15.83:0/3383149414 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55e700bda380 0x55e700bd67c0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.828+0000 7f4b4742b740 1 --2- 172.21.15.83:0/3383149414 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e700b5ddc0 0x55e700b68840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.828+0000 7f4b4742b740 1 --2- 172.21.15.83:0/3383149414 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e700b58c50 0x55e700b68300 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.828+0000 7f4b27400640 0 client.15555 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:51:25.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.828+0000 7f4b4742b740 1 --2- 172.21.15.83:0/3383149414 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e700b58260 0x55e700b67dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.828+0000 7f4b4742b740 1 -- 172.21.15.83:0/3383149414 >> 172.21.15.83:0/3383149414 conn(0x55e700b538d0 msgr2=0x55e700b5d240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:25.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.829+0000 7f4b4742b740 1 -- 172.21.15.83:0/3383149414 shutdown_connections 2024-04-24T08:51:25.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.829+0000 7f4b4742b740 1 -- 172.21.15.83:0/3383149414 wait complete. 2024-04-24T08:51:25.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.829+0000 7f4b4742b740 20 client.15555 trim_cache size 0 max 16384 2024-04-24T08:51:25.835 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T08:51:25.835 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_fsid": "bd3e039e-0215-11ef-bc93-c7b262605968", 2024-04-24T08:51:25.835 INFO:teuthology.orchestra.run.smithi083.stdout: "whoami": 0, 2024-04-24T08:51:25.835 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 34520, 2024-04-24T08:51:25.835 INFO:teuthology.orchestra.run.smithi083.stdout: "want_state": "up:active", 2024-04-24T08:51:25.835 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "up:active", 2024-04-24T08:51:25.835 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs", 2024-04-24T08:51:25.835 INFO:teuthology.orchestra.run.smithi083.stdout: "rank_uptime": 175.61541443600001, 2024-04-24T08:51:25.835 INFO:teuthology.orchestra.run.smithi083.stdout: "mdsmap_epoch": 27, 2024-04-24T08:51:25.835 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch": 93, 2024-04-24T08:51:25.836 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch_barrier": 89, 2024-04-24T08:51:25.836 INFO:teuthology.orchestra.run.smithi083.stdout: "uptime": 183.800542785 2024-04-24T08:51:25.836 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T08:51:25.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.829+0000 7f4b43000640 1 -- 172.21.15.83:0/3045874040 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f4b200a3bb0 msgr2=0x7f4b200a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:25.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.829+0000 7f4b43000640 1 --2- 172.21.15.83:0/3045874040 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f4b200a3bb0 0x7f4b200a6070 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x55e700b594b0 tx=0x7f4b34031a40 comp rx=0 tx=0).stop 2024-04-24T08:51:25.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.830+0000 7f4b43000640 1 -- 172.21.15.83:0/3045874040 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b3c103080 msgr2=0x7f4b3c198600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:25.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.830+0000 7f4b43000640 1 --2- 172.21.15.83:0/3045874040 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b3c103080 0x7f4b3c198600 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f4b2800e9c0 tx=0x7f4b2800ee80 comp rx=0 tx=0).stop 2024-04-24T08:51:25.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.830+0000 7f4b43000640 1 -- 172.21.15.83:0/3045874040 shutdown_connections 2024-04-24T08:51:25.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.830+0000 7f4b43000640 1 --2- 172.21.15.83:0/3045874040 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b3c103a40 0x7f4b3c19f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.830+0000 7f4b43000640 1 --2- 172.21.15.83:0/3045874040 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f4b200a3bb0 0x7f4b200a6070 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.830+0000 7f4b43000640 1 --2- 172.21.15.83:0/3045874040 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b3c103080 0x7f4b3c198600 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.830+0000 7f4b43000640 1 --2- 172.21.15.83:0/3045874040 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4b3c101f30 0x7f4b3c1980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:25.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.830+0000 7f4b43000640 1 -- 172.21.15.83:0/3045874040 >> 172.21.15.83:0/3045874040 conn(0x7f4b3c0fd4a0 msgr2=0x7f4b3c1063b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:25.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.830+0000 7f4b43000640 1 -- 172.21.15.83:0/3045874040 shutdown_connections 2024-04-24T08:51:25.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:25.830+0000 7f4b43000640 1 -- 172.21.15.83:0/3045874040 wait complete. 2024-04-24T08:51:25.848 INFO:teuthology.run_tasks:Running task sequential... 2024-04-24T08:51:25.859 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-24T08:51:25.859 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-24T08:51:25.859 INFO:teuthology.task.print:Enabling mgr modules 2024-04-24T08:51:25.860 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-04-24T08:51:25.860 INFO:teuthology.task.exec:Executing custom commands... 2024-04-24T08:51:25.860 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi083.front.sepia.ceph.com 2024-04-24T08:51:25.861 DEBUG:teuthology.orchestra.run.smithi083:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-04-24T08:51:26.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:26.065+0000 7f5a1b800640 1 -- 172.21.15.83:0/367056532 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a141012b0 msgr2=0x7f5a14101710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:26.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:26.065+0000 7f5a1b800640 1 --2- 172.21.15.83:0/367056532 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a141012b0 0x7f5a14101710 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f59fc009900 tx=0x7f59fc02a480 comp rx=0 tx=0).stop 2024-04-24T08:51:26.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:26.065+0000 7f5a1b800640 1 -- 172.21.15.83:0/367056532 shutdown_connections 2024-04-24T08:51:26.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:26.065+0000 7f5a1b800640 1 --2- 172.21.15.83:0/367056532 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5a14101c50 0x7f5a141020d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:26.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:26.065+0000 7f5a1b800640 1 --2- 172.21.15.83:0/367056532 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a141012b0 0x7f5a14101710 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:26.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:26.065+0000 7f5a1b800640 1 --2- 172.21.15.83:0/367056532 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5a14100950 0x7f5a14100d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:26.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:26.065+0000 7f5a1b800640 1 -- 172.21.15.83:0/367056532 >> 172.21.15.83:0/367056532 conn(0x7f5a140fc4d0 msgr2=0x7f5a140fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:26.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:26.065+0000 7f5a1b800640 1 -- 172.21.15.83:0/367056532 shutdown_connections 2024-04-24T08:51:26.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:26.065+0000 7f5a1b800640 1 -- 172.21.15.83:0/367056532 wait complete. 2024-04-24T08:51:26.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:26.066+0000 7f5a1b800640 1 Processor -- start 2024-04-24T08:51:26.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:26.066+0000 7f5a1b800640 1 -- start start 2024-04-24T08:51:26.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:26.067+0000 7f5a1b800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5a14100950 0x7f5a1419a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:26.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:26.067+0000 7f5a1b800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5a141012b0 0x7f5a1419a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:26.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:26.067+0000 7f5a1b800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a14101c50 0x7f5a141a1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:26.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:26.067+0000 7f5a1b800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5a1410fdf0 con 0x7f5a14101c50 2024-04-24T08:51:26.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:26.067+0000 7f5a1b800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5a1410fc70 con 0x7f5a14100950 2024-04-24T08:51:26.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:26.067+0000 7f5a1b800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5a1410ff70 con 0x7f5a141012b0 2024-04-24T08:51:26.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:26.067+0000 7f5a19800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a14101c50 0x7f5a141a1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:26.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:26.067+0000 7f5a19800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a14101c50 0x7f5a141a1a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41586/0 (socket says 172.21.15.83:41586) 2024-04-24T08:51:26.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:26.067+0000 7f5a19800640 1 -- 172.21.15.83:0/1271352400 learned_addr learned my addr 172.21.15.83:0/1271352400 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:51:26.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:26.067+0000 7f5a13e00640 1 --2- 172.21.15.83:0/1271352400 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5a141012b0 0x7f5a1419a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:26.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:26.067+0000 7f5a18e00640 1 --2- 172.21.15.83:0/1271352400 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5a14100950 0x7f5a1419a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:26.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:26.067+0000 7f5a19800640 1 -- 172.21.15.83:0/1271352400 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5a141012b0 msgr2=0x7f5a1419a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:26.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:26.067+0000 7f5a19800640 1 --2- 172.21.15.83:0/1271352400 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5a141012b0 0x7f5a1419a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:26.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:26.067+0000 7f5a19800640 1 -- 172.21.15.83:0/1271352400 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5a14100950 msgr2=0x7f5a1419a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:26.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:26.067+0000 7f5a19800640 1 --2- 172.21.15.83:0/1271352400 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5a14100950 0x7f5a1419a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:26.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:26.067+0000 7f5a19800640 1 -- 172.21.15.83:0/1271352400 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a141a2090 con 0x7f5a14101c50 2024-04-24T08:51:26.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:26.068+0000 7f5a19800640 1 --2- 172.21.15.83:0/1271352400 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a14101c50 0x7f5a141a1a20 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c00c970 tx=0x7f5a0c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:26.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:26.068+0000 7f5a11600640 1 -- 172.21.15.83:0/1271352400 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a0c007bb0 con 0x7f5a14101c50 2024-04-24T08:51:26.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:26.068+0000 7f5a11600640 1 -- 172.21.15.83:0/1271352400 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5a0c007d50 con 0x7f5a14101c50 2024-04-24T08:51:26.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:26.068+0000 7f5a1b800640 1 -- 172.21.15.83:0/1271352400 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a141a2380 con 0x7f5a14101c50 2024-04-24T08:51:26.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:26.068+0000 7f5a11600640 1 -- 172.21.15.83:0/1271352400 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a0c004e10 con 0x7f5a14101c50 2024-04-24T08:51:26.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:26.068+0000 7f5a1b800640 1 -- 172.21.15.83:0/1271352400 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a141a28a0 con 0x7f5a14101c50 2024-04-24T08:51:26.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:26.069+0000 7f5a1b800640 1 -- 172.21.15.83:0/1271352400 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a1410b6a0 con 0x7f5a14101c50 2024-04-24T08:51:26.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:26.071+0000 7f5a11600640 1 -- 172.21.15.83:0/1271352400 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f5a0c005020 con 0x7f5a14101c50 2024-04-24T08:51:26.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:26.073+0000 7f5a11600640 1 --2- 172.21.15.83:0/1271352400 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f59f80a3cd0 0x7f59f80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:26.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:26.073+0000 7f5a11600640 1 -- 172.21.15.83:0/1271352400 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9237+0+0 (secure 0 0 0) 0x7f5a0c0d0540 con 0x7f5a14101c50 2024-04-24T08:51:26.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:26.073+0000 7f5a18e00640 1 --2- 172.21.15.83:0/1271352400 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f59f80a3cd0 0x7f59f80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:26.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:26.074+0000 7f5a18e00640 1 --2- 172.21.15.83:0/1271352400 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f59f80a3cd0 0x7f59f80a6190 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f5a04005fd0 tx=0x7f5a04009870 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:26.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:26.077+0000 7f5a11600640 1 -- 172.21.15.83:0/1271352400 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5a0c014030 con 0x7f5a14101c50 2024-04-24T08:51:26.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:26 smithi083 ceph-mon[27907]: pgmap v465: 129 pgs: 129 active+clean; 644 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-04-24T08:51:26.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:26.332+0000 7f5a1b800640 1 -- 172.21.15.83:0/1271352400 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7f5a14062c00 con 0x7f5a14101c50 2024-04-24T08:51:26.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:26 smithi138 ceph-mon[31441]: pgmap v465: 129 pgs: 129 active+clean; 644 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-04-24T08:51:26.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:26 smithi189 ceph-mon[31791]: pgmap v465: 129 pgs: 129 active+clean; 644 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-04-24T08:51:27.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.030+0000 7f5a11600640 1 -- 172.21.15.83:0/1271352400 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7f5a0c09cad0 con 0x7f5a14101c50 2024-04-24T08:51:27.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.034+0000 7f5a1b800640 1 -- 172.21.15.83:0/1271352400 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f59f80a3cd0 msgr2=0x7f59f80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:27.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.034+0000 7f5a1b800640 1 --2- 172.21.15.83:0/1271352400 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f59f80a3cd0 0x7f59f80a6190 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f5a04005fd0 tx=0x7f5a04009870 comp rx=0 tx=0).stop 2024-04-24T08:51:27.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.034+0000 7f5a1b800640 1 -- 172.21.15.83:0/1271352400 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a14101c50 msgr2=0x7f5a141a1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:27.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.034+0000 7f5a1b800640 1 --2- 172.21.15.83:0/1271352400 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a14101c50 0x7f5a141a1a20 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c00c970 tx=0x7f5a0c00ce30 comp rx=0 tx=0).stop 2024-04-24T08:51:27.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.038+0000 7f5a1b800640 1 -- 172.21.15.83:0/1271352400 shutdown_connections 2024-04-24T08:51:27.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.038+0000 7f5a1b800640 1 --2- 172.21.15.83:0/1271352400 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f59f80a3cd0 0x7f59f80a6190 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:27.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.038+0000 7f5a1b800640 1 --2- 172.21.15.83:0/1271352400 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a14101c50 0x7f5a141a1a20 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:27.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.038+0000 7f5a1b800640 1 --2- 172.21.15.83:0/1271352400 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5a141012b0 0x7f5a1419a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:27.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.038+0000 7f5a1b800640 1 --2- 172.21.15.83:0/1271352400 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5a14100950 0x7f5a1419a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:27.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.038+0000 7f5a1b800640 1 -- 172.21.15.83:0/1271352400 >> 172.21.15.83:0/1271352400 conn(0x7f5a140fc4d0 msgr2=0x7f5a140fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:27.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.041+0000 7f5a1b800640 1 -- 172.21.15.83:0/1271352400 shutdown_connections 2024-04-24T08:51:27.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.041+0000 7f5a1b800640 1 -- 172.21.15.83:0/1271352400 wait complete. 2024-04-24T08:51:27.069 DEBUG:teuthology.orchestra.run.smithi083:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-04-24T08:51:27.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.247+0000 7f92f9600640 1 -- 172.21.15.83:0/2284420242 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92f4105480 msgr2=0x7f92f4105900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:27.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.247+0000 7f92f9600640 1 --2- 172.21.15.83:0/2284420242 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92f4105480 0x7f92f4105900 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f92dc009900 tx=0x7f92dc02a410 comp rx=0 tx=0).stop 2024-04-24T08:51:27.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.247+0000 7f92f9600640 1 -- 172.21.15.83:0/2284420242 shutdown_connections 2024-04-24T08:51:27.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.247+0000 7f92f9600640 1 --2- 172.21.15.83:0/2284420242 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f92f4105e40 0x7f92f410c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:27.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.247+0000 7f92f9600640 1 --2- 172.21.15.83:0/2284420242 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92f4105480 0x7f92f4105900 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:27.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.247+0000 7f92f9600640 1 --2- 172.21.15.83:0/2284420242 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f92f4104290 0x7f92f41046b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:27.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.247+0000 7f92f9600640 1 -- 172.21.15.83:0/2284420242 >> 172.21.15.83:0/2284420242 conn(0x7f92f40ff820 msgr2=0x7f92f4101c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:27.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.248+0000 7f92f9600640 1 -- 172.21.15.83:0/2284420242 shutdown_connections 2024-04-24T08:51:27.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.248+0000 7f92f9600640 1 -- 172.21.15.83:0/2284420242 wait complete. 2024-04-24T08:51:27.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.248+0000 7f92f9600640 1 Processor -- start 2024-04-24T08:51:27.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.249+0000 7f92f9600640 1 -- start start 2024-04-24T08:51:27.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.249+0000 7f92f9600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f92f4104290 0x7f92f419a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:27.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.249+0000 7f92f9600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f92f4105480 0x7f92f419a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:27.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.249+0000 7f92f9600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92f4105e40 0x7f92f41a19c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:27.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.249+0000 7f92f9600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f92f410fe20 con 0x7f92f4105e40 2024-04-24T08:51:27.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.249+0000 7f92f9600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f92f410fca0 con 0x7f92f4104290 2024-04-24T08:51:27.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.249+0000 7f92f9600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f92f410ffa0 con 0x7f92f4105480 2024-04-24T08:51:27.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.249+0000 7f92f3400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92f4105e40 0x7f92f41a19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:27.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.249+0000 7f92f3400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92f4105e40 0x7f92f41a19c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41622/0 (socket says 172.21.15.83:41622) 2024-04-24T08:51:27.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.249+0000 7f92f3400640 1 -- 172.21.15.83:0/921726943 learned_addr learned my addr 172.21.15.83:0/921726943 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:51:27.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.249+0000 7f92f2a00640 1 --2- 172.21.15.83:0/921726943 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f92f4104290 0x7f92f419a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:27.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.249+0000 7f92ea000640 1 --2- 172.21.15.83:0/921726943 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f92f4105480 0x7f92f419a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:27.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.250+0000 7f92f3400640 1 -- 172.21.15.83:0/921726943 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f92f4105480 msgr2=0x7f92f419a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:27.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.250+0000 7f92f3400640 1 --2- 172.21.15.83:0/921726943 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f92f4105480 0x7f92f419a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:27.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.250+0000 7f92f3400640 1 -- 172.21.15.83:0/921726943 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f92f4104290 msgr2=0x7f92f419a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:27.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.250+0000 7f92f3400640 1 --2- 172.21.15.83:0/921726943 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f92f4104290 0x7f92f419a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:27.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.250+0000 7f92f3400640 1 -- 172.21.15.83:0/921726943 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92f41a2030 con 0x7f92f4105e40 2024-04-24T08:51:27.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.250+0000 7f92f3400640 1 --2- 172.21.15.83:0/921726943 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92f4105e40 0x7f92f41a19c0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f92ec00d750 tx=0x7f92ec00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:27.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.250+0000 7f92ebe00640 1 -- 172.21.15.83:0/921726943 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92ec017070 con 0x7f92f4105e40 2024-04-24T08:51:27.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.250+0000 7f92ebe00640 1 -- 172.21.15.83:0/921726943 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f92ec0054d0 con 0x7f92f4105e40 2024-04-24T08:51:27.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.250+0000 7f92ebe00640 1 -- 172.21.15.83:0/921726943 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92ec004c60 con 0x7f92f4105e40 2024-04-24T08:51:27.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.250+0000 7f92f9600640 1 -- 172.21.15.83:0/921726943 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92f41a2320 con 0x7f92f4105e40 2024-04-24T08:51:27.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.251+0000 7f92f9600640 1 -- 172.21.15.83:0/921726943 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92f41a2840 con 0x7f92f4105e40 2024-04-24T08:51:27.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.252+0000 7f92f9600640 1 -- 172.21.15.83:0/921726943 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92f410b550 con 0x7f92f4105e40 2024-04-24T08:51:27.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.254+0000 7f92ebe00640 1 -- 172.21.15.83:0/921726943 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f92ec004e00 con 0x7f92f4105e40 2024-04-24T08:51:27.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.255+0000 7f92ebe00640 1 --2- 172.21.15.83:0/921726943 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f92d80a3ca0 0x7f92d80a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:27.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.255+0000 7f92f2a00640 1 -- 172.21.15.83:0/921726943 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f92d80a3ca0 msgr2=0x7f92d80a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.83:6800/2550826560 2024-04-24T08:51:27.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.255+0000 7f92ebe00640 1 -- 172.21.15.83:0/921726943 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9237+0+0 (secure 0 0 0) 0x7f92ec0d0280 con 0x7f92f4105e40 2024-04-24T08:51:27.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.255+0000 7f92f2a00640 1 --2- 172.21.15.83:0/921726943 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f92d80a3ca0 0x7f92d80a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T08:51:27.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.259+0000 7f92ebe00640 1 -- 172.21.15.83:0/921726943 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f92ec09c870 con 0x7f92f4105e40 2024-04-24T08:51:27.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:27 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/1271352400' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-24T08:51:27.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:27 smithi083 ceph-mon[27907]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:51:27.303 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:51:27 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: ignoring --setuser ceph since I am not root 2024-04-24T08:51:27.303 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:51:27 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: ignoring --setgroup ceph since I am not root 2024-04-24T08:51:27.338 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:27 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/1271352400' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-24T08:51:27.338 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:27 smithi138 ceph-mon[31441]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:51:27.339 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Apr 24 08:51:27 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-y[32810]: ignoring --setuser ceph since I am not root 2024-04-24T08:51:27.339 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Apr 24 08:51:27 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-y[32810]: ignoring --setgroup ceph since I am not root 2024-04-24T08:51:27.339 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Apr 24 08:51:27 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-y[32810]: 2024-04-24T08:51:27.338+0000 7f492924e200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-24T08:51:27.344 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:27 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/1271352400' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-24T08:51:27.344 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:27 smithi189 ceph-mon[31791]: from='mgr.14152 172.21.15.83:0/1987671572' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:51:27.345 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 24 08:51:27 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-z[33677]: ignoring --setuser ceph since I am not root 2024-04-24T08:51:27.345 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 24 08:51:27 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-z[33677]: ignoring --setgroup ceph since I am not root 2024-04-24T08:51:27.345 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 24 08:51:27 smithi189 ceph-mgr[33681]: -- 172.21.15.189:0/2567116828 <== mon.1 v2:172.21.15.189:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a83f3465a0 con 0x55a83f31b000 2024-04-24T08:51:27.345 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 24 08:51:27 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-z[33677]: 2024-04-24T08:51:27.343+0000 7fef15b51200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-24T08:51:27.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.456+0000 7f92f2a00640 1 -- 172.21.15.83:0/921726943 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f92d80a3ca0 msgr2=0x7f92d80a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.83:6800/2550826560 2024-04-24T08:51:27.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.456+0000 7f92f2a00640 1 --2- 172.21.15.83:0/921726943 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f92d80a3ca0 0x7f92d80a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-24T08:51:27.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.460+0000 7f92f9600640 1 -- 172.21.15.83:0/921726943 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f92f4062c00 con 0x7f92f4105e40 2024-04-24T08:51:27.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.466+0000 7f92ebe00640 1 -- 172.21.15.83:0/921726943 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v32)=0 v32) ==== 138+0+0 (secure 0 0 0) 0x7f92ec09c170 con 0x7f92f4105e40 2024-04-24T08:51:27.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.470+0000 7f92f9600640 1 -- 172.21.15.83:0/921726943 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f92d80a3ca0 msgr2=0x7f92d80a6160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T08:51:27.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.470+0000 7f92f9600640 1 --2- 172.21.15.83:0/921726943 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f92d80a3ca0 0x7f92d80a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:27.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.470+0000 7f92f9600640 1 -- 172.21.15.83:0/921726943 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92f4105e40 msgr2=0x7f92f41a19c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:27.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.470+0000 7f92f9600640 1 --2- 172.21.15.83:0/921726943 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92f4105e40 0x7f92f41a19c0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f92ec00d750 tx=0x7f92ec00dc10 comp rx=0 tx=0).stop 2024-04-24T08:51:27.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.470+0000 7f92f9600640 1 -- 172.21.15.83:0/921726943 shutdown_connections 2024-04-24T08:51:27.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.470+0000 7f92f9600640 1 --2- 172.21.15.83:0/921726943 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f92d80a3ca0 0x7f92d80a6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:27.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.470+0000 7f92f9600640 1 --2- 172.21.15.83:0/921726943 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92f4105e40 0x7f92f41a19c0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:27.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.470+0000 7f92f9600640 1 --2- 172.21.15.83:0/921726943 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f92f4105480 0x7f92f419a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:27.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.470+0000 7f92f9600640 1 --2- 172.21.15.83:0/921726943 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f92f4104290 0x7f92f419a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:27.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.470+0000 7f92f9600640 1 -- 172.21.15.83:0/921726943 >> 172.21.15.83:0/921726943 conn(0x7f92f40ff820 msgr2=0x7f92f4108720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:27.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.470+0000 7f92f9600640 1 -- 172.21.15.83:0/921726943 shutdown_connections 2024-04-24T08:51:27.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.470+0000 7f92f9600640 1 -- 172.21.15.83:0/921726943 wait complete. 2024-04-24T08:51:27.494 DEBUG:teuthology.orchestra.run.smithi083:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-04-24T08:51:27.618 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:51:27 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:51:27.358+0000 7f3b97266200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-24T08:51:27.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.668+0000 7f0779000640 1 -- 172.21.15.83:0/2355205422 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0774074a60 msgr2=0x7f0774074ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:27.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.668+0000 7f0779000640 1 --2- 172.21.15.83:0/2355205422 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0774074a60 0x7f0774074ee0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f075c009900 tx=0x7f075c02a3f0 comp rx=0 tx=0).stop 2024-04-24T08:51:27.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.668+0000 7f0779000640 1 -- 172.21.15.83:0/2355205422 shutdown_connections 2024-04-24T08:51:27.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.668+0000 7f0779000640 1 --2- 172.21.15.83:0/2355205422 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0774109f40 0x7f077410c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:27.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.668+0000 7f0779000640 1 --2- 172.21.15.83:0/2355205422 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0774074a60 0x7f0774074ee0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:27.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.668+0000 7f0779000640 1 --2- 172.21.15.83:0/2355205422 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0774076040 0x7f0774074490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:27.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.668+0000 7f0779000640 1 -- 172.21.15.83:0/2355205422 >> 172.21.15.83:0/2355205422 conn(0x7f07740fd5f0 msgr2=0x7f07740ffa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:27.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.668+0000 7f0779000640 1 -- 172.21.15.83:0/2355205422 shutdown_connections 2024-04-24T08:51:27.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.668+0000 7f0779000640 1 -- 172.21.15.83:0/2355205422 wait complete. 2024-04-24T08:51:27.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.669+0000 7f0779000640 1 Processor -- start 2024-04-24T08:51:27.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.669+0000 7f0779000640 1 -- start start 2024-04-24T08:51:27.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.670+0000 7f0779000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0774074a60 0x7f077419a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:27.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.670+0000 7f0779000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0774076040 0x7f077419a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:27.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.670+0000 7f0779000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0774109f40 0x7f07741a1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:27.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.670+0000 7f0779000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f077410fe00 con 0x7f0774109f40 2024-04-24T08:51:27.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.670+0000 7f0779000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f077410fc80 con 0x7f0774076040 2024-04-24T08:51:27.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.670+0000 7f0779000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f077410ff80 con 0x7f0774074a60 2024-04-24T08:51:27.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.670+0000 7f0772800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0774109f40 0x7f07741a1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:27.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.670+0000 7f0772800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0774109f40 0x7f07741a1930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41632/0 (socket says 172.21.15.83:41632) 2024-04-24T08:51:27.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.670+0000 7f076be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0774076040 0x7f077419a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:27.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.670+0000 7f0772800640 1 -- 172.21.15.83:0/2560159720 learned_addr learned my addr 172.21.15.83:0/2560159720 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:51:27.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.670+0000 7f0771e00640 1 --2- 172.21.15.83:0/2560159720 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0774074a60 0x7f077419a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:27.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.670+0000 7f0772800640 1 -- 172.21.15.83:0/2560159720 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0774074a60 msgr2=0x7f077419a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:27.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.670+0000 7f0772800640 1 --2- 172.21.15.83:0/2560159720 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0774074a60 0x7f077419a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:27.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.671+0000 7f0772800640 1 -- 172.21.15.83:0/2560159720 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0774076040 msgr2=0x7f077419a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:27.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.671+0000 7f0772800640 1 --2- 172.21.15.83:0/2560159720 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0774076040 0x7f077419a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:27.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.671+0000 7f0772800640 1 -- 172.21.15.83:0/2560159720 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07741a1fa0 con 0x7f0774109f40 2024-04-24T08:51:27.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.671+0000 7f0772800640 1 --2- 172.21.15.83:0/2560159720 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0774109f40 0x7f07741a1930 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f076c00cce0 tx=0x7f076c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:27.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.671+0000 7f076aa00640 1 -- 172.21.15.83:0/2560159720 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f076c00f450 con 0x7f0774109f40 2024-04-24T08:51:27.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.671+0000 7f076aa00640 1 -- 172.21.15.83:0/2560159720 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f076c00faa0 con 0x7f0774109f40 2024-04-24T08:51:27.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.671+0000 7f0779000640 1 -- 172.21.15.83:0/2560159720 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07741a2290 con 0x7f0774109f40 2024-04-24T08:51:27.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.671+0000 7f076aa00640 1 -- 172.21.15.83:0/2560159720 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f076c018b70 con 0x7f0774109f40 2024-04-24T08:51:27.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.671+0000 7f0779000640 1 -- 172.21.15.83:0/2560159720 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07741a2760 con 0x7f0774109f40 2024-04-24T08:51:27.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.672+0000 7f0779000640 1 -- 172.21.15.83:0/2560159720 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0774108c70 con 0x7f0774109f40 2024-04-24T08:51:27.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.674+0000 7f076aa00640 1 -- 172.21.15.83:0/2560159720 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f076c018d10 con 0x7f0774109f40 2024-04-24T08:51:27.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.675+0000 7f076aa00640 1 --2- 172.21.15.83:0/2560159720 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f07580a3ca0 0x7f07580a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:27.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.676+0000 7f0771e00640 1 -- 172.21.15.83:0/2560159720 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f07580a3ca0 msgr2=0x7f07580a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.83:6800/2550826560 2024-04-24T08:51:27.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.676+0000 7f076aa00640 1 -- 172.21.15.83:0/2560159720 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9237+0+0 (secure 0 0 0) 0x7f076c0d0c80 con 0x7f0774109f40 2024-04-24T08:51:27.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.676+0000 7f0771e00640 1 --2- 172.21.15.83:0/2560159720 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f07580a3ca0 0x7f07580a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T08:51:27.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.679+0000 7f076aa00640 1 -- 172.21.15.83:0/2560159720 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f076c014030 con 0x7f0774109f40 2024-04-24T08:51:27.704 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Apr 24 08:51:27 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-y[32810]: 2024-04-24T08:51:27.576+0000 7f492924e200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-24T08:51:27.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.877+0000 7f0771e00640 1 -- 172.21.15.83:0/2560159720 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f07580a3ca0 msgr2=0x7f07580a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.83:6800/2550826560 2024-04-24T08:51:27.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.877+0000 7f0771e00640 1 --2- 172.21.15.83:0/2560159720 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f07580a3ca0 0x7f07580a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-24T08:51:27.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.878+0000 7f0779000640 1 -- 172.21.15.83:0/2560159720 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f077402d620 con 0x7f0774109f40 2024-04-24T08:51:27.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.884+0000 7f076aa00640 1 -- 172.21.15.83:0/2560159720 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v33)=0 v33) ==== 133+0+0 (secure 0 0 0) 0x7f076c09d270 con 0x7f0774109f40 2024-04-24T08:51:27.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.887+0000 7f0779000640 1 -- 172.21.15.83:0/2560159720 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f07580a3ca0 msgr2=0x7f07580a6160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T08:51:27.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.887+0000 7f0779000640 1 --2- 172.21.15.83:0/2560159720 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f07580a3ca0 0x7f07580a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:27.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.887+0000 7f0779000640 1 -- 172.21.15.83:0/2560159720 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0774109f40 msgr2=0x7f07741a1930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:27.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.887+0000 7f0779000640 1 --2- 172.21.15.83:0/2560159720 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0774109f40 0x7f07741a1930 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f076c00cce0 tx=0x7f076c00a620 comp rx=0 tx=0).stop 2024-04-24T08:51:27.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.887+0000 7f0779000640 1 -- 172.21.15.83:0/2560159720 shutdown_connections 2024-04-24T08:51:27.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.887+0000 7f0779000640 1 --2- 172.21.15.83:0/2560159720 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f07580a3ca0 0x7f07580a6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:27.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.887+0000 7f0779000640 1 --2- 172.21.15.83:0/2560159720 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0774109f40 0x7f07741a1930 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:27.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.887+0000 7f0779000640 1 --2- 172.21.15.83:0/2560159720 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0774076040 0x7f077419a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:27.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.887+0000 7f0779000640 1 --2- 172.21.15.83:0/2560159720 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0774074a60 0x7f077419a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:27.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.887+0000 7f0779000640 1 -- 172.21.15.83:0/2560159720 >> 172.21.15.83:0/2560159720 conn(0x7f07740fd5f0 msgr2=0x7f0774105e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:27.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.887+0000 7f0779000640 1 -- 172.21.15.83:0/2560159720 shutdown_connections 2024-04-24T08:51:27.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:27.888+0000 7f0779000640 1 -- 172.21.15.83:0/2560159720 wait complete. 2024-04-24T08:51:27.903 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:51:27 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:51:27.617+0000 7f3b97266200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-24T08:51:27.903 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:51:27 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:51:27.757+0000 7f3b97266200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-24T08:51:27.906 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-04-24T08:51:27.916 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-04-24T08:51:27.916 INFO:tasks.ceph_fuse:config is {'client.0': {}} 2024-04-24T08:51:27.916 INFO:tasks.ceph_fuse:client.0 config is {'mount_subvol_num': 0} 2024-04-24T08:51:27.916 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-24T08:51:27.917 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-24T08:51:27.917 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T08:51:27.917 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list 2024-04-24T08:51:27.957 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 24 08:51:27 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-z[33677]: 2024-04-24T08:51:27.598+0000 7fef15b51200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-24T08:51:27.957 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 24 08:51:27 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-z[33677]: 2024-04-24T08:51:27.731+0000 7fef15b51200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-24T08:51:27.959 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-04-24T08:51:27.960 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T08:51:27.960 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-24T08:51:28.034 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Apr 24 08:51:27 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-y[32810]: 2024-04-24T08:51:27.705+0000 7f492924e200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-24T08:51:28.070 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T08:51:28.070 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link delete ceph-brx 2024-04-24T08:51:28.164 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:28 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/1271352400' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-24T08:51:28.164 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:28 smithi083 ceph-mon[27907]: mgrmap e19: x(active, since 12m), standbys: z, y 2024-04-24T08:51:28.164 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:28 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/921726943' entity='client.admin' 2024-04-24T08:51:28.164 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:28 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/2560159720' entity='client.admin' 2024-04-24T08:51:28.176 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-04-24T08:51:28.176 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid bd3e039e-0215-11ef-bc93-c7b262605968 -- ceph fs ls 2024-04-24T08:51:28.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:28 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/1271352400' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-24T08:51:28.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:28 smithi138 ceph-mon[31441]: mgrmap e19: x(active, since 12m), standbys: z, y 2024-04-24T08:51:28.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:28 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/921726943' entity='client.admin' 2024-04-24T08:51:28.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:28 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/2560159720' entity='client.admin' 2024-04-24T08:51:28.455 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Apr 24 08:51:28 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-y[32810]: 2024-04-24T08:51:28.099+0000 7f492924e200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-24T08:51:28.455 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Apr 24 08:51:28 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-y[32810]: 2024-04-24T08:51:28.220+0000 7f492924e200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-24T08:51:28.457 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 24 08:51:28 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-z[33677]: 2024-04-24T08:51:28.154+0000 7fef15b51200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-24T08:51:28.457 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 24 08:51:28 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-z[33677]: 2024-04-24T08:51:28.266+0000 7fef15b51200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-24T08:51:28.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:28 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/1271352400' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-24T08:51:28.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:28 smithi189 ceph-mon[31791]: mgrmap e19: x(active, since 12m), standbys: z, y 2024-04-24T08:51:28.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:28 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/921726943' entity='client.admin' 2024-04-24T08:51:28.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:28 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/2560159720' entity='client.admin' 2024-04-24T08:51:28.552 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:51:28 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:51:28.173+0000 7f3b97266200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-24T08:51:28.553 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:51:28 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:51:28.291+0000 7f3b97266200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-24T08:51:28.581 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T08:51:30.043 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:51:29 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:51:29.870+0000 7f3b97266200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-24T08:51:30.171 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Apr 24 08:51:29 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-y[32810]: 2024-04-24T08:51:29.730+0000 7f492924e200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-24T08:51:30.171 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Apr 24 08:51:29 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-y[32810]: 2024-04-24T08:51:29.956+0000 7f492924e200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-24T08:51:30.171 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Apr 24 08:51:30 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-y[32810]: 2024-04-24T08:51:30.171+0000 7f492924e200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-24T08:51:30.207 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 24 08:51:29 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-z[33677]: 2024-04-24T08:51:29.800+0000 7fef15b51200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-24T08:51:30.207 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 24 08:51:30 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-z[33677]: 2024-04-24T08:51:30.028+0000 7fef15b51200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-24T08:51:30.303 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:51:30 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:51:30.103+0000 7f3b97266200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-24T08:51:30.607 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:51:30 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:51:30.347+0000 7f3b97266200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-24T08:51:30.646 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 24 08:51:30 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-z[33677]: 2024-04-24T08:51:30.244+0000 7fef15b51200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-24T08:51:30.923 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Apr 24 08:51:30 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-y[32810]: 2024-04-24T08:51:30.550+0000 7f492924e200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-24T08:51:30.923 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Apr 24 08:51:30 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-y[32810]: 2024-04-24T08:51:30.669+0000 7f492924e200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-24T08:51:30.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:30.930+0000 7f7bc3c00700 1 -- 172.21.15.83:0/2519168330 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7bbc108c50 msgr2=0x7f7bbc109030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:30.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:30.931+0000 7f7bc3c00700 1 --2- 172.21.15.83:0/2519168330 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7bbc108c50 0x7f7bbc109030 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f7bb0009990 tx=0x7f7bb0009ca0 comp rx=0 tx=0).stop 2024-04-24T08:51:30.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:30.931+0000 7f7bc3c00700 1 -- 172.21.15.83:0/2519168330 shutdown_connections 2024-04-24T08:51:30.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:30.931+0000 7f7bc3c00700 1 --2- 172.21.15.83:0/2519168330 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7bbc108c50 0x7f7bbc109030 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:30.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:30.931+0000 7f7bc3c00700 1 --2- 172.21.15.83:0/2519168330 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7bbc103500 0x7f7bbc10f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:30.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:30.931+0000 7f7bc3c00700 1 --2- 172.21.15.83:0/2519168330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7bbc102b40 0x7f7bbc102fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:30.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:30.931+0000 7f7bc3c00700 1 -- 172.21.15.83:0/2519168330 >> 172.21.15.83:0/2519168330 conn(0x7f7bbc079ca0 msgr2=0x7f7bbc07a0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:30.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:30.931+0000 7f7bc3c00700 1 -- 172.21.15.83:0/2519168330 shutdown_connections 2024-04-24T08:51:30.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:30.931+0000 7f7bc3c00700 1 -- 172.21.15.83:0/2519168330 wait complete. 2024-04-24T08:51:30.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:30.932+0000 7f7bc3c00700 1 Processor -- start 2024-04-24T08:51:30.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:30.932+0000 7f7bc3c00700 1 -- start start 2024-04-24T08:51:30.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:30.932+0000 7f7bc3c00700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7bbc102b40 0x7f7bbc19b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:30.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:30.932+0000 7f7bc3c00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7bbc103500 0x7f7bbc19b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:30.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:30.932+0000 7f7bc3c00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7bbc108c50 0x7f7bbc19fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:30.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:30.932+0000 7f7bc3c00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7bbc112d10 con 0x7f7bbc108c50 2024-04-24T08:51:30.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:30.932+0000 7f7bc3c00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7bbc112b90 con 0x7f7bbc103500 2024-04-24T08:51:30.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:30.932+0000 7f7bc3c00700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7bbc112e90 con 0x7f7bbc102b40 2024-04-24T08:51:30.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:30.933+0000 7f7bc1c00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7bbc108c50 0x7f7bbc19fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:30.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:30.933+0000 7f7bc1c00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7bbc108c50 0x7f7bbc19fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50394/0 (socket says 172.21.15.83:50394) 2024-04-24T08:51:30.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:30.933+0000 7f7bc1c00700 1 -- 172.21.15.83:0/160415481 learned_addr learned my addr 172.21.15.83:0/160415481 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:51:30.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:30.933+0000 7f7bc1200700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7bbc102b40 0x7f7bbc19b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:30.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:30.933+0000 7f7bbbe00700 1 --2- 172.21.15.83:0/160415481 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7bbc103500 0x7f7bbc19b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:30.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:30.933+0000 7f7bc1c00700 1 -- 172.21.15.83:0/160415481 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7bbc102b40 msgr2=0x7f7bbc19b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:30.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:30.933+0000 7f7bc1c00700 1 --2- 172.21.15.83:0/160415481 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7bbc102b40 0x7f7bbc19b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:30.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:30.933+0000 7f7bc1c00700 1 -- 172.21.15.83:0/160415481 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7bbc103500 msgr2=0x7f7bbc19b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:30.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:30.933+0000 7f7bc1c00700 1 --2- 172.21.15.83:0/160415481 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7bbc103500 0x7f7bbc19b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:30.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:30.933+0000 7f7bc1c00700 1 -- 172.21.15.83:0/160415481 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bbc1a03a0 con 0x7f7bbc108c50 2024-04-24T08:51:30.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:30.934+0000 7f7bc1c00700 1 --2- 172.21.15.83:0/160415481 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7bbc108c50 0x7f7bbc19fc60 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f7bb400ab50 tx=0x7f7bb400ae60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:30.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:30.934+0000 7f7bb9600700 1 -- 172.21.15.83:0/160415481 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bb4017020 con 0x7f7bbc108c50 2024-04-24T08:51:30.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:30.934+0000 7f7bc3c00700 1 -- 172.21.15.83:0/160415481 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bbc1a0690 con 0x7f7bbc108c50 2024-04-24T08:51:30.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:30.934+0000 7f7bb9600700 1 -- 172.21.15.83:0/160415481 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7bb4004bb0 con 0x7f7bbc108c50 2024-04-24T08:51:30.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:30.934+0000 7f7bb9600700 1 -- 172.21.15.83:0/160415481 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bb4005350 con 0x7f7bbc108c50 2024-04-24T08:51:30.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:30.934+0000 7f7bc3c00700 1 -- 172.21.15.83:0/160415481 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bbc1a7f10 con 0x7f7bbc108c50 2024-04-24T08:51:30.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:30.936+0000 7f7bc3c00700 1 -- 172.21.15.83:0/160415481 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bbc0697e0 con 0x7f7bbc108c50 2024-04-24T08:51:30.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:30.940+0000 7f7bb9600700 1 -- 172.21.15.83:0/160415481 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f7bb40054f0 con 0x7f7bbc108c50 2024-04-24T08:51:30.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:30.941+0000 7f7bb9600700 1 --2- 172.21.15.83:0/160415481 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f7ba40a3c90 0x7f7ba40a6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:30.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:30.941+0000 7f7bb9600700 1 -- 172.21.15.83:0/160415481 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9237+0+0 (secure 0 0 0) 0x7f7bb40d0490 con 0x7f7bbc108c50 2024-04-24T08:51:30.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:30.941+0000 7f7bc1200700 1 -- 172.21.15.83:0/160415481 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f7ba40a3c90 msgr2=0x7f7ba40a6150 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.83:6800/2550826560 2024-04-24T08:51:30.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:30.941+0000 7f7bc1200700 1 --2- 172.21.15.83:0/160415481 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f7ba40a3c90 0x7f7ba40a6150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T08:51:30.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:30.943+0000 7f7bb9600700 1 -- 172.21.15.83:0/160415481 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7bb4021020 con 0x7f7bbc108c50 2024-04-24T08:51:30.956 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 24 08:51:30 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-z[33677]: 2024-04-24T08:51:30.645+0000 7fef15b51200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-24T08:51:30.957 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 24 08:51:30 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-z[33677]: 2024-04-24T08:51:30.769+0000 7fef15b51200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-24T08:51:31.053 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:51:30 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:51:30.751+0000 7f3b97266200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-24T08:51:31.053 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:51:30 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:51:30.879+0000 7f3b97266200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-24T08:51:31.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:31.142+0000 7f7bc1200700 1 -- 172.21.15.83:0/160415481 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f7ba40a3c90 msgr2=0x7f7ba40a6150 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.83:6800/2550826560 2024-04-24T08:51:31.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:31.142+0000 7f7bc1200700 1 --2- 172.21.15.83:0/160415481 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f7ba40a3c90 0x7f7ba40a6150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-24T08:51:31.177 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Apr 24 08:51:30 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-y[32810]: 2024-04-24T08:51:30.922+0000 7f492924e200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-24T08:51:31.177 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Apr 24 08:51:31 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-y[32810]: 2024-04-24T08:51:31.176+0000 7f492924e200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-24T08:51:31.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:31.216+0000 7f7bc3c00700 1 -- 172.21.15.83:0/160415481 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f7bbc1a80e0 con 0x7f7bbc108c50 2024-04-24T08:51:31.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:31.216+0000 7f7bb9600700 1 -- 172.21.15.83:0/160415481 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v27) ==== 53+0+88 (secure 0 0 0) 0x7f7bb409cb10 con 0x7f7bbc108c50 2024-04-24T08:51:31.217 INFO:teuthology.orchestra.run.smithi083.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-24T08:51:31.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:31.219+0000 7f7bc3c00700 1 -- 172.21.15.83:0/160415481 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f7ba40a3c90 msgr2=0x7f7ba40a6150 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T08:51:31.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:31.219+0000 7f7bc3c00700 1 --2- 172.21.15.83:0/160415481 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f7ba40a3c90 0x7f7ba40a6150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:31.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:31.219+0000 7f7bc3c00700 1 -- 172.21.15.83:0/160415481 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7bbc108c50 msgr2=0x7f7bbc19fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:31.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:31.219+0000 7f7bc3c00700 1 --2- 172.21.15.83:0/160415481 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7bbc108c50 0x7f7bbc19fc60 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f7bb400ab50 tx=0x7f7bb400ae60 comp rx=0 tx=0).stop 2024-04-24T08:51:31.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:31.219+0000 7f7bc3c00700 1 -- 172.21.15.83:0/160415481 shutdown_connections 2024-04-24T08:51:31.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:31.219+0000 7f7bc3c00700 1 --2- 172.21.15.83:0/160415481 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f7ba40a3c90 0x7f7ba40a6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:31.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:31.219+0000 7f7bc3c00700 1 --2- 172.21.15.83:0/160415481 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7bbc108c50 0x7f7bbc19fc60 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:31.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:31.219+0000 7f7bc3c00700 1 --2- 172.21.15.83:0/160415481 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7bbc102b40 0x7f7bbc19b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:31.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:31.219+0000 7f7bc3c00700 1 --2- 172.21.15.83:0/160415481 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7bbc103500 0x7f7bbc19b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:31.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:31.219+0000 7f7bc3c00700 1 -- 172.21.15.83:0/160415481 >> 172.21.15.83:0/160415481 conn(0x7f7bbc079ca0 msgr2=0x7f7bbc1015b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:31.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:31.219+0000 7f7bc3c00700 1 -- 172.21.15.83:0/160415481 shutdown_connections 2024-04-24T08:51:31.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:31.219+0000 7f7bc3c00700 1 -- 172.21.15.83:0/160415481 wait complete. 2024-04-24T08:51:31.281 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 24 08:51:31 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-z[33677]: 2024-04-24T08:51:31.023+0000 7fef15b51200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-24T08:51:31.281 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:31 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/160415481' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-24T08:51:31.415 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:51:31 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:51:31.145+0000 7f3b97266200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-24T08:51:31.415 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:31 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/160415481' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-24T08:51:31.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:31 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/160415481' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-24T08:51:31.676 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:51:31 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:51:31.414+0000 7f3b97266200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-24T08:51:31.706 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 24 08:51:31 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-z[33677]: 2024-04-24T08:51:31.280+0000 7fef15b51200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-24T08:51:32.133 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-24T08:51:32.133 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-24T08:51:32.133 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi083.front.sepia.ceph.com 2024-04-24T08:51:32.134 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-24T08:51:32.134 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-24T08:51:32.134 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-24T08:51:32.134 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-24T08:51:32.134 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-24T08:51:32.134 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-24T08:51:32.134 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T08:51:32.134 DEBUG:teuthology.orchestra.run.smithi083:> ip addr 2024-04-24T08:51:32.185 INFO:teuthology.orchestra.run.smithi083.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-24T08:51:32.185 INFO:teuthology.orchestra.run.smithi083.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-24T08:51:32.185 INFO:teuthology.orchestra.run.smithi083.stdout: inet 127.0.0.1/8 scope host lo 2024-04-24T08:51:32.185 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-24T08:51:32.185 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 ::1/128 scope host 2024-04-24T08:51:32.185 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-24T08:51:32.185 INFO:teuthology.orchestra.run.smithi083.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T08:51:32.185 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1e brd ff:ff:ff:ff:ff:ff 2024-04-24T08:51:32.185 INFO:teuthology.orchestra.run.smithi083.stdout: altname enp7s0f0 2024-04-24T08:51:32.185 INFO:teuthology.orchestra.run.smithi083.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T08:51:32.185 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1f brd ff:ff:ff:ff:ff:ff 2024-04-24T08:51:32.186 INFO:teuthology.orchestra.run.smithi083.stdout: altname enp7s0f1 2024-04-24T08:51:32.186 INFO:teuthology.orchestra.run.smithi083.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T08:51:32.186 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:90 brd ff:ff:ff:ff:ff:ff 2024-04-24T08:51:32.186 INFO:teuthology.orchestra.run.smithi083.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-24T08:51:32.186 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:91 brd ff:ff:ff:ff:ff:ff 2024-04-24T08:51:32.186 INFO:teuthology.orchestra.run.smithi083.stdout: inet 172.21.15.83/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-24T08:51:32.186 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft 41808sec preferred_lft 41808sec 2024-04-24T08:51:32.186 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::ec4:7aff:fe8f:ce91/64 scope link 2024-04-24T08:51:32.186 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-24T08:51:32.186 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-24T08:51:32.187 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T08:51:32.187 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-24T08:51:32.187 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link add name ceph-brx type bridge 2024-04-24T08:51:32.187 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip addr flush dev ceph-brx 2024-04-24T08:51:32.187 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link set ceph-brx up 2024-04-24T08:51:32.187 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-24T08:51:32.187 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-24T08:51:32.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T08:51:32.339 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:51:32 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:51:32.228+0000 7f3b97266200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-24T08:51:32.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T08:51:32.411 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T08:51:32.412 DEBUG:teuthology.orchestra.run.smithi083:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-24T08:51:32.454 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Apr 24 08:51:31 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-y[32810]: 2024-04-24T08:51:31.955+0000 7f492924e200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-24T08:51:32.456 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 24 08:51:32 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-z[33677]: 2024-04-24T08:51:32.066+0000 7fef15b51200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-24T08:51:32.469 INFO:teuthology.orchestra.run.smithi083.stdout:1 2024-04-24T08:51:32.472 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T08:51:32.472 DEBUG:teuthology.orchestra.run.smithi083:> route 2024-04-24T08:51:32.510 INFO:teuthology.orchestra.run.smithi083.stdout:Kernel IP routing table 2024-04-24T08:51:32.510 INFO:teuthology.orchestra.run.smithi083.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-24T08:51:32.510 INFO:teuthology.orchestra.run.smithi083.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-24T08:51:32.510 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-24T08:51:32.510 INFO:teuthology.orchestra.run.smithi083.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-24T08:51:32.511 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T08:51:32.511 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-24T08:51:32.511 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-24T08:51:32.511 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-24T08:51:32.511 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-24T08:51:32.512 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-24T08:51:32.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T08:51:32.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T08:51:32.754 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T08:51:32.755 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list 2024-04-24T08:51:32.803 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T08:51:32.803 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list-id 2024-04-24T08:51:32.847 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T08:51:32.847 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-24T08:51:32.847 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T08:51:32.847 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-24T08:51:32.847 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-24T08:51:32.887 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Apr 24 08:51:32 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-y[32810]: 2024-04-24T08:51:32.500+0000 7f492924e200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-24T08:51:32.887 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Apr 24 08:51:32 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-y[32810]: 2024-04-24T08:51:32.625+0000 7f492924e200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-24T08:51:32.887 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Apr 24 08:51:32 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-y[32810]: 2024-04-24T08:51:32.887+0000 7f492924e200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-24T08:51:32.956 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 24 08:51:32 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-z[33677]: 2024-04-24T08:51:32.618+0000 7fef15b51200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-24T08:51:32.957 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 24 08:51:32 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-z[33677]: 2024-04-24T08:51:32.747+0000 7fef15b51200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-24T08:51:32.959 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:51:32 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:51:32.823+0000 7f3b97266200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-24T08:51:32.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T08:51:33.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T08:51:33.058 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-24T08:51:33.058 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T08:51:33.058 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-24T08:51:33.058 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-24T08:51:33.058 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-24T08:51:33.058 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-24T08:51:33.058 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-24T08:51:33.058 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-24T08:51:33.059 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-24T08:51:33.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:33 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T08:51:33.212 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:51:32 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:51:32.958+0000 7f3b97266200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-24T08:51:33.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T08:51:33.305 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T08:51:33.306 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-24T08:51:33.306 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link set brx.0 up 2024-04-24T08:51:33.306 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link set dev brx.0 master ceph-brx 2024-04-24T08:51:33.306 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-24T08:51:33.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:33 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T08:51:33.456 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 24 08:51:33 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-z[33677]: 2024-04-24T08:51:33.018+0000 7fef15b51200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-24T08:51:33.467 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:51:33 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:51:33.249+0000 7f3b97266200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-24T08:51:33.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T08:51:33.491 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {'mount_subvol_num': 0} 2024-04-24T08:51:33.491 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T08:51:33.491 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-24T08:51:33.532 INFO:teuthology.orchestra.run.smithi083.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-24T08:51:33.532 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T08:51:33.532 DEBUG:teuthology.orchestra.run.smithi083:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-24T08:51:33.624 DEBUG:teuthology.orchestra.run.smithi083:> sudo modprobe fuse 2024-04-24T08:51:33.705 DEBUG:teuthology.orchestra.run.smithi083:> cat /proc/self/mounts | awk '{print $2}' 2024-04-24T08:51:33.754 INFO:teuthology.orchestra.run.smithi083.stdout:/proc 2024-04-24T08:51:33.754 INFO:teuthology.orchestra.run.smithi083.stdout:/sys 2024-04-24T08:51:33.754 INFO:teuthology.orchestra.run.smithi083.stdout:/dev 2024-04-24T08:51:33.754 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/security 2024-04-24T08:51:33.754 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/shm 2024-04-24T08:51:33.754 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/pts 2024-04-24T08:51:33.754 INFO:teuthology.orchestra.run.smithi083.stdout:/run 2024-04-24T08:51:33.754 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/cgroup 2024-04-24T08:51:33.754 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/pstore 2024-04-24T08:51:33.754 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/bpf 2024-04-24T08:51:33.754 INFO:teuthology.orchestra.run.smithi083.stdout:/ 2024-04-24T08:51:33.755 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/selinux 2024-04-24T08:51:33.755 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T08:51:33.755 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/hugepages 2024-04-24T08:51:33.755 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/mqueue 2024-04-24T08:51:33.755 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/debug 2024-04-24T08:51:33.755 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/tracing 2024-04-24T08:51:33.755 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/fuse/connections 2024-04-24T08:51:33.755 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/config 2024-04-24T08:51:33.755 INFO:teuthology.orchestra.run.smithi083.stdout:/run/credentials/systemd-sysctl.service 2024-04-24T08:51:33.755 INFO:teuthology.orchestra.run.smithi083.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-24T08:51:33.755 INFO:teuthology.orchestra.run.smithi083.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-24T08:51:33.755 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/nfs/rpc_pipefs 2024-04-24T08:51:33.755 INFO:teuthology.orchestra.run.smithi083.stdout:/run/user/1000 2024-04-24T08:51:33.756 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph 2024-04-24T08:51:33.756 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/containers/storage/overlay 2024-04-24T08:51:33.756 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/containers/storage/overlay/49049192a6c660dfa299f98afedd81704406f70e80d6c99e3017cc3798469627/merged 2024-04-24T08:51:33.756 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/containers/storage/overlay/6a0c2bc8270c7dfd42477801d69ee13df03cea164cd5ecacf24b5fdc496904a4/merged 2024-04-24T08:51:33.756 INFO:teuthology.orchestra.run.smithi083.stdout:/run/user/0 2024-04-24T08:51:33.756 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T08:51:33.756 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/containers/storage/overlay/c92130de88cd4e5f8b30940a442652d0f71b4f10030df7685cf09f54ee334ca9/merged 2024-04-24T08:51:33.756 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/containers/storage/overlay/91a1ec8e5a8b56746793a65be525d445eb58a998d01ab8a7be11fefc80543fc1/merged 2024-04-24T08:51:33.756 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/containers/storage/overlay/59c7644b789b5c770c3accf58b941cb591889e0fcd9d79c001122751ebd34006/merged 2024-04-24T08:51:33.756 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/containers/storage/overlay/c39d5dae117cb52392575e92167a37143623a99da0c2d54cdd6ba44559ddd9b1/merged 2024-04-24T08:51:33.756 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/containers/storage/overlay/2aaa102c1272f2328e73534c89583df21950659a9917506415c62d4dc7cfd384/merged 2024-04-24T08:51:33.756 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/containers/storage/overlay/d997c279ce46cf62112b51cf6a3efea5734bbd6c9e363b3d45dcf7418695073a/merged 2024-04-24T08:51:33.756 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/containers/storage/overlay/01054164ae515e45a02a7aae9331e96774dfdccb9a503b4ee1bc6532a01ca330/merged 2024-04-24T08:51:33.757 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/containers/storage/overlay/f3868c721fea77ab7c659487533247b0fbc23b8ef9840112767a07b6744f8a12/merged 2024-04-24T08:51:33.757 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns 2024-04-24T08:51:33.757 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T08:51:33.757 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T08:51:33.758 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T08:51:33.758 DEBUG:teuthology.orchestra.run.smithi083:> ls /sys/fs/fuse/connections 2024-04-24T08:51:33.797 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-24T08:51:33.797 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0 --client_mountpoint=/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2 --client_fs=cephfs) 2024-04-24T08:51:33.843 DEBUG:teuthology.orchestra.run.smithi083:> sudo modprobe fuse 2024-04-24T08:51:33.910 DEBUG:teuthology.orchestra.run.smithi083:> cat /proc/self/mounts | awk '{print $2}' 2024-04-24T08:51:33.953 INFO:teuthology.orchestra.run.smithi083.stdout:/proc 2024-04-24T08:51:33.953 INFO:teuthology.orchestra.run.smithi083.stdout:/sys 2024-04-24T08:51:33.953 INFO:teuthology.orchestra.run.smithi083.stdout:/dev 2024-04-24T08:51:33.953 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/security 2024-04-24T08:51:33.953 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/shm 2024-04-24T08:51:33.953 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/pts 2024-04-24T08:51:33.953 INFO:teuthology.orchestra.run.smithi083.stdout:/run 2024-04-24T08:51:33.953 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/cgroup 2024-04-24T08:51:33.953 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/pstore 2024-04-24T08:51:33.953 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/bpf 2024-04-24T08:51:33.953 INFO:teuthology.orchestra.run.smithi083.stdout:/ 2024-04-24T08:51:33.953 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/selinux 2024-04-24T08:51:33.954 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T08:51:33.954 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/hugepages 2024-04-24T08:51:33.954 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/mqueue 2024-04-24T08:51:33.954 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/debug 2024-04-24T08:51:33.954 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/tracing 2024-04-24T08:51:33.954 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/fuse/connections 2024-04-24T08:51:33.954 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/config 2024-04-24T08:51:33.954 INFO:teuthology.orchestra.run.smithi083.stdout:/run/credentials/systemd-sysctl.service 2024-04-24T08:51:33.954 INFO:teuthology.orchestra.run.smithi083.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-24T08:51:33.954 INFO:teuthology.orchestra.run.smithi083.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-24T08:51:33.954 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/nfs/rpc_pipefs 2024-04-24T08:51:33.954 INFO:teuthology.orchestra.run.smithi083.stdout:/run/user/1000 2024-04-24T08:51:33.954 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph 2024-04-24T08:51:33.954 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/containers/storage/overlay 2024-04-24T08:51:33.955 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/containers/storage/overlay/49049192a6c660dfa299f98afedd81704406f70e80d6c99e3017cc3798469627/merged 2024-04-24T08:51:33.955 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/containers/storage/overlay/6a0c2bc8270c7dfd42477801d69ee13df03cea164cd5ecacf24b5fdc496904a4/merged 2024-04-24T08:51:33.955 INFO:teuthology.orchestra.run.smithi083.stdout:/run/user/0 2024-04-24T08:51:33.955 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T08:51:33.955 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/containers/storage/overlay/c92130de88cd4e5f8b30940a442652d0f71b4f10030df7685cf09f54ee334ca9/merged 2024-04-24T08:51:33.955 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/containers/storage/overlay/91a1ec8e5a8b56746793a65be525d445eb58a998d01ab8a7be11fefc80543fc1/merged 2024-04-24T08:51:33.955 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/containers/storage/overlay/59c7644b789b5c770c3accf58b941cb591889e0fcd9d79c001122751ebd34006/merged 2024-04-24T08:51:33.955 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/containers/storage/overlay/c39d5dae117cb52392575e92167a37143623a99da0c2d54cdd6ba44559ddd9b1/merged 2024-04-24T08:51:33.955 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/containers/storage/overlay/2aaa102c1272f2328e73534c89583df21950659a9917506415c62d4dc7cfd384/merged 2024-04-24T08:51:33.955 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/containers/storage/overlay/d997c279ce46cf62112b51cf6a3efea5734bbd6c9e363b3d45dcf7418695073a/merged 2024-04-24T08:51:33.955 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/containers/storage/overlay/01054164ae515e45a02a7aae9331e96774dfdccb9a503b4ee1bc6532a01ca330/merged 2024-04-24T08:51:33.955 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/containers/storage/overlay/f3868c721fea77ab7c659487533247b0fbc23b8ef9840112767a07b6744f8a12/merged 2024-04-24T08:51:33.955 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns 2024-04-24T08:51:33.955 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T08:51:33.956 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T08:51:33.956 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Apr 24 08:51:33 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-y[32810]: 2024-04-24T08:51:33.538+0000 7f492924e200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-24T08:51:33.956 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Apr 24 08:51:33 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-y[32810]: 2024-04-24T08:51:33.660+0000 7f492924e200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-24T08:51:33.957 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T08:51:33.957 DEBUG:teuthology.orchestra.run.smithi083:> ls /sys/fs/fuse/connections 2024-04-24T08:51:33.957 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 24 08:51:33 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-z[33677]: 2024-04-24T08:51:33.670+0000 7fef15b51200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-24T08:51:33.958 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 24 08:51:33 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-z[33677]: 2024-04-24T08:51:33.795+0000 7fef15b51200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-24T08:51:34.015 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi083.stderr:2024-04-24T08:51:34.015+0000 7f4fbb81b480 -1 init, newargv = 0x561c4fb3e380 newargc=15 2024-04-24T08:51:34.016 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi083.stderr:ceph-fuse[82517]: starting ceph client 2024-04-24T08:51:34.069 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi083.stderr:ceph-fuse[82517]: starting fuse 2024-04-24T08:51:34.293 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Apr 24 08:51:33 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-y[32810]: 2024-04-24T08:51:33.975+0000 7f492924e200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-24T08:51:34.293 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Apr 24 08:51:34 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-y[32810]: 2024-04-24T08:51:34.166+0000 7f492924e200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-24T08:51:34.303 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:51:33 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:51:33.933+0000 7f3b97266200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-24T08:51:34.303 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:51:34 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:51:34.076+0000 7f3b97266200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-24T08:51:34.438 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 24 08:51:34 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-z[33677]: 2024-04-24T08:51:34.129+0000 7fef15b51200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-24T08:51:34.438 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 24 08:51:34 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-z[33677]: 2024-04-24T08:51:34.310+0000 7fef15b51200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-24T08:51:34.705 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Apr 24 08:51:34 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-y[32810]: 2024-04-24T08:51:34.291+0000 7f492924e200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-24T08:51:34.705 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Apr 24 08:51:34 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-y[32810]: 2024-04-24T08:51:34.442+0000 7f492924e200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-24T08:51:34.706 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 24 08:51:34 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-z[33677]: 2024-04-24T08:51:34.438+0000 7fef15b51200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-24T08:51:34.706 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 24 08:51:34 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-z[33677]: 2024-04-24T08:51:34.589+0000 7fef15b51200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-24T08:51:34.707 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:51:34 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:51:34.400+0000 7f3b97266200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-24T08:51:34.707 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:51:34 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:51:34.578+0000 7f3b97266200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-24T08:51:35.053 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:51:34 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:51:34.706+0000 7f3b97266200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-24T08:51:35.053 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:51:34 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:51:34.858+0000 7f3b97266200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-24T08:51:35.140 DEBUG:teuthology.orchestra.run.smithi083:> sudo modprobe fuse 2024-04-24T08:51:35.205 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Apr 24 08:51:34 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-y[32810]: 2024-04-24T08:51:34.812+0000 7f492924e200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-24T08:51:35.220 DEBUG:teuthology.orchestra.run.smithi083:> cat /proc/self/mounts | awk '{print $2}' 2024-04-24T08:51:35.271 INFO:teuthology.orchestra.run.smithi083.stdout:/proc 2024-04-24T08:51:35.271 INFO:teuthology.orchestra.run.smithi083.stdout:/sys 2024-04-24T08:51:35.271 INFO:teuthology.orchestra.run.smithi083.stdout:/dev 2024-04-24T08:51:35.271 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/security 2024-04-24T08:51:35.271 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/shm 2024-04-24T08:51:35.271 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/pts 2024-04-24T08:51:35.271 INFO:teuthology.orchestra.run.smithi083.stdout:/run 2024-04-24T08:51:35.271 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/cgroup 2024-04-24T08:51:35.271 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/pstore 2024-04-24T08:51:35.271 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/bpf 2024-04-24T08:51:35.272 INFO:teuthology.orchestra.run.smithi083.stdout:/ 2024-04-24T08:51:35.272 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/selinux 2024-04-24T08:51:35.272 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T08:51:35.272 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/hugepages 2024-04-24T08:51:35.272 INFO:teuthology.orchestra.run.smithi083.stdout:/dev/mqueue 2024-04-24T08:51:35.272 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/debug 2024-04-24T08:51:35.272 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/tracing 2024-04-24T08:51:35.272 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/fs/fuse/connections 2024-04-24T08:51:35.272 INFO:teuthology.orchestra.run.smithi083.stdout:/sys/kernel/config 2024-04-24T08:51:35.272 INFO:teuthology.orchestra.run.smithi083.stdout:/run/credentials/systemd-sysctl.service 2024-04-24T08:51:35.272 INFO:teuthology.orchestra.run.smithi083.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-24T08:51:35.272 INFO:teuthology.orchestra.run.smithi083.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-24T08:51:35.272 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/nfs/rpc_pipefs 2024-04-24T08:51:35.273 INFO:teuthology.orchestra.run.smithi083.stdout:/run/user/1000 2024-04-24T08:51:35.273 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/ceph 2024-04-24T08:51:35.273 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/containers/storage/overlay 2024-04-24T08:51:35.273 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/containers/storage/overlay/49049192a6c660dfa299f98afedd81704406f70e80d6c99e3017cc3798469627/merged 2024-04-24T08:51:35.273 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/containers/storage/overlay/6a0c2bc8270c7dfd42477801d69ee13df03cea164cd5ecacf24b5fdc496904a4/merged 2024-04-24T08:51:35.273 INFO:teuthology.orchestra.run.smithi083.stdout:/run/user/0 2024-04-24T08:51:35.273 INFO:teuthology.orchestra.run.smithi083.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T08:51:35.273 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/containers/storage/overlay/c92130de88cd4e5f8b30940a442652d0f71b4f10030df7685cf09f54ee334ca9/merged 2024-04-24T08:51:35.273 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/containers/storage/overlay/91a1ec8e5a8b56746793a65be525d445eb58a998d01ab8a7be11fefc80543fc1/merged 2024-04-24T08:51:35.273 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/containers/storage/overlay/59c7644b789b5c770c3accf58b941cb591889e0fcd9d79c001122751ebd34006/merged 2024-04-24T08:51:35.273 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/containers/storage/overlay/c39d5dae117cb52392575e92167a37143623a99da0c2d54cdd6ba44559ddd9b1/merged 2024-04-24T08:51:35.273 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/containers/storage/overlay/2aaa102c1272f2328e73534c89583df21950659a9917506415c62d4dc7cfd384/merged 2024-04-24T08:51:35.273 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/containers/storage/overlay/d997c279ce46cf62112b51cf6a3efea5734bbd6c9e363b3d45dcf7418695073a/merged 2024-04-24T08:51:35.274 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/containers/storage/overlay/01054164ae515e45a02a7aae9331e96774dfdccb9a503b4ee1bc6532a01ca330/merged 2024-04-24T08:51:35.274 INFO:teuthology.orchestra.run.smithi083.stdout:/var/lib/containers/storage/overlay/f3868c721fea77ab7c659487533247b0fbc23b8ef9840112767a07b6744f8a12/merged 2024-04-24T08:51:35.274 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns 2024-04-24T08:51:35.274 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T08:51:35.274 INFO:teuthology.orchestra.run.smithi083.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T08:51:35.274 INFO:teuthology.orchestra.run.smithi083.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-24T08:51:35.275 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T08:51:35.275 DEBUG:teuthology.orchestra.run.smithi083:> ls /sys/fs/fuse/connections 2024-04-24T08:51:35.322 INFO:teuthology.orchestra.run.smithi083.stdout:114 2024-04-24T08:51:35.323 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [114] 2024-04-24T08:51:35.323 DEBUG:teuthology.orchestra.run.smithi083:> sudo stdin-killer -- python3 -c ' 2024-04-24T08:51:35.323 DEBUG:teuthology.orchestra.run.smithi083:> import glob 2024-04-24T08:51:35.323 DEBUG:teuthology.orchestra.run.smithi083:> import re 2024-04-24T08:51:35.323 DEBUG:teuthology.orchestra.run.smithi083:> import os 2024-04-24T08:51:35.323 DEBUG:teuthology.orchestra.run.smithi083:> import subprocess 2024-04-24T08:51:35.323 DEBUG:teuthology.orchestra.run.smithi083:> 2024-04-24T08:51:35.323 DEBUG:teuthology.orchestra.run.smithi083:> def _find_admin_socket(client_name): 2024-04-24T08:51:35.323 DEBUG:teuthology.orchestra.run.smithi083:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-24T08:51:35.323 DEBUG:teuthology.orchestra.run.smithi083:> files = glob.glob(asok_path) 2024-04-24T08:51:35.323 DEBUG:teuthology.orchestra.run.smithi083:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-24T08:51:35.323 DEBUG:teuthology.orchestra.run.smithi083:> 2024-04-24T08:51:35.324 DEBUG:teuthology.orchestra.run.smithi083:> # Given a non-glob path, it better be there 2024-04-24T08:51:35.324 DEBUG:teuthology.orchestra.run.smithi083:> if "*" not in asok_path: 2024-04-24T08:51:35.324 DEBUG:teuthology.orchestra.run.smithi083:> assert(len(files) == 1) 2024-04-24T08:51:35.324 DEBUG:teuthology.orchestra.run.smithi083:> return files[0] 2024-04-24T08:51:35.324 DEBUG:teuthology.orchestra.run.smithi083:> 2024-04-24T08:51:35.324 DEBUG:teuthology.orchestra.run.smithi083:> for f in files: 2024-04-24T08:51:35.324 DEBUG:teuthology.orchestra.run.smithi083:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-24T08:51:35.324 DEBUG:teuthology.orchestra.run.smithi083:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-24T08:51:35.324 DEBUG:teuthology.orchestra.run.smithi083:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-24T08:51:35.324 DEBUG:teuthology.orchestra.run.smithi083:> contents = proc_f.read() 2024-04-24T08:51:35.324 DEBUG:teuthology.orchestra.run.smithi083:> if mountpoint in contents: 2024-04-24T08:51:35.324 DEBUG:teuthology.orchestra.run.smithi083:> return f 2024-04-24T08:51:35.324 DEBUG:teuthology.orchestra.run.smithi083:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-24T08:51:35.324 DEBUG:teuthology.orchestra.run.smithi083:> 2024-04-24T08:51:35.325 DEBUG:teuthology.orchestra.run.smithi083:> print(_find_admin_socket("client.0")) 2024-04-24T08:51:35.325 DEBUG:teuthology.orchestra.run.smithi083:> ' 2024-04-24T08:51:35.456 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 24 08:51:34 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-z[33677]: 2024-04-24T08:51:34.964+0000 7fef15b51200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-24T08:51:35.521 INFO:teuthology.orchestra.run.smithi083.stdout:/var/run/ceph/ceph-client.0.82517.asok 2024-04-24T08:51:35.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T08:51:35.535 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:51:35 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:51:35.233+0000 7f3b97266200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-24T08:51:35.538 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.82517.asok 2024-04-24T08:51:35.538 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T08:51:35.539 DEBUG:teuthology.orchestra.run.smithi083:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.82517.asok status 2024-04-24T08:51:35.756 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T08:51:35.757 INFO:teuthology.orchestra.run.smithi083.stdout: "metadata": { 2024-04-24T08:51:35.757 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_sha1": "1f81fb397ae98da7563d451a78c61574c8f4e6e0", 2024-04-24T08:51:35.757 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_version": "ceph version 19.0.0-3116-g1f81fb39 (1f81fb397ae98da7563d451a78c61574c8f4e6e0) squid (dev)", 2024-04-24T08:51:35.757 INFO:teuthology.orchestra.run.smithi083.stdout: "entity_id": "0", 2024-04-24T08:51:35.757 INFO:teuthology.orchestra.run.smithi083.stdout: "hostname": "smithi083", 2024-04-24T08:51:35.757 INFO:teuthology.orchestra.run.smithi083.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-24T08:51:35.757 INFO:teuthology.orchestra.run.smithi083.stdout: "pid": "82517", 2024-04-24T08:51:35.757 INFO:teuthology.orchestra.run.smithi083.stdout: "root": "/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2" 2024-04-24T08:51:35.757 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T08:51:35.757 INFO:teuthology.orchestra.run.smithi083.stdout: "dentry_count": 0, 2024-04-24T08:51:35.757 INFO:teuthology.orchestra.run.smithi083.stdout: "dentry_pinned_count": 0, 2024-04-24T08:51:35.757 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 15603, 2024-04-24T08:51:35.757 INFO:teuthology.orchestra.run.smithi083.stdout: "inst": { 2024-04-24T08:51:35.757 INFO:teuthology.orchestra.run.smithi083.stdout: "name": { 2024-04-24T08:51:35.758 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "client", 2024-04-24T08:51:35.758 INFO:teuthology.orchestra.run.smithi083.stdout: "num": 15603 2024-04-24T08:51:35.758 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T08:51:35.758 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": { 2024-04-24T08:51:35.758 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-04-24T08:51:35.758 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "192.168.0.1:0", 2024-04-24T08:51:35.758 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 3843544189 2024-04-24T08:51:35.758 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T08:51:35.758 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T08:51:35.758 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": { 2024-04-24T08:51:35.758 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-04-24T08:51:35.758 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "192.168.0.1:0", 2024-04-24T08:51:35.758 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 3843544189 2024-04-24T08:51:35.758 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T08:51:35.758 INFO:teuthology.orchestra.run.smithi083.stdout: "inst_str": "client.15603 192.168.0.1:0/3843544189", 2024-04-24T08:51:35.758 INFO:teuthology.orchestra.run.smithi083.stdout: "addr_str": "192.168.0.1:0/3843544189", 2024-04-24T08:51:35.759 INFO:teuthology.orchestra.run.smithi083.stdout: "inode_count": 5, 2024-04-24T08:51:35.759 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_epoch": 27, 2024-04-24T08:51:35.759 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_epoch": 93, 2024-04-24T08:51:35.759 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_epoch_barrier": 0, 2024-04-24T08:51:35.759 INFO:teuthology.orchestra.run.smithi083.stdout: "blocklisted": false, 2024-04-24T08:51:35.759 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs" 2024-04-24T08:51:35.759 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T08:51:35.771 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T08:51:35.771 DEBUG:teuthology.orchestra.run.smithi083:> stat --file-system '--printf=%T 2024-04-24T08:51:35.771 DEBUG:teuthology.orchestra.run.smithi083:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-24T08:51:35.820 INFO:teuthology.orchestra.run.smithi083.stdout:fuseblk 2024-04-24T08:51:35.820 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-24T08:51:35.820 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T08:51:35.820 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-24T08:51:35.837 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Apr 24 08:51:35 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-y[32810]: 2024-04-24T08:51:35.557+0000 7f492924e200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-24T08:51:35.899 INFO:teuthology.run_tasks:Running task exec... 2024-04-24T08:51:35.910 INFO:teuthology.task.exec:Executing custom commands... 2024-04-24T08:51:35.910 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi083.front.sepia.ceph.com 2024-04-24T08:51:35.910 DEBUG:teuthology.orchestra.run.smithi083:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-04-24T08:51:36.015 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 24 08:51:35 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-z[33677]: 2024-04-24T08:51:35.725+0000 7fef15b51200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-24T08:51:36.015 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:35 smithi189 ceph-mon[31791]: Standby manager daemon y restarted 2024-04-24T08:51:36.016 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:35 smithi189 ceph-mon[31791]: Standby manager daemon y started 2024-04-24T08:51:36.016 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:35 smithi189 ceph-mon[31791]: from='mgr.? 172.21.15.138:0/449961047' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-24T08:51:36.016 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:35 smithi189 ceph-mon[31791]: from='mgr.? 172.21.15.138:0/449961047' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T08:51:36.016 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:35 smithi189 ceph-mon[31791]: from='mgr.? 172.21.15.138:0/449961047' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-24T08:51:36.016 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:35 smithi189 ceph-mon[31791]: from='mgr.? 172.21.15.138:0/449961047' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T08:51:36.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:36.090+0000 7f4607e00640 1 -- 172.21.15.83:0/3962754972 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46081039a0 msgr2=0x7f460810a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:36.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:36.090+0000 7f4607e00640 1 --2- 172.21.15.83:0/3962754972 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46081039a0 0x7f460810a2c0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f4600009ca0 tx=0x7f460002a8c0 comp rx=0 tx=0).stop 2024-04-24T08:51:36.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:36.090+0000 7f4607e00640 1 -- 172.21.15.83:0/3962754972 shutdown_connections 2024-04-24T08:51:36.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:36.090+0000 7f4607e00640 1 --2- 172.21.15.83:0/3962754972 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46081039a0 0x7f460810a2c0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:36.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:36.090+0000 7f4607e00640 1 --2- 172.21.15.83:0/3962754972 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4608102fe0 0x7f4608103460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:36.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:36.090+0000 7f4607e00640 1 --2- 172.21.15.83:0/3962754972 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4608101e90 0x7f46081022b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:36.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:36.090+0000 7f4607e00640 1 -- 172.21.15.83:0/3962754972 >> 172.21.15.83:0/3962754972 conn(0x7f46080fd400 msgr2=0x7f46080ff840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:36.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:36.090+0000 7f4607e00640 1 -- 172.21.15.83:0/3962754972 shutdown_connections 2024-04-24T08:51:36.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:36.090+0000 7f4607e00640 1 -- 172.21.15.83:0/3962754972 wait complete. 2024-04-24T08:51:36.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:36.091+0000 7f4607e00640 1 Processor -- start 2024-04-24T08:51:36.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:36.091+0000 7f4607e00640 1 -- start start 2024-04-24T08:51:36.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:36.092+0000 7f4607e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4608101e90 0x7f460819a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:36.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:36.092+0000 7f4607e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4608102fe0 0x7f460819a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:36.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:36.092+0000 7f4607e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f46081039a0 0x7f46081a1810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:36.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:36.092+0000 7f4607e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f460810da30 con 0x7f4608101e90 2024-04-24T08:51:36.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:36.092+0000 7f4607e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f460810d8b0 con 0x7f4608102fe0 2024-04-24T08:51:36.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:36.092+0000 7f4607e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f460810dbb0 con 0x7f46081039a0 2024-04-24T08:51:36.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:36.092+0000 7f4606a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4608101e90 0x7f460819a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:36.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:36.092+0000 7f4606a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4608101e90 0x7f460819a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50414/0 (socket says 172.21.15.83:50414) 2024-04-24T08:51:36.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:36.092+0000 7f4606a00640 1 -- 172.21.15.83:0/233630380 learned_addr learned my addr 172.21.15.83:0/233630380 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:51:36.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:36.092+0000 7f4607400640 1 --2- 172.21.15.83:0/233630380 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f46081039a0 0x7f46081a1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:36.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:36.092+0000 7f4606000640 1 --2- 172.21.15.83:0/233630380 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4608102fe0 0x7f460819a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:36.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:36.092+0000 7f4606a00640 1 -- 172.21.15.83:0/233630380 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f46081039a0 msgr2=0x7f46081a1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:36.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:36.092+0000 7f4606a00640 1 --2- 172.21.15.83:0/233630380 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f46081039a0 0x7f46081a1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:36.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:36.092+0000 7f4606a00640 1 -- 172.21.15.83:0/233630380 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4608102fe0 msgr2=0x7f460819a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:36.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:36.092+0000 7f4606a00640 1 --2- 172.21.15.83:0/233630380 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4608102fe0 0x7f460819a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:36.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:36.092+0000 7f4606a00640 1 -- 172.21.15.83:0/233630380 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46081a1e80 con 0x7f4608101e90 2024-04-24T08:51:36.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:36.093+0000 7f4606a00640 1 --2- 172.21.15.83:0/233630380 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4608101e90 0x7f460819a260 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f45f800d950 tx=0x7f45f800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:36.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:36.093+0000 7f45ff400640 1 -- 172.21.15.83:0/233630380 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f8017070 con 0x7f4608101e90 2024-04-24T08:51:36.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:36.093+0000 7f45ff400640 1 -- 172.21.15.83:0/233630380 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f45f8009c30 con 0x7f4608101e90 2024-04-24T08:51:36.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:36.093+0000 7f4607e00640 1 -- 172.21.15.83:0/233630380 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46081a2170 con 0x7f4608101e90 2024-04-24T08:51:36.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:36.093+0000 7f45ff400640 1 -- 172.21.15.83:0/233630380 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f800f620 con 0x7f4608101e90 2024-04-24T08:51:36.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:36.093+0000 7f4607e00640 1 -- 172.21.15.83:0/233630380 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46081a26c0 con 0x7f4608101e90 2024-04-24T08:51:36.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:36.094+0000 7f4607e00640 1 -- 172.21.15.83:0/233630380 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46081a2920 con 0x7f4608101e90 2024-04-24T08:51:36.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:36.098+0000 7f45ff400640 1 -- 172.21.15.83:0/233630380 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 20) ==== 138663+0+0 (secure 0 0 0) 0x7f45f80109a0 con 0x7f4608101e90 2024-04-24T08:51:36.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:36.100+0000 7f45ff400640 1 --2- 172.21.15.83:0/233630380 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f45ec0a3bf0 0x7f45ec0a60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:36.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:36.100+0000 7f4606000640 1 -- 172.21.15.83:0/233630380 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f45ec0a3bf0 msgr2=0x7f45ec0a60b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.83:6800/2550826560 2024-04-24T08:51:36.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:36.100+0000 7f45ff400640 1 -- 172.21.15.83:0/233630380 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9237+0+0 (secure 0 0 0) 0x7f45f80d03e0 con 0x7f4608101e90 2024-04-24T08:51:36.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:36.100+0000 7f4606000640 1 --2- 172.21.15.83:0/233630380 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f45ec0a3bf0 0x7f45ec0a60b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T08:51:36.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:36.105+0000 7f45ff400640 1 -- 172.21.15.83:0/233630380 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f45f8009de0 con 0x7f4608101e90 2024-04-24T08:51:36.205 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Apr 24 08:51:35 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-y[32810]: 2024-04-24T08:51:35.836+0000 7f492924e200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-24T08:51:36.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:35 smithi138 ceph-mon[31441]: Standby manager daemon y restarted 2024-04-24T08:51:36.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:35 smithi138 ceph-mon[31441]: Standby manager daemon y started 2024-04-24T08:51:36.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:35 smithi138 ceph-mon[31441]: from='mgr.? 172.21.15.138:0/449961047' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-24T08:51:36.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:35 smithi138 ceph-mon[31441]: from='mgr.? 172.21.15.138:0/449961047' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T08:51:36.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:35 smithi138 ceph-mon[31441]: from='mgr.? 172.21.15.138:0/449961047' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-24T08:51:36.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:35 smithi138 ceph-mon[31441]: from='mgr.? 172.21.15.138:0/449961047' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T08:51:36.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:36.301+0000 7f4606000640 1 -- 172.21.15.83:0/233630380 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f45ec0a3bf0 msgr2=0x7f45ec0a60b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.83:6800/2550826560 2024-04-24T08:51:36.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:36.301+0000 7f4606000640 1 --2- 172.21.15.83:0/233630380 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f45ec0a3bf0 0x7f45ec0a60b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-24T08:51:36.302 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:51:36 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:51:36.045+0000 7f3b97266200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-24T08:51:36.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:35 smithi083 ceph-mon[27907]: Standby manager daemon y restarted 2024-04-24T08:51:36.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:35 smithi083 ceph-mon[27907]: Standby manager daemon y started 2024-04-24T08:51:36.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:35 smithi083 ceph-mon[27907]: from='mgr.? 172.21.15.138:0/449961047' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-24T08:51:36.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:35 smithi083 ceph-mon[27907]: from='mgr.? 172.21.15.138:0/449961047' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T08:51:36.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:35 smithi083 ceph-mon[27907]: from='mgr.? 172.21.15.138:0/449961047' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-24T08:51:36.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:35 smithi083 ceph-mon[27907]: from='mgr.? 172.21.15.138:0/449961047' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T08:51:36.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:36.353+0000 7f4607e00640 1 -- 172.21.15.83:0/233630380 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7f46081a2c70 con 0x7f4608101e90 2024-04-24T08:51:36.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:36.366+0000 7f45ff400640 1 -- 172.21.15.83:0/233630380 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mgrmap(e 21) ==== 138331+0+0 (secure 0 0 0) 0x7f45f80989e0 con 0x7f4608101e90 2024-04-24T08:51:36.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:36.367+0000 7f45ff400640 1 -- 172.21.15.83:0/233630380 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f45ec0a3bf0 msgr2=0x7f45ec0a60b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T08:51:36.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:36.367+0000 7f45ff400640 1 --2- 172.21.15.83:0/233630380 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f45ec0a3bf0 0x7f45ec0a60b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:36.457 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 24 08:51:36 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-z[33677]: 2024-04-24T08:51:36.015+0000 7fef15b51200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-24T08:51:36.713 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:51:36 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:51:36.344+0000 7f3b97266200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-24T08:51:36.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:36.936+0000 7f45ff400640 1 -- 172.21.15.83:0/233630380 <== mon.0 v2:172.21.15.83:3300/0 8 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v28) ==== 117+0+0 (secure 0 0 0) 0x7f45f809ca70 con 0x7f4608101e90 2024-04-24T08:51:36.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:36.939+0000 7f4607e00640 1 -- 172.21.15.83:0/233630380 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4608101e90 msgr2=0x7f460819a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:36.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:36.939+0000 7f4607e00640 1 --2- 172.21.15.83:0/233630380 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4608101e90 0x7f460819a260 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f45f800d950 tx=0x7f45f800de10 comp rx=0 tx=0).stop 2024-04-24T08:51:36.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:36.939+0000 7f4607e00640 1 -- 172.21.15.83:0/233630380 shutdown_connections 2024-04-24T08:51:36.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:36.939+0000 7f4607e00640 1 --2- 172.21.15.83:0/233630380 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f46081039a0 0x7f46081a1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:36.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:36.939+0000 7f4607e00640 1 --2- 172.21.15.83:0/233630380 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4608102fe0 0x7f460819a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:36.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:36.939+0000 7f4607e00640 1 --2- 172.21.15.83:0/233630380 >> [v2:172.21.15.83:6800/2550826560,v1:172.21.15.83:6801/2550826560] conn(0x7f45ec0a3bf0 0x7f45ec0a60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:36.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:36.939+0000 7f4607e00640 1 --2- 172.21.15.83:0/233630380 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4608101e90 0x7f460819a260 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:36.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:36.939+0000 7f4607e00640 1 -- 172.21.15.83:0/233630380 >> 172.21.15.83:0/233630380 conn(0x7f46080fd400 msgr2=0x7f4608074230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:36.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:36.939+0000 7f4607e00640 1 -- 172.21.15.83:0/233630380 shutdown_connections 2024-04-24T08:51:36.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:36.939+0000 7f4607e00640 1 -- 172.21.15.83:0/233630380 wait complete. 2024-04-24T08:51:36.957 INFO:teuthology.run_tasks:Running task check-counter... 2024-04-24T08:51:36.968 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-04-24T08:51:37.010 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T08:51:37.053 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:51:36 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:51:36.882+0000 7f3aed600700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T08:51:37.053 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:51:36 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:51:36.882+0000 7f3aed600700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T08:51:37.053 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:51:36 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:51:36.882+0000 7f3aed600700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T08:51:37.053 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:51:36 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:51:36.882+0000 7f3aed600700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T08:51:37.053 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:51:36 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:51:36.882+0000 7f3aed600700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T08:51:37.053 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:51:36 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:51:36.883+0000 7f3ae7c00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T08:51:37.053 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:51:36 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:51:36.883+0000 7f3ae7c00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T08:51:37.053 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:51:36 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:51:36.883+0000 7f3ae7c00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T08:51:37.053 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:51:36 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:51:36.883+0000 7f3ae7c00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T08:51:37.054 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:51:36 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:51:36.883+0000 7f3ae7c00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T08:51:37.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:36 smithi083 ceph-mon[27907]: mgrmap e20: x(active, since 12m), standbys: z, y 2024-04-24T08:51:37.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:36 smithi083 ceph-mon[27907]: Standby manager daemon z restarted 2024-04-24T08:51:37.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:36 smithi083 ceph-mon[27907]: Standby manager daemon z started 2024-04-24T08:51:37.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:36 smithi083 ceph-mon[27907]: from='mgr.? 172.21.15.189:0/3053766956' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-24T08:51:37.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:36 smithi083 ceph-mon[27907]: from='mgr.? 172.21.15.189:0/3053766956' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T08:51:37.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:36 smithi083 ceph-mon[27907]: from='mgr.? 172.21.15.189:0/3053766956' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-24T08:51:37.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:36 smithi083 ceph-mon[27907]: from='mgr.? 172.21.15.189:0/3053766956' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T08:51:37.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:36 smithi083 ceph-mon[27907]: Active manager daemon x restarted 2024-04-24T08:51:37.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:36 smithi083 ceph-mon[27907]: Activating manager daemon x 2024-04-24T08:51:37.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:36 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/233630380' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-04-24T08:51:37.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:36 smithi083 ceph-mon[27907]: osdmap e94: 12 total, 12 up, 12 in 2024-04-24T08:51:37.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:36 smithi083 ceph-mon[27907]: mgrmap e21: x(active, starting, since 0.0222939s), standbys: z, y 2024-04-24T08:51:37.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:36 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T08:51:37.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:36 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T08:51:37.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:36 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T08:51:37.056 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:36 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-24T08:51:37.056 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:36 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-24T08:51:37.056 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:36 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-24T08:51:37.056 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:36 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-24T08:51:37.056 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:36 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-24T08:51:37.056 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:36 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-24T08:51:37.056 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:36 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-24T08:51:37.056 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:36 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-24T08:51:37.056 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:36 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-24T08:51:37.056 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:36 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-24T08:51:37.056 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:36 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-24T08:51:37.056 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:36 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-24T08:51:37.056 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:36 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-24T08:51:37.057 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:36 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-24T08:51:37.057 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:36 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-24T08:51:37.057 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:36 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T08:51:37.057 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:36 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T08:51:37.057 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:36 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T08:51:37.057 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:36 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T08:51:37.057 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:36 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T08:51:37.057 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:36 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T08:51:37.057 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:36 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T08:51:37.057 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:36 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T08:51:37.057 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:36 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T08:51:37.057 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:36 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T08:51:37.057 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:36 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T08:51:37.057 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:36 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T08:51:37.058 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:36 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-24T08:51:37.058 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:36 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-24T08:51:37.058 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:36 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-24T08:51:37.058 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:36 smithi083 ceph-mon[27907]: Manager daemon x is now available 2024-04-24T08:51:37.058 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:36 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:51:37.058 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:36 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-24T08:51:37.058 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:36 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-24T08:51:37.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:37.179+0000 7f81abe00640 1 -- 172.21.15.83:0/630927787 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81ac109d60 msgr2=0x7f81ac10c1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:37.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:37.179+0000 7f81abe00640 1 --2- 172.21.15.83:0/630927787 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81ac109d60 0x7f81ac10c1f0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f81a4009ca0 tx=0x7f81a402a8c0 comp rx=0 tx=0).stop 2024-04-24T08:51:37.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:37.179+0000 7f81abe00640 1 -- 172.21.15.83:0/630927787 shutdown_connections 2024-04-24T08:51:37.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:37.179+0000 7f81abe00640 1 --2- 172.21.15.83:0/630927787 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81ac109d60 0x7f81ac10c1f0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:37.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:37.179+0000 7f81abe00640 1 --2- 172.21.15.83:0/630927787 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f81ac074a60 0x7f81ac074ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:37.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:37.179+0000 7f81abe00640 1 --2- 172.21.15.83:0/630927787 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81ac076040 0x7f81ac074490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:37.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:37.179+0000 7f81abe00640 1 -- 172.21.15.83:0/630927787 >> 172.21.15.83:0/630927787 conn(0x7f81ac0fd410 msgr2=0x7f81ac0ff850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:37.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:37.180+0000 7f81abe00640 1 -- 172.21.15.83:0/630927787 shutdown_connections 2024-04-24T08:51:37.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:37.180+0000 7f81abe00640 1 -- 172.21.15.83:0/630927787 wait complete. 2024-04-24T08:51:37.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:37.180+0000 7f81abe00640 1 Processor -- start 2024-04-24T08:51:37.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:37.181+0000 7f81abe00640 1 -- start start 2024-04-24T08:51:37.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:37.181+0000 7f81abe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f81ac074a60 0x7f81ac19a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:37.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:37.181+0000 7f81abe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81ac076040 0x7f81ac19a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:37.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:37.181+0000 7f81abe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81ac109d60 0x7f81ac1a17e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:37.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:37.181+0000 7f81abe00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f81ac10fc20 con 0x7f81ac109d60 2024-04-24T08:51:37.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:37.181+0000 7f81abe00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f81ac10faa0 con 0x7f81ac076040 2024-04-24T08:51:37.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:37.181+0000 7f81abe00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f81ac10fda0 con 0x7f81ac074a60 2024-04-24T08:51:37.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:37.181+0000 7f81ab400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81ac109d60 0x7f81ac1a17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:37.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:37.181+0000 7f81aaa00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f81ac074a60 0x7f81ac19a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:37.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:37.181+0000 7f81aaa00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f81ac074a60 0x7f81ac19a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:39816/0 (socket says 172.21.15.83:39816) 2024-04-24T08:51:37.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:37.181+0000 7f81aaa00640 1 -- 172.21.15.83:0/2099136067 learned_addr learned my addr 172.21.15.83:0/2099136067 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:51:37.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:37.181+0000 7f81aa000640 1 --2- 172.21.15.83:0/2099136067 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81ac076040 0x7f81ac19a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:37.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:37.182+0000 7f81aaa00640 1 -- 172.21.15.83:0/2099136067 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81ac076040 msgr2=0x7f81ac19a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:37.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:37.182+0000 7f81aaa00640 1 --2- 172.21.15.83:0/2099136067 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81ac076040 0x7f81ac19a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:37.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:37.182+0000 7f81aaa00640 1 -- 172.21.15.83:0/2099136067 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81ac109d60 msgr2=0x7f81ac1a17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:37.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:37.182+0000 7f81aaa00640 1 --2- 172.21.15.83:0/2099136067 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81ac109d60 0x7f81ac1a17e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:37.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:37.182+0000 7f81aaa00640 1 -- 172.21.15.83:0/2099136067 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81ac1a1e50 con 0x7f81ac074a60 2024-04-24T08:51:37.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:37.182+0000 7f81ab400640 1 --2- 172.21.15.83:0/2099136067 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81ac109d60 0x7f81ac1a17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T08:51:37.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:37.182+0000 7f81aaa00640 1 --2- 172.21.15.83:0/2099136067 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f81ac074a60 0x7f81ac19a280 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f819c00d950 tx=0x7f819c00de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:37.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:37.182+0000 7f81a3400640 1 -- 172.21.15.83:0/2099136067 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f819c017070 con 0x7f81ac074a60 2024-04-24T08:51:37.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:37.182+0000 7f81abe00640 1 -- 172.21.15.83:0/2099136067 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81ac1a2140 con 0x7f81ac074a60 2024-04-24T08:51:37.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:37.183+0000 7f81a3400640 1 -- 172.21.15.83:0/2099136067 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f819c009c30 con 0x7f81ac074a60 2024-04-24T08:51:37.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:37.183+0000 7f81abe00640 1 -- 172.21.15.83:0/2099136067 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81ac1a26e0 con 0x7f81ac074a60 2024-04-24T08:51:37.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:37.183+0000 7f81a3400640 1 -- 172.21.15.83:0/2099136067 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f819c020060 con 0x7f81ac074a60 2024-04-24T08:51:37.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:37.184+0000 7f81abe00640 1 -- 172.21.15.83:0/2099136067 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81ac108a90 con 0x7f81ac074a60 2024-04-24T08:51:37.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:37.186+0000 7f81a3400640 1 -- 172.21.15.83:0/2099136067 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 21) ==== 138331+0+0 (secure 0 0 0) 0x7f819c0109a0 con 0x7f81ac074a60 2024-04-24T08:51:37.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:37.189+0000 7f81a3400640 1 -- 172.21.15.83:0/2099136067 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f819c0cfbb0 con 0x7f81ac074a60 2024-04-24T08:51:37.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:37.194+0000 7f81a3400640 1 -- 172.21.15.83:0/2099136067 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f819c009de0 con 0x7f81ac074a60 2024-04-24T08:51:37.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:36 smithi138 ceph-mon[31441]: mgrmap e20: x(active, since 12m), standbys: z, y 2024-04-24T08:51:37.207 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:36 smithi138 ceph-mon[31441]: Standby manager daemon z restarted 2024-04-24T08:51:37.207 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:36 smithi138 ceph-mon[31441]: Standby manager daemon z started 2024-04-24T08:51:37.207 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:36 smithi138 ceph-mon[31441]: from='mgr.? 172.21.15.189:0/3053766956' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-24T08:51:37.207 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:36 smithi138 ceph-mon[31441]: from='mgr.? 172.21.15.189:0/3053766956' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T08:51:37.207 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:36 smithi138 ceph-mon[31441]: from='mgr.? 172.21.15.189:0/3053766956' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-24T08:51:37.207 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:36 smithi138 ceph-mon[31441]: from='mgr.? 172.21.15.189:0/3053766956' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T08:51:37.207 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:36 smithi138 ceph-mon[31441]: Active manager daemon x restarted 2024-04-24T08:51:37.207 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:36 smithi138 ceph-mon[31441]: Activating manager daemon x 2024-04-24T08:51:37.207 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:36 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/233630380' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-04-24T08:51:37.207 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:36 smithi138 ceph-mon[31441]: osdmap e94: 12 total, 12 up, 12 in 2024-04-24T08:51:37.207 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:36 smithi138 ceph-mon[31441]: mgrmap e21: x(active, starting, since 0.0222939s), standbys: z, y 2024-04-24T08:51:37.207 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:36 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T08:51:37.207 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:36 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T08:51:37.207 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:36 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T08:51:37.208 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:36 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-24T08:51:37.208 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:36 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-24T08:51:37.208 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:36 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-24T08:51:37.208 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:36 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-24T08:51:37.208 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:36 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-24T08:51:37.208 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:36 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-24T08:51:37.208 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:36 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-24T08:51:37.208 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:36 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-24T08:51:37.208 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:36 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-24T08:51:37.208 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:36 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-24T08:51:37.208 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:36 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-24T08:51:37.208 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:36 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-24T08:51:37.209 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:36 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-24T08:51:37.209 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:36 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-24T08:51:37.209 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:36 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-24T08:51:37.209 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:36 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T08:51:37.210 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:36 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T08:51:37.210 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:36 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T08:51:37.210 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:36 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T08:51:37.210 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:36 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T08:51:37.210 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:36 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T08:51:37.210 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:36 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T08:51:37.210 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:36 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T08:51:37.210 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:36 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T08:51:37.210 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:36 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T08:51:37.210 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:36 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T08:51:37.210 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:36 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T08:51:37.210 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:36 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-24T08:51:37.210 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:36 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-24T08:51:37.211 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:36 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-24T08:51:37.211 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:36 smithi138 ceph-mon[31441]: Manager daemon x is now available 2024-04-24T08:51:37.211 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:36 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:51:37.211 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:36 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-24T08:51:37.211 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:36 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-24T08:51:37.211 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:36 smithi189 ceph-mon[31791]: mgrmap e20: x(active, since 12m), standbys: z, y 2024-04-24T08:51:37.211 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:36 smithi189 ceph-mon[31791]: Standby manager daemon z restarted 2024-04-24T08:51:37.211 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:36 smithi189 ceph-mon[31791]: Standby manager daemon z started 2024-04-24T08:51:37.211 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:36 smithi189 ceph-mon[31791]: from='mgr.? 172.21.15.189:0/3053766956' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-24T08:51:37.211 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:36 smithi189 ceph-mon[31791]: from='mgr.? 172.21.15.189:0/3053766956' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T08:51:37.211 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:36 smithi189 ceph-mon[31791]: from='mgr.? 172.21.15.189:0/3053766956' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-24T08:51:37.211 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:36 smithi189 ceph-mon[31791]: from='mgr.? 172.21.15.189:0/3053766956' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T08:51:37.212 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:36 smithi189 ceph-mon[31791]: Active manager daemon x restarted 2024-04-24T08:51:37.212 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:36 smithi189 ceph-mon[31791]: Activating manager daemon x 2024-04-24T08:51:37.212 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:36 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/233630380' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-04-24T08:51:37.212 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:36 smithi189 ceph-mon[31791]: osdmap e94: 12 total, 12 up, 12 in 2024-04-24T08:51:37.212 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:36 smithi189 ceph-mon[31791]: mgrmap e21: x(active, starting, since 0.0222939s), standbys: z, y 2024-04-24T08:51:37.212 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:36 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T08:51:37.212 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:36 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T08:51:37.212 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:36 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T08:51:37.212 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:36 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-24T08:51:37.212 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:36 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-24T08:51:37.212 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:36 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-24T08:51:37.212 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:36 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-24T08:51:37.212 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:36 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-24T08:51:37.212 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:36 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-24T08:51:37.213 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:36 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-24T08:51:37.213 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:36 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-24T08:51:37.213 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:36 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-24T08:51:37.213 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:36 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-24T08:51:37.213 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:36 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-24T08:51:37.213 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:36 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-24T08:51:37.213 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:36 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-24T08:51:37.213 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:36 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-24T08:51:37.213 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:36 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-24T08:51:37.213 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:36 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T08:51:37.213 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:36 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T08:51:37.213 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:36 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T08:51:37.213 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:36 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T08:51:37.213 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:36 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T08:51:37.213 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:36 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T08:51:37.213 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:36 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T08:51:37.214 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:36 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T08:51:37.214 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:36 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T08:51:37.214 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:36 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T08:51:37.214 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:36 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T08:51:37.214 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:36 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T08:51:37.214 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:36 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-24T08:51:37.214 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:36 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-24T08:51:37.214 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:36 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-24T08:51:37.214 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:36 smithi189 ceph-mon[31791]: Manager daemon x is now available 2024-04-24T08:51:37.214 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:36 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:51:37.214 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:36 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-24T08:51:37.214 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:36 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-24T08:51:37.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:37.392+0000 7f81a3400640 1 -- 172.21.15.83:0/2099136067 <== mon.1 v2:172.21.15.189:3300/0 7 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7f819c097b10 con 0x7f81ac074a60 2024-04-24T08:51:37.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:37.392+0000 7f81a3400640 1 --2- 172.21.15.83:0/2099136067 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f81900ac290 0x7f81900ae680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:37.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:37.396+0000 7f81aa000640 1 --2- 172.21.15.83:0/2099136067 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f81900ac290 0x7f81900ae680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:37.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:37.399+0000 7f81aa000640 1 --2- 172.21.15.83:0/2099136067 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f81900ac290 0x7f81900ae680 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f81ac19b740 tx=0x7f8198005f90 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:37.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:37.413+0000 7f81abe00640 1 -- 172.21.15.83:0/2099136067 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f81ac062c00 con 0x7f81ac074a60 2024-04-24T08:51:37.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:37.418+0000 7f81a3400640 1 -- 172.21.15.83:0/2099136067 <== mon.1 v2:172.21.15.189:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+25265 (secure 0 0 0) 0x7f819c09c4c0 con 0x7f81ac074a60 2024-04-24T08:51:37.419 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:51:37.419 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":94,"fsid":"bd3e039e-0215-11ef-bc93-c7b262605968","created":"2024-04-24T08:37:32.209944+0000","modified":"2024-04-24T08:51:36.347834+0000","last_up_change":"2024-04-24T08:47:17.966041+0000","last_in_change":"2024-04-24T08:46:53.835936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T08:41:56.834372+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T08:48:26.978311+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T08:48:28.352731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T08:48:32.275167+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2cd77f3a-19dd-4c6e-a199-1dcbd5faa679","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2132675555},{"type":"v1","addr":"172.21.15.83:6803","nonce":2132675555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2132675555},{"type":"v1","addr":"172.21.15.83:6805","nonce":2132675555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2132675555},{"type":"v1","addr":"172.21.15.83:6809","nonce":2132675555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2132675555},{"type":"v1","addr":"172.21.15.83:6807","nonce":2132675555}]},"public_addr":"172.21.15.83:6803/2132675555","cluster_addr":"172.21.15.83:6805/2132675555","heartbeat_back_addr":"172.21.15.83:6809/2132675555","heartbeat_front_addr":"172.21.15.83:6807/2132675555","state":["exists","up"]},{"osd":1,"uuid":"aba7f5df-2a4c-4c23-b502-d193605a5b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":3589158880},{"type":"v1","addr":"172.21.15.138:6801","nonce":3589158880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":3589158880},{"type":"v1","addr":"172.21.15.138:6803","nonce":3589158880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":3589158880},{"type":"v1","addr":"172.21.15.138:6807","nonce":3589158880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":3589158880},{"type":"v1","addr":"172.21.15.138:6805","nonce":3589158880}]},"public_addr":"172.21.15.138:6801/3589158880","cluster_addr":"172.21.15.138:6803/3589158880","heartbeat_back_addr":"172.21.15.138:6807/3589158880","heartbeat_front_addr":"172.21.15.138:6805/3589158880","state":["exists","up"]},{"osd":2,"uuid":"071bc167-2333-4420-bb58-bffbc10d7641","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":3169072688},{"type":"v1","addr":"172.21.15.189:6801","nonce":3169072688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":3169072688},{"type":"v1","addr":"172.21.15.189:6803","nonce":3169072688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":3169072688},{"type":"v1","addr":"172.21.15.189:6807","nonce":3169072688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":3169072688},{"type":"v1","addr":"172.21.15.189:6805","nonce":3169072688}]},"public_addr":"172.21.15.189:6801/3169072688","cluster_addr":"172.21.15.189:6803/3169072688","heartbeat_back_addr":"172.21.15.189:6807/3169072688","heartbeat_front_addr":"172.21.15.189:6805/3169072688","state":["exists","up"]},{"osd":3,"uuid":"f5624d78-4455-4e07-8c1a-f47c94cddb23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3131092450},{"type":"v1","addr":"172.21.15.83:6811","nonce":3131092450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3131092450},{"type":"v1","addr":"172.21.15.83:6813","nonce":3131092450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3131092450},{"type":"v1","addr":"172.21.15.83:6817","nonce":3131092450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3131092450},{"type":"v1","addr":"172.21.15.83:6815","nonce":3131092450}]},"public_addr":"172.21.15.83:6811/3131092450","cluster_addr":"172.21.15.83:6813/3131092450","heartbeat_back_addr":"172.21.15.83:6817/3131092450","heartbeat_front_addr":"172.21.15.83:6815/3131092450","state":["exists","up"]},{"osd":4,"uuid":"7b50cd14-9385-4535-a7b0-1c1da93c8548","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":823775360},{"type":"v1","addr":"172.21.15.138:6809","nonce":823775360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":823775360},{"type":"v1","addr":"172.21.15.138:6811","nonce":823775360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":823775360},{"type":"v1","addr":"172.21.15.138:6815","nonce":823775360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":823775360},{"type":"v1","addr":"172.21.15.138:6813","nonce":823775360}]},"public_addr":"172.21.15.138:6809/823775360","cluster_addr":"172.21.15.138:6811/823775360","heartbeat_back_addr":"172.21.15.138:6815/823775360","heartbeat_front_addr":"172.21.15.138:6813/823775360","state":["exists","up"]},{"osd":5,"uuid":"9c647797-35bc-449f-be84-af97d2c69c2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":1399817431},{"type":"v1","addr":"172.21.15.189:6809","nonce":1399817431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":1399817431},{"type":"v1","addr":"172.21.15.189:6811","nonce":1399817431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":1399817431},{"type":"v1","addr":"172.21.15.189:6815","nonce":1399817431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":1399817431},{"type":"v1","addr":"172.21.15.189:6813","nonce":1399817431}]},"public_addr":"172.21.15.189:6809/1399817431","cluster_addr":"172.21.15.189:6811/1399817431","heartbeat_back_addr":"172.21.15.189:6815/1399817431","heartbeat_front_addr":"172.21.15.189:6813/1399817431","state":["exists","up"]},{"osd":6,"uuid":"8f056584-16c8-4454-b445-3f780f26c196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":1611358483},{"type":"v1","addr":"172.21.15.83:6819","nonce":1611358483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1611358483},{"type":"v1","addr":"172.21.15.83:6821","nonce":1611358483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1611358483},{"type":"v1","addr":"172.21.15.83:6825","nonce":1611358483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":1611358483},{"type":"v1","addr":"172.21.15.83:6823","nonce":1611358483}]},"public_addr":"172.21.15.83:6819/1611358483","cluster_addr":"172.21.15.83:6821/1611358483","heartbeat_back_addr":"172.21.15.83:6825/1611358483","heartbeat_front_addr":"172.21.15.83:6823/1611358483","state":["exists","up"]},{"osd":7,"uuid":"7603efe3-da77-4b7f-9d03-29c4f293b6d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":3557856706},{"type":"v1","addr":"172.21.15.138:6817","nonce":3557856706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":3557856706},{"type":"v1","addr":"172.21.15.138:6819","nonce":3557856706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":3557856706},{"type":"v1","addr":"172.21.15.138:6823","nonce":3557856706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":3557856706},{"type":"v1","addr":"172.21.15.138:6821","nonce":3557856706}]},"public_addr":"172.21.15.138:6817/3557856706","cluster_addr":"172.21.15.138:6819/3557856706","heartbeat_back_addr":"172.21.15.138:6823/3557856706","heartbeat_front_addr":"172.21.15.138:6821/3557856706","state":["exists","up"]},{"osd":8,"uuid":"27d7d1b8-d6a0-417e-b8aa-596325aa1a77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":207018554},{"type":"v1","addr":"172.21.15.189:6817","nonce":207018554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":207018554},{"type":"v1","addr":"172.21.15.189:6819","nonce":207018554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":207018554},{"type":"v1","addr":"172.21.15.189:6823","nonce":207018554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":207018554},{"type":"v1","addr":"172.21.15.189:6821","nonce":207018554}]},"public_addr":"172.21.15.189:6817/207018554","cluster_addr":"172.21.15.189:6819/207018554","heartbeat_back_addr":"172.21.15.189:6823/207018554","heartbeat_front_addr":"172.21.15.189:6821/207018554","state":["exists","up"]},{"osd":9,"uuid":"baf19b98-881b-4d3a-9f72-8aa2dd73cad6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":519636751},{"type":"v1","addr":"172.21.15.83:6827","nonce":519636751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":519636751},{"type":"v1","addr":"172.21.15.83:6829","nonce":519636751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":519636751},{"type":"v1","addr":"172.21.15.83:6833","nonce":519636751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":519636751},{"type":"v1","addr":"172.21.15.83:6831","nonce":519636751}]},"public_addr":"172.21.15.83:6827/519636751","cluster_addr":"172.21.15.83:6829/519636751","heartbeat_back_addr":"172.21.15.83:6833/519636751","heartbeat_front_addr":"172.21.15.83:6831/519636751","state":["exists","up"]},{"osd":10,"uuid":"eb346717-5c07-47c8-95be-0e68fe449fdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1130872098},{"type":"v1","addr":"172.21.15.138:6825","nonce":1130872098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1130872098},{"type":"v1","addr":"172.21.15.138:6827","nonce":1130872098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1130872098},{"type":"v1","addr":"172.21.15.138:6831","nonce":1130872098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1130872098},{"type":"v1","addr":"172.21.15.138:6829","nonce":1130872098}]},"public_addr":"172.21.15.138:6825/1130872098","cluster_addr":"172.21.15.138:6827/1130872098","heartbeat_back_addr":"172.21.15.138:6831/1130872098","heartbeat_front_addr":"172.21.15.138:6829/1130872098","state":["exists","up"]},{"osd":11,"uuid":"8e37568e-981c-45b8-b2af-2f1b73898542","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":2998782690},{"type":"v1","addr":"172.21.15.189:6825","nonce":2998782690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":2998782690},{"type":"v1","addr":"172.21.15.189:6827","nonce":2998782690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":2998782690},{"type":"v1","addr":"172.21.15.189:6831","nonce":2998782690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":2998782690},{"type":"v1","addr":"172.21.15.189:6829","nonce":2998782690}]},"public_addr":"172.21.15.189:6825/2998782690","cluster_addr":"172.21.15.189:6827/2998782690","heartbeat_back_addr":"172.21.15.189:6831/2998782690","heartbeat_front_addr":"172.21.15.189:6829/2998782690","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:40:47.980103+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:41:21.970795+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:41:55.160426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:42:30.400475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:43:05.705663+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:43:40.112312+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:44:16.781289+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:44:52.151187+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:45:28.055944+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:46:04.969301+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:46:40.918306+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:47:16.886874+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"2.1d","mappings":[{"from":2,"to":1}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.2a","mappings":[{"from":3,"to":8}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.a","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:0/1096290901":"2024-04-25T08:51:36.347763+0000","172.21.15.83:6800/2550826560":"2024-04-25T08:51:36.347763+0000","172.21.15.83:0/1743153945":"2024-04-25T08:51:36.347763+0000","172.21.15.83:0/3306681366":"2024-04-25T08:51:36.347763+0000","172.21.15.83:6801/820049883":"2024-04-25T08:38:40.661315+0000","172.21.15.83:6801/2550826560":"2024-04-25T08:51:36.347763+0000","172.21.15.83:0/4272874138":"2024-04-25T08:38:40.661315+0000","172.21.15.83:0/4212994335":"2024-04-25T08:38:40.661315+0000","172.21.15.83:6801/1996371091":"2024-04-25T08:38:05.800856+0000","172.21.15.83:0/1820663683":"2024-04-25T08:38:05.800856+0000","172.21.15.83:0/3802897262":"2024-04-25T08:38:05.800856+0000","172.21.15.83:6800/1996371091":"2024-04-25T08:38:05.800856+0000","172.21.15.83:6800/820049883":"2024-04-25T08:38:40.661315+0000","172.21.15.83:0/1366125049":"2024-04-25T08:38:05.800856+0000","172.21.15.83:0/4159820288":"2024-04-25T08:51:36.347763+0000","172.21.15.83:0/340452868":"2024-04-25T08:38:40.661315+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T08:51:37.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:37.420+0000 7f81abe00640 1 -- 172.21.15.83:0/2099136067 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f81900ac290 msgr2=0x7f81900ae680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:37.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:37.420+0000 7f81abe00640 1 --2- 172.21.15.83:0/2099136067 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f81900ac290 0x7f81900ae680 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f81ac19b740 tx=0x7f8198005f90 comp rx=0 tx=0).stop 2024-04-24T08:51:37.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:37.420+0000 7f81abe00640 1 -- 172.21.15.83:0/2099136067 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f81ac074a60 msgr2=0x7f81ac19a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:37.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:37.420+0000 7f81abe00640 1 --2- 172.21.15.83:0/2099136067 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f81ac074a60 0x7f81ac19a280 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f819c00d950 tx=0x7f819c00de10 comp rx=0 tx=0).stop 2024-04-24T08:51:37.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:37.420+0000 7f81abe00640 1 -- 172.21.15.83:0/2099136067 shutdown_connections 2024-04-24T08:51:37.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:37.420+0000 7f81abe00640 1 --2- 172.21.15.83:0/2099136067 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f81900ac290 0x7f81900ae680 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:37.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:37.420+0000 7f81abe00640 1 --2- 172.21.15.83:0/2099136067 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81ac109d60 0x7f81ac1a17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:37.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:37.420+0000 7f81abe00640 1 --2- 172.21.15.83:0/2099136067 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81ac076040 0x7f81ac19a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:37.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:37.420+0000 7f81abe00640 1 --2- 172.21.15.83:0/2099136067 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f81ac074a60 0x7f81ac19a280 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:37.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:37.420+0000 7f81abe00640 1 -- 172.21.15.83:0/2099136067 >> 172.21.15.83:0/2099136067 conn(0x7f81ac0fd410 msgr2=0x7f81ac105c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:37.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:37.421+0000 7f81abe00640 1 -- 172.21.15.83:0/2099136067 shutdown_connections 2024-04-24T08:51:37.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:37.421+0000 7f81abe00640 1 -- 172.21.15.83:0/2099136067 wait complete. 2024-04-24T08:51:37.438 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T08:41:56.834372+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T08:48:26.978311+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T08:48:28.352731+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-24T08:48:32.275167+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-24T08:51:37.438 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T08:51:37.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:37.654+0000 7f3bb2e00640 1 -- 172.21.15.83:0/1890116849 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3bac102990 msgr2=0x7f3bac104d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:37.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:37.654+0000 7f3bb2e00640 1 --2- 172.21.15.83:0/1890116849 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3bac102990 0x7f3bac104d80 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3ba0009860 tx=0x7f3ba002a340 comp rx=0 tx=0).stop 2024-04-24T08:51:37.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:37.654+0000 7f3bb2e00640 1 -- 172.21.15.83:0/1890116849 shutdown_connections 2024-04-24T08:51:37.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:37.654+0000 7f3bb2e00640 1 --2- 172.21.15.83:0/1890116849 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3bac107bf0 0x7f3bac10a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:37.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:37.654+0000 7f3bb2e00640 1 --2- 172.21.15.83:0/1890116849 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bac1052c0 0x7f3bac1076b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:37.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:37.654+0000 7f3bb2e00640 1 --2- 172.21.15.83:0/1890116849 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3bac102990 0x7f3bac104d80 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:37.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:37.654+0000 7f3bb2e00640 1 -- 172.21.15.83:0/1890116849 >> 172.21.15.83:0/1890116849 conn(0x7f3bac0fc3a0 msgr2=0x7f3bac0fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:37.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:37.654+0000 7f3bb2e00640 1 -- 172.21.15.83:0/1890116849 shutdown_connections 2024-04-24T08:51:37.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:37.654+0000 7f3bb2e00640 1 -- 172.21.15.83:0/1890116849 wait complete. 2024-04-24T08:51:37.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:37.655+0000 7f3bb2e00640 1 Processor -- start 2024-04-24T08:51:37.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:37.655+0000 7f3bb2e00640 1 -- start start 2024-04-24T08:51:37.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:37.656+0000 7f3bb2e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bac102990 0x7f3bac19a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:37.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:37.656+0000 7f3bb2e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3bac1052c0 0x7f3bac19a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:37.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:37.656+0000 7f3bb2e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3bac107bf0 0x7f3bac1a1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:37.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:37.656+0000 7f3bb2e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3bac10da30 con 0x7f3bac102990 2024-04-24T08:51:37.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:37.656+0000 7f3bb2e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3bac10d8b0 con 0x7f3bac1052c0 2024-04-24T08:51:37.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:37.656+0000 7f3bb2e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3bac10dbb0 con 0x7f3bac107bf0 2024-04-24T08:51:37.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:37.656+0000 7f3bb1a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bac102990 0x7f3bac19a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:37.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:37.656+0000 7f3bb1a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bac102990 0x7f3bac19a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50572/0 (socket says 172.21.15.83:50572) 2024-04-24T08:51:37.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:37.656+0000 7f3bb1a00640 1 -- 172.21.15.83:0/780067902 learned_addr learned my addr 172.21.15.83:0/780067902 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:51:37.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:37.656+0000 7f3bb1000640 1 --2- 172.21.15.83:0/780067902 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3bac1052c0 0x7f3bac19a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:37.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:37.656+0000 7f3bb2400640 1 --2- 172.21.15.83:0/780067902 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3bac107bf0 0x7f3bac1a1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:37.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:37.656+0000 7f3bb1a00640 1 -- 172.21.15.83:0/780067902 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3bac107bf0 msgr2=0x7f3bac1a1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:37.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:37.656+0000 7f3bb1a00640 1 --2- 172.21.15.83:0/780067902 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3bac107bf0 0x7f3bac1a1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:37.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:37.656+0000 7f3bb1a00640 1 -- 172.21.15.83:0/780067902 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3bac1052c0 msgr2=0x7f3bac19a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:37.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:37.656+0000 7f3bb1a00640 1 --2- 172.21.15.83:0/780067902 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3bac1052c0 0x7f3bac19a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:37.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:37.656+0000 7f3bb1a00640 1 -- 172.21.15.83:0/780067902 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bac1a1f70 con 0x7f3bac102990 2024-04-24T08:51:37.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:37.657+0000 7f3bb1a00640 1 --2- 172.21.15.83:0/780067902 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bac102990 0x7f3bac19a350 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f3ba00057c0 tx=0x7f3ba0030e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:37.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:37.657+0000 7f3baa000640 1 -- 172.21.15.83:0/780067902 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba0040030 con 0x7f3bac102990 2024-04-24T08:51:37.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:37.657+0000 7f3baa000640 1 -- 172.21.15.83:0/780067902 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3ba0038040 con 0x7f3bac102990 2024-04-24T08:51:37.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:37.657+0000 7f3bb2e00640 1 -- 172.21.15.83:0/780067902 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bac1a2200 con 0x7f3bac102990 2024-04-24T08:51:37.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:37.657+0000 7f3baa000640 1 -- 172.21.15.83:0/780067902 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba0037070 con 0x7f3bac102990 2024-04-24T08:51:37.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:37.657+0000 7f3bb2e00640 1 -- 172.21.15.83:0/780067902 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bac1a26f0 con 0x7f3bac102990 2024-04-24T08:51:37.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:37.658+0000 7f3bb2e00640 1 -- 172.21.15.83:0/780067902 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bac109740 con 0x7f3bac102990 2024-04-24T08:51:37.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:37.660+0000 7f3baa000640 1 -- 172.21.15.83:0/780067902 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7f3ba0043070 con 0x7f3bac102990 2024-04-24T08:51:37.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:37.662+0000 7f3baa000640 1 --2- 172.21.15.83:0/780067902 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f3b940a39b0 0x7f3b940a5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:37.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:37.662+0000 7f3baa000640 1 -- 172.21.15.83:0/780067902 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f3ba00efcf0 con 0x7f3bac102990 2024-04-24T08:51:37.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:37.662+0000 7f3bb1000640 1 --2- 172.21.15.83:0/780067902 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f3b940a39b0 0x7f3b940a5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:37.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:37.663+0000 7f3bb1000640 1 --2- 172.21.15.83:0/780067902 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f3b940a39b0 0x7f3b940a5e70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f3bac19b860 tx=0x7f3b98009260 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:37.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:37.667+0000 7f3baa000640 1 -- 172.21.15.83:0/780067902 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3ba004a060 con 0x7f3bac102990 2024-04-24T08:51:37.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:37.874+0000 7f3bb2e00640 1 -- 172.21.15.83:0/780067902 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3bac062c00 con 0x7f3bac102990 2024-04-24T08:51:37.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:37.875+0000 7f3baa000640 1 -- 172.21.15.83:0/780067902 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7f3ba00bc490 con 0x7f3bac102990 2024-04-24T08:51:37.876 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-24T08:51:37.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:37.878+0000 7f3bb2e00640 1 -- 172.21.15.83:0/780067902 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f3b940a39b0 msgr2=0x7f3b940a5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:37.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:37.878+0000 7f3bb2e00640 1 --2- 172.21.15.83:0/780067902 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f3b940a39b0 0x7f3b940a5e70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f3bac19b860 tx=0x7f3b98009260 comp rx=0 tx=0).stop 2024-04-24T08:51:37.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:37.878+0000 7f3bb2e00640 1 -- 172.21.15.83:0/780067902 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bac102990 msgr2=0x7f3bac19a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:37.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:37.878+0000 7f3bb2e00640 1 --2- 172.21.15.83:0/780067902 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bac102990 0x7f3bac19a350 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f3ba00057c0 tx=0x7f3ba0030e00 comp rx=0 tx=0).stop 2024-04-24T08:51:37.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:37.878+0000 7f3bb2e00640 1 -- 172.21.15.83:0/780067902 shutdown_connections 2024-04-24T08:51:37.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:37.878+0000 7f3bb2e00640 1 --2- 172.21.15.83:0/780067902 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f3b940a39b0 0x7f3b940a5e70 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:37.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:37.878+0000 7f3bb2e00640 1 --2- 172.21.15.83:0/780067902 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3bac107bf0 0x7f3bac1a1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:37.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:37.878+0000 7f3bb2e00640 1 --2- 172.21.15.83:0/780067902 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3bac1052c0 0x7f3bac19a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:37.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:37.878+0000 7f3bb2e00640 1 --2- 172.21.15.83:0/780067902 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bac102990 0x7f3bac19a350 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:37.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:37.878+0000 7f3bb2e00640 1 -- 172.21.15.83:0/780067902 >> 172.21.15.83:0/780067902 conn(0x7f3bac0fc3a0 msgr2=0x7f3bac1021e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:37.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:37.878+0000 7f3bb2e00640 1 -- 172.21.15.83:0/780067902 shutdown_connections 2024-04-24T08:51:37.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:37.878+0000 7f3bb2e00640 1 -- 172.21.15.83:0/780067902 wait complete. 2024-04-24T08:51:37.895 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T08:51:38.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.084+0000 7f3eef200640 1 -- 172.21.15.83:0/3486497529 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ef0104140 msgr2=0x7f3ef0104560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:38.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.084+0000 7f3eef200640 1 --2- 172.21.15.83:0/3486497529 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ef0104140 0x7f3ef0104560 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f3ee0009900 tx=0x7f3ee002a3f0 comp rx=0 tx=0).stop 2024-04-24T08:51:38.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.084+0000 7f3eef200640 1 -- 172.21.15.83:0/3486497529 shutdown_connections 2024-04-24T08:51:38.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.084+0000 7f3eef200640 1 --2- 172.21.15.83:0/3486497529 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3ef0105cf0 0x7f3ef010c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:38.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.084+0000 7f3eef200640 1 --2- 172.21.15.83:0/3486497529 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3ef0105330 0x7f3ef01057b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:38.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.084+0000 7f3eef200640 1 --2- 172.21.15.83:0/3486497529 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ef0104140 0x7f3ef0104560 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:38.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.084+0000 7f3eef200640 1 -- 172.21.15.83:0/3486497529 >> 172.21.15.83:0/3486497529 conn(0x7f3ef00ff6b0 msgr2=0x7f3ef0101af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:38.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.084+0000 7f3eef200640 1 -- 172.21.15.83:0/3486497529 shutdown_connections 2024-04-24T08:51:38.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.084+0000 7f3eef200640 1 -- 172.21.15.83:0/3486497529 wait complete. 2024-04-24T08:51:38.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.086+0000 7f3eef200640 1 Processor -- start 2024-04-24T08:51:38.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.086+0000 7f3eef200640 1 -- start start 2024-04-24T08:51:38.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.086+0000 7f3eef200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ef0104140 0x7f3ef019a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:38.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.086+0000 7f3eef200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3ef0105330 0x7f3ef019a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:38.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.086+0000 7f3eef200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3ef0105cf0 0x7f3ef01a1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:38.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.086+0000 7f3eef200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef010fcd0 con 0x7f3ef0104140 2024-04-24T08:51:38.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.086+0000 7f3eef200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef010fb50 con 0x7f3ef0105cf0 2024-04-24T08:51:38.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.086+0000 7f3eef200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef010fe50 con 0x7f3ef0105330 2024-04-24T08:51:38.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.086+0000 7f3eede00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ef0104140 0x7f3ef019a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:38.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.086+0000 7f3eede00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ef0104140 0x7f3ef019a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50586/0 (socket says 172.21.15.83:50586) 2024-04-24T08:51:38.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.086+0000 7f3eede00640 1 -- 172.21.15.83:0/903175057 learned_addr learned my addr 172.21.15.83:0/903175057 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:51:38.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.087+0000 7f3eede00640 1 -- 172.21.15.83:0/903175057 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3ef0105330 msgr2=0x7f3ef019a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:38.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.087+0000 7f3eede00640 1 --2- 172.21.15.83:0/903175057 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3ef0105330 0x7f3ef019a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:38.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.087+0000 7f3eede00640 1 -- 172.21.15.83:0/903175057 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3ef0105cf0 msgr2=0x7f3ef01a1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:38.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.087+0000 7f3eede00640 1 --2- 172.21.15.83:0/903175057 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3ef0105cf0 0x7f3ef01a1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:38.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.087+0000 7f3eede00640 1 -- 172.21.15.83:0/903175057 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ef01a1e70 con 0x7f3ef0104140 2024-04-24T08:51:38.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.087+0000 7f3eede00640 1 --2- 172.21.15.83:0/903175057 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ef0104140 0x7f3ef019a250 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f3ee00314e0 tx=0x7f3ee0003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:38.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.087+0000 7f3ee6a00640 1 -- 172.21.15.83:0/903175057 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee003f070 con 0x7f3ef0104140 2024-04-24T08:51:38.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.087+0000 7f3ee6a00640 1 -- 172.21.15.83:0/903175057 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3ee0031e40 con 0x7f3ef0104140 2024-04-24T08:51:38.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.087+0000 7f3eef200640 1 -- 172.21.15.83:0/903175057 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ef01a2100 con 0x7f3ef0104140 2024-04-24T08:51:38.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.087+0000 7f3ee6a00640 1 -- 172.21.15.83:0/903175057 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee0035030 con 0x7f3ef0104140 2024-04-24T08:51:38.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.087+0000 7f3eef200640 1 -- 172.21.15.83:0/903175057 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ef01a2640 con 0x7f3ef0104140 2024-04-24T08:51:38.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.089+0000 7f3eef200640 1 -- 172.21.15.83:0/903175057 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ef010b460 con 0x7f3ef0104140 2024-04-24T08:51:38.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.090+0000 7f3ee6a00640 1 -- 172.21.15.83:0/903175057 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7f3ee0042070 con 0x7f3ef0104140 2024-04-24T08:51:38.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.093+0000 7f3ee6a00640 1 --2- 172.21.15.83:0/903175057 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f3ed40a3a30 0x7f3ed40a5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:38.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.093+0000 7f3ee6a00640 1 -- 172.21.15.83:0/903175057 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f3ee00f0f20 con 0x7f3ef0104140 2024-04-24T08:51:38.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.093+0000 7f3eed400640 1 --2- 172.21.15.83:0/903175057 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f3ed40a3a30 0x7f3ed40a5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:38.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.093+0000 7f3eed400640 1 --2- 172.21.15.83:0/903175057 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f3ed40a3a30 0x7f3ed40a5ef0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3ef019b760 tx=0x7f3edc005c70 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:38.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.097+0000 7f3ee6a00640 1 -- 172.21.15.83:0/903175057 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3ee00bd6c0 con 0x7f3ef0104140 2024-04-24T08:51:38.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:37 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/233630380' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-04-24T08:51:38.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:37 smithi138 ceph-mon[31441]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T08:51:38.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:37 smithi138 ceph-mon[31441]: mgrmap e22: x(active, since 1.03472s), standbys: z, y 2024-04-24T08:51:38.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:37 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/2099136067' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T08:51:38.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:37 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/780067902' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T08:51:38.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:37 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/233630380' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-04-24T08:51:38.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:37 smithi189 ceph-mon[31791]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T08:51:38.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:37 smithi189 ceph-mon[31791]: mgrmap e22: x(active, since 1.03472s), standbys: z, y 2024-04-24T08:51:38.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:37 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/2099136067' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T08:51:38.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:37 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/780067902' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T08:51:38.237 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:37 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/233630380' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-04-24T08:51:38.237 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:38 smithi083 ceph-mon[27907]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T08:51:38.238 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:38 smithi083 ceph-mon[27907]: mgrmap e22: x(active, since 1.03472s), standbys: z, y 2024-04-24T08:51:38.238 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:38 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/2099136067' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T08:51:38.238 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:38 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/780067902' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T08:51:38.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.299+0000 7f3eef200640 1 -- 172.21.15.83:0/903175057 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f3ef0062c00 con 0x7f3ef0104140 2024-04-24T08:51:38.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.300+0000 7f3ee6a00640 1 -- 172.21.15.83:0/903175057 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7f3ee00bcfc0 con 0x7f3ef0104140 2024-04-24T08:51:38.301 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-24T08:51:38.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.303+0000 7f3eef200640 1 -- 172.21.15.83:0/903175057 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f3ed40a3a30 msgr2=0x7f3ed40a5ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:38.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.303+0000 7f3eef200640 1 --2- 172.21.15.83:0/903175057 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f3ed40a3a30 0x7f3ed40a5ef0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3ef019b760 tx=0x7f3edc005c70 comp rx=0 tx=0).stop 2024-04-24T08:51:38.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.303+0000 7f3eef200640 1 -- 172.21.15.83:0/903175057 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ef0104140 msgr2=0x7f3ef019a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:38.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.303+0000 7f3eef200640 1 --2- 172.21.15.83:0/903175057 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ef0104140 0x7f3ef019a250 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f3ee00314e0 tx=0x7f3ee0003d50 comp rx=0 tx=0).stop 2024-04-24T08:51:38.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.303+0000 7f3eef200640 1 -- 172.21.15.83:0/903175057 shutdown_connections 2024-04-24T08:51:38.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.303+0000 7f3eef200640 1 --2- 172.21.15.83:0/903175057 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f3ed40a3a30 0x7f3ed40a5ef0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:38.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.303+0000 7f3eef200640 1 --2- 172.21.15.83:0/903175057 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3ef0105cf0 0x7f3ef01a1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:38.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.303+0000 7f3eef200640 1 --2- 172.21.15.83:0/903175057 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3ef0105330 0x7f3ef019a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:38.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.303+0000 7f3eef200640 1 --2- 172.21.15.83:0/903175057 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ef0104140 0x7f3ef019a250 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:38.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.303+0000 7f3eef200640 1 -- 172.21.15.83:0/903175057 >> 172.21.15.83:0/903175057 conn(0x7f3ef00ff6b0 msgr2=0x7f3ef0108630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:38.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.303+0000 7f3eef200640 1 -- 172.21.15.83:0/903175057 shutdown_connections 2024-04-24T08:51:38.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.303+0000 7f3eef200640 1 -- 172.21.15.83:0/903175057 wait complete. 2024-04-24T08:51:38.321 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T08:51:38.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.508+0000 7f2609e00640 1 -- 172.21.15.83:0/2759940194 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2604103a40 msgr2=0x7f260410a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:38.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.508+0000 7f2609e00640 1 --2- 172.21.15.83:0/2759940194 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2604103a40 0x7f260410a360 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f25fc009ca0 tx=0x7f25fc02a8c0 comp rx=0 tx=0).stop 2024-04-24T08:51:38.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.508+0000 7f2609e00640 1 -- 172.21.15.83:0/2759940194 shutdown_connections 2024-04-24T08:51:38.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.508+0000 7f2609e00640 1 --2- 172.21.15.83:0/2759940194 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2604103a40 0x7f260410a360 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:38.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.508+0000 7f2609e00640 1 --2- 172.21.15.83:0/2759940194 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2604103080 0x7f2604103500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:38.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.508+0000 7f2609e00640 1 --2- 172.21.15.83:0/2759940194 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2604101f30 0x7f2604102350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:38.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.508+0000 7f2609e00640 1 -- 172.21.15.83:0/2759940194 >> 172.21.15.83:0/2759940194 conn(0x7f26040fd4a0 msgr2=0x7f26040ff8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:38.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.508+0000 7f2609e00640 1 -- 172.21.15.83:0/2759940194 shutdown_connections 2024-04-24T08:51:38.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.508+0000 7f2609e00640 1 -- 172.21.15.83:0/2759940194 wait complete. 2024-04-24T08:51:38.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.509+0000 7f2609e00640 1 Processor -- start 2024-04-24T08:51:38.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.509+0000 7f2609e00640 1 -- start start 2024-04-24T08:51:38.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.510+0000 7f2609e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2604101f30 0x7f2604198130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:38.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.510+0000 7f2609e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2604103080 0x7f2604198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:38.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.510+0000 7f2609e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2604103a40 0x7f260419f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:38.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.510+0000 7f2609e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f260410da90 con 0x7f2604103a40 2024-04-24T08:51:38.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.510+0000 7f2609e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f260410d910 con 0x7f2604101f30 2024-04-24T08:51:38.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.510+0000 7f2609e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f260410dc10 con 0x7f2604103080 2024-04-24T08:51:38.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.510+0000 7f2609400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2604103a40 0x7f260419f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:38.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.510+0000 7f2609400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2604103a40 0x7f260419f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50610/0 (socket says 172.21.15.83:50610) 2024-04-24T08:51:38.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.510+0000 7f2609400640 1 -- 172.21.15.83:0/1827112841 learned_addr learned my addr 172.21.15.83:0/1827112841 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:51:38.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.510+0000 7f2608a00640 1 --2- 172.21.15.83:0/1827112841 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2604101f30 0x7f2604198130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:38.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.510+0000 7f2603e00640 1 --2- 172.21.15.83:0/1827112841 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2604103080 0x7f2604198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:38.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.510+0000 7f2609400640 1 -- 172.21.15.83:0/1827112841 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2604103080 msgr2=0x7f2604198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:38.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.510+0000 7f2609400640 1 --2- 172.21.15.83:0/1827112841 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2604103080 0x7f2604198670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:38.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.510+0000 7f2609400640 1 -- 172.21.15.83:0/1827112841 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2604101f30 msgr2=0x7f2604198130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:38.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.510+0000 7f2609400640 1 --2- 172.21.15.83:0/1827112841 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2604101f30 0x7f2604198130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:38.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.510+0000 7f2609400640 1 -- 172.21.15.83:0/1827112841 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f260419fd50 con 0x7f2604103a40 2024-04-24T08:51:38.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.510+0000 7f2608a00640 1 --2- 172.21.15.83:0/1827112841 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2604101f30 0x7f2604198130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:51:38.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.511+0000 7f2603e00640 1 --2- 172.21.15.83:0/1827112841 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2604103080 0x7f2604198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:51:38.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.511+0000 7f2609400640 1 --2- 172.21.15.83:0/1827112841 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2604103a40 0x7f260419f6e0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f25fc009dc0 tx=0x7f25fc003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:38.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.511+0000 7f2601600640 1 -- 172.21.15.83:0/1827112841 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25fc032ec0 con 0x7f2604103a40 2024-04-24T08:51:38.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.511+0000 7f2601600640 1 -- 172.21.15.83:0/1827112841 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f25fc0044f0 con 0x7f2604103a40 2024-04-24T08:51:38.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.511+0000 7f2609e00640 1 -- 172.21.15.83:0/1827112841 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f260419ffe0 con 0x7f2604103a40 2024-04-24T08:51:38.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.511+0000 7f2601600640 1 -- 172.21.15.83:0/1827112841 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25fc030d40 con 0x7f2604103a40 2024-04-24T08:51:38.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.511+0000 7f2609e00640 1 -- 172.21.15.83:0/1827112841 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26041a04a0 con 0x7f2604103a40 2024-04-24T08:51:38.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.512+0000 7f2609e00640 1 -- 172.21.15.83:0/1827112841 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2604109240 con 0x7f2604103a40 2024-04-24T08:51:38.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.514+0000 7f2601600640 1 -- 172.21.15.83:0/1827112841 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7f25fc004040 con 0x7f2604103a40 2024-04-24T08:51:38.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.515+0000 7f2601600640 1 --2- 172.21.15.83:0/1827112841 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f25ec0a3b50 0x7f25ec0a6010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:38.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.516+0000 7f2601600640 1 -- 172.21.15.83:0/1827112841 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f25fc0f1b80 con 0x7f2604103a40 2024-04-24T08:51:38.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.516+0000 7f2608a00640 1 --2- 172.21.15.83:0/1827112841 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f25ec0a3b50 0x7f25ec0a6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:38.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.517+0000 7f2608a00640 1 --2- 172.21.15.83:0/1827112841 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f25ec0a3b50 0x7f25ec0a6010 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f25f8009b40 tx=0x7f25f8005e50 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:38.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.520+0000 7f2601600640 1 -- 172.21.15.83:0/1827112841 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f25fc0be360 con 0x7f2604103a40 2024-04-24T08:51:38.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.723+0000 7f2609e00640 1 -- 172.21.15.83:0/1827112841 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f2604062c00 con 0x7f2604103a40 2024-04-24T08:51:38.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.724+0000 7f2601600640 1 -- 172.21.15.83:0/1827112841 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7f25fc0dce20 con 0x7f2604103a40 2024-04-24T08:51:38.724 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2024-04-24T08:51:38.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.727+0000 7f2609e00640 1 -- 172.21.15.83:0/1827112841 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f25ec0a3b50 msgr2=0x7f25ec0a6010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:38.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.727+0000 7f2609e00640 1 --2- 172.21.15.83:0/1827112841 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f25ec0a3b50 0x7f25ec0a6010 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f25f8009b40 tx=0x7f25f8005e50 comp rx=0 tx=0).stop 2024-04-24T08:51:38.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.727+0000 7f2609e00640 1 -- 172.21.15.83:0/1827112841 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2604103a40 msgr2=0x7f260419f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:38.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.727+0000 7f2609e00640 1 --2- 172.21.15.83:0/1827112841 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2604103a40 0x7f260419f6e0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f25fc009dc0 tx=0x7f25fc003ba0 comp rx=0 tx=0).stop 2024-04-24T08:51:38.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.727+0000 7f2609e00640 1 -- 172.21.15.83:0/1827112841 shutdown_connections 2024-04-24T08:51:38.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.727+0000 7f2609e00640 1 --2- 172.21.15.83:0/1827112841 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f25ec0a3b50 0x7f25ec0a6010 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:38.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.727+0000 7f2609e00640 1 --2- 172.21.15.83:0/1827112841 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2604103a40 0x7f260419f6e0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:38.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.727+0000 7f2609e00640 1 --2- 172.21.15.83:0/1827112841 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2604103080 0x7f2604198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:38.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.727+0000 7f2609e00640 1 --2- 172.21.15.83:0/1827112841 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2604101f30 0x7f2604198130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:38.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.727+0000 7f2609e00640 1 -- 172.21.15.83:0/1827112841 >> 172.21.15.83:0/1827112841 conn(0x7f26040fd4a0 msgr2=0x7f2604106410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:38.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.728+0000 7f2609e00640 1 -- 172.21.15.83:0/1827112841 shutdown_connections 2024-04-24T08:51:38.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.728+0000 7f2609e00640 1 -- 172.21.15.83:0/1827112841 wait complete. 2024-04-24T08:51:38.745 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-24T08:51:38.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.929+0000 7f1843600640 1 -- 172.21.15.83:0/1243292036 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f183c109ef0 msgr2=0x7f183c10c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:38.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.929+0000 7f1843600640 1 --2- 172.21.15.83:0/1243292036 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f183c109ef0 0x7f183c10c360 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f1834009ca0 tx=0x7f183402a8c0 comp rx=0 tx=0).stop 2024-04-24T08:51:38.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.930+0000 7f1843600640 1 -- 172.21.15.83:0/1243292036 shutdown_connections 2024-04-24T08:51:38.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.930+0000 7f1843600640 1 --2- 172.21.15.83:0/1243292036 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f183c109ef0 0x7f183c10c360 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:38.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.930+0000 7f1843600640 1 --2- 172.21.15.83:0/1243292036 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f183c074a20 0x7f183c074ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:38.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.930+0000 7f1843600640 1 --2- 172.21.15.83:0/1243292036 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f183c076000 0x7f183c074450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:38.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.930+0000 7f1843600640 1 -- 172.21.15.83:0/1243292036 >> 172.21.15.83:0/1243292036 conn(0x7f183c0fd5b0 msgr2=0x7f183c0ff9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:38.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.930+0000 7f1843600640 1 -- 172.21.15.83:0/1243292036 shutdown_connections 2024-04-24T08:51:38.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.930+0000 7f1843600640 1 -- 172.21.15.83:0/1243292036 wait complete. 2024-04-24T08:51:38.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.931+0000 7f1843600640 1 Processor -- start 2024-04-24T08:51:38.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.931+0000 7f1843600640 1 -- start start 2024-04-24T08:51:38.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.931+0000 7f1843600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f183c074a20 0x7f183c1053a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:38.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.931+0000 7f1843600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f183c076000 0x7f183c1058e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:38.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.931+0000 7f1843600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f183c109ef0 0x7f183c101f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:38.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.931+0000 7f1843600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f183c10fd70 con 0x7f183c074a20 2024-04-24T08:51:38.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.931+0000 7f1843600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f183c10fbf0 con 0x7f183c076000 2024-04-24T08:51:38.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.931+0000 7f1843600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f183c10fef0 con 0x7f183c109ef0 2024-04-24T08:51:38.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.931+0000 7f1840c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f183c074a20 0x7f183c1053a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:38.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.931+0000 7f1840c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f183c074a20 0x7f183c1053a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55668/0 (socket says 172.21.15.83:55668) 2024-04-24T08:51:38.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.931+0000 7f1840c00640 1 -- 172.21.15.83:0/236693550 learned_addr learned my addr 172.21.15.83:0/236693550 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:51:38.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.932+0000 7f183be00640 1 --2- 172.21.15.83:0/236693550 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f183c076000 0x7f183c1058e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:38.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.932+0000 7f1841600640 1 --2- 172.21.15.83:0/236693550 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f183c109ef0 0x7f183c101f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:38.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.932+0000 7f1840c00640 1 -- 172.21.15.83:0/236693550 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f183c109ef0 msgr2=0x7f183c101f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:38.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.932+0000 7f1840c00640 1 --2- 172.21.15.83:0/236693550 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f183c109ef0 0x7f183c101f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:38.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.932+0000 7f1840c00640 1 -- 172.21.15.83:0/236693550 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f183c076000 msgr2=0x7f183c1058e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:38.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.932+0000 7f1840c00640 1 --2- 172.21.15.83:0/236693550 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f183c076000 0x7f183c1058e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:38.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.932+0000 7f1840c00640 1 -- 172.21.15.83:0/236693550 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f183c1027d0 con 0x7f183c074a20 2024-04-24T08:51:38.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.932+0000 7f1840c00640 1 --2- 172.21.15.83:0/236693550 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f183c074a20 0x7f183c1053a0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f183000e9f0 tx=0x7f183000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:38.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.933+0000 7f1839600640 1 -- 172.21.15.83:0/236693550 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f183000cd80 con 0x7f183c074a20 2024-04-24T08:51:38.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.933+0000 7f1839600640 1 -- 172.21.15.83:0/236693550 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f18300054d0 con 0x7f183c074a20 2024-04-24T08:51:38.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.933+0000 7f1839600640 1 -- 172.21.15.83:0/236693550 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1830010490 con 0x7f183c074a20 2024-04-24T08:51:38.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.933+0000 7f1843600640 1 -- 172.21.15.83:0/236693550 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f183c1a68a0 con 0x7f183c074a20 2024-04-24T08:51:38.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.933+0000 7f1843600640 1 -- 172.21.15.83:0/236693550 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f183c1a6cb0 con 0x7f183c074a20 2024-04-24T08:51:38.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.934+0000 7f1843600640 1 -- 172.21.15.83:0/236693550 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f183c108c70 con 0x7f183c074a20 2024-04-24T08:51:38.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.937+0000 7f1839600640 1 -- 172.21.15.83:0/236693550 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7f1830005020 con 0x7f183c074a20 2024-04-24T08:51:38.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.939+0000 7f1839600640 1 --2- 172.21.15.83:0/236693550 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f18240a39b0 0x7f18240a5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:38.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.939+0000 7f1839600640 1 -- 172.21.15.83:0/236693550 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f18300d0950 con 0x7f183c074a20 2024-04-24T08:51:38.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.939+0000 7f183be00640 1 --2- 172.21.15.83:0/236693550 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f18240a39b0 0x7f18240a5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:38.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.940+0000 7f183be00640 1 --2- 172.21.15.83:0/236693550 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f18240a39b0 0x7f18240a5e70 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f183c103320 tx=0x7f1828005f90 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:38.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.943+0000 7f1839600640 1 -- 172.21.15.83:0/236693550 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f183009d0f0 con 0x7f183c074a20 2024-04-24T08:51:38.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:38.965+0000 7f1839600640 1 -- 172.21.15.83:0/236693550 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mgrmap(e 23) ==== 138766+0+0 (secure 0 0 0) 0x7f1830099060 con 0x7f183c074a20 2024-04-24T08:51:39.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:38 smithi083 ceph-mon[27907]: [24/Apr/2024:08:51:37] ENGINE Bus STARTING 2024-04-24T08:51:39.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:38 smithi083 ceph-mon[27907]: [24/Apr/2024:08:51:37] ENGINE Serving on https://172.21.15.83:7150 2024-04-24T08:51:39.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:38 smithi083 ceph-mon[27907]: [24/Apr/2024:08:51:37] ENGINE Serving on http://172.21.15.83:8765 2024-04-24T08:51:39.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:38 smithi083 ceph-mon[27907]: [24/Apr/2024:08:51:37] ENGINE Bus STARTED 2024-04-24T08:51:39.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:38 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/903175057' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T08:51:39.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:38 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/1827112841' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T08:51:39.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:39.164+0000 7f1843600640 1 -- 172.21.15.83:0/236693550 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f183c062c00 con 0x7f183c074a20 2024-04-24T08:51:39.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:39.164+0000 7f1839600640 1 -- 172.21.15.83:0/236693550 <== mon.0 v2:172.21.15.83:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) ==== 103+0+11 (secure 0 0 0) 0x7f183005c9d0 con 0x7f183c074a20 2024-04-24T08:51:39.164 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-24T08:51:39.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:39.168+0000 7f1843600640 1 -- 172.21.15.83:0/236693550 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f18240a39b0 msgr2=0x7f18240a5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:39.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:39.168+0000 7f1843600640 1 --2- 172.21.15.83:0/236693550 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f18240a39b0 0x7f18240a5e70 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f183c103320 tx=0x7f1828005f90 comp rx=0 tx=0).stop 2024-04-24T08:51:39.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:39.168+0000 7f1843600640 1 -- 172.21.15.83:0/236693550 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f183c074a20 msgr2=0x7f183c1053a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:39.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:39.168+0000 7f1843600640 1 --2- 172.21.15.83:0/236693550 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f183c074a20 0x7f183c1053a0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f183000e9f0 tx=0x7f183000edf0 comp rx=0 tx=0).stop 2024-04-24T08:51:39.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:39.168+0000 7f1843600640 1 -- 172.21.15.83:0/236693550 shutdown_connections 2024-04-24T08:51:39.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:39.168+0000 7f1843600640 1 --2- 172.21.15.83:0/236693550 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f18240a39b0 0x7f18240a5e70 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:39.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:39.168+0000 7f1843600640 1 --2- 172.21.15.83:0/236693550 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f183c109ef0 0x7f183c101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:39.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:39.168+0000 7f1843600640 1 --2- 172.21.15.83:0/236693550 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f183c076000 0x7f183c1058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:39.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:39.168+0000 7f1843600640 1 --2- 172.21.15.83:0/236693550 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f183c074a20 0x7f183c1053a0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:39.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:39.168+0000 7f1843600640 1 -- 172.21.15.83:0/236693550 >> 172.21.15.83:0/236693550 conn(0x7f183c0fd5b0 msgr2=0x7f183c105e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:39.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:39.168+0000 7f1843600640 1 -- 172.21.15.83:0/236693550 shutdown_connections 2024-04-24T08:51:39.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:39.168+0000 7f1843600640 1 -- 172.21.15.83:0/236693550 wait complete. 2024-04-24T08:51:39.185 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T08:51:39.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:39.365+0000 7f224ac00640 1 -- 172.21.15.83:0/657516916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2244107b60 msgr2=0x7f2244109fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:39.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:39.365+0000 7f224ac00640 1 --2- 172.21.15.83:0/657516916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2244107b60 0x7f2244109fb0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f223c009ca0 tx=0x7f223c02a8c0 comp rx=0 tx=0).stop 2024-04-24T08:51:39.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:39.366+0000 7f224ac00640 1 -- 172.21.15.83:0/657516916 shutdown_connections 2024-04-24T08:51:39.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:39.366+0000 7f224ac00640 1 --2- 172.21.15.83:0/657516916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2244107b60 0x7f2244109fb0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:39.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:39.366+0000 7f224ac00640 1 --2- 172.21.15.83:0/657516916 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2244105230 0x7f2244107620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:39.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:39.366+0000 7f224ac00640 1 --2- 172.21.15.83:0/657516916 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2244102900 0x7f2244104cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:39.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:39.366+0000 7f224ac00640 1 -- 172.21.15.83:0/657516916 >> 172.21.15.83:0/657516916 conn(0x7f22440fc310 msgr2=0x7f22440fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:39.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:39.366+0000 7f224ac00640 1 -- 172.21.15.83:0/657516916 shutdown_connections 2024-04-24T08:51:39.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:39.366+0000 7f224ac00640 1 -- 172.21.15.83:0/657516916 wait complete. 2024-04-24T08:51:39.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:39.367+0000 7f224ac00640 1 Processor -- start 2024-04-24T08:51:39.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:39.367+0000 7f224ac00640 1 -- start start 2024-04-24T08:51:39.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:39.367+0000 7f224ac00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2244102900 0x7f224419a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:39.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:39.367+0000 7f224ac00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2244105230 0x7f224419a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:39.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:39.367+0000 7f224ac00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2244107b60 0x7f22441a18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:39.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:39.367+0000 7f224ac00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f224410da20 con 0x7f2244105230 2024-04-24T08:51:39.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:39.367+0000 7f224ac00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f224410d8a0 con 0x7f2244107b60 2024-04-24T08:51:39.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:39.367+0000 7f224ac00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f224410dba0 con 0x7f2244102900 2024-04-24T08:51:39.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:39.367+0000 7f2248e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2244105230 0x7f224419a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:39.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:39.367+0000 7f2248e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2244105230 0x7f224419a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55686/0 (socket says 172.21.15.83:55686) 2024-04-24T08:51:39.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:39.368+0000 7f2248e00640 1 -- 172.21.15.83:0/1254968157 learned_addr learned my addr 172.21.15.83:0/1254968157 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:51:39.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:39.368+0000 7f2249800640 1 --2- 172.21.15.83:0/1254968157 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2244102900 0x7f224419a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:39.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:39.368+0000 7f224a200640 1 --2- 172.21.15.83:0/1254968157 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2244107b60 0x7f22441a18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:39.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:39.368+0000 7f2248e00640 1 -- 172.21.15.83:0/1254968157 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2244102900 msgr2=0x7f224419a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:39.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:39.368+0000 7f2248e00640 1 --2- 172.21.15.83:0/1254968157 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2244102900 0x7f224419a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:39.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:39.368+0000 7f2248e00640 1 -- 172.21.15.83:0/1254968157 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2244107b60 msgr2=0x7f22441a18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:39.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:39.368+0000 7f2248e00640 1 --2- 172.21.15.83:0/1254968157 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2244107b60 0x7f22441a18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:39.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:39.368+0000 7f2248e00640 1 -- 172.21.15.83:0/1254968157 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22441a1f10 con 0x7f2244105230 2024-04-24T08:51:39.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:39.368+0000 7f2248e00640 1 --2- 172.21.15.83:0/1254968157 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2244105230 0x7f224419a830 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f223000e9c0 tx=0x7f223000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:39.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:39.369+0000 7f2242000640 1 -- 172.21.15.83:0/1254968157 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f223000cd80 con 0x7f2244105230 2024-04-24T08:51:39.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:39.369+0000 7f2242000640 1 -- 172.21.15.83:0/1254968157 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f22300054d0 con 0x7f2244105230 2024-04-24T08:51:39.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:39.369+0000 7f224ac00640 1 -- 172.21.15.83:0/1254968157 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22441a2200 con 0x7f2244105230 2024-04-24T08:51:39.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:39.369+0000 7f2242000640 1 -- 172.21.15.83:0/1254968157 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2230010430 con 0x7f2244105230 2024-04-24T08:51:39.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:39.369+0000 7f224ac00640 1 -- 172.21.15.83:0/1254968157 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22441a27a0 con 0x7f2244105230 2024-04-24T08:51:39.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:39.371+0000 7f224ac00640 1 -- 172.21.15.83:0/1254968157 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2244109650 con 0x7f2244105230 2024-04-24T08:51:39.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:39.373+0000 7f2242000640 1 -- 172.21.15.83:0/1254968157 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 23) ==== 138766+0+0 (secure 0 0 0) 0x7f2230004fd0 con 0x7f2244105230 2024-04-24T08:51:39.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:39.374+0000 7f2242000640 1 --2- 172.21.15.83:0/1254968157 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f222c0a3da0 0x7f222c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:39.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:39.374+0000 7f2242000640 1 -- 172.21.15.83:0/1254968157 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f22300d0f50 con 0x7f2244105230 2024-04-24T08:51:39.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:39.375+0000 7f2249800640 1 --2- 172.21.15.83:0/1254968157 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f222c0a3da0 0x7f222c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:39.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:39.376+0000 7f2249800640 1 --2- 172.21.15.83:0/1254968157 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f222c0a3da0 0x7f222c0a6260 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f2238009bd0 tx=0x7f2238005f90 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:39.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:39.381+0000 7f2242000640 1 -- 172.21.15.83:0/1254968157 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f223009d220 con 0x7f2244105230 2024-04-24T08:51:39.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:38 smithi138 ceph-mon[31441]: [24/Apr/2024:08:51:37] ENGINE Bus STARTING 2024-04-24T08:51:39.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:38 smithi138 ceph-mon[31441]: [24/Apr/2024:08:51:37] ENGINE Serving on https://172.21.15.83:7150 2024-04-24T08:51:39.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:38 smithi138 ceph-mon[31441]: [24/Apr/2024:08:51:37] ENGINE Serving on http://172.21.15.83:8765 2024-04-24T08:51:39.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:38 smithi138 ceph-mon[31441]: [24/Apr/2024:08:51:37] ENGINE Bus STARTED 2024-04-24T08:51:39.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:38 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/903175057' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T08:51:39.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:38 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/1827112841' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T08:51:39.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:38 smithi189 ceph-mon[31791]: [24/Apr/2024:08:51:37] ENGINE Bus STARTING 2024-04-24T08:51:39.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:38 smithi189 ceph-mon[31791]: [24/Apr/2024:08:51:37] ENGINE Serving on https://172.21.15.83:7150 2024-04-24T08:51:39.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:38 smithi189 ceph-mon[31791]: [24/Apr/2024:08:51:37] ENGINE Serving on http://172.21.15.83:8765 2024-04-24T08:51:39.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:38 smithi189 ceph-mon[31791]: [24/Apr/2024:08:51:37] ENGINE Bus STARTED 2024-04-24T08:51:39.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:38 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/903175057' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T08:51:39.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:38 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/1827112841' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T08:51:39.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:39.594+0000 7f224ac00640 1 -- 172.21.15.83:0/1254968157 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f224402d7a0 con 0x7f2244105230 2024-04-24T08:51:39.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:39.596+0000 7f2242000640 1 -- 172.21.15.83:0/1254968157 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+25265 (secure 0 0 0) 0x7f223009cb20 con 0x7f2244105230 2024-04-24T08:51:39.596 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:51:39.596 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":94,"fsid":"bd3e039e-0215-11ef-bc93-c7b262605968","created":"2024-04-24T08:37:32.209944+0000","modified":"2024-04-24T08:51:36.347834+0000","last_up_change":"2024-04-24T08:47:17.966041+0000","last_in_change":"2024-04-24T08:46:53.835936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T08:41:56.834372+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T08:48:26.978311+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T08:48:28.352731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T08:48:32.275167+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2cd77f3a-19dd-4c6e-a199-1dcbd5faa679","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2132675555},{"type":"v1","addr":"172.21.15.83:6803","nonce":2132675555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2132675555},{"type":"v1","addr":"172.21.15.83:6805","nonce":2132675555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2132675555},{"type":"v1","addr":"172.21.15.83:6809","nonce":2132675555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2132675555},{"type":"v1","addr":"172.21.15.83:6807","nonce":2132675555}]},"public_addr":"172.21.15.83:6803/2132675555","cluster_addr":"172.21.15.83:6805/2132675555","heartbeat_back_addr":"172.21.15.83:6809/2132675555","heartbeat_front_addr":"172.21.15.83:6807/2132675555","state":["exists","up"]},{"osd":1,"uuid":"aba7f5df-2a4c-4c23-b502-d193605a5b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":3589158880},{"type":"v1","addr":"172.21.15.138:6801","nonce":3589158880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":3589158880},{"type":"v1","addr":"172.21.15.138:6803","nonce":3589158880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":3589158880},{"type":"v1","addr":"172.21.15.138:6807","nonce":3589158880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":3589158880},{"type":"v1","addr":"172.21.15.138:6805","nonce":3589158880}]},"public_addr":"172.21.15.138:6801/3589158880","cluster_addr":"172.21.15.138:6803/3589158880","heartbeat_back_addr":"172.21.15.138:6807/3589158880","heartbeat_front_addr":"172.21.15.138:6805/3589158880","state":["exists","up"]},{"osd":2,"uuid":"071bc167-2333-4420-bb58-bffbc10d7641","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":3169072688},{"type":"v1","addr":"172.21.15.189:6801","nonce":3169072688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":3169072688},{"type":"v1","addr":"172.21.15.189:6803","nonce":3169072688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":3169072688},{"type":"v1","addr":"172.21.15.189:6807","nonce":3169072688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":3169072688},{"type":"v1","addr":"172.21.15.189:6805","nonce":3169072688}]},"public_addr":"172.21.15.189:6801/3169072688","cluster_addr":"172.21.15.189:6803/3169072688","heartbeat_back_addr":"172.21.15.189:6807/3169072688","heartbeat_front_addr":"172.21.15.189:6805/3169072688","state":["exists","up"]},{"osd":3,"uuid":"f5624d78-4455-4e07-8c1a-f47c94cddb23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3131092450},{"type":"v1","addr":"172.21.15.83:6811","nonce":3131092450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3131092450},{"type":"v1","addr":"172.21.15.83:6813","nonce":3131092450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3131092450},{"type":"v1","addr":"172.21.15.83:6817","nonce":3131092450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3131092450},{"type":"v1","addr":"172.21.15.83:6815","nonce":3131092450}]},"public_addr":"172.21.15.83:6811/3131092450","cluster_addr":"172.21.15.83:6813/3131092450","heartbeat_back_addr":"172.21.15.83:6817/3131092450","heartbeat_front_addr":"172.21.15.83:6815/3131092450","state":["exists","up"]},{"osd":4,"uuid":"7b50cd14-9385-4535-a7b0-1c1da93c8548","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":823775360},{"type":"v1","addr":"172.21.15.138:6809","nonce":823775360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":823775360},{"type":"v1","addr":"172.21.15.138:6811","nonce":823775360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":823775360},{"type":"v1","addr":"172.21.15.138:6815","nonce":823775360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":823775360},{"type":"v1","addr":"172.21.15.138:6813","nonce":823775360}]},"public_addr":"172.21.15.138:6809/823775360","cluster_addr":"172.21.15.138:6811/823775360","heartbeat_back_addr":"172.21.15.138:6815/823775360","heartbeat_front_addr":"172.21.15.138:6813/823775360","state":["exists","up"]},{"osd":5,"uuid":"9c647797-35bc-449f-be84-af97d2c69c2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":1399817431},{"type":"v1","addr":"172.21.15.189:6809","nonce":1399817431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":1399817431},{"type":"v1","addr":"172.21.15.189:6811","nonce":1399817431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":1399817431},{"type":"v1","addr":"172.21.15.189:6815","nonce":1399817431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":1399817431},{"type":"v1","addr":"172.21.15.189:6813","nonce":1399817431}]},"public_addr":"172.21.15.189:6809/1399817431","cluster_addr":"172.21.15.189:6811/1399817431","heartbeat_back_addr":"172.21.15.189:6815/1399817431","heartbeat_front_addr":"172.21.15.189:6813/1399817431","state":["exists","up"]},{"osd":6,"uuid":"8f056584-16c8-4454-b445-3f780f26c196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":1611358483},{"type":"v1","addr":"172.21.15.83:6819","nonce":1611358483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1611358483},{"type":"v1","addr":"172.21.15.83:6821","nonce":1611358483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1611358483},{"type":"v1","addr":"172.21.15.83:6825","nonce":1611358483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":1611358483},{"type":"v1","addr":"172.21.15.83:6823","nonce":1611358483}]},"public_addr":"172.21.15.83:6819/1611358483","cluster_addr":"172.21.15.83:6821/1611358483","heartbeat_back_addr":"172.21.15.83:6825/1611358483","heartbeat_front_addr":"172.21.15.83:6823/1611358483","state":["exists","up"]},{"osd":7,"uuid":"7603efe3-da77-4b7f-9d03-29c4f293b6d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":3557856706},{"type":"v1","addr":"172.21.15.138:6817","nonce":3557856706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":3557856706},{"type":"v1","addr":"172.21.15.138:6819","nonce":3557856706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":3557856706},{"type":"v1","addr":"172.21.15.138:6823","nonce":3557856706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":3557856706},{"type":"v1","addr":"172.21.15.138:6821","nonce":3557856706}]},"public_addr":"172.21.15.138:6817/3557856706","cluster_addr":"172.21.15.138:6819/3557856706","heartbeat_back_addr":"172.21.15.138:6823/3557856706","heartbeat_front_addr":"172.21.15.138:6821/3557856706","state":["exists","up"]},{"osd":8,"uuid":"27d7d1b8-d6a0-417e-b8aa-596325aa1a77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":207018554},{"type":"v1","addr":"172.21.15.189:6817","nonce":207018554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":207018554},{"type":"v1","addr":"172.21.15.189:6819","nonce":207018554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":207018554},{"type":"v1","addr":"172.21.15.189:6823","nonce":207018554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":207018554},{"type":"v1","addr":"172.21.15.189:6821","nonce":207018554}]},"public_addr":"172.21.15.189:6817/207018554","cluster_addr":"172.21.15.189:6819/207018554","heartbeat_back_addr":"172.21.15.189:6823/207018554","heartbeat_front_addr":"172.21.15.189:6821/207018554","state":["exists","up"]},{"osd":9,"uuid":"baf19b98-881b-4d3a-9f72-8aa2dd73cad6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":519636751},{"type":"v1","addr":"172.21.15.83:6827","nonce":519636751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":519636751},{"type":"v1","addr":"172.21.15.83:6829","nonce":519636751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":519636751},{"type":"v1","addr":"172.21.15.83:6833","nonce":519636751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":519636751},{"type":"v1","addr":"172.21.15.83:6831","nonce":519636751}]},"public_addr":"172.21.15.83:6827/519636751","cluster_addr":"172.21.15.83:6829/519636751","heartbeat_back_addr":"172.21.15.83:6833/519636751","heartbeat_front_addr":"172.21.15.83:6831/519636751","state":["exists","up"]},{"osd":10,"uuid":"eb346717-5c07-47c8-95be-0e68fe449fdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1130872098},{"type":"v1","addr":"172.21.15.138:6825","nonce":1130872098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1130872098},{"type":"v1","addr":"172.21.15.138:6827","nonce":1130872098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1130872098},{"type":"v1","addr":"172.21.15.138:6831","nonce":1130872098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1130872098},{"type":"v1","addr":"172.21.15.138:6829","nonce":1130872098}]},"public_addr":"172.21.15.138:6825/1130872098","cluster_addr":"172.21.15.138:6827/1130872098","heartbeat_back_addr":"172.21.15.138:6831/1130872098","heartbeat_front_addr":"172.21.15.138:6829/1130872098","state":["exists","up"]},{"osd":11,"uuid":"8e37568e-981c-45b8-b2af-2f1b73898542","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":2998782690},{"type":"v1","addr":"172.21.15.189:6825","nonce":2998782690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":2998782690},{"type":"v1","addr":"172.21.15.189:6827","nonce":2998782690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":2998782690},{"type":"v1","addr":"172.21.15.189:6831","nonce":2998782690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":2998782690},{"type":"v1","addr":"172.21.15.189:6829","nonce":2998782690}]},"public_addr":"172.21.15.189:6825/2998782690","cluster_addr":"172.21.15.189:6827/2998782690","heartbeat_back_addr":"172.21.15.189:6831/2998782690","heartbeat_front_addr":"172.21.15.189:6829/2998782690","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:40:47.980103+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:41:21.970795+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:41:55.160426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:42:30.400475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:43:05.705663+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:43:40.112312+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:44:16.781289+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:44:52.151187+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:45:28.055944+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:46:04.969301+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:46:40.918306+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:47:16.886874+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"2.1d","mappings":[{"from":2,"to":1}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.2a","mappings":[{"from":3,"to":8}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.a","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:0/1096290901":"2024-04-25T08:51:36.347763+0000","172.21.15.83:6800/2550826560":"2024-04-25T08:51:36.347763+0000","172.21.15.83:0/1743153945":"2024-04-25T08:51:36.347763+0000","172.21.15.83:0/3306681366":"2024-04-25T08:51:36.347763+0000","172.21.15.83:6801/820049883":"2024-04-25T08:38:40.661315+0000","172.21.15.83:6801/2550826560":"2024-04-25T08:51:36.347763+0000","172.21.15.83:0/4272874138":"2024-04-25T08:38:40.661315+0000","172.21.15.83:0/4212994335":"2024-04-25T08:38:40.661315+0000","172.21.15.83:6801/1996371091":"2024-04-25T08:38:05.800856+0000","172.21.15.83:0/1820663683":"2024-04-25T08:38:05.800856+0000","172.21.15.83:0/3802897262":"2024-04-25T08:38:05.800856+0000","172.21.15.83:6800/1996371091":"2024-04-25T08:38:05.800856+0000","172.21.15.83:6800/820049883":"2024-04-25T08:38:40.661315+0000","172.21.15.83:0/1366125049":"2024-04-25T08:38:05.800856+0000","172.21.15.83:0/4159820288":"2024-04-25T08:51:36.347763+0000","172.21.15.83:0/340452868":"2024-04-25T08:38:40.661315+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T08:51:39.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:39.598+0000 7f224ac00640 1 -- 172.21.15.83:0/1254968157 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f222c0a3da0 msgr2=0x7f222c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:39.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:39.598+0000 7f224ac00640 1 --2- 172.21.15.83:0/1254968157 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f222c0a3da0 0x7f222c0a6260 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f2238009bd0 tx=0x7f2238005f90 comp rx=0 tx=0).stop 2024-04-24T08:51:39.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:39.598+0000 7f224ac00640 1 -- 172.21.15.83:0/1254968157 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2244105230 msgr2=0x7f224419a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:39.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:39.598+0000 7f224ac00640 1 --2- 172.21.15.83:0/1254968157 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2244105230 0x7f224419a830 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f223000e9c0 tx=0x7f223000ee80 comp rx=0 tx=0).stop 2024-04-24T08:51:39.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:39.599+0000 7f224ac00640 1 -- 172.21.15.83:0/1254968157 shutdown_connections 2024-04-24T08:51:39.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:39.599+0000 7f224ac00640 1 --2- 172.21.15.83:0/1254968157 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f222c0a3da0 0x7f222c0a6260 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:39.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:39.599+0000 7f224ac00640 1 --2- 172.21.15.83:0/1254968157 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2244107b60 0x7f22441a18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:39.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:39.599+0000 7f224ac00640 1 --2- 172.21.15.83:0/1254968157 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2244105230 0x7f224419a830 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:39.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:39.599+0000 7f224ac00640 1 --2- 172.21.15.83:0/1254968157 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2244102900 0x7f224419a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:39.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:39.599+0000 7f224ac00640 1 -- 172.21.15.83:0/1254968157 >> 172.21.15.83:0/1254968157 conn(0x7f22440fc310 msgr2=0x7f2244102060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:39.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:39.599+0000 7f224ac00640 1 -- 172.21.15.83:0/1254968157 shutdown_connections 2024-04-24T08:51:39.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:39.599+0000 7f224ac00640 1 -- 172.21.15.83:0/1254968157 wait complete. 2024-04-24T08:51:39.622 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T08:41:56.834372+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T08:48:26.978311+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T08:48:28.352731+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-24T08:48:32.275167+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-24T08:51:39.622 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T08:51:39.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:39.820+0000 7fb2ffc00640 1 -- 172.21.15.83:0/908287226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb2f8101c40 msgr2=0x7fb2f81020c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:39.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:39.820+0000 7fb2ffc00640 1 --2- 172.21.15.83:0/908287226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb2f8101c40 0x7fb2f81020c0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fb2f0009ca0 tx=0x7fb2f002a950 comp rx=0 tx=0).stop 2024-04-24T08:51:39.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:39.820+0000 7fb2ffc00640 1 -- 172.21.15.83:0/908287226 shutdown_connections 2024-04-24T08:51:39.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:39.820+0000 7fb2ffc00640 1 --2- 172.21.15.83:0/908287226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb2f8101c40 0x7fb2f81020c0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:39.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:39.820+0000 7fb2ffc00640 1 --2- 172.21.15.83:0/908287226 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2f81012a0 0x7fb2f8101700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:39.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:39.820+0000 7fb2ffc00640 1 --2- 172.21.15.83:0/908287226 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb2f8100940 0x7fb2f8100d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:39.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:39.820+0000 7fb2ffc00640 1 -- 172.21.15.83:0/908287226 >> 172.21.15.83:0/908287226 conn(0x7fb2f80fc4c0 msgr2=0x7fb2f80fe900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:39.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:39.821+0000 7fb2ffc00640 1 -- 172.21.15.83:0/908287226 shutdown_connections 2024-04-24T08:51:39.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:39.821+0000 7fb2ffc00640 1 -- 172.21.15.83:0/908287226 wait complete. 2024-04-24T08:51:39.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:39.821+0000 7fb2ffc00640 1 Processor -- start 2024-04-24T08:51:39.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:39.822+0000 7fb2ffc00640 1 -- start start 2024-04-24T08:51:39.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:39.822+0000 7fb2ffc00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb2f8100940 0x7fb2f819a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:39.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:39.822+0000 7fb2ffc00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2f81012a0 0x7fb2f819a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:39.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:39.822+0000 7fb2ffc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb2f8101c40 0x7fb2f81a19d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:39.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:39.822+0000 7fb2ffc00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f810fd90 con 0x7fb2f8101c40 2024-04-24T08:51:39.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:39.822+0000 7fb2ffc00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f810fc10 con 0x7fb2f81012a0 2024-04-24T08:51:39.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:39.822+0000 7fb2ffc00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f810ff10 con 0x7fb2f8100940 2024-04-24T08:51:39.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:39.822+0000 7fb2fdc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb2f8101c40 0x7fb2f81a19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:39.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:39.822+0000 7fb2fdc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb2f8101c40 0x7fb2f81a19d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55708/0 (socket says 172.21.15.83:55708) 2024-04-24T08:51:39.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:39.822+0000 7fb2fdc00640 1 -- 172.21.15.83:0/2629580367 learned_addr learned my addr 172.21.15.83:0/2629580367 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:51:39.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:39.822+0000 7fb2fd200640 1 --2- 172.21.15.83:0/2629580367 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb2f8100940 0x7fb2f819a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:39.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:39.823+0000 7fb2f7e00640 1 --2- 172.21.15.83:0/2629580367 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2f81012a0 0x7fb2f819a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:39.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:39.823+0000 7fb2fdc00640 1 -- 172.21.15.83:0/2629580367 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb2f8100940 msgr2=0x7fb2f819a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:39.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:39.823+0000 7fb2fdc00640 1 --2- 172.21.15.83:0/2629580367 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb2f8100940 0x7fb2f819a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:39.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:39.823+0000 7fb2fdc00640 1 -- 172.21.15.83:0/2629580367 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2f81012a0 msgr2=0x7fb2f819a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:39.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:39.823+0000 7fb2fdc00640 1 --2- 172.21.15.83:0/2629580367 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2f81012a0 0x7fb2f819a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:39.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:39.823+0000 7fb2fdc00640 1 -- 172.21.15.83:0/2629580367 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2f81a2040 con 0x7fb2f8101c40 2024-04-24T08:51:39.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:39.823+0000 7fb2fdc00640 1 --2- 172.21.15.83:0/2629580367 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb2f8101c40 0x7fb2f81a19d0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fb2f0006010 tx=0x7fb2f002fc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:39.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:39.823+0000 7fb2f5600640 1 -- 172.21.15.83:0/2629580367 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2f0034040 con 0x7fb2f8101c40 2024-04-24T08:51:39.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:39.823+0000 7fb2f5600640 1 -- 172.21.15.83:0/2629580367 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb2f00308f0 con 0x7fb2f8101c40 2024-04-24T08:51:39.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:39.823+0000 7fb2f5600640 1 -- 172.21.15.83:0/2629580367 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2f0039450 con 0x7fb2f8101c40 2024-04-24T08:51:39.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:39.823+0000 7fb2ffc00640 1 -- 172.21.15.83:0/2629580367 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2f81a22d0 con 0x7fb2f8101c40 2024-04-24T08:51:39.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:39.824+0000 7fb2ffc00640 1 -- 172.21.15.83:0/2629580367 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2f81a27c0 con 0x7fb2f8101c40 2024-04-24T08:51:39.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:39.825+0000 7fb2ffc00640 1 -- 172.21.15.83:0/2629580367 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2f810b880 con 0x7fb2f8101c40 2024-04-24T08:51:39.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:39.827+0000 7fb2f5600640 1 -- 172.21.15.83:0/2629580367 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 23) ==== 138766+0+0 (secure 0 0 0) 0x7fb2f0037020 con 0x7fb2f8101c40 2024-04-24T08:51:39.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:39.828+0000 7fb2f5600640 1 --2- 172.21.15.83:0/2629580367 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb2e00a3d20 0x7fb2e00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:39.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:39.828+0000 7fb2f5600640 1 -- 172.21.15.83:0/2629580367 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7fb2f00f1da0 con 0x7fb2f8101c40 2024-04-24T08:51:39.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:39.828+0000 7fb2fd200640 1 --2- 172.21.15.83:0/2629580367 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb2e00a3d20 0x7fb2e00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:39.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:39.829+0000 7fb2fd200640 1 --2- 172.21.15.83:0/2629580367 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb2e00a3d20 0x7fb2e00a61e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec0058c0 tx=0x7fb2ec00a4c0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:39.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:39.833+0000 7fb2f5600640 1 -- 172.21.15.83:0/2629580367 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb2f00be180 con 0x7fb2f8101c40 2024-04-24T08:51:40.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.038+0000 7fb2ffc00640 1 -- 172.21.15.83:0/2629580367 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb2f8062c00 con 0x7fb2f8101c40 2024-04-24T08:51:40.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.039+0000 7fb2f5600640 1 -- 172.21.15.83:0/2629580367 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7fb2f0030ac0 con 0x7fb2f8101c40 2024-04-24T08:51:40.038 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-24T08:51:40.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.042+0000 7fb2ffc00640 1 -- 172.21.15.83:0/2629580367 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb2e00a3d20 msgr2=0x7fb2e00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:40.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.042+0000 7fb2ffc00640 1 --2- 172.21.15.83:0/2629580367 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb2e00a3d20 0x7fb2e00a61e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec0058c0 tx=0x7fb2ec00a4c0 comp rx=0 tx=0).stop 2024-04-24T08:51:40.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.042+0000 7fb2ffc00640 1 -- 172.21.15.83:0/2629580367 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb2f8101c40 msgr2=0x7fb2f81a19d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:40.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.042+0000 7fb2ffc00640 1 --2- 172.21.15.83:0/2629580367 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb2f8101c40 0x7fb2f81a19d0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fb2f0006010 tx=0x7fb2f002fc00 comp rx=0 tx=0).stop 2024-04-24T08:51:40.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.042+0000 7fb2ffc00640 1 -- 172.21.15.83:0/2629580367 shutdown_connections 2024-04-24T08:51:40.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.042+0000 7fb2ffc00640 1 --2- 172.21.15.83:0/2629580367 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb2e00a3d20 0x7fb2e00a61e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:40.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.042+0000 7fb2ffc00640 1 --2- 172.21.15.83:0/2629580367 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb2f8101c40 0x7fb2f81a19d0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:40.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.042+0000 7fb2ffc00640 1 --2- 172.21.15.83:0/2629580367 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2f81012a0 0x7fb2f819a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:40.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.042+0000 7fb2ffc00640 1 --2- 172.21.15.83:0/2629580367 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb2f8100940 0x7fb2f819a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:40.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.042+0000 7fb2ffc00640 1 -- 172.21.15.83:0/2629580367 >> 172.21.15.83:0/2629580367 conn(0x7fb2f80fc4c0 msgr2=0x7fb2f80fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:40.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.042+0000 7fb2ffc00640 1 -- 172.21.15.83:0/2629580367 shutdown_connections 2024-04-24T08:51:40.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.042+0000 7fb2ffc00640 1 -- 172.21.15.83:0/2629580367 wait complete. 2024-04-24T08:51:40.059 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T08:51:40.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.253+0000 7f15f5e00640 1 -- 172.21.15.83:0/4169014963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f15f0105470 msgr2=0x7f15f01058f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:40.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.253+0000 7f15f5e00640 1 --2- 172.21.15.83:0/4169014963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f15f0105470 0x7f15f01058f0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f15d8009900 tx=0x7f15d802a410 comp rx=0 tx=0).stop 2024-04-24T08:51:40.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.253+0000 7f15f5e00640 1 -- 172.21.15.83:0/4169014963 shutdown_connections 2024-04-24T08:51:40.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.253+0000 7f15f5e00640 1 --2- 172.21.15.83:0/4169014963 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f15f0105e30 0x7f15f010c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:40.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.253+0000 7f15f5e00640 1 --2- 172.21.15.83:0/4169014963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f15f0105470 0x7f15f01058f0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:40.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.253+0000 7f15f5e00640 1 --2- 172.21.15.83:0/4169014963 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f15f0104280 0x7f15f01046a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:40.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.253+0000 7f15f5e00640 1 -- 172.21.15.83:0/4169014963 >> 172.21.15.83:0/4169014963 conn(0x7f15f00ff7f0 msgr2=0x7f15f0101c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:40.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.253+0000 7f15f5e00640 1 -- 172.21.15.83:0/4169014963 shutdown_connections 2024-04-24T08:51:40.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.253+0000 7f15f5e00640 1 -- 172.21.15.83:0/4169014963 wait complete. 2024-04-24T08:51:40.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.254+0000 7f15f5e00640 1 Processor -- start 2024-04-24T08:51:40.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.255+0000 7f15f5e00640 1 -- start start 2024-04-24T08:51:40.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.255+0000 7f15f5e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f15f0104280 0x7f15f019a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:40.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.255+0000 7f15f5e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f15f0105470 0x7f15f019a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:40.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.255+0000 7f15f5e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f15f0105e30 0x7f15f01a1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:40.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.255+0000 7f15f5e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f15f010fe10 con 0x7f15f0105470 2024-04-24T08:51:40.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.255+0000 7f15f5e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f15f010fc90 con 0x7f15f0104280 2024-04-24T08:51:40.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.255+0000 7f15f5e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f15f010ff90 con 0x7f15f0105e30 2024-04-24T08:51:40.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.255+0000 7f15ee000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f15f0105470 0x7f15f019a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:40.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.255+0000 7f15ee000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f15f0105470 0x7f15f019a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55720/0 (socket says 172.21.15.83:55720) 2024-04-24T08:51:40.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.255+0000 7f15ee000640 1 -- 172.21.15.83:0/186215961 learned_addr learned my addr 172.21.15.83:0/186215961 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:51:40.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.255+0000 7f15ef400640 1 --2- 172.21.15.83:0/186215961 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f15f0105e30 0x7f15f01a1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:40.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.255+0000 7f15eea00640 1 --2- 172.21.15.83:0/186215961 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f15f0104280 0x7f15f019a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:40.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.256+0000 7f15ee000640 1 -- 172.21.15.83:0/186215961 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f15f0105e30 msgr2=0x7f15f01a1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:40.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.256+0000 7f15ee000640 1 --2- 172.21.15.83:0/186215961 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f15f0105e30 0x7f15f01a1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:40.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.256+0000 7f15ee000640 1 -- 172.21.15.83:0/186215961 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f15f0104280 msgr2=0x7f15f019a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:40.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.256+0000 7f15ee000640 1 --2- 172.21.15.83:0/186215961 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f15f0104280 0x7f15f019a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:40.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.256+0000 7f15ee000640 1 -- 172.21.15.83:0/186215961 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15f01a20b0 con 0x7f15f0105470 2024-04-24T08:51:40.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.256+0000 7f15eea00640 1 --2- 172.21.15.83:0/186215961 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f15f0104280 0x7f15f019a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:51:40.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.256+0000 7f15ee000640 1 --2- 172.21.15.83:0/186215961 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f15f0105470 0x7f15f019a9d0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f15d80314e0 tx=0x7f15d8003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:40.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.256+0000 7f15f4a00640 1 -- 172.21.15.83:0/186215961 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d803f070 con 0x7f15f0105470 2024-04-24T08:51:40.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.256+0000 7f15f4a00640 1 -- 172.21.15.83:0/186215961 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f15d8031c40 con 0x7f15f0105470 2024-04-24T08:51:40.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.256+0000 7f15f4a00640 1 -- 172.21.15.83:0/186215961 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d8035030 con 0x7f15f0105470 2024-04-24T08:51:40.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.256+0000 7f15f5e00640 1 -- 172.21.15.83:0/186215961 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15f01a2340 con 0x7f15f0105470 2024-04-24T08:51:40.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.256+0000 7f15f5e00640 1 -- 172.21.15.83:0/186215961 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15f01a27e0 con 0x7f15f0105470 2024-04-24T08:51:40.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.258+0000 7f15f5e00640 1 -- 172.21.15.83:0/186215961 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15f010b620 con 0x7f15f0105470 2024-04-24T08:51:40.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.260+0000 7f15f4a00640 1 -- 172.21.15.83:0/186215961 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 23) ==== 138766+0+0 (secure 0 0 0) 0x7f15d8042070 con 0x7f15f0105470 2024-04-24T08:51:40.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.261+0000 7f15f4a00640 1 --2- 172.21.15.83:0/186215961 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f15d40a3da0 0x7f15d40a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:40.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.261+0000 7f15f4a00640 1 -- 172.21.15.83:0/186215961 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f15d80f1400 con 0x7f15f0105470 2024-04-24T08:51:40.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.261+0000 7f15eea00640 1 --2- 172.21.15.83:0/186215961 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f15d40a3da0 0x7f15d40a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:40.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.262+0000 7f15eea00640 1 --2- 172.21.15.83:0/186215961 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f15d40a3da0 0x7f15d40a6260 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f15e00058c0 tx=0x7f15e0005850 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:40.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.265+0000 7f15f4a00640 1 -- 172.21.15.83:0/186215961 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f15d80bd670 con 0x7f15f0105470 2024-04-24T08:51:40.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:39 smithi083 ceph-mon[27907]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:51:40.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:39 smithi083 ceph-mon[27907]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-04-24T08:51:40.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:39 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/236693550' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T08:51:40.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:39 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/1254968157' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T08:51:40.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:39 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T08:51:40.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:39 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T08:51:40.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:39 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T08:51:40.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:39 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T08:51:40.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:39 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T08:51:40.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:39 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T08:51:40.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:39 smithi138 ceph-mon[31441]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:51:40.485 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:39 smithi138 ceph-mon[31441]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-04-24T08:51:40.485 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:39 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/236693550' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T08:51:40.485 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:39 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/1254968157' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T08:51:40.486 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:39 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T08:51:40.486 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:39 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T08:51:40.486 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:39 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T08:51:40.486 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:39 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T08:51:40.486 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:39 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T08:51:40.486 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:39 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T08:51:40.487 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:39 smithi189 ceph-mon[31791]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:51:40.487 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:39 smithi189 ceph-mon[31791]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-04-24T08:51:40.487 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:39 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/236693550' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T08:51:40.487 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:39 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/1254968157' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T08:51:40.487 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:39 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T08:51:40.487 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:39 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T08:51:40.487 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:39 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T08:51:40.487 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:39 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T08:51:40.488 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:39 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T08:51:40.488 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:39 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T08:51:40.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.478+0000 7f15f5e00640 1 -- 172.21.15.83:0/186215961 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f15f0062c00 con 0x7f15f0105470 2024-04-24T08:51:40.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.478+0000 7f15f4a00640 1 -- 172.21.15.83:0/186215961 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7f15d80bcf70 con 0x7f15f0105470 2024-04-24T08:51:40.488 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-24T08:51:40.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.481+0000 7f15f5e00640 1 -- 172.21.15.83:0/186215961 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f15d40a3da0 msgr2=0x7f15d40a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:40.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.481+0000 7f15f5e00640 1 --2- 172.21.15.83:0/186215961 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f15d40a3da0 0x7f15d40a6260 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f15e00058c0 tx=0x7f15e0005850 comp rx=0 tx=0).stop 2024-04-24T08:51:40.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.481+0000 7f15f5e00640 1 -- 172.21.15.83:0/186215961 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f15f0105470 msgr2=0x7f15f019a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:40.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.481+0000 7f15f5e00640 1 --2- 172.21.15.83:0/186215961 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f15f0105470 0x7f15f019a9d0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f15d80314e0 tx=0x7f15d8003c10 comp rx=0 tx=0).stop 2024-04-24T08:51:40.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.481+0000 7f15f5e00640 1 -- 172.21.15.83:0/186215961 shutdown_connections 2024-04-24T08:51:40.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.481+0000 7f15f5e00640 1 --2- 172.21.15.83:0/186215961 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f15d40a3da0 0x7f15d40a6260 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:40.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.481+0000 7f15f5e00640 1 --2- 172.21.15.83:0/186215961 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f15f0105e30 0x7f15f01a1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:40.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.481+0000 7f15f5e00640 1 --2- 172.21.15.83:0/186215961 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f15f0105470 0x7f15f019a9d0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:40.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.481+0000 7f15f5e00640 1 --2- 172.21.15.83:0/186215961 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f15f0104280 0x7f15f019a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:40.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.481+0000 7f15f5e00640 1 -- 172.21.15.83:0/186215961 >> 172.21.15.83:0/186215961 conn(0x7f15f00ff7f0 msgr2=0x7f15f01087f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:40.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.481+0000 7f15f5e00640 1 -- 172.21.15.83:0/186215961 shutdown_connections 2024-04-24T08:51:40.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.482+0000 7f15f5e00640 1 -- 172.21.15.83:0/186215961 wait complete. 2024-04-24T08:51:40.499 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T08:51:40.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.691+0000 7f40fe000640 1 -- 172.21.15.83:0/4058104770 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f40f8104140 msgr2=0x7f40f8104560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:40.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.691+0000 7f40fe000640 1 --2- 172.21.15.83:0/4058104770 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f40f8104140 0x7f40f8104560 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f40e8009900 tx=0x7f40e802a410 comp rx=0 tx=0).stop 2024-04-24T08:51:40.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.692+0000 7f40fe000640 1 -- 172.21.15.83:0/4058104770 shutdown_connections 2024-04-24T08:51:40.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.692+0000 7f40fe000640 1 --2- 172.21.15.83:0/4058104770 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f40f8105cf0 0x7f40f810c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:40.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.692+0000 7f40fe000640 1 --2- 172.21.15.83:0/4058104770 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f40f8105330 0x7f40f81057b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:40.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.692+0000 7f40fe000640 1 --2- 172.21.15.83:0/4058104770 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f40f8104140 0x7f40f8104560 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:40.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.692+0000 7f40fe000640 1 -- 172.21.15.83:0/4058104770 >> 172.21.15.83:0/4058104770 conn(0x7f40f80ff6b0 msgr2=0x7f40f8101af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:40.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.692+0000 7f40fe000640 1 -- 172.21.15.83:0/4058104770 shutdown_connections 2024-04-24T08:51:40.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.692+0000 7f40fe000640 1 -- 172.21.15.83:0/4058104770 wait complete. 2024-04-24T08:51:40.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.693+0000 7f40fe000640 1 Processor -- start 2024-04-24T08:51:40.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.693+0000 7f40fe000640 1 -- start start 2024-04-24T08:51:40.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.693+0000 7f40fe000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f40f8104140 0x7f40f819a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:40.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.693+0000 7f40fe000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f40f8105330 0x7f40f819a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:40.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.693+0000 7f40fe000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f40f8105cf0 0x7f40f81a1910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:40.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.693+0000 7f40fe000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f40f810fcd0 con 0x7f40f8105cf0 2024-04-24T08:51:40.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.693+0000 7f40fe000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f40f810fb50 con 0x7f40f8104140 2024-04-24T08:51:40.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.693+0000 7f40fe000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f40f810fe50 con 0x7f40f8105330 2024-04-24T08:51:40.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.693+0000 7f40fd600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f40f8105cf0 0x7f40f81a1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:40.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.693+0000 7f40fd600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f40f8105cf0 0x7f40f81a1910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55748/0 (socket says 172.21.15.83:55748) 2024-04-24T08:51:40.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.693+0000 7f40fd600640 1 -- 172.21.15.83:0/4134749279 learned_addr learned my addr 172.21.15.83:0/4134749279 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:51:40.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.694+0000 7f40f7e00640 1 --2- 172.21.15.83:0/4134749279 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f40f8105330 0x7f40f819a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:40.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.694+0000 7f40fcc00640 1 --2- 172.21.15.83:0/4134749279 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f40f8104140 0x7f40f819a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:40.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.694+0000 7f40fd600640 1 -- 172.21.15.83:0/4134749279 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f40f8105330 msgr2=0x7f40f819a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:40.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.694+0000 7f40fd600640 1 --2- 172.21.15.83:0/4134749279 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f40f8105330 0x7f40f819a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:40.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.694+0000 7f40fd600640 1 -- 172.21.15.83:0/4134749279 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f40f8104140 msgr2=0x7f40f819a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:40.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.694+0000 7f40fd600640 1 --2- 172.21.15.83:0/4134749279 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f40f8104140 0x7f40f819a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:40.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.694+0000 7f40fd600640 1 -- 172.21.15.83:0/4134749279 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40f81a1f80 con 0x7f40f8105cf0 2024-04-24T08:51:40.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.694+0000 7f40fcc00640 1 --2- 172.21.15.83:0/4134749279 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f40f8104140 0x7f40f819a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:51:40.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.694+0000 7f40fd600640 1 --2- 172.21.15.83:0/4134749279 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f40f8105cf0 0x7f40f81a1910 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f40f000eaa0 tx=0x7f40f000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:40.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.694+0000 7f40f5600640 1 -- 172.21.15.83:0/4134749279 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f000cdb0 con 0x7f40f8105cf0 2024-04-24T08:51:40.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.694+0000 7f40f5600640 1 -- 172.21.15.83:0/4134749279 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f40f0005020 con 0x7f40f8105cf0 2024-04-24T08:51:40.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.694+0000 7f40fe000640 1 -- 172.21.15.83:0/4134749279 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40f81a2270 con 0x7f40f8105cf0 2024-04-24T08:51:40.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.694+0000 7f40f5600640 1 -- 172.21.15.83:0/4134749279 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f00116d0 con 0x7f40f8105cf0 2024-04-24T08:51:40.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.695+0000 7f40fe000640 1 -- 172.21.15.83:0/4134749279 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40f81a2770 con 0x7f40f8105cf0 2024-04-24T08:51:40.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.696+0000 7f40fe000640 1 -- 172.21.15.83:0/4134749279 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40f810b500 con 0x7f40f8105cf0 2024-04-24T08:51:40.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.698+0000 7f40f5600640 1 -- 172.21.15.83:0/4134749279 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 23) ==== 138766+0+0 (secure 0 0 0) 0x7f40f00051d0 con 0x7f40f8105cf0 2024-04-24T08:51:40.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.700+0000 7f40f5600640 1 --2- 172.21.15.83:0/4134749279 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f40dc0a3d20 0x7f40dc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:40.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.700+0000 7f40f5600640 1 -- 172.21.15.83:0/4134749279 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f40f00d0dc0 con 0x7f40f8105cf0 2024-04-24T08:51:40.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.700+0000 7f40fcc00640 1 --2- 172.21.15.83:0/4134749279 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f40dc0a3d20 0x7f40dc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:40.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.701+0000 7f40fcc00640 1 --2- 172.21.15.83:0/4134749279 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f40dc0a3d20 0x7f40dc0a61e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f40e8006010 tx=0x7f40e8000bc0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:40.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.704+0000 7f40f5600640 1 -- 172.21.15.83:0/4134749279 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f40f0016030 con 0x7f40f8105cf0 2024-04-24T08:51:40.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.911+0000 7f40fe000640 1 -- 172.21.15.83:0/4134749279 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f40f8062c00 con 0x7f40f8105cf0 2024-04-24T08:51:40.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.912+0000 7f40f5600640 1 -- 172.21.15.83:0/4134749279 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7f40f009d090 con 0x7f40f8105cf0 2024-04-24T08:51:40.911 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2024-04-24T08:51:40.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.914+0000 7f40fe000640 1 -- 172.21.15.83:0/4134749279 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f40dc0a3d20 msgr2=0x7f40dc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:40.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.914+0000 7f40fe000640 1 --2- 172.21.15.83:0/4134749279 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f40dc0a3d20 0x7f40dc0a61e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f40e8006010 tx=0x7f40e8000bc0 comp rx=0 tx=0).stop 2024-04-24T08:51:40.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.914+0000 7f40fe000640 1 -- 172.21.15.83:0/4134749279 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f40f8105cf0 msgr2=0x7f40f81a1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:40.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.914+0000 7f40fe000640 1 --2- 172.21.15.83:0/4134749279 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f40f8105cf0 0x7f40f81a1910 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f40f000eaa0 tx=0x7f40f000ef60 comp rx=0 tx=0).stop 2024-04-24T08:51:40.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.914+0000 7f40fe000640 1 -- 172.21.15.83:0/4134749279 shutdown_connections 2024-04-24T08:51:40.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.914+0000 7f40fe000640 1 --2- 172.21.15.83:0/4134749279 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f40dc0a3d20 0x7f40dc0a61e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:40.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.914+0000 7f40fe000640 1 --2- 172.21.15.83:0/4134749279 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f40f8105cf0 0x7f40f81a1910 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:40.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.914+0000 7f40fe000640 1 --2- 172.21.15.83:0/4134749279 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f40f8105330 0x7f40f819a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:40.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.914+0000 7f40fe000640 1 --2- 172.21.15.83:0/4134749279 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f40f8104140 0x7f40f819a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:40.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.914+0000 7f40fe000640 1 -- 172.21.15.83:0/4134749279 >> 172.21.15.83:0/4134749279 conn(0x7f40f80ff6b0 msgr2=0x7f40f81086d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:40.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.915+0000 7f40fe000640 1 -- 172.21.15.83:0/4134749279 shutdown_connections 2024-04-24T08:51:40.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:40.915+0000 7f40fe000640 1 -- 172.21.15.83:0/4134749279 wait complete. 2024-04-24T08:51:40.931 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-24T08:51:41.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:41.140+0000 7fe744200640 1 -- 172.21.15.83:0/736452583 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe73c077c00 msgr2=0x7fe73c078060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:41.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:41.140+0000 7fe744200640 1 --2- 172.21.15.83:0/736452583 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe73c077c00 0x7fe73c078060 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fe728009900 tx=0x7fe72802a3f0 comp rx=0 tx=0).stop 2024-04-24T08:51:41.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:41.140+0000 7fe744200640 1 -- 172.21.15.83:0/736452583 shutdown_connections 2024-04-24T08:51:41.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:41.140+0000 7fe744200640 1 --2- 172.21.15.83:0/736452583 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe73c0785a0 0x7fe73c078e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:41.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:41.140+0000 7fe744200640 1 --2- 172.21.15.83:0/736452583 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe73c077c00 0x7fe73c078060 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:41.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:41.140+0000 7fe744200640 1 --2- 172.21.15.83:0/736452583 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe73c0769c0 0x7fe73c076de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:41.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:41.140+0000 7fe744200640 1 -- 172.21.15.83:0/736452583 >> 172.21.15.83:0/736452583 conn(0x7fe73c0ff7f0 msgr2=0x7fe73c101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:41.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:41.140+0000 7fe744200640 1 -- 172.21.15.83:0/736452583 shutdown_connections 2024-04-24T08:51:41.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:41.140+0000 7fe744200640 1 -- 172.21.15.83:0/736452583 wait complete. 2024-04-24T08:51:41.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:41.141+0000 7fe744200640 1 Processor -- start 2024-04-24T08:51:41.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:41.141+0000 7fe744200640 1 -- start start 2024-04-24T08:51:41.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:41.141+0000 7fe744200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe73c0769c0 0x7fe73c19e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:41.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:41.141+0000 7fe744200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe73c077c00 0x7fe73c19edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:41.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:41.141+0000 7fe744200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe73c0785a0 0x7fe73c1a5e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:41.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:41.141+0000 7fe744200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe73c1141e0 con 0x7fe73c077c00 2024-04-24T08:51:41.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:41.141+0000 7fe744200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe73c114060 con 0x7fe73c0769c0 2024-04-24T08:51:41.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:41.141+0000 7fe744200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe73c114360 con 0x7fe73c0785a0 2024-04-24T08:51:41.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:41.142+0000 7fe740e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe73c077c00 0x7fe73c19edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:41.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:41.142+0000 7fe741800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe73c0769c0 0x7fe73c19e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:41.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:41.142+0000 7fe740e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe73c077c00 0x7fe73c19edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55774/0 (socket says 172.21.15.83:55774) 2024-04-24T08:51:41.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:41.142+0000 7fe740e00640 1 -- 172.21.15.83:0/2082720260 learned_addr learned my addr 172.21.15.83:0/2082720260 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:51:41.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:41.142+0000 7fe742200640 1 --2- 172.21.15.83:0/2082720260 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe73c0785a0 0x7fe73c1a5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:41.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:41.142+0000 7fe740e00640 1 -- 172.21.15.83:0/2082720260 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe73c0785a0 msgr2=0x7fe73c1a5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:41.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:41.142+0000 7fe740e00640 1 --2- 172.21.15.83:0/2082720260 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe73c0785a0 0x7fe73c1a5e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:41.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:41.142+0000 7fe740e00640 1 -- 172.21.15.83:0/2082720260 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe73c0769c0 msgr2=0x7fe73c19e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:41.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:41.142+0000 7fe740e00640 1 --2- 172.21.15.83:0/2082720260 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe73c0769c0 0x7fe73c19e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:41.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:41.142+0000 7fe740e00640 1 -- 172.21.15.83:0/2082720260 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe73c1a64d0 con 0x7fe73c077c00 2024-04-24T08:51:41.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:41.143+0000 7fe740e00640 1 --2- 172.21.15.83:0/2082720260 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe73c077c00 0x7fe73c19edf0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fe7280314e0 tx=0x7fe728003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:41.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:41.143+0000 7fe73a000640 1 -- 172.21.15.83:0/2082720260 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe72803f070 con 0x7fe73c077c00 2024-04-24T08:51:41.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:41.143+0000 7fe73a000640 1 -- 172.21.15.83:0/2082720260 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe728031c40 con 0x7fe73c077c00 2024-04-24T08:51:41.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:41.143+0000 7fe73a000640 1 -- 172.21.15.83:0/2082720260 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe728035030 con 0x7fe73c077c00 2024-04-24T08:51:41.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:41.143+0000 7fe744200640 1 -- 172.21.15.83:0/2082720260 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe73c1a6760 con 0x7fe73c077c00 2024-04-24T08:51:41.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:41.143+0000 7fe744200640 1 -- 172.21.15.83:0/2082720260 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe73c1a6c00 con 0x7fe73c077c00 2024-04-24T08:51:41.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:41.145+0000 7fe744200640 1 -- 172.21.15.83:0/2082720260 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe73c110bd0 con 0x7fe73c077c00 2024-04-24T08:51:41.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:41.148+0000 7fe73a000640 1 -- 172.21.15.83:0/2082720260 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7fe728042070 con 0x7fe73c077c00 2024-04-24T08:51:41.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:41.151+0000 7fe73a000640 1 --2- 172.21.15.83:0/2082720260 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe7240a3e30 0x7fe7240a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:41.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:41.152+0000 7fe73a000640 1 -- 172.21.15.83:0/2082720260 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7fe7280f1a60 con 0x7fe73c077c00 2024-04-24T08:51:41.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:41.152+0000 7fe741800640 1 --2- 172.21.15.83:0/2082720260 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe7240a3e30 0x7fe7240a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:41.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:41.152+0000 7fe73a000640 1 -- 172.21.15.83:0/2082720260 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe7280bd570 con 0x7fe73c077c00 2024-04-24T08:51:41.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:41.153+0000 7fe741800640 1 --2- 172.21.15.83:0/2082720260 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe7240a3e30 0x7fe7240a62f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe730009bd0 tx=0x7fe730005f90 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:41.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:40 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/2629580367' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T08:51:41.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:40 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/186215961' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T08:51:41.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:40 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/4134749279' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T08:51:41.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:41.357+0000 7fe744200640 1 -- 172.21.15.83:0/2082720260 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fe73c02d7b0 con 0x7fe73c077c00 2024-04-24T08:51:41.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:41.358+0000 7fe73a000640 1 -- 172.21.15.83:0/2082720260 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) ==== 103+0+11 (secure 0 0 0) 0x7fe7280c1870 con 0x7fe73c077c00 2024-04-24T08:51:41.357 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-24T08:51:41.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:41.361+0000 7fe744200640 1 -- 172.21.15.83:0/2082720260 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe7240a3e30 msgr2=0x7fe7240a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:41.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:41.361+0000 7fe744200640 1 --2- 172.21.15.83:0/2082720260 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe7240a3e30 0x7fe7240a62f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe730009bd0 tx=0x7fe730005f90 comp rx=0 tx=0).stop 2024-04-24T08:51:41.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:41.361+0000 7fe744200640 1 -- 172.21.15.83:0/2082720260 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe73c077c00 msgr2=0x7fe73c19edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:41.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:41.361+0000 7fe744200640 1 --2- 172.21.15.83:0/2082720260 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe73c077c00 0x7fe73c19edf0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fe7280314e0 tx=0x7fe728003c10 comp rx=0 tx=0).stop 2024-04-24T08:51:41.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:41.361+0000 7fe744200640 1 -- 172.21.15.83:0/2082720260 shutdown_connections 2024-04-24T08:51:41.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:41.361+0000 7fe744200640 1 --2- 172.21.15.83:0/2082720260 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe7240a3e30 0x7fe7240a62f0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:41.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:41.361+0000 7fe744200640 1 --2- 172.21.15.83:0/2082720260 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe73c0785a0 0x7fe73c1a5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:41.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:41.361+0000 7fe744200640 1 --2- 172.21.15.83:0/2082720260 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe73c077c00 0x7fe73c19edf0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:41.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:41.361+0000 7fe744200640 1 --2- 172.21.15.83:0/2082720260 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe73c0769c0 0x7fe73c19e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:41.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:41.361+0000 7fe744200640 1 -- 172.21.15.83:0/2082720260 >> 172.21.15.83:0/2082720260 conn(0x7fe73c0ff7f0 msgr2=0x7fe73c102b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:41.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:41.361+0000 7fe744200640 1 -- 172.21.15.83:0/2082720260 shutdown_connections 2024-04-24T08:51:41.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:41.361+0000 7fe744200640 1 -- 172.21.15.83:0/2082720260 wait complete. 2024-04-24T08:51:41.379 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-04-24T08:51:41.379 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T08:51:41.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:40 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/2629580367' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T08:51:41.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:40 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/186215961' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T08:51:41.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:40 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/4134749279' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T08:51:41.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:40 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/2629580367' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T08:51:41.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:40 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/186215961' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T08:51:41.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:40 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/4134749279' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T08:51:41.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:41.572+0000 7f67abe00640 1 -- 172.21.15.83:0/3287365267 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67ac102900 msgr2=0x7f67ac104cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:41.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:41.572+0000 7f67abe00640 1 --2- 172.21.15.83:0/3287365267 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67ac102900 0x7f67ac104cf0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f679c009900 tx=0x7f679c02a3f0 comp rx=0 tx=0).stop 2024-04-24T08:51:41.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:41.572+0000 7f67abe00640 1 -- 172.21.15.83:0/3287365267 shutdown_connections 2024-04-24T08:51:41.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:41.572+0000 7f67abe00640 1 --2- 172.21.15.83:0/3287365267 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f67ac107b60 0x7f67ac109fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:41.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:41.572+0000 7f67abe00640 1 --2- 172.21.15.83:0/3287365267 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f67ac105230 0x7f67ac107620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:41.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:41.572+0000 7f67abe00640 1 --2- 172.21.15.83:0/3287365267 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67ac102900 0x7f67ac104cf0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:41.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:41.572+0000 7f67abe00640 1 -- 172.21.15.83:0/3287365267 >> 172.21.15.83:0/3287365267 conn(0x7f67ac0fc2d0 msgr2=0x7f67ac0fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:41.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:41.572+0000 7f67abe00640 1 -- 172.21.15.83:0/3287365267 shutdown_connections 2024-04-24T08:51:41.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:41.572+0000 7f67abe00640 1 -- 172.21.15.83:0/3287365267 wait complete. 2024-04-24T08:51:41.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:41.573+0000 7f67abe00640 1 Processor -- start 2024-04-24T08:51:41.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:41.573+0000 7f67abe00640 1 -- start start 2024-04-24T08:51:41.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:41.573+0000 7f67abe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f67ac102900 0x7f67ac19a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:41.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:41.573+0000 7f67abe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67ac105230 0x7f67ac19a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:41.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:41.573+0000 7f67abe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f67ac107b60 0x7f67ac1a1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:41.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:41.573+0000 7f67abe00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f67ac10da20 con 0x7f67ac105230 2024-04-24T08:51:41.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:41.573+0000 7f67abe00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f67ac10d8a0 con 0x7f67ac107b60 2024-04-24T08:51:41.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:41.573+0000 7f67abe00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f67ac10dba0 con 0x7f67ac102900 2024-04-24T08:51:41.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:41.574+0000 7f67aa000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67ac105230 0x7f67ac19a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:41.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:41.574+0000 7f67aa000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67ac105230 0x7f67ac19a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55782/0 (socket says 172.21.15.83:55782) 2024-04-24T08:51:41.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:41.574+0000 7f67aa000640 1 -- 172.21.15.83:0/3076566445 learned_addr learned my addr 172.21.15.83:0/3076566445 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:51:41.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:41.574+0000 7f67aaa00640 1 --2- 172.21.15.83:0/3076566445 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f67ac102900 0x7f67ac19a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:41.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:41.574+0000 7f67ab400640 1 --2- 172.21.15.83:0/3076566445 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f67ac107b60 0x7f67ac1a1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:41.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:41.574+0000 7f67aa000640 1 -- 172.21.15.83:0/3076566445 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f67ac102900 msgr2=0x7f67ac19a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:41.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:41.574+0000 7f67aa000640 1 --2- 172.21.15.83:0/3076566445 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f67ac102900 0x7f67ac19a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:41.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:41.574+0000 7f67aa000640 1 -- 172.21.15.83:0/3076566445 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f67ac107b60 msgr2=0x7f67ac1a1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:41.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:41.574+0000 7f67aa000640 1 --2- 172.21.15.83:0/3076566445 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f67ac107b60 0x7f67ac1a1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:41.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:41.574+0000 7f67aa000640 1 -- 172.21.15.83:0/3076566445 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67ac1a1ed0 con 0x7f67ac105230 2024-04-24T08:51:41.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:41.575+0000 7f67aa000640 1 --2- 172.21.15.83:0/3076566445 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67ac105230 0x7f67ac19a7f0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f679800d770 tx=0x7f679800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:41.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:41.575+0000 7f67a3400640 1 -- 172.21.15.83:0/3076566445 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6798017070 con 0x7f67ac105230 2024-04-24T08:51:41.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:41.575+0000 7f67a3400640 1 -- 172.21.15.83:0/3076566445 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6798009c30 con 0x7f67ac105230 2024-04-24T08:51:41.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:41.575+0000 7f67abe00640 1 -- 172.21.15.83:0/3076566445 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67ac1a21c0 con 0x7f67ac105230 2024-04-24T08:51:41.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:41.575+0000 7f67a3400640 1 -- 172.21.15.83:0/3076566445 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f679800f620 con 0x7f67ac105230 2024-04-24T08:51:41.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:41.575+0000 7f67abe00640 1 -- 172.21.15.83:0/3076566445 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67ac1a2710 con 0x7f67ac105230 2024-04-24T08:51:41.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:41.576+0000 7f67abe00640 1 -- 172.21.15.83:0/3076566445 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67ac1096b0 con 0x7f67ac105230 2024-04-24T08:51:41.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:41.579+0000 7f67a3400640 1 -- 172.21.15.83:0/3076566445 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f679801a070 con 0x7f67ac105230 2024-04-24T08:51:41.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:41.580+0000 7f67a3400640 1 --2- 172.21.15.83:0/3076566445 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f67900a3db0 0x7f67900a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:41.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:41.581+0000 7f67a3400640 1 -- 172.21.15.83:0/3076566445 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f67980d0840 con 0x7f67ac105230 2024-04-24T08:51:41.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:41.581+0000 7f67aaa00640 1 --2- 172.21.15.83:0/3076566445 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f67900a3db0 0x7f67900a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:41.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:41.582+0000 7f67aaa00640 1 --2- 172.21.15.83:0/3076566445 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f67900a3db0 0x7f67900a6270 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f679c0314e0 tx=0x7f679c005a20 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:41.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:41.586+0000 7f67a3400640 1 -- 172.21.15.83:0/3076566445 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6798009de0 con 0x7f67ac105230 2024-04-24T08:51:41.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:41.841+0000 7f67abe00640 1 -- 172.21.15.83:0/3076566445 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f67ac02d870 con 0x7f67ac105230 2024-04-24T08:51:41.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:41.843+0000 7f67a3400640 1 -- 172.21.15.83:0/3076566445 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11944 (secure 0 0 0) 0x7f679809c2f0 con 0x7f67ac105230 2024-04-24T08:51:41.843 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:51:41.843 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/358872257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":358872257},{"type":"v1","addr":"172.21.15.83:6835","nonce":358872257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3157070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3157070851},{"type":"v1","addr":"172.21.15.83:6837","nonce":3157070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T08:48:29.422576+0000","modified":"2024-04-24T08:51:36.355488+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34520,"mds_1":34493,"mds_2":34451,"mds_3":34427,"mds_4":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14658":{"gid":14658,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6839/4085470844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":4085470844},{"type":"v1","addr":"172.21.15.83:6839","nonce":4085470844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14676":{"gid":14676,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6841/2634754210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":2634754210},{"type":"v1","addr":"172.21.15.83:6841","nonce":2634754210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24382":{"gid":24382,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/1859921310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1859921310},{"type":"v1","addr":"172.21.15.189:6833","nonce":1859921310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/2241923197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2241923197},{"type":"v1","addr":"172.21.15.189:6835","nonce":2241923197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6837/3172311220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":3172311220},{"type":"v1","addr":"172.21.15.189:6837","nonce":3172311220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.189:6839/2548965336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":2548965336},{"type":"v1","addr":"172.21.15.189:6839","nonce":2548965336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.138:6833/1157967658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1157967658},{"type":"v1","addr":"172.21.15.138:6833","nonce":1157967658}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.138:6835/1909219758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1909219758},{"type":"v1","addr":"172.21.15.138:6835","nonce":1909219758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34493":{"gid":34493,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.138:6837/900566754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":900566754},{"type":"v1","addr":"172.21.15.138:6837","nonce":900566754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34520":{"gid":34520,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.138:6839/2236932393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2236932393},{"type":"v1","addr":"172.21.15.138:6839","nonce":2236932393}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34520,"qdb_cluster":[34451,34493,34427,34520,24454]},"id":1}]} 2024-04-24T08:51:41.844 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 28 2024-04-24T08:51:41.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:41.846+0000 7f67abe00640 1 -- 172.21.15.83:0/3076566445 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f67900a3db0 msgr2=0x7f67900a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:41.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:41.846+0000 7f67abe00640 1 --2- 172.21.15.83:0/3076566445 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f67900a3db0 0x7f67900a6270 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f679c0314e0 tx=0x7f679c005a20 comp rx=0 tx=0).stop 2024-04-24T08:51:41.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:41.846+0000 7f67abe00640 1 -- 172.21.15.83:0/3076566445 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67ac105230 msgr2=0x7f67ac19a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:41.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:41.846+0000 7f67abe00640 1 --2- 172.21.15.83:0/3076566445 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67ac105230 0x7f67ac19a7f0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f679800d770 tx=0x7f679800dc30 comp rx=0 tx=0).stop 2024-04-24T08:51:41.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:41.847+0000 7f67abe00640 1 -- 172.21.15.83:0/3076566445 shutdown_connections 2024-04-24T08:51:41.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:41.847+0000 7f67abe00640 1 --2- 172.21.15.83:0/3076566445 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f67900a3db0 0x7f67900a6270 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:41.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:41.847+0000 7f67abe00640 1 --2- 172.21.15.83:0/3076566445 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f67ac107b60 0x7f67ac1a1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:41.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:41.847+0000 7f67abe00640 1 --2- 172.21.15.83:0/3076566445 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67ac105230 0x7f67ac19a7f0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:41.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:41.847+0000 7f67abe00640 1 --2- 172.21.15.83:0/3076566445 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f67ac102900 0x7f67ac19a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:41.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:41.847+0000 7f67abe00640 1 -- 172.21.15.83:0/3076566445 >> 172.21.15.83:0/3076566445 conn(0x7f67ac0fc2d0 msgr2=0x7f67ac101fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:41.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:41.847+0000 7f67abe00640 1 -- 172.21.15.83:0/3076566445 shutdown_connections 2024-04-24T08:51:41.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:41.847+0000 7f67abe00640 1 -- 172.21.15.83:0/3076566445 wait complete. 2024-04-24T08:51:41.866 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-04-24T08:51:41.866 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-04-24T08:51:41.866 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T08:51:42.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.040+0000 7f9ef8e00640 1 -- 172.21.15.83:0/3376202227 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ef410e620 msgr2=0x7f9ef4110a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:42.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.040+0000 7f9ef8e00640 1 --2- 172.21.15.83:0/3376202227 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ef410e620 0x7f9ef4110a10 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f9eec009cd0 tx=0x7f9eec02a8e0 comp rx=0 tx=0).stop 2024-04-24T08:51:42.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.040+0000 7f9ef8e00640 1 -- 172.21.15.83:0/3376202227 shutdown_connections 2024-04-24T08:51:42.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.040+0000 7f9ef8e00640 1 --2- 172.21.15.83:0/3376202227 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ef410e620 0x7f9ef4110a10 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:42.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.040+0000 7f9ef8e00640 1 --2- 172.21.15.83:0/3376202227 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9ef4074a60 0x7f9ef4074ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:42.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.040+0000 7f9ef8e00640 1 --2- 172.21.15.83:0/3376202227 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9ef4076040 0x7f9ef4074490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:42.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.040+0000 7f9ef8e00640 1 -- 172.21.15.83:0/3376202227 >> 172.21.15.83:0/3376202227 conn(0x7f9ef40fd5e0 msgr2=0x7f9ef40ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:42.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.040+0000 7f9ef8e00640 1 -- 172.21.15.83:0/3376202227 shutdown_connections 2024-04-24T08:51:42.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.040+0000 7f9ef8e00640 1 -- 172.21.15.83:0/3376202227 wait complete. 2024-04-24T08:51:42.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.041+0000 7f9ef8e00640 1 Processor -- start 2024-04-24T08:51:42.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.041+0000 7f9ef8e00640 1 -- start start 2024-04-24T08:51:42.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.041+0000 7f9ef8e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9ef4074a60 0x7f9ef419e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:42.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.041+0000 7f9ef8e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ef4076040 0x7f9ef419eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:42.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.041+0000 7f9ef8e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9ef410e620 0x7f9ef41a5e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:42.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.041+0000 7f9ef8e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef41141c0 con 0x7f9ef4076040 2024-04-24T08:51:42.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.041+0000 7f9ef8e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef4114040 con 0x7f9ef410e620 2024-04-24T08:51:42.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.041+0000 7f9ef8e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef4114340 con 0x7f9ef4074a60 2024-04-24T08:51:42.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.042+0000 7f9ef1400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ef4076040 0x7f9ef419eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:42.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.042+0000 7f9ef1e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9ef4074a60 0x7f9ef419e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:42.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.042+0000 7f9ef1400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ef4076040 0x7f9ef419eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55802/0 (socket says 172.21.15.83:55802) 2024-04-24T08:51:42.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.042+0000 7f9ef1400640 1 -- 172.21.15.83:0/2563760206 learned_addr learned my addr 172.21.15.83:0/2563760206 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:51:42.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.042+0000 7f9ef2800640 1 --2- 172.21.15.83:0/2563760206 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9ef410e620 0x7f9ef41a5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:42.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.042+0000 7f9ef1e00640 1 -- 172.21.15.83:0/2563760206 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9ef410e620 msgr2=0x7f9ef41a5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:42.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.042+0000 7f9ef1e00640 1 --2- 172.21.15.83:0/2563760206 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9ef410e620 0x7f9ef41a5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:42.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.042+0000 7f9ef1e00640 1 -- 172.21.15.83:0/2563760206 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ef4076040 msgr2=0x7f9ef419eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:42.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.042+0000 7f9ef1e00640 1 --2- 172.21.15.83:0/2563760206 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ef4076040 0x7f9ef419eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:42.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.042+0000 7f9ef1e00640 1 -- 172.21.15.83:0/2563760206 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ef41a6480 con 0x7f9ef4074a60 2024-04-24T08:51:42.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.042+0000 7f9ef1e00640 1 --2- 172.21.15.83:0/2563760206 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9ef4074a60 0x7f9ef419e860 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9ee400e9f0 tx=0x7f9ee400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:42.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.043+0000 7f9eeaa00640 1 -- 172.21.15.83:0/2563760206 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ee400ccc0 con 0x7f9ef4074a60 2024-04-24T08:51:42.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.043+0000 7f9eeaa00640 1 -- 172.21.15.83:0/2563760206 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9ee4005020 con 0x7f9ef4074a60 2024-04-24T08:51:42.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.043+0000 7f9ef8e00640 1 -- 172.21.15.83:0/2563760206 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ef41a6770 con 0x7f9ef4074a60 2024-04-24T08:51:42.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.043+0000 7f9eeaa00640 1 -- 172.21.15.83:0/2563760206 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ee4004a00 con 0x7f9ef4074a60 2024-04-24T08:51:42.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.043+0000 7f9ef8e00640 1 -- 172.21.15.83:0/2563760206 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ef41a6cc0 con 0x7f9ef4074a60 2024-04-24T08:51:42.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.044+0000 7f9ef8e00640 1 -- 172.21.15.83:0/2563760206 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ef41046e0 con 0x7f9ef4074a60 2024-04-24T08:51:42.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.046+0000 7f9eeaa00640 1 -- 172.21.15.83:0/2563760206 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f9ee4004ba0 con 0x7f9ef4074a60 2024-04-24T08:51:42.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.049+0000 7f9eeaa00640 1 --2- 172.21.15.83:0/2563760206 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f9ed80a3e30 0x7f9ed80a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:42.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.049+0000 7f9eeaa00640 1 -- 172.21.15.83:0/2563760206 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f9ee40d0d00 con 0x7f9ef4074a60 2024-04-24T08:51:42.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.049+0000 7f9ef1400640 1 --2- 172.21.15.83:0/2563760206 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f9ed80a3e30 0x7f9ed80a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:42.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.050+0000 7f9ef1400640 1 --2- 172.21.15.83:0/2563760206 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f9ed80a3e30 0x7f9ed80a62f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9ef419fd70 tx=0x7f9ee000b4a0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:42.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.053+0000 7f9eeaa00640 1 -- 172.21.15.83:0/2563760206 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9ee409ce00 con 0x7f9ef4074a60 2024-04-24T08:51:42.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.261+0000 7f9ef8e00640 1 -- 172.21.15.83:0/2563760206 --> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9ef4062440 con 0x7f9ed80a3e30 2024-04-24T08:51:42.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.274+0000 7f9eeaa00640 1 -- 172.21.15.83:0/2563760206 <== mgr.15579 v2:172.21.15.83:6800/3594769126 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339543 (secure 0 0 0) 0x7f9ef4062440 con 0x7f9ed80a3e30 2024-04-24T08:51:42.278 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:51:42.280 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-04-24T08:51:42.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.278+0000 7f9ef8e00640 1 -- 172.21.15.83:0/2563760206 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f9ed80a3e30 msgr2=0x7f9ed80a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:42.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.278+0000 7f9ef8e00640 1 --2- 172.21.15.83:0/2563760206 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f9ed80a3e30 0x7f9ed80a62f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9ef419fd70 tx=0x7f9ee000b4a0 comp rx=0 tx=0).stop 2024-04-24T08:51:42.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.278+0000 7f9ef8e00640 1 -- 172.21.15.83:0/2563760206 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9ef4074a60 msgr2=0x7f9ef419e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:42.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.278+0000 7f9ef8e00640 1 --2- 172.21.15.83:0/2563760206 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9ef4074a60 0x7f9ef419e860 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9ee400e9f0 tx=0x7f9ee400eeb0 comp rx=0 tx=0).stop 2024-04-24T08:51:42.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.278+0000 7f9ef8e00640 1 -- 172.21.15.83:0/2563760206 shutdown_connections 2024-04-24T08:51:42.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.278+0000 7f9ef8e00640 1 --2- 172.21.15.83:0/2563760206 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f9ed80a3e30 0x7f9ed80a62f0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:42.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.278+0000 7f9ef8e00640 1 --2- 172.21.15.83:0/2563760206 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9ef410e620 0x7f9ef41a5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:42.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.278+0000 7f9ef8e00640 1 --2- 172.21.15.83:0/2563760206 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ef4076040 0x7f9ef419eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:42.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.278+0000 7f9ef8e00640 1 --2- 172.21.15.83:0/2563760206 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9ef4074a60 0x7f9ef419e860 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:42.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.279+0000 7f9ef8e00640 1 -- 172.21.15.83:0/2563760206 >> 172.21.15.83:0/2563760206 conn(0x7f9ef40fd5e0 msgr2=0x7f9ef41018b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:42.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.279+0000 7f9ef8e00640 1 -- 172.21.15.83:0/2563760206 shutdown_connections 2024-04-24T08:51:42.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.279+0000 7f9ef8e00640 1 -- 172.21.15.83:0/2563760206 wait complete. 2024-04-24T08:51:42.294 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:41 smithi083 ceph-mon[27907]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:51:42.294 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:41 smithi083 ceph-mon[27907]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-04-24T08:51:42.294 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:41 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/2082720260' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T08:51:42.295 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:41 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T08:51:42.295 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:41 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T08:51:42.295 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:41 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:51:42.295 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:41 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:51:42.295 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:41 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:51:42.295 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:41 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:51:42.295 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:41 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T08:51:42.295 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:41 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T08:51:42.295 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:41 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T08:51:42.295 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:41 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:51:42.295 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:41 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T08:51:42.295 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:41 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:51:42.295 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:41 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:51:42.296 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:41 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:51:42.296 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:41 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:51:42.296 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:41 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/3076566445' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:51:42.296 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:41 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:51:42.296 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:41 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:51:42.296 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:41 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:51:42.296 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:41 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:51:42.296 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:41 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:51:42.297 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-04-24T08:51:40.381360+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1192897,"num_objects":104,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1800,"num_read_kb":1061,"num_write":569,"num_write_kb":4415,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":1180870,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":949,"ondisk_log_size":949,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":385836,"kb_used_data":15732,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124555988,"statfs":{"total":1151940427776,"available":1151545331712,"internally_reserved":0,"allocated":16109568,"data_stored":9278423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":378713401},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.970609"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.701720+0000","last_change":"2024-04-24T08:48:29.993009+0000","last_active":"2024-04-24T08:51:36.701720+0000","last_peered":"2024-04-24T08:51:36.701720+0000","last_clean":"2024-04-24T08:51:36.701720+0000","last_became_active":"2024-04-24T08:48:29.990613+0000","last_became_peered":"2024-04-24T08:48:29.990613+0000","last_unstale":"2024-04-24T08:51:36.701720+0000","last_undegraded":"2024-04-24T08:51:36.701720+0000","last_fullsized":"2024-04-24T08:51:36.701720+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:13:43.871742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.893581+0000","last_change":"2024-04-24T08:48:29.986957+0000","last_active":"2024-04-24T08:51:36.893581+0000","last_peered":"2024-04-24T08:51:36.893581+0000","last_clean":"2024-04-24T08:51:36.893581+0000","last_became_active":"2024-04-24T08:48:29.984691+0000","last_became_peered":"2024-04-24T08:48:29.984691+0000","last_unstale":"2024-04-24T08:51:36.893581+0000","last_undegraded":"2024-04-24T08:51:36.893581+0000","last_fullsized":"2024-04-24T08:51:36.893581+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:07:38.031213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.893523+0000","last_change":"2024-04-24T08:48:29.991478+0000","last_active":"2024-04-24T08:51:36.893523+0000","last_peered":"2024-04-24T08:51:36.893523+0000","last_clean":"2024-04-24T08:51:36.893523+0000","last_became_active":"2024-04-24T08:48:29.991013+0000","last_became_peered":"2024-04-24T08:48:29.991013+0000","last_unstale":"2024-04-24T08:51:36.893523+0000","last_undegraded":"2024-04-24T08:51:36.893523+0000","last_fullsized":"2024-04-24T08:51:36.893523+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:10:39.381353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.738223+0000","last_change":"2024-04-24T08:48:29.983664+0000","last_active":"2024-04-24T08:51:36.738223+0000","last_peered":"2024-04-24T08:51:36.738223+0000","last_clean":"2024-04-24T08:51:36.738223+0000","last_became_active":"2024-04-24T08:48:29.978835+0000","last_became_peered":"2024-04-24T08:48:29.978835+0000","last_unstale":"2024-04-24T08:51:36.738223+0000","last_undegraded":"2024-04-24T08:51:36.738223+0000","last_fullsized":"2024-04-24T08:51:36.738223+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:29:20.501652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.503590+0000","last_change":"2024-04-24T08:48:29.989886+0000","last_active":"2024-04-24T08:51:36.503590+0000","last_peered":"2024-04-24T08:51:36.503590+0000","last_clean":"2024-04-24T08:51:36.503590+0000","last_became_active":"2024-04-24T08:48:29.989574+0000","last_became_peered":"2024-04-24T08:48:29.989574+0000","last_unstale":"2024-04-24T08:51:36.503590+0000","last_undegraded":"2024-04-24T08:51:36.503590+0000","last_fullsized":"2024-04-24T08:51:36.503590+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:14:01.085543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.368540+0000","last_change":"2024-04-24T08:48:29.978045+0000","last_active":"2024-04-24T08:51:36.368540+0000","last_peered":"2024-04-24T08:51:36.368540+0000","last_clean":"2024-04-24T08:51:36.368540+0000","last_became_active":"2024-04-24T08:48:29.977729+0000","last_became_peered":"2024-04-24T08:48:29.977729+0000","last_unstale":"2024-04-24T08:51:36.368540+0000","last_undegraded":"2024-04-24T08:51:36.368540+0000","last_fullsized":"2024-04-24T08:51:36.368540+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:03:33.382473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.657826+0000","last_change":"2024-04-24T08:48:29.976355+0000","last_active":"2024-04-24T08:51:36.657826+0000","last_peered":"2024-04-24T08:51:36.657826+0000","last_clean":"2024-04-24T08:51:36.657826+0000","last_became_active":"2024-04-24T08:48:29.975849+0000","last_became_peered":"2024-04-24T08:48:29.975849+0000","last_unstale":"2024-04-24T08:51:36.657826+0000","last_undegraded":"2024-04-24T08:51:36.657826+0000","last_fullsized":"2024-04-24T08:51:36.657826+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:13:51.302995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.359785+0000","last_change":"2024-04-24T08:48:29.983149+0000","last_active":"2024-04-24T08:51:36.359785+0000","last_peered":"2024-04-24T08:51:36.359785+0000","last_clean":"2024-04-24T08:51:36.359785+0000","last_became_active":"2024-04-24T08:48:29.982692+0000","last_became_peered":"2024-04-24T08:48:29.982692+0000","last_unstale":"2024-04-24T08:51:36.359785+0000","last_undegraded":"2024-04-24T08:51:36.359785+0000","last_fullsized":"2024-04-24T08:51:36.359785+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:59:06.151441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.893672+0000","last_change":"2024-04-24T08:48:29.985531+0000","last_active":"2024-04-24T08:51:36.893672+0000","last_peered":"2024-04-24T08:51:36.893672+0000","last_clean":"2024-04-24T08:51:36.893672+0000","last_became_active":"2024-04-24T08:48:29.982484+0000","last_became_peered":"2024-04-24T08:48:29.982484+0000","last_unstale":"2024-04-24T08:51:36.893672+0000","last_undegraded":"2024-04-24T08:51:36.893672+0000","last_fullsized":"2024-04-24T08:51:36.893672+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:03:50.114096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":47,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.368859+0000","last_change":"2024-04-24T08:48:29.982016+0000","last_active":"2024-04-24T08:51:36.368859+0000","last_peered":"2024-04-24T08:51:36.368859+0000","last_clean":"2024-04-24T08:51:36.368859+0000","last_became_active":"2024-04-24T08:48:29.977639+0000","last_became_peered":"2024-04-24T08:48:29.977639+0000","last_unstale":"2024-04-24T08:51:36.368859+0000","last_undegraded":"2024-04-24T08:51:36.368859+0000","last_fullsized":"2024-04-24T08:51:36.368859+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:35:48.676822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.659055+0000","last_change":"2024-04-24T08:48:29.977982+0000","last_active":"2024-04-24T08:51:36.659055+0000","last_peered":"2024-04-24T08:51:36.659055+0000","last_clean":"2024-04-24T08:51:36.659055+0000","last_became_active":"2024-04-24T08:48:29.977664+0000","last_became_peered":"2024-04-24T08:48:29.977664+0000","last_unstale":"2024-04-24T08:51:36.659055+0000","last_undegraded":"2024-04-24T08:51:36.659055+0000","last_fullsized":"2024-04-24T08:51:36.659055+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:45:17.597291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.738302+0000","last_change":"2024-04-24T08:48:29.981615+0000","last_active":"2024-04-24T08:51:36.738302+0000","last_peered":"2024-04-24T08:51:36.738302+0000","last_clean":"2024-04-24T08:51:36.738302+0000","last_became_active":"2024-04-24T08:48:29.979292+0000","last_became_peered":"2024-04-24T08:48:29.979292+0000","last_unstale":"2024-04-24T08:51:36.738302+0000","last_undegraded":"2024-04-24T08:51:36.738302+0000","last_fullsized":"2024-04-24T08:51:36.738302+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:48:44.195626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.730843+0000","last_change":"2024-04-24T08:48:29.952632+0000","last_active":"2024-04-24T08:51:36.730843+0000","last_peered":"2024-04-24T08:51:36.730843+0000","last_clean":"2024-04-24T08:51:36.730843+0000","last_became_active":"2024-04-24T08:48:29.951862+0000","last_became_peered":"2024-04-24T08:48:29.951862+0000","last_unstale":"2024-04-24T08:51:36.730843+0000","last_undegraded":"2024-04-24T08:51:36.730843+0000","last_fullsized":"2024-04-24T08:51:36.730843+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:23:40.038517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.893175+0000","last_change":"2024-04-24T08:48:29.984727+0000","last_active":"2024-04-24T08:51:36.893175+0000","last_peered":"2024-04-24T08:51:36.893175+0000","last_clean":"2024-04-24T08:51:36.893175+0000","last_became_active":"2024-04-24T08:48:29.983530+0000","last_became_peered":"2024-04-24T08:48:29.983530+0000","last_unstale":"2024-04-24T08:51:36.893175+0000","last_undegraded":"2024-04-24T08:51:36.893175+0000","last_fullsized":"2024-04-24T08:51:36.893175+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:00:12.729972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.893421+0000","last_change":"2024-04-24T08:48:29.981023+0000","last_active":"2024-04-24T08:51:36.893421+0000","last_peered":"2024-04-24T08:51:36.893421+0000","last_clean":"2024-04-24T08:51:36.893421+0000","last_became_active":"2024-04-24T08:48:29.979986+0000","last_became_peered":"2024-04-24T08:48:29.979986+0000","last_unstale":"2024-04-24T08:51:36.893421+0000","last_undegraded":"2024-04-24T08:51:36.893421+0000","last_fullsized":"2024-04-24T08:51:36.893421+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:05:36.163209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.892674+0000","last_change":"2024-04-24T08:48:29.989036+0000","last_active":"2024-04-24T08:51:36.892674+0000","last_peered":"2024-04-24T08:51:36.892674+0000","last_clean":"2024-04-24T08:51:36.892674+0000","last_became_active":"2024-04-24T08:48:29.988676+0000","last_became_peered":"2024-04-24T08:48:29.988676+0000","last_unstale":"2024-04-24T08:51:36.892674+0000","last_undegraded":"2024-04-24T08:51:36.892674+0000","last_fullsized":"2024-04-24T08:51:36.892674+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:03:38.544596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.702193+0000","last_change":"2024-04-24T08:48:29.986424+0000","last_active":"2024-04-24T08:51:36.702193+0000","last_peered":"2024-04-24T08:51:36.702193+0000","last_clean":"2024-04-24T08:51:36.702193+0000","last_became_active":"2024-04-24T08:48:29.985973+0000","last_became_peered":"2024-04-24T08:48:29.985973+0000","last_unstale":"2024-04-24T08:51:36.702193+0000","last_undegraded":"2024-04-24T08:51:36.702193+0000","last_fullsized":"2024-04-24T08:51:36.702193+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:48:31.533282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.498447+0000","last_change":"2024-04-24T08:48:29.975470+0000","last_active":"2024-04-24T08:51:36.498447+0000","last_peered":"2024-04-24T08:51:36.498447+0000","last_clean":"2024-04-24T08:51:36.498447+0000","last_became_active":"2024-04-24T08:48:29.974777+0000","last_became_peered":"2024-04-24T08:48:29.974777+0000","last_unstale":"2024-04-24T08:51:36.498447+0000","last_undegraded":"2024-04-24T08:51:36.498447+0000","last_fullsized":"2024-04-24T08:51:36.498447+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:04:26.423840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.368791+0000","last_change":"2024-04-24T08:48:29.988001+0000","last_active":"2024-04-24T08:51:36.368791+0000","last_peered":"2024-04-24T08:51:36.368791+0000","last_clean":"2024-04-24T08:51:36.368791+0000","last_became_active":"2024-04-24T08:48:29.987669+0000","last_became_peered":"2024-04-24T08:48:29.987669+0000","last_unstale":"2024-04-24T08:51:36.368791+0000","last_undegraded":"2024-04-24T08:51:36.368791+0000","last_fullsized":"2024-04-24T08:51:36.368791+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:29:24.457369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.367087+0000","last_change":"2024-04-24T08:48:34.308732+0000","last_active":"2024-04-24T08:51:36.367087+0000","last_peered":"2024-04-24T08:51:36.367087+0000","last_clean":"2024-04-24T08:51:36.367087+0000","last_became_active":"2024-04-24T08:48:34.307598+0000","last_became_peered":"2024-04-24T08:48:34.307598+0000","last_unstale":"2024-04-24T08:51:36.367087+0000","last_undegraded":"2024-04-24T08:51:36.367087+0000","last_fullsized":"2024-04-24T08:51:36.367087+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:28:57.260836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"93'14","reported_seq":201,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.499394+0000","last_change":"2024-04-24T08:48:28.970230+0000","last_active":"2024-04-24T08:51:36.499394+0000","last_peered":"2024-04-24T08:51:36.499394+0000","last_clean":"2024-04-24T08:51:36.499394+0000","last_became_active":"2024-04-24T08:48:28.969878+0000","last_became_peered":"2024-04-24T08:48:28.969878+0000","last_unstale":"2024-04-24T08:51:36.499394+0000","last_undegraded":"2024-04-24T08:51:36.499394+0000","last_fullsized":"2024-04-24T08:51:36.499394+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:05:19.206557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":154,"num_read_kb":154,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.360088+0000","last_change":"2024-04-24T08:48:29.986350+0000","last_active":"2024-04-24T08:51:36.360088+0000","last_peered":"2024-04-24T08:51:36.360088+0000","last_clean":"2024-04-24T08:51:36.360088+0000","last_became_active":"2024-04-24T08:48:29.985167+0000","last_became_peered":"2024-04-24T08:48:29.985167+0000","last_unstale":"2024-04-24T08:51:36.360088+0000","last_undegraded":"2024-04-24T08:51:36.360088+0000","last_fullsized":"2024-04-24T08:51:36.360088+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:43:33.326808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.736357+0000","last_change":"2024-04-24T08:48:34.315159+0000","last_active":"2024-04-24T08:51:36.736357+0000","last_peered":"2024-04-24T08:51:36.736357+0000","last_clean":"2024-04-24T08:51:36.736357+0000","last_became_active":"2024-04-24T08:48:34.312119+0000","last_became_peered":"2024-04-24T08:48:34.312119+0000","last_unstale":"2024-04-24T08:51:36.736357+0000","last_undegraded":"2024-04-24T08:51:36.736357+0000","last_fullsized":"2024-04-24T08:51:36.736357+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:56:08.024123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.360224+0000","last_change":"2024-04-24T08:48:28.971787+0000","last_active":"2024-04-24T08:51:36.360224+0000","last_peered":"2024-04-24T08:51:36.360224+0000","last_clean":"2024-04-24T08:51:36.360224+0000","last_became_active":"2024-04-24T08:48:28.971479+0000","last_became_peered":"2024-04-24T08:48:28.971479+0000","last_unstale":"2024-04-24T08:51:36.360224+0000","last_undegraded":"2024-04-24T08:51:36.360224+0000","last_fullsized":"2024-04-24T08:51:36.360224+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:09:52.498833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.893290+0000","last_change":"2024-04-24T08:48:29.989737+0000","last_active":"2024-04-24T08:51:36.893290+0000","last_peered":"2024-04-24T08:51:36.893290+0000","last_clean":"2024-04-24T08:51:36.893290+0000","last_became_active":"2024-04-24T08:48:29.989341+0000","last_became_peered":"2024-04-24T08:48:29.989341+0000","last_unstale":"2024-04-24T08:51:36.893290+0000","last_undegraded":"2024-04-24T08:51:36.893290+0000","last_fullsized":"2024-04-24T08:51:36.893290+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:02:32.714848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.736714+0000","last_change":"2024-04-24T08:48:34.322666+0000","last_active":"2024-04-24T08:51:36.736714+0000","last_peered":"2024-04-24T08:51:36.736714+0000","last_clean":"2024-04-24T08:51:36.736714+0000","last_became_active":"2024-04-24T08:48:34.321546+0000","last_became_peered":"2024-04-24T08:48:34.321546+0000","last_unstale":"2024-04-24T08:51:36.736714+0000","last_undegraded":"2024-04-24T08:51:36.736714+0000","last_fullsized":"2024-04-24T08:51:36.736714+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:42:45.066043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"94'13","reported_seq":70,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.505002+0000","last_change":"2024-04-24T08:50:43.151013+0000","last_active":"2024-04-24T08:51:36.505002+0000","last_peered":"2024-04-24T08:51:36.505002+0000","last_clean":"2024-04-24T08:51:36.505002+0000","last_became_active":"2024-04-24T08:50:43.123152+0000","last_became_peered":"2024-04-24T08:50:43.123152+0000","last_unstale":"2024-04-24T08:51:36.505002+0000","last_undegraded":"2024-04-24T08:51:36.505002+0000","last_fullsized":"2024-04-24T08:51:36.505002+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:56:45.260680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,7],"acting":[8,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.893187+0000","last_change":"2024-04-24T08:48:29.974588+0000","last_active":"2024-04-24T08:51:36.893187+0000","last_peered":"2024-04-24T08:51:36.893187+0000","last_clean":"2024-04-24T08:51:36.893187+0000","last_became_active":"2024-04-24T08:48:29.974190+0000","last_became_peered":"2024-04-24T08:48:29.974190+0000","last_unstale":"2024-04-24T08:51:36.893187+0000","last_undegraded":"2024-04-24T08:51:36.893187+0000","last_fullsized":"2024-04-24T08:51:36.893187+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:07:12.948660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.736446+0000","last_change":"2024-04-24T08:48:34.321954+0000","last_active":"2024-04-24T08:51:36.736446+0000","last_peered":"2024-04-24T08:51:36.736446+0000","last_clean":"2024-04-24T08:51:36.736446+0000","last_became_active":"2024-04-24T08:48:34.321417+0000","last_became_peered":"2024-04-24T08:48:34.321417+0000","last_unstale":"2024-04-24T08:51:36.736446+0000","last_undegraded":"2024-04-24T08:51:36.736446+0000","last_fullsized":"2024-04-24T08:51:36.736446+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:39:07.205709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.504769+0000","last_change":"2024-04-24T08:48:28.974464+0000","last_active":"2024-04-24T08:51:36.504769+0000","last_peered":"2024-04-24T08:51:36.504769+0000","last_clean":"2024-04-24T08:51:36.504769+0000","last_became_active":"2024-04-24T08:48:28.974001+0000","last_became_peered":"2024-04-24T08:48:28.974001+0000","last_unstale":"2024-04-24T08:51:36.504769+0000","last_undegraded":"2024-04-24T08:51:36.504769+0000","last_fullsized":"2024-04-24T08:51:36.504769+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:38:55.193427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":47,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.514535+0000","last_change":"2024-04-24T08:48:29.981336+0000","last_active":"2024-04-24T08:51:36.514535+0000","last_peered":"2024-04-24T08:51:36.514535+0000","last_clean":"2024-04-24T08:51:36.514535+0000","last_became_active":"2024-04-24T08:48:29.980333+0000","last_became_peered":"2024-04-24T08:48:29.980333+0000","last_unstale":"2024-04-24T08:51:36.514535+0000","last_undegraded":"2024-04-24T08:51:36.514535+0000","last_fullsized":"2024-04-24T08:51:36.514535+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:46:03.640332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.368326+0000","last_change":"2024-04-24T08:48:34.323099+0000","last_active":"2024-04-24T08:51:36.368326+0000","last_peered":"2024-04-24T08:51:36.368326+0000","last_clean":"2024-04-24T08:51:36.368326+0000","last_became_active":"2024-04-24T08:48:34.322557+0000","last_became_peered":"2024-04-24T08:48:34.322557+0000","last_unstale":"2024-04-24T08:51:36.368326+0000","last_undegraded":"2024-04-24T08:51:36.368326+0000","last_fullsized":"2024-04-24T08:51:36.368326+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:59:59.887087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"94'9","reported_seq":200,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.699273+0000","last_change":"2024-04-24T08:48:28.978427+0000","last_active":"2024-04-24T08:51:36.699273+0000","last_peered":"2024-04-24T08:51:36.699273+0000","last_clean":"2024-04-24T08:51:36.699273+0000","last_became_active":"2024-04-24T08:48:28.977291+0000","last_became_peered":"2024-04-24T08:48:28.977291+0000","last_unstale":"2024-04-24T08:51:36.699273+0000","last_undegraded":"2024-04-24T08:51:36.699273+0000","last_fullsized":"2024-04-24T08:51:36.699273+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:03:03.085265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2038,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":157,"num_read_kb":4,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.503358+0000","last_change":"2024-04-24T08:48:29.978008+0000","last_active":"2024-04-24T08:51:36.503358+0000","last_peered":"2024-04-24T08:51:36.503358+0000","last_clean":"2024-04-24T08:51:36.503358+0000","last_became_active":"2024-04-24T08:48:29.977389+0000","last_became_peered":"2024-04-24T08:48:29.977389+0000","last_unstale":"2024-04-24T08:51:36.503358+0000","last_undegraded":"2024-04-24T08:51:36.503358+0000","last_fullsized":"2024-04-24T08:51:36.503358+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:35:28.035378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.368135+0000","last_change":"2024-04-24T08:48:34.321477+0000","last_active":"2024-04-24T08:51:36.368135+0000","last_peered":"2024-04-24T08:51:36.368135+0000","last_clean":"2024-04-24T08:51:36.368135+0000","last_became_active":"2024-04-24T08:48:34.320860+0000","last_became_peered":"2024-04-24T08:48:34.320860+0000","last_unstale":"2024-04-24T08:51:36.368135+0000","last_undegraded":"2024-04-24T08:51:36.368135+0000","last_fullsized":"2024-04-24T08:51:36.368135+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:26:07.862590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":54,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.368151+0000","last_change":"2024-04-24T08:50:43.014290+0000","last_active":"2024-04-24T08:51:36.368151+0000","last_peered":"2024-04-24T08:51:36.368151+0000","last_clean":"2024-04-24T08:51:36.368151+0000","last_became_active":"2024-04-24T08:50:42.995733+0000","last_became_peered":"2024-04-24T08:50:42.995733+0000","last_unstale":"2024-04-24T08:51:36.368151+0000","last_undegraded":"2024-04-24T08:51:36.368151+0000","last_fullsized":"2024-04-24T08:51:36.368151+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:54:50.270110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.514617+0000","last_change":"2024-04-24T08:48:29.990788+0000","last_active":"2024-04-24T08:51:36.514617+0000","last_peered":"2024-04-24T08:51:36.514617+0000","last_clean":"2024-04-24T08:51:36.514617+0000","last_became_active":"2024-04-24T08:48:29.988476+0000","last_became_peered":"2024-04-24T08:48:29.988476+0000","last_unstale":"2024-04-24T08:51:36.514617+0000","last_undegraded":"2024-04-24T08:51:36.514617+0000","last_fullsized":"2024-04-24T08:51:36.514617+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:08:48.389877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.701568+0000","last_change":"2024-04-24T08:48:34.311020+0000","last_active":"2024-04-24T08:51:36.701568+0000","last_peered":"2024-04-24T08:51:36.701568+0000","last_clean":"2024-04-24T08:51:36.701568+0000","last_became_active":"2024-04-24T08:48:34.310395+0000","last_became_peered":"2024-04-24T08:48:34.310395+0000","last_unstale":"2024-04-24T08:51:36.701568+0000","last_undegraded":"2024-04-24T08:51:36.701568+0000","last_fullsized":"2024-04-24T08:51:36.701568+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:55:38.704954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.729835+0000","last_change":"2024-04-24T08:48:28.969991+0000","last_active":"2024-04-24T08:51:36.729835+0000","last_peered":"2024-04-24T08:51:36.729835+0000","last_clean":"2024-04-24T08:51:36.729835+0000","last_became_active":"2024-04-24T08:48:28.967937+0000","last_became_peered":"2024-04-24T08:48:28.967937+0000","last_unstale":"2024-04-24T08:51:36.729835+0000","last_undegraded":"2024-04-24T08:51:36.729835+0000","last_fullsized":"2024-04-24T08:51:36.729835+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:09:49.736108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.359593+0000","last_change":"2024-04-24T08:48:29.992814+0000","last_active":"2024-04-24T08:51:36.359593+0000","last_peered":"2024-04-24T08:51:36.359593+0000","last_clean":"2024-04-24T08:51:36.359593+0000","last_became_active":"2024-04-24T08:48:29.992188+0000","last_became_peered":"2024-04-24T08:48:29.992188+0000","last_unstale":"2024-04-24T08:51:36.359593+0000","last_undegraded":"2024-04-24T08:51:36.359593+0000","last_fullsized":"2024-04-24T08:51:36.359593+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:29:33.974958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.893584+0000","last_change":"2024-04-24T08:48:34.311196+0000","last_active":"2024-04-24T08:51:36.893584+0000","last_peered":"2024-04-24T08:51:36.893584+0000","last_clean":"2024-04-24T08:51:36.893584+0000","last_became_active":"2024-04-24T08:48:34.310723+0000","last_became_peered":"2024-04-24T08:48:34.310723+0000","last_unstale":"2024-04-24T08:51:36.893584+0000","last_undegraded":"2024-04-24T08:51:36.893584+0000","last_fullsized":"2024-04-24T08:51:36.893584+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:04:30.149510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.513501+0000","last_change":"2024-04-24T08:48:28.978534+0000","last_active":"2024-04-24T08:51:36.513501+0000","last_peered":"2024-04-24T08:51:36.513501+0000","last_clean":"2024-04-24T08:51:36.513501+0000","last_became_active":"2024-04-24T08:48:28.975407+0000","last_became_peered":"2024-04-24T08:48:28.975407+0000","last_unstale":"2024-04-24T08:51:36.513501+0000","last_undegraded":"2024-04-24T08:51:36.513501+0000","last_fullsized":"2024-04-24T08:51:36.513501+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:10:45.355039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.497595+0000","last_change":"2024-04-24T08:48:29.997682+0000","last_active":"2024-04-24T08:51:36.497595+0000","last_peered":"2024-04-24T08:51:36.497595+0000","last_clean":"2024-04-24T08:51:36.497595+0000","last_became_active":"2024-04-24T08:48:29.997223+0000","last_became_peered":"2024-04-24T08:48:29.997223+0000","last_unstale":"2024-04-24T08:51:36.497595+0000","last_undegraded":"2024-04-24T08:51:36.497595+0000","last_fullsized":"2024-04-24T08:51:36.497595+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:03:00.807249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.359842+0000","last_change":"2024-04-24T08:48:34.321477+0000","last_active":"2024-04-24T08:51:36.359842+0000","last_peered":"2024-04-24T08:51:36.359842+0000","last_clean":"2024-04-24T08:51:36.359842+0000","last_became_active":"2024-04-24T08:48:34.321018+0000","last_became_peered":"2024-04-24T08:48:34.321018+0000","last_unstale":"2024-04-24T08:51:36.359842+0000","last_undegraded":"2024-04-24T08:51:36.359842+0000","last_fullsized":"2024-04-24T08:51:36.359842+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:56:50.657298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"93'7","reported_seq":346,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.680385+0000","last_change":"2024-04-24T08:48:28.983722+0000","last_active":"2024-04-24T08:51:36.680385+0000","last_peered":"2024-04-24T08:51:36.680385+0000","last_clean":"2024-04-24T08:51:36.680385+0000","last_became_active":"2024-04-24T08:48:28.983394+0000","last_became_peered":"2024-04-24T08:48:28.983394+0000","last_unstale":"2024-04-24T08:51:36.680385+0000","last_undegraded":"2024-04-24T08:51:36.680385+0000","last_fullsized":"2024-04-24T08:51:36.680385+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:53:14.288742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":306,"num_read_kb":306,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.893545+0000","last_change":"2024-04-24T08:48:29.975009+0000","last_active":"2024-04-24T08:51:36.893545+0000","last_peered":"2024-04-24T08:51:36.893545+0000","last_clean":"2024-04-24T08:51:36.893545+0000","last_became_active":"2024-04-24T08:48:29.974137+0000","last_became_peered":"2024-04-24T08:48:29.974137+0000","last_unstale":"2024-04-24T08:51:36.893545+0000","last_undegraded":"2024-04-24T08:51:36.893545+0000","last_fullsized":"2024-04-24T08:51:36.893545+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:13:16.381923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.497502+0000","last_change":"2024-04-24T08:48:34.312126+0000","last_active":"2024-04-24T08:51:36.497502+0000","last_peered":"2024-04-24T08:51:36.497502+0000","last_clean":"2024-04-24T08:51:36.497502+0000","last_became_active":"2024-04-24T08:48:34.311689+0000","last_became_peered":"2024-04-24T08:48:34.311689+0000","last_unstale":"2024-04-24T08:51:36.497502+0000","last_undegraded":"2024-04-24T08:51:36.497502+0000","last_fullsized":"2024-04-24T08:51:36.497502+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:11:07.942412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.659196+0000","last_change":"2024-04-24T08:48:28.978355+0000","last_active":"2024-04-24T08:51:36.659196+0000","last_peered":"2024-04-24T08:51:36.659196+0000","last_clean":"2024-04-24T08:51:36.659196+0000","last_became_active":"2024-04-24T08:48:28.977351+0000","last_became_peered":"2024-04-24T08:48:28.977351+0000","last_unstale":"2024-04-24T08:51:36.659196+0000","last_undegraded":"2024-04-24T08:51:36.659196+0000","last_fullsized":"2024-04-24T08:51:36.659196+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:00:12.836569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.893780+0000","last_change":"2024-04-24T08:48:29.989015+0000","last_active":"2024-04-24T08:51:36.893780+0000","last_peered":"2024-04-24T08:51:36.893780+0000","last_clean":"2024-04-24T08:51:36.893780+0000","last_became_active":"2024-04-24T08:48:29.988648+0000","last_became_peered":"2024-04-24T08:48:29.988648+0000","last_unstale":"2024-04-24T08:51:36.893780+0000","last_undegraded":"2024-04-24T08:51:36.893780+0000","last_fullsized":"2024-04-24T08:51:36.893780+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:04:52.824894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.737315+0000","last_change":"2024-04-24T08:48:34.314230+0000","last_active":"2024-04-24T08:51:36.737315+0000","last_peered":"2024-04-24T08:51:36.737315+0000","last_clean":"2024-04-24T08:51:36.737315+0000","last_became_active":"2024-04-24T08:48:34.313617+0000","last_became_peered":"2024-04-24T08:48:34.313617+0000","last_unstale":"2024-04-24T08:51:36.737315+0000","last_undegraded":"2024-04-24T08:51:36.737315+0000","last_fullsized":"2024-04-24T08:51:36.737315+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:06:02.307575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.514177+0000","last_change":"2024-04-24T08:48:28.974516+0000","last_active":"2024-04-24T08:51:36.514177+0000","last_peered":"2024-04-24T08:51:36.514177+0000","last_clean":"2024-04-24T08:51:36.514177+0000","last_became_active":"2024-04-24T08:48:28.974194+0000","last_became_peered":"2024-04-24T08:48:28.974194+0000","last_unstale":"2024-04-24T08:51:36.514177+0000","last_undegraded":"2024-04-24T08:51:36.514177+0000","last_fullsized":"2024-04-24T08:51:36.514177+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:03:38.582610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":48,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.893356+0000","last_change":"2024-04-24T08:49:42.128553+0000","last_active":"2024-04-24T08:51:36.893356+0000","last_peered":"2024-04-24T08:51:36.893356+0000","last_clean":"2024-04-24T08:51:36.893356+0000","last_became_active":"2024-04-24T08:49:42.127061+0000","last_became_peered":"2024-04-24T08:49:42.127061+0000","last_unstale":"2024-04-24T08:51:36.893356+0000","last_undegraded":"2024-04-24T08:51:36.893356+0000","last_fullsized":"2024-04-24T08:51:36.893356+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:07:06.798284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.367516+0000","last_change":"2024-04-24T08:49:42.131305+0000","last_active":"2024-04-24T08:51:36.367516+0000","last_peered":"2024-04-24T08:51:36.367516+0000","last_clean":"2024-04-24T08:51:36.367516+0000","last_became_active":"2024-04-24T08:49:42.129680+0000","last_became_peered":"2024-04-24T08:49:42.129680+0000","last_unstale":"2024-04-24T08:51:36.367516+0000","last_undegraded":"2024-04-24T08:51:36.367516+0000","last_fullsized":"2024-04-24T08:51:36.367516+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:14:09.579649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"94'40","reported_seq":387,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.894275+0000","last_change":"2024-04-24T08:48:28.981675+0000","last_active":"2024-04-24T08:51:36.894275+0000","last_peered":"2024-04-24T08:51:36.894275+0000","last_clean":"2024-04-24T08:51:36.894275+0000","last_became_active":"2024-04-24T08:48:28.981082+0000","last_became_peered":"2024-04-24T08:48:28.981082+0000","last_unstale":"2024-04-24T08:51:36.894275+0000","last_undegraded":"2024-04-24T08:51:36.894275+0000","last_fullsized":"2024-04-24T08:51:36.894275+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:22:07.826011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66989,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":314,"num_read_kb":217,"num_write":42,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.504369+0000","last_change":"2024-04-24T08:48:29.986932+0000","last_active":"2024-04-24T08:51:36.504369+0000","last_peered":"2024-04-24T08:51:36.504369+0000","last_clean":"2024-04-24T08:51:36.504369+0000","last_became_active":"2024-04-24T08:48:29.986505+0000","last_became_peered":"2024-04-24T08:48:29.986505+0000","last_unstale":"2024-04-24T08:51:36.504369+0000","last_undegraded":"2024-04-24T08:51:36.504369+0000","last_fullsized":"2024-04-24T08:51:36.504369+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:53:36.050294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.515187+0000","last_change":"2024-04-24T08:48:34.327127+0000","last_active":"2024-04-24T08:51:36.515187+0000","last_peered":"2024-04-24T08:51:36.515187+0000","last_clean":"2024-04-24T08:51:36.515187+0000","last_became_active":"2024-04-24T08:48:34.323942+0000","last_became_peered":"2024-04-24T08:48:34.323942+0000","last_unstale":"2024-04-24T08:51:36.515187+0000","last_undegraded":"2024-04-24T08:51:36.515187+0000","last_fullsized":"2024-04-24T08:51:36.515187+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:22:49.637513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.367615+0000","last_change":"2024-04-24T08:48:28.978618+0000","last_active":"2024-04-24T08:51:36.367615+0000","last_peered":"2024-04-24T08:51:36.367615+0000","last_clean":"2024-04-24T08:51:36.367615+0000","last_became_active":"2024-04-24T08:48:28.978309+0000","last_became_peered":"2024-04-24T08:48:28.978309+0000","last_unstale":"2024-04-24T08:51:36.367615+0000","last_undegraded":"2024-04-24T08:51:36.367615+0000","last_fullsized":"2024-04-24T08:51:36.367615+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:35:50.091398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.729979+0000","last_change":"2024-04-24T08:48:29.985629+0000","last_active":"2024-04-24T08:51:36.729979+0000","last_peered":"2024-04-24T08:51:36.729979+0000","last_clean":"2024-04-24T08:51:36.729979+0000","last_became_active":"2024-04-24T08:48:29.985258+0000","last_became_peered":"2024-04-24T08:48:29.985258+0000","last_unstale":"2024-04-24T08:51:36.729979+0000","last_undegraded":"2024-04-24T08:51:36.729979+0000","last_fullsized":"2024-04-24T08:51:36.729979+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:57:29.302175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.701030+0000","last_change":"2024-04-24T08:49:42.129242+0000","last_active":"2024-04-24T08:51:36.701030+0000","last_peered":"2024-04-24T08:51:36.701030+0000","last_clean":"2024-04-24T08:51:36.701030+0000","last_became_active":"2024-04-24T08:49:42.127967+0000","last_became_peered":"2024-04-24T08:49:42.127967+0000","last_unstale":"2024-04-24T08:51:36.701030+0000","last_undegraded":"2024-04-24T08:51:36.701030+0000","last_fullsized":"2024-04-24T08:51:36.701030+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:42:11.206123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.513770+0000","last_change":"2024-04-24T08:48:28.977840+0000","last_active":"2024-04-24T08:51:36.513770+0000","last_peered":"2024-04-24T08:51:36.513770+0000","last_clean":"2024-04-24T08:51:36.513770+0000","last_became_active":"2024-04-24T08:48:28.975482+0000","last_became_peered":"2024-04-24T08:48:28.975482+0000","last_unstale":"2024-04-24T08:51:36.513770+0000","last_undegraded":"2024-04-24T08:51:36.513770+0000","last_fullsized":"2024-04-24T08:51:36.513770+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:51:52.468627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":52,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.658628+0000","last_change":"2024-04-24T08:49:42.129823+0000","last_active":"2024-04-24T08:51:36.658628+0000","last_peered":"2024-04-24T08:51:36.658628+0000","last_clean":"2024-04-24T08:51:36.658628+0000","last_became_active":"2024-04-24T08:49:42.127254+0000","last_became_peered":"2024-04-24T08:49:42.127254+0000","last_unstale":"2024-04-24T08:51:36.658628+0000","last_undegraded":"2024-04-24T08:51:36.658628+0000","last_fullsized":"2024-04-24T08:51:36.658628+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:51:10.804703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.499181+0000","last_change":"2024-04-24T08:48:34.314956+0000","last_active":"2024-04-24T08:51:36.499181+0000","last_peered":"2024-04-24T08:51:36.499181+0000","last_clean":"2024-04-24T08:51:36.499181+0000","last_became_active":"2024-04-24T08:48:34.314541+0000","last_became_peered":"2024-04-24T08:48:34.314541+0000","last_unstale":"2024-04-24T08:51:36.499181+0000","last_undegraded":"2024-04-24T08:51:36.499181+0000","last_fullsized":"2024-04-24T08:51:36.499181+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:42:48.614000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"93'6","reported_seq":196,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.737247+0000","last_change":"2024-04-24T08:48:28.974529+0000","last_active":"2024-04-24T08:51:36.737247+0000","last_peered":"2024-04-24T08:51:36.737247+0000","last_clean":"2024-04-24T08:51:36.737247+0000","last_became_active":"2024-04-24T08:48:28.973957+0000","last_became_peered":"2024-04-24T08:48:28.973957+0000","last_unstale":"2024-04-24T08:51:36.737247+0000","last_undegraded":"2024-04-24T08:51:36.737247+0000","last_fullsized":"2024-04-24T08:51:36.737247+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:44:46.216389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1081,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":157,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.658134+0000","last_change":"2024-04-24T08:48:29.981995+0000","last_active":"2024-04-24T08:51:36.658134+0000","last_peered":"2024-04-24T08:51:36.658134+0000","last_clean":"2024-04-24T08:51:36.658134+0000","last_became_active":"2024-04-24T08:48:29.980756+0000","last_became_peered":"2024-04-24T08:48:29.980756+0000","last_unstale":"2024-04-24T08:51:36.658134+0000","last_undegraded":"2024-04-24T08:51:36.658134+0000","last_fullsized":"2024-04-24T08:51:36.658134+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:42:52.770069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.731767+0000","last_change":"2024-04-24T08:48:34.308836+0000","last_active":"2024-04-24T08:51:36.731767+0000","last_peered":"2024-04-24T08:51:36.731767+0000","last_clean":"2024-04-24T08:51:36.731767+0000","last_became_active":"2024-04-24T08:48:34.308470+0000","last_became_peered":"2024-04-24T08:48:34.308470+0000","last_unstale":"2024-04-24T08:51:36.731767+0000","last_undegraded":"2024-04-24T08:51:36.731767+0000","last_fullsized":"2024-04-24T08:51:36.731767+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:38:14.906117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":76,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.503278+0000","last_change":"2024-04-24T08:50:43.053334+0000","last_active":"2024-04-24T08:51:36.503278+0000","last_peered":"2024-04-24T08:51:36.503278+0000","last_clean":"2024-04-24T08:51:36.503278+0000","last_became_active":"2024-04-24T08:50:43.007120+0000","last_became_peered":"2024-04-24T08:50:43.007120+0000","last_unstale":"2024-04-24T08:51:36.503278+0000","last_undegraded":"2024-04-24T08:51:36.503278+0000","last_fullsized":"2024-04-24T08:51:36.503278+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:27:37.158069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.513585+0000","last_change":"2024-04-24T08:48:29.980632+0000","last_active":"2024-04-24T08:51:36.513585+0000","last_peered":"2024-04-24T08:51:36.513585+0000","last_clean":"2024-04-24T08:51:36.513585+0000","last_became_active":"2024-04-24T08:48:29.980174+0000","last_became_peered":"2024-04-24T08:48:29.980174+0000","last_unstale":"2024-04-24T08:51:36.513585+0000","last_undegraded":"2024-04-24T08:51:36.513585+0000","last_fullsized":"2024-04-24T08:51:36.513585+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:45:32.709091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.513372+0000","last_change":"2024-04-24T08:48:34.316302+0000","last_active":"2024-04-24T08:51:36.513372+0000","last_peered":"2024-04-24T08:51:36.513372+0000","last_clean":"2024-04-24T08:51:36.513372+0000","last_became_active":"2024-04-24T08:48:34.315783+0000","last_became_peered":"2024-04-24T08:48:34.315783+0000","last_unstale":"2024-04-24T08:51:36.513372+0000","last_undegraded":"2024-04-24T08:51:36.513372+0000","last_fullsized":"2024-04-24T08:51:36.513372+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:29:01.733931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"93'3","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.701302+0000","last_change":"2024-04-24T08:50:43.023981+0000","last_active":"2024-04-24T08:51:36.701302+0000","last_peered":"2024-04-24T08:51:36.701302+0000","last_clean":"2024-04-24T08:51:36.701302+0000","last_became_active":"2024-04-24T08:50:43.003967+0000","last_became_peered":"2024-04-24T08:50:43.003967+0000","last_unstale":"2024-04-24T08:51:36.701302+0000","last_undegraded":"2024-04-24T08:51:36.701302+0000","last_fullsized":"2024-04-24T08:51:36.701302+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:13:22.381793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.701617+0000","last_change":"2024-04-24T08:49:41.975577+0000","last_active":"2024-04-24T08:51:36.701617+0000","last_peered":"2024-04-24T08:51:36.701617+0000","last_clean":"2024-04-24T08:51:36.701617+0000","last_became_active":"2024-04-24T08:49:41.975018+0000","last_became_peered":"2024-04-24T08:49:41.975018+0000","last_unstale":"2024-04-24T08:51:36.701617+0000","last_undegraded":"2024-04-24T08:51:36.701617+0000","last_fullsized":"2024-04-24T08:51:36.701617+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:07:53.051847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.503774+0000","last_change":"2024-04-24T08:49:41.978827+0000","last_active":"2024-04-24T08:51:36.503774+0000","last_peered":"2024-04-24T08:51:36.503774+0000","last_clean":"2024-04-24T08:51:36.503774+0000","last_became_active":"2024-04-24T08:49:41.977741+0000","last_became_peered":"2024-04-24T08:49:41.977741+0000","last_unstale":"2024-04-24T08:51:36.503774+0000","last_undegraded":"2024-04-24T08:51:36.503774+0000","last_fullsized":"2024-04-24T08:51:36.503774+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:16:54.754341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.498955+0000","last_change":"2024-04-24T08:48:28.970726+0000","last_active":"2024-04-24T08:51:36.498955+0000","last_peered":"2024-04-24T08:51:36.498955+0000","last_clean":"2024-04-24T08:51:36.498955+0000","last_became_active":"2024-04-24T08:48:28.969911+0000","last_became_peered":"2024-04-24T08:48:28.969911+0000","last_unstale":"2024-04-24T08:51:36.498955+0000","last_undegraded":"2024-04-24T08:51:36.498955+0000","last_fullsized":"2024-04-24T08:51:36.498955+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:55:26.357256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'4","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.369616+0000","last_change":"2024-04-24T08:48:29.967804+0000","last_active":"2024-04-24T08:51:36.369616+0000","last_peered":"2024-04-24T08:51:36.369616+0000","last_clean":"2024-04-24T08:51:36.369616+0000","last_became_active":"2024-04-24T08:48:29.967431+0000","last_became_peered":"2024-04-24T08:48:29.967431+0000","last_unstale":"2024-04-24T08:51:36.369616+0000","last_undegraded":"2024-04-24T08:51:36.369616+0000","last_fullsized":"2024-04-24T08:51:36.369616+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:00:02.779543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.731564+0000","last_change":"2024-04-24T08:48:34.316844+0000","last_active":"2024-04-24T08:51:36.731564+0000","last_peered":"2024-04-24T08:51:36.731564+0000","last_clean":"2024-04-24T08:51:36.731564+0000","last_became_active":"2024-04-24T08:48:34.316426+0000","last_became_peered":"2024-04-24T08:48:34.316426+0000","last_unstale":"2024-04-24T08:51:36.731564+0000","last_undegraded":"2024-04-24T08:51:36.731564+0000","last_fullsized":"2024-04-24T08:51:36.731564+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:10:13.378443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"94'7","reported_seq":195,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.658832+0000","last_change":"2024-04-24T08:48:28.971544+0000","last_active":"2024-04-24T08:51:36.658832+0000","last_peered":"2024-04-24T08:51:36.658832+0000","last_clean":"2024-04-24T08:51:36.658832+0000","last_became_active":"2024-04-24T08:48:28.970417+0000","last_became_peered":"2024-04-24T08:48:28.970417+0000","last_unstale":"2024-04-24T08:51:36.658832+0000","last_undegraded":"2024-04-24T08:51:36.658832+0000","last_fullsized":"2024-04-24T08:51:36.658832+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:35:04.506126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1472,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":155,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.368499+0000","last_change":"2024-04-24T08:48:29.987542+0000","last_active":"2024-04-24T08:51:36.368499+0000","last_peered":"2024-04-24T08:51:36.368499+0000","last_clean":"2024-04-24T08:51:36.368499+0000","last_became_active":"2024-04-24T08:48:29.987258+0000","last_became_peered":"2024-04-24T08:48:29.987258+0000","last_unstale":"2024-04-24T08:51:36.368499+0000","last_undegraded":"2024-04-24T08:51:36.368499+0000","last_fullsized":"2024-04-24T08:51:36.368499+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:37:23.341132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.732011+0000","last_change":"2024-04-24T08:48:34.322636+0000","last_active":"2024-04-24T08:51:36.732011+0000","last_peered":"2024-04-24T08:51:36.732011+0000","last_clean":"2024-04-24T08:51:36.732011+0000","last_became_active":"2024-04-24T08:48:34.321764+0000","last_became_peered":"2024-04-24T08:48:34.321764+0000","last_unstale":"2024-04-24T08:51:36.732011+0000","last_undegraded":"2024-04-24T08:51:36.732011+0000","last_fullsized":"2024-04-24T08:51:36.732011+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:56:10.563907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.514664+0000","last_change":"2024-04-24T08:48:29.981734+0000","last_active":"2024-04-24T08:51:36.514664+0000","last_peered":"2024-04-24T08:51:36.514664+0000","last_clean":"2024-04-24T08:51:36.514664+0000","last_became_active":"2024-04-24T08:48:29.980491+0000","last_became_peered":"2024-04-24T08:48:29.980491+0000","last_unstale":"2024-04-24T08:51:36.514664+0000","last_undegraded":"2024-04-24T08:51:36.514664+0000","last_fullsized":"2024-04-24T08:51:36.514664+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:08:45.189026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"93'6","reported_seq":194,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.729652+0000","last_change":"2024-04-24T08:48:28.954621+0000","last_active":"2024-04-24T08:51:36.729652+0000","last_peered":"2024-04-24T08:51:36.729652+0000","last_clean":"2024-04-24T08:51:36.729652+0000","last_became_active":"2024-04-24T08:48:28.954139+0000","last_became_peered":"2024-04-24T08:48:28.954139+0000","last_unstale":"2024-04-24T08:51:36.729652+0000","last_undegraded":"2024-04-24T08:51:36.729652+0000","last_fullsized":"2024-04-24T08:51:36.729652+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:59:19.360414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":154,"num_read_kb":154,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.658849+0000","last_change":"2024-04-24T08:48:34.323774+0000","last_active":"2024-04-24T08:51:36.658849+0000","last_peered":"2024-04-24T08:51:36.658849+0000","last_clean":"2024-04-24T08:51:36.658849+0000","last_became_active":"2024-04-24T08:48:34.322214+0000","last_became_peered":"2024-04-24T08:48:34.322214+0000","last_unstale":"2024-04-24T08:51:36.658849+0000","last_undegraded":"2024-04-24T08:51:36.658849+0000","last_fullsized":"2024-04-24T08:51:36.658849+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:51:03.971107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.360105+0000","last_change":"2024-04-24T08:48:29.995372+0000","last_active":"2024-04-24T08:51:36.360105+0000","last_peered":"2024-04-24T08:51:36.360105+0000","last_clean":"2024-04-24T08:51:36.360105+0000","last_became_active":"2024-04-24T08:48:29.994744+0000","last_became_peered":"2024-04-24T08:48:29.994744+0000","last_unstale":"2024-04-24T08:51:36.360105+0000","last_undegraded":"2024-04-24T08:51:36.360105+0000","last_fullsized":"2024-04-24T08:51:36.360105+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:28:55.356390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.737024+0000","last_change":"2024-04-24T08:48:28.974921+0000","last_active":"2024-04-24T08:51:36.737024+0000","last_peered":"2024-04-24T08:51:36.737024+0000","last_clean":"2024-04-24T08:51:36.737024+0000","last_became_active":"2024-04-24T08:48:28.974113+0000","last_became_peered":"2024-04-24T08:48:28.974113+0000","last_unstale":"2024-04-24T08:51:36.737024+0000","last_undegraded":"2024-04-24T08:51:36.737024+0000","last_fullsized":"2024-04-24T08:51:36.737024+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:35:12.635253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":367,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.657967+0000","last_change":"2024-04-24T08:46:45.190615+0000","last_active":"2024-04-24T08:51:36.657967+0000","last_peered":"2024-04-24T08:51:36.657967+0000","last_clean":"2024-04-24T08:51:36.657967+0000","last_became_active":"2024-04-24T08:46:45.190011+0000","last_became_peered":"2024-04-24T08:46:45.190011+0000","last_unstale":"2024-04-24T08:51:36.657967+0000","last_undegraded":"2024-04-24T08:51:36.657967+0000","last_fullsized":"2024-04-24T08:51:36.657967+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:41:57.806569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:41:57.806569+0000","last_clean_scrub_stamp":"2024-04-24T08:41:57.806569+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:13:08.283236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.513988+0000","last_change":"2024-04-24T08:48:34.324588+0000","last_active":"2024-04-24T08:51:36.513988+0000","last_peered":"2024-04-24T08:51:36.513988+0000","last_clean":"2024-04-24T08:51:36.513988+0000","last_became_active":"2024-04-24T08:48:34.321969+0000","last_became_peered":"2024-04-24T08:48:34.321969+0000","last_unstale":"2024-04-24T08:51:36.513988+0000","last_undegraded":"2024-04-24T08:51:36.513988+0000","last_fullsized":"2024-04-24T08:51:36.513988+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:19:56.921198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.729531+0000","last_change":"2024-04-24T08:48:29.981573+0000","last_active":"2024-04-24T08:51:36.729531+0000","last_peered":"2024-04-24T08:51:36.729531+0000","last_clean":"2024-04-24T08:51:36.729531+0000","last_became_active":"2024-04-24T08:48:29.979216+0000","last_became_peered":"2024-04-24T08:48:29.979216+0000","last_unstale":"2024-04-24T08:51:36.729531+0000","last_undegraded":"2024-04-24T08:51:36.729531+0000","last_fullsized":"2024-04-24T08:51:36.729531+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:38:09.164883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":17,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.700456+0000","last_change":"2024-04-24T08:50:43.030115+0000","last_active":"2024-04-24T08:51:36.700456+0000","last_peered":"2024-04-24T08:51:36.700456+0000","last_clean":"2024-04-24T08:51:36.700456+0000","last_became_active":"2024-04-24T08:50:43.005669+0000","last_became_peered":"2024-04-24T08:50:43.005669+0000","last_unstale":"2024-04-24T08:51:36.700456+0000","last_undegraded":"2024-04-24T08:51:36.700456+0000","last_fullsized":"2024-04-24T08:51:36.700456+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:32:32.226068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.700661+0000","last_change":"2024-04-24T08:48:34.317502+0000","last_active":"2024-04-24T08:51:36.700661+0000","last_peered":"2024-04-24T08:51:36.700661+0000","last_clean":"2024-04-24T08:51:36.700661+0000","last_became_active":"2024-04-24T08:48:34.316614+0000","last_became_peered":"2024-04-24T08:48:34.316614+0000","last_unstale":"2024-04-24T08:51:36.700661+0000","last_undegraded":"2024-04-24T08:51:36.700661+0000","last_fullsized":"2024-04-24T08:51:36.700661+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:44:12.469926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.504469+0000","last_change":"2024-04-24T08:48:28.970261+0000","last_active":"2024-04-24T08:51:36.504469+0000","last_peered":"2024-04-24T08:51:36.504469+0000","last_clean":"2024-04-24T08:51:36.504469+0000","last_became_active":"2024-04-24T08:48:28.969813+0000","last_became_peered":"2024-04-24T08:48:28.969813+0000","last_unstale":"2024-04-24T08:51:36.504469+0000","last_undegraded":"2024-04-24T08:51:36.504469+0000","last_fullsized":"2024-04-24T08:51:36.504469+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:46:12.311756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.731284+0000","last_change":"2024-04-24T08:48:29.976712+0000","last_active":"2024-04-24T08:51:36.731284+0000","last_peered":"2024-04-24T08:51:36.731284+0000","last_clean":"2024-04-24T08:51:36.731284+0000","last_became_active":"2024-04-24T08:48:29.976166+0000","last_became_peered":"2024-04-24T08:48:29.976166+0000","last_unstale":"2024-04-24T08:51:36.731284+0000","last_undegraded":"2024-04-24T08:51:36.731284+0000","last_fullsized":"2024-04-24T08:51:36.731284+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:52:53.757829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.359483+0000","last_change":"2024-04-24T08:48:34.310173+0000","last_active":"2024-04-24T08:51:36.359483+0000","last_peered":"2024-04-24T08:51:36.359483+0000","last_clean":"2024-04-24T08:51:36.359483+0000","last_became_active":"2024-04-24T08:48:34.309772+0000","last_became_peered":"2024-04-24T08:48:34.309772+0000","last_unstale":"2024-04-24T08:51:36.359483+0000","last_undegraded":"2024-04-24T08:51:36.359483+0000","last_fullsized":"2024-04-24T08:51:36.359483+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:35:50.302769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"94'50","reported_seq":106,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.770470+0000","last_change":"2024-04-24T08:48:28.970127+0000","last_active":"2024-04-24T08:51:36.770470+0000","last_peered":"2024-04-24T08:51:36.770470+0000","last_clean":"2024-04-24T08:51:36.770470+0000","last_became_active":"2024-04-24T08:48:28.968803+0000","last_became_peered":"2024-04-24T08:48:28.968803+0000","last_unstale":"2024-04-24T08:51:36.770470+0000","last_undegraded":"2024-04-24T08:51:36.770470+0000","last_fullsized":"2024-04-24T08:51:36.770470+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:28:47.782276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":88,"num_write_kb":837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.360223+0000","last_change":"2024-04-24T08:48:29.992964+0000","last_active":"2024-04-24T08:51:36.360223+0000","last_peered":"2024-04-24T08:51:36.360223+0000","last_clean":"2024-04-24T08:51:36.360223+0000","last_became_active":"2024-04-24T08:48:29.987501+0000","last_became_peered":"2024-04-24T08:48:29.987501+0000","last_unstale":"2024-04-24T08:51:36.360223+0000","last_undegraded":"2024-04-24T08:51:36.360223+0000","last_fullsized":"2024-04-24T08:51:36.360223+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:49:19.157052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.366605+0000","last_change":"2024-04-24T08:48:34.315393+0000","last_active":"2024-04-24T08:51:36.366605+0000","last_peered":"2024-04-24T08:51:36.366605+0000","last_clean":"2024-04-24T08:51:36.366605+0000","last_became_active":"2024-04-24T08:48:34.314943+0000","last_became_peered":"2024-04-24T08:48:34.314943+0000","last_unstale":"2024-04-24T08:51:36.366605+0000","last_undegraded":"2024-04-24T08:51:36.366605+0000","last_fullsized":"2024-04-24T08:51:36.366605+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:03:13.312223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.737303+0000","last_change":"2024-04-24T08:48:28.969813+0000","last_active":"2024-04-24T08:51:36.737303+0000","last_peered":"2024-04-24T08:51:36.737303+0000","last_clean":"2024-04-24T08:51:36.737303+0000","last_became_active":"2024-04-24T08:48:28.968485+0000","last_became_peered":"2024-04-24T08:48:28.968485+0000","last_unstale":"2024-04-24T08:51:36.737303+0000","last_undegraded":"2024-04-24T08:51:36.737303+0000","last_fullsized":"2024-04-24T08:51:36.737303+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:32:10.095723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.893652+0000","last_change":"2024-04-24T08:48:29.997731+0000","last_active":"2024-04-24T08:51:36.893652+0000","last_peered":"2024-04-24T08:51:36.893652+0000","last_clean":"2024-04-24T08:51:36.893652+0000","last_became_active":"2024-04-24T08:48:29.997373+0000","last_became_peered":"2024-04-24T08:48:29.997373+0000","last_unstale":"2024-04-24T08:51:36.893652+0000","last_undegraded":"2024-04-24T08:51:36.893652+0000","last_fullsized":"2024-04-24T08:51:36.893652+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:57:19.531173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.736525+0000","last_change":"2024-04-24T08:48:34.326170+0000","last_active":"2024-04-24T08:51:36.736525+0000","last_peered":"2024-04-24T08:51:36.736525+0000","last_clean":"2024-04-24T08:51:36.736525+0000","last_became_active":"2024-04-24T08:48:34.325678+0000","last_became_peered":"2024-04-24T08:48:34.325678+0000","last_unstale":"2024-04-24T08:51:36.736525+0000","last_undegraded":"2024-04-24T08:51:36.736525+0000","last_fullsized":"2024-04-24T08:51:36.736525+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:12:47.003864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.892690+0000","last_change":"2024-04-24T08:48:28.975522+0000","last_active":"2024-04-24T08:51:36.892690+0000","last_peered":"2024-04-24T08:51:36.892690+0000","last_clean":"2024-04-24T08:51:36.892690+0000","last_became_active":"2024-04-24T08:48:28.975093+0000","last_became_peered":"2024-04-24T08:48:28.975093+0000","last_unstale":"2024-04-24T08:51:36.892690+0000","last_undegraded":"2024-04-24T08:51:36.892690+0000","last_fullsized":"2024-04-24T08:51:36.892690+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:37:31.809694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.503358+0000","last_change":"2024-04-24T08:48:29.982502+0000","last_active":"2024-04-24T08:51:36.503358+0000","last_peered":"2024-04-24T08:51:36.503358+0000","last_clean":"2024-04-24T08:51:36.503358+0000","last_became_active":"2024-04-24T08:48:29.982141+0000","last_became_peered":"2024-04-24T08:48:29.982141+0000","last_unstale":"2024-04-24T08:51:36.503358+0000","last_undegraded":"2024-04-24T08:51:36.503358+0000","last_fullsized":"2024-04-24T08:51:36.503358+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:46:15.851359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.499230+0000","last_change":"2024-04-24T08:48:34.322717+0000","last_active":"2024-04-24T08:51:36.499230+0000","last_peered":"2024-04-24T08:51:36.499230+0000","last_clean":"2024-04-24T08:51:36.499230+0000","last_became_active":"2024-04-24T08:48:34.321837+0000","last_became_peered":"2024-04-24T08:48:34.321837+0000","last_unstale":"2024-04-24T08:51:36.499230+0000","last_undegraded":"2024-04-24T08:51:36.499230+0000","last_fullsized":"2024-04-24T08:51:36.499230+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:40:11.557436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.359786+0000","last_change":"2024-04-24T08:48:28.973951+0000","last_active":"2024-04-24T08:51:36.359786+0000","last_peered":"2024-04-24T08:51:36.359786+0000","last_clean":"2024-04-24T08:51:36.359786+0000","last_became_active":"2024-04-24T08:48:28.973131+0000","last_became_peered":"2024-04-24T08:48:28.973131+0000","last_unstale":"2024-04-24T08:51:36.359786+0000","last_undegraded":"2024-04-24T08:51:36.359786+0000","last_fullsized":"2024-04-24T08:51:36.359786+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:45:45.197337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.701038+0000","last_change":"2024-04-24T08:48:29.992689+0000","last_active":"2024-04-24T08:51:36.701038+0000","last_peered":"2024-04-24T08:51:36.701038+0000","last_clean":"2024-04-24T08:51:36.701038+0000","last_became_active":"2024-04-24T08:48:29.988874+0000","last_became_peered":"2024-04-24T08:48:29.988874+0000","last_unstale":"2024-04-24T08:51:36.701038+0000","last_undegraded":"2024-04-24T08:51:36.701038+0000","last_fullsized":"2024-04-24T08:51:36.701038+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:42:41.578424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.737734+0000","last_change":"2024-04-24T08:48:34.318640+0000","last_active":"2024-04-24T08:51:36.737734+0000","last_peered":"2024-04-24T08:51:36.737734+0000","last_clean":"2024-04-24T08:51:36.737734+0000","last_became_active":"2024-04-24T08:48:34.318039+0000","last_became_peered":"2024-04-24T08:48:34.318039+0000","last_unstale":"2024-04-24T08:51:36.737734+0000","last_undegraded":"2024-04-24T08:51:36.737734+0000","last_fullsized":"2024-04-24T08:51:36.737734+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:17:27.110930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"94'6","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.367758+0000","last_change":"2024-04-24T08:48:28.980828+0000","last_active":"2024-04-24T08:51:36.367758+0000","last_peered":"2024-04-24T08:51:36.367758+0000","last_clean":"2024-04-24T08:51:36.367758+0000","last_became_active":"2024-04-24T08:48:28.979814+0000","last_became_peered":"2024-04-24T08:48:28.979814+0000","last_unstale":"2024-04-24T08:51:36.367758+0000","last_undegraded":"2024-04-24T08:51:36.367758+0000","last_fullsized":"2024-04-24T08:51:36.367758+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:02:28.145847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.359850+0000","last_change":"2024-04-24T08:48:29.995003+0000","last_active":"2024-04-24T08:51:36.359850+0000","last_peered":"2024-04-24T08:51:36.359850+0000","last_clean":"2024-04-24T08:51:36.359850+0000","last_became_active":"2024-04-24T08:48:29.994516+0000","last_became_peered":"2024-04-24T08:48:29.994516+0000","last_unstale":"2024-04-24T08:51:36.359850+0000","last_undegraded":"2024-04-24T08:51:36.359850+0000","last_fullsized":"2024-04-24T08:51:36.359850+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:15:46.094534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.738075+0000","last_change":"2024-04-24T08:48:34.311721+0000","last_active":"2024-04-24T08:51:36.738075+0000","last_peered":"2024-04-24T08:51:36.738075+0000","last_clean":"2024-04-24T08:51:36.738075+0000","last_became_active":"2024-04-24T08:48:34.311241+0000","last_became_peered":"2024-04-24T08:48:34.311241+0000","last_unstale":"2024-04-24T08:51:36.738075+0000","last_undegraded":"2024-04-24T08:51:36.738075+0000","last_fullsized":"2024-04-24T08:51:36.738075+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:16:17.679237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.503671+0000","last_change":"2024-04-24T08:48:28.973336+0000","last_active":"2024-04-24T08:51:36.503671+0000","last_peered":"2024-04-24T08:51:36.503671+0000","last_clean":"2024-04-24T08:51:36.503671+0000","last_became_active":"2024-04-24T08:48:28.972666+0000","last_became_peered":"2024-04-24T08:48:28.972666+0000","last_unstale":"2024-04-24T08:51:36.503671+0000","last_undegraded":"2024-04-24T08:51:36.503671+0000","last_fullsized":"2024-04-24T08:51:36.503671+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:53:28.358291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.658302+0000","last_change":"2024-04-24T08:48:29.983498+0000","last_active":"2024-04-24T08:51:36.658302+0000","last_peered":"2024-04-24T08:51:36.658302+0000","last_clean":"2024-04-24T08:51:36.658302+0000","last_became_active":"2024-04-24T08:48:29.983167+0000","last_became_peered":"2024-04-24T08:48:29.983167+0000","last_unstale":"2024-04-24T08:51:36.658302+0000","last_undegraded":"2024-04-24T08:51:36.658302+0000","last_fullsized":"2024-04-24T08:51:36.658302+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:09:56.648047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.893130+0000","last_change":"2024-04-24T08:49:42.129192+0000","last_active":"2024-04-24T08:51:36.893130+0000","last_peered":"2024-04-24T08:51:36.893130+0000","last_clean":"2024-04-24T08:51:36.893130+0000","last_became_active":"2024-04-24T08:49:42.124814+0000","last_became_peered":"2024-04-24T08:49:42.124814+0000","last_unstale":"2024-04-24T08:51:36.893130+0000","last_undegraded":"2024-04-24T08:51:36.893130+0000","last_fullsized":"2024-04-24T08:51:36.893130+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:52:46.273836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6,7],"acting":[1,8,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"94'8","reported_seq":197,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.893736+0000","last_change":"2024-04-24T08:48:28.974528+0000","last_active":"2024-04-24T08:51:36.893736+0000","last_peered":"2024-04-24T08:51:36.893736+0000","last_clean":"2024-04-24T08:51:36.893736+0000","last_became_active":"2024-04-24T08:48:28.974203+0000","last_became_peered":"2024-04-24T08:48:28.974203+0000","last_unstale":"2024-04-24T08:51:36.893736+0000","last_undegraded":"2024-04-24T08:51:36.893736+0000","last_fullsized":"2024-04-24T08:51:36.893736+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:26:15.701204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2445,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":156,"num_read_kb":3,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.659205+0000","last_change":"2024-04-24T08:48:29.984356+0000","last_active":"2024-04-24T08:51:36.659205+0000","last_peered":"2024-04-24T08:51:36.659205+0000","last_clean":"2024-04-24T08:51:36.659205+0000","last_became_active":"2024-04-24T08:48:29.983949+0000","last_became_peered":"2024-04-24T08:48:29.983949+0000","last_unstale":"2024-04-24T08:51:36.659205+0000","last_undegraded":"2024-04-24T08:51:36.659205+0000","last_fullsized":"2024-04-24T08:51:36.659205+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:51:59.409835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.893130+0000","last_change":"2024-04-24T08:48:34.312418+0000","last_active":"2024-04-24T08:51:36.893130+0000","last_peered":"2024-04-24T08:51:36.893130+0000","last_clean":"2024-04-24T08:51:36.893130+0000","last_became_active":"2024-04-24T08:48:34.312035+0000","last_became_peered":"2024-04-24T08:48:34.312035+0000","last_unstale":"2024-04-24T08:51:36.893130+0000","last_undegraded":"2024-04-24T08:51:36.893130+0000","last_fullsized":"2024-04-24T08:51:36.893130+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:27:33.461732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"94'41","reported_seq":83,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.765572+0000","last_change":"2024-04-24T08:48:28.976070+0000","last_active":"2024-04-24T08:51:36.765572+0000","last_peered":"2024-04-24T08:51:36.765572+0000","last_clean":"2024-04-24T08:51:36.765572+0000","last_became_active":"2024-04-24T08:48:28.975703+0000","last_became_peered":"2024-04-24T08:48:28.975703+0000","last_unstale":"2024-04-24T08:51:36.765572+0000","last_undegraded":"2024-04-24T08:51:36.765572+0000","last_fullsized":"2024-04-24T08:51:36.765572+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:46:13.227529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'6","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.658745+0000","last_change":"2024-04-24T08:49:42.128814+0000","last_active":"2024-04-24T08:51:36.658745+0000","last_peered":"2024-04-24T08:51:36.658745+0000","last_clean":"2024-04-24T08:51:36.658745+0000","last_became_active":"2024-04-24T08:49:42.127422+0000","last_became_peered":"2024-04-24T08:49:42.127422+0000","last_unstale":"2024-04-24T08:51:36.658745+0000","last_undegraded":"2024-04-24T08:51:36.658745+0000","last_fullsized":"2024-04-24T08:51:36.658745+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:58:25.181798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.512831+0000","last_change":"2024-04-24T08:48:34.320191+0000","last_active":"2024-04-24T08:51:36.512831+0000","last_peered":"2024-04-24T08:51:36.512831+0000","last_clean":"2024-04-24T08:51:36.512831+0000","last_became_active":"2024-04-24T08:48:34.315415+0000","last_became_peered":"2024-04-24T08:48:34.315415+0000","last_unstale":"2024-04-24T08:51:36.512831+0000","last_undegraded":"2024-04-24T08:51:36.512831+0000","last_fullsized":"2024-04-24T08:51:36.512831+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:55:11.372789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.731727+0000","last_change":"2024-04-24T08:48:28.968550+0000","last_active":"2024-04-24T08:51:36.731727+0000","last_peered":"2024-04-24T08:51:36.731727+0000","last_clean":"2024-04-24T08:51:36.731727+0000","last_became_active":"2024-04-24T08:48:28.968120+0000","last_became_peered":"2024-04-24T08:48:28.968120+0000","last_unstale":"2024-04-24T08:51:36.731727+0000","last_undegraded":"2024-04-24T08:51:36.731727+0000","last_fullsized":"2024-04-24T08:51:36.731727+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:46:00.870285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.659317+0000","last_change":"2024-04-24T08:48:29.984755+0000","last_active":"2024-04-24T08:51:36.659317+0000","last_peered":"2024-04-24T08:51:36.659317+0000","last_clean":"2024-04-24T08:51:36.659317+0000","last_became_active":"2024-04-24T08:48:29.982093+0000","last_became_peered":"2024-04-24T08:48:29.982093+0000","last_unstale":"2024-04-24T08:51:36.659317+0000","last_undegraded":"2024-04-24T08:51:36.659317+0000","last_fullsized":"2024-04-24T08:51:36.659317+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:25:22.620152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.700841+0000","last_change":"2024-04-24T08:48:29.983101+0000","last_active":"2024-04-24T08:51:36.700841+0000","last_peered":"2024-04-24T08:51:36.700841+0000","last_clean":"2024-04-24T08:51:36.700841+0000","last_became_active":"2024-04-24T08:48:29.981871+0000","last_became_peered":"2024-04-24T08:48:29.981871+0000","last_unstale":"2024-04-24T08:51:36.700841+0000","last_undegraded":"2024-04-24T08:51:36.700841+0000","last_fullsized":"2024-04-24T08:51:36.700841+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:02:44.582111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":49,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.367516+0000","last_change":"2024-04-24T08:49:42.128974+0000","last_active":"2024-04-24T08:51:36.367516+0000","last_peered":"2024-04-24T08:51:36.367516+0000","last_clean":"2024-04-24T08:51:36.367516+0000","last_became_active":"2024-04-24T08:49:42.127710+0000","last_became_peered":"2024-04-24T08:49:42.127710+0000","last_unstale":"2024-04-24T08:51:36.367516+0000","last_undegraded":"2024-04-24T08:51:36.367516+0000","last_fullsized":"2024-04-24T08:51:36.367516+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:09:20.279852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.367156+0000","last_change":"2024-04-24T08:48:29.988241+0000","last_active":"2024-04-24T08:51:36.367156+0000","last_peered":"2024-04-24T08:51:36.367156+0000","last_clean":"2024-04-24T08:51:36.367156+0000","last_became_active":"2024-04-24T08:48:29.987930+0000","last_became_peered":"2024-04-24T08:48:29.987930+0000","last_unstale":"2024-04-24T08:51:36.367156+0000","last_undegraded":"2024-04-24T08:51:36.367156+0000","last_fullsized":"2024-04-24T08:51:36.367156+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:42:24.687624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.893443+0000","last_change":"2024-04-24T08:48:29.992441+0000","last_active":"2024-04-24T08:51:36.893443+0000","last_peered":"2024-04-24T08:51:36.893443+0000","last_clean":"2024-04-24T08:51:36.893443+0000","last_became_active":"2024-04-24T08:48:29.988788+0000","last_became_peered":"2024-04-24T08:48:29.988788+0000","last_unstale":"2024-04-24T08:51:36.893443+0000","last_undegraded":"2024-04-24T08:51:36.893443+0000","last_fullsized":"2024-04-24T08:51:36.893443+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:08:31.549509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.731077+0000","last_change":"2024-04-24T08:48:29.982116+0000","last_active":"2024-04-24T08:51:36.731077+0000","last_peered":"2024-04-24T08:51:36.731077+0000","last_clean":"2024-04-24T08:51:36.731077+0000","last_became_active":"2024-04-24T08:48:29.981636+0000","last_became_peered":"2024-04-24T08:48:29.981636+0000","last_unstale":"2024-04-24T08:51:36.731077+0000","last_undegraded":"2024-04-24T08:51:36.731077+0000","last_fullsized":"2024-04-24T08:51:36.731077+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:11:01.943425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.701572+0000","last_change":"2024-04-24T08:48:29.982244+0000","last_active":"2024-04-24T08:51:36.701572+0000","last_peered":"2024-04-24T08:51:36.701572+0000","last_clean":"2024-04-24T08:51:36.701572+0000","last_became_active":"2024-04-24T08:48:29.981622+0000","last_became_peered":"2024-04-24T08:48:29.981622+0000","last_unstale":"2024-04-24T08:51:36.701572+0000","last_undegraded":"2024-04-24T08:51:36.701572+0000","last_fullsized":"2024-04-24T08:51:36.701572+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:02:57.723949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":46,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.369396+0000","last_change":"2024-04-24T08:48:29.985867+0000","last_active":"2024-04-24T08:51:36.369396+0000","last_peered":"2024-04-24T08:51:36.369396+0000","last_clean":"2024-04-24T08:51:36.369396+0000","last_became_active":"2024-04-24T08:48:29.985430+0000","last_became_peered":"2024-04-24T08:48:29.985430+0000","last_unstale":"2024-04-24T08:51:36.369396+0000","last_undegraded":"2024-04-24T08:51:36.369396+0000","last_fullsized":"2024-04-24T08:51:36.369396+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:09:40.467623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.360401+0000","last_change":"2024-04-24T08:48:29.996474+0000","last_active":"2024-04-24T08:51:36.360401+0000","last_peered":"2024-04-24T08:51:36.360401+0000","last_clean":"2024-04-24T08:51:36.360401+0000","last_became_active":"2024-04-24T08:48:29.995958+0000","last_became_peered":"2024-04-24T08:48:29.995958+0000","last_unstale":"2024-04-24T08:51:36.360401+0000","last_undegraded":"2024-04-24T08:51:36.360401+0000","last_fullsized":"2024-04-24T08:51:36.360401+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:50:19.865125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.513993+0000","last_change":"2024-04-24T08:48:29.978285+0000","last_active":"2024-04-24T08:51:36.513993+0000","last_peered":"2024-04-24T08:51:36.513993+0000","last_clean":"2024-04-24T08:51:36.513993+0000","last_became_active":"2024-04-24T08:48:29.976445+0000","last_became_peered":"2024-04-24T08:48:29.976445+0000","last_unstale":"2024-04-24T08:51:36.513993+0000","last_undegraded":"2024-04-24T08:51:36.513993+0000","last_fullsized":"2024-04-24T08:51:36.513993+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:30:29.598733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.359571+0000","last_change":"2024-04-24T08:48:29.994442+0000","last_active":"2024-04-24T08:51:36.359571+0000","last_peered":"2024-04-24T08:51:36.359571+0000","last_clean":"2024-04-24T08:51:36.359571+0000","last_became_active":"2024-04-24T08:48:29.993896+0000","last_became_peered":"2024-04-24T08:48:29.993896+0000","last_unstale":"2024-04-24T08:51:36.359571+0000","last_undegraded":"2024-04-24T08:51:36.359571+0000","last_fullsized":"2024-04-24T08:51:36.359571+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:37:14.023511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":47,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.893266+0000","last_change":"2024-04-24T08:49:41.981252+0000","last_active":"2024-04-24T08:51:36.893266+0000","last_peered":"2024-04-24T08:51:36.893266+0000","last_clean":"2024-04-24T08:51:36.893266+0000","last_became_active":"2024-04-24T08:49:41.980495+0000","last_became_peered":"2024-04-24T08:49:41.980495+0000","last_unstale":"2024-04-24T08:51:36.893266+0000","last_undegraded":"2024-04-24T08:51:36.893266+0000","last_fullsized":"2024-04-24T08:51:36.893266+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:47:33.579204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,8],"acting":[1,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.730358+0000","last_change":"2024-04-24T08:48:29.986021+0000","last_active":"2024-04-24T08:51:36.730358+0000","last_peered":"2024-04-24T08:51:36.730358+0000","last_clean":"2024-04-24T08:51:36.730358+0000","last_became_active":"2024-04-24T08:48:29.985182+0000","last_became_peered":"2024-04-24T08:48:29.985182+0000","last_unstale":"2024-04-24T08:51:36.730358+0000","last_undegraded":"2024-04-24T08:51:36.730358+0000","last_fullsized":"2024-04-24T08:51:36.730358+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:10:57.143065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.700856+0000","last_change":"2024-04-24T08:48:29.979886+0000","last_active":"2024-04-24T08:51:36.700856+0000","last_peered":"2024-04-24T08:51:36.700856+0000","last_clean":"2024-04-24T08:51:36.700856+0000","last_became_active":"2024-04-24T08:48:29.978779+0000","last_became_peered":"2024-04-24T08:48:29.978779+0000","last_unstale":"2024-04-24T08:51:36.700856+0000","last_undegraded":"2024-04-24T08:51:36.700856+0000","last_fullsized":"2024-04-24T08:51:36.700856+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:51:51.303990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":602295,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1664,"num_read_kb":942,"num_write":386,"num_write_kb":1475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":134,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2174976,"data_stored":1806885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":310,"ondisk_log_size":310,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383733,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1069056,"data_stored":520983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645372,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":33100,"kb_used_data":1864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712052,"statfs":{"total":95995035648,"available":95961141248,"internally_reserved":0,"allocated":1908736,"data_stored":1311223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808899,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32648,"kb_used_data":1468,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712504,"statfs":{"total":95995035648,"available":95961604096,"internally_reserved":0,"allocated":1503232,"data_stored":927663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070538,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28476,"kb_used_data":1456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716676,"statfs":{"total":95995035648,"available":95965876224,"internally_reserved":0,"allocated":1490944,"data_stored":923923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332178,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32840,"kb_used_data":1660,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712312,"statfs":{"total":95995035648,"available":95961407488,"internally_reserved":0,"allocated":1699840,"data_stored":1128846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495705,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32216,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712936,"statfs":{"total":95995035648,"available":95962046464,"internally_reserved":0,"allocated":1064960,"data_stored":516395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574973,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36468,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708684,"statfs":{"total":95995035648,"available":95957692416,"internally_reserved":0,"allocated":1155072,"data_stored":594758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738500,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36420,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708732,"statfs":{"total":95995035648,"available":95957741568,"internally_reserved":0,"allocated":1105920,"data_stored":528586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411445,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28152,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717000,"statfs":{"total":95995035648,"available":95966208000,"internally_reserved":0,"allocated":1159168,"data_stored":590404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182511,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32248,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712904,"statfs":{"total":95995035648,"available":95962013696,"internally_reserved":0,"allocated":1093632,"data_stored":525563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920871,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32232,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712920,"statfs":{"total":95995035648,"available":95962030080,"internally_reserved":0,"allocated":1081344,"data_stored":520581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757345,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32976,"kb_used_data":1736,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712176,"statfs":{"total":95995035648,"available":95961268224,"internally_reserved":0,"allocated":1777664,"data_stored":1189498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":69625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":71540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":557056,"data_stored":525246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":180224,"data_stored":135836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":421888,"data_stored":394694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":258048,"data_stored":200818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T08:51:42.303 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T08:51:42.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:41 smithi138 ceph-mon[31441]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:51:42.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:41 smithi138 ceph-mon[31441]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-04-24T08:51:42.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:41 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/2082720260' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T08:51:42.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:41 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T08:51:42.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:41 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T08:51:42.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:41 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:51:42.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:41 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:51:42.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:41 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:51:42.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:41 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:51:42.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:41 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T08:51:42.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:41 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T08:51:42.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:41 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T08:51:42.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:41 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:51:42.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:41 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T08:51:42.457 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:41 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:51:42.457 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:41 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:51:42.457 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:41 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:51:42.457 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:41 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:51:42.457 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:41 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/3076566445' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:51:42.457 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:41 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:51:42.457 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:41 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:51:42.457 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:41 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:51:42.457 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:41 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:51:42.457 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:41 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:51:42.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:41 smithi189 ceph-mon[31791]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:51:42.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:41 smithi189 ceph-mon[31791]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-04-24T08:51:42.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:41 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/2082720260' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T08:51:42.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:41 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T08:51:42.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:41 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T08:51:42.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:41 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:51:42.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:41 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:51:42.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:41 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:51:42.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:41 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:51:42.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:41 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T08:51:42.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:41 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T08:51:42.459 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:41 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T08:51:42.459 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:41 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:51:42.459 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:41 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T08:51:42.459 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:41 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:51:42.459 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:41 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:51:42.459 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:41 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:51:42.459 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:41 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:51:42.459 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:41 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/3076566445' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:51:42.459 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:41 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:51:42.459 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:41 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:51:42.459 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:41 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-24T08:51:42.459 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:41 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:51:42.459 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:41 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:51:42.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.515+0000 7f4e05600640 1 -- 172.21.15.83:0/3517570460 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e00103240 msgr2=0x7f4e001036c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:42.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.515+0000 7f4e05600640 1 --2- 172.21.15.83:0/3517570460 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e00103240 0x7f4e001036c0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f4dec009900 tx=0x7f4dec02a410 comp rx=0 tx=0).stop 2024-04-24T08:51:42.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.515+0000 7f4e05600640 1 -- 172.21.15.83:0/3517570460 shutdown_connections 2024-04-24T08:51:42.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.515+0000 7f4e05600640 1 --2- 172.21.15.83:0/3517570460 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4e00103c00 0x7f4e0010a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:42.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.515+0000 7f4e05600640 1 --2- 172.21.15.83:0/3517570460 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e00103240 0x7f4e001036c0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:42.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.515+0000 7f4e05600640 1 --2- 172.21.15.83:0/3517570460 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e00102050 0x7f4e00102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:42.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.515+0000 7f4e05600640 1 -- 172.21.15.83:0/3517570460 >> 172.21.15.83:0/3517570460 conn(0x7f4e000fd5e0 msgr2=0x7f4e000ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:42.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.515+0000 7f4e05600640 1 -- 172.21.15.83:0/3517570460 shutdown_connections 2024-04-24T08:51:42.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.515+0000 7f4e05600640 1 -- 172.21.15.83:0/3517570460 wait complete. 2024-04-24T08:51:42.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.516+0000 7f4e05600640 1 Processor -- start 2024-04-24T08:51:42.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.516+0000 7f4e05600640 1 -- start start 2024-04-24T08:51:42.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.517+0000 7f4e05600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e00102050 0x7f4e00198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:42.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.517+0000 7f4e05600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4e00103240 0x7f4e001987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:42.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.517+0000 7f4e05600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e00103c00 0x7f4e0019f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:42.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.517+0000 7f4e05600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4e0010db90 con 0x7f4e00102050 2024-04-24T08:51:42.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.517+0000 7f4e05600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4e0010da10 con 0x7f4e00103c00 2024-04-24T08:51:42.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.517+0000 7f4e05600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4e0010dd10 con 0x7f4e00103240 2024-04-24T08:51:42.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.517+0000 7f4dfea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e00102050 0x7f4e00198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:42.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.517+0000 7f4dfea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e00102050 0x7f4e00198270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55820/0 (socket says 172.21.15.83:55820) 2024-04-24T08:51:42.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.517+0000 7f4dfea00640 1 -- 172.21.15.83:0/1084941550 learned_addr learned my addr 172.21.15.83:0/1084941550 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:51:42.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.517+0000 7f4dff400640 1 --2- 172.21.15.83:0/1084941550 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e00103c00 0x7f4e0019f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:42.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.517+0000 7f4dfe000640 1 --2- 172.21.15.83:0/1084941550 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4e00103240 0x7f4e001987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:42.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.518+0000 7f4dfea00640 1 -- 172.21.15.83:0/1084941550 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4e00103240 msgr2=0x7f4e001987b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:42.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.518+0000 7f4dfea00640 1 --2- 172.21.15.83:0/1084941550 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4e00103240 0x7f4e001987b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:42.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.518+0000 7f4dfea00640 1 -- 172.21.15.83:0/1084941550 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e00103c00 msgr2=0x7f4e0019f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:42.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.518+0000 7f4dfea00640 1 --2- 172.21.15.83:0/1084941550 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e00103c00 0x7f4e0019f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:42.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.518+0000 7f4dfea00640 1 -- 172.21.15.83:0/1084941550 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e0019fe90 con 0x7f4e00102050 2024-04-24T08:51:42.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.518+0000 7f4dfea00640 1 --2- 172.21.15.83:0/1084941550 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e00102050 0x7f4e00198270 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f4df000e9c0 tx=0x7f4df000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:42.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.518+0000 7f4df7400640 1 -- 172.21.15.83:0/1084941550 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4df000cd80 con 0x7f4e00102050 2024-04-24T08:51:42.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.518+0000 7f4df7400640 1 -- 172.21.15.83:0/1084941550 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4df00035f0 con 0x7f4e00102050 2024-04-24T08:51:42.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.518+0000 7f4e05600640 1 -- 172.21.15.83:0/1084941550 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e001a0180 con 0x7f4e00102050 2024-04-24T08:51:42.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.518+0000 7f4df7400640 1 -- 172.21.15.83:0/1084941550 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4df0010690 con 0x7f4e00102050 2024-04-24T08:51:42.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.519+0000 7f4e05600640 1 -- 172.21.15.83:0/1084941550 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e001a06f0 con 0x7f4e00102050 2024-04-24T08:51:42.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.520+0000 7f4e05600640 1 -- 172.21.15.83:0/1084941550 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e001093f0 con 0x7f4e00102050 2024-04-24T08:51:42.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.522+0000 7f4df7400640 1 -- 172.21.15.83:0/1084941550 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f4df00037a0 con 0x7f4e00102050 2024-04-24T08:51:42.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.524+0000 7f4df7400640 1 --2- 172.21.15.83:0/1084941550 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f4de40a3e30 0x7f4de40a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:42.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.524+0000 7f4df7400640 1 -- 172.21.15.83:0/1084941550 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f4df00d10a0 con 0x7f4e00102050 2024-04-24T08:51:42.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.524+0000 7f4dfe000640 1 --2- 172.21.15.83:0/1084941550 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f4de40a3e30 0x7f4de40a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:42.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.525+0000 7f4dfe000640 1 --2- 172.21.15.83:0/1084941550 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f4de40a3e30 0x7f4de40a62f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f4dec006010 tx=0x7f4dec000bc0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:42.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.528+0000 7f4df7400640 1 -- 172.21.15.83:0/1084941550 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4df009d200 con 0x7f4e00102050 2024-04-24T08:51:42.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.738+0000 7f4e05600640 1 -- 172.21.15.83:0/1084941550 --> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4e00062440 con 0x7f4de40a3e30 2024-04-24T08:51:42.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.749+0000 7f4df7400640 1 -- 172.21.15.83:0/1084941550 <== mgr.15579 v2:172.21.15.83:6800/3594769126 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339543 (secure 0 0 0) 0x7f4e00062440 con 0x7f4de40a3e30 2024-04-24T08:51:42.753 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:51:42.755 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-04-24T08:51:42.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.754+0000 7f4e05600640 1 -- 172.21.15.83:0/1084941550 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f4de40a3e30 msgr2=0x7f4de40a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:42.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.754+0000 7f4e05600640 1 --2- 172.21.15.83:0/1084941550 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f4de40a3e30 0x7f4de40a62f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f4dec006010 tx=0x7f4dec000bc0 comp rx=0 tx=0).stop 2024-04-24T08:51:42.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.754+0000 7f4e05600640 1 -- 172.21.15.83:0/1084941550 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e00102050 msgr2=0x7f4e00198270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:42.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.754+0000 7f4e05600640 1 --2- 172.21.15.83:0/1084941550 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e00102050 0x7f4e00198270 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f4df000e9c0 tx=0x7f4df000ee80 comp rx=0 tx=0).stop 2024-04-24T08:51:42.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.754+0000 7f4e05600640 1 -- 172.21.15.83:0/1084941550 shutdown_connections 2024-04-24T08:51:42.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.754+0000 7f4e05600640 1 --2- 172.21.15.83:0/1084941550 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f4de40a3e30 0x7f4de40a62f0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:42.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.754+0000 7f4e05600640 1 --2- 172.21.15.83:0/1084941550 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e00103c00 0x7f4e0019f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:42.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.754+0000 7f4e05600640 1 --2- 172.21.15.83:0/1084941550 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4e00103240 0x7f4e001987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:42.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.754+0000 7f4e05600640 1 --2- 172.21.15.83:0/1084941550 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e00102050 0x7f4e00198270 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:42.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.754+0000 7f4e05600640 1 -- 172.21.15.83:0/1084941550 >> 172.21.15.83:0/1084941550 conn(0x7f4e000fd5e0 msgr2=0x7f4e001065c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:42.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.754+0000 7f4e05600640 1 -- 172.21.15.83:0/1084941550 shutdown_connections 2024-04-24T08:51:42.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.754+0000 7f4e05600640 1 -- 172.21.15.83:0/1084941550 wait complete. 2024-04-24T08:51:42.770 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":6,"stamp":"2024-04-24T08:51:42.381846+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1192897,"num_objects":104,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1800,"num_read_kb":1061,"num_write":569,"num_write_kb":4415,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":1180870,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":949,"ondisk_log_size":949,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":385836,"kb_used_data":15732,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124555988,"statfs":{"total":1151940427776,"available":1151545331712,"internally_reserved":0,"allocated":16109568,"data_stored":9278423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":378713401},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.971095"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.701720+0000","last_change":"2024-04-24T08:48:29.993009+0000","last_active":"2024-04-24T08:51:36.701720+0000","last_peered":"2024-04-24T08:51:36.701720+0000","last_clean":"2024-04-24T08:51:36.701720+0000","last_became_active":"2024-04-24T08:48:29.990613+0000","last_became_peered":"2024-04-24T08:48:29.990613+0000","last_unstale":"2024-04-24T08:51:36.701720+0000","last_undegraded":"2024-04-24T08:51:36.701720+0000","last_fullsized":"2024-04-24T08:51:36.701720+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:13:43.871742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.893581+0000","last_change":"2024-04-24T08:48:29.986957+0000","last_active":"2024-04-24T08:51:36.893581+0000","last_peered":"2024-04-24T08:51:36.893581+0000","last_clean":"2024-04-24T08:51:36.893581+0000","last_became_active":"2024-04-24T08:48:29.984691+0000","last_became_peered":"2024-04-24T08:48:29.984691+0000","last_unstale":"2024-04-24T08:51:36.893581+0000","last_undegraded":"2024-04-24T08:51:36.893581+0000","last_fullsized":"2024-04-24T08:51:36.893581+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:07:38.031213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.893523+0000","last_change":"2024-04-24T08:48:29.991478+0000","last_active":"2024-04-24T08:51:36.893523+0000","last_peered":"2024-04-24T08:51:36.893523+0000","last_clean":"2024-04-24T08:51:36.893523+0000","last_became_active":"2024-04-24T08:48:29.991013+0000","last_became_peered":"2024-04-24T08:48:29.991013+0000","last_unstale":"2024-04-24T08:51:36.893523+0000","last_undegraded":"2024-04-24T08:51:36.893523+0000","last_fullsized":"2024-04-24T08:51:36.893523+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:10:39.381353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.738223+0000","last_change":"2024-04-24T08:48:29.983664+0000","last_active":"2024-04-24T08:51:36.738223+0000","last_peered":"2024-04-24T08:51:36.738223+0000","last_clean":"2024-04-24T08:51:36.738223+0000","last_became_active":"2024-04-24T08:48:29.978835+0000","last_became_peered":"2024-04-24T08:48:29.978835+0000","last_unstale":"2024-04-24T08:51:36.738223+0000","last_undegraded":"2024-04-24T08:51:36.738223+0000","last_fullsized":"2024-04-24T08:51:36.738223+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:29:20.501652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.503590+0000","last_change":"2024-04-24T08:48:29.989886+0000","last_active":"2024-04-24T08:51:36.503590+0000","last_peered":"2024-04-24T08:51:36.503590+0000","last_clean":"2024-04-24T08:51:36.503590+0000","last_became_active":"2024-04-24T08:48:29.989574+0000","last_became_peered":"2024-04-24T08:48:29.989574+0000","last_unstale":"2024-04-24T08:51:36.503590+0000","last_undegraded":"2024-04-24T08:51:36.503590+0000","last_fullsized":"2024-04-24T08:51:36.503590+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:14:01.085543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.368540+0000","last_change":"2024-04-24T08:48:29.978045+0000","last_active":"2024-04-24T08:51:36.368540+0000","last_peered":"2024-04-24T08:51:36.368540+0000","last_clean":"2024-04-24T08:51:36.368540+0000","last_became_active":"2024-04-24T08:48:29.977729+0000","last_became_peered":"2024-04-24T08:48:29.977729+0000","last_unstale":"2024-04-24T08:51:36.368540+0000","last_undegraded":"2024-04-24T08:51:36.368540+0000","last_fullsized":"2024-04-24T08:51:36.368540+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:03:33.382473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.657826+0000","last_change":"2024-04-24T08:48:29.976355+0000","last_active":"2024-04-24T08:51:36.657826+0000","last_peered":"2024-04-24T08:51:36.657826+0000","last_clean":"2024-04-24T08:51:36.657826+0000","last_became_active":"2024-04-24T08:48:29.975849+0000","last_became_peered":"2024-04-24T08:48:29.975849+0000","last_unstale":"2024-04-24T08:51:36.657826+0000","last_undegraded":"2024-04-24T08:51:36.657826+0000","last_fullsized":"2024-04-24T08:51:36.657826+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:13:51.302995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.359785+0000","last_change":"2024-04-24T08:48:29.983149+0000","last_active":"2024-04-24T08:51:36.359785+0000","last_peered":"2024-04-24T08:51:36.359785+0000","last_clean":"2024-04-24T08:51:36.359785+0000","last_became_active":"2024-04-24T08:48:29.982692+0000","last_became_peered":"2024-04-24T08:48:29.982692+0000","last_unstale":"2024-04-24T08:51:36.359785+0000","last_undegraded":"2024-04-24T08:51:36.359785+0000","last_fullsized":"2024-04-24T08:51:36.359785+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:59:06.151441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.893672+0000","last_change":"2024-04-24T08:48:29.985531+0000","last_active":"2024-04-24T08:51:36.893672+0000","last_peered":"2024-04-24T08:51:36.893672+0000","last_clean":"2024-04-24T08:51:36.893672+0000","last_became_active":"2024-04-24T08:48:29.982484+0000","last_became_peered":"2024-04-24T08:48:29.982484+0000","last_unstale":"2024-04-24T08:51:36.893672+0000","last_undegraded":"2024-04-24T08:51:36.893672+0000","last_fullsized":"2024-04-24T08:51:36.893672+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:03:50.114096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":47,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.368859+0000","last_change":"2024-04-24T08:48:29.982016+0000","last_active":"2024-04-24T08:51:36.368859+0000","last_peered":"2024-04-24T08:51:36.368859+0000","last_clean":"2024-04-24T08:51:36.368859+0000","last_became_active":"2024-04-24T08:48:29.977639+0000","last_became_peered":"2024-04-24T08:48:29.977639+0000","last_unstale":"2024-04-24T08:51:36.368859+0000","last_undegraded":"2024-04-24T08:51:36.368859+0000","last_fullsized":"2024-04-24T08:51:36.368859+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:35:48.676822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.659055+0000","last_change":"2024-04-24T08:48:29.977982+0000","last_active":"2024-04-24T08:51:36.659055+0000","last_peered":"2024-04-24T08:51:36.659055+0000","last_clean":"2024-04-24T08:51:36.659055+0000","last_became_active":"2024-04-24T08:48:29.977664+0000","last_became_peered":"2024-04-24T08:48:29.977664+0000","last_unstale":"2024-04-24T08:51:36.659055+0000","last_undegraded":"2024-04-24T08:51:36.659055+0000","last_fullsized":"2024-04-24T08:51:36.659055+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:45:17.597291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.738302+0000","last_change":"2024-04-24T08:48:29.981615+0000","last_active":"2024-04-24T08:51:36.738302+0000","last_peered":"2024-04-24T08:51:36.738302+0000","last_clean":"2024-04-24T08:51:36.738302+0000","last_became_active":"2024-04-24T08:48:29.979292+0000","last_became_peered":"2024-04-24T08:48:29.979292+0000","last_unstale":"2024-04-24T08:51:36.738302+0000","last_undegraded":"2024-04-24T08:51:36.738302+0000","last_fullsized":"2024-04-24T08:51:36.738302+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:48:44.195626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.730843+0000","last_change":"2024-04-24T08:48:29.952632+0000","last_active":"2024-04-24T08:51:36.730843+0000","last_peered":"2024-04-24T08:51:36.730843+0000","last_clean":"2024-04-24T08:51:36.730843+0000","last_became_active":"2024-04-24T08:48:29.951862+0000","last_became_peered":"2024-04-24T08:48:29.951862+0000","last_unstale":"2024-04-24T08:51:36.730843+0000","last_undegraded":"2024-04-24T08:51:36.730843+0000","last_fullsized":"2024-04-24T08:51:36.730843+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:23:40.038517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.893175+0000","last_change":"2024-04-24T08:48:29.984727+0000","last_active":"2024-04-24T08:51:36.893175+0000","last_peered":"2024-04-24T08:51:36.893175+0000","last_clean":"2024-04-24T08:51:36.893175+0000","last_became_active":"2024-04-24T08:48:29.983530+0000","last_became_peered":"2024-04-24T08:48:29.983530+0000","last_unstale":"2024-04-24T08:51:36.893175+0000","last_undegraded":"2024-04-24T08:51:36.893175+0000","last_fullsized":"2024-04-24T08:51:36.893175+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:00:12.729972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.893421+0000","last_change":"2024-04-24T08:48:29.981023+0000","last_active":"2024-04-24T08:51:36.893421+0000","last_peered":"2024-04-24T08:51:36.893421+0000","last_clean":"2024-04-24T08:51:36.893421+0000","last_became_active":"2024-04-24T08:48:29.979986+0000","last_became_peered":"2024-04-24T08:48:29.979986+0000","last_unstale":"2024-04-24T08:51:36.893421+0000","last_undegraded":"2024-04-24T08:51:36.893421+0000","last_fullsized":"2024-04-24T08:51:36.893421+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:05:36.163209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.892674+0000","last_change":"2024-04-24T08:48:29.989036+0000","last_active":"2024-04-24T08:51:36.892674+0000","last_peered":"2024-04-24T08:51:36.892674+0000","last_clean":"2024-04-24T08:51:36.892674+0000","last_became_active":"2024-04-24T08:48:29.988676+0000","last_became_peered":"2024-04-24T08:48:29.988676+0000","last_unstale":"2024-04-24T08:51:36.892674+0000","last_undegraded":"2024-04-24T08:51:36.892674+0000","last_fullsized":"2024-04-24T08:51:36.892674+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:03:38.544596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.702193+0000","last_change":"2024-04-24T08:48:29.986424+0000","last_active":"2024-04-24T08:51:36.702193+0000","last_peered":"2024-04-24T08:51:36.702193+0000","last_clean":"2024-04-24T08:51:36.702193+0000","last_became_active":"2024-04-24T08:48:29.985973+0000","last_became_peered":"2024-04-24T08:48:29.985973+0000","last_unstale":"2024-04-24T08:51:36.702193+0000","last_undegraded":"2024-04-24T08:51:36.702193+0000","last_fullsized":"2024-04-24T08:51:36.702193+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:48:31.533282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.498447+0000","last_change":"2024-04-24T08:48:29.975470+0000","last_active":"2024-04-24T08:51:36.498447+0000","last_peered":"2024-04-24T08:51:36.498447+0000","last_clean":"2024-04-24T08:51:36.498447+0000","last_became_active":"2024-04-24T08:48:29.974777+0000","last_became_peered":"2024-04-24T08:48:29.974777+0000","last_unstale":"2024-04-24T08:51:36.498447+0000","last_undegraded":"2024-04-24T08:51:36.498447+0000","last_fullsized":"2024-04-24T08:51:36.498447+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:04:26.423840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.368791+0000","last_change":"2024-04-24T08:48:29.988001+0000","last_active":"2024-04-24T08:51:36.368791+0000","last_peered":"2024-04-24T08:51:36.368791+0000","last_clean":"2024-04-24T08:51:36.368791+0000","last_became_active":"2024-04-24T08:48:29.987669+0000","last_became_peered":"2024-04-24T08:48:29.987669+0000","last_unstale":"2024-04-24T08:51:36.368791+0000","last_undegraded":"2024-04-24T08:51:36.368791+0000","last_fullsized":"2024-04-24T08:51:36.368791+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:29:24.457369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.367087+0000","last_change":"2024-04-24T08:48:34.308732+0000","last_active":"2024-04-24T08:51:36.367087+0000","last_peered":"2024-04-24T08:51:36.367087+0000","last_clean":"2024-04-24T08:51:36.367087+0000","last_became_active":"2024-04-24T08:48:34.307598+0000","last_became_peered":"2024-04-24T08:48:34.307598+0000","last_unstale":"2024-04-24T08:51:36.367087+0000","last_undegraded":"2024-04-24T08:51:36.367087+0000","last_fullsized":"2024-04-24T08:51:36.367087+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:28:57.260836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"93'14","reported_seq":201,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.499394+0000","last_change":"2024-04-24T08:48:28.970230+0000","last_active":"2024-04-24T08:51:36.499394+0000","last_peered":"2024-04-24T08:51:36.499394+0000","last_clean":"2024-04-24T08:51:36.499394+0000","last_became_active":"2024-04-24T08:48:28.969878+0000","last_became_peered":"2024-04-24T08:48:28.969878+0000","last_unstale":"2024-04-24T08:51:36.499394+0000","last_undegraded":"2024-04-24T08:51:36.499394+0000","last_fullsized":"2024-04-24T08:51:36.499394+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:05:19.206557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":154,"num_read_kb":154,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.360088+0000","last_change":"2024-04-24T08:48:29.986350+0000","last_active":"2024-04-24T08:51:36.360088+0000","last_peered":"2024-04-24T08:51:36.360088+0000","last_clean":"2024-04-24T08:51:36.360088+0000","last_became_active":"2024-04-24T08:48:29.985167+0000","last_became_peered":"2024-04-24T08:48:29.985167+0000","last_unstale":"2024-04-24T08:51:36.360088+0000","last_undegraded":"2024-04-24T08:51:36.360088+0000","last_fullsized":"2024-04-24T08:51:36.360088+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:43:33.326808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.736357+0000","last_change":"2024-04-24T08:48:34.315159+0000","last_active":"2024-04-24T08:51:36.736357+0000","last_peered":"2024-04-24T08:51:36.736357+0000","last_clean":"2024-04-24T08:51:36.736357+0000","last_became_active":"2024-04-24T08:48:34.312119+0000","last_became_peered":"2024-04-24T08:48:34.312119+0000","last_unstale":"2024-04-24T08:51:36.736357+0000","last_undegraded":"2024-04-24T08:51:36.736357+0000","last_fullsized":"2024-04-24T08:51:36.736357+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:56:08.024123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.360224+0000","last_change":"2024-04-24T08:48:28.971787+0000","last_active":"2024-04-24T08:51:36.360224+0000","last_peered":"2024-04-24T08:51:36.360224+0000","last_clean":"2024-04-24T08:51:36.360224+0000","last_became_active":"2024-04-24T08:48:28.971479+0000","last_became_peered":"2024-04-24T08:48:28.971479+0000","last_unstale":"2024-04-24T08:51:36.360224+0000","last_undegraded":"2024-04-24T08:51:36.360224+0000","last_fullsized":"2024-04-24T08:51:36.360224+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:09:52.498833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.893290+0000","last_change":"2024-04-24T08:48:29.989737+0000","last_active":"2024-04-24T08:51:36.893290+0000","last_peered":"2024-04-24T08:51:36.893290+0000","last_clean":"2024-04-24T08:51:36.893290+0000","last_became_active":"2024-04-24T08:48:29.989341+0000","last_became_peered":"2024-04-24T08:48:29.989341+0000","last_unstale":"2024-04-24T08:51:36.893290+0000","last_undegraded":"2024-04-24T08:51:36.893290+0000","last_fullsized":"2024-04-24T08:51:36.893290+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:02:32.714848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.736714+0000","last_change":"2024-04-24T08:48:34.322666+0000","last_active":"2024-04-24T08:51:36.736714+0000","last_peered":"2024-04-24T08:51:36.736714+0000","last_clean":"2024-04-24T08:51:36.736714+0000","last_became_active":"2024-04-24T08:48:34.321546+0000","last_became_peered":"2024-04-24T08:48:34.321546+0000","last_unstale":"2024-04-24T08:51:36.736714+0000","last_undegraded":"2024-04-24T08:51:36.736714+0000","last_fullsized":"2024-04-24T08:51:36.736714+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:42:45.066043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"94'13","reported_seq":70,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.505002+0000","last_change":"2024-04-24T08:50:43.151013+0000","last_active":"2024-04-24T08:51:36.505002+0000","last_peered":"2024-04-24T08:51:36.505002+0000","last_clean":"2024-04-24T08:51:36.505002+0000","last_became_active":"2024-04-24T08:50:43.123152+0000","last_became_peered":"2024-04-24T08:50:43.123152+0000","last_unstale":"2024-04-24T08:51:36.505002+0000","last_undegraded":"2024-04-24T08:51:36.505002+0000","last_fullsized":"2024-04-24T08:51:36.505002+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:56:45.260680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,7],"acting":[8,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.893187+0000","last_change":"2024-04-24T08:48:29.974588+0000","last_active":"2024-04-24T08:51:36.893187+0000","last_peered":"2024-04-24T08:51:36.893187+0000","last_clean":"2024-04-24T08:51:36.893187+0000","last_became_active":"2024-04-24T08:48:29.974190+0000","last_became_peered":"2024-04-24T08:48:29.974190+0000","last_unstale":"2024-04-24T08:51:36.893187+0000","last_undegraded":"2024-04-24T08:51:36.893187+0000","last_fullsized":"2024-04-24T08:51:36.893187+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:07:12.948660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.736446+0000","last_change":"2024-04-24T08:48:34.321954+0000","last_active":"2024-04-24T08:51:36.736446+0000","last_peered":"2024-04-24T08:51:36.736446+0000","last_clean":"2024-04-24T08:51:36.736446+0000","last_became_active":"2024-04-24T08:48:34.321417+0000","last_became_peered":"2024-04-24T08:48:34.321417+0000","last_unstale":"2024-04-24T08:51:36.736446+0000","last_undegraded":"2024-04-24T08:51:36.736446+0000","last_fullsized":"2024-04-24T08:51:36.736446+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:39:07.205709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.504769+0000","last_change":"2024-04-24T08:48:28.974464+0000","last_active":"2024-04-24T08:51:36.504769+0000","last_peered":"2024-04-24T08:51:36.504769+0000","last_clean":"2024-04-24T08:51:36.504769+0000","last_became_active":"2024-04-24T08:48:28.974001+0000","last_became_peered":"2024-04-24T08:48:28.974001+0000","last_unstale":"2024-04-24T08:51:36.504769+0000","last_undegraded":"2024-04-24T08:51:36.504769+0000","last_fullsized":"2024-04-24T08:51:36.504769+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:38:55.193427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":47,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.514535+0000","last_change":"2024-04-24T08:48:29.981336+0000","last_active":"2024-04-24T08:51:36.514535+0000","last_peered":"2024-04-24T08:51:36.514535+0000","last_clean":"2024-04-24T08:51:36.514535+0000","last_became_active":"2024-04-24T08:48:29.980333+0000","last_became_peered":"2024-04-24T08:48:29.980333+0000","last_unstale":"2024-04-24T08:51:36.514535+0000","last_undegraded":"2024-04-24T08:51:36.514535+0000","last_fullsized":"2024-04-24T08:51:36.514535+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:46:03.640332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.368326+0000","last_change":"2024-04-24T08:48:34.323099+0000","last_active":"2024-04-24T08:51:36.368326+0000","last_peered":"2024-04-24T08:51:36.368326+0000","last_clean":"2024-04-24T08:51:36.368326+0000","last_became_active":"2024-04-24T08:48:34.322557+0000","last_became_peered":"2024-04-24T08:48:34.322557+0000","last_unstale":"2024-04-24T08:51:36.368326+0000","last_undegraded":"2024-04-24T08:51:36.368326+0000","last_fullsized":"2024-04-24T08:51:36.368326+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:59:59.887087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"94'9","reported_seq":200,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.699273+0000","last_change":"2024-04-24T08:48:28.978427+0000","last_active":"2024-04-24T08:51:36.699273+0000","last_peered":"2024-04-24T08:51:36.699273+0000","last_clean":"2024-04-24T08:51:36.699273+0000","last_became_active":"2024-04-24T08:48:28.977291+0000","last_became_peered":"2024-04-24T08:48:28.977291+0000","last_unstale":"2024-04-24T08:51:36.699273+0000","last_undegraded":"2024-04-24T08:51:36.699273+0000","last_fullsized":"2024-04-24T08:51:36.699273+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:03:03.085265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2038,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":157,"num_read_kb":4,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.503358+0000","last_change":"2024-04-24T08:48:29.978008+0000","last_active":"2024-04-24T08:51:36.503358+0000","last_peered":"2024-04-24T08:51:36.503358+0000","last_clean":"2024-04-24T08:51:36.503358+0000","last_became_active":"2024-04-24T08:48:29.977389+0000","last_became_peered":"2024-04-24T08:48:29.977389+0000","last_unstale":"2024-04-24T08:51:36.503358+0000","last_undegraded":"2024-04-24T08:51:36.503358+0000","last_fullsized":"2024-04-24T08:51:36.503358+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:35:28.035378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.368135+0000","last_change":"2024-04-24T08:48:34.321477+0000","last_active":"2024-04-24T08:51:36.368135+0000","last_peered":"2024-04-24T08:51:36.368135+0000","last_clean":"2024-04-24T08:51:36.368135+0000","last_became_active":"2024-04-24T08:48:34.320860+0000","last_became_peered":"2024-04-24T08:48:34.320860+0000","last_unstale":"2024-04-24T08:51:36.368135+0000","last_undegraded":"2024-04-24T08:51:36.368135+0000","last_fullsized":"2024-04-24T08:51:36.368135+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:26:07.862590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":54,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.368151+0000","last_change":"2024-04-24T08:50:43.014290+0000","last_active":"2024-04-24T08:51:36.368151+0000","last_peered":"2024-04-24T08:51:36.368151+0000","last_clean":"2024-04-24T08:51:36.368151+0000","last_became_active":"2024-04-24T08:50:42.995733+0000","last_became_peered":"2024-04-24T08:50:42.995733+0000","last_unstale":"2024-04-24T08:51:36.368151+0000","last_undegraded":"2024-04-24T08:51:36.368151+0000","last_fullsized":"2024-04-24T08:51:36.368151+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:54:50.270110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.514617+0000","last_change":"2024-04-24T08:48:29.990788+0000","last_active":"2024-04-24T08:51:36.514617+0000","last_peered":"2024-04-24T08:51:36.514617+0000","last_clean":"2024-04-24T08:51:36.514617+0000","last_became_active":"2024-04-24T08:48:29.988476+0000","last_became_peered":"2024-04-24T08:48:29.988476+0000","last_unstale":"2024-04-24T08:51:36.514617+0000","last_undegraded":"2024-04-24T08:51:36.514617+0000","last_fullsized":"2024-04-24T08:51:36.514617+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:08:48.389877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.701568+0000","last_change":"2024-04-24T08:48:34.311020+0000","last_active":"2024-04-24T08:51:36.701568+0000","last_peered":"2024-04-24T08:51:36.701568+0000","last_clean":"2024-04-24T08:51:36.701568+0000","last_became_active":"2024-04-24T08:48:34.310395+0000","last_became_peered":"2024-04-24T08:48:34.310395+0000","last_unstale":"2024-04-24T08:51:36.701568+0000","last_undegraded":"2024-04-24T08:51:36.701568+0000","last_fullsized":"2024-04-24T08:51:36.701568+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:55:38.704954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.729835+0000","last_change":"2024-04-24T08:48:28.969991+0000","last_active":"2024-04-24T08:51:36.729835+0000","last_peered":"2024-04-24T08:51:36.729835+0000","last_clean":"2024-04-24T08:51:36.729835+0000","last_became_active":"2024-04-24T08:48:28.967937+0000","last_became_peered":"2024-04-24T08:48:28.967937+0000","last_unstale":"2024-04-24T08:51:36.729835+0000","last_undegraded":"2024-04-24T08:51:36.729835+0000","last_fullsized":"2024-04-24T08:51:36.729835+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:09:49.736108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.359593+0000","last_change":"2024-04-24T08:48:29.992814+0000","last_active":"2024-04-24T08:51:36.359593+0000","last_peered":"2024-04-24T08:51:36.359593+0000","last_clean":"2024-04-24T08:51:36.359593+0000","last_became_active":"2024-04-24T08:48:29.992188+0000","last_became_peered":"2024-04-24T08:48:29.992188+0000","last_unstale":"2024-04-24T08:51:36.359593+0000","last_undegraded":"2024-04-24T08:51:36.359593+0000","last_fullsized":"2024-04-24T08:51:36.359593+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:29:33.974958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.893584+0000","last_change":"2024-04-24T08:48:34.311196+0000","last_active":"2024-04-24T08:51:36.893584+0000","last_peered":"2024-04-24T08:51:36.893584+0000","last_clean":"2024-04-24T08:51:36.893584+0000","last_became_active":"2024-04-24T08:48:34.310723+0000","last_became_peered":"2024-04-24T08:48:34.310723+0000","last_unstale":"2024-04-24T08:51:36.893584+0000","last_undegraded":"2024-04-24T08:51:36.893584+0000","last_fullsized":"2024-04-24T08:51:36.893584+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:04:30.149510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.513501+0000","last_change":"2024-04-24T08:48:28.978534+0000","last_active":"2024-04-24T08:51:36.513501+0000","last_peered":"2024-04-24T08:51:36.513501+0000","last_clean":"2024-04-24T08:51:36.513501+0000","last_became_active":"2024-04-24T08:48:28.975407+0000","last_became_peered":"2024-04-24T08:48:28.975407+0000","last_unstale":"2024-04-24T08:51:36.513501+0000","last_undegraded":"2024-04-24T08:51:36.513501+0000","last_fullsized":"2024-04-24T08:51:36.513501+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:10:45.355039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.497595+0000","last_change":"2024-04-24T08:48:29.997682+0000","last_active":"2024-04-24T08:51:36.497595+0000","last_peered":"2024-04-24T08:51:36.497595+0000","last_clean":"2024-04-24T08:51:36.497595+0000","last_became_active":"2024-04-24T08:48:29.997223+0000","last_became_peered":"2024-04-24T08:48:29.997223+0000","last_unstale":"2024-04-24T08:51:36.497595+0000","last_undegraded":"2024-04-24T08:51:36.497595+0000","last_fullsized":"2024-04-24T08:51:36.497595+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:03:00.807249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.359842+0000","last_change":"2024-04-24T08:48:34.321477+0000","last_active":"2024-04-24T08:51:36.359842+0000","last_peered":"2024-04-24T08:51:36.359842+0000","last_clean":"2024-04-24T08:51:36.359842+0000","last_became_active":"2024-04-24T08:48:34.321018+0000","last_became_peered":"2024-04-24T08:48:34.321018+0000","last_unstale":"2024-04-24T08:51:36.359842+0000","last_undegraded":"2024-04-24T08:51:36.359842+0000","last_fullsized":"2024-04-24T08:51:36.359842+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:56:50.657298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"93'7","reported_seq":346,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.680385+0000","last_change":"2024-04-24T08:48:28.983722+0000","last_active":"2024-04-24T08:51:36.680385+0000","last_peered":"2024-04-24T08:51:36.680385+0000","last_clean":"2024-04-24T08:51:36.680385+0000","last_became_active":"2024-04-24T08:48:28.983394+0000","last_became_peered":"2024-04-24T08:48:28.983394+0000","last_unstale":"2024-04-24T08:51:36.680385+0000","last_undegraded":"2024-04-24T08:51:36.680385+0000","last_fullsized":"2024-04-24T08:51:36.680385+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:53:14.288742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":306,"num_read_kb":306,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.893545+0000","last_change":"2024-04-24T08:48:29.975009+0000","last_active":"2024-04-24T08:51:36.893545+0000","last_peered":"2024-04-24T08:51:36.893545+0000","last_clean":"2024-04-24T08:51:36.893545+0000","last_became_active":"2024-04-24T08:48:29.974137+0000","last_became_peered":"2024-04-24T08:48:29.974137+0000","last_unstale":"2024-04-24T08:51:36.893545+0000","last_undegraded":"2024-04-24T08:51:36.893545+0000","last_fullsized":"2024-04-24T08:51:36.893545+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:13:16.381923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.497502+0000","last_change":"2024-04-24T08:48:34.312126+0000","last_active":"2024-04-24T08:51:36.497502+0000","last_peered":"2024-04-24T08:51:36.497502+0000","last_clean":"2024-04-24T08:51:36.497502+0000","last_became_active":"2024-04-24T08:48:34.311689+0000","last_became_peered":"2024-04-24T08:48:34.311689+0000","last_unstale":"2024-04-24T08:51:36.497502+0000","last_undegraded":"2024-04-24T08:51:36.497502+0000","last_fullsized":"2024-04-24T08:51:36.497502+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:11:07.942412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.659196+0000","last_change":"2024-04-24T08:48:28.978355+0000","last_active":"2024-04-24T08:51:36.659196+0000","last_peered":"2024-04-24T08:51:36.659196+0000","last_clean":"2024-04-24T08:51:36.659196+0000","last_became_active":"2024-04-24T08:48:28.977351+0000","last_became_peered":"2024-04-24T08:48:28.977351+0000","last_unstale":"2024-04-24T08:51:36.659196+0000","last_undegraded":"2024-04-24T08:51:36.659196+0000","last_fullsized":"2024-04-24T08:51:36.659196+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:00:12.836569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.893780+0000","last_change":"2024-04-24T08:48:29.989015+0000","last_active":"2024-04-24T08:51:36.893780+0000","last_peered":"2024-04-24T08:51:36.893780+0000","last_clean":"2024-04-24T08:51:36.893780+0000","last_became_active":"2024-04-24T08:48:29.988648+0000","last_became_peered":"2024-04-24T08:48:29.988648+0000","last_unstale":"2024-04-24T08:51:36.893780+0000","last_undegraded":"2024-04-24T08:51:36.893780+0000","last_fullsized":"2024-04-24T08:51:36.893780+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:04:52.824894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.737315+0000","last_change":"2024-04-24T08:48:34.314230+0000","last_active":"2024-04-24T08:51:36.737315+0000","last_peered":"2024-04-24T08:51:36.737315+0000","last_clean":"2024-04-24T08:51:36.737315+0000","last_became_active":"2024-04-24T08:48:34.313617+0000","last_became_peered":"2024-04-24T08:48:34.313617+0000","last_unstale":"2024-04-24T08:51:36.737315+0000","last_undegraded":"2024-04-24T08:51:36.737315+0000","last_fullsized":"2024-04-24T08:51:36.737315+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:06:02.307575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.514177+0000","last_change":"2024-04-24T08:48:28.974516+0000","last_active":"2024-04-24T08:51:36.514177+0000","last_peered":"2024-04-24T08:51:36.514177+0000","last_clean":"2024-04-24T08:51:36.514177+0000","last_became_active":"2024-04-24T08:48:28.974194+0000","last_became_peered":"2024-04-24T08:48:28.974194+0000","last_unstale":"2024-04-24T08:51:36.514177+0000","last_undegraded":"2024-04-24T08:51:36.514177+0000","last_fullsized":"2024-04-24T08:51:36.514177+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:03:38.582610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":48,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.893356+0000","last_change":"2024-04-24T08:49:42.128553+0000","last_active":"2024-04-24T08:51:36.893356+0000","last_peered":"2024-04-24T08:51:36.893356+0000","last_clean":"2024-04-24T08:51:36.893356+0000","last_became_active":"2024-04-24T08:49:42.127061+0000","last_became_peered":"2024-04-24T08:49:42.127061+0000","last_unstale":"2024-04-24T08:51:36.893356+0000","last_undegraded":"2024-04-24T08:51:36.893356+0000","last_fullsized":"2024-04-24T08:51:36.893356+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:07:06.798284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.367516+0000","last_change":"2024-04-24T08:49:42.131305+0000","last_active":"2024-04-24T08:51:36.367516+0000","last_peered":"2024-04-24T08:51:36.367516+0000","last_clean":"2024-04-24T08:51:36.367516+0000","last_became_active":"2024-04-24T08:49:42.129680+0000","last_became_peered":"2024-04-24T08:49:42.129680+0000","last_unstale":"2024-04-24T08:51:36.367516+0000","last_undegraded":"2024-04-24T08:51:36.367516+0000","last_fullsized":"2024-04-24T08:51:36.367516+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:14:09.579649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"94'40","reported_seq":387,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.894275+0000","last_change":"2024-04-24T08:48:28.981675+0000","last_active":"2024-04-24T08:51:36.894275+0000","last_peered":"2024-04-24T08:51:36.894275+0000","last_clean":"2024-04-24T08:51:36.894275+0000","last_became_active":"2024-04-24T08:48:28.981082+0000","last_became_peered":"2024-04-24T08:48:28.981082+0000","last_unstale":"2024-04-24T08:51:36.894275+0000","last_undegraded":"2024-04-24T08:51:36.894275+0000","last_fullsized":"2024-04-24T08:51:36.894275+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:22:07.826011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66989,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":314,"num_read_kb":217,"num_write":42,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.504369+0000","last_change":"2024-04-24T08:48:29.986932+0000","last_active":"2024-04-24T08:51:36.504369+0000","last_peered":"2024-04-24T08:51:36.504369+0000","last_clean":"2024-04-24T08:51:36.504369+0000","last_became_active":"2024-04-24T08:48:29.986505+0000","last_became_peered":"2024-04-24T08:48:29.986505+0000","last_unstale":"2024-04-24T08:51:36.504369+0000","last_undegraded":"2024-04-24T08:51:36.504369+0000","last_fullsized":"2024-04-24T08:51:36.504369+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:53:36.050294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.515187+0000","last_change":"2024-04-24T08:48:34.327127+0000","last_active":"2024-04-24T08:51:36.515187+0000","last_peered":"2024-04-24T08:51:36.515187+0000","last_clean":"2024-04-24T08:51:36.515187+0000","last_became_active":"2024-04-24T08:48:34.323942+0000","last_became_peered":"2024-04-24T08:48:34.323942+0000","last_unstale":"2024-04-24T08:51:36.515187+0000","last_undegraded":"2024-04-24T08:51:36.515187+0000","last_fullsized":"2024-04-24T08:51:36.515187+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:22:49.637513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.367615+0000","last_change":"2024-04-24T08:48:28.978618+0000","last_active":"2024-04-24T08:51:36.367615+0000","last_peered":"2024-04-24T08:51:36.367615+0000","last_clean":"2024-04-24T08:51:36.367615+0000","last_became_active":"2024-04-24T08:48:28.978309+0000","last_became_peered":"2024-04-24T08:48:28.978309+0000","last_unstale":"2024-04-24T08:51:36.367615+0000","last_undegraded":"2024-04-24T08:51:36.367615+0000","last_fullsized":"2024-04-24T08:51:36.367615+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:35:50.091398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.729979+0000","last_change":"2024-04-24T08:48:29.985629+0000","last_active":"2024-04-24T08:51:36.729979+0000","last_peered":"2024-04-24T08:51:36.729979+0000","last_clean":"2024-04-24T08:51:36.729979+0000","last_became_active":"2024-04-24T08:48:29.985258+0000","last_became_peered":"2024-04-24T08:48:29.985258+0000","last_unstale":"2024-04-24T08:51:36.729979+0000","last_undegraded":"2024-04-24T08:51:36.729979+0000","last_fullsized":"2024-04-24T08:51:36.729979+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:57:29.302175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.701030+0000","last_change":"2024-04-24T08:49:42.129242+0000","last_active":"2024-04-24T08:51:36.701030+0000","last_peered":"2024-04-24T08:51:36.701030+0000","last_clean":"2024-04-24T08:51:36.701030+0000","last_became_active":"2024-04-24T08:49:42.127967+0000","last_became_peered":"2024-04-24T08:49:42.127967+0000","last_unstale":"2024-04-24T08:51:36.701030+0000","last_undegraded":"2024-04-24T08:51:36.701030+0000","last_fullsized":"2024-04-24T08:51:36.701030+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:42:11.206123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.513770+0000","last_change":"2024-04-24T08:48:28.977840+0000","last_active":"2024-04-24T08:51:36.513770+0000","last_peered":"2024-04-24T08:51:36.513770+0000","last_clean":"2024-04-24T08:51:36.513770+0000","last_became_active":"2024-04-24T08:48:28.975482+0000","last_became_peered":"2024-04-24T08:48:28.975482+0000","last_unstale":"2024-04-24T08:51:36.513770+0000","last_undegraded":"2024-04-24T08:51:36.513770+0000","last_fullsized":"2024-04-24T08:51:36.513770+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:51:52.468627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":52,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.658628+0000","last_change":"2024-04-24T08:49:42.129823+0000","last_active":"2024-04-24T08:51:36.658628+0000","last_peered":"2024-04-24T08:51:36.658628+0000","last_clean":"2024-04-24T08:51:36.658628+0000","last_became_active":"2024-04-24T08:49:42.127254+0000","last_became_peered":"2024-04-24T08:49:42.127254+0000","last_unstale":"2024-04-24T08:51:36.658628+0000","last_undegraded":"2024-04-24T08:51:36.658628+0000","last_fullsized":"2024-04-24T08:51:36.658628+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:51:10.804703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.499181+0000","last_change":"2024-04-24T08:48:34.314956+0000","last_active":"2024-04-24T08:51:36.499181+0000","last_peered":"2024-04-24T08:51:36.499181+0000","last_clean":"2024-04-24T08:51:36.499181+0000","last_became_active":"2024-04-24T08:48:34.314541+0000","last_became_peered":"2024-04-24T08:48:34.314541+0000","last_unstale":"2024-04-24T08:51:36.499181+0000","last_undegraded":"2024-04-24T08:51:36.499181+0000","last_fullsized":"2024-04-24T08:51:36.499181+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:42:48.614000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"93'6","reported_seq":196,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.737247+0000","last_change":"2024-04-24T08:48:28.974529+0000","last_active":"2024-04-24T08:51:36.737247+0000","last_peered":"2024-04-24T08:51:36.737247+0000","last_clean":"2024-04-24T08:51:36.737247+0000","last_became_active":"2024-04-24T08:48:28.973957+0000","last_became_peered":"2024-04-24T08:48:28.973957+0000","last_unstale":"2024-04-24T08:51:36.737247+0000","last_undegraded":"2024-04-24T08:51:36.737247+0000","last_fullsized":"2024-04-24T08:51:36.737247+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:44:46.216389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1081,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":157,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.658134+0000","last_change":"2024-04-24T08:48:29.981995+0000","last_active":"2024-04-24T08:51:36.658134+0000","last_peered":"2024-04-24T08:51:36.658134+0000","last_clean":"2024-04-24T08:51:36.658134+0000","last_became_active":"2024-04-24T08:48:29.980756+0000","last_became_peered":"2024-04-24T08:48:29.980756+0000","last_unstale":"2024-04-24T08:51:36.658134+0000","last_undegraded":"2024-04-24T08:51:36.658134+0000","last_fullsized":"2024-04-24T08:51:36.658134+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:42:52.770069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.731767+0000","last_change":"2024-04-24T08:48:34.308836+0000","last_active":"2024-04-24T08:51:36.731767+0000","last_peered":"2024-04-24T08:51:36.731767+0000","last_clean":"2024-04-24T08:51:36.731767+0000","last_became_active":"2024-04-24T08:48:34.308470+0000","last_became_peered":"2024-04-24T08:48:34.308470+0000","last_unstale":"2024-04-24T08:51:36.731767+0000","last_undegraded":"2024-04-24T08:51:36.731767+0000","last_fullsized":"2024-04-24T08:51:36.731767+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:38:14.906117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":76,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.503278+0000","last_change":"2024-04-24T08:50:43.053334+0000","last_active":"2024-04-24T08:51:36.503278+0000","last_peered":"2024-04-24T08:51:36.503278+0000","last_clean":"2024-04-24T08:51:36.503278+0000","last_became_active":"2024-04-24T08:50:43.007120+0000","last_became_peered":"2024-04-24T08:50:43.007120+0000","last_unstale":"2024-04-24T08:51:36.503278+0000","last_undegraded":"2024-04-24T08:51:36.503278+0000","last_fullsized":"2024-04-24T08:51:36.503278+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:27:37.158069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.513585+0000","last_change":"2024-04-24T08:48:29.980632+0000","last_active":"2024-04-24T08:51:36.513585+0000","last_peered":"2024-04-24T08:51:36.513585+0000","last_clean":"2024-04-24T08:51:36.513585+0000","last_became_active":"2024-04-24T08:48:29.980174+0000","last_became_peered":"2024-04-24T08:48:29.980174+0000","last_unstale":"2024-04-24T08:51:36.513585+0000","last_undegraded":"2024-04-24T08:51:36.513585+0000","last_fullsized":"2024-04-24T08:51:36.513585+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:45:32.709091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.513372+0000","last_change":"2024-04-24T08:48:34.316302+0000","last_active":"2024-04-24T08:51:36.513372+0000","last_peered":"2024-04-24T08:51:36.513372+0000","last_clean":"2024-04-24T08:51:36.513372+0000","last_became_active":"2024-04-24T08:48:34.315783+0000","last_became_peered":"2024-04-24T08:48:34.315783+0000","last_unstale":"2024-04-24T08:51:36.513372+0000","last_undegraded":"2024-04-24T08:51:36.513372+0000","last_fullsized":"2024-04-24T08:51:36.513372+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:29:01.733931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"93'3","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.701302+0000","last_change":"2024-04-24T08:50:43.023981+0000","last_active":"2024-04-24T08:51:36.701302+0000","last_peered":"2024-04-24T08:51:36.701302+0000","last_clean":"2024-04-24T08:51:36.701302+0000","last_became_active":"2024-04-24T08:50:43.003967+0000","last_became_peered":"2024-04-24T08:50:43.003967+0000","last_unstale":"2024-04-24T08:51:36.701302+0000","last_undegraded":"2024-04-24T08:51:36.701302+0000","last_fullsized":"2024-04-24T08:51:36.701302+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:13:22.381793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.701617+0000","last_change":"2024-04-24T08:49:41.975577+0000","last_active":"2024-04-24T08:51:36.701617+0000","last_peered":"2024-04-24T08:51:36.701617+0000","last_clean":"2024-04-24T08:51:36.701617+0000","last_became_active":"2024-04-24T08:49:41.975018+0000","last_became_peered":"2024-04-24T08:49:41.975018+0000","last_unstale":"2024-04-24T08:51:36.701617+0000","last_undegraded":"2024-04-24T08:51:36.701617+0000","last_fullsized":"2024-04-24T08:51:36.701617+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:07:53.051847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.503774+0000","last_change":"2024-04-24T08:49:41.978827+0000","last_active":"2024-04-24T08:51:36.503774+0000","last_peered":"2024-04-24T08:51:36.503774+0000","last_clean":"2024-04-24T08:51:36.503774+0000","last_became_active":"2024-04-24T08:49:41.977741+0000","last_became_peered":"2024-04-24T08:49:41.977741+0000","last_unstale":"2024-04-24T08:51:36.503774+0000","last_undegraded":"2024-04-24T08:51:36.503774+0000","last_fullsized":"2024-04-24T08:51:36.503774+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:16:54.754341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.498955+0000","last_change":"2024-04-24T08:48:28.970726+0000","last_active":"2024-04-24T08:51:36.498955+0000","last_peered":"2024-04-24T08:51:36.498955+0000","last_clean":"2024-04-24T08:51:36.498955+0000","last_became_active":"2024-04-24T08:48:28.969911+0000","last_became_peered":"2024-04-24T08:48:28.969911+0000","last_unstale":"2024-04-24T08:51:36.498955+0000","last_undegraded":"2024-04-24T08:51:36.498955+0000","last_fullsized":"2024-04-24T08:51:36.498955+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:55:26.357256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'4","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.369616+0000","last_change":"2024-04-24T08:48:29.967804+0000","last_active":"2024-04-24T08:51:36.369616+0000","last_peered":"2024-04-24T08:51:36.369616+0000","last_clean":"2024-04-24T08:51:36.369616+0000","last_became_active":"2024-04-24T08:48:29.967431+0000","last_became_peered":"2024-04-24T08:48:29.967431+0000","last_unstale":"2024-04-24T08:51:36.369616+0000","last_undegraded":"2024-04-24T08:51:36.369616+0000","last_fullsized":"2024-04-24T08:51:36.369616+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:00:02.779543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.731564+0000","last_change":"2024-04-24T08:48:34.316844+0000","last_active":"2024-04-24T08:51:36.731564+0000","last_peered":"2024-04-24T08:51:36.731564+0000","last_clean":"2024-04-24T08:51:36.731564+0000","last_became_active":"2024-04-24T08:48:34.316426+0000","last_became_peered":"2024-04-24T08:48:34.316426+0000","last_unstale":"2024-04-24T08:51:36.731564+0000","last_undegraded":"2024-04-24T08:51:36.731564+0000","last_fullsized":"2024-04-24T08:51:36.731564+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:10:13.378443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"94'7","reported_seq":195,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.658832+0000","last_change":"2024-04-24T08:48:28.971544+0000","last_active":"2024-04-24T08:51:36.658832+0000","last_peered":"2024-04-24T08:51:36.658832+0000","last_clean":"2024-04-24T08:51:36.658832+0000","last_became_active":"2024-04-24T08:48:28.970417+0000","last_became_peered":"2024-04-24T08:48:28.970417+0000","last_unstale":"2024-04-24T08:51:36.658832+0000","last_undegraded":"2024-04-24T08:51:36.658832+0000","last_fullsized":"2024-04-24T08:51:36.658832+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:35:04.506126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1472,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":155,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.368499+0000","last_change":"2024-04-24T08:48:29.987542+0000","last_active":"2024-04-24T08:51:36.368499+0000","last_peered":"2024-04-24T08:51:36.368499+0000","last_clean":"2024-04-24T08:51:36.368499+0000","last_became_active":"2024-04-24T08:48:29.987258+0000","last_became_peered":"2024-04-24T08:48:29.987258+0000","last_unstale":"2024-04-24T08:51:36.368499+0000","last_undegraded":"2024-04-24T08:51:36.368499+0000","last_fullsized":"2024-04-24T08:51:36.368499+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:37:23.341132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.732011+0000","last_change":"2024-04-24T08:48:34.322636+0000","last_active":"2024-04-24T08:51:36.732011+0000","last_peered":"2024-04-24T08:51:36.732011+0000","last_clean":"2024-04-24T08:51:36.732011+0000","last_became_active":"2024-04-24T08:48:34.321764+0000","last_became_peered":"2024-04-24T08:48:34.321764+0000","last_unstale":"2024-04-24T08:51:36.732011+0000","last_undegraded":"2024-04-24T08:51:36.732011+0000","last_fullsized":"2024-04-24T08:51:36.732011+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:56:10.563907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.514664+0000","last_change":"2024-04-24T08:48:29.981734+0000","last_active":"2024-04-24T08:51:36.514664+0000","last_peered":"2024-04-24T08:51:36.514664+0000","last_clean":"2024-04-24T08:51:36.514664+0000","last_became_active":"2024-04-24T08:48:29.980491+0000","last_became_peered":"2024-04-24T08:48:29.980491+0000","last_unstale":"2024-04-24T08:51:36.514664+0000","last_undegraded":"2024-04-24T08:51:36.514664+0000","last_fullsized":"2024-04-24T08:51:36.514664+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:08:45.189026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"93'6","reported_seq":194,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.729652+0000","last_change":"2024-04-24T08:48:28.954621+0000","last_active":"2024-04-24T08:51:36.729652+0000","last_peered":"2024-04-24T08:51:36.729652+0000","last_clean":"2024-04-24T08:51:36.729652+0000","last_became_active":"2024-04-24T08:48:28.954139+0000","last_became_peered":"2024-04-24T08:48:28.954139+0000","last_unstale":"2024-04-24T08:51:36.729652+0000","last_undegraded":"2024-04-24T08:51:36.729652+0000","last_fullsized":"2024-04-24T08:51:36.729652+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:59:19.360414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":154,"num_read_kb":154,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.658849+0000","last_change":"2024-04-24T08:48:34.323774+0000","last_active":"2024-04-24T08:51:36.658849+0000","last_peered":"2024-04-24T08:51:36.658849+0000","last_clean":"2024-04-24T08:51:36.658849+0000","last_became_active":"2024-04-24T08:48:34.322214+0000","last_became_peered":"2024-04-24T08:48:34.322214+0000","last_unstale":"2024-04-24T08:51:36.658849+0000","last_undegraded":"2024-04-24T08:51:36.658849+0000","last_fullsized":"2024-04-24T08:51:36.658849+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:51:03.971107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.360105+0000","last_change":"2024-04-24T08:48:29.995372+0000","last_active":"2024-04-24T08:51:36.360105+0000","last_peered":"2024-04-24T08:51:36.360105+0000","last_clean":"2024-04-24T08:51:36.360105+0000","last_became_active":"2024-04-24T08:48:29.994744+0000","last_became_peered":"2024-04-24T08:48:29.994744+0000","last_unstale":"2024-04-24T08:51:36.360105+0000","last_undegraded":"2024-04-24T08:51:36.360105+0000","last_fullsized":"2024-04-24T08:51:36.360105+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:28:55.356390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.737024+0000","last_change":"2024-04-24T08:48:28.974921+0000","last_active":"2024-04-24T08:51:36.737024+0000","last_peered":"2024-04-24T08:51:36.737024+0000","last_clean":"2024-04-24T08:51:36.737024+0000","last_became_active":"2024-04-24T08:48:28.974113+0000","last_became_peered":"2024-04-24T08:48:28.974113+0000","last_unstale":"2024-04-24T08:51:36.737024+0000","last_undegraded":"2024-04-24T08:51:36.737024+0000","last_fullsized":"2024-04-24T08:51:36.737024+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:35:12.635253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":367,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.657967+0000","last_change":"2024-04-24T08:46:45.190615+0000","last_active":"2024-04-24T08:51:36.657967+0000","last_peered":"2024-04-24T08:51:36.657967+0000","last_clean":"2024-04-24T08:51:36.657967+0000","last_became_active":"2024-04-24T08:46:45.190011+0000","last_became_peered":"2024-04-24T08:46:45.190011+0000","last_unstale":"2024-04-24T08:51:36.657967+0000","last_undegraded":"2024-04-24T08:51:36.657967+0000","last_fullsized":"2024-04-24T08:51:36.657967+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:41:57.806569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:41:57.806569+0000","last_clean_scrub_stamp":"2024-04-24T08:41:57.806569+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:13:08.283236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.513988+0000","last_change":"2024-04-24T08:48:34.324588+0000","last_active":"2024-04-24T08:51:36.513988+0000","last_peered":"2024-04-24T08:51:36.513988+0000","last_clean":"2024-04-24T08:51:36.513988+0000","last_became_active":"2024-04-24T08:48:34.321969+0000","last_became_peered":"2024-04-24T08:48:34.321969+0000","last_unstale":"2024-04-24T08:51:36.513988+0000","last_undegraded":"2024-04-24T08:51:36.513988+0000","last_fullsized":"2024-04-24T08:51:36.513988+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:19:56.921198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.729531+0000","last_change":"2024-04-24T08:48:29.981573+0000","last_active":"2024-04-24T08:51:36.729531+0000","last_peered":"2024-04-24T08:51:36.729531+0000","last_clean":"2024-04-24T08:51:36.729531+0000","last_became_active":"2024-04-24T08:48:29.979216+0000","last_became_peered":"2024-04-24T08:48:29.979216+0000","last_unstale":"2024-04-24T08:51:36.729531+0000","last_undegraded":"2024-04-24T08:51:36.729531+0000","last_fullsized":"2024-04-24T08:51:36.729531+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:38:09.164883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":17,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.700456+0000","last_change":"2024-04-24T08:50:43.030115+0000","last_active":"2024-04-24T08:51:36.700456+0000","last_peered":"2024-04-24T08:51:36.700456+0000","last_clean":"2024-04-24T08:51:36.700456+0000","last_became_active":"2024-04-24T08:50:43.005669+0000","last_became_peered":"2024-04-24T08:50:43.005669+0000","last_unstale":"2024-04-24T08:51:36.700456+0000","last_undegraded":"2024-04-24T08:51:36.700456+0000","last_fullsized":"2024-04-24T08:51:36.700456+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:32:32.226068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.700661+0000","last_change":"2024-04-24T08:48:34.317502+0000","last_active":"2024-04-24T08:51:36.700661+0000","last_peered":"2024-04-24T08:51:36.700661+0000","last_clean":"2024-04-24T08:51:36.700661+0000","last_became_active":"2024-04-24T08:48:34.316614+0000","last_became_peered":"2024-04-24T08:48:34.316614+0000","last_unstale":"2024-04-24T08:51:36.700661+0000","last_undegraded":"2024-04-24T08:51:36.700661+0000","last_fullsized":"2024-04-24T08:51:36.700661+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:44:12.469926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.504469+0000","last_change":"2024-04-24T08:48:28.970261+0000","last_active":"2024-04-24T08:51:36.504469+0000","last_peered":"2024-04-24T08:51:36.504469+0000","last_clean":"2024-04-24T08:51:36.504469+0000","last_became_active":"2024-04-24T08:48:28.969813+0000","last_became_peered":"2024-04-24T08:48:28.969813+0000","last_unstale":"2024-04-24T08:51:36.504469+0000","last_undegraded":"2024-04-24T08:51:36.504469+0000","last_fullsized":"2024-04-24T08:51:36.504469+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:46:12.311756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.731284+0000","last_change":"2024-04-24T08:48:29.976712+0000","last_active":"2024-04-24T08:51:36.731284+0000","last_peered":"2024-04-24T08:51:36.731284+0000","last_clean":"2024-04-24T08:51:36.731284+0000","last_became_active":"2024-04-24T08:48:29.976166+0000","last_became_peered":"2024-04-24T08:48:29.976166+0000","last_unstale":"2024-04-24T08:51:36.731284+0000","last_undegraded":"2024-04-24T08:51:36.731284+0000","last_fullsized":"2024-04-24T08:51:36.731284+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:52:53.757829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.359483+0000","last_change":"2024-04-24T08:48:34.310173+0000","last_active":"2024-04-24T08:51:36.359483+0000","last_peered":"2024-04-24T08:51:36.359483+0000","last_clean":"2024-04-24T08:51:36.359483+0000","last_became_active":"2024-04-24T08:48:34.309772+0000","last_became_peered":"2024-04-24T08:48:34.309772+0000","last_unstale":"2024-04-24T08:51:36.359483+0000","last_undegraded":"2024-04-24T08:51:36.359483+0000","last_fullsized":"2024-04-24T08:51:36.359483+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:35:50.302769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"94'50","reported_seq":106,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.770470+0000","last_change":"2024-04-24T08:48:28.970127+0000","last_active":"2024-04-24T08:51:36.770470+0000","last_peered":"2024-04-24T08:51:36.770470+0000","last_clean":"2024-04-24T08:51:36.770470+0000","last_became_active":"2024-04-24T08:48:28.968803+0000","last_became_peered":"2024-04-24T08:48:28.968803+0000","last_unstale":"2024-04-24T08:51:36.770470+0000","last_undegraded":"2024-04-24T08:51:36.770470+0000","last_fullsized":"2024-04-24T08:51:36.770470+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:28:47.782276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":88,"num_write_kb":837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.360223+0000","last_change":"2024-04-24T08:48:29.992964+0000","last_active":"2024-04-24T08:51:36.360223+0000","last_peered":"2024-04-24T08:51:36.360223+0000","last_clean":"2024-04-24T08:51:36.360223+0000","last_became_active":"2024-04-24T08:48:29.987501+0000","last_became_peered":"2024-04-24T08:48:29.987501+0000","last_unstale":"2024-04-24T08:51:36.360223+0000","last_undegraded":"2024-04-24T08:51:36.360223+0000","last_fullsized":"2024-04-24T08:51:36.360223+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:49:19.157052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.366605+0000","last_change":"2024-04-24T08:48:34.315393+0000","last_active":"2024-04-24T08:51:36.366605+0000","last_peered":"2024-04-24T08:51:36.366605+0000","last_clean":"2024-04-24T08:51:36.366605+0000","last_became_active":"2024-04-24T08:48:34.314943+0000","last_became_peered":"2024-04-24T08:48:34.314943+0000","last_unstale":"2024-04-24T08:51:36.366605+0000","last_undegraded":"2024-04-24T08:51:36.366605+0000","last_fullsized":"2024-04-24T08:51:36.366605+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:03:13.312223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.737303+0000","last_change":"2024-04-24T08:48:28.969813+0000","last_active":"2024-04-24T08:51:36.737303+0000","last_peered":"2024-04-24T08:51:36.737303+0000","last_clean":"2024-04-24T08:51:36.737303+0000","last_became_active":"2024-04-24T08:48:28.968485+0000","last_became_peered":"2024-04-24T08:48:28.968485+0000","last_unstale":"2024-04-24T08:51:36.737303+0000","last_undegraded":"2024-04-24T08:51:36.737303+0000","last_fullsized":"2024-04-24T08:51:36.737303+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:32:10.095723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.893652+0000","last_change":"2024-04-24T08:48:29.997731+0000","last_active":"2024-04-24T08:51:36.893652+0000","last_peered":"2024-04-24T08:51:36.893652+0000","last_clean":"2024-04-24T08:51:36.893652+0000","last_became_active":"2024-04-24T08:48:29.997373+0000","last_became_peered":"2024-04-24T08:48:29.997373+0000","last_unstale":"2024-04-24T08:51:36.893652+0000","last_undegraded":"2024-04-24T08:51:36.893652+0000","last_fullsized":"2024-04-24T08:51:36.893652+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:57:19.531173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.736525+0000","last_change":"2024-04-24T08:48:34.326170+0000","last_active":"2024-04-24T08:51:36.736525+0000","last_peered":"2024-04-24T08:51:36.736525+0000","last_clean":"2024-04-24T08:51:36.736525+0000","last_became_active":"2024-04-24T08:48:34.325678+0000","last_became_peered":"2024-04-24T08:48:34.325678+0000","last_unstale":"2024-04-24T08:51:36.736525+0000","last_undegraded":"2024-04-24T08:51:36.736525+0000","last_fullsized":"2024-04-24T08:51:36.736525+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:12:47.003864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.892690+0000","last_change":"2024-04-24T08:48:28.975522+0000","last_active":"2024-04-24T08:51:36.892690+0000","last_peered":"2024-04-24T08:51:36.892690+0000","last_clean":"2024-04-24T08:51:36.892690+0000","last_became_active":"2024-04-24T08:48:28.975093+0000","last_became_peered":"2024-04-24T08:48:28.975093+0000","last_unstale":"2024-04-24T08:51:36.892690+0000","last_undegraded":"2024-04-24T08:51:36.892690+0000","last_fullsized":"2024-04-24T08:51:36.892690+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:37:31.809694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.503358+0000","last_change":"2024-04-24T08:48:29.982502+0000","last_active":"2024-04-24T08:51:36.503358+0000","last_peered":"2024-04-24T08:51:36.503358+0000","last_clean":"2024-04-24T08:51:36.503358+0000","last_became_active":"2024-04-24T08:48:29.982141+0000","last_became_peered":"2024-04-24T08:48:29.982141+0000","last_unstale":"2024-04-24T08:51:36.503358+0000","last_undegraded":"2024-04-24T08:51:36.503358+0000","last_fullsized":"2024-04-24T08:51:36.503358+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:46:15.851359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.499230+0000","last_change":"2024-04-24T08:48:34.322717+0000","last_active":"2024-04-24T08:51:36.499230+0000","last_peered":"2024-04-24T08:51:36.499230+0000","last_clean":"2024-04-24T08:51:36.499230+0000","last_became_active":"2024-04-24T08:48:34.321837+0000","last_became_peered":"2024-04-24T08:48:34.321837+0000","last_unstale":"2024-04-24T08:51:36.499230+0000","last_undegraded":"2024-04-24T08:51:36.499230+0000","last_fullsized":"2024-04-24T08:51:36.499230+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:40:11.557436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.359786+0000","last_change":"2024-04-24T08:48:28.973951+0000","last_active":"2024-04-24T08:51:36.359786+0000","last_peered":"2024-04-24T08:51:36.359786+0000","last_clean":"2024-04-24T08:51:36.359786+0000","last_became_active":"2024-04-24T08:48:28.973131+0000","last_became_peered":"2024-04-24T08:48:28.973131+0000","last_unstale":"2024-04-24T08:51:36.359786+0000","last_undegraded":"2024-04-24T08:51:36.359786+0000","last_fullsized":"2024-04-24T08:51:36.359786+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:45:45.197337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.701038+0000","last_change":"2024-04-24T08:48:29.992689+0000","last_active":"2024-04-24T08:51:36.701038+0000","last_peered":"2024-04-24T08:51:36.701038+0000","last_clean":"2024-04-24T08:51:36.701038+0000","last_became_active":"2024-04-24T08:48:29.988874+0000","last_became_peered":"2024-04-24T08:48:29.988874+0000","last_unstale":"2024-04-24T08:51:36.701038+0000","last_undegraded":"2024-04-24T08:51:36.701038+0000","last_fullsized":"2024-04-24T08:51:36.701038+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:42:41.578424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.737734+0000","last_change":"2024-04-24T08:48:34.318640+0000","last_active":"2024-04-24T08:51:36.737734+0000","last_peered":"2024-04-24T08:51:36.737734+0000","last_clean":"2024-04-24T08:51:36.737734+0000","last_became_active":"2024-04-24T08:48:34.318039+0000","last_became_peered":"2024-04-24T08:48:34.318039+0000","last_unstale":"2024-04-24T08:51:36.737734+0000","last_undegraded":"2024-04-24T08:51:36.737734+0000","last_fullsized":"2024-04-24T08:51:36.737734+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:17:27.110930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"94'6","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.367758+0000","last_change":"2024-04-24T08:48:28.980828+0000","last_active":"2024-04-24T08:51:36.367758+0000","last_peered":"2024-04-24T08:51:36.367758+0000","last_clean":"2024-04-24T08:51:36.367758+0000","last_became_active":"2024-04-24T08:48:28.979814+0000","last_became_peered":"2024-04-24T08:48:28.979814+0000","last_unstale":"2024-04-24T08:51:36.367758+0000","last_undegraded":"2024-04-24T08:51:36.367758+0000","last_fullsized":"2024-04-24T08:51:36.367758+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:02:28.145847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.359850+0000","last_change":"2024-04-24T08:48:29.995003+0000","last_active":"2024-04-24T08:51:36.359850+0000","last_peered":"2024-04-24T08:51:36.359850+0000","last_clean":"2024-04-24T08:51:36.359850+0000","last_became_active":"2024-04-24T08:48:29.994516+0000","last_became_peered":"2024-04-24T08:48:29.994516+0000","last_unstale":"2024-04-24T08:51:36.359850+0000","last_undegraded":"2024-04-24T08:51:36.359850+0000","last_fullsized":"2024-04-24T08:51:36.359850+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:15:46.094534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.738075+0000","last_change":"2024-04-24T08:48:34.311721+0000","last_active":"2024-04-24T08:51:36.738075+0000","last_peered":"2024-04-24T08:51:36.738075+0000","last_clean":"2024-04-24T08:51:36.738075+0000","last_became_active":"2024-04-24T08:48:34.311241+0000","last_became_peered":"2024-04-24T08:48:34.311241+0000","last_unstale":"2024-04-24T08:51:36.738075+0000","last_undegraded":"2024-04-24T08:51:36.738075+0000","last_fullsized":"2024-04-24T08:51:36.738075+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:16:17.679237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.503671+0000","last_change":"2024-04-24T08:48:28.973336+0000","last_active":"2024-04-24T08:51:36.503671+0000","last_peered":"2024-04-24T08:51:36.503671+0000","last_clean":"2024-04-24T08:51:36.503671+0000","last_became_active":"2024-04-24T08:48:28.972666+0000","last_became_peered":"2024-04-24T08:48:28.972666+0000","last_unstale":"2024-04-24T08:51:36.503671+0000","last_undegraded":"2024-04-24T08:51:36.503671+0000","last_fullsized":"2024-04-24T08:51:36.503671+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:53:28.358291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.658302+0000","last_change":"2024-04-24T08:48:29.983498+0000","last_active":"2024-04-24T08:51:36.658302+0000","last_peered":"2024-04-24T08:51:36.658302+0000","last_clean":"2024-04-24T08:51:36.658302+0000","last_became_active":"2024-04-24T08:48:29.983167+0000","last_became_peered":"2024-04-24T08:48:29.983167+0000","last_unstale":"2024-04-24T08:51:36.658302+0000","last_undegraded":"2024-04-24T08:51:36.658302+0000","last_fullsized":"2024-04-24T08:51:36.658302+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:09:56.648047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.893130+0000","last_change":"2024-04-24T08:49:42.129192+0000","last_active":"2024-04-24T08:51:36.893130+0000","last_peered":"2024-04-24T08:51:36.893130+0000","last_clean":"2024-04-24T08:51:36.893130+0000","last_became_active":"2024-04-24T08:49:42.124814+0000","last_became_peered":"2024-04-24T08:49:42.124814+0000","last_unstale":"2024-04-24T08:51:36.893130+0000","last_undegraded":"2024-04-24T08:51:36.893130+0000","last_fullsized":"2024-04-24T08:51:36.893130+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:52:46.273836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6,7],"acting":[1,8,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"94'8","reported_seq":197,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.893736+0000","last_change":"2024-04-24T08:48:28.974528+0000","last_active":"2024-04-24T08:51:36.893736+0000","last_peered":"2024-04-24T08:51:36.893736+0000","last_clean":"2024-04-24T08:51:36.893736+0000","last_became_active":"2024-04-24T08:48:28.974203+0000","last_became_peered":"2024-04-24T08:48:28.974203+0000","last_unstale":"2024-04-24T08:51:36.893736+0000","last_undegraded":"2024-04-24T08:51:36.893736+0000","last_fullsized":"2024-04-24T08:51:36.893736+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:26:15.701204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2445,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":156,"num_read_kb":3,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.659205+0000","last_change":"2024-04-24T08:48:29.984356+0000","last_active":"2024-04-24T08:51:36.659205+0000","last_peered":"2024-04-24T08:51:36.659205+0000","last_clean":"2024-04-24T08:51:36.659205+0000","last_became_active":"2024-04-24T08:48:29.983949+0000","last_became_peered":"2024-04-24T08:48:29.983949+0000","last_unstale":"2024-04-24T08:51:36.659205+0000","last_undegraded":"2024-04-24T08:51:36.659205+0000","last_fullsized":"2024-04-24T08:51:36.659205+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:51:59.409835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.893130+0000","last_change":"2024-04-24T08:48:34.312418+0000","last_active":"2024-04-24T08:51:36.893130+0000","last_peered":"2024-04-24T08:51:36.893130+0000","last_clean":"2024-04-24T08:51:36.893130+0000","last_became_active":"2024-04-24T08:48:34.312035+0000","last_became_peered":"2024-04-24T08:48:34.312035+0000","last_unstale":"2024-04-24T08:51:36.893130+0000","last_undegraded":"2024-04-24T08:51:36.893130+0000","last_fullsized":"2024-04-24T08:51:36.893130+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:27:33.461732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"94'41","reported_seq":83,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.765572+0000","last_change":"2024-04-24T08:48:28.976070+0000","last_active":"2024-04-24T08:51:36.765572+0000","last_peered":"2024-04-24T08:51:36.765572+0000","last_clean":"2024-04-24T08:51:36.765572+0000","last_became_active":"2024-04-24T08:48:28.975703+0000","last_became_peered":"2024-04-24T08:48:28.975703+0000","last_unstale":"2024-04-24T08:51:36.765572+0000","last_undegraded":"2024-04-24T08:51:36.765572+0000","last_fullsized":"2024-04-24T08:51:36.765572+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:46:13.227529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'6","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.658745+0000","last_change":"2024-04-24T08:49:42.128814+0000","last_active":"2024-04-24T08:51:36.658745+0000","last_peered":"2024-04-24T08:51:36.658745+0000","last_clean":"2024-04-24T08:51:36.658745+0000","last_became_active":"2024-04-24T08:49:42.127422+0000","last_became_peered":"2024-04-24T08:49:42.127422+0000","last_unstale":"2024-04-24T08:51:36.658745+0000","last_undegraded":"2024-04-24T08:51:36.658745+0000","last_fullsized":"2024-04-24T08:51:36.658745+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:58:25.181798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.512831+0000","last_change":"2024-04-24T08:48:34.320191+0000","last_active":"2024-04-24T08:51:36.512831+0000","last_peered":"2024-04-24T08:51:36.512831+0000","last_clean":"2024-04-24T08:51:36.512831+0000","last_became_active":"2024-04-24T08:48:34.315415+0000","last_became_peered":"2024-04-24T08:48:34.315415+0000","last_unstale":"2024-04-24T08:51:36.512831+0000","last_undegraded":"2024-04-24T08:51:36.512831+0000","last_fullsized":"2024-04-24T08:51:36.512831+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:55:11.372789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.731727+0000","last_change":"2024-04-24T08:48:28.968550+0000","last_active":"2024-04-24T08:51:36.731727+0000","last_peered":"2024-04-24T08:51:36.731727+0000","last_clean":"2024-04-24T08:51:36.731727+0000","last_became_active":"2024-04-24T08:48:28.968120+0000","last_became_peered":"2024-04-24T08:48:28.968120+0000","last_unstale":"2024-04-24T08:51:36.731727+0000","last_undegraded":"2024-04-24T08:51:36.731727+0000","last_fullsized":"2024-04-24T08:51:36.731727+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:46:00.870285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.659317+0000","last_change":"2024-04-24T08:48:29.984755+0000","last_active":"2024-04-24T08:51:36.659317+0000","last_peered":"2024-04-24T08:51:36.659317+0000","last_clean":"2024-04-24T08:51:36.659317+0000","last_became_active":"2024-04-24T08:48:29.982093+0000","last_became_peered":"2024-04-24T08:48:29.982093+0000","last_unstale":"2024-04-24T08:51:36.659317+0000","last_undegraded":"2024-04-24T08:51:36.659317+0000","last_fullsized":"2024-04-24T08:51:36.659317+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:25:22.620152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.700841+0000","last_change":"2024-04-24T08:48:29.983101+0000","last_active":"2024-04-24T08:51:36.700841+0000","last_peered":"2024-04-24T08:51:36.700841+0000","last_clean":"2024-04-24T08:51:36.700841+0000","last_became_active":"2024-04-24T08:48:29.981871+0000","last_became_peered":"2024-04-24T08:48:29.981871+0000","last_unstale":"2024-04-24T08:51:36.700841+0000","last_undegraded":"2024-04-24T08:51:36.700841+0000","last_fullsized":"2024-04-24T08:51:36.700841+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:02:44.582111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":49,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.367516+0000","last_change":"2024-04-24T08:49:42.128974+0000","last_active":"2024-04-24T08:51:36.367516+0000","last_peered":"2024-04-24T08:51:36.367516+0000","last_clean":"2024-04-24T08:51:36.367516+0000","last_became_active":"2024-04-24T08:49:42.127710+0000","last_became_peered":"2024-04-24T08:49:42.127710+0000","last_unstale":"2024-04-24T08:51:36.367516+0000","last_undegraded":"2024-04-24T08:51:36.367516+0000","last_fullsized":"2024-04-24T08:51:36.367516+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:09:20.279852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.367156+0000","last_change":"2024-04-24T08:48:29.988241+0000","last_active":"2024-04-24T08:51:36.367156+0000","last_peered":"2024-04-24T08:51:36.367156+0000","last_clean":"2024-04-24T08:51:36.367156+0000","last_became_active":"2024-04-24T08:48:29.987930+0000","last_became_peered":"2024-04-24T08:48:29.987930+0000","last_unstale":"2024-04-24T08:51:36.367156+0000","last_undegraded":"2024-04-24T08:51:36.367156+0000","last_fullsized":"2024-04-24T08:51:36.367156+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:42:24.687624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.893443+0000","last_change":"2024-04-24T08:48:29.992441+0000","last_active":"2024-04-24T08:51:36.893443+0000","last_peered":"2024-04-24T08:51:36.893443+0000","last_clean":"2024-04-24T08:51:36.893443+0000","last_became_active":"2024-04-24T08:48:29.988788+0000","last_became_peered":"2024-04-24T08:48:29.988788+0000","last_unstale":"2024-04-24T08:51:36.893443+0000","last_undegraded":"2024-04-24T08:51:36.893443+0000","last_fullsized":"2024-04-24T08:51:36.893443+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:08:31.549509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.731077+0000","last_change":"2024-04-24T08:48:29.982116+0000","last_active":"2024-04-24T08:51:36.731077+0000","last_peered":"2024-04-24T08:51:36.731077+0000","last_clean":"2024-04-24T08:51:36.731077+0000","last_became_active":"2024-04-24T08:48:29.981636+0000","last_became_peered":"2024-04-24T08:48:29.981636+0000","last_unstale":"2024-04-24T08:51:36.731077+0000","last_undegraded":"2024-04-24T08:51:36.731077+0000","last_fullsized":"2024-04-24T08:51:36.731077+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:11:01.943425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.701572+0000","last_change":"2024-04-24T08:48:29.982244+0000","last_active":"2024-04-24T08:51:36.701572+0000","last_peered":"2024-04-24T08:51:36.701572+0000","last_clean":"2024-04-24T08:51:36.701572+0000","last_became_active":"2024-04-24T08:48:29.981622+0000","last_became_peered":"2024-04-24T08:48:29.981622+0000","last_unstale":"2024-04-24T08:51:36.701572+0000","last_undegraded":"2024-04-24T08:51:36.701572+0000","last_fullsized":"2024-04-24T08:51:36.701572+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:02:57.723949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":46,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.369396+0000","last_change":"2024-04-24T08:48:29.985867+0000","last_active":"2024-04-24T08:51:36.369396+0000","last_peered":"2024-04-24T08:51:36.369396+0000","last_clean":"2024-04-24T08:51:36.369396+0000","last_became_active":"2024-04-24T08:48:29.985430+0000","last_became_peered":"2024-04-24T08:48:29.985430+0000","last_unstale":"2024-04-24T08:51:36.369396+0000","last_undegraded":"2024-04-24T08:51:36.369396+0000","last_fullsized":"2024-04-24T08:51:36.369396+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:09:40.467623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.360401+0000","last_change":"2024-04-24T08:48:29.996474+0000","last_active":"2024-04-24T08:51:36.360401+0000","last_peered":"2024-04-24T08:51:36.360401+0000","last_clean":"2024-04-24T08:51:36.360401+0000","last_became_active":"2024-04-24T08:48:29.995958+0000","last_became_peered":"2024-04-24T08:48:29.995958+0000","last_unstale":"2024-04-24T08:51:36.360401+0000","last_undegraded":"2024-04-24T08:51:36.360401+0000","last_fullsized":"2024-04-24T08:51:36.360401+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:50:19.865125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.513993+0000","last_change":"2024-04-24T08:48:29.978285+0000","last_active":"2024-04-24T08:51:36.513993+0000","last_peered":"2024-04-24T08:51:36.513993+0000","last_clean":"2024-04-24T08:51:36.513993+0000","last_became_active":"2024-04-24T08:48:29.976445+0000","last_became_peered":"2024-04-24T08:48:29.976445+0000","last_unstale":"2024-04-24T08:51:36.513993+0000","last_undegraded":"2024-04-24T08:51:36.513993+0000","last_fullsized":"2024-04-24T08:51:36.513993+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:30:29.598733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.359571+0000","last_change":"2024-04-24T08:48:29.994442+0000","last_active":"2024-04-24T08:51:36.359571+0000","last_peered":"2024-04-24T08:51:36.359571+0000","last_clean":"2024-04-24T08:51:36.359571+0000","last_became_active":"2024-04-24T08:48:29.993896+0000","last_became_peered":"2024-04-24T08:48:29.993896+0000","last_unstale":"2024-04-24T08:51:36.359571+0000","last_undegraded":"2024-04-24T08:51:36.359571+0000","last_fullsized":"2024-04-24T08:51:36.359571+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:37:14.023511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":47,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.893266+0000","last_change":"2024-04-24T08:49:41.981252+0000","last_active":"2024-04-24T08:51:36.893266+0000","last_peered":"2024-04-24T08:51:36.893266+0000","last_clean":"2024-04-24T08:51:36.893266+0000","last_became_active":"2024-04-24T08:49:41.980495+0000","last_became_peered":"2024-04-24T08:49:41.980495+0000","last_unstale":"2024-04-24T08:51:36.893266+0000","last_undegraded":"2024-04-24T08:51:36.893266+0000","last_fullsized":"2024-04-24T08:51:36.893266+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:47:33.579204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,8],"acting":[1,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.730358+0000","last_change":"2024-04-24T08:48:29.986021+0000","last_active":"2024-04-24T08:51:36.730358+0000","last_peered":"2024-04-24T08:51:36.730358+0000","last_clean":"2024-04-24T08:51:36.730358+0000","last_became_active":"2024-04-24T08:48:29.985182+0000","last_became_peered":"2024-04-24T08:48:29.985182+0000","last_unstale":"2024-04-24T08:51:36.730358+0000","last_undegraded":"2024-04-24T08:51:36.730358+0000","last_fullsized":"2024-04-24T08:51:36.730358+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:10:57.143065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.700856+0000","last_change":"2024-04-24T08:48:29.979886+0000","last_active":"2024-04-24T08:51:36.700856+0000","last_peered":"2024-04-24T08:51:36.700856+0000","last_clean":"2024-04-24T08:51:36.700856+0000","last_became_active":"2024-04-24T08:48:29.978779+0000","last_became_peered":"2024-04-24T08:48:29.978779+0000","last_unstale":"2024-04-24T08:51:36.700856+0000","last_undegraded":"2024-04-24T08:51:36.700856+0000","last_fullsized":"2024-04-24T08:51:36.700856+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:51:51.303990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":602295,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1664,"num_read_kb":942,"num_write":386,"num_write_kb":1475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":134,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2174976,"data_stored":1806885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":310,"ondisk_log_size":310,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383733,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1069056,"data_stored":520983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645372,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":33100,"kb_used_data":1864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712052,"statfs":{"total":95995035648,"available":95961141248,"internally_reserved":0,"allocated":1908736,"data_stored":1311223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808899,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32648,"kb_used_data":1468,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712504,"statfs":{"total":95995035648,"available":95961604096,"internally_reserved":0,"allocated":1503232,"data_stored":927663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070538,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28476,"kb_used_data":1456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716676,"statfs":{"total":95995035648,"available":95965876224,"internally_reserved":0,"allocated":1490944,"data_stored":923923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332178,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32840,"kb_used_data":1660,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712312,"statfs":{"total":95995035648,"available":95961407488,"internally_reserved":0,"allocated":1699840,"data_stored":1128846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495705,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32216,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712936,"statfs":{"total":95995035648,"available":95962046464,"internally_reserved":0,"allocated":1064960,"data_stored":516395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574973,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36468,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708684,"statfs":{"total":95995035648,"available":95957692416,"internally_reserved":0,"allocated":1155072,"data_stored":594758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738500,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36420,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708732,"statfs":{"total":95995035648,"available":95957741568,"internally_reserved":0,"allocated":1105920,"data_stored":528586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411445,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28152,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717000,"statfs":{"total":95995035648,"available":95966208000,"internally_reserved":0,"allocated":1159168,"data_stored":590404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182511,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32248,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712904,"statfs":{"total":95995035648,"available":95962013696,"internally_reserved":0,"allocated":1093632,"data_stored":525563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920871,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32232,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712920,"statfs":{"total":95995035648,"available":95962030080,"internally_reserved":0,"allocated":1081344,"data_stored":520581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757345,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32976,"kb_used_data":1736,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712176,"statfs":{"total":95995035648,"available":95961268224,"internally_reserved":0,"allocated":1777664,"data_stored":1189498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":69625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":71540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":557056,"data_stored":525246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":180224,"data_stored":135836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":421888,"data_stored":394694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":258048,"data_stored":200818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T08:51:42.775 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-04-24T08:51:42.775 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T08:51:42.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.978+0000 7f4b0e800640 1 -- 172.21.15.83:0/2369249343 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b08105450 msgr2=0x7f4b081058d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:42.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.978+0000 7f4b0e800640 1 --2- 172.21.15.83:0/2369249343 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b08105450 0x7f4b081058d0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f4af0009900 tx=0x7f4af002a3f0 comp rx=0 tx=0).stop 2024-04-24T08:51:42.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.978+0000 7f4b0e800640 1 -- 172.21.15.83:0/2369249343 shutdown_connections 2024-04-24T08:51:42.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.978+0000 7f4b0e800640 1 --2- 172.21.15.83:0/2369249343 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4b08105e10 0x7f4b0810c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:42.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.978+0000 7f4b0e800640 1 --2- 172.21.15.83:0/2369249343 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b08105450 0x7f4b081058d0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:42.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.978+0000 7f4b0e800640 1 --2- 172.21.15.83:0/2369249343 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b08104260 0x7f4b08104680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:42.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.978+0000 7f4b0e800640 1 -- 172.21.15.83:0/2369249343 >> 172.21.15.83:0/2369249343 conn(0x7f4b080ff7f0 msgr2=0x7f4b08101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:42.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.979+0000 7f4b0e800640 1 -- 172.21.15.83:0/2369249343 shutdown_connections 2024-04-24T08:51:42.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.979+0000 7f4b0e800640 1 -- 172.21.15.83:0/2369249343 wait complete. 2024-04-24T08:51:42.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.979+0000 7f4b0e800640 1 Processor -- start 2024-04-24T08:51:42.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.980+0000 7f4b0e800640 1 -- start start 2024-04-24T08:51:42.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.980+0000 7f4b0e800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b08104260 0x7f4b0819a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:42.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.980+0000 7f4b0e800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b08105450 0x7f4b0819a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:42.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.980+0000 7f4b0e800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4b08105e10 0x7f4b081a19e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:42.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.980+0000 7f4b0e800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4b0810fda0 con 0x7f4b08104260 2024-04-24T08:51:42.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.980+0000 7f4b0e800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4b0810fc20 con 0x7f4b08105450 2024-04-24T08:51:42.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.980+0000 7f4b0e800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4b0810ff20 con 0x7f4b08105e10 2024-04-24T08:51:42.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.980+0000 7f4b07400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b08104260 0x7f4b0819a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:42.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.980+0000 7f4b07400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b08104260 0x7f4b0819a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55848/0 (socket says 172.21.15.83:55848) 2024-04-24T08:51:42.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.980+0000 7f4b07400640 1 -- 172.21.15.83:0/1860208863 learned_addr learned my addr 172.21.15.83:0/1860208863 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:51:42.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.980+0000 7f4afea00640 1 --2- 172.21.15.83:0/1860208863 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b08105450 0x7f4b0819a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:42.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.980+0000 7f4b07e00640 1 --2- 172.21.15.83:0/1860208863 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4b08105e10 0x7f4b081a19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:42.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.980+0000 7f4b07400640 1 -- 172.21.15.83:0/1860208863 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4b08105e10 msgr2=0x7f4b081a19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:42.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.980+0000 7f4b07400640 1 --2- 172.21.15.83:0/1860208863 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4b08105e10 0x7f4b081a19e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:42.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.980+0000 7f4b07400640 1 -- 172.21.15.83:0/1860208863 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b08105450 msgr2=0x7f4b0819a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:42.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.980+0000 7f4b07400640 1 --2- 172.21.15.83:0/1860208863 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b08105450 0x7f4b0819a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:42.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.980+0000 7f4b07400640 1 -- 172.21.15.83:0/1860208863 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b081a2050 con 0x7f4b08104260 2024-04-24T08:51:42.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.981+0000 7f4b07400640 1 --2- 172.21.15.83:0/1860208863 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b08104260 0x7f4b0819a430 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f4af800ccb0 tx=0x7f4af800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:42.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.981+0000 7f4b04c00640 1 -- 172.21.15.83:0/1860208863 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4af8003990 con 0x7f4b08104260 2024-04-24T08:51:42.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.981+0000 7f4b04c00640 1 -- 172.21.15.83:0/1860208863 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4af8004020 con 0x7f4b08104260 2024-04-24T08:51:42.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.981+0000 7f4b0e800640 1 -- 172.21.15.83:0/1860208863 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b081a2340 con 0x7f4b08104260 2024-04-24T08:51:42.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.981+0000 7f4b04c00640 1 -- 172.21.15.83:0/1860208863 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4af800f6b0 con 0x7f4b08104260 2024-04-24T08:51:42.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.981+0000 7f4b0e800640 1 -- 172.21.15.83:0/1860208863 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b081a2890 con 0x7f4b08104260 2024-04-24T08:51:42.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.983+0000 7f4b0e800640 1 -- 172.21.15.83:0/1860208863 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b0810b580 con 0x7f4b08104260 2024-04-24T08:51:42.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.985+0000 7f4b04c00640 1 -- 172.21.15.83:0/1860208863 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f4af8003b30 con 0x7f4b08104260 2024-04-24T08:51:42.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.986+0000 7f4b04c00640 1 --2- 172.21.15.83:0/1860208863 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f4aec0a3db0 0x7f4aec0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:42.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.986+0000 7f4b04c00640 1 -- 172.21.15.83:0/1860208863 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f4af80d1250 con 0x7f4b08104260 2024-04-24T08:51:42.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.986+0000 7f4afea00640 1 --2- 172.21.15.83:0/1860208863 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f4aec0a3db0 0x7f4aec0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:42.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.987+0000 7f4afea00640 1 --2- 172.21.15.83:0/1860208863 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f4aec0a3db0 0x7f4aec0a6270 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f4b0819b940 tx=0x7f4af00314a0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:42.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:42.990+0000 7f4b04c00640 1 -- 172.21.15.83:0/1860208863 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4af8014030 con 0x7f4b08104260 2024-04-24T08:51:43.052 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:42 smithi083 ceph-mon[27907]: Adjusting osd_memory_target on smithi083 to 176.4M 2024-04-24T08:51:43.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:42 smithi083 ceph-mon[27907]: Unable to set osd_memory_target on smithi083 to 185061990: error parsing value: Value '185061990' is below minimum 939524096 2024-04-24T08:51:43.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:42 smithi083 ceph-mon[27907]: Adjusting osd_memory_target on smithi189 to 176.4M 2024-04-24T08:51:43.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:42 smithi083 ceph-mon[27907]: Unable to set osd_memory_target on smithi189 to 185064857: error parsing value: Value '185064857' is below minimum 939524096 2024-04-24T08:51:43.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:42 smithi083 ceph-mon[27907]: Adjusting osd_memory_target on smithi138 to 176.4M 2024-04-24T08:51:43.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:42 smithi083 ceph-mon[27907]: Unable to set osd_memory_target on smithi138 to 185047654: error parsing value: Value '185047654' is below minimum 939524096 2024-04-24T08:51:43.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:42 smithi083 ceph-mon[27907]: Updating smithi083:/etc/ceph/ceph.conf 2024-04-24T08:51:43.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:42 smithi083 ceph-mon[27907]: Updating smithi138:/etc/ceph/ceph.conf 2024-04-24T08:51:43.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:42 smithi083 ceph-mon[27907]: Updating smithi189:/etc/ceph/ceph.conf 2024-04-24T08:51:43.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:43.186+0000 7f4b0e800640 1 -- 172.21.15.83:0/1860208863 --> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4b08062440 con 0x7f4aec0a3db0 2024-04-24T08:51:43.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:43.196+0000 7f4b04c00640 1 -- 172.21.15.83:0/1860208863 <== mgr.15579 v2:172.21.15.83:6800/3594769126 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339543 (secure 0 0 0) 0x7f4b08062440 con 0x7f4aec0a3db0 2024-04-24T08:51:43.199 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:51:43.201 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-04-24T08:51:43.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:43.201+0000 7f4b0e800640 1 -- 172.21.15.83:0/1860208863 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f4aec0a3db0 msgr2=0x7f4aec0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:43.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:43.201+0000 7f4b0e800640 1 --2- 172.21.15.83:0/1860208863 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f4aec0a3db0 0x7f4aec0a6270 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f4b0819b940 tx=0x7f4af00314a0 comp rx=0 tx=0).stop 2024-04-24T08:51:43.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:43.201+0000 7f4b0e800640 1 -- 172.21.15.83:0/1860208863 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b08104260 msgr2=0x7f4b0819a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:43.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:43.201+0000 7f4b0e800640 1 --2- 172.21.15.83:0/1860208863 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b08104260 0x7f4b0819a430 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f4af800ccb0 tx=0x7f4af800a620 comp rx=0 tx=0).stop 2024-04-24T08:51:43.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:43.201+0000 7f4b0e800640 1 -- 172.21.15.83:0/1860208863 shutdown_connections 2024-04-24T08:51:43.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:43.201+0000 7f4b0e800640 1 --2- 172.21.15.83:0/1860208863 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f4aec0a3db0 0x7f4aec0a6270 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:43.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:43.201+0000 7f4b0e800640 1 --2- 172.21.15.83:0/1860208863 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4b08105e10 0x7f4b081a19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:43.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:43.201+0000 7f4b0e800640 1 --2- 172.21.15.83:0/1860208863 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b08105450 0x7f4b0819a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:43.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:43.201+0000 7f4b0e800640 1 --2- 172.21.15.83:0/1860208863 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b08104260 0x7f4b0819a430 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:43.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:43.201+0000 7f4b0e800640 1 -- 172.21.15.83:0/1860208863 >> 172.21.15.83:0/1860208863 conn(0x7f4b080ff7f0 msgr2=0x7f4b08108750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:43.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:43.201+0000 7f4b0e800640 1 -- 172.21.15.83:0/1860208863 shutdown_connections 2024-04-24T08:51:43.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:43.201+0000 7f4b0e800640 1 -- 172.21.15.83:0/1860208863 wait complete. 2024-04-24T08:51:43.217 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":6,"stamp":"2024-04-24T08:51:42.381846+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1192897,"num_objects":104,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1800,"num_read_kb":1061,"num_write":569,"num_write_kb":4415,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":1180870,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":949,"ondisk_log_size":949,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":385836,"kb_used_data":15732,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124555988,"statfs":{"total":1151940427776,"available":1151545331712,"internally_reserved":0,"allocated":16109568,"data_stored":9278423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":378713401},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.971095"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.701720+0000","last_change":"2024-04-24T08:48:29.993009+0000","last_active":"2024-04-24T08:51:36.701720+0000","last_peered":"2024-04-24T08:51:36.701720+0000","last_clean":"2024-04-24T08:51:36.701720+0000","last_became_active":"2024-04-24T08:48:29.990613+0000","last_became_peered":"2024-04-24T08:48:29.990613+0000","last_unstale":"2024-04-24T08:51:36.701720+0000","last_undegraded":"2024-04-24T08:51:36.701720+0000","last_fullsized":"2024-04-24T08:51:36.701720+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:13:43.871742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.893581+0000","last_change":"2024-04-24T08:48:29.986957+0000","last_active":"2024-04-24T08:51:36.893581+0000","last_peered":"2024-04-24T08:51:36.893581+0000","last_clean":"2024-04-24T08:51:36.893581+0000","last_became_active":"2024-04-24T08:48:29.984691+0000","last_became_peered":"2024-04-24T08:48:29.984691+0000","last_unstale":"2024-04-24T08:51:36.893581+0000","last_undegraded":"2024-04-24T08:51:36.893581+0000","last_fullsized":"2024-04-24T08:51:36.893581+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:07:38.031213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.893523+0000","last_change":"2024-04-24T08:48:29.991478+0000","last_active":"2024-04-24T08:51:36.893523+0000","last_peered":"2024-04-24T08:51:36.893523+0000","last_clean":"2024-04-24T08:51:36.893523+0000","last_became_active":"2024-04-24T08:48:29.991013+0000","last_became_peered":"2024-04-24T08:48:29.991013+0000","last_unstale":"2024-04-24T08:51:36.893523+0000","last_undegraded":"2024-04-24T08:51:36.893523+0000","last_fullsized":"2024-04-24T08:51:36.893523+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:10:39.381353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.738223+0000","last_change":"2024-04-24T08:48:29.983664+0000","last_active":"2024-04-24T08:51:36.738223+0000","last_peered":"2024-04-24T08:51:36.738223+0000","last_clean":"2024-04-24T08:51:36.738223+0000","last_became_active":"2024-04-24T08:48:29.978835+0000","last_became_peered":"2024-04-24T08:48:29.978835+0000","last_unstale":"2024-04-24T08:51:36.738223+0000","last_undegraded":"2024-04-24T08:51:36.738223+0000","last_fullsized":"2024-04-24T08:51:36.738223+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:29:20.501652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.503590+0000","last_change":"2024-04-24T08:48:29.989886+0000","last_active":"2024-04-24T08:51:36.503590+0000","last_peered":"2024-04-24T08:51:36.503590+0000","last_clean":"2024-04-24T08:51:36.503590+0000","last_became_active":"2024-04-24T08:48:29.989574+0000","last_became_peered":"2024-04-24T08:48:29.989574+0000","last_unstale":"2024-04-24T08:51:36.503590+0000","last_undegraded":"2024-04-24T08:51:36.503590+0000","last_fullsized":"2024-04-24T08:51:36.503590+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:14:01.085543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.368540+0000","last_change":"2024-04-24T08:48:29.978045+0000","last_active":"2024-04-24T08:51:36.368540+0000","last_peered":"2024-04-24T08:51:36.368540+0000","last_clean":"2024-04-24T08:51:36.368540+0000","last_became_active":"2024-04-24T08:48:29.977729+0000","last_became_peered":"2024-04-24T08:48:29.977729+0000","last_unstale":"2024-04-24T08:51:36.368540+0000","last_undegraded":"2024-04-24T08:51:36.368540+0000","last_fullsized":"2024-04-24T08:51:36.368540+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:03:33.382473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.657826+0000","last_change":"2024-04-24T08:48:29.976355+0000","last_active":"2024-04-24T08:51:36.657826+0000","last_peered":"2024-04-24T08:51:36.657826+0000","last_clean":"2024-04-24T08:51:36.657826+0000","last_became_active":"2024-04-24T08:48:29.975849+0000","last_became_peered":"2024-04-24T08:48:29.975849+0000","last_unstale":"2024-04-24T08:51:36.657826+0000","last_undegraded":"2024-04-24T08:51:36.657826+0000","last_fullsized":"2024-04-24T08:51:36.657826+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:13:51.302995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.359785+0000","last_change":"2024-04-24T08:48:29.983149+0000","last_active":"2024-04-24T08:51:36.359785+0000","last_peered":"2024-04-24T08:51:36.359785+0000","last_clean":"2024-04-24T08:51:36.359785+0000","last_became_active":"2024-04-24T08:48:29.982692+0000","last_became_peered":"2024-04-24T08:48:29.982692+0000","last_unstale":"2024-04-24T08:51:36.359785+0000","last_undegraded":"2024-04-24T08:51:36.359785+0000","last_fullsized":"2024-04-24T08:51:36.359785+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:59:06.151441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.893672+0000","last_change":"2024-04-24T08:48:29.985531+0000","last_active":"2024-04-24T08:51:36.893672+0000","last_peered":"2024-04-24T08:51:36.893672+0000","last_clean":"2024-04-24T08:51:36.893672+0000","last_became_active":"2024-04-24T08:48:29.982484+0000","last_became_peered":"2024-04-24T08:48:29.982484+0000","last_unstale":"2024-04-24T08:51:36.893672+0000","last_undegraded":"2024-04-24T08:51:36.893672+0000","last_fullsized":"2024-04-24T08:51:36.893672+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:03:50.114096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":47,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.368859+0000","last_change":"2024-04-24T08:48:29.982016+0000","last_active":"2024-04-24T08:51:36.368859+0000","last_peered":"2024-04-24T08:51:36.368859+0000","last_clean":"2024-04-24T08:51:36.368859+0000","last_became_active":"2024-04-24T08:48:29.977639+0000","last_became_peered":"2024-04-24T08:48:29.977639+0000","last_unstale":"2024-04-24T08:51:36.368859+0000","last_undegraded":"2024-04-24T08:51:36.368859+0000","last_fullsized":"2024-04-24T08:51:36.368859+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:35:48.676822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.659055+0000","last_change":"2024-04-24T08:48:29.977982+0000","last_active":"2024-04-24T08:51:36.659055+0000","last_peered":"2024-04-24T08:51:36.659055+0000","last_clean":"2024-04-24T08:51:36.659055+0000","last_became_active":"2024-04-24T08:48:29.977664+0000","last_became_peered":"2024-04-24T08:48:29.977664+0000","last_unstale":"2024-04-24T08:51:36.659055+0000","last_undegraded":"2024-04-24T08:51:36.659055+0000","last_fullsized":"2024-04-24T08:51:36.659055+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:45:17.597291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.738302+0000","last_change":"2024-04-24T08:48:29.981615+0000","last_active":"2024-04-24T08:51:36.738302+0000","last_peered":"2024-04-24T08:51:36.738302+0000","last_clean":"2024-04-24T08:51:36.738302+0000","last_became_active":"2024-04-24T08:48:29.979292+0000","last_became_peered":"2024-04-24T08:48:29.979292+0000","last_unstale":"2024-04-24T08:51:36.738302+0000","last_undegraded":"2024-04-24T08:51:36.738302+0000","last_fullsized":"2024-04-24T08:51:36.738302+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:48:44.195626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.730843+0000","last_change":"2024-04-24T08:48:29.952632+0000","last_active":"2024-04-24T08:51:36.730843+0000","last_peered":"2024-04-24T08:51:36.730843+0000","last_clean":"2024-04-24T08:51:36.730843+0000","last_became_active":"2024-04-24T08:48:29.951862+0000","last_became_peered":"2024-04-24T08:48:29.951862+0000","last_unstale":"2024-04-24T08:51:36.730843+0000","last_undegraded":"2024-04-24T08:51:36.730843+0000","last_fullsized":"2024-04-24T08:51:36.730843+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:23:40.038517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.893175+0000","last_change":"2024-04-24T08:48:29.984727+0000","last_active":"2024-04-24T08:51:36.893175+0000","last_peered":"2024-04-24T08:51:36.893175+0000","last_clean":"2024-04-24T08:51:36.893175+0000","last_became_active":"2024-04-24T08:48:29.983530+0000","last_became_peered":"2024-04-24T08:48:29.983530+0000","last_unstale":"2024-04-24T08:51:36.893175+0000","last_undegraded":"2024-04-24T08:51:36.893175+0000","last_fullsized":"2024-04-24T08:51:36.893175+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:00:12.729972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.893421+0000","last_change":"2024-04-24T08:48:29.981023+0000","last_active":"2024-04-24T08:51:36.893421+0000","last_peered":"2024-04-24T08:51:36.893421+0000","last_clean":"2024-04-24T08:51:36.893421+0000","last_became_active":"2024-04-24T08:48:29.979986+0000","last_became_peered":"2024-04-24T08:48:29.979986+0000","last_unstale":"2024-04-24T08:51:36.893421+0000","last_undegraded":"2024-04-24T08:51:36.893421+0000","last_fullsized":"2024-04-24T08:51:36.893421+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:05:36.163209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.892674+0000","last_change":"2024-04-24T08:48:29.989036+0000","last_active":"2024-04-24T08:51:36.892674+0000","last_peered":"2024-04-24T08:51:36.892674+0000","last_clean":"2024-04-24T08:51:36.892674+0000","last_became_active":"2024-04-24T08:48:29.988676+0000","last_became_peered":"2024-04-24T08:48:29.988676+0000","last_unstale":"2024-04-24T08:51:36.892674+0000","last_undegraded":"2024-04-24T08:51:36.892674+0000","last_fullsized":"2024-04-24T08:51:36.892674+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:03:38.544596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.702193+0000","last_change":"2024-04-24T08:48:29.986424+0000","last_active":"2024-04-24T08:51:36.702193+0000","last_peered":"2024-04-24T08:51:36.702193+0000","last_clean":"2024-04-24T08:51:36.702193+0000","last_became_active":"2024-04-24T08:48:29.985973+0000","last_became_peered":"2024-04-24T08:48:29.985973+0000","last_unstale":"2024-04-24T08:51:36.702193+0000","last_undegraded":"2024-04-24T08:51:36.702193+0000","last_fullsized":"2024-04-24T08:51:36.702193+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:48:31.533282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.498447+0000","last_change":"2024-04-24T08:48:29.975470+0000","last_active":"2024-04-24T08:51:36.498447+0000","last_peered":"2024-04-24T08:51:36.498447+0000","last_clean":"2024-04-24T08:51:36.498447+0000","last_became_active":"2024-04-24T08:48:29.974777+0000","last_became_peered":"2024-04-24T08:48:29.974777+0000","last_unstale":"2024-04-24T08:51:36.498447+0000","last_undegraded":"2024-04-24T08:51:36.498447+0000","last_fullsized":"2024-04-24T08:51:36.498447+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:04:26.423840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.368791+0000","last_change":"2024-04-24T08:48:29.988001+0000","last_active":"2024-04-24T08:51:36.368791+0000","last_peered":"2024-04-24T08:51:36.368791+0000","last_clean":"2024-04-24T08:51:36.368791+0000","last_became_active":"2024-04-24T08:48:29.987669+0000","last_became_peered":"2024-04-24T08:48:29.987669+0000","last_unstale":"2024-04-24T08:51:36.368791+0000","last_undegraded":"2024-04-24T08:51:36.368791+0000","last_fullsized":"2024-04-24T08:51:36.368791+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:29:24.457369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.367087+0000","last_change":"2024-04-24T08:48:34.308732+0000","last_active":"2024-04-24T08:51:36.367087+0000","last_peered":"2024-04-24T08:51:36.367087+0000","last_clean":"2024-04-24T08:51:36.367087+0000","last_became_active":"2024-04-24T08:48:34.307598+0000","last_became_peered":"2024-04-24T08:48:34.307598+0000","last_unstale":"2024-04-24T08:51:36.367087+0000","last_undegraded":"2024-04-24T08:51:36.367087+0000","last_fullsized":"2024-04-24T08:51:36.367087+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:28:57.260836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"93'14","reported_seq":201,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.499394+0000","last_change":"2024-04-24T08:48:28.970230+0000","last_active":"2024-04-24T08:51:36.499394+0000","last_peered":"2024-04-24T08:51:36.499394+0000","last_clean":"2024-04-24T08:51:36.499394+0000","last_became_active":"2024-04-24T08:48:28.969878+0000","last_became_peered":"2024-04-24T08:48:28.969878+0000","last_unstale":"2024-04-24T08:51:36.499394+0000","last_undegraded":"2024-04-24T08:51:36.499394+0000","last_fullsized":"2024-04-24T08:51:36.499394+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:05:19.206557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":154,"num_read_kb":154,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.360088+0000","last_change":"2024-04-24T08:48:29.986350+0000","last_active":"2024-04-24T08:51:36.360088+0000","last_peered":"2024-04-24T08:51:36.360088+0000","last_clean":"2024-04-24T08:51:36.360088+0000","last_became_active":"2024-04-24T08:48:29.985167+0000","last_became_peered":"2024-04-24T08:48:29.985167+0000","last_unstale":"2024-04-24T08:51:36.360088+0000","last_undegraded":"2024-04-24T08:51:36.360088+0000","last_fullsized":"2024-04-24T08:51:36.360088+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:43:33.326808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.736357+0000","last_change":"2024-04-24T08:48:34.315159+0000","last_active":"2024-04-24T08:51:36.736357+0000","last_peered":"2024-04-24T08:51:36.736357+0000","last_clean":"2024-04-24T08:51:36.736357+0000","last_became_active":"2024-04-24T08:48:34.312119+0000","last_became_peered":"2024-04-24T08:48:34.312119+0000","last_unstale":"2024-04-24T08:51:36.736357+0000","last_undegraded":"2024-04-24T08:51:36.736357+0000","last_fullsized":"2024-04-24T08:51:36.736357+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:56:08.024123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.360224+0000","last_change":"2024-04-24T08:48:28.971787+0000","last_active":"2024-04-24T08:51:36.360224+0000","last_peered":"2024-04-24T08:51:36.360224+0000","last_clean":"2024-04-24T08:51:36.360224+0000","last_became_active":"2024-04-24T08:48:28.971479+0000","last_became_peered":"2024-04-24T08:48:28.971479+0000","last_unstale":"2024-04-24T08:51:36.360224+0000","last_undegraded":"2024-04-24T08:51:36.360224+0000","last_fullsized":"2024-04-24T08:51:36.360224+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:09:52.498833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.893290+0000","last_change":"2024-04-24T08:48:29.989737+0000","last_active":"2024-04-24T08:51:36.893290+0000","last_peered":"2024-04-24T08:51:36.893290+0000","last_clean":"2024-04-24T08:51:36.893290+0000","last_became_active":"2024-04-24T08:48:29.989341+0000","last_became_peered":"2024-04-24T08:48:29.989341+0000","last_unstale":"2024-04-24T08:51:36.893290+0000","last_undegraded":"2024-04-24T08:51:36.893290+0000","last_fullsized":"2024-04-24T08:51:36.893290+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:02:32.714848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.736714+0000","last_change":"2024-04-24T08:48:34.322666+0000","last_active":"2024-04-24T08:51:36.736714+0000","last_peered":"2024-04-24T08:51:36.736714+0000","last_clean":"2024-04-24T08:51:36.736714+0000","last_became_active":"2024-04-24T08:48:34.321546+0000","last_became_peered":"2024-04-24T08:48:34.321546+0000","last_unstale":"2024-04-24T08:51:36.736714+0000","last_undegraded":"2024-04-24T08:51:36.736714+0000","last_fullsized":"2024-04-24T08:51:36.736714+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:42:45.066043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"94'13","reported_seq":70,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.505002+0000","last_change":"2024-04-24T08:50:43.151013+0000","last_active":"2024-04-24T08:51:36.505002+0000","last_peered":"2024-04-24T08:51:36.505002+0000","last_clean":"2024-04-24T08:51:36.505002+0000","last_became_active":"2024-04-24T08:50:43.123152+0000","last_became_peered":"2024-04-24T08:50:43.123152+0000","last_unstale":"2024-04-24T08:51:36.505002+0000","last_undegraded":"2024-04-24T08:51:36.505002+0000","last_fullsized":"2024-04-24T08:51:36.505002+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:56:45.260680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,7],"acting":[8,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.893187+0000","last_change":"2024-04-24T08:48:29.974588+0000","last_active":"2024-04-24T08:51:36.893187+0000","last_peered":"2024-04-24T08:51:36.893187+0000","last_clean":"2024-04-24T08:51:36.893187+0000","last_became_active":"2024-04-24T08:48:29.974190+0000","last_became_peered":"2024-04-24T08:48:29.974190+0000","last_unstale":"2024-04-24T08:51:36.893187+0000","last_undegraded":"2024-04-24T08:51:36.893187+0000","last_fullsized":"2024-04-24T08:51:36.893187+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:07:12.948660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.736446+0000","last_change":"2024-04-24T08:48:34.321954+0000","last_active":"2024-04-24T08:51:36.736446+0000","last_peered":"2024-04-24T08:51:36.736446+0000","last_clean":"2024-04-24T08:51:36.736446+0000","last_became_active":"2024-04-24T08:48:34.321417+0000","last_became_peered":"2024-04-24T08:48:34.321417+0000","last_unstale":"2024-04-24T08:51:36.736446+0000","last_undegraded":"2024-04-24T08:51:36.736446+0000","last_fullsized":"2024-04-24T08:51:36.736446+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:39:07.205709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.504769+0000","last_change":"2024-04-24T08:48:28.974464+0000","last_active":"2024-04-24T08:51:36.504769+0000","last_peered":"2024-04-24T08:51:36.504769+0000","last_clean":"2024-04-24T08:51:36.504769+0000","last_became_active":"2024-04-24T08:48:28.974001+0000","last_became_peered":"2024-04-24T08:48:28.974001+0000","last_unstale":"2024-04-24T08:51:36.504769+0000","last_undegraded":"2024-04-24T08:51:36.504769+0000","last_fullsized":"2024-04-24T08:51:36.504769+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:38:55.193427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":47,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.514535+0000","last_change":"2024-04-24T08:48:29.981336+0000","last_active":"2024-04-24T08:51:36.514535+0000","last_peered":"2024-04-24T08:51:36.514535+0000","last_clean":"2024-04-24T08:51:36.514535+0000","last_became_active":"2024-04-24T08:48:29.980333+0000","last_became_peered":"2024-04-24T08:48:29.980333+0000","last_unstale":"2024-04-24T08:51:36.514535+0000","last_undegraded":"2024-04-24T08:51:36.514535+0000","last_fullsized":"2024-04-24T08:51:36.514535+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:46:03.640332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.368326+0000","last_change":"2024-04-24T08:48:34.323099+0000","last_active":"2024-04-24T08:51:36.368326+0000","last_peered":"2024-04-24T08:51:36.368326+0000","last_clean":"2024-04-24T08:51:36.368326+0000","last_became_active":"2024-04-24T08:48:34.322557+0000","last_became_peered":"2024-04-24T08:48:34.322557+0000","last_unstale":"2024-04-24T08:51:36.368326+0000","last_undegraded":"2024-04-24T08:51:36.368326+0000","last_fullsized":"2024-04-24T08:51:36.368326+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:59:59.887087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"94'9","reported_seq":200,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.699273+0000","last_change":"2024-04-24T08:48:28.978427+0000","last_active":"2024-04-24T08:51:36.699273+0000","last_peered":"2024-04-24T08:51:36.699273+0000","last_clean":"2024-04-24T08:51:36.699273+0000","last_became_active":"2024-04-24T08:48:28.977291+0000","last_became_peered":"2024-04-24T08:48:28.977291+0000","last_unstale":"2024-04-24T08:51:36.699273+0000","last_undegraded":"2024-04-24T08:51:36.699273+0000","last_fullsized":"2024-04-24T08:51:36.699273+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:03:03.085265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2038,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":157,"num_read_kb":4,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.503358+0000","last_change":"2024-04-24T08:48:29.978008+0000","last_active":"2024-04-24T08:51:36.503358+0000","last_peered":"2024-04-24T08:51:36.503358+0000","last_clean":"2024-04-24T08:51:36.503358+0000","last_became_active":"2024-04-24T08:48:29.977389+0000","last_became_peered":"2024-04-24T08:48:29.977389+0000","last_unstale":"2024-04-24T08:51:36.503358+0000","last_undegraded":"2024-04-24T08:51:36.503358+0000","last_fullsized":"2024-04-24T08:51:36.503358+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:35:28.035378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.368135+0000","last_change":"2024-04-24T08:48:34.321477+0000","last_active":"2024-04-24T08:51:36.368135+0000","last_peered":"2024-04-24T08:51:36.368135+0000","last_clean":"2024-04-24T08:51:36.368135+0000","last_became_active":"2024-04-24T08:48:34.320860+0000","last_became_peered":"2024-04-24T08:48:34.320860+0000","last_unstale":"2024-04-24T08:51:36.368135+0000","last_undegraded":"2024-04-24T08:51:36.368135+0000","last_fullsized":"2024-04-24T08:51:36.368135+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:26:07.862590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":54,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.368151+0000","last_change":"2024-04-24T08:50:43.014290+0000","last_active":"2024-04-24T08:51:36.368151+0000","last_peered":"2024-04-24T08:51:36.368151+0000","last_clean":"2024-04-24T08:51:36.368151+0000","last_became_active":"2024-04-24T08:50:42.995733+0000","last_became_peered":"2024-04-24T08:50:42.995733+0000","last_unstale":"2024-04-24T08:51:36.368151+0000","last_undegraded":"2024-04-24T08:51:36.368151+0000","last_fullsized":"2024-04-24T08:51:36.368151+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:54:50.270110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.514617+0000","last_change":"2024-04-24T08:48:29.990788+0000","last_active":"2024-04-24T08:51:36.514617+0000","last_peered":"2024-04-24T08:51:36.514617+0000","last_clean":"2024-04-24T08:51:36.514617+0000","last_became_active":"2024-04-24T08:48:29.988476+0000","last_became_peered":"2024-04-24T08:48:29.988476+0000","last_unstale":"2024-04-24T08:51:36.514617+0000","last_undegraded":"2024-04-24T08:51:36.514617+0000","last_fullsized":"2024-04-24T08:51:36.514617+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:08:48.389877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.701568+0000","last_change":"2024-04-24T08:48:34.311020+0000","last_active":"2024-04-24T08:51:36.701568+0000","last_peered":"2024-04-24T08:51:36.701568+0000","last_clean":"2024-04-24T08:51:36.701568+0000","last_became_active":"2024-04-24T08:48:34.310395+0000","last_became_peered":"2024-04-24T08:48:34.310395+0000","last_unstale":"2024-04-24T08:51:36.701568+0000","last_undegraded":"2024-04-24T08:51:36.701568+0000","last_fullsized":"2024-04-24T08:51:36.701568+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:55:38.704954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.729835+0000","last_change":"2024-04-24T08:48:28.969991+0000","last_active":"2024-04-24T08:51:36.729835+0000","last_peered":"2024-04-24T08:51:36.729835+0000","last_clean":"2024-04-24T08:51:36.729835+0000","last_became_active":"2024-04-24T08:48:28.967937+0000","last_became_peered":"2024-04-24T08:48:28.967937+0000","last_unstale":"2024-04-24T08:51:36.729835+0000","last_undegraded":"2024-04-24T08:51:36.729835+0000","last_fullsized":"2024-04-24T08:51:36.729835+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:09:49.736108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.359593+0000","last_change":"2024-04-24T08:48:29.992814+0000","last_active":"2024-04-24T08:51:36.359593+0000","last_peered":"2024-04-24T08:51:36.359593+0000","last_clean":"2024-04-24T08:51:36.359593+0000","last_became_active":"2024-04-24T08:48:29.992188+0000","last_became_peered":"2024-04-24T08:48:29.992188+0000","last_unstale":"2024-04-24T08:51:36.359593+0000","last_undegraded":"2024-04-24T08:51:36.359593+0000","last_fullsized":"2024-04-24T08:51:36.359593+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:29:33.974958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.893584+0000","last_change":"2024-04-24T08:48:34.311196+0000","last_active":"2024-04-24T08:51:36.893584+0000","last_peered":"2024-04-24T08:51:36.893584+0000","last_clean":"2024-04-24T08:51:36.893584+0000","last_became_active":"2024-04-24T08:48:34.310723+0000","last_became_peered":"2024-04-24T08:48:34.310723+0000","last_unstale":"2024-04-24T08:51:36.893584+0000","last_undegraded":"2024-04-24T08:51:36.893584+0000","last_fullsized":"2024-04-24T08:51:36.893584+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:04:30.149510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.513501+0000","last_change":"2024-04-24T08:48:28.978534+0000","last_active":"2024-04-24T08:51:36.513501+0000","last_peered":"2024-04-24T08:51:36.513501+0000","last_clean":"2024-04-24T08:51:36.513501+0000","last_became_active":"2024-04-24T08:48:28.975407+0000","last_became_peered":"2024-04-24T08:48:28.975407+0000","last_unstale":"2024-04-24T08:51:36.513501+0000","last_undegraded":"2024-04-24T08:51:36.513501+0000","last_fullsized":"2024-04-24T08:51:36.513501+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:10:45.355039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.497595+0000","last_change":"2024-04-24T08:48:29.997682+0000","last_active":"2024-04-24T08:51:36.497595+0000","last_peered":"2024-04-24T08:51:36.497595+0000","last_clean":"2024-04-24T08:51:36.497595+0000","last_became_active":"2024-04-24T08:48:29.997223+0000","last_became_peered":"2024-04-24T08:48:29.997223+0000","last_unstale":"2024-04-24T08:51:36.497595+0000","last_undegraded":"2024-04-24T08:51:36.497595+0000","last_fullsized":"2024-04-24T08:51:36.497595+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:03:00.807249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.359842+0000","last_change":"2024-04-24T08:48:34.321477+0000","last_active":"2024-04-24T08:51:36.359842+0000","last_peered":"2024-04-24T08:51:36.359842+0000","last_clean":"2024-04-24T08:51:36.359842+0000","last_became_active":"2024-04-24T08:48:34.321018+0000","last_became_peered":"2024-04-24T08:48:34.321018+0000","last_unstale":"2024-04-24T08:51:36.359842+0000","last_undegraded":"2024-04-24T08:51:36.359842+0000","last_fullsized":"2024-04-24T08:51:36.359842+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:56:50.657298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"93'7","reported_seq":346,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.680385+0000","last_change":"2024-04-24T08:48:28.983722+0000","last_active":"2024-04-24T08:51:36.680385+0000","last_peered":"2024-04-24T08:51:36.680385+0000","last_clean":"2024-04-24T08:51:36.680385+0000","last_became_active":"2024-04-24T08:48:28.983394+0000","last_became_peered":"2024-04-24T08:48:28.983394+0000","last_unstale":"2024-04-24T08:51:36.680385+0000","last_undegraded":"2024-04-24T08:51:36.680385+0000","last_fullsized":"2024-04-24T08:51:36.680385+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:53:14.288742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":306,"num_read_kb":306,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.893545+0000","last_change":"2024-04-24T08:48:29.975009+0000","last_active":"2024-04-24T08:51:36.893545+0000","last_peered":"2024-04-24T08:51:36.893545+0000","last_clean":"2024-04-24T08:51:36.893545+0000","last_became_active":"2024-04-24T08:48:29.974137+0000","last_became_peered":"2024-04-24T08:48:29.974137+0000","last_unstale":"2024-04-24T08:51:36.893545+0000","last_undegraded":"2024-04-24T08:51:36.893545+0000","last_fullsized":"2024-04-24T08:51:36.893545+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:13:16.381923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.497502+0000","last_change":"2024-04-24T08:48:34.312126+0000","last_active":"2024-04-24T08:51:36.497502+0000","last_peered":"2024-04-24T08:51:36.497502+0000","last_clean":"2024-04-24T08:51:36.497502+0000","last_became_active":"2024-04-24T08:48:34.311689+0000","last_became_peered":"2024-04-24T08:48:34.311689+0000","last_unstale":"2024-04-24T08:51:36.497502+0000","last_undegraded":"2024-04-24T08:51:36.497502+0000","last_fullsized":"2024-04-24T08:51:36.497502+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:11:07.942412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.659196+0000","last_change":"2024-04-24T08:48:28.978355+0000","last_active":"2024-04-24T08:51:36.659196+0000","last_peered":"2024-04-24T08:51:36.659196+0000","last_clean":"2024-04-24T08:51:36.659196+0000","last_became_active":"2024-04-24T08:48:28.977351+0000","last_became_peered":"2024-04-24T08:48:28.977351+0000","last_unstale":"2024-04-24T08:51:36.659196+0000","last_undegraded":"2024-04-24T08:51:36.659196+0000","last_fullsized":"2024-04-24T08:51:36.659196+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:00:12.836569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.893780+0000","last_change":"2024-04-24T08:48:29.989015+0000","last_active":"2024-04-24T08:51:36.893780+0000","last_peered":"2024-04-24T08:51:36.893780+0000","last_clean":"2024-04-24T08:51:36.893780+0000","last_became_active":"2024-04-24T08:48:29.988648+0000","last_became_peered":"2024-04-24T08:48:29.988648+0000","last_unstale":"2024-04-24T08:51:36.893780+0000","last_undegraded":"2024-04-24T08:51:36.893780+0000","last_fullsized":"2024-04-24T08:51:36.893780+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:04:52.824894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.737315+0000","last_change":"2024-04-24T08:48:34.314230+0000","last_active":"2024-04-24T08:51:36.737315+0000","last_peered":"2024-04-24T08:51:36.737315+0000","last_clean":"2024-04-24T08:51:36.737315+0000","last_became_active":"2024-04-24T08:48:34.313617+0000","last_became_peered":"2024-04-24T08:48:34.313617+0000","last_unstale":"2024-04-24T08:51:36.737315+0000","last_undegraded":"2024-04-24T08:51:36.737315+0000","last_fullsized":"2024-04-24T08:51:36.737315+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:06:02.307575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.514177+0000","last_change":"2024-04-24T08:48:28.974516+0000","last_active":"2024-04-24T08:51:36.514177+0000","last_peered":"2024-04-24T08:51:36.514177+0000","last_clean":"2024-04-24T08:51:36.514177+0000","last_became_active":"2024-04-24T08:48:28.974194+0000","last_became_peered":"2024-04-24T08:48:28.974194+0000","last_unstale":"2024-04-24T08:51:36.514177+0000","last_undegraded":"2024-04-24T08:51:36.514177+0000","last_fullsized":"2024-04-24T08:51:36.514177+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:03:38.582610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":48,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.893356+0000","last_change":"2024-04-24T08:49:42.128553+0000","last_active":"2024-04-24T08:51:36.893356+0000","last_peered":"2024-04-24T08:51:36.893356+0000","last_clean":"2024-04-24T08:51:36.893356+0000","last_became_active":"2024-04-24T08:49:42.127061+0000","last_became_peered":"2024-04-24T08:49:42.127061+0000","last_unstale":"2024-04-24T08:51:36.893356+0000","last_undegraded":"2024-04-24T08:51:36.893356+0000","last_fullsized":"2024-04-24T08:51:36.893356+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:07:06.798284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.367516+0000","last_change":"2024-04-24T08:49:42.131305+0000","last_active":"2024-04-24T08:51:36.367516+0000","last_peered":"2024-04-24T08:51:36.367516+0000","last_clean":"2024-04-24T08:51:36.367516+0000","last_became_active":"2024-04-24T08:49:42.129680+0000","last_became_peered":"2024-04-24T08:49:42.129680+0000","last_unstale":"2024-04-24T08:51:36.367516+0000","last_undegraded":"2024-04-24T08:51:36.367516+0000","last_fullsized":"2024-04-24T08:51:36.367516+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:14:09.579649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"94'40","reported_seq":387,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.894275+0000","last_change":"2024-04-24T08:48:28.981675+0000","last_active":"2024-04-24T08:51:36.894275+0000","last_peered":"2024-04-24T08:51:36.894275+0000","last_clean":"2024-04-24T08:51:36.894275+0000","last_became_active":"2024-04-24T08:48:28.981082+0000","last_became_peered":"2024-04-24T08:48:28.981082+0000","last_unstale":"2024-04-24T08:51:36.894275+0000","last_undegraded":"2024-04-24T08:51:36.894275+0000","last_fullsized":"2024-04-24T08:51:36.894275+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:22:07.826011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66989,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":314,"num_read_kb":217,"num_write":42,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.504369+0000","last_change":"2024-04-24T08:48:29.986932+0000","last_active":"2024-04-24T08:51:36.504369+0000","last_peered":"2024-04-24T08:51:36.504369+0000","last_clean":"2024-04-24T08:51:36.504369+0000","last_became_active":"2024-04-24T08:48:29.986505+0000","last_became_peered":"2024-04-24T08:48:29.986505+0000","last_unstale":"2024-04-24T08:51:36.504369+0000","last_undegraded":"2024-04-24T08:51:36.504369+0000","last_fullsized":"2024-04-24T08:51:36.504369+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:53:36.050294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.515187+0000","last_change":"2024-04-24T08:48:34.327127+0000","last_active":"2024-04-24T08:51:36.515187+0000","last_peered":"2024-04-24T08:51:36.515187+0000","last_clean":"2024-04-24T08:51:36.515187+0000","last_became_active":"2024-04-24T08:48:34.323942+0000","last_became_peered":"2024-04-24T08:48:34.323942+0000","last_unstale":"2024-04-24T08:51:36.515187+0000","last_undegraded":"2024-04-24T08:51:36.515187+0000","last_fullsized":"2024-04-24T08:51:36.515187+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:22:49.637513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.367615+0000","last_change":"2024-04-24T08:48:28.978618+0000","last_active":"2024-04-24T08:51:36.367615+0000","last_peered":"2024-04-24T08:51:36.367615+0000","last_clean":"2024-04-24T08:51:36.367615+0000","last_became_active":"2024-04-24T08:48:28.978309+0000","last_became_peered":"2024-04-24T08:48:28.978309+0000","last_unstale":"2024-04-24T08:51:36.367615+0000","last_undegraded":"2024-04-24T08:51:36.367615+0000","last_fullsized":"2024-04-24T08:51:36.367615+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:35:50.091398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.729979+0000","last_change":"2024-04-24T08:48:29.985629+0000","last_active":"2024-04-24T08:51:36.729979+0000","last_peered":"2024-04-24T08:51:36.729979+0000","last_clean":"2024-04-24T08:51:36.729979+0000","last_became_active":"2024-04-24T08:48:29.985258+0000","last_became_peered":"2024-04-24T08:48:29.985258+0000","last_unstale":"2024-04-24T08:51:36.729979+0000","last_undegraded":"2024-04-24T08:51:36.729979+0000","last_fullsized":"2024-04-24T08:51:36.729979+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:57:29.302175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.701030+0000","last_change":"2024-04-24T08:49:42.129242+0000","last_active":"2024-04-24T08:51:36.701030+0000","last_peered":"2024-04-24T08:51:36.701030+0000","last_clean":"2024-04-24T08:51:36.701030+0000","last_became_active":"2024-04-24T08:49:42.127967+0000","last_became_peered":"2024-04-24T08:49:42.127967+0000","last_unstale":"2024-04-24T08:51:36.701030+0000","last_undegraded":"2024-04-24T08:51:36.701030+0000","last_fullsized":"2024-04-24T08:51:36.701030+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:42:11.206123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.513770+0000","last_change":"2024-04-24T08:48:28.977840+0000","last_active":"2024-04-24T08:51:36.513770+0000","last_peered":"2024-04-24T08:51:36.513770+0000","last_clean":"2024-04-24T08:51:36.513770+0000","last_became_active":"2024-04-24T08:48:28.975482+0000","last_became_peered":"2024-04-24T08:48:28.975482+0000","last_unstale":"2024-04-24T08:51:36.513770+0000","last_undegraded":"2024-04-24T08:51:36.513770+0000","last_fullsized":"2024-04-24T08:51:36.513770+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:51:52.468627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":52,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.658628+0000","last_change":"2024-04-24T08:49:42.129823+0000","last_active":"2024-04-24T08:51:36.658628+0000","last_peered":"2024-04-24T08:51:36.658628+0000","last_clean":"2024-04-24T08:51:36.658628+0000","last_became_active":"2024-04-24T08:49:42.127254+0000","last_became_peered":"2024-04-24T08:49:42.127254+0000","last_unstale":"2024-04-24T08:51:36.658628+0000","last_undegraded":"2024-04-24T08:51:36.658628+0000","last_fullsized":"2024-04-24T08:51:36.658628+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:51:10.804703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.499181+0000","last_change":"2024-04-24T08:48:34.314956+0000","last_active":"2024-04-24T08:51:36.499181+0000","last_peered":"2024-04-24T08:51:36.499181+0000","last_clean":"2024-04-24T08:51:36.499181+0000","last_became_active":"2024-04-24T08:48:34.314541+0000","last_became_peered":"2024-04-24T08:48:34.314541+0000","last_unstale":"2024-04-24T08:51:36.499181+0000","last_undegraded":"2024-04-24T08:51:36.499181+0000","last_fullsized":"2024-04-24T08:51:36.499181+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:42:48.614000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"93'6","reported_seq":196,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.737247+0000","last_change":"2024-04-24T08:48:28.974529+0000","last_active":"2024-04-24T08:51:36.737247+0000","last_peered":"2024-04-24T08:51:36.737247+0000","last_clean":"2024-04-24T08:51:36.737247+0000","last_became_active":"2024-04-24T08:48:28.973957+0000","last_became_peered":"2024-04-24T08:48:28.973957+0000","last_unstale":"2024-04-24T08:51:36.737247+0000","last_undegraded":"2024-04-24T08:51:36.737247+0000","last_fullsized":"2024-04-24T08:51:36.737247+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:44:46.216389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1081,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":157,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.658134+0000","last_change":"2024-04-24T08:48:29.981995+0000","last_active":"2024-04-24T08:51:36.658134+0000","last_peered":"2024-04-24T08:51:36.658134+0000","last_clean":"2024-04-24T08:51:36.658134+0000","last_became_active":"2024-04-24T08:48:29.980756+0000","last_became_peered":"2024-04-24T08:48:29.980756+0000","last_unstale":"2024-04-24T08:51:36.658134+0000","last_undegraded":"2024-04-24T08:51:36.658134+0000","last_fullsized":"2024-04-24T08:51:36.658134+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:42:52.770069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.731767+0000","last_change":"2024-04-24T08:48:34.308836+0000","last_active":"2024-04-24T08:51:36.731767+0000","last_peered":"2024-04-24T08:51:36.731767+0000","last_clean":"2024-04-24T08:51:36.731767+0000","last_became_active":"2024-04-24T08:48:34.308470+0000","last_became_peered":"2024-04-24T08:48:34.308470+0000","last_unstale":"2024-04-24T08:51:36.731767+0000","last_undegraded":"2024-04-24T08:51:36.731767+0000","last_fullsized":"2024-04-24T08:51:36.731767+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:38:14.906117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":76,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.503278+0000","last_change":"2024-04-24T08:50:43.053334+0000","last_active":"2024-04-24T08:51:36.503278+0000","last_peered":"2024-04-24T08:51:36.503278+0000","last_clean":"2024-04-24T08:51:36.503278+0000","last_became_active":"2024-04-24T08:50:43.007120+0000","last_became_peered":"2024-04-24T08:50:43.007120+0000","last_unstale":"2024-04-24T08:51:36.503278+0000","last_undegraded":"2024-04-24T08:51:36.503278+0000","last_fullsized":"2024-04-24T08:51:36.503278+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:27:37.158069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.513585+0000","last_change":"2024-04-24T08:48:29.980632+0000","last_active":"2024-04-24T08:51:36.513585+0000","last_peered":"2024-04-24T08:51:36.513585+0000","last_clean":"2024-04-24T08:51:36.513585+0000","last_became_active":"2024-04-24T08:48:29.980174+0000","last_became_peered":"2024-04-24T08:48:29.980174+0000","last_unstale":"2024-04-24T08:51:36.513585+0000","last_undegraded":"2024-04-24T08:51:36.513585+0000","last_fullsized":"2024-04-24T08:51:36.513585+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:45:32.709091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.513372+0000","last_change":"2024-04-24T08:48:34.316302+0000","last_active":"2024-04-24T08:51:36.513372+0000","last_peered":"2024-04-24T08:51:36.513372+0000","last_clean":"2024-04-24T08:51:36.513372+0000","last_became_active":"2024-04-24T08:48:34.315783+0000","last_became_peered":"2024-04-24T08:48:34.315783+0000","last_unstale":"2024-04-24T08:51:36.513372+0000","last_undegraded":"2024-04-24T08:51:36.513372+0000","last_fullsized":"2024-04-24T08:51:36.513372+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:29:01.733931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"93'3","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.701302+0000","last_change":"2024-04-24T08:50:43.023981+0000","last_active":"2024-04-24T08:51:36.701302+0000","last_peered":"2024-04-24T08:51:36.701302+0000","last_clean":"2024-04-24T08:51:36.701302+0000","last_became_active":"2024-04-24T08:50:43.003967+0000","last_became_peered":"2024-04-24T08:50:43.003967+0000","last_unstale":"2024-04-24T08:51:36.701302+0000","last_undegraded":"2024-04-24T08:51:36.701302+0000","last_fullsized":"2024-04-24T08:51:36.701302+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:13:22.381793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.701617+0000","last_change":"2024-04-24T08:49:41.975577+0000","last_active":"2024-04-24T08:51:36.701617+0000","last_peered":"2024-04-24T08:51:36.701617+0000","last_clean":"2024-04-24T08:51:36.701617+0000","last_became_active":"2024-04-24T08:49:41.975018+0000","last_became_peered":"2024-04-24T08:49:41.975018+0000","last_unstale":"2024-04-24T08:51:36.701617+0000","last_undegraded":"2024-04-24T08:51:36.701617+0000","last_fullsized":"2024-04-24T08:51:36.701617+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:07:53.051847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.503774+0000","last_change":"2024-04-24T08:49:41.978827+0000","last_active":"2024-04-24T08:51:36.503774+0000","last_peered":"2024-04-24T08:51:36.503774+0000","last_clean":"2024-04-24T08:51:36.503774+0000","last_became_active":"2024-04-24T08:49:41.977741+0000","last_became_peered":"2024-04-24T08:49:41.977741+0000","last_unstale":"2024-04-24T08:51:36.503774+0000","last_undegraded":"2024-04-24T08:51:36.503774+0000","last_fullsized":"2024-04-24T08:51:36.503774+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:16:54.754341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.498955+0000","last_change":"2024-04-24T08:48:28.970726+0000","last_active":"2024-04-24T08:51:36.498955+0000","last_peered":"2024-04-24T08:51:36.498955+0000","last_clean":"2024-04-24T08:51:36.498955+0000","last_became_active":"2024-04-24T08:48:28.969911+0000","last_became_peered":"2024-04-24T08:48:28.969911+0000","last_unstale":"2024-04-24T08:51:36.498955+0000","last_undegraded":"2024-04-24T08:51:36.498955+0000","last_fullsized":"2024-04-24T08:51:36.498955+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:55:26.357256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'4","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.369616+0000","last_change":"2024-04-24T08:48:29.967804+0000","last_active":"2024-04-24T08:51:36.369616+0000","last_peered":"2024-04-24T08:51:36.369616+0000","last_clean":"2024-04-24T08:51:36.369616+0000","last_became_active":"2024-04-24T08:48:29.967431+0000","last_became_peered":"2024-04-24T08:48:29.967431+0000","last_unstale":"2024-04-24T08:51:36.369616+0000","last_undegraded":"2024-04-24T08:51:36.369616+0000","last_fullsized":"2024-04-24T08:51:36.369616+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:00:02.779543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.731564+0000","last_change":"2024-04-24T08:48:34.316844+0000","last_active":"2024-04-24T08:51:36.731564+0000","last_peered":"2024-04-24T08:51:36.731564+0000","last_clean":"2024-04-24T08:51:36.731564+0000","last_became_active":"2024-04-24T08:48:34.316426+0000","last_became_peered":"2024-04-24T08:48:34.316426+0000","last_unstale":"2024-04-24T08:51:36.731564+0000","last_undegraded":"2024-04-24T08:51:36.731564+0000","last_fullsized":"2024-04-24T08:51:36.731564+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:10:13.378443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"94'7","reported_seq":195,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.658832+0000","last_change":"2024-04-24T08:48:28.971544+0000","last_active":"2024-04-24T08:51:36.658832+0000","last_peered":"2024-04-24T08:51:36.658832+0000","last_clean":"2024-04-24T08:51:36.658832+0000","last_became_active":"2024-04-24T08:48:28.970417+0000","last_became_peered":"2024-04-24T08:48:28.970417+0000","last_unstale":"2024-04-24T08:51:36.658832+0000","last_undegraded":"2024-04-24T08:51:36.658832+0000","last_fullsized":"2024-04-24T08:51:36.658832+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:35:04.506126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1472,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":155,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.368499+0000","last_change":"2024-04-24T08:48:29.987542+0000","last_active":"2024-04-24T08:51:36.368499+0000","last_peered":"2024-04-24T08:51:36.368499+0000","last_clean":"2024-04-24T08:51:36.368499+0000","last_became_active":"2024-04-24T08:48:29.987258+0000","last_became_peered":"2024-04-24T08:48:29.987258+0000","last_unstale":"2024-04-24T08:51:36.368499+0000","last_undegraded":"2024-04-24T08:51:36.368499+0000","last_fullsized":"2024-04-24T08:51:36.368499+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:37:23.341132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.732011+0000","last_change":"2024-04-24T08:48:34.322636+0000","last_active":"2024-04-24T08:51:36.732011+0000","last_peered":"2024-04-24T08:51:36.732011+0000","last_clean":"2024-04-24T08:51:36.732011+0000","last_became_active":"2024-04-24T08:48:34.321764+0000","last_became_peered":"2024-04-24T08:48:34.321764+0000","last_unstale":"2024-04-24T08:51:36.732011+0000","last_undegraded":"2024-04-24T08:51:36.732011+0000","last_fullsized":"2024-04-24T08:51:36.732011+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:56:10.563907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.514664+0000","last_change":"2024-04-24T08:48:29.981734+0000","last_active":"2024-04-24T08:51:36.514664+0000","last_peered":"2024-04-24T08:51:36.514664+0000","last_clean":"2024-04-24T08:51:36.514664+0000","last_became_active":"2024-04-24T08:48:29.980491+0000","last_became_peered":"2024-04-24T08:48:29.980491+0000","last_unstale":"2024-04-24T08:51:36.514664+0000","last_undegraded":"2024-04-24T08:51:36.514664+0000","last_fullsized":"2024-04-24T08:51:36.514664+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:08:45.189026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"93'6","reported_seq":194,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.729652+0000","last_change":"2024-04-24T08:48:28.954621+0000","last_active":"2024-04-24T08:51:36.729652+0000","last_peered":"2024-04-24T08:51:36.729652+0000","last_clean":"2024-04-24T08:51:36.729652+0000","last_became_active":"2024-04-24T08:48:28.954139+0000","last_became_peered":"2024-04-24T08:48:28.954139+0000","last_unstale":"2024-04-24T08:51:36.729652+0000","last_undegraded":"2024-04-24T08:51:36.729652+0000","last_fullsized":"2024-04-24T08:51:36.729652+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:59:19.360414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":154,"num_read_kb":154,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.658849+0000","last_change":"2024-04-24T08:48:34.323774+0000","last_active":"2024-04-24T08:51:36.658849+0000","last_peered":"2024-04-24T08:51:36.658849+0000","last_clean":"2024-04-24T08:51:36.658849+0000","last_became_active":"2024-04-24T08:48:34.322214+0000","last_became_peered":"2024-04-24T08:48:34.322214+0000","last_unstale":"2024-04-24T08:51:36.658849+0000","last_undegraded":"2024-04-24T08:51:36.658849+0000","last_fullsized":"2024-04-24T08:51:36.658849+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:51:03.971107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.360105+0000","last_change":"2024-04-24T08:48:29.995372+0000","last_active":"2024-04-24T08:51:36.360105+0000","last_peered":"2024-04-24T08:51:36.360105+0000","last_clean":"2024-04-24T08:51:36.360105+0000","last_became_active":"2024-04-24T08:48:29.994744+0000","last_became_peered":"2024-04-24T08:48:29.994744+0000","last_unstale":"2024-04-24T08:51:36.360105+0000","last_undegraded":"2024-04-24T08:51:36.360105+0000","last_fullsized":"2024-04-24T08:51:36.360105+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:28:55.356390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.737024+0000","last_change":"2024-04-24T08:48:28.974921+0000","last_active":"2024-04-24T08:51:36.737024+0000","last_peered":"2024-04-24T08:51:36.737024+0000","last_clean":"2024-04-24T08:51:36.737024+0000","last_became_active":"2024-04-24T08:48:28.974113+0000","last_became_peered":"2024-04-24T08:48:28.974113+0000","last_unstale":"2024-04-24T08:51:36.737024+0000","last_undegraded":"2024-04-24T08:51:36.737024+0000","last_fullsized":"2024-04-24T08:51:36.737024+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:35:12.635253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":367,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.657967+0000","last_change":"2024-04-24T08:46:45.190615+0000","last_active":"2024-04-24T08:51:36.657967+0000","last_peered":"2024-04-24T08:51:36.657967+0000","last_clean":"2024-04-24T08:51:36.657967+0000","last_became_active":"2024-04-24T08:46:45.190011+0000","last_became_peered":"2024-04-24T08:46:45.190011+0000","last_unstale":"2024-04-24T08:51:36.657967+0000","last_undegraded":"2024-04-24T08:51:36.657967+0000","last_fullsized":"2024-04-24T08:51:36.657967+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:41:57.806569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:41:57.806569+0000","last_clean_scrub_stamp":"2024-04-24T08:41:57.806569+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:13:08.283236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.513988+0000","last_change":"2024-04-24T08:48:34.324588+0000","last_active":"2024-04-24T08:51:36.513988+0000","last_peered":"2024-04-24T08:51:36.513988+0000","last_clean":"2024-04-24T08:51:36.513988+0000","last_became_active":"2024-04-24T08:48:34.321969+0000","last_became_peered":"2024-04-24T08:48:34.321969+0000","last_unstale":"2024-04-24T08:51:36.513988+0000","last_undegraded":"2024-04-24T08:51:36.513988+0000","last_fullsized":"2024-04-24T08:51:36.513988+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:19:56.921198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.729531+0000","last_change":"2024-04-24T08:48:29.981573+0000","last_active":"2024-04-24T08:51:36.729531+0000","last_peered":"2024-04-24T08:51:36.729531+0000","last_clean":"2024-04-24T08:51:36.729531+0000","last_became_active":"2024-04-24T08:48:29.979216+0000","last_became_peered":"2024-04-24T08:48:29.979216+0000","last_unstale":"2024-04-24T08:51:36.729531+0000","last_undegraded":"2024-04-24T08:51:36.729531+0000","last_fullsized":"2024-04-24T08:51:36.729531+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:38:09.164883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":17,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.700456+0000","last_change":"2024-04-24T08:50:43.030115+0000","last_active":"2024-04-24T08:51:36.700456+0000","last_peered":"2024-04-24T08:51:36.700456+0000","last_clean":"2024-04-24T08:51:36.700456+0000","last_became_active":"2024-04-24T08:50:43.005669+0000","last_became_peered":"2024-04-24T08:50:43.005669+0000","last_unstale":"2024-04-24T08:51:36.700456+0000","last_undegraded":"2024-04-24T08:51:36.700456+0000","last_fullsized":"2024-04-24T08:51:36.700456+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:32:32.226068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.700661+0000","last_change":"2024-04-24T08:48:34.317502+0000","last_active":"2024-04-24T08:51:36.700661+0000","last_peered":"2024-04-24T08:51:36.700661+0000","last_clean":"2024-04-24T08:51:36.700661+0000","last_became_active":"2024-04-24T08:48:34.316614+0000","last_became_peered":"2024-04-24T08:48:34.316614+0000","last_unstale":"2024-04-24T08:51:36.700661+0000","last_undegraded":"2024-04-24T08:51:36.700661+0000","last_fullsized":"2024-04-24T08:51:36.700661+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:44:12.469926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.504469+0000","last_change":"2024-04-24T08:48:28.970261+0000","last_active":"2024-04-24T08:51:36.504469+0000","last_peered":"2024-04-24T08:51:36.504469+0000","last_clean":"2024-04-24T08:51:36.504469+0000","last_became_active":"2024-04-24T08:48:28.969813+0000","last_became_peered":"2024-04-24T08:48:28.969813+0000","last_unstale":"2024-04-24T08:51:36.504469+0000","last_undegraded":"2024-04-24T08:51:36.504469+0000","last_fullsized":"2024-04-24T08:51:36.504469+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:46:12.311756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.731284+0000","last_change":"2024-04-24T08:48:29.976712+0000","last_active":"2024-04-24T08:51:36.731284+0000","last_peered":"2024-04-24T08:51:36.731284+0000","last_clean":"2024-04-24T08:51:36.731284+0000","last_became_active":"2024-04-24T08:48:29.976166+0000","last_became_peered":"2024-04-24T08:48:29.976166+0000","last_unstale":"2024-04-24T08:51:36.731284+0000","last_undegraded":"2024-04-24T08:51:36.731284+0000","last_fullsized":"2024-04-24T08:51:36.731284+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:52:53.757829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.359483+0000","last_change":"2024-04-24T08:48:34.310173+0000","last_active":"2024-04-24T08:51:36.359483+0000","last_peered":"2024-04-24T08:51:36.359483+0000","last_clean":"2024-04-24T08:51:36.359483+0000","last_became_active":"2024-04-24T08:48:34.309772+0000","last_became_peered":"2024-04-24T08:48:34.309772+0000","last_unstale":"2024-04-24T08:51:36.359483+0000","last_undegraded":"2024-04-24T08:51:36.359483+0000","last_fullsized":"2024-04-24T08:51:36.359483+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:35:50.302769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"94'50","reported_seq":106,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.770470+0000","last_change":"2024-04-24T08:48:28.970127+0000","last_active":"2024-04-24T08:51:36.770470+0000","last_peered":"2024-04-24T08:51:36.770470+0000","last_clean":"2024-04-24T08:51:36.770470+0000","last_became_active":"2024-04-24T08:48:28.968803+0000","last_became_peered":"2024-04-24T08:48:28.968803+0000","last_unstale":"2024-04-24T08:51:36.770470+0000","last_undegraded":"2024-04-24T08:51:36.770470+0000","last_fullsized":"2024-04-24T08:51:36.770470+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:28:47.782276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":88,"num_write_kb":837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.360223+0000","last_change":"2024-04-24T08:48:29.992964+0000","last_active":"2024-04-24T08:51:36.360223+0000","last_peered":"2024-04-24T08:51:36.360223+0000","last_clean":"2024-04-24T08:51:36.360223+0000","last_became_active":"2024-04-24T08:48:29.987501+0000","last_became_peered":"2024-04-24T08:48:29.987501+0000","last_unstale":"2024-04-24T08:51:36.360223+0000","last_undegraded":"2024-04-24T08:51:36.360223+0000","last_fullsized":"2024-04-24T08:51:36.360223+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:49:19.157052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.366605+0000","last_change":"2024-04-24T08:48:34.315393+0000","last_active":"2024-04-24T08:51:36.366605+0000","last_peered":"2024-04-24T08:51:36.366605+0000","last_clean":"2024-04-24T08:51:36.366605+0000","last_became_active":"2024-04-24T08:48:34.314943+0000","last_became_peered":"2024-04-24T08:48:34.314943+0000","last_unstale":"2024-04-24T08:51:36.366605+0000","last_undegraded":"2024-04-24T08:51:36.366605+0000","last_fullsized":"2024-04-24T08:51:36.366605+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:03:13.312223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.737303+0000","last_change":"2024-04-24T08:48:28.969813+0000","last_active":"2024-04-24T08:51:36.737303+0000","last_peered":"2024-04-24T08:51:36.737303+0000","last_clean":"2024-04-24T08:51:36.737303+0000","last_became_active":"2024-04-24T08:48:28.968485+0000","last_became_peered":"2024-04-24T08:48:28.968485+0000","last_unstale":"2024-04-24T08:51:36.737303+0000","last_undegraded":"2024-04-24T08:51:36.737303+0000","last_fullsized":"2024-04-24T08:51:36.737303+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:32:10.095723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.893652+0000","last_change":"2024-04-24T08:48:29.997731+0000","last_active":"2024-04-24T08:51:36.893652+0000","last_peered":"2024-04-24T08:51:36.893652+0000","last_clean":"2024-04-24T08:51:36.893652+0000","last_became_active":"2024-04-24T08:48:29.997373+0000","last_became_peered":"2024-04-24T08:48:29.997373+0000","last_unstale":"2024-04-24T08:51:36.893652+0000","last_undegraded":"2024-04-24T08:51:36.893652+0000","last_fullsized":"2024-04-24T08:51:36.893652+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:57:19.531173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.736525+0000","last_change":"2024-04-24T08:48:34.326170+0000","last_active":"2024-04-24T08:51:36.736525+0000","last_peered":"2024-04-24T08:51:36.736525+0000","last_clean":"2024-04-24T08:51:36.736525+0000","last_became_active":"2024-04-24T08:48:34.325678+0000","last_became_peered":"2024-04-24T08:48:34.325678+0000","last_unstale":"2024-04-24T08:51:36.736525+0000","last_undegraded":"2024-04-24T08:51:36.736525+0000","last_fullsized":"2024-04-24T08:51:36.736525+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:12:47.003864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.892690+0000","last_change":"2024-04-24T08:48:28.975522+0000","last_active":"2024-04-24T08:51:36.892690+0000","last_peered":"2024-04-24T08:51:36.892690+0000","last_clean":"2024-04-24T08:51:36.892690+0000","last_became_active":"2024-04-24T08:48:28.975093+0000","last_became_peered":"2024-04-24T08:48:28.975093+0000","last_unstale":"2024-04-24T08:51:36.892690+0000","last_undegraded":"2024-04-24T08:51:36.892690+0000","last_fullsized":"2024-04-24T08:51:36.892690+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:37:31.809694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.503358+0000","last_change":"2024-04-24T08:48:29.982502+0000","last_active":"2024-04-24T08:51:36.503358+0000","last_peered":"2024-04-24T08:51:36.503358+0000","last_clean":"2024-04-24T08:51:36.503358+0000","last_became_active":"2024-04-24T08:48:29.982141+0000","last_became_peered":"2024-04-24T08:48:29.982141+0000","last_unstale":"2024-04-24T08:51:36.503358+0000","last_undegraded":"2024-04-24T08:51:36.503358+0000","last_fullsized":"2024-04-24T08:51:36.503358+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:46:15.851359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.499230+0000","last_change":"2024-04-24T08:48:34.322717+0000","last_active":"2024-04-24T08:51:36.499230+0000","last_peered":"2024-04-24T08:51:36.499230+0000","last_clean":"2024-04-24T08:51:36.499230+0000","last_became_active":"2024-04-24T08:48:34.321837+0000","last_became_peered":"2024-04-24T08:48:34.321837+0000","last_unstale":"2024-04-24T08:51:36.499230+0000","last_undegraded":"2024-04-24T08:51:36.499230+0000","last_fullsized":"2024-04-24T08:51:36.499230+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:40:11.557436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.359786+0000","last_change":"2024-04-24T08:48:28.973951+0000","last_active":"2024-04-24T08:51:36.359786+0000","last_peered":"2024-04-24T08:51:36.359786+0000","last_clean":"2024-04-24T08:51:36.359786+0000","last_became_active":"2024-04-24T08:48:28.973131+0000","last_became_peered":"2024-04-24T08:48:28.973131+0000","last_unstale":"2024-04-24T08:51:36.359786+0000","last_undegraded":"2024-04-24T08:51:36.359786+0000","last_fullsized":"2024-04-24T08:51:36.359786+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:45:45.197337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.701038+0000","last_change":"2024-04-24T08:48:29.992689+0000","last_active":"2024-04-24T08:51:36.701038+0000","last_peered":"2024-04-24T08:51:36.701038+0000","last_clean":"2024-04-24T08:51:36.701038+0000","last_became_active":"2024-04-24T08:48:29.988874+0000","last_became_peered":"2024-04-24T08:48:29.988874+0000","last_unstale":"2024-04-24T08:51:36.701038+0000","last_undegraded":"2024-04-24T08:51:36.701038+0000","last_fullsized":"2024-04-24T08:51:36.701038+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:42:41.578424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.737734+0000","last_change":"2024-04-24T08:48:34.318640+0000","last_active":"2024-04-24T08:51:36.737734+0000","last_peered":"2024-04-24T08:51:36.737734+0000","last_clean":"2024-04-24T08:51:36.737734+0000","last_became_active":"2024-04-24T08:48:34.318039+0000","last_became_peered":"2024-04-24T08:48:34.318039+0000","last_unstale":"2024-04-24T08:51:36.737734+0000","last_undegraded":"2024-04-24T08:51:36.737734+0000","last_fullsized":"2024-04-24T08:51:36.737734+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:17:27.110930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"94'6","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.367758+0000","last_change":"2024-04-24T08:48:28.980828+0000","last_active":"2024-04-24T08:51:36.367758+0000","last_peered":"2024-04-24T08:51:36.367758+0000","last_clean":"2024-04-24T08:51:36.367758+0000","last_became_active":"2024-04-24T08:48:28.979814+0000","last_became_peered":"2024-04-24T08:48:28.979814+0000","last_unstale":"2024-04-24T08:51:36.367758+0000","last_undegraded":"2024-04-24T08:51:36.367758+0000","last_fullsized":"2024-04-24T08:51:36.367758+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:02:28.145847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.359850+0000","last_change":"2024-04-24T08:48:29.995003+0000","last_active":"2024-04-24T08:51:36.359850+0000","last_peered":"2024-04-24T08:51:36.359850+0000","last_clean":"2024-04-24T08:51:36.359850+0000","last_became_active":"2024-04-24T08:48:29.994516+0000","last_became_peered":"2024-04-24T08:48:29.994516+0000","last_unstale":"2024-04-24T08:51:36.359850+0000","last_undegraded":"2024-04-24T08:51:36.359850+0000","last_fullsized":"2024-04-24T08:51:36.359850+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:15:46.094534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.738075+0000","last_change":"2024-04-24T08:48:34.311721+0000","last_active":"2024-04-24T08:51:36.738075+0000","last_peered":"2024-04-24T08:51:36.738075+0000","last_clean":"2024-04-24T08:51:36.738075+0000","last_became_active":"2024-04-24T08:48:34.311241+0000","last_became_peered":"2024-04-24T08:48:34.311241+0000","last_unstale":"2024-04-24T08:51:36.738075+0000","last_undegraded":"2024-04-24T08:51:36.738075+0000","last_fullsized":"2024-04-24T08:51:36.738075+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:16:17.679237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.503671+0000","last_change":"2024-04-24T08:48:28.973336+0000","last_active":"2024-04-24T08:51:36.503671+0000","last_peered":"2024-04-24T08:51:36.503671+0000","last_clean":"2024-04-24T08:51:36.503671+0000","last_became_active":"2024-04-24T08:48:28.972666+0000","last_became_peered":"2024-04-24T08:48:28.972666+0000","last_unstale":"2024-04-24T08:51:36.503671+0000","last_undegraded":"2024-04-24T08:51:36.503671+0000","last_fullsized":"2024-04-24T08:51:36.503671+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:53:28.358291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.658302+0000","last_change":"2024-04-24T08:48:29.983498+0000","last_active":"2024-04-24T08:51:36.658302+0000","last_peered":"2024-04-24T08:51:36.658302+0000","last_clean":"2024-04-24T08:51:36.658302+0000","last_became_active":"2024-04-24T08:48:29.983167+0000","last_became_peered":"2024-04-24T08:48:29.983167+0000","last_unstale":"2024-04-24T08:51:36.658302+0000","last_undegraded":"2024-04-24T08:51:36.658302+0000","last_fullsized":"2024-04-24T08:51:36.658302+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:09:56.648047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.893130+0000","last_change":"2024-04-24T08:49:42.129192+0000","last_active":"2024-04-24T08:51:36.893130+0000","last_peered":"2024-04-24T08:51:36.893130+0000","last_clean":"2024-04-24T08:51:36.893130+0000","last_became_active":"2024-04-24T08:49:42.124814+0000","last_became_peered":"2024-04-24T08:49:42.124814+0000","last_unstale":"2024-04-24T08:51:36.893130+0000","last_undegraded":"2024-04-24T08:51:36.893130+0000","last_fullsized":"2024-04-24T08:51:36.893130+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:52:46.273836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6,7],"acting":[1,8,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"94'8","reported_seq":197,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.893736+0000","last_change":"2024-04-24T08:48:28.974528+0000","last_active":"2024-04-24T08:51:36.893736+0000","last_peered":"2024-04-24T08:51:36.893736+0000","last_clean":"2024-04-24T08:51:36.893736+0000","last_became_active":"2024-04-24T08:48:28.974203+0000","last_became_peered":"2024-04-24T08:48:28.974203+0000","last_unstale":"2024-04-24T08:51:36.893736+0000","last_undegraded":"2024-04-24T08:51:36.893736+0000","last_fullsized":"2024-04-24T08:51:36.893736+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:26:15.701204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2445,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":156,"num_read_kb":3,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.659205+0000","last_change":"2024-04-24T08:48:29.984356+0000","last_active":"2024-04-24T08:51:36.659205+0000","last_peered":"2024-04-24T08:51:36.659205+0000","last_clean":"2024-04-24T08:51:36.659205+0000","last_became_active":"2024-04-24T08:48:29.983949+0000","last_became_peered":"2024-04-24T08:48:29.983949+0000","last_unstale":"2024-04-24T08:51:36.659205+0000","last_undegraded":"2024-04-24T08:51:36.659205+0000","last_fullsized":"2024-04-24T08:51:36.659205+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:51:59.409835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.893130+0000","last_change":"2024-04-24T08:48:34.312418+0000","last_active":"2024-04-24T08:51:36.893130+0000","last_peered":"2024-04-24T08:51:36.893130+0000","last_clean":"2024-04-24T08:51:36.893130+0000","last_became_active":"2024-04-24T08:48:34.312035+0000","last_became_peered":"2024-04-24T08:48:34.312035+0000","last_unstale":"2024-04-24T08:51:36.893130+0000","last_undegraded":"2024-04-24T08:51:36.893130+0000","last_fullsized":"2024-04-24T08:51:36.893130+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:27:33.461732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"94'41","reported_seq":83,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.765572+0000","last_change":"2024-04-24T08:48:28.976070+0000","last_active":"2024-04-24T08:51:36.765572+0000","last_peered":"2024-04-24T08:51:36.765572+0000","last_clean":"2024-04-24T08:51:36.765572+0000","last_became_active":"2024-04-24T08:48:28.975703+0000","last_became_peered":"2024-04-24T08:48:28.975703+0000","last_unstale":"2024-04-24T08:51:36.765572+0000","last_undegraded":"2024-04-24T08:51:36.765572+0000","last_fullsized":"2024-04-24T08:51:36.765572+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T16:46:13.227529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'6","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.658745+0000","last_change":"2024-04-24T08:49:42.128814+0000","last_active":"2024-04-24T08:51:36.658745+0000","last_peered":"2024-04-24T08:51:36.658745+0000","last_clean":"2024-04-24T08:51:36.658745+0000","last_became_active":"2024-04-24T08:49:42.127422+0000","last_became_peered":"2024-04-24T08:49:42.127422+0000","last_unstale":"2024-04-24T08:51:36.658745+0000","last_undegraded":"2024-04-24T08:51:36.658745+0000","last_fullsized":"2024-04-24T08:51:36.658745+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:58:25.181798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.512831+0000","last_change":"2024-04-24T08:48:34.320191+0000","last_active":"2024-04-24T08:51:36.512831+0000","last_peered":"2024-04-24T08:51:36.512831+0000","last_clean":"2024-04-24T08:51:36.512831+0000","last_became_active":"2024-04-24T08:48:34.315415+0000","last_became_peered":"2024-04-24T08:48:34.315415+0000","last_unstale":"2024-04-24T08:51:36.512831+0000","last_undegraded":"2024-04-24T08:51:36.512831+0000","last_fullsized":"2024-04-24T08:51:36.512831+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:33.250416+0000","last_clean_scrub_stamp":"2024-04-24T08:48:33.250416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:55:11.372789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.731727+0000","last_change":"2024-04-24T08:48:28.968550+0000","last_active":"2024-04-24T08:51:36.731727+0000","last_peered":"2024-04-24T08:51:36.731727+0000","last_clean":"2024-04-24T08:51:36.731727+0000","last_became_active":"2024-04-24T08:48:28.968120+0000","last_became_peered":"2024-04-24T08:48:28.968120+0000","last_unstale":"2024-04-24T08:51:36.731727+0000","last_undegraded":"2024-04-24T08:51:36.731727+0000","last_fullsized":"2024-04-24T08:51:36.731727+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:27.917599+0000","last_clean_scrub_stamp":"2024-04-24T08:48:27.917599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:46:00.870285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.659317+0000","last_change":"2024-04-24T08:48:29.984755+0000","last_active":"2024-04-24T08:51:36.659317+0000","last_peered":"2024-04-24T08:51:36.659317+0000","last_clean":"2024-04-24T08:51:36.659317+0000","last_became_active":"2024-04-24T08:48:29.982093+0000","last_became_peered":"2024-04-24T08:48:29.982093+0000","last_unstale":"2024-04-24T08:51:36.659317+0000","last_undegraded":"2024-04-24T08:51:36.659317+0000","last_fullsized":"2024-04-24T08:51:36.659317+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:25:22.620152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.700841+0000","last_change":"2024-04-24T08:48:29.983101+0000","last_active":"2024-04-24T08:51:36.700841+0000","last_peered":"2024-04-24T08:51:36.700841+0000","last_clean":"2024-04-24T08:51:36.700841+0000","last_became_active":"2024-04-24T08:48:29.981871+0000","last_became_peered":"2024-04-24T08:48:29.981871+0000","last_unstale":"2024-04-24T08:51:36.700841+0000","last_undegraded":"2024-04-24T08:51:36.700841+0000","last_fullsized":"2024-04-24T08:51:36.700841+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:02:44.582111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":49,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.367516+0000","last_change":"2024-04-24T08:49:42.128974+0000","last_active":"2024-04-24T08:51:36.367516+0000","last_peered":"2024-04-24T08:51:36.367516+0000","last_clean":"2024-04-24T08:51:36.367516+0000","last_became_active":"2024-04-24T08:49:42.127710+0000","last_became_peered":"2024-04-24T08:49:42.127710+0000","last_unstale":"2024-04-24T08:51:36.367516+0000","last_undegraded":"2024-04-24T08:51:36.367516+0000","last_fullsized":"2024-04-24T08:51:36.367516+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:09:20.279852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,8],"acting":[2,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.367156+0000","last_change":"2024-04-24T08:48:29.988241+0000","last_active":"2024-04-24T08:51:36.367156+0000","last_peered":"2024-04-24T08:51:36.367156+0000","last_clean":"2024-04-24T08:51:36.367156+0000","last_became_active":"2024-04-24T08:48:29.987930+0000","last_became_peered":"2024-04-24T08:48:29.987930+0000","last_unstale":"2024-04-24T08:51:36.367156+0000","last_undegraded":"2024-04-24T08:51:36.367156+0000","last_fullsized":"2024-04-24T08:51:36.367156+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T14:42:24.687624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.893443+0000","last_change":"2024-04-24T08:48:29.992441+0000","last_active":"2024-04-24T08:51:36.893443+0000","last_peered":"2024-04-24T08:51:36.893443+0000","last_clean":"2024-04-24T08:51:36.893443+0000","last_became_active":"2024-04-24T08:48:29.988788+0000","last_became_peered":"2024-04-24T08:48:29.988788+0000","last_unstale":"2024-04-24T08:51:36.893443+0000","last_undegraded":"2024-04-24T08:51:36.893443+0000","last_fullsized":"2024-04-24T08:51:36.893443+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:08:31.549509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.731077+0000","last_change":"2024-04-24T08:48:29.982116+0000","last_active":"2024-04-24T08:51:36.731077+0000","last_peered":"2024-04-24T08:51:36.731077+0000","last_clean":"2024-04-24T08:51:36.731077+0000","last_became_active":"2024-04-24T08:48:29.981636+0000","last_became_peered":"2024-04-24T08:48:29.981636+0000","last_unstale":"2024-04-24T08:51:36.731077+0000","last_undegraded":"2024-04-24T08:51:36.731077+0000","last_fullsized":"2024-04-24T08:51:36.731077+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T17:11:01.943425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.701572+0000","last_change":"2024-04-24T08:48:29.982244+0000","last_active":"2024-04-24T08:51:36.701572+0000","last_peered":"2024-04-24T08:51:36.701572+0000","last_clean":"2024-04-24T08:51:36.701572+0000","last_became_active":"2024-04-24T08:48:29.981622+0000","last_became_peered":"2024-04-24T08:48:29.981622+0000","last_unstale":"2024-04-24T08:51:36.701572+0000","last_undegraded":"2024-04-24T08:51:36.701572+0000","last_fullsized":"2024-04-24T08:51:36.701572+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T13:02:57.723949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":46,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.369396+0000","last_change":"2024-04-24T08:48:29.985867+0000","last_active":"2024-04-24T08:51:36.369396+0000","last_peered":"2024-04-24T08:51:36.369396+0000","last_clean":"2024-04-24T08:51:36.369396+0000","last_became_active":"2024-04-24T08:48:29.985430+0000","last_became_peered":"2024-04-24T08:48:29.985430+0000","last_unstale":"2024-04-24T08:51:36.369396+0000","last_undegraded":"2024-04-24T08:51:36.369396+0000","last_fullsized":"2024-04-24T08:51:36.369396+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:09:40.467623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.360401+0000","last_change":"2024-04-24T08:48:29.996474+0000","last_active":"2024-04-24T08:51:36.360401+0000","last_peered":"2024-04-24T08:51:36.360401+0000","last_clean":"2024-04-24T08:51:36.360401+0000","last_became_active":"2024-04-24T08:48:29.995958+0000","last_became_peered":"2024-04-24T08:48:29.995958+0000","last_unstale":"2024-04-24T08:51:36.360401+0000","last_undegraded":"2024-04-24T08:51:36.360401+0000","last_fullsized":"2024-04-24T08:51:36.360401+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:50:19.865125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.513993+0000","last_change":"2024-04-24T08:48:29.978285+0000","last_active":"2024-04-24T08:51:36.513993+0000","last_peered":"2024-04-24T08:51:36.513993+0000","last_clean":"2024-04-24T08:51:36.513993+0000","last_became_active":"2024-04-24T08:48:29.976445+0000","last_became_peered":"2024-04-24T08:48:29.976445+0000","last_unstale":"2024-04-24T08:51:36.513993+0000","last_undegraded":"2024-04-24T08:51:36.513993+0000","last_fullsized":"2024-04-24T08:51:36.513993+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T15:30:29.598733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.359571+0000","last_change":"2024-04-24T08:48:29.994442+0000","last_active":"2024-04-24T08:51:36.359571+0000","last_peered":"2024-04-24T08:51:36.359571+0000","last_clean":"2024-04-24T08:51:36.359571+0000","last_became_active":"2024-04-24T08:48:29.993896+0000","last_became_peered":"2024-04-24T08:48:29.993896+0000","last_unstale":"2024-04-24T08:51:36.359571+0000","last_undegraded":"2024-04-24T08:51:36.359571+0000","last_fullsized":"2024-04-24T08:51:36.359571+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T12:37:14.023511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":47,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.893266+0000","last_change":"2024-04-24T08:49:41.981252+0000","last_active":"2024-04-24T08:51:36.893266+0000","last_peered":"2024-04-24T08:51:36.893266+0000","last_clean":"2024-04-24T08:51:36.893266+0000","last_became_active":"2024-04-24T08:49:41.980495+0000","last_became_peered":"2024-04-24T08:49:41.980495+0000","last_unstale":"2024-04-24T08:51:36.893266+0000","last_undegraded":"2024-04-24T08:51:36.893266+0000","last_fullsized":"2024-04-24T08:51:36.893266+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T11:47:33.579204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,8],"acting":[1,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.730358+0000","last_change":"2024-04-24T08:48:29.986021+0000","last_active":"2024-04-24T08:51:36.730358+0000","last_peered":"2024-04-24T08:51:36.730358+0000","last_clean":"2024-04-24T08:51:36.730358+0000","last_became_active":"2024-04-24T08:48:29.985182+0000","last_became_peered":"2024-04-24T08:48:29.985182+0000","last_unstale":"2024-04-24T08:51:36.730358+0000","last_undegraded":"2024-04-24T08:51:36.730358+0000","last_fullsized":"2024-04-24T08:51:36.730358+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:10:57.143065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T08:51:36.700856+0000","last_change":"2024-04-24T08:48:29.979886+0000","last_active":"2024-04-24T08:51:36.700856+0000","last_peered":"2024-04-24T08:51:36.700856+0000","last_clean":"2024-04-24T08:51:36.700856+0000","last_became_active":"2024-04-24T08:48:29.978779+0000","last_became_peered":"2024-04-24T08:48:29.978779+0000","last_unstale":"2024-04-24T08:51:36.700856+0000","last_undegraded":"2024-04-24T08:51:36.700856+0000","last_fullsized":"2024-04-24T08:51:36.700856+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T08:48:28.922311+0000","last_clean_scrub_stamp":"2024-04-24T08:48:28.922311+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T10:51:51.303990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":602295,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1664,"num_read_kb":942,"num_write":386,"num_write_kb":1475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":134,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2174976,"data_stored":1806885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":310,"ondisk_log_size":310,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383733,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1069056,"data_stored":520983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645372,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":33100,"kb_used_data":1864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712052,"statfs":{"total":95995035648,"available":95961141248,"internally_reserved":0,"allocated":1908736,"data_stored":1311223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808899,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32648,"kb_used_data":1468,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712504,"statfs":{"total":95995035648,"available":95961604096,"internally_reserved":0,"allocated":1503232,"data_stored":927663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070538,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28476,"kb_used_data":1456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716676,"statfs":{"total":95995035648,"available":95965876224,"internally_reserved":0,"allocated":1490944,"data_stored":923923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332178,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32840,"kb_used_data":1660,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712312,"statfs":{"total":95995035648,"available":95961407488,"internally_reserved":0,"allocated":1699840,"data_stored":1128846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495705,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32216,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712936,"statfs":{"total":95995035648,"available":95962046464,"internally_reserved":0,"allocated":1064960,"data_stored":516395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574973,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36468,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708684,"statfs":{"total":95995035648,"available":95957692416,"internally_reserved":0,"allocated":1155072,"data_stored":594758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738500,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36420,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708732,"statfs":{"total":95995035648,"available":95957741568,"internally_reserved":0,"allocated":1105920,"data_stored":528586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411445,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28152,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717000,"statfs":{"total":95995035648,"available":95966208000,"internally_reserved":0,"allocated":1159168,"data_stored":590404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182511,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32248,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712904,"statfs":{"total":95995035648,"available":95962013696,"internally_reserved":0,"allocated":1093632,"data_stored":525563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920871,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32232,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712920,"statfs":{"total":95995035648,"available":95962030080,"internally_reserved":0,"allocated":1081344,"data_stored":520581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757345,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32976,"kb_used_data":1736,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712176,"statfs":{"total":95995035648,"available":95961268224,"internally_reserved":0,"allocated":1777664,"data_stored":1189498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":69625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":71540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":557056,"data_stored":525246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":180224,"data_stored":135836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":421888,"data_stored":394694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":258048,"data_stored":200818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T08:51:43.222 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T08:51:43.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:43.429+0000 7f83bb000640 1 -- 172.21.15.83:0/164450718 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83b4104260 msgr2=0x7f83b4104680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:43.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:43.429+0000 7f83bb000640 1 --2- 172.21.15.83:0/164450718 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83b4104260 0x7f83b4104680 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f83a4009900 tx=0x7f83a402a3f0 comp rx=0 tx=0).stop 2024-04-24T08:51:43.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:43.429+0000 7f83bb000640 1 -- 172.21.15.83:0/164450718 shutdown_connections 2024-04-24T08:51:43.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:43.429+0000 7f83bb000640 1 --2- 172.21.15.83:0/164450718 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f83b4105e10 0x7f83b410c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:43.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:43.429+0000 7f83bb000640 1 --2- 172.21.15.83:0/164450718 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83b4105450 0x7f83b41058d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:43.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:43.429+0000 7f83bb000640 1 --2- 172.21.15.83:0/164450718 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83b4104260 0x7f83b4104680 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:43.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:43.429+0000 7f83bb000640 1 -- 172.21.15.83:0/164450718 >> 172.21.15.83:0/164450718 conn(0x7f83b40ff7f0 msgr2=0x7f83b4101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:43.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:43.429+0000 7f83bb000640 1 -- 172.21.15.83:0/164450718 shutdown_connections 2024-04-24T08:51:43.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:43.429+0000 7f83bb000640 1 -- 172.21.15.83:0/164450718 wait complete. 2024-04-24T08:51:43.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:43.430+0000 7f83bb000640 1 Processor -- start 2024-04-24T08:51:43.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:43.430+0000 7f83bb000640 1 -- start start 2024-04-24T08:51:43.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:43.430+0000 7f83bb000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f83b4104260 0x7f83b419a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:43.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:43.430+0000 7f83bb000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83b4105450 0x7f83b419a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:43.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:43.430+0000 7f83bb000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83b4105e10 0x7f83b41a1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:43.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:43.430+0000 7f83bb000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f83b410fdf0 con 0x7f83b4105e10 2024-04-24T08:51:43.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:43.430+0000 7f83bb000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f83b410fc70 con 0x7f83b4105450 2024-04-24T08:51:43.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:43.430+0000 7f83bb000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f83b410ff70 con 0x7f83b4104260 2024-04-24T08:51:43.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:43.430+0000 7f83b9000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83b4105e10 0x7f83b41a1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:43.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:43.431+0000 7f83b9000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83b4105e10 0x7f83b41a1a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55872/0 (socket says 172.21.15.83:55872) 2024-04-24T08:51:43.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:43.431+0000 7f83b9000640 1 -- 172.21.15.83:0/2011602406 learned_addr learned my addr 172.21.15.83:0/2011602406 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:51:43.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:43.431+0000 7f83b3e00640 1 --2- 172.21.15.83:0/2011602406 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f83b4104260 0x7f83b419a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:43.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:43.431+0000 7f83b3400640 1 --2- 172.21.15.83:0/2011602406 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83b4105450 0x7f83b419a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:43.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:43.431+0000 7f83b9000640 1 -- 172.21.15.83:0/2011602406 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f83b4104260 msgr2=0x7f83b419a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:43.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:43.431+0000 7f83b9000640 1 --2- 172.21.15.83:0/2011602406 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f83b4104260 0x7f83b419a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:43.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:43.431+0000 7f83b9000640 1 -- 172.21.15.83:0/2011602406 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83b4105450 msgr2=0x7f83b419a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:43.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:43.431+0000 7f83b9000640 1 --2- 172.21.15.83:0/2011602406 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83b4105450 0x7f83b419a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:43.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:43.431+0000 7f83b9000640 1 -- 172.21.15.83:0/2011602406 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83b41a20d0 con 0x7f83b4105e10 2024-04-24T08:51:43.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:43.431+0000 7f83b9000640 1 --2- 172.21.15.83:0/2011602406 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83b4105e10 0x7f83b41a1a60 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f83ac00eb10 tx=0x7f83ac00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:43.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:43.431+0000 7f83b0c00640 1 -- 172.21.15.83:0/2011602406 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83ac004b20 con 0x7f83b4105e10 2024-04-24T08:51:43.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:43.432+0000 7f83b0c00640 1 -- 172.21.15.83:0/2011602406 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f83ac004cc0 con 0x7f83b4105e10 2024-04-24T08:51:43.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:43.432+0000 7f83b0c00640 1 -- 172.21.15.83:0/2011602406 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83ac010690 con 0x7f83b4105e10 2024-04-24T08:51:43.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:43.432+0000 7f83bb000640 1 -- 172.21.15.83:0/2011602406 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83b41a23c0 con 0x7f83b4105e10 2024-04-24T08:51:43.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:43.432+0000 7f83bb000640 1 -- 172.21.15.83:0/2011602406 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83b41a28c0 con 0x7f83b4105e10 2024-04-24T08:51:43.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:43.433+0000 7f83bb000640 1 -- 172.21.15.83:0/2011602406 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83b410b650 con 0x7f83b4105e10 2024-04-24T08:51:43.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:43.435+0000 7f83b0c00640 1 -- 172.21.15.83:0/2011602406 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f83ac005020 con 0x7f83b4105e10 2024-04-24T08:51:43.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:43.437+0000 7f83b0c00640 1 --2- 172.21.15.83:0/2011602406 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f83980a3e30 0x7f83980a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:43.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:43.437+0000 7f83b0c00640 1 -- 172.21.15.83:0/2011602406 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f83ac0d1320 con 0x7f83b4105e10 2024-04-24T08:51:43.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:43.437+0000 7f83b3e00640 1 --2- 172.21.15.83:0/2011602406 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f83980a3e30 0x7f83980a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:43.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:43.438+0000 7f83b3e00640 1 --2- 172.21.15.83:0/2011602406 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f83980a3e30 0x7f83980a62f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f83a4005ab0 tx=0x7f83a4005a20 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:43.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:43.441+0000 7f83b0c00640 1 -- 172.21.15.83:0/2011602406 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f83ac09d480 con 0x7f83b4105e10 2024-04-24T08:51:43.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:42 smithi138 ceph-mon[31441]: Adjusting osd_memory_target on smithi083 to 176.4M 2024-04-24T08:51:43.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:42 smithi138 ceph-mon[31441]: Unable to set osd_memory_target on smithi083 to 185061990: error parsing value: Value '185061990' is below minimum 939524096 2024-04-24T08:51:43.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:42 smithi138 ceph-mon[31441]: Adjusting osd_memory_target on smithi189 to 176.4M 2024-04-24T08:51:43.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:42 smithi138 ceph-mon[31441]: Unable to set osd_memory_target on smithi189 to 185064857: error parsing value: Value '185064857' is below minimum 939524096 2024-04-24T08:51:43.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:42 smithi138 ceph-mon[31441]: Adjusting osd_memory_target on smithi138 to 176.4M 2024-04-24T08:51:43.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:42 smithi138 ceph-mon[31441]: Unable to set osd_memory_target on smithi138 to 185047654: error parsing value: Value '185047654' is below minimum 939524096 2024-04-24T08:51:43.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:42 smithi138 ceph-mon[31441]: Updating smithi083:/etc/ceph/ceph.conf 2024-04-24T08:51:43.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:42 smithi138 ceph-mon[31441]: Updating smithi138:/etc/ceph/ceph.conf 2024-04-24T08:51:43.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:42 smithi138 ceph-mon[31441]: Updating smithi189:/etc/ceph/ceph.conf 2024-04-24T08:51:43.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:43 smithi189 ceph-mon[31791]: Adjusting osd_memory_target on smithi083 to 176.4M 2024-04-24T08:51:43.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:43 smithi189 ceph-mon[31791]: Unable to set osd_memory_target on smithi083 to 185061990: error parsing value: Value '185061990' is below minimum 939524096 2024-04-24T08:51:43.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:43 smithi189 ceph-mon[31791]: Adjusting osd_memory_target on smithi189 to 176.4M 2024-04-24T08:51:43.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:43 smithi189 ceph-mon[31791]: Unable to set osd_memory_target on smithi189 to 185064857: error parsing value: Value '185064857' is below minimum 939524096 2024-04-24T08:51:43.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:43 smithi189 ceph-mon[31791]: Adjusting osd_memory_target on smithi138 to 176.4M 2024-04-24T08:51:43.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:43 smithi189 ceph-mon[31791]: Unable to set osd_memory_target on smithi138 to 185047654: error parsing value: Value '185047654' is below minimum 939524096 2024-04-24T08:51:43.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:43 smithi189 ceph-mon[31791]: Updating smithi083:/etc/ceph/ceph.conf 2024-04-24T08:51:43.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:43 smithi189 ceph-mon[31791]: Updating smithi138:/etc/ceph/ceph.conf 2024-04-24T08:51:43.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:43 smithi189 ceph-mon[31791]: Updating smithi189:/etc/ceph/ceph.conf 2024-04-24T08:51:43.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:43.650+0000 7f83bb000640 1 -- 172.21.15.83:0/2011602406 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f83b4062c00 con 0x7f83b4105e10 2024-04-24T08:51:43.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:43.652+0000 7f83b0c00640 1 -- 172.21.15.83:0/2011602406 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+25265 (secure 0 0 0) 0x7f83ac09cd80 con 0x7f83b4105e10 2024-04-24T08:51:43.653 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:51:43.653 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":94,"fsid":"bd3e039e-0215-11ef-bc93-c7b262605968","created":"2024-04-24T08:37:32.209944+0000","modified":"2024-04-24T08:51:36.347834+0000","last_up_change":"2024-04-24T08:47:17.966041+0000","last_in_change":"2024-04-24T08:46:53.835936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T08:41:56.834372+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T08:48:26.978311+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T08:48:28.352731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T08:48:32.275167+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2cd77f3a-19dd-4c6e-a199-1dcbd5faa679","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2132675555},{"type":"v1","addr":"172.21.15.83:6803","nonce":2132675555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2132675555},{"type":"v1","addr":"172.21.15.83:6805","nonce":2132675555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2132675555},{"type":"v1","addr":"172.21.15.83:6809","nonce":2132675555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2132675555},{"type":"v1","addr":"172.21.15.83:6807","nonce":2132675555}]},"public_addr":"172.21.15.83:6803/2132675555","cluster_addr":"172.21.15.83:6805/2132675555","heartbeat_back_addr":"172.21.15.83:6809/2132675555","heartbeat_front_addr":"172.21.15.83:6807/2132675555","state":["exists","up"]},{"osd":1,"uuid":"aba7f5df-2a4c-4c23-b502-d193605a5b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":3589158880},{"type":"v1","addr":"172.21.15.138:6801","nonce":3589158880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":3589158880},{"type":"v1","addr":"172.21.15.138:6803","nonce":3589158880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":3589158880},{"type":"v1","addr":"172.21.15.138:6807","nonce":3589158880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":3589158880},{"type":"v1","addr":"172.21.15.138:6805","nonce":3589158880}]},"public_addr":"172.21.15.138:6801/3589158880","cluster_addr":"172.21.15.138:6803/3589158880","heartbeat_back_addr":"172.21.15.138:6807/3589158880","heartbeat_front_addr":"172.21.15.138:6805/3589158880","state":["exists","up"]},{"osd":2,"uuid":"071bc167-2333-4420-bb58-bffbc10d7641","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":3169072688},{"type":"v1","addr":"172.21.15.189:6801","nonce":3169072688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":3169072688},{"type":"v1","addr":"172.21.15.189:6803","nonce":3169072688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":3169072688},{"type":"v1","addr":"172.21.15.189:6807","nonce":3169072688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":3169072688},{"type":"v1","addr":"172.21.15.189:6805","nonce":3169072688}]},"public_addr":"172.21.15.189:6801/3169072688","cluster_addr":"172.21.15.189:6803/3169072688","heartbeat_back_addr":"172.21.15.189:6807/3169072688","heartbeat_front_addr":"172.21.15.189:6805/3169072688","state":["exists","up"]},{"osd":3,"uuid":"f5624d78-4455-4e07-8c1a-f47c94cddb23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3131092450},{"type":"v1","addr":"172.21.15.83:6811","nonce":3131092450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3131092450},{"type":"v1","addr":"172.21.15.83:6813","nonce":3131092450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3131092450},{"type":"v1","addr":"172.21.15.83:6817","nonce":3131092450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3131092450},{"type":"v1","addr":"172.21.15.83:6815","nonce":3131092450}]},"public_addr":"172.21.15.83:6811/3131092450","cluster_addr":"172.21.15.83:6813/3131092450","heartbeat_back_addr":"172.21.15.83:6817/3131092450","heartbeat_front_addr":"172.21.15.83:6815/3131092450","state":["exists","up"]},{"osd":4,"uuid":"7b50cd14-9385-4535-a7b0-1c1da93c8548","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":823775360},{"type":"v1","addr":"172.21.15.138:6809","nonce":823775360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":823775360},{"type":"v1","addr":"172.21.15.138:6811","nonce":823775360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":823775360},{"type":"v1","addr":"172.21.15.138:6815","nonce":823775360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":823775360},{"type":"v1","addr":"172.21.15.138:6813","nonce":823775360}]},"public_addr":"172.21.15.138:6809/823775360","cluster_addr":"172.21.15.138:6811/823775360","heartbeat_back_addr":"172.21.15.138:6815/823775360","heartbeat_front_addr":"172.21.15.138:6813/823775360","state":["exists","up"]},{"osd":5,"uuid":"9c647797-35bc-449f-be84-af97d2c69c2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":1399817431},{"type":"v1","addr":"172.21.15.189:6809","nonce":1399817431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":1399817431},{"type":"v1","addr":"172.21.15.189:6811","nonce":1399817431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":1399817431},{"type":"v1","addr":"172.21.15.189:6815","nonce":1399817431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":1399817431},{"type":"v1","addr":"172.21.15.189:6813","nonce":1399817431}]},"public_addr":"172.21.15.189:6809/1399817431","cluster_addr":"172.21.15.189:6811/1399817431","heartbeat_back_addr":"172.21.15.189:6815/1399817431","heartbeat_front_addr":"172.21.15.189:6813/1399817431","state":["exists","up"]},{"osd":6,"uuid":"8f056584-16c8-4454-b445-3f780f26c196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":1611358483},{"type":"v1","addr":"172.21.15.83:6819","nonce":1611358483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1611358483},{"type":"v1","addr":"172.21.15.83:6821","nonce":1611358483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1611358483},{"type":"v1","addr":"172.21.15.83:6825","nonce":1611358483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":1611358483},{"type":"v1","addr":"172.21.15.83:6823","nonce":1611358483}]},"public_addr":"172.21.15.83:6819/1611358483","cluster_addr":"172.21.15.83:6821/1611358483","heartbeat_back_addr":"172.21.15.83:6825/1611358483","heartbeat_front_addr":"172.21.15.83:6823/1611358483","state":["exists","up"]},{"osd":7,"uuid":"7603efe3-da77-4b7f-9d03-29c4f293b6d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":3557856706},{"type":"v1","addr":"172.21.15.138:6817","nonce":3557856706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":3557856706},{"type":"v1","addr":"172.21.15.138:6819","nonce":3557856706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":3557856706},{"type":"v1","addr":"172.21.15.138:6823","nonce":3557856706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":3557856706},{"type":"v1","addr":"172.21.15.138:6821","nonce":3557856706}]},"public_addr":"172.21.15.138:6817/3557856706","cluster_addr":"172.21.15.138:6819/3557856706","heartbeat_back_addr":"172.21.15.138:6823/3557856706","heartbeat_front_addr":"172.21.15.138:6821/3557856706","state":["exists","up"]},{"osd":8,"uuid":"27d7d1b8-d6a0-417e-b8aa-596325aa1a77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":207018554},{"type":"v1","addr":"172.21.15.189:6817","nonce":207018554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":207018554},{"type":"v1","addr":"172.21.15.189:6819","nonce":207018554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":207018554},{"type":"v1","addr":"172.21.15.189:6823","nonce":207018554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":207018554},{"type":"v1","addr":"172.21.15.189:6821","nonce":207018554}]},"public_addr":"172.21.15.189:6817/207018554","cluster_addr":"172.21.15.189:6819/207018554","heartbeat_back_addr":"172.21.15.189:6823/207018554","heartbeat_front_addr":"172.21.15.189:6821/207018554","state":["exists","up"]},{"osd":9,"uuid":"baf19b98-881b-4d3a-9f72-8aa2dd73cad6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":519636751},{"type":"v1","addr":"172.21.15.83:6827","nonce":519636751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":519636751},{"type":"v1","addr":"172.21.15.83:6829","nonce":519636751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":519636751},{"type":"v1","addr":"172.21.15.83:6833","nonce":519636751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":519636751},{"type":"v1","addr":"172.21.15.83:6831","nonce":519636751}]},"public_addr":"172.21.15.83:6827/519636751","cluster_addr":"172.21.15.83:6829/519636751","heartbeat_back_addr":"172.21.15.83:6833/519636751","heartbeat_front_addr":"172.21.15.83:6831/519636751","state":["exists","up"]},{"osd":10,"uuid":"eb346717-5c07-47c8-95be-0e68fe449fdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1130872098},{"type":"v1","addr":"172.21.15.138:6825","nonce":1130872098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1130872098},{"type":"v1","addr":"172.21.15.138:6827","nonce":1130872098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1130872098},{"type":"v1","addr":"172.21.15.138:6831","nonce":1130872098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1130872098},{"type":"v1","addr":"172.21.15.138:6829","nonce":1130872098}]},"public_addr":"172.21.15.138:6825/1130872098","cluster_addr":"172.21.15.138:6827/1130872098","heartbeat_back_addr":"172.21.15.138:6831/1130872098","heartbeat_front_addr":"172.21.15.138:6829/1130872098","state":["exists","up"]},{"osd":11,"uuid":"8e37568e-981c-45b8-b2af-2f1b73898542","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":2998782690},{"type":"v1","addr":"172.21.15.189:6825","nonce":2998782690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":2998782690},{"type":"v1","addr":"172.21.15.189:6827","nonce":2998782690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":2998782690},{"type":"v1","addr":"172.21.15.189:6831","nonce":2998782690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":2998782690},{"type":"v1","addr":"172.21.15.189:6829","nonce":2998782690}]},"public_addr":"172.21.15.189:6825/2998782690","cluster_addr":"172.21.15.189:6827/2998782690","heartbeat_back_addr":"172.21.15.189:6831/2998782690","heartbeat_front_addr":"172.21.15.189:6829/2998782690","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:40:47.980103+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:41:21.970795+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:41:55.160426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:42:30.400475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:43:05.705663+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:43:40.112312+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:44:16.781289+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:44:52.151187+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:45:28.055944+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:46:04.969301+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:46:40.918306+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:47:16.886874+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"2.1d","mappings":[{"from":2,"to":1}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.2a","mappings":[{"from":3,"to":8}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.a","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:0/1096290901":"2024-04-25T08:51:36.347763+0000","172.21.15.83:6800/2550826560":"2024-04-25T08:51:36.347763+0000","172.21.15.83:0/1743153945":"2024-04-25T08:51:36.347763+0000","172.21.15.83:0/3306681366":"2024-04-25T08:51:36.347763+0000","172.21.15.83:6801/820049883":"2024-04-25T08:38:40.661315+0000","172.21.15.83:6801/2550826560":"2024-04-25T08:51:36.347763+0000","172.21.15.83:0/4272874138":"2024-04-25T08:38:40.661315+0000","172.21.15.83:0/4212994335":"2024-04-25T08:38:40.661315+0000","172.21.15.83:6801/1996371091":"2024-04-25T08:38:05.800856+0000","172.21.15.83:0/1820663683":"2024-04-25T08:38:05.800856+0000","172.21.15.83:0/3802897262":"2024-04-25T08:38:05.800856+0000","172.21.15.83:6800/1996371091":"2024-04-25T08:38:05.800856+0000","172.21.15.83:6800/820049883":"2024-04-25T08:38:40.661315+0000","172.21.15.83:0/1366125049":"2024-04-25T08:38:05.800856+0000","172.21.15.83:0/4159820288":"2024-04-25T08:51:36.347763+0000","172.21.15.83:0/340452868":"2024-04-25T08:38:40.661315+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T08:51:43.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:43.654+0000 7f83bb000640 1 -- 172.21.15.83:0/2011602406 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f83980a3e30 msgr2=0x7f83980a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:43.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:43.654+0000 7f83bb000640 1 --2- 172.21.15.83:0/2011602406 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f83980a3e30 0x7f83980a62f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f83a4005ab0 tx=0x7f83a4005a20 comp rx=0 tx=0).stop 2024-04-24T08:51:43.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:43.654+0000 7f83bb000640 1 -- 172.21.15.83:0/2011602406 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83b4105e10 msgr2=0x7f83b41a1a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:43.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:43.654+0000 7f83bb000640 1 --2- 172.21.15.83:0/2011602406 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83b4105e10 0x7f83b41a1a60 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f83ac00eb10 tx=0x7f83ac00c2d0 comp rx=0 tx=0).stop 2024-04-24T08:51:43.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:43.654+0000 7f83bb000640 1 -- 172.21.15.83:0/2011602406 shutdown_connections 2024-04-24T08:51:43.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:43.654+0000 7f83bb000640 1 --2- 172.21.15.83:0/2011602406 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f83980a3e30 0x7f83980a62f0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:43.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:43.654+0000 7f83bb000640 1 --2- 172.21.15.83:0/2011602406 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83b4105e10 0x7f83b41a1a60 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:43.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:43.654+0000 7f83bb000640 1 --2- 172.21.15.83:0/2011602406 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83b4105450 0x7f83b419a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:43.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:43.654+0000 7f83bb000640 1 --2- 172.21.15.83:0/2011602406 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f83b4104260 0x7f83b419a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:43.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:43.654+0000 7f83bb000640 1 -- 172.21.15.83:0/2011602406 >> 172.21.15.83:0/2011602406 conn(0x7f83b40ff7f0 msgr2=0x7f83b4108820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:43.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:43.655+0000 7f83bb000640 1 -- 172.21.15.83:0/2011602406 shutdown_connections 2024-04-24T08:51:43.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:43.655+0000 7f83bb000640 1 -- 172.21.15.83:0/2011602406 wait complete. 2024-04-24T08:51:43.670 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T08:41:56.834372+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T08:48:26.978311+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T08:48:28.352731+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-24T08:48:32.275167+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-24T08:51:43.671 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T08:51:43.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:43.844+0000 7ff266a00640 1 -- 172.21.15.83:0/1447834980 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff260074cf0 msgr2=0x7ff26010e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:43.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:43.844+0000 7ff266a00640 1 --2- 172.21.15.83:0/1447834980 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff260074cf0 0x7ff26010e200 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7ff24c009900 tx=0x7ff24c02a480 comp rx=0 tx=0).stop 2024-04-24T08:51:43.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:43.844+0000 7ff266a00640 1 -- 172.21.15.83:0/1447834980 shutdown_connections 2024-04-24T08:51:43.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:43.844+0000 7ff266a00640 1 --2- 172.21.15.83:0/1447834980 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff26010e740 0x7ff260110b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:43.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:43.844+0000 7ff266a00640 1 --2- 172.21.15.83:0/1447834980 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff260074cf0 0x7ff26010e200 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:43.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:43.844+0000 7ff266a00640 1 --2- 172.21.15.83:0/1447834980 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff2600743d0 0x7ff2600747b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:43.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:43.844+0000 7ff266a00640 1 -- 172.21.15.83:0/1447834980 >> 172.21.15.83:0/1447834980 conn(0x7ff2600fd9d0 msgr2=0x7ff2600ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:43.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:43.844+0000 7ff266a00640 1 -- 172.21.15.83:0/1447834980 shutdown_connections 2024-04-24T08:51:43.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:43.844+0000 7ff266a00640 1 -- 172.21.15.83:0/1447834980 wait complete. 2024-04-24T08:51:43.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:43.845+0000 7ff266a00640 1 Processor -- start 2024-04-24T08:51:43.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:43.845+0000 7ff266a00640 1 -- start start 2024-04-24T08:51:43.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:43.846+0000 7ff266a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2600743d0 0x7ff2601a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:43.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:43.846+0000 7ff266a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff260074cf0 0x7ff2601a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:43.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:43.846+0000 7ff266a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff26010e740 0x7ff26019a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:43.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:43.846+0000 7ff266a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff2601143c0 con 0x7ff2600743d0 2024-04-24T08:51:43.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:43.846+0000 7ff266a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff260114240 con 0x7ff26010e740 2024-04-24T08:51:43.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:43.846+0000 7ff266a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff260114540 con 0x7ff260074cf0 2024-04-24T08:51:43.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:43.846+0000 7ff265600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2600743d0 0x7ff2601a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:43.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:43.846+0000 7ff265600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2600743d0 0x7ff2601a0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55886/0 (socket says 172.21.15.83:55886) 2024-04-24T08:51:43.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:43.846+0000 7ff265600640 1 -- 172.21.15.83:0/984658657 learned_addr learned my addr 172.21.15.83:0/984658657 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:51:43.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:43.846+0000 7ff266000640 1 --2- 172.21.15.83:0/984658657 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff26010e740 0x7ff26019a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:43.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:43.846+0000 7ff264c00640 1 --2- 172.21.15.83:0/984658657 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff260074cf0 0x7ff2601a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:43.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:43.846+0000 7ff265600640 1 -- 172.21.15.83:0/984658657 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff260074cf0 msgr2=0x7ff2601a0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:43.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:43.846+0000 7ff265600640 1 --2- 172.21.15.83:0/984658657 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff260074cf0 0x7ff2601a0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:43.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:43.846+0000 7ff265600640 1 -- 172.21.15.83:0/984658657 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff26010e740 msgr2=0x7ff26019a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:43.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:43.846+0000 7ff265600640 1 --2- 172.21.15.83:0/984658657 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff26010e740 0x7ff26019a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:43.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:43.846+0000 7ff265600640 1 -- 172.21.15.83:0/984658657 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff26019b000 con 0x7ff2600743d0 2024-04-24T08:51:43.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:43.846+0000 7ff264c00640 1 --2- 172.21.15.83:0/984658657 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff260074cf0 0x7ff2601a0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:51:43.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:43.847+0000 7ff265600640 1 --2- 172.21.15.83:0/984658657 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2600743d0 0x7ff2601a0680 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7ff25400db80 tx=0x7ff25400b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:43.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:43.847+0000 7ff25e000640 1 -- 172.21.15.83:0/984658657 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff254017070 con 0x7ff2600743d0 2024-04-24T08:51:43.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:43.847+0000 7ff25e000640 1 -- 172.21.15.83:0/984658657 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff254003b70 con 0x7ff2600743d0 2024-04-24T08:51:43.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:43.847+0000 7ff25e000640 1 -- 172.21.15.83:0/984658657 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2540104c0 con 0x7ff2600743d0 2024-04-24T08:51:43.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:43.847+0000 7ff266a00640 1 -- 172.21.15.83:0/984658657 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff26019b2f0 con 0x7ff2600743d0 2024-04-24T08:51:43.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:43.847+0000 7ff266a00640 1 -- 172.21.15.83:0/984658657 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2601a7600 con 0x7ff2600743d0 2024-04-24T08:51:43.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:43.848+0000 7ff266a00640 1 -- 172.21.15.83:0/984658657 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff26019b630 con 0x7ff2600743d0 2024-04-24T08:51:43.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:43.851+0000 7ff25e000640 1 -- 172.21.15.83:0/984658657 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7ff254004020 con 0x7ff2600743d0 2024-04-24T08:51:43.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:43.853+0000 7ff25e000640 1 --2- 172.21.15.83:0/984658657 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff2480a3db0 0x7ff2480a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:43.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:43.854+0000 7ff25e000640 1 -- 172.21.15.83:0/984658657 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7ff2540d0ec0 con 0x7ff2600743d0 2024-04-24T08:51:43.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:43.854+0000 7ff264c00640 1 --2- 172.21.15.83:0/984658657 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff2480a3db0 0x7ff2480a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:43.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:43.854+0000 7ff264c00640 1 --2- 172.21.15.83:0/984658657 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff2480a3db0 0x7ff2480a6270 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff24c000c00 tx=0x7ff24c005a20 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:43.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:43.858+0000 7ff25e000640 1 -- 172.21.15.83:0/984658657 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff25409cfc0 con 0x7ff2600743d0 2024-04-24T08:51:44.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.094+0000 7ff266a00640 1 -- 172.21.15.83:0/984658657 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff26019bc70 con 0x7ff2600743d0 2024-04-24T08:51:44.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.095+0000 7ff25e000640 1 -- 172.21.15.83:0/984658657 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7ff25409c8c0 con 0x7ff2600743d0 2024-04-24T08:51:44.095 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-24T08:51:44.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.098+0000 7ff266a00640 1 -- 172.21.15.83:0/984658657 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff2480a3db0 msgr2=0x7ff2480a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:44.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.098+0000 7ff266a00640 1 --2- 172.21.15.83:0/984658657 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff2480a3db0 0x7ff2480a6270 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff24c000c00 tx=0x7ff24c005a20 comp rx=0 tx=0).stop 2024-04-24T08:51:44.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.098+0000 7ff266a00640 1 -- 172.21.15.83:0/984658657 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2600743d0 msgr2=0x7ff2601a0680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:44.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.098+0000 7ff266a00640 1 --2- 172.21.15.83:0/984658657 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2600743d0 0x7ff2601a0680 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7ff25400db80 tx=0x7ff25400b630 comp rx=0 tx=0).stop 2024-04-24T08:51:44.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.098+0000 7ff266a00640 1 -- 172.21.15.83:0/984658657 shutdown_connections 2024-04-24T08:51:44.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.098+0000 7ff266a00640 1 --2- 172.21.15.83:0/984658657 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff2480a3db0 0x7ff2480a6270 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:44.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.098+0000 7ff266a00640 1 --2- 172.21.15.83:0/984658657 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff26010e740 0x7ff26019a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:44.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.098+0000 7ff266a00640 1 --2- 172.21.15.83:0/984658657 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff260074cf0 0x7ff2601a0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:44.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.098+0000 7ff266a00640 1 --2- 172.21.15.83:0/984658657 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2600743d0 0x7ff2601a0680 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:44.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.098+0000 7ff266a00640 1 -- 172.21.15.83:0/984658657 >> 172.21.15.83:0/984658657 conn(0x7ff2600fd9d0 msgr2=0x7ff2600fe740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:44.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.098+0000 7ff266a00640 1 -- 172.21.15.83:0/984658657 shutdown_connections 2024-04-24T08:51:44.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.098+0000 7ff266a00640 1 -- 172.21.15.83:0/984658657 wait complete. 2024-04-24T08:51:44.116 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T08:51:44.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:44 smithi083 ceph-mon[27907]: from='client.25081 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T08:51:44.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:44 smithi083 ceph-mon[27907]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:51:44.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:44 smithi083 ceph-mon[27907]: Updating smithi083:/var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T08:51:44.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:44 smithi083 ceph-mon[27907]: Updating smithi189:/var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T08:51:44.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:44 smithi083 ceph-mon[27907]: Updating smithi138:/var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T08:51:44.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:44 smithi083 ceph-mon[27907]: from='client.15729 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T08:51:44.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:44 smithi083 ceph-mon[27907]: Updating smithi083:/etc/ceph/ceph.client.admin.keyring 2024-04-24T08:51:44.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:44 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/2011602406' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T08:51:44.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.307+0000 7fafa7e00640 1 -- 172.21.15.83:0/1105684441 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fafa8103980 msgr2=0x7fafa810be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:44.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.307+0000 7fafa7e00640 1 --2- 172.21.15.83:0/1105684441 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fafa8103980 0x7fafa810be60 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7faf94009900 tx=0x7faf9402a480 comp rx=0 tx=0).stop 2024-04-24T08:51:44.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.308+0000 7fafa7e00640 1 -- 172.21.15.83:0/1105684441 shutdown_connections 2024-04-24T08:51:44.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.308+0000 7fafa7e00640 1 --2- 172.21.15.83:0/1105684441 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fafa810c480 0x7fafa810e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:44.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.308+0000 7fafa7e00640 1 --2- 172.21.15.83:0/1105684441 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fafa8103980 0x7fafa810be60 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:44.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.308+0000 7fafa7e00640 1 --2- 172.21.15.83:0/1105684441 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fafa8103060 0x7fafa8103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:44.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.308+0000 7fafa7e00640 1 -- 172.21.15.83:0/1105684441 >> 172.21.15.83:0/1105684441 conn(0x7fafa80fc880 msgr2=0x7fafa80feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:44.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.308+0000 7fafa7e00640 1 -- 172.21.15.83:0/1105684441 shutdown_connections 2024-04-24T08:51:44.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.308+0000 7fafa7e00640 1 -- 172.21.15.83:0/1105684441 wait complete. 2024-04-24T08:51:44.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.309+0000 7fafa7e00640 1 Processor -- start 2024-04-24T08:51:44.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.309+0000 7fafa7e00640 1 -- start start 2024-04-24T08:51:44.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.309+0000 7fafa7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fafa8103060 0x7fafa81a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:44.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.309+0000 7fafa7e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fafa8103980 0x7fafa81a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:44.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.309+0000 7fafa7e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fafa810c480 0x7fafa819a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:44.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.309+0000 7fafa7e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fafa8112160 con 0x7fafa8103060 2024-04-24T08:51:44.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.309+0000 7fafa7e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fafa8111fe0 con 0x7fafa8103980 2024-04-24T08:51:44.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.309+0000 7fafa7e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fafa81122e0 con 0x7fafa810c480 2024-04-24T08:51:44.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.309+0000 7fafa6a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fafa8103060 0x7fafa81a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:44.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.309+0000 7fafa6a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fafa8103060 0x7fafa81a0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55904/0 (socket says 172.21.15.83:55904) 2024-04-24T08:51:44.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.309+0000 7fafa6a00640 1 -- 172.21.15.83:0/45149222 learned_addr learned my addr 172.21.15.83:0/45149222 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:51:44.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.310+0000 7fafa6000640 1 --2- 172.21.15.83:0/45149222 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fafa8103980 0x7fafa81a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:44.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.310+0000 7fafa7400640 1 --2- 172.21.15.83:0/45149222 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fafa810c480 0x7fafa819a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:44.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.310+0000 7fafa6a00640 1 -- 172.21.15.83:0/45149222 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fafa810c480 msgr2=0x7fafa819a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:44.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.310+0000 7fafa6a00640 1 --2- 172.21.15.83:0/45149222 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fafa810c480 0x7fafa819a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:44.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.310+0000 7fafa6a00640 1 -- 172.21.15.83:0/45149222 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fafa8103980 msgr2=0x7fafa81a0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:44.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.310+0000 7fafa6a00640 1 --2- 172.21.15.83:0/45149222 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fafa8103980 0x7fafa81a0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:44.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.310+0000 7fafa6a00640 1 -- 172.21.15.83:0/45149222 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafa819aff0 con 0x7fafa8103060 2024-04-24T08:51:44.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.310+0000 7fafa6a00640 1 --2- 172.21.15.83:0/45149222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fafa8103060 0x7fafa81a0620 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7faf9800e9f0 tx=0x7faf9800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:44.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.310+0000 7faf9f400640 1 -- 172.21.15.83:0/45149222 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf9800cd80 con 0x7fafa8103060 2024-04-24T08:51:44.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.310+0000 7faf9f400640 1 -- 172.21.15.83:0/45149222 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faf980054d0 con 0x7fafa8103060 2024-04-24T08:51:44.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.310+0000 7fafa7e00640 1 -- 172.21.15.83:0/45149222 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafa819b2e0 con 0x7fafa8103060 2024-04-24T08:51:44.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.310+0000 7faf9f400640 1 -- 172.21.15.83:0/45149222 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf98010490 con 0x7fafa8103060 2024-04-24T08:51:44.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.310+0000 7fafa7e00640 1 -- 172.21.15.83:0/45149222 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafa81a75a0 con 0x7fafa8103060 2024-04-24T08:51:44.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.311+0000 7fafa7e00640 1 -- 172.21.15.83:0/45149222 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafa8069a70 con 0x7fafa8103060 2024-04-24T08:51:44.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.313+0000 7faf9f400640 1 -- 172.21.15.83:0/45149222 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7faf98005020 con 0x7fafa8103060 2024-04-24T08:51:44.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.315+0000 7faf9f400640 1 --2- 172.21.15.83:0/45149222 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7faf8c0a3db0 0x7faf8c0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:44.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.315+0000 7faf9f400640 1 -- 172.21.15.83:0/45149222 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7faf980d1290 con 0x7fafa8103060 2024-04-24T08:51:44.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.315+0000 7fafa6000640 1 --2- 172.21.15.83:0/45149222 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7faf8c0a3db0 0x7faf8c0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:44.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.316+0000 7fafa6000640 1 --2- 172.21.15.83:0/45149222 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7faf8c0a3db0 0x7faf8c0a6270 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7faf94000c00 tx=0x7faf94031400 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:44.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.319+0000 7faf9f400640 1 -- 172.21.15.83:0/45149222 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7faf9809d390 con 0x7fafa8103060 2024-04-24T08:51:44.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:44 smithi138 ceph-mon[31441]: from='client.25081 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T08:51:44.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:44 smithi138 ceph-mon[31441]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:51:44.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:44 smithi138 ceph-mon[31441]: Updating smithi083:/var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T08:51:44.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:44 smithi138 ceph-mon[31441]: Updating smithi189:/var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T08:51:44.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:44 smithi138 ceph-mon[31441]: Updating smithi138:/var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T08:51:44.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:44 smithi138 ceph-mon[31441]: from='client.15729 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T08:51:44.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:44 smithi138 ceph-mon[31441]: Updating smithi083:/etc/ceph/ceph.client.admin.keyring 2024-04-24T08:51:44.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:44 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/2011602406' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T08:51:44.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:44 smithi189 ceph-mon[31791]: from='client.25081 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T08:51:44.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:44 smithi189 ceph-mon[31791]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T08:51:44.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:44 smithi189 ceph-mon[31791]: Updating smithi083:/var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T08:51:44.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:44 smithi189 ceph-mon[31791]: Updating smithi189:/var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T08:51:44.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:44 smithi189 ceph-mon[31791]: Updating smithi138:/var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T08:51:44.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:44 smithi189 ceph-mon[31791]: from='client.15729 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T08:51:44.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:44 smithi189 ceph-mon[31791]: Updating smithi083:/etc/ceph/ceph.client.admin.keyring 2024-04-24T08:51:44.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:44 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/2011602406' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T08:51:44.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.511+0000 7fafa7e00640 1 -- 172.21.15.83:0/45149222 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fafa819bc10 con 0x7fafa8103060 2024-04-24T08:51:44.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.512+0000 7faf9f400640 1 -- 172.21.15.83:0/45149222 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7faf9809cc90 con 0x7fafa8103060 2024-04-24T08:51:44.511 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-24T08:51:44.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.514+0000 7fafa7e00640 1 -- 172.21.15.83:0/45149222 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7faf8c0a3db0 msgr2=0x7faf8c0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:44.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.515+0000 7fafa7e00640 1 --2- 172.21.15.83:0/45149222 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7faf8c0a3db0 0x7faf8c0a6270 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7faf94000c00 tx=0x7faf94031400 comp rx=0 tx=0).stop 2024-04-24T08:51:44.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.515+0000 7fafa7e00640 1 -- 172.21.15.83:0/45149222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fafa8103060 msgr2=0x7fafa81a0620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:44.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.515+0000 7fafa7e00640 1 --2- 172.21.15.83:0/45149222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fafa8103060 0x7fafa81a0620 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7faf9800e9f0 tx=0x7faf9800edf0 comp rx=0 tx=0).stop 2024-04-24T08:51:44.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.515+0000 7fafa7e00640 1 -- 172.21.15.83:0/45149222 shutdown_connections 2024-04-24T08:51:44.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.515+0000 7fafa7e00640 1 --2- 172.21.15.83:0/45149222 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7faf8c0a3db0 0x7faf8c0a6270 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:44.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.515+0000 7fafa7e00640 1 --2- 172.21.15.83:0/45149222 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fafa810c480 0x7fafa819a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:44.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.515+0000 7fafa7e00640 1 --2- 172.21.15.83:0/45149222 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fafa8103980 0x7fafa81a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:44.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.515+0000 7fafa7e00640 1 --2- 172.21.15.83:0/45149222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fafa8103060 0x7fafa81a0620 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:44.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.515+0000 7fafa7e00640 1 -- 172.21.15.83:0/45149222 >> 172.21.15.83:0/45149222 conn(0x7fafa80fc880 msgr2=0x7fafa8100c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:44.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.515+0000 7fafa7e00640 1 -- 172.21.15.83:0/45149222 shutdown_connections 2024-04-24T08:51:44.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.515+0000 7fafa7e00640 1 -- 172.21.15.83:0/45149222 wait complete. 2024-04-24T08:51:44.531 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T08:51:44.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.738+0000 7f020b200640 1 -- 172.21.15.83:0/659291382 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f02041039e0 msgr2=0x7f020410bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:44.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.738+0000 7f020b200640 1 --2- 172.21.15.83:0/659291382 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f02041039e0 0x7f020410bec0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f01f4009900 tx=0x7f01f402a480 comp rx=0 tx=0).stop 2024-04-24T08:51:44.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.738+0000 7f020b200640 1 -- 172.21.15.83:0/659291382 shutdown_connections 2024-04-24T08:51:44.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.738+0000 7f020b200640 1 --2- 172.21.15.83:0/659291382 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f020410c490 0x7f020410e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:44.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.738+0000 7f020b200640 1 --2- 172.21.15.83:0/659291382 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f02041039e0 0x7f020410bec0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:44.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.738+0000 7f020b200640 1 --2- 172.21.15.83:0/659291382 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f02041030c0 0x7f02041034a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:44.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.738+0000 7f020b200640 1 -- 172.21.15.83:0/659291382 >> 172.21.15.83:0/659291382 conn(0x7f02040fc880 msgr2=0x7f02040feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:44.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.738+0000 7f020b200640 1 -- 172.21.15.83:0/659291382 shutdown_connections 2024-04-24T08:51:44.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.738+0000 7f020b200640 1 -- 172.21.15.83:0/659291382 wait complete. 2024-04-24T08:51:44.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.739+0000 7f020b200640 1 Processor -- start 2024-04-24T08:51:44.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.740+0000 7f020b200640 1 -- start start 2024-04-24T08:51:44.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.740+0000 7f020b200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02041030c0 0x7f02041a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:44.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.740+0000 7f020b200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f02041039e0 0x7f02041a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:44.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.740+0000 7f020b200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f020410c490 0x7f020419a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:44.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.740+0000 7f020b200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0204112140 con 0x7f020410c490 2024-04-24T08:51:44.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.740+0000 7f020b200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0204111fc0 con 0x7f02041030c0 2024-04-24T08:51:44.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.740+0000 7f020b200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f02041122c0 con 0x7f02041039e0 2024-04-24T08:51:44.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.740+0000 7f020a800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f020410c490 0x7f020419a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:44.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.740+0000 7f020a800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f020410c490 0x7f020419a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55920/0 (socket says 172.21.15.83:55920) 2024-04-24T08:51:44.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.740+0000 7f020a800640 1 -- 172.21.15.83:0/810356271 learned_addr learned my addr 172.21.15.83:0/810356271 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:51:44.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.741+0000 7f0209e00640 1 --2- 172.21.15.83:0/810356271 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02041030c0 0x7f02041a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:44.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.741+0000 7f0209400640 1 --2- 172.21.15.83:0/810356271 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f02041039e0 0x7f02041a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:44.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.741+0000 7f020a800640 1 -- 172.21.15.83:0/810356271 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f02041039e0 msgr2=0x7f02041a0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:44.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.741+0000 7f020a800640 1 --2- 172.21.15.83:0/810356271 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f02041039e0 0x7f02041a0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:44.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.741+0000 7f020a800640 1 -- 172.21.15.83:0/810356271 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02041030c0 msgr2=0x7f02041a0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:44.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.741+0000 7f020a800640 1 --2- 172.21.15.83:0/810356271 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02041030c0 0x7f02041a0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:44.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.741+0000 7f020a800640 1 -- 172.21.15.83:0/810356271 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f020419afe0 con 0x7f020410c490 2024-04-24T08:51:44.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.741+0000 7f020a800640 1 --2- 172.21.15.83:0/810356271 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f020410c490 0x7f020419a720 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f020000eaa0 tx=0x7f020000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:44.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.741+0000 7f01fea00640 1 -- 172.21.15.83:0/810356271 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f020000cdb0 con 0x7f020410c490 2024-04-24T08:51:44.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.741+0000 7f01fea00640 1 -- 172.21.15.83:0/810356271 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0200005020 con 0x7f020410c490 2024-04-24T08:51:44.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.741+0000 7f020b200640 1 -- 172.21.15.83:0/810356271 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f020419b2d0 con 0x7f020410c490 2024-04-24T08:51:44.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.741+0000 7f01fea00640 1 -- 172.21.15.83:0/810356271 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02000116d0 con 0x7f020410c490 2024-04-24T08:51:44.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.742+0000 7f020b200640 1 -- 172.21.15.83:0/810356271 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02041a7590 con 0x7f020410c490 2024-04-24T08:51:44.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.743+0000 7f020b200640 1 -- 172.21.15.83:0/810356271 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f020419b610 con 0x7f020410c490 2024-04-24T08:51:44.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.746+0000 7f01fea00640 1 -- 172.21.15.83:0/810356271 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f02000051d0 con 0x7f020410c490 2024-04-24T08:51:44.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.747+0000 7f01fea00640 1 --2- 172.21.15.83:0/810356271 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f01ec0a3e30 0x7f01ec0a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:44.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.748+0000 7f01fea00640 1 -- 172.21.15.83:0/810356271 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f02000d12f0 con 0x7f020410c490 2024-04-24T08:51:44.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.748+0000 7f0209e00640 1 --2- 172.21.15.83:0/810356271 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f01ec0a3e30 0x7f01ec0a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:44.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.748+0000 7f0209e00640 1 --2- 172.21.15.83:0/810356271 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f01ec0a3e30 0x7f01ec0a62f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f01f8005fd0 tx=0x7f01f8005850 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:44.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.750+0000 7f01fea00640 1 -- 172.21.15.83:0/810356271 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0200016030 con 0x7f020410c490 2024-04-24T08:51:44.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.955+0000 7f020b200640 1 -- 172.21.15.83:0/810356271 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f020419bc00 con 0x7f020410c490 2024-04-24T08:51:44.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.955+0000 7f01fea00640 1 -- 172.21.15.83:0/810356271 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7f020009d470 con 0x7f020410c490 2024-04-24T08:51:44.955 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2024-04-24T08:51:44.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.958+0000 7f020b200640 1 -- 172.21.15.83:0/810356271 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f01ec0a3e30 msgr2=0x7f01ec0a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:44.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.958+0000 7f020b200640 1 --2- 172.21.15.83:0/810356271 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f01ec0a3e30 0x7f01ec0a62f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f01f8005fd0 tx=0x7f01f8005850 comp rx=0 tx=0).stop 2024-04-24T08:51:44.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.958+0000 7f020b200640 1 -- 172.21.15.83:0/810356271 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f020410c490 msgr2=0x7f020419a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:44.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.958+0000 7f020b200640 1 --2- 172.21.15.83:0/810356271 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f020410c490 0x7f020419a720 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f020000eaa0 tx=0x7f020000ef60 comp rx=0 tx=0).stop 2024-04-24T08:51:44.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.958+0000 7f020b200640 1 -- 172.21.15.83:0/810356271 shutdown_connections 2024-04-24T08:51:44.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.958+0000 7f020b200640 1 --2- 172.21.15.83:0/810356271 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f01ec0a3e30 0x7f01ec0a62f0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:44.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.958+0000 7f020b200640 1 --2- 172.21.15.83:0/810356271 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f020410c490 0x7f020419a720 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:44.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.958+0000 7f020b200640 1 --2- 172.21.15.83:0/810356271 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f02041039e0 0x7f02041a0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:44.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.958+0000 7f020b200640 1 --2- 172.21.15.83:0/810356271 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02041030c0 0x7f02041a0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:44.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.958+0000 7f020b200640 1 -- 172.21.15.83:0/810356271 >> 172.21.15.83:0/810356271 conn(0x7f02040fc880 msgr2=0x7f0204101f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:44.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.958+0000 7f020b200640 1 -- 172.21.15.83:0/810356271 shutdown_connections 2024-04-24T08:51:44.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:44.958+0000 7f020b200640 1 -- 172.21.15.83:0/810356271 wait complete. 2024-04-24T08:51:44.975 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-24T08:51:45.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:45.183+0000 7f843cc00640 1 -- 172.21.15.83:0/968057297 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8438100e50 msgr2=0x7f8438101230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:45.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:45.183+0000 7f843cc00640 1 --2- 172.21.15.83:0/968057297 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8438100e50 0x7f8438101230 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f8420009860 tx=0x7f842002a3b0 comp rx=0 tx=0).stop 2024-04-24T08:51:45.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:45.183+0000 7f843cc00640 1 -- 172.21.15.83:0/968057297 shutdown_connections 2024-04-24T08:51:45.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:45.183+0000 7f843cc00640 1 --2- 172.21.15.83:0/968057297 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8438106250 0x7f84381066e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:45.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:45.183+0000 7f843cc00640 1 --2- 172.21.15.83:0/968057297 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8438101770 0x7f8438105c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:45.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:45.183+0000 7f843cc00640 1 --2- 172.21.15.83:0/968057297 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8438100e50 0x7f8438101230 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:45.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:45.183+0000 7f843cc00640 1 -- 172.21.15.83:0/968057297 >> 172.21.15.83:0/968057297 conn(0x7f84380fc880 msgr2=0x7f84380feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:45.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:45.183+0000 7f843cc00640 1 -- 172.21.15.83:0/968057297 shutdown_connections 2024-04-24T08:51:45.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:45.183+0000 7f843cc00640 1 -- 172.21.15.83:0/968057297 wait complete. 2024-04-24T08:51:45.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:45.184+0000 7f843cc00640 1 Processor -- start 2024-04-24T08:51:45.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:45.184+0000 7f843cc00640 1 -- start start 2024-04-24T08:51:45.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:45.185+0000 7f843cc00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8438100e50 0x7f84381a4950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:45.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:45.185+0000 7f843cc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8438101770 0x7f84381a4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:45.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:45.185+0000 7f843cc00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8438106250 0x7f843819eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:45.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:45.185+0000 7f843cc00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8438118700 con 0x7f8438101770 2024-04-24T08:51:45.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:45.185+0000 7f843cc00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8438118580 con 0x7f8438100e50 2024-04-24T08:51:45.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:45.185+0000 7f843cc00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8438118880 con 0x7f8438106250 2024-04-24T08:51:45.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:45.185+0000 7f8436a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8438101770 0x7f84381a4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:45.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:45.185+0000 7f8436a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8438101770 0x7f84381a4e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55938/0 (socket says 172.21.15.83:55938) 2024-04-24T08:51:45.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:45.185+0000 7f8436a00640 1 -- 172.21.15.83:0/3154348841 learned_addr learned my addr 172.21.15.83:0/3154348841 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:51:45.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:45.185+0000 7f8437e00640 1 --2- 172.21.15.83:0/3154348841 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8438106250 0x7f843819eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:45.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:45.185+0000 7f8437400640 1 --2- 172.21.15.83:0/3154348841 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8438100e50 0x7f84381a4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:45.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:45.186+0000 7f8436a00640 1 -- 172.21.15.83:0/3154348841 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8438106250 msgr2=0x7f843819eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:45.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:45.186+0000 7f8436a00640 1 --2- 172.21.15.83:0/3154348841 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8438106250 0x7f843819eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:45.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:45.186+0000 7f8436a00640 1 -- 172.21.15.83:0/3154348841 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8438100e50 msgr2=0x7f84381a4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:45.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:45.186+0000 7f8436a00640 1 --2- 172.21.15.83:0/3154348841 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8438100e50 0x7f84381a4950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:45.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:45.186+0000 7f8436a00640 1 -- 172.21.15.83:0/3154348841 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f843819f280 con 0x7f8438101770 2024-04-24T08:51:45.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:45.186+0000 7f8436a00640 1 --2- 172.21.15.83:0/3154348841 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8438101770 0x7f84381a4e90 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f842800eb10 tx=0x7f842800ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:45.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:45.186+0000 7f842fe00640 1 -- 172.21.15.83:0/3154348841 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f842800ce20 con 0x7f8438101770 2024-04-24T08:51:45.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:45.186+0000 7f842fe00640 1 -- 172.21.15.83:0/3154348841 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f84280108e0 con 0x7f8438101770 2024-04-24T08:51:45.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:45.186+0000 7f842fe00640 1 -- 172.21.15.83:0/3154348841 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84280189b0 con 0x7f8438101770 2024-04-24T08:51:45.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:45.186+0000 7f843cc00640 1 -- 172.21.15.83:0/3154348841 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f843819f570 con 0x7f8438101770 2024-04-24T08:51:45.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:45.186+0000 7f843cc00640 1 -- 172.21.15.83:0/3154348841 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84381ab810 con 0x7f8438101770 2024-04-24T08:51:45.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:45.188+0000 7f843cc00640 1 -- 172.21.15.83:0/3154348841 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8438068d40 con 0x7f8438101770 2024-04-24T08:51:45.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:45.190+0000 7f842fe00640 1 -- 172.21.15.83:0/3154348841 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f8428010a90 con 0x7f8438101770 2024-04-24T08:51:45.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:45.191+0000 7f842fe00640 1 --2- 172.21.15.83:0/3154348841 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f841c0a3e30 0x7f841c0a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:45.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:45.191+0000 7f842fe00640 1 -- 172.21.15.83:0/3154348841 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f84280d12b0 con 0x7f8438101770 2024-04-24T08:51:45.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:45.191+0000 7f8437400640 1 --2- 172.21.15.83:0/3154348841 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f841c0a3e30 0x7f841c0a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:45.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:45.192+0000 7f8437400640 1 --2- 172.21.15.83:0/3154348841 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f841c0a3e30 0x7f841c0a62f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8420000c00 tx=0x7f84200058b0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:45.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:45.197+0000 7f842fe00640 1 -- 172.21.15.83:0/3154348841 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f842809d3b0 con 0x7f8438101770 2024-04-24T08:51:45.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:45 smithi083 ceph-mon[27907]: from='client.15735 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T08:51:45.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:45 smithi083 ceph-mon[27907]: Updating smithi189:/etc/ceph/ceph.client.admin.keyring 2024-04-24T08:51:45.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:45 smithi083 ceph-mon[27907]: Updating smithi138:/etc/ceph/ceph.client.admin.keyring 2024-04-24T08:51:45.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:45 smithi083 ceph-mon[27907]: Updating smithi083:/var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T08:51:45.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:45 smithi083 ceph-mon[27907]: Updating smithi189:/var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T08:51:45.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:45 smithi083 ceph-mon[27907]: Updating smithi138:/var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T08:51:45.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:45 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/984658657' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T08:51:45.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:45 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T08:51:45.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:45 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T08:51:45.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:45 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/45149222' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T08:51:45.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:45 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T08:51:45.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:45 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T08:51:45.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:45 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T08:51:45.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:45 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T08:51:45.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:45 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T08:51:45.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:45 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/810356271' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T08:51:45.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:45.401+0000 7f843cc00640 1 -- 172.21.15.83:0/3154348841 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f84381aba10 con 0x7f8438101770 2024-04-24T08:51:45.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:45.402+0000 7f842fe00640 1 -- 172.21.15.83:0/3154348841 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) ==== 103+0+11 (secure 0 0 0) 0x7f842809ccb0 con 0x7f8438101770 2024-04-24T08:51:45.402 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-24T08:51:45.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:45.404+0000 7f843cc00640 1 -- 172.21.15.83:0/3154348841 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f841c0a3e30 msgr2=0x7f841c0a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:45.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:45.404+0000 7f843cc00640 1 --2- 172.21.15.83:0/3154348841 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f841c0a3e30 0x7f841c0a62f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8420000c00 tx=0x7f84200058b0 comp rx=0 tx=0).stop 2024-04-24T08:51:45.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:45.404+0000 7f843cc00640 1 -- 172.21.15.83:0/3154348841 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8438101770 msgr2=0x7f84381a4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:45.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:45.404+0000 7f843cc00640 1 --2- 172.21.15.83:0/3154348841 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8438101770 0x7f84381a4e90 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f842800eb10 tx=0x7f842800ef10 comp rx=0 tx=0).stop 2024-04-24T08:51:45.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:45.405+0000 7f843cc00640 1 -- 172.21.15.83:0/3154348841 shutdown_connections 2024-04-24T08:51:45.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:45.405+0000 7f843cc00640 1 --2- 172.21.15.83:0/3154348841 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f841c0a3e30 0x7f841c0a62f0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:45.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:45.405+0000 7f843cc00640 1 --2- 172.21.15.83:0/3154348841 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8438106250 0x7f843819eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:45.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:45.405+0000 7f843cc00640 1 --2- 172.21.15.83:0/3154348841 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8438101770 0x7f84381a4e90 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:45.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:45.405+0000 7f843cc00640 1 --2- 172.21.15.83:0/3154348841 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8438100e50 0x7f84381a4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:45.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:45.405+0000 7f843cc00640 1 -- 172.21.15.83:0/3154348841 >> 172.21.15.83:0/3154348841 conn(0x7f84380fc880 msgr2=0x7f84380fd900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:45.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:45.405+0000 7f843cc00640 1 -- 172.21.15.83:0/3154348841 shutdown_connections 2024-04-24T08:51:45.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:45.405+0000 7f843cc00640 1 -- 172.21.15.83:0/3154348841 wait complete. 2024-04-24T08:51:45.422 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T08:51:45.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:45 smithi138 ceph-mon[31441]: from='client.15735 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T08:51:45.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:45 smithi138 ceph-mon[31441]: Updating smithi189:/etc/ceph/ceph.client.admin.keyring 2024-04-24T08:51:45.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:45 smithi138 ceph-mon[31441]: Updating smithi138:/etc/ceph/ceph.client.admin.keyring 2024-04-24T08:51:45.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:45 smithi138 ceph-mon[31441]: Updating smithi083:/var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T08:51:45.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:45 smithi138 ceph-mon[31441]: Updating smithi189:/var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T08:51:45.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:45 smithi138 ceph-mon[31441]: Updating smithi138:/var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T08:51:45.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:45 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/984658657' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T08:51:45.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:45 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T08:51:45.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:45 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T08:51:45.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:45 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/45149222' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T08:51:45.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:45 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T08:51:45.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:45 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T08:51:45.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:45 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T08:51:45.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:45 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T08:51:45.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:45 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T08:51:45.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:45 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/810356271' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T08:51:45.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:45 smithi189 ceph-mon[31791]: from='client.15735 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T08:51:45.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:45 smithi189 ceph-mon[31791]: Updating smithi189:/etc/ceph/ceph.client.admin.keyring 2024-04-24T08:51:45.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:45 smithi189 ceph-mon[31791]: Updating smithi138:/etc/ceph/ceph.client.admin.keyring 2024-04-24T08:51:45.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:45 smithi189 ceph-mon[31791]: Updating smithi083:/var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T08:51:45.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:45 smithi189 ceph-mon[31791]: Updating smithi189:/var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T08:51:45.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:45 smithi189 ceph-mon[31791]: Updating smithi138:/var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T08:51:45.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:45 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/984658657' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T08:51:45.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:45 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T08:51:45.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:45 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T08:51:45.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:45 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/45149222' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T08:51:45.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:45 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T08:51:45.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:45 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T08:51:45.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:45 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T08:51:45.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:45 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T08:51:45.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:45 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T08:51:45.459 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:45 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/810356271' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T08:51:45.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:45.615+0000 7ff4e6200640 1 -- 172.21.15.83:0/1288385194 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4e01084a0 msgr2=0x7ff4e0108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:45.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:45.615+0000 7ff4e6200640 1 --2- 172.21.15.83:0/1288385194 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4e01084a0 0x7ff4e0108900 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7ff4c8009900 tx=0x7ff4c802a490 comp rx=0 tx=0).stop 2024-04-24T08:51:45.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:45.615+0000 7ff4e6200640 1 -- 172.21.15.83:0/1288385194 shutdown_connections 2024-04-24T08:51:45.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:45.616+0000 7ff4e6200640 1 --2- 172.21.15.83:0/1288385194 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff4e0108e40 0x7ff4e01152b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:45.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:45.616+0000 7ff4e6200640 1 --2- 172.21.15.83:0/1288385194 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4e01084a0 0x7ff4e0108900 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:45.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:45.616+0000 7ff4e6200640 1 --2- 172.21.15.83:0/1288385194 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff4e010d3c0 0x7ff4e010d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:45.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:45.616+0000 7ff4e6200640 1 -- 172.21.15.83:0/1288385194 >> 172.21.15.83:0/1288385194 conn(0x7ff4e0077b80 msgr2=0x7ff4e0079fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:45.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:45.616+0000 7ff4e6200640 1 -- 172.21.15.83:0/1288385194 shutdown_connections 2024-04-24T08:51:45.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:45.616+0000 7ff4e6200640 1 -- 172.21.15.83:0/1288385194 wait complete. 2024-04-24T08:51:45.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:45.617+0000 7ff4e6200640 1 Processor -- start 2024-04-24T08:51:45.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:45.617+0000 7ff4e6200640 1 -- start start 2024-04-24T08:51:45.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:45.617+0000 7ff4e6200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff4e01084a0 0x7ff4e019ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:45.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:45.617+0000 7ff4e6200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4e0108e40 0x7ff4e019f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:45.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:45.617+0000 7ff4e6200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff4e010d3c0 0x7ff4e01a3510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:45.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:45.617+0000 7ff4e6200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e01186e0 con 0x7ff4e0108e40 2024-04-24T08:51:45.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:45.617+0000 7ff4e6200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e0118560 con 0x7ff4e01084a0 2024-04-24T08:51:45.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:45.617+0000 7ff4e6200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e0118860 con 0x7ff4e010d3c0 2024-04-24T08:51:45.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:45.617+0000 7ff4d7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4e0108e40 0x7ff4e019f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:45.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:45.617+0000 7ff4d7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4e0108e40 0x7ff4e019f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55954/0 (socket says 172.21.15.83:55954) 2024-04-24T08:51:45.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:45.617+0000 7ff4d7e00640 1 -- 172.21.15.83:0/2300771843 learned_addr learned my addr 172.21.15.83:0/2300771843 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:51:45.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:45.618+0000 7ff4e5800640 1 --2- 172.21.15.83:0/2300771843 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff4e010d3c0 0x7ff4e01a3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:45.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:45.618+0000 7ff4e4e00640 1 --2- 172.21.15.83:0/2300771843 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff4e01084a0 0x7ff4e019ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:45.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:45.618+0000 7ff4d7e00640 1 -- 172.21.15.83:0/2300771843 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff4e010d3c0 msgr2=0x7ff4e01a3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:45.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:45.618+0000 7ff4d7e00640 1 --2- 172.21.15.83:0/2300771843 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff4e010d3c0 0x7ff4e01a3510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:45.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:45.618+0000 7ff4d7e00640 1 -- 172.21.15.83:0/2300771843 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff4e01084a0 msgr2=0x7ff4e019ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:45.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:45.618+0000 7ff4d7e00640 1 --2- 172.21.15.83:0/2300771843 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff4e01084a0 0x7ff4e019ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:45.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:45.618+0000 7ff4d7e00640 1 -- 172.21.15.83:0/2300771843 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4e01a3c40 con 0x7ff4e0108e40 2024-04-24T08:51:45.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:45.618+0000 7ff4e4e00640 1 --2- 172.21.15.83:0/2300771843 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff4e01084a0 0x7ff4e019ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:51:45.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:45.618+0000 7ff4d7e00640 1 --2- 172.21.15.83:0/2300771843 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4e0108e40 0x7ff4e019f160 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7ff4c8005ab0 tx=0x7ff4c8003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:45.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:45.619+0000 7ff4de000640 1 -- 172.21.15.83:0/2300771843 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4c803f070 con 0x7ff4e0108e40 2024-04-24T08:51:45.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:45.619+0000 7ff4e6200640 1 -- 172.21.15.83:0/2300771843 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4e01a3ed0 con 0x7ff4e0108e40 2024-04-24T08:51:45.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:45.619+0000 7ff4de000640 1 -- 172.21.15.83:0/2300771843 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff4c8031d60 con 0x7ff4e0108e40 2024-04-24T08:51:45.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:45.619+0000 7ff4e6200640 1 -- 172.21.15.83:0/2300771843 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4e01ab7c0 con 0x7ff4e0108e40 2024-04-24T08:51:45.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:45.619+0000 7ff4de000640 1 -- 172.21.15.83:0/2300771843 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4c8049060 con 0x7ff4e0108e40 2024-04-24T08:51:45.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:45.621+0000 7ff4e6200640 1 -- 172.21.15.83:0/2300771843 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4e0069190 con 0x7ff4e0108e40 2024-04-24T08:51:45.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:45.622+0000 7ff4de000640 1 -- 172.21.15.83:0/2300771843 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7ff4c8042070 con 0x7ff4e0108e40 2024-04-24T08:51:45.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:45.623+0000 7ff4de000640 1 --2- 172.21.15.83:0/2300771843 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff4c40a3db0 0x7ff4c40a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:45.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:45.624+0000 7ff4de000640 1 -- 172.21.15.83:0/2300771843 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7ff4c80f13c0 con 0x7ff4e0108e40 2024-04-24T08:51:45.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:45.624+0000 7ff4e4e00640 1 --2- 172.21.15.83:0/2300771843 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff4c40a3db0 0x7ff4c40a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:45.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:45.624+0000 7ff4e4e00640 1 --2- 172.21.15.83:0/2300771843 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff4c40a3db0 0x7ff4c40a6270 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff4d0005fd0 tx=0x7ff4d00094d0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:45.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:45.628+0000 7ff4de000640 1 -- 172.21.15.83:0/2300771843 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff4c80bd4c0 con 0x7ff4e0108e40 2024-04-24T08:51:45.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:45.872+0000 7ff4e6200640 1 -- 172.21.15.83:0/2300771843 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff4e0110900 con 0x7ff4e0108e40 2024-04-24T08:51:45.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:45.874+0000 7ff4de000640 1 -- 172.21.15.83:0/2300771843 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11944 (secure 0 0 0) 0x7ff4c80bcdc0 con 0x7ff4e0108e40 2024-04-24T08:51:45.874 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:51:45.874 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/358872257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":358872257},{"type":"v1","addr":"172.21.15.83:6835","nonce":358872257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3157070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3157070851},{"type":"v1","addr":"172.21.15.83:6837","nonce":3157070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T08:48:29.422576+0000","modified":"2024-04-24T08:51:36.355488+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34520,"mds_1":34493,"mds_2":34451,"mds_3":34427,"mds_4":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14658":{"gid":14658,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6839/4085470844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":4085470844},{"type":"v1","addr":"172.21.15.83:6839","nonce":4085470844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14676":{"gid":14676,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6841/2634754210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":2634754210},{"type":"v1","addr":"172.21.15.83:6841","nonce":2634754210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24382":{"gid":24382,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/1859921310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1859921310},{"type":"v1","addr":"172.21.15.189:6833","nonce":1859921310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/2241923197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2241923197},{"type":"v1","addr":"172.21.15.189:6835","nonce":2241923197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6837/3172311220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":3172311220},{"type":"v1","addr":"172.21.15.189:6837","nonce":3172311220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.189:6839/2548965336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":2548965336},{"type":"v1","addr":"172.21.15.189:6839","nonce":2548965336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.138:6833/1157967658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1157967658},{"type":"v1","addr":"172.21.15.138:6833","nonce":1157967658}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.138:6835/1909219758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1909219758},{"type":"v1","addr":"172.21.15.138:6835","nonce":1909219758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34493":{"gid":34493,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.138:6837/900566754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":900566754},{"type":"v1","addr":"172.21.15.138:6837","nonce":900566754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34520":{"gid":34520,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.138:6839/2236932393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2236932393},{"type":"v1","addr":"172.21.15.138:6839","nonce":2236932393}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34520,"qdb_cluster":[34451,34493,34427,34520,24454]},"id":1}]} 2024-04-24T08:51:45.874 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 28 2024-04-24T08:51:45.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:45.877+0000 7ff4e6200640 1 -- 172.21.15.83:0/2300771843 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff4c40a3db0 msgr2=0x7ff4c40a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:45.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:45.877+0000 7ff4e6200640 1 --2- 172.21.15.83:0/2300771843 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff4c40a3db0 0x7ff4c40a6270 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff4d0005fd0 tx=0x7ff4d00094d0 comp rx=0 tx=0).stop 2024-04-24T08:51:45.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:45.877+0000 7ff4e6200640 1 -- 172.21.15.83:0/2300771843 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4e0108e40 msgr2=0x7ff4e019f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:45.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:45.877+0000 7ff4e6200640 1 --2- 172.21.15.83:0/2300771843 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4e0108e40 0x7ff4e019f160 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7ff4c8005ab0 tx=0x7ff4c8003cd0 comp rx=0 tx=0).stop 2024-04-24T08:51:45.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:45.877+0000 7ff4e6200640 1 -- 172.21.15.83:0/2300771843 shutdown_connections 2024-04-24T08:51:45.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:45.877+0000 7ff4e6200640 1 --2- 172.21.15.83:0/2300771843 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff4c40a3db0 0x7ff4c40a6270 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:45.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:45.877+0000 7ff4e6200640 1 --2- 172.21.15.83:0/2300771843 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff4e010d3c0 0x7ff4e01a3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:45.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:45.877+0000 7ff4e6200640 1 --2- 172.21.15.83:0/2300771843 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4e0108e40 0x7ff4e019f160 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:45.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:45.877+0000 7ff4e6200640 1 --2- 172.21.15.83:0/2300771843 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff4e01084a0 0x7ff4e019ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:45.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:45.877+0000 7ff4e6200640 1 -- 172.21.15.83:0/2300771843 >> 172.21.15.83:0/2300771843 conn(0x7ff4e0077b80 msgr2=0x7ff4e00781a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:45.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:45.877+0000 7ff4e6200640 1 -- 172.21.15.83:0/2300771843 shutdown_connections 2024-04-24T08:51:45.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:45.877+0000 7ff4e6200640 1 -- 172.21.15.83:0/2300771843 wait complete. 2024-04-24T08:51:45.895 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T08:51:46.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.100+0000 7f8cf4e00640 1 -- 172.21.15.83:0/1813232792 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8cf0074430 msgr2=0x7f8cf0074810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:46.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.100+0000 7f8cf4e00640 1 --2- 172.21.15.83:0/1813232792 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8cf0074430 0x7f8cf0074810 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f8ce0009900 tx=0x7f8ce002a480 comp rx=0 tx=0).stop 2024-04-24T08:51:46.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.100+0000 7f8cf4e00640 1 -- 172.21.15.83:0/1813232792 shutdown_connections 2024-04-24T08:51:46.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.100+0000 7f8cf4e00640 1 --2- 172.21.15.83:0/1813232792 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8cf010e800 0x7f8cf0110c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:46.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.100+0000 7f8cf4e00640 1 --2- 172.21.15.83:0/1813232792 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8cf0074d50 0x7f8cf010e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:46.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.100+0000 7f8cf4e00640 1 --2- 172.21.15.83:0/1813232792 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8cf0074430 0x7f8cf0074810 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:46.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.100+0000 7f8cf4e00640 1 -- 172.21.15.83:0/1813232792 >> 172.21.15.83:0/1813232792 conn(0x7f8cf0079400 msgr2=0x7f8cf0079810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:46.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.100+0000 7f8cf4e00640 1 -- 172.21.15.83:0/1813232792 shutdown_connections 2024-04-24T08:51:46.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.100+0000 7f8cf4e00640 1 -- 172.21.15.83:0/1813232792 wait complete. 2024-04-24T08:51:46.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.101+0000 7f8cf4e00640 1 Processor -- start 2024-04-24T08:51:46.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.101+0000 7f8cf4e00640 1 -- start start 2024-04-24T08:51:46.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.101+0000 7f8cf4e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8cf0074430 0x7f8cf01a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:46.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.101+0000 7f8cf4e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8cf0074d50 0x7f8cf01a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:46.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.101+0000 7f8cf4e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8cf010e800 0x7f8cf019a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:46.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.101+0000 7f8cf4e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf0114490 con 0x7f8cf0074430 2024-04-24T08:51:46.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.101+0000 7f8cf4e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf0114310 con 0x7f8cf0074d50 2024-04-24T08:51:46.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.101+0000 7f8cf4e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf0114610 con 0x7f8cf010e800 2024-04-24T08:51:46.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.102+0000 7f8cede00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8cf0074430 0x7f8cf01a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:46.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.102+0000 7f8cede00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8cf0074430 0x7f8cf01a0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55976/0 (socket says 172.21.15.83:55976) 2024-04-24T08:51:46.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.102+0000 7f8cede00640 1 -- 172.21.15.83:0/982917112 learned_addr learned my addr 172.21.15.83:0/982917112 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:51:46.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.102+0000 7f8cede00640 1 -- 172.21.15.83:0/982917112 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8cf010e800 msgr2=0x7f8cf019a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:46.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.102+0000 7f8cede00640 1 --2- 172.21.15.83:0/982917112 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8cf010e800 0x7f8cf019a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:46.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.102+0000 7f8cede00640 1 -- 172.21.15.83:0/982917112 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8cf0074d50 msgr2=0x7f8cf01a0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:46.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.102+0000 7f8cede00640 1 --2- 172.21.15.83:0/982917112 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8cf0074d50 0x7f8cf01a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:46.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.102+0000 7f8cede00640 1 -- 172.21.15.83:0/982917112 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cf019afe0 con 0x7f8cf0074430 2024-04-24T08:51:46.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.102+0000 7f8cede00640 1 --2- 172.21.15.83:0/982917112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8cf0074430 0x7f8cf01a0650 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f8ce0005ab0 tx=0x7f8ce0003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:46.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.102+0000 7f8ce6a00640 1 -- 172.21.15.83:0/982917112 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ce003f070 con 0x7f8cf0074430 2024-04-24T08:51:46.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.103+0000 7f8ce6a00640 1 -- 172.21.15.83:0/982917112 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8ce0031d60 con 0x7f8cf0074430 2024-04-24T08:51:46.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.103+0000 7f8cf4e00640 1 -- 172.21.15.83:0/982917112 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cf019b270 con 0x7f8cf0074430 2024-04-24T08:51:46.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.103+0000 7f8ce6a00640 1 -- 172.21.15.83:0/982917112 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ce0035030 con 0x7f8cf0074430 2024-04-24T08:51:46.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.103+0000 7f8cf4e00640 1 -- 172.21.15.83:0/982917112 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cf01a74a0 con 0x7f8cf0074430 2024-04-24T08:51:46.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.104+0000 7f8cf4e00640 1 -- 172.21.15.83:0/982917112 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8cf0075ae0 con 0x7f8cf0074430 2024-04-24T08:51:46.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.106+0000 7f8ce6a00640 1 -- 172.21.15.83:0/982917112 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f8ce0042070 con 0x7f8cf0074430 2024-04-24T08:51:46.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.107+0000 7f8ce6a00640 1 --2- 172.21.15.83:0/982917112 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f8cd40a3db0 0x7f8cd40a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:46.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.107+0000 7f8ce6a00640 1 -- 172.21.15.83:0/982917112 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f8ce00f1770 con 0x7f8cf0074430 2024-04-24T08:51:46.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.108+0000 7f8ce7e00640 1 --2- 172.21.15.83:0/982917112 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f8cd40a3db0 0x7f8cd40a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:46.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.108+0000 7f8ce7e00640 1 --2- 172.21.15.83:0/982917112 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f8cd40a3db0 0x7f8cd40a6270 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8cd8005fd0 tx=0x7f8cd8005850 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:46.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.112+0000 7f8ce6a00640 1 -- 172.21.15.83:0/982917112 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8ce00bd870 con 0x7f8cf0074430 2024-04-24T08:51:46.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.317+0000 7f8cf4e00640 1 -- 172.21.15.83:0/982917112 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8cf0109980 con 0x7f8cf0074430 2024-04-24T08:51:46.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.321+0000 7f8ce6a00640 1 -- 172.21.15.83:0/982917112 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+25265 (secure 0 0 0) 0x7f8ce00bd170 con 0x7f8cf0074430 2024-04-24T08:51:46.321 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:51:46.322 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":94,"fsid":"bd3e039e-0215-11ef-bc93-c7b262605968","created":"2024-04-24T08:37:32.209944+0000","modified":"2024-04-24T08:51:36.347834+0000","last_up_change":"2024-04-24T08:47:17.966041+0000","last_in_change":"2024-04-24T08:46:53.835936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T08:41:56.834372+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T08:48:26.978311+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T08:48:28.352731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T08:48:32.275167+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2cd77f3a-19dd-4c6e-a199-1dcbd5faa679","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2132675555},{"type":"v1","addr":"172.21.15.83:6803","nonce":2132675555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2132675555},{"type":"v1","addr":"172.21.15.83:6805","nonce":2132675555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2132675555},{"type":"v1","addr":"172.21.15.83:6809","nonce":2132675555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2132675555},{"type":"v1","addr":"172.21.15.83:6807","nonce":2132675555}]},"public_addr":"172.21.15.83:6803/2132675555","cluster_addr":"172.21.15.83:6805/2132675555","heartbeat_back_addr":"172.21.15.83:6809/2132675555","heartbeat_front_addr":"172.21.15.83:6807/2132675555","state":["exists","up"]},{"osd":1,"uuid":"aba7f5df-2a4c-4c23-b502-d193605a5b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":3589158880},{"type":"v1","addr":"172.21.15.138:6801","nonce":3589158880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":3589158880},{"type":"v1","addr":"172.21.15.138:6803","nonce":3589158880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":3589158880},{"type":"v1","addr":"172.21.15.138:6807","nonce":3589158880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":3589158880},{"type":"v1","addr":"172.21.15.138:6805","nonce":3589158880}]},"public_addr":"172.21.15.138:6801/3589158880","cluster_addr":"172.21.15.138:6803/3589158880","heartbeat_back_addr":"172.21.15.138:6807/3589158880","heartbeat_front_addr":"172.21.15.138:6805/3589158880","state":["exists","up"]},{"osd":2,"uuid":"071bc167-2333-4420-bb58-bffbc10d7641","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":3169072688},{"type":"v1","addr":"172.21.15.189:6801","nonce":3169072688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":3169072688},{"type":"v1","addr":"172.21.15.189:6803","nonce":3169072688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":3169072688},{"type":"v1","addr":"172.21.15.189:6807","nonce":3169072688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":3169072688},{"type":"v1","addr":"172.21.15.189:6805","nonce":3169072688}]},"public_addr":"172.21.15.189:6801/3169072688","cluster_addr":"172.21.15.189:6803/3169072688","heartbeat_back_addr":"172.21.15.189:6807/3169072688","heartbeat_front_addr":"172.21.15.189:6805/3169072688","state":["exists","up"]},{"osd":3,"uuid":"f5624d78-4455-4e07-8c1a-f47c94cddb23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3131092450},{"type":"v1","addr":"172.21.15.83:6811","nonce":3131092450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3131092450},{"type":"v1","addr":"172.21.15.83:6813","nonce":3131092450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3131092450},{"type":"v1","addr":"172.21.15.83:6817","nonce":3131092450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3131092450},{"type":"v1","addr":"172.21.15.83:6815","nonce":3131092450}]},"public_addr":"172.21.15.83:6811/3131092450","cluster_addr":"172.21.15.83:6813/3131092450","heartbeat_back_addr":"172.21.15.83:6817/3131092450","heartbeat_front_addr":"172.21.15.83:6815/3131092450","state":["exists","up"]},{"osd":4,"uuid":"7b50cd14-9385-4535-a7b0-1c1da93c8548","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":823775360},{"type":"v1","addr":"172.21.15.138:6809","nonce":823775360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":823775360},{"type":"v1","addr":"172.21.15.138:6811","nonce":823775360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":823775360},{"type":"v1","addr":"172.21.15.138:6815","nonce":823775360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":823775360},{"type":"v1","addr":"172.21.15.138:6813","nonce":823775360}]},"public_addr":"172.21.15.138:6809/823775360","cluster_addr":"172.21.15.138:6811/823775360","heartbeat_back_addr":"172.21.15.138:6815/823775360","heartbeat_front_addr":"172.21.15.138:6813/823775360","state":["exists","up"]},{"osd":5,"uuid":"9c647797-35bc-449f-be84-af97d2c69c2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":1399817431},{"type":"v1","addr":"172.21.15.189:6809","nonce":1399817431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":1399817431},{"type":"v1","addr":"172.21.15.189:6811","nonce":1399817431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":1399817431},{"type":"v1","addr":"172.21.15.189:6815","nonce":1399817431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":1399817431},{"type":"v1","addr":"172.21.15.189:6813","nonce":1399817431}]},"public_addr":"172.21.15.189:6809/1399817431","cluster_addr":"172.21.15.189:6811/1399817431","heartbeat_back_addr":"172.21.15.189:6815/1399817431","heartbeat_front_addr":"172.21.15.189:6813/1399817431","state":["exists","up"]},{"osd":6,"uuid":"8f056584-16c8-4454-b445-3f780f26c196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":1611358483},{"type":"v1","addr":"172.21.15.83:6819","nonce":1611358483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1611358483},{"type":"v1","addr":"172.21.15.83:6821","nonce":1611358483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1611358483},{"type":"v1","addr":"172.21.15.83:6825","nonce":1611358483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":1611358483},{"type":"v1","addr":"172.21.15.83:6823","nonce":1611358483}]},"public_addr":"172.21.15.83:6819/1611358483","cluster_addr":"172.21.15.83:6821/1611358483","heartbeat_back_addr":"172.21.15.83:6825/1611358483","heartbeat_front_addr":"172.21.15.83:6823/1611358483","state":["exists","up"]},{"osd":7,"uuid":"7603efe3-da77-4b7f-9d03-29c4f293b6d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":3557856706},{"type":"v1","addr":"172.21.15.138:6817","nonce":3557856706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":3557856706},{"type":"v1","addr":"172.21.15.138:6819","nonce":3557856706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":3557856706},{"type":"v1","addr":"172.21.15.138:6823","nonce":3557856706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":3557856706},{"type":"v1","addr":"172.21.15.138:6821","nonce":3557856706}]},"public_addr":"172.21.15.138:6817/3557856706","cluster_addr":"172.21.15.138:6819/3557856706","heartbeat_back_addr":"172.21.15.138:6823/3557856706","heartbeat_front_addr":"172.21.15.138:6821/3557856706","state":["exists","up"]},{"osd":8,"uuid":"27d7d1b8-d6a0-417e-b8aa-596325aa1a77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":207018554},{"type":"v1","addr":"172.21.15.189:6817","nonce":207018554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":207018554},{"type":"v1","addr":"172.21.15.189:6819","nonce":207018554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":207018554},{"type":"v1","addr":"172.21.15.189:6823","nonce":207018554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":207018554},{"type":"v1","addr":"172.21.15.189:6821","nonce":207018554}]},"public_addr":"172.21.15.189:6817/207018554","cluster_addr":"172.21.15.189:6819/207018554","heartbeat_back_addr":"172.21.15.189:6823/207018554","heartbeat_front_addr":"172.21.15.189:6821/207018554","state":["exists","up"]},{"osd":9,"uuid":"baf19b98-881b-4d3a-9f72-8aa2dd73cad6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":519636751},{"type":"v1","addr":"172.21.15.83:6827","nonce":519636751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":519636751},{"type":"v1","addr":"172.21.15.83:6829","nonce":519636751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":519636751},{"type":"v1","addr":"172.21.15.83:6833","nonce":519636751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":519636751},{"type":"v1","addr":"172.21.15.83:6831","nonce":519636751}]},"public_addr":"172.21.15.83:6827/519636751","cluster_addr":"172.21.15.83:6829/519636751","heartbeat_back_addr":"172.21.15.83:6833/519636751","heartbeat_front_addr":"172.21.15.83:6831/519636751","state":["exists","up"]},{"osd":10,"uuid":"eb346717-5c07-47c8-95be-0e68fe449fdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1130872098},{"type":"v1","addr":"172.21.15.138:6825","nonce":1130872098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1130872098},{"type":"v1","addr":"172.21.15.138:6827","nonce":1130872098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1130872098},{"type":"v1","addr":"172.21.15.138:6831","nonce":1130872098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1130872098},{"type":"v1","addr":"172.21.15.138:6829","nonce":1130872098}]},"public_addr":"172.21.15.138:6825/1130872098","cluster_addr":"172.21.15.138:6827/1130872098","heartbeat_back_addr":"172.21.15.138:6831/1130872098","heartbeat_front_addr":"172.21.15.138:6829/1130872098","state":["exists","up"]},{"osd":11,"uuid":"8e37568e-981c-45b8-b2af-2f1b73898542","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":2998782690},{"type":"v1","addr":"172.21.15.189:6825","nonce":2998782690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":2998782690},{"type":"v1","addr":"172.21.15.189:6827","nonce":2998782690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":2998782690},{"type":"v1","addr":"172.21.15.189:6831","nonce":2998782690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":2998782690},{"type":"v1","addr":"172.21.15.189:6829","nonce":2998782690}]},"public_addr":"172.21.15.189:6825/2998782690","cluster_addr":"172.21.15.189:6827/2998782690","heartbeat_back_addr":"172.21.15.189:6831/2998782690","heartbeat_front_addr":"172.21.15.189:6829/2998782690","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:40:47.980103+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:41:21.970795+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:41:55.160426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:42:30.400475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:43:05.705663+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:43:40.112312+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:44:16.781289+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:44:52.151187+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:45:28.055944+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:46:04.969301+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:46:40.918306+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:47:16.886874+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"2.1d","mappings":[{"from":2,"to":1}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.2a","mappings":[{"from":3,"to":8}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.a","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:0/1096290901":"2024-04-25T08:51:36.347763+0000","172.21.15.83:6800/2550826560":"2024-04-25T08:51:36.347763+0000","172.21.15.83:0/1743153945":"2024-04-25T08:51:36.347763+0000","172.21.15.83:0/3306681366":"2024-04-25T08:51:36.347763+0000","172.21.15.83:6801/820049883":"2024-04-25T08:38:40.661315+0000","172.21.15.83:6801/2550826560":"2024-04-25T08:51:36.347763+0000","172.21.15.83:0/4272874138":"2024-04-25T08:38:40.661315+0000","172.21.15.83:0/4212994335":"2024-04-25T08:38:40.661315+0000","172.21.15.83:6801/1996371091":"2024-04-25T08:38:05.800856+0000","172.21.15.83:0/1820663683":"2024-04-25T08:38:05.800856+0000","172.21.15.83:0/3802897262":"2024-04-25T08:38:05.800856+0000","172.21.15.83:6800/1996371091":"2024-04-25T08:38:05.800856+0000","172.21.15.83:6800/820049883":"2024-04-25T08:38:40.661315+0000","172.21.15.83:0/1366125049":"2024-04-25T08:38:05.800856+0000","172.21.15.83:0/4159820288":"2024-04-25T08:51:36.347763+0000","172.21.15.83:0/340452868":"2024-04-25T08:38:40.661315+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T08:51:46.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.325+0000 7f8cf4e00640 1 -- 172.21.15.83:0/982917112 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f8cd40a3db0 msgr2=0x7f8cd40a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:46.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.325+0000 7f8cf4e00640 1 --2- 172.21.15.83:0/982917112 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f8cd40a3db0 0x7f8cd40a6270 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8cd8005fd0 tx=0x7f8cd8005850 comp rx=0 tx=0).stop 2024-04-24T08:51:46.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.325+0000 7f8cf4e00640 1 -- 172.21.15.83:0/982917112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8cf0074430 msgr2=0x7f8cf01a0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:46.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.325+0000 7f8cf4e00640 1 --2- 172.21.15.83:0/982917112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8cf0074430 0x7f8cf01a0650 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f8ce0005ab0 tx=0x7f8ce0003cd0 comp rx=0 tx=0).stop 2024-04-24T08:51:46.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.325+0000 7f8cf4e00640 1 -- 172.21.15.83:0/982917112 shutdown_connections 2024-04-24T08:51:46.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.325+0000 7f8cf4e00640 1 --2- 172.21.15.83:0/982917112 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f8cd40a3db0 0x7f8cd40a6270 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:46.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.325+0000 7f8cf4e00640 1 --2- 172.21.15.83:0/982917112 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8cf010e800 0x7f8cf019a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:46.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.325+0000 7f8cf4e00640 1 --2- 172.21.15.83:0/982917112 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8cf0074d50 0x7f8cf01a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:46.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.325+0000 7f8cf4e00640 1 --2- 172.21.15.83:0/982917112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8cf0074430 0x7f8cf01a0650 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:46.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.325+0000 7f8cf4e00640 1 -- 172.21.15.83:0/982917112 >> 172.21.15.83:0/982917112 conn(0x7f8cf0079400 msgr2=0x7f8cf00fef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:46.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.325+0000 7f8cf4e00640 1 -- 172.21.15.83:0/982917112 shutdown_connections 2024-04-24T08:51:46.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.326+0000 7f8cf4e00640 1 -- 172.21.15.83:0/982917112 wait complete. 2024-04-24T08:51:46.345 DEBUG:tasks.fwd_scrub:Yielding 2024-04-24T08:51:46.346 INFO:teuthology.run_tasks:Running task exec... 2024-04-24T08:51:46.356 INFO:teuthology.task.exec:Executing custom commands... 2024-04-24T08:51:46.356 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi083.front.sepia.ceph.com 2024-04-24T08:51:46.356 DEBUG:teuthology.orchestra.run.smithi083:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-04-24T08:51:46.357 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-04-24T08:51:46.357 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T08:51:46.357 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T08:51:46.357 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T08:51:46.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:46 smithi138 ceph-mon[31441]: pgmap v7: 129 pgs: 129 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 0 B/s wr, 25 op/s 2024-04-24T08:51:46.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:46 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:51:46.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:46 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:51:46.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:46 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:51:46.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:46 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T08:51:46.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:46 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/3154348841' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T08:51:46.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:46 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/2300771843' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:51:46.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:46 smithi189 ceph-mon[31791]: pgmap v7: 129 pgs: 129 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 0 B/s wr, 25 op/s 2024-04-24T08:51:46.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:46 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:51:46.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:46 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:51:46.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:46 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:51:46.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:46 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T08:51:46.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:46 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/3154348841' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T08:51:46.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:46 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/2300771843' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:51:46.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.545+0000 7fa990200640 1 -- 172.21.15.83:0/2118822813 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa98810e8f0 msgr2=0x7fa988110ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:46.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.545+0000 7fa990200640 1 --2- 172.21.15.83:0/2118822813 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa98810e8f0 0x7fa988110ce0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fa980009ca0 tx=0x7fa98002a950 comp rx=0 tx=0).stop 2024-04-24T08:51:46.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.545+0000 7fa990200640 1 -- 172.21.15.83:0/2118822813 shutdown_connections 2024-04-24T08:51:46.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.545+0000 7fa990200640 1 --2- 172.21.15.83:0/2118822813 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa98810e8f0 0x7fa988110ce0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:46.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.545+0000 7fa990200640 1 --2- 172.21.15.83:0/2118822813 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa988074c20 0x7fa98810e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:46.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.545+0000 7fa990200640 1 --2- 172.21.15.83:0/2118822813 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa988074300 0x7fa9880746e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:46.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.545+0000 7fa990200640 1 -- 172.21.15.83:0/2118822813 >> 172.21.15.83:0/2118822813 conn(0x7fa988079490 msgr2=0x7fa9880798a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:46.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.546+0000 7fa990200640 1 -- 172.21.15.83:0/2118822813 shutdown_connections 2024-04-24T08:51:46.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.546+0000 7fa990200640 1 -- 172.21.15.83:0/2118822813 wait complete. 2024-04-24T08:51:46.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.546+0000 7fd84b200640 1 -- 172.21.15.83:0/2946722544 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd844104e00 msgr2=0x7fd844108e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:46.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.546+0000 7fd84b200640 1 --2- 172.21.15.83:0/2946722544 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd844104e00 0x7fd844108e40 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fd830009900 tx=0x7fd83002a410 comp rx=0 tx=0).stop 2024-04-24T08:51:46.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.547+0000 7fa990200640 1 Processor -- start 2024-04-24T08:51:46.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.547+0000 7fd84b200640 1 -- 172.21.15.83:0/2946722544 shutdown_connections 2024-04-24T08:51:46.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.547+0000 7fd84b200640 1 --2- 172.21.15.83:0/2946722544 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd844109520 0x7fd8441110f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:46.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.547+0000 7fd84b200640 1 --2- 172.21.15.83:0/2946722544 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd844104e00 0x7fd844108e40 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:46.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.547+0000 7fd84b200640 1 --2- 172.21.15.83:0/2946722544 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd844104450 0x7fd844104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:46.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.547+0000 7fd84b200640 1 -- 172.21.15.83:0/2946722544 >> 172.21.15.83:0/2946722544 conn(0x7fd8440ffc70 msgr2=0x7fd844102090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:46.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.547+0000 7fa990200640 1 -- start start 2024-04-24T08:51:46.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.547+0000 7fd84b200640 1 -- 172.21.15.83:0/2946722544 shutdown_connections 2024-04-24T08:51:46.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.547+0000 7fd84b200640 1 -- 172.21.15.83:0/2946722544 wait complete. 2024-04-24T08:51:46.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.547+0000 7fa990200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa988074300 0x7fa9881a05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:46.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.547+0000 7fa990200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa988074c20 0x7fa9881a0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:46.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.547+0000 7fa990200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa98810e8f0 0x7fa98819a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:46.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.547+0000 7fa990200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa9881144e0 con 0x7fa988074c20 2024-04-24T08:51:46.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.547+0000 7fa990200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa988114360 con 0x7fa98810e8f0 2024-04-24T08:51:46.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.547+0000 7fa990200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa988114660 con 0x7fa988074300 2024-04-24T08:51:46.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.547+0000 7fa98ce00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa988074c20 0x7fa9881a0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:46.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.547+0000 7fa98ce00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa988074c20 0x7fa9881a0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56008/0 (socket says 172.21.15.83:56008) 2024-04-24T08:51:46.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.548+0000 7fa98ce00640 1 -- 172.21.15.83:0/3677731376 learned_addr learned my addr 172.21.15.83:0/3677731376 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:51:46.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.548+0000 7fa98d800640 1 --2- 172.21.15.83:0/3677731376 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa988074300 0x7fa9881a05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:46.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.548+0000 7fa98e200640 1 --2- 172.21.15.83:0/3677731376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa98810e8f0 0x7fa98819a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:46.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.548+0000 7fa98ce00640 1 -- 172.21.15.83:0/3677731376 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa988074300 msgr2=0x7fa9881a05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:46.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.548+0000 7fa98ce00640 1 --2- 172.21.15.83:0/3677731376 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa988074300 0x7fa9881a05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:46.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.548+0000 7fa98ce00640 1 -- 172.21.15.83:0/3677731376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa98810e8f0 msgr2=0x7fa98819a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:46.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.548+0000 7fa98ce00640 1 --2- 172.21.15.83:0/3677731376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa98810e8f0 0x7fa98819a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:46.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.548+0000 7fa98ce00640 1 -- 172.21.15.83:0/3677731376 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa98819af40 con 0x7fa988074c20 2024-04-24T08:51:46.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.548+0000 7fa98e200640 1 --2- 172.21.15.83:0/3677731376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa98810e8f0 0x7fa98819a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:51:46.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.548+0000 7fd84b200640 1 Processor -- start 2024-04-24T08:51:46.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.548+0000 7fa98ce00640 1 --2- 172.21.15.83:0/3677731376 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa988074c20 0x7fa9881a0b00 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fa97400d770 tx=0x7fa97400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:46.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.548+0000 7fd84b200640 1 -- start start 2024-04-24T08:51:46.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.549+0000 7fa986000640 1 -- 172.21.15.83:0/3677731376 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa974017070 con 0x7fa988074c20 2024-04-24T08:51:46.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.549+0000 7fa986000640 1 -- 172.21.15.83:0/3677731376 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa9740054d0 con 0x7fa988074c20 2024-04-24T08:51:46.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.549+0000 7fa986000640 1 -- 172.21.15.83:0/3677731376 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa974004cf0 con 0x7fa988074c20 2024-04-24T08:51:46.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.549+0000 7fa990200640 1 -- 172.21.15.83:0/3677731376 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa98819b230 con 0x7fa988074c20 2024-04-24T08:51:46.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.549+0000 7fd84b200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd844104450 0x7fd8441a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:46.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.549+0000 7fd84b200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd844109520 0x7fd8441a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:46.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.549+0000 7fa990200640 1 -- 172.21.15.83:0/3677731376 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9881a7540 con 0x7fa988074c20 2024-04-24T08:51:46.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.549+0000 7fd84b200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd84419a830 0x7fd84419ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:46.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.549+0000 7fd84b200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd844114490 con 0x7fd844104450 2024-04-24T08:51:46.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.549+0000 7fd84b200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd844114310 con 0x7fd844109520 2024-04-24T08:51:46.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.549+0000 7fd84b200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd844114610 con 0x7fd84419a830 2024-04-24T08:51:46.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.549+0000 7fd843e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd844104450 0x7fd8441a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:46.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.549+0000 7fd843e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd844104450 0x7fd8441a0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56024/0 (socket says 172.21.15.83:56024) 2024-04-24T08:51:46.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.549+0000 7fd843e00640 1 -- 172.21.15.83:0/1407811744 learned_addr learned my addr 172.21.15.83:0/1407811744 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:51:46.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.550+0000 7fd843400640 1 --2- 172.21.15.83:0/1407811744 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd844109520 0x7fd8441a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:46.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.550+0000 7fd849200640 1 --2- 172.21.15.83:0/1407811744 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd84419a830 0x7fd84419ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:46.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.550+0000 7fd843e00640 1 -- 172.21.15.83:0/1407811744 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd84419a830 msgr2=0x7fd84419ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:46.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.550+0000 7fd843e00640 1 --2- 172.21.15.83:0/1407811744 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd84419a830 0x7fd84419ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:46.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.550+0000 7fd843e00640 1 -- 172.21.15.83:0/1407811744 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd844109520 msgr2=0x7fd8441a0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:46.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.550+0000 7fd843e00640 1 --2- 172.21.15.83:0/1407811744 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd844109520 0x7fd8441a0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:46.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.550+0000 7fd843e00640 1 -- 172.21.15.83:0/1407811744 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd84419b3c0 con 0x7fd844104450 2024-04-24T08:51:46.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.550+0000 7fd843400640 1 --2- 172.21.15.83:0/1407811744 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd844109520 0x7fd8441a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:51:46.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.550+0000 7fd849200640 1 --2- 172.21.15.83:0/1407811744 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd84419a830 0x7fd84419ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:51:46.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.550+0000 7fd843e00640 1 --2- 172.21.15.83:0/1407811744 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd844104450 0x7fd8441a0670 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fd83400c890 tx=0x7fd83400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:46.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.550+0000 7fa990200640 1 -- 172.21.15.83:0/3677731376 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa988075a40 con 0x7fa988074c20 2024-04-24T08:51:46.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.551+0000 7fd840c00640 1 -- 172.21.15.83:0/1407811744 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd834004220 con 0x7fd844104450 2024-04-24T08:51:46.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.551+0000 7fd840c00640 1 -- 172.21.15.83:0/1407811744 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd8340043c0 con 0x7fd844104450 2024-04-24T08:51:46.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.551+0000 7fd840c00640 1 -- 172.21.15.83:0/1407811744 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd834004a00 con 0x7fd844104450 2024-04-24T08:51:46.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.551+0000 7fd84b200640 1 -- 172.21.15.83:0/1407811744 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd84419b6b0 con 0x7fd844104450 2024-04-24T08:51:46.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.551+0000 7fd84b200640 1 -- 172.21.15.83:0/1407811744 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8441a7890 con 0x7fd844104450 2024-04-24T08:51:46.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.553+0000 7fa986000640 1 -- 172.21.15.83:0/3677731376 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7fa974004e90 con 0x7fa988074c20 2024-04-24T08:51:46.557 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:46 smithi083 ceph-mon[27907]: pgmap v7: 129 pgs: 129 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 0 B/s wr, 25 op/s 2024-04-24T08:51:46.557 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:46 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:51:46.557 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:46 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:51:46.557 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:46 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:51:46.557 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:46 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T08:51:46.557 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:46 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/3154348841' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T08:51:46.557 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:46 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/2300771843' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:51:46.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.554+0000 7fd84dfe1740 1 Processor -- start 2024-04-24T08:51:46.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.554+0000 7fd84dfe1740 1 -- start start 2024-04-24T08:51:46.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.555+0000 7fd840c00640 1 -- 172.21.15.83:0/1407811744 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7fd834004ba0 con 0x7fd844104450 2024-04-24T08:51:46.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.555+0000 7fd84dfe1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d0844b6090 0x55d0844b6470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:46.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.555+0000 7fd84dfe1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0844b69b0 0x55d0844c0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:46.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.555+0000 7fd84dfe1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d0844c1080 0x55d0844c3470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:46.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.555+0000 7fd84dfe1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d08441be70 con 0x55d0844b69b0 2024-04-24T08:51:46.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.555+0000 7fd84dfe1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d0843c2380 con 0x55d0844c1080 2024-04-24T08:51:46.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.555+0000 7fd84dfe1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d084417e30 con 0x55d0844b6090 2024-04-24T08:51:46.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.555+0000 7fd843e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0844b69b0 0x55d0844c0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:46.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.555+0000 7fa986000640 1 --2- 172.21.15.83:0/3677731376 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa9700a3ee0 0x7fa9700a63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:46.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.555+0000 7fd843e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0844b69b0 0x55d0844c0b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56038/0 (socket says 172.21.15.83:56038) 2024-04-24T08:51:46.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.555+0000 7fd843e00640 1 -- 172.21.15.83:0/1362554965 learned_addr learned my addr 172.21.15.83:0/1362554965 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:51:46.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.556+0000 7fa986000640 1 -- 172.21.15.83:0/3677731376 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7fa9740d0780 con 0x7fa988074c20 2024-04-24T08:51:46.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.556+0000 7fa98d800640 1 --2- 172.21.15.83:0/3677731376 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa9700a3ee0 0x7fa9700a63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:46.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.556+0000 7fd843400640 1 --2- 172.21.15.83:0/1362554965 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d0844c1080 0x55d0844c3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:46.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.556+0000 7fd849200640 1 --2- 172.21.15.83:0/1362554965 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d0844b6090 0x55d0844b6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:46.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.556+0000 7fd843e00640 1 -- 172.21.15.83:0/1362554965 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d0844b6090 msgr2=0x55d0844b6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:51:46.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.556+0000 7fd843e00640 1 --2- 172.21.15.83:0/1362554965 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d0844b6090 0x55d0844b6470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:46.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.556+0000 7fd843e00640 1 -- 172.21.15.83:0/1362554965 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d0844c1080 msgr2=0x55d0844c3470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:51:46.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.556+0000 7fd843e00640 1 --2- 172.21.15.83:0/1362554965 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d0844c1080 0x55d0844c3470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:46.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.556+0000 7fd843e00640 1 -- 172.21.15.83:0/1362554965 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0841f67e0 con 0x55d0844b69b0 2024-04-24T08:51:46.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.556+0000 7fd843400640 1 --2- 172.21.15.83:0/1362554965 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d0844c1080 0x55d0844c3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:51:46.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.556+0000 7fd840c00640 1 --2- 172.21.15.83:0/1407811744 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fd8280a3e30 0x7fd8280a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:46.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.556+0000 7fd843e00640 1 --2- 172.21.15.83:0/1362554965 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0844b69b0 0x55d0844c0b40 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fd8340cdca0 tx=0x7fd8340d9580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71316fdb8489644d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:46.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.557+0000 7fa98d800640 1 --2- 172.21.15.83:0/3677731376 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa9700a3ee0 0x7fa9700a63a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa97c005fd0 tx=0x7fa97c005850 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:46.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.557+0000 7fd849200640 1 --2- 172.21.15.83:0/1407811744 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fd8280a3e30 0x7fd8280a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:46.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.557+0000 7fd840c00640 1 -- 172.21.15.83:0/1407811744 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7fd8340d0a40 con 0x7fd844104450 2024-04-24T08:51:46.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.557+0000 7fd849200640 1 --2- 172.21.15.83:0/1407811744 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fd8280a3e30 0x7fd8280a62f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd83c005cc0 tx=0x7fd83c0083f0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:46.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.560+0000 7fd839600640 1 -- 172.21.15.83:0/1362554965 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8340d4e60 con 0x55d0844b69b0 2024-04-24T08:51:46.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.560+0000 7fd839600640 1 -- 172.21.15.83:0/1362554965 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd83409cba0 con 0x55d0844b69b0 2024-04-24T08:51:46.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.560+0000 7fd839600640 1 -- 172.21.15.83:0/1362554965 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd83409be00 con 0x55d0844b69b0 2024-04-24T08:51:46.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.560+0000 7fa986000640 1 -- 172.21.15.83:0/3677731376 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa97409c8e0 con 0x7fa988074c20 2024-04-24T08:51:46.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.561+0000 7fd84dfe1740 1 -- 172.21.15.83:0/1362554965 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0844b69b0 msgr2=0x55d0844c0b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:46.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.561+0000 7fd84dfe1740 1 --2- 172.21.15.83:0/1362554965 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0844b69b0 0x55d0844c0b40 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fd8340cdca0 tx=0x7fd8340d9580 comp rx=0 tx=0).stop 2024-04-24T08:51:46.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.561+0000 7fd84dfe1740 1 -- 172.21.15.83:0/1362554965 shutdown_connections 2024-04-24T08:51:46.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.561+0000 7fd84dfe1740 1 --2- 172.21.15.83:0/1362554965 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d0844c1080 0x55d0844c3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:46.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.561+0000 7fd84dfe1740 1 --2- 172.21.15.83:0/1362554965 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0844b69b0 0x55d0844c0b40 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:46.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.561+0000 7fd84dfe1740 1 --2- 172.21.15.83:0/1362554965 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d0844b6090 0x55d0844b6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:46.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.561+0000 7fd84dfe1740 1 -- 172.21.15.83:0/1362554965 >> 172.21.15.83:0/1362554965 conn(0x55d0844b23b0 msgr2=0x55d0844b2790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:46.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.561+0000 7fd84dfe1740 1 -- 172.21.15.83:0/1362554965 shutdown_connections 2024-04-24T08:51:46.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.561+0000 7fd84dfe1740 1 -- 172.21.15.83:0/1362554965 wait complete. 2024-04-24T08:51:46.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.562+0000 7fd84dfe1740 1 -- start start 2024-04-24T08:51:46.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.562+0000 7fd84dfe1740 1 Processor -- start 2024-04-24T08:51:46.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.562+0000 7fd84dfe1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d0844b6090 0x55d0844be0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:46.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.562+0000 7fd84dfe1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d0844b69b0 0x55d0844be5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:46.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.562+0000 7fd84dfe1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0844c1080 0x55d0844beb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:46.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.562+0000 7fd84dfe1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d0844c3e70 con 0x55d0844c1080 2024-04-24T08:51:46.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.562+0000 7fd84dfe1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d0843d5660 con 0x55d0844b69b0 2024-04-24T08:51:46.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.562+0000 7fd84dfe1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d0844c3ff0 con 0x55d0844b6090 2024-04-24T08:51:46.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.562+0000 7fd84dfe1740 10 client.0 _ll_register_callbacks cb 0x55d0843c3aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:51:46.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.563+0000 7fd849200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0844c1080 0x55d0844beb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:46.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.563+0000 7fd849200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0844c1080 0x55d0844beb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56050/0 (socket says 172.21.15.83:56050) 2024-04-24T08:51:46.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.563+0000 7fd849200640 1 -- 172.21.15.83:0/2250627225 learned_addr learned my addr 172.21.15.83:0/2250627225 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:51:46.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.563+0000 7fd843e00640 1 --2- 172.21.15.83:0/2250627225 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d0844b6090 0x55d0844be0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:46.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.563+0000 7fd843400640 1 --2- 172.21.15.83:0/2250627225 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d0844b69b0 0x55d0844be5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:46.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.563+0000 7fd849200640 1 -- 172.21.15.83:0/2250627225 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d0844b6090 msgr2=0x55d0844be0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:51:46.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.563+0000 7fd849200640 1 --2- 172.21.15.83:0/2250627225 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d0844b6090 0x55d0844be0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:46.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.563+0000 7fd849200640 1 -- 172.21.15.83:0/2250627225 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d0844b69b0 msgr2=0x55d0844be5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:51:46.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.563+0000 7fd849200640 1 --2- 172.21.15.83:0/2250627225 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d0844b69b0 0x55d0844be5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:46.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.563+0000 7fd849200640 1 -- 172.21.15.83:0/2250627225 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d084256f70 con 0x55d0844c1080 2024-04-24T08:51:46.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.564+0000 7fd849200640 1 --2- 172.21.15.83:0/2250627225 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0844c1080 0x55d0844beb20 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fd83c010d90 tx=0x7fd83c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97306b477ae6ffcb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:46.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.564+0000 7fd82e000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:51:46.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.564+0000 7fd82e000640 1 -- 172.21.15.83:0/2250627225 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd83c01d620 con 0x55d0844c1080 2024-04-24T08:51:46.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.564+0000 7fd82e000640 1 -- 172.21.15.83:0/2250627225 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7fd83c01d7c0 con 0x55d0844c1080 2024-04-24T08:51:46.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.564+0000 7fd84dfe1740 1 -- 172.21.15.83:0/2250627225 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd83409cba0 con 0x55d0844c1080 2024-04-24T08:51:46.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.564+0000 7fd82e000640 1 -- 172.21.15.83:0/2250627225 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd83c02e050 con 0x55d0844c1080 2024-04-24T08:51:46.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.564+0000 7fd84dfe1740 10 client.15795 fetch_fsmap learned FSMap version 28 2024-04-24T08:51:46.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.564+0000 7fd84dfe1740 1 -- 172.21.15.83:0/2250627225 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d0840ee170 con 0x55d0844c1080 2024-04-24T08:51:46.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.565+0000 7fd82e000640 1 -- 172.21.15.83:0/2250627225 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd83409cba0 con 0x55d0844c1080 2024-04-24T08:51:46.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.565+0000 7fd82e000640 1 -- 172.21.15.83:0/2250627225 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd83c01b940 con 0x55d0844c1080 2024-04-24T08:51:46.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.565+0000 7fd84dfe1740 10 client.15795 fetch_fsmap finished waiting for FSMap version 28 2024-04-24T08:51:46.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.565+0000 7fd84dfe1740 10 client.15795 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:51:46.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.565+0000 7fd84dfe1740 20 client.15795 populate_metadata read hostname 'smithi083' 2024-04-24T08:51:46.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.565+0000 7fd84dfe1740 1 --2- 172.21.15.83:0/2250627225 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d0844d14f0 0x55d0844d3960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:46.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.565+0000 7fd84dfe1740 4 client.15795 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:51:46.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.565+0000 7fd84dfe1740 1 -- 172.21.15.83:0/2250627225 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d084429890 con 0x55d0844d14f0 2024-04-24T08:51:46.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.566+0000 7fd843e00640 1 --2- 172.21.15.83:0/2250627225 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d0844d14f0 0x55d0844d3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:46.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.566+0000 7fd843e00640 1 --2- 172.21.15.83:0/2250627225 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d0844d14f0 0x55d0844d3960 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=166cc2ce23201233 server_cookie=9fce8b9ce327e04 in_seq=0 out_seq=0 2024-04-24T08:51:46.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.566+0000 7fd82e000640 10 client.15795 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:51:46.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.568+0000 7fd82e000640 1 -- 172.21.15.83:0/2250627225 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d084429890 con 0x55d0844d14f0 2024-04-24T08:51:46.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.568+0000 7fd82e000640 10 client.15795 handle_command_reply: tid=0 2024-04-24T08:51:46.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.568+0000 7fd84dfe1740 1 client.15795 shutdown 2024-04-24T08:51:46.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.568+0000 7fd84dfe1740 1 -- 172.21.15.83:0/2250627225 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0844c1080 msgr2=0x55d0844beb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:46.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.568+0000 7fd84dfe1740 1 --2- 172.21.15.83:0/2250627225 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0844c1080 0x55d0844beb20 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fd83c010d90 tx=0x7fd83c004a40 comp rx=0 tx=0).stop 2024-04-24T08:51:46.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.569+0000 7fd84dfe1740 1 -- 172.21.15.83:0/2250627225 shutdown_connections 2024-04-24T08:51:46.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.569+0000 7fd84dfe1740 1 --2- 172.21.15.83:0/2250627225 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d0844d14f0 0x55d0844d3960 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:46.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.569+0000 7fd84dfe1740 1 --2- 172.21.15.83:0/2250627225 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0844c1080 0x55d0844beb20 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:46.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.569+0000 7fd84dfe1740 1 --2- 172.21.15.83:0/2250627225 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d0844b69b0 0x55d0844be5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:46.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.569+0000 7fd84dfe1740 1 --2- 172.21.15.83:0/2250627225 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d0844b6090 0x55d0844be0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:46.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.569+0000 7fd84dfe1740 1 -- 172.21.15.83:0/2250627225 >> 172.21.15.83:0/2250627225 conn(0x55d0844b23b0 msgr2=0x55d0844b81b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:46.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.569+0000 7fd82e000640 0 client.15795 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:51:46.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.569+0000 7fd84dfe1740 1 -- 172.21.15.83:0/2250627225 shutdown_connections 2024-04-24T08:51:46.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.569+0000 7fd84dfe1740 1 -- 172.21.15.83:0/2250627225 wait complete. 2024-04-24T08:51:46.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.569+0000 7fd84dfe1740 20 client.15795 trim_cache size 0 max 16384 2024-04-24T08:51:46.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.588+0000 7fd84dfe1740 1 Processor -- start 2024-04-24T08:51:46.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.588+0000 7fd84dfe1740 1 -- start start 2024-04-24T08:51:46.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.588+0000 7fd84dfe1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0844b6090 0x55d0844ccd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:46.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.588+0000 7fd84dfe1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d0844b69b0 0x55d0844b8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:46.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.588+0000 7fd84dfe1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d0844c1080 0x55d0844d9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:46.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.588+0000 7fd84dfe1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd8441b17e0 con 0x55d0844b6090 2024-04-24T08:51:46.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.588+0000 7fd84dfe1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd8441b3320 con 0x55d0844c1080 2024-04-24T08:51:46.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.588+0000 7fd84dfe1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd8441b1eb0 con 0x55d0844b69b0 2024-04-24T08:51:46.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.589+0000 7fd843e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0844b6090 0x55d0844ccd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:46.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.589+0000 7fd843e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0844b6090 0x55d0844ccd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56056/0 (socket says 172.21.15.83:56056) 2024-04-24T08:51:46.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.589+0000 7fd843e00640 1 -- 172.21.15.83:0/1999275762 learned_addr learned my addr 172.21.15.83:0/1999275762 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:51:46.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.589+0000 7fd843400640 1 --2- 172.21.15.83:0/1999275762 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d0844b69b0 0x55d0844b8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:46.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.589+0000 7fd849200640 1 --2- 172.21.15.83:0/1999275762 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d0844c1080 0x55d0844d9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:46.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.589+0000 7fd843e00640 1 -- 172.21.15.83:0/1999275762 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d0844b69b0 msgr2=0x55d0844b8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:51:46.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.589+0000 7fd843e00640 1 --2- 172.21.15.83:0/1999275762 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d0844b69b0 0x55d0844b8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:46.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.589+0000 7fd843e00640 1 -- 172.21.15.83:0/1999275762 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d0844c1080 msgr2=0x55d0844d9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:51:46.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.589+0000 7fd843e00640 1 --2- 172.21.15.83:0/1999275762 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d0844c1080 0x55d0844d9230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:46.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.589+0000 7fd843e00640 1 -- 172.21.15.83:0/1999275762 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0844c37e0 con 0x55d0844b6090 2024-04-24T08:51:46.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.590+0000 7fd843e00640 1 --2- 172.21.15.83:0/1999275762 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0844b6090 0x55d0844ccd30 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fd8340ce3c0 tx=0x7fd834000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e9c085c1162d5d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:46.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.590+0000 7fd838c00640 1 -- 172.21.15.83:0/1999275762 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd83401d040 con 0x55d0844b6090 2024-04-24T08:51:46.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.590+0000 7fd838c00640 1 -- 172.21.15.83:0/1999275762 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd83409cba0 con 0x55d0844b6090 2024-04-24T08:51:46.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.590+0000 7fd838c00640 1 -- 172.21.15.83:0/1999275762 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd83409bfe0 con 0x55d0844b6090 2024-04-24T08:51:46.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.591+0000 7fd84dfe1740 1 -- 172.21.15.83:0/1999275762 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0844b6090 msgr2=0x55d0844ccd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:46.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.591+0000 7fd84dfe1740 1 --2- 172.21.15.83:0/1999275762 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0844b6090 0x55d0844ccd30 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fd8340ce3c0 tx=0x7fd834000bc0 comp rx=0 tx=0).stop 2024-04-24T08:51:46.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.591+0000 7fd84dfe1740 1 -- 172.21.15.83:0/1999275762 shutdown_connections 2024-04-24T08:51:46.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.591+0000 7fd84dfe1740 1 --2- 172.21.15.83:0/1999275762 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d0844c1080 0x55d0844d9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:46.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.591+0000 7fd84dfe1740 1 --2- 172.21.15.83:0/1999275762 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d0844b69b0 0x55d0844b8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:46.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.591+0000 7fd84dfe1740 1 --2- 172.21.15.83:0/1999275762 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0844b6090 0x55d0844ccd30 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:46.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.591+0000 7fd84dfe1740 1 -- 172.21.15.83:0/1999275762 >> 172.21.15.83:0/1999275762 conn(0x55d0844b23b0 msgr2=0x55d0844c0160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:46.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.591+0000 7fd84dfe1740 1 -- 172.21.15.83:0/1999275762 shutdown_connections 2024-04-24T08:51:46.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.591+0000 7fd84dfe1740 1 -- 172.21.15.83:0/1999275762 wait complete. 2024-04-24T08:51:46.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.591+0000 7fd84dfe1740 1 -- start start 2024-04-24T08:51:46.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.591+0000 7fd84dfe1740 1 Processor -- start 2024-04-24T08:51:46.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.592+0000 7fd84dfe1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d0844b6090 0x55d0844c9240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:46.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.592+0000 7fd84dfe1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0844b69b0 0x55d0844c9780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:46.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.592+0000 7fd84dfe1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d0844c1080 0x55d0844c6270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:46.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.592+0000 7fd84dfe1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd8441b2c50 con 0x55d0844b69b0 2024-04-24T08:51:46.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.592+0000 7fd84dfe1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd8441b2580 con 0x55d0844c1080 2024-04-24T08:51:46.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.592+0000 7fd84dfe1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd8441af7c0 con 0x55d0844b6090 2024-04-24T08:51:46.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.592+0000 7fd84dfe1740 10 client.0 _ll_register_callbacks cb 0x7fd83c008a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:51:46.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.592+0000 7fd843400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0844b69b0 0x55d0844c9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:46.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.592+0000 7fd843400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0844b69b0 0x55d0844c9780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56060/0 (socket says 172.21.15.83:56060) 2024-04-24T08:51:46.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.592+0000 7fd843400640 1 -- 172.21.15.83:0/1555328045 learned_addr learned my addr 172.21.15.83:0/1555328045 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:51:46.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.592+0000 7fd843e00640 1 --2- 172.21.15.83:0/1555328045 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d0844b6090 0x55d0844c9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:46.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.593+0000 7fd843400640 1 -- 172.21.15.83:0/1555328045 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d0844b6090 msgr2=0x55d0844c9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:51:46.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.593+0000 7fd843400640 1 --2- 172.21.15.83:0/1555328045 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d0844b6090 0x55d0844c9240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:46.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.593+0000 7fd843400640 1 -- 172.21.15.83:0/1555328045 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d0844c1080 msgr2=0x55d0844c6270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:51:46.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.593+0000 7fd843400640 1 --2- 172.21.15.83:0/1555328045 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d0844c1080 0x55d0844c6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:46.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.593+0000 7fd843400640 1 -- 172.21.15.83:0/1555328045 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0844c6aa0 con 0x55d0844b69b0 2024-04-24T08:51:46.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.593+0000 7fd843400640 1 --2- 172.21.15.83:0/1555328045 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0844b69b0 0x55d0844c9780 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fd83002a930 tx=0x7fd8300053b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d7c9eae4b3989d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:46.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.593+0000 7fd82e000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:51:46.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.593+0000 7fd82e000640 1 -- 172.21.15.83:0/1555328045 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd830035070 con 0x55d0844b69b0 2024-04-24T08:51:46.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.593+0000 7fd82e000640 1 -- 172.21.15.83:0/1555328045 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7fd8300048c0 con 0x55d0844b69b0 2024-04-24T08:51:46.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.593+0000 7fd84dfe1740 1 -- 172.21.15.83:0/1555328045 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd83409cba0 con 0x55d0844b69b0 2024-04-24T08:51:46.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.593+0000 7fd82e000640 1 -- 172.21.15.83:0/1555328045 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd83004d050 con 0x55d0844b69b0 2024-04-24T08:51:46.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.593+0000 7fd84dfe1740 10 client.15801 fetch_fsmap learned FSMap version 28 2024-04-24T08:51:46.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.593+0000 7fd84dfe1740 1 -- 172.21.15.83:0/1555328045 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d0845386a0 con 0x55d0844b69b0 2024-04-24T08:51:46.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.594+0000 7fd82e000640 1 -- 172.21.15.83:0/1555328045 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd83409cba0 con 0x55d0844b69b0 2024-04-24T08:51:46.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.594+0000 7fd82e000640 1 -- 172.21.15.83:0/1555328045 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd830042020 con 0x55d0844b69b0 2024-04-24T08:51:46.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.594+0000 7fd84dfe1740 10 client.15801 fetch_fsmap finished waiting for FSMap version 28 2024-04-24T08:51:46.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.594+0000 7fd84dfe1740 10 client.15801 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:51:46.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.594+0000 7fd84dfe1740 20 client.15801 populate_metadata read hostname 'smithi083' 2024-04-24T08:51:46.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.594+0000 7fd84dfe1740 1 --2- 172.21.15.83:0/1555328045 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d084538e50 0x55d08453b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:46.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.594+0000 7fd84dfe1740 4 client.15801 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T08:51:46.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.594+0000 7fd84dfe1740 1 -- 172.21.15.83:0/1555328045 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55d084423700 con 0x55d084538e50 2024-04-24T08:51:46.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.595+0000 7fd843e00640 1 --2- 172.21.15.83:0/1555328045 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d084538e50 0x55d08453b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:46.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.596+0000 7fd843e00640 1 --2- 172.21.15.83:0/1555328045 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d084538e50 0x55d08453b290 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e7a132e671b09ea server_cookie=88404142c3fa3c06 in_seq=0 out_seq=0 2024-04-24T08:51:46.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.596+0000 7fd82e000640 10 client.15801 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:51:46.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.598+0000 7fd82e000640 1 -- 172.21.15.83:0/1555328045 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d084423700 con 0x55d084538e50 2024-04-24T08:51:46.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.598+0000 7fd82e000640 10 client.15801 handle_command_reply: tid=0 2024-04-24T08:51:46.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.598+0000 7fd84dfe1740 1 client.15801 shutdown 2024-04-24T08:51:46.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.599+0000 7fd84dfe1740 1 -- 172.21.15.83:0/1555328045 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0844b69b0 msgr2=0x55d0844c9780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:46.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.599+0000 7fd84dfe1740 1 --2- 172.21.15.83:0/1555328045 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0844b69b0 0x55d0844c9780 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fd83002a930 tx=0x7fd8300053b0 comp rx=0 tx=0).stop 2024-04-24T08:51:46.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.599+0000 7fd84dfe1740 1 -- 172.21.15.83:0/1555328045 shutdown_connections 2024-04-24T08:51:46.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.599+0000 7fd84dfe1740 1 --2- 172.21.15.83:0/1555328045 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d084538e50 0x55d08453b290 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:46.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.599+0000 7fd84dfe1740 1 --2- 172.21.15.83:0/1555328045 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d0844c1080 0x55d0844c6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:46.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.599+0000 7fd84dfe1740 1 --2- 172.21.15.83:0/1555328045 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0844b69b0 0x55d0844c9780 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:46.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.599+0000 7fd84dfe1740 1 --2- 172.21.15.83:0/1555328045 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d0844b6090 0x55d0844c9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:46.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.599+0000 7fd82e000640 0 client.15801 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:51:46.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.599+0000 7fd84dfe1740 1 -- 172.21.15.83:0/1555328045 >> 172.21.15.83:0/1555328045 conn(0x55d0844b23b0 msgr2=0x55d0844be060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:46.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.599+0000 7fd84dfe1740 1 -- 172.21.15.83:0/1555328045 shutdown_connections 2024-04-24T08:51:46.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.599+0000 7fd84dfe1740 1 -- 172.21.15.83:0/1555328045 wait complete. 2024-04-24T08:51:46.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.599+0000 7fd84dfe1740 20 client.15801 trim_cache size 0 max 16384 2024-04-24T08:51:46.600 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T08:51:46.600 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-04-24T08:51:46.600 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "62d86377-f426-4a96-89d0-27abf9b5772e", 2024-04-24T08:51:46.600 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-04-24T08:51:46.600 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T08:51:46.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.600+0000 7fd84b200640 1 -- 172.21.15.83:0/1407811744 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fd8280a3e30 msgr2=0x7fd8280a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:46.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.600+0000 7fd84b200640 1 --2- 172.21.15.83:0/1407811744 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fd8280a3e30 0x7fd8280a62f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd83c005cc0 tx=0x7fd83c0083f0 comp rx=0 tx=0).stop 2024-04-24T08:51:46.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.600+0000 7fd84b200640 1 -- 172.21.15.83:0/1407811744 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd844104450 msgr2=0x7fd8441a0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:46.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.600+0000 7fd84b200640 1 --2- 172.21.15.83:0/1407811744 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd844104450 0x7fd8441a0670 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fd83400c890 tx=0x7fd83400cd50 comp rx=0 tx=0).stop 2024-04-24T08:51:46.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.600+0000 7fd84b200640 1 -- 172.21.15.83:0/1407811744 shutdown_connections 2024-04-24T08:51:46.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.600+0000 7fd84b200640 1 --2- 172.21.15.83:0/1407811744 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fd8280a3e30 0x7fd8280a62f0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:46.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.600+0000 7fd84b200640 1 --2- 172.21.15.83:0/1407811744 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd84419a830 0x7fd84419ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:46.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.600+0000 7fd84b200640 1 --2- 172.21.15.83:0/1407811744 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd844109520 0x7fd8441a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:46.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.600+0000 7fd84b200640 1 --2- 172.21.15.83:0/1407811744 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd844104450 0x7fd8441a0670 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:46.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.600+0000 7fd84b200640 1 -- 172.21.15.83:0/1407811744 >> 172.21.15.83:0/1407811744 conn(0x7fd8440ffc70 msgr2=0x7fd844101060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:46.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.600+0000 7fd84b200640 1 -- 172.21.15.83:0/1407811744 shutdown_connections 2024-04-24T08:51:46.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.600+0000 7fd84b200640 1 -- 172.21.15.83:0/1407811744 wait complete. 2024-04-24T08:51:46.616 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T08:51:46.616 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T08:51:46.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.793+0000 7fa990200640 1 -- 172.21.15.83:0/3677731376 --> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7fa9880ff550 con 0x7fa9700a3ee0 2024-04-24T08:51:46.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.800+0000 7fc4c0e00640 1 -- 172.21.15.83:0/116796308 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc4bc1071e0 msgr2=0x7fc4bc10edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:46.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.800+0000 7fc4c0e00640 1 --2- 172.21.15.83:0/116796308 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc4bc1071e0 0x7fc4bc10edb0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fc4b4009ca0 tx=0x7fc4b402a8c0 comp rx=0 tx=0).stop 2024-04-24T08:51:46.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.800+0000 7fc4c0e00640 1 -- 172.21.15.83:0/116796308 shutdown_connections 2024-04-24T08:51:46.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.800+0000 7fc4c0e00640 1 --2- 172.21.15.83:0/116796308 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc4bc1071e0 0x7fc4bc10edb0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:46.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.800+0000 7fc4c0e00640 1 --2- 172.21.15.83:0/116796308 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc4bc102ac0 0x7fc4bc106b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:46.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.800+0000 7fc4c0e00640 1 --2- 172.21.15.83:0/116796308 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc4bc102110 0x7fc4bc1024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:46.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.800+0000 7fc4c0e00640 1 -- 172.21.15.83:0/116796308 >> 172.21.15.83:0/116796308 conn(0x7fc4bc0fd9d0 msgr2=0x7fc4bc0ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:46.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.800+0000 7fc4c0e00640 1 -- 172.21.15.83:0/116796308 shutdown_connections 2024-04-24T08:51:46.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.800+0000 7fc4c0e00640 1 -- 172.21.15.83:0/116796308 wait complete. 2024-04-24T08:51:46.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.801+0000 7fc4c0e00640 1 Processor -- start 2024-04-24T08:51:46.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.801+0000 7fc4c0e00640 1 -- start start 2024-04-24T08:51:46.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.802+0000 7fc4c0e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc4bc102110 0x7fc4bc198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:46.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.802+0000 7fc4c0e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc4bc1071e0 0x7fc4bc198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:46.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.802+0000 7fc4c0e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc4bc19cf70 0x7fc4bc19d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:46.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.802+0000 7fc4c0e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc4bc1121a0 con 0x7fc4bc102110 2024-04-24T08:51:46.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.802+0000 7fc4c0e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc4bc112020 con 0x7fc4bc19cf70 2024-04-24T08:51:46.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.802+0000 7fc4c0e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc4bc112320 con 0x7fc4bc1071e0 2024-04-24T08:51:46.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.802+0000 7fc4bb400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc4bc102110 0x7fc4bc198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:46.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.802+0000 7fc4bb400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc4bc102110 0x7fc4bc198700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56082/0 (socket says 172.21.15.83:56082) 2024-04-24T08:51:46.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.802+0000 7fc4bb400640 1 -- 172.21.15.83:0/915514581 learned_addr learned my addr 172.21.15.83:0/915514581 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:51:46.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.802+0000 7fc4bbe00640 1 --2- 172.21.15.83:0/915514581 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc4bc19cf70 0x7fc4bc19d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:46.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.802+0000 7fc4bb400640 1 -- 172.21.15.83:0/915514581 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc4bc1071e0 msgr2=0x7fc4bc198c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:46.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.802+0000 7fc4baa00640 1 --2- 172.21.15.83:0/915514581 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc4bc1071e0 0x7fc4bc198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:46.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.802+0000 7fc4bb400640 1 --2- 172.21.15.83:0/915514581 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc4bc1071e0 0x7fc4bc198c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:46.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.803+0000 7fc4bb400640 1 -- 172.21.15.83:0/915514581 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc4bc19cf70 msgr2=0x7fc4bc19d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:46.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.803+0000 7fc4bb400640 1 --2- 172.21.15.83:0/915514581 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc4bc19cf70 0x7fc4bc19d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:46.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.803+0000 7fc4bb400640 1 -- 172.21.15.83:0/915514581 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4bc19da60 con 0x7fc4bc102110 2024-04-24T08:51:46.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.803+0000 7fc4bbe00640 1 --2- 172.21.15.83:0/915514581 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc4bc19cf70 0x7fc4bc19d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:51:46.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.803+0000 7fc4baa00640 1 --2- 172.21.15.83:0/915514581 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc4bc1071e0 0x7fc4bc198c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:51:46.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.803+0000 7fc4bb400640 1 --2- 172.21.15.83:0/915514581 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc4bc102110 0x7fc4bc198700 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac00d950 tx=0x7fc4ac00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:46.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.803+0000 7fc4b3e00640 1 -- 172.21.15.83:0/915514581 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4ac017070 con 0x7fc4bc102110 2024-04-24T08:51:46.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.803+0000 7fc4b3e00640 1 -- 172.21.15.83:0/915514581 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc4ac009c30 con 0x7fc4bc102110 2024-04-24T08:51:46.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.803+0000 7fc4c0e00640 1 -- 172.21.15.83:0/915514581 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4bc19dd50 con 0x7fc4bc102110 2024-04-24T08:51:46.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.803+0000 7fc4b3e00640 1 -- 172.21.15.83:0/915514581 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4ac00f620 con 0x7fc4bc102110 2024-04-24T08:51:46.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.803+0000 7fc4c0e00640 1 -- 172.21.15.83:0/915514581 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4bc1a5590 con 0x7fc4bc102110 2024-04-24T08:51:46.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.807+0000 7fc4b3e00640 1 -- 172.21.15.83:0/915514581 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7fc4ac0109a0 con 0x7fc4bc102110 2024-04-24T08:51:46.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.807+0000 7fc4c53bf740 1 Processor -- start 2024-04-24T08:51:46.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.807+0000 7fc4c53bf740 1 -- start start 2024-04-24T08:51:46.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.808+0000 7fc4c53bf740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fbc299be40 0x55fbc299c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:46.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.808+0000 7fc4c53bf740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fbc299c760 0x55fbc29a68f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:46.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.808+0000 7fc4c53bf740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbc29a6e30 0x55fbc29a9220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:46.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.808+0000 7fc4c53bf740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55fbc28fd1f0 con 0x55fbc29a6e30 2024-04-24T08:51:46.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.808+0000 7fc4c53bf740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55fbc28a3350 con 0x55fbc299c760 2024-04-24T08:51:46.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.808+0000 7fc4c53bf740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55fbc28f91b0 con 0x55fbc299be40 2024-04-24T08:51:46.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.808+0000 7fc4baa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbc29a6e30 0x55fbc29a9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:46.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.808+0000 7fc4baa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbc29a6e30 0x55fbc29a9220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56094/0 (socket says 172.21.15.83:56094) 2024-04-24T08:51:46.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.808+0000 7fc4baa00640 1 -- 172.21.15.83:0/738036307 learned_addr learned my addr 172.21.15.83:0/738036307 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:51:46.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.808+0000 7fc4bbe00640 1 --2- 172.21.15.83:0/738036307 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fbc299be40 0x55fbc299c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:46.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.808+0000 7fc4bb400640 1 --2- 172.21.15.83:0/738036307 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fbc299c760 0x55fbc29a68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:46.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.809+0000 7fc4baa00640 1 -- 172.21.15.83:0/738036307 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fbc299be40 msgr2=0x55fbc299c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:51:46.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.809+0000 7fc4baa00640 1 --2- 172.21.15.83:0/738036307 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fbc299be40 0x55fbc299c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:46.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.809+0000 7fc4baa00640 1 -- 172.21.15.83:0/738036307 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fbc299c760 msgr2=0x55fbc29a68f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:51:46.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.809+0000 7fc4baa00640 1 --2- 172.21.15.83:0/738036307 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fbc299c760 0x55fbc29a68f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:46.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.809+0000 7fc4baa00640 1 -- 172.21.15.83:0/738036307 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fbc28f7f50 con 0x55fbc29a6e30 2024-04-24T08:51:46.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.809+0000 7fc4b3e00640 1 --2- 172.21.15.83:0/915514581 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fc4a00a3e30 0x7fc4a00a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:46.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.809+0000 7fc4b3e00640 1 -- 172.21.15.83:0/915514581 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7fc4ac0d09e0 con 0x7fc4bc102110 2024-04-24T08:51:46.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.809+0000 7fc4bbe00640 1 --2- 172.21.15.83:0/915514581 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fc4a00a3e30 0x7fc4a00a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:46.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.809+0000 7fc4baa00640 1 --2- 172.21.15.83:0/738036307 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbc29a6e30 0x55fbc29a9220 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fc4a800ba70 tx=0x7fc4a800bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7934ded597516ec9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:46.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.810+0000 7fc4b0c00640 1 -- 172.21.15.83:0/738036307 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a8017420 con 0x55fbc29a6e30 2024-04-24T08:51:46.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.810+0000 7fc4b0c00640 1 -- 172.21.15.83:0/738036307 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc4a8017a70 con 0x55fbc29a6e30 2024-04-24T08:51:46.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.810+0000 7fc4b0c00640 1 -- 172.21.15.83:0/738036307 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a800fa90 con 0x55fbc29a6e30 2024-04-24T08:51:46.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.810+0000 7fc4c53bf740 1 -- 172.21.15.83:0/738036307 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbc29a6e30 msgr2=0x55fbc29a9220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:46.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.810+0000 7fc4bbe00640 1 --2- 172.21.15.83:0/915514581 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fc4a00a3e30 0x7fc4a00a62f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc4b4004af0 tx=0x7fc4b4004a40 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:46.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.810+0000 7fc4c53bf740 1 --2- 172.21.15.83:0/738036307 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbc29a6e30 0x55fbc29a9220 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fc4a800ba70 tx=0x7fc4a800bf30 comp rx=0 tx=0).stop 2024-04-24T08:51:46.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.811+0000 7fc4c53bf740 1 -- 172.21.15.83:0/738036307 shutdown_connections 2024-04-24T08:51:46.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.811+0000 7fc4c53bf740 1 --2- 172.21.15.83:0/738036307 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbc29a6e30 0x55fbc29a9220 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:46.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.811+0000 7fc4c53bf740 1 --2- 172.21.15.83:0/738036307 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fbc299c760 0x55fbc29a68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:46.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.811+0000 7fc4c53bf740 1 --2- 172.21.15.83:0/738036307 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fbc299be40 0x55fbc299c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:46.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.811+0000 7fc4c53bf740 1 -- 172.21.15.83:0/738036307 >> 172.21.15.83:0/738036307 conn(0x55fbc2998160 msgr2=0x55fbc2998540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:46.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.811+0000 7fc4c53bf740 1 -- 172.21.15.83:0/738036307 shutdown_connections 2024-04-24T08:51:46.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.811+0000 7fc4c53bf740 1 -- 172.21.15.83:0/738036307 wait complete. 2024-04-24T08:51:46.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.811+0000 7fc4c53bf740 1 -- start start 2024-04-24T08:51:46.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.811+0000 7fc4c53bf740 1 Processor -- start 2024-04-24T08:51:46.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.812+0000 7fc4c53bf740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fbc299be40 0x55fbc29a6860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:46.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.812+0000 7fc4c53bf740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbc299c760 0x55fbc29a3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:46.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.812+0000 7fc4c53bf740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fbc29a6e30 0x55fbc29a42f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:46.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.812+0000 7fc4c53bf740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55fbc29a9c20 con 0x55fbc299c760 2024-04-24T08:51:46.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.812+0000 7fc4c53bf740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55fbc28b6690 con 0x55fbc299be40 2024-04-24T08:51:46.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.812+0000 7fc4c53bf740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55fbc29a9da0 con 0x55fbc29a6e30 2024-04-24T08:51:46.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.812+0000 7fc4c53bf740 10 client.0 _ll_register_callbacks cb 0x55fbc28f2650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:51:46.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.812+0000 7fc4baa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbc299c760 0x55fbc29a3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:46.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.812+0000 7fc4baa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbc299c760 0x55fbc29a3db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56102/0 (socket says 172.21.15.83:56102) 2024-04-24T08:51:46.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.812+0000 7fc4baa00640 1 -- 172.21.15.83:0/495443623 learned_addr learned my addr 172.21.15.83:0/495443623 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:51:46.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.812+0000 7fc4bb400640 1 --2- 172.21.15.83:0/495443623 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fbc299be40 0x55fbc29a6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:46.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.813+0000 7fc4bbe00640 1 --2- 172.21.15.83:0/495443623 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fbc29a6e30 0x55fbc29a42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:46.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.813+0000 7fc4baa00640 1 -- 172.21.15.83:0/495443623 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fbc29a6e30 msgr2=0x55fbc29a42f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:51:46.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.813+0000 7fc4baa00640 1 --2- 172.21.15.83:0/495443623 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fbc29a6e30 0x55fbc29a42f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:46.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.813+0000 7fc4baa00640 1 -- 172.21.15.83:0/495443623 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fbc299be40 msgr2=0x55fbc29a6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:51:46.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.813+0000 7fc4baa00640 1 --2- 172.21.15.83:0/495443623 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fbc299be40 0x55fbc29a6860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:46.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.813+0000 7fc4baa00640 1 -- 172.21.15.83:0/495443623 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fbc28b5920 con 0x55fbc299c760 2024-04-24T08:51:46.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.813+0000 7fc4bbe00640 1 --2- 172.21.15.83:0/495443623 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fbc29a6e30 0x55fbc29a42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:51:46.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.813+0000 7fc4bb400640 1 --2- 172.21.15.83:0/495443623 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fbc299be40 0x55fbc29a6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:51:46.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.813+0000 7fc4baa00640 1 --2- 172.21.15.83:0/495443623 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbc299c760 0x55fbc29a3db0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fc4a800f710 tx=0x7fc4a800b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaac171fe596cb82 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:46.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.813+0000 7fc4a5600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:51:46.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.813+0000 7fc4a5600640 1 -- 172.21.15.83:0/495443623 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a8027020 con 0x55fbc299c760 2024-04-24T08:51:46.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.813+0000 7fc4a5600640 1 -- 172.21.15.83:0/495443623 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7fc4a8017830 con 0x55fbc299c760 2024-04-24T08:51:46.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.813+0000 7fc4c53bf740 1 -- 172.21.15.83:0/495443623 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4a8017a70 con 0x55fbc299c760 2024-04-24T08:51:46.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.814+0000 7fc4a5600640 1 -- 172.21.15.83:0/495443623 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc4a8032050 con 0x55fbc299c760 2024-04-24T08:51:46.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.814+0000 7fc4c53bf740 10 client.15813 fetch_fsmap learned FSMap version 28 2024-04-24T08:51:46.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.814+0000 7fc4c53bf740 1 -- 172.21.15.83:0/495443623 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55fbc26d97d0 con 0x55fbc299c760 2024-04-24T08:51:46.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.814+0000 7fc4a5600640 1 -- 172.21.15.83:0/495443623 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc4a8017a70 con 0x55fbc299c760 2024-04-24T08:51:46.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.814+0000 7fc4a5600640 1 -- 172.21.15.83:0/495443623 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc4a8023890 con 0x55fbc299c760 2024-04-24T08:51:46.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.814+0000 7fc4c53bf740 10 client.15813 fetch_fsmap finished waiting for FSMap version 28 2024-04-24T08:51:46.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.814+0000 7fc4c53bf740 10 client.15813 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:51:46.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.814+0000 7fc4c53bf740 20 client.15813 populate_metadata read hostname 'smithi083' 2024-04-24T08:51:46.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.814+0000 7fc4c53bf740 1 --2- 172.21.15.83:0/495443623 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55fbc29bb5d0 0x55fbc29bd9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:46.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.814+0000 7fc4c53bf740 4 client.15813 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:51:46.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.814+0000 7fc4c53bf740 1 -- 172.21.15.83:0/495443623 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fbc290ac10 con 0x55fbc29bb5d0 2024-04-24T08:51:46.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.815+0000 7fc4bb400640 1 --2- 172.21.15.83:0/495443623 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55fbc29bb5d0 0x55fbc29bd9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:46.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.816+0000 7fc4bb400640 1 --2- 172.21.15.83:0/495443623 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55fbc29bb5d0 0x55fbc29bd9c0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=621edfd2298d26 server_cookie=7a50112253eb612e in_seq=0 out_seq=0 2024-04-24T08:51:46.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.816+0000 7fc4a5600640 10 client.15813 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:51:46.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.818+0000 7fc4a5600640 1 -- 172.21.15.83:0/495443623 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fbc290ac10 con 0x55fbc29bb5d0 2024-04-24T08:51:46.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.818+0000 7fc4a5600640 10 client.15813 handle_command_reply: tid=0 2024-04-24T08:51:46.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.818+0000 7fc4c53bf740 1 client.15813 shutdown 2024-04-24T08:51:46.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.818+0000 7fc4c53bf740 1 -- 172.21.15.83:0/495443623 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbc299c760 msgr2=0x55fbc29a3db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:46.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.818+0000 7fc4c53bf740 1 --2- 172.21.15.83:0/495443623 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbc299c760 0x55fbc29a3db0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fc4a800f710 tx=0x7fc4a800b7c0 comp rx=0 tx=0).stop 2024-04-24T08:51:46.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.818+0000 7fc4c53bf740 1 -- 172.21.15.83:0/495443623 shutdown_connections 2024-04-24T08:51:46.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.819+0000 7fc4c53bf740 1 --2- 172.21.15.83:0/495443623 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55fbc29bb5d0 0x55fbc29bd9c0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:46.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.819+0000 7fc4c53bf740 1 --2- 172.21.15.83:0/495443623 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fbc29a6e30 0x55fbc29a42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:46.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.819+0000 7fc4c53bf740 1 --2- 172.21.15.83:0/495443623 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbc299c760 0x55fbc29a3db0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:46.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.819+0000 7fc4c53bf740 1 --2- 172.21.15.83:0/495443623 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fbc299be40 0x55fbc29a6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:46.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.819+0000 7fc4c53bf740 1 -- 172.21.15.83:0/495443623 >> 172.21.15.83:0/495443623 conn(0x55fbc2998160 msgr2=0x55fbc2992fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:46.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.819+0000 7fc4a5600640 0 client.15813 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:51:46.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.819+0000 7fc4c53bf740 1 -- 172.21.15.83:0/495443623 shutdown_connections 2024-04-24T08:51:46.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.819+0000 7fc4c53bf740 1 -- 172.21.15.83:0/495443623 wait complete. 2024-04-24T08:51:46.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.819+0000 7fc4c53bf740 20 client.15813 trim_cache size 0 max 16384 2024-04-24T08:51:46.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.840+0000 7fc4c53bf740 1 Processor -- start 2024-04-24T08:51:46.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.840+0000 7fc4c53bf740 1 -- start start 2024-04-24T08:51:46.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.841+0000 7fc4c53bf740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fbc299be40 0x55fbc299e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:46.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.841+0000 7fc4c53bf740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbc299c760 0x55fbc29c30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:46.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.841+0000 7fc4c53bf740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fbc29a6e30 0x55fbc29c35f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:46.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.841+0000 7fc4c53bf740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc4bc1ad0c0 con 0x55fbc299c760 2024-04-24T08:51:46.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.841+0000 7fc4c53bf740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc4bc1aec00 con 0x55fbc299be40 2024-04-24T08:51:46.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.841+0000 7fc4c53bf740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc4bc1ad790 con 0x55fbc29a6e30 2024-04-24T08:51:46.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.841+0000 7fc4baa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbc299c760 0x55fbc29c30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:46.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.841+0000 7fc4baa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbc299c760 0x55fbc29c30b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56116/0 (socket says 172.21.15.83:56116) 2024-04-24T08:51:46.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.841+0000 7fc4baa00640 1 -- 172.21.15.83:0/3978766744 learned_addr learned my addr 172.21.15.83:0/3978766744 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:51:46.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.841+0000 7fc4bbe00640 1 --2- 172.21.15.83:0/3978766744 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fbc29a6e30 0x55fbc29c35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:46.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.842+0000 7fc4bb400640 1 --2- 172.21.15.83:0/3978766744 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fbc299be40 0x55fbc299e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:46.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.842+0000 7fc4baa00640 1 -- 172.21.15.83:0/3978766744 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fbc29a6e30 msgr2=0x55fbc29c35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:51:46.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.842+0000 7fc4baa00640 1 --2- 172.21.15.83:0/3978766744 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fbc29a6e30 0x55fbc29c35f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:46.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.842+0000 7fc4baa00640 1 -- 172.21.15.83:0/3978766744 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fbc299be40 msgr2=0x55fbc299e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:51:46.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.842+0000 7fc4baa00640 1 --2- 172.21.15.83:0/3978766744 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fbc299be40 0x55fbc299e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:46.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.842+0000 7fc4baa00640 1 -- 172.21.15.83:0/3978766744 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fbc260f3f0 con 0x55fbc299c760 2024-04-24T08:51:46.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.842+0000 7fc4bb400640 1 --2- 172.21.15.83:0/3978766744 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fbc299be40 0x55fbc299e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:51:46.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.842+0000 7fc4baa00640 1 --2- 172.21.15.83:0/3978766744 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbc299c760 0x55fbc29c30b0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fc4a800bf70 tx=0x7fc4a800b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25d84f88a13dd69 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:46.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.842+0000 7fc4a7e00640 1 -- 172.21.15.83:0/3978766744 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a803b070 con 0x55fbc299c760 2024-04-24T08:51:46.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.842+0000 7fc4a7e00640 1 -- 172.21.15.83:0/3978766744 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc4a800dcb0 con 0x55fbc299c760 2024-04-24T08:51:46.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.842+0000 7fc4a7e00640 1 -- 172.21.15.83:0/3978766744 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a8027040 con 0x55fbc299c760 2024-04-24T08:51:46.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.843+0000 7fc4c53bf740 1 -- 172.21.15.83:0/3978766744 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbc299c760 msgr2=0x55fbc29c30b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:46.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.843+0000 7fc4c53bf740 1 --2- 172.21.15.83:0/3978766744 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbc299c760 0x55fbc29c30b0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fc4a800bf70 tx=0x7fc4a800b9a0 comp rx=0 tx=0).stop 2024-04-24T08:51:46.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.843+0000 7fc4c53bf740 1 -- 172.21.15.83:0/3978766744 shutdown_connections 2024-04-24T08:51:46.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.843+0000 7fc4c53bf740 1 --2- 172.21.15.83:0/3978766744 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fbc29a6e30 0x55fbc29c35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:46.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.843+0000 7fc4c53bf740 1 --2- 172.21.15.83:0/3978766744 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbc299c760 0x55fbc29c30b0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:46.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.843+0000 7fc4c53bf740 1 --2- 172.21.15.83:0/3978766744 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fbc299be40 0x55fbc299e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:46.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.843+0000 7fc4c53bf740 1 -- 172.21.15.83:0/3978766744 >> 172.21.15.83:0/3978766744 conn(0x55fbc2998160 msgr2=0x55fbc29a2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:46.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.843+0000 7fc4c53bf740 1 -- 172.21.15.83:0/3978766744 shutdown_connections 2024-04-24T08:51:46.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.843+0000 7fc4c53bf740 1 -- 172.21.15.83:0/3978766744 wait complete. 2024-04-24T08:51:46.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.844+0000 7fc4c53bf740 1 -- start start 2024-04-24T08:51:46.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.844+0000 7fc4c53bf740 1 Processor -- start 2024-04-24T08:51:46.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.844+0000 7fc4c53bf740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fbc299be40 0x55fbc29b7b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:46.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.844+0000 7fc4c53bf740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fbc299c760 0x55fbc29b8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:46.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.844+0000 7fc4c53bf740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbc29a6e30 0x55fbc29a06e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:46.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.844+0000 7fc4c53bf740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc4bc1ae530 con 0x55fbc29a6e30 2024-04-24T08:51:46.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.844+0000 7fc4c53bf740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc4bc1ade60 con 0x55fbc299be40 2024-04-24T08:51:46.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.844+0000 7fc4c53bf740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc4bc1ab0a0 con 0x55fbc299c760 2024-04-24T08:51:46.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.844+0000 7fc4c53bf740 10 client.0 _ll_register_callbacks cb 0x7fc4a800b8f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:51:46.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.845+0000 7fc4bbe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbc29a6e30 0x55fbc29a06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:46.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.845+0000 7fc4bbe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbc29a6e30 0x55fbc29a06e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56122/0 (socket says 172.21.15.83:56122) 2024-04-24T08:51:46.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.845+0000 7fc4baa00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fbc299c760 0x55fbc29b8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:46.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.845+0000 7fc4bbe00640 1 -- 172.21.15.83:0/2683852124 learned_addr learned my addr 172.21.15.83:0/2683852124 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:51:46.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.845+0000 7fc4bb400640 1 --2- 172.21.15.83:0/2683852124 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fbc299be40 0x55fbc29b7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:46.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.845+0000 7fc4bbe00640 1 -- 172.21.15.83:0/2683852124 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fbc299c760 msgr2=0x55fbc29b8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:51:46.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.845+0000 7fc4bbe00640 1 --2- 172.21.15.83:0/2683852124 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fbc299c760 0x55fbc29b8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:46.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.845+0000 7fc4bbe00640 1 -- 172.21.15.83:0/2683852124 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fbc299be40 msgr2=0x55fbc29b7b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:51:46.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.845+0000 7fc4bbe00640 1 --2- 172.21.15.83:0/2683852124 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fbc299be40 0x55fbc29b7b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:46.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.845+0000 7fc4bbe00640 1 -- 172.21.15.83:0/2683852124 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fbc25cf0f0 con 0x55fbc29a6e30 2024-04-24T08:51:46.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.845+0000 7fc4bb400640 1 --2- 172.21.15.83:0/2683852124 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fbc299be40 0x55fbc29b7b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:51:46.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.846+0000 7fc4bbe00640 1 --2- 172.21.15.83:0/2683852124 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbc29a6e30 0x55fbc29a06e0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fc4b4009ca0 tx=0x7fc4b40095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22b0ac8a6f5d24c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:46.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.846+0000 7fc4a5600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:51:46.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.846+0000 7fc4a5600640 1 -- 172.21.15.83:0/2683852124 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b4003e00 con 0x55fbc29a6e30 2024-04-24T08:51:46.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.846+0000 7fc4a5600640 1 -- 172.21.15.83:0/2683852124 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7fc4b40046a0 con 0x55fbc29a6e30 2024-04-24T08:51:46.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.846+0000 7fc4c53bf740 1 -- 172.21.15.83:0/2683852124 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4a800dcb0 con 0x55fbc29a6e30 2024-04-24T08:51:46.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.846+0000 7fc4a5600640 1 -- 172.21.15.83:0/2683852124 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc4b40305c0 con 0x55fbc29a6e30 2024-04-24T08:51:46.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.846+0000 7fc4c53bf740 10 client.15819 fetch_fsmap learned FSMap version 28 2024-04-24T08:51:46.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.846+0000 7fc4c53bf740 1 -- 172.21.15.83:0/2683852124 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55fbc28c2810 con 0x55fbc29a6e30 2024-04-24T08:51:46.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.846+0000 7fc4a5600640 1 -- 172.21.15.83:0/2683852124 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc4a800dcb0 con 0x55fbc29a6e30 2024-04-24T08:51:46.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.847+0000 7fc4a5600640 1 -- 172.21.15.83:0/2683852124 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc4b4043040 con 0x55fbc29a6e30 2024-04-24T08:51:46.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.847+0000 7fc4c53bf740 10 client.15819 fetch_fsmap finished waiting for FSMap version 28 2024-04-24T08:51:46.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.847+0000 7fc4c53bf740 10 client.15819 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:51:46.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.847+0000 7fc4c53bf740 20 client.15819 populate_metadata read hostname 'smithi083' 2024-04-24T08:51:46.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.847+0000 7fc4c53bf740 1 --2- 172.21.15.83:0/2683852124 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55fbc2a21730 0x55fbc2a23b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:46.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.847+0000 7fc4c53bf740 4 client.15819 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T08:51:46.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.847+0000 7fc4c53bf740 1 -- 172.21.15.83:0/2683852124 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55fbc2904a80 con 0x55fbc2a21730 2024-04-24T08:51:46.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.847+0000 7fc4bb400640 1 --2- 172.21.15.83:0/2683852124 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55fbc2a21730 0x55fbc2a23b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:46.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.848+0000 7fc4bb400640 1 --2- 172.21.15.83:0/2683852124 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55fbc2a21730 0x55fbc2a23b50 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d254cfce32dde3c8 server_cookie=362932e5bcf52357 in_seq=0 out_seq=0 2024-04-24T08:51:46.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.848+0000 7fc4a5600640 10 client.15819 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:51:46.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.849+0000 7fc4a5600640 1 -- 172.21.15.83:0/2683852124 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55fbc2904a80 con 0x55fbc2a21730 2024-04-24T08:51:46.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.849+0000 7fc4a5600640 10 client.15819 handle_command_reply: tid=0 2024-04-24T08:51:46.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.849+0000 7fc4c53bf740 1 client.15819 shutdown 2024-04-24T08:51:46.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.849+0000 7fc4c53bf740 1 -- 172.21.15.83:0/2683852124 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbc29a6e30 msgr2=0x55fbc29a06e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:46.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.849+0000 7fc4c53bf740 1 --2- 172.21.15.83:0/2683852124 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbc29a6e30 0x55fbc29a06e0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fc4b4009ca0 tx=0x7fc4b40095c0 comp rx=0 tx=0).stop 2024-04-24T08:51:46.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.849+0000 7fc4c53bf740 1 -- 172.21.15.83:0/2683852124 shutdown_connections 2024-04-24T08:51:46.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.849+0000 7fc4c53bf740 1 --2- 172.21.15.83:0/2683852124 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55fbc2a21730 0x55fbc2a23b50 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:46.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.849+0000 7fc4c53bf740 1 --2- 172.21.15.83:0/2683852124 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbc29a6e30 0x55fbc29a06e0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:46.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.849+0000 7fc4c53bf740 1 --2- 172.21.15.83:0/2683852124 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fbc299c760 0x55fbc29b8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:46.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.849+0000 7fc4c53bf740 1 --2- 172.21.15.83:0/2683852124 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fbc299be40 0x55fbc29b7b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:46.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.849+0000 7fc4c53bf740 1 -- 172.21.15.83:0/2683852124 >> 172.21.15.83:0/2683852124 conn(0x55fbc2998160 msgr2=0x55fbc299d590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:46.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.849+0000 7fc4a5600640 0 client.15819 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:51:46.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.849+0000 7fc4c53bf740 1 -- 172.21.15.83:0/2683852124 shutdown_connections 2024-04-24T08:51:46.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.849+0000 7fc4c53bf740 1 -- 172.21.15.83:0/2683852124 wait complete. 2024-04-24T08:51:46.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.849+0000 7fc4c53bf740 20 client.15819 trim_cache size 0 max 16384 2024-04-24T08:51:46.852 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T08:51:46.852 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T08:51:46.852 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-04-24T08:51:46.852 INFO:teuthology.orchestra.run.smithi083.stdout: "62d86377-f426-4a96-89d0-27abf9b5772e": { 2024-04-24T08:51:46.852 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-04-24T08:51:46.852 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "62d86377-f426-4a96-89d0-27abf9b5772e", 2024-04-24T08:51:46.852 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-04-24T08:51:46.852 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T08:51:46.852 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T08:51:46.852 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T08:51:46.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.850+0000 7fc4c0e00640 1 -- 172.21.15.83:0/915514581 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fc4a00a3e30 msgr2=0x7fc4a00a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:46.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.850+0000 7fc4c0e00640 1 --2- 172.21.15.83:0/915514581 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fc4a00a3e30 0x7fc4a00a62f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc4b4004af0 tx=0x7fc4b4004a40 comp rx=0 tx=0).stop 2024-04-24T08:51:46.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.850+0000 7fc4c0e00640 1 -- 172.21.15.83:0/915514581 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc4bc102110 msgr2=0x7fc4bc198700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:46.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.850+0000 7fc4c0e00640 1 --2- 172.21.15.83:0/915514581 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc4bc102110 0x7fc4bc198700 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac00d950 tx=0x7fc4ac00de10 comp rx=0 tx=0).stop 2024-04-24T08:51:46.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.850+0000 7fc4c0e00640 1 -- 172.21.15.83:0/915514581 shutdown_connections 2024-04-24T08:51:46.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.850+0000 7fc4c0e00640 1 --2- 172.21.15.83:0/915514581 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fc4a00a3e30 0x7fc4a00a62f0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:46.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.850+0000 7fc4c0e00640 1 --2- 172.21.15.83:0/915514581 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc4bc19cf70 0x7fc4bc19d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:46.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.850+0000 7fc4c0e00640 1 --2- 172.21.15.83:0/915514581 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc4bc1071e0 0x7fc4bc198c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:46.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.850+0000 7fc4c0e00640 1 --2- 172.21.15.83:0/915514581 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc4bc102110 0x7fc4bc198700 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:46.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.850+0000 7fc4c0e00640 1 -- 172.21.15.83:0/915514581 >> 172.21.15.83:0/915514581 conn(0x7fc4bc0fd9d0 msgr2=0x7fc4bc0fedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:46.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.850+0000 7fc4c0e00640 1 -- 172.21.15.83:0/915514581 shutdown_connections 2024-04-24T08:51:46.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.850+0000 7fc4c0e00640 1 -- 172.21.15.83:0/915514581 wait complete. 2024-04-24T08:51:46.867 INFO:tasks.cephfs.filesystem:scrub status for tag:62d86377-f426-4a96-89d0-27abf9b5772e - {'path': '/', 'tag': '62d86377-f426-4a96-89d0-27abf9b5772e', 'options': 'recursive,force'} 2024-04-24T08:51:46.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.949+0000 7fa986000640 1 -- 172.21.15.83:0/3677731376 <== mgr.15579 v2:172.21.15.83:6800/3594769126 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fa9880ff550 con 0x7fa9700a3ee0 2024-04-24T08:51:46.949 INFO:teuthology.orchestra.run.smithi083.stdout:Schedule set for path / 2024-04-24T08:51:46.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.953+0000 7fa990200640 1 -- 172.21.15.83:0/3677731376 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa9700a3ee0 msgr2=0x7fa9700a63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:46.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.953+0000 7fa990200640 1 --2- 172.21.15.83:0/3677731376 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa9700a3ee0 0x7fa9700a63a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa97c005fd0 tx=0x7fa97c005850 comp rx=0 tx=0).stop 2024-04-24T08:51:46.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.953+0000 7fa990200640 1 -- 172.21.15.83:0/3677731376 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa988074c20 msgr2=0x7fa9881a0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:46.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.953+0000 7fa990200640 1 --2- 172.21.15.83:0/3677731376 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa988074c20 0x7fa9881a0b00 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fa97400d770 tx=0x7fa97400dc30 comp rx=0 tx=0).stop 2024-04-24T08:51:46.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.953+0000 7fa990200640 1 -- 172.21.15.83:0/3677731376 shutdown_connections 2024-04-24T08:51:46.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.953+0000 7fa990200640 1 --2- 172.21.15.83:0/3677731376 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa9700a3ee0 0x7fa9700a63a0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:46.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.953+0000 7fa990200640 1 --2- 172.21.15.83:0/3677731376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa98810e8f0 0x7fa98819a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:46.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.953+0000 7fa990200640 1 --2- 172.21.15.83:0/3677731376 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa988074c20 0x7fa9881a0b00 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:46.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.953+0000 7fa990200640 1 --2- 172.21.15.83:0/3677731376 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa988074300 0x7fa9881a05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:46.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.953+0000 7fa990200640 1 -- 172.21.15.83:0/3677731376 >> 172.21.15.83:0/3677731376 conn(0x7fa988079490 msgr2=0x7fa9880fee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:46.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.953+0000 7fa990200640 1 -- 172.21.15.83:0/3677731376 shutdown_connections 2024-04-24T08:51:46.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:46.953+0000 7fa990200640 1 -- 172.21.15.83:0/3677731376 wait complete. 2024-04-24T08:51:46.970 DEBUG:teuthology.orchestra.run.smithi083:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-04-24T08:51:47.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:47.160+0000 7f41d9000640 1 -- 172.21.15.83:0/3345645416 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f41d4104c30 msgr2=0x7f41d4108c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:47.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:47.160+0000 7f41d9000640 1 --2- 172.21.15.83:0/3345645416 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f41d4104c30 0x7f41d4108c70 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f41bc009900 tx=0x7f41bc02a3f0 comp rx=0 tx=0).stop 2024-04-24T08:51:47.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:47.160+0000 7f41d9000640 1 -- 172.21.15.83:0/3345645416 shutdown_connections 2024-04-24T08:51:47.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:47.160+0000 7f41d9000640 1 --2- 172.21.15.83:0/3345645416 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f41d4109350 0x7f41d4110f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:47.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:47.160+0000 7f41d9000640 1 --2- 172.21.15.83:0/3345645416 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f41d4104c30 0x7f41d4108c70 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:47.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:47.160+0000 7f41d9000640 1 --2- 172.21.15.83:0/3345645416 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41d4104280 0x7f41d4104660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:47.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:47.160+0000 7f41d9000640 1 -- 172.21.15.83:0/3345645416 >> 172.21.15.83:0/3345645416 conn(0x7f41d40ffb20 msgr2=0x7f41d4101f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:47.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:47.161+0000 7f41d9000640 1 -- 172.21.15.83:0/3345645416 shutdown_connections 2024-04-24T08:51:47.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:47.161+0000 7f41d9000640 1 -- 172.21.15.83:0/3345645416 wait complete. 2024-04-24T08:51:47.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:47.162+0000 7f41d9000640 1 Processor -- start 2024-04-24T08:51:47.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:47.162+0000 7f41d9000640 1 -- start start 2024-04-24T08:51:47.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:47.162+0000 7f41d9000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41d4104280 0x7f41d41a0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:47.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:47.162+0000 7f41d9000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f41d4109350 0x7f41d41a0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:47.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:47.162+0000 7f41d9000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f41d419a5b0 0x7f41d419aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:47.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:47.162+0000 7f41d9000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f41d4114290 con 0x7f41d419a5b0 2024-04-24T08:51:47.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:47.162+0000 7f41d9000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f41d4114110 con 0x7f41d4104280 2024-04-24T08:51:47.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:47.162+0000 7f41d9000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f41d4114410 con 0x7f41d4109350 2024-04-24T08:51:47.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:47.162+0000 7f41d3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f41d419a5b0 0x7f41d419aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:47.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:47.162+0000 7f41d3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f41d419a5b0 0x7f41d419aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56148/0 (socket says 172.21.15.83:56148) 2024-04-24T08:51:47.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:47.162+0000 7f41d3e00640 1 -- 172.21.15.83:0/467676244 learned_addr learned my addr 172.21.15.83:0/467676244 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:51:47.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:47.163+0000 7f41cbe00640 1 --2- 172.21.15.83:0/467676244 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f41d4109350 0x7f41d41a0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:47.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:47.163+0000 7f41d3400640 1 --2- 172.21.15.83:0/467676244 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41d4104280 0x7f41d41a0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:47.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:47.163+0000 7f41d3e00640 1 -- 172.21.15.83:0/467676244 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f41d4109350 msgr2=0x7f41d41a0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:47.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:47.163+0000 7f41d3e00640 1 --2- 172.21.15.83:0/467676244 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f41d4109350 0x7f41d41a0940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:47.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:47.163+0000 7f41d3e00640 1 -- 172.21.15.83:0/467676244 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41d4104280 msgr2=0x7f41d41a0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:47.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:47.163+0000 7f41d3e00640 1 --2- 172.21.15.83:0/467676244 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41d4104280 0x7f41d41a0400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:47.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:47.163+0000 7f41d3e00640 1 -- 172.21.15.83:0/467676244 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41d419b0d0 con 0x7f41d419a5b0 2024-04-24T08:51:47.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:47.163+0000 7f41d3e00640 1 --2- 172.21.15.83:0/467676244 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f41d419a5b0 0x7f41d419aa60 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f41cc00eaa0 tx=0x7f41cc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:47.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:47.163+0000 7f41d0c00640 1 -- 172.21.15.83:0/467676244 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41cc00cdb0 con 0x7f41d419a5b0 2024-04-24T08:51:47.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:47.163+0000 7f41d0c00640 1 -- 172.21.15.83:0/467676244 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f41cc005020 con 0x7f41d419a5b0 2024-04-24T08:51:47.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:47.163+0000 7f41d9000640 1 -- 172.21.15.83:0/467676244 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41d419b3c0 con 0x7f41d419a5b0 2024-04-24T08:51:47.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:47.163+0000 7f41d0c00640 1 -- 172.21.15.83:0/467676244 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41cc0116d0 con 0x7f41d419a5b0 2024-04-24T08:51:47.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:47.163+0000 7f41d9000640 1 -- 172.21.15.83:0/467676244 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41d41a7660 con 0x7f41d419a5b0 2024-04-24T08:51:47.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:47.165+0000 7f41d9000640 1 -- 172.21.15.83:0/467676244 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41d40505e0 con 0x7f41d419a5b0 2024-04-24T08:51:47.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:47.167+0000 7f41d0c00640 1 -- 172.21.15.83:0/467676244 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f41cc0051d0 con 0x7f41d419a5b0 2024-04-24T08:51:47.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:47.168+0000 7f41d0c00640 1 --2- 172.21.15.83:0/467676244 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f41b80a3e20 0x7f41b80a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:47.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:47.168+0000 7f41d0c00640 1 -- 172.21.15.83:0/467676244 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f41cc0d0f30 con 0x7f41d419a5b0 2024-04-24T08:51:47.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:47.168+0000 7f41d3400640 1 --2- 172.21.15.83:0/467676244 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f41b80a3e20 0x7f41b80a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:47.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:47.169+0000 7f41d3400640 1 --2- 172.21.15.83:0/467676244 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f41b80a3e20 0x7f41b80a62e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f41c400ba40 tx=0x7f41c4005cc0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:47.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:47.172+0000 7f41d0c00640 1 -- 172.21.15.83:0/467676244 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f41cc016030 con 0x7f41d419a5b0 2024-04-24T08:51:47.187 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:47 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/982917112' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T08:51:47.187 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:47 smithi083 ceph-mon[27907]: scrub queued for path: / 2024-04-24T08:51:47.187 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:47 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T08:51:47.188 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:47 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T08:51:47.188 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:47 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T08:51:47.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:47.385+0000 7f41d9000640 1 -- 172.21.15.83:0/467676244 --> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7f41d4104c30 con 0x7f41b80a3e20 2024-04-24T08:51:47.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:47 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/982917112' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T08:51:47.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:47 smithi138 ceph-mon[31441]: scrub queued for path: / 2024-04-24T08:51:47.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:47 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T08:51:47.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:47 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T08:51:47.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:47 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T08:51:47.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:47 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/982917112' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T08:51:47.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:47 smithi189 ceph-mon[31791]: scrub queued for path: / 2024-04-24T08:51:47.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:47 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T08:51:47.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:47 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T08:51:47.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:47 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T08:51:47.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:47.551+0000 7f41d0c00640 1 -- 172.21.15.83:0/467676244 <== mgr.15579 v2:172.21.15.83:6800/3594769126 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7f41d4104c30 con 0x7f41b80a3e20 2024-04-24T08:51:47.551 INFO:teuthology.orchestra.run.smithi083.stdout:Retention added to path / 2024-04-24T08:51:47.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:47.556+0000 7f41d9000640 1 -- 172.21.15.83:0/467676244 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f41b80a3e20 msgr2=0x7f41b80a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:47.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:47.556+0000 7f41d9000640 1 --2- 172.21.15.83:0/467676244 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f41b80a3e20 0x7f41b80a62e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f41c400ba40 tx=0x7f41c4005cc0 comp rx=0 tx=0).stop 2024-04-24T08:51:47.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:47.556+0000 7f41d9000640 1 -- 172.21.15.83:0/467676244 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f41d419a5b0 msgr2=0x7f41d419aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:47.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:47.556+0000 7f41d9000640 1 --2- 172.21.15.83:0/467676244 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f41d419a5b0 0x7f41d419aa60 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f41cc00eaa0 tx=0x7f41cc00ef60 comp rx=0 tx=0).stop 2024-04-24T08:51:47.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:47.556+0000 7f41d9000640 1 -- 172.21.15.83:0/467676244 shutdown_connections 2024-04-24T08:51:47.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:47.556+0000 7f41d9000640 1 --2- 172.21.15.83:0/467676244 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f41b80a3e20 0x7f41b80a62e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:47.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:47.556+0000 7f41d9000640 1 --2- 172.21.15.83:0/467676244 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f41d419a5b0 0x7f41d419aa60 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:47.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:47.556+0000 7f41d9000640 1 --2- 172.21.15.83:0/467676244 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f41d4109350 0x7f41d41a0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:47.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:47.556+0000 7f41d9000640 1 --2- 172.21.15.83:0/467676244 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41d4104280 0x7f41d41a0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:47.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:47.556+0000 7f41d9000640 1 -- 172.21.15.83:0/467676244 >> 172.21.15.83:0/467676244 conn(0x7f41d40ffb20 msgr2=0x7f41d4101e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:47.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:47.556+0000 7f41d9000640 1 -- 172.21.15.83:0/467676244 shutdown_connections 2024-04-24T08:51:47.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:47.556+0000 7f41d9000640 1 -- 172.21.15.83:0/467676244 wait complete. 2024-04-24T08:51:47.575 DEBUG:teuthology.orchestra.run.smithi083:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-24T08:51:47.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:47.770+0000 7f358dc00640 1 -- 172.21.15.83:0/2711306842 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3588112ce0 msgr2=0x7f35881150d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:47.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:47.770+0000 7f358dc00640 1 --2- 172.21.15.83:0/2711306842 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3588112ce0 0x7f35881150d0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f3580009ca0 tx=0x7f358002a8b0 comp rx=0 tx=0).stop 2024-04-24T08:51:47.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:47.770+0000 7f358dc00640 1 -- 172.21.15.83:0/2711306842 shutdown_connections 2024-04-24T08:51:47.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:47.770+0000 7f358dc00640 1 --2- 172.21.15.83:0/2711306842 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3588112ce0 0x7f35881150d0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:47.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:47.770+0000 7f358dc00640 1 --2- 172.21.15.83:0/2711306842 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3588076e50 0x7f35881127a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:47.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:47.770+0000 7f358dc00640 1 --2- 172.21.15.83:0/2711306842 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3588076530 0x7f3588076910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:47.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:47.770+0000 7f358dc00640 1 -- 172.21.15.83:0/2711306842 >> 172.21.15.83:0/2711306842 conn(0x7f35880ffb10 msgr2=0x7f3588101f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:47.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:47.770+0000 7f358dc00640 1 -- 172.21.15.83:0/2711306842 shutdown_connections 2024-04-24T08:51:47.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:47.770+0000 7f358dc00640 1 -- 172.21.15.83:0/2711306842 wait complete. 2024-04-24T08:51:47.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:47.771+0000 7f358dc00640 1 Processor -- start 2024-04-24T08:51:47.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:47.771+0000 7f358dc00640 1 -- start start 2024-04-24T08:51:47.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:47.771+0000 7f358dc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3588076530 0x7f35881a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:47.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:47.771+0000 7f358dc00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3588076e50 0x7f35881a4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:47.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:47.771+0000 7f358dc00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3588112ce0 0x7f358819e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:47.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:47.771+0000 7f358dc00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3588118700 con 0x7f3588076530 2024-04-24T08:51:47.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:47.771+0000 7f358dc00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3588118580 con 0x7f3588112ce0 2024-04-24T08:51:47.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:47.771+0000 7f358dc00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3588118880 con 0x7f3588076e50 2024-04-24T08:51:47.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:47.772+0000 7f3587e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3588076530 0x7f35881a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:47.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:47.772+0000 7f3587e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3588076530 0x7f35881a4770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56160/0 (socket says 172.21.15.83:56160) 2024-04-24T08:51:47.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:47.772+0000 7f3587e00640 1 -- 172.21.15.83:0/2878773851 learned_addr learned my addr 172.21.15.83:0/2878773851 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:51:47.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:47.772+0000 7f3587400640 1 --2- 172.21.15.83:0/2878773851 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3588076e50 0x7f35881a4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:47.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:47.772+0000 7f3587e00640 1 -- 172.21.15.83:0/2878773851 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3588076e50 msgr2=0x7f35881a4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:47.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:47.772+0000 7f3587e00640 1 --2- 172.21.15.83:0/2878773851 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3588076e50 0x7f35881a4cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:47.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:47.772+0000 7f358d200640 1 --2- 172.21.15.83:0/2878773851 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3588112ce0 0x7f358819e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:47.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:47.772+0000 7f3587e00640 1 -- 172.21.15.83:0/2878773851 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3588112ce0 msgr2=0x7f358819e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:47.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:47.772+0000 7f3587e00640 1 --2- 172.21.15.83:0/2878773851 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3588112ce0 0x7f358819e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:47.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:47.772+0000 7f3587e00640 1 -- 172.21.15.83:0/2878773851 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f358819f0a0 con 0x7f3588076530 2024-04-24T08:51:47.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:47.772+0000 7f3587400640 1 --2- 172.21.15.83:0/2878773851 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3588076e50 0x7f35881a4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:51:47.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:47.772+0000 7f3587e00640 1 --2- 172.21.15.83:0/2878773851 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3588076530 0x7f35881a4770 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f357800c920 tx=0x7f357800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:47.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:47.773+0000 7f3584c00640 1 -- 172.21.15.83:0/2878773851 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3578007c70 con 0x7f3588076530 2024-04-24T08:51:47.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:47.773+0000 7f3584c00640 1 -- 172.21.15.83:0/2878773851 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3578007e10 con 0x7f3588076530 2024-04-24T08:51:47.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:47.773+0000 7f3584c00640 1 -- 172.21.15.83:0/2878773851 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3578018820 con 0x7f3588076530 2024-04-24T08:51:47.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:47.773+0000 7f358dc00640 1 -- 172.21.15.83:0/2878773851 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f358819f390 con 0x7f3588076530 2024-04-24T08:51:47.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:47.773+0000 7f358dc00640 1 -- 172.21.15.83:0/2878773851 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35881ab630 con 0x7f3588076530 2024-04-24T08:51:47.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:47.774+0000 7f358dc00640 1 -- 172.21.15.83:0/2878773851 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35880505e0 con 0x7f3588076530 2024-04-24T08:51:47.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:47.776+0000 7f3584c00640 1 -- 172.21.15.83:0/2878773851 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f35780189c0 con 0x7f3588076530 2024-04-24T08:51:47.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:47.777+0000 7f3584c00640 1 --2- 172.21.15.83:0/2878773851 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f356c0a3da0 0x7f356c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:47.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:47.777+0000 7f3584c00640 1 -- 172.21.15.83:0/2878773851 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f35780d0fd0 con 0x7f3588076530 2024-04-24T08:51:47.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:47.777+0000 7f3587400640 1 --2- 172.21.15.83:0/2878773851 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f356c0a3da0 0x7f356c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:47.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:47.778+0000 7f3587400640 1 --2- 172.21.15.83:0/2878773851 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f356c0a3da0 0x7f356c0a6260 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3588103560 tx=0x7f3574005d20 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:47.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:47.782+0000 7f3584c00640 1 -- 172.21.15.83:0/2878773851 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3578014030 con 0x7f3588076530 2024-04-24T08:51:48.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:48.009+0000 7f358dc00640 1 -- 172.21.15.83:0/2878773851 --> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f3588114e40 con 0x7f356c0a3da0 2024-04-24T08:51:48.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:48.024+0000 7f3584c00640 1 -- 172.21.15.83:0/2878773851 <== mgr.15579 v2:172.21.15.83:6800/3594769126 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7f3588114e40 con 0x7f356c0a3da0 2024-04-24T08:51:48.023 INFO:teuthology.orchestra.run.smithi083.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-24T00:00:00", "created": "2024-04-24T08:51:46", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-04-24T08:51:48.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:48.027+0000 7f358dc00640 1 -- 172.21.15.83:0/2878773851 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f356c0a3da0 msgr2=0x7f356c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:48.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:48.027+0000 7f358dc00640 1 --2- 172.21.15.83:0/2878773851 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f356c0a3da0 0x7f356c0a6260 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3588103560 tx=0x7f3574005d20 comp rx=0 tx=0).stop 2024-04-24T08:51:48.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:48.027+0000 7f358dc00640 1 -- 172.21.15.83:0/2878773851 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3588076530 msgr2=0x7f35881a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:48.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:48.027+0000 7f358dc00640 1 --2- 172.21.15.83:0/2878773851 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3588076530 0x7f35881a4770 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f357800c920 tx=0x7f357800cde0 comp rx=0 tx=0).stop 2024-04-24T08:51:48.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:48.027+0000 7f358dc00640 1 -- 172.21.15.83:0/2878773851 shutdown_connections 2024-04-24T08:51:48.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:48.027+0000 7f358dc00640 1 --2- 172.21.15.83:0/2878773851 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f356c0a3da0 0x7f356c0a6260 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:48.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:48.027+0000 7f358dc00640 1 --2- 172.21.15.83:0/2878773851 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3588112ce0 0x7f358819e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:48.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:48.027+0000 7f358dc00640 1 --2- 172.21.15.83:0/2878773851 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3588076e50 0x7f35881a4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:48.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:48.027+0000 7f358dc00640 1 --2- 172.21.15.83:0/2878773851 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3588076530 0x7f35881a4770 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:48.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:48.027+0000 7f358dc00640 1 -- 172.21.15.83:0/2878773851 >> 172.21.15.83:0/2878773851 conn(0x7f35880ffb10 msgr2=0x7f3588101f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:48.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:48.027+0000 7f358dc00640 1 -- 172.21.15.83:0/2878773851 shutdown_connections 2024-04-24T08:51:48.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:48.027+0000 7f358dc00640 1 -- 172.21.15.83:0/2878773851 wait complete. 2024-04-24T08:51:48.046 DEBUG:teuthology.orchestra.run.smithi083:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-24T08:51:48.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:48.245+0000 7fa09c600640 1 -- 172.21.15.83:0/1099091930 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa094102c30 msgr2=0x7fa094106c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:48.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:48.245+0000 7fa09c600640 1 --2- 172.21.15.83:0/1099091930 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa094102c30 0x7fa094106c70 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fa080009900 tx=0x7fa08002a410 comp rx=0 tx=0).stop 2024-04-24T08:51:48.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:48.245+0000 7fa09c600640 1 -- 172.21.15.83:0/1099091930 shutdown_connections 2024-04-24T08:51:48.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:48.245+0000 7fa09c600640 1 --2- 172.21.15.83:0/1099091930 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa094107350 0x7fa09410ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:48.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:48.245+0000 7fa09c600640 1 --2- 172.21.15.83:0/1099091930 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa094102c30 0x7fa094106c70 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:48.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:48.245+0000 7fa09c600640 1 --2- 172.21.15.83:0/1099091930 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa094102280 0x7fa094102660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:48.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:48.245+0000 7fa09c600640 1 -- 172.21.15.83:0/1099091930 >> 172.21.15.83:0/1099091930 conn(0x7fa094079490 msgr2=0x7fa0940798a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:48.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:48.245+0000 7fa09c600640 1 -- 172.21.15.83:0/1099091930 shutdown_connections 2024-04-24T08:51:48.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:48.245+0000 7fa09c600640 1 -- 172.21.15.83:0/1099091930 wait complete. 2024-04-24T08:51:48.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:48.246+0000 7fa09c600640 1 Processor -- start 2024-04-24T08:51:48.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:48.246+0000 7fa09c600640 1 -- start start 2024-04-24T08:51:48.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:48.247+0000 7fa09c600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa094102280 0x7fa0941a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:48.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:48.247+0000 7fa09c600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa094107350 0x7fa0941a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:48.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:48.247+0000 7fa09c600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa09419a790 0x7fa09419ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:48.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:48.247+0000 7fa09c600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa0941122e0 con 0x7fa094107350 2024-04-24T08:51:48.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:48.247+0000 7fa09c600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa094112160 con 0x7fa09419a790 2024-04-24T08:51:48.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:48.247+0000 7fa09c600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa094112460 con 0x7fa094102280 2024-04-24T08:51:48.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:48.247+0000 7fa099200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa094107350 0x7fa0941a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:48.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:48.247+0000 7fa099200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa094107350 0x7fa0941a0b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56178/0 (socket says 172.21.15.83:56178) 2024-04-24T08:51:48.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:48.247+0000 7fa099200640 1 -- 172.21.15.83:0/4038207500 learned_addr learned my addr 172.21.15.83:0/4038207500 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:51:48.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:48.247+0000 7fa099c00640 1 --2- 172.21.15.83:0/4038207500 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa094102280 0x7fa0941a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:48.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:48.248+0000 7fa09a600640 1 --2- 172.21.15.83:0/4038207500 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa09419a790 0x7fa09419ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:48.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:48.248+0000 7fa099200640 1 -- 172.21.15.83:0/4038207500 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa094102280 msgr2=0x7fa0941a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:48.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:48.248+0000 7fa099200640 1 --2- 172.21.15.83:0/4038207500 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa094102280 0x7fa0941a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:48.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:48.248+0000 7fa099200640 1 -- 172.21.15.83:0/4038207500 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa09419a790 msgr2=0x7fa09419ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:48.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:48.248+0000 7fa099200640 1 --2- 172.21.15.83:0/4038207500 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa09419a790 0x7fa09419ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:48.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:48.248+0000 7fa099200640 1 -- 172.21.15.83:0/4038207500 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa09419b320 con 0x7fa094107350 2024-04-24T08:51:48.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:48.248+0000 7fa099c00640 1 --2- 172.21.15.83:0/4038207500 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa094102280 0x7fa0941a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:51:48.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:48.248+0000 7fa09a600640 1 --2- 172.21.15.83:0/4038207500 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa09419a790 0x7fa09419ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:51:48.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:48.248+0000 7fa099200640 1 --2- 172.21.15.83:0/4038207500 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa094107350 0x7fa0941a0b60 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fa080005ab0 tx=0x7fa080003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:48.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:48.248+0000 7fa092000640 1 -- 172.21.15.83:0/4038207500 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa08003f070 con 0x7fa094107350 2024-04-24T08:51:48.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:48.248+0000 7fa092000640 1 -- 172.21.15.83:0/4038207500 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa080031d60 con 0x7fa094107350 2024-04-24T08:51:48.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:48.248+0000 7fa092000640 1 -- 172.21.15.83:0/4038207500 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa080035030 con 0x7fa094107350 2024-04-24T08:51:48.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:48.248+0000 7fa09c600640 1 -- 172.21.15.83:0/4038207500 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa09419b5b0 con 0x7fa094107350 2024-04-24T08:51:48.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:48.248+0000 7fa09c600640 1 -- 172.21.15.83:0/4038207500 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0941a7760 con 0x7fa094107350 2024-04-24T08:51:48.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:48.250+0000 7fa09c600640 1 -- 172.21.15.83:0/4038207500 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa094103750 con 0x7fa094107350 2024-04-24T08:51:48.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:48.252+0000 7fa092000640 1 -- 172.21.15.83:0/4038207500 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7fa080042070 con 0x7fa094107350 2024-04-24T08:51:48.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:48.253+0000 7fa092000640 1 --2- 172.21.15.83:0/4038207500 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa07c0a3e60 0x7fa07c0a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:51:48.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:48.253+0000 7fa092000640 1 -- 172.21.15.83:0/4038207500 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7fa0800f1300 con 0x7fa094107350 2024-04-24T08:51:48.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:48.253+0000 7fa099c00640 1 --2- 172.21.15.83:0/4038207500 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa07c0a3e60 0x7fa07c0a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:51:48.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:48.254+0000 7fa099c00640 1 --2- 172.21.15.83:0/4038207500 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa07c0a3e60 0x7fa07c0a6320 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa088009b40 tx=0x7fa088005e50 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:51:48.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:48.257+0000 7fa092000640 1 -- 172.21.15.83:0/4038207500 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa0800bd400 con 0x7fa094107350 2024-04-24T08:51:48.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:48 smithi083 ceph-mon[27907]: pgmap v8: 129 pgs: 129 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 0 B/s wr, 19 op/s 2024-04-24T08:51:48.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:48 smithi083 ceph-mon[27907]: from='client.15786 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:51:48.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:48 smithi083 ceph-mon[27907]: from='client.15825 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:51:48.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:48 smithi083 ceph-mon[27907]: from='client.15831 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:51:48.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:48 smithi138 ceph-mon[31441]: pgmap v8: 129 pgs: 129 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 0 B/s wr, 19 op/s 2024-04-24T08:51:48.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:48 smithi138 ceph-mon[31441]: from='client.15786 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:51:48.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:48 smithi138 ceph-mon[31441]: from='client.15825 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:51:48.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:48 smithi138 ceph-mon[31441]: from='client.15831 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:51:48.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:48 smithi189 ceph-mon[31791]: pgmap v8: 129 pgs: 129 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 0 B/s wr, 19 op/s 2024-04-24T08:51:48.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:48 smithi189 ceph-mon[31791]: from='client.15786 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:51:48.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:48 smithi189 ceph-mon[31791]: from='client.15825 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:51:48.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:48 smithi189 ceph-mon[31791]: from='client.15831 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:51:48.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:48.483+0000 7fa09c600640 1 -- 172.21.15.83:0/4038207500 --> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7fa094102ca0 con 0x7fa07c0a3e60 2024-04-24T08:51:48.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:48.498+0000 7fa092000640 1 -- 172.21.15.83:0/4038207500 <== mgr.15579 v2:172.21.15.83:6800/3594769126 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7fa094102ca0 con 0x7fa07c0a3e60 2024-04-24T08:51:48.498 INFO:teuthology.orchestra.run.smithi083.stdout:/ 1m 6m3h 2024-04-24T08:51:48.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:48.501+0000 7fa09c600640 1 -- 172.21.15.83:0/4038207500 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa07c0a3e60 msgr2=0x7fa07c0a6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:48.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:48.501+0000 7fa09c600640 1 --2- 172.21.15.83:0/4038207500 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa07c0a3e60 0x7fa07c0a6320 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa088009b40 tx=0x7fa088005e50 comp rx=0 tx=0).stop 2024-04-24T08:51:48.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:48.501+0000 7fa09c600640 1 -- 172.21.15.83:0/4038207500 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa094107350 msgr2=0x7fa0941a0b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:51:48.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:48.501+0000 7fa09c600640 1 --2- 172.21.15.83:0/4038207500 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa094107350 0x7fa0941a0b60 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fa080005ab0 tx=0x7fa080003cd0 comp rx=0 tx=0).stop 2024-04-24T08:51:48.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:48.501+0000 7fa09c600640 1 -- 172.21.15.83:0/4038207500 shutdown_connections 2024-04-24T08:51:48.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:48.501+0000 7fa09c600640 1 --2- 172.21.15.83:0/4038207500 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa07c0a3e60 0x7fa07c0a6320 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:48.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:48.501+0000 7fa09c600640 1 --2- 172.21.15.83:0/4038207500 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa09419a790 0x7fa09419ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:48.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:48.501+0000 7fa09c600640 1 --2- 172.21.15.83:0/4038207500 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa094107350 0x7fa0941a0b60 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:48.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:48.501+0000 7fa09c600640 1 --2- 172.21.15.83:0/4038207500 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa094102280 0x7fa0941a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:51:48.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:48.501+0000 7fa09c600640 1 -- 172.21.15.83:0/4038207500 >> 172.21.15.83:0/4038207500 conn(0x7fa094079490 msgr2=0x7fa0940fff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:51:48.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:48.501+0000 7fa09c600640 1 -- 172.21.15.83:0/4038207500 shutdown_connections 2024-04-24T08:51:48.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:51:48.501+0000 7fa09c600640 1 -- 172.21.15.83:0/4038207500 wait complete. 2024-04-24T08:51:48.520 DEBUG:teuthology.orchestra.run.smithi083:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-04-24T08:51:48.609 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-04-24T08:51:48.619 INFO:teuthology.run_tasks:Running task background_exec... 2024-04-24T08:51:48.630 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi083.front.sepia.ceph.com 2024-04-24T08:51:48.630 DEBUG:teuthology.orchestra.run.smithi083:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-04-24T08:51:48.637 INFO:teuthology.run_tasks:Running task workunit... 2024-04-24T08:51:48.650 INFO:tasks.workunit:Pulling workunits from ref 1f81fb397ae98da7563d451a78c61574c8f4e6e0 2024-04-24T08:51:48.650 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-24T08:51:48.650 INFO:tasks.workunit:timeout=3h 2024-04-24T08:51:48.651 INFO:tasks.workunit:cleanup=True 2024-04-24T08:51:48.651 DEBUG:teuthology.orchestra.run.smithi083:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-24T08:51:48.701 INFO:teuthology.orchestra.run.smithi083.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-04-24T08:51:48.702 INFO:teuthology.orchestra.run.smithi083.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-04-24T08:51:48.702 INFO:teuthology.orchestra.run.smithi083.stdout:Device: 72h/114d Inode: 1 Links: 2 2024-04-24T08:51:48.702 INFO:teuthology.orchestra.run.smithi083.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T08:51:48.702 INFO:teuthology.orchestra.run.smithi083.stdout:Context: system_u:object_r:fusefs_t:s0 2024-04-24T08:51:48.702 INFO:teuthology.orchestra.run.smithi083.stdout:Access: 2024-04-24 08:49:04.052478823 +0000 2024-04-24T08:51:48.702 INFO:teuthology.orchestra.run.smithi083.stdout:Modify: 2024-04-24 08:49:04.052478823 +0000 2024-04-24T08:51:48.702 INFO:teuthology.orchestra.run.smithi083.stdout:Change: 2024-04-24 08:51:35.892235510 +0000 2024-04-24T08:51:48.702 INFO:teuthology.orchestra.run.smithi083.stdout: Birth: - 2024-04-24T08:51:48.702 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-04-24T08:51:48.702 DEBUG:teuthology.orchestra.run.smithi083:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-04-24T08:51:48.787 DEBUG:teuthology.orchestra.run.smithi083:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 1f81fb397ae98da7563d451a78c61574c8f4e6e0 2024-04-24T08:51:48.840 INFO:tasks.workunit.client.0.smithi083.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-24T08:51:49.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:49 smithi138 ceph-mon[31441]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 60 KiB/s wr, 36 op/s 2024-04-24T08:51:49.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:49 smithi138 ceph-mon[31441]: from='client.15837 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:51:49.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:49 smithi189 ceph-mon[31791]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 60 KiB/s wr, 36 op/s 2024-04-24T08:51:49.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:49 smithi189 ceph-mon[31791]: from='client.15837 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:51:49.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:49 smithi083 ceph-mon[27907]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 60 KiB/s wr, 36 op/s 2024-04-24T08:51:49.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:49 smithi083 ceph-mon[27907]: from='client.15837 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T08:51:51.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:51 smithi138 ceph-mon[31441]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 54 KiB/s wr, 33 op/s 2024-04-24T08:51:51.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:51 smithi189 ceph-mon[31791]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 54 KiB/s wr, 33 op/s 2024-04-24T08:51:51.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:51 smithi083 ceph-mon[27907]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 54 KiB/s wr, 33 op/s 2024-04-24T08:51:53.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:53 smithi138 ceph-mon[31441]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 54 KiB/s wr, 33 op/s 2024-04-24T08:51:53.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:53 smithi189 ceph-mon[31791]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 54 KiB/s wr, 33 op/s 2024-04-24T08:51:53.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:53 smithi083 ceph-mon[27907]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 54 KiB/s wr, 33 op/s 2024-04-24T08:51:55.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:55 smithi083 ceph-mon[27907]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 77 KiB/s wr, 55 op/s 2024-04-24T08:51:55.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:55 smithi138 ceph-mon[31441]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 77 KiB/s wr, 55 op/s 2024-04-24T08:51:55.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:55 smithi189 ceph-mon[31791]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 77 KiB/s wr, 55 op/s 2024-04-24T08:51:57.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:57 smithi083 ceph-mon[27907]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 40 op/s 2024-04-24T08:51:57.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:57 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T08:51:57.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:57 smithi138 ceph-mon[31441]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 40 op/s 2024-04-24T08:51:57.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:57 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T08:51:57.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:57 smithi189 ceph-mon[31791]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 40 op/s 2024-04-24T08:51:57.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:57 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T08:51:59.759 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:51:59 smithi189 ceph-mon[31791]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 78 KiB/s wr, 45 op/s 2024-04-24T08:51:59.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:51:59 smithi083 ceph-mon[27907]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 78 KiB/s wr, 45 op/s 2024-04-24T08:51:59.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:51:59 smithi138 ceph-mon[31441]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 78 KiB/s wr, 45 op/s 2024-04-24T08:52:01.053 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:52:00 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:52:00.573+0000 7f3abde00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T08:52:01.053 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:52:00 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:52:00.573+0000 7f3abde00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T08:52:01.054 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:52:00 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:52:00.573+0000 7f3abde00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T08:52:01.054 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:52:00 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:52:00.573+0000 7f3abde00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T08:52:01.054 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 08:52:00 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T08:52:00.573+0000 7f3abde00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T08:52:01.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:52:01 smithi189 ceph-mon[31791]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 25 op/s 2024-04-24T08:52:01.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:52:01 smithi083 ceph-mon[27907]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 25 op/s 2024-04-24T08:52:01.879 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.878+0000 7f8196a00640 1 -- 172.21.15.83:0/890183925 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8190104e20 msgr2=0x7f8190108e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:01.879 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.878+0000 7f8196a00640 1 --2- 172.21.15.83:0/890183925 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8190104e20 0x7f8190108e60 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f8178009900 tx=0x7f817802a3f0 comp rx=0 tx=0).stop 2024-04-24T08:52:01.879 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.878+0000 7f8196a00640 1 -- 172.21.15.83:0/890183925 shutdown_connections 2024-04-24T08:52:01.879 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.878+0000 7f8196a00640 1 --2- 172.21.15.83:0/890183925 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8190109540 0x7f8190111110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:01.879 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.878+0000 7f8196a00640 1 --2- 172.21.15.83:0/890183925 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8190104e20 0x7f8190108e60 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:01.879 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.878+0000 7f8196a00640 1 --2- 172.21.15.83:0/890183925 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8190104470 0x7f8190104850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:01.879 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.878+0000 7f8196a00640 1 -- 172.21.15.83:0/890183925 >> 172.21.15.83:0/890183925 conn(0x7f81900ffd30 msgr2=0x7f8190102150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:01.879 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.878+0000 7f8196a00640 1 -- 172.21.15.83:0/890183925 shutdown_connections 2024-04-24T08:52:01.879 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.878+0000 7f8196a00640 1 -- 172.21.15.83:0/890183925 wait complete. 2024-04-24T08:52:01.882 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.879+0000 7f8196a00640 1 Processor -- start 2024-04-24T08:52:01.882 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.879+0000 7f8196a00640 1 -- start start 2024-04-24T08:52:01.882 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.880+0000 7f8196a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8190104470 0x7f81901a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:01.882 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.880+0000 7f8196a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8190109540 0x7f81901a0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:01.882 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.880+0000 7f8196a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f819019a700 0x7f819019ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:01.882 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.880+0000 7f8196a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8190114460 con 0x7f819019a700 2024-04-24T08:52:01.882 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.880+0000 7f8196a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f81901142e0 con 0x7f8190104470 2024-04-24T08:52:01.882 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.880+0000 7f8196a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f81901145e0 con 0x7f8190109540 2024-04-24T08:52:01.882 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.880+0000 7f8194a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f819019a700 0x7f819019ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:01.882 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.880+0000 7f8194a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f819019a700 0x7f819019ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44950/0 (socket says 172.21.15.83:44950) 2024-04-24T08:52:01.882 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.880+0000 7f8194a00640 1 -- 172.21.15.83:0/1421669390 learned_addr learned my addr 172.21.15.83:0/1421669390 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:52:01.882 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.880+0000 7f818fe00640 1 --2- 172.21.15.83:0/1421669390 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8190104470 0x7f81901a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:01.882 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.880+0000 7f8194a00640 1 -- 172.21.15.83:0/1421669390 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8190109540 msgr2=0x7f81901a0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:01.883 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.880+0000 7f8194a00640 1 --2- 172.21.15.83:0/1421669390 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8190109540 0x7f81901a0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:01.883 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.880+0000 7f8194a00640 1 -- 172.21.15.83:0/1421669390 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8190104470 msgr2=0x7f81901a0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:01.883 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.880+0000 7f8194a00640 1 --2- 172.21.15.83:0/1421669390 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8190104470 0x7f81901a0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:01.883 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.880+0000 7f8194a00640 1 -- 172.21.15.83:0/1421669390 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f819019b300 con 0x7f819019a700 2024-04-24T08:52:01.883 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.881+0000 7f8194a00640 1 --2- 172.21.15.83:0/1421669390 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f819019a700 0x7f819019ab80 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f818800cce0 tx=0x7f818800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:01.883 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.881+0000 7f818cc00640 1 -- 172.21.15.83:0/1421669390 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f818800f450 con 0x7f819019a700 2024-04-24T08:52:01.883 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.881+0000 7f818cc00640 1 -- 172.21.15.83:0/1421669390 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f818800faa0 con 0x7f819019a700 2024-04-24T08:52:01.883 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.881+0000 7f8196a00640 1 -- 172.21.15.83:0/1421669390 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81901a73e0 con 0x7f819019a700 2024-04-24T08:52:01.883 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.881+0000 7f818cc00640 1 -- 172.21.15.83:0/1421669390 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8188018b70 con 0x7f819019a700 2024-04-24T08:52:01.883 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.881+0000 7f8196a00640 1 -- 172.21.15.83:0/1421669390 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81901a77e0 con 0x7f819019a700 2024-04-24T08:52:01.885 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.885+0000 7f81999ef740 1 Processor -- start 2024-04-24T08:52:01.886 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.885+0000 7f818cc00640 1 -- 172.21.15.83:0/1421669390 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f8188018d10 con 0x7f819019a700 2024-04-24T08:52:01.886 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.886+0000 7f81999ef740 1 -- start start 2024-04-24T08:52:01.888 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.886+0000 7f81999ef740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561d508711b0 0x561d50871590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:01.888 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.886+0000 7f81999ef740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561d50871ad0 0x561d5087be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:01.888 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.886+0000 7f81999ef740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d5087c390 0x561d5087e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:01.888 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.886+0000 7f81999ef740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561d507d7900 con 0x561d5087c390 2024-04-24T08:52:01.888 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.886+0000 7f81999ef740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x561d5077df20 con 0x561d50871ad0 2024-04-24T08:52:01.888 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.886+0000 7f81999ef740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x561d507d38c0 con 0x561d508711b0 2024-04-24T08:52:01.888 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.886+0000 7f818f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d5087c390 0x561d5087e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:01.888 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.887+0000 7f818f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d5087c390 0x561d5087e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44966/0 (socket says 172.21.15.83:44966) 2024-04-24T08:52:01.888 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.887+0000 7f818f400640 1 -- 172.21.15.83:0/2300848957 learned_addr learned my addr 172.21.15.83:0/2300848957 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:52:01.888 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.887+0000 7f818fe00640 1 --2- 172.21.15.83:0/2300848957 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561d50871ad0 0x561d5087be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:01.888 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.887+0000 7f818f400640 1 --2- 172.21.15.83:0/2300848957 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561d508711b0 0x561d50871590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:01.889 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.887+0000 7f818f400640 1 -- 172.21.15.83:0/2300848957 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561d508711b0 msgr2=0x561d50871590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:01.889 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.887+0000 7f818f400640 1 --2- 172.21.15.83:0/2300848957 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561d508711b0 0x561d50871590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:01.889 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.887+0000 7f818f400640 1 -- 172.21.15.83:0/2300848957 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561d50871ad0 msgr2=0x561d5087be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:01.889 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.887+0000 7f818f400640 1 --2- 172.21.15.83:0/2300848957 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561d50871ad0 0x561d5087be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:01.889 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.887+0000 7f818f400640 1 -- 172.21.15.83:0/2300848957 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d507a2e10 con 0x561d5087c390 2024-04-24T08:52:01.889 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.887+0000 7f818cc00640 1 --2- 172.21.15.83:0/1421669390 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f81740a3ee0 0x7f81740a63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:01.889 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.887+0000 7f818cc00640 1 -- 172.21.15.83:0/1421669390 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f81880d11e0 con 0x7f819019a700 2024-04-24T08:52:01.889 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.887+0000 7f818f400640 1 --2- 172.21.15.83:0/2300848957 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d5087c390 0x561d5087e780 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f8178031ef0 tx=0x7f8178000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c8806971ab33647 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:01.889 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.887+0000 7f818fe00640 1 --2- 172.21.15.83:0/1421669390 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f81740a3ee0 0x7f81740a63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:01.889 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.888+0000 7f8185200640 1 -- 172.21.15.83:0/2300848957 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8178035070 con 0x561d5087c390 2024-04-24T08:52:01.889 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.888+0000 7f8185200640 1 -- 172.21.15.83:0/2300848957 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8178044070 con 0x561d5087c390 2024-04-24T08:52:01.889 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.888+0000 7f8185200640 1 -- 172.21.15.83:0/2300848957 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8178003bb0 con 0x561d5087c390 2024-04-24T08:52:01.893 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.889+0000 7f81999ef740 1 -- 172.21.15.83:0/2300848957 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d5087c390 msgr2=0x561d5087e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:01.893 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.889+0000 7f81999ef740 1 --2- 172.21.15.83:0/2300848957 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d5087c390 0x561d5087e780 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f8178031ef0 tx=0x7f8178000bc0 comp rx=0 tx=0).stop 2024-04-24T08:52:01.893 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.889+0000 7f818fe00640 1 --2- 172.21.15.83:0/1421669390 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f81740a3ee0 0x7f81740a63a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f81800058c0 tx=0x7f8180005850 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:01.893 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.889+0000 7f81999ef740 1 -- 172.21.15.83:0/2300848957 shutdown_connections 2024-04-24T08:52:01.893 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.889+0000 7f81999ef740 1 --2- 172.21.15.83:0/2300848957 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d5087c390 0x561d5087e780 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:01.893 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.889+0000 7f81999ef740 1 --2- 172.21.15.83:0/2300848957 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561d50871ad0 0x561d5087be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:01.894 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.889+0000 7f81999ef740 1 --2- 172.21.15.83:0/2300848957 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561d508711b0 0x561d50871590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:01.894 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.889+0000 7f81999ef740 1 -- 172.21.15.83:0/2300848957 >> 172.21.15.83:0/2300848957 conn(0x561d5086d6c0 msgr2=0x561d5086daa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:01.894 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.889+0000 7f81999ef740 1 -- 172.21.15.83:0/2300848957 shutdown_connections 2024-04-24T08:52:01.894 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.889+0000 7f81999ef740 1 -- 172.21.15.83:0/2300848957 wait complete. 2024-04-24T08:52:01.894 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.889+0000 7f81999ef740 1 -- start start 2024-04-24T08:52:01.894 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.889+0000 7f81999ef740 1 Processor -- start 2024-04-24T08:52:01.894 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.890+0000 7f81999ef740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561d508711b0 0x561d50879330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:01.894 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.890+0000 7f81999ef740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d50871ad0 0x561d50879870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:01.894 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.890+0000 7f81999ef740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561d5087c390 0x561d50879db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:01.894 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.890+0000 7f81999ef740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561d5087ef60 con 0x561d50871ad0 2024-04-24T08:52:01.894 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.890+0000 7f81999ef740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x561d507912c0 con 0x561d5087c390 2024-04-24T08:52:01.895 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.890+0000 7f81999ef740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x561d5087f0e0 con 0x561d508711b0 2024-04-24T08:52:01.895 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.890+0000 7f81999ef740 10 client.0 _ll_register_callbacks cb 0x561d504e9a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:52:01.895 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.890+0000 7f818f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d50871ad0 0x561d50879870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:01.895 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.890+0000 7f818f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d50871ad0 0x561d50879870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44970/0 (socket says 172.21.15.83:44970) 2024-04-24T08:52:01.895 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.890+0000 7f818f400640 1 -- 172.21.15.83:0/2548749336 learned_addr learned my addr 172.21.15.83:0/2548749336 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:52:01.895 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.891+0000 7f818fe00640 1 --2- 172.21.15.83:0/2548749336 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561d508711b0 0x561d50879330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:01.895 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.891+0000 7f8194a00640 1 --2- 172.21.15.83:0/2548749336 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561d5087c390 0x561d50879db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:01.895 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.891+0000 7f818f400640 1 -- 172.21.15.83:0/2548749336 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561d508711b0 msgr2=0x561d50879330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:01.895 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.891+0000 7f818f400640 1 --2- 172.21.15.83:0/2548749336 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561d508711b0 0x561d50879330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:01.895 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.891+0000 7f818f400640 1 -- 172.21.15.83:0/2548749336 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561d5087c390 msgr2=0x561d50879db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:01.895 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.891+0000 7f818f400640 1 --2- 172.21.15.83:0/2548749336 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561d5087c390 0x561d50879db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:01.895 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.891+0000 7f818f400640 1 -- 172.21.15.83:0/2548749336 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d507a1fd0 con 0x561d50871ad0 2024-04-24T08:52:01.895 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.891+0000 7f8194a00640 1 --2- 172.21.15.83:0/2548749336 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561d5087c390 0x561d50879db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:52:01.895 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.891+0000 7f818f400640 1 --2- 172.21.15.83:0/2548749336 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d50871ad0 0x561d50879870 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f8178033000 tx=0x7f817802ab10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fac4b8f4a786a92f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:01.896 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.891+0000 7f817d600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:52:01.896 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.891+0000 7f817d600640 1 -- 172.21.15.83:0/2548749336 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f817804c070 con 0x561d50871ad0 2024-04-24T08:52:01.896 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.891+0000 7f817d600640 1 -- 172.21.15.83:0/2548749336 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f817803d480 con 0x561d50871ad0 2024-04-24T08:52:01.896 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.891+0000 7f81999ef740 1 -- 172.21.15.83:0/2548749336 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8178044070 con 0x561d50871ad0 2024-04-24T08:52:01.896 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.892+0000 7f817d600640 1 -- 172.21.15.83:0/2548749336 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8178058050 con 0x561d50871ad0 2024-04-24T08:52:01.896 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.892+0000 7f81999ef740 10 client.15855 fetch_fsmap learned FSMap version 28 2024-04-24T08:52:01.896 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.892+0000 7f81999ef740 1 -- 172.21.15.83:0/2548749336 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561d504ea210 con 0x561d50871ad0 2024-04-24T08:52:01.896 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.892+0000 7f817d600640 1 -- 172.21.15.83:0/2548749336 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8178044070 con 0x561d50871ad0 2024-04-24T08:52:01.896 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.892+0000 7f817d600640 1 -- 172.21.15.83:0/2548749336 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f817803a040 con 0x561d50871ad0 2024-04-24T08:52:01.896 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.892+0000 7f81999ef740 10 client.15855 fetch_fsmap finished waiting for FSMap version 28 2024-04-24T08:52:01.896 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.892+0000 7f81999ef740 10 client.15855 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T08:52:01.896 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.892+0000 7f81999ef740 20 client.15855 populate_metadata read hostname 'smithi083' 2024-04-24T08:52:01.896 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.892+0000 7f81999ef740 1 --2- 172.21.15.83:0/2548749336 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x561d5088c320 0x561d5088e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:01.896 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.892+0000 7f81999ef740 4 client.15855 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:52:01.896 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.892+0000 7f81999ef740 1 -- 172.21.15.83:0/2548749336 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561d507e5320 con 0x561d5088c320 2024-04-24T08:52:01.897 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.893+0000 7f818fe00640 1 --2- 172.21.15.83:0/2548749336 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x561d5088c320 0x561d5088e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:01.898 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.894+0000 7f818fe00640 1 --2- 172.21.15.83:0/2548749336 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x561d5088c320 0x561d5088e710 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4c3302f8219415 server_cookie=8865e88113718fbe in_seq=0 out_seq=0 2024-04-24T08:52:01.898 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.894+0000 7f817d600640 10 client.15855 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:52:01.898 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.896+0000 7f817d600640 1 -- 172.21.15.83:0/2548749336 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561d507e5320 con 0x561d5088c320 2024-04-24T08:52:01.898 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.896+0000 7f817d600640 10 client.15855 handle_command_reply: tid=0 2024-04-24T08:52:01.898 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.896+0000 7f81999ef740 1 client.15855 shutdown 2024-04-24T08:52:01.898 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.896+0000 7f81999ef740 1 -- 172.21.15.83:0/2548749336 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d50871ad0 msgr2=0x561d50879870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:01.898 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.896+0000 7f81999ef740 1 --2- 172.21.15.83:0/2548749336 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d50871ad0 0x561d50879870 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f8178033000 tx=0x7f817802ab10 comp rx=0 tx=0).stop 2024-04-24T08:52:01.898 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.896+0000 7f81999ef740 1 -- 172.21.15.83:0/2548749336 shutdown_connections 2024-04-24T08:52:01.898 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.896+0000 7f81999ef740 1 --2- 172.21.15.83:0/2548749336 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x561d5088c320 0x561d5088e710 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:01.899 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.896+0000 7f81999ef740 1 --2- 172.21.15.83:0/2548749336 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561d5087c390 0x561d50879db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:01.899 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.896+0000 7f817d600640 0 client.15855 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:52:01.899 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.896+0000 7f81999ef740 1 --2- 172.21.15.83:0/2548749336 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d50871ad0 0x561d50879870 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:01.899 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.896+0000 7f81999ef740 1 --2- 172.21.15.83:0/2548749336 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561d508711b0 0x561d50879330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:01.899 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.896+0000 7f81999ef740 1 -- 172.21.15.83:0/2548749336 >> 172.21.15.83:0/2548749336 conn(0x561d5086d6c0 msgr2=0x561d508733f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:01.899 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.897+0000 7f81999ef740 1 -- 172.21.15.83:0/2548749336 shutdown_connections 2024-04-24T08:52:01.899 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.897+0000 7f81999ef740 1 -- 172.21.15.83:0/2548749336 wait complete. 2024-04-24T08:52:01.899 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.897+0000 7f81999ef740 20 client.15855 trim_cache size 0 max 16384 2024-04-24T08:52:01.913 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.913+0000 7f81999ef740 1 Processor -- start 2024-04-24T08:52:01.913 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.913+0000 7f81999ef740 1 -- start start 2024-04-24T08:52:01.913 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.913+0000 7f81999ef740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561d508711b0 0x561d50873b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:01.914 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.913+0000 7f81999ef740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561d50871ad0 0x561d508765b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:01.914 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.913+0000 7f81999ef740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d5087c390 0x561d50876af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:01.914 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.913+0000 7f81999ef740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f81901b1480 con 0x561d5087c390 2024-04-24T08:52:01.914 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.913+0000 7f81999ef740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f81901b2fc0 con 0x561d508711b0 2024-04-24T08:52:01.914 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.913+0000 7f81999ef740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f81901b1b50 con 0x561d50871ad0 2024-04-24T08:52:01.914 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.913+0000 7f8194a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d5087c390 0x561d50876af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:01.914 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.913+0000 7f8194a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d5087c390 0x561d50876af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44978/0 (socket says 172.21.15.83:44978) 2024-04-24T08:52:01.914 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.913+0000 7f8194a00640 1 -- 172.21.15.83:0/4200136640 learned_addr learned my addr 172.21.15.83:0/4200136640 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:52:01.916 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.914+0000 7f818f400640 1 --2- 172.21.15.83:0/4200136640 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561d50871ad0 0x561d508765b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:01.916 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.914+0000 7f8194a00640 1 -- 172.21.15.83:0/4200136640 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561d50871ad0 msgr2=0x561d508765b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:01.916 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.914+0000 7f8194a00640 1 --2- 172.21.15.83:0/4200136640 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561d50871ad0 0x561d508765b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:01.916 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.914+0000 7f818fe00640 1 --2- 172.21.15.83:0/4200136640 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561d508711b0 0x561d50873b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:01.916 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.914+0000 7f8194a00640 1 -- 172.21.15.83:0/4200136640 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561d508711b0 msgr2=0x561d50873b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:01.916 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.914+0000 7f8194a00640 1 --2- 172.21.15.83:0/4200136640 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561d508711b0 0x561d50873b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:01.916 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.914+0000 7f8194a00640 1 -- 172.21.15.83:0/4200136640 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d505b6620 con 0x561d5087c390 2024-04-24T08:52:01.916 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.914+0000 7f818f400640 1 --2- 172.21.15.83:0/4200136640 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561d50871ad0 0x561d508765b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:52:01.916 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.914+0000 7f818fe00640 1 --2- 172.21.15.83:0/4200136640 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561d508711b0 0x561d50873b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:52:01.916 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.914+0000 7f8194a00640 1 --2- 172.21.15.83:0/4200136640 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d5087c390 0x561d50876af0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f8188099040 tx=0x7f818809d2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2462985978767efe server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:01.916 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.914+0000 7f817fe00640 1 -- 172.21.15.83:0/4200136640 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81880b6070 con 0x561d5087c390 2024-04-24T08:52:01.917 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.914+0000 7f817fe00640 1 -- 172.21.15.83:0/4200136640 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f81880a5850 con 0x561d5087c390 2024-04-24T08:52:01.917 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.914+0000 7f817fe00640 1 -- 172.21.15.83:0/4200136640 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81880ce6f0 con 0x561d5087c390 2024-04-24T08:52:01.917 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.915+0000 7f81999ef740 1 -- 172.21.15.83:0/4200136640 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d5087c390 msgr2=0x561d50876af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:01.917 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.915+0000 7f81999ef740 1 --2- 172.21.15.83:0/4200136640 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d5087c390 0x561d50876af0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f8188099040 tx=0x7f818809d2e0 comp rx=0 tx=0).stop 2024-04-24T08:52:01.917 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.915+0000 7f81999ef740 1 -- 172.21.15.83:0/4200136640 shutdown_connections 2024-04-24T08:52:01.917 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.915+0000 7f81999ef740 1 --2- 172.21.15.83:0/4200136640 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d5087c390 0x561d50876af0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:01.917 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.915+0000 7f81999ef740 1 --2- 172.21.15.83:0/4200136640 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561d50871ad0 0x561d508765b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:01.917 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.915+0000 7f81999ef740 1 --2- 172.21.15.83:0/4200136640 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561d508711b0 0x561d50873b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:01.917 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.915+0000 7f81999ef740 1 -- 172.21.15.83:0/4200136640 >> 172.21.15.83:0/4200136640 conn(0x561d5086d6c0 msgr2=0x561d508745d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:01.917 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.915+0000 7f81999ef740 1 -- 172.21.15.83:0/4200136640 shutdown_connections 2024-04-24T08:52:01.917 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.915+0000 7f81999ef740 1 -- 172.21.15.83:0/4200136640 wait complete. 2024-04-24T08:52:01.917 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.916+0000 7f81999ef740 1 -- start start 2024-04-24T08:52:01.917 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.916+0000 7f81999ef740 1 Processor -- start 2024-04-24T08:52:01.920 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.916+0000 7f81999ef740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561d508711b0 0x561d50891bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:01.920 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.916+0000 7f81999ef740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561d50871ad0 0x561d5088eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:01.920 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.916+0000 7f81999ef740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d5087c390 0x561d5088f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:01.920 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.916+0000 7f81999ef740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f81901b28f0 con 0x561d5087c390 2024-04-24T08:52:01.920 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.916+0000 7f81999ef740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f81901b2220 con 0x561d50871ad0 2024-04-24T08:52:01.920 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.916+0000 7f81999ef740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f81901af460 con 0x561d508711b0 2024-04-24T08:52:01.921 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.916+0000 7f81999ef740 10 client.0 _ll_register_callbacks cb 0x7f8178031e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:52:01.921 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.917+0000 7f8194a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d5087c390 0x561d5088f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:01.921 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.917+0000 7f8194a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d5087c390 0x561d5088f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44990/0 (socket says 172.21.15.83:44990) 2024-04-24T08:52:01.921 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.917+0000 7f8194a00640 1 -- 172.21.15.83:0/2379289470 learned_addr learned my addr 172.21.15.83:0/2379289470 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:52:01.921 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.917+0000 7f818f400640 1 --2- 172.21.15.83:0/2379289470 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561d50871ad0 0x561d5088eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:01.921 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.917+0000 7f818fe00640 1 --2- 172.21.15.83:0/2379289470 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561d508711b0 0x561d50891bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:01.921 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.917+0000 7f8194a00640 1 -- 172.21.15.83:0/2379289470 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561d508711b0 msgr2=0x561d50891bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:01.921 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.917+0000 7f8194a00640 1 --2- 172.21.15.83:0/2379289470 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561d508711b0 0x561d50891bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:01.921 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.917+0000 7f8194a00640 1 -- 172.21.15.83:0/2379289470 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561d50871ad0 msgr2=0x561d5088eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:01.921 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.917+0000 7f8194a00640 1 --2- 172.21.15.83:0/2379289470 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561d50871ad0 0x561d5088eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:01.921 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.917+0000 7f8194a00640 1 -- 172.21.15.83:0/2379289470 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d5088fdc0 con 0x561d5087c390 2024-04-24T08:52:01.921 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.917+0000 7f8194a00640 1 --2- 172.21.15.83:0/2379289470 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d5087c390 0x561d5088f590 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f81880078b0 tx=0x7f818800fc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a42df33ff8c4c619 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:01.922 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.917+0000 7f817d600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:52:01.922 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.917+0000 7f817d600640 1 -- 172.21.15.83:0/2379289470 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8188080060 con 0x561d5087c390 2024-04-24T08:52:01.922 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.917+0000 7f817d600640 1 -- 172.21.15.83:0/2379289470 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f81880a53c0 con 0x561d5087c390 2024-04-24T08:52:01.922 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.917+0000 7f81999ef740 1 -- 172.21.15.83:0/2379289470 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81880a5850 con 0x561d5087c390 2024-04-24T08:52:01.922 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.918+0000 7f817d600640 1 -- 172.21.15.83:0/2379289470 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f81880b6040 con 0x561d5087c390 2024-04-24T08:52:01.922 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.918+0000 7f81999ef740 10 client.15861 fetch_fsmap learned FSMap version 28 2024-04-24T08:52:01.922 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.918+0000 7f81999ef740 1 -- 172.21.15.83:0/2379289470 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561d508fad60 con 0x561d5087c390 2024-04-24T08:52:01.922 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.918+0000 7f817d600640 1 -- 172.21.15.83:0/2379289470 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f81880a5850 con 0x561d5087c390 2024-04-24T08:52:01.922 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.918+0000 7f817d600640 1 -- 172.21.15.83:0/2379289470 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f8188098930 con 0x561d5087c390 2024-04-24T08:52:01.922 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.918+0000 7f81999ef740 10 client.15861 fetch_fsmap finished waiting for FSMap version 28 2024-04-24T08:52:01.922 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.918+0000 7f81999ef740 10 client.15861 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T08:52:01.922 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.918+0000 7f81999ef740 20 client.15861 populate_metadata read hostname 'smithi083' 2024-04-24T08:52:01.922 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.918+0000 7f81999ef740 1 --2- 172.21.15.83:0/2379289470 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x561d508fb380 0x561d508fd7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:01.923 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.918+0000 7f81999ef740 4 client.15861 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T08:52:01.923 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.918+0000 7f81999ef740 1 -- 172.21.15.83:0/2379289470 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561d507df190 con 0x561d508fb380 2024-04-24T08:52:01.923 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.919+0000 7f818fe00640 1 --2- 172.21.15.83:0/2379289470 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x561d508fb380 0x561d508fd7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:01.923 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.920+0000 7f818fe00640 1 --2- 172.21.15.83:0/2379289470 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x561d508fb380 0x561d508fd7a0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c26050f57962c92 server_cookie=58fe68dc6934871f in_seq=0 out_seq=0 2024-04-24T08:52:01.923 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:01.920+0000 7f817d600640 10 client.15861 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:52:01.954 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:52:01 smithi138 ceph-mon[31441]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 25 op/s 2024-04-24T08:52:02.045 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:02.045+0000 7f817d600640 1 -- 172.21.15.83:0/2379289470 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561d507df190 con 0x561d508fb380 2024-04-24T08:52:02.046 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:02.045+0000 7f817d600640 10 client.15861 handle_command_reply: tid=0 2024-04-24T08:52:02.046 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:02.045+0000 7f81999ef740 1 client.15861 shutdown 2024-04-24T08:52:02.046 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:02.046+0000 7f81999ef740 1 -- 172.21.15.83:0/2379289470 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d5087c390 msgr2=0x561d5088f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:02.046 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:02.046+0000 7f81999ef740 1 --2- 172.21.15.83:0/2379289470 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d5087c390 0x561d5088f590 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f81880078b0 tx=0x7f818800fc50 comp rx=0 tx=0).stop 2024-04-24T08:52:02.046 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:02.046+0000 7f81999ef740 1 -- 172.21.15.83:0/2379289470 shutdown_connections 2024-04-24T08:52:02.047 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:02.046+0000 7f81999ef740 1 --2- 172.21.15.83:0/2379289470 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x561d508fb380 0x561d508fd7a0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:02.047 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:02.046+0000 7f81999ef740 1 --2- 172.21.15.83:0/2379289470 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561d5087c390 0x561d5088f590 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:02.047 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:02.046+0000 7f81999ef740 1 --2- 172.21.15.83:0/2379289470 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561d50871ad0 0x561d5088eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:02.047 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:02.046+0000 7f81999ef740 1 --2- 172.21.15.83:0/2379289470 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561d508711b0 0x561d50891bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:02.047 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:02.046+0000 7f81999ef740 1 -- 172.21.15.83:0/2379289470 >> 172.21.15.83:0/2379289470 conn(0x561d5086d6c0 msgr2=0x561d508792d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:02.047 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:02.046+0000 7f817d600640 0 client.15861 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:52:02.047 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:02.047+0000 7f81999ef740 1 -- 172.21.15.83:0/2379289470 shutdown_connections 2024-04-24T08:52:02.048 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:02.047+0000 7f81999ef740 1 -- 172.21.15.83:0/2379289470 wait complete. 2024-04-24T08:52:02.048 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:02.047+0000 7f81999ef740 20 client.15861 trim_cache size 0 max 16384 2024-04-24T08:52:02.048 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T08:52:02.048 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T08:52:02.048 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T08:52:02.048 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T08:52:02.049 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:02.048+0000 7f8196a00640 1 -- 172.21.15.83:0/1421669390 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f81740a3ee0 msgr2=0x7f81740a63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:02.049 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:02.048+0000 7f8196a00640 1 --2- 172.21.15.83:0/1421669390 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f81740a3ee0 0x7f81740a63a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f81800058c0 tx=0x7f8180005850 comp rx=0 tx=0).stop 2024-04-24T08:52:02.049 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:02.048+0000 7f8196a00640 1 -- 172.21.15.83:0/1421669390 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f819019a700 msgr2=0x7f819019ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:02.049 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:02.048+0000 7f8196a00640 1 --2- 172.21.15.83:0/1421669390 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f819019a700 0x7f819019ab80 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f818800cce0 tx=0x7f818800a620 comp rx=0 tx=0).stop 2024-04-24T08:52:02.049 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:02.048+0000 7f8196a00640 1 -- 172.21.15.83:0/1421669390 shutdown_connections 2024-04-24T08:52:02.049 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:02.048+0000 7f8196a00640 1 --2- 172.21.15.83:0/1421669390 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f81740a3ee0 0x7f81740a63a0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:02.050 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:02.048+0000 7f8196a00640 1 --2- 172.21.15.83:0/1421669390 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f819019a700 0x7f819019ab80 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:02.050 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:02.048+0000 7f8196a00640 1 --2- 172.21.15.83:0/1421669390 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8190109540 0x7f81901a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:02.050 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:02.048+0000 7f8196a00640 1 --2- 172.21.15.83:0/1421669390 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8190104470 0x7f81901a0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:02.050 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:02.048+0000 7f8196a00640 1 -- 172.21.15.83:0/1421669390 >> 172.21.15.83:0/1421669390 conn(0x7f81900ffd30 msgr2=0x7f8190101ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:02.050 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:02.049+0000 7f8196a00640 1 -- 172.21.15.83:0/1421669390 shutdown_connections 2024-04-24T08:52:02.050 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:02.049+0000 7f8196a00640 1 -- 172.21.15.83:0/1421669390 wait complete. 2024-04-24T08:52:03.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:52:03 smithi083 ceph-mon[27907]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 25 op/s 2024-04-24T08:52:03.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:52:03 smithi083 ceph-mon[27907]: mgrmap e25: x(active, since 26s), standbys: z, y 2024-04-24T08:52:03.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:52:03 smithi138 ceph-mon[31441]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 25 op/s 2024-04-24T08:52:03.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:52:03 smithi138 ceph-mon[31441]: mgrmap e25: x(active, since 26s), standbys: z, y 2024-04-24T08:52:03.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:52:03 smithi189 ceph-mon[31791]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 25 op/s 2024-04-24T08:52:03.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:52:03 smithi189 ceph-mon[31791]: mgrmap e25: x(active, since 26s), standbys: z, y 2024-04-24T08:52:05.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:52:05 smithi083 ceph-mon[27907]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 47 KiB/s wr, 50 op/s 2024-04-24T08:52:05.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:52:05 smithi138 ceph-mon[31441]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 47 KiB/s wr, 50 op/s 2024-04-24T08:52:05.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:52:05 smithi189 ceph-mon[31791]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 47 KiB/s wr, 50 op/s 2024-04-24T08:52:07.760 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:52:07 smithi138 ceph-mon[31441]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 25 KiB/s wr, 29 op/s 2024-04-24T08:52:07.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:52:07 smithi083 ceph-mon[27907]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 25 KiB/s wr, 29 op/s 2024-04-24T08:52:07.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:52:07 smithi189 ceph-mon[31791]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 25 KiB/s wr, 29 op/s 2024-04-24T08:52:09.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:52:09 smithi138 ceph-mon[31441]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 25 KiB/s wr, 33 op/s 2024-04-24T08:52:09.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:52:09 smithi083 ceph-mon[27907]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 25 KiB/s wr, 33 op/s 2024-04-24T08:52:09.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:52:09 smithi189 ceph-mon[31791]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 25 KiB/s wr, 33 op/s 2024-04-24T08:52:11.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:52:11 smithi083 ceph-mon[27907]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 28 op/s 2024-04-24T08:52:11.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:52:11 smithi138 ceph-mon[31441]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 28 op/s 2024-04-24T08:52:11.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:52:11 smithi189 ceph-mon[31791]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 28 op/s 2024-04-24T08:52:13.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:52:13 smithi083 ceph-mon[27907]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 29 op/s 2024-04-24T08:52:13.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:52:13 smithi138 ceph-mon[31441]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 29 op/s 2024-04-24T08:52:13.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:52:13 smithi189 ceph-mon[31791]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 29 op/s 2024-04-24T08:52:15.200 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.199+0000 7fa10ae00640 1 -- 172.21.15.83:0/72439707 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa104074430 msgr2=0x7fa104074810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:15.200 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.199+0000 7fa10ae00640 1 --2- 172.21.15.83:0/72439707 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa104074430 0x7fa104074810 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fa0f8009900 tx=0x7fa0f802a480 comp rx=0 tx=0).stop 2024-04-24T08:52:15.200 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.199+0000 7fa10ae00640 1 -- 172.21.15.83:0/72439707 shutdown_connections 2024-04-24T08:52:15.200 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.199+0000 7fa10ae00640 1 --2- 172.21.15.83:0/72439707 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa10410e650 0x7fa104110aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:15.200 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.199+0000 7fa10ae00640 1 --2- 172.21.15.83:0/72439707 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa104074d50 0x7fa10410e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:15.200 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.199+0000 7fa10ae00640 1 --2- 172.21.15.83:0/72439707 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa104074430 0x7fa104074810 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:15.200 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.199+0000 7fa10ae00640 1 -- 172.21.15.83:0/72439707 >> 172.21.15.83:0/72439707 conn(0x7fa1040fd940 msgr2=0x7fa1040ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:15.200 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.199+0000 7fa10ae00640 1 -- 172.21.15.83:0/72439707 shutdown_connections 2024-04-24T08:52:15.200 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.199+0000 7fa10ae00640 1 -- 172.21.15.83:0/72439707 wait complete. 2024-04-24T08:52:15.202 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.200+0000 7fa10ae00640 1 Processor -- start 2024-04-24T08:52:15.203 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.200+0000 7fa10ae00640 1 -- start start 2024-04-24T08:52:15.203 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.201+0000 7fa10ae00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa104074430 0x7fa1041a0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:15.203 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.201+0000 7fa10ae00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa104074d50 0x7fa1041a0950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:15.203 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.201+0000 7fa10ae00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa10410e650 0x7fa10419a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:15.203 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.201+0000 7fa10ae00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa1041142d0 con 0x7fa10410e650 2024-04-24T08:52:15.203 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.201+0000 7fa10ae00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa104114150 con 0x7fa104074430 2024-04-24T08:52:15.203 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.201+0000 7fa10ae00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa104114450 con 0x7fa104074d50 2024-04-24T08:52:15.203 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.201+0000 7fa10a400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa10410e650 0x7fa10419a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:15.203 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.201+0000 7fa10a400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa10410e650 0x7fa10419a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55596/0 (socket says 172.21.15.83:55596) 2024-04-24T08:52:15.203 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.201+0000 7fa10a400640 1 -- 172.21.15.83:0/841477484 learned_addr learned my addr 172.21.15.83:0/841477484 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:52:15.203 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.201+0000 7fa109a00640 1 --2- 172.21.15.83:0/841477484 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa104074430 0x7fa1041a0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:15.203 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.202+0000 7fa109000640 1 --2- 172.21.15.83:0/841477484 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa104074d50 0x7fa1041a0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:15.203 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.202+0000 7fa10a400640 1 -- 172.21.15.83:0/841477484 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa104074d50 msgr2=0x7fa1041a0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:15.203 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.202+0000 7fa10a400640 1 --2- 172.21.15.83:0/841477484 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa104074d50 0x7fa1041a0950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:15.204 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.202+0000 7fa10a400640 1 -- 172.21.15.83:0/841477484 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa104074430 msgr2=0x7fa1041a0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:15.204 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.202+0000 7fa10a400640 1 --2- 172.21.15.83:0/841477484 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa104074430 0x7fa1041a0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:15.204 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.202+0000 7fa10a400640 1 -- 172.21.15.83:0/841477484 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa10419ad90 con 0x7fa10410e650 2024-04-24T08:52:15.204 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.202+0000 7fa10a400640 1 --2- 172.21.15.83:0/841477484 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa10410e650 0x7fa10419a610 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc00eaa0 tx=0x7fa0fc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:15.204 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.202+0000 7fa102000640 1 -- 172.21.15.83:0/841477484 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0fc00cdb0 con 0x7fa10410e650 2024-04-24T08:52:15.204 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.202+0000 7fa102000640 1 -- 172.21.15.83:0/841477484 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa0fc005020 con 0x7fa10410e650 2024-04-24T08:52:15.204 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.202+0000 7fa10ae00640 1 -- 172.21.15.83:0/841477484 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa10419b080 con 0x7fa10410e650 2024-04-24T08:52:15.204 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.202+0000 7fa102000640 1 -- 172.21.15.83:0/841477484 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0fc0116d0 con 0x7fa10410e650 2024-04-24T08:52:15.204 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.202+0000 7fa10ae00640 1 -- 172.21.15.83:0/841477484 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1041a72d0 con 0x7fa10410e650 2024-04-24T08:52:15.206 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.206+0000 7fa10f23f740 1 Processor -- start 2024-04-24T08:52:15.207 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.207+0000 7fa102000640 1 -- 172.21.15.83:0/841477484 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7fa0fc0051d0 con 0x7fa10410e650 2024-04-24T08:52:15.207 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.207+0000 7fa10f23f740 1 -- start start 2024-04-24T08:52:15.207 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.207+0000 7fa10f23f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a5228763c0 0x55a5228767a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:15.207 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.207+0000 7fa10f23f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a522876ce0 0x55a522881060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:15.208 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.207+0000 7fa10f23f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a5228815a0 0x55a522883990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:15.208 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.207+0000 7fa10f23f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a5227dc900 con 0x55a5228815a0 2024-04-24T08:52:15.208 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.207+0000 7fa10f23f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a522782f20 con 0x55a5228763c0 2024-04-24T08:52:15.208 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.207+0000 7fa10f23f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a5227d88c0 con 0x55a522876ce0 2024-04-24T08:52:15.209 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.208+0000 7fa109000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a5228815a0 0x55a522883990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:15.209 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.208+0000 7fa109000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a5228815a0 0x55a522883990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55606/0 (socket says 172.21.15.83:55606) 2024-04-24T08:52:15.209 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.208+0000 7fa109000640 1 -- 172.21.15.83:0/2375594944 learned_addr learned my addr 172.21.15.83:0/2375594944 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:52:15.209 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.208+0000 7fa109a00640 1 --2- 172.21.15.83:0/2375594944 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a522876ce0 0x55a522881060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:15.209 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.208+0000 7fa109000640 1 --2- 172.21.15.83:0/2375594944 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a5228763c0 0x55a5228767a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:15.210 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.208+0000 7fa109000640 1 -- 172.21.15.83:0/2375594944 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a522876ce0 msgr2=0x55a522881060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:15.210 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.208+0000 7fa109000640 1 --2- 172.21.15.83:0/2375594944 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a522876ce0 0x55a522881060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:15.210 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.208+0000 7fa109000640 1 -- 172.21.15.83:0/2375594944 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a5228763c0 msgr2=0x55a5228767a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:15.210 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.208+0000 7fa109000640 1 --2- 172.21.15.83:0/2375594944 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a5228763c0 0x55a5228767a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:15.210 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.208+0000 7fa109000640 1 -- 172.21.15.83:0/2375594944 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5227a7e10 con 0x55a5228815a0 2024-04-24T08:52:15.210 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.209+0000 7fa109000640 1 --2- 172.21.15.83:0/2375594944 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a5228815a0 0x55a522883990 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fa0f0014520 tx=0x7fa0f000d540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe4c9dc136e3756b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:15.210 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.209+0000 7fa102000640 1 --2- 172.21.15.83:0/841477484 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa0ec0a3e30 0x7fa0ec0a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:15.210 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.209+0000 7fa102000640 1 -- 172.21.15.83:0/841477484 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7fa0fc0d1130 con 0x7fa10410e650 2024-04-24T08:52:15.210 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.209+0000 7fa0f6a00640 1 -- 172.21.15.83:0/2375594944 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f0012710 con 0x55a5228815a0 2024-04-24T08:52:15.210 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.209+0000 7fa0f6a00640 1 -- 172.21.15.83:0/2375594944 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa0f001a070 con 0x55a5228815a0 2024-04-24T08:52:15.210 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.209+0000 7fa0f6a00640 1 -- 172.21.15.83:0/2375594944 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f0011b10 con 0x55a5228815a0 2024-04-24T08:52:15.210 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.209+0000 7fa109a00640 1 --2- 172.21.15.83:0/841477484 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa0ec0a3e30 0x7fa0ec0a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:15.214 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.210+0000 7fa10f23f740 1 -- 172.21.15.83:0/2375594944 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a5228815a0 msgr2=0x55a522883990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:15.214 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.210+0000 7fa10f23f740 1 --2- 172.21.15.83:0/2375594944 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a5228815a0 0x55a522883990 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fa0f0014520 tx=0x7fa0f000d540 comp rx=0 tx=0).stop 2024-04-24T08:52:15.214 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.210+0000 7fa109a00640 1 --2- 172.21.15.83:0/841477484 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa0ec0a3e30 0x7fa0ec0a62f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa0f8000c00 tx=0x7fa0f8005a20 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:15.214 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.210+0000 7fa10f23f740 1 -- 172.21.15.83:0/2375594944 shutdown_connections 2024-04-24T08:52:15.214 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.210+0000 7fa10f23f740 1 --2- 172.21.15.83:0/2375594944 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a5228815a0 0x55a522883990 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:15.215 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.210+0000 7fa10f23f740 1 --2- 172.21.15.83:0/2375594944 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a522876ce0 0x55a522881060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:15.215 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.210+0000 7fa10f23f740 1 --2- 172.21.15.83:0/2375594944 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a5228763c0 0x55a5228767a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:15.215 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.210+0000 7fa10f23f740 1 -- 172.21.15.83:0/2375594944 >> 172.21.15.83:0/2375594944 conn(0x55a5228728d0 msgr2=0x55a522872cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:15.215 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.210+0000 7fa10f23f740 1 -- 172.21.15.83:0/2375594944 shutdown_connections 2024-04-24T08:52:15.215 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.210+0000 7fa10f23f740 1 -- 172.21.15.83:0/2375594944 wait complete. 2024-04-24T08:52:15.215 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.211+0000 7fa10f23f740 1 -- start start 2024-04-24T08:52:15.215 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.211+0000 7fa10f23f740 1 Processor -- start 2024-04-24T08:52:15.215 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.211+0000 7fa10f23f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a5228763c0 0x55a52287e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:15.215 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.211+0000 7fa10f23f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a522876ce0 0x55a52287ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:15.215 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.211+0000 7fa10f23f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a5228815a0 0x55a52287f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:15.215 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.211+0000 7fa10f23f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a5228840e0 con 0x55a5228815a0 2024-04-24T08:52:15.215 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.211+0000 7fa10f23f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a5227962c0 con 0x55a522876ce0 2024-04-24T08:52:15.215 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.211+0000 7fa10f23f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a522884260 con 0x55a5228763c0 2024-04-24T08:52:15.215 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.211+0000 7fa10f23f740 10 client.0 _ll_register_callbacks cb 0x55a5224eea00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:52:15.215 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.212+0000 7fa10a400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a5228815a0 0x55a52287f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:15.216 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.212+0000 7fa10a400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a5228815a0 0x55a52287f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55614/0 (socket says 172.21.15.83:55614) 2024-04-24T08:52:15.216 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.212+0000 7fa10a400640 1 -- 172.21.15.83:0/3863743883 learned_addr learned my addr 172.21.15.83:0/3863743883 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:52:15.216 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.212+0000 7fa109a00640 1 --2- 172.21.15.83:0/3863743883 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a5228763c0 0x55a52287e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:15.216 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.212+0000 7fa109000640 1 --2- 172.21.15.83:0/3863743883 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a522876ce0 0x55a52287ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:15.216 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.212+0000 7fa10a400640 1 -- 172.21.15.83:0/3863743883 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a5228763c0 msgr2=0x55a52287e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:15.216 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.212+0000 7fa10a400640 1 --2- 172.21.15.83:0/3863743883 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a5228763c0 0x55a52287e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:15.216 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.212+0000 7fa10a400640 1 -- 172.21.15.83:0/3863743883 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a522876ce0 msgr2=0x55a52287ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:15.216 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.212+0000 7fa10a400640 1 --2- 172.21.15.83:0/3863743883 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a522876ce0 0x55a52287ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:15.216 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.212+0000 7fa10a400640 1 -- 172.21.15.83:0/3863743883 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5227a6fd0 con 0x55a5228815a0 2024-04-24T08:52:15.216 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.212+0000 7fa109000640 1 --2- 172.21.15.83:0/3863743883 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a522876ce0 0x55a52287ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:52:15.216 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.212+0000 7fa10a400640 1 --2- 172.21.15.83:0/3863743883 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a5228815a0 0x55a52287f120 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc00efd0 tx=0x7fa0fc09ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a76f1546be01f2d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:15.216 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.212+0000 7fa0eb400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:52:15.216 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.212+0000 7fa0eb400640 1 -- 172.21.15.83:0/3863743883 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0fc01f040 con 0x55a5228815a0 2024-04-24T08:52:15.216 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.213+0000 7fa0eb400640 1 -- 172.21.15.83:0/3863743883 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7fa0fc0ce7b0 con 0x55a5228815a0 2024-04-24T08:52:15.216 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.213+0000 7fa10f23f740 1 -- 172.21.15.83:0/3863743883 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0f001a070 con 0x55a5228815a0 2024-04-24T08:52:15.216 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.213+0000 7fa0eb400640 1 -- 172.21.15.83:0/3863743883 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa0fc0bd0d0 con 0x55a5228815a0 2024-04-24T08:52:15.217 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.213+0000 7fa10f23f740 10 client.15873 fetch_fsmap learned FSMap version 28 2024-04-24T08:52:15.217 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.213+0000 7fa10f23f740 1 -- 172.21.15.83:0/3863743883 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a5224ef210 con 0x55a5228815a0 2024-04-24T08:52:15.217 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.213+0000 7fa0eb400640 1 -- 172.21.15.83:0/3863743883 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa0f001a070 con 0x55a5228815a0 2024-04-24T08:52:15.217 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.213+0000 7fa0eb400640 1 -- 172.21.15.83:0/3863743883 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa0fc00f040 con 0x55a5228815a0 2024-04-24T08:52:15.217 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.213+0000 7fa10f23f740 10 client.15873 fetch_fsmap finished waiting for FSMap version 28 2024-04-24T08:52:15.217 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.213+0000 7fa10f23f740 10 client.15873 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T08:52:15.217 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.213+0000 7fa10f23f740 20 client.15873 populate_metadata read hostname 'smithi083' 2024-04-24T08:52:15.217 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.213+0000 7fa10f23f740 1 --2- 172.21.15.83:0/3863743883 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55a5228918a0 0x55a522893c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:15.217 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.213+0000 7fa10f23f740 4 client.15873 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:52:15.217 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.213+0000 7fa10f23f740 1 -- 172.21.15.83:0/3863743883 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a5227ea320 con 0x55a5228918a0 2024-04-24T08:52:15.217 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.214+0000 7fa109a00640 1 --2- 172.21.15.83:0/3863743883 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55a5228918a0 0x55a522893c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:15.218 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.215+0000 7fa109a00640 1 --2- 172.21.15.83:0/3863743883 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55a5228918a0 0x55a522893c90 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62fff65e9911409c server_cookie=868652859f0ffd6 in_seq=0 out_seq=0 2024-04-24T08:52:15.219 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.215+0000 7fa0eb400640 10 client.15873 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:52:15.219 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.217+0000 7fa0eb400640 1 -- 172.21.15.83:0/3863743883 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a5227ea320 con 0x55a5228918a0 2024-04-24T08:52:15.219 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.217+0000 7fa0eb400640 10 client.15873 handle_command_reply: tid=0 2024-04-24T08:52:15.219 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.217+0000 7fa10f23f740 1 client.15873 shutdown 2024-04-24T08:52:15.219 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.217+0000 7fa10f23f740 1 -- 172.21.15.83:0/3863743883 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a5228815a0 msgr2=0x55a52287f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:15.219 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.217+0000 7fa10f23f740 1 --2- 172.21.15.83:0/3863743883 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a5228815a0 0x55a52287f120 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc00efd0 tx=0x7fa0fc09ca50 comp rx=0 tx=0).stop 2024-04-24T08:52:15.219 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.217+0000 7fa10f23f740 1 -- 172.21.15.83:0/3863743883 shutdown_connections 2024-04-24T08:52:15.219 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.217+0000 7fa10f23f740 1 --2- 172.21.15.83:0/3863743883 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55a5228918a0 0x55a522893c90 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:15.219 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.217+0000 7fa10f23f740 1 --2- 172.21.15.83:0/3863743883 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a5228815a0 0x55a52287f120 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:15.219 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.217+0000 7fa10f23f740 1 --2- 172.21.15.83:0/3863743883 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a522876ce0 0x55a52287ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:15.219 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.217+0000 7fa10f23f740 1 --2- 172.21.15.83:0/3863743883 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a5228763c0 0x55a52287e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:15.219 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.217+0000 7fa10f23f740 1 -- 172.21.15.83:0/3863743883 >> 172.21.15.83:0/3863743883 conn(0x55a5228728d0 msgr2=0x55a522878600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:15.219 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.217+0000 7fa0eb400640 0 client.15873 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:52:15.219 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.218+0000 7fa10f23f740 1 -- 172.21.15.83:0/3863743883 shutdown_connections 2024-04-24T08:52:15.220 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.218+0000 7fa10f23f740 1 -- 172.21.15.83:0/3863743883 wait complete. 2024-04-24T08:52:15.220 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.218+0000 7fa10f23f740 20 client.15873 trim_cache size 0 max 16384 2024-04-24T08:52:15.236 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.234+0000 7fa10f23f740 1 Processor -- start 2024-04-24T08:52:15.236 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.234+0000 7fa10f23f740 1 -- start start 2024-04-24T08:52:15.236 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.234+0000 7fa10f23f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a5228763c0 0x55a522880dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:15.236 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.234+0000 7fa10f23f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a522876ce0 0x55a522872210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:15.236 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.234+0000 7fa10f23f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a5228815a0 0x55a522878d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:15.236 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.234+0000 7fa10f23f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa1041b1540 con 0x55a5228815a0 2024-04-24T08:52:15.236 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.234+0000 7fa10f23f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa1041b3080 con 0x55a522876ce0 2024-04-24T08:52:15.236 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.234+0000 7fa10f23f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa1041b1c10 con 0x55a5228763c0 2024-04-24T08:52:15.236 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.235+0000 7fa10a400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a5228815a0 0x55a522878d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:15.236 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.235+0000 7fa10a400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a5228815a0 0x55a522878d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55624/0 (socket says 172.21.15.83:55624) 2024-04-24T08:52:15.236 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.235+0000 7fa10a400640 1 -- 172.21.15.83:0/2181362071 learned_addr learned my addr 172.21.15.83:0/2181362071 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:52:15.236 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.235+0000 7fa109000640 1 --2- 172.21.15.83:0/2181362071 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a522876ce0 0x55a522872210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:15.237 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.235+0000 7fa10a400640 1 -- 172.21.15.83:0/2181362071 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a5228763c0 msgr2=0x55a522880dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:15.237 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.235+0000 7fa10a400640 1 --2- 172.21.15.83:0/2181362071 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a5228763c0 0x55a522880dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:15.237 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.235+0000 7fa10a400640 1 -- 172.21.15.83:0/2181362071 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a522876ce0 msgr2=0x55a522872210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:15.237 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.235+0000 7fa10a400640 1 --2- 172.21.15.83:0/2181362071 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a522876ce0 0x55a522872210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:15.237 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.235+0000 7fa10a400640 1 -- 172.21.15.83:0/2181362071 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5225bb620 con 0x55a5228815a0 2024-04-24T08:52:15.237 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.235+0000 7fa109000640 1 --2- 172.21.15.83:0/2181362071 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a522876ce0 0x55a522872210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:52:15.237 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.236+0000 7fa10a400640 1 --2- 172.21.15.83:0/2181362071 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a5228815a0 0x55a522878d80 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc09ad00 tx=0x7fa0fc09d150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b152573ae4732b22 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:15.237 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.236+0000 7fa0f6000640 1 -- 172.21.15.83:0/2181362071 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0fc01f070 con 0x55a5228815a0 2024-04-24T08:52:15.237 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.236+0000 7fa0f6000640 1 -- 172.21.15.83:0/2181362071 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa0fc066070 con 0x55a5228815a0 2024-04-24T08:52:15.237 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.236+0000 7fa0f6000640 1 -- 172.21.15.83:0/2181362071 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0fc0ce7f0 con 0x55a5228815a0 2024-04-24T08:52:15.240 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.236+0000 7fa10f23f740 1 -- 172.21.15.83:0/2181362071 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a5228815a0 msgr2=0x55a522878d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:15.240 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.236+0000 7fa10f23f740 1 --2- 172.21.15.83:0/2181362071 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a5228815a0 0x55a522878d80 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc09ad00 tx=0x7fa0fc09d150 comp rx=0 tx=0).stop 2024-04-24T08:52:15.240 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.236+0000 7fa10f23f740 1 -- 172.21.15.83:0/2181362071 shutdown_connections 2024-04-24T08:52:15.240 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.236+0000 7fa10f23f740 1 --2- 172.21.15.83:0/2181362071 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a5228815a0 0x55a522878d80 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:15.240 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.236+0000 7fa10f23f740 1 --2- 172.21.15.83:0/2181362071 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a522876ce0 0x55a522872210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:15.241 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.236+0000 7fa10f23f740 1 --2- 172.21.15.83:0/2181362071 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a5228763c0 0x55a522880dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:15.241 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.236+0000 7fa10f23f740 1 -- 172.21.15.83:0/2181362071 >> 172.21.15.83:0/2181362071 conn(0x55a5228728d0 msgr2=0x55a52287eef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:15.241 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.237+0000 7fa10f23f740 1 -- 172.21.15.83:0/2181362071 shutdown_connections 2024-04-24T08:52:15.241 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.237+0000 7fa10f23f740 1 -- 172.21.15.83:0/2181362071 wait complete. 2024-04-24T08:52:15.241 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.237+0000 7fa10f23f740 1 -- start start 2024-04-24T08:52:15.241 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.237+0000 7fa10f23f740 1 Processor -- start 2024-04-24T08:52:15.241 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.238+0000 7fa10f23f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a5228763c0 0x55a522885730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:15.241 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.238+0000 7fa10f23f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a522876ce0 0x55a522885c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:15.241 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.238+0000 7fa10f23f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a5228815a0 0x55a5228ff620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:15.241 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.238+0000 7fa10f23f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa1041b29b0 con 0x55a522876ce0 2024-04-24T08:52:15.241 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.238+0000 7fa10f23f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa1041b22e0 con 0x55a5228815a0 2024-04-24T08:52:15.241 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.238+0000 7fa10f23f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa1041af520 con 0x55a5228763c0 2024-04-24T08:52:15.241 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.238+0000 7fa10f23f740 10 client.0 _ll_register_callbacks cb 0x7fa0fc098cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:52:15.241 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.238+0000 7fa109000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a522876ce0 0x55a522885c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:15.242 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.238+0000 7fa109000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a522876ce0 0x55a522885c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55628/0 (socket says 172.21.15.83:55628) 2024-04-24T08:52:15.242 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.238+0000 7fa109000640 1 -- 172.21.15.83:0/317613081 learned_addr learned my addr 172.21.15.83:0/317613081 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:52:15.242 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.238+0000 7fa10a400640 1 --2- 172.21.15.83:0/317613081 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a5228815a0 0x55a5228ff620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:15.242 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.238+0000 7fa109000640 1 -- 172.21.15.83:0/317613081 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a5228763c0 msgr2=0x55a522885730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:15.242 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.238+0000 7fa109000640 1 --2- 172.21.15.83:0/317613081 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a5228763c0 0x55a522885730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:15.242 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.238+0000 7fa109000640 1 -- 172.21.15.83:0/317613081 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a5228815a0 msgr2=0x55a5228ff620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:15.242 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.238+0000 7fa109000640 1 --2- 172.21.15.83:0/317613081 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a5228815a0 0x55a5228ff620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:15.242 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.238+0000 7fa109000640 1 -- 172.21.15.83:0/317613081 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5228ffb60 con 0x55a522876ce0 2024-04-24T08:52:15.242 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.239+0000 7fa109000640 1 --2- 172.21.15.83:0/317613081 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a522876ce0 0x55a522885c70 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fa0f0014520 tx=0x7fa0f0005e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab360d5c2678ad67 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:15.242 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.239+0000 7fa0eb400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:52:15.242 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.239+0000 7fa0eb400640 1 -- 172.21.15.83:0/317613081 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f0014380 con 0x55a522876ce0 2024-04-24T08:52:15.242 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.239+0000 7fa0eb400640 1 -- 172.21.15.83:0/317613081 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7fa0f0005450 con 0x55a522876ce0 2024-04-24T08:52:15.242 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.239+0000 7fa10f23f740 1 -- 172.21.15.83:0/317613081 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0fc066070 con 0x55a522876ce0 2024-04-24T08:52:15.242 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.239+0000 7fa0eb400640 1 -- 172.21.15.83:0/317613081 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa0f002f050 con 0x55a522876ce0 2024-04-24T08:52:15.243 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.239+0000 7fa10f23f740 10 client.15879 fetch_fsmap learned FSMap version 28 2024-04-24T08:52:15.243 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.239+0000 7fa10f23f740 1 -- 172.21.15.83:0/317613081 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a522900e10 con 0x55a522876ce0 2024-04-24T08:52:15.243 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.240+0000 7fa0eb400640 1 -- 172.21.15.83:0/317613081 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa0fc066070 con 0x55a522876ce0 2024-04-24T08:52:15.243 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.240+0000 7fa0eb400640 1 -- 172.21.15.83:0/317613081 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa0f0012780 con 0x55a522876ce0 2024-04-24T08:52:15.243 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.240+0000 7fa10f23f740 10 client.15879 fetch_fsmap finished waiting for FSMap version 28 2024-04-24T08:52:15.243 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.240+0000 7fa10f23f740 10 client.15879 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T08:52:15.243 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.240+0000 7fa10f23f740 20 client.15879 populate_metadata read hostname 'smithi083' 2024-04-24T08:52:15.243 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.240+0000 7fa10f23f740 1 --2- 172.21.15.83:0/317613081 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55a522901250 0x55a522903640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:15.243 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.240+0000 7fa10f23f740 4 client.15879 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T08:52:15.243 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.240+0000 7fa10f23f740 1 -- 172.21.15.83:0/317613081 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a5227e4190 con 0x55a522901250 2024-04-24T08:52:15.244 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.241+0000 7fa109a00640 1 --2- 172.21.15.83:0/317613081 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55a522901250 0x55a522903640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:15.244 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.241+0000 7fa109a00640 1 --2- 172.21.15.83:0/317613081 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55a522901250 0x55a522903640 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ea60ca2f477c344 server_cookie=cac2d1c7cdad1aa3 in_seq=0 out_seq=0 2024-04-24T08:52:15.244 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.241+0000 7fa0eb400640 10 client.15879 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:52:15.251 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.251+0000 7fa0eb400640 1 -- 172.21.15.83:0/317613081 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a5227e4190 con 0x55a522901250 2024-04-24T08:52:15.251 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.251+0000 7fa0eb400640 10 client.15879 handle_command_reply: tid=0 2024-04-24T08:52:15.251 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.251+0000 7fa10f23f740 1 client.15879 shutdown 2024-04-24T08:52:15.252 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.251+0000 7fa10f23f740 1 -- 172.21.15.83:0/317613081 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a522876ce0 msgr2=0x55a522885c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:15.252 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.251+0000 7fa10f23f740 1 --2- 172.21.15.83:0/317613081 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a522876ce0 0x55a522885c70 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fa0f0014520 tx=0x7fa0f0005e00 comp rx=0 tx=0).stop 2024-04-24T08:52:15.252 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.251+0000 7fa10f23f740 1 -- 172.21.15.83:0/317613081 shutdown_connections 2024-04-24T08:52:15.252 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.251+0000 7fa10f23f740 1 --2- 172.21.15.83:0/317613081 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55a522901250 0x55a522903640 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:15.252 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.251+0000 7fa10f23f740 1 --2- 172.21.15.83:0/317613081 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a5228815a0 0x55a5228ff620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:15.252 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.251+0000 7fa0eb400640 0 client.15879 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:52:15.252 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.251+0000 7fa10f23f740 1 --2- 172.21.15.83:0/317613081 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a522876ce0 0x55a522885c70 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:15.252 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.251+0000 7fa10f23f740 1 --2- 172.21.15.83:0/317613081 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a5228763c0 0x55a522885730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:15.252 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.251+0000 7fa10f23f740 1 -- 172.21.15.83:0/317613081 >> 172.21.15.83:0/317613081 conn(0x55a5228728d0 msgr2=0x55a522871200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:15.252 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.251+0000 7fa10f23f740 1 -- 172.21.15.83:0/317613081 shutdown_connections 2024-04-24T08:52:15.252 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.251+0000 7fa10f23f740 1 -- 172.21.15.83:0/317613081 wait complete. 2024-04-24T08:52:15.253 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.252+0000 7fa10f23f740 20 client.15879 trim_cache size 0 max 16384 2024-04-24T08:52:15.253 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T08:52:15.253 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T08:52:15.253 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T08:52:15.253 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T08:52:15.254 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.252+0000 7fa10ae00640 1 -- 172.21.15.83:0/841477484 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa0ec0a3e30 msgr2=0x7fa0ec0a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:15.254 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.252+0000 7fa10ae00640 1 --2- 172.21.15.83:0/841477484 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa0ec0a3e30 0x7fa0ec0a62f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa0f8000c00 tx=0x7fa0f8005a20 comp rx=0 tx=0).stop 2024-04-24T08:52:15.254 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.252+0000 7fa10ae00640 1 -- 172.21.15.83:0/841477484 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa10410e650 msgr2=0x7fa10419a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:15.254 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.252+0000 7fa10ae00640 1 --2- 172.21.15.83:0/841477484 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa10410e650 0x7fa10419a610 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc00eaa0 tx=0x7fa0fc00ef60 comp rx=0 tx=0).stop 2024-04-24T08:52:15.254 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.252+0000 7fa10ae00640 1 -- 172.21.15.83:0/841477484 shutdown_connections 2024-04-24T08:52:15.254 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.252+0000 7fa10ae00640 1 --2- 172.21.15.83:0/841477484 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa0ec0a3e30 0x7fa0ec0a62f0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:15.254 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.252+0000 7fa10ae00640 1 --2- 172.21.15.83:0/841477484 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa10410e650 0x7fa10419a610 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:15.254 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.252+0000 7fa10ae00640 1 --2- 172.21.15.83:0/841477484 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa104074d50 0x7fa1041a0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:15.254 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.252+0000 7fa10ae00640 1 --2- 172.21.15.83:0/841477484 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa104074430 0x7fa1041a0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:15.254 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.252+0000 7fa10ae00640 1 -- 172.21.15.83:0/841477484 >> 172.21.15.83:0/841477484 conn(0x7fa1040fd940 msgr2=0x7fa1040ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:15.255 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.253+0000 7fa10ae00640 1 -- 172.21.15.83:0/841477484 shutdown_connections 2024-04-24T08:52:15.255 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:15.253+0000 7fa10ae00640 1 -- 172.21.15.83:0/841477484 wait complete. 2024-04-24T08:52:15.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:52:15 smithi083 ceph-mon[27907]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 24 KiB/s wr, 33 op/s 2024-04-24T08:52:15.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:52:15 smithi138 ceph-mon[31441]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 24 KiB/s wr, 33 op/s 2024-04-24T08:52:15.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:52:15 smithi189 ceph-mon[31791]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 24 KiB/s wr, 33 op/s 2024-04-24T08:52:16.868 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T08:52:16.868 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T08:52:17.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.078+0000 7f42ea000640 1 -- 172.21.15.83:0/555249423 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42e4103a40 msgr2=0x7f42e410bf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:17.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.078+0000 7f42ea000640 1 --2- 172.21.15.83:0/555249423 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42e4103a40 0x7f42e410bf20 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f42cc009900 tx=0x7f42cc02a480 comp rx=0 tx=0).stop 2024-04-24T08:52:17.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.079+0000 7f42ea000640 1 -- 172.21.15.83:0/555249423 shutdown_connections 2024-04-24T08:52:17.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.079+0000 7f42ea000640 1 --2- 172.21.15.83:0/555249423 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f42e410c4f0 0x7f42e410e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:17.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.079+0000 7f42ea000640 1 --2- 172.21.15.83:0/555249423 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42e4103a40 0x7f42e410bf20 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:17.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.079+0000 7f42ea000640 1 --2- 172.21.15.83:0/555249423 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f42e4103120 0x7f42e4103500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:17.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.079+0000 7f42ea000640 1 -- 172.21.15.83:0/555249423 >> 172.21.15.83:0/555249423 conn(0x7f42e40fc880 msgr2=0x7f42e40feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:17.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.079+0000 7f42ea000640 1 -- 172.21.15.83:0/555249423 shutdown_connections 2024-04-24T08:52:17.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.079+0000 7f42ea000640 1 -- 172.21.15.83:0/555249423 wait complete. 2024-04-24T08:52:17.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.080+0000 7f42ea000640 1 Processor -- start 2024-04-24T08:52:17.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.080+0000 7f42ea000640 1 -- start start 2024-04-24T08:52:17.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.080+0000 7f42ea000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f42e4103120 0x7f42e41a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:17.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.080+0000 7f42ea000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42e4103a40 0x7f42e41a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:17.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.080+0000 7f42ea000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f42e410c4f0 0x7f42e419a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:17.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.080+0000 7f42ea000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f42e4112160 con 0x7f42e4103a40 2024-04-24T08:52:17.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.080+0000 7f42ea000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f42e4111fe0 con 0x7f42e4103120 2024-04-24T08:52:17.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.080+0000 7f42ea000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f42e41122e0 con 0x7f42e410c4f0 2024-04-24T08:52:17.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.080+0000 7f42e3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42e4103a40 0x7f42e41a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:17.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.080+0000 7f42e3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42e4103a40 0x7f42e41a0b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55660/0 (socket says 172.21.15.83:55660) 2024-04-24T08:52:17.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.080+0000 7f42e3e00640 1 -- 172.21.15.83:0/3406009536 learned_addr learned my addr 172.21.15.83:0/3406009536 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:52:17.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.081+0000 7f42e8c00640 1 --2- 172.21.15.83:0/3406009536 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f42e4103120 0x7f42e41a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:17.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.081+0000 7f42e9600640 1 --2- 172.21.15.83:0/3406009536 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f42e410c4f0 0x7f42e419a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:17.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.081+0000 7f42e3e00640 1 -- 172.21.15.83:0/3406009536 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f42e410c4f0 msgr2=0x7f42e419a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:17.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.081+0000 7f42e3e00640 1 --2- 172.21.15.83:0/3406009536 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f42e410c4f0 0x7f42e419a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:17.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.081+0000 7f42e3e00640 1 -- 172.21.15.83:0/3406009536 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f42e4103120 msgr2=0x7f42e41a0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:17.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.081+0000 7f42e3e00640 1 --2- 172.21.15.83:0/3406009536 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f42e4103120 0x7f42e41a0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:17.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.081+0000 7f42e3e00640 1 -- 172.21.15.83:0/3406009536 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42e419b000 con 0x7f42e4103a40 2024-04-24T08:52:17.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.081+0000 7f42e3e00640 1 --2- 172.21.15.83:0/3406009536 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42e4103a40 0x7f42e41a0b70 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f42cc005ab0 tx=0x7f42cc003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:17.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.081+0000 7f42e1600640 1 -- 172.21.15.83:0/3406009536 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42cc03f070 con 0x7f42e4103a40 2024-04-24T08:52:17.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.081+0000 7f42e1600640 1 -- 172.21.15.83:0/3406009536 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f42cc031d60 con 0x7f42e4103a40 2024-04-24T08:52:17.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.081+0000 7f42e1600640 1 -- 172.21.15.83:0/3406009536 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42cc035030 con 0x7f42e4103a40 2024-04-24T08:52:17.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.082+0000 7f42ea000640 1 -- 172.21.15.83:0/3406009536 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42e419b290 con 0x7f42e4103a40 2024-04-24T08:52:17.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.082+0000 7f42ea000640 1 -- 172.21.15.83:0/3406009536 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42e41a73d0 con 0x7f42e4103a40 2024-04-24T08:52:17.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.085+0000 7f42ee5de740 1 Processor -- start 2024-04-24T08:52:17.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.085+0000 7f42e1600640 1 -- 172.21.15.83:0/3406009536 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7f42cc042070 con 0x7f42e4103a40 2024-04-24T08:52:17.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.085+0000 7f42ee5de740 1 -- start start 2024-04-24T08:52:17.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.085+0000 7f42ee5de740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561307e73a30 0x561307e73e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:17.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.085+0000 7f42ee5de740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561307e74350 0x561307e7e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:17.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.085+0000 7f42ee5de740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561307e7ea20 0x561307e80e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:17.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.085+0000 7f42ee5de740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561307dd91f0 con 0x561307e73a30 2024-04-24T08:52:17.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.085+0000 7f42ee5de740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x561307d7f350 con 0x561307e74350 2024-04-24T08:52:17.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.085+0000 7f42ee5de740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x561307dd51b0 con 0x561307e7ea20 2024-04-24T08:52:17.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.086+0000 7f42e9600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561307e73a30 0x561307e73e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:17.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.086+0000 7f42e9600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561307e73a30 0x561307e73e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55676/0 (socket says 172.21.15.83:55676) 2024-04-24T08:52:17.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.086+0000 7f42e9600640 1 -- 172.21.15.83:0/2197430204 learned_addr learned my addr 172.21.15.83:0/2197430204 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:52:17.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.086+0000 7f42e8c00640 1 --2- 172.21.15.83:0/2197430204 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561307e74350 0x561307e7e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:17.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.086+0000 7f42e9600640 1 -- 172.21.15.83:0/2197430204 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561307e7ea20 msgr2=0x561307e80e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:17.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.086+0000 7f42e9600640 1 --2- 172.21.15.83:0/2197430204 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561307e7ea20 0x561307e80e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:17.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.086+0000 7f42e9600640 1 -- 172.21.15.83:0/2197430204 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561307e74350 msgr2=0x561307e7e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:17.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.086+0000 7f42e9600640 1 --2- 172.21.15.83:0/2197430204 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561307e74350 0x561307e7e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:17.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.086+0000 7f42e9600640 1 -- 172.21.15.83:0/2197430204 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561307dd3f50 con 0x561307e73a30 2024-04-24T08:52:17.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.086+0000 7f42e8c00640 1 --2- 172.21.15.83:0/2197430204 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561307e74350 0x561307e7e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:52:17.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.086+0000 7f42e1600640 1 --2- 172.21.15.83:0/3406009536 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f42c80a3ec0 0x7f42c80a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:17.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.086+0000 7f42e9600640 1 --2- 172.21.15.83:0/2197430204 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561307e73a30 0x561307e73e10 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f42d400bb70 tx=0x7f42d400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2de0ddd754861927 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:17.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.086+0000 7f42e1600640 1 -- 172.21.15.83:0/3406009536 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f42cc0f1740 con 0x7f42e4103a40 2024-04-24T08:52:17.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.087+0000 7f42e8c00640 1 --2- 172.21.15.83:0/3406009536 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f42c80a3ec0 0x7f42c80a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:17.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.087+0000 7f42d9c00640 1 -- 172.21.15.83:0/2197430204 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d4015070 con 0x561307e73a30 2024-04-24T08:52:17.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.087+0000 7f42d9c00640 1 -- 172.21.15.83:0/2197430204 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f42d40054d0 con 0x561307e73a30 2024-04-24T08:52:17.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.087+0000 7f42d9c00640 1 -- 172.21.15.83:0/2197430204 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d4011440 con 0x561307e73a30 2024-04-24T08:52:17.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.087+0000 7f42ee5de740 1 -- 172.21.15.83:0/2197430204 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561307e73a30 msgr2=0x561307e73e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:17.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.087+0000 7f42ee5de740 1 --2- 172.21.15.83:0/2197430204 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561307e73a30 0x561307e73e10 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f42d400bb70 tx=0x7f42d400f730 comp rx=0 tx=0).stop 2024-04-24T08:52:17.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.087+0000 7f42e8c00640 1 --2- 172.21.15.83:0/3406009536 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f42c80a3ec0 0x7f42c80a6380 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x561307e74ff0 tx=0x7f42dc009500 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:17.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.088+0000 7f42ee5de740 1 -- 172.21.15.83:0/2197430204 shutdown_connections 2024-04-24T08:52:17.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.088+0000 7f42ee5de740 1 --2- 172.21.15.83:0/2197430204 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561307e7ea20 0x561307e80e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:17.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.088+0000 7f42ee5de740 1 --2- 172.21.15.83:0/2197430204 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561307e74350 0x561307e7e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:17.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.088+0000 7f42ee5de740 1 --2- 172.21.15.83:0/2197430204 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561307e73a30 0x561307e73e10 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:17.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.088+0000 7f42ee5de740 1 -- 172.21.15.83:0/2197430204 >> 172.21.15.83:0/2197430204 conn(0x561307e6fd50 msgr2=0x561307e70130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:17.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.088+0000 7f42ee5de740 1 -- 172.21.15.83:0/2197430204 shutdown_connections 2024-04-24T08:52:17.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.088+0000 7f42ee5de740 1 -- 172.21.15.83:0/2197430204 wait complete. 2024-04-24T08:52:17.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.088+0000 7f42ee5de740 1 -- start start 2024-04-24T08:52:17.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.088+0000 7f42ee5de740 1 Processor -- start 2024-04-24T08:52:17.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.089+0000 7f42ee5de740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561307e73a30 0x561307e7b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:17.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.089+0000 7f42ee5de740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561307e74350 0x561307e7bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:17.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.089+0000 7f42ee5de740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561307e7ea20 0x561307e7c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:17.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.089+0000 7f42ee5de740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561307e81810 con 0x561307e74350 2024-04-24T08:52:17.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.089+0000 7f42ee5de740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x561307d92690 con 0x561307e7ea20 2024-04-24T08:52:17.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.089+0000 7f42ee5de740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x561307e81990 con 0x561307e73a30 2024-04-24T08:52:17.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.089+0000 7f42ee5de740 10 client.0 _ll_register_callbacks cb 0x561307dce650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:52:17.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.089+0000 7f42e3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561307e74350 0x561307e7bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:17.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.089+0000 7f42e3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561307e74350 0x561307e7bef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55680/0 (socket says 172.21.15.83:55680) 2024-04-24T08:52:17.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.089+0000 7f42e3e00640 1 -- 172.21.15.83:0/4180453441 learned_addr learned my addr 172.21.15.83:0/4180453441 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:52:17.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.089+0000 7f42e9600640 1 --2- 172.21.15.83:0/4180453441 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561307e7ea20 0x561307e7c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:17.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.090+0000 7f42e3e00640 1 -- 172.21.15.83:0/4180453441 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561307e73a30 msgr2=0x561307e7b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:17.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.090+0000 7f42e3e00640 1 --2- 172.21.15.83:0/4180453441 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561307e73a30 0x561307e7b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:17.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.090+0000 7f42e3e00640 1 -- 172.21.15.83:0/4180453441 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561307e7ea20 msgr2=0x561307e7c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:17.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.090+0000 7f42e3e00640 1 --2- 172.21.15.83:0/4180453441 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561307e7ea20 0x561307e7c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:17.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.090+0000 7f42e3e00640 1 -- 172.21.15.83:0/4180453441 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561307d91920 con 0x561307e74350 2024-04-24T08:52:17.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.090+0000 7f42e3e00640 1 --2- 172.21.15.83:0/4180453441 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561307e74350 0x561307e7bef0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f42cc000c00 tx=0x7f42cc0bcfc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fee1386e6cfeb305 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:17.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.090+0000 7f42d2000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:52:17.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.090+0000 7f42d2000640 1 -- 172.21.15.83:0/4180453441 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42cc049070 con 0x561307e74350 2024-04-24T08:52:17.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.090+0000 7f42d2000640 1 -- 172.21.15.83:0/4180453441 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f42cc03b5d0 con 0x561307e74350 2024-04-24T08:52:17.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.090+0000 7f42ee5de740 1 -- 172.21.15.83:0/4180453441 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42d40054d0 con 0x561307e74350 2024-04-24T08:52:17.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.090+0000 7f42d2000640 1 -- 172.21.15.83:0/4180453441 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f42cc0bd830 con 0x561307e74350 2024-04-24T08:52:17.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.090+0000 7f42ee5de740 10 client.15891 fetch_fsmap learned FSMap version 28 2024-04-24T08:52:17.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.090+0000 7f42ee5de740 1 -- 172.21.15.83:0/4180453441 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561307bb57d0 con 0x561307e74350 2024-04-24T08:52:17.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.091+0000 7f42d2000640 1 -- 172.21.15.83:0/4180453441 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f42d40054d0 con 0x561307e74350 2024-04-24T08:52:17.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.091+0000 7f42d2000640 1 -- 172.21.15.83:0/4180453441 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f42cc0dd040 con 0x561307e74350 2024-04-24T08:52:17.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.091+0000 7f42ee5de740 10 client.15891 fetch_fsmap finished waiting for FSMap version 28 2024-04-24T08:52:17.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.091+0000 7f42ee5de740 10 client.15891 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:52:17.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.091+0000 7f42ee5de740 20 client.15891 populate_metadata read hostname 'smithi083' 2024-04-24T08:52:17.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.091+0000 7f42ee5de740 1 --2- 172.21.15.83:0/4180453441 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x561307e8eba0 0x561307e90ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:17.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.091+0000 7f42ee5de740 4 client.15891 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:52:17.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.091+0000 7f42ee5de740 1 -- 172.21.15.83:0/4180453441 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561307de6c10 con 0x561307e8eba0 2024-04-24T08:52:17.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.092+0000 7f42e8c00640 1 --2- 172.21.15.83:0/4180453441 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x561307e8eba0 0x561307e90ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:17.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.093+0000 7f42e8c00640 1 --2- 172.21.15.83:0/4180453441 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x561307e8eba0 0x561307e90ff0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8415d5755a8ca44 server_cookie=67dce0840cbee9f in_seq=0 out_seq=0 2024-04-24T08:52:17.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.093+0000 7f42d2000640 10 client.15891 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:52:17.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.095+0000 7f42d2000640 1 -- 172.21.15.83:0/4180453441 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561307de6c10 con 0x561307e8eba0 2024-04-24T08:52:17.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.095+0000 7f42d2000640 10 client.15891 handle_command_reply: tid=0 2024-04-24T08:52:17.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.095+0000 7f42ee5de740 1 client.15891 shutdown 2024-04-24T08:52:17.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.095+0000 7f42ee5de740 1 -- 172.21.15.83:0/4180453441 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561307e74350 msgr2=0x561307e7bef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:17.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.095+0000 7f42ee5de740 1 --2- 172.21.15.83:0/4180453441 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561307e74350 0x561307e7bef0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f42cc000c00 tx=0x7f42cc0bcfc0 comp rx=0 tx=0).stop 2024-04-24T08:52:17.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.095+0000 7f42ee5de740 1 -- 172.21.15.83:0/4180453441 shutdown_connections 2024-04-24T08:52:17.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.095+0000 7f42ee5de740 1 --2- 172.21.15.83:0/4180453441 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x561307e8eba0 0x561307e90ff0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:17.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.095+0000 7f42ee5de740 1 --2- 172.21.15.83:0/4180453441 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561307e7ea20 0x561307e7c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:17.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.095+0000 7f42ee5de740 1 --2- 172.21.15.83:0/4180453441 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561307e74350 0x561307e7bef0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:17.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.095+0000 7f42d2000640 0 client.15891 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:52:17.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.095+0000 7f42ee5de740 1 --2- 172.21.15.83:0/4180453441 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561307e73a30 0x561307e7b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:17.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.095+0000 7f42ee5de740 1 -- 172.21.15.83:0/4180453441 >> 172.21.15.83:0/4180453441 conn(0x561307e6fd50 msgr2=0x561307e6efd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:17.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.095+0000 7f42ee5de740 1 -- 172.21.15.83:0/4180453441 shutdown_connections 2024-04-24T08:52:17.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.095+0000 7f42ee5de740 1 -- 172.21.15.83:0/4180453441 wait complete. 2024-04-24T08:52:17.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.095+0000 7f42ee5de740 20 client.15891 trim_cache size 0 max 16384 2024-04-24T08:52:17.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.114+0000 7f42ee5de740 1 Processor -- start 2024-04-24T08:52:17.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.114+0000 7f42ee5de740 1 -- start start 2024-04-24T08:52:17.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.114+0000 7f42ee5de740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561307e73a30 0x561307e8a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:17.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.114+0000 7f42ee5de740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561307e74350 0x561307e7d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:17.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.115+0000 7f42ee5de740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561307e7ea20 0x561307e968c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:17.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.115+0000 7f42ee5de740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f42e41b1540 con 0x561307e7ea20 2024-04-24T08:52:17.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.115+0000 7f42ee5de740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f42e41b3080 con 0x561307e74350 2024-04-24T08:52:17.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.115+0000 7f42ee5de740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f42e41b1c10 con 0x561307e73a30 2024-04-24T08:52:17.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.115+0000 7f42e9600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561307e7ea20 0x561307e968c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:17.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.115+0000 7f42e9600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561307e7ea20 0x561307e968c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55684/0 (socket says 172.21.15.83:55684) 2024-04-24T08:52:17.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.115+0000 7f42e9600640 1 -- 172.21.15.83:0/4120812976 learned_addr learned my addr 172.21.15.83:0/4120812976 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:52:17.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.115+0000 7f42e9600640 1 -- 172.21.15.83:0/4120812976 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561307e73a30 msgr2=0x561307e8a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:17.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.115+0000 7f42e8c00640 1 --2- 172.21.15.83:0/4120812976 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561307e73a30 0x561307e8a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:17.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.115+0000 7f42e9600640 1 --2- 172.21.15.83:0/4120812976 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561307e73a30 0x561307e8a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:17.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.115+0000 7f42e9600640 1 -- 172.21.15.83:0/4120812976 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561307e74350 msgr2=0x561307e7d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:17.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.115+0000 7f42e9600640 1 --2- 172.21.15.83:0/4120812976 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561307e74350 0x561307e7d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:17.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.115+0000 7f42e9600640 1 -- 172.21.15.83:0/4120812976 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561307aeb3f0 con 0x561307e7ea20 2024-04-24T08:52:17.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.115+0000 7f42e8c00640 1 --2- 172.21.15.83:0/4120812976 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561307e73a30 0x561307e8a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:52:17.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.116+0000 7f42e9600640 1 --2- 172.21.15.83:0/4120812976 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561307e7ea20 0x561307e968c0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f42d400b8f0 tx=0x7f42d40058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfb6539ab77f4da5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:17.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.116+0000 7f42d9200640 1 -- 172.21.15.83:0/4120812976 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d4015070 con 0x561307e7ea20 2024-04-24T08:52:17.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.116+0000 7f42d9200640 1 -- 172.21.15.83:0/4120812976 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f42d400be10 con 0x561307e7ea20 2024-04-24T08:52:17.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.116+0000 7f42d9200640 1 -- 172.21.15.83:0/4120812976 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d40116a0 con 0x561307e7ea20 2024-04-24T08:52:17.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.116+0000 7f42ee5de740 1 -- 172.21.15.83:0/4120812976 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561307e7ea20 msgr2=0x561307e968c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:17.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.116+0000 7f42ee5de740 1 --2- 172.21.15.83:0/4120812976 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561307e7ea20 0x561307e968c0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f42d400b8f0 tx=0x7f42d40058a0 comp rx=0 tx=0).stop 2024-04-24T08:52:17.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.117+0000 7f42ee5de740 1 -- 172.21.15.83:0/4120812976 shutdown_connections 2024-04-24T08:52:17.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.117+0000 7f42ee5de740 1 --2- 172.21.15.83:0/4120812976 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561307e7ea20 0x561307e968c0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:17.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.117+0000 7f42ee5de740 1 --2- 172.21.15.83:0/4120812976 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561307e74350 0x561307e7d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:17.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.117+0000 7f42ee5de740 1 --2- 172.21.15.83:0/4120812976 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561307e73a30 0x561307e8a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:17.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.117+0000 7f42ee5de740 1 -- 172.21.15.83:0/4120812976 >> 172.21.15.83:0/4120812976 conn(0x561307e6fd50 msgr2=0x561307e7d1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:17.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.117+0000 7f42ee5de740 1 -- 172.21.15.83:0/4120812976 shutdown_connections 2024-04-24T08:52:17.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.117+0000 7f42ee5de740 1 -- 172.21.15.83:0/4120812976 wait complete. 2024-04-24T08:52:17.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.117+0000 7f42ee5de740 1 -- start start 2024-04-24T08:52:17.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.117+0000 7f42ee5de740 1 Processor -- start 2024-04-24T08:52:17.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.118+0000 7f42ee5de740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561307e73a30 0x561307e78790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:17.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.118+0000 7f42ee5de740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561307e74350 0x561307e78cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:17.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.118+0000 7f42ee5de740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561307e7ea20 0x561307e79210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:17.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.118+0000 7f42ee5de740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f42e41b29b0 con 0x561307e7ea20 2024-04-24T08:52:17.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.118+0000 7f42ee5de740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f42e41b22e0 con 0x561307e73a30 2024-04-24T08:52:17.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.118+0000 7f42ee5de740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f42e41af520 con 0x561307e74350 2024-04-24T08:52:17.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.118+0000 7f42ee5de740 10 client.0 _ll_register_callbacks cb 0x7f42cc003ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:52:17.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.118+0000 7f42e9600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561307e7ea20 0x561307e79210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:17.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.118+0000 7f42e9600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561307e7ea20 0x561307e79210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55692/0 (socket says 172.21.15.83:55692) 2024-04-24T08:52:17.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.118+0000 7f42e9600640 1 -- 172.21.15.83:0/839904801 learned_addr learned my addr 172.21.15.83:0/839904801 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:52:17.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.118+0000 7f42e9600640 1 -- 172.21.15.83:0/839904801 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561307e74350 msgr2=0x561307e78cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:17.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.118+0000 7f42e9600640 1 --2- 172.21.15.83:0/839904801 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561307e74350 0x561307e78cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:17.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.118+0000 7f42e9600640 1 -- 172.21.15.83:0/839904801 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561307e73a30 msgr2=0x561307e78790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:17.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.118+0000 7f42e9600640 1 --2- 172.21.15.83:0/839904801 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561307e73a30 0x561307e78790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:17.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.118+0000 7f42e9600640 1 -- 172.21.15.83:0/839904801 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561307aab0f0 con 0x561307e7ea20 2024-04-24T08:52:17.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.119+0000 7f42e9600640 1 --2- 172.21.15.83:0/839904801 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561307e7ea20 0x561307e79210 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f42d4005870 tx=0x7f42d40049b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f132cc9737643231 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:17.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.119+0000 7f42d2000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:52:17.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.119+0000 7f42d2000640 1 -- 172.21.15.83:0/839904801 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d4015030 con 0x561307e7ea20 2024-04-24T08:52:17.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.119+0000 7f42d2000640 1 -- 172.21.15.83:0/839904801 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f42d4005050 con 0x561307e7ea20 2024-04-24T08:52:17.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.119+0000 7f42ee5de740 1 -- 172.21.15.83:0/839904801 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42d400be10 con 0x561307e7ea20 2024-04-24T08:52:17.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.119+0000 7f42d2000640 1 -- 172.21.15.83:0/839904801 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f42d4033050 con 0x561307e7ea20 2024-04-24T08:52:17.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.119+0000 7f42ee5de740 10 client.15897 fetch_fsmap learned FSMap version 28 2024-04-24T08:52:17.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.119+0000 7f42ee5de740 1 -- 172.21.15.83:0/839904801 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561307d9e810 con 0x561307e7ea20 2024-04-24T08:52:17.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.119+0000 7f42d2000640 1 -- 172.21.15.83:0/839904801 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f42d400be10 con 0x561307e7ea20 2024-04-24T08:52:17.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.120+0000 7f42d2000640 1 -- 172.21.15.83:0/839904801 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f42d4011c00 con 0x561307e7ea20 2024-04-24T08:52:17.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.120+0000 7f42ee5de740 10 client.15897 fetch_fsmap finished waiting for FSMap version 28 2024-04-24T08:52:17.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.120+0000 7f42ee5de740 10 client.15897 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:52:17.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.120+0000 7f42ee5de740 20 client.15897 populate_metadata read hostname 'smithi083' 2024-04-24T08:52:17.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.120+0000 7f42ee5de740 1 --2- 172.21.15.83:0/839904801 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x561307ef55f0 0x561307ef7a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:17.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.120+0000 7f42ee5de740 4 client.15897 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T08:52:17.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.120+0000 7f42ee5de740 1 -- 172.21.15.83:0/839904801 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561307de0a80 con 0x561307ef55f0 2024-04-24T08:52:17.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.120+0000 7f42e8c00640 1 --2- 172.21.15.83:0/839904801 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x561307ef55f0 0x561307ef7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:17.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.121+0000 7f42e8c00640 1 --2- 172.21.15.83:0/839904801 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x561307ef55f0 0x561307ef7a30 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79e847c299df4ca9 server_cookie=840c8657145fad24 in_seq=0 out_seq=0 2024-04-24T08:52:17.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.121+0000 7f42d2000640 10 client.15897 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:52:17.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.122+0000 7f42d2000640 1 -- 172.21.15.83:0/839904801 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x561307de0a80 con 0x561307ef55f0 2024-04-24T08:52:17.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.122+0000 7f42d2000640 10 client.15897 handle_command_reply: tid=0 2024-04-24T08:52:17.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.122+0000 7f42ee5de740 1 client.15897 shutdown 2024-04-24T08:52:17.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.122+0000 7f42ee5de740 1 -- 172.21.15.83:0/839904801 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561307e7ea20 msgr2=0x561307e79210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:17.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.122+0000 7f42ee5de740 1 --2- 172.21.15.83:0/839904801 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561307e7ea20 0x561307e79210 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f42d4005870 tx=0x7f42d40049b0 comp rx=0 tx=0).stop 2024-04-24T08:52:17.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.122+0000 7f42ee5de740 1 -- 172.21.15.83:0/839904801 shutdown_connections 2024-04-24T08:52:17.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.122+0000 7f42ee5de740 1 --2- 172.21.15.83:0/839904801 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x561307ef55f0 0x561307ef7a30 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:17.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.122+0000 7f42ee5de740 1 --2- 172.21.15.83:0/839904801 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561307e7ea20 0x561307e79210 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:17.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.122+0000 7f42ee5de740 1 --2- 172.21.15.83:0/839904801 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561307e74350 0x561307e78cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:17.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.122+0000 7f42d2000640 0 client.15897 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:52:17.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.122+0000 7f42ee5de740 1 --2- 172.21.15.83:0/839904801 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561307e73a30 0x561307e78790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:17.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.122+0000 7f42ee5de740 1 -- 172.21.15.83:0/839904801 >> 172.21.15.83:0/839904801 conn(0x561307e6fd50 msgr2=0x561307e7da70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:17.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.122+0000 7f42ee5de740 1 -- 172.21.15.83:0/839904801 shutdown_connections 2024-04-24T08:52:17.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.122+0000 7f42ee5de740 1 -- 172.21.15.83:0/839904801 wait complete. 2024-04-24T08:52:17.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.122+0000 7f42ee5de740 20 client.15897 trim_cache size 0 max 16384 2024-04-24T08:52:17.126 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T08:52:17.126 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-04-24T08:52:17.126 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-04-24T08:52:17.127 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T08:52:17.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.123+0000 7f42ea000640 1 -- 172.21.15.83:0/3406009536 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f42c80a3ec0 msgr2=0x7f42c80a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:17.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.123+0000 7f42ea000640 1 --2- 172.21.15.83:0/3406009536 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f42c80a3ec0 0x7f42c80a6380 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x561307e74ff0 tx=0x7f42dc009500 comp rx=0 tx=0).stop 2024-04-24T08:52:17.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.123+0000 7f42ea000640 1 -- 172.21.15.83:0/3406009536 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42e4103a40 msgr2=0x7f42e41a0b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:17.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.123+0000 7f42ea000640 1 --2- 172.21.15.83:0/3406009536 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42e4103a40 0x7f42e41a0b70 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f42cc005ab0 tx=0x7f42cc003cd0 comp rx=0 tx=0).stop 2024-04-24T08:52:17.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.123+0000 7f42ea000640 1 -- 172.21.15.83:0/3406009536 shutdown_connections 2024-04-24T08:52:17.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.123+0000 7f42ea000640 1 --2- 172.21.15.83:0/3406009536 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f42c80a3ec0 0x7f42c80a6380 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:17.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.123+0000 7f42ea000640 1 --2- 172.21.15.83:0/3406009536 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f42e410c4f0 0x7f42e419a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:17.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.123+0000 7f42ea000640 1 --2- 172.21.15.83:0/3406009536 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42e4103a40 0x7f42e41a0b70 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:17.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.123+0000 7f42ea000640 1 --2- 172.21.15.83:0/3406009536 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f42e4103120 0x7f42e41a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:17.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.123+0000 7f42ea000640 1 -- 172.21.15.83:0/3406009536 >> 172.21.15.83:0/3406009536 conn(0x7f42e40fc880 msgr2=0x7f42e4100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:17.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.123+0000 7f42ea000640 1 -- 172.21.15.83:0/3406009536 shutdown_connections 2024-04-24T08:52:17.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.123+0000 7f42ea000640 1 -- 172.21.15.83:0/3406009536 wait complete. 2024-04-24T08:52:17.141 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T08:52:17.141 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T08:52:17.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.327+0000 7f9a7a600640 1 -- 172.21.15.83:0/2928966791 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a74108e40 msgr2=0x7f9a741152b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:17.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.327+0000 7f9a7a600640 1 --2- 172.21.15.83:0/2928966791 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a74108e40 0x7f9a741152b0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c009ca0 tx=0x7f9a6c02a950 comp rx=0 tx=0).stop 2024-04-24T08:52:17.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.327+0000 7f9a7a600640 1 -- 172.21.15.83:0/2928966791 shutdown_connections 2024-04-24T08:52:17.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.327+0000 7f9a7a600640 1 --2- 172.21.15.83:0/2928966791 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a74108e40 0x7f9a741152b0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:17.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.327+0000 7f9a7a600640 1 --2- 172.21.15.83:0/2928966791 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a741084a0 0x7f9a74108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:17.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.327+0000 7f9a7a600640 1 --2- 172.21.15.83:0/2928966791 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9a7410d3c0 0x7f9a7410d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:17.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.327+0000 7f9a7a600640 1 -- 172.21.15.83:0/2928966791 >> 172.21.15.83:0/2928966791 conn(0x7f9a74077b80 msgr2=0x7f9a74079fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:17.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.327+0000 7f9a7a600640 1 -- 172.21.15.83:0/2928966791 shutdown_connections 2024-04-24T08:52:17.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.327+0000 7f9a7a600640 1 -- 172.21.15.83:0/2928966791 wait complete. 2024-04-24T08:52:17.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.328+0000 7f9a7a600640 1 Processor -- start 2024-04-24T08:52:17.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.329+0000 7f9a7a600640 1 -- start start 2024-04-24T08:52:17.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.329+0000 7f9a7a600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a741084a0 0x7f9a741a47b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:17.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.329+0000 7f9a7a600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a74108e40 0x7f9a741a4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:17.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.329+0000 7f9a7a600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9a7410d3c0 0x7f9a7419e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:17.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.329+0000 7f9a7a600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9a74118700 con 0x7f9a74108e40 2024-04-24T08:52:17.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.329+0000 7f9a7a600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9a74118580 con 0x7f9a741084a0 2024-04-24T08:52:17.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.329+0000 7f9a7a600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9a74118880 con 0x7f9a7410d3c0 2024-04-24T08:52:17.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.329+0000 7f9a73e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a74108e40 0x7f9a741a4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:17.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.329+0000 7f9a73e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a74108e40 0x7f9a741a4cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55706/0 (socket says 172.21.15.83:55706) 2024-04-24T08:52:17.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.329+0000 7f9a73e00640 1 -- 172.21.15.83:0/1122855404 learned_addr learned my addr 172.21.15.83:0/1122855404 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:52:17.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.330+0000 7f9a79c00640 1 --2- 172.21.15.83:0/1122855404 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9a7410d3c0 0x7f9a7419e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:17.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.330+0000 7f9a79200640 1 --2- 172.21.15.83:0/1122855404 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a741084a0 0x7f9a741a47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:17.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.330+0000 7f9a73e00640 1 -- 172.21.15.83:0/1122855404 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9a7410d3c0 msgr2=0x7f9a7419e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:17.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.330+0000 7f9a73e00640 1 --2- 172.21.15.83:0/1122855404 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9a7410d3c0 0x7f9a7419e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:17.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.330+0000 7f9a73e00640 1 -- 172.21.15.83:0/1122855404 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a741084a0 msgr2=0x7f9a741a47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:17.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.330+0000 7f9a73e00640 1 --2- 172.21.15.83:0/1122855404 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a741084a0 0x7f9a741a47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:17.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.330+0000 7f9a73e00640 1 -- 172.21.15.83:0/1122855404 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a7419f0e0 con 0x7f9a74108e40 2024-04-24T08:52:17.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.330+0000 7f9a79200640 1 --2- 172.21.15.83:0/1122855404 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a741084a0 0x7f9a741a47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:52:17.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.331+0000 7f9a73e00640 1 --2- 172.21.15.83:0/1122855404 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a74108e40 0x7f9a741a4cf0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f9a6000c920 tx=0x7f9a6000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:17.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.331+0000 7f9a71600640 1 -- 172.21.15.83:0/1122855404 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a60007c70 con 0x7f9a74108e40 2024-04-24T08:52:17.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.331+0000 7f9a7a600640 1 -- 172.21.15.83:0/1122855404 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a7419f3d0 con 0x7f9a74108e40 2024-04-24T08:52:17.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.331+0000 7f9a71600640 1 -- 172.21.15.83:0/1122855404 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9a60007e10 con 0x7f9a74108e40 2024-04-24T08:52:17.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.331+0000 7f9a71600640 1 -- 172.21.15.83:0/1122855404 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a60018820 con 0x7f9a74108e40 2024-04-24T08:52:17.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.331+0000 7f9a7a600640 1 -- 172.21.15.83:0/1122855404 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a741ab670 con 0x7f9a74108e40 2024-04-24T08:52:17.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.332+0000 7f9a7a600640 1 -- 172.21.15.83:0/1122855404 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a74069190 con 0x7f9a74108e40 2024-04-24T08:52:17.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.334+0000 7f9a71600640 1 -- 172.21.15.83:0/1122855404 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7f9a600189c0 con 0x7f9a74108e40 2024-04-24T08:52:17.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.336+0000 7f9a71600640 1 --2- 172.21.15.83:0/1122855404 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f9a5c0a3e40 0x7f9a5c0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:17.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.336+0000 7f9a71600640 1 -- 172.21.15.83:0/1122855404 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f9a600d1320 con 0x7f9a74108e40 2024-04-24T08:52:17.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.336+0000 7f9a79200640 1 --2- 172.21.15.83:0/1122855404 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f9a5c0a3e40 0x7f9a5c0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:17.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.337+0000 7f9a79200640 1 --2- 172.21.15.83:0/1122855404 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f9a5c0a3e40 0x7f9a5c0a6300 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9a68005fd0 tx=0x7f9a68005cf0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:17.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.339+0000 7f9a71600640 1 -- 172.21.15.83:0/1122855404 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9a60014030 con 0x7f9a74108e40 2024-04-24T08:52:17.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.586+0000 7f9a7a600640 1 -- 172.21.15.83:0/1122855404 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9a7419fe90 con 0x7f9a74108e40 2024-04-24T08:52:17.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.587+0000 7f9a71600640 1 -- 172.21.15.83:0/1122855404 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11944 (secure 0 0 0) 0x7f9a6009d3d0 con 0x7f9a74108e40 2024-04-24T08:52:17.588 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:52:17.588 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/358872257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":358872257},{"type":"v1","addr":"172.21.15.83:6835","nonce":358872257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3157070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3157070851},{"type":"v1","addr":"172.21.15.83:6837","nonce":3157070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T08:48:29.422576+0000","modified":"2024-04-24T08:51:36.355488+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34520,"mds_1":34493,"mds_2":34451,"mds_3":34427,"mds_4":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14658":{"gid":14658,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6839/4085470844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":4085470844},{"type":"v1","addr":"172.21.15.83:6839","nonce":4085470844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14676":{"gid":14676,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6841/2634754210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":2634754210},{"type":"v1","addr":"172.21.15.83:6841","nonce":2634754210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24382":{"gid":24382,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/1859921310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1859921310},{"type":"v1","addr":"172.21.15.189:6833","nonce":1859921310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/2241923197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2241923197},{"type":"v1","addr":"172.21.15.189:6835","nonce":2241923197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6837/3172311220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":3172311220},{"type":"v1","addr":"172.21.15.189:6837","nonce":3172311220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.189:6839/2548965336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":2548965336},{"type":"v1","addr":"172.21.15.189:6839","nonce":2548965336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.138:6833/1157967658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1157967658},{"type":"v1","addr":"172.21.15.138:6833","nonce":1157967658}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.138:6835/1909219758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1909219758},{"type":"v1","addr":"172.21.15.138:6835","nonce":1909219758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34493":{"gid":34493,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.138:6837/900566754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":900566754},{"type":"v1","addr":"172.21.15.138:6837","nonce":900566754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34520":{"gid":34520,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.138:6839/2236932393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2236932393},{"type":"v1","addr":"172.21.15.138:6839","nonce":2236932393}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34520,"qdb_cluster":[34451,34493,34427,34520,24454]},"id":1}]} 2024-04-24T08:52:17.588 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 28 2024-04-24T08:52:17.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.590+0000 7f9a7a600640 1 -- 172.21.15.83:0/1122855404 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f9a5c0a3e40 msgr2=0x7f9a5c0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:17.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.590+0000 7f9a7a600640 1 --2- 172.21.15.83:0/1122855404 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f9a5c0a3e40 0x7f9a5c0a6300 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9a68005fd0 tx=0x7f9a68005cf0 comp rx=0 tx=0).stop 2024-04-24T08:52:17.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.590+0000 7f9a7a600640 1 -- 172.21.15.83:0/1122855404 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a74108e40 msgr2=0x7f9a741a4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:17.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.590+0000 7f9a7a600640 1 --2- 172.21.15.83:0/1122855404 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a74108e40 0x7f9a741a4cf0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f9a6000c920 tx=0x7f9a6000cde0 comp rx=0 tx=0).stop 2024-04-24T08:52:17.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.590+0000 7f9a7a600640 1 -- 172.21.15.83:0/1122855404 shutdown_connections 2024-04-24T08:52:17.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.590+0000 7f9a7a600640 1 --2- 172.21.15.83:0/1122855404 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f9a5c0a3e40 0x7f9a5c0a6300 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:17.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.590+0000 7f9a7a600640 1 --2- 172.21.15.83:0/1122855404 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9a7410d3c0 0x7f9a7419e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:17.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.590+0000 7f9a7a600640 1 --2- 172.21.15.83:0/1122855404 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a74108e40 0x7f9a741a4cf0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:17.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.590+0000 7f9a7a600640 1 --2- 172.21.15.83:0/1122855404 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a741084a0 0x7f9a741a47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:17.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.590+0000 7f9a7a600640 1 -- 172.21.15.83:0/1122855404 >> 172.21.15.83:0/1122855404 conn(0x7f9a74077b80 msgr2=0x7f9a74078320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:17.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.591+0000 7f9a7a600640 1 -- 172.21.15.83:0/1122855404 shutdown_connections 2024-04-24T08:52:17.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.591+0000 7f9a7a600640 1 -- 172.21.15.83:0/1122855404 wait complete. 2024-04-24T08:52:17.608 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-24T08:52:17.802 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:52:17 smithi083 ceph-mon[27907]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-24T08:52:17.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.807+0000 7f22a7200640 1 -- 172.21.15.83:0/2676187080 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22a8104cc0 msgr2=0x7f22a8108d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:17.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.807+0000 7f22a7200640 1 --2- 172.21.15.83:0/2676187080 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22a8104cc0 0x7f22a8108d00 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f2294009900 tx=0x7f229402a400 comp rx=0 tx=0).stop 2024-04-24T08:52:17.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.807+0000 7f22a7200640 1 -- 172.21.15.83:0/2676187080 shutdown_connections 2024-04-24T08:52:17.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.807+0000 7f22a7200640 1 --2- 172.21.15.83:0/2676187080 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f22a81093e0 0x7f22a8110fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:17.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.807+0000 7f22a7200640 1 --2- 172.21.15.83:0/2676187080 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22a8104cc0 0x7f22a8108d00 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:17.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.807+0000 7f22a7200640 1 --2- 172.21.15.83:0/2676187080 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f22a8104310 0x7f22a81046f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:17.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.807+0000 7f22a7200640 1 -- 172.21.15.83:0/2676187080 >> 172.21.15.83:0/2676187080 conn(0x7f22a80ffb70 msgr2=0x7f22a8101f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:17.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.807+0000 7f22a7200640 1 -- 172.21.15.83:0/2676187080 shutdown_connections 2024-04-24T08:52:17.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.807+0000 7f22a7200640 1 -- 172.21.15.83:0/2676187080 wait complete. 2024-04-24T08:52:17.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.808+0000 7f22a7200640 1 Processor -- start 2024-04-24T08:52:17.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.808+0000 7f22a7200640 1 -- start start 2024-04-24T08:52:17.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.809+0000 7f22a7200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f22a8104310 0x7f22a81a0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:17.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.809+0000 7f22a7200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f22a81093e0 0x7f22a81a0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:17.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.809+0000 7f22a7200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22a819a6c0 0x7f22a819ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:17.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.809+0000 7f22a7200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f22a8114380 con 0x7f22a819a6c0 2024-04-24T08:52:17.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.809+0000 7f22a7200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f22a8114200 con 0x7f22a81093e0 2024-04-24T08:52:17.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.809+0000 7f22a7200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f22a8114500 con 0x7f22a8104310 2024-04-24T08:52:17.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.809+0000 7f22a6800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22a819a6c0 0x7f22a819ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:17.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.809+0000 7f22a6800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22a819a6c0 0x7f22a819ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55726/0 (socket says 172.21.15.83:55726) 2024-04-24T08:52:17.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.809+0000 7f22a6800640 1 -- 172.21.15.83:0/3753586690 learned_addr learned my addr 172.21.15.83:0/3753586690 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:52:17.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.809+0000 7f22a5e00640 1 --2- 172.21.15.83:0/3753586690 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f22a8104310 0x7f22a81a0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:17.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.810+0000 7f22a5400640 1 --2- 172.21.15.83:0/3753586690 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f22a81093e0 0x7f22a81a0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:17.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.810+0000 7f22a6800640 1 -- 172.21.15.83:0/3753586690 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f22a8104310 msgr2=0x7f22a81a0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:17.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.810+0000 7f22a6800640 1 --2- 172.21.15.83:0/3753586690 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f22a8104310 0x7f22a81a0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:17.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.810+0000 7f22a6800640 1 -- 172.21.15.83:0/3753586690 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f22a81093e0 msgr2=0x7f22a81a0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:17.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.810+0000 7f22a6800640 1 --2- 172.21.15.83:0/3753586690 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f22a81093e0 0x7f22a81a0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:17.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.810+0000 7f22a6800640 1 -- 172.21.15.83:0/3753586690 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22a819b2c0 con 0x7f22a819a6c0 2024-04-24T08:52:17.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.810+0000 7f22a5400640 1 --2- 172.21.15.83:0/3753586690 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f22a81093e0 0x7f22a81a0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:52:17.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.810+0000 7f22a6800640 1 --2- 172.21.15.83:0/3753586690 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22a819a6c0 0x7f22a819ab40 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f22a000b970 tx=0x7f22a000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:17.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.810+0000 7f229ea00640 1 -- 172.21.15.83:0/3753586690 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a0019070 con 0x7f22a819a6c0 2024-04-24T08:52:17.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.810+0000 7f229ea00640 1 -- 172.21.15.83:0/3753586690 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f22a0014440 con 0x7f22a819a6c0 2024-04-24T08:52:17.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.810+0000 7f22a7200640 1 -- 172.21.15.83:0/3753586690 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22a81a73a0 con 0x7f22a819a6c0 2024-04-24T08:52:17.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.810+0000 7f229ea00640 1 -- 172.21.15.83:0/3753586690 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a00075d0 con 0x7f22a819a6c0 2024-04-24T08:52:17.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.810+0000 7f22a7200640 1 -- 172.21.15.83:0/3753586690 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22a81a7750 con 0x7f22a819a6c0 2024-04-24T08:52:17.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.814+0000 7f22af9b8740 1 Processor -- start 2024-04-24T08:52:17.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.815+0000 7f229ea00640 1 -- 172.21.15.83:0/3753586690 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7f22a001c070 con 0x7f22a819a6c0 2024-04-24T08:52:17.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.815+0000 7f22af9b8740 1 -- start start 2024-04-24T08:52:17.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.815+0000 7f22af9b8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7940a7b40 0x55a7940a7f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:17.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.815+0000 7f22af9b8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7940a8460 0x55a7940b25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:17.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.815+0000 7f22af9b8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a7940b2b30 0x55a7940b4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:17.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.815+0000 7f22af9b8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a79400d1d0 con 0x55a7940a8460 2024-04-24T08:52:17.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.815+0000 7f22af9b8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a793fb3450 con 0x55a7940a7b40 2024-04-24T08:52:17.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.815+0000 7f22af9b8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a794009190 con 0x55a7940b2b30 2024-04-24T08:52:17.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.816+0000 7f22a5e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7940a8460 0x55a7940b25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:17.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.816+0000 7f22a5e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7940a8460 0x55a7940b25f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55742/0 (socket says 172.21.15.83:55742) 2024-04-24T08:52:17.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.816+0000 7f22a5e00640 1 -- 172.21.15.83:0/3466107201 learned_addr learned my addr 172.21.15.83:0/3466107201 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:52:17.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.816+0000 7f22a5400640 1 --2- 172.21.15.83:0/3466107201 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7940a7b40 0x55a7940a7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:17.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.816+0000 7f22a5400640 1 --2- 172.21.15.83:0/3466107201 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a7940b2b30 0x55a7940b4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:17.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.816+0000 7f22a5e00640 1 -- 172.21.15.83:0/3466107201 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a7940b2b30 msgr2=0x55a7940b4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:17.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.816+0000 7f22a5e00640 1 --2- 172.21.15.83:0/3466107201 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a7940b2b30 0x55a7940b4f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:17.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.816+0000 7f22a5e00640 1 -- 172.21.15.83:0/3466107201 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7940a7b40 msgr2=0x55a7940a7f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:17.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.816+0000 7f22a5e00640 1 --2- 172.21.15.83:0/3466107201 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7940a7b40 0x55a7940a7f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:17.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.816+0000 7f22a5e00640 1 -- 172.21.15.83:0/3466107201 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a794070310 con 0x55a7940a8460 2024-04-24T08:52:17.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.816+0000 7f22a5400640 1 --2- 172.21.15.83:0/3466107201 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a7940b2b30 0x55a7940b4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:52:17.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.816+0000 7f229ea00640 1 --2- 172.21.15.83:0/3753586690 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f228c0a3e40 0x7f228c0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:17.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.816+0000 7f229ea00640 1 -- 172.21.15.83:0/3753586690 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f22a00d0400 con 0x7f22a819a6c0 2024-04-24T08:52:17.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.817+0000 7f22a5e00640 1 --2- 172.21.15.83:0/3466107201 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7940a8460 0x55a7940b25f0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f229800cc60 tx=0x7f229800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d661cbb70386f44a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:17.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.817+0000 7f22a5400640 1 --2- 172.21.15.83:0/3753586690 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f228c0a3e40 0x7f228c0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:17.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.817+0000 7f2293000640 1 -- 172.21.15.83:0/3466107201 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2298014070 con 0x55a7940a8460 2024-04-24T08:52:17.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.817+0000 7f2293000640 1 -- 172.21.15.83:0/3466107201 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f22980047a0 con 0x55a7940a8460 2024-04-24T08:52:17.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.817+0000 7f2293000640 1 -- 172.21.15.83:0/3466107201 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22980106c0 con 0x55a7940a8460 2024-04-24T08:52:17.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.818+0000 7f22af9b8740 1 -- 172.21.15.83:0/3466107201 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7940a8460 msgr2=0x55a7940b25f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:17.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.818+0000 7f22a5400640 1 --2- 172.21.15.83:0/3753586690 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f228c0a3e40 0x7f228c0a6300 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2294005c20 tx=0x7f2294031400 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:17.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.818+0000 7f22af9b8740 1 --2- 172.21.15.83:0/3466107201 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7940a8460 0x55a7940b25f0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f229800cc60 tx=0x7f229800e720 comp rx=0 tx=0).stop 2024-04-24T08:52:17.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.818+0000 7f22af9b8740 1 -- 172.21.15.83:0/3466107201 shutdown_connections 2024-04-24T08:52:17.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.818+0000 7f22af9b8740 1 --2- 172.21.15.83:0/3466107201 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a7940b2b30 0x55a7940b4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:17.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.818+0000 7f22af9b8740 1 --2- 172.21.15.83:0/3466107201 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7940a8460 0x55a7940b25f0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:17.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.818+0000 7f22af9b8740 1 --2- 172.21.15.83:0/3466107201 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7940a7b40 0x55a7940a7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:17.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.818+0000 7f22af9b8740 1 -- 172.21.15.83:0/3466107201 >> 172.21.15.83:0/3466107201 conn(0x55a7940a3e60 msgr2=0x55a7940a4240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:17.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.818+0000 7f22af9b8740 1 -- 172.21.15.83:0/3466107201 shutdown_connections 2024-04-24T08:52:17.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.818+0000 7f22af9b8740 1 -- 172.21.15.83:0/3466107201 wait complete. 2024-04-24T08:52:17.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.818+0000 7f22af9b8740 1 -- start start 2024-04-24T08:52:17.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.819+0000 7f22af9b8740 1 Processor -- start 2024-04-24T08:52:17.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.819+0000 7f22af9b8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a7940a7b40 0x55a7940b25b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:17.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.819+0000 7f22af9b8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7940a8460 0x55a7940af8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:17.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.819+0000 7f22af9b8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7940b2b30 0x55a7940affe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:17.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.819+0000 7f22af9b8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a7940b59c0 con 0x55a7940b2b30 2024-04-24T08:52:17.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.819+0000 7f22af9b8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a793fc6810 con 0x55a7940a8460 2024-04-24T08:52:17.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.819+0000 7f22af9b8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a7940b5b40 con 0x55a7940a7b40 2024-04-24T08:52:17.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.819+0000 7f22af9b8740 10 client.0 _ll_register_callbacks cb 0x55a793f94140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:52:17.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.819+0000 7f22a6800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7940b2b30 0x55a7940affe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:17.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.819+0000 7f22a6800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7940b2b30 0x55a7940affe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55758/0 (socket says 172.21.15.83:55758) 2024-04-24T08:52:17.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.819+0000 7f22a6800640 1 -- 172.21.15.83:0/134156004 learned_addr learned my addr 172.21.15.83:0/134156004 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:52:17.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.820+0000 7f22a5400640 1 --2- 172.21.15.83:0/134156004 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7940a8460 0x55a7940af8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:17.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.820+0000 7f22a5e00640 1 --2- 172.21.15.83:0/134156004 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a7940a7b40 0x55a7940b25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:17.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.820+0000 7f22a6800640 1 -- 172.21.15.83:0/134156004 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a7940a7b40 msgr2=0x55a7940b25b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:17.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.820+0000 7f22a6800640 1 --2- 172.21.15.83:0/134156004 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a7940a7b40 0x55a7940b25b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:17.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.820+0000 7f22a6800640 1 -- 172.21.15.83:0/134156004 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7940a8460 msgr2=0x55a7940af8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:17.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.820+0000 7f22a6800640 1 --2- 172.21.15.83:0/134156004 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7940a8460 0x55a7940af8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:17.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.820+0000 7f22a6800640 1 -- 172.21.15.83:0/134156004 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a793fd73f0 con 0x55a7940b2b30 2024-04-24T08:52:17.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.820+0000 7f22a6800640 1 --2- 172.21.15.83:0/134156004 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7940b2b30 0x55a7940affe0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f22a00a4570 tx=0x7f22a0014aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90ae2a3471f370a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:17.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.820+0000 7f2287400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:52:17.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.820+0000 7f2287400640 1 -- 172.21.15.83:0/134156004 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a00b5060 con 0x55a7940b2b30 2024-04-24T08:52:17.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.820+0000 7f2287400640 1 -- 172.21.15.83:0/134156004 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f22a00a4610 con 0x55a7940b2b30 2024-04-24T08:52:17.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.820+0000 7f22af9b8740 1 -- 172.21.15.83:0/134156004 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22980047a0 con 0x55a7940b2b30 2024-04-24T08:52:17.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.820+0000 7f2287400640 1 -- 172.21.15.83:0/134156004 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f22a00a4aa0 con 0x55a7940b2b30 2024-04-24T08:52:17.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.821+0000 7f22af9b8740 10 client.15915 fetch_fsmap learned FSMap version 28 2024-04-24T08:52:17.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.821+0000 7f22af9b8740 1 -- 172.21.15.83:0/134156004 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a793d1f3f0 con 0x55a7940b2b30 2024-04-24T08:52:17.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.821+0000 7f2287400640 1 -- 172.21.15.83:0/134156004 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f22980047a0 con 0x55a7940b2b30 2024-04-24T08:52:17.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.821+0000 7f2287400640 1 -- 172.21.15.83:0/134156004 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f22a00cd9a0 con 0x55a7940b2b30 2024-04-24T08:52:17.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.821+0000 7f22af9b8740 10 client.15915 fetch_fsmap finished waiting for FSMap version 28 2024-04-24T08:52:17.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.821+0000 7f22af9b8740 10 client.15915 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-24T08:52:17.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.821+0000 7f22af9b8740 20 client.15915 populate_metadata read hostname 'smithi083' 2024-04-24T08:52:17.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.821+0000 7f22af9b8740 1 --2- 172.21.15.83:0/134156004 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55a7940c2d50 0x55a7940c51c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:17.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.821+0000 7f22af9b8740 4 client.15915 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:52:17.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.821+0000 7f22af9b8740 1 -- 172.21.15.83:0/134156004 --> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a79401abf0 con 0x55a7940c2d50 2024-04-24T08:52:17.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.822+0000 7f22a5e00640 1 --2- 172.21.15.83:0/134156004 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55a7940c2d50 0x55a7940c51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:17.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.823+0000 7f22a5e00640 1 --2- 172.21.15.83:0/134156004 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55a7940c2d50 0x55a7940c51c0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b328fc74c30b9dc9 server_cookie=a6369e5cfed61591 in_seq=0 out_seq=0 2024-04-24T08:52:17.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.823+0000 7f2287400640 10 client.15915 ms_handle_connect on v2:172.21.15.189:6838/2548965336 2024-04-24T08:52:17.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.825+0000 7f2287400640 1 -- 172.21.15.83:0/134156004 <== mds.4 v2:172.21.15.189:6838/2548965336 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a79401abf0 con 0x55a7940c2d50 2024-04-24T08:52:17.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.825+0000 7f2287400640 10 client.15915 handle_command_reply: tid=0 2024-04-24T08:52:17.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.825+0000 7f22af9b8740 1 client.15915 shutdown 2024-04-24T08:52:17.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.826+0000 7f22af9b8740 1 -- 172.21.15.83:0/134156004 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7940b2b30 msgr2=0x55a7940affe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:17.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.826+0000 7f22af9b8740 1 --2- 172.21.15.83:0/134156004 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7940b2b30 0x55a7940affe0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f22a00a4570 tx=0x7f22a0014aa0 comp rx=0 tx=0).stop 2024-04-24T08:52:17.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.826+0000 7f22af9b8740 1 -- 172.21.15.83:0/134156004 shutdown_connections 2024-04-24T08:52:17.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.826+0000 7f22af9b8740 1 --2- 172.21.15.83:0/134156004 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55a7940c2d50 0x55a7940c51c0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:17.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.826+0000 7f22af9b8740 1 --2- 172.21.15.83:0/134156004 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7940b2b30 0x55a7940affe0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:17.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.826+0000 7f22af9b8740 1 --2- 172.21.15.83:0/134156004 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7940a8460 0x55a7940af8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:17.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.826+0000 7f22af9b8740 1 --2- 172.21.15.83:0/134156004 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a7940a7b40 0x55a7940b25b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:17.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.826+0000 7f2287400640 0 client.15915 ms_handle_reset on v2:172.21.15.189:6838/2548965336 2024-04-24T08:52:17.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.826+0000 7f22af9b8740 1 -- 172.21.15.83:0/134156004 >> 172.21.15.83:0/134156004 conn(0x55a7940a3e60 msgr2=0x55a7940a9b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:17.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.826+0000 7f22af9b8740 1 -- 172.21.15.83:0/134156004 shutdown_connections 2024-04-24T08:52:17.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.826+0000 7f22af9b8740 1 -- 172.21.15.83:0/134156004 wait complete. 2024-04-24T08:52:17.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.826+0000 7f22af9b8740 20 client.15915 trim_cache size 0 max 16384 2024-04-24T08:52:17.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.840+0000 7f22af9b8740 1 Processor -- start 2024-04-24T08:52:17.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.840+0000 7f22af9b8740 1 -- start start 2024-04-24T08:52:17.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.841+0000 7f22af9b8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7940a7b40 0x55a7940aeb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:17.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.841+0000 7f22af9b8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7940a8460 0x55a7940be840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:17.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.841+0000 7f22af9b8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a7940b2b30 0x55a7940a9d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:17.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.841+0000 7f22af9b8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f22a81b1460 con 0x55a7940a7b40 2024-04-24T08:52:17.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.841+0000 7f22af9b8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f22a81b2fa0 con 0x55a7940a8460 2024-04-24T08:52:17.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.841+0000 7f22af9b8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f22a81b1b30 con 0x55a7940b2b30 2024-04-24T08:52:17.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.841+0000 7f22a5e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7940a7b40 0x55a7940aeb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:17.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.841+0000 7f22a5e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7940a7b40 0x55a7940aeb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55764/0 (socket says 172.21.15.83:55764) 2024-04-24T08:52:17.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.841+0000 7f22a5e00640 1 -- 172.21.15.83:0/103817219 learned_addr learned my addr 172.21.15.83:0/103817219 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:52:17.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.841+0000 7f22a5e00640 1 -- 172.21.15.83:0/103817219 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a7940b2b30 msgr2=0x55a7940a9d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:17.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.841+0000 7f22a5e00640 1 --2- 172.21.15.83:0/103817219 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a7940b2b30 0x55a7940a9d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:17.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.841+0000 7f22a5e00640 1 -- 172.21.15.83:0/103817219 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7940a8460 msgr2=0x55a7940be840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:17.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.841+0000 7f22a5e00640 1 --2- 172.21.15.83:0/103817219 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7940a8460 0x55a7940be840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:17.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.841+0000 7f22a5e00640 1 -- 172.21.15.83:0/103817219 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7940a8290 con 0x55a7940a7b40 2024-04-24T08:52:17.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.842+0000 7f22a5e00640 1 --2- 172.21.15.83:0/103817219 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7940a7b40 0x55a7940aeb20 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f229800cc60 tx=0x7f229800c550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad14d7e332824f58 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:17.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.842+0000 7f2292600640 1 -- 172.21.15.83:0/103817219 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2298014070 con 0x55a7940a7b40 2024-04-24T08:52:17.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.842+0000 7f2292600640 1 -- 172.21.15.83:0/103817219 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f22980047a0 con 0x55a7940a7b40 2024-04-24T08:52:17.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.842+0000 7f2292600640 1 -- 172.21.15.83:0/103817219 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2298025280 con 0x55a7940a7b40 2024-04-24T08:52:17.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.842+0000 7f22af9b8740 1 -- 172.21.15.83:0/103817219 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7940a7b40 msgr2=0x55a7940aeb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:17.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.842+0000 7f22af9b8740 1 --2- 172.21.15.83:0/103817219 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7940a7b40 0x55a7940aeb20 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f229800cc60 tx=0x7f229800c550 comp rx=0 tx=0).stop 2024-04-24T08:52:17.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.842+0000 7f22af9b8740 1 -- 172.21.15.83:0/103817219 shutdown_connections 2024-04-24T08:52:17.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.842+0000 7f22af9b8740 1 --2- 172.21.15.83:0/103817219 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a7940b2b30 0x55a7940a9d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:17.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.842+0000 7f22af9b8740 1 --2- 172.21.15.83:0/103817219 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7940a8460 0x55a7940be840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:17.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.842+0000 7f22af9b8740 1 --2- 172.21.15.83:0/103817219 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7940a7b40 0x55a7940aeb20 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:17.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.842+0000 7f22af9b8740 1 -- 172.21.15.83:0/103817219 >> 172.21.15.83:0/103817219 conn(0x55a7940a3e60 msgr2=0x55a7940be130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:17.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.843+0000 7f22af9b8740 1 -- 172.21.15.83:0/103817219 shutdown_connections 2024-04-24T08:52:17.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.843+0000 7f22af9b8740 1 -- 172.21.15.83:0/103817219 wait complete. 2024-04-24T08:52:17.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.843+0000 7f22af9b8740 1 -- start start 2024-04-24T08:52:17.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.843+0000 7f22af9b8740 1 Processor -- start 2024-04-24T08:52:17.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.843+0000 7f22af9b8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7940a7b40 0x55a7940badc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:17.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.843+0000 7f22af9b8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a7940a8460 0x55a7940bb300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:17.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.843+0000 7f22af9b8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7940b2b30 0x55a7940b3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:17.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.844+0000 7f22af9b8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f22a81b28d0 con 0x55a7940a7b40 2024-04-24T08:52:17.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.844+0000 7f22af9b8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f22a81b2200 con 0x55a7940b2b30 2024-04-24T08:52:17.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.844+0000 7f22af9b8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f22a81af440 con 0x55a7940a8460 2024-04-24T08:52:17.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.844+0000 7f22af9b8740 10 client.0 _ll_register_callbacks cb 0x7f22a00980d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:52:17.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.844+0000 7f22a5e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7940a7b40 0x55a7940badc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:17.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.844+0000 7f22a5e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7940a7b40 0x55a7940badc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55770/0 (socket says 172.21.15.83:55770) 2024-04-24T08:52:17.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.844+0000 7f22a5e00640 1 -- 172.21.15.83:0/3031043100 learned_addr learned my addr 172.21.15.83:0/3031043100 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:52:17.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.844+0000 7f22a5e00640 1 -- 172.21.15.83:0/3031043100 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a7940a8460 msgr2=0x55a7940bb300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:17.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.844+0000 7f22a5e00640 1 --2- 172.21.15.83:0/3031043100 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a7940a8460 0x55a7940bb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:17.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.844+0000 7f22a5e00640 1 -- 172.21.15.83:0/3031043100 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7940b2b30 msgr2=0x55a7940b3ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:17.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.844+0000 7f22a5e00640 1 --2- 172.21.15.83:0/3031043100 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7940b2b30 0x55a7940b3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:17.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.844+0000 7f22a5e00640 1 -- 172.21.15.83:0/3031043100 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a793de97d0 con 0x55a7940a7b40 2024-04-24T08:52:17.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.844+0000 7f22a5e00640 1 --2- 172.21.15.83:0/3031043100 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7940a7b40 0x55a7940badc0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f229800ec20 tx=0x7f22980041b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7173367f4032bbf9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:17.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.844+0000 7f2287400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:52:17.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.845+0000 7f2287400640 1 -- 172.21.15.83:0/3031043100 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2298014040 con 0x55a7940a7b40 2024-04-24T08:52:17.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.845+0000 7f2287400640 1 -- 172.21.15.83:0/3031043100 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f22980035f0 con 0x55a7940a7b40 2024-04-24T08:52:17.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.845+0000 7f22af9b8740 1 -- 172.21.15.83:0/3031043100 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22980047a0 con 0x55a7940a7b40 2024-04-24T08:52:17.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.845+0000 7f2287400640 1 -- 172.21.15.83:0/3031043100 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f229803c050 con 0x55a7940a7b40 2024-04-24T08:52:17.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.845+0000 7f22af9b8740 10 client.15921 fetch_fsmap learned FSMap version 28 2024-04-24T08:52:17.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.845+0000 7f22af9b8740 1 -- 172.21.15.83:0/3031043100 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a794132030 con 0x55a7940a7b40 2024-04-24T08:52:17.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.845+0000 7f2287400640 1 -- 172.21.15.83:0/3031043100 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f22980047a0 con 0x55a7940a7b40 2024-04-24T08:52:17.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.845+0000 7f2287400640 1 -- 172.21.15.83:0/3031043100 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f229802e9e0 con 0x55a7940a7b40 2024-04-24T08:52:17.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.845+0000 7f22af9b8740 10 client.15921 fetch_fsmap finished waiting for FSMap version 28 2024-04-24T08:52:17.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.845+0000 7f22af9b8740 10 client.15921 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-24T08:52:17.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.845+0000 7f22af9b8740 20 client.15921 populate_metadata read hostname 'smithi083' 2024-04-24T08:52:17.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.845+0000 7f22af9b8740 1 --2- 172.21.15.83:0/3031043100 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55a794132550 0x55a794134970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:17.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.846+0000 7f22af9b8740 4 client.15921 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T08:52:17.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.846+0000 7f22af9b8740 1 -- 172.21.15.83:0/3031043100 --> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a794014a60 con 0x55a794132550 2024-04-24T08:52:17.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.846+0000 7f22a5400640 1 --2- 172.21.15.83:0/3031043100 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55a794132550 0x55a794134970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:17.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.847+0000 7f22a5400640 1 --2- 172.21.15.83:0/3031043100 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55a794132550 0x55a794134970 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=79866c476a45a811 server_cookie=aba3e586fbd72b46 in_seq=0 out_seq=0 2024-04-24T08:52:17.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.847+0000 7f2287400640 10 client.15921 ms_handle_connect on v2:172.21.15.189:6838/2548965336 2024-04-24T08:52:17.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.848+0000 7f2287400640 1 -- 172.21.15.83:0/3031043100 <== mds.4 v2:172.21.15.189:6838/2548965336 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a794014a60 con 0x55a794132550 2024-04-24T08:52:17.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.848+0000 7f2287400640 10 client.15921 handle_command_reply: tid=0 2024-04-24T08:52:17.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.848+0000 7f22af9b8740 1 client.15921 shutdown 2024-04-24T08:52:17.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.848+0000 7f22af9b8740 1 -- 172.21.15.83:0/3031043100 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7940a7b40 msgr2=0x55a7940badc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:17.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.848+0000 7f22af9b8740 1 --2- 172.21.15.83:0/3031043100 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7940a7b40 0x55a7940badc0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f229800ec20 tx=0x7f22980041b0 comp rx=0 tx=0).stop 2024-04-24T08:52:17.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.848+0000 7f22af9b8740 1 -- 172.21.15.83:0/3031043100 shutdown_connections 2024-04-24T08:52:17.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.848+0000 7f22af9b8740 1 --2- 172.21.15.83:0/3031043100 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55a794132550 0x55a794134970 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:17.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.848+0000 7f22af9b8740 1 --2- 172.21.15.83:0/3031043100 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7940b2b30 0x55a7940b3ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:17.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.848+0000 7f22af9b8740 1 --2- 172.21.15.83:0/3031043100 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a7940a8460 0x55a7940bb300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:17.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.848+0000 7f2287400640 0 client.15921 ms_handle_reset on v2:172.21.15.189:6838/2548965336 2024-04-24T08:52:17.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.848+0000 7f22af9b8740 1 --2- 172.21.15.83:0/3031043100 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7940a7b40 0x55a7940badc0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:17.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.848+0000 7f22af9b8740 1 -- 172.21.15.83:0/3031043100 >> 172.21.15.83:0/3031043100 conn(0x55a7940a3e60 msgr2=0x55a7940adb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:17.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.848+0000 7f22af9b8740 1 -- 172.21.15.83:0/3031043100 shutdown_connections 2024-04-24T08:52:17.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.848+0000 7f22af9b8740 1 -- 172.21.15.83:0/3031043100 wait complete. 2024-04-24T08:52:17.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.849+0000 7f22af9b8740 20 client.15921 trim_cache size 0 max 16384 2024-04-24T08:52:17.851 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T08:52:17.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.849+0000 7f22a7200640 1 -- 172.21.15.83:0/3753586690 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f228c0a3e40 msgr2=0x7f228c0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:17.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.849+0000 7f22a7200640 1 --2- 172.21.15.83:0/3753586690 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f228c0a3e40 0x7f228c0a6300 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2294005c20 tx=0x7f2294031400 comp rx=0 tx=0).stop 2024-04-24T08:52:17.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.849+0000 7f22a7200640 1 -- 172.21.15.83:0/3753586690 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22a819a6c0 msgr2=0x7f22a819ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:17.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.849+0000 7f22a7200640 1 --2- 172.21.15.83:0/3753586690 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22a819a6c0 0x7f22a819ab40 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f22a000b970 tx=0x7f22a000be30 comp rx=0 tx=0).stop 2024-04-24T08:52:17.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.849+0000 7f22a7200640 1 -- 172.21.15.83:0/3753586690 shutdown_connections 2024-04-24T08:52:17.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.849+0000 7f22a7200640 1 --2- 172.21.15.83:0/3753586690 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f228c0a3e40 0x7f228c0a6300 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:17.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.849+0000 7f22a7200640 1 --2- 172.21.15.83:0/3753586690 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22a819a6c0 0x7f22a819ab40 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:17.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.849+0000 7f22a7200640 1 --2- 172.21.15.83:0/3753586690 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f22a81093e0 0x7f22a81a0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:17.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.849+0000 7f22a7200640 1 --2- 172.21.15.83:0/3753586690 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f22a8104310 0x7f22a81a0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:17.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.849+0000 7f22a7200640 1 -- 172.21.15.83:0/3753586690 >> 172.21.15.83:0/3753586690 conn(0x7f22a80ffb70 msgr2=0x7f22a8101ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:17.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.849+0000 7f22a7200640 1 -- 172.21.15.83:0/3753586690 shutdown_connections 2024-04-24T08:52:17.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:17.849+0000 7f22a7200640 1 -- 172.21.15.83:0/3753586690 wait complete. 2024-04-24T08:52:17.867 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-24T08:52:17.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:52:17 smithi138 ceph-mon[31441]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-24T08:52:17.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:52:17 smithi189 ceph-mon[31791]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-24T08:52:18.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.067+0000 7fdcf5c00640 1 -- 172.21.15.83:0/28118631 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdcf0104cd0 msgr2=0x7fdcf0108d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:18.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.067+0000 7fdcf5c00640 1 --2- 172.21.15.83:0/28118631 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdcf0104cd0 0x7fdcf0108d10 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fdcd8009900 tx=0x7fdcd802a400 comp rx=0 tx=0).stop 2024-04-24T08:52:18.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.067+0000 7fdcf5c00640 1 -- 172.21.15.83:0/28118631 shutdown_connections 2024-04-24T08:52:18.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.067+0000 7fdcf5c00640 1 --2- 172.21.15.83:0/28118631 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdcf01093f0 0x7fdcf0110fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.067+0000 7fdcf5c00640 1 --2- 172.21.15.83:0/28118631 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdcf0104cd0 0x7fdcf0108d10 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.067+0000 7fdcf5c00640 1 --2- 172.21.15.83:0/28118631 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdcf0104320 0x7fdcf0104700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.067+0000 7fdcf5c00640 1 -- 172.21.15.83:0/28118631 >> 172.21.15.83:0/28118631 conn(0x7fdcf00ffb40 msgr2=0x7fdcf0101f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:18.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.067+0000 7fdcf5c00640 1 -- 172.21.15.83:0/28118631 shutdown_connections 2024-04-24T08:52:18.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.067+0000 7fdcf5c00640 1 -- 172.21.15.83:0/28118631 wait complete. 2024-04-24T08:52:18.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.068+0000 7fdcf5c00640 1 Processor -- start 2024-04-24T08:52:18.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.068+0000 7fdcf5c00640 1 -- start start 2024-04-24T08:52:18.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.068+0000 7fdcf5c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdcf0104320 0x7fdcf01a05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:18.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.069+0000 7fdcf5c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdcf01093f0 0x7fdcf01a0b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:18.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.069+0000 7fdcf5c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdcf019a6f0 0x7fdcf019ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:18.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.069+0000 7fdcf5c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdcf0114310 con 0x7fdcf01093f0 2024-04-24T08:52:18.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.069+0000 7fdcf5c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdcf0114190 con 0x7fdcf0104320 2024-04-24T08:52:18.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.069+0000 7fdcf5c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdcf0114490 con 0x7fdcf019a6f0 2024-04-24T08:52:18.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.069+0000 7fdce7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdcf01093f0 0x7fdcf01a0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:18.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.069+0000 7fdce7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdcf01093f0 0x7fdcf01a0b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55790/0 (socket says 172.21.15.83:55790) 2024-04-24T08:52:18.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.069+0000 7fdce7e00640 1 -- 172.21.15.83:0/1557414429 learned_addr learned my addr 172.21.15.83:0/1557414429 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:52:18.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.069+0000 7fdcefe00640 1 --2- 172.21.15.83:0/1557414429 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdcf0104320 0x7fdcf01a05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:18.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.069+0000 7fdce7e00640 1 -- 172.21.15.83:0/1557414429 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdcf019a6f0 msgr2=0x7fdcf019ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:18.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.069+0000 7fdce7e00640 1 --2- 172.21.15.83:0/1557414429 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdcf019a6f0 0x7fdcf019ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.069+0000 7fdce7e00640 1 -- 172.21.15.83:0/1557414429 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdcf0104320 msgr2=0x7fdcf01a05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:18.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.069+0000 7fdce7e00640 1 --2- 172.21.15.83:0/1557414429 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdcf0104320 0x7fdcf01a05e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.069+0000 7fdce7e00640 1 -- 172.21.15.83:0/1557414429 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcf019b2f0 con 0x7fdcf01093f0 2024-04-24T08:52:18.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.070+0000 7fdce7e00640 1 --2- 172.21.15.83:0/1557414429 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdcf01093f0 0x7fdcf01a0b20 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fdcd8004270 tx=0x7fdcd80042a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:18.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.070+0000 7fdced600640 1 -- 172.21.15.83:0/1557414429 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd803f030 con 0x7fdcf01093f0 2024-04-24T08:52:18.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.070+0000 7fdced600640 1 -- 172.21.15.83:0/1557414429 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdcd80044c0 con 0x7fdcf01093f0 2024-04-24T08:52:18.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.070+0000 7fdced600640 1 -- 172.21.15.83:0/1557414429 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd8035070 con 0x7fdcf01093f0 2024-04-24T08:52:18.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.070+0000 7fdcf5c00640 1 -- 172.21.15.83:0/1557414429 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcf019b580 con 0x7fdcf01093f0 2024-04-24T08:52:18.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.070+0000 7fdcf5c00640 1 -- 172.21.15.83:0/1557414429 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcf01a76c0 con 0x7fdcf01093f0 2024-04-24T08:52:18.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.073+0000 7fdced600640 1 -- 172.21.15.83:0/1557414429 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7fdcd8042030 con 0x7fdcf01093f0 2024-04-24T08:52:18.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.073+0000 7fdcfa010740 1 Processor -- start 2024-04-24T08:52:18.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.073+0000 7fdcfa010740 1 -- start start 2024-04-24T08:52:18.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.074+0000 7fdcfa010740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5598a71eeb40 0x5598a71eef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:18.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.074+0000 7fdcfa010740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5598a71ef460 0x5598a71f95f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:18.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.074+0000 7fdcfa010740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5598a71f9b30 0x5598a71fbf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:18.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.074+0000 7fdcfa010740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5598a71541d0 con 0x5598a71f9b30 2024-04-24T08:52:18.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.074+0000 7fdcfa010740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5598a70fa450 con 0x5598a71ef460 2024-04-24T08:52:18.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.074+0000 7fdcfa010740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5598a7150190 con 0x5598a71eeb40 2024-04-24T08:52:18.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.074+0000 7fdce7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5598a71f9b30 0x5598a71fbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:18.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.074+0000 7fdce7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5598a71f9b30 0x5598a71fbf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55804/0 (socket says 172.21.15.83:55804) 2024-04-24T08:52:18.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.074+0000 7fdce7e00640 1 -- 172.21.15.83:0/43490182 learned_addr learned my addr 172.21.15.83:0/43490182 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:52:18.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.074+0000 7fdcefe00640 1 --2- 172.21.15.83:0/43490182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5598a71ef460 0x5598a71f95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:18.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.074+0000 7fdcf5200640 1 --2- 172.21.15.83:0/43490182 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5598a71eeb40 0x5598a71eef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:18.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.075+0000 7fdce7e00640 1 -- 172.21.15.83:0/43490182 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5598a71eeb40 msgr2=0x5598a71eef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:18.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.075+0000 7fdce7e00640 1 --2- 172.21.15.83:0/43490182 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5598a71eeb40 0x5598a71eef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.075+0000 7fdce7e00640 1 -- 172.21.15.83:0/43490182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5598a71ef460 msgr2=0x5598a71f95f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:18.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.075+0000 7fdce7e00640 1 --2- 172.21.15.83:0/43490182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5598a71ef460 0x5598a71f95f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.075+0000 7fdce7e00640 1 -- 172.21.15.83:0/43490182 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598a71b7310 con 0x5598a71f9b30 2024-04-24T08:52:18.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.075+0000 7fdcefe00640 1 --2- 172.21.15.83:0/43490182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5598a71ef460 0x5598a71f95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:52:18.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.075+0000 7fdce7e00640 1 --2- 172.21.15.83:0/43490182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5598a71f9b30 0x5598a71fbf20 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fdcd80ee650 tx=0x7fdcd80f7750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=881c0b8632033de4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:18.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.075+0000 7fdced600640 1 --2- 172.21.15.83:0/1557414429 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fdcd40a3ec0 0x7fdcd40a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:18.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.075+0000 7fdced600640 1 -- 172.21.15.83:0/1557414429 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7fdcd80f13c0 con 0x7fdcf01093f0 2024-04-24T08:52:18.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.075+0000 7fdce5600640 1 -- 172.21.15.83:0/43490182 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd80b90b0 con 0x5598a71f9b30 2024-04-24T08:52:18.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.075+0000 7fdce5600640 1 -- 172.21.15.83:0/43490182 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdcd80fe030 con 0x5598a71f9b30 2024-04-24T08:52:18.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.075+0000 7fdcf5200640 1 --2- 172.21.15.83:0/1557414429 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fdcd40a3ec0 0x7fdcd40a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:18.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.075+0000 7fdce5600640 1 -- 172.21.15.83:0/43490182 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd80bc770 con 0x5598a71f9b30 2024-04-24T08:52:18.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.076+0000 7fdcfa010740 1 -- 172.21.15.83:0/43490182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5598a71f9b30 msgr2=0x5598a71fbf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:18.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.076+0000 7fdcfa010740 1 --2- 172.21.15.83:0/43490182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5598a71f9b30 0x5598a71fbf20 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fdcd80ee650 tx=0x7fdcd80f7750 comp rx=0 tx=0).stop 2024-04-24T08:52:18.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.076+0000 7fdcf5200640 1 --2- 172.21.15.83:0/1557414429 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fdcd40a3ec0 0x7fdcd40a6380 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fdce800aca0 tx=0x7fdce800a2b0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:18.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.076+0000 7fdcfa010740 1 -- 172.21.15.83:0/43490182 shutdown_connections 2024-04-24T08:52:18.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.076+0000 7fdcfa010740 1 --2- 172.21.15.83:0/43490182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5598a71f9b30 0x5598a71fbf20 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.076+0000 7fdcfa010740 1 --2- 172.21.15.83:0/43490182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5598a71ef460 0x5598a71f95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.076+0000 7fdcfa010740 1 --2- 172.21.15.83:0/43490182 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5598a71eeb40 0x5598a71eef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.076+0000 7fdcfa010740 1 -- 172.21.15.83:0/43490182 >> 172.21.15.83:0/43490182 conn(0x5598a71eae60 msgr2=0x5598a71eb240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:18.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.076+0000 7fdcfa010740 1 -- 172.21.15.83:0/43490182 shutdown_connections 2024-04-24T08:52:18.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.076+0000 7fdcfa010740 1 -- 172.21.15.83:0/43490182 wait complete. 2024-04-24T08:52:18.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.077+0000 7fdcfa010740 1 -- start start 2024-04-24T08:52:18.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.077+0000 7fdcfa010740 1 Processor -- start 2024-04-24T08:52:18.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.078+0000 7fdcfa010740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5598a71eeb40 0x5598a71f9530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:18.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.078+0000 7fdcfa010740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5598a71ef460 0x5598a71f6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:18.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.078+0000 7fdcfa010740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5598a71f9b30 0x5598a71f6e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:18.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.078+0000 7fdcfa010740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5598a71fc6e0 con 0x5598a71eeb40 2024-04-24T08:52:18.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.078+0000 7fdcfa010740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5598a710d810 con 0x5598a71f9b30 2024-04-24T08:52:18.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.078+0000 7fdcfa010740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5598a71fc860 con 0x5598a71ef460 2024-04-24T08:52:18.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.078+0000 7fdcfa010740 10 client.0 _ll_register_callbacks cb 0x5598a70db140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:52:18.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.078+0000 7fdcefe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5598a71eeb40 0x5598a71f9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:18.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.078+0000 7fdcefe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5598a71eeb40 0x5598a71f9530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55818/0 (socket says 172.21.15.83:55818) 2024-04-24T08:52:18.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.078+0000 7fdcefe00640 1 -- 172.21.15.83:0/2812576819 learned_addr learned my addr 172.21.15.83:0/2812576819 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:52:18.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.078+0000 7fdcf5200640 1 --2- 172.21.15.83:0/2812576819 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5598a71f9b30 0x5598a71f6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:18.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.078+0000 7fdcefe00640 1 -- 172.21.15.83:0/2812576819 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5598a71ef460 msgr2=0x5598a71f6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:18.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.078+0000 7fdcefe00640 1 --2- 172.21.15.83:0/2812576819 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5598a71ef460 0x5598a71f6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.078+0000 7fdcefe00640 1 -- 172.21.15.83:0/2812576819 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5598a71f9b30 msgr2=0x5598a71f6e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:18.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.078+0000 7fdcefe00640 1 --2- 172.21.15.83:0/2812576819 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5598a71f9b30 0x5598a71f6e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.078+0000 7fdcefe00640 1 -- 172.21.15.83:0/2812576819 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598a711e3f0 con 0x5598a71eeb40 2024-04-24T08:52:18.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.078+0000 7fdcf5200640 1 --2- 172.21.15.83:0/2812576819 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5598a71f9b30 0x5598a71f6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:52:18.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.079+0000 7fdcefe00640 1 --2- 172.21.15.83:0/2812576819 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5598a71eeb40 0x5598a71f9530 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fdce000f6f0 tx=0x7fdce000fbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23a2f5f6b0f9265 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:18.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.079+0000 7fdcde000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:52:18.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.079+0000 7fdcde000640 1 -- 172.21.15.83:0/2812576819 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce0016070 con 0x5598a71eeb40 2024-04-24T08:52:18.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.079+0000 7fdcde000640 1 -- 172.21.15.83:0/2812576819 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7fdce000bb00 con 0x5598a71eeb40 2024-04-24T08:52:18.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.079+0000 7fdcfa010740 1 -- 172.21.15.83:0/2812576819 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdcd80fe030 con 0x5598a71eeb40 2024-04-24T08:52:18.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.079+0000 7fdcde000640 1 -- 172.21.15.83:0/2812576819 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fdce000bd00 con 0x5598a71eeb40 2024-04-24T08:52:18.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.079+0000 7fdcfa010740 10 client.15933 fetch_fsmap learned FSMap version 28 2024-04-24T08:52:18.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.079+0000 7fdcfa010740 1 -- 172.21.15.83:0/2812576819 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5598a6e663f0 con 0x5598a71eeb40 2024-04-24T08:52:18.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.079+0000 7fdcde000640 1 -- 172.21.15.83:0/2812576819 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdcd80fe030 con 0x5598a71eeb40 2024-04-24T08:52:18.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.079+0000 7fdcde000640 1 -- 172.21.15.83:0/2812576819 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fdce0004bf0 con 0x5598a71eeb40 2024-04-24T08:52:18.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.080+0000 7fdcfa010740 10 client.15933 fetch_fsmap finished waiting for FSMap version 28 2024-04-24T08:52:18.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.080+0000 7fdcfa010740 10 client.15933 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-24T08:52:18.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.080+0000 7fdcfa010740 20 client.15933 populate_metadata read hostname 'smithi083' 2024-04-24T08:52:18.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.080+0000 7fdcfa010740 1 --2- 172.21.15.83:0/2812576819 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x5598a7209a70 0x5598a720bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:18.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.080+0000 7fdcfa010740 4 client.15933 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:52:18.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.080+0000 7fdcfa010740 1 -- 172.21.15.83:0/2812576819 --> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5598a7161bf0 con 0x5598a7209a70 2024-04-24T08:52:18.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.080+0000 7fdce7e00640 1 --2- 172.21.15.83:0/2812576819 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x5598a7209a70 0x5598a720bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:18.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.081+0000 7fdce7e00640 1 --2- 172.21.15.83:0/2812576819 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x5598a7209a70 0x5598a720bee0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2da902192eee185c server_cookie=9ee3b5a5f3c8e282 in_seq=0 out_seq=0 2024-04-24T08:52:18.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.081+0000 7fdcde000640 10 client.15933 ms_handle_connect on v2:172.21.15.138:6832/1157967658 2024-04-24T08:52:18.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.083+0000 7fdcde000640 1 -- 172.21.15.83:0/2812576819 <== mds.3 v2:172.21.15.138:6832/1157967658 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5598a7161bf0 con 0x5598a7209a70 2024-04-24T08:52:18.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.083+0000 7fdcde000640 10 client.15933 handle_command_reply: tid=0 2024-04-24T08:52:18.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.083+0000 7fdcfa010740 1 client.15933 shutdown 2024-04-24T08:52:18.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.084+0000 7fdcfa010740 1 -- 172.21.15.83:0/2812576819 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5598a71eeb40 msgr2=0x5598a71f9530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:18.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.084+0000 7fdcfa010740 1 --2- 172.21.15.83:0/2812576819 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5598a71eeb40 0x5598a71f9530 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fdce000f6f0 tx=0x7fdce000fbb0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.084+0000 7fdcfa010740 1 -- 172.21.15.83:0/2812576819 shutdown_connections 2024-04-24T08:52:18.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.084+0000 7fdcfa010740 1 --2- 172.21.15.83:0/2812576819 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5598a71f9b30 0x5598a71f6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.084+0000 7fdcfa010740 1 --2- 172.21.15.83:0/2812576819 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x5598a7209a70 0x5598a720bee0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.084+0000 7fdcfa010740 1 --2- 172.21.15.83:0/2812576819 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5598a71ef460 0x5598a71f6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.084+0000 7fdcfa010740 1 --2- 172.21.15.83:0/2812576819 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5598a71eeb40 0x5598a71f9530 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.084+0000 7fdcde000640 0 client.15933 ms_handle_reset on v2:172.21.15.138:6832/1157967658 2024-04-24T08:52:18.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.084+0000 7fdcfa010740 1 -- 172.21.15.83:0/2812576819 >> 172.21.15.83:0/2812576819 conn(0x5598a71eae60 msgr2=0x5598a71f0c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:18.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.084+0000 7fdcfa010740 1 -- 172.21.15.83:0/2812576819 shutdown_connections 2024-04-24T08:52:18.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.084+0000 7fdcfa010740 1 -- 172.21.15.83:0/2812576819 wait complete. 2024-04-24T08:52:18.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.084+0000 7fdcfa010740 20 client.15933 trim_cache size 0 max 16384 2024-04-24T08:52:18.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.098+0000 7fdcfa010740 1 Processor -- start 2024-04-24T08:52:18.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.098+0000 7fdcfa010740 1 -- start start 2024-04-24T08:52:18.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.099+0000 7fdcfa010740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5598a71eeb40 0x5598a72055a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:18.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.099+0000 7fdcfa010740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5598a71ef460 0x5598a71f9060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:18.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.099+0000 7fdcfa010740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5598a71f9b30 0x5598a72117b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:18.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.099+0000 7fdcfa010740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdcf01b14f0 con 0x5598a71eeb40 2024-04-24T08:52:18.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.099+0000 7fdcfa010740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdcf01b3030 con 0x5598a71f9b30 2024-04-24T08:52:18.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.099+0000 7fdcfa010740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdcf01b1bc0 con 0x5598a71ef460 2024-04-24T08:52:18.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.099+0000 7fdcefe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5598a71eeb40 0x5598a72055a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:18.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.099+0000 7fdcefe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5598a71eeb40 0x5598a72055a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55832/0 (socket says 172.21.15.83:55832) 2024-04-24T08:52:18.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.099+0000 7fdcefe00640 1 -- 172.21.15.83:0/2163731538 learned_addr learned my addr 172.21.15.83:0/2163731538 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:52:18.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.099+0000 7fdcf5200640 1 --2- 172.21.15.83:0/2163731538 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5598a71f9b30 0x5598a72117b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:18.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.099+0000 7fdce7e00640 1 --2- 172.21.15.83:0/2163731538 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5598a71ef460 0x5598a71f9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:18.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.100+0000 7fdcefe00640 1 -- 172.21.15.83:0/2163731538 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5598a71ef460 msgr2=0x5598a71f9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:18.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.100+0000 7fdcefe00640 1 --2- 172.21.15.83:0/2163731538 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5598a71ef460 0x5598a71f9060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.100+0000 7fdcefe00640 1 -- 172.21.15.83:0/2163731538 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5598a71f9b30 msgr2=0x5598a72117b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:18.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.100+0000 7fdcefe00640 1 --2- 172.21.15.83:0/2163731538 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5598a71f9b30 0x5598a72117b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.100+0000 7fdcefe00640 1 -- 172.21.15.83:0/2163731538 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598a6f307d0 con 0x5598a71eeb40 2024-04-24T08:52:18.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.100+0000 7fdcefe00640 1 --2- 172.21.15.83:0/2163731538 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5598a71eeb40 0x5598a72055a0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fdce0000c00 tx=0x7fdce000d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8862e7ac095cf25 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:18.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.100+0000 7fdce4c00640 1 -- 172.21.15.83:0/2163731538 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce002d070 con 0x5598a71eeb40 2024-04-24T08:52:18.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.100+0000 7fdce4c00640 1 -- 172.21.15.83:0/2163731538 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdce00054b0 con 0x5598a71eeb40 2024-04-24T08:52:18.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.100+0000 7fdce4c00640 1 -- 172.21.15.83:0/2163731538 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce0016040 con 0x5598a71eeb40 2024-04-24T08:52:18.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.101+0000 7fdcfa010740 1 -- 172.21.15.83:0/2163731538 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5598a71eeb40 msgr2=0x5598a72055a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:18.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.101+0000 7fdcfa010740 1 --2- 172.21.15.83:0/2163731538 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5598a71eeb40 0x5598a72055a0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fdce0000c00 tx=0x7fdce000d560 comp rx=0 tx=0).stop 2024-04-24T08:52:18.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.101+0000 7fdcfa010740 1 -- 172.21.15.83:0/2163731538 shutdown_connections 2024-04-24T08:52:18.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.101+0000 7fdcfa010740 1 --2- 172.21.15.83:0/2163731538 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5598a71f9b30 0x5598a72117b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.101+0000 7fdcfa010740 1 --2- 172.21.15.83:0/2163731538 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5598a71ef460 0x5598a71f9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.101+0000 7fdcfa010740 1 --2- 172.21.15.83:0/2163731538 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5598a71eeb40 0x5598a72055a0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.101+0000 7fdcfa010740 1 -- 172.21.15.83:0/2163731538 >> 172.21.15.83:0/2163731538 conn(0x5598a71eae60 msgr2=0x5598a71f50f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:18.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.101+0000 7fdcfa010740 1 -- 172.21.15.83:0/2163731538 shutdown_connections 2024-04-24T08:52:18.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.101+0000 7fdcfa010740 1 -- 172.21.15.83:0/2163731538 wait complete. 2024-04-24T08:52:18.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.101+0000 7fdcfa010740 1 -- start start 2024-04-24T08:52:18.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.102+0000 7fdcfa010740 1 Processor -- start 2024-04-24T08:52:18.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.102+0000 7fdcfa010740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5598a71eeb40 0x5598a71fa340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:18.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.102+0000 7fdcfa010740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5598a71ef460 0x5598a71fa880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:18.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.102+0000 7fdcfa010740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5598a71f9b30 0x5598a7200ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:18.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.102+0000 7fdcfa010740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdcf01b2960 con 0x5598a71ef460 2024-04-24T08:52:18.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.102+0000 7fdcfa010740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdcf01b2290 con 0x5598a71f9b30 2024-04-24T08:52:18.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.102+0000 7fdcfa010740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdcf01af4d0 con 0x5598a71eeb40 2024-04-24T08:52:18.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.102+0000 7fdcfa010740 10 client.0 _ll_register_callbacks cb 0x7fdce000d340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:52:18.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.102+0000 7fdce7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5598a71ef460 0x5598a71fa880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:18.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.102+0000 7fdce7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5598a71ef460 0x5598a71fa880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55838/0 (socket says 172.21.15.83:55838) 2024-04-24T08:52:18.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.102+0000 7fdce7e00640 1 -- 172.21.15.83:0/2868115321 learned_addr learned my addr 172.21.15.83:0/2868115321 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:52:18.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.102+0000 7fdcefe00640 1 --2- 172.21.15.83:0/2868115321 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5598a71eeb40 0x5598a71fa340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:18.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.103+0000 7fdce7e00640 1 -- 172.21.15.83:0/2868115321 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5598a71eeb40 msgr2=0x5598a71fa340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:18.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.103+0000 7fdce7e00640 1 --2- 172.21.15.83:0/2868115321 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5598a71eeb40 0x5598a71fa340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.103+0000 7fdce7e00640 1 -- 172.21.15.83:0/2868115321 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5598a71f9b30 msgr2=0x5598a7200ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:18.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.103+0000 7fdce7e00640 1 --2- 172.21.15.83:0/2868115321 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5598a71f9b30 0x5598a7200ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.103+0000 7fdce7e00640 1 -- 172.21.15.83:0/2868115321 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598a7200ff0 con 0x5598a71ef460 2024-04-24T08:52:18.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.103+0000 7fdce7e00640 1 --2- 172.21.15.83:0/2868115321 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5598a71ef460 0x5598a71fa880 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fdcd80bcda0 tx=0x7fdcd8003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c66f32b6c31b6849 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:18.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.103+0000 7fdcde000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:52:18.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.103+0000 7fdcde000640 1 -- 172.21.15.83:0/2868115321 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd8049040 con 0x5598a71ef460 2024-04-24T08:52:18.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.103+0000 7fdcde000640 1 -- 172.21.15.83:0/2868115321 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7fdcd80f1170 con 0x5598a71ef460 2024-04-24T08:52:18.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.103+0000 7fdcfa010740 1 -- 172.21.15.83:0/2868115321 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdce00054b0 con 0x5598a71ef460 2024-04-24T08:52:18.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.103+0000 7fdcde000640 1 -- 172.21.15.83:0/2868115321 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fdcd80bc950 con 0x5598a71ef460 2024-04-24T08:52:18.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.103+0000 7fdcfa010740 10 client.15939 fetch_fsmap learned FSMap version 28 2024-04-24T08:52:18.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.103+0000 7fdcfa010740 1 -- 172.21.15.83:0/2868115321 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5598a7202250 con 0x5598a71ef460 2024-04-24T08:52:18.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.104+0000 7fdcde000640 1 -- 172.21.15.83:0/2868115321 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdce00054b0 con 0x5598a71ef460 2024-04-24T08:52:18.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.104+0000 7fdcde000640 1 -- 172.21.15.83:0/2868115321 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fdcd80b9480 con 0x5598a71ef460 2024-04-24T08:52:18.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.104+0000 7fdcfa010740 10 client.15939 fetch_fsmap finished waiting for FSMap version 28 2024-04-24T08:52:18.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.104+0000 7fdcfa010740 10 client.15939 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-24T08:52:18.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.104+0000 7fdcfa010740 20 client.15939 populate_metadata read hostname 'smithi083' 2024-04-24T08:52:18.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.104+0000 7fdcfa010740 1 --2- 172.21.15.83:0/2868115321 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x5598a7270280 0x5598a72726a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:18.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.104+0000 7fdcfa010740 4 client.15939 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T08:52:18.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.104+0000 7fdcfa010740 1 -- 172.21.15.83:0/2868115321 --> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5598a715ba60 con 0x5598a7270280 2024-04-24T08:52:18.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.105+0000 7fdcefe00640 1 --2- 172.21.15.83:0/2868115321 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x5598a7270280 0x5598a72726a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:18.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.105+0000 7fdcefe00640 1 --2- 172.21.15.83:0/2868115321 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x5598a7270280 0x5598a72726a0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=793c7f7b00ba8f28 server_cookie=96969a07985cc2ad in_seq=0 out_seq=0 2024-04-24T08:52:18.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.105+0000 7fdcde000640 10 client.15939 ms_handle_connect on v2:172.21.15.138:6832/1157967658 2024-04-24T08:52:18.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.106+0000 7fdcde000640 1 -- 172.21.15.83:0/2868115321 <== mds.3 v2:172.21.15.138:6832/1157967658 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5598a715ba60 con 0x5598a7270280 2024-04-24T08:52:18.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.106+0000 7fdcde000640 10 client.15939 handle_command_reply: tid=0 2024-04-24T08:52:18.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.106+0000 7fdcfa010740 1 client.15939 shutdown 2024-04-24T08:52:18.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.106+0000 7fdcfa010740 1 -- 172.21.15.83:0/2868115321 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5598a71ef460 msgr2=0x5598a71fa880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:18.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.106+0000 7fdcfa010740 1 --2- 172.21.15.83:0/2868115321 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5598a71ef460 0x5598a71fa880 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fdcd80bcda0 tx=0x7fdcd8003f60 comp rx=0 tx=0).stop 2024-04-24T08:52:18.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.106+0000 7fdcfa010740 1 -- 172.21.15.83:0/2868115321 shutdown_connections 2024-04-24T08:52:18.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.106+0000 7fdcfa010740 1 --2- 172.21.15.83:0/2868115321 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5598a71f9b30 0x5598a7200ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.106+0000 7fdcfa010740 1 --2- 172.21.15.83:0/2868115321 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5598a71ef460 0x5598a71fa880 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.106+0000 7fdcfa010740 1 --2- 172.21.15.83:0/2868115321 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x5598a7270280 0x5598a72726a0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.106+0000 7fdcfa010740 1 --2- 172.21.15.83:0/2868115321 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5598a71eeb40 0x5598a71fa340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.106+0000 7fdcde000640 0 client.15939 ms_handle_reset on v2:172.21.15.138:6832/1157967658 2024-04-24T08:52:18.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.106+0000 7fdcfa010740 1 -- 172.21.15.83:0/2868115321 >> 172.21.15.83:0/2868115321 conn(0x5598a71eae60 msgr2=0x5598a71e9f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:18.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.106+0000 7fdcfa010740 1 -- 172.21.15.83:0/2868115321 shutdown_connections 2024-04-24T08:52:18.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.106+0000 7fdcfa010740 1 -- 172.21.15.83:0/2868115321 wait complete. 2024-04-24T08:52:18.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.106+0000 7fdcfa010740 20 client.15939 trim_cache size 0 max 16384 2024-04-24T08:52:18.110 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T08:52:18.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.107+0000 7fdcf5c00640 1 -- 172.21.15.83:0/1557414429 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fdcd40a3ec0 msgr2=0x7fdcd40a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:18.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.107+0000 7fdcf5c00640 1 --2- 172.21.15.83:0/1557414429 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fdcd40a3ec0 0x7fdcd40a6380 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fdce800aca0 tx=0x7fdce800a2b0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.107+0000 7fdcf5c00640 1 -- 172.21.15.83:0/1557414429 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdcf01093f0 msgr2=0x7fdcf01a0b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:18.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.107+0000 7fdcf5c00640 1 --2- 172.21.15.83:0/1557414429 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdcf01093f0 0x7fdcf01a0b20 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fdcd8004270 tx=0x7fdcd80042a0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.107+0000 7fdcf5c00640 1 -- 172.21.15.83:0/1557414429 shutdown_connections 2024-04-24T08:52:18.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.107+0000 7fdcf5c00640 1 --2- 172.21.15.83:0/1557414429 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fdcd40a3ec0 0x7fdcd40a6380 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.107+0000 7fdcf5c00640 1 --2- 172.21.15.83:0/1557414429 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdcf019a6f0 0x7fdcf019ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.107+0000 7fdcf5c00640 1 --2- 172.21.15.83:0/1557414429 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdcf01093f0 0x7fdcf01a0b20 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.107+0000 7fdcf5c00640 1 --2- 172.21.15.83:0/1557414429 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdcf0104320 0x7fdcf01a05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.107+0000 7fdcf5c00640 1 -- 172.21.15.83:0/1557414429 >> 172.21.15.83:0/1557414429 conn(0x7fdcf00ffb40 msgr2=0x7fdcf0100f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:18.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.108+0000 7fdcf5c00640 1 -- 172.21.15.83:0/1557414429 shutdown_connections 2024-04-24T08:52:18.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.108+0000 7fdcf5c00640 1 -- 172.21.15.83:0/1557414429 wait complete. 2024-04-24T08:52:18.126 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T08:52:18.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.319+0000 7f522f200640 1 -- 172.21.15.83:0/3037565226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5230108f40 msgr2=0x7f52301153b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:18.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.319+0000 7f522f200640 1 --2- 172.21.15.83:0/3037565226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5230108f40 0x7f52301153b0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f5228009cd0 tx=0x7f522802a8f0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.320+0000 7f522f200640 1 -- 172.21.15.83:0/3037565226 shutdown_connections 2024-04-24T08:52:18.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.320+0000 7f522f200640 1 --2- 172.21.15.83:0/3037565226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5230108f40 0x7f52301153b0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.320+0000 7f522f200640 1 --2- 172.21.15.83:0/3037565226 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5230108580 0x7f5230108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.320+0000 7f522f200640 1 --2- 172.21.15.83:0/3037565226 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f523010d500 0x7f523010d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.320+0000 7f522f200640 1 -- 172.21.15.83:0/3037565226 >> 172.21.15.83:0/3037565226 conn(0x7f5230077e20 msgr2=0x7f5230106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:18.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.320+0000 7f522f200640 1 -- 172.21.15.83:0/3037565226 shutdown_connections 2024-04-24T08:52:18.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.320+0000 7f522f200640 1 -- 172.21.15.83:0/3037565226 wait complete. 2024-04-24T08:52:18.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.321+0000 7f522f200640 1 Processor -- start 2024-04-24T08:52:18.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.321+0000 7f522f200640 1 -- start start 2024-04-24T08:52:18.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.322+0000 7f522f200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5230108580 0x7f52301a4870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:18.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.322+0000 7f522f200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5230108f40 0x7f52301a4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:18.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.322+0000 7f522f200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f523010d500 0x7f523019ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:18.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.322+0000 7f522f200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f52301187d0 con 0x7f5230108580 2024-04-24T08:52:18.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.322+0000 7f522f200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5230118650 con 0x7f5230108f40 2024-04-24T08:52:18.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.322+0000 7f522f200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5230118950 con 0x7f523010d500 2024-04-24T08:52:18.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.322+0000 7f522de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5230108580 0x7f52301a4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:18.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.322+0000 7f522de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5230108580 0x7f52301a4870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55866/0 (socket says 172.21.15.83:55866) 2024-04-24T08:52:18.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.322+0000 7f522de00640 1 -- 172.21.15.83:0/2556522951 learned_addr learned my addr 172.21.15.83:0/2556522951 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:52:18.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.323+0000 7f5225400640 1 --2- 172.21.15.83:0/2556522951 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5230108f40 0x7f52301a4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:18.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.323+0000 7f522e800640 1 --2- 172.21.15.83:0/2556522951 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f523010d500 0x7f523019ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:18.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.323+0000 7f522de00640 1 -- 172.21.15.83:0/2556522951 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f523010d500 msgr2=0x7f523019ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:18.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.323+0000 7f522de00640 1 --2- 172.21.15.83:0/2556522951 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f523010d500 0x7f523019ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.323+0000 7f522de00640 1 -- 172.21.15.83:0/2556522951 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5230108f40 msgr2=0x7f52301a4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:18.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.323+0000 7f522de00640 1 --2- 172.21.15.83:0/2556522951 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5230108f40 0x7f52301a4db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.323+0000 7f522de00640 1 -- 172.21.15.83:0/2556522951 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f523019f1f0 con 0x7f5230108580 2024-04-24T08:52:18.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.323+0000 7f5225400640 1 --2- 172.21.15.83:0/2556522951 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5230108f40 0x7f52301a4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:52:18.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.323+0000 7f522de00640 1 --2- 172.21.15.83:0/2556522951 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5230108580 0x7f52301a4870 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f522000d950 tx=0x7f522000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:18.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.324+0000 7f5227400640 1 -- 172.21.15.83:0/2556522951 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5220017070 con 0x7f5230108580 2024-04-24T08:52:18.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.324+0000 7f5227400640 1 -- 172.21.15.83:0/2556522951 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5220009c30 con 0x7f5230108580 2024-04-24T08:52:18.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.324+0000 7f522f200640 1 -- 172.21.15.83:0/2556522951 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f523019f4e0 con 0x7f5230108580 2024-04-24T08:52:18.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.324+0000 7f5227400640 1 -- 172.21.15.83:0/2556522951 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f522000f620 con 0x7f5230108580 2024-04-24T08:52:18.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.324+0000 7f522f200640 1 -- 172.21.15.83:0/2556522951 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52301ab730 con 0x7f5230108580 2024-04-24T08:52:18.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.328+0000 7f5227400640 1 -- 172.21.15.83:0/2556522951 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7f522001a070 con 0x7f5230108580 2024-04-24T08:52:18.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.329+0000 7f5237df6740 1 Processor -- start 2024-04-24T08:52:18.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.329+0000 7f5237df6740 1 -- start start 2024-04-24T08:52:18.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.329+0000 7f5237df6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557900abfb40 0x557900abff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:18.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.329+0000 7f5237df6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557900ac0460 0x557900aca5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:18.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.329+0000 7f5237df6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557900acab30 0x557900accf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:18.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.329+0000 7f5237df6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x557900a251d0 con 0x557900ac0460 2024-04-24T08:52:18.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.329+0000 7f5237df6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5579009cb450 con 0x557900acab30 2024-04-24T08:52:18.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.329+0000 7f5237df6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x557900a21190 con 0x557900abfb40 2024-04-24T08:52:18.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.330+0000 7f522de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557900ac0460 0x557900aca5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:18.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.330+0000 7f522de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557900ac0460 0x557900aca5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55878/0 (socket says 172.21.15.83:55878) 2024-04-24T08:52:18.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.330+0000 7f522de00640 1 -- 172.21.15.83:0/2726717113 learned_addr learned my addr 172.21.15.83:0/2726717113 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:52:18.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.330+0000 7f522e800640 1 --2- 172.21.15.83:0/2726717113 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557900abfb40 0x557900abff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:18.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.330+0000 7f5225400640 1 --2- 172.21.15.83:0/2726717113 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557900acab30 0x557900accf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:18.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.330+0000 7f522de00640 1 -- 172.21.15.83:0/2726717113 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557900abfb40 msgr2=0x557900abff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:18.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.330+0000 7f522de00640 1 --2- 172.21.15.83:0/2726717113 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557900abfb40 0x557900abff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.330+0000 7f522de00640 1 -- 172.21.15.83:0/2726717113 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557900acab30 msgr2=0x557900accf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:18.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.330+0000 7f522de00640 1 --2- 172.21.15.83:0/2726717113 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557900acab30 0x557900accf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.330+0000 7f522de00640 1 -- 172.21.15.83:0/2726717113 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557900a88310 con 0x557900ac0460 2024-04-24T08:52:18.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.330+0000 7f5227400640 1 --2- 172.21.15.83:0/2556522951 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f52140a3ec0 0x7f52140a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:18.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.331+0000 7f522de00640 1 --2- 172.21.15.83:0/2726717113 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557900ac0460 0x557900aca5f0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f52200cdac0 tx=0x7f52200d8750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cf87cb6c7a77a55 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:18.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.331+0000 7f5227400640 1 -- 172.21.15.83:0/2556522951 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f52200d0920 con 0x7f5230108580 2024-04-24T08:52:18.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.331+0000 7f522e800640 1 --2- 172.21.15.83:0/2556522951 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f52140a3ec0 0x7f52140a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:18.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.331+0000 7f521f400640 1 -- 172.21.15.83:0/2726717113 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5220021030 con 0x557900ac0460 2024-04-24T08:52:18.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.331+0000 7f521f400640 1 -- 172.21.15.83:0/2726717113 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f522009c9a0 con 0x557900ac0460 2024-04-24T08:52:18.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.331+0000 7f521f400640 1 -- 172.21.15.83:0/2726717113 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f522009bc00 con 0x557900ac0460 2024-04-24T08:52:18.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.332+0000 7f5237df6740 1 -- 172.21.15.83:0/2726717113 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557900ac0460 msgr2=0x557900aca5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:18.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.332+0000 7f5237df6740 1 --2- 172.21.15.83:0/2726717113 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557900ac0460 0x557900aca5f0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f52200cdac0 tx=0x7f52200d8750 comp rx=0 tx=0).stop 2024-04-24T08:52:18.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.332+0000 7f522e800640 1 --2- 172.21.15.83:0/2556522951 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f52140a3ec0 0x7f52140a6380 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f522802aee0 tx=0x7f5228031000 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:18.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.332+0000 7f5237df6740 1 -- 172.21.15.83:0/2726717113 shutdown_connections 2024-04-24T08:52:18.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.332+0000 7f5237df6740 1 --2- 172.21.15.83:0/2726717113 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557900acab30 0x557900accf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.332+0000 7f5237df6740 1 --2- 172.21.15.83:0/2726717113 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557900ac0460 0x557900aca5f0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.332+0000 7f5237df6740 1 --2- 172.21.15.83:0/2726717113 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557900abfb40 0x557900abff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.332+0000 7f5237df6740 1 -- 172.21.15.83:0/2726717113 >> 172.21.15.83:0/2726717113 conn(0x557900abbe60 msgr2=0x557900abc240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:18.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.332+0000 7f5237df6740 1 -- 172.21.15.83:0/2726717113 shutdown_connections 2024-04-24T08:52:18.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.332+0000 7f5237df6740 1 -- 172.21.15.83:0/2726717113 wait complete. 2024-04-24T08:52:18.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.333+0000 7f5237df6740 1 -- start start 2024-04-24T08:52:18.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.333+0000 7f5237df6740 1 Processor -- start 2024-04-24T08:52:18.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.334+0000 7f5237df6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557900abfb40 0x557900aca5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:18.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.334+0000 7f5237df6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557900ac0460 0x557900ac78b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:18.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.334+0000 7f5237df6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557900acab30 0x557900ac7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:18.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.334+0000 7f5237df6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x557900acd9c0 con 0x557900ac0460 2024-04-24T08:52:18.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.334+0000 7f5237df6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5579009de810 con 0x557900acab30 2024-04-24T08:52:18.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.334+0000 7f5237df6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x557900acdb40 con 0x557900abfb40 2024-04-24T08:52:18.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.334+0000 7f5237df6740 10 client.0 _ll_register_callbacks cb 0x5579009ac140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:52:18.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.334+0000 7f5225400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557900ac0460 0x557900ac78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:18.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.334+0000 7f5225400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557900ac0460 0x557900ac78b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55890/0 (socket says 172.21.15.83:55890) 2024-04-24T08:52:18.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.334+0000 7f5225400640 1 -- 172.21.15.83:0/936860920 learned_addr learned my addr 172.21.15.83:0/936860920 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:52:18.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.334+0000 7f522de00640 1 --2- 172.21.15.83:0/936860920 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557900abfb40 0x557900aca5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:18.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.334+0000 7f5225400640 1 -- 172.21.15.83:0/936860920 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557900abfb40 msgr2=0x557900aca5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:18.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.334+0000 7f5225400640 1 --2- 172.21.15.83:0/936860920 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557900abfb40 0x557900aca5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.334+0000 7f5225400640 1 -- 172.21.15.83:0/936860920 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557900acab30 msgr2=0x557900ac7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:18.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.334+0000 7f5225400640 1 --2- 172.21.15.83:0/936860920 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557900acab30 0x557900ac7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.335+0000 7f5225400640 1 -- 172.21.15.83:0/936860920 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579009ef3f0 con 0x557900ac0460 2024-04-24T08:52:18.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.335+0000 7f5225400640 1 --2- 172.21.15.83:0/936860920 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557900ac0460 0x557900ac78b0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f5218010670 tx=0x7f5218010b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d57da8130363028 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:18.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.335+0000 7f5213e00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:52:18.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.335+0000 7f5213e00640 1 -- 172.21.15.83:0/936860920 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5218004220 con 0x557900ac0460 2024-04-24T08:52:18.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.335+0000 7f5213e00640 1 -- 172.21.15.83:0/936860920 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f52180043c0 con 0x557900ac0460 2024-04-24T08:52:18.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.335+0000 7f5237df6740 1 -- 172.21.15.83:0/936860920 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f522009c9a0 con 0x557900ac0460 2024-04-24T08:52:18.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.335+0000 7f5213e00640 1 -- 172.21.15.83:0/936860920 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5218023050 con 0x557900ac0460 2024-04-24T08:52:18.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.335+0000 7f5237df6740 10 client.15951 fetch_fsmap learned FSMap version 28 2024-04-24T08:52:18.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.335+0000 7f5237df6740 1 -- 172.21.15.83:0/936860920 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5579007373f0 con 0x557900ac0460 2024-04-24T08:52:18.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.336+0000 7f5213e00640 1 -- 172.21.15.83:0/936860920 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f522009c9a0 con 0x557900ac0460 2024-04-24T08:52:18.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.336+0000 7f5213e00640 1 -- 172.21.15.83:0/936860920 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f521800f040 con 0x557900ac0460 2024-04-24T08:52:18.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.336+0000 7f5237df6740 10 client.15951 fetch_fsmap finished waiting for FSMap version 28 2024-04-24T08:52:18.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.336+0000 7f5237df6740 10 client.15951 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-24T08:52:18.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.336+0000 7f5237df6740 20 client.15951 populate_metadata read hostname 'smithi083' 2024-04-24T08:52:18.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.336+0000 7f5237df6740 1 --2- 172.21.15.83:0/936860920 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x557900adae40 0x557900add260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:18.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.336+0000 7f5237df6740 4 client.15951 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:52:18.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.336+0000 7f5237df6740 1 -- 172.21.15.83:0/936860920 --> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557900a32bf0 con 0x557900adae40 2024-04-24T08:52:18.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.337+0000 7f522de00640 1 --2- 172.21.15.83:0/936860920 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x557900adae40 0x557900add260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:18.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.338+0000 7f522de00640 1 --2- 172.21.15.83:0/936860920 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x557900adae40 0x557900add260 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b9768cc7c651d53e server_cookie=4c11c01a92706bc9 in_seq=0 out_seq=0 2024-04-24T08:52:18.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.338+0000 7f5213e00640 10 client.15951 ms_handle_connect on v2:172.21.15.138:6834/1909219758 2024-04-24T08:52:18.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.340+0000 7f5213e00640 1 -- 172.21.15.83:0/936860920 <== mds.2 v2:172.21.15.138:6834/1909219758 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557900a32bf0 con 0x557900adae40 2024-04-24T08:52:18.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.340+0000 7f5213e00640 10 client.15951 handle_command_reply: tid=0 2024-04-24T08:52:18.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.340+0000 7f5237df6740 1 client.15951 shutdown 2024-04-24T08:52:18.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.340+0000 7f5237df6740 1 -- 172.21.15.83:0/936860920 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557900ac0460 msgr2=0x557900ac78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:18.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.340+0000 7f5237df6740 1 --2- 172.21.15.83:0/936860920 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557900ac0460 0x557900ac78b0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f5218010670 tx=0x7f5218010b30 comp rx=0 tx=0).stop 2024-04-24T08:52:18.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.340+0000 7f5237df6740 1 -- 172.21.15.83:0/936860920 shutdown_connections 2024-04-24T08:52:18.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.340+0000 7f5237df6740 1 --2- 172.21.15.83:0/936860920 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x557900adae40 0x557900add260 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.340+0000 7f5237df6740 1 --2- 172.21.15.83:0/936860920 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557900acab30 0x557900ac7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.340+0000 7f5237df6740 1 --2- 172.21.15.83:0/936860920 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557900ac0460 0x557900ac78b0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.340+0000 7f5237df6740 1 --2- 172.21.15.83:0/936860920 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557900abfb40 0x557900aca5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.340+0000 7f5213e00640 0 client.15951 ms_handle_reset on v2:172.21.15.138:6834/1909219758 2024-04-24T08:52:18.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.340+0000 7f5237df6740 1 -- 172.21.15.83:0/936860920 >> 172.21.15.83:0/936860920 conn(0x557900abbe60 msgr2=0x557900ac1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:18.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.340+0000 7f5237df6740 1 -- 172.21.15.83:0/936860920 shutdown_connections 2024-04-24T08:52:18.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.340+0000 7f5237df6740 1 -- 172.21.15.83:0/936860920 wait complete. 2024-04-24T08:52:18.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.341+0000 7f5237df6740 20 client.15951 trim_cache size 0 max 16384 2024-04-24T08:52:18.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.355+0000 7f5237df6740 1 Processor -- start 2024-04-24T08:52:18.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.355+0000 7f5237df6740 1 -- start start 2024-04-24T08:52:18.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.355+0000 7f5237df6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557900abfb40 0x557900ac7850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:18.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.355+0000 7f5237df6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557900ac0460 0x557900ad68e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:18.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.356+0000 7f5237df6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557900acab30 0x557900aca0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:18.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.356+0000 7f5237df6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f52301b5810 con 0x557900ac0460 2024-04-24T08:52:18.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.356+0000 7f5237df6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f52301b7350 con 0x557900abfb40 2024-04-24T08:52:18.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.356+0000 7f5237df6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f52301b5ee0 con 0x557900acab30 2024-04-24T08:52:18.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.356+0000 7f5225400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557900ac0460 0x557900ad68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:18.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.356+0000 7f5225400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557900ac0460 0x557900ad68e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55892/0 (socket says 172.21.15.83:55892) 2024-04-24T08:52:18.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.356+0000 7f5225400640 1 -- 172.21.15.83:0/3763426509 learned_addr learned my addr 172.21.15.83:0/3763426509 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:52:18.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.356+0000 7f522de00640 1 --2- 172.21.15.83:0/3763426509 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557900abfb40 0x557900ac7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:18.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.356+0000 7f5225400640 1 -- 172.21.15.83:0/3763426509 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557900acab30 msgr2=0x557900aca0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:18.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.356+0000 7f5225400640 1 --2- 172.21.15.83:0/3763426509 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557900acab30 0x557900aca0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.356+0000 7f5225400640 1 -- 172.21.15.83:0/3763426509 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557900abfb40 msgr2=0x557900ac7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:18.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.356+0000 7f5225400640 1 --2- 172.21.15.83:0/3763426509 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557900abfb40 0x557900ac7850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.356+0000 7f5225400640 1 -- 172.21.15.83:0/3763426509 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557900ac8310 con 0x557900ac0460 2024-04-24T08:52:18.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.356+0000 7f522de00640 1 --2- 172.21.15.83:0/3763426509 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557900abfb40 0x557900ac7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:52:18.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.357+0000 7f5225400640 1 --2- 172.21.15.83:0/3763426509 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557900ac0460 0x557900ad68e0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f5218003cd0 tx=0x7f5218009d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f3357b4eabab3a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:18.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.357+0000 7f521ea00640 1 -- 172.21.15.83:0/3763426509 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f521800e6c0 con 0x557900ac0460 2024-04-24T08:52:18.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.357+0000 7f521ea00640 1 -- 172.21.15.83:0/3763426509 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f52180095f0 con 0x557900ac0460 2024-04-24T08:52:18.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.357+0000 7f521ea00640 1 -- 172.21.15.83:0/3763426509 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f521800ed00 con 0x557900ac0460 2024-04-24T08:52:18.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.357+0000 7f5237df6740 1 -- 172.21.15.83:0/3763426509 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557900ac0460 msgr2=0x557900ad68e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:18.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.357+0000 7f5237df6740 1 --2- 172.21.15.83:0/3763426509 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557900ac0460 0x557900ad68e0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f5218003cd0 tx=0x7f5218009d80 comp rx=0 tx=0).stop 2024-04-24T08:52:18.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.358+0000 7f5237df6740 1 -- 172.21.15.83:0/3763426509 shutdown_connections 2024-04-24T08:52:18.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.358+0000 7f5237df6740 1 --2- 172.21.15.83:0/3763426509 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557900acab30 0x557900aca0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.358+0000 7f5237df6740 1 --2- 172.21.15.83:0/3763426509 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557900ac0460 0x557900ad68e0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.358+0000 7f5237df6740 1 --2- 172.21.15.83:0/3763426509 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557900abfb40 0x557900ac7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.358+0000 7f5237df6740 1 -- 172.21.15.83:0/3763426509 >> 172.21.15.83:0/3763426509 conn(0x557900abbe60 msgr2=0x557900ac7140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:18.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.358+0000 7f5237df6740 1 -- 172.21.15.83:0/3763426509 shutdown_connections 2024-04-24T08:52:18.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.358+0000 7f5237df6740 1 -- 172.21.15.83:0/3763426509 wait complete. 2024-04-24T08:52:18.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.358+0000 7f5237df6740 1 -- start start 2024-04-24T08:52:18.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.358+0000 7f5237df6740 1 Processor -- start 2024-04-24T08:52:18.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.359+0000 7f5237df6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557900abfb40 0x557900ad3310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:18.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.359+0000 7f5237df6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557900acab30 0x557900ac4e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:18.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.359+0000 7f5237df6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557900ac5520 0x557900acb9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:18.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.359+0000 7f5237df6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f52301b6c80 con 0x557900acab30 2024-04-24T08:52:18.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.359+0000 7f5237df6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f52301b65b0 con 0x557900abfb40 2024-04-24T08:52:18.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.359+0000 7f5237df6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f52301b37f0 con 0x557900ac5520 2024-04-24T08:52:18.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.359+0000 7f5237df6740 10 client.0 _ll_register_callbacks cb 0x7f5218009450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:52:18.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.359+0000 7f5225400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557900acab30 0x557900ac4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:18.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.359+0000 7f5225400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557900acab30 0x557900ac4e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55898/0 (socket says 172.21.15.83:55898) 2024-04-24T08:52:18.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.359+0000 7f5225400640 1 -- 172.21.15.83:0/3341901787 learned_addr learned my addr 172.21.15.83:0/3341901787 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:52:18.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.359+0000 7f522de00640 1 --2- 172.21.15.83:0/3341901787 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557900abfb40 0x557900ad3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:18.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.359+0000 7f522e800640 1 --2- 172.21.15.83:0/3341901787 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557900ac5520 0x557900acb9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:18.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.359+0000 7f5225400640 1 -- 172.21.15.83:0/3341901787 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557900ac5520 msgr2=0x557900acb9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:18.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.359+0000 7f5225400640 1 --2- 172.21.15.83:0/3341901787 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557900ac5520 0x557900acb9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.359+0000 7f5225400640 1 -- 172.21.15.83:0/3341901787 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557900abfb40 msgr2=0x557900ad3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:18.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.359+0000 7f5225400640 1 --2- 172.21.15.83:0/3341901787 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557900abfb40 0x557900ad3310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.360+0000 7f5225400640 1 -- 172.21.15.83:0/3341901787 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579008017d0 con 0x557900acab30 2024-04-24T08:52:18.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.360+0000 7f522e800640 1 --2- 172.21.15.83:0/3341901787 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557900ac5520 0x557900acb9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:52:18.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.360+0000 7f5225400640 1 --2- 172.21.15.83:0/3341901787 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557900acab30 0x557900ac4e80 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f5218009eb0 tx=0x7f5218010300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d174f9fee7c1280d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:18.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.360+0000 7f5213e00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:52:18.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.360+0000 7f5213e00640 1 -- 172.21.15.83:0/3341901787 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5218003880 con 0x557900acab30 2024-04-24T08:52:18.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.360+0000 7f5213e00640 1 -- 172.21.15.83:0/3341901787 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f521800e8d0 con 0x557900acab30 2024-04-24T08:52:18.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.360+0000 7f5237df6740 1 -- 172.21.15.83:0/3341901787 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52180095f0 con 0x557900acab30 2024-04-24T08:52:18.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.360+0000 7f5213e00640 1 -- 172.21.15.83:0/3341901787 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5218003a20 con 0x557900acab30 2024-04-24T08:52:18.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.360+0000 7f5237df6740 10 client.15957 fetch_fsmap learned FSMap version 28 2024-04-24T08:52:18.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.360+0000 7f5237df6740 1 -- 172.21.15.83:0/3341901787 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557900b41960 con 0x557900acab30 2024-04-24T08:52:18.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.361+0000 7f5213e00640 1 -- 172.21.15.83:0/3341901787 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f52180095f0 con 0x557900acab30 2024-04-24T08:52:18.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.361+0000 7f5213e00640 1 -- 172.21.15.83:0/3341901787 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f52180126d0 con 0x557900acab30 2024-04-24T08:52:18.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.361+0000 7f5237df6740 10 client.15957 fetch_fsmap finished waiting for FSMap version 28 2024-04-24T08:52:18.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.361+0000 7f5237df6740 10 client.15957 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-24T08:52:18.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.361+0000 7f5237df6740 20 client.15957 populate_metadata read hostname 'smithi083' 2024-04-24T08:52:18.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.361+0000 7f5237df6740 1 --2- 172.21.15.83:0/3341901787 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x557900b41fc0 0x557900b443b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:18.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.361+0000 7f5237df6740 4 client.15957 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T08:52:18.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.361+0000 7f5237df6740 1 -- 172.21.15.83:0/3341901787 --> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557900a2ca60 con 0x557900b41fc0 2024-04-24T08:52:18.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.362+0000 7f522de00640 1 --2- 172.21.15.83:0/3341901787 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x557900b41fc0 0x557900b443b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:18.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.362+0000 7f522de00640 1 --2- 172.21.15.83:0/3341901787 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x557900b41fc0 0x557900b443b0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=72b457a770fdcad2 server_cookie=25f8f656e00622c0 in_seq=0 out_seq=0 2024-04-24T08:52:18.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.362+0000 7f5213e00640 10 client.15957 ms_handle_connect on v2:172.21.15.138:6834/1909219758 2024-04-24T08:52:18.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.363+0000 7f5213e00640 1 -- 172.21.15.83:0/3341901787 <== mds.2 v2:172.21.15.138:6834/1909219758 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557900a2ca60 con 0x557900b41fc0 2024-04-24T08:52:18.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.363+0000 7f5213e00640 10 client.15957 handle_command_reply: tid=0 2024-04-24T08:52:18.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.363+0000 7f5237df6740 1 client.15957 shutdown 2024-04-24T08:52:18.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.363+0000 7f5237df6740 1 -- 172.21.15.83:0/3341901787 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557900acab30 msgr2=0x557900ac4e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:18.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.363+0000 7f5237df6740 1 --2- 172.21.15.83:0/3341901787 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557900acab30 0x557900ac4e80 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f5218009eb0 tx=0x7f5218010300 comp rx=0 tx=0).stop 2024-04-24T08:52:18.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.363+0000 7f5237df6740 1 -- 172.21.15.83:0/3341901787 shutdown_connections 2024-04-24T08:52:18.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.363+0000 7f5237df6740 1 --2- 172.21.15.83:0/3341901787 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x557900b41fc0 0x557900b443b0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.363+0000 7f5237df6740 1 --2- 172.21.15.83:0/3341901787 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557900ac5520 0x557900acb9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.363+0000 7f5237df6740 1 --2- 172.21.15.83:0/3341901787 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557900acab30 0x557900ac4e80 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.363+0000 7f5237df6740 1 --2- 172.21.15.83:0/3341901787 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557900abfb40 0x557900ad3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.363+0000 7f5237df6740 1 -- 172.21.15.83:0/3341901787 >> 172.21.15.83:0/3341901787 conn(0x557900abbe60 msgr2=0x557900aba6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:18.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.363+0000 7f5213e00640 0 client.15957 ms_handle_reset on v2:172.21.15.138:6834/1909219758 2024-04-24T08:52:18.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.363+0000 7f5237df6740 1 -- 172.21.15.83:0/3341901787 shutdown_connections 2024-04-24T08:52:18.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.363+0000 7f5237df6740 1 -- 172.21.15.83:0/3341901787 wait complete. 2024-04-24T08:52:18.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.364+0000 7f5237df6740 20 client.15957 trim_cache size 0 max 16384 2024-04-24T08:52:18.367 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T08:52:18.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.364+0000 7f522f200640 1 -- 172.21.15.83:0/2556522951 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f52140a3ec0 msgr2=0x7f52140a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:18.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.364+0000 7f522f200640 1 --2- 172.21.15.83:0/2556522951 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f52140a3ec0 0x7f52140a6380 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f522802aee0 tx=0x7f5228031000 comp rx=0 tx=0).stop 2024-04-24T08:52:18.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.364+0000 7f522f200640 1 -- 172.21.15.83:0/2556522951 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5230108580 msgr2=0x7f52301a4870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:18.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.364+0000 7f522f200640 1 --2- 172.21.15.83:0/2556522951 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5230108580 0x7f52301a4870 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f522000d950 tx=0x7f522000de10 comp rx=0 tx=0).stop 2024-04-24T08:52:18.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.364+0000 7f522f200640 1 -- 172.21.15.83:0/2556522951 shutdown_connections 2024-04-24T08:52:18.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.364+0000 7f522f200640 1 --2- 172.21.15.83:0/2556522951 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f52140a3ec0 0x7f52140a6380 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.364+0000 7f522f200640 1 --2- 172.21.15.83:0/2556522951 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f523010d500 0x7f523019ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.364+0000 7f522f200640 1 --2- 172.21.15.83:0/2556522951 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5230108f40 0x7f52301a4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.364+0000 7f522f200640 1 --2- 172.21.15.83:0/2556522951 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5230108580 0x7f52301a4870 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.364+0000 7f522f200640 1 -- 172.21.15.83:0/2556522951 >> 172.21.15.83:0/2556522951 conn(0x7f5230077e20 msgr2=0x7f52300783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:18.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.364+0000 7f522f200640 1 -- 172.21.15.83:0/2556522951 shutdown_connections 2024-04-24T08:52:18.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.364+0000 7f522f200640 1 -- 172.21.15.83:0/2556522951 wait complete. 2024-04-24T08:52:18.383 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T08:52:18.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.587+0000 7f668c200640 1 -- 172.21.15.83:0/3809812840 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66840743a0 msgr2=0x7f6684074780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:18.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.587+0000 7f668c200640 1 --2- 172.21.15.83:0/3809812840 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66840743a0 0x7f6684074780 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f6678009900 tx=0x7f667802a480 comp rx=0 tx=0).stop 2024-04-24T08:52:18.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.587+0000 7f668c200640 1 -- 172.21.15.83:0/3809812840 shutdown_connections 2024-04-24T08:52:18.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.587+0000 7f668c200640 1 --2- 172.21.15.83:0/3809812840 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6684113030 0x7f6684115420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.587+0000 7f668c200640 1 --2- 172.21.15.83:0/3809812840 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6684074cc0 0x7f6684112af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.587+0000 7f668c200640 1 --2- 172.21.15.83:0/3809812840 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66840743a0 0x7f6684074780 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.587+0000 7f668c200640 1 -- 172.21.15.83:0/3809812840 >> 172.21.15.83:0/3809812840 conn(0x7f66840fdb10 msgr2=0x7f66840fff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:18.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.587+0000 7f668c200640 1 -- 172.21.15.83:0/3809812840 shutdown_connections 2024-04-24T08:52:18.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.587+0000 7f668c200640 1 -- 172.21.15.83:0/3809812840 wait complete. 2024-04-24T08:52:18.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.588+0000 7f668c200640 1 Processor -- start 2024-04-24T08:52:18.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.588+0000 7f668c200640 1 -- start start 2024-04-24T08:52:18.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.589+0000 7f668c200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f66840743a0 0x7f668419ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:18.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.589+0000 7f668c200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6684074cc0 0x7f668419f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:18.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.589+0000 7f668c200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6684113030 0x7f66841a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:18.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.589+0000 7f668c200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6684118930 con 0x7f6684113030 2024-04-24T08:52:18.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.589+0000 7f668c200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f66841187b0 con 0x7f66840743a0 2024-04-24T08:52:18.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.589+0000 7f668c200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6684118ab0 con 0x7f6684074cc0 2024-04-24T08:52:18.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.589+0000 7f668a200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6684113030 0x7f66841a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:18.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.589+0000 7f668a200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6684113030 0x7f66841a36f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55916/0 (socket says 172.21.15.83:55916) 2024-04-24T08:52:18.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.589+0000 7f668a200640 1 -- 172.21.15.83:0/3014445795 learned_addr learned my addr 172.21.15.83:0/3014445795 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:52:18.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.589+0000 7f6689800640 1 --2- 172.21.15.83:0/3014445795 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f66840743a0 0x7f668419ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:18.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.589+0000 7f6688e00640 1 --2- 172.21.15.83:0/3014445795 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6684074cc0 0x7f668419f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:18.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.589+0000 7f668a200640 1 -- 172.21.15.83:0/3014445795 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6684074cc0 msgr2=0x7f668419f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:18.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.589+0000 7f668a200640 1 --2- 172.21.15.83:0/3014445795 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6684074cc0 0x7f668419f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.589+0000 7f668a200640 1 -- 172.21.15.83:0/3014445795 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f66840743a0 msgr2=0x7f668419ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:18.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.589+0000 7f668a200640 1 --2- 172.21.15.83:0/3014445795 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f66840743a0 0x7f668419ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.589+0000 7f668a200640 1 -- 172.21.15.83:0/3014445795 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66841a3e20 con 0x7f6684113030 2024-04-24T08:52:18.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.589+0000 7f6688e00640 1 --2- 172.21.15.83:0/3014445795 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6684074cc0 0x7f668419f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:52:18.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.590+0000 7f668a200640 1 --2- 172.21.15.83:0/3014445795 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6684113030 0x7f66841a36f0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f667c00ccb0 tx=0x7f667c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:18.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.590+0000 7f6682000640 1 -- 172.21.15.83:0/3014445795 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f667c00f450 con 0x7f6684113030 2024-04-24T08:52:18.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.590+0000 7f668c200640 1 -- 172.21.15.83:0/3014445795 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66841a4110 con 0x7f6684113030 2024-04-24T08:52:18.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.590+0000 7f6682000640 1 -- 172.21.15.83:0/3014445795 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f667c00faa0 con 0x7f6684113030 2024-04-24T08:52:18.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.590+0000 7f6682000640 1 -- 172.21.15.83:0/3014445795 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f667c018b10 con 0x7f6684113030 2024-04-24T08:52:18.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.590+0000 7f668c200640 1 -- 172.21.15.83:0/3014445795 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66841ab9a0 con 0x7f6684113030 2024-04-24T08:52:18.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.594+0000 7f668efc6740 1 Processor -- start 2024-04-24T08:52:18.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.594+0000 7f6682000640 1 -- 172.21.15.83:0/3014445795 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7f667c00f5f0 con 0x7f6684113030 2024-04-24T08:52:18.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.595+0000 7f668efc6740 1 -- start start 2024-04-24T08:52:18.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.595+0000 7f668efc6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5631a1662950 0x5631a1662d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:18.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.595+0000 7f668efc6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631a1663270 0x5631a166d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:18.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.595+0000 7f668efc6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631a166d940 0x5631a166fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:18.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.595+0000 7f668efc6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5631a15c81d0 con 0x5631a1663270 2024-04-24T08:52:18.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.595+0000 7f668efc6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5631a156e450 con 0x5631a1662950 2024-04-24T08:52:18.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.595+0000 7f668efc6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5631a15c4190 con 0x5631a166d940 2024-04-24T08:52:18.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.595+0000 7f6689800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631a1663270 0x5631a166d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:18.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.595+0000 7f6689800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631a1663270 0x5631a166d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55926/0 (socket says 172.21.15.83:55926) 2024-04-24T08:52:18.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.595+0000 7f6689800640 1 -- 172.21.15.83:0/870232709 learned_addr learned my addr 172.21.15.83:0/870232709 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:52:18.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.596+0000 7f6688e00640 1 --2- 172.21.15.83:0/870232709 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5631a1662950 0x5631a1662d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:18.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.596+0000 7f6688e00640 1 --2- 172.21.15.83:0/870232709 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631a166d940 0x5631a166fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:18.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.596+0000 7f6689800640 1 -- 172.21.15.83:0/870232709 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631a166d940 msgr2=0x5631a166fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:18.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.596+0000 7f6689800640 1 --2- 172.21.15.83:0/870232709 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631a166d940 0x5631a166fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.596+0000 7f6689800640 1 -- 172.21.15.83:0/870232709 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5631a1662950 msgr2=0x5631a1662d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:18.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.596+0000 7f6689800640 1 --2- 172.21.15.83:0/870232709 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5631a1662950 0x5631a1662d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.596+0000 7f6689800640 1 -- 172.21.15.83:0/870232709 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631a162b310 con 0x5631a1663270 2024-04-24T08:52:18.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.596+0000 7f6688e00640 1 --2- 172.21.15.83:0/870232709 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5631a1662950 0x5631a1662d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:52:18.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.596+0000 7f6682000640 1 --2- 172.21.15.83:0/3014445795 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f666c0a3ec0 0x7f666c0a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:18.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.596+0000 7f6689800640 1 --2- 172.21.15.83:0/870232709 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631a1663270 0x5631a166d400 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f6678009230 tx=0x7f6678004390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49387e8b131706e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:18.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.596+0000 7f6682000640 1 -- 172.21.15.83:0/3014445795 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f667c0d14c0 con 0x7f6684113030 2024-04-24T08:52:18.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.597+0000 7f6688e00640 1 --2- 172.21.15.83:0/3014445795 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f666c0a3ec0 0x7f666c0a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:18.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.597+0000 7f6676a00640 1 -- 172.21.15.83:0/870232709 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6678035030 con 0x5631a1663270 2024-04-24T08:52:18.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.597+0000 7f6676a00640 1 -- 172.21.15.83:0/870232709 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f66780046c0 con 0x5631a1663270 2024-04-24T08:52:18.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.597+0000 7f6676a00640 1 -- 172.21.15.83:0/870232709 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6678003850 con 0x5631a1663270 2024-04-24T08:52:18.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.597+0000 7f668efc6740 1 -- 172.21.15.83:0/870232709 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631a1663270 msgr2=0x5631a166d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:18.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.597+0000 7f668efc6740 1 --2- 172.21.15.83:0/870232709 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631a1663270 0x5631a166d400 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f6678009230 tx=0x7f6678004390 comp rx=0 tx=0).stop 2024-04-24T08:52:18.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.597+0000 7f6688e00640 1 --2- 172.21.15.83:0/3014445795 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f666c0a3ec0 0x7f666c0a6380 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f66700058c0 tx=0x7f6670005e50 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:18.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.597+0000 7f668efc6740 1 -- 172.21.15.83:0/870232709 shutdown_connections 2024-04-24T08:52:18.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.597+0000 7f668efc6740 1 --2- 172.21.15.83:0/870232709 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631a166d940 0x5631a166fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.597+0000 7f668efc6740 1 --2- 172.21.15.83:0/870232709 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631a1663270 0x5631a166d400 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.597+0000 7f668efc6740 1 --2- 172.21.15.83:0/870232709 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5631a1662950 0x5631a1662d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.597+0000 7f668efc6740 1 -- 172.21.15.83:0/870232709 >> 172.21.15.83:0/870232709 conn(0x5631a165ec70 msgr2=0x5631a165f050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:18.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.598+0000 7f668efc6740 1 -- 172.21.15.83:0/870232709 shutdown_connections 2024-04-24T08:52:18.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.598+0000 7f668efc6740 1 -- 172.21.15.83:0/870232709 wait complete. 2024-04-24T08:52:18.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.598+0000 7f668efc6740 1 -- start start 2024-04-24T08:52:18.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.598+0000 7f668efc6740 1 Processor -- start 2024-04-24T08:52:18.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.599+0000 7f668efc6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631a1662950 0x5631a166d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:18.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.599+0000 7f668efc6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631a1663270 0x5631a166a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:18.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.599+0000 7f668efc6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5631a166d940 0x5631a166ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:18.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.599+0000 7f668efc6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5631a1670770 con 0x5631a1663270 2024-04-24T08:52:18.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.599+0000 7f668efc6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5631a1581810 con 0x5631a166d940 2024-04-24T08:52:18.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.599+0000 7f668efc6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5631a16708f0 con 0x5631a1662950 2024-04-24T08:52:18.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.599+0000 7f668efc6740 10 client.0 _ll_register_callbacks cb 0x5631a154f140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:52:18.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.599+0000 7f6688e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631a1663270 0x5631a166a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:18.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.599+0000 7f6688e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631a1663270 0x5631a166a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55930/0 (socket says 172.21.15.83:55930) 2024-04-24T08:52:18.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.599+0000 7f6688e00640 1 -- 172.21.15.83:0/1911533361 learned_addr learned my addr 172.21.15.83:0/1911533361 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:52:18.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.599+0000 7f668a200640 1 --2- 172.21.15.83:0/1911533361 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5631a166d940 0x5631a166ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:18.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.599+0000 7f6689800640 1 --2- 172.21.15.83:0/1911533361 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631a1662950 0x5631a166d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:18.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.599+0000 7f6688e00640 1 -- 172.21.15.83:0/1911533361 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631a1662950 msgr2=0x5631a166d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:18.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.599+0000 7f6688e00640 1 --2- 172.21.15.83:0/1911533361 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631a1662950 0x5631a166d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.599+0000 7f6688e00640 1 -- 172.21.15.83:0/1911533361 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5631a166d940 msgr2=0x5631a166ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:18.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.599+0000 7f6688e00640 1 --2- 172.21.15.83:0/1911533361 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5631a166d940 0x5631a166ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.599+0000 7f6688e00640 1 -- 172.21.15.83:0/1911533361 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631a15923f0 con 0x5631a1663270 2024-04-24T08:52:18.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.599+0000 7f668a200640 1 --2- 172.21.15.83:0/1911533361 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5631a166d940 0x5631a166ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:52:18.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.599+0000 7f6689800640 1 --2- 172.21.15.83:0/1911533361 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631a1662950 0x5631a166d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:52:18.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.600+0000 7f6688e00640 1 --2- 172.21.15.83:0/1911533361 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631a1663270 0x5631a166a630 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f66700126b0 tx=0x7f6670005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d27f286aea73d856 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:18.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.600+0000 7f666b400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:52:18.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.600+0000 7f666b400640 1 -- 172.21.15.83:0/1911533361 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6670010a60 con 0x5631a1663270 2024-04-24T08:52:18.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.600+0000 7f666b400640 1 -- 172.21.15.83:0/1911533361 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f6670010c00 con 0x5631a1663270 2024-04-24T08:52:18.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.600+0000 7f668efc6740 1 -- 172.21.15.83:0/1911533361 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66780046c0 con 0x5631a1663270 2024-04-24T08:52:18.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.600+0000 7f666b400640 1 -- 172.21.15.83:0/1911533361 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f667002f050 con 0x5631a1663270 2024-04-24T08:52:18.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.600+0000 7f668efc6740 10 client.15969 fetch_fsmap learned FSMap version 28 2024-04-24T08:52:18.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.600+0000 7f668efc6740 1 -- 172.21.15.83:0/1911533361 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5631a12da3f0 con 0x5631a1663270 2024-04-24T08:52:18.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.601+0000 7f666b400640 1 -- 172.21.15.83:0/1911533361 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f66780046c0 con 0x5631a1663270 2024-04-24T08:52:18.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.601+0000 7f666b400640 1 -- 172.21.15.83:0/1911533361 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f66700273f0 con 0x5631a1663270 2024-04-24T08:52:18.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.601+0000 7f668efc6740 10 client.15969 fetch_fsmap finished waiting for FSMap version 28 2024-04-24T08:52:18.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.601+0000 7f668efc6740 10 client.15969 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-24T08:52:18.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.601+0000 7f668efc6740 20 client.15969 populate_metadata read hostname 'smithi083' 2024-04-24T08:52:18.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.601+0000 7f668efc6740 1 --2- 172.21.15.83:0/1911533361 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x5631a167d9a0 0x5631a167fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:18.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.601+0000 7f668efc6740 4 client.15969 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:52:18.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.601+0000 7f668efc6740 1 -- 172.21.15.83:0/1911533361 --> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5631a15d5bf0 con 0x5631a167d9a0 2024-04-24T08:52:18.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.602+0000 7f6689800640 1 --2- 172.21.15.83:0/1911533361 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x5631a167d9a0 0x5631a167fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:18.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.602+0000 7f6689800640 1 --2- 172.21.15.83:0/1911533361 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x5631a167d9a0 0x5631a167fe60 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9408720f9cd14a5e server_cookie=ccbeaa2840330a11 in_seq=0 out_seq=0 2024-04-24T08:52:18.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.602+0000 7f666b400640 10 client.15969 ms_handle_connect on v2:172.21.15.138:6836/900566754 2024-04-24T08:52:18.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.604+0000 7f666b400640 1 -- 172.21.15.83:0/1911533361 <== mds.1 v2:172.21.15.138:6836/900566754 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5631a15d5bf0 con 0x5631a167d9a0 2024-04-24T08:52:18.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.604+0000 7f666b400640 10 client.15969 handle_command_reply: tid=0 2024-04-24T08:52:18.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.604+0000 7f668efc6740 1 client.15969 shutdown 2024-04-24T08:52:18.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.605+0000 7f668efc6740 1 -- 172.21.15.83:0/1911533361 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631a1663270 msgr2=0x5631a166a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:18.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.605+0000 7f668efc6740 1 --2- 172.21.15.83:0/1911533361 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631a1663270 0x5631a166a630 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f66700126b0 tx=0x7f6670005850 comp rx=0 tx=0).stop 2024-04-24T08:52:18.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.605+0000 7f668efc6740 1 -- 172.21.15.83:0/1911533361 shutdown_connections 2024-04-24T08:52:18.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.605+0000 7f668efc6740 1 --2- 172.21.15.83:0/1911533361 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5631a166d940 0x5631a166ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.605+0000 7f668efc6740 1 --2- 172.21.15.83:0/1911533361 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631a1663270 0x5631a166a630 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.605+0000 7f668efc6740 1 --2- 172.21.15.83:0/1911533361 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x5631a167d9a0 0x5631a167fe60 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.605+0000 7f668efc6740 1 --2- 172.21.15.83:0/1911533361 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631a1662950 0x5631a166d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.605+0000 7f666b400640 0 client.15969 ms_handle_reset on v2:172.21.15.138:6836/900566754 2024-04-24T08:52:18.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.605+0000 7f668efc6740 1 -- 172.21.15.83:0/1911533361 >> 172.21.15.83:0/1911533361 conn(0x5631a165ec70 msgr2=0x5631a1664970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:18.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.605+0000 7f668efc6740 1 -- 172.21.15.83:0/1911533361 shutdown_connections 2024-04-24T08:52:18.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.605+0000 7f668efc6740 1 -- 172.21.15.83:0/1911533361 wait complete. 2024-04-24T08:52:18.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.605+0000 7f668efc6740 20 client.15969 trim_cache size 0 max 16384 2024-04-24T08:52:18.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.620+0000 7f668efc6740 1 Processor -- start 2024-04-24T08:52:18.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.620+0000 7f668efc6740 1 -- start start 2024-04-24T08:52:18.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.620+0000 7f668efc6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631a1662950 0x5631a1679660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:18.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.620+0000 7f668efc6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631a1663270 0x5631a166ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:18.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.620+0000 7f668efc6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5631a166d940 0x5631a1685730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:18.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.620+0000 7f668efc6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f66841b5a70 con 0x5631a1663270 2024-04-24T08:52:18.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.620+0000 7f668efc6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f66841b75b0 con 0x5631a166d940 2024-04-24T08:52:18.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.620+0000 7f668efc6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f66841b6140 con 0x5631a1662950 2024-04-24T08:52:18.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.620+0000 7f6688e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631a1663270 0x5631a166ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:18.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.620+0000 7f6688e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631a1663270 0x5631a166ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55940/0 (socket says 172.21.15.83:55940) 2024-04-24T08:52:18.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.621+0000 7f6688e00640 1 -- 172.21.15.83:0/1746551080 learned_addr learned my addr 172.21.15.83:0/1746551080 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:52:18.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.621+0000 7f6689800640 1 --2- 172.21.15.83:0/1746551080 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631a1662950 0x5631a1679660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:18.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.621+0000 7f668a200640 1 --2- 172.21.15.83:0/1746551080 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5631a166d940 0x5631a1685730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:18.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.621+0000 7f6688e00640 1 -- 172.21.15.83:0/1746551080 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631a1662950 msgr2=0x5631a1679660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:18.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.621+0000 7f6688e00640 1 --2- 172.21.15.83:0/1746551080 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631a1662950 0x5631a1679660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.621+0000 7f6688e00640 1 -- 172.21.15.83:0/1746551080 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5631a166d940 msgr2=0x5631a1685730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:18.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.621+0000 7f6688e00640 1 --2- 172.21.15.83:0/1746551080 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5631a166d940 0x5631a1685730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.621+0000 7f6688e00640 1 -- 172.21.15.83:0/1746551080 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631a13a47d0 con 0x5631a1663270 2024-04-24T08:52:18.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.621+0000 7f6688e00640 1 --2- 172.21.15.83:0/1746551080 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631a1663270 0x5631a166ce70 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f6670005d00 tx=0x7f6670027f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dac2635cb52665a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:18.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.621+0000 7f6676000640 1 -- 172.21.15.83:0/1746551080 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66700273f0 con 0x5631a1663270 2024-04-24T08:52:18.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.622+0000 7f6676000640 1 -- 172.21.15.83:0/1746551080 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6670027590 con 0x5631a1663270 2024-04-24T08:52:18.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.622+0000 7f6676000640 1 -- 172.21.15.83:0/1746551080 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6670003cd0 con 0x5631a1663270 2024-04-24T08:52:18.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.622+0000 7f668efc6740 1 -- 172.21.15.83:0/1746551080 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631a1663270 msgr2=0x5631a166ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:18.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.622+0000 7f668efc6740 1 --2- 172.21.15.83:0/1746551080 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631a1663270 0x5631a166ce70 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f6670005d00 tx=0x7f6670027f90 comp rx=0 tx=0).stop 2024-04-24T08:52:18.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.622+0000 7f668efc6740 1 -- 172.21.15.83:0/1746551080 shutdown_connections 2024-04-24T08:52:18.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.622+0000 7f668efc6740 1 --2- 172.21.15.83:0/1746551080 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5631a166d940 0x5631a1685730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.622+0000 7f668efc6740 1 --2- 172.21.15.83:0/1746551080 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631a1663270 0x5631a166ce70 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.622+0000 7f668efc6740 1 --2- 172.21.15.83:0/1746551080 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631a1662950 0x5631a1679660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.622+0000 7f668efc6740 1 -- 172.21.15.83:0/1746551080 >> 172.21.15.83:0/1746551080 conn(0x5631a165ec70 msgr2=0x5631a165f050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:18.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.622+0000 7f668efc6740 1 -- 172.21.15.83:0/1746551080 shutdown_connections 2024-04-24T08:52:18.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.622+0000 7f668efc6740 1 -- 172.21.15.83:0/1746551080 wait complete. 2024-04-24T08:52:18.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.623+0000 7f668efc6740 1 -- start start 2024-04-24T08:52:18.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.623+0000 7f668efc6740 1 Processor -- start 2024-04-24T08:52:18.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.623+0000 7f668efc6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631a1662950 0x5631a1675db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:18.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.623+0000 7f668efc6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631a1663270 0x5631a166e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:18.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.623+0000 7f668efc6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5631a166d940 0x5631a166e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:18.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.623+0000 7f668efc6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f66841b6ee0 con 0x5631a1662950 2024-04-24T08:52:18.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.623+0000 7f668efc6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f66841b6810 con 0x5631a166d940 2024-04-24T08:52:18.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.623+0000 7f668efc6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f66841b3a50 con 0x5631a1663270 2024-04-24T08:52:18.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.623+0000 7f668efc6740 10 client.0 _ll_register_callbacks cb 0x7f667000de20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:52:18.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.623+0000 7f6689800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631a1662950 0x5631a1675db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:18.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.624+0000 7f6689800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631a1662950 0x5631a1675db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55948/0 (socket says 172.21.15.83:55948) 2024-04-24T08:52:18.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.624+0000 7f6689800640 1 -- 172.21.15.83:0/4198349313 learned_addr learned my addr 172.21.15.83:0/4198349313 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:52:18.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.624+0000 7f6688e00640 1 --2- 172.21.15.83:0/4198349313 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631a1663270 0x5631a166e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:18.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.624+0000 7f668a200640 1 --2- 172.21.15.83:0/4198349313 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5631a166d940 0x5631a166e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:18.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.624+0000 7f6689800640 1 -- 172.21.15.83:0/4198349313 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631a1663270 msgr2=0x5631a166e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:18.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.624+0000 7f6689800640 1 --2- 172.21.15.83:0/4198349313 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631a1663270 0x5631a166e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.624+0000 7f6689800640 1 -- 172.21.15.83:0/4198349313 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5631a166d940 msgr2=0x5631a166e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:18.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.624+0000 7f6689800640 1 --2- 172.21.15.83:0/4198349313 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5631a166d940 0x5631a166e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.624+0000 7f6689800640 1 -- 172.21.15.83:0/4198349313 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631a166eea0 con 0x5631a1662950 2024-04-24T08:52:18.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.624+0000 7f6688e00640 1 --2- 172.21.15.83:0/4198349313 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631a1663270 0x5631a166e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:52:18.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.624+0000 7f6689800640 1 --2- 172.21.15.83:0/4198349313 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631a1662950 0x5631a1675db0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f6678031440 tx=0x7f667802a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=922660dc62ab36b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:18.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.624+0000 7f666b400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:52:18.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.624+0000 7f666b400640 1 -- 172.21.15.83:0/4198349313 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6678055060 con 0x5631a1662950 2024-04-24T08:52:18.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.625+0000 7f666b400640 1 -- 172.21.15.83:0/4198349313 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f6678031c90 con 0x5631a1662950 2024-04-24T08:52:18.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.625+0000 7f668efc6740 1 -- 172.21.15.83:0/4198349313 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6670027590 con 0x5631a1662950 2024-04-24T08:52:18.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.625+0000 7f666b400640 1 -- 172.21.15.83:0/4198349313 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f667805c050 con 0x5631a1662950 2024-04-24T08:52:18.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.625+0000 7f668efc6740 10 client.15975 fetch_fsmap learned FSMap version 28 2024-04-24T08:52:18.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.625+0000 7f668efc6740 1 -- 172.21.15.83:0/4198349313 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5631a16e3fa0 con 0x5631a1662950 2024-04-24T08:52:18.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.625+0000 7f666b400640 1 -- 172.21.15.83:0/4198349313 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6670027590 con 0x5631a1662950 2024-04-24T08:52:18.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.625+0000 7f666b400640 1 -- 172.21.15.83:0/4198349313 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f6678004ae0 con 0x5631a1662950 2024-04-24T08:52:18.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.625+0000 7f668efc6740 10 client.15975 fetch_fsmap finished waiting for FSMap version 28 2024-04-24T08:52:18.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.625+0000 7f668efc6740 10 client.15975 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-24T08:52:18.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.625+0000 7f668efc6740 20 client.15975 populate_metadata read hostname 'smithi083' 2024-04-24T08:52:18.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.625+0000 7f668efc6740 1 --2- 172.21.15.83:0/4198349313 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x5631a16e4230 0x5631a16e6620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:18.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.625+0000 7f668efc6740 4 client.15975 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T08:52:18.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.625+0000 7f668efc6740 1 -- 172.21.15.83:0/4198349313 --> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5631a15cfa60 con 0x5631a16e4230 2024-04-24T08:52:18.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.626+0000 7f6688e00640 1 --2- 172.21.15.83:0/4198349313 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x5631a16e4230 0x5631a16e6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:18.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.627+0000 7f6688e00640 1 --2- 172.21.15.83:0/4198349313 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x5631a16e4230 0x5631a16e6620 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=35dc58c5d31fb5c8 server_cookie=88d448d62ea4e7c1 in_seq=0 out_seq=0 2024-04-24T08:52:18.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.627+0000 7f666b400640 10 client.15975 ms_handle_connect on v2:172.21.15.138:6836/900566754 2024-04-24T08:52:18.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.628+0000 7f666b400640 1 -- 172.21.15.83:0/4198349313 <== mds.1 v2:172.21.15.138:6836/900566754 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5631a15cfa60 con 0x5631a16e4230 2024-04-24T08:52:18.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.628+0000 7f666b400640 10 client.15975 handle_command_reply: tid=0 2024-04-24T08:52:18.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.628+0000 7f668efc6740 1 client.15975 shutdown 2024-04-24T08:52:18.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.628+0000 7f668efc6740 1 -- 172.21.15.83:0/4198349313 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631a1662950 msgr2=0x5631a1675db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:18.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.628+0000 7f668efc6740 1 --2- 172.21.15.83:0/4198349313 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631a1662950 0x5631a1675db0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f6678031440 tx=0x7f667802a480 comp rx=0 tx=0).stop 2024-04-24T08:52:18.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.628+0000 7f668efc6740 1 -- 172.21.15.83:0/4198349313 shutdown_connections 2024-04-24T08:52:18.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.628+0000 7f668efc6740 1 --2- 172.21.15.83:0/4198349313 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5631a166d940 0x5631a166e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.628+0000 7f668efc6740 1 --2- 172.21.15.83:0/4198349313 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x5631a16e4230 0x5631a16e6620 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.628+0000 7f668efc6740 1 --2- 172.21.15.83:0/4198349313 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631a1663270 0x5631a166e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.628+0000 7f666b400640 0 client.15975 ms_handle_reset on v2:172.21.15.138:6836/900566754 2024-04-24T08:52:18.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.628+0000 7f668efc6740 1 --2- 172.21.15.83:0/4198349313 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631a1662950 0x5631a1675db0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.628+0000 7f668efc6740 1 -- 172.21.15.83:0/4198349313 >> 172.21.15.83:0/4198349313 conn(0x5631a165ec70 msgr2=0x5631a1668b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:18.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.628+0000 7f668efc6740 1 -- 172.21.15.83:0/4198349313 shutdown_connections 2024-04-24T08:52:18.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.628+0000 7f668efc6740 1 -- 172.21.15.83:0/4198349313 wait complete. 2024-04-24T08:52:18.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.628+0000 7f668efc6740 20 client.15975 trim_cache size 0 max 16384 2024-04-24T08:52:18.631 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T08:52:18.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.629+0000 7f668c200640 1 -- 172.21.15.83:0/3014445795 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f666c0a3ec0 msgr2=0x7f666c0a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:18.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.629+0000 7f668c200640 1 --2- 172.21.15.83:0/3014445795 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f666c0a3ec0 0x7f666c0a6380 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f66700058c0 tx=0x7f6670005e50 comp rx=0 tx=0).stop 2024-04-24T08:52:18.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.629+0000 7f668c200640 1 -- 172.21.15.83:0/3014445795 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6684113030 msgr2=0x7f66841a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:18.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.629+0000 7f668c200640 1 --2- 172.21.15.83:0/3014445795 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6684113030 0x7f66841a36f0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f667c00ccb0 tx=0x7f667c00a620 comp rx=0 tx=0).stop 2024-04-24T08:52:18.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.629+0000 7f668c200640 1 -- 172.21.15.83:0/3014445795 shutdown_connections 2024-04-24T08:52:18.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.629+0000 7f668c200640 1 --2- 172.21.15.83:0/3014445795 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f666c0a3ec0 0x7f666c0a6380 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.629+0000 7f668c200640 1 --2- 172.21.15.83:0/3014445795 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6684113030 0x7f66841a36f0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.629+0000 7f668c200640 1 --2- 172.21.15.83:0/3014445795 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6684074cc0 0x7f668419f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.629+0000 7f668c200640 1 --2- 172.21.15.83:0/3014445795 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f66840743a0 0x7f668419ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.629+0000 7f668c200640 1 -- 172.21.15.83:0/3014445795 >> 172.21.15.83:0/3014445795 conn(0x7f66840fdb10 msgr2=0x7f66840ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:18.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.629+0000 7f668c200640 1 -- 172.21.15.83:0/3014445795 shutdown_connections 2024-04-24T08:52:18.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.629+0000 7f668c200640 1 -- 172.21.15.83:0/3014445795 wait complete. 2024-04-24T08:52:18.643 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:52:18 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/1122855404' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:52:18.646 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T08:52:18.821 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:52:18 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/1122855404' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:52:18.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.845+0000 7f4c47e00640 1 -- 172.21.15.83:0/123246188 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c48104cc0 msgr2=0x7f4c48108d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:18.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.845+0000 7f4c47e00640 1 --2- 172.21.15.83:0/123246188 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c48104cc0 0x7f4c48108d00 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f4c30009900 tx=0x7f4c3002a3f0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.845+0000 7f4c47e00640 1 -- 172.21.15.83:0/123246188 shutdown_connections 2024-04-24T08:52:18.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.845+0000 7f4c47e00640 1 --2- 172.21.15.83:0/123246188 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c481093e0 0x7f4c48110fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.845+0000 7f4c47e00640 1 --2- 172.21.15.83:0/123246188 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c48104cc0 0x7f4c48108d00 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.845+0000 7f4c47e00640 1 --2- 172.21.15.83:0/123246188 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4c48104310 0x7f4c481046f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.845+0000 7f4c47e00640 1 -- 172.21.15.83:0/123246188 >> 172.21.15.83:0/123246188 conn(0x7f4c480ffb70 msgr2=0x7f4c48101f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:18.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.845+0000 7f4c47e00640 1 -- 172.21.15.83:0/123246188 shutdown_connections 2024-04-24T08:52:18.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.845+0000 7f4c47e00640 1 -- 172.21.15.83:0/123246188 wait complete. 2024-04-24T08:52:18.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.847+0000 7f4c47e00640 1 Processor -- start 2024-04-24T08:52:18.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.847+0000 7f4c47e00640 1 -- start start 2024-04-24T08:52:18.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.847+0000 7f4c47e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c48104310 0x7f4c481a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:18.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.847+0000 7f4c47e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4c481093e0 0x7f4c481a0b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:18.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.847+0000 7f4c47e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c4819a6b0 0x7f4c4819ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:18.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.847+0000 7f4c47e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4c48114370 con 0x7f4c4819a6b0 2024-04-24T08:52:18.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.847+0000 7f4c47e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4c481141f0 con 0x7f4c48104310 2024-04-24T08:52:18.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.847+0000 7f4c47e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4c481144f0 con 0x7f4c481093e0 2024-04-24T08:52:18.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.848+0000 7f4c47400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c4819a6b0 0x7f4c4819ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:18.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.848+0000 7f4c47400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c4819a6b0 0x7f4c4819ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55902/0 (socket says 172.21.15.83:55902) 2024-04-24T08:52:18.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.848+0000 7f4c47400640 1 -- 172.21.15.83:0/2226932848 learned_addr learned my addr 172.21.15.83:0/2226932848 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:52:18.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.848+0000 7f4c46a00640 1 --2- 172.21.15.83:0/2226932848 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c48104310 0x7f4c481a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:18.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.848+0000 7f4c3e000640 1 --2- 172.21.15.83:0/2226932848 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4c481093e0 0x7f4c481a0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:18.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.848+0000 7f4c47400640 1 -- 172.21.15.83:0/2226932848 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4c481093e0 msgr2=0x7f4c481a0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:18.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.848+0000 7f4c47400640 1 --2- 172.21.15.83:0/2226932848 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4c481093e0 0x7f4c481a0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.848+0000 7f4c47400640 1 -- 172.21.15.83:0/2226932848 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c48104310 msgr2=0x7f4c481a05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:18.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.848+0000 7f4c47400640 1 --2- 172.21.15.83:0/2226932848 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c48104310 0x7f4c481a05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.848+0000 7f4c47400640 1 -- 172.21.15.83:0/2226932848 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c4819b2b0 con 0x7f4c4819a6b0 2024-04-24T08:52:18.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.848+0000 7f4c47400640 1 --2- 172.21.15.83:0/2226932848 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c4819a6b0 0x7f4c4819ab30 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f4c4000c970 tx=0x7f4c4000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:18.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.849+0000 7f4c3fe00640 1 -- 172.21.15.83:0/2226932848 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c40007bb0 con 0x7f4c4819a6b0 2024-04-24T08:52:18.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.849+0000 7f4c3fe00640 1 -- 172.21.15.83:0/2226932848 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4c40007d50 con 0x7f4c4819a6b0 2024-04-24T08:52:18.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.849+0000 7f4c3fe00640 1 -- 172.21.15.83:0/2226932848 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c40018700 con 0x7f4c4819a6b0 2024-04-24T08:52:18.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.849+0000 7f4c47e00640 1 -- 172.21.15.83:0/2226932848 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c481a7390 con 0x7f4c4819a6b0 2024-04-24T08:52:18.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.849+0000 7f4c47e00640 1 -- 172.21.15.83:0/2226932848 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c481a7740 con 0x7f4c4819a6b0 2024-04-24T08:52:18.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.852+0000 7f4c50384740 1 Processor -- start 2024-04-24T08:52:18.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.852+0000 7f4c3fe00640 1 -- 172.21.15.83:0/2226932848 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7f4c4000fa20 con 0x7f4c4819a6b0 2024-04-24T08:52:18.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.852+0000 7f4c50384740 1 -- start start 2024-04-24T08:52:18.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.853+0000 7f4c50384740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f6c63f8b40 0x55f6c63f8f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:18.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.853+0000 7f4c50384740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6c63f9460 0x55f6c64035f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:18.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.853+0000 7f4c50384740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6c6403b30 0x55f6c6405f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:18.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.853+0000 7f4c50384740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55f6c635e1d0 con 0x55f6c6403b30 2024-04-24T08:52:18.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.853+0000 7f4c50384740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f6c6304450 con 0x55f6c63f9460 2024-04-24T08:52:18.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.853+0000 7f4c50384740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55f6c635a190 con 0x55f6c63f8b40 2024-04-24T08:52:18.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.853+0000 7f4c3e000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6c6403b30 0x55f6c6405f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:18.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.853+0000 7f4c3e000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6c6403b30 0x55f6c6405f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55906/0 (socket says 172.21.15.83:55906) 2024-04-24T08:52:18.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.853+0000 7f4c3e000640 1 -- 172.21.15.83:0/2429523996 learned_addr learned my addr 172.21.15.83:0/2429523996 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:52:18.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.853+0000 7f4c46a00640 1 --2- 172.21.15.83:0/2429523996 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6c63f9460 0x55f6c64035f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:18.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.854+0000 7f4c3e000640 1 --2- 172.21.15.83:0/2429523996 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f6c63f8b40 0x55f6c63f8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:18.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.854+0000 7f4c3e000640 1 -- 172.21.15.83:0/2429523996 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f6c63f8b40 msgr2=0x55f6c63f8f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:18.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.854+0000 7f4c3e000640 1 --2- 172.21.15.83:0/2429523996 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f6c63f8b40 0x55f6c63f8f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.854+0000 7f4c3e000640 1 -- 172.21.15.83:0/2429523996 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6c63f9460 msgr2=0x55f6c64035f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:18.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.854+0000 7f4c3e000640 1 --2- 172.21.15.83:0/2429523996 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6c63f9460 0x55f6c64035f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.854+0000 7f4c3e000640 1 -- 172.21.15.83:0/2429523996 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6c63c1310 con 0x55f6c6403b30 2024-04-24T08:52:18.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.854+0000 7f4c3fe00640 1 --2- 172.21.15.83:0/2226932848 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f4c2c0a3ec0 0x7f4c2c0a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:18.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.854+0000 7f4c3fe00640 1 -- 172.21.15.83:0/2226932848 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f4c400d0f70 con 0x7f4c4819a6b0 2024-04-24T08:52:18.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.854+0000 7f4c46a00640 1 --2- 172.21.15.83:0/2226932848 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f4c2c0a3ec0 0x7f4c2c0a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:18.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.855+0000 7f4c3e000640 1 --2- 172.21.15.83:0/2429523996 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6c6403b30 0x55f6c6405f20 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f4c300096c0 tx=0x7f4c30000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=668475706a64356c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:18.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.855+0000 7f4c37400640 1 -- 172.21.15.83:0/2429523996 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c30035040 con 0x55f6c6403b30 2024-04-24T08:52:18.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.855+0000 7f4c37400640 1 -- 172.21.15.83:0/2429523996 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4c300038c0 con 0x55f6c6403b30 2024-04-24T08:52:18.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.855+0000 7f4c37400640 1 -- 172.21.15.83:0/2429523996 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c300048c0 con 0x55f6c6403b30 2024-04-24T08:52:18.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.856+0000 7f4c50384740 1 -- 172.21.15.83:0/2429523996 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6c6403b30 msgr2=0x55f6c6405f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:18.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.856+0000 7f4c46a00640 1 --2- 172.21.15.83:0/2226932848 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f4c2c0a3ec0 0x7f4c2c0a6380 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x55f6c63fa100 tx=0x7f4c380097c0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:18.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.856+0000 7f4c50384740 1 --2- 172.21.15.83:0/2429523996 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6c6403b30 0x55f6c6405f20 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f4c300096c0 tx=0x7f4c30000bc0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.856+0000 7f4c50384740 1 -- 172.21.15.83:0/2429523996 shutdown_connections 2024-04-24T08:52:18.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.856+0000 7f4c50384740 1 --2- 172.21.15.83:0/2429523996 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6c6403b30 0x55f6c6405f20 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.856+0000 7f4c50384740 1 --2- 172.21.15.83:0/2429523996 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6c63f9460 0x55f6c64035f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.856+0000 7f4c50384740 1 --2- 172.21.15.83:0/2429523996 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f6c63f8b40 0x55f6c63f8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.856+0000 7f4c50384740 1 -- 172.21.15.83:0/2429523996 >> 172.21.15.83:0/2429523996 conn(0x55f6c63f4e60 msgr2=0x55f6c63f5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:18.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.856+0000 7f4c50384740 1 -- 172.21.15.83:0/2429523996 shutdown_connections 2024-04-24T08:52:18.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.856+0000 7f4c50384740 1 -- 172.21.15.83:0/2429523996 wait complete. 2024-04-24T08:52:18.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.856+0000 7f4c50384740 1 -- start start 2024-04-24T08:52:18.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.856+0000 7f4c50384740 1 Processor -- start 2024-04-24T08:52:18.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.857+0000 7f4c50384740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f6c63f8b40 0x55f6c64035c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:18.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.857+0000 7f4c50384740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6c63f9460 0x55f6c64008b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:18.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.857+0000 7f4c50384740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6c6403b30 0x55f6c6400fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:18.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.857+0000 7f4c50384740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55f6c6406960 con 0x55f6c6403b30 2024-04-24T08:52:18.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.857+0000 7f4c50384740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f6c6317810 con 0x55f6c63f9460 2024-04-24T08:52:18.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.857+0000 7f4c50384740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55f6c6406ae0 con 0x55f6c63f8b40 2024-04-24T08:52:18.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.857+0000 7f4c50384740 10 client.0 _ll_register_callbacks cb 0x55f6c62e5140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:52:18.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.857+0000 7f4c47400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6c6403b30 0x55f6c6400fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:18.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.857+0000 7f4c47400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6c6403b30 0x55f6c6400fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55918/0 (socket says 172.21.15.83:55918) 2024-04-24T08:52:18.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.857+0000 7f4c47400640 1 -- 172.21.15.83:0/2624421290 learned_addr learned my addr 172.21.15.83:0/2624421290 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:52:18.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.857+0000 7f4c46a00640 1 --2- 172.21.15.83:0/2624421290 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f6c63f8b40 0x55f6c64035c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:18.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.858+0000 7f4c3e000640 1 --2- 172.21.15.83:0/2624421290 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6c63f9460 0x55f6c64008b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:18.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.858+0000 7f4c47400640 1 -- 172.21.15.83:0/2624421290 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f6c63f8b40 msgr2=0x55f6c64035c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:18.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.858+0000 7f4c47400640 1 --2- 172.21.15.83:0/2624421290 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f6c63f8b40 0x55f6c64035c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.858+0000 7f4c47400640 1 -- 172.21.15.83:0/2624421290 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6c63f9460 msgr2=0x55f6c64008b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:18.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.858+0000 7f4c47400640 1 --2- 172.21.15.83:0/2624421290 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6c63f9460 0x55f6c64008b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.858+0000 7f4c47400640 1 -- 172.21.15.83:0/2624421290 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6c63283f0 con 0x55f6c6403b30 2024-04-24T08:52:18.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.858+0000 7f4c47400640 1 --2- 172.21.15.83:0/2624421290 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6c6403b30 0x55f6c6400fa0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f4c40098cf0 tx=0x7f4c4009c2b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1408235e7473eb04 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:18.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.858+0000 7f4c2be00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:52:18.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.858+0000 7f4c2be00640 1 -- 172.21.15.83:0/2624421290 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c400a5270 con 0x55f6c6403b30 2024-04-24T08:52:18.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.858+0000 7f4c2be00640 1 -- 172.21.15.83:0/2624421290 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f4c4009dbc0 con 0x55f6c6403b30 2024-04-24T08:52:18.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.858+0000 7f4c50384740 1 -- 172.21.15.83:0/2624421290 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c300038c0 con 0x55f6c6403b30 2024-04-24T08:52:18.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.859+0000 7f4c2be00640 1 -- 172.21.15.83:0/2624421290 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4c4009de40 con 0x55f6c6403b30 2024-04-24T08:52:18.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.859+0000 7f4c50384740 10 client.15987 fetch_fsmap learned FSMap version 28 2024-04-24T08:52:18.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.859+0000 7f4c50384740 1 -- 172.21.15.83:0/2624421290 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f6c60703f0 con 0x55f6c6403b30 2024-04-24T08:52:18.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.859+0000 7f4c2be00640 1 -- 172.21.15.83:0/2624421290 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4c300038c0 con 0x55f6c6403b30 2024-04-24T08:52:18.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.859+0000 7f4c2be00640 1 -- 172.21.15.83:0/2624421290 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4c4000f450 con 0x55f6c6403b30 2024-04-24T08:52:18.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.859+0000 7f4c50384740 10 client.15987 fetch_fsmap finished waiting for FSMap version 28 2024-04-24T08:52:18.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.859+0000 7f4c50384740 10 client.15987 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:52:18.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.859+0000 7f4c50384740 20 client.15987 populate_metadata read hostname 'smithi083' 2024-04-24T08:52:18.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.859+0000 7f4c50384740 1 --2- 172.21.15.83:0/2624421290 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55f6c6413d70 0x55f6c6416160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:18.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.859+0000 7f4c50384740 4 client.15987 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:52:18.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.859+0000 7f4c50384740 1 -- 172.21.15.83:0/2624421290 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f6c636bbf0 con 0x55f6c6413d70 2024-04-24T08:52:18.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.860+0000 7f4c46a00640 1 --2- 172.21.15.83:0/2624421290 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55f6c6413d70 0x55f6c6416160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:18.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.861+0000 7f4c46a00640 1 --2- 172.21.15.83:0/2624421290 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55f6c6413d70 0x55f6c6416160 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5e7a4c0fe4820c0 server_cookie=f75b11ac9a623139 in_seq=0 out_seq=0 2024-04-24T08:52:18.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.861+0000 7f4c2be00640 10 client.15987 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:52:18.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.863+0000 7f4c2be00640 1 -- 172.21.15.83:0/2624421290 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f6c636bbf0 con 0x55f6c6413d70 2024-04-24T08:52:18.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.863+0000 7f4c2be00640 10 client.15987 handle_command_reply: tid=0 2024-04-24T08:52:18.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.863+0000 7f4c50384740 1 client.15987 shutdown 2024-04-24T08:52:18.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.863+0000 7f4c50384740 1 -- 172.21.15.83:0/2624421290 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6c6403b30 msgr2=0x55f6c6400fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:18.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.863+0000 7f4c50384740 1 --2- 172.21.15.83:0/2624421290 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6c6403b30 0x55f6c6400fa0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f4c40098cf0 tx=0x7f4c4009c2b0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.863+0000 7f4c50384740 1 -- 172.21.15.83:0/2624421290 shutdown_connections 2024-04-24T08:52:18.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.863+0000 7f4c50384740 1 --2- 172.21.15.83:0/2624421290 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55f6c6413d70 0x55f6c6416160 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.863+0000 7f4c50384740 1 --2- 172.21.15.83:0/2624421290 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6c6403b30 0x55f6c6400fa0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.863+0000 7f4c50384740 1 --2- 172.21.15.83:0/2624421290 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6c63f9460 0x55f6c64008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.863+0000 7f4c2be00640 0 client.15987 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:52:18.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.863+0000 7f4c50384740 1 --2- 172.21.15.83:0/2624421290 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f6c63f8b40 0x55f6c64035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.863+0000 7f4c50384740 1 -- 172.21.15.83:0/2624421290 >> 172.21.15.83:0/2624421290 conn(0x55f6c63f4e60 msgr2=0x55f6c63fac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:18.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.863+0000 7f4c50384740 1 -- 172.21.15.83:0/2624421290 shutdown_connections 2024-04-24T08:52:18.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.863+0000 7f4c50384740 1 -- 172.21.15.83:0/2624421290 wait complete. 2024-04-24T08:52:18.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.863+0000 7f4c50384740 20 client.15987 trim_cache size 0 max 16384 2024-04-24T08:52:18.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.878+0000 7f4c50384740 1 Processor -- start 2024-04-24T08:52:18.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.878+0000 7f4c50384740 1 -- start start 2024-04-24T08:52:18.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.878+0000 7f4c50384740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6c63f8b40 0x55f6c63fb2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:18.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.878+0000 7f4c50384740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6c63f9460 0x55f6c64030f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:18.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.878+0000 7f4c50384740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f6c6403b30 0x55f6c641b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:18.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.878+0000 7f4c50384740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4c481b1440 con 0x55f6c63f8b40 2024-04-24T08:52:18.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.878+0000 7f4c50384740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4c481b2f80 con 0x55f6c63f9460 2024-04-24T08:52:18.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.878+0000 7f4c50384740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4c481b1b10 con 0x55f6c6403b30 2024-04-24T08:52:18.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.879+0000 7f4c46a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6c63f8b40 0x55f6c63fb2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:18.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.879+0000 7f4c46a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6c63f8b40 0x55f6c63fb2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55928/0 (socket says 172.21.15.83:55928) 2024-04-24T08:52:18.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.879+0000 7f4c46a00640 1 -- 172.21.15.83:0/2446281876 learned_addr learned my addr 172.21.15.83:0/2446281876 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:52:18.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.879+0000 7f4c47400640 1 --2- 172.21.15.83:0/2446281876 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f6c6403b30 0x55f6c641b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:18.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.879+0000 7f4c46a00640 1 -- 172.21.15.83:0/2446281876 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f6c6403b30 msgr2=0x55f6c641b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:18.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.879+0000 7f4c46a00640 1 --2- 172.21.15.83:0/2446281876 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f6c6403b30 0x55f6c641b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.879+0000 7f4c46a00640 1 -- 172.21.15.83:0/2446281876 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6c63f9460 msgr2=0x55f6c64030f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:18.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.879+0000 7f4c46a00640 1 --2- 172.21.15.83:0/2446281876 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6c63f9460 0x55f6c64030f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.879+0000 7f4c46a00640 1 -- 172.21.15.83:0/2446281876 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6c64164d0 con 0x55f6c63f8b40 2024-04-24T08:52:18.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.879+0000 7f4c46a00640 1 --2- 172.21.15.83:0/2446281876 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6c63f8b40 0x55f6c63fb2a0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x55f6c6416920 tx=0x7f4c38015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6447bfee33f5f3e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:18.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.880+0000 7f4c36a00640 1 -- 172.21.15.83:0/2446281876 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c38004b20 con 0x55f6c63f8b40 2024-04-24T08:52:18.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.880+0000 7f4c36a00640 1 -- 172.21.15.83:0/2446281876 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4c3801dd80 con 0x55f6c63f8b40 2024-04-24T08:52:18.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.880+0000 7f4c36a00640 1 -- 172.21.15.83:0/2446281876 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c38028640 con 0x55f6c63f8b40 2024-04-24T08:52:18.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.880+0000 7f4c50384740 1 -- 172.21.15.83:0/2446281876 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6c63f8b40 msgr2=0x55f6c63fb2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:18.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.880+0000 7f4c50384740 1 --2- 172.21.15.83:0/2446281876 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6c63f8b40 0x55f6c63fb2a0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x55f6c6416920 tx=0x7f4c38015000 comp rx=0 tx=0).stop 2024-04-24T08:52:18.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.881+0000 7f4c50384740 1 -- 172.21.15.83:0/2446281876 shutdown_connections 2024-04-24T08:52:18.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.881+0000 7f4c50384740 1 --2- 172.21.15.83:0/2446281876 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f6c6403b30 0x55f6c641b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.881+0000 7f4c50384740 1 --2- 172.21.15.83:0/2446281876 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6c63f9460 0x55f6c64030f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.881+0000 7f4c50384740 1 --2- 172.21.15.83:0/2446281876 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6c63f8b40 0x55f6c63fb2a0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.881+0000 7f4c50384740 1 -- 172.21.15.83:0/2446281876 >> 172.21.15.83:0/2446281876 conn(0x55f6c63f4e60 msgr2=0x55f6c63fbbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:18.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.881+0000 7f4c50384740 1 -- 172.21.15.83:0/2446281876 shutdown_connections 2024-04-24T08:52:18.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.881+0000 7f4c50384740 1 -- 172.21.15.83:0/2446281876 wait complete. 2024-04-24T08:52:18.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.881+0000 7f4c50384740 1 -- start start 2024-04-24T08:52:18.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.881+0000 7f4c50384740 1 Processor -- start 2024-04-24T08:52:18.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.882+0000 7f4c50384740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f6c63f8b40 0x55f6c640bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:18.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.882+0000 7f4c50384740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6c63f9460 0x55f6c640c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:18.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.882+0000 7f4c50384740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6c6403b30 0x55f6c6404e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:18.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.882+0000 7f4c50384740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4c481b28b0 con 0x55f6c63f9460 2024-04-24T08:52:18.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.882+0000 7f4c50384740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4c481b21e0 con 0x55f6c6403b30 2024-04-24T08:52:18.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.882+0000 7f4c50384740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4c481af420 con 0x55f6c63f8b40 2024-04-24T08:52:18.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.882+0000 7f4c50384740 10 client.0 _ll_register_callbacks cb 0x7f4c4000a4c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:52:18.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.882+0000 7f4c3e000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6c63f9460 0x55f6c640c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:18.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.882+0000 7f4c3e000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6c63f9460 0x55f6c640c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55934/0 (socket says 172.21.15.83:55934) 2024-04-24T08:52:18.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.882+0000 7f4c3e000640 1 -- 172.21.15.83:0/4171367233 learned_addr learned my addr 172.21.15.83:0/4171367233 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:52:18.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.882+0000 7f4c46a00640 1 --2- 172.21.15.83:0/4171367233 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f6c63f8b40 0x55f6c640bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:18.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.882+0000 7f4c47400640 1 --2- 172.21.15.83:0/4171367233 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6c6403b30 0x55f6c6404e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:18.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.882+0000 7f4c3e000640 1 -- 172.21.15.83:0/4171367233 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f6c63f8b40 msgr2=0x55f6c640bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:18.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.882+0000 7f4c3e000640 1 --2- 172.21.15.83:0/4171367233 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f6c63f8b40 0x55f6c640bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.882+0000 7f4c3e000640 1 -- 172.21.15.83:0/4171367233 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6c6403b30 msgr2=0x55f6c6404e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:18.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.882+0000 7f4c3e000640 1 --2- 172.21.15.83:0/4171367233 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6c6403b30 0x55f6c6404e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.882+0000 7f4c3e000640 1 -- 172.21.15.83:0/4171367233 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6c613a7d0 con 0x55f6c63f9460 2024-04-24T08:52:18.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.882+0000 7f4c47400640 1 --2- 172.21.15.83:0/4171367233 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6c6403b30 0x55f6c6404e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:52:18.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.883+0000 7f4c3e000640 1 --2- 172.21.15.83:0/4171367233 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6c63f9460 0x55f6c640c2a0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x55f6c63f5ff0 tx=0x7f4c30003f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a2ba1ef247b1cb3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:18.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.883+0000 7f4c2be00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:52:18.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.883+0000 7f4c2be00640 1 -- 172.21.15.83:0/4171367233 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c30035070 con 0x55f6c63f9460 2024-04-24T08:52:18.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.883+0000 7f4c2be00640 1 -- 172.21.15.83:0/4171367233 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f4c300318b0 con 0x55f6c63f9460 2024-04-24T08:52:18.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.883+0000 7f4c50384740 1 -- 172.21.15.83:0/4171367233 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c3801dd80 con 0x55f6c63f9460 2024-04-24T08:52:18.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.883+0000 7f4c2be00640 1 -- 172.21.15.83:0/4171367233 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4c30058050 con 0x55f6c63f9460 2024-04-24T08:52:18.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.883+0000 7f4c50384740 10 client.15993 fetch_fsmap learned FSMap version 28 2024-04-24T08:52:18.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.883+0000 7f4c50384740 1 -- 172.21.15.83:0/4171367233 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f6c647a850 con 0x55f6c63f9460 2024-04-24T08:52:18.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.884+0000 7f4c2be00640 1 -- 172.21.15.83:0/4171367233 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4c3801dd80 con 0x55f6c63f9460 2024-04-24T08:52:18.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.884+0000 7f4c2be00640 1 -- 172.21.15.83:0/4171367233 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4c3003a040 con 0x55f6c63f9460 2024-04-24T08:52:18.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.884+0000 7f4c50384740 10 client.15993 fetch_fsmap finished waiting for FSMap version 28 2024-04-24T08:52:18.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.884+0000 7f4c50384740 10 client.15993 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:52:18.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.884+0000 7f4c50384740 20 client.15993 populate_metadata read hostname 'smithi083' 2024-04-24T08:52:18.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.884+0000 7f4c50384740 1 --2- 172.21.15.83:0/4171367233 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55f6c647ab30 0x55f6c647cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:18.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.884+0000 7f4c50384740 4 client.15993 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T08:52:18.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.884+0000 7f4c50384740 1 -- 172.21.15.83:0/4171367233 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f6c6365a60 con 0x55f6c647ab30 2024-04-24T08:52:18.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.885+0000 7f4c46a00640 1 --2- 172.21.15.83:0/4171367233 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55f6c647ab30 0x55f6c647cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:18.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.885+0000 7f4c46a00640 1 --2- 172.21.15.83:0/4171367233 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55f6c647ab30 0x55f6c647cf20 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6787ec947b93fa4 server_cookie=3d3ad7604f396580 in_seq=0 out_seq=0 2024-04-24T08:52:18.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.885+0000 7f4c2be00640 10 client.15993 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:52:18.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.886+0000 7f4c2be00640 1 -- 172.21.15.83:0/4171367233 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f6c6365a60 con 0x55f6c647ab30 2024-04-24T08:52:18.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.886+0000 7f4c2be00640 10 client.15993 handle_command_reply: tid=0 2024-04-24T08:52:18.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.886+0000 7f4c50384740 1 client.15993 shutdown 2024-04-24T08:52:18.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.886+0000 7f4c50384740 1 -- 172.21.15.83:0/4171367233 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6c63f9460 msgr2=0x55f6c640c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:18.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.886+0000 7f4c50384740 1 --2- 172.21.15.83:0/4171367233 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6c63f9460 0x55f6c640c2a0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x55f6c63f5ff0 tx=0x7f4c30003f20 comp rx=0 tx=0).stop 2024-04-24T08:52:18.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.887+0000 7f4c50384740 1 -- 172.21.15.83:0/4171367233 shutdown_connections 2024-04-24T08:52:18.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.887+0000 7f4c50384740 1 --2- 172.21.15.83:0/4171367233 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55f6c647ab30 0x55f6c647cf20 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.887+0000 7f4c50384740 1 --2- 172.21.15.83:0/4171367233 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6c6403b30 0x55f6c6404e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.887+0000 7f4c50384740 1 --2- 172.21.15.83:0/4171367233 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6c63f9460 0x55f6c640c2a0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.887+0000 7f4c50384740 1 --2- 172.21.15.83:0/4171367233 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f6c63f8b40 0x55f6c640bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.887+0000 7f4c2be00640 0 client.15993 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:52:18.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.887+0000 7f4c50384740 1 -- 172.21.15.83:0/4171367233 >> 172.21.15.83:0/4171367233 conn(0x55f6c63f4e60 msgr2=0x55f6c63f36c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:18.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.887+0000 7f4c50384740 1 -- 172.21.15.83:0/4171367233 shutdown_connections 2024-04-24T08:52:18.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.887+0000 7f4c50384740 1 -- 172.21.15.83:0/4171367233 wait complete. 2024-04-24T08:52:18.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.887+0000 7f4c50384740 20 client.15993 trim_cache size 0 max 16384 2024-04-24T08:52:18.890 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T08:52:18.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.887+0000 7f4c47e00640 1 -- 172.21.15.83:0/2226932848 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f4c2c0a3ec0 msgr2=0x7f4c2c0a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:18.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.887+0000 7f4c47e00640 1 --2- 172.21.15.83:0/2226932848 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f4c2c0a3ec0 0x7f4c2c0a6380 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x55f6c63fa100 tx=0x7f4c380097c0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.887+0000 7f4c47e00640 1 -- 172.21.15.83:0/2226932848 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c4819a6b0 msgr2=0x7f4c4819ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:18.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.887+0000 7f4c47e00640 1 --2- 172.21.15.83:0/2226932848 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c4819a6b0 0x7f4c4819ab30 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f4c4000c970 tx=0x7f4c4000ce30 comp rx=0 tx=0).stop 2024-04-24T08:52:18.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.888+0000 7f4c47e00640 1 -- 172.21.15.83:0/2226932848 shutdown_connections 2024-04-24T08:52:18.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.888+0000 7f4c47e00640 1 --2- 172.21.15.83:0/2226932848 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f4c2c0a3ec0 0x7f4c2c0a6380 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.888+0000 7f4c47e00640 1 --2- 172.21.15.83:0/2226932848 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c4819a6b0 0x7f4c4819ab30 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.888+0000 7f4c47e00640 1 --2- 172.21.15.83:0/2226932848 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4c481093e0 0x7f4c481a0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.888+0000 7f4c47e00640 1 --2- 172.21.15.83:0/2226932848 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c48104310 0x7f4c481a05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:18.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.888+0000 7f4c47e00640 1 -- 172.21.15.83:0/2226932848 >> 172.21.15.83:0/2226932848 conn(0x7f4c480ffb70 msgr2=0x7f4c48101ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:18.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.888+0000 7f4c47e00640 1 -- 172.21.15.83:0/2226932848 shutdown_connections 2024-04-24T08:52:18.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:18.888+0000 7f4c47e00640 1 -- 172.21.15.83:0/2226932848 wait complete. 2024-04-24T08:52:18.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:52:18 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/1122855404' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:52:19.906 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T08:52:19.906 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T08:52:19.906 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T08:52:19.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:52:19 smithi138 ceph-mon[31441]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-24T08:52:19.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:52:19 smithi189 ceph-mon[31791]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-24T08:52:19.980 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:52:19 smithi083 ceph-mon[27907]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-24T08:52:20.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.119+0000 7fbaba800640 1 -- 172.21.15.83:0/3468970764 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbab4076e50 msgr2=0x7fbab4112800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:20.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.119+0000 7fbaba800640 1 --2- 172.21.15.83:0/3468970764 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbab4076e50 0x7fbab4112800 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fbaa0009900 tx=0x7fbaa002a400 comp rx=0 tx=0).stop 2024-04-24T08:52:20.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.119+0000 7fbaba800640 1 -- 172.21.15.83:0/3468970764 shutdown_connections 2024-04-24T08:52:20.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.119+0000 7fbaba800640 1 --2- 172.21.15.83:0/3468970764 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbab4112d40 0x7fbab4115130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:20.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.119+0000 7fbaba800640 1 --2- 172.21.15.83:0/3468970764 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbab4076e50 0x7fbab4112800 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:20.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.119+0000 7fbaba800640 1 --2- 172.21.15.83:0/3468970764 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbab4076530 0x7fbab4076910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:20.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.119+0000 7fbaba800640 1 -- 172.21.15.83:0/3468970764 >> 172.21.15.83:0/3468970764 conn(0x7fbab40ffb70 msgr2=0x7fbab4101f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:20.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.119+0000 7fbaba800640 1 -- 172.21.15.83:0/3468970764 shutdown_connections 2024-04-24T08:52:20.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.119+0000 7fbaba800640 1 -- 172.21.15.83:0/3468970764 wait complete. 2024-04-24T08:52:20.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.120+0000 7fbaba800640 1 Processor -- start 2024-04-24T08:52:20.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.120+0000 7fbaba800640 1 -- start start 2024-04-24T08:52:20.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.120+0000 7fbaba800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbab4076530 0x7fbab41a4860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:20.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.120+0000 7fbaba800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbab4076e50 0x7fbab41a4da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:20.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.120+0000 7fbaba800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbab4112d40 0x7fbab419ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:20.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.120+0000 7fbaba800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbab4118790 con 0x7fbab4076530 2024-04-24T08:52:20.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.120+0000 7fbaba800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbab4118610 con 0x7fbab4076e50 2024-04-24T08:52:20.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.120+0000 7fbaba800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fbab4118910 con 0x7fbab4112d40 2024-04-24T08:52:20.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.121+0000 7fbab9400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbab4076530 0x7fbab41a4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:20.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.121+0000 7fbab9400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbab4076530 0x7fbab41a4860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55952/0 (socket says 172.21.15.83:55952) 2024-04-24T08:52:20.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.121+0000 7fbab9400640 1 -- 172.21.15.83:0/3281641309 learned_addr learned my addr 172.21.15.83:0/3281641309 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:52:20.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.121+0000 7fbab8a00640 1 --2- 172.21.15.83:0/3281641309 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbab4076e50 0x7fbab41a4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:20.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.121+0000 7fbab9400640 1 -- 172.21.15.83:0/3281641309 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbab4112d40 msgr2=0x7fbab419ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:20.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.121+0000 7fbab9e00640 1 --2- 172.21.15.83:0/3281641309 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbab4112d40 0x7fbab419ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:20.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.121+0000 7fbab9400640 1 --2- 172.21.15.83:0/3281641309 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbab4112d40 0x7fbab419ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:20.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.121+0000 7fbab9400640 1 -- 172.21.15.83:0/3281641309 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbab4076e50 msgr2=0x7fbab41a4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:20.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.121+0000 7fbab9400640 1 --2- 172.21.15.83:0/3281641309 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbab4076e50 0x7fbab41a4da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:20.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.121+0000 7fbab9400640 1 -- 172.21.15.83:0/3281641309 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbab419f190 con 0x7fbab4076530 2024-04-24T08:52:20.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.121+0000 7fbab8a00640 1 --2- 172.21.15.83:0/3281641309 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbab4076e50 0x7fbab41a4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:52:20.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.122+0000 7fbab9400640 1 --2- 172.21.15.83:0/3281641309 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbab4076530 0x7fbab41a4860 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fbaa800c920 tx=0x7fbaa800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:20.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.122+0000 7fbab2000640 1 -- 172.21.15.83:0/3281641309 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaa8007c70 con 0x7fbab4076530 2024-04-24T08:52:20.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.122+0000 7fbaba800640 1 -- 172.21.15.83:0/3281641309 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbab419f480 con 0x7fbab4076530 2024-04-24T08:52:20.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.122+0000 7fbab2000640 1 -- 172.21.15.83:0/3281641309 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbaa8007e10 con 0x7fbab4076530 2024-04-24T08:52:20.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.122+0000 7fbab2000640 1 -- 172.21.15.83:0/3281641309 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaa8018820 con 0x7fbab4076530 2024-04-24T08:52:20.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.122+0000 7fbaba800640 1 -- 172.21.15.83:0/3281641309 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbab41ab720 con 0x7fbab4076530 2024-04-24T08:52:20.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.125+0000 7fbabed8a740 1 Processor -- start 2024-04-24T08:52:20.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.125+0000 7fbabed8a740 1 -- start start 2024-04-24T08:52:20.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.126+0000 7fbabed8a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d20876270 0x558d20876650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:20.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.126+0000 7fbabed8a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558d20876b90 0x558d20880d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:20.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.126+0000 7fbabed8a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d20881260 0x558d20883650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:20.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.126+0000 7fbabed8a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x558d207dbe70 con 0x558d20881260 2024-04-24T08:52:20.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.126+0000 7fbabed8a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558d20782380 con 0x558d20876270 2024-04-24T08:52:20.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.126+0000 7fbabed8a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x558d207d7e30 con 0x558d20876b90 2024-04-24T08:52:20.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.126+0000 7fbab2000640 1 -- 172.21.15.83:0/3281641309 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7fbaa80189c0 con 0x7fbab4076530 2024-04-24T08:52:20.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.126+0000 7fbab8a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d20881260 0x558d20883650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:20.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.126+0000 7fbab8a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d20881260 0x558d20883650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55956/0 (socket says 172.21.15.83:55956) 2024-04-24T08:52:20.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.126+0000 7fbab8a00640 1 -- 172.21.15.83:0/874221385 learned_addr learned my addr 172.21.15.83:0/874221385 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:52:20.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.126+0000 7fbab9e00640 1 --2- 172.21.15.83:0/874221385 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d20876270 0x558d20876650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:20.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.126+0000 7fbab8a00640 1 -- 172.21.15.83:0/874221385 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558d20876b90 msgr2=0x558d20880d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:20.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.126+0000 7fbab9400640 1 --2- 172.21.15.83:0/874221385 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558d20876b90 0x558d20880d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:20.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.126+0000 7fbab8a00640 1 --2- 172.21.15.83:0/874221385 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558d20876b90 0x558d20880d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:20.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.126+0000 7fbab8a00640 1 -- 172.21.15.83:0/874221385 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d20876270 msgr2=0x558d20876650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:20.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.126+0000 7fbab8a00640 1 --2- 172.21.15.83:0/874221385 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d20876270 0x558d20876650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:20.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.126+0000 7fbab8a00640 1 -- 172.21.15.83:0/874221385 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d205b67e0 con 0x558d20881260 2024-04-24T08:52:20.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.127+0000 7fbab9400640 1 --2- 172.21.15.83:0/874221385 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558d20876b90 0x558d20880d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:52:20.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.127+0000 7fbab8a00640 1 --2- 172.21.15.83:0/874221385 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d20881260 0x558d20883650 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fbaa0006010 tx=0x7fbaa0005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4514a186869fd1d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:20.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.127+0000 7fbaa6600640 1 -- 172.21.15.83:0/874221385 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaa003f070 con 0x558d20881260 2024-04-24T08:52:20.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.127+0000 7fbaa6600640 1 -- 172.21.15.83:0/874221385 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbaa00046c0 con 0x558d20881260 2024-04-24T08:52:20.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.127+0000 7fbaa6600640 1 -- 172.21.15.83:0/874221385 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaa0035030 con 0x558d20881260 2024-04-24T08:52:20.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.127+0000 7fbab2000640 1 --2- 172.21.15.83:0/3281641309 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fba9c0a3e40 0x7fba9c0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:20.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.127+0000 7fbab2000640 1 -- 172.21.15.83:0/3281641309 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7fbaa80d10e0 con 0x7fbab4076530 2024-04-24T08:52:20.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.128+0000 7fbab9e00640 1 --2- 172.21.15.83:0/3281641309 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fba9c0a3e40 0x7fba9c0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:20.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.128+0000 7fbabed8a740 1 -- 172.21.15.83:0/874221385 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d20881260 msgr2=0x558d20883650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:20.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.128+0000 7fbabed8a740 1 --2- 172.21.15.83:0/874221385 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d20881260 0x558d20883650 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fbaa0006010 tx=0x7fbaa0005e80 comp rx=0 tx=0).stop 2024-04-24T08:52:20.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.128+0000 7fbabed8a740 1 -- 172.21.15.83:0/874221385 shutdown_connections 2024-04-24T08:52:20.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.128+0000 7fbabed8a740 1 --2- 172.21.15.83:0/874221385 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d20881260 0x558d20883650 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:20.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.128+0000 7fbabed8a740 1 --2- 172.21.15.83:0/874221385 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558d20876b90 0x558d20880d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:20.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.128+0000 7fbabed8a740 1 --2- 172.21.15.83:0/874221385 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d20876270 0x558d20876650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:20.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.128+0000 7fbabed8a740 1 -- 172.21.15.83:0/874221385 >> 172.21.15.83:0/874221385 conn(0x558d20872590 msgr2=0x558d20872970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:20.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.128+0000 7fbabed8a740 1 -- 172.21.15.83:0/874221385 shutdown_connections 2024-04-24T08:52:20.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.128+0000 7fbab9e00640 1 --2- 172.21.15.83:0/3281641309 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fba9c0a3e40 0x7fba9c0a6300 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fbaac005cc0 tx=0x7fbaac00b340 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:20.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.128+0000 7fbabed8a740 1 -- 172.21.15.83:0/874221385 wait complete. 2024-04-24T08:52:20.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.129+0000 7fbabed8a740 1 -- start start 2024-04-24T08:52:20.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.129+0000 7fbabed8a740 1 Processor -- start 2024-04-24T08:52:20.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.130+0000 7fbabed8a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d20876270 0x558d2087e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:20.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.130+0000 7fbabed8a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d20876b90 0x558d2087e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:20.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.130+0000 7fbabed8a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558d20881260 0x558d2087ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:20.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.130+0000 7fbabed8a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x558d20883ff0 con 0x558d20876270 2024-04-24T08:52:20.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.130+0000 7fbabed8a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558d20795660 con 0x558d20876b90 2024-04-24T08:52:20.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.130+0000 7fbabed8a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x558d20884170 con 0x558d20881260 2024-04-24T08:52:20.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.130+0000 7fbabed8a740 10 client.0 _ll_register_callbacks cb 0x558d20783aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:52:20.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.130+0000 7fbab9400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d20876270 0x558d2087e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:20.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.130+0000 7fbab9400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d20876270 0x558d2087e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55958/0 (socket says 172.21.15.83:55958) 2024-04-24T08:52:20.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.130+0000 7fbab9400640 1 -- 172.21.15.83:0/2085323971 learned_addr learned my addr 172.21.15.83:0/2085323971 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:52:20.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.130+0000 7fbab8a00640 1 --2- 172.21.15.83:0/2085323971 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d20876b90 0x558d2087e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:20.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.130+0000 7fbab9400640 1 -- 172.21.15.83:0/2085323971 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558d20881260 msgr2=0x558d2087ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:20.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.130+0000 7fbab9400640 1 --2- 172.21.15.83:0/2085323971 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558d20881260 0x558d2087ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:20.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.130+0000 7fbab9400640 1 -- 172.21.15.83:0/2085323971 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d20876b90 msgr2=0x558d2087e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:20.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.130+0000 7fbab9400640 1 --2- 172.21.15.83:0/2085323971 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d20876b90 0x558d2087e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:20.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.130+0000 7fbab9400640 1 -- 172.21.15.83:0/2085323971 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d20616f70 con 0x558d20876270 2024-04-24T08:52:20.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.131+0000 7fbab9400640 1 --2- 172.21.15.83:0/2085323971 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d20876270 0x558d2087e160 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fbaa809c700 tx=0x7fbaa809c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a6a08fbe540f02f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:20.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.131+0000 7fba9aa00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:52:20.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.131+0000 7fba9aa00640 1 -- 172.21.15.83:0/2085323971 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaa800de20 con 0x558d20876270 2024-04-24T08:52:20.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.131+0000 7fba9aa00640 1 -- 172.21.15.83:0/2085323971 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7fbaa80bbbc0 con 0x558d20876270 2024-04-24T08:52:20.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.131+0000 7fbabed8a740 1 -- 172.21.15.83:0/2085323971 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbaa00046c0 con 0x558d20876270 2024-04-24T08:52:20.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.131+0000 7fba9aa00640 1 -- 172.21.15.83:0/2085323971 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbaa809d100 con 0x558d20876270 2024-04-24T08:52:20.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.131+0000 7fbabed8a740 10 client.16005 fetch_fsmap learned FSMap version 28 2024-04-24T08:52:20.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.131+0000 7fbabed8a740 1 -- 172.21.15.83:0/2085323971 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558d204ae170 con 0x558d20876270 2024-04-24T08:52:20.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.131+0000 7fba9aa00640 1 -- 172.21.15.83:0/2085323971 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbaa00046c0 con 0x558d20876270 2024-04-24T08:52:20.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.132+0000 7fba9aa00640 1 -- 172.21.15.83:0/2085323971 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fbaa80d3100 con 0x558d20876270 2024-04-24T08:52:20.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.132+0000 7fbabed8a740 10 client.16005 fetch_fsmap finished waiting for FSMap version 28 2024-04-24T08:52:20.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.132+0000 7fbabed8a740 10 client.16005 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:52:20.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.132+0000 7fbabed8a740 20 client.16005 populate_metadata read hostname 'smithi083' 2024-04-24T08:52:20.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.132+0000 7fbabed8a740 1 --2- 172.21.15.83:0/2085323971 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x558d208914a0 0x558d20893960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:20.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.132+0000 7fbabed8a740 4 client.16005 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:52:20.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.132+0000 7fbabed8a740 1 -- 172.21.15.83:0/2085323971 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558d207e9890 con 0x558d208914a0 2024-04-24T08:52:20.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.132+0000 7fbab8a00640 1 --2- 172.21.15.83:0/2085323971 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x558d208914a0 0x558d20893960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:20.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.133+0000 7fbab8a00640 1 --2- 172.21.15.83:0/2085323971 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x558d208914a0 0x558d20893960 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47cabca07f21f822 server_cookie=a48c91f87f57114f in_seq=0 out_seq=0 2024-04-24T08:52:20.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.133+0000 7fba9aa00640 10 client.16005 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:52:20.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.135+0000 7fba9aa00640 1 -- 172.21.15.83:0/2085323971 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558d207e9890 con 0x558d208914a0 2024-04-24T08:52:20.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.135+0000 7fba9aa00640 10 client.16005 handle_command_reply: tid=0 2024-04-24T08:52:20.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.136+0000 7fbabed8a740 1 client.16005 shutdown 2024-04-24T08:52:20.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.136+0000 7fbabed8a740 1 -- 172.21.15.83:0/2085323971 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d20876270 msgr2=0x558d2087e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:20.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.136+0000 7fbabed8a740 1 --2- 172.21.15.83:0/2085323971 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d20876270 0x558d2087e160 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fbaa809c700 tx=0x7fbaa809c420 comp rx=0 tx=0).stop 2024-04-24T08:52:20.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.136+0000 7fbabed8a740 1 -- 172.21.15.83:0/2085323971 shutdown_connections 2024-04-24T08:52:20.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.136+0000 7fbabed8a740 1 --2- 172.21.15.83:0/2085323971 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x558d208914a0 0x558d20893960 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:20.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.136+0000 7fbabed8a740 1 --2- 172.21.15.83:0/2085323971 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558d20881260 0x558d2087ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:20.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.136+0000 7fbabed8a740 1 --2- 172.21.15.83:0/2085323971 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d20876b90 0x558d2087e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:20.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.136+0000 7fbabed8a740 1 --2- 172.21.15.83:0/2085323971 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d20876270 0x558d2087e160 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:20.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.136+0000 7fba9aa00640 0 client.16005 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:52:20.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.136+0000 7fbabed8a740 1 -- 172.21.15.83:0/2085323971 >> 172.21.15.83:0/2085323971 conn(0x558d20872590 msgr2=0x558d20878390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:20.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.136+0000 7fbabed8a740 1 -- 172.21.15.83:0/2085323971 shutdown_connections 2024-04-24T08:52:20.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.136+0000 7fbabed8a740 1 -- 172.21.15.83:0/2085323971 wait complete. 2024-04-24T08:52:20.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.136+0000 7fbabed8a740 20 client.16005 trim_cache size 0 max 16384 2024-04-24T08:52:20.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.155+0000 7fbabed8a740 1 Processor -- start 2024-04-24T08:52:20.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.155+0000 7fbabed8a740 1 -- start start 2024-04-24T08:52:20.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.156+0000 7fbabed8a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d20876270 0x558d2088ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:20.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.156+0000 7fbabed8a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d20876b90 0x558d208789d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:20.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.156+0000 7fbabed8a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558d20881260 0x558d20899280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:20.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.156+0000 7fbabed8a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbab41b5810 con 0x558d20876b90 2024-04-24T08:52:20.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.156+0000 7fbabed8a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbab41b7350 con 0x558d20876270 2024-04-24T08:52:20.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.156+0000 7fbabed8a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fbab41b5ee0 con 0x558d20881260 2024-04-24T08:52:20.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.156+0000 7fbab8a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d20876b90 0x558d208789d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:20.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.156+0000 7fbab8a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d20876b90 0x558d208789d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55968/0 (socket says 172.21.15.83:55968) 2024-04-24T08:52:20.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.156+0000 7fbab8a00640 1 -- 172.21.15.83:0/1549764130 learned_addr learned my addr 172.21.15.83:0/1549764130 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:52:20.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.156+0000 7fbab8a00640 1 -- 172.21.15.83:0/1549764130 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558d20881260 msgr2=0x558d20899280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:20.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.156+0000 7fbab8a00640 1 --2- 172.21.15.83:0/1549764130 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558d20881260 0x558d20899280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:20.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.156+0000 7fbab8a00640 1 -- 172.21.15.83:0/1549764130 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d20876270 msgr2=0x558d2088ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:20.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.156+0000 7fbab9400640 1 --2- 172.21.15.83:0/1549764130 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d20876270 0x558d2088ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:20.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.157+0000 7fbab8a00640 1 --2- 172.21.15.83:0/1549764130 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d20876270 0x558d2088ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:20.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.157+0000 7fbab8a00640 1 -- 172.21.15.83:0/1549764130 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d2087d110 con 0x558d20876b90 2024-04-24T08:52:20.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.157+0000 7fbab8a00640 1 --2- 172.21.15.83:0/1549764130 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d20876b90 0x558d208789d0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fbaa0000c00 tx=0x7fbaa0005de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c8cf16a2049104a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:20.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.157+0000 7fbaa5c00640 1 -- 172.21.15.83:0/1549764130 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaa0035040 con 0x558d20876b90 2024-04-24T08:52:20.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.157+0000 7fbaa5c00640 1 -- 172.21.15.83:0/1549764130 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbaa00046c0 con 0x558d20876b90 2024-04-24T08:52:20.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.157+0000 7fbaa5c00640 1 -- 172.21.15.83:0/1549764130 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaa003f040 con 0x558d20876b90 2024-04-24T08:52:20.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.158+0000 7fbabed8a740 1 -- 172.21.15.83:0/1549764130 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d20876b90 msgr2=0x558d208789d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:20.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.158+0000 7fbabed8a740 1 --2- 172.21.15.83:0/1549764130 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d20876b90 0x558d208789d0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fbaa0000c00 tx=0x7fbaa0005de0 comp rx=0 tx=0).stop 2024-04-24T08:52:20.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.158+0000 7fbabed8a740 1 -- 172.21.15.83:0/1549764130 shutdown_connections 2024-04-24T08:52:20.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.158+0000 7fbabed8a740 1 --2- 172.21.15.83:0/1549764130 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558d20881260 0x558d20899280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:20.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.158+0000 7fbabed8a740 1 --2- 172.21.15.83:0/1549764130 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d20876b90 0x558d208789d0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:20.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.158+0000 7fbabed8a740 1 --2- 172.21.15.83:0/1549764130 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d20876270 0x558d2088ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:20.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.158+0000 7fbabed8a740 1 -- 172.21.15.83:0/1549764130 >> 172.21.15.83:0/1549764130 conn(0x558d20872590 msgr2=0x558d208716f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:20.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.158+0000 7fbabed8a740 1 -- 172.21.15.83:0/1549764130 shutdown_connections 2024-04-24T08:52:20.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.158+0000 7fbabed8a740 1 -- 172.21.15.83:0/1549764130 wait complete. 2024-04-24T08:52:20.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.158+0000 7fbabed8a740 1 -- start start 2024-04-24T08:52:20.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.159+0000 7fbabed8a740 1 Processor -- start 2024-04-24T08:52:20.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.159+0000 7fbabed8a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558d20876270 0x558d208890e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:20.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.159+0000 7fbabed8a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d20876b90 0x558d20889620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:20.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.159+0000 7fbabed8a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d20881260 0x558d20885600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:20.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.159+0000 7fbabed8a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbab41b6c80 con 0x558d20876b90 2024-04-24T08:52:20.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.159+0000 7fbabed8a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbab41b65b0 con 0x558d20881260 2024-04-24T08:52:20.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.159+0000 7fbabed8a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fbab41b37f0 con 0x558d20876270 2024-04-24T08:52:20.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.159+0000 7fbabed8a740 10 client.0 _ll_register_callbacks cb 0x7fbaa809c540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:52:20.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.159+0000 7fbab8a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d20876b90 0x558d20889620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:20.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.159+0000 7fbab8a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d20876b90 0x558d20889620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55974/0 (socket says 172.21.15.83:55974) 2024-04-24T08:52:20.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.159+0000 7fbab8a00640 1 -- 172.21.15.83:0/2713871116 learned_addr learned my addr 172.21.15.83:0/2713871116 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:52:20.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.160+0000 7fbab9400640 1 --2- 172.21.15.83:0/2713871116 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558d20876270 0x558d208890e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:20.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.160+0000 7fbab9e00640 1 --2- 172.21.15.83:0/2713871116 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d20881260 0x558d20885600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:20.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.160+0000 7fbab8a00640 1 -- 172.21.15.83:0/2713871116 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558d20876270 msgr2=0x558d208890e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:20.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.160+0000 7fbab8a00640 1 --2- 172.21.15.83:0/2713871116 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558d20876270 0x558d208890e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:20.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.160+0000 7fbab8a00640 1 -- 172.21.15.83:0/2713871116 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d20881260 msgr2=0x558d20885600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:20.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.160+0000 7fbab8a00640 1 --2- 172.21.15.83:0/2713871116 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d20881260 0x558d20885600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:20.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.160+0000 7fbab8a00640 1 -- 172.21.15.83:0/2713871116 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d20885de0 con 0x558d20876b90 2024-04-24T08:52:20.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.160+0000 7fbab9400640 1 --2- 172.21.15.83:0/2713871116 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558d20876270 0x558d208890e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:52:20.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.160+0000 7fbab8a00640 1 --2- 172.21.15.83:0/2713871116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d20876b90 0x558d20889620 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fbaa0009600 tx=0x7fbaa0004570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c386ef6f918a9dc8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:20.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.160+0000 7fba9aa00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:52:20.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.160+0000 7fba9aa00640 1 -- 172.21.15.83:0/2713871116 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaa003f050 con 0x558d20876b90 2024-04-24T08:52:20.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.160+0000 7fbabed8a740 1 -- 172.21.15.83:0/2713871116 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbaa00046c0 con 0x558d20876b90 2024-04-24T08:52:20.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.160+0000 7fba9aa00640 1 -- 172.21.15.83:0/2713871116 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7fbaa003ba80 con 0x558d20876b90 2024-04-24T08:52:20.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.161+0000 7fba9aa00640 1 -- 172.21.15.83:0/2713871116 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbaa0035040 con 0x558d20876b90 2024-04-24T08:52:20.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.161+0000 7fbabed8a740 10 client.16011 fetch_fsmap learned FSMap version 28 2024-04-24T08:52:20.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.161+0000 7fbabed8a740 1 -- 172.21.15.83:0/2713871116 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558d208f85c0 con 0x558d20876b90 2024-04-24T08:52:20.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.161+0000 7fba9aa00640 1 -- 172.21.15.83:0/2713871116 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbaa00046c0 con 0x558d20876b90 2024-04-24T08:52:20.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.161+0000 7fba9aa00640 1 -- 172.21.15.83:0/2713871116 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fbaa004d070 con 0x558d20876b90 2024-04-24T08:52:20.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.161+0000 7fbabed8a740 10 client.16011 fetch_fsmap finished waiting for FSMap version 28 2024-04-24T08:52:20.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.161+0000 7fbabed8a740 10 client.16011 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:52:20.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.161+0000 7fbabed8a740 20 client.16011 populate_metadata read hostname 'smithi083' 2024-04-24T08:52:20.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.161+0000 7fbabed8a740 1 --2- 172.21.15.83:0/2713871116 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x558d208f8890 0x558d208fac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:20.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.161+0000 7fbabed8a740 4 client.16011 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T08:52:20.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.161+0000 7fbabed8a740 1 -- 172.21.15.83:0/2713871116 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x558d207e3700 con 0x558d208f8890 2024-04-24T08:52:20.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.162+0000 7fbab9400640 1 --2- 172.21.15.83:0/2713871116 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x558d208f8890 0x558d208fac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:20.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.163+0000 7fbab9400640 1 --2- 172.21.15.83:0/2713871116 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x558d208f8890 0x558d208fac80 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53cf8bbaee80719f server_cookie=f51271abc783ad3d in_seq=0 out_seq=0 2024-04-24T08:52:20.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.163+0000 7fba9aa00640 10 client.16011 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:52:20.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.165+0000 7fba9aa00640 1 -- 172.21.15.83:0/2713871116 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x558d207e3700 con 0x558d208f8890 2024-04-24T08:52:20.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.165+0000 7fba9aa00640 10 client.16011 handle_command_reply: tid=0 2024-04-24T08:52:20.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.165+0000 7fbabed8a740 1 client.16011 shutdown 2024-04-24T08:52:20.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.165+0000 7fbabed8a740 1 -- 172.21.15.83:0/2713871116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d20876b90 msgr2=0x558d20889620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:20.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.165+0000 7fbabed8a740 1 --2- 172.21.15.83:0/2713871116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d20876b90 0x558d20889620 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fbaa0009600 tx=0x7fbaa0004570 comp rx=0 tx=0).stop 2024-04-24T08:52:20.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.165+0000 7fbabed8a740 1 -- 172.21.15.83:0/2713871116 shutdown_connections 2024-04-24T08:52:20.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.165+0000 7fbabed8a740 1 --2- 172.21.15.83:0/2713871116 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x558d208f8890 0x558d208fac80 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:20.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.165+0000 7fbabed8a740 1 --2- 172.21.15.83:0/2713871116 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d20881260 0x558d20885600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:20.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.165+0000 7fbabed8a740 1 --2- 172.21.15.83:0/2713871116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d20876b90 0x558d20889620 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:20.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.165+0000 7fbabed8a740 1 --2- 172.21.15.83:0/2713871116 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558d20876270 0x558d208890e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:20.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.165+0000 7fba9aa00640 0 client.16011 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:52:20.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.165+0000 7fbabed8a740 1 -- 172.21.15.83:0/2713871116 >> 172.21.15.83:0/2713871116 conn(0x558d20872590 msgr2=0x558d20870e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:20.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.166+0000 7fbabed8a740 1 -- 172.21.15.83:0/2713871116 shutdown_connections 2024-04-24T08:52:20.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.166+0000 7fbabed8a740 1 -- 172.21.15.83:0/2713871116 wait complete. 2024-04-24T08:52:20.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.166+0000 7fbabed8a740 20 client.16011 trim_cache size 0 max 16384 2024-04-24T08:52:20.168 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T08:52:20.168 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-04-24T08:52:20.168 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "ab5f12b7-963c-4982-abdb-2faac1314a4a", 2024-04-24T08:52:20.168 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-04-24T08:52:20.168 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T08:52:20.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.166+0000 7fbaba800640 1 -- 172.21.15.83:0/3281641309 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fba9c0a3e40 msgr2=0x7fba9c0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:20.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.166+0000 7fbaba800640 1 --2- 172.21.15.83:0/3281641309 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fba9c0a3e40 0x7fba9c0a6300 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fbaac005cc0 tx=0x7fbaac00b340 comp rx=0 tx=0).stop 2024-04-24T08:52:20.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.166+0000 7fbaba800640 1 -- 172.21.15.83:0/3281641309 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbab4076530 msgr2=0x7fbab41a4860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:20.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.166+0000 7fbaba800640 1 --2- 172.21.15.83:0/3281641309 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbab4076530 0x7fbab41a4860 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fbaa800c920 tx=0x7fbaa800cde0 comp rx=0 tx=0).stop 2024-04-24T08:52:20.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.167+0000 7fbaba800640 1 -- 172.21.15.83:0/3281641309 shutdown_connections 2024-04-24T08:52:20.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.167+0000 7fbaba800640 1 --2- 172.21.15.83:0/3281641309 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fba9c0a3e40 0x7fba9c0a6300 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:20.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.167+0000 7fbaba800640 1 --2- 172.21.15.83:0/3281641309 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbab4112d40 0x7fbab419ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:20.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.167+0000 7fbaba800640 1 --2- 172.21.15.83:0/3281641309 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbab4076e50 0x7fbab41a4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:20.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.167+0000 7fbaba800640 1 --2- 172.21.15.83:0/3281641309 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbab4076530 0x7fbab41a4860 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:20.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.167+0000 7fbaba800640 1 -- 172.21.15.83:0/3281641309 >> 172.21.15.83:0/3281641309 conn(0x7fbab40ffb70 msgr2=0x7fbab4101f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:20.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.167+0000 7fbaba800640 1 -- 172.21.15.83:0/3281641309 shutdown_connections 2024-04-24T08:52:20.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.167+0000 7fbaba800640 1 -- 172.21.15.83:0/3281641309 wait complete. 2024-04-24T08:52:20.185 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T08:52:20.185 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T08:52:20.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.380+0000 7f9875800640 1 -- 172.21.15.83:0/2744158194 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9870107310 msgr2=0x7f987010eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:20.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.380+0000 7f9875800640 1 --2- 172.21.15.83:0/2744158194 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9870107310 0x7f987010eee0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f9868009ca0 tx=0x7f986802a8c0 comp rx=0 tx=0).stop 2024-04-24T08:52:20.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.380+0000 7f9875800640 1 -- 172.21.15.83:0/2744158194 shutdown_connections 2024-04-24T08:52:20.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.380+0000 7f9875800640 1 --2- 172.21.15.83:0/2744158194 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9870107310 0x7f987010eee0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:20.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.380+0000 7f9875800640 1 --2- 172.21.15.83:0/2744158194 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9870102bf0 0x7f9870106c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:20.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.380+0000 7f9875800640 1 --2- 172.21.15.83:0/2744158194 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9870102240 0x7f9870102620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:20.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.380+0000 7f9875800640 1 -- 172.21.15.83:0/2744158194 >> 172.21.15.83:0/2744158194 conn(0x7f9870079450 msgr2=0x7f9870079860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:20.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.380+0000 7f9875800640 1 -- 172.21.15.83:0/2744158194 shutdown_connections 2024-04-24T08:52:20.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.380+0000 7f9875800640 1 -- 172.21.15.83:0/2744158194 wait complete. 2024-04-24T08:52:20.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.381+0000 7f9875800640 1 Processor -- start 2024-04-24T08:52:20.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.381+0000 7f9875800640 1 -- start start 2024-04-24T08:52:20.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.381+0000 7f9875800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9870102240 0x7f9870074300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:20.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.381+0000 7f9875800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9870107310 0x7f9870074840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:20.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.382+0000 7f9875800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9870074d80 0x7f98701a69a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:20.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.382+0000 7f9875800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f98701122d0 con 0x7f9870102240 2024-04-24T08:52:20.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.382+0000 7f9875800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9870112150 con 0x7f9870074d80 2024-04-24T08:52:20.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.382+0000 7f9875800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9870112450 con 0x7f9870107310 2024-04-24T08:52:20.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.382+0000 7f986ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9870102240 0x7f9870074300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:20.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.382+0000 7f986ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9870102240 0x7f9870074300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55990/0 (socket says 172.21.15.83:55990) 2024-04-24T08:52:20.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.382+0000 7f986ea00640 1 -- 172.21.15.83:0/42631744 learned_addr learned my addr 172.21.15.83:0/42631744 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:52:20.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.382+0000 7f986f400640 1 --2- 172.21.15.83:0/42631744 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9870074d80 0x7f98701a69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:20.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.382+0000 7f986e000640 1 --2- 172.21.15.83:0/42631744 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9870107310 0x7f9870074840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:20.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.382+0000 7f986ea00640 1 -- 172.21.15.83:0/42631744 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9870107310 msgr2=0x7f9870074840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:20.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.382+0000 7f986ea00640 1 --2- 172.21.15.83:0/42631744 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9870107310 0x7f9870074840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:20.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.382+0000 7f986ea00640 1 -- 172.21.15.83:0/42631744 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9870074d80 msgr2=0x7f98701a69a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:20.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.382+0000 7f986ea00640 1 --2- 172.21.15.83:0/42631744 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9870074d80 0x7f98701a69a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:20.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.382+0000 7f986ea00640 1 -- 172.21.15.83:0/42631744 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98701a70a0 con 0x7f9870102240 2024-04-24T08:52:20.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.383+0000 7f986ea00640 1 --2- 172.21.15.83:0/42631744 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9870102240 0x7f9870074300 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f986000d950 tx=0x7f986000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:20.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.383+0000 7f9867400640 1 -- 172.21.15.83:0/42631744 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9860017070 con 0x7f9870102240 2024-04-24T08:52:20.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.383+0000 7f9867400640 1 -- 172.21.15.83:0/42631744 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9860009c30 con 0x7f9870102240 2024-04-24T08:52:20.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.383+0000 7f9867400640 1 -- 172.21.15.83:0/42631744 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f986000f620 con 0x7f9870102240 2024-04-24T08:52:20.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.383+0000 7f9875800640 1 -- 172.21.15.83:0/42631744 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98701a7390 con 0x7f9870102240 2024-04-24T08:52:20.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.383+0000 7f9875800640 1 -- 172.21.15.83:0/42631744 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98701a7890 con 0x7f9870102240 2024-04-24T08:52:20.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.387+0000 7f9878584740 1 Processor -- start 2024-04-24T08:52:20.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.387+0000 7f9867400640 1 -- 172.21.15.83:0/42631744 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7f98600109a0 con 0x7f9870102240 2024-04-24T08:52:20.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.387+0000 7f9878584740 1 -- start start 2024-04-24T08:52:20.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.387+0000 7f9878584740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5575581fa860 0x5575581fac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:20.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.387+0000 7f9878584740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5575581fb180 0x557558205310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:20.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.387+0000 7f9878584740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557558205850 0x557558207c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:20.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.387+0000 7f9878584740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5575581601f0 con 0x557558205850 2024-04-24T08:52:20.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.387+0000 7f9878584740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x557558106350 con 0x5575581fa860 2024-04-24T08:52:20.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.387+0000 7f9878584740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55755815c1b0 con 0x5575581fb180 2024-04-24T08:52:20.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.388+0000 7f986e000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557558205850 0x557558207c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:20.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.388+0000 7f986e000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557558205850 0x557558207c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56004/0 (socket says 172.21.15.83:56004) 2024-04-24T08:52:20.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.388+0000 7f986e000640 1 -- 172.21.15.83:0/1991172264 learned_addr learned my addr 172.21.15.83:0/1991172264 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:52:20.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.388+0000 7f986ea00640 1 --2- 172.21.15.83:0/1991172264 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5575581fb180 0x557558205310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:20.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.388+0000 7f986f400640 1 --2- 172.21.15.83:0/1991172264 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5575581fa860 0x5575581fac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:20.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.388+0000 7f986e000640 1 -- 172.21.15.83:0/1991172264 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5575581fb180 msgr2=0x557558205310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:20.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.388+0000 7f986e000640 1 --2- 172.21.15.83:0/1991172264 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5575581fb180 0x557558205310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:20.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.388+0000 7f986e000640 1 -- 172.21.15.83:0/1991172264 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5575581fa860 msgr2=0x5575581fac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:20.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.388+0000 7f986e000640 1 --2- 172.21.15.83:0/1991172264 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5575581fa860 0x5575581fac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:20.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.388+0000 7f986e000640 1 -- 172.21.15.83:0/1991172264 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55755815af50 con 0x557558205850 2024-04-24T08:52:20.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.389+0000 7f986e000640 1 --2- 172.21.15.83:0/1991172264 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557558205850 0x557558207c40 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f985c00bbc0 tx=0x7f985c00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=190fc5f0d0d9e606 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:20.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.389+0000 7f985be00640 1 -- 172.21.15.83:0/1991172264 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f985c015070 con 0x557558205850 2024-04-24T08:52:20.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.389+0000 7f985be00640 1 -- 172.21.15.83:0/1991172264 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f985c0054d0 con 0x557558205850 2024-04-24T08:52:20.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.389+0000 7f985be00640 1 -- 172.21.15.83:0/1991172264 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f985c011440 con 0x557558205850 2024-04-24T08:52:20.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.389+0000 7f9867400640 1 --2- 172.21.15.83:0/42631744 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f98540a3e70 0x7f98540a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:20.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.389+0000 7f9867400640 1 -- 172.21.15.83:0/42631744 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f98600d0b40 con 0x7f9870102240 2024-04-24T08:52:20.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.389+0000 7f986f400640 1 --2- 172.21.15.83:0/42631744 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f98540a3e70 0x7f98540a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:20.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.390+0000 7f9878584740 1 -- 172.21.15.83:0/1991172264 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557558205850 msgr2=0x557558207c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:20.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.390+0000 7f9878584740 1 --2- 172.21.15.83:0/1991172264 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557558205850 0x557558207c40 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f985c00bbc0 tx=0x7f985c00f730 comp rx=0 tx=0).stop 2024-04-24T08:52:20.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.390+0000 7f9878584740 1 -- 172.21.15.83:0/1991172264 shutdown_connections 2024-04-24T08:52:20.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.390+0000 7f9878584740 1 --2- 172.21.15.83:0/1991172264 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557558205850 0x557558207c40 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:20.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.390+0000 7f9878584740 1 --2- 172.21.15.83:0/1991172264 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5575581fb180 0x557558205310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:20.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.390+0000 7f9878584740 1 --2- 172.21.15.83:0/1991172264 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5575581fa860 0x5575581fac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:20.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.390+0000 7f9878584740 1 -- 172.21.15.83:0/1991172264 >> 172.21.15.83:0/1991172264 conn(0x5575581f6b80 msgr2=0x5575581f6f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:20.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.390+0000 7f9878584740 1 -- 172.21.15.83:0/1991172264 shutdown_connections 2024-04-24T08:52:20.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.390+0000 7f986f400640 1 --2- 172.21.15.83:0/42631744 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f98540a3e70 0x7f98540a6330 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9868009430 tx=0x7f986802f310 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:20.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.390+0000 7f9878584740 1 -- 172.21.15.83:0/1991172264 wait complete. 2024-04-24T08:52:20.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.390+0000 7f9878584740 1 -- start start 2024-04-24T08:52:20.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.391+0000 7f9878584740 1 Processor -- start 2024-04-24T08:52:20.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.391+0000 7f9878584740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5575581fa860 0x5575582027b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:20.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.391+0000 7f9878584740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5575581fb180 0x557558202cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:20.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.391+0000 7f9878584740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557558205850 0x557558203230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:20.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.391+0000 7f9878584740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x557558208640 con 0x557558205850 2024-04-24T08:52:20.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.391+0000 7f9878584740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x557558119690 con 0x5575581fa860 2024-04-24T08:52:20.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.391+0000 7f9878584740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5575582087c0 con 0x5575581fb180 2024-04-24T08:52:20.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.391+0000 7f9878584740 10 client.0 _ll_register_callbacks cb 0x557558155650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:52:20.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.391+0000 7f986f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557558205850 0x557558203230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:20.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.392+0000 7f986f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557558205850 0x557558203230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56018/0 (socket says 172.21.15.83:56018) 2024-04-24T08:52:20.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.392+0000 7f986f400640 1 -- 172.21.15.83:0/3332683800 learned_addr learned my addr 172.21.15.83:0/3332683800 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:52:20.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.392+0000 7f986e000640 1 --2- 172.21.15.83:0/3332683800 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5575581fb180 0x557558202cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:20.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.392+0000 7f986ea00640 1 --2- 172.21.15.83:0/3332683800 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5575581fa860 0x5575582027b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:20.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.392+0000 7f986f400640 1 -- 172.21.15.83:0/3332683800 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5575581fb180 msgr2=0x557558202cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:20.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.392+0000 7f986f400640 1 --2- 172.21.15.83:0/3332683800 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5575581fb180 0x557558202cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:20.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.392+0000 7f986f400640 1 -- 172.21.15.83:0/3332683800 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5575581fa860 msgr2=0x5575582027b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:20.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.392+0000 7f986f400640 1 --2- 172.21.15.83:0/3332683800 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5575581fa860 0x5575582027b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:20.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.392+0000 7f986f400640 1 -- 172.21.15.83:0/3332683800 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557558118920 con 0x557558205850 2024-04-24T08:52:20.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.392+0000 7f986ea00640 1 --2- 172.21.15.83:0/3332683800 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5575581fa860 0x5575582027b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:52:20.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.392+0000 7f986f400640 1 --2- 172.21.15.83:0/3332683800 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557558205850 0x557558203230 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f9868035000 tx=0x7f98680095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f60224103522fbe server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:20.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.392+0000 7f9858c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:52:20.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.393+0000 7f9858c00640 1 -- 172.21.15.83:0/3332683800 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f986804f070 con 0x557558205850 2024-04-24T08:52:20.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.393+0000 7f9858c00640 1 -- 172.21.15.83:0/3332683800 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f98680041b0 con 0x557558205850 2024-04-24T08:52:20.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.393+0000 7f9878584740 1 -- 172.21.15.83:0/3332683800 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f985c0054d0 con 0x557558205850 2024-04-24T08:52:20.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.393+0000 7f9858c00640 1 -- 172.21.15.83:0/3332683800 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f98680043b0 con 0x557558205850 2024-04-24T08:52:20.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.393+0000 7f9878584740 10 client.16023 fetch_fsmap learned FSMap version 28 2024-04-24T08:52:20.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.393+0000 7f9878584740 1 -- 172.21.15.83:0/3332683800 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557557f3c7d0 con 0x557558205850 2024-04-24T08:52:20.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.393+0000 7f9858c00640 1 -- 172.21.15.83:0/3332683800 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f985c0054d0 con 0x557558205850 2024-04-24T08:52:20.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.394+0000 7f9858c00640 1 -- 172.21.15.83:0/3332683800 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f9868044040 con 0x557558205850 2024-04-24T08:52:20.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.394+0000 7f9878584740 10 client.16023 fetch_fsmap finished waiting for FSMap version 28 2024-04-24T08:52:20.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.394+0000 7f9878584740 10 client.16023 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:52:20.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.394+0000 7f9878584740 20 client.16023 populate_metadata read hostname 'smithi083' 2024-04-24T08:52:20.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.394+0000 7f9878584740 1 --2- 172.21.15.83:0/3332683800 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x557558215ae0 0x557558217f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:20.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.394+0000 7f9878584740 4 client.16023 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:52:20.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.394+0000 7f9878584740 1 -- 172.21.15.83:0/3332683800 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55755816dc10 con 0x557558215ae0 2024-04-24T08:52:20.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.394+0000 7f986ea00640 1 --2- 172.21.15.83:0/3332683800 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x557558215ae0 0x557558217f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:20.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.395+0000 7f986ea00640 1 --2- 172.21.15.83:0/3332683800 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x557558215ae0 0x557558217f80 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6e50db0743b057f server_cookie=f015a203782c67ea in_seq=0 out_seq=0 2024-04-24T08:52:20.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.395+0000 7f9858c00640 10 client.16023 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:52:20.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.397+0000 7f9858c00640 1 -- 172.21.15.83:0/3332683800 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55755816dc10 con 0x557558215ae0 2024-04-24T08:52:20.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.397+0000 7f9858c00640 10 client.16023 handle_command_reply: tid=0 2024-04-24T08:52:20.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.397+0000 7f9878584740 1 client.16023 shutdown 2024-04-24T08:52:20.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.397+0000 7f9878584740 1 -- 172.21.15.83:0/3332683800 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557558205850 msgr2=0x557558203230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:20.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.397+0000 7f9878584740 1 --2- 172.21.15.83:0/3332683800 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557558205850 0x557558203230 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f9868035000 tx=0x7f98680095c0 comp rx=0 tx=0).stop 2024-04-24T08:52:20.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.397+0000 7f9878584740 1 -- 172.21.15.83:0/3332683800 shutdown_connections 2024-04-24T08:52:20.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.397+0000 7f9878584740 1 --2- 172.21.15.83:0/3332683800 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x557558215ae0 0x557558217f80 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:20.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.397+0000 7f9878584740 1 --2- 172.21.15.83:0/3332683800 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557558205850 0x557558203230 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:20.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.397+0000 7f9878584740 1 --2- 172.21.15.83:0/3332683800 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5575581fb180 0x557558202cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:20.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.397+0000 7f9858c00640 0 client.16023 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:52:20.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.397+0000 7f9878584740 1 --2- 172.21.15.83:0/3332683800 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5575581fa860 0x5575582027b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:20.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.397+0000 7f9878584740 1 -- 172.21.15.83:0/3332683800 >> 172.21.15.83:0/3332683800 conn(0x5575581f6b80 msgr2=0x5575581fc980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:20.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.397+0000 7f9878584740 1 -- 172.21.15.83:0/3332683800 shutdown_connections 2024-04-24T08:52:20.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.397+0000 7f9878584740 1 -- 172.21.15.83:0/3332683800 wait complete. 2024-04-24T08:52:20.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.398+0000 7f9878584740 20 client.16023 trim_cache size 0 max 16384 2024-04-24T08:52:20.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.417+0000 7f9878584740 1 Processor -- start 2024-04-24T08:52:20.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.417+0000 7f9878584740 1 -- start start 2024-04-24T08:52:20.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.417+0000 7f9878584740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5575581fa860 0x5575582016f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:20.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.417+0000 7f9878584740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5575581fb180 0x557558211500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:20.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.417+0000 7f9878584740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557558205850 0x5575581fd0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:20.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.417+0000 7f9878584740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f98701b1720 con 0x557558205850 2024-04-24T08:52:20.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.417+0000 7f9878584740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f98701b3260 con 0x5575581fb180 2024-04-24T08:52:20.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.417+0000 7f9878584740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f98701b1df0 con 0x5575581fa860 2024-04-24T08:52:20.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.418+0000 7f986f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557558205850 0x5575581fd0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:20.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.418+0000 7f986f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557558205850 0x5575581fd0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56034/0 (socket says 172.21.15.83:56034) 2024-04-24T08:52:20.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.418+0000 7f986f400640 1 -- 172.21.15.83:0/315802016 learned_addr learned my addr 172.21.15.83:0/315802016 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:52:20.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.418+0000 7f986ea00640 1 --2- 172.21.15.83:0/315802016 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5575581fa860 0x5575582016f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:20.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.418+0000 7f986e000640 1 --2- 172.21.15.83:0/315802016 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5575581fb180 0x557558211500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:20.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.418+0000 7f986f400640 1 -- 172.21.15.83:0/315802016 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5575581fa860 msgr2=0x5575582016f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:20.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.418+0000 7f986f400640 1 --2- 172.21.15.83:0/315802016 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5575581fa860 0x5575582016f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:20.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.418+0000 7f986f400640 1 -- 172.21.15.83:0/315802016 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5575581fb180 msgr2=0x557558211500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:20.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.418+0000 7f986f400640 1 --2- 172.21.15.83:0/315802016 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5575581fb180 0x557558211500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:20.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.418+0000 7f986f400640 1 -- 172.21.15.83:0/315802016 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557557e723f0 con 0x557558205850 2024-04-24T08:52:20.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.419+0000 7f986f400640 1 --2- 172.21.15.83:0/315802016 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557558205850 0x5575581fd0d0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f9868030ef0 tx=0x7f986802afc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b6d1468befd54a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:20.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.419+0000 7f985b400640 1 -- 172.21.15.83:0/315802016 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f986804f070 con 0x557558205850 2024-04-24T08:52:20.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.419+0000 7f985b400640 1 -- 172.21.15.83:0/315802016 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9868036040 con 0x557558205850 2024-04-24T08:52:20.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.419+0000 7f985b400640 1 -- 172.21.15.83:0/315802016 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98680309c0 con 0x557558205850 2024-04-24T08:52:20.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.420+0000 7f9878584740 1 -- 172.21.15.83:0/315802016 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557558205850 msgr2=0x5575581fd0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:20.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.420+0000 7f9878584740 1 --2- 172.21.15.83:0/315802016 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557558205850 0x5575581fd0d0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f9868030ef0 tx=0x7f986802afc0 comp rx=0 tx=0).stop 2024-04-24T08:52:20.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.420+0000 7f9878584740 1 -- 172.21.15.83:0/315802016 shutdown_connections 2024-04-24T08:52:20.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.420+0000 7f9878584740 1 --2- 172.21.15.83:0/315802016 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557558205850 0x5575581fd0d0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:20.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.420+0000 7f9878584740 1 --2- 172.21.15.83:0/315802016 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5575581fb180 0x557558211500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:20.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.420+0000 7f9878584740 1 --2- 172.21.15.83:0/315802016 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5575581fa860 0x5575582016f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:20.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.420+0000 7f9878584740 1 -- 172.21.15.83:0/315802016 >> 172.21.15.83:0/315802016 conn(0x5575581f6b80 msgr2=0x5575581f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:20.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.420+0000 7f9878584740 1 -- 172.21.15.83:0/315802016 shutdown_connections 2024-04-24T08:52:20.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.420+0000 7f9878584740 1 -- 172.21.15.83:0/315802016 wait complete. 2024-04-24T08:52:20.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.420+0000 7f9878584740 1 -- start start 2024-04-24T08:52:20.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.420+0000 7f9878584740 1 Processor -- start 2024-04-24T08:52:20.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.421+0000 7f9878584740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5575581fa860 0x557558209540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:20.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.421+0000 7f9878584740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5575581fb180 0x557558209a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:20.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.421+0000 7f9878584740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557558205850 0x557558209fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:20.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.421+0000 7f9878584740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f98701b2b90 con 0x5575581fa860 2024-04-24T08:52:20.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.421+0000 7f9878584740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f98701b24c0 con 0x557558205850 2024-04-24T08:52:20.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.421+0000 7f9878584740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f98701af700 con 0x5575581fb180 2024-04-24T08:52:20.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.421+0000 7f9878584740 10 client.0 _ll_register_callbacks cb 0x7f9868009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:52:20.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.421+0000 7f986ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5575581fa860 0x557558209540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:20.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.421+0000 7f986ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5575581fa860 0x557558209540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56036/0 (socket says 172.21.15.83:56036) 2024-04-24T08:52:20.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.421+0000 7f986ea00640 1 -- 172.21.15.83:0/1156012773 learned_addr learned my addr 172.21.15.83:0/1156012773 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:52:20.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.421+0000 7f986e000640 1 --2- 172.21.15.83:0/1156012773 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5575581fb180 0x557558209a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:20.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.422+0000 7f986f400640 1 --2- 172.21.15.83:0/1156012773 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557558205850 0x557558209fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:20.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.422+0000 7f986ea00640 1 -- 172.21.15.83:0/1156012773 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5575581fb180 msgr2=0x557558209a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:20.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.422+0000 7f986ea00640 1 --2- 172.21.15.83:0/1156012773 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5575581fb180 0x557558209a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:20.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.422+0000 7f986ea00640 1 -- 172.21.15.83:0/1156012773 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557558205850 msgr2=0x557558209fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:20.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.422+0000 7f986ea00640 1 --2- 172.21.15.83:0/1156012773 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557558205850 0x557558209fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:20.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.422+0000 7f986ea00640 1 -- 172.21.15.83:0/1156012773 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557557e320f0 con 0x5575581fa860 2024-04-24T08:52:20.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.422+0000 7f986ea00640 1 --2- 172.21.15.83:0/1156012773 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5575581fa860 0x557558209540 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f986009c1a0 tx=0x7f9860005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=184c7ba62962cf9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:20.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.422+0000 7f9858c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:52:20.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.422+0000 7f9858c00640 1 -- 172.21.15.83:0/1156012773 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9860020040 con 0x5575581fa860 2024-04-24T08:52:20.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.422+0000 7f9858c00640 1 -- 172.21.15.83:0/1156012773 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f9860098b10 con 0x5575581fa860 2024-04-24T08:52:20.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.422+0000 7f9878584740 1 -- 172.21.15.83:0/1156012773 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9868036040 con 0x5575581fa860 2024-04-24T08:52:20.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.422+0000 7f9858c00640 1 -- 172.21.15.83:0/1156012773 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f986009cba0 con 0x5575581fa860 2024-04-24T08:52:20.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.422+0000 7f9878584740 10 client.16029 fetch_fsmap learned FSMap version 28 2024-04-24T08:52:20.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.422+0000 7f9878584740 1 -- 172.21.15.83:0/1156012773 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557558125810 con 0x5575581fa860 2024-04-24T08:52:20.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.423+0000 7f9858c00640 1 -- 172.21.15.83:0/1156012773 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9868036040 con 0x5575581fa860 2024-04-24T08:52:20.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.423+0000 7f9858c00640 1 -- 172.21.15.83:0/1156012773 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f986009ba10 con 0x5575581fa860 2024-04-24T08:52:20.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.423+0000 7f9878584740 10 client.16029 fetch_fsmap finished waiting for FSMap version 28 2024-04-24T08:52:20.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.423+0000 7f9878584740 10 client.16029 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:52:20.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.423+0000 7f9878584740 20 client.16029 populate_metadata read hostname 'smithi083' 2024-04-24T08:52:20.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.423+0000 7f9878584740 1 --2- 172.21.15.83:0/1156012773 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55755827c8d0 0x55755827ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:20.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.423+0000 7f9878584740 4 client.16029 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T08:52:20.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.423+0000 7f9878584740 1 -- 172.21.15.83:0/1156012773 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557558167a80 con 0x55755827c8d0 2024-04-24T08:52:20.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.423+0000 7f986e000640 1 --2- 172.21.15.83:0/1156012773 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55755827c8d0 0x55755827ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:20.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.424+0000 7f986e000640 1 --2- 172.21.15.83:0/1156012773 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55755827c8d0 0x55755827ecf0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5715c4404fb91c12 server_cookie=750c22478b0e3f79 in_seq=0 out_seq=0 2024-04-24T08:52:20.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.424+0000 7f9858c00640 10 client.16029 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:52:20.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.425+0000 7f9858c00640 1 -- 172.21.15.83:0/1156012773 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x557558167a80 con 0x55755827c8d0 2024-04-24T08:52:20.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.425+0000 7f9858c00640 10 client.16029 handle_command_reply: tid=0 2024-04-24T08:52:20.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.425+0000 7f9878584740 1 client.16029 shutdown 2024-04-24T08:52:20.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.425+0000 7f9878584740 1 -- 172.21.15.83:0/1156012773 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5575581fa860 msgr2=0x557558209540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:20.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.425+0000 7f9878584740 1 --2- 172.21.15.83:0/1156012773 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5575581fa860 0x557558209540 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f986009c1a0 tx=0x7f9860005880 comp rx=0 tx=0).stop 2024-04-24T08:52:20.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.425+0000 7f9878584740 1 -- 172.21.15.83:0/1156012773 shutdown_connections 2024-04-24T08:52:20.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.425+0000 7f9878584740 1 --2- 172.21.15.83:0/1156012773 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55755827c8d0 0x55755827ecf0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:20.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.425+0000 7f9878584740 1 --2- 172.21.15.83:0/1156012773 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557558205850 0x557558209fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:20.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.425+0000 7f9878584740 1 --2- 172.21.15.83:0/1156012773 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5575581fb180 0x557558209a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:20.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.425+0000 7f9858c00640 0 client.16029 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:52:20.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.425+0000 7f9878584740 1 --2- 172.21.15.83:0/1156012773 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5575581fa860 0x557558209540 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:20.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.425+0000 7f9878584740 1 -- 172.21.15.83:0/1156012773 >> 172.21.15.83:0/1156012773 conn(0x5575581f6b80 msgr2=0x5575581f5390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:20.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.425+0000 7f9878584740 1 -- 172.21.15.83:0/1156012773 shutdown_connections 2024-04-24T08:52:20.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.425+0000 7f9878584740 1 -- 172.21.15.83:0/1156012773 wait complete. 2024-04-24T08:52:20.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.426+0000 7f9878584740 20 client.16029 trim_cache size 0 max 16384 2024-04-24T08:52:20.429 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T08:52:20.429 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T08:52:20.429 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-04-24T08:52:20.429 INFO:teuthology.orchestra.run.smithi083.stdout: "ab5f12b7-963c-4982-abdb-2faac1314a4a": { 2024-04-24T08:52:20.429 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-04-24T08:52:20.430 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "ab5f12b7-963c-4982-abdb-2faac1314a4a", 2024-04-24T08:52:20.430 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-04-24T08:52:20.430 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T08:52:20.430 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T08:52:20.430 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T08:52:20.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.426+0000 7f9875800640 1 -- 172.21.15.83:0/42631744 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f98540a3e70 msgr2=0x7f98540a6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:20.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.426+0000 7f9875800640 1 --2- 172.21.15.83:0/42631744 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f98540a3e70 0x7f98540a6330 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9868009430 tx=0x7f986802f310 comp rx=0 tx=0).stop 2024-04-24T08:52:20.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.426+0000 7f9875800640 1 -- 172.21.15.83:0/42631744 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9870102240 msgr2=0x7f9870074300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:20.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.426+0000 7f9875800640 1 --2- 172.21.15.83:0/42631744 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9870102240 0x7f9870074300 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f986000d950 tx=0x7f986000de10 comp rx=0 tx=0).stop 2024-04-24T08:52:20.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.426+0000 7f9875800640 1 -- 172.21.15.83:0/42631744 shutdown_connections 2024-04-24T08:52:20.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.426+0000 7f9875800640 1 --2- 172.21.15.83:0/42631744 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f98540a3e70 0x7f98540a6330 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:20.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.426+0000 7f9875800640 1 --2- 172.21.15.83:0/42631744 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9870074d80 0x7f98701a69a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:20.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.426+0000 7f9875800640 1 --2- 172.21.15.83:0/42631744 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9870107310 0x7f9870074840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:20.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.426+0000 7f9875800640 1 --2- 172.21.15.83:0/42631744 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9870102240 0x7f9870074300 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:20.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.426+0000 7f9875800640 1 -- 172.21.15.83:0/42631744 >> 172.21.15.83:0/42631744 conn(0x7f9870079450 msgr2=0x7f98700fff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:20.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.426+0000 7f9875800640 1 -- 172.21.15.83:0/42631744 shutdown_connections 2024-04-24T08:52:20.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:20.426+0000 7f9875800640 1 -- 172.21.15.83:0/42631744 wait complete. 2024-04-24T08:52:20.444 INFO:tasks.cephfs.filesystem:scrub status for tag:ab5f12b7-963c-4982-abdb-2faac1314a4a - {'path': '/', 'tag': 'ab5f12b7-963c-4982-abdb-2faac1314a4a', 'options': 'recursive,force'} 2024-04-24T08:52:21.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:52:21 smithi138 ceph-mon[31441]: scrub queued for path: / 2024-04-24T08:52:21.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:52:21 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T08:52:21.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:52:21 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T08:52:21.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:52:21 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T08:52:21.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:52:21 smithi138 ceph-mon[31441]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-24T08:52:21.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:52:21 smithi189 ceph-mon[31791]: scrub queued for path: / 2024-04-24T08:52:21.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:52:21 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T08:52:21.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:52:21 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T08:52:21.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:52:21 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T08:52:21.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:52:21 smithi189 ceph-mon[31791]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-24T08:52:22.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:52:21 smithi083 ceph-mon[27907]: scrub queued for path: / 2024-04-24T08:52:22.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:52:21 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T08:52:22.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:52:21 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T08:52:22.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:52:21 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T08:52:22.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:52:21 smithi083 ceph-mon[27907]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-24T08:52:22.954 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:52:22 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T08:52:22.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:52:22 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T08:52:23.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:52:22 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T08:52:23.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:52:23 smithi138 ceph-mon[31441]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-24T08:52:23.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:52:23 smithi189 ceph-mon[31791]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-24T08:52:24.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:52:23 smithi083 ceph-mon[27907]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-24T08:52:25.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:52:25 smithi138 ceph-mon[31441]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-24T08:52:25.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:52:25 smithi189 ceph-mon[31791]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-24T08:52:26.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:52:25 smithi083 ceph-mon[27907]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-24T08:52:27.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:52:27 smithi138 ceph-mon[31441]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-24T08:52:27.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:52:27 smithi189 ceph-mon[31791]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-24T08:52:28.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:52:27 smithi083 ceph-mon[27907]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-24T08:52:28.394 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.395+0000 7f828a200640 1 -- 172.21.15.83:0/3664207333 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8284112e90 msgr2=0x7f8284115280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:28.395 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.395+0000 7f828a200640 1 --2- 172.21.15.83:0/3664207333 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8284112e90 0x7f8284115280 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f827c009ca0 tx=0x7f827c02a8c0 comp rx=0 tx=0).stop 2024-04-24T08:52:28.395 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.395+0000 7f828a200640 1 -- 172.21.15.83:0/3664207333 shutdown_connections 2024-04-24T08:52:28.395 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.395+0000 7f828a200640 1 --2- 172.21.15.83:0/3664207333 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8284112e90 0x7f8284115280 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:28.395 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.395+0000 7f828a200640 1 --2- 172.21.15.83:0/3664207333 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8284076e50 0x7f8284112950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:28.395 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.395+0000 7f828a200640 1 --2- 172.21.15.83:0/3664207333 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8284076530 0x7f8284076910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:28.395 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.395+0000 7f828a200640 1 -- 172.21.15.83:0/3664207333 >> 172.21.15.83:0/3664207333 conn(0x7f82840ffd00 msgr2=0x7f8284102120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:28.395 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.395+0000 7f828a200640 1 -- 172.21.15.83:0/3664207333 shutdown_connections 2024-04-24T08:52:28.395 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.395+0000 7f828a200640 1 -- 172.21.15.83:0/3664207333 wait complete. 2024-04-24T08:52:28.398 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.396+0000 7f828a200640 1 Processor -- start 2024-04-24T08:52:28.398 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.396+0000 7f828a200640 1 -- start start 2024-04-24T08:52:28.398 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.396+0000 7f828a200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8284076530 0x7f82841a4a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:28.398 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.396+0000 7f828a200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8284076e50 0x7f82841a4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:28.398 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.396+0000 7f828a200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8284112e90 0x7f828419eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:28.398 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.396+0000 7f828a200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f82841188f0 con 0x7f8284112e90 2024-04-24T08:52:28.398 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.396+0000 7f828a200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8284118770 con 0x7f8284076e50 2024-04-24T08:52:28.398 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.396+0000 7f828a200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8284118a70 con 0x7f8284076530 2024-04-24T08:52:28.398 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.397+0000 7f8283e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8284112e90 0x7f828419eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:28.398 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.397+0000 7f8283e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8284112e90 0x7f828419eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56062/0 (socket says 172.21.15.83:56062) 2024-04-24T08:52:28.398 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.397+0000 7f8283e00640 1 -- 172.21.15.83:0/1851976797 learned_addr learned my addr 172.21.15.83:0/1851976797 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:52:28.399 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.397+0000 7f8283400640 1 --2- 172.21.15.83:0/1851976797 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8284076530 0x7f82841a4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:28.399 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.397+0000 7f8282a00640 1 --2- 172.21.15.83:0/1851976797 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8284076e50 0x7f82841a4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:28.399 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.397+0000 7f8283e00640 1 -- 172.21.15.83:0/1851976797 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8284076530 msgr2=0x7f82841a4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:28.399 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.397+0000 7f8283e00640 1 --2- 172.21.15.83:0/1851976797 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8284076530 0x7f82841a4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:28.399 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.397+0000 7f8283e00640 1 -- 172.21.15.83:0/1851976797 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8284076e50 msgr2=0x7f82841a4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:28.399 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.397+0000 7f8283e00640 1 --2- 172.21.15.83:0/1851976797 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8284076e50 0x7f82841a4fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:28.399 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.397+0000 7f8283e00640 1 -- 172.21.15.83:0/1851976797 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f828419f430 con 0x7f8284112e90 2024-04-24T08:52:28.399 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.397+0000 7f8282a00640 1 --2- 172.21.15.83:0/1851976797 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8284076e50 0x7f82841a4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:52:28.399 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.398+0000 7f8283e00640 1 --2- 172.21.15.83:0/1851976797 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8284112e90 0x7f828419eb70 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f827c031040 tx=0x7f827c003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:28.399 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.398+0000 7f827be00640 1 -- 172.21.15.83:0/1851976797 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f827c003ce0 con 0x7f8284112e90 2024-04-24T08:52:28.399 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.398+0000 7f827be00640 1 -- 172.21.15.83:0/1851976797 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f827c0044f0 con 0x7f8284112e90 2024-04-24T08:52:28.399 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.398+0000 7f827be00640 1 -- 172.21.15.83:0/1851976797 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f827c030c60 con 0x7f8284112e90 2024-04-24T08:52:28.400 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.398+0000 7f828a200640 1 -- 172.21.15.83:0/1851976797 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f828419f6c0 con 0x7f8284112e90 2024-04-24T08:52:28.400 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.398+0000 7f828a200640 1 -- 172.21.15.83:0/1851976797 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82841ab800 con 0x7f8284112e90 2024-04-24T08:52:28.401 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.401+0000 7f827be00640 1 -- 172.21.15.83:0/1851976797 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7f827c0046a0 con 0x7f8284112e90 2024-04-24T08:52:28.401 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.402+0000 7f828d072740 1 Processor -- start 2024-04-24T08:52:28.401 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.402+0000 7f828d072740 1 -- start start 2024-04-24T08:52:28.403 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.402+0000 7f828d072740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564942c421b0 0x564942c42590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:28.403 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.402+0000 7f828d072740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564942c42ad0 0x564942c4ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:28.403 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.402+0000 7f828d072740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564942c4d390 0x564942c4f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:28.403 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.402+0000 7f828d072740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x564942ba8900 con 0x564942c421b0 2024-04-24T08:52:28.403 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.402+0000 7f828d072740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x564942b4ef20 con 0x564942c4d390 2024-04-24T08:52:28.403 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.402+0000 7f828d072740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x564942ba48c0 con 0x564942c42ad0 2024-04-24T08:52:28.403 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.402+0000 7f8282a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564942c421b0 0x564942c42590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:28.403 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.402+0000 7f8282a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564942c421b0 0x564942c42590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56064/0 (socket says 172.21.15.83:56064) 2024-04-24T08:52:28.403 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.402+0000 7f8282a00640 1 -- 172.21.15.83:0/2795234980 learned_addr learned my addr 172.21.15.83:0/2795234980 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:52:28.403 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.403+0000 7f8283400640 1 --2- 172.21.15.83:0/2795234980 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564942c42ad0 0x564942c4ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:28.403 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.403+0000 7f8282a00640 1 -- 172.21.15.83:0/2795234980 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564942c42ad0 msgr2=0x564942c4ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:28.403 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.403+0000 7f8282a00640 1 --2- 172.21.15.83:0/2795234980 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564942c42ad0 0x564942c4ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:28.404 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.403+0000 7f8282a00640 1 -- 172.21.15.83:0/2795234980 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564942c4d390 msgr2=0x564942c4f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:28.404 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.403+0000 7f8282a00640 1 --2- 172.21.15.83:0/2795234980 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564942c4d390 0x564942c4f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:28.404 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.403+0000 7f8282a00640 1 -- 172.21.15.83:0/2795234980 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564942b73e10 con 0x564942c421b0 2024-04-24T08:52:28.404 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.403+0000 7f827be00640 1 --2- 172.21.15.83:0/1851976797 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f82680a3f60 0x7f82680a6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:28.404 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.403+0000 7f8282a00640 1 --2- 172.21.15.83:0/2795234980 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564942c421b0 0x564942c42590 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f82700058c0 tx=0x7f82700106e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a837eadcbd5b72fa server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:28.404 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.403+0000 7f827be00640 1 -- 172.21.15.83:0/1851976797 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f827c0f2250 con 0x7f8284112e90 2024-04-24T08:52:28.404 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.403+0000 7f8283400640 1 --2- 172.21.15.83:0/1851976797 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f82680a3f60 0x7f82680a6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:28.404 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.404+0000 7f826fe00640 1 -- 172.21.15.83:0/2795234980 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8270016070 con 0x564942c421b0 2024-04-24T08:52:28.408 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.404+0000 7f826fe00640 1 -- 172.21.15.83:0/2795234980 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f82700054d0 con 0x564942c421b0 2024-04-24T08:52:28.408 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.404+0000 7f826fe00640 1 -- 172.21.15.83:0/2795234980 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8270012420 con 0x564942c421b0 2024-04-24T08:52:28.408 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.404+0000 7f8283400640 1 --2- 172.21.15.83:0/1851976797 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f82680a3f60 0x7f82680a6420 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8274005fd0 tx=0x7f827400b650 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:28.408 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.405+0000 7f828d072740 1 -- 172.21.15.83:0/2795234980 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564942c421b0 msgr2=0x564942c42590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:28.408 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.405+0000 7f828d072740 1 --2- 172.21.15.83:0/2795234980 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564942c421b0 0x564942c42590 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f82700058c0 tx=0x7f82700106e0 comp rx=0 tx=0).stop 2024-04-24T08:52:28.408 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.405+0000 7f828d072740 1 -- 172.21.15.83:0/2795234980 shutdown_connections 2024-04-24T08:52:28.408 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.405+0000 7f828d072740 1 --2- 172.21.15.83:0/2795234980 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564942c4d390 0x564942c4f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:28.409 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.405+0000 7f828d072740 1 --2- 172.21.15.83:0/2795234980 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564942c42ad0 0x564942c4ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:28.409 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.405+0000 7f828d072740 1 --2- 172.21.15.83:0/2795234980 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564942c421b0 0x564942c42590 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:28.409 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.405+0000 7f828d072740 1 -- 172.21.15.83:0/2795234980 >> 172.21.15.83:0/2795234980 conn(0x564942c3e6c0 msgr2=0x564942c3eaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:28.409 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.405+0000 7f828d072740 1 -- 172.21.15.83:0/2795234980 shutdown_connections 2024-04-24T08:52:28.409 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.405+0000 7f828d072740 1 -- 172.21.15.83:0/2795234980 wait complete. 2024-04-24T08:52:28.409 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.406+0000 7f828d072740 1 -- start start 2024-04-24T08:52:28.409 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.406+0000 7f828d072740 1 Processor -- start 2024-04-24T08:52:28.409 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.406+0000 7f828d072740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564942c421b0 0x564942c4a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:28.409 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.406+0000 7f828d072740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564942c42ad0 0x564942c4a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:28.409 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.406+0000 7f828d072740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564942c4d390 0x564942c4ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:28.409 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.406+0000 7f828d072740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x564942c4ffc0 con 0x564942c421b0 2024-04-24T08:52:28.409 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.406+0000 7f828d072740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x564942b622c0 con 0x564942c4d390 2024-04-24T08:52:28.410 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.406+0000 7f828d072740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x564942c50140 con 0x564942c42ad0 2024-04-24T08:52:28.410 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.406+0000 7f828d072740 10 client.0 _ll_register_callbacks cb 0x5649428baa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:52:28.410 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.407+0000 7f8283400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564942c421b0 0x564942c4a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:28.410 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.407+0000 7f8283400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564942c421b0 0x564942c4a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56078/0 (socket says 172.21.15.83:56078) 2024-04-24T08:52:28.410 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.407+0000 7f8283400640 1 -- 172.21.15.83:0/2352037380 learned_addr learned my addr 172.21.15.83:0/2352037380 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:52:28.410 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.407+0000 7f8282a00640 1 --2- 172.21.15.83:0/2352037380 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564942c42ad0 0x564942c4a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:28.410 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.407+0000 7f8283e00640 1 --2- 172.21.15.83:0/2352037380 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564942c4d390 0x564942c4ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:28.410 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.407+0000 7f8283400640 1 -- 172.21.15.83:0/2352037380 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564942c42ad0 msgr2=0x564942c4a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:28.410 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.407+0000 7f8283400640 1 --2- 172.21.15.83:0/2352037380 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564942c42ad0 0x564942c4a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:28.410 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.407+0000 7f8283400640 1 -- 172.21.15.83:0/2352037380 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564942c4d390 msgr2=0x564942c4ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:28.410 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.407+0000 7f8283400640 1 --2- 172.21.15.83:0/2352037380 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564942c4d390 0x564942c4ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:28.410 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.407+0000 7f8283400640 1 -- 172.21.15.83:0/2352037380 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564942b72fd0 con 0x564942c421b0 2024-04-24T08:52:28.411 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.407+0000 7f8283400640 1 --2- 172.21.15.83:0/2352037380 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564942c421b0 0x564942c4a380 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f8274012700 tx=0x7f82740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87bc0bdc766f9a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:28.411 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.407+0000 7f826cc00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:52:28.411 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.408+0000 7f826cc00640 1 -- 172.21.15.83:0/2352037380 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f827401f8f0 con 0x564942c421b0 2024-04-24T08:52:28.411 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.408+0000 7f826cc00640 1 -- 172.21.15.83:0/2352037380 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f827401fa90 con 0x564942c421b0 2024-04-24T08:52:28.411 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.408+0000 7f828d072740 1 -- 172.21.15.83:0/2352037380 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82700054d0 con 0x564942c421b0 2024-04-24T08:52:28.411 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.408+0000 7f826cc00640 1 -- 172.21.15.83:0/2352037380 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f827401fc90 con 0x564942c421b0 2024-04-24T08:52:28.411 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.408+0000 7f828d072740 10 client.16041 fetch_fsmap learned FSMap version 28 2024-04-24T08:52:28.411 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.408+0000 7f828d072740 1 -- 172.21.15.83:0/2352037380 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5649428bb210 con 0x564942c421b0 2024-04-24T08:52:28.411 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.408+0000 7f826cc00640 1 -- 172.21.15.83:0/2352037380 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f82700054d0 con 0x564942c421b0 2024-04-24T08:52:28.411 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.409+0000 7f826cc00640 1 -- 172.21.15.83:0/2352037380 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f8274027400 con 0x564942c421b0 2024-04-24T08:52:28.411 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.409+0000 7f828d072740 10 client.16041 fetch_fsmap finished waiting for FSMap version 28 2024-04-24T08:52:28.411 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.409+0000 7f828d072740 10 client.16041 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T08:52:28.411 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.409+0000 7f828d072740 20 client.16041 populate_metadata read hostname 'smithi083' 2024-04-24T08:52:28.411 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.409+0000 7f828d072740 1 --2- 172.21.15.83:0/2352037380 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x564942c5d5e0 0x564942c5fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:28.412 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.409+0000 7f828d072740 4 client.16041 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:52:28.412 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.409+0000 7f828d072740 1 -- 172.21.15.83:0/2352037380 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564942bb6320 con 0x564942c5d5e0 2024-04-24T08:52:28.413 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.409+0000 7f8282a00640 1 --2- 172.21.15.83:0/2352037380 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x564942c5d5e0 0x564942c5fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:28.413 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.410+0000 7f8282a00640 1 --2- 172.21.15.83:0/2352037380 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x564942c5d5e0 0x564942c5fa40 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5e34cd52408a074 server_cookie=39a9d7d8948cec87 in_seq=0 out_seq=0 2024-04-24T08:52:28.413 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.410+0000 7f826cc00640 10 client.16041 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:52:28.413 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.412+0000 7f826cc00640 1 -- 172.21.15.83:0/2352037380 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564942bb6320 con 0x564942c5d5e0 2024-04-24T08:52:28.413 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.412+0000 7f826cc00640 10 client.16041 handle_command_reply: tid=0 2024-04-24T08:52:28.413 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.412+0000 7f828d072740 1 client.16041 shutdown 2024-04-24T08:52:28.413 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.412+0000 7f828d072740 1 -- 172.21.15.83:0/2352037380 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564942c421b0 msgr2=0x564942c4a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:28.413 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.412+0000 7f828d072740 1 --2- 172.21.15.83:0/2352037380 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564942c421b0 0x564942c4a380 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f8274012700 tx=0x7f82740046c0 comp rx=0 tx=0).stop 2024-04-24T08:52:28.413 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.412+0000 7f828d072740 1 -- 172.21.15.83:0/2352037380 shutdown_connections 2024-04-24T08:52:28.413 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.412+0000 7f828d072740 1 --2- 172.21.15.83:0/2352037380 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x564942c5d5e0 0x564942c5fa40 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:28.413 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.413+0000 7f828d072740 1 --2- 172.21.15.83:0/2352037380 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564942c4d390 0x564942c4ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:28.414 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.413+0000 7f828d072740 1 --2- 172.21.15.83:0/2352037380 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564942c42ad0 0x564942c4a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:28.414 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.413+0000 7f828d072740 1 --2- 172.21.15.83:0/2352037380 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564942c421b0 0x564942c4a380 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:28.414 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.413+0000 7f826cc00640 0 client.16041 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:52:28.414 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.413+0000 7f828d072740 1 -- 172.21.15.83:0/2352037380 >> 172.21.15.83:0/2352037380 conn(0x564942c3e6c0 msgr2=0x564942c44440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:28.414 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.413+0000 7f828d072740 1 -- 172.21.15.83:0/2352037380 shutdown_connections 2024-04-24T08:52:28.414 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.413+0000 7f828d072740 1 -- 172.21.15.83:0/2352037380 wait complete. 2024-04-24T08:52:28.414 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.413+0000 7f828d072740 20 client.16041 trim_cache size 0 max 16384 2024-04-24T08:52:28.430 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.429+0000 7f828d072740 1 Processor -- start 2024-04-24T08:52:28.430 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.429+0000 7f828d072740 1 -- start start 2024-04-24T08:52:28.430 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.429+0000 7f828d072740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564942c421b0 0x564942c58e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:28.430 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.429+0000 7f828d072740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564942c42ad0 0x564942c44b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:28.430 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.429+0000 7f828d072740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564942c4d390 0x564942c65310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:28.430 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.429+0000 7f828d072740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f82841b5aa0 con 0x564942c42ad0 2024-04-24T08:52:28.430 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.429+0000 7f828d072740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f82841b75e0 con 0x564942c4d390 2024-04-24T08:52:28.430 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.429+0000 7f828d072740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f82841b6170 con 0x564942c421b0 2024-04-24T08:52:28.430 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.430+0000 7f8282a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564942c42ad0 0x564942c44b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:28.430 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.430+0000 7f8282a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564942c42ad0 0x564942c44b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56084/0 (socket says 172.21.15.83:56084) 2024-04-24T08:52:28.430 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.430+0000 7f8282a00640 1 -- 172.21.15.83:0/3809668535 learned_addr learned my addr 172.21.15.83:0/3809668535 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:52:28.431 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.430+0000 7f8283e00640 1 --2- 172.21.15.83:0/3809668535 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564942c4d390 0x564942c65310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:28.431 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.430+0000 7f8283400640 1 --2- 172.21.15.83:0/3809668535 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564942c421b0 0x564942c58e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:28.431 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.430+0000 7f8282a00640 1 -- 172.21.15.83:0/3809668535 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564942c421b0 msgr2=0x564942c58e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:28.431 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.430+0000 7f8282a00640 1 --2- 172.21.15.83:0/3809668535 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564942c421b0 0x564942c58e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:28.431 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.430+0000 7f8282a00640 1 -- 172.21.15.83:0/3809668535 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564942c4d390 msgr2=0x564942c65310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:28.431 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.430+0000 7f8282a00640 1 --2- 172.21.15.83:0/3809668535 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564942c4d390 0x564942c65310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:28.431 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.430+0000 7f8282a00640 1 -- 172.21.15.83:0/3809668535 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564942987620 con 0x564942c42ad0 2024-04-24T08:52:28.431 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.430+0000 7f8283e00640 1 --2- 172.21.15.83:0/3809668535 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564942c4d390 0x564942c65310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:52:28.431 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.430+0000 7f8283400640 1 --2- 172.21.15.83:0/3809668535 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564942c421b0 0x564942c58e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:52:28.431 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.431+0000 7f8282a00640 1 --2- 172.21.15.83:0/3809668535 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564942c42ad0 0x564942c44b70 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f8270005fd0 tx=0x7f8270009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcf6db55719bfcd4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:28.435 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.431+0000 7f826f400640 1 -- 172.21.15.83:0/3809668535 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8270016040 con 0x564942c42ad0 2024-04-24T08:52:28.435 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.431+0000 7f826f400640 1 -- 172.21.15.83:0/3809668535 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f82700284d0 con 0x564942c42ad0 2024-04-24T08:52:28.435 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.431+0000 7f826f400640 1 -- 172.21.15.83:0/3809668535 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8270027a10 con 0x564942c42ad0 2024-04-24T08:52:28.436 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.432+0000 7f828d072740 1 -- 172.21.15.83:0/3809668535 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564942c42ad0 msgr2=0x564942c44b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:28.436 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.432+0000 7f828d072740 1 --2- 172.21.15.83:0/3809668535 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564942c42ad0 0x564942c44b70 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f8270005fd0 tx=0x7f8270009510 comp rx=0 tx=0).stop 2024-04-24T08:52:28.436 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.432+0000 7f828d072740 1 -- 172.21.15.83:0/3809668535 shutdown_connections 2024-04-24T08:52:28.436 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.432+0000 7f828d072740 1 --2- 172.21.15.83:0/3809668535 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564942c4d390 0x564942c65310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:28.436 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.432+0000 7f828d072740 1 --2- 172.21.15.83:0/3809668535 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564942c42ad0 0x564942c44b70 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:28.436 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.432+0000 7f828d072740 1 --2- 172.21.15.83:0/3809668535 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564942c421b0 0x564942c58e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:28.436 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.432+0000 7f828d072740 1 -- 172.21.15.83:0/3809668535 >> 172.21.15.83:0/3809668535 conn(0x564942c3e6c0 msgr2=0x564942c3d840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:28.436 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.432+0000 7f828d072740 1 -- 172.21.15.83:0/3809668535 shutdown_connections 2024-04-24T08:52:28.436 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.432+0000 7f828d072740 1 -- 172.21.15.83:0/3809668535 wait complete. 2024-04-24T08:52:28.436 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.432+0000 7f828d072740 1 -- start start 2024-04-24T08:52:28.436 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.432+0000 7f828d072740 1 Processor -- start 2024-04-24T08:52:28.437 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.433+0000 7f828d072740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564942c421b0 0x564942c552f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:28.437 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.433+0000 7f828d072740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564942c42ad0 0x564942c55830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:28.437 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.433+0000 7f828d072740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564942c4d390 0x564942c50850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:28.437 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.433+0000 7f828d072740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f82841b6f10 con 0x564942c42ad0 2024-04-24T08:52:28.437 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.433+0000 7f828d072740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f82841b6840 con 0x564942c421b0 2024-04-24T08:52:28.437 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.433+0000 7f828d072740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f82841b3a80 con 0x564942c4d390 2024-04-24T08:52:28.437 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.433+0000 7f828d072740 10 client.0 _ll_register_callbacks cb 0x7f827400b3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:52:28.437 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.433+0000 7f8282a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564942c42ad0 0x564942c55830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:28.437 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.433+0000 7f8282a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564942c42ad0 0x564942c55830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56098/0 (socket says 172.21.15.83:56098) 2024-04-24T08:52:28.437 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.433+0000 7f8282a00640 1 -- 172.21.15.83:0/2120803953 learned_addr learned my addr 172.21.15.83:0/2120803953 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:52:28.437 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.433+0000 7f8283400640 1 --2- 172.21.15.83:0/2120803953 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564942c421b0 0x564942c552f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:28.437 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.433+0000 7f8282a00640 1 -- 172.21.15.83:0/2120803953 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564942c4d390 msgr2=0x564942c50850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:28.437 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.433+0000 7f8283e00640 1 --2- 172.21.15.83:0/2120803953 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564942c4d390 0x564942c50850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:28.437 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.433+0000 7f8282a00640 1 --2- 172.21.15.83:0/2120803953 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564942c4d390 0x564942c50850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:28.438 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.433+0000 7f8282a00640 1 -- 172.21.15.83:0/2120803953 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564942c421b0 msgr2=0x564942c552f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:28.438 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.433+0000 7f8282a00640 1 --2- 172.21.15.83:0/2120803953 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564942c421b0 0x564942c552f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:28.438 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.433+0000 7f8282a00640 1 -- 172.21.15.83:0/2120803953 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564942c51050 con 0x564942c42ad0 2024-04-24T08:52:28.438 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.433+0000 7f8283400640 1 --2- 172.21.15.83:0/2120803953 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564942c421b0 0x564942c552f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:52:28.438 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.434+0000 7f8282a00640 1 --2- 172.21.15.83:0/2120803953 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564942c42ad0 0x564942c55830 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f8270009c00 tx=0x7f82700093b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a72c69010c9495f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:28.438 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.434+0000 7f826cc00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:52:28.438 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.434+0000 7f826cc00640 1 -- 172.21.15.83:0/2120803953 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8270016070 con 0x564942c42ad0 2024-04-24T08:52:28.438 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.434+0000 7f826cc00640 1 -- 172.21.15.83:0/2120803953 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f8270028020 con 0x564942c42ad0 2024-04-24T08:52:28.438 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.434+0000 7f828d072740 1 -- 172.21.15.83:0/2120803953 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82700284d0 con 0x564942c42ad0 2024-04-24T08:52:28.438 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.434+0000 7f826cc00640 1 -- 172.21.15.83:0/2120803953 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8270028220 con 0x564942c42ad0 2024-04-24T08:52:28.438 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.434+0000 7f828d072740 10 client.16047 fetch_fsmap learned FSMap version 28 2024-04-24T08:52:28.438 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.434+0000 7f828d072740 1 -- 172.21.15.83:0/2120803953 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564942cc36e0 con 0x564942c42ad0 2024-04-24T08:52:28.438 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.435+0000 7f826cc00640 1 -- 172.21.15.83:0/2120803953 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f82700284d0 con 0x564942c42ad0 2024-04-24T08:52:28.439 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.435+0000 7f826cc00640 1 -- 172.21.15.83:0/2120803953 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f8270003af0 con 0x564942c42ad0 2024-04-24T08:52:28.439 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.435+0000 7f828d072740 10 client.16047 fetch_fsmap finished waiting for FSMap version 28 2024-04-24T08:52:28.439 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.435+0000 7f828d072740 10 client.16047 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T08:52:28.439 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.435+0000 7f828d072740 20 client.16047 populate_metadata read hostname 'smithi083' 2024-04-24T08:52:28.439 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.435+0000 7f828d072740 1 --2- 172.21.15.83:0/2120803953 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x564942cc3bd0 0x564942cc5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:28.439 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.435+0000 7f828d072740 4 client.16047 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T08:52:28.439 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.435+0000 7f828d072740 1 -- 172.21.15.83:0/2120803953 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564942bb0190 con 0x564942cc3bd0 2024-04-24T08:52:28.439 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.435+0000 7f8283400640 1 --2- 172.21.15.83:0/2120803953 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x564942cc3bd0 0x564942cc5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:28.439 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.437+0000 7f826cc00640 10 client.16047 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:52:28.440 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.437+0000 7f8283400640 1 --2- 172.21.15.83:0/2120803953 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x564942cc3bd0 0x564942cc5ff0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24a105fd7c9d1ca6 server_cookie=460ab9ef3c405a87 in_seq=0 out_seq=0 2024-04-24T08:52:28.447 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.447+0000 7f826cc00640 1 -- 172.21.15.83:0/2120803953 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564942bb0190 con 0x564942cc3bd0 2024-04-24T08:52:28.447 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.447+0000 7f826cc00640 10 client.16047 handle_command_reply: tid=0 2024-04-24T08:52:28.447 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.447+0000 7f828d072740 1 client.16047 shutdown 2024-04-24T08:52:28.447 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.447+0000 7f828d072740 1 -- 172.21.15.83:0/2120803953 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564942c42ad0 msgr2=0x564942c55830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:28.447 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.447+0000 7f828d072740 1 --2- 172.21.15.83:0/2120803953 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564942c42ad0 0x564942c55830 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f8270009c00 tx=0x7f82700093b0 comp rx=0 tx=0).stop 2024-04-24T08:52:28.447 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.447+0000 7f828d072740 1 -- 172.21.15.83:0/2120803953 shutdown_connections 2024-04-24T08:52:28.447 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.447+0000 7f828d072740 1 --2- 172.21.15.83:0/2120803953 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x564942cc3bd0 0x564942cc5ff0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:28.447 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.447+0000 7f828d072740 1 --2- 172.21.15.83:0/2120803953 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564942c4d390 0x564942c50850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:28.448 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.447+0000 7f828d072740 1 --2- 172.21.15.83:0/2120803953 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564942c42ad0 0x564942c55830 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:28.448 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.447+0000 7f828d072740 1 --2- 172.21.15.83:0/2120803953 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564942c421b0 0x564942c552f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:28.448 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.447+0000 7f826cc00640 0 client.16047 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:52:28.448 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.447+0000 7f828d072740 1 -- 172.21.15.83:0/2120803953 >> 172.21.15.83:0/2120803953 conn(0x564942c3e6c0 msgr2=0x564942c3cfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:28.448 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.447+0000 7f828d072740 1 -- 172.21.15.83:0/2120803953 shutdown_connections 2024-04-24T08:52:28.448 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.447+0000 7f828d072740 1 -- 172.21.15.83:0/2120803953 wait complete. 2024-04-24T08:52:28.448 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.448+0000 7f828d072740 20 client.16047 trim_cache size 0 max 16384 2024-04-24T08:52:28.448 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T08:52:28.448 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T08:52:28.449 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T08:52:28.449 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T08:52:28.449 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.448+0000 7f828a200640 1 -- 172.21.15.83:0/1851976797 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f82680a3f60 msgr2=0x7f82680a6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:28.449 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.448+0000 7f828a200640 1 --2- 172.21.15.83:0/1851976797 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f82680a3f60 0x7f82680a6420 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8274005fd0 tx=0x7f827400b650 comp rx=0 tx=0).stop 2024-04-24T08:52:28.450 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.448+0000 7f828a200640 1 -- 172.21.15.83:0/1851976797 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8284112e90 msgr2=0x7f828419eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:28.450 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.448+0000 7f828a200640 1 --2- 172.21.15.83:0/1851976797 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8284112e90 0x7f828419eb70 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f827c031040 tx=0x7f827c003970 comp rx=0 tx=0).stop 2024-04-24T08:52:28.450 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.448+0000 7f828a200640 1 -- 172.21.15.83:0/1851976797 shutdown_connections 2024-04-24T08:52:28.450 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.448+0000 7f828a200640 1 --2- 172.21.15.83:0/1851976797 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f82680a3f60 0x7f82680a6420 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:28.450 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.448+0000 7f828a200640 1 --2- 172.21.15.83:0/1851976797 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8284112e90 0x7f828419eb70 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:28.450 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.448+0000 7f828a200640 1 --2- 172.21.15.83:0/1851976797 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8284076e50 0x7f82841a4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:28.450 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.448+0000 7f828a200640 1 --2- 172.21.15.83:0/1851976797 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8284076530 0x7f82841a4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:28.450 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.448+0000 7f828a200640 1 -- 172.21.15.83:0/1851976797 >> 172.21.15.83:0/1851976797 conn(0x7f82840ffd00 msgr2=0x7f8284114920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:28.450 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.449+0000 7f828a200640 1 -- 172.21.15.83:0/1851976797 shutdown_connections 2024-04-24T08:52:28.450 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:28.449+0000 7f828a200640 1 -- 172.21.15.83:0/1851976797 wait complete. 2024-04-24T08:52:29.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:52:29 smithi138 ceph-mon[31441]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-24T08:52:29.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:52:29 smithi189 ceph-mon[31791]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-24T08:52:30.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:52:29 smithi083 ceph-mon[27907]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-24T08:52:31.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:52:31 smithi138 ceph-mon[31441]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-24T08:52:31.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:52:31 smithi189 ceph-mon[31791]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-24T08:52:32.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:52:31 smithi083 ceph-mon[27907]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-24T08:52:33.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:52:33 smithi138 ceph-mon[31441]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-24T08:52:33.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:52:33 smithi189 ceph-mon[31791]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-24T08:52:34.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:52:33 smithi083 ceph-mon[27907]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-24T08:52:35.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:52:35 smithi138 ceph-mon[31441]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-24T08:52:35.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:52:35 smithi189 ceph-mon[31791]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-24T08:52:36.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:52:35 smithi083 ceph-mon[27907]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-24T08:52:37.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:52:37 smithi138 ceph-mon[31441]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-24T08:52:37.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:52:37 smithi189 ceph-mon[31791]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-24T08:52:38.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:52:37 smithi083 ceph-mon[27907]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-24T08:52:39.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:52:39 smithi138 ceph-mon[31441]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-24T08:52:39.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:52:39 smithi189 ceph-mon[31791]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-24T08:52:40.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:52:39 smithi083 ceph-mon[27907]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-24T08:52:41.583 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.582+0000 7f7fd1200640 1 -- 172.21.15.83:0/2183703512 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7fcc0743a0 msgr2=0x7f7fcc074780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:41.583 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.582+0000 7f7fd1200640 1 --2- 172.21.15.83:0/2183703512 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7fcc0743a0 0x7f7fcc074780 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc009900 tx=0x7f7fbc02a480 comp rx=0 tx=0).stop 2024-04-24T08:52:41.583 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.582+0000 7f7fd1200640 1 -- 172.21.15.83:0/2183703512 shutdown_connections 2024-04-24T08:52:41.583 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.582+0000 7f7fd1200640 1 --2- 172.21.15.83:0/2183703512 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7fcc112e10 0x7f7fcc115200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:41.583 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.582+0000 7f7fd1200640 1 --2- 172.21.15.83:0/2183703512 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7fcc074cc0 0x7f7fcc1128d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:41.583 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.582+0000 7f7fd1200640 1 --2- 172.21.15.83:0/2183703512 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7fcc0743a0 0x7f7fcc074780 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:41.584 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.582+0000 7f7fd1200640 1 -- 172.21.15.83:0/2183703512 >> 172.21.15.83:0/2183703512 conn(0x7f7fcc0fd950 msgr2=0x7f7fcc0ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:41.584 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.582+0000 7f7fd1200640 1 -- 172.21.15.83:0/2183703512 shutdown_connections 2024-04-24T08:52:41.584 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.583+0000 7f7fd1200640 1 -- 172.21.15.83:0/2183703512 wait complete. 2024-04-24T08:52:41.586 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.583+0000 7f7fd1200640 1 Processor -- start 2024-04-24T08:52:41.586 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.584+0000 7f7fd1200640 1 -- start start 2024-04-24T08:52:41.586 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.584+0000 7f7fd1200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7fcc0743a0 0x7f7fcc1a48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:41.586 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.584+0000 7f7fd1200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7fcc074cc0 0x7f7fcc1a4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:41.586 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.584+0000 7f7fd1200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7fcc112e10 0x7f7fcc19eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:41.586 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.584+0000 7f7fd1200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7fcc118700 con 0x7f7fcc112e10 2024-04-24T08:52:41.586 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.584+0000 7f7fd1200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7fcc118580 con 0x7f7fcc0743a0 2024-04-24T08:52:41.587 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.584+0000 7f7fd1200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7fcc118880 con 0x7f7fcc074cc0 2024-04-24T08:52:41.587 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.584+0000 7f7fcbe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7fcc112e10 0x7f7fcc19eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:41.587 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.584+0000 7f7fcbe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7fcc112e10 0x7f7fcc19eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41536/0 (socket says 172.21.15.83:41536) 2024-04-24T08:52:41.587 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.584+0000 7f7fcbe00640 1 -- 172.21.15.83:0/921497634 learned_addr learned my addr 172.21.15.83:0/921497634 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:52:41.587 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.585+0000 7f7fc2a00640 1 --2- 172.21.15.83:0/921497634 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7fcc074cc0 0x7f7fcc1a4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:41.587 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.585+0000 7f7fcb400640 1 --2- 172.21.15.83:0/921497634 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7fcc0743a0 0x7f7fcc1a48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:41.587 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.585+0000 7f7fcbe00640 1 -- 172.21.15.83:0/921497634 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7fcc074cc0 msgr2=0x7f7fcc1a4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:41.587 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.585+0000 7f7fcbe00640 1 --2- 172.21.15.83:0/921497634 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7fcc074cc0 0x7f7fcc1a4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:41.587 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.585+0000 7f7fcbe00640 1 -- 172.21.15.83:0/921497634 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7fcc0743a0 msgr2=0x7f7fcc1a48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:41.587 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.585+0000 7f7fcbe00640 1 --2- 172.21.15.83:0/921497634 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7fcc0743a0 0x7f7fcc1a48e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:41.587 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.585+0000 7f7fcbe00640 1 -- 172.21.15.83:0/921497634 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fcc19f260 con 0x7f7fcc112e10 2024-04-24T08:52:41.587 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.585+0000 7f7fc2a00640 1 --2- 172.21.15.83:0/921497634 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7fcc074cc0 0x7f7fcc1a4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:52:41.587 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.585+0000 7f7fcbe00640 1 --2- 172.21.15.83:0/921497634 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7fcc112e10 0x7f7fcc19eae0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f7fc400e9d0 tx=0x7f7fc400ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:41.587 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.585+0000 7f7fc8c00640 1 -- 172.21.15.83:0/921497634 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc400cd80 con 0x7f7fcc112e10 2024-04-24T08:52:41.588 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.585+0000 7f7fc8c00640 1 -- 172.21.15.83:0/921497634 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7fc4005020 con 0x7f7fcc112e10 2024-04-24T08:52:41.588 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.585+0000 7f7fd1200640 1 -- 172.21.15.83:0/921497634 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fcc19f550 con 0x7f7fcc112e10 2024-04-24T08:52:41.588 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.585+0000 7f7fc8c00640 1 -- 172.21.15.83:0/921497634 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc40116d0 con 0x7f7fcc112e10 2024-04-24T08:52:41.588 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.585+0000 7f7fd1200640 1 -- 172.21.15.83:0/921497634 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fcc1ab7a0 con 0x7f7fcc112e10 2024-04-24T08:52:41.589 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.589+0000 7f7fd5630740 1 Processor -- start 2024-04-24T08:52:41.589 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.589+0000 7f7fc8c00640 1 -- 172.21.15.83:0/921497634 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7f7fc40051d0 con 0x7f7fcc112e10 2024-04-24T08:52:41.589 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.589+0000 7f7fd5630740 1 -- start start 2024-04-24T08:52:41.590 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.589+0000 7f7fd5630740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563f567a33c0 0x563f567a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:41.590 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.589+0000 7f7fd5630740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563f567a3ce0 0x563f567ae060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:41.590 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.589+0000 7f7fd5630740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563f567ae5a0 0x563f567b0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:41.590 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.589+0000 7f7fd5630740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x563f56709900 con 0x563f567ae5a0 2024-04-24T08:52:41.590 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.589+0000 7f7fd5630740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563f566aff20 con 0x563f567a33c0 2024-04-24T08:52:41.590 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.589+0000 7f7fd5630740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x563f567058c0 con 0x563f567a3ce0 2024-04-24T08:52:41.592 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.590+0000 7f7fc2a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563f567ae5a0 0x563f567b0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:41.592 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.590+0000 7f7fc2a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563f567ae5a0 0x563f567b0990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41540/0 (socket says 172.21.15.83:41540) 2024-04-24T08:52:41.592 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.590+0000 7f7fc2a00640 1 -- 172.21.15.83:0/1455377532 learned_addr learned my addr 172.21.15.83:0/1455377532 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:52:41.592 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.590+0000 7f7fcb400640 1 --2- 172.21.15.83:0/1455377532 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563f567a3ce0 0x563f567ae060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:41.593 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.590+0000 7f7fc2a00640 1 --2- 172.21.15.83:0/1455377532 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563f567a33c0 0x563f567a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:41.593 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.590+0000 7f7fc2a00640 1 -- 172.21.15.83:0/1455377532 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563f567a3ce0 msgr2=0x563f567ae060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:41.593 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.590+0000 7f7fc2a00640 1 --2- 172.21.15.83:0/1455377532 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563f567a3ce0 0x563f567ae060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:41.593 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.590+0000 7f7fc2a00640 1 -- 172.21.15.83:0/1455377532 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563f567a33c0 msgr2=0x563f567a37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:41.593 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.590+0000 7f7fc2a00640 1 --2- 172.21.15.83:0/1455377532 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563f567a33c0 0x563f567a37a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:41.593 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.590+0000 7f7fc2a00640 1 -- 172.21.15.83:0/1455377532 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f566d4e10 con 0x563f567ae5a0 2024-04-24T08:52:41.593 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.590+0000 7f7fc8c00640 1 --2- 172.21.15.83:0/921497634 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f7fb00a3e30 0x7f7fb00a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:41.593 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.590+0000 7f7fc2a00640 1 --2- 172.21.15.83:0/1455377532 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563f567ae5a0 0x563f567b0990 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f7fb400de20 tx=0x7f7fb400d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db5b2ce110558d88 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:41.593 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.590+0000 7f7fc8c00640 1 -- 172.21.15.83:0/921497634 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f7fc40d1460 con 0x7f7fcc112e10 2024-04-24T08:52:41.593 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.591+0000 7f7fc0c00640 1 -- 172.21.15.83:0/1455377532 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fb4012650 con 0x563f567ae5a0 2024-04-24T08:52:41.593 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.591+0000 7f7fcb400640 1 --2- 172.21.15.83:0/921497634 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f7fb00a3e30 0x7f7fb00a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:41.593 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.591+0000 7f7fc0c00640 1 -- 172.21.15.83:0/1455377532 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7fb4012ca0 con 0x563f567ae5a0 2024-04-24T08:52:41.593 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.591+0000 7f7fc0c00640 1 -- 172.21.15.83:0/1455377532 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fb4011dc0 con 0x563f567ae5a0 2024-04-24T08:52:41.594 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.591+0000 7f7fcb400640 1 --2- 172.21.15.83:0/921497634 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f7fb00a3e30 0x7f7fb00a62f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x563f567a4960 tx=0x7f7fbc031400 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:41.594 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.591+0000 7f7fd5630740 1 -- 172.21.15.83:0/1455377532 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563f567ae5a0 msgr2=0x563f567b0990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:41.594 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.591+0000 7f7fd5630740 1 --2- 172.21.15.83:0/1455377532 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563f567ae5a0 0x563f567b0990 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f7fb400de20 tx=0x7f7fb400d560 comp rx=0 tx=0).stop 2024-04-24T08:52:41.594 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.592+0000 7f7fd5630740 1 -- 172.21.15.83:0/1455377532 shutdown_connections 2024-04-24T08:52:41.594 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.592+0000 7f7fd5630740 1 --2- 172.21.15.83:0/1455377532 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563f567ae5a0 0x563f567b0990 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:41.594 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.592+0000 7f7fd5630740 1 --2- 172.21.15.83:0/1455377532 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563f567a3ce0 0x563f567ae060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:41.594 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.592+0000 7f7fd5630740 1 --2- 172.21.15.83:0/1455377532 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563f567a33c0 0x563f567a37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:41.594 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.592+0000 7f7fd5630740 1 -- 172.21.15.83:0/1455377532 >> 172.21.15.83:0/1455377532 conn(0x563f5679f8d0 msgr2=0x563f5679fcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:41.594 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.592+0000 7f7fd5630740 1 -- 172.21.15.83:0/1455377532 shutdown_connections 2024-04-24T08:52:41.594 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.592+0000 7f7fd5630740 1 -- 172.21.15.83:0/1455377532 wait complete. 2024-04-24T08:52:41.594 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.592+0000 7f7fd5630740 1 -- start start 2024-04-24T08:52:41.594 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.592+0000 7f7fd5630740 1 Processor -- start 2024-04-24T08:52:41.597 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.593+0000 7f7fd5630740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563f567a33c0 0x563f567ab610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:41.597 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.593+0000 7f7fd5630740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563f567a3ce0 0x563f567abb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:41.597 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.593+0000 7f7fd5630740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563f567ae5a0 0x563f567ac090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:41.597 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.593+0000 7f7fd5630740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x563f567b10e0 con 0x563f567a3ce0 2024-04-24T08:52:41.597 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.593+0000 7f7fd5630740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563f566c32c0 con 0x563f567ae5a0 2024-04-24T08:52:41.598 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.593+0000 7f7fd5630740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x563f567b1260 con 0x563f567a33c0 2024-04-24T08:52:41.598 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.593+0000 7f7fd5630740 10 client.0 _ll_register_callbacks cb 0x563f5641ba00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:52:41.598 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.593+0000 7f7fc2a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563f567a3ce0 0x563f567abb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:41.598 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.593+0000 7f7fc2a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563f567a3ce0 0x563f567abb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41552/0 (socket says 172.21.15.83:41552) 2024-04-24T08:52:41.598 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.593+0000 7f7fc2a00640 1 -- 172.21.15.83:0/3539466340 learned_addr learned my addr 172.21.15.83:0/3539466340 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:52:41.598 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.593+0000 7f7fc2a00640 1 -- 172.21.15.83:0/3539466340 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563f567a33c0 msgr2=0x563f567ab610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:41.598 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.593+0000 7f7fc2a00640 1 --2- 172.21.15.83:0/3539466340 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563f567a33c0 0x563f567ab610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:41.598 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.593+0000 7f7fc2a00640 1 -- 172.21.15.83:0/3539466340 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563f567ae5a0 msgr2=0x563f567ac090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:41.598 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.593+0000 7f7fcbe00640 1 --2- 172.21.15.83:0/3539466340 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563f567ae5a0 0x563f567ac090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:41.598 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.593+0000 7f7fc2a00640 1 --2- 172.21.15.83:0/3539466340 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563f567ae5a0 0x563f567ac090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:41.598 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.593+0000 7f7fc2a00640 1 -- 172.21.15.83:0/3539466340 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f566d3fd0 con 0x563f567a3ce0 2024-04-24T08:52:41.598 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.594+0000 7f7fc2a00640 1 --2- 172.21.15.83:0/3539466340 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563f567a3ce0 0x563f567abb50 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f7fb400d3d0 tx=0x7f7fb40118f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a17ff118a75ded6e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:41.598 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.594+0000 7f7fb9600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:52:41.598 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.594+0000 7f7fb9600640 1 -- 172.21.15.83:0/3539466340 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fb4011aa0 con 0x563f567a3ce0 2024-04-24T08:52:41.599 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.594+0000 7f7fb9600640 1 -- 172.21.15.83:0/3539466340 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f7fb4011c40 con 0x563f567a3ce0 2024-04-24T08:52:41.599 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.594+0000 7f7fd5630740 1 -- 172.21.15.83:0/3539466340 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7fb4012ca0 con 0x563f567a3ce0 2024-04-24T08:52:41.599 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.594+0000 7f7fb9600640 1 -- 172.21.15.83:0/3539466340 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7fb4032050 con 0x563f567a3ce0 2024-04-24T08:52:41.599 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.594+0000 7f7fd5630740 10 client.16059 fetch_fsmap learned FSMap version 28 2024-04-24T08:52:41.599 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.594+0000 7f7fd5630740 1 -- 172.21.15.83:0/3539466340 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563f5641c210 con 0x563f567a3ce0 2024-04-24T08:52:41.599 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.595+0000 7f7fb9600640 1 -- 172.21.15.83:0/3539466340 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7fb4012ca0 con 0x563f567a3ce0 2024-04-24T08:52:41.599 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.595+0000 7f7fb9600640 1 -- 172.21.15.83:0/3539466340 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7fb4024670 con 0x563f567a3ce0 2024-04-24T08:52:41.599 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.595+0000 7f7fd5630740 10 client.16059 fetch_fsmap finished waiting for FSMap version 28 2024-04-24T08:52:41.599 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.595+0000 7f7fd5630740 10 client.16059 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T08:52:41.599 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.595+0000 7f7fd5630740 20 client.16059 populate_metadata read hostname 'smithi083' 2024-04-24T08:52:41.599 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.595+0000 7f7fd5630740 1 --2- 172.21.15.83:0/3539466340 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x563f567be770 0x563f567c0b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:41.599 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.595+0000 7f7fd5630740 4 client.16059 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:52:41.599 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.595+0000 7f7fd5630740 1 -- 172.21.15.83:0/3539466340 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563f56717320 con 0x563f567be770 2024-04-24T08:52:41.599 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.596+0000 7f7fcb400640 1 --2- 172.21.15.83:0/3539466340 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x563f567be770 0x563f567c0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:41.600 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.597+0000 7f7fb9600640 10 client.16059 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:52:41.600 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.597+0000 7f7fcb400640 1 --2- 172.21.15.83:0/3539466340 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x563f567be770 0x563f567c0b60 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c29edcac8cdb677c server_cookie=d43916a28f191ca6 in_seq=0 out_seq=0 2024-04-24T08:52:41.601 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.598+0000 7f7fb9600640 1 -- 172.21.15.83:0/3539466340 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563f56717320 con 0x563f567be770 2024-04-24T08:52:41.601 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.598+0000 7f7fb9600640 10 client.16059 handle_command_reply: tid=0 2024-04-24T08:52:41.601 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.598+0000 7f7fd5630740 1 client.16059 shutdown 2024-04-24T08:52:41.601 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.599+0000 7f7fd5630740 1 -- 172.21.15.83:0/3539466340 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563f567a3ce0 msgr2=0x563f567abb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:41.601 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.599+0000 7f7fd5630740 1 --2- 172.21.15.83:0/3539466340 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563f567a3ce0 0x563f567abb50 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f7fb400d3d0 tx=0x7f7fb40118f0 comp rx=0 tx=0).stop 2024-04-24T08:52:41.601 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.599+0000 7f7fd5630740 1 -- 172.21.15.83:0/3539466340 shutdown_connections 2024-04-24T08:52:41.601 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.599+0000 7f7fd5630740 1 --2- 172.21.15.83:0/3539466340 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x563f567be770 0x563f567c0b60 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:41.601 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.599+0000 7f7fd5630740 1 --2- 172.21.15.83:0/3539466340 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563f567ae5a0 0x563f567ac090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:41.602 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.599+0000 7f7fd5630740 1 --2- 172.21.15.83:0/3539466340 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563f567a3ce0 0x563f567abb50 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:41.602 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.599+0000 7f7fd5630740 1 --2- 172.21.15.83:0/3539466340 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563f567a33c0 0x563f567ab610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:41.602 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.599+0000 7f7fd5630740 1 -- 172.21.15.83:0/3539466340 >> 172.21.15.83:0/3539466340 conn(0x563f5679f8d0 msgr2=0x563f567a5600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:41.602 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.599+0000 7f7fb9600640 0 client.16059 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:52:41.602 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.599+0000 7f7fd5630740 1 -- 172.21.15.83:0/3539466340 shutdown_connections 2024-04-24T08:52:41.602 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.599+0000 7f7fd5630740 1 -- 172.21.15.83:0/3539466340 wait complete. 2024-04-24T08:52:41.602 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.599+0000 7f7fd5630740 20 client.16059 trim_cache size 0 max 16384 2024-04-24T08:52:41.620 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.620+0000 7f7fd5630740 1 Processor -- start 2024-04-24T08:52:41.620 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.620+0000 7f7fd5630740 1 -- start start 2024-04-24T08:52:41.620 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.620+0000 7f7fd5630740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563f567a33c0 0x563f567add40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:41.620 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.620+0000 7f7fd5630740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563f567a3ce0 0x563f5679f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:41.620 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.620+0000 7f7fd5630740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563f567ae5a0 0x563f567a6920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:41.621 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.620+0000 7f7fd5630740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7fcc1b5950 con 0x563f567a33c0 2024-04-24T08:52:41.621 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.620+0000 7f7fd5630740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7fcc1b7490 con 0x563f567ae5a0 2024-04-24T08:52:41.621 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.620+0000 7f7fd5630740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7fcc1b6020 con 0x563f567a3ce0 2024-04-24T08:52:41.622 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.620+0000 7f7fcb400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563f567a33c0 0x563f567add40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:41.622 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.621+0000 7f7fcb400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563f567a33c0 0x563f567add40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41566/0 (socket says 172.21.15.83:41566) 2024-04-24T08:52:41.622 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.621+0000 7f7fcb400640 1 -- 172.21.15.83:0/4128572715 learned_addr learned my addr 172.21.15.83:0/4128572715 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:52:41.622 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.621+0000 7f7fc2a00640 1 --2- 172.21.15.83:0/4128572715 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563f567a3ce0 0x563f5679f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:41.622 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.621+0000 7f7fcbe00640 1 --2- 172.21.15.83:0/4128572715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563f567ae5a0 0x563f567a6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:41.622 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.621+0000 7f7fcb400640 1 -- 172.21.15.83:0/4128572715 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563f567a3ce0 msgr2=0x563f5679f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:41.622 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.621+0000 7f7fcb400640 1 --2- 172.21.15.83:0/4128572715 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563f567a3ce0 0x563f5679f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:41.622 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.621+0000 7f7fcb400640 1 -- 172.21.15.83:0/4128572715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563f567ae5a0 msgr2=0x563f567a6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:41.626 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.621+0000 7f7fcb400640 1 --2- 172.21.15.83:0/4128572715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563f567ae5a0 0x563f567a6920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:41.626 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.621+0000 7f7fcb400640 1 -- 172.21.15.83:0/4128572715 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f564e8620 con 0x563f567a33c0 2024-04-24T08:52:41.626 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.621+0000 7f7fcb400640 1 --2- 172.21.15.83:0/4128572715 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563f567a33c0 0x563f567add40 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc009230 tx=0x7f7fbc009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4d1be0558e128e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:41.626 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.622+0000 7f7fbbe00640 1 -- 172.21.15.83:0/4128572715 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fbc035050 con 0x563f567a33c0 2024-04-24T08:52:41.626 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.622+0000 7f7fbbe00640 1 -- 172.21.15.83:0/4128572715 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7fbc052030 con 0x563f567a33c0 2024-04-24T08:52:41.626 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.622+0000 7f7fbbe00640 1 -- 172.21.15.83:0/4128572715 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fbc003ab0 con 0x563f567a33c0 2024-04-24T08:52:41.629 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.622+0000 7f7fd5630740 1 -- 172.21.15.83:0/4128572715 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563f567a33c0 msgr2=0x563f567add40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:41.629 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.622+0000 7f7fd5630740 1 --2- 172.21.15.83:0/4128572715 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563f567a33c0 0x563f567add40 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc009230 tx=0x7f7fbc009260 comp rx=0 tx=0).stop 2024-04-24T08:52:41.629 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.622+0000 7f7fd5630740 1 -- 172.21.15.83:0/4128572715 shutdown_connections 2024-04-24T08:52:41.629 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.622+0000 7f7fd5630740 1 --2- 172.21.15.83:0/4128572715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563f567ae5a0 0x563f567a6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:41.629 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.622+0000 7f7fd5630740 1 --2- 172.21.15.83:0/4128572715 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563f567a3ce0 0x563f5679f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:41.629 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.622+0000 7f7fd5630740 1 --2- 172.21.15.83:0/4128572715 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563f567a33c0 0x563f567add40 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:41.629 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.622+0000 7f7fd5630740 1 -- 172.21.15.83:0/4128572715 >> 172.21.15.83:0/4128572715 conn(0x563f5679f8d0 msgr2=0x563f567abe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:41.630 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.623+0000 7f7fd5630740 1 -- 172.21.15.83:0/4128572715 shutdown_connections 2024-04-24T08:52:41.630 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.623+0000 7f7fd5630740 1 -- 172.21.15.83:0/4128572715 wait complete. 2024-04-24T08:52:41.630 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.623+0000 7f7fd5630740 1 -- start start 2024-04-24T08:52:41.630 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.623+0000 7f7fd5630740 1 Processor -- start 2024-04-24T08:52:41.630 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.624+0000 7f7fd5630740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563f567a33c0 0x563f567b6250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:41.630 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.624+0000 7f7fd5630740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563f567a3ce0 0x563f567b6790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:41.630 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.624+0000 7f7fd5630740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563f567ae5a0 0x563f567b1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:41.630 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.624+0000 7f7fd5630740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7fcc1b6dc0 con 0x563f567a33c0 2024-04-24T08:52:41.630 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.624+0000 7f7fd5630740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7fcc1b66f0 con 0x563f567a3ce0 2024-04-24T08:52:41.630 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.624+0000 7f7fd5630740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7fcc1b3930 con 0x563f567ae5a0 2024-04-24T08:52:41.630 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.624+0000 7f7fd5630740 10 client.0 _ll_register_callbacks cb 0x7f7fb400dc50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:52:41.630 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.624+0000 7f7fcb400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563f567a33c0 0x563f567b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:41.630 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.624+0000 7f7fcb400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563f567a33c0 0x563f567b6250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41582/0 (socket says 172.21.15.83:41582) 2024-04-24T08:52:41.631 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.624+0000 7f7fcb400640 1 -- 172.21.15.83:0/3078445588 learned_addr learned my addr 172.21.15.83:0/3078445588 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:52:41.631 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.624+0000 7f7fc2a00640 1 --2- 172.21.15.83:0/3078445588 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563f567a3ce0 0x563f567b6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:41.631 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.624+0000 7f7fcbe00640 1 --2- 172.21.15.83:0/3078445588 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563f567ae5a0 0x563f567b1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:41.631 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.624+0000 7f7fcb400640 1 -- 172.21.15.83:0/3078445588 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563f567ae5a0 msgr2=0x563f567b1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:41.631 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.624+0000 7f7fcb400640 1 --2- 172.21.15.83:0/3078445588 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563f567ae5a0 0x563f567b1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:41.631 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.624+0000 7f7fcb400640 1 -- 172.21.15.83:0/3078445588 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563f567a3ce0 msgr2=0x563f567b6790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:41.631 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.624+0000 7f7fcb400640 1 --2- 172.21.15.83:0/3078445588 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563f567a3ce0 0x563f567b6790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:41.631 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.624+0000 7f7fcb400640 1 -- 172.21.15.83:0/3078445588 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f567b1ec0 con 0x563f567a33c0 2024-04-24T08:52:41.631 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.625+0000 7f7fcb400640 1 --2- 172.21.15.83:0/3078445588 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563f567a33c0 0x563f567b6250 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc02a9a0 tx=0x7f7fbc003830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dea7286f6606339b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:41.631 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.625+0000 7f7fb9600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:52:41.631 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.625+0000 7f7fb9600640 1 -- 172.21.15.83:0/3078445588 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fbc035070 con 0x563f567a33c0 2024-04-24T08:52:41.631 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.625+0000 7f7fb9600640 1 -- 172.21.15.83:0/3078445588 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f7fbc0039e0 con 0x563f567a33c0 2024-04-24T08:52:41.631 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.625+0000 7f7fd5630740 1 -- 172.21.15.83:0/3078445588 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7fbc052030 con 0x563f567a33c0 2024-04-24T08:52:41.631 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.625+0000 7f7fb9600640 1 -- 172.21.15.83:0/3078445588 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7fbc003ce0 con 0x563f567a33c0 2024-04-24T08:52:41.632 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.625+0000 7f7fd5630740 10 client.16065 fetch_fsmap learned FSMap version 28 2024-04-24T08:52:41.632 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.625+0000 7f7fd5630740 1 -- 172.21.15.83:0/3078445588 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563f568244a0 con 0x563f567a33c0 2024-04-24T08:52:41.632 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.626+0000 7f7fb9600640 1 -- 172.21.15.83:0/3078445588 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7fbc052030 con 0x563f567a33c0 2024-04-24T08:52:41.632 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.626+0000 7f7fb9600640 1 -- 172.21.15.83:0/3078445588 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7fbc0324b0 con 0x563f567a33c0 2024-04-24T08:52:41.632 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.626+0000 7f7fd5630740 10 client.16065 fetch_fsmap finished waiting for FSMap version 28 2024-04-24T08:52:41.632 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.626+0000 7f7fd5630740 10 client.16065 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T08:52:41.632 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.626+0000 7f7fd5630740 20 client.16065 populate_metadata read hostname 'smithi083' 2024-04-24T08:52:41.632 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.626+0000 7f7fd5630740 1 --2- 172.21.15.83:0/3078445588 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x563f56824b60 0x563f56826f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:41.632 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.626+0000 7f7fd5630740 4 client.16065 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T08:52:41.632 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.626+0000 7f7fd5630740 1 -- 172.21.15.83:0/3078445588 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563f56711190 con 0x563f56824b60 2024-04-24T08:52:41.632 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.627+0000 7f7fc2a00640 1 --2- 172.21.15.83:0/3078445588 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x563f56824b60 0x563f56826f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:41.632 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.627+0000 7f7fc2a00640 1 --2- 172.21.15.83:0/3078445588 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x563f56824b60 0x563f56826f80 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0d9baf869a0ac76 server_cookie=594f7fb1ff5461a9 in_seq=0 out_seq=0 2024-04-24T08:52:41.632 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.627+0000 7f7fb9600640 10 client.16065 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:52:41.639 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.638+0000 7f7fb9600640 1 -- 172.21.15.83:0/3078445588 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563f56711190 con 0x563f56824b60 2024-04-24T08:52:41.639 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.638+0000 7f7fb9600640 10 client.16065 handle_command_reply: tid=0 2024-04-24T08:52:41.639 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.638+0000 7f7fd5630740 1 client.16065 shutdown 2024-04-24T08:52:41.639 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.638+0000 7f7fd5630740 1 -- 172.21.15.83:0/3078445588 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563f567a33c0 msgr2=0x563f567b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:41.639 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.638+0000 7f7fd5630740 1 --2- 172.21.15.83:0/3078445588 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563f567a33c0 0x563f567b6250 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc02a9a0 tx=0x7f7fbc003830 comp rx=0 tx=0).stop 2024-04-24T08:52:41.639 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.638+0000 7f7fd5630740 1 -- 172.21.15.83:0/3078445588 shutdown_connections 2024-04-24T08:52:41.640 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.638+0000 7f7fd5630740 1 --2- 172.21.15.83:0/3078445588 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x563f56824b60 0x563f56826f80 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:41.640 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.638+0000 7f7fd5630740 1 --2- 172.21.15.83:0/3078445588 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563f567ae5a0 0x563f567b1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:41.640 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.638+0000 7f7fb9600640 0 client.16065 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:52:41.640 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.638+0000 7f7fd5630740 1 --2- 172.21.15.83:0/3078445588 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563f567a3ce0 0x563f567b6790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:41.640 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.638+0000 7f7fd5630740 1 --2- 172.21.15.83:0/3078445588 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563f567a33c0 0x563f567b6250 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:41.640 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.638+0000 7f7fd5630740 1 -- 172.21.15.83:0/3078445588 >> 172.21.15.83:0/3078445588 conn(0x563f5679f8d0 msgr2=0x563f567accc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:41.640 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.638+0000 7f7fd5630740 1 -- 172.21.15.83:0/3078445588 shutdown_connections 2024-04-24T08:52:41.640 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.639+0000 7f7fd5630740 1 -- 172.21.15.83:0/3078445588 wait complete. 2024-04-24T08:52:41.640 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.639+0000 7f7fd5630740 20 client.16065 trim_cache size 0 max 16384 2024-04-24T08:52:41.641 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T08:52:41.641 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T08:52:41.641 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T08:52:41.641 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T08:52:41.642 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.639+0000 7f7fd1200640 1 -- 172.21.15.83:0/921497634 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f7fb00a3e30 msgr2=0x7f7fb00a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:41.642 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.639+0000 7f7fd1200640 1 --2- 172.21.15.83:0/921497634 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f7fb00a3e30 0x7f7fb00a62f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x563f567a4960 tx=0x7f7fbc031400 comp rx=0 tx=0).stop 2024-04-24T08:52:41.642 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.639+0000 7f7fd1200640 1 -- 172.21.15.83:0/921497634 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7fcc112e10 msgr2=0x7f7fcc19eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:41.642 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.639+0000 7f7fd1200640 1 --2- 172.21.15.83:0/921497634 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7fcc112e10 0x7f7fcc19eae0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f7fc400e9d0 tx=0x7f7fc400ee90 comp rx=0 tx=0).stop 2024-04-24T08:52:41.643 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.640+0000 7f7fd1200640 1 -- 172.21.15.83:0/921497634 shutdown_connections 2024-04-24T08:52:41.643 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.640+0000 7f7fd1200640 1 --2- 172.21.15.83:0/921497634 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f7fb00a3e30 0x7f7fb00a62f0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:41.643 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.640+0000 7f7fd1200640 1 --2- 172.21.15.83:0/921497634 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7fcc112e10 0x7f7fcc19eae0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:41.643 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.640+0000 7f7fd1200640 1 --2- 172.21.15.83:0/921497634 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7fcc074cc0 0x7f7fcc1a4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:41.643 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.640+0000 7f7fd1200640 1 --2- 172.21.15.83:0/921497634 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7fcc0743a0 0x7f7fcc1a48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:41.643 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.640+0000 7f7fd1200640 1 -- 172.21.15.83:0/921497634 >> 172.21.15.83:0/921497634 conn(0x7f7fcc0fd950 msgr2=0x7f7fcc0ff250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:41.643 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.640+0000 7f7fd1200640 1 -- 172.21.15.83:0/921497634 shutdown_connections 2024-04-24T08:52:41.643 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:41.640+0000 7f7fd1200640 1 -- 172.21.15.83:0/921497634 wait complete. 2024-04-24T08:52:41.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:52:41 smithi138 ceph-mon[31441]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-24T08:52:41.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:52:41 smithi189 ceph-mon[31791]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-24T08:52:42.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:52:41 smithi083 ceph-mon[27907]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-24T08:52:43.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:52:43 smithi138 ceph-mon[31441]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-24T08:52:43.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:52:43 smithi189 ceph-mon[31791]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-24T08:52:44.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:52:43 smithi083 ceph-mon[27907]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-24T08:52:45.954 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:52:45 smithi138 ceph-mon[31441]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 511 B/s wr, 12 op/s 2024-04-24T08:52:45.956 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:52:45 smithi189 ceph-mon[31791]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 511 B/s wr, 12 op/s 2024-04-24T08:52:46.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:52:45 smithi083 ceph-mon[27907]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 511 B/s wr, 12 op/s 2024-04-24T08:52:46.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:52:46 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:52:46.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:52:46 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:52:47.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:52:46 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:52:47.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:52:47 smithi138 ceph-mon[31441]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-24T08:52:47.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:52:47 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:52:47.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:52:47 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:52:47.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:52:47 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T08:52:47.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:52:47 smithi189 ceph-mon[31791]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-24T08:52:47.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:52:47 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:52:47.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:52:47 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:52:47.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:52:47 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T08:52:48.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:52:47 smithi083 ceph-mon[27907]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-24T08:52:48.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:52:47 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:52:48.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:52:47 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:52:48.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:52:47 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T08:52:49.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:52:49 smithi138 ceph-mon[31441]: pgmap v39: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-24T08:52:49.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:52:49 smithi189 ceph-mon[31791]: pgmap v39: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-24T08:52:50.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:52:49 smithi083 ceph-mon[27907]: pgmap v39: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-24T08:52:50.445 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T08:52:50.445 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T08:52:50.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.627+0000 7fe8aea00640 1 -- 172.21.15.83:0/3463022875 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8a8102240 msgr2=0x7fe8a8102620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:50.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.627+0000 7fe8aea00640 1 --2- 172.21.15.83:0/3463022875 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8a8102240 0x7fe8a8102620 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fe898009900 tx=0x7fe89802a410 comp rx=0 tx=0).stop 2024-04-24T08:52:50.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.627+0000 7fe8aea00640 1 -- 172.21.15.83:0/3463022875 shutdown_connections 2024-04-24T08:52:50.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.627+0000 7fe8aea00640 1 --2- 172.21.15.83:0/3463022875 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe8a8107310 0x7fe8a810eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:50.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.627+0000 7fe8aea00640 1 --2- 172.21.15.83:0/3463022875 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe8a8102bf0 0x7fe8a8106c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:50.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.627+0000 7fe8aea00640 1 --2- 172.21.15.83:0/3463022875 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8a8102240 0x7fe8a8102620 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:50.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.627+0000 7fe8aea00640 1 -- 172.21.15.83:0/3463022875 >> 172.21.15.83:0/3463022875 conn(0x7fe8a8079450 msgr2=0x7fe8a8079860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:50.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.628+0000 7fe8aea00640 1 -- 172.21.15.83:0/3463022875 shutdown_connections 2024-04-24T08:52:50.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.628+0000 7fe8aea00640 1 -- 172.21.15.83:0/3463022875 wait complete. 2024-04-24T08:52:50.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.628+0000 7fe8aea00640 1 Processor -- start 2024-04-24T08:52:50.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.629+0000 7fe8aea00640 1 -- start start 2024-04-24T08:52:50.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.629+0000 7fe8aea00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe8a8102240 0x7fe8a8074300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:50.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.629+0000 7fe8aea00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe8a8107310 0x7fe8a8074840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:50.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.629+0000 7fe8aea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8a8074d80 0x7fe8a81a6a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:50.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.629+0000 7fe8aea00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a81122d0 con 0x7fe8a8074d80 2024-04-24T08:52:50.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.629+0000 7fe8aea00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a8112150 con 0x7fe8a8107310 2024-04-24T08:52:50.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.629+0000 7fe8aea00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a8112450 con 0x7fe8a8102240 2024-04-24T08:52:50.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.630+0000 7fe8aca00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8a8074d80 0x7fe8a81a6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:50.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.630+0000 7fe8aca00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8a8074d80 0x7fe8a81a6a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54360/0 (socket says 172.21.15.83:54360) 2024-04-24T08:52:50.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.630+0000 7fe8aca00640 1 -- 172.21.15.83:0/1930728342 learned_addr learned my addr 172.21.15.83:0/1930728342 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:52:50.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.630+0000 7fe8a7e00640 1 --2- 172.21.15.83:0/1930728342 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe8a8102240 0x7fe8a8074300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:50.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.630+0000 7fe8a7400640 1 --2- 172.21.15.83:0/1930728342 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe8a8107310 0x7fe8a8074840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:50.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.630+0000 7fe8aca00640 1 -- 172.21.15.83:0/1930728342 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe8a8102240 msgr2=0x7fe8a8074300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:50.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.630+0000 7fe8aca00640 1 --2- 172.21.15.83:0/1930728342 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe8a8102240 0x7fe8a8074300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:50.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.630+0000 7fe8aca00640 1 -- 172.21.15.83:0/1930728342 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe8a8107310 msgr2=0x7fe8a8074840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:50.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.630+0000 7fe8aca00640 1 --2- 172.21.15.83:0/1930728342 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe8a8107310 0x7fe8a8074840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:50.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.630+0000 7fe8aca00640 1 -- 172.21.15.83:0/1930728342 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8a81a7100 con 0x7fe8a8074d80 2024-04-24T08:52:50.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.630+0000 7fe8aca00640 1 --2- 172.21.15.83:0/1930728342 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8a8074d80 0x7fe8a81a6a00 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fe8a000eaa0 tx=0x7fe8a000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:50.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.631+0000 7fe8a4c00640 1 -- 172.21.15.83:0/1930728342 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8a000cdb0 con 0x7fe8a8074d80 2024-04-24T08:52:50.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.631+0000 7fe8a4c00640 1 -- 172.21.15.83:0/1930728342 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe8a0005020 con 0x7fe8a8074d80 2024-04-24T08:52:50.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.631+0000 7fe8aea00640 1 -- 172.21.15.83:0/1930728342 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8a81a73f0 con 0x7fe8a8074d80 2024-04-24T08:52:50.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.631+0000 7fe8a4c00640 1 -- 172.21.15.83:0/1930728342 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8a00116d0 con 0x7fe8a8074d80 2024-04-24T08:52:50.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.631+0000 7fe8aea00640 1 -- 172.21.15.83:0/1930728342 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8a81a7940 con 0x7fe8a8074d80 2024-04-24T08:52:50.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.635+0000 7fe8b17fd740 1 Processor -- start 2024-04-24T08:52:50.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.636+0000 7fe8a4c00640 1 -- 172.21.15.83:0/1930728342 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7fe8a00051d0 con 0x7fe8a8074d80 2024-04-24T08:52:50.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.636+0000 7fe8b17fd740 1 -- start start 2024-04-24T08:52:50.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.637+0000 7fe8b17fd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5566ff0fe860 0x5566ff0fec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:50.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.637+0000 7fe8b17fd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5566ff0ff180 0x5566ff109310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:50.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.637+0000 7fe8b17fd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5566ff109850 0x5566ff10bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:50.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.637+0000 7fe8b17fd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5566ff0641f0 con 0x5566ff0ff180 2024-04-24T08:52:50.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.637+0000 7fe8b17fd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5566ff00a350 con 0x5566ff0fe860 2024-04-24T08:52:50.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.637+0000 7fe8b17fd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5566ff0601b0 con 0x5566ff109850 2024-04-24T08:52:50.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.637+0000 7fe8a7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5566ff0ff180 0x5566ff109310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:50.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.637+0000 7fe8a7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5566ff0ff180 0x5566ff109310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54374/0 (socket says 172.21.15.83:54374) 2024-04-24T08:52:50.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.637+0000 7fe8a7e00640 1 -- 172.21.15.83:0/754480713 learned_addr learned my addr 172.21.15.83:0/754480713 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:52:50.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.637+0000 7fe8a7400640 1 --2- 172.21.15.83:0/754480713 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5566ff0fe860 0x5566ff0fec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:50.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.638+0000 7fe8a7400640 1 --2- 172.21.15.83:0/754480713 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5566ff109850 0x5566ff10bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:50.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.638+0000 7fe8a7e00640 1 -- 172.21.15.83:0/754480713 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5566ff109850 msgr2=0x5566ff10bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:50.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.638+0000 7fe8a7e00640 1 --2- 172.21.15.83:0/754480713 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5566ff109850 0x5566ff10bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:50.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.638+0000 7fe8a7e00640 1 -- 172.21.15.83:0/754480713 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5566ff0fe860 msgr2=0x5566ff0fec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:50.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.638+0000 7fe8a7e00640 1 --2- 172.21.15.83:0/754480713 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5566ff0fe860 0x5566ff0fec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:50.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.638+0000 7fe8a7e00640 1 -- 172.21.15.83:0/754480713 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566ff05ef50 con 0x5566ff0ff180 2024-04-24T08:52:50.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.638+0000 7fe8a4c00640 1 --2- 172.21.15.83:0/1930728342 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe88c0a3e70 0x7fe88c0a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:50.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.638+0000 7fe8a4c00640 1 -- 172.21.15.83:0/1930728342 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7fe8a00d10a0 con 0x7fe8a8074d80 2024-04-24T08:52:50.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.638+0000 7fe8a7e00640 1 --2- 172.21.15.83:0/754480713 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5566ff0ff180 0x5566ff109310 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fe898009900 tx=0x7fe898004170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bd100bffdcee2df server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:50.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.639+0000 7fe8a7400640 1 --2- 172.21.15.83:0/1930728342 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe88c0a3e70 0x7fe88c0a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:50.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.639+0000 7fe89d200640 1 -- 172.21.15.83:0/754480713 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe898035030 con 0x5566ff0ff180 2024-04-24T08:52:50.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.639+0000 7fe89d200640 1 -- 172.21.15.83:0/754480713 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe8980054f0 con 0x5566ff0ff180 2024-04-24T08:52:50.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.639+0000 7fe89d200640 1 -- 172.21.15.83:0/754480713 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8980035f0 con 0x5566ff0ff180 2024-04-24T08:52:50.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.640+0000 7fe8b17fd740 1 -- 172.21.15.83:0/754480713 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5566ff0ff180 msgr2=0x5566ff109310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:50.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.640+0000 7fe8a7400640 1 --2- 172.21.15.83:0/1930728342 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe88c0a3e70 0x7fe88c0a6330 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x5566ff0fbd10 tx=0x7fe894004730 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:50.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.640+0000 7fe8b17fd740 1 --2- 172.21.15.83:0/754480713 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5566ff0ff180 0x5566ff109310 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fe898009900 tx=0x7fe898004170 comp rx=0 tx=0).stop 2024-04-24T08:52:50.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.640+0000 7fe8b17fd740 1 -- 172.21.15.83:0/754480713 shutdown_connections 2024-04-24T08:52:50.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.640+0000 7fe8b17fd740 1 --2- 172.21.15.83:0/754480713 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5566ff109850 0x5566ff10bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:50.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.640+0000 7fe8b17fd740 1 --2- 172.21.15.83:0/754480713 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5566ff0ff180 0x5566ff109310 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:50.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.640+0000 7fe8b17fd740 1 --2- 172.21.15.83:0/754480713 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5566ff0fe860 0x5566ff0fec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:50.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.640+0000 7fe8b17fd740 1 -- 172.21.15.83:0/754480713 >> 172.21.15.83:0/754480713 conn(0x5566ff0fab80 msgr2=0x5566ff0faf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:50.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.640+0000 7fe8b17fd740 1 -- 172.21.15.83:0/754480713 shutdown_connections 2024-04-24T08:52:50.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.641+0000 7fe8b17fd740 1 -- 172.21.15.83:0/754480713 wait complete. 2024-04-24T08:52:50.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.641+0000 7fe8b17fd740 1 -- start start 2024-04-24T08:52:50.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.641+0000 7fe8b17fd740 1 Processor -- start 2024-04-24T08:52:50.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.642+0000 7fe8b17fd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5566ff0fe860 0x5566ff106760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:50.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.642+0000 7fe8b17fd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5566ff0ff180 0x5566ff106ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:50.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.642+0000 7fe8b17fd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5566ff109850 0x5566ff1071e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:50.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.642+0000 7fe8b17fd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5566ff10c5e0 con 0x5566ff0fe860 2024-04-24T08:52:50.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.642+0000 7fe8b17fd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5566ff01d690 con 0x5566ff0ff180 2024-04-24T08:52:50.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.642+0000 7fe8b17fd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5566ff10c760 con 0x5566ff109850 2024-04-24T08:52:50.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.642+0000 7fe8b17fd740 10 client.0 _ll_register_callbacks cb 0x5566ff059650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:52:50.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.642+0000 7fe8a7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5566ff0fe860 0x5566ff106760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:50.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.642+0000 7fe8a7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5566ff0fe860 0x5566ff106760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54376/0 (socket says 172.21.15.83:54376) 2024-04-24T08:52:50.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.642+0000 7fe8a7400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5566ff0ff180 0x5566ff106ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:50.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.642+0000 7fe8a7e00640 1 -- 172.21.15.83:0/3656459127 learned_addr learned my addr 172.21.15.83:0/3656459127 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:52:50.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.643+0000 7fe8aca00640 1 --2- 172.21.15.83:0/3656459127 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5566ff109850 0x5566ff1071e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:50.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.643+0000 7fe8a7400640 1 -- 172.21.15.83:0/3656459127 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5566ff109850 msgr2=0x5566ff1071e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:50.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.643+0000 7fe8a7400640 1 --2- 172.21.15.83:0/3656459127 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5566ff109850 0x5566ff1071e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:50.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.643+0000 7fe8a7400640 1 -- 172.21.15.83:0/3656459127 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5566ff0fe860 msgr2=0x5566ff106760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:50.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.643+0000 7fe8a7400640 1 --2- 172.21.15.83:0/3656459127 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5566ff0fe860 0x5566ff106760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:50.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.643+0000 7fe8a7400640 1 -- 172.21.15.83:0/3656459127 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566ff01c920 con 0x5566ff0ff180 2024-04-24T08:52:50.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.644+0000 7fe8a7400640 1 --2- 172.21.15.83:0/3656459127 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5566ff0ff180 0x5566ff106ca0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe8a8075d90 tx=0x7fe8940046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b1e2151c1c3fbc4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:50.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.644+0000 7fe891600640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T08:52:50.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.644+0000 7fe891600640 1 -- 172.21.15.83:0/3656459127 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe894024070 con 0x5566ff0ff180 2024-04-24T08:52:50.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.644+0000 7fe8b17fd740 1 -- 172.21.15.83:0/3656459127 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8980054f0 con 0x5566ff0ff180 2024-04-24T08:52:50.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.644+0000 7fe891600640 1 -- 172.21.15.83:0/3656459127 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7fe894010dc0 con 0x5566ff0ff180 2024-04-24T08:52:50.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.645+0000 7fe891600640 1 -- 172.21.15.83:0/3656459127 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe894029460 con 0x5566ff0ff180 2024-04-24T08:52:50.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.645+0000 7fe8b17fd740 10 client.35327 fetch_fsmap learned FSMap version 28 2024-04-24T08:52:50.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.645+0000 7fe8b17fd740 1 -- 172.21.15.83:0/3656459127 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5566fee407d0 con 0x5566ff0ff180 2024-04-24T08:52:50.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.645+0000 7fe891600640 1 -- 172.21.15.83:0/3656459127 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe8980054f0 con 0x5566ff0ff180 2024-04-24T08:52:50.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.646+0000 7fe891600640 1 -- 172.21.15.83:0/3656459127 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe894020580 con 0x5566ff0ff180 2024-04-24T08:52:50.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.646+0000 7fe8b17fd740 10 client.35327 fetch_fsmap finished waiting for FSMap version 28 2024-04-24T08:52:50.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.646+0000 7fe8b17fd740 10 client.35327 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:52:50.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.646+0000 7fe8b17fd740 20 client.35327 populate_metadata read hostname 'smithi083' 2024-04-24T08:52:50.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.646+0000 7fe8b17fd740 1 --2- 172.21.15.83:0/3656459127 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5566ff119b20 0x5566ff11bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:50.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.646+0000 7fe8b17fd740 4 client.35327 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:52:50.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.646+0000 7fe8b17fd740 1 -- 172.21.15.83:0/3656459127 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5566ff071c10 con 0x5566ff119b20 2024-04-24T08:52:50.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.647+0000 7fe8a7e00640 1 --2- 172.21.15.83:0/3656459127 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5566ff119b20 0x5566ff11bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:50.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.647+0000 7fe8a7e00640 1 --2- 172.21.15.83:0/3656459127 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5566ff119b20 0x5566ff11bf50 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8006f71c127910f6 server_cookie=ab57f7990b239a2c in_seq=0 out_seq=0 2024-04-24T08:52:50.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.647+0000 7fe891600640 10 client.35327 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:52:50.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.649+0000 7fe891600640 1 -- 172.21.15.83:0/3656459127 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5566ff071c10 con 0x5566ff119b20 2024-04-24T08:52:50.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.649+0000 7fe891600640 10 client.35327 handle_command_reply: tid=0 2024-04-24T08:52:50.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.649+0000 7fe8b17fd740 1 client.35327 shutdown 2024-04-24T08:52:50.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.650+0000 7fe8b17fd740 1 -- 172.21.15.83:0/3656459127 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5566ff0ff180 msgr2=0x5566ff106ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:50.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.650+0000 7fe8b17fd740 1 --2- 172.21.15.83:0/3656459127 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5566ff0ff180 0x5566ff106ca0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe8a8075d90 tx=0x7fe8940046c0 comp rx=0 tx=0).stop 2024-04-24T08:52:50.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.650+0000 7fe8b17fd740 1 -- 172.21.15.83:0/3656459127 shutdown_connections 2024-04-24T08:52:50.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.650+0000 7fe8b17fd740 1 --2- 172.21.15.83:0/3656459127 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5566ff119b20 0x5566ff11bf50 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:50.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.650+0000 7fe8b17fd740 1 --2- 172.21.15.83:0/3656459127 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5566ff109850 0x5566ff1071e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:50.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.650+0000 7fe8b17fd740 1 --2- 172.21.15.83:0/3656459127 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5566ff0ff180 0x5566ff106ca0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:50.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.650+0000 7fe8b17fd740 1 --2- 172.21.15.83:0/3656459127 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5566ff0fe860 0x5566ff106760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:50.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.650+0000 7fe8b17fd740 1 -- 172.21.15.83:0/3656459127 >> 172.21.15.83:0/3656459127 conn(0x5566ff0fab80 msgr2=0x5566ff100880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:50.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.650+0000 7fe891600640 0 client.35327 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:52:50.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.650+0000 7fe8b17fd740 1 -- 172.21.15.83:0/3656459127 shutdown_connections 2024-04-24T08:52:50.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.650+0000 7fe8b17fd740 1 -- 172.21.15.83:0/3656459127 wait complete. 2024-04-24T08:52:50.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.650+0000 7fe8b17fd740 20 client.35327 trim_cache size 0 max 16384 2024-04-24T08:52:50.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.669+0000 7fe8b17fd740 1 Processor -- start 2024-04-24T08:52:50.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.669+0000 7fe8b17fd740 1 -- start start 2024-04-24T08:52:50.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.670+0000 7fe8b17fd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5566ff0fe860 0x5566ff1154d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:50.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.670+0000 7fe8b17fd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5566ff0ff180 0x5566ff1056a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:50.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.670+0000 7fe8b17fd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5566ff109850 0x5566ff121860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:50.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.670+0000 7fe8b17fd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a81b17b0 con 0x5566ff0ff180 2024-04-24T08:52:50.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.670+0000 7fe8b17fd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a81b32f0 con 0x5566ff109850 2024-04-24T08:52:50.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.670+0000 7fe8b17fd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a81b1e80 con 0x5566ff0fe860 2024-04-24T08:52:50.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.670+0000 7fe8a7400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5566ff0ff180 0x5566ff1056a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:50.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.670+0000 7fe8a7400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5566ff0ff180 0x5566ff1056a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54390/0 (socket says 172.21.15.83:54390) 2024-04-24T08:52:50.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.670+0000 7fe8a7400640 1 -- 172.21.15.83:0/3661378759 learned_addr learned my addr 172.21.15.83:0/3661378759 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:52:50.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.671+0000 7fe8a7e00640 1 --2- 172.21.15.83:0/3661378759 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5566ff0fe860 0x5566ff1154d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:50.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.671+0000 7fe8aca00640 1 --2- 172.21.15.83:0/3661378759 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5566ff109850 0x5566ff121860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:50.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.671+0000 7fe8a7400640 1 -- 172.21.15.83:0/3661378759 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5566ff0fe860 msgr2=0x5566ff1154d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:50.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.671+0000 7fe8a7400640 1 --2- 172.21.15.83:0/3661378759 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5566ff0fe860 0x5566ff1154d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:50.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.671+0000 7fe8a7400640 1 -- 172.21.15.83:0/3661378759 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5566ff109850 msgr2=0x5566ff121860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:50.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.671+0000 7fe8a7400640 1 --2- 172.21.15.83:0/3661378759 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5566ff109850 0x5566ff121860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:50.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.671+0000 7fe8a7400640 1 -- 172.21.15.83:0/3661378759 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566ff0fefb0 con 0x5566ff0ff180 2024-04-24T08:52:50.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.671+0000 7fe8a7e00640 1 --2- 172.21.15.83:0/3661378759 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5566ff0fe860 0x5566ff1154d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:52:50.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.671+0000 7fe8aca00640 1 --2- 172.21.15.83:0/3661378759 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5566ff109850 0x5566ff121860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:52:50.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.671+0000 7fe8a7400640 1 --2- 172.21.15.83:0/3661378759 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5566ff0ff180 0x5566ff1056a0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fe894004010 tx=0x7fe894010830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3d70a0ae6500260 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:50.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.672+0000 7fe893e00640 1 -- 172.21.15.83:0/3661378759 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe894024070 con 0x5566ff0ff180 2024-04-24T08:52:50.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.672+0000 7fe893e00640 1 -- 172.21.15.83:0/3661378759 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe894003900 con 0x5566ff0ff180 2024-04-24T08:52:50.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.672+0000 7fe893e00640 1 -- 172.21.15.83:0/3661378759 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe894029d20 con 0x5566ff0ff180 2024-04-24T08:52:50.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.673+0000 7fe8b17fd740 1 -- 172.21.15.83:0/3661378759 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5566ff0ff180 msgr2=0x5566ff1056a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:50.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.673+0000 7fe8b17fd740 1 --2- 172.21.15.83:0/3661378759 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5566ff0ff180 0x5566ff1056a0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fe894004010 tx=0x7fe894010830 comp rx=0 tx=0).stop 2024-04-24T08:52:50.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.673+0000 7fe8b17fd740 1 -- 172.21.15.83:0/3661378759 shutdown_connections 2024-04-24T08:52:50.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.673+0000 7fe8b17fd740 1 --2- 172.21.15.83:0/3661378759 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5566ff109850 0x5566ff121860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:50.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.673+0000 7fe8b17fd740 1 --2- 172.21.15.83:0/3661378759 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5566ff0ff180 0x5566ff1056a0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:50.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.673+0000 7fe8b17fd740 1 --2- 172.21.15.83:0/3661378759 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5566ff0fe860 0x5566ff1154d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:50.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.673+0000 7fe8b17fd740 1 -- 172.21.15.83:0/3661378759 >> 172.21.15.83:0/3661378759 conn(0x5566ff0fab80 msgr2=0x5566ff108820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:50.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.673+0000 7fe8b17fd740 1 -- 172.21.15.83:0/3661378759 shutdown_connections 2024-04-24T08:52:50.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.673+0000 7fe8b17fd740 1 -- 172.21.15.83:0/3661378759 wait complete. 2024-04-24T08:52:50.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.673+0000 7fe8b17fd740 1 -- start start 2024-04-24T08:52:50.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.673+0000 7fe8b17fd740 1 Processor -- start 2024-04-24T08:52:50.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.674+0000 7fe8b17fd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5566ff0fe860 0x5566ff111bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:50.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.674+0000 7fe8b17fd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5566ff0ff180 0x5566ff10e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:50.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.674+0000 7fe8b17fd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5566ff109850 0x5566ff10eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:50.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.674+0000 7fe8b17fd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a81b2c20 con 0x5566ff0ff180 2024-04-24T08:52:50.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.674+0000 7fe8b17fd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a81b2550 con 0x5566ff109850 2024-04-24T08:52:50.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.674+0000 7fe8b17fd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a81af790 con 0x5566ff0fe860 2024-04-24T08:52:50.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.674+0000 7fe8b17fd740 10 client.0 _ll_register_callbacks cb 0x7fe8940045b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:52:50.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.674+0000 7fe8a7400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5566ff0ff180 0x5566ff10e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:50.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.674+0000 7fe8a7400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5566ff0ff180 0x5566ff10e9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54394/0 (socket says 172.21.15.83:54394) 2024-04-24T08:52:50.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.674+0000 7fe8a7400640 1 -- 172.21.15.83:0/3303186510 learned_addr learned my addr 172.21.15.83:0/3303186510 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:52:50.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.675+0000 7fe8a7e00640 1 --2- 172.21.15.83:0/3303186510 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5566ff0fe860 0x5566ff111bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:50.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.675+0000 7fe8a7400640 1 -- 172.21.15.83:0/3303186510 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5566ff0fe860 msgr2=0x5566ff111bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:50.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.675+0000 7fe8a7400640 1 --2- 172.21.15.83:0/3303186510 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5566ff0fe860 0x5566ff111bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:50.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.675+0000 7fe8a7400640 1 -- 172.21.15.83:0/3303186510 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5566ff109850 msgr2=0x5566ff10eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:50.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.675+0000 7fe8aca00640 1 --2- 172.21.15.83:0/3303186510 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5566ff109850 0x5566ff10eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:50.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.675+0000 7fe8a7400640 1 --2- 172.21.15.83:0/3303186510 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5566ff109850 0x5566ff10eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:50.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.675+0000 7fe8a7400640 1 -- 172.21.15.83:0/3303186510 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566fed763f0 con 0x5566ff0ff180 2024-04-24T08:52:50.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.675+0000 7fe8aca00640 1 --2- 172.21.15.83:0/3303186510 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5566ff109850 0x5566ff10eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:52:50.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.675+0000 7fe8a7400640 1 --2- 172.21.15.83:0/3303186510 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5566ff0ff180 0x5566ff10e9b0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fe89400f010 tx=0x7fe894011fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99b2b7206160fa18 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:50.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.676+0000 7fe891600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:52:50.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.678+0000 7fe891600640 1 -- 172.21.15.83:0/3303186510 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe894024040 con 0x5566ff0ff180 2024-04-24T08:52:50.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.678+0000 7fe8b17fd740 1 -- 172.21.15.83:0/3303186510 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5566ff06ba80 con 0x5566ff0ff180 2024-04-24T08:52:50.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.678+0000 7fe891600640 1 -- 172.21.15.83:0/3303186510 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7fe8940205f0 con 0x5566ff0ff180 2024-04-24T08:52:50.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.679+0000 7fe891600640 1 -- 172.21.15.83:0/3303186510 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe894029960 con 0x5566ff0ff180 2024-04-24T08:52:50.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.679+0000 7fe8b17fd740 10 client.16083 fetch_fsmap learned FSMap version 28 2024-04-24T08:52:50.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.679+0000 7fe8b17fd740 1 -- 172.21.15.83:0/3303186510 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5566fed360f0 con 0x5566ff0ff180 2024-04-24T08:52:50.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.679+0000 7fe891600640 1 -- 172.21.15.83:0/3303186510 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x5566ff06ba80 con 0x5566ff0ff180 2024-04-24T08:52:50.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.679+0000 7fe891600640 1 -- 172.21.15.83:0/3303186510 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe894013c20 con 0x5566ff0ff180 2024-04-24T08:52:50.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.679+0000 7fe8b17fd740 10 client.16083 fetch_fsmap finished waiting for FSMap version 28 2024-04-24T08:52:50.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.679+0000 7fe8b17fd740 10 client.16083 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:52:50.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.679+0000 7fe8b17fd740 20 client.16083 populate_metadata read hostname 'smithi083' 2024-04-24T08:52:50.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.679+0000 7fe8b17fd740 1 --2- 172.21.15.83:0/3303186510 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5566ff180ff0 0x5566ff183410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:50.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.679+0000 7fe8b17fd740 4 client.16083 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T08:52:50.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.679+0000 7fe8b17fd740 1 -- 172.21.15.83:0/3303186510 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5566fefe7360 con 0x5566ff180ff0 2024-04-24T08:52:50.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.680+0000 7fe8a7e00640 1 --2- 172.21.15.83:0/3303186510 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5566ff180ff0 0x5566ff183410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:50.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.681+0000 7fe8a7e00640 1 --2- 172.21.15.83:0/3303186510 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5566ff180ff0 0x5566ff183410 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed6deff76396c9b9 server_cookie=baa138eaf2bd818e in_seq=0 out_seq=0 2024-04-24T08:52:50.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.681+0000 7fe891600640 10 client.16083 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:52:50.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.682+0000 7fe891600640 1 -- 172.21.15.83:0/3303186510 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5566fefe7360 con 0x5566ff180ff0 2024-04-24T08:52:50.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.682+0000 7fe891600640 10 client.16083 handle_command_reply: tid=0 2024-04-24T08:52:50.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.682+0000 7fe8b17fd740 1 client.16083 shutdown 2024-04-24T08:52:50.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.682+0000 7fe8b17fd740 1 -- 172.21.15.83:0/3303186510 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5566ff0ff180 msgr2=0x5566ff10e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:50.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.682+0000 7fe8b17fd740 1 --2- 172.21.15.83:0/3303186510 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5566ff0ff180 0x5566ff10e9b0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fe89400f010 tx=0x7fe894011fd0 comp rx=0 tx=0).stop 2024-04-24T08:52:50.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.682+0000 7fe8b17fd740 1 -- 172.21.15.83:0/3303186510 shutdown_connections 2024-04-24T08:52:50.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.682+0000 7fe8b17fd740 1 --2- 172.21.15.83:0/3303186510 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5566ff180ff0 0x5566ff183410 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:50.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.682+0000 7fe8b17fd740 1 --2- 172.21.15.83:0/3303186510 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5566ff109850 0x5566ff10eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:50.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.682+0000 7fe8b17fd740 1 --2- 172.21.15.83:0/3303186510 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5566ff0ff180 0x5566ff10e9b0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:50.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.682+0000 7fe8b17fd740 1 --2- 172.21.15.83:0/3303186510 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5566ff0fe860 0x5566ff111bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:50.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.682+0000 7fe8b17fd740 1 -- 172.21.15.83:0/3303186510 >> 172.21.15.83:0/3303186510 conn(0x5566ff0fab80 msgr2=0x5566ff0f9390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:50.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.682+0000 7fe891600640 0 client.16083 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:52:50.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.682+0000 7fe8b17fd740 1 -- 172.21.15.83:0/3303186510 shutdown_connections 2024-04-24T08:52:50.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.682+0000 7fe8b17fd740 1 -- 172.21.15.83:0/3303186510 wait complete. 2024-04-24T08:52:50.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.683+0000 7fe8b17fd740 20 client.16083 trim_cache size 0 max 16384 2024-04-24T08:52:50.684 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T08:52:50.684 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-04-24T08:52:50.684 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-04-24T08:52:50.684 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T08:52:50.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.683+0000 7fe8aea00640 1 -- 172.21.15.83:0/1930728342 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe88c0a3e70 msgr2=0x7fe88c0a6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:50.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.683+0000 7fe8aea00640 1 --2- 172.21.15.83:0/1930728342 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe88c0a3e70 0x7fe88c0a6330 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x5566ff0fbd10 tx=0x7fe894004730 comp rx=0 tx=0).stop 2024-04-24T08:52:50.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.683+0000 7fe8aea00640 1 -- 172.21.15.83:0/1930728342 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8a8074d80 msgr2=0x7fe8a81a6a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:50.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.683+0000 7fe8aea00640 1 --2- 172.21.15.83:0/1930728342 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8a8074d80 0x7fe8a81a6a00 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fe8a000eaa0 tx=0x7fe8a000ef60 comp rx=0 tx=0).stop 2024-04-24T08:52:50.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.684+0000 7fe8aea00640 1 -- 172.21.15.83:0/1930728342 shutdown_connections 2024-04-24T08:52:50.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.684+0000 7fe8aea00640 1 --2- 172.21.15.83:0/1930728342 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe88c0a3e70 0x7fe88c0a6330 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:50.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.684+0000 7fe8aea00640 1 --2- 172.21.15.83:0/1930728342 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8a8074d80 0x7fe8a81a6a00 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:50.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.684+0000 7fe8aea00640 1 --2- 172.21.15.83:0/1930728342 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe8a8107310 0x7fe8a8074840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:50.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.684+0000 7fe8aea00640 1 --2- 172.21.15.83:0/1930728342 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe8a8102240 0x7fe8a8074300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:50.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.684+0000 7fe8aea00640 1 -- 172.21.15.83:0/1930728342 >> 172.21.15.83:0/1930728342 conn(0x7fe8a8079450 msgr2=0x7fe8a80fffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:50.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.684+0000 7fe8aea00640 1 -- 172.21.15.83:0/1930728342 shutdown_connections 2024-04-24T08:52:50.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.684+0000 7fe8aea00640 1 -- 172.21.15.83:0/1930728342 wait complete. 2024-04-24T08:52:50.702 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T08:52:50.703 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T08:52:50.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.868+0000 7f3a4b000640 1 -- 172.21.15.83:0/1746441829 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a440743b0 msgr2=0x7f3a44074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:50.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.868+0000 7f3a4b000640 1 --2- 172.21.15.83:0/1746441829 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a440743b0 0x7f3a44074790 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f3a38009900 tx=0x7f3a3802a480 comp rx=0 tx=0).stop 2024-04-24T08:52:50.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.868+0000 7f3a4b000640 1 -- 172.21.15.83:0/1746441829 shutdown_connections 2024-04-24T08:52:50.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.868+0000 7f3a4b000640 1 --2- 172.21.15.83:0/1746441829 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3a4410e6e0 0x7f3a44110ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:50.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.868+0000 7f3a4b000640 1 --2- 172.21.15.83:0/1746441829 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a44074cd0 0x7f3a4410e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:50.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.868+0000 7f3a4b000640 1 --2- 172.21.15.83:0/1746441829 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a440743b0 0x7f3a44074790 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:50.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.868+0000 7f3a4b000640 1 -- 172.21.15.83:0/1746441829 >> 172.21.15.83:0/1746441829 conn(0x7f3a440fd930 msgr2=0x7f3a440ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:50.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.868+0000 7f3a4b000640 1 -- 172.21.15.83:0/1746441829 shutdown_connections 2024-04-24T08:52:50.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.868+0000 7f3a4b000640 1 -- 172.21.15.83:0/1746441829 wait complete. 2024-04-24T08:52:50.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.870+0000 7f3a4b000640 1 Processor -- start 2024-04-24T08:52:50.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.870+0000 7f3a4b000640 1 -- start start 2024-04-24T08:52:50.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.870+0000 7f3a4b000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a440743b0 0x7f3a441a05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:50.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.870+0000 7f3a4b000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3a44074cd0 0x7f3a441a0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:50.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.870+0000 7f3a4b000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a4410e6e0 0x7f3a4419a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:50.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.870+0000 7f3a4b000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3a441142f0 con 0x7f3a4410e6e0 2024-04-24T08:52:50.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.870+0000 7f3a4b000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3a44114170 con 0x7f3a440743b0 2024-04-24T08:52:50.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.870+0000 7f3a4b000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3a44114470 con 0x7f3a44074cd0 2024-04-24T08:52:50.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.871+0000 7f3a4a600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a4410e6e0 0x7f3a4419a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:50.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.871+0000 7f3a4a600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a4410e6e0 0x7f3a4419a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54408/0 (socket says 172.21.15.83:54408) 2024-04-24T08:52:50.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.871+0000 7f3a4a600640 1 -- 172.21.15.83:0/2316371904 learned_addr learned my addr 172.21.15.83:0/2316371904 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:52:50.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.871+0000 7f3a49c00640 1 --2- 172.21.15.83:0/2316371904 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a440743b0 0x7f3a441a05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:50.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.871+0000 7f3a49200640 1 --2- 172.21.15.83:0/2316371904 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3a44074cd0 0x7f3a441a0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:50.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.871+0000 7f3a4a600640 1 -- 172.21.15.83:0/2316371904 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3a44074cd0 msgr2=0x7f3a441a0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:50.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.871+0000 7f3a4a600640 1 --2- 172.21.15.83:0/2316371904 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3a44074cd0 0x7f3a441a0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:50.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.871+0000 7f3a4a600640 1 -- 172.21.15.83:0/2316371904 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a440743b0 msgr2=0x7f3a441a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:50.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.871+0000 7f3a4a600640 1 --2- 172.21.15.83:0/2316371904 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a440743b0 0x7f3a441a05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:50.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.871+0000 7f3a4a600640 1 -- 172.21.15.83:0/2316371904 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a4419af70 con 0x7f3a4410e6e0 2024-04-24T08:52:50.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.871+0000 7f3a49c00640 1 --2- 172.21.15.83:0/2316371904 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a440743b0 0x7f3a441a05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:52:50.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.872+0000 7f3a4a600640 1 --2- 172.21.15.83:0/2316371904 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a4410e6e0 0x7f3a4419a6b0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c00eaa0 tx=0x7f3a3c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:50.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.872+0000 7f3a42000640 1 -- 172.21.15.83:0/2316371904 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a3c00cdb0 con 0x7f3a4410e6e0 2024-04-24T08:52:50.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.872+0000 7f3a42000640 1 -- 172.21.15.83:0/2316371904 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3a3c005020 con 0x7f3a4410e6e0 2024-04-24T08:52:50.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.872+0000 7f3a4b000640 1 -- 172.21.15.83:0/2316371904 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a4419b260 con 0x7f3a4410e6e0 2024-04-24T08:52:50.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.872+0000 7f3a42000640 1 -- 172.21.15.83:0/2316371904 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a3c0116d0 con 0x7f3a4410e6e0 2024-04-24T08:52:50.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.872+0000 7f3a4b000640 1 -- 172.21.15.83:0/2316371904 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a441a7520 con 0x7f3a4410e6e0 2024-04-24T08:52:50.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.874+0000 7f3a4b000640 1 -- 172.21.15.83:0/2316371904 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a44075a60 con 0x7f3a4410e6e0 2024-04-24T08:52:50.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.877+0000 7f3a42000640 1 -- 172.21.15.83:0/2316371904 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7f3a3c0051d0 con 0x7f3a4410e6e0 2024-04-24T08:52:50.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.879+0000 7f3a42000640 1 --2- 172.21.15.83:0/2316371904 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f3a2c0a3f90 0x7f3a2c0a6450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:50.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.879+0000 7f3a42000640 1 -- 172.21.15.83:0/2316371904 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f3a3c0d0f20 con 0x7f3a4410e6e0 2024-04-24T08:52:50.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.879+0000 7f3a49c00640 1 --2- 172.21.15.83:0/2316371904 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f3a2c0a3f90 0x7f3a2c0a6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:50.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.880+0000 7f3a49c00640 1 --2- 172.21.15.83:0/2316371904 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f3a2c0a3f90 0x7f3a2c0a6450 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3a38000c00 tx=0x7f3a38005a20 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:50.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:50.887+0000 7f3a42000640 1 -- 172.21.15.83:0/2316371904 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3a3c016030 con 0x7f3a4410e6e0 2024-04-24T08:52:51.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.224+0000 7f3a4b000640 1 -- 172.21.15.83:0/2316371904 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3a44068d40 con 0x7f3a4410e6e0 2024-04-24T08:52:51.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.225+0000 7f3a42000640 1 -- 172.21.15.83:0/2316371904 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11944 (secure 0 0 0) 0x7f3a3c09cf40 con 0x7f3a4410e6e0 2024-04-24T08:52:51.225 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:52:51.225 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/358872257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":358872257},{"type":"v1","addr":"172.21.15.83:6835","nonce":358872257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3157070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3157070851},{"type":"v1","addr":"172.21.15.83:6837","nonce":3157070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T08:48:29.422576+0000","modified":"2024-04-24T08:51:36.355488+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34520,"mds_1":34493,"mds_2":34451,"mds_3":34427,"mds_4":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14658":{"gid":14658,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6839/4085470844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":4085470844},{"type":"v1","addr":"172.21.15.83:6839","nonce":4085470844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14676":{"gid":14676,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6841/2634754210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":2634754210},{"type":"v1","addr":"172.21.15.83:6841","nonce":2634754210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24382":{"gid":24382,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/1859921310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1859921310},{"type":"v1","addr":"172.21.15.189:6833","nonce":1859921310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/2241923197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2241923197},{"type":"v1","addr":"172.21.15.189:6835","nonce":2241923197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6837/3172311220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":3172311220},{"type":"v1","addr":"172.21.15.189:6837","nonce":3172311220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.189:6839/2548965336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":2548965336},{"type":"v1","addr":"172.21.15.189:6839","nonce":2548965336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.138:6833/1157967658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1157967658},{"type":"v1","addr":"172.21.15.138:6833","nonce":1157967658}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.138:6835/1909219758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1909219758},{"type":"v1","addr":"172.21.15.138:6835","nonce":1909219758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34493":{"gid":34493,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.138:6837/900566754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":900566754},{"type":"v1","addr":"172.21.15.138:6837","nonce":900566754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34520":{"gid":34520,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.138:6839/2236932393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2236932393},{"type":"v1","addr":"172.21.15.138:6839","nonce":2236932393}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34520,"qdb_cluster":[34451,34493,34427,34520,24454]},"id":1}]} 2024-04-24T08:52:51.226 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 28 2024-04-24T08:52:51.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.227+0000 7f3a4b000640 1 -- 172.21.15.83:0/2316371904 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f3a2c0a3f90 msgr2=0x7f3a2c0a6450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:51.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.227+0000 7f3a4b000640 1 --2- 172.21.15.83:0/2316371904 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f3a2c0a3f90 0x7f3a2c0a6450 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3a38000c00 tx=0x7f3a38005a20 comp rx=0 tx=0).stop 2024-04-24T08:52:51.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.227+0000 7f3a4b000640 1 -- 172.21.15.83:0/2316371904 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a4410e6e0 msgr2=0x7f3a4419a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:51.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.227+0000 7f3a4b000640 1 --2- 172.21.15.83:0/2316371904 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a4410e6e0 0x7f3a4419a6b0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c00eaa0 tx=0x7f3a3c00ef60 comp rx=0 tx=0).stop 2024-04-24T08:52:51.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.227+0000 7f3a4b000640 1 -- 172.21.15.83:0/2316371904 shutdown_connections 2024-04-24T08:52:51.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.227+0000 7f3a4b000640 1 --2- 172.21.15.83:0/2316371904 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f3a2c0a3f90 0x7f3a2c0a6450 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:51.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.227+0000 7f3a4b000640 1 --2- 172.21.15.83:0/2316371904 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a4410e6e0 0x7f3a4419a6b0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:51.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.227+0000 7f3a4b000640 1 --2- 172.21.15.83:0/2316371904 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3a44074cd0 0x7f3a441a0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:51.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.227+0000 7f3a4b000640 1 --2- 172.21.15.83:0/2316371904 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a440743b0 0x7f3a441a05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:51.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.227+0000 7f3a4b000640 1 -- 172.21.15.83:0/2316371904 >> 172.21.15.83:0/2316371904 conn(0x7f3a440fd930 msgr2=0x7f3a440fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:51.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.227+0000 7f3a4b000640 1 -- 172.21.15.83:0/2316371904 shutdown_connections 2024-04-24T08:52:51.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.227+0000 7f3a4b000640 1 -- 172.21.15.83:0/2316371904 wait complete. 2024-04-24T08:52:51.251 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-24T08:52:51.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.415+0000 7ffab8c00640 1 -- 172.21.15.83:0/2096603465 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffab4102ac0 msgr2=0x7ffab4106b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:51.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.415+0000 7ffab8c00640 1 --2- 172.21.15.83:0/2096603465 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffab4102ac0 0x7ffab4106b00 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c009900 tx=0x7ffa9c02a410 comp rx=0 tx=0).stop 2024-04-24T08:52:51.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.416+0000 7ffab8c00640 1 -- 172.21.15.83:0/2096603465 shutdown_connections 2024-04-24T08:52:51.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.416+0000 7ffab8c00640 1 --2- 172.21.15.83:0/2096603465 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffab41071e0 0x7ffab410edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:51.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.416+0000 7ffab8c00640 1 --2- 172.21.15.83:0/2096603465 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffab4102ac0 0x7ffab4106b00 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:51.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.416+0000 7ffab8c00640 1 --2- 172.21.15.83:0/2096603465 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffab4102110 0x7ffab41024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:51.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.416+0000 7ffab8c00640 1 -- 172.21.15.83:0/2096603465 >> 172.21.15.83:0/2096603465 conn(0x7ffab40fd9d0 msgr2=0x7ffab40ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:51.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.416+0000 7ffab8c00640 1 -- 172.21.15.83:0/2096603465 shutdown_connections 2024-04-24T08:52:51.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.416+0000 7ffab8c00640 1 -- 172.21.15.83:0/2096603465 wait complete. 2024-04-24T08:52:51.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.417+0000 7ffab8c00640 1 Processor -- start 2024-04-24T08:52:51.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.417+0000 7ffab8c00640 1 -- start start 2024-04-24T08:52:51.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.418+0000 7ffab8c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffab4102110 0x7ffab41a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:51.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.418+0000 7ffab8c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffab41071e0 0x7ffab41a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:51.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.418+0000 7ffab8c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffab419a730 0x7ffab419abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:51.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.418+0000 7ffab8c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ffab4112170 con 0x7ffab419a730 2024-04-24T08:52:51.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.418+0000 7ffab8c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ffab4111ff0 con 0x7ffab4102110 2024-04-24T08:52:51.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.418+0000 7ffab8c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ffab41122f0 con 0x7ffab41071e0 2024-04-24T08:52:51.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.419+0000 7ffab3400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffab4102110 0x7ffab41a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:51.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.419+0000 7ffab3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffab419a730 0x7ffab419abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:51.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.419+0000 7ffab3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffab419a730 0x7ffab419abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54420/0 (socket says 172.21.15.83:54420) 2024-04-24T08:52:51.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.419+0000 7ffab3e00640 1 -- 172.21.15.83:0/901367837 learned_addr learned my addr 172.21.15.83:0/901367837 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:52:51.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.419+0000 7ffaabe00640 1 --2- 172.21.15.83:0/901367837 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffab41071e0 0x7ffab41a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:51.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.419+0000 7ffab3e00640 1 -- 172.21.15.83:0/901367837 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffab41071e0 msgr2=0x7ffab41a0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:51.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.419+0000 7ffab3e00640 1 --2- 172.21.15.83:0/901367837 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffab41071e0 0x7ffab41a0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:51.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.419+0000 7ffab3e00640 1 -- 172.21.15.83:0/901367837 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffab4102110 msgr2=0x7ffab41a0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:51.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.419+0000 7ffab3e00640 1 --2- 172.21.15.83:0/901367837 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffab4102110 0x7ffab41a0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:51.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.419+0000 7ffab3e00640 1 -- 172.21.15.83:0/901367837 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffab419b330 con 0x7ffab419a730 2024-04-24T08:52:51.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.420+0000 7ffab3e00640 1 --2- 172.21.15.83:0/901367837 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffab419a730 0x7ffab419abb0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7ffaac00eaa0 tx=0x7ffaac00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:51.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.420+0000 7ffab0c00640 1 -- 172.21.15.83:0/901367837 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaac00cdb0 con 0x7ffab419a730 2024-04-24T08:52:51.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.420+0000 7ffab0c00640 1 -- 172.21.15.83:0/901367837 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffaac005020 con 0x7ffab419a730 2024-04-24T08:52:51.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.420+0000 7ffab8c00640 1 -- 172.21.15.83:0/901367837 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffab41a7410 con 0x7ffab419a730 2024-04-24T08:52:51.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.420+0000 7ffab0c00640 1 -- 172.21.15.83:0/901367837 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaac0116d0 con 0x7ffab419a730 2024-04-24T08:52:51.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.421+0000 7ffab8c00640 1 -- 172.21.15.83:0/901367837 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffab41a77c0 con 0x7ffab419a730 2024-04-24T08:52:51.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.424+0000 7ffab0c00640 1 -- 172.21.15.83:0/901367837 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7ffaac0051d0 con 0x7ffab419a730 2024-04-24T08:52:51.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.425+0000 7ffabd037740 1 Processor -- start 2024-04-24T08:52:51.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.425+0000 7ffabd037740 1 -- start start 2024-04-24T08:52:51.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.425+0000 7ffabd037740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559c4159cb40 0x559c4159cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:51.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.425+0000 7ffabd037740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c4159d460 0x559c415a75f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:51.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.426+0000 7ffabd037740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c415a7b30 0x559c415a9f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:51.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.426+0000 7ffabd037740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559c415021d0 con 0x559c415a7b30 2024-04-24T08:52:51.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.426+0000 7ffabd037740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559c414a8450 con 0x559c4159cb40 2024-04-24T08:52:51.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.426+0000 7ffabd037740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559c414fe190 con 0x559c4159d460 2024-04-24T08:52:51.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.426+0000 7ffab0c00640 1 --2- 172.21.15.83:0/901367837 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ffa980a3f40 0x7ffa980a6400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:51.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.426+0000 7ffab0c00640 1 -- 172.21.15.83:0/901367837 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7ffaac0d0ed0 con 0x7ffab419a730 2024-04-24T08:52:51.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.426+0000 7ffaabe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c415a7b30 0x559c415a9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:51.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.426+0000 7ffaabe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c415a7b30 0x559c415a9f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54424/0 (socket says 172.21.15.83:54424) 2024-04-24T08:52:51.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.426+0000 7ffaabe00640 1 -- 172.21.15.83:0/1940829989 learned_addr learned my addr 172.21.15.83:0/1940829989 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:52:51.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.426+0000 7ffaabe00640 1 --2- 172.21.15.83:0/1940829989 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559c4159cb40 0x559c4159cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:51.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.426+0000 7ffab3e00640 1 --2- 172.21.15.83:0/901367837 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ffa980a3f40 0x7ffa980a6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:51.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.426+0000 7ffab3400640 1 --2- 172.21.15.83:0/1940829989 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c4159d460 0x559c415a75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:51.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.427+0000 7ffaabe00640 1 -- 172.21.15.83:0/1940829989 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c4159d460 msgr2=0x559c415a75f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:51.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.427+0000 7ffaabe00640 1 --2- 172.21.15.83:0/1940829989 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c4159d460 0x559c415a75f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:51.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.427+0000 7ffaabe00640 1 -- 172.21.15.83:0/1940829989 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559c4159cb40 msgr2=0x559c4159cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:51.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.427+0000 7ffaabe00640 1 --2- 172.21.15.83:0/1940829989 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559c4159cb40 0x559c4159cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:51.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.427+0000 7ffaabe00640 1 -- 172.21.15.83:0/1940829989 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c41565310 con 0x559c415a7b30 2024-04-24T08:52:51.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.427+0000 7ffaabe00640 1 --2- 172.21.15.83:0/1940829989 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c415a7b30 0x559c415a9f20 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c006010 tx=0x7ffa9c005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e763e1b87f646851 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:51.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.427+0000 7ffab3e00640 1 --2- 172.21.15.83:0/901367837 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ffa980a3f40 0x7ffa980a6400 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ffaac0ce0d0 tx=0x7ffaac01cfd0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:51.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.427+0000 7ffaa3e00640 1 -- 172.21.15.83:0/1940829989 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa9c035030 con 0x559c415a7b30 2024-04-24T08:52:51.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.428+0000 7ffaa3e00640 1 -- 172.21.15.83:0/1940829989 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffa9c03d450 con 0x559c415a7b30 2024-04-24T08:52:51.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.428+0000 7ffaa3e00640 1 -- 172.21.15.83:0/1940829989 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa9c03c6a0 con 0x559c415a7b30 2024-04-24T08:52:51.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.429+0000 7ffabd037740 1 -- 172.21.15.83:0/1940829989 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c415a7b30 msgr2=0x559c415a9f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:51.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.429+0000 7ffabd037740 1 --2- 172.21.15.83:0/1940829989 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c415a7b30 0x559c415a9f20 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c006010 tx=0x7ffa9c005780 comp rx=0 tx=0).stop 2024-04-24T08:52:51.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.429+0000 7ffabd037740 1 -- 172.21.15.83:0/1940829989 shutdown_connections 2024-04-24T08:52:51.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.429+0000 7ffabd037740 1 --2- 172.21.15.83:0/1940829989 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c415a7b30 0x559c415a9f20 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:51.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.429+0000 7ffabd037740 1 --2- 172.21.15.83:0/1940829989 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c4159d460 0x559c415a75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:51.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.429+0000 7ffabd037740 1 --2- 172.21.15.83:0/1940829989 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559c4159cb40 0x559c4159cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:51.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.429+0000 7ffabd037740 1 -- 172.21.15.83:0/1940829989 >> 172.21.15.83:0/1940829989 conn(0x559c41598e60 msgr2=0x559c41599240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:51.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.429+0000 7ffabd037740 1 -- 172.21.15.83:0/1940829989 shutdown_connections 2024-04-24T08:52:51.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.429+0000 7ffabd037740 1 -- 172.21.15.83:0/1940829989 wait complete. 2024-04-24T08:52:51.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.430+0000 7ffabd037740 1 -- start start 2024-04-24T08:52:51.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.430+0000 7ffabd037740 1 Processor -- start 2024-04-24T08:52:51.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.431+0000 7ffabd037740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559c4159cb40 0x559c415a7540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:51.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.431+0000 7ffabd037740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c4159d460 0x559c415a4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:51.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.431+0000 7ffabd037740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c415a7b30 0x559c415a4e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:51.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.431+0000 7ffabd037740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559c415aa960 con 0x559c4159d460 2024-04-24T08:52:51.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.431+0000 7ffabd037740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559c414bb810 con 0x559c4159cb40 2024-04-24T08:52:51.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.431+0000 7ffabd037740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559c415aaae0 con 0x559c415a7b30 2024-04-24T08:52:51.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.431+0000 7ffabd037740 10 client.0 _ll_register_callbacks cb 0x559c41489140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:52:51.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.431+0000 7ffab3400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c4159d460 0x559c415a4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:51.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.431+0000 7ffab3400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c4159d460 0x559c415a4830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54428/0 (socket says 172.21.15.83:54428) 2024-04-24T08:52:51.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.431+0000 7ffab3400640 1 -- 172.21.15.83:0/1878813132 learned_addr learned my addr 172.21.15.83:0/1878813132 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:52:51.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.431+0000 7ffaabe00640 1 --2- 172.21.15.83:0/1878813132 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559c4159cb40 0x559c415a7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:51.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.432+0000 7ffaabe00640 1 --2- 172.21.15.83:0/1878813132 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c415a7b30 0x559c415a4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:51.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.432+0000 7ffab3400640 1 -- 172.21.15.83:0/1878813132 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c415a7b30 msgr2=0x559c415a4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:51.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.432+0000 7ffab3400640 1 --2- 172.21.15.83:0/1878813132 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c415a7b30 0x559c415a4e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:51.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.432+0000 7ffab3400640 1 -- 172.21.15.83:0/1878813132 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559c4159cb40 msgr2=0x559c415a7540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:51.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.432+0000 7ffab3400640 1 --2- 172.21.15.83:0/1878813132 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559c4159cb40 0x559c415a7540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:51.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.432+0000 7ffab3400640 1 -- 172.21.15.83:0/1878813132 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c414cc3f0 con 0x559c4159d460 2024-04-24T08:52:51.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.432+0000 7ffaabe00640 1 --2- 172.21.15.83:0/1878813132 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c415a7b30 0x559c415a4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:52:51.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.432+0000 7ffab3400640 1 --2- 172.21.15.83:0/1878813132 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c4159d460 0x559c415a4830 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7ffaa400ba10 tx=0x7ffaa400bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d59174192a9ec06 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:51.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.432+0000 7ffaa0c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:52:51.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.432+0000 7ffaa0c00640 1 -- 172.21.15.83:0/1878813132 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaa4003e80 con 0x559c4159d460 2024-04-24T08:52:51.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.432+0000 7ffaa0c00640 1 -- 172.21.15.83:0/1878813132 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7ffaa4011440 con 0x559c4159d460 2024-04-24T08:52:51.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.432+0000 7ffabd037740 1 -- 172.21.15.83:0/1878813132 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa9c03d450 con 0x559c4159d460 2024-04-24T08:52:51.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.433+0000 7ffaa0c00640 1 -- 172.21.15.83:0/1878813132 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ffaa4011640 con 0x559c4159d460 2024-04-24T08:52:51.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.433+0000 7ffabd037740 10 client.16101 fetch_fsmap learned FSMap version 28 2024-04-24T08:52:51.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.433+0000 7ffabd037740 1 -- 172.21.15.83:0/1878813132 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559c412143f0 con 0x559c4159d460 2024-04-24T08:52:51.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.433+0000 7ffaa0c00640 1 -- 172.21.15.83:0/1878813132 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffa9c03d450 con 0x559c4159d460 2024-04-24T08:52:51.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.434+0000 7ffaa0c00640 1 -- 172.21.15.83:0/1878813132 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ffaa4025990 con 0x559c4159d460 2024-04-24T08:52:51.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.434+0000 7ffabd037740 10 client.16101 fetch_fsmap finished waiting for FSMap version 28 2024-04-24T08:52:51.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.434+0000 7ffabd037740 10 client.16101 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-24T08:52:51.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.434+0000 7ffabd037740 20 client.16101 populate_metadata read hostname 'smithi083' 2024-04-24T08:52:51.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.434+0000 7ffabd037740 1 --2- 172.21.15.83:0/1878813132 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x559c415b7ce0 0x559c415ba150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:51.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.434+0000 7ffabd037740 4 client.16101 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:52:51.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.434+0000 7ffabd037740 1 -- 172.21.15.83:0/1878813132 --> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559c4150fbf0 con 0x559c415b7ce0 2024-04-24T08:52:51.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.435+0000 7ffaabe00640 1 --2- 172.21.15.83:0/1878813132 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x559c415b7ce0 0x559c415ba150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:51.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.436+0000 7ffaabe00640 1 --2- 172.21.15.83:0/1878813132 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x559c415b7ce0 0x559c415ba150 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ccfef49ab8e126ec server_cookie=94352f110e77b17b in_seq=0 out_seq=0 2024-04-24T08:52:51.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.436+0000 7ffaa0c00640 10 client.16101 ms_handle_connect on v2:172.21.15.189:6838/2548965336 2024-04-24T08:52:51.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.437+0000 7ffaa0c00640 1 -- 172.21.15.83:0/1878813132 <== mds.4 v2:172.21.15.189:6838/2548965336 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559c4150fbf0 con 0x559c415b7ce0 2024-04-24T08:52:51.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.437+0000 7ffaa0c00640 10 client.16101 handle_command_reply: tid=0 2024-04-24T08:52:51.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.438+0000 7ffabd037740 1 client.16101 shutdown 2024-04-24T08:52:51.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.438+0000 7ffabd037740 1 -- 172.21.15.83:0/1878813132 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c4159d460 msgr2=0x559c415a4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:51.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.438+0000 7ffabd037740 1 --2- 172.21.15.83:0/1878813132 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c4159d460 0x559c415a4830 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7ffaa400ba10 tx=0x7ffaa400bed0 comp rx=0 tx=0).stop 2024-04-24T08:52:51.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.438+0000 7ffabd037740 1 -- 172.21.15.83:0/1878813132 shutdown_connections 2024-04-24T08:52:51.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.438+0000 7ffabd037740 1 --2- 172.21.15.83:0/1878813132 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x559c415b7ce0 0x559c415ba150 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:51.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.438+0000 7ffabd037740 1 --2- 172.21.15.83:0/1878813132 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c415a7b30 0x559c415a4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:51.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.438+0000 7ffabd037740 1 --2- 172.21.15.83:0/1878813132 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c4159d460 0x559c415a4830 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:51.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.438+0000 7ffabd037740 1 --2- 172.21.15.83:0/1878813132 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559c4159cb40 0x559c415a7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:51.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.438+0000 7ffabd037740 1 -- 172.21.15.83:0/1878813132 >> 172.21.15.83:0/1878813132 conn(0x559c41598e60 msgr2=0x559c4159ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:51.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.438+0000 7ffaa0c00640 0 client.16101 ms_handle_reset on v2:172.21.15.189:6838/2548965336 2024-04-24T08:52:51.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.438+0000 7ffabd037740 1 -- 172.21.15.83:0/1878813132 shutdown_connections 2024-04-24T08:52:51.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.438+0000 7ffabd037740 1 -- 172.21.15.83:0/1878813132 wait complete. 2024-04-24T08:52:51.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.439+0000 7ffabd037740 20 client.16101 trim_cache size 0 max 16384 2024-04-24T08:52:51.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.458+0000 7ffabd037740 1 Processor -- start 2024-04-24T08:52:51.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.458+0000 7ffabd037740 1 -- start start 2024-04-24T08:52:51.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.458+0000 7ffabd037740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c4159cb40 0x559c415b3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:51.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.458+0000 7ffabd037740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559c4159d460 0x559c415a7070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:51.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.458+0000 7ffabd037740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c415a7b30 0x559c415bfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:51.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.458+0000 7ffabd037740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ffab41b1500 con 0x559c415a7b30 2024-04-24T08:52:51.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.458+0000 7ffabd037740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ffab41b3040 con 0x559c4159d460 2024-04-24T08:52:51.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.458+0000 7ffabd037740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ffab41b1bd0 con 0x559c4159cb40 2024-04-24T08:52:51.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.459+0000 7ffaabe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c415a7b30 0x559c415bfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:51.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.459+0000 7ffaabe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c415a7b30 0x559c415bfa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54440/0 (socket says 172.21.15.83:54440) 2024-04-24T08:52:51.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.459+0000 7ffaabe00640 1 -- 172.21.15.83:0/3564569972 learned_addr learned my addr 172.21.15.83:0/3564569972 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:52:51.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.459+0000 7ffaabe00640 1 --2- 172.21.15.83:0/3564569972 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c4159cb40 0x559c415b3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:51.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.459+0000 7ffab3400640 1 --2- 172.21.15.83:0/3564569972 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559c4159d460 0x559c415a7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:51.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.459+0000 7ffaabe00640 1 -- 172.21.15.83:0/3564569972 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c4159cb40 msgr2=0x559c415b3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:51.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.459+0000 7ffaabe00640 1 --2- 172.21.15.83:0/3564569972 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c4159cb40 0x559c415b3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:51.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.459+0000 7ffaabe00640 1 -- 172.21.15.83:0/3564569972 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559c4159d460 msgr2=0x559c415a7070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:51.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.459+0000 7ffaabe00640 1 --2- 172.21.15.83:0/3564569972 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559c4159d460 0x559c415a7070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:51.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.459+0000 7ffaabe00640 1 -- 172.21.15.83:0/3564569972 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c412de7d0 con 0x559c415a7b30 2024-04-24T08:52:51.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.460+0000 7ffaabe00640 1 --2- 172.21.15.83:0/3564569972 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c415a7b30 0x559c415bfa20 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c02a9d0 tx=0x7ffa9c03ddb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce67ee38d8f56bb0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:51.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.460+0000 7ffaa3400640 1 -- 172.21.15.83:0/3564569972 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa9c035040 con 0x559c415a7b30 2024-04-24T08:52:51.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.460+0000 7ffaa3400640 1 -- 172.21.15.83:0/3564569972 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffa9c052dc0 con 0x559c415a7b30 2024-04-24T08:52:51.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.460+0000 7ffaa3400640 1 -- 172.21.15.83:0/3564569972 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa9c0526d0 con 0x559c415a7b30 2024-04-24T08:52:51.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.461+0000 7ffabd037740 1 -- 172.21.15.83:0/3564569972 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c415a7b30 msgr2=0x559c415bfa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:51.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.461+0000 7ffabd037740 1 --2- 172.21.15.83:0/3564569972 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c415a7b30 0x559c415bfa20 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c02a9d0 tx=0x7ffa9c03ddb0 comp rx=0 tx=0).stop 2024-04-24T08:52:51.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.461+0000 7ffabd037740 1 -- 172.21.15.83:0/3564569972 shutdown_connections 2024-04-24T08:52:51.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.461+0000 7ffabd037740 1 --2- 172.21.15.83:0/3564569972 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c415a7b30 0x559c415bfa20 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:51.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.461+0000 7ffabd037740 1 --2- 172.21.15.83:0/3564569972 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559c4159d460 0x559c415a7070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:51.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.461+0000 7ffabd037740 1 --2- 172.21.15.83:0/3564569972 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c4159cb40 0x559c415b3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:51.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.461+0000 7ffabd037740 1 -- 172.21.15.83:0/3564569972 >> 172.21.15.83:0/3564569972 conn(0x559c41598e60 msgr2=0x559c4159fb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:51.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.461+0000 7ffabd037740 1 -- 172.21.15.83:0/3564569972 shutdown_connections 2024-04-24T08:52:51.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.461+0000 7ffabd037740 1 -- 172.21.15.83:0/3564569972 wait complete. 2024-04-24T08:52:51.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.461+0000 7ffabd037740 1 -- start start 2024-04-24T08:52:51.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.462+0000 7ffabd037740 1 Processor -- start 2024-04-24T08:52:51.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.462+0000 7ffabd037740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c4159cb40 0x559c415b0260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:51.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.462+0000 7ffabd037740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559c4159d460 0x559c415a1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:51.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.462+0000 7ffabd037740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c415a7b30 0x559c415a2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:51.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.462+0000 7ffabd037740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ffab41b2970 con 0x559c415a7b30 2024-04-24T08:52:51.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.462+0000 7ffabd037740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ffab41b22a0 con 0x559c4159d460 2024-04-24T08:52:51.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.462+0000 7ffabd037740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ffab41af4e0 con 0x559c4159cb40 2024-04-24T08:52:51.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.462+0000 7ffabd037740 10 client.0 _ll_register_callbacks cb 0x7ffaa40042b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:52:51.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.463+0000 7ffaabe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c415a7b30 0x559c415a2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:51.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.463+0000 7ffaabe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c415a7b30 0x559c415a2310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54448/0 (socket says 172.21.15.83:54448) 2024-04-24T08:52:51.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.463+0000 7ffaabe00640 1 -- 172.21.15.83:0/241670731 learned_addr learned my addr 172.21.15.83:0/241670731 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:52:51.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.463+0000 7ffab3400640 1 --2- 172.21.15.83:0/241670731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559c4159d460 0x559c415a1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:51.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.463+0000 7ffaabe00640 1 --2- 172.21.15.83:0/241670731 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c4159cb40 0x559c415b0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:51.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.463+0000 7ffaabe00640 1 -- 172.21.15.83:0/241670731 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c4159cb40 msgr2=0x559c415b0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:51.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.463+0000 7ffaabe00640 1 --2- 172.21.15.83:0/241670731 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c4159cb40 0x559c415b0260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:51.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.463+0000 7ffaabe00640 1 -- 172.21.15.83:0/241670731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559c4159d460 msgr2=0x559c415a1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:51.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.463+0000 7ffaabe00640 1 --2- 172.21.15.83:0/241670731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559c4159d460 0x559c415a1ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:51.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.463+0000 7ffaabe00640 1 -- 172.21.15.83:0/241670731 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c415a8990 con 0x559c415a7b30 2024-04-24T08:52:51.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.464+0000 7ffaabe00640 1 --2- 172.21.15.83:0/241670731 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c415a7b30 0x559c415a2310 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c009340 tx=0x7ffa9c004220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f013b99c7db28522 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:51.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.464+0000 7ffaa0c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:52:51.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.464+0000 7ffaa0c00640 1 -- 172.21.15.83:0/241670731 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa9c035040 con 0x559c415a7b30 2024-04-24T08:52:51.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.464+0000 7ffaa0c00640 1 -- 172.21.15.83:0/241670731 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7ffa9c003cd0 con 0x559c415a7b30 2024-04-24T08:52:51.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.464+0000 7ffabd037740 1 -- 172.21.15.83:0/241670731 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa9c052dc0 con 0x559c415a7b30 2024-04-24T08:52:51.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.464+0000 7ffaa0c00640 1 -- 172.21.15.83:0/241670731 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ffa9c0521e0 con 0x559c415a7b30 2024-04-24T08:52:51.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.464+0000 7ffabd037740 10 client.16107 fetch_fsmap learned FSMap version 28 2024-04-24T08:52:51.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.464+0000 7ffabd037740 1 -- 172.21.15.83:0/241670731 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559c4161e5c0 con 0x559c415a7b30 2024-04-24T08:52:51.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.465+0000 7ffaa0c00640 1 -- 172.21.15.83:0/241670731 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffa9c052dc0 con 0x559c415a7b30 2024-04-24T08:52:51.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.465+0000 7ffaa0c00640 1 -- 172.21.15.83:0/241670731 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ffa9c03d780 con 0x559c415a7b30 2024-04-24T08:52:51.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.466+0000 7ffabd037740 10 client.16107 fetch_fsmap finished waiting for FSMap version 28 2024-04-24T08:52:51.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.466+0000 7ffabd037740 10 client.16107 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-24T08:52:51.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.466+0000 7ffabd037740 20 client.16107 populate_metadata read hostname 'smithi083' 2024-04-24T08:52:51.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.466+0000 7ffabd037740 1 --2- 172.21.15.83:0/241670731 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x559c4161e790 0x559c41620b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:51.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.466+0000 7ffabd037740 4 client.16107 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T08:52:51.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.466+0000 7ffabd037740 1 -- 172.21.15.83:0/241670731 --> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559c41509a60 con 0x559c4161e790 2024-04-24T08:52:51.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.466+0000 7ffab3400640 1 --2- 172.21.15.83:0/241670731 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x559c4161e790 0x559c41620b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:51.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.468+0000 7ffab3400640 1 --2- 172.21.15.83:0/241670731 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x559c4161e790 0x559c41620b80 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e5ba87315b1ce4f1 server_cookie=245abddb7162dc8a in_seq=0 out_seq=0 2024-04-24T08:52:51.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.468+0000 7ffaa0c00640 10 client.16107 ms_handle_connect on v2:172.21.15.189:6838/2548965336 2024-04-24T08:52:51.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.468+0000 7ffaa0c00640 1 -- 172.21.15.83:0/241670731 <== mds.4 v2:172.21.15.189:6838/2548965336 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559c41509a60 con 0x559c4161e790 2024-04-24T08:52:51.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.468+0000 7ffaa0c00640 10 client.16107 handle_command_reply: tid=0 2024-04-24T08:52:51.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.468+0000 7ffabd037740 1 client.16107 shutdown 2024-04-24T08:52:51.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.469+0000 7ffabd037740 1 -- 172.21.15.83:0/241670731 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c415a7b30 msgr2=0x559c415a2310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:51.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.469+0000 7ffabd037740 1 --2- 172.21.15.83:0/241670731 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c415a7b30 0x559c415a2310 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c009340 tx=0x7ffa9c004220 comp rx=0 tx=0).stop 2024-04-24T08:52:51.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.469+0000 7ffabd037740 1 -- 172.21.15.83:0/241670731 shutdown_connections 2024-04-24T08:52:51.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.469+0000 7ffabd037740 1 --2- 172.21.15.83:0/241670731 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x559c4161e790 0x559c41620b80 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:51.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.469+0000 7ffabd037740 1 --2- 172.21.15.83:0/241670731 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559c415a7b30 0x559c415a2310 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:51.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.469+0000 7ffabd037740 1 --2- 172.21.15.83:0/241670731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559c4159d460 0x559c415a1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:51.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.469+0000 7ffabd037740 1 --2- 172.21.15.83:0/241670731 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c4159cb40 0x559c415b0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:51.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.469+0000 7ffabd037740 1 -- 172.21.15.83:0/241670731 >> 172.21.15.83:0/241670731 conn(0x559c41598e60 msgr2=0x559c415976c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:51.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.469+0000 7ffaa0c00640 0 client.16107 ms_handle_reset on v2:172.21.15.189:6838/2548965336 2024-04-24T08:52:51.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.469+0000 7ffabd037740 1 -- 172.21.15.83:0/241670731 shutdown_connections 2024-04-24T08:52:51.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.469+0000 7ffabd037740 1 -- 172.21.15.83:0/241670731 wait complete. 2024-04-24T08:52:51.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.469+0000 7ffabd037740 20 client.16107 trim_cache size 0 max 16384 2024-04-24T08:52:51.471 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T08:52:51.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.470+0000 7ffab8c00640 1 -- 172.21.15.83:0/901367837 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ffa980a3f40 msgr2=0x7ffa980a6400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:51.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.470+0000 7ffab8c00640 1 --2- 172.21.15.83:0/901367837 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ffa980a3f40 0x7ffa980a6400 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ffaac0ce0d0 tx=0x7ffaac01cfd0 comp rx=0 tx=0).stop 2024-04-24T08:52:51.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.470+0000 7ffab8c00640 1 -- 172.21.15.83:0/901367837 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffab419a730 msgr2=0x7ffab419abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:51.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.470+0000 7ffab8c00640 1 --2- 172.21.15.83:0/901367837 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffab419a730 0x7ffab419abb0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7ffaac00eaa0 tx=0x7ffaac00ef60 comp rx=0 tx=0).stop 2024-04-24T08:52:51.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.470+0000 7ffab8c00640 1 -- 172.21.15.83:0/901367837 shutdown_connections 2024-04-24T08:52:51.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.470+0000 7ffab8c00640 1 --2- 172.21.15.83:0/901367837 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ffa980a3f40 0x7ffa980a6400 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:51.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.470+0000 7ffab8c00640 1 --2- 172.21.15.83:0/901367837 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffab419a730 0x7ffab419abb0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:51.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.470+0000 7ffab8c00640 1 --2- 172.21.15.83:0/901367837 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffab41071e0 0x7ffab41a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:51.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.470+0000 7ffab8c00640 1 --2- 172.21.15.83:0/901367837 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffab4102110 0x7ffab41a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:51.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.470+0000 7ffab8c00640 1 -- 172.21.15.83:0/901367837 >> 172.21.15.83:0/901367837 conn(0x7ffab40fd9d0 msgr2=0x7ffab40ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:51.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.470+0000 7ffab8c00640 1 -- 172.21.15.83:0/901367837 shutdown_connections 2024-04-24T08:52:51.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.470+0000 7ffab8c00640 1 -- 172.21.15.83:0/901367837 wait complete. 2024-04-24T08:52:51.492 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-24T08:52:51.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.674+0000 7fbac6e00640 1 -- 172.21.15.83:0/3694380722 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbac01071e0 msgr2=0x7fbac010edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:51.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.674+0000 7fbac6e00640 1 --2- 172.21.15.83:0/3694380722 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbac01071e0 0x7fbac010edb0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fbab8009ca0 tx=0x7fbab802a8c0 comp rx=0 tx=0).stop 2024-04-24T08:52:51.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.674+0000 7fbac6e00640 1 -- 172.21.15.83:0/3694380722 shutdown_connections 2024-04-24T08:52:51.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.674+0000 7fbac6e00640 1 --2- 172.21.15.83:0/3694380722 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbac01071e0 0x7fbac010edb0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:51.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.674+0000 7fbac6e00640 1 --2- 172.21.15.83:0/3694380722 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbac0102ac0 0x7fbac0106b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:51.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.674+0000 7fbac6e00640 1 --2- 172.21.15.83:0/3694380722 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbac0102110 0x7fbac01024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:51.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.674+0000 7fbac6e00640 1 -- 172.21.15.83:0/3694380722 >> 172.21.15.83:0/3694380722 conn(0x7fbac00fd9d0 msgr2=0x7fbac00ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:51.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.675+0000 7fbac6e00640 1 -- 172.21.15.83:0/3694380722 shutdown_connections 2024-04-24T08:52:51.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.675+0000 7fbac6e00640 1 -- 172.21.15.83:0/3694380722 wait complete. 2024-04-24T08:52:51.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.676+0000 7fbac6e00640 1 Processor -- start 2024-04-24T08:52:51.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.676+0000 7fbac6e00640 1 -- start start 2024-04-24T08:52:51.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.677+0000 7fbac6e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbac0102110 0x7fbac01a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:51.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.677+0000 7fbac6e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbac01071e0 0x7fbac01a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:51.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.677+0000 7fbac6e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbac019a6f0 0x7fbac019ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:51.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.677+0000 7fbac6e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbac0112150 con 0x7fbac019a6f0 2024-04-24T08:52:51.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.677+0000 7fbac6e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbac0111fd0 con 0x7fbac01071e0 2024-04-24T08:52:51.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.677+0000 7fbac6e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fbac01122d0 con 0x7fbac0102110 2024-04-24T08:52:51.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.677+0000 7fbac6400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbac019a6f0 0x7fbac019ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:51.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.677+0000 7fbac5a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbac0102110 0x7fbac01a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:51.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.677+0000 7fbac6400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbac019a6f0 0x7fbac019ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54458/0 (socket says 172.21.15.83:54458) 2024-04-24T08:52:51.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.677+0000 7fbac6400640 1 -- 172.21.15.83:0/2314227570 learned_addr learned my addr 172.21.15.83:0/2314227570 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:52:51.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.678+0000 7fbac6400640 1 -- 172.21.15.83:0/2314227570 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbac0102110 msgr2=0x7fbac01a0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:51.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.678+0000 7fbac6400640 1 --2- 172.21.15.83:0/2314227570 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbac0102110 0x7fbac01a0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:51.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.678+0000 7fbac6400640 1 -- 172.21.15.83:0/2314227570 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbac01071e0 msgr2=0x7fbac01a0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:51.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.678+0000 7fbac6400640 1 --2- 172.21.15.83:0/2314227570 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbac01071e0 0x7fbac01a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:51.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.678+0000 7fbac6400640 1 -- 172.21.15.83:0/2314227570 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbac019b2f0 con 0x7fbac019a6f0 2024-04-24T08:52:51.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.678+0000 7fbac6400640 1 --2- 172.21.15.83:0/2314227570 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbac019a6f0 0x7fbac019ab70 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fbab8009dc0 tx=0x7fbab8032ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:51.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.682+0000 7fbabe000640 1 -- 172.21.15.83:0/2314227570 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbab8003cf0 con 0x7fbac019a6f0 2024-04-24T08:52:51.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.682+0000 7fbabe000640 1 -- 172.21.15.83:0/2314227570 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbab80044f0 con 0x7fbac019a6f0 2024-04-24T08:52:51.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.682+0000 7fbac6e00640 1 -- 172.21.15.83:0/2314227570 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbac019b580 con 0x7fbac019a6f0 2024-04-24T08:52:51.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.682+0000 7fbabe000640 1 -- 172.21.15.83:0/2314227570 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbab8030bf0 con 0x7fbac019a6f0 2024-04-24T08:52:51.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.682+0000 7fbac6e00640 1 -- 172.21.15.83:0/2314227570 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbac01a7770 con 0x7fbac019a6f0 2024-04-24T08:52:51.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.686+0000 7fbacb232740 1 Processor -- start 2024-04-24T08:52:51.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.686+0000 7fbabe000640 1 -- 172.21.15.83:0/2314227570 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7fbab8004040 con 0x7fbac019a6f0 2024-04-24T08:52:51.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.687+0000 7fbacb232740 1 -- start start 2024-04-24T08:52:51.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.687+0000 7fbacb232740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558ad7ebfb40 0x558ad7ebff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:51.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.687+0000 7fbacb232740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558ad7ec0460 0x558ad7eca5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:51.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.687+0000 7fbacb232740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ad7ecab30 0x558ad7eccf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:51.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.687+0000 7fbacb232740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x558ad7e251d0 con 0x558ad7ebfb40 2024-04-24T08:52:51.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.687+0000 7fbacb232740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558ad7dcb450 con 0x558ad7ecab30 2024-04-24T08:52:51.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.687+0000 7fbacb232740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x558ad7e21190 con 0x558ad7ec0460 2024-04-24T08:52:51.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.687+0000 7fbac5000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558ad7ebfb40 0x558ad7ebff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:51.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.688+0000 7fbac5000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558ad7ebfb40 0x558ad7ebff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54462/0 (socket says 172.21.15.83:54462) 2024-04-24T08:52:51.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.688+0000 7fbac5000640 1 -- 172.21.15.83:0/1444174791 learned_addr learned my addr 172.21.15.83:0/1444174791 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:52:51.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.688+0000 7fbac5000640 1 -- 172.21.15.83:0/1444174791 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558ad7ec0460 msgr2=0x558ad7eca5f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T08:52:51.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.688+0000 7fbac5000640 1 --2- 172.21.15.83:0/1444174791 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558ad7ec0460 0x558ad7eca5f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:51.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.688+0000 7fbac5000640 1 -- 172.21.15.83:0/1444174791 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ad7ecab30 msgr2=0x558ad7eccf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:51.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.688+0000 7fbac5000640 1 --2- 172.21.15.83:0/1444174791 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ad7ecab30 0x558ad7eccf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:51.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.688+0000 7fbac5000640 1 -- 172.21.15.83:0/1444174791 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ad7e88310 con 0x558ad7ebfb40 2024-04-24T08:52:51.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.688+0000 7fbabe000640 1 --2- 172.21.15.83:0/2314227570 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fbaa80a3ec0 0x7fbaa80a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:51.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.689+0000 7fbac5000640 1 --2- 172.21.15.83:0/1444174791 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558ad7ebfb40 0x558ad7ebff20 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fbaac00bac0 tx=0x7fbaac00bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b554e1d2da21b21 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:51.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.689+0000 7fbabe000640 1 -- 172.21.15.83:0/2314227570 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7fbab80f2500 con 0x7fbac019a6f0 2024-04-24T08:52:51.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.689+0000 7fbac5a00640 1 --2- 172.21.15.83:0/2314227570 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fbaa80a3ec0 0x7fbaa80a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:51.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.689+0000 7fbab2a00640 1 -- 172.21.15.83:0/1444174791 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaac015030 con 0x558ad7ebfb40 2024-04-24T08:52:51.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.689+0000 7fbab2a00640 1 -- 172.21.15.83:0/1444174791 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbaac005020 con 0x558ad7ebfb40 2024-04-24T08:52:51.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.689+0000 7fbab2a00640 1 -- 172.21.15.83:0/1444174791 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaac004be0 con 0x558ad7ebfb40 2024-04-24T08:52:51.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.690+0000 7fbacb232740 1 -- 172.21.15.83:0/1444174791 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558ad7ebfb40 msgr2=0x558ad7ebff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:51.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.690+0000 7fbac5a00640 1 --2- 172.21.15.83:0/2314227570 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fbaa80a3ec0 0x7fbaa80a6380 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x558ad7ec1100 tx=0x7fbab400b390 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:51.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.690+0000 7fbacb232740 1 --2- 172.21.15.83:0/1444174791 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558ad7ebfb40 0x558ad7ebff20 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fbaac00bac0 tx=0x7fbaac00bf80 comp rx=0 tx=0).stop 2024-04-24T08:52:51.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.690+0000 7fbacb232740 1 -- 172.21.15.83:0/1444174791 shutdown_connections 2024-04-24T08:52:51.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.690+0000 7fbacb232740 1 --2- 172.21.15.83:0/1444174791 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ad7ecab30 0x558ad7eccf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:51.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.690+0000 7fbacb232740 1 --2- 172.21.15.83:0/1444174791 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558ad7ec0460 0x558ad7eca5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:51.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.690+0000 7fbacb232740 1 --2- 172.21.15.83:0/1444174791 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558ad7ebfb40 0x558ad7ebff20 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:51.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.690+0000 7fbacb232740 1 -- 172.21.15.83:0/1444174791 >> 172.21.15.83:0/1444174791 conn(0x558ad7ebbe60 msgr2=0x558ad7ebc240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:51.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.691+0000 7fbacb232740 1 -- 172.21.15.83:0/1444174791 shutdown_connections 2024-04-24T08:52:51.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.691+0000 7fbacb232740 1 -- 172.21.15.83:0/1444174791 wait complete. 2024-04-24T08:52:51.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.691+0000 7fbacb232740 1 -- start start 2024-04-24T08:52:51.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.692+0000 7fbacb232740 1 Processor -- start 2024-04-24T08:52:51.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.692+0000 7fbacb232740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ad7ebfb40 0x558ad7eca5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:51.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.692+0000 7fbacb232740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558ad7ec0460 0x558ad7ec7930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:51.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.692+0000 7fbacb232740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558ad7ecab30 0x558ad7ec8020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:51.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.692+0000 7fbacb232740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x558ad7ecd9c0 con 0x558ad7ec0460 2024-04-24T08:52:51.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.692+0000 7fbacb232740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558ad7dde810 con 0x558ad7ebfb40 2024-04-24T08:52:51.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.692+0000 7fbacb232740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x558ad7ecdb40 con 0x558ad7ecab30 2024-04-24T08:52:51.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.692+0000 7fbacb232740 10 client.0 _ll_register_callbacks cb 0x558ad7dac140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:52:51.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.693+0000 7fbac5000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558ad7ec0460 0x558ad7ec7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:51.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.693+0000 7fbac5000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558ad7ec0460 0x558ad7ec7930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54464/0 (socket says 172.21.15.83:54464) 2024-04-24T08:52:51.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.693+0000 7fbac5000640 1 -- 172.21.15.83:0/3754422323 learned_addr learned my addr 172.21.15.83:0/3754422323 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:52:51.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.693+0000 7fbac5a00640 1 --2- 172.21.15.83:0/3754422323 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ad7ebfb40 0x558ad7eca5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:51.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.693+0000 7fbac6400640 1 --2- 172.21.15.83:0/3754422323 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558ad7ecab30 0x558ad7ec8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:51.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.693+0000 7fbac5000640 1 -- 172.21.15.83:0/3754422323 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558ad7ecab30 msgr2=0x558ad7ec8020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:51.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.693+0000 7fbac5000640 1 --2- 172.21.15.83:0/3754422323 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558ad7ecab30 0x558ad7ec8020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:51.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.693+0000 7fbac5000640 1 -- 172.21.15.83:0/3754422323 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ad7ebfb40 msgr2=0x558ad7eca5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:51.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.693+0000 7fbac5000640 1 --2- 172.21.15.83:0/3754422323 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ad7ebfb40 0x558ad7eca5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:51.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.693+0000 7fbac5000640 1 -- 172.21.15.83:0/3754422323 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ad7def3f0 con 0x558ad7ec0460 2024-04-24T08:52:51.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.693+0000 7fbac6400640 1 --2- 172.21.15.83:0/3754422323 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558ad7ecab30 0x558ad7ec8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:52:51.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.693+0000 7fbac5000640 1 --2- 172.21.15.83:0/3754422323 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558ad7ec0460 0x558ad7ec7930 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fbaac005400 tx=0x7fbaac005430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90a36a3b6e9b879b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:51.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.693+0000 7fbaa7400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:52:51.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.694+0000 7fbaa7400640 1 -- 172.21.15.83:0/3754422323 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaac015070 con 0x558ad7ec0460 2024-04-24T08:52:51.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.694+0000 7fbaa7400640 1 -- 172.21.15.83:0/3754422323 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7fbaac004d30 con 0x558ad7ec0460 2024-04-24T08:52:51.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.694+0000 7fbacb232740 1 -- 172.21.15.83:0/3754422323 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbaac005020 con 0x558ad7ec0460 2024-04-24T08:52:51.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.694+0000 7fbaa7400640 1 -- 172.21.15.83:0/3754422323 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbaac01a460 con 0x558ad7ec0460 2024-04-24T08:52:51.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.694+0000 7fbacb232740 10 client.16119 fetch_fsmap learned FSMap version 28 2024-04-24T08:52:51.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.694+0000 7fbacb232740 1 -- 172.21.15.83:0/3754422323 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558ad7b373f0 con 0x558ad7ec0460 2024-04-24T08:52:51.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.695+0000 7fbaa7400640 1 -- 172.21.15.83:0/3754422323 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbaac005020 con 0x558ad7ec0460 2024-04-24T08:52:51.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.696+0000 7fbaa7400640 1 -- 172.21.15.83:0/3754422323 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fbaac022070 con 0x558ad7ec0460 2024-04-24T08:52:51.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.696+0000 7fbacb232740 10 client.16119 fetch_fsmap finished waiting for FSMap version 28 2024-04-24T08:52:51.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.696+0000 7fbacb232740 10 client.16119 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-24T08:52:51.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.696+0000 7fbacb232740 20 client.16119 populate_metadata read hostname 'smithi083' 2024-04-24T08:52:51.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.696+0000 7fbacb232740 1 --2- 172.21.15.83:0/3754422323 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x558ad7edad30 0x558ad7edd120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:51.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.696+0000 7fbacb232740 4 client.16119 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:52:51.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.696+0000 7fbacb232740 1 -- 172.21.15.83:0/3754422323 --> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558ad7e32bf0 con 0x558ad7edad30 2024-04-24T08:52:51.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.696+0000 7fbac5a00640 1 --2- 172.21.15.83:0/3754422323 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x558ad7edad30 0x558ad7edd120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:51.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.697+0000 7fbac5a00640 1 --2- 172.21.15.83:0/3754422323 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x558ad7edad30 0x558ad7edd120 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e22364bcfacb8fbb server_cookie=c4ce017fce5407b7 in_seq=0 out_seq=0 2024-04-24T08:52:51.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.697+0000 7fbaa7400640 10 client.16119 ms_handle_connect on v2:172.21.15.138:6832/1157967658 2024-04-24T08:52:51.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.699+0000 7fbaa7400640 1 -- 172.21.15.83:0/3754422323 <== mds.3 v2:172.21.15.138:6832/1157967658 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558ad7e32bf0 con 0x558ad7edad30 2024-04-24T08:52:51.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.699+0000 7fbaa7400640 10 client.16119 handle_command_reply: tid=0 2024-04-24T08:52:51.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.699+0000 7fbacb232740 1 client.16119 shutdown 2024-04-24T08:52:51.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.699+0000 7fbacb232740 1 -- 172.21.15.83:0/3754422323 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558ad7ec0460 msgr2=0x558ad7ec7930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:51.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.699+0000 7fbacb232740 1 --2- 172.21.15.83:0/3754422323 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558ad7ec0460 0x558ad7ec7930 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fbaac005400 tx=0x7fbaac005430 comp rx=0 tx=0).stop 2024-04-24T08:52:51.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.700+0000 7fbacb232740 1 -- 172.21.15.83:0/3754422323 shutdown_connections 2024-04-24T08:52:51.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.700+0000 7fbacb232740 1 --2- 172.21.15.83:0/3754422323 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x558ad7edad30 0x558ad7edd120 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:51.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.700+0000 7fbacb232740 1 --2- 172.21.15.83:0/3754422323 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558ad7ecab30 0x558ad7ec8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:51.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.700+0000 7fbacb232740 1 --2- 172.21.15.83:0/3754422323 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558ad7ec0460 0x558ad7ec7930 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:51.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.700+0000 7fbacb232740 1 --2- 172.21.15.83:0/3754422323 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ad7ebfb40 0x558ad7eca5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:51.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.700+0000 7fbacb232740 1 -- 172.21.15.83:0/3754422323 >> 172.21.15.83:0/3754422323 conn(0x558ad7ebbe60 msgr2=0x558ad7ec1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:51.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.700+0000 7fbaa7400640 0 client.16119 ms_handle_reset on v2:172.21.15.138:6832/1157967658 2024-04-24T08:52:51.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.700+0000 7fbacb232740 1 -- 172.21.15.83:0/3754422323 shutdown_connections 2024-04-24T08:52:51.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.700+0000 7fbacb232740 1 -- 172.21.15.83:0/3754422323 wait complete. 2024-04-24T08:52:51.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.700+0000 7fbacb232740 20 client.16119 trim_cache size 0 max 16384 2024-04-24T08:52:51.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.727+0000 7fbacb232740 1 Processor -- start 2024-04-24T08:52:51.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.727+0000 7fbacb232740 1 -- start start 2024-04-24T08:52:51.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.727+0000 7fbacb232740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ad7ebfb40 0x558ad7ed68b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:51.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.727+0000 7fbacb232740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558ad7ec0460 0x558ad7ee28b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:51.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.727+0000 7fbacb232740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558ad7ecab30 0x558ad7ee2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:51.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.727+0000 7fbacb232740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbac01b1500 con 0x558ad7ecab30 2024-04-24T08:52:51.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.727+0000 7fbacb232740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbac01b3040 con 0x558ad7ebfb40 2024-04-24T08:52:51.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.727+0000 7fbacb232740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fbac01b1bd0 con 0x558ad7ec0460 2024-04-24T08:52:51.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.728+0000 7fbac6400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558ad7ecab30 0x558ad7ee2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:51.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.728+0000 7fbac6400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558ad7ecab30 0x558ad7ee2df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54466/0 (socket says 172.21.15.83:54466) 2024-04-24T08:52:51.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.728+0000 7fbac6400640 1 -- 172.21.15.83:0/3682894874 learned_addr learned my addr 172.21.15.83:0/3682894874 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:52:51.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.728+0000 7fbac5a00640 1 --2- 172.21.15.83:0/3682894874 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ad7ebfb40 0x558ad7ed68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:51.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.728+0000 7fbac5000640 1 --2- 172.21.15.83:0/3682894874 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558ad7ec0460 0x558ad7ee28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:51.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.728+0000 7fbac6400640 1 -- 172.21.15.83:0/3682894874 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558ad7ec0460 msgr2=0x558ad7ee28b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:51.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.728+0000 7fbac6400640 1 --2- 172.21.15.83:0/3682894874 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558ad7ec0460 0x558ad7ee28b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:51.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.728+0000 7fbac6400640 1 -- 172.21.15.83:0/3682894874 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ad7ebfb40 msgr2=0x558ad7ed68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:51.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.728+0000 7fbac6400640 1 --2- 172.21.15.83:0/3682894874 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ad7ebfb40 0x558ad7ed68b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:51.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.728+0000 7fbac6400640 1 -- 172.21.15.83:0/3682894874 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ad7ec0290 con 0x558ad7ecab30 2024-04-24T08:52:51.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.728+0000 7fbac5a00640 1 --2- 172.21.15.83:0/3682894874 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ad7ebfb40 0x558ad7ed68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:52:51.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.728+0000 7fbac6400640 1 --2- 172.21.15.83:0/3682894874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558ad7ecab30 0x558ad7ee2df0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fbab8005e30 tx=0x7fbab80305c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aea4ccdc507acd92 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:51.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.729+0000 7fbab2000640 1 -- 172.21.15.83:0/3682894874 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbab80d6020 con 0x558ad7ecab30 2024-04-24T08:52:51.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.729+0000 7fbab2000640 1 -- 172.21.15.83:0/3682894874 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbab8036030 con 0x558ad7ecab30 2024-04-24T08:52:51.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.729+0000 7fbab2000640 1 -- 172.21.15.83:0/3682894874 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbab80bc360 con 0x558ad7ecab30 2024-04-24T08:52:51.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.730+0000 7fbacb232740 1 -- 172.21.15.83:0/3682894874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558ad7ecab30 msgr2=0x558ad7ee2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:51.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.730+0000 7fbacb232740 1 --2- 172.21.15.83:0/3682894874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558ad7ecab30 0x558ad7ee2df0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fbab8005e30 tx=0x7fbab80305c0 comp rx=0 tx=0).stop 2024-04-24T08:52:51.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.730+0000 7fbacb232740 1 -- 172.21.15.83:0/3682894874 shutdown_connections 2024-04-24T08:52:51.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.730+0000 7fbacb232740 1 --2- 172.21.15.83:0/3682894874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558ad7ecab30 0x558ad7ee2df0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:51.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.730+0000 7fbacb232740 1 --2- 172.21.15.83:0/3682894874 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558ad7ec0460 0x558ad7ee28b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:51.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.730+0000 7fbacb232740 1 --2- 172.21.15.83:0/3682894874 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ad7ebfb40 0x558ad7ed68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:51.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.730+0000 7fbacb232740 1 -- 172.21.15.83:0/3682894874 >> 172.21.15.83:0/3682894874 conn(0x558ad7ebbe60 msgr2=0x558ad7ebc240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:51.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.730+0000 7fbacb232740 1 -- 172.21.15.83:0/3682894874 shutdown_connections 2024-04-24T08:52:51.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.730+0000 7fbacb232740 1 -- 172.21.15.83:0/3682894874 wait complete. 2024-04-24T08:52:51.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.730+0000 7fbacb232740 1 -- start start 2024-04-24T08:52:51.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.730+0000 7fbacb232740 1 Processor -- start 2024-04-24T08:52:51.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.731+0000 7fbacb232740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558ad7ebfb40 0x558ad7ed2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:51.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.731+0000 7fbacb232740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558ad7ec0460 0x558ad7ed3300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:51.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.731+0000 7fbacb232740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ad7ecab30 0x558ad7ec4d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:51.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.731+0000 7fbacb232740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbac01b2970 con 0x558ad7ec0460 2024-04-24T08:52:51.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.731+0000 7fbacb232740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbac01b22a0 con 0x558ad7ecab30 2024-04-24T08:52:51.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.731+0000 7fbacb232740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fbac01af4e0 con 0x558ad7ebfb40 2024-04-24T08:52:51.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.731+0000 7fbacb232740 10 client.0 _ll_register_callbacks cb 0x7fbaac00b9c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:52:51.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.731+0000 7fbac5000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558ad7ec0460 0x558ad7ed3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:51.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.731+0000 7fbac5000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558ad7ec0460 0x558ad7ed3300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54478/0 (socket says 172.21.15.83:54478) 2024-04-24T08:52:51.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.731+0000 7fbac5000640 1 -- 172.21.15.83:0/1218799533 learned_addr learned my addr 172.21.15.83:0/1218799533 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:52:51.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.732+0000 7fbac5a00640 1 --2- 172.21.15.83:0/1218799533 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558ad7ebfb40 0x558ad7ed2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:51.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.732+0000 7fbac5000640 1 -- 172.21.15.83:0/1218799533 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558ad7ebfb40 msgr2=0x558ad7ed2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:51.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.732+0000 7fbac5000640 1 --2- 172.21.15.83:0/1218799533 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558ad7ebfb40 0x558ad7ed2dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:51.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.732+0000 7fbac5000640 1 -- 172.21.15.83:0/1218799533 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ad7ecab30 msgr2=0x558ad7ec4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:51.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.732+0000 7fbac5000640 1 --2- 172.21.15.83:0/1218799533 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ad7ecab30 0x558ad7ec4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:51.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.732+0000 7fbac5000640 1 -- 172.21.15.83:0/1218799533 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ad7c017d0 con 0x558ad7ec0460 2024-04-24T08:52:51.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.732+0000 7fbac5000640 1 --2- 172.21.15.83:0/1218799533 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558ad7ec0460 0x558ad7ed3300 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fbaac00c010 tx=0x7fbaac00ff60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a94a2551e08a7a8b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:51.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.732+0000 7fbaa7400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:52:51.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.733+0000 7fbaa7400640 1 -- 172.21.15.83:0/1218799533 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaac035060 con 0x558ad7ec0460 2024-04-24T08:52:51.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.733+0000 7fbaa7400640 1 -- 172.21.15.83:0/1218799533 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7fbaac004d30 con 0x558ad7ec0460 2024-04-24T08:52:51.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.733+0000 7fbacb232740 1 -- 172.21.15.83:0/1218799533 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbab8036030 con 0x558ad7ec0460 2024-04-24T08:52:51.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.733+0000 7fbaa7400640 1 -- 172.21.15.83:0/1218799533 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbaac03c050 con 0x558ad7ec0460 2024-04-24T08:52:51.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.733+0000 7fbacb232740 10 client.16125 fetch_fsmap learned FSMap version 28 2024-04-24T08:52:51.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.733+0000 7fbacb232740 1 -- 172.21.15.83:0/1218799533 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558ad7ecbc40 con 0x558ad7ec0460 2024-04-24T08:52:51.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.733+0000 7fbaa7400640 1 -- 172.21.15.83:0/1218799533 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbab8036030 con 0x558ad7ec0460 2024-04-24T08:52:51.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.733+0000 7fbaa7400640 1 -- 172.21.15.83:0/1218799533 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fbaac022040 con 0x558ad7ec0460 2024-04-24T08:52:51.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.734+0000 7fbacb232740 10 client.16125 fetch_fsmap finished waiting for FSMap version 28 2024-04-24T08:52:51.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.734+0000 7fbacb232740 10 client.16125 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-24T08:52:51.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.734+0000 7fbacb232740 20 client.16125 populate_metadata read hostname 'smithi083' 2024-04-24T08:52:51.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.734+0000 7fbacb232740 1 --2- 172.21.15.83:0/1218799533 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x558ad7f41230 0x558ad7f43650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:51.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.734+0000 7fbacb232740 4 client.16125 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T08:52:51.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.734+0000 7fbacb232740 1 -- 172.21.15.83:0/1218799533 --> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558ad7e2ca60 con 0x558ad7f41230 2024-04-24T08:52:51.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.735+0000 7fbac5a00640 1 --2- 172.21.15.83:0/1218799533 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x558ad7f41230 0x558ad7f43650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:51.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.735+0000 7fbac5a00640 1 --2- 172.21.15.83:0/1218799533 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x558ad7f41230 0x558ad7f43650 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d99ba2640b198c9b server_cookie=5610b8baef5e75ef in_seq=0 out_seq=0 2024-04-24T08:52:51.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.735+0000 7fbaa7400640 10 client.16125 ms_handle_connect on v2:172.21.15.138:6832/1157967658 2024-04-24T08:52:51.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.736+0000 7fbaa7400640 1 -- 172.21.15.83:0/1218799533 <== mds.3 v2:172.21.15.138:6832/1157967658 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558ad7e2ca60 con 0x558ad7f41230 2024-04-24T08:52:51.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.736+0000 7fbaa7400640 10 client.16125 handle_command_reply: tid=0 2024-04-24T08:52:51.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.736+0000 7fbacb232740 1 client.16125 shutdown 2024-04-24T08:52:51.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.736+0000 7fbacb232740 1 -- 172.21.15.83:0/1218799533 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558ad7ec0460 msgr2=0x558ad7ed3300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:51.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.736+0000 7fbacb232740 1 --2- 172.21.15.83:0/1218799533 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558ad7ec0460 0x558ad7ed3300 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fbaac00c010 tx=0x7fbaac00ff60 comp rx=0 tx=0).stop 2024-04-24T08:52:51.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.736+0000 7fbacb232740 1 -- 172.21.15.83:0/1218799533 shutdown_connections 2024-04-24T08:52:51.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.736+0000 7fbacb232740 1 --2- 172.21.15.83:0/1218799533 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ad7ecab30 0x558ad7ec4d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:51.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.736+0000 7fbacb232740 1 --2- 172.21.15.83:0/1218799533 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558ad7ec0460 0x558ad7ed3300 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:51.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.736+0000 7fbacb232740 1 --2- 172.21.15.83:0/1218799533 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x558ad7f41230 0x558ad7f43650 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:51.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.736+0000 7fbacb232740 1 --2- 172.21.15.83:0/1218799533 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558ad7ebfb40 0x558ad7ed2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:51.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.736+0000 7fbacb232740 1 -- 172.21.15.83:0/1218799533 >> 172.21.15.83:0/1218799533 conn(0x558ad7ebbe60 msgr2=0x558ad7ec2bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:51.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.736+0000 7fbaa7400640 0 client.16125 ms_handle_reset on v2:172.21.15.138:6832/1157967658 2024-04-24T08:52:51.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.737+0000 7fbacb232740 1 -- 172.21.15.83:0/1218799533 shutdown_connections 2024-04-24T08:52:51.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.737+0000 7fbacb232740 1 -- 172.21.15.83:0/1218799533 wait complete. 2024-04-24T08:52:51.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.737+0000 7fbacb232740 20 client.16125 trim_cache size 0 max 16384 2024-04-24T08:52:51.738 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T08:52:51.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.737+0000 7fbac6e00640 1 -- 172.21.15.83:0/2314227570 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fbaa80a3ec0 msgr2=0x7fbaa80a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:51.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.737+0000 7fbac6e00640 1 --2- 172.21.15.83:0/2314227570 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fbaa80a3ec0 0x7fbaa80a6380 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x558ad7ec1100 tx=0x7fbab400b390 comp rx=0 tx=0).stop 2024-04-24T08:52:51.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.737+0000 7fbac6e00640 1 -- 172.21.15.83:0/2314227570 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbac019a6f0 msgr2=0x7fbac019ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:51.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.737+0000 7fbac6e00640 1 --2- 172.21.15.83:0/2314227570 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbac019a6f0 0x7fbac019ab70 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fbab8009dc0 tx=0x7fbab8032ff0 comp rx=0 tx=0).stop 2024-04-24T08:52:51.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.738+0000 7fbac6e00640 1 -- 172.21.15.83:0/2314227570 shutdown_connections 2024-04-24T08:52:51.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.738+0000 7fbac6e00640 1 --2- 172.21.15.83:0/2314227570 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fbaa80a3ec0 0x7fbaa80a6380 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:51.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.738+0000 7fbac6e00640 1 --2- 172.21.15.83:0/2314227570 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbac019a6f0 0x7fbac019ab70 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:51.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.738+0000 7fbac6e00640 1 --2- 172.21.15.83:0/2314227570 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbac01071e0 0x7fbac01a0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:51.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.738+0000 7fbac6e00640 1 --2- 172.21.15.83:0/2314227570 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbac0102110 0x7fbac01a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:51.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.738+0000 7fbac6e00640 1 -- 172.21.15.83:0/2314227570 >> 172.21.15.83:0/2314227570 conn(0x7fbac00fd9d0 msgr2=0x7fbac00fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:51.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.738+0000 7fbac6e00640 1 -- 172.21.15.83:0/2314227570 shutdown_connections 2024-04-24T08:52:51.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.738+0000 7fbac6e00640 1 -- 172.21.15.83:0/2314227570 wait complete. 2024-04-24T08:52:51.758 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T08:52:51.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:52:51 smithi083 ceph-mon[27907]: pgmap v40: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-24T08:52:51.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:52:51 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/2316371904' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:52:51.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:52:51 smithi138 ceph-mon[31441]: pgmap v40: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-24T08:52:51.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:52:51 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/2316371904' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:52:51.956 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:52:51 smithi189 ceph-mon[31791]: pgmap v40: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-24T08:52:51.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:52:51 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/2316371904' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:52:51.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.976+0000 7f297b200640 1 -- 172.21.15.83:0/2140832364 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f297c101000 msgr2=0x7f297c1013e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:51.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.976+0000 7f297b200640 1 --2- 172.21.15.83:0/2140832364 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f297c101000 0x7f297c1013e0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f296c009900 tx=0x7f296c02a3f0 comp rx=0 tx=0).stop 2024-04-24T08:52:51.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.976+0000 7f297b200640 1 -- 172.21.15.83:0/2140832364 shutdown_connections 2024-04-24T08:52:51.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.976+0000 7f297b200640 1 --2- 172.21.15.83:0/2140832364 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f297c1060d0 0x7f297c10dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:51.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.976+0000 7f297b200640 1 --2- 172.21.15.83:0/2140832364 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f297c1019b0 0x7f297c1059f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:51.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.976+0000 7f297b200640 1 --2- 172.21.15.83:0/2140832364 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f297c101000 0x7f297c1013e0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:51.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.976+0000 7f297b200640 1 -- 172.21.15.83:0/2140832364 >> 172.21.15.83:0/2140832364 conn(0x7f297c0fc880 msgr2=0x7f297c0feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:51.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.976+0000 7f297b200640 1 -- 172.21.15.83:0/2140832364 shutdown_connections 2024-04-24T08:52:51.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.976+0000 7f297b200640 1 -- 172.21.15.83:0/2140832364 wait complete. 2024-04-24T08:52:51.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.977+0000 7f297b200640 1 Processor -- start 2024-04-24T08:52:51.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.978+0000 7f297b200640 1 -- start start 2024-04-24T08:52:51.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.978+0000 7f297b200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f297c101000 0x7f297c1a05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:51.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.978+0000 7f297b200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f297c1060d0 0x7f297c1a0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:51.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.978+0000 7f297b200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f297c19a660 0x7f297c19aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:51.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.978+0000 7f297b200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f297c111040 con 0x7f297c19a660 2024-04-24T08:52:51.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.978+0000 7f297b200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f297c110ec0 con 0x7f297c1060d0 2024-04-24T08:52:51.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.978+0000 7f297b200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f297c1111c0 con 0x7f297c101000 2024-04-24T08:52:51.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.978+0000 7f297a800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f297c19a660 0x7f297c19aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:51.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.979+0000 7f297a800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f297c19a660 0x7f297c19aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54490/0 (socket says 172.21.15.83:54490) 2024-04-24T08:52:51.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.979+0000 7f297a800640 1 -- 172.21.15.83:0/1309366807 learned_addr learned my addr 172.21.15.83:0/1309366807 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:52:51.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.979+0000 7f2979e00640 1 --2- 172.21.15.83:0/1309366807 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f297c101000 0x7f297c1a05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:51.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.979+0000 7f2979400640 1 --2- 172.21.15.83:0/1309366807 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f297c1060d0 0x7f297c1a0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:51.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.979+0000 7f297a800640 1 -- 172.21.15.83:0/1309366807 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f297c101000 msgr2=0x7f297c1a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:51.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.979+0000 7f297a800640 1 --2- 172.21.15.83:0/1309366807 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f297c101000 0x7f297c1a05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:51.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.979+0000 7f297a800640 1 -- 172.21.15.83:0/1309366807 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f297c1060d0 msgr2=0x7f297c1a0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:51.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.979+0000 7f297a800640 1 --2- 172.21.15.83:0/1309366807 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f297c1060d0 0x7f297c1a0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:51.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.979+0000 7f297a800640 1 -- 172.21.15.83:0/1309366807 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f297c19b260 con 0x7f297c19a660 2024-04-24T08:52:51.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.979+0000 7f2979400640 1 --2- 172.21.15.83:0/1309366807 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f297c1060d0 0x7f297c1a0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:52:51.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.980+0000 7f297a800640 1 --2- 172.21.15.83:0/1309366807 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f297c19a660 0x7f297c19aae0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f297400eaa0 tx=0x7f297400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:51.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.980+0000 7f2972a00640 1 -- 172.21.15.83:0/1309366807 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f297400cdb0 con 0x7f297c19a660 2024-04-24T08:52:51.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.980+0000 7f2972a00640 1 -- 172.21.15.83:0/1309366807 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2974005020 con 0x7f297c19a660 2024-04-24T08:52:51.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.980+0000 7f2972a00640 1 -- 172.21.15.83:0/1309366807 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29740116d0 con 0x7f297c19a660 2024-04-24T08:52:51.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.980+0000 7f297b200640 1 -- 172.21.15.83:0/1309366807 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f297c1a7340 con 0x7f297c19a660 2024-04-24T08:52:51.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.980+0000 7f297b200640 1 -- 172.21.15.83:0/1309366807 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f297c1a76f0 con 0x7f297c19a660 2024-04-24T08:52:51.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.984+0000 7f29837fb740 1 Processor -- start 2024-04-24T08:52:51.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.984+0000 7f2972a00640 1 -- 172.21.15.83:0/1309366807 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7f29740051d0 con 0x7f297c19a660 2024-04-24T08:52:51.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.985+0000 7f29837fb740 1 -- start start 2024-04-24T08:52:51.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.985+0000 7f29837fb740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5557b01b0b40 0x5557b01b0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:51.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.985+0000 7f29837fb740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5557b01b1460 0x5557b01bb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:51.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.985+0000 7f29837fb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5557b01bbb30 0x5557b01bdf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:51.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.985+0000 7f29837fb740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5557b01161d0 con 0x5557b01b1460 2024-04-24T08:52:51.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.985+0000 7f29837fb740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5557b00bc450 con 0x5557b01bbb30 2024-04-24T08:52:51.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.985+0000 7f29837fb740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5557b0112190 con 0x5557b01b0b40 2024-04-24T08:52:51.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.986+0000 7f2979e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5557b01b1460 0x5557b01bb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:51.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.986+0000 7f2979400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5557b01b0b40 0x5557b01b0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:51.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.986+0000 7f2979e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5557b01b1460 0x5557b01bb5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54498/0 (socket says 172.21.15.83:54498) 2024-04-24T08:52:51.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.986+0000 7f2979e00640 1 -- 172.21.15.83:0/2373451206 learned_addr learned my addr 172.21.15.83:0/2373451206 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:52:51.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.986+0000 7f2979400640 1 --2- 172.21.15.83:0/2373451206 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5557b01bbb30 0x5557b01bdf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:51.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.986+0000 7f2979e00640 1 -- 172.21.15.83:0/2373451206 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5557b01b0b40 msgr2=0x5557b01b0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:51.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.986+0000 7f2979e00640 1 --2- 172.21.15.83:0/2373451206 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5557b01b0b40 0x5557b01b0f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:51.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.986+0000 7f2979e00640 1 -- 172.21.15.83:0/2373451206 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5557b01bbb30 msgr2=0x5557b01bdf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:51.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.986+0000 7f2979e00640 1 --2- 172.21.15.83:0/2373451206 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5557b01bbb30 0x5557b01bdf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:51.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.986+0000 7f2979e00640 1 -- 172.21.15.83:0/2373451206 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557b0179310 con 0x5557b01b1460 2024-04-24T08:52:51.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.986+0000 7f2972a00640 1 --2- 172.21.15.83:0/1309366807 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f29600a3e40 0x7f29600a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:51.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.987+0000 7f2972a00640 1 -- 172.21.15.83:0/1309366807 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f29740d0c80 con 0x7f297c19a660 2024-04-24T08:52:51.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.987+0000 7f2979400640 1 --2- 172.21.15.83:0/1309366807 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f29600a3e40 0x7f29600a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:51.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.987+0000 7f2979e00640 1 --2- 172.21.15.83:0/2373451206 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5557b01b1460 0x5557b01bb5f0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f296c009900 tx=0x7f296c032e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80b9671d955e369c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:51.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.988+0000 7f2979400640 1 --2- 172.21.15.83:0/1309366807 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f29600a3e40 0x7f29600a6300 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x5557b01adff0 tx=0x7f2968015040 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:51.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.988+0000 7f2967000640 1 -- 172.21.15.83:0/2373451206 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f296c040070 con 0x5557b01b1460 2024-04-24T08:52:51.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.988+0000 7f2967000640 1 -- 172.21.15.83:0/2373451206 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f296c0316a0 con 0x5557b01b1460 2024-04-24T08:52:51.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.988+0000 7f2967000640 1 -- 172.21.15.83:0/2373451206 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f296c035040 con 0x5557b01b1460 2024-04-24T08:52:51.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.989+0000 7f29837fb740 1 -- 172.21.15.83:0/2373451206 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5557b01b1460 msgr2=0x5557b01bb5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:51.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.989+0000 7f29837fb740 1 --2- 172.21.15.83:0/2373451206 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5557b01b1460 0x5557b01bb5f0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f296c009900 tx=0x7f296c032e40 comp rx=0 tx=0).stop 2024-04-24T08:52:51.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.989+0000 7f29837fb740 1 -- 172.21.15.83:0/2373451206 shutdown_connections 2024-04-24T08:52:51.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.989+0000 7f29837fb740 1 --2- 172.21.15.83:0/2373451206 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5557b01bbb30 0x5557b01bdf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:51.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.989+0000 7f29837fb740 1 --2- 172.21.15.83:0/2373451206 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5557b01b1460 0x5557b01bb5f0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:51.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.989+0000 7f29837fb740 1 --2- 172.21.15.83:0/2373451206 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5557b01b0b40 0x5557b01b0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:51.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.989+0000 7f29837fb740 1 -- 172.21.15.83:0/2373451206 >> 172.21.15.83:0/2373451206 conn(0x5557b01ace60 msgr2=0x5557b01ad240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:51.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.989+0000 7f29837fb740 1 -- 172.21.15.83:0/2373451206 shutdown_connections 2024-04-24T08:52:51.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.989+0000 7f29837fb740 1 -- 172.21.15.83:0/2373451206 wait complete. 2024-04-24T08:52:51.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.989+0000 7f29837fb740 1 -- start start 2024-04-24T08:52:51.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.990+0000 7f29837fb740 1 Processor -- start 2024-04-24T08:52:51.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.990+0000 7f29837fb740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5557b01b0b40 0x5557b01bb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:51.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.990+0000 7f29837fb740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5557b01b1460 0x5557b01b88a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:51.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.990+0000 7f29837fb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5557b01bbb30 0x5557b01b8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:51.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.990+0000 7f29837fb740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5557b01be960 con 0x5557b01b0b40 2024-04-24T08:52:51.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.990+0000 7f29837fb740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5557b00cf810 con 0x5557b01bbb30 2024-04-24T08:52:51.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.990+0000 7f29837fb740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5557b01beae0 con 0x5557b01b1460 2024-04-24T08:52:51.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.990+0000 7f29837fb740 10 client.0 _ll_register_callbacks cb 0x5557b009d140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:52:51.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.990+0000 7f2979e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5557b01b0b40 0x5557b01bb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:51.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.991+0000 7f2979e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5557b01b0b40 0x5557b01bb5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54512/0 (socket says 172.21.15.83:54512) 2024-04-24T08:52:51.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.991+0000 7f2979e00640 1 -- 172.21.15.83:0/2007001551 learned_addr learned my addr 172.21.15.83:0/2007001551 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:52:51.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.991+0000 7f2979400640 1 --2- 172.21.15.83:0/2007001551 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5557b01b1460 0x5557b01b88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:51.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.991+0000 7f297a800640 1 --2- 172.21.15.83:0/2007001551 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5557b01bbb30 0x5557b01b8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:51.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.991+0000 7f2979e00640 1 -- 172.21.15.83:0/2007001551 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5557b01b1460 msgr2=0x5557b01b88a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:51.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.991+0000 7f2979e00640 1 --2- 172.21.15.83:0/2007001551 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5557b01b1460 0x5557b01b88a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:51.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.991+0000 7f2979e00640 1 -- 172.21.15.83:0/2007001551 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5557b01bbb30 msgr2=0x5557b01b8fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:51.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.991+0000 7f2979e00640 1 --2- 172.21.15.83:0/2007001551 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5557b01bbb30 0x5557b01b8fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:51.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.991+0000 7f2979e00640 1 -- 172.21.15.83:0/2007001551 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557b00e03f0 con 0x5557b01b0b40 2024-04-24T08:52:51.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.991+0000 7f297a800640 1 --2- 172.21.15.83:0/2007001551 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5557b01bbb30 0x5557b01b8fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:52:51.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.991+0000 7f2979e00640 1 --2- 172.21.15.83:0/2007001551 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5557b01b0b40 0x5557b01bb5b0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f296c005180 tx=0x7f296c031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5a01f06302cb6ff server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:51.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.991+0000 7f295b400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:52:51.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.992+0000 7f295b400640 1 -- 172.21.15.83:0/2007001551 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f296c035040 con 0x5557b01b0b40 2024-04-24T08:52:51.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.992+0000 7f295b400640 1 -- 172.21.15.83:0/2007001551 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f296c004030 con 0x5557b01b0b40 2024-04-24T08:52:51.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.992+0000 7f29837fb740 1 -- 172.21.15.83:0/2007001551 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f296c0316a0 con 0x5557b01b0b40 2024-04-24T08:52:51.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.992+0000 7f295b400640 1 -- 172.21.15.83:0/2007001551 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f296c040040 con 0x5557b01b0b40 2024-04-24T08:52:51.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.992+0000 7f29837fb740 10 client.16137 fetch_fsmap learned FSMap version 28 2024-04-24T08:52:51.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.992+0000 7f29837fb740 1 -- 172.21.15.83:0/2007001551 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5557afe283f0 con 0x5557b01b0b40 2024-04-24T08:52:51.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.992+0000 7f295b400640 1 -- 172.21.15.83:0/2007001551 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f296c0316a0 con 0x5557b01b0b40 2024-04-24T08:52:51.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.993+0000 7f295b400640 1 -- 172.21.15.83:0/2007001551 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f296c0035f0 con 0x5557b01b0b40 2024-04-24T08:52:51.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.993+0000 7f29837fb740 10 client.16137 fetch_fsmap finished waiting for FSMap version 28 2024-04-24T08:52:51.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.993+0000 7f29837fb740 10 client.16137 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-24T08:52:51.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.993+0000 7f29837fb740 20 client.16137 populate_metadata read hostname 'smithi083' 2024-04-24T08:52:51.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.993+0000 7f29837fb740 1 --2- 172.21.15.83:0/2007001551 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x5557b01cbbd0 0x5557b01cdfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:51.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.993+0000 7f29837fb740 4 client.16137 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:52:51.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.993+0000 7f29837fb740 1 -- 172.21.15.83:0/2007001551 --> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5557b0123bf0 con 0x5557b01cbbd0 2024-04-24T08:52:51.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.994+0000 7f2979400640 1 --2- 172.21.15.83:0/2007001551 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x5557b01cbbd0 0x5557b01cdfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:51.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.995+0000 7f2979400640 1 --2- 172.21.15.83:0/2007001551 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x5557b01cbbd0 0x5557b01cdfc0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a0f22240ed879a6c server_cookie=7ac950210233d2bb in_seq=0 out_seq=0 2024-04-24T08:52:51.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.995+0000 7f295b400640 10 client.16137 ms_handle_connect on v2:172.21.15.138:6834/1909219758 2024-04-24T08:52:51.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.997+0000 7f295b400640 1 -- 172.21.15.83:0/2007001551 <== mds.2 v2:172.21.15.138:6834/1909219758 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5557b0123bf0 con 0x5557b01cbbd0 2024-04-24T08:52:51.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.997+0000 7f295b400640 10 client.16137 handle_command_reply: tid=0 2024-04-24T08:52:51.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.997+0000 7f29837fb740 1 client.16137 shutdown 2024-04-24T08:52:51.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.997+0000 7f29837fb740 1 -- 172.21.15.83:0/2007001551 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5557b01b0b40 msgr2=0x5557b01bb5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:51.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.997+0000 7f29837fb740 1 --2- 172.21.15.83:0/2007001551 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5557b01b0b40 0x5557b01bb5b0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f296c005180 tx=0x7f296c031400 comp rx=0 tx=0).stop 2024-04-24T08:52:51.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.997+0000 7f29837fb740 1 -- 172.21.15.83:0/2007001551 shutdown_connections 2024-04-24T08:52:51.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.997+0000 7f29837fb740 1 --2- 172.21.15.83:0/2007001551 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x5557b01cbbd0 0x5557b01cdfc0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:51.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.997+0000 7f29837fb740 1 --2- 172.21.15.83:0/2007001551 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5557b01bbb30 0x5557b01b8fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:51.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.997+0000 7f29837fb740 1 --2- 172.21.15.83:0/2007001551 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5557b01b1460 0x5557b01b88a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:51.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.997+0000 7f29837fb740 1 --2- 172.21.15.83:0/2007001551 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5557b01b0b40 0x5557b01bb5b0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:51.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.997+0000 7f29837fb740 1 -- 172.21.15.83:0/2007001551 >> 172.21.15.83:0/2007001551 conn(0x5557b01ace60 msgr2=0x5557b01b2b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:51.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.997+0000 7f295b400640 0 client.16137 ms_handle_reset on v2:172.21.15.138:6834/1909219758 2024-04-24T08:52:51.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.997+0000 7f29837fb740 1 -- 172.21.15.83:0/2007001551 shutdown_connections 2024-04-24T08:52:51.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.998+0000 7f29837fb740 1 -- 172.21.15.83:0/2007001551 wait complete. 2024-04-24T08:52:51.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:51.998+0000 7f29837fb740 20 client.16137 trim_cache size 0 max 16384 2024-04-24T08:52:52.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.012+0000 7f29837fb740 1 Processor -- start 2024-04-24T08:52:52.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.012+0000 7f29837fb740 1 -- start start 2024-04-24T08:52:52.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.012+0000 7f29837fb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5557b01b0b40 0x5557b01c7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:52.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.012+0000 7f29837fb740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5557b01b1460 0x5557b01b7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:52.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.012+0000 7f29837fb740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5557b01bbb30 0x5557b01bb0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:52.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.012+0000 7f29837fb740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f297c1b1430 con 0x5557b01b1460 2024-04-24T08:52:52.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.012+0000 7f29837fb740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f297c1b2f70 con 0x5557b01b0b40 2024-04-24T08:52:52.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.012+0000 7f29837fb740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f297c1b1b00 con 0x5557b01bbb30 2024-04-24T08:52:52.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.012+0000 7f2979400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5557b01b1460 0x5557b01b7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:52.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.012+0000 7f2979400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5557b01b1460 0x5557b01b7880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54518/0 (socket says 172.21.15.83:54518) 2024-04-24T08:52:52.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.012+0000 7f2979400640 1 -- 172.21.15.83:0/692500012 learned_addr learned my addr 172.21.15.83:0/692500012 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:52:52.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.013+0000 7f2979e00640 1 --2- 172.21.15.83:0/692500012 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5557b01b0b40 0x5557b01c7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:52.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.013+0000 7f297a800640 1 --2- 172.21.15.83:0/692500012 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5557b01bbb30 0x5557b01bb0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:52.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.013+0000 7f2979400640 1 -- 172.21.15.83:0/692500012 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5557b01bbb30 msgr2=0x5557b01bb0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:52.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.013+0000 7f2979400640 1 --2- 172.21.15.83:0/692500012 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5557b01bbb30 0x5557b01bb0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:52.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.013+0000 7f2979400640 1 -- 172.21.15.83:0/692500012 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5557b01b0b40 msgr2=0x5557b01c7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:52.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.013+0000 7f2979400640 1 --2- 172.21.15.83:0/692500012 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5557b01b0b40 0x5557b01c7880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:52.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.013+0000 7f2979400640 1 -- 172.21.15.83:0/692500012 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557afef27d0 con 0x5557b01b1460 2024-04-24T08:52:52.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.013+0000 7f2979400640 1 --2- 172.21.15.83:0/692500012 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5557b01b1460 0x5557b01b7880 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f297c19beb0 tx=0x7f2968005da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd31b06ddda8bd2d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:52.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.014+0000 7f2966600640 1 -- 172.21.15.83:0/692500012 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2968013e60 con 0x5557b01b1460 2024-04-24T08:52:52.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.014+0000 7f2966600640 1 -- 172.21.15.83:0/692500012 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f296802f070 con 0x5557b01b1460 2024-04-24T08:52:52.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.014+0000 7f2966600640 1 -- 172.21.15.83:0/692500012 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f296802a4c0 con 0x5557b01b1460 2024-04-24T08:52:52.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.015+0000 7f29837fb740 1 -- 172.21.15.83:0/692500012 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5557b01b1460 msgr2=0x5557b01b7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:52.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.015+0000 7f29837fb740 1 --2- 172.21.15.83:0/692500012 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5557b01b1460 0x5557b01b7880 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f297c19beb0 tx=0x7f2968005da0 comp rx=0 tx=0).stop 2024-04-24T08:52:52.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.015+0000 7f29837fb740 1 -- 172.21.15.83:0/692500012 shutdown_connections 2024-04-24T08:52:52.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.015+0000 7f29837fb740 1 --2- 172.21.15.83:0/692500012 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5557b01bbb30 0x5557b01bb0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:52.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.015+0000 7f29837fb740 1 --2- 172.21.15.83:0/692500012 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5557b01b1460 0x5557b01b7880 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:52.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.015+0000 7f29837fb740 1 --2- 172.21.15.83:0/692500012 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5557b01b0b40 0x5557b01c7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:52.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.015+0000 7f29837fb740 1 -- 172.21.15.83:0/692500012 >> 172.21.15.83:0/692500012 conn(0x5557b01ace60 msgr2=0x5557b01b7170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:52.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.015+0000 7f29837fb740 1 -- 172.21.15.83:0/692500012 shutdown_connections 2024-04-24T08:52:52.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.015+0000 7f29837fb740 1 -- 172.21.15.83:0/692500012 wait complete. 2024-04-24T08:52:52.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.015+0000 7f29837fb740 1 -- start start 2024-04-24T08:52:52.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.016+0000 7f29837fb740 1 Processor -- start 2024-04-24T08:52:52.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.016+0000 7f29837fb740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5557b01b0b40 0x5557b01bc030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:52.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.016+0000 7f29837fb740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5557b01b1460 0x5557b01bc570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:52.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.016+0000 7f29837fb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5557b01bbb30 0x5557b01bcab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:52.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.016+0000 7f29837fb740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f297c1b28a0 con 0x5557b01b1460 2024-04-24T08:52:52.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.016+0000 7f29837fb740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f297c1b21d0 con 0x5557b01bbb30 2024-04-24T08:52:52.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.016+0000 7f29837fb740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f297c1af410 con 0x5557b01b0b40 2024-04-24T08:52:52.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.016+0000 7f29837fb740 10 client.0 _ll_register_callbacks cb 0x7f296800df30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:52:52.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.016+0000 7f2979400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5557b01b1460 0x5557b01bc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:52.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.016+0000 7f2979400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5557b01b1460 0x5557b01bc570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54534/0 (socket says 172.21.15.83:54534) 2024-04-24T08:52:52.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.016+0000 7f2979400640 1 -- 172.21.15.83:0/3516070364 learned_addr learned my addr 172.21.15.83:0/3516070364 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:52:52.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.017+0000 7f2979e00640 1 --2- 172.21.15.83:0/3516070364 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5557b01b0b40 0x5557b01bc030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:52.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.017+0000 7f2979400640 1 -- 172.21.15.83:0/3516070364 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5557b01b0b40 msgr2=0x5557b01bc030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:52.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.017+0000 7f2979400640 1 --2- 172.21.15.83:0/3516070364 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5557b01b0b40 0x5557b01bc030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:52.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.017+0000 7f297a800640 1 --2- 172.21.15.83:0/3516070364 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5557b01bbb30 0x5557b01bcab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:52.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.017+0000 7f2979400640 1 -- 172.21.15.83:0/3516070364 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5557b01bbb30 msgr2=0x5557b01bcab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:52.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.017+0000 7f2979400640 1 --2- 172.21.15.83:0/3516070364 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5557b01bbb30 0x5557b01bcab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:52.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.017+0000 7f2979400640 1 -- 172.21.15.83:0/3516070364 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557b01c3030 con 0x5557b01b1460 2024-04-24T08:52:52.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.017+0000 7f2979400640 1 --2- 172.21.15.83:0/3516070364 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5557b01b1460 0x5557b01bc570 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f2968004580 tx=0x7f2968003e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86347ed3a81538d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:52.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.017+0000 7f295b400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:52:52.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.017+0000 7f295b400640 1 -- 172.21.15.83:0/3516070364 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2968013e60 con 0x5557b01b1460 2024-04-24T08:52:52.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.017+0000 7f295b400640 1 -- 172.21.15.83:0/3516070364 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f296800cdf0 con 0x5557b01b1460 2024-04-24T08:52:52.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.017+0000 7f29837fb740 1 -- 172.21.15.83:0/3516070364 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f296802f070 con 0x5557b01b1460 2024-04-24T08:52:52.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.018+0000 7f295b400640 1 -- 172.21.15.83:0/3516070364 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2968041050 con 0x5557b01b1460 2024-04-24T08:52:52.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.018+0000 7f29837fb740 10 client.16143 fetch_fsmap learned FSMap version 28 2024-04-24T08:52:52.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.018+0000 7f29837fb740 1 -- 172.21.15.83:0/3516070364 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5557b01c42f0 con 0x5557b01b1460 2024-04-24T08:52:52.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.018+0000 7f295b400640 1 -- 172.21.15.83:0/3516070364 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f296802f070 con 0x5557b01b1460 2024-04-24T08:52:52.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.018+0000 7f295b400640 1 -- 172.21.15.83:0/3516070364 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f296801f040 con 0x5557b01b1460 2024-04-24T08:52:52.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.018+0000 7f29837fb740 10 client.16143 fetch_fsmap finished waiting for FSMap version 28 2024-04-24T08:52:52.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.018+0000 7f29837fb740 10 client.16143 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-24T08:52:52.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.018+0000 7f29837fb740 20 client.16143 populate_metadata read hostname 'smithi083' 2024-04-24T08:52:52.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.018+0000 7f29837fb740 1 --2- 172.21.15.83:0/3516070364 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x5557b0231fb0 0x5557b02343d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:52.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.018+0000 7f29837fb740 4 client.16143 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T08:52:52.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.018+0000 7f29837fb740 1 -- 172.21.15.83:0/3516070364 --> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5557b011da60 con 0x5557b0231fb0 2024-04-24T08:52:52.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.019+0000 7f2979e00640 1 --2- 172.21.15.83:0/3516070364 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x5557b0231fb0 0x5557b02343d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:52.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.020+0000 7f2979e00640 1 --2- 172.21.15.83:0/3516070364 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x5557b0231fb0 0x5557b02343d0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cc392d68cac6fbad server_cookie=8ba36334f900e58 in_seq=0 out_seq=0 2024-04-24T08:52:52.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.020+0000 7f295b400640 10 client.16143 ms_handle_connect on v2:172.21.15.138:6834/1909219758 2024-04-24T08:52:52.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.021+0000 7f295b400640 1 -- 172.21.15.83:0/3516070364 <== mds.2 v2:172.21.15.138:6834/1909219758 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5557b011da60 con 0x5557b0231fb0 2024-04-24T08:52:52.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.021+0000 7f295b400640 10 client.16143 handle_command_reply: tid=0 2024-04-24T08:52:52.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.021+0000 7f29837fb740 1 client.16143 shutdown 2024-04-24T08:52:52.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.021+0000 7f29837fb740 1 -- 172.21.15.83:0/3516070364 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5557b01b1460 msgr2=0x5557b01bc570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:52.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.021+0000 7f29837fb740 1 --2- 172.21.15.83:0/3516070364 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5557b01b1460 0x5557b01bc570 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f2968004580 tx=0x7f2968003e50 comp rx=0 tx=0).stop 2024-04-24T08:52:52.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.021+0000 7f29837fb740 1 -- 172.21.15.83:0/3516070364 shutdown_connections 2024-04-24T08:52:52.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.021+0000 7f29837fb740 1 --2- 172.21.15.83:0/3516070364 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x5557b0231fb0 0x5557b02343d0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:52.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.021+0000 7f29837fb740 1 --2- 172.21.15.83:0/3516070364 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5557b01bbb30 0x5557b01bcab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:52.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.021+0000 7f29837fb740 1 --2- 172.21.15.83:0/3516070364 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5557b01b1460 0x5557b01bc570 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:52.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.021+0000 7f295b400640 0 client.16143 ms_handle_reset on v2:172.21.15.138:6834/1909219758 2024-04-24T08:52:52.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.021+0000 7f29837fb740 1 --2- 172.21.15.83:0/3516070364 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5557b01b0b40 0x5557b01bc030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:52.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.021+0000 7f29837fb740 1 -- 172.21.15.83:0/3516070364 >> 172.21.15.83:0/3516070364 conn(0x5557b01ace60 msgr2=0x5557b01b7a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:52.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.021+0000 7f29837fb740 1 -- 172.21.15.83:0/3516070364 shutdown_connections 2024-04-24T08:52:52.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.021+0000 7f29837fb740 1 -- 172.21.15.83:0/3516070364 wait complete. 2024-04-24T08:52:52.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.022+0000 7f29837fb740 20 client.16143 trim_cache size 0 max 16384 2024-04-24T08:52:52.023 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T08:52:52.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.022+0000 7f297b200640 1 -- 172.21.15.83:0/1309366807 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f29600a3e40 msgr2=0x7f29600a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:52.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.022+0000 7f297b200640 1 --2- 172.21.15.83:0/1309366807 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f29600a3e40 0x7f29600a6300 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x5557b01adff0 tx=0x7f2968015040 comp rx=0 tx=0).stop 2024-04-24T08:52:52.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.022+0000 7f297b200640 1 -- 172.21.15.83:0/1309366807 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f297c19a660 msgr2=0x7f297c19aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:52.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.022+0000 7f297b200640 1 --2- 172.21.15.83:0/1309366807 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f297c19a660 0x7f297c19aae0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f297400eaa0 tx=0x7f297400ef60 comp rx=0 tx=0).stop 2024-04-24T08:52:52.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.022+0000 7f297b200640 1 -- 172.21.15.83:0/1309366807 shutdown_connections 2024-04-24T08:52:52.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.022+0000 7f297b200640 1 --2- 172.21.15.83:0/1309366807 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f29600a3e40 0x7f29600a6300 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:52.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.022+0000 7f297b200640 1 --2- 172.21.15.83:0/1309366807 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f297c19a660 0x7f297c19aae0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:52.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.022+0000 7f297b200640 1 --2- 172.21.15.83:0/1309366807 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f297c1060d0 0x7f297c1a0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:52.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.022+0000 7f297b200640 1 --2- 172.21.15.83:0/1309366807 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f297c101000 0x7f297c1a05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:52.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.022+0000 7f297b200640 1 -- 172.21.15.83:0/1309366807 >> 172.21.15.83:0/1309366807 conn(0x7f297c0fc880 msgr2=0x7f297c0fdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:52.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.023+0000 7f297b200640 1 -- 172.21.15.83:0/1309366807 shutdown_connections 2024-04-24T08:52:52.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.023+0000 7f297b200640 1 -- 172.21.15.83:0/1309366807 wait complete. 2024-04-24T08:52:52.040 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T08:52:52.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.297+0000 7fae0f200640 1 -- 172.21.15.83:0/1364225596 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae10101020 msgr2=0x7fae10106b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:52.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.297+0000 7fae0f200640 1 --2- 172.21.15.83:0/1364225596 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae10101020 0x7fae10106b50 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fadfc009900 tx=0x7fadfc02a410 comp rx=0 tx=0).stop 2024-04-24T08:52:52.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.297+0000 7fae0f200640 1 -- 172.21.15.83:0/1364225596 shutdown_connections 2024-04-24T08:52:52.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.297+0000 7fae0f200640 1 --2- 172.21.15.83:0/1364225596 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae10107280 0x7fae1010ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:52.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.297+0000 7fae0f200640 1 --2- 172.21.15.83:0/1364225596 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae10101020 0x7fae10106b50 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:52.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.297+0000 7fae0f200640 1 --2- 172.21.15.83:0/1364225596 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fae10100670 0x7fae10100a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:52.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.297+0000 7fae0f200640 1 -- 172.21.15.83:0/1364225596 >> 172.21.15.83:0/1364225596 conn(0x7fae10077390 msgr2=0x7fae100fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:52.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.297+0000 7fae0f200640 1 -- 172.21.15.83:0/1364225596 shutdown_connections 2024-04-24T08:52:52.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.297+0000 7fae0f200640 1 -- 172.21.15.83:0/1364225596 wait complete. 2024-04-24T08:52:52.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.298+0000 7fae0f200640 1 Processor -- start 2024-04-24T08:52:52.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.299+0000 7fae0f200640 1 -- start start 2024-04-24T08:52:52.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.299+0000 7fae0f200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae10100670 0x7fae101a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:52.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.299+0000 7fae0f200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fae10107280 0x7fae101a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:52.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.299+0000 7fae0f200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae1019a780 0x7fae1019ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:52.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.299+0000 7fae0f200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fae10112150 con 0x7fae1019a780 2024-04-24T08:52:52.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.299+0000 7fae0f200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fae10111fd0 con 0x7fae10100670 2024-04-24T08:52:52.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.299+0000 7fae0f200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fae101122d0 con 0x7fae10107280 2024-04-24T08:52:52.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.299+0000 7fae0e800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae1019a780 0x7fae1019ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:52.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.300+0000 7fae0e800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae1019a780 0x7fae1019ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54560/0 (socket says 172.21.15.83:54560) 2024-04-24T08:52:52.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.300+0000 7fae0e800640 1 -- 172.21.15.83:0/555166173 learned_addr learned my addr 172.21.15.83:0/555166173 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:52:52.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.300+0000 7fae0de00640 1 --2- 172.21.15.83:0/555166173 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae10100670 0x7fae101a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:52.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.300+0000 7fae0d400640 1 --2- 172.21.15.83:0/555166173 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fae10107280 0x7fae101a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:52.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.300+0000 7fae0e800640 1 -- 172.21.15.83:0/555166173 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fae10107280 msgr2=0x7fae101a0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:52.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.300+0000 7fae0e800640 1 --2- 172.21.15.83:0/555166173 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fae10107280 0x7fae101a0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:52.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.300+0000 7fae0e800640 1 -- 172.21.15.83:0/555166173 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae10100670 msgr2=0x7fae101a0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:52.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.300+0000 7fae0e800640 1 --2- 172.21.15.83:0/555166173 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae10100670 0x7fae101a0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:52.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.300+0000 7fae0e800640 1 -- 172.21.15.83:0/555166173 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae1019b310 con 0x7fae1019a780 2024-04-24T08:52:52.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.300+0000 7fae0d400640 1 --2- 172.21.15.83:0/555166173 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fae10107280 0x7fae101a0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:52:52.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.300+0000 7fae0e800640 1 --2- 172.21.15.83:0/555166173 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae1019a780 0x7fae1019ac30 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fae0800eaa0 tx=0x7fae0800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:52.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.301+0000 7fae06a00640 1 -- 172.21.15.83:0/555166173 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae0800cdb0 con 0x7fae1019a780 2024-04-24T08:52:52.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.301+0000 7fae06a00640 1 -- 172.21.15.83:0/555166173 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fae08005020 con 0x7fae1019a780 2024-04-24T08:52:52.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.301+0000 7fae06a00640 1 -- 172.21.15.83:0/555166173 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae080116d0 con 0x7fae1019a780 2024-04-24T08:52:52.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.301+0000 7fae0f200640 1 -- 172.21.15.83:0/555166173 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae1019b600 con 0x7fae1019a780 2024-04-24T08:52:52.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.301+0000 7fae0f200640 1 -- 172.21.15.83:0/555166173 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae101a7760 con 0x7fae1019a780 2024-04-24T08:52:52.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.305+0000 7fae17bb2740 1 Processor -- start 2024-04-24T08:52:52.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.306+0000 7fae06a00640 1 -- 172.21.15.83:0/555166173 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7fae080051d0 con 0x7fae1019a780 2024-04-24T08:52:52.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.306+0000 7fae17bb2740 1 -- start start 2024-04-24T08:52:52.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.307+0000 7fae17bb2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561e6ebecb40 0x561e6ebecf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:52.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.307+0000 7fae17bb2740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561e6ebed460 0x561e6ebf75f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:52.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.307+0000 7fae17bb2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e6ebf7b30 0x561e6ebf9f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:52.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.307+0000 7fae17bb2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561e6eb521d0 con 0x561e6ebecb40 2024-04-24T08:52:52.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.307+0000 7fae17bb2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x561e6eaf8450 con 0x561e6ebf7b30 2024-04-24T08:52:52.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.307+0000 7fae17bb2740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x561e6eb4e190 con 0x561e6ebed460 2024-04-24T08:52:52.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.307+0000 7fae0d400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561e6ebecb40 0x561e6ebecf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:52.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.307+0000 7fae0d400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561e6ebecb40 0x561e6ebecf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54564/0 (socket says 172.21.15.83:54564) 2024-04-24T08:52:52.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.307+0000 7fae0d400640 1 -- 172.21.15.83:0/3513054160 learned_addr learned my addr 172.21.15.83:0/3513054160 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:52:52.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.308+0000 7fae0d400640 1 --2- 172.21.15.83:0/3513054160 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e6ebf7b30 0x561e6ebf9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:52.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.308+0000 7fae0d400640 1 -- 172.21.15.83:0/3513054160 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561e6ebed460 msgr2=0x561e6ebf75f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:52.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.308+0000 7fae0d400640 1 --2- 172.21.15.83:0/3513054160 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561e6ebed460 0x561e6ebf75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:52.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.308+0000 7fae0d400640 1 -- 172.21.15.83:0/3513054160 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e6ebf7b30 msgr2=0x561e6ebf9f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:52.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.308+0000 7fae0d400640 1 --2- 172.21.15.83:0/3513054160 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e6ebf7b30 0x561e6ebf9f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:52.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.308+0000 7fae06a00640 1 --2- 172.21.15.83:0/555166173 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fadf40a3ec0 0x7fadf40a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:52.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.308+0000 7fae0d400640 1 -- 172.21.15.83:0/3513054160 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e6ebb5310 con 0x561e6ebecb40 2024-04-24T08:52:52.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.308+0000 7fae06a00640 1 -- 172.21.15.83:0/555166173 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7fae080d0fe0 con 0x7fae1019a780 2024-04-24T08:52:52.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.308+0000 7fae0e800640 1 --2- 172.21.15.83:0/555166173 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fadf40a3ec0 0x7fadf40a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:52.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.308+0000 7fae0d400640 1 --2- 172.21.15.83:0/3513054160 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561e6ebecb40 0x561e6ebecf20 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fadfc031440 tx=0x7fadfc005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1a8a1404545b3a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:52.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.309+0000 7fadfb000640 1 -- 172.21.15.83:0/3513054160 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadfc035050 con 0x561e6ebecb40 2024-04-24T08:52:52.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.309+0000 7fadfb000640 1 -- 172.21.15.83:0/3513054160 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fadfc041070 con 0x561e6ebecb40 2024-04-24T08:52:52.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.309+0000 7fadfb000640 1 -- 172.21.15.83:0/3513054160 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadfc005530 con 0x561e6ebecb40 2024-04-24T08:52:52.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.309+0000 7fae17bb2740 1 -- 172.21.15.83:0/3513054160 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561e6ebecb40 msgr2=0x561e6ebecf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:52.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.309+0000 7fae17bb2740 1 --2- 172.21.15.83:0/3513054160 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561e6ebecb40 0x561e6ebecf20 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fadfc031440 tx=0x7fadfc005bd0 comp rx=0 tx=0).stop 2024-04-24T08:52:52.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.309+0000 7fae0e800640 1 --2- 172.21.15.83:0/555166173 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fadf40a3ec0 0x7fadf40a6380 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fae080ce240 tx=0x7fae0800cf50 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:52.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.310+0000 7fae17bb2740 1 -- 172.21.15.83:0/3513054160 shutdown_connections 2024-04-24T08:52:52.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.310+0000 7fae17bb2740 1 --2- 172.21.15.83:0/3513054160 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e6ebf7b30 0x561e6ebf9f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:52.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.310+0000 7fae17bb2740 1 --2- 172.21.15.83:0/3513054160 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561e6ebed460 0x561e6ebf75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:52.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.310+0000 7fae17bb2740 1 --2- 172.21.15.83:0/3513054160 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561e6ebecb40 0x561e6ebecf20 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:52.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.310+0000 7fae17bb2740 1 -- 172.21.15.83:0/3513054160 >> 172.21.15.83:0/3513054160 conn(0x561e6ebe8e60 msgr2=0x561e6ebe9240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:52.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.310+0000 7fae17bb2740 1 -- 172.21.15.83:0/3513054160 shutdown_connections 2024-04-24T08:52:52.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.310+0000 7fae17bb2740 1 -- 172.21.15.83:0/3513054160 wait complete. 2024-04-24T08:52:52.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.310+0000 7fae17bb2740 1 -- start start 2024-04-24T08:52:52.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.311+0000 7fae17bb2740 1 Processor -- start 2024-04-24T08:52:52.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.311+0000 7fae17bb2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561e6ebecb40 0x561e6ebf4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:52.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.311+0000 7fae17bb2740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561e6ebed460 0x561e6ebf4e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:52.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.311+0000 7fae17bb2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e6ebf7b30 0x561e6ebf5380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:52.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.311+0000 7fae17bb2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561e6ebfa960 con 0x561e6ebecb40 2024-04-24T08:52:52.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.311+0000 7fae17bb2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x561e6eb0b810 con 0x561e6ebf7b30 2024-04-24T08:52:52.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.311+0000 7fae17bb2740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x561e6ebfaae0 con 0x561e6ebed460 2024-04-24T08:52:52.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.311+0000 7fae17bb2740 10 client.0 _ll_register_callbacks cb 0x561e6ead9140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:52:52.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.312+0000 7fae0d400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561e6ebecb40 0x561e6ebf4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:52.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.312+0000 7fae0d400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561e6ebecb40 0x561e6ebf4900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54568/0 (socket says 172.21.15.83:54568) 2024-04-24T08:52:52.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.312+0000 7fae0d400640 1 -- 172.21.15.83:0/2981003542 learned_addr learned my addr 172.21.15.83:0/2981003542 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:52:52.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.312+0000 7fae0de00640 1 --2- 172.21.15.83:0/2981003542 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561e6ebed460 0x561e6ebf4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:52.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.312+0000 7fae0d400640 1 -- 172.21.15.83:0/2981003542 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561e6ebed460 msgr2=0x561e6ebf4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:52.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.312+0000 7fae0d400640 1 --2- 172.21.15.83:0/2981003542 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561e6ebed460 0x561e6ebf4e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:52.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.312+0000 7fae0d400640 1 -- 172.21.15.83:0/2981003542 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e6ebf7b30 msgr2=0x561e6ebf5380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:52.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.312+0000 7fae0d400640 1 --2- 172.21.15.83:0/2981003542 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e6ebf7b30 0x561e6ebf5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:52.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.312+0000 7fae0d400640 1 -- 172.21.15.83:0/2981003542 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e6eb1c3f0 con 0x561e6ebecb40 2024-04-24T08:52:52.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.313+0000 7fae0d400640 1 --2- 172.21.15.83:0/2981003542 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561e6ebecb40 0x561e6ebf4900 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fadfc006010 tx=0x7fadfc031f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa7482398454e429 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:52.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.313+0000 7fadef400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:52:52.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.313+0000 7fadef400640 1 -- 172.21.15.83:0/2981003542 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadfc035070 con 0x561e6ebecb40 2024-04-24T08:52:52.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.313+0000 7fae17bb2740 1 -- 172.21.15.83:0/2981003542 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fadfc041070 con 0x561e6ebecb40 2024-04-24T08:52:52.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.313+0000 7fadef400640 1 -- 172.21.15.83:0/2981003542 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7fadfc004030 con 0x561e6ebecb40 2024-04-24T08:52:52.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.314+0000 7fadef400640 1 -- 172.21.15.83:0/2981003542 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fadfc055050 con 0x561e6ebecb40 2024-04-24T08:52:52.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.314+0000 7fae17bb2740 10 client.16155 fetch_fsmap learned FSMap version 28 2024-04-24T08:52:52.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.314+0000 7fae17bb2740 1 -- 172.21.15.83:0/2981003542 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561e6e8643f0 con 0x561e6ebecb40 2024-04-24T08:52:52.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.314+0000 7fadef400640 1 -- 172.21.15.83:0/2981003542 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fadfc041070 con 0x561e6ebecb40 2024-04-24T08:52:52.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.315+0000 7fadef400640 1 -- 172.21.15.83:0/2981003542 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fadfc0046c0 con 0x561e6ebecb40 2024-04-24T08:52:52.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.315+0000 7fae17bb2740 10 client.16155 fetch_fsmap finished waiting for FSMap version 28 2024-04-24T08:52:52.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.315+0000 7fae17bb2740 10 client.16155 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-24T08:52:52.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.315+0000 7fae17bb2740 20 client.16155 populate_metadata read hostname 'smithi083' 2024-04-24T08:52:52.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.315+0000 7fae17bb2740 1 --2- 172.21.15.83:0/2981003542 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x561e6ec103e0 0x561e6ec127d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:52.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.315+0000 7fae17bb2740 4 client.16155 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:52:52.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.315+0000 7fae17bb2740 1 -- 172.21.15.83:0/2981003542 --> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561e6eb5fbf0 con 0x561e6ec103e0 2024-04-24T08:52:52.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.315+0000 7fae0de00640 1 --2- 172.21.15.83:0/2981003542 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x561e6ec103e0 0x561e6ec127d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:52.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.316+0000 7fadef400640 10 client.16155 ms_handle_connect on v2:172.21.15.138:6836/900566754 2024-04-24T08:52:52.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.316+0000 7fae0de00640 1 --2- 172.21.15.83:0/2981003542 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x561e6ec103e0 0x561e6ec127d0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8949d6daf84c84ac server_cookie=fc67e50da0423517 in_seq=0 out_seq=0 2024-04-24T08:52:52.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.318+0000 7fadef400640 1 -- 172.21.15.83:0/2981003542 <== mds.1 v2:172.21.15.138:6836/900566754 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561e6eb5fbf0 con 0x561e6ec103e0 2024-04-24T08:52:52.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.318+0000 7fadef400640 10 client.16155 handle_command_reply: tid=0 2024-04-24T08:52:52.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.318+0000 7fae17bb2740 1 client.16155 shutdown 2024-04-24T08:52:52.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.319+0000 7fae17bb2740 1 -- 172.21.15.83:0/2981003542 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561e6ebecb40 msgr2=0x561e6ebf4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:52.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.319+0000 7fae17bb2740 1 --2- 172.21.15.83:0/2981003542 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561e6ebecb40 0x561e6ebf4900 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fadfc006010 tx=0x7fadfc031f60 comp rx=0 tx=0).stop 2024-04-24T08:52:52.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.319+0000 7fae17bb2740 1 -- 172.21.15.83:0/2981003542 shutdown_connections 2024-04-24T08:52:52.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.319+0000 7fae17bb2740 1 --2- 172.21.15.83:0/2981003542 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e6ebf7b30 0x561e6ebf5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:52.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.319+0000 7fae17bb2740 1 --2- 172.21.15.83:0/2981003542 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x561e6ec103e0 0x561e6ec127d0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:52.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.319+0000 7fae17bb2740 1 --2- 172.21.15.83:0/2981003542 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561e6ebed460 0x561e6ebf4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:52.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.319+0000 7fae17bb2740 1 --2- 172.21.15.83:0/2981003542 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561e6ebecb40 0x561e6ebf4900 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:52.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.319+0000 7fae17bb2740 1 -- 172.21.15.83:0/2981003542 >> 172.21.15.83:0/2981003542 conn(0x561e6ebe8e60 msgr2=0x561e6ebeec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:52.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.319+0000 7fadef400640 0 client.16155 ms_handle_reset on v2:172.21.15.138:6836/900566754 2024-04-24T08:52:52.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.319+0000 7fae17bb2740 1 -- 172.21.15.83:0/2981003542 shutdown_connections 2024-04-24T08:52:52.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.319+0000 7fae17bb2740 1 -- 172.21.15.83:0/2981003542 wait complete. 2024-04-24T08:52:52.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.319+0000 7fae17bb2740 20 client.16155 trim_cache size 0 max 16384 2024-04-24T08:52:52.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.333+0000 7fae17bb2740 1 Processor -- start 2024-04-24T08:52:52.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.333+0000 7fae17bb2740 1 -- start start 2024-04-24T08:52:52.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.334+0000 7fae17bb2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e6ebecb40 0x561e6ebe7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:52.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.334+0000 7fae17bb2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561e6ebed460 0x561e6ec17d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:52.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.334+0000 7fae17bb2740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561e6ebf7b30 0x561e6ec182b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:52.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.334+0000 7fae17bb2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fae101ad0d0 con 0x561e6ebed460 2024-04-24T08:52:52.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.334+0000 7fae17bb2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fae101aec10 con 0x561e6ebecb40 2024-04-24T08:52:52.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.334+0000 7fae17bb2740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fae101ad7a0 con 0x561e6ebf7b30 2024-04-24T08:52:52.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.334+0000 7fae0de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561e6ebed460 0x561e6ec17d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:52.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.334+0000 7fae0de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561e6ebed460 0x561e6ec17d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54580/0 (socket says 172.21.15.83:54580) 2024-04-24T08:52:52.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.334+0000 7fae0de00640 1 -- 172.21.15.83:0/3275113283 learned_addr learned my addr 172.21.15.83:0/3275113283 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:52:52.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.334+0000 7fae0d400640 1 --2- 172.21.15.83:0/3275113283 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e6ebecb40 0x561e6ebe7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:52.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.334+0000 7fae0d400640 1 --2- 172.21.15.83:0/3275113283 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561e6ebf7b30 0x561e6ec182b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:52.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.335+0000 7fae0de00640 1 -- 172.21.15.83:0/3275113283 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561e6ebf7b30 msgr2=0x561e6ec182b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:52.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.335+0000 7fae0de00640 1 --2- 172.21.15.83:0/3275113283 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561e6ebf7b30 0x561e6ec182b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:52.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.335+0000 7fae0de00640 1 -- 172.21.15.83:0/3275113283 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e6ebecb40 msgr2=0x561e6ebe7dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:52.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.335+0000 7fae0de00640 1 --2- 172.21.15.83:0/3275113283 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e6ebecb40 0x561e6ebe7dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:52.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.335+0000 7fae0de00640 1 -- 172.21.15.83:0/3275113283 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e6e92e7d0 con 0x561e6ebed460 2024-04-24T08:52:52.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.335+0000 7fae0de00640 1 --2- 172.21.15.83:0/3275113283 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561e6ebed460 0x561e6ec17d70 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fae0000b700 tx=0x7fae0000b3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fefd7f276d5e602 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:52.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.335+0000 7fadfa600640 1 -- 172.21.15.83:0/3275113283 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae000139c0 con 0x561e6ebed460 2024-04-24T08:52:52.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.335+0000 7fadfa600640 1 -- 172.21.15.83:0/3275113283 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fae0001a330 con 0x561e6ebed460 2024-04-24T08:52:52.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.335+0000 7fadfa600640 1 -- 172.21.15.83:0/3275113283 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae00011860 con 0x561e6ebed460 2024-04-24T08:52:52.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.336+0000 7fae17bb2740 1 -- 172.21.15.83:0/3275113283 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561e6ebed460 msgr2=0x561e6ec17d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:52.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.336+0000 7fae17bb2740 1 --2- 172.21.15.83:0/3275113283 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561e6ebed460 0x561e6ec17d70 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fae0000b700 tx=0x7fae0000b3d0 comp rx=0 tx=0).stop 2024-04-24T08:52:52.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.336+0000 7fae17bb2740 1 -- 172.21.15.83:0/3275113283 shutdown_connections 2024-04-24T08:52:52.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.336+0000 7fae17bb2740 1 --2- 172.21.15.83:0/3275113283 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561e6ebf7b30 0x561e6ec182b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:52.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.336+0000 7fae17bb2740 1 --2- 172.21.15.83:0/3275113283 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561e6ebed460 0x561e6ec17d70 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:52.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.336+0000 7fae17bb2740 1 --2- 172.21.15.83:0/3275113283 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e6ebecb40 0x561e6ebe7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:52.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.336+0000 7fae17bb2740 1 -- 172.21.15.83:0/3275113283 >> 172.21.15.83:0/3275113283 conn(0x561e6ebe8e60 msgr2=0x561e6ebe76c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:52.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.336+0000 7fae17bb2740 1 -- 172.21.15.83:0/3275113283 shutdown_connections 2024-04-24T08:52:52.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.336+0000 7fae17bb2740 1 -- 172.21.15.83:0/3275113283 wait complete. 2024-04-24T08:52:52.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.337+0000 7fae17bb2740 1 -- start start 2024-04-24T08:52:52.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.337+0000 7fae17bb2740 1 Processor -- start 2024-04-24T08:52:52.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.337+0000 7fae17bb2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e6ebecb40 0x561e6ebf1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:52.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.337+0000 7fae17bb2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561e6ebed460 0x561e6ebf1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:52.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.337+0000 7fae17bb2740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561e6ebf7b30 0x561e6ebf2080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:52.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.337+0000 7fae17bb2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fae101ae540 con 0x561e6ebed460 2024-04-24T08:52:52.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.337+0000 7fae17bb2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fae101ade70 con 0x561e6ebecb40 2024-04-24T08:52:52.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.337+0000 7fae17bb2740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fae101ab0b0 con 0x561e6ebf7b30 2024-04-24T08:52:52.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.337+0000 7fae17bb2740 10 client.0 _ll_register_callbacks cb 0x561e6ebf7a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:52:52.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.338+0000 7fae0de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561e6ebed460 0x561e6ebf1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:52.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.338+0000 7fae0de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561e6ebed460 0x561e6ebf1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54596/0 (socket says 172.21.15.83:54596) 2024-04-24T08:52:52.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.338+0000 7fae0de00640 1 -- 172.21.15.83:0/198359226 learned_addr learned my addr 172.21.15.83:0/198359226 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:52:52.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.338+0000 7fae0d400640 1 --2- 172.21.15.83:0/198359226 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e6ebecb40 0x561e6ebf1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:52.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.338+0000 7fae0d400640 1 --2- 172.21.15.83:0/198359226 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561e6ebf7b30 0x561e6ebf2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:52.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.338+0000 7fae0de00640 1 -- 172.21.15.83:0/198359226 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561e6ebf7b30 msgr2=0x561e6ebf2080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:52.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.338+0000 7fae0de00640 1 --2- 172.21.15.83:0/198359226 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561e6ebf7b30 0x561e6ebf2080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:52.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.338+0000 7fae0de00640 1 -- 172.21.15.83:0/198359226 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e6ebecb40 msgr2=0x561e6ebf1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:52.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.338+0000 7fae0de00640 1 --2- 172.21.15.83:0/198359226 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e6ebecb40 0x561e6ebf1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:52.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.338+0000 7fae0de00640 1 -- 172.21.15.83:0/198359226 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e6ec06770 con 0x561e6ebed460 2024-04-24T08:52:52.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.338+0000 7fae0de00640 1 --2- 172.21.15.83:0/198359226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561e6ebed460 0x561e6ebf1b40 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fae00000c00 tx=0x7fae00013310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bf5d0e4def5fcb7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:52.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.338+0000 7fadef400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:52:52.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.339+0000 7fadef400640 1 -- 172.21.15.83:0/198359226 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae00013d80 con 0x561e6ebed460 2024-04-24T08:52:52.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.339+0000 7fadef400640 1 -- 172.21.15.83:0/198359226 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7fae0001a4e0 con 0x561e6ebed460 2024-04-24T08:52:52.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.339+0000 7fae17bb2740 1 -- 172.21.15.83:0/198359226 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae0001a330 con 0x561e6ebed460 2024-04-24T08:52:52.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.339+0000 7fadef400640 1 -- 172.21.15.83:0/198359226 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fae00035050 con 0x561e6ebed460 2024-04-24T08:52:52.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.339+0000 7fae17bb2740 10 client.16161 fetch_fsmap learned FSMap version 28 2024-04-24T08:52:52.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.339+0000 7fae17bb2740 1 -- 172.21.15.83:0/198359226 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561e6ec768b0 con 0x561e6ebed460 2024-04-24T08:52:52.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.339+0000 7fadef400640 1 -- 172.21.15.83:0/198359226 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fae0001a330 con 0x561e6ebed460 2024-04-24T08:52:52.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.339+0000 7fadef400640 1 -- 172.21.15.83:0/198359226 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fae0002f4b0 con 0x561e6ebed460 2024-04-24T08:52:52.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.340+0000 7fae17bb2740 10 client.16161 fetch_fsmap finished waiting for FSMap version 28 2024-04-24T08:52:52.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.340+0000 7fae17bb2740 10 client.16161 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-24T08:52:52.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.340+0000 7fae17bb2740 20 client.16161 populate_metadata read hostname 'smithi083' 2024-04-24T08:52:52.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.340+0000 7fae17bb2740 1 --2- 172.21.15.83:0/198359226 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x561e6ec76f20 0x561e6ec79340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:52.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.340+0000 7fae17bb2740 4 client.16161 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T08:52:52.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.340+0000 7fae17bb2740 1 -- 172.21.15.83:0/198359226 --> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561e6eb59a60 con 0x561e6ec76f20 2024-04-24T08:52:52.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.340+0000 7fae0d400640 1 --2- 172.21.15.83:0/198359226 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x561e6ec76f20 0x561e6ec79340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:52.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.341+0000 7fae0d400640 1 --2- 172.21.15.83:0/198359226 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x561e6ec76f20 0x561e6ec79340 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8c8c81da8712b45f server_cookie=beb7ffb7490da25d in_seq=0 out_seq=0 2024-04-24T08:52:52.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.341+0000 7fadef400640 10 client.16161 ms_handle_connect on v2:172.21.15.138:6836/900566754 2024-04-24T08:52:52.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.342+0000 7fadef400640 1 -- 172.21.15.83:0/198359226 <== mds.1 v2:172.21.15.138:6836/900566754 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561e6eb59a60 con 0x561e6ec76f20 2024-04-24T08:52:52.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.342+0000 7fadef400640 10 client.16161 handle_command_reply: tid=0 2024-04-24T08:52:52.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.342+0000 7fae17bb2740 1 client.16161 shutdown 2024-04-24T08:52:52.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.342+0000 7fae17bb2740 1 -- 172.21.15.83:0/198359226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561e6ebed460 msgr2=0x561e6ebf1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:52.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.342+0000 7fae17bb2740 1 --2- 172.21.15.83:0/198359226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561e6ebed460 0x561e6ebf1b40 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fae00000c00 tx=0x7fae00013310 comp rx=0 tx=0).stop 2024-04-24T08:52:52.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.342+0000 7fae17bb2740 1 -- 172.21.15.83:0/198359226 shutdown_connections 2024-04-24T08:52:52.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.342+0000 7fae17bb2740 1 --2- 172.21.15.83:0/198359226 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x561e6ec76f20 0x561e6ec79340 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:52.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.342+0000 7fae17bb2740 1 --2- 172.21.15.83:0/198359226 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561e6ebf7b30 0x561e6ebf2080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:52.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.342+0000 7fadef400640 0 client.16161 ms_handle_reset on v2:172.21.15.138:6836/900566754 2024-04-24T08:52:52.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.342+0000 7fae17bb2740 1 --2- 172.21.15.83:0/198359226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561e6ebed460 0x561e6ebf1b40 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:52.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.342+0000 7fae17bb2740 1 --2- 172.21.15.83:0/198359226 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e6ebecb40 0x561e6ebf1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:52.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.342+0000 7fae17bb2740 1 -- 172.21.15.83:0/198359226 >> 172.21.15.83:0/198359226 conn(0x561e6ebe8e60 msgr2=0x561e6ebe7f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:52.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.342+0000 7fae17bb2740 1 -- 172.21.15.83:0/198359226 shutdown_connections 2024-04-24T08:52:52.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.342+0000 7fae17bb2740 1 -- 172.21.15.83:0/198359226 wait complete. 2024-04-24T08:52:52.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.342+0000 7fae17bb2740 20 client.16161 trim_cache size 0 max 16384 2024-04-24T08:52:52.345 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T08:52:52.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.343+0000 7fae0f200640 1 -- 172.21.15.83:0/555166173 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fadf40a3ec0 msgr2=0x7fadf40a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:52.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.343+0000 7fae0f200640 1 --2- 172.21.15.83:0/555166173 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fadf40a3ec0 0x7fadf40a6380 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fae080ce240 tx=0x7fae0800cf50 comp rx=0 tx=0).stop 2024-04-24T08:52:52.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.343+0000 7fae0f200640 1 -- 172.21.15.83:0/555166173 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae1019a780 msgr2=0x7fae1019ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:52.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.343+0000 7fae0f200640 1 --2- 172.21.15.83:0/555166173 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae1019a780 0x7fae1019ac30 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fae0800eaa0 tx=0x7fae0800ef60 comp rx=0 tx=0).stop 2024-04-24T08:52:52.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.343+0000 7fae0f200640 1 -- 172.21.15.83:0/555166173 shutdown_connections 2024-04-24T08:52:52.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.343+0000 7fae0f200640 1 --2- 172.21.15.83:0/555166173 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fadf40a3ec0 0x7fadf40a6380 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:52.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.343+0000 7fae0f200640 1 --2- 172.21.15.83:0/555166173 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae1019a780 0x7fae1019ac30 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:52.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.343+0000 7fae0f200640 1 --2- 172.21.15.83:0/555166173 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fae10107280 0x7fae101a0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:52.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.343+0000 7fae0f200640 1 --2- 172.21.15.83:0/555166173 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae10100670 0x7fae101a0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:52.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.343+0000 7fae0f200640 1 -- 172.21.15.83:0/555166173 >> 172.21.15.83:0/555166173 conn(0x7fae10077390 msgr2=0x7fae101057c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:52.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.344+0000 7fae0f200640 1 -- 172.21.15.83:0/555166173 shutdown_connections 2024-04-24T08:52:52.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.344+0000 7fae0f200640 1 -- 172.21.15.83:0/555166173 wait complete. 2024-04-24T08:52:52.360 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T08:52:52.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.578+0000 7f214be00640 1 -- 172.21.15.83:0/377689401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f214c10e740 msgr2=0x7f214c110b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:52.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.578+0000 7f214be00640 1 --2- 172.21.15.83:0/377689401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f214c10e740 0x7f214c110b50 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f2144009ca0 tx=0x7f214402a950 comp rx=0 tx=0).stop 2024-04-24T08:52:52.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.578+0000 7f214be00640 1 -- 172.21.15.83:0/377689401 shutdown_connections 2024-04-24T08:52:52.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.578+0000 7f214be00640 1 --2- 172.21.15.83:0/377689401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f214c10e740 0x7f214c110b50 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:52.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.578+0000 7f214be00640 1 --2- 172.21.15.83:0/377689401 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f214c074cf0 0x7f214c10e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:52.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.578+0000 7f214be00640 1 --2- 172.21.15.83:0/377689401 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f214c0743d0 0x7f214c0747b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:52.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.578+0000 7f214be00640 1 -- 172.21.15.83:0/377689401 >> 172.21.15.83:0/377689401 conn(0x7f214c0fd9d0 msgr2=0x7f214c0ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:52.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.578+0000 7f214be00640 1 -- 172.21.15.83:0/377689401 shutdown_connections 2024-04-24T08:52:52.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.578+0000 7f214be00640 1 -- 172.21.15.83:0/377689401 wait complete. 2024-04-24T08:52:52.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.579+0000 7f214be00640 1 Processor -- start 2024-04-24T08:52:52.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.579+0000 7f214be00640 1 -- start start 2024-04-24T08:52:52.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.580+0000 7f214be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f214c0743d0 0x7f214c1a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:52.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.580+0000 7f214be00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f214c074cf0 0x7f214c1a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:52.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.580+0000 7f214be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f214c10e740 0x7f214c19a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:52.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.580+0000 7f214be00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f214c114330 con 0x7f214c10e740 2024-04-24T08:52:52.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.580+0000 7f214be00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f214c1141b0 con 0x7f214c0743d0 2024-04-24T08:52:52.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.580+0000 7f214be00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f214c1144b0 con 0x7f214c074cf0 2024-04-24T08:52:52.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.580+0000 7f214b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f214c10e740 0x7f214c19a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:52.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.580+0000 7f214b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f214c10e740 0x7f214c19a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54616/0 (socket says 172.21.15.83:54616) 2024-04-24T08:52:52.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.580+0000 7f214b400640 1 -- 172.21.15.83:0/4195709744 learned_addr learned my addr 172.21.15.83:0/4195709744 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:52:52.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.580+0000 7f214a000640 1 --2- 172.21.15.83:0/4195709744 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f214c074cf0 0x7f214c1a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:52.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.581+0000 7f214aa00640 1 --2- 172.21.15.83:0/4195709744 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f214c0743d0 0x7f214c1a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:52.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.581+0000 7f214b400640 1 -- 172.21.15.83:0/4195709744 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f214c074cf0 msgr2=0x7f214c1a0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:52.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.581+0000 7f214b400640 1 --2- 172.21.15.83:0/4195709744 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f214c074cf0 0x7f214c1a0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:52.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.581+0000 7f214b400640 1 -- 172.21.15.83:0/4195709744 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f214c0743d0 msgr2=0x7f214c1a06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:52.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.581+0000 7f214b400640 1 --2- 172.21.15.83:0/4195709744 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f214c0743d0 0x7f214c1a06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:52.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.581+0000 7f214b400640 1 -- 172.21.15.83:0/4195709744 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f214c19b080 con 0x7f214c10e740 2024-04-24T08:52:52.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.581+0000 7f214b400640 1 --2- 172.21.15.83:0/4195709744 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f214c10e740 0x7f214c19a7c0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f2144005e30 tx=0x7f214402f650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:52.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.581+0000 7f2143400640 1 -- 172.21.15.83:0/4195709744 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2144034040 con 0x7f214c10e740 2024-04-24T08:52:52.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.582+0000 7f214be00640 1 -- 172.21.15.83:0/4195709744 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f214c19b310 con 0x7f214c10e740 2024-04-24T08:52:52.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.582+0000 7f2143400640 1 -- 172.21.15.83:0/4195709744 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f214402fe80 con 0x7f214c10e740 2024-04-24T08:52:52.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.582+0000 7f2143400640 1 -- 172.21.15.83:0/4195709744 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2144004670 con 0x7f214c10e740 2024-04-24T08:52:52.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.582+0000 7f214be00640 1 -- 172.21.15.83:0/4195709744 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f214c1a7450 con 0x7f214c10e740 2024-04-24T08:52:52.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.586+0000 7f215481e740 1 Processor -- start 2024-04-24T08:52:52.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.586+0000 7f2143400640 1 -- 172.21.15.83:0/4195709744 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7f2144037020 con 0x7f214c10e740 2024-04-24T08:52:52.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.588+0000 7f215481e740 1 -- start start 2024-04-24T08:52:52.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.588+0000 7f215481e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55764b05fb40 0x55764b05ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:52.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.588+0000 7f215481e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55764b060460 0x55764b06a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:52.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.588+0000 7f215481e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55764b06ab30 0x55764b06cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:52.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.588+0000 7f215481e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55764afc51d0 con 0x55764b05fb40 2024-04-24T08:52:52.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.588+0000 7f215481e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55764af6b450 con 0x55764b06ab30 2024-04-24T08:52:52.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.588+0000 7f215481e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55764afc1190 con 0x55764b060460 2024-04-24T08:52:52.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.588+0000 7f2143400640 1 --2- 172.21.15.83:0/4195709744 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f21300a3f40 0x7f21300a6400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:52.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.588+0000 7f214a000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55764b05fb40 0x55764b05ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:52.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.589+0000 7f2143400640 1 -- 172.21.15.83:0/4195709744 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f21440f1c80 con 0x7f214c10e740 2024-04-24T08:52:52.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.589+0000 7f214a000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55764b05fb40 0x55764b05ff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54630/0 (socket says 172.21.15.83:54630) 2024-04-24T08:52:52.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.589+0000 7f214a000640 1 -- 172.21.15.83:0/3142873509 learned_addr learned my addr 172.21.15.83:0/3142873509 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:52:52.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.589+0000 7f214aa00640 1 --2- 172.21.15.83:0/3142873509 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55764b060460 0x55764b06a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:52.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.589+0000 7f214a000640 1 -- 172.21.15.83:0/3142873509 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55764b060460 msgr2=0x55764b06a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:52.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.589+0000 7f214a000640 1 --2- 172.21.15.83:0/3142873509 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55764b060460 0x55764b06a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:52.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.589+0000 7f214a000640 1 -- 172.21.15.83:0/3142873509 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55764b06ab30 msgr2=0x55764b06cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:52.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.589+0000 7f214a000640 1 --2- 172.21.15.83:0/3142873509 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55764b06ab30 0x55764b06cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:52.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.589+0000 7f214a000640 1 -- 172.21.15.83:0/3142873509 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55764b028310 con 0x55764b05fb40 2024-04-24T08:52:52.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.589+0000 7f214aa00640 1 --2- 172.21.15.83:0/3142873509 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55764b060460 0x55764b06a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:52:52.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.589+0000 7f214b400640 1 --2- 172.21.15.83:0/4195709744 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f21300a3f40 0x7f21300a6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:52.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.590+0000 7f214a000640 1 --2- 172.21.15.83:0/3142873509 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55764b05fb40 0x55764b05ff20 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f213800bcd0 tx=0x7f21380127c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=715bf8ded4f1c9fc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:52.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.590+0000 7f2137e00640 1 -- 172.21.15.83:0/3142873509 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21380120d0 con 0x55764b05fb40 2024-04-24T08:52:52.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.590+0000 7f2137e00640 1 -- 172.21.15.83:0/3142873509 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2138016070 con 0x55764b05fb40 2024-04-24T08:52:52.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.590+0000 7f2137e00640 1 -- 172.21.15.83:0/3142873509 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2138010430 con 0x55764b05fb40 2024-04-24T08:52:52.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.591+0000 7f215481e740 1 -- 172.21.15.83:0/3142873509 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55764b05fb40 msgr2=0x55764b05ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:52.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.591+0000 7f214b400640 1 --2- 172.21.15.83:0/4195709744 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f21300a3f40 0x7f21300a6400 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f21440eef90 tx=0x7f21440095c0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:52.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.591+0000 7f215481e740 1 --2- 172.21.15.83:0/3142873509 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55764b05fb40 0x55764b05ff20 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f213800bcd0 tx=0x7f21380127c0 comp rx=0 tx=0).stop 2024-04-24T08:52:52.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.591+0000 7f215481e740 1 -- 172.21.15.83:0/3142873509 shutdown_connections 2024-04-24T08:52:52.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.591+0000 7f215481e740 1 --2- 172.21.15.83:0/3142873509 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55764b06ab30 0x55764b06cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:52.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.591+0000 7f215481e740 1 --2- 172.21.15.83:0/3142873509 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55764b060460 0x55764b06a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:52.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.591+0000 7f215481e740 1 --2- 172.21.15.83:0/3142873509 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55764b05fb40 0x55764b05ff20 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:52.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.591+0000 7f215481e740 1 -- 172.21.15.83:0/3142873509 >> 172.21.15.83:0/3142873509 conn(0x55764b05be60 msgr2=0x55764b05c240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:52.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.591+0000 7f215481e740 1 -- 172.21.15.83:0/3142873509 shutdown_connections 2024-04-24T08:52:52.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.591+0000 7f215481e740 1 -- 172.21.15.83:0/3142873509 wait complete. 2024-04-24T08:52:52.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.592+0000 7f215481e740 1 -- start start 2024-04-24T08:52:52.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.592+0000 7f215481e740 1 Processor -- start 2024-04-24T08:52:52.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.593+0000 7f215481e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55764b05fb40 0x55764b067910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:52.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.593+0000 7f215481e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55764b060460 0x55764b067e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:52.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.593+0000 7f215481e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55764b06ab30 0x55764b068390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:52.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.593+0000 7f215481e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55764b06d9c0 con 0x55764b060460 2024-04-24T08:52:52.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.593+0000 7f215481e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55764af7e810 con 0x55764b05fb40 2024-04-24T08:52:52.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.593+0000 7f215481e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55764b06db40 con 0x55764b06ab30 2024-04-24T08:52:52.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.593+0000 7f215481e740 10 client.0 _ll_register_callbacks cb 0x55764af4c140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:52:52.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.593+0000 7f214aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55764b060460 0x55764b067e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:52.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.593+0000 7f214aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55764b060460 0x55764b067e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54634/0 (socket says 172.21.15.83:54634) 2024-04-24T08:52:52.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.593+0000 7f214aa00640 1 -- 172.21.15.83:0/733015818 learned_addr learned my addr 172.21.15.83:0/733015818 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:52:52.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.593+0000 7f214a000640 1 --2- 172.21.15.83:0/733015818 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55764b05fb40 0x55764b067910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:52.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.594+0000 7f214a000640 1 --2- 172.21.15.83:0/733015818 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55764b06ab30 0x55764b068390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:52.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.594+0000 7f214aa00640 1 -- 172.21.15.83:0/733015818 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55764b06ab30 msgr2=0x55764b068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:52.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.594+0000 7f214aa00640 1 --2- 172.21.15.83:0/733015818 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55764b06ab30 0x55764b068390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:52.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.594+0000 7f214aa00640 1 -- 172.21.15.83:0/733015818 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55764b05fb40 msgr2=0x55764b067910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:52.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.594+0000 7f214aa00640 1 --2- 172.21.15.83:0/733015818 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55764b05fb40 0x55764b067910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:52.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.594+0000 7f214aa00640 1 -- 172.21.15.83:0/733015818 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55764af8f3f0 con 0x55764b060460 2024-04-24T08:52:52.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.594+0000 7f214a000640 1 --2- 172.21.15.83:0/733015818 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55764b06ab30 0x55764b068390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:52:52.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.594+0000 7f214aa00640 1 --2- 172.21.15.83:0/733015818 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55764b060460 0x55764b067e50 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f213c0106a0 tx=0x7f213c010b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=142606cac8782e00 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:52.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.594+0000 7f2134c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:52:52.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.594+0000 7f2134c00640 1 -- 172.21.15.83:0/733015818 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f213c010ea0 con 0x55764b060460 2024-04-24T08:52:52.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.594+0000 7f2134c00640 1 -- 172.21.15.83:0/733015818 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f213c00ead0 con 0x55764b060460 2024-04-24T08:52:52.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.594+0000 7f215481e740 1 -- 172.21.15.83:0/733015818 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2138016070 con 0x55764b060460 2024-04-24T08:52:52.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.595+0000 7f2134c00640 1 -- 172.21.15.83:0/733015818 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f213c025050 con 0x55764b060460 2024-04-24T08:52:52.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.595+0000 7f215481e740 10 client.16173 fetch_fsmap learned FSMap version 28 2024-04-24T08:52:52.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.595+0000 7f215481e740 1 -- 172.21.15.83:0/733015818 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55764acd73f0 con 0x55764b060460 2024-04-24T08:52:52.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.595+0000 7f2134c00640 1 -- 172.21.15.83:0/733015818 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2138016070 con 0x55764b060460 2024-04-24T08:52:52.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.595+0000 7f2134c00640 1 -- 172.21.15.83:0/733015818 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f213c00f040 con 0x55764b060460 2024-04-24T08:52:52.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.595+0000 7f215481e740 10 client.16173 fetch_fsmap finished waiting for FSMap version 28 2024-04-24T08:52:52.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.595+0000 7f215481e740 10 client.16173 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:52:52.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.595+0000 7f215481e740 20 client.16173 populate_metadata read hostname 'smithi083' 2024-04-24T08:52:52.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.595+0000 7f215481e740 1 --2- 172.21.15.83:0/733015818 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55764b07ada0 0x55764b07d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:52.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.596+0000 7f215481e740 4 client.16173 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:52:52.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.596+0000 7f215481e740 1 -- 172.21.15.83:0/733015818 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55764afd2bf0 con 0x55764b07ada0 2024-04-24T08:52:52.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.596+0000 7f214a000640 1 --2- 172.21.15.83:0/733015818 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55764b07ada0 0x55764b07d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:52.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.597+0000 7f2134c00640 10 client.16173 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:52:52.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.597+0000 7f214a000640 1 --2- 172.21.15.83:0/733015818 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55764b07ada0 0x55764b07d1f0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31079e5f923f5bb2 server_cookie=64c78fccc046e63b in_seq=0 out_seq=0 2024-04-24T08:52:52.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.599+0000 7f2134c00640 1 -- 172.21.15.83:0/733015818 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55764afd2bf0 con 0x55764b07ada0 2024-04-24T08:52:52.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.599+0000 7f2134c00640 10 client.16173 handle_command_reply: tid=0 2024-04-24T08:52:52.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.599+0000 7f215481e740 1 client.16173 shutdown 2024-04-24T08:52:52.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.599+0000 7f215481e740 1 -- 172.21.15.83:0/733015818 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55764b060460 msgr2=0x55764b067e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:52.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.599+0000 7f215481e740 1 --2- 172.21.15.83:0/733015818 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55764b060460 0x55764b067e50 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f213c0106a0 tx=0x7f213c010b60 comp rx=0 tx=0).stop 2024-04-24T08:52:52.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.600+0000 7f215481e740 1 -- 172.21.15.83:0/733015818 shutdown_connections 2024-04-24T08:52:52.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.600+0000 7f215481e740 1 --2- 172.21.15.83:0/733015818 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55764b07ada0 0x55764b07d1f0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:52.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.600+0000 7f215481e740 1 --2- 172.21.15.83:0/733015818 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55764b06ab30 0x55764b068390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:52.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.600+0000 7f215481e740 1 --2- 172.21.15.83:0/733015818 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55764b060460 0x55764b067e50 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:52.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.600+0000 7f215481e740 1 --2- 172.21.15.83:0/733015818 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55764b05fb40 0x55764b067910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:52.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.600+0000 7f215481e740 1 -- 172.21.15.83:0/733015818 >> 172.21.15.83:0/733015818 conn(0x55764b05be60 msgr2=0x55764b061b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:52.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.600+0000 7f2134c00640 0 client.16173 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:52:52.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.600+0000 7f215481e740 1 -- 172.21.15.83:0/733015818 shutdown_connections 2024-04-24T08:52:52.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.600+0000 7f215481e740 1 -- 172.21.15.83:0/733015818 wait complete. 2024-04-24T08:52:52.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.600+0000 7f215481e740 20 client.16173 trim_cache size 0 max 16384 2024-04-24T08:52:52.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.614+0000 7f215481e740 1 Processor -- start 2024-04-24T08:52:52.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.614+0000 7f215481e740 1 -- start start 2024-04-24T08:52:52.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.614+0000 7f215481e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55764b05fb40 0x55764b0622a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:52.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.614+0000 7f215481e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55764b060460 0x55764b0768b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:52.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.614+0000 7f215481e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55764b06ab30 0x55764b082ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:52.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.614+0000 7f215481e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f214c1b14c0 con 0x55764b06ab30 2024-04-24T08:52:52.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.614+0000 7f215481e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f214c1b3000 con 0x55764b05fb40 2024-04-24T08:52:52.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.614+0000 7f215481e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f214c1b1b90 con 0x55764b060460 2024-04-24T08:52:52.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.614+0000 7f214a000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55764b06ab30 0x55764b082ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:52.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.615+0000 7f214a000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55764b06ab30 0x55764b082ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54648/0 (socket says 172.21.15.83:54648) 2024-04-24T08:52:52.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.615+0000 7f214a000640 1 -- 172.21.15.83:0/1097528745 learned_addr learned my addr 172.21.15.83:0/1097528745 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:52:52.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.615+0000 7f214a000640 1 --2- 172.21.15.83:0/1097528745 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55764b05fb40 0x55764b0622a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:52.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.615+0000 7f214aa00640 1 --2- 172.21.15.83:0/1097528745 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55764b060460 0x55764b0768b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:52.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.615+0000 7f214a000640 1 -- 172.21.15.83:0/1097528745 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55764b060460 msgr2=0x55764b0768b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:52.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.615+0000 7f214a000640 1 --2- 172.21.15.83:0/1097528745 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55764b060460 0x55764b0768b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:52.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.615+0000 7f214a000640 1 -- 172.21.15.83:0/1097528745 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55764b05fb40 msgr2=0x55764b0622a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:52.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.615+0000 7f214a000640 1 --2- 172.21.15.83:0/1097528745 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55764b05fb40 0x55764b0622a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:52.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.615+0000 7f214a000640 1 -- 172.21.15.83:0/1097528745 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55764b060290 con 0x55764b06ab30 2024-04-24T08:52:52.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.615+0000 7f214a000640 1 --2- 172.21.15.83:0/1097528745 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55764b06ab30 0x55764b082ac0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f213800bb10 tx=0x7f213800b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99d1332d83ab7cf9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:52.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.616+0000 7f2137400640 1 -- 172.21.15.83:0/1097528745 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21380035f0 con 0x55764b06ab30 2024-04-24T08:52:52.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.616+0000 7f2137400640 1 -- 172.21.15.83:0/1097528745 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2138016070 con 0x55764b06ab30 2024-04-24T08:52:52.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.616+0000 7f2137400640 1 -- 172.21.15.83:0/1097528745 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2138003ad0 con 0x55764b06ab30 2024-04-24T08:52:52.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.616+0000 7f215481e740 1 -- 172.21.15.83:0/1097528745 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55764b06ab30 msgr2=0x55764b082ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:52.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.616+0000 7f215481e740 1 --2- 172.21.15.83:0/1097528745 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55764b06ab30 0x55764b082ac0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f213800bb10 tx=0x7f213800b7c0 comp rx=0 tx=0).stop 2024-04-24T08:52:52.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.617+0000 7f215481e740 1 -- 172.21.15.83:0/1097528745 shutdown_connections 2024-04-24T08:52:52.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.617+0000 7f215481e740 1 --2- 172.21.15.83:0/1097528745 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55764b06ab30 0x55764b082ac0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:52.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.617+0000 7f215481e740 1 --2- 172.21.15.83:0/1097528745 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55764b060460 0x55764b0768b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:52.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.617+0000 7f215481e740 1 --2- 172.21.15.83:0/1097528745 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55764b05fb40 0x55764b0622a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:52.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.617+0000 7f215481e740 1 -- 172.21.15.83:0/1097528745 >> 172.21.15.83:0/1097528745 conn(0x55764b05be60 msgr2=0x55764b068160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:52.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.617+0000 7f215481e740 1 -- 172.21.15.83:0/1097528745 shutdown_connections 2024-04-24T08:52:52.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.617+0000 7f215481e740 1 -- 172.21.15.83:0/1097528745 wait complete. 2024-04-24T08:52:52.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.617+0000 7f215481e740 1 -- start start 2024-04-24T08:52:52.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.618+0000 7f215481e740 1 Processor -- start 2024-04-24T08:52:52.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.618+0000 7f215481e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55764b05fb40 0x55764b073230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:52.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.618+0000 7f215481e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55764b060460 0x55764b06b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:52.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.618+0000 7f215481e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55764b06ab30 0x55764b06bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:52.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.618+0000 7f215481e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f214c1b2930 con 0x55764b05fb40 2024-04-24T08:52:52.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.618+0000 7f215481e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f214c1b2260 con 0x55764b06ab30 2024-04-24T08:52:52.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.618+0000 7f215481e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f214c1af4a0 con 0x55764b060460 2024-04-24T08:52:52.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.618+0000 7f215481e740 10 client.0 _ll_register_callbacks cb 0x7f213c00b400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:52:52.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.619+0000 7f214a000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55764b05fb40 0x55764b073230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:52.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.619+0000 7f214a000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55764b05fb40 0x55764b073230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54650/0 (socket says 172.21.15.83:54650) 2024-04-24T08:52:52.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.619+0000 7f214a000640 1 -- 172.21.15.83:0/1978536598 learned_addr learned my addr 172.21.15.83:0/1978536598 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:52:52.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.619+0000 7f214aa00640 1 --2- 172.21.15.83:0/1978536598 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55764b060460 0x55764b06b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:52.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.619+0000 7f214a000640 1 -- 172.21.15.83:0/1978536598 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55764b060460 msgr2=0x55764b06b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:52.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.619+0000 7f214a000640 1 --2- 172.21.15.83:0/1978536598 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55764b060460 0x55764b06b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:52.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.619+0000 7f214a000640 1 -- 172.21.15.83:0/1978536598 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55764b06ab30 msgr2=0x55764b06bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:52.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.619+0000 7f214a000640 1 --2- 172.21.15.83:0/1978536598 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55764b06ab30 0x55764b06bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:52.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.619+0000 7f214a000640 1 -- 172.21.15.83:0/1978536598 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55764ada17d0 con 0x55764b05fb40 2024-04-24T08:52:52.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.619+0000 7f214a000640 1 --2- 172.21.15.83:0/1978536598 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55764b05fb40 0x55764b073230 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f213800be70 tx=0x7f2138012d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dbb83178c4d65e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:52.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.619+0000 7f2134c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:52:52.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.620+0000 7f2134c00640 1 -- 172.21.15.83:0/1978536598 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2138029030 con 0x55764b05fb40 2024-04-24T08:52:52.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.620+0000 7f2134c00640 1 -- 172.21.15.83:0/1978536598 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f21380291d0 con 0x55764b05fb40 2024-04-24T08:52:52.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.620+0000 7f215481e740 1 -- 172.21.15.83:0/1978536598 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2138016070 con 0x55764b05fb40 2024-04-24T08:52:52.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.620+0000 7f2134c00640 1 -- 172.21.15.83:0/1978536598 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2138039050 con 0x55764b05fb40 2024-04-24T08:52:52.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.620+0000 7f215481e740 10 client.16179 fetch_fsmap learned FSMap version 28 2024-04-24T08:52:52.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.620+0000 7f215481e740 1 -- 172.21.15.83:0/1978536598 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55764b0e9b40 con 0x55764b05fb40 2024-04-24T08:52:52.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.620+0000 7f2134c00640 1 -- 172.21.15.83:0/1978536598 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2138016070 con 0x55764b05fb40 2024-04-24T08:52:52.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.621+0000 7f2134c00640 1 -- 172.21.15.83:0/1978536598 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f2138028450 con 0x55764b05fb40 2024-04-24T08:52:52.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.621+0000 7f215481e740 10 client.16179 fetch_fsmap finished waiting for FSMap version 28 2024-04-24T08:52:52.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.621+0000 7f215481e740 10 client.16179 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:52:52.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.621+0000 7f215481e740 20 client.16179 populate_metadata read hostname 'smithi083' 2024-04-24T08:52:52.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.621+0000 7f215481e740 1 --2- 172.21.15.83:0/1978536598 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55764b0e9e20 0x55764b0ec210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:52.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.621+0000 7f215481e740 4 client.16179 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T08:52:52.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.621+0000 7f215481e740 1 -- 172.21.15.83:0/1978536598 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55764afcca60 con 0x55764b0e9e20 2024-04-24T08:52:52.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.621+0000 7f214aa00640 1 --2- 172.21.15.83:0/1978536598 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55764b0e9e20 0x55764b0ec210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:52.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.622+0000 7f2134c00640 10 client.16179 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:52:52.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.622+0000 7f214aa00640 1 --2- 172.21.15.83:0/1978536598 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55764b0e9e20 0x55764b0ec210 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fda72e3ddf033d2 server_cookie=3ae743b7de817ae8 in_seq=0 out_seq=0 2024-04-24T08:52:52.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.623+0000 7f2134c00640 1 -- 172.21.15.83:0/1978536598 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55764afcca60 con 0x55764b0e9e20 2024-04-24T08:52:52.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.623+0000 7f2134c00640 10 client.16179 handle_command_reply: tid=0 2024-04-24T08:52:52.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.623+0000 7f215481e740 1 client.16179 shutdown 2024-04-24T08:52:52.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.623+0000 7f215481e740 1 -- 172.21.15.83:0/1978536598 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55764b05fb40 msgr2=0x55764b073230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:52.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.623+0000 7f215481e740 1 --2- 172.21.15.83:0/1978536598 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55764b05fb40 0x55764b073230 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f213800be70 tx=0x7f2138012d50 comp rx=0 tx=0).stop 2024-04-24T08:52:52.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.623+0000 7f215481e740 1 -- 172.21.15.83:0/1978536598 shutdown_connections 2024-04-24T08:52:52.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.623+0000 7f215481e740 1 --2- 172.21.15.83:0/1978536598 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55764b0e9e20 0x55764b0ec210 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:52.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.623+0000 7f215481e740 1 --2- 172.21.15.83:0/1978536598 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55764b06ab30 0x55764b06bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:52.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.623+0000 7f2134c00640 0 client.16179 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:52:52.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.623+0000 7f215481e740 1 --2- 172.21.15.83:0/1978536598 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55764b060460 0x55764b06b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:52.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.624+0000 7f215481e740 1 --2- 172.21.15.83:0/1978536598 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55764b05fb40 0x55764b073230 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:52.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.624+0000 7f215481e740 1 -- 172.21.15.83:0/1978536598 >> 172.21.15.83:0/1978536598 conn(0x55764b05be60 msgr2=0x55764b05a6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:52.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.624+0000 7f215481e740 1 -- 172.21.15.83:0/1978536598 shutdown_connections 2024-04-24T08:52:52.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.624+0000 7f215481e740 1 -- 172.21.15.83:0/1978536598 wait complete. 2024-04-24T08:52:52.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.624+0000 7f215481e740 20 client.16179 trim_cache size 0 max 16384 2024-04-24T08:52:52.627 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T08:52:52.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.625+0000 7f214be00640 1 -- 172.21.15.83:0/4195709744 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f21300a3f40 msgr2=0x7f21300a6400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:52.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.625+0000 7f214be00640 1 --2- 172.21.15.83:0/4195709744 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f21300a3f40 0x7f21300a6400 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f21440eef90 tx=0x7f21440095c0 comp rx=0 tx=0).stop 2024-04-24T08:52:52.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.625+0000 7f214be00640 1 -- 172.21.15.83:0/4195709744 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f214c10e740 msgr2=0x7f214c19a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:52.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.625+0000 7f214be00640 1 --2- 172.21.15.83:0/4195709744 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f214c10e740 0x7f214c19a7c0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f2144005e30 tx=0x7f214402f650 comp rx=0 tx=0).stop 2024-04-24T08:52:52.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.625+0000 7f214be00640 1 -- 172.21.15.83:0/4195709744 shutdown_connections 2024-04-24T08:52:52.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.625+0000 7f214be00640 1 --2- 172.21.15.83:0/4195709744 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f21300a3f40 0x7f21300a6400 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:52.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.625+0000 7f214be00640 1 --2- 172.21.15.83:0/4195709744 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f214c10e740 0x7f214c19a7c0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:52.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.625+0000 7f214be00640 1 --2- 172.21.15.83:0/4195709744 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f214c074cf0 0x7f214c1a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:52.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.625+0000 7f214be00640 1 --2- 172.21.15.83:0/4195709744 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f214c0743d0 0x7f214c1a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:52.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.625+0000 7f214be00640 1 -- 172.21.15.83:0/4195709744 >> 172.21.15.83:0/4195709744 conn(0x7f214c0fd9d0 msgr2=0x7f214c0fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:52.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.625+0000 7f214be00640 1 -- 172.21.15.83:0/4195709744 shutdown_connections 2024-04-24T08:52:52.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:52.625+0000 7f214be00640 1 -- 172.21.15.83:0/4195709744 wait complete. 2024-04-24T08:52:53.648 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T08:52:53.648 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T08:52:53.648 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T08:52:53.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.835+0000 7f5ca9000640 1 -- 172.21.15.83:0/4120191863 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ca4104320 msgr2=0x7f5ca4104700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:53.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.835+0000 7f5ca9000640 1 --2- 172.21.15.83:0/4120191863 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ca4104320 0x7f5ca4104700 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f5c94009900 tx=0x7f5c9402a3f0 comp rx=0 tx=0).stop 2024-04-24T08:52:53.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.835+0000 7f5ca9000640 1 -- 172.21.15.83:0/4120191863 shutdown_connections 2024-04-24T08:52:53.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.835+0000 7f5ca9000640 1 --2- 172.21.15.83:0/4120191863 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ca41093f0 0x7f5ca4110fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:53.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.835+0000 7f5ca9000640 1 --2- 172.21.15.83:0/4120191863 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5ca4104cd0 0x7f5ca4108d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:53.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.835+0000 7f5ca9000640 1 --2- 172.21.15.83:0/4120191863 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ca4104320 0x7f5ca4104700 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:53.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.835+0000 7f5ca9000640 1 -- 172.21.15.83:0/4120191863 >> 172.21.15.83:0/4120191863 conn(0x7f5ca40ffb40 msgr2=0x7f5ca4101f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:53.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.835+0000 7f5ca9000640 1 -- 172.21.15.83:0/4120191863 shutdown_connections 2024-04-24T08:52:53.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.835+0000 7f5ca9000640 1 -- 172.21.15.83:0/4120191863 wait complete. 2024-04-24T08:52:53.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.836+0000 7f5ca9000640 1 Processor -- start 2024-04-24T08:52:53.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.836+0000 7f5ca9000640 1 -- start start 2024-04-24T08:52:53.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.837+0000 7f5ca9000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5ca4104320 0x7f5ca41a05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:53.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.837+0000 7f5ca9000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ca41093f0 0x7f5ca41a0b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:53.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.837+0000 7f5ca9000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ca419a6a0 0x7f5ca419ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:53.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.837+0000 7f5ca9000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca4114380 con 0x7f5ca419a6a0 2024-04-24T08:52:53.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.837+0000 7f5ca9000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca4114200 con 0x7f5ca41093f0 2024-04-24T08:52:53.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.837+0000 7f5ca9000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca4114500 con 0x7f5ca4104320 2024-04-24T08:52:53.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.837+0000 7f5ca3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ca419a6a0 0x7f5ca419ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:53.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.837+0000 7f5ca3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ca419a6a0 0x7f5ca419ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54668/0 (socket says 172.21.15.83:54668) 2024-04-24T08:52:53.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.837+0000 7f5ca3e00640 1 -- 172.21.15.83:0/1794889733 learned_addr learned my addr 172.21.15.83:0/1794889733 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:52:53.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.837+0000 7f5ca2a00640 1 --2- 172.21.15.83:0/1794889733 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ca41093f0 0x7f5ca41a0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:53.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.837+0000 7f5ca3400640 1 --2- 172.21.15.83:0/1794889733 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5ca4104320 0x7f5ca41a05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:53.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.838+0000 7f5ca3e00640 1 -- 172.21.15.83:0/1794889733 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5ca4104320 msgr2=0x7f5ca41a05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:53.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.838+0000 7f5ca3e00640 1 --2- 172.21.15.83:0/1794889733 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5ca4104320 0x7f5ca41a05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:53.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.838+0000 7f5ca3e00640 1 -- 172.21.15.83:0/1794889733 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ca41093f0 msgr2=0x7f5ca41a0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:53.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.838+0000 7f5ca3e00640 1 --2- 172.21.15.83:0/1794889733 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ca41093f0 0x7f5ca41a0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:53.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.838+0000 7f5ca3e00640 1 -- 172.21.15.83:0/1794889733 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ca419b2a0 con 0x7f5ca419a6a0 2024-04-24T08:52:53.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.838+0000 7f5ca2a00640 1 --2- 172.21.15.83:0/1794889733 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ca41093f0 0x7f5ca41a0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:52:53.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.838+0000 7f5ca3e00640 1 --2- 172.21.15.83:0/1794889733 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ca419a6a0 0x7f5ca419ab20 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c00cad0 tx=0x7f5c9c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:53.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.838+0000 7f5c9be00640 1 -- 172.21.15.83:0/1794889733 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c9c00ae20 con 0x7f5ca419a6a0 2024-04-24T08:52:53.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.838+0000 7f5c9be00640 1 -- 172.21.15.83:0/1794889733 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5c9c005020 con 0x7f5ca419a6a0 2024-04-24T08:52:53.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.838+0000 7f5c9be00640 1 -- 172.21.15.83:0/1794889733 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c9c00f6b0 con 0x7f5ca419a6a0 2024-04-24T08:52:53.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.838+0000 7f5ca9000640 1 -- 172.21.15.83:0/1794889733 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ca41a7380 con 0x7f5ca419a6a0 2024-04-24T08:52:53.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.839+0000 7f5ca9000640 1 -- 172.21.15.83:0/1794889733 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ca41a7730 con 0x7f5ca419a6a0 2024-04-24T08:52:53.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.843+0000 7f5c9be00640 1 -- 172.21.15.83:0/1794889733 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7f5c9c0051d0 con 0x7f5ca419a6a0 2024-04-24T08:52:53.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.843+0000 7f5cad460740 1 Processor -- start 2024-04-24T08:52:53.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.844+0000 7f5cad460740 1 -- start start 2024-04-24T08:52:53.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.844+0000 7f5cad460740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae840ae270 0x55ae840ae650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:53.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.844+0000 7f5cad460740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ae840aeb90 0x55ae840b8d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:53.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.844+0000 7f5cad460740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae840b9260 0x55ae840bb650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:53.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.844+0000 7f5cad460740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ae84013e70 con 0x55ae840aeb90 2024-04-24T08:52:53.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.844+0000 7f5cad460740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ae83fba380 con 0x55ae840b9260 2024-04-24T08:52:53.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.844+0000 7f5cad460740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55ae8400fe30 con 0x55ae840ae270 2024-04-24T08:52:53.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.845+0000 7f5ca3400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ae840aeb90 0x55ae840b8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:53.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.845+0000 7f5ca2a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae840ae270 0x55ae840ae650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:53.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.845+0000 7f5ca3400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ae840aeb90 0x55ae840b8d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54670/0 (socket says 172.21.15.83:54670) 2024-04-24T08:52:53.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.845+0000 7f5ca3400640 1 -- 172.21.15.83:0/3027556943 learned_addr learned my addr 172.21.15.83:0/3027556943 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:52:53.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.845+0000 7f5ca2a00640 1 --2- 172.21.15.83:0/3027556943 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae840b9260 0x55ae840bb650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:53.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.845+0000 7f5ca3400640 1 -- 172.21.15.83:0/3027556943 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae840ae270 msgr2=0x55ae840ae650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:53.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.845+0000 7f5ca3400640 1 --2- 172.21.15.83:0/3027556943 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae840ae270 0x55ae840ae650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:53.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.845+0000 7f5ca3400640 1 -- 172.21.15.83:0/3027556943 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae840b9260 msgr2=0x55ae840bb650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:53.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.845+0000 7f5ca3400640 1 --2- 172.21.15.83:0/3027556943 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae840b9260 0x55ae840bb650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:53.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.845+0000 7f5ca3400640 1 -- 172.21.15.83:0/3027556943 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae83dee7e0 con 0x55ae840aeb90 2024-04-24T08:52:53.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.846+0000 7f5c9be00640 1 --2- 172.21.15.83:0/1794889733 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f5c880a3ec0 0x7f5c880a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:53.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.845+0000 7f5ca2a00640 1 --2- 172.21.15.83:0/3027556943 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae840b9260 0x55ae840bb650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:52:53.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.846+0000 7f5c9be00640 1 -- 172.21.15.83:0/1794889733 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f5c9c0d0f00 con 0x7f5ca419a6a0 2024-04-24T08:52:53.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.846+0000 7f5ca3400640 1 --2- 172.21.15.83:0/3027556943 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ae840aeb90 0x55ae840b8d20 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f5c94031440 tx=0x7f5c94004390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cb454f8939c5a13 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:53.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.846+0000 7f5ca2a00640 1 --2- 172.21.15.83:0/1794889733 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f5c880a3ec0 0x7f5c880a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:53.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.846+0000 7f5c98c00640 1 -- 172.21.15.83:0/3027556943 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c9403f070 con 0x55ae840aeb90 2024-04-24T08:52:53.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.846+0000 7f5c98c00640 1 -- 172.21.15.83:0/3027556943 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5c940047a0 con 0x55ae840aeb90 2024-04-24T08:52:53.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.846+0000 7f5c98c00640 1 -- 172.21.15.83:0/3027556943 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c94035070 con 0x55ae840aeb90 2024-04-24T08:52:53.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.847+0000 7f5cad460740 1 -- 172.21.15.83:0/3027556943 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ae840aeb90 msgr2=0x55ae840b8d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:53.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.847+0000 7f5ca2a00640 1 --2- 172.21.15.83:0/1794889733 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f5c880a3ec0 0x7f5c880a6380 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x55ae840ab720 tx=0x7f5c90005f90 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:53.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.847+0000 7f5cad460740 1 --2- 172.21.15.83:0/3027556943 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ae840aeb90 0x55ae840b8d20 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f5c94031440 tx=0x7f5c94004390 comp rx=0 tx=0).stop 2024-04-24T08:52:53.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.848+0000 7f5cad460740 1 -- 172.21.15.83:0/3027556943 shutdown_connections 2024-04-24T08:52:53.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.848+0000 7f5cad460740 1 --2- 172.21.15.83:0/3027556943 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae840b9260 0x55ae840bb650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:53.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.848+0000 7f5cad460740 1 --2- 172.21.15.83:0/3027556943 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ae840aeb90 0x55ae840b8d20 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:53.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.848+0000 7f5cad460740 1 --2- 172.21.15.83:0/3027556943 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae840ae270 0x55ae840ae650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:53.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.848+0000 7f5cad460740 1 -- 172.21.15.83:0/3027556943 >> 172.21.15.83:0/3027556943 conn(0x55ae840aa590 msgr2=0x55ae840aa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:53.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.848+0000 7f5cad460740 1 -- 172.21.15.83:0/3027556943 shutdown_connections 2024-04-24T08:52:53.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.848+0000 7f5cad460740 1 -- 172.21.15.83:0/3027556943 wait complete. 2024-04-24T08:52:53.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.848+0000 7f5cad460740 1 -- start start 2024-04-24T08:52:53.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.849+0000 7f5cad460740 1 Processor -- start 2024-04-24T08:52:53.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.849+0000 7f5cad460740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae840ae270 0x55ae840b63f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:53.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.849+0000 7f5cad460740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ae840b9260 0x55ae840b6930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:53.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.849+0000 7f5cad460740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae840b6e70 0x55ae840c5490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:53.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.849+0000 7f5cad460740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ae840bbff0 con 0x55ae840b9260 2024-04-24T08:52:53.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.849+0000 7f5cad460740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ae83fcd660 con 0x55ae840ae270 2024-04-24T08:52:53.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.849+0000 7f5cad460740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55ae840bc170 con 0x55ae840b6e70 2024-04-24T08:52:53.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.849+0000 7f5cad460740 10 client.0 _ll_register_callbacks cb 0x55ae83fbbaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:52:53.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.849+0000 7f5ca2a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ae840b9260 0x55ae840b6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:53.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.850+0000 7f5ca2a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ae840b9260 0x55ae840b6930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54682/0 (socket says 172.21.15.83:54682) 2024-04-24T08:52:53.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.850+0000 7f5ca2a00640 1 -- 172.21.15.83:0/718295207 learned_addr learned my addr 172.21.15.83:0/718295207 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:52:53.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.850+0000 7f5ca3e00640 1 --2- 172.21.15.83:0/718295207 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae840b6e70 0x55ae840c5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:53.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.850+0000 7f5ca3400640 1 --2- 172.21.15.83:0/718295207 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae840ae270 0x55ae840b63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:53.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.850+0000 7f5ca2a00640 1 -- 172.21.15.83:0/718295207 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae840b6e70 msgr2=0x55ae840c5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:53.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.850+0000 7f5ca2a00640 1 --2- 172.21.15.83:0/718295207 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae840b6e70 0x55ae840c5490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:53.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.850+0000 7f5ca2a00640 1 -- 172.21.15.83:0/718295207 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae840ae270 msgr2=0x55ae840b63f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:53.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.850+0000 7f5ca2a00640 1 --2- 172.21.15.83:0/718295207 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae840ae270 0x55ae840b63f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:53.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.850+0000 7f5ca2a00640 1 -- 172.21.15.83:0/718295207 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae83e4ef70 con 0x55ae840b9260 2024-04-24T08:52:53.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.851+0000 7f5ca2a00640 1 --2- 172.21.15.83:0/718295207 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ae840b9260 0x55ae840b6930 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f5c90012fd0 tx=0x7f5c900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2f242b2322b2232 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:53.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.851+0000 7f5c8d600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:52:53.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.852+0000 7f5c8d600640 1 -- 172.21.15.83:0/718295207 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c90023070 con 0x55ae840b9260 2024-04-24T08:52:53.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.852+0000 7f5c8d600640 1 -- 172.21.15.83:0/718295207 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f5c9000ee10 con 0x55ae840b9260 2024-04-24T08:52:53.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.852+0000 7f5cad460740 1 -- 172.21.15.83:0/718295207 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c940047a0 con 0x55ae840b9260 2024-04-24T08:52:53.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.852+0000 7f5c8d600640 1 -- 172.21.15.83:0/718295207 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5c90031050 con 0x55ae840b9260 2024-04-24T08:52:53.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.852+0000 7f5cad460740 10 client.16191 fetch_fsmap learned FSMap version 28 2024-04-24T08:52:53.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.852+0000 7f5cad460740 1 -- 172.21.15.83:0/718295207 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ae83ce6170 con 0x55ae840b9260 2024-04-24T08:52:53.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.853+0000 7f5c8d600640 1 -- 172.21.15.83:0/718295207 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5c940047a0 con 0x55ae840b9260 2024-04-24T08:52:53.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.853+0000 7f5c8d600640 1 -- 172.21.15.83:0/718295207 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f5c9001f650 con 0x55ae840b9260 2024-04-24T08:52:53.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.853+0000 7f5cad460740 10 client.16191 fetch_fsmap finished waiting for FSMap version 28 2024-04-24T08:52:53.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.853+0000 7f5cad460740 10 client.16191 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:52:53.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.853+0000 7f5cad460740 20 client.16191 populate_metadata read hostname 'smithi083' 2024-04-24T08:52:53.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.853+0000 7f5cad460740 1 --2- 172.21.15.83:0/718295207 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ae840c9c30 0x55ae840cc050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:53.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.853+0000 7f5cad460740 4 client.16191 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:52:53.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.853+0000 7f5cad460740 1 -- 172.21.15.83:0/718295207 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ae84021890 con 0x55ae840c9c30 2024-04-24T08:52:53.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.854+0000 7f5ca3400640 1 --2- 172.21.15.83:0/718295207 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ae840c9c30 0x55ae840cc050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:53.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.855+0000 7f5ca3400640 1 --2- 172.21.15.83:0/718295207 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ae840c9c30 0x55ae840cc050 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b60d603d7833d13 server_cookie=4549acc73ab71431 in_seq=0 out_seq=0 2024-04-24T08:52:53.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.855+0000 7f5c8d600640 10 client.16191 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:52:53.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.857+0000 7f5c8d600640 1 -- 172.21.15.83:0/718295207 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ae84021890 con 0x55ae840c9c30 2024-04-24T08:52:53.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.857+0000 7f5c8d600640 10 client.16191 handle_command_reply: tid=0 2024-04-24T08:52:53.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.857+0000 7f5cad460740 1 client.16191 shutdown 2024-04-24T08:52:53.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.857+0000 7f5cad460740 1 -- 172.21.15.83:0/718295207 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ae840b9260 msgr2=0x55ae840b6930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:53.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.857+0000 7f5cad460740 1 --2- 172.21.15.83:0/718295207 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ae840b9260 0x55ae840b6930 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f5c90012fd0 tx=0x7f5c900046c0 comp rx=0 tx=0).stop 2024-04-24T08:52:53.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.857+0000 7f5cad460740 1 -- 172.21.15.83:0/718295207 shutdown_connections 2024-04-24T08:52:53.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.857+0000 7f5cad460740 1 --2- 172.21.15.83:0/718295207 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ae840c9c30 0x55ae840cc050 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:53.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.857+0000 7f5cad460740 1 --2- 172.21.15.83:0/718295207 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae840b6e70 0x55ae840c5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:53.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.857+0000 7f5cad460740 1 --2- 172.21.15.83:0/718295207 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ae840b9260 0x55ae840b6930 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:53.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.857+0000 7f5cad460740 1 --2- 172.21.15.83:0/718295207 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae840ae270 0x55ae840b63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:53.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.857+0000 7f5cad460740 1 -- 172.21.15.83:0/718295207 >> 172.21.15.83:0/718295207 conn(0x55ae840aa590 msgr2=0x55ae840b0290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:53.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.857+0000 7f5c8d600640 0 client.16191 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:52:53.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.858+0000 7f5cad460740 1 -- 172.21.15.83:0/718295207 shutdown_connections 2024-04-24T08:52:53.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.858+0000 7f5cad460740 1 -- 172.21.15.83:0/718295207 wait complete. 2024-04-24T08:52:53.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.858+0000 7f5cad460740 20 client.16191 trim_cache size 0 max 16384 2024-04-24T08:52:53.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.893+0000 7f5cad460740 1 Processor -- start 2024-04-24T08:52:53.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.893+0000 7f5cad460740 1 -- start start 2024-04-24T08:52:53.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.894+0000 7f5cad460740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae840ae270 0x55ae840d1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:53.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.894+0000 7f5cad460740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae840b6e70 0x55ae840d1e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:53.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.894+0000 7f5cad460740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ae840b6a20 0x55ae840b00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:53.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.894+0000 7f5cad460740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca41b1480 con 0x55ae840b6a20 2024-04-24T08:52:53.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.894+0000 7f5cad460740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca41b2fc0 con 0x55ae840b6e70 2024-04-24T08:52:53.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.894+0000 7f5cad460740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca41b1b50 con 0x55ae840ae270 2024-04-24T08:52:53.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.894+0000 7f5ca3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ae840b6a20 0x55ae840b00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:53.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.894+0000 7f5ca3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ae840b6a20 0x55ae840b00d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54694/0 (socket says 172.21.15.83:54694) 2024-04-24T08:52:53.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.894+0000 7f5ca3e00640 1 -- 172.21.15.83:0/106444581 learned_addr learned my addr 172.21.15.83:0/106444581 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:52:53.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.895+0000 7f5ca2a00640 1 --2- 172.21.15.83:0/106444581 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae840b6e70 0x55ae840d1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:53.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.895+0000 7f5ca3400640 1 --2- 172.21.15.83:0/106444581 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae840ae270 0x55ae840d1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:53.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.895+0000 7f5ca3e00640 1 -- 172.21.15.83:0/106444581 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae840ae270 msgr2=0x55ae840d1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:53.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.895+0000 7f5ca3e00640 1 --2- 172.21.15.83:0/106444581 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae840ae270 0x55ae840d1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:53.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.895+0000 7f5ca3e00640 1 -- 172.21.15.83:0/106444581 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae840b6e70 msgr2=0x55ae840d1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:53.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.895+0000 7f5ca3e00640 1 --2- 172.21.15.83:0/106444581 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae840b6e70 0x55ae840d1e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:53.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.895+0000 7f5ca3e00640 1 -- 172.21.15.83:0/106444581 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae840b5120 con 0x55ae840b6a20 2024-04-24T08:52:53.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.895+0000 7f5ca3e00640 1 --2- 172.21.15.83:0/106444581 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ae840b6a20 0x55ae840b00d0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c09c580 tx=0x7f5c9c09c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ca2a99983cef1e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:53.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.896+0000 7f5c8fe00640 1 -- 172.21.15.83:0/106444581 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c9c01d040 con 0x55ae840b6a20 2024-04-24T08:52:53.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.896+0000 7f5c8fe00640 1 -- 172.21.15.83:0/106444581 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5c9c0a54f0 con 0x55ae840b6a20 2024-04-24T08:52:53.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.896+0000 7f5c8fe00640 1 -- 172.21.15.83:0/106444581 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c9c09de10 con 0x55ae840b6a20 2024-04-24T08:52:53.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.897+0000 7f5cad460740 1 -- 172.21.15.83:0/106444581 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ae840b6a20 msgr2=0x55ae840b00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:53.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.897+0000 7f5cad460740 1 --2- 172.21.15.83:0/106444581 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ae840b6a20 0x55ae840b00d0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c09c580 tx=0x7f5c9c09c4d0 comp rx=0 tx=0).stop 2024-04-24T08:52:53.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.897+0000 7f5cad460740 1 -- 172.21.15.83:0/106444581 shutdown_connections 2024-04-24T08:52:53.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.897+0000 7f5cad460740 1 --2- 172.21.15.83:0/106444581 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ae840b6a20 0x55ae840b00d0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:53.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.897+0000 7f5cad460740 1 --2- 172.21.15.83:0/106444581 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae840b6e70 0x55ae840d1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:53.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.897+0000 7f5cad460740 1 --2- 172.21.15.83:0/106444581 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae840ae270 0x55ae840d1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:53.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.897+0000 7f5cad460740 1 -- 172.21.15.83:0/106444581 >> 172.21.15.83:0/106444581 conn(0x55ae840aa590 msgr2=0x55ae840af9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:53.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.897+0000 7f5cad460740 1 -- 172.21.15.83:0/106444581 shutdown_connections 2024-04-24T08:52:53.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.897+0000 7f5cad460740 1 -- 172.21.15.83:0/106444581 wait complete. 2024-04-24T08:52:53.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.897+0000 7f5cad460740 1 -- start start 2024-04-24T08:52:53.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.898+0000 7f5cad460740 1 Processor -- start 2024-04-24T08:52:53.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.898+0000 7f5cad460740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae840ae270 0x55ae840be3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:53.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.898+0000 7f5cad460740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae840b6a20 0x55ae840be900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:53.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.898+0000 7f5cad460740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ae840b6e70 0x55ae840bee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:53.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.898+0000 7f5cad460740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca41b28f0 con 0x55ae840b6e70 2024-04-24T08:52:53.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.898+0000 7f5cad460740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca41b2220 con 0x55ae840b6a20 2024-04-24T08:52:53.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.898+0000 7f5cad460740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca41af460 con 0x55ae840ae270 2024-04-24T08:52:53.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.898+0000 7f5cad460740 10 client.0 _ll_register_callbacks cb 0x7f5c90011440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:52:53.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.899+0000 7f5ca3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ae840b6e70 0x55ae840bee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:53.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.899+0000 7f5ca3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ae840b6e70 0x55ae840bee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54710/0 (socket says 172.21.15.83:54710) 2024-04-24T08:52:53.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.899+0000 7f5ca3e00640 1 -- 172.21.15.83:0/2773113071 learned_addr learned my addr 172.21.15.83:0/2773113071 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:52:53.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.899+0000 7f5ca3400640 1 --2- 172.21.15.83:0/2773113071 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae840ae270 0x55ae840be3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:53.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.899+0000 7f5ca2a00640 1 --2- 172.21.15.83:0/2773113071 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae840b6a20 0x55ae840be900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:53.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.899+0000 7f5ca3e00640 1 -- 172.21.15.83:0/2773113071 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae840ae270 msgr2=0x55ae840be3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:53.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.899+0000 7f5ca3e00640 1 --2- 172.21.15.83:0/2773113071 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae840ae270 0x55ae840be3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:53.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.899+0000 7f5ca3e00640 1 -- 172.21.15.83:0/2773113071 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae840b6a20 msgr2=0x55ae840be900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:53.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.899+0000 7f5ca3e00640 1 --2- 172.21.15.83:0/2773113071 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae840b6a20 0x55ae840be900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:53.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.899+0000 7f5ca3e00640 1 -- 172.21.15.83:0/2773113071 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae841389b0 con 0x55ae840b6e70 2024-04-24T08:52:53.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.899+0000 7f5ca3400640 1 --2- 172.21.15.83:0/2773113071 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae840ae270 0x55ae840be3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:52:53.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.900+0000 7f5ca3e00640 1 --2- 172.21.15.83:0/2773113071 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ae840b6e70 0x55ae840bee40 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c00a6e0 tx=0x7f5c9c098c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=678d37fc3279d54e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:53.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.900+0000 7f5c8d600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:52:53.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.900+0000 7f5c8d600640 1 -- 172.21.15.83:0/2773113071 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c9c01d040 con 0x55ae840b6e70 2024-04-24T08:52:53.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.900+0000 7f5c8d600640 1 -- 172.21.15.83:0/2773113071 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f5c9c09dbb0 con 0x55ae840b6e70 2024-04-24T08:52:53.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.900+0000 7f5cad460740 1 -- 172.21.15.83:0/2773113071 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c9c0a54f0 con 0x55ae840b6e70 2024-04-24T08:52:53.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.900+0000 7f5c8d600640 1 -- 172.21.15.83:0/2773113071 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5c9c005620 con 0x55ae840b6e70 2024-04-24T08:52:53.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.900+0000 7f5cad460740 10 client.16197 fetch_fsmap learned FSMap version 28 2024-04-24T08:52:53.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.900+0000 7f5cad460740 1 -- 172.21.15.83:0/2773113071 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ae84139c90 con 0x55ae840b6e70 2024-04-24T08:52:53.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.900+0000 7f5c8d600640 1 -- 172.21.15.83:0/2773113071 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5c9c0a54f0 con 0x55ae840b6e70 2024-04-24T08:52:53.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.901+0000 7f5c8d600640 1 -- 172.21.15.83:0/2773113071 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f5c9c0986f0 con 0x55ae840b6e70 2024-04-24T08:52:53.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.901+0000 7f5cad460740 10 client.16197 fetch_fsmap finished waiting for FSMap version 28 2024-04-24T08:52:53.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.901+0000 7f5cad460740 10 client.16197 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:52:53.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.901+0000 7f5cad460740 20 client.16197 populate_metadata read hostname 'smithi083' 2024-04-24T08:52:53.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.901+0000 7f5cad460740 1 --2- 172.21.15.83:0/2773113071 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ae8413a2a0 0x55ae8413c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:53.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.901+0000 7f5cad460740 4 client.16197 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T08:52:53.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.901+0000 7f5cad460740 1 -- 172.21.15.83:0/2773113071 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ae8401b700 con 0x55ae8413a2a0 2024-04-24T08:52:53.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.901+0000 7f5ca3400640 1 --2- 172.21.15.83:0/2773113071 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ae8413a2a0 0x55ae8413c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:53.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.902+0000 7f5ca3400640 1 --2- 172.21.15.83:0/2773113071 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ae8413a2a0 0x55ae8413c6c0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9bc5155de0e9b27f server_cookie=b5507a6197e60bdb in_seq=0 out_seq=0 2024-04-24T08:52:53.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.902+0000 7f5c8d600640 10 client.16197 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:52:53.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.905+0000 7f5c8d600640 1 -- 172.21.15.83:0/2773113071 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ae8401b700 con 0x55ae8413a2a0 2024-04-24T08:52:53.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.905+0000 7f5c8d600640 10 client.16197 handle_command_reply: tid=0 2024-04-24T08:52:53.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.905+0000 7f5cad460740 1 client.16197 shutdown 2024-04-24T08:52:53.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.906+0000 7f5cad460740 1 -- 172.21.15.83:0/2773113071 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ae840b6e70 msgr2=0x55ae840bee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:53.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.906+0000 7f5cad460740 1 --2- 172.21.15.83:0/2773113071 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ae840b6e70 0x55ae840bee40 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c00a6e0 tx=0x7f5c9c098c50 comp rx=0 tx=0).stop 2024-04-24T08:52:53.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.906+0000 7f5cad460740 1 -- 172.21.15.83:0/2773113071 shutdown_connections 2024-04-24T08:52:53.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.906+0000 7f5cad460740 1 --2- 172.21.15.83:0/2773113071 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ae8413a2a0 0x55ae8413c6c0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:53.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.906+0000 7f5cad460740 1 --2- 172.21.15.83:0/2773113071 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ae840b6e70 0x55ae840bee40 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:53.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.906+0000 7f5cad460740 1 --2- 172.21.15.83:0/2773113071 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae840b6a20 0x55ae840be900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:53.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.906+0000 7f5cad460740 1 --2- 172.21.15.83:0/2773113071 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae840ae270 0x55ae840be3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:53.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.906+0000 7f5cad460740 1 -- 172.21.15.83:0/2773113071 >> 172.21.15.83:0/2773113071 conn(0x55ae840aa590 msgr2=0x55ae840b0250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:53.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.906+0000 7f5c8d600640 0 client.16197 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:52:53.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.906+0000 7f5cad460740 1 -- 172.21.15.83:0/2773113071 shutdown_connections 2024-04-24T08:52:53.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.906+0000 7f5cad460740 1 -- 172.21.15.83:0/2773113071 wait complete. 2024-04-24T08:52:53.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.906+0000 7f5cad460740 20 client.16197 trim_cache size 0 max 16384 2024-04-24T08:52:53.908 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T08:52:53.908 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-04-24T08:52:53.908 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "6614d42a-77ef-43cd-bd4f-53a11e538c45", 2024-04-24T08:52:53.908 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-04-24T08:52:53.908 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T08:52:53.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.907+0000 7f5ca9000640 1 -- 172.21.15.83:0/1794889733 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f5c880a3ec0 msgr2=0x7f5c880a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:53.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.907+0000 7f5ca9000640 1 --2- 172.21.15.83:0/1794889733 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f5c880a3ec0 0x7f5c880a6380 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x55ae840ab720 tx=0x7f5c90005f90 comp rx=0 tx=0).stop 2024-04-24T08:52:53.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.907+0000 7f5ca9000640 1 -- 172.21.15.83:0/1794889733 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ca419a6a0 msgr2=0x7f5ca419ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:53.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.907+0000 7f5ca9000640 1 --2- 172.21.15.83:0/1794889733 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ca419a6a0 0x7f5ca419ab20 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c00cad0 tx=0x7f5c9c00cf90 comp rx=0 tx=0).stop 2024-04-24T08:52:53.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.907+0000 7f5ca9000640 1 -- 172.21.15.83:0/1794889733 shutdown_connections 2024-04-24T08:52:53.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.907+0000 7f5ca9000640 1 --2- 172.21.15.83:0/1794889733 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f5c880a3ec0 0x7f5c880a6380 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:53.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.907+0000 7f5ca9000640 1 --2- 172.21.15.83:0/1794889733 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ca419a6a0 0x7f5ca419ab20 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:53.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.907+0000 7f5ca9000640 1 --2- 172.21.15.83:0/1794889733 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ca41093f0 0x7f5ca41a0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:53.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.907+0000 7f5ca9000640 1 --2- 172.21.15.83:0/1794889733 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5ca4104320 0x7f5ca41a05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:53.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.907+0000 7f5ca9000640 1 -- 172.21.15.83:0/1794889733 >> 172.21.15.83:0/1794889733 conn(0x7f5ca40ffb40 msgr2=0x7f5ca4101ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:53.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.907+0000 7f5ca9000640 1 -- 172.21.15.83:0/1794889733 shutdown_connections 2024-04-24T08:52:53.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:53.907+0000 7f5ca9000640 1 -- 172.21.15.83:0/1794889733 wait complete. 2024-04-24T08:52:53.926 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T08:52:53.927 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T08:52:53.954 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:52:53 smithi138 ceph-mon[31441]: pgmap v41: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-24T08:52:53.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:52:53 smithi189 ceph-mon[31791]: pgmap v41: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-24T08:52:53.977 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:52:53 smithi083 ceph-mon[27907]: pgmap v41: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-24T08:52:54.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.148+0000 7fb847e00640 1 -- 172.21.15.83:0/3158861245 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb840102240 msgr2=0x7fb840102620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:54.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.148+0000 7fb847e00640 1 --2- 172.21.15.83:0/3158861245 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb840102240 0x7fb840102620 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fb834009900 tx=0x7fb83402a3f0 comp rx=0 tx=0).stop 2024-04-24T08:52:54.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.148+0000 7fb847e00640 1 -- 172.21.15.83:0/3158861245 shutdown_connections 2024-04-24T08:52:54.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.148+0000 7fb847e00640 1 --2- 172.21.15.83:0/3158861245 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb840107310 0x7fb84010eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:54.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.148+0000 7fb847e00640 1 --2- 172.21.15.83:0/3158861245 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb840102bf0 0x7fb840106c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:54.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.148+0000 7fb847e00640 1 --2- 172.21.15.83:0/3158861245 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb840102240 0x7fb840102620 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:54.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.148+0000 7fb847e00640 1 -- 172.21.15.83:0/3158861245 >> 172.21.15.83:0/3158861245 conn(0x7fb840079450 msgr2=0x7fb840079860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:54.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.148+0000 7fb847e00640 1 -- 172.21.15.83:0/3158861245 shutdown_connections 2024-04-24T08:52:54.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.148+0000 7fb847e00640 1 -- 172.21.15.83:0/3158861245 wait complete. 2024-04-24T08:52:54.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.149+0000 7fb847e00640 1 Processor -- start 2024-04-24T08:52:54.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.150+0000 7fb847e00640 1 -- start start 2024-04-24T08:52:54.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.150+0000 7fb847e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb840102240 0x7fb84019e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:54.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.150+0000 7fb847e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb840107310 0x7fb84019e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:54.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.150+0000 7fb847e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb8401985f0 0x7fb840198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:54.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.150+0000 7fb847e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb8401122d0 con 0x7fb8401985f0 2024-04-24T08:52:54.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.150+0000 7fb847e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb840112150 con 0x7fb840102240 2024-04-24T08:52:54.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.150+0000 7fb847e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb840112450 con 0x7fb840107310 2024-04-24T08:52:54.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.151+0000 7fb845e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb8401985f0 0x7fb840198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:54.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.151+0000 7fb845e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb8401985f0 0x7fb840198aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54724/0 (socket says 172.21.15.83:54724) 2024-04-24T08:52:54.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.151+0000 7fb845e00640 1 -- 172.21.15.83:0/3142947468 learned_addr learned my addr 172.21.15.83:0/3142947468 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:52:54.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.151+0000 7fb844a00640 1 --2- 172.21.15.83:0/3142947468 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb840107310 0x7fb84019e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:54.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.151+0000 7fb845400640 1 --2- 172.21.15.83:0/3142947468 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb840102240 0x7fb84019e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:54.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.151+0000 7fb845e00640 1 -- 172.21.15.83:0/3142947468 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb840107310 msgr2=0x7fb84019e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:54.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.151+0000 7fb845e00640 1 --2- 172.21.15.83:0/3142947468 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb840107310 0x7fb84019e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:54.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.151+0000 7fb845e00640 1 -- 172.21.15.83:0/3142947468 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb840102240 msgr2=0x7fb84019e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:54.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.151+0000 7fb845e00640 1 --2- 172.21.15.83:0/3142947468 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb840102240 0x7fb84019e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:54.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.151+0000 7fb845e00640 1 -- 172.21.15.83:0/3142947468 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb840199110 con 0x7fb8401985f0 2024-04-24T08:52:54.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.151+0000 7fb844a00640 1 --2- 172.21.15.83:0/3142947468 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb840107310 0x7fb84019e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:52:54.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.152+0000 7fb845e00640 1 --2- 172.21.15.83:0/3142947468 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb8401985f0 0x7fb840198aa0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fb83800eaa0 tx=0x7fb83800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:54.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.152+0000 7fb83e000640 1 -- 172.21.15.83:0/3142947468 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb83800cdb0 con 0x7fb8401985f0 2024-04-24T08:52:54.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.152+0000 7fb83e000640 1 -- 172.21.15.83:0/3142947468 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb838005020 con 0x7fb8401985f0 2024-04-24T08:52:54.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.152+0000 7fb83e000640 1 -- 172.21.15.83:0/3142947468 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8380116d0 con 0x7fb8401985f0 2024-04-24T08:52:54.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.152+0000 7fb847e00640 1 -- 172.21.15.83:0/3142947468 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb840199400 con 0x7fb8401985f0 2024-04-24T08:52:54.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.152+0000 7fb847e00640 1 -- 172.21.15.83:0/3142947468 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8400745d0 con 0x7fb8401985f0 2024-04-24T08:52:54.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.156+0000 7fb84ac3a740 1 Processor -- start 2024-04-24T08:52:54.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.157+0000 7fb83e000640 1 -- 172.21.15.83:0/3142947468 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7fb8380051d0 con 0x7fb8401985f0 2024-04-24T08:52:54.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.157+0000 7fb84ac3a740 1 -- start start 2024-04-24T08:52:54.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.157+0000 7fb84ac3a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561561e8d860 0x561561e8dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:54.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.157+0000 7fb84ac3a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561561e8e180 0x561561e98310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:54.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.158+0000 7fb84ac3a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561561e98850 0x561561e9ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:54.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.158+0000 7fb84ac3a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561561df31f0 con 0x561561e8e180 2024-04-24T08:52:54.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.158+0000 7fb84ac3a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x561561d99350 con 0x561561e98850 2024-04-24T08:52:54.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.158+0000 7fb84ac3a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x561561def1b0 con 0x561561e8d860 2024-04-24T08:52:54.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.158+0000 7fb845400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561561e8e180 0x561561e98310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:54.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.158+0000 7fb845400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561561e8e180 0x561561e98310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54732/0 (socket says 172.21.15.83:54732) 2024-04-24T08:52:54.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.158+0000 7fb845400640 1 -- 172.21.15.83:0/2348563218 learned_addr learned my addr 172.21.15.83:0/2348563218 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:52:54.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.158+0000 7fb844a00640 1 --2- 172.21.15.83:0/2348563218 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561561e8d860 0x561561e8dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:54.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.158+0000 7fb844a00640 1 --2- 172.21.15.83:0/2348563218 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561561e98850 0x561561e9ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:54.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.159+0000 7fb83e000640 1 --2- 172.21.15.83:0/3142947468 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb8280a3ec0 0x7fb8280a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:54.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.159+0000 7fb83e000640 1 -- 172.21.15.83:0/3142947468 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7fb8380d0c10 con 0x7fb8401985f0 2024-04-24T08:52:54.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.159+0000 7fb845400640 1 -- 172.21.15.83:0/2348563218 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561561e8d860 msgr2=0x561561e8dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:54.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.159+0000 7fb845400640 1 --2- 172.21.15.83:0/2348563218 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561561e8d860 0x561561e8dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:54.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.159+0000 7fb845400640 1 -- 172.21.15.83:0/2348563218 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561561e98850 msgr2=0x561561e9ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:54.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.159+0000 7fb845400640 1 --2- 172.21.15.83:0/2348563218 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561561e98850 0x561561e9ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:54.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.159+0000 7fb845400640 1 -- 172.21.15.83:0/2348563218 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561561dedf50 con 0x561561e8e180 2024-04-24T08:52:54.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.159+0000 7fb845e00640 1 --2- 172.21.15.83:0/3142947468 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb8280a3ec0 0x7fb8280a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:54.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.160+0000 7fb845400640 1 --2- 172.21.15.83:0/2348563218 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561561e8e180 0x561561e98310 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fb834009230 tx=0x7fb834004310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9da0b6c6ce01e9f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:54.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.160+0000 7fb832600640 1 -- 172.21.15.83:0/2348563218 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb834035040 con 0x561561e8e180 2024-04-24T08:52:54.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.160+0000 7fb832600640 1 -- 172.21.15.83:0/2348563218 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb834004770 con 0x561561e8e180 2024-04-24T08:52:54.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.160+0000 7fb832600640 1 -- 172.21.15.83:0/2348563218 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb834003850 con 0x561561e8e180 2024-04-24T08:52:54.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.161+0000 7fb84ac3a740 1 -- 172.21.15.83:0/2348563218 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561561e8e180 msgr2=0x561561e98310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:54.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.161+0000 7fb845e00640 1 --2- 172.21.15.83:0/3142947468 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb8280a3ec0 0x7fb8280a6380 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb8380cde70 tx=0x7fb83800cf50 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:54.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.161+0000 7fb84ac3a740 1 --2- 172.21.15.83:0/2348563218 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561561e8e180 0x561561e98310 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fb834009230 tx=0x7fb834004310 comp rx=0 tx=0).stop 2024-04-24T08:52:54.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.161+0000 7fb84ac3a740 1 -- 172.21.15.83:0/2348563218 shutdown_connections 2024-04-24T08:52:54.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.161+0000 7fb84ac3a740 1 --2- 172.21.15.83:0/2348563218 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561561e98850 0x561561e9ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:54.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.161+0000 7fb84ac3a740 1 --2- 172.21.15.83:0/2348563218 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561561e8e180 0x561561e98310 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:54.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.161+0000 7fb84ac3a740 1 --2- 172.21.15.83:0/2348563218 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561561e8d860 0x561561e8dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:54.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.161+0000 7fb84ac3a740 1 -- 172.21.15.83:0/2348563218 >> 172.21.15.83:0/2348563218 conn(0x561561e89b80 msgr2=0x561561e89f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:54.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.161+0000 7fb84ac3a740 1 -- 172.21.15.83:0/2348563218 shutdown_connections 2024-04-24T08:52:54.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.161+0000 7fb84ac3a740 1 -- 172.21.15.83:0/2348563218 wait complete. 2024-04-24T08:52:54.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.162+0000 7fb84ac3a740 1 -- start start 2024-04-24T08:52:54.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.162+0000 7fb84ac3a740 1 Processor -- start 2024-04-24T08:52:54.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.163+0000 7fb84ac3a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561561e8d860 0x561561e957b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:54.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.163+0000 7fb84ac3a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561561e8e180 0x561561e95cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:54.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.163+0000 7fb84ac3a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561561e98850 0x561561e96230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:54.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.163+0000 7fb84ac3a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561561e9b5e0 con 0x561561e98850 2024-04-24T08:52:54.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.163+0000 7fb84ac3a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x561561dac690 con 0x561561e8d860 2024-04-24T08:52:54.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.163+0000 7fb84ac3a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x561561e9b760 con 0x561561e8e180 2024-04-24T08:52:54.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.163+0000 7fb84ac3a740 10 client.0 _ll_register_callbacks cb 0x561561de8650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:52:54.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.163+0000 7fb844a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561561e98850 0x561561e96230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:54.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.163+0000 7fb844a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561561e98850 0x561561e96230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54738/0 (socket says 172.21.15.83:54738) 2024-04-24T08:52:54.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.163+0000 7fb844a00640 1 -- 172.21.15.83:0/1499403636 learned_addr learned my addr 172.21.15.83:0/1499403636 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:52:54.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.163+0000 7fb844a00640 1 --2- 172.21.15.83:0/1499403636 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561561e8d860 0x561561e957b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:54.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.164+0000 7fb844a00640 1 -- 172.21.15.83:0/1499403636 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561561e8e180 msgr2=0x561561e95cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:54.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.164+0000 7fb845400640 1 --2- 172.21.15.83:0/1499403636 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561561e8e180 0x561561e95cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:54.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.164+0000 7fb844a00640 1 --2- 172.21.15.83:0/1499403636 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561561e8e180 0x561561e95cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:54.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.164+0000 7fb844a00640 1 -- 172.21.15.83:0/1499403636 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561561e8d860 msgr2=0x561561e957b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:54.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.164+0000 7fb844a00640 1 --2- 172.21.15.83:0/1499403636 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561561e8d860 0x561561e957b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:54.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.164+0000 7fb844a00640 1 -- 172.21.15.83:0/1499403636 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561561dab920 con 0x561561e98850 2024-04-24T08:52:54.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.164+0000 7fb844a00640 1 --2- 172.21.15.83:0/1499403636 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561561e98850 0x561561e96230 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fb82c0131f0 tx=0x7fb82c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d5c74e3ec8b27bc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:54.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.164+0000 7fb826a00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:52:54.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.164+0000 7fb826a00640 1 -- 172.21.15.83:0/1499403636 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb82c004030 con 0x561561e98850 2024-04-24T08:52:54.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.164+0000 7fb84ac3a740 1 -- 172.21.15.83:0/1499403636 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb834004770 con 0x561561e98850 2024-04-24T08:52:54.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.164+0000 7fb826a00640 1 -- 172.21.15.83:0/1499403636 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7fb82c0041d0 con 0x561561e98850 2024-04-24T08:52:54.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.165+0000 7fb826a00640 1 -- 172.21.15.83:0/1499403636 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb82c028050 con 0x561561e98850 2024-04-24T08:52:54.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.165+0000 7fb84ac3a740 10 client.16209 fetch_fsmap learned FSMap version 28 2024-04-24T08:52:54.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.165+0000 7fb84ac3a740 1 -- 172.21.15.83:0/1499403636 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561561bcf7d0 con 0x561561e98850 2024-04-24T08:52:54.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.165+0000 7fb826a00640 1 -- 172.21.15.83:0/1499403636 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb834004770 con 0x561561e98850 2024-04-24T08:52:54.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.165+0000 7fb826a00640 1 -- 172.21.15.83:0/1499403636 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb82c020420 con 0x561561e98850 2024-04-24T08:52:54.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.165+0000 7fb84ac3a740 10 client.16209 fetch_fsmap finished waiting for FSMap version 28 2024-04-24T08:52:54.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.165+0000 7fb84ac3a740 10 client.16209 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:52:54.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.165+0000 7fb84ac3a740 20 client.16209 populate_metadata read hostname 'smithi083' 2024-04-24T08:52:54.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.165+0000 7fb84ac3a740 1 --2- 172.21.15.83:0/1499403636 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x561561ea8920 0x561561eaad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:54.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.165+0000 7fb84ac3a740 4 client.16209 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:52:54.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.165+0000 7fb84ac3a740 1 -- 172.21.15.83:0/1499403636 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561561e00c10 con 0x561561ea8920 2024-04-24T08:52:54.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.166+0000 7fb845400640 1 --2- 172.21.15.83:0/1499403636 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x561561ea8920 0x561561eaad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:54.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.167+0000 7fb845400640 1 --2- 172.21.15.83:0/1499403636 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x561561ea8920 0x561561eaad80 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2bdd96ff925314a6 server_cookie=db02fbb655ae9765 in_seq=0 out_seq=0 2024-04-24T08:52:54.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.167+0000 7fb826a00640 10 client.16209 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:52:54.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.169+0000 7fb826a00640 1 -- 172.21.15.83:0/1499403636 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561561e00c10 con 0x561561ea8920 2024-04-24T08:52:54.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.169+0000 7fb826a00640 10 client.16209 handle_command_reply: tid=0 2024-04-24T08:52:54.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.169+0000 7fb84ac3a740 1 client.16209 shutdown 2024-04-24T08:52:54.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.169+0000 7fb84ac3a740 1 -- 172.21.15.83:0/1499403636 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561561e98850 msgr2=0x561561e96230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:54.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.169+0000 7fb84ac3a740 1 --2- 172.21.15.83:0/1499403636 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561561e98850 0x561561e96230 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fb82c0131f0 tx=0x7fb82c0046c0 comp rx=0 tx=0).stop 2024-04-24T08:52:54.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.170+0000 7fb84ac3a740 1 -- 172.21.15.83:0/1499403636 shutdown_connections 2024-04-24T08:52:54.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.170+0000 7fb84ac3a740 1 --2- 172.21.15.83:0/1499403636 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x561561ea8920 0x561561eaad80 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:54.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.170+0000 7fb84ac3a740 1 --2- 172.21.15.83:0/1499403636 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561561e98850 0x561561e96230 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:54.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.170+0000 7fb84ac3a740 1 --2- 172.21.15.83:0/1499403636 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561561e8e180 0x561561e95cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:54.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.170+0000 7fb84ac3a740 1 --2- 172.21.15.83:0/1499403636 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561561e8d860 0x561561e957b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:54.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.170+0000 7fb84ac3a740 1 -- 172.21.15.83:0/1499403636 >> 172.21.15.83:0/1499403636 conn(0x561561e89b80 msgr2=0x561561e8f980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:54.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.170+0000 7fb826a00640 0 client.16209 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:52:54.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.170+0000 7fb84ac3a740 1 -- 172.21.15.83:0/1499403636 shutdown_connections 2024-04-24T08:52:54.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.170+0000 7fb84ac3a740 1 -- 172.21.15.83:0/1499403636 wait complete. 2024-04-24T08:52:54.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.170+0000 7fb84ac3a740 20 client.16209 trim_cache size 0 max 16384 2024-04-24T08:52:54.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.194+0000 7fb84ac3a740 1 Processor -- start 2024-04-24T08:52:54.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.194+0000 7fb84ac3a740 1 -- start start 2024-04-24T08:52:54.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.194+0000 7fb84ac3a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561561e8d860 0x561561ea44a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:54.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.194+0000 7fb84ac3a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561561e8e180 0x561561e90000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:54.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.194+0000 7fb84ac3a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561561e98850 0x561561e88aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:54.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.194+0000 7fb84ac3a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb8401b1710 con 0x561561e8d860 2024-04-24T08:52:54.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.194+0000 7fb84ac3a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb8401b3250 con 0x561561e98850 2024-04-24T08:52:54.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.194+0000 7fb84ac3a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb8401b1de0 con 0x561561e8e180 2024-04-24T08:52:54.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.194+0000 7fb844a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561561e8d860 0x561561ea44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:54.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.195+0000 7fb844a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561561e8d860 0x561561ea44a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54748/0 (socket says 172.21.15.83:54748) 2024-04-24T08:52:54.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.195+0000 7fb844a00640 1 -- 172.21.15.83:0/4228338790 learned_addr learned my addr 172.21.15.83:0/4228338790 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:52:54.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.195+0000 7fb844a00640 1 -- 172.21.15.83:0/4228338790 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561561e8e180 msgr2=0x561561e90000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:54.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.195+0000 7fb844a00640 1 --2- 172.21.15.83:0/4228338790 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561561e8e180 0x561561e90000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:54.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.195+0000 7fb844a00640 1 -- 172.21.15.83:0/4228338790 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561561e98850 msgr2=0x561561e88aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:54.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.195+0000 7fb844a00640 1 --2- 172.21.15.83:0/4228338790 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561561e98850 0x561561e88aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:54.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.195+0000 7fb844a00640 1 -- 172.21.15.83:0/4228338790 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561561b053f0 con 0x561561e8d860 2024-04-24T08:52:54.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.195+0000 7fb844a00640 1 --2- 172.21.15.83:0/4228338790 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561561e8d860 0x561561ea44a0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fb82c026040 tx=0x7fb82c014870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96bcd18531ce4741 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:54.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.195+0000 7fb831c00640 1 -- 172.21.15.83:0/4228338790 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb82c024030 con 0x561561e8d860 2024-04-24T08:52:54.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.196+0000 7fb831c00640 1 -- 172.21.15.83:0/4228338790 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb82c00b7c0 con 0x561561e8d860 2024-04-24T08:52:54.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.196+0000 7fb831c00640 1 -- 172.21.15.83:0/4228338790 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb82c0205f0 con 0x561561e8d860 2024-04-24T08:52:54.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.196+0000 7fb84ac3a740 1 -- 172.21.15.83:0/4228338790 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561561e8d860 msgr2=0x561561ea44a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:54.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.196+0000 7fb84ac3a740 1 --2- 172.21.15.83:0/4228338790 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561561e8d860 0x561561ea44a0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fb82c026040 tx=0x7fb82c014870 comp rx=0 tx=0).stop 2024-04-24T08:52:54.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.196+0000 7fb84ac3a740 1 -- 172.21.15.83:0/4228338790 shutdown_connections 2024-04-24T08:52:54.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.196+0000 7fb84ac3a740 1 --2- 172.21.15.83:0/4228338790 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561561e98850 0x561561e88aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:54.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.196+0000 7fb84ac3a740 1 --2- 172.21.15.83:0/4228338790 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561561e8e180 0x561561e90000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:54.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.196+0000 7fb84ac3a740 1 --2- 172.21.15.83:0/4228338790 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561561e8d860 0x561561ea44a0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:54.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.196+0000 7fb84ac3a740 1 -- 172.21.15.83:0/4228338790 >> 172.21.15.83:0/4228338790 conn(0x561561e89b80 msgr2=0x561561e88390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:54.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.197+0000 7fb84ac3a740 1 -- 172.21.15.83:0/4228338790 shutdown_connections 2024-04-24T08:52:54.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.197+0000 7fb84ac3a740 1 -- 172.21.15.83:0/4228338790 wait complete. 2024-04-24T08:52:54.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.197+0000 7fb84ac3a740 1 -- start start 2024-04-24T08:52:54.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.197+0000 7fb84ac3a740 1 Processor -- start 2024-04-24T08:52:54.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.198+0000 7fb84ac3a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561561e8d860 0x561561ea09b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:54.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.198+0000 7fb84ac3a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561561e8e180 0x561561ea0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:54.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.198+0000 7fb84ac3a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561561e98850 0x561561e9c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:54.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.198+0000 7fb84ac3a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb8401b2b80 con 0x561561e8e180 2024-04-24T08:52:54.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.198+0000 7fb84ac3a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb8401b24b0 con 0x561561e8d860 2024-04-24T08:52:54.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.198+0000 7fb84ac3a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb8401af6f0 con 0x561561e98850 2024-04-24T08:52:54.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.198+0000 7fb84ac3a740 10 client.0 _ll_register_callbacks cb 0x7fb82c014c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:52:54.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.198+0000 7fb845400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561561e8e180 0x561561ea0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:54.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.198+0000 7fb845400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561561e8e180 0x561561ea0ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54762/0 (socket says 172.21.15.83:54762) 2024-04-24T08:52:54.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.198+0000 7fb845400640 1 -- 172.21.15.83:0/3603524354 learned_addr learned my addr 172.21.15.83:0/3603524354 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:52:54.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.198+0000 7fb844a00640 1 --2- 172.21.15.83:0/3603524354 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561561e8d860 0x561561ea09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:54.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.198+0000 7fb844a00640 1 --2- 172.21.15.83:0/3603524354 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561561e98850 0x561561e9c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:54.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.199+0000 7fb845400640 1 -- 172.21.15.83:0/3603524354 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561561e98850 msgr2=0x561561e9c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:54.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.199+0000 7fb845400640 1 --2- 172.21.15.83:0/3603524354 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561561e98850 0x561561e9c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:54.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.199+0000 7fb845400640 1 -- 172.21.15.83:0/3603524354 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561561e8d860 msgr2=0x561561ea09b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:54.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.199+0000 7fb845400640 1 --2- 172.21.15.83:0/3603524354 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561561e8d860 0x561561ea09b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:54.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.199+0000 7fb845400640 1 -- 172.21.15.83:0/3603524354 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561561ac50f0 con 0x561561e8e180 2024-04-24T08:52:54.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.199+0000 7fb845400640 1 --2- 172.21.15.83:0/3603524354 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561561e8e180 0x561561ea0ef0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fb834005dc0 tx=0x7fb8340316d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=595237c664501519 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:54.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.199+0000 7fb826a00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:52:54.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.199+0000 7fb826a00640 1 -- 172.21.15.83:0/3603524354 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb834035040 con 0x561561e8e180 2024-04-24T08:52:54.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.199+0000 7fb826a00640 1 -- 172.21.15.83:0/3603524354 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7fb834031bd0 con 0x561561e8e180 2024-04-24T08:52:54.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.199+0000 7fb84ac3a740 1 -- 172.21.15.83:0/3603524354 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb82c00b7c0 con 0x561561e8e180 2024-04-24T08:52:54.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.200+0000 7fb826a00640 1 -- 172.21.15.83:0/3603524354 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb834004770 con 0x561561e8e180 2024-04-24T08:52:54.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.200+0000 7fb84ac3a740 10 client.16215 fetch_fsmap learned FSMap version 28 2024-04-24T08:52:54.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.200+0000 7fb84ac3a740 1 -- 172.21.15.83:0/3603524354 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561561db8810 con 0x561561e8e180 2024-04-24T08:52:54.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.200+0000 7fb826a00640 1 -- 172.21.15.83:0/3603524354 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb82c00b7c0 con 0x561561e8e180 2024-04-24T08:52:54.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.200+0000 7fb826a00640 1 -- 172.21.15.83:0/3603524354 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb83404e470 con 0x561561e8e180 2024-04-24T08:52:54.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.200+0000 7fb84ac3a740 10 client.16215 fetch_fsmap finished waiting for FSMap version 28 2024-04-24T08:52:54.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.200+0000 7fb84ac3a740 10 client.16215 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:52:54.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.200+0000 7fb84ac3a740 20 client.16215 populate_metadata read hostname 'smithi083' 2024-04-24T08:52:54.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.200+0000 7fb84ac3a740 1 --2- 172.21.15.83:0/3603524354 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x561561f0eb10 0x561561f10f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:54.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.200+0000 7fb84ac3a740 4 client.16215 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T08:52:54.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.200+0000 7fb84ac3a740 1 -- 172.21.15.83:0/3603524354 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561561dfaa80 con 0x561561f0eb10 2024-04-24T08:52:54.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.201+0000 7fb844a00640 1 --2- 172.21.15.83:0/3603524354 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x561561f0eb10 0x561561f10f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:54.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.202+0000 7fb844a00640 1 --2- 172.21.15.83:0/3603524354 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x561561f0eb10 0x561561f10f00 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15333245d386b1a2 server_cookie=8f2e5ebc1de3f4c1 in_seq=0 out_seq=0 2024-04-24T08:52:54.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.202+0000 7fb826a00640 10 client.16215 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:52:54.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.202+0000 7fb826a00640 1 -- 172.21.15.83:0/3603524354 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x561561dfaa80 con 0x561561f0eb10 2024-04-24T08:52:54.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.202+0000 7fb826a00640 10 client.16215 handle_command_reply: tid=0 2024-04-24T08:52:54.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.202+0000 7fb84ac3a740 1 client.16215 shutdown 2024-04-24T08:52:54.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.202+0000 7fb84ac3a740 1 -- 172.21.15.83:0/3603524354 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561561e8e180 msgr2=0x561561ea0ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:54.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.202+0000 7fb84ac3a740 1 --2- 172.21.15.83:0/3603524354 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561561e8e180 0x561561ea0ef0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fb834005dc0 tx=0x7fb8340316d0 comp rx=0 tx=0).stop 2024-04-24T08:52:54.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.202+0000 7fb84ac3a740 1 -- 172.21.15.83:0/3603524354 shutdown_connections 2024-04-24T08:52:54.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.202+0000 7fb84ac3a740 1 --2- 172.21.15.83:0/3603524354 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x561561f0eb10 0x561561f10f00 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:54.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.202+0000 7fb84ac3a740 1 --2- 172.21.15.83:0/3603524354 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561561e98850 0x561561e9c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:54.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.202+0000 7fb84ac3a740 1 --2- 172.21.15.83:0/3603524354 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561561e8e180 0x561561ea0ef0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:54.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.202+0000 7fb84ac3a740 1 --2- 172.21.15.83:0/3603524354 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561561e8d860 0x561561ea09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:54.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.202+0000 7fb84ac3a740 1 -- 172.21.15.83:0/3603524354 >> 172.21.15.83:0/3603524354 conn(0x561561e89b80 msgr2=0x561561e88c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:54.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.202+0000 7fb826a00640 0 client.16215 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:52:54.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.203+0000 7fb84ac3a740 1 -- 172.21.15.83:0/3603524354 shutdown_connections 2024-04-24T08:52:54.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.203+0000 7fb84ac3a740 1 -- 172.21.15.83:0/3603524354 wait complete. 2024-04-24T08:52:54.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.203+0000 7fb84ac3a740 20 client.16215 trim_cache size 0 max 16384 2024-04-24T08:52:54.206 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T08:52:54.206 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T08:52:54.206 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-04-24T08:52:54.206 INFO:teuthology.orchestra.run.smithi083.stdout: "6614d42a-77ef-43cd-bd4f-53a11e538c45": { 2024-04-24T08:52:54.206 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-04-24T08:52:54.206 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "6614d42a-77ef-43cd-bd4f-53a11e538c45", 2024-04-24T08:52:54.206 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-04-24T08:52:54.206 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T08:52:54.206 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T08:52:54.206 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T08:52:54.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.203+0000 7fb847e00640 1 -- 172.21.15.83:0/3142947468 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb8280a3ec0 msgr2=0x7fb8280a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:54.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.203+0000 7fb847e00640 1 --2- 172.21.15.83:0/3142947468 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb8280a3ec0 0x7fb8280a6380 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb8380cde70 tx=0x7fb83800cf50 comp rx=0 tx=0).stop 2024-04-24T08:52:54.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.203+0000 7fb847e00640 1 -- 172.21.15.83:0/3142947468 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb8401985f0 msgr2=0x7fb840198aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:54.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.203+0000 7fb847e00640 1 --2- 172.21.15.83:0/3142947468 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb8401985f0 0x7fb840198aa0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fb83800eaa0 tx=0x7fb83800ef60 comp rx=0 tx=0).stop 2024-04-24T08:52:54.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.204+0000 7fb847e00640 1 -- 172.21.15.83:0/3142947468 shutdown_connections 2024-04-24T08:52:54.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.204+0000 7fb847e00640 1 --2- 172.21.15.83:0/3142947468 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb8280a3ec0 0x7fb8280a6380 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:54.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.204+0000 7fb847e00640 1 --2- 172.21.15.83:0/3142947468 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb8401985f0 0x7fb840198aa0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:54.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.204+0000 7fb847e00640 1 --2- 172.21.15.83:0/3142947468 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb840107310 0x7fb84019e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:54.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.204+0000 7fb847e00640 1 --2- 172.21.15.83:0/3142947468 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb840102240 0x7fb84019e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:54.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.204+0000 7fb847e00640 1 -- 172.21.15.83:0/3142947468 >> 172.21.15.83:0/3142947468 conn(0x7fb840079450 msgr2=0x7fb8400ffee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:54.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.204+0000 7fb847e00640 1 -- 172.21.15.83:0/3142947468 shutdown_connections 2024-04-24T08:52:54.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:52:54.204+0000 7fb847e00640 1 -- 172.21.15.83:0/3142947468 wait complete. 2024-04-24T08:52:54.220 INFO:tasks.cephfs.filesystem:scrub status for tag:6614d42a-77ef-43cd-bd4f-53a11e538c45 - {'path': '/', 'tag': '6614d42a-77ef-43cd-bd4f-53a11e538c45', 'options': 'recursive,force'} 2024-04-24T08:52:54.790 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.790+0000 7fc660800640 1 -- 172.21.15.83:0/2906288630 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc658107310 msgr2=0x7fc65810eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:54.790 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.790+0000 7fc660800640 1 --2- 172.21.15.83:0/2906288630 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc658107310 0x7fc65810eee0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fc654009ca0 tx=0x7fc65402a8c0 comp rx=0 tx=0).stop 2024-04-24T08:52:54.791 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.790+0000 7fc660800640 1 -- 172.21.15.83:0/2906288630 shutdown_connections 2024-04-24T08:52:54.791 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.790+0000 7fc660800640 1 --2- 172.21.15.83:0/2906288630 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc658107310 0x7fc65810eee0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:54.791 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.790+0000 7fc660800640 1 --2- 172.21.15.83:0/2906288630 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc65806b450 0x7fc658106c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:54.791 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.790+0000 7fc660800640 1 --2- 172.21.15.83:0/2906288630 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc65806aaa0 0x7fc65806ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:54.791 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.790+0000 7fc660800640 1 -- 172.21.15.83:0/2906288630 >> 172.21.15.83:0/2906288630 conn(0x7fc658076c30 msgr2=0x7fc658077040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:54.791 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.790+0000 7fc660800640 1 -- 172.21.15.83:0/2906288630 shutdown_connections 2024-04-24T08:52:54.791 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.790+0000 7fc660800640 1 -- 172.21.15.83:0/2906288630 wait complete. 2024-04-24T08:52:54.791 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.791+0000 7fc660800640 1 Processor -- start 2024-04-24T08:52:54.792 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.791+0000 7fc660800640 1 -- start start 2024-04-24T08:52:54.793 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.792+0000 7fc660800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc65806aaa0 0x7fc658198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:54.793 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.792+0000 7fc660800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc658107310 0x7fc658198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:54.793 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.792+0000 7fc660800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc65819cfa0 0x7fc65819d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:54.793 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.792+0000 7fc660800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc6581122d0 con 0x7fc658107310 2024-04-24T08:52:54.793 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.792+0000 7fc660800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc658112150 con 0x7fc65819cfa0 2024-04-24T08:52:54.794 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.792+0000 7fc660800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc658112450 con 0x7fc65806aaa0 2024-04-24T08:52:54.794 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.792+0000 7fc65d400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc658107310 0x7fc658198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:54.794 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.792+0000 7fc65d400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc658107310 0x7fc658198c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54782/0 (socket says 172.21.15.83:54782) 2024-04-24T08:52:54.794 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.792+0000 7fc65d400640 1 -- 172.21.15.83:0/2949943431 learned_addr learned my addr 172.21.15.83:0/2949943431 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:52:54.794 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.793+0000 7fc65de00640 1 --2- 172.21.15.83:0/2949943431 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc65806aaa0 0x7fc658198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:54.794 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.793+0000 7fc65e800640 1 --2- 172.21.15.83:0/2949943431 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc65819cfa0 0x7fc65819d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:54.794 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.793+0000 7fc65d400640 1 -- 172.21.15.83:0/2949943431 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc65806aaa0 msgr2=0x7fc658198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:54.794 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.793+0000 7fc65d400640 1 --2- 172.21.15.83:0/2949943431 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc65806aaa0 0x7fc658198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:54.794 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.793+0000 7fc65d400640 1 -- 172.21.15.83:0/2949943431 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc65819cfa0 msgr2=0x7fc65819d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:54.794 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.793+0000 7fc65d400640 1 --2- 172.21.15.83:0/2949943431 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc65819cfa0 0x7fc65819d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:54.794 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.793+0000 7fc65d400640 1 -- 172.21.15.83:0/2949943431 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc65819da90 con 0x7fc658107310 2024-04-24T08:52:54.794 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.793+0000 7fc65e800640 1 --2- 172.21.15.83:0/2949943431 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc65819cfa0 0x7fc65819d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:52:54.794 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.793+0000 7fc65de00640 1 --2- 172.21.15.83:0/2949943431 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc65806aaa0 0x7fc658198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:52:54.795 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.793+0000 7fc65d400640 1 --2- 172.21.15.83:0/2949943431 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc658107310 0x7fc658198c70 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fc648005fd0 tx=0x7fc64800dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:54.795 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.794+0000 7fc652a00640 1 -- 172.21.15.83:0/2949943431 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc648017070 con 0x7fc658107310 2024-04-24T08:52:54.795 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.794+0000 7fc652a00640 1 -- 172.21.15.83:0/2949943431 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc648009c30 con 0x7fc658107310 2024-04-24T08:52:54.796 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.794+0000 7fc652a00640 1 -- 172.21.15.83:0/2949943431 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc648003d40 con 0x7fc658107310 2024-04-24T08:52:54.796 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.794+0000 7fc660800640 1 -- 172.21.15.83:0/2949943431 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc65819dd80 con 0x7fc658107310 2024-04-24T08:52:54.796 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.794+0000 7fc660800640 1 -- 172.21.15.83:0/2949943431 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6581a5520 con 0x7fc658107310 2024-04-24T08:52:54.798 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.798+0000 7fc6635f1740 1 Processor -- start 2024-04-24T08:52:54.798 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.798+0000 7fc6635f1740 1 -- start start 2024-04-24T08:52:54.799 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.799+0000 7fc6635f1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c923d1f0d0 0x55c923d1f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:54.799 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.799+0000 7fc652a00640 1 -- 172.21.15.83:0/2949943431 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7fc648003ee0 con 0x7fc658107310 2024-04-24T08:52:54.799 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.799+0000 7fc6635f1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c923d1f9f0 0x55c923d2e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:54.799 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.799+0000 7fc6635f1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c923d2e7a0 0x55c923d30b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:54.799 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.799+0000 7fc6635f1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c923c85900 con 0x55c923d2e7a0 2024-04-24T08:52:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.799+0000 7fc6635f1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c923c2bf20 con 0x55c923d1f0d0 2024-04-24T08:52:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.799+0000 7fc6635f1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55c923c818c0 con 0x55c923d1f9f0 2024-04-24T08:52:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.799+0000 7fc65d400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c923d2e7a0 0x55c923d30b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.799+0000 7fc65d400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c923d2e7a0 0x55c923d30b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54798/0 (socket says 172.21.15.83:54798) 2024-04-24T08:52:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.799+0000 7fc65d400640 1 -- 172.21.15.83:0/2145715380 learned_addr learned my addr 172.21.15.83:0/2145715380 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:52:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.799+0000 7fc65e800640 1 --2- 172.21.15.83:0/2145715380 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c923d1f0d0 0x55c923d1f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:54.801 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.799+0000 7fc65de00640 1 --2- 172.21.15.83:0/2145715380 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c923d1f9f0 0x55c923d2e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:54.801 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.800+0000 7fc65d400640 1 -- 172.21.15.83:0/2145715380 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c923d1f9f0 msgr2=0x55c923d2e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:54.801 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.800+0000 7fc65d400640 1 --2- 172.21.15.83:0/2145715380 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c923d1f9f0 0x55c923d2e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:54.801 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.800+0000 7fc65d400640 1 -- 172.21.15.83:0/2145715380 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c923d1f0d0 msgr2=0x55c923d1f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:54.801 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.800+0000 7fc65d400640 1 --2- 172.21.15.83:0/2145715380 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c923d1f0d0 0x55c923d1f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:54.801 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.800+0000 7fc65d400640 1 -- 172.21.15.83:0/2145715380 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c923c50e10 con 0x55c923d2e7a0 2024-04-24T08:52:54.802 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.800+0000 7fc65d400640 1 --2- 172.21.15.83:0/2145715380 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c923d2e7a0 0x55c923d30b90 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fc6480cde60 tx=0x7fc6480d9690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84347c9b92f1ea9a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:54.802 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.801+0000 7fc652a00640 1 --2- 172.21.15.83:0/2949943431 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fc6400a3ee0 0x7fc6400a63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:54.802 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.801+0000 7fc647000640 1 -- 172.21.15.83:0/2145715380 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6480e0020 con 0x55c923d2e7a0 2024-04-24T08:52:54.802 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.801+0000 7fc647000640 1 -- 172.21.15.83:0/2145715380 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc6480dcaf0 con 0x55c923d2e7a0 2024-04-24T08:52:54.802 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.801+0000 7fc652a00640 1 -- 172.21.15.83:0/2949943431 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7fc6480d0b50 con 0x7fc658107310 2024-04-24T08:52:54.802 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.801+0000 7fc647000640 1 -- 172.21.15.83:0/2145715380 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6480db990 con 0x55c923d2e7a0 2024-04-24T08:52:54.802 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.801+0000 7fc65e800640 1 --2- 172.21.15.83:0/2949943431 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fc6400a3ee0 0x7fc6400a63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:54.803 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.802+0000 7fc6635f1740 1 -- 172.21.15.83:0/2145715380 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c923d2e7a0 msgr2=0x55c923d30b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:54.803 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.802+0000 7fc6635f1740 1 --2- 172.21.15.83:0/2145715380 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c923d2e7a0 0x55c923d30b90 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fc6480cde60 tx=0x7fc6480d9690 comp rx=0 tx=0).stop 2024-04-24T08:52:54.803 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.802+0000 7fc65e800640 1 --2- 172.21.15.83:0/2949943431 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fc6400a3ee0 0x7fc6400a63a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc65402ade0 tx=0x7fc654005da0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:54.803 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.802+0000 7fc6635f1740 1 -- 172.21.15.83:0/2145715380 shutdown_connections 2024-04-24T08:52:54.803 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.802+0000 7fc6635f1740 1 --2- 172.21.15.83:0/2145715380 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c923d2e7a0 0x55c923d30b90 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:54.803 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.802+0000 7fc6635f1740 1 --2- 172.21.15.83:0/2145715380 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c923d1f9f0 0x55c923d2e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:54.803 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.802+0000 7fc6635f1740 1 --2- 172.21.15.83:0/2145715380 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c923d1f0d0 0x55c923d1f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:54.803 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.802+0000 7fc6635f1740 1 -- 172.21.15.83:0/2145715380 >> 172.21.15.83:0/2145715380 conn(0x55c923d1b6c0 msgr2=0x55c923d1baa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:54.803 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.802+0000 7fc6635f1740 1 -- 172.21.15.83:0/2145715380 shutdown_connections 2024-04-24T08:52:54.803 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.803+0000 7fc6635f1740 1 -- 172.21.15.83:0/2145715380 wait complete. 2024-04-24T08:52:54.804 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.803+0000 7fc6635f1740 1 -- start start 2024-04-24T08:52:54.804 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.803+0000 7fc6635f1740 1 Processor -- start 2024-04-24T08:52:54.805 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.804+0000 7fc6635f1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c923d1f0d0 0x55c923d2b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:54.805 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.804+0000 7fc6635f1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c923d1f9f0 0x55c923d2bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:54.805 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.804+0000 7fc6635f1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c923d2e7a0 0x55c923d2c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:54.805 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.804+0000 7fc6635f1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c923d313d0 con 0x55c923d2e7a0 2024-04-24T08:52:54.805 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.804+0000 7fc6635f1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c923c3f2c0 con 0x55c923d1f0d0 2024-04-24T08:52:54.805 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.804+0000 7fc6635f1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55c923d31550 con 0x55c923d1f9f0 2024-04-24T08:52:54.805 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.804+0000 7fc6635f1740 10 client.0 _ll_register_callbacks cb 0x55c923997a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:52:54.806 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.805+0000 7fc65e800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c923d2e7a0 0x55c923d2c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:54.806 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.805+0000 7fc65d400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c923d1f9f0 0x55c923d2bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:54.806 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.805+0000 7fc65d400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c923d1f9f0 0x55c923d2bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:45762/0 (socket says 172.21.15.83:45762) 2024-04-24T08:52:54.806 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.805+0000 7fc65de00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c923d1f0d0 0x55c923d2b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:54.806 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.805+0000 7fc65d400640 1 -- 172.21.15.83:0/2637771242 learned_addr learned my addr 172.21.15.83:0/2637771242 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:52:54.806 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.805+0000 7fc65d400640 1 -- 172.21.15.83:0/2637771242 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c923d1f0d0 msgr2=0x55c923d2b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:54.806 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.805+0000 7fc65d400640 1 --2- 172.21.15.83:0/2637771242 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c923d1f0d0 0x55c923d2b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:54.806 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.805+0000 7fc65d400640 1 -- 172.21.15.83:0/2637771242 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c923d2e7a0 msgr2=0x55c923d2c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:54.806 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.805+0000 7fc65d400640 1 --2- 172.21.15.83:0/2637771242 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c923d2e7a0 0x55c923d2c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:54.807 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.805+0000 7fc65d400640 1 -- 172.21.15.83:0/2637771242 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c923c4ffd0 con 0x55c923d1f9f0 2024-04-24T08:52:54.807 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.806+0000 7fc65d400640 1 --2- 172.21.15.83:0/2637771242 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c923d1f9f0 0x55c923d2bce0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc648098ea0 tx=0x7fc6480d4370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27cd51e79243cf8b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:54.807 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.806+0000 7fc63b400640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-04-24T08:52:54.807 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.806+0000 7fc63b400640 1 -- 172.21.15.83:0/2637771242 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6480e0050 con 0x55c923d1f9f0 2024-04-24T08:52:54.807 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.806+0000 7fc6635f1740 1 -- 172.21.15.83:0/2637771242 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6480dcaf0 con 0x55c923d1f9f0 2024-04-24T08:52:54.808 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.806+0000 7fc63b400640 1 -- 172.21.15.83:0/2637771242 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7fc6480dcca0 con 0x55c923d1f9f0 2024-04-24T08:52:54.808 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.807+0000 7fc63b400640 1 -- 172.21.15.83:0/2637771242 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc648098770 con 0x55c923d1f9f0 2024-04-24T08:52:54.808 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.807+0000 7fc6635f1740 10 client.25399 fetch_fsmap learned FSMap version 28 2024-04-24T08:52:54.808 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.807+0000 7fc6635f1740 1 -- 172.21.15.83:0/2637771242 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c923998210 con 0x55c923d1f9f0 2024-04-24T08:52:54.808 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.808+0000 7fc63b400640 1 -- 172.21.15.83:0/2637771242 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc6480dcaf0 con 0x55c923d1f9f0 2024-04-24T08:52:54.809 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.808+0000 7fc63b400640 1 -- 172.21.15.83:0/2637771242 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc648098b40 con 0x55c923d1f9f0 2024-04-24T08:52:54.809 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.808+0000 7fc6635f1740 10 client.25399 fetch_fsmap finished waiting for FSMap version 28 2024-04-24T08:52:54.809 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.808+0000 7fc6635f1740 10 client.25399 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T08:52:54.809 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.808+0000 7fc6635f1740 20 client.25399 populate_metadata read hostname 'smithi083' 2024-04-24T08:52:54.809 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.808+0000 7fc6635f1740 1 --2- 172.21.15.83:0/2637771242 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55c923d3e850 0x55c923d40c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:54.809 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.808+0000 7fc6635f1740 4 client.25399 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:52:54.809 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.808+0000 7fc6635f1740 1 -- 172.21.15.83:0/2637771242 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c923c93320 con 0x55c923d3e850 2024-04-24T08:52:54.809 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.809+0000 7fc65de00640 1 --2- 172.21.15.83:0/2637771242 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55c923d3e850 0x55c923d40c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:54.810 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.810+0000 7fc63b400640 10 client.25399 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:52:54.810 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.810+0000 7fc65de00640 1 --2- 172.21.15.83:0/2637771242 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55c923d3e850 0x55c923d40c40 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ad857b2176b254b server_cookie=3d0bb706a40c3ef1 in_seq=0 out_seq=0 2024-04-24T08:52:54.812 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.811+0000 7fc63b400640 1 -- 172.21.15.83:0/2637771242 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c923c93320 con 0x55c923d3e850 2024-04-24T08:52:54.812 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.812+0000 7fc63b400640 10 client.25399 handle_command_reply: tid=0 2024-04-24T08:52:54.812 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.812+0000 7fc6635f1740 1 client.25399 shutdown 2024-04-24T08:52:54.812 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.812+0000 7fc6635f1740 1 -- 172.21.15.83:0/2637771242 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c923d1f9f0 msgr2=0x55c923d2bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:54.812 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.812+0000 7fc6635f1740 1 --2- 172.21.15.83:0/2637771242 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c923d1f9f0 0x55c923d2bce0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc648098ea0 tx=0x7fc6480d4370 comp rx=0 tx=0).stop 2024-04-24T08:52:54.813 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.812+0000 7fc6635f1740 1 -- 172.21.15.83:0/2637771242 shutdown_connections 2024-04-24T08:52:54.813 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.812+0000 7fc6635f1740 1 --2- 172.21.15.83:0/2637771242 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55c923d3e850 0x55c923d40c40 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:54.813 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.812+0000 7fc6635f1740 1 --2- 172.21.15.83:0/2637771242 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c923d2e7a0 0x55c923d2c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:54.813 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.812+0000 7fc6635f1740 1 --2- 172.21.15.83:0/2637771242 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c923d1f9f0 0x55c923d2bce0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:54.813 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.812+0000 7fc6635f1740 1 --2- 172.21.15.83:0/2637771242 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c923d1f0d0 0x55c923d2b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:54.813 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.812+0000 7fc6635f1740 1 -- 172.21.15.83:0/2637771242 >> 172.21.15.83:0/2637771242 conn(0x55c923d1b6c0 msgr2=0x55c923d21530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:54.813 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.812+0000 7fc63b400640 0 client.25399 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:52:54.813 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.812+0000 7fc6635f1740 1 -- 172.21.15.83:0/2637771242 shutdown_connections 2024-04-24T08:52:54.813 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.812+0000 7fc6635f1740 1 -- 172.21.15.83:0/2637771242 wait complete. 2024-04-24T08:52:54.813 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.813+0000 7fc6635f1740 20 client.25399 trim_cache size 0 max 16384 2024-04-24T08:52:54.828 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.828+0000 7fc6635f1740 1 Processor -- start 2024-04-24T08:52:54.828 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.828+0000 7fc6635f1740 1 -- start start 2024-04-24T08:52:54.829 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.828+0000 7fc6635f1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c923d1f0d0 0x55c923d39eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:54.829 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.828+0000 7fc6635f1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c923d1f9f0 0x55c923d43c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:54.829 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.829+0000 7fc6635f1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c923d2e7a0 0x55c923d441a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:54.829 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.829+0000 7fc6635f1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc6581ad250 con 0x55c923d1f0d0 2024-04-24T08:52:54.829 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.829+0000 7fc6635f1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc6581aed90 con 0x55c923d2e7a0 2024-04-24T08:52:54.829 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.829+0000 7fc6635f1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc6581ad920 con 0x55c923d1f9f0 2024-04-24T08:52:54.829 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.829+0000 7fc65de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c923d1f0d0 0x55c923d39eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:54.829 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.829+0000 7fc65de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c923d1f0d0 0x55c923d39eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54826/0 (socket says 172.21.15.83:54826) 2024-04-24T08:52:54.829 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.829+0000 7fc65de00640 1 -- 172.21.15.83:0/3981699111 learned_addr learned my addr 172.21.15.83:0/3981699111 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:52:54.830 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.829+0000 7fc65de00640 1 -- 172.21.15.83:0/3981699111 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c923d1f9f0 msgr2=0x55c923d43c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:54.830 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.829+0000 7fc65de00640 1 --2- 172.21.15.83:0/3981699111 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c923d1f9f0 0x55c923d43c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:54.831 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.829+0000 7fc65de00640 1 -- 172.21.15.83:0/3981699111 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c923d2e7a0 msgr2=0x55c923d441a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:54.831 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.829+0000 7fc65de00640 1 --2- 172.21.15.83:0/3981699111 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c923d2e7a0 0x55c923d441a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:54.831 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.829+0000 7fc65de00640 1 -- 172.21.15.83:0/3981699111 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c923d1f820 con 0x55c923d1f0d0 2024-04-24T08:52:54.831 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.830+0000 7fc65de00640 1 --2- 172.21.15.83:0/3981699111 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c923d1f0d0 0x55c923d39eb0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x55c923d31610 tx=0x7fc64c014840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=975445f25e29b65e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:54.831 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.830+0000 7fc646600640 1 -- 172.21.15.83:0/3981699111 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc64c005020 con 0x55c923d1f0d0 2024-04-24T08:52:54.831 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.830+0000 7fc646600640 1 -- 172.21.15.83:0/3981699111 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc64c0047a0 con 0x55c923d1f0d0 2024-04-24T08:52:54.831 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.830+0000 7fc646600640 1 -- 172.21.15.83:0/3981699111 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc64c01e3f0 con 0x55c923d1f0d0 2024-04-24T08:52:54.832 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.831+0000 7fc6635f1740 1 -- 172.21.15.83:0/3981699111 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c923d1f0d0 msgr2=0x55c923d39eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:54.832 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.831+0000 7fc6635f1740 1 --2- 172.21.15.83:0/3981699111 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c923d1f0d0 0x55c923d39eb0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x55c923d31610 tx=0x7fc64c014840 comp rx=0 tx=0).stop 2024-04-24T08:52:54.832 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.831+0000 7fc6635f1740 1 -- 172.21.15.83:0/3981699111 shutdown_connections 2024-04-24T08:52:54.832 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.831+0000 7fc6635f1740 1 --2- 172.21.15.83:0/3981699111 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c923d2e7a0 0x55c923d441a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:54.832 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.831+0000 7fc6635f1740 1 --2- 172.21.15.83:0/3981699111 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c923d1f9f0 0x55c923d43c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:54.832 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.831+0000 7fc6635f1740 1 --2- 172.21.15.83:0/3981699111 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c923d1f0d0 0x55c923d39eb0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:54.832 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.831+0000 7fc6635f1740 1 -- 172.21.15.83:0/3981699111 >> 172.21.15.83:0/3981699111 conn(0x55c923d1b6c0 msgr2=0x55c923d397a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:54.832 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.831+0000 7fc6635f1740 1 -- 172.21.15.83:0/3981699111 shutdown_connections 2024-04-24T08:52:54.833 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.831+0000 7fc6635f1740 1 -- 172.21.15.83:0/3981699111 wait complete. 2024-04-24T08:52:54.833 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.832+0000 7fc6635f1740 1 -- start start 2024-04-24T08:52:54.833 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.832+0000 7fc6635f1740 1 Processor -- start 2024-04-24T08:52:54.834 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.833+0000 7fc6635f1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c923d1f0d0 0x55c923d415c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:54.835 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.833+0000 7fc6635f1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c923d1f9f0 0x55c923d41b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:54.835 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.833+0000 7fc6635f1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c923d2e7a0 0x55c923d42040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:54.835 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.833+0000 7fc6635f1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc6581ae6c0 con 0x55c923d1f0d0 2024-04-24T08:52:54.835 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.833+0000 7fc6635f1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc6581adff0 con 0x55c923d1f9f0 2024-04-24T08:52:54.835 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.833+0000 7fc6635f1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc6581ab230 con 0x55c923d2e7a0 2024-04-24T08:52:54.835 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.833+0000 7fc6635f1740 10 client.0 _ll_register_callbacks cb 0x7fc64c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:52:54.835 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.833+0000 7fc65de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c923d1f0d0 0x55c923d415c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:54.835 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.833+0000 7fc65de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c923d1f0d0 0x55c923d415c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54840/0 (socket says 172.21.15.83:54840) 2024-04-24T08:52:54.835 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.833+0000 7fc65de00640 1 -- 172.21.15.83:0/1253203827 learned_addr learned my addr 172.21.15.83:0/1253203827 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:52:54.835 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.833+0000 7fc65d400640 1 --2- 172.21.15.83:0/1253203827 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c923d1f9f0 0x55c923d41b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:54.835 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.833+0000 7fc65e800640 1 --2- 172.21.15.83:0/1253203827 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c923d2e7a0 0x55c923d42040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:54.835 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.833+0000 7fc65de00640 1 -- 172.21.15.83:0/1253203827 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c923d2e7a0 msgr2=0x55c923d42040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:54.836 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.833+0000 7fc65de00640 1 --2- 172.21.15.83:0/1253203827 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c923d2e7a0 0x55c923d42040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:54.836 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.833+0000 7fc65de00640 1 -- 172.21.15.83:0/1253203827 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c923d1f9f0 msgr2=0x55c923d41b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:52:54.836 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.833+0000 7fc65de00640 1 --2- 172.21.15.83:0/1253203827 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c923d1f9f0 0x55c923d41b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:54.836 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.833+0000 7fc65de00640 1 -- 172.21.15.83:0/1253203827 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c923a64620 con 0x55c923d1f0d0 2024-04-24T08:52:54.836 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.834+0000 7fc65de00640 1 --2- 172.21.15.83:0/1253203827 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c923d1f0d0 0x55c923d415c0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fc64c014410 tx=0x7fc64c004950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7b2518b3949316 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:52:54.836 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.834+0000 7fc63b400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:52:54.836 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.834+0000 7fc63b400640 1 -- 172.21.15.83:0/1253203827 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc64c02d070 con 0x55c923d1f0d0 2024-04-24T08:52:54.836 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.834+0000 7fc63b400640 1 -- 172.21.15.83:0/1253203827 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7fc64c005200 con 0x55c923d1f0d0 2024-04-24T08:52:54.836 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.834+0000 7fc6635f1740 1 -- 172.21.15.83:0/1253203827 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc64c0047a0 con 0x55c923d1f0d0 2024-04-24T08:52:54.836 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.834+0000 7fc63b400640 1 -- 172.21.15.83:0/1253203827 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc64c039050 con 0x55c923d1f0d0 2024-04-24T08:52:54.837 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.835+0000 7fc6635f1740 10 client.16233 fetch_fsmap learned FSMap version 28 2024-04-24T08:52:54.837 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.835+0000 7fc6635f1740 1 -- 172.21.15.83:0/1253203827 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c923da4ad0 con 0x55c923d1f0d0 2024-04-24T08:52:54.837 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.835+0000 7fc63b400640 1 -- 172.21.15.83:0/1253203827 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc64c0047a0 con 0x55c923d1f0d0 2024-04-24T08:52:54.837 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.835+0000 7fc63b400640 1 -- 172.21.15.83:0/1253203827 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc64c0124e0 con 0x55c923d1f0d0 2024-04-24T08:52:54.837 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.835+0000 7fc6635f1740 10 client.16233 fetch_fsmap finished waiting for FSMap version 28 2024-04-24T08:52:54.837 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.835+0000 7fc6635f1740 10 client.16233 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T08:52:54.838 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.835+0000 7fc6635f1740 20 client.16233 populate_metadata read hostname 'smithi083' 2024-04-24T08:52:54.838 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.835+0000 7fc6635f1740 1 --2- 172.21.15.83:0/1253203827 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55c923da5170 0x55c923da7590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:52:54.838 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.835+0000 7fc6635f1740 4 client.16233 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T08:52:54.838 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.835+0000 7fc6635f1740 1 -- 172.21.15.83:0/1253203827 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c923c8d190 con 0x55c923da5170 2024-04-24T08:52:54.838 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.836+0000 7fc65d400640 1 --2- 172.21.15.83:0/1253203827 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55c923da5170 0x55c923da7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:52:54.838 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.837+0000 7fc63b400640 10 client.16233 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:52:54.838 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.837+0000 7fc65d400640 1 --2- 172.21.15.83:0/1253203827 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55c923da5170 0x55c923da7590 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b593eccf2ef5b8d server_cookie=e2bfe8363f507c55 in_seq=0 out_seq=0 2024-04-24T08:52:54.847 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.846+0000 7fc63b400640 1 -- 172.21.15.83:0/1253203827 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c923c8d190 con 0x55c923da5170 2024-04-24T08:52:54.847 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.846+0000 7fc63b400640 10 client.16233 handle_command_reply: tid=0 2024-04-24T08:52:54.847 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.846+0000 7fc6635f1740 1 client.16233 shutdown 2024-04-24T08:52:54.847 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.846+0000 7fc6635f1740 1 -- 172.21.15.83:0/1253203827 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c923d1f0d0 msgr2=0x55c923d415c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:54.847 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.846+0000 7fc6635f1740 1 --2- 172.21.15.83:0/1253203827 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c923d1f0d0 0x55c923d415c0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fc64c014410 tx=0x7fc64c004950 comp rx=0 tx=0).stop 2024-04-24T08:52:54.847 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.846+0000 7fc6635f1740 1 -- 172.21.15.83:0/1253203827 shutdown_connections 2024-04-24T08:52:54.847 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.846+0000 7fc6635f1740 1 --2- 172.21.15.83:0/1253203827 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55c923da5170 0x55c923da7590 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:54.847 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.846+0000 7fc6635f1740 1 --2- 172.21.15.83:0/1253203827 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c923d2e7a0 0x55c923d42040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:54.848 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.846+0000 7fc6635f1740 1 --2- 172.21.15.83:0/1253203827 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c923d1f9f0 0x55c923d41b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:54.848 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.846+0000 7fc6635f1740 1 --2- 172.21.15.83:0/1253203827 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c923d1f0d0 0x55c923d415c0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:54.848 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.846+0000 7fc63b400640 0 client.16233 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:52:54.848 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.846+0000 7fc6635f1740 1 -- 172.21.15.83:0/1253203827 >> 172.21.15.83:0/1253203827 conn(0x55c923d1b6c0 msgr2=0x55c923d21420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:54.848 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.847+0000 7fc6635f1740 1 -- 172.21.15.83:0/1253203827 shutdown_connections 2024-04-24T08:52:54.848 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.847+0000 7fc6635f1740 1 -- 172.21.15.83:0/1253203827 wait complete. 2024-04-24T08:52:54.848 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.847+0000 7fc6635f1740 20 client.16233 trim_cache size 0 max 16384 2024-04-24T08:52:54.848 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T08:52:54.848 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T08:52:54.848 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T08:52:54.848 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T08:52:54.849 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.848+0000 7fc660800640 1 -- 172.21.15.83:0/2949943431 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fc6400a3ee0 msgr2=0x7fc6400a63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:54.849 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.848+0000 7fc660800640 1 --2- 172.21.15.83:0/2949943431 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fc6400a3ee0 0x7fc6400a63a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc65402ade0 tx=0x7fc654005da0 comp rx=0 tx=0).stop 2024-04-24T08:52:54.849 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.848+0000 7fc660800640 1 -- 172.21.15.83:0/2949943431 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc658107310 msgr2=0x7fc658198c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:52:54.849 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.848+0000 7fc660800640 1 --2- 172.21.15.83:0/2949943431 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc658107310 0x7fc658198c70 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fc648005fd0 tx=0x7fc64800dde0 comp rx=0 tx=0).stop 2024-04-24T08:52:54.849 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.848+0000 7fc660800640 1 -- 172.21.15.83:0/2949943431 shutdown_connections 2024-04-24T08:52:54.849 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.848+0000 7fc660800640 1 --2- 172.21.15.83:0/2949943431 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fc6400a3ee0 0x7fc6400a63a0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:54.849 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.848+0000 7fc660800640 1 --2- 172.21.15.83:0/2949943431 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc65819cfa0 0x7fc65819d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:54.849 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.848+0000 7fc660800640 1 --2- 172.21.15.83:0/2949943431 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc658107310 0x7fc658198c70 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:54.850 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.848+0000 7fc660800640 1 --2- 172.21.15.83:0/2949943431 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc65806aaa0 0x7fc658198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:52:54.850 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.848+0000 7fc660800640 1 -- 172.21.15.83:0/2949943431 >> 172.21.15.83:0/2949943431 conn(0x7fc658076c30 msgr2=0x7fc6580ff180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:52:54.850 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.848+0000 7fc660800640 1 -- 172.21.15.83:0/2949943431 shutdown_connections 2024-04-24T08:52:54.850 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:52:54.848+0000 7fc660800640 1 -- 172.21.15.83:0/2949943431 wait complete. 2024-04-24T08:52:54.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:52:54 smithi138 ceph-mon[31441]: scrub queued for path: / 2024-04-24T08:52:54.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:52:54 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T08:52:54.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:52:54 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T08:52:54.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:52:54 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T08:52:54.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:52:54 smithi189 ceph-mon[31791]: scrub queued for path: / 2024-04-24T08:52:54.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:52:54 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T08:52:54.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:52:54 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T08:52:54.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:52:54 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T08:52:55.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:52:54 smithi083 ceph-mon[27907]: scrub queued for path: / 2024-04-24T08:52:55.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:52:54 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T08:52:55.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:52:54 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T08:52:55.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:52:54 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T08:52:55.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:52:55 smithi138 ceph-mon[31441]: pgmap v42: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 10 op/s 2024-04-24T08:52:55.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:52:55 smithi189 ceph-mon[31791]: pgmap v42: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 10 op/s 2024-04-24T08:52:56.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:52:55 smithi083 ceph-mon[27907]: pgmap v42: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 10 op/s 2024-04-24T08:52:57.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:52:57 smithi189 ceph-mon[31791]: pgmap v43: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-24T08:52:57.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:52:57 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T08:52:58.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:52:57 smithi083 ceph-mon[27907]: pgmap v43: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-24T08:52:58.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:52:57 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T08:52:58.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:52:57 smithi138 ceph-mon[31441]: pgmap v43: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-24T08:52:58.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:52:57 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T08:53:00.052 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:52:59 smithi083 ceph-mon[27907]: pgmap v44: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-24T08:53:00.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:52:59 smithi138 ceph-mon[31441]: pgmap v44: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-24T08:53:00.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:52:59 smithi189 ceph-mon[31791]: pgmap v44: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-24T08:53:02.052 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:53:01 smithi083 ceph-mon[27907]: pgmap v45: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-24T08:53:02.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:53:01 smithi138 ceph-mon[31441]: pgmap v45: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-24T08:53:02.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:53:01 smithi189 ceph-mon[31791]: pgmap v45: 129 pgs: 129 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-24T08:53:04.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:53:03 smithi083 ceph-mon[27907]: pgmap v46: 129 pgs: 129 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 26 op/s 2024-04-24T08:53:04.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:53:03 smithi138 ceph-mon[31441]: pgmap v46: 129 pgs: 129 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 26 op/s 2024-04-24T08:53:04.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:53:03 smithi189 ceph-mon[31791]: pgmap v46: 129 pgs: 129 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 26 op/s 2024-04-24T08:53:06.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:53:05 smithi083 ceph-mon[27907]: pgmap v47: 129 pgs: 129 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 26 op/s 2024-04-24T08:53:06.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:53:05 smithi138 ceph-mon[31441]: pgmap v47: 129 pgs: 129 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 26 op/s 2024-04-24T08:53:06.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:53:05 smithi189 ceph-mon[31791]: pgmap v47: 129 pgs: 129 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 26 op/s 2024-04-24T08:53:08.007 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:53:07 smithi138 ceph-mon[31441]: pgmap v48: 129 pgs: 129 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-04-24T08:53:08.037 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.037+0000 7f59eda00640 1 -- 172.21.15.83:0/3874008111 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59e80743d0 msgr2=0x7f59e80747b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:08.037 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.037+0000 7f59eda00640 1 --2- 172.21.15.83:0/3874008111 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59e80743d0 0x7f59e80747b0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f59d8009900 tx=0x7f59d802a480 comp rx=0 tx=0).stop 2024-04-24T08:53:08.037 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.037+0000 7f59eda00640 1 -- 172.21.15.83:0/3874008111 shutdown_connections 2024-04-24T08:53:08.037 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.037+0000 7f59eda00640 1 --2- 172.21.15.83:0/3874008111 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59e810e6c0 0x7f59e8110ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:08.037 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.037+0000 7f59eda00640 1 --2- 172.21.15.83:0/3874008111 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f59e8074cf0 0x7f59e810e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:08.037 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.037+0000 7f59eda00640 1 --2- 172.21.15.83:0/3874008111 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59e80743d0 0x7f59e80747b0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:08.037 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.037+0000 7f59eda00640 1 -- 172.21.15.83:0/3874008111 >> 172.21.15.83:0/3874008111 conn(0x7f59e80fd950 msgr2=0x7f59e80ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:08.038 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.037+0000 7f59eda00640 1 -- 172.21.15.83:0/3874008111 shutdown_connections 2024-04-24T08:53:08.038 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.037+0000 7f59eda00640 1 -- 172.21.15.83:0/3874008111 wait complete. 2024-04-24T08:53:08.039 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.039+0000 7f59eda00640 1 Processor -- start 2024-04-24T08:53:08.039 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.039+0000 7f59eda00640 1 -- start start 2024-04-24T08:53:08.040 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.039+0000 7f59eda00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59e80743d0 0x7f59e81a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:08.040 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.039+0000 7f59eda00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f59e8074cf0 0x7f59e81a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:08.040 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.039+0000 7f59eda00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59e810e6c0 0x7f59e819a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:08.040 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.039+0000 7f59eda00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f59e8114350 con 0x7f59e810e6c0 2024-04-24T08:53:08.040 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.039+0000 7f59eda00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f59e81141d0 con 0x7f59e80743d0 2024-04-24T08:53:08.040 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.039+0000 7f59eda00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f59e81144d0 con 0x7f59e8074cf0 2024-04-24T08:53:08.040 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.040+0000 7f59ed000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59e810e6c0 0x7f59e819a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:08.040 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.040+0000 7f59e7e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59e80743d0 0x7f59e81a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:08.041 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.040+0000 7f59e7e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59e80743d0 0x7f59e81a0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:40086/0 (socket says 172.21.15.83:40086) 2024-04-24T08:53:08.041 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.040+0000 7f59e7e00640 1 -- 172.21.15.83:0/783176793 learned_addr learned my addr 172.21.15.83:0/783176793 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:53:08.041 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.040+0000 7f59e7400640 1 --2- 172.21.15.83:0/783176793 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f59e8074cf0 0x7f59e81a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:08.042 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.041+0000 7f59ed000640 1 -- 172.21.15.83:0/783176793 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f59e8074cf0 msgr2=0x7f59e81a0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:08.042 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.041+0000 7f59ed000640 1 --2- 172.21.15.83:0/783176793 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f59e8074cf0 0x7f59e81a0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:08.042 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.041+0000 7f59ed000640 1 -- 172.21.15.83:0/783176793 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59e80743d0 msgr2=0x7f59e81a0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:08.042 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.041+0000 7f59ed000640 1 --2- 172.21.15.83:0/783176793 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59e80743d0 0x7f59e81a0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:08.042 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.041+0000 7f59ed000640 1 -- 172.21.15.83:0/783176793 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59e819b030 con 0x7f59e810e6c0 2024-04-24T08:53:08.043 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.041+0000 7f59ed000640 1 --2- 172.21.15.83:0/783176793 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59e810e6c0 0x7f59e819a770 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f59e8075320 tx=0x7f59e000edd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:08.043 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.041+0000 7f59e4c00640 1 -- 172.21.15.83:0/783176793 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59e0004d20 con 0x7f59e810e6c0 2024-04-24T08:53:08.043 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.041+0000 7f59e4c00640 1 -- 172.21.15.83:0/783176793 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f59e0005020 con 0x7f59e810e6c0 2024-04-24T08:53:08.043 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.042+0000 7f59e4c00640 1 -- 172.21.15.83:0/783176793 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59e00116b0 con 0x7f59e810e6c0 2024-04-24T08:53:08.043 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.042+0000 7f59eda00640 1 -- 172.21.15.83:0/783176793 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59e819b320 con 0x7f59e810e6c0 2024-04-24T08:53:08.043 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.042+0000 7f59eda00640 1 -- 172.21.15.83:0/783176793 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59e81a75e0 con 0x7f59e810e6c0 2024-04-24T08:53:08.046 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.047+0000 7f59f1d76740 1 Processor -- start 2024-04-24T08:53:08.047 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.048+0000 7f59e4c00640 1 -- 172.21.15.83:0/783176793 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7f59e000caf0 con 0x7f59e810e6c0 2024-04-24T08:53:08.047 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.048+0000 7f59f1d76740 1 -- start start 2024-04-24T08:53:08.047 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.048+0000 7f59f1d76740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563acd16a3c0 0x563acd16a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:08.048 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.048+0000 7f59f1d76740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563acd16ace0 0x563acd175060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:08.048 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.048+0000 7f59f1d76740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563acd1755a0 0x563acd177990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:08.048 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.048+0000 7f59f1d76740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x563acd0d0900 con 0x563acd16a3c0 2024-04-24T08:53:08.048 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.048+0000 7f59f1d76740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563acd076f20 con 0x563acd1755a0 2024-04-24T08:53:08.048 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.048+0000 7f59f1d76740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x563acd0cc8c0 con 0x563acd16ace0 2024-04-24T08:53:08.048 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.049+0000 7f59e7400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563acd16a3c0 0x563acd16a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:08.049 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.049+0000 7f59e7400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563acd16a3c0 0x563acd16a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38906/0 (socket says 172.21.15.83:38906) 2024-04-24T08:53:08.049 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.049+0000 7f59e7400640 1 -- 172.21.15.83:0/164570632 learned_addr learned my addr 172.21.15.83:0/164570632 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:53:08.050 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.049+0000 7f59e7e00640 1 --2- 172.21.15.83:0/164570632 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563acd16ace0 0x563acd175060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:08.050 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.049+0000 7f59e7400640 1 --2- 172.21.15.83:0/164570632 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563acd1755a0 0x563acd177990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:08.050 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.050+0000 7f59e7400640 1 -- 172.21.15.83:0/164570632 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563acd16ace0 msgr2=0x563acd175060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:08.050 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.050+0000 7f59e7400640 1 --2- 172.21.15.83:0/164570632 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563acd16ace0 0x563acd175060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:08.050 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.050+0000 7f59e7400640 1 -- 172.21.15.83:0/164570632 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563acd1755a0 msgr2=0x563acd177990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:08.051 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.050+0000 7f59e7400640 1 --2- 172.21.15.83:0/164570632 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563acd1755a0 0x563acd177990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:08.051 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.050+0000 7f59e7400640 1 -- 172.21.15.83:0/164570632 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563acd09be10 con 0x563acd16a3c0 2024-04-24T08:53:08.051 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.050+0000 7f59e4c00640 1 --2- 172.21.15.83:0/783176793 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f59cc0a3eb0 0x7f59cc0a6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:08.051 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.050+0000 7f59e7400640 1 --2- 172.21.15.83:0/164570632 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563acd16a3c0 0x563acd16a7a0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f59d400bfd0 tx=0x7f59d400b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afaea47626a6ad81 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:08.051 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.050+0000 7f59e4c00640 1 -- 172.21.15.83:0/783176793 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f59e00d0df0 con 0x7f59e810e6c0 2024-04-24T08:53:08.051 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.050+0000 7f59dd600640 1 -- 172.21.15.83:0/164570632 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d4005220 con 0x563acd16a3c0 2024-04-24T08:53:08.051 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.051+0000 7f59e7e00640 1 --2- 172.21.15.83:0/783176793 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f59cc0a3eb0 0x7f59cc0a6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:08.051 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.051+0000 7f59dd600640 1 -- 172.21.15.83:0/164570632 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f59d4004c50 con 0x563acd16a3c0 2024-04-24T08:53:08.051 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.051+0000 7f59dd600640 1 -- 172.21.15.83:0/164570632 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d40107b0 con 0x563acd16a3c0 2024-04-24T08:53:08.052 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.052+0000 7f59f1d76740 1 -- 172.21.15.83:0/164570632 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563acd16a3c0 msgr2=0x563acd16a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:08.053 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.052+0000 7f59e7e00640 1 --2- 172.21.15.83:0/783176793 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f59cc0a3eb0 0x7f59cc0a6370 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x563acd16b960 tx=0x7f59d8005a20 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:08.053 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.052+0000 7f59f1d76740 1 --2- 172.21.15.83:0/164570632 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563acd16a3c0 0x563acd16a7a0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f59d400bfd0 tx=0x7f59d400b820 comp rx=0 tx=0).stop 2024-04-24T08:53:08.053 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.052+0000 7f59f1d76740 1 -- 172.21.15.83:0/164570632 shutdown_connections 2024-04-24T08:53:08.053 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.052+0000 7f59f1d76740 1 --2- 172.21.15.83:0/164570632 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563acd1755a0 0x563acd177990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:08.053 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.052+0000 7f59f1d76740 1 --2- 172.21.15.83:0/164570632 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563acd16ace0 0x563acd175060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:08.053 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.052+0000 7f59f1d76740 1 --2- 172.21.15.83:0/164570632 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563acd16a3c0 0x563acd16a7a0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:08.053 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.052+0000 7f59f1d76740 1 -- 172.21.15.83:0/164570632 >> 172.21.15.83:0/164570632 conn(0x563acd1668d0 msgr2=0x563acd166cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:08.053 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.052+0000 7f59f1d76740 1 -- 172.21.15.83:0/164570632 shutdown_connections 2024-04-24T08:53:08.053 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.052+0000 7f59f1d76740 1 -- 172.21.15.83:0/164570632 wait complete. 2024-04-24T08:53:08.053 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.053+0000 7f59f1d76740 1 -- start start 2024-04-24T08:53:08.053 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.053+0000 7f59f1d76740 1 Processor -- start 2024-04-24T08:53:08.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:53:07 smithi083 ceph-mon[27907]: pgmap v48: 129 pgs: 129 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-04-24T08:53:08.056 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.054+0000 7f59f1d76740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563acd16a3c0 0x563acd172570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:08.056 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.054+0000 7f59f1d76740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563acd16ace0 0x563acd172ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:08.056 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.054+0000 7f59f1d76740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563acd1755a0 0x563acd172ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:08.056 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.054+0000 7f59f1d76740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x563acd1780e0 con 0x563acd16ace0 2024-04-24T08:53:08.056 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.054+0000 7f59f1d76740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563acd08a2c0 con 0x563acd16a3c0 2024-04-24T08:53:08.057 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.054+0000 7f59f1d76740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x563acd178260 con 0x563acd1755a0 2024-04-24T08:53:08.057 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.054+0000 7f59f1d76740 10 client.0 _ll_register_callbacks cb 0x563accde2a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:53:08.057 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.055+0000 7f59e7400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563acd16ace0 0x563acd172ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:08.057 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.055+0000 7f59e7e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563acd16a3c0 0x563acd172570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:08.057 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.055+0000 7f59e7400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563acd16ace0 0x563acd172ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38916/0 (socket says 172.21.15.83:38916) 2024-04-24T08:53:08.057 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.055+0000 7f59e7400640 1 -- 172.21.15.83:0/276007920 learned_addr learned my addr 172.21.15.83:0/276007920 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:53:08.057 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.055+0000 7f59ed000640 1 --2- 172.21.15.83:0/276007920 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563acd1755a0 0x563acd172ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:08.057 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.055+0000 7f59e7400640 1 -- 172.21.15.83:0/276007920 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563acd1755a0 msgr2=0x563acd172ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:08.057 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.055+0000 7f59e7400640 1 --2- 172.21.15.83:0/276007920 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563acd1755a0 0x563acd172ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:08.057 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.055+0000 7f59e7400640 1 -- 172.21.15.83:0/276007920 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563acd16a3c0 msgr2=0x563acd172570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:08.057 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.055+0000 7f59e7400640 1 --2- 172.21.15.83:0/276007920 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563acd16a3c0 0x563acd172570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:08.057 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.055+0000 7f59e7400640 1 -- 172.21.15.83:0/276007920 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563acd09afd0 con 0x563acd16ace0 2024-04-24T08:53:08.057 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.056+0000 7f59e7400640 1 --2- 172.21.15.83:0/276007920 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563acd16ace0 0x563acd172ab0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f59d400be30 tx=0x7f59d400be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51d3ffc023288808 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:08.058 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.056+0000 7f59d2000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:53:08.058 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.056+0000 7f59d2000640 1 -- 172.21.15.83:0/276007920 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d402b020 con 0x563acd16ace0 2024-04-24T08:53:08.058 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.056+0000 7f59f1d76740 1 -- 172.21.15.83:0/276007920 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59d4004c50 con 0x563acd16ace0 2024-04-24T08:53:08.058 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.056+0000 7f59d2000640 1 -- 172.21.15.83:0/276007920 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f59d40048d0 con 0x563acd16ace0 2024-04-24T08:53:08.058 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.057+0000 7f59d2000640 1 -- 172.21.15.83:0/276007920 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f59d4036050 con 0x563acd16ace0 2024-04-24T08:53:08.058 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.057+0000 7f59f1d76740 10 client.16245 fetch_fsmap learned FSMap version 28 2024-04-24T08:53:08.058 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.057+0000 7f59f1d76740 1 -- 172.21.15.83:0/276007920 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563accde3210 con 0x563acd16ace0 2024-04-24T08:53:08.058 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.057+0000 7f59d2000640 1 -- 172.21.15.83:0/276007920 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f59d4004c50 con 0x563acd16ace0 2024-04-24T08:53:08.059 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.058+0000 7f59d2000640 1 -- 172.21.15.83:0/276007920 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f59d401c070 con 0x563acd16ace0 2024-04-24T08:53:08.059 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.058+0000 7f59f1d76740 10 client.16245 fetch_fsmap finished waiting for FSMap version 28 2024-04-24T08:53:08.059 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.058+0000 7f59f1d76740 10 client.16245 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T08:53:08.059 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.058+0000 7f59f1d76740 20 client.16245 populate_metadata read hostname 'smithi083' 2024-04-24T08:53:08.060 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.058+0000 7f59f1d76740 1 --2- 172.21.15.83:0/276007920 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x563acd18de90 0x563acd190280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:08.060 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.058+0000 7f59f1d76740 4 client.16245 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:53:08.060 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.058+0000 7f59f1d76740 1 -- 172.21.15.83:0/276007920 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563acd0de320 con 0x563acd18de90 2024-04-24T08:53:08.060 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.059+0000 7f59e7e00640 1 --2- 172.21.15.83:0/276007920 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x563acd18de90 0x563acd190280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:08.060 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.060+0000 7f59e7e00640 1 --2- 172.21.15.83:0/276007920 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x563acd18de90 0x563acd190280 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c93f5d2a1900cb0f server_cookie=e291a3268499972b in_seq=0 out_seq=0 2024-04-24T08:53:08.060 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.060+0000 7f59d2000640 10 client.16245 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:53:08.061 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.062+0000 7f59d2000640 1 -- 172.21.15.83:0/276007920 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563acd0de320 con 0x563acd18de90 2024-04-24T08:53:08.061 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.062+0000 7f59d2000640 10 client.16245 handle_command_reply: tid=0 2024-04-24T08:53:08.061 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.062+0000 7f59f1d76740 1 client.16245 shutdown 2024-04-24T08:53:08.062 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.062+0000 7f59f1d76740 1 -- 172.21.15.83:0/276007920 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563acd16ace0 msgr2=0x563acd172ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:08.062 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.062+0000 7f59f1d76740 1 --2- 172.21.15.83:0/276007920 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563acd16ace0 0x563acd172ab0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f59d400be30 tx=0x7f59d400be60 comp rx=0 tx=0).stop 2024-04-24T08:53:08.062 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.062+0000 7f59f1d76740 1 -- 172.21.15.83:0/276007920 shutdown_connections 2024-04-24T08:53:08.062 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.062+0000 7f59f1d76740 1 --2- 172.21.15.83:0/276007920 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x563acd18de90 0x563acd190280 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:08.062 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.062+0000 7f59f1d76740 1 --2- 172.21.15.83:0/276007920 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563acd1755a0 0x563acd172ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:08.062 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.062+0000 7f59f1d76740 1 --2- 172.21.15.83:0/276007920 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563acd16ace0 0x563acd172ab0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:08.063 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.062+0000 7f59d2000640 0 client.16245 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:53:08.063 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.062+0000 7f59f1d76740 1 --2- 172.21.15.83:0/276007920 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563acd16a3c0 0x563acd172570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:08.063 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.062+0000 7f59f1d76740 1 -- 172.21.15.83:0/276007920 >> 172.21.15.83:0/276007920 conn(0x563acd1668d0 msgr2=0x563acd16c600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:08.063 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.062+0000 7f59f1d76740 1 -- 172.21.15.83:0/276007920 shutdown_connections 2024-04-24T08:53:08.063 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.062+0000 7f59f1d76740 1 -- 172.21.15.83:0/276007920 wait complete. 2024-04-24T08:53:08.063 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.063+0000 7f59f1d76740 20 client.16245 trim_cache size 0 max 16384 2024-04-24T08:53:08.078 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.079+0000 7f59f1d76740 1 Processor -- start 2024-04-24T08:53:08.078 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.079+0000 7f59f1d76740 1 -- start start 2024-04-24T08:53:08.078 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.079+0000 7f59f1d76740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563acd16a3c0 0x563acd174ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:08.078 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.079+0000 7f59f1d76740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563acd16ace0 0x563acd166210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:08.078 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.079+0000 7f59f1d76740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563acd1755a0 0x563acd16c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:08.078 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.079+0000 7f59f1d76740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f59e81ad210 con 0x563acd16a3c0 2024-04-24T08:53:08.079 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.079+0000 7f59f1d76740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f59e81aed50 con 0x563acd16ace0 2024-04-24T08:53:08.079 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.079+0000 7f59f1d76740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f59e81ad8e0 con 0x563acd1755a0 2024-04-24T08:53:08.080 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.079+0000 7f59e7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563acd16a3c0 0x563acd174ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:08.080 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.080+0000 7f59e7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563acd16a3c0 0x563acd174ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38918/0 (socket says 172.21.15.83:38918) 2024-04-24T08:53:08.080 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.080+0000 7f59e7e00640 1 -- 172.21.15.83:0/3739856614 learned_addr learned my addr 172.21.15.83:0/3739856614 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:53:08.080 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.080+0000 7f59ed000640 1 --2- 172.21.15.83:0/3739856614 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563acd1755a0 0x563acd16c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:08.080 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.080+0000 7f59e7400640 1 --2- 172.21.15.83:0/3739856614 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563acd16ace0 0x563acd166210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:08.080 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.080+0000 7f59e7e00640 1 -- 172.21.15.83:0/3739856614 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563acd1755a0 msgr2=0x563acd16c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:08.080 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.080+0000 7f59e7e00640 1 --2- 172.21.15.83:0/3739856614 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563acd1755a0 0x563acd16c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:08.080 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.080+0000 7f59e7e00640 1 -- 172.21.15.83:0/3739856614 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563acd16ace0 msgr2=0x563acd166210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:08.080 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.080+0000 7f59e7e00640 1 --2- 172.21.15.83:0/3739856614 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563acd16ace0 0x563acd166210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:08.081 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.080+0000 7f59e7e00640 1 -- 172.21.15.83:0/3739856614 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563acd177d00 con 0x563acd16a3c0 2024-04-24T08:53:08.081 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.080+0000 7f59e7400640 1 --2- 172.21.15.83:0/3739856614 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563acd16ace0 0x563acd166210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:53:08.081 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.080+0000 7f59e7e00640 1 --2- 172.21.15.83:0/3739856614 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563acd16a3c0 0x563acd174ca0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f59d802aa70 tx=0x7f59d8040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4cec94a54cab9d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:08.081 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.081+0000 7f59dcc00640 1 -- 172.21.15.83:0/3739856614 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d805a070 con 0x563acd16a3c0 2024-04-24T08:53:08.081 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.081+0000 7f59dcc00640 1 -- 172.21.15.83:0/3739856614 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f59d8054780 con 0x563acd16a3c0 2024-04-24T08:53:08.081 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.081+0000 7f59dcc00640 1 -- 172.21.15.83:0/3739856614 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d8035040 con 0x563acd16a3c0 2024-04-24T08:53:08.084 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.081+0000 7f59f1d76740 1 -- 172.21.15.83:0/3739856614 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563acd16a3c0 msgr2=0x563acd174ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:08.084 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.081+0000 7f59f1d76740 1 --2- 172.21.15.83:0/3739856614 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563acd16a3c0 0x563acd174ca0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f59d802aa70 tx=0x7f59d8040000 comp rx=0 tx=0).stop 2024-04-24T08:53:08.084 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.082+0000 7f59f1d76740 1 -- 172.21.15.83:0/3739856614 shutdown_connections 2024-04-24T08:53:08.084 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.082+0000 7f59f1d76740 1 --2- 172.21.15.83:0/3739856614 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563acd1755a0 0x563acd16c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:08.085 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.082+0000 7f59f1d76740 1 --2- 172.21.15.83:0/3739856614 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563acd16ace0 0x563acd166210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:08.085 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.082+0000 7f59f1d76740 1 --2- 172.21.15.83:0/3739856614 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563acd16a3c0 0x563acd174ca0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:08.085 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.082+0000 7f59f1d76740 1 -- 172.21.15.83:0/3739856614 >> 172.21.15.83:0/3739856614 conn(0x563acd1668d0 msgr2=0x563acd16bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:08.085 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.082+0000 7f59f1d76740 1 -- 172.21.15.83:0/3739856614 shutdown_connections 2024-04-24T08:53:08.085 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.082+0000 7f59f1d76740 1 -- 172.21.15.83:0/3739856614 wait complete. 2024-04-24T08:53:08.085 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.082+0000 7f59f1d76740 1 -- start start 2024-04-24T08:53:08.085 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.082+0000 7f59f1d76740 1 Processor -- start 2024-04-24T08:53:08.085 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.083+0000 7f59f1d76740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563acd16a3c0 0x563acd185e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:08.085 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.083+0000 7f59f1d76740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563acd16ace0 0x563acd186340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:08.085 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.083+0000 7f59f1d76740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563acd1755a0 0x563acd182dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:08.086 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.083+0000 7f59f1d76740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f59e81ae680 con 0x563acd16a3c0 2024-04-24T08:53:08.086 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.083+0000 7f59f1d76740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f59e81adfb0 con 0x563acd16ace0 2024-04-24T08:53:08.086 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.083+0000 7f59f1d76740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f59e81ab1f0 con 0x563acd1755a0 2024-04-24T08:53:08.086 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.083+0000 7f59f1d76740 10 client.0 _ll_register_callbacks cb 0x7f59d8005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:53:08.086 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.083+0000 7f59e7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563acd16a3c0 0x563acd185e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:08.086 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.083+0000 7f59e7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563acd16a3c0 0x563acd185e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38932/0 (socket says 172.21.15.83:38932) 2024-04-24T08:53:08.086 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.083+0000 7f59e7e00640 1 -- 172.21.15.83:0/4209299007 learned_addr learned my addr 172.21.15.83:0/4209299007 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:53:08.086 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.083+0000 7f59e7400640 1 --2- 172.21.15.83:0/4209299007 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563acd16ace0 0x563acd186340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:08.086 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.083+0000 7f59e7e00640 1 -- 172.21.15.83:0/4209299007 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563acd1755a0 msgr2=0x563acd182dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:08.086 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.083+0000 7f59e7e00640 1 --2- 172.21.15.83:0/4209299007 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563acd1755a0 0x563acd182dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:08.087 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.083+0000 7f59e7e00640 1 -- 172.21.15.83:0/4209299007 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563acd16ace0 msgr2=0x563acd186340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:08.087 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.083+0000 7f59e7e00640 1 --2- 172.21.15.83:0/4209299007 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563acd16ace0 0x563acd186340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:08.087 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.083+0000 7f59e7e00640 1 -- 172.21.15.83:0/4209299007 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563acceaf620 con 0x563acd16a3c0 2024-04-24T08:53:08.087 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.084+0000 7f59e7e00640 1 --2- 172.21.15.83:0/4209299007 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563acd16a3c0 0x563acd185e00 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f59d8031440 tx=0x7f59d80058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43d4c231fae104a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:08.087 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.084+0000 7f59d2000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:53:08.087 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.084+0000 7f59d2000640 1 -- 172.21.15.83:0/4209299007 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d805a070 con 0x563acd16a3c0 2024-04-24T08:53:08.087 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.084+0000 7f59f1d76740 1 -- 172.21.15.83:0/4209299007 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59d8054780 con 0x563acd16a3c0 2024-04-24T08:53:08.087 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.084+0000 7f59d2000640 1 -- 172.21.15.83:0/4209299007 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f59d80544e0 con 0x563acd16a3c0 2024-04-24T08:53:08.087 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.084+0000 7f59d2000640 1 -- 172.21.15.83:0/4209299007 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f59d8054930 con 0x563acd16a3c0 2024-04-24T08:53:08.087 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.084+0000 7f59f1d76740 10 client.16251 fetch_fsmap learned FSMap version 28 2024-04-24T08:53:08.087 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.084+0000 7f59f1d76740 1 -- 172.21.15.83:0/4209299007 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563acd1f49b0 con 0x563acd16a3c0 2024-04-24T08:53:08.087 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.085+0000 7f59d2000640 1 -- 172.21.15.83:0/4209299007 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f59d8054780 con 0x563acd16a3c0 2024-04-24T08:53:08.088 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.085+0000 7f59d2000640 1 -- 172.21.15.83:0/4209299007 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f59d8044070 con 0x563acd16a3c0 2024-04-24T08:53:08.088 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.085+0000 7f59f1d76740 10 client.16251 fetch_fsmap finished waiting for FSMap version 28 2024-04-24T08:53:08.088 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.085+0000 7f59f1d76740 10 client.16251 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T08:53:08.088 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.085+0000 7f59f1d76740 20 client.16251 populate_metadata read hostname 'smithi083' 2024-04-24T08:53:08.088 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.085+0000 7f59f1d76740 1 --2- 172.21.15.83:0/4209299007 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x563acd1f50e0 0x563acd1f7500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:08.088 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.085+0000 7f59f1d76740 4 client.16251 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T08:53:08.088 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.085+0000 7f59f1d76740 1 -- 172.21.15.83:0/4209299007 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563acd0d8190 con 0x563acd1f50e0 2024-04-24T08:53:08.089 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.086+0000 7f59e7400640 1 --2- 172.21.15.83:0/4209299007 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x563acd1f50e0 0x563acd1f7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:08.089 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.087+0000 7f59d2000640 10 client.16251 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:53:08.089 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.087+0000 7f59e7400640 1 --2- 172.21.15.83:0/4209299007 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x563acd1f50e0 0x563acd1f7500 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e0c8eda1cbb7b23 server_cookie=76bb38d62035b1e4 in_seq=0 out_seq=0 2024-04-24T08:53:08.103 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.103+0000 7f59d2000640 1 -- 172.21.15.83:0/4209299007 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563acd0d8190 con 0x563acd1f50e0 2024-04-24T08:53:08.103 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.103+0000 7f59d2000640 10 client.16251 handle_command_reply: tid=0 2024-04-24T08:53:08.103 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.103+0000 7f59f1d76740 1 client.16251 shutdown 2024-04-24T08:53:08.103 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.103+0000 7f59f1d76740 1 -- 172.21.15.83:0/4209299007 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563acd16a3c0 msgr2=0x563acd185e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:08.103 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.103+0000 7f59f1d76740 1 --2- 172.21.15.83:0/4209299007 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563acd16a3c0 0x563acd185e00 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f59d8031440 tx=0x7f59d80058d0 comp rx=0 tx=0).stop 2024-04-24T08:53:08.103 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.103+0000 7f59f1d76740 1 -- 172.21.15.83:0/4209299007 shutdown_connections 2024-04-24T08:53:08.104 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.103+0000 7f59f1d76740 1 --2- 172.21.15.83:0/4209299007 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x563acd1f50e0 0x563acd1f7500 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:08.104 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.103+0000 7f59f1d76740 1 --2- 172.21.15.83:0/4209299007 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563acd1755a0 0x563acd182dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:08.104 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.103+0000 7f59f1d76740 1 --2- 172.21.15.83:0/4209299007 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563acd16ace0 0x563acd186340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:08.104 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.104+0000 7f59f1d76740 1 --2- 172.21.15.83:0/4209299007 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563acd16a3c0 0x563acd185e00 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:08.104 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.104+0000 7f59f1d76740 1 -- 172.21.15.83:0/4209299007 >> 172.21.15.83:0/4209299007 conn(0x563acd1668d0 msgr2=0x563acd16c600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:08.104 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.104+0000 7f59d2000640 0 client.16251 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:53:08.104 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.104+0000 7f59f1d76740 1 -- 172.21.15.83:0/4209299007 shutdown_connections 2024-04-24T08:53:08.104 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.104+0000 7f59f1d76740 1 -- 172.21.15.83:0/4209299007 wait complete. 2024-04-24T08:53:08.104 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.104+0000 7f59f1d76740 20 client.16251 trim_cache size 0 max 16384 2024-04-24T08:53:08.105 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T08:53:08.105 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T08:53:08.105 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T08:53:08.105 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T08:53:08.106 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.105+0000 7f59eda00640 1 -- 172.21.15.83:0/783176793 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f59cc0a3eb0 msgr2=0x7f59cc0a6370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:08.106 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.105+0000 7f59eda00640 1 --2- 172.21.15.83:0/783176793 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f59cc0a3eb0 0x7f59cc0a6370 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x563acd16b960 tx=0x7f59d8005a20 comp rx=0 tx=0).stop 2024-04-24T08:53:08.106 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.105+0000 7f59eda00640 1 -- 172.21.15.83:0/783176793 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59e810e6c0 msgr2=0x7f59e819a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:08.106 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.105+0000 7f59eda00640 1 --2- 172.21.15.83:0/783176793 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59e810e6c0 0x7f59e819a770 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f59e8075320 tx=0x7f59e000edd0 comp rx=0 tx=0).stop 2024-04-24T08:53:08.106 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.105+0000 7f59eda00640 1 -- 172.21.15.83:0/783176793 shutdown_connections 2024-04-24T08:53:08.106 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.105+0000 7f59eda00640 1 --2- 172.21.15.83:0/783176793 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f59cc0a3eb0 0x7f59cc0a6370 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:08.106 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.105+0000 7f59eda00640 1 --2- 172.21.15.83:0/783176793 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59e810e6c0 0x7f59e819a770 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:08.106 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.105+0000 7f59eda00640 1 --2- 172.21.15.83:0/783176793 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f59e8074cf0 0x7f59e81a0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:08.106 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.105+0000 7f59eda00640 1 --2- 172.21.15.83:0/783176793 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59e80743d0 0x7f59e81a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:08.107 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.105+0000 7f59eda00640 1 -- 172.21.15.83:0/783176793 >> 172.21.15.83:0/783176793 conn(0x7f59e80fd950 msgr2=0x7f59e80fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:08.107 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.105+0000 7f59eda00640 1 -- 172.21.15.83:0/783176793 shutdown_connections 2024-04-24T08:53:08.107 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:08.105+0000 7f59eda00640 1 -- 172.21.15.83:0/783176793 wait complete. 2024-04-24T08:53:08.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:53:07 smithi189 ceph-mon[31791]: pgmap v48: 129 pgs: 129 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-04-24T08:53:10.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:53:09 smithi083 ceph-mon[27907]: pgmap v49: 129 pgs: 129 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 23 KiB/s wr, 30 op/s 2024-04-24T08:53:10.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:53:09 smithi083 ceph-mon[27907]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-04-24T08:53:10.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:53:09 smithi138 ceph-mon[31441]: pgmap v49: 129 pgs: 129 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 23 KiB/s wr, 30 op/s 2024-04-24T08:53:10.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:53:09 smithi138 ceph-mon[31441]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-04-24T08:53:10.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:53:09 smithi189 ceph-mon[31791]: pgmap v49: 129 pgs: 129 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 23 KiB/s wr, 30 op/s 2024-04-24T08:53:10.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:53:09 smithi189 ceph-mon[31791]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-04-24T08:53:12.052 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:53:11 smithi083 ceph-mon[27907]: pgmap v50: 129 pgs: 129 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-04-24T08:53:12.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:53:11 smithi138 ceph-mon[31441]: pgmap v50: 129 pgs: 129 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-04-24T08:53:12.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:53:11 smithi189 ceph-mon[31791]: pgmap v50: 129 pgs: 129 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-04-24T08:53:14.052 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:53:13 smithi083 ceph-mon[27907]: pgmap v51: 129 pgs: 129 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 29 op/s 2024-04-24T08:53:14.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:53:13 smithi138 ceph-mon[31441]: pgmap v51: 129 pgs: 129 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 29 op/s 2024-04-24T08:53:14.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:53:13 smithi189 ceph-mon[31791]: pgmap v51: 129 pgs: 129 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 29 op/s 2024-04-24T08:53:16.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:53:15 smithi083 ceph-mon[27907]: pgmap v52: 129 pgs: 129 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2024-04-24T08:53:16.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:53:15 smithi138 ceph-mon[31441]: pgmap v52: 129 pgs: 129 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2024-04-24T08:53:16.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:53:15 smithi189 ceph-mon[31791]: pgmap v52: 129 pgs: 129 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2024-04-24T08:53:18.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:53:17 smithi083 ceph-mon[27907]: pgmap v53: 129 pgs: 129 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-04-24T08:53:18.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:53:17 smithi138 ceph-mon[31441]: pgmap v53: 129 pgs: 129 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-04-24T08:53:18.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:53:17 smithi189 ceph-mon[31791]: pgmap v53: 129 pgs: 129 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-04-24T08:53:20.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:53:19 smithi083 ceph-mon[27907]: pgmap v54: 129 pgs: 129 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.3 KiB/s wr, 14 op/s 2024-04-24T08:53:20.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:53:19 smithi138 ceph-mon[31441]: pgmap v54: 129 pgs: 129 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.3 KiB/s wr, 14 op/s 2024-04-24T08:53:20.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:53:19 smithi189 ceph-mon[31791]: pgmap v54: 129 pgs: 129 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.3 KiB/s wr, 14 op/s 2024-04-24T08:53:21.243 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.242+0000 7efcafe00640 1 -- 172.21.15.83:0/252787370 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efca8074d50 msgr2=0x7efca810e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:21.243 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.242+0000 7efcafe00640 1 --2- 172.21.15.83:0/252787370 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efca8074d50 0x7efca810e300 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7efc94009900 tx=0x7efc9402a480 comp rx=0 tx=0).stop 2024-04-24T08:53:21.244 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.243+0000 7efcafe00640 1 -- 172.21.15.83:0/252787370 shutdown_connections 2024-04-24T08:53:21.244 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.243+0000 7efcafe00640 1 --2- 172.21.15.83:0/252787370 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efca810e840 0x7efca8110c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:21.244 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.243+0000 7efcafe00640 1 --2- 172.21.15.83:0/252787370 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efca8074d50 0x7efca810e300 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:21.244 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.243+0000 7efcafe00640 1 --2- 172.21.15.83:0/252787370 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efca8074430 0x7efca8074810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:21.244 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.243+0000 7efcafe00640 1 -- 172.21.15.83:0/252787370 >> 172.21.15.83:0/252787370 conn(0x7efca80fda90 msgr2=0x7efca80ffeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:21.244 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.243+0000 7efcafe00640 1 -- 172.21.15.83:0/252787370 shutdown_connections 2024-04-24T08:53:21.244 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.243+0000 7efcafe00640 1 -- 172.21.15.83:0/252787370 wait complete. 2024-04-24T08:53:21.247 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.244+0000 7efcafe00640 1 Processor -- start 2024-04-24T08:53:21.247 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.244+0000 7efcafe00640 1 -- start start 2024-04-24T08:53:21.247 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.244+0000 7efcafe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efca8074430 0x7efca819aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:21.247 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.244+0000 7efcafe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efca8074d50 0x7efca819b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:21.247 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.244+0000 7efcafe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efca810e840 0x7efca819f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:21.247 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.244+0000 7efcafe00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7efca81144e0 con 0x7efca8074430 2024-04-24T08:53:21.247 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.244+0000 7efcafe00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efca8114360 con 0x7efca810e840 2024-04-24T08:53:21.247 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.244+0000 7efcafe00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7efca8114660 con 0x7efca8074d50 2024-04-24T08:53:21.248 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.244+0000 7efcad400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efca8074430 0x7efca819aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:21.248 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.244+0000 7efcad400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efca8074430 0x7efca819aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50386/0 (socket says 172.21.15.83:50386) 2024-04-24T08:53:21.248 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.244+0000 7efcad400640 1 -- 172.21.15.83:0/1226179754 learned_addr learned my addr 172.21.15.83:0/1226179754 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:53:21.248 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.245+0000 7efcaca00640 1 --2- 172.21.15.83:0/1226179754 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efca8074d50 0x7efca819b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:21.248 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.245+0000 7efcad400640 1 -- 172.21.15.83:0/1226179754 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efca8074d50 msgr2=0x7efca819b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:21.248 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.245+0000 7efcad400640 1 --2- 172.21.15.83:0/1226179754 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efca8074d50 0x7efca819b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:21.248 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.245+0000 7efcad400640 1 -- 172.21.15.83:0/1226179754 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efca810e840 msgr2=0x7efca819f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:21.248 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.245+0000 7efcade00640 1 --2- 172.21.15.83:0/1226179754 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efca810e840 0x7efca819f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:21.248 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.245+0000 7efcad400640 1 --2- 172.21.15.83:0/1226179754 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efca810e840 0x7efca819f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:21.248 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.245+0000 7efcad400640 1 -- 172.21.15.83:0/1226179754 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efca819fae0 con 0x7efca8074430 2024-04-24T08:53:21.248 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.245+0000 7efcad400640 1 --2- 172.21.15.83:0/1226179754 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efca8074430 0x7efca819aac0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7efc9c00b4a0 tx=0x7efc9c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:21.248 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.245+0000 7efca6000640 1 -- 172.21.15.83:0/1226179754 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc9c014070 con 0x7efca8074430 2024-04-24T08:53:21.249 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.245+0000 7efca6000640 1 -- 172.21.15.83:0/1226179754 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efc9c010ac0 con 0x7efca8074430 2024-04-24T08:53:21.249 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.245+0000 7efca6000640 1 -- 172.21.15.83:0/1226179754 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc9c017c30 con 0x7efca8074430 2024-04-24T08:53:21.249 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.246+0000 7efcafe00640 1 -- 172.21.15.83:0/1226179754 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efca819fdd0 con 0x7efca8074430 2024-04-24T08:53:21.249 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.246+0000 7efcafe00640 1 -- 172.21.15.83:0/1226179754 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efca81a7660 con 0x7efca8074430 2024-04-24T08:53:21.251 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.249+0000 7efcb2bf3740 1 Processor -- start 2024-04-24T08:53:21.251 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.249+0000 7efcb2bf3740 1 -- start start 2024-04-24T08:53:21.251 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.249+0000 7efcb2bf3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f28c691b0 0x556f28c69590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:21.251 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.249+0000 7efcb2bf3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f28c69ad0 0x556f28c73e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:21.251 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.249+0000 7efcb2bf3740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f28c74390 0x556f28c76780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:21.251 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.249+0000 7efcb2bf3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556f28bcf900 con 0x556f28c691b0 2024-04-24T08:53:21.251 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.249+0000 7efcb2bf3740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556f28b75f20 con 0x556f28c74390 2024-04-24T08:53:21.251 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.249+0000 7efcb2bf3740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x556f28bcb8c0 con 0x556f28c69ad0 2024-04-24T08:53:21.251 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.250+0000 7efcade00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f28c691b0 0x556f28c69590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:21.251 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.250+0000 7efcade00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f28c691b0 0x556f28c69590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50392/0 (socket says 172.21.15.83:50392) 2024-04-24T08:53:21.252 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.250+0000 7efcade00640 1 -- 172.21.15.83:0/856902523 learned_addr learned my addr 172.21.15.83:0/856902523 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:53:21.252 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.250+0000 7efcaca00640 1 --2- 172.21.15.83:0/856902523 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f28c74390 0x556f28c76780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:21.252 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.250+0000 7efcade00640 1 -- 172.21.15.83:0/856902523 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f28c69ad0 msgr2=0x556f28c73e50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T08:53:21.252 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.250+0000 7efcade00640 1 --2- 172.21.15.83:0/856902523 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f28c69ad0 0x556f28c73e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:21.252 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.250+0000 7efcade00640 1 -- 172.21.15.83:0/856902523 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f28c74390 msgr2=0x556f28c76780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:21.252 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.250+0000 7efca6000640 1 -- 172.21.15.83:0/1226179754 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7efc9c017070 con 0x7efca8074430 2024-04-24T08:53:21.252 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.250+0000 7efcade00640 1 --2- 172.21.15.83:0/856902523 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f28c74390 0x556f28c76780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:21.252 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.250+0000 7efcade00640 1 -- 172.21.15.83:0/856902523 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f28b9ae10 con 0x556f28c691b0 2024-04-24T08:53:21.252 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.250+0000 7efcade00640 1 --2- 172.21.15.83:0/856902523 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f28c691b0 0x556f28c69590 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7efca0008da0 tx=0x7efca000d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f55293a4ab6fea79 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:21.252 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.251+0000 7efc9a600640 1 -- 172.21.15.83:0/856902523 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca000f830 con 0x556f28c691b0 2024-04-24T08:53:21.252 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.251+0000 7efc9a600640 1 -- 172.21.15.83:0/856902523 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efca000de20 con 0x556f28c691b0 2024-04-24T08:53:21.252 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.251+0000 7efc9a600640 1 -- 172.21.15.83:0/856902523 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca0018dc0 con 0x556f28c691b0 2024-04-24T08:53:21.256 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.252+0000 7efca6000640 1 --2- 172.21.15.83:0/1226179754 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7efc900a3dc0 0x7efc900a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:21.256 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.252+0000 7efcb2bf3740 1 -- 172.21.15.83:0/856902523 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f28c691b0 msgr2=0x556f28c69590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:21.256 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.252+0000 7efcb2bf3740 1 --2- 172.21.15.83:0/856902523 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f28c691b0 0x556f28c69590 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7efca0008da0 tx=0x7efca000d710 comp rx=0 tx=0).stop 2024-04-24T08:53:21.256 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.252+0000 7efcb2bf3740 1 -- 172.21.15.83:0/856902523 shutdown_connections 2024-04-24T08:53:21.256 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.252+0000 7efcb2bf3740 1 --2- 172.21.15.83:0/856902523 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f28c74390 0x556f28c76780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:21.256 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.252+0000 7efcb2bf3740 1 --2- 172.21.15.83:0/856902523 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f28c69ad0 0x556f28c73e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:21.256 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.252+0000 7efcb2bf3740 1 --2- 172.21.15.83:0/856902523 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f28c691b0 0x556f28c69590 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:21.256 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.252+0000 7efcb2bf3740 1 -- 172.21.15.83:0/856902523 >> 172.21.15.83:0/856902523 conn(0x556f28c656c0 msgr2=0x556f28c65aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:21.257 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.252+0000 7efcad400640 1 --2- 172.21.15.83:0/1226179754 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7efc900a3dc0 0x7efc900a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:21.257 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.252+0000 7efca6000640 1 -- 172.21.15.83:0/1226179754 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7efc9c0d0270 con 0x7efca8074430 2024-04-24T08:53:21.257 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.252+0000 7efcb2bf3740 1 -- 172.21.15.83:0/856902523 shutdown_connections 2024-04-24T08:53:21.257 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.252+0000 7efcb2bf3740 1 -- 172.21.15.83:0/856902523 wait complete. 2024-04-24T08:53:21.257 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.252+0000 7efcad400640 1 --2- 172.21.15.83:0/1226179754 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7efc900a3dc0 0x7efc900a6280 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x556f28c6a750 tx=0x7efc9c09bb30 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:21.257 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.253+0000 7efcb2bf3740 1 -- start start 2024-04-24T08:53:21.257 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.253+0000 7efcb2bf3740 1 Processor -- start 2024-04-24T08:53:21.257 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.254+0000 7efcb2bf3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f28c691b0 0x556f28c712a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:21.257 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.254+0000 7efcb2bf3740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f28c69ad0 0x556f28c717e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:21.257 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.254+0000 7efcb2bf3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f28c74390 0x556f28c71d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:21.257 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.254+0000 7efcb2bf3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556f28c76fc0 con 0x556f28c691b0 2024-04-24T08:53:21.257 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.254+0000 7efcb2bf3740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556f28b892c0 con 0x556f28c69ad0 2024-04-24T08:53:21.257 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.254+0000 7efcb2bf3740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x556f28c77140 con 0x556f28c74390 2024-04-24T08:53:21.257 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.254+0000 7efcb2bf3740 10 client.0 _ll_register_callbacks cb 0x556f288e1a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:53:21.257 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.254+0000 7efcade00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f28c691b0 0x556f28c712a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:21.257 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.254+0000 7efcade00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f28c691b0 0x556f28c712a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50396/0 (socket says 172.21.15.83:50396) 2024-04-24T08:53:21.258 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.254+0000 7efcade00640 1 -- 172.21.15.83:0/1485875324 learned_addr learned my addr 172.21.15.83:0/1485875324 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:53:21.258 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.254+0000 7efcade00640 1 -- 172.21.15.83:0/1485875324 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f28c74390 msgr2=0x556f28c71d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:21.258 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.254+0000 7efcade00640 1 --2- 172.21.15.83:0/1485875324 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f28c74390 0x556f28c71d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:21.258 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.254+0000 7efcade00640 1 -- 172.21.15.83:0/1485875324 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f28c69ad0 msgr2=0x556f28c717e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:21.258 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.254+0000 7efcaca00640 1 --2- 172.21.15.83:0/1485875324 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f28c69ad0 0x556f28c717e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:21.258 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.254+0000 7efcade00640 1 --2- 172.21.15.83:0/1485875324 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f28c69ad0 0x556f28c717e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:21.258 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.254+0000 7efcade00640 1 -- 172.21.15.83:0/1485875324 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f28b99fd0 con 0x556f28c691b0 2024-04-24T08:53:21.258 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.255+0000 7efcade00640 1 --2- 172.21.15.83:0/1485875324 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f28c691b0 0x556f28c712a0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7efca000f840 tx=0x7efca000f870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbf8c1aa41055ada server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:21.258 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.255+0000 7efc8ea00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:53:21.258 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.255+0000 7efc8ea00640 1 -- 172.21.15.83:0/1485875324 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca0018480 con 0x556f28c691b0 2024-04-24T08:53:21.258 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.255+0000 7efc8ea00640 1 -- 172.21.15.83:0/1485875324 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7efca00223f0 con 0x556f28c691b0 2024-04-24T08:53:21.258 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.255+0000 7efcb2bf3740 1 -- 172.21.15.83:0/1485875324 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efca000de20 con 0x556f28c691b0 2024-04-24T08:53:21.258 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.255+0000 7efc8ea00640 1 -- 172.21.15.83:0/1485875324 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7efca0030050 con 0x556f28c691b0 2024-04-24T08:53:21.258 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.255+0000 7efcb2bf3740 10 client.16263 fetch_fsmap learned FSMap version 28 2024-04-24T08:53:21.259 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.255+0000 7efcb2bf3740 1 -- 172.21.15.83:0/1485875324 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556f288e2210 con 0x556f28c691b0 2024-04-24T08:53:21.259 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.255+0000 7efc8ea00640 1 -- 172.21.15.83:0/1485875324 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efca000de20 con 0x556f28c691b0 2024-04-24T08:53:21.259 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.256+0000 7efc8ea00640 1 -- 172.21.15.83:0/1485875324 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7efca002b3b0 con 0x556f28c691b0 2024-04-24T08:53:21.259 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.256+0000 7efcb2bf3740 10 client.16263 fetch_fsmap finished waiting for FSMap version 28 2024-04-24T08:53:21.259 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.256+0000 7efcb2bf3740 10 client.16263 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T08:53:21.259 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.256+0000 7efcb2bf3740 20 client.16263 populate_metadata read hostname 'smithi083' 2024-04-24T08:53:21.259 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.256+0000 7efcb2bf3740 1 --2- 172.21.15.83:0/1485875324 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556f28c84390 0x556f28c86780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:21.259 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.256+0000 7efcb2bf3740 4 client.16263 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:53:21.259 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.256+0000 7efcb2bf3740 1 -- 172.21.15.83:0/1485875324 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556f28bdd320 con 0x556f28c84390 2024-04-24T08:53:21.260 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.256+0000 7efcaca00640 1 --2- 172.21.15.83:0/1485875324 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556f28c84390 0x556f28c86780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:21.260 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.259+0000 7efcaca00640 1 --2- 172.21.15.83:0/1485875324 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556f28c84390 0x556f28c86780 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cfcef9a9fb998eca server_cookie=5b5ef024fe55be04 in_seq=0 out_seq=0 2024-04-24T08:53:21.260 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.259+0000 7efc8ea00640 10 client.16263 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:53:21.261 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.261+0000 7efc8ea00640 1 -- 172.21.15.83:0/1485875324 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556f28bdd320 con 0x556f28c84390 2024-04-24T08:53:21.261 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.261+0000 7efc8ea00640 10 client.16263 handle_command_reply: tid=0 2024-04-24T08:53:21.261 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.261+0000 7efcb2bf3740 1 client.16263 shutdown 2024-04-24T08:53:21.261 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.261+0000 7efcb2bf3740 1 -- 172.21.15.83:0/1485875324 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f28c691b0 msgr2=0x556f28c712a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:21.261 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.261+0000 7efcb2bf3740 1 --2- 172.21.15.83:0/1485875324 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f28c691b0 0x556f28c712a0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7efca000f840 tx=0x7efca000f870 comp rx=0 tx=0).stop 2024-04-24T08:53:21.262 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.261+0000 7efcb2bf3740 1 -- 172.21.15.83:0/1485875324 shutdown_connections 2024-04-24T08:53:21.262 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.261+0000 7efcb2bf3740 1 --2- 172.21.15.83:0/1485875324 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556f28c84390 0x556f28c86780 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:21.262 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.261+0000 7efcb2bf3740 1 --2- 172.21.15.83:0/1485875324 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f28c74390 0x556f28c71d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:21.262 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.261+0000 7efcb2bf3740 1 --2- 172.21.15.83:0/1485875324 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f28c69ad0 0x556f28c717e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:21.262 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.261+0000 7efcb2bf3740 1 --2- 172.21.15.83:0/1485875324 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f28c691b0 0x556f28c712a0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:21.262 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.261+0000 7efc8ea00640 0 client.16263 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:53:21.262 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.261+0000 7efcb2bf3740 1 -- 172.21.15.83:0/1485875324 >> 172.21.15.83:0/1485875324 conn(0x556f28c656c0 msgr2=0x556f28c6b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:21.262 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.261+0000 7efcb2bf3740 1 -- 172.21.15.83:0/1485875324 shutdown_connections 2024-04-24T08:53:21.262 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.262+0000 7efcb2bf3740 1 -- 172.21.15.83:0/1485875324 wait complete. 2024-04-24T08:53:21.263 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.262+0000 7efcb2bf3740 20 client.16263 trim_cache size 0 max 16384 2024-04-24T08:53:21.284 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.277+0000 7efcb2bf3740 1 Processor -- start 2024-04-24T08:53:21.284 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.277+0000 7efcb2bf3740 1 -- start start 2024-04-24T08:53:21.284 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.277+0000 7efcb2bf3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f28c691b0 0x556f28c7fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:21.284 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.277+0000 7efcb2bf3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f28c69ad0 0x556f28c70a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:21.284 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.277+0000 7efcb2bf3740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f28c74390 0x556f28c8bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:21.284 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.277+0000 7efcb2bf3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7efca81b1780 con 0x556f28c691b0 2024-04-24T08:53:21.284 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.277+0000 7efcb2bf3740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efca81b32c0 con 0x556f28c74390 2024-04-24T08:53:21.284 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.277+0000 7efcb2bf3740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7efca81b1e50 con 0x556f28c69ad0 2024-04-24T08:53:21.284 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.278+0000 7efcade00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f28c691b0 0x556f28c7fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:21.284 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.278+0000 7efcade00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f28c691b0 0x556f28c7fe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50406/0 (socket says 172.21.15.83:50406) 2024-04-24T08:53:21.285 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.278+0000 7efcade00640 1 -- 172.21.15.83:0/3577392991 learned_addr learned my addr 172.21.15.83:0/3577392991 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:53:21.285 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.278+0000 7efcaca00640 1 --2- 172.21.15.83:0/3577392991 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f28c69ad0 0x556f28c70a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:21.285 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.278+0000 7efcade00640 1 -- 172.21.15.83:0/3577392991 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f28c69ad0 msgr2=0x556f28c70a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:21.285 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.278+0000 7efcade00640 1 --2- 172.21.15.83:0/3577392991 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f28c69ad0 0x556f28c70a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:21.285 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.278+0000 7efcade00640 1 -- 172.21.15.83:0/3577392991 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f28c74390 msgr2=0x556f28c8bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:21.285 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.278+0000 7efcade00640 1 --2- 172.21.15.83:0/3577392991 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f28c74390 0x556f28c8bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:21.285 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.278+0000 7efcade00640 1 -- 172.21.15.83:0/3577392991 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f28c69900 con 0x556f28c691b0 2024-04-24T08:53:21.285 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.278+0000 7efcaca00640 1 --2- 172.21.15.83:0/3577392991 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f28c69ad0 0x556f28c70a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:53:21.285 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.279+0000 7efcade00640 1 --2- 172.21.15.83:0/3577392991 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f28c691b0 0x556f28c7fe80 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7efca0005f10 tx=0x7efca0008570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a09ad5d5a854eaa server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:21.285 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.279+0000 7efc99c00640 1 -- 172.21.15.83:0/3577392991 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca0018480 con 0x556f28c691b0 2024-04-24T08:53:21.285 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.279+0000 7efc99c00640 1 -- 172.21.15.83:0/3577392991 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efca0027070 con 0x556f28c691b0 2024-04-24T08:53:21.285 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.279+0000 7efc99c00640 1 -- 172.21.15.83:0/3577392991 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca002b3b0 con 0x556f28c691b0 2024-04-24T08:53:21.285 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.280+0000 7efcb2bf3740 1 -- 172.21.15.83:0/3577392991 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f28c691b0 msgr2=0x556f28c7fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:21.286 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.280+0000 7efcb2bf3740 1 --2- 172.21.15.83:0/3577392991 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f28c691b0 0x556f28c7fe80 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7efca0005f10 tx=0x7efca0008570 comp rx=0 tx=0).stop 2024-04-24T08:53:21.286 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.280+0000 7efcb2bf3740 1 -- 172.21.15.83:0/3577392991 shutdown_connections 2024-04-24T08:53:21.286 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.280+0000 7efcb2bf3740 1 --2- 172.21.15.83:0/3577392991 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f28c74390 0x556f28c8bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:21.286 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.280+0000 7efcb2bf3740 1 --2- 172.21.15.83:0/3577392991 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f28c69ad0 0x556f28c70a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:21.286 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.280+0000 7efcb2bf3740 1 --2- 172.21.15.83:0/3577392991 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f28c691b0 0x556f28c7fe80 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:21.286 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.280+0000 7efcb2bf3740 1 -- 172.21.15.83:0/3577392991 >> 172.21.15.83:0/3577392991 conn(0x556f28c656c0 msgr2=0x556f28c72ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:21.286 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.280+0000 7efcb2bf3740 1 -- 172.21.15.83:0/3577392991 shutdown_connections 2024-04-24T08:53:21.286 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.280+0000 7efcb2bf3740 1 -- 172.21.15.83:0/3577392991 wait complete. 2024-04-24T08:53:21.286 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.280+0000 7efcb2bf3740 1 -- start start 2024-04-24T08:53:21.286 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.280+0000 7efcb2bf3740 1 Processor -- start 2024-04-24T08:53:21.286 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.281+0000 7efcb2bf3740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f28c691b0 0x556f28c6e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:21.286 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.281+0000 7efcb2bf3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f28c69ad0 0x556f28c6e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:21.286 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.281+0000 7efcb2bf3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f28c74390 0x556f28c6ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:21.286 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.281+0000 7efcb2bf3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7efca81b2bf0 con 0x556f28c69ad0 2024-04-24T08:53:21.287 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.281+0000 7efcb2bf3740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efca81b2520 con 0x556f28c691b0 2024-04-24T08:53:21.287 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.281+0000 7efcb2bf3740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7efca81af760 con 0x556f28c74390 2024-04-24T08:53:21.287 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.281+0000 7efcb2bf3740 10 client.0 _ll_register_callbacks cb 0x7efca0008bf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:53:21.287 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.281+0000 7efcaca00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f28c69ad0 0x556f28c6e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:21.287 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.281+0000 7efcaca00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f28c69ad0 0x556f28c6e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50416/0 (socket says 172.21.15.83:50416) 2024-04-24T08:53:21.287 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.281+0000 7efcaca00640 1 -- 172.21.15.83:0/198185984 learned_addr learned my addr 172.21.15.83:0/198185984 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:53:21.287 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.281+0000 7efcade00640 1 --2- 172.21.15.83:0/198185984 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f28c691b0 0x556f28c6e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:21.287 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.281+0000 7efcaca00640 1 -- 172.21.15.83:0/198185984 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f28c74390 msgr2=0x556f28c6ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:21.287 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.281+0000 7efcaca00640 1 --2- 172.21.15.83:0/198185984 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f28c74390 0x556f28c6ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:21.287 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.281+0000 7efcaca00640 1 -- 172.21.15.83:0/198185984 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f28c691b0 msgr2=0x556f28c6e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:21.287 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.281+0000 7efcaca00640 1 --2- 172.21.15.83:0/198185984 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f28c691b0 0x556f28c6e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:21.287 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.281+0000 7efcaca00640 1 -- 172.21.15.83:0/198185984 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f289ae620 con 0x556f28c69ad0 2024-04-24T08:53:21.287 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.281+0000 7efcade00640 1 --2- 172.21.15.83:0/198185984 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f28c691b0 0x556f28c6e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:53:21.287 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.282+0000 7efcaca00640 1 --2- 172.21.15.83:0/198185984 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f28c69ad0 0x556f28c6e550 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7efc940092a0 tx=0x7efc94004300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5ec7e353af4f5d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:21.288 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.282+0000 7efc8ea00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:53:21.288 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.282+0000 7efc8ea00640 1 -- 172.21.15.83:0/198185984 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc94035040 con 0x556f28c69ad0 2024-04-24T08:53:21.288 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.282+0000 7efc8ea00640 1 -- 172.21.15.83:0/198185984 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7efc940316a0 con 0x556f28c69ad0 2024-04-24T08:53:21.288 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.282+0000 7efcb2bf3740 1 -- 172.21.15.83:0/198185984 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efca0027070 con 0x556f28c69ad0 2024-04-24T08:53:21.288 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.282+0000 7efc8ea00640 1 -- 172.21.15.83:0/198185984 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7efc94051050 con 0x556f28c69ad0 2024-04-24T08:53:21.288 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.282+0000 7efcb2bf3740 10 client.16269 fetch_fsmap learned FSMap version 28 2024-04-24T08:53:21.288 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.282+0000 7efcb2bf3740 1 -- 172.21.15.83:0/198185984 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556f28cea760 con 0x556f28c69ad0 2024-04-24T08:53:21.288 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.282+0000 7efc8ea00640 1 -- 172.21.15.83:0/198185984 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efca0027070 con 0x556f28c69ad0 2024-04-24T08:53:21.288 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.283+0000 7efc8ea00640 1 -- 172.21.15.83:0/198185984 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7efc94004b40 con 0x556f28c69ad0 2024-04-24T08:53:21.288 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.283+0000 7efcb2bf3740 10 client.16269 fetch_fsmap finished waiting for FSMap version 28 2024-04-24T08:53:21.288 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.283+0000 7efcb2bf3740 10 client.16269 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T08:53:21.288 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.283+0000 7efcb2bf3740 20 client.16269 populate_metadata read hostname 'smithi083' 2024-04-24T08:53:21.288 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.283+0000 7efcb2bf3740 1 --2- 172.21.15.83:0/198185984 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556f28ceaec0 0x556f28ced2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:21.288 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.283+0000 7efcb2bf3740 4 client.16269 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T08:53:21.289 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.283+0000 7efcb2bf3740 1 -- 172.21.15.83:0/198185984 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556f28bd7190 con 0x556f28ceaec0 2024-04-24T08:53:21.289 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.283+0000 7efcade00640 1 --2- 172.21.15.83:0/198185984 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556f28ceaec0 0x556f28ced2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:21.289 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.284+0000 7efcade00640 1 --2- 172.21.15.83:0/198185984 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556f28ceaec0 0x556f28ced2e0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0f623f23b8e905f server_cookie=dbf164a41a49f24 in_seq=0 out_seq=0 2024-04-24T08:53:21.290 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.284+0000 7efc8ea00640 10 client.16269 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:53:21.296 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.295+0000 7efc8ea00640 1 -- 172.21.15.83:0/198185984 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556f28bd7190 con 0x556f28ceaec0 2024-04-24T08:53:21.296 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.295+0000 7efc8ea00640 10 client.16269 handle_command_reply: tid=0 2024-04-24T08:53:21.296 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.295+0000 7efcb2bf3740 1 client.16269 shutdown 2024-04-24T08:53:21.296 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.295+0000 7efcb2bf3740 1 -- 172.21.15.83:0/198185984 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f28c69ad0 msgr2=0x556f28c6e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:21.296 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.295+0000 7efcb2bf3740 1 --2- 172.21.15.83:0/198185984 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f28c69ad0 0x556f28c6e550 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7efc940092a0 tx=0x7efc94004300 comp rx=0 tx=0).stop 2024-04-24T08:53:21.296 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.295+0000 7efcb2bf3740 1 -- 172.21.15.83:0/198185984 shutdown_connections 2024-04-24T08:53:21.297 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.295+0000 7efcb2bf3740 1 --2- 172.21.15.83:0/198185984 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556f28ceaec0 0x556f28ced2e0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:21.297 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.295+0000 7efcb2bf3740 1 --2- 172.21.15.83:0/198185984 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f28c74390 0x556f28c6ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:21.297 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.295+0000 7efcb2bf3740 1 --2- 172.21.15.83:0/198185984 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f28c69ad0 0x556f28c6e550 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:21.297 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.295+0000 7efcb2bf3740 1 --2- 172.21.15.83:0/198185984 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f28c691b0 0x556f28c6e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:21.297 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.295+0000 7efc8ea00640 0 client.16269 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:53:21.297 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.295+0000 7efcb2bf3740 1 -- 172.21.15.83:0/198185984 >> 172.21.15.83:0/198185984 conn(0x556f28c656c0 msgr2=0x556f28c73360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:21.297 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.295+0000 7efcb2bf3740 1 -- 172.21.15.83:0/198185984 shutdown_connections 2024-04-24T08:53:21.297 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.295+0000 7efcb2bf3740 1 -- 172.21.15.83:0/198185984 wait complete. 2024-04-24T08:53:21.297 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.295+0000 7efcb2bf3740 20 client.16269 trim_cache size 0 max 16384 2024-04-24T08:53:21.298 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T08:53:21.298 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T08:53:21.298 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T08:53:21.298 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T08:53:21.298 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.296+0000 7efcafe00640 1 -- 172.21.15.83:0/1226179754 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7efc900a3dc0 msgr2=0x7efc900a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:21.299 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.296+0000 7efcafe00640 1 --2- 172.21.15.83:0/1226179754 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7efc900a3dc0 0x7efc900a6280 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x556f28c6a750 tx=0x7efc9c09bb30 comp rx=0 tx=0).stop 2024-04-24T08:53:21.299 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.296+0000 7efcafe00640 1 -- 172.21.15.83:0/1226179754 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efca8074430 msgr2=0x7efca819aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:21.299 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.296+0000 7efcafe00640 1 --2- 172.21.15.83:0/1226179754 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efca8074430 0x7efca819aac0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7efc9c00b4a0 tx=0x7efc9c00b960 comp rx=0 tx=0).stop 2024-04-24T08:53:21.299 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.296+0000 7efcafe00640 1 -- 172.21.15.83:0/1226179754 shutdown_connections 2024-04-24T08:53:21.299 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.296+0000 7efcafe00640 1 --2- 172.21.15.83:0/1226179754 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7efc900a3dc0 0x7efc900a6280 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:21.299 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.296+0000 7efcafe00640 1 --2- 172.21.15.83:0/1226179754 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efca810e840 0x7efca819f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:21.299 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.296+0000 7efcafe00640 1 --2- 172.21.15.83:0/1226179754 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efca8074d50 0x7efca819b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:21.299 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.296+0000 7efcafe00640 1 --2- 172.21.15.83:0/1226179754 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efca8074430 0x7efca819aac0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:21.299 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.296+0000 7efcafe00640 1 -- 172.21.15.83:0/1226179754 >> 172.21.15.83:0/1226179754 conn(0x7efca80fda90 msgr2=0x7efca80fe800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:21.299 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.296+0000 7efcafe00640 1 -- 172.21.15.83:0/1226179754 shutdown_connections 2024-04-24T08:53:21.299 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:21.296+0000 7efcafe00640 1 -- 172.21.15.83:0/1226179754 wait complete. 2024-04-24T08:53:22.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:53:21 smithi083 ceph-mon[27907]: pgmap v55: 129 pgs: 129 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 511 B/s wr, 9 op/s 2024-04-24T08:53:22.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:53:21 smithi138 ceph-mon[31441]: pgmap v55: 129 pgs: 129 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 511 B/s wr, 9 op/s 2024-04-24T08:53:22.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:53:21 smithi189 ceph-mon[31791]: pgmap v55: 129 pgs: 129 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 511 B/s wr, 9 op/s 2024-04-24T08:53:23.869 INFO:tasks.workunit.client.0.smithi083.stderr:Updating files: 60% (7438/12271) Updating files: 61% (7486/12271) Updating files: 62% (7609/12271) Updating files: 63% (7731/12271) Updating files: 64% (7854/12271) Updating files: 65% (7977/12271) Updating files: 66% (8099/12271) Updating files: 67% (8222/12271) Updating files: 68% (8345/12271) Updating files: 69% (8467/12271) Updating files: 70% (8590/12271) Updating files: 71% (8713/12271) Updating files: 72% (8836/12271) Updating files: 73% (8958/12271) Updating files: 74% (9081/12271) Updating files: 75% (9204/12271) Updating files: 76% (9326/12271) Updating files: 77% (9449/12271) Updating files: 78% (9572/12271) Updating files: 79% (9695/12271) Updating files: 80% (9817/12271) Updating files: 81% (9940/12271) Updating files: 82% (10063/12271) Updating files: 83% (10185/12271) Updating files: 84% (10308/12271) Updating files: 85% (10431/12271) Updating files: 86% (10554/12271) Updating files: 87% (10676/12271) Updating files: 88% (10799/12271) Updating files: 89% (10922/12271) Updating files: 90% (11044/12271) Updating files: 91% (11167/12271) Updating files: 92% (11290/12271) Updating files: 93% (11413/12271) Updating files: 94% (11535/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-04-24T08:53:24.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:53:23 smithi083 ceph-mon[27907]: pgmap v56: 129 pgs: 129 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 767 B/s wr, 13 op/s 2024-04-24T08:53:24.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:53:23 smithi138 ceph-mon[31441]: pgmap v56: 129 pgs: 129 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 767 B/s wr, 13 op/s 2024-04-24T08:53:24.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:53:23 smithi189 ceph-mon[31791]: pgmap v56: 129 pgs: 129 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 767 B/s wr, 13 op/s 2024-04-24T08:53:24.221 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T08:53:24.221 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T08:53:24.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.416+0000 7fd59fe00640 1 -- 172.21.15.83:0/640468869 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd5a0100c70 msgr2=0x7fd5a0101050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:24.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.416+0000 7fd59fe00640 1 --2- 172.21.15.83:0/640468869 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd5a0100c70 0x7fd5a0101050 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fd590009900 tx=0x7fd59002a420 comp rx=0 tx=0).stop 2024-04-24T08:53:24.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.416+0000 7fd59fe00640 1 -- 172.21.15.83:0/640468869 shutdown_connections 2024-04-24T08:53:24.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.416+0000 7fd59fe00640 1 --2- 172.21.15.83:0/640468869 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd5a010e820 0x7fd5a0110c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:24.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.416+0000 7fd59fe00640 1 --2- 172.21.15.83:0/640468869 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd5a0101620 0x7fd5a010e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:24.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.416+0000 7fd59fe00640 1 --2- 172.21.15.83:0/640468869 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd5a0100c70 0x7fd5a0101050 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:24.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.416+0000 7fd59fe00640 1 -- 172.21.15.83:0/640468869 >> 172.21.15.83:0/640468869 conn(0x7fd5a00fc880 msgr2=0x7fd5a00feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:24.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.416+0000 7fd59fe00640 1 -- 172.21.15.83:0/640468869 shutdown_connections 2024-04-24T08:53:24.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.416+0000 7fd59fe00640 1 -- 172.21.15.83:0/640468869 wait complete. 2024-04-24T08:53:24.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.417+0000 7fd59fe00640 1 Processor -- start 2024-04-24T08:53:24.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.417+0000 7fd59fe00640 1 -- start start 2024-04-24T08:53:24.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.418+0000 7fd59fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd5a0100c70 0x7fd5a01a06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:24.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.418+0000 7fd59fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd5a0101620 0x7fd5a01a0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:24.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.418+0000 7fd59fe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd5a010e820 0x7fd5a019a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:24.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.418+0000 7fd59fe00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a0114420 con 0x7fd5a0100c70 2024-04-24T08:53:24.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.418+0000 7fd59fe00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a01142a0 con 0x7fd5a0101620 2024-04-24T08:53:24.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.418+0000 7fd59fe00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a01145a0 con 0x7fd5a010e820 2024-04-24T08:53:24.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.418+0000 7fd59ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd5a0100c70 0x7fd5a01a06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:24.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.418+0000 7fd59ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd5a0100c70 0x7fd5a01a06d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50428/0 (socket says 172.21.15.83:50428) 2024-04-24T08:53:24.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.418+0000 7fd59ea00640 1 -- 172.21.15.83:0/727152824 learned_addr learned my addr 172.21.15.83:0/727152824 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:53:24.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.418+0000 7fd59e000640 1 --2- 172.21.15.83:0/727152824 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd5a0101620 0x7fd5a01a0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:24.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.419+0000 7fd59f400640 1 --2- 172.21.15.83:0/727152824 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd5a010e820 0x7fd5a019a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:24.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.419+0000 7fd59ea00640 1 -- 172.21.15.83:0/727152824 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd5a010e820 msgr2=0x7fd5a019a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:24.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.419+0000 7fd59ea00640 1 --2- 172.21.15.83:0/727152824 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd5a010e820 0x7fd5a019a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:24.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.419+0000 7fd59ea00640 1 -- 172.21.15.83:0/727152824 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd5a0101620 msgr2=0x7fd5a01a0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:24.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.419+0000 7fd59ea00640 1 --2- 172.21.15.83:0/727152824 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd5a0101620 0x7fd5a01a0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:24.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.419+0000 7fd59ea00640 1 -- 172.21.15.83:0/727152824 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5a019b010 con 0x7fd5a0100c70 2024-04-24T08:53:24.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.419+0000 7fd59f400640 1 --2- 172.21.15.83:0/727152824 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd5a010e820 0x7fd5a019a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:53:24.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.419+0000 7fd59ea00640 1 --2- 172.21.15.83:0/727152824 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd5a0100c70 0x7fd5a01a06d0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fd590005940 tx=0x7fd590004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:24.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.419+0000 7fd597400640 1 -- 172.21.15.83:0/727152824 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd59003f070 con 0x7fd5a0100c70 2024-04-24T08:53:24.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.419+0000 7fd597400640 1 -- 172.21.15.83:0/727152824 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd590003aa0 con 0x7fd5a0100c70 2024-04-24T08:53:24.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.419+0000 7fd59fe00640 1 -- 172.21.15.83:0/727152824 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5a019b2a0 con 0x7fd5a0100c70 2024-04-24T08:53:24.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.419+0000 7fd597400640 1 -- 172.21.15.83:0/727152824 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd590035030 con 0x7fd5a0100c70 2024-04-24T08:53:24.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.419+0000 7fd59fe00640 1 -- 172.21.15.83:0/727152824 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5a01a7520 con 0x7fd5a0100c70 2024-04-24T08:53:24.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.423+0000 7fd5a8966740 1 Processor -- start 2024-04-24T08:53:24.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.423+0000 7fd597400640 1 -- 172.21.15.83:0/727152824 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd590042070 con 0x7fd5a0100c70 2024-04-24T08:53:24.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.423+0000 7fd5a8966740 1 -- start start 2024-04-24T08:53:24.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.423+0000 7fd5a8966740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557e0ffada30 0x557e0ffade10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:24.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.423+0000 7fd5a8966740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557e0ffae350 0x557e0ffb84e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:24.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.423+0000 7fd5a8966740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557e0ffb8a20 0x557e0ffbae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:24.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.423+0000 7fd5a8966740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x557e0ff131f0 con 0x557e0ffae350 2024-04-24T08:53:24.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.423+0000 7fd5a8966740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x557e0feb9350 con 0x557e0ffb8a20 2024-04-24T08:53:24.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.423+0000 7fd5a8966740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x557e0ff0f1b0 con 0x557e0ffada30 2024-04-24T08:53:24.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.423+0000 7fd59ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557e0ffae350 0x557e0ffb84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:24.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.423+0000 7fd59ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557e0ffae350 0x557e0ffb84e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50436/0 (socket says 172.21.15.83:50436) 2024-04-24T08:53:24.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.423+0000 7fd59ea00640 1 -- 172.21.15.83:0/1470356604 learned_addr learned my addr 172.21.15.83:0/1470356604 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:53:24.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.424+0000 7fd59f400640 1 --2- 172.21.15.83:0/1470356604 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557e0ffada30 0x557e0ffade10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:24.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.424+0000 7fd59e000640 1 --2- 172.21.15.83:0/1470356604 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557e0ffb8a20 0x557e0ffbae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:24.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.424+0000 7fd59ea00640 1 -- 172.21.15.83:0/1470356604 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557e0ffada30 msgr2=0x557e0ffade10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:24.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.424+0000 7fd59ea00640 1 --2- 172.21.15.83:0/1470356604 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557e0ffada30 0x557e0ffade10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:24.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.424+0000 7fd59ea00640 1 -- 172.21.15.83:0/1470356604 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557e0ffb8a20 msgr2=0x557e0ffbae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:24.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.424+0000 7fd59ea00640 1 --2- 172.21.15.83:0/1470356604 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557e0ffb8a20 0x557e0ffbae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:24.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.424+0000 7fd59ea00640 1 -- 172.21.15.83:0/1470356604 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e0ff0df50 con 0x557e0ffae350 2024-04-24T08:53:24.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.424+0000 7fd597400640 1 --2- 172.21.15.83:0/727152824 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fd5840a3d20 0x7fd5840a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:24.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.424+0000 7fd59ea00640 1 --2- 172.21.15.83:0/1470356604 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557e0ffae350 0x557e0ffb84e0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fd5900ee910 tx=0x7fd5900f7cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3db05f2b95afc94c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:24.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.425+0000 7fd597400640 1 -- 172.21.15.83:0/727152824 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7fd5900f16b0 con 0x7fd5a0100c70 2024-04-24T08:53:24.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.425+0000 7fd58be00640 1 -- 172.21.15.83:0/1470356604 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd590100070 con 0x557e0ffae350 2024-04-24T08:53:24.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.425+0000 7fd59f400640 1 --2- 172.21.15.83:0/727152824 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fd5840a3d20 0x7fd5840a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:24.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.425+0000 7fd58be00640 1 -- 172.21.15.83:0/1470356604 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd5900bd920 con 0x557e0ffae350 2024-04-24T08:53:24.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.425+0000 7fd58be00640 1 -- 172.21.15.83:0/1470356604 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5900bcd60 con 0x557e0ffae350 2024-04-24T08:53:24.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.426+0000 7fd5a8966740 1 -- 172.21.15.83:0/1470356604 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557e0ffae350 msgr2=0x557e0ffb84e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:24.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.426+0000 7fd5a8966740 1 --2- 172.21.15.83:0/1470356604 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557e0ffae350 0x557e0ffb84e0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fd5900ee910 tx=0x7fd5900f7cb0 comp rx=0 tx=0).stop 2024-04-24T08:53:24.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.426+0000 7fd59f400640 1 --2- 172.21.15.83:0/727152824 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fd5840a3d20 0x7fd5840a61e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd5a006b6b0 tx=0x7fd598007740 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:24.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.426+0000 7fd5a8966740 1 -- 172.21.15.83:0/1470356604 shutdown_connections 2024-04-24T08:53:24.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.426+0000 7fd5a8966740 1 --2- 172.21.15.83:0/1470356604 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557e0ffb8a20 0x557e0ffbae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:24.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.426+0000 7fd5a8966740 1 --2- 172.21.15.83:0/1470356604 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557e0ffae350 0x557e0ffb84e0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:24.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.426+0000 7fd5a8966740 1 --2- 172.21.15.83:0/1470356604 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557e0ffada30 0x557e0ffade10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:24.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.426+0000 7fd5a8966740 1 -- 172.21.15.83:0/1470356604 >> 172.21.15.83:0/1470356604 conn(0x557e0ffa9d50 msgr2=0x557e0ffaa130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:24.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.426+0000 7fd5a8966740 1 -- 172.21.15.83:0/1470356604 shutdown_connections 2024-04-24T08:53:24.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.426+0000 7fd5a8966740 1 -- 172.21.15.83:0/1470356604 wait complete. 2024-04-24T08:53:24.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.426+0000 7fd5a8966740 1 -- start start 2024-04-24T08:53:24.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.426+0000 7fd5a8966740 1 Processor -- start 2024-04-24T08:53:24.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.427+0000 7fd5a8966740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557e0ffada30 0x557e0ffb59d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:24.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.427+0000 7fd5a8966740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557e0ffae350 0x557e0ffb5f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:24.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.427+0000 7fd5a8966740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557e0ffb8a20 0x557e0ffb6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:24.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.427+0000 7fd5a8966740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x557e0ffbb5d0 con 0x557e0ffb8a20 2024-04-24T08:53:24.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.427+0000 7fd5a8966740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x557e0fecc690 con 0x557e0ffada30 2024-04-24T08:53:24.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.427+0000 7fd5a8966740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x557e0ffbb750 con 0x557e0ffae350 2024-04-24T08:53:24.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.427+0000 7fd5a8966740 10 client.0 _ll_register_callbacks cb 0x557e0ff08650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:53:24.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.427+0000 7fd59f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557e0ffb8a20 0x557e0ffb6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:24.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.427+0000 7fd59f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557e0ffb8a20 0x557e0ffb6450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50446/0 (socket says 172.21.15.83:50446) 2024-04-24T08:53:24.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.427+0000 7fd59f400640 1 -- 172.21.15.83:0/4141508921 learned_addr learned my addr 172.21.15.83:0/4141508921 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:53:24.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.428+0000 7fd59ea00640 1 --2- 172.21.15.83:0/4141508921 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557e0ffada30 0x557e0ffb59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:24.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.428+0000 7fd59f400640 1 -- 172.21.15.83:0/4141508921 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557e0ffae350 msgr2=0x557e0ffb5f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:24.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.428+0000 7fd59f400640 1 --2- 172.21.15.83:0/4141508921 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557e0ffae350 0x557e0ffb5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:24.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.428+0000 7fd59f400640 1 -- 172.21.15.83:0/4141508921 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557e0ffada30 msgr2=0x557e0ffb59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:24.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.428+0000 7fd59f400640 1 --2- 172.21.15.83:0/4141508921 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557e0ffada30 0x557e0ffb59d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:24.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.428+0000 7fd59f400640 1 -- 172.21.15.83:0/4141508921 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e0fecb920 con 0x557e0ffb8a20 2024-04-24T08:53:24.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.428+0000 7fd59f400640 1 --2- 172.21.15.83:0/4141508921 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557e0ffb8a20 0x557e0ffb6450 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fd598010f10 tx=0x7fd598004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3acfb2b5c87d8a3c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:24.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.428+0000 7fd588c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:53:24.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.428+0000 7fd588c00640 1 -- 172.21.15.83:0/4141508921 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd59801bea0 con 0x557e0ffb8a20 2024-04-24T08:53:24.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.428+0000 7fd588c00640 1 -- 172.21.15.83:0/4141508921 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7fd598026680 con 0x557e0ffb8a20 2024-04-24T08:53:24.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.428+0000 7fd5a8966740 1 -- 172.21.15.83:0/4141508921 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5900bd920 con 0x557e0ffb8a20 2024-04-24T08:53:24.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.429+0000 7fd588c00640 1 -- 172.21.15.83:0/4141508921 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd59802f050 con 0x557e0ffb8a20 2024-04-24T08:53:24.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.429+0000 7fd5a8966740 10 client.16281 fetch_fsmap learned FSMap version 28 2024-04-24T08:53:24.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.429+0000 7fd5a8966740 1 -- 172.21.15.83:0/4141508921 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557e0fcef7d0 con 0x557e0ffb8a20 2024-04-24T08:53:24.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.429+0000 7fd588c00640 1 -- 172.21.15.83:0/4141508921 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd5900bd920 con 0x557e0ffb8a20 2024-04-24T08:53:24.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.429+0000 7fd588c00640 1 -- 172.21.15.83:0/4141508921 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd59801c570 con 0x557e0ffb8a20 2024-04-24T08:53:24.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.429+0000 7fd5a8966740 10 client.16281 fetch_fsmap finished waiting for FSMap version 28 2024-04-24T08:53:24.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.429+0000 7fd5a8966740 10 client.16281 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:53:24.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.429+0000 7fd5a8966740 20 client.16281 populate_metadata read hostname 'smithi083' 2024-04-24T08:53:24.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.429+0000 7fd5a8966740 1 --2- 172.21.15.83:0/4141508921 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x557e0ffc8ba0 0x557e0ffcb000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:24.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.429+0000 7fd5a8966740 4 client.16281 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:53:24.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.429+0000 7fd5a8966740 1 -- 172.21.15.83:0/4141508921 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557e0ff20c10 con 0x557e0ffc8ba0 2024-04-24T08:53:24.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.430+0000 7fd59ea00640 1 --2- 172.21.15.83:0/4141508921 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x557e0ffc8ba0 0x557e0ffcb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:24.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.431+0000 7fd59ea00640 1 --2- 172.21.15.83:0/4141508921 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x557e0ffc8ba0 0x557e0ffcb000 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e23ab5f5f018f3a7 server_cookie=33987d9ef25d652f in_seq=0 out_seq=0 2024-04-24T08:53:24.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.431+0000 7fd588c00640 10 client.16281 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:53:24.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.433+0000 7fd588c00640 1 -- 172.21.15.83:0/4141508921 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557e0ff20c10 con 0x557e0ffc8ba0 2024-04-24T08:53:24.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.433+0000 7fd588c00640 10 client.16281 handle_command_reply: tid=0 2024-04-24T08:53:24.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.433+0000 7fd5a8966740 1 client.16281 shutdown 2024-04-24T08:53:24.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.433+0000 7fd5a8966740 1 -- 172.21.15.83:0/4141508921 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557e0ffb8a20 msgr2=0x557e0ffb6450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:24.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.433+0000 7fd5a8966740 1 --2- 172.21.15.83:0/4141508921 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557e0ffb8a20 0x557e0ffb6450 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fd598010f10 tx=0x7fd598004b50 comp rx=0 tx=0).stop 2024-04-24T08:53:24.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.433+0000 7fd5a8966740 1 -- 172.21.15.83:0/4141508921 shutdown_connections 2024-04-24T08:53:24.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.433+0000 7fd5a8966740 1 --2- 172.21.15.83:0/4141508921 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x557e0ffc8ba0 0x557e0ffcb000 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:24.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.433+0000 7fd5a8966740 1 --2- 172.21.15.83:0/4141508921 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557e0ffb8a20 0x557e0ffb6450 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:24.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.433+0000 7fd5a8966740 1 --2- 172.21.15.83:0/4141508921 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557e0ffae350 0x557e0ffb5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:24.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.433+0000 7fd588c00640 0 client.16281 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:53:24.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.433+0000 7fd5a8966740 1 --2- 172.21.15.83:0/4141508921 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557e0ffada30 0x557e0ffb59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:24.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.433+0000 7fd5a8966740 1 -- 172.21.15.83:0/4141508921 >> 172.21.15.83:0/4141508921 conn(0x557e0ffa9d50 msgr2=0x557e0ffafb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:24.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.434+0000 7fd5a8966740 1 -- 172.21.15.83:0/4141508921 shutdown_connections 2024-04-24T08:53:24.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.434+0000 7fd5a8966740 1 -- 172.21.15.83:0/4141508921 wait complete. 2024-04-24T08:53:24.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.434+0000 7fd5a8966740 20 client.16281 trim_cache size 0 max 16384 2024-04-24T08:53:24.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.452+0000 7fd5a8966740 1 Processor -- start 2024-04-24T08:53:24.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.452+0000 7fd5a8966740 1 -- start start 2024-04-24T08:53:24.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.453+0000 7fd5a8966740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557e0ffa9d50 0x557e0ffb0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:24.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.453+0000 7fd5a8966740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557e0ffada30 0x557e0ffc42d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:24.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.453+0000 7fd5a8966740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557e0ffae350 0x557e0ffd08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:24.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.453+0000 7fd5a8966740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a01b16d0 con 0x557e0ffa9d50 2024-04-24T08:53:24.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.453+0000 7fd5a8966740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a01b3210 con 0x557e0ffae350 2024-04-24T08:53:24.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.453+0000 7fd5a8966740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a01b1da0 con 0x557e0ffada30 2024-04-24T08:53:24.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.453+0000 7fd59ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557e0ffa9d50 0x557e0ffb0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:24.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.453+0000 7fd59ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557e0ffa9d50 0x557e0ffb0190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50448/0 (socket says 172.21.15.83:50448) 2024-04-24T08:53:24.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.453+0000 7fd59ea00640 1 -- 172.21.15.83:0/1199467441 learned_addr learned my addr 172.21.15.83:0/1199467441 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:53:24.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.453+0000 7fd59ea00640 1 -- 172.21.15.83:0/1199467441 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557e0ffada30 msgr2=0x557e0ffc42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:24.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.453+0000 7fd59ea00640 1 --2- 172.21.15.83:0/1199467441 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557e0ffada30 0x557e0ffc42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:24.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.453+0000 7fd59ea00640 1 -- 172.21.15.83:0/1199467441 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557e0ffae350 msgr2=0x557e0ffd08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:24.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.453+0000 7fd59ea00640 1 --2- 172.21.15.83:0/1199467441 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557e0ffae350 0x557e0ffd08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:24.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.453+0000 7fd59ea00640 1 -- 172.21.15.83:0/1199467441 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e0fc253f0 con 0x557e0ffa9d50 2024-04-24T08:53:24.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.454+0000 7fd59ea00640 1 --2- 172.21.15.83:0/1199467441 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557e0ffa9d50 0x557e0ffb0190 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fd5900bcf20 tx=0x7fd59002a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbc1a2fadc6361ee server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:24.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.454+0000 7fd58b400640 1 -- 172.21.15.83:0/1199467441 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd590049040 con 0x557e0ffa9d50 2024-04-24T08:53:24.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.454+0000 7fd58b400640 1 -- 172.21.15.83:0/1199467441 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd5900bd920 con 0x557e0ffa9d50 2024-04-24T08:53:24.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.454+0000 7fd58b400640 1 -- 172.21.15.83:0/1199467441 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd590100040 con 0x557e0ffa9d50 2024-04-24T08:53:24.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.455+0000 7fd5a8966740 1 -- 172.21.15.83:0/1199467441 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557e0ffa9d50 msgr2=0x557e0ffb0190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:24.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.455+0000 7fd5a8966740 1 --2- 172.21.15.83:0/1199467441 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557e0ffa9d50 0x557e0ffb0190 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fd5900bcf20 tx=0x7fd59002a420 comp rx=0 tx=0).stop 2024-04-24T08:53:24.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.455+0000 7fd5a8966740 1 -- 172.21.15.83:0/1199467441 shutdown_connections 2024-04-24T08:53:24.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.455+0000 7fd5a8966740 1 --2- 172.21.15.83:0/1199467441 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557e0ffae350 0x557e0ffd08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:24.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.455+0000 7fd5a8966740 1 --2- 172.21.15.83:0/1199467441 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557e0ffada30 0x557e0ffc42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:24.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.455+0000 7fd5a8966740 1 --2- 172.21.15.83:0/1199467441 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557e0ffa9d50 0x557e0ffb0190 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:24.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.455+0000 7fd5a8966740 1 -- 172.21.15.83:0/1199467441 >> 172.21.15.83:0/1199467441 conn(0x557e0ffb4610 msgr2=0x557e0ffa8fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:24.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.455+0000 7fd5a8966740 1 -- 172.21.15.83:0/1199467441 shutdown_connections 2024-04-24T08:53:24.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.455+0000 7fd5a8966740 1 -- 172.21.15.83:0/1199467441 wait complete. 2024-04-24T08:53:24.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.455+0000 7fd5a8966740 1 -- start start 2024-04-24T08:53:24.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.456+0000 7fd5a8966740 1 Processor -- start 2024-04-24T08:53:24.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.456+0000 7fd5a8966740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557e0ffa9d50 0x557e0ffbc360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:24.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.456+0000 7fd5a8966740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557e0ffada30 0x557e0ffbc8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:24.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.456+0000 7fd5a8966740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557e0ffae350 0x557e0ffbcde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:24.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.456+0000 7fd5a8966740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a01b2b40 con 0x557e0ffa9d50 2024-04-24T08:53:24.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.456+0000 7fd5a8966740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a01b2470 con 0x557e0ffada30 2024-04-24T08:53:24.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.456+0000 7fd5a8966740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a01af6b0 con 0x557e0ffae350 2024-04-24T08:53:24.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.457+0000 7fd5a8966740 10 client.0 _ll_register_callbacks cb 0x7fd598010b60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:53:24.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.457+0000 7fd59ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557e0ffa9d50 0x557e0ffbc360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:24.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.457+0000 7fd59ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557e0ffa9d50 0x557e0ffbc360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50462/0 (socket says 172.21.15.83:50462) 2024-04-24T08:53:24.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.457+0000 7fd59ea00640 1 -- 172.21.15.83:0/4048387166 learned_addr learned my addr 172.21.15.83:0/4048387166 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:53:24.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.457+0000 7fd59ea00640 1 -- 172.21.15.83:0/4048387166 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557e0ffae350 msgr2=0x557e0ffbcde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:24.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.457+0000 7fd59ea00640 1 --2- 172.21.15.83:0/4048387166 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557e0ffae350 0x557e0ffbcde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:24.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.457+0000 7fd59ea00640 1 -- 172.21.15.83:0/4048387166 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557e0ffada30 msgr2=0x557e0ffbc8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:24.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.457+0000 7fd59ea00640 1 --2- 172.21.15.83:0/4048387166 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557e0ffada30 0x557e0ffbc8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:24.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.457+0000 7fd59ea00640 1 -- 172.21.15.83:0/4048387166 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e0fbe50f0 con 0x557e0ffa9d50 2024-04-24T08:53:24.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.457+0000 7fd59ea00640 1 --2- 172.21.15.83:0/4048387166 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557e0ffa9d50 0x557e0ffbc360 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fd5900f7b40 tx=0x7fd5900f9700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab389c69f66a3176 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:24.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.457+0000 7fd588c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:53:24.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.457+0000 7fd588c00640 1 -- 172.21.15.83:0/4048387166 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5900d5030 con 0x557e0ffa9d50 2024-04-24T08:53:24.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.458+0000 7fd588c00640 1 -- 172.21.15.83:0/4048387166 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7fd5900bb2c0 con 0x557e0ffa9d50 2024-04-24T08:53:24.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.458+0000 7fd5a8966740 1 -- 172.21.15.83:0/4048387166 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5900bd920 con 0x557e0ffa9d50 2024-04-24T08:53:24.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.458+0000 7fd588c00640 1 -- 172.21.15.83:0/4048387166 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd590049070 con 0x557e0ffa9d50 2024-04-24T08:53:24.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.458+0000 7fd5a8966740 10 client.16287 fetch_fsmap learned FSMap version 28 2024-04-24T08:53:24.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.458+0000 7fd5a8966740 1 -- 172.21.15.83:0/4048387166 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557e0fed8810 con 0x557e0ffa9d50 2024-04-24T08:53:24.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.458+0000 7fd588c00640 1 -- 172.21.15.83:0/4048387166 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd5900bd920 con 0x557e0ffa9d50 2024-04-24T08:53:24.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.458+0000 7fd588c00640 1 -- 172.21.15.83:0/4048387166 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd5900f7300 con 0x557e0ffa9d50 2024-04-24T08:53:24.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.458+0000 7fd5a8966740 10 client.16287 fetch_fsmap finished waiting for FSMap version 28 2024-04-24T08:53:24.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.458+0000 7fd5a8966740 10 client.16287 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:53:24.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.458+0000 7fd5a8966740 20 client.16287 populate_metadata read hostname 'smithi083' 2024-04-24T08:53:24.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.458+0000 7fd5a8966740 1 --2- 172.21.15.83:0/4048387166 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x557e0ffb8a20 0x557e100313d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:24.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.458+0000 7fd5a8966740 4 client.16287 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T08:53:24.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.458+0000 7fd5a8966740 1 -- 172.21.15.83:0/4048387166 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557e0ff1aa80 con 0x557e0ffb8a20 2024-04-24T08:53:24.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.459+0000 7fd59e000640 1 --2- 172.21.15.83:0/4048387166 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x557e0ffb8a20 0x557e100313d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:24.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.460+0000 7fd59e000640 1 --2- 172.21.15.83:0/4048387166 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x557e0ffb8a20 0x557e100313d0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce383160dd864d0f server_cookie=e5066480f94acd59 in_seq=0 out_seq=0 2024-04-24T08:53:24.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.460+0000 7fd588c00640 10 client.16287 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:53:24.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.460+0000 7fd588c00640 1 -- 172.21.15.83:0/4048387166 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x557e0ff1aa80 con 0x557e0ffb8a20 2024-04-24T08:53:24.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.460+0000 7fd588c00640 10 client.16287 handle_command_reply: tid=0 2024-04-24T08:53:24.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.460+0000 7fd5a8966740 1 client.16287 shutdown 2024-04-24T08:53:24.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.461+0000 7fd5a8966740 1 -- 172.21.15.83:0/4048387166 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557e0ffa9d50 msgr2=0x557e0ffbc360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:24.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.461+0000 7fd5a8966740 1 --2- 172.21.15.83:0/4048387166 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557e0ffa9d50 0x557e0ffbc360 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fd5900f7b40 tx=0x7fd5900f9700 comp rx=0 tx=0).stop 2024-04-24T08:53:24.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.461+0000 7fd5a8966740 1 -- 172.21.15.83:0/4048387166 shutdown_connections 2024-04-24T08:53:24.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.461+0000 7fd5a8966740 1 --2- 172.21.15.83:0/4048387166 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x557e0ffb8a20 0x557e100313d0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:24.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.461+0000 7fd5a8966740 1 --2- 172.21.15.83:0/4048387166 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557e0ffae350 0x557e0ffbcde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:24.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.461+0000 7fd588c00640 0 client.16287 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:53:24.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.461+0000 7fd5a8966740 1 --2- 172.21.15.83:0/4048387166 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557e0ffada30 0x557e0ffbc8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:24.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.461+0000 7fd5a8966740 1 --2- 172.21.15.83:0/4048387166 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557e0ffa9d50 0x557e0ffbc360 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:24.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.461+0000 7fd5a8966740 1 -- 172.21.15.83:0/4048387166 >> 172.21.15.83:0/4048387166 conn(0x557e0ffb4610 msgr2=0x557e0ffa8fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:24.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.461+0000 7fd5a8966740 1 -- 172.21.15.83:0/4048387166 shutdown_connections 2024-04-24T08:53:24.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.461+0000 7fd5a8966740 1 -- 172.21.15.83:0/4048387166 wait complete. 2024-04-24T08:53:24.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.461+0000 7fd5a8966740 20 client.16287 trim_cache size 0 max 16384 2024-04-24T08:53:24.464 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T08:53:24.464 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-04-24T08:53:24.464 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-04-24T08:53:24.464 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T08:53:24.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.462+0000 7fd59fe00640 1 -- 172.21.15.83:0/727152824 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fd5840a3d20 msgr2=0x7fd5840a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:24.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.462+0000 7fd59fe00640 1 --2- 172.21.15.83:0/727152824 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fd5840a3d20 0x7fd5840a61e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd5a006b6b0 tx=0x7fd598007740 comp rx=0 tx=0).stop 2024-04-24T08:53:24.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.462+0000 7fd59fe00640 1 -- 172.21.15.83:0/727152824 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd5a0100c70 msgr2=0x7fd5a01a06d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:24.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.462+0000 7fd59fe00640 1 --2- 172.21.15.83:0/727152824 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd5a0100c70 0x7fd5a01a06d0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fd590005940 tx=0x7fd590004430 comp rx=0 tx=0).stop 2024-04-24T08:53:24.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.462+0000 7fd59fe00640 1 -- 172.21.15.83:0/727152824 shutdown_connections 2024-04-24T08:53:24.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.462+0000 7fd59fe00640 1 --2- 172.21.15.83:0/727152824 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fd5840a3d20 0x7fd5840a61e0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:24.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.462+0000 7fd59fe00640 1 --2- 172.21.15.83:0/727152824 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd5a010e820 0x7fd5a019a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:24.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.462+0000 7fd59fe00640 1 --2- 172.21.15.83:0/727152824 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd5a0101620 0x7fd5a01a0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:24.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.462+0000 7fd59fe00640 1 --2- 172.21.15.83:0/727152824 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd5a0100c70 0x7fd5a01a06d0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:24.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.462+0000 7fd59fe00640 1 -- 172.21.15.83:0/727152824 >> 172.21.15.83:0/727152824 conn(0x7fd5a00fc880 msgr2=0x7fd5a00fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:24.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.462+0000 7fd59fe00640 1 -- 172.21.15.83:0/727152824 shutdown_connections 2024-04-24T08:53:24.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.462+0000 7fd59fe00640 1 -- 172.21.15.83:0/727152824 wait complete. 2024-04-24T08:53:24.479 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T08:53:24.479 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T08:53:24.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.660+0000 7f6bdf000640 1 -- 172.21.15.83:0/228637379 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6bd8102bf0 msgr2=0x7f6bd8106c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:24.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.660+0000 7f6bdf000640 1 --2- 172.21.15.83:0/228637379 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6bd8102bf0 0x7f6bd8106c30 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f6bc0009900 tx=0x7f6bc002a410 comp rx=0 tx=0).stop 2024-04-24T08:53:24.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.660+0000 7f6bdf000640 1 -- 172.21.15.83:0/228637379 shutdown_connections 2024-04-24T08:53:24.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.660+0000 7f6bdf000640 1 --2- 172.21.15.83:0/228637379 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6bd8107310 0x7f6bd810eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:24.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.660+0000 7f6bdf000640 1 --2- 172.21.15.83:0/228637379 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6bd8102bf0 0x7f6bd8106c30 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:24.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.660+0000 7f6bdf000640 1 --2- 172.21.15.83:0/228637379 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6bd8102240 0x7f6bd8102620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:24.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.660+0000 7f6bdf000640 1 -- 172.21.15.83:0/228637379 >> 172.21.15.83:0/228637379 conn(0x7f6bd80793c0 msgr2=0x7f6bd80797d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:24.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.660+0000 7f6bdf000640 1 -- 172.21.15.83:0/228637379 shutdown_connections 2024-04-24T08:53:24.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.660+0000 7f6bdf000640 1 -- 172.21.15.83:0/228637379 wait complete. 2024-04-24T08:53:24.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.661+0000 7f6bdf000640 1 Processor -- start 2024-04-24T08:53:24.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.661+0000 7f6bdf000640 1 -- start start 2024-04-24T08:53:24.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.662+0000 7f6bdf000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6bd8102240 0x7f6bd81a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:24.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.662+0000 7f6bdf000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6bd8107310 0x7f6bd81a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:24.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.662+0000 7f6bdf000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6bd819a860 0x7f6bd819ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:24.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.662+0000 7f6bdf000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6bd81122a0 con 0x7f6bd8102240 2024-04-24T08:53:24.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.662+0000 7f6bdf000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6bd8112120 con 0x7f6bd8107310 2024-04-24T08:53:24.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.662+0000 7f6bdf000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6bd8112420 con 0x7f6bd819a860 2024-04-24T08:53:24.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.662+0000 7f6bd7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6bd8102240 0x7f6bd81a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:24.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.662+0000 7f6bd7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6bd8102240 0x7f6bd81a06b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50478/0 (socket says 172.21.15.83:50478) 2024-04-24T08:53:24.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.662+0000 7f6bd7e00640 1 -- 172.21.15.83:0/2342997206 learned_addr learned my addr 172.21.15.83:0/2342997206 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:53:24.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.662+0000 7f6bcfe00640 1 --2- 172.21.15.83:0/2342997206 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6bd8107310 0x7f6bd81a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:24.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.662+0000 7f6bd7e00640 1 -- 172.21.15.83:0/2342997206 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6bd819a860 msgr2=0x7f6bd819ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:24.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.662+0000 7f6bd7e00640 1 --2- 172.21.15.83:0/2342997206 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6bd819a860 0x7f6bd819ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:24.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.662+0000 7f6bd7e00640 1 -- 172.21.15.83:0/2342997206 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6bd8107310 msgr2=0x7f6bd81a0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:24.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.663+0000 7f6bd7e00640 1 --2- 172.21.15.83:0/2342997206 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6bd8107310 0x7f6bd81a0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:24.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.663+0000 7f6bd7e00640 1 -- 172.21.15.83:0/2342997206 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bd819b380 con 0x7f6bd8102240 2024-04-24T08:53:24.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.663+0000 7f6bcfe00640 1 --2- 172.21.15.83:0/2342997206 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6bd8107310 0x7f6bd81a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:53:24.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.663+0000 7f6bd7e00640 1 --2- 172.21.15.83:0/2342997206 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6bd8102240 0x7f6bd81a06b0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f6bc800d770 tx=0x7f6bc800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:24.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.663+0000 7f6bd5600640 1 -- 172.21.15.83:0/2342997206 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bc8017070 con 0x7f6bd8102240 2024-04-24T08:53:24.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.663+0000 7f6bdf000640 1 -- 172.21.15.83:0/2342997206 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bd819b670 con 0x7f6bd8102240 2024-04-24T08:53:24.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.663+0000 7f6bd5600640 1 -- 172.21.15.83:0/2342997206 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6bc80054d0 con 0x7f6bd8102240 2024-04-24T08:53:24.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.663+0000 7f6bd5600640 1 -- 172.21.15.83:0/2342997206 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bc8020060 con 0x7f6bd8102240 2024-04-24T08:53:24.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.663+0000 7f6bdf000640 1 -- 172.21.15.83:0/2342997206 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bd81a7870 con 0x7f6bd8102240 2024-04-24T08:53:24.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.664+0000 7f6bdf000640 1 -- 172.21.15.83:0/2342997206 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bd8069270 con 0x7f6bd8102240 2024-04-24T08:53:24.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.666+0000 7f6bd5600640 1 -- 172.21.15.83:0/2342997206 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6bc8004c90 con 0x7f6bd8102240 2024-04-24T08:53:24.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.668+0000 7f6bd5600640 1 --2- 172.21.15.83:0/2342997206 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f6bbc0a3d20 0x7f6bbc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:24.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.668+0000 7f6bd5600640 1 -- 172.21.15.83:0/2342997206 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f6bc80d0ac0 con 0x7f6bd8102240 2024-04-24T08:53:24.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.668+0000 7f6bcfe00640 1 --2- 172.21.15.83:0/2342997206 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f6bbc0a3d20 0x7f6bbc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:24.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.669+0000 7f6bcfe00640 1 --2- 172.21.15.83:0/2342997206 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f6bbc0a3d20 0x7f6bbc0a61e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6bc002a930 tx=0x7f6bc0000bc0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:24.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.677+0000 7f6bd5600640 1 -- 172.21.15.83:0/2342997206 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6bc809cd30 con 0x7f6bd8102240 2024-04-24T08:53:24.874 INFO:tasks.workunit.client.0.smithi083.stderr:Note: switching to '1f81fb397ae98da7563d451a78c61574c8f4e6e0'. 2024-04-24T08:53:24.875 INFO:tasks.workunit.client.0.smithi083.stderr: 2024-04-24T08:53:24.875 INFO:tasks.workunit.client.0.smithi083.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-24T08:53:24.875 INFO:tasks.workunit.client.0.smithi083.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-24T08:53:24.875 INFO:tasks.workunit.client.0.smithi083.stderr:state without impacting any branches by switching back to a branch. 2024-04-24T08:53:24.875 INFO:tasks.workunit.client.0.smithi083.stderr: 2024-04-24T08:53:24.875 INFO:tasks.workunit.client.0.smithi083.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-24T08:53:24.875 INFO:tasks.workunit.client.0.smithi083.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-24T08:53:24.875 INFO:tasks.workunit.client.0.smithi083.stderr: 2024-04-24T08:53:24.875 INFO:tasks.workunit.client.0.smithi083.stderr: git switch -c 2024-04-24T08:53:24.875 INFO:tasks.workunit.client.0.smithi083.stderr: 2024-04-24T08:53:24.875 INFO:tasks.workunit.client.0.smithi083.stderr:Or undo this operation with: 2024-04-24T08:53:24.875 INFO:tasks.workunit.client.0.smithi083.stderr: 2024-04-24T08:53:24.875 INFO:tasks.workunit.client.0.smithi083.stderr: git switch - 2024-04-24T08:53:24.875 INFO:tasks.workunit.client.0.smithi083.stderr: 2024-04-24T08:53:24.875 INFO:tasks.workunit.client.0.smithi083.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-24T08:53:24.875 INFO:tasks.workunit.client.0.smithi083.stderr: 2024-04-24T08:53:24.876 INFO:tasks.workunit.client.0.smithi083.stderr:HEAD is now at 1f81fb397ae DNM: trigger builds 2024-04-24T08:53:24.888 DEBUG:teuthology.orchestra.run.smithi083:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-24T08:53:24.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.921+0000 7f6bdf000640 1 -- 172.21.15.83:0/2342997206 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6bd819bd90 con 0x7f6bd8102240 2024-04-24T08:53:24.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.922+0000 7f6bd5600640 1 -- 172.21.15.83:0/2342997206 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11944 (secure 0 0 0) 0x7f6bc809c630 con 0x7f6bd8102240 2024-04-24T08:53:24.922 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:53:24.923 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/358872257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":358872257},{"type":"v1","addr":"172.21.15.83:6835","nonce":358872257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3157070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3157070851},{"type":"v1","addr":"172.21.15.83:6837","nonce":3157070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T08:48:29.422576+0000","modified":"2024-04-24T08:51:36.355488+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34520,"mds_1":34493,"mds_2":34451,"mds_3":34427,"mds_4":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14658":{"gid":14658,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6839/4085470844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":4085470844},{"type":"v1","addr":"172.21.15.83:6839","nonce":4085470844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14676":{"gid":14676,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6841/2634754210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":2634754210},{"type":"v1","addr":"172.21.15.83:6841","nonce":2634754210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24382":{"gid":24382,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/1859921310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1859921310},{"type":"v1","addr":"172.21.15.189:6833","nonce":1859921310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/2241923197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2241923197},{"type":"v1","addr":"172.21.15.189:6835","nonce":2241923197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6837/3172311220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":3172311220},{"type":"v1","addr":"172.21.15.189:6837","nonce":3172311220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.189:6839/2548965336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":2548965336},{"type":"v1","addr":"172.21.15.189:6839","nonce":2548965336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.138:6833/1157967658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1157967658},{"type":"v1","addr":"172.21.15.138:6833","nonce":1157967658}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.138:6835/1909219758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1909219758},{"type":"v1","addr":"172.21.15.138:6835","nonce":1909219758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34493":{"gid":34493,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.138:6837/900566754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":900566754},{"type":"v1","addr":"172.21.15.138:6837","nonce":900566754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34520":{"gid":34520,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.138:6839/2236932393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2236932393},{"type":"v1","addr":"172.21.15.138:6839","nonce":2236932393}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34520,"qdb_cluster":[34451,34493,34427,34520,24454]},"id":1}]} 2024-04-24T08:53:24.923 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 28 2024-04-24T08:53:24.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.924+0000 7f6bdf000640 1 -- 172.21.15.83:0/2342997206 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f6bbc0a3d20 msgr2=0x7f6bbc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:24.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.924+0000 7f6bdf000640 1 --2- 172.21.15.83:0/2342997206 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f6bbc0a3d20 0x7f6bbc0a61e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6bc002a930 tx=0x7f6bc0000bc0 comp rx=0 tx=0).stop 2024-04-24T08:53:24.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.924+0000 7f6bdf000640 1 -- 172.21.15.83:0/2342997206 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6bd8102240 msgr2=0x7f6bd81a06b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:24.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.924+0000 7f6bdf000640 1 --2- 172.21.15.83:0/2342997206 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6bd8102240 0x7f6bd81a06b0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f6bc800d770 tx=0x7f6bc800dc30 comp rx=0 tx=0).stop 2024-04-24T08:53:24.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.924+0000 7f6bdf000640 1 -- 172.21.15.83:0/2342997206 shutdown_connections 2024-04-24T08:53:24.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.924+0000 7f6bdf000640 1 --2- 172.21.15.83:0/2342997206 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f6bbc0a3d20 0x7f6bbc0a61e0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:24.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.924+0000 7f6bdf000640 1 --2- 172.21.15.83:0/2342997206 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6bd819a860 0x7f6bd819ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:24.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.924+0000 7f6bdf000640 1 --2- 172.21.15.83:0/2342997206 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6bd8107310 0x7f6bd81a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:24.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.924+0000 7f6bdf000640 1 --2- 172.21.15.83:0/2342997206 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6bd8102240 0x7f6bd81a06b0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:24.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.924+0000 7f6bdf000640 1 -- 172.21.15.83:0/2342997206 >> 172.21.15.83:0/2342997206 conn(0x7f6bd80793c0 msgr2=0x7f6bd80fffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:24.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.925+0000 7f6bdf000640 1 -- 172.21.15.83:0/2342997206 shutdown_connections 2024-04-24T08:53:24.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:24.925+0000 7f6bdf000640 1 -- 172.21.15.83:0/2342997206 wait complete. 2024-04-24T08:53:24.943 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-24T08:53:24.948 INFO:tasks.workunit.client.0.smithi083.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-24T08:53:24.954 INFO:tasks.workunit.client.0.smithi083.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-24T08:53:24.954 INFO:tasks.workunit.client.0.smithi083.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-24T08:53:25.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.125+0000 7fccb4a00640 1 -- 172.21.15.83:0/641217469 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fccb0108580 msgr2=0x7fccb0108a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:25.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.125+0000 7fccb4a00640 1 --2- 172.21.15.83:0/641217469 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fccb0108580 0x7fccb0108a00 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fcc98009930 tx=0x7fcc9802a420 comp rx=0 tx=0).stop 2024-04-24T08:53:25.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.125+0000 7fccb4a00640 1 -- 172.21.15.83:0/641217469 shutdown_connections 2024-04-24T08:53:25.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.125+0000 7fccb4a00640 1 --2- 172.21.15.83:0/641217469 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fccb0108f40 0x7fccb01153b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.125+0000 7fccb4a00640 1 --2- 172.21.15.83:0/641217469 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fccb0108580 0x7fccb0108a00 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.125+0000 7fccb4a00640 1 --2- 172.21.15.83:0/641217469 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fccb010d500 0x7fccb010d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.125+0000 7fccb4a00640 1 -- 172.21.15.83:0/641217469 >> 172.21.15.83:0/641217469 conn(0x7fccb0077e20 msgr2=0x7fccb0106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:25.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.125+0000 7fccb4a00640 1 -- 172.21.15.83:0/641217469 shutdown_connections 2024-04-24T08:53:25.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.125+0000 7fccb4a00640 1 -- 172.21.15.83:0/641217469 wait complete. 2024-04-24T08:53:25.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.126+0000 7fccb4a00640 1 Processor -- start 2024-04-24T08:53:25.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.126+0000 7fccb4a00640 1 -- start start 2024-04-24T08:53:25.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.126+0000 7fccb4a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fccb0108580 0x7fccb01a48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:25.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.126+0000 7fccb4a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fccb0108f40 0x7fccb01a4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:25.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.126+0000 7fccb4a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fccb010d500 0x7fccb019eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:25.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.126+0000 7fccb4a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fccb0118830 con 0x7fccb010d500 2024-04-24T08:53:25.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.126+0000 7fccb4a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fccb01186b0 con 0x7fccb0108f40 2024-04-24T08:53:25.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.126+0000 7fccb4a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fccb01189b0 con 0x7fccb0108580 2024-04-24T08:53:25.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.127+0000 7fccafe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fccb010d500 0x7fccb019eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:25.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.127+0000 7fccafe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fccb010d500 0x7fccb019eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50494/0 (socket says 172.21.15.83:50494) 2024-04-24T08:53:25.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.127+0000 7fccafe00640 1 -- 172.21.15.83:0/3757276239 learned_addr learned my addr 172.21.15.83:0/3757276239 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:53:25.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.127+0000 7fcca6a00640 1 --2- 172.21.15.83:0/3757276239 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fccb0108f40 0x7fccb01a4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:25.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.127+0000 7fccaf400640 1 --2- 172.21.15.83:0/3757276239 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fccb0108580 0x7fccb01a48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:25.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.127+0000 7fccafe00640 1 -- 172.21.15.83:0/3757276239 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fccb0108580 msgr2=0x7fccb01a48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:25.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.127+0000 7fccafe00640 1 --2- 172.21.15.83:0/3757276239 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fccb0108580 0x7fccb01a48b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.127+0000 7fccafe00640 1 -- 172.21.15.83:0/3757276239 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fccb0108f40 msgr2=0x7fccb01a4df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:25.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.127+0000 7fccafe00640 1 --2- 172.21.15.83:0/3757276239 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fccb0108f40 0x7fccb01a4df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.127+0000 7fccafe00640 1 -- 172.21.15.83:0/3757276239 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccb019f230 con 0x7fccb010d500 2024-04-24T08:53:25.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.127+0000 7fccafe00640 1 --2- 172.21.15.83:0/3757276239 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fccb010d500 0x7fccb019eab0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fcca800b9e0 tx=0x7fcca800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:25.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.128+0000 7fccacc00640 1 -- 172.21.15.83:0/3757276239 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcca8019070 con 0x7fccb010d500 2024-04-24T08:53:25.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.128+0000 7fccacc00640 1 -- 172.21.15.83:0/3757276239 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcca8014440 con 0x7fccb010d500 2024-04-24T08:53:25.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.128+0000 7fccacc00640 1 -- 172.21.15.83:0/3757276239 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcca80075d0 con 0x7fccb010d500 2024-04-24T08:53:25.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.128+0000 7fccb4a00640 1 -- 172.21.15.83:0/3757276239 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccb019f520 con 0x7fccb010d500 2024-04-24T08:53:25.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.128+0000 7fccb4a00640 1 -- 172.21.15.83:0/3757276239 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccb01ab770 con 0x7fccb010d500 2024-04-24T08:53:25.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.131+0000 7fccacc00640 1 -- 172.21.15.83:0/3757276239 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fcca801c070 con 0x7fccb010d500 2024-04-24T08:53:25.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.131+0000 7fccb8f9e740 1 Processor -- start 2024-04-24T08:53:25.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.131+0000 7fccb8f9e740 1 -- start start 2024-04-24T08:53:25.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.131+0000 7fccb8f9e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55674f9a1b40 0x55674f9a1f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:25.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.132+0000 7fccb8f9e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55674f9a2460 0x55674f9ac5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:25.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.132+0000 7fccb8f9e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55674f9acb30 0x55674f9aef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:25.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.132+0000 7fccb8f9e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55674f9071d0 con 0x55674f9a2460 2024-04-24T08:53:25.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.132+0000 7fccb8f9e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55674f8ad450 con 0x55674f9a1b40 2024-04-24T08:53:25.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.132+0000 7fccb8f9e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55674f903190 con 0x55674f9acb30 2024-04-24T08:53:25.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.132+0000 7fccaf400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55674f9a2460 0x55674f9ac5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:25.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.132+0000 7fccaf400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55674f9a2460 0x55674f9ac5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50500/0 (socket says 172.21.15.83:50500) 2024-04-24T08:53:25.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.132+0000 7fccaf400640 1 -- 172.21.15.83:0/1552390517 learned_addr learned my addr 172.21.15.83:0/1552390517 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:53:25.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.132+0000 7fcca6a00640 1 --2- 172.21.15.83:0/1552390517 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55674f9a1b40 0x55674f9a1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:25.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.132+0000 7fccaf400640 1 -- 172.21.15.83:0/1552390517 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55674f9acb30 msgr2=0x55674f9aef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:25.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.132+0000 7fccaf400640 1 --2- 172.21.15.83:0/1552390517 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55674f9acb30 0x55674f9aef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.132+0000 7fccaf400640 1 -- 172.21.15.83:0/1552390517 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55674f9a1b40 msgr2=0x55674f9a1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:25.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.132+0000 7fccaf400640 1 --2- 172.21.15.83:0/1552390517 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55674f9a1b40 0x55674f9a1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.132+0000 7fccaf400640 1 -- 172.21.15.83:0/1552390517 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55674f96a310 con 0x55674f9a2460 2024-04-24T08:53:25.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.132+0000 7fcca6a00640 1 --2- 172.21.15.83:0/1552390517 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55674f9a1b40 0x55674f9a1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:53:25.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.132+0000 7fccacc00640 1 --2- 172.21.15.83:0/3757276239 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fcc940a3da0 0x7fcc940a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:25.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.133+0000 7fccaf400640 1 --2- 172.21.15.83:0/1552390517 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55674f9a2460 0x55674f9ac5f0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fcca000bb10 tx=0x7fcca000d340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd0bfb4dc7125da7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:25.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.133+0000 7fccacc00640 1 -- 172.21.15.83:0/3757276239 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7fcca80d00d0 con 0x7fccb010d500 2024-04-24T08:53:25.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.133+0000 7fcca6a00640 1 --2- 172.21.15.83:0/3757276239 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fcc940a3da0 0x7fcc940a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:25.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.133+0000 7fcc9fe00640 1 -- 172.21.15.83:0/1552390517 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcca0017420 con 0x55674f9a2460 2024-04-24T08:53:25.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.133+0000 7fcc9fe00640 1 -- 172.21.15.83:0/1552390517 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcca0017a70 con 0x55674f9a2460 2024-04-24T08:53:25.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.133+0000 7fcc9fe00640 1 -- 172.21.15.83:0/1552390517 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcca000f9c0 con 0x55674f9a2460 2024-04-24T08:53:25.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.134+0000 7fccb8f9e740 1 -- 172.21.15.83:0/1552390517 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55674f9a2460 msgr2=0x55674f9ac5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:25.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.134+0000 7fccb8f9e740 1 --2- 172.21.15.83:0/1552390517 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55674f9a2460 0x55674f9ac5f0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fcca000bb10 tx=0x7fcca000d340 comp rx=0 tx=0).stop 2024-04-24T08:53:25.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.134+0000 7fcca6a00640 1 --2- 172.21.15.83:0/3757276239 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fcc940a3da0 0x7fcc940a6260 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fcc98006010 tx=0x7fcc98000bc0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:25.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.134+0000 7fccb8f9e740 1 -- 172.21.15.83:0/1552390517 shutdown_connections 2024-04-24T08:53:25.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.134+0000 7fccb8f9e740 1 --2- 172.21.15.83:0/1552390517 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55674f9acb30 0x55674f9aef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.134+0000 7fccb8f9e740 1 --2- 172.21.15.83:0/1552390517 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55674f9a2460 0x55674f9ac5f0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.134+0000 7fccb8f9e740 1 --2- 172.21.15.83:0/1552390517 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55674f9a1b40 0x55674f9a1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.134+0000 7fccb8f9e740 1 -- 172.21.15.83:0/1552390517 >> 172.21.15.83:0/1552390517 conn(0x55674f99de60 msgr2=0x55674f99e240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:25.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.134+0000 7fccb8f9e740 1 -- 172.21.15.83:0/1552390517 shutdown_connections 2024-04-24T08:53:25.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.134+0000 7fccb8f9e740 1 -- 172.21.15.83:0/1552390517 wait complete. 2024-04-24T08:53:25.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.134+0000 7fccb8f9e740 1 -- start start 2024-04-24T08:53:25.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.135+0000 7fccb8f9e740 1 Processor -- start 2024-04-24T08:53:25.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.135+0000 7fccb8f9e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55674f9a1b40 0x55674f9ac4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:25.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.135+0000 7fccb8f9e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55674f9a2460 0x55674f9a9790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:25.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.135+0000 7fccb8f9e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55674f9acb30 0x55674f9a9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:25.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.135+0000 7fccb8f9e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55674f9af9c0 con 0x55674f9acb30 2024-04-24T08:53:25.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.135+0000 7fccb8f9e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55674f8c0810 con 0x55674f9a2460 2024-04-24T08:53:25.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.135+0000 7fccb8f9e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55674f9afb40 con 0x55674f9a1b40 2024-04-24T08:53:25.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.135+0000 7fccb8f9e740 10 client.0 _ll_register_callbacks cb 0x55674f88e140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:53:25.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.135+0000 7fccafe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55674f9acb30 0x55674f9a9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:25.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.135+0000 7fccafe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55674f9acb30 0x55674f9a9cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50514/0 (socket says 172.21.15.83:50514) 2024-04-24T08:53:25.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.135+0000 7fccafe00640 1 -- 172.21.15.83:0/460089750 learned_addr learned my addr 172.21.15.83:0/460089750 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:53:25.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.136+0000 7fcca6a00640 1 --2- 172.21.15.83:0/460089750 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55674f9a2460 0x55674f9a9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:25.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.136+0000 7fccaf400640 1 --2- 172.21.15.83:0/460089750 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55674f9a1b40 0x55674f9ac4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:25.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.136+0000 7fccafe00640 1 -- 172.21.15.83:0/460089750 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55674f9a1b40 msgr2=0x55674f9ac4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:25.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.136+0000 7fccafe00640 1 --2- 172.21.15.83:0/460089750 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55674f9a1b40 0x55674f9ac4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.136+0000 7fccafe00640 1 -- 172.21.15.83:0/460089750 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55674f9a2460 msgr2=0x55674f9a9790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:25.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.136+0000 7fccafe00640 1 --2- 172.21.15.83:0/460089750 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55674f9a2460 0x55674f9a9790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.136+0000 7fccafe00640 1 -- 172.21.15.83:0/460089750 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55674f8d13f0 con 0x55674f9acb30 2024-04-24T08:53:25.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.136+0000 7fccafe00640 1 --2- 172.21.15.83:0/460089750 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55674f9acb30 0x55674f9a9cf0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fcca8098680 tx=0x7fcca8014aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fca7344f181da2c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:25.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.136+0000 7fcc9cc00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:53:25.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.136+0000 7fcc9cc00640 1 -- 172.21.15.83:0/460089750 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcca80a0060 con 0x55674f9acb30 2024-04-24T08:53:25.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.136+0000 7fcc9cc00640 1 -- 172.21.15.83:0/460089750 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7fcca80bc0b0 con 0x55674f9acb30 2024-04-24T08:53:25.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.136+0000 7fccb8f9e740 1 -- 172.21.15.83:0/460089750 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcca0017a70 con 0x55674f9acb30 2024-04-24T08:53:25.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.136+0000 7fcc9cc00640 1 -- 172.21.15.83:0/460089750 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fcca80a4920 con 0x55674f9acb30 2024-04-24T08:53:25.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.137+0000 7fccb8f9e740 10 client.16305 fetch_fsmap learned FSMap version 28 2024-04-24T08:53:25.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.137+0000 7fccb8f9e740 1 -- 172.21.15.83:0/460089750 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55674f6193f0 con 0x55674f9acb30 2024-04-24T08:53:25.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.137+0000 7fcc9cc00640 1 -- 172.21.15.83:0/460089750 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcca0017a70 con 0x55674f9acb30 2024-04-24T08:53:25.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.137+0000 7fcc9cc00640 1 -- 172.21.15.83:0/460089750 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fcca80cdb40 con 0x55674f9acb30 2024-04-24T08:53:25.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.137+0000 7fccb8f9e740 10 client.16305 fetch_fsmap finished waiting for FSMap version 28 2024-04-24T08:53:25.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.137+0000 7fccb8f9e740 10 client.16305 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-24T08:53:25.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.137+0000 7fccb8f9e740 20 client.16305 populate_metadata read hostname 'smithi083' 2024-04-24T08:53:25.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.137+0000 7fccb8f9e740 1 --2- 172.21.15.83:0/460089750 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55674f9bcb10 0x55674f9bef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:25.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.137+0000 7fccb8f9e740 4 client.16305 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:53:25.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.137+0000 7fccb8f9e740 1 -- 172.21.15.83:0/460089750 --> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55674f914bf0 con 0x55674f9bcb10 2024-04-24T08:53:25.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.138+0000 7fccaf400640 1 --2- 172.21.15.83:0/460089750 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55674f9bcb10 0x55674f9bef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:25.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.139+0000 7fccaf400640 1 --2- 172.21.15.83:0/460089750 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55674f9bcb10 0x55674f9bef00 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d82708bef4c5b26b server_cookie=c92cf20c26dcf71 in_seq=0 out_seq=0 2024-04-24T08:53:25.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.139+0000 7fcc9cc00640 10 client.16305 ms_handle_connect on v2:172.21.15.189:6838/2548965336 2024-04-24T08:53:25.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.141+0000 7fcc9cc00640 1 -- 172.21.15.83:0/460089750 <== mds.4 v2:172.21.15.189:6838/2548965336 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55674f914bf0 con 0x55674f9bcb10 2024-04-24T08:53:25.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.141+0000 7fcc9cc00640 10 client.16305 handle_command_reply: tid=0 2024-04-24T08:53:25.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.141+0000 7fccb8f9e740 1 client.16305 shutdown 2024-04-24T08:53:25.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.141+0000 7fccb8f9e740 1 -- 172.21.15.83:0/460089750 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55674f9acb30 msgr2=0x55674f9a9cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:25.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.141+0000 7fccb8f9e740 1 --2- 172.21.15.83:0/460089750 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55674f9acb30 0x55674f9a9cf0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fcca8098680 tx=0x7fcca8014aa0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.141+0000 7fccb8f9e740 1 -- 172.21.15.83:0/460089750 shutdown_connections 2024-04-24T08:53:25.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.141+0000 7fccb8f9e740 1 --2- 172.21.15.83:0/460089750 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55674f9bcb10 0x55674f9bef00 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.141+0000 7fccb8f9e740 1 --2- 172.21.15.83:0/460089750 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55674f9acb30 0x55674f9a9cf0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.141+0000 7fccb8f9e740 1 --2- 172.21.15.83:0/460089750 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55674f9a2460 0x55674f9a9790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.141+0000 7fcc9cc00640 0 client.16305 ms_handle_reset on v2:172.21.15.189:6838/2548965336 2024-04-24T08:53:25.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.141+0000 7fccb8f9e740 1 --2- 172.21.15.83:0/460089750 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55674f9a1b40 0x55674f9ac4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.141+0000 7fccb8f9e740 1 -- 172.21.15.83:0/460089750 >> 172.21.15.83:0/460089750 conn(0x55674f99de60 msgr2=0x55674f9a3b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:25.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.141+0000 7fccb8f9e740 1 -- 172.21.15.83:0/460089750 shutdown_connections 2024-04-24T08:53:25.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.141+0000 7fccb8f9e740 1 -- 172.21.15.83:0/460089750 wait complete. 2024-04-24T08:53:25.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.141+0000 7fccb8f9e740 20 client.16305 trim_cache size 0 max 16384 2024-04-24T08:53:25.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.155+0000 7fccb8f9e740 1 Processor -- start 2024-04-24T08:53:25.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.155+0000 7fccb8f9e740 1 -- start start 2024-04-24T08:53:25.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.155+0000 7fccb8f9e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55674f9a1b40 0x55674f99cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:25.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.155+0000 7fccb8f9e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55674f9a2460 0x55674f9abfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:25.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.155+0000 7fccb8f9e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55674f9acb30 0x55674f9c4780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:25.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.155+0000 7fccb8f9e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fccb01b57d0 con 0x55674f9a1b40 2024-04-24T08:53:25.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.155+0000 7fccb8f9e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fccb01b7310 con 0x55674f9acb30 2024-04-24T08:53:25.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.155+0000 7fccb8f9e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fccb01b5ea0 con 0x55674f9a2460 2024-04-24T08:53:25.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.156+0000 7fccaf400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55674f9a1b40 0x55674f99cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:25.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.156+0000 7fccaf400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55674f9a1b40 0x55674f99cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50526/0 (socket says 172.21.15.83:50526) 2024-04-24T08:53:25.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.156+0000 7fccaf400640 1 -- 172.21.15.83:0/593074103 learned_addr learned my addr 172.21.15.83:0/593074103 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:53:25.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.156+0000 7fccaf400640 1 -- 172.21.15.83:0/593074103 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55674f9a2460 msgr2=0x55674f9abfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:25.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.156+0000 7fccaf400640 1 --2- 172.21.15.83:0/593074103 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55674f9a2460 0x55674f9abfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.156+0000 7fccaf400640 1 -- 172.21.15.83:0/593074103 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55674f9acb30 msgr2=0x55674f9c4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:25.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.156+0000 7fccaf400640 1 --2- 172.21.15.83:0/593074103 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55674f9acb30 0x55674f9c4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.156+0000 7fccaf400640 1 -- 172.21.15.83:0/593074103 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55674f6e37d0 con 0x55674f9a1b40 2024-04-24T08:53:25.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.156+0000 7fccaf400640 1 --2- 172.21.15.83:0/593074103 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55674f9a1b40 0x55674f99cdd0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fcca000b630 tx=0x7fcca000d340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afe1de4e6c0f2fcb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:25.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.157+0000 7fcc9f400640 1 -- 172.21.15.83:0/593074103 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcca0028d50 con 0x55674f9a1b40 2024-04-24T08:53:25.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.157+0000 7fcc9f400640 1 -- 172.21.15.83:0/593074103 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcca002d070 con 0x55674f9a1b40 2024-04-24T08:53:25.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.157+0000 7fcc9f400640 1 -- 172.21.15.83:0/593074103 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcca000fc80 con 0x55674f9a1b40 2024-04-24T08:53:25.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.157+0000 7fccb8f9e740 1 -- 172.21.15.83:0/593074103 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55674f9a1b40 msgr2=0x55674f99cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:25.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.157+0000 7fccb8f9e740 1 --2- 172.21.15.83:0/593074103 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55674f9a1b40 0x55674f99cdd0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fcca000b630 tx=0x7fcca000d340 comp rx=0 tx=0).stop 2024-04-24T08:53:25.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.157+0000 7fccb8f9e740 1 -- 172.21.15.83:0/593074103 shutdown_connections 2024-04-24T08:53:25.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.157+0000 7fccb8f9e740 1 --2- 172.21.15.83:0/593074103 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55674f9acb30 0x55674f9c4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.157+0000 7fccb8f9e740 1 --2- 172.21.15.83:0/593074103 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55674f9a2460 0x55674f9abfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.157+0000 7fccb8f9e740 1 --2- 172.21.15.83:0/593074103 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55674f9a1b40 0x55674f99cdd0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.157+0000 7fccb8f9e740 1 -- 172.21.15.83:0/593074103 >> 172.21.15.83:0/593074103 conn(0x55674f99de60 msgr2=0x55674f99c6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:25.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.157+0000 7fccb8f9e740 1 -- 172.21.15.83:0/593074103 shutdown_connections 2024-04-24T08:53:25.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.157+0000 7fccb8f9e740 1 -- 172.21.15.83:0/593074103 wait complete. 2024-04-24T08:53:25.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.158+0000 7fccb8f9e740 1 -- start start 2024-04-24T08:53:25.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.158+0000 7fccb8f9e740 1 Processor -- start 2024-04-24T08:53:25.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.158+0000 7fccb8f9e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55674f9a1b40 0x55674f9a7490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:25.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.158+0000 7fccb8f9e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55674f9a2460 0x55674f9b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:25.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.158+0000 7fccb8f9e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55674f9acb30 0x55674f9adc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:25.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.158+0000 7fccb8f9e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fccb01b6c40 con 0x55674f9a1b40 2024-04-24T08:53:25.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.158+0000 7fccb8f9e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fccb01b6570 con 0x55674f9a2460 2024-04-24T08:53:25.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.158+0000 7fccb8f9e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fccb01b37b0 con 0x55674f9acb30 2024-04-24T08:53:25.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.158+0000 7fccb8f9e740 10 client.0 _ll_register_callbacks cb 0x7fcca8097f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:53:25.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.159+0000 7fccaf400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55674f9a1b40 0x55674f9a7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:25.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.159+0000 7fccaf400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55674f9a1b40 0x55674f9a7490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50536/0 (socket says 172.21.15.83:50536) 2024-04-24T08:53:25.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.159+0000 7fccaf400640 1 -- 172.21.15.83:0/3994268346 learned_addr learned my addr 172.21.15.83:0/3994268346 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:53:25.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.159+0000 7fccaf400640 1 -- 172.21.15.83:0/3994268346 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55674f9acb30 msgr2=0x55674f9adc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:25.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.159+0000 7fccaf400640 1 --2- 172.21.15.83:0/3994268346 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55674f9acb30 0x55674f9adc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.159+0000 7fcca6a00640 1 --2- 172.21.15.83:0/3994268346 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55674f9a2460 0x55674f9b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:25.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.159+0000 7fccaf400640 1 -- 172.21.15.83:0/3994268346 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55674f9a2460 msgr2=0x55674f9b4dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:25.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.159+0000 7fccaf400640 1 --2- 172.21.15.83:0/3994268346 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55674f9a2460 0x55674f9b4dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.159+0000 7fccaf400640 1 -- 172.21.15.83:0/3994268346 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55674f9ae150 con 0x55674f9a1b40 2024-04-24T08:53:25.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.159+0000 7fccaf400640 1 --2- 172.21.15.83:0/3994268346 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55674f9a1b40 0x55674f9a7490 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fcca0019040 tx=0x7fcca000b420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a239c54774a90f1d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:25.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.159+0000 7fcc9cc00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:53:25.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.160+0000 7fcc9cc00640 1 -- 172.21.15.83:0/3994268346 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcca0028ae0 con 0x55674f9a1b40 2024-04-24T08:53:25.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.160+0000 7fccb8f9e740 1 -- 172.21.15.83:0/3994268346 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcca002d070 con 0x55674f9a1b40 2024-04-24T08:53:25.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.160+0000 7fcc9cc00640 1 -- 172.21.15.83:0/3994268346 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7fcca0028c80 con 0x55674f9a1b40 2024-04-24T08:53:25.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.160+0000 7fcc9cc00640 1 -- 172.21.15.83:0/3994268346 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fcca0044050 con 0x55674f9a1b40 2024-04-24T08:53:25.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.160+0000 7fccb8f9e740 10 client.16311 fetch_fsmap learned FSMap version 28 2024-04-24T08:53:25.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.160+0000 7fccb8f9e740 1 -- 172.21.15.83:0/3994268346 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55674fa23730 con 0x55674f9a1b40 2024-04-24T08:53:25.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.160+0000 7fcc9cc00640 1 -- 172.21.15.83:0/3994268346 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcca002d070 con 0x55674f9a1b40 2024-04-24T08:53:25.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.161+0000 7fcc9cc00640 1 -- 172.21.15.83:0/3994268346 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fcca003e4f0 con 0x55674f9a1b40 2024-04-24T08:53:25.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.161+0000 7fccb8f9e740 10 client.16311 fetch_fsmap finished waiting for FSMap version 28 2024-04-24T08:53:25.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.161+0000 7fccb8f9e740 10 client.16311 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-24T08:53:25.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.161+0000 7fccb8f9e740 20 client.16311 populate_metadata read hostname 'smithi083' 2024-04-24T08:53:25.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.161+0000 7fccb8f9e740 1 --2- 172.21.15.83:0/3994268346 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55674fa23c60 0x55674fa26080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:25.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.161+0000 7fccb8f9e740 4 client.16311 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T08:53:25.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.161+0000 7fccb8f9e740 1 -- 172.21.15.83:0/3994268346 --> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55674f99dcb0 con 0x55674fa23c60 2024-04-24T08:53:25.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.161+0000 7fcca6a00640 1 --2- 172.21.15.83:0/3994268346 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55674fa23c60 0x55674fa26080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:25.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.162+0000 7fcc9cc00640 10 client.16311 ms_handle_connect on v2:172.21.15.189:6838/2548965336 2024-04-24T08:53:25.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.162+0000 7fcca6a00640 1 --2- 172.21.15.83:0/3994268346 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55674fa23c60 0x55674fa26080 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=21b96fd923d7059e server_cookie=45b2e52e30fe1763 in_seq=0 out_seq=0 2024-04-24T08:53:25.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.163+0000 7fcc9cc00640 1 -- 172.21.15.83:0/3994268346 <== mds.4 v2:172.21.15.189:6838/2548965336 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55674f99dcb0 con 0x55674fa23c60 2024-04-24T08:53:25.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.163+0000 7fcc9cc00640 10 client.16311 handle_command_reply: tid=0 2024-04-24T08:53:25.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.163+0000 7fccb8f9e740 1 client.16311 shutdown 2024-04-24T08:53:25.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.163+0000 7fccb8f9e740 1 -- 172.21.15.83:0/3994268346 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55674f9a1b40 msgr2=0x55674f9a7490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:25.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.163+0000 7fccb8f9e740 1 --2- 172.21.15.83:0/3994268346 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55674f9a1b40 0x55674f9a7490 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fcca0019040 tx=0x7fcca000b420 comp rx=0 tx=0).stop 2024-04-24T08:53:25.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.163+0000 7fccb8f9e740 1 -- 172.21.15.83:0/3994268346 shutdown_connections 2024-04-24T08:53:25.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.163+0000 7fccb8f9e740 1 --2- 172.21.15.83:0/3994268346 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55674fa23c60 0x55674fa26080 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.163+0000 7fccb8f9e740 1 --2- 172.21.15.83:0/3994268346 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55674f9acb30 0x55674f9adc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.163+0000 7fccb8f9e740 1 --2- 172.21.15.83:0/3994268346 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55674f9a2460 0x55674f9b4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.163+0000 7fcc9cc00640 0 client.16311 ms_handle_reset on v2:172.21.15.189:6838/2548965336 2024-04-24T08:53:25.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.163+0000 7fccb8f9e740 1 --2- 172.21.15.83:0/3994268346 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55674f9a1b40 0x55674f9a7490 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.163+0000 7fccb8f9e740 1 -- 172.21.15.83:0/3994268346 >> 172.21.15.83:0/3994268346 conn(0x55674f99de60 msgr2=0x55674f9a7c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:25.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.163+0000 7fccb8f9e740 1 -- 172.21.15.83:0/3994268346 shutdown_connections 2024-04-24T08:53:25.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.163+0000 7fccb8f9e740 1 -- 172.21.15.83:0/3994268346 wait complete. 2024-04-24T08:53:25.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.163+0000 7fccb8f9e740 20 client.16311 trim_cache size 0 max 16384 2024-04-24T08:53:25.166 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T08:53:25.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.164+0000 7fccb4a00640 1 -- 172.21.15.83:0/3757276239 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fcc940a3da0 msgr2=0x7fcc940a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:25.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.164+0000 7fccb4a00640 1 --2- 172.21.15.83:0/3757276239 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fcc940a3da0 0x7fcc940a6260 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fcc98006010 tx=0x7fcc98000bc0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.164+0000 7fccb4a00640 1 -- 172.21.15.83:0/3757276239 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fccb010d500 msgr2=0x7fccb019eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:25.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.164+0000 7fccb4a00640 1 --2- 172.21.15.83:0/3757276239 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fccb010d500 0x7fccb019eab0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fcca800b9e0 tx=0x7fcca800bea0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.164+0000 7fccb4a00640 1 -- 172.21.15.83:0/3757276239 shutdown_connections 2024-04-24T08:53:25.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.164+0000 7fccb4a00640 1 --2- 172.21.15.83:0/3757276239 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fcc940a3da0 0x7fcc940a6260 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.164+0000 7fccb4a00640 1 --2- 172.21.15.83:0/3757276239 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fccb010d500 0x7fccb019eab0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.164+0000 7fccb4a00640 1 --2- 172.21.15.83:0/3757276239 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fccb0108f40 0x7fccb01a4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.164+0000 7fccb4a00640 1 --2- 172.21.15.83:0/3757276239 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fccb0108580 0x7fccb01a48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.164+0000 7fccb4a00640 1 -- 172.21.15.83:0/3757276239 >> 172.21.15.83:0/3757276239 conn(0x7fccb0077e20 msgr2=0x7fccb0078340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:25.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.164+0000 7fccb4a00640 1 -- 172.21.15.83:0/3757276239 shutdown_connections 2024-04-24T08:53:25.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.164+0000 7fccb4a00640 1 -- 172.21.15.83:0/3757276239 wait complete. 2024-04-24T08:53:25.179 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-24T08:53:25.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.409+0000 7f3037200640 1 -- 172.21.15.83:0/118887645 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3038108580 msgr2=0x7f3038108a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:25.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.409+0000 7f3037200640 1 --2- 172.21.15.83:0/118887645 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3038108580 0x7f3038108a00 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f3024009930 tx=0x7f302402a420 comp rx=0 tx=0).stop 2024-04-24T08:53:25.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.409+0000 7f3037200640 1 -- 172.21.15.83:0/118887645 shutdown_connections 2024-04-24T08:53:25.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.409+0000 7f3037200640 1 --2- 172.21.15.83:0/118887645 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3038108f40 0x7f30381153b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.409+0000 7f3037200640 1 --2- 172.21.15.83:0/118887645 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3038108580 0x7f3038108a00 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.409+0000 7f3037200640 1 --2- 172.21.15.83:0/118887645 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f303810d500 0x7f303810d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.409+0000 7f3037200640 1 -- 172.21.15.83:0/118887645 >> 172.21.15.83:0/118887645 conn(0x7f3038077e20 msgr2=0x7f3038106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:25.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.409+0000 7f3037200640 1 -- 172.21.15.83:0/118887645 shutdown_connections 2024-04-24T08:53:25.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.409+0000 7f3037200640 1 -- 172.21.15.83:0/118887645 wait complete. 2024-04-24T08:53:25.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.410+0000 7f3037200640 1 Processor -- start 2024-04-24T08:53:25.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.410+0000 7f3037200640 1 -- start start 2024-04-24T08:53:25.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.410+0000 7f3037200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3038108580 0x7f30381a4860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:25.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.410+0000 7f3037200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3038108f40 0x7f30381a4da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:25.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.410+0000 7f3037200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f303810d500 0x7f303819ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:25.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.410+0000 7f3037200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3038118760 con 0x7f3038108580 2024-04-24T08:53:25.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.410+0000 7f3037200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f30381185e0 con 0x7f3038108f40 2024-04-24T08:53:25.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.410+0000 7f3037200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f30381188e0 con 0x7f303810d500 2024-04-24T08:53:25.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.410+0000 7f3035e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3038108580 0x7f30381a4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:25.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.410+0000 7f3035e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3038108580 0x7f30381a4860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50550/0 (socket says 172.21.15.83:50550) 2024-04-24T08:53:25.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.410+0000 7f3035e00640 1 -- 172.21.15.83:0/3183353622 learned_addr learned my addr 172.21.15.83:0/3183353622 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:53:25.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.411+0000 7f3035400640 1 --2- 172.21.15.83:0/3183353622 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3038108f40 0x7f30381a4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:25.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.411+0000 7f3035e00640 1 -- 172.21.15.83:0/3183353622 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f303810d500 msgr2=0x7f303819ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:25.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.411+0000 7f3035e00640 1 --2- 172.21.15.83:0/3183353622 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f303810d500 0x7f303819ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.411+0000 7f3035e00640 1 -- 172.21.15.83:0/3183353622 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3038108f40 msgr2=0x7f30381a4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:25.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.411+0000 7f3035e00640 1 --2- 172.21.15.83:0/3183353622 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3038108f40 0x7f30381a4da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.411+0000 7f3035e00640 1 -- 172.21.15.83:0/3183353622 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f303819f1e0 con 0x7f3038108580 2024-04-24T08:53:25.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.411+0000 7f3035e00640 1 --2- 172.21.15.83:0/3183353622 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3038108580 0x7f30381a4860 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f302800c970 tx=0x7f302800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:25.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.411+0000 7f302ea00640 1 -- 172.21.15.83:0/3183353622 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3028004220 con 0x7f3038108580 2024-04-24T08:53:25.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.411+0000 7f302ea00640 1 -- 172.21.15.83:0/3183353622 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f30280043c0 con 0x7f3038108580 2024-04-24T08:53:25.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.411+0000 7f3037200640 1 -- 172.21.15.83:0/3183353622 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f303819f4d0 con 0x7f3038108580 2024-04-24T08:53:25.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.411+0000 7f302ea00640 1 -- 172.21.15.83:0/3183353622 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f302800f450 con 0x7f3038108580 2024-04-24T08:53:25.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.412+0000 7f3037200640 1 -- 172.21.15.83:0/3183353622 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30381ab720 con 0x7f3038108580 2024-04-24T08:53:25.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.414+0000 7f303f9e5740 1 Processor -- start 2024-04-24T08:53:25.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.415+0000 7f302ea00640 1 -- 172.21.15.83:0/3183353622 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f30280079b0 con 0x7f3038108580 2024-04-24T08:53:25.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.415+0000 7f303f9e5740 1 -- start start 2024-04-24T08:53:25.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.415+0000 7f303f9e5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dd49392b40 0x55dd49392f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:25.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.415+0000 7f303f9e5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd49393460 0x55dd4939d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:25.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.415+0000 7f303f9e5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd4939db30 0x55dd4939ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:25.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.415+0000 7f303f9e5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55dd492f81d0 con 0x55dd4939db30 2024-04-24T08:53:25.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.415+0000 7f303f9e5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55dd4929e450 con 0x55dd49392b40 2024-04-24T08:53:25.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.415+0000 7f303f9e5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55dd492f4190 con 0x55dd49393460 2024-04-24T08:53:25.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.415+0000 7f3035400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd4939db30 0x55dd4939ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:25.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.415+0000 7f3035400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd4939db30 0x55dd4939ff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50556/0 (socket says 172.21.15.83:50556) 2024-04-24T08:53:25.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.415+0000 7f3035400640 1 -- 172.21.15.83:0/3877931667 learned_addr learned my addr 172.21.15.83:0/3877931667 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:53:25.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.416+0000 7f3036800640 1 --2- 172.21.15.83:0/3877931667 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dd49392b40 0x55dd49392f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:25.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.416+0000 7f3035e00640 1 --2- 172.21.15.83:0/3877931667 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd49393460 0x55dd4939d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:25.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.416+0000 7f3035400640 1 -- 172.21.15.83:0/3877931667 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd49393460 msgr2=0x55dd4939d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:25.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.416+0000 7f3035400640 1 --2- 172.21.15.83:0/3877931667 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd49393460 0x55dd4939d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.416+0000 7f3035400640 1 -- 172.21.15.83:0/3877931667 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dd49392b40 msgr2=0x55dd49392f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:25.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.416+0000 7f3035400640 1 --2- 172.21.15.83:0/3877931667 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dd49392b40 0x55dd49392f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.416+0000 7f3035400640 1 -- 172.21.15.83:0/3877931667 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd4935b310 con 0x55dd4939db30 2024-04-24T08:53:25.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.416+0000 7f3035400640 1 --2- 172.21.15.83:0/3877931667 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd4939db30 0x55dd4939ff20 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f3024009930 tx=0x7f3024032e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=859daef2501cbc8d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:25.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.416+0000 7f302ea00640 1 --2- 172.21.15.83:0/3183353622 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f301c0a3d20 0x7f301c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:25.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.416+0000 7f302ea00640 1 -- 172.21.15.83:0/3183353622 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f30280d0a20 con 0x7f3038108580 2024-04-24T08:53:25.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.417+0000 7f3023000640 1 -- 172.21.15.83:0/3877931667 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f302403f070 con 0x55dd4939db30 2024-04-24T08:53:25.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.417+0000 7f3036800640 1 --2- 172.21.15.83:0/3183353622 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f301c0a3d20 0x7f301c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:25.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.417+0000 7f3023000640 1 -- 172.21.15.83:0/3877931667 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f30240316a0 con 0x55dd4939db30 2024-04-24T08:53:25.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.417+0000 7f3023000640 1 -- 172.21.15.83:0/3877931667 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3024035070 con 0x55dd4939db30 2024-04-24T08:53:25.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.417+0000 7f303f9e5740 1 -- 172.21.15.83:0/3877931667 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd4939db30 msgr2=0x55dd4939ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:25.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.417+0000 7f303f9e5740 1 --2- 172.21.15.83:0/3877931667 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd4939db30 0x55dd4939ff20 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f3024009930 tx=0x7f3024032e40 comp rx=0 tx=0).stop 2024-04-24T08:53:25.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.417+0000 7f3036800640 1 --2- 172.21.15.83:0/3183353622 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f301c0a3d20 0x7f301c0a61e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3030007e00 tx=0x7f3030007410 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:25.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.417+0000 7f303f9e5740 1 -- 172.21.15.83:0/3877931667 shutdown_connections 2024-04-24T08:53:25.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.417+0000 7f303f9e5740 1 --2- 172.21.15.83:0/3877931667 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd4939db30 0x55dd4939ff20 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.417+0000 7f303f9e5740 1 --2- 172.21.15.83:0/3877931667 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd49393460 0x55dd4939d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.417+0000 7f303f9e5740 1 --2- 172.21.15.83:0/3877931667 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dd49392b40 0x55dd49392f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.417+0000 7f303f9e5740 1 -- 172.21.15.83:0/3877931667 >> 172.21.15.83:0/3877931667 conn(0x55dd4938ee60 msgr2=0x55dd4938f240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:25.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.418+0000 7f303f9e5740 1 -- 172.21.15.83:0/3877931667 shutdown_connections 2024-04-24T08:53:25.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.418+0000 7f303f9e5740 1 -- 172.21.15.83:0/3877931667 wait complete. 2024-04-24T08:53:25.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.418+0000 7f303f9e5740 1 -- start start 2024-04-24T08:53:25.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.418+0000 7f303f9e5740 1 Processor -- start 2024-04-24T08:53:25.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.419+0000 7f303f9e5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dd49392b40 0x55dd4939d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:25.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.419+0000 7f303f9e5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd49393460 0x55dd4939a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:25.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.419+0000 7f303f9e5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd4939db30 0x55dd4939ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:25.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.419+0000 7f303f9e5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55dd493a09b0 con 0x55dd49393460 2024-04-24T08:53:25.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.419+0000 7f303f9e5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55dd492b1810 con 0x55dd49392b40 2024-04-24T08:53:25.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.419+0000 7f303f9e5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55dd493a0b30 con 0x55dd4939db30 2024-04-24T08:53:25.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.419+0000 7f303f9e5740 10 client.0 _ll_register_callbacks cb 0x55dd4927f140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:53:25.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.419+0000 7f3035400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd49393460 0x55dd4939a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:25.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.419+0000 7f3035400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd49393460 0x55dd4939a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50570/0 (socket says 172.21.15.83:50570) 2024-04-24T08:53:25.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.419+0000 7f3035400640 1 -- 172.21.15.83:0/917064287 learned_addr learned my addr 172.21.15.83:0/917064287 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:53:25.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.419+0000 7f3036800640 1 --2- 172.21.15.83:0/917064287 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd4939db30 0x55dd4939ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:25.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.419+0000 7f3035400640 1 -- 172.21.15.83:0/917064287 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd4939db30 msgr2=0x55dd4939ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:25.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.419+0000 7f3035e00640 1 --2- 172.21.15.83:0/917064287 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dd49392b40 0x55dd4939d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:25.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.419+0000 7f3035400640 1 --2- 172.21.15.83:0/917064287 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd4939db30 0x55dd4939ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.419+0000 7f3035400640 1 -- 172.21.15.83:0/917064287 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dd49392b40 msgr2=0x55dd4939d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:25.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.419+0000 7f3035400640 1 --2- 172.21.15.83:0/917064287 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dd49392b40 0x55dd4939d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.419+0000 7f3035400640 1 -- 172.21.15.83:0/917064287 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd492c23f0 con 0x55dd49393460 2024-04-24T08:53:25.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.419+0000 7f3036800640 1 --2- 172.21.15.83:0/917064287 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd4939db30 0x55dd4939ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:53:25.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.419+0000 7f3035e00640 1 --2- 172.21.15.83:0/917064287 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dd49392b40 0x55dd4939d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:53:25.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.420+0000 7f3035400640 1 --2- 172.21.15.83:0/917064287 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd49393460 0x55dd4939a750 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f3024032fd0 tx=0x7f30240314a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b3064d5df9767df server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:25.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.420+0000 7f3017400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:53:25.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.420+0000 7f3017400640 1 -- 172.21.15.83:0/917064287 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f302403f070 con 0x55dd49393460 2024-04-24T08:53:25.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.420+0000 7f3017400640 1 -- 172.21.15.83:0/917064287 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f3024004020 con 0x55dd49393460 2024-04-24T08:53:25.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.420+0000 7f303f9e5740 1 -- 172.21.15.83:0/917064287 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30240316a0 con 0x55dd49393460 2024-04-24T08:53:25.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.420+0000 7f3017400640 1 -- 172.21.15.83:0/917064287 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3024054050 con 0x55dd49393460 2024-04-24T08:53:25.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.420+0000 7f303f9e5740 10 client.16323 fetch_fsmap learned FSMap version 28 2024-04-24T08:53:25.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.420+0000 7f303f9e5740 1 -- 172.21.15.83:0/917064287 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55dd4900a3f0 con 0x55dd49393460 2024-04-24T08:53:25.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.421+0000 7f3017400640 1 -- 172.21.15.83:0/917064287 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f30240316a0 con 0x55dd49393460 2024-04-24T08:53:25.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.421+0000 7f3017400640 1 -- 172.21.15.83:0/917064287 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f3024003680 con 0x55dd49393460 2024-04-24T08:53:25.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.421+0000 7f303f9e5740 10 client.16323 fetch_fsmap finished waiting for FSMap version 28 2024-04-24T08:53:25.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.421+0000 7f303f9e5740 10 client.16323 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-24T08:53:25.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.421+0000 7f303f9e5740 20 client.16323 populate_metadata read hostname 'smithi083' 2024-04-24T08:53:25.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.421+0000 7f303f9e5740 1 --2- 172.21.15.83:0/917064287 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55dd493adab0 0x55dd493afea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:25.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.421+0000 7f303f9e5740 4 client.16323 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:53:25.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.421+0000 7f303f9e5740 1 -- 172.21.15.83:0/917064287 --> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dd49305bf0 con 0x55dd493adab0 2024-04-24T08:53:25.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.422+0000 7f3035e00640 1 --2- 172.21.15.83:0/917064287 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55dd493adab0 0x55dd493afea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:25.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.423+0000 7f3035e00640 1 --2- 172.21.15.83:0/917064287 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55dd493adab0 0x55dd493afea0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a8df46e811d5f179 server_cookie=606e9ee0fb48798d in_seq=0 out_seq=0 2024-04-24T08:53:25.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.423+0000 7f3017400640 10 client.16323 ms_handle_connect on v2:172.21.15.138:6832/1157967658 2024-04-24T08:53:25.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.425+0000 7f3017400640 1 -- 172.21.15.83:0/917064287 <== mds.3 v2:172.21.15.138:6832/1157967658 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55dd49305bf0 con 0x55dd493adab0 2024-04-24T08:53:25.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.425+0000 7f3017400640 10 client.16323 handle_command_reply: tid=0 2024-04-24T08:53:25.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.425+0000 7f303f9e5740 1 client.16323 shutdown 2024-04-24T08:53:25.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.425+0000 7f303f9e5740 1 -- 172.21.15.83:0/917064287 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd49393460 msgr2=0x55dd4939a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:25.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.425+0000 7f303f9e5740 1 --2- 172.21.15.83:0/917064287 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd49393460 0x55dd4939a750 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f3024032fd0 tx=0x7f30240314a0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.425+0000 7f303f9e5740 1 -- 172.21.15.83:0/917064287 shutdown_connections 2024-04-24T08:53:25.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.425+0000 7f303f9e5740 1 --2- 172.21.15.83:0/917064287 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55dd493adab0 0x55dd493afea0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.425+0000 7f303f9e5740 1 --2- 172.21.15.83:0/917064287 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd4939db30 0x55dd4939ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.425+0000 7f303f9e5740 1 --2- 172.21.15.83:0/917064287 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd49393460 0x55dd4939a750 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.425+0000 7f303f9e5740 1 --2- 172.21.15.83:0/917064287 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dd49392b40 0x55dd4939d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.425+0000 7f3017400640 0 client.16323 ms_handle_reset on v2:172.21.15.138:6832/1157967658 2024-04-24T08:53:25.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.425+0000 7f303f9e5740 1 -- 172.21.15.83:0/917064287 >> 172.21.15.83:0/917064287 conn(0x55dd4938ee60 msgr2=0x55dd49394c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:25.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.425+0000 7f303f9e5740 1 -- 172.21.15.83:0/917064287 shutdown_connections 2024-04-24T08:53:25.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.425+0000 7f303f9e5740 1 -- 172.21.15.83:0/917064287 wait complete. 2024-04-24T08:53:25.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.425+0000 7f303f9e5740 20 client.16323 trim_cache size 0 max 16384 2024-04-24T08:53:25.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.440+0000 7f303f9e5740 1 Processor -- start 2024-04-24T08:53:25.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.441+0000 7f303f9e5740 1 -- start start 2024-04-24T08:53:25.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.441+0000 7f303f9e5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd49392b40 0x55dd493a98a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:25.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.441+0000 7f303f9e5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dd49393460 0x55dd4939ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:25.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.441+0000 7f303f9e5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd4939db30 0x55dd4939cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:25.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.441+0000 7f303f9e5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f30381b58e0 con 0x55dd49392b40 2024-04-24T08:53:25.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.441+0000 7f303f9e5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f30381b7420 con 0x55dd49393460 2024-04-24T08:53:25.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.441+0000 7f303f9e5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f30381b5fb0 con 0x55dd4939db30 2024-04-24T08:53:25.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.441+0000 7f3035e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd49392b40 0x55dd493a98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:25.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.441+0000 7f3035e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd49392b40 0x55dd493a98a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50586/0 (socket says 172.21.15.83:50586) 2024-04-24T08:53:25.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.441+0000 7f3035e00640 1 -- 172.21.15.83:0/249213287 learned_addr learned my addr 172.21.15.83:0/249213287 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:53:25.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.442+0000 7f3035400640 1 --2- 172.21.15.83:0/249213287 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dd49393460 0x55dd4939ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:25.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.442+0000 7f3036800640 1 --2- 172.21.15.83:0/249213287 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd4939db30 0x55dd4939cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:25.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.442+0000 7f3035e00640 1 -- 172.21.15.83:0/249213287 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd4939db30 msgr2=0x55dd4939cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:25.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.442+0000 7f3035e00640 1 --2- 172.21.15.83:0/249213287 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd4939db30 0x55dd4939cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.442+0000 7f3035e00640 1 -- 172.21.15.83:0/249213287 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dd49393460 msgr2=0x55dd4939ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:25.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.442+0000 7f3035e00640 1 --2- 172.21.15.83:0/249213287 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dd49393460 0x55dd4939ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.442+0000 7f3035e00640 1 -- 172.21.15.83:0/249213287 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd490d47d0 con 0x55dd49392b40 2024-04-24T08:53:25.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.442+0000 7f3035400640 1 --2- 172.21.15.83:0/249213287 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dd49393460 0x55dd4939ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:53:25.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.442+0000 7f3036800640 1 --2- 172.21.15.83:0/249213287 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd4939db30 0x55dd4939cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:53:25.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.442+0000 7f3035e00640 1 --2- 172.21.15.83:0/249213287 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd49392b40 0x55dd493a98a0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f3028098a50 tx=0x7f3028000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12c6da72af441182 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:25.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.442+0000 7f3022600640 1 -- 172.21.15.83:0/249213287 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f302801d040 con 0x55dd49392b40 2024-04-24T08:53:25.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.442+0000 7f3022600640 1 -- 172.21.15.83:0/249213287 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f302809ccf0 con 0x55dd49392b40 2024-04-24T08:53:25.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.442+0000 7f3022600640 1 -- 172.21.15.83:0/249213287 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3028007ca0 con 0x55dd49392b40 2024-04-24T08:53:25.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.443+0000 7f303f9e5740 1 -- 172.21.15.83:0/249213287 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd49392b40 msgr2=0x55dd493a98a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:25.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.443+0000 7f303f9e5740 1 --2- 172.21.15.83:0/249213287 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd49392b40 0x55dd493a98a0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f3028098a50 tx=0x7f3028000bc0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.443+0000 7f303f9e5740 1 -- 172.21.15.83:0/249213287 shutdown_connections 2024-04-24T08:53:25.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.443+0000 7f303f9e5740 1 --2- 172.21.15.83:0/249213287 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd4939db30 0x55dd4939cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.443+0000 7f303f9e5740 1 --2- 172.21.15.83:0/249213287 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dd49393460 0x55dd4939ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.443+0000 7f303f9e5740 1 --2- 172.21.15.83:0/249213287 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd49392b40 0x55dd493a98a0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.443+0000 7f303f9e5740 1 -- 172.21.15.83:0/249213287 >> 172.21.15.83:0/249213287 conn(0x55dd4938ee60 msgr2=0x55dd49399020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:25.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.443+0000 7f303f9e5740 1 -- 172.21.15.83:0/249213287 shutdown_connections 2024-04-24T08:53:25.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.443+0000 7f303f9e5740 1 -- 172.21.15.83:0/249213287 wait complete. 2024-04-24T08:53:25.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.443+0000 7f303f9e5740 1 -- start start 2024-04-24T08:53:25.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.444+0000 7f303f9e5740 1 Processor -- start 2024-04-24T08:53:25.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.444+0000 7f303f9e5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd49392b40 0x55dd4939e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:25.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.444+0000 7f303f9e5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd49393460 0x55dd4939e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:25.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.444+0000 7f303f9e5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dd4939db30 0x55dd493a4db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:25.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.444+0000 7f303f9e5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f30381b6d50 con 0x55dd49392b40 2024-04-24T08:53:25.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.444+0000 7f303f9e5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f30381b6680 con 0x55dd4939db30 2024-04-24T08:53:25.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.444+0000 7f303f9e5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f30381b38c0 con 0x55dd49393460 2024-04-24T08:53:25.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.444+0000 7f303f9e5740 10 client.0 _ll_register_callbacks cb 0x7f3024032d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:53:25.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.444+0000 7f3035e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd49392b40 0x55dd4939e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:25.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.444+0000 7f3035e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd49392b40 0x55dd4939e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50598/0 (socket says 172.21.15.83:50598) 2024-04-24T08:53:25.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.444+0000 7f3035e00640 1 -- 172.21.15.83:0/1090293864 learned_addr learned my addr 172.21.15.83:0/1090293864 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:53:25.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.445+0000 7f3035e00640 1 -- 172.21.15.83:0/1090293864 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd49393460 msgr2=0x55dd4939e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:25.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.445+0000 7f3035e00640 1 --2- 172.21.15.83:0/1090293864 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd49393460 0x55dd4939e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.445+0000 7f3035e00640 1 -- 172.21.15.83:0/1090293864 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dd4939db30 msgr2=0x55dd493a4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:25.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.445+0000 7f3035e00640 1 --2- 172.21.15.83:0/1090293864 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dd4939db30 0x55dd493a4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.445+0000 7f3035e00640 1 -- 172.21.15.83:0/1090293864 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd493a52f0 con 0x55dd49392b40 2024-04-24T08:53:25.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.445+0000 7f3035e00640 1 --2- 172.21.15.83:0/1090293864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd49392b40 0x55dd4939e090 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f30280cde50 tx=0x7f302809bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=782f1907d76e17d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:25.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.445+0000 7f3017400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:53:25.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.445+0000 7f3017400640 1 -- 172.21.15.83:0/1090293864 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f302801d030 con 0x55dd49392b40 2024-04-24T08:53:25.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.445+0000 7f3017400640 1 -- 172.21.15.83:0/1090293864 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f3028003610 con 0x55dd49392b40 2024-04-24T08:53:25.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.445+0000 7f303f9e5740 1 -- 172.21.15.83:0/1090293864 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f302809ccf0 con 0x55dd49392b40 2024-04-24T08:53:25.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.445+0000 7f3017400640 1 -- 172.21.15.83:0/1090293864 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3028067120 con 0x55dd49392b40 2024-04-24T08:53:25.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.445+0000 7f303f9e5740 10 client.16329 fetch_fsmap learned FSMap version 28 2024-04-24T08:53:25.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.445+0000 7f303f9e5740 1 -- 172.21.15.83:0/1090293864 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55dd493a6560 con 0x55dd49392b40 2024-04-24T08:53:25.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.446+0000 7f3017400640 1 -- 172.21.15.83:0/1090293864 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f302809ccf0 con 0x55dd49392b40 2024-04-24T08:53:25.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.446+0000 7f3017400640 1 -- 172.21.15.83:0/1090293864 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f30280ce020 con 0x55dd49392b40 2024-04-24T08:53:25.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.446+0000 7f303f9e5740 10 client.16329 fetch_fsmap finished waiting for FSMap version 28 2024-04-24T08:53:25.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.446+0000 7f303f9e5740 10 client.16329 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-24T08:53:25.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.446+0000 7f303f9e5740 20 client.16329 populate_metadata read hostname 'smithi083' 2024-04-24T08:53:25.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.446+0000 7f303f9e5740 1 --2- 172.21.15.83:0/1090293864 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55dd494140c0 0x55dd49416520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:25.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.446+0000 7f303f9e5740 4 client.16329 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T08:53:25.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.446+0000 7f303f9e5740 1 -- 172.21.15.83:0/1090293864 --> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dd492ffa60 con 0x55dd494140c0 2024-04-24T08:53:25.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.446+0000 7f3035400640 1 --2- 172.21.15.83:0/1090293864 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55dd494140c0 0x55dd49416520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:25.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.447+0000 7f3035400640 1 --2- 172.21.15.83:0/1090293864 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55dd494140c0 0x55dd49416520 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d23028f766e5e622 server_cookie=cda789eeae064de1 in_seq=0 out_seq=0 2024-04-24T08:53:25.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.447+0000 7f3017400640 10 client.16329 ms_handle_connect on v2:172.21.15.138:6832/1157967658 2024-04-24T08:53:25.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.448+0000 7f3017400640 1 -- 172.21.15.83:0/1090293864 <== mds.3 v2:172.21.15.138:6832/1157967658 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dd492ffa60 con 0x55dd494140c0 2024-04-24T08:53:25.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.448+0000 7f3017400640 10 client.16329 handle_command_reply: tid=0 2024-04-24T08:53:25.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.448+0000 7f303f9e5740 1 client.16329 shutdown 2024-04-24T08:53:25.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.448+0000 7f303f9e5740 1 -- 172.21.15.83:0/1090293864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd49392b40 msgr2=0x55dd4939e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:25.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.448+0000 7f303f9e5740 1 --2- 172.21.15.83:0/1090293864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd49392b40 0x55dd4939e090 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f30280cde50 tx=0x7f302809bf50 comp rx=0 tx=0).stop 2024-04-24T08:53:25.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.448+0000 7f303f9e5740 1 -- 172.21.15.83:0/1090293864 shutdown_connections 2024-04-24T08:53:25.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.448+0000 7f303f9e5740 1 --2- 172.21.15.83:0/1090293864 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dd4939db30 0x55dd493a4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.448+0000 7f303f9e5740 1 --2- 172.21.15.83:0/1090293864 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55dd494140c0 0x55dd49416520 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.448+0000 7f303f9e5740 1 --2- 172.21.15.83:0/1090293864 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd49393460 0x55dd4939e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.448+0000 7f303f9e5740 1 --2- 172.21.15.83:0/1090293864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd49392b40 0x55dd4939e090 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.448+0000 7f3017400640 0 client.16329 ms_handle_reset on v2:172.21.15.138:6832/1157967658 2024-04-24T08:53:25.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.448+0000 7f303f9e5740 1 -- 172.21.15.83:0/1090293864 >> 172.21.15.83:0/1090293864 conn(0x55dd4938ee60 msgr2=0x55dd4938d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:25.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.448+0000 7f303f9e5740 1 -- 172.21.15.83:0/1090293864 shutdown_connections 2024-04-24T08:53:25.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.448+0000 7f303f9e5740 1 -- 172.21.15.83:0/1090293864 wait complete. 2024-04-24T08:53:25.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.448+0000 7f303f9e5740 20 client.16329 trim_cache size 0 max 16384 2024-04-24T08:53:25.452 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T08:53:25.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.449+0000 7f3037200640 1 -- 172.21.15.83:0/3183353622 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f301c0a3d20 msgr2=0x7f301c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:25.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.449+0000 7f3037200640 1 --2- 172.21.15.83:0/3183353622 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f301c0a3d20 0x7f301c0a61e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3030007e00 tx=0x7f3030007410 comp rx=0 tx=0).stop 2024-04-24T08:53:25.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.449+0000 7f3037200640 1 -- 172.21.15.83:0/3183353622 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3038108580 msgr2=0x7f30381a4860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:25.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.449+0000 7f3037200640 1 --2- 172.21.15.83:0/3183353622 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3038108580 0x7f30381a4860 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f302800c970 tx=0x7f302800ce30 comp rx=0 tx=0).stop 2024-04-24T08:53:25.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.449+0000 7f3037200640 1 -- 172.21.15.83:0/3183353622 shutdown_connections 2024-04-24T08:53:25.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.449+0000 7f3037200640 1 --2- 172.21.15.83:0/3183353622 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f301c0a3d20 0x7f301c0a61e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.449+0000 7f3037200640 1 --2- 172.21.15.83:0/3183353622 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f303810d500 0x7f303819ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.449+0000 7f3037200640 1 --2- 172.21.15.83:0/3183353622 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3038108f40 0x7f30381a4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.449+0000 7f3037200640 1 --2- 172.21.15.83:0/3183353622 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3038108580 0x7f30381a4860 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.449+0000 7f3037200640 1 -- 172.21.15.83:0/3183353622 >> 172.21.15.83:0/3183353622 conn(0x7f3038077e20 msgr2=0x7f30380783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:25.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.449+0000 7f3037200640 1 -- 172.21.15.83:0/3183353622 shutdown_connections 2024-04-24T08:53:25.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.449+0000 7f3037200640 1 -- 172.21.15.83:0/3183353622 wait complete. 2024-04-24T08:53:25.466 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T08:53:25.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.639+0000 7f8f92200640 1 -- 172.21.15.83:0/3767883849 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f8c104cd0 msgr2=0x7f8f8c108d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:25.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.639+0000 7f8f92200640 1 --2- 172.21.15.83:0/3767883849 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f8c104cd0 0x7f8f8c108d10 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f8f74009900 tx=0x7f8f7402a3f0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.639+0000 7f8f92200640 1 -- 172.21.15.83:0/3767883849 shutdown_connections 2024-04-24T08:53:25.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.639+0000 7f8f92200640 1 --2- 172.21.15.83:0/3767883849 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f8c1093f0 0x7f8f8c110fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.639+0000 7f8f92200640 1 --2- 172.21.15.83:0/3767883849 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f8c104cd0 0x7f8f8c108d10 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.639+0000 7f8f92200640 1 --2- 172.21.15.83:0/3767883849 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8f8c104320 0x7f8f8c104700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.639+0000 7f8f92200640 1 -- 172.21.15.83:0/3767883849 >> 172.21.15.83:0/3767883849 conn(0x7f8f8c0ffb40 msgr2=0x7f8f8c101f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:25.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.639+0000 7f8f92200640 1 -- 172.21.15.83:0/3767883849 shutdown_connections 2024-04-24T08:53:25.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.639+0000 7f8f92200640 1 -- 172.21.15.83:0/3767883849 wait complete. 2024-04-24T08:53:25.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.640+0000 7f8f92200640 1 Processor -- start 2024-04-24T08:53:25.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.640+0000 7f8f92200640 1 -- start start 2024-04-24T08:53:25.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.640+0000 7f8f92200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f8c104320 0x7f8f8c1a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:25.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.640+0000 7f8f92200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f8c1093f0 0x7f8f8c1a0b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:25.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.640+0000 7f8f92200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8f8c19a6b0 0x7f8f8c19ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:25.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.640+0000 7f8f92200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8f8c114330 con 0x7f8f8c104320 2024-04-24T08:53:25.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.640+0000 7f8f92200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8f8c1141b0 con 0x7f8f8c1093f0 2024-04-24T08:53:25.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.640+0000 7f8f92200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8f8c1144b0 con 0x7f8f8c19a6b0 2024-04-24T08:53:25.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.641+0000 7f8f90e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f8c104320 0x7f8f8c1a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:25.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.641+0000 7f8f90e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f8c104320 0x7f8f8c1a05f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50614/0 (socket says 172.21.15.83:50614) 2024-04-24T08:53:25.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.641+0000 7f8f90e00640 1 -- 172.21.15.83:0/3114476126 learned_addr learned my addr 172.21.15.83:0/3114476126 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:53:25.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.641+0000 7f8f90e00640 1 -- 172.21.15.83:0/3114476126 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8f8c19a6b0 msgr2=0x7f8f8c19ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:25.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.641+0000 7f8f90e00640 1 --2- 172.21.15.83:0/3114476126 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8f8c19a6b0 0x7f8f8c19ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.641+0000 7f8f90e00640 1 -- 172.21.15.83:0/3114476126 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f8c1093f0 msgr2=0x7f8f8c1a0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:25.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.641+0000 7f8f83e00640 1 --2- 172.21.15.83:0/3114476126 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f8c1093f0 0x7f8f8c1a0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:25.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.641+0000 7f8f90e00640 1 --2- 172.21.15.83:0/3114476126 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f8c1093f0 0x7f8f8c1a0b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.641+0000 7f8f90e00640 1 -- 172.21.15.83:0/3114476126 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f8c19b2b0 con 0x7f8f8c104320 2024-04-24T08:53:25.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.641+0000 7f8f90e00640 1 --2- 172.21.15.83:0/3114476126 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f8c104320 0x7f8f8c1a05f0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c00c970 tx=0x7f8f7c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:25.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.642+0000 7f8f8a000640 1 -- 172.21.15.83:0/3114476126 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f7c007bb0 con 0x7f8f8c104320 2024-04-24T08:53:25.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.642+0000 7f8f8a000640 1 -- 172.21.15.83:0/3114476126 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8f7c007d50 con 0x7f8f8c104320 2024-04-24T08:53:25.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.642+0000 7f8f8a000640 1 -- 172.21.15.83:0/3114476126 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f7c00f450 con 0x7f8f8c104320 2024-04-24T08:53:25.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.642+0000 7f8f92200640 1 -- 172.21.15.83:0/3114476126 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f8c1a7390 con 0x7f8f8c104320 2024-04-24T08:53:25.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.642+0000 7f8f92200640 1 -- 172.21.15.83:0/3114476126 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f8c1a7740 con 0x7f8f8c104320 2024-04-24T08:53:25.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.646+0000 7f8f9662e740 1 Processor -- start 2024-04-24T08:53:25.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.646+0000 7f8f9662e740 1 -- start start 2024-04-24T08:53:25.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.646+0000 7f8f8a000640 1 -- 172.21.15.83:0/3114476126 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8f7c004f20 con 0x7f8f8c104320 2024-04-24T08:53:25.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.646+0000 7f8f9662e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbc2fdeb40 0x55bbc2fdef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:25.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.646+0000 7f8f9662e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bbc2fdf460 0x55bbc2fe95f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:25.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.646+0000 7f8f9662e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bbc2fe9b30 0x55bbc2febf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:25.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.646+0000 7f8f9662e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55bbc2f441d0 con 0x55bbc2fdf460 2024-04-24T08:53:25.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.646+0000 7f8f9662e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55bbc2eea450 con 0x55bbc2fdeb40 2024-04-24T08:53:25.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.646+0000 7f8f9662e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55bbc2f40190 con 0x55bbc2fe9b30 2024-04-24T08:53:25.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.647+0000 7f8f90e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bbc2fdf460 0x55bbc2fe95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:25.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.647+0000 7f8f90e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bbc2fdf460 0x55bbc2fe95f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50624/0 (socket says 172.21.15.83:50624) 2024-04-24T08:53:25.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.647+0000 7f8f90e00640 1 -- 172.21.15.83:0/1054769664 learned_addr learned my addr 172.21.15.83:0/1054769664 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:53:25.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.647+0000 7f8f91800640 1 --2- 172.21.15.83:0/1054769664 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbc2fdeb40 0x55bbc2fdef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:25.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.647+0000 7f8f83e00640 1 --2- 172.21.15.83:0/1054769664 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bbc2fe9b30 0x55bbc2febf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:25.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.647+0000 7f8f90e00640 1 -- 172.21.15.83:0/1054769664 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bbc2fe9b30 msgr2=0x55bbc2febf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:25.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.647+0000 7f8f90e00640 1 --2- 172.21.15.83:0/1054769664 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bbc2fe9b30 0x55bbc2febf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.647+0000 7f8f90e00640 1 -- 172.21.15.83:0/1054769664 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbc2fdeb40 msgr2=0x55bbc2fdef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:25.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.647+0000 7f8f90e00640 1 --2- 172.21.15.83:0/1054769664 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbc2fdeb40 0x55bbc2fdef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.647+0000 7f8f90e00640 1 -- 172.21.15.83:0/1054769664 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbc2fa7310 con 0x55bbc2fdf460 2024-04-24T08:53:25.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.648+0000 7f8f90e00640 1 --2- 172.21.15.83:0/1054769664 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bbc2fdf460 0x55bbc2fe95f0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c0cdc80 tx=0x7f8f7c0d94d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14aa364d12fd701b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:25.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.648+0000 7f8f8a000640 1 --2- 172.21.15.83:0/3114476126 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f8f700a3da0 0x7f8f700a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:25.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.648+0000 7f8f82000640 1 -- 172.21.15.83:0/1054769664 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f7c0e4070 con 0x55bbc2fdf460 2024-04-24T08:53:25.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.648+0000 7f8f82000640 1 -- 172.21.15.83:0/1054769664 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8f7c09cc60 con 0x55bbc2fdf460 2024-04-24T08:53:25.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.648+0000 7f8f82000640 1 -- 172.21.15.83:0/1054769664 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f7c0db040 con 0x55bbc2fdf460 2024-04-24T08:53:25.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.648+0000 7f8f8a000640 1 -- 172.21.15.83:0/3114476126 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f8f7c0d0990 con 0x7f8f8c104320 2024-04-24T08:53:25.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.648+0000 7f8f91800640 1 --2- 172.21.15.83:0/3114476126 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f8f700a3da0 0x7f8f700a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:25.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.649+0000 7f8f9662e740 1 -- 172.21.15.83:0/1054769664 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bbc2fdf460 msgr2=0x55bbc2fe95f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:25.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.649+0000 7f8f9662e740 1 --2- 172.21.15.83:0/1054769664 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bbc2fdf460 0x55bbc2fe95f0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c0cdc80 tx=0x7f8f7c0d94d0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.649+0000 7f8f9662e740 1 -- 172.21.15.83:0/1054769664 shutdown_connections 2024-04-24T08:53:25.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.649+0000 7f8f9662e740 1 --2- 172.21.15.83:0/1054769664 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bbc2fe9b30 0x55bbc2febf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.649+0000 7f8f9662e740 1 --2- 172.21.15.83:0/1054769664 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bbc2fdf460 0x55bbc2fe95f0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.649+0000 7f8f9662e740 1 --2- 172.21.15.83:0/1054769664 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbc2fdeb40 0x55bbc2fdef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.649+0000 7f8f91800640 1 --2- 172.21.15.83:0/3114476126 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f8f700a3da0 0x7f8f700a6260 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8f84005cc0 tx=0x7f8f8400a330 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:25.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.649+0000 7f8f9662e740 1 -- 172.21.15.83:0/1054769664 >> 172.21.15.83:0/1054769664 conn(0x55bbc2fdae60 msgr2=0x55bbc2fdb240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:25.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.649+0000 7f8f9662e740 1 -- 172.21.15.83:0/1054769664 shutdown_connections 2024-04-24T08:53:25.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.649+0000 7f8f9662e740 1 -- 172.21.15.83:0/1054769664 wait complete. 2024-04-24T08:53:25.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.649+0000 7f8f9662e740 1 -- start start 2024-04-24T08:53:25.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.650+0000 7f8f9662e740 1 Processor -- start 2024-04-24T08:53:25.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.650+0000 7f8f9662e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bbc2fdeb40 0x55bbc2fe95d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:25.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.650+0000 7f8f9662e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bbc2fdf460 0x55bbc2fe68c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:25.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.650+0000 7f8f9662e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbc2fe9b30 0x55bbc2fe6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:25.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.650+0000 7f8f9662e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55bbc2fec960 con 0x55bbc2fdf460 2024-04-24T08:53:25.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.650+0000 7f8f9662e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55bbc2efd810 con 0x55bbc2fe9b30 2024-04-24T08:53:25.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.650+0000 7f8f9662e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55bbc2fecae0 con 0x55bbc2fdeb40 2024-04-24T08:53:25.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.650+0000 7f8f9662e740 10 client.0 _ll_register_callbacks cb 0x55bbc2ecb140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:53:25.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.650+0000 7f8f83e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bbc2fdf460 0x55bbc2fe68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:25.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.650+0000 7f8f83e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bbc2fdf460 0x55bbc2fe68c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50630/0 (socket says 172.21.15.83:50630) 2024-04-24T08:53:25.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.650+0000 7f8f83e00640 1 -- 172.21.15.83:0/149181132 learned_addr learned my addr 172.21.15.83:0/149181132 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:53:25.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.651+0000 7f8f90e00640 1 --2- 172.21.15.83:0/149181132 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bbc2fdeb40 0x55bbc2fe95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:25.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.651+0000 7f8f91800640 1 --2- 172.21.15.83:0/149181132 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbc2fe9b30 0x55bbc2fe6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:25.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.651+0000 7f8f83e00640 1 -- 172.21.15.83:0/149181132 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bbc2fdeb40 msgr2=0x55bbc2fe95d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:25.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.651+0000 7f8f83e00640 1 --2- 172.21.15.83:0/149181132 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bbc2fdeb40 0x55bbc2fe95d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.651+0000 7f8f83e00640 1 -- 172.21.15.83:0/149181132 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbc2fe9b30 msgr2=0x55bbc2fe6fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:25.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.651+0000 7f8f83e00640 1 --2- 172.21.15.83:0/149181132 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbc2fe9b30 0x55bbc2fe6fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.651+0000 7f8f83e00640 1 -- 172.21.15.83:0/149181132 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bbc2f0e3f0 con 0x55bbc2fdf460 2024-04-24T08:53:25.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.651+0000 7f8f83e00640 1 --2- 172.21.15.83:0/149181132 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bbc2fdf460 0x55bbc2fe68c0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f8f74000c00 tx=0x7f8f74032ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d83fbbaf1f7a7d8a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:25.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.651+0000 7f8f7aa00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:53:25.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.651+0000 7f8f7aa00640 1 -- 172.21.15.83:0/149181132 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f7403f070 con 0x55bbc2fdf460 2024-04-24T08:53:25.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.651+0000 7f8f7aa00640 1 -- 172.21.15.83:0/149181132 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f8f74004040 con 0x55bbc2fdf460 2024-04-24T08:53:25.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.651+0000 7f8f9662e740 1 -- 172.21.15.83:0/149181132 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f7c09cc60 con 0x55bbc2fdf460 2024-04-24T08:53:25.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.652+0000 7f8f7aa00640 1 -- 172.21.15.83:0/149181132 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8f7404b050 con 0x55bbc2fdf460 2024-04-24T08:53:25.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.652+0000 7f8f9662e740 10 client.16341 fetch_fsmap learned FSMap version 28 2024-04-24T08:53:25.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.652+0000 7f8f9662e740 1 -- 172.21.15.83:0/149181132 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55bbc2c563f0 con 0x55bbc2fdf460 2024-04-24T08:53:25.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.652+0000 7f8f7aa00640 1 -- 172.21.15.83:0/149181132 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8f7c09cc60 con 0x55bbc2fdf460 2024-04-24T08:53:25.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.652+0000 7f8f7aa00640 1 -- 172.21.15.83:0/149181132 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f8f740047e0 con 0x55bbc2fdf460 2024-04-24T08:53:25.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.652+0000 7f8f9662e740 10 client.16341 fetch_fsmap finished waiting for FSMap version 28 2024-04-24T08:53:25.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.652+0000 7f8f9662e740 10 client.16341 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-24T08:53:25.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.652+0000 7f8f9662e740 20 client.16341 populate_metadata read hostname 'smithi083' 2024-04-24T08:53:25.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.652+0000 7f8f9662e740 1 --2- 172.21.15.83:0/149181132 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55bbc2ff9cb0 0x55bbc2ffc0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:25.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.652+0000 7f8f9662e740 4 client.16341 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:53:25.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.652+0000 7f8f9662e740 1 -- 172.21.15.83:0/149181132 --> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bbc2f51bf0 con 0x55bbc2ff9cb0 2024-04-24T08:53:25.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.653+0000 7f8f90e00640 1 --2- 172.21.15.83:0/149181132 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55bbc2ff9cb0 0x55bbc2ffc0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:25.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.654+0000 7f8f90e00640 1 --2- 172.21.15.83:0/149181132 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55bbc2ff9cb0 0x55bbc2ffc0e0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=11c7e9ac705d2541 server_cookie=d7ab3a49f656251a in_seq=0 out_seq=0 2024-04-24T08:53:25.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.654+0000 7f8f7aa00640 10 client.16341 ms_handle_connect on v2:172.21.15.138:6834/1909219758 2024-04-24T08:53:25.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.655+0000 7f8f7aa00640 1 -- 172.21.15.83:0/149181132 <== mds.2 v2:172.21.15.138:6834/1909219758 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bbc2f51bf0 con 0x55bbc2ff9cb0 2024-04-24T08:53:25.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.656+0000 7f8f7aa00640 10 client.16341 handle_command_reply: tid=0 2024-04-24T08:53:25.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.656+0000 7f8f9662e740 1 client.16341 shutdown 2024-04-24T08:53:25.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.656+0000 7f8f9662e740 1 -- 172.21.15.83:0/149181132 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bbc2fdf460 msgr2=0x55bbc2fe68c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:25.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.656+0000 7f8f9662e740 1 --2- 172.21.15.83:0/149181132 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bbc2fdf460 0x55bbc2fe68c0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f8f74000c00 tx=0x7f8f74032ed0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.656+0000 7f8f9662e740 1 -- 172.21.15.83:0/149181132 shutdown_connections 2024-04-24T08:53:25.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.656+0000 7f8f9662e740 1 --2- 172.21.15.83:0/149181132 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55bbc2ff9cb0 0x55bbc2ffc0e0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.656+0000 7f8f9662e740 1 --2- 172.21.15.83:0/149181132 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbc2fe9b30 0x55bbc2fe6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.656+0000 7f8f9662e740 1 --2- 172.21.15.83:0/149181132 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bbc2fdf460 0x55bbc2fe68c0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.656+0000 7f8f7aa00640 0 client.16341 ms_handle_reset on v2:172.21.15.138:6834/1909219758 2024-04-24T08:53:25.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.656+0000 7f8f9662e740 1 --2- 172.21.15.83:0/149181132 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bbc2fdeb40 0x55bbc2fe95d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.656+0000 7f8f9662e740 1 -- 172.21.15.83:0/149181132 >> 172.21.15.83:0/149181132 conn(0x55bbc2fdae60 msgr2=0x55bbc2fe0b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:25.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.656+0000 7f8f9662e740 1 -- 172.21.15.83:0/149181132 shutdown_connections 2024-04-24T08:53:25.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.656+0000 7f8f9662e740 1 -- 172.21.15.83:0/149181132 wait complete. 2024-04-24T08:53:25.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.656+0000 7f8f9662e740 20 client.16341 trim_cache size 0 max 16384 2024-04-24T08:53:25.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.671+0000 7f8f9662e740 1 Processor -- start 2024-04-24T08:53:25.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.671+0000 7f8f9662e740 1 -- start start 2024-04-24T08:53:25.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.671+0000 7f8f9662e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bbc2fdeb40 0x55bbc2fe12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:25.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.671+0000 7f8f9662e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bbc2fe9b30 0x55bbc2fd9f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:25.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.671+0000 7f8f9662e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbc2fda490 0x55bbc2fe9100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:25.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.671+0000 7f8f9662e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8f8c1b1500 con 0x55bbc2fe9b30 2024-04-24T08:53:25.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.671+0000 7f8f9662e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8f8c1b3040 con 0x55bbc2fda490 2024-04-24T08:53:25.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.671+0000 7f8f9662e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8f8c1b1bd0 con 0x55bbc2fdeb40 2024-04-24T08:53:25.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.671+0000 7f8f83e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bbc2fe9b30 0x55bbc2fd9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:25.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.671+0000 7f8f83e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bbc2fe9b30 0x55bbc2fd9f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50642/0 (socket says 172.21.15.83:50642) 2024-04-24T08:53:25.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.671+0000 7f8f83e00640 1 -- 172.21.15.83:0/3529788955 learned_addr learned my addr 172.21.15.83:0/3529788955 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:53:25.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.672+0000 7f8f90e00640 1 --2- 172.21.15.83:0/3529788955 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bbc2fdeb40 0x55bbc2fe12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:25.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.672+0000 7f8f83e00640 1 -- 172.21.15.83:0/3529788955 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bbc2fdeb40 msgr2=0x55bbc2fe12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:25.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.672+0000 7f8f83e00640 1 --2- 172.21.15.83:0/3529788955 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bbc2fdeb40 0x55bbc2fe12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.672+0000 7f8f83e00640 1 -- 172.21.15.83:0/3529788955 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbc2fda490 msgr2=0x55bbc2fe9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:25.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.672+0000 7f8f83e00640 1 --2- 172.21.15.83:0/3529788955 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbc2fda490 0x55bbc2fe9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.672+0000 7f8f83e00640 1 -- 172.21.15.83:0/3529788955 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbc2fe5870 con 0x55bbc2fe9b30 2024-04-24T08:53:25.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.672+0000 7f8f83e00640 1 --2- 172.21.15.83:0/3529788955 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bbc2fe9b30 0x55bbc2fd9f50 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f8f74009850 tx=0x7f8f74031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb36dfa8d2534124 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:25.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.672+0000 7f8f81600640 1 -- 172.21.15.83:0/3529788955 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f74000e60 con 0x55bbc2fe9b30 2024-04-24T08:53:25.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.672+0000 7f8f81600640 1 -- 172.21.15.83:0/3529788955 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8f740046c0 con 0x55bbc2fe9b30 2024-04-24T08:53:25.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.672+0000 7f8f81600640 1 -- 172.21.15.83:0/3529788955 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f7403f070 con 0x55bbc2fe9b30 2024-04-24T08:53:25.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.673+0000 7f8f9662e740 1 -- 172.21.15.83:0/3529788955 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bbc2fe9b30 msgr2=0x55bbc2fd9f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:25.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.673+0000 7f8f9662e740 1 --2- 172.21.15.83:0/3529788955 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bbc2fe9b30 0x55bbc2fd9f50 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f8f74009850 tx=0x7f8f74031400 comp rx=0 tx=0).stop 2024-04-24T08:53:25.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.673+0000 7f8f9662e740 1 -- 172.21.15.83:0/3529788955 shutdown_connections 2024-04-24T08:53:25.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.673+0000 7f8f9662e740 1 --2- 172.21.15.83:0/3529788955 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbc2fda490 0x55bbc2fe9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.673+0000 7f8f9662e740 1 --2- 172.21.15.83:0/3529788955 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bbc2fe9b30 0x55bbc2fd9f50 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.673+0000 7f8f9662e740 1 --2- 172.21.15.83:0/3529788955 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bbc2fdeb40 0x55bbc2fe12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.673+0000 7f8f9662e740 1 -- 172.21.15.83:0/3529788955 >> 172.21.15.83:0/3529788955 conn(0x55bbc2fdae60 msgr2=0x55bbc2fdb240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:25.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.673+0000 7f8f9662e740 1 -- 172.21.15.83:0/3529788955 shutdown_connections 2024-04-24T08:53:25.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.673+0000 7f8f9662e740 1 -- 172.21.15.83:0/3529788955 wait complete. 2024-04-24T08:53:25.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.674+0000 7f8f9662e740 1 -- start start 2024-04-24T08:53:25.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.674+0000 7f8f9662e740 1 Processor -- start 2024-04-24T08:53:25.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.674+0000 7f8f9662e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bbc2fda490 0x55bbc2ff1f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:25.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.674+0000 7f8f9662e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bbc2fdeb40 0x55bbc2fecd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:25.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.674+0000 7f8f9662e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbc2fe9b30 0x55bbc2fed260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:25.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.674+0000 7f8f9662e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8f8c1b2970 con 0x55bbc2fdeb40 2024-04-24T08:53:25.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.674+0000 7f8f9662e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8f8c1b22a0 con 0x55bbc2fe9b30 2024-04-24T08:53:25.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.674+0000 7f8f9662e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8f8c1af4e0 con 0x55bbc2fda490 2024-04-24T08:53:25.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.674+0000 7f8f9662e740 10 client.0 _ll_register_callbacks cb 0x7f8f74009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:53:25.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.674+0000 7f8f83e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bbc2fdeb40 0x55bbc2fecd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:25.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.674+0000 7f8f83e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bbc2fdeb40 0x55bbc2fecd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50652/0 (socket says 172.21.15.83:50652) 2024-04-24T08:53:25.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.674+0000 7f8f83e00640 1 -- 172.21.15.83:0/2443413825 learned_addr learned my addr 172.21.15.83:0/2443413825 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:53:25.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.675+0000 7f8f90e00640 1 --2- 172.21.15.83:0/2443413825 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bbc2fda490 0x55bbc2ff1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:25.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.675+0000 7f8f83e00640 1 -- 172.21.15.83:0/2443413825 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bbc2fda490 msgr2=0x55bbc2ff1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:25.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.675+0000 7f8f91800640 1 --2- 172.21.15.83:0/2443413825 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbc2fe9b30 0x55bbc2fed260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:25.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.675+0000 7f8f83e00640 1 --2- 172.21.15.83:0/2443413825 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bbc2fda490 0x55bbc2ff1f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.675+0000 7f8f83e00640 1 -- 172.21.15.83:0/2443413825 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbc2fe9b30 msgr2=0x55bbc2fed260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:25.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.675+0000 7f8f83e00640 1 --2- 172.21.15.83:0/2443413825 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbc2fe9b30 0x55bbc2fed260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.675+0000 7f8f83e00640 1 -- 172.21.15.83:0/2443413825 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bbc2d207d0 con 0x55bbc2fdeb40 2024-04-24T08:53:25.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.675+0000 7f8f83e00640 1 --2- 172.21.15.83:0/2443413825 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bbc2fdeb40 0x55bbc2fecd20 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f8f7402a910 tx=0x7f8f7402a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b77705f2707598fa server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:25.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.675+0000 7f8f7aa00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:53:25.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.675+0000 7f8f7aa00640 1 -- 172.21.15.83:0/2443413825 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f7405c020 con 0x55bbc2fdeb40 2024-04-24T08:53:25.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.676+0000 7f8f7aa00640 1 -- 172.21.15.83:0/2443413825 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f8f74004870 con 0x55bbc2fdeb40 2024-04-24T08:53:25.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.676+0000 7f8f9662e740 1 -- 172.21.15.83:0/2443413825 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f740046c0 con 0x55bbc2fdeb40 2024-04-24T08:53:25.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.676+0000 7f8f7aa00640 1 -- 172.21.15.83:0/2443413825 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8f74067050 con 0x55bbc2fdeb40 2024-04-24T08:53:25.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.676+0000 7f8f9662e740 10 client.16347 fetch_fsmap learned FSMap version 28 2024-04-24T08:53:25.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.676+0000 7f8f9662e740 1 -- 172.21.15.83:0/2443413825 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55bbc3060d60 con 0x55bbc2fdeb40 2024-04-24T08:53:25.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.676+0000 7f8f7aa00640 1 -- 172.21.15.83:0/2443413825 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8f740046c0 con 0x55bbc2fdeb40 2024-04-24T08:53:25.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.676+0000 7f8f7aa00640 1 -- 172.21.15.83:0/2443413825 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f8f7403b9d0 con 0x55bbc2fdeb40 2024-04-24T08:53:25.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.676+0000 7f8f9662e740 10 client.16347 fetch_fsmap finished waiting for FSMap version 28 2024-04-24T08:53:25.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.676+0000 7f8f9662e740 10 client.16347 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-24T08:53:25.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.676+0000 7f8f9662e740 20 client.16347 populate_metadata read hostname 'smithi083' 2024-04-24T08:53:25.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.676+0000 7f8f9662e740 1 --2- 172.21.15.83:0/2443413825 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55bbc3061470 0x55bbc3063890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:25.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.676+0000 7f8f9662e740 4 client.16347 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T08:53:25.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.676+0000 7f8f9662e740 1 -- 172.21.15.83:0/2443413825 --> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bbc2f4ba60 con 0x55bbc3061470 2024-04-24T08:53:25.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.677+0000 7f8f90e00640 1 --2- 172.21.15.83:0/2443413825 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55bbc3061470 0x55bbc3063890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:25.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.678+0000 7f8f90e00640 1 --2- 172.21.15.83:0/2443413825 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55bbc3061470 0x55bbc3063890 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=db8fc2b308335272 server_cookie=8cc066b82430351b in_seq=0 out_seq=0 2024-04-24T08:53:25.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.678+0000 7f8f7aa00640 10 client.16347 ms_handle_connect on v2:172.21.15.138:6834/1909219758 2024-04-24T08:53:25.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.678+0000 7f8f7aa00640 1 -- 172.21.15.83:0/2443413825 <== mds.2 v2:172.21.15.138:6834/1909219758 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bbc2f4ba60 con 0x55bbc3061470 2024-04-24T08:53:25.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.678+0000 7f8f7aa00640 10 client.16347 handle_command_reply: tid=0 2024-04-24T08:53:25.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.679+0000 7f8f9662e740 1 client.16347 shutdown 2024-04-24T08:53:25.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.679+0000 7f8f9662e740 1 -- 172.21.15.83:0/2443413825 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bbc2fdeb40 msgr2=0x55bbc2fecd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:25.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.679+0000 7f8f9662e740 1 --2- 172.21.15.83:0/2443413825 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bbc2fdeb40 0x55bbc2fecd20 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f8f7402a910 tx=0x7f8f7402a3f0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.679+0000 7f8f9662e740 1 -- 172.21.15.83:0/2443413825 shutdown_connections 2024-04-24T08:53:25.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.679+0000 7f8f9662e740 1 --2- 172.21.15.83:0/2443413825 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55bbc3061470 0x55bbc3063890 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.679+0000 7f8f9662e740 1 --2- 172.21.15.83:0/2443413825 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbc2fe9b30 0x55bbc2fed260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.679+0000 7f8f9662e740 1 --2- 172.21.15.83:0/2443413825 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bbc2fdeb40 0x55bbc2fecd20 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.679+0000 7f8f7aa00640 0 client.16347 ms_handle_reset on v2:172.21.15.138:6834/1909219758 2024-04-24T08:53:25.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.679+0000 7f8f9662e740 1 --2- 172.21.15.83:0/2443413825 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bbc2fda490 0x55bbc2ff1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.679+0000 7f8f9662e740 1 -- 172.21.15.83:0/2443413825 >> 172.21.15.83:0/2443413825 conn(0x55bbc2fdae60 msgr2=0x55bbc2fe0290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:25.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.679+0000 7f8f9662e740 1 -- 172.21.15.83:0/2443413825 shutdown_connections 2024-04-24T08:53:25.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.679+0000 7f8f9662e740 1 -- 172.21.15.83:0/2443413825 wait complete. 2024-04-24T08:53:25.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.679+0000 7f8f9662e740 20 client.16347 trim_cache size 0 max 16384 2024-04-24T08:53:25.689 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T08:53:25.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.680+0000 7f8f92200640 1 -- 172.21.15.83:0/3114476126 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f8f700a3da0 msgr2=0x7f8f700a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:25.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.680+0000 7f8f92200640 1 --2- 172.21.15.83:0/3114476126 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f8f700a3da0 0x7f8f700a6260 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8f84005cc0 tx=0x7f8f8400a330 comp rx=0 tx=0).stop 2024-04-24T08:53:25.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.680+0000 7f8f92200640 1 -- 172.21.15.83:0/3114476126 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f8c104320 msgr2=0x7f8f8c1a05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:25.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.680+0000 7f8f92200640 1 --2- 172.21.15.83:0/3114476126 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f8c104320 0x7f8f8c1a05f0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c00c970 tx=0x7f8f7c00ce30 comp rx=0 tx=0).stop 2024-04-24T08:53:25.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.680+0000 7f8f92200640 1 -- 172.21.15.83:0/3114476126 shutdown_connections 2024-04-24T08:53:25.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.680+0000 7f8f92200640 1 --2- 172.21.15.83:0/3114476126 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f8f700a3da0 0x7f8f700a6260 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.680+0000 7f8f92200640 1 --2- 172.21.15.83:0/3114476126 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8f8c19a6b0 0x7f8f8c19ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.680+0000 7f8f92200640 1 --2- 172.21.15.83:0/3114476126 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f8c1093f0 0x7f8f8c1a0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.680+0000 7f8f92200640 1 --2- 172.21.15.83:0/3114476126 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f8c104320 0x7f8f8c1a05f0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.680+0000 7f8f92200640 1 -- 172.21.15.83:0/3114476126 >> 172.21.15.83:0/3114476126 conn(0x7f8f8c0ffb40 msgr2=0x7f8f8c100e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:25.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.680+0000 7f8f92200640 1 -- 172.21.15.83:0/3114476126 shutdown_connections 2024-04-24T08:53:25.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.680+0000 7f8f92200640 1 -- 172.21.15.83:0/3114476126 wait complete. 2024-04-24T08:53:25.697 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T08:53:25.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.890+0000 7f9c86a00640 1 -- 172.21.15.83:0/2065385063 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c801130b0 msgr2=0x7f9c801154a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:25.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.890+0000 7f9c86a00640 1 --2- 172.21.15.83:0/2065385063 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c801130b0 0x7f9c801154a0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f9c78009930 tx=0x7f9c7802a4d0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.890+0000 7f9c86a00640 1 -- 172.21.15.83:0/2065385063 shutdown_connections 2024-04-24T08:53:25.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.890+0000 7f9c86a00640 1 --2- 172.21.15.83:0/2065385063 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c801130b0 0x7f9c801154a0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.890+0000 7f9c86a00640 1 --2- 172.21.15.83:0/2065385063 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9c80074bf0 0x7f9c80112b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.890+0000 7f9c86a00640 1 --2- 172.21.15.83:0/2065385063 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c800742d0 0x7f9c800746b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.890+0000 7f9c86a00640 1 -- 172.21.15.83:0/2065385063 >> 172.21.15.83:0/2065385063 conn(0x7f9c80079450 msgr2=0x7f9c80079860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:25.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.890+0000 7f9c86a00640 1 -- 172.21.15.83:0/2065385063 shutdown_connections 2024-04-24T08:53:25.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.890+0000 7f9c86a00640 1 -- 172.21.15.83:0/2065385063 wait complete. 2024-04-24T08:53:25.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.891+0000 7f9c86a00640 1 Processor -- start 2024-04-24T08:53:25.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.891+0000 7f9c86a00640 1 -- start start 2024-04-24T08:53:25.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.892+0000 7f9c86a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c800742d0 0x7f9c8019ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:25.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.892+0000 7f9c86a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9c80074bf0 0x7f9c8019f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:25.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.892+0000 7f9c86a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c801130b0 0x7f9c801a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:25.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.892+0000 7f9c86a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9c80118880 con 0x7f9c801130b0 2024-04-24T08:53:25.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.892+0000 7f9c86a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9c80118700 con 0x7f9c800742d0 2024-04-24T08:53:25.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.892+0000 7f9c86a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9c80118a00 con 0x7f9c80074bf0 2024-04-24T08:53:25.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.892+0000 7f9c84a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c801130b0 0x7f9c801a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:25.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.892+0000 7f9c84a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c801130b0 0x7f9c801a3690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50670/0 (socket says 172.21.15.83:50670) 2024-04-24T08:53:25.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.892+0000 7f9c84a00640 1 -- 172.21.15.83:0/2609966859 learned_addr learned my addr 172.21.15.83:0/2609966859 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:53:25.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.892+0000 7f9c7fe00640 1 --2- 172.21.15.83:0/2609966859 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c800742d0 0x7f9c8019ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:25.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.892+0000 7f9c77400640 1 --2- 172.21.15.83:0/2609966859 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9c80074bf0 0x7f9c8019f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:25.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.892+0000 7f9c84a00640 1 -- 172.21.15.83:0/2609966859 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9c80074bf0 msgr2=0x7f9c8019f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:25.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.892+0000 7f9c84a00640 1 --2- 172.21.15.83:0/2609966859 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9c80074bf0 0x7f9c8019f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.892+0000 7f9c84a00640 1 -- 172.21.15.83:0/2609966859 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c800742d0 msgr2=0x7f9c8019ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:25.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.892+0000 7f9c84a00640 1 --2- 172.21.15.83:0/2609966859 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c800742d0 0x7f9c8019ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.892+0000 7f9c84a00640 1 -- 172.21.15.83:0/2609966859 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c801a3dc0 con 0x7f9c801130b0 2024-04-24T08:53:25.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.892+0000 7f9c7fe00640 1 --2- 172.21.15.83:0/2609966859 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c800742d0 0x7f9c8019ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:53:25.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.893+0000 7f9c84a00640 1 --2- 172.21.15.83:0/2609966859 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c801130b0 0x7f9c801a3690 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f9c78005940 tx=0x7f9c780042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:25.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.893+0000 7f9c7d600640 1 -- 172.21.15.83:0/2609966859 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c7803f030 con 0x7f9c801130b0 2024-04-24T08:53:25.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.893+0000 7f9c7d600640 1 -- 172.21.15.83:0/2609966859 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9c78003cb0 con 0x7f9c801130b0 2024-04-24T08:53:25.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.893+0000 7f9c7d600640 1 -- 172.21.15.83:0/2609966859 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c78035070 con 0x7f9c801130b0 2024-04-24T08:53:25.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.893+0000 7f9c86a00640 1 -- 172.21.15.83:0/2609966859 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c801a4050 con 0x7f9c801130b0 2024-04-24T08:53:25.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.893+0000 7f9c86a00640 1 -- 172.21.15.83:0/2609966859 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c801ab940 con 0x7f9c801130b0 2024-04-24T08:53:25.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.896+0000 7f9c7d600640 1 -- 172.21.15.83:0/2609966859 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9c78042030 con 0x7f9c801130b0 2024-04-24T08:53:25.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.896+0000 7f9c89781740 1 Processor -- start 2024-04-24T08:53:25.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.897+0000 7f9c89781740 1 -- start start 2024-04-24T08:53:25.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.897+0000 7f9c89781740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d9e8eb1950 0x55d9e8eb1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:25.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.897+0000 7f9c89781740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9e8eb2270 0x55d9e8ebc400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:25.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.897+0000 7f9c89781740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d9e8ebc940 0x55d9e8ebed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:25.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.897+0000 7f9c89781740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d9e8e171d0 con 0x55d9e8eb2270 2024-04-24T08:53:25.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.897+0000 7f9c89781740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d9e8dbd450 con 0x55d9e8ebc940 2024-04-24T08:53:25.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.897+0000 7f9c89781740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d9e8e13190 con 0x55d9e8eb1950 2024-04-24T08:53:25.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.897+0000 7f9c7fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9e8eb2270 0x55d9e8ebc400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:25.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.897+0000 7f9c7fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9e8eb2270 0x55d9e8ebc400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50684/0 (socket says 172.21.15.83:50684) 2024-04-24T08:53:25.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.897+0000 7f9c7fe00640 1 -- 172.21.15.83:0/1212876244 learned_addr learned my addr 172.21.15.83:0/1212876244 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:53:25.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.898+0000 7f9c77400640 1 --2- 172.21.15.83:0/1212876244 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d9e8eb1950 0x55d9e8eb1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:25.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.898+0000 7f9c7fe00640 1 -- 172.21.15.83:0/1212876244 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d9e8eb1950 msgr2=0x55d9e8eb1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:25.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.898+0000 7f9c7fe00640 1 --2- 172.21.15.83:0/1212876244 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d9e8eb1950 0x55d9e8eb1d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.898+0000 7f9c7fe00640 1 -- 172.21.15.83:0/1212876244 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d9e8ebc940 msgr2=0x55d9e8ebed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:25.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.898+0000 7f9c7fe00640 1 --2- 172.21.15.83:0/1212876244 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d9e8ebc940 0x55d9e8ebed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.898+0000 7f9c7fe00640 1 -- 172.21.15.83:0/1212876244 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9e8e7a310 con 0x55d9e8eb2270 2024-04-24T08:53:25.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.898+0000 7f9c7d600640 1 --2- 172.21.15.83:0/2609966859 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f9c640a3da0 0x7f9c640a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:25.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.898+0000 7f9c7fe00640 1 --2- 172.21.15.83:0/1212876244 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9e8eb2270 0x55d9e8ebc400 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f9c70008f10 tx=0x7f9c7000d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a40cc21cf307a8d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:25.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.898+0000 7f9c7d600640 1 -- 172.21.15.83:0/2609966859 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f9c780f10f0 con 0x7f9c801130b0 2024-04-24T08:53:25.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.898+0000 7f9c77400640 1 --2- 172.21.15.83:0/2609966859 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f9c640a3da0 0x7f9c640a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:25.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.899+0000 7f9c75200640 1 -- 172.21.15.83:0/1212876244 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c7000f830 con 0x55d9e8eb2270 2024-04-24T08:53:25.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.899+0000 7f9c75200640 1 -- 172.21.15.83:0/1212876244 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9c70018480 con 0x55d9e8eb2270 2024-04-24T08:53:25.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.899+0000 7f9c75200640 1 -- 172.21.15.83:0/1212876244 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c7000e5b0 con 0x55d9e8eb2270 2024-04-24T08:53:25.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.899+0000 7f9c89781740 1 -- 172.21.15.83:0/1212876244 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9e8eb2270 msgr2=0x55d9e8ebc400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:25.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.899+0000 7f9c89781740 1 --2- 172.21.15.83:0/1212876244 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9e8eb2270 0x55d9e8ebc400 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f9c70008f10 tx=0x7f9c7000d710 comp rx=0 tx=0).stop 2024-04-24T08:53:25.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.899+0000 7f9c77400640 1 --2- 172.21.15.83:0/2609966859 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f9c640a3da0 0x7f9c640a6260 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x55d9e8eaee00 tx=0x7f9c68005e90 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:25.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.899+0000 7f9c89781740 1 -- 172.21.15.83:0/1212876244 shutdown_connections 2024-04-24T08:53:25.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.899+0000 7f9c89781740 1 --2- 172.21.15.83:0/1212876244 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d9e8ebc940 0x55d9e8ebed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.899+0000 7f9c89781740 1 --2- 172.21.15.83:0/1212876244 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9e8eb2270 0x55d9e8ebc400 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.899+0000 7f9c89781740 1 --2- 172.21.15.83:0/1212876244 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d9e8eb1950 0x55d9e8eb1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.899+0000 7f9c89781740 1 -- 172.21.15.83:0/1212876244 >> 172.21.15.83:0/1212876244 conn(0x55d9e8eadc70 msgr2=0x55d9e8eae050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:25.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.899+0000 7f9c89781740 1 -- 172.21.15.83:0/1212876244 shutdown_connections 2024-04-24T08:53:25.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.900+0000 7f9c89781740 1 -- 172.21.15.83:0/1212876244 wait complete. 2024-04-24T08:53:25.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.900+0000 7f9c89781740 1 -- start start 2024-04-24T08:53:25.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.900+0000 7f9c89781740 1 Processor -- start 2024-04-24T08:53:25.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.900+0000 7f9c89781740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9e8eb1950 0x55d9e8ebc300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:25.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.901+0000 7f9c89781740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d9e8eb2270 0x55d9e8eb95f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:25.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.901+0000 7f9c89781740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d9e8ebc940 0x55d9e8eb9c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:25.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.901+0000 7f9c89781740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d9e8ebf7d0 con 0x55d9e8eb1950 2024-04-24T08:53:25.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.901+0000 7f9c89781740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d9e8dd0810 con 0x55d9e8eb2270 2024-04-24T08:53:25.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.901+0000 7f9c89781740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d9e8ebf950 con 0x55d9e8ebc940 2024-04-24T08:53:25.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.901+0000 7f9c89781740 10 client.0 _ll_register_callbacks cb 0x55d9e8d9e140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:53:25.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.901+0000 7f9c7fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9e8eb1950 0x55d9e8ebc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:25.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.901+0000 7f9c7fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9e8eb1950 0x55d9e8ebc300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50694/0 (socket says 172.21.15.83:50694) 2024-04-24T08:53:25.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.901+0000 7f9c7fe00640 1 -- 172.21.15.83:0/3953698826 learned_addr learned my addr 172.21.15.83:0/3953698826 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:53:25.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.901+0000 7f9c77400640 1 --2- 172.21.15.83:0/3953698826 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d9e8eb2270 0x55d9e8eb95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:25.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.901+0000 7f9c7fe00640 1 -- 172.21.15.83:0/3953698826 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d9e8ebc940 msgr2=0x55d9e8eb9c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:25.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.901+0000 7f9c7fe00640 1 --2- 172.21.15.83:0/3953698826 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d9e8ebc940 0x55d9e8eb9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.901+0000 7f9c7fe00640 1 -- 172.21.15.83:0/3953698826 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d9e8eb2270 msgr2=0x55d9e8eb95f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:25.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.901+0000 7f9c7fe00640 1 --2- 172.21.15.83:0/3953698826 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d9e8eb2270 0x55d9e8eb95f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.901+0000 7f9c7fe00640 1 -- 172.21.15.83:0/3953698826 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9e8de13f0 con 0x55d9e8eb1950 2024-04-24T08:53:25.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.902+0000 7f9c7fe00640 1 --2- 172.21.15.83:0/3953698826 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9e8eb1950 0x55d9e8ebc300 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f9c70018c80 tx=0x7f9c70008630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3f95737ddb8481c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:25.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.902+0000 7f9c6d600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:53:25.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.902+0000 7f9c6d600640 1 -- 172.21.15.83:0/3953698826 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c70028060 con 0x55d9e8eb1950 2024-04-24T08:53:25.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.902+0000 7f9c89781740 1 -- 172.21.15.83:0/3953698826 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c70018480 con 0x55d9e8eb1950 2024-04-24T08:53:25.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.902+0000 7f9c6d600640 1 -- 172.21.15.83:0/3953698826 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f9c70004d10 con 0x55d9e8eb1950 2024-04-24T08:53:25.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.902+0000 7f9c6d600640 1 -- 172.21.15.83:0/3953698826 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9c7002f050 con 0x55d9e8eb1950 2024-04-24T08:53:25.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.902+0000 7f9c89781740 10 client.16359 fetch_fsmap learned FSMap version 28 2024-04-24T08:53:25.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.902+0000 7f9c89781740 1 -- 172.21.15.83:0/3953698826 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d9e8b293f0 con 0x55d9e8eb1950 2024-04-24T08:53:25.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.902+0000 7f9c6d600640 1 -- 172.21.15.83:0/3953698826 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9c70018480 con 0x55d9e8eb1950 2024-04-24T08:53:25.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.903+0000 7f9c6d600640 1 -- 172.21.15.83:0/3953698826 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f9c700053a0 con 0x55d9e8eb1950 2024-04-24T08:53:25.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.903+0000 7f9c89781740 10 client.16359 fetch_fsmap finished waiting for FSMap version 28 2024-04-24T08:53:25.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.903+0000 7f9c89781740 10 client.16359 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-24T08:53:25.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.903+0000 7f9c89781740 20 client.16359 populate_metadata read hostname 'smithi083' 2024-04-24T08:53:25.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.903+0000 7f9c89781740 1 --2- 172.21.15.83:0/3953698826 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x55d9e8ecca60 0x55d9e8ecee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:25.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.903+0000 7f9c89781740 4 client.16359 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:53:25.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.903+0000 7f9c89781740 1 -- 172.21.15.83:0/3953698826 --> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d9e8e24bf0 con 0x55d9e8ecca60 2024-04-24T08:53:25.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.903+0000 7f9c77400640 1 --2- 172.21.15.83:0/3953698826 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x55d9e8ecca60 0x55d9e8ecee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:25.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.905+0000 7f9c6d600640 10 client.16359 ms_handle_connect on v2:172.21.15.138:6836/900566754 2024-04-24T08:53:25.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.904+0000 7f9c77400640 1 --2- 172.21.15.83:0/3953698826 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x55d9e8ecca60 0x55d9e8ecee50 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cd3d4980b05763a8 server_cookie=f76721c84a1d2d5e in_seq=0 out_seq=0 2024-04-24T08:53:25.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.906+0000 7f9c6d600640 1 -- 172.21.15.83:0/3953698826 <== mds.1 v2:172.21.15.138:6836/900566754 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d9e8e24bf0 con 0x55d9e8ecca60 2024-04-24T08:53:25.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.906+0000 7f9c6d600640 10 client.16359 handle_command_reply: tid=0 2024-04-24T08:53:25.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.906+0000 7f9c89781740 1 client.16359 shutdown 2024-04-24T08:53:25.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.907+0000 7f9c89781740 1 -- 172.21.15.83:0/3953698826 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9e8eb1950 msgr2=0x55d9e8ebc300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:25.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.907+0000 7f9c89781740 1 --2- 172.21.15.83:0/3953698826 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9e8eb1950 0x55d9e8ebc300 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f9c70018c80 tx=0x7f9c70008630 comp rx=0 tx=0).stop 2024-04-24T08:53:25.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.907+0000 7f9c89781740 1 -- 172.21.15.83:0/3953698826 shutdown_connections 2024-04-24T08:53:25.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.907+0000 7f9c89781740 1 --2- 172.21.15.83:0/3953698826 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x55d9e8ecca60 0x55d9e8ecee50 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.907+0000 7f9c89781740 1 --2- 172.21.15.83:0/3953698826 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d9e8ebc940 0x55d9e8eb9c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.907+0000 7f9c89781740 1 --2- 172.21.15.83:0/3953698826 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d9e8eb2270 0x55d9e8eb95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.907+0000 7f9c6d600640 0 client.16359 ms_handle_reset on v2:172.21.15.138:6836/900566754 2024-04-24T08:53:25.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.907+0000 7f9c89781740 1 --2- 172.21.15.83:0/3953698826 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9e8eb1950 0x55d9e8ebc300 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.907+0000 7f9c89781740 1 -- 172.21.15.83:0/3953698826 >> 172.21.15.83:0/3953698826 conn(0x55d9e8eadc70 msgr2=0x55d9e8eacdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:25.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.907+0000 7f9c89781740 1 -- 172.21.15.83:0/3953698826 shutdown_connections 2024-04-24T08:53:25.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.907+0000 7f9c89781740 1 -- 172.21.15.83:0/3953698826 wait complete. 2024-04-24T08:53:25.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.907+0000 7f9c89781740 20 client.16359 trim_cache size 0 max 16384 2024-04-24T08:53:25.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.921+0000 7f9c89781740 1 Processor -- start 2024-04-24T08:53:25.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.921+0000 7f9c89781740 1 -- start start 2024-04-24T08:53:25.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.921+0000 7f9c89781740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9e8eb1950 0x55d9e8eb8770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:25.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.921+0000 7f9c89781740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d9e8eb2270 0x55d9e8eb37f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:25.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.922+0000 7f9c89781740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d9e8ebc940 0x55d9e8ebbe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:25.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.922+0000 7f9c89781740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9c801b5b30 con 0x55d9e8eb1950 2024-04-24T08:53:25.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.922+0000 7f9c89781740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9c801b7670 con 0x55d9e8ebc940 2024-04-24T08:53:25.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.922+0000 7f9c89781740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9c801b6200 con 0x55d9e8eb2270 2024-04-24T08:53:25.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.922+0000 7f9c7fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9e8eb1950 0x55d9e8eb8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:25.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.922+0000 7f9c7fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9e8eb1950 0x55d9e8eb8770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50702/0 (socket says 172.21.15.83:50702) 2024-04-24T08:53:25.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.922+0000 7f9c7fe00640 1 -- 172.21.15.83:0/2083480195 learned_addr learned my addr 172.21.15.83:0/2083480195 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:53:25.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.922+0000 7f9c84a00640 1 --2- 172.21.15.83:0/2083480195 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d9e8ebc940 0x55d9e8ebbe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:25.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.922+0000 7f9c77400640 1 --2- 172.21.15.83:0/2083480195 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d9e8eb2270 0x55d9e8eb37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:25.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.922+0000 7f9c7fe00640 1 -- 172.21.15.83:0/2083480195 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d9e8eb2270 msgr2=0x55d9e8eb37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:25.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.922+0000 7f9c7fe00640 1 --2- 172.21.15.83:0/2083480195 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d9e8eb2270 0x55d9e8eb37f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.922+0000 7f9c7fe00640 1 -- 172.21.15.83:0/2083480195 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d9e8ebc940 msgr2=0x55d9e8ebbe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:25.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.922+0000 7f9c7fe00640 1 --2- 172.21.15.83:0/2083480195 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d9e8ebc940 0x55d9e8ebbe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.922+0000 7f9c7fe00640 1 -- 172.21.15.83:0/2083480195 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9e8bf37d0 con 0x55d9e8eb1950 2024-04-24T08:53:25.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.922+0000 7f9c84a00640 1 --2- 172.21.15.83:0/2083480195 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d9e8ebc940 0x55d9e8ebbe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:53:25.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.922+0000 7f9c77400640 1 --2- 172.21.15.83:0/2083480195 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d9e8eb2270 0x55d9e8eb37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:53:25.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.923+0000 7f9c7fe00640 1 --2- 172.21.15.83:0/2083480195 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9e8eb1950 0x55d9e8eb8770 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f9c70008ca0 tx=0x7f9c7000b420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ced86c308a07558 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:25.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.923+0000 7f9c6fe00640 1 -- 172.21.15.83:0/2083480195 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c70028040 con 0x55d9e8eb1950 2024-04-24T08:53:25.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.923+0000 7f9c6fe00640 1 -- 172.21.15.83:0/2083480195 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9c70018a60 con 0x55d9e8eb1950 2024-04-24T08:53:25.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.923+0000 7f9c6fe00640 1 -- 172.21.15.83:0/2083480195 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c70004f70 con 0x55d9e8eb1950 2024-04-24T08:53:25.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.924+0000 7f9c89781740 1 -- 172.21.15.83:0/2083480195 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9e8eb1950 msgr2=0x55d9e8eb8770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:25.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.924+0000 7f9c89781740 1 --2- 172.21.15.83:0/2083480195 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9e8eb1950 0x55d9e8eb8770 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f9c70008ca0 tx=0x7f9c7000b420 comp rx=0 tx=0).stop 2024-04-24T08:53:25.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.924+0000 7f9c89781740 1 -- 172.21.15.83:0/2083480195 shutdown_connections 2024-04-24T08:53:25.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.924+0000 7f9c89781740 1 --2- 172.21.15.83:0/2083480195 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d9e8ebc940 0x55d9e8ebbe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.924+0000 7f9c89781740 1 --2- 172.21.15.83:0/2083480195 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d9e8eb2270 0x55d9e8eb37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.924+0000 7f9c89781740 1 --2- 172.21.15.83:0/2083480195 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9e8eb1950 0x55d9e8eb8770 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.924+0000 7f9c89781740 1 -- 172.21.15.83:0/2083480195 >> 172.21.15.83:0/2083480195 conn(0x55d9e8eadc70 msgr2=0x55d9e8eb30e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:25.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.924+0000 7f9c89781740 1 -- 172.21.15.83:0/2083480195 shutdown_connections 2024-04-24T08:53:25.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.924+0000 7f9c89781740 1 -- 172.21.15.83:0/2083480195 wait complete. 2024-04-24T08:53:25.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.924+0000 7f9c89781740 1 -- start start 2024-04-24T08:53:25.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.925+0000 7f9c89781740 1 Processor -- start 2024-04-24T08:53:25.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.925+0000 7f9c89781740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9e8eb1950 0x55d9e8ec4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:25.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.925+0000 7f9c89781740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d9e8eb2270 0x55d9e8ec5110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:25.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.925+0000 7f9c89781740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d9e8ebc940 0x55d9e8ebdad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:25.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.925+0000 7f9c89781740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9c801b6fa0 con 0x55d9e8eb1950 2024-04-24T08:53:25.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.925+0000 7f9c89781740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9c801b68d0 con 0x55d9e8eb2270 2024-04-24T08:53:25.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.925+0000 7f9c89781740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9c801b3b10 con 0x55d9e8ebc940 2024-04-24T08:53:25.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.925+0000 7f9c89781740 10 client.0 _ll_register_callbacks cb 0x7f9c70008d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:53:25.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.925+0000 7f9c7fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9e8eb1950 0x55d9e8ec4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:25.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.925+0000 7f9c7fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9e8eb1950 0x55d9e8ec4bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50712/0 (socket says 172.21.15.83:50712) 2024-04-24T08:53:25.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.925+0000 7f9c7fe00640 1 -- 172.21.15.83:0/3261458605 learned_addr learned my addr 172.21.15.83:0/3261458605 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:53:25.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.926+0000 7f9c7fe00640 1 -- 172.21.15.83:0/3261458605 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d9e8ebc940 msgr2=0x55d9e8ebdad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:25.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.926+0000 7f9c7fe00640 1 --2- 172.21.15.83:0/3261458605 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d9e8ebc940 0x55d9e8ebdad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.926+0000 7f9c7fe00640 1 -- 172.21.15.83:0/3261458605 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d9e8eb2270 msgr2=0x55d9e8ec5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:25.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.926+0000 7f9c7fe00640 1 --2- 172.21.15.83:0/3261458605 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d9e8eb2270 0x55d9e8ec5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.926+0000 7f9c7fe00640 1 -- 172.21.15.83:0/3261458605 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9e8ebe010 con 0x55d9e8eb1950 2024-04-24T08:53:25.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.926+0000 7f9c7fe00640 1 --2- 172.21.15.83:0/3261458605 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9e8eb1950 0x55d9e8ec4bd0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f9c7000b4c0 tx=0x7f9c70008800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78d04414ff0c313d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:25.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.926+0000 7f9c6d600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:53:25.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.926+0000 7f9c6d600640 1 -- 172.21.15.83:0/3261458605 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c70028070 con 0x55d9e8eb1950 2024-04-24T08:53:25.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.926+0000 7f9c6d600640 1 -- 172.21.15.83:0/3261458605 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f9c7000f980 con 0x55d9e8eb1950 2024-04-24T08:53:25.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.926+0000 7f9c89781740 1 -- 172.21.15.83:0/3261458605 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c70018a60 con 0x55d9e8eb1950 2024-04-24T08:53:25.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.926+0000 7f9c6d600640 1 -- 172.21.15.83:0/3261458605 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9c7000fb80 con 0x55d9e8eb1950 2024-04-24T08:53:25.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.926+0000 7f9c89781740 10 client.16365 fetch_fsmap learned FSMap version 28 2024-04-24T08:53:25.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.926+0000 7f9c89781740 1 -- 172.21.15.83:0/3261458605 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d9e8f33450 con 0x55d9e8eb1950 2024-04-24T08:53:25.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.927+0000 7f9c6d600640 1 -- 172.21.15.83:0/3261458605 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9c70018a60 con 0x55d9e8eb1950 2024-04-24T08:53:25.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.927+0000 7f9c6d600640 1 -- 172.21.15.83:0/3261458605 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f9c7000b660 con 0x55d9e8eb1950 2024-04-24T08:53:25.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.927+0000 7f9c89781740 10 client.16365 fetch_fsmap finished waiting for FSMap version 28 2024-04-24T08:53:25.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.927+0000 7f9c89781740 10 client.16365 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-24T08:53:25.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.927+0000 7f9c89781740 20 client.16365 populate_metadata read hostname 'smithi083' 2024-04-24T08:53:25.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.927+0000 7f9c89781740 1 --2- 172.21.15.83:0/3261458605 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x55d9e8f33ab0 0x55d9e8f35ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:25.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.927+0000 7f9c89781740 4 client.16365 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T08:53:25.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.927+0000 7f9c89781740 1 -- 172.21.15.83:0/3261458605 --> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d9e8e1ea60 con 0x55d9e8f33ab0 2024-04-24T08:53:25.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.928+0000 7f9c77400640 1 --2- 172.21.15.83:0/3261458605 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x55d9e8f33ab0 0x55d9e8f35ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:25.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.928+0000 7f9c77400640 1 --2- 172.21.15.83:0/3261458605 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x55d9e8f33ab0 0x55d9e8f35ec0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1fdbfe13489bce78 server_cookie=26546c8147ee0ae7 in_seq=0 out_seq=0 2024-04-24T08:53:25.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.929+0000 7f9c6d600640 10 client.16365 ms_handle_connect on v2:172.21.15.138:6836/900566754 2024-04-24T08:53:25.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.929+0000 7f9c6d600640 1 -- 172.21.15.83:0/3261458605 <== mds.1 v2:172.21.15.138:6836/900566754 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d9e8e1ea60 con 0x55d9e8f33ab0 2024-04-24T08:53:25.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.929+0000 7f9c6d600640 10 client.16365 handle_command_reply: tid=0 2024-04-24T08:53:25.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.929+0000 7f9c89781740 1 client.16365 shutdown 2024-04-24T08:53:25.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.929+0000 7f9c89781740 1 -- 172.21.15.83:0/3261458605 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9e8eb1950 msgr2=0x55d9e8ec4bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:25.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.929+0000 7f9c89781740 1 --2- 172.21.15.83:0/3261458605 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9e8eb1950 0x55d9e8ec4bd0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f9c7000b4c0 tx=0x7f9c70008800 comp rx=0 tx=0).stop 2024-04-24T08:53:25.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.929+0000 7f9c89781740 1 -- 172.21.15.83:0/3261458605 shutdown_connections 2024-04-24T08:53:25.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.929+0000 7f9c89781740 1 --2- 172.21.15.83:0/3261458605 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x55d9e8f33ab0 0x55d9e8f35ec0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.929+0000 7f9c89781740 1 --2- 172.21.15.83:0/3261458605 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d9e8ebc940 0x55d9e8ebdad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.929+0000 7f9c89781740 1 --2- 172.21.15.83:0/3261458605 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d9e8eb2270 0x55d9e8ec5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.929+0000 7f9c6d600640 0 client.16365 ms_handle_reset on v2:172.21.15.138:6836/900566754 2024-04-24T08:53:25.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.929+0000 7f9c89781740 1 --2- 172.21.15.83:0/3261458605 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9e8eb1950 0x55d9e8ec4bd0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.929+0000 7f9c89781740 1 -- 172.21.15.83:0/3261458605 >> 172.21.15.83:0/3261458605 conn(0x55d9e8eadc70 msgr2=0x55d9e8eb3930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:25.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.929+0000 7f9c89781740 1 -- 172.21.15.83:0/3261458605 shutdown_connections 2024-04-24T08:53:25.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.929+0000 7f9c89781740 1 -- 172.21.15.83:0/3261458605 wait complete. 2024-04-24T08:53:25.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.930+0000 7f9c89781740 20 client.16365 trim_cache size 0 max 16384 2024-04-24T08:53:25.933 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T08:53:25.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.930+0000 7f9c86a00640 1 -- 172.21.15.83:0/2609966859 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f9c640a3da0 msgr2=0x7f9c640a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:25.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.930+0000 7f9c86a00640 1 --2- 172.21.15.83:0/2609966859 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f9c640a3da0 0x7f9c640a6260 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x55d9e8eaee00 tx=0x7f9c68005e90 comp rx=0 tx=0).stop 2024-04-24T08:53:25.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.930+0000 7f9c86a00640 1 -- 172.21.15.83:0/2609966859 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c801130b0 msgr2=0x7f9c801a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:25.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.930+0000 7f9c86a00640 1 --2- 172.21.15.83:0/2609966859 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c801130b0 0x7f9c801a3690 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f9c78005940 tx=0x7f9c780042f0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.930+0000 7f9c86a00640 1 -- 172.21.15.83:0/2609966859 shutdown_connections 2024-04-24T08:53:25.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.930+0000 7f9c86a00640 1 --2- 172.21.15.83:0/2609966859 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f9c640a3da0 0x7f9c640a6260 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.930+0000 7f9c86a00640 1 --2- 172.21.15.83:0/2609966859 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c801130b0 0x7f9c801a3690 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.930+0000 7f9c86a00640 1 --2- 172.21.15.83:0/2609966859 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9c80074bf0 0x7f9c8019f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.930+0000 7f9c86a00640 1 --2- 172.21.15.83:0/2609966859 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c800742d0 0x7f9c8019ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:25.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.930+0000 7f9c86a00640 1 -- 172.21.15.83:0/2609966859 >> 172.21.15.83:0/2609966859 conn(0x7f9c80079450 msgr2=0x7f9c800fefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:25.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.930+0000 7f9c86a00640 1 -- 172.21.15.83:0/2609966859 shutdown_connections 2024-04-24T08:53:25.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:25.931+0000 7f9c86a00640 1 -- 172.21.15.83:0/2609966859 wait complete. 2024-04-24T08:53:25.939 INFO:tasks.workunit.client.0.smithi083.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-24T08:53:25.945 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:53:25 smithi083 ceph-mon[27907]: pgmap v57: 129 pgs: 129 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-24T08:53:25.945 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:53:25 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/2342997206' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:53:25.946 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T08:53:26.080 INFO:tasks.workunit.client.0.smithi083.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-24T08:53:26.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.128+0000 7f5461000640 1 -- 172.21.15.83:0/53283039 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f545c109580 msgr2=0x7f545c111150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:26.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.128+0000 7f5461000640 1 --2- 172.21.15.83:0/53283039 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f545c109580 0x7f545c111150 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f5454009ca0 tx=0x7f545402a8c0 comp rx=0 tx=0).stop 2024-04-24T08:53:26.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.128+0000 7f5461000640 1 -- 172.21.15.83:0/53283039 shutdown_connections 2024-04-24T08:53:26.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.128+0000 7f5461000640 1 --2- 172.21.15.83:0/53283039 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f545c109580 0x7f545c111150 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:26.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.128+0000 7f5461000640 1 --2- 172.21.15.83:0/53283039 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f545c104e60 0x7f545c108ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:26.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.128+0000 7f5461000640 1 --2- 172.21.15.83:0/53283039 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f545c1044b0 0x7f545c104890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:26.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.128+0000 7f5461000640 1 -- 172.21.15.83:0/53283039 >> 172.21.15.83:0/53283039 conn(0x7f545c0ffd90 msgr2=0x7f545c1021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:26.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.128+0000 7f5461000640 1 -- 172.21.15.83:0/53283039 shutdown_connections 2024-04-24T08:53:26.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.128+0000 7f5461000640 1 -- 172.21.15.83:0/53283039 wait complete. 2024-04-24T08:53:26.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.129+0000 7f5461000640 1 Processor -- start 2024-04-24T08:53:26.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.129+0000 7f5461000640 1 -- start start 2024-04-24T08:53:26.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.129+0000 7f5461000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f545c1044b0 0x7f545c1a06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:26.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.129+0000 7f5461000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f545c109580 0x7f545c1a0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:26.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.129+0000 7f5461000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f545c19a890 0x7f545c19ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:26.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.129+0000 7f5461000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f545c1144f0 con 0x7f545c109580 2024-04-24T08:53:26.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.129+0000 7f5461000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f545c114370 con 0x7f545c1044b0 2024-04-24T08:53:26.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.129+0000 7f5461000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f545c114670 con 0x7f545c19a890 2024-04-24T08:53:26.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.130+0000 7f5453e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f545c109580 0x7f545c1a0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:26.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.130+0000 7f5453e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f545c109580 0x7f545c1a0c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50738/0 (socket says 172.21.15.83:50738) 2024-04-24T08:53:26.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.130+0000 7f5453e00640 1 -- 172.21.15.83:0/2208163665 learned_addr learned my addr 172.21.15.83:0/2208163665 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:53:26.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.130+0000 7f5459e00640 1 --2- 172.21.15.83:0/2208163665 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f545c1044b0 0x7f545c1a06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:26.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.130+0000 7f5453e00640 1 -- 172.21.15.83:0/2208163665 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f545c19a890 msgr2=0x7f545c19ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:26.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.130+0000 7f5453e00640 1 --2- 172.21.15.83:0/2208163665 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f545c19a890 0x7f545c19ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:26.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.130+0000 7f5453e00640 1 -- 172.21.15.83:0/2208163665 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f545c1044b0 msgr2=0x7f545c1a06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:26.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.130+0000 7f5453e00640 1 --2- 172.21.15.83:0/2208163665 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f545c1044b0 0x7f545c1a06d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:26.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.130+0000 7f5453e00640 1 -- 172.21.15.83:0/2208163665 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f545c19b420 con 0x7f545c109580 2024-04-24T08:53:26.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.130+0000 7f5453e00640 1 --2- 172.21.15.83:0/2208163665 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f545c109580 0x7f545c1a0c10 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f544400c970 tx=0x7f544400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:26.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.131+0000 7f5452a00640 1 -- 172.21.15.83:0/2208163665 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5444014070 con 0x7f545c109580 2024-04-24T08:53:26.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.131+0000 7f5461000640 1 -- 172.21.15.83:0/2208163665 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f545c19b710 con 0x7f545c109580 2024-04-24T08:53:26.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.131+0000 7f5452a00640 1 -- 172.21.15.83:0/2208163665 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5444004440 con 0x7f545c109580 2024-04-24T08:53:26.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.131+0000 7f5461000640 1 -- 172.21.15.83:0/2208163665 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f545c1a78a0 con 0x7f545c109580 2024-04-24T08:53:26.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.131+0000 7f5452a00640 1 -- 172.21.15.83:0/2208163665 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5444003df0 con 0x7f545c109580 2024-04-24T08:53:26.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.134+0000 7f5463e80740 1 Processor -- start 2024-04-24T08:53:26.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.134+0000 7f5463e80740 1 -- start start 2024-04-24T08:53:26.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.135+0000 7f5463e80740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ceafed6950 0x55ceafed6d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:26.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.135+0000 7f5463e80740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ceafed7270 0x55ceafee1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:26.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.135+0000 7f5463e80740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ceafee1940 0x55ceafee3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:26.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.135+0000 7f5463e80740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ceafe3c1d0 con 0x55ceafed7270 2024-04-24T08:53:26.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.135+0000 7f5463e80740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ceafde2450 con 0x55ceafee1940 2024-04-24T08:53:26.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.135+0000 7f5463e80740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55ceafe38190 con 0x55ceafed6950 2024-04-24T08:53:26.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.135+0000 7f5452a00640 1 -- 172.21.15.83:0/2208163665 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5444017070 con 0x7f545c109580 2024-04-24T08:53:26.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.135+0000 7f5459e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ceafed7270 0x55ceafee1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:26.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.135+0000 7f5459e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ceafed7270 0x55ceafee1400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50752/0 (socket says 172.21.15.83:50752) 2024-04-24T08:53:26.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.135+0000 7f5459e00640 1 -- 172.21.15.83:0/4033072223 learned_addr learned my addr 172.21.15.83:0/4033072223 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:53:26.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.135+0000 7f5459e00640 1 -- 172.21.15.83:0/4033072223 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ceafed6950 msgr2=0x55ceafed6d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:26.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.135+0000 7f5453e00640 1 --2- 172.21.15.83:0/4033072223 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ceafee1940 0x55ceafee3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:26.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.135+0000 7f545a800640 1 --2- 172.21.15.83:0/4033072223 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ceafed6950 0x55ceafed6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:26.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.135+0000 7f5459e00640 1 --2- 172.21.15.83:0/4033072223 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ceafed6950 0x55ceafed6d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:26.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.135+0000 7f5459e00640 1 -- 172.21.15.83:0/4033072223 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ceafee1940 msgr2=0x55ceafee3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:26.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.135+0000 7f5459e00640 1 --2- 172.21.15.83:0/4033072223 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ceafee1940 0x55ceafee3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:26.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.135+0000 7f5459e00640 1 -- 172.21.15.83:0/4033072223 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ceafe9f310 con 0x55ceafed7270 2024-04-24T08:53:26.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.135+0000 7f545a800640 1 --2- 172.21.15.83:0/4033072223 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ceafed6950 0x55ceafed6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:53:26.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.136+0000 7f5453e00640 1 --2- 172.21.15.83:0/4033072223 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ceafee1940 0x55ceafee3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:53:26.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.136+0000 7f5459e00640 1 --2- 172.21.15.83:0/4033072223 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ceafed7270 0x55ceafee1400 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f544c009bc0 tx=0x7f544c00e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca65ba630fcbe9b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:26.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.136+0000 7f544b400640 1 -- 172.21.15.83:0/4033072223 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f544c016070 con 0x55ceafed7270 2024-04-24T08:53:26.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.136+0000 7f544b400640 1 -- 172.21.15.83:0/4033072223 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f544c011920 con 0x55ceafed7270 2024-04-24T08:53:26.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.136+0000 7f544b400640 1 -- 172.21.15.83:0/4033072223 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f544c010890 con 0x55ceafed7270 2024-04-24T08:53:26.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.136+0000 7f5452a00640 1 --2- 172.21.15.83:0/2208163665 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f54400a3d20 0x7f54400a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:26.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.137+0000 7f5452a00640 1 -- 172.21.15.83:0/2208163665 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f54440d06c0 con 0x7f545c109580 2024-04-24T08:53:26.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.137+0000 7f545a800640 1 --2- 172.21.15.83:0/2208163665 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f54400a3d20 0x7f54400a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:26.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.137+0000 7f5463e80740 1 -- 172.21.15.83:0/4033072223 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ceafed7270 msgr2=0x55ceafee1400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:26.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.137+0000 7f5463e80740 1 --2- 172.21.15.83:0/4033072223 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ceafed7270 0x55ceafee1400 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f544c009bc0 tx=0x7f544c00e6c0 comp rx=0 tx=0).stop 2024-04-24T08:53:26.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.137+0000 7f5463e80740 1 -- 172.21.15.83:0/4033072223 shutdown_connections 2024-04-24T08:53:26.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.137+0000 7f5463e80740 1 --2- 172.21.15.83:0/4033072223 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ceafee1940 0x55ceafee3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:26.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.137+0000 7f5463e80740 1 --2- 172.21.15.83:0/4033072223 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ceafed7270 0x55ceafee1400 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:26.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.137+0000 7f5463e80740 1 --2- 172.21.15.83:0/4033072223 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ceafed6950 0x55ceafed6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:26.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.137+0000 7f5463e80740 1 -- 172.21.15.83:0/4033072223 >> 172.21.15.83:0/4033072223 conn(0x55ceafed2c70 msgr2=0x55ceafed3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:26.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.137+0000 7f5463e80740 1 -- 172.21.15.83:0/4033072223 shutdown_connections 2024-04-24T08:53:26.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.138+0000 7f5463e80740 1 -- 172.21.15.83:0/4033072223 wait complete. 2024-04-24T08:53:26.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.138+0000 7f545a800640 1 --2- 172.21.15.83:0/2208163665 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f54400a3d20 0x7f54400a61e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5454005e30 tx=0x7f5454005d80 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:26.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.138+0000 7f5463e80740 1 -- start start 2024-04-24T08:53:26.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.138+0000 7f5463e80740 1 Processor -- start 2024-04-24T08:53:26.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.139+0000 7f5463e80740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ceafed6950 0x55ceafee1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:26.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.139+0000 7f5463e80740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ceafed7270 0x55ceafede5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:26.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.139+0000 7f5463e80740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ceafee1940 0x55ceafedec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:26.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.139+0000 7f5463e80740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ceafee47d0 con 0x55ceafed6950 2024-04-24T08:53:26.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.139+0000 7f5463e80740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ceafdf5810 con 0x55ceafed7270 2024-04-24T08:53:26.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.139+0000 7f5463e80740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55ceafee4950 con 0x55ceafee1940 2024-04-24T08:53:26.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.139+0000 7f5463e80740 10 client.0 _ll_register_callbacks cb 0x55ceafdc3140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:53:26.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.139+0000 7f5459e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ceafed6950 0x55ceafee1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:26.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.139+0000 7f5459e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ceafed6950 0x55ceafee1300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50758/0 (socket says 172.21.15.83:50758) 2024-04-24T08:53:26.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.139+0000 7f5459e00640 1 -- 172.21.15.83:0/2444393103 learned_addr learned my addr 172.21.15.83:0/2444393103 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:53:26.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.139+0000 7f5453e00640 1 --2- 172.21.15.83:0/2444393103 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ceafed7270 0x55ceafede5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:26.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.139+0000 7f5459e00640 1 -- 172.21.15.83:0/2444393103 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ceafee1940 msgr2=0x55ceafedec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:26.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.139+0000 7f5459e00640 1 --2- 172.21.15.83:0/2444393103 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ceafee1940 0x55ceafedec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:26.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.139+0000 7f5459e00640 1 -- 172.21.15.83:0/2444393103 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ceafed7270 msgr2=0x55ceafede5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:26.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.139+0000 7f5459e00640 1 --2- 172.21.15.83:0/2444393103 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ceafed7270 0x55ceafede5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:26.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.139+0000 7f5459e00640 1 -- 172.21.15.83:0/2444393103 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ceafe063f0 con 0x55ceafed6950 2024-04-24T08:53:26.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.140+0000 7f5459e00640 1 --2- 172.21.15.83:0/2444393103 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ceafed6950 0x55ceafee1300 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f544c0116a0 tx=0x7f544c0116d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ceb1e382c6f0bbe server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:26.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.140+0000 7f543fe00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:53:26.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.140+0000 7f543fe00640 1 -- 172.21.15.83:0/2444393103 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f544c016050 con 0x55ceafed6950 2024-04-24T08:53:26.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.140+0000 7f543fe00640 1 -- 172.21.15.83:0/2444393103 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f544c005040 con 0x55ceafed6950 2024-04-24T08:53:26.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.140+0000 7f5463e80740 1 -- 172.21.15.83:0/2444393103 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f544c011920 con 0x55ceafed6950 2024-04-24T08:53:26.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.140+0000 7f543fe00640 1 -- 172.21.15.83:0/2444393103 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f544c005260 con 0x55ceafed6950 2024-04-24T08:53:26.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.140+0000 7f5463e80740 10 client.16377 fetch_fsmap learned FSMap version 28 2024-04-24T08:53:26.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.140+0000 7f5463e80740 1 -- 172.21.15.83:0/2444393103 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ceafb4e3f0 con 0x55ceafed6950 2024-04-24T08:53:26.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.141+0000 7f543fe00640 1 -- 172.21.15.83:0/2444393103 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f544c011920 con 0x55ceafed6950 2024-04-24T08:53:26.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.141+0000 7f543fe00640 1 -- 172.21.15.83:0/2444393103 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f544c02b030 con 0x55ceafed6950 2024-04-24T08:53:26.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.141+0000 7f5463e80740 10 client.16377 fetch_fsmap finished waiting for FSMap version 28 2024-04-24T08:53:26.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.141+0000 7f5463e80740 10 client.16377 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:53:26.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.141+0000 7f5463e80740 20 client.16377 populate_metadata read hostname 'smithi083' 2024-04-24T08:53:26.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.141+0000 7f5463e80740 1 --2- 172.21.15.83:0/2444393103 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ceafef19a0 0x55ceafef3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:26.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.141+0000 7f5463e80740 4 client.16377 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:53:26.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.141+0000 7f5463e80740 1 -- 172.21.15.83:0/2444393103 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ceafe49bf0 con 0x55ceafef19a0 2024-04-24T08:53:26.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.142+0000 7f5453e00640 1 --2- 172.21.15.83:0/2444393103 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ceafef19a0 0x55ceafef3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:26.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.142+0000 7f5453e00640 1 --2- 172.21.15.83:0/2444393103 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ceafef19a0 0x55ceafef3d90 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1fb6565af8e5e97c server_cookie=a72883cc0d2a359f in_seq=0 out_seq=0 2024-04-24T08:53:26.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.142+0000 7f543fe00640 10 client.16377 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:53:26.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.144+0000 7f543fe00640 1 -- 172.21.15.83:0/2444393103 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ceafe49bf0 con 0x55ceafef19a0 2024-04-24T08:53:26.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.144+0000 7f543fe00640 10 client.16377 handle_command_reply: tid=0 2024-04-24T08:53:26.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.144+0000 7f5463e80740 1 client.16377 shutdown 2024-04-24T08:53:26.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.145+0000 7f5463e80740 1 -- 172.21.15.83:0/2444393103 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ceafed6950 msgr2=0x55ceafee1300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:26.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.145+0000 7f5463e80740 1 --2- 172.21.15.83:0/2444393103 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ceafed6950 0x55ceafee1300 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f544c0116a0 tx=0x7f544c0116d0 comp rx=0 tx=0).stop 2024-04-24T08:53:26.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.145+0000 7f5463e80740 1 -- 172.21.15.83:0/2444393103 shutdown_connections 2024-04-24T08:53:26.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.145+0000 7f5463e80740 1 --2- 172.21.15.83:0/2444393103 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ceafef19a0 0x55ceafef3d90 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:26.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.145+0000 7f5463e80740 1 --2- 172.21.15.83:0/2444393103 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ceafee1940 0x55ceafedec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:26.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.145+0000 7f5463e80740 1 --2- 172.21.15.83:0/2444393103 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ceafed7270 0x55ceafede5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:26.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.145+0000 7f543fe00640 0 client.16377 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:53:26.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.145+0000 7f5463e80740 1 --2- 172.21.15.83:0/2444393103 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ceafed6950 0x55ceafee1300 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:26.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.145+0000 7f5463e80740 1 -- 172.21.15.83:0/2444393103 >> 172.21.15.83:0/2444393103 conn(0x55ceafed2c70 msgr2=0x55ceafed8a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:26.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.145+0000 7f5463e80740 1 -- 172.21.15.83:0/2444393103 shutdown_connections 2024-04-24T08:53:26.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.145+0000 7f5463e80740 1 -- 172.21.15.83:0/2444393103 wait complete. 2024-04-24T08:53:26.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.145+0000 7f5463e80740 20 client.16377 trim_cache size 0 max 16384 2024-04-24T08:53:26.158 INFO:tasks.workunit.client.0.smithi083.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-24T08:53:26.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.159+0000 7f5463e80740 1 Processor -- start 2024-04-24T08:53:26.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.159+0000 7f5463e80740 1 -- start start 2024-04-24T08:53:26.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.159+0000 7f5463e80740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ceafed6950 0x55ceafeed690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:26.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.159+0000 7f5463e80740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ceafed7270 0x55ceafee0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:26.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.159+0000 7f5463e80740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ceafee1940 0x55ceafef93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:26.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.159+0000 7f5463e80740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f545c1b1790 con 0x55ceafed6950 2024-04-24T08:53:26.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.159+0000 7f5463e80740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f545c1b32d0 con 0x55ceafed7270 2024-04-24T08:53:26.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.159+0000 7f5463e80740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f545c1b1e60 con 0x55ceafee1940 2024-04-24T08:53:26.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.160+0000 7f5459e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ceafed6950 0x55ceafeed690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:26.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.160+0000 7f5459e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ceafed6950 0x55ceafeed690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50774/0 (socket says 172.21.15.83:50774) 2024-04-24T08:53:26.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.160+0000 7f5459e00640 1 -- 172.21.15.83:0/1352914939 learned_addr learned my addr 172.21.15.83:0/1352914939 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:53:26.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.160+0000 7f545a800640 1 --2- 172.21.15.83:0/1352914939 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ceafee1940 0x55ceafef93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:26.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.160+0000 7f5459e00640 1 -- 172.21.15.83:0/1352914939 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ceafee1940 msgr2=0x55ceafef93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:26.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.160+0000 7f5459e00640 1 --2- 172.21.15.83:0/1352914939 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ceafee1940 0x55ceafef93c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:26.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.160+0000 7f5459e00640 1 -- 172.21.15.83:0/1352914939 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ceafed7270 msgr2=0x55ceafee0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:26.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.160+0000 7f5459e00640 1 --2- 172.21.15.83:0/1352914939 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ceafed7270 0x55ceafee0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:26.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.160+0000 7f5459e00640 1 -- 172.21.15.83:0/1352914939 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ceafc187d0 con 0x55ceafed6950 2024-04-24T08:53:26.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.160+0000 7f545a800640 1 --2- 172.21.15.83:0/1352914939 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ceafee1940 0x55ceafef93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:53:26.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.160+0000 7f5459e00640 1 --2- 172.21.15.83:0/1352914939 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ceafed6950 0x55ceafeed690 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f544c00eda0 tx=0x7f544c010470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25c0c8be90742c97 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:26.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.161+0000 7f544aa00640 1 -- 172.21.15.83:0/1352914939 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f544c016040 con 0x55ceafed6950 2024-04-24T08:53:26.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.161+0000 7f544aa00640 1 -- 172.21.15.83:0/1352914939 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f544c005040 con 0x55ceafed6950 2024-04-24T08:53:26.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.161+0000 7f544aa00640 1 -- 172.21.15.83:0/1352914939 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f544c005660 con 0x55ceafed6950 2024-04-24T08:53:26.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.161+0000 7f5463e80740 1 -- 172.21.15.83:0/1352914939 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ceafed6950 msgr2=0x55ceafeed690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:26.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.161+0000 7f5463e80740 1 --2- 172.21.15.83:0/1352914939 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ceafed6950 0x55ceafeed690 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f544c00eda0 tx=0x7f544c010470 comp rx=0 tx=0).stop 2024-04-24T08:53:26.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.162+0000 7f5463e80740 1 -- 172.21.15.83:0/1352914939 shutdown_connections 2024-04-24T08:53:26.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.162+0000 7f5463e80740 1 --2- 172.21.15.83:0/1352914939 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ceafee1940 0x55ceafef93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:26.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.162+0000 7f5463e80740 1 --2- 172.21.15.83:0/1352914939 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ceafed7270 0x55ceafee0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:26.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.162+0000 7f5463e80740 1 --2- 172.21.15.83:0/1352914939 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ceafed6950 0x55ceafeed690 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:26.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.162+0000 7f5463e80740 1 -- 172.21.15.83:0/1352914939 >> 172.21.15.83:0/1352914939 conn(0x55ceafed2c70 msgr2=0x55ceafed98a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:26.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.162+0000 7f5463e80740 1 -- 172.21.15.83:0/1352914939 shutdown_connections 2024-04-24T08:53:26.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.162+0000 7f5463e80740 1 -- 172.21.15.83:0/1352914939 wait complete. 2024-04-24T08:53:26.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.162+0000 7f5463e80740 1 -- start start 2024-04-24T08:53:26.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.162+0000 7f5463e80740 1 Processor -- start 2024-04-24T08:53:26.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.163+0000 7f5463e80740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ceafed6950 0x55ceafee1d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:26.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.163+0000 7f5463e80740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ceafed7270 0x55ceafee2260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:26.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.163+0000 7f5463e80740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ceafee1940 0x55ceafee27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:26.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.163+0000 7f5463e80740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f545c1b2c00 con 0x55ceafed6950 2024-04-24T08:53:26.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.163+0000 7f5463e80740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f545c1b2530 con 0x55ceafee1940 2024-04-24T08:53:26.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.163+0000 7f5463e80740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f545c1af770 con 0x55ceafed7270 2024-04-24T08:53:26.164 INFO:tasks.workunit.client.0.smithi083.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-24T08:53:26.164 INFO:tasks.workunit.client.0.smithi083.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-24T08:53:26.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.163+0000 7f5463e80740 10 client.0 _ll_register_callbacks cb 0x7f544c009a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:53:26.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.163+0000 7f5459e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ceafed6950 0x55ceafee1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:26.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.163+0000 7f5459e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ceafed6950 0x55ceafee1d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50786/0 (socket says 172.21.15.83:50786) 2024-04-24T08:53:26.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.163+0000 7f5459e00640 1 -- 172.21.15.83:0/1256537413 learned_addr learned my addr 172.21.15.83:0/1256537413 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:53:26.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.163+0000 7f545a800640 1 --2- 172.21.15.83:0/1256537413 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ceafee1940 0x55ceafee27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:26.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.163+0000 7f5459e00640 1 -- 172.21.15.83:0/1256537413 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ceafed7270 msgr2=0x55ceafee2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:26.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.163+0000 7f5459e00640 1 --2- 172.21.15.83:0/1256537413 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ceafed7270 0x55ceafee2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:26.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.163+0000 7f5459e00640 1 -- 172.21.15.83:0/1256537413 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ceafee1940 msgr2=0x55ceafee27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:26.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.163+0000 7f5459e00640 1 --2- 172.21.15.83:0/1256537413 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ceafee1940 0x55ceafee27a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:26.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.163+0000 7f5459e00640 1 -- 172.21.15.83:0/1256537413 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ceafee8ba0 con 0x55ceafed6950 2024-04-24T08:53:26.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.163+0000 7f545a800640 1 --2- 172.21.15.83:0/1256537413 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ceafee1940 0x55ceafee27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:53:26.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.164+0000 7f5459e00640 1 --2- 172.21.15.83:0/1256537413 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ceafed6950 0x55ceafee1d20 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f544c010fc0 tx=0x7f544c011ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98511a1207cb26fc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:26.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.164+0000 7f543fe00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:53:26.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.164+0000 7f543fe00640 1 -- 172.21.15.83:0/1256537413 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f544c016030 con 0x55ceafed6950 2024-04-24T08:53:26.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.164+0000 7f543fe00640 1 -- 172.21.15.83:0/1256537413 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f544c005400 con 0x55ceafed6950 2024-04-24T08:53:26.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.164+0000 7f5463e80740 1 -- 172.21.15.83:0/1256537413 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f544c005040 con 0x55ceafed6950 2024-04-24T08:53:26.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.164+0000 7f543fe00640 1 -- 172.21.15.83:0/1256537413 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f544c005600 con 0x55ceafed6950 2024-04-24T08:53:26.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.164+0000 7f5463e80740 10 client.16383 fetch_fsmap learned FSMap version 28 2024-04-24T08:53:26.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.164+0000 7f5463e80740 1 -- 172.21.15.83:0/1256537413 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ceafee9e10 con 0x55ceafed6950 2024-04-24T08:53:26.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.165+0000 7f543fe00640 1 -- 172.21.15.83:0/1256537413 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f544c005040 con 0x55ceafed6950 2024-04-24T08:53:26.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.165+0000 7f543fe00640 1 -- 172.21.15.83:0/1256537413 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f544c02b070 con 0x55ceafed6950 2024-04-24T08:53:26.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.165+0000 7f5463e80740 10 client.16383 fetch_fsmap finished waiting for FSMap version 28 2024-04-24T08:53:26.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.165+0000 7f5463e80740 10 client.16383 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:53:26.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.165+0000 7f5463e80740 20 client.16383 populate_metadata read hostname 'smithi083' 2024-04-24T08:53:26.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.165+0000 7f5463e80740 1 --2- 172.21.15.83:0/1256537413 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ceaff57c30 0x55ceaff5a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:26.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.165+0000 7f5463e80740 4 client.16383 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T08:53:26.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.165+0000 7f5463e80740 1 -- 172.21.15.83:0/1256537413 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ceafe43a60 con 0x55ceaff57c30 2024-04-24T08:53:26.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.166+0000 7f5453e00640 1 --2- 172.21.15.83:0/1256537413 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ceaff57c30 0x55ceaff5a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:26.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.167+0000 7f5453e00640 1 --2- 172.21.15.83:0/1256537413 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ceaff57c30 0x55ceaff5a020 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46f36fd414991ea8 server_cookie=61e5bbd05b8566c9 in_seq=0 out_seq=0 2024-04-24T08:53:26.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.167+0000 7f543fe00640 10 client.16383 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:53:26.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.168+0000 7f543fe00640 1 -- 172.21.15.83:0/1256537413 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ceafe43a60 con 0x55ceaff57c30 2024-04-24T08:53:26.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.168+0000 7f543fe00640 10 client.16383 handle_command_reply: tid=0 2024-04-24T08:53:26.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.168+0000 7f5463e80740 1 client.16383 shutdown 2024-04-24T08:53:26.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.168+0000 7f5463e80740 1 -- 172.21.15.83:0/1256537413 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ceafed6950 msgr2=0x55ceafee1d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:26.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.168+0000 7f5463e80740 1 --2- 172.21.15.83:0/1256537413 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ceafed6950 0x55ceafee1d20 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f544c010fc0 tx=0x7f544c011ce0 comp rx=0 tx=0).stop 2024-04-24T08:53:26.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.168+0000 7f5463e80740 1 -- 172.21.15.83:0/1256537413 shutdown_connections 2024-04-24T08:53:26.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.168+0000 7f5463e80740 1 --2- 172.21.15.83:0/1256537413 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ceaff57c30 0x55ceaff5a020 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:26.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.168+0000 7f5463e80740 1 --2- 172.21.15.83:0/1256537413 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ceafee1940 0x55ceafee27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:26.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.168+0000 7f5463e80740 1 --2- 172.21.15.83:0/1256537413 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ceafed7270 0x55ceafee2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:26.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.168+0000 7f543fe00640 0 client.16383 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:53:26.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.168+0000 7f5463e80740 1 --2- 172.21.15.83:0/1256537413 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ceafed6950 0x55ceafee1d20 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:26.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.168+0000 7f5463e80740 1 -- 172.21.15.83:0/1256537413 >> 172.21.15.83:0/1256537413 conn(0x55ceafed2c70 msgr2=0x55ceafedcec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:26.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.168+0000 7f5463e80740 1 -- 172.21.15.83:0/1256537413 shutdown_connections 2024-04-24T08:53:26.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.169+0000 7f5463e80740 1 -- 172.21.15.83:0/1256537413 wait complete. 2024-04-24T08:53:26.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.169+0000 7f5463e80740 20 client.16383 trim_cache size 0 max 16384 2024-04-24T08:53:26.171 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T08:53:26.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.169+0000 7f5461000640 1 -- 172.21.15.83:0/2208163665 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f54400a3d20 msgr2=0x7f54400a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:26.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.169+0000 7f5461000640 1 --2- 172.21.15.83:0/2208163665 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f54400a3d20 0x7f54400a61e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5454005e30 tx=0x7f5454005d80 comp rx=0 tx=0).stop 2024-04-24T08:53:26.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.169+0000 7f5461000640 1 -- 172.21.15.83:0/2208163665 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f545c109580 msgr2=0x7f545c1a0c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:26.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.169+0000 7f5461000640 1 --2- 172.21.15.83:0/2208163665 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f545c109580 0x7f545c1a0c10 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f544400c970 tx=0x7f544400ce30 comp rx=0 tx=0).stop 2024-04-24T08:53:26.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.169+0000 7f5461000640 1 -- 172.21.15.83:0/2208163665 shutdown_connections 2024-04-24T08:53:26.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.169+0000 7f5461000640 1 --2- 172.21.15.83:0/2208163665 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f54400a3d20 0x7f54400a61e0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:26.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.169+0000 7f5461000640 1 --2- 172.21.15.83:0/2208163665 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f545c19a890 0x7f545c19ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:26.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.169+0000 7f5461000640 1 --2- 172.21.15.83:0/2208163665 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f545c109580 0x7f545c1a0c10 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:26.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.169+0000 7f5461000640 1 --2- 172.21.15.83:0/2208163665 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f545c1044b0 0x7f545c1a06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:26.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.169+0000 7f5461000640 1 -- 172.21.15.83:0/2208163665 >> 172.21.15.83:0/2208163665 conn(0x7f545c0ffd90 msgr2=0x7f545c102060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:26.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.170+0000 7f5461000640 1 -- 172.21.15.83:0/2208163665 shutdown_connections 2024-04-24T08:53:26.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:26.170+0000 7f5461000640 1 -- 172.21.15.83:0/2208163665 wait complete. 2024-04-24T08:53:26.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:53:25 smithi138 ceph-mon[31441]: pgmap v57: 129 pgs: 129 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-24T08:53:26.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:53:25 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/2342997206' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:53:26.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:53:25 smithi189 ceph-mon[31791]: pgmap v57: 129 pgs: 129 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-24T08:53:26.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:53:25 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/2342997206' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:53:26.235 INFO:tasks.workunit.client.0.smithi083.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-24T08:53:26.245 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-24T08:53:26.245 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-24T08:53:26.297 INFO:tasks.workunit:Running workunits matching suites/dbench.sh on client.0... 2024-04-24T08:53:26.298 INFO:tasks.workunit:Running workunit suites/dbench.sh... 2024-04-24T08:53:26.299 DEBUG:teuthology.orchestra.run.smithi083:workunit test suites/dbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1f81fb397ae98da7563d451a78c61574c8f4e6e0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh 2024-04-24T08:53:26.438 INFO:tasks.workunit.client.0.smithi083.stderr:+ dbench 1 2024-04-24T08:53:26.444 INFO:tasks.workunit.client.0.smithi083.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-04-24T08:53:26.444 INFO:tasks.workunit.client.0.smithi083.stdout: 2024-04-24T08:53:26.444 INFO:tasks.workunit.client.0.smithi083.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-04-24T08:53:26.452 INFO:tasks.workunit.client.0.smithi083.stdout:0 of 1 processes prepared for launch 0 sec 2024-04-24T08:53:26.553 INFO:tasks.workunit.client.0.smithi083.stdout:1 of 1 processes prepared for launch 0 sec 2024-04-24T08:53:26.553 INFO:tasks.workunit.client.0.smithi083.stdout:releasing clients 2024-04-24T08:53:27.186 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T08:53:27.187 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T08:53:27.187 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T08:53:27.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.408+0000 7f2eea800640 1 -- 172.21.15.83:0/3272399751 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ee40743a0 msgr2=0x7f2ee4074780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:27.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.408+0000 7f2eea800640 1 --2- 172.21.15.83:0/3272399751 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ee40743a0 0x7f2ee4074780 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f2ed4009900 tx=0x7f2ed402a480 comp rx=0 tx=0).stop 2024-04-24T08:53:27.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.408+0000 7f2eea800640 1 -- 172.21.15.83:0/3272399751 shutdown_connections 2024-04-24T08:53:27.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.408+0000 7f2eea800640 1 --2- 172.21.15.83:0/3272399751 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2ee4112eb0 0x7f2ee41152a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:27.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.408+0000 7f2eea800640 1 --2- 172.21.15.83:0/3272399751 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ee4074cc0 0x7f2ee4112970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:27.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.408+0000 7f2eea800640 1 --2- 172.21.15.83:0/3272399751 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ee40743a0 0x7f2ee4074780 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:27.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.408+0000 7f2eea800640 1 -- 172.21.15.83:0/3272399751 >> 172.21.15.83:0/3272399751 conn(0x7f2ee40fd9d0 msgr2=0x7f2ee40ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:27.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.408+0000 7f2eea800640 1 -- 172.21.15.83:0/3272399751 shutdown_connections 2024-04-24T08:53:27.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.408+0000 7f2eea800640 1 -- 172.21.15.83:0/3272399751 wait complete. 2024-04-24T08:53:27.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.409+0000 7f2eea800640 1 Processor -- start 2024-04-24T08:53:27.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.409+0000 7f2eea800640 1 -- start start 2024-04-24T08:53:27.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.410+0000 7f2eea800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2ee40743a0 0x7f2ee419edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:27.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.410+0000 7f2eea800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ee4074cc0 0x7f2ee419f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:27.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.410+0000 7f2eea800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ee4112eb0 0x7f2ee41a3620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:27.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.410+0000 7f2eea800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee41187f0 con 0x7f2ee4074cc0 2024-04-24T08:53:27.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.410+0000 7f2eea800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee4118670 con 0x7f2ee4112eb0 2024-04-24T08:53:27.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.410+0000 7f2eea800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee4118970 con 0x7f2ee40743a0 2024-04-24T08:53:27.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.410+0000 7f2ee8a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ee4074cc0 0x7f2ee419f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:27.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.410+0000 7f2ee8a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ee4074cc0 0x7f2ee419f2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50804/0 (socket says 172.21.15.83:50804) 2024-04-24T08:53:27.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.410+0000 7f2ee8a00640 1 -- 172.21.15.83:0/1870934306 learned_addr learned my addr 172.21.15.83:0/1870934306 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:53:27.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.410+0000 7f2ee9400640 1 --2- 172.21.15.83:0/1870934306 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2ee40743a0 0x7f2ee419edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:27.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.410+0000 7f2ee8a00640 1 -- 172.21.15.83:0/1870934306 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2ee40743a0 msgr2=0x7f2ee419edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:27.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.410+0000 7f2ee8a00640 1 --2- 172.21.15.83:0/1870934306 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2ee40743a0 0x7f2ee419edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:27.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.410+0000 7f2ee8a00640 1 -- 172.21.15.83:0/1870934306 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ee4112eb0 msgr2=0x7f2ee41a3620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:27.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.410+0000 7f2ee8a00640 1 --2- 172.21.15.83:0/1870934306 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ee4112eb0 0x7f2ee41a3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:27.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.411+0000 7f2ee8a00640 1 -- 172.21.15.83:0/1870934306 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ee41a3d50 con 0x7f2ee4074cc0 2024-04-24T08:53:27.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.411+0000 7f2ee8a00640 1 --2- 172.21.15.83:0/1870934306 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ee4074cc0 0x7f2ee419f2f0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f2ed800c890 tx=0x7f2ed800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:27.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.411+0000 7f2ee2000640 1 -- 172.21.15.83:0/1870934306 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed8004220 con 0x7f2ee4074cc0 2024-04-24T08:53:27.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.411+0000 7f2eea800640 1 -- 172.21.15.83:0/1870934306 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ee41a4040 con 0x7f2ee4074cc0 2024-04-24T08:53:27.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.411+0000 7f2ee2000640 1 -- 172.21.15.83:0/1870934306 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2ed80043c0 con 0x7f2ee4074cc0 2024-04-24T08:53:27.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.411+0000 7f2ee2000640 1 -- 172.21.15.83:0/1870934306 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed801d060 con 0x7f2ee4074cc0 2024-04-24T08:53:27.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.411+0000 7f2eea800640 1 -- 172.21.15.83:0/1870934306 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ee41ab8d0 con 0x7f2ee4074cc0 2024-04-24T08:53:27.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.415+0000 7f2ee2000640 1 -- 172.21.15.83:0/1870934306 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2ed80079b0 con 0x7f2ee4074cc0 2024-04-24T08:53:27.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.415+0000 7f2eeec41740 1 Processor -- start 2024-04-24T08:53:27.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.416+0000 7f2eeec41740 1 -- start start 2024-04-24T08:53:27.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.416+0000 7f2eeec41740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56509fb6a270 0x56509fb6a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:27.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.416+0000 7f2eeec41740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56509fb6ab90 0x56509fb74d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:27.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.416+0000 7f2eeec41740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56509fb75260 0x56509fb77650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:27.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.416+0000 7f2eeec41740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56509facfe70 con 0x56509fb6a270 2024-04-24T08:53:27.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.416+0000 7f2eeec41740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56509fa76380 con 0x56509fb75260 2024-04-24T08:53:27.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.416+0000 7f2eeec41740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56509facbe30 con 0x56509fb6ab90 2024-04-24T08:53:27.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.416+0000 7f2ee9e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56509fb6a270 0x56509fb6a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:27.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.417+0000 7f2ee9e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56509fb6a270 0x56509fb6a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50820/0 (socket says 172.21.15.83:50820) 2024-04-24T08:53:27.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.417+0000 7f2ee9e00640 1 -- 172.21.15.83:0/4005485979 learned_addr learned my addr 172.21.15.83:0/4005485979 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:53:27.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.417+0000 7f2ee8a00640 1 --2- 172.21.15.83:0/4005485979 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56509fb75260 0x56509fb77650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:27.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.417+0000 7f2ee9400640 1 --2- 172.21.15.83:0/4005485979 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56509fb6ab90 0x56509fb74d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:27.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.417+0000 7f2ee9e00640 1 -- 172.21.15.83:0/4005485979 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56509fb6ab90 msgr2=0x56509fb74d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:27.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.417+0000 7f2ee9e00640 1 --2- 172.21.15.83:0/4005485979 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56509fb6ab90 0x56509fb74d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:27.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.417+0000 7f2ee9e00640 1 -- 172.21.15.83:0/4005485979 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56509fb75260 msgr2=0x56509fb77650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:27.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.417+0000 7f2ee9e00640 1 --2- 172.21.15.83:0/4005485979 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56509fb75260 0x56509fb77650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:27.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.417+0000 7f2ee9e00640 1 -- 172.21.15.83:0/4005485979 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56509f8aa7e0 con 0x56509fb6a270 2024-04-24T08:53:27.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.417+0000 7f2ee2000640 1 --2- 172.21.15.83:0/1870934306 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f2ecc0a3d20 0x7f2ecc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:27.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.417+0000 7f2ee9e00640 1 --2- 172.21.15.83:0/4005485979 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56509fb6a270 0x56509fb6a650 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f2edc009cd0 tx=0x7f2edc00b420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7707f564fae16089 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:27.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.418+0000 7f2ee2000640 1 -- 172.21.15.83:0/1870934306 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f2ed80d0d60 con 0x7f2ee4074cc0 2024-04-24T08:53:27.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.418+0000 7f2ee9400640 1 --2- 172.21.15.83:0/1870934306 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f2ecc0a3d20 0x7f2ecc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:27.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.418+0000 7f2ed2600640 1 -- 172.21.15.83:0/4005485979 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2edc00bd40 con 0x56509fb6a270 2024-04-24T08:53:27.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.418+0000 7f2ed2600640 1 -- 172.21.15.83:0/4005485979 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2edc017070 con 0x56509fb6a270 2024-04-24T08:53:27.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.418+0000 7f2ed2600640 1 -- 172.21.15.83:0/4005485979 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2edc0115f0 con 0x56509fb6a270 2024-04-24T08:53:27.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.419+0000 7f2eeec41740 1 -- 172.21.15.83:0/4005485979 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56509fb6a270 msgr2=0x56509fb6a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:27.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.419+0000 7f2ee9400640 1 --2- 172.21.15.83:0/1870934306 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f2ecc0a3d20 0x7f2ecc0a61e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2ed4000c00 tx=0x7f2ed4031400 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:27.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.419+0000 7f2eeec41740 1 --2- 172.21.15.83:0/4005485979 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56509fb6a270 0x56509fb6a650 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f2edc009cd0 tx=0x7f2edc00b420 comp rx=0 tx=0).stop 2024-04-24T08:53:27.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.419+0000 7f2eeec41740 1 -- 172.21.15.83:0/4005485979 shutdown_connections 2024-04-24T08:53:27.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.419+0000 7f2eeec41740 1 --2- 172.21.15.83:0/4005485979 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56509fb75260 0x56509fb77650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:27.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.419+0000 7f2eeec41740 1 --2- 172.21.15.83:0/4005485979 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56509fb6ab90 0x56509fb74d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:27.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.419+0000 7f2eeec41740 1 --2- 172.21.15.83:0/4005485979 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56509fb6a270 0x56509fb6a650 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:27.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.419+0000 7f2eeec41740 1 -- 172.21.15.83:0/4005485979 >> 172.21.15.83:0/4005485979 conn(0x56509fb66590 msgr2=0x56509fb66970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:27.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.419+0000 7f2eeec41740 1 -- 172.21.15.83:0/4005485979 shutdown_connections 2024-04-24T08:53:27.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.419+0000 7f2eeec41740 1 -- 172.21.15.83:0/4005485979 wait complete. 2024-04-24T08:53:27.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.420+0000 7f2eeec41740 1 -- start start 2024-04-24T08:53:27.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.420+0000 7f2eeec41740 1 Processor -- start 2024-04-24T08:53:27.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.420+0000 7f2eeec41740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56509fb6a270 0x56509fb721e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:27.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.420+0000 7f2eeec41740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56509fb6ab90 0x56509fb72720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:27.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.420+0000 7f2eeec41740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56509fb75260 0x56509fb72c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:27.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.420+0000 7f2eeec41740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56509fb78050 con 0x56509fb6ab90 2024-04-24T08:53:27.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.420+0000 7f2eeec41740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56509fa89660 con 0x56509fb75260 2024-04-24T08:53:27.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.420+0000 7f2eeec41740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56509fb781d0 con 0x56509fb6a270 2024-04-24T08:53:27.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.420+0000 7f2eeec41740 10 client.0 _ll_register_callbacks cb 0x56509fa77aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:53:27.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.421+0000 7f2ee8a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56509fb6ab90 0x56509fb72720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:27.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.421+0000 7f2ee8a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56509fb6ab90 0x56509fb72720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50832/0 (socket says 172.21.15.83:50832) 2024-04-24T08:53:27.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.421+0000 7f2ee8a00640 1 -- 172.21.15.83:0/1332079683 learned_addr learned my addr 172.21.15.83:0/1332079683 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:53:27.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.421+0000 7f2ee9400640 1 --2- 172.21.15.83:0/1332079683 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56509fb6a270 0x56509fb721e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:27.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.421+0000 7f2ee9e00640 1 --2- 172.21.15.83:0/1332079683 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56509fb75260 0x56509fb72c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:27.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.421+0000 7f2ee8a00640 1 -- 172.21.15.83:0/1332079683 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56509fb6a270 msgr2=0x56509fb721e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:27.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.421+0000 7f2ee8a00640 1 --2- 172.21.15.83:0/1332079683 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56509fb6a270 0x56509fb721e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:27.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.421+0000 7f2ee8a00640 1 -- 172.21.15.83:0/1332079683 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56509fb75260 msgr2=0x56509fb72c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:27.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.421+0000 7f2ee8a00640 1 --2- 172.21.15.83:0/1332079683 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56509fb75260 0x56509fb72c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:27.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.421+0000 7f2ee8a00640 1 -- 172.21.15.83:0/1332079683 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56509f90af70 con 0x56509fb6ab90 2024-04-24T08:53:27.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.421+0000 7f2ee9e00640 1 --2- 172.21.15.83:0/1332079683 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56509fb75260 0x56509fb72c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:53:27.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.422+0000 7f2ee8a00640 1 --2- 172.21.15.83:0/1332079683 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56509fb6ab90 0x56509fb72720 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f2ed809c860 tx=0x7f2ed809c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62cd3ee9168717f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:27.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.422+0000 7f2ec6a00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:53:27.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.422+0000 7f2ec6a00640 1 -- 172.21.15.83:0/1332079683 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed809c8d0 con 0x56509fb6ab90 2024-04-24T08:53:27.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.422+0000 7f2ec6a00640 1 -- 172.21.15.83:0/1332079683 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f2ed80ce300 con 0x56509fb6ab90 2024-04-24T08:53:27.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.422+0000 7f2eeec41740 1 -- 172.21.15.83:0/1332079683 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2edc017070 con 0x56509fb6ab90 2024-04-24T08:53:27.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.422+0000 7f2ec6a00640 1 -- 172.21.15.83:0/1332079683 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2ed80ce500 con 0x56509fb6ab90 2024-04-24T08:53:27.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.422+0000 7f2eeec41740 10 client.16395 fetch_fsmap learned FSMap version 28 2024-04-24T08:53:27.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.422+0000 7f2eeec41740 1 -- 172.21.15.83:0/1332079683 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56509f7a2170 con 0x56509fb6ab90 2024-04-24T08:53:27.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.423+0000 7f2ec6a00640 1 -- 172.21.15.83:0/1332079683 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2edc017070 con 0x56509fb6ab90 2024-04-24T08:53:27.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.423+0000 7f2ec6a00640 1 -- 172.21.15.83:0/1332079683 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f2ed80a50b0 con 0x56509fb6ab90 2024-04-24T08:53:27.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.423+0000 7f2eeec41740 10 client.16395 fetch_fsmap finished waiting for FSMap version 28 2024-04-24T08:53:27.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.423+0000 7f2eeec41740 10 client.16395 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:53:27.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.423+0000 7f2eeec41740 20 client.16395 populate_metadata read hostname 'smithi083' 2024-04-24T08:53:27.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.423+0000 7f2eeec41740 1 --2- 172.21.15.83:0/1332079683 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56509fb85620 0x56509fb87a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:27.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.423+0000 7f2eeec41740 4 client.16395 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:53:27.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.423+0000 7f2eeec41740 1 -- 172.21.15.83:0/1332079683 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56509fadd890 con 0x56509fb85620 2024-04-24T08:53:27.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.424+0000 7f2ee9400640 1 --2- 172.21.15.83:0/1332079683 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56509fb85620 0x56509fb87a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:27.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.425+0000 7f2ee9400640 1 --2- 172.21.15.83:0/1332079683 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56509fb85620 0x56509fb87a10 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d96171a5b37aa59 server_cookie=d35feaebaef3d3e1 in_seq=0 out_seq=0 2024-04-24T08:53:27.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.425+0000 7f2ec6a00640 10 client.16395 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:53:27.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.427+0000 7f2ec6a00640 1 -- 172.21.15.83:0/1332079683 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56509fadd890 con 0x56509fb85620 2024-04-24T08:53:27.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.427+0000 7f2ec6a00640 10 client.16395 handle_command_reply: tid=0 2024-04-24T08:53:27.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.427+0000 7f2eeec41740 1 client.16395 shutdown 2024-04-24T08:53:27.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.428+0000 7f2eeec41740 1 -- 172.21.15.83:0/1332079683 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56509fb6ab90 msgr2=0x56509fb72720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:27.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.428+0000 7f2eeec41740 1 --2- 172.21.15.83:0/1332079683 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56509fb6ab90 0x56509fb72720 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f2ed809c860 tx=0x7f2ed809c410 comp rx=0 tx=0).stop 2024-04-24T08:53:27.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.428+0000 7f2eeec41740 1 -- 172.21.15.83:0/1332079683 shutdown_connections 2024-04-24T08:53:27.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.428+0000 7f2eeec41740 1 --2- 172.21.15.83:0/1332079683 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56509fb85620 0x56509fb87a10 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:27.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.428+0000 7f2eeec41740 1 --2- 172.21.15.83:0/1332079683 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56509fb75260 0x56509fb72c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:27.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.428+0000 7f2eeec41740 1 --2- 172.21.15.83:0/1332079683 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56509fb6ab90 0x56509fb72720 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:27.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.428+0000 7f2ec6a00640 0 client.16395 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:53:27.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.428+0000 7f2eeec41740 1 --2- 172.21.15.83:0/1332079683 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56509fb6a270 0x56509fb721e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:27.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.428+0000 7f2eeec41740 1 -- 172.21.15.83:0/1332079683 >> 172.21.15.83:0/1332079683 conn(0x56509fb66590 msgr2=0x56509fb6c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:27.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.428+0000 7f2eeec41740 1 -- 172.21.15.83:0/1332079683 shutdown_connections 2024-04-24T08:53:27.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.428+0000 7f2eeec41740 1 -- 172.21.15.83:0/1332079683 wait complete. 2024-04-24T08:53:27.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.428+0000 7f2eeec41740 20 client.16395 trim_cache size 0 max 16384 2024-04-24T08:53:27.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.450+0000 7f2eeec41740 1 Processor -- start 2024-04-24T08:53:27.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.450+0000 7f2eeec41740 1 -- start start 2024-04-24T08:53:27.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.451+0000 7f2eeec41740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56509fb6a270 0x56509fb80f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:27.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.451+0000 7f2eeec41740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56509fb6ab90 0x56509fb65570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:27.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.451+0000 7f2eeec41740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56509fb75260 0x56509fb8d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:27.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.451+0000 7f2eeec41740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee41b5a40 con 0x56509fb6ab90 2024-04-24T08:53:27.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.451+0000 7f2eeec41740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee41b7580 con 0x56509fb75260 2024-04-24T08:53:27.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.451+0000 7f2eeec41740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee41b6110 con 0x56509fb6a270 2024-04-24T08:53:27.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.451+0000 7f2ee8a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56509fb6ab90 0x56509fb65570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:27.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.451+0000 7f2ee8a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56509fb6ab90 0x56509fb65570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50838/0 (socket says 172.21.15.83:50838) 2024-04-24T08:53:27.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.452+0000 7f2ee8a00640 1 -- 172.21.15.83:0/2806905877 learned_addr learned my addr 172.21.15.83:0/2806905877 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:53:27.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.452+0000 7f2ee9e00640 1 --2- 172.21.15.83:0/2806905877 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56509fb75260 0x56509fb8d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:27.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.452+0000 7f2ee9400640 1 --2- 172.21.15.83:0/2806905877 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56509fb6a270 0x56509fb80f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:27.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.452+0000 7f2ee8a00640 1 -- 172.21.15.83:0/2806905877 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56509fb6a270 msgr2=0x56509fb80f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:27.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.452+0000 7f2ee8a00640 1 --2- 172.21.15.83:0/2806905877 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56509fb6a270 0x56509fb80f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:27.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.452+0000 7f2ee8a00640 1 -- 172.21.15.83:0/2806905877 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56509fb75260 msgr2=0x56509fb8d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:27.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.452+0000 7f2ee8a00640 1 --2- 172.21.15.83:0/2806905877 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56509fb75260 0x56509fb8d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:27.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.452+0000 7f2ee8a00640 1 -- 172.21.15.83:0/2806905877 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56509fb6a9c0 con 0x56509fb6ab90 2024-04-24T08:53:27.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.453+0000 7f2ee8a00640 1 --2- 172.21.15.83:0/2806905877 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56509fb6ab90 0x56509fb65570 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f2ed80a5010 tx=0x7f2ed809c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c914e3385052b4ef server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:27.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.453+0000 7f2ed1c00640 1 -- 172.21.15.83:0/2806905877 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed80a5540 con 0x56509fb6ab90 2024-04-24T08:53:27.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.453+0000 7f2ed1c00640 1 -- 172.21.15.83:0/2806905877 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2ed80a50b0 con 0x56509fb6ab90 2024-04-24T08:53:27.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.453+0000 7f2ed1c00640 1 -- 172.21.15.83:0/2806905877 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed8007ca0 con 0x56509fb6ab90 2024-04-24T08:53:27.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.454+0000 7f2eeec41740 1 -- 172.21.15.83:0/2806905877 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56509fb6ab90 msgr2=0x56509fb65570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:27.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.454+0000 7f2eeec41740 1 --2- 172.21.15.83:0/2806905877 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56509fb6ab90 0x56509fb65570 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f2ed80a5010 tx=0x7f2ed809c540 comp rx=0 tx=0).stop 2024-04-24T08:53:27.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.454+0000 7f2eeec41740 1 -- 172.21.15.83:0/2806905877 shutdown_connections 2024-04-24T08:53:27.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.454+0000 7f2eeec41740 1 --2- 172.21.15.83:0/2806905877 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56509fb75260 0x56509fb8d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:27.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.454+0000 7f2eeec41740 1 --2- 172.21.15.83:0/2806905877 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56509fb6ab90 0x56509fb65570 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:27.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.454+0000 7f2eeec41740 1 --2- 172.21.15.83:0/2806905877 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56509fb6a270 0x56509fb80f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:27.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.454+0000 7f2eeec41740 1 -- 172.21.15.83:0/2806905877 >> 172.21.15.83:0/2806905877 conn(0x56509fb66590 msgr2=0x56509fb64e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:27.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.455+0000 7f2eeec41740 1 -- 172.21.15.83:0/2806905877 shutdown_connections 2024-04-24T08:53:27.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.455+0000 7f2eeec41740 1 -- 172.21.15.83:0/2806905877 wait complete. 2024-04-24T08:53:27.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.455+0000 7f2eeec41740 1 -- start start 2024-04-24T08:53:27.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.455+0000 7f2eeec41740 1 Processor -- start 2024-04-24T08:53:27.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.456+0000 7f2eeec41740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56509fb6a270 0x56509fb799d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:27.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.456+0000 7f2eeec41740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56509fb6ab90 0x56509fb79f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:27.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.456+0000 7f2eeec41740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56509fb75260 0x56509fbe3b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:27.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.456+0000 7f2eeec41740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee41b6eb0 con 0x56509fb6a270 2024-04-24T08:53:27.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.456+0000 7f2eeec41740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee41b67e0 con 0x56509fb75260 2024-04-24T08:53:27.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.456+0000 7f2eeec41740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee41b3a20 con 0x56509fb6ab90 2024-04-24T08:53:27.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.456+0000 7f2eeec41740 10 client.0 _ll_register_callbacks cb 0x7f2ed80058c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:53:27.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.456+0000 7f2ee9400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56509fb6a270 0x56509fb799d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:27.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.456+0000 7f2ee9400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56509fb6a270 0x56509fb799d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50850/0 (socket says 172.21.15.83:50850) 2024-04-24T08:53:27.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.456+0000 7f2ee9400640 1 -- 172.21.15.83:0/2506049741 learned_addr learned my addr 172.21.15.83:0/2506049741 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:53:27.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.456+0000 7f2ee9400640 1 -- 172.21.15.83:0/2506049741 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56509fb6ab90 msgr2=0x56509fb79f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:27.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.456+0000 7f2ee8a00640 1 --2- 172.21.15.83:0/2506049741 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56509fb6ab90 0x56509fb79f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:27.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.456+0000 7f2ee9400640 1 --2- 172.21.15.83:0/2506049741 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56509fb6ab90 0x56509fb79f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:27.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.456+0000 7f2ee9400640 1 -- 172.21.15.83:0/2506049741 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56509fb75260 msgr2=0x56509fbe3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:27.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.456+0000 7f2ee9400640 1 --2- 172.21.15.83:0/2506049741 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56509fb75260 0x56509fbe3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:27.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.456+0000 7f2ee9400640 1 -- 172.21.15.83:0/2506049741 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56509fbe41b0 con 0x56509fb6a270 2024-04-24T08:53:27.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.457+0000 7f2ee9400640 1 --2- 172.21.15.83:0/2506049741 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56509fb6a270 0x56509fb799d0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f2ed4009230 tx=0x7f2ed4009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d0ccb7eb1d6983a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:27.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.457+0000 7f2ec6a00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:53:27.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.457+0000 7f2ec6a00640 1 -- 172.21.15.83:0/2506049741 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed4035050 con 0x56509fb6a270 2024-04-24T08:53:27.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.457+0000 7f2ec6a00640 1 -- 172.21.15.83:0/2506049741 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f2ed4031c90 con 0x56509fb6a270 2024-04-24T08:53:27.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.457+0000 7f2eeec41740 1 -- 172.21.15.83:0/2506049741 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ed80a50b0 con 0x56509fb6a270 2024-04-24T08:53:27.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.457+0000 7f2ec6a00640 1 -- 172.21.15.83:0/2506049741 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2ed40501a0 con 0x56509fb6a270 2024-04-24T08:53:27.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.457+0000 7f2eeec41740 10 client.16401 fetch_fsmap learned FSMap version 28 2024-04-24T08:53:27.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.457+0000 7f2eeec41740 1 -- 172.21.15.83:0/2506049741 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56509fbec7a0 con 0x56509fb6a270 2024-04-24T08:53:27.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.458+0000 7f2ec6a00640 1 -- 172.21.15.83:0/2506049741 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2ed80a50b0 con 0x56509fb6a270 2024-04-24T08:53:27.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.458+0000 7f2ec6a00640 1 -- 172.21.15.83:0/2506049741 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f2ed4003950 con 0x56509fb6a270 2024-04-24T08:53:27.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.458+0000 7f2eeec41740 10 client.16401 fetch_fsmap finished waiting for FSMap version 28 2024-04-24T08:53:27.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.458+0000 7f2eeec41740 10 client.16401 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:53:27.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.458+0000 7f2eeec41740 20 client.16401 populate_metadata read hostname 'smithi083' 2024-04-24T08:53:27.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.458+0000 7f2eeec41740 1 --2- 172.21.15.83:0/2506049741 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56509fbecf00 0x56509fbef340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:27.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.458+0000 7f2eeec41740 4 client.16401 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T08:53:27.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.458+0000 7f2eeec41740 1 -- 172.21.15.83:0/2506049741 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56509fad7700 con 0x56509fbecf00 2024-04-24T08:53:27.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.459+0000 7f2ee8a00640 1 --2- 172.21.15.83:0/2506049741 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56509fbecf00 0x56509fbef340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:27.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.460+0000 7f2ee8a00640 1 --2- 172.21.15.83:0/2506049741 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56509fbecf00 0x56509fbef340 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3ecd082912ac927 server_cookie=1fca7ca24e4f90bc in_seq=0 out_seq=0 2024-04-24T08:53:27.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.460+0000 7f2ec6a00640 10 client.16401 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:53:27.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.463+0000 7f2ec6a00640 1 -- 172.21.15.83:0/2506049741 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56509fad7700 con 0x56509fbecf00 2024-04-24T08:53:27.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.463+0000 7f2ec6a00640 10 client.16401 handle_command_reply: tid=0 2024-04-24T08:53:27.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.463+0000 7f2eeec41740 1 client.16401 shutdown 2024-04-24T08:53:27.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.463+0000 7f2eeec41740 1 -- 172.21.15.83:0/2506049741 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56509fb6a270 msgr2=0x56509fb799d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:27.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.463+0000 7f2eeec41740 1 --2- 172.21.15.83:0/2506049741 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56509fb6a270 0x56509fb799d0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f2ed4009230 tx=0x7f2ed4009260 comp rx=0 tx=0).stop 2024-04-24T08:53:27.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.463+0000 7f2eeec41740 1 -- 172.21.15.83:0/2506049741 shutdown_connections 2024-04-24T08:53:27.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.463+0000 7f2eeec41740 1 --2- 172.21.15.83:0/2506049741 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56509fbecf00 0x56509fbef340 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:27.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.463+0000 7f2eeec41740 1 --2- 172.21.15.83:0/2506049741 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56509fb75260 0x56509fbe3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:27.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.463+0000 7f2eeec41740 1 --2- 172.21.15.83:0/2506049741 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56509fb6ab90 0x56509fb79f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:27.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.463+0000 7f2eeec41740 1 --2- 172.21.15.83:0/2506049741 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56509fb6a270 0x56509fb799d0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:27.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.463+0000 7f2ec6a00640 0 client.16401 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:53:27.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.463+0000 7f2eeec41740 1 -- 172.21.15.83:0/2506049741 >> 172.21.15.83:0/2506049741 conn(0x56509fb66590 msgr2=0x56509fb656f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:27.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.463+0000 7f2eeec41740 1 -- 172.21.15.83:0/2506049741 shutdown_connections 2024-04-24T08:53:27.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.463+0000 7f2eeec41740 1 -- 172.21.15.83:0/2506049741 wait complete. 2024-04-24T08:53:27.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.464+0000 7f2eeec41740 20 client.16401 trim_cache size 0 max 16384 2024-04-24T08:53:27.465 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T08:53:27.465 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-04-24T08:53:27.465 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "98f1c9e6-a0a9-4235-9983-e7955f7dc728", 2024-04-24T08:53:27.465 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-04-24T08:53:27.465 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T08:53:27.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.464+0000 7f2eea800640 1 -- 172.21.15.83:0/1870934306 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f2ecc0a3d20 msgr2=0x7f2ecc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:27.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.464+0000 7f2eea800640 1 --2- 172.21.15.83:0/1870934306 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f2ecc0a3d20 0x7f2ecc0a61e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2ed4000c00 tx=0x7f2ed4031400 comp rx=0 tx=0).stop 2024-04-24T08:53:27.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.464+0000 7f2eea800640 1 -- 172.21.15.83:0/1870934306 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ee4074cc0 msgr2=0x7f2ee419f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:27.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.464+0000 7f2eea800640 1 --2- 172.21.15.83:0/1870934306 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ee4074cc0 0x7f2ee419f2f0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f2ed800c890 tx=0x7f2ed800cd50 comp rx=0 tx=0).stop 2024-04-24T08:53:27.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.464+0000 7f2eea800640 1 -- 172.21.15.83:0/1870934306 shutdown_connections 2024-04-24T08:53:27.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.464+0000 7f2eea800640 1 --2- 172.21.15.83:0/1870934306 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f2ecc0a3d20 0x7f2ecc0a61e0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:27.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.464+0000 7f2eea800640 1 --2- 172.21.15.83:0/1870934306 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ee4112eb0 0x7f2ee41a3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:27.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.464+0000 7f2eea800640 1 --2- 172.21.15.83:0/1870934306 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ee4074cc0 0x7f2ee419f2f0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:27.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.464+0000 7f2eea800640 1 --2- 172.21.15.83:0/1870934306 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2ee40743a0 0x7f2ee419edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:27.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.464+0000 7f2eea800640 1 -- 172.21.15.83:0/1870934306 >> 172.21.15.83:0/1870934306 conn(0x7f2ee40fd9d0 msgr2=0x7f2ee40ff2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:27.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.465+0000 7f2eea800640 1 -- 172.21.15.83:0/1870934306 shutdown_connections 2024-04-24T08:53:27.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.465+0000 7f2eea800640 1 -- 172.21.15.83:0/1870934306 wait complete. 2024-04-24T08:53:27.483 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T08:53:27.483 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T08:53:27.553 INFO:tasks.workunit.client.0.smithi083.stdout: 1 279 10.44 MB/sec warmup 1 sec latency 31.771 ms 2024-04-24T08:53:27.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.692+0000 7fdb0ba00640 1 -- 172.21.15.83:0/3777741228 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb0410e660 msgr2=0x7fdb0410ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:27.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.692+0000 7fdb0ba00640 1 --2- 172.21.15.83:0/3777741228 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb0410e660 0x7fdb0410ea40 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fdaf4009cd0 tx=0x7fdaf402a8e0 comp rx=0 tx=0).stop 2024-04-24T08:53:27.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.692+0000 7fdb0ba00640 1 -- 172.21.15.83:0/3777741228 shutdown_connections 2024-04-24T08:53:27.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.692+0000 7fdb0ba00640 1 --2- 172.21.15.83:0/3777741228 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdb04108fd0 0x7fdb04115490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:27.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.692+0000 7fdb0ba00640 1 --2- 172.21.15.83:0/3777741228 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb04108610 0x7fdb04108a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:27.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.692+0000 7fdb0ba00640 1 --2- 172.21.15.83:0/3777741228 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb0410e660 0x7fdb0410ea40 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:27.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.692+0000 7fdb0ba00640 1 -- 172.21.15.83:0/3777741228 >> 172.21.15.83:0/3777741228 conn(0x7fdb04076540 msgr2=0x7fdb04078960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:27.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.692+0000 7fdb0ba00640 1 -- 172.21.15.83:0/3777741228 shutdown_connections 2024-04-24T08:53:27.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.693+0000 7fdb0ba00640 1 -- 172.21.15.83:0/3777741228 wait complete. 2024-04-24T08:53:27.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.694+0000 7fdb0ba00640 1 Processor -- start 2024-04-24T08:53:27.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.694+0000 7fdb0ba00640 1 -- start start 2024-04-24T08:53:27.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.694+0000 7fdb0ba00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb04108610 0x7fdb0419ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:27.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.694+0000 7fdb0ba00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdb04108fd0 0x7fdb0419f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:27.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.694+0000 7fdb0ba00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb0410e660 0x7fdb041a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:27.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.694+0000 7fdb0ba00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdb041188e0 con 0x7fdb0410e660 2024-04-24T08:53:27.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.694+0000 7fdb0ba00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdb04118760 con 0x7fdb04108610 2024-04-24T08:53:27.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.694+0000 7fdb0ba00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdb04118a60 con 0x7fdb04108fd0 2024-04-24T08:53:27.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.695+0000 7fdb09a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb0410e660 0x7fdb041a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:27.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.695+0000 7fdb09a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb0410e660 0x7fdb041a3670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50864/0 (socket says 172.21.15.83:50864) 2024-04-24T08:53:27.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.695+0000 7fdb09a00640 1 -- 172.21.15.83:0/946486112 learned_addr learned my addr 172.21.15.83:0/946486112 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:53:27.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.695+0000 7fdb09000640 1 --2- 172.21.15.83:0/946486112 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb04108610 0x7fdb0419ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:27.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.695+0000 7fdafbe00640 1 --2- 172.21.15.83:0/946486112 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdb04108fd0 0x7fdb0419f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:27.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.695+0000 7fdb09a00640 1 -- 172.21.15.83:0/946486112 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdb04108fd0 msgr2=0x7fdb0419f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:27.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.695+0000 7fdb09a00640 1 --2- 172.21.15.83:0/946486112 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdb04108fd0 0x7fdb0419f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:27.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.695+0000 7fdb09a00640 1 -- 172.21.15.83:0/946486112 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb04108610 msgr2=0x7fdb0419ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:27.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.695+0000 7fdb09a00640 1 --2- 172.21.15.83:0/946486112 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb04108610 0x7fdb0419ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:27.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.695+0000 7fdb09a00640 1 -- 172.21.15.83:0/946486112 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb041a3da0 con 0x7fdb0410e660 2024-04-24T08:53:27.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.695+0000 7fdb09000640 1 --2- 172.21.15.83:0/946486112 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb04108610 0x7fdb0419ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:53:27.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.695+0000 7fdb09a00640 1 --2- 172.21.15.83:0/946486112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb0410e660 0x7fdb041a3670 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fdafc00d750 tx=0x7fdafc00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:27.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.696+0000 7fdb02000640 1 -- 172.21.15.83:0/946486112 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdafc017070 con 0x7fdb0410e660 2024-04-24T08:53:27.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.696+0000 7fdb02000640 1 -- 172.21.15.83:0/946486112 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdafc0054d0 con 0x7fdb0410e660 2024-04-24T08:53:27.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.696+0000 7fdb0ba00640 1 -- 172.21.15.83:0/946486112 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb041a4090 con 0x7fdb0410e660 2024-04-24T08:53:27.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.696+0000 7fdb02000640 1 -- 172.21.15.83:0/946486112 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdafc004c60 con 0x7fdb0410e660 2024-04-24T08:53:27.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.696+0000 7fdb0ba00640 1 -- 172.21.15.83:0/946486112 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb041ab920 con 0x7fdb0410e660 2024-04-24T08:53:27.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.699+0000 7fdb0e82d740 1 Processor -- start 2024-04-24T08:53:27.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.700+0000 7fdb02000640 1 -- 172.21.15.83:0/946486112 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fdafc004e00 con 0x7fdb0410e660 2024-04-24T08:53:27.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.700+0000 7fdb0e82d740 1 -- start start 2024-04-24T08:53:27.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.700+0000 7fdb0e82d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e22e77d860 0x55e22e77dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:27.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.700+0000 7fdb0e82d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e22e77e180 0x55e22e788310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:27.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.700+0000 7fdb0e82d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e22e788850 0x55e22e78ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:27.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.700+0000 7fdb0e82d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e22e6e31f0 con 0x55e22e788850 2024-04-24T08:53:27.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.700+0000 7fdb0e82d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e22e689350 con 0x55e22e77d860 2024-04-24T08:53:27.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.700+0000 7fdb0e82d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55e22e6df1b0 con 0x55e22e77e180 2024-04-24T08:53:27.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.701+0000 7fdafbe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e22e788850 0x55e22e78ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:27.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.701+0000 7fdafbe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e22e788850 0x55e22e78ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50870/0 (socket says 172.21.15.83:50870) 2024-04-24T08:53:27.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.701+0000 7fdafbe00640 1 -- 172.21.15.83:0/1784079496 learned_addr learned my addr 172.21.15.83:0/1784079496 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:53:27.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.701+0000 7fdb09000640 1 --2- 172.21.15.83:0/1784079496 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e22e77e180 0x55e22e788310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:27.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.701+0000 7fdafbe00640 1 --2- 172.21.15.83:0/1784079496 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e22e77d860 0x55e22e77dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:27.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.701+0000 7fdafbe00640 1 -- 172.21.15.83:0/1784079496 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e22e77e180 msgr2=0x55e22e788310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:27.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.701+0000 7fdafbe00640 1 --2- 172.21.15.83:0/1784079496 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e22e77e180 0x55e22e788310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:27.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.701+0000 7fdafbe00640 1 -- 172.21.15.83:0/1784079496 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e22e77d860 msgr2=0x55e22e77dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:27.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.701+0000 7fdafbe00640 1 --2- 172.21.15.83:0/1784079496 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e22e77d860 0x55e22e77dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:27.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.701+0000 7fdafbe00640 1 -- 172.21.15.83:0/1784079496 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e22e6ddf50 con 0x55e22e788850 2024-04-24T08:53:27.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.701+0000 7fdb02000640 1 --2- 172.21.15.83:0/946486112 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fdae80a3d20 0x7fdae80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:27.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.702+0000 7fdafbe00640 1 --2- 172.21.15.83:0/1784079496 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e22e788850 0x55e22e78ac40 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fdaec00efd0 tx=0x7fdaec00e5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=331d6bb954a1c7da server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:27.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.702+0000 7fdb02000640 1 -- 172.21.15.83:0/946486112 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7fdafc0d0620 con 0x7fdb0410e660 2024-04-24T08:53:27.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.702+0000 7fdb09000640 1 --2- 172.21.15.83:0/946486112 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fdae80a3d20 0x7fdae80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:27.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.702+0000 7fdafa000640 1 -- 172.21.15.83:0/1784079496 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaec018030 con 0x55e22e788850 2024-04-24T08:53:27.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.702+0000 7fdafa000640 1 -- 172.21.15.83:0/1784079496 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdaec01c870 con 0x55e22e788850 2024-04-24T08:53:27.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.702+0000 7fdafa000640 1 -- 172.21.15.83:0/1784079496 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaec010930 con 0x55e22e788850 2024-04-24T08:53:27.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.703+0000 7fdb0e82d740 1 -- 172.21.15.83:0/1784079496 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e22e788850 msgr2=0x55e22e78ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:27.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.703+0000 7fdb09000640 1 --2- 172.21.15.83:0/946486112 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fdae80a3d20 0x7fdae80a61e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fdaf402ae00 tx=0x7fdaf4000f40 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:27.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.703+0000 7fdb0e82d740 1 --2- 172.21.15.83:0/1784079496 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e22e788850 0x55e22e78ac40 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fdaec00efd0 tx=0x7fdaec00e5b0 comp rx=0 tx=0).stop 2024-04-24T08:53:27.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.703+0000 7fdb0e82d740 1 -- 172.21.15.83:0/1784079496 shutdown_connections 2024-04-24T08:53:27.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.703+0000 7fdb0e82d740 1 --2- 172.21.15.83:0/1784079496 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e22e788850 0x55e22e78ac40 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:27.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.703+0000 7fdb0e82d740 1 --2- 172.21.15.83:0/1784079496 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e22e77e180 0x55e22e788310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:27.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.703+0000 7fdb0e82d740 1 --2- 172.21.15.83:0/1784079496 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e22e77d860 0x55e22e77dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:27.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.703+0000 7fdb0e82d740 1 -- 172.21.15.83:0/1784079496 >> 172.21.15.83:0/1784079496 conn(0x55e22e779b80 msgr2=0x55e22e779f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:27.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.703+0000 7fdb0e82d740 1 -- 172.21.15.83:0/1784079496 shutdown_connections 2024-04-24T08:53:27.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.703+0000 7fdb0e82d740 1 -- 172.21.15.83:0/1784079496 wait complete. 2024-04-24T08:53:27.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.704+0000 7fdb0e82d740 1 -- start start 2024-04-24T08:53:27.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.704+0000 7fdb0e82d740 1 Processor -- start 2024-04-24T08:53:27.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.704+0000 7fdb0e82d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e22e77d860 0x55e22e7857d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:27.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.704+0000 7fdb0e82d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e22e77e180 0x55e22e785d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:27.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.705+0000 7fdb0e82d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e22e788850 0x55e22e786250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:27.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.705+0000 7fdb0e82d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e22e78b640 con 0x55e22e788850 2024-04-24T08:53:27.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.705+0000 7fdb0e82d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e22e69c690 con 0x55e22e77d860 2024-04-24T08:53:27.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.705+0000 7fdb0e82d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55e22e78b7c0 con 0x55e22e77e180 2024-04-24T08:53:27.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.705+0000 7fdb0e82d740 10 client.0 _ll_register_callbacks cb 0x55e22e6d8650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:53:27.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.705+0000 7fdb09a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e22e788850 0x55e22e786250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:27.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.705+0000 7fdb09a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e22e788850 0x55e22e786250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50880/0 (socket says 172.21.15.83:50880) 2024-04-24T08:53:27.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.705+0000 7fdb09a00640 1 -- 172.21.15.83:0/3041065875 learned_addr learned my addr 172.21.15.83:0/3041065875 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:53:27.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.705+0000 7fdb09000640 1 --2- 172.21.15.83:0/3041065875 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e22e77d860 0x55e22e7857d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:27.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.705+0000 7fdafbe00640 1 --2- 172.21.15.83:0/3041065875 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e22e77e180 0x55e22e785d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:27.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.705+0000 7fdb09a00640 1 -- 172.21.15.83:0/3041065875 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e22e77e180 msgr2=0x55e22e785d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:27.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.705+0000 7fdb09a00640 1 --2- 172.21.15.83:0/3041065875 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e22e77e180 0x55e22e785d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:27.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.705+0000 7fdb09a00640 1 -- 172.21.15.83:0/3041065875 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e22e77d860 msgr2=0x55e22e7857d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:27.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.705+0000 7fdb09a00640 1 --2- 172.21.15.83:0/3041065875 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e22e77d860 0x55e22e7857d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:27.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.705+0000 7fdb09a00640 1 -- 172.21.15.83:0/3041065875 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e22e69b920 con 0x55e22e788850 2024-04-24T08:53:27.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.705+0000 7fdb09000640 1 --2- 172.21.15.83:0/3041065875 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e22e77d860 0x55e22e7857d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:53:27.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.706+0000 7fdb09a00640 1 --2- 172.21.15.83:0/3041065875 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e22e788850 0x55e22e786250 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fdafc098650 tx=0x7fdafc0057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8632444283aa5b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:27.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.706+0000 7fdaf2a00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:53:27.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.706+0000 7fdaf2a00640 1 -- 172.21.15.83:0/3041065875 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdafc0d0070 con 0x55e22e788850 2024-04-24T08:53:27.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.706+0000 7fdaf2a00640 1 -- 172.21.15.83:0/3041065875 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7fdafc0b5680 con 0x55e22e788850 2024-04-24T08:53:27.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.706+0000 7fdb0e82d740 1 -- 172.21.15.83:0/3041065875 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdaec01c870 con 0x55e22e788850 2024-04-24T08:53:27.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.706+0000 7fdaf2a00640 1 -- 172.21.15.83:0/3041065875 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fdafc020040 con 0x55e22e788850 2024-04-24T08:53:27.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.706+0000 7fdb0e82d740 10 client.16413 fetch_fsmap learned FSMap version 28 2024-04-24T08:53:27.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.706+0000 7fdb0e82d740 1 -- 172.21.15.83:0/3041065875 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e22e4bf7d0 con 0x55e22e788850 2024-04-24T08:53:27.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.707+0000 7fdaf2a00640 1 -- 172.21.15.83:0/3041065875 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdaec01c870 con 0x55e22e788850 2024-04-24T08:53:27.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.707+0000 7fdaf2a00640 1 -- 172.21.15.83:0/3041065875 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fdafc0a0120 con 0x55e22e788850 2024-04-24T08:53:27.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.707+0000 7fdb0e82d740 10 client.16413 fetch_fsmap finished waiting for FSMap version 28 2024-04-24T08:53:27.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.707+0000 7fdb0e82d740 10 client.16413 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:53:27.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.707+0000 7fdb0e82d740 20 client.16413 populate_metadata read hostname 'smithi083' 2024-04-24T08:53:27.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.707+0000 7fdb0e82d740 1 --2- 172.21.15.83:0/3041065875 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55e22e798c40 0x55e22e79b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:27.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.707+0000 7fdb0e82d740 4 client.16413 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:53:27.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.707+0000 7fdb0e82d740 1 -- 172.21.15.83:0/3041065875 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e22e6f0c10 con 0x55e22e798c40 2024-04-24T08:53:27.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.708+0000 7fdb09000640 1 --2- 172.21.15.83:0/3041065875 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55e22e798c40 0x55e22e79b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:27.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.709+0000 7fdaf2a00640 10 client.16413 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:53:27.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.709+0000 7fdb09000640 1 --2- 172.21.15.83:0/3041065875 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55e22e798c40 0x55e22e79b070 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=abcce2808123f80f server_cookie=ceeba60ae61904eb in_seq=0 out_seq=0 2024-04-24T08:53:27.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.713+0000 7fdaf2a00640 1 -- 172.21.15.83:0/3041065875 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e22e6f0c10 con 0x55e22e798c40 2024-04-24T08:53:27.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.713+0000 7fdaf2a00640 10 client.16413 handle_command_reply: tid=0 2024-04-24T08:53:27.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.713+0000 7fdb0e82d740 1 client.16413 shutdown 2024-04-24T08:53:27.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.714+0000 7fdb0e82d740 1 -- 172.21.15.83:0/3041065875 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e22e788850 msgr2=0x55e22e786250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:27.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.714+0000 7fdb0e82d740 1 --2- 172.21.15.83:0/3041065875 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e22e788850 0x55e22e786250 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fdafc098650 tx=0x7fdafc0057c0 comp rx=0 tx=0).stop 2024-04-24T08:53:27.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.714+0000 7fdb0e82d740 1 -- 172.21.15.83:0/3041065875 shutdown_connections 2024-04-24T08:53:27.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.714+0000 7fdb0e82d740 1 --2- 172.21.15.83:0/3041065875 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55e22e798c40 0x55e22e79b070 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:27.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.714+0000 7fdb0e82d740 1 --2- 172.21.15.83:0/3041065875 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e22e788850 0x55e22e786250 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:27.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.714+0000 7fdb0e82d740 1 --2- 172.21.15.83:0/3041065875 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e22e77e180 0x55e22e785d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:27.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.714+0000 7fdb0e82d740 1 --2- 172.21.15.83:0/3041065875 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e22e77d860 0x55e22e7857d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:27.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.714+0000 7fdaf2a00640 0 client.16413 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:53:27.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.714+0000 7fdb0e82d740 1 -- 172.21.15.83:0/3041065875 >> 172.21.15.83:0/3041065875 conn(0x55e22e779b80 msgr2=0x55e22e77f980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:27.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.714+0000 7fdb0e82d740 1 -- 172.21.15.83:0/3041065875 shutdown_connections 2024-04-24T08:53:27.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.714+0000 7fdb0e82d740 1 -- 172.21.15.83:0/3041065875 wait complete. 2024-04-24T08:53:27.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.714+0000 7fdb0e82d740 20 client.16413 trim_cache size 0 max 16384 2024-04-24T08:53:27.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.737+0000 7fdb0e82d740 1 Processor -- start 2024-04-24T08:53:27.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.737+0000 7fdb0e82d740 1 -- start start 2024-04-24T08:53:27.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.737+0000 7fdb0e82d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e22e77d860 0x55e22e794530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:27.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.737+0000 7fdb0e82d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e22e77e180 0x55e22e784930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:27.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.737+0000 7fdb0e82d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e22e788850 0x55e22e77ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:27.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.737+0000 7fdb0e82d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdb041b5ba0 con 0x55e22e788850 2024-04-24T08:53:27.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.737+0000 7fdb0e82d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdb041b76e0 con 0x55e22e77d860 2024-04-24T08:53:27.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.737+0000 7fdb0e82d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdb041b6270 con 0x55e22e77e180 2024-04-24T08:53:27.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.738+0000 7fdb09a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e22e788850 0x55e22e77ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:27.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.738+0000 7fdb09a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e22e788850 0x55e22e77ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50894/0 (socket says 172.21.15.83:50894) 2024-04-24T08:53:27.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.738+0000 7fdb09a00640 1 -- 172.21.15.83:0/3735853659 learned_addr learned my addr 172.21.15.83:0/3735853659 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:53:27.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.738+0000 7fdafbe00640 1 --2- 172.21.15.83:0/3735853659 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e22e77e180 0x55e22e784930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:27.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.738+0000 7fdb09000640 1 --2- 172.21.15.83:0/3735853659 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e22e77d860 0x55e22e794530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:27.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.738+0000 7fdb09a00640 1 -- 172.21.15.83:0/3735853659 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e22e77e180 msgr2=0x55e22e784930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:27.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.738+0000 7fdb09a00640 1 --2- 172.21.15.83:0/3735853659 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e22e77e180 0x55e22e784930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:27.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.738+0000 7fdb09a00640 1 -- 172.21.15.83:0/3735853659 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e22e77d860 msgr2=0x55e22e794530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:27.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.738+0000 7fdb09a00640 1 --2- 172.21.15.83:0/3735853659 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e22e77d860 0x55e22e794530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:27.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.738+0000 7fdb09a00640 1 -- 172.21.15.83:0/3735853659 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e22e3f53f0 con 0x55e22e788850 2024-04-24T08:53:27.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.739+0000 7fdb09a00640 1 --2- 172.21.15.83:0/3735853659 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e22e788850 0x55e22e77ffc0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fdafc00d720 tx=0x7fdafc000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbf0ee1a1d7f3122 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:27.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.739+0000 7fdaf9600640 1 -- 172.21.15.83:0/3735853659 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdafc062070 con 0x55e22e788850 2024-04-24T08:53:27.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.739+0000 7fdaf9600640 1 -- 172.21.15.83:0/3735853659 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdafc0a4c50 con 0x55e22e788850 2024-04-24T08:53:27.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.739+0000 7fdaf9600640 1 -- 172.21.15.83:0/3735853659 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdafc0d0040 con 0x55e22e788850 2024-04-24T08:53:27.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.740+0000 7fdb0e82d740 1 -- 172.21.15.83:0/3735853659 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e22e788850 msgr2=0x55e22e77ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:27.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.740+0000 7fdb0e82d740 1 --2- 172.21.15.83:0/3735853659 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e22e788850 0x55e22e77ffc0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fdafc00d720 tx=0x7fdafc000bc0 comp rx=0 tx=0).stop 2024-04-24T08:53:27.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.740+0000 7fdb0e82d740 1 -- 172.21.15.83:0/3735853659 shutdown_connections 2024-04-24T08:53:27.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.740+0000 7fdb0e82d740 1 --2- 172.21.15.83:0/3735853659 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e22e788850 0x55e22e77ffc0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:27.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.740+0000 7fdb0e82d740 1 --2- 172.21.15.83:0/3735853659 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e22e77e180 0x55e22e784930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:27.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.740+0000 7fdb0e82d740 1 --2- 172.21.15.83:0/3735853659 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e22e77d860 0x55e22e794530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:27.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.740+0000 7fdb0e82d740 1 -- 172.21.15.83:0/3735853659 >> 172.21.15.83:0/3735853659 conn(0x55e22e779b80 msgr2=0x55e22e778390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:27.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.740+0000 7fdb0e82d740 1 -- 172.21.15.83:0/3735853659 shutdown_connections 2024-04-24T08:53:27.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.740+0000 7fdb0e82d740 1 -- 172.21.15.83:0/3735853659 wait complete. 2024-04-24T08:53:27.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.740+0000 7fdb0e82d740 1 -- start start 2024-04-24T08:53:27.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.740+0000 7fdb0e82d740 1 Processor -- start 2024-04-24T08:53:27.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.741+0000 7fdb0e82d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e22e77d860 0x55e22e7906a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:27.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.741+0000 7fdb0e82d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e22e77e180 0x55e22e790be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:27.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.741+0000 7fdb0e82d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e22e788850 0x55e22e78bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:27.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.741+0000 7fdb0e82d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdb041b7010 con 0x55e22e788850 2024-04-24T08:53:27.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.741+0000 7fdb0e82d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdb041b6940 con 0x55e22e77d860 2024-04-24T08:53:27.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.741+0000 7fdb0e82d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdb041b3b80 con 0x55e22e77e180 2024-04-24T08:53:27.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.741+0000 7fdb0e82d740 10 client.0 _ll_register_callbacks cb 0x7fdafc09bfc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:53:27.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.741+0000 7fdb09a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e22e788850 0x55e22e78bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:27.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.741+0000 7fdb09a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e22e788850 0x55e22e78bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50910/0 (socket says 172.21.15.83:50910) 2024-04-24T08:53:27.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.741+0000 7fdb09a00640 1 -- 172.21.15.83:0/2943191301 learned_addr learned my addr 172.21.15.83:0/2943191301 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:53:27.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.741+0000 7fdafbe00640 1 --2- 172.21.15.83:0/2943191301 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e22e77e180 0x55e22e790be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:27.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.741+0000 7fdb09000640 1 --2- 172.21.15.83:0/2943191301 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e22e77d860 0x55e22e7906a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:27.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.742+0000 7fdb09a00640 1 -- 172.21.15.83:0/2943191301 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e22e77e180 msgr2=0x55e22e790be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:27.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.742+0000 7fdb09a00640 1 --2- 172.21.15.83:0/2943191301 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e22e77e180 0x55e22e790be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:27.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.742+0000 7fdb09a00640 1 -- 172.21.15.83:0/2943191301 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e22e77d860 msgr2=0x55e22e7906a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:27.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.742+0000 7fdb09a00640 1 --2- 172.21.15.83:0/2943191301 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e22e77d860 0x55e22e7906a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:27.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.742+0000 7fdb09a00640 1 -- 172.21.15.83:0/2943191301 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e22e3b50f0 con 0x55e22e788850 2024-04-24T08:53:27.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.742+0000 7fdafbe00640 1 --2- 172.21.15.83:0/2943191301 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e22e77e180 0x55e22e790be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:53:27.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.742+0000 7fdb09a00640 1 --2- 172.21.15.83:0/2943191301 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e22e788850 0x55e22e78bb60 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fdafc0cda50 tx=0x7fdafc0cde50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b14a00711da0b6b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:27.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.742+0000 7fdaf2a00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:53:27.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.742+0000 7fdaf2a00640 1 -- 172.21.15.83:0/2943191301 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdafc062070 con 0x55e22e788850 2024-04-24T08:53:27.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.742+0000 7fdaf2a00640 1 -- 172.21.15.83:0/2943191301 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7fdafc098140 con 0x55e22e788850 2024-04-24T08:53:27.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.742+0000 7fdb0e82d740 1 -- 172.21.15.83:0/2943191301 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdafc0a4c50 con 0x55e22e788850 2024-04-24T08:53:27.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.742+0000 7fdaf2a00640 1 -- 172.21.15.83:0/2943191301 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fdafc0d0040 con 0x55e22e788850 2024-04-24T08:53:27.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.742+0000 7fdb0e82d740 10 client.16419 fetch_fsmap learned FSMap version 28 2024-04-24T08:53:27.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.742+0000 7fdb0e82d740 1 -- 172.21.15.83:0/2943191301 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e22e6a8810 con 0x55e22e788850 2024-04-24T08:53:27.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.743+0000 7fdaf2a00640 1 -- 172.21.15.83:0/2943191301 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdafc0a4c50 con 0x55e22e788850 2024-04-24T08:53:27.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.743+0000 7fdaf2a00640 1 -- 172.21.15.83:0/2943191301 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fdafc0a1730 con 0x55e22e788850 2024-04-24T08:53:27.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.743+0000 7fdb0e82d740 10 client.16419 fetch_fsmap finished waiting for FSMap version 28 2024-04-24T08:53:27.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.743+0000 7fdb0e82d740 10 client.16419 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:53:27.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.743+0000 7fdb0e82d740 20 client.16419 populate_metadata read hostname 'smithi083' 2024-04-24T08:53:27.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.743+0000 7fdb0e82d740 1 --2- 172.21.15.83:0/2943191301 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55e22e7feaf0 0x55e22e800ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:27.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.743+0000 7fdb0e82d740 4 client.16419 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T08:53:27.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.743+0000 7fdb0e82d740 1 -- 172.21.15.83:0/2943191301 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e22e6eaa80 con 0x55e22e7feaf0 2024-04-24T08:53:27.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.743+0000 7fdb09000640 1 --2- 172.21.15.83:0/2943191301 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55e22e7feaf0 0x55e22e800ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:27.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.745+0000 7fdaf2a00640 10 client.16419 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:53:27.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.745+0000 7fdb09000640 1 --2- 172.21.15.83:0/2943191301 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55e22e7feaf0 0x55e22e800ee0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a764b919edd88869 server_cookie=8b1c2c6ba0e233cf in_seq=0 out_seq=0 2024-04-24T08:53:27.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.748+0000 7fdaf2a00640 1 -- 172.21.15.83:0/2943191301 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55e22e6eaa80 con 0x55e22e7feaf0 2024-04-24T08:53:27.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.748+0000 7fdaf2a00640 10 client.16419 handle_command_reply: tid=0 2024-04-24T08:53:27.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.748+0000 7fdb0e82d740 1 client.16419 shutdown 2024-04-24T08:53:27.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.749+0000 7fdb0e82d740 1 -- 172.21.15.83:0/2943191301 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e22e788850 msgr2=0x55e22e78bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:27.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.749+0000 7fdb0e82d740 1 --2- 172.21.15.83:0/2943191301 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e22e788850 0x55e22e78bb60 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fdafc0cda50 tx=0x7fdafc0cde50 comp rx=0 tx=0).stop 2024-04-24T08:53:27.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.749+0000 7fdb0e82d740 1 -- 172.21.15.83:0/2943191301 shutdown_connections 2024-04-24T08:53:27.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.749+0000 7fdb0e82d740 1 --2- 172.21.15.83:0/2943191301 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55e22e7feaf0 0x55e22e800ee0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:27.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.749+0000 7fdb0e82d740 1 --2- 172.21.15.83:0/2943191301 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e22e788850 0x55e22e78bb60 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:27.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.749+0000 7fdb0e82d740 1 --2- 172.21.15.83:0/2943191301 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e22e77e180 0x55e22e790be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:27.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.749+0000 7fdb0e82d740 1 --2- 172.21.15.83:0/2943191301 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e22e77d860 0x55e22e7906a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:27.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.749+0000 7fdaf2a00640 0 client.16419 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:53:27.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.749+0000 7fdb0e82d740 1 -- 172.21.15.83:0/2943191301 >> 172.21.15.83:0/2943191301 conn(0x55e22e779b80 msgr2=0x55e22e786020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:27.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.749+0000 7fdb0e82d740 1 -- 172.21.15.83:0/2943191301 shutdown_connections 2024-04-24T08:53:27.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.749+0000 7fdb0e82d740 1 -- 172.21.15.83:0/2943191301 wait complete. 2024-04-24T08:53:27.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.749+0000 7fdb0e82d740 20 client.16419 trim_cache size 0 max 16384 2024-04-24T08:53:27.750 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T08:53:27.751 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T08:53:27.751 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-04-24T08:53:27.751 INFO:teuthology.orchestra.run.smithi083.stdout: "98f1c9e6-a0a9-4235-9983-e7955f7dc728": { 2024-04-24T08:53:27.751 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-04-24T08:53:27.751 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "98f1c9e6-a0a9-4235-9983-e7955f7dc728", 2024-04-24T08:53:27.751 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-04-24T08:53:27.751 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T08:53:27.751 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T08:53:27.751 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T08:53:27.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.750+0000 7fdb0ba00640 1 -- 172.21.15.83:0/946486112 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fdae80a3d20 msgr2=0x7fdae80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:27.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.750+0000 7fdb0ba00640 1 --2- 172.21.15.83:0/946486112 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fdae80a3d20 0x7fdae80a61e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fdaf402ae00 tx=0x7fdaf4000f40 comp rx=0 tx=0).stop 2024-04-24T08:53:27.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.750+0000 7fdb0ba00640 1 -- 172.21.15.83:0/946486112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb0410e660 msgr2=0x7fdb041a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:27.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.750+0000 7fdb0ba00640 1 --2- 172.21.15.83:0/946486112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb0410e660 0x7fdb041a3670 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fdafc00d750 tx=0x7fdafc00dc10 comp rx=0 tx=0).stop 2024-04-24T08:53:27.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.750+0000 7fdb0ba00640 1 -- 172.21.15.83:0/946486112 shutdown_connections 2024-04-24T08:53:27.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.750+0000 7fdb0ba00640 1 --2- 172.21.15.83:0/946486112 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fdae80a3d20 0x7fdae80a61e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:27.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.750+0000 7fdb0ba00640 1 --2- 172.21.15.83:0/946486112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb0410e660 0x7fdb041a3670 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:27.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.750+0000 7fdb0ba00640 1 --2- 172.21.15.83:0/946486112 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdb04108fd0 0x7fdb0419f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:27.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.750+0000 7fdb0ba00640 1 --2- 172.21.15.83:0/946486112 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb04108610 0x7fdb0419ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:27.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.750+0000 7fdb0ba00640 1 -- 172.21.15.83:0/946486112 >> 172.21.15.83:0/946486112 conn(0x7fdb04076540 msgr2=0x7fdb04077d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:27.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.750+0000 7fdb0ba00640 1 -- 172.21.15.83:0/946486112 shutdown_connections 2024-04-24T08:53:27.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:27.750+0000 7fdb0ba00640 1 -- 172.21.15.83:0/946486112 wait complete. 2024-04-24T08:53:27.770 INFO:tasks.cephfs.filesystem:scrub status for tag:98f1c9e6-a0a9-4235-9983-e7955f7dc728 - {'path': '/', 'tag': '98f1c9e6-a0a9-4235-9983-e7955f7dc728', 'options': 'recursive,force'} 2024-04-24T08:53:27.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:53:27 smithi083 ceph-mon[27907]: pgmap v58: 129 pgs: 129 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-24T08:53:28.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:53:27 smithi138 ceph-mon[31441]: pgmap v58: 129 pgs: 129 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-24T08:53:28.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:53:27 smithi189 ceph-mon[31791]: pgmap v58: 129 pgs: 129 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-24T08:53:28.554 INFO:tasks.workunit.client.0.smithi083.stdout: 1 522 9.02 MB/sec warmup 2 sec latency 31.896 ms 2024-04-24T08:53:29.109 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:53:28 smithi189 ceph-mon[31791]: scrub queued for path: / 2024-04-24T08:53:29.109 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:53:28 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T08:53:29.109 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:53:28 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T08:53:29.110 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:53:28 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T08:53:29.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:53:28 smithi138 ceph-mon[31441]: scrub queued for path: / 2024-04-24T08:53:29.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:53:28 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T08:53:29.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:53:28 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T08:53:29.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:53:28 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T08:53:29.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:53:28 smithi083 ceph-mon[27907]: scrub queued for path: / 2024-04-24T08:53:29.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:53:28 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T08:53:29.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:53:28 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T08:53:29.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:53:28 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T08:53:29.554 INFO:tasks.workunit.client.0.smithi083.stdout: 1 728 7.10 MB/sec warmup 3 sec latency 54.476 ms 2024-04-24T08:53:30.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:53:29 smithi138 ceph-mon[31441]: pgmap v59: 129 pgs: 129 active+clean; 9.6 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 708 KiB/s wr, 16 op/s 2024-04-24T08:53:30.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:53:29 smithi189 ceph-mon[31791]: pgmap v59: 129 pgs: 129 active+clean; 9.6 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 708 KiB/s wr, 16 op/s 2024-04-24T08:53:30.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:53:29 smithi083 ceph-mon[27907]: pgmap v59: 129 pgs: 129 active+clean; 9.6 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 708 KiB/s wr, 16 op/s 2024-04-24T08:53:30.554 INFO:tasks.workunit.client.0.smithi083.stdout: 1 864 5.39 MB/sec warmup 4 sec latency 56.738 ms 2024-04-24T08:53:31.554 INFO:tasks.workunit.client.0.smithi083.stdout: 1 1027 4.58 MB/sec warmup 5 sec latency 49.250 ms 2024-04-24T08:53:32.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:53:31 smithi138 ceph-mon[31441]: pgmap v60: 129 pgs: 129 active+clean; 9.6 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 707 KiB/s wr, 11 op/s 2024-04-24T08:53:32.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:53:31 smithi189 ceph-mon[31791]: pgmap v60: 129 pgs: 129 active+clean; 9.6 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 707 KiB/s wr, 11 op/s 2024-04-24T08:53:32.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:53:31 smithi083 ceph-mon[27907]: pgmap v60: 129 pgs: 129 active+clean; 9.6 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 707 KiB/s wr, 11 op/s 2024-04-24T08:53:32.554 INFO:tasks.workunit.client.0.smithi083.stdout: 1 1156 3.83 MB/sec warmup 6 sec latency 54.686 ms 2024-04-24T08:53:33.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:53:32 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T08:53:33.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:53:32 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T08:53:33.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:53:32 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T08:53:33.554 INFO:tasks.workunit.client.0.smithi083.stdout: 1 1278 3.31 MB/sec warmup 7 sec latency 78.241 ms 2024-04-24T08:53:34.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:53:33 smithi138 ceph-mon[31441]: pgmap v61: 129 pgs: 129 active+clean; 22 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 1.8 MiB/s wr, 32 op/s 2024-04-24T08:53:34.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:53:33 smithi189 ceph-mon[31791]: pgmap v61: 129 pgs: 129 active+clean; 22 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 1.8 MiB/s wr, 32 op/s 2024-04-24T08:53:34.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:53:33 smithi083 ceph-mon[27907]: pgmap v61: 129 pgs: 129 active+clean; 22 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 1.8 MiB/s wr, 32 op/s 2024-04-24T08:53:34.450 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.450+0000 7f2fb9200640 1 -- 172.21.15.83:0/1794339872 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fb4105560 msgr2=0x7f2fb410d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:34.451 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.450+0000 7f2fb9200640 1 --2- 172.21.15.83:0/1794339872 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fb4105560 0x7f2fb410d130 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f2fac00b3f0 tx=0x7f2fac02ac90 comp rx=0 tx=0).stop 2024-04-24T08:53:34.451 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.450+0000 7f2fb9200640 1 -- 172.21.15.83:0/1794339872 shutdown_connections 2024-04-24T08:53:34.451 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.450+0000 7f2fb9200640 1 --2- 172.21.15.83:0/1794339872 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fb4105560 0x7f2fb410d130 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:34.451 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.450+0000 7f2fb9200640 1 --2- 172.21.15.83:0/1794339872 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2fb4100e40 0x7f2fb4104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:34.451 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.450+0000 7f2fb9200640 1 --2- 172.21.15.83:0/1794339872 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2fb4100490 0x7f2fb4100870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:34.451 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.450+0000 7f2fb9200640 1 -- 172.21.15.83:0/1794339872 >> 172.21.15.83:0/1794339872 conn(0x7f2fb4077000 msgr2=0x7f2fb40fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:34.451 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.450+0000 7f2fb9200640 1 -- 172.21.15.83:0/1794339872 shutdown_connections 2024-04-24T08:53:34.451 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.450+0000 7f2fb9200640 1 -- 172.21.15.83:0/1794339872 wait complete. 2024-04-24T08:53:34.453 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.451+0000 7f2fb9200640 1 Processor -- start 2024-04-24T08:53:34.453 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.451+0000 7f2fb9200640 1 -- start start 2024-04-24T08:53:34.454 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.451+0000 7f2fb9200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2fb4100490 0x7f2fb4198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:34.454 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.451+0000 7f2fb9200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fb4105560 0x7f2fb4198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:34.454 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.451+0000 7f2fb9200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2fb419cff0 0x7f2fb419d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:34.454 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.451+0000 7f2fb9200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb41100d0 con 0x7f2fb4105560 2024-04-24T08:53:34.454 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.451+0000 7f2fb9200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb410ff50 con 0x7f2fb4100490 2024-04-24T08:53:34.454 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.451+0000 7f2fb9200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb4110250 con 0x7f2fb419cff0 2024-04-24T08:53:34.454 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.452+0000 7f2fa9400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fb4105560 0x7f2fb4198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:34.454 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.452+0000 7f2fa9400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fb4105560 0x7f2fb4198c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38360/0 (socket says 172.21.15.83:38360) 2024-04-24T08:53:34.454 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.452+0000 7f2fa9400640 1 -- 172.21.15.83:0/3909960907 learned_addr learned my addr 172.21.15.83:0/3909960907 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:53:34.454 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.452+0000 7f2fb2800640 1 --2- 172.21.15.83:0/3909960907 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2fb419cff0 0x7f2fb419d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:34.454 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.452+0000 7f2fa9400640 1 -- 172.21.15.83:0/3909960907 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2fb419cff0 msgr2=0x7f2fb419d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:34.454 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.452+0000 7f2fa9400640 1 --2- 172.21.15.83:0/3909960907 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2fb419cff0 0x7f2fb419d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:34.454 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.452+0000 7f2fa9400640 1 -- 172.21.15.83:0/3909960907 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2fb4100490 msgr2=0x7f2fb4198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:34.454 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.452+0000 7f2fa9400640 1 --2- 172.21.15.83:0/3909960907 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2fb4100490 0x7f2fb4198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:34.455 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.452+0000 7f2fa9400640 1 -- 172.21.15.83:0/3909960907 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fb419dae0 con 0x7f2fb4105560 2024-04-24T08:53:34.455 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.452+0000 7f2fb2800640 1 --2- 172.21.15.83:0/3909960907 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2fb419cff0 0x7f2fb419d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:53:34.455 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.453+0000 7f2fa9400640 1 --2- 172.21.15.83:0/3909960907 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fb4105560 0x7f2fb4198c70 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c00ccb0 tx=0x7f2f9c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:34.455 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.453+0000 7f2fab400640 1 -- 172.21.15.83:0/3909960907 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f9c004b20 con 0x7f2fb4105560 2024-04-24T08:53:34.455 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.453+0000 7f2fab400640 1 -- 172.21.15.83:0/3909960907 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2f9c004cc0 con 0x7f2fb4105560 2024-04-24T08:53:34.455 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.453+0000 7f2fab400640 1 -- 172.21.15.83:0/3909960907 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f9c00f7c0 con 0x7f2fb4105560 2024-04-24T08:53:34.455 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.453+0000 7f2fb9200640 1 -- 172.21.15.83:0/3909960907 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fb419ddd0 con 0x7f2fb4105560 2024-04-24T08:53:34.455 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.453+0000 7f2fb9200640 1 -- 172.21.15.83:0/3909960907 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fb41a5610 con 0x7f2fb4105560 2024-04-24T08:53:34.456 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.456+0000 7f2fbc1c6740 1 Processor -- start 2024-04-24T08:53:34.456 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.456+0000 7f2fab400640 1 -- 172.21.15.83:0/3909960907 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2f9c00f960 con 0x7f2fb4105560 2024-04-24T08:53:34.458 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.457+0000 7f2fbc1c6740 1 -- start start 2024-04-24T08:53:34.458 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.457+0000 7f2fbc1c6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef3b82c0d0 0x55ef3b82c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:34.458 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.457+0000 7f2fbc1c6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef3b82c9f0 0x55ef3b83fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:34.458 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.457+0000 7f2fbc1c6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef3b832e10 0x55ef3b841fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:34.459 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.457+0000 7f2fbc1c6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ef3b792900 con 0x55ef3b82c0d0 2024-04-24T08:53:34.459 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.457+0000 7f2fbc1c6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ef3b738f20 con 0x55ef3b82c9f0 2024-04-24T08:53:34.459 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.457+0000 7f2fbc1c6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55ef3b78e8c0 con 0x55ef3b832e10 2024-04-24T08:53:34.459 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.457+0000 7f2fb2800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef3b82c0d0 0x55ef3b82c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:34.459 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.457+0000 7f2fb2800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef3b82c0d0 0x55ef3b82c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38372/0 (socket says 172.21.15.83:38372) 2024-04-24T08:53:34.459 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.457+0000 7f2fb2800640 1 -- 172.21.15.83:0/1093714544 learned_addr learned my addr 172.21.15.83:0/1093714544 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:53:34.459 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.458+0000 7f2fa9400640 1 --2- 172.21.15.83:0/1093714544 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef3b832e10 0x55ef3b841fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:34.459 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.458+0000 7f2fb1e00640 1 --2- 172.21.15.83:0/1093714544 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef3b82c9f0 0x55ef3b83fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:34.459 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.458+0000 7f2fb2800640 1 -- 172.21.15.83:0/1093714544 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef3b832e10 msgr2=0x55ef3b841fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:34.459 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.458+0000 7f2fb2800640 1 --2- 172.21.15.83:0/1093714544 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef3b832e10 0x55ef3b841fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:34.459 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.458+0000 7f2fb2800640 1 -- 172.21.15.83:0/1093714544 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef3b82c9f0 msgr2=0x55ef3b83fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:34.459 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.458+0000 7f2fb2800640 1 --2- 172.21.15.83:0/1093714544 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef3b82c9f0 0x55ef3b83fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:34.459 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.458+0000 7f2fb2800640 1 -- 172.21.15.83:0/1093714544 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef3b75de10 con 0x55ef3b82c0d0 2024-04-24T08:53:34.459 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.458+0000 7f2fb1e00640 1 --2- 172.21.15.83:0/1093714544 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef3b82c9f0 0x55ef3b83fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:53:34.464 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.458+0000 7f2fb2800640 1 --2- 172.21.15.83:0/1093714544 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef3b82c0d0 0x55ef3b82c4b0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f2fac005ab0 tx=0x7f2fac02b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3788020aa6903e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:34.464 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.459+0000 7f2fa3400640 1 -- 172.21.15.83:0/1093714544 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fac041020 con 0x55ef3b82c0d0 2024-04-24T08:53:34.464 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.459+0000 7f2fa3400640 1 -- 172.21.15.83:0/1093714544 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2fac02b7e0 con 0x55ef3b82c0d0 2024-04-24T08:53:34.464 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.459+0000 7f2fa3400640 1 -- 172.21.15.83:0/1093714544 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fac005640 con 0x55ef3b82c0d0 2024-04-24T08:53:34.464 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.459+0000 7f2fab400640 1 --2- 172.21.15.83:0/3909960907 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f2f980a3e50 0x7f2f980a6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:34.465 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.459+0000 7f2fab400640 1 -- 172.21.15.83:0/3909960907 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f2f9c0d0ff0 con 0x7f2fb4105560 2024-04-24T08:53:34.465 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.459+0000 7f2fb1e00640 1 --2- 172.21.15.83:0/3909960907 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f2f980a3e50 0x7f2f980a6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:34.465 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.459+0000 7f2fbc1c6740 1 -- 172.21.15.83:0/1093714544 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef3b82c0d0 msgr2=0x55ef3b82c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:34.465 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.459+0000 7f2fbc1c6740 1 --2- 172.21.15.83:0/1093714544 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef3b82c0d0 0x55ef3b82c4b0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f2fac005ab0 tx=0x7f2fac02b730 comp rx=0 tx=0).stop 2024-04-24T08:53:34.465 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.460+0000 7f2fbc1c6740 1 -- 172.21.15.83:0/1093714544 shutdown_connections 2024-04-24T08:53:34.465 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.460+0000 7f2fbc1c6740 1 --2- 172.21.15.83:0/1093714544 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef3b832e10 0x55ef3b841fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:34.465 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.460+0000 7f2fbc1c6740 1 --2- 172.21.15.83:0/1093714544 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef3b82c9f0 0x55ef3b83fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:34.465 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.460+0000 7f2fbc1c6740 1 --2- 172.21.15.83:0/1093714544 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef3b82c0d0 0x55ef3b82c4b0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:34.465 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.460+0000 7f2fbc1c6740 1 -- 172.21.15.83:0/1093714544 >> 172.21.15.83:0/1093714544 conn(0x55ef3b8286c0 msgr2=0x55ef3b828aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:34.465 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.460+0000 7f2fbc1c6740 1 -- 172.21.15.83:0/1093714544 shutdown_connections 2024-04-24T08:53:34.465 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.460+0000 7f2fbc1c6740 1 -- 172.21.15.83:0/1093714544 wait complete. 2024-04-24T08:53:34.465 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.460+0000 7f2fb1e00640 1 --2- 172.21.15.83:0/3909960907 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f2f980a3e50 0x7f2f980a6310 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x55ef3b82ecd0 tx=0x7f2fa4005d90 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:34.465 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.460+0000 7f2fbc1c6740 1 -- start start 2024-04-24T08:53:34.466 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.461+0000 7f2fbc1c6740 1 Processor -- start 2024-04-24T08:53:34.466 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.461+0000 7f2fbc1c6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef3b82c0d0 0x55ef3b834270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:34.466 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.461+0000 7f2fbc1c6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef3b82c9f0 0x55ef3b8347b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:34.466 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.461+0000 7f2fbc1c6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef3b834cf0 0x55ef3b84bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:34.466 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.461+0000 7f2fbc1c6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ef3b842780 con 0x55ef3b82c9f0 2024-04-24T08:53:34.466 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.461+0000 7f2fbc1c6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ef3b74c2c0 con 0x55ef3b82c0d0 2024-04-24T08:53:34.466 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.461+0000 7f2fbc1c6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55ef3b842900 con 0x55ef3b834cf0 2024-04-24T08:53:34.466 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.461+0000 7f2fbc1c6740 10 client.0 _ll_register_callbacks cb 0x55ef3b4a4a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:53:34.466 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.461+0000 7f2fa9400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef3b82c9f0 0x55ef3b8347b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:34.466 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.461+0000 7f2fa9400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef3b82c9f0 0x55ef3b8347b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38374/0 (socket says 172.21.15.83:38374) 2024-04-24T08:53:34.466 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.461+0000 7f2fa9400640 1 -- 172.21.15.83:0/3978128899 learned_addr learned my addr 172.21.15.83:0/3978128899 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:53:34.466 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.462+0000 7f2fb1e00640 1 --2- 172.21.15.83:0/3978128899 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef3b82c0d0 0x55ef3b834270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:34.466 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.462+0000 7f2fa9400640 1 -- 172.21.15.83:0/3978128899 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef3b834cf0 msgr2=0x55ef3b84bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:34.466 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.462+0000 7f2fb2800640 1 --2- 172.21.15.83:0/3978128899 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef3b834cf0 0x55ef3b84bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:34.467 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.462+0000 7f2fa9400640 1 --2- 172.21.15.83:0/3978128899 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef3b834cf0 0x55ef3b84bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:34.467 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.462+0000 7f2fa9400640 1 -- 172.21.15.83:0/3978128899 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef3b82c0d0 msgr2=0x55ef3b834270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:34.467 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.462+0000 7f2fa9400640 1 --2- 172.21.15.83:0/3978128899 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef3b82c0d0 0x55ef3b834270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:34.467 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.462+0000 7f2fa9400640 1 -- 172.21.15.83:0/3978128899 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef3b75cfd0 con 0x55ef3b82c9f0 2024-04-24T08:53:34.467 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.462+0000 7f2fb1e00640 1 --2- 172.21.15.83:0/3978128899 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef3b82c0d0 0x55ef3b834270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:53:34.467 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.462+0000 7f2fa9400640 1 --2- 172.21.15.83:0/3978128899 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef3b82c9f0 0x55ef3b8347b0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c000f80 tx=0x7f2f9c09c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8c3152edc3e495c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:34.467 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.462+0000 7f2f97e00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:53:34.467 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.462+0000 7f2f97e00640 1 -- 172.21.15.83:0/3978128899 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f9c01d040 con 0x55ef3b82c9f0 2024-04-24T08:53:34.467 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.462+0000 7f2f97e00640 1 -- 172.21.15.83:0/3978128899 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f2f9c00fc50 con 0x55ef3b82c9f0 2024-04-24T08:53:34.467 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.462+0000 7f2fbc1c6740 1 -- 172.21.15.83:0/3978128899 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2fac02b7e0 con 0x55ef3b82c9f0 2024-04-24T08:53:34.467 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.463+0000 7f2f97e00640 1 -- 172.21.15.83:0/3978128899 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2f9c00fe50 con 0x55ef3b82c9f0 2024-04-24T08:53:34.467 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.463+0000 7f2fbc1c6740 10 client.16431 fetch_fsmap learned FSMap version 28 2024-04-24T08:53:34.467 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.463+0000 7f2fbc1c6740 1 -- 172.21.15.83:0/3978128899 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ef3b4a5210 con 0x55ef3b82c9f0 2024-04-24T08:53:34.468 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.463+0000 7f2f97e00640 1 -- 172.21.15.83:0/3978128899 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2fac02b7e0 con 0x55ef3b82c9f0 2024-04-24T08:53:34.468 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.463+0000 7f2f97e00640 1 -- 172.21.15.83:0/3978128899 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f2f9c0a5340 con 0x55ef3b82c9f0 2024-04-24T08:53:34.468 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.463+0000 7f2fbc1c6740 10 client.16431 fetch_fsmap finished waiting for FSMap version 28 2024-04-24T08:53:34.468 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.463+0000 7f2fbc1c6740 10 client.16431 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T08:53:34.468 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.463+0000 7f2fbc1c6740 20 client.16431 populate_metadata read hostname 'smithi083' 2024-04-24T08:53:34.468 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.463+0000 7f2fbc1c6740 1 --2- 172.21.15.83:0/3978128899 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ef3b8501a0 0x55ef3b852590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:34.468 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.463+0000 7f2fbc1c6740 4 client.16431 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:53:34.468 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.463+0000 7f2fbc1c6740 1 -- 172.21.15.83:0/3978128899 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ef3b7a0320 con 0x55ef3b8501a0 2024-04-24T08:53:34.468 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.464+0000 7f2fb1e00640 1 --2- 172.21.15.83:0/3978128899 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ef3b8501a0 0x55ef3b852590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:34.469 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.465+0000 7f2fb1e00640 1 --2- 172.21.15.83:0/3978128899 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ef3b8501a0 0x55ef3b852590 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0769335ef9e95ee server_cookie=ea643aba8b0a2bf9 in_seq=0 out_seq=0 2024-04-24T08:53:34.469 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.465+0000 7f2f97e00640 10 client.16431 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:53:34.470 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.467+0000 7f2f97e00640 1 -- 172.21.15.83:0/3978128899 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ef3b7a0320 con 0x55ef3b8501a0 2024-04-24T08:53:34.470 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.467+0000 7f2f97e00640 10 client.16431 handle_command_reply: tid=0 2024-04-24T08:53:34.470 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.467+0000 7f2fbc1c6740 1 client.16431 shutdown 2024-04-24T08:53:34.470 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.467+0000 7f2fbc1c6740 1 -- 172.21.15.83:0/3978128899 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef3b82c9f0 msgr2=0x55ef3b8347b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:34.470 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.467+0000 7f2fbc1c6740 1 --2- 172.21.15.83:0/3978128899 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef3b82c9f0 0x55ef3b8347b0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c000f80 tx=0x7f2f9c09c760 comp rx=0 tx=0).stop 2024-04-24T08:53:34.470 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.467+0000 7f2fbc1c6740 1 -- 172.21.15.83:0/3978128899 shutdown_connections 2024-04-24T08:53:34.470 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.467+0000 7f2fbc1c6740 1 --2- 172.21.15.83:0/3978128899 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ef3b8501a0 0x55ef3b852590 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:34.470 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.467+0000 7f2fbc1c6740 1 --2- 172.21.15.83:0/3978128899 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef3b834cf0 0x55ef3b84bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:34.470 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.468+0000 7f2fbc1c6740 1 --2- 172.21.15.83:0/3978128899 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef3b82c9f0 0x55ef3b8347b0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:34.470 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.468+0000 7f2fbc1c6740 1 --2- 172.21.15.83:0/3978128899 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef3b82c0d0 0x55ef3b834270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:34.470 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.468+0000 7f2fbc1c6740 1 -- 172.21.15.83:0/3978128899 >> 172.21.15.83:0/3978128899 conn(0x55ef3b8286c0 msgr2=0x55ef3b82e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:34.470 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.468+0000 7f2f97e00640 0 client.16431 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:53:34.471 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.468+0000 7f2fbc1c6740 1 -- 172.21.15.83:0/3978128899 shutdown_connections 2024-04-24T08:53:34.471 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.468+0000 7f2fbc1c6740 1 -- 172.21.15.83:0/3978128899 wait complete. 2024-04-24T08:53:34.471 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.468+0000 7f2fbc1c6740 20 client.16431 trim_cache size 0 max 16384 2024-04-24T08:53:34.484 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.484+0000 7f2fbc1c6740 1 Processor -- start 2024-04-24T08:53:34.484 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.484+0000 7f2fbc1c6740 1 -- start start 2024-04-24T08:53:34.486 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.484+0000 7f2fbc1c6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef3b82c0d0 0x55ef3b857e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:34.486 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.484+0000 7f2fbc1c6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef3b82c9f0 0x55ef3b858340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:34.486 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.485+0000 7f2fbc1c6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef3b834cf0 0x55ef3b84b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:34.486 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.485+0000 7f2fbc1c6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb41ad260 con 0x55ef3b82c0d0 2024-04-24T08:53:34.486 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.485+0000 7f2fbc1c6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb41aeda0 con 0x55ef3b82c9f0 2024-04-24T08:53:34.486 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.485+0000 7f2fbc1c6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb41ad930 con 0x55ef3b834cf0 2024-04-24T08:53:34.486 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.485+0000 7f2fb1e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef3b82c0d0 0x55ef3b857e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:34.486 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.485+0000 7f2fb1e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef3b82c0d0 0x55ef3b857e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38382/0 (socket says 172.21.15.83:38382) 2024-04-24T08:53:34.486 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.485+0000 7f2fb1e00640 1 -- 172.21.15.83:0/3132481585 learned_addr learned my addr 172.21.15.83:0/3132481585 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:53:34.486 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.485+0000 7f2fa9400640 1 --2- 172.21.15.83:0/3132481585 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef3b82c9f0 0x55ef3b858340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:34.487 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.485+0000 7f2fb1e00640 1 -- 172.21.15.83:0/3132481585 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef3b834cf0 msgr2=0x55ef3b84b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:34.487 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.485+0000 7f2fb1e00640 1 --2- 172.21.15.83:0/3132481585 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef3b834cf0 0x55ef3b84b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:34.487 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.485+0000 7f2fb1e00640 1 -- 172.21.15.83:0/3132481585 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef3b82c9f0 msgr2=0x55ef3b858340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:34.487 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.485+0000 7f2fb1e00640 1 --2- 172.21.15.83:0/3132481585 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef3b82c9f0 0x55ef3b858340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:34.487 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.485+0000 7f2fb1e00640 1 -- 172.21.15.83:0/3132481585 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef3b82c820 con 0x55ef3b82c0d0 2024-04-24T08:53:34.487 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.486+0000 7f2fb1e00640 1 --2- 172.21.15.83:0/3132481585 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef3b82c0d0 0x55ef3b857e00 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f2fa400bfd0 tx=0x7f2fa4013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4bc2f9b8ac6229f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:34.487 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.486+0000 7f2fa2a00640 1 -- 172.21.15.83:0/3132481585 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fa4004020 con 0x55ef3b82c0d0 2024-04-24T08:53:34.487 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.486+0000 7f2fa2a00640 1 -- 172.21.15.83:0/3132481585 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2fa401dcb0 con 0x55ef3b82c0d0 2024-04-24T08:53:34.487 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.486+0000 7f2fa2a00640 1 -- 172.21.15.83:0/3132481585 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fa40237a0 con 0x55ef3b82c0d0 2024-04-24T08:53:34.490 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.487+0000 7f2fbc1c6740 1 -- 172.21.15.83:0/3132481585 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef3b82c0d0 msgr2=0x55ef3b857e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:34.490 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.487+0000 7f2fbc1c6740 1 --2- 172.21.15.83:0/3132481585 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef3b82c0d0 0x55ef3b857e00 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f2fa400bfd0 tx=0x7f2fa4013000 comp rx=0 tx=0).stop 2024-04-24T08:53:34.490 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.487+0000 7f2fbc1c6740 1 -- 172.21.15.83:0/3132481585 shutdown_connections 2024-04-24T08:53:34.490 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.487+0000 7f2fbc1c6740 1 --2- 172.21.15.83:0/3132481585 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef3b834cf0 0x55ef3b84b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:34.491 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.487+0000 7f2fbc1c6740 1 --2- 172.21.15.83:0/3132481585 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef3b82c9f0 0x55ef3b858340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:34.491 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.487+0000 7f2fbc1c6740 1 --2- 172.21.15.83:0/3132481585 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef3b82c0d0 0x55ef3b857e00 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:34.491 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.487+0000 7f2fbc1c6740 1 -- 172.21.15.83:0/3132481585 >> 172.21.15.83:0/3132481585 conn(0x55ef3b8286c0 msgr2=0x55ef3b833a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:34.491 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.487+0000 7f2fbc1c6740 1 -- 172.21.15.83:0/3132481585 shutdown_connections 2024-04-24T08:53:34.491 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.487+0000 7f2fbc1c6740 1 -- 172.21.15.83:0/3132481585 wait complete. 2024-04-24T08:53:34.491 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.487+0000 7f2fbc1c6740 1 -- start start 2024-04-24T08:53:34.491 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.487+0000 7f2fbc1c6740 1 Processor -- start 2024-04-24T08:53:34.491 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.488+0000 7f2fbc1c6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef3b82c0d0 0x55ef3b843580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:34.491 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.488+0000 7f2fbc1c6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef3b82c9f0 0x55ef3b843ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:34.491 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.488+0000 7f2fbc1c6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef3b834cf0 0x55ef3b844000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:34.491 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.488+0000 7f2fbc1c6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb41ae6d0 con 0x55ef3b834cf0 2024-04-24T08:53:34.491 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.488+0000 7f2fbc1c6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb41ae000 con 0x55ef3b82c9f0 2024-04-24T08:53:34.491 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.488+0000 7f2fbc1c6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb41ab240 con 0x55ef3b82c0d0 2024-04-24T08:53:34.491 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.488+0000 7f2fbc1c6740 10 client.0 _ll_register_callbacks cb 0x7f2f9c098e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:53:34.491 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.488+0000 7f2fb2800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef3b834cf0 0x55ef3b844000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:34.491 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.488+0000 7f2fb2800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef3b834cf0 0x55ef3b844000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38386/0 (socket says 172.21.15.83:38386) 2024-04-24T08:53:34.492 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.488+0000 7f2fb2800640 1 -- 172.21.15.83:0/2082744609 learned_addr learned my addr 172.21.15.83:0/2082744609 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:53:34.492 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.489+0000 7f2fa9400640 1 --2- 172.21.15.83:0/2082744609 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef3b82c9f0 0x55ef3b843ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:34.492 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.489+0000 7f2fb1e00640 1 --2- 172.21.15.83:0/2082744609 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef3b82c0d0 0x55ef3b843580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:34.492 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.489+0000 7f2fb2800640 1 -- 172.21.15.83:0/2082744609 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef3b82c0d0 msgr2=0x55ef3b843580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:34.492 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.489+0000 7f2fb2800640 1 --2- 172.21.15.83:0/2082744609 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef3b82c0d0 0x55ef3b843580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:34.492 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.489+0000 7f2fb2800640 1 -- 172.21.15.83:0/2082744609 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef3b82c9f0 msgr2=0x55ef3b843ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:34.492 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.489+0000 7f2fb2800640 1 --2- 172.21.15.83:0/2082744609 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef3b82c9f0 0x55ef3b843ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:34.492 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.489+0000 7f2fb2800640 1 -- 172.21.15.83:0/2082744609 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef3b571620 con 0x55ef3b834cf0 2024-04-24T08:53:34.492 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.489+0000 7f2fb2800640 1 --2- 172.21.15.83:0/2082744609 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef3b834cf0 0x55ef3b844000 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f2fac009e60 tx=0x7f2fac02ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18f1e0147b959eaa server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:34.492 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.489+0000 7f2f97e00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:53:34.492 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.489+0000 7f2f97e00640 1 -- 172.21.15.83:0/2082744609 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fac041040 con 0x55ef3b834cf0 2024-04-24T08:53:34.492 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.489+0000 7f2f97e00640 1 -- 172.21.15.83:0/2082744609 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f2fac005040 con 0x55ef3b834cf0 2024-04-24T08:53:34.492 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.489+0000 7f2fbc1c6740 1 -- 172.21.15.83:0/2082744609 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2fa401dcb0 con 0x55ef3b834cf0 2024-04-24T08:53:34.492 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.490+0000 7f2f97e00640 1 -- 172.21.15.83:0/2082744609 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2fac057050 con 0x55ef3b834cf0 2024-04-24T08:53:34.492 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.490+0000 7f2fbc1c6740 10 client.16437 fetch_fsmap learned FSMap version 28 2024-04-24T08:53:34.493 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.490+0000 7f2fbc1c6740 1 -- 172.21.15.83:0/2082744609 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ef3b8b67a0 con 0x55ef3b834cf0 2024-04-24T08:53:34.493 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.490+0000 7f2f97e00640 1 -- 172.21.15.83:0/2082744609 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2fa401dcb0 con 0x55ef3b834cf0 2024-04-24T08:53:34.493 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.490+0000 7f2f97e00640 1 -- 172.21.15.83:0/2082744609 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f2fac0046c0 con 0x55ef3b834cf0 2024-04-24T08:53:34.493 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.490+0000 7f2fbc1c6740 10 client.16437 fetch_fsmap finished waiting for FSMap version 28 2024-04-24T08:53:34.493 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.490+0000 7f2fbc1c6740 10 client.16437 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T08:53:34.493 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.490+0000 7f2fbc1c6740 20 client.16437 populate_metadata read hostname 'smithi083' 2024-04-24T08:53:34.493 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.490+0000 7f2fbc1c6740 1 --2- 172.21.15.83:0/2082744609 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ef3b8b6ad0 0x55ef3b8b8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:34.493 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.490+0000 7f2fbc1c6740 4 client.16437 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T08:53:34.493 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.490+0000 7f2fbc1c6740 1 -- 172.21.15.83:0/2082744609 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ef3b79a190 con 0x55ef3b8b6ad0 2024-04-24T08:53:34.494 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.491+0000 7f2fb1e00640 1 --2- 172.21.15.83:0/2082744609 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ef3b8b6ad0 0x55ef3b8b8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:34.494 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.492+0000 7f2fb1e00640 1 --2- 172.21.15.83:0/2082744609 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ef3b8b6ad0 0x55ef3b8b8ec0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=887deba7e401bd0 server_cookie=76ca509dddbde9c4 in_seq=0 out_seq=0 2024-04-24T08:53:34.494 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.492+0000 7f2f97e00640 10 client.16437 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:53:34.554 INFO:tasks.workunit.client.0.smithi083.stdout: 1 1453 3.25 MB/sec warmup 8 sec latency 59.445 ms 2024-04-24T08:53:34.589 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.588+0000 7f2f97e00640 1 -- 172.21.15.83:0/2082744609 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ef3b79a190 con 0x55ef3b8b6ad0 2024-04-24T08:53:34.589 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.588+0000 7f2f97e00640 10 client.16437 handle_command_reply: tid=0 2024-04-24T08:53:34.589 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.588+0000 7f2fbc1c6740 1 client.16437 shutdown 2024-04-24T08:53:34.589 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.588+0000 7f2fbc1c6740 1 -- 172.21.15.83:0/2082744609 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef3b834cf0 msgr2=0x55ef3b844000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:34.589 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.588+0000 7f2fbc1c6740 1 --2- 172.21.15.83:0/2082744609 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef3b834cf0 0x55ef3b844000 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f2fac009e60 tx=0x7f2fac02ba10 comp rx=0 tx=0).stop 2024-04-24T08:53:34.589 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.588+0000 7f2fbc1c6740 1 -- 172.21.15.83:0/2082744609 shutdown_connections 2024-04-24T08:53:34.589 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.588+0000 7f2fbc1c6740 1 --2- 172.21.15.83:0/2082744609 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ef3b8b6ad0 0x55ef3b8b8ec0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:34.589 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.588+0000 7f2fbc1c6740 1 --2- 172.21.15.83:0/2082744609 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef3b834cf0 0x55ef3b844000 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:34.589 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.588+0000 7f2fbc1c6740 1 --2- 172.21.15.83:0/2082744609 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef3b82c9f0 0x55ef3b843ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:34.589 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.588+0000 7f2fbc1c6740 1 --2- 172.21.15.83:0/2082744609 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef3b82c0d0 0x55ef3b843580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:34.589 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.588+0000 7f2fbc1c6740 1 -- 172.21.15.83:0/2082744609 >> 172.21.15.83:0/2082744609 conn(0x55ef3b8286c0 msgr2=0x55ef3b835aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:34.589 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.588+0000 7f2f97e00640 0 client.16437 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:53:34.590 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.588+0000 7f2fbc1c6740 1 -- 172.21.15.83:0/2082744609 shutdown_connections 2024-04-24T08:53:34.590 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.588+0000 7f2fbc1c6740 1 -- 172.21.15.83:0/2082744609 wait complete. 2024-04-24T08:53:34.590 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.588+0000 7f2fbc1c6740 20 client.16437 trim_cache size 0 max 16384 2024-04-24T08:53:34.590 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T08:53:34.590 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T08:53:34.590 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T08:53:34.590 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T08:53:34.591 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.589+0000 7f2fb9200640 1 -- 172.21.15.83:0/3909960907 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f2f980a3e50 msgr2=0x7f2f980a6310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:34.591 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.589+0000 7f2fb9200640 1 --2- 172.21.15.83:0/3909960907 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f2f980a3e50 0x7f2f980a6310 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x55ef3b82ecd0 tx=0x7f2fa4005d90 comp rx=0 tx=0).stop 2024-04-24T08:53:34.591 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.589+0000 7f2fb9200640 1 -- 172.21.15.83:0/3909960907 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fb4105560 msgr2=0x7f2fb4198c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:34.591 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.589+0000 7f2fb9200640 1 --2- 172.21.15.83:0/3909960907 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fb4105560 0x7f2fb4198c70 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c00ccb0 tx=0x7f2f9c00a620 comp rx=0 tx=0).stop 2024-04-24T08:53:34.591 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.589+0000 7f2fb9200640 1 -- 172.21.15.83:0/3909960907 shutdown_connections 2024-04-24T08:53:34.591 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.589+0000 7f2fb9200640 1 --2- 172.21.15.83:0/3909960907 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f2f980a3e50 0x7f2f980a6310 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:34.591 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.589+0000 7f2fb9200640 1 --2- 172.21.15.83:0/3909960907 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2fb419cff0 0x7f2fb419d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:34.591 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.589+0000 7f2fb9200640 1 --2- 172.21.15.83:0/3909960907 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fb4105560 0x7f2fb4198c70 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:34.591 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.589+0000 7f2fb9200640 1 --2- 172.21.15.83:0/3909960907 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2fb4100490 0x7f2fb4198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:34.592 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.589+0000 7f2fb9200640 1 -- 172.21.15.83:0/3909960907 >> 172.21.15.83:0/3909960907 conn(0x7f2fb4077000 msgr2=0x7f2fb40fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:34.592 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.589+0000 7f2fb9200640 1 -- 172.21.15.83:0/3909960907 shutdown_connections 2024-04-24T08:53:34.592 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:34.589+0000 7f2fb9200640 1 -- 172.21.15.83:0/3909960907 wait complete. 2024-04-24T08:53:35.554 INFO:tasks.workunit.client.0.smithi083.stdout: 1 1603 2.91 MB/sec warmup 9 sec latency 53.383 ms 2024-04-24T08:53:35.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:53:35 smithi138 ceph-mon[31441]: pgmap v62: 129 pgs: 129 active+clean; 22 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 1.8 MiB/s wr, 28 op/s 2024-04-24T08:53:35.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:53:35 smithi189 ceph-mon[31791]: pgmap v62: 129 pgs: 129 active+clean; 22 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 1.8 MiB/s wr, 28 op/s 2024-04-24T08:53:35.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:53:35 smithi083 ceph-mon[27907]: pgmap v62: 129 pgs: 129 active+clean; 22 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 1.8 MiB/s wr, 28 op/s 2024-04-24T08:53:36.555 INFO:tasks.workunit.client.0.smithi083.stdout: 1 1758 2.63 MB/sec warmup 10 sec latency 53.851 ms 2024-04-24T08:53:37.555 INFO:tasks.workunit.client.0.smithi083.stdout: 1 1869 2.40 MB/sec warmup 11 sec latency 59.180 ms 2024-04-24T08:53:37.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:53:37 smithi083 ceph-mon[27907]: pgmap v63: 129 pgs: 129 active+clean; 22 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 1.8 MiB/s wr, 28 op/s 2024-04-24T08:53:37.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:53:37 smithi138 ceph-mon[31441]: pgmap v63: 129 pgs: 129 active+clean; 22 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 1.8 MiB/s wr, 28 op/s 2024-04-24T08:53:37.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:53:37 smithi189 ceph-mon[31791]: pgmap v63: 129 pgs: 129 active+clean; 22 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 1.8 MiB/s wr, 28 op/s 2024-04-24T08:53:38.555 INFO:tasks.workunit.client.0.smithi083.stdout: 1 2034 2.21 MB/sec warmup 12 sec latency 62.068 ms 2024-04-24T08:53:38.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:53:38 smithi083 ceph-mon[27907]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T08:53:38.857 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:53:38 smithi138 ceph-mon[31441]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T08:53:38.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:53:38 smithi189 ceph-mon[31791]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T08:53:39.557 INFO:tasks.workunit.client.0.smithi083.stdout: 1 2173 2.06 MB/sec warmup 13 sec latency 54.808 ms 2024-04-24T08:53:39.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:53:39 smithi083 ceph-mon[27907]: pgmap v64: 129 pgs: 129 active+clean; 22 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 1.9 MiB/s wr, 70 op/s 2024-04-24T08:53:39.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:53:39 smithi138 ceph-mon[31441]: pgmap v64: 129 pgs: 129 active+clean; 22 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 1.9 MiB/s wr, 70 op/s 2024-04-24T08:53:39.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:53:39 smithi189 ceph-mon[31791]: pgmap v64: 129 pgs: 129 active+clean; 22 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 1.9 MiB/s wr, 70 op/s 2024-04-24T08:53:40.555 INFO:tasks.workunit.client.0.smithi083.stdout: 1 2333 1.99 MB/sec warmup 14 sec latency 48.552 ms 2024-04-24T08:53:41.555 INFO:tasks.workunit.client.0.smithi083.stdout: 1 2428 1.87 MB/sec warmup 15 sec latency 58.195 ms 2024-04-24T08:53:41.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:53:41 smithi083 ceph-mon[27907]: pgmap v65: 129 pgs: 129 active+clean; 22 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 1.2 MiB/s wr, 63 op/s 2024-04-24T08:53:41.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:53:41 smithi138 ceph-mon[31441]: pgmap v65: 129 pgs: 129 active+clean; 22 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 1.2 MiB/s wr, 63 op/s 2024-04-24T08:53:41.956 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:53:41 smithi189 ceph-mon[31791]: pgmap v65: 129 pgs: 129 active+clean; 22 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 1.2 MiB/s wr, 63 op/s 2024-04-24T08:53:42.556 INFO:tasks.workunit.client.0.smithi083.stdout: 1 2560 1.81 MB/sec warmup 16 sec latency 56.293 ms 2024-04-24T08:53:43.556 INFO:tasks.workunit.client.0.smithi083.stdout: 1 2718 1.88 MB/sec warmup 17 sec latency 65.604 ms 2024-04-24T08:53:43.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:53:43 smithi083 ceph-mon[27907]: pgmap v66: 129 pgs: 129 active+clean; 24 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 1.4 MiB/s wr, 79 op/s 2024-04-24T08:53:43.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:53:43 smithi083 ceph-mon[27907]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T08:53:43.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:53:43 smithi138 ceph-mon[31441]: pgmap v66: 129 pgs: 129 active+clean; 24 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 1.4 MiB/s wr, 79 op/s 2024-04-24T08:53:43.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:53:43 smithi138 ceph-mon[31441]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T08:53:43.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:53:43 smithi189 ceph-mon[31791]: pgmap v66: 129 pgs: 129 active+clean; 24 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 1.4 MiB/s wr, 79 op/s 2024-04-24T08:53:43.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:53:43 smithi189 ceph-mon[31791]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T08:53:44.556 INFO:tasks.workunit.client.0.smithi083.stdout: 1 2926 1.86 MB/sec warmup 18 sec latency 49.227 ms 2024-04-24T08:53:45.556 INFO:tasks.workunit.client.0.smithi083.stdout: 1 3025 1.77 MB/sec warmup 19 sec latency 50.236 ms 2024-04-24T08:53:45.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:53:45 smithi083 ceph-mon[27907]: pgmap v67: 129 pgs: 129 active+clean; 24 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 242 KiB/s wr, 58 op/s 2024-04-24T08:53:45.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:53:45 smithi138 ceph-mon[31441]: pgmap v67: 129 pgs: 129 active+clean; 24 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 242 KiB/s wr, 58 op/s 2024-04-24T08:53:45.956 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:53:45 smithi189 ceph-mon[31791]: pgmap v67: 129 pgs: 129 active+clean; 24 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 242 KiB/s wr, 58 op/s 2024-04-24T08:53:46.556 INFO:tasks.workunit.client.0.smithi083.stdout: 1 3285 1.77 MB/sec warmup 20 sec latency 49.381 ms 2024-04-24T08:53:47.556 INFO:tasks.workunit.client.0.smithi083.stdout: 1 3313 1.68 MB/sec warmup 21 sec latency 58.885 ms 2024-04-24T08:53:47.737 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.737+0000 7f81b3200640 1 -- 172.21.15.83:0/2485152646 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81b4103940 msgr2=0x7f81b410be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:47.737 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.737+0000 7f81b3200640 1 --2- 172.21.15.83:0/2485152646 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81b4103940 0x7f81b410be20 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f819c009900 tx=0x7f819c02a480 comp rx=0 tx=0).stop 2024-04-24T08:53:47.737 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.737+0000 7f81b3200640 1 -- 172.21.15.83:0/2485152646 shutdown_connections 2024-04-24T08:53:47.738 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.737+0000 7f81b3200640 1 --2- 172.21.15.83:0/2485152646 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81b410c3f0 0x7f81b410e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:47.738 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.737+0000 7f81b3200640 1 --2- 172.21.15.83:0/2485152646 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81b4103940 0x7f81b410be20 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:47.738 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.737+0000 7f81b3200640 1 --2- 172.21.15.83:0/2485152646 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f81b4103020 0x7f81b4103400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:47.738 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.737+0000 7f81b3200640 1 -- 172.21.15.83:0/2485152646 >> 172.21.15.83:0/2485152646 conn(0x7f81b40fc820 msgr2=0x7f81b40fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:47.738 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.738+0000 7f81b3200640 1 -- 172.21.15.83:0/2485152646 shutdown_connections 2024-04-24T08:53:47.738 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.738+0000 7f81b3200640 1 -- 172.21.15.83:0/2485152646 wait complete. 2024-04-24T08:53:47.740 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.739+0000 7f81b3200640 1 Processor -- start 2024-04-24T08:53:47.741 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.739+0000 7f81b3200640 1 -- start start 2024-04-24T08:53:47.741 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.739+0000 7f81b3200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81b4103020 0x7f81b41a05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:47.741 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.739+0000 7f81b3200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81b4103940 0x7f81b41a0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:47.741 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.739+0000 7f81b3200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f81b410c3f0 0x7f81b419a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:47.741 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.739+0000 7f81b3200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f81b41120c0 con 0x7f81b4103940 2024-04-24T08:53:47.741 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.739+0000 7f81b3200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f81b4111f40 con 0x7f81b4103020 2024-04-24T08:53:47.741 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.740+0000 7f81b3200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f81b4112240 con 0x7f81b410c3f0 2024-04-24T08:53:47.741 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.740+0000 7f81abe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81b4103940 0x7f81b41a0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:47.741 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.740+0000 7f81abe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81b4103940 0x7f81b41a0b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52158/0 (socket says 172.21.15.83:52158) 2024-04-24T08:53:47.741 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.740+0000 7f81abe00640 1 -- 172.21.15.83:0/499690966 learned_addr learned my addr 172.21.15.83:0/499690966 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:53:47.741 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.740+0000 7f81b1e00640 1 --2- 172.21.15.83:0/499690966 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81b4103020 0x7f81b41a05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:47.741 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.740+0000 7f81abe00640 1 -- 172.21.15.83:0/499690966 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f81b410c3f0 msgr2=0x7f81b419a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:47.742 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.740+0000 7f81b2800640 1 --2- 172.21.15.83:0/499690966 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f81b410c3f0 0x7f81b419a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:47.742 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.740+0000 7f81abe00640 1 --2- 172.21.15.83:0/499690966 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f81b410c3f0 0x7f81b419a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:47.742 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.741+0000 7f81abe00640 1 -- 172.21.15.83:0/499690966 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81b4103020 msgr2=0x7f81b41a05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:47.742 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.741+0000 7f81abe00640 1 --2- 172.21.15.83:0/499690966 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81b4103020 0x7f81b41a05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:47.742 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.741+0000 7f81abe00640 1 -- 172.21.15.83:0/499690966 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81b419af50 con 0x7f81b4103940 2024-04-24T08:53:47.742 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.741+0000 7f81b2800640 1 --2- 172.21.15.83:0/499690966 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f81b410c3f0 0x7f81b419a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:53:47.742 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.741+0000 7f81b1e00640 1 --2- 172.21.15.83:0/499690966 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81b4103020 0x7f81b41a05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T08:53:47.742 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.741+0000 7f81abe00640 1 --2- 172.21.15.83:0/499690966 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81b4103940 0x7f81b41a0b10 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f819c005ab0 tx=0x7f819c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:47.742 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.741+0000 7f81aaa00640 1 -- 172.21.15.83:0/499690966 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f819c03f070 con 0x7f81b4103940 2024-04-24T08:53:47.742 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.741+0000 7f81b3200640 1 -- 172.21.15.83:0/499690966 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81b419b1e0 con 0x7f81b4103940 2024-04-24T08:53:47.742 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.741+0000 7f81aaa00640 1 -- 172.21.15.83:0/499690966 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f819c031b60 con 0x7f81b4103940 2024-04-24T08:53:47.742 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.741+0000 7f81aaa00640 1 -- 172.21.15.83:0/499690966 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f819c035030 con 0x7f81b4103940 2024-04-24T08:53:47.742 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.742+0000 7f81b3200640 1 -- 172.21.15.83:0/499690966 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81b41a7370 con 0x7f81b4103940 2024-04-24T08:53:47.744 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.745+0000 7f81bbe24740 1 Processor -- start 2024-04-24T08:53:47.744 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.745+0000 7f81aaa00640 1 -- 172.21.15.83:0/499690966 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f819c042070 con 0x7f81b4103940 2024-04-24T08:53:47.745 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.745+0000 7f81bbe24740 1 -- start start 2024-04-24T08:53:47.745 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.746+0000 7f81bbe24740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f531d93c0 0x559f531d97a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:47.745 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.746+0000 7f81bbe24740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559f531d9ce0 0x559f531e4060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:47.746 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.746+0000 7f81bbe24740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f531e45a0 0x559f531e6990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:47.746 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.746+0000 7f81bbe24740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559f5313f900 con 0x559f531d93c0 2024-04-24T08:53:47.746 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.746+0000 7f81bbe24740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559f530e5f20 con 0x559f531e45a0 2024-04-24T08:53:47.746 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.746+0000 7f81bbe24740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559f5313b8c0 con 0x559f531d9ce0 2024-04-24T08:53:47.746 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.746+0000 7f81abe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f531e45a0 0x559f531e6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:47.746 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.746+0000 7f81b1e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559f531d9ce0 0x559f531e4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:47.746 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.746+0000 7f81abe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f531e45a0 0x559f531e6990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:48968/0 (socket says 172.21.15.83:48968) 2024-04-24T08:53:47.746 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.746+0000 7f81b2800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f531d93c0 0x559f531d97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:47.746 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.746+0000 7f81abe00640 1 -- 172.21.15.83:0/2252032013 learned_addr learned my addr 172.21.15.83:0/2252032013 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:53:47.748 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.747+0000 7f81b1e00640 1 -- 172.21.15.83:0/2252032013 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f531e45a0 msgr2=0x559f531e6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:47.748 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.747+0000 7f81b1e00640 1 --2- 172.21.15.83:0/2252032013 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f531e45a0 0x559f531e6990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:47.748 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.747+0000 7f81b1e00640 1 -- 172.21.15.83:0/2252032013 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f531d93c0 msgr2=0x559f531d97a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:47.748 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.747+0000 7f81b1e00640 1 --2- 172.21.15.83:0/2252032013 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f531d93c0 0x559f531d97a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:47.748 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.747+0000 7f81b1e00640 1 -- 172.21.15.83:0/2252032013 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f5310ae10 con 0x559f531d9ce0 2024-04-24T08:53:47.748 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.747+0000 7f81b2800640 1 --2- 172.21.15.83:0/2252032013 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f531d93c0 0x559f531d97a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T08:53:47.748 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.747+0000 7f81aaa00640 1 --2- 172.21.15.83:0/499690966 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f81980a3d10 0x7f81980a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:47.749 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.747+0000 7f81aaa00640 1 -- 172.21.15.83:0/499690966 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f819c0f0fe0 con 0x7f81b4103940 2024-04-24T08:53:47.749 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.747+0000 7f81b1e00640 1 --2- 172.21.15.83:0/2252032013 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559f531d9ce0 0x559f531e4060 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f81a4009fd0 tx=0x7f81a40118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4757cbe18cb6a376 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:47.749 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.747+0000 7f81b2800640 1 --2- 172.21.15.83:0/499690966 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f81980a3d10 0x7f81980a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:47.749 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.748+0000 7f81a3400640 1 -- 172.21.15.83:0/2252032013 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a4004310 con 0x559f531d9ce0 2024-04-24T08:53:47.749 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.748+0000 7f81a3400640 1 -- 172.21.15.83:0/2252032013 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f81a40044b0 con 0x559f531d9ce0 2024-04-24T08:53:47.749 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.748+0000 7f81a3400640 1 -- 172.21.15.83:0/2252032013 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a40035f0 con 0x559f531d9ce0 2024-04-24T08:53:47.749 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.749+0000 7f81bbe24740 1 -- 172.21.15.83:0/2252032013 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559f531d9ce0 msgr2=0x559f531e4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:47.749 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.749+0000 7f81bbe24740 1 --2- 172.21.15.83:0/2252032013 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559f531d9ce0 0x559f531e4060 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f81a4009fd0 tx=0x7f81a40118d0 comp rx=0 tx=0).stop 2024-04-24T08:53:47.749 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.749+0000 7f81b2800640 1 --2- 172.21.15.83:0/499690966 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f81980a3d10 0x7f81980a61d0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f81ac005fd0 tx=0x7f81ac009500 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:47.749 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.749+0000 7f81bbe24740 1 -- 172.21.15.83:0/2252032013 shutdown_connections 2024-04-24T08:53:47.749 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.749+0000 7f81bbe24740 1 --2- 172.21.15.83:0/2252032013 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f531e45a0 0x559f531e6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:47.749 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.749+0000 7f81bbe24740 1 --2- 172.21.15.83:0/2252032013 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559f531d9ce0 0x559f531e4060 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:47.749 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.749+0000 7f81bbe24740 1 --2- 172.21.15.83:0/2252032013 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f531d93c0 0x559f531d97a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:47.749 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.749+0000 7f81bbe24740 1 -- 172.21.15.83:0/2252032013 >> 172.21.15.83:0/2252032013 conn(0x559f531d58d0 msgr2=0x559f531d5cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:47.749 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.749+0000 7f81bbe24740 1 -- 172.21.15.83:0/2252032013 shutdown_connections 2024-04-24T08:53:47.750 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.749+0000 7f81bbe24740 1 -- 172.21.15.83:0/2252032013 wait complete. 2024-04-24T08:53:47.753 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.750+0000 7f81bbe24740 1 -- start start 2024-04-24T08:53:47.753 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.750+0000 7f81bbe24740 1 Processor -- start 2024-04-24T08:53:47.753 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.750+0000 7f81bbe24740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f531d93c0 0x559f531e15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:47.753 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.750+0000 7f81bbe24740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559f531d9ce0 0x559f531e1b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:47.753 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.750+0000 7f81bbe24740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f531e45a0 0x559f531e2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:47.753 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.750+0000 7f81bbe24740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559f531e7140 con 0x559f531e45a0 2024-04-24T08:53:47.753 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.750+0000 7f81bbe24740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559f530f92c0 con 0x559f531d93c0 2024-04-24T08:53:47.753 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.750+0000 7f81bbe24740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559f531e72c0 con 0x559f531d9ce0 2024-04-24T08:53:47.754 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.751+0000 7f81bbe24740 10 client.0 _ll_register_callbacks cb 0x559f52e51a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:53:47.754 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.751+0000 7f81b1e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f531d93c0 0x559f531e15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:47.754 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.751+0000 7f81b1e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f531d93c0 0x559f531e15c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:48970/0 (socket says 172.21.15.83:48970) 2024-04-24T08:53:47.754 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.751+0000 7f81b1e00640 1 -- 172.21.15.83:0/2668811783 learned_addr learned my addr 172.21.15.83:0/2668811783 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:53:47.754 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.751+0000 7f81b2800640 1 --2- 172.21.15.83:0/2668811783 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f531e45a0 0x559f531e2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:47.754 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.751+0000 7f81abe00640 1 --2- 172.21.15.83:0/2668811783 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559f531d9ce0 0x559f531e1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:47.754 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.751+0000 7f81b1e00640 1 -- 172.21.15.83:0/2668811783 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559f531d9ce0 msgr2=0x559f531e1b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:47.754 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.751+0000 7f81b1e00640 1 --2- 172.21.15.83:0/2668811783 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559f531d9ce0 0x559f531e1b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:47.754 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.751+0000 7f81b1e00640 1 -- 172.21.15.83:0/2668811783 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f531e45a0 msgr2=0x559f531e2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:47.754 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.751+0000 7f81b1e00640 1 --2- 172.21.15.83:0/2668811783 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f531e45a0 0x559f531e2040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:47.754 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.751+0000 7f81b1e00640 1 -- 172.21.15.83:0/2668811783 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f53109fd0 con 0x559f531d93c0 2024-04-24T08:53:47.754 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.752+0000 7f81abe00640 1 --2- 172.21.15.83:0/2668811783 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559f531d9ce0 0x559f531e1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:53:47.754 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.752+0000 7f81b1e00640 1 --2- 172.21.15.83:0/2668811783 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f531d93c0 0x559f531e15c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f81a4005890 tx=0x7f81a4004970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f584ebd6fd1eff20 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:47.755 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.752+0000 7f8197e00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T08:53:47.755 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.752+0000 7f8197e00640 1 -- 172.21.15.83:0/2668811783 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a4029070 con 0x559f531d93c0 2024-04-24T08:53:47.755 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.752+0000 7f81bbe24740 1 -- 172.21.15.83:0/2668811783 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81a40044b0 con 0x559f531d93c0 2024-04-24T08:53:47.755 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.752+0000 7f8197e00640 1 -- 172.21.15.83:0/2668811783 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f81a4003a80 con 0x559f531d93c0 2024-04-24T08:53:47.755 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.753+0000 7f8197e00640 1 -- 172.21.15.83:0/2668811783 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f81a4035050 con 0x559f531d93c0 2024-04-24T08:53:47.755 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.753+0000 7f81bbe24740 10 client.35489 fetch_fsmap learned FSMap version 30 2024-04-24T08:53:47.755 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.753+0000 7f81bbe24740 1 -- 172.21.15.83:0/2668811783 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559f52e52210 con 0x559f531d93c0 2024-04-24T08:53:47.755 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.753+0000 7f8197e00640 1 -- 172.21.15.83:0/2668811783 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f81a40044b0 con 0x559f531d93c0 2024-04-24T08:53:47.755 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.753+0000 7f8197e00640 1 -- 172.21.15.83:0/2668811783 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7f81a4023430 con 0x559f531d93c0 2024-04-24T08:53:47.755 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.753+0000 7f81bbe24740 10 client.35489 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T08:53:47.755 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.753+0000 7f81bbe24740 10 client.35489 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T08:53:47.755 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.753+0000 7f81bbe24740 20 client.35489 populate_metadata read hostname 'smithi083' 2024-04-24T08:53:47.755 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.753+0000 7f81bbe24740 1 --2- 172.21.15.83:0/2668811783 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x559f531f4670 0x559f531f6a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:47.755 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.753+0000 7f81bbe24740 4 client.35489 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:53:47.755 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.753+0000 7f81bbe24740 1 -- 172.21.15.83:0/2668811783 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559f5314d320 con 0x559f531f4670 2024-04-24T08:53:47.756 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.754+0000 7f81abe00640 1 --2- 172.21.15.83:0/2668811783 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x559f531f4670 0x559f531f6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:47.757 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.755+0000 7f81abe00640 1 --2- 172.21.15.83:0/2668811783 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x559f531f4670 0x559f531f6a60 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3303ed31c183de15 server_cookie=cc27e0276b3e63d2 in_seq=0 out_seq=0 2024-04-24T08:53:47.757 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.755+0000 7f8197e00640 10 client.35489 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:53:47.757 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.756+0000 7f8197e00640 1 -- 172.21.15.83:0/2668811783 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559f5314d320 con 0x559f531f4670 2024-04-24T08:53:47.757 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.756+0000 7f8197e00640 10 client.35489 handle_command_reply: tid=0 2024-04-24T08:53:47.757 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.756+0000 7f81bbe24740 1 client.35489 shutdown 2024-04-24T08:53:47.757 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.756+0000 7f81bbe24740 1 -- 172.21.15.83:0/2668811783 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f531d93c0 msgr2=0x559f531e15c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:47.757 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.756+0000 7f81bbe24740 1 --2- 172.21.15.83:0/2668811783 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f531d93c0 0x559f531e15c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f81a4005890 tx=0x7f81a4004970 comp rx=0 tx=0).stop 2024-04-24T08:53:47.757 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.756+0000 7f81bbe24740 1 -- 172.21.15.83:0/2668811783 shutdown_connections 2024-04-24T08:53:47.757 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.756+0000 7f81bbe24740 1 --2- 172.21.15.83:0/2668811783 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x559f531f4670 0x559f531f6a60 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:47.757 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.756+0000 7f81bbe24740 1 --2- 172.21.15.83:0/2668811783 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f531e45a0 0x559f531e2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:47.757 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.756+0000 7f81bbe24740 1 --2- 172.21.15.83:0/2668811783 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559f531d9ce0 0x559f531e1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:47.757 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.756+0000 7f81bbe24740 1 --2- 172.21.15.83:0/2668811783 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f531d93c0 0x559f531e15c0 secure :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f81a4005890 tx=0x7f81a4004970 comp rx=0 tx=0).stop 2024-04-24T08:53:47.757 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.756+0000 7f81bbe24740 1 -- 172.21.15.83:0/2668811783 >> 172.21.15.83:0/2668811783 conn(0x559f531d58d0 msgr2=0x559f531db650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:47.758 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.756+0000 7f8197e00640 0 client.35489 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:53:47.758 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.757+0000 7f81bbe24740 1 -- 172.21.15.83:0/2668811783 shutdown_connections 2024-04-24T08:53:47.758 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.757+0000 7f81bbe24740 1 -- 172.21.15.83:0/2668811783 wait complete. 2024-04-24T08:53:47.758 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.757+0000 7f81bbe24740 20 client.35489 trim_cache size 0 max 16384 2024-04-24T08:53:47.774 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.774+0000 7f81bbe24740 1 Processor -- start 2024-04-24T08:53:47.774 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.774+0000 7f81bbe24740 1 -- start start 2024-04-24T08:53:47.774 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.774+0000 7f81bbe24740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f531d9ce0 0x559f531e3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:47.774 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.774+0000 7f81bbe24740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559f531e45a0 0x559f531f00f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:47.774 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.774+0000 7f81bbe24740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f531e0e80 0x559f531fc2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:47.774 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.774+0000 7f81bbe24740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f81b41b15b0 con 0x559f531e0e80 2024-04-24T08:53:47.775 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.774+0000 7f81bbe24740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f81b41b3180 con 0x559f531d9ce0 2024-04-24T08:53:47.775 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.774+0000 7f81bbe24740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f81b41b1c80 con 0x559f531e45a0 2024-04-24T08:53:47.775 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.775+0000 7f81b2800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f531e0e80 0x559f531fc2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:47.775 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.775+0000 7f81b1e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f531d9ce0 0x559f531e3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:47.775 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.775+0000 7f81b2800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f531e0e80 0x559f531fc2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52186/0 (socket says 172.21.15.83:52186) 2024-04-24T08:53:47.775 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.775+0000 7f81b2800640 1 -- 172.21.15.83:0/1798595446 learned_addr learned my addr 172.21.15.83:0/1798595446 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:53:47.777 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.775+0000 7f81abe00640 1 --2- 172.21.15.83:0/1798595446 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559f531e45a0 0x559f531f00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:47.777 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.775+0000 7f81b1e00640 1 -- 172.21.15.83:0/1798595446 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559f531e45a0 msgr2=0x559f531f00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:47.777 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.775+0000 7f81b1e00640 1 --2- 172.21.15.83:0/1798595446 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559f531e45a0 0x559f531f00f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:47.777 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.775+0000 7f81b1e00640 1 -- 172.21.15.83:0/1798595446 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f531e0e80 msgr2=0x559f531fc2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:47.777 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.775+0000 7f81b1e00640 1 --2- 172.21.15.83:0/1798595446 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f531e0e80 0x559f531fc2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:47.777 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.775+0000 7f81b1e00640 1 -- 172.21.15.83:0/1798595446 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f52f1e620 con 0x559f531d9ce0 2024-04-24T08:53:47.777 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.775+0000 7f81b2800640 1 --2- 172.21.15.83:0/1798595446 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f531e0e80 0x559f531fc2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T08:53:47.777 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.776+0000 7f81b1e00640 1 --2- 172.21.15.83:0/1798595446 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f531d9ce0 0x559f531e3cf0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f81a40058c0 tx=0x7f81a4005b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db9f42b6fb305faa server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:47.778 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.776+0000 7f81a2a00640 1 -- 172.21.15.83:0/1798595446 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a4029070 con 0x559f531d9ce0 2024-04-24T08:53:47.778 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.776+0000 7f81a2a00640 1 -- 172.21.15.83:0/1798595446 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f81a4023e40 con 0x559f531d9ce0 2024-04-24T08:53:47.778 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.776+0000 7f81a2a00640 1 -- 172.21.15.83:0/1798595446 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a400f4e0 con 0x559f531d9ce0 2024-04-24T08:53:47.778 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.777+0000 7f81bbe24740 1 -- 172.21.15.83:0/1798595446 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f531d9ce0 msgr2=0x559f531e3cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:47.778 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.777+0000 7f81bbe24740 1 --2- 172.21.15.83:0/1798595446 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f531d9ce0 0x559f531e3cf0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f81a40058c0 tx=0x7f81a4005b90 comp rx=0 tx=0).stop 2024-04-24T08:53:47.778 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.777+0000 7f81bbe24740 1 -- 172.21.15.83:0/1798595446 shutdown_connections 2024-04-24T08:53:47.778 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.777+0000 7f81bbe24740 1 --2- 172.21.15.83:0/1798595446 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f531e0e80 0x559f531fc2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:47.778 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.777+0000 7f81bbe24740 1 --2- 172.21.15.83:0/1798595446 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559f531e45a0 0x559f531f00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:47.778 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.777+0000 7f81bbe24740 1 --2- 172.21.15.83:0/1798595446 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f531d9ce0 0x559f531e3cf0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:47.778 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.777+0000 7f81bbe24740 1 -- 172.21.15.83:0/1798595446 >> 172.21.15.83:0/1798595446 conn(0x559f531d58d0 msgr2=0x559f531d5cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:47.778 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.777+0000 7f81bbe24740 1 -- 172.21.15.83:0/1798595446 shutdown_connections 2024-04-24T08:53:47.778 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.777+0000 7f81bbe24740 1 -- 172.21.15.83:0/1798595446 wait complete. 2024-04-24T08:53:47.779 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.778+0000 7f81bbe24740 1 -- start start 2024-04-24T08:53:47.779 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.778+0000 7f81bbe24740 1 Processor -- start 2024-04-24T08:53:47.781 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.778+0000 7f81bbe24740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f531d9ce0 0x559f531ec920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:47.781 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.778+0000 7f81bbe24740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f531e0e80 0x559f531e7c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:47.782 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.778+0000 7f81bbe24740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559f531e45a0 0x559f531e8180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:47.782 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.778+0000 7f81bbe24740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f81b41b2a20 con 0x559f531e0e80 2024-04-24T08:53:47.782 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.778+0000 7f81bbe24740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f81b41b2350 con 0x559f531d9ce0 2024-04-24T08:53:47.782 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.778+0000 7f81bbe24740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f81b41af590 con 0x559f531e45a0 2024-04-24T08:53:47.782 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.778+0000 7f81bbe24740 10 client.0 _ll_register_callbacks cb 0x7f81ac0098e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:53:47.782 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.779+0000 7f81abe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f531e0e80 0x559f531e7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:47.782 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.779+0000 7f81abe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f531e0e80 0x559f531e7c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52200/0 (socket says 172.21.15.83:52200) 2024-04-24T08:53:47.782 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.779+0000 7f81abe00640 1 -- 172.21.15.83:0/554389503 learned_addr learned my addr 172.21.15.83:0/554389503 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:53:47.782 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.779+0000 7f81b1e00640 1 --2- 172.21.15.83:0/554389503 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f531d9ce0 0x559f531ec920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:47.782 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.779+0000 7f81b2800640 1 --2- 172.21.15.83:0/554389503 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559f531e45a0 0x559f531e8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:47.782 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.779+0000 7f81b1e00640 1 -- 172.21.15.83:0/554389503 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559f531e45a0 msgr2=0x559f531e8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:47.782 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.779+0000 7f81b1e00640 1 --2- 172.21.15.83:0/554389503 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559f531e45a0 0x559f531e8180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:47.783 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.779+0000 7f81b1e00640 1 -- 172.21.15.83:0/554389503 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f531e0e80 msgr2=0x559f531e7c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:47.783 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.779+0000 7f81b1e00640 1 --2- 172.21.15.83:0/554389503 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f531e0e80 0x559f531e7c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:47.783 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.779+0000 7f81b1e00640 1 -- 172.21.15.83:0/554389503 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f531e88c0 con 0x559f531d9ce0 2024-04-24T08:53:47.783 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.780+0000 7f81b1e00640 1 --2- 172.21.15.83:0/554389503 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f531d9ce0 0x559f531ec920 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f81a40058c0 tx=0x7f81a4023640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0a0cf6c58a33d3b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:47.783 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.780+0000 7f8197e00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T08:53:47.783 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.780+0000 7f8197e00640 1 -- 172.21.15.83:0/554389503 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a4029040 con 0x559f531d9ce0 2024-04-24T08:53:47.783 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.780+0000 7f81bbe24740 1 -- 172.21.15.83:0/554389503 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559f53147190 con 0x559f531d9ce0 2024-04-24T08:53:47.783 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.780+0000 7f8197e00640 1 -- 172.21.15.83:0/554389503 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f81a4004490 con 0x559f531d9ce0 2024-04-24T08:53:47.783 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.780+0000 7f8197e00640 1 -- 172.21.15.83:0/554389503 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f81a4050050 con 0x559f531d9ce0 2024-04-24T08:53:47.783 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.781+0000 7f81bbe24740 10 client.35495 fetch_fsmap learned FSMap version 30 2024-04-24T08:53:47.783 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.781+0000 7f81bbe24740 1 -- 172.21.15.83:0/554389503 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559f5325aa90 con 0x559f531d9ce0 2024-04-24T08:53:47.783 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.781+0000 7f8197e00640 1 -- 172.21.15.83:0/554389503 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x559f53147190 con 0x559f531d9ce0 2024-04-24T08:53:47.783 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.781+0000 7f8197e00640 1 -- 172.21.15.83:0/554389503 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7f81a404b030 con 0x559f531d9ce0 2024-04-24T08:53:47.783 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.781+0000 7f81bbe24740 10 client.35495 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T08:53:47.784 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.781+0000 7f81bbe24740 10 client.35495 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T08:53:47.784 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.781+0000 7f81bbe24740 20 client.35495 populate_metadata read hostname 'smithi083' 2024-04-24T08:53:47.784 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.781+0000 7f81bbe24740 1 --2- 172.21.15.83:0/554389503 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x559f5325b1e0 0x559f5325d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:47.784 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.781+0000 7f81bbe24740 4 client.35495 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T08:53:47.784 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.781+0000 7f81bbe24740 1 -- 172.21.15.83:0/554389503 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559f530c34b0 con 0x559f5325b1e0 2024-04-24T08:53:47.784 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.782+0000 7f81abe00640 1 --2- 172.21.15.83:0/554389503 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x559f5325b1e0 0x559f5325d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:47.784 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.782+0000 7f81abe00640 1 --2- 172.21.15.83:0/554389503 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x559f5325b1e0 0x559f5325d600 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=216c551b98de5208 server_cookie=227d488c14574325 in_seq=0 out_seq=0 2024-04-24T08:53:47.784 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.782+0000 7f8197e00640 10 client.35495 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:53:47.818 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.818+0000 7f8197e00640 1 -- 172.21.15.83:0/554389503 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559f530c34b0 con 0x559f5325b1e0 2024-04-24T08:53:47.818 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.818+0000 7f8197e00640 10 client.35495 handle_command_reply: tid=0 2024-04-24T08:53:47.819 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.818+0000 7f81bbe24740 1 client.35495 shutdown 2024-04-24T08:53:47.819 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.819+0000 7f81bbe24740 1 -- 172.21.15.83:0/554389503 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f531d9ce0 msgr2=0x559f531ec920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:47.819 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.819+0000 7f81bbe24740 1 --2- 172.21.15.83:0/554389503 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f531d9ce0 0x559f531ec920 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f81a40058c0 tx=0x7f81a4023640 comp rx=0 tx=0).stop 2024-04-24T08:53:47.819 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.819+0000 7f81bbe24740 1 -- 172.21.15.83:0/554389503 shutdown_connections 2024-04-24T08:53:47.819 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.819+0000 7f81bbe24740 1 --2- 172.21.15.83:0/554389503 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x559f5325b1e0 0x559f5325d600 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:47.819 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.819+0000 7f81bbe24740 1 --2- 172.21.15.83:0/554389503 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559f531e45a0 0x559f531e8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:47.819 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.819+0000 7f81bbe24740 1 --2- 172.21.15.83:0/554389503 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f531e0e80 0x559f531e7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:47.819 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.819+0000 7f81bbe24740 1 --2- 172.21.15.83:0/554389503 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f531d9ce0 0x559f531ec920 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:47.819 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.819+0000 7f8197e00640 0 client.35495 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:53:47.819 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.819+0000 7f81bbe24740 1 -- 172.21.15.83:0/554389503 >> 172.21.15.83:0/554389503 conn(0x559f531d58d0 msgr2=0x559f531e1580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:47.819 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.819+0000 7f81bbe24740 1 -- 172.21.15.83:0/554389503 shutdown_connections 2024-04-24T08:53:47.819 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.819+0000 7f81bbe24740 1 -- 172.21.15.83:0/554389503 wait complete. 2024-04-24T08:53:47.820 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.820+0000 7f81bbe24740 20 client.35495 trim_cache size 0 max 16384 2024-04-24T08:53:47.820 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T08:53:47.820 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T08:53:47.820 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T08:53:47.820 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T08:53:47.821 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.820+0000 7f81b3200640 1 -- 172.21.15.83:0/499690966 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f81980a3d10 msgr2=0x7f81980a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:47.821 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.820+0000 7f81b3200640 1 --2- 172.21.15.83:0/499690966 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f81980a3d10 0x7f81980a61d0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f81ac005fd0 tx=0x7f81ac009500 comp rx=0 tx=0).stop 2024-04-24T08:53:47.821 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.820+0000 7f81b3200640 1 -- 172.21.15.83:0/499690966 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81b4103940 msgr2=0x7f81b41a0b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:47.821 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.820+0000 7f81b3200640 1 --2- 172.21.15.83:0/499690966 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81b4103940 0x7f81b41a0b10 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f819c005ab0 tx=0x7f819c003c10 comp rx=0 tx=0).stop 2024-04-24T08:53:47.821 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.821+0000 7f81b3200640 1 -- 172.21.15.83:0/499690966 shutdown_connections 2024-04-24T08:53:47.821 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.821+0000 7f81b3200640 1 --2- 172.21.15.83:0/499690966 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f81980a3d10 0x7f81980a61d0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:47.821 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.821+0000 7f81b3200640 1 --2- 172.21.15.83:0/499690966 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f81b410c3f0 0x7f81b419a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:47.821 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.821+0000 7f81b3200640 1 --2- 172.21.15.83:0/499690966 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81b4103940 0x7f81b41a0b10 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:47.822 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.821+0000 7f81b3200640 1 --2- 172.21.15.83:0/499690966 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81b4103020 0x7f81b41a05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:47.822 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.821+0000 7f81b3200640 1 -- 172.21.15.83:0/499690966 >> 172.21.15.83:0/499690966 conn(0x7f81b40fc820 msgr2=0x7f81b4100c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:47.822 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.821+0000 7f81b3200640 1 -- 172.21.15.83:0/499690966 shutdown_connections 2024-04-24T08:53:47.822 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:53:47.821+0000 7f81b3200640 1 -- 172.21.15.83:0/499690966 wait complete. 2024-04-24T08:53:47.954 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:53:47 smithi138 ceph-mon[31441]: pgmap v68: 129 pgs: 129 active+clean; 24 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 242 KiB/s wr, 58 op/s 2024-04-24T08:53:47.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:53:47 smithi189 ceph-mon[31791]: pgmap v68: 129 pgs: 129 active+clean; 24 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 242 KiB/s wr, 58 op/s 2024-04-24T08:53:48.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:53:47 smithi083 ceph-mon[27907]: pgmap v68: 129 pgs: 129 active+clean; 24 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 242 KiB/s wr, 58 op/s 2024-04-24T08:53:48.556 INFO:tasks.workunit.client.0.smithi083.stdout: 1 3379 1.61 MB/sec warmup 22 sec latency 49.826 ms 2024-04-24T08:53:48.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:53:48 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:53:48.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:53:48 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:53:49.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:53:48 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:53:49.556 INFO:tasks.workunit.client.0.smithi083.stdout: 1 3588 1.67 MB/sec warmup 23 sec latency 48.148 ms 2024-04-24T08:53:49.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:53:49 smithi138 ceph-mon[31441]: pgmap v69: 129 pgs: 129 active+clean; 25 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 427 KiB/s wr, 77 op/s 2024-04-24T08:53:49.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:53:49 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:53:49.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:53:49 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:53:49.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:53:49 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T08:53:49.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:53:49 smithi189 ceph-mon[31791]: pgmap v69: 129 pgs: 129 active+clean; 25 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 427 KiB/s wr, 77 op/s 2024-04-24T08:53:49.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:53:49 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:53:49.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:53:49 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:53:49.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:53:49 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T08:53:50.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:53:49 smithi083 ceph-mon[27907]: pgmap v69: 129 pgs: 129 active+clean; 25 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 427 KiB/s wr, 77 op/s 2024-04-24T08:53:50.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:53:49 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:53:50.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:53:49 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:53:50.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:53:49 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T08:53:50.556 INFO:tasks.workunit.client.0.smithi083.stdout: 1 3715 1.62 MB/sec warmup 24 sec latency 47.485 ms 2024-04-24T08:53:51.557 INFO:tasks.workunit.client.0.smithi083.stdout: 1 3805 1.59 MB/sec warmup 25 sec latency 44.538 ms 2024-04-24T08:53:51.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:53:51 smithi138 ceph-mon[31441]: pgmap v70: 129 pgs: 129 active+clean; 25 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 371 KiB/s wr, 35 op/s 2024-04-24T08:53:51.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:53:51 smithi189 ceph-mon[31791]: pgmap v70: 129 pgs: 129 active+clean; 25 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 371 KiB/s wr, 35 op/s 2024-04-24T08:53:52.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:53:51 smithi083 ceph-mon[27907]: pgmap v70: 129 pgs: 129 active+clean; 25 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 371 KiB/s wr, 35 op/s 2024-04-24T08:53:52.557 INFO:tasks.workunit.client.0.smithi083.stdout: 1 3867 1.53 MB/sec warmup 26 sec latency 41.651 ms 2024-04-24T08:53:53.557 INFO:tasks.workunit.client.0.smithi083.stdout: 1 3936 1.48 MB/sec warmup 27 sec latency 49.297 ms 2024-04-24T08:53:53.954 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:53:53 smithi138 ceph-mon[31441]: pgmap v71: 129 pgs: 129 active+clean; 26 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 585 KiB/s wr, 63 op/s 2024-04-24T08:53:53.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:53:53 smithi189 ceph-mon[31791]: pgmap v71: 129 pgs: 129 active+clean; 26 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 585 KiB/s wr, 63 op/s 2024-04-24T08:53:54.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:53:53 smithi083 ceph-mon[27907]: pgmap v71: 129 pgs: 129 active+clean; 26 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 585 KiB/s wr, 63 op/s 2024-04-24T08:53:54.557 INFO:tasks.workunit.client.0.smithi083.stdout: 1 3991 1.43 MB/sec warmup 28 sec latency 44.781 ms 2024-04-24T08:53:55.557 INFO:tasks.workunit.client.0.smithi083.stdout: 1 4047 1.38 MB/sec warmup 29 sec latency 49.953 ms 2024-04-24T08:53:55.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:53:55 smithi138 ceph-mon[31441]: pgmap v72: 129 pgs: 129 active+clean; 26 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 399 KiB/s wr, 47 op/s 2024-04-24T08:53:55.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:53:55 smithi189 ceph-mon[31791]: pgmap v72: 129 pgs: 129 active+clean; 26 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 399 KiB/s wr, 47 op/s 2024-04-24T08:53:56.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:53:55 smithi083 ceph-mon[27907]: pgmap v72: 129 pgs: 129 active+clean; 26 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 399 KiB/s wr, 47 op/s 2024-04-24T08:53:56.557 INFO:tasks.workunit.client.0.smithi083.stdout: 1 4129 1.34 MB/sec warmup 30 sec latency 49.824 ms 2024-04-24T08:53:57.557 INFO:tasks.workunit.client.0.smithi083.stdout: 1 4217 1.30 MB/sec warmup 31 sec latency 48.389 ms 2024-04-24T08:53:57.771 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T08:53:57.771 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T08:53:57.846 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:53:57 smithi083 ceph-mon[27907]: pgmap v73: 129 pgs: 129 active+clean; 26 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 399 KiB/s wr, 47 op/s 2024-04-24T08:53:57.954 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:53:57 smithi138 ceph-mon[31441]: pgmap v73: 129 pgs: 129 active+clean; 26 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 399 KiB/s wr, 47 op/s 2024-04-24T08:53:57.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:53:57 smithi189 ceph-mon[31791]: pgmap v73: 129 pgs: 129 active+clean; 26 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 399 KiB/s wr, 47 op/s 2024-04-24T08:53:57.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:57.988+0000 7fee34a00640 1 -- 172.21.15.83:0/2856463105 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee3010c4f0 msgr2=0x7fee3010e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:57.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:57.988+0000 7fee34a00640 1 --2- 172.21.15.83:0/2856463105 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee3010c4f0 0x7fee3010e920 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fee28009ca0 tx=0x7fee2802a950 comp rx=0 tx=0).stop 2024-04-24T08:53:57.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:57.988+0000 7fee34a00640 1 -- 172.21.15.83:0/2856463105 shutdown_connections 2024-04-24T08:53:57.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:57.988+0000 7fee34a00640 1 --2- 172.21.15.83:0/2856463105 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee3010c4f0 0x7fee3010e920 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:57.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:57.988+0000 7fee34a00640 1 --2- 172.21.15.83:0/2856463105 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee30103a40 0x7fee3010bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:57.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:57.988+0000 7fee34a00640 1 --2- 172.21.15.83:0/2856463105 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fee30103120 0x7fee30103500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:57.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:57.988+0000 7fee34a00640 1 -- 172.21.15.83:0/2856463105 >> 172.21.15.83:0/2856463105 conn(0x7fee300fc880 msgr2=0x7fee300feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:57.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:57.988+0000 7fee34a00640 1 -- 172.21.15.83:0/2856463105 shutdown_connections 2024-04-24T08:53:57.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:57.988+0000 7fee34a00640 1 -- 172.21.15.83:0/2856463105 wait complete. 2024-04-24T08:53:57.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:57.989+0000 7fee34a00640 1 Processor -- start 2024-04-24T08:53:57.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:57.989+0000 7fee34a00640 1 -- start start 2024-04-24T08:53:57.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:57.989+0000 7fee34a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fee30103120 0x7fee301a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:57.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:57.989+0000 7fee34a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee30103a40 0x7fee301a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:57.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:57.990+0000 7fee34a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee3010c4f0 0x7fee3019a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:57.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:57.990+0000 7fee34a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fee30112110 con 0x7fee3010c4f0 2024-04-24T08:53:57.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:57.990+0000 7fee34a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fee30111f90 con 0x7fee30103a40 2024-04-24T08:53:57.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:57.990+0000 7fee34a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fee30112290 con 0x7fee30103120 2024-04-24T08:53:57.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:57.990+0000 7fee2fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee3010c4f0 0x7fee3019a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:57.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:57.990+0000 7fee2fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee3010c4f0 0x7fee3019a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39390/0 (socket says 172.21.15.83:39390) 2024-04-24T08:53:57.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:57.990+0000 7fee2fe00640 1 -- 172.21.15.83:0/3130322910 learned_addr learned my addr 172.21.15.83:0/3130322910 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:53:57.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:57.990+0000 7fee2f400640 1 --2- 172.21.15.83:0/3130322910 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fee30103120 0x7fee301a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:57.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:57.990+0000 7fee2fe00640 1 -- 172.21.15.83:0/3130322910 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fee30103120 msgr2=0x7fee301a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:57.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:57.990+0000 7fee2fe00640 1 --2- 172.21.15.83:0/3130322910 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fee30103120 0x7fee301a0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:57.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:57.990+0000 7fee2ea00640 1 --2- 172.21.15.83:0/3130322910 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee30103a40 0x7fee301a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:57.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:57.990+0000 7fee2fe00640 1 -- 172.21.15.83:0/3130322910 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee30103a40 msgr2=0x7fee301a0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:57.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:57.990+0000 7fee2fe00640 1 --2- 172.21.15.83:0/3130322910 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee30103a40 0x7fee301a0b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:57.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:57.990+0000 7fee2fe00640 1 -- 172.21.15.83:0/3130322910 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee3019afa0 con 0x7fee3010c4f0 2024-04-24T08:53:57.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:57.991+0000 7fee2fe00640 1 --2- 172.21.15.83:0/3130322910 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee3010c4f0 0x7fee3019a6e0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fee2802ae70 tx=0x7fee2802f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:57.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:57.991+0000 7fee27e00640 1 -- 172.21.15.83:0/3130322910 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee28034040 con 0x7fee3010c4f0 2024-04-24T08:53:57.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:57.991+0000 7fee27e00640 1 -- 172.21.15.83:0/3130322910 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fee280308f0 con 0x7fee3010c4f0 2024-04-24T08:53:57.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:57.991+0000 7fee34a00640 1 -- 172.21.15.83:0/3130322910 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee3019b230 con 0x7fee3010c4f0 2024-04-24T08:53:57.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:57.991+0000 7fee27e00640 1 -- 172.21.15.83:0/3130322910 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee280047c0 con 0x7fee3010c4f0 2024-04-24T08:53:57.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:57.991+0000 7fee34a00640 1 -- 172.21.15.83:0/3130322910 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee301a75a0 con 0x7fee3010c4f0 2024-04-24T08:53:57.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:57.995+0000 7fee27e00640 1 -- 172.21.15.83:0/3130322910 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fee28037020 con 0x7fee3010c4f0 2024-04-24T08:53:57.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:57.995+0000 7fee38e4e740 1 Processor -- start 2024-04-24T08:53:57.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:57.995+0000 7fee38e4e740 1 -- start start 2024-04-24T08:53:57.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:57.996+0000 7fee38e4e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55889ed65a30 0x55889ed65e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:57.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:57.996+0000 7fee38e4e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55889ed66350 0x55889ed704e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:57.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:57.996+0000 7fee38e4e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55889ed70a20 0x55889ed72e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:57.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:57.996+0000 7fee38e4e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55889eccb1f0 con 0x55889ed66350 2024-04-24T08:53:57.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:57.996+0000 7fee38e4e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55889ec71350 con 0x55889ed65a30 2024-04-24T08:53:57.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:57.996+0000 7fee38e4e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55889ecc71b0 con 0x55889ed70a20 2024-04-24T08:53:57.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:57.996+0000 7fee2f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55889ed66350 0x55889ed704e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:57.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:57.996+0000 7fee2f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55889ed66350 0x55889ed704e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39396/0 (socket says 172.21.15.83:39396) 2024-04-24T08:53:57.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:57.996+0000 7fee2f400640 1 -- 172.21.15.83:0/1445578522 learned_addr learned my addr 172.21.15.83:0/1445578522 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:53:57.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:57.997+0000 7fee2ea00640 1 --2- 172.21.15.83:0/1445578522 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55889ed70a20 0x55889ed72e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:57.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:57.997+0000 7fee2f400640 1 -- 172.21.15.83:0/1445578522 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55889ed70a20 msgr2=0x55889ed72e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:57.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:57.997+0000 7fee2f400640 1 --2- 172.21.15.83:0/1445578522 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55889ed70a20 0x55889ed72e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:57.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:57.997+0000 7fee2f400640 1 -- 172.21.15.83:0/1445578522 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55889ed65a30 msgr2=0x55889ed65e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:57.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:57.997+0000 7fee2f400640 1 --2- 172.21.15.83:0/1445578522 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55889ed65a30 0x55889ed65e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:57.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:57.997+0000 7fee2f400640 1 -- 172.21.15.83:0/1445578522 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55889ecc5f50 con 0x55889ed66350 2024-04-24T08:53:57.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:57.997+0000 7fee27e00640 1 --2- 172.21.15.83:0/3130322910 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fee140a3da0 0x7fee140a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:57.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:57.997+0000 7fee2f400640 1 --2- 172.21.15.83:0/1445578522 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55889ed66350 0x55889ed704e0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fee20009bc0 tx=0x7fee2000e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40c1a35bce9253f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:57.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:57.997+0000 7fee27e00640 1 -- 172.21.15.83:0/3130322910 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7fee280f1b40 con 0x7fee3010c4f0 2024-04-24T08:53:57.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:57.997+0000 7fee2ea00640 1 --2- 172.21.15.83:0/3130322910 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fee140a3da0 0x7fee140a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:57.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:57.998+0000 7fee1be00640 1 -- 172.21.15.83:0/1445578522 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee20016070 con 0x55889ed66350 2024-04-24T08:53:57.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:57.998+0000 7fee1be00640 1 -- 172.21.15.83:0/1445578522 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fee20011920 con 0x55889ed66350 2024-04-24T08:53:57.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:57.998+0000 7fee1be00640 1 -- 172.21.15.83:0/1445578522 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee20010890 con 0x55889ed66350 2024-04-24T08:53:57.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:57.999+0000 7fee38e4e740 1 -- 172.21.15.83:0/1445578522 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55889ed66350 msgr2=0x55889ed704e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:57.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:57.999+0000 7fee2ea00640 1 --2- 172.21.15.83:0/3130322910 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fee140a3da0 0x7fee140a6260 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fee1c005d80 tx=0x7fee1c005e50 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:57.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:57.999+0000 7fee38e4e740 1 --2- 172.21.15.83:0/1445578522 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55889ed66350 0x55889ed704e0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fee20009bc0 tx=0x7fee2000e6c0 comp rx=0 tx=0).stop 2024-04-24T08:53:57.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:57.999+0000 7fee38e4e740 1 -- 172.21.15.83:0/1445578522 shutdown_connections 2024-04-24T08:53:57.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:57.999+0000 7fee38e4e740 1 --2- 172.21.15.83:0/1445578522 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55889ed70a20 0x55889ed72e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:58.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:57.999+0000 7fee38e4e740 1 --2- 172.21.15.83:0/1445578522 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55889ed66350 0x55889ed704e0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:58.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:57.999+0000 7fee38e4e740 1 --2- 172.21.15.83:0/1445578522 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55889ed65a30 0x55889ed65e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:58.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:57.999+0000 7fee38e4e740 1 -- 172.21.15.83:0/1445578522 >> 172.21.15.83:0/1445578522 conn(0x55889ed61d50 msgr2=0x55889ed62130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:58.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:57.999+0000 7fee38e4e740 1 -- 172.21.15.83:0/1445578522 shutdown_connections 2024-04-24T08:53:58.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:57.999+0000 7fee38e4e740 1 -- 172.21.15.83:0/1445578522 wait complete. 2024-04-24T08:53:58.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.000+0000 7fee38e4e740 1 -- start start 2024-04-24T08:53:58.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.000+0000 7fee38e4e740 1 Processor -- start 2024-04-24T08:53:58.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.000+0000 7fee38e4e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55889ed65a30 0x55889ed6d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:58.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.000+0000 7fee38e4e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55889ed66350 0x55889ed6df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:58.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.001+0000 7fee38e4e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55889ed70a20 0x55889ed6e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:58.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.001+0000 7fee38e4e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55889ed73810 con 0x55889ed70a20 2024-04-24T08:53:58.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.001+0000 7fee38e4e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55889ec84690 con 0x55889ed65a30 2024-04-24T08:53:58.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.001+0000 7fee38e4e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55889ed73990 con 0x55889ed66350 2024-04-24T08:53:58.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.001+0000 7fee38e4e740 10 client.0 _ll_register_callbacks cb 0x55889ecc0650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:53:58.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.001+0000 7fee2fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55889ed70a20 0x55889ed6e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:58.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.001+0000 7fee2fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55889ed70a20 0x55889ed6e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39406/0 (socket says 172.21.15.83:39406) 2024-04-24T08:53:58.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.001+0000 7fee2fe00640 1 -- 172.21.15.83:0/3307017658 learned_addr learned my addr 172.21.15.83:0/3307017658 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:53:58.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.002+0000 7fee2fe00640 1 -- 172.21.15.83:0/3307017658 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55889ed66350 msgr2=0x55889ed6df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:58.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.002+0000 7fee2fe00640 1 --2- 172.21.15.83:0/3307017658 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55889ed66350 0x55889ed6df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:58.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.002+0000 7fee2fe00640 1 -- 172.21.15.83:0/3307017658 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55889ed65a30 msgr2=0x55889ed6d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:58.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.002+0000 7fee2fe00640 1 --2- 172.21.15.83:0/3307017658 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55889ed65a30 0x55889ed6d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:58.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.002+0000 7fee2fe00640 1 -- 172.21.15.83:0/3307017658 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55889ec83920 con 0x55889ed70a20 2024-04-24T08:53:58.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.002+0000 7fee2fe00640 1 --2- 172.21.15.83:0/3307017658 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55889ed70a20 0x55889ed6e470 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fee280ba190 tx=0x7fee280bd650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77ec8a31b5194bb6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:58.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.002+0000 7fee18c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:53:58.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.002+0000 7fee18c00640 1 -- 172.21.15.83:0/3307017658 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee2802c040 con 0x55889ed70a20 2024-04-24T08:53:58.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.002+0000 7fee18c00640 1 -- 172.21.15.83:0/3307017658 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7fee28004040 con 0x55889ed70a20 2024-04-24T08:53:58.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.002+0000 7fee38e4e740 1 -- 172.21.15.83:0/3307017658 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee20011920 con 0x55889ed70a20 2024-04-24T08:53:58.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.002+0000 7fee18c00640 1 -- 172.21.15.83:0/3307017658 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fee280db050 con 0x55889ed70a20 2024-04-24T08:53:58.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.003+0000 7fee38e4e740 10 client.16467 fetch_fsmap learned FSMap version 30 2024-04-24T08:53:58.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.003+0000 7fee38e4e740 1 -- 172.21.15.83:0/3307017658 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55889eaa77d0 con 0x55889ed70a20 2024-04-24T08:53:58.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.003+0000 7fee18c00640 1 -- 172.21.15.83:0/3307017658 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fee20011920 con 0x55889ed70a20 2024-04-24T08:53:58.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.003+0000 7fee18c00640 1 -- 172.21.15.83:0/3307017658 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7fee2804d030 con 0x55889ed70a20 2024-04-24T08:53:58.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.003+0000 7fee38e4e740 10 client.16467 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T08:53:58.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.003+0000 7fee38e4e740 10 client.16467 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:53:58.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.003+0000 7fee38e4e740 20 client.16467 populate_metadata read hostname 'smithi083' 2024-04-24T08:53:58.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.003+0000 7fee38e4e740 1 --2- 172.21.15.83:0/3307017658 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55889ed80e20 0x55889ed832e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:58.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.003+0000 7fee38e4e740 4 client.16467 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:53:58.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.003+0000 7fee38e4e740 1 -- 172.21.15.83:0/3307017658 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55889ecd8c10 con 0x55889ed80e20 2024-04-24T08:53:58.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.006+0000 7fee2f400640 1 --2- 172.21.15.83:0/3307017658 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55889ed80e20 0x55889ed832e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:58.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.008+0000 7fee2f400640 1 --2- 172.21.15.83:0/3307017658 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55889ed80e20 0x55889ed832e0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6aaf310e7cc3e68 server_cookie=7fcbaf9babbf626d in_seq=0 out_seq=0 2024-04-24T08:53:58.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.008+0000 7fee18c00640 10 client.16467 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:53:58.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.011+0000 7fee18c00640 1 -- 172.21.15.83:0/3307017658 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55889ecd8c10 con 0x55889ed80e20 2024-04-24T08:53:58.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.011+0000 7fee18c00640 10 client.16467 handle_command_reply: tid=0 2024-04-24T08:53:58.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.011+0000 7fee38e4e740 1 client.16467 shutdown 2024-04-24T08:53:58.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.012+0000 7fee38e4e740 1 -- 172.21.15.83:0/3307017658 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55889ed70a20 msgr2=0x55889ed6e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:58.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.012+0000 7fee38e4e740 1 --2- 172.21.15.83:0/3307017658 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55889ed70a20 0x55889ed6e470 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fee280ba190 tx=0x7fee280bd650 comp rx=0 tx=0).stop 2024-04-24T08:53:58.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.012+0000 7fee38e4e740 1 -- 172.21.15.83:0/3307017658 shutdown_connections 2024-04-24T08:53:58.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.012+0000 7fee38e4e740 1 --2- 172.21.15.83:0/3307017658 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55889ed80e20 0x55889ed832e0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:58.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.012+0000 7fee38e4e740 1 --2- 172.21.15.83:0/3307017658 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55889ed70a20 0x55889ed6e470 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:58.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.012+0000 7fee38e4e740 1 --2- 172.21.15.83:0/3307017658 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55889ed66350 0x55889ed6df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:58.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.012+0000 7fee38e4e740 1 --2- 172.21.15.83:0/3307017658 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55889ed65a30 0x55889ed6d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:58.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.012+0000 7fee38e4e740 1 -- 172.21.15.83:0/3307017658 >> 172.21.15.83:0/3307017658 conn(0x55889ed61d50 msgr2=0x55889ed60fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:58.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.012+0000 7fee18c00640 0 client.16467 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:53:58.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.012+0000 7fee38e4e740 1 -- 172.21.15.83:0/3307017658 shutdown_connections 2024-04-24T08:53:58.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.012+0000 7fee38e4e740 1 -- 172.21.15.83:0/3307017658 wait complete. 2024-04-24T08:53:58.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.012+0000 7fee38e4e740 20 client.16467 trim_cache size 0 max 16384 2024-04-24T08:53:58.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.032+0000 7fee38e4e740 1 Processor -- start 2024-04-24T08:53:58.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.032+0000 7fee38e4e740 1 -- start start 2024-04-24T08:53:58.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.032+0000 7fee38e4e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55889ed65a30 0x55889ed7c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:58.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.032+0000 7fee38e4e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55889ed66350 0x55889ed67a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:58.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.032+0000 7fee38e4e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55889ed70a20 0x55889ed6e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:58.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.032+0000 7fee38e4e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fee301b13f0 con 0x55889ed66350 2024-04-24T08:53:58.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.032+0000 7fee38e4e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fee301b2fc0 con 0x55889ed65a30 2024-04-24T08:53:58.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.032+0000 7fee38e4e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fee301b1ac0 con 0x55889ed70a20 2024-04-24T08:53:58.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.033+0000 7fee2ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55889ed66350 0x55889ed67a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:58.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.033+0000 7fee2ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55889ed66350 0x55889ed67a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39416/0 (socket says 172.21.15.83:39416) 2024-04-24T08:53:58.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.033+0000 7fee2ea00640 1 -- 172.21.15.83:0/3577913196 learned_addr learned my addr 172.21.15.83:0/3577913196 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:53:58.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.033+0000 7fee2f400640 1 --2- 172.21.15.83:0/3577913196 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55889ed65a30 0x55889ed7c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:58.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.033+0000 7fee2fe00640 1 --2- 172.21.15.83:0/3577913196 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55889ed70a20 0x55889ed6e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:58.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.034+0000 7fee2ea00640 1 -- 172.21.15.83:0/3577913196 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55889ed70a20 msgr2=0x55889ed6e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:58.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.034+0000 7fee2ea00640 1 --2- 172.21.15.83:0/3577913196 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55889ed70a20 0x55889ed6e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:58.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.034+0000 7fee2ea00640 1 -- 172.21.15.83:0/3577913196 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55889ed65a30 msgr2=0x55889ed7c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:58.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.034+0000 7fee2ea00640 1 --2- 172.21.15.83:0/3577913196 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55889ed65a30 0x55889ed7c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:58.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.034+0000 7fee2ea00640 1 -- 172.21.15.83:0/3577913196 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55889ed68090 con 0x55889ed66350 2024-04-24T08:53:58.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.034+0000 7fee2fe00640 1 --2- 172.21.15.83:0/3577913196 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55889ed70a20 0x55889ed6e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:53:58.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.034+0000 7fee2ea00640 1 --2- 172.21.15.83:0/3577913196 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55889ed66350 0x55889ed67a10 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fee1c00fc10 tx=0x7fee1c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bfac311084ec653 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:58.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.034+0000 7fee1b400640 1 -- 172.21.15.83:0/3577913196 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee1c022070 con 0x55889ed66350 2024-04-24T08:53:58.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.034+0000 7fee1b400640 1 -- 172.21.15.83:0/3577913196 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fee1c0054d0 con 0x55889ed66350 2024-04-24T08:53:58.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.034+0000 7fee1b400640 1 -- 172.21.15.83:0/3577913196 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee1c01e450 con 0x55889ed66350 2024-04-24T08:53:58.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.035+0000 7fee38e4e740 1 -- 172.21.15.83:0/3577913196 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55889ed66350 msgr2=0x55889ed67a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:58.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.035+0000 7fee38e4e740 1 --2- 172.21.15.83:0/3577913196 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55889ed66350 0x55889ed67a10 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fee1c00fc10 tx=0x7fee1c004730 comp rx=0 tx=0).stop 2024-04-24T08:53:58.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.035+0000 7fee38e4e740 1 -- 172.21.15.83:0/3577913196 shutdown_connections 2024-04-24T08:53:58.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.035+0000 7fee38e4e740 1 --2- 172.21.15.83:0/3577913196 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55889ed70a20 0x55889ed6e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:58.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.035+0000 7fee38e4e740 1 --2- 172.21.15.83:0/3577913196 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55889ed66350 0x55889ed67a10 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:58.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.035+0000 7fee38e4e740 1 --2- 172.21.15.83:0/3577913196 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55889ed65a30 0x55889ed7c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:58.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.035+0000 7fee38e4e740 1 -- 172.21.15.83:0/3577913196 >> 172.21.15.83:0/3577913196 conn(0x55889ed61d50 msgr2=0x55889ed6d9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:58.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.035+0000 7fee38e4e740 1 -- 172.21.15.83:0/3577913196 shutdown_connections 2024-04-24T08:53:58.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.035+0000 7fee38e4e740 1 -- 172.21.15.83:0/3577913196 wait complete. 2024-04-24T08:53:58.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.036+0000 7fee38e4e740 1 -- start start 2024-04-24T08:53:58.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.036+0000 7fee38e4e740 1 Processor -- start 2024-04-24T08:53:58.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.036+0000 7fee38e4e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55889ed65a30 0x55889ed78c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:58.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.036+0000 7fee38e4e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55889ed66350 0x55889ed79180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:58.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.036+0000 7fee38e4e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55889ed70a20 0x55889ed75c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:58.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.036+0000 7fee38e4e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fee301b2860 con 0x55889ed65a30 2024-04-24T08:53:58.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.036+0000 7fee38e4e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fee301b2190 con 0x55889ed66350 2024-04-24T08:53:58.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.036+0000 7fee38e4e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fee301af3d0 con 0x55889ed70a20 2024-04-24T08:53:58.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.036+0000 7fee38e4e740 10 client.0 _ll_register_callbacks cb 0x7fee28005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:53:58.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.037+0000 7fee2f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55889ed65a30 0x55889ed78c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:58.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.037+0000 7fee2f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55889ed65a30 0x55889ed78c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39418/0 (socket says 172.21.15.83:39418) 2024-04-24T08:53:58.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.037+0000 7fee2f400640 1 -- 172.21.15.83:0/2490556875 learned_addr learned my addr 172.21.15.83:0/2490556875 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:53:58.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.037+0000 7fee2ea00640 1 --2- 172.21.15.83:0/2490556875 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55889ed66350 0x55889ed79180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:58.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.037+0000 7fee2f400640 1 -- 172.21.15.83:0/2490556875 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55889ed70a20 msgr2=0x55889ed75c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:58.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.037+0000 7fee2f400640 1 --2- 172.21.15.83:0/2490556875 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55889ed70a20 0x55889ed75c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:58.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.037+0000 7fee2f400640 1 -- 172.21.15.83:0/2490556875 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55889ed66350 msgr2=0x55889ed79180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:58.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.037+0000 7fee2f400640 1 --2- 172.21.15.83:0/2490556875 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55889ed66350 0x55889ed79180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:58.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.037+0000 7fee2f400640 1 -- 172.21.15.83:0/2490556875 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55889ed73180 con 0x55889ed65a30 2024-04-24T08:53:58.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.037+0000 7fee2f400640 1 --2- 172.21.15.83:0/2490556875 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55889ed65a30 0x55889ed78c40 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fee20009b10 tx=0x7fee20009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=756254c1bfcd86bb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:58.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.037+0000 7fee18c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:53:58.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.037+0000 7fee18c00640 1 -- 172.21.15.83:0/2490556875 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee20028070 con 0x55889ed65a30 2024-04-24T08:53:58.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.038+0000 7fee18c00640 1 -- 172.21.15.83:0/2490556875 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7fee200247a0 con 0x55889ed65a30 2024-04-24T08:53:58.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.038+0000 7fee38e4e740 1 -- 172.21.15.83:0/2490556875 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee1c0054d0 con 0x55889ed65a30 2024-04-24T08:53:58.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.038+0000 7fee18c00640 1 -- 172.21.15.83:0/2490556875 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fee20031050 con 0x55889ed65a30 2024-04-24T08:53:58.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.038+0000 7fee38e4e740 10 client.16473 fetch_fsmap learned FSMap version 30 2024-04-24T08:53:58.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.038+0000 7fee38e4e740 1 -- 172.21.15.83:0/2490556875 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55889e9dd3f0 con 0x55889ed65a30 2024-04-24T08:53:58.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.038+0000 7fee18c00640 1 -- 172.21.15.83:0/2490556875 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fee1c0054d0 con 0x55889ed65a30 2024-04-24T08:53:58.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.038+0000 7fee18c00640 1 -- 172.21.15.83:0/2490556875 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7fee2002b020 con 0x55889ed65a30 2024-04-24T08:53:58.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.038+0000 7fee38e4e740 10 client.16473 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T08:53:58.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.038+0000 7fee38e4e740 10 client.16473 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:53:58.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.038+0000 7fee38e4e740 20 client.16473 populate_metadata read hostname 'smithi083' 2024-04-24T08:53:58.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.038+0000 7fee38e4e740 1 --2- 172.21.15.83:0/2490556875 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55889ede7fb0 0x55889edea3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:58.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.039+0000 7fee38e4e740 4 client.16473 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T08:53:58.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.039+0000 7fee38e4e740 1 -- 172.21.15.83:0/2490556875 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55889ecd2a80 con 0x55889ede7fb0 2024-04-24T08:53:58.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.039+0000 7fee2ea00640 1 --2- 172.21.15.83:0/2490556875 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55889ede7fb0 0x55889edea3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:58.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.040+0000 7fee2ea00640 1 --2- 172.21.15.83:0/2490556875 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55889ede7fb0 0x55889edea3f0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5b8a06f417ada72 server_cookie=7b65d4764729d493 in_seq=0 out_seq=0 2024-04-24T08:53:58.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.040+0000 7fee18c00640 10 client.16473 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:53:58.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.040+0000 7fee18c00640 1 -- 172.21.15.83:0/2490556875 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55889ecd2a80 con 0x55889ede7fb0 2024-04-24T08:53:58.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.040+0000 7fee18c00640 10 client.16473 handle_command_reply: tid=0 2024-04-24T08:53:58.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.040+0000 7fee38e4e740 1 client.16473 shutdown 2024-04-24T08:53:58.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.041+0000 7fee38e4e740 1 -- 172.21.15.83:0/2490556875 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55889ed65a30 msgr2=0x55889ed78c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:58.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.041+0000 7fee38e4e740 1 --2- 172.21.15.83:0/2490556875 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55889ed65a30 0x55889ed78c40 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fee20009b10 tx=0x7fee20009f60 comp rx=0 tx=0).stop 2024-04-24T08:53:58.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.041+0000 7fee38e4e740 1 -- 172.21.15.83:0/2490556875 shutdown_connections 2024-04-24T08:53:58.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.041+0000 7fee38e4e740 1 --2- 172.21.15.83:0/2490556875 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55889ede7fb0 0x55889edea3f0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:58.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.041+0000 7fee38e4e740 1 --2- 172.21.15.83:0/2490556875 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55889ed70a20 0x55889ed75c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:58.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.041+0000 7fee38e4e740 1 --2- 172.21.15.83:0/2490556875 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55889ed66350 0x55889ed79180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:58.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.041+0000 7fee38e4e740 1 --2- 172.21.15.83:0/2490556875 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55889ed65a30 0x55889ed78c40 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:58.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.041+0000 7fee38e4e740 1 -- 172.21.15.83:0/2490556875 >> 172.21.15.83:0/2490556875 conn(0x55889ed61d50 msgr2=0x55889ed67890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:58.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.041+0000 7fee18c00640 0 client.16473 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:53:58.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.041+0000 7fee38e4e740 1 -- 172.21.15.83:0/2490556875 shutdown_connections 2024-04-24T08:53:58.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.041+0000 7fee38e4e740 1 -- 172.21.15.83:0/2490556875 wait complete. 2024-04-24T08:53:58.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.041+0000 7fee38e4e740 20 client.16473 trim_cache size 0 max 16384 2024-04-24T08:53:58.043 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T08:53:58.043 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-04-24T08:53:58.044 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-04-24T08:53:58.044 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T08:53:58.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.042+0000 7fee34a00640 1 -- 172.21.15.83:0/3130322910 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fee140a3da0 msgr2=0x7fee140a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:58.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.042+0000 7fee34a00640 1 --2- 172.21.15.83:0/3130322910 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fee140a3da0 0x7fee140a6260 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fee1c005d80 tx=0x7fee1c005e50 comp rx=0 tx=0).stop 2024-04-24T08:53:58.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.042+0000 7fee34a00640 1 -- 172.21.15.83:0/3130322910 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee3010c4f0 msgr2=0x7fee3019a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:58.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.042+0000 7fee34a00640 1 --2- 172.21.15.83:0/3130322910 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee3010c4f0 0x7fee3019a6e0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fee2802ae70 tx=0x7fee2802f5d0 comp rx=0 tx=0).stop 2024-04-24T08:53:58.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.042+0000 7fee34a00640 1 -- 172.21.15.83:0/3130322910 shutdown_connections 2024-04-24T08:53:58.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.042+0000 7fee34a00640 1 --2- 172.21.15.83:0/3130322910 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fee140a3da0 0x7fee140a6260 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:58.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.042+0000 7fee34a00640 1 --2- 172.21.15.83:0/3130322910 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee3010c4f0 0x7fee3019a6e0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:58.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.042+0000 7fee34a00640 1 --2- 172.21.15.83:0/3130322910 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee30103a40 0x7fee301a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:58.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.042+0000 7fee34a00640 1 --2- 172.21.15.83:0/3130322910 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fee30103120 0x7fee301a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:58.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.042+0000 7fee34a00640 1 -- 172.21.15.83:0/3130322910 >> 172.21.15.83:0/3130322910 conn(0x7fee300fc880 msgr2=0x7fee30100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:58.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.042+0000 7fee34a00640 1 -- 172.21.15.83:0/3130322910 shutdown_connections 2024-04-24T08:53:58.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.042+0000 7fee34a00640 1 -- 172.21.15.83:0/3130322910 wait complete. 2024-04-24T08:53:58.059 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T08:53:58.060 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T08:53:58.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.259+0000 7fe9f7000640 1 -- 172.21.15.83:0/3697914333 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9f00743d0 msgr2=0x7fe9f00747b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:58.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.259+0000 7fe9f7000640 1 --2- 172.21.15.83:0/3697914333 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9f00743d0 0x7fe9f00747b0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fe9e4009900 tx=0x7fe9e402a4a0 comp rx=0 tx=0).stop 2024-04-24T08:53:58.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.259+0000 7fe9f7000640 1 -- 172.21.15.83:0/3697914333 shutdown_connections 2024-04-24T08:53:58.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.259+0000 7fe9f7000640 1 --2- 172.21.15.83:0/3697914333 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe9f010e6c0 0x7fe9f0110ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:58.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.259+0000 7fe9f7000640 1 --2- 172.21.15.83:0/3697914333 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe9f0074cf0 0x7fe9f010e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:58.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.259+0000 7fe9f7000640 1 --2- 172.21.15.83:0/3697914333 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9f00743d0 0x7fe9f00747b0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:58.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.259+0000 7fe9f7000640 1 -- 172.21.15.83:0/3697914333 >> 172.21.15.83:0/3697914333 conn(0x7fe9f00fd930 msgr2=0x7fe9f00ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:58.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.259+0000 7fe9f7000640 1 -- 172.21.15.83:0/3697914333 shutdown_connections 2024-04-24T08:53:58.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.259+0000 7fe9f7000640 1 -- 172.21.15.83:0/3697914333 wait complete. 2024-04-24T08:53:58.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.260+0000 7fe9f7000640 1 Processor -- start 2024-04-24T08:53:58.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.260+0000 7fe9f7000640 1 -- start start 2024-04-24T08:53:58.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.261+0000 7fe9f7000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe9f00743d0 0x7fe9f01a05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:58.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.261+0000 7fe9f7000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9f0074cf0 0x7fe9f01a0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:58.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.261+0000 7fe9f7000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe9f010e6c0 0x7fe9f019a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:58.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.261+0000 7fe9f7000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe9f0114330 con 0x7fe9f0074cf0 2024-04-24T08:53:58.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.261+0000 7fe9f7000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe9f01141b0 con 0x7fe9f00743d0 2024-04-24T08:53:58.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.261+0000 7fe9f7000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe9f01144b0 con 0x7fe9f010e6c0 2024-04-24T08:53:58.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.261+0000 7fe9f5200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9f0074cf0 0x7fe9f01a0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:58.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.261+0000 7fe9f5200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9f0074cf0 0x7fe9f01a0af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39440/0 (socket says 172.21.15.83:39440) 2024-04-24T08:53:58.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.261+0000 7fe9f5200640 1 -- 172.21.15.83:0/4228035668 learned_addr learned my addr 172.21.15.83:0/4228035668 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:53:58.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.261+0000 7fe9f5c00640 1 --2- 172.21.15.83:0/4228035668 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe9f00743d0 0x7fe9f01a05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:58.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.261+0000 7fe9f5200640 1 -- 172.21.15.83:0/4228035668 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe9f010e6c0 msgr2=0x7fe9f019a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:58.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.261+0000 7fe9f6600640 1 --2- 172.21.15.83:0/4228035668 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe9f010e6c0 0x7fe9f019a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:58.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.261+0000 7fe9f5200640 1 --2- 172.21.15.83:0/4228035668 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe9f010e6c0 0x7fe9f019a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:58.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.261+0000 7fe9f5200640 1 -- 172.21.15.83:0/4228035668 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe9f00743d0 msgr2=0x7fe9f01a05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:58.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.261+0000 7fe9f5200640 1 --2- 172.21.15.83:0/4228035668 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe9f00743d0 0x7fe9f01a05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:58.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.262+0000 7fe9f5200640 1 -- 172.21.15.83:0/4228035668 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9f019af30 con 0x7fe9f0074cf0 2024-04-24T08:53:58.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.262+0000 7fe9f6600640 1 --2- 172.21.15.83:0/4228035668 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe9f010e6c0 0x7fe9f019a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:53:58.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.262+0000 7fe9f5200640 1 --2- 172.21.15.83:0/4228035668 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9f0074cf0 0x7fe9f01a0af0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc00c920 tx=0x7fe9dc00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:58.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.262+0000 7fe9ee000640 1 -- 172.21.15.83:0/4228035668 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9dc007c70 con 0x7fe9f0074cf0 2024-04-24T08:53:58.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.262+0000 7fe9ee000640 1 -- 172.21.15.83:0/4228035668 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe9dc007e10 con 0x7fe9f0074cf0 2024-04-24T08:53:58.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.262+0000 7fe9f7000640 1 -- 172.21.15.83:0/4228035668 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9f019b220 con 0x7fe9f0074cf0 2024-04-24T08:53:58.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.262+0000 7fe9ee000640 1 -- 172.21.15.83:0/4228035668 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9dc018820 con 0x7fe9f0074cf0 2024-04-24T08:53:58.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.262+0000 7fe9f7000640 1 -- 172.21.15.83:0/4228035668 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9f01a7530 con 0x7fe9f0074cf0 2024-04-24T08:53:58.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.264+0000 7fe9f7000640 1 -- 172.21.15.83:0/4228035668 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9f0075a80 con 0x7fe9f0074cf0 2024-04-24T08:53:58.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.266+0000 7fe9ee000640 1 -- 172.21.15.83:0/4228035668 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe9dc0189c0 con 0x7fe9f0074cf0 2024-04-24T08:53:58.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.267+0000 7fe9ee000640 1 --2- 172.21.15.83:0/4228035668 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe9d80a3da0 0x7fe9d80a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:58.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.267+0000 7fe9ee000640 1 -- 172.21.15.83:0/4228035668 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7fe9dc0d0db0 con 0x7fe9f0074cf0 2024-04-24T08:53:58.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.268+0000 7fe9f5c00640 1 --2- 172.21.15.83:0/4228035668 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe9d80a3da0 0x7fe9d80a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:58.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.268+0000 7fe9f5c00640 1 --2- 172.21.15.83:0/4228035668 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe9d80a3da0 0x7fe9d80a6260 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe9e4031440 tx=0x7fe9e4031c30 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:58.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.272+0000 7fe9ee000640 1 -- 172.21.15.83:0/4228035668 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe9dc0d5070 con 0x7fe9f0074cf0 2024-04-24T08:53:58.557 INFO:tasks.workunit.client.0.smithi083.stdout: 1 4362 1.27 MB/sec warmup 32 sec latency 48.115 ms 2024-04-24T08:53:58.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.562+0000 7fe9f7000640 1 -- 172.21.15.83:0/4228035668 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe9f0109840 con 0x7fe9f0074cf0 2024-04-24T08:53:58.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.564+0000 7fe9ee000640 1 -- 172.21.15.83:0/4228035668 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11949 (secure 0 0 0) 0x7fe9dc014030 con 0x7fe9f0074cf0 2024-04-24T08:53:58.564 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:53:58.564 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/358872257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":358872257},{"type":"v1","addr":"172.21.15.83:6835","nonce":358872257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3157070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3157070851},{"type":"v1","addr":"172.21.15.83:6837","nonce":3157070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T08:48:29.422576+0000","modified":"2024-04-24T08:53:42.044317+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34520,"mds_1":34493,"mds_2":34451,"mds_3":34427,"mds_4":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14658":{"gid":14658,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6839/4085470844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":4085470844},{"type":"v1","addr":"172.21.15.83:6839","nonce":4085470844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14676":{"gid":14676,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6841/2634754210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":2634754210},{"type":"v1","addr":"172.21.15.83:6841","nonce":2634754210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24382":{"gid":24382,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/1859921310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1859921310},{"type":"v1","addr":"172.21.15.189:6833","nonce":1859921310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/2241923197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2241923197},{"type":"v1","addr":"172.21.15.189:6835","nonce":2241923197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6837/3172311220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":3172311220},{"type":"v1","addr":"172.21.15.189:6837","nonce":3172311220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.189:6839/2548965336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":2548965336},{"type":"v1","addr":"172.21.15.189:6839","nonce":2548965336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.138:6833/1157967658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1157967658},{"type":"v1","addr":"172.21.15.138:6833","nonce":1157967658}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.138:6835/1909219758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1909219758},{"type":"v1","addr":"172.21.15.138:6835","nonce":1909219758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34493":{"gid":34493,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.138:6837/900566754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":900566754},{"type":"v1","addr":"172.21.15.138:6837","nonce":900566754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34520":{"gid":34520,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.138:6839/2236932393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2236932393},{"type":"v1","addr":"172.21.15.138:6839","nonce":2236932393}]},"join_fscid":-1,"export_targets":[1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34520,"qdb_cluster":[34493,34451,34427,24454,34520]},"id":1}]} 2024-04-24T08:53:58.564 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 30 2024-04-24T08:53:58.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.567+0000 7fe9f7000640 1 -- 172.21.15.83:0/4228035668 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe9d80a3da0 msgr2=0x7fe9d80a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:58.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.567+0000 7fe9f7000640 1 --2- 172.21.15.83:0/4228035668 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe9d80a3da0 0x7fe9d80a6260 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe9e4031440 tx=0x7fe9e4031c30 comp rx=0 tx=0).stop 2024-04-24T08:53:58.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.567+0000 7fe9f7000640 1 -- 172.21.15.83:0/4228035668 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9f0074cf0 msgr2=0x7fe9f01a0af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:58.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.567+0000 7fe9f7000640 1 --2- 172.21.15.83:0/4228035668 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9f0074cf0 0x7fe9f01a0af0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc00c920 tx=0x7fe9dc00cde0 comp rx=0 tx=0).stop 2024-04-24T08:53:58.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.567+0000 7fe9f7000640 1 -- 172.21.15.83:0/4228035668 shutdown_connections 2024-04-24T08:53:58.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.567+0000 7fe9f7000640 1 --2- 172.21.15.83:0/4228035668 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe9d80a3da0 0x7fe9d80a6260 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:58.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.567+0000 7fe9f7000640 1 --2- 172.21.15.83:0/4228035668 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe9f010e6c0 0x7fe9f019a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:58.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.567+0000 7fe9f7000640 1 --2- 172.21.15.83:0/4228035668 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9f0074cf0 0x7fe9f01a0af0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:58.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.567+0000 7fe9f7000640 1 --2- 172.21.15.83:0/4228035668 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe9f00743d0 0x7fe9f01a05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:58.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.567+0000 7fe9f7000640 1 -- 172.21.15.83:0/4228035668 >> 172.21.15.83:0/4228035668 conn(0x7fe9f00fd930 msgr2=0x7fe9f00fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:58.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.567+0000 7fe9f7000640 1 -- 172.21.15.83:0/4228035668 shutdown_connections 2024-04-24T08:53:58.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.567+0000 7fe9f7000640 1 -- 172.21.15.83:0/4228035668 wait complete. 2024-04-24T08:53:58.586 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-24T08:53:58.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.787+0000 7f5691800640 1 -- 172.21.15.83:0/2912496739 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f568c108580 msgr2=0x7f568c108a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:58.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.787+0000 7f5691800640 1 --2- 172.21.15.83:0/2912496739 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f568c108580 0x7f568c108a00 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f5678009930 tx=0x7f567802a440 comp rx=0 tx=0).stop 2024-04-24T08:53:58.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.787+0000 7f5691800640 1 -- 172.21.15.83:0/2912496739 shutdown_connections 2024-04-24T08:53:58.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.787+0000 7f5691800640 1 --2- 172.21.15.83:0/2912496739 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f568c108f40 0x7f568c1153b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:58.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.787+0000 7f5691800640 1 --2- 172.21.15.83:0/2912496739 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f568c108580 0x7f568c108a00 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:58.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.787+0000 7f5691800640 1 --2- 172.21.15.83:0/2912496739 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f568c10d500 0x7f568c10d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:58.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.787+0000 7f5691800640 1 -- 172.21.15.83:0/2912496739 >> 172.21.15.83:0/2912496739 conn(0x7f568c077e20 msgr2=0x7f568c106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:58.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.787+0000 7f5691800640 1 -- 172.21.15.83:0/2912496739 shutdown_connections 2024-04-24T08:53:58.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.787+0000 7f5691800640 1 -- 172.21.15.83:0/2912496739 wait complete. 2024-04-24T08:53:58.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.788+0000 7f5691800640 1 Processor -- start 2024-04-24T08:53:58.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.788+0000 7f5691800640 1 -- start start 2024-04-24T08:53:58.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.789+0000 7f5691800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f568c108580 0x7f568c1a47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:58.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.789+0000 7f5691800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f568c108f40 0x7f568c1a4d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:58.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.789+0000 7f5691800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f568c10d500 0x7f568c19e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:58.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.789+0000 7f5691800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f568c1187b0 con 0x7f568c108f40 2024-04-24T08:53:58.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.789+0000 7f5691800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f568c118630 con 0x7f568c10d500 2024-04-24T08:53:58.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.789+0000 7f5691800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f568c118930 con 0x7f568c108580 2024-04-24T08:53:58.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.789+0000 7f568b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f568c108f40 0x7f568c1a4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:58.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.789+0000 7f568b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f568c108f40 0x7f568c1a4d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39472/0 (socket says 172.21.15.83:39472) 2024-04-24T08:53:58.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.789+0000 7f568b400640 1 -- 172.21.15.83:0/3173793376 learned_addr learned my addr 172.21.15.83:0/3173793376 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:53:58.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.789+0000 7f568be00640 1 --2- 172.21.15.83:0/3173793376 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f568c108580 0x7f568c1a47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:58.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.790+0000 7f5690e00640 1 --2- 172.21.15.83:0/3173793376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f568c10d500 0x7f568c19e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:58.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.790+0000 7f568b400640 1 -- 172.21.15.83:0/3173793376 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f568c108580 msgr2=0x7f568c1a47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:58.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.790+0000 7f568b400640 1 --2- 172.21.15.83:0/3173793376 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f568c108580 0x7f568c1a47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:58.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.790+0000 7f568b400640 1 -- 172.21.15.83:0/3173793376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f568c10d500 msgr2=0x7f568c19e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:58.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.790+0000 7f568b400640 1 --2- 172.21.15.83:0/3173793376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f568c10d500 0x7f568c19e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:58.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.790+0000 7f568b400640 1 -- 172.21.15.83:0/3173793376 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f568c19f150 con 0x7f568c108f40 2024-04-24T08:53:58.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.790+0000 7f5690e00640 1 --2- 172.21.15.83:0/3173793376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f568c10d500 0x7f568c19e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:53:58.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.790+0000 7f568b400640 1 --2- 172.21.15.83:0/3173793376 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f568c108f40 0x7f568c1a4d10 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f5678031440 tx=0x7f5678003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:58.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.790+0000 7f5688c00640 1 -- 172.21.15.83:0/3173793376 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f567803f070 con 0x7f568c108f40 2024-04-24T08:53:58.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.790+0000 7f5688c00640 1 -- 172.21.15.83:0/3173793376 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5678031d60 con 0x7f568c108f40 2024-04-24T08:53:58.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.790+0000 7f5688c00640 1 -- 172.21.15.83:0/3173793376 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5678035030 con 0x7f568c108f40 2024-04-24T08:53:58.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.790+0000 7f5691800640 1 -- 172.21.15.83:0/3173793376 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f568c19f3e0 con 0x7f568c108f40 2024-04-24T08:53:58.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.791+0000 7f5691800640 1 -- 172.21.15.83:0/3173793376 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f568c1ab690 con 0x7f568c108f40 2024-04-24T08:53:58.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.794+0000 7f5695c00740 1 Processor -- start 2024-04-24T08:53:58.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.794+0000 7f5688c00640 1 -- 172.21.15.83:0/3173793376 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5678042070 con 0x7f568c108f40 2024-04-24T08:53:58.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.795+0000 7f5695c00740 1 -- start start 2024-04-24T08:53:58.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.795+0000 7f5695c00740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a613c6cb40 0x55a613c6cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:58.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.795+0000 7f5695c00740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a613c6d460 0x55a613c775f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:58.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.795+0000 7f5695c00740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a613c77b30 0x55a613c79f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:58.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.795+0000 7f5695c00740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a613bd21d0 con 0x55a613c6cb40 2024-04-24T08:53:58.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.795+0000 7f5695c00740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a613b78450 con 0x55a613c6d460 2024-04-24T08:53:58.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.795+0000 7f5695c00740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a613bce190 con 0x55a613c77b30 2024-04-24T08:53:58.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.795+0000 7f5690e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a613c6cb40 0x55a613c6cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:58.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.795+0000 7f5690e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a613c6cb40 0x55a613c6cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39488/0 (socket says 172.21.15.83:39488) 2024-04-24T08:53:58.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.795+0000 7f5690e00640 1 -- 172.21.15.83:0/3457908607 learned_addr learned my addr 172.21.15.83:0/3457908607 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:53:58.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.796+0000 7f568be00640 1 --2- 172.21.15.83:0/3457908607 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a613c6d460 0x55a613c775f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:58.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.796+0000 7f568b400640 1 --2- 172.21.15.83:0/3457908607 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a613c77b30 0x55a613c79f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:58.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.796+0000 7f5690e00640 1 -- 172.21.15.83:0/3457908607 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a613c77b30 msgr2=0x55a613c79f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:58.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.796+0000 7f5690e00640 1 --2- 172.21.15.83:0/3457908607 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a613c77b30 0x55a613c79f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:58.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.796+0000 7f5690e00640 1 -- 172.21.15.83:0/3457908607 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a613c6d460 msgr2=0x55a613c775f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:58.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.796+0000 7f5690e00640 1 --2- 172.21.15.83:0/3457908607 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a613c6d460 0x55a613c775f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:58.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.796+0000 7f5690e00640 1 -- 172.21.15.83:0/3457908607 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a613c35310 con 0x55a613c6cb40 2024-04-24T08:53:58.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.796+0000 7f5690e00640 1 --2- 172.21.15.83:0/3457908607 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a613c6cb40 0x55a613c6cf20 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f5684008e50 tx=0x7f568400d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c15bccbbde7629e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:58.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.796+0000 7f5688c00640 1 --2- 172.21.15.83:0/3173793376 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f56700a3da0 0x7f56700a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:58.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.797+0000 7f5688c00640 1 -- 172.21.15.83:0/3173793376 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f56780f1520 con 0x7f568c108f40 2024-04-24T08:53:58.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.797+0000 7f5681600640 1 -- 172.21.15.83:0/3457908607 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f568400f830 con 0x55a613c6cb40 2024-04-24T08:53:58.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.797+0000 7f5681600640 1 -- 172.21.15.83:0/3457908607 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f568400e3b0 con 0x55a613c6cb40 2024-04-24T08:53:58.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.797+0000 7f568be00640 1 --2- 172.21.15.83:0/3173793376 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f56700a3da0 0x7f56700a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:58.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.797+0000 7f5681600640 1 -- 172.21.15.83:0/3457908607 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f568400b600 con 0x55a613c6cb40 2024-04-24T08:53:58.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.798+0000 7f5695c00740 1 -- 172.21.15.83:0/3457908607 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a613c6cb40 msgr2=0x55a613c6cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:58.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.798+0000 7f568be00640 1 --2- 172.21.15.83:0/3173793376 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f56700a3da0 0x7f56700a6260 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f567c0058c0 tx=0x7f567c005850 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:58.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.798+0000 7f5695c00740 1 --2- 172.21.15.83:0/3457908607 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a613c6cb40 0x55a613c6cf20 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f5684008e50 tx=0x7f568400d710 comp rx=0 tx=0).stop 2024-04-24T08:53:58.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.798+0000 7f5695c00740 1 -- 172.21.15.83:0/3457908607 shutdown_connections 2024-04-24T08:53:58.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.798+0000 7f5695c00740 1 --2- 172.21.15.83:0/3457908607 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a613c77b30 0x55a613c79f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:58.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.798+0000 7f5695c00740 1 --2- 172.21.15.83:0/3457908607 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a613c6d460 0x55a613c775f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:58.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.798+0000 7f5695c00740 1 --2- 172.21.15.83:0/3457908607 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a613c6cb40 0x55a613c6cf20 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:58.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.798+0000 7f5695c00740 1 -- 172.21.15.83:0/3457908607 >> 172.21.15.83:0/3457908607 conn(0x55a613c68e60 msgr2=0x55a613c69240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:58.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.798+0000 7f5695c00740 1 -- 172.21.15.83:0/3457908607 shutdown_connections 2024-04-24T08:53:58.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.798+0000 7f5695c00740 1 -- 172.21.15.83:0/3457908607 wait complete. 2024-04-24T08:53:58.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.798+0000 7f5695c00740 1 -- start start 2024-04-24T08:53:58.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.799+0000 7f5695c00740 1 Processor -- start 2024-04-24T08:53:58.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.799+0000 7f5695c00740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a613c6cb40 0x55a613c775e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:58.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.799+0000 7f5695c00740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a613c6d460 0x55a613c748d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:58.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.799+0000 7f5695c00740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a613c77b30 0x55a613c75010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:58.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.799+0000 7f5695c00740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a613c7a9c0 con 0x55a613c6cb40 2024-04-24T08:53:58.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.799+0000 7f5695c00740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a613b8b810 con 0x55a613c77b30 2024-04-24T08:53:58.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.799+0000 7f5695c00740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a613c7ab40 con 0x55a613c6d460 2024-04-24T08:53:58.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.799+0000 7f5695c00740 10 client.0 _ll_register_callbacks cb 0x55a613b59140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:53:58.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.799+0000 7f568be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a613c6cb40 0x55a613c775e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:58.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.800+0000 7f568be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a613c6cb40 0x55a613c775e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39490/0 (socket says 172.21.15.83:39490) 2024-04-24T08:53:58.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.800+0000 7f568be00640 1 -- 172.21.15.83:0/3868809877 learned_addr learned my addr 172.21.15.83:0/3868809877 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:53:58.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.800+0000 7f568b400640 1 --2- 172.21.15.83:0/3868809877 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a613c6d460 0x55a613c748d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:58.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.800+0000 7f5690e00640 1 --2- 172.21.15.83:0/3868809877 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a613c77b30 0x55a613c75010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:58.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.800+0000 7f568be00640 1 -- 172.21.15.83:0/3868809877 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a613c6d460 msgr2=0x55a613c748d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:58.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.800+0000 7f568be00640 1 --2- 172.21.15.83:0/3868809877 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a613c6d460 0x55a613c748d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:58.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.800+0000 7f568be00640 1 -- 172.21.15.83:0/3868809877 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a613c77b30 msgr2=0x55a613c75010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:58.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.800+0000 7f568be00640 1 --2- 172.21.15.83:0/3868809877 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a613c77b30 0x55a613c75010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:58.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.800+0000 7f568be00640 1 -- 172.21.15.83:0/3868809877 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a613b9c3f0 con 0x55a613c6cb40 2024-04-24T08:53:58.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.800+0000 7f5690e00640 1 --2- 172.21.15.83:0/3868809877 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a613c77b30 0x55a613c75010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:53:58.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.800+0000 7f568be00640 1 --2- 172.21.15.83:0/3868809877 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a613c6cb40 0x55a613c775e0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f567c011e10 tx=0x7f567c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70896a2e5910389f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:58.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.800+0000 7f5676000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:53:58.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.801+0000 7f5676000640 1 -- 172.21.15.83:0/3868809877 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f567c01e420 con 0x55a613c6cb40 2024-04-24T08:53:58.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.801+0000 7f5676000640 1 -- 172.21.15.83:0/3868809877 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f567c01e5c0 con 0x55a613c6cb40 2024-04-24T08:53:58.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.801+0000 7f5695c00740 1 -- 172.21.15.83:0/3868809877 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f568400e3b0 con 0x55a613c6cb40 2024-04-24T08:53:58.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.801+0000 7f5676000640 1 -- 172.21.15.83:0/3868809877 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f567c01e7c0 con 0x55a613c6cb40 2024-04-24T08:53:58.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.801+0000 7f5695c00740 10 client.16491 fetch_fsmap learned FSMap version 30 2024-04-24T08:53:58.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.801+0000 7f5695c00740 1 -- 172.21.15.83:0/3868809877 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a6138e43f0 con 0x55a613c6cb40 2024-04-24T08:53:58.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.801+0000 7f5676000640 1 -- 172.21.15.83:0/3868809877 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f568400e3b0 con 0x55a613c6cb40 2024-04-24T08:53:58.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.802+0000 7f5676000640 1 -- 172.21.15.83:0/3868809877 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7f567c0268d0 con 0x55a613c6cb40 2024-04-24T08:53:58.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.802+0000 7f5695c00740 10 client.16491 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T08:53:58.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.802+0000 7f5695c00740 10 client.16491 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-24T08:53:58.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.802+0000 7f5695c00740 20 client.16491 populate_metadata read hostname 'smithi083' 2024-04-24T08:53:58.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.802+0000 7f5695c00740 1 --2- 172.21.15.83:0/3868809877 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55a613c87e70 0x55a613c8a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:58.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.802+0000 7f5695c00740 4 client.16491 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:53:58.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.802+0000 7f5695c00740 1 -- 172.21.15.83:0/3868809877 --> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a613bdfbf0 con 0x55a613c87e70 2024-04-24T08:53:58.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.802+0000 7f568b400640 1 --2- 172.21.15.83:0/3868809877 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55a613c87e70 0x55a613c8a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:58.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.803+0000 7f568b400640 1 --2- 172.21.15.83:0/3868809877 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55a613c87e70 0x55a613c8a2c0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f98b17024efec06 server_cookie=109ab575f79428cb in_seq=0 out_seq=0 2024-04-24T08:53:58.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.803+0000 7f5676000640 10 client.16491 ms_handle_connect on v2:172.21.15.189:6838/2548965336 2024-04-24T08:53:58.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.805+0000 7f5676000640 1 -- 172.21.15.83:0/3868809877 <== mds.4 v2:172.21.15.189:6838/2548965336 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a613bdfbf0 con 0x55a613c87e70 2024-04-24T08:53:58.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.805+0000 7f5676000640 10 client.16491 handle_command_reply: tid=0 2024-04-24T08:53:58.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.805+0000 7f5695c00740 1 client.16491 shutdown 2024-04-24T08:53:58.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.805+0000 7f5695c00740 1 -- 172.21.15.83:0/3868809877 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a613c6cb40 msgr2=0x55a613c775e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:58.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.805+0000 7f5695c00740 1 --2- 172.21.15.83:0/3868809877 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a613c6cb40 0x55a613c775e0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f567c011e10 tx=0x7f567c0046c0 comp rx=0 tx=0).stop 2024-04-24T08:53:58.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.806+0000 7f5695c00740 1 -- 172.21.15.83:0/3868809877 shutdown_connections 2024-04-24T08:53:58.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.806+0000 7f5695c00740 1 --2- 172.21.15.83:0/3868809877 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55a613c87e70 0x55a613c8a2c0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:58.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.806+0000 7f5695c00740 1 --2- 172.21.15.83:0/3868809877 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a613c77b30 0x55a613c75010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:58.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.806+0000 7f5695c00740 1 --2- 172.21.15.83:0/3868809877 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a613c6d460 0x55a613c748d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:58.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.806+0000 7f5695c00740 1 --2- 172.21.15.83:0/3868809877 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a613c6cb40 0x55a613c775e0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:58.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.806+0000 7f5695c00740 1 -- 172.21.15.83:0/3868809877 >> 172.21.15.83:0/3868809877 conn(0x55a613c68e60 msgr2=0x55a613c6eb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:58.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.806+0000 7f5676000640 0 client.16491 ms_handle_reset on v2:172.21.15.189:6838/2548965336 2024-04-24T08:53:58.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.806+0000 7f5695c00740 1 -- 172.21.15.83:0/3868809877 shutdown_connections 2024-04-24T08:53:58.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.806+0000 7f5695c00740 1 -- 172.21.15.83:0/3868809877 wait complete. 2024-04-24T08:53:58.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.806+0000 7f5695c00740 20 client.16491 trim_cache size 0 max 16384 2024-04-24T08:53:58.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.821+0000 7f5695c00740 1 Processor -- start 2024-04-24T08:53:58.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.821+0000 7f5695c00740 1 -- start start 2024-04-24T08:53:58.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.821+0000 7f5695c00740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a613c6cb40 0x55a613c741b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:58.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.821+0000 7f5695c00740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a613c6d460 0x55a613c77110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:58.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.821+0000 7f5695c00740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a613c77b30 0x55a613c8fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:58.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.821+0000 7f5695c00740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f568c1b57a0 con 0x55a613c6cb40 2024-04-24T08:53:58.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.821+0000 7f5695c00740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f568c1b7370 con 0x55a613c6d460 2024-04-24T08:53:58.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.821+0000 7f5695c00740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f568c1b5e70 con 0x55a613c77b30 2024-04-24T08:53:58.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.822+0000 7f568be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a613c6cb40 0x55a613c741b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:58.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.822+0000 7f568be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a613c6cb40 0x55a613c741b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39502/0 (socket says 172.21.15.83:39502) 2024-04-24T08:53:58.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.822+0000 7f568be00640 1 -- 172.21.15.83:0/1239246874 learned_addr learned my addr 172.21.15.83:0/1239246874 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:53:58.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.822+0000 7f5690e00640 1 --2- 172.21.15.83:0/1239246874 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a613c77b30 0x55a613c8fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:58.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.822+0000 7f568b400640 1 --2- 172.21.15.83:0/1239246874 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a613c6d460 0x55a613c77110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:58.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.822+0000 7f568be00640 1 -- 172.21.15.83:0/1239246874 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a613c77b30 msgr2=0x55a613c8fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:58.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.822+0000 7f568be00640 1 --2- 172.21.15.83:0/1239246874 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a613c77b30 0x55a613c8fb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:58.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.822+0000 7f568be00640 1 -- 172.21.15.83:0/1239246874 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a613c6d460 msgr2=0x55a613c77110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:58.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.822+0000 7f568be00640 1 --2- 172.21.15.83:0/1239246874 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a613c6d460 0x55a613c77110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:58.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.822+0000 7f568be00640 1 -- 172.21.15.83:0/1239246874 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6139ae7d0 con 0x55a613c6cb40 2024-04-24T08:53:58.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.823+0000 7f568be00640 1 --2- 172.21.15.83:0/1239246874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a613c6cb40 0x55a613c741b0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f567c00cb10 tx=0x7f567c00cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd4dc791059eb8e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:58.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.823+0000 7f5680c00640 1 -- 172.21.15.83:0/1239246874 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f567c039070 con 0x55a613c6cb40 2024-04-24T08:53:58.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.823+0000 7f5680c00640 1 -- 172.21.15.83:0/1239246874 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f567c01ee10 con 0x55a613c6cb40 2024-04-24T08:53:58.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.823+0000 7f5680c00640 1 -- 172.21.15.83:0/1239246874 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f567c005500 con 0x55a613c6cb40 2024-04-24T08:53:58.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.824+0000 7f5695c00740 1 -- 172.21.15.83:0/1239246874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a613c6cb40 msgr2=0x55a613c741b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:58.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.824+0000 7f5695c00740 1 --2- 172.21.15.83:0/1239246874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a613c6cb40 0x55a613c741b0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f567c00cb10 tx=0x7f567c00cb40 comp rx=0 tx=0).stop 2024-04-24T08:53:58.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.824+0000 7f5695c00740 1 -- 172.21.15.83:0/1239246874 shutdown_connections 2024-04-24T08:53:58.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.824+0000 7f5695c00740 1 --2- 172.21.15.83:0/1239246874 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a613c77b30 0x55a613c8fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:58.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.824+0000 7f5695c00740 1 --2- 172.21.15.83:0/1239246874 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a613c6d460 0x55a613c77110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:58.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.824+0000 7f5695c00740 1 --2- 172.21.15.83:0/1239246874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a613c6cb40 0x55a613c741b0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:58.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.824+0000 7f5695c00740 1 -- 172.21.15.83:0/1239246874 >> 172.21.15.83:0/1239246874 conn(0x55a613c68e60 msgr2=0x55a613c89e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:58.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.824+0000 7f5695c00740 1 -- 172.21.15.83:0/1239246874 shutdown_connections 2024-04-24T08:53:58.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.824+0000 7f5695c00740 1 -- 172.21.15.83:0/1239246874 wait complete. 2024-04-24T08:53:58.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.825+0000 7f5695c00740 1 -- start start 2024-04-24T08:53:58.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.825+0000 7f5695c00740 1 Processor -- start 2024-04-24T08:53:58.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.826+0000 7f5695c00740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a613c6cb40 0x55a613c7fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:58.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.826+0000 7f5695c00740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a613c6d460 0x55a613c80330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:58.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.826+0000 7f5695c00740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a613c77b30 0x55a613c71b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:58.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.826+0000 7f5695c00740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f568c1b6c10 con 0x55a613c6d460 2024-04-24T08:53:58.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.826+0000 7f5695c00740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f568c1b6540 con 0x55a613c77b30 2024-04-24T08:53:58.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.826+0000 7f5695c00740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f568c1b3780 con 0x55a613c6cb40 2024-04-24T08:53:58.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.826+0000 7f5695c00740 10 client.0 _ll_register_callbacks cb 0x7f567c00c960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:53:58.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.826+0000 7f568b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a613c6d460 0x55a613c80330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:58.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.826+0000 7f568b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a613c6d460 0x55a613c80330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39518/0 (socket says 172.21.15.83:39518) 2024-04-24T08:53:58.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.826+0000 7f568b400640 1 -- 172.21.15.83:0/4239832403 learned_addr learned my addr 172.21.15.83:0/4239832403 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:53:58.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.826+0000 7f5690e00640 1 --2- 172.21.15.83:0/4239832403 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a613c77b30 0x55a613c71b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:58.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.826+0000 7f568b400640 1 -- 172.21.15.83:0/4239832403 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a613c6cb40 msgr2=0x55a613c7fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:58.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.826+0000 7f568be00640 1 --2- 172.21.15.83:0/4239832403 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a613c6cb40 0x55a613c7fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:58.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.826+0000 7f568b400640 1 --2- 172.21.15.83:0/4239832403 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a613c6cb40 0x55a613c7fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:58.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.826+0000 7f568b400640 1 -- 172.21.15.83:0/4239832403 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a613c77b30 msgr2=0x55a613c71b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:58.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.826+0000 7f568b400640 1 --2- 172.21.15.83:0/4239832403 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a613c77b30 0x55a613c71b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:58.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.826+0000 7f568b400640 1 -- 172.21.15.83:0/4239832403 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a613c72380 con 0x55a613c6d460 2024-04-24T08:53:58.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.826+0000 7f5690e00640 1 --2- 172.21.15.83:0/4239832403 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a613c77b30 0x55a613c71b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:53:58.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.826+0000 7f568be00640 1 --2- 172.21.15.83:0/4239832403 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a613c6cb40 0x55a613c7fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:53:58.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.827+0000 7f568b400640 1 --2- 172.21.15.83:0/4239832403 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a613c6d460 0x55a613c80330 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f56780bcd90 tx=0x7f5678031f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99d911f20cf06f4e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:58.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.827+0000 7f5676000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:53:58.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.827+0000 7f5676000640 1 -- 172.21.15.83:0/4239832403 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5678049040 con 0x55a613c6d460 2024-04-24T08:53:58.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.827+0000 7f5676000640 1 -- 172.21.15.83:0/4239832403 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f56780f4d80 con 0x55a613c6d460 2024-04-24T08:53:58.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.827+0000 7f5695c00740 1 -- 172.21.15.83:0/4239832403 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f567c01ee10 con 0x55a613c6d460 2024-04-24T08:53:58.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.827+0000 7f5676000640 1 -- 172.21.15.83:0/4239832403 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f56780c1440 con 0x55a613c6d460 2024-04-24T08:53:58.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.827+0000 7f5695c00740 10 client.16497 fetch_fsmap learned FSMap version 30 2024-04-24T08:53:58.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.827+0000 7f5695c00740 1 -- 172.21.15.83:0/4239832403 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a613c78bd0 con 0x55a613c6d460 2024-04-24T08:53:58.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.828+0000 7f5676000640 1 -- 172.21.15.83:0/4239832403 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f567c01ee10 con 0x55a613c6d460 2024-04-24T08:53:58.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.828+0000 7f5676000640 1 -- 172.21.15.83:0/4239832403 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7f56780f1150 con 0x55a613c6d460 2024-04-24T08:53:58.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.828+0000 7f5695c00740 10 client.16497 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T08:53:58.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.828+0000 7f5695c00740 10 client.16497 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-24T08:53:58.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.828+0000 7f5695c00740 20 client.16497 populate_metadata read hostname 'smithi083' 2024-04-24T08:53:58.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.828+0000 7f5695c00740 1 --2- 172.21.15.83:0/4239832403 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55a613cee5b0 0x55a613cf09d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:58.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.828+0000 7f5695c00740 4 client.16497 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T08:53:58.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.828+0000 7f5695c00740 1 -- 172.21.15.83:0/4239832403 --> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a613bd9a60 con 0x55a613cee5b0 2024-04-24T08:53:58.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.829+0000 7f568be00640 1 --2- 172.21.15.83:0/4239832403 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55a613cee5b0 0x55a613cf09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:58.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.830+0000 7f5676000640 10 client.16497 ms_handle_connect on v2:172.21.15.189:6838/2548965336 2024-04-24T08:53:58.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.830+0000 7f568be00640 1 --2- 172.21.15.83:0/4239832403 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55a613cee5b0 0x55a613cf09d0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=afee24ddb7b6c22d server_cookie=3950cb25694fa76e in_seq=0 out_seq=0 2024-04-24T08:53:58.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.830+0000 7f5676000640 1 -- 172.21.15.83:0/4239832403 <== mds.4 v2:172.21.15.189:6838/2548965336 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a613bd9a60 con 0x55a613cee5b0 2024-04-24T08:53:58.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.830+0000 7f5676000640 10 client.16497 handle_command_reply: tid=0 2024-04-24T08:53:58.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.830+0000 7f5695c00740 1 client.16497 shutdown 2024-04-24T08:53:58.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.830+0000 7f5695c00740 1 -- 172.21.15.83:0/4239832403 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a613c6d460 msgr2=0x55a613c80330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:58.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.830+0000 7f5695c00740 1 --2- 172.21.15.83:0/4239832403 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a613c6d460 0x55a613c80330 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f56780bcd90 tx=0x7f5678031f10 comp rx=0 tx=0).stop 2024-04-24T08:53:58.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.830+0000 7f5695c00740 1 -- 172.21.15.83:0/4239832403 shutdown_connections 2024-04-24T08:53:58.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.830+0000 7f5695c00740 1 --2- 172.21.15.83:0/4239832403 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55a613cee5b0 0x55a613cf09d0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:58.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.830+0000 7f5695c00740 1 --2- 172.21.15.83:0/4239832403 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a613c77b30 0x55a613c71b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:58.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.830+0000 7f5695c00740 1 --2- 172.21.15.83:0/4239832403 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a613c6d460 0x55a613c80330 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:58.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.830+0000 7f5695c00740 1 --2- 172.21.15.83:0/4239832403 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a613c6cb40 0x55a613c7fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:58.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.830+0000 7f5676000640 0 client.16497 ms_handle_reset on v2:172.21.15.189:6838/2548965336 2024-04-24T08:53:58.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.831+0000 7f5695c00740 1 -- 172.21.15.83:0/4239832403 >> 172.21.15.83:0/4239832403 conn(0x55a613c68e60 msgr2=0x55a613c6fbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:58.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.831+0000 7f5695c00740 1 -- 172.21.15.83:0/4239832403 shutdown_connections 2024-04-24T08:53:58.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.831+0000 7f5695c00740 1 -- 172.21.15.83:0/4239832403 wait complete. 2024-04-24T08:53:58.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.831+0000 7f5695c00740 20 client.16497 trim_cache size 0 max 16384 2024-04-24T08:53:58.834 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T08:53:58.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.832+0000 7f5691800640 1 -- 172.21.15.83:0/3173793376 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f56700a3da0 msgr2=0x7f56700a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:58.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.832+0000 7f5691800640 1 --2- 172.21.15.83:0/3173793376 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f56700a3da0 0x7f56700a6260 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f567c0058c0 tx=0x7f567c005850 comp rx=0 tx=0).stop 2024-04-24T08:53:58.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.832+0000 7f5691800640 1 -- 172.21.15.83:0/3173793376 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f568c108f40 msgr2=0x7f568c1a4d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:58.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.832+0000 7f5691800640 1 --2- 172.21.15.83:0/3173793376 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f568c108f40 0x7f568c1a4d10 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f5678031440 tx=0x7f5678003d50 comp rx=0 tx=0).stop 2024-04-24T08:53:58.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.832+0000 7f5691800640 1 -- 172.21.15.83:0/3173793376 shutdown_connections 2024-04-24T08:53:58.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.832+0000 7f5691800640 1 --2- 172.21.15.83:0/3173793376 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f56700a3da0 0x7f56700a6260 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:58.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.832+0000 7f5691800640 1 --2- 172.21.15.83:0/3173793376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f568c10d500 0x7f568c19e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:58.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.832+0000 7f5691800640 1 --2- 172.21.15.83:0/3173793376 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f568c108f40 0x7f568c1a4d10 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:58.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.832+0000 7f5691800640 1 --2- 172.21.15.83:0/3173793376 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f568c108580 0x7f568c1a47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:58.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.832+0000 7f5691800640 1 -- 172.21.15.83:0/3173793376 >> 172.21.15.83:0/3173793376 conn(0x7f568c077e20 msgr2=0x7f568c0782e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:58.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.832+0000 7f5691800640 1 -- 172.21.15.83:0/3173793376 shutdown_connections 2024-04-24T08:53:58.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:58.832+0000 7f5691800640 1 -- 172.21.15.83:0/3173793376 wait complete. 2024-04-24T08:53:58.846 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:53:58 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/4228035668' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:53:58.850 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-24T08:53:58.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:53:58 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/4228035668' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:53:58.956 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:53:58 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/4228035668' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:53:59.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.053+0000 7f02fa800640 1 -- 172.21.15.83:0/792828619 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f02f4103060 msgr2=0x7f02f4103440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:59.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.053+0000 7f02fa800640 1 --2- 172.21.15.83:0/792828619 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f02f4103060 0x7f02f4103440 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f02e8009900 tx=0x7f02e802a480 comp rx=0 tx=0).stop 2024-04-24T08:53:59.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.053+0000 7f02fa800640 1 -- 172.21.15.83:0/792828619 shutdown_connections 2024-04-24T08:53:59.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.053+0000 7f02fa800640 1 --2- 172.21.15.83:0/792828619 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02f410c480 0x7f02f410e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.053+0000 7f02fa800640 1 --2- 172.21.15.83:0/792828619 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f02f4103980 0x7f02f410be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.053+0000 7f02fa800640 1 --2- 172.21.15.83:0/792828619 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f02f4103060 0x7f02f4103440 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.053+0000 7f02fa800640 1 -- 172.21.15.83:0/792828619 >> 172.21.15.83:0/792828619 conn(0x7f02f40fc880 msgr2=0x7f02f40feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:59.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.053+0000 7f02fa800640 1 -- 172.21.15.83:0/792828619 shutdown_connections 2024-04-24T08:53:59.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.053+0000 7f02fa800640 1 -- 172.21.15.83:0/792828619 wait complete. 2024-04-24T08:53:59.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.054+0000 7f02fa800640 1 Processor -- start 2024-04-24T08:53:59.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.054+0000 7f02fa800640 1 -- start start 2024-04-24T08:53:59.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.054+0000 7f02fa800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02f4103060 0x7f02f41a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:59.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.054+0000 7f02fa800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f02f4103980 0x7f02f41a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:59.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.054+0000 7f02fa800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f02f410c480 0x7f02f419a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:59.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.054+0000 7f02fa800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f02f4112170 con 0x7f02f4103980 2024-04-24T08:53:59.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.054+0000 7f02fa800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f02f4111ff0 con 0x7f02f4103060 2024-04-24T08:53:59.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.054+0000 7f02fa800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f02f41122f0 con 0x7f02f410c480 2024-04-24T08:53:59.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.055+0000 7f02f8a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f02f4103980 0x7f02f41a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:59.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.055+0000 7f02f8a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f02f4103980 0x7f02f41a0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52902/0 (socket says 172.21.15.83:52902) 2024-04-24T08:53:59.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.055+0000 7f02f8a00640 1 -- 172.21.15.83:0/1130572260 learned_addr learned my addr 172.21.15.83:0/1130572260 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:53:59.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.055+0000 7f02f9400640 1 --2- 172.21.15.83:0/1130572260 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02f4103060 0x7f02f41a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:59.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.055+0000 7f02f9e00640 1 --2- 172.21.15.83:0/1130572260 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f02f410c480 0x7f02f419a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:59.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.055+0000 7f02f8a00640 1 -- 172.21.15.83:0/1130572260 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f02f410c480 msgr2=0x7f02f419a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:59.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.055+0000 7f02f8a00640 1 --2- 172.21.15.83:0/1130572260 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f02f410c480 0x7f02f419a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.055+0000 7f02f8a00640 1 -- 172.21.15.83:0/1130572260 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02f4103060 msgr2=0x7f02f41a06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:59.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.055+0000 7f02f8a00640 1 --2- 172.21.15.83:0/1130572260 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02f4103060 0x7f02f41a06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.055+0000 7f02f8a00640 1 -- 172.21.15.83:0/1130572260 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02f419b070 con 0x7f02f4103980 2024-04-24T08:53:59.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.055+0000 7f02f8a00640 1 --2- 172.21.15.83:0/1130572260 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f02f4103980 0x7f02f41a0be0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f02e000b3f0 tx=0x7f02e000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:59.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.056+0000 7f02f2000640 1 -- 172.21.15.83:0/1130572260 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e0015020 con 0x7f02f4103980 2024-04-24T08:53:59.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.056+0000 7f02f2000640 1 -- 172.21.15.83:0/1130572260 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f02e0011ce0 con 0x7f02f4103980 2024-04-24T08:53:59.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.056+0000 7f02f2000640 1 -- 172.21.15.83:0/1130572260 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e0010d80 con 0x7f02f4103980 2024-04-24T08:53:59.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.056+0000 7f02fa800640 1 -- 172.21.15.83:0/1130572260 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02f419b360 con 0x7f02f4103980 2024-04-24T08:53:59.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.056+0000 7f02fa800640 1 -- 172.21.15.83:0/1130572260 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02f41a7620 con 0x7f02f4103980 2024-04-24T08:53:59.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.060+0000 7f02fedbc740 1 Processor -- start 2024-04-24T08:53:59.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.060+0000 7f02fedbc740 1 -- start start 2024-04-24T08:53:59.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.061+0000 7f02f2000640 1 -- 172.21.15.83:0/1130572260 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f02e0018020 con 0x7f02f4103980 2024-04-24T08:53:59.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.061+0000 7f02fedbc740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555a663e2b40 0x555a663e2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:59.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.061+0000 7f02fedbc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a663e3460 0x555a663ed5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:59.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.061+0000 7f02fedbc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a663edb30 0x555a663eff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:59.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.061+0000 7f02fedbc740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x555a663481d0 con 0x555a663edb30 2024-04-24T08:53:59.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.061+0000 7f02fedbc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555a662ee450 con 0x555a663e3460 2024-04-24T08:53:59.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.061+0000 7f02fedbc740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x555a66344190 con 0x555a663e2b40 2024-04-24T08:53:59.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.061+0000 7f02f8a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a663edb30 0x555a663eff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:59.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.061+0000 7f02f8a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a663edb30 0x555a663eff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52906/0 (socket says 172.21.15.83:52906) 2024-04-24T08:53:59.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.061+0000 7f02f8a00640 1 -- 172.21.15.83:0/23731359 learned_addr learned my addr 172.21.15.83:0/23731359 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:53:59.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.061+0000 7f02f9e00640 1 --2- 172.21.15.83:0/23731359 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555a663e2b40 0x555a663e2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:59.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.061+0000 7f02f9400640 1 --2- 172.21.15.83:0/23731359 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a663e3460 0x555a663ed5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:59.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.061+0000 7f02f8a00640 1 -- 172.21.15.83:0/23731359 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555a663e2b40 msgr2=0x555a663e2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:59.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.061+0000 7f02f8a00640 1 --2- 172.21.15.83:0/23731359 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555a663e2b40 0x555a663e2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.062+0000 7f02f8a00640 1 -- 172.21.15.83:0/23731359 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a663e3460 msgr2=0x555a663ed5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:59.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.062+0000 7f02f8a00640 1 --2- 172.21.15.83:0/23731359 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a663e3460 0x555a663ed5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.062+0000 7f02f8a00640 1 -- 172.21.15.83:0/23731359 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a663ab310 con 0x555a663edb30 2024-04-24T08:53:59.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.062+0000 7f02f9400640 1 --2- 172.21.15.83:0/23731359 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a663e3460 0x555a663ed5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:53:59.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.062+0000 7f02f8a00640 1 --2- 172.21.15.83:0/23731359 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a663edb30 0x555a663eff20 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f02e00cd7f0 tx=0x7f02e00d6a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3df1acb6133e1318 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:59.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.062+0000 7f02e6600640 1 -- 172.21.15.83:0/23731359 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e00e3070 con 0x555a663edb30 2024-04-24T08:53:59.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.062+0000 7f02e6600640 1 -- 172.21.15.83:0/23731359 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f02e00d2520 con 0x555a663edb30 2024-04-24T08:53:59.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.062+0000 7f02e6600640 1 -- 172.21.15.83:0/23731359 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e00de9d0 con 0x555a663edb30 2024-04-24T08:53:59.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.062+0000 7f02f2000640 1 --2- 172.21.15.83:0/1130572260 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f02dc0a3d20 0x7f02dc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:59.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.063+0000 7f02f2000640 1 -- 172.21.15.83:0/1130572260 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f02e00d0440 con 0x7f02f4103980 2024-04-24T08:53:59.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.063+0000 7f02f9e00640 1 --2- 172.21.15.83:0/1130572260 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f02dc0a3d20 0x7f02dc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:59.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.063+0000 7f02fedbc740 1 -- 172.21.15.83:0/23731359 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a663edb30 msgr2=0x555a663eff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:59.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.063+0000 7f02fedbc740 1 --2- 172.21.15.83:0/23731359 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a663edb30 0x555a663eff20 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f02e00cd7f0 tx=0x7f02e00d6a50 comp rx=0 tx=0).stop 2024-04-24T08:53:59.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.063+0000 7f02fedbc740 1 -- 172.21.15.83:0/23731359 shutdown_connections 2024-04-24T08:53:59.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.063+0000 7f02fedbc740 1 --2- 172.21.15.83:0/23731359 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a663edb30 0x555a663eff20 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.063+0000 7f02fedbc740 1 --2- 172.21.15.83:0/23731359 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a663e3460 0x555a663ed5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.063+0000 7f02fedbc740 1 --2- 172.21.15.83:0/23731359 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555a663e2b40 0x555a663e2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.063+0000 7f02fedbc740 1 -- 172.21.15.83:0/23731359 >> 172.21.15.83:0/23731359 conn(0x555a663dee60 msgr2=0x555a663df240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:59.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.063+0000 7f02fedbc740 1 -- 172.21.15.83:0/23731359 shutdown_connections 2024-04-24T08:53:59.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.064+0000 7f02f9e00640 1 --2- 172.21.15.83:0/1130572260 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f02dc0a3d20 0x7f02dc0a61e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f02ec008d80 tx=0x7f02ec008370 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:59.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.064+0000 7f02fedbc740 1 -- 172.21.15.83:0/23731359 wait complete. 2024-04-24T08:53:59.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.064+0000 7f02fedbc740 1 -- start start 2024-04-24T08:53:59.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.064+0000 7f02fedbc740 1 Processor -- start 2024-04-24T08:53:59.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.065+0000 7f02fedbc740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555a663e2b40 0x555a663ed560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:59.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.065+0000 7f02fedbc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a663e3460 0x555a663ea850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:59.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.065+0000 7f02fedbc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a663edb30 0x555a663eaf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:59.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.065+0000 7f02fedbc740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x555a663f0840 con 0x555a663edb30 2024-04-24T08:53:59.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.065+0000 7f02fedbc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555a66301810 con 0x555a663e3460 2024-04-24T08:53:59.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.065+0000 7f02fedbc740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x555a663f09c0 con 0x555a663e2b40 2024-04-24T08:53:59.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.065+0000 7f02fedbc740 10 client.0 _ll_register_callbacks cb 0x555a662cf140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:53:59.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.065+0000 7f02f9e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a663edb30 0x555a663eaf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:59.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.065+0000 7f02f9e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a663edb30 0x555a663eaf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52910/0 (socket says 172.21.15.83:52910) 2024-04-24T08:53:59.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.065+0000 7f02f9e00640 1 -- 172.21.15.83:0/3499730635 learned_addr learned my addr 172.21.15.83:0/3499730635 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:53:59.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.065+0000 7f02f8a00640 1 --2- 172.21.15.83:0/3499730635 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a663e3460 0x555a663ea850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:59.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.065+0000 7f02f9400640 1 --2- 172.21.15.83:0/3499730635 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555a663e2b40 0x555a663ed560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:59.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.066+0000 7f02f9e00640 1 -- 172.21.15.83:0/3499730635 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555a663e2b40 msgr2=0x555a663ed560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:59.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.066+0000 7f02f9e00640 1 --2- 172.21.15.83:0/3499730635 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555a663e2b40 0x555a663ed560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.066+0000 7f02f9e00640 1 -- 172.21.15.83:0/3499730635 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a663e3460 msgr2=0x555a663ea850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:59.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.066+0000 7f02f9e00640 1 --2- 172.21.15.83:0/3499730635 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a663e3460 0x555a663ea850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.066+0000 7f02f9e00640 1 -- 172.21.15.83:0/3499730635 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a663123f0 con 0x555a663edb30 2024-04-24T08:53:59.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.066+0000 7f02f9e00640 1 --2- 172.21.15.83:0/3499730635 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a663edb30 0x555a663eaf90 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f02ec011e10 tx=0x7f02ec004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1f9b985bc21f67c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:59.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.066+0000 7f02daa00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:53:59.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.066+0000 7f02daa00640 1 -- 172.21.15.83:0/3499730635 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02ec01e420 con 0x555a663edb30 2024-04-24T08:53:59.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.066+0000 7f02daa00640 1 -- 172.21.15.83:0/3499730635 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f02ec01e5c0 con 0x555a663edb30 2024-04-24T08:53:59.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.066+0000 7f02fedbc740 1 -- 172.21.15.83:0/3499730635 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02e00d2520 con 0x555a663edb30 2024-04-24T08:53:59.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.067+0000 7f02daa00640 1 -- 172.21.15.83:0/3499730635 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f02ec02f050 con 0x555a663edb30 2024-04-24T08:53:59.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.067+0000 7f02fedbc740 10 client.16509 fetch_fsmap learned FSMap version 30 2024-04-24T08:53:59.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.067+0000 7f02fedbc740 1 -- 172.21.15.83:0/3499730635 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555a6605a3f0 con 0x555a663edb30 2024-04-24T08:53:59.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.067+0000 7f02daa00640 1 -- 172.21.15.83:0/3499730635 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f02e00d2520 con 0x555a663edb30 2024-04-24T08:53:59.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.067+0000 7f02daa00640 1 -- 172.21.15.83:0/3499730635 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7f02ec026860 con 0x555a663edb30 2024-04-24T08:53:59.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.067+0000 7f02fedbc740 10 client.16509 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T08:53:59.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.067+0000 7f02fedbc740 10 client.16509 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-24T08:53:59.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.067+0000 7f02fedbc740 20 client.16509 populate_metadata read hostname 'smithi083' 2024-04-24T08:53:59.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.067+0000 7f02fedbc740 1 --2- 172.21.15.83:0/3499730635 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x555a663fdb20 0x555a663fff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:59.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.067+0000 7f02fedbc740 4 client.16509 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:53:59.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.067+0000 7f02fedbc740 1 -- 172.21.15.83:0/3499730635 --> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555a66355bf0 con 0x555a663fdb20 2024-04-24T08:53:59.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.068+0000 7f02f9400640 1 --2- 172.21.15.83:0/3499730635 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x555a663fdb20 0x555a663fff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:59.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.069+0000 7f02f9400640 1 --2- 172.21.15.83:0/3499730635 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x555a663fdb20 0x555a663fff30 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=587e9434da6f57b1 server_cookie=16a99b23d76a9d4e in_seq=0 out_seq=0 2024-04-24T08:53:59.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.069+0000 7f02daa00640 10 client.16509 ms_handle_connect on v2:172.21.15.138:6832/1157967658 2024-04-24T08:53:59.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.071+0000 7f02daa00640 1 -- 172.21.15.83:0/3499730635 <== mds.3 v2:172.21.15.138:6832/1157967658 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555a66355bf0 con 0x555a663fdb20 2024-04-24T08:53:59.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.071+0000 7f02daa00640 10 client.16509 handle_command_reply: tid=0 2024-04-24T08:53:59.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.071+0000 7f02fedbc740 1 client.16509 shutdown 2024-04-24T08:53:59.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.072+0000 7f02fedbc740 1 -- 172.21.15.83:0/3499730635 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a663edb30 msgr2=0x555a663eaf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:59.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.072+0000 7f02fedbc740 1 --2- 172.21.15.83:0/3499730635 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a663edb30 0x555a663eaf90 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f02ec011e10 tx=0x7f02ec004a40 comp rx=0 tx=0).stop 2024-04-24T08:53:59.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.072+0000 7f02fedbc740 1 -- 172.21.15.83:0/3499730635 shutdown_connections 2024-04-24T08:53:59.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.072+0000 7f02fedbc740 1 --2- 172.21.15.83:0/3499730635 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a663edb30 0x555a663eaf90 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.072+0000 7f02fedbc740 1 --2- 172.21.15.83:0/3499730635 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a663e3460 0x555a663ea850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.072+0000 7f02fedbc740 1 --2- 172.21.15.83:0/3499730635 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x555a663fdb20 0x555a663fff30 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.072+0000 7f02fedbc740 1 --2- 172.21.15.83:0/3499730635 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555a663e2b40 0x555a663ed560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.072+0000 7f02fedbc740 1 -- 172.21.15.83:0/3499730635 >> 172.21.15.83:0/3499730635 conn(0x555a663dee60 msgr2=0x555a663e4c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:59.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.072+0000 7f02daa00640 0 client.16509 ms_handle_reset on v2:172.21.15.138:6832/1157967658 2024-04-24T08:53:59.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.072+0000 7f02fedbc740 1 -- 172.21.15.83:0/3499730635 shutdown_connections 2024-04-24T08:53:59.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.072+0000 7f02fedbc740 1 -- 172.21.15.83:0/3499730635 wait complete. 2024-04-24T08:53:59.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.072+0000 7f02fedbc740 20 client.16509 trim_cache size 0 max 16384 2024-04-24T08:53:59.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.089+0000 7f02fedbc740 1 Processor -- start 2024-04-24T08:53:59.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.089+0000 7f02fedbc740 1 -- start start 2024-04-24T08:53:59.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.089+0000 7f02fedbc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a663e2b40 0x555a663ea7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:59.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.089+0000 7f02fedbc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a663e3460 0x555a663f9700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:59.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.089+0000 7f02fedbc740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555a663edb30 0x555a663ed090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:59.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.089+0000 7f02fedbc740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f02f41b15b0 con 0x555a663e2b40 2024-04-24T08:53:59.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.089+0000 7f02fedbc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f02f41b3180 con 0x555a663e3460 2024-04-24T08:53:59.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.089+0000 7f02fedbc740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f02f41b1c80 con 0x555a663edb30 2024-04-24T08:53:59.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.090+0000 7f02f9400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a663e2b40 0x555a663ea7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:59.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.090+0000 7f02f9400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a663e2b40 0x555a663ea7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52912/0 (socket says 172.21.15.83:52912) 2024-04-24T08:53:59.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.090+0000 7f02f9400640 1 -- 172.21.15.83:0/2276684961 learned_addr learned my addr 172.21.15.83:0/2276684961 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:53:59.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.090+0000 7f02f8a00640 1 --2- 172.21.15.83:0/2276684961 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a663e3460 0x555a663f9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:59.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.090+0000 7f02f9400640 1 -- 172.21.15.83:0/2276684961 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555a663edb30 msgr2=0x555a663ed090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:59.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.090+0000 7f02f9400640 1 --2- 172.21.15.83:0/2276684961 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555a663edb30 0x555a663ed090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.090+0000 7f02f9400640 1 -- 172.21.15.83:0/2276684961 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a663e3460 msgr2=0x555a663f9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:59.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.090+0000 7f02f9400640 1 --2- 172.21.15.83:0/2276684961 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a663e3460 0x555a663f9700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.090+0000 7f02f9400640 1 -- 172.21.15.83:0/2276684961 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a663e3290 con 0x555a663e2b40 2024-04-24T08:53:59.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.090+0000 7f02f9400640 1 --2- 172.21.15.83:0/2276684961 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a663e2b40 0x555a663ea7f0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f02e8005ab0 tx=0x7f02e8005e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dc50e91316c73d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:59.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.091+0000 7f02e5c00640 1 -- 172.21.15.83:0/2276684961 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e8035040 con 0x555a663e2b40 2024-04-24T08:53:59.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.091+0000 7f02e5c00640 1 -- 172.21.15.83:0/2276684961 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f02e8031e50 con 0x555a663e2b40 2024-04-24T08:53:59.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.091+0000 7f02e5c00640 1 -- 172.21.15.83:0/2276684961 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e8005280 con 0x555a663e2b40 2024-04-24T08:53:59.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.092+0000 7f02fedbc740 1 -- 172.21.15.83:0/2276684961 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a663e2b40 msgr2=0x555a663ea7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:59.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.092+0000 7f02fedbc740 1 --2- 172.21.15.83:0/2276684961 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a663e2b40 0x555a663ea7f0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f02e8005ab0 tx=0x7f02e8005e00 comp rx=0 tx=0).stop 2024-04-24T08:53:59.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.092+0000 7f02fedbc740 1 -- 172.21.15.83:0/2276684961 shutdown_connections 2024-04-24T08:53:59.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.092+0000 7f02fedbc740 1 --2- 172.21.15.83:0/2276684961 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555a663edb30 0x555a663ed090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.092+0000 7f02fedbc740 1 --2- 172.21.15.83:0/2276684961 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a663e3460 0x555a663f9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.092+0000 7f02fedbc740 1 --2- 172.21.15.83:0/2276684961 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a663e2b40 0x555a663ea7f0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.092+0000 7f02fedbc740 1 -- 172.21.15.83:0/2276684961 >> 172.21.15.83:0/2276684961 conn(0x555a663dee60 msgr2=0x555a663ea0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:59.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.092+0000 7f02fedbc740 1 -- 172.21.15.83:0/2276684961 shutdown_connections 2024-04-24T08:53:59.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.092+0000 7f02fedbc740 1 -- 172.21.15.83:0/2276684961 wait complete. 2024-04-24T08:53:59.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.092+0000 7f02fedbc740 1 -- start start 2024-04-24T08:53:59.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.092+0000 7f02fedbc740 1 Processor -- start 2024-04-24T08:53:59.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.093+0000 7f02fedbc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a663e2b40 0x555a663eea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:59.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.093+0000 7f02fedbc740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555a663e3460 0x555a663eefc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:59.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.093+0000 7f02fedbc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a663edb30 0x555a663ef500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:59.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.093+0000 7f02fedbc740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f02f41b2a20 con 0x555a663edb30 2024-04-24T08:53:59.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.093+0000 7f02fedbc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f02f41b2350 con 0x555a663e2b40 2024-04-24T08:53:59.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.093+0000 7f02fedbc740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f02f41af590 con 0x555a663e3460 2024-04-24T08:53:59.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.093+0000 7f02fedbc740 10 client.0 _ll_register_callbacks cb 0x7f02ec008990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:53:59.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.093+0000 7f02f9e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a663edb30 0x555a663ef500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:59.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.093+0000 7f02f9e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a663edb30 0x555a663ef500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52916/0 (socket says 172.21.15.83:52916) 2024-04-24T08:53:59.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.093+0000 7f02f9e00640 1 -- 172.21.15.83:0/2038540987 learned_addr learned my addr 172.21.15.83:0/2038540987 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:53:59.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.093+0000 7f02f9400640 1 --2- 172.21.15.83:0/2038540987 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a663e2b40 0x555a663eea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:59.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.093+0000 7f02f8a00640 1 --2- 172.21.15.83:0/2038540987 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555a663e3460 0x555a663eefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:59.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.093+0000 7f02f9e00640 1 -- 172.21.15.83:0/2038540987 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555a663e3460 msgr2=0x555a663eefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:59.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.093+0000 7f02f9e00640 1 --2- 172.21.15.83:0/2038540987 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555a663e3460 0x555a663eefc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.094+0000 7f02f9e00640 1 -- 172.21.15.83:0/2038540987 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a663e2b40 msgr2=0x555a663eea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:59.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.094+0000 7f02f9e00640 1 --2- 172.21.15.83:0/2038540987 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a663e2b40 0x555a663eea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.094+0000 7f02f9e00640 1 -- 172.21.15.83:0/2038540987 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a661247d0 con 0x555a663edb30 2024-04-24T08:53:59.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.094+0000 7f02f9e00640 1 --2- 172.21.15.83:0/2038540987 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a663edb30 0x555a663ef500 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f02ec005e10 tx=0x7f02ec008a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb4ca047fa2e8dd6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:59.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.094+0000 7f02daa00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:53:59.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.094+0000 7f02daa00640 1 -- 172.21.15.83:0/2038540987 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02ec026400 con 0x555a663edb30 2024-04-24T08:53:59.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.094+0000 7f02daa00640 1 -- 172.21.15.83:0/2038540987 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f02ec0265a0 con 0x555a663edb30 2024-04-24T08:53:59.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.094+0000 7f02fedbc740 1 -- 172.21.15.83:0/2038540987 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02e8031e50 con 0x555a663edb30 2024-04-24T08:53:59.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.094+0000 7f02daa00640 1 -- 172.21.15.83:0/2038540987 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f02ec0267a0 con 0x555a663edb30 2024-04-24T08:53:59.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.094+0000 7f02fedbc740 10 client.16515 fetch_fsmap learned FSMap version 30 2024-04-24T08:53:59.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.094+0000 7f02fedbc740 1 -- 172.21.15.83:0/2038540987 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555a66464310 con 0x555a663edb30 2024-04-24T08:53:59.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.095+0000 7f02daa00640 1 -- 172.21.15.83:0/2038540987 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f02e8031e50 con 0x555a663edb30 2024-04-24T08:53:59.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.095+0000 7f02daa00640 1 -- 172.21.15.83:0/2038540987 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7f02ec03c420 con 0x555a663edb30 2024-04-24T08:53:59.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.095+0000 7f02fedbc740 10 client.16515 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T08:53:59.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.095+0000 7f02fedbc740 10 client.16515 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-24T08:53:59.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.095+0000 7f02fedbc740 20 client.16515 populate_metadata read hostname 'smithi083' 2024-04-24T08:53:59.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.095+0000 7f02fedbc740 1 --2- 172.21.15.83:0/2038540987 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x555a66464a90 0x555a66466eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:59.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.095+0000 7f02fedbc740 4 client.16515 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T08:53:59.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.095+0000 7f02fedbc740 1 -- 172.21.15.83:0/2038540987 --> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555a6634fa60 con 0x555a66464a90 2024-04-24T08:53:59.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.096+0000 7f02f9400640 1 --2- 172.21.15.83:0/2038540987 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x555a66464a90 0x555a66466eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:59.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.097+0000 7f02f9400640 1 --2- 172.21.15.83:0/2038540987 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x555a66464a90 0x555a66466eb0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ca90ff42c08c72b server_cookie=f01747d7d08214d9 in_seq=0 out_seq=0 2024-04-24T08:53:59.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.097+0000 7f02daa00640 10 client.16515 ms_handle_connect on v2:172.21.15.138:6832/1157967658 2024-04-24T08:53:59.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.097+0000 7f02daa00640 1 -- 172.21.15.83:0/2038540987 <== mds.3 v2:172.21.15.138:6832/1157967658 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555a6634fa60 con 0x555a66464a90 2024-04-24T08:53:59.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.097+0000 7f02daa00640 10 client.16515 handle_command_reply: tid=0 2024-04-24T08:53:59.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.097+0000 7f02fedbc740 1 client.16515 shutdown 2024-04-24T08:53:59.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.097+0000 7f02fedbc740 1 -- 172.21.15.83:0/2038540987 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a663edb30 msgr2=0x555a663ef500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:59.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.097+0000 7f02fedbc740 1 --2- 172.21.15.83:0/2038540987 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a663edb30 0x555a663ef500 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f02ec005e10 tx=0x7f02ec008a70 comp rx=0 tx=0).stop 2024-04-24T08:53:59.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.097+0000 7f02fedbc740 1 -- 172.21.15.83:0/2038540987 shutdown_connections 2024-04-24T08:53:59.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.097+0000 7f02fedbc740 1 --2- 172.21.15.83:0/2038540987 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a663edb30 0x555a663ef500 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.097+0000 7f02fedbc740 1 --2- 172.21.15.83:0/2038540987 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x555a66464a90 0x555a66466eb0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.097+0000 7f02fedbc740 1 --2- 172.21.15.83:0/2038540987 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555a663e3460 0x555a663eefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.097+0000 7f02fedbc740 1 --2- 172.21.15.83:0/2038540987 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a663e2b40 0x555a663eea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.097+0000 7f02daa00640 0 client.16515 ms_handle_reset on v2:172.21.15.138:6832/1157967658 2024-04-24T08:53:59.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.097+0000 7f02fedbc740 1 -- 172.21.15.83:0/2038540987 >> 172.21.15.83:0/2038540987 conn(0x555a663dee60 msgr2=0x555a663dd6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:59.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.098+0000 7f02fedbc740 1 -- 172.21.15.83:0/2038540987 shutdown_connections 2024-04-24T08:53:59.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.098+0000 7f02fedbc740 1 -- 172.21.15.83:0/2038540987 wait complete. 2024-04-24T08:53:59.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.098+0000 7f02fedbc740 20 client.16515 trim_cache size 0 max 16384 2024-04-24T08:53:59.103 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T08:53:59.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.098+0000 7f02fa800640 1 -- 172.21.15.83:0/1130572260 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f02dc0a3d20 msgr2=0x7f02dc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:59.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.098+0000 7f02fa800640 1 --2- 172.21.15.83:0/1130572260 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f02dc0a3d20 0x7f02dc0a61e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f02ec008d80 tx=0x7f02ec008370 comp rx=0 tx=0).stop 2024-04-24T08:53:59.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.098+0000 7f02fa800640 1 -- 172.21.15.83:0/1130572260 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f02f4103980 msgr2=0x7f02f41a0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:59.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.098+0000 7f02fa800640 1 --2- 172.21.15.83:0/1130572260 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f02f4103980 0x7f02f41a0be0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f02e000b3f0 tx=0x7f02e000b8b0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.098+0000 7f02fa800640 1 -- 172.21.15.83:0/1130572260 shutdown_connections 2024-04-24T08:53:59.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.099+0000 7f02fa800640 1 --2- 172.21.15.83:0/1130572260 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f02dc0a3d20 0x7f02dc0a61e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.099+0000 7f02fa800640 1 --2- 172.21.15.83:0/1130572260 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f02f410c480 0x7f02f419a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.099+0000 7f02fa800640 1 --2- 172.21.15.83:0/1130572260 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f02f4103980 0x7f02f41a0be0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.099+0000 7f02fa800640 1 --2- 172.21.15.83:0/1130572260 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02f4103060 0x7f02f41a06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.099+0000 7f02fa800640 1 -- 172.21.15.83:0/1130572260 >> 172.21.15.83:0/1130572260 conn(0x7f02f40fc880 msgr2=0x7f02f4101d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:59.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.099+0000 7f02fa800640 1 -- 172.21.15.83:0/1130572260 shutdown_connections 2024-04-24T08:53:59.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.099+0000 7f02fa800640 1 -- 172.21.15.83:0/1130572260 wait complete. 2024-04-24T08:53:59.116 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T08:53:59.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.318+0000 7f1443200640 1 -- 172.21.15.83:0/1831865261 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f143c108610 msgr2=0x7f143c108a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:59.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.318+0000 7f1443200640 1 --2- 172.21.15.83:0/1831865261 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f143c108610 0x7f143c108a90 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f1424009930 tx=0x7f142402a420 comp rx=0 tx=0).stop 2024-04-24T08:53:59.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.318+0000 7f1443200640 1 -- 172.21.15.83:0/1831865261 shutdown_connections 2024-04-24T08:53:59.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.318+0000 7f1443200640 1 --2- 172.21.15.83:0/1831865261 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f143c108fd0 0x7f143c115490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.318+0000 7f1443200640 1 --2- 172.21.15.83:0/1831865261 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f143c108610 0x7f143c108a90 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.318+0000 7f1443200640 1 --2- 172.21.15.83:0/1831865261 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f143c10e660 0x7f143c10ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.318+0000 7f1443200640 1 -- 172.21.15.83:0/1831865261 >> 172.21.15.83:0/1831865261 conn(0x7f143c076540 msgr2=0x7f143c078960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:59.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.318+0000 7f1443200640 1 -- 172.21.15.83:0/1831865261 shutdown_connections 2024-04-24T08:53:59.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.318+0000 7f1443200640 1 -- 172.21.15.83:0/1831865261 wait complete. 2024-04-24T08:53:59.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.319+0000 7f1443200640 1 Processor -- start 2024-04-24T08:53:59.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.319+0000 7f1443200640 1 -- start start 2024-04-24T08:53:59.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.320+0000 7f1443200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f143c108610 0x7f143c19efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:59.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.320+0000 7f1443200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f143c108fd0 0x7f143c19f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:59.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.320+0000 7f1443200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f143c10e660 0x7f143c1a3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:59.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.320+0000 7f1443200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f143c118980 con 0x7f143c108fd0 2024-04-24T08:53:59.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.320+0000 7f1443200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f143c118800 con 0x7f143c10e660 2024-04-24T08:53:59.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.320+0000 7f1443200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f143c118b00 con 0x7f143c108610 2024-04-24T08:53:59.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.320+0000 7f1433400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f143c108fd0 0x7f143c19f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:59.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.320+0000 7f1433400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f143c108fd0 0x7f143c19f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52928/0 (socket says 172.21.15.83:52928) 2024-04-24T08:53:59.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.320+0000 7f1433400640 1 -- 172.21.15.83:0/3101256500 learned_addr learned my addr 172.21.15.83:0/3101256500 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:53:59.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.320+0000 7f1433400640 1 -- 172.21.15.83:0/3101256500 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f143c108610 msgr2=0x7f143c19efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:59.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.320+0000 7f143be00640 1 --2- 172.21.15.83:0/3101256500 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f143c108610 0x7f143c19efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:59.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.320+0000 7f1441200640 1 --2- 172.21.15.83:0/3101256500 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f143c10e660 0x7f143c1a3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:59.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.321+0000 7f1433400640 1 --2- 172.21.15.83:0/3101256500 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f143c108610 0x7f143c19efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.321+0000 7f1433400640 1 -- 172.21.15.83:0/3101256500 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f143c10e660 msgr2=0x7f143c1a3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:59.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.321+0000 7f1433400640 1 --2- 172.21.15.83:0/3101256500 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f143c10e660 0x7f143c1a3860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.321+0000 7f1433400640 1 -- 172.21.15.83:0/3101256500 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f143c1a3f90 con 0x7f143c108fd0 2024-04-24T08:53:59.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.321+0000 7f143be00640 1 --2- 172.21.15.83:0/3101256500 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f143c108610 0x7f143c19efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:53:59.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.321+0000 7f1441200640 1 --2- 172.21.15.83:0/3101256500 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f143c10e660 0x7f143c1a3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:53:59.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.321+0000 7f1433400640 1 --2- 172.21.15.83:0/3101256500 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f143c108fd0 0x7f143c19f4e0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f1424031440 tx=0x7f1424003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:59.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.321+0000 7f1439600640 1 -- 172.21.15.83:0/3101256500 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f142403f070 con 0x7f143c108fd0 2024-04-24T08:53:59.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.321+0000 7f1439600640 1 -- 172.21.15.83:0/3101256500 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1424031d60 con 0x7f143c108fd0 2024-04-24T08:53:59.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.321+0000 7f1439600640 1 -- 172.21.15.83:0/3101256500 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1424035030 con 0x7f143c108fd0 2024-04-24T08:53:59.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.321+0000 7f1443200640 1 -- 172.21.15.83:0/3101256500 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f143c1a4220 con 0x7f143c108fd0 2024-04-24T08:53:59.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.322+0000 7f1443200640 1 -- 172.21.15.83:0/3101256500 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f143c1abb10 con 0x7f143c108fd0 2024-04-24T08:53:59.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.325+0000 7f1439600640 1 -- 172.21.15.83:0/3101256500 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1424042070 con 0x7f143c108fd0 2024-04-24T08:53:59.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.325+0000 7f1446022740 1 Processor -- start 2024-04-24T08:53:59.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.325+0000 7f1446022740 1 -- start start 2024-04-24T08:53:59.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.325+0000 7f1446022740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563aeef3c950 0x563aeef3cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:59.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.325+0000 7f1446022740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563aeef3d270 0x563aeef47400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:59.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.325+0000 7f1446022740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563aeef47940 0x563aeef49d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:59.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.325+0000 7f1446022740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x563aeeea21d0 con 0x563aeef3c950 2024-04-24T08:53:59.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.325+0000 7f1446022740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563aeee48450 con 0x563aeef3d270 2024-04-24T08:53:59.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.325+0000 7f1446022740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x563aeee9e190 con 0x563aeef47940 2024-04-24T08:53:59.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.326+0000 7f1441200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563aeef3c950 0x563aeef3cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:59.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.326+0000 7f1441200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563aeef3c950 0x563aeef3cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52942/0 (socket says 172.21.15.83:52942) 2024-04-24T08:53:59.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.326+0000 7f1441200640 1 -- 172.21.15.83:0/3402306318 learned_addr learned my addr 172.21.15.83:0/3402306318 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:53:59.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.326+0000 7f1433400640 1 --2- 172.21.15.83:0/3402306318 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563aeef47940 0x563aeef49d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:59.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.326+0000 7f1441200640 1 -- 172.21.15.83:0/3402306318 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563aeef47940 msgr2=0x563aeef49d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:59.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.326+0000 7f1441200640 1 --2- 172.21.15.83:0/3402306318 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563aeef47940 0x563aeef49d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.326+0000 7f1441200640 1 -- 172.21.15.83:0/3402306318 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563aeef3d270 msgr2=0x563aeef47400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:59.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.326+0000 7f1441200640 1 --2- 172.21.15.83:0/3402306318 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563aeef3d270 0x563aeef47400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.326+0000 7f1441200640 1 -- 172.21.15.83:0/3402306318 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563aeef05310 con 0x563aeef3c950 2024-04-24T08:53:59.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.326+0000 7f1441200640 1 --2- 172.21.15.83:0/3402306318 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563aeef3c950 0x563aeef3cd30 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f143400a990 tx=0x7f143400ae50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6ccd8c8ca289fdd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:59.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.327+0000 7f1439600640 1 --2- 172.21.15.83:0/3101256500 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f14200a3da0 0x7f14200a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:59.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.327+0000 7f1431600640 1 -- 172.21.15.83:0/3402306318 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1434015070 con 0x563aeef3c950 2024-04-24T08:53:59.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.327+0000 7f1431600640 1 -- 172.21.15.83:0/3402306318 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f143400cbe0 con 0x563aeef3c950 2024-04-24T08:53:59.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.327+0000 7f1431600640 1 -- 172.21.15.83:0/3402306318 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14340105b0 con 0x563aeef3c950 2024-04-24T08:53:59.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.327+0000 7f1439600640 1 -- 172.21.15.83:0/3101256500 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f14240f1500 con 0x7f143c108fd0 2024-04-24T08:53:59.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.327+0000 7f143be00640 1 --2- 172.21.15.83:0/3101256500 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f14200a3da0 0x7f14200a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:59.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.328+0000 7f1446022740 1 -- 172.21.15.83:0/3402306318 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563aeef3c950 msgr2=0x563aeef3cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:59.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.328+0000 7f1446022740 1 --2- 172.21.15.83:0/3402306318 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563aeef3c950 0x563aeef3cd30 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f143400a990 tx=0x7f143400ae50 comp rx=0 tx=0).stop 2024-04-24T08:53:59.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.328+0000 7f143be00640 1 --2- 172.21.15.83:0/3101256500 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f14200a3da0 0x7f14200a6260 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x563aeef3df10 tx=0x7f142c008350 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:59.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.328+0000 7f1446022740 1 -- 172.21.15.83:0/3402306318 shutdown_connections 2024-04-24T08:53:59.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.328+0000 7f1446022740 1 --2- 172.21.15.83:0/3402306318 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563aeef47940 0x563aeef49d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.328+0000 7f1446022740 1 --2- 172.21.15.83:0/3402306318 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563aeef3d270 0x563aeef47400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.328+0000 7f1446022740 1 --2- 172.21.15.83:0/3402306318 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563aeef3c950 0x563aeef3cd30 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.328+0000 7f1446022740 1 -- 172.21.15.83:0/3402306318 >> 172.21.15.83:0/3402306318 conn(0x563aeef38c70 msgr2=0x563aeef39050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:59.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.328+0000 7f1446022740 1 -- 172.21.15.83:0/3402306318 shutdown_connections 2024-04-24T08:53:59.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.328+0000 7f1446022740 1 -- 172.21.15.83:0/3402306318 wait complete. 2024-04-24T08:53:59.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.329+0000 7f1446022740 1 -- start start 2024-04-24T08:53:59.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.329+0000 7f1446022740 1 Processor -- start 2024-04-24T08:53:59.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.329+0000 7f1446022740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563aeef3c950 0x563aeef47370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:59.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.329+0000 7f1446022740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563aeef3d270 0x563aeef44660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:59.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.329+0000 7f1446022740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563aeef47940 0x563aeef44da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:59.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.329+0000 7f1446022740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x563aeef4a7d0 con 0x563aeef3c950 2024-04-24T08:53:59.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.329+0000 7f1446022740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563aeee5b810 con 0x563aeef3d270 2024-04-24T08:53:59.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.329+0000 7f1446022740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x563aeef4a950 con 0x563aeef47940 2024-04-24T08:53:59.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.329+0000 7f1446022740 10 client.0 _ll_register_callbacks cb 0x563aeee29140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:53:59.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.330+0000 7f143be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563aeef3c950 0x563aeef47370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:59.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.330+0000 7f143be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563aeef3c950 0x563aeef47370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52950/0 (socket says 172.21.15.83:52950) 2024-04-24T08:53:59.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.330+0000 7f143be00640 1 -- 172.21.15.83:0/230589362 learned_addr learned my addr 172.21.15.83:0/230589362 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:53:59.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.330+0000 7f1441200640 1 --2- 172.21.15.83:0/230589362 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563aeef47940 0x563aeef44da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:59.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.330+0000 7f1433400640 1 --2- 172.21.15.83:0/230589362 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563aeef3d270 0x563aeef44660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:59.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.330+0000 7f143be00640 1 -- 172.21.15.83:0/230589362 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563aeef47940 msgr2=0x563aeef44da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:59.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.330+0000 7f143be00640 1 --2- 172.21.15.83:0/230589362 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563aeef47940 0x563aeef44da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.330+0000 7f143be00640 1 -- 172.21.15.83:0/230589362 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563aeef3d270 msgr2=0x563aeef44660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:59.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.330+0000 7f143be00640 1 --2- 172.21.15.83:0/230589362 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563aeef3d270 0x563aeef44660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.330+0000 7f143be00640 1 -- 172.21.15.83:0/230589362 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563aeee6c3f0 con 0x563aeef3c950 2024-04-24T08:53:59.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.330+0000 7f143be00640 1 --2- 172.21.15.83:0/230589362 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563aeef3c950 0x563aeef47370 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f142c010e50 tx=0x7f142c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d8397431e720669 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:59.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.331+0000 7f142a000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:53:59.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.332+0000 7f142a000640 1 -- 172.21.15.83:0/230589362 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f142c021020 con 0x563aeef3c950 2024-04-24T08:53:59.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.332+0000 7f142a000640 1 -- 172.21.15.83:0/230589362 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f142c00fd40 con 0x563aeef3c950 2024-04-24T08:53:59.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.332+0000 7f1446022740 1 -- 172.21.15.83:0/230589362 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f143400cbe0 con 0x563aeef3c950 2024-04-24T08:53:59.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.332+0000 7f142a000640 1 -- 172.21.15.83:0/230589362 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f142c02c050 con 0x563aeef3c950 2024-04-24T08:53:59.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.332+0000 7f1446022740 10 client.16527 fetch_fsmap learned FSMap version 30 2024-04-24T08:53:59.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.332+0000 7f1446022740 1 -- 172.21.15.83:0/230589362 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563aeebb43f0 con 0x563aeef3c950 2024-04-24T08:53:59.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.332+0000 7f142a000640 1 -- 172.21.15.83:0/230589362 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f143400cbe0 con 0x563aeef3c950 2024-04-24T08:53:59.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.333+0000 7f142a000640 1 -- 172.21.15.83:0/230589362 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7f142c01c3a0 con 0x563aeef3c950 2024-04-24T08:53:59.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.333+0000 7f1446022740 10 client.16527 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T08:53:59.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.333+0000 7f1446022740 10 client.16527 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-24T08:53:59.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.333+0000 7f1446022740 20 client.16527 populate_metadata read hostname 'smithi083' 2024-04-24T08:53:59.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.333+0000 7f1446022740 1 --2- 172.21.15.83:0/230589362 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x563aeef57b90 0x563aeef5a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:59.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.333+0000 7f1446022740 4 client.16527 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:53:59.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.333+0000 7f1446022740 1 -- 172.21.15.83:0/230589362 --> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563aeeeafbf0 con 0x563aeef57b90 2024-04-24T08:53:59.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.334+0000 7f1433400640 1 --2- 172.21.15.83:0/230589362 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x563aeef57b90 0x563aeef5a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:59.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.334+0000 7f1433400640 1 --2- 172.21.15.83:0/230589362 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x563aeef57b90 0x563aeef5a000 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c3b1efa09f11bf1e server_cookie=3e7fd51f7b037d03 in_seq=0 out_seq=0 2024-04-24T08:53:59.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.334+0000 7f142a000640 10 client.16527 ms_handle_connect on v2:172.21.15.138:6834/1909219758 2024-04-24T08:53:59.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.336+0000 7f142a000640 1 -- 172.21.15.83:0/230589362 <== mds.2 v2:172.21.15.138:6834/1909219758 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563aeeeafbf0 con 0x563aeef57b90 2024-04-24T08:53:59.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.336+0000 7f142a000640 10 client.16527 handle_command_reply: tid=0 2024-04-24T08:53:59.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.336+0000 7f1446022740 1 client.16527 shutdown 2024-04-24T08:53:59.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.336+0000 7f1446022740 1 -- 172.21.15.83:0/230589362 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563aeef3c950 msgr2=0x563aeef47370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:59.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.336+0000 7f1446022740 1 --2- 172.21.15.83:0/230589362 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563aeef3c950 0x563aeef47370 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f142c010e50 tx=0x7f142c004a40 comp rx=0 tx=0).stop 2024-04-24T08:53:59.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.337+0000 7f1446022740 1 -- 172.21.15.83:0/230589362 shutdown_connections 2024-04-24T08:53:59.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.337+0000 7f1446022740 1 --2- 172.21.15.83:0/230589362 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x563aeef57b90 0x563aeef5a000 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.337+0000 7f1446022740 1 --2- 172.21.15.83:0/230589362 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563aeef47940 0x563aeef44da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.337+0000 7f1446022740 1 --2- 172.21.15.83:0/230589362 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563aeef3d270 0x563aeef44660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.337+0000 7f1446022740 1 --2- 172.21.15.83:0/230589362 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563aeef3c950 0x563aeef47370 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.337+0000 7f142a000640 0 client.16527 ms_handle_reset on v2:172.21.15.138:6834/1909219758 2024-04-24T08:53:59.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.337+0000 7f1446022740 1 -- 172.21.15.83:0/230589362 >> 172.21.15.83:0/230589362 conn(0x563aeef38c70 msgr2=0x563aeef3e970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:59.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.337+0000 7f1446022740 1 -- 172.21.15.83:0/230589362 shutdown_connections 2024-04-24T08:53:59.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.337+0000 7f1446022740 1 -- 172.21.15.83:0/230589362 wait complete. 2024-04-24T08:53:59.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.337+0000 7f1446022740 20 client.16527 trim_cache size 0 max 16384 2024-04-24T08:53:59.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.352+0000 7f1446022740 1 Processor -- start 2024-04-24T08:53:59.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.352+0000 7f1446022740 1 -- start start 2024-04-24T08:53:59.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.352+0000 7f1446022740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563aeef3c950 0x563aeef37c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:59.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.352+0000 7f1446022740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563aeef3d270 0x563aeef46ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:59.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.352+0000 7f1446022740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563aeef47940 0x563aeef473e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:59.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.352+0000 7f1446022740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f143c1b5bf0 con 0x563aeef47940 2024-04-24T08:53:59.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.352+0000 7f1446022740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f143c1b77c0 con 0x563aeef3c950 2024-04-24T08:53:59.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.352+0000 7f1446022740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f143c1b62c0 con 0x563aeef3d270 2024-04-24T08:53:59.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.353+0000 7f1441200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563aeef47940 0x563aeef473e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:59.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.353+0000 7f1441200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563aeef47940 0x563aeef473e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52954/0 (socket says 172.21.15.83:52954) 2024-04-24T08:53:59.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.353+0000 7f1441200640 1 -- 172.21.15.83:0/4103053794 learned_addr learned my addr 172.21.15.83:0/4103053794 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:53:59.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.353+0000 7f1433400640 1 --2- 172.21.15.83:0/4103053794 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563aeef3d270 0x563aeef46ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:59.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.353+0000 7f143be00640 1 --2- 172.21.15.83:0/4103053794 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563aeef3c950 0x563aeef37c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:59.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.353+0000 7f1441200640 1 -- 172.21.15.83:0/4103053794 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563aeef3d270 msgr2=0x563aeef46ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:59.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.353+0000 7f1441200640 1 --2- 172.21.15.83:0/4103053794 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563aeef3d270 0x563aeef46ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.353+0000 7f1441200640 1 -- 172.21.15.83:0/4103053794 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563aeef3c950 msgr2=0x563aeef37c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:59.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.353+0000 7f1441200640 1 --2- 172.21.15.83:0/4103053794 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563aeef3c950 0x563aeef37c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.353+0000 7f1441200640 1 -- 172.21.15.83:0/4103053794 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563aeec7e7d0 con 0x563aeef47940 2024-04-24T08:53:59.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.353+0000 7f1433400640 1 --2- 172.21.15.83:0/4103053794 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563aeef3d270 0x563aeef46ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:53:59.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.354+0000 7f1441200640 1 --2- 172.21.15.83:0/4103053794 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563aeef47940 0x563aeef473e0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f143400a8b0 tx=0x7f1434011db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4fd1641138f1cc0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:59.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.354+0000 7f1430c00640 1 -- 172.21.15.83:0/4103053794 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1434015040 con 0x563aeef47940 2024-04-24T08:53:59.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.354+0000 7f1430c00640 1 -- 172.21.15.83:0/4103053794 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f143400cb90 con 0x563aeef47940 2024-04-24T08:53:59.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.354+0000 7f1430c00640 1 -- 172.21.15.83:0/4103053794 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f143401a810 con 0x563aeef47940 2024-04-24T08:53:59.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.355+0000 7f1446022740 1 -- 172.21.15.83:0/4103053794 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563aeef47940 msgr2=0x563aeef473e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:59.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.355+0000 7f1446022740 1 --2- 172.21.15.83:0/4103053794 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563aeef47940 0x563aeef473e0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f143400a8b0 tx=0x7f1434011db0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.355+0000 7f1446022740 1 -- 172.21.15.83:0/4103053794 shutdown_connections 2024-04-24T08:53:59.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.355+0000 7f1446022740 1 --2- 172.21.15.83:0/4103053794 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563aeef47940 0x563aeef473e0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.355+0000 7f1446022740 1 --2- 172.21.15.83:0/4103053794 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563aeef3d270 0x563aeef46ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.355+0000 7f1446022740 1 --2- 172.21.15.83:0/4103053794 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563aeef3c950 0x563aeef37c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.355+0000 7f1446022740 1 -- 172.21.15.83:0/4103053794 >> 172.21.15.83:0/4103053794 conn(0x563aeef38c70 msgr2=0x563aeef37560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:59.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.355+0000 7f1446022740 1 -- 172.21.15.83:0/4103053794 shutdown_connections 2024-04-24T08:53:59.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.355+0000 7f1446022740 1 -- 172.21.15.83:0/4103053794 wait complete. 2024-04-24T08:53:59.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.355+0000 7f1446022740 1 -- start start 2024-04-24T08:53:59.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.356+0000 7f1446022740 1 Processor -- start 2024-04-24T08:53:59.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.356+0000 7f1446022740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563aeef3c950 0x563aeef4fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:59.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.356+0000 7f1446022740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563aeef3d270 0x563aeef500f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:59.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.356+0000 7f1446022740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563aeef47940 0x563aeef4ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:59.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.356+0000 7f1446022740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f143c1b7060 con 0x563aeef3c950 2024-04-24T08:53:59.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.356+0000 7f1446022740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f143c1b6990 con 0x563aeef3d270 2024-04-24T08:53:59.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.356+0000 7f1446022740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f143c1b3bd0 con 0x563aeef47940 2024-04-24T08:53:59.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.356+0000 7f1446022740 10 client.0 _ll_register_callbacks cb 0x7f142c008b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:53:59.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.356+0000 7f143be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563aeef3c950 0x563aeef4fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:59.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.356+0000 7f143be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563aeef3c950 0x563aeef4fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52960/0 (socket says 172.21.15.83:52960) 2024-04-24T08:53:59.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.357+0000 7f143be00640 1 -- 172.21.15.83:0/977593171 learned_addr learned my addr 172.21.15.83:0/977593171 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:53:59.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.357+0000 7f1433400640 1 --2- 172.21.15.83:0/977593171 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563aeef3d270 0x563aeef500f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:59.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.357+0000 7f1441200640 1 --2- 172.21.15.83:0/977593171 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563aeef47940 0x563aeef4ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:59.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.357+0000 7f143be00640 1 -- 172.21.15.83:0/977593171 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563aeef47940 msgr2=0x563aeef4ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:59.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.357+0000 7f143be00640 1 --2- 172.21.15.83:0/977593171 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563aeef47940 0x563aeef4ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.357+0000 7f143be00640 1 -- 172.21.15.83:0/977593171 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563aeef3d270 msgr2=0x563aeef500f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:59.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.357+0000 7f143be00640 1 --2- 172.21.15.83:0/977593171 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563aeef3d270 0x563aeef500f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.357+0000 7f143be00640 1 -- 172.21.15.83:0/977593171 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563aeef4b150 con 0x563aeef3c950 2024-04-24T08:53:59.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.357+0000 7f1433400640 1 --2- 172.21.15.83:0/977593171 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563aeef3d270 0x563aeef500f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:53:59.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.357+0000 7f143be00640 1 --2- 172.21.15.83:0/977593171 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563aeef3c950 0x563aeef4fbb0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f142c000f80 tx=0x7f142c00dc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d2bae7188e8573 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:59.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.357+0000 7f142a000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:53:59.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.357+0000 7f142a000640 1 -- 172.21.15.83:0/977593171 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f142c034070 con 0x563aeef3c950 2024-04-24T08:53:59.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.358+0000 7f142a000640 1 -- 172.21.15.83:0/977593171 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f142c01c9e0 con 0x563aeef3c950 2024-04-24T08:53:59.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.358+0000 7f1446022740 1 -- 172.21.15.83:0/977593171 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f143400cb90 con 0x563aeef3c950 2024-04-24T08:53:59.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.358+0000 7f142a000640 1 -- 172.21.15.83:0/977593171 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f142c040050 con 0x563aeef3c950 2024-04-24T08:53:59.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.358+0000 7f1446022740 10 client.16533 fetch_fsmap learned FSMap version 30 2024-04-24T08:53:59.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.358+0000 7f1446022740 1 -- 172.21.15.83:0/977593171 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563aeefbe9d0 con 0x563aeef3c950 2024-04-24T08:53:59.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.358+0000 7f142a000640 1 -- 172.21.15.83:0/977593171 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f143400cb90 con 0x563aeef3c950 2024-04-24T08:53:59.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.358+0000 7f142a000640 1 -- 172.21.15.83:0/977593171 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7f142c01d980 con 0x563aeef3c950 2024-04-24T08:53:59.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.358+0000 7f1446022740 10 client.16533 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T08:53:59.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.358+0000 7f1446022740 10 client.16533 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-24T08:53:59.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.358+0000 7f1446022740 20 client.16533 populate_metadata read hostname 'smithi083' 2024-04-24T08:53:59.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.358+0000 7f1446022740 1 --2- 172.21.15.83:0/977593171 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x563aeefbf150 0x563aeefc1570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:59.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.358+0000 7f1446022740 4 client.16533 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T08:53:59.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.358+0000 7f1446022740 1 -- 172.21.15.83:0/977593171 --> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563aeeea9a60 con 0x563aeefbf150 2024-04-24T08:53:59.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.359+0000 7f1433400640 1 --2- 172.21.15.83:0/977593171 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x563aeefbf150 0x563aeefc1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:59.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.360+0000 7f142a000640 10 client.16533 ms_handle_connect on v2:172.21.15.138:6834/1909219758 2024-04-24T08:53:59.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.360+0000 7f1433400640 1 --2- 172.21.15.83:0/977593171 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x563aeefbf150 0x563aeefc1570 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=620419ab6e539936 server_cookie=17028d8cf52c6b03 in_seq=0 out_seq=0 2024-04-24T08:53:59.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.360+0000 7f142a000640 1 -- 172.21.15.83:0/977593171 <== mds.2 v2:172.21.15.138:6834/1909219758 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563aeeea9a60 con 0x563aeefbf150 2024-04-24T08:53:59.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.360+0000 7f142a000640 10 client.16533 handle_command_reply: tid=0 2024-04-24T08:53:59.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.360+0000 7f1446022740 1 client.16533 shutdown 2024-04-24T08:53:59.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.361+0000 7f1446022740 1 -- 172.21.15.83:0/977593171 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563aeef3c950 msgr2=0x563aeef4fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:59.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.361+0000 7f1446022740 1 --2- 172.21.15.83:0/977593171 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563aeef3c950 0x563aeef4fbb0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f142c000f80 tx=0x7f142c00dc50 comp rx=0 tx=0).stop 2024-04-24T08:53:59.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.361+0000 7f1446022740 1 -- 172.21.15.83:0/977593171 shutdown_connections 2024-04-24T08:53:59.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.361+0000 7f1446022740 1 --2- 172.21.15.83:0/977593171 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x563aeefbf150 0x563aeefc1570 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.361+0000 7f1446022740 1 --2- 172.21.15.83:0/977593171 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563aeef47940 0x563aeef4ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.361+0000 7f1446022740 1 --2- 172.21.15.83:0/977593171 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563aeef3d270 0x563aeef500f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.361+0000 7f142a000640 0 client.16533 ms_handle_reset on v2:172.21.15.138:6834/1909219758 2024-04-24T08:53:59.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.361+0000 7f1446022740 1 --2- 172.21.15.83:0/977593171 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563aeef3c950 0x563aeef4fbb0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.361+0000 7f1446022740 1 -- 172.21.15.83:0/977593171 >> 172.21.15.83:0/977593171 conn(0x563aeef38c70 msgr2=0x563aeef37df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:59.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.361+0000 7f1446022740 1 -- 172.21.15.83:0/977593171 shutdown_connections 2024-04-24T08:53:59.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.361+0000 7f1446022740 1 -- 172.21.15.83:0/977593171 wait complete. 2024-04-24T08:53:59.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.361+0000 7f1446022740 20 client.16533 trim_cache size 0 max 16384 2024-04-24T08:53:59.366 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T08:53:59.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.362+0000 7f1443200640 1 -- 172.21.15.83:0/3101256500 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f14200a3da0 msgr2=0x7f14200a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:59.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.362+0000 7f1443200640 1 --2- 172.21.15.83:0/3101256500 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f14200a3da0 0x7f14200a6260 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x563aeef3df10 tx=0x7f142c008350 comp rx=0 tx=0).stop 2024-04-24T08:53:59.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.362+0000 7f1443200640 1 -- 172.21.15.83:0/3101256500 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f143c108fd0 msgr2=0x7f143c19f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:59.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.362+0000 7f1443200640 1 --2- 172.21.15.83:0/3101256500 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f143c108fd0 0x7f143c19f4e0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f1424031440 tx=0x7f1424003d50 comp rx=0 tx=0).stop 2024-04-24T08:53:59.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.362+0000 7f1443200640 1 -- 172.21.15.83:0/3101256500 shutdown_connections 2024-04-24T08:53:59.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.362+0000 7f1443200640 1 --2- 172.21.15.83:0/3101256500 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f14200a3da0 0x7f14200a6260 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.362+0000 7f1443200640 1 --2- 172.21.15.83:0/3101256500 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f143c10e660 0x7f143c1a3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.362+0000 7f1443200640 1 --2- 172.21.15.83:0/3101256500 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f143c108fd0 0x7f143c19f4e0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.362+0000 7f1443200640 1 --2- 172.21.15.83:0/3101256500 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f143c108610 0x7f143c19efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.362+0000 7f1443200640 1 -- 172.21.15.83:0/3101256500 >> 172.21.15.83:0/3101256500 conn(0x7f143c076540 msgr2=0x7f143c077d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:59.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.362+0000 7f1443200640 1 -- 172.21.15.83:0/3101256500 shutdown_connections 2024-04-24T08:53:59.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.362+0000 7f1443200640 1 -- 172.21.15.83:0/3101256500 wait complete. 2024-04-24T08:53:59.380 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T08:53:59.557 INFO:tasks.workunit.client.0.smithi083.stdout: 1 4506 1.24 MB/sec warmup 33 sec latency 44.046 ms 2024-04-24T08:53:59.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.588+0000 7f3ffb600640 1 -- 172.21.15.83:0/1981284210 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ff4107320 msgr2=0x7f3ff410eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:59.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.588+0000 7f3ffb600640 1 --2- 172.21.15.83:0/1981284210 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ff4107320 0x7f3ff410eef0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f3fec009ca0 tx=0x7f3fec02a8c0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.588+0000 7f3ffb600640 1 -- 172.21.15.83:0/1981284210 shutdown_connections 2024-04-24T08:53:59.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.588+0000 7f3ffb600640 1 --2- 172.21.15.83:0/1981284210 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ff4107320 0x7f3ff410eef0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.588+0000 7f3ffb600640 1 --2- 172.21.15.83:0/1981284210 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3ff406b2a0 0x7f3ff4106c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.588+0000 7f3ffb600640 1 --2- 172.21.15.83:0/1981284210 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3ff406a8f0 0x7f3ff406acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.588+0000 7f3ffb600640 1 -- 172.21.15.83:0/1981284210 >> 172.21.15.83:0/1981284210 conn(0x7f3ff4077f60 msgr2=0x7f3ff4078370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:59.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.588+0000 7f3ffb600640 1 -- 172.21.15.83:0/1981284210 shutdown_connections 2024-04-24T08:53:59.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.588+0000 7f3ffb600640 1 -- 172.21.15.83:0/1981284210 wait complete. 2024-04-24T08:53:59.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.589+0000 7f3ffb600640 1 Processor -- start 2024-04-24T08:53:59.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.589+0000 7f3ffb600640 1 -- start start 2024-04-24T08:53:59.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.590+0000 7f3ffb600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3ff406a8f0 0x7f3ff41a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:59.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.590+0000 7f3ffb600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ff4107320 0x7f3ff41a0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:59.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.590+0000 7f3ffb600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3ff419a700 0x7f3ff419ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:59.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.590+0000 7f3ffb600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff41122b0 con 0x7f3ff4107320 2024-04-24T08:53:59.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.590+0000 7f3ffb600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff4112130 con 0x7f3ff419a700 2024-04-24T08:53:59.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.590+0000 7f3ffb600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff4112430 con 0x7f3ff406a8f0 2024-04-24T08:53:59.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.590+0000 7f3ff3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ff4107320 0x7f3ff41a0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:59.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.590+0000 7f3ff3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ff4107320 0x7f3ff41a0b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52972/0 (socket says 172.21.15.83:52972) 2024-04-24T08:53:59.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.590+0000 7f3ff3e00640 1 -- 172.21.15.83:0/2890021718 learned_addr learned my addr 172.21.15.83:0/2890021718 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:53:59.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.590+0000 7f3ff8c00640 1 --2- 172.21.15.83:0/2890021718 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3ff406a8f0 0x7f3ff41a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:59.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.590+0000 7f3ff9600640 1 --2- 172.21.15.83:0/2890021718 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3ff419a700 0x7f3ff419ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:59.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.591+0000 7f3ff3e00640 1 -- 172.21.15.83:0/2890021718 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3ff406a8f0 msgr2=0x7f3ff41a0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:59.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.591+0000 7f3ff3e00640 1 --2- 172.21.15.83:0/2890021718 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3ff406a8f0 0x7f3ff41a0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.591+0000 7f3ff3e00640 1 -- 172.21.15.83:0/2890021718 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3ff419a700 msgr2=0x7f3ff419ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:59.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.591+0000 7f3ff3e00640 1 --2- 172.21.15.83:0/2890021718 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3ff419a700 0x7f3ff419ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.591+0000 7f3ff3e00640 1 -- 172.21.15.83:0/2890021718 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ff419b300 con 0x7f3ff4107320 2024-04-24T08:53:59.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.591+0000 7f3ff3e00640 1 --2- 172.21.15.83:0/2890021718 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ff4107320 0x7f3ff41a0b80 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f3fe000d770 tx=0x7f3fe000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:59.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.591+0000 7f3ff1600640 1 -- 172.21.15.83:0/2890021718 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe0017070 con 0x7f3ff4107320 2024-04-24T08:53:59.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.591+0000 7f3ff1600640 1 -- 172.21.15.83:0/2890021718 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3fe0009c30 con 0x7f3ff4107320 2024-04-24T08:53:59.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.591+0000 7f3ff1600640 1 -- 172.21.15.83:0/2890021718 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe000f620 con 0x7f3ff4107320 2024-04-24T08:53:59.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.591+0000 7f3ffb600640 1 -- 172.21.15.83:0/2890021718 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ff41a73e0 con 0x7f3ff4107320 2024-04-24T08:53:59.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.592+0000 7f3ffb600640 1 -- 172.21.15.83:0/2890021718 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ff41a77e0 con 0x7f3ff4107320 2024-04-24T08:53:59.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.595+0000 7f3ff1600640 1 -- 172.21.15.83:0/2890021718 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3fe001a070 con 0x7f3ff4107320 2024-04-24T08:53:59.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.595+0000 7f3ffe36d740 1 Processor -- start 2024-04-24T08:53:59.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.595+0000 7f3ffe36d740 1 -- start start 2024-04-24T08:53:59.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.596+0000 7f3ffe36d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5647da056950 0x5647da056d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:59.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.596+0000 7f3ffe36d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647da057270 0x5647da061400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:59.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.596+0000 7f3ffe36d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5647da061940 0x5647da063d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:59.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.596+0000 7f3ffe36d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5647d9fbc1d0 con 0x5647da061940 2024-04-24T08:53:59.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.596+0000 7f3ffe36d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5647d9f62450 con 0x5647da057270 2024-04-24T08:53:59.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.596+0000 7f3ffe36d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5647d9fb8190 con 0x5647da056950 2024-04-24T08:53:59.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.596+0000 7f3ff3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5647da061940 0x5647da063d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:59.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.596+0000 7f3ff3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5647da061940 0x5647da063d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52974/0 (socket says 172.21.15.83:52974) 2024-04-24T08:53:59.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.596+0000 7f3ff3e00640 1 -- 172.21.15.83:0/3176674795 learned_addr learned my addr 172.21.15.83:0/3176674795 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:53:59.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.596+0000 7f3ff8c00640 1 --2- 172.21.15.83:0/3176674795 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647da057270 0x5647da061400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:59.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.596+0000 7f3ff9600640 1 --2- 172.21.15.83:0/3176674795 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5647da056950 0x5647da056d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:59.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.597+0000 7f3ff3e00640 1 -- 172.21.15.83:0/3176674795 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5647da056950 msgr2=0x5647da056d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:59.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.597+0000 7f3ff3e00640 1 --2- 172.21.15.83:0/3176674795 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5647da056950 0x5647da056d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.597+0000 7f3ff3e00640 1 -- 172.21.15.83:0/3176674795 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647da057270 msgr2=0x5647da061400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:59.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.597+0000 7f3ff3e00640 1 --2- 172.21.15.83:0/3176674795 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647da057270 0x5647da061400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.597+0000 7f3ff3e00640 1 -- 172.21.15.83:0/3176674795 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647da01f310 con 0x5647da061940 2024-04-24T08:53:59.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.597+0000 7f3ff9600640 1 --2- 172.21.15.83:0/3176674795 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5647da056950 0x5647da056d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:53:59.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.597+0000 7f3ff3e00640 1 --2- 172.21.15.83:0/3176674795 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5647da061940 0x5647da063d30 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f3fe00cd9c0 tx=0x7f3fe00d8670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d50b8b7c7dbece9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:59.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.597+0000 7f3fe5c00640 1 -- 172.21.15.83:0/3176674795 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe00df070 con 0x5647da061940 2024-04-24T08:53:59.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.597+0000 7f3ff1600640 1 --2- 172.21.15.83:0/2890021718 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f3fdc0a3d20 0x7f3fdc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:59.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.597+0000 7f3fe5c00640 1 -- 172.21.15.83:0/3176674795 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3fe00d0d50 con 0x5647da061940 2024-04-24T08:53:59.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.597+0000 7f3fe5c00640 1 -- 172.21.15.83:0/3176674795 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe00db680 con 0x5647da061940 2024-04-24T08:53:59.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.598+0000 7f3ff1600640 1 -- 172.21.15.83:0/2890021718 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f3fe00d0760 con 0x7f3ff4107320 2024-04-24T08:53:59.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.598+0000 7f3ff9600640 1 --2- 172.21.15.83:0/2890021718 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f3fdc0a3d20 0x7f3fdc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:59.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.598+0000 7f3ffe36d740 1 -- 172.21.15.83:0/3176674795 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5647da061940 msgr2=0x5647da063d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:59.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.598+0000 7f3ffe36d740 1 --2- 172.21.15.83:0/3176674795 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5647da061940 0x5647da063d30 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f3fe00cd9c0 tx=0x7f3fe00d8670 comp rx=0 tx=0).stop 2024-04-24T08:53:59.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.599+0000 7f3ffe36d740 1 -- 172.21.15.83:0/3176674795 shutdown_connections 2024-04-24T08:53:59.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.599+0000 7f3ffe36d740 1 --2- 172.21.15.83:0/3176674795 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5647da061940 0x5647da063d30 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.599+0000 7f3ffe36d740 1 --2- 172.21.15.83:0/3176674795 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647da057270 0x5647da061400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.599+0000 7f3ffe36d740 1 --2- 172.21.15.83:0/3176674795 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5647da056950 0x5647da056d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.599+0000 7f3ffe36d740 1 -- 172.21.15.83:0/3176674795 >> 172.21.15.83:0/3176674795 conn(0x5647da052c70 msgr2=0x5647da053050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:59.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.599+0000 7f3ffe36d740 1 -- 172.21.15.83:0/3176674795 shutdown_connections 2024-04-24T08:53:59.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.599+0000 7f3ffe36d740 1 -- 172.21.15.83:0/3176674795 wait complete. 2024-04-24T08:53:59.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.599+0000 7f3ff9600640 1 --2- 172.21.15.83:0/2890021718 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f3fdc0a3d20 0x7f3fdc0a61e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f3fec02f3a0 tx=0x7f3fec02f310 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:59.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.599+0000 7f3ffe36d740 1 -- start start 2024-04-24T08:53:59.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.599+0000 7f3ffe36d740 1 Processor -- start 2024-04-24T08:53:59.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.600+0000 7f3ffe36d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647da056950 0x5647da061310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:59.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.600+0000 7f3ffe36d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5647da057270 0x5647da05e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:59.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.600+0000 7f3ffe36d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5647da061940 0x5647da05ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:59.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.600+0000 7f3ffe36d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5647da0647d0 con 0x5647da061940 2024-04-24T08:53:59.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.600+0000 7f3ffe36d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5647d9f75810 con 0x5647da056950 2024-04-24T08:53:59.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.600+0000 7f3ffe36d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5647da064950 con 0x5647da057270 2024-04-24T08:53:59.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.600+0000 7f3ffe36d740 10 client.0 _ll_register_callbacks cb 0x5647d9f43140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:53:59.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.600+0000 7f3ff9600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5647da061940 0x5647da05ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:59.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.600+0000 7f3ff9600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5647da061940 0x5647da05ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52980/0 (socket says 172.21.15.83:52980) 2024-04-24T08:53:59.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.600+0000 7f3ff9600640 1 -- 172.21.15.83:0/3020095626 learned_addr learned my addr 172.21.15.83:0/3020095626 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:53:59.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.600+0000 7f3ff8c00640 1 --2- 172.21.15.83:0/3020095626 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647da056950 0x5647da061310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:59.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.601+0000 7f3ff3e00640 1 --2- 172.21.15.83:0/3020095626 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5647da057270 0x5647da05e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:59.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.601+0000 7f3ff9600640 1 -- 172.21.15.83:0/3020095626 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5647da057270 msgr2=0x5647da05e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:59.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.601+0000 7f3ff9600640 1 --2- 172.21.15.83:0/3020095626 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5647da057270 0x5647da05e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.601+0000 7f3ff9600640 1 -- 172.21.15.83:0/3020095626 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647da056950 msgr2=0x5647da061310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:59.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.601+0000 7f3ff9600640 1 --2- 172.21.15.83:0/3020095626 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647da056950 0x5647da061310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.601+0000 7f3ff9600640 1 -- 172.21.15.83:0/3020095626 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647d9f863f0 con 0x5647da061940 2024-04-24T08:53:59.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.601+0000 7f3ff9600640 1 --2- 172.21.15.83:0/3020095626 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5647da061940 0x5647da05ecf0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f3fec031850 tx=0x7f3fec0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d05a18f254d2f58 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:59.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.601+0000 7f3fda000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:53:59.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.601+0000 7f3fda000640 1 -- 172.21.15.83:0/3020095626 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fec04f070 con 0x5647da061940 2024-04-24T08:53:59.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.601+0000 7f3ffe36d740 1 -- 172.21.15.83:0/3020095626 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3fe00d0d50 con 0x5647da061940 2024-04-24T08:53:59.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.601+0000 7f3fda000640 1 -- 172.21.15.83:0/3020095626 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f3fec004450 con 0x5647da061940 2024-04-24T08:53:59.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.602+0000 7f3fda000640 1 -- 172.21.15.83:0/3020095626 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3fec004650 con 0x5647da061940 2024-04-24T08:53:59.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.602+0000 7f3ffe36d740 10 client.16545 fetch_fsmap learned FSMap version 30 2024-04-24T08:53:59.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.602+0000 7f3ffe36d740 1 -- 172.21.15.83:0/3020095626 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5647d9cce3f0 con 0x5647da061940 2024-04-24T08:53:59.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.602+0000 7f3fda000640 1 -- 172.21.15.83:0/3020095626 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3fe00d0d50 con 0x5647da061940 2024-04-24T08:53:59.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.602+0000 7f3fda000640 1 -- 172.21.15.83:0/3020095626 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7f3fec044040 con 0x5647da061940 2024-04-24T08:53:59.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.602+0000 7f3ffe36d740 10 client.16545 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T08:53:59.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.602+0000 7f3ffe36d740 10 client.16545 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-24T08:53:59.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.602+0000 7f3ffe36d740 20 client.16545 populate_metadata read hostname 'smithi083' 2024-04-24T08:53:59.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.602+0000 7f3ffe36d740 1 --2- 172.21.15.83:0/3020095626 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x5647da07a280 0x5647da07c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:59.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.602+0000 7f3ffe36d740 4 client.16545 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:53:59.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.602+0000 7f3ffe36d740 1 -- 172.21.15.83:0/3020095626 --> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5647d9fc9bf0 con 0x5647da07a280 2024-04-24T08:53:59.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.603+0000 7f3ff8c00640 1 --2- 172.21.15.83:0/3020095626 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x5647da07a280 0x5647da07c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:59.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.604+0000 7f3ff8c00640 1 --2- 172.21.15.83:0/3020095626 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x5647da07a280 0x5647da07c720 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bc42efb12c03ec1c server_cookie=645ba406676e2aeb in_seq=0 out_seq=0 2024-04-24T08:53:59.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.604+0000 7f3fda000640 10 client.16545 ms_handle_connect on v2:172.21.15.138:6836/900566754 2024-04-24T08:53:59.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.606+0000 7f3fda000640 1 -- 172.21.15.83:0/3020095626 <== mds.1 v2:172.21.15.138:6836/900566754 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5647d9fc9bf0 con 0x5647da07a280 2024-04-24T08:53:59.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.606+0000 7f3fda000640 10 client.16545 handle_command_reply: tid=0 2024-04-24T08:53:59.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.606+0000 7f3ffe36d740 1 client.16545 shutdown 2024-04-24T08:53:59.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.606+0000 7f3ffe36d740 1 -- 172.21.15.83:0/3020095626 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5647da061940 msgr2=0x5647da05ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:59.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.606+0000 7f3ffe36d740 1 --2- 172.21.15.83:0/3020095626 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5647da061940 0x5647da05ecf0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f3fec031850 tx=0x7f3fec0095c0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.606+0000 7f3ffe36d740 1 -- 172.21.15.83:0/3020095626 shutdown_connections 2024-04-24T08:53:59.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.606+0000 7f3ffe36d740 1 --2- 172.21.15.83:0/3020095626 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5647da061940 0x5647da05ecf0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.606+0000 7f3ffe36d740 1 --2- 172.21.15.83:0/3020095626 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x5647da07a280 0x5647da07c720 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.606+0000 7f3ffe36d740 1 --2- 172.21.15.83:0/3020095626 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5647da057270 0x5647da05e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.606+0000 7f3ffe36d740 1 --2- 172.21.15.83:0/3020095626 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647da056950 0x5647da061310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.606+0000 7f3ffe36d740 1 -- 172.21.15.83:0/3020095626 >> 172.21.15.83:0/3020095626 conn(0x5647da052c70 msgr2=0x5647da058a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:59.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.606+0000 7f3fda000640 0 client.16545 ms_handle_reset on v2:172.21.15.138:6836/900566754 2024-04-24T08:53:59.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.607+0000 7f3ffe36d740 1 -- 172.21.15.83:0/3020095626 shutdown_connections 2024-04-24T08:53:59.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.607+0000 7f3ffe36d740 1 -- 172.21.15.83:0/3020095626 wait complete. 2024-04-24T08:53:59.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.607+0000 7f3ffe36d740 20 client.16545 trim_cache size 0 max 16384 2024-04-24T08:53:59.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.624+0000 7f3ffe36d740 1 Processor -- start 2024-04-24T08:53:59.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.624+0000 7f3ffe36d740 1 -- start start 2024-04-24T08:53:59.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.624+0000 7f3ffe36d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5647da056950 0x5647da081ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:59.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.624+0000 7f3ffe36d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5647da057270 0x5647da082530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:59.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.624+0000 7f3ffe36d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647da061940 0x5647da059940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:59.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.624+0000 7f3ffe36d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff41ad250 con 0x5647da056950 2024-04-24T08:53:59.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.624+0000 7f3ffe36d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff41aee20 con 0x5647da061940 2024-04-24T08:53:59.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.624+0000 7f3ffe36d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff41ad920 con 0x5647da057270 2024-04-24T08:53:59.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.625+0000 7f3ff8c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5647da056950 0x5647da081ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:59.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.625+0000 7f3ff8c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5647da056950 0x5647da081ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52986/0 (socket says 172.21.15.83:52986) 2024-04-24T08:53:59.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.625+0000 7f3ff8c00640 1 -- 172.21.15.83:0/594261558 learned_addr learned my addr 172.21.15.83:0/594261558 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:53:59.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.625+0000 7f3ff9600640 1 --2- 172.21.15.83:0/594261558 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647da061940 0x5647da059940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:59.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.625+0000 7f3ff3e00640 1 --2- 172.21.15.83:0/594261558 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5647da057270 0x5647da082530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:59.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.625+0000 7f3ff8c00640 1 -- 172.21.15.83:0/594261558 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5647da057270 msgr2=0x5647da082530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:59.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.625+0000 7f3ff8c00640 1 --2- 172.21.15.83:0/594261558 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5647da057270 0x5647da082530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.625+0000 7f3ff8c00640 1 -- 172.21.15.83:0/594261558 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647da061940 msgr2=0x5647da059940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:59.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.625+0000 7f3ff8c00640 1 --2- 172.21.15.83:0/594261558 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647da061940 0x5647da059940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.625+0000 7f3ff8c00640 1 -- 172.21.15.83:0/594261558 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647da0570a0 con 0x5647da056950 2024-04-24T08:53:59.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.626+0000 7f3ff8c00640 1 --2- 172.21.15.83:0/594261558 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5647da056950 0x5647da081ff0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f3fe8005940 tx=0x7f3fe80058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb1b5e6c973ddfa0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:59.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.626+0000 7f3fe5200640 1 -- 172.21.15.83:0/594261558 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe8013190 con 0x5647da056950 2024-04-24T08:53:59.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.626+0000 7f3fe5200640 1 -- 172.21.15.83:0/594261558 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3fe801e020 con 0x5647da056950 2024-04-24T08:53:59.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.626+0000 7f3fe5200640 1 -- 172.21.15.83:0/594261558 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe801a6a0 con 0x5647da056950 2024-04-24T08:53:59.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.627+0000 7f3ffe36d740 1 -- 172.21.15.83:0/594261558 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5647da056950 msgr2=0x5647da081ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:59.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.627+0000 7f3ffe36d740 1 --2- 172.21.15.83:0/594261558 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5647da056950 0x5647da081ff0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f3fe8005940 tx=0x7f3fe80058b0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.627+0000 7f3ffe36d740 1 -- 172.21.15.83:0/594261558 shutdown_connections 2024-04-24T08:53:59.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.627+0000 7f3ffe36d740 1 --2- 172.21.15.83:0/594261558 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647da061940 0x5647da059940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.627+0000 7f3ffe36d740 1 --2- 172.21.15.83:0/594261558 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5647da057270 0x5647da082530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.627+0000 7f3ffe36d740 1 --2- 172.21.15.83:0/594261558 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5647da056950 0x5647da081ff0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.627+0000 7f3ffe36d740 1 -- 172.21.15.83:0/594261558 >> 172.21.15.83:0/594261558 conn(0x5647da052c70 msgr2=0x5647da05dbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:59.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.627+0000 7f3ffe36d740 1 -- 172.21.15.83:0/594261558 shutdown_connections 2024-04-24T08:53:59.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.627+0000 7f3ffe36d740 1 -- 172.21.15.83:0/594261558 wait complete. 2024-04-24T08:53:59.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.628+0000 7f3ffe36d740 1 -- start start 2024-04-24T08:53:59.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.628+0000 7f3ffe36d740 1 Processor -- start 2024-04-24T08:53:59.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.628+0000 7f3ffe36d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5647da056950 0x5647da072650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:59.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.628+0000 7f3ffe36d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5647da057270 0x5647da05b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:59.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.628+0000 7f3ffe36d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647da061940 0x5647da05be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:59.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.628+0000 7f3ffe36d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff41ae6c0 con 0x5647da057270 2024-04-24T08:53:59.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.628+0000 7f3ffe36d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff41adff0 con 0x5647da061940 2024-04-24T08:53:59.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.628+0000 7f3ffe36d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff41ab250 con 0x5647da056950 2024-04-24T08:53:59.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.628+0000 7f3ffe36d740 10 client.0 _ll_register_callbacks cb 0x7f3fec009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:53:59.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.628+0000 7f3ff3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5647da057270 0x5647da05b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:59.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.629+0000 7f3ff3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5647da057270 0x5647da05b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53000/0 (socket says 172.21.15.83:53000) 2024-04-24T08:53:59.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.629+0000 7f3ff3e00640 1 -- 172.21.15.83:0/2197185727 learned_addr learned my addr 172.21.15.83:0/2197185727 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:53:59.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.629+0000 7f3ff8c00640 1 --2- 172.21.15.83:0/2197185727 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5647da056950 0x5647da072650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:59.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.629+0000 7f3ff3e00640 1 -- 172.21.15.83:0/2197185727 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5647da056950 msgr2=0x5647da072650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:59.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.629+0000 7f3ff9600640 1 --2- 172.21.15.83:0/2197185727 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647da061940 0x5647da05be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:59.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.629+0000 7f3ff3e00640 1 --2- 172.21.15.83:0/2197185727 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5647da056950 0x5647da072650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.629+0000 7f3ff3e00640 1 -- 172.21.15.83:0/2197185727 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647da061940 msgr2=0x5647da05be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:59.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.629+0000 7f3ff3e00640 1 --2- 172.21.15.83:0/2197185727 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647da061940 0x5647da05be30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.629+0000 7f3ff3e00640 1 -- 172.21.15.83:0/2197185727 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647d9d987d0 con 0x5647da057270 2024-04-24T08:53:59.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.629+0000 7f3ff3e00640 1 --2- 172.21.15.83:0/2197185727 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5647da057270 0x5647da05b8f0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f3fe000d740 tx=0x7f3fe00097b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d1ba76f30ace4a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:59.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.629+0000 7f3fda000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:53:59.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.629+0000 7f3fda000640 1 -- 172.21.15.83:0/2197185727 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe0021030 con 0x5647da057270 2024-04-24T08:53:59.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.630+0000 7f3fda000640 1 -- 172.21.15.83:0/2197185727 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f3fe009a5d0 con 0x5647da057270 2024-04-24T08:53:59.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.630+0000 7f3ffe36d740 1 -- 172.21.15.83:0/2197185727 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3fe801e020 con 0x5647da057270 2024-04-24T08:53:59.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.630+0000 7f3fda000640 1 -- 172.21.15.83:0/2197185727 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3fe00df070 con 0x5647da057270 2024-04-24T08:53:59.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.630+0000 7f3ffe36d740 10 client.16551 fetch_fsmap learned FSMap version 30 2024-04-24T08:53:59.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.630+0000 7f3ffe36d740 1 -- 172.21.15.83:0/2197185727 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5647da062cd0 con 0x5647da057270 2024-04-24T08:53:59.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.630+0000 7f3fda000640 1 -- 172.21.15.83:0/2197185727 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3fe801e020 con 0x5647da057270 2024-04-24T08:53:59.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.630+0000 7f3fda000640 1 -- 172.21.15.83:0/2197185727 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7f3fe0098940 con 0x5647da057270 2024-04-24T08:53:59.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.630+0000 7f3ffe36d740 10 client.16551 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T08:53:59.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.630+0000 7f3ffe36d740 10 client.16551 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-24T08:53:59.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.630+0000 7f3ffe36d740 20 client.16551 populate_metadata read hostname 'smithi083' 2024-04-24T08:53:59.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.630+0000 7f3ffe36d740 1 --2- 172.21.15.83:0/2197185727 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x5647da0e0860 0x5647da0e2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:59.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.630+0000 7f3ffe36d740 4 client.16551 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T08:53:59.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.630+0000 7f3ffe36d740 1 -- 172.21.15.83:0/2197185727 --> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5647d9fc3a60 con 0x5647da0e0860 2024-04-24T08:53:59.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.631+0000 7f3ff8c00640 1 --2- 172.21.15.83:0/2197185727 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x5647da0e0860 0x5647da0e2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:59.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.632+0000 7f3ff8c00640 1 --2- 172.21.15.83:0/2197185727 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x5647da0e0860 0x5647da0e2c50 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cfb8f341aff43afe server_cookie=e1cbfebe285696ab in_seq=0 out_seq=0 2024-04-24T08:53:59.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.632+0000 7f3fda000640 10 client.16551 ms_handle_connect on v2:172.21.15.138:6836/900566754 2024-04-24T08:53:59.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.633+0000 7f3fda000640 1 -- 172.21.15.83:0/2197185727 <== mds.1 v2:172.21.15.138:6836/900566754 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5647d9fc3a60 con 0x5647da0e0860 2024-04-24T08:53:59.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.633+0000 7f3fda000640 10 client.16551 handle_command_reply: tid=0 2024-04-24T08:53:59.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.633+0000 7f3ffe36d740 1 client.16551 shutdown 2024-04-24T08:53:59.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.633+0000 7f3ffe36d740 1 -- 172.21.15.83:0/2197185727 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5647da057270 msgr2=0x5647da05b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:59.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.633+0000 7f3ffe36d740 1 --2- 172.21.15.83:0/2197185727 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5647da057270 0x5647da05b8f0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f3fe000d740 tx=0x7f3fe00097b0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.633+0000 7f3ffe36d740 1 -- 172.21.15.83:0/2197185727 shutdown_connections 2024-04-24T08:53:59.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.633+0000 7f3ffe36d740 1 --2- 172.21.15.83:0/2197185727 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647da061940 0x5647da05be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.633+0000 7f3ffe36d740 1 --2- 172.21.15.83:0/2197185727 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5647da057270 0x5647da05b8f0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.633+0000 7f3ffe36d740 1 --2- 172.21.15.83:0/2197185727 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x5647da0e0860 0x5647da0e2c50 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.633+0000 7f3ffe36d740 1 --2- 172.21.15.83:0/2197185727 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5647da056950 0x5647da072650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.633+0000 7f3ffe36d740 1 -- 172.21.15.83:0/2197185727 >> 172.21.15.83:0/2197185727 conn(0x5647da052c70 msgr2=0x5647da059940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:59.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.633+0000 7f3fda000640 0 client.16551 ms_handle_reset on v2:172.21.15.138:6836/900566754 2024-04-24T08:53:59.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.633+0000 7f3ffe36d740 1 -- 172.21.15.83:0/2197185727 shutdown_connections 2024-04-24T08:53:59.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.633+0000 7f3ffe36d740 1 -- 172.21.15.83:0/2197185727 wait complete. 2024-04-24T08:53:59.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.633+0000 7f3ffe36d740 20 client.16551 trim_cache size 0 max 16384 2024-04-24T08:53:59.635 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T08:53:59.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.634+0000 7f3ffb600640 1 -- 172.21.15.83:0/2890021718 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f3fdc0a3d20 msgr2=0x7f3fdc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:59.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.634+0000 7f3ffb600640 1 --2- 172.21.15.83:0/2890021718 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f3fdc0a3d20 0x7f3fdc0a61e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f3fec02f3a0 tx=0x7f3fec02f310 comp rx=0 tx=0).stop 2024-04-24T08:53:59.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.634+0000 7f3ffb600640 1 -- 172.21.15.83:0/2890021718 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ff4107320 msgr2=0x7f3ff41a0b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:59.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.634+0000 7f3ffb600640 1 --2- 172.21.15.83:0/2890021718 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ff4107320 0x7f3ff41a0b80 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f3fe000d770 tx=0x7f3fe000dc30 comp rx=0 tx=0).stop 2024-04-24T08:53:59.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.634+0000 7f3ffb600640 1 -- 172.21.15.83:0/2890021718 shutdown_connections 2024-04-24T08:53:59.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.634+0000 7f3ffb600640 1 --2- 172.21.15.83:0/2890021718 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f3fdc0a3d20 0x7f3fdc0a61e0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.634+0000 7f3ffb600640 1 --2- 172.21.15.83:0/2890021718 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3ff419a700 0x7f3ff419ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.634+0000 7f3ffb600640 1 --2- 172.21.15.83:0/2890021718 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ff4107320 0x7f3ff41a0b80 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.634+0000 7f3ffb600640 1 --2- 172.21.15.83:0/2890021718 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3ff406a8f0 0x7f3ff41a0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.634+0000 7f3ffb600640 1 -- 172.21.15.83:0/2890021718 >> 172.21.15.83:0/2890021718 conn(0x7f3ff4077f60 msgr2=0x7f3ff410d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:59.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.634+0000 7f3ffb600640 1 -- 172.21.15.83:0/2890021718 shutdown_connections 2024-04-24T08:53:59.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.634+0000 7f3ffb600640 1 -- 172.21.15.83:0/2890021718 wait complete. 2024-04-24T08:53:59.648 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:53:59 smithi083 ceph-mon[27907]: pgmap v74: 129 pgs: 129 active+clean; 27 MiB data, 468 MiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 468 KiB/s wr, 69 op/s 2024-04-24T08:53:59.651 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T08:53:59.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.846+0000 7f4212800640 1 -- 172.21.15.83:0/3434929525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f420c074430 msgr2=0x7f420c074810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:59.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.846+0000 7f4212800640 1 --2- 172.21.15.83:0/3434929525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f420c074430 0x7f420c074810 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f4200009900 tx=0x7f420002a480 comp rx=0 tx=0).stop 2024-04-24T08:53:59.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.847+0000 7f4212800640 1 -- 172.21.15.83:0/3434929525 shutdown_connections 2024-04-24T08:53:59.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.847+0000 7f4212800640 1 --2- 172.21.15.83:0/3434929525 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f420c10e6e0 0x7f420c110b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.847+0000 7f4212800640 1 --2- 172.21.15.83:0/3434929525 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f420c074d50 0x7f420c10e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.847+0000 7f4212800640 1 --2- 172.21.15.83:0/3434929525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f420c074430 0x7f420c074810 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.847+0000 7f4212800640 1 -- 172.21.15.83:0/3434929525 >> 172.21.15.83:0/3434929525 conn(0x7f420c0fd9d0 msgr2=0x7f420c0ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:59.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.847+0000 7f4212800640 1 -- 172.21.15.83:0/3434929525 shutdown_connections 2024-04-24T08:53:59.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.847+0000 7f4212800640 1 -- 172.21.15.83:0/3434929525 wait complete. 2024-04-24T08:53:59.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.848+0000 7f4212800640 1 Processor -- start 2024-04-24T08:53:59.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.848+0000 7f4212800640 1 -- start start 2024-04-24T08:53:59.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.848+0000 7f4212800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f420c074430 0x7f420c1a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:59.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.848+0000 7f4212800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f420c074d50 0x7f420c1a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:59.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.848+0000 7f4212800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f420c10e6e0 0x7f420c19a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:59.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.848+0000 7f4212800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f420c114320 con 0x7f420c10e6e0 2024-04-24T08:53:59.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.848+0000 7f4212800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f420c1141a0 con 0x7f420c074d50 2024-04-24T08:53:59.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.848+0000 7f4212800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f420c1144a0 con 0x7f420c074430 2024-04-24T08:53:59.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.849+0000 7f4211e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f420c10e6e0 0x7f420c19a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:59.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.849+0000 7f4211e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f420c10e6e0 0x7f420c19a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53016/0 (socket says 172.21.15.83:53016) 2024-04-24T08:53:59.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.849+0000 7f4211e00640 1 -- 172.21.15.83:0/259367946 learned_addr learned my addr 172.21.15.83:0/259367946 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:53:59.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.849+0000 7f4211400640 1 --2- 172.21.15.83:0/259367946 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f420c074430 0x7f420c1a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:59.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.849+0000 7f4210a00640 1 --2- 172.21.15.83:0/259367946 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f420c074d50 0x7f420c1a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:59.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.849+0000 7f4211e00640 1 -- 172.21.15.83:0/259367946 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f420c074430 msgr2=0x7f420c1a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:59.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.849+0000 7f4211e00640 1 --2- 172.21.15.83:0/259367946 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f420c074430 0x7f420c1a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.849+0000 7f4211e00640 1 -- 172.21.15.83:0/259367946 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f420c074d50 msgr2=0x7f420c1a0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:59.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.849+0000 7f4211e00640 1 --2- 172.21.15.83:0/259367946 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f420c074d50 0x7f420c1a0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.849+0000 7f4211e00640 1 -- 172.21.15.83:0/259367946 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f420c19aff0 con 0x7f420c10e6e0 2024-04-24T08:53:59.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.850+0000 7f4211e00640 1 --2- 172.21.15.83:0/259367946 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f420c10e6e0 0x7f420c19a730 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f420400cad0 tx=0x7f420400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:59.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.850+0000 7f420a000640 1 -- 172.21.15.83:0/259367946 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f420400ae20 con 0x7f420c10e6e0 2024-04-24T08:53:59.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.850+0000 7f420a000640 1 -- 172.21.15.83:0/259367946 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4204005020 con 0x7f420c10e6e0 2024-04-24T08:53:59.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.850+0000 7f4212800640 1 -- 172.21.15.83:0/259367946 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f420c19b2e0 con 0x7f420c10e6e0 2024-04-24T08:53:59.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.850+0000 7f420a000640 1 -- 172.21.15.83:0/259367946 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f420400f6b0 con 0x7f420c10e6e0 2024-04-24T08:53:59.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.850+0000 7f4212800640 1 -- 172.21.15.83:0/259367946 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f420c1a75a0 con 0x7f420c10e6e0 2024-04-24T08:53:59.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.854+0000 7f4216bed740 1 Processor -- start 2024-04-24T08:53:59.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.854+0000 7f420a000640 1 -- 172.21.15.83:0/259367946 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f42040051d0 con 0x7f420c10e6e0 2024-04-24T08:53:59.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.854+0000 7f4216bed740 1 -- start start 2024-04-24T08:53:59.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.854+0000 7f4216bed740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d25a5abb40 0x55d25a5abf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:59.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.855+0000 7f4216bed740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d25a5ac460 0x55d25a5b65f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:59.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.855+0000 7f4216bed740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d25a5b6b30 0x55d25a5b8f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:59.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.855+0000 7f4216bed740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d25a5111d0 con 0x55d25a5b6b30 2024-04-24T08:53:59.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.855+0000 7f4216bed740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d25a4b7450 con 0x55d25a5ac460 2024-04-24T08:53:59.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.855+0000 7f4216bed740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d25a50d190 con 0x55d25a5abb40 2024-04-24T08:53:59.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.855+0000 7f4210a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d25a5b6b30 0x55d25a5b8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:59.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.855+0000 7f4210a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d25a5b6b30 0x55d25a5b8f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53026/0 (socket says 172.21.15.83:53026) 2024-04-24T08:53:59.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.855+0000 7f4210a00640 1 -- 172.21.15.83:0/4281736790 learned_addr learned my addr 172.21.15.83:0/4281736790 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:53:59.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.855+0000 7f4210a00640 1 --2- 172.21.15.83:0/4281736790 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d25a5abb40 0x55d25a5abf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:59.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.855+0000 7f4211400640 1 --2- 172.21.15.83:0/4281736790 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d25a5ac460 0x55d25a5b65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:59.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.856+0000 7f4210a00640 1 -- 172.21.15.83:0/4281736790 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d25a5abb40 msgr2=0x55d25a5abf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:59.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.856+0000 7f4210a00640 1 --2- 172.21.15.83:0/4281736790 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d25a5abb40 0x55d25a5abf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.856+0000 7f4210a00640 1 -- 172.21.15.83:0/4281736790 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d25a5ac460 msgr2=0x55d25a5b65f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:59.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.856+0000 7f4210a00640 1 --2- 172.21.15.83:0/4281736790 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d25a5ac460 0x55d25a5b65f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.856+0000 7f4210a00640 1 -- 172.21.15.83:0/4281736790 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d25a574310 con 0x55d25a5b6b30 2024-04-24T08:53:59.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.856+0000 7f4211400640 1 --2- 172.21.15.83:0/4281736790 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d25a5ac460 0x55d25a5b65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:53:59.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.856+0000 7f4210a00640 1 --2- 172.21.15.83:0/4281736790 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d25a5b6b30 0x55d25a5b8f20 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f41f80149c0 tx=0x7f41f800c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0537dc1b1802715 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:59.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.856+0000 7f420a000640 1 --2- 172.21.15.83:0/259367946 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f41f40a3da0 0x7f41f40a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:59.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.856+0000 7f41fe600640 1 -- 172.21.15.83:0/4281736790 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f80142d0 con 0x55d25a5b6b30 2024-04-24T08:53:59.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.856+0000 7f41fe600640 1 -- 172.21.15.83:0/4281736790 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f41f800fc10 con 0x55d25a5b6b30 2024-04-24T08:53:59.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.856+0000 7f41fe600640 1 -- 172.21.15.83:0/4281736790 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f8012b00 con 0x55d25a5b6b30 2024-04-24T08:53:59.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.856+0000 7f420a000640 1 -- 172.21.15.83:0/259367946 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f42040d0ba0 con 0x7f420c10e6e0 2024-04-24T08:53:59.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.856+0000 7f4211400640 1 --2- 172.21.15.83:0/259367946 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f41f40a3da0 0x7f41f40a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:59.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.857+0000 7f4216bed740 1 -- 172.21.15.83:0/4281736790 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d25a5b6b30 msgr2=0x55d25a5b8f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:59.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.857+0000 7f4216bed740 1 --2- 172.21.15.83:0/4281736790 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d25a5b6b30 0x55d25a5b8f20 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f41f80149c0 tx=0x7f41f800c4d0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.857+0000 7f4216bed740 1 -- 172.21.15.83:0/4281736790 shutdown_connections 2024-04-24T08:53:59.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.857+0000 7f4216bed740 1 --2- 172.21.15.83:0/4281736790 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d25a5b6b30 0x55d25a5b8f20 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.857+0000 7f4216bed740 1 --2- 172.21.15.83:0/4281736790 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d25a5ac460 0x55d25a5b65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.857+0000 7f4216bed740 1 --2- 172.21.15.83:0/4281736790 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d25a5abb40 0x55d25a5abf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.857+0000 7f4216bed740 1 -- 172.21.15.83:0/4281736790 >> 172.21.15.83:0/4281736790 conn(0x55d25a5a7e60 msgr2=0x55d25a5a8240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:59.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.857+0000 7f4211400640 1 --2- 172.21.15.83:0/259367946 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f41f40a3da0 0x7f41f40a6260 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4200031440 tx=0x7f42000058b0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:59.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.857+0000 7f4216bed740 1 -- 172.21.15.83:0/4281736790 shutdown_connections 2024-04-24T08:53:59.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.857+0000 7f4216bed740 1 -- 172.21.15.83:0/4281736790 wait complete. 2024-04-24T08:53:59.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.858+0000 7f4216bed740 1 -- start start 2024-04-24T08:53:59.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.858+0000 7f4216bed740 1 Processor -- start 2024-04-24T08:53:59.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.858+0000 7f4216bed740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d25a5abb40 0x55d25a5b6550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:59.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.858+0000 7f4216bed740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d25a5ac460 0x55d25a5b3840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:59.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.858+0000 7f4216bed740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d25a5b6b30 0x55d25a5b3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:59.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.859+0000 7f4216bed740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d25a5b99c0 con 0x55d25a5abb40 2024-04-24T08:53:59.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.859+0000 7f4216bed740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d25a4ca810 con 0x55d25a5ac460 2024-04-24T08:53:59.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.859+0000 7f4216bed740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d25a5b9b40 con 0x55d25a5b6b30 2024-04-24T08:53:59.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.859+0000 7f4216bed740 10 client.0 _ll_register_callbacks cb 0x55d25a498140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:53:59.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.859+0000 7f4211400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d25a5abb40 0x55d25a5b6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:59.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.859+0000 7f4211400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d25a5abb40 0x55d25a5b6550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53036/0 (socket says 172.21.15.83:53036) 2024-04-24T08:53:59.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.859+0000 7f4211400640 1 -- 172.21.15.83:0/919547113 learned_addr learned my addr 172.21.15.83:0/919547113 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:53:59.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.859+0000 7f4210a00640 1 --2- 172.21.15.83:0/919547113 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d25a5ac460 0x55d25a5b3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:59.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.859+0000 7f4211400640 1 -- 172.21.15.83:0/919547113 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d25a5b6b30 msgr2=0x55d25a5b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:59.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.859+0000 7f4211400640 1 --2- 172.21.15.83:0/919547113 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d25a5b6b30 0x55d25a5b3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.859+0000 7f4211400640 1 -- 172.21.15.83:0/919547113 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d25a5ac460 msgr2=0x55d25a5b3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:59.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.859+0000 7f4211400640 1 --2- 172.21.15.83:0/919547113 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d25a5ac460 0x55d25a5b3840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.859+0000 7f4211400640 1 -- 172.21.15.83:0/919547113 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d25a4db3f0 con 0x55d25a5abb40 2024-04-24T08:53:59.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.859+0000 7f4210a00640 1 --2- 172.21.15.83:0/919547113 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d25a5ac460 0x55d25a5b3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:53:59.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.860+0000 7f4211400640 1 --2- 172.21.15.83:0/919547113 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d25a5abb40 0x55d25a5b6550 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f4200009900 tx=0x7f420002a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cf373c5929efe0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:59.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.860+0000 7f41f2a00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:53:59.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.860+0000 7f41f2a00640 1 -- 172.21.15.83:0/919547113 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f420004b070 con 0x55d25a5abb40 2024-04-24T08:53:59.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.860+0000 7f41f2a00640 1 -- 172.21.15.83:0/919547113 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f4200003b40 con 0x55d25a5abb40 2024-04-24T08:53:59.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.860+0000 7f4216bed740 1 -- 172.21.15.83:0/919547113 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41f800fc10 con 0x55d25a5abb40 2024-04-24T08:53:59.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.860+0000 7f41f2a00640 1 -- 172.21.15.83:0/919547113 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4200057050 con 0x55d25a5abb40 2024-04-24T08:53:59.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.860+0000 7f4216bed740 10 client.16563 fetch_fsmap learned FSMap version 30 2024-04-24T08:53:59.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.860+0000 7f4216bed740 1 -- 172.21.15.83:0/919547113 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d25a2233f0 con 0x55d25a5abb40 2024-04-24T08:53:59.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.860+0000 7f41f2a00640 1 -- 172.21.15.83:0/919547113 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f41f800fc10 con 0x55d25a5abb40 2024-04-24T08:53:59.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.861+0000 7f41f2a00640 1 -- 172.21.15.83:0/919547113 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7f4200047ba0 con 0x55d25a5abb40 2024-04-24T08:53:59.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.861+0000 7f4216bed740 10 client.16563 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T08:53:59.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.861+0000 7f4216bed740 10 client.16563 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:53:59.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.861+0000 7f4216bed740 20 client.16563 populate_metadata read hostname 'smithi083' 2024-04-24T08:53:59.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.861+0000 7f4216bed740 1 --2- 172.21.15.83:0/919547113 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d25a5c6c90 0x55d25a5c9130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:59.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.861+0000 7f4216bed740 4 client.16563 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:53:59.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.861+0000 7f4216bed740 1 -- 172.21.15.83:0/919547113 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d25a51ebf0 con 0x55d25a5c6c90 2024-04-24T08:53:59.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.861+0000 7f4210a00640 1 --2- 172.21.15.83:0/919547113 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d25a5c6c90 0x55d25a5c9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:59.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.863+0000 7f4210a00640 1 --2- 172.21.15.83:0/919547113 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d25a5c6c90 0x55d25a5c9130 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e1fcbe65a8a4175 server_cookie=857f6b32cab22893 in_seq=0 out_seq=0 2024-04-24T08:53:59.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.863+0000 7f41f2a00640 10 client.16563 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:53:59.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.864+0000 7f41f2a00640 1 -- 172.21.15.83:0/919547113 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d25a51ebf0 con 0x55d25a5c6c90 2024-04-24T08:53:59.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.864+0000 7f41f2a00640 10 client.16563 handle_command_reply: tid=0 2024-04-24T08:53:59.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.865+0000 7f4216bed740 1 client.16563 shutdown 2024-04-24T08:53:59.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.865+0000 7f4216bed740 1 -- 172.21.15.83:0/919547113 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d25a5abb40 msgr2=0x55d25a5b6550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:59.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.865+0000 7f4216bed740 1 --2- 172.21.15.83:0/919547113 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d25a5abb40 0x55d25a5b6550 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f4200009900 tx=0x7f420002a480 comp rx=0 tx=0).stop 2024-04-24T08:53:59.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.865+0000 7f4216bed740 1 -- 172.21.15.83:0/919547113 shutdown_connections 2024-04-24T08:53:59.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.865+0000 7f4216bed740 1 --2- 172.21.15.83:0/919547113 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d25a5c6c90 0x55d25a5c9130 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.865+0000 7f4216bed740 1 --2- 172.21.15.83:0/919547113 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d25a5b6b30 0x55d25a5b3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.865+0000 7f4216bed740 1 --2- 172.21.15.83:0/919547113 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d25a5ac460 0x55d25a5b3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.865+0000 7f4216bed740 1 --2- 172.21.15.83:0/919547113 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d25a5abb40 0x55d25a5b6550 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.865+0000 7f41f2a00640 0 client.16563 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:53:59.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.865+0000 7f4216bed740 1 -- 172.21.15.83:0/919547113 >> 172.21.15.83:0/919547113 conn(0x55d25a5a7e60 msgr2=0x55d25a5adc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:59.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.865+0000 7f4216bed740 1 -- 172.21.15.83:0/919547113 shutdown_connections 2024-04-24T08:53:59.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.865+0000 7f4216bed740 1 -- 172.21.15.83:0/919547113 wait complete. 2024-04-24T08:53:59.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.865+0000 7f4216bed740 20 client.16563 trim_cache size 0 max 16384 2024-04-24T08:53:59.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.881+0000 7f4216bed740 1 Processor -- start 2024-04-24T08:53:59.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.881+0000 7f4216bed740 1 -- start start 2024-04-24T08:53:59.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.881+0000 7f4216bed740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d25a5abb40 0x55d25a5a76d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:59.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.881+0000 7f4216bed740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d25a5ac460 0x55d25a5b6080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:59.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.881+0000 7f4216bed740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d25a5b6b30 0x55d25a5cea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:59.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.881+0000 7f4216bed740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f420c1b15e0 con 0x55d25a5b6b30 2024-04-24T08:53:59.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.881+0000 7f4216bed740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f420c1b31b0 con 0x55d25a5abb40 2024-04-24T08:53:59.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.881+0000 7f4216bed740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f420c1b1cb0 con 0x55d25a5ac460 2024-04-24T08:53:59.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.882+0000 7f4211e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d25a5b6b30 0x55d25a5cea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:59.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.882+0000 7f4211e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d25a5b6b30 0x55d25a5cea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53050/0 (socket says 172.21.15.83:53050) 2024-04-24T08:53:59.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.882+0000 7f4211e00640 1 -- 172.21.15.83:0/2845548254 learned_addr learned my addr 172.21.15.83:0/2845548254 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:53:59.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.882+0000 7f4210a00640 1 --2- 172.21.15.83:0/2845548254 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d25a5ac460 0x55d25a5b6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:59.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.882+0000 7f4211400640 1 --2- 172.21.15.83:0/2845548254 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d25a5abb40 0x55d25a5a76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:59.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.882+0000 7f4211e00640 1 -- 172.21.15.83:0/2845548254 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d25a5ac460 msgr2=0x55d25a5b6080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:59.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.882+0000 7f4211e00640 1 --2- 172.21.15.83:0/2845548254 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d25a5ac460 0x55d25a5b6080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.882+0000 7f4211e00640 1 -- 172.21.15.83:0/2845548254 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d25a5abb40 msgr2=0x55d25a5a76d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:59.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.882+0000 7f4211e00640 1 --2- 172.21.15.83:0/2845548254 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d25a5abb40 0x55d25a5a76d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.882+0000 7f4211e00640 1 -- 172.21.15.83:0/2845548254 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d25a2ed7d0 con 0x55d25a5b6b30 2024-04-24T08:53:59.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.882+0000 7f4210a00640 1 --2- 172.21.15.83:0/2845548254 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d25a5ac460 0x55d25a5b6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:53:59.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.883+0000 7f4211e00640 1 --2- 172.21.15.83:0/2845548254 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d25a5b6b30 0x55d25a5cea40 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f4204099140 tx=0x7f420409ce70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e74a453f5ceb0097 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:59.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.883+0000 7f41fdc00640 1 -- 172.21.15.83:0/2845548254 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f420401d040 con 0x55d25a5b6b30 2024-04-24T08:53:59.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.883+0000 7f41fdc00640 1 -- 172.21.15.83:0/2845548254 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f42040a0b20 con 0x55d25a5b6b30 2024-04-24T08:53:59.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.883+0000 7f41fdc00640 1 -- 172.21.15.83:0/2845548254 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42040984c0 con 0x55d25a5b6b30 2024-04-24T08:53:59.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.883+0000 7f4216bed740 1 -- 172.21.15.83:0/2845548254 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d25a5b6b30 msgr2=0x55d25a5cea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:59.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.883+0000 7f4216bed740 1 --2- 172.21.15.83:0/2845548254 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d25a5b6b30 0x55d25a5cea40 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f4204099140 tx=0x7f420409ce70 comp rx=0 tx=0).stop 2024-04-24T08:53:59.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.884+0000 7f4216bed740 1 -- 172.21.15.83:0/2845548254 shutdown_connections 2024-04-24T08:53:59.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.884+0000 7f4216bed740 1 --2- 172.21.15.83:0/2845548254 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d25a5b6b30 0x55d25a5cea40 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.884+0000 7f4216bed740 1 --2- 172.21.15.83:0/2845548254 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d25a5ac460 0x55d25a5b6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.884+0000 7f4216bed740 1 --2- 172.21.15.83:0/2845548254 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d25a5abb40 0x55d25a5a76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.884+0000 7f4216bed740 1 -- 172.21.15.83:0/2845548254 >> 172.21.15.83:0/2845548254 conn(0x55d25a5a7e60 msgr2=0x55d25a5aead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:59.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.884+0000 7f4216bed740 1 -- 172.21.15.83:0/2845548254 shutdown_connections 2024-04-24T08:53:59.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.884+0000 7f4216bed740 1 -- 172.21.15.83:0/2845548254 wait complete. 2024-04-24T08:53:59.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.884+0000 7f4216bed740 1 -- start start 2024-04-24T08:53:59.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.884+0000 7f4216bed740 1 Processor -- start 2024-04-24T08:53:59.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.885+0000 7f4216bed740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d25a5abb40 0x55d25a5b7530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:59.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.885+0000 7f4216bed740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d25a5ac460 0x55d25a5b7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:59.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.885+0000 7f4216bed740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d25a5b6b30 0x55d25a5bdd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:59.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.885+0000 7f4216bed740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f420c1b2a50 con 0x55d25a5ac460 2024-04-24T08:53:59.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.885+0000 7f4216bed740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f420c1b2380 con 0x55d25a5abb40 2024-04-24T08:53:59.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.885+0000 7f4216bed740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f420c1af5c0 con 0x55d25a5b6b30 2024-04-24T08:53:59.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.885+0000 7f4216bed740 10 client.0 _ll_register_callbacks cb 0x7f420003f070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:53:59.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.885+0000 7f4210a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d25a5ac460 0x55d25a5b7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:59.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.885+0000 7f4210a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d25a5ac460 0x55d25a5b7a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53054/0 (socket says 172.21.15.83:53054) 2024-04-24T08:53:59.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.885+0000 7f4210a00640 1 -- 172.21.15.83:0/4286263541 learned_addr learned my addr 172.21.15.83:0/4286263541 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:53:59.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.885+0000 7f4211400640 1 --2- 172.21.15.83:0/4286263541 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d25a5abb40 0x55d25a5b7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:59.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.885+0000 7f4210a00640 1 -- 172.21.15.83:0/4286263541 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d25a5b6b30 msgr2=0x55d25a5bdd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:59.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.885+0000 7f4210a00640 1 --2- 172.21.15.83:0/4286263541 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d25a5b6b30 0x55d25a5bdd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.885+0000 7f4210a00640 1 -- 172.21.15.83:0/4286263541 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d25a5abb40 msgr2=0x55d25a5b7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:53:59.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.885+0000 7f4210a00640 1 --2- 172.21.15.83:0/4286263541 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d25a5abb40 0x55d25a5b7530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.885+0000 7f4210a00640 1 -- 172.21.15.83:0/4286263541 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d25a5be2d0 con 0x55d25a5ac460 2024-04-24T08:53:59.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.885+0000 7f4211400640 1 --2- 172.21.15.83:0/4286263541 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d25a5abb40 0x55d25a5b7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:53:59.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.886+0000 7f4210a00640 1 --2- 172.21.15.83:0/4286263541 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d25a5ac460 0x55d25a5b7a70 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f41f8014990 tx=0x7f41f8005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b1723d02c067666 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:53:59.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.886+0000 7f41f2a00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:53:59.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.886+0000 7f41f2a00640 1 -- 172.21.15.83:0/4286263541 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f802f060 con 0x55d25a5ac460 2024-04-24T08:53:59.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.886+0000 7f41f2a00640 1 -- 172.21.15.83:0/4286263541 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f41f80142c0 con 0x55d25a5ac460 2024-04-24T08:53:59.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.886+0000 7f4216bed740 1 -- 172.21.15.83:0/4286263541 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42040a0b20 con 0x55d25a5ac460 2024-04-24T08:53:59.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.886+0000 7f41f2a00640 1 -- 172.21.15.83:0/4286263541 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f41f800c990 con 0x55d25a5ac460 2024-04-24T08:53:59.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.886+0000 7f4216bed740 10 client.16569 fetch_fsmap learned FSMap version 30 2024-04-24T08:53:59.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.886+0000 7f4216bed740 1 -- 172.21.15.83:0/4286263541 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d25a5bf5d0 con 0x55d25a5ac460 2024-04-24T08:53:59.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.887+0000 7f41f2a00640 1 -- 172.21.15.83:0/4286263541 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f42040a0b20 con 0x55d25a5ac460 2024-04-24T08:53:59.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.887+0000 7f41f2a00640 1 -- 172.21.15.83:0/4286263541 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7f41f8012750 con 0x55d25a5ac460 2024-04-24T08:53:59.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.887+0000 7f4216bed740 10 client.16569 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T08:53:59.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.887+0000 7f4216bed740 10 client.16569 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:53:59.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.887+0000 7f4216bed740 20 client.16569 populate_metadata read hostname 'smithi083' 2024-04-24T08:53:59.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.887+0000 7f4216bed740 1 --2- 172.21.15.83:0/4286263541 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d25a62d520 0x55d25a62f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:53:59.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.887+0000 7f4216bed740 4 client.16569 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T08:53:59.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.887+0000 7f4216bed740 1 -- 172.21.15.83:0/4286263541 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d25a518a60 con 0x55d25a62d520 2024-04-24T08:53:59.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.888+0000 7f4211400640 1 --2- 172.21.15.83:0/4286263541 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d25a62d520 0x55d25a62f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:53:59.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.889+0000 7f4211400640 1 --2- 172.21.15.83:0/4286263541 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d25a62d520 0x55d25a62f940 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92169c1bcc6e173 server_cookie=8d83499b96a03626 in_seq=0 out_seq=0 2024-04-24T08:53:59.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.889+0000 7f41f2a00640 10 client.16569 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:53:59.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.891+0000 7f41f2a00640 1 -- 172.21.15.83:0/4286263541 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d25a518a60 con 0x55d25a62d520 2024-04-24T08:53:59.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.891+0000 7f41f2a00640 10 client.16569 handle_command_reply: tid=0 2024-04-24T08:53:59.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.891+0000 7f4216bed740 1 client.16569 shutdown 2024-04-24T08:53:59.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.891+0000 7f4216bed740 1 -- 172.21.15.83:0/4286263541 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d25a5ac460 msgr2=0x55d25a5b7a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:59.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.891+0000 7f4216bed740 1 --2- 172.21.15.83:0/4286263541 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d25a5ac460 0x55d25a5b7a70 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f41f8014990 tx=0x7f41f8005850 comp rx=0 tx=0).stop 2024-04-24T08:53:59.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.891+0000 7f4216bed740 1 -- 172.21.15.83:0/4286263541 shutdown_connections 2024-04-24T08:53:59.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.891+0000 7f4216bed740 1 --2- 172.21.15.83:0/4286263541 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d25a62d520 0x55d25a62f940 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.891+0000 7f4216bed740 1 --2- 172.21.15.83:0/4286263541 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d25a5b6b30 0x55d25a5bdd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.891+0000 7f4216bed740 1 --2- 172.21.15.83:0/4286263541 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d25a5ac460 0x55d25a5b7a70 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.891+0000 7f4216bed740 1 --2- 172.21.15.83:0/4286263541 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d25a5abb40 0x55d25a5b7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.891+0000 7f41f2a00640 0 client.16569 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:53:59.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.891+0000 7f4216bed740 1 -- 172.21.15.83:0/4286263541 >> 172.21.15.83:0/4286263541 conn(0x55d25a5a7e60 msgr2=0x55d25a5ad3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:59.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.891+0000 7f4216bed740 1 -- 172.21.15.83:0/4286263541 shutdown_connections 2024-04-24T08:53:59.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.891+0000 7f4216bed740 1 -- 172.21.15.83:0/4286263541 wait complete. 2024-04-24T08:53:59.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.891+0000 7f4216bed740 20 client.16569 trim_cache size 0 max 16384 2024-04-24T08:53:59.892 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T08:53:59.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.892+0000 7f4212800640 1 -- 172.21.15.83:0/259367946 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f41f40a3da0 msgr2=0x7f41f40a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:59.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.892+0000 7f4212800640 1 --2- 172.21.15.83:0/259367946 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f41f40a3da0 0x7f41f40a6260 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4200031440 tx=0x7f42000058b0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.892+0000 7f4212800640 1 -- 172.21.15.83:0/259367946 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f420c10e6e0 msgr2=0x7f420c19a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:53:59.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.892+0000 7f4212800640 1 --2- 172.21.15.83:0/259367946 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f420c10e6e0 0x7f420c19a730 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f420400cad0 tx=0x7f420400cf90 comp rx=0 tx=0).stop 2024-04-24T08:53:59.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.892+0000 7f4212800640 1 -- 172.21.15.83:0/259367946 shutdown_connections 2024-04-24T08:53:59.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.892+0000 7f4212800640 1 --2- 172.21.15.83:0/259367946 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f41f40a3da0 0x7f41f40a6260 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.892+0000 7f4212800640 1 --2- 172.21.15.83:0/259367946 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f420c10e6e0 0x7f420c19a730 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.892+0000 7f4212800640 1 --2- 172.21.15.83:0/259367946 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f420c074d50 0x7f420c1a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.892+0000 7f4212800640 1 --2- 172.21.15.83:0/259367946 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f420c074430 0x7f420c1a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:53:59.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.892+0000 7f4212800640 1 -- 172.21.15.83:0/259367946 >> 172.21.15.83:0/259367946 conn(0x7f420c0fd9d0 msgr2=0x7f420c0fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:53:59.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.892+0000 7f4212800640 1 -- 172.21.15.83:0/259367946 shutdown_connections 2024-04-24T08:53:59.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:53:59.892+0000 7f4212800640 1 -- 172.21.15.83:0/259367946 wait complete. 2024-04-24T08:53:59.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:53:59 smithi138 ceph-mon[31441]: pgmap v74: 129 pgs: 129 active+clean; 27 MiB data, 468 MiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 468 KiB/s wr, 69 op/s 2024-04-24T08:53:59.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:53:59 smithi189 ceph-mon[31791]: pgmap v74: 129 pgs: 129 active+clean; 27 MiB data, 468 MiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 468 KiB/s wr, 69 op/s 2024-04-24T08:54:00.557 INFO:tasks.workunit.client.0.smithi083.stdout: 1 4653 1.24 MB/sec warmup 34 sec latency 49.341 ms 2024-04-24T08:54:00.911 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T08:54:00.912 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T08:54:00.912 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T08:54:00.970 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.971+0000 7f9d0e800640 1 -- 172.21.15.83:0/1828663565 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d081042a0 msgr2=0x7f9d08104680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:00.971 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.971+0000 7f9d0e800640 1 --2- 172.21.15.83:0/1828663565 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d081042a0 0x7f9d08104680 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc009900 tx=0x7f9cfc02a410 comp rx=0 tx=0).stop 2024-04-24T08:54:00.971 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.971+0000 7f9d0e800640 1 -- 172.21.15.83:0/1828663565 shutdown_connections 2024-04-24T08:54:00.971 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.971+0000 7f9d0e800640 1 --2- 172.21.15.83:0/1828663565 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d08109370 0x7f9d08110f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:00.971 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.971+0000 7f9d0e800640 1 --2- 172.21.15.83:0/1828663565 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9d08104c50 0x7f9d08108c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:00.972 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.971+0000 7f9d0e800640 1 --2- 172.21.15.83:0/1828663565 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d081042a0 0x7f9d08104680 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:00.972 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.971+0000 7f9d0e800640 1 -- 172.21.15.83:0/1828663565 >> 172.21.15.83:0/1828663565 conn(0x7f9d080ffae0 msgr2=0x7f9d08101f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:00.972 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.971+0000 7f9d0e800640 1 -- 172.21.15.83:0/1828663565 shutdown_connections 2024-04-24T08:54:00.972 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.971+0000 7f9d0e800640 1 -- 172.21.15.83:0/1828663565 wait complete. 2024-04-24T08:54:00.974 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.972+0000 7f9d0e800640 1 Processor -- start 2024-04-24T08:54:00.974 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.972+0000 7f9d0e800640 1 -- start start 2024-04-24T08:54:00.974 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.973+0000 7f9d0e800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9d081042a0 0x7f9d081a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:00.974 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.973+0000 7f9d0e800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d08109370 0x7f9d081a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:00.974 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.973+0000 7f9d0e800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d0819a7a0 0x7f9d0819ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:00.975 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.973+0000 7f9d0e800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9d08114320 con 0x7f9d08109370 2024-04-24T08:54:00.975 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.973+0000 7f9d0e800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9d081141a0 con 0x7f9d0819a7a0 2024-04-24T08:54:00.975 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.973+0000 7f9d0e800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9d081144a0 con 0x7f9d081042a0 2024-04-24T08:54:00.975 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.973+0000 7f9d0ca00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d08109370 0x7f9d081a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:00.975 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.973+0000 7f9d0ca00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d08109370 0x7f9d081a0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53068/0 (socket says 172.21.15.83:53068) 2024-04-24T08:54:00.975 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.973+0000 7f9d0ca00640 1 -- 172.21.15.83:0/1994810088 learned_addr learned my addr 172.21.15.83:0/1994810088 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:54:00.975 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.974+0000 7f9d0d400640 1 --2- 172.21.15.83:0/1994810088 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9d081042a0 0x7f9d081a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:00.975 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.974+0000 7f9d0de00640 1 --2- 172.21.15.83:0/1994810088 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d0819a7a0 0x7f9d0819ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:00.975 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.974+0000 7f9d0ca00640 1 -- 172.21.15.83:0/1994810088 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9d081042a0 msgr2=0x7f9d081a0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:00.976 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.974+0000 7f9d0ca00640 1 --2- 172.21.15.83:0/1994810088 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9d081042a0 0x7f9d081a0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:00.976 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.974+0000 7f9d0ca00640 1 -- 172.21.15.83:0/1994810088 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d0819a7a0 msgr2=0x7f9d0819ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:00.976 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.974+0000 7f9d0ca00640 1 --2- 172.21.15.83:0/1994810088 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d0819a7a0 0x7f9d0819ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:00.976 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.974+0000 7f9d0ca00640 1 -- 172.21.15.83:0/1994810088 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d0819b3a0 con 0x7f9d08109370 2024-04-24T08:54:00.976 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.974+0000 7f9d0de00640 1 --2- 172.21.15.83:0/1994810088 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d0819a7a0 0x7f9d0819ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:54:00.976 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.974+0000 7f9d0ca00640 1 --2- 172.21.15.83:0/1994810088 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d08109370 0x7f9d081a0bd0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f9cf400d740 tx=0x7f9cf400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:00.977 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.974+0000 7f9d06000640 1 -- 172.21.15.83:0/1994810088 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cf4017070 con 0x7f9d08109370 2024-04-24T08:54:00.977 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.975+0000 7f9d06000640 1 -- 172.21.15.83:0/1994810088 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9cf400de50 con 0x7f9d08109370 2024-04-24T08:54:00.977 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.975+0000 7f9d0e800640 1 -- 172.21.15.83:0/1994810088 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d081a7430 con 0x7f9d08109370 2024-04-24T08:54:00.977 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.975+0000 7f9d06000640 1 -- 172.21.15.83:0/1994810088 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cf4004d60 con 0x7f9d08109370 2024-04-24T08:54:00.977 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.975+0000 7f9d0e800640 1 -- 172.21.15.83:0/1994810088 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d081a77e0 con 0x7f9d08109370 2024-04-24T08:54:00.978 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.979+0000 7f9d06000640 1 -- 172.21.15.83:0/1994810088 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9cf4004f00 con 0x7f9d08109370 2024-04-24T08:54:00.980 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.979+0000 7f9d12b87740 1 Processor -- start 2024-04-24T08:54:00.980 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.980+0000 7f9d12b87740 1 -- start start 2024-04-24T08:54:00.980 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.980+0000 7f9d12b87740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646806fb3c0 0x5646806fb7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:00.980 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.980+0000 7f9d12b87740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646806fbce0 0x564680706060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:00.980 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.980+0000 7f9d12b87740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5646807065a0 0x564680708990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:00.980 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.980+0000 7f9d12b87740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x564680661900 con 0x5646806fb3c0 2024-04-24T08:54:00.980 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.980+0000 7f9d12b87740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x564680607f20 con 0x5646806fbce0 2024-04-24T08:54:00.980 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.980+0000 7f9d12b87740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56468065d8c0 con 0x5646807065a0 2024-04-24T08:54:00.980 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.980+0000 7f9d06000640 1 --2- 172.21.15.83:0/1994810088 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f9cf00a3d10 0x7f9cf00a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:00.980 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.980+0000 7f9d06000640 1 -- 172.21.15.83:0/1994810088 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f9cf40d06b0 con 0x7f9d08109370 2024-04-24T08:54:00.981 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.981+0000 7f9d0de00640 1 --2- 172.21.15.83:0/1994810088 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f9cf00a3d10 0x7f9cf00a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:00.981 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.981+0000 7f9d0de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646806fb3c0 0x5646806fb7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:00.981 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.981+0000 7f9d0d400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646806fbce0 0x564680706060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:00.981 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.981+0000 7f9d0de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646806fb3c0 0x5646806fb7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53078/0 (socket says 172.21.15.83:53078) 2024-04-24T08:54:00.981 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.981+0000 7f9d0de00640 1 -- 172.21.15.83:0/535638327 learned_addr learned my addr 172.21.15.83:0/535638327 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:54:00.982 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.981+0000 7f9d0de00640 1 -- 172.21.15.83:0/535638327 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5646807065a0 msgr2=0x564680708990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:00.982 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.981+0000 7f9d0de00640 1 --2- 172.21.15.83:0/535638327 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5646807065a0 0x564680708990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:00.982 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.981+0000 7f9d0de00640 1 -- 172.21.15.83:0/535638327 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646806fbce0 msgr2=0x564680706060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:00.982 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.981+0000 7f9d0de00640 1 --2- 172.21.15.83:0/535638327 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646806fbce0 0x564680706060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:00.982 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.981+0000 7f9d0de00640 1 -- 172.21.15.83:0/535638327 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56468062ce10 con 0x5646806fb3c0 2024-04-24T08:54:00.982 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.982+0000 7f9d0de00640 1 --2- 172.21.15.83:0/1994810088 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f9cf00a3d10 0x7f9cf00a61d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9d00009fd0 tx=0x7f9d00009d80 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:00.982 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.982+0000 7f9d0de00640 1 --2- 172.21.15.83:0/535638327 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646806fb3c0 0x5646806fb7a0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f9d00010cd0 tx=0x7f9d0001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37819a455d68e4d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:00.982 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.982+0000 7f9cfa600640 1 -- 172.21.15.83:0/535638327 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d00021070 con 0x5646806fb3c0 2024-04-24T08:54:00.982 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.982+0000 7f9cfa600640 1 -- 172.21.15.83:0/535638327 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9d0000ebf0 con 0x5646806fb3c0 2024-04-24T08:54:00.982 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.982+0000 7f9cfa600640 1 -- 172.21.15.83:0/535638327 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d00025e50 con 0x5646806fb3c0 2024-04-24T08:54:00.983 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.983+0000 7f9d12b87740 1 -- 172.21.15.83:0/535638327 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646806fb3c0 msgr2=0x5646806fb7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:00.983 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.983+0000 7f9d12b87740 1 --2- 172.21.15.83:0/535638327 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646806fb3c0 0x5646806fb7a0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f9d00010cd0 tx=0x7f9d0001a000 comp rx=0 tx=0).stop 2024-04-24T08:54:00.983 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.984+0000 7f9d12b87740 1 -- 172.21.15.83:0/535638327 shutdown_connections 2024-04-24T08:54:00.984 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.984+0000 7f9d12b87740 1 --2- 172.21.15.83:0/535638327 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5646807065a0 0x564680708990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:00.984 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.984+0000 7f9d12b87740 1 --2- 172.21.15.83:0/535638327 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646806fbce0 0x564680706060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:00.984 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.984+0000 7f9d12b87740 1 --2- 172.21.15.83:0/535638327 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646806fb3c0 0x5646806fb7a0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:00.984 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.984+0000 7f9d12b87740 1 -- 172.21.15.83:0/535638327 >> 172.21.15.83:0/535638327 conn(0x5646806f78d0 msgr2=0x5646806f7cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:00.984 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.984+0000 7f9d12b87740 1 -- 172.21.15.83:0/535638327 shutdown_connections 2024-04-24T08:54:00.984 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.984+0000 7f9d12b87740 1 -- 172.21.15.83:0/535638327 wait complete. 2024-04-24T08:54:00.984 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.984+0000 7f9d12b87740 1 -- start start 2024-04-24T08:54:00.987 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.985+0000 7f9d12b87740 1 Processor -- start 2024-04-24T08:54:00.987 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.985+0000 7f9d12b87740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646806fb3c0 0x5646807035f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:00.987 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.985+0000 7f9d12b87740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5646806fbce0 0x564680703b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:00.987 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.985+0000 7f9d12b87740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646807065a0 0x564680704070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:00.987 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.985+0000 7f9d12b87740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x564680709020 con 0x5646807065a0 2024-04-24T08:54:00.987 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.985+0000 7f9d12b87740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56468061b2c0 con 0x5646806fb3c0 2024-04-24T08:54:00.987 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.985+0000 7f9d12b87740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5646807091a0 con 0x5646806fbce0 2024-04-24T08:54:00.987 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.985+0000 7f9d12b87740 10 client.0 _ll_register_callbacks cb 0x564680373a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:54:00.987 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.986+0000 7f9d0de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646807065a0 0x564680704070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:00.987 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.986+0000 7f9d0de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646807065a0 0x564680704070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53090/0 (socket says 172.21.15.83:53090) 2024-04-24T08:54:00.987 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.986+0000 7f9d0de00640 1 -- 172.21.15.83:0/2403488689 learned_addr learned my addr 172.21.15.83:0/2403488689 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:54:00.987 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.986+0000 7f9d0ca00640 1 --2- 172.21.15.83:0/2403488689 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5646806fbce0 0x564680703b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:00.988 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.986+0000 7f9d0d400640 1 --2- 172.21.15.83:0/2403488689 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646806fb3c0 0x5646807035f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:00.988 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.986+0000 7f9d0de00640 1 -- 172.21.15.83:0/2403488689 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5646806fbce0 msgr2=0x564680703b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:00.988 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.986+0000 7f9d0de00640 1 --2- 172.21.15.83:0/2403488689 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5646806fbce0 0x564680703b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:00.988 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.986+0000 7f9d0de00640 1 -- 172.21.15.83:0/2403488689 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646806fb3c0 msgr2=0x5646807035f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:00.988 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.986+0000 7f9d0de00640 1 --2- 172.21.15.83:0/2403488689 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646806fb3c0 0x5646807035f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:00.988 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.986+0000 7f9d0de00640 1 -- 172.21.15.83:0/2403488689 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56468062bfd0 con 0x5646807065a0 2024-04-24T08:54:00.988 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.986+0000 7f9d0ca00640 1 --2- 172.21.15.83:0/2403488689 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5646806fbce0 0x564680703b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:54:00.988 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.987+0000 7f9d0de00640 1 --2- 172.21.15.83:0/2403488689 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646807065a0 0x564680704070 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f9d00009500 tx=0x7f9d0000efd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6c2724854b2eb1d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:00.988 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.987+0000 7f9ceea00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:54:00.988 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.987+0000 7f9ceea00640 1 -- 172.21.15.83:0/2403488689 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d00021040 con 0x5646807065a0 2024-04-24T08:54:00.988 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.987+0000 7f9ceea00640 1 -- 172.21.15.83:0/2403488689 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f9d00025e00 con 0x5646807065a0 2024-04-24T08:54:00.988 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.987+0000 7f9d12b87740 1 -- 172.21.15.83:0/2403488689 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d0000ebf0 con 0x5646807065a0 2024-04-24T08:54:00.988 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.988+0000 7f9ceea00640 1 -- 172.21.15.83:0/2403488689 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9d0003c050 con 0x5646807065a0 2024-04-24T08:54:00.988 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.988+0000 7f9d12b87740 10 client.16581 fetch_fsmap learned FSMap version 30 2024-04-24T08:54:00.989 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.988+0000 7f9d12b87740 1 -- 172.21.15.83:0/2403488689 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564680374210 con 0x5646807065a0 2024-04-24T08:54:00.990 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.988+0000 7f9ceea00640 1 -- 172.21.15.83:0/2403488689 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9d0000ebf0 con 0x5646807065a0 2024-04-24T08:54:00.990 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.988+0000 7f9ceea00640 1 -- 172.21.15.83:0/2403488689 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7f9d00037030 con 0x5646807065a0 2024-04-24T08:54:00.990 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.988+0000 7f9d12b87740 10 client.16581 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T08:54:00.990 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.988+0000 7f9d12b87740 10 client.16581 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T08:54:00.990 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.988+0000 7f9d12b87740 20 client.16581 populate_metadata read hostname 'smithi083' 2024-04-24T08:54:00.990 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.988+0000 7f9d12b87740 1 --2- 172.21.15.83:0/2403488689 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x564680716670 0x564680718a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:00.990 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.989+0000 7f9d12b87740 4 client.16581 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:54:00.990 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.989+0000 7f9d12b87740 1 -- 172.21.15.83:0/2403488689 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56468066f320 con 0x564680716670 2024-04-24T08:54:00.990 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.989+0000 7f9d0d400640 1 --2- 172.21.15.83:0/2403488689 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x564680716670 0x564680718a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:00.990 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.990+0000 7f9d0d400640 1 --2- 172.21.15.83:0/2403488689 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x564680716670 0x564680718a60 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=657d75758960c4d0 server_cookie=1556c3ca11454dee in_seq=0 out_seq=0 2024-04-24T08:54:00.990 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.990+0000 7f9ceea00640 10 client.16581 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:54:00.991 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.992+0000 7f9ceea00640 1 -- 172.21.15.83:0/2403488689 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56468066f320 con 0x564680716670 2024-04-24T08:54:00.992 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.992+0000 7f9ceea00640 10 client.16581 handle_command_reply: tid=0 2024-04-24T08:54:00.992 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.993+0000 7f9d12b87740 1 client.16581 shutdown 2024-04-24T08:54:00.993 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.993+0000 7f9d12b87740 1 -- 172.21.15.83:0/2403488689 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646807065a0 msgr2=0x564680704070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:00.993 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.993+0000 7f9d12b87740 1 --2- 172.21.15.83:0/2403488689 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646807065a0 0x564680704070 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f9d00009500 tx=0x7f9d0000efd0 comp rx=0 tx=0).stop 2024-04-24T08:54:00.993 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.993+0000 7f9d12b87740 1 -- 172.21.15.83:0/2403488689 shutdown_connections 2024-04-24T08:54:00.993 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.993+0000 7f9d12b87740 1 --2- 172.21.15.83:0/2403488689 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x564680716670 0x564680718a60 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:00.993 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.993+0000 7f9d12b87740 1 --2- 172.21.15.83:0/2403488689 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646807065a0 0x564680704070 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:00.993 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.993+0000 7f9d12b87740 1 --2- 172.21.15.83:0/2403488689 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5646806fbce0 0x564680703b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:00.993 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.993+0000 7f9d12b87740 1 --2- 172.21.15.83:0/2403488689 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646806fb3c0 0x5646807035f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:00.993 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.993+0000 7f9d12b87740 1 -- 172.21.15.83:0/2403488689 >> 172.21.15.83:0/2403488689 conn(0x5646806f78d0 msgr2=0x5646806fd600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:00.993 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.993+0000 7f9ceea00640 0 client.16581 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:54:00.994 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.994+0000 7f9d12b87740 1 -- 172.21.15.83:0/2403488689 shutdown_connections 2024-04-24T08:54:00.994 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.994+0000 7f9d12b87740 1 -- 172.21.15.83:0/2403488689 wait complete. 2024-04-24T08:54:00.994 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:00.994+0000 7f9d12b87740 20 client.16581 trim_cache size 0 max 16384 2024-04-24T08:54:01.018 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:01.018+0000 7f9d12b87740 1 Processor -- start 2024-04-24T08:54:01.018 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:01.018+0000 7f9d12b87740 1 -- start start 2024-04-24T08:54:01.018 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:01.018+0000 7f9d12b87740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5646806f78d0 0x564680705d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:01.018 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:01.018+0000 7f9d12b87740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646806fb3c0 0x5646806f7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:01.018 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:01.018+0000 7f9d12b87740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646806fbce0 0x564680701d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:01.018 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:01.018+0000 7f9d12b87740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9d081b1590 con 0x5646806fb3c0 2024-04-24T08:54:01.018 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:01.018+0000 7f9d12b87740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9d081b3160 con 0x5646806fbce0 2024-04-24T08:54:01.018 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:01.018+0000 7f9d12b87740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9d081b1c60 con 0x5646806f78d0 2024-04-24T08:54:01.019 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:01.019+0000 7f9d0ca00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646806fb3c0 0x5646806f7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:01.019 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:01.019+0000 7f9d0ca00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646806fb3c0 0x5646806f7210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53102/0 (socket says 172.21.15.83:53102) 2024-04-24T08:54:01.019 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:01.019+0000 7f9d0ca00640 1 -- 172.21.15.83:0/4149592260 learned_addr learned my addr 172.21.15.83:0/4149592260 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:54:01.020 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:01.019+0000 7f9d0d400640 1 --2- 172.21.15.83:0/4149592260 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5646806f78d0 0x564680705d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:01.020 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:01.019+0000 7f9d0de00640 1 --2- 172.21.15.83:0/4149592260 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646806fbce0 0x564680701d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:01.020 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:01.020+0000 7f9d0ca00640 1 -- 172.21.15.83:0/4149592260 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5646806f78d0 msgr2=0x564680705d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:01.020 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:01.020+0000 7f9d0ca00640 1 --2- 172.21.15.83:0/4149592260 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5646806f78d0 0x564680705d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:01.020 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:01.020+0000 7f9d0ca00640 1 -- 172.21.15.83:0/4149592260 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646806fbce0 msgr2=0x564680701d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:01.020 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:01.020+0000 7f9d0ca00640 1 --2- 172.21.15.83:0/4149592260 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646806fbce0 0x564680701d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:01.020 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:01.020+0000 7f9d0ca00640 1 -- 172.21.15.83:0/4149592260 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564680708d00 con 0x5646806fb3c0 2024-04-24T08:54:01.020 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:01.020+0000 7f9d0de00640 1 --2- 172.21.15.83:0/4149592260 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646806fbce0 0x564680701d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:54:01.020 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:01.020+0000 7f9d0ca00640 1 --2- 172.21.15.83:0/4149592260 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646806fb3c0 0x5646806f7210 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f9cf4009610 tx=0x7f9cf409be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21776695c602cd57 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:01.020 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:01.021+0000 7f9cf9c00640 1 -- 172.21.15.83:0/4149592260 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cf4084070 con 0x5646806fb3c0 2024-04-24T08:54:01.020 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:01.021+0000 7f9cf9c00640 1 -- 172.21.15.83:0/4149592260 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9cf409c220 con 0x5646806fb3c0 2024-04-24T08:54:01.021 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:01.021+0000 7f9cf9c00640 1 -- 172.21.15.83:0/4149592260 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cf4020040 con 0x5646806fb3c0 2024-04-24T08:54:01.021 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:01.022+0000 7f9d12b87740 1 -- 172.21.15.83:0/4149592260 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646806fb3c0 msgr2=0x5646806f7210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:01.021 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:01.022+0000 7f9d12b87740 1 --2- 172.21.15.83:0/4149592260 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646806fb3c0 0x5646806f7210 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f9cf4009610 tx=0x7f9cf409be90 comp rx=0 tx=0).stop 2024-04-24T08:54:01.021 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:01.022+0000 7f9d12b87740 1 -- 172.21.15.83:0/4149592260 shutdown_connections 2024-04-24T08:54:01.022 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:01.022+0000 7f9d12b87740 1 --2- 172.21.15.83:0/4149592260 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646806fbce0 0x564680701d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:01.022 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:01.022+0000 7f9d12b87740 1 --2- 172.21.15.83:0/4149592260 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646806fb3c0 0x5646806f7210 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:01.022 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:01.022+0000 7f9d12b87740 1 --2- 172.21.15.83:0/4149592260 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5646806f78d0 0x564680705d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:01.022 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:01.022+0000 7f9d12b87740 1 -- 172.21.15.83:0/4149592260 >> 172.21.15.83:0/4149592260 conn(0x5646806f2e60 msgr2=0x5646806fd600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:01.022 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:01.022+0000 7f9d12b87740 1 -- 172.21.15.83:0/4149592260 shutdown_connections 2024-04-24T08:54:01.022 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:01.022+0000 7f9d12b87740 1 -- 172.21.15.83:0/4149592260 wait complete. 2024-04-24T08:54:01.023 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:01.023+0000 7f9d12b87740 1 -- start start 2024-04-24T08:54:01.024 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:01.023+0000 7f9d12b87740 1 Processor -- start 2024-04-24T08:54:01.024 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:01.024+0000 7f9d12b87740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5646806f78d0 0x56468070e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:01.024 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:01.024+0000 7f9d12b87740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646806fb3c0 0x5646806ffff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:01.024 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:01.024+0000 7f9d12b87740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646806fbce0 0x564680700570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:01.024 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:01.024+0000 7f9d12b87740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9d081b2a00 con 0x5646806fbce0 2024-04-24T08:54:01.024 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:01.024+0000 7f9d12b87740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9d081b2330 con 0x5646806fb3c0 2024-04-24T08:54:01.024 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:01.024+0000 7f9d12b87740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9d081af570 con 0x5646806f78d0 2024-04-24T08:54:01.024 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:01.024+0000 7f9d12b87740 10 client.0 _ll_register_callbacks cb 0x7f9cfc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:54:01.024 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:01.024+0000 7f9d0de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646806fbce0 0x564680700570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:01.024 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:01.024+0000 7f9d0de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646806fbce0 0x564680700570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53104/0 (socket says 172.21.15.83:53104) 2024-04-24T08:54:01.024 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:01.024+0000 7f9d0de00640 1 -- 172.21.15.83:0/3867212929 learned_addr learned my addr 172.21.15.83:0/3867212929 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:54:01.024 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:01.024+0000 7f9d0ca00640 1 --2- 172.21.15.83:0/3867212929 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646806fb3c0 0x5646806ffff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:01.024 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:01.024+0000 7f9d0de00640 1 -- 172.21.15.83:0/3867212929 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5646806f78d0 msgr2=0x56468070e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:01.024 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:01.024+0000 7f9d0d400640 1 --2- 172.21.15.83:0/3867212929 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5646806f78d0 0x56468070e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:01.025 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:01.025+0000 7f9d0de00640 1 --2- 172.21.15.83:0/3867212929 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5646806f78d0 0x56468070e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:01.025 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:01.025+0000 7f9d0de00640 1 -- 172.21.15.83:0/3867212929 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646806fb3c0 msgr2=0x5646806ffff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:01.025 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:01.025+0000 7f9d0de00640 1 --2- 172.21.15.83:0/3867212929 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646806fb3c0 0x5646806ffff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:01.025 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:01.025+0000 7f9d0de00640 1 -- 172.21.15.83:0/3867212929 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564680440620 con 0x5646806fbce0 2024-04-24T08:54:01.025 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:01.025+0000 7f9d0d400640 1 --2- 172.21.15.83:0/3867212929 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5646806f78d0 0x56468070e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:54:01.026 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:01.025+0000 7f9d0de00640 1 --2- 172.21.15.83:0/3867212929 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646806fbce0 0x564680700570 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f9d00025bc0 tx=0x7f9d00010eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=910dc33e78e5cbe6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:01.026 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:01.025+0000 7f9ceea00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:54:01.026 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:01.025+0000 7f9ceea00640 1 -- 172.21.15.83:0/3867212929 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d00021040 con 0x5646806fbce0 2024-04-24T08:54:01.026 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:01.025+0000 7f9ceea00640 1 -- 172.21.15.83:0/3867212929 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f9d00004510 con 0x5646806fbce0 2024-04-24T08:54:01.026 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:01.026+0000 7f9d12b87740 1 -- 172.21.15.83:0/3867212929 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9cf409c220 con 0x5646806fbce0 2024-04-24T08:54:01.027 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:01.026+0000 7f9ceea00640 1 -- 172.21.15.83:0/3867212929 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9d00009970 con 0x5646806fbce0 2024-04-24T08:54:01.027 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:01.026+0000 7f9d12b87740 10 client.16587 fetch_fsmap learned FSMap version 30 2024-04-24T08:54:01.027 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:01.026+0000 7f9d12b87740 1 -- 172.21.15.83:0/3867212929 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56468077c840 con 0x5646806fbce0 2024-04-24T08:54:01.027 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:01.026+0000 7f9ceea00640 1 -- 172.21.15.83:0/3867212929 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9cf409c220 con 0x5646806fbce0 2024-04-24T08:54:01.027 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:01.026+0000 7f9ceea00640 1 -- 172.21.15.83:0/3867212929 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7f9d00037040 con 0x5646806fbce0 2024-04-24T08:54:01.027 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:01.027+0000 7f9d12b87740 10 client.16587 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T08:54:01.027 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:01.027+0000 7f9d12b87740 10 client.16587 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T08:54:01.027 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:01.027+0000 7f9d12b87740 20 client.16587 populate_metadata read hostname 'smithi083' 2024-04-24T08:54:01.027 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:01.027+0000 7f9d12b87740 1 --2- 172.21.15.83:0/3867212929 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5646807065a0 0x56468077f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:01.027 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:01.027+0000 7f9d12b87740 4 client.16587 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T08:54:01.027 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:01.027+0000 7f9d12b87740 1 -- 172.21.15.83:0/3867212929 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564680669190 con 0x5646807065a0 2024-04-24T08:54:01.027 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:01.027+0000 7f9d0d400640 1 --2- 172.21.15.83:0/3867212929 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5646807065a0 0x56468077f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:01.028 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:01.029+0000 7f9d0d400640 1 --2- 172.21.15.83:0/3867212929 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5646807065a0 0x56468077f000 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e44ee46eddb64fc3 server_cookie=57f9365874da5feb in_seq=0 out_seq=0 2024-04-24T08:54:01.028 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:01.029+0000 7f9ceea00640 10 client.16587 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:54:01.076 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:01.076+0000 7f9ceea00640 1 -- 172.21.15.83:0/3867212929 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564680669190 con 0x5646807065a0 2024-04-24T08:54:01.076 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:01.076+0000 7f9ceea00640 10 client.16587 handle_command_reply: tid=0 2024-04-24T08:54:01.076 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:01.076+0000 7f9d12b87740 1 client.16587 shutdown 2024-04-24T08:54:01.076 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:01.077+0000 7f9d12b87740 1 -- 172.21.15.83:0/3867212929 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646806fbce0 msgr2=0x564680700570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:01.076 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:01.077+0000 7f9d12b87740 1 --2- 172.21.15.83:0/3867212929 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646806fbce0 0x564680700570 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f9d00025bc0 tx=0x7f9d00010eb0 comp rx=0 tx=0).stop 2024-04-24T08:54:01.076 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:01.077+0000 7f9d12b87740 1 -- 172.21.15.83:0/3867212929 shutdown_connections 2024-04-24T08:54:01.076 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:01.077+0000 7f9d12b87740 1 --2- 172.21.15.83:0/3867212929 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5646807065a0 0x56468077f000 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:01.076 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:01.077+0000 7f9d12b87740 1 --2- 172.21.15.83:0/3867212929 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646806fbce0 0x564680700570 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:01.077 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:01.077+0000 7f9d12b87740 1 --2- 172.21.15.83:0/3867212929 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646806fb3c0 0x5646806ffff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:01.077 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:01.077+0000 7f9d12b87740 1 --2- 172.21.15.83:0/3867212929 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5646806f78d0 0x56468070e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:01.077 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:01.077+0000 7f9d12b87740 1 -- 172.21.15.83:0/3867212929 >> 172.21.15.83:0/3867212929 conn(0x5646806f2e60 msgr2=0x5646806f6200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:01.077 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:01.077+0000 7f9ceea00640 0 client.16587 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:54:01.077 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:01.077+0000 7f9d12b87740 1 -- 172.21.15.83:0/3867212929 shutdown_connections 2024-04-24T08:54:01.077 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:01.077+0000 7f9d12b87740 1 -- 172.21.15.83:0/3867212929 wait complete. 2024-04-24T08:54:01.077 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:01.077+0000 7f9d12b87740 20 client.16587 trim_cache size 0 max 16384 2024-04-24T08:54:01.078 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T08:54:01.078 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T08:54:01.078 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T08:54:01.078 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T08:54:01.079 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:01.078+0000 7f9d0e800640 1 -- 172.21.15.83:0/1994810088 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f9cf00a3d10 msgr2=0x7f9cf00a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:01.079 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:01.078+0000 7f9d0e800640 1 --2- 172.21.15.83:0/1994810088 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f9cf00a3d10 0x7f9cf00a61d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9d00009fd0 tx=0x7f9d00009d80 comp rx=0 tx=0).stop 2024-04-24T08:54:01.079 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:01.078+0000 7f9d0e800640 1 -- 172.21.15.83:0/1994810088 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d08109370 msgr2=0x7f9d081a0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:01.079 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:01.078+0000 7f9d0e800640 1 --2- 172.21.15.83:0/1994810088 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d08109370 0x7f9d081a0bd0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f9cf400d740 tx=0x7f9cf400dc00 comp rx=0 tx=0).stop 2024-04-24T08:54:01.079 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:01.078+0000 7f9d0e800640 1 -- 172.21.15.83:0/1994810088 shutdown_connections 2024-04-24T08:54:01.079 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:01.078+0000 7f9d0e800640 1 --2- 172.21.15.83:0/1994810088 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f9cf00a3d10 0x7f9cf00a61d0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:01.079 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:01.078+0000 7f9d0e800640 1 --2- 172.21.15.83:0/1994810088 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d0819a7a0 0x7f9d0819ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:01.079 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:01.078+0000 7f9d0e800640 1 --2- 172.21.15.83:0/1994810088 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d08109370 0x7f9d081a0bd0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:01.079 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:01.078+0000 7f9d0e800640 1 --2- 172.21.15.83:0/1994810088 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9d081042a0 0x7f9d081a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:01.079 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:01.078+0000 7f9d0e800640 1 -- 172.21.15.83:0/1994810088 >> 172.21.15.83:0/1994810088 conn(0x7f9d080ffae0 msgr2=0x7f9d08100ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:01.080 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:01.079+0000 7f9d0e800640 1 -- 172.21.15.83:0/1994810088 shutdown_connections 2024-04-24T08:54:01.080 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:01.079+0000 7f9d0e800640 1 -- 172.21.15.83:0/1994810088 wait complete. 2024-04-24T08:54:01.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.138+0000 7f73a3a00640 1 -- 172.21.15.83:0/2051083802 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f739c107360 msgr2=0x7f739c10ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:01.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.138+0000 7f73a3a00640 1 --2- 172.21.15.83:0/2051083802 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f739c107360 0x7f739c10ef30 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f7394009ca0 tx=0x7f739402a8c0 comp rx=0 tx=0).stop 2024-04-24T08:54:01.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.139+0000 7f73a3a00640 1 -- 172.21.15.83:0/2051083802 shutdown_connections 2024-04-24T08:54:01.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.139+0000 7f73a3a00640 1 --2- 172.21.15.83:0/2051083802 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f739c107360 0x7f739c10ef30 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:01.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.139+0000 7f73a3a00640 1 --2- 172.21.15.83:0/2051083802 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f739c102c40 0x7f739c106c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:01.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.139+0000 7f73a3a00640 1 --2- 172.21.15.83:0/2051083802 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f739c102290 0x7f739c102670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:01.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.139+0000 7f73a3a00640 1 -- 172.21.15.83:0/2051083802 >> 172.21.15.83:0/2051083802 conn(0x7f739c0fdb10 msgr2=0x7f739c0fff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:01.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.139+0000 7f73a3a00640 1 -- 172.21.15.83:0/2051083802 shutdown_connections 2024-04-24T08:54:01.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.139+0000 7f73a3a00640 1 -- 172.21.15.83:0/2051083802 wait complete. 2024-04-24T08:54:01.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.140+0000 7f73a3a00640 1 Processor -- start 2024-04-24T08:54:01.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.140+0000 7f73a3a00640 1 -- start start 2024-04-24T08:54:01.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.140+0000 7f73a3a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f739c102290 0x7f739c1a0720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:01.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.140+0000 7f73a3a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f739c107360 0x7f739c1a0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:01.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.140+0000 7f73a3a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f739c19a8d0 0x7f739c19ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:01.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.140+0000 7f73a3a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f739c112370 con 0x7f739c19a8d0 2024-04-24T08:54:01.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.140+0000 7f73a3a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f739c1121f0 con 0x7f739c102290 2024-04-24T08:54:01.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.140+0000 7f73a3a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f739c1124f0 con 0x7f739c107360 2024-04-24T08:54:01.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.141+0000 7f73a1a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f739c19a8d0 0x7f739c19ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:01.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.141+0000 7f73a1a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f739c19a8d0 0x7f739c19ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53118/0 (socket says 172.21.15.83:53118) 2024-04-24T08:54:01.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.141+0000 7f73a1a00640 1 -- 172.21.15.83:0/1775655907 learned_addr learned my addr 172.21.15.83:0/1775655907 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:54:01.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.141+0000 7f73a1000640 1 --2- 172.21.15.83:0/1775655907 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f739c102290 0x7f739c1a0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:01.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.141+0000 7f739be00640 1 --2- 172.21.15.83:0/1775655907 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f739c107360 0x7f739c1a0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:01.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.141+0000 7f73a1a00640 1 -- 172.21.15.83:0/1775655907 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f739c107360 msgr2=0x7f739c1a0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:01.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.141+0000 7f73a1a00640 1 --2- 172.21.15.83:0/1775655907 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f739c107360 0x7f739c1a0c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:01.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.141+0000 7f73a1a00640 1 -- 172.21.15.83:0/1775655907 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f739c102290 msgr2=0x7f739c1a0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:01.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.141+0000 7f73a1a00640 1 --2- 172.21.15.83:0/1775655907 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f739c102290 0x7f739c1a0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:01.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.141+0000 7f73a1a00640 1 -- 172.21.15.83:0/1775655907 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f739c19b3f0 con 0x7f739c19a8d0 2024-04-24T08:54:01.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.141+0000 7f739be00640 1 --2- 172.21.15.83:0/1775655907 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f739c107360 0x7f739c1a0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:54:01.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.142+0000 7f73a1a00640 1 --2- 172.21.15.83:0/1775655907 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f739c19a8d0 0x7f739c19ad80 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f7394031040 tx=0x7f7394003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:01.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.142+0000 7f7399600640 1 -- 172.21.15.83:0/1775655907 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7394003ce0 con 0x7f739c19a8d0 2024-04-24T08:54:01.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.142+0000 7f7399600640 1 -- 172.21.15.83:0/1775655907 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f73940044f0 con 0x7f739c19a8d0 2024-04-24T08:54:01.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.142+0000 7f7399600640 1 -- 172.21.15.83:0/1775655907 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7394030c90 con 0x7f739c19a8d0 2024-04-24T08:54:01.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.142+0000 7f73a3a00640 1 -- 172.21.15.83:0/1775655907 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f739c19b680 con 0x7f739c19a8d0 2024-04-24T08:54:01.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.142+0000 7f73a3a00640 1 -- 172.21.15.83:0/1775655907 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f739c1a7880 con 0x7f739c19a8d0 2024-04-24T08:54:01.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.146+0000 7f7399600640 1 -- 172.21.15.83:0/1775655907 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f73940046a0 con 0x7f739c19a8d0 2024-04-24T08:54:01.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.146+0000 7f73a6777740 1 Processor -- start 2024-04-24T08:54:01.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.147+0000 7f73a6777740 1 -- start start 2024-04-24T08:54:01.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.147+0000 7f73a6777740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559356d9f090 0x559356d9f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:01.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.147+0000 7f73a6777740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559356d9f9b0 0x559356da9b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:01.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.147+0000 7f73a6777740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559356daa080 0x559356dac470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:01.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.147+0000 7f73a6777740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559356d04e70 con 0x559356d9f9b0 2024-04-24T08:54:01.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.147+0000 7f73a6777740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559356cab380 con 0x559356d9f090 2024-04-24T08:54:01.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.147+0000 7f73a6777740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559356d00e30 con 0x559356daa080 2024-04-24T08:54:01.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.147+0000 7f73a1000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559356d9f9b0 0x559356da9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:01.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.148+0000 7f73a1000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559356d9f9b0 0x559356da9b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53120/0 (socket says 172.21.15.83:53120) 2024-04-24T08:54:01.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.148+0000 7f73a1000640 1 -- 172.21.15.83:0/3931504603 learned_addr learned my addr 172.21.15.83:0/3931504603 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:54:01.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.148+0000 7f739be00640 1 --2- 172.21.15.83:0/3931504603 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559356d9f090 0x559356d9f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:01.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.148+0000 7f739be00640 1 --2- 172.21.15.83:0/3931504603 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559356daa080 0x559356dac470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:01.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.148+0000 7f73a1000640 1 -- 172.21.15.83:0/3931504603 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559356daa080 msgr2=0x559356dac470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:01.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.148+0000 7f73a1000640 1 --2- 172.21.15.83:0/3931504603 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559356daa080 0x559356dac470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:01.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.148+0000 7f73a1000640 1 -- 172.21.15.83:0/3931504603 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559356d9f090 msgr2=0x559356d9f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:01.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.148+0000 7f73a1000640 1 --2- 172.21.15.83:0/3931504603 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559356d9f090 0x559356d9f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:01.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.148+0000 7f73a1000640 1 -- 172.21.15.83:0/3931504603 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559356adf7e0 con 0x559356d9f9b0 2024-04-24T08:54:01.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.148+0000 7f739be00640 1 --2- 172.21.15.83:0/3931504603 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559356daa080 0x559356dac470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:54:01.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.148+0000 7f7399600640 1 --2- 172.21.15.83:0/1775655907 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f73840a3da0 0x7f73840a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:01.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.148+0000 7f73a1000640 1 --2- 172.21.15.83:0/3931504603 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559356d9f9b0 0x559356da9b40 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f739000b7f0 tx=0x7f739000bcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e89e81f4da556eeb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:01.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.149+0000 7f7399600640 1 -- 172.21.15.83:0/1775655907 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f73940f2170 con 0x7f739c19a8d0 2024-04-24T08:54:01.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.149+0000 7f739be00640 1 --2- 172.21.15.83:0/1775655907 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f73840a3da0 0x7f73840a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:01.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.149+0000 7f738e000640 1 -- 172.21.15.83:0/3931504603 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f739000d950 con 0x559356d9f9b0 2024-04-24T08:54:01.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.149+0000 7f738e000640 1 -- 172.21.15.83:0/3931504603 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7390018070 con 0x559356d9f9b0 2024-04-24T08:54:01.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.149+0000 7f738e000640 1 -- 172.21.15.83:0/3931504603 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73900123e0 con 0x559356d9f9b0 2024-04-24T08:54:01.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.150+0000 7f739be00640 1 --2- 172.21.15.83:0/1775655907 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f73840a3da0 0x7f73840a6260 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f73880058c0 tx=0x7f7388015040 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:01.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.150+0000 7f73a6777740 1 -- 172.21.15.83:0/3931504603 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559356d9f9b0 msgr2=0x559356da9b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:01.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.150+0000 7f73a6777740 1 --2- 172.21.15.83:0/3931504603 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559356d9f9b0 0x559356da9b40 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f739000b7f0 tx=0x7f739000bcb0 comp rx=0 tx=0).stop 2024-04-24T08:54:01.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.150+0000 7f73a6777740 1 -- 172.21.15.83:0/3931504603 shutdown_connections 2024-04-24T08:54:01.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.150+0000 7f73a6777740 1 --2- 172.21.15.83:0/3931504603 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559356daa080 0x559356dac470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:01.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.150+0000 7f73a6777740 1 --2- 172.21.15.83:0/3931504603 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559356d9f9b0 0x559356da9b40 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:01.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.150+0000 7f73a6777740 1 --2- 172.21.15.83:0/3931504603 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559356d9f090 0x559356d9f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:01.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.150+0000 7f73a6777740 1 -- 172.21.15.83:0/3931504603 >> 172.21.15.83:0/3931504603 conn(0x559356d9b3b0 msgr2=0x559356d9b790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:01.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.150+0000 7f73a6777740 1 -- 172.21.15.83:0/3931504603 shutdown_connections 2024-04-24T08:54:01.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.150+0000 7f73a6777740 1 -- 172.21.15.83:0/3931504603 wait complete. 2024-04-24T08:54:01.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.150+0000 7f73a6777740 1 -- start start 2024-04-24T08:54:01.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.151+0000 7f73a6777740 1 Processor -- start 2024-04-24T08:54:01.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.151+0000 7f73a6777740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559356d9f090 0x559356da7010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:01.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.151+0000 7f73a6777740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559356d9f9b0 0x559356da7550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:01.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.151+0000 7f73a6777740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559356daa080 0x559356da7a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:01.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.151+0000 7f73a6777740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559356dace70 con 0x559356d9f9b0 2024-04-24T08:54:01.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.151+0000 7f73a6777740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559356cbe660 con 0x559356daa080 2024-04-24T08:54:01.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.151+0000 7f73a6777740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559356dacff0 con 0x559356d9f090 2024-04-24T08:54:01.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.151+0000 7f73a6777740 10 client.0 _ll_register_callbacks cb 0x559356cacaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:54:01.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.151+0000 7f739be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559356d9f9b0 0x559356da7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:01.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.151+0000 7f739be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559356d9f9b0 0x559356da7550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53130/0 (socket says 172.21.15.83:53130) 2024-04-24T08:54:01.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.151+0000 7f739be00640 1 -- 172.21.15.83:0/2038848147 learned_addr learned my addr 172.21.15.83:0/2038848147 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:54:01.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.152+0000 7f73a1000640 1 --2- 172.21.15.83:0/2038848147 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559356d9f090 0x559356da7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:01.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.152+0000 7f73a1a00640 1 --2- 172.21.15.83:0/2038848147 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559356daa080 0x559356da7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:01.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.152+0000 7f739be00640 1 -- 172.21.15.83:0/2038848147 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559356d9f090 msgr2=0x559356da7010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:01.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.152+0000 7f739be00640 1 --2- 172.21.15.83:0/2038848147 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559356d9f090 0x559356da7010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:01.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.152+0000 7f739be00640 1 -- 172.21.15.83:0/2038848147 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559356daa080 msgr2=0x559356da7a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:01.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.152+0000 7f739be00640 1 --2- 172.21.15.83:0/2038848147 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559356daa080 0x559356da7a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:01.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.152+0000 7f739be00640 1 -- 172.21.15.83:0/2038848147 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559356b3ff70 con 0x559356d9f9b0 2024-04-24T08:54:01.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.152+0000 7f73a1a00640 1 --2- 172.21.15.83:0/2038848147 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559356daa080 0x559356da7a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:54:01.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.152+0000 7f739be00640 1 --2- 172.21.15.83:0/2038848147 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559356d9f9b0 0x559356da7550 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f7388010b30 tx=0x7f7388004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0b8f35aba0170b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:01.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.152+0000 7f7382a00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:54:01.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.153+0000 7f7382a00640 1 -- 172.21.15.83:0/2038848147 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7388024030 con 0x559356d9f9b0 2024-04-24T08:54:01.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.153+0000 7f7382a00640 1 -- 172.21.15.83:0/2038848147 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f738800ec30 con 0x559356d9f9b0 2024-04-24T08:54:01.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.153+0000 7f73a6777740 1 -- 172.21.15.83:0/2038848147 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7390018070 con 0x559356d9f9b0 2024-04-24T08:54:01.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.153+0000 7f7382a00640 1 -- 172.21.15.83:0/2038848147 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f738802e050 con 0x559356d9f9b0 2024-04-24T08:54:01.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.153+0000 7f73a6777740 10 client.16599 fetch_fsmap learned FSMap version 30 2024-04-24T08:54:01.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.153+0000 7f73a6777740 1 -- 172.21.15.83:0/2038848147 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5593569d7170 con 0x559356d9f9b0 2024-04-24T08:54:01.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.153+0000 7f7382a00640 1 -- 172.21.15.83:0/2038848147 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7390018070 con 0x559356d9f9b0 2024-04-24T08:54:01.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.153+0000 7f7382a00640 1 -- 172.21.15.83:0/2038848147 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7f738800f040 con 0x559356d9f9b0 2024-04-24T08:54:01.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.153+0000 7f73a6777740 10 client.16599 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T08:54:01.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.153+0000 7f73a6777740 10 client.16599 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:54:01.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.154+0000 7f73a6777740 20 client.16599 populate_metadata read hostname 'smithi083' 2024-04-24T08:54:01.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.154+0000 7f73a6777740 1 --2- 172.21.15.83:0/2038848147 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x559356dba4a0 0x559356dbc930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:01.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.154+0000 7f73a6777740 4 client.16599 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:54:01.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.154+0000 7f73a6777740 1 -- 172.21.15.83:0/2038848147 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559356d12890 con 0x559356dba4a0 2024-04-24T08:54:01.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.154+0000 7f73a1000640 1 --2- 172.21.15.83:0/2038848147 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x559356dba4a0 0x559356dbc930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:01.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.155+0000 7f73a1000640 1 --2- 172.21.15.83:0/2038848147 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x559356dba4a0 0x559356dbc930 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72ad3237c6c7bc4 server_cookie=cfee868c29cfee3e in_seq=0 out_seq=0 2024-04-24T08:54:01.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.155+0000 7f7382a00640 10 client.16599 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:54:01.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.158+0000 7f7382a00640 1 -- 172.21.15.83:0/2038848147 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559356d12890 con 0x559356dba4a0 2024-04-24T08:54:01.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.158+0000 7f7382a00640 10 client.16599 handle_command_reply: tid=0 2024-04-24T08:54:01.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.158+0000 7f73a6777740 1 client.16599 shutdown 2024-04-24T08:54:01.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.158+0000 7f73a6777740 1 -- 172.21.15.83:0/2038848147 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559356d9f9b0 msgr2=0x559356da7550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:01.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.158+0000 7f73a6777740 1 --2- 172.21.15.83:0/2038848147 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559356d9f9b0 0x559356da7550 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f7388010b30 tx=0x7f7388004730 comp rx=0 tx=0).stop 2024-04-24T08:54:01.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.158+0000 7f73a6777740 1 -- 172.21.15.83:0/2038848147 shutdown_connections 2024-04-24T08:54:01.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.158+0000 7f73a6777740 1 --2- 172.21.15.83:0/2038848147 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x559356dba4a0 0x559356dbc930 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:01.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.158+0000 7f73a6777740 1 --2- 172.21.15.83:0/2038848147 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559356daa080 0x559356da7a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:01.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.158+0000 7f73a6777740 1 --2- 172.21.15.83:0/2038848147 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559356d9f9b0 0x559356da7550 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:01.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.158+0000 7f73a6777740 1 --2- 172.21.15.83:0/2038848147 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559356d9f090 0x559356da7010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:01.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.158+0000 7f7382a00640 0 client.16599 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:54:01.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.158+0000 7f73a6777740 1 -- 172.21.15.83:0/2038848147 >> 172.21.15.83:0/2038848147 conn(0x559356d9b3b0 msgr2=0x559356da10b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:01.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.158+0000 7f73a6777740 1 -- 172.21.15.83:0/2038848147 shutdown_connections 2024-04-24T08:54:01.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.158+0000 7f73a6777740 1 -- 172.21.15.83:0/2038848147 wait complete. 2024-04-24T08:54:01.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.159+0000 7f73a6777740 20 client.16599 trim_cache size 0 max 16384 2024-04-24T08:54:01.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.180+0000 7f73a6777740 1 Processor -- start 2024-04-24T08:54:01.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.180+0000 7f73a6777740 1 -- start start 2024-04-24T08:54:01.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.181+0000 7f73a6777740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559356d9f090 0x559356db5d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:01.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.181+0000 7f73a6777740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559356d9f9b0 0x559356da5f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:01.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.181+0000 7f73a6777740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559356daa080 0x559356dc2240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:01.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.181+0000 7f73a6777740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f739c1b16e0 con 0x559356d9f9b0 2024-04-24T08:54:01.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.181+0000 7f73a6777740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f739c1b32b0 con 0x559356d9f090 2024-04-24T08:54:01.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.181+0000 7f73a6777740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f739c1b1db0 con 0x559356daa080 2024-04-24T08:54:01.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.181+0000 7f739be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559356d9f9b0 0x559356da5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:01.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.181+0000 7f739be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559356d9f9b0 0x559356da5f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53146/0 (socket says 172.21.15.83:53146) 2024-04-24T08:54:01.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.181+0000 7f739be00640 1 -- 172.21.15.83:0/3241563917 learned_addr learned my addr 172.21.15.83:0/3241563917 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:54:01.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.181+0000 7f73a1000640 1 --2- 172.21.15.83:0/3241563917 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559356d9f090 0x559356db5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:01.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.181+0000 7f73a1a00640 1 --2- 172.21.15.83:0/3241563917 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559356daa080 0x559356dc2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:01.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.181+0000 7f739be00640 1 -- 172.21.15.83:0/3241563917 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559356daa080 msgr2=0x559356dc2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:01.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.181+0000 7f739be00640 1 --2- 172.21.15.83:0/3241563917 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559356daa080 0x559356dc2240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:01.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.182+0000 7f739be00640 1 -- 172.21.15.83:0/3241563917 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559356d9f090 msgr2=0x559356db5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:01.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.182+0000 7f739be00640 1 --2- 172.21.15.83:0/3241563917 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559356d9f090 0x559356db5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:01.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.182+0000 7f739be00640 1 -- 172.21.15.83:0/3241563917 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559356d9f7a0 con 0x559356d9f9b0 2024-04-24T08:54:01.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.182+0000 7f739be00640 1 --2- 172.21.15.83:0/3241563917 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559356d9f9b0 0x559356da5f50 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f7388010b00 tx=0x7f738800e8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38b33c5a9bb27f1e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:01.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.182+0000 7f738d600640 1 -- 172.21.15.83:0/3241563917 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7388024070 con 0x559356d9f9b0 2024-04-24T08:54:01.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.182+0000 7f738d600640 1 -- 172.21.15.83:0/3241563917 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7388011000 con 0x559356d9f9b0 2024-04-24T08:54:01.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.182+0000 7f738d600640 1 -- 172.21.15.83:0/3241563917 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f738801a070 con 0x559356d9f9b0 2024-04-24T08:54:01.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.183+0000 7f73a6777740 1 -- 172.21.15.83:0/3241563917 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559356d9f9b0 msgr2=0x559356da5f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:01.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.183+0000 7f73a6777740 1 --2- 172.21.15.83:0/3241563917 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559356d9f9b0 0x559356da5f50 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f7388010b00 tx=0x7f738800e8f0 comp rx=0 tx=0).stop 2024-04-24T08:54:01.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.183+0000 7f73a6777740 1 -- 172.21.15.83:0/3241563917 shutdown_connections 2024-04-24T08:54:01.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.183+0000 7f73a6777740 1 --2- 172.21.15.83:0/3241563917 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559356daa080 0x559356dc2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:01.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.183+0000 7f73a6777740 1 --2- 172.21.15.83:0/3241563917 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559356d9f9b0 0x559356da5f50 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:01.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.183+0000 7f73a6777740 1 --2- 172.21.15.83:0/3241563917 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559356d9f090 0x559356db5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:01.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.183+0000 7f73a6777740 1 -- 172.21.15.83:0/3241563917 >> 172.21.15.83:0/3241563917 conn(0x559356d9b3b0 msgr2=0x559356da7860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:01.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.183+0000 7f73a6777740 1 -- 172.21.15.83:0/3241563917 shutdown_connections 2024-04-24T08:54:01.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.183+0000 7f73a6777740 1 -- 172.21.15.83:0/3241563917 wait complete. 2024-04-24T08:54:01.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.183+0000 7f73a6777740 1 -- start start 2024-04-24T08:54:01.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.184+0000 7f73a6777740 1 Processor -- start 2024-04-24T08:54:01.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.184+0000 7f73a6777740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559356d9f090 0x559356db2750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:01.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.184+0000 7f73a6777740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559356d9f9b0 0x559356dadb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:01.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.184+0000 7f73a6777740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559356daa080 0x559356dae250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:01.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.184+0000 7f73a6777740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f739c1b2b50 con 0x559356daa080 2024-04-24T08:54:01.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.184+0000 7f73a6777740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f739c1b2480 con 0x559356d9f090 2024-04-24T08:54:01.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.184+0000 7f73a6777740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f739c1af6c0 con 0x559356d9f9b0 2024-04-24T08:54:01.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.184+0000 7f73a6777740 10 client.0 _ll_register_callbacks cb 0x7f73880107b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:54:01.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.184+0000 7f73a1a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559356daa080 0x559356dae250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:01.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.185+0000 7f73a1a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559356daa080 0x559356dae250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53148/0 (socket says 172.21.15.83:53148) 2024-04-24T08:54:01.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.185+0000 7f73a1a00640 1 -- 172.21.15.83:0/1488039329 learned_addr learned my addr 172.21.15.83:0/1488039329 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:54:01.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.185+0000 7f739be00640 1 --2- 172.21.15.83:0/1488039329 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559356d9f9b0 0x559356dadb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:01.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.185+0000 7f73a1000640 1 --2- 172.21.15.83:0/1488039329 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559356d9f090 0x559356db2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:01.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.185+0000 7f73a1a00640 1 -- 172.21.15.83:0/1488039329 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559356d9f9b0 msgr2=0x559356dadb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:01.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.185+0000 7f73a1a00640 1 --2- 172.21.15.83:0/1488039329 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559356d9f9b0 0x559356dadb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:01.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.185+0000 7f73a1a00640 1 -- 172.21.15.83:0/1488039329 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559356d9f090 msgr2=0x559356db2750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:01.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.185+0000 7f73a1a00640 1 --2- 172.21.15.83:0/1488039329 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559356d9f090 0x559356db2750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:01.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.185+0000 7f73a1a00640 1 -- 172.21.15.83:0/1488039329 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559356e1fab0 con 0x559356daa080 2024-04-24T08:54:01.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.185+0000 7f73a1000640 1 --2- 172.21.15.83:0/1488039329 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559356d9f090 0x559356db2750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:54:01.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.185+0000 7f73a1a00640 1 --2- 172.21.15.83:0/1488039329 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559356daa080 0x559356dae250 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f73940ba820 tx=0x7f73940098b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccba71ba161a7041 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:01.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.185+0000 7f7382a00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:54:01.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.185+0000 7f7382a00640 1 -- 172.21.15.83:0/1488039329 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73940d6070 con 0x559356daa080 2024-04-24T08:54:01.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.186+0000 7f7382a00640 1 -- 172.21.15.83:0/1488039329 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f73940d7260 con 0x559356daa080 2024-04-24T08:54:01.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.186+0000 7f73a6777740 1 -- 172.21.15.83:0/1488039329 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7388011000 con 0x559356daa080 2024-04-24T08:54:01.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.186+0000 7f7382a00640 1 -- 172.21.15.83:0/1488039329 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f73940c2200 con 0x559356daa080 2024-04-24T08:54:01.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.186+0000 7f73a6777740 10 client.16605 fetch_fsmap learned FSMap version 30 2024-04-24T08:54:01.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.186+0000 7f73a6777740 1 -- 172.21.15.83:0/1488039329 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559356e20c90 con 0x559356daa080 2024-04-24T08:54:01.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.186+0000 7f7382a00640 1 -- 172.21.15.83:0/1488039329 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7388011000 con 0x559356daa080 2024-04-24T08:54:01.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.186+0000 7f7382a00640 1 -- 172.21.15.83:0/1488039329 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7f73940c6630 con 0x559356daa080 2024-04-24T08:54:01.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.186+0000 7f73a6777740 10 client.16605 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T08:54:01.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.186+0000 7f73a6777740 10 client.16605 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:54:01.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.186+0000 7f73a6777740 20 client.16605 populate_metadata read hostname 'smithi083' 2024-04-24T08:54:01.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.186+0000 7f73a6777740 1 --2- 172.21.15.83:0/1488039329 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x559356e21440 0x559356e23860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:01.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.186+0000 7f73a6777740 4 client.16605 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T08:54:01.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.186+0000 7f73a6777740 1 -- 172.21.15.83:0/1488039329 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x559356d0c700 con 0x559356e21440 2024-04-24T08:54:01.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.187+0000 7f73a1000640 1 --2- 172.21.15.83:0/1488039329 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x559356e21440 0x559356e23860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:01.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.188+0000 7f73a1000640 1 --2- 172.21.15.83:0/1488039329 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x559356e21440 0x559356e23860 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be637d18f1c8c4b5 server_cookie=f25dc19d09311920 in_seq=0 out_seq=0 2024-04-24T08:54:01.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.188+0000 7f7382a00640 10 client.16605 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:54:01.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.191+0000 7f7382a00640 1 -- 172.21.15.83:0/1488039329 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559356d0c700 con 0x559356e21440 2024-04-24T08:54:01.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.191+0000 7f7382a00640 10 client.16605 handle_command_reply: tid=0 2024-04-24T08:54:01.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.191+0000 7f73a6777740 1 client.16605 shutdown 2024-04-24T08:54:01.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.192+0000 7f73a6777740 1 -- 172.21.15.83:0/1488039329 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559356daa080 msgr2=0x559356dae250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:01.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.192+0000 7f73a6777740 1 --2- 172.21.15.83:0/1488039329 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559356daa080 0x559356dae250 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f73940ba820 tx=0x7f73940098b0 comp rx=0 tx=0).stop 2024-04-24T08:54:01.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.192+0000 7f73a6777740 1 -- 172.21.15.83:0/1488039329 shutdown_connections 2024-04-24T08:54:01.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.192+0000 7f73a6777740 1 --2- 172.21.15.83:0/1488039329 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x559356e21440 0x559356e23860 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:01.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.192+0000 7f73a6777740 1 --2- 172.21.15.83:0/1488039329 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559356daa080 0x559356dae250 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:01.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.192+0000 7f73a6777740 1 --2- 172.21.15.83:0/1488039329 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559356d9f9b0 0x559356dadb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:01.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.192+0000 7f73a6777740 1 --2- 172.21.15.83:0/1488039329 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559356d9f090 0x559356db2750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:01.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.192+0000 7f7382a00640 0 client.16605 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:54:01.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.192+0000 7f73a6777740 1 -- 172.21.15.83:0/1488039329 >> 172.21.15.83:0/1488039329 conn(0x559356d9b3b0 msgr2=0x559356da07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:01.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.192+0000 7f73a6777740 1 -- 172.21.15.83:0/1488039329 shutdown_connections 2024-04-24T08:54:01.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.192+0000 7f73a6777740 1 -- 172.21.15.83:0/1488039329 wait complete. 2024-04-24T08:54:01.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.192+0000 7f73a6777740 20 client.16605 trim_cache size 0 max 16384 2024-04-24T08:54:01.193 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T08:54:01.193 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-04-24T08:54:01.194 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "d6e993f5-7ce9-47b0-9e7d-5d7112a8e7f7", 2024-04-24T08:54:01.194 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-04-24T08:54:01.194 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T08:54:01.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.193+0000 7f73a3a00640 1 -- 172.21.15.83:0/1775655907 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f73840a3da0 msgr2=0x7f73840a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:01.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.193+0000 7f73a3a00640 1 --2- 172.21.15.83:0/1775655907 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f73840a3da0 0x7f73840a6260 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f73880058c0 tx=0x7f7388015040 comp rx=0 tx=0).stop 2024-04-24T08:54:01.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.193+0000 7f73a3a00640 1 -- 172.21.15.83:0/1775655907 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f739c19a8d0 msgr2=0x7f739c19ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:01.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.193+0000 7f73a3a00640 1 --2- 172.21.15.83:0/1775655907 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f739c19a8d0 0x7f739c19ad80 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f7394031040 tx=0x7f7394003970 comp rx=0 tx=0).stop 2024-04-24T08:54:01.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.193+0000 7f73a3a00640 1 -- 172.21.15.83:0/1775655907 shutdown_connections 2024-04-24T08:54:01.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.193+0000 7f73a3a00640 1 --2- 172.21.15.83:0/1775655907 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f73840a3da0 0x7f73840a6260 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:01.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.193+0000 7f73a3a00640 1 --2- 172.21.15.83:0/1775655907 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f739c19a8d0 0x7f739c19ad80 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:01.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.193+0000 7f73a3a00640 1 --2- 172.21.15.83:0/1775655907 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f739c107360 0x7f739c1a0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:01.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.193+0000 7f73a3a00640 1 --2- 172.21.15.83:0/1775655907 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f739c102290 0x7f739c1a0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:01.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.193+0000 7f73a3a00640 1 -- 172.21.15.83:0/1775655907 >> 172.21.15.83:0/1775655907 conn(0x7f739c0fdb10 msgr2=0x7f739c0fee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:01.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.193+0000 7f73a3a00640 1 -- 172.21.15.83:0/1775655907 shutdown_connections 2024-04-24T08:54:01.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.193+0000 7f73a3a00640 1 -- 172.21.15.83:0/1775655907 wait complete. 2024-04-24T08:54:01.210 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T08:54:01.210 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T08:54:01.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.480+0000 7f6d54000640 1 -- 172.21.15.83:0/2199011728 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d4c10ea10 msgr2=0x7f6d4c110e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:01.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.480+0000 7f6d54000640 1 --2- 172.21.15.83:0/2199011728 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d4c10ea10 0x7f6d4c110e00 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f6d44009ca0 tx=0x7f6d4402a8c0 comp rx=0 tx=0).stop 2024-04-24T08:54:01.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.480+0000 7f6d54000640 1 -- 172.21.15.83:0/2199011728 shutdown_connections 2024-04-24T08:54:01.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.480+0000 7f6d54000640 1 --2- 172.21.15.83:0/2199011728 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d4c10ea10 0x7f6d4c110e00 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:01.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.480+0000 7f6d54000640 1 --2- 172.21.15.83:0/2199011728 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d4c06afa0 0x7f6d4c10e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:01.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.480+0000 7f6d54000640 1 --2- 172.21.15.83:0/2199011728 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6d4c06a680 0x7f6d4c06aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:01.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.480+0000 7f6d54000640 1 -- 172.21.15.83:0/2199011728 >> 172.21.15.83:0/2199011728 conn(0x7f6d4c077f60 msgr2=0x7f6d4c078370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:01.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.480+0000 7f6d54000640 1 -- 172.21.15.83:0/2199011728 shutdown_connections 2024-04-24T08:54:01.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.480+0000 7f6d54000640 1 -- 172.21.15.83:0/2199011728 wait complete. 2024-04-24T08:54:01.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.481+0000 7f6d54000640 1 Processor -- start 2024-04-24T08:54:01.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.481+0000 7f6d54000640 1 -- start start 2024-04-24T08:54:01.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.482+0000 7f6d54000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6d4c06a680 0x7f6d4c1a0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:01.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.482+0000 7f6d54000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d4c06afa0 0x7f6d4c1a0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:01.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.482+0000 7f6d54000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d4c10ea10 0x7f6d4c19a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:01.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.482+0000 7f6d54000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6d4c114590 con 0x7f6d4c10ea10 2024-04-24T08:54:01.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.482+0000 7f6d54000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6d4c114410 con 0x7f6d4c06afa0 2024-04-24T08:54:01.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.482+0000 7f6d54000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6d4c114710 con 0x7f6d4c06a680 2024-04-24T08:54:01.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.482+0000 7f6d52000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d4c10ea10 0x7f6d4c19a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:01.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.482+0000 7f6d52000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d4c10ea10 0x7f6d4c19a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53178/0 (socket says 172.21.15.83:53178) 2024-04-24T08:54:01.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.482+0000 7f6d52000640 1 -- 172.21.15.83:0/1958541460 learned_addr learned my addr 172.21.15.83:0/1958541460 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:54:01.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.483+0000 7f6d50c00640 1 --2- 172.21.15.83:0/1958541460 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d4c06afa0 0x7f6d4c1a0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:01.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.483+0000 7f6d51600640 1 --2- 172.21.15.83:0/1958541460 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6d4c06a680 0x7f6d4c1a0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:01.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.483+0000 7f6d52000640 1 -- 172.21.15.83:0/1958541460 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6d4c06a680 msgr2=0x7f6d4c1a0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:01.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.483+0000 7f6d52000640 1 --2- 172.21.15.83:0/1958541460 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6d4c06a680 0x7f6d4c1a0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:01.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.483+0000 7f6d52000640 1 -- 172.21.15.83:0/1958541460 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d4c06afa0 msgr2=0x7f6d4c1a0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:01.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.483+0000 7f6d52000640 1 --2- 172.21.15.83:0/1958541460 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d4c06afa0 0x7f6d4c1a0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:01.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.483+0000 7f6d52000640 1 -- 172.21.15.83:0/1958541460 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d4c19b0b0 con 0x7f6d4c10ea10 2024-04-24T08:54:01.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.483+0000 7f6d51600640 1 --2- 172.21.15.83:0/1958541460 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6d4c06a680 0x7f6d4c1a0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:54:01.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.483+0000 7f6d52000640 1 --2- 172.21.15.83:0/1958541460 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d4c10ea10 0x7f6d4c19a930 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f6d44031040 tx=0x7f6d44003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:01.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.483+0000 7f6d4a000640 1 -- 172.21.15.83:0/1958541460 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d44004040 con 0x7f6d4c10ea10 2024-04-24T08:54:01.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.484+0000 7f6d54000640 1 -- 172.21.15.83:0/1958541460 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d4c19b340 con 0x7f6d4c10ea10 2024-04-24T08:54:01.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.484+0000 7f6d54000640 1 -- 172.21.15.83:0/1958541460 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d4c1a7640 con 0x7f6d4c10ea10 2024-04-24T08:54:01.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.484+0000 7f6d4a000640 1 -- 172.21.15.83:0/1958541460 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6d44004690 con 0x7f6d4c10ea10 2024-04-24T08:54:01.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.484+0000 7f6d4a000640 1 -- 172.21.15.83:0/1958541460 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d44030db0 con 0x7f6d4c10ea10 2024-04-24T08:54:01.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.487+0000 7f6d4a000640 1 -- 172.21.15.83:0/1958541460 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6d440305c0 con 0x7f6d4c10ea10 2024-04-24T08:54:01.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.487+0000 7f6d56fc8740 1 Processor -- start 2024-04-24T08:54:01.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.487+0000 7f6d56fc8740 1 -- start start 2024-04-24T08:54:01.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.488+0000 7f6d56fc8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bf95714860 0x55bf95714c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:01.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.488+0000 7f6d56fc8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bf95715180 0x55bf9571f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:01.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.488+0000 7f6d56fc8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bf9571f850 0x55bf95721c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:01.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.488+0000 7f6d56fc8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55bf9567a1f0 con 0x55bf95715180 2024-04-24T08:54:01.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.488+0000 7f6d56fc8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55bf95620350 con 0x55bf9571f850 2024-04-24T08:54:01.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.488+0000 7f6d56fc8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55bf956761b0 con 0x55bf95714860 2024-04-24T08:54:01.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.488+0000 7f6d51600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bf95715180 0x55bf9571f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:01.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.488+0000 7f6d51600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bf95715180 0x55bf9571f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53182/0 (socket says 172.21.15.83:53182) 2024-04-24T08:54:01.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.488+0000 7f6d51600640 1 -- 172.21.15.83:0/3751467384 learned_addr learned my addr 172.21.15.83:0/3751467384 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:54:01.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.488+0000 7f6d50c00640 1 --2- 172.21.15.83:0/3751467384 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bf95714860 0x55bf95714c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:01.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.489+0000 7f6d51600640 1 -- 172.21.15.83:0/3751467384 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bf95714860 msgr2=0x55bf95714c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:01.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.489+0000 7f6d51600640 1 --2- 172.21.15.83:0/3751467384 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bf95714860 0x55bf95714c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:01.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.489+0000 7f6d51600640 1 -- 172.21.15.83:0/3751467384 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bf9571f850 msgr2=0x55bf95721c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:01.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.489+0000 7f6d51600640 1 --2- 172.21.15.83:0/3751467384 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bf9571f850 0x55bf95721c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:01.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.489+0000 7f6d51600640 1 -- 172.21.15.83:0/3751467384 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf95674f50 con 0x55bf95715180 2024-04-24T08:54:01.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.489+0000 7f6d51600640 1 --2- 172.21.15.83:0/3751467384 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bf95715180 0x55bf9571f310 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f6d4000a9d0 tx=0x7f6d4000ae90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff8a74f7b7e48ff2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:01.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.489+0000 7f6d4a000640 1 --2- 172.21.15.83:0/1958541460 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f6d340a3d20 0x7f6d340a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:01.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.489+0000 7f6d4a000640 1 -- 172.21.15.83:0/1958541460 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f6d440f2320 con 0x7f6d4c10ea10 2024-04-24T08:54:01.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.489+0000 7f6d50c00640 1 --2- 172.21.15.83:0/1958541460 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f6d340a3d20 0x7f6d340a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:01.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.489+0000 7f6d3e600640 1 -- 172.21.15.83:0/3751467384 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d40015070 con 0x55bf95715180 2024-04-24T08:54:01.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.489+0000 7f6d3e600640 1 -- 172.21.15.83:0/3751467384 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6d4000cc60 con 0x55bf95715180 2024-04-24T08:54:01.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.489+0000 7f6d3e600640 1 -- 172.21.15.83:0/3751467384 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d400185c0 con 0x55bf95715180 2024-04-24T08:54:01.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.490+0000 7f6d56fc8740 1 -- 172.21.15.83:0/3751467384 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bf95715180 msgr2=0x55bf9571f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:01.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.490+0000 7f6d56fc8740 1 --2- 172.21.15.83:0/3751467384 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bf95715180 0x55bf9571f310 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f6d4000a9d0 tx=0x7f6d4000ae90 comp rx=0 tx=0).stop 2024-04-24T08:54:01.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.490+0000 7f6d50c00640 1 --2- 172.21.15.83:0/1958541460 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f6d340a3d20 0x7f6d340a61e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x55bf95711d10 tx=0x7f6d3800d1c0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:01.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.491+0000 7f6d56fc8740 1 -- 172.21.15.83:0/3751467384 shutdown_connections 2024-04-24T08:54:01.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.491+0000 7f6d56fc8740 1 --2- 172.21.15.83:0/3751467384 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bf9571f850 0x55bf95721c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:01.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.491+0000 7f6d56fc8740 1 --2- 172.21.15.83:0/3751467384 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bf95715180 0x55bf9571f310 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:01.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.491+0000 7f6d56fc8740 1 --2- 172.21.15.83:0/3751467384 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bf95714860 0x55bf95714c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:01.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.491+0000 7f6d56fc8740 1 -- 172.21.15.83:0/3751467384 >> 172.21.15.83:0/3751467384 conn(0x55bf95710b80 msgr2=0x55bf95710f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:01.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.491+0000 7f6d56fc8740 1 -- 172.21.15.83:0/3751467384 shutdown_connections 2024-04-24T08:54:01.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.491+0000 7f6d56fc8740 1 -- 172.21.15.83:0/3751467384 wait complete. 2024-04-24T08:54:01.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.491+0000 7f6d56fc8740 1 -- start start 2024-04-24T08:54:01.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.491+0000 7f6d56fc8740 1 Processor -- start 2024-04-24T08:54:01.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.492+0000 7f6d56fc8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bf95714860 0x55bf9571c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:01.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.492+0000 7f6d56fc8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bf95715180 0x55bf9571ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:01.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.492+0000 7f6d56fc8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bf9571f850 0x55bf9571d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:01.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.492+0000 7f6d56fc8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55bf95722640 con 0x55bf95714860 2024-04-24T08:54:01.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.492+0000 7f6d56fc8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55bf95633690 con 0x55bf95715180 2024-04-24T08:54:01.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.492+0000 7f6d56fc8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55bf957227c0 con 0x55bf9571f850 2024-04-24T08:54:01.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.492+0000 7f6d56fc8740 10 client.0 _ll_register_callbacks cb 0x55bf9566f650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:54:01.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.492+0000 7f6d51600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bf95714860 0x55bf9571c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:01.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.492+0000 7f6d51600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bf95714860 0x55bf9571c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53198/0 (socket says 172.21.15.83:53198) 2024-04-24T08:54:01.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.492+0000 7f6d51600640 1 -- 172.21.15.83:0/219561282 learned_addr learned my addr 172.21.15.83:0/219561282 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:54:01.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.492+0000 7f6d52000640 1 --2- 172.21.15.83:0/219561282 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bf9571f850 0x55bf9571d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:01.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.493+0000 7f6d50c00640 1 --2- 172.21.15.83:0/219561282 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bf95715180 0x55bf9571ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:01.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.493+0000 7f6d51600640 1 -- 172.21.15.83:0/219561282 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bf9571f850 msgr2=0x55bf9571d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:01.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.493+0000 7f6d51600640 1 --2- 172.21.15.83:0/219561282 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bf9571f850 0x55bf9571d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:01.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.493+0000 7f6d51600640 1 -- 172.21.15.83:0/219561282 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bf95715180 msgr2=0x55bf9571ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:01.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.493+0000 7f6d51600640 1 --2- 172.21.15.83:0/219561282 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bf95715180 0x55bf9571ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:01.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.493+0000 7f6d51600640 1 -- 172.21.15.83:0/219561282 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf95632920 con 0x55bf95714860 2024-04-24T08:54:01.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.493+0000 7f6d51600640 1 --2- 172.21.15.83:0/219561282 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bf95714860 0x55bf9571c7b0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f6d40006aa0 tx=0x7f6d40011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6bac541c4f1483d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:01.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.493+0000 7f6d32a00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:54:01.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.493+0000 7f6d32a00640 1 -- 172.21.15.83:0/219561282 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d40020070 con 0x55bf95714860 2024-04-24T08:54:01.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.493+0000 7f6d32a00640 1 -- 172.21.15.83:0/219561282 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f6d4001a450 con 0x55bf95714860 2024-04-24T08:54:01.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.493+0000 7f6d56fc8740 1 -- 172.21.15.83:0/219561282 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d4000cc60 con 0x55bf95714860 2024-04-24T08:54:01.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.494+0000 7f6d32a00640 1 -- 172.21.15.83:0/219561282 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6d40015040 con 0x55bf95714860 2024-04-24T08:54:01.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.494+0000 7f6d56fc8740 10 client.16617 fetch_fsmap learned FSMap version 30 2024-04-24T08:54:01.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.494+0000 7f6d56fc8740 1 -- 172.21.15.83:0/219561282 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55bf954567d0 con 0x55bf95714860 2024-04-24T08:54:01.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.494+0000 7f6d32a00640 1 -- 172.21.15.83:0/219561282 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6d4000cc60 con 0x55bf95714860 2024-04-24T08:54:01.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.494+0000 7f6d32a00640 1 -- 172.21.15.83:0/219561282 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7f6d40023540 con 0x55bf95714860 2024-04-24T08:54:01.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.494+0000 7f6d56fc8740 10 client.16617 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T08:54:01.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.494+0000 7f6d56fc8740 10 client.16617 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:54:01.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.494+0000 7f6d56fc8740 20 client.16617 populate_metadata read hostname 'smithi083' 2024-04-24T08:54:01.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.494+0000 7f6d56fc8740 1 --2- 172.21.15.83:0/219561282 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55bf9572fab0 0x55bf95731ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:01.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.494+0000 7f6d56fc8740 4 client.16617 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:54:01.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.494+0000 7f6d56fc8740 1 -- 172.21.15.83:0/219561282 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bf95687c10 con 0x55bf9572fab0 2024-04-24T08:54:01.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.495+0000 7f6d50c00640 1 --2- 172.21.15.83:0/219561282 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55bf9572fab0 0x55bf95731ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:01.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.496+0000 7f6d50c00640 1 --2- 172.21.15.83:0/219561282 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55bf9572fab0 0x55bf95731ea0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9684540b394493c3 server_cookie=41ddeb4b198eaed7 in_seq=0 out_seq=0 2024-04-24T08:54:01.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.496+0000 7f6d32a00640 10 client.16617 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:54:01.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.500+0000 7f6d32a00640 1 -- 172.21.15.83:0/219561282 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bf95687c10 con 0x55bf9572fab0 2024-04-24T08:54:01.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.500+0000 7f6d32a00640 10 client.16617 handle_command_reply: tid=0 2024-04-24T08:54:01.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.500+0000 7f6d56fc8740 1 client.16617 shutdown 2024-04-24T08:54:01.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.500+0000 7f6d56fc8740 1 -- 172.21.15.83:0/219561282 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bf95714860 msgr2=0x55bf9571c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:01.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.500+0000 7f6d56fc8740 1 --2- 172.21.15.83:0/219561282 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bf95714860 0x55bf9571c7b0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f6d40006aa0 tx=0x7f6d40011b00 comp rx=0 tx=0).stop 2024-04-24T08:54:01.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.500+0000 7f6d56fc8740 1 -- 172.21.15.83:0/219561282 shutdown_connections 2024-04-24T08:54:01.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.500+0000 7f6d56fc8740 1 --2- 172.21.15.83:0/219561282 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55bf9572fab0 0x55bf95731ea0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:01.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.500+0000 7f6d56fc8740 1 --2- 172.21.15.83:0/219561282 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bf9571f850 0x55bf9571d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:01.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.500+0000 7f6d56fc8740 1 --2- 172.21.15.83:0/219561282 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bf95715180 0x55bf9571ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:01.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.500+0000 7f6d32a00640 0 client.16617 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:54:01.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.500+0000 7f6d56fc8740 1 --2- 172.21.15.83:0/219561282 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bf95714860 0x55bf9571c7b0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:01.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.500+0000 7f6d56fc8740 1 -- 172.21.15.83:0/219561282 >> 172.21.15.83:0/219561282 conn(0x55bf95710b80 msgr2=0x55bf95716880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:01.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.501+0000 7f6d56fc8740 1 -- 172.21.15.83:0/219561282 shutdown_connections 2024-04-24T08:54:01.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.501+0000 7f6d56fc8740 1 -- 172.21.15.83:0/219561282 wait complete. 2024-04-24T08:54:01.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.501+0000 7f6d56fc8740 20 client.16617 trim_cache size 0 max 16384 2024-04-24T08:54:01.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.521+0000 7f6d56fc8740 1 Processor -- start 2024-04-24T08:54:01.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.521+0000 7f6d56fc8740 1 -- start start 2024-04-24T08:54:01.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.521+0000 7f6d56fc8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bf95714860 0x55bf9572b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:01.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.521+0000 7f6d56fc8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bf95715180 0x55bf95717000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:01.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.521+0000 7f6d56fc8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bf9571f850 0x55bf95737660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:01.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.521+0000 7f6d56fc8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6d4c1b1680 con 0x55bf9571f850 2024-04-24T08:54:01.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.521+0000 7f6d56fc8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6d4c1b3250 con 0x55bf95714860 2024-04-24T08:54:01.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.521+0000 7f6d56fc8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6d4c1b1d50 con 0x55bf95715180 2024-04-24T08:54:01.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.522+0000 7f6d52000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bf9571f850 0x55bf95737660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:01.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.522+0000 7f6d52000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bf9571f850 0x55bf95737660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53200/0 (socket says 172.21.15.83:53200) 2024-04-24T08:54:01.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.522+0000 7f6d52000640 1 -- 172.21.15.83:0/3015701330 learned_addr learned my addr 172.21.15.83:0/3015701330 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:54:01.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.522+0000 7f6d50c00640 1 --2- 172.21.15.83:0/3015701330 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bf95715180 0x55bf95717000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:01.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.522+0000 7f6d51600640 1 --2- 172.21.15.83:0/3015701330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bf95714860 0x55bf9572b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:01.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.522+0000 7f6d52000640 1 -- 172.21.15.83:0/3015701330 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bf95715180 msgr2=0x55bf95717000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:01.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.522+0000 7f6d52000640 1 --2- 172.21.15.83:0/3015701330 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bf95715180 0x55bf95717000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:01.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.522+0000 7f6d52000640 1 -- 172.21.15.83:0/3015701330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bf95714860 msgr2=0x55bf9572b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:01.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.522+0000 7f6d52000640 1 --2- 172.21.15.83:0/3015701330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bf95714860 0x55bf9572b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:01.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.522+0000 7f6d52000640 1 -- 172.21.15.83:0/3015701330 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf95732210 con 0x55bf9571f850 2024-04-24T08:54:01.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.522+0000 7f6d50c00640 1 --2- 172.21.15.83:0/3015701330 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bf95715180 0x55bf95717000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:54:01.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.522+0000 7f6d51600640 1 --2- 172.21.15.83:0/3015701330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bf95714860 0x55bf9572b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:54:01.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.522+0000 7f6d52000640 1 --2- 172.21.15.83:0/3015701330 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bf9571f850 0x55bf95737660 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f6d44030f50 tx=0x7f6d44009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6734064b7481b21 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:01.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.523+0000 7f6d3dc00640 1 -- 172.21.15.83:0/3015701330 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d440bdfa0 con 0x55bf9571f850 2024-04-24T08:54:01.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.523+0000 7f6d3dc00640 1 -- 172.21.15.83:0/3015701330 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6d440d7070 con 0x55bf9571f850 2024-04-24T08:54:01.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.523+0000 7f6d3dc00640 1 -- 172.21.15.83:0/3015701330 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d440ba610 con 0x55bf9571f850 2024-04-24T08:54:01.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.523+0000 7f6d56fc8740 1 -- 172.21.15.83:0/3015701330 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bf9571f850 msgr2=0x55bf95737660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:01.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.523+0000 7f6d56fc8740 1 --2- 172.21.15.83:0/3015701330 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bf9571f850 0x55bf95737660 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f6d44030f50 tx=0x7f6d44009360 comp rx=0 tx=0).stop 2024-04-24T08:54:01.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.523+0000 7f6d56fc8740 1 -- 172.21.15.83:0/3015701330 shutdown_connections 2024-04-24T08:54:01.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.523+0000 7f6d56fc8740 1 --2- 172.21.15.83:0/3015701330 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bf9571f850 0x55bf95737660 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:01.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.523+0000 7f6d56fc8740 1 --2- 172.21.15.83:0/3015701330 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bf95715180 0x55bf95717000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:01.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.523+0000 7f6d56fc8740 1 --2- 172.21.15.83:0/3015701330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bf95714860 0x55bf9572b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:01.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.524+0000 7f6d56fc8740 1 -- 172.21.15.83:0/3015701330 >> 172.21.15.83:0/3015701330 conn(0x55bf95710b80 msgr2=0x55bf9570fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:01.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.524+0000 7f6d56fc8740 1 -- 172.21.15.83:0/3015701330 shutdown_connections 2024-04-24T08:54:01.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.524+0000 7f6d56fc8740 1 -- 172.21.15.83:0/3015701330 wait complete. 2024-04-24T08:54:01.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.524+0000 7f6d56fc8740 1 -- start start 2024-04-24T08:54:01.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.524+0000 7f6d56fc8740 1 Processor -- start 2024-04-24T08:54:01.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.525+0000 7f6d56fc8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bf95714860 0x55bf95727e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:01.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.525+0000 7f6d56fc8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bf95715180 0x55bf95719160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:01.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.525+0000 7f6d56fc8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bf9571f850 0x55bf957196a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:01.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.525+0000 7f6d56fc8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6d4c1b2af0 con 0x55bf9571f850 2024-04-24T08:54:01.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.525+0000 7f6d56fc8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6d4c1b2420 con 0x55bf95714860 2024-04-24T08:54:01.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.525+0000 7f6d56fc8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6d4c1af660 con 0x55bf95715180 2024-04-24T08:54:01.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.525+0000 7f6d56fc8740 10 client.0 _ll_register_callbacks cb 0x7f6d40011f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:54:01.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.525+0000 7f6d52000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bf9571f850 0x55bf957196a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:01.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.525+0000 7f6d52000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bf9571f850 0x55bf957196a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53202/0 (socket says 172.21.15.83:53202) 2024-04-24T08:54:01.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.525+0000 7f6d52000640 1 -- 172.21.15.83:0/3885663290 learned_addr learned my addr 172.21.15.83:0/3885663290 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:54:01.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.525+0000 7f6d51600640 1 --2- 172.21.15.83:0/3885663290 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bf95714860 0x55bf95727e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:01.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.525+0000 7f6d50c00640 1 --2- 172.21.15.83:0/3885663290 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bf95715180 0x55bf95719160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:01.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.525+0000 7f6d52000640 1 -- 172.21.15.83:0/3885663290 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bf95715180 msgr2=0x55bf95719160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:01.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.525+0000 7f6d52000640 1 --2- 172.21.15.83:0/3885663290 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bf95715180 0x55bf95719160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:01.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.526+0000 7f6d52000640 1 -- 172.21.15.83:0/3885663290 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bf95714860 msgr2=0x55bf95727e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:01.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.526+0000 7f6d52000640 1 --2- 172.21.15.83:0/3885663290 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bf95714860 0x55bf95727e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:01.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.526+0000 7f6d52000640 1 -- 172.21.15.83:0/3885663290 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf9538c3f0 con 0x55bf9571f850 2024-04-24T08:54:01.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.526+0000 7f6d52000640 1 --2- 172.21.15.83:0/3885663290 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bf9571f850 0x55bf957196a0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f6d440bdca0 tx=0x7f6d440093f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd5a6ce9a1e25b20 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:01.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.526+0000 7f6d32a00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:54:01.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.526+0000 7f6d32a00640 1 -- 172.21.15.83:0/3885663290 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d44087060 con 0x55bf9571f850 2024-04-24T08:54:01.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.526+0000 7f6d32a00640 1 -- 172.21.15.83:0/3885663290 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f6d440efa30 con 0x55bf9571f850 2024-04-24T08:54:01.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.526+0000 7f6d56fc8740 1 -- 172.21.15.83:0/3885663290 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d440d7070 con 0x55bf9571f850 2024-04-24T08:54:01.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.526+0000 7f6d32a00640 1 -- 172.21.15.83:0/3885663290 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6d440c2350 con 0x55bf9571f850 2024-04-24T08:54:01.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.526+0000 7f6d56fc8740 10 client.16623 fetch_fsmap learned FSMap version 30 2024-04-24T08:54:01.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.526+0000 7f6d56fc8740 1 -- 172.21.15.83:0/3885663290 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55bf9534c0f0 con 0x55bf9571f850 2024-04-24T08:54:01.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.527+0000 7f6d32a00640 1 -- 172.21.15.83:0/3885663290 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6d440d7070 con 0x55bf9571f850 2024-04-24T08:54:01.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.527+0000 7f6d32a00640 1 -- 172.21.15.83:0/3885663290 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7f6d440a0030 con 0x55bf9571f850 2024-04-24T08:54:01.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.527+0000 7f6d56fc8740 10 client.16623 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T08:54:01.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.527+0000 7f6d56fc8740 10 client.16623 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:54:01.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.527+0000 7f6d56fc8740 20 client.16623 populate_metadata read hostname 'smithi083' 2024-04-24T08:54:01.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.527+0000 7f6d56fc8740 1 --2- 172.21.15.83:0/3885663290 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55bf95796180 0x55bf957985a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:01.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.527+0000 7f6d56fc8740 4 client.16623 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T08:54:01.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.527+0000 7f6d56fc8740 1 -- 172.21.15.83:0/3885663290 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55bf95681a80 con 0x55bf95796180 2024-04-24T08:54:01.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.527+0000 7f6d51600640 1 --2- 172.21.15.83:0/3885663290 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55bf95796180 0x55bf957985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:01.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.528+0000 7f6d51600640 1 --2- 172.21.15.83:0/3885663290 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55bf95796180 0x55bf957985a0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4761c6232aa87eef server_cookie=750424f3f21d4ace in_seq=0 out_seq=0 2024-04-24T08:54:01.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.528+0000 7f6d32a00640 10 client.16623 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:54:01.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.531+0000 7f6d32a00640 1 -- 172.21.15.83:0/3885663290 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55bf95681a80 con 0x55bf95796180 2024-04-24T08:54:01.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.531+0000 7f6d32a00640 10 client.16623 handle_command_reply: tid=0 2024-04-24T08:54:01.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.531+0000 7f6d56fc8740 1 client.16623 shutdown 2024-04-24T08:54:01.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.531+0000 7f6d56fc8740 1 -- 172.21.15.83:0/3885663290 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bf9571f850 msgr2=0x55bf957196a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:01.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.531+0000 7f6d56fc8740 1 --2- 172.21.15.83:0/3885663290 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bf9571f850 0x55bf957196a0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f6d440bdca0 tx=0x7f6d440093f0 comp rx=0 tx=0).stop 2024-04-24T08:54:01.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.532+0000 7f6d56fc8740 1 -- 172.21.15.83:0/3885663290 shutdown_connections 2024-04-24T08:54:01.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.532+0000 7f6d56fc8740 1 --2- 172.21.15.83:0/3885663290 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55bf95796180 0x55bf957985a0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:01.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.532+0000 7f6d56fc8740 1 --2- 172.21.15.83:0/3885663290 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bf9571f850 0x55bf957196a0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:01.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.532+0000 7f6d56fc8740 1 --2- 172.21.15.83:0/3885663290 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bf95715180 0x55bf95719160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:01.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.532+0000 7f6d56fc8740 1 --2- 172.21.15.83:0/3885663290 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bf95714860 0x55bf95727e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:01.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.532+0000 7f6d56fc8740 1 -- 172.21.15.83:0/3885663290 >> 172.21.15.83:0/3885663290 conn(0x55bf95710b80 msgr2=0x55bf9571c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:01.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.532+0000 7f6d32a00640 0 client.16623 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:54:01.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.532+0000 7f6d56fc8740 1 -- 172.21.15.83:0/3885663290 shutdown_connections 2024-04-24T08:54:01.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.532+0000 7f6d56fc8740 1 -- 172.21.15.83:0/3885663290 wait complete. 2024-04-24T08:54:01.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.532+0000 7f6d56fc8740 20 client.16623 trim_cache size 0 max 16384 2024-04-24T08:54:01.533 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T08:54:01.533 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T08:54:01.533 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-04-24T08:54:01.533 INFO:teuthology.orchestra.run.smithi083.stdout: "d6e993f5-7ce9-47b0-9e7d-5d7112a8e7f7": { 2024-04-24T08:54:01.533 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-04-24T08:54:01.533 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "d6e993f5-7ce9-47b0-9e7d-5d7112a8e7f7", 2024-04-24T08:54:01.533 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-04-24T08:54:01.533 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T08:54:01.534 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T08:54:01.534 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T08:54:01.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.533+0000 7f6d54000640 1 -- 172.21.15.83:0/1958541460 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f6d340a3d20 msgr2=0x7f6d340a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:01.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.533+0000 7f6d54000640 1 --2- 172.21.15.83:0/1958541460 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f6d340a3d20 0x7f6d340a61e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x55bf95711d10 tx=0x7f6d3800d1c0 comp rx=0 tx=0).stop 2024-04-24T08:54:01.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.533+0000 7f6d54000640 1 -- 172.21.15.83:0/1958541460 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d4c10ea10 msgr2=0x7f6d4c19a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:01.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.533+0000 7f6d54000640 1 --2- 172.21.15.83:0/1958541460 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d4c10ea10 0x7f6d4c19a930 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f6d44031040 tx=0x7f6d44003ba0 comp rx=0 tx=0).stop 2024-04-24T08:54:01.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.533+0000 7f6d54000640 1 -- 172.21.15.83:0/1958541460 shutdown_connections 2024-04-24T08:54:01.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.533+0000 7f6d54000640 1 --2- 172.21.15.83:0/1958541460 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f6d340a3d20 0x7f6d340a61e0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:01.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.533+0000 7f6d54000640 1 --2- 172.21.15.83:0/1958541460 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d4c10ea10 0x7f6d4c19a930 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:01.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.533+0000 7f6d54000640 1 --2- 172.21.15.83:0/1958541460 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d4c06afa0 0x7f6d4c1a0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:01.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.533+0000 7f6d54000640 1 --2- 172.21.15.83:0/1958541460 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6d4c06a680 0x7f6d4c1a0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:01.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.533+0000 7f6d54000640 1 -- 172.21.15.83:0/1958541460 >> 172.21.15.83:0/1958541460 conn(0x7f6d4c077f60 msgr2=0x7f6d4c10ff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:01.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.533+0000 7f6d54000640 1 -- 172.21.15.83:0/1958541460 shutdown_connections 2024-04-24T08:54:01.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:01.533+0000 7f6d54000640 1 -- 172.21.15.83:0/1958541460 wait complete. 2024-04-24T08:54:01.554 INFO:tasks.cephfs.filesystem:scrub status for tag:d6e993f5-7ce9-47b0-9e7d-5d7112a8e7f7 - {'path': '/', 'tag': 'd6e993f5-7ce9-47b0-9e7d-5d7112a8e7f7', 'options': 'recursive,force'} 2024-04-24T08:54:01.557 INFO:tasks.workunit.client.0.smithi083.stdout: 1 4780 1.21 MB/sec warmup 35 sec latency 62.118 ms 2024-04-24T08:54:01.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:54:01 smithi138 ceph-mon[31441]: pgmap v75: 129 pgs: 129 active+clean; 27 MiB data, 468 MiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 283 KiB/s wr, 49 op/s 2024-04-24T08:54:01.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:54:01 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T08:54:01.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:54:01 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T08:54:01.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:54:01 smithi189 ceph-mon[31791]: pgmap v75: 129 pgs: 129 active+clean; 27 MiB data, 468 MiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 283 KiB/s wr, 49 op/s 2024-04-24T08:54:01.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:54:01 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T08:54:01.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:54:01 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T08:54:02.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:54:01 smithi083 ceph-mon[27907]: pgmap v75: 129 pgs: 129 active+clean; 27 MiB data, 468 MiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 283 KiB/s wr, 49 op/s 2024-04-24T08:54:02.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:54:01 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T08:54:02.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:54:01 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T08:54:02.558 INFO:tasks.workunit.client.0.smithi083.stdout: 1 4951 1.25 MB/sec warmup 36 sec latency 52.627 ms 2024-04-24T08:54:02.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:54:02 smithi138 ceph-mon[31441]: scrub queued for path: / 2024-04-24T08:54:02.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:54:02 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T08:54:02.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:54:02 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T08:54:02.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:54:02 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T08:54:02.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:54:02 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T08:54:02.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:54:02 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T08:54:02.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:54:02 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T08:54:02.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:54:02 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T08:54:02.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:54:02 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T08:54:02.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:54:02 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T08:54:02.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:54:02 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T08:54:02.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:54:02 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T08:54:02.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:54:02 smithi189 ceph-mon[31791]: scrub queued for path: / 2024-04-24T08:54:02.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:54:02 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T08:54:02.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:54:02 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T08:54:02.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:54:02 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T08:54:02.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:54:02 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T08:54:02.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:54:02 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T08:54:02.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:54:02 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T08:54:02.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:54:02 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T08:54:02.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:54:02 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T08:54:02.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:54:02 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T08:54:02.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:54:02 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T08:54:02.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:54:02 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T08:54:03.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:54:02 smithi083 ceph-mon[27907]: scrub queued for path: / 2024-04-24T08:54:03.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:54:02 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T08:54:03.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:54:02 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T08:54:03.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:54:02 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T08:54:03.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:54:02 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T08:54:03.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:54:02 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T08:54:03.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:54:02 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T08:54:03.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:54:02 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T08:54:03.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:54:02 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T08:54:03.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:54:02 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T08:54:03.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:54:02 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T08:54:03.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:54:02 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T08:54:03.558 INFO:tasks.workunit.client.0.smithi083.stdout: 1 5058 1.22 MB/sec warmup 37 sec latency 53.527 ms 2024-04-24T08:54:03.954 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:54:03 smithi138 ceph-mon[31441]: pgmap v76: 129 pgs: 129 active+clean; 25 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 489 KiB/s wr, 108 op/s 2024-04-24T08:54:03.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:54:03 smithi189 ceph-mon[31791]: pgmap v76: 129 pgs: 129 active+clean; 25 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 489 KiB/s wr, 108 op/s 2024-04-24T08:54:04.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:54:03 smithi083 ceph-mon[27907]: pgmap v76: 129 pgs: 129 active+clean; 25 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 489 KiB/s wr, 108 op/s 2024-04-24T08:54:04.558 INFO:tasks.workunit.client.0.smithi083.stdout: 1 5178 1.19 MB/sec warmup 38 sec latency 57.050 ms 2024-04-24T08:54:05.558 INFO:tasks.workunit.client.0.smithi083.stdout: 1 5334 1.16 MB/sec warmup 39 sec latency 49.285 ms 2024-04-24T08:54:05.954 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:54:05 smithi138 ceph-mon[31441]: pgmap v77: 129 pgs: 129 active+clean; 25 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 276 KiB/s wr, 81 op/s 2024-04-24T08:54:05.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:54:05 smithi189 ceph-mon[31791]: pgmap v77: 129 pgs: 129 active+clean; 25 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 276 KiB/s wr, 81 op/s 2024-04-24T08:54:06.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:54:05 smithi083 ceph-mon[27907]: pgmap v77: 129 pgs: 129 active+clean; 25 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 276 KiB/s wr, 81 op/s 2024-04-24T08:54:06.558 INFO:tasks.workunit.client.0.smithi083.stdout: 1 5434 1.14 MB/sec warmup 40 sec latency 55.260 ms 2024-04-24T08:54:07.558 INFO:tasks.workunit.client.0.smithi083.stdout: 1 5597 1.11 MB/sec warmup 41 sec latency 50.126 ms 2024-04-24T08:54:07.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:54:07 smithi138 ceph-mon[31441]: pgmap v78: 129 pgs: 129 active+clean; 25 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 276 KiB/s wr, 81 op/s 2024-04-24T08:54:07.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:54:07 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T08:54:07.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:54:07 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T08:54:07.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:54:07 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T08:54:07.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:54:07 smithi189 ceph-mon[31791]: pgmap v78: 129 pgs: 129 active+clean; 25 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 276 KiB/s wr, 81 op/s 2024-04-24T08:54:07.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:54:07 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T08:54:07.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:54:07 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T08:54:07.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:54:07 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T08:54:08.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:54:07 smithi083 ceph-mon[27907]: pgmap v78: 129 pgs: 129 active+clean; 25 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 276 KiB/s wr, 81 op/s 2024-04-24T08:54:08.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:54:07 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T08:54:08.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:54:07 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T08:54:08.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:54:07 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T08:54:08.558 INFO:tasks.workunit.client.0.smithi083.stdout: 1 5730 1.09 MB/sec warmup 42 sec latency 56.835 ms 2024-04-24T08:54:08.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:54:08 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T08:54:08.956 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:54:08 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T08:54:09.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:54:08 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T08:54:09.558 INFO:tasks.workunit.client.0.smithi083.stdout: 1 5881 1.09 MB/sec warmup 43 sec latency 54.870 ms 2024-04-24T08:54:09.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:54:09 smithi138 ceph-mon[31441]: pgmap v79: 129 pgs: 129 active+clean; 25 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 320 KiB/s wr, 101 op/s 2024-04-24T08:54:09.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:54:09 smithi189 ceph-mon[31791]: pgmap v79: 129 pgs: 129 active+clean; 25 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 320 KiB/s wr, 101 op/s 2024-04-24T08:54:10.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:54:09 smithi083 ceph-mon[27907]: pgmap v79: 129 pgs: 129 active+clean; 25 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 320 KiB/s wr, 101 op/s 2024-04-24T08:54:10.559 INFO:tasks.workunit.client.0.smithi083.stdout: 1 5958 1.07 MB/sec warmup 44 sec latency 50.516 ms 2024-04-24T08:54:11.559 INFO:tasks.workunit.client.0.smithi083.stdout: 1 6049 1.06 MB/sec warmup 45 sec latency 58.316 ms 2024-04-24T08:54:11.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:54:11 smithi138 ceph-mon[31441]: pgmap v80: 129 pgs: 129 active+clean; 25 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 251 KiB/s wr, 79 op/s 2024-04-24T08:54:11.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:54:11 smithi189 ceph-mon[31791]: pgmap v80: 129 pgs: 129 active+clean; 25 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 251 KiB/s wr, 79 op/s 2024-04-24T08:54:12.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:54:11 smithi083 ceph-mon[27907]: pgmap v80: 129 pgs: 129 active+clean; 25 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 251 KiB/s wr, 79 op/s 2024-04-24T08:54:12.559 INFO:tasks.workunit.client.0.smithi083.stdout: 1 6158 1.05 MB/sec warmup 46 sec latency 53.770 ms 2024-04-24T08:54:13.559 INFO:tasks.workunit.client.0.smithi083.stdout: 1 6353 1.09 MB/sec warmup 47 sec latency 57.014 ms 2024-04-24T08:54:13.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:54:13 smithi138 ceph-mon[31441]: pgmap v81: 129 pgs: 129 active+clean; 27 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 437 KiB/s wr, 98 op/s 2024-04-24T08:54:13.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:54:13 smithi189 ceph-mon[31791]: pgmap v81: 129 pgs: 129 active+clean; 27 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 437 KiB/s wr, 98 op/s 2024-04-24T08:54:14.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:54:13 smithi083 ceph-mon[27907]: pgmap v81: 129 pgs: 129 active+clean; 27 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 437 KiB/s wr, 98 op/s 2024-04-24T08:54:14.228 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.227+0000 7fa099e00640 1 -- 172.21.15.83:0/678859850 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa094108ec0 msgr2=0x7fa094115300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:14.228 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.227+0000 7fa099e00640 1 --2- 172.21.15.83:0/678859850 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa094108ec0 0x7fa094115300 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fa08c009cd0 tx=0x7fa08c02a8e0 comp rx=0 tx=0).stop 2024-04-24T08:54:14.228 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.227+0000 7fa099e00640 1 -- 172.21.15.83:0/678859850 shutdown_connections 2024-04-24T08:54:14.228 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.227+0000 7fa099e00640 1 --2- 172.21.15.83:0/678859850 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa094108ec0 0x7fa094115300 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:14.228 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.227+0000 7fa099e00640 1 --2- 172.21.15.83:0/678859850 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa094108540 0x7fa094108980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:14.228 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.227+0000 7fa099e00640 1 --2- 172.21.15.83:0/678859850 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa09410e4d0 0x7fa09410e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:14.228 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.227+0000 7fa099e00640 1 -- 172.21.15.83:0/678859850 >> 172.21.15.83:0/678859850 conn(0x7fa094077de0 msgr2=0x7fa094106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:14.228 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.227+0000 7fa099e00640 1 -- 172.21.15.83:0/678859850 shutdown_connections 2024-04-24T08:54:14.229 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.227+0000 7fa099e00640 1 -- 172.21.15.83:0/678859850 wait complete. 2024-04-24T08:54:14.231 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.228+0000 7fa099e00640 1 Processor -- start 2024-04-24T08:54:14.231 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.228+0000 7fa099e00640 1 -- start start 2024-04-24T08:54:14.231 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.229+0000 7fa099e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa094108540 0x7fa0941a4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:14.231 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.229+0000 7fa099e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa094108ec0 0x7fa0941a4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:14.231 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.229+0000 7fa099e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa09410e4d0 0x7fa09419ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:14.231 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.229+0000 7fa099e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa0941186e0 con 0x7fa09410e4d0 2024-04-24T08:54:14.231 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.229+0000 7fa099e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa094118560 con 0x7fa094108540 2024-04-24T08:54:14.231 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.229+0000 7fa099e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa094118860 con 0x7fa094108ec0 2024-04-24T08:54:14.231 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.229+0000 7fa099400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa09410e4d0 0x7fa09419ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:14.231 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.229+0000 7fa099400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa09410e4d0 0x7fa09419ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40136/0 (socket says 172.21.15.83:40136) 2024-04-24T08:54:14.232 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.229+0000 7fa099400640 1 -- 172.21.15.83:0/324033804 learned_addr learned my addr 172.21.15.83:0/324033804 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:54:14.232 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.230+0000 7fa093e00640 1 --2- 172.21.15.83:0/324033804 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa094108ec0 0x7fa0941a4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:14.232 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.229+0000 7fa098a00640 1 --2- 172.21.15.83:0/324033804 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa094108540 0x7fa0941a4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:14.232 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.230+0000 7fa099400640 1 -- 172.21.15.83:0/324033804 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa094108ec0 msgr2=0x7fa0941a4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:14.232 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.230+0000 7fa099400640 1 --2- 172.21.15.83:0/324033804 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa094108ec0 0x7fa0941a4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:14.232 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.230+0000 7fa099400640 1 -- 172.21.15.83:0/324033804 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa094108540 msgr2=0x7fa0941a4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:14.232 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.230+0000 7fa099400640 1 --2- 172.21.15.83:0/324033804 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa094108540 0x7fa0941a4850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:14.232 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.230+0000 7fa099400640 1 -- 172.21.15.83:0/324033804 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa09419f180 con 0x7fa09410e4d0 2024-04-24T08:54:14.232 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.230+0000 7fa099400640 1 --2- 172.21.15.83:0/324033804 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa09410e4d0 0x7fa09419ea00 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fa08c005e30 tx=0x7fa08c003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:14.232 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.230+0000 7fa091600640 1 -- 172.21.15.83:0/324033804 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa08c004240 con 0x7fa09410e4d0 2024-04-24T08:54:14.232 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.230+0000 7fa091600640 1 -- 172.21.15.83:0/324033804 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa08c031ce0 con 0x7fa09410e4d0 2024-04-24T08:54:14.232 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.231+0000 7fa091600640 1 -- 172.21.15.83:0/324033804 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa08c046480 con 0x7fa09410e4d0 2024-04-24T08:54:14.232 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.231+0000 7fa099e00640 1 -- 172.21.15.83:0/324033804 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa09419f410 con 0x7fa09410e4d0 2024-04-24T08:54:14.233 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.231+0000 7fa099e00640 1 -- 172.21.15.83:0/324033804 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0941ab710 con 0x7fa09410e4d0 2024-04-24T08:54:14.234 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.234+0000 7fa091600640 1 -- 172.21.15.83:0/324033804 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa08c0043e0 con 0x7fa09410e4d0 2024-04-24T08:54:14.234 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.234+0000 7fa09e1af740 1 Processor -- start 2024-04-24T08:54:14.234 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.234+0000 7fa09e1af740 1 -- start start 2024-04-24T08:54:14.236 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.235+0000 7fa09e1af740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b019fe63c0 0x55b019fe67a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:14.236 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.235+0000 7fa09e1af740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b019fe6ce0 0x55b019ff1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:14.236 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.235+0000 7fa09e1af740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b019ff15a0 0x55b019ff3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:14.236 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.235+0000 7fa09e1af740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55b019f4c900 con 0x55b019fe63c0 2024-04-24T08:54:14.236 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.235+0000 7fa09e1af740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b019ef2f20 con 0x55b019ff15a0 2024-04-24T08:54:14.236 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.235+0000 7fa09e1af740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55b019f488c0 con 0x55b019fe6ce0 2024-04-24T08:54:14.236 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.235+0000 7fa093e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b019fe63c0 0x55b019fe67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:14.236 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.235+0000 7fa093e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b019fe63c0 0x55b019fe67a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40140/0 (socket says 172.21.15.83:40140) 2024-04-24T08:54:14.237 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.235+0000 7fa093e00640 1 -- 172.21.15.83:0/764033873 learned_addr learned my addr 172.21.15.83:0/764033873 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:54:14.237 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.235+0000 7fa098a00640 1 --2- 172.21.15.83:0/764033873 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b019fe6ce0 0x55b019ff1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:14.237 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.236+0000 7fa093e00640 1 --2- 172.21.15.83:0/764033873 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b019ff15a0 0x55b019ff3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:14.237 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.236+0000 7fa093e00640 1 -- 172.21.15.83:0/764033873 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b019fe6ce0 msgr2=0x55b019ff1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:14.237 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.236+0000 7fa093e00640 1 --2- 172.21.15.83:0/764033873 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b019fe6ce0 0x55b019ff1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:14.237 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.236+0000 7fa093e00640 1 -- 172.21.15.83:0/764033873 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b019ff15a0 msgr2=0x55b019ff3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:14.237 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.236+0000 7fa093e00640 1 --2- 172.21.15.83:0/764033873 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b019ff15a0 0x55b019ff3990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:14.237 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.236+0000 7fa093e00640 1 -- 172.21.15.83:0/764033873 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b019f17e10 con 0x55b019fe63c0 2024-04-24T08:54:14.240 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.236+0000 7fa093e00640 1 --2- 172.21.15.83:0/764033873 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b019fe63c0 0x55b019fe67a0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fa080005d00 tx=0x7fa08000b660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=130ff4392c9618ae server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:14.240 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.236+0000 7fa091600640 1 --2- 172.21.15.83:0/324033804 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa07c0a3d90 0x7fa07c0a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:14.240 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.236+0000 7fa085c00640 1 -- 172.21.15.83:0/764033873 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa080005020 con 0x55b019fe63c0 2024-04-24T08:54:14.240 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.236+0000 7fa085c00640 1 -- 172.21.15.83:0/764033873 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa080017030 con 0x55b019fe63c0 2024-04-24T08:54:14.240 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.236+0000 7fa085c00640 1 -- 172.21.15.83:0/764033873 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0800125f0 con 0x55b019fe63c0 2024-04-24T08:54:14.240 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.237+0000 7fa091600640 1 -- 172.21.15.83:0/324033804 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7fa08c02e030 con 0x7fa09410e4d0 2024-04-24T08:54:14.240 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.237+0000 7fa098a00640 1 --2- 172.21.15.83:0/324033804 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa07c0a3d90 0x7fa07c0a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:14.240 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.237+0000 7fa09e1af740 1 -- 172.21.15.83:0/764033873 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b019fe63c0 msgr2=0x55b019fe67a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:14.240 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.237+0000 7fa09e1af740 1 --2- 172.21.15.83:0/764033873 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b019fe63c0 0x55b019fe67a0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fa080005d00 tx=0x7fa08000b660 comp rx=0 tx=0).stop 2024-04-24T08:54:14.240 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.237+0000 7fa09e1af740 1 -- 172.21.15.83:0/764033873 shutdown_connections 2024-04-24T08:54:14.240 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.237+0000 7fa09e1af740 1 --2- 172.21.15.83:0/764033873 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b019ff15a0 0x55b019ff3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:14.241 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.237+0000 7fa09e1af740 1 --2- 172.21.15.83:0/764033873 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b019fe6ce0 0x55b019ff1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:14.241 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.237+0000 7fa09e1af740 1 --2- 172.21.15.83:0/764033873 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b019fe63c0 0x55b019fe67a0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:14.241 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.238+0000 7fa09e1af740 1 -- 172.21.15.83:0/764033873 >> 172.21.15.83:0/764033873 conn(0x55b019fe28d0 msgr2=0x55b019fe2cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:14.241 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.238+0000 7fa09e1af740 1 -- 172.21.15.83:0/764033873 shutdown_connections 2024-04-24T08:54:14.241 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.238+0000 7fa098a00640 1 --2- 172.21.15.83:0/324033804 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa07c0a3d90 0x7fa07c0a6250 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa088005fd0 tx=0x7fa08800b5d0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:14.241 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.238+0000 7fa09e1af740 1 -- 172.21.15.83:0/764033873 wait complete. 2024-04-24T08:54:14.241 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.238+0000 7fa09e1af740 1 -- start start 2024-04-24T08:54:14.241 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.238+0000 7fa09e1af740 1 Processor -- start 2024-04-24T08:54:14.241 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.239+0000 7fa09e1af740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b019fe63c0 0x55b019fee640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:14.241 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.239+0000 7fa09e1af740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b019fe6ce0 0x55b019feeb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:14.241 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.239+0000 7fa09e1af740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b019ff15a0 0x55b019fef0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:14.241 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.239+0000 7fa09e1af740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55b019ff40e0 con 0x55b019fe6ce0 2024-04-24T08:54:14.242 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.239+0000 7fa09e1af740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b019f062c0 con 0x55b019fe63c0 2024-04-24T08:54:14.242 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.239+0000 7fa09e1af740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55b019ff4260 con 0x55b019ff15a0 2024-04-24T08:54:14.242 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.239+0000 7fa09e1af740 10 client.0 _ll_register_callbacks cb 0x55b019c5ea00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:54:14.242 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.239+0000 7fa093e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b019fe6ce0 0x55b019feeb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:14.242 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.240+0000 7fa093e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b019fe6ce0 0x55b019feeb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40148/0 (socket says 172.21.15.83:40148) 2024-04-24T08:54:14.242 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.240+0000 7fa093e00640 1 -- 172.21.15.83:0/2571847015 learned_addr learned my addr 172.21.15.83:0/2571847015 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:54:14.242 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.240+0000 7fa098a00640 1 --2- 172.21.15.83:0/2571847015 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b019fe63c0 0x55b019fee640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:14.242 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.240+0000 7fa099400640 1 --2- 172.21.15.83:0/2571847015 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b019ff15a0 0x55b019fef0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:14.244 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.240+0000 7fa093e00640 1 -- 172.21.15.83:0/2571847015 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b019ff15a0 msgr2=0x55b019fef0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:14.244 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.240+0000 7fa093e00640 1 --2- 172.21.15.83:0/2571847015 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b019ff15a0 0x55b019fef0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:14.244 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.240+0000 7fa093e00640 1 -- 172.21.15.83:0/2571847015 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b019fe63c0 msgr2=0x55b019fee640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:14.244 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.240+0000 7fa093e00640 1 --2- 172.21.15.83:0/2571847015 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b019fe63c0 0x55b019fee640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:14.244 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.240+0000 7fa093e00640 1 -- 172.21.15.83:0/2571847015 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b019f16fd0 con 0x55b019fe6ce0 2024-04-24T08:54:14.244 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.240+0000 7fa093e00640 1 --2- 172.21.15.83:0/2571847015 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b019fe6ce0 0x55b019feeb80 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fa080005bc0 tx=0x7fa080011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81f546a89410f70e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:14.244 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.241+0000 7fa07a000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:54:14.245 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.241+0000 7fa07a000640 1 -- 172.21.15.83:0/2571847015 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa08002a060 con 0x55b019fe6ce0 2024-04-24T08:54:14.245 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.241+0000 7fa07a000640 1 -- 172.21.15.83:0/2571847015 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7fa0800048c0 con 0x55b019fe6ce0 2024-04-24T08:54:14.245 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.241+0000 7fa09e1af740 1 -- 172.21.15.83:0/2571847015 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa080017030 con 0x55b019fe6ce0 2024-04-24T08:54:14.245 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.241+0000 7fa07a000640 1 -- 172.21.15.83:0/2571847015 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa080031050 con 0x55b019fe6ce0 2024-04-24T08:54:14.245 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.241+0000 7fa09e1af740 10 client.16635 fetch_fsmap learned FSMap version 30 2024-04-24T08:54:14.245 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.241+0000 7fa09e1af740 1 -- 172.21.15.83:0/2571847015 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b019c5f210 con 0x55b019fe6ce0 2024-04-24T08:54:14.245 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.242+0000 7fa07a000640 1 -- 172.21.15.83:0/2571847015 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa080017030 con 0x55b019fe6ce0 2024-04-24T08:54:14.245 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.242+0000 7fa07a000640 1 -- 172.21.15.83:0/2571847015 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7fa080025810 con 0x55b019fe6ce0 2024-04-24T08:54:14.245 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.242+0000 7fa09e1af740 10 client.16635 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T08:54:14.245 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.242+0000 7fa09e1af740 10 client.16635 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T08:54:14.245 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.242+0000 7fa09e1af740 20 client.16635 populate_metadata read hostname 'smithi083' 2024-04-24T08:54:14.245 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.242+0000 7fa09e1af740 1 --2- 172.21.15.83:0/2571847015 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55b01a001690 0x55b01a003a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:14.245 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.242+0000 7fa09e1af740 4 client.16635 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:54:14.245 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.242+0000 7fa09e1af740 1 -- 172.21.15.83:0/2571847015 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b019f5a320 con 0x55b01a001690 2024-04-24T08:54:14.245 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.242+0000 7fa098a00640 1 --2- 172.21.15.83:0/2571847015 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55b01a001690 0x55b01a003a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:14.246 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.243+0000 7fa098a00640 1 --2- 172.21.15.83:0/2571847015 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55b01a001690 0x55b01a003a80 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a2a8453d2029f5e server_cookie=6e0d21a60805651d in_seq=0 out_seq=0 2024-04-24T08:54:14.246 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.243+0000 7fa07a000640 10 client.16635 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:54:14.247 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.246+0000 7fa07a000640 1 -- 172.21.15.83:0/2571847015 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b019f5a320 con 0x55b01a001690 2024-04-24T08:54:14.247 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.246+0000 7fa07a000640 10 client.16635 handle_command_reply: tid=0 2024-04-24T08:54:14.247 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.246+0000 7fa09e1af740 1 client.16635 shutdown 2024-04-24T08:54:14.247 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.246+0000 7fa09e1af740 1 -- 172.21.15.83:0/2571847015 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b019fe6ce0 msgr2=0x55b019feeb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:14.247 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.246+0000 7fa09e1af740 1 --2- 172.21.15.83:0/2571847015 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b019fe6ce0 0x55b019feeb80 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fa080005bc0 tx=0x7fa080011e20 comp rx=0 tx=0).stop 2024-04-24T08:54:14.247 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.247+0000 7fa09e1af740 1 -- 172.21.15.83:0/2571847015 shutdown_connections 2024-04-24T08:54:14.247 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.247+0000 7fa09e1af740 1 --2- 172.21.15.83:0/2571847015 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55b01a001690 0x55b01a003a80 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:14.247 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.247+0000 7fa09e1af740 1 --2- 172.21.15.83:0/2571847015 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b019ff15a0 0x55b019fef0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:14.247 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.247+0000 7fa09e1af740 1 --2- 172.21.15.83:0/2571847015 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b019fe6ce0 0x55b019feeb80 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:14.247 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.247+0000 7fa07a000640 0 client.16635 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:54:14.247 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.247+0000 7fa09e1af740 1 --2- 172.21.15.83:0/2571847015 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b019fe63c0 0x55b019fee640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:14.247 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.247+0000 7fa09e1af740 1 -- 172.21.15.83:0/2571847015 >> 172.21.15.83:0/2571847015 conn(0x55b019fe28d0 msgr2=0x55b019fe8650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:14.248 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.247+0000 7fa09e1af740 1 -- 172.21.15.83:0/2571847015 shutdown_connections 2024-04-24T08:54:14.248 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.247+0000 7fa09e1af740 1 -- 172.21.15.83:0/2571847015 wait complete. 2024-04-24T08:54:14.248 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.247+0000 7fa09e1af740 20 client.16635 trim_cache size 0 max 16384 2024-04-24T08:54:14.264 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.264+0000 7fa09e1af740 1 Processor -- start 2024-04-24T08:54:14.265 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.265+0000 7fa09e1af740 1 -- start start 2024-04-24T08:54:14.266 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.265+0000 7fa09e1af740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b019fe63c0 0x55b019ffd060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:14.266 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.265+0000 7fa09e1af740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b019fe6ce0 0x55b019fe2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:14.266 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.265+0000 7fa09e1af740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b019ff15a0 0x55b01a0090b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:14.266 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.265+0000 7fa09e1af740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa0941b58f0 con 0x55b019fe63c0 2024-04-24T08:54:14.266 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.265+0000 7fa09e1af740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa0941b74c0 con 0x55b019fe6ce0 2024-04-24T08:54:14.266 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.265+0000 7fa09e1af740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa0941b5fc0 con 0x55b019ff15a0 2024-04-24T08:54:14.266 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.265+0000 7fa098a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b019fe63c0 0x55b019ffd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:14.266 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.265+0000 7fa098a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b019fe63c0 0x55b019ffd060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40150/0 (socket says 172.21.15.83:40150) 2024-04-24T08:54:14.266 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.265+0000 7fa098a00640 1 -- 172.21.15.83:0/201328549 learned_addr learned my addr 172.21.15.83:0/201328549 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:54:14.266 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.266+0000 7fa098a00640 1 -- 172.21.15.83:0/201328549 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b019ff15a0 msgr2=0x55b01a0090b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:14.267 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.266+0000 7fa098a00640 1 --2- 172.21.15.83:0/201328549 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b019ff15a0 0x55b01a0090b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:14.267 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.266+0000 7fa098a00640 1 -- 172.21.15.83:0/201328549 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b019fe6ce0 msgr2=0x55b019fe2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:14.267 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.266+0000 7fa098a00640 1 --2- 172.21.15.83:0/201328549 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b019fe6ce0 0x55b019fe2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:14.267 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.266+0000 7fa098a00640 1 -- 172.21.15.83:0/201328549 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b019d2b620 con 0x55b019fe63c0 2024-04-24T08:54:14.267 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.266+0000 7fa098a00640 1 --2- 172.21.15.83:0/201328549 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b019fe63c0 0x55b019ffd060 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fa08800b370 tx=0x7fa088014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3baac03c9fbb8923 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:14.271 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.266+0000 7fa085200640 1 -- 172.21.15.83:0/201328549 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa08802d070 con 0x55b019fe63c0 2024-04-24T08:54:14.271 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.266+0000 7fa085200640 1 -- 172.21.15.83:0/201328549 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa088003d80 con 0x55b019fe63c0 2024-04-24T08:54:14.271 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.266+0000 7fa085200640 1 -- 172.21.15.83:0/201328549 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa088012800 con 0x55b019fe63c0 2024-04-24T08:54:14.271 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.267+0000 7fa09e1af740 1 -- 172.21.15.83:0/201328549 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b019fe63c0 msgr2=0x55b019ffd060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:14.271 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.267+0000 7fa09e1af740 1 --2- 172.21.15.83:0/201328549 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b019fe63c0 0x55b019ffd060 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fa08800b370 tx=0x7fa088014000 comp rx=0 tx=0).stop 2024-04-24T08:54:14.271 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.267+0000 7fa09e1af740 1 -- 172.21.15.83:0/201328549 shutdown_connections 2024-04-24T08:54:14.271 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.267+0000 7fa09e1af740 1 --2- 172.21.15.83:0/201328549 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b019ff15a0 0x55b01a0090b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:14.271 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.267+0000 7fa09e1af740 1 --2- 172.21.15.83:0/201328549 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b019fe6ce0 0x55b019fe2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:14.271 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.267+0000 7fa09e1af740 1 --2- 172.21.15.83:0/201328549 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b019fe63c0 0x55b019ffd060 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:14.271 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.267+0000 7fa09e1af740 1 -- 172.21.15.83:0/201328549 >> 172.21.15.83:0/201328549 conn(0x55b019fe28d0 msgr2=0x55b019feee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:14.271 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.267+0000 7fa09e1af740 1 -- 172.21.15.83:0/201328549 shutdown_connections 2024-04-24T08:54:14.271 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.267+0000 7fa09e1af740 1 -- 172.21.15.83:0/201328549 wait complete. 2024-04-24T08:54:14.272 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.268+0000 7fa09e1af740 1 -- start start 2024-04-24T08:54:14.272 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.268+0000 7fa09e1af740 1 Processor -- start 2024-04-24T08:54:14.272 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.268+0000 7fa09e1af740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b019fe63c0 0x55b019ff5230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:14.272 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.268+0000 7fa09e1af740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b019fe6ce0 0x55b019ff5770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:14.272 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.268+0000 7fa09e1af740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b019ff15a0 0x55b019ff5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:14.272 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.268+0000 7fa09e1af740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa0941b6d60 con 0x55b019fe6ce0 2024-04-24T08:54:14.272 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.268+0000 7fa09e1af740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa0941b6690 con 0x55b019fe63c0 2024-04-24T08:54:14.272 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.268+0000 7fa09e1af740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa0941b38d0 con 0x55b019ff15a0 2024-04-24T08:54:14.272 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.268+0000 7fa09e1af740 10 client.0 _ll_register_callbacks cb 0x7fa08000bd70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:54:14.272 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.268+0000 7fa093e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b019fe6ce0 0x55b019ff5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:14.272 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.269+0000 7fa093e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b019fe6ce0 0x55b019ff5770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40156/0 (socket says 172.21.15.83:40156) 2024-04-24T08:54:14.272 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.269+0000 7fa093e00640 1 -- 172.21.15.83:0/2912367772 learned_addr learned my addr 172.21.15.83:0/2912367772 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:54:14.272 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.269+0000 7fa099400640 1 --2- 172.21.15.83:0/2912367772 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b019ff15a0 0x55b019ff5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:14.273 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.269+0000 7fa098a00640 1 --2- 172.21.15.83:0/2912367772 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b019fe63c0 0x55b019ff5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:14.273 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.269+0000 7fa093e00640 1 -- 172.21.15.83:0/2912367772 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b019ff15a0 msgr2=0x55b019ff5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:14.273 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.269+0000 7fa093e00640 1 --2- 172.21.15.83:0/2912367772 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b019ff15a0 0x55b019ff5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:14.273 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.269+0000 7fa093e00640 1 -- 172.21.15.83:0/2912367772 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b019fe63c0 msgr2=0x55b019ff5230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:14.273 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.269+0000 7fa093e00640 1 --2- 172.21.15.83:0/2912367772 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b019fe63c0 0x55b019ff5230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:14.273 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.269+0000 7fa093e00640 1 -- 172.21.15.83:0/2912367772 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b01a066bc0 con 0x55b019fe6ce0 2024-04-24T08:54:14.273 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.269+0000 7fa093e00640 1 --2- 172.21.15.83:0/2912367772 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b019fe6ce0 0x55b019ff5770 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fa08000bd00 tx=0x7fa080011be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a4a4b725e172c48 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:14.273 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.269+0000 7fa07a000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:54:14.273 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.270+0000 7fa07a000640 1 -- 172.21.15.83:0/2912367772 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa080039020 con 0x55b019fe6ce0 2024-04-24T08:54:14.273 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.270+0000 7fa07a000640 1 -- 172.21.15.83:0/2912367772 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7fa080025480 con 0x55b019fe6ce0 2024-04-24T08:54:14.273 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.270+0000 7fa09e1af740 1 -- 172.21.15.83:0/2912367772 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa088003d80 con 0x55b019fe6ce0 2024-04-24T08:54:14.273 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.270+0000 7fa07a000640 1 -- 172.21.15.83:0/2912367772 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa080041050 con 0x55b019fe6ce0 2024-04-24T08:54:14.273 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.270+0000 7fa09e1af740 10 client.16641 fetch_fsmap learned FSMap version 30 2024-04-24T08:54:14.273 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.270+0000 7fa09e1af740 1 -- 172.21.15.83:0/2912367772 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b01a067ed0 con 0x55b019fe6ce0 2024-04-24T08:54:14.273 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.270+0000 7fa07a000640 1 -- 172.21.15.83:0/2912367772 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa088003d80 con 0x55b019fe6ce0 2024-04-24T08:54:14.274 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.271+0000 7fa07a000640 1 -- 172.21.15.83:0/2912367772 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7fa08001c6f0 con 0x55b019fe6ce0 2024-04-24T08:54:14.274 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.271+0000 7fa09e1af740 10 client.16641 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T08:54:14.274 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.271+0000 7fa09e1af740 10 client.16641 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T08:54:14.274 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.271+0000 7fa09e1af740 20 client.16641 populate_metadata read hostname 'smithi083' 2024-04-24T08:54:14.274 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.271+0000 7fa09e1af740 1 --2- 172.21.15.83:0/2912367772 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55b01a068670 0x55b01a06aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:14.274 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.271+0000 7fa09e1af740 4 client.16641 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T08:54:14.274 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.271+0000 7fa09e1af740 1 -- 172.21.15.83:0/2912367772 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b019f54190 con 0x55b01a068670 2024-04-24T08:54:14.275 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.271+0000 7fa098a00640 1 --2- 172.21.15.83:0/2912367772 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55b01a068670 0x55b01a06aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:14.275 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.272+0000 7fa07a000640 10 client.16641 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:54:14.275 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.272+0000 7fa098a00640 1 --2- 172.21.15.83:0/2912367772 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55b01a068670 0x55b01a06aa90 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bfc9d47fb817beeb server_cookie=6e5a027275470609 in_seq=0 out_seq=0 2024-04-24T08:54:14.325 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.325+0000 7fa07a000640 1 -- 172.21.15.83:0/2912367772 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b019f54190 con 0x55b01a068670 2024-04-24T08:54:14.325 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.325+0000 7fa07a000640 10 client.16641 handle_command_reply: tid=0 2024-04-24T08:54:14.325 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.325+0000 7fa09e1af740 1 client.16641 shutdown 2024-04-24T08:54:14.326 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.326+0000 7fa09e1af740 1 -- 172.21.15.83:0/2912367772 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b019fe6ce0 msgr2=0x55b019ff5770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:14.327 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.326+0000 7fa09e1af740 1 --2- 172.21.15.83:0/2912367772 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b019fe6ce0 0x55b019ff5770 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fa08000bd00 tx=0x7fa080011be0 comp rx=0 tx=0).stop 2024-04-24T08:54:14.327 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.326+0000 7fa09e1af740 1 -- 172.21.15.83:0/2912367772 shutdown_connections 2024-04-24T08:54:14.327 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.326+0000 7fa09e1af740 1 --2- 172.21.15.83:0/2912367772 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55b01a068670 0x55b01a06aa90 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:14.327 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.326+0000 7fa09e1af740 1 --2- 172.21.15.83:0/2912367772 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b019ff15a0 0x55b019ff5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:14.327 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.326+0000 7fa09e1af740 1 --2- 172.21.15.83:0/2912367772 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b019fe6ce0 0x55b019ff5770 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:14.327 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.326+0000 7fa09e1af740 1 --2- 172.21.15.83:0/2912367772 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b019fe63c0 0x55b019ff5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:14.327 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.326+0000 7fa09e1af740 1 -- 172.21.15.83:0/2912367772 >> 172.21.15.83:0/2912367772 conn(0x55b019fe28d0 msgr2=0x55b019fee430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:14.327 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.326+0000 7fa07a000640 0 client.16641 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:54:14.327 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.326+0000 7fa09e1af740 1 -- 172.21.15.83:0/2912367772 shutdown_connections 2024-04-24T08:54:14.327 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.326+0000 7fa09e1af740 1 -- 172.21.15.83:0/2912367772 wait complete. 2024-04-24T08:54:14.327 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.327+0000 7fa09e1af740 20 client.16641 trim_cache size 0 max 16384 2024-04-24T08:54:14.328 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T08:54:14.328 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T08:54:14.328 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T08:54:14.328 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T08:54:14.329 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.327+0000 7fa099e00640 1 -- 172.21.15.83:0/324033804 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa07c0a3d90 msgr2=0x7fa07c0a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:14.329 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.327+0000 7fa099e00640 1 --2- 172.21.15.83:0/324033804 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa07c0a3d90 0x7fa07c0a6250 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa088005fd0 tx=0x7fa08800b5d0 comp rx=0 tx=0).stop 2024-04-24T08:54:14.329 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.328+0000 7fa099e00640 1 -- 172.21.15.83:0/324033804 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa09410e4d0 msgr2=0x7fa09419ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:14.329 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.328+0000 7fa099e00640 1 --2- 172.21.15.83:0/324033804 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa09410e4d0 0x7fa09419ea00 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fa08c005e30 tx=0x7fa08c003ba0 comp rx=0 tx=0).stop 2024-04-24T08:54:14.329 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.328+0000 7fa099e00640 1 -- 172.21.15.83:0/324033804 shutdown_connections 2024-04-24T08:54:14.329 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.328+0000 7fa099e00640 1 --2- 172.21.15.83:0/324033804 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa07c0a3d90 0x7fa07c0a6250 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:14.329 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.328+0000 7fa099e00640 1 --2- 172.21.15.83:0/324033804 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa09410e4d0 0x7fa09419ea00 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:14.329 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.328+0000 7fa099e00640 1 --2- 172.21.15.83:0/324033804 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa094108ec0 0x7fa0941a4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:14.329 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.328+0000 7fa099e00640 1 --2- 172.21.15.83:0/324033804 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa094108540 0x7fa0941a4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:14.330 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.328+0000 7fa099e00640 1 -- 172.21.15.83:0/324033804 >> 172.21.15.83:0/324033804 conn(0x7fa094077de0 msgr2=0x7fa094078400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:14.330 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.328+0000 7fa099e00640 1 -- 172.21.15.83:0/324033804 shutdown_connections 2024-04-24T08:54:14.330 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:14.328+0000 7fa099e00640 1 -- 172.21.15.83:0/324033804 wait complete. 2024-04-24T08:54:14.559 INFO:tasks.workunit.client.0.smithi083.stdout: 1 6544 1.10 MB/sec warmup 48 sec latency 54.140 ms 2024-04-24T08:54:15.559 INFO:tasks.workunit.client.0.smithi083.stdout: 1 6767 1.09 MB/sec warmup 49 sec latency 58.200 ms 2024-04-24T08:54:15.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:54:15 smithi138 ceph-mon[31441]: pgmap v82: 129 pgs: 129 active+clean; 27 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 231 KiB/s wr, 39 op/s 2024-04-24T08:54:15.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:54:15 smithi189 ceph-mon[31791]: pgmap v82: 129 pgs: 129 active+clean; 27 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 231 KiB/s wr, 39 op/s 2024-04-24T08:54:16.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:54:15 smithi083 ceph-mon[27907]: pgmap v82: 129 pgs: 129 active+clean; 27 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 231 KiB/s wr, 39 op/s 2024-04-24T08:54:16.559 INFO:tasks.workunit.client.0.smithi083.stdout: 1 6858 1.09 MB/sec warmup 50 sec latency 54.828 ms 2024-04-24T08:54:17.559 INFO:tasks.workunit.client.0.smithi083.stdout: 1 6882 1.07 MB/sec warmup 51 sec latency 56.607 ms 2024-04-24T08:54:17.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:54:17 smithi138 ceph-mon[31441]: pgmap v83: 129 pgs: 129 active+clean; 27 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 231 KiB/s wr, 39 op/s 2024-04-24T08:54:17.956 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:54:17 smithi189 ceph-mon[31791]: pgmap v83: 129 pgs: 129 active+clean; 27 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 231 KiB/s wr, 39 op/s 2024-04-24T08:54:18.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:54:17 smithi083 ceph-mon[27907]: pgmap v83: 129 pgs: 129 active+clean; 27 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 231 KiB/s wr, 39 op/s 2024-04-24T08:54:18.559 INFO:tasks.workunit.client.0.smithi083.stdout: 1 6946 1.05 MB/sec warmup 52 sec latency 49.398 ms 2024-04-24T08:54:19.560 INFO:tasks.workunit.client.0.smithi083.stdout: 1 7130 1.09 MB/sec warmup 53 sec latency 51.317 ms 2024-04-24T08:54:19.954 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:54:19 smithi138 ceph-mon[31441]: pgmap v84: 129 pgs: 129 active+clean; 29 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 406 KiB/s wr, 61 op/s 2024-04-24T08:54:19.956 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:54:19 smithi189 ceph-mon[31791]: pgmap v84: 129 pgs: 129 active+clean; 29 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 406 KiB/s wr, 61 op/s 2024-04-24T08:54:20.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:54:19 smithi083 ceph-mon[27907]: pgmap v84: 129 pgs: 129 active+clean; 29 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 406 KiB/s wr, 61 op/s 2024-04-24T08:54:20.560 INFO:tasks.workunit.client.0.smithi083.stdout: 1 7247 1.07 MB/sec warmup 54 sec latency 51.410 ms 2024-04-24T08:54:21.560 INFO:tasks.workunit.client.0.smithi083.stdout: 1 7344 1.07 MB/sec warmup 55 sec latency 46.619 ms 2024-04-24T08:54:21.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:54:21 smithi189 ceph-mon[31791]: pgmap v85: 129 pgs: 129 active+clean; 29 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 361 KiB/s wr, 41 op/s 2024-04-24T08:54:22.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:54:21 smithi083 ceph-mon[27907]: pgmap v85: 129 pgs: 129 active+clean; 29 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 361 KiB/s wr, 41 op/s 2024-04-24T08:54:22.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:54:21 smithi138 ceph-mon[31441]: pgmap v85: 129 pgs: 129 active+clean; 29 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 361 KiB/s wr, 41 op/s 2024-04-24T08:54:22.561 INFO:tasks.workunit.client.0.smithi083.stdout: 1 7402 1.05 MB/sec warmup 56 sec latency 50.895 ms 2024-04-24T08:54:23.560 INFO:tasks.workunit.client.0.smithi083.stdout: 1 7471 1.04 MB/sec warmup 57 sec latency 49.001 ms 2024-04-24T08:54:24.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:54:23 smithi083 ceph-mon[27907]: pgmap v86: 129 pgs: 129 active+clean; 29 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 578 KiB/s wr, 63 op/s 2024-04-24T08:54:24.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:54:23 smithi138 ceph-mon[31441]: pgmap v86: 129 pgs: 129 active+clean; 29 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 578 KiB/s wr, 63 op/s 2024-04-24T08:54:24.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:54:23 smithi189 ceph-mon[31791]: pgmap v86: 129 pgs: 129 active+clean; 29 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 578 KiB/s wr, 63 op/s 2024-04-24T08:54:24.560 INFO:tasks.workunit.client.0.smithi083.stdout: 1 7527 1.02 MB/sec warmup 58 sec latency 45.323 ms 2024-04-24T08:54:25.560 INFO:tasks.workunit.client.0.smithi083.stdout: 1 7581 1.00 MB/sec warmup 59 sec latency 53.414 ms 2024-04-24T08:54:26.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:54:25 smithi083 ceph-mon[27907]: pgmap v87: 129 pgs: 129 active+clean; 29 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 392 KiB/s wr, 43 op/s 2024-04-24T08:54:26.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:54:25 smithi138 ceph-mon[31441]: pgmap v87: 129 pgs: 129 active+clean; 29 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 392 KiB/s wr, 43 op/s 2024-04-24T08:54:26.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:54:25 smithi189 ceph-mon[31791]: pgmap v87: 129 pgs: 129 active+clean; 29 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 392 KiB/s wr, 43 op/s 2024-04-24T08:54:26.560 INFO:tasks.workunit.client.0.smithi083.stdout: 1 7663 0.99 MB/sec warmup 60 sec latency 56.926 ms 2024-04-24T08:54:27.482 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.482+0000 7f85b2200640 1 -- 172.21.15.83:0/2632221182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85ac0743d0 msgr2=0x7f85ac0747b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:27.482 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.482+0000 7f85b2200640 1 --2- 172.21.15.83:0/2632221182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85ac0743d0 0x7f85ac0747b0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f85a0009900 tx=0x7f85a002a480 comp rx=0 tx=0).stop 2024-04-24T08:54:27.482 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.482+0000 7f85b2200640 1 -- 172.21.15.83:0/2632221182 shutdown_connections 2024-04-24T08:54:27.482 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.482+0000 7f85b2200640 1 --2- 172.21.15.83:0/2632221182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f85ac10e6b0 0x7f85ac110ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:27.482 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.482+0000 7f85b2200640 1 --2- 172.21.15.83:0/2632221182 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f85ac074cf0 0x7f85ac10e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:27.482 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.482+0000 7f85b2200640 1 --2- 172.21.15.83:0/2632221182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85ac0743d0 0x7f85ac0747b0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:27.482 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.482+0000 7f85b2200640 1 -- 172.21.15.83:0/2632221182 >> 172.21.15.83:0/2632221182 conn(0x7f85ac0fd940 msgr2=0x7f85ac0ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:27.482 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.482+0000 7f85b2200640 1 -- 172.21.15.83:0/2632221182 shutdown_connections 2024-04-24T08:54:27.482 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.482+0000 7f85b2200640 1 -- 172.21.15.83:0/2632221182 wait complete. 2024-04-24T08:54:27.483 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.483+0000 7f85b2200640 1 Processor -- start 2024-04-24T08:54:27.485 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.483+0000 7f85b2200640 1 -- start start 2024-04-24T08:54:27.485 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.483+0000 7f85b2200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85ac0743d0 0x7f85ac1a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:27.485 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.483+0000 7f85b2200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f85ac074cf0 0x7f85ac1a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:27.485 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.483+0000 7f85b2200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f85ac10e6b0 0x7f85ac19a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:27.485 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.483+0000 7f85b2200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f85ac114390 con 0x7f85ac0743d0 2024-04-24T08:54:27.485 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.483+0000 7f85b2200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f85ac114210 con 0x7f85ac10e6b0 2024-04-24T08:54:27.485 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.483+0000 7f85b2200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f85ac114510 con 0x7f85ac074cf0 2024-04-24T08:54:27.485 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.484+0000 7f85b0e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85ac0743d0 0x7f85ac1a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:27.485 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.484+0000 7f85b0e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85ac0743d0 0x7f85ac1a0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42062/0 (socket says 172.21.15.83:42062) 2024-04-24T08:54:27.486 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.484+0000 7f85b0e00640 1 -- 172.21.15.83:0/2103720867 learned_addr learned my addr 172.21.15.83:0/2103720867 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:54:27.486 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.484+0000 7f85b1800640 1 --2- 172.21.15.83:0/2103720867 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f85ac10e6b0 0x7f85ac19a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:27.486 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.484+0000 7f85abe00640 1 --2- 172.21.15.83:0/2103720867 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f85ac074cf0 0x7f85ac1a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:27.486 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.484+0000 7f85b0e00640 1 -- 172.21.15.83:0/2103720867 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f85ac074cf0 msgr2=0x7f85ac1a0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:27.486 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.484+0000 7f85b0e00640 1 --2- 172.21.15.83:0/2103720867 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f85ac074cf0 0x7f85ac1a0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:27.486 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.484+0000 7f85b0e00640 1 -- 172.21.15.83:0/2103720867 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f85ac10e6b0 msgr2=0x7f85ac19a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:27.486 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.484+0000 7f85b0e00640 1 --2- 172.21.15.83:0/2103720867 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f85ac10e6b0 0x7f85ac19a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:27.486 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.484+0000 7f85b0e00640 1 -- 172.21.15.83:0/2103720867 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85ac19aff0 con 0x7f85ac0743d0 2024-04-24T08:54:27.486 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.484+0000 7f85b1800640 1 --2- 172.21.15.83:0/2103720867 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f85ac10e6b0 0x7f85ac19a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:54:27.486 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.484+0000 7f85abe00640 1 --2- 172.21.15.83:0/2103720867 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f85ac074cf0 0x7f85ac1a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:54:27.486 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.484+0000 7f85b0e00640 1 --2- 172.21.15.83:0/2103720867 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85ac0743d0 0x7f85ac1a0670 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f85a0005ab0 tx=0x7f85a0003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:27.487 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.485+0000 7f85a9600640 1 -- 172.21.15.83:0/2103720867 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a003f070 con 0x7f85ac0743d0 2024-04-24T08:54:27.487 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.485+0000 7f85a9600640 1 -- 172.21.15.83:0/2103720867 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f85a0031d60 con 0x7f85ac0743d0 2024-04-24T08:54:27.487 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.485+0000 7f85a9600640 1 -- 172.21.15.83:0/2103720867 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a0035030 con 0x7f85ac0743d0 2024-04-24T08:54:27.487 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.485+0000 7f85b2200640 1 -- 172.21.15.83:0/2103720867 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85ac19b280 con 0x7f85ac0743d0 2024-04-24T08:54:27.487 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.485+0000 7f85b2200640 1 -- 172.21.15.83:0/2103720867 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85ac1a75f0 con 0x7f85ac0743d0 2024-04-24T08:54:27.487 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.488+0000 7f85b65fc740 1 Processor -- start 2024-04-24T08:54:27.487 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.488+0000 7f85a9600640 1 -- 172.21.15.83:0/2103720867 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f85a0042070 con 0x7f85ac0743d0 2024-04-24T08:54:27.488 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.488+0000 7f85b65fc740 1 -- start start 2024-04-24T08:54:27.489 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.489+0000 7f85b65fc740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5608e43f43c0 0x5608e43f47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:27.489 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.489+0000 7f85b65fc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608e43f4ce0 0x5608e43ff060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:27.489 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.489+0000 7f85b65fc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5608e43ff5a0 0x5608e4401990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:27.489 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.489+0000 7f85b65fc740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5608e435a900 con 0x5608e43f4ce0 2024-04-24T08:54:27.490 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.489+0000 7f85b65fc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5608e4300f20 con 0x5608e43ff5a0 2024-04-24T08:54:27.490 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.489+0000 7f85b65fc740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5608e43568c0 con 0x5608e43f43c0 2024-04-24T08:54:27.490 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.489+0000 7f85b0e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608e43f4ce0 0x5608e43ff060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:27.490 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.489+0000 7f85b0e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608e43f4ce0 0x5608e43ff060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42068/0 (socket says 172.21.15.83:42068) 2024-04-24T08:54:27.490 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.489+0000 7f85b0e00640 1 -- 172.21.15.83:0/254324040 learned_addr learned my addr 172.21.15.83:0/254324040 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:54:27.490 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.489+0000 7f85abe00640 1 --2- 172.21.15.83:0/254324040 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5608e43ff5a0 0x5608e4401990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:27.490 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.490+0000 7f85b0e00640 1 -- 172.21.15.83:0/254324040 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5608e43f43c0 msgr2=0x5608e43f47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:27.490 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.490+0000 7f85b0e00640 1 --2- 172.21.15.83:0/254324040 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5608e43f43c0 0x5608e43f47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:27.490 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.490+0000 7f85b0e00640 1 -- 172.21.15.83:0/254324040 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5608e43ff5a0 msgr2=0x5608e4401990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:27.490 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.490+0000 7f85b0e00640 1 --2- 172.21.15.83:0/254324040 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5608e43ff5a0 0x5608e4401990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:27.490 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.490+0000 7f85b0e00640 1 -- 172.21.15.83:0/254324040 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608e4325e10 con 0x5608e43f4ce0 2024-04-24T08:54:27.491 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.490+0000 7f85a9600640 1 --2- 172.21.15.83:0/2103720867 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f85940a3d90 0x7f85940a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:27.491 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.490+0000 7f85b0e00640 1 --2- 172.21.15.83:0/254324040 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608e43f4ce0 0x5608e43ff060 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f85a00ee7c0 tx=0x7f85a00f7840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57563ac75f02745a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:27.491 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.490+0000 7f85a9600640 1 -- 172.21.15.83:0/2103720867 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f85a00f1560 con 0x7f85ac0743d0 2024-04-24T08:54:27.491 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.490+0000 7f85b1800640 1 --2- 172.21.15.83:0/2103720867 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f85940a3d90 0x7f85940a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:27.491 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.490+0000 7f859e000640 1 -- 172.21.15.83:0/254324040 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a00b9370 con 0x5608e43f4ce0 2024-04-24T08:54:27.491 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.490+0000 7f859e000640 1 -- 172.21.15.83:0/254324040 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f85a00bd7d0 con 0x5608e43f4ce0 2024-04-24T08:54:27.491 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.490+0000 7f859e000640 1 -- 172.21.15.83:0/254324040 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a00bca30 con 0x5608e43f4ce0 2024-04-24T08:54:27.493 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.491+0000 7f85b65fc740 1 -- 172.21.15.83:0/254324040 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608e43f4ce0 msgr2=0x5608e43ff060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:27.494 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.491+0000 7f85b1800640 1 --2- 172.21.15.83:0/2103720867 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f85940a3d90 0x7f85940a6250 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f85ac06b560 tx=0x7f85a4009440 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:27.494 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.491+0000 7f85b65fc740 1 --2- 172.21.15.83:0/254324040 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608e43f4ce0 0x5608e43ff060 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f85a00ee7c0 tx=0x7f85a00f7840 comp rx=0 tx=0).stop 2024-04-24T08:54:27.494 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.491+0000 7f85b65fc740 1 -- 172.21.15.83:0/254324040 shutdown_connections 2024-04-24T08:54:27.494 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.491+0000 7f85b65fc740 1 --2- 172.21.15.83:0/254324040 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5608e43ff5a0 0x5608e4401990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:27.494 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.491+0000 7f85b65fc740 1 --2- 172.21.15.83:0/254324040 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608e43f4ce0 0x5608e43ff060 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:27.494 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.491+0000 7f85b65fc740 1 --2- 172.21.15.83:0/254324040 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5608e43f43c0 0x5608e43f47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:27.494 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.491+0000 7f85b65fc740 1 -- 172.21.15.83:0/254324040 >> 172.21.15.83:0/254324040 conn(0x5608e43f08d0 msgr2=0x5608e43f0cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:27.494 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.491+0000 7f85b65fc740 1 -- 172.21.15.83:0/254324040 shutdown_connections 2024-04-24T08:54:27.494 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.492+0000 7f85b65fc740 1 -- 172.21.15.83:0/254324040 wait complete. 2024-04-24T08:54:27.494 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.492+0000 7f85b65fc740 1 -- start start 2024-04-24T08:54:27.494 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.492+0000 7f85b65fc740 1 Processor -- start 2024-04-24T08:54:27.494 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.493+0000 7f85b65fc740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5608e43f43c0 0x5608e43fc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:27.495 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.493+0000 7f85b65fc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5608e43f4ce0 0x5608e43fcaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:27.495 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.493+0000 7f85b65fc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608e43ff5a0 0x5608e43fd030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:27.495 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.493+0000 7f85b65fc740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5608e4401f90 con 0x5608e43ff5a0 2024-04-24T08:54:27.495 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.493+0000 7f85b65fc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5608e43142c0 con 0x5608e43f4ce0 2024-04-24T08:54:27.495 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.493+0000 7f85b65fc740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5608e4402110 con 0x5608e43f43c0 2024-04-24T08:54:27.495 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.493+0000 7f85b65fc740 10 client.0 _ll_register_callbacks cb 0x5608e406ca00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:54:27.495 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.493+0000 7f85b1800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608e43ff5a0 0x5608e43fd030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:27.495 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.493+0000 7f85b1800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608e43ff5a0 0x5608e43fd030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42072/0 (socket says 172.21.15.83:42072) 2024-04-24T08:54:27.495 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.493+0000 7f85b1800640 1 -- 172.21.15.83:0/1353106462 learned_addr learned my addr 172.21.15.83:0/1353106462 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:54:27.495 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.494+0000 7f85b1800640 1 -- 172.21.15.83:0/1353106462 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5608e43f43c0 msgr2=0x5608e43fc5b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-24T08:54:27.495 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.494+0000 7f85b1800640 1 --2- 172.21.15.83:0/1353106462 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5608e43f43c0 0x5608e43fc5b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:27.495 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.494+0000 7f85b1800640 1 -- 172.21.15.83:0/1353106462 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5608e43f4ce0 msgr2=0x5608e43fcaf0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-24T08:54:27.495 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.494+0000 7f85b1800640 1 --2- 172.21.15.83:0/1353106462 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5608e43f4ce0 0x5608e43fcaf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:27.495 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.494+0000 7f85b1800640 1 -- 172.21.15.83:0/1353106462 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608e4324fd0 con 0x5608e43ff5a0 2024-04-24T08:54:27.496 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.494+0000 7f85b1800640 1 --2- 172.21.15.83:0/1353106462 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608e43ff5a0 0x5608e43fd030 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f85a4004a80 tx=0x7f85a4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b438e8fdb7b0197a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:27.496 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.494+0000 7f8592a00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:54:27.496 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.494+0000 7f8592a00640 1 -- 172.21.15.83:0/1353106462 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a4022070 con 0x5608e43ff5a0 2024-04-24T08:54:27.496 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.494+0000 7f8592a00640 1 -- 172.21.15.83:0/1353106462 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f85a4010920 con 0x5608e43ff5a0 2024-04-24T08:54:27.496 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.494+0000 7f85b65fc740 1 -- 172.21.15.83:0/1353106462 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85a00bd7d0 con 0x5608e43ff5a0 2024-04-24T08:54:27.496 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.495+0000 7f8592a00640 1 -- 172.21.15.83:0/1353106462 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f85a402e050 con 0x5608e43ff5a0 2024-04-24T08:54:27.496 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.495+0000 7f85b65fc740 10 client.16653 fetch_fsmap learned FSMap version 30 2024-04-24T08:54:27.496 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.495+0000 7f85b65fc740 1 -- 172.21.15.83:0/1353106462 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5608e406d210 con 0x5608e43ff5a0 2024-04-24T08:54:27.497 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.495+0000 7f8592a00640 1 -- 172.21.15.83:0/1353106462 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f85a00bd7d0 con 0x5608e43ff5a0 2024-04-24T08:54:27.497 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.495+0000 7f8592a00640 1 -- 172.21.15.83:0/1353106462 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7f85a40263b0 con 0x5608e43ff5a0 2024-04-24T08:54:27.497 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.495+0000 7f85b65fc740 10 client.16653 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T08:54:27.497 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.495+0000 7f85b65fc740 10 client.16653 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T08:54:27.497 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.495+0000 7f85b65fc740 20 client.16653 populate_metadata read hostname 'smithi083' 2024-04-24T08:54:27.497 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.495+0000 7f85b65fc740 1 --2- 172.21.15.83:0/1353106462 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5608e440f620 0x5608e4411ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:27.497 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.496+0000 7f85b65fc740 4 client.16653 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:54:27.498 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.496+0000 7f85b65fc740 1 -- 172.21.15.83:0/1353106462 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5608e4368320 con 0x5608e440f620 2024-04-24T08:54:27.498 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.498+0000 7f85b0e00640 1 --2- 172.21.15.83:0/1353106462 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5608e440f620 0x5608e4411ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:27.500 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.501+0000 7f85b0e00640 1 --2- 172.21.15.83:0/1353106462 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5608e440f620 0x5608e4411ac0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9fb61cbe5b8b6a server_cookie=28a69060fc080c09 in_seq=0 out_seq=0 2024-04-24T08:54:27.500 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.501+0000 7f8592a00640 10 client.16653 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:54:27.505 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.506+0000 7f8592a00640 1 -- 172.21.15.83:0/1353106462 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5608e4368320 con 0x5608e440f620 2024-04-24T08:54:27.506 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.506+0000 7f8592a00640 10 client.16653 handle_command_reply: tid=0 2024-04-24T08:54:27.506 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.506+0000 7f85b65fc740 1 client.16653 shutdown 2024-04-24T08:54:27.506 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.506+0000 7f85b65fc740 1 -- 172.21.15.83:0/1353106462 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608e43ff5a0 msgr2=0x5608e43fd030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:27.506 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.506+0000 7f85b65fc740 1 --2- 172.21.15.83:0/1353106462 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608e43ff5a0 0x5608e43fd030 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f85a4004a80 tx=0x7f85a4004ab0 comp rx=0 tx=0).stop 2024-04-24T08:54:27.506 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.506+0000 7f85b65fc740 1 -- 172.21.15.83:0/1353106462 shutdown_connections 2024-04-24T08:54:27.506 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.506+0000 7f85b65fc740 1 --2- 172.21.15.83:0/1353106462 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5608e440f620 0x5608e4411ac0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:27.506 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.506+0000 7f85b65fc740 1 --2- 172.21.15.83:0/1353106462 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608e43ff5a0 0x5608e43fd030 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:27.506 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.506+0000 7f85b65fc740 1 --2- 172.21.15.83:0/1353106462 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5608e43f4ce0 0x5608e43fcaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:27.507 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.506+0000 7f85b65fc740 1 --2- 172.21.15.83:0/1353106462 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5608e43f43c0 0x5608e43fc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:27.507 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.506+0000 7f8592a00640 0 client.16653 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:54:27.507 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.506+0000 7f85b65fc740 1 -- 172.21.15.83:0/1353106462 >> 172.21.15.83:0/1353106462 conn(0x5608e43f08d0 msgr2=0x5608e43efa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:27.507 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.506+0000 7f85b65fc740 1 -- 172.21.15.83:0/1353106462 shutdown_connections 2024-04-24T08:54:27.507 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.506+0000 7f85b65fc740 1 -- 172.21.15.83:0/1353106462 wait complete. 2024-04-24T08:54:27.507 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.506+0000 7f85b65fc740 20 client.16653 trim_cache size 0 max 16384 2024-04-24T08:54:27.529 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.529+0000 7f85b65fc740 1 Processor -- start 2024-04-24T08:54:27.530 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.529+0000 7f85b65fc740 1 -- start start 2024-04-24T08:54:27.530 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.530+0000 7f85b65fc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5608e43f43c0 0x5608e440ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:27.530 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.530+0000 7f85b65fc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608e43f4ce0 0x5608e43fce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:27.530 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.530+0000 7f85b65fc740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5608e43ff5a0 0x5608e43f6480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:27.530 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.530+0000 7f85b65fc740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f85ac1b1640 con 0x5608e43f4ce0 2024-04-24T08:54:27.530 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.530+0000 7f85b65fc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f85ac1b3210 con 0x5608e43f43c0 2024-04-24T08:54:27.530 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.530+0000 7f85b65fc740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f85ac1b1d10 con 0x5608e43ff5a0 2024-04-24T08:54:27.530 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.530+0000 7f85abe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608e43f4ce0 0x5608e43fce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:27.530 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.530+0000 7f85abe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608e43f4ce0 0x5608e43fce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42076/0 (socket says 172.21.15.83:42076) 2024-04-24T08:54:27.530 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.530+0000 7f85abe00640 1 -- 172.21.15.83:0/3591625267 learned_addr learned my addr 172.21.15.83:0/3591625267 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:54:27.531 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.530+0000 7f85b1800640 1 --2- 172.21.15.83:0/3591625267 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5608e43ff5a0 0x5608e43f6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:27.531 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.531+0000 7f85b0e00640 1 --2- 172.21.15.83:0/3591625267 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5608e43f43c0 0x5608e440ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:27.532 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.531+0000 7f85abe00640 1 -- 172.21.15.83:0/3591625267 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5608e43ff5a0 msgr2=0x5608e43f6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:27.532 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.531+0000 7f85abe00640 1 --2- 172.21.15.83:0/3591625267 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5608e43ff5a0 0x5608e43f6480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:27.532 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.531+0000 7f85abe00640 1 -- 172.21.15.83:0/3591625267 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5608e43f43c0 msgr2=0x5608e440ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:27.532 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.531+0000 7f85abe00640 1 --2- 172.21.15.83:0/3591625267 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5608e43f43c0 0x5608e440ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:27.532 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.531+0000 7f85abe00640 1 -- 172.21.15.83:0/3591625267 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608e4401d00 con 0x5608e43f4ce0 2024-04-24T08:54:27.533 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.531+0000 7f85abe00640 1 --2- 172.21.15.83:0/3591625267 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608e43f4ce0 0x5608e43fce00 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f85ac19bf80 tx=0x7f859800ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acbceb281af42ff0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:27.533 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.531+0000 7f859d600640 1 -- 172.21.15.83:0/3591625267 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8598018030 con 0x5608e43f4ce0 2024-04-24T08:54:27.533 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.531+0000 7f859d600640 1 -- 172.21.15.83:0/3591625267 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f859800ee70 con 0x5608e43f4ce0 2024-04-24T08:54:27.533 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.532+0000 7f859d600640 1 -- 172.21.15.83:0/3591625267 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8598013d30 con 0x5608e43f4ce0 2024-04-24T08:54:27.533 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.532+0000 7f85b65fc740 1 -- 172.21.15.83:0/3591625267 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608e43f4ce0 msgr2=0x5608e43fce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:27.533 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.532+0000 7f85b65fc740 1 --2- 172.21.15.83:0/3591625267 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608e43f4ce0 0x5608e43fce00 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f85ac19bf80 tx=0x7f859800ccd0 comp rx=0 tx=0).stop 2024-04-24T08:54:27.533 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.533+0000 7f85b65fc740 1 -- 172.21.15.83:0/3591625267 shutdown_connections 2024-04-24T08:54:27.533 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.533+0000 7f85b65fc740 1 --2- 172.21.15.83:0/3591625267 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5608e43ff5a0 0x5608e43f6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:27.533 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.533+0000 7f85b65fc740 1 --2- 172.21.15.83:0/3591625267 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608e43f4ce0 0x5608e43fce00 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:27.533 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.533+0000 7f85b65fc740 1 --2- 172.21.15.83:0/3591625267 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5608e43f43c0 0x5608e440ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:27.533 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.533+0000 7f85b65fc740 1 -- 172.21.15.83:0/3591625267 >> 172.21.15.83:0/3591625267 conn(0x5608e43f08d0 msgr2=0x5608e43f5d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:27.533 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.533+0000 7f85b65fc740 1 -- 172.21.15.83:0/3591625267 shutdown_connections 2024-04-24T08:54:27.533 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.533+0000 7f85b65fc740 1 -- 172.21.15.83:0/3591625267 wait complete. 2024-04-24T08:54:27.534 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.533+0000 7f85b65fc740 1 -- start start 2024-04-24T08:54:27.534 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.533+0000 7f85b65fc740 1 Processor -- start 2024-04-24T08:54:27.537 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.534+0000 7f85b65fc740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5608e43f43c0 0x5608e4407260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:27.538 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.534+0000 7f85b65fc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5608e43f4ce0 0x5608e44077a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:27.538 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.534+0000 7f85b65fc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608e43ff5a0 0x5608e4404040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:27.538 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.534+0000 7f85b65fc740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f85ac1b2ab0 con 0x5608e43ff5a0 2024-04-24T08:54:27.538 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.534+0000 7f85b65fc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f85ac1b23e0 con 0x5608e43f4ce0 2024-04-24T08:54:27.538 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.534+0000 7f85b65fc740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f85ac1af620 con 0x5608e43f43c0 2024-04-24T08:54:27.538 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.534+0000 7f85b65fc740 10 client.0 _ll_register_callbacks cb 0x7f85a4015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:54:27.538 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.534+0000 7f85b1800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608e43ff5a0 0x5608e4404040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:27.538 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.534+0000 7f85b1800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608e43ff5a0 0x5608e4404040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42080/0 (socket says 172.21.15.83:42080) 2024-04-24T08:54:27.538 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.534+0000 7f85b1800640 1 -- 172.21.15.83:0/3214531191 learned_addr learned my addr 172.21.15.83:0/3214531191 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:54:27.538 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.534+0000 7f85b0e00640 1 --2- 172.21.15.83:0/3214531191 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5608e43f43c0 0x5608e4407260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:27.538 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.535+0000 7f85abe00640 1 --2- 172.21.15.83:0/3214531191 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5608e43f4ce0 0x5608e44077a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:27.538 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.535+0000 7f85b1800640 1 -- 172.21.15.83:0/3214531191 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5608e43f43c0 msgr2=0x5608e4407260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:27.539 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.535+0000 7f85b1800640 1 --2- 172.21.15.83:0/3214531191 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5608e43f43c0 0x5608e4407260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:27.539 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.535+0000 7f85b1800640 1 -- 172.21.15.83:0/3214531191 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5608e43f4ce0 msgr2=0x5608e44077a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:27.539 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.535+0000 7f85b1800640 1 --2- 172.21.15.83:0/3214531191 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5608e43f4ce0 0x5608e44077a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:27.539 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.535+0000 7f85b1800640 1 -- 172.21.15.83:0/3214531191 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608e4139620 con 0x5608e43ff5a0 2024-04-24T08:54:27.539 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.535+0000 7f85b1800640 1 --2- 172.21.15.83:0/3214531191 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608e43ff5a0 0x5608e4404040 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f85a4026750 tx=0x7f85a40265e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=defdbd7e4d53f74a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:27.539 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.535+0000 7f8592a00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:54:27.539 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.535+0000 7f8592a00640 1 -- 172.21.15.83:0/3214531191 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a4022050 con 0x5608e43ff5a0 2024-04-24T08:54:27.539 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.535+0000 7f8592a00640 1 -- 172.21.15.83:0/3214531191 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f85a4010720 con 0x5608e43ff5a0 2024-04-24T08:54:27.539 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.535+0000 7f85b65fc740 1 -- 172.21.15.83:0/3214531191 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f859800ee70 con 0x5608e43ff5a0 2024-04-24T08:54:27.539 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.536+0000 7f8592a00640 1 -- 172.21.15.83:0/3214531191 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f85a4010920 con 0x5608e43ff5a0 2024-04-24T08:54:27.539 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.536+0000 7f85b65fc740 10 client.16659 fetch_fsmap learned FSMap version 30 2024-04-24T08:54:27.539 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.536+0000 7f85b65fc740 1 -- 172.21.15.83:0/3214531191 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5608e447ed10 con 0x5608e43ff5a0 2024-04-24T08:54:27.539 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.536+0000 7f8592a00640 1 -- 172.21.15.83:0/3214531191 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f859800ee70 con 0x5608e43ff5a0 2024-04-24T08:54:27.539 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.536+0000 7f8592a00640 1 -- 172.21.15.83:0/3214531191 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7f85a401ebc0 con 0x5608e43ff5a0 2024-04-24T08:54:27.540 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.536+0000 7f85b65fc740 10 client.16659 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T08:54:27.540 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.536+0000 7f85b65fc740 10 client.16659 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T08:54:27.540 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.536+0000 7f85b65fc740 20 client.16659 populate_metadata read hostname 'smithi083' 2024-04-24T08:54:27.540 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.536+0000 7f85b65fc740 1 --2- 172.21.15.83:0/3214531191 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5608e447f570 0x5608e4481990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:27.540 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.536+0000 7f85b65fc740 4 client.16659 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T08:54:27.540 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.536+0000 7f85b65fc740 1 -- 172.21.15.83:0/3214531191 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5608e4362190 con 0x5608e447f570 2024-04-24T08:54:27.540 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.537+0000 7f85b0e00640 1 --2- 172.21.15.83:0/3214531191 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5608e447f570 0x5608e4481990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:27.540 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.538+0000 7f85b0e00640 1 --2- 172.21.15.83:0/3214531191 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5608e447f570 0x5608e4481990 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7460e9333b92edb server_cookie=72f77c5f60309047 in_seq=0 out_seq=0 2024-04-24T08:54:27.540 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.538+0000 7f8592a00640 10 client.16659 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:54:27.560 INFO:tasks.workunit.client.0.smithi083.stdout: 1 7758 0.97 MB/sec warmup 61 sec latency 53.497 ms 2024-04-24T08:54:27.593 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.593+0000 7f8592a00640 1 -- 172.21.15.83:0/3214531191 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5608e4362190 con 0x5608e447f570 2024-04-24T08:54:27.593 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.593+0000 7f8592a00640 10 client.16659 handle_command_reply: tid=0 2024-04-24T08:54:27.593 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.594+0000 7f85b65fc740 1 client.16659 shutdown 2024-04-24T08:54:27.593 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.594+0000 7f85b65fc740 1 -- 172.21.15.83:0/3214531191 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608e43ff5a0 msgr2=0x5608e4404040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:27.593 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.594+0000 7f85b65fc740 1 --2- 172.21.15.83:0/3214531191 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608e43ff5a0 0x5608e4404040 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f85a4026750 tx=0x7f85a40265e0 comp rx=0 tx=0).stop 2024-04-24T08:54:27.594 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.594+0000 7f85b65fc740 1 -- 172.21.15.83:0/3214531191 shutdown_connections 2024-04-24T08:54:27.594 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.594+0000 7f85b65fc740 1 --2- 172.21.15.83:0/3214531191 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5608e447f570 0x5608e4481990 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:27.594 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.594+0000 7f85b65fc740 1 --2- 172.21.15.83:0/3214531191 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608e43ff5a0 0x5608e4404040 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:27.594 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.594+0000 7f85b65fc740 1 --2- 172.21.15.83:0/3214531191 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5608e43f4ce0 0x5608e44077a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:27.594 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.594+0000 7f85b65fc740 1 --2- 172.21.15.83:0/3214531191 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5608e43f43c0 0x5608e4407260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:27.594 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.594+0000 7f8592a00640 0 client.16659 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:54:27.594 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.594+0000 7f85b65fc740 1 -- 172.21.15.83:0/3214531191 >> 172.21.15.83:0/3214531191 conn(0x5608e43f08d0 msgr2=0x5608e43f5d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:27.595 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.594+0000 7f85b65fc740 1 -- 172.21.15.83:0/3214531191 shutdown_connections 2024-04-24T08:54:27.595 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.595+0000 7f85b65fc740 1 -- 172.21.15.83:0/3214531191 wait complete. 2024-04-24T08:54:27.595 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.595+0000 7f85b65fc740 20 client.16659 trim_cache size 0 max 16384 2024-04-24T08:54:27.595 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T08:54:27.595 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T08:54:27.595 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T08:54:27.595 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T08:54:27.596 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.596+0000 7f85b2200640 1 -- 172.21.15.83:0/2103720867 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f85940a3d90 msgr2=0x7f85940a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:27.596 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.596+0000 7f85b2200640 1 --2- 172.21.15.83:0/2103720867 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f85940a3d90 0x7f85940a6250 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f85ac06b560 tx=0x7f85a4009440 comp rx=0 tx=0).stop 2024-04-24T08:54:27.596 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.596+0000 7f85b2200640 1 -- 172.21.15.83:0/2103720867 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85ac0743d0 msgr2=0x7f85ac1a0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:27.596 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.596+0000 7f85b2200640 1 --2- 172.21.15.83:0/2103720867 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85ac0743d0 0x7f85ac1a0670 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f85a0005ab0 tx=0x7f85a0003cd0 comp rx=0 tx=0).stop 2024-04-24T08:54:27.596 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.596+0000 7f85b2200640 1 -- 172.21.15.83:0/2103720867 shutdown_connections 2024-04-24T08:54:27.596 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.596+0000 7f85b2200640 1 --2- 172.21.15.83:0/2103720867 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f85940a3d90 0x7f85940a6250 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:27.596 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.596+0000 7f85b2200640 1 --2- 172.21.15.83:0/2103720867 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f85ac10e6b0 0x7f85ac19a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:27.596 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.596+0000 7f85b2200640 1 --2- 172.21.15.83:0/2103720867 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f85ac074cf0 0x7f85ac1a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:27.596 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.596+0000 7f85b2200640 1 --2- 172.21.15.83:0/2103720867 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85ac0743d0 0x7f85ac1a0670 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:27.597 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.596+0000 7f85b2200640 1 -- 172.21.15.83:0/2103720867 >> 172.21.15.83:0/2103720867 conn(0x7f85ac0fd940 msgr2=0x7f85ac0ff200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:27.597 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.596+0000 7f85b2200640 1 -- 172.21.15.83:0/2103720867 shutdown_connections 2024-04-24T08:54:27.597 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:27.596+0000 7f85b2200640 1 -- 172.21.15.83:0/2103720867 wait complete. 2024-04-24T08:54:28.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:54:27 smithi083 ceph-mon[27907]: pgmap v88: 129 pgs: 129 active+clean; 29 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 392 KiB/s wr, 43 op/s 2024-04-24T08:54:28.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:54:27 smithi138 ceph-mon[31441]: pgmap v88: 129 pgs: 129 active+clean; 29 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 392 KiB/s wr, 43 op/s 2024-04-24T08:54:28.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:54:27 smithi189 ceph-mon[31791]: pgmap v88: 129 pgs: 129 active+clean; 29 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 392 KiB/s wr, 43 op/s 2024-04-24T08:54:28.560 INFO:tasks.workunit.client.0.smithi083.stdout: 1 7884 0.96 MB/sec warmup 62 sec latency 58.831 ms 2024-04-24T08:54:29.561 INFO:tasks.workunit.client.0.smithi083.stdout: 1 8000 0.95 MB/sec warmup 63 sec latency 57.105 ms 2024-04-24T08:54:30.010 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:54:29 smithi189 ceph-mon[31791]: pgmap v89: 129 pgs: 129 active+clean; 30 MiB data, 494 MiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 469 KiB/s wr, 66 op/s 2024-04-24T08:54:30.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:54:29 smithi083 ceph-mon[27907]: pgmap v89: 129 pgs: 129 active+clean; 30 MiB data, 494 MiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 469 KiB/s wr, 66 op/s 2024-04-24T08:54:30.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:54:29 smithi138 ceph-mon[31441]: pgmap v89: 129 pgs: 129 active+clean; 30 MiB data, 494 MiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 469 KiB/s wr, 66 op/s 2024-04-24T08:54:30.561 INFO:tasks.workunit.client.0.smithi083.stdout: 1 8104 0.94 MB/sec warmup 64 sec latency 53.811 ms 2024-04-24T08:54:31.555 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T08:54:31.555 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T08:54:31.564 INFO:tasks.workunit.client.0.smithi083.stdout: 1 8223 0.94 MB/sec warmup 65 sec latency 52.227 ms 2024-04-24T08:54:31.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.778+0000 7f4a95e00640 1 -- 172.21.15.83:0/183542970 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a901071e0 msgr2=0x7f4a9010edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:31.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.778+0000 7f4a95e00640 1 --2- 172.21.15.83:0/183542970 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a901071e0 0x7f4a9010edb0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f4a88009ca0 tx=0x7f4a8802a8c0 comp rx=0 tx=0).stop 2024-04-24T08:54:31.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.778+0000 7f4a95e00640 1 -- 172.21.15.83:0/183542970 shutdown_connections 2024-04-24T08:54:31.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.778+0000 7f4a95e00640 1 --2- 172.21.15.83:0/183542970 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a901071e0 0x7f4a9010edb0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:31.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.778+0000 7f4a95e00640 1 --2- 172.21.15.83:0/183542970 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4a90102ac0 0x7f4a90106b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:31.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.778+0000 7f4a95e00640 1 --2- 172.21.15.83:0/183542970 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a90102110 0x7f4a901024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:31.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.778+0000 7f4a95e00640 1 -- 172.21.15.83:0/183542970 >> 172.21.15.83:0/183542970 conn(0x7f4a900fd9d0 msgr2=0x7f4a900ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:31.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.778+0000 7f4a95e00640 1 -- 172.21.15.83:0/183542970 shutdown_connections 2024-04-24T08:54:31.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.778+0000 7f4a95e00640 1 -- 172.21.15.83:0/183542970 wait complete. 2024-04-24T08:54:31.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.779+0000 7f4a95e00640 1 Processor -- start 2024-04-24T08:54:31.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.779+0000 7f4a95e00640 1 -- start start 2024-04-24T08:54:31.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.780+0000 7f4a95e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a90102110 0x7f4a901a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:31.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.780+0000 7f4a95e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4a901071e0 0x7f4a901a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:31.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.780+0000 7f4a95e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a9019a6f0 0x7f4a9019ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:31.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.780+0000 7f4a95e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4a90112150 con 0x7f4a9019a6f0 2024-04-24T08:54:31.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.780+0000 7f4a95e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4a90111fd0 con 0x7f4a90102110 2024-04-24T08:54:31.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.780+0000 7f4a95e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4a901122d0 con 0x7f4a901071e0 2024-04-24T08:54:31.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.780+0000 7f4a95400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a9019a6f0 0x7f4a9019ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:31.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.780+0000 7f4a95400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a9019a6f0 0x7f4a9019ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41488/0 (socket says 172.21.15.83:41488) 2024-04-24T08:54:31.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.780+0000 7f4a95400640 1 -- 172.21.15.83:0/1394583378 learned_addr learned my addr 172.21.15.83:0/1394583378 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:54:31.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.780+0000 7f4a94a00640 1 --2- 172.21.15.83:0/1394583378 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a90102110 0x7f4a901a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:31.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.780+0000 7f4a8fe00640 1 --2- 172.21.15.83:0/1394583378 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4a901071e0 0x7f4a901a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:31.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.780+0000 7f4a95400640 1 -- 172.21.15.83:0/1394583378 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4a901071e0 msgr2=0x7f4a901a0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:31.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.781+0000 7f4a95400640 1 --2- 172.21.15.83:0/1394583378 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4a901071e0 0x7f4a901a0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:31.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.781+0000 7f4a95400640 1 -- 172.21.15.83:0/1394583378 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a90102110 msgr2=0x7f4a901a0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:31.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.781+0000 7f4a95400640 1 --2- 172.21.15.83:0/1394583378 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a90102110 0x7f4a901a0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:31.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.781+0000 7f4a95400640 1 -- 172.21.15.83:0/1394583378 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a9019b2f0 con 0x7f4a9019a6f0 2024-04-24T08:54:31.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.781+0000 7f4a94a00640 1 --2- 172.21.15.83:0/1394583378 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a90102110 0x7f4a901a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:54:31.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.781+0000 7f4a95400640 1 --2- 172.21.15.83:0/1394583378 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a9019a6f0 0x7f4a9019ab70 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f4a88009dc0 tx=0x7f4a88032ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:31.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.781+0000 7f4a8d600640 1 -- 172.21.15.83:0/1394583378 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a88003cf0 con 0x7f4a9019a6f0 2024-04-24T08:54:31.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.781+0000 7f4a8d600640 1 -- 172.21.15.83:0/1394583378 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4a880044f0 con 0x7f4a9019a6f0 2024-04-24T08:54:31.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.781+0000 7f4a95e00640 1 -- 172.21.15.83:0/1394583378 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a9019b580 con 0x7f4a9019a6f0 2024-04-24T08:54:31.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.781+0000 7f4a8d600640 1 -- 172.21.15.83:0/1394583378 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a88030bf0 con 0x7f4a9019a6f0 2024-04-24T08:54:31.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.781+0000 7f4a95e00640 1 -- 172.21.15.83:0/1394583378 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a901a7720 con 0x7f4a9019a6f0 2024-04-24T08:54:31.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.785+0000 7f4a8d600640 1 -- 172.21.15.83:0/1394583378 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4a88004040 con 0x7f4a9019a6f0 2024-04-24T08:54:31.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.785+0000 7f4a9a1d9740 1 Processor -- start 2024-04-24T08:54:31.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.785+0000 7f4a9a1d9740 1 -- start start 2024-04-24T08:54:31.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.785+0000 7f4a9a1d9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d75f71da30 0x55d75f71de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:31.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.785+0000 7f4a9a1d9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d75f71e350 0x55d75f7284e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:31.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.785+0000 7f4a9a1d9740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d75f728a20 0x55d75f72ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:31.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.785+0000 7f4a9a1d9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d75f6831f0 con 0x55d75f71da30 2024-04-24T08:54:31.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.785+0000 7f4a9a1d9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d75f629350 con 0x55d75f71e350 2024-04-24T08:54:31.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.785+0000 7f4a9a1d9740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d75f67f1b0 con 0x55d75f728a20 2024-04-24T08:54:31.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.785+0000 7f4a8fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d75f71da30 0x55d75f71de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:31.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.785+0000 7f4a8fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d75f71da30 0x55d75f71de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41496/0 (socket says 172.21.15.83:41496) 2024-04-24T08:54:31.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.785+0000 7f4a8fe00640 1 -- 172.21.15.83:0/2126488199 learned_addr learned my addr 172.21.15.83:0/2126488199 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:54:31.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.786+0000 7f4a8fe00640 1 -- 172.21.15.83:0/2126488199 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d75f728a20 msgr2=0x55d75f72ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:31.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.786+0000 7f4a8fe00640 1 --2- 172.21.15.83:0/2126488199 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d75f728a20 0x55d75f72ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:31.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.786+0000 7f4a8fe00640 1 -- 172.21.15.83:0/2126488199 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d75f71e350 msgr2=0x55d75f7284e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:31.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.786+0000 7f4a8fe00640 1 --2- 172.21.15.83:0/2126488199 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d75f71e350 0x55d75f7284e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:31.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.786+0000 7f4a8fe00640 1 -- 172.21.15.83:0/2126488199 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d75f67df50 con 0x55d75f71da30 2024-04-24T08:54:31.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.786+0000 7f4a8fe00640 1 --2- 172.21.15.83:0/2126488199 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d75f71da30 0x55d75f71de10 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c00bc70 tx=0x7f4a7c011750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2810adea1dfc41b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:31.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.786+0000 7f4a81c00640 1 -- 172.21.15.83:0/2126488199 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a7c01a400 con 0x55d75f71da30 2024-04-24T08:54:31.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.786+0000 7f4a81c00640 1 -- 172.21.15.83:0/2126488199 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4a7c005bf0 con 0x55d75f71da30 2024-04-24T08:54:31.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.787+0000 7f4a81c00640 1 -- 172.21.15.83:0/2126488199 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a7c0128b0 con 0x55d75f71da30 2024-04-24T08:54:31.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.787+0000 7f4a8d600640 1 --2- 172.21.15.83:0/1394583378 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f4a780a3d20 0x7f4a780a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:31.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.787+0000 7f4a8d600640 1 -- 172.21.15.83:0/1394583378 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f4a880f21d0 con 0x7f4a9019a6f0 2024-04-24T08:54:31.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.787+0000 7f4a94a00640 1 --2- 172.21.15.83:0/1394583378 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f4a780a3d20 0x7f4a780a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:31.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.787+0000 7f4a9a1d9740 1 -- 172.21.15.83:0/2126488199 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d75f71da30 msgr2=0x55d75f71de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:31.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.787+0000 7f4a9a1d9740 1 --2- 172.21.15.83:0/2126488199 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d75f71da30 0x55d75f71de10 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c00bc70 tx=0x7f4a7c011750 comp rx=0 tx=0).stop 2024-04-24T08:54:31.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.788+0000 7f4a9a1d9740 1 -- 172.21.15.83:0/2126488199 shutdown_connections 2024-04-24T08:54:31.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.788+0000 7f4a9a1d9740 1 --2- 172.21.15.83:0/2126488199 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d75f728a20 0x55d75f72ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:31.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.788+0000 7f4a9a1d9740 1 --2- 172.21.15.83:0/2126488199 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d75f71e350 0x55d75f7284e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:31.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.788+0000 7f4a9a1d9740 1 --2- 172.21.15.83:0/2126488199 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d75f71da30 0x55d75f71de10 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:31.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.788+0000 7f4a9a1d9740 1 -- 172.21.15.83:0/2126488199 >> 172.21.15.83:0/2126488199 conn(0x55d75f719d50 msgr2=0x55d75f71a130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:31.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.788+0000 7f4a9a1d9740 1 -- 172.21.15.83:0/2126488199 shutdown_connections 2024-04-24T08:54:31.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.788+0000 7f4a9a1d9740 1 -- 172.21.15.83:0/2126488199 wait complete. 2024-04-24T08:54:31.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.788+0000 7f4a94a00640 1 --2- 172.21.15.83:0/1394583378 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f4a780a3d20 0x7f4a780a61e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f4a84005fd0 tx=0x7f4a8400b2c0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:31.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.788+0000 7f4a9a1d9740 1 -- start start 2024-04-24T08:54:31.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.788+0000 7f4a9a1d9740 1 Processor -- start 2024-04-24T08:54:31.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.789+0000 7f4a9a1d9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d75f71da30 0x55d75f725a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:31.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.789+0000 7f4a9a1d9740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d75f71e350 0x55d75f725f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:31.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.789+0000 7f4a9a1d9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d75f728a20 0x55d75f7264b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:31.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.789+0000 7f4a9a1d9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d75f72b810 con 0x55d75f71da30 2024-04-24T08:54:31.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.789+0000 7f4a9a1d9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d75f63c690 con 0x55d75f728a20 2024-04-24T08:54:31.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.789+0000 7f4a9a1d9740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d75f72b990 con 0x55d75f71e350 2024-04-24T08:54:31.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.789+0000 7f4a9a1d9740 10 client.0 _ll_register_callbacks cb 0x55d75f678650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:54:31.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.789+0000 7f4a94a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d75f71da30 0x55d75f725a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:31.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.789+0000 7f4a94a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d75f71da30 0x55d75f725a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41500/0 (socket says 172.21.15.83:41500) 2024-04-24T08:54:31.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.789+0000 7f4a94a00640 1 -- 172.21.15.83:0/2658843082 learned_addr learned my addr 172.21.15.83:0/2658843082 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:54:31.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.790+0000 7f4a8fe00640 1 --2- 172.21.15.83:0/2658843082 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d75f71e350 0x55d75f725f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:31.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.790+0000 7f4a95400640 1 --2- 172.21.15.83:0/2658843082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d75f728a20 0x55d75f7264b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:31.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.790+0000 7f4a94a00640 1 -- 172.21.15.83:0/2658843082 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d75f71e350 msgr2=0x55d75f725f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:31.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.790+0000 7f4a94a00640 1 --2- 172.21.15.83:0/2658843082 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d75f71e350 0x55d75f725f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:31.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.790+0000 7f4a94a00640 1 -- 172.21.15.83:0/2658843082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d75f728a20 msgr2=0x55d75f7264b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:31.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.790+0000 7f4a94a00640 1 --2- 172.21.15.83:0/2658843082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d75f728a20 0x55d75f7264b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:31.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.790+0000 7f4a94a00640 1 -- 172.21.15.83:0/2658843082 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d75f63b920 con 0x55d75f71da30 2024-04-24T08:54:31.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.790+0000 7f4a95400640 1 --2- 172.21.15.83:0/2658843082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d75f728a20 0x55d75f7264b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:54:31.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.790+0000 7f4a94a00640 1 --2- 172.21.15.83:0/2658843082 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d75f71da30 0x55d75f725a30 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f4a84004700 tx=0x7f4a84004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1b230060c17b732 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:31.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.790+0000 7f4a76000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:54:31.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.790+0000 7f4a76000640 1 -- 172.21.15.83:0/2658843082 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a84022070 con 0x55d75f71da30 2024-04-24T08:54:31.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.790+0000 7f4a9a1d9740 1 -- 172.21.15.83:0/2658843082 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d75f690c10 con 0x55d75f71da30 2024-04-24T08:54:31.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.790+0000 7f4a76000640 1 -- 172.21.15.83:0/2658843082 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f4a84004220 con 0x55d75f71da30 2024-04-24T08:54:31.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.791+0000 7f4a76000640 1 -- 172.21.15.83:0/2658843082 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4a8402d050 con 0x55d75f71da30 2024-04-24T08:54:31.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.791+0000 7f4a9a1d9740 10 client.16671 fetch_fsmap learned FSMap version 30 2024-04-24T08:54:31.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.791+0000 7f4a9a1d9740 1 -- 172.21.15.83:0/2658843082 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d75f45f7d0 con 0x55d75f71da30 2024-04-24T08:54:31.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.791+0000 7f4a76000640 1 -- 172.21.15.83:0/2658843082 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55d75f690c10 con 0x55d75f71da30 2024-04-24T08:54:31.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.791+0000 7f4a76000640 1 -- 172.21.15.83:0/2658843082 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7f4a84003820 con 0x55d75f71da30 2024-04-24T08:54:31.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.791+0000 7f4a9a1d9740 10 client.16671 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T08:54:31.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.791+0000 7f4a9a1d9740 10 client.16671 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:54:31.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.791+0000 7f4a9a1d9740 20 client.16671 populate_metadata read hostname 'smithi083' 2024-04-24T08:54:31.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.791+0000 7f4a9a1d9740 1 --2- 172.21.15.83:0/2658843082 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d75f738ea0 0x55d75f73b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:31.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.791+0000 7f4a9a1d9740 4 client.16671 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:54:31.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.791+0000 7f4a9a1d9740 1 -- 172.21.15.83:0/2658843082 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d75f68aa80 con 0x55d75f738ea0 2024-04-24T08:54:31.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.792+0000 7f4a8fe00640 1 --2- 172.21.15.83:0/2658843082 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d75f738ea0 0x55d75f73b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:31.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.793+0000 7f4a8fe00640 1 --2- 172.21.15.83:0/2658843082 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d75f738ea0 0x55d75f73b340 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=817cd5aecf70604e server_cookie=5f9959659b147225 in_seq=0 out_seq=0 2024-04-24T08:54:31.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.793+0000 7f4a76000640 10 client.16671 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:54:31.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.796+0000 7f4a76000640 1 -- 172.21.15.83:0/2658843082 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d75f68aa80 con 0x55d75f738ea0 2024-04-24T08:54:31.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.796+0000 7f4a76000640 10 client.16671 handle_command_reply: tid=0 2024-04-24T08:54:31.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.796+0000 7f4a9a1d9740 1 client.16671 shutdown 2024-04-24T08:54:31.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.796+0000 7f4a9a1d9740 1 -- 172.21.15.83:0/2658843082 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d75f71da30 msgr2=0x55d75f725a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:31.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.796+0000 7f4a9a1d9740 1 --2- 172.21.15.83:0/2658843082 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d75f71da30 0x55d75f725a30 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f4a84004700 tx=0x7f4a84004730 comp rx=0 tx=0).stop 2024-04-24T08:54:31.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.796+0000 7f4a9a1d9740 1 -- 172.21.15.83:0/2658843082 shutdown_connections 2024-04-24T08:54:31.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.796+0000 7f4a9a1d9740 1 --2- 172.21.15.83:0/2658843082 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d75f738ea0 0x55d75f73b340 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:31.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.796+0000 7f4a9a1d9740 1 --2- 172.21.15.83:0/2658843082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d75f728a20 0x55d75f7264b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:31.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.796+0000 7f4a9a1d9740 1 --2- 172.21.15.83:0/2658843082 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d75f71e350 0x55d75f725f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:31.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.796+0000 7f4a9a1d9740 1 --2- 172.21.15.83:0/2658843082 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d75f71da30 0x55d75f725a30 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:31.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.796+0000 7f4a76000640 0 client.16671 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:54:31.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.796+0000 7f4a9a1d9740 1 -- 172.21.15.83:0/2658843082 >> 172.21.15.83:0/2658843082 conn(0x55d75f719d50 msgr2=0x55d75f718fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:31.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.797+0000 7f4a9a1d9740 1 -- 172.21.15.83:0/2658843082 shutdown_connections 2024-04-24T08:54:31.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.797+0000 7f4a9a1d9740 1 -- 172.21.15.83:0/2658843082 wait complete. 2024-04-24T08:54:31.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.797+0000 7f4a9a1d9740 20 client.16671 trim_cache size 0 max 16384 2024-04-24T08:54:31.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.816+0000 7f4a9a1d9740 1 Processor -- start 2024-04-24T08:54:31.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.816+0000 7f4a9a1d9740 1 -- start start 2024-04-24T08:54:31.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.816+0000 7f4a9a1d9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d75f71da30 0x55d75f719750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:31.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.816+0000 7f4a9a1d9740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d75f71e350 0x55d75f720190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:31.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.817+0000 7f4a9a1d9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d75f728a20 0x55d75f726100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:31.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.817+0000 7f4a9a1d9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4a901b14f0 con 0x55d75f728a20 2024-04-24T08:54:31.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.817+0000 7f4a9a1d9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4a901b30c0 con 0x55d75f71da30 2024-04-24T08:54:31.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.817+0000 7f4a9a1d9740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4a901b1bc0 con 0x55d75f71e350 2024-04-24T08:54:31.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.817+0000 7f4a95400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d75f728a20 0x55d75f726100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:31.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.817+0000 7f4a95400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d75f728a20 0x55d75f726100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41512/0 (socket says 172.21.15.83:41512) 2024-04-24T08:54:31.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.817+0000 7f4a95400640 1 -- 172.21.15.83:0/3823008500 learned_addr learned my addr 172.21.15.83:0/3823008500 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:54:31.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.817+0000 7f4a94a00640 1 --2- 172.21.15.83:0/3823008500 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d75f71da30 0x55d75f719750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:31.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.817+0000 7f4a8fe00640 1 --2- 172.21.15.83:0/3823008500 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d75f71e350 0x55d75f720190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:31.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.817+0000 7f4a95400640 1 -- 172.21.15.83:0/3823008500 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d75f71e350 msgr2=0x55d75f720190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:31.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.817+0000 7f4a95400640 1 --2- 172.21.15.83:0/3823008500 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d75f71e350 0x55d75f720190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:31.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.817+0000 7f4a95400640 1 -- 172.21.15.83:0/3823008500 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d75f71da30 msgr2=0x55d75f719750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:31.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.817+0000 7f4a95400640 1 --2- 172.21.15.83:0/3823008500 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d75f71da30 0x55d75f719750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:31.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.817+0000 7f4a95400640 1 -- 172.21.15.83:0/3823008500 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d75f3953f0 con 0x55d75f728a20 2024-04-24T08:54:31.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.817+0000 7f4a94a00640 1 --2- 172.21.15.83:0/3823008500 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d75f71da30 0x55d75f719750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:54:31.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.818+0000 7f4a95400640 1 --2- 172.21.15.83:0/3823008500 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d75f728a20 0x55d75f726100 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f4a880ef6c0 tx=0x7f4a88030640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ce378514053e1f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:31.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.818+0000 7f4a81200640 1 -- 172.21.15.83:0/3823008500 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a880d6020 con 0x55d75f728a20 2024-04-24T08:54:31.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.818+0000 7f4a81200640 1 -- 172.21.15.83:0/3823008500 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4a88036030 con 0x55d75f728a20 2024-04-24T08:54:31.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.818+0000 7f4a81200640 1 -- 172.21.15.83:0/3823008500 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a880bc1b0 con 0x55d75f728a20 2024-04-24T08:54:31.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.819+0000 7f4a9a1d9740 1 -- 172.21.15.83:0/3823008500 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d75f728a20 msgr2=0x55d75f726100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:31.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.819+0000 7f4a9a1d9740 1 --2- 172.21.15.83:0/3823008500 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d75f728a20 0x55d75f726100 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f4a880ef6c0 tx=0x7f4a88030640 comp rx=0 tx=0).stop 2024-04-24T08:54:31.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.819+0000 7f4a9a1d9740 1 -- 172.21.15.83:0/3823008500 shutdown_connections 2024-04-24T08:54:31.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.819+0000 7f4a9a1d9740 1 --2- 172.21.15.83:0/3823008500 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d75f728a20 0x55d75f726100 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:31.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.819+0000 7f4a9a1d9740 1 --2- 172.21.15.83:0/3823008500 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d75f71e350 0x55d75f720190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:31.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.819+0000 7f4a9a1d9740 1 --2- 172.21.15.83:0/3823008500 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d75f71da30 0x55d75f719750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:31.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.819+0000 7f4a9a1d9740 1 -- 172.21.15.83:0/3823008500 >> 172.21.15.83:0/3823008500 conn(0x55d75f719d50 msgr2=0x55d75f7259f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:31.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.819+0000 7f4a9a1d9740 1 -- 172.21.15.83:0/3823008500 shutdown_connections 2024-04-24T08:54:31.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.819+0000 7f4a9a1d9740 1 -- 172.21.15.83:0/3823008500 wait complete. 2024-04-24T08:54:31.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.819+0000 7f4a9a1d9740 1 -- start start 2024-04-24T08:54:31.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.820+0000 7f4a9a1d9740 1 Processor -- start 2024-04-24T08:54:31.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.820+0000 7f4a9a1d9740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d75f71da30 0x55d75f72be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:31.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.820+0000 7f4a9a1d9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d75f71e350 0x55d75f72c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:31.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.820+0000 7f4a9a1d9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d75f728a20 0x55d75f79fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:31.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.820+0000 7f4a9a1d9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4a901b2960 con 0x55d75f728a20 2024-04-24T08:54:31.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.820+0000 7f4a9a1d9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4a901b2290 con 0x55d75f71e350 2024-04-24T08:54:31.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.820+0000 7f4a9a1d9740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4a901af4d0 con 0x55d75f71da30 2024-04-24T08:54:31.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.820+0000 7f4a9a1d9740 10 client.0 _ll_register_callbacks cb 0x7f4a84015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:54:31.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.820+0000 7f4a95400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d75f728a20 0x55d75f79fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:31.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.820+0000 7f4a95400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d75f728a20 0x55d75f79fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41524/0 (socket says 172.21.15.83:41524) 2024-04-24T08:54:31.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.820+0000 7f4a95400640 1 -- 172.21.15.83:0/659632962 learned_addr learned my addr 172.21.15.83:0/659632962 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:54:31.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.821+0000 7f4a94a00640 1 --2- 172.21.15.83:0/659632962 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d75f71da30 0x55d75f72be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:31.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.821+0000 7f4a8fe00640 1 --2- 172.21.15.83:0/659632962 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d75f71e350 0x55d75f72c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:31.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.821+0000 7f4a95400640 1 -- 172.21.15.83:0/659632962 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d75f71da30 msgr2=0x55d75f72be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:31.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.821+0000 7f4a95400640 1 --2- 172.21.15.83:0/659632962 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d75f71da30 0x55d75f72be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:31.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.821+0000 7f4a95400640 1 -- 172.21.15.83:0/659632962 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d75f71e350 msgr2=0x55d75f72c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:31.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.821+0000 7f4a95400640 1 --2- 172.21.15.83:0/659632962 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d75f71e350 0x55d75f72c3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:31.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.821+0000 7f4a95400640 1 -- 172.21.15.83:0/659632962 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d75f3550f0 con 0x55d75f728a20 2024-04-24T08:54:31.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.821+0000 7f4a95400640 1 --2- 172.21.15.83:0/659632962 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d75f728a20 0x55d75f79fc90 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f4a880efb50 tx=0x7f4a880bdfe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce78d7f60574b363 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:31.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.821+0000 7f4a76000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:54:31.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.821+0000 7f4a76000640 1 -- 172.21.15.83:0/659632962 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a880d6070 con 0x55d75f728a20 2024-04-24T08:54:31.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.821+0000 7f4a76000640 1 -- 172.21.15.83:0/659632962 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f4a880046a0 con 0x55d75f728a20 2024-04-24T08:54:31.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.821+0000 7f4a9a1d9740 1 -- 172.21.15.83:0/659632962 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a88036030 con 0x55d75f728a20 2024-04-24T08:54:31.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.822+0000 7f4a76000640 1 -- 172.21.15.83:0/659632962 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4a880c6b20 con 0x55d75f728a20 2024-04-24T08:54:31.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.822+0000 7f4a9a1d9740 10 client.16677 fetch_fsmap learned FSMap version 30 2024-04-24T08:54:31.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.822+0000 7f4a9a1d9740 1 -- 172.21.15.83:0/659632962 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d75f648810 con 0x55d75f728a20 2024-04-24T08:54:31.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.822+0000 7f4a76000640 1 -- 172.21.15.83:0/659632962 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4a88036030 con 0x55d75f728a20 2024-04-24T08:54:31.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.822+0000 7f4a76000640 1 -- 172.21.15.83:0/659632962 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7f4a880d7260 con 0x55d75f728a20 2024-04-24T08:54:31.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.822+0000 7f4a9a1d9740 10 client.16677 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T08:54:31.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.822+0000 7f4a9a1d9740 10 client.16677 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:54:31.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.822+0000 7f4a9a1d9740 20 client.16677 populate_metadata read hostname 'smithi083' 2024-04-24T08:54:31.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.822+0000 7f4a9a1d9740 1 --2- 172.21.15.83:0/659632962 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d75f7a7780 0x55d75f7a9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:31.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.822+0000 7f4a9a1d9740 4 client.16677 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T08:54:31.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.822+0000 7f4a9a1d9740 1 -- 172.21.15.83:0/659632962 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d75f606360 con 0x55d75f7a7780 2024-04-24T08:54:31.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.823+0000 7f4a94a00640 1 --2- 172.21.15.83:0/659632962 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d75f7a7780 0x55d75f7a9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:31.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.824+0000 7f4a94a00640 1 --2- 172.21.15.83:0/659632962 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d75f7a7780 0x55d75f7a9ba0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35bc1136452c97ac server_cookie=2a2da8c18e914d3 in_seq=0 out_seq=0 2024-04-24T08:54:31.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.824+0000 7f4a76000640 10 client.16677 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:54:31.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.828+0000 7f4a76000640 1 -- 172.21.15.83:0/659632962 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55d75f606360 con 0x55d75f7a7780 2024-04-24T08:54:31.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.828+0000 7f4a76000640 10 client.16677 handle_command_reply: tid=0 2024-04-24T08:54:31.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.828+0000 7f4a9a1d9740 1 client.16677 shutdown 2024-04-24T08:54:31.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.828+0000 7f4a9a1d9740 1 -- 172.21.15.83:0/659632962 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d75f728a20 msgr2=0x55d75f79fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:31.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.828+0000 7f4a9a1d9740 1 --2- 172.21.15.83:0/659632962 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d75f728a20 0x55d75f79fc90 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f4a880efb50 tx=0x7f4a880bdfe0 comp rx=0 tx=0).stop 2024-04-24T08:54:31.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.828+0000 7f4a9a1d9740 1 -- 172.21.15.83:0/659632962 shutdown_connections 2024-04-24T08:54:31.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.828+0000 7f4a9a1d9740 1 --2- 172.21.15.83:0/659632962 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d75f7a7780 0x55d75f7a9ba0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:31.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.828+0000 7f4a9a1d9740 1 --2- 172.21.15.83:0/659632962 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d75f728a20 0x55d75f79fc90 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:31.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.828+0000 7f4a9a1d9740 1 --2- 172.21.15.83:0/659632962 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d75f71e350 0x55d75f72c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:31.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.828+0000 7f4a9a1d9740 1 --2- 172.21.15.83:0/659632962 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d75f71da30 0x55d75f72be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:31.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.828+0000 7f4a76000640 0 client.16677 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:54:31.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.828+0000 7f4a9a1d9740 1 -- 172.21.15.83:0/659632962 >> 172.21.15.83:0/659632962 conn(0x55d75f719d50 msgr2=0x55d75f7261f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:31.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.828+0000 7f4a9a1d9740 1 -- 172.21.15.83:0/659632962 shutdown_connections 2024-04-24T08:54:31.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.828+0000 7f4a9a1d9740 1 -- 172.21.15.83:0/659632962 wait complete. 2024-04-24T08:54:31.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.828+0000 7f4a9a1d9740 20 client.16677 trim_cache size 0 max 16384 2024-04-24T08:54:31.829 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T08:54:31.829 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-04-24T08:54:31.829 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-04-24T08:54:31.829 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T08:54:31.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.829+0000 7f4a95e00640 1 -- 172.21.15.83:0/1394583378 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f4a780a3d20 msgr2=0x7f4a780a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:31.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.829+0000 7f4a95e00640 1 --2- 172.21.15.83:0/1394583378 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f4a780a3d20 0x7f4a780a61e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f4a84005fd0 tx=0x7f4a8400b2c0 comp rx=0 tx=0).stop 2024-04-24T08:54:31.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.829+0000 7f4a95e00640 1 -- 172.21.15.83:0/1394583378 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a9019a6f0 msgr2=0x7f4a9019ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:31.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.829+0000 7f4a95e00640 1 --2- 172.21.15.83:0/1394583378 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a9019a6f0 0x7f4a9019ab70 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f4a88009dc0 tx=0x7f4a88032ff0 comp rx=0 tx=0).stop 2024-04-24T08:54:31.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.829+0000 7f4a95e00640 1 -- 172.21.15.83:0/1394583378 shutdown_connections 2024-04-24T08:54:31.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.829+0000 7f4a95e00640 1 --2- 172.21.15.83:0/1394583378 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f4a780a3d20 0x7f4a780a61e0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:31.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.829+0000 7f4a95e00640 1 --2- 172.21.15.83:0/1394583378 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a9019a6f0 0x7f4a9019ab70 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:31.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.829+0000 7f4a95e00640 1 --2- 172.21.15.83:0/1394583378 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4a901071e0 0x7f4a901a0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:31.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.829+0000 7f4a95e00640 1 --2- 172.21.15.83:0/1394583378 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a90102110 0x7f4a901a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:31.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.829+0000 7f4a95e00640 1 -- 172.21.15.83:0/1394583378 >> 172.21.15.83:0/1394583378 conn(0x7f4a900fd9d0 msgr2=0x7f4a900feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:31.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.829+0000 7f4a95e00640 1 -- 172.21.15.83:0/1394583378 shutdown_connections 2024-04-24T08:54:31.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:31.829+0000 7f4a95e00640 1 -- 172.21.15.83:0/1394583378 wait complete. 2024-04-24T08:54:31.846 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T08:54:31.846 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T08:54:31.912 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:54:31 smithi083 ceph-mon[27907]: pgmap v90: 129 pgs: 129 active+clean; 30 MiB data, 494 MiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 294 KiB/s wr, 44 op/s 2024-04-24T08:54:32.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.045+0000 7f6898c00640 1 -- 172.21.15.83:0/1722616521 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6894108e40 msgr2=0x7f68941152b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:32.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.045+0000 7f6898c00640 1 --2- 172.21.15.83:0/1722616521 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6894108e40 0x7f68941152b0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f688c009ca0 tx=0x7f688c02a950 comp rx=0 tx=0).stop 2024-04-24T08:54:32.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.045+0000 7f6898c00640 1 -- 172.21.15.83:0/1722616521 shutdown_connections 2024-04-24T08:54:32.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.045+0000 7f6898c00640 1 --2- 172.21.15.83:0/1722616521 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6894108e40 0x7f68941152b0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:32.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.045+0000 7f6898c00640 1 --2- 172.21.15.83:0/1722616521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68941084a0 0x7f6894108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:32.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.045+0000 7f6898c00640 1 --2- 172.21.15.83:0/1722616521 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f689410d3c0 0x7f689410d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:32.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.045+0000 7f6898c00640 1 -- 172.21.15.83:0/1722616521 >> 172.21.15.83:0/1722616521 conn(0x7f6894077b80 msgr2=0x7f6894079fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:32.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.045+0000 7f6898c00640 1 -- 172.21.15.83:0/1722616521 shutdown_connections 2024-04-24T08:54:32.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.045+0000 7f6898c00640 1 -- 172.21.15.83:0/1722616521 wait complete. 2024-04-24T08:54:32.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.046+0000 7f6898c00640 1 Processor -- start 2024-04-24T08:54:32.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.046+0000 7f6898c00640 1 -- start start 2024-04-24T08:54:32.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.047+0000 7f6898c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68941084a0 0x7f68941a4830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:32.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.047+0000 7f6898c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6894108e40 0x7f68941a4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:32.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.047+0000 7f6898c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f689410d3c0 0x7f689419e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:32.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.047+0000 7f6898c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6894118700 con 0x7f689410d3c0 2024-04-24T08:54:32.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.047+0000 7f6898c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6894118580 con 0x7f68941084a0 2024-04-24T08:54:32.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.047+0000 7f6898c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6894118880 con 0x7f6894108e40 2024-04-24T08:54:32.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.047+0000 7f6893e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f689410d3c0 0x7f689419e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:32.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.047+0000 7f6893e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f689410d3c0 0x7f689419e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41548/0 (socket says 172.21.15.83:41548) 2024-04-24T08:54:32.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.047+0000 7f6893e00640 1 -- 172.21.15.83:0/329192880 learned_addr learned my addr 172.21.15.83:0/329192880 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:54:32.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.048+0000 7f6893400640 1 --2- 172.21.15.83:0/329192880 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68941084a0 0x7f68941a4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:32.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.048+0000 7f6892a00640 1 --2- 172.21.15.83:0/329192880 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6894108e40 0x7f68941a4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:32.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.048+0000 7f6893e00640 1 -- 172.21.15.83:0/329192880 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6894108e40 msgr2=0x7f68941a4d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:32.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.048+0000 7f6893e00640 1 --2- 172.21.15.83:0/329192880 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6894108e40 0x7f68941a4d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:32.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.048+0000 7f6893e00640 1 -- 172.21.15.83:0/329192880 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68941084a0 msgr2=0x7f68941a4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:32.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.048+0000 7f6893e00640 1 --2- 172.21.15.83:0/329192880 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68941084a0 0x7f68941a4830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:32.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.048+0000 7f6893e00640 1 -- 172.21.15.83:0/329192880 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f689419f160 con 0x7f689410d3c0 2024-04-24T08:54:32.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.048+0000 7f6893e00640 1 --2- 172.21.15.83:0/329192880 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f689410d3c0 0x7f689419e9e0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f688c009dc0 tx=0x7f688c003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:32.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.048+0000 7f688be00640 1 -- 172.21.15.83:0/329192880 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f688c034040 con 0x7f689410d3c0 2024-04-24T08:54:32.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.048+0000 7f688be00640 1 -- 172.21.15.83:0/329192880 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f688c003e50 con 0x7f689410d3c0 2024-04-24T08:54:32.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.048+0000 7f688be00640 1 -- 172.21.15.83:0/329192880 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f688c0045b0 con 0x7f689410d3c0 2024-04-24T08:54:32.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.048+0000 7f6898c00640 1 -- 172.21.15.83:0/329192880 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f689419f3f0 con 0x7f689410d3c0 2024-04-24T08:54:32.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.049+0000 7f6898c00640 1 -- 172.21.15.83:0/329192880 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68941ab6f0 con 0x7f689410d3c0 2024-04-24T08:54:32.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.050+0000 7f6898c00640 1 -- 172.21.15.83:0/329192880 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6894069190 con 0x7f689410d3c0 2024-04-24T08:54:32.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.052+0000 7f688be00640 1 -- 172.21.15.83:0/329192880 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f688c037020 con 0x7f689410d3c0 2024-04-24T08:54:32.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.054+0000 7f688be00640 1 --2- 172.21.15.83:0/329192880 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f68780a3da0 0x7f68780a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:32.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.054+0000 7f688be00640 1 -- 172.21.15.83:0/329192880 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f688c0f1a80 con 0x7f689410d3c0 2024-04-24T08:54:32.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.054+0000 7f6893400640 1 --2- 172.21.15.83:0/329192880 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f68780a3da0 0x7f68780a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:32.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.055+0000 7f6893400640 1 --2- 172.21.15.83:0/329192880 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f68780a3da0 0x7f68780a6260 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f689419fb20 tx=0x7f6884005f90 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:32.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.057+0000 7f688be00640 1 -- 172.21.15.83:0/329192880 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f688c080ff0 con 0x7f689410d3c0 2024-04-24T08:54:32.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:54:31 smithi138 ceph-mon[31441]: pgmap v90: 129 pgs: 129 active+clean; 30 MiB data, 494 MiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 294 KiB/s wr, 44 op/s 2024-04-24T08:54:32.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:54:31 smithi189 ceph-mon[31791]: pgmap v90: 129 pgs: 129 active+clean; 30 MiB data, 494 MiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 294 KiB/s wr, 44 op/s 2024-04-24T08:54:32.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.327+0000 7f6898c00640 1 -- 172.21.15.83:0/329192880 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f689419ff10 con 0x7f689410d3c0 2024-04-24T08:54:32.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.330+0000 7f688be00640 1 -- 172.21.15.83:0/329192880 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11949 (secure 0 0 0) 0x7f689419ff10 con 0x7f689410d3c0 2024-04-24T08:54:32.329 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:54:32.329 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/358872257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":358872257},{"type":"v1","addr":"172.21.15.83:6835","nonce":358872257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3157070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3157070851},{"type":"v1","addr":"172.21.15.83:6837","nonce":3157070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T08:48:29.422576+0000","modified":"2024-04-24T08:53:42.044317+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34520,"mds_1":34493,"mds_2":34451,"mds_3":34427,"mds_4":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14658":{"gid":14658,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6839/4085470844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":4085470844},{"type":"v1","addr":"172.21.15.83:6839","nonce":4085470844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14676":{"gid":14676,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6841/2634754210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":2634754210},{"type":"v1","addr":"172.21.15.83:6841","nonce":2634754210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24382":{"gid":24382,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/1859921310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1859921310},{"type":"v1","addr":"172.21.15.189:6833","nonce":1859921310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/2241923197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2241923197},{"type":"v1","addr":"172.21.15.189:6835","nonce":2241923197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6837/3172311220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":3172311220},{"type":"v1","addr":"172.21.15.189:6837","nonce":3172311220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.189:6839/2548965336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":2548965336},{"type":"v1","addr":"172.21.15.189:6839","nonce":2548965336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.138:6833/1157967658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1157967658},{"type":"v1","addr":"172.21.15.138:6833","nonce":1157967658}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.138:6835/1909219758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1909219758},{"type":"v1","addr":"172.21.15.138:6835","nonce":1909219758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34493":{"gid":34493,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.138:6837/900566754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":900566754},{"type":"v1","addr":"172.21.15.138:6837","nonce":900566754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34520":{"gid":34520,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.138:6839/2236932393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2236932393},{"type":"v1","addr":"172.21.15.138:6839","nonce":2236932393}]},"join_fscid":-1,"export_targets":[1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34520,"qdb_cluster":[34493,34451,34427,24454,34520]},"id":1}]} 2024-04-24T08:54:32.329 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 30 2024-04-24T08:54:32.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.332+0000 7f6898c00640 1 -- 172.21.15.83:0/329192880 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f68780a3da0 msgr2=0x7f68780a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:32.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.332+0000 7f6898c00640 1 --2- 172.21.15.83:0/329192880 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f68780a3da0 0x7f68780a6260 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f689419fb20 tx=0x7f6884005f90 comp rx=0 tx=0).stop 2024-04-24T08:54:32.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.333+0000 7f6898c00640 1 -- 172.21.15.83:0/329192880 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f689410d3c0 msgr2=0x7f689419e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:32.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.333+0000 7f6898c00640 1 --2- 172.21.15.83:0/329192880 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f689410d3c0 0x7f689419e9e0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f688c009dc0 tx=0x7f688c003ce0 comp rx=0 tx=0).stop 2024-04-24T08:54:32.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.333+0000 7f6898c00640 1 -- 172.21.15.83:0/329192880 shutdown_connections 2024-04-24T08:54:32.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.333+0000 7f6898c00640 1 --2- 172.21.15.83:0/329192880 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f68780a3da0 0x7f68780a6260 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:32.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.333+0000 7f6898c00640 1 --2- 172.21.15.83:0/329192880 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f689410d3c0 0x7f689419e9e0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:32.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.333+0000 7f6898c00640 1 --2- 172.21.15.83:0/329192880 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6894108e40 0x7f68941a4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:32.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.333+0000 7f6898c00640 1 --2- 172.21.15.83:0/329192880 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68941084a0 0x7f68941a4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:32.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.333+0000 7f6898c00640 1 -- 172.21.15.83:0/329192880 >> 172.21.15.83:0/329192880 conn(0x7f6894077b80 msgr2=0x7f68940783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:32.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.333+0000 7f6898c00640 1 -- 172.21.15.83:0/329192880 shutdown_connections 2024-04-24T08:54:32.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.333+0000 7f6898c00640 1 -- 172.21.15.83:0/329192880 wait complete. 2024-04-24T08:54:32.352 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-24T08:54:32.561 INFO:tasks.workunit.client.0.smithi083.stdout: 1 8356 0.93 MB/sec warmup 66 sec latency 49.068 ms 2024-04-24T08:54:32.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.565+0000 7fcd2e400640 1 -- 172.21.15.83:0/2649373466 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd28074bf0 msgr2=0x7fcd2810e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:32.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.565+0000 7fcd2e400640 1 --2- 172.21.15.83:0/2649373466 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd28074bf0 0x7fcd2810e3a0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fcd14009900 tx=0x7fcd1402a490 comp rx=0 tx=0).stop 2024-04-24T08:54:32.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.565+0000 7fcd2e400640 1 -- 172.21.15.83:0/2649373466 shutdown_connections 2024-04-24T08:54:32.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.565+0000 7fcd2e400640 1 --2- 172.21.15.83:0/2649373466 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcd2810e8e0 0x7fcd28110cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:32.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.565+0000 7fcd2e400640 1 --2- 172.21.15.83:0/2649373466 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd28074bf0 0x7fcd2810e3a0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:32.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.565+0000 7fcd2e400640 1 --2- 172.21.15.83:0/2649373466 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd280742d0 0x7fcd280746b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:32.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.565+0000 7fcd2e400640 1 -- 172.21.15.83:0/2649373466 >> 172.21.15.83:0/2649373466 conn(0x7fcd28079450 msgr2=0x7fcd28079860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:32.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.565+0000 7fcd2e400640 1 -- 172.21.15.83:0/2649373466 shutdown_connections 2024-04-24T08:54:32.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.565+0000 7fcd2e400640 1 -- 172.21.15.83:0/2649373466 wait complete. 2024-04-24T08:54:32.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.566+0000 7fcd2e400640 1 Processor -- start 2024-04-24T08:54:32.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.567+0000 7fcd2e400640 1 -- start start 2024-04-24T08:54:32.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.567+0000 7fcd2e400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd280742d0 0x7fcd281a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:32.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.567+0000 7fcd2e400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcd28074bf0 0x7fcd281a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:32.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.567+0000 7fcd2e400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd2810e8e0 0x7fcd2819a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:32.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.567+0000 7fcd2e400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcd281144d0 con 0x7fcd280742d0 2024-04-24T08:54:32.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.567+0000 7fcd2e400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcd28114350 con 0x7fcd2810e8e0 2024-04-24T08:54:32.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.567+0000 7fcd2e400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fcd28114650 con 0x7fcd28074bf0 2024-04-24T08:54:32.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.567+0000 7fcd27400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd280742d0 0x7fcd281a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:32.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.567+0000 7fcd27400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd280742d0 0x7fcd281a0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41566/0 (socket says 172.21.15.83:41566) 2024-04-24T08:54:32.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.567+0000 7fcd27400640 1 -- 172.21.15.83:0/1794971392 learned_addr learned my addr 172.21.15.83:0/1794971392 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:54:32.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.568+0000 7fcd27e00640 1 --2- 172.21.15.83:0/1794971392 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd2810e8e0 0x7fcd2819a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:32.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.568+0000 7fcd26a00640 1 --2- 172.21.15.83:0/1794971392 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcd28074bf0 0x7fcd281a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:32.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.568+0000 7fcd27400640 1 -- 172.21.15.83:0/1794971392 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcd28074bf0 msgr2=0x7fcd281a0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:32.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.568+0000 7fcd27400640 1 --2- 172.21.15.83:0/1794971392 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcd28074bf0 0x7fcd281a0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:32.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.568+0000 7fcd27400640 1 -- 172.21.15.83:0/1794971392 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd2810e8e0 msgr2=0x7fcd2819a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:32.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.568+0000 7fcd27400640 1 --2- 172.21.15.83:0/1794971392 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd2810e8e0 0x7fcd2819a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:32.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.568+0000 7fcd27400640 1 -- 172.21.15.83:0/1794971392 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd2819afa0 con 0x7fcd280742d0 2024-04-24T08:54:32.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.568+0000 7fcd27400640 1 --2- 172.21.15.83:0/1794971392 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd280742d0 0x7fcd281a0610 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fcd1800c920 tx=0x7fcd1800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:32.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.568+0000 7fcd1fe00640 1 -- 172.21.15.83:0/1794971392 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd18007c70 con 0x7fcd280742d0 2024-04-24T08:54:32.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.568+0000 7fcd1fe00640 1 -- 172.21.15.83:0/1794971392 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcd18007e10 con 0x7fcd280742d0 2024-04-24T08:54:32.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.568+0000 7fcd2e400640 1 -- 172.21.15.83:0/1794971392 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd2819b290 con 0x7fcd280742d0 2024-04-24T08:54:32.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.569+0000 7fcd1fe00640 1 -- 172.21.15.83:0/1794971392 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd18018820 con 0x7fcd280742d0 2024-04-24T08:54:32.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.569+0000 7fcd2e400640 1 -- 172.21.15.83:0/1794971392 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd281a7460 con 0x7fcd280742d0 2024-04-24T08:54:32.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.572+0000 7fcd313db740 1 Processor -- start 2024-04-24T08:54:32.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.572+0000 7fcd313db740 1 -- start start 2024-04-24T08:54:32.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.572+0000 7fcd1fe00640 1 -- 172.21.15.83:0/1794971392 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fcd180189c0 con 0x7fcd280742d0 2024-04-24T08:54:32.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.572+0000 7fcd313db740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f9b4569950 0x55f9b4569d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:32.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.572+0000 7fcd313db740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f9b456a270 0x55f9b4574400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:32.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.572+0000 7fcd313db740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9b4574940 0x55f9b4576d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:32.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.572+0000 7fcd313db740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55f9b44cf1d0 con 0x55f9b4569950 2024-04-24T08:54:32.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.572+0000 7fcd313db740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f9b4475450 con 0x55f9b4574940 2024-04-24T08:54:32.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.572+0000 7fcd313db740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55f9b44cb190 con 0x55f9b456a270 2024-04-24T08:54:32.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.573+0000 7fcd27e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f9b4569950 0x55f9b4569d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:32.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.573+0000 7fcd27e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f9b4569950 0x55f9b4569d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41570/0 (socket says 172.21.15.83:41570) 2024-04-24T08:54:32.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.573+0000 7fcd27e00640 1 -- 172.21.15.83:0/4235675031 learned_addr learned my addr 172.21.15.83:0/4235675031 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:54:32.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.573+0000 7fcd27400640 1 --2- 172.21.15.83:0/4235675031 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f9b456a270 0x55f9b4574400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:32.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.573+0000 7fcd26a00640 1 --2- 172.21.15.83:0/4235675031 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9b4574940 0x55f9b4576d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:32.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.573+0000 7fcd27e00640 1 -- 172.21.15.83:0/4235675031 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f9b456a270 msgr2=0x55f9b4574400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:32.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.573+0000 7fcd27e00640 1 --2- 172.21.15.83:0/4235675031 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f9b456a270 0x55f9b4574400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:32.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.573+0000 7fcd27e00640 1 -- 172.21.15.83:0/4235675031 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9b4574940 msgr2=0x55f9b4576d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:32.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.573+0000 7fcd27e00640 1 --2- 172.21.15.83:0/4235675031 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9b4574940 0x55f9b4576d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:32.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.573+0000 7fcd27e00640 1 -- 172.21.15.83:0/4235675031 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9b4532310 con 0x55f9b4569950 2024-04-24T08:54:32.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.573+0000 7fcd27400640 1 --2- 172.21.15.83:0/4235675031 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f9b456a270 0x55f9b4574400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:54:32.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.574+0000 7fcd27e00640 1 --2- 172.21.15.83:0/4235675031 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f9b4569950 0x55f9b4569d30 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fcd20007da0 tx=0x7fcd2000e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8045e14c567f5316 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:32.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.574+0000 7fcd1fe00640 1 --2- 172.21.15.83:0/1794971392 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fcd0c0a3d20 0x7fcd0c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:32.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.574+0000 7fcd1cc00640 1 -- 172.21.15.83:0/4235675031 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd20014070 con 0x55f9b4569950 2024-04-24T08:54:32.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.574+0000 7fcd1cc00640 1 -- 172.21.15.83:0/4235675031 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcd20004d20 con 0x55f9b4569950 2024-04-24T08:54:32.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.574+0000 7fcd1cc00640 1 -- 172.21.15.83:0/4235675031 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd200107a0 con 0x55f9b4569950 2024-04-24T08:54:32.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.574+0000 7fcd1fe00640 1 -- 172.21.15.83:0/1794971392 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7fcd180d0e20 con 0x7fcd280742d0 2024-04-24T08:54:32.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.574+0000 7fcd27400640 1 --2- 172.21.15.83:0/1794971392 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fcd0c0a3d20 0x7fcd0c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:32.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.575+0000 7fcd313db740 1 -- 172.21.15.83:0/4235675031 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f9b4569950 msgr2=0x55f9b4569d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:32.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.575+0000 7fcd27400640 1 --2- 172.21.15.83:0/1794971392 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fcd0c0a3d20 0x7fcd0c0a61e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x55f9b456af10 tx=0x7fcd180bd040 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:32.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.575+0000 7fcd313db740 1 --2- 172.21.15.83:0/4235675031 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f9b4569950 0x55f9b4569d30 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fcd20007da0 tx=0x7fcd2000e720 comp rx=0 tx=0).stop 2024-04-24T08:54:32.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.575+0000 7fcd313db740 1 -- 172.21.15.83:0/4235675031 shutdown_connections 2024-04-24T08:54:32.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.575+0000 7fcd313db740 1 --2- 172.21.15.83:0/4235675031 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9b4574940 0x55f9b4576d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:32.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.575+0000 7fcd313db740 1 --2- 172.21.15.83:0/4235675031 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f9b456a270 0x55f9b4574400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:32.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.575+0000 7fcd313db740 1 --2- 172.21.15.83:0/4235675031 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f9b4569950 0x55f9b4569d30 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:32.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.575+0000 7fcd313db740 1 -- 172.21.15.83:0/4235675031 >> 172.21.15.83:0/4235675031 conn(0x55f9b4565c70 msgr2=0x55f9b4566050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:32.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.575+0000 7fcd313db740 1 -- 172.21.15.83:0/4235675031 shutdown_connections 2024-04-24T08:54:32.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.575+0000 7fcd313db740 1 -- 172.21.15.83:0/4235675031 wait complete. 2024-04-24T08:54:32.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.576+0000 7fcd313db740 1 -- start start 2024-04-24T08:54:32.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.576+0000 7fcd313db740 1 Processor -- start 2024-04-24T08:54:32.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.576+0000 7fcd313db740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f9b4569950 0x55f9b4574380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:32.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.576+0000 7fcd313db740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9b456a270 0x55f9b4571670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:32.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.576+0000 7fcd313db740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f9b4574940 0x55f9b4571db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:32.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.576+0000 7fcd313db740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55f9b45777d0 con 0x55f9b4569950 2024-04-24T08:54:32.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.576+0000 7fcd313db740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f9b4488810 con 0x55f9b456a270 2024-04-24T08:54:32.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.576+0000 7fcd313db740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55f9b4577950 con 0x55f9b4574940 2024-04-24T08:54:32.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.577+0000 7fcd313db740 10 client.0 _ll_register_callbacks cb 0x55f9b4456140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:54:32.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.577+0000 7fcd27e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f9b4569950 0x55f9b4574380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:32.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.577+0000 7fcd27e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f9b4569950 0x55f9b4574380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41584/0 (socket says 172.21.15.83:41584) 2024-04-24T08:54:32.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.577+0000 7fcd27e00640 1 -- 172.21.15.83:0/526984731 learned_addr learned my addr 172.21.15.83:0/526984731 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:54:32.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.577+0000 7fcd26a00640 1 --2- 172.21.15.83:0/526984731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9b456a270 0x55f9b4571670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:32.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.577+0000 7fcd27e00640 1 -- 172.21.15.83:0/526984731 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f9b4574940 msgr2=0x55f9b4571db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:32.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.577+0000 7fcd27e00640 1 --2- 172.21.15.83:0/526984731 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f9b4574940 0x55f9b4571db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:32.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.577+0000 7fcd27e00640 1 -- 172.21.15.83:0/526984731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9b456a270 msgr2=0x55f9b4571670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:32.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.577+0000 7fcd27e00640 1 --2- 172.21.15.83:0/526984731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9b456a270 0x55f9b4571670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:32.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.577+0000 7fcd27e00640 1 -- 172.21.15.83:0/526984731 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9b44993f0 con 0x55f9b4569950 2024-04-24T08:54:32.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.578+0000 7fcd27e00640 1 --2- 172.21.15.83:0/526984731 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f9b4569950 0x55f9b4574380 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fcd20005610 tx=0x7fcd20005640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=127352cad8b42442 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:32.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.578+0000 7fcd11600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:54:32.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.578+0000 7fcd11600640 1 -- 172.21.15.83:0/526984731 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd20014070 con 0x55f9b4569950 2024-04-24T08:54:32.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.578+0000 7fcd11600640 1 -- 172.21.15.83:0/526984731 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7fcd20004a40 con 0x55f9b4569950 2024-04-24T08:54:32.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.578+0000 7fcd313db740 1 -- 172.21.15.83:0/526984731 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd20004d20 con 0x55f9b4569950 2024-04-24T08:54:32.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.578+0000 7fcd11600640 1 -- 172.21.15.83:0/526984731 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcd20003970 con 0x55f9b4569950 2024-04-24T08:54:32.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.578+0000 7fcd313db740 10 client.16695 fetch_fsmap learned FSMap version 30 2024-04-24T08:54:32.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.578+0000 7fcd313db740 1 -- 172.21.15.83:0/526984731 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f9b41e13f0 con 0x55f9b4569950 2024-04-24T08:54:32.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.578+0000 7fcd11600640 1 -- 172.21.15.83:0/526984731 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcd20004d20 con 0x55f9b4569950 2024-04-24T08:54:32.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.579+0000 7fcd11600640 1 -- 172.21.15.83:0/526984731 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7fcd200104a0 con 0x55f9b4569950 2024-04-24T08:54:32.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.579+0000 7fcd313db740 10 client.16695 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T08:54:32.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.579+0000 7fcd313db740 10 client.16695 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-24T08:54:32.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.579+0000 7fcd313db740 20 client.16695 populate_metadata read hostname 'smithi083' 2024-04-24T08:54:32.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.579+0000 7fcd313db740 1 --2- 172.21.15.83:0/526984731 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55f9b45848f0 0x55f9b4586ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:32.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.579+0000 7fcd313db740 4 client.16695 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:54:32.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.579+0000 7fcd313db740 1 -- 172.21.15.83:0/526984731 --> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f9b44dcbf0 con 0x55f9b45848f0 2024-04-24T08:54:32.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.580+0000 7fcd26a00640 1 --2- 172.21.15.83:0/526984731 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55f9b45848f0 0x55f9b4586ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:32.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.581+0000 7fcd11600640 10 client.16695 ms_handle_connect on v2:172.21.15.189:6838/2548965336 2024-04-24T08:54:32.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.581+0000 7fcd26a00640 1 --2- 172.21.15.83:0/526984731 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55f9b45848f0 0x55f9b4586ce0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f8485f174d98541b server_cookie=31ce68e68558fc84 in_seq=0 out_seq=0 2024-04-24T08:54:32.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.582+0000 7fcd11600640 1 -- 172.21.15.83:0/526984731 <== mds.4 v2:172.21.15.189:6838/2548965336 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f9b44dcbf0 con 0x55f9b45848f0 2024-04-24T08:54:32.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.582+0000 7fcd11600640 10 client.16695 handle_command_reply: tid=0 2024-04-24T08:54:32.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.582+0000 7fcd313db740 1 client.16695 shutdown 2024-04-24T08:54:32.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.583+0000 7fcd313db740 1 -- 172.21.15.83:0/526984731 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f9b4569950 msgr2=0x55f9b4574380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:32.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.583+0000 7fcd313db740 1 --2- 172.21.15.83:0/526984731 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f9b4569950 0x55f9b4574380 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fcd20005610 tx=0x7fcd20005640 comp rx=0 tx=0).stop 2024-04-24T08:54:32.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.583+0000 7fcd313db740 1 -- 172.21.15.83:0/526984731 shutdown_connections 2024-04-24T08:54:32.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.583+0000 7fcd313db740 1 --2- 172.21.15.83:0/526984731 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55f9b45848f0 0x55f9b4586ce0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:32.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.583+0000 7fcd313db740 1 --2- 172.21.15.83:0/526984731 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f9b4574940 0x55f9b4571db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:32.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.583+0000 7fcd313db740 1 --2- 172.21.15.83:0/526984731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9b456a270 0x55f9b4571670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:32.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.583+0000 7fcd313db740 1 --2- 172.21.15.83:0/526984731 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f9b4569950 0x55f9b4574380 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:32.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.583+0000 7fcd313db740 1 -- 172.21.15.83:0/526984731 >> 172.21.15.83:0/526984731 conn(0x55f9b4565c70 msgr2=0x55f9b456ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:32.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.583+0000 7fcd11600640 0 client.16695 ms_handle_reset on v2:172.21.15.189:6838/2548965336 2024-04-24T08:54:32.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.583+0000 7fcd313db740 1 -- 172.21.15.83:0/526984731 shutdown_connections 2024-04-24T08:54:32.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.583+0000 7fcd313db740 1 -- 172.21.15.83:0/526984731 wait complete. 2024-04-24T08:54:32.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.583+0000 7fcd313db740 20 client.16695 trim_cache size 0 max 16384 2024-04-24T08:54:32.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.600+0000 7fcd313db740 1 Processor -- start 2024-04-24T08:54:32.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.600+0000 7fcd313db740 1 -- start start 2024-04-24T08:54:32.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.600+0000 7fcd313db740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9b4569950 0x55f9b4570f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:32.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.600+0000 7fcd313db740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f9b456a270 0x55f9b4573eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:32.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.600+0000 7fcd313db740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f9b4574940 0x55f9b45743f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:32.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.600+0000 7fcd313db740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcd281b1650 con 0x55f9b4574940 2024-04-24T08:54:32.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.600+0000 7fcd313db740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcd281b3220 con 0x55f9b4569950 2024-04-24T08:54:32.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.600+0000 7fcd313db740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fcd281b1d20 con 0x55f9b456a270 2024-04-24T08:54:32.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.601+0000 7fcd27e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f9b4574940 0x55f9b45743f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:32.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.601+0000 7fcd27e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f9b4574940 0x55f9b45743f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41598/0 (socket says 172.21.15.83:41598) 2024-04-24T08:54:32.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.601+0000 7fcd27e00640 1 -- 172.21.15.83:0/1462632451 learned_addr learned my addr 172.21.15.83:0/1462632451 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:54:32.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.601+0000 7fcd26a00640 1 --2- 172.21.15.83:0/1462632451 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f9b456a270 0x55f9b4573eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:32.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.601+0000 7fcd27e00640 1 --2- 172.21.15.83:0/1462632451 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9b4569950 0x55f9b4570f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:32.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.601+0000 7fcd27e00640 1 -- 172.21.15.83:0/1462632451 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f9b456a270 msgr2=0x55f9b4573eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:32.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.601+0000 7fcd27e00640 1 --2- 172.21.15.83:0/1462632451 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f9b456a270 0x55f9b4573eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:32.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.601+0000 7fcd27e00640 1 -- 172.21.15.83:0/1462632451 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9b4569950 msgr2=0x55f9b4570f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:32.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.601+0000 7fcd27e00640 1 --2- 172.21.15.83:0/1462632451 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9b4569950 0x55f9b4570f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:32.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.601+0000 7fcd27e00640 1 -- 172.21.15.83:0/1462632451 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9b4572120 con 0x55f9b4574940 2024-04-24T08:54:32.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.602+0000 7fcd27e00640 1 --2- 172.21.15.83:0/1462632451 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f9b4574940 0x55f9b45743f0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fcd20002230 tx=0x7fcd20012040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45b264546ee98c04 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:32.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.602+0000 7fcd13e00640 1 -- 172.21.15.83:0/1462632451 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd20014070 con 0x55f9b4574940 2024-04-24T08:54:32.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.602+0000 7fcd13e00640 1 -- 172.21.15.83:0/1462632451 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcd20003fc0 con 0x55f9b4574940 2024-04-24T08:54:32.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.602+0000 7fcd13e00640 1 -- 172.21.15.83:0/1462632451 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd20010520 con 0x55f9b4574940 2024-04-24T08:54:32.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.603+0000 7fcd313db740 1 -- 172.21.15.83:0/1462632451 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f9b4574940 msgr2=0x55f9b45743f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:32.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.603+0000 7fcd313db740 1 --2- 172.21.15.83:0/1462632451 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f9b4574940 0x55f9b45743f0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fcd20002230 tx=0x7fcd20012040 comp rx=0 tx=0).stop 2024-04-24T08:54:32.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.603+0000 7fcd313db740 1 -- 172.21.15.83:0/1462632451 shutdown_connections 2024-04-24T08:54:32.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.603+0000 7fcd313db740 1 --2- 172.21.15.83:0/1462632451 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f9b4574940 0x55f9b45743f0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:32.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.603+0000 7fcd313db740 1 --2- 172.21.15.83:0/1462632451 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f9b456a270 0x55f9b4573eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:32.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.603+0000 7fcd313db740 1 --2- 172.21.15.83:0/1462632451 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9b4569950 0x55f9b4570f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:32.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.603+0000 7fcd313db740 1 -- 172.21.15.83:0/1462632451 >> 172.21.15.83:0/1462632451 conn(0x55f9b4565c70 msgr2=0x55f9b4564df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:32.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.603+0000 7fcd313db740 1 -- 172.21.15.83:0/1462632451 shutdown_connections 2024-04-24T08:54:32.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.603+0000 7fcd313db740 1 -- 172.21.15.83:0/1462632451 wait complete. 2024-04-24T08:54:32.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.603+0000 7fcd313db740 1 -- start start 2024-04-24T08:54:32.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.604+0000 7fcd313db740 1 Processor -- start 2024-04-24T08:54:32.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.604+0000 7fcd313db740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9b4569950 0x55f9b457ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:32.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.604+0000 7fcd313db740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f9b456a270 0x55f9b456e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:32.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.604+0000 7fcd313db740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f9b4574940 0x55f9b456ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:32.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.604+0000 7fcd313db740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcd281b2ac0 con 0x55f9b4574940 2024-04-24T08:54:32.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.604+0000 7fcd313db740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcd281b23f0 con 0x55f9b4569950 2024-04-24T08:54:32.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.604+0000 7fcd313db740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fcd281af630 con 0x55f9b456a270 2024-04-24T08:54:32.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.604+0000 7fcd313db740 10 client.0 _ll_register_callbacks cb 0x7fcd20007c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:54:32.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.604+0000 7fcd27e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f9b4574940 0x55f9b456ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:32.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.605+0000 7fcd27e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f9b4574940 0x55f9b456ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41602/0 (socket says 172.21.15.83:41602) 2024-04-24T08:54:32.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.605+0000 7fcd27e00640 1 -- 172.21.15.83:0/3891866661 learned_addr learned my addr 172.21.15.83:0/3891866661 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:54:32.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.605+0000 7fcd26a00640 1 --2- 172.21.15.83:0/3891866661 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f9b456a270 0x55f9b456e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:32.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.605+0000 7fcd27e00640 1 --2- 172.21.15.83:0/3891866661 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9b4569950 0x55f9b457ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:32.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.605+0000 7fcd27e00640 1 -- 172.21.15.83:0/3891866661 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f9b456a270 msgr2=0x55f9b456e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:32.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.605+0000 7fcd27e00640 1 --2- 172.21.15.83:0/3891866661 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f9b456a270 0x55f9b456e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:32.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.605+0000 7fcd27e00640 1 -- 172.21.15.83:0/3891866661 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9b4569950 msgr2=0x55f9b457ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:32.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.605+0000 7fcd27e00640 1 --2- 172.21.15.83:0/3891866661 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9b4569950 0x55f9b457ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:32.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.605+0000 7fcd27e00640 1 -- 172.21.15.83:0/3891866661 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9b42ab7d0 con 0x55f9b4574940 2024-04-24T08:54:32.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.605+0000 7fcd27e00640 1 --2- 172.21.15.83:0/3891866661 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f9b4574940 0x55f9b456ef10 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fcd200109e0 tx=0x7fcd20010d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c37aa54c4412a94 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:32.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.605+0000 7fcd11600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:54:32.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.606+0000 7fcd11600640 1 -- 172.21.15.83:0/3891866661 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd20014030 con 0x55f9b4574940 2024-04-24T08:54:32.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.606+0000 7fcd313db740 1 -- 172.21.15.83:0/3891866661 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd20003fc0 con 0x55f9b4574940 2024-04-24T08:54:32.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.606+0000 7fcd11600640 1 -- 172.21.15.83:0/3891866661 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7fcd20005ad0 con 0x55f9b4574940 2024-04-24T08:54:32.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.606+0000 7fcd11600640 1 -- 172.21.15.83:0/3891866661 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcd20003d20 con 0x55f9b4574940 2024-04-24T08:54:32.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.606+0000 7fcd313db740 10 client.16701 fetch_fsmap learned FSMap version 30 2024-04-24T08:54:32.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.606+0000 7fcd313db740 1 -- 172.21.15.83:0/3891866661 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f9b45eac70 con 0x55f9b4574940 2024-04-24T08:54:32.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.606+0000 7fcd11600640 1 -- 172.21.15.83:0/3891866661 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcd20003fc0 con 0x55f9b4574940 2024-04-24T08:54:32.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.606+0000 7fcd11600640 1 -- 172.21.15.83:0/3891866661 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7fcd20033040 con 0x55f9b4574940 2024-04-24T08:54:32.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.607+0000 7fcd313db740 10 client.16701 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T08:54:32.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.607+0000 7fcd313db740 10 client.16701 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-24T08:54:32.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.607+0000 7fcd313db740 20 client.16701 populate_metadata read hostname 'smithi083' 2024-04-24T08:54:32.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.607+0000 7fcd313db740 1 --2- 172.21.15.83:0/3891866661 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55f9b45eafa0 0x55f9b45ed3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:32.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.607+0000 7fcd313db740 4 client.16701 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T08:54:32.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.607+0000 7fcd313db740 1 -- 172.21.15.83:0/3891866661 --> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f9b44d6a60 con 0x55f9b45eafa0 2024-04-24T08:54:32.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.607+0000 7fcd26a00640 1 --2- 172.21.15.83:0/3891866661 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55f9b45eafa0 0x55f9b45ed3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:32.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.608+0000 7fcd26a00640 1 --2- 172.21.15.83:0/3891866661 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55f9b45eafa0 0x55f9b45ed3c0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5bb83c9b4a467c21 server_cookie=59913d4236561f32 in_seq=0 out_seq=0 2024-04-24T08:54:32.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.608+0000 7fcd11600640 10 client.16701 ms_handle_connect on v2:172.21.15.189:6838/2548965336 2024-04-24T08:54:32.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.609+0000 7fcd11600640 1 -- 172.21.15.83:0/3891866661 <== mds.4 v2:172.21.15.189:6838/2548965336 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f9b44d6a60 con 0x55f9b45eafa0 2024-04-24T08:54:32.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.609+0000 7fcd11600640 10 client.16701 handle_command_reply: tid=0 2024-04-24T08:54:32.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.609+0000 7fcd313db740 1 client.16701 shutdown 2024-04-24T08:54:32.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.609+0000 7fcd313db740 1 -- 172.21.15.83:0/3891866661 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f9b4574940 msgr2=0x55f9b456ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:32.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.609+0000 7fcd313db740 1 --2- 172.21.15.83:0/3891866661 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f9b4574940 0x55f9b456ef10 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fcd200109e0 tx=0x7fcd20010d40 comp rx=0 tx=0).stop 2024-04-24T08:54:32.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.609+0000 7fcd313db740 1 -- 172.21.15.83:0/3891866661 shutdown_connections 2024-04-24T08:54:32.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.609+0000 7fcd313db740 1 --2- 172.21.15.83:0/3891866661 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55f9b45eafa0 0x55f9b45ed3c0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:32.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.609+0000 7fcd313db740 1 --2- 172.21.15.83:0/3891866661 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f9b4574940 0x55f9b456ef10 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:32.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.609+0000 7fcd313db740 1 --2- 172.21.15.83:0/3891866661 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f9b456a270 0x55f9b456e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:32.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.609+0000 7fcd313db740 1 --2- 172.21.15.83:0/3891866661 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9b4569950 0x55f9b457ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:32.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.609+0000 7fcd11600640 0 client.16701 ms_handle_reset on v2:172.21.15.189:6838/2548965336 2024-04-24T08:54:32.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.609+0000 7fcd313db740 1 -- 172.21.15.83:0/3891866661 >> 172.21.15.83:0/3891866661 conn(0x55f9b4565c70 msgr2=0x55f9b456b140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:32.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.609+0000 7fcd313db740 1 -- 172.21.15.83:0/3891866661 shutdown_connections 2024-04-24T08:54:32.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.609+0000 7fcd313db740 1 -- 172.21.15.83:0/3891866661 wait complete. 2024-04-24T08:54:32.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.610+0000 7fcd313db740 20 client.16701 trim_cache size 0 max 16384 2024-04-24T08:54:32.611 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T08:54:32.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.610+0000 7fcd2e400640 1 -- 172.21.15.83:0/1794971392 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fcd0c0a3d20 msgr2=0x7fcd0c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:32.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.610+0000 7fcd2e400640 1 --2- 172.21.15.83:0/1794971392 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fcd0c0a3d20 0x7fcd0c0a61e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x55f9b456af10 tx=0x7fcd180bd040 comp rx=0 tx=0).stop 2024-04-24T08:54:32.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.610+0000 7fcd2e400640 1 -- 172.21.15.83:0/1794971392 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd280742d0 msgr2=0x7fcd281a0610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:32.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.610+0000 7fcd2e400640 1 --2- 172.21.15.83:0/1794971392 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd280742d0 0x7fcd281a0610 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fcd1800c920 tx=0x7fcd1800cde0 comp rx=0 tx=0).stop 2024-04-24T08:54:32.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.610+0000 7fcd2e400640 1 -- 172.21.15.83:0/1794971392 shutdown_connections 2024-04-24T08:54:32.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.610+0000 7fcd2e400640 1 --2- 172.21.15.83:0/1794971392 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fcd0c0a3d20 0x7fcd0c0a61e0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:32.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.610+0000 7fcd2e400640 1 --2- 172.21.15.83:0/1794971392 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd2810e8e0 0x7fcd2819a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:32.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.610+0000 7fcd2e400640 1 --2- 172.21.15.83:0/1794971392 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcd28074bf0 0x7fcd281a0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:32.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.610+0000 7fcd2e400640 1 --2- 172.21.15.83:0/1794971392 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd280742d0 0x7fcd281a0610 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:32.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.610+0000 7fcd2e400640 1 -- 172.21.15.83:0/1794971392 >> 172.21.15.83:0/1794971392 conn(0x7fcd28079450 msgr2=0x7fcd280fef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:32.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.611+0000 7fcd2e400640 1 -- 172.21.15.83:0/1794971392 shutdown_connections 2024-04-24T08:54:32.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.611+0000 7fcd2e400640 1 -- 172.21.15.83:0/1794971392 wait complete. 2024-04-24T08:54:32.627 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-24T08:54:32.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.848+0000 7ff28da00640 1 -- 172.21.15.83:0/2552015835 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff28810c480 msgr2=0x7ff28810e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:32.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.848+0000 7ff28da00640 1 --2- 172.21.15.83:0/2552015835 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff28810c480 0x7ff28810e920 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7ff280009ca0 tx=0x7ff28002a950 comp rx=0 tx=0).stop 2024-04-24T08:54:32.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.848+0000 7ff28da00640 1 -- 172.21.15.83:0/2552015835 shutdown_connections 2024-04-24T08:54:32.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.848+0000 7ff28da00640 1 --2- 172.21.15.83:0/2552015835 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff28810c480 0x7ff28810e920 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:32.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.848+0000 7ff28da00640 1 --2- 172.21.15.83:0/2552015835 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff288103980 0x7ff28810be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:32.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.848+0000 7ff28da00640 1 --2- 172.21.15.83:0/2552015835 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff288103060 0x7ff288103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:32.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.848+0000 7ff28da00640 1 -- 172.21.15.83:0/2552015835 >> 172.21.15.83:0/2552015835 conn(0x7ff2880fc880 msgr2=0x7ff2880feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:32.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.848+0000 7ff28da00640 1 -- 172.21.15.83:0/2552015835 shutdown_connections 2024-04-24T08:54:32.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.848+0000 7ff28da00640 1 -- 172.21.15.83:0/2552015835 wait complete. 2024-04-24T08:54:32.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.849+0000 7ff28da00640 1 Processor -- start 2024-04-24T08:54:32.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.849+0000 7ff28da00640 1 -- start start 2024-04-24T08:54:32.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.850+0000 7ff28da00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff288103060 0x7ff2881a05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:32.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.850+0000 7ff28da00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff288103980 0x7ff2881a0b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:32.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.850+0000 7ff28da00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff28810c480 0x7ff28819a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:32.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.850+0000 7ff28da00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff288112160 con 0x7ff288103060 2024-04-24T08:54:32.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.850+0000 7ff28da00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff288111fe0 con 0x7ff288103980 2024-04-24T08:54:32.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.850+0000 7ff28da00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff2881122e0 con 0x7ff28810c480 2024-04-24T08:54:32.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.850+0000 7ff287e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff288103060 0x7ff2881a05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:32.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.850+0000 7ff287e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff288103060 0x7ff2881a05e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41634/0 (socket says 172.21.15.83:41634) 2024-04-24T08:54:32.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.850+0000 7ff287e00640 1 -- 172.21.15.83:0/1971222317 learned_addr learned my addr 172.21.15.83:0/1971222317 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:54:32.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.850+0000 7ff287400640 1 --2- 172.21.15.83:0/1971222317 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff288103980 0x7ff2881a0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:32.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.850+0000 7ff287e00640 1 -- 172.21.15.83:0/1971222317 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff28810c480 msgr2=0x7ff28819a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:32.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.850+0000 7ff28d000640 1 --2- 172.21.15.83:0/1971222317 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff28810c480 0x7ff28819a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:32.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.850+0000 7ff287e00640 1 --2- 172.21.15.83:0/1971222317 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff28810c480 0x7ff28819a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:32.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.850+0000 7ff287e00640 1 -- 172.21.15.83:0/1971222317 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff288103980 msgr2=0x7ff2881a0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:32.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.851+0000 7ff287e00640 1 --2- 172.21.15.83:0/1971222317 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff288103980 0x7ff2881a0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:32.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.851+0000 7ff287e00640 1 -- 172.21.15.83:0/1971222317 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff28819af60 con 0x7ff288103060 2024-04-24T08:54:32.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.851+0000 7ff287400640 1 --2- 172.21.15.83:0/1971222317 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff288103980 0x7ff2881a0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:54:32.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.851+0000 7ff28d000640 1 --2- 172.21.15.83:0/1971222317 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff28810c480 0x7ff28819a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:54:32.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.851+0000 7ff287e00640 1 --2- 172.21.15.83:0/1971222317 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff288103060 0x7ff2881a05e0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7ff27800d770 tx=0x7ff27800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:32.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.851+0000 7ff284c00640 1 -- 172.21.15.83:0/1971222317 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff278017070 con 0x7ff288103060 2024-04-24T08:54:32.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.851+0000 7ff28da00640 1 -- 172.21.15.83:0/1971222317 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff28819b250 con 0x7ff288103060 2024-04-24T08:54:32.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.851+0000 7ff284c00640 1 -- 172.21.15.83:0/1971222317 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff2780054d0 con 0x7ff288103060 2024-04-24T08:54:32.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.851+0000 7ff284c00640 1 -- 172.21.15.83:0/1971222317 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff278004cf0 con 0x7ff288103060 2024-04-24T08:54:32.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.851+0000 7ff28da00640 1 -- 172.21.15.83:0/1971222317 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2881a7560 con 0x7ff288103060 2024-04-24T08:54:32.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.854+0000 7ff291f89740 1 Processor -- start 2024-04-24T08:54:32.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.855+0000 7ff291f89740 1 -- start start 2024-04-24T08:54:32.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.855+0000 7ff291f89740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d58703bb40 0x55d58703bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:32.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.855+0000 7ff291f89740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d58703c460 0x55d5870465f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:32.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.855+0000 7ff284c00640 1 -- 172.21.15.83:0/1971222317 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff278004e90 con 0x7ff288103060 2024-04-24T08:54:32.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.855+0000 7ff291f89740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d587046b30 0x55d587048f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:32.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.855+0000 7ff291f89740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d586fa11d0 con 0x55d58703c460 2024-04-24T08:54:32.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.855+0000 7ff291f89740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d586f47450 con 0x55d587046b30 2024-04-24T08:54:32.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.855+0000 7ff291f89740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d586f9d190 con 0x55d58703bb40 2024-04-24T08:54:32.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.855+0000 7ff287e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d58703c460 0x55d5870465f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:32.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.855+0000 7ff287e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d58703c460 0x55d5870465f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41636/0 (socket says 172.21.15.83:41636) 2024-04-24T08:54:32.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.855+0000 7ff287e00640 1 -- 172.21.15.83:0/3843712908 learned_addr learned my addr 172.21.15.83:0/3843712908 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:54:32.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.856+0000 7ff28d000640 1 --2- 172.21.15.83:0/3843712908 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d58703bb40 0x55d58703bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:32.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.856+0000 7ff287400640 1 --2- 172.21.15.83:0/3843712908 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d587046b30 0x55d587048f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:32.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.856+0000 7ff287e00640 1 -- 172.21.15.83:0/3843712908 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d58703bb40 msgr2=0x55d58703bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:32.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.856+0000 7ff287e00640 1 --2- 172.21.15.83:0/3843712908 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d58703bb40 0x55d58703bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:32.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.856+0000 7ff287e00640 1 -- 172.21.15.83:0/3843712908 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d587046b30 msgr2=0x55d587048f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:32.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.856+0000 7ff287e00640 1 --2- 172.21.15.83:0/3843712908 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d587046b30 0x55d587048f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:32.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.856+0000 7ff287e00640 1 -- 172.21.15.83:0/3843712908 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d587004310 con 0x55d58703c460 2024-04-24T08:54:32.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.856+0000 7ff287e00640 1 --2- 172.21.15.83:0/3843712908 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d58703c460 0x55d5870465f0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7ff2780cd9c0 tx=0x7ff2780d8750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf0631806321e1c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:32.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.857+0000 7ff27d600640 1 -- 172.21.15.83:0/3843712908 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2780d0d50 con 0x55d58703c460 2024-04-24T08:54:32.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.857+0000 7ff27d600640 1 -- 172.21.15.83:0/3843712908 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff2780dd070 con 0x55d58703c460 2024-04-24T08:54:32.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.857+0000 7ff27d600640 1 -- 172.21.15.83:0/3843712908 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2780e2750 con 0x55d58703c460 2024-04-24T08:54:32.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.857+0000 7ff284c00640 1 --2- 172.21.15.83:0/1971222317 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff26c0a3d20 0x7ff26c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:32.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.857+0000 7ff284c00640 1 -- 172.21.15.83:0/1971222317 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7ff2780d0760 con 0x7ff288103060 2024-04-24T08:54:32.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.857+0000 7ff28d000640 1 --2- 172.21.15.83:0/1971222317 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff26c0a3d20 0x7ff26c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:32.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.858+0000 7ff291f89740 1 -- 172.21.15.83:0/3843712908 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d58703c460 msgr2=0x55d5870465f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:32.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.858+0000 7ff291f89740 1 --2- 172.21.15.83:0/3843712908 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d58703c460 0x55d5870465f0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7ff2780cd9c0 tx=0x7ff2780d8750 comp rx=0 tx=0).stop 2024-04-24T08:54:32.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.858+0000 7ff291f89740 1 -- 172.21.15.83:0/3843712908 shutdown_connections 2024-04-24T08:54:32.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.858+0000 7ff291f89740 1 --2- 172.21.15.83:0/3843712908 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d587046b30 0x55d587048f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:32.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.858+0000 7ff291f89740 1 --2- 172.21.15.83:0/3843712908 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d58703c460 0x55d5870465f0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:32.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.858+0000 7ff291f89740 1 --2- 172.21.15.83:0/3843712908 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d58703bb40 0x55d58703bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:32.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.858+0000 7ff291f89740 1 -- 172.21.15.83:0/3843712908 >> 172.21.15.83:0/3843712908 conn(0x55d587037e60 msgr2=0x55d587038240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:32.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.858+0000 7ff291f89740 1 -- 172.21.15.83:0/3843712908 shutdown_connections 2024-04-24T08:54:32.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.858+0000 7ff291f89740 1 -- 172.21.15.83:0/3843712908 wait complete. 2024-04-24T08:54:32.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.858+0000 7ff28d000640 1 --2- 172.21.15.83:0/1971222317 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff26c0a3d20 0x7ff26c0a61e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff280009a60 tx=0x7ff28003c040 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:32.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.858+0000 7ff291f89740 1 -- start start 2024-04-24T08:54:32.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.859+0000 7ff291f89740 1 Processor -- start 2024-04-24T08:54:32.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.859+0000 7ff291f89740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d58703bb40 0x55d587043910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:32.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.859+0000 7ff291f89740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d58703c460 0x55d587043e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:32.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.859+0000 7ff291f89740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d587046b30 0x55d587044390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:32.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.859+0000 7ff291f89740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d5870499c0 con 0x55d58703bb40 2024-04-24T08:54:32.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.859+0000 7ff291f89740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d586f5a810 con 0x55d58703c460 2024-04-24T08:54:32.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.859+0000 7ff291f89740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d587049b40 con 0x55d587046b30 2024-04-24T08:54:32.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.859+0000 7ff291f89740 10 client.0 _ll_register_callbacks cb 0x55d586f28140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:54:32.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.859+0000 7ff287e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d58703bb40 0x55d587043910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:32.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.860+0000 7ff287e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d58703bb40 0x55d587043910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41646/0 (socket says 172.21.15.83:41646) 2024-04-24T08:54:32.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.860+0000 7ff287e00640 1 -- 172.21.15.83:0/1639254725 learned_addr learned my addr 172.21.15.83:0/1639254725 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:54:32.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.860+0000 7ff28d000640 1 --2- 172.21.15.83:0/1639254725 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d587046b30 0x55d587044390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:32.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.860+0000 7ff287400640 1 --2- 172.21.15.83:0/1639254725 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d58703c460 0x55d587043e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:32.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.860+0000 7ff287e00640 1 -- 172.21.15.83:0/1639254725 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d587046b30 msgr2=0x55d587044390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:32.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.860+0000 7ff287e00640 1 --2- 172.21.15.83:0/1639254725 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d587046b30 0x55d587044390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:32.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.860+0000 7ff287e00640 1 -- 172.21.15.83:0/1639254725 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d58703c460 msgr2=0x55d587043e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:32.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.860+0000 7ff287e00640 1 --2- 172.21.15.83:0/1639254725 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d58703c460 0x55d587043e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:32.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.860+0000 7ff287e00640 1 -- 172.21.15.83:0/1639254725 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d586f6b3f0 con 0x55d58703bb40 2024-04-24T08:54:32.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.860+0000 7ff287e00640 1 --2- 172.21.15.83:0/1639254725 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d58703bb40 0x55d587043910 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7ff278098730 tx=0x7ff278009e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13f01a2420c85082 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:32.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.860+0000 7ff272000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:54:32.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.860+0000 7ff272000640 1 -- 172.21.15.83:0/1639254725 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff278020050 con 0x55d58703bb40 2024-04-24T08:54:32.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.861+0000 7ff272000640 1 -- 172.21.15.83:0/1639254725 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7ff2780d67b0 con 0x55d58703bb40 2024-04-24T08:54:32.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.861+0000 7ff291f89740 1 -- 172.21.15.83:0/1639254725 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2780dd070 con 0x55d58703bb40 2024-04-24T08:54:32.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.861+0000 7ff272000640 1 -- 172.21.15.83:0/1639254725 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff278098570 con 0x55d58703bb40 2024-04-24T08:54:32.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.861+0000 7ff291f89740 10 client.16713 fetch_fsmap learned FSMap version 30 2024-04-24T08:54:32.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.861+0000 7ff291f89740 1 -- 172.21.15.83:0/1639254725 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d586cb33f0 con 0x55d58703bb40 2024-04-24T08:54:32.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.861+0000 7ff272000640 1 -- 172.21.15.83:0/1639254725 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff2780dd070 con 0x55d58703bb40 2024-04-24T08:54:32.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.861+0000 7ff272000640 1 -- 172.21.15.83:0/1639254725 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7ff278098940 con 0x55d58703bb40 2024-04-24T08:54:32.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.861+0000 7ff291f89740 10 client.16713 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T08:54:32.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.861+0000 7ff291f89740 10 client.16713 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-24T08:54:32.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.862+0000 7ff291f89740 20 client.16713 populate_metadata read hostname 'smithi083' 2024-04-24T08:54:32.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.862+0000 7ff291f89740 1 --2- 172.21.15.83:0/1639254725 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55d587056c60 0x55d587059050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:32.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.862+0000 7ff291f89740 4 client.16713 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:54:32.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.862+0000 7ff291f89740 1 -- 172.21.15.83:0/1639254725 --> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d586faebf0 con 0x55d587056c60 2024-04-24T08:54:32.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.862+0000 7ff287400640 1 --2- 172.21.15.83:0/1639254725 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55d587056c60 0x55d587059050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:32.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.863+0000 7ff287400640 1 --2- 172.21.15.83:0/1639254725 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55d587056c60 0x55d587059050 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=18548dcecd024572 server_cookie=4cc538259cbec0af in_seq=0 out_seq=0 2024-04-24T08:54:32.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.863+0000 7ff272000640 10 client.16713 ms_handle_connect on v2:172.21.15.138:6832/1157967658 2024-04-24T08:54:32.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.866+0000 7ff272000640 1 -- 172.21.15.83:0/1639254725 <== mds.3 v2:172.21.15.138:6832/1157967658 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d586faebf0 con 0x55d587056c60 2024-04-24T08:54:32.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.866+0000 7ff272000640 10 client.16713 handle_command_reply: tid=0 2024-04-24T08:54:32.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.866+0000 7ff291f89740 1 client.16713 shutdown 2024-04-24T08:54:32.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.866+0000 7ff291f89740 1 -- 172.21.15.83:0/1639254725 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d58703bb40 msgr2=0x55d587043910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:32.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.866+0000 7ff291f89740 1 --2- 172.21.15.83:0/1639254725 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d58703bb40 0x55d587043910 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7ff278098730 tx=0x7ff278009e20 comp rx=0 tx=0).stop 2024-04-24T08:54:32.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.866+0000 7ff291f89740 1 -- 172.21.15.83:0/1639254725 shutdown_connections 2024-04-24T08:54:32.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.866+0000 7ff291f89740 1 --2- 172.21.15.83:0/1639254725 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55d587056c60 0x55d587059050 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:32.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.866+0000 7ff291f89740 1 --2- 172.21.15.83:0/1639254725 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d587046b30 0x55d587044390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:32.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.866+0000 7ff272000640 0 client.16713 ms_handle_reset on v2:172.21.15.138:6832/1157967658 2024-04-24T08:54:32.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.866+0000 7ff291f89740 1 --2- 172.21.15.83:0/1639254725 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d58703c460 0x55d587043e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:32.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.866+0000 7ff291f89740 1 --2- 172.21.15.83:0/1639254725 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d58703bb40 0x55d587043910 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:32.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.866+0000 7ff291f89740 1 -- 172.21.15.83:0/1639254725 >> 172.21.15.83:0/1639254725 conn(0x55d587037e60 msgr2=0x55d58703db40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:32.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.866+0000 7ff291f89740 1 -- 172.21.15.83:0/1639254725 shutdown_connections 2024-04-24T08:54:32.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.866+0000 7ff291f89740 1 -- 172.21.15.83:0/1639254725 wait complete. 2024-04-24T08:54:32.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.866+0000 7ff291f89740 20 client.16713 trim_cache size 0 max 16384 2024-04-24T08:54:32.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.882+0000 7ff291f89740 1 Processor -- start 2024-04-24T08:54:32.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.882+0000 7ff291f89740 1 -- start start 2024-04-24T08:54:32.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.883+0000 7ff291f89740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d58703bb40 0x55d587042a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:32.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.883+0000 7ff291f89740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d58703c460 0x55d5870524d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:32.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.883+0000 7ff291f89740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d587046b30 0x55d587040b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:32.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.883+0000 7ff291f89740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff2881b15a0 con 0x55d58703bb40 2024-04-24T08:54:32.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.883+0000 7ff291f89740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff2881b3170 con 0x55d587046b30 2024-04-24T08:54:32.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.883+0000 7ff291f89740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff2881b1c70 con 0x55d58703c460 2024-04-24T08:54:32.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.883+0000 7ff287e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d58703bb40 0x55d587042a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:32.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.883+0000 7ff287e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d58703bb40 0x55d587042a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41656/0 (socket says 172.21.15.83:41656) 2024-04-24T08:54:32.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.883+0000 7ff287e00640 1 -- 172.21.15.83:0/2956096344 learned_addr learned my addr 172.21.15.83:0/2956096344 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:54:32.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.883+0000 7ff287e00640 1 -- 172.21.15.83:0/2956096344 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d58703c460 msgr2=0x55d5870524d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:32.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.883+0000 7ff287e00640 1 --2- 172.21.15.83:0/2956096344 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d58703c460 0x55d5870524d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:32.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.883+0000 7ff287e00640 1 -- 172.21.15.83:0/2956096344 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d587046b30 msgr2=0x55d587040b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:32.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.883+0000 7ff287e00640 1 --2- 172.21.15.83:0/2956096344 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d587046b30 0x55d587040b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:32.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.883+0000 7ff287e00640 1 -- 172.21.15.83:0/2956096344 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d586d7d7d0 con 0x55d58703bb40 2024-04-24T08:54:32.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.884+0000 7ff287e00640 1 --2- 172.21.15.83:0/2956096344 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d58703bb40 0x55d587042a70 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7ff27809c260 tx=0x7ff27809a7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=291018901b39770b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:32.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.884+0000 7ff27cc00640 1 -- 172.21.15.83:0/2956096344 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff278020070 con 0x55d58703bb40 2024-04-24T08:54:32.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.884+0000 7ff27cc00640 1 -- 172.21.15.83:0/2956096344 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff2780dd030 con 0x55d58703bb40 2024-04-24T08:54:32.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.884+0000 7ff27cc00640 1 -- 172.21.15.83:0/2956096344 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff278005180 con 0x55d58703bb40 2024-04-24T08:54:32.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.885+0000 7ff291f89740 1 -- 172.21.15.83:0/2956096344 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d58703bb40 msgr2=0x55d587042a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:32.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.885+0000 7ff291f89740 1 --2- 172.21.15.83:0/2956096344 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d58703bb40 0x55d587042a70 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7ff27809c260 tx=0x7ff27809a7e0 comp rx=0 tx=0).stop 2024-04-24T08:54:32.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.885+0000 7ff291f89740 1 -- 172.21.15.83:0/2956096344 shutdown_connections 2024-04-24T08:54:32.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.885+0000 7ff291f89740 1 --2- 172.21.15.83:0/2956096344 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d587046b30 0x55d587040b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:32.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.885+0000 7ff291f89740 1 --2- 172.21.15.83:0/2956096344 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d58703c460 0x55d5870524d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:32.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.885+0000 7ff291f89740 1 --2- 172.21.15.83:0/2956096344 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d58703bb40 0x55d587042a70 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:32.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.885+0000 7ff291f89740 1 -- 172.21.15.83:0/2956096344 >> 172.21.15.83:0/2956096344 conn(0x55d587037e60 msgr2=0x55d587051dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:32.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.885+0000 7ff291f89740 1 -- 172.21.15.83:0/2956096344 shutdown_connections 2024-04-24T08:54:32.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.885+0000 7ff291f89740 1 -- 172.21.15.83:0/2956096344 wait complete. 2024-04-24T08:54:32.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.885+0000 7ff291f89740 1 -- start start 2024-04-24T08:54:32.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.885+0000 7ff291f89740 1 Processor -- start 2024-04-24T08:54:32.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.886+0000 7ff291f89740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d58703bb40 0x55d58705c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:32.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.886+0000 7ff291f89740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d58703c460 0x55d5870576e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:32.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.886+0000 7ff291f89740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d587046b30 0x55d587057c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:32.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.886+0000 7ff291f89740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff2881b2a10 con 0x55d58703c460 2024-04-24T08:54:32.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.886+0000 7ff291f89740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff2881b2340 con 0x55d58703bb40 2024-04-24T08:54:32.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.886+0000 7ff291f89740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff2881af580 con 0x55d587046b30 2024-04-24T08:54:32.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.886+0000 7ff291f89740 10 client.0 _ll_register_callbacks cb 0x7ff280009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:54:32.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.886+0000 7ff287400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d58703c460 0x55d5870576e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:32.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.886+0000 7ff287400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d58703c460 0x55d5870576e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41668/0 (socket says 172.21.15.83:41668) 2024-04-24T08:54:32.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.886+0000 7ff287400640 1 -- 172.21.15.83:0/243954034 learned_addr learned my addr 172.21.15.83:0/243954034 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:54:32.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.886+0000 7ff287e00640 1 --2- 172.21.15.83:0/243954034 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d58703bb40 0x55d58705c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:32.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.887+0000 7ff28d000640 1 --2- 172.21.15.83:0/243954034 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d587046b30 0x55d587057c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:32.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.887+0000 7ff287400640 1 -- 172.21.15.83:0/243954034 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d587046b30 msgr2=0x55d587057c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:32.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.887+0000 7ff287400640 1 --2- 172.21.15.83:0/243954034 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d587046b30 0x55d587057c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:32.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.887+0000 7ff287400640 1 -- 172.21.15.83:0/243954034 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d58703bb40 msgr2=0x55d58705c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:32.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.887+0000 7ff287400640 1 --2- 172.21.15.83:0/243954034 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d58703bb40 0x55d58705c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:32.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.887+0000 7ff287400640 1 -- 172.21.15.83:0/243954034 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d587058360 con 0x55d58703c460 2024-04-24T08:54:32.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.887+0000 7ff287e00640 1 --2- 172.21.15.83:0/243954034 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d58703bb40 0x55d58705c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:54:32.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.887+0000 7ff28d000640 1 --2- 172.21.15.83:0/243954034 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d587046b30 0x55d587057c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:54:32.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.887+0000 7ff287400640 1 --2- 172.21.15.83:0/243954034 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d58703c460 0x55d5870576e0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x55d587038ff0 tx=0x7ff27401af90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d0249a96e5d63ae server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:32.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.887+0000 7ff272000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:54:32.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.887+0000 7ff272000640 1 -- 172.21.15.83:0/243954034 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff274013d40 con 0x55d58703c460 2024-04-24T08:54:32.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.887+0000 7ff272000640 1 -- 172.21.15.83:0/243954034 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7ff274005050 con 0x55d58703c460 2024-04-24T08:54:32.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.887+0000 7ff291f89740 1 -- 172.21.15.83:0/243954034 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2780dd030 con 0x55d58703c460 2024-04-24T08:54:32.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.888+0000 7ff272000640 1 -- 172.21.15.83:0/243954034 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff27402a050 con 0x55d58703c460 2024-04-24T08:54:32.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.888+0000 7ff291f89740 10 client.16719 fetch_fsmap learned FSMap version 30 2024-04-24T08:54:32.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.888+0000 7ff291f89740 1 -- 172.21.15.83:0/243954034 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d5870bcdf0 con 0x55d58703c460 2024-04-24T08:54:32.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.888+0000 7ff272000640 1 -- 172.21.15.83:0/243954034 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff2780dd030 con 0x55d58703c460 2024-04-24T08:54:32.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.888+0000 7ff272000640 1 -- 172.21.15.83:0/243954034 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7ff27400f040 con 0x55d58703c460 2024-04-24T08:54:32.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.888+0000 7ff291f89740 10 client.16719 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T08:54:32.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.888+0000 7ff291f89740 10 client.16719 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-24T08:54:32.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.888+0000 7ff291f89740 20 client.16719 populate_metadata read hostname 'smithi083' 2024-04-24T08:54:32.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.888+0000 7ff291f89740 1 --2- 172.21.15.83:0/243954034 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55d5870bd520 0x55d5870bf940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:32.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.888+0000 7ff291f89740 4 client.16719 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T08:54:32.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.888+0000 7ff291f89740 1 -- 172.21.15.83:0/243954034 --> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d586fa8a60 con 0x55d5870bd520 2024-04-24T08:54:32.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.889+0000 7ff287e00640 1 --2- 172.21.15.83:0/243954034 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55d5870bd520 0x55d5870bf940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:32.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.890+0000 7ff287e00640 1 --2- 172.21.15.83:0/243954034 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55d5870bd520 0x55d5870bf940 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=edd43dc4daf97009 server_cookie=c325b734b5aa2b56 in_seq=0 out_seq=0 2024-04-24T08:54:32.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.890+0000 7ff272000640 10 client.16719 ms_handle_connect on v2:172.21.15.138:6832/1157967658 2024-04-24T08:54:32.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.890+0000 7ff272000640 1 -- 172.21.15.83:0/243954034 <== mds.3 v2:172.21.15.138:6832/1157967658 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d586fa8a60 con 0x55d5870bd520 2024-04-24T08:54:32.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.890+0000 7ff272000640 10 client.16719 handle_command_reply: tid=0 2024-04-24T08:54:32.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.890+0000 7ff291f89740 1 client.16719 shutdown 2024-04-24T08:54:32.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.890+0000 7ff291f89740 1 -- 172.21.15.83:0/243954034 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d58703c460 msgr2=0x55d5870576e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:32.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.890+0000 7ff291f89740 1 --2- 172.21.15.83:0/243954034 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d58703c460 0x55d5870576e0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x55d587038ff0 tx=0x7ff27401af90 comp rx=0 tx=0).stop 2024-04-24T08:54:32.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.890+0000 7ff291f89740 1 -- 172.21.15.83:0/243954034 shutdown_connections 2024-04-24T08:54:32.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.890+0000 7ff291f89740 1 --2- 172.21.15.83:0/243954034 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55d5870bd520 0x55d5870bf940 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:32.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.891+0000 7ff291f89740 1 --2- 172.21.15.83:0/243954034 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d587046b30 0x55d587057c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:32.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.891+0000 7ff291f89740 1 --2- 172.21.15.83:0/243954034 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d58703c460 0x55d5870576e0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:32.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.891+0000 7ff291f89740 1 --2- 172.21.15.83:0/243954034 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d58703bb40 0x55d58705c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:32.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.891+0000 7ff272000640 0 client.16719 ms_handle_reset on v2:172.21.15.138:6832/1157967658 2024-04-24T08:54:32.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.891+0000 7ff291f89740 1 -- 172.21.15.83:0/243954034 >> 172.21.15.83:0/243954034 conn(0x55d587037e60 msgr2=0x55d587036f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:32.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.891+0000 7ff291f89740 1 -- 172.21.15.83:0/243954034 shutdown_connections 2024-04-24T08:54:32.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.891+0000 7ff291f89740 1 -- 172.21.15.83:0/243954034 wait complete. 2024-04-24T08:54:32.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.891+0000 7ff291f89740 20 client.16719 trim_cache size 0 max 16384 2024-04-24T08:54:32.893 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T08:54:32.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.892+0000 7ff28da00640 1 -- 172.21.15.83:0/1971222317 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff26c0a3d20 msgr2=0x7ff26c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:32.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.892+0000 7ff28da00640 1 --2- 172.21.15.83:0/1971222317 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff26c0a3d20 0x7ff26c0a61e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff280009a60 tx=0x7ff28003c040 comp rx=0 tx=0).stop 2024-04-24T08:54:32.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.892+0000 7ff28da00640 1 -- 172.21.15.83:0/1971222317 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff288103060 msgr2=0x7ff2881a05e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:32.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.892+0000 7ff28da00640 1 --2- 172.21.15.83:0/1971222317 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff288103060 0x7ff2881a05e0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7ff27800d770 tx=0x7ff27800dc30 comp rx=0 tx=0).stop 2024-04-24T08:54:32.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.892+0000 7ff28da00640 1 -- 172.21.15.83:0/1971222317 shutdown_connections 2024-04-24T08:54:32.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.892+0000 7ff28da00640 1 --2- 172.21.15.83:0/1971222317 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff26c0a3d20 0x7ff26c0a61e0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:32.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.892+0000 7ff28da00640 1 --2- 172.21.15.83:0/1971222317 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff28810c480 0x7ff28819a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:32.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.892+0000 7ff28da00640 1 --2- 172.21.15.83:0/1971222317 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff288103980 0x7ff2881a0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:32.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.892+0000 7ff28da00640 1 --2- 172.21.15.83:0/1971222317 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff288103060 0x7ff2881a05e0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:32.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.892+0000 7ff28da00640 1 -- 172.21.15.83:0/1971222317 >> 172.21.15.83:0/1971222317 conn(0x7ff2880fc880 msgr2=0x7ff288100be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:32.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.892+0000 7ff28da00640 1 -- 172.21.15.83:0/1971222317 shutdown_connections 2024-04-24T08:54:32.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:32.892+0000 7ff28da00640 1 -- 172.21.15.83:0/1971222317 wait complete. 2024-04-24T08:54:32.907 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:54:32 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/329192880' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:54:32.910 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T08:54:33.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.120+0000 7f4337e00640 1 -- 172.21.15.83:0/1497892374 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4338074cf0 msgr2=0x7f433810e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:33.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.120+0000 7f4337e00640 1 --2- 172.21.15.83:0/1497892374 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4338074cf0 0x7f433810e200 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f4320009900 tx=0x7f432002a480 comp rx=0 tx=0).stop 2024-04-24T08:54:33.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.120+0000 7f4337e00640 1 -- 172.21.15.83:0/1497892374 shutdown_connections 2024-04-24T08:54:33.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.120+0000 7f4337e00640 1 --2- 172.21.15.83:0/1497892374 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f433810e740 0x7f4338110b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:33.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.120+0000 7f4337e00640 1 --2- 172.21.15.83:0/1497892374 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4338074cf0 0x7f433810e200 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:33.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.120+0000 7f4337e00640 1 --2- 172.21.15.83:0/1497892374 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f43380743d0 0x7f43380747b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:33.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.120+0000 7f4337e00640 1 -- 172.21.15.83:0/1497892374 >> 172.21.15.83:0/1497892374 conn(0x7f43380fd9d0 msgr2=0x7f43380ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:33.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.120+0000 7f4337e00640 1 -- 172.21.15.83:0/1497892374 shutdown_connections 2024-04-24T08:54:33.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.121+0000 7f4337e00640 1 -- 172.21.15.83:0/1497892374 wait complete. 2024-04-24T08:54:33.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.122+0000 7f4337e00640 1 Processor -- start 2024-04-24T08:54:33.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.122+0000 7f4337e00640 1 -- start start 2024-04-24T08:54:33.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.122+0000 7f4337e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f43380743d0 0x7f433810e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:33.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.122+0000 7f4337e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4338074cf0 0x7f433810b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:33.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.122+0000 7f4337e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f433810e740 0x7f433810b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:33.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.122+0000 7f4337e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f43381143e0 con 0x7f4338074cf0 2024-04-24T08:54:33.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.122+0000 7f4337e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4338114260 con 0x7f433810e740 2024-04-24T08:54:33.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.122+0000 7f4337e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4338114560 con 0x7f43380743d0 2024-04-24T08:54:33.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.123+0000 7f432e000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4338074cf0 0x7f433810b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:33.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.123+0000 7f432e000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4338074cf0 0x7f433810b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41688/0 (socket says 172.21.15.83:41688) 2024-04-24T08:54:33.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.123+0000 7f432e000640 1 -- 172.21.15.83:0/2869298610 learned_addr learned my addr 172.21.15.83:0/2869298610 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:54:33.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.123+0000 7f4336a00640 1 --2- 172.21.15.83:0/2869298610 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f43380743d0 0x7f433810e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:33.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.123+0000 7f4337400640 1 --2- 172.21.15.83:0/2869298610 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f433810e740 0x7f433810b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:33.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.123+0000 7f432e000640 1 -- 172.21.15.83:0/2869298610 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f43380743d0 msgr2=0x7f433810e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:33.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.123+0000 7f432e000640 1 --2- 172.21.15.83:0/2869298610 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f43380743d0 0x7f433810e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:33.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.123+0000 7f432e000640 1 -- 172.21.15.83:0/2869298610 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f433810e740 msgr2=0x7f433810b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:33.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.123+0000 7f432e000640 1 --2- 172.21.15.83:0/2869298610 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f433810e740 0x7f433810b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:33.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.123+0000 7f432e000640 1 -- 172.21.15.83:0/2869298610 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43381ab120 con 0x7f4338074cf0 2024-04-24T08:54:33.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.123+0000 7f432e000640 1 --2- 172.21.15.83:0/2869298610 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4338074cf0 0x7f433810b200 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f4320005ab0 tx=0x7f4320003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:33.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.124+0000 7f432fe00640 1 -- 172.21.15.83:0/2869298610 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f432003f070 con 0x7f4338074cf0 2024-04-24T08:54:33.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.124+0000 7f432fe00640 1 -- 172.21.15.83:0/2869298610 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4320031d60 con 0x7f4338074cf0 2024-04-24T08:54:33.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.124+0000 7f432fe00640 1 -- 172.21.15.83:0/2869298610 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4320035030 con 0x7f4338074cf0 2024-04-24T08:54:33.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.124+0000 7f4337e00640 1 -- 172.21.15.83:0/2869298610 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43381ab2f0 con 0x7f4338074cf0 2024-04-24T08:54:33.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.124+0000 7f4337e00640 1 -- 172.21.15.83:0/2869298610 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43381ab700 con 0x7f4338074cf0 2024-04-24T08:54:33.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.128+0000 7f4340402740 1 Processor -- start 2024-04-24T08:54:33.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.128+0000 7f4340402740 1 -- start start 2024-04-24T08:54:33.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.128+0000 7f432fe00640 1 -- 172.21.15.83:0/2869298610 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4320042070 con 0x7f4338074cf0 2024-04-24T08:54:33.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.128+0000 7f4340402740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55823534fb40 0x55823534ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:33.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.128+0000 7f4340402740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558235350460 0x55823535a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:33.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.128+0000 7f4340402740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55823535ab30 0x55823535cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:33.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.128+0000 7f4340402740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5582352b51d0 con 0x558235350460 2024-04-24T08:54:33.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.128+0000 7f4340402740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55823525b450 con 0x55823534fb40 2024-04-24T08:54:33.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.128+0000 7f4340402740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5582352b1190 con 0x55823535ab30 2024-04-24T08:54:33.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.129+0000 7f4336a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558235350460 0x55823535a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:33.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.129+0000 7f4336a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558235350460 0x55823535a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41692/0 (socket says 172.21.15.83:41692) 2024-04-24T08:54:33.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.129+0000 7f4336a00640 1 -- 172.21.15.83:0/831607006 learned_addr learned my addr 172.21.15.83:0/831607006 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:54:33.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.129+0000 7f432e000640 1 --2- 172.21.15.83:0/831607006 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55823535ab30 0x55823535cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:33.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.129+0000 7f4337400640 1 --2- 172.21.15.83:0/831607006 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55823534fb40 0x55823534ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:33.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.129+0000 7f4336a00640 1 -- 172.21.15.83:0/831607006 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55823535ab30 msgr2=0x55823535cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:33.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.129+0000 7f4336a00640 1 --2- 172.21.15.83:0/831607006 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55823535ab30 0x55823535cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:33.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.129+0000 7f4336a00640 1 -- 172.21.15.83:0/831607006 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55823534fb40 msgr2=0x55823534ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:33.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.129+0000 7f4336a00640 1 --2- 172.21.15.83:0/831607006 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55823534fb40 0x55823534ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:33.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.129+0000 7f4336a00640 1 -- 172.21.15.83:0/831607006 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558235318310 con 0x558235350460 2024-04-24T08:54:33.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.129+0000 7f432e000640 1 --2- 172.21.15.83:0/831607006 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55823535ab30 0x55823535cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:54:33.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.130+0000 7f4336a00640 1 --2- 172.21.15.83:0/831607006 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558235350460 0x55823535a5f0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f432800bcf0 tx=0x7f432800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb1c5dca7a357f56 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:33.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.130+0000 7f4327400640 1 -- 172.21.15.83:0/831607006 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4328015070 con 0x558235350460 2024-04-24T08:54:33.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.130+0000 7f4327400640 1 -- 172.21.15.83:0/831607006 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f432801a910 con 0x558235350460 2024-04-24T08:54:33.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.130+0000 7f4327400640 1 -- 172.21.15.83:0/831607006 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4328011930 con 0x558235350460 2024-04-24T08:54:33.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.130+0000 7f432fe00640 1 --2- 172.21.15.83:0/2869298610 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f43180a3d20 0x7f43180a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:33.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.130+0000 7f4337400640 1 --2- 172.21.15.83:0/2869298610 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f43180a3d20 0x7f43180a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:33.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.130+0000 7f432fe00640 1 -- 172.21.15.83:0/2869298610 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f43200f1530 con 0x7f4338074cf0 2024-04-24T08:54:33.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.131+0000 7f4340402740 1 -- 172.21.15.83:0/831607006 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558235350460 msgr2=0x55823535a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:33.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.131+0000 7f4340402740 1 --2- 172.21.15.83:0/831607006 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558235350460 0x55823535a5f0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f432800bcf0 tx=0x7f432800f730 comp rx=0 tx=0).stop 2024-04-24T08:54:33.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.131+0000 7f4340402740 1 -- 172.21.15.83:0/831607006 shutdown_connections 2024-04-24T08:54:33.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.131+0000 7f4340402740 1 --2- 172.21.15.83:0/831607006 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55823535ab30 0x55823535cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:33.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.131+0000 7f4340402740 1 --2- 172.21.15.83:0/831607006 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558235350460 0x55823535a5f0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:33.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.131+0000 7f4340402740 1 --2- 172.21.15.83:0/831607006 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55823534fb40 0x55823534ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:33.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.131+0000 7f4340402740 1 -- 172.21.15.83:0/831607006 >> 172.21.15.83:0/831607006 conn(0x55823534be60 msgr2=0x55823534c240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:33.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.131+0000 7f4337400640 1 --2- 172.21.15.83:0/2869298610 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f43180a3d20 0x7f43180a61e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f433806b560 tx=0x7f433000c330 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:33.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.131+0000 7f4340402740 1 -- 172.21.15.83:0/831607006 shutdown_connections 2024-04-24T08:54:33.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.131+0000 7f4340402740 1 -- 172.21.15.83:0/831607006 wait complete. 2024-04-24T08:54:33.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.132+0000 7f4340402740 1 -- start start 2024-04-24T08:54:33.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.132+0000 7f4340402740 1 Processor -- start 2024-04-24T08:54:33.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.133+0000 7f4340402740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55823534fb40 0x55823535a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:33.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.133+0000 7f4340402740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558235350460 0x5582353578b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:33.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.133+0000 7f4340402740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55823535ab30 0x558235357fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:33.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.133+0000 7f4340402740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55823535d9c0 con 0x558235350460 2024-04-24T08:54:33.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.133+0000 7f4340402740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55823526e810 con 0x55823535ab30 2024-04-24T08:54:33.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.133+0000 7f4340402740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55823535db40 con 0x55823534fb40 2024-04-24T08:54:33.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.133+0000 7f4340402740 10 client.0 _ll_register_callbacks cb 0x55823523c140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:54:33.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.133+0000 7f432e000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558235350460 0x5582353578b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:33.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.133+0000 7f432e000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558235350460 0x5582353578b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41696/0 (socket says 172.21.15.83:41696) 2024-04-24T08:54:33.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.133+0000 7f432e000640 1 -- 172.21.15.83:0/948030215 learned_addr learned my addr 172.21.15.83:0/948030215 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:54:33.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.133+0000 7f4337400640 1 --2- 172.21.15.83:0/948030215 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55823535ab30 0x558235357fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:33.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.133+0000 7f432e000640 1 -- 172.21.15.83:0/948030215 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55823534fb40 msgr2=0x55823535a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:33.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.133+0000 7f432e000640 1 --2- 172.21.15.83:0/948030215 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55823534fb40 0x55823535a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:33.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.133+0000 7f432e000640 1 -- 172.21.15.83:0/948030215 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55823535ab30 msgr2=0x558235357fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:33.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.133+0000 7f432e000640 1 --2- 172.21.15.83:0/948030215 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55823535ab30 0x558235357fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:33.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.133+0000 7f432e000640 1 -- 172.21.15.83:0/948030215 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55823527f3f0 con 0x558235350460 2024-04-24T08:54:33.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.134+0000 7f432e000640 1 --2- 172.21.15.83:0/948030215 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558235350460 0x5582353578b0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f4320000c00 tx=0x7f43200091c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75db55004de794c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:33.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.134+0000 7f431fe00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:54:33.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.134+0000 7f431fe00640 1 -- 172.21.15.83:0/948030215 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4320049070 con 0x558235350460 2024-04-24T08:54:33.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.134+0000 7f431fe00640 1 -- 172.21.15.83:0/948030215 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f432003b5f0 con 0x558235350460 2024-04-24T08:54:33.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.134+0000 7f4340402740 1 -- 172.21.15.83:0/948030215 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f432801a910 con 0x558235350460 2024-04-24T08:54:33.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.134+0000 7f431fe00640 1 -- 172.21.15.83:0/948030215 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f43200b93a0 con 0x558235350460 2024-04-24T08:54:33.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.134+0000 7f4340402740 10 client.16731 fetch_fsmap learned FSMap version 30 2024-04-24T08:54:33.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.134+0000 7f4340402740 1 -- 172.21.15.83:0/948030215 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558234fc73f0 con 0x558235350460 2024-04-24T08:54:33.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.135+0000 7f431fe00640 1 -- 172.21.15.83:0/948030215 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f432801a910 con 0x558235350460 2024-04-24T08:54:33.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.135+0000 7f431fe00640 1 -- 172.21.15.83:0/948030215 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7f43200c1110 con 0x558235350460 2024-04-24T08:54:33.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.135+0000 7f4340402740 10 client.16731 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T08:54:33.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.135+0000 7f4340402740 10 client.16731 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-24T08:54:33.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.135+0000 7f4340402740 20 client.16731 populate_metadata read hostname 'smithi083' 2024-04-24T08:54:33.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.135+0000 7f4340402740 1 --2- 172.21.15.83:0/948030215 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55823536acc0 0x55823536d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:33.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.135+0000 7f4340402740 4 client.16731 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:54:33.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.135+0000 7f4340402740 1 -- 172.21.15.83:0/948030215 --> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5582352c2bf0 con 0x55823536acc0 2024-04-24T08:54:33.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.135+0000 7f4336a00640 1 --2- 172.21.15.83:0/948030215 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55823536acc0 0x55823536d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:33.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.137+0000 7f4336a00640 1 --2- 172.21.15.83:0/948030215 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55823536acc0 0x55823536d130 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5b69ebb5203d6755 server_cookie=fae281876dc59b26 in_seq=0 out_seq=0 2024-04-24T08:54:33.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.137+0000 7f431fe00640 10 client.16731 ms_handle_connect on v2:172.21.15.138:6834/1909219758 2024-04-24T08:54:33.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.138+0000 7f431fe00640 1 -- 172.21.15.83:0/948030215 <== mds.2 v2:172.21.15.138:6834/1909219758 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5582352c2bf0 con 0x55823536acc0 2024-04-24T08:54:33.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.138+0000 7f431fe00640 10 client.16731 handle_command_reply: tid=0 2024-04-24T08:54:33.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.139+0000 7f4340402740 1 client.16731 shutdown 2024-04-24T08:54:33.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.139+0000 7f4340402740 1 -- 172.21.15.83:0/948030215 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558235350460 msgr2=0x5582353578b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:33.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.139+0000 7f4340402740 1 --2- 172.21.15.83:0/948030215 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558235350460 0x5582353578b0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f4320000c00 tx=0x7f43200091c0 comp rx=0 tx=0).stop 2024-04-24T08:54:33.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.139+0000 7f4340402740 1 -- 172.21.15.83:0/948030215 shutdown_connections 2024-04-24T08:54:33.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.139+0000 7f4340402740 1 --2- 172.21.15.83:0/948030215 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55823536acc0 0x55823536d130 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:33.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.139+0000 7f4340402740 1 --2- 172.21.15.83:0/948030215 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55823535ab30 0x558235357fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:33.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.139+0000 7f4340402740 1 --2- 172.21.15.83:0/948030215 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558235350460 0x5582353578b0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:33.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.139+0000 7f4340402740 1 --2- 172.21.15.83:0/948030215 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55823534fb40 0x55823535a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:33.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.139+0000 7f431fe00640 0 client.16731 ms_handle_reset on v2:172.21.15.138:6834/1909219758 2024-04-24T08:54:33.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.139+0000 7f4340402740 1 -- 172.21.15.83:0/948030215 >> 172.21.15.83:0/948030215 conn(0x55823534be60 msgr2=0x558235351c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:33.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.139+0000 7f4340402740 1 -- 172.21.15.83:0/948030215 shutdown_connections 2024-04-24T08:54:33.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.139+0000 7f4340402740 1 -- 172.21.15.83:0/948030215 wait complete. 2024-04-24T08:54:33.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.139+0000 7f4340402740 20 client.16731 trim_cache size 0 max 16384 2024-04-24T08:54:33.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.155+0000 7f4340402740 1 Processor -- start 2024-04-24T08:54:33.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.155+0000 7f4340402740 1 -- start start 2024-04-24T08:54:33.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.155+0000 7f4340402740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55823534fb40 0x558235366880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:33.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.155+0000 7f4340402740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558235350460 0x55823535a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:33.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.155+0000 7f4340402740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55823535ab30 0x558235372a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:33.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.155+0000 7f4340402740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f43381b59b0 con 0x55823535ab30 2024-04-24T08:54:33.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.155+0000 7f4340402740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f43381b7580 con 0x55823534fb40 2024-04-24T08:54:33.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.155+0000 7f4340402740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f43381b6080 con 0x558235350460 2024-04-24T08:54:33.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.155+0000 7f4337400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55823535ab30 0x558235372a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:33.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.155+0000 7f4337400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55823535ab30 0x558235372a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41700/0 (socket says 172.21.15.83:41700) 2024-04-24T08:54:33.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.155+0000 7f4337400640 1 -- 172.21.15.83:0/1549671002 learned_addr learned my addr 172.21.15.83:0/1549671002 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:54:33.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.156+0000 7f432e000640 1 --2- 172.21.15.83:0/1549671002 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558235350460 0x55823535a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:33.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.156+0000 7f4337400640 1 -- 172.21.15.83:0/1549671002 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558235350460 msgr2=0x55823535a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:33.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.156+0000 7f4337400640 1 --2- 172.21.15.83:0/1549671002 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558235350460 0x55823535a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:33.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.156+0000 7f4337400640 1 -- 172.21.15.83:0/1549671002 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55823534fb40 msgr2=0x558235366880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:33.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.156+0000 7f4337400640 1 --2- 172.21.15.83:0/1549671002 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55823534fb40 0x558235366880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:33.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.156+0000 7f4337400640 1 -- 172.21.15.83:0/1549671002 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582350917d0 con 0x55823535ab30 2024-04-24T08:54:33.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.156+0000 7f432e000640 1 --2- 172.21.15.83:0/1549671002 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558235350460 0x55823535a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:54:33.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.156+0000 7f4337400640 1 --2- 172.21.15.83:0/1549671002 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55823535ab30 0x558235372a40 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f43300114f0 tx=0x7f4330013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e181e6f0a8e4871 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:33.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.156+0000 7f4326a00640 1 -- 172.21.15.83:0/1549671002 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f433001e830 con 0x55823535ab30 2024-04-24T08:54:33.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.156+0000 7f4326a00640 1 -- 172.21.15.83:0/1549671002 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4330022070 con 0x55823535ab30 2024-04-24T08:54:33.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.156+0000 7f4326a00640 1 -- 172.21.15.83:0/1549671002 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4330027da0 con 0x55823535ab30 2024-04-24T08:54:33.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.157+0000 7f4340402740 1 -- 172.21.15.83:0/1549671002 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55823535ab30 msgr2=0x558235372a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:33.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.157+0000 7f4340402740 1 --2- 172.21.15.83:0/1549671002 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55823535ab30 0x558235372a40 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f43300114f0 tx=0x7f4330013000 comp rx=0 tx=0).stop 2024-04-24T08:54:33.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.157+0000 7f4340402740 1 -- 172.21.15.83:0/1549671002 shutdown_connections 2024-04-24T08:54:33.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.157+0000 7f4340402740 1 --2- 172.21.15.83:0/1549671002 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55823535ab30 0x558235372a40 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:33.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.157+0000 7f4340402740 1 --2- 172.21.15.83:0/1549671002 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558235350460 0x55823535a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:33.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.157+0000 7f4340402740 1 --2- 172.21.15.83:0/1549671002 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55823534fb40 0x558235366880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:33.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.157+0000 7f4340402740 1 -- 172.21.15.83:0/1549671002 >> 172.21.15.83:0/1549671002 conn(0x55823534be60 msgr2=0x558235352b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:33.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.157+0000 7f4340402740 1 -- 172.21.15.83:0/1549671002 shutdown_connections 2024-04-24T08:54:33.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.157+0000 7f4340402740 1 -- 172.21.15.83:0/1549671002 wait complete. 2024-04-24T08:54:33.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.158+0000 7f4340402740 1 -- start start 2024-04-24T08:54:33.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.158+0000 7f4340402740 1 Processor -- start 2024-04-24T08:54:33.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.158+0000 7f4340402740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55823534fb40 0x55823535bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:33.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.158+0000 7f4340402740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558235350460 0x55823535c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:33.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.158+0000 7f4340402740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55823535ab30 0x5582353612b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:33.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.158+0000 7f4340402740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f43381b6e20 con 0x558235350460 2024-04-24T08:54:33.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.158+0000 7f4340402740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f43381b6750 con 0x55823535ab30 2024-04-24T08:54:33.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.158+0000 7f4340402740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f43381b3990 con 0x55823534fb40 2024-04-24T08:54:33.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.158+0000 7f4340402740 10 client.0 _ll_register_callbacks cb 0x7f43200f4f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:54:33.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.159+0000 7f432e000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558235350460 0x55823535c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:33.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.159+0000 7f432e000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558235350460 0x55823535c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41714/0 (socket says 172.21.15.83:41714) 2024-04-24T08:54:33.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.159+0000 7f432e000640 1 -- 172.21.15.83:0/2590846174 learned_addr learned my addr 172.21.15.83:0/2590846174 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:54:33.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.159+0000 7f4336a00640 1 --2- 172.21.15.83:0/2590846174 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55823534fb40 0x55823535bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:33.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.159+0000 7f4337400640 1 --2- 172.21.15.83:0/2590846174 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55823535ab30 0x5582353612b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:33.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.159+0000 7f432e000640 1 -- 172.21.15.83:0/2590846174 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55823534fb40 msgr2=0x55823535bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:33.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.159+0000 7f432e000640 1 --2- 172.21.15.83:0/2590846174 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55823534fb40 0x55823535bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:33.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.159+0000 7f432e000640 1 -- 172.21.15.83:0/2590846174 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55823535ab30 msgr2=0x5582353612b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:33.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.159+0000 7f432e000640 1 --2- 172.21.15.83:0/2590846174 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55823535ab30 0x5582353612b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:33.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.159+0000 7f432e000640 1 -- 172.21.15.83:0/2590846174 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55823535ca10 con 0x558235350460 2024-04-24T08:54:33.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.159+0000 7f432e000640 1 --2- 172.21.15.83:0/2590846174 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558235350460 0x55823535c150 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f43200eeeb0 tx=0x7f43200c5c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=834cb4adb4a354e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:33.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.160+0000 7f431fe00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:54:33.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.160+0000 7f431fe00640 1 -- 172.21.15.83:0/2590846174 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4320049070 con 0x558235350460 2024-04-24T08:54:33.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.160+0000 7f4340402740 1 -- 172.21.15.83:0/2590846174 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4330022070 con 0x558235350460 2024-04-24T08:54:33.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.160+0000 7f431fe00640 1 -- 172.21.15.83:0/2590846174 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f43200f1120 con 0x558235350460 2024-04-24T08:54:33.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.160+0000 7f431fe00640 1 -- 172.21.15.83:0/2590846174 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f43200bca60 con 0x558235350460 2024-04-24T08:54:33.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.160+0000 7f4340402740 10 client.16737 fetch_fsmap learned FSMap version 30 2024-04-24T08:54:33.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.160+0000 7f4340402740 1 -- 172.21.15.83:0/2590846174 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5582353d1850 con 0x558235350460 2024-04-24T08:54:33.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.160+0000 7f431fe00640 1 -- 172.21.15.83:0/2590846174 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4330022070 con 0x558235350460 2024-04-24T08:54:33.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.160+0000 7f431fe00640 1 -- 172.21.15.83:0/2590846174 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7f43200d1ba0 con 0x558235350460 2024-04-24T08:54:33.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.160+0000 7f4340402740 10 client.16737 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T08:54:33.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.160+0000 7f4340402740 10 client.16737 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-24T08:54:33.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.160+0000 7f4340402740 20 client.16737 populate_metadata read hostname 'smithi083' 2024-04-24T08:54:33.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.161+0000 7f4340402740 1 --2- 172.21.15.83:0/2590846174 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x5582353d1e30 0x5582353d4220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:33.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.161+0000 7f4340402740 4 client.16737 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T08:54:33.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.161+0000 7f4340402740 1 -- 172.21.15.83:0/2590846174 --> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5582352bca60 con 0x5582353d1e30 2024-04-24T08:54:33.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.161+0000 7f4336a00640 1 --2- 172.21.15.83:0/2590846174 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x5582353d1e30 0x5582353d4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:33.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.162+0000 7f4336a00640 1 --2- 172.21.15.83:0/2590846174 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x5582353d1e30 0x5582353d4220 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bc673630469349f7 server_cookie=17485bd141e71898 in_seq=0 out_seq=0 2024-04-24T08:54:33.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.162+0000 7f431fe00640 10 client.16737 ms_handle_connect on v2:172.21.15.138:6834/1909219758 2024-04-24T08:54:33.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.163+0000 7f431fe00640 1 -- 172.21.15.83:0/2590846174 <== mds.2 v2:172.21.15.138:6834/1909219758 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5582352bca60 con 0x5582353d1e30 2024-04-24T08:54:33.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.163+0000 7f431fe00640 10 client.16737 handle_command_reply: tid=0 2024-04-24T08:54:33.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.163+0000 7f4340402740 1 client.16737 shutdown 2024-04-24T08:54:33.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.163+0000 7f4340402740 1 -- 172.21.15.83:0/2590846174 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558235350460 msgr2=0x55823535c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:33.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.163+0000 7f4340402740 1 --2- 172.21.15.83:0/2590846174 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558235350460 0x55823535c150 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f43200eeeb0 tx=0x7f43200c5c90 comp rx=0 tx=0).stop 2024-04-24T08:54:33.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.163+0000 7f4340402740 1 -- 172.21.15.83:0/2590846174 shutdown_connections 2024-04-24T08:54:33.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.163+0000 7f4340402740 1 --2- 172.21.15.83:0/2590846174 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x5582353d1e30 0x5582353d4220 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:33.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.163+0000 7f4340402740 1 --2- 172.21.15.83:0/2590846174 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55823535ab30 0x5582353612b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:33.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.163+0000 7f4340402740 1 --2- 172.21.15.83:0/2590846174 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558235350460 0x55823535c150 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:33.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.163+0000 7f431fe00640 0 client.16737 ms_handle_reset on v2:172.21.15.138:6834/1909219758 2024-04-24T08:54:33.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.163+0000 7f4340402740 1 --2- 172.21.15.83:0/2590846174 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55823534fb40 0x55823535bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:33.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.163+0000 7f4340402740 1 -- 172.21.15.83:0/2590846174 >> 172.21.15.83:0/2590846174 conn(0x55823534be60 msgr2=0x55823534a6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:33.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.163+0000 7f4340402740 1 -- 172.21.15.83:0/2590846174 shutdown_connections 2024-04-24T08:54:33.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.163+0000 7f4340402740 1 -- 172.21.15.83:0/2590846174 wait complete. 2024-04-24T08:54:33.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.163+0000 7f4340402740 20 client.16737 trim_cache size 0 max 16384 2024-04-24T08:54:33.165 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T08:54:33.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.164+0000 7f4337e00640 1 -- 172.21.15.83:0/2869298610 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f43180a3d20 msgr2=0x7f43180a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:33.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.164+0000 7f4337e00640 1 --2- 172.21.15.83:0/2869298610 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f43180a3d20 0x7f43180a61e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f433806b560 tx=0x7f433000c330 comp rx=0 tx=0).stop 2024-04-24T08:54:33.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.164+0000 7f4337e00640 1 -- 172.21.15.83:0/2869298610 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4338074cf0 msgr2=0x7f433810b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:33.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.164+0000 7f4337e00640 1 --2- 172.21.15.83:0/2869298610 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4338074cf0 0x7f433810b200 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f4320005ab0 tx=0x7f4320003cd0 comp rx=0 tx=0).stop 2024-04-24T08:54:33.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.164+0000 7f4337e00640 1 -- 172.21.15.83:0/2869298610 shutdown_connections 2024-04-24T08:54:33.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.164+0000 7f4337e00640 1 --2- 172.21.15.83:0/2869298610 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f43180a3d20 0x7f43180a61e0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:33.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.164+0000 7f4337e00640 1 --2- 172.21.15.83:0/2869298610 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f433810e740 0x7f433810b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:33.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.164+0000 7f4337e00640 1 --2- 172.21.15.83:0/2869298610 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4338074cf0 0x7f433810b200 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:33.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.164+0000 7f4337e00640 1 --2- 172.21.15.83:0/2869298610 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f43380743d0 0x7f433810e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:33.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.164+0000 7f4337e00640 1 -- 172.21.15.83:0/2869298610 >> 172.21.15.83:0/2869298610 conn(0x7f43380fd9d0 msgr2=0x7f43380fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:33.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.164+0000 7f4337e00640 1 -- 172.21.15.83:0/2869298610 shutdown_connections 2024-04-24T08:54:33.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.165+0000 7f4337e00640 1 -- 172.21.15.83:0/2869298610 wait complete. 2024-04-24T08:54:33.182 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T08:54:33.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:54:32 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/329192880' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:54:33.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:54:32 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/329192880' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:54:33.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.378+0000 7fdfc2600640 1 -- 172.21.15.83:0/3342623201 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdfbc107300 msgr2=0x7fdfbc10eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:33.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.378+0000 7fdfc2600640 1 --2- 172.21.15.83:0/3342623201 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdfbc107300 0x7fdfbc10eed0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fdfb4009ca0 tx=0x7fdfb402a8c0 comp rx=0 tx=0).stop 2024-04-24T08:54:33.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.378+0000 7fdfc2600640 1 -- 172.21.15.83:0/3342623201 shutdown_connections 2024-04-24T08:54:33.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.378+0000 7fdfc2600640 1 --2- 172.21.15.83:0/3342623201 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdfbc107300 0x7fdfbc10eed0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:33.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.378+0000 7fdfc2600640 1 --2- 172.21.15.83:0/3342623201 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdfbc102be0 0x7fdfbc106c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:33.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.378+0000 7fdfc2600640 1 --2- 172.21.15.83:0/3342623201 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdfbc102230 0x7fdfbc102610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:33.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.378+0000 7fdfc2600640 1 -- 172.21.15.83:0/3342623201 >> 172.21.15.83:0/3342623201 conn(0x7fdfbc0fda70 msgr2=0x7fdfbc0ffe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:33.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.378+0000 7fdfc2600640 1 -- 172.21.15.83:0/3342623201 shutdown_connections 2024-04-24T08:54:33.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.378+0000 7fdfc2600640 1 -- 172.21.15.83:0/3342623201 wait complete. 2024-04-24T08:54:33.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.379+0000 7fdfc2600640 1 Processor -- start 2024-04-24T08:54:33.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.380+0000 7fdfc2600640 1 -- start start 2024-04-24T08:54:33.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.380+0000 7fdfc2600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdfbc102230 0x7fdfbc074300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:33.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.380+0000 7fdfc2600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdfbc107300 0x7fdfbc074840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:33.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.380+0000 7fdfc2600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdfbc074d80 0x7fdfbc1a6a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:33.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.380+0000 7fdfc2600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdfbc1122c0 con 0x7fdfbc074d80 2024-04-24T08:54:33.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.380+0000 7fdfc2600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdfbc112140 con 0x7fdfbc107300 2024-04-24T08:54:33.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.380+0000 7fdfc2600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdfbc112440 con 0x7fdfbc102230 2024-04-24T08:54:33.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.380+0000 7fdfbbe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdfbc074d80 0x7fdfbc1a6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:33.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.380+0000 7fdfbbe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdfbc074d80 0x7fdfbc1a6a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41728/0 (socket says 172.21.15.83:41728) 2024-04-24T08:54:33.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.380+0000 7fdfbbe00640 1 -- 172.21.15.83:0/3224194043 learned_addr learned my addr 172.21.15.83:0/3224194043 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:54:33.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.381+0000 7fdfbb400640 1 --2- 172.21.15.83:0/3224194043 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdfbc102230 0x7fdfbc074300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:33.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.381+0000 7fdfbaa00640 1 --2- 172.21.15.83:0/3224194043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdfbc107300 0x7fdfbc074840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:33.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.381+0000 7fdfbbe00640 1 -- 172.21.15.83:0/3224194043 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdfbc102230 msgr2=0x7fdfbc074300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:33.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.381+0000 7fdfbbe00640 1 --2- 172.21.15.83:0/3224194043 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdfbc102230 0x7fdfbc074300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:33.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.381+0000 7fdfbbe00640 1 -- 172.21.15.83:0/3224194043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdfbc107300 msgr2=0x7fdfbc074840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:33.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.381+0000 7fdfbbe00640 1 --2- 172.21.15.83:0/3224194043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdfbc107300 0x7fdfbc074840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:33.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.381+0000 7fdfbbe00640 1 -- 172.21.15.83:0/3224194043 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfbc1a7170 con 0x7fdfbc074d80 2024-04-24T08:54:33.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.381+0000 7fdfbbe00640 1 --2- 172.21.15.83:0/3224194043 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdfbc074d80 0x7fdfbc1a6a70 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fdfb4005e30 tx=0x7fdfb40093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:33.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.381+0000 7fdfb3e00640 1 -- 172.21.15.83:0/3224194043 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfb4003c70 con 0x7fdfbc074d80 2024-04-24T08:54:33.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.381+0000 7fdfb3e00640 1 -- 172.21.15.83:0/3224194043 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdfb4003e10 con 0x7fdfbc074d80 2024-04-24T08:54:33.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.381+0000 7fdfc2600640 1 -- 172.21.15.83:0/3224194043 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfbc1a7400 con 0x7fdfbc074d80 2024-04-24T08:54:33.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.382+0000 7fdfb3e00640 1 -- 172.21.15.83:0/3224194043 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfb4030c10 con 0x7fdfbc074d80 2024-04-24T08:54:33.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.382+0000 7fdfc2600640 1 -- 172.21.15.83:0/3224194043 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfbc1a7950 con 0x7fdfbc074d80 2024-04-24T08:54:33.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.386+0000 7fdfc55e9740 1 Processor -- start 2024-04-24T08:54:33.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.386+0000 7fdfb3e00640 1 -- 172.21.15.83:0/3224194043 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fdfb4004040 con 0x7fdfbc074d80 2024-04-24T08:54:33.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.386+0000 7fdfc55e9740 1 -- start start 2024-04-24T08:54:33.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.387+0000 7fdfc55e9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561023e9c950 0x561023e9cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:33.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.387+0000 7fdfc55e9740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561023e9d270 0x561023ea7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:33.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.387+0000 7fdfc55e9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561023ea7940 0x561023ea9d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:33.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.387+0000 7fdfc55e9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561023e021d0 con 0x561023ea7940 2024-04-24T08:54:33.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.387+0000 7fdfc55e9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x561023da8450 con 0x561023e9c950 2024-04-24T08:54:33.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.387+0000 7fdfc55e9740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x561023dfe190 con 0x561023e9d270 2024-04-24T08:54:33.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.387+0000 7fdfbaa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561023ea7940 0x561023ea9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:33.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.387+0000 7fdfbaa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561023ea7940 0x561023ea9d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41742/0 (socket says 172.21.15.83:41742) 2024-04-24T08:54:33.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.387+0000 7fdfbaa00640 1 -- 172.21.15.83:0/3309842253 learned_addr learned my addr 172.21.15.83:0/3309842253 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:54:33.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.388+0000 7fdfbaa00640 1 --2- 172.21.15.83:0/3309842253 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561023e9c950 0x561023e9cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:33.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.388+0000 7fdfbb400640 1 --2- 172.21.15.83:0/3309842253 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561023e9d270 0x561023ea7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:33.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.388+0000 7fdfbaa00640 1 -- 172.21.15.83:0/3309842253 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561023e9d270 msgr2=0x561023ea7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:33.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.388+0000 7fdfbaa00640 1 --2- 172.21.15.83:0/3309842253 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561023e9d270 0x561023ea7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:33.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.388+0000 7fdfbaa00640 1 -- 172.21.15.83:0/3309842253 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561023e9c950 msgr2=0x561023e9cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:33.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.388+0000 7fdfbaa00640 1 --2- 172.21.15.83:0/3309842253 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561023e9c950 0x561023e9cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:33.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.388+0000 7fdfbaa00640 1 -- 172.21.15.83:0/3309842253 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561023e65310 con 0x561023ea7940 2024-04-24T08:54:33.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.388+0000 7fdfb3e00640 1 --2- 172.21.15.83:0/3224194043 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fdfa00a3d50 0x7fdfa00a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:33.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.388+0000 7fdfbaa00640 1 --2- 172.21.15.83:0/3309842253 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561023ea7940 0x561023ea9d30 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fdfa80124d0 tx=0x7fdfa8009710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=755082a51c21f346 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:33.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.388+0000 7fdfb3e00640 1 -- 172.21.15.83:0/3224194043 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7fdfb40f1fb0 con 0x7fdfbc074d80 2024-04-24T08:54:33.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.388+0000 7fdfb0c00640 1 -- 172.21.15.83:0/3309842253 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa8004300 con 0x561023ea7940 2024-04-24T08:54:33.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.389+0000 7fdfbb400640 1 --2- 172.21.15.83:0/3224194043 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fdfa00a3d50 0x7fdfa00a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:33.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.389+0000 7fdfb0c00640 1 -- 172.21.15.83:0/3309842253 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdfa80044a0 con 0x561023ea7940 2024-04-24T08:54:33.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.389+0000 7fdfb0c00640 1 -- 172.21.15.83:0/3309842253 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa800fa10 con 0x561023ea7940 2024-04-24T08:54:33.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.389+0000 7fdfc55e9740 1 -- 172.21.15.83:0/3309842253 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561023ea7940 msgr2=0x561023ea9d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:33.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.389+0000 7fdfc55e9740 1 --2- 172.21.15.83:0/3309842253 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561023ea7940 0x561023ea9d30 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fdfa80124d0 tx=0x7fdfa8009710 comp rx=0 tx=0).stop 2024-04-24T08:54:33.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.389+0000 7fdfbb400640 1 --2- 172.21.15.83:0/3224194043 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fdfa00a3d50 0x7fdfa00a6210 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fdfac005fd0 tx=0x7fdfac00b650 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:33.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.390+0000 7fdfc55e9740 1 -- 172.21.15.83:0/3309842253 shutdown_connections 2024-04-24T08:54:33.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.390+0000 7fdfc55e9740 1 --2- 172.21.15.83:0/3309842253 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561023ea7940 0x561023ea9d30 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:33.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.390+0000 7fdfc55e9740 1 --2- 172.21.15.83:0/3309842253 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561023e9d270 0x561023ea7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:33.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.390+0000 7fdfc55e9740 1 --2- 172.21.15.83:0/3309842253 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561023e9c950 0x561023e9cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:33.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.390+0000 7fdfc55e9740 1 -- 172.21.15.83:0/3309842253 >> 172.21.15.83:0/3309842253 conn(0x561023e98c70 msgr2=0x561023e99050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:33.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.390+0000 7fdfc55e9740 1 -- 172.21.15.83:0/3309842253 shutdown_connections 2024-04-24T08:54:33.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.390+0000 7fdfc55e9740 1 -- 172.21.15.83:0/3309842253 wait complete. 2024-04-24T08:54:33.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.390+0000 7fdfc55e9740 1 -- start start 2024-04-24T08:54:33.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.391+0000 7fdfc55e9740 1 Processor -- start 2024-04-24T08:54:33.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.391+0000 7fdfc55e9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561023e9c950 0x561023ea4720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:33.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.391+0000 7fdfc55e9740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561023e9d270 0x561023ea4c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:33.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.391+0000 7fdfc55e9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561023ea7940 0x561023ea51a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:33.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.391+0000 7fdfc55e9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561023eaa7d0 con 0x561023e9c950 2024-04-24T08:54:33.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.391+0000 7fdfc55e9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x561023dbb810 con 0x561023ea7940 2024-04-24T08:54:33.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.391+0000 7fdfc55e9740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x561023eaa950 con 0x561023e9d270 2024-04-24T08:54:33.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.391+0000 7fdfc55e9740 10 client.0 _ll_register_callbacks cb 0x561023d89140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:54:33.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.391+0000 7fdfbb400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561023e9c950 0x561023ea4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:33.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.391+0000 7fdfbb400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561023e9c950 0x561023ea4720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41748/0 (socket says 172.21.15.83:41748) 2024-04-24T08:54:33.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.391+0000 7fdfbb400640 1 -- 172.21.15.83:0/3085389535 learned_addr learned my addr 172.21.15.83:0/3085389535 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:54:33.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.392+0000 7fdfbaa00640 1 --2- 172.21.15.83:0/3085389535 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561023e9d270 0x561023ea4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:33.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.392+0000 7fdfbbe00640 1 --2- 172.21.15.83:0/3085389535 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561023ea7940 0x561023ea51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:33.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.392+0000 7fdfbb400640 1 -- 172.21.15.83:0/3085389535 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561023e9d270 msgr2=0x561023ea4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:33.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.392+0000 7fdfbb400640 1 --2- 172.21.15.83:0/3085389535 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561023e9d270 0x561023ea4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:33.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.392+0000 7fdfbb400640 1 -- 172.21.15.83:0/3085389535 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561023ea7940 msgr2=0x561023ea51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:33.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.392+0000 7fdfbb400640 1 --2- 172.21.15.83:0/3085389535 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561023ea7940 0x561023ea51a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:33.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.392+0000 7fdfbb400640 1 -- 172.21.15.83:0/3085389535 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561023dcc3f0 con 0x561023e9c950 2024-04-24T08:54:33.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.392+0000 7fdfbb400640 1 --2- 172.21.15.83:0/3085389535 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561023e9c950 0x561023ea4720 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fdfac0126d0 tx=0x7fdfac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=245fe613967524bf server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:33.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.392+0000 7fdfa5600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:54:33.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.393+0000 7fdfa5600640 1 -- 172.21.15.83:0/3085389535 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfac005220 con 0x561023e9c950 2024-04-24T08:54:33.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.393+0000 7fdfa5600640 1 -- 172.21.15.83:0/3085389535 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7fdfac0053c0 con 0x561023e9c950 2024-04-24T08:54:33.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.393+0000 7fdfc55e9740 1 -- 172.21.15.83:0/3085389535 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdfa80044a0 con 0x561023e9c950 2024-04-24T08:54:33.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.393+0000 7fdfa5600640 1 -- 172.21.15.83:0/3085389535 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fdfac02f050 con 0x561023e9c950 2024-04-24T08:54:33.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.393+0000 7fdfc55e9740 10 client.16749 fetch_fsmap learned FSMap version 30 2024-04-24T08:54:33.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.393+0000 7fdfc55e9740 1 -- 172.21.15.83:0/3085389535 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561023b143f0 con 0x561023e9c950 2024-04-24T08:54:33.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.393+0000 7fdfa5600640 1 -- 172.21.15.83:0/3085389535 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdfa80044a0 con 0x561023e9c950 2024-04-24T08:54:33.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.393+0000 7fdfa5600640 1 -- 172.21.15.83:0/3085389535 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7fdfac00fa70 con 0x561023e9c950 2024-04-24T08:54:33.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.394+0000 7fdfc55e9740 10 client.16749 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T08:54:33.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.394+0000 7fdfc55e9740 10 client.16749 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-24T08:54:33.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.394+0000 7fdfc55e9740 20 client.16749 populate_metadata read hostname 'smithi083' 2024-04-24T08:54:33.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.394+0000 7fdfc55e9740 1 --2- 172.21.15.83:0/3085389535 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x561023eb7cb0 0x561023eba110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:33.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.394+0000 7fdfc55e9740 4 client.16749 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:54:33.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.394+0000 7fdfc55e9740 1 -- 172.21.15.83:0/3085389535 --> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561023e0fbf0 con 0x561023eb7cb0 2024-04-24T08:54:33.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.394+0000 7fdfbaa00640 1 --2- 172.21.15.83:0/3085389535 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x561023eb7cb0 0x561023eba110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:33.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.395+0000 7fdfbaa00640 1 --2- 172.21.15.83:0/3085389535 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x561023eb7cb0 0x561023eba110 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d7688378bf80596c server_cookie=299cf5b05918c2db in_seq=0 out_seq=0 2024-04-24T08:54:33.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.395+0000 7fdfa5600640 10 client.16749 ms_handle_connect on v2:172.21.15.138:6836/900566754 2024-04-24T08:54:33.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.397+0000 7fdfa5600640 1 -- 172.21.15.83:0/3085389535 <== mds.1 v2:172.21.15.138:6836/900566754 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561023e0fbf0 con 0x561023eb7cb0 2024-04-24T08:54:33.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.397+0000 7fdfa5600640 10 client.16749 handle_command_reply: tid=0 2024-04-24T08:54:33.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.397+0000 7fdfc55e9740 1 client.16749 shutdown 2024-04-24T08:54:33.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.397+0000 7fdfc55e9740 1 -- 172.21.15.83:0/3085389535 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561023e9c950 msgr2=0x561023ea4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:33.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.397+0000 7fdfc55e9740 1 --2- 172.21.15.83:0/3085389535 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561023e9c950 0x561023ea4720 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fdfac0126d0 tx=0x7fdfac0046c0 comp rx=0 tx=0).stop 2024-04-24T08:54:33.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.397+0000 7fdfc55e9740 1 -- 172.21.15.83:0/3085389535 shutdown_connections 2024-04-24T08:54:33.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.397+0000 7fdfc55e9740 1 --2- 172.21.15.83:0/3085389535 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561023ea7940 0x561023ea51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:33.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.397+0000 7fdfc55e9740 1 --2- 172.21.15.83:0/3085389535 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x561023eb7cb0 0x561023eba110 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:33.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.397+0000 7fdfc55e9740 1 --2- 172.21.15.83:0/3085389535 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561023e9d270 0x561023ea4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:33.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.397+0000 7fdfc55e9740 1 --2- 172.21.15.83:0/3085389535 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561023e9c950 0x561023ea4720 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:33.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.397+0000 7fdfc55e9740 1 -- 172.21.15.83:0/3085389535 >> 172.21.15.83:0/3085389535 conn(0x561023e98c70 msgr2=0x561023e9ea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:33.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.397+0000 7fdfa5600640 0 client.16749 ms_handle_reset on v2:172.21.15.138:6836/900566754 2024-04-24T08:54:33.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.397+0000 7fdfc55e9740 1 -- 172.21.15.83:0/3085389535 shutdown_connections 2024-04-24T08:54:33.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.398+0000 7fdfc55e9740 1 -- 172.21.15.83:0/3085389535 wait complete. 2024-04-24T08:54:33.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.398+0000 7fdfc55e9740 20 client.16749 trim_cache size 0 max 16384 2024-04-24T08:54:33.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.414+0000 7fdfc55e9740 1 Processor -- start 2024-04-24T08:54:33.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.414+0000 7fdfc55e9740 1 -- start start 2024-04-24T08:54:33.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.414+0000 7fdfc55e9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561023e9c950 0x561023eb3690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:33.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.414+0000 7fdfc55e9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561023e9d270 0x561023ea2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:33.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.414+0000 7fdfc55e9740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561023ea7940 0x561023e98570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:33.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.414+0000 7fdfc55e9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdfbc1b16e0 con 0x561023e9c950 2024-04-24T08:54:33.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.414+0000 7fdfc55e9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdfbc1b32b0 con 0x561023e9d270 2024-04-24T08:54:33.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.414+0000 7fdfc55e9740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdfbc1b1db0 con 0x561023ea7940 2024-04-24T08:54:33.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.415+0000 7fdfbb400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561023e9c950 0x561023eb3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:33.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.415+0000 7fdfbb400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561023e9c950 0x561023eb3690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41756/0 (socket says 172.21.15.83:41756) 2024-04-24T08:54:33.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.415+0000 7fdfbb400640 1 -- 172.21.15.83:0/862475594 learned_addr learned my addr 172.21.15.83:0/862475594 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:54:33.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.415+0000 7fdfbaa00640 1 --2- 172.21.15.83:0/862475594 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561023e9d270 0x561023ea2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:33.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.415+0000 7fdfbb400640 1 -- 172.21.15.83:0/862475594 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561023ea7940 msgr2=0x561023e98570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:33.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.415+0000 7fdfbb400640 1 --2- 172.21.15.83:0/862475594 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561023ea7940 0x561023e98570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:33.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.415+0000 7fdfbb400640 1 -- 172.21.15.83:0/862475594 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561023e9d270 msgr2=0x561023ea2ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:33.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.415+0000 7fdfbb400640 1 --2- 172.21.15.83:0/862475594 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561023e9d270 0x561023ea2ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:33.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.415+0000 7fdfbb400640 1 -- 172.21.15.83:0/862475594 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561023e9d0a0 con 0x561023e9c950 2024-04-24T08:54:33.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.415+0000 7fdfbb400640 1 --2- 172.21.15.83:0/862475594 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561023e9c950 0x561023eb3690 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fdfac00b590 tx=0x7fdfac01f7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f4b21ffbe1f9ed1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:33.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.416+0000 7fdfa7e00640 1 -- 172.21.15.83:0/862475594 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfac00f9a0 con 0x561023e9c950 2024-04-24T08:54:33.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.416+0000 7fdfa7e00640 1 -- 172.21.15.83:0/862475594 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdfac02c070 con 0x561023e9c950 2024-04-24T08:54:33.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.416+0000 7fdfa7e00640 1 -- 172.21.15.83:0/862475594 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfac005450 con 0x561023e9c950 2024-04-24T08:54:33.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.417+0000 7fdfc55e9740 1 -- 172.21.15.83:0/862475594 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561023e9c950 msgr2=0x561023eb3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:33.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.417+0000 7fdfc55e9740 1 --2- 172.21.15.83:0/862475594 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561023e9c950 0x561023eb3690 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fdfac00b590 tx=0x7fdfac01f7b0 comp rx=0 tx=0).stop 2024-04-24T08:54:33.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.417+0000 7fdfc55e9740 1 -- 172.21.15.83:0/862475594 shutdown_connections 2024-04-24T08:54:33.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.417+0000 7fdfc55e9740 1 --2- 172.21.15.83:0/862475594 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561023ea7940 0x561023e98570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:33.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.417+0000 7fdfc55e9740 1 --2- 172.21.15.83:0/862475594 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561023e9d270 0x561023ea2ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:33.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.417+0000 7fdfc55e9740 1 --2- 172.21.15.83:0/862475594 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561023e9c950 0x561023eb3690 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:33.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.417+0000 7fdfc55e9740 1 -- 172.21.15.83:0/862475594 >> 172.21.15.83:0/862475594 conn(0x561023e98c70 msgr2=0x561023ea4f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:33.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.417+0000 7fdfc55e9740 1 -- 172.21.15.83:0/862475594 shutdown_connections 2024-04-24T08:54:33.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.417+0000 7fdfc55e9740 1 -- 172.21.15.83:0/862475594 wait complete. 2024-04-24T08:54:33.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.417+0000 7fdfc55e9740 1 -- start start 2024-04-24T08:54:33.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.417+0000 7fdfc55e9740 1 Processor -- start 2024-04-24T08:54:33.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.418+0000 7fdfc55e9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561023e9c950 0x561023ea86e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:33.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.418+0000 7fdfc55e9740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561023e9d270 0x561023ea8c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:33.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.418+0000 7fdfc55e9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561023ea7940 0x561023ea9160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:33.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.418+0000 7fdfc55e9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdfbc1b2b50 con 0x561023ea7940 2024-04-24T08:54:33.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.418+0000 7fdfc55e9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdfbc1b2480 con 0x561023e9c950 2024-04-24T08:54:33.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.418+0000 7fdfc55e9740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdfbc1af6c0 con 0x561023e9d270 2024-04-24T08:54:33.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.418+0000 7fdfc55e9740 10 client.0 _ll_register_callbacks cb 0x7fdfac00bef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:54:33.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.418+0000 7fdfbbe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561023ea7940 0x561023ea9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:33.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.418+0000 7fdfbbe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561023ea7940 0x561023ea9160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41758/0 (socket says 172.21.15.83:41758) 2024-04-24T08:54:33.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.418+0000 7fdfbbe00640 1 -- 172.21.15.83:0/1194195770 learned_addr learned my addr 172.21.15.83:0/1194195770 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:54:33.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.418+0000 7fdfbaa00640 1 --2- 172.21.15.83:0/1194195770 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561023e9d270 0x561023ea8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:33.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.419+0000 7fdfbb400640 1 --2- 172.21.15.83:0/1194195770 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561023e9c950 0x561023ea86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:33.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.419+0000 7fdfbbe00640 1 -- 172.21.15.83:0/1194195770 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561023e9d270 msgr2=0x561023ea8c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:33.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.419+0000 7fdfbbe00640 1 --2- 172.21.15.83:0/1194195770 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561023e9d270 0x561023ea8c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:33.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.419+0000 7fdfbbe00640 1 -- 172.21.15.83:0/1194195770 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561023e9c950 msgr2=0x561023ea86e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:33.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.419+0000 7fdfbbe00640 1 --2- 172.21.15.83:0/1194195770 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561023e9c950 0x561023ea86e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:33.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.419+0000 7fdfbbe00640 1 -- 172.21.15.83:0/1194195770 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561023bde7d0 con 0x561023ea7940 2024-04-24T08:54:33.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.419+0000 7fdfbbe00640 1 --2- 172.21.15.83:0/1194195770 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561023ea7940 0x561023ea9160 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fdfb40bdbc0 tx=0x7fdfb4030e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a75477029753670d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:33.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.419+0000 7fdfa5600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:54:33.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.419+0000 7fdfa5600640 1 -- 172.21.15.83:0/1194195770 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfb40a8040 con 0x561023ea7940 2024-04-24T08:54:33.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.419+0000 7fdfa5600640 1 -- 172.21.15.83:0/1194195770 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7fdfb40ba2a0 con 0x561023ea7940 2024-04-24T08:54:33.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.419+0000 7fdfc55e9740 1 -- 172.21.15.83:0/1194195770 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdfac02c070 con 0x561023ea7940 2024-04-24T08:54:33.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.419+0000 7fdfa5600640 1 -- 172.21.15.83:0/1194195770 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fdfb40c1fe0 con 0x561023ea7940 2024-04-24T08:54:33.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.419+0000 7fdfc55e9740 10 client.16755 fetch_fsmap learned FSMap version 30 2024-04-24T08:54:33.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.419+0000 7fdfc55e9740 1 -- 172.21.15.83:0/1194195770 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561023f26bd0 con 0x561023ea7940 2024-04-24T08:54:33.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.420+0000 7fdfa5600640 1 -- 172.21.15.83:0/1194195770 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdfac02c070 con 0x561023ea7940 2024-04-24T08:54:33.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.420+0000 7fdfa5600640 1 -- 172.21.15.83:0/1194195770 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7fdfb40bde90 con 0x561023ea7940 2024-04-24T08:54:33.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.420+0000 7fdfc55e9740 10 client.16755 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T08:54:33.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.420+0000 7fdfc55e9740 10 client.16755 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-24T08:54:33.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.420+0000 7fdfc55e9740 20 client.16755 populate_metadata read hostname 'smithi083' 2024-04-24T08:54:33.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.420+0000 7fdfc55e9740 1 --2- 172.21.15.83:0/1194195770 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x561023f27320 0x561023f29740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:33.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.420+0000 7fdfc55e9740 4 client.16755 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T08:54:33.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.420+0000 7fdfc55e9740 1 -- 172.21.15.83:0/1194195770 --> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561023e09a60 con 0x561023f27320 2024-04-24T08:54:33.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.420+0000 7fdfbb400640 1 --2- 172.21.15.83:0/1194195770 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x561023f27320 0x561023f29740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:33.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.421+0000 7fdfbb400640 1 --2- 172.21.15.83:0/1194195770 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x561023f27320 0x561023f29740 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=18204f0781c8092 server_cookie=7662d7234cab1c71 in_seq=0 out_seq=0 2024-04-24T08:54:33.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.421+0000 7fdfa5600640 10 client.16755 ms_handle_connect on v2:172.21.15.138:6836/900566754 2024-04-24T08:54:33.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.422+0000 7fdfa5600640 1 -- 172.21.15.83:0/1194195770 <== mds.1 v2:172.21.15.138:6836/900566754 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561023e09a60 con 0x561023f27320 2024-04-24T08:54:33.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.422+0000 7fdfa5600640 10 client.16755 handle_command_reply: tid=0 2024-04-24T08:54:33.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.422+0000 7fdfc55e9740 1 client.16755 shutdown 2024-04-24T08:54:33.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.422+0000 7fdfc55e9740 1 -- 172.21.15.83:0/1194195770 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561023ea7940 msgr2=0x561023ea9160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:33.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.422+0000 7fdfc55e9740 1 --2- 172.21.15.83:0/1194195770 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561023ea7940 0x561023ea9160 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fdfb40bdbc0 tx=0x7fdfb4030e70 comp rx=0 tx=0).stop 2024-04-24T08:54:33.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.422+0000 7fdfc55e9740 1 -- 172.21.15.83:0/1194195770 shutdown_connections 2024-04-24T08:54:33.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.422+0000 7fdfc55e9740 1 --2- 172.21.15.83:0/1194195770 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561023ea7940 0x561023ea9160 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:33.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.422+0000 7fdfc55e9740 1 --2- 172.21.15.83:0/1194195770 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x561023f27320 0x561023f29740 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:33.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.422+0000 7fdfc55e9740 1 --2- 172.21.15.83:0/1194195770 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561023e9d270 0x561023ea8c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:33.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.422+0000 7fdfc55e9740 1 --2- 172.21.15.83:0/1194195770 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561023e9c950 0x561023ea86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:33.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.422+0000 7fdfa5600640 0 client.16755 ms_handle_reset on v2:172.21.15.138:6836/900566754 2024-04-24T08:54:33.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.422+0000 7fdfc55e9740 1 -- 172.21.15.83:0/1194195770 >> 172.21.15.83:0/1194195770 conn(0x561023e98c70 msgr2=0x561023e9e180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:33.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.422+0000 7fdfc55e9740 1 -- 172.21.15.83:0/1194195770 shutdown_connections 2024-04-24T08:54:33.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.422+0000 7fdfc55e9740 1 -- 172.21.15.83:0/1194195770 wait complete. 2024-04-24T08:54:33.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.423+0000 7fdfc55e9740 20 client.16755 trim_cache size 0 max 16384 2024-04-24T08:54:33.426 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T08:54:33.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.423+0000 7fdfc2600640 1 -- 172.21.15.83:0/3224194043 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fdfa00a3d50 msgr2=0x7fdfa00a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:33.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.423+0000 7fdfc2600640 1 --2- 172.21.15.83:0/3224194043 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fdfa00a3d50 0x7fdfa00a6210 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fdfac005fd0 tx=0x7fdfac00b650 comp rx=0 tx=0).stop 2024-04-24T08:54:33.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.423+0000 7fdfc2600640 1 -- 172.21.15.83:0/3224194043 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdfbc074d80 msgr2=0x7fdfbc1a6a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:33.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.423+0000 7fdfc2600640 1 --2- 172.21.15.83:0/3224194043 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdfbc074d80 0x7fdfbc1a6a70 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fdfb4005e30 tx=0x7fdfb40093d0 comp rx=0 tx=0).stop 2024-04-24T08:54:33.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.423+0000 7fdfc2600640 1 -- 172.21.15.83:0/3224194043 shutdown_connections 2024-04-24T08:54:33.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.423+0000 7fdfc2600640 1 --2- 172.21.15.83:0/3224194043 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fdfa00a3d50 0x7fdfa00a6210 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:33.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.423+0000 7fdfc2600640 1 --2- 172.21.15.83:0/3224194043 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdfbc074d80 0x7fdfbc1a6a70 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:33.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.423+0000 7fdfc2600640 1 --2- 172.21.15.83:0/3224194043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdfbc107300 0x7fdfbc074840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:33.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.423+0000 7fdfc2600640 1 --2- 172.21.15.83:0/3224194043 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdfbc102230 0x7fdfbc074300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:33.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.423+0000 7fdfc2600640 1 -- 172.21.15.83:0/3224194043 >> 172.21.15.83:0/3224194043 conn(0x7fdfbc0fda70 msgr2=0x7fdfbc0ffe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:33.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.423+0000 7fdfc2600640 1 -- 172.21.15.83:0/3224194043 shutdown_connections 2024-04-24T08:54:33.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.423+0000 7fdfc2600640 1 -- 172.21.15.83:0/3224194043 wait complete. 2024-04-24T08:54:33.440 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T08:54:33.561 INFO:tasks.workunit.client.0.smithi083.stdout: 1 8573 0.96 MB/sec warmup 67 sec latency 51.135 ms 2024-04-24T08:54:33.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.655+0000 7f0991a00640 1 -- 172.21.15.83:0/2756335096 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f098c1044b0 msgr2=0x7f098c104890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:33.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.655+0000 7f0991a00640 1 --2- 172.21.15.83:0/2756335096 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f098c1044b0 0x7f098c104890 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f097c009900 tx=0x7f097c02a410 comp rx=0 tx=0).stop 2024-04-24T08:54:33.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.655+0000 7f0991a00640 1 -- 172.21.15.83:0/2756335096 shutdown_connections 2024-04-24T08:54:33.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.655+0000 7f0991a00640 1 --2- 172.21.15.83:0/2756335096 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f098c109580 0x7f098c111150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:33.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.655+0000 7f0991a00640 1 --2- 172.21.15.83:0/2756335096 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f098c104e60 0x7f098c108ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:33.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.655+0000 7f0991a00640 1 --2- 172.21.15.83:0/2756335096 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f098c1044b0 0x7f098c104890 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:33.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.655+0000 7f0991a00640 1 -- 172.21.15.83:0/2756335096 >> 172.21.15.83:0/2756335096 conn(0x7f098c0ffd90 msgr2=0x7f098c1021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:33.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.655+0000 7f0991a00640 1 -- 172.21.15.83:0/2756335096 shutdown_connections 2024-04-24T08:54:33.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.656+0000 7f0991a00640 1 -- 172.21.15.83:0/2756335096 wait complete. 2024-04-24T08:54:33.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.656+0000 7f0991a00640 1 Processor -- start 2024-04-24T08:54:33.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.657+0000 7f0991a00640 1 -- start start 2024-04-24T08:54:33.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.657+0000 7f0991a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f098c1044b0 0x7f098c1a07d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:33.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.657+0000 7f0991a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f098c109580 0x7f098c1a0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:33.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.657+0000 7f0991a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f098c19a9d0 0x7f098c19ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:33.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.657+0000 7f0991a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f098c1144f0 con 0x7f098c109580 2024-04-24T08:54:33.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.657+0000 7f0991a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f098c114370 con 0x7f098c19a9d0 2024-04-24T08:54:33.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.657+0000 7f0991a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f098c114670 con 0x7f098c1044b0 2024-04-24T08:54:33.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.657+0000 7f0983e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f098c109580 0x7f098c1a0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:33.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.657+0000 7f0983e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f098c109580 0x7f098c1a0d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41786/0 (socket says 172.21.15.83:41786) 2024-04-24T08:54:33.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.657+0000 7f0983e00640 1 -- 172.21.15.83:0/3772969663 learned_addr learned my addr 172.21.15.83:0/3772969663 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:54:33.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.658+0000 7f098aa00640 1 --2- 172.21.15.83:0/3772969663 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f098c1044b0 0x7f098c1a07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:33.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.658+0000 7f098b400640 1 --2- 172.21.15.83:0/3772969663 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f098c19a9d0 0x7f098c19ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:33.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.658+0000 7f0983e00640 1 -- 172.21.15.83:0/3772969663 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f098c1044b0 msgr2=0x7f098c1a07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:33.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.658+0000 7f0983e00640 1 --2- 172.21.15.83:0/3772969663 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f098c1044b0 0x7f098c1a07d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:33.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.658+0000 7f0983e00640 1 -- 172.21.15.83:0/3772969663 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f098c19a9d0 msgr2=0x7f098c19ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:33.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.658+0000 7f0983e00640 1 --2- 172.21.15.83:0/3772969663 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f098c19a9d0 0x7f098c19ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:33.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.658+0000 7f0983e00640 1 -- 172.21.15.83:0/3772969663 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f098c19b4f0 con 0x7f098c109580 2024-04-24T08:54:33.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.658+0000 7f0983e00640 1 --2- 172.21.15.83:0/3772969663 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f098c109580 0x7f098c1a0d10 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f097400c970 tx=0x7f097400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:33.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.659+0000 7f0983400640 1 -- 172.21.15.83:0/3772969663 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0974007bb0 con 0x7f098c109580 2024-04-24T08:54:33.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.659+0000 7f0983400640 1 -- 172.21.15.83:0/3772969663 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0974007d50 con 0x7f098c109580 2024-04-24T08:54:33.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.659+0000 7f0991a00640 1 -- 172.21.15.83:0/3772969663 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f098c19b7e0 con 0x7f098c109580 2024-04-24T08:54:33.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.659+0000 7f0991a00640 1 -- 172.21.15.83:0/3772969663 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f098c1a79e0 con 0x7f098c109580 2024-04-24T08:54:33.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.659+0000 7f0983400640 1 -- 172.21.15.83:0/3772969663 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f097400f450 con 0x7f098c109580 2024-04-24T08:54:33.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.664+0000 7f0983400640 1 -- 172.21.15.83:0/3772969663 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0974004f20 con 0x7f098c109580 2024-04-24T08:54:33.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.664+0000 7f099481f740 1 Processor -- start 2024-04-24T08:54:33.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.665+0000 7f099481f740 1 -- start start 2024-04-24T08:54:33.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.665+0000 7f099481f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec21dbe950 0x55ec21dbed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:33.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.665+0000 7f099481f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec21dbf270 0x55ec21dc9400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:33.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.665+0000 7f099481f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ec21dc9940 0x55ec21dcbd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:33.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.665+0000 7f099481f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ec21d241d0 con 0x55ec21dbf270 2024-04-24T08:54:33.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.665+0000 7f099481f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ec21cca450 con 0x55ec21dbe950 2024-04-24T08:54:33.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.665+0000 7f099481f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55ec21d20190 con 0x55ec21dc9940 2024-04-24T08:54:33.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.665+0000 7f098aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec21dbf270 0x55ec21dc9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:33.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.665+0000 7f098aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec21dbf270 0x55ec21dc9400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41796/0 (socket says 172.21.15.83:41796) 2024-04-24T08:54:33.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.665+0000 7f098aa00640 1 -- 172.21.15.83:0/3181094839 learned_addr learned my addr 172.21.15.83:0/3181094839 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:54:33.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.666+0000 7f098b400640 1 --2- 172.21.15.83:0/3181094839 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec21dbe950 0x55ec21dbed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:33.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.666+0000 7f0983e00640 1 --2- 172.21.15.83:0/3181094839 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ec21dc9940 0x55ec21dcbd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:33.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.666+0000 7f098aa00640 1 -- 172.21.15.83:0/3181094839 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ec21dc9940 msgr2=0x55ec21dcbd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:33.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.666+0000 7f098aa00640 1 --2- 172.21.15.83:0/3181094839 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ec21dc9940 0x55ec21dcbd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:33.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.666+0000 7f098aa00640 1 -- 172.21.15.83:0/3181094839 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec21dbe950 msgr2=0x55ec21dbed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:33.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.666+0000 7f098aa00640 1 --2- 172.21.15.83:0/3181094839 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec21dbe950 0x55ec21dbed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:33.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.666+0000 7f098aa00640 1 -- 172.21.15.83:0/3181094839 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec21d87310 con 0x55ec21dbf270 2024-04-24T08:54:33.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.666+0000 7f098b400640 1 --2- 172.21.15.83:0/3181094839 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec21dbe950 0x55ec21dbed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:54:33.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.666+0000 7f098aa00640 1 --2- 172.21.15.83:0/3181094839 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec21dbf270 0x55ec21dc9400 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f097c02a930 tx=0x7f097c032cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b8abcfd94d21055 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:33.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.667+0000 7f0983400640 1 --2- 172.21.15.83:0/3772969663 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f09700a3e70 0x7f09700a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:33.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.667+0000 7f097be00640 1 -- 172.21.15.83:0/3181094839 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f097c03f070 con 0x55ec21dbf270 2024-04-24T08:54:33.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.667+0000 7f097be00640 1 -- 172.21.15.83:0/3181094839 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f097c0316a0 con 0x55ec21dbf270 2024-04-24T08:54:33.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.667+0000 7f097be00640 1 -- 172.21.15.83:0/3181094839 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f097c035070 con 0x55ec21dbf270 2024-04-24T08:54:33.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.667+0000 7f0983400640 1 -- 172.21.15.83:0/3772969663 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f09740d0750 con 0x7f098c109580 2024-04-24T08:54:33.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.667+0000 7f098b400640 1 --2- 172.21.15.83:0/3772969663 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f09700a3e70 0x7f09700a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:33.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.668+0000 7f099481f740 1 -- 172.21.15.83:0/3181094839 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec21dbf270 msgr2=0x55ec21dc9400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:33.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.668+0000 7f099481f740 1 --2- 172.21.15.83:0/3181094839 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec21dbf270 0x55ec21dc9400 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f097c02a930 tx=0x7f097c032cc0 comp rx=0 tx=0).stop 2024-04-24T08:54:33.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.668+0000 7f098b400640 1 --2- 172.21.15.83:0/3772969663 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f09700a3e70 0x7f09700a6330 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f09840092f0 tx=0x7f0984009280 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:33.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.668+0000 7f099481f740 1 -- 172.21.15.83:0/3181094839 shutdown_connections 2024-04-24T08:54:33.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.668+0000 7f099481f740 1 --2- 172.21.15.83:0/3181094839 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ec21dc9940 0x55ec21dcbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:33.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.668+0000 7f099481f740 1 --2- 172.21.15.83:0/3181094839 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec21dbf270 0x55ec21dc9400 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:33.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.668+0000 7f099481f740 1 --2- 172.21.15.83:0/3181094839 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec21dbe950 0x55ec21dbed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:33.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.668+0000 7f099481f740 1 -- 172.21.15.83:0/3181094839 >> 172.21.15.83:0/3181094839 conn(0x55ec21dbac70 msgr2=0x55ec21dbb050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:33.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.668+0000 7f099481f740 1 -- 172.21.15.83:0/3181094839 shutdown_connections 2024-04-24T08:54:33.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.668+0000 7f099481f740 1 -- 172.21.15.83:0/3181094839 wait complete. 2024-04-24T08:54:33.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.669+0000 7f099481f740 1 -- start start 2024-04-24T08:54:33.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.669+0000 7f099481f740 1 Processor -- start 2024-04-24T08:54:33.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.669+0000 7f099481f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ec21dbe950 0x55ec21dc9320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:33.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.669+0000 7f099481f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec21dbf270 0x55ec21dc6610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:33.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.669+0000 7f099481f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec21dc9940 0x55ec21dc6c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:33.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.669+0000 7f099481f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ec21dcc770 con 0x55ec21dc9940 2024-04-24T08:54:33.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.669+0000 7f099481f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ec21cdd810 con 0x55ec21dbf270 2024-04-24T08:54:33.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.669+0000 7f099481f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55ec21dcc8f0 con 0x55ec21dbe950 2024-04-24T08:54:33.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.670+0000 7f099481f740 10 client.0 _ll_register_callbacks cb 0x55ec21cab140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:54:33.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.670+0000 7f098b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec21dc9940 0x55ec21dc6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:33.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.670+0000 7f098b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec21dc9940 0x55ec21dc6c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41808/0 (socket says 172.21.15.83:41808) 2024-04-24T08:54:33.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.670+0000 7f098b400640 1 -- 172.21.15.83:0/3673721269 learned_addr learned my addr 172.21.15.83:0/3673721269 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:54:33.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.670+0000 7f0983e00640 1 --2- 172.21.15.83:0/3673721269 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec21dbf270 0x55ec21dc6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:33.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.670+0000 7f098aa00640 1 --2- 172.21.15.83:0/3673721269 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ec21dbe950 0x55ec21dc9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:33.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.670+0000 7f098b400640 1 -- 172.21.15.83:0/3673721269 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ec21dbe950 msgr2=0x55ec21dc9320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:33.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.670+0000 7f098b400640 1 --2- 172.21.15.83:0/3673721269 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ec21dbe950 0x55ec21dc9320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:33.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.670+0000 7f098b400640 1 -- 172.21.15.83:0/3673721269 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec21dbf270 msgr2=0x55ec21dc6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:33.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.670+0000 7f098b400640 1 --2- 172.21.15.83:0/3673721269 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec21dbf270 0x55ec21dc6610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:33.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.670+0000 7f098b400640 1 -- 172.21.15.83:0/3673721269 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec21cee3f0 con 0x55ec21dc9940 2024-04-24T08:54:33.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.671+0000 7f098b400640 1 --2- 172.21.15.83:0/3673721269 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec21dc9940 0x55ec21dc6c20 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f0984009d40 tx=0x7f0984004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50c66ceb63195807 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:33.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.671+0000 7f0978c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:54:33.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.671+0000 7f0978c00640 1 -- 172.21.15.83:0/3673721269 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f098400eaa0 con 0x55ec21dc9940 2024-04-24T08:54:33.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.671+0000 7f0978c00640 1 -- 172.21.15.83:0/3673721269 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f098400ec40 con 0x55ec21dc9940 2024-04-24T08:54:33.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.671+0000 7f099481f740 1 -- 172.21.15.83:0/3673721269 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f097c0316a0 con 0x55ec21dc9940 2024-04-24T08:54:33.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.671+0000 7f0978c00640 1 -- 172.21.15.83:0/3673721269 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0984030050 con 0x55ec21dc9940 2024-04-24T08:54:33.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.671+0000 7f099481f740 10 client.16767 fetch_fsmap learned FSMap version 30 2024-04-24T08:54:33.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.671+0000 7f099481f740 1 -- 172.21.15.83:0/3673721269 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ec21a363f0 con 0x55ec21dc9940 2024-04-24T08:54:33.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.672+0000 7f0978c00640 1 -- 172.21.15.83:0/3673721269 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f097c0316a0 con 0x55ec21dc9940 2024-04-24T08:54:33.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.672+0000 7f0978c00640 1 -- 172.21.15.83:0/3673721269 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7f098400f040 con 0x55ec21dc9940 2024-04-24T08:54:33.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.672+0000 7f099481f740 10 client.16767 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T08:54:33.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.672+0000 7f099481f740 10 client.16767 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:54:33.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.672+0000 7f099481f740 20 client.16767 populate_metadata read hostname 'smithi083' 2024-04-24T08:54:33.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.672+0000 7f099481f740 1 --2- 172.21.15.83:0/3673721269 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ec21dd9be0 0x55ec21ddbff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:33.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.672+0000 7f099481f740 4 client.16767 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:54:33.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.672+0000 7f099481f740 1 -- 172.21.15.83:0/3673721269 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ec21d31bf0 con 0x55ec21dd9be0 2024-04-24T08:54:33.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.673+0000 7f098aa00640 1 --2- 172.21.15.83:0/3673721269 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ec21dd9be0 0x55ec21ddbff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:33.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.674+0000 7f098aa00640 1 --2- 172.21.15.83:0/3673721269 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ec21dd9be0 0x55ec21ddbff0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7f4a443bcc9c613 server_cookie=934f13610bc89faf in_seq=0 out_seq=0 2024-04-24T08:54:33.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.674+0000 7f0978c00640 10 client.16767 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:54:33.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.676+0000 7f0978c00640 1 -- 172.21.15.83:0/3673721269 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ec21d31bf0 con 0x55ec21dd9be0 2024-04-24T08:54:33.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.676+0000 7f0978c00640 10 client.16767 handle_command_reply: tid=0 2024-04-24T08:54:33.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.676+0000 7f099481f740 1 client.16767 shutdown 2024-04-24T08:54:33.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.677+0000 7f099481f740 1 -- 172.21.15.83:0/3673721269 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec21dc9940 msgr2=0x55ec21dc6c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:33.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.677+0000 7f099481f740 1 --2- 172.21.15.83:0/3673721269 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec21dc9940 0x55ec21dc6c20 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f0984009d40 tx=0x7f0984004a40 comp rx=0 tx=0).stop 2024-04-24T08:54:33.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.677+0000 7f099481f740 1 -- 172.21.15.83:0/3673721269 shutdown_connections 2024-04-24T08:54:33.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.677+0000 7f099481f740 1 --2- 172.21.15.83:0/3673721269 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ec21dd9be0 0x55ec21ddbff0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:33.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.677+0000 7f099481f740 1 --2- 172.21.15.83:0/3673721269 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec21dc9940 0x55ec21dc6c20 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:33.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.677+0000 7f099481f740 1 --2- 172.21.15.83:0/3673721269 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec21dbf270 0x55ec21dc6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:33.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.677+0000 7f099481f740 1 --2- 172.21.15.83:0/3673721269 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ec21dbe950 0x55ec21dc9320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:33.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.677+0000 7f099481f740 1 -- 172.21.15.83:0/3673721269 >> 172.21.15.83:0/3673721269 conn(0x55ec21dbac70 msgr2=0x55ec21db9df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:33.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.677+0000 7f0978c00640 0 client.16767 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:54:33.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.677+0000 7f099481f740 1 -- 172.21.15.83:0/3673721269 shutdown_connections 2024-04-24T08:54:33.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.677+0000 7f099481f740 1 -- 172.21.15.83:0/3673721269 wait complete. 2024-04-24T08:54:33.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.677+0000 7f099481f740 20 client.16767 trim_cache size 0 max 16384 2024-04-24T08:54:33.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.692+0000 7f099481f740 1 Processor -- start 2024-04-24T08:54:33.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.692+0000 7f099481f740 1 -- start start 2024-04-24T08:54:33.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.692+0000 7f099481f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec21dbe950 0x55ec21dd56c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:33.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.693+0000 7f099481f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec21dbf270 0x55ec21dc5770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:33.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.693+0000 7f099481f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ec21dc9940 0x55ec21dc07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:33.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.693+0000 7f099481f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f096800fff0 con 0x55ec21dbf270 2024-04-24T08:54:33.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.693+0000 7f099481f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0968011bc0 con 0x55ec21dbe950 2024-04-24T08:54:33.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.693+0000 7f099481f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f09680106c0 con 0x55ec21dc9940 2024-04-24T08:54:33.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.693+0000 7f0983e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec21dbf270 0x55ec21dc5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:33.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.693+0000 7f0983e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec21dbf270 0x55ec21dc5770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41818/0 (socket says 172.21.15.83:41818) 2024-04-24T08:54:33.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.693+0000 7f0983e00640 1 -- 172.21.15.83:0/1670931101 learned_addr learned my addr 172.21.15.83:0/1670931101 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:54:33.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.693+0000 7f098aa00640 1 --2- 172.21.15.83:0/1670931101 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec21dbe950 0x55ec21dd56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:33.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.693+0000 7f0983e00640 1 -- 172.21.15.83:0/1670931101 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ec21dc9940 msgr2=0x55ec21dc07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:33.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.693+0000 7f0983e00640 1 --2- 172.21.15.83:0/1670931101 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ec21dc9940 0x55ec21dc07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:33.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.693+0000 7f0983e00640 1 -- 172.21.15.83:0/1670931101 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec21dbe950 msgr2=0x55ec21dd56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:33.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.693+0000 7f0983e00640 1 --2- 172.21.15.83:0/1670931101 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec21dbe950 0x55ec21dd56c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:33.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.693+0000 7f0983e00640 1 -- 172.21.15.83:0/1670931101 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec21db6450 con 0x55ec21dbf270 2024-04-24T08:54:33.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.694+0000 7f0983e00640 1 --2- 172.21.15.83:0/1670931101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec21dbf270 0x55ec21dc5770 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f097400c940 tx=0x7f097409bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17a86da986cdc96a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:33.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.694+0000 7f097b400640 1 -- 172.21.15.83:0/1670931101 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f097401d040 con 0x55ec21dbf270 2024-04-24T08:54:33.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.694+0000 7f097b400640 1 -- 172.21.15.83:0/1670931101 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f09740a0670 con 0x55ec21dbf270 2024-04-24T08:54:33.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.694+0000 7f097b400640 1 -- 172.21.15.83:0/1670931101 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09740d02a0 con 0x55ec21dbf270 2024-04-24T08:54:33.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.695+0000 7f099481f740 1 -- 172.21.15.83:0/1670931101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec21dbf270 msgr2=0x55ec21dc5770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:33.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.695+0000 7f099481f740 1 --2- 172.21.15.83:0/1670931101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec21dbf270 0x55ec21dc5770 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f097400c940 tx=0x7f097409bec0 comp rx=0 tx=0).stop 2024-04-24T08:54:33.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.695+0000 7f099481f740 1 -- 172.21.15.83:0/1670931101 shutdown_connections 2024-04-24T08:54:33.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.695+0000 7f099481f740 1 --2- 172.21.15.83:0/1670931101 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ec21dc9940 0x55ec21dc07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:33.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.695+0000 7f099481f740 1 --2- 172.21.15.83:0/1670931101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec21dbf270 0x55ec21dc5770 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:33.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.695+0000 7f099481f740 1 --2- 172.21.15.83:0/1670931101 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec21dbe950 0x55ec21dd56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:33.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.695+0000 7f099481f740 1 -- 172.21.15.83:0/1670931101 >> 172.21.15.83:0/1670931101 conn(0x55ec21dbac70 msgr2=0x55ec21dc00a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:33.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.695+0000 7f099481f740 1 -- 172.21.15.83:0/1670931101 shutdown_connections 2024-04-24T08:54:33.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.695+0000 7f099481f740 1 -- 172.21.15.83:0/1670931101 wait complete. 2024-04-24T08:54:33.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.695+0000 7f099481f740 1 -- start start 2024-04-24T08:54:33.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.696+0000 7f099481f740 1 Processor -- start 2024-04-24T08:54:33.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.696+0000 7f099481f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec21dbe950 0x55ec21dc9f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:33.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.696+0000 7f099481f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ec21dbf270 0x55ec21dca480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:33.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.696+0000 7f099481f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec21dc9940 0x55ec21dd0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:33.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.696+0000 7f099481f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0968011460 con 0x55ec21dc9940 2024-04-24T08:54:33.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.696+0000 7f099481f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0968010d90 con 0x55ec21dbe950 2024-04-24T08:54:33.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.696+0000 7f099481f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f096800dfd0 con 0x55ec21dbf270 2024-04-24T08:54:33.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.696+0000 7f099481f740 10 client.0 _ll_register_callbacks cb 0x7f09840096e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:54:33.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.696+0000 7f098b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec21dc9940 0x55ec21dd0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:33.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.696+0000 7f098b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec21dc9940 0x55ec21dd0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41828/0 (socket says 172.21.15.83:41828) 2024-04-24T08:54:33.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.696+0000 7f098b400640 1 -- 172.21.15.83:0/2965145319 learned_addr learned my addr 172.21.15.83:0/2965145319 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:54:33.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.697+0000 7f0983e00640 1 --2- 172.21.15.83:0/2965145319 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ec21dbf270 0x55ec21dca480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:33.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.697+0000 7f098aa00640 1 --2- 172.21.15.83:0/2965145319 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec21dbe950 0x55ec21dc9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:33.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.697+0000 7f098b400640 1 -- 172.21.15.83:0/2965145319 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ec21dbf270 msgr2=0x55ec21dca480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:33.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.697+0000 7f098b400640 1 --2- 172.21.15.83:0/2965145319 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ec21dbf270 0x55ec21dca480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:33.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.697+0000 7f098b400640 1 -- 172.21.15.83:0/2965145319 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec21dbe950 msgr2=0x55ec21dc9f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:33.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.697+0000 7f098b400640 1 --2- 172.21.15.83:0/2965145319 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec21dbe950 0x55ec21dc9f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:33.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.697+0000 7f098b400640 1 -- 172.21.15.83:0/2965145319 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec21b007d0 con 0x55ec21dc9940 2024-04-24T08:54:33.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.697+0000 7f098aa00640 1 --2- 172.21.15.83:0/2965145319 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec21dbe950 0x55ec21dc9f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:54:33.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.697+0000 7f098b400640 1 --2- 172.21.15.83:0/2965145319 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec21dc9940 0x55ec21dd0bd0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f098401d540 tx=0x7f098401ddb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cefdc79bb283aa42 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:33.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.697+0000 7f0978c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:54:33.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.697+0000 7f0978c00640 1 -- 172.21.15.83:0/2965145319 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f098403a020 con 0x55ec21dc9940 2024-04-24T08:54:33.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.697+0000 7f0978c00640 1 -- 172.21.15.83:0/2965145319 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f098400eb30 con 0x55ec21dc9940 2024-04-24T08:54:33.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.697+0000 7f099481f740 1 -- 172.21.15.83:0/2965145319 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09740a0670 con 0x55ec21dc9940 2024-04-24T08:54:33.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.698+0000 7f0978c00640 1 -- 172.21.15.83:0/2965145319 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0984011c90 con 0x55ec21dc9940 2024-04-24T08:54:33.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.698+0000 7f099481f740 10 client.16773 fetch_fsmap learned FSMap version 30 2024-04-24T08:54:33.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.698+0000 7f099481f740 1 -- 172.21.15.83:0/2965145319 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ec21dd2170 con 0x55ec21dc9940 2024-04-24T08:54:33.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.698+0000 7f0978c00640 1 -- 172.21.15.83:0/2965145319 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f09740a0670 con 0x55ec21dc9940 2024-04-24T08:54:33.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.698+0000 7f0978c00640 1 -- 172.21.15.83:0/2965145319 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7f0984033070 con 0x55ec21dc9940 2024-04-24T08:54:33.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.698+0000 7f099481f740 10 client.16773 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T08:54:33.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.698+0000 7f099481f740 10 client.16773 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:54:33.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.698+0000 7f099481f740 20 client.16773 populate_metadata read hostname 'smithi083' 2024-04-24T08:54:33.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.698+0000 7f099481f740 1 --2- 172.21.15.83:0/2965145319 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ec21e40420 0x55ec21e42840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:33.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.698+0000 7f099481f740 4 client.16773 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T08:54:33.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.698+0000 7f099481f740 1 -- 172.21.15.83:0/2965145319 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ec21d2ba60 con 0x55ec21e40420 2024-04-24T08:54:33.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.699+0000 7f098aa00640 1 --2- 172.21.15.83:0/2965145319 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ec21e40420 0x55ec21e42840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:33.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.700+0000 7f098aa00640 1 --2- 172.21.15.83:0/2965145319 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ec21e40420 0x55ec21e42840 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c33a21252de187e server_cookie=fb3526aed02c8f66 in_seq=0 out_seq=0 2024-04-24T08:54:33.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.700+0000 7f0978c00640 10 client.16773 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:54:33.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.703+0000 7f0978c00640 1 -- 172.21.15.83:0/2965145319 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ec21d2ba60 con 0x55ec21e40420 2024-04-24T08:54:33.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.703+0000 7f0978c00640 10 client.16773 handle_command_reply: tid=0 2024-04-24T08:54:33.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.703+0000 7f099481f740 1 client.16773 shutdown 2024-04-24T08:54:33.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.703+0000 7f099481f740 1 -- 172.21.15.83:0/2965145319 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec21dc9940 msgr2=0x55ec21dd0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:33.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.703+0000 7f099481f740 1 --2- 172.21.15.83:0/2965145319 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec21dc9940 0x55ec21dd0bd0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f098401d540 tx=0x7f098401ddb0 comp rx=0 tx=0).stop 2024-04-24T08:54:33.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.703+0000 7f099481f740 1 -- 172.21.15.83:0/2965145319 shutdown_connections 2024-04-24T08:54:33.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.703+0000 7f099481f740 1 --2- 172.21.15.83:0/2965145319 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ec21e40420 0x55ec21e42840 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:33.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.703+0000 7f099481f740 1 --2- 172.21.15.83:0/2965145319 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec21dc9940 0x55ec21dd0bd0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:33.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.703+0000 7f099481f740 1 --2- 172.21.15.83:0/2965145319 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ec21dbf270 0x55ec21dca480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:33.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.703+0000 7f0978c00640 0 client.16773 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:54:33.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.703+0000 7f099481f740 1 --2- 172.21.15.83:0/2965145319 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec21dbe950 0x55ec21dc9f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:33.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.703+0000 7f099481f740 1 -- 172.21.15.83:0/2965145319 >> 172.21.15.83:0/2965145319 conn(0x55ec21dbac70 msgr2=0x55ec21dc0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:33.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.703+0000 7f099481f740 1 -- 172.21.15.83:0/2965145319 shutdown_connections 2024-04-24T08:54:33.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.703+0000 7f099481f740 1 -- 172.21.15.83:0/2965145319 wait complete. 2024-04-24T08:54:33.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.703+0000 7f099481f740 20 client.16773 trim_cache size 0 max 16384 2024-04-24T08:54:33.704 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T08:54:33.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.704+0000 7f0991a00640 1 -- 172.21.15.83:0/3772969663 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f09700a3e70 msgr2=0x7f09700a6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:33.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.704+0000 7f0991a00640 1 --2- 172.21.15.83:0/3772969663 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f09700a3e70 0x7f09700a6330 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f09840092f0 tx=0x7f0984009280 comp rx=0 tx=0).stop 2024-04-24T08:54:33.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.704+0000 7f0991a00640 1 -- 172.21.15.83:0/3772969663 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f098c109580 msgr2=0x7f098c1a0d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:33.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.704+0000 7f0991a00640 1 --2- 172.21.15.83:0/3772969663 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f098c109580 0x7f098c1a0d10 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f097400c970 tx=0x7f097400ce30 comp rx=0 tx=0).stop 2024-04-24T08:54:33.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.704+0000 7f0991a00640 1 -- 172.21.15.83:0/3772969663 shutdown_connections 2024-04-24T08:54:33.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.704+0000 7f0991a00640 1 --2- 172.21.15.83:0/3772969663 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f09700a3e70 0x7f09700a6330 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:33.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.704+0000 7f0991a00640 1 --2- 172.21.15.83:0/3772969663 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f098c19a9d0 0x7f098c19ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:33.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.704+0000 7f0991a00640 1 --2- 172.21.15.83:0/3772969663 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f098c109580 0x7f098c1a0d10 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:33.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.704+0000 7f0991a00640 1 --2- 172.21.15.83:0/3772969663 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f098c1044b0 0x7f098c1a07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:33.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.704+0000 7f0991a00640 1 -- 172.21.15.83:0/3772969663 >> 172.21.15.83:0/3772969663 conn(0x7f098c0ffd90 msgr2=0x7f098c101180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:33.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.704+0000 7f0991a00640 1 -- 172.21.15.83:0/3772969663 shutdown_connections 2024-04-24T08:54:33.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:33.704+0000 7f0991a00640 1 -- 172.21.15.83:0/3772969663 wait complete. 2024-04-24T08:54:34.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:54:33 smithi083 ceph-mon[27907]: pgmap v91: 129 pgs: 129 active+clean; 28 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 475 KiB/s wr, 76 op/s 2024-04-24T08:54:34.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:54:33 smithi138 ceph-mon[31441]: pgmap v91: 129 pgs: 129 active+clean; 28 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 475 KiB/s wr, 76 op/s 2024-04-24T08:54:34.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:54:33 smithi189 ceph-mon[31791]: pgmap v91: 129 pgs: 129 active+clean; 28 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 475 KiB/s wr, 76 op/s 2024-04-24T08:54:34.561 INFO:tasks.workunit.client.0.smithi083.stdout: 1 8697 0.95 MB/sec warmup 68 sec latency 51.319 ms 2024-04-24T08:54:34.722 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T08:54:34.723 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T08:54:34.723 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T08:54:34.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.954+0000 7f5379a00640 1 -- 172.21.15.83:0/3171453628 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f537410c480 msgr2=0x7f537410e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:34.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.954+0000 7f5379a00640 1 --2- 172.21.15.83:0/3171453628 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f537410c480 0x7f537410e920 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f536c009ca0 tx=0x7f536c02a940 comp rx=0 tx=0).stop 2024-04-24T08:54:34.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.954+0000 7f5379a00640 1 -- 172.21.15.83:0/3171453628 shutdown_connections 2024-04-24T08:54:34.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.954+0000 7f5379a00640 1 --2- 172.21.15.83:0/3171453628 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f537410c480 0x7f537410e920 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:34.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.954+0000 7f5379a00640 1 --2- 172.21.15.83:0/3171453628 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5374103980 0x7f537410be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:34.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.954+0000 7f5379a00640 1 --2- 172.21.15.83:0/3171453628 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5374103060 0x7f5374103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:34.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.954+0000 7f5379a00640 1 -- 172.21.15.83:0/3171453628 >> 172.21.15.83:0/3171453628 conn(0x7f53740fc880 msgr2=0x7f53740feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:34.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.954+0000 7f5379a00640 1 -- 172.21.15.83:0/3171453628 shutdown_connections 2024-04-24T08:54:34.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.954+0000 7f5379a00640 1 -- 172.21.15.83:0/3171453628 wait complete. 2024-04-24T08:54:34.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.956+0000 7f5379a00640 1 Processor -- start 2024-04-24T08:54:34.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.956+0000 7f5379a00640 1 -- start start 2024-04-24T08:54:34.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.956+0000 7f5379a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5374103060 0x7f537419a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:34.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.956+0000 7f5379a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5374103980 0x7f537419ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:34.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.956+0000 7f5379a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f537410c480 0x7f537419eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:34.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.956+0000 7f5379a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f53741120f0 con 0x7f537410c480 2024-04-24T08:54:34.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.956+0000 7f5379a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5374111f70 con 0x7f5374103060 2024-04-24T08:54:34.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.956+0000 7f5379a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5374112270 con 0x7f5374103980 2024-04-24T08:54:34.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.956+0000 7f5379000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f537410c480 0x7f537419eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:34.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.956+0000 7f5379000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f537410c480 0x7f537419eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41846/0 (socket says 172.21.15.83:41846) 2024-04-24T08:54:34.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.956+0000 7f5379000640 1 -- 172.21.15.83:0/835641227 learned_addr learned my addr 172.21.15.83:0/835641227 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:54:34.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.957+0000 7f5373e00640 1 --2- 172.21.15.83:0/835641227 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5374103060 0x7f537419a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:34.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.957+0000 7f5373400640 1 --2- 172.21.15.83:0/835641227 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5374103980 0x7f537419ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:34.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.957+0000 7f5379000640 1 -- 172.21.15.83:0/835641227 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5374103980 msgr2=0x7f537419ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:34.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.957+0000 7f5379000640 1 --2- 172.21.15.83:0/835641227 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5374103980 0x7f537419ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:34.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.957+0000 7f5379000640 1 -- 172.21.15.83:0/835641227 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5374103060 msgr2=0x7f537419a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:34.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.957+0000 7f5379000640 1 --2- 172.21.15.83:0/835641227 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5374103060 0x7f537419a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:34.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.957+0000 7f5379000640 1 -- 172.21.15.83:0/835641227 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f537419f720 con 0x7f537410c480 2024-04-24T08:54:34.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.957+0000 7f5379000640 1 --2- 172.21.15.83:0/835641227 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f537410c480 0x7f537419eff0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f536c009dc0 tx=0x7f536c02fa90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:34.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.957+0000 7f5370c00640 1 -- 172.21.15.83:0/835641227 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f536c034040 con 0x7f537410c480 2024-04-24T08:54:34.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.958+0000 7f5370c00640 1 -- 172.21.15.83:0/835641227 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f536c02fcb0 con 0x7f537410c480 2024-04-24T08:54:34.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.958+0000 7f5379a00640 1 -- 172.21.15.83:0/835641227 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f537419f9b0 con 0x7f537410c480 2024-04-24T08:54:34.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.958+0000 7f5370c00640 1 -- 172.21.15.83:0/835641227 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f536c039430 con 0x7f537410c480 2024-04-24T08:54:34.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.958+0000 7f5379a00640 1 -- 172.21.15.83:0/835641227 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53741a72a0 con 0x7f537410c480 2024-04-24T08:54:34.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.961+0000 7f5370c00640 1 -- 172.21.15.83:0/835641227 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f536c037020 con 0x7f537410c480 2024-04-24T08:54:34.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.961+0000 7f537df7d740 1 Processor -- start 2024-04-24T08:54:34.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.962+0000 7f537df7d740 1 -- start start 2024-04-24T08:54:34.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.962+0000 7f537df7d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569e8e88270 0x5569e8e88650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:34.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.962+0000 7f537df7d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569e8e88b90 0x5569e8e92d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:34.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.962+0000 7f537df7d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5569e8e93260 0x5569e8e95650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:34.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.962+0000 7f537df7d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5569e8dede70 con 0x5569e8e88270 2024-04-24T08:54:34.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.962+0000 7f537df7d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5569e8d94380 con 0x5569e8e88b90 2024-04-24T08:54:34.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.962+0000 7f537df7d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5569e8de9e30 con 0x5569e8e93260 2024-04-24T08:54:34.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.962+0000 7f5373400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569e8e88270 0x5569e8e88650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:34.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.962+0000 7f5373400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569e8e88270 0x5569e8e88650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41862/0 (socket says 172.21.15.83:41862) 2024-04-24T08:54:34.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.962+0000 7f5373400640 1 -- 172.21.15.83:0/1572911883 learned_addr learned my addr 172.21.15.83:0/1572911883 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:54:34.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.963+0000 7f5373e00640 1 --2- 172.21.15.83:0/1572911883 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569e8e88b90 0x5569e8e92d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:34.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.963+0000 7f5373400640 1 -- 172.21.15.83:0/1572911883 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5569e8e93260 msgr2=0x5569e8e95650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:34.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.963+0000 7f5373400640 1 --2- 172.21.15.83:0/1572911883 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5569e8e93260 0x5569e8e95650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:34.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.963+0000 7f5373400640 1 -- 172.21.15.83:0/1572911883 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569e8e88b90 msgr2=0x5569e8e92d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:34.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.963+0000 7f5373400640 1 --2- 172.21.15.83:0/1572911883 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569e8e88b90 0x5569e8e92d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:34.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.963+0000 7f5373400640 1 -- 172.21.15.83:0/1572911883 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569e8bc87e0 con 0x5569e8e88270 2024-04-24T08:54:34.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.963+0000 7f5370c00640 1 --2- 172.21.15.83:0/835641227 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f53580a3da0 0x7f53580a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:34.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.963+0000 7f5373400640 1 --2- 172.21.15.83:0/1572911883 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569e8e88270 0x5569e8e88650 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f5360008bc0 tx=0x7f5360010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f843a5006100c470 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:34.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.963+0000 7f5370c00640 1 -- 172.21.15.83:0/835641227 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f536c0f1d90 con 0x7f537410c480 2024-04-24T08:54:34.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.963+0000 7f5369600640 1 -- 172.21.15.83:0/1572911883 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5360016070 con 0x5569e8e88270 2024-04-24T08:54:34.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.963+0000 7f5373e00640 1 --2- 172.21.15.83:0/835641227 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f53580a3da0 0x7f53580a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:34.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.964+0000 7f5369600640 1 -- 172.21.15.83:0/1572911883 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5360012b30 con 0x5569e8e88270 2024-04-24T08:54:34.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.964+0000 7f5369600640 1 -- 172.21.15.83:0/1572911883 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5360011a40 con 0x5569e8e88270 2024-04-24T08:54:34.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.964+0000 7f537df7d740 1 -- 172.21.15.83:0/1572911883 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569e8e88270 msgr2=0x5569e8e88650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:34.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.964+0000 7f5373e00640 1 --2- 172.21.15.83:0/835641227 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f53580a3da0 0x7f53580a6260 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f53640058c0 tx=0x7f5364005850 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:34.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.964+0000 7f537df7d740 1 --2- 172.21.15.83:0/1572911883 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569e8e88270 0x5569e8e88650 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f5360008bc0 tx=0x7f5360010740 comp rx=0 tx=0).stop 2024-04-24T08:54:34.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.964+0000 7f537df7d740 1 -- 172.21.15.83:0/1572911883 shutdown_connections 2024-04-24T08:54:34.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.965+0000 7f537df7d740 1 --2- 172.21.15.83:0/1572911883 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5569e8e93260 0x5569e8e95650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:34.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.965+0000 7f537df7d740 1 --2- 172.21.15.83:0/1572911883 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569e8e88b90 0x5569e8e92d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:34.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.965+0000 7f537df7d740 1 --2- 172.21.15.83:0/1572911883 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569e8e88270 0x5569e8e88650 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:34.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.965+0000 7f537df7d740 1 -- 172.21.15.83:0/1572911883 >> 172.21.15.83:0/1572911883 conn(0x5569e8e84590 msgr2=0x5569e8e84970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:34.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.965+0000 7f537df7d740 1 -- 172.21.15.83:0/1572911883 shutdown_connections 2024-04-24T08:54:34.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.965+0000 7f537df7d740 1 -- 172.21.15.83:0/1572911883 wait complete. 2024-04-24T08:54:34.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.965+0000 7f537df7d740 1 -- start start 2024-04-24T08:54:34.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.965+0000 7f537df7d740 1 Processor -- start 2024-04-24T08:54:34.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.966+0000 7f537df7d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569e8e88270 0x5569e8e90270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:34.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.966+0000 7f537df7d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569e8e88b90 0x5569e8e907b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:34.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.966+0000 7f537df7d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5569e8e93260 0x5569e8e90cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:34.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.966+0000 7f537df7d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5569e8e96050 con 0x5569e8e88270 2024-04-24T08:54:34.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.966+0000 7f537df7d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5569e8da7660 con 0x5569e8e88b90 2024-04-24T08:54:34.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.966+0000 7f537df7d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5569e8e961d0 con 0x5569e8e93260 2024-04-24T08:54:34.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.966+0000 7f537df7d740 10 client.0 _ll_register_callbacks cb 0x5569e8d95aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:54:34.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.966+0000 7f5373e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569e8e88270 0x5569e8e90270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:34.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.966+0000 7f5373e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569e8e88270 0x5569e8e90270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41864/0 (socket says 172.21.15.83:41864) 2024-04-24T08:54:34.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.966+0000 7f5373e00640 1 -- 172.21.15.83:0/3010943264 learned_addr learned my addr 172.21.15.83:0/3010943264 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:54:34.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.966+0000 7f5373400640 1 --2- 172.21.15.83:0/3010943264 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569e8e88b90 0x5569e8e907b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:34.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.967+0000 7f5379000640 1 --2- 172.21.15.83:0/3010943264 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5569e8e93260 0x5569e8e90cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:34.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.967+0000 7f5373e00640 1 -- 172.21.15.83:0/3010943264 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5569e8e93260 msgr2=0x5569e8e90cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:34.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.967+0000 7f5373e00640 1 --2- 172.21.15.83:0/3010943264 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5569e8e93260 0x5569e8e90cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:34.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.967+0000 7f5373e00640 1 -- 172.21.15.83:0/3010943264 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569e8e88b90 msgr2=0x5569e8e907b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:34.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.967+0000 7f5373e00640 1 --2- 172.21.15.83:0/3010943264 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569e8e88b90 0x5569e8e907b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:34.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.967+0000 7f5373e00640 1 -- 172.21.15.83:0/3010943264 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569e8c28f70 con 0x5569e8e88270 2024-04-24T08:54:34.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.967+0000 7f5379000640 1 --2- 172.21.15.83:0/3010943264 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5569e8e93260 0x5569e8e90cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:54:34.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.967+0000 7f5373e00640 1 --2- 172.21.15.83:0/3010943264 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569e8e88270 0x5569e8e90270 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f5364011e10 tx=0x7f53640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8cc91d779c72fb9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:34.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.967+0000 7f535e000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:54:34.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.967+0000 7f535e000640 1 -- 172.21.15.83:0/3010943264 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5364004030 con 0x5569e8e88270 2024-04-24T08:54:34.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.967+0000 7f535e000640 1 -- 172.21.15.83:0/3010943264 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f53640041d0 con 0x5569e8e88270 2024-04-24T08:54:34.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.967+0000 7f537df7d740 1 -- 172.21.15.83:0/3010943264 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5360012b30 con 0x5569e8e88270 2024-04-24T08:54:34.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.968+0000 7f535e000640 1 -- 172.21.15.83:0/3010943264 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f536402e050 con 0x5569e8e88270 2024-04-24T08:54:34.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.968+0000 7f537df7d740 10 client.16785 fetch_fsmap learned FSMap version 30 2024-04-24T08:54:34.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.968+0000 7f537df7d740 1 -- 172.21.15.83:0/3010943264 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5569e8ac0170 con 0x5569e8e88270 2024-04-24T08:54:34.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.968+0000 7f535e000640 1 -- 172.21.15.83:0/3010943264 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5360012b30 con 0x5569e8e88270 2024-04-24T08:54:34.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.968+0000 7f535e000640 1 -- 172.21.15.83:0/3010943264 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7f536401e420 con 0x5569e8e88270 2024-04-24T08:54:34.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.968+0000 7f537df7d740 10 client.16785 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T08:54:34.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.968+0000 7f537df7d740 10 client.16785 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:54:34.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.968+0000 7f537df7d740 20 client.16785 populate_metadata read hostname 'smithi083' 2024-04-24T08:54:34.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.968+0000 7f537df7d740 1 --2- 172.21.15.83:0/3010943264 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5569e8ea37f0 0x5569e8ea5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:34.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.968+0000 7f537df7d740 4 client.16785 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:54:34.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.968+0000 7f537df7d740 1 -- 172.21.15.83:0/3010943264 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5569e8dfb890 con 0x5569e8ea37f0 2024-04-24T08:54:34.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.969+0000 7f5373400640 1 --2- 172.21.15.83:0/3010943264 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5569e8ea37f0 0x5569e8ea5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:34.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.970+0000 7f5373400640 1 --2- 172.21.15.83:0/3010943264 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5569e8ea37f0 0x5569e8ea5c70 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9fbb748518d26e7b server_cookie=3ee6eb3e5921bf63 in_seq=0 out_seq=0 2024-04-24T08:54:34.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.970+0000 7f535e000640 10 client.16785 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:54:34.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.972+0000 7f535e000640 1 -- 172.21.15.83:0/3010943264 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5569e8dfb890 con 0x5569e8ea37f0 2024-04-24T08:54:34.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.972+0000 7f535e000640 10 client.16785 handle_command_reply: tid=0 2024-04-24T08:54:34.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.972+0000 7f537df7d740 1 client.16785 shutdown 2024-04-24T08:54:34.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.972+0000 7f537df7d740 1 -- 172.21.15.83:0/3010943264 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569e8e88270 msgr2=0x5569e8e90270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:34.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.972+0000 7f537df7d740 1 --2- 172.21.15.83:0/3010943264 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569e8e88270 0x5569e8e90270 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f5364011e10 tx=0x7f53640046c0 comp rx=0 tx=0).stop 2024-04-24T08:54:34.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.972+0000 7f537df7d740 1 -- 172.21.15.83:0/3010943264 shutdown_connections 2024-04-24T08:54:34.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.972+0000 7f537df7d740 1 --2- 172.21.15.83:0/3010943264 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5569e8ea37f0 0x5569e8ea5c70 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:34.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.972+0000 7f537df7d740 1 --2- 172.21.15.83:0/3010943264 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5569e8e93260 0x5569e8e90cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:34.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.972+0000 7f537df7d740 1 --2- 172.21.15.83:0/3010943264 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569e8e88b90 0x5569e8e907b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:34.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.972+0000 7f537df7d740 1 --2- 172.21.15.83:0/3010943264 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569e8e88270 0x5569e8e90270 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:34.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.972+0000 7f535e000640 0 client.16785 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:54:34.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.972+0000 7f537df7d740 1 -- 172.21.15.83:0/3010943264 >> 172.21.15.83:0/3010943264 conn(0x5569e8e84590 msgr2=0x5569e8e8a290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:34.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.973+0000 7f537df7d740 1 -- 172.21.15.83:0/3010943264 shutdown_connections 2024-04-24T08:54:34.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.973+0000 7f537df7d740 1 -- 172.21.15.83:0/3010943264 wait complete. 2024-04-24T08:54:34.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.973+0000 7f537df7d740 20 client.16785 trim_cache size 0 max 16384 2024-04-24T08:54:34.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.994+0000 7f537df7d740 1 Processor -- start 2024-04-24T08:54:34.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.994+0000 7f537df7d740 1 -- start start 2024-04-24T08:54:34.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.994+0000 7f537df7d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569e8e88270 0x5569e8e9ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:34.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.994+0000 7f537df7d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569e8e88b90 0x5569e8e8a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:34.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.994+0000 7f537df7d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5569e8e93260 0x5569e8e83570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:34.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.994+0000 7f537df7d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f53741b12e0 con 0x5569e8e88b90 2024-04-24T08:54:34.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.994+0000 7f537df7d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f53741b2eb0 con 0x5569e8e88270 2024-04-24T08:54:34.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.994+0000 7f537df7d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f53741b19b0 con 0x5569e8e93260 2024-04-24T08:54:34.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.994+0000 7f5373400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569e8e88b90 0x5569e8e8a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:34.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.994+0000 7f5373400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569e8e88b90 0x5569e8e8a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41870/0 (socket says 172.21.15.83:41870) 2024-04-24T08:54:34.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.994+0000 7f5373400640 1 -- 172.21.15.83:0/800695427 learned_addr learned my addr 172.21.15.83:0/800695427 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:54:34.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.995+0000 7f5373400640 1 -- 172.21.15.83:0/800695427 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5569e8e93260 msgr2=0x5569e8e83570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:34.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.995+0000 7f5373400640 1 --2- 172.21.15.83:0/800695427 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5569e8e93260 0x5569e8e83570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:34.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.995+0000 7f5373400640 1 -- 172.21.15.83:0/800695427 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569e8e88270 msgr2=0x5569e8e9ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:34.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.995+0000 7f5373400640 1 --2- 172.21.15.83:0/800695427 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569e8e88270 0x5569e8e9ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:34.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.995+0000 7f5373400640 1 -- 172.21.15.83:0/800695427 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569e8e7fe80 con 0x5569e8e88b90 2024-04-24T08:54:34.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.995+0000 7f5373400640 1 --2- 172.21.15.83:0/800695427 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569e8e88b90 0x5569e8e8a9d0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f5360008b60 tx=0x7f5360010f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85df444ff28103fd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:34.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.995+0000 7f5368c00640 1 -- 172.21.15.83:0/800695427 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5360016070 con 0x5569e8e88b90 2024-04-24T08:54:34.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.995+0000 7f5368c00640 1 -- 172.21.15.83:0/800695427 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5360012480 con 0x5569e8e88b90 2024-04-24T08:54:34.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.995+0000 7f5368c00640 1 -- 172.21.15.83:0/800695427 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5360003850 con 0x5569e8e88b90 2024-04-24T08:54:34.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.996+0000 7f537df7d740 1 -- 172.21.15.83:0/800695427 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569e8e88b90 msgr2=0x5569e8e8a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:34.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.996+0000 7f537df7d740 1 --2- 172.21.15.83:0/800695427 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569e8e88b90 0x5569e8e8a9d0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f5360008b60 tx=0x7f5360010f70 comp rx=0 tx=0).stop 2024-04-24T08:54:34.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.996+0000 7f537df7d740 1 -- 172.21.15.83:0/800695427 shutdown_connections 2024-04-24T08:54:34.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.996+0000 7f537df7d740 1 --2- 172.21.15.83:0/800695427 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5569e8e93260 0x5569e8e83570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:34.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.996+0000 7f537df7d740 1 --2- 172.21.15.83:0/800695427 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569e8e88b90 0x5569e8e8a9d0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:34.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.996+0000 7f537df7d740 1 --2- 172.21.15.83:0/800695427 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569e8e88270 0x5569e8e9ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:34.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.996+0000 7f537df7d740 1 -- 172.21.15.83:0/800695427 >> 172.21.15.83:0/800695427 conn(0x5569e8e84590 msgr2=0x5569e8e82e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:34.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.996+0000 7f537df7d740 1 -- 172.21.15.83:0/800695427 shutdown_connections 2024-04-24T08:54:34.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.996+0000 7f537df7d740 1 -- 172.21.15.83:0/800695427 wait complete. 2024-04-24T08:54:34.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.997+0000 7f537df7d740 1 -- start start 2024-04-24T08:54:34.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.997+0000 7f537df7d740 1 Processor -- start 2024-04-24T08:54:34.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.997+0000 7f537df7d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569e8e88270 0x5569e8e9b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:34.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.997+0000 7f537df7d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5569e8e93260 0x5569e8e97930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:34.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.997+0000 7f537df7d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569e8e97e70 0x5569e8e982f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:34.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.997+0000 7f537df7d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f53741b2750 con 0x5569e8e88270 2024-04-24T08:54:34.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.997+0000 7f537df7d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f53741b2080 con 0x5569e8e97e70 2024-04-24T08:54:34.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.997+0000 7f537df7d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f53741af2c0 con 0x5569e8e93260 2024-04-24T08:54:34.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.997+0000 7f537df7d740 10 client.0 _ll_register_callbacks cb 0x7f536400c960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:54:34.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.998+0000 7f5373e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569e8e88270 0x5569e8e9b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:34.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.998+0000 7f5373e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569e8e88270 0x5569e8e9b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41880/0 (socket says 172.21.15.83:41880) 2024-04-24T08:54:34.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.998+0000 7f5373e00640 1 -- 172.21.15.83:0/591750976 learned_addr learned my addr 172.21.15.83:0/591750976 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:54:35.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.998+0000 7f5373400640 1 --2- 172.21.15.83:0/591750976 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5569e8e93260 0x5569e8e97930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:35.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.998+0000 7f5373e00640 1 -- 172.21.15.83:0/591750976 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5569e8e93260 msgr2=0x5569e8e97930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:35.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.998+0000 7f5373e00640 1 --2- 172.21.15.83:0/591750976 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5569e8e93260 0x5569e8e97930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:35.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.998+0000 7f5379000640 1 --2- 172.21.15.83:0/591750976 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569e8e97e70 0x5569e8e982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:35.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.998+0000 7f5373e00640 1 -- 172.21.15.83:0/591750976 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569e8e97e70 msgr2=0x5569e8e982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:35.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.998+0000 7f5373e00640 1 --2- 172.21.15.83:0/591750976 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569e8e97e70 0x5569e8e982f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:35.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.998+0000 7f5373e00640 1 -- 172.21.15.83:0/591750976 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569e8f09e00 con 0x5569e8e88270 2024-04-24T08:54:35.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.998+0000 7f5373e00640 1 --2- 172.21.15.83:0/591750976 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569e8e88270 0x5569e8e9b650 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f536401e810 tx=0x7f536400c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e2f5e0cf6d33df7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:35.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.998+0000 7f535e000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:54:35.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.999+0000 7f535e000640 1 -- 172.21.15.83:0/591750976 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f536401e620 con 0x5569e8e88270 2024-04-24T08:54:35.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.999+0000 7f535e000640 1 -- 172.21.15.83:0/591750976 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f536401ea80 con 0x5569e8e88270 2024-04-24T08:54:35.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.999+0000 7f537df7d740 1 -- 172.21.15.83:0/591750976 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5360012480 con 0x5569e8e88270 2024-04-24T08:54:35.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.999+0000 7f535e000640 1 -- 172.21.15.83:0/591750976 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f536401ecc0 con 0x5569e8e88270 2024-04-24T08:54:35.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.999+0000 7f537df7d740 10 client.16791 fetch_fsmap learned FSMap version 30 2024-04-24T08:54:35.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.999+0000 7f537df7d740 1 -- 172.21.15.83:0/591750976 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5569e8f0b050 con 0x5569e8e88270 2024-04-24T08:54:35.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.999+0000 7f535e000640 1 -- 172.21.15.83:0/591750976 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5360012480 con 0x5569e8e88270 2024-04-24T08:54:35.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.999+0000 7f535e000640 1 -- 172.21.15.83:0/591750976 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7f5364003960 con 0x5569e8e88270 2024-04-24T08:54:35.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.999+0000 7f537df7d740 10 client.16791 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T08:54:35.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.999+0000 7f537df7d740 10 client.16791 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:54:35.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.999+0000 7f537df7d740 20 client.16791 populate_metadata read hostname 'smithi083' 2024-04-24T08:54:35.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:34.999+0000 7f537df7d740 1 --2- 172.21.15.83:0/591750976 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5569e8f0b890 0x5569e8f0dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:35.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.000+0000 7f537df7d740 4 client.16791 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T08:54:35.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.000+0000 7f537df7d740 1 -- 172.21.15.83:0/591750976 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5569e8df5700 con 0x5569e8f0b890 2024-04-24T08:54:35.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.000+0000 7f5373400640 1 --2- 172.21.15.83:0/591750976 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5569e8f0b890 0x5569e8f0dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:35.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.001+0000 7f5373400640 1 --2- 172.21.15.83:0/591750976 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5569e8f0b890 0x5569e8f0dcb0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90112012686022d server_cookie=e1b80651ea7141b9 in_seq=0 out_seq=0 2024-04-24T08:54:35.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.001+0000 7f535e000640 10 client.16791 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:54:35.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.003+0000 7f535e000640 1 -- 172.21.15.83:0/591750976 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5569e8df5700 con 0x5569e8f0b890 2024-04-24T08:54:35.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.003+0000 7f535e000640 10 client.16791 handle_command_reply: tid=0 2024-04-24T08:54:35.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.004+0000 7f537df7d740 1 client.16791 shutdown 2024-04-24T08:54:35.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.004+0000 7f537df7d740 1 -- 172.21.15.83:0/591750976 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569e8e88270 msgr2=0x5569e8e9b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:35.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.004+0000 7f537df7d740 1 --2- 172.21.15.83:0/591750976 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569e8e88270 0x5569e8e9b650 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f536401e810 tx=0x7f536400c460 comp rx=0 tx=0).stop 2024-04-24T08:54:35.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.004+0000 7f537df7d740 1 -- 172.21.15.83:0/591750976 shutdown_connections 2024-04-24T08:54:35.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.004+0000 7f537df7d740 1 --2- 172.21.15.83:0/591750976 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5569e8f0b890 0x5569e8f0dcb0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:35.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.004+0000 7f537df7d740 1 --2- 172.21.15.83:0/591750976 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569e8e97e70 0x5569e8e982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:35.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.004+0000 7f537df7d740 1 --2- 172.21.15.83:0/591750976 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5569e8e93260 0x5569e8e97930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:35.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.004+0000 7f537df7d740 1 --2- 172.21.15.83:0/591750976 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569e8e88270 0x5569e8e9b650 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:35.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.004+0000 7f535e000640 0 client.16791 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:54:35.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.004+0000 7f537df7d740 1 -- 172.21.15.83:0/591750976 >> 172.21.15.83:0/591750976 conn(0x5569e8e84590 msgr2=0x5569e8e90aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:35.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.004+0000 7f537df7d740 1 -- 172.21.15.83:0/591750976 shutdown_connections 2024-04-24T08:54:35.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.004+0000 7f537df7d740 1 -- 172.21.15.83:0/591750976 wait complete. 2024-04-24T08:54:35.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.004+0000 7f537df7d740 20 client.16791 trim_cache size 0 max 16384 2024-04-24T08:54:35.005 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T08:54:35.005 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-04-24T08:54:35.005 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "0347b3ec-c9af-4549-9480-e0a992eb1783", 2024-04-24T08:54:35.005 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-04-24T08:54:35.005 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T08:54:35.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.005+0000 7f5379a00640 1 -- 172.21.15.83:0/835641227 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f53580a3da0 msgr2=0x7f53580a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:35.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.005+0000 7f5379a00640 1 --2- 172.21.15.83:0/835641227 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f53580a3da0 0x7f53580a6260 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f53640058c0 tx=0x7f5364005850 comp rx=0 tx=0).stop 2024-04-24T08:54:35.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.005+0000 7f5379a00640 1 -- 172.21.15.83:0/835641227 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f537410c480 msgr2=0x7f537419eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:35.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.005+0000 7f5379a00640 1 --2- 172.21.15.83:0/835641227 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f537410c480 0x7f537419eff0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f536c009dc0 tx=0x7f536c02fa90 comp rx=0 tx=0).stop 2024-04-24T08:54:35.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.005+0000 7f5379a00640 1 -- 172.21.15.83:0/835641227 shutdown_connections 2024-04-24T08:54:35.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.005+0000 7f5379a00640 1 --2- 172.21.15.83:0/835641227 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f53580a3da0 0x7f53580a6260 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:35.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.005+0000 7f5379a00640 1 --2- 172.21.15.83:0/835641227 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f537410c480 0x7f537419eff0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:35.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.005+0000 7f5379a00640 1 --2- 172.21.15.83:0/835641227 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5374103980 0x7f537419ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:35.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.005+0000 7f5379a00640 1 --2- 172.21.15.83:0/835641227 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5374103060 0x7f537419a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:35.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.005+0000 7f5379a00640 1 -- 172.21.15.83:0/835641227 >> 172.21.15.83:0/835641227 conn(0x7f53740fc880 msgr2=0x7f537410de50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:35.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.005+0000 7f5379a00640 1 -- 172.21.15.83:0/835641227 shutdown_connections 2024-04-24T08:54:35.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.005+0000 7f5379a00640 1 -- 172.21.15.83:0/835641227 wait complete. 2024-04-24T08:54:35.022 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T08:54:35.022 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T08:54:35.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.240+0000 7f2850a00640 1 -- 172.21.15.83:0/4160898219 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f284c1015f0 msgr2=0x7f284c10e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:35.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.240+0000 7f2850a00640 1 --2- 172.21.15.83:0/4160898219 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f284c1015f0 0x7f284c10e2b0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f2834009ca0 tx=0x7f283402a8c0 comp rx=0 tx=0).stop 2024-04-24T08:54:35.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.240+0000 7f2850a00640 1 -- 172.21.15.83:0/4160898219 shutdown_connections 2024-04-24T08:54:35.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.240+0000 7f2850a00640 1 --2- 172.21.15.83:0/4160898219 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f284c10e7f0 0x7f284c110be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:35.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.240+0000 7f2850a00640 1 --2- 172.21.15.83:0/4160898219 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f284c1015f0 0x7f284c10e2b0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:35.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.240+0000 7f2850a00640 1 --2- 172.21.15.83:0/4160898219 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f284c100c40 0x7f284c101020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:35.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.240+0000 7f2850a00640 1 -- 172.21.15.83:0/4160898219 >> 172.21.15.83:0/4160898219 conn(0x7f284c0fc880 msgr2=0x7f284c0feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:35.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.240+0000 7f2850a00640 1 -- 172.21.15.83:0/4160898219 shutdown_connections 2024-04-24T08:54:35.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.240+0000 7f2850a00640 1 -- 172.21.15.83:0/4160898219 wait complete. 2024-04-24T08:54:35.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.241+0000 7f2850a00640 1 Processor -- start 2024-04-24T08:54:35.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.242+0000 7f2850a00640 1 -- start start 2024-04-24T08:54:35.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.242+0000 7f2850a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f284c100c40 0x7f284c102f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:35.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.242+0000 7f2850a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f284c1015f0 0x7f284c103460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:35.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.242+0000 7f2850a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f284c10e7f0 0x7f284c1039a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:35.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.242+0000 7f2850a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f284c114320 con 0x7f284c10e7f0 2024-04-24T08:54:35.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.242+0000 7f2850a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f284c1141a0 con 0x7f284c1015f0 2024-04-24T08:54:35.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.242+0000 7f2850a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f284c1144a0 con 0x7f284c100c40 2024-04-24T08:54:35.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.242+0000 7f284b200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f284c10e7f0 0x7f284c1039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:35.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.242+0000 7f284b200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f284c10e7f0 0x7f284c1039a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41894/0 (socket says 172.21.15.83:41894) 2024-04-24T08:54:35.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.242+0000 7f284b200640 1 -- 172.21.15.83:0/2292825320 learned_addr learned my addr 172.21.15.83:0/2292825320 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:54:35.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.243+0000 7f284a800640 1 --2- 172.21.15.83:0/2292825320 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f284c100c40 0x7f284c102f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:35.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.243+0000 7f2849e00640 1 --2- 172.21.15.83:0/2292825320 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f284c1015f0 0x7f284c103460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:35.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.243+0000 7f284b200640 1 -- 172.21.15.83:0/2292825320 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f284c100c40 msgr2=0x7f284c102f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:35.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.243+0000 7f284b200640 1 --2- 172.21.15.83:0/2292825320 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f284c100c40 0x7f284c102f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:35.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.243+0000 7f284b200640 1 -- 172.21.15.83:0/2292825320 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f284c1015f0 msgr2=0x7f284c103460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:35.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.243+0000 7f284b200640 1 --2- 172.21.15.83:0/2292825320 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f284c1015f0 0x7f284c103460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:35.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.243+0000 7f284b200640 1 -- 172.21.15.83:0/2292825320 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f284c1aaec0 con 0x7f284c10e7f0 2024-04-24T08:54:35.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.243+0000 7f284b200640 1 --2- 172.21.15.83:0/2292825320 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f284c10e7f0 0x7f284c1039a0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f284400d720 tx=0x7f284400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:35.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.243+0000 7f2843400640 1 -- 172.21.15.83:0/2292825320 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2844017070 con 0x7f284c10e7f0 2024-04-24T08:54:35.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.243+0000 7f2843400640 1 -- 172.21.15.83:0/2292825320 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f284400de30 con 0x7f284c10e7f0 2024-04-24T08:54:35.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.243+0000 7f2843400640 1 -- 172.21.15.83:0/2292825320 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2844004cf0 con 0x7f284c10e7f0 2024-04-24T08:54:35.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.243+0000 7f2850a00640 1 -- 172.21.15.83:0/2292825320 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f284c1ab1b0 con 0x7f284c10e7f0 2024-04-24T08:54:35.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.244+0000 7f2850a00640 1 -- 172.21.15.83:0/2292825320 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f284c1ab6b0 con 0x7f284c10e7f0 2024-04-24T08:54:35.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.247+0000 7f28543d9740 1 Processor -- start 2024-04-24T08:54:35.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.247+0000 7f2843400640 1 -- 172.21.15.83:0/2292825320 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2844004e90 con 0x7f284c10e7f0 2024-04-24T08:54:35.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.247+0000 7f28543d9740 1 -- start start 2024-04-24T08:54:35.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.248+0000 7f28543d9740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560dd2bf3a30 0x560dd2bf3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:35.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.248+0000 7f28543d9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560dd2bf4350 0x560dd2bfe4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:35.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.248+0000 7f28543d9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560dd2bfea20 0x560dd2c00e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:35.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.248+0000 7f28543d9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x560dd2b591f0 con 0x560dd2bfea20 2024-04-24T08:54:35.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.248+0000 7f28543d9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560dd2aff350 con 0x560dd2bf4350 2024-04-24T08:54:35.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.248+0000 7f28543d9740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x560dd2b551b0 con 0x560dd2bf3a30 2024-04-24T08:54:35.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.248+0000 7f2849e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560dd2bfea20 0x560dd2c00e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:35.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.248+0000 7f2849e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560dd2bfea20 0x560dd2c00e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41906/0 (socket says 172.21.15.83:41906) 2024-04-24T08:54:35.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.248+0000 7f2849e00640 1 -- 172.21.15.83:0/62225409 learned_addr learned my addr 172.21.15.83:0/62225409 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:54:35.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.248+0000 7f284a800640 1 --2- 172.21.15.83:0/62225409 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560dd2bf4350 0x560dd2bfe4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:35.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.248+0000 7f2849e00640 1 --2- 172.21.15.83:0/62225409 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560dd2bf3a30 0x560dd2bf3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:35.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.249+0000 7f2849e00640 1 -- 172.21.15.83:0/62225409 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560dd2bf3a30 msgr2=0x560dd2bf3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:35.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.249+0000 7f2849e00640 1 --2- 172.21.15.83:0/62225409 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560dd2bf3a30 0x560dd2bf3e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:35.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.249+0000 7f2849e00640 1 -- 172.21.15.83:0/62225409 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560dd2bf4350 msgr2=0x560dd2bfe4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:35.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.249+0000 7f2849e00640 1 --2- 172.21.15.83:0/62225409 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560dd2bf4350 0x560dd2bfe4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:35.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.249+0000 7f2849e00640 1 -- 172.21.15.83:0/62225409 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560dd2b53f50 con 0x560dd2bfea20 2024-04-24T08:54:35.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.249+0000 7f2843400640 1 --2- 172.21.15.83:0/2292825320 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f282c0a3da0 0x7f282c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:35.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.249+0000 7f2849e00640 1 --2- 172.21.15.83:0/62225409 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560dd2bfea20 0x560dd2c00e10 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f283402ade0 tx=0x7f2834005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23e8dd038a6955a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:35.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.249+0000 7f2843400640 1 -- 172.21.15.83:0/2292825320 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f28440d0460 con 0x7f284c10e7f0 2024-04-24T08:54:35.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.249+0000 7f284a800640 1 --2- 172.21.15.83:0/2292825320 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f282c0a3da0 0x7f282c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:35.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.250+0000 7f283b400640 1 -- 172.21.15.83:0/62225409 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2834049070 con 0x560dd2bfea20 2024-04-24T08:54:35.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.250+0000 7f283b400640 1 -- 172.21.15.83:0/62225409 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2834042030 con 0x560dd2bfea20 2024-04-24T08:54:35.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.250+0000 7f283b400640 1 -- 172.21.15.83:0/62225409 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28340044d0 con 0x560dd2bfea20 2024-04-24T08:54:35.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.250+0000 7f28543d9740 1 -- 172.21.15.83:0/62225409 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560dd2bfea20 msgr2=0x560dd2c00e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:35.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.250+0000 7f28543d9740 1 --2- 172.21.15.83:0/62225409 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560dd2bfea20 0x560dd2c00e10 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f283402ade0 tx=0x7f2834005c50 comp rx=0 tx=0).stop 2024-04-24T08:54:35.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.250+0000 7f284a800640 1 --2- 172.21.15.83:0/2292825320 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f282c0a3da0 0x7f282c0a6260 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x560dd2bf4ff0 tx=0x7f283c00c4b0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:35.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.251+0000 7f28543d9740 1 -- 172.21.15.83:0/62225409 shutdown_connections 2024-04-24T08:54:35.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.251+0000 7f28543d9740 1 --2- 172.21.15.83:0/62225409 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560dd2bfea20 0x560dd2c00e10 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:35.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.251+0000 7f28543d9740 1 --2- 172.21.15.83:0/62225409 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560dd2bf4350 0x560dd2bfe4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:35.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.251+0000 7f28543d9740 1 --2- 172.21.15.83:0/62225409 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560dd2bf3a30 0x560dd2bf3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:35.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.251+0000 7f28543d9740 1 -- 172.21.15.83:0/62225409 >> 172.21.15.83:0/62225409 conn(0x560dd2befd50 msgr2=0x560dd2bf0130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:35.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.251+0000 7f28543d9740 1 -- 172.21.15.83:0/62225409 shutdown_connections 2024-04-24T08:54:35.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.251+0000 7f28543d9740 1 -- 172.21.15.83:0/62225409 wait complete. 2024-04-24T08:54:35.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.251+0000 7f28543d9740 1 -- start start 2024-04-24T08:54:35.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.251+0000 7f28543d9740 1 Processor -- start 2024-04-24T08:54:35.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.252+0000 7f28543d9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560dd2bf3a30 0x560dd2bfba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:35.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.252+0000 7f28543d9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560dd2bf4350 0x560dd2bfbf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:35.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.252+0000 7f28543d9740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560dd2bfea20 0x560dd2bfc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:35.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.252+0000 7f28543d9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x560dd2c017b0 con 0x560dd2bf4350 2024-04-24T08:54:35.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.252+0000 7f28543d9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560dd2b12690 con 0x560dd2bf3a30 2024-04-24T08:54:35.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.252+0000 7f28543d9740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x560dd2c01930 con 0x560dd2bfea20 2024-04-24T08:54:35.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.252+0000 7f28543d9740 10 client.0 _ll_register_callbacks cb 0x560dd2b4e650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:54:35.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.252+0000 7f2849e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560dd2bf4350 0x560dd2bfbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:35.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.252+0000 7f2849e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560dd2bf4350 0x560dd2bfbf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41918/0 (socket says 172.21.15.83:41918) 2024-04-24T08:54:35.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.252+0000 7f2849e00640 1 -- 172.21.15.83:0/3565058015 learned_addr learned my addr 172.21.15.83:0/3565058015 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:54:35.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.253+0000 7f284b200640 1 --2- 172.21.15.83:0/3565058015 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560dd2bfea20 0x560dd2bfc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:35.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.253+0000 7f284a800640 1 --2- 172.21.15.83:0/3565058015 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560dd2bf3a30 0x560dd2bfba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:35.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.253+0000 7f2849e00640 1 -- 172.21.15.83:0/3565058015 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560dd2bfea20 msgr2=0x560dd2bfc4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:35.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.253+0000 7f2849e00640 1 --2- 172.21.15.83:0/3565058015 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560dd2bfea20 0x560dd2bfc4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:35.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.253+0000 7f2849e00640 1 -- 172.21.15.83:0/3565058015 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560dd2bf3a30 msgr2=0x560dd2bfba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:35.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.253+0000 7f2849e00640 1 --2- 172.21.15.83:0/3565058015 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560dd2bf3a30 0x560dd2bfba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:35.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.253+0000 7f2849e00640 1 -- 172.21.15.83:0/3565058015 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560dd2b11920 con 0x560dd2bf4350 2024-04-24T08:54:35.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.253+0000 7f284a800640 1 --2- 172.21.15.83:0/3565058015 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560dd2bf3a30 0x560dd2bfba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:54:35.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.253+0000 7f2849e00640 1 --2- 172.21.15.83:0/3565058015 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560dd2bf4350 0x560dd2bfbf80 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f2834005cc0 tx=0x7f2834031f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1a5d27914a7409 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:35.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.253+0000 7f2833e00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:54:35.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.253+0000 7f2833e00640 1 -- 172.21.15.83:0/3565058015 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2834052060 con 0x560dd2bf4350 2024-04-24T08:54:35.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.253+0000 7f2833e00640 1 -- 172.21.15.83:0/3565058015 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f2834003b70 con 0x560dd2bf4350 2024-04-24T08:54:35.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.253+0000 7f28543d9740 1 -- 172.21.15.83:0/3565058015 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2834042030 con 0x560dd2bf4350 2024-04-24T08:54:35.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.254+0000 7f2833e00640 1 -- 172.21.15.83:0/3565058015 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2834059050 con 0x560dd2bf4350 2024-04-24T08:54:35.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.254+0000 7f28543d9740 10 client.16803 fetch_fsmap learned FSMap version 30 2024-04-24T08:54:35.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.254+0000 7f28543d9740 1 -- 172.21.15.83:0/3565058015 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560dd29357d0 con 0x560dd2bf4350 2024-04-24T08:54:35.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.254+0000 7f2833e00640 1 -- 172.21.15.83:0/3565058015 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2834042030 con 0x560dd2bf4350 2024-04-24T08:54:35.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.254+0000 7f2833e00640 1 -- 172.21.15.83:0/3565058015 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7f283404c070 con 0x560dd2bf4350 2024-04-24T08:54:35.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.254+0000 7f28543d9740 10 client.16803 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T08:54:35.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.254+0000 7f28543d9740 10 client.16803 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:54:35.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.254+0000 7f28543d9740 20 client.16803 populate_metadata read hostname 'smithi083' 2024-04-24T08:54:35.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.254+0000 7f28543d9740 1 --2- 172.21.15.83:0/3565058015 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x560dd2c0ec60 0x560dd2c11050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:35.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.254+0000 7f28543d9740 4 client.16803 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:54:35.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.254+0000 7f28543d9740 1 -- 172.21.15.83:0/3565058015 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560dd2b66c10 con 0x560dd2c0ec60 2024-04-24T08:54:35.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.255+0000 7f284a800640 1 --2- 172.21.15.83:0/3565058015 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x560dd2c0ec60 0x560dd2c11050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:35.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.257+0000 7f284a800640 1 --2- 172.21.15.83:0/3565058015 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x560dd2c0ec60 0x560dd2c11050 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5dcd991352c5e94 server_cookie=bfebe67d3129005 in_seq=0 out_seq=0 2024-04-24T08:54:35.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.257+0000 7f2833e00640 10 client.16803 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:54:35.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.259+0000 7f2833e00640 1 -- 172.21.15.83:0/3565058015 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560dd2b66c10 con 0x560dd2c0ec60 2024-04-24T08:54:35.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.259+0000 7f2833e00640 10 client.16803 handle_command_reply: tid=0 2024-04-24T08:54:35.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.259+0000 7f28543d9740 1 client.16803 shutdown 2024-04-24T08:54:35.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.259+0000 7f28543d9740 1 -- 172.21.15.83:0/3565058015 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560dd2bf4350 msgr2=0x560dd2bfbf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:35.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.259+0000 7f28543d9740 1 --2- 172.21.15.83:0/3565058015 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560dd2bf4350 0x560dd2bfbf80 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f2834005cc0 tx=0x7f2834031f00 comp rx=0 tx=0).stop 2024-04-24T08:54:35.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.259+0000 7f28543d9740 1 -- 172.21.15.83:0/3565058015 shutdown_connections 2024-04-24T08:54:35.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.259+0000 7f28543d9740 1 --2- 172.21.15.83:0/3565058015 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x560dd2c0ec60 0x560dd2c11050 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:35.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.259+0000 7f28543d9740 1 --2- 172.21.15.83:0/3565058015 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560dd2bfea20 0x560dd2bfc4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:35.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.259+0000 7f28543d9740 1 --2- 172.21.15.83:0/3565058015 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560dd2bf4350 0x560dd2bfbf80 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:35.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.259+0000 7f28543d9740 1 --2- 172.21.15.83:0/3565058015 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560dd2bf3a30 0x560dd2bfba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:35.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.259+0000 7f2833e00640 0 client.16803 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:54:35.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.259+0000 7f28543d9740 1 -- 172.21.15.83:0/3565058015 >> 172.21.15.83:0/3565058015 conn(0x560dd2befd50 msgr2=0x560dd2bf5b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:35.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.259+0000 7f28543d9740 1 -- 172.21.15.83:0/3565058015 shutdown_connections 2024-04-24T08:54:35.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.259+0000 7f28543d9740 1 -- 172.21.15.83:0/3565058015 wait complete. 2024-04-24T08:54:35.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.259+0000 7f28543d9740 20 client.16803 trim_cache size 0 max 16384 2024-04-24T08:54:35.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.278+0000 7f28543d9740 1 Processor -- start 2024-04-24T08:54:35.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.279+0000 7f28543d9740 1 -- start start 2024-04-24T08:54:35.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.279+0000 7f28543d9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560dd2bf3a30 0x560dd2c0a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:35.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.279+0000 7f28543d9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560dd2bf4350 0x560dd2beae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:35.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.279+0000 7f28543d9740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560dd2bfea20 0x560dd2bf61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:35.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.279+0000 7f28543d9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f284c1b5800 con 0x560dd2bf4350 2024-04-24T08:54:35.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.279+0000 7f28543d9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f284c1b73d0 con 0x560dd2bf3a30 2024-04-24T08:54:35.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.279+0000 7f28543d9740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f284c1b5ed0 con 0x560dd2bfea20 2024-04-24T08:54:35.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.279+0000 7f2849e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560dd2bf4350 0x560dd2beae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:35.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.279+0000 7f2849e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560dd2bf4350 0x560dd2beae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41920/0 (socket says 172.21.15.83:41920) 2024-04-24T08:54:35.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.279+0000 7f2849e00640 1 -- 172.21.15.83:0/3723695353 learned_addr learned my addr 172.21.15.83:0/3723695353 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:54:35.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.279+0000 7f284a800640 1 --2- 172.21.15.83:0/3723695353 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560dd2bf3a30 0x560dd2c0a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:35.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.280+0000 7f284b200640 1 --2- 172.21.15.83:0/3723695353 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560dd2bfea20 0x560dd2bf61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:35.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.280+0000 7f2849e00640 1 -- 172.21.15.83:0/3723695353 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560dd2bfea20 msgr2=0x560dd2bf61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:35.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.280+0000 7f2849e00640 1 --2- 172.21.15.83:0/3723695353 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560dd2bfea20 0x560dd2bf61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:35.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.280+0000 7f2849e00640 1 -- 172.21.15.83:0/3723695353 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560dd2bf3a30 msgr2=0x560dd2c0a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:35.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.280+0000 7f2849e00640 1 --2- 172.21.15.83:0/3723695353 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560dd2bf3a30 0x560dd2c0a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:35.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.280+0000 7f2849e00640 1 -- 172.21.15.83:0/3723695353 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560dd2bfa9f0 con 0x560dd2bf4350 2024-04-24T08:54:35.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.280+0000 7f2849e00640 1 --2- 172.21.15.83:0/3723695353 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560dd2bf4350 0x560dd2beae60 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f28340307e0 tx=0x7f2834031940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39c9c889f5f35477 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:35.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.280+0000 7f283aa00640 1 -- 172.21.15.83:0/3723695353 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2834052070 con 0x560dd2bf4350 2024-04-24T08:54:35.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.280+0000 7f283aa00640 1 -- 172.21.15.83:0/3723695353 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2834042040 con 0x560dd2bf4350 2024-04-24T08:54:35.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.280+0000 7f283aa00640 1 -- 172.21.15.83:0/3723695353 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2834049040 con 0x560dd2bf4350 2024-04-24T08:54:35.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.281+0000 7f28543d9740 1 -- 172.21.15.83:0/3723695353 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560dd2bf4350 msgr2=0x560dd2beae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:35.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.281+0000 7f28543d9740 1 --2- 172.21.15.83:0/3723695353 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560dd2bf4350 0x560dd2beae60 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f28340307e0 tx=0x7f2834031940 comp rx=0 tx=0).stop 2024-04-24T08:54:35.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.281+0000 7f28543d9740 1 -- 172.21.15.83:0/3723695353 shutdown_connections 2024-04-24T08:54:35.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.281+0000 7f28543d9740 1 --2- 172.21.15.83:0/3723695353 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560dd2bfea20 0x560dd2bf61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:35.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.281+0000 7f28543d9740 1 --2- 172.21.15.83:0/3723695353 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560dd2bf4350 0x560dd2beae60 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:35.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.281+0000 7f28543d9740 1 --2- 172.21.15.83:0/3723695353 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560dd2bf3a30 0x560dd2c0a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:35.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.281+0000 7f28543d9740 1 -- 172.21.15.83:0/3723695353 >> 172.21.15.83:0/3723695353 conn(0x560dd2befd50 msgr2=0x560dd2bfc290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:35.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.281+0000 7f28543d9740 1 -- 172.21.15.83:0/3723695353 shutdown_connections 2024-04-24T08:54:35.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.281+0000 7f28543d9740 1 -- 172.21.15.83:0/3723695353 wait complete. 2024-04-24T08:54:35.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.282+0000 7f28543d9740 1 -- start start 2024-04-24T08:54:35.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.282+0000 7f28543d9740 1 Processor -- start 2024-04-24T08:54:35.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.282+0000 7f28543d9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560dd2bf3a30 0x560dd2c026e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:35.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.282+0000 7f28543d9740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560dd2bf4350 0x560dd2c02c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:35.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.282+0000 7f28543d9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560dd2bfea20 0x560dd2c03160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:35.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.282+0000 7f28543d9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f284c1b6c70 con 0x560dd2bf3a30 2024-04-24T08:54:35.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.282+0000 7f28543d9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f284c1b65a0 con 0x560dd2bfea20 2024-04-24T08:54:35.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.282+0000 7f28543d9740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f284c1b37e0 con 0x560dd2bf4350 2024-04-24T08:54:35.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.282+0000 7f28543d9740 10 client.0 _ll_register_callbacks cb 0x7f2834047070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:54:35.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.282+0000 7f284a800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560dd2bf3a30 0x560dd2c026e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:35.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.283+0000 7f284a800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560dd2bf3a30 0x560dd2c026e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41926/0 (socket says 172.21.15.83:41926) 2024-04-24T08:54:35.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.283+0000 7f284a800640 1 -- 172.21.15.83:0/466036268 learned_addr learned my addr 172.21.15.83:0/466036268 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:54:35.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.283+0000 7f2849e00640 1 --2- 172.21.15.83:0/466036268 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560dd2bf4350 0x560dd2c02c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:35.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.283+0000 7f284b200640 1 --2- 172.21.15.83:0/466036268 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560dd2bfea20 0x560dd2c03160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:35.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.283+0000 7f284a800640 1 -- 172.21.15.83:0/466036268 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560dd2bf4350 msgr2=0x560dd2c02c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:35.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.283+0000 7f284a800640 1 --2- 172.21.15.83:0/466036268 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560dd2bf4350 0x560dd2c02c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:35.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.283+0000 7f284a800640 1 -- 172.21.15.83:0/466036268 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560dd2bfea20 msgr2=0x560dd2c03160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:35.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.283+0000 7f284a800640 1 --2- 172.21.15.83:0/466036268 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560dd2bfea20 0x560dd2c03160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:35.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.283+0000 7f284a800640 1 -- 172.21.15.83:0/466036268 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560dd2bf4180 con 0x560dd2bf3a30 2024-04-24T08:54:35.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.283+0000 7f284b200640 1 --2- 172.21.15.83:0/466036268 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560dd2bfea20 0x560dd2c03160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:54:35.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.283+0000 7f284a800640 1 --2- 172.21.15.83:0/466036268 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560dd2bf3a30 0x560dd2c026e0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x560dd2c11590 tx=0x7f283c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea81809c0e42c3c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:35.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.283+0000 7f2833e00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:54:35.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.283+0000 7f2833e00640 1 -- 172.21.15.83:0/466036268 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f283c027620 con 0x560dd2bf3a30 2024-04-24T08:54:35.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.283+0000 7f2833e00640 1 -- 172.21.15.83:0/466036268 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f283c0277c0 con 0x560dd2bf3a30 2024-04-24T08:54:35.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.283+0000 7f28543d9740 1 -- 172.21.15.83:0/466036268 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2834042040 con 0x560dd2bf3a30 2024-04-24T08:54:35.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.284+0000 7f2833e00640 1 -- 172.21.15.83:0/466036268 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f283c037050 con 0x560dd2bf3a30 2024-04-24T08:54:35.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.284+0000 7f28543d9740 10 client.16809 fetch_fsmap learned FSMap version 30 2024-04-24T08:54:35.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.284+0000 7f28543d9740 1 -- 172.21.15.83:0/466036268 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560dd286b3f0 con 0x560dd2bf3a30 2024-04-24T08:54:35.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.284+0000 7f2833e00640 1 -- 172.21.15.83:0/466036268 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2834042040 con 0x560dd2bf3a30 2024-04-24T08:54:35.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.284+0000 7f2833e00640 1 -- 172.21.15.83:0/466036268 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7f283c018040 con 0x560dd2bf3a30 2024-04-24T08:54:35.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.284+0000 7f28543d9740 10 client.16809 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T08:54:35.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.284+0000 7f28543d9740 10 client.16809 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:54:35.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.284+0000 7f28543d9740 20 client.16809 populate_metadata read hostname 'smithi083' 2024-04-24T08:54:35.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.284+0000 7f28543d9740 1 --2- 172.21.15.83:0/466036268 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x560dd2c75730 0x560dd2c77b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:35.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.284+0000 7f28543d9740 4 client.16809 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T08:54:35.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.284+0000 7f28543d9740 1 -- 172.21.15.83:0/466036268 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560dd2b60a80 con 0x560dd2c75730 2024-04-24T08:54:35.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.285+0000 7f2849e00640 1 --2- 172.21.15.83:0/466036268 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x560dd2c75730 0x560dd2c77b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:35.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.286+0000 7f2849e00640 1 --2- 172.21.15.83:0/466036268 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x560dd2c75730 0x560dd2c77b50 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4437d835cbc92c89 server_cookie=48f876d0bca05b37 in_seq=0 out_seq=0 2024-04-24T08:54:35.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.286+0000 7f2833e00640 10 client.16809 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:54:35.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.286+0000 7f2833e00640 1 -- 172.21.15.83:0/466036268 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x560dd2b60a80 con 0x560dd2c75730 2024-04-24T08:54:35.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.286+0000 7f2833e00640 10 client.16809 handle_command_reply: tid=0 2024-04-24T08:54:35.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.286+0000 7f28543d9740 1 client.16809 shutdown 2024-04-24T08:54:35.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.287+0000 7f28543d9740 1 -- 172.21.15.83:0/466036268 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560dd2bf3a30 msgr2=0x560dd2c026e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:35.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.287+0000 7f28543d9740 1 --2- 172.21.15.83:0/466036268 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560dd2bf3a30 0x560dd2c026e0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x560dd2c11590 tx=0x7f283c0046c0 comp rx=0 tx=0).stop 2024-04-24T08:54:35.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.287+0000 7f28543d9740 1 -- 172.21.15.83:0/466036268 shutdown_connections 2024-04-24T08:54:35.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.287+0000 7f28543d9740 1 --2- 172.21.15.83:0/466036268 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x560dd2c75730 0x560dd2c77b50 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:35.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.287+0000 7f28543d9740 1 --2- 172.21.15.83:0/466036268 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560dd2bfea20 0x560dd2c03160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:35.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.287+0000 7f28543d9740 1 --2- 172.21.15.83:0/466036268 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560dd2bf4350 0x560dd2c02c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:35.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.287+0000 7f28543d9740 1 --2- 172.21.15.83:0/466036268 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560dd2bf3a30 0x560dd2c026e0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:35.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.287+0000 7f2833e00640 0 client.16809 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:54:35.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.287+0000 7f28543d9740 1 -- 172.21.15.83:0/466036268 >> 172.21.15.83:0/466036268 conn(0x560dd2befd50 msgr2=0x560dd2bee740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:35.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.287+0000 7f28543d9740 1 -- 172.21.15.83:0/466036268 shutdown_connections 2024-04-24T08:54:35.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.287+0000 7f28543d9740 1 -- 172.21.15.83:0/466036268 wait complete. 2024-04-24T08:54:35.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.287+0000 7f28543d9740 20 client.16809 trim_cache size 0 max 16384 2024-04-24T08:54:35.290 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T08:54:35.290 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T08:54:35.290 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-04-24T08:54:35.290 INFO:teuthology.orchestra.run.smithi083.stdout: "0347b3ec-c9af-4549-9480-e0a992eb1783": { 2024-04-24T08:54:35.290 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-04-24T08:54:35.290 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "0347b3ec-c9af-4549-9480-e0a992eb1783", 2024-04-24T08:54:35.291 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-04-24T08:54:35.291 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T08:54:35.291 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T08:54:35.291 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T08:54:35.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.288+0000 7f2850a00640 1 -- 172.21.15.83:0/2292825320 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f282c0a3da0 msgr2=0x7f282c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:35.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.288+0000 7f2850a00640 1 --2- 172.21.15.83:0/2292825320 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f282c0a3da0 0x7f282c0a6260 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x560dd2bf4ff0 tx=0x7f283c00c4b0 comp rx=0 tx=0).stop 2024-04-24T08:54:35.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.288+0000 7f2850a00640 1 -- 172.21.15.83:0/2292825320 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f284c10e7f0 msgr2=0x7f284c1039a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:35.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.288+0000 7f2850a00640 1 --2- 172.21.15.83:0/2292825320 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f284c10e7f0 0x7f284c1039a0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f284400d720 tx=0x7f284400dbe0 comp rx=0 tx=0).stop 2024-04-24T08:54:35.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.288+0000 7f2850a00640 1 -- 172.21.15.83:0/2292825320 shutdown_connections 2024-04-24T08:54:35.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.288+0000 7f2850a00640 1 --2- 172.21.15.83:0/2292825320 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f282c0a3da0 0x7f282c0a6260 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:35.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.288+0000 7f2850a00640 1 --2- 172.21.15.83:0/2292825320 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f284c10e7f0 0x7f284c1039a0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:35.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.288+0000 7f2850a00640 1 --2- 172.21.15.83:0/2292825320 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f284c1015f0 0x7f284c103460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:35.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.288+0000 7f2850a00640 1 --2- 172.21.15.83:0/2292825320 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f284c100c40 0x7f284c102f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:35.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.288+0000 7f2850a00640 1 -- 172.21.15.83:0/2292825320 >> 172.21.15.83:0/2292825320 conn(0x7f284c0fc880 msgr2=0x7f284c0fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:35.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.289+0000 7f2850a00640 1 -- 172.21.15.83:0/2292825320 shutdown_connections 2024-04-24T08:54:35.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:54:35.289+0000 7f2850a00640 1 -- 172.21.15.83:0/2292825320 wait complete. 2024-04-24T08:54:35.308 INFO:tasks.cephfs.filesystem:scrub status for tag:0347b3ec-c9af-4549-9480-e0a992eb1783 - {'path': '/', 'tag': '0347b3ec-c9af-4549-9480-e0a992eb1783', 'options': 'recursive,force'} 2024-04-24T08:54:35.561 INFO:tasks.workunit.client.0.smithi083.stdout: 1 8866 0.93 MB/sec warmup 69 sec latency 52.521 ms 2024-04-24T08:54:36.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:54:35 smithi083 ceph-mon[27907]: pgmap v92: 129 pgs: 129 active+clean; 28 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 258 KiB/s wr, 54 op/s 2024-04-24T08:54:36.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:54:35 smithi083 ceph-mon[27907]: scrub queued for path: / 2024-04-24T08:54:36.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:54:35 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T08:54:36.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:54:35 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T08:54:36.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:54:35 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T08:54:36.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:54:35 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T08:54:36.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:54:35 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T08:54:36.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:54:35 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T08:54:36.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:54:35 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T08:54:36.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:54:35 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T08:54:36.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:54:35 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T08:54:36.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:54:35 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T08:54:36.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:54:35 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T08:54:36.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:54:35 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T08:54:36.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:54:35 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T08:54:36.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:54:35 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T08:54:36.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:54:35 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T08:54:36.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:54:35 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T08:54:36.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:54:35 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T08:54:36.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:54:35 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T08:54:36.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:54:35 smithi138 ceph-mon[31441]: pgmap v92: 129 pgs: 129 active+clean; 28 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 258 KiB/s wr, 54 op/s 2024-04-24T08:54:36.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:54:35 smithi138 ceph-mon[31441]: scrub queued for path: / 2024-04-24T08:54:36.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:54:35 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T08:54:36.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:54:35 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T08:54:36.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:54:35 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T08:54:36.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:54:35 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T08:54:36.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:54:35 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T08:54:36.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:54:35 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T08:54:36.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:54:35 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T08:54:36.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:54:35 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T08:54:36.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:54:35 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T08:54:36.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:54:35 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T08:54:36.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:54:35 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T08:54:36.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:54:35 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T08:54:36.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:54:35 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T08:54:36.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:54:35 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T08:54:36.207 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:54:35 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T08:54:36.207 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:54:35 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T08:54:36.207 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:54:35 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T08:54:36.207 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:54:35 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T08:54:36.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:54:35 smithi189 ceph-mon[31791]: pgmap v92: 129 pgs: 129 active+clean; 28 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 258 KiB/s wr, 54 op/s 2024-04-24T08:54:36.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:54:35 smithi189 ceph-mon[31791]: scrub queued for path: / 2024-04-24T08:54:36.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:54:35 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T08:54:36.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:54:35 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T08:54:36.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:54:35 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T08:54:36.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:54:35 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T08:54:36.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:54:35 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T08:54:36.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:54:35 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T08:54:36.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:54:35 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T08:54:36.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:54:35 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T08:54:36.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:54:35 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T08:54:36.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:54:35 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T08:54:36.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:54:35 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T08:54:36.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:54:35 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T08:54:36.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:54:35 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T08:54:36.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:54:35 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T08:54:36.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:54:35 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T08:54:36.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:54:35 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T08:54:36.209 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:54:35 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T08:54:36.209 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:54:35 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T08:54:36.561 INFO:tasks.workunit.client.0.smithi083.stdout: 1 8976 0.92 MB/sec warmup 70 sec latency 54.278 ms 2024-04-24T08:54:37.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:54:36 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T08:54:37.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:54:36 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T08:54:37.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:54:36 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T08:54:37.561 INFO:tasks.workunit.client.0.smithi083.stdout: 1 9139 0.91 MB/sec warmup 71 sec latency 52.523 ms 2024-04-24T08:54:38.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:54:37 smithi083 ceph-mon[27907]: pgmap v93: 129 pgs: 129 active+clean; 28 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 258 KiB/s wr, 54 op/s 2024-04-24T08:54:38.106 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:54:37 smithi138 ceph-mon[31441]: pgmap v93: 129 pgs: 129 active+clean; 28 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 258 KiB/s wr, 54 op/s 2024-04-24T08:54:38.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:54:37 smithi189 ceph-mon[31791]: pgmap v93: 129 pgs: 129 active+clean; 28 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 258 KiB/s wr, 54 op/s 2024-04-24T08:54:38.561 INFO:tasks.workunit.client.0.smithi083.stdout: 1 9256 0.90 MB/sec warmup 72 sec latency 55.910 ms 2024-04-24T08:54:39.562 INFO:tasks.workunit.client.0.smithi083.stdout: 1 9411 0.90 MB/sec warmup 73 sec latency 54.809 ms 2024-04-24T08:54:40.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:54:39 smithi083 ceph-mon[27907]: pgmap v94: 129 pgs: 129 active+clean; 29 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 311 KiB/s wr, 83 op/s 2024-04-24T08:54:40.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:54:39 smithi138 ceph-mon[31441]: pgmap v94: 129 pgs: 129 active+clean; 29 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 311 KiB/s wr, 83 op/s 2024-04-24T08:54:40.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:54:39 smithi189 ceph-mon[31791]: pgmap v94: 129 pgs: 129 active+clean; 29 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 311 KiB/s wr, 83 op/s 2024-04-24T08:54:40.562 INFO:tasks.workunit.client.0.smithi083.stdout: 1 9498 0.89 MB/sec warmup 74 sec latency 53.694 ms 2024-04-24T08:54:40.751 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.751+0000 7f0563200640 1 -- 172.21.15.83:0/2903762265 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f055c104e20 msgr2=0x7f055c108e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:40.751 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.751+0000 7f0563200640 1 --2- 172.21.15.83:0/2903762265 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f055c104e20 0x7f055c108e60 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f0548009900 tx=0x7f054802a3f0 comp rx=0 tx=0).stop 2024-04-24T08:54:40.751 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.751+0000 7f0563200640 1 -- 172.21.15.83:0/2903762265 shutdown_connections 2024-04-24T08:54:40.751 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.751+0000 7f0563200640 1 --2- 172.21.15.83:0/2903762265 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f055c109540 0x7f055c111110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:40.751 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.751+0000 7f0563200640 1 --2- 172.21.15.83:0/2903762265 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f055c104e20 0x7f055c108e60 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:40.751 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.751+0000 7f0563200640 1 --2- 172.21.15.83:0/2903762265 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f055c104470 0x7f055c104850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:40.752 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.751+0000 7f0563200640 1 -- 172.21.15.83:0/2903762265 >> 172.21.15.83:0/2903762265 conn(0x7f055c0ffd30 msgr2=0x7f055c102150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:40.752 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.751+0000 7f0563200640 1 -- 172.21.15.83:0/2903762265 shutdown_connections 2024-04-24T08:54:40.752 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.751+0000 7f0563200640 1 -- 172.21.15.83:0/2903762265 wait complete. 2024-04-24T08:54:40.754 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.752+0000 7f0563200640 1 Processor -- start 2024-04-24T08:54:40.754 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.753+0000 7f0563200640 1 -- start start 2024-04-24T08:54:40.754 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.753+0000 7f0563200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f055c104470 0x7f055c1a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:40.754 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.753+0000 7f0563200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f055c109540 0x7f055c1a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:40.754 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.753+0000 7f0563200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f055c19a770 0x7f055c19abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:40.754 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.753+0000 7f0563200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f055c1144b0 con 0x7f055c19a770 2024-04-24T08:54:40.754 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.753+0000 7f0563200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f055c114330 con 0x7f055c109540 2024-04-24T08:54:40.754 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.753+0000 7f0563200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f055c114630 con 0x7f055c104470 2024-04-24T08:54:40.754 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.753+0000 7f0561200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f055c19a770 0x7f055c19abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:40.755 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.753+0000 7f0561200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f055c19a770 0x7f055c19abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42800/0 (socket says 172.21.15.83:42800) 2024-04-24T08:54:40.755 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.753+0000 7f0561200640 1 -- 172.21.15.83:0/2720971811 learned_addr learned my addr 172.21.15.83:0/2720971811 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:54:40.755 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.754+0000 7f055be00640 1 --2- 172.21.15.83:0/2720971811 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f055c104470 0x7f055c1a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:40.755 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.754+0000 7f055b400640 1 --2- 172.21.15.83:0/2720971811 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f055c109540 0x7f055c1a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:40.755 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.754+0000 7f0561200640 1 -- 172.21.15.83:0/2720971811 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f055c104470 msgr2=0x7f055c1a0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:40.755 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.754+0000 7f0561200640 1 --2- 172.21.15.83:0/2720971811 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f055c104470 0x7f055c1a0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:40.755 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.754+0000 7f0561200640 1 -- 172.21.15.83:0/2720971811 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f055c109540 msgr2=0x7f055c1a0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:40.755 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.754+0000 7f0561200640 1 --2- 172.21.15.83:0/2720971811 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f055c109540 0x7f055c1a0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:40.755 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.754+0000 7f0561200640 1 -- 172.21.15.83:0/2720971811 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f055c19b370 con 0x7f055c19a770 2024-04-24T08:54:40.755 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.754+0000 7f055b400640 1 --2- 172.21.15.83:0/2720971811 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f055c109540 0x7f055c1a0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:54:40.755 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.754+0000 7f0561200640 1 --2- 172.21.15.83:0/2720971811 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f055c19a770 0x7f055c19abf0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f055400eaa0 tx=0x7f055400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:40.755 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.755+0000 7f0558c00640 1 -- 172.21.15.83:0/2720971811 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f055400cdb0 con 0x7f055c19a770 2024-04-24T08:54:40.755 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.755+0000 7f0558c00640 1 -- 172.21.15.83:0/2720971811 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0554005020 con 0x7f055c19a770 2024-04-24T08:54:40.755 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.755+0000 7f0563200640 1 -- 172.21.15.83:0/2720971811 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f055c1a7400 con 0x7f055c19a770 2024-04-24T08:54:40.755 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.755+0000 7f0558c00640 1 -- 172.21.15.83:0/2720971811 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05540116d0 con 0x7f055c19a770 2024-04-24T08:54:40.756 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.755+0000 7f0563200640 1 -- 172.21.15.83:0/2720971811 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f055c1a77b0 con 0x7f055c19a770 2024-04-24T08:54:40.757 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.758+0000 7f056607c740 1 Processor -- start 2024-04-24T08:54:40.757 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.758+0000 7f0558c00640 1 -- 172.21.15.83:0/2720971811 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f05540051d0 con 0x7f055c19a770 2024-04-24T08:54:40.757 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.758+0000 7f056607c740 1 -- start start 2024-04-24T08:54:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.759+0000 7f056607c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588ad9151b0 0x5588ad915590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.759+0000 7f056607c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588ad915ad0 0x5588ad91fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.759+0000 7f056607c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5588ad920390 0x5588ad922780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.759+0000 7f056607c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5588ad87b900 con 0x5588ad9151b0 2024-04-24T08:54:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.759+0000 7f056607c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5588ad821f20 con 0x5588ad920390 2024-04-24T08:54:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.759+0000 7f056607c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5588ad8778c0 con 0x5588ad915ad0 2024-04-24T08:54:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.759+0000 7f055b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588ad9151b0 0x5588ad915590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.759+0000 7f055b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588ad9151b0 0x5588ad915590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42804/0 (socket says 172.21.15.83:42804) 2024-04-24T08:54:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.759+0000 7f055b400640 1 -- 172.21.15.83:0/1229741231 learned_addr learned my addr 172.21.15.83:0/1229741231 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:54:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.759+0000 7f055be00640 1 --2- 172.21.15.83:0/1229741231 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588ad915ad0 0x5588ad91fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.760+0000 7f055b400640 1 -- 172.21.15.83:0/1229741231 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588ad915ad0 msgr2=0x5588ad91fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.760+0000 7f055b400640 1 --2- 172.21.15.83:0/1229741231 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588ad915ad0 0x5588ad91fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.760+0000 7f055b400640 1 -- 172.21.15.83:0/1229741231 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5588ad920390 msgr2=0x5588ad922780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:40.760 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.760+0000 7f055b400640 1 --2- 172.21.15.83:0/1229741231 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5588ad920390 0x5588ad922780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:40.760 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.760+0000 7f055b400640 1 -- 172.21.15.83:0/1229741231 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588ad846e10 con 0x5588ad9151b0 2024-04-24T08:54:40.760 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.760+0000 7f0558c00640 1 --2- 172.21.15.83:0/2720971811 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f05400a3dc0 0x7f05400a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:40.764 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.760+0000 7f055b400640 1 --2- 172.21.15.83:0/1229741231 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588ad9151b0 0x5588ad915590 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f05480096c0 tx=0x7f0548005e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fae5dde2a701201d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:40.764 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.760+0000 7f0558c00640 1 -- 172.21.15.83:0/2720971811 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f05540d0e80 con 0x7f055c19a770 2024-04-24T08:54:40.764 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.760+0000 7f055be00640 1 --2- 172.21.15.83:0/2720971811 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f05400a3dc0 0x7f05400a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:40.764 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.760+0000 7f0551600640 1 -- 172.21.15.83:0/1229741231 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0548040020 con 0x5588ad9151b0 2024-04-24T08:54:40.764 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.760+0000 7f0551600640 1 -- 172.21.15.83:0/1229741231 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f05480044d0 con 0x5588ad9151b0 2024-04-24T08:54:40.764 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.760+0000 7f0551600640 1 -- 172.21.15.83:0/1229741231 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0548035070 con 0x5588ad9151b0 2024-04-24T08:54:40.764 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.761+0000 7f056607c740 1 -- 172.21.15.83:0/1229741231 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588ad9151b0 msgr2=0x5588ad915590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:40.764 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.761+0000 7f055be00640 1 --2- 172.21.15.83:0/2720971811 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f05400a3dc0 0x7f05400a6280 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f054c0058c0 tx=0x7f054c005850 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:40.764 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.761+0000 7f056607c740 1 --2- 172.21.15.83:0/1229741231 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588ad9151b0 0x5588ad915590 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f05480096c0 tx=0x7f0548005e30 comp rx=0 tx=0).stop 2024-04-24T08:54:40.764 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.761+0000 7f056607c740 1 -- 172.21.15.83:0/1229741231 shutdown_connections 2024-04-24T08:54:40.765 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.761+0000 7f056607c740 1 --2- 172.21.15.83:0/1229741231 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5588ad920390 0x5588ad922780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:40.765 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.761+0000 7f056607c740 1 --2- 172.21.15.83:0/1229741231 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588ad915ad0 0x5588ad91fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:40.765 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.761+0000 7f056607c740 1 --2- 172.21.15.83:0/1229741231 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588ad9151b0 0x5588ad915590 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:40.765 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.761+0000 7f056607c740 1 -- 172.21.15.83:0/1229741231 >> 172.21.15.83:0/1229741231 conn(0x5588ad9116c0 msgr2=0x5588ad911aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:40.765 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.761+0000 7f056607c740 1 -- 172.21.15.83:0/1229741231 shutdown_connections 2024-04-24T08:54:40.765 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.761+0000 7f056607c740 1 -- 172.21.15.83:0/1229741231 wait complete. 2024-04-24T08:54:40.765 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.762+0000 7f056607c740 1 -- start start 2024-04-24T08:54:40.765 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.762+0000 7f056607c740 1 Processor -- start 2024-04-24T08:54:40.765 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.762+0000 7f056607c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588ad9151b0 0x5588ad91d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:40.765 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.762+0000 7f056607c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5588ad915ad0 0x5588ad91d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:40.765 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.763+0000 7f056607c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588ad920390 0x5588ad91ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:40.765 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.763+0000 7f056607c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5588ad922f60 con 0x5588ad920390 2024-04-24T08:54:40.765 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.763+0000 7f056607c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5588ad8352c0 con 0x5588ad915ad0 2024-04-24T08:54:40.765 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.763+0000 7f056607c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5588ad9230e0 con 0x5588ad9151b0 2024-04-24T08:54:40.765 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.763+0000 7f056607c740 10 client.0 _ll_register_callbacks cb 0x5588ad58da00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:54:40.766 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.763+0000 7f0561200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588ad920390 0x5588ad91ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:40.766 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.763+0000 7f0561200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588ad920390 0x5588ad91ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42818/0 (socket says 172.21.15.83:42818) 2024-04-24T08:54:40.766 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.763+0000 7f0561200640 1 -- 172.21.15.83:0/1960585810 learned_addr learned my addr 172.21.15.83:0/1960585810 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:54:40.766 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.763+0000 7f055b400640 1 --2- 172.21.15.83:0/1960585810 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5588ad915ad0 0x5588ad91d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:40.766 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.763+0000 7f055be00640 1 --2- 172.21.15.83:0/1960585810 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588ad9151b0 0x5588ad91d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:40.766 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.763+0000 7f0561200640 1 -- 172.21.15.83:0/1960585810 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588ad9151b0 msgr2=0x5588ad91d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:40.766 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.763+0000 7f0561200640 1 --2- 172.21.15.83:0/1960585810 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588ad9151b0 0x5588ad91d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:40.766 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.763+0000 7f0561200640 1 -- 172.21.15.83:0/1960585810 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5588ad915ad0 msgr2=0x5588ad91d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:40.766 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.763+0000 7f0561200640 1 --2- 172.21.15.83:0/1960585810 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5588ad915ad0 0x5588ad91d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:40.766 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.763+0000 7f0561200640 1 -- 172.21.15.83:0/1960585810 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588ad845fd0 con 0x5588ad920390 2024-04-24T08:54:40.766 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.763+0000 7f055b400640 1 --2- 172.21.15.83:0/1960585810 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5588ad915ad0 0x5588ad91d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:54:40.766 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.764+0000 7f0561200640 1 --2- 172.21.15.83:0/1960585810 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588ad920390 0x5588ad91ddd0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f0554098e50 tx=0x7f055409d1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fc3fa24b7a908ff server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:40.766 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.764+0000 7f0546000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:54:40.766 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.764+0000 7f0546000640 1 -- 172.21.15.83:0/1960585810 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f055401f040 con 0x5588ad920390 2024-04-24T08:54:40.767 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.764+0000 7f056607c740 1 -- 172.21.15.83:0/1960585810 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05480044d0 con 0x5588ad920390 2024-04-24T08:54:40.767 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.764+0000 7f0546000640 1 -- 172.21.15.83:0/1960585810 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f05540bbbb0 con 0x5588ad920390 2024-04-24T08:54:40.767 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.764+0000 7f0546000640 1 -- 172.21.15.83:0/1960585810 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f05540a5660 con 0x5588ad920390 2024-04-24T08:54:40.767 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.764+0000 7f056607c740 10 client.16821 fetch_fsmap learned FSMap version 30 2024-04-24T08:54:40.767 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.764+0000 7f056607c740 1 -- 172.21.15.83:0/1960585810 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5588ad58e210 con 0x5588ad920390 2024-04-24T08:54:40.767 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.764+0000 7f0546000640 1 -- 172.21.15.83:0/1960585810 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f05480044d0 con 0x5588ad920390 2024-04-24T08:54:40.767 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.765+0000 7f0546000640 1 -- 172.21.15.83:0/1960585810 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7f055400f040 con 0x5588ad920390 2024-04-24T08:54:40.767 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.765+0000 7f056607c740 10 client.16821 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T08:54:40.767 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.765+0000 7f056607c740 10 client.16821 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T08:54:40.767 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.765+0000 7f056607c740 20 client.16821 populate_metadata read hostname 'smithi083' 2024-04-24T08:54:40.767 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.765+0000 7f056607c740 1 --2- 172.21.15.83:0/1960585810 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5588ad930570 0x5588ad9329c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:40.767 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.765+0000 7f056607c740 4 client.16821 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:54:40.767 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.765+0000 7f056607c740 1 -- 172.21.15.83:0/1960585810 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5588ad889320 con 0x5588ad930570 2024-04-24T08:54:40.769 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.765+0000 7f055be00640 1 --2- 172.21.15.83:0/1960585810 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5588ad930570 0x5588ad9329c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:40.769 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.767+0000 7f055be00640 1 --2- 172.21.15.83:0/1960585810 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5588ad930570 0x5588ad9329c0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f859d96ac43e6ad0 server_cookie=220eb54f5f087c11 in_seq=0 out_seq=0 2024-04-24T08:54:40.769 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.767+0000 7f0546000640 10 client.16821 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:54:40.769 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.769+0000 7f0546000640 1 -- 172.21.15.83:0/1960585810 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5588ad889320 con 0x5588ad930570 2024-04-24T08:54:40.769 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.769+0000 7f0546000640 10 client.16821 handle_command_reply: tid=0 2024-04-24T08:54:40.769 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.769+0000 7f056607c740 1 client.16821 shutdown 2024-04-24T08:54:40.769 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.769+0000 7f056607c740 1 -- 172.21.15.83:0/1960585810 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588ad920390 msgr2=0x5588ad91ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:40.769 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.769+0000 7f056607c740 1 --2- 172.21.15.83:0/1960585810 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588ad920390 0x5588ad91ddd0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f0554098e50 tx=0x7f055409d1b0 comp rx=0 tx=0).stop 2024-04-24T08:54:40.769 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.770+0000 7f056607c740 1 -- 172.21.15.83:0/1960585810 shutdown_connections 2024-04-24T08:54:40.769 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.770+0000 7f056607c740 1 --2- 172.21.15.83:0/1960585810 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5588ad930570 0x5588ad9329c0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:40.769 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.770+0000 7f056607c740 1 --2- 172.21.15.83:0/1960585810 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588ad920390 0x5588ad91ddd0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:40.769 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.770+0000 7f056607c740 1 --2- 172.21.15.83:0/1960585810 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5588ad915ad0 0x5588ad91d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:40.770 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.770+0000 7f0546000640 0 client.16821 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:54:40.770 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.770+0000 7f056607c740 1 --2- 172.21.15.83:0/1960585810 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588ad9151b0 0x5588ad91d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:40.770 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.770+0000 7f056607c740 1 -- 172.21.15.83:0/1960585810 >> 172.21.15.83:0/1960585810 conn(0x5588ad9116c0 msgr2=0x5588ad917440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:40.770 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.770+0000 7f056607c740 1 -- 172.21.15.83:0/1960585810 shutdown_connections 2024-04-24T08:54:40.770 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.770+0000 7f056607c740 1 -- 172.21.15.83:0/1960585810 wait complete. 2024-04-24T08:54:40.770 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.770+0000 7f056607c740 20 client.16821 trim_cache size 0 max 16384 2024-04-24T08:54:40.789 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.788+0000 7f056607c740 1 Processor -- start 2024-04-24T08:54:40.789 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.788+0000 7f056607c740 1 -- start start 2024-04-24T08:54:40.789 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.788+0000 7f056607c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588ad9151b0 0x5588ad92be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:40.789 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.788+0000 7f056607c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588ad915ad0 0x5588ad91f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:40.790 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.788+0000 7f056607c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5588ad920390 0x5588ad9382d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:40.790 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.788+0000 7f056607c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f055c1b1560 con 0x5588ad9151b0 2024-04-24T08:54:40.790 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.788+0000 7f056607c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f055c1b3130 con 0x5588ad920390 2024-04-24T08:54:40.790 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.788+0000 7f056607c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f055c1b1c30 con 0x5588ad915ad0 2024-04-24T08:54:40.790 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.789+0000 7f055be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588ad9151b0 0x5588ad92be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:40.790 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.789+0000 7f055be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588ad9151b0 0x5588ad92be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42824/0 (socket says 172.21.15.83:42824) 2024-04-24T08:54:40.790 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.789+0000 7f055be00640 1 -- 172.21.15.83:0/214763359 learned_addr learned my addr 172.21.15.83:0/214763359 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:54:40.790 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.789+0000 7f055b400640 1 --2- 172.21.15.83:0/214763359 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588ad915ad0 0x5588ad91f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:40.790 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.789+0000 7f055be00640 1 -- 172.21.15.83:0/214763359 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588ad915ad0 msgr2=0x5588ad91f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:40.790 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.789+0000 7f0561200640 1 --2- 172.21.15.83:0/214763359 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5588ad920390 0x5588ad9382d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:40.790 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.789+0000 7f055be00640 1 --2- 172.21.15.83:0/214763359 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588ad915ad0 0x5588ad91f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:40.790 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.789+0000 7f055be00640 1 -- 172.21.15.83:0/214763359 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5588ad920390 msgr2=0x5588ad9382d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:40.790 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.789+0000 7f055be00640 1 --2- 172.21.15.83:0/214763359 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5588ad920390 0x5588ad9382d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:40.790 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.789+0000 7f055be00640 1 -- 172.21.15.83:0/214763359 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588ad65a620 con 0x5588ad9151b0 2024-04-24T08:54:40.791 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.789+0000 7f055b400640 1 --2- 172.21.15.83:0/214763359 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588ad915ad0 0x5588ad91f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:54:40.791 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.789+0000 7f0561200640 1 --2- 172.21.15.83:0/214763359 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5588ad920390 0x5588ad9382d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:54:40.791 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.790+0000 7f055be00640 1 --2- 172.21.15.83:0/214763359 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588ad9151b0 0x5588ad92be20 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x5588ad933240 tx=0x7f054c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a65fa123071c34f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:40.791 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.790+0000 7f0550c00640 1 -- 172.21.15.83:0/214763359 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f054c011c30 con 0x5588ad9151b0 2024-04-24T08:54:40.791 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.790+0000 7f0550c00640 1 -- 172.21.15.83:0/214763359 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f054c011dd0 con 0x5588ad9151b0 2024-04-24T08:54:40.791 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.790+0000 7f0550c00640 1 -- 172.21.15.83:0/214763359 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f054c003850 con 0x5588ad9151b0 2024-04-24T08:54:40.794 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.791+0000 7f056607c740 1 -- 172.21.15.83:0/214763359 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588ad9151b0 msgr2=0x5588ad92be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:40.794 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.791+0000 7f056607c740 1 --2- 172.21.15.83:0/214763359 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588ad9151b0 0x5588ad92be20 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x5588ad933240 tx=0x7f054c013000 comp rx=0 tx=0).stop 2024-04-24T08:54:40.794 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.791+0000 7f056607c740 1 -- 172.21.15.83:0/214763359 shutdown_connections 2024-04-24T08:54:40.794 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.791+0000 7f056607c740 1 --2- 172.21.15.83:0/214763359 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5588ad920390 0x5588ad9382d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:40.795 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.791+0000 7f056607c740 1 --2- 172.21.15.83:0/214763359 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588ad915ad0 0x5588ad91f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:40.795 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.791+0000 7f056607c740 1 --2- 172.21.15.83:0/214763359 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588ad9151b0 0x5588ad92be20 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:40.795 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.791+0000 7f056607c740 1 -- 172.21.15.83:0/214763359 >> 172.21.15.83:0/214763359 conn(0x5588ad9116c0 msgr2=0x5588ad91eb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:40.795 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.791+0000 7f056607c740 1 -- 172.21.15.83:0/214763359 shutdown_connections 2024-04-24T08:54:40.795 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.791+0000 7f056607c740 1 -- 172.21.15.83:0/214763359 wait complete. 2024-04-24T08:54:40.795 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.791+0000 7f056607c740 1 -- start start 2024-04-24T08:54:40.795 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.792+0000 7f056607c740 1 Processor -- start 2024-04-24T08:54:40.795 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.792+0000 7f056607c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5588ad9151b0 0x5588ad928520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:40.795 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.792+0000 7f056607c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588ad915ad0 0x5588ad925300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:40.795 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.792+0000 7f056607c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588ad920390 0x5588ad925840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:40.795 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.792+0000 7f056607c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f055c1b29d0 con 0x5588ad915ad0 2024-04-24T08:54:40.795 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.792+0000 7f056607c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f055c1b2300 con 0x5588ad9151b0 2024-04-24T08:54:40.795 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.792+0000 7f056607c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f055c1af540 con 0x5588ad920390 2024-04-24T08:54:40.795 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.792+0000 7f056607c740 10 client.0 _ll_register_callbacks cb 0x7f0554005e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:54:40.796 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.792+0000 7f055b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588ad915ad0 0x5588ad925300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:40.796 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.792+0000 7f055b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588ad915ad0 0x5588ad925300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42838/0 (socket says 172.21.15.83:42838) 2024-04-24T08:54:40.796 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.792+0000 7f055b400640 1 -- 172.21.15.83:0/3209216918 learned_addr learned my addr 172.21.15.83:0/3209216918 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:54:40.796 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.793+0000 7f055be00640 1 --2- 172.21.15.83:0/3209216918 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5588ad9151b0 0x5588ad928520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:40.796 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.793+0000 7f0561200640 1 --2- 172.21.15.83:0/3209216918 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588ad920390 0x5588ad925840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:40.796 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.793+0000 7f055b400640 1 -- 172.21.15.83:0/3209216918 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588ad920390 msgr2=0x5588ad925840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:40.796 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.793+0000 7f055b400640 1 --2- 172.21.15.83:0/3209216918 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588ad920390 0x5588ad925840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:40.796 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.793+0000 7f055b400640 1 -- 172.21.15.83:0/3209216918 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5588ad9151b0 msgr2=0x5588ad928520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:40.796 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.793+0000 7f055b400640 1 --2- 172.21.15.83:0/3209216918 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5588ad9151b0 0x5588ad928520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:40.796 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.793+0000 7f055b400640 1 -- 172.21.15.83:0/3209216918 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588ad925d80 con 0x5588ad915ad0 2024-04-24T08:54:40.796 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.793+0000 7f055be00640 1 --2- 172.21.15.83:0/3209216918 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5588ad9151b0 0x5588ad928520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:54:40.796 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.793+0000 7f0561200640 1 --2- 172.21.15.83:0/3209216918 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588ad920390 0x5588ad925840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:54:40.796 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.793+0000 7f055b400640 1 --2- 172.21.15.83:0/3209216918 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588ad915ad0 0x5588ad925300 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f054802a9b0 tx=0x7f054802a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2d3d1ce159ce80 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:40.796 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.793+0000 7f0546000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:54:40.796 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.793+0000 7f0546000640 1 -- 172.21.15.83:0/3209216918 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0548035070 con 0x5588ad915ad0 2024-04-24T08:54:40.797 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.793+0000 7f0546000640 1 -- 172.21.15.83:0/3209216918 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f05480048c0 con 0x5588ad915ad0 2024-04-24T08:54:40.797 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.793+0000 7f056607c740 1 -- 172.21.15.83:0/3209216918 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f054c011dd0 con 0x5588ad915ad0 2024-04-24T08:54:40.797 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.794+0000 7f0546000640 1 -- 172.21.15.83:0/3209216918 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0548040040 con 0x5588ad915ad0 2024-04-24T08:54:40.797 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.794+0000 7f056607c740 10 client.16827 fetch_fsmap learned FSMap version 30 2024-04-24T08:54:40.797 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.794+0000 7f056607c740 1 -- 172.21.15.83:0/3209216918 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5588ad997740 con 0x5588ad915ad0 2024-04-24T08:54:40.797 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.794+0000 7f0546000640 1 -- 172.21.15.83:0/3209216918 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f054c011dd0 con 0x5588ad915ad0 2024-04-24T08:54:40.797 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.794+0000 7f0546000640 1 -- 172.21.15.83:0/3209216918 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7f0548031b00 con 0x5588ad915ad0 2024-04-24T08:54:40.797 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.794+0000 7f056607c740 10 client.16827 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T08:54:40.797 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.794+0000 7f056607c740 10 client.16827 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T08:54:40.797 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.794+0000 7f056607c740 20 client.16827 populate_metadata read hostname 'smithi083' 2024-04-24T08:54:40.797 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.794+0000 7f056607c740 1 --2- 172.21.15.83:0/3209216918 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5588ad997ad0 0x5588ad999ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:40.797 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.794+0000 7f056607c740 4 client.16827 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T08:54:40.797 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.794+0000 7f056607c740 1 -- 172.21.15.83:0/3209216918 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5588ad883190 con 0x5588ad997ad0 2024-04-24T08:54:40.797 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.795+0000 7f055be00640 1 --2- 172.21.15.83:0/3209216918 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5588ad997ad0 0x5588ad999ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:40.798 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.796+0000 7f055be00640 1 --2- 172.21.15.83:0/3209216918 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5588ad997ad0 0x5588ad999ef0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17ed444f548ac7ad server_cookie=d5894d60fc4c6c90 in_seq=0 out_seq=0 2024-04-24T08:54:40.798 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.796+0000 7f0546000640 10 client.16827 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:54:40.845 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.845+0000 7f0546000640 1 -- 172.21.15.83:0/3209216918 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5588ad883190 con 0x5588ad997ad0 2024-04-24T08:54:40.845 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.845+0000 7f0546000640 10 client.16827 handle_command_reply: tid=0 2024-04-24T08:54:40.845 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.845+0000 7f056607c740 1 client.16827 shutdown 2024-04-24T08:54:40.845 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.846+0000 7f056607c740 1 -- 172.21.15.83:0/3209216918 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588ad915ad0 msgr2=0x5588ad925300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:40.846 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.846+0000 7f056607c740 1 --2- 172.21.15.83:0/3209216918 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588ad915ad0 0x5588ad925300 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f054802a9b0 tx=0x7f054802a3f0 comp rx=0 tx=0).stop 2024-04-24T08:54:40.846 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.846+0000 7f056607c740 1 -- 172.21.15.83:0/3209216918 shutdown_connections 2024-04-24T08:54:40.846 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.846+0000 7f056607c740 1 --2- 172.21.15.83:0/3209216918 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5588ad997ad0 0x5588ad999ef0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:40.846 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.846+0000 7f056607c740 1 --2- 172.21.15.83:0/3209216918 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588ad920390 0x5588ad925840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:40.846 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.846+0000 7f056607c740 1 --2- 172.21.15.83:0/3209216918 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588ad915ad0 0x5588ad925300 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:40.846 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.846+0000 7f056607c740 1 --2- 172.21.15.83:0/3209216918 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5588ad9151b0 0x5588ad928520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:40.846 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.846+0000 7f0546000640 0 client.16827 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:54:40.846 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.846+0000 7f056607c740 1 -- 172.21.15.83:0/3209216918 >> 172.21.15.83:0/3209216918 conn(0x5588ad9116c0 msgr2=0x5588ad91f410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:40.846 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.846+0000 7f056607c740 1 -- 172.21.15.83:0/3209216918 shutdown_connections 2024-04-24T08:54:40.846 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.846+0000 7f056607c740 1 -- 172.21.15.83:0/3209216918 wait complete. 2024-04-24T08:54:40.846 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.846+0000 7f056607c740 20 client.16827 trim_cache size 0 max 16384 2024-04-24T08:54:40.847 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T08:54:40.847 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T08:54:40.847 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T08:54:40.847 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T08:54:40.848 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.847+0000 7f0563200640 1 -- 172.21.15.83:0/2720971811 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f05400a3dc0 msgr2=0x7f05400a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:40.848 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.847+0000 7f0563200640 1 --2- 172.21.15.83:0/2720971811 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f05400a3dc0 0x7f05400a6280 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f054c0058c0 tx=0x7f054c005850 comp rx=0 tx=0).stop 2024-04-24T08:54:40.848 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.847+0000 7f0563200640 1 -- 172.21.15.83:0/2720971811 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f055c19a770 msgr2=0x7f055c19abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:40.848 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.847+0000 7f0563200640 1 --2- 172.21.15.83:0/2720971811 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f055c19a770 0x7f055c19abf0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f055400eaa0 tx=0x7f055400ef60 comp rx=0 tx=0).stop 2024-04-24T08:54:40.848 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.847+0000 7f0563200640 1 -- 172.21.15.83:0/2720971811 shutdown_connections 2024-04-24T08:54:40.848 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.847+0000 7f0563200640 1 --2- 172.21.15.83:0/2720971811 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f05400a3dc0 0x7f05400a6280 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:40.848 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.848+0000 7f0563200640 1 --2- 172.21.15.83:0/2720971811 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f055c19a770 0x7f055c19abf0 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:40.848 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.848+0000 7f0563200640 1 --2- 172.21.15.83:0/2720971811 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f055c109540 0x7f055c1a0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:40.848 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.848+0000 7f0563200640 1 --2- 172.21.15.83:0/2720971811 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f055c104470 0x7f055c1a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:40.848 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.848+0000 7f0563200640 1 -- 172.21.15.83:0/2720971811 >> 172.21.15.83:0/2720971811 conn(0x7f055c0ffd30 msgr2=0x7f055c102030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:40.848 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.848+0000 7f0563200640 1 -- 172.21.15.83:0/2720971811 shutdown_connections 2024-04-24T08:54:40.849 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:40.848+0000 7f0563200640 1 -- 172.21.15.83:0/2720971811 wait complete. 2024-04-24T08:54:41.562 INFO:tasks.workunit.client.0.smithi083.stdout: 1 9593 0.89 MB/sec warmup 75 sec latency 54.284 ms 2024-04-24T08:54:42.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:54:41 smithi083 ceph-mon[27907]: pgmap v95: 129 pgs: 129 active+clean; 29 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 234 KiB/s wr, 61 op/s 2024-04-24T08:54:42.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:54:41 smithi138 ceph-mon[31441]: pgmap v95: 129 pgs: 129 active+clean; 29 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 234 KiB/s wr, 61 op/s 2024-04-24T08:54:42.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:54:41 smithi189 ceph-mon[31791]: pgmap v95: 129 pgs: 129 active+clean; 29 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 234 KiB/s wr, 61 op/s 2024-04-24T08:54:42.562 INFO:tasks.workunit.client.0.smithi083.stdout: 1 9702 0.88 MB/sec warmup 76 sec latency 59.126 ms 2024-04-24T08:54:43.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:54:42 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T08:54:43.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:54:42 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T08:54:43.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:54:42 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T08:54:43.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:54:42 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T08:54:43.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:54:42 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T08:54:43.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:54:42 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T08:54:43.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:54:42 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T08:54:43.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:54:42 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T08:54:43.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:54:42 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T08:54:43.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:54:42 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T08:54:43.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:54:42 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T08:54:43.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:54:42 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T08:54:43.562 INFO:tasks.workunit.client.0.smithi083.stdout: 1 9892 0.91 MB/sec warmup 77 sec latency 56.124 ms 2024-04-24T08:54:44.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:54:43 smithi138 ceph-mon[31441]: pgmap v96: 129 pgs: 129 active+clean; 30 MiB data, 494 MiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 425 KiB/s wr, 84 op/s 2024-04-24T08:54:44.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:54:43 smithi189 ceph-mon[31791]: pgmap v96: 129 pgs: 129 active+clean; 30 MiB data, 494 MiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 425 KiB/s wr, 84 op/s 2024-04-24T08:54:44.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:54:43 smithi083 ceph-mon[27907]: pgmap v96: 129 pgs: 129 active+clean; 30 MiB data, 494 MiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 425 KiB/s wr, 84 op/s 2024-04-24T08:54:44.562 INFO:tasks.workunit.client.0.smithi083.stdout: 1 10083 0.92 MB/sec warmup 78 sec latency 48.426 ms 2024-04-24T08:54:45.562 INFO:tasks.workunit.client.0.smithi083.stdout: 1 10306 0.92 MB/sec warmup 79 sec latency 63.622 ms 2024-04-24T08:54:46.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:54:45 smithi138 ceph-mon[31441]: pgmap v97: 129 pgs: 129 active+clean; 30 MiB data, 494 MiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 245 KiB/s wr, 52 op/s 2024-04-24T08:54:46.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:54:45 smithi189 ceph-mon[31791]: pgmap v97: 129 pgs: 129 active+clean; 30 MiB data, 494 MiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 245 KiB/s wr, 52 op/s 2024-04-24T08:54:46.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:54:45 smithi083 ceph-mon[27907]: pgmap v97: 129 pgs: 129 active+clean; 30 MiB data, 494 MiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 245 KiB/s wr, 52 op/s 2024-04-24T08:54:46.562 INFO:tasks.workunit.client.0.smithi083.stdout: 1 10397 0.92 MB/sec warmup 80 sec latency 55.657 ms 2024-04-24T08:54:47.562 INFO:tasks.workunit.client.0.smithi083.stdout: 1 10424 0.91 MB/sec warmup 81 sec latency 66.519 ms 2024-04-24T08:54:48.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:54:47 smithi138 ceph-mon[31441]: pgmap v98: 129 pgs: 129 active+clean; 30 MiB data, 494 MiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 245 KiB/s wr, 52 op/s 2024-04-24T08:54:48.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:54:47 smithi189 ceph-mon[31791]: pgmap v98: 129 pgs: 129 active+clean; 30 MiB data, 494 MiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 245 KiB/s wr, 52 op/s 2024-04-24T08:54:48.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:54:47 smithi083 ceph-mon[27907]: pgmap v98: 129 pgs: 129 active+clean; 30 MiB data, 494 MiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 245 KiB/s wr, 52 op/s 2024-04-24T08:54:48.563 INFO:tasks.workunit.client.0.smithi083.stdout: 1 10477 0.90 MB/sec warmup 82 sec latency 53.268 ms 2024-04-24T08:54:49.562 INFO:tasks.workunit.client.0.smithi083.stdout: 1 10679 0.92 MB/sec warmup 83 sec latency 53.370 ms 2024-04-24T08:54:50.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:54:49 smithi138 ceph-mon[31441]: pgmap v99: 129 pgs: 129 active+clean; 32 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 415 KiB/s wr, 71 op/s 2024-04-24T08:54:50.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:54:49 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:54:50.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:54:49 smithi189 ceph-mon[31791]: pgmap v99: 129 pgs: 129 active+clean; 32 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 415 KiB/s wr, 71 op/s 2024-04-24T08:54:50.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:54:49 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:54:50.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:54:49 smithi083 ceph-mon[27907]: pgmap v99: 129 pgs: 129 active+clean; 32 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 415 KiB/s wr, 71 op/s 2024-04-24T08:54:50.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:54:49 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:54:50.563 INFO:tasks.workunit.client.0.smithi083.stdout: 1 10850 0.93 MB/sec warmup 84 sec latency 48.405 ms 2024-04-24T08:54:51.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:54:50 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:54:51.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:54:50 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:54:51.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:54:50 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T08:54:51.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:54:50 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:54:51.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:54:50 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:54:51.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:54:50 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T08:54:51.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:54:50 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:54:51.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:54:50 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:54:51.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:54:50 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T08:54:51.563 INFO:tasks.workunit.client.0.smithi083.stdout: 1 10911 0.92 MB/sec warmup 85 sec latency 46.894 ms 2024-04-24T08:54:52.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:54:51 smithi138 ceph-mon[31441]: pgmap v100: 129 pgs: 129 active+clean; 32 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 362 KiB/s wr, 41 op/s 2024-04-24T08:54:52.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:54:51 smithi189 ceph-mon[31791]: pgmap v100: 129 pgs: 129 active+clean; 32 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 362 KiB/s wr, 41 op/s 2024-04-24T08:54:52.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:54:51 smithi083 ceph-mon[27907]: pgmap v100: 129 pgs: 129 active+clean; 32 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 362 KiB/s wr, 41 op/s 2024-04-24T08:54:52.563 INFO:tasks.workunit.client.0.smithi083.stdout: 1 10970 0.91 MB/sec warmup 86 sec latency 50.015 ms 2024-04-24T08:54:53.563 INFO:tasks.workunit.client.0.smithi083.stdout: 1 11038 0.90 MB/sec warmup 87 sec latency 49.278 ms 2024-04-24T08:54:54.000 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.000+0000 7f1207e00640 1 -- 172.21.15.83:0/1217822323 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f120810c3e0 msgr2=0x7f120810e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:54.000 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.000+0000 7f1207e00640 1 --2- 172.21.15.83:0/1217822323 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f120810c3e0 0x7f120810e880 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f1200009ca0 tx=0x7f120002a950 comp rx=0 tx=0).stop 2024-04-24T08:54:54.001 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.000+0000 7f1207e00640 1 -- 172.21.15.83:0/1217822323 shutdown_connections 2024-04-24T08:54:54.001 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.000+0000 7f1207e00640 1 --2- 172.21.15.83:0/1217822323 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f120810c3e0 0x7f120810e880 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:54.001 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.000+0000 7f1207e00640 1 --2- 172.21.15.83:0/1217822323 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f12081038e0 0x7f120810bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:54.001 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.000+0000 7f1207e00640 1 --2- 172.21.15.83:0/1217822323 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1208102fc0 0x7f12081033a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:54.001 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.000+0000 7f1207e00640 1 -- 172.21.15.83:0/1217822323 >> 172.21.15.83:0/1217822323 conn(0x7f12080fc820 msgr2=0x7f12080fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:54.001 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.000+0000 7f1207e00640 1 -- 172.21.15.83:0/1217822323 shutdown_connections 2024-04-24T08:54:54.001 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.000+0000 7f1207e00640 1 -- 172.21.15.83:0/1217822323 wait complete. 2024-04-24T08:54:54.001 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.001+0000 7f1207e00640 1 Processor -- start 2024-04-24T08:54:54.002 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.002+0000 7f1207e00640 1 -- start start 2024-04-24T08:54:54.002 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.002+0000 7f1207e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1208102fc0 0x7f12081a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:54.002 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.002+0000 7f1207e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12081038e0 0x7f12081a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:54.003 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.002+0000 7f1207e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f120810c3e0 0x7f120819a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:54.003 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.002+0000 7f1207e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1208112150 con 0x7f12081038e0 2024-04-24T08:54:54.003 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.002+0000 7f1207e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1208111fd0 con 0x7f1208102fc0 2024-04-24T08:54:54.003 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.002+0000 7f1207e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f12081122d0 con 0x7f120810c3e0 2024-04-24T08:54:54.003 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.002+0000 7f1206000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12081038e0 0x7f12081a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:54.003 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.003+0000 7f1206000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12081038e0 0x7f12081a0b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46968/0 (socket says 172.21.15.83:46968) 2024-04-24T08:54:54.003 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.003+0000 7f1206000640 1 -- 172.21.15.83:0/886135490 learned_addr learned my addr 172.21.15.83:0/886135490 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:54:54.004 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.003+0000 7f1206a00640 1 --2- 172.21.15.83:0/886135490 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1208102fc0 0x7f12081a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:54.004 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.003+0000 7f1206000640 1 -- 172.21.15.83:0/886135490 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f120810c3e0 msgr2=0x7f120819a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:54.004 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.003+0000 7f1206000640 1 --2- 172.21.15.83:0/886135490 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f120810c3e0 0x7f120819a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:54.004 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.003+0000 7f1206000640 1 -- 172.21.15.83:0/886135490 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1208102fc0 msgr2=0x7f12081a0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:54.004 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.003+0000 7f1206000640 1 --2- 172.21.15.83:0/886135490 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1208102fc0 0x7f12081a0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:54.004 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.003+0000 7f1206000640 1 -- 172.21.15.83:0/886135490 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f120819afa0 con 0x7f12081038e0 2024-04-24T08:54:54.004 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.003+0000 7f1206000640 1 --2- 172.21.15.83:0/886135490 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12081038e0 0x7f12081a0b50 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f11f400b3f0 tx=0x7f11f400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:54.004 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.004+0000 7f11ff400640 1 -- 172.21.15.83:0/886135490 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f4015020 con 0x7f12081038e0 2024-04-24T08:54:54.004 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.004+0000 7f1207e00640 1 -- 172.21.15.83:0/886135490 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f120819b290 con 0x7f12081038e0 2024-04-24T08:54:54.005 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.004+0000 7f11ff400640 1 -- 172.21.15.83:0/886135490 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f11f4011ce0 con 0x7f12081038e0 2024-04-24T08:54:54.005 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.004+0000 7f11ff400640 1 -- 172.21.15.83:0/886135490 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f4010d80 con 0x7f12081038e0 2024-04-24T08:54:54.005 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.004+0000 7f1207e00640 1 -- 172.21.15.83:0/886135490 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12081a7460 con 0x7f12081038e0 2024-04-24T08:54:54.008 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.008+0000 7f12109e3740 1 Processor -- start 2024-04-24T08:54:54.008 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.008+0000 7f12109e3740 1 -- start start 2024-04-24T08:54:54.009 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.008+0000 7f12109e3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56179d0913c0 0x56179d0917a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:54.009 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.008+0000 7f12109e3740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56179d091ce0 0x56179d09c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:54.009 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.008+0000 7f12109e3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56179d09c5a0 0x56179d09e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:54.009 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.008+0000 7f12109e3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56179cff7900 con 0x56179d09c5a0 2024-04-24T08:54:54.009 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.008+0000 7f12109e3740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56179cf9df20 con 0x56179d091ce0 2024-04-24T08:54:54.009 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.008+0000 7f12109e3740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56179cff38c0 con 0x56179d0913c0 2024-04-24T08:54:54.009 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.008+0000 7f11ff400640 1 -- 172.21.15.83:0/886135490 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f11f4018020 con 0x7f12081038e0 2024-04-24T08:54:54.009 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.009+0000 7f1206000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56179d09c5a0 0x56179d09e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:54.009 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.009+0000 7f1206000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56179d09c5a0 0x56179d09e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46984/0 (socket says 172.21.15.83:46984) 2024-04-24T08:54:54.009 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.009+0000 7f1206000640 1 -- 172.21.15.83:0/3697818224 learned_addr learned my addr 172.21.15.83:0/3697818224 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:54:54.010 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.009+0000 7f1207400640 1 --2- 172.21.15.83:0/3697818224 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56179d0913c0 0x56179d0917a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:54.010 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.009+0000 7f1206000640 1 -- 172.21.15.83:0/3697818224 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56179d0913c0 msgr2=0x56179d0917a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:54.010 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.009+0000 7f1206000640 1 --2- 172.21.15.83:0/3697818224 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56179d0913c0 0x56179d0917a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:54.010 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.009+0000 7f1206000640 1 -- 172.21.15.83:0/3697818224 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56179d091ce0 msgr2=0x56179d09c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:54.010 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.009+0000 7f1206000640 1 --2- 172.21.15.83:0/3697818224 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56179d091ce0 0x56179d09c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:54.010 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.009+0000 7f1206000640 1 -- 172.21.15.83:0/3697818224 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56179cfc2e10 con 0x56179d09c5a0 2024-04-24T08:54:54.010 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.010+0000 7f1206000640 1 --2- 172.21.15.83:0/3697818224 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56179d09c5a0 0x56179d09e990 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f11f40cd780 tx=0x7f11f40d8a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b5a083bba3edc1e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.010+0000 7f11f3e00640 1 -- 172.21.15.83:0/3697818224 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f40e4070 con 0x56179d09c5a0 2024-04-24T08:54:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.010+0000 7f11f3e00640 1 -- 172.21.15.83:0/3697818224 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f11f40d24e0 con 0x56179d09c5a0 2024-04-24T08:54:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.010+0000 7f11f3e00640 1 -- 172.21.15.83:0/3697818224 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f40dc9b0 con 0x56179d09c5a0 2024-04-24T08:54:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.011+0000 7f11ff400640 1 --2- 172.21.15.83:0/886135490 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f11ec0a3d90 0x7f11ec0a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.011+0000 7f11ff400640 1 -- 172.21.15.83:0/886135490 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f11f40d0350 con 0x7f12081038e0 2024-04-24T08:54:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.011+0000 7f1207400640 1 --2- 172.21.15.83:0/886135490 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f11ec0a3d90 0x7f11ec0a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.011+0000 7f12109e3740 1 -- 172.21.15.83:0/3697818224 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56179d09c5a0 msgr2=0x56179d09e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.011+0000 7f12109e3740 1 --2- 172.21.15.83:0/3697818224 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56179d09c5a0 0x56179d09e990 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f11f40cd780 tx=0x7f11f40d8a10 comp rx=0 tx=0).stop 2024-04-24T08:54:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.012+0000 7f12109e3740 1 -- 172.21.15.83:0/3697818224 shutdown_connections 2024-04-24T08:54:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.012+0000 7f12109e3740 1 --2- 172.21.15.83:0/3697818224 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56179d09c5a0 0x56179d09e990 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.012+0000 7f12109e3740 1 --2- 172.21.15.83:0/3697818224 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56179d091ce0 0x56179d09c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.012+0000 7f12109e3740 1 --2- 172.21.15.83:0/3697818224 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56179d0913c0 0x56179d0917a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.012+0000 7f12109e3740 1 -- 172.21.15.83:0/3697818224 >> 172.21.15.83:0/3697818224 conn(0x56179d08d8d0 msgr2=0x56179d08dcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:54.013 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.012+0000 7f1207400640 1 --2- 172.21.15.83:0/886135490 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f11ec0a3d90 0x7f11ec0a6250 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1200009a60 tx=0x7f1200005d80 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:54.013 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.012+0000 7f12109e3740 1 -- 172.21.15.83:0/3697818224 shutdown_connections 2024-04-24T08:54:54.013 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.012+0000 7f12109e3740 1 -- 172.21.15.83:0/3697818224 wait complete. 2024-04-24T08:54:54.013 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.013+0000 7f12109e3740 1 -- start start 2024-04-24T08:54:54.013 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.013+0000 7f12109e3740 1 Processor -- start 2024-04-24T08:54:54.014 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.014+0000 7f12109e3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56179d0913c0 0x56179d099530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:54.014 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.014+0000 7f12109e3740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56179d091ce0 0x56179d099a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:54.014 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.014+0000 7f12109e3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56179d09c5a0 0x56179d099fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:54.014 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.014+0000 7f12109e3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56179d09eff0 con 0x56179d09c5a0 2024-04-24T08:54:54.014 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.014+0000 7f12109e3740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56179cfb12c0 con 0x56179d091ce0 2024-04-24T08:54:54.014 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.014+0000 7f12109e3740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56179d09f170 con 0x56179d0913c0 2024-04-24T08:54:54.015 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.014+0000 7f12109e3740 10 client.0 _ll_register_callbacks cb 0x56179cd09a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:54:54.017 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.014+0000 7f1207400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56179d09c5a0 0x56179d099fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:54.017 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.014+0000 7f1207400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56179d09c5a0 0x56179d099fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46992/0 (socket says 172.21.15.83:46992) 2024-04-24T08:54:54.017 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.014+0000 7f1207400640 1 -- 172.21.15.83:0/2641631921 learned_addr learned my addr 172.21.15.83:0/2641631921 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:54:54.017 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.015+0000 7f1206000640 1 --2- 172.21.15.83:0/2641631921 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56179d091ce0 0x56179d099a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:54.017 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.015+0000 7f1206a00640 1 --2- 172.21.15.83:0/2641631921 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56179d0913c0 0x56179d099530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:54.017 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.015+0000 7f1207400640 1 -- 172.21.15.83:0/2641631921 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56179d0913c0 msgr2=0x56179d099530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:54.017 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.015+0000 7f1207400640 1 --2- 172.21.15.83:0/2641631921 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56179d0913c0 0x56179d099530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:54.017 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.015+0000 7f1207400640 1 -- 172.21.15.83:0/2641631921 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56179d091ce0 msgr2=0x56179d099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:54.017 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.015+0000 7f1207400640 1 --2- 172.21.15.83:0/2641631921 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56179d091ce0 0x56179d099a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:54.017 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.015+0000 7f1207400640 1 -- 172.21.15.83:0/2641631921 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56179cfc1fd0 con 0x56179d09c5a0 2024-04-24T08:54:54.017 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.015+0000 7f1206000640 1 --2- 172.21.15.83:0/2641631921 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56179d091ce0 0x56179d099a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:54:54.017 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.015+0000 7f1207400640 1 --2- 172.21.15.83:0/2641631921 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56179d09c5a0 0x56179d099fb0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f120002f680 tx=0x7f12000095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceb61cb3f157cc8e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:54.018 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.015+0000 7f11f0c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:54:54.018 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.016+0000 7f11f0c00640 1 -- 172.21.15.83:0/2641631921 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1200034070 con 0x56179d09c5a0 2024-04-24T08:54:54.018 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.016+0000 7f11f0c00640 1 -- 172.21.15.83:0/2641631921 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f1200004450 con 0x56179d09c5a0 2024-04-24T08:54:54.018 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.016+0000 7f12109e3740 1 -- 172.21.15.83:0/2641631921 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11f40d24e0 con 0x56179d09c5a0 2024-04-24T08:54:54.018 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.016+0000 7f11f0c00640 1 -- 172.21.15.83:0/2641631921 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1200004650 con 0x56179d09c5a0 2024-04-24T08:54:54.018 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.016+0000 7f12109e3740 10 client.16839 fetch_fsmap learned FSMap version 30 2024-04-24T08:54:54.018 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.016+0000 7f12109e3740 1 -- 172.21.15.83:0/2641631921 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56179cd0a210 con 0x56179d09c5a0 2024-04-24T08:54:54.018 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.016+0000 7f11f0c00640 1 -- 172.21.15.83:0/2641631921 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f11f40d24e0 con 0x56179d09c5a0 2024-04-24T08:54:54.018 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.016+0000 7f11f0c00640 1 -- 172.21.15.83:0/2641631921 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7f120004c020 con 0x56179d09c5a0 2024-04-24T08:54:54.018 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.017+0000 7f12109e3740 10 client.16839 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T08:54:54.018 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.017+0000 7f12109e3740 10 client.16839 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T08:54:54.018 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.017+0000 7f12109e3740 20 client.16839 populate_metadata read hostname 'smithi083' 2024-04-24T08:54:54.018 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.017+0000 7f12109e3740 1 --2- 172.21.15.83:0/2641631921 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56179d0b4f00 0x56179d0b7360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:54.018 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.017+0000 7f12109e3740 4 client.16839 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:54:54.018 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.017+0000 7f12109e3740 1 -- 172.21.15.83:0/2641631921 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56179d005320 con 0x56179d0b4f00 2024-04-24T08:54:54.019 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.017+0000 7f1206a00640 1 --2- 172.21.15.83:0/2641631921 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56179d0b4f00 0x56179d0b7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:54.019 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.019+0000 7f1206a00640 1 --2- 172.21.15.83:0/2641631921 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56179d0b4f00 0x56179d0b7360 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf23b7d9076a42a0 server_cookie=76a6882783d153de in_seq=0 out_seq=0 2024-04-24T08:54:54.019 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.019+0000 7f11f0c00640 10 client.16839 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:54:54.021 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.021+0000 7f11f0c00640 1 -- 172.21.15.83:0/2641631921 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56179d005320 con 0x56179d0b4f00 2024-04-24T08:54:54.021 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.021+0000 7f11f0c00640 10 client.16839 handle_command_reply: tid=0 2024-04-24T08:54:54.021 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.021+0000 7f12109e3740 1 client.16839 shutdown 2024-04-24T08:54:54.021 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.021+0000 7f12109e3740 1 -- 172.21.15.83:0/2641631921 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56179d09c5a0 msgr2=0x56179d099fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:54.021 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.021+0000 7f12109e3740 1 --2- 172.21.15.83:0/2641631921 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56179d09c5a0 0x56179d099fb0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f120002f680 tx=0x7f12000095c0 comp rx=0 tx=0).stop 2024-04-24T08:54:54.022 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.021+0000 7f12109e3740 1 -- 172.21.15.83:0/2641631921 shutdown_connections 2024-04-24T08:54:54.022 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.021+0000 7f12109e3740 1 --2- 172.21.15.83:0/2641631921 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56179d0b4f00 0x56179d0b7360 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:54.023 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.021+0000 7f12109e3740 1 --2- 172.21.15.83:0/2641631921 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56179d09c5a0 0x56179d099fb0 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:54.023 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.021+0000 7f12109e3740 1 --2- 172.21.15.83:0/2641631921 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56179d091ce0 0x56179d099a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:54.023 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.021+0000 7f12109e3740 1 --2- 172.21.15.83:0/2641631921 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56179d0913c0 0x56179d099530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:54.023 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.021+0000 7f12109e3740 1 -- 172.21.15.83:0/2641631921 >> 172.21.15.83:0/2641631921 conn(0x56179d08d8d0 msgr2=0x56179d08ca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:54.023 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.021+0000 7f11f0c00640 0 client.16839 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:54:54.023 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.021+0000 7f12109e3740 1 -- 172.21.15.83:0/2641631921 shutdown_connections 2024-04-24T08:54:54.023 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.021+0000 7f12109e3740 1 -- 172.21.15.83:0/2641631921 wait complete. 2024-04-24T08:54:54.023 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.022+0000 7f12109e3740 20 client.16839 trim_cache size 0 max 16384 2024-04-24T08:54:54.039 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.039+0000 7f12109e3740 1 Processor -- start 2024-04-24T08:54:54.039 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.039+0000 7f12109e3740 1 -- start start 2024-04-24T08:54:54.039 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.039+0000 7f12109e3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56179d0913c0 0x56179d0b0790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:54.039 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.039+0000 7f12109e3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56179d091ce0 0x56179d093d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:54.039 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.039+0000 7f12109e3740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56179d09c5a0 0x56179d0bcc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:54.039 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.039+0000 7f12109e3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f12081ad310 con 0x56179d091ce0 2024-04-24T08:54:54.039 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.039+0000 7f12109e3740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f12081aeee0 con 0x56179d09c5a0 2024-04-24T08:54:54.039 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.039+0000 7f12109e3740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f12081ad9e0 con 0x56179d0913c0 2024-04-24T08:54:54.041 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.039+0000 7f1206000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56179d091ce0 0x56179d093d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:54.041 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.039+0000 7f1206000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56179d091ce0 0x56179d093d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46994/0 (socket says 172.21.15.83:46994) 2024-04-24T08:54:54.041 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.040+0000 7f1206000640 1 -- 172.21.15.83:0/497864756 learned_addr learned my addr 172.21.15.83:0/497864756 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:54:54.041 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.040+0000 7f1206a00640 1 --2- 172.21.15.83:0/497864756 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56179d0913c0 0x56179d0b0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:54.041 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.040+0000 7f1207400640 1 --2- 172.21.15.83:0/497864756 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56179d09c5a0 0x56179d0bcc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:54.041 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.040+0000 7f1206000640 1 -- 172.21.15.83:0/497864756 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56179d0913c0 msgr2=0x56179d0b0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:54.041 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.040+0000 7f1206000640 1 --2- 172.21.15.83:0/497864756 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56179d0913c0 0x56179d0b0790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:54.041 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.040+0000 7f1206000640 1 -- 172.21.15.83:0/497864756 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56179d09c5a0 msgr2=0x56179d0bcc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:54.041 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.040+0000 7f1206000640 1 --2- 172.21.15.83:0/497864756 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56179d09c5a0 0x56179d0bcc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:54.041 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.040+0000 7f1206000640 1 -- 172.21.15.83:0/497864756 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56179cdd6620 con 0x56179d091ce0 2024-04-24T08:54:54.041 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.040+0000 7f1206a00640 1 --2- 172.21.15.83:0/497864756 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56179d0913c0 0x56179d0b0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:54:54.042 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.040+0000 7f1206000640 1 --2- 172.21.15.83:0/497864756 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56179d091ce0 0x56179d093d80 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f11f400bdb0 tx=0x7f11f40d39b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebd4f6a925cb8be4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:54.042 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.040+0000 7f11f3400640 1 -- 172.21.15.83:0/497864756 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f409b990 con 0x56179d091ce0 2024-04-24T08:54:54.042 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.041+0000 7f11f3400640 1 -- 172.21.15.83:0/497864756 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f11f4010480 con 0x56179d091ce0 2024-04-24T08:54:54.042 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.041+0000 7f11f3400640 1 -- 172.21.15.83:0/497864756 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f40a4ca0 con 0x56179d091ce0 2024-04-24T08:54:54.045 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.041+0000 7f12109e3740 1 -- 172.21.15.83:0/497864756 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56179d091ce0 msgr2=0x56179d093d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:54.045 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.041+0000 7f12109e3740 1 --2- 172.21.15.83:0/497864756 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56179d091ce0 0x56179d093d80 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f11f400bdb0 tx=0x7f11f40d39b0 comp rx=0 tx=0).stop 2024-04-24T08:54:54.045 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.041+0000 7f12109e3740 1 -- 172.21.15.83:0/497864756 shutdown_connections 2024-04-24T08:54:54.046 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.041+0000 7f12109e3740 1 --2- 172.21.15.83:0/497864756 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56179d09c5a0 0x56179d0bcc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:54.046 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.041+0000 7f12109e3740 1 --2- 172.21.15.83:0/497864756 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56179d091ce0 0x56179d093d80 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:54.046 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.041+0000 7f12109e3740 1 --2- 172.21.15.83:0/497864756 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56179d0913c0 0x56179d0b0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:54.046 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.041+0000 7f12109e3740 1 -- 172.21.15.83:0/497864756 >> 172.21.15.83:0/497864756 conn(0x56179d08d8d0 msgr2=0x56179d088a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:54.046 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.041+0000 7f12109e3740 1 -- 172.21.15.83:0/497864756 shutdown_connections 2024-04-24T08:54:54.046 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.042+0000 7f12109e3740 1 -- 172.21.15.83:0/497864756 wait complete. 2024-04-24T08:54:54.046 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.042+0000 7f12109e3740 1 -- start start 2024-04-24T08:54:54.046 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.042+0000 7f12109e3740 1 Processor -- start 2024-04-24T08:54:54.046 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.043+0000 7f12109e3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56179d0913c0 0x56179d0acd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:54.046 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.043+0000 7f12109e3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56179d091ce0 0x56179d0a8280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:54.046 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.043+0000 7f12109e3740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56179d09c5a0 0x56179d0a87c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:54.046 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.043+0000 7f12109e3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f12081ae780 con 0x56179d091ce0 2024-04-24T08:54:54.047 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.043+0000 7f12109e3740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f12081ae0b0 con 0x56179d09c5a0 2024-04-24T08:54:54.047 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.043+0000 7f12109e3740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f12081ab310 con 0x56179d0913c0 2024-04-24T08:54:54.047 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.043+0000 7f12109e3740 10 client.0 _ll_register_callbacks cb 0x7f1200009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:54:54.047 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.043+0000 7f1206000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56179d091ce0 0x56179d0a8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:54.047 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.043+0000 7f1206000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56179d091ce0 0x56179d0a8280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46998/0 (socket says 172.21.15.83:46998) 2024-04-24T08:54:54.047 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.043+0000 7f1206000640 1 -- 172.21.15.83:0/1741348064 learned_addr learned my addr 172.21.15.83:0/1741348064 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:54:54.047 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.043+0000 7f1207400640 1 --2- 172.21.15.83:0/1741348064 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56179d09c5a0 0x56179d0a87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:54.047 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.043+0000 7f1206a00640 1 --2- 172.21.15.83:0/1741348064 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56179d0913c0 0x56179d0acd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:54.047 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.043+0000 7f1206000640 1 -- 172.21.15.83:0/1741348064 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56179d0913c0 msgr2=0x56179d0acd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:54.047 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.043+0000 7f1206000640 1 --2- 172.21.15.83:0/1741348064 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56179d0913c0 0x56179d0acd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:54.047 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.043+0000 7f1206000640 1 -- 172.21.15.83:0/1741348064 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56179d09c5a0 msgr2=0x56179d0a87c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:54:54.047 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.043+0000 7f1206000640 1 --2- 172.21.15.83:0/1741348064 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56179d09c5a0 0x56179d0a87c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:54.047 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.043+0000 7f1206000640 1 -- 172.21.15.83:0/1741348064 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56179d0a8d00 con 0x56179d091ce0 2024-04-24T08:54:54.047 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.043+0000 7f1206a00640 1 --2- 172.21.15.83:0/1741348064 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56179d0913c0 0x56179d0acd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:54:54.048 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.044+0000 7f1206000640 1 --2- 172.21.15.83:0/1741348064 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56179d091ce0 0x56179d0a8280 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f11f409bd30 tx=0x7f11f409b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a2b8859f75d2d2a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:54:54.048 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.044+0000 7f11f0c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:54:54.048 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.044+0000 7f11f0c00640 1 -- 172.21.15.83:0/1741348064 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f40b4070 con 0x56179d091ce0 2024-04-24T08:54:54.048 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.044+0000 7f11f0c00640 1 -- 172.21.15.83:0/1741348064 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f11f409a330 con 0x56179d091ce0 2024-04-24T08:54:54.048 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.044+0000 7f12109e3740 1 -- 172.21.15.83:0/1741348064 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56179cfff190 con 0x56179d091ce0 2024-04-24T08:54:54.048 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.044+0000 7f11f0c00640 1 -- 172.21.15.83:0/1741348064 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f11f409c030 con 0x56179d091ce0 2024-04-24T08:54:54.048 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.044+0000 7f12109e3740 10 client.16845 fetch_fsmap learned FSMap version 30 2024-04-24T08:54:54.048 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.044+0000 7f12109e3740 1 -- 172.21.15.83:0/1741348064 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56179d11b1a0 con 0x56179d091ce0 2024-04-24T08:54:54.048 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.044+0000 7f11f0c00640 1 -- 172.21.15.83:0/1741348064 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f11f407ae50 con 0x56179d091ce0 2024-04-24T08:54:54.048 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.045+0000 7f11f0c00640 1 -- 172.21.15.83:0/1741348064 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7f11f40cd890 con 0x56179d091ce0 2024-04-24T08:54:54.048 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.045+0000 7f12109e3740 10 client.16845 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T08:54:54.048 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.045+0000 7f12109e3740 10 client.16845 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T08:54:54.048 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.045+0000 7f12109e3740 20 client.16845 populate_metadata read hostname 'smithi083' 2024-04-24T08:54:54.048 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.045+0000 7f12109e3740 1 --2- 172.21.15.83:0/1741348064 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56179d11b600 0x56179d11da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:54:54.049 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.045+0000 7f12109e3740 4 client.16845 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T08:54:54.049 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.045+0000 7f12109e3740 1 -- 172.21.15.83:0/1741348064 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56179cf7b4b0 con 0x56179d11b600 2024-04-24T08:54:54.049 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.045+0000 7f1206a00640 1 --2- 172.21.15.83:0/1741348064 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56179d11b600 0x56179d11da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:54:54.049 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.046+0000 7f1206a00640 1 --2- 172.21.15.83:0/1741348064 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56179d11b600 0x56179d11da20 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4933dcecb343560 server_cookie=e652cc027257fe9d in_seq=0 out_seq=0 2024-04-24T08:54:54.049 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.046+0000 7f11f0c00640 10 client.16845 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:54:54.097 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.097+0000 7f11f0c00640 1 -- 172.21.15.83:0/1741348064 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56179cf7b4b0 con 0x56179d11b600 2024-04-24T08:54:54.097 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.097+0000 7f11f0c00640 10 client.16845 handle_command_reply: tid=0 2024-04-24T08:54:54.097 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.097+0000 7f12109e3740 1 client.16845 shutdown 2024-04-24T08:54:54.098 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.097+0000 7f12109e3740 1 -- 172.21.15.83:0/1741348064 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56179d091ce0 msgr2=0x56179d0a8280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:54.098 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.097+0000 7f12109e3740 1 --2- 172.21.15.83:0/1741348064 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56179d091ce0 0x56179d0a8280 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f11f409bd30 tx=0x7f11f409b990 comp rx=0 tx=0).stop 2024-04-24T08:54:54.098 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.097+0000 7f12109e3740 1 -- 172.21.15.83:0/1741348064 shutdown_connections 2024-04-24T08:54:54.099 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.097+0000 7f12109e3740 1 --2- 172.21.15.83:0/1741348064 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56179d11b600 0x56179d11da20 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:54.099 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.097+0000 7f12109e3740 1 --2- 172.21.15.83:0/1741348064 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56179d09c5a0 0x56179d0a87c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:54.099 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.097+0000 7f12109e3740 1 --2- 172.21.15.83:0/1741348064 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56179d091ce0 0x56179d0a8280 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:54.099 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.097+0000 7f12109e3740 1 --2- 172.21.15.83:0/1741348064 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56179d0913c0 0x56179d0acd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:54.099 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.097+0000 7f11f0c00640 0 client.16845 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:54:54.099 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.097+0000 7f12109e3740 1 -- 172.21.15.83:0/1741348064 >> 172.21.15.83:0/1741348064 conn(0x56179d08d8d0 msgr2=0x56179d08c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:54.099 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.098+0000 7f12109e3740 1 -- 172.21.15.83:0/1741348064 shutdown_connections 2024-04-24T08:54:54.099 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.098+0000 7f12109e3740 1 -- 172.21.15.83:0/1741348064 wait complete. 2024-04-24T08:54:54.099 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.098+0000 7f12109e3740 20 client.16845 trim_cache size 0 max 16384 2024-04-24T08:54:54.100 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T08:54:54.100 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T08:54:54.100 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T08:54:54.100 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T08:54:54.100 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.099+0000 7f1207e00640 1 -- 172.21.15.83:0/886135490 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f11ec0a3d90 msgr2=0x7f11ec0a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:54.101 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.099+0000 7f1207e00640 1 --2- 172.21.15.83:0/886135490 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f11ec0a3d90 0x7f11ec0a6250 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1200009a60 tx=0x7f1200005d80 comp rx=0 tx=0).stop 2024-04-24T08:54:54.101 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.099+0000 7f1207e00640 1 -- 172.21.15.83:0/886135490 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12081038e0 msgr2=0x7f12081a0b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:54:54.101 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.099+0000 7f1207e00640 1 --2- 172.21.15.83:0/886135490 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12081038e0 0x7f12081a0b50 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f11f400b3f0 tx=0x7f11f400b8b0 comp rx=0 tx=0).stop 2024-04-24T08:54:54.101 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.099+0000 7f1207e00640 1 -- 172.21.15.83:0/886135490 shutdown_connections 2024-04-24T08:54:54.101 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.099+0000 7f1207e00640 1 --2- 172.21.15.83:0/886135490 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f11ec0a3d90 0x7f11ec0a6250 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:54.101 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.099+0000 7f1207e00640 1 --2- 172.21.15.83:0/886135490 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f120810c3e0 0x7f120819a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:54.101 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.099+0000 7f1207e00640 1 --2- 172.21.15.83:0/886135490 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12081038e0 0x7f12081a0b50 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:54.101 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.099+0000 7f1207e00640 1 --2- 172.21.15.83:0/886135490 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1208102fc0 0x7f12081a0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:54:54.101 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.099+0000 7f1207e00640 1 -- 172.21.15.83:0/886135490 >> 172.21.15.83:0/886135490 conn(0x7f12080fc820 msgr2=0x7f1208101cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:54:54.101 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.100+0000 7f1207e00640 1 -- 172.21.15.83:0/886135490 shutdown_connections 2024-04-24T08:54:54.101 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:54:54.100+0000 7f1207e00640 1 -- 172.21.15.83:0/886135490 wait complete. 2024-04-24T08:54:54.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:54:53 smithi138 ceph-mon[31441]: pgmap v101: 129 pgs: 129 active+clean; 32 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 579 KiB/s wr, 64 op/s 2024-04-24T08:54:54.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:54:53 smithi189 ceph-mon[31791]: pgmap v101: 129 pgs: 129 active+clean; 32 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 579 KiB/s wr, 64 op/s 2024-04-24T08:54:54.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:54:53 smithi083 ceph-mon[27907]: pgmap v101: 129 pgs: 129 active+clean; 32 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 579 KiB/s wr, 64 op/s 2024-04-24T08:54:54.563 INFO:tasks.workunit.client.0.smithi083.stdout: 1 11100 0.89 MB/sec warmup 88 sec latency 51.007 ms 2024-04-24T08:54:55.563 INFO:tasks.workunit.client.0.smithi083.stdout: 1 11169 0.88 MB/sec warmup 89 sec latency 55.285 ms 2024-04-24T08:54:56.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:54:55 smithi138 ceph-mon[31441]: pgmap v102: 129 pgs: 129 active+clean; 32 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 387 KiB/s wr, 41 op/s 2024-04-24T08:54:56.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:54:55 smithi189 ceph-mon[31791]: pgmap v102: 129 pgs: 129 active+clean; 32 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 387 KiB/s wr, 41 op/s 2024-04-24T08:54:56.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:54:55 smithi083 ceph-mon[27907]: pgmap v102: 129 pgs: 129 active+clean; 32 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 387 KiB/s wr, 41 op/s 2024-04-24T08:54:56.563 INFO:tasks.workunit.client.0.smithi083.stdout: 1 11245 0.87 MB/sec warmup 90 sec latency 55.327 ms 2024-04-24T08:54:57.564 INFO:tasks.workunit.client.0.smithi083.stdout: 1 11343 0.86 MB/sec warmup 91 sec latency 57.321 ms 2024-04-24T08:54:58.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:54:57 smithi138 ceph-mon[31441]: pgmap v103: 129 pgs: 129 active+clean; 32 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 387 KiB/s wr, 41 op/s 2024-04-24T08:54:58.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:54:57 smithi189 ceph-mon[31791]: pgmap v103: 129 pgs: 129 active+clean; 32 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 387 KiB/s wr, 41 op/s 2024-04-24T08:54:58.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:54:57 smithi083 ceph-mon[27907]: pgmap v103: 129 pgs: 129 active+clean; 32 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 387 KiB/s wr, 41 op/s 2024-04-24T08:54:58.564 INFO:tasks.workunit.client.0.smithi083.stdout: 1 11473 0.86 MB/sec warmup 92 sec latency 59.152 ms 2024-04-24T08:54:59.564 INFO:tasks.workunit.client.0.smithi083.stdout: 1 11588 0.85 MB/sec warmup 93 sec latency 58.451 ms 2024-04-24T08:55:00.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:54:59 smithi138 ceph-mon[31441]: pgmap v104: 129 pgs: 129 active+clean; 29 MiB data, 511 MiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 478 KiB/s wr, 67 op/s 2024-04-24T08:55:00.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:54:59 smithi189 ceph-mon[31791]: pgmap v104: 129 pgs: 129 active+clean; 29 MiB data, 511 MiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 478 KiB/s wr, 67 op/s 2024-04-24T08:55:00.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:54:59 smithi083 ceph-mon[27907]: pgmap v104: 129 pgs: 129 active+clean; 29 MiB data, 511 MiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 478 KiB/s wr, 67 op/s 2024-04-24T08:55:00.564 INFO:tasks.workunit.client.0.smithi083.stdout: 1 11705 0.85 MB/sec warmup 94 sec latency 55.426 ms 2024-04-24T08:55:01.564 INFO:tasks.workunit.client.0.smithi083.stdout: 1 11824 0.85 MB/sec warmup 95 sec latency 54.354 ms 2024-04-24T08:55:02.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:55:01 smithi138 ceph-mon[31441]: pgmap v105: 129 pgs: 129 active+clean; 29 MiB data, 511 MiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 307 KiB/s wr, 48 op/s 2024-04-24T08:55:02.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:55:01 smithi189 ceph-mon[31791]: pgmap v105: 129 pgs: 129 active+clean; 29 MiB data, 511 MiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 307 KiB/s wr, 48 op/s 2024-04-24T08:55:02.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:55:01 smithi083 ceph-mon[27907]: pgmap v105: 129 pgs: 129 active+clean; 29 MiB data, 511 MiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 307 KiB/s wr, 48 op/s 2024-04-24T08:55:02.564 INFO:tasks.workunit.client.0.smithi083.stdout: 1 11913 0.84 MB/sec warmup 96 sec latency 81.839 ms 2024-04-24T08:55:03.564 INFO:tasks.workunit.client.0.smithi083.stdout: 1 12096 0.86 MB/sec warmup 97 sec latency 57.234 ms 2024-04-24T08:55:04.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:55:03 smithi138 ceph-mon[31441]: pgmap v106: 129 pgs: 129 active+clean; 28 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 502 KiB/s wr, 93 op/s 2024-04-24T08:55:04.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:55:03 smithi189 ceph-mon[31791]: pgmap v106: 129 pgs: 129 active+clean; 28 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 502 KiB/s wr, 93 op/s 2024-04-24T08:55:04.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:55:03 smithi083 ceph-mon[27907]: pgmap v106: 129 pgs: 129 active+clean; 28 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 502 KiB/s wr, 93 op/s 2024-04-24T08:55:04.564 INFO:tasks.workunit.client.0.smithi083.stdout: 1 12207 0.85 MB/sec warmup 98 sec latency 54.282 ms 2024-04-24T08:55:05.310 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T08:55:05.310 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T08:55:05.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.526+0000 7f6d5ce00640 1 -- 172.21.15.83:0/1450935385 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d58107880 msgr2=0x7f6d58105bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:05.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.526+0000 7f6d5ce00640 1 --2- 172.21.15.83:0/1450935385 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d58107880 0x7f6d58105bc0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f6d44009900 tx=0x7f6d4402a3f0 comp rx=0 tx=0).stop 2024-04-24T08:55:05.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.526+0000 7f6d5ce00640 1 -- 172.21.15.83:0/1450935385 shutdown_connections 2024-04-24T08:55:05.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.526+0000 7f6d5ce00640 1 --2- 172.21.15.83:0/1450935385 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6d581062a0 0x7f6d5810de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:05.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.526+0000 7f6d5ce00640 1 --2- 172.21.15.83:0/1450935385 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d58107880 0x7f6d58105bc0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:05.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.526+0000 7f6d5ce00640 1 --2- 172.21.15.83:0/1450935385 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d58106ed0 0x7f6d581072b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:05.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.526+0000 7f6d5ce00640 1 -- 172.21.15.83:0/1450935385 >> 172.21.15.83:0/1450935385 conn(0x7f6d58077060 msgr2=0x7f6d580fe450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:05.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.527+0000 7f6d5ce00640 1 -- 172.21.15.83:0/1450935385 shutdown_connections 2024-04-24T08:55:05.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.527+0000 7f6d5ce00640 1 -- 172.21.15.83:0/1450935385 wait complete. 2024-04-24T08:55:05.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.528+0000 7f6d5ce00640 1 Processor -- start 2024-04-24T08:55:05.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.528+0000 7f6d5ce00640 1 -- start start 2024-04-24T08:55:05.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.529+0000 7f6d5ce00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6d581062a0 0x7f6d581a4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:05.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.529+0000 7f6d5ce00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d58106ed0 0x7f6d581a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:05.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.529+0000 7f6d5ce00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d5810d090 0x7f6d5810d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:05.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.529+0000 7f6d5ce00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6d581015d0 con 0x7f6d58106ed0 2024-04-24T08:55:05.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.529+0000 7f6d5ce00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6d58101450 con 0x7f6d5810d090 2024-04-24T08:55:05.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.529+0000 7f6d5ce00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6d58101750 con 0x7f6d581062a0 2024-04-24T08:55:05.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.529+0000 7f6d55400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d58106ed0 0x7f6d581a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:05.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.529+0000 7f6d55400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d58106ed0 0x7f6d581a50f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34586/0 (socket says 172.21.15.83:34586) 2024-04-24T08:55:05.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.529+0000 7f6d55400640 1 -- 172.21.15.83:0/3341246062 learned_addr learned my addr 172.21.15.83:0/3341246062 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:55:05.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.529+0000 7f6d55e00640 1 --2- 172.21.15.83:0/3341246062 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6d581062a0 0x7f6d581a4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:05.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.529+0000 7f6d56800640 1 --2- 172.21.15.83:0/3341246062 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d5810d090 0x7f6d5810d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:05.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.530+0000 7f6d55400640 1 -- 172.21.15.83:0/3341246062 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6d581062a0 msgr2=0x7f6d581a4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:05.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.530+0000 7f6d55400640 1 --2- 172.21.15.83:0/3341246062 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6d581062a0 0x7f6d581a4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:05.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.530+0000 7f6d55400640 1 -- 172.21.15.83:0/3341246062 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d5810d090 msgr2=0x7f6d5810d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:05.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.530+0000 7f6d55400640 1 --2- 172.21.15.83:0/3341246062 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d5810d090 0x7f6d5810d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:05.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.530+0000 7f6d55400640 1 -- 172.21.15.83:0/3341246062 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d5810dc10 con 0x7f6d58106ed0 2024-04-24T08:55:05.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.530+0000 7f6d55400640 1 --2- 172.21.15.83:0/3341246062 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d58106ed0 0x7f6d581a50f0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f6d44031440 tx=0x7f6d44003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:05.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.530+0000 7f6d4ea00640 1 -- 172.21.15.83:0/3341246062 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d4403f070 con 0x7f6d58106ed0 2024-04-24T08:55:05.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.530+0000 7f6d5ce00640 1 -- 172.21.15.83:0/3341246062 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d5810dea0 con 0x7f6d58106ed0 2024-04-24T08:55:05.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.530+0000 7f6d4ea00640 1 -- 172.21.15.83:0/3341246062 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6d44031d60 con 0x7f6d58106ed0 2024-04-24T08:55:05.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.530+0000 7f6d4ea00640 1 -- 172.21.15.83:0/3341246062 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d44035030 con 0x7f6d58106ed0 2024-04-24T08:55:05.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.531+0000 7f6d5ce00640 1 -- 172.21.15.83:0/3341246062 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d581abc80 con 0x7f6d58106ed0 2024-04-24T08:55:05.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.534+0000 7f6d5fdd6740 1 Processor -- start 2024-04-24T08:55:05.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.534+0000 7f6d5fdd6740 1 -- start start 2024-04-24T08:55:05.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.534+0000 7f6d4ea00640 1 -- 172.21.15.83:0/3341246062 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6d44042070 con 0x7f6d58106ed0 2024-04-24T08:55:05.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.535+0000 7f6d5fdd6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bb2eef8860 0x55bb2eef8c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:05.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.535+0000 7f6d5fdd6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bb2eef9180 0x55bb2ef03310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:05.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.535+0000 7f6d5fdd6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bb2ef03850 0x55bb2ef05c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:05.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.535+0000 7f6d5fdd6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55bb2ee5e1f0 con 0x55bb2eef8860 2024-04-24T08:55:05.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.535+0000 7f6d5fdd6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55bb2ee04350 con 0x55bb2ef03850 2024-04-24T08:55:05.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.535+0000 7f6d5fdd6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55bb2ee5a1b0 con 0x55bb2eef9180 2024-04-24T08:55:05.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.535+0000 7f6d56800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bb2eef8860 0x55bb2eef8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:05.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.535+0000 7f6d56800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bb2eef8860 0x55bb2eef8c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34598/0 (socket says 172.21.15.83:34598) 2024-04-24T08:55:05.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.535+0000 7f6d56800640 1 -- 172.21.15.83:0/3213049691 learned_addr learned my addr 172.21.15.83:0/3213049691 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:55:05.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.535+0000 7f6d55e00640 1 --2- 172.21.15.83:0/3213049691 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bb2eef9180 0x55bb2ef03310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:05.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.535+0000 7f6d55400640 1 --2- 172.21.15.83:0/3213049691 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bb2ef03850 0x55bb2ef05c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:05.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.536+0000 7f6d56800640 1 -- 172.21.15.83:0/3213049691 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bb2eef9180 msgr2=0x55bb2ef03310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:05.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.536+0000 7f6d56800640 1 --2- 172.21.15.83:0/3213049691 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bb2eef9180 0x55bb2ef03310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:05.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.536+0000 7f6d56800640 1 -- 172.21.15.83:0/3213049691 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bb2ef03850 msgr2=0x55bb2ef05c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:05.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.536+0000 7f6d56800640 1 --2- 172.21.15.83:0/3213049691 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bb2ef03850 0x55bb2ef05c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:05.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.536+0000 7f6d56800640 1 -- 172.21.15.83:0/3213049691 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb2ee58f50 con 0x55bb2eef8860 2024-04-24T08:55:05.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.536+0000 7f6d56800640 1 --2- 172.21.15.83:0/3213049691 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bb2eef8860 0x55bb2eef8c40 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f6d5000ae30 tx=0x7f6d5000e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a26fdff033f13f2f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:05.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.536+0000 7f6d43400640 1 -- 172.21.15.83:0/3213049691 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d50003b70 con 0x55bb2eef8860 2024-04-24T08:55:05.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.536+0000 7f6d43400640 1 -- 172.21.15.83:0/3213049691 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6d50003d10 con 0x55bb2eef8860 2024-04-24T08:55:05.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.536+0000 7f6d4ea00640 1 --2- 172.21.15.83:0/3341246062 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f6d3c0a3d20 0x7f6d3c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:05.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.536+0000 7f6d43400640 1 -- 172.21.15.83:0/3213049691 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d500109e0 con 0x55bb2eef8860 2024-04-24T08:55:05.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.537+0000 7f6d4ea00640 1 -- 172.21.15.83:0/3341246062 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f6d440f1560 con 0x7f6d58106ed0 2024-04-24T08:55:05.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.537+0000 7f6d55e00640 1 --2- 172.21.15.83:0/3341246062 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f6d3c0a3d20 0x7f6d3c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:05.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.537+0000 7f6d5fdd6740 1 -- 172.21.15.83:0/3213049691 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bb2eef8860 msgr2=0x55bb2eef8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:05.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.537+0000 7f6d5fdd6740 1 --2- 172.21.15.83:0/3213049691 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bb2eef8860 0x55bb2eef8c40 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f6d5000ae30 tx=0x7f6d5000e720 comp rx=0 tx=0).stop 2024-04-24T08:55:05.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.537+0000 7f6d5fdd6740 1 -- 172.21.15.83:0/3213049691 shutdown_connections 2024-04-24T08:55:05.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.537+0000 7f6d5fdd6740 1 --2- 172.21.15.83:0/3213049691 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bb2ef03850 0x55bb2ef05c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:05.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.537+0000 7f6d5fdd6740 1 --2- 172.21.15.83:0/3213049691 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bb2eef9180 0x55bb2ef03310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:05.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.537+0000 7f6d5fdd6740 1 --2- 172.21.15.83:0/3213049691 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bb2eef8860 0x55bb2eef8c40 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:05.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.537+0000 7f6d5fdd6740 1 -- 172.21.15.83:0/3213049691 >> 172.21.15.83:0/3213049691 conn(0x55bb2eef4b80 msgr2=0x55bb2eef4f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:05.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.538+0000 7f6d5fdd6740 1 -- 172.21.15.83:0/3213049691 shutdown_connections 2024-04-24T08:55:05.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.538+0000 7f6d55e00640 1 --2- 172.21.15.83:0/3341246062 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f6d3c0a3d20 0x7f6d3c0a61e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6d48005e10 tx=0x7f6d48005da0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:05.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.538+0000 7f6d5fdd6740 1 -- 172.21.15.83:0/3213049691 wait complete. 2024-04-24T08:55:05.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.538+0000 7f6d5fdd6740 1 -- start start 2024-04-24T08:55:05.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.538+0000 7f6d5fdd6740 1 Processor -- start 2024-04-24T08:55:05.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.539+0000 7f6d5fdd6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bb2eef8860 0x55bb2ef007a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:05.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.539+0000 7f6d5fdd6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bb2eef9180 0x55bb2ef00ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:05.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.539+0000 7f6d5fdd6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bb2ef03850 0x55bb2ef01220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:05.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.539+0000 7f6d5fdd6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55bb2ef06640 con 0x55bb2eef8860 2024-04-24T08:55:05.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.539+0000 7f6d5fdd6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55bb2ee17690 con 0x55bb2ef03850 2024-04-24T08:55:05.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.539+0000 7f6d5fdd6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55bb2ef067c0 con 0x55bb2eef9180 2024-04-24T08:55:05.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.539+0000 7f6d5fdd6740 10 client.0 _ll_register_callbacks cb 0x55bb2ee53650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:55:05.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.539+0000 7f6d55e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bb2eef8860 0x55bb2ef007a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:05.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.539+0000 7f6d55e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bb2eef8860 0x55bb2ef007a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34614/0 (socket says 172.21.15.83:34614) 2024-04-24T08:55:05.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.539+0000 7f6d55e00640 1 -- 172.21.15.83:0/4174262068 learned_addr learned my addr 172.21.15.83:0/4174262068 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:55:05.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.539+0000 7f6d56800640 1 --2- 172.21.15.83:0/4174262068 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bb2ef03850 0x55bb2ef01220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:05.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.539+0000 7f6d55400640 1 --2- 172.21.15.83:0/4174262068 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bb2eef9180 0x55bb2ef00ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:05.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.539+0000 7f6d55e00640 1 -- 172.21.15.83:0/4174262068 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bb2eef9180 msgr2=0x55bb2ef00ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:05.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.539+0000 7f6d55e00640 1 --2- 172.21.15.83:0/4174262068 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bb2eef9180 0x55bb2ef00ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:05.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.539+0000 7f6d55e00640 1 -- 172.21.15.83:0/4174262068 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bb2ef03850 msgr2=0x55bb2ef01220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:05.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.539+0000 7f6d55e00640 1 --2- 172.21.15.83:0/4174262068 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bb2ef03850 0x55bb2ef01220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:05.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.539+0000 7f6d55e00640 1 -- 172.21.15.83:0/4174262068 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb2ee16920 con 0x55bb2eef8860 2024-04-24T08:55:05.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.539+0000 7f6d56800640 1 --2- 172.21.15.83:0/4174262068 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bb2ef03850 0x55bb2ef01220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:55:05.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.540+0000 7f6d55e00640 1 --2- 172.21.15.83:0/4174262068 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bb2eef8860 0x55bb2ef007a0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f6d48011b00 tx=0x7f6d480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b98a45c1e3ad3b96 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:05.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.540+0000 7f6d37e00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:55:05.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.540+0000 7f6d37e00640 1 -- 172.21.15.83:0/4174262068 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d4800ee10 con 0x55bb2eef8860 2024-04-24T08:55:05.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.540+0000 7f6d37e00640 1 -- 172.21.15.83:0/4174262068 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f6d4801e420 con 0x55bb2eef8860 2024-04-24T08:55:05.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.540+0000 7f6d5fdd6740 1 -- 172.21.15.83:0/4174262068 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d50003d10 con 0x55bb2eef8860 2024-04-24T08:55:05.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.540+0000 7f6d37e00640 1 -- 172.21.15.83:0/4174262068 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6d4801e620 con 0x55bb2eef8860 2024-04-24T08:55:05.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.540+0000 7f6d5fdd6740 10 client.16857 fetch_fsmap learned FSMap version 30 2024-04-24T08:55:05.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.540+0000 7f6d5fdd6740 1 -- 172.21.15.83:0/4174262068 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55bb2ec3a7d0 con 0x55bb2eef8860 2024-04-24T08:55:05.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.541+0000 7f6d37e00640 1 -- 172.21.15.83:0/4174262068 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6d50003d10 con 0x55bb2eef8860 2024-04-24T08:55:05.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.541+0000 7f6d37e00640 1 -- 172.21.15.83:0/4174262068 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7f6d48026930 con 0x55bb2eef8860 2024-04-24T08:55:05.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.541+0000 7f6d5fdd6740 10 client.16857 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T08:55:05.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.541+0000 7f6d5fdd6740 10 client.16857 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:55:05.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.541+0000 7f6d5fdd6740 20 client.16857 populate_metadata read hostname 'smithi083' 2024-04-24T08:55:05.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.541+0000 7f6d5fdd6740 1 --2- 172.21.15.83:0/4174262068 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55bb2ef13c90 0x55bb2ef16130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:05.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.541+0000 7f6d5fdd6740 4 client.16857 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:55:05.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.541+0000 7f6d5fdd6740 1 -- 172.21.15.83:0/4174262068 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bb2ee6bc10 con 0x55bb2ef13c90 2024-04-24T08:55:05.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.542+0000 7f6d55400640 1 --2- 172.21.15.83:0/4174262068 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55bb2ef13c90 0x55bb2ef16130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:05.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.543+0000 7f6d55400640 1 --2- 172.21.15.83:0/4174262068 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55bb2ef13c90 0x55bb2ef16130 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64ffedfca8c4b055 server_cookie=2504ed33b8ce94a8 in_seq=0 out_seq=0 2024-04-24T08:55:05.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.543+0000 7f6d37e00640 10 client.16857 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:55:05.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.545+0000 7f6d37e00640 1 -- 172.21.15.83:0/4174262068 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bb2ee6bc10 con 0x55bb2ef13c90 2024-04-24T08:55:05.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.545+0000 7f6d37e00640 10 client.16857 handle_command_reply: tid=0 2024-04-24T08:55:05.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.545+0000 7f6d5fdd6740 1 client.16857 shutdown 2024-04-24T08:55:05.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.545+0000 7f6d5fdd6740 1 -- 172.21.15.83:0/4174262068 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bb2eef8860 msgr2=0x55bb2ef007a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:05.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.545+0000 7f6d5fdd6740 1 --2- 172.21.15.83:0/4174262068 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bb2eef8860 0x55bb2ef007a0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f6d48011b00 tx=0x7f6d480046c0 comp rx=0 tx=0).stop 2024-04-24T08:55:05.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.545+0000 7f6d5fdd6740 1 -- 172.21.15.83:0/4174262068 shutdown_connections 2024-04-24T08:55:05.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.545+0000 7f6d5fdd6740 1 --2- 172.21.15.83:0/4174262068 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55bb2ef13c90 0x55bb2ef16130 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:05.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.545+0000 7f6d5fdd6740 1 --2- 172.21.15.83:0/4174262068 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bb2ef03850 0x55bb2ef01220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:05.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.545+0000 7f6d5fdd6740 1 --2- 172.21.15.83:0/4174262068 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bb2eef9180 0x55bb2ef00ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:05.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.545+0000 7f6d5fdd6740 1 --2- 172.21.15.83:0/4174262068 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bb2eef8860 0x55bb2ef007a0 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:05.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.545+0000 7f6d5fdd6740 1 -- 172.21.15.83:0/4174262068 >> 172.21.15.83:0/4174262068 conn(0x55bb2eef4b80 msgr2=0x55bb2eefa880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:05.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.545+0000 7f6d37e00640 0 client.16857 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:55:05.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.545+0000 7f6d5fdd6740 1 -- 172.21.15.83:0/4174262068 shutdown_connections 2024-04-24T08:55:05.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.545+0000 7f6d5fdd6740 1 -- 172.21.15.83:0/4174262068 wait complete. 2024-04-24T08:55:05.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.545+0000 7f6d5fdd6740 20 client.16857 trim_cache size 0 max 16384 2024-04-24T08:55:05.565 INFO:tasks.workunit.client.0.smithi083.stdout: 1 12337 0.84 MB/sec warmup 99 sec latency 55.878 ms 2024-04-24T08:55:05.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.569+0000 7f6d5fdd6740 1 Processor -- start 2024-04-24T08:55:05.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.569+0000 7f6d5fdd6740 1 -- start start 2024-04-24T08:55:05.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.570+0000 7f6d5fdd6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bb2eef8860 0x55bb2ef0f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:05.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.570+0000 7f6d5fdd6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bb2eef9180 0x55bb2eeff900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:05.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.570+0000 7f6d5fdd6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bb2ef03850 0x55bb2ef00e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:05.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.570+0000 7f6d5fdd6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6d581b5b70 con 0x55bb2eef9180 2024-04-24T08:55:05.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.570+0000 7f6d5fdd6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6d581b7740 con 0x55bb2eef8860 2024-04-24T08:55:05.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.570+0000 7f6d5fdd6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6d581b6240 con 0x55bb2ef03850 2024-04-24T08:55:05.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.570+0000 7f6d55400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bb2eef9180 0x55bb2eeff900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:05.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.570+0000 7f6d55400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bb2eef9180 0x55bb2eeff900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34618/0 (socket says 172.21.15.83:34618) 2024-04-24T08:55:05.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.570+0000 7f6d55400640 1 -- 172.21.15.83:0/3710767015 learned_addr learned my addr 172.21.15.83:0/3710767015 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:55:05.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.570+0000 7f6d55e00640 1 --2- 172.21.15.83:0/3710767015 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bb2eef8860 0x55bb2ef0f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:05.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.570+0000 7f6d56800640 1 --2- 172.21.15.83:0/3710767015 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bb2ef03850 0x55bb2ef00e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:05.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.571+0000 7f6d55400640 1 -- 172.21.15.83:0/3710767015 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bb2ef03850 msgr2=0x55bb2ef00e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:05.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.571+0000 7f6d55400640 1 --2- 172.21.15.83:0/3710767015 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bb2ef03850 0x55bb2ef00e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:05.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.571+0000 7f6d55400640 1 -- 172.21.15.83:0/3710767015 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bb2eef8860 msgr2=0x55bb2ef0f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:05.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.571+0000 7f6d55400640 1 --2- 172.21.15.83:0/3710767015 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bb2eef8860 0x55bb2ef0f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:05.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.571+0000 7f6d55400640 1 -- 172.21.15.83:0/3710767015 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb2eb703f0 con 0x55bb2eef9180 2024-04-24T08:55:05.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.571+0000 7f6d55400640 1 --2- 172.21.15.83:0/3710767015 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bb2eef9180 0x55bb2eeff900 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f6d440b9aa0 tx=0x7f6d44031f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48f7c7a962ccc4f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:05.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.571+0000 7f6d42a00640 1 -- 172.21.15.83:0/3710767015 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d44049040 con 0x55bb2eef9180 2024-04-24T08:55:05.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.571+0000 7f6d42a00640 1 -- 172.21.15.83:0/3710767015 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6d440c1480 con 0x55bb2eef9180 2024-04-24T08:55:05.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.571+0000 7f6d42a00640 1 -- 172.21.15.83:0/3710767015 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d440f4ca0 con 0x55bb2eef9180 2024-04-24T08:55:05.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.572+0000 7f6d5fdd6740 1 -- 172.21.15.83:0/3710767015 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bb2eef9180 msgr2=0x55bb2eeff900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:05.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.572+0000 7f6d5fdd6740 1 --2- 172.21.15.83:0/3710767015 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bb2eef9180 0x55bb2eeff900 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f6d440b9aa0 tx=0x7f6d44031f10 comp rx=0 tx=0).stop 2024-04-24T08:55:05.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.572+0000 7f6d5fdd6740 1 -- 172.21.15.83:0/3710767015 shutdown_connections 2024-04-24T08:55:05.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.572+0000 7f6d5fdd6740 1 --2- 172.21.15.83:0/3710767015 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bb2ef03850 0x55bb2ef00e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:05.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.572+0000 7f6d5fdd6740 1 --2- 172.21.15.83:0/3710767015 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bb2eef9180 0x55bb2eeff900 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:05.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.572+0000 7f6d5fdd6740 1 --2- 172.21.15.83:0/3710767015 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bb2eef8860 0x55bb2ef0f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:05.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.572+0000 7f6d5fdd6740 1 -- 172.21.15.83:0/3710767015 >> 172.21.15.83:0/3710767015 conn(0x55bb2eef4b80 msgr2=0x55bb2ef00760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:05.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.572+0000 7f6d5fdd6740 1 -- 172.21.15.83:0/3710767015 shutdown_connections 2024-04-24T08:55:05.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.572+0000 7f6d5fdd6740 1 -- 172.21.15.83:0/3710767015 wait complete. 2024-04-24T08:55:05.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.572+0000 7f6d5fdd6740 1 -- start start 2024-04-24T08:55:05.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.573+0000 7f6d5fdd6740 1 Processor -- start 2024-04-24T08:55:05.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.573+0000 7f6d5fdd6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bb2eef8860 0x55bb2ef06a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:05.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.573+0000 7f6d5fdd6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bb2eef9180 0x55bb2ef06f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:05.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.573+0000 7f6d5fdd6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bb2ef03850 0x55bb2ef782a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:05.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.573+0000 7f6d5fdd6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6d581b6fe0 con 0x55bb2eef8860 2024-04-24T08:55:05.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.573+0000 7f6d5fdd6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6d581b6910 con 0x55bb2ef03850 2024-04-24T08:55:05.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.573+0000 7f6d5fdd6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6d581b3b50 con 0x55bb2eef9180 2024-04-24T08:55:05.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.573+0000 7f6d5fdd6740 10 client.0 _ll_register_callbacks cb 0x7f6d4800c8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:55:05.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.573+0000 7f6d55e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bb2eef8860 0x55bb2ef06a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:05.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.573+0000 7f6d55e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bb2eef8860 0x55bb2ef06a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34634/0 (socket says 172.21.15.83:34634) 2024-04-24T08:55:05.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.573+0000 7f6d55e00640 1 -- 172.21.15.83:0/915138916 learned_addr learned my addr 172.21.15.83:0/915138916 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:55:05.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.574+0000 7f6d55400640 1 --2- 172.21.15.83:0/915138916 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bb2eef9180 0x55bb2ef06f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:05.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.574+0000 7f6d56800640 1 --2- 172.21.15.83:0/915138916 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bb2ef03850 0x55bb2ef782a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:05.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.574+0000 7f6d55e00640 1 -- 172.21.15.83:0/915138916 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bb2eef9180 msgr2=0x55bb2ef06f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:05.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.574+0000 7f6d55e00640 1 --2- 172.21.15.83:0/915138916 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bb2eef9180 0x55bb2ef06f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:05.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.574+0000 7f6d55e00640 1 -- 172.21.15.83:0/915138916 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bb2ef03850 msgr2=0x55bb2ef782a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:05.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.574+0000 7f6d55e00640 1 --2- 172.21.15.83:0/915138916 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bb2ef03850 0x55bb2ef782a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:05.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.574+0000 7f6d55e00640 1 -- 172.21.15.83:0/915138916 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb2eb300f0 con 0x55bb2eef8860 2024-04-24T08:55:05.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.574+0000 7f6d55e00640 1 --2- 172.21.15.83:0/915138916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bb2eef8860 0x55bb2ef06a30 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f6d48027fd0 tx=0x7f6d480052f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6b54d8af719435f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:05.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.574+0000 7f6d37e00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:55:05.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.574+0000 7f6d37e00640 1 -- 172.21.15.83:0/915138916 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d4800e790 con 0x55bb2eef8860 2024-04-24T08:55:05.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.575+0000 7f6d37e00640 1 -- 172.21.15.83:0/915138916 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f6d480054e0 con 0x55bb2eef8860 2024-04-24T08:55:05.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.575+0000 7f6d5fdd6740 1 -- 172.21.15.83:0/915138916 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d440c1480 con 0x55bb2eef8860 2024-04-24T08:55:05.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.575+0000 7f6d37e00640 1 -- 172.21.15.83:0/915138916 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6d4801e7c0 con 0x55bb2eef8860 2024-04-24T08:55:05.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.575+0000 7f6d5fdd6740 10 client.16863 fetch_fsmap learned FSMap version 30 2024-04-24T08:55:05.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.575+0000 7f6d5fdd6740 1 -- 172.21.15.83:0/915138916 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55bb2ee23810 con 0x55bb2eef8860 2024-04-24T08:55:05.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.575+0000 7f6d37e00640 1 -- 172.21.15.83:0/915138916 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6d440c1480 con 0x55bb2eef8860 2024-04-24T08:55:05.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.575+0000 7f6d37e00640 1 -- 172.21.15.83:0/915138916 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7f6d48004b10 con 0x55bb2eef8860 2024-04-24T08:55:05.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.575+0000 7f6d5fdd6740 10 client.16863 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T08:55:05.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.575+0000 7f6d5fdd6740 10 client.16863 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:55:05.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.575+0000 7f6d5fdd6740 20 client.16863 populate_metadata read hostname 'smithi083' 2024-04-24T08:55:05.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.575+0000 7f6d5fdd6740 1 --2- 172.21.15.83:0/915138916 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55bb2ef79e20 0x55bb2ef7c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:05.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.575+0000 7f6d5fdd6740 4 client.16863 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T08:55:05.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.575+0000 7f6d5fdd6740 1 -- 172.21.15.83:0/915138916 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55bb2ee65a80 con 0x55bb2ef79e20 2024-04-24T08:55:05.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.576+0000 7f6d55400640 1 --2- 172.21.15.83:0/915138916 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55bb2ef79e20 0x55bb2ef7c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:05.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.577+0000 7f6d55400640 1 --2- 172.21.15.83:0/915138916 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55bb2ef79e20 0x55bb2ef7c240 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3099e22a4b37d4f0 server_cookie=4d0ca86e3fda4f6d in_seq=0 out_seq=0 2024-04-24T08:55:05.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.577+0000 7f6d37e00640 10 client.16863 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:55:05.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.578+0000 7f6d37e00640 1 -- 172.21.15.83:0/915138916 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55bb2ee65a80 con 0x55bb2ef79e20 2024-04-24T08:55:05.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.578+0000 7f6d37e00640 10 client.16863 handle_command_reply: tid=0 2024-04-24T08:55:05.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.578+0000 7f6d5fdd6740 1 client.16863 shutdown 2024-04-24T08:55:05.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.578+0000 7f6d5fdd6740 1 -- 172.21.15.83:0/915138916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bb2eef8860 msgr2=0x55bb2ef06a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:05.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.578+0000 7f6d5fdd6740 1 --2- 172.21.15.83:0/915138916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bb2eef8860 0x55bb2ef06a30 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f6d48027fd0 tx=0x7f6d480052f0 comp rx=0 tx=0).stop 2024-04-24T08:55:05.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.578+0000 7f6d5fdd6740 1 -- 172.21.15.83:0/915138916 shutdown_connections 2024-04-24T08:55:05.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.578+0000 7f6d5fdd6740 1 --2- 172.21.15.83:0/915138916 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55bb2ef79e20 0x55bb2ef7c240 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:05.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.578+0000 7f6d5fdd6740 1 --2- 172.21.15.83:0/915138916 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bb2ef03850 0x55bb2ef782a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:05.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.578+0000 7f6d5fdd6740 1 --2- 172.21.15.83:0/915138916 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bb2eef9180 0x55bb2ef06f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:05.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.578+0000 7f6d37e00640 0 client.16863 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:55:05.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.578+0000 7f6d5fdd6740 1 --2- 172.21.15.83:0/915138916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bb2eef8860 0x55bb2ef06a30 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:05.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.578+0000 7f6d5fdd6740 1 -- 172.21.15.83:0/915138916 >> 172.21.15.83:0/915138916 conn(0x55bb2eef4b80 msgr2=0x55bb2eef3390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:05.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.578+0000 7f6d5fdd6740 1 -- 172.21.15.83:0/915138916 shutdown_connections 2024-04-24T08:55:05.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.578+0000 7f6d5fdd6740 1 -- 172.21.15.83:0/915138916 wait complete. 2024-04-24T08:55:05.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.578+0000 7f6d5fdd6740 20 client.16863 trim_cache size 0 max 16384 2024-04-24T08:55:05.583 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T08:55:05.583 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-04-24T08:55:05.583 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-04-24T08:55:05.583 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T08:55:05.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.579+0000 7f6d5ce00640 1 -- 172.21.15.83:0/3341246062 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f6d3c0a3d20 msgr2=0x7f6d3c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:05.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.579+0000 7f6d5ce00640 1 --2- 172.21.15.83:0/3341246062 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f6d3c0a3d20 0x7f6d3c0a61e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6d48005e10 tx=0x7f6d48005da0 comp rx=0 tx=0).stop 2024-04-24T08:55:05.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.579+0000 7f6d5ce00640 1 -- 172.21.15.83:0/3341246062 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d58106ed0 msgr2=0x7f6d581a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:05.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.579+0000 7f6d5ce00640 1 --2- 172.21.15.83:0/3341246062 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d58106ed0 0x7f6d581a50f0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f6d44031440 tx=0x7f6d44003d50 comp rx=0 tx=0).stop 2024-04-24T08:55:05.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.579+0000 7f6d5ce00640 1 -- 172.21.15.83:0/3341246062 shutdown_connections 2024-04-24T08:55:05.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.579+0000 7f6d5ce00640 1 --2- 172.21.15.83:0/3341246062 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f6d3c0a3d20 0x7f6d3c0a61e0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:05.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.579+0000 7f6d5ce00640 1 --2- 172.21.15.83:0/3341246062 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d5810d090 0x7f6d5810d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:05.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.579+0000 7f6d5ce00640 1 --2- 172.21.15.83:0/3341246062 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d58106ed0 0x7f6d581a50f0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:05.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.579+0000 7f6d5ce00640 1 --2- 172.21.15.83:0/3341246062 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6d581062a0 0x7f6d581a4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:05.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.579+0000 7f6d5ce00640 1 -- 172.21.15.83:0/3341246062 >> 172.21.15.83:0/3341246062 conn(0x7f6d58077060 msgr2=0x7f6d58111240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:05.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.579+0000 7f6d5ce00640 1 -- 172.21.15.83:0/3341246062 shutdown_connections 2024-04-24T08:55:05.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.579+0000 7f6d5ce00640 1 -- 172.21.15.83:0/3341246062 wait complete. 2024-04-24T08:55:05.596 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T08:55:05.596 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T08:55:05.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.796+0000 7f3053000640 1 -- 172.21.15.83:0/2359670080 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f304c074cd0 msgr2=0x7f304c10e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:05.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.796+0000 7f3053000640 1 --2- 172.21.15.83:0/2359670080 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f304c074cd0 0x7f304c10e1a0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f3038009900 tx=0x7f303802a480 comp rx=0 tx=0).stop 2024-04-24T08:55:05.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.796+0000 7f3053000640 1 -- 172.21.15.83:0/2359670080 shutdown_connections 2024-04-24T08:55:05.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.796+0000 7f3053000640 1 --2- 172.21.15.83:0/2359670080 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f304c10e6e0 0x7f304c110ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:05.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.796+0000 7f3053000640 1 --2- 172.21.15.83:0/2359670080 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f304c074cd0 0x7f304c10e1a0 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:05.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.796+0000 7f3053000640 1 --2- 172.21.15.83:0/2359670080 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f304c0743b0 0x7f304c074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:05.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.796+0000 7f3053000640 1 -- 172.21.15.83:0/2359670080 >> 172.21.15.83:0/2359670080 conn(0x7f304c0fd930 msgr2=0x7f304c0ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:05.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.796+0000 7f3053000640 1 -- 172.21.15.83:0/2359670080 shutdown_connections 2024-04-24T08:55:05.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.796+0000 7f3053000640 1 -- 172.21.15.83:0/2359670080 wait complete. 2024-04-24T08:55:05.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.797+0000 7f3053000640 1 Processor -- start 2024-04-24T08:55:05.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.797+0000 7f3053000640 1 -- start start 2024-04-24T08:55:05.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.797+0000 7f3053000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f304c0743b0 0x7f304c1a05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:05.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.797+0000 7f3053000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f304c074cd0 0x7f304c1a0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:05.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.797+0000 7f3053000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f304c10e6e0 0x7f304c19a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:05.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.797+0000 7f3053000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f304c1142d0 con 0x7f304c10e6e0 2024-04-24T08:55:05.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.797+0000 7f3053000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f304c114150 con 0x7f304c0743b0 2024-04-24T08:55:05.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.797+0000 7f3053000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f304c114450 con 0x7f304c074cd0 2024-04-24T08:55:05.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.798+0000 7f3052600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f304c10e6e0 0x7f304c19a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:05.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.798+0000 7f3052600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f304c10e6e0 0x7f304c19a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34646/0 (socket says 172.21.15.83:34646) 2024-04-24T08:55:05.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.798+0000 7f3052600640 1 -- 172.21.15.83:0/1290071886 learned_addr learned my addr 172.21.15.83:0/1290071886 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:55:05.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.798+0000 7f3051c00640 1 --2- 172.21.15.83:0/1290071886 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f304c0743b0 0x7f304c1a05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:05.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.798+0000 7f3051200640 1 --2- 172.21.15.83:0/1290071886 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f304c074cd0 0x7f304c1a0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:05.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.798+0000 7f3052600640 1 -- 172.21.15.83:0/1290071886 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f304c074cd0 msgr2=0x7f304c1a0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:05.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.798+0000 7f3052600640 1 --2- 172.21.15.83:0/1290071886 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f304c074cd0 0x7f304c1a0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:05.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.798+0000 7f3052600640 1 -- 172.21.15.83:0/1290071886 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f304c0743b0 msgr2=0x7f304c1a05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:05.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.798+0000 7f3052600640 1 --2- 172.21.15.83:0/1290071886 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f304c0743b0 0x7f304c1a05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:05.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.798+0000 7f3052600640 1 -- 172.21.15.83:0/1290071886 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f304c19af50 con 0x7f304c10e6e0 2024-04-24T08:55:05.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.798+0000 7f3051c00640 1 --2- 172.21.15.83:0/1290071886 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f304c0743b0 0x7f304c1a05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:55:05.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.799+0000 7f3052600640 1 --2- 172.21.15.83:0/1290071886 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f304c10e6e0 0x7f304c19a690 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f304400eaa0 tx=0x7f304400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:05.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.799+0000 7f304a000640 1 -- 172.21.15.83:0/1290071886 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f304400cdb0 con 0x7f304c10e6e0 2024-04-24T08:55:05.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.799+0000 7f304a000640 1 -- 172.21.15.83:0/1290071886 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3044005020 con 0x7f304c10e6e0 2024-04-24T08:55:05.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.799+0000 7f304a000640 1 -- 172.21.15.83:0/1290071886 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30440116d0 con 0x7f304c10e6e0 2024-04-24T08:55:05.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.799+0000 7f3053000640 1 -- 172.21.15.83:0/1290071886 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f304c19b240 con 0x7f304c10e6e0 2024-04-24T08:55:05.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.799+0000 7f3053000640 1 -- 172.21.15.83:0/1290071886 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f304c1a7550 con 0x7f304c10e6e0 2024-04-24T08:55:05.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.800+0000 7f3053000640 1 -- 172.21.15.83:0/1290071886 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f304c075a60 con 0x7f304c10e6e0 2024-04-24T08:55:05.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.802+0000 7f304a000640 1 -- 172.21.15.83:0/1290071886 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f30440051d0 con 0x7f304c10e6e0 2024-04-24T08:55:05.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.804+0000 7f304a000640 1 --2- 172.21.15.83:0/1290071886 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f30340a3d20 0x7f30340a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:05.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.804+0000 7f304a000640 1 -- 172.21.15.83:0/1290071886 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f30440d0e80 con 0x7f304c10e6e0 2024-04-24T08:55:05.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.804+0000 7f3051c00640 1 --2- 172.21.15.83:0/1290071886 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f30340a3d20 0x7f30340a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:05.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.805+0000 7f3051c00640 1 --2- 172.21.15.83:0/1290071886 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f30340a3d20 0x7f30340a61e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f30400058c0 tx=0x7f3040005850 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:05.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:05.809+0000 7f304a000640 1 -- 172.21.15.83:0/1290071886 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3044016030 con 0x7f304c10e6e0 2024-04-24T08:55:06.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:55:05 smithi083 ceph-mon[27907]: pgmap v107: 129 pgs: 129 active+clean; 28 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 285 KiB/s wr, 70 op/s 2024-04-24T08:55:06.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.069+0000 7f3053000640 1 -- 172.21.15.83:0/1290071886 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f304c19b470 con 0x7f304c10e6e0 2024-04-24T08:55:06.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.071+0000 7f304a000640 1 -- 172.21.15.83:0/1290071886 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11949 (secure 0 0 0) 0x7f304409d0f0 con 0x7f304c10e6e0 2024-04-24T08:55:06.071 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:55:06.072 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/358872257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":358872257},{"type":"v1","addr":"172.21.15.83:6835","nonce":358872257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3157070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3157070851},{"type":"v1","addr":"172.21.15.83:6837","nonce":3157070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T08:48:29.422576+0000","modified":"2024-04-24T08:53:42.044317+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34520,"mds_1":34493,"mds_2":34451,"mds_3":34427,"mds_4":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14658":{"gid":14658,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6839/4085470844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":4085470844},{"type":"v1","addr":"172.21.15.83:6839","nonce":4085470844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14676":{"gid":14676,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6841/2634754210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":2634754210},{"type":"v1","addr":"172.21.15.83:6841","nonce":2634754210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24382":{"gid":24382,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/1859921310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1859921310},{"type":"v1","addr":"172.21.15.189:6833","nonce":1859921310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/2241923197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2241923197},{"type":"v1","addr":"172.21.15.189:6835","nonce":2241923197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6837/3172311220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":3172311220},{"type":"v1","addr":"172.21.15.189:6837","nonce":3172311220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.189:6839/2548965336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":2548965336},{"type":"v1","addr":"172.21.15.189:6839","nonce":2548965336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.138:6833/1157967658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1157967658},{"type":"v1","addr":"172.21.15.138:6833","nonce":1157967658}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.138:6835/1909219758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1909219758},{"type":"v1","addr":"172.21.15.138:6835","nonce":1909219758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34493":{"gid":34493,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.138:6837/900566754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":900566754},{"type":"v1","addr":"172.21.15.138:6837","nonce":900566754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34520":{"gid":34520,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.138:6839/2236932393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2236932393},{"type":"v1","addr":"172.21.15.138:6839","nonce":2236932393}]},"join_fscid":-1,"export_targets":[1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34520,"qdb_cluster":[34493,34451,34427,24454,34520]},"id":1}]} 2024-04-24T08:55:06.072 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 30 2024-04-24T08:55:06.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.074+0000 7f3053000640 1 -- 172.21.15.83:0/1290071886 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f30340a3d20 msgr2=0x7f30340a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:06.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.074+0000 7f3053000640 1 --2- 172.21.15.83:0/1290071886 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f30340a3d20 0x7f30340a61e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f30400058c0 tx=0x7f3040005850 comp rx=0 tx=0).stop 2024-04-24T08:55:06.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.074+0000 7f3053000640 1 -- 172.21.15.83:0/1290071886 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f304c10e6e0 msgr2=0x7f304c19a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:06.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.074+0000 7f3053000640 1 --2- 172.21.15.83:0/1290071886 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f304c10e6e0 0x7f304c19a690 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f304400eaa0 tx=0x7f304400ef60 comp rx=0 tx=0).stop 2024-04-24T08:55:06.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.074+0000 7f3053000640 1 -- 172.21.15.83:0/1290071886 shutdown_connections 2024-04-24T08:55:06.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.074+0000 7f3053000640 1 --2- 172.21.15.83:0/1290071886 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f30340a3d20 0x7f30340a61e0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:06.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.074+0000 7f3053000640 1 --2- 172.21.15.83:0/1290071886 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f304c10e6e0 0x7f304c19a690 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:06.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.074+0000 7f3053000640 1 --2- 172.21.15.83:0/1290071886 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f304c074cd0 0x7f304c1a0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:06.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.074+0000 7f3053000640 1 --2- 172.21.15.83:0/1290071886 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f304c0743b0 0x7f304c1a05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:06.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.074+0000 7f3053000640 1 -- 172.21.15.83:0/1290071886 >> 172.21.15.83:0/1290071886 conn(0x7f304c0fd930 msgr2=0x7f304c0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:06.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.074+0000 7f3053000640 1 -- 172.21.15.83:0/1290071886 shutdown_connections 2024-04-24T08:55:06.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.074+0000 7f3053000640 1 -- 172.21.15.83:0/1290071886 wait complete. 2024-04-24T08:55:06.096 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-24T08:55:06.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:55:05 smithi138 ceph-mon[31441]: pgmap v107: 129 pgs: 129 active+clean; 28 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 285 KiB/s wr, 70 op/s 2024-04-24T08:55:06.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:55:05 smithi189 ceph-mon[31791]: pgmap v107: 129 pgs: 129 active+clean; 28 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 285 KiB/s wr, 70 op/s 2024-04-24T08:55:06.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.307+0000 7f75d3e00640 1 -- 172.21.15.83:0/1960448065 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c8031880 con 0x7f75d4104320 2024-04-24T08:55:06.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.307+0000 7f75dac00640 1 -- 172.21.15.83:0/1960448065 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75d4104320 msgr2=0x7f75d4104700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:06.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.307+0000 7f75dac00640 1 --2- 172.21.15.83:0/1960448065 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75d4104320 0x7f75d4104700 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f75c8009900 tx=0x7f75c802a3f0 comp rx=0 tx=0).stop 2024-04-24T08:55:06.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.307+0000 7f75dac00640 1 -- 172.21.15.83:0/1960448065 shutdown_connections 2024-04-24T08:55:06.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.307+0000 7f75dac00640 1 --2- 172.21.15.83:0/1960448065 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f75d41093f0 0x7f75d4110fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:06.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.307+0000 7f75dac00640 1 --2- 172.21.15.83:0/1960448065 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f75d4104cd0 0x7f75d4108d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:06.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.307+0000 7f75dac00640 1 --2- 172.21.15.83:0/1960448065 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75d4104320 0x7f75d4104700 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:06.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.307+0000 7f75dac00640 1 -- 172.21.15.83:0/1960448065 >> 172.21.15.83:0/1960448065 conn(0x7f75d40ffb40 msgr2=0x7f75d4101f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:06.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.308+0000 7f75dac00640 1 -- 172.21.15.83:0/1960448065 shutdown_connections 2024-04-24T08:55:06.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.308+0000 7f75dac00640 1 -- 172.21.15.83:0/1960448065 wait complete. 2024-04-24T08:55:06.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.309+0000 7f75dac00640 1 Processor -- start 2024-04-24T08:55:06.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.309+0000 7f75dac00640 1 -- start start 2024-04-24T08:55:06.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.309+0000 7f75dac00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75d4104320 0x7f75d41a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:06.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.309+0000 7f75dac00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f75d41093f0 0x7f75d41a0b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:06.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.309+0000 7f75dac00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f75d419a6b0 0x7f75d419ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:06.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.309+0000 7f75dac00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f75d41143a0 con 0x7f75d4104320 2024-04-24T08:55:06.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.309+0000 7f75dac00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f75d4114220 con 0x7f75d419a6b0 2024-04-24T08:55:06.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.309+0000 7f75dac00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f75d4114520 con 0x7f75d41093f0 2024-04-24T08:55:06.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.309+0000 7f75d9800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75d4104320 0x7f75d41a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:06.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.309+0000 7f75d9800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75d4104320 0x7f75d41a05f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34668/0 (socket says 172.21.15.83:34668) 2024-04-24T08:55:06.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.310+0000 7f75d9800640 1 -- 172.21.15.83:0/477772030 learned_addr learned my addr 172.21.15.83:0/477772030 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:55:06.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.310+0000 7f75d8e00640 1 --2- 172.21.15.83:0/477772030 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f75d41093f0 0x7f75d41a0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:06.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.310+0000 7f75da200640 1 --2- 172.21.15.83:0/477772030 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f75d419a6b0 0x7f75d419ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:06.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.310+0000 7f75d9800640 1 -- 172.21.15.83:0/477772030 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f75d41093f0 msgr2=0x7f75d41a0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:06.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.310+0000 7f75d9800640 1 --2- 172.21.15.83:0/477772030 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f75d41093f0 0x7f75d41a0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:06.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.310+0000 7f75d9800640 1 -- 172.21.15.83:0/477772030 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f75d419a6b0 msgr2=0x7f75d419ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:06.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.310+0000 7f75d9800640 1 --2- 172.21.15.83:0/477772030 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f75d419a6b0 0x7f75d419ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:06.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.310+0000 7f75d9800640 1 -- 172.21.15.83:0/477772030 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75d419b2b0 con 0x7f75d4104320 2024-04-24T08:55:06.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.310+0000 7f75d9800640 1 --2- 172.21.15.83:0/477772030 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75d4104320 0x7f75d41a05f0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f75c8005940 tx=0x7f75c8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:06.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.311+0000 7f75d2000640 1 -- 172.21.15.83:0/477772030 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c803f030 con 0x7f75d4104320 2024-04-24T08:55:06.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.311+0000 7f75d2000640 1 -- 172.21.15.83:0/477772030 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f75c8003aa0 con 0x7f75d4104320 2024-04-24T08:55:06.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.311+0000 7f75dac00640 1 -- 172.21.15.83:0/477772030 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75d419b540 con 0x7f75d4104320 2024-04-24T08:55:06.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.311+0000 7f75d2000640 1 -- 172.21.15.83:0/477772030 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c8035070 con 0x7f75d4104320 2024-04-24T08:55:06.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.311+0000 7f75dac00640 1 -- 172.21.15.83:0/477772030 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75d41a7680 con 0x7f75d4104320 2024-04-24T08:55:06.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.314+0000 7f75def95740 1 Processor -- start 2024-04-24T08:55:06.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.314+0000 7f75d2000640 1 -- 172.21.15.83:0/477772030 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f75c8042030 con 0x7f75d4104320 2024-04-24T08:55:06.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.315+0000 7f75def95740 1 -- start start 2024-04-24T08:55:06.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.315+0000 7f75def95740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55940628ab40 0x55940628af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:06.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.315+0000 7f75def95740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55940628b460 0x5594062955f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:06.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.315+0000 7f75def95740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559406295b30 0x559406297f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:06.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.315+0000 7f75def95740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5594061f01d0 con 0x55940628ab40 2024-04-24T08:55:06.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.315+0000 7f75def95740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559406196450 con 0x55940628b460 2024-04-24T08:55:06.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.315+0000 7f75def95740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5594061ec190 con 0x559406295b30 2024-04-24T08:55:06.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.315+0000 7f75da200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55940628ab40 0x55940628af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:06.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.315+0000 7f75da200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55940628ab40 0x55940628af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34682/0 (socket says 172.21.15.83:34682) 2024-04-24T08:55:06.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.315+0000 7f75da200640 1 -- 172.21.15.83:0/899169753 learned_addr learned my addr 172.21.15.83:0/899169753 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:55:06.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.316+0000 7f75d9800640 1 --2- 172.21.15.83:0/899169753 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55940628b460 0x5594062955f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:06.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.316+0000 7f75d8e00640 1 --2- 172.21.15.83:0/899169753 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559406295b30 0x559406297f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:06.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.316+0000 7f75da200640 1 -- 172.21.15.83:0/899169753 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559406295b30 msgr2=0x559406297f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:06.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.316+0000 7f75da200640 1 --2- 172.21.15.83:0/899169753 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559406295b30 0x559406297f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:06.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.316+0000 7f75da200640 1 -- 172.21.15.83:0/899169753 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55940628b460 msgr2=0x5594062955f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:06.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.316+0000 7f75da200640 1 --2- 172.21.15.83:0/899169753 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55940628b460 0x5594062955f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:06.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.316+0000 7f75da200640 1 -- 172.21.15.83:0/899169753 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559406253310 con 0x55940628ab40 2024-04-24T08:55:06.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.316+0000 7f75d2000640 1 --2- 172.21.15.83:0/477772030 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f75bc0a3d20 0x7f75bc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:06.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.316+0000 7f75d2000640 1 -- 172.21.15.83:0/477772030 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f75c80f1210 con 0x7f75d4104320 2024-04-24T08:55:06.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.316+0000 7f75d9800640 1 --2- 172.21.15.83:0/477772030 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f75bc0a3d20 0x7f75bc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:06.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.316+0000 7f75da200640 1 --2- 172.21.15.83:0/899169753 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55940628ab40 0x55940628af20 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f75cc007dd0 tx=0x7f75cc00e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4724653db0550bd9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:06.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.317+0000 7f75c6a00640 1 -- 172.21.15.83:0/899169753 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75cc014070 con 0x55940628ab40 2024-04-24T08:55:06.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.317+0000 7f75c6a00640 1 -- 172.21.15.83:0/899169753 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f75cc004d20 con 0x55940628ab40 2024-04-24T08:55:06.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.317+0000 7f75c6a00640 1 -- 172.21.15.83:0/899169753 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75cc010790 con 0x55940628ab40 2024-04-24T08:55:06.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.318+0000 7f75def95740 1 -- 172.21.15.83:0/899169753 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55940628ab40 msgr2=0x55940628af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:06.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.318+0000 7f75d9800640 1 --2- 172.21.15.83:0/477772030 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f75bc0a3d20 0x7f75bc0a61e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f75c80ee640 tx=0x7f75c802a3f0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:06.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.318+0000 7f75def95740 1 --2- 172.21.15.83:0/899169753 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55940628ab40 0x55940628af20 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f75cc007dd0 tx=0x7f75cc00e720 comp rx=0 tx=0).stop 2024-04-24T08:55:06.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.318+0000 7f75def95740 1 -- 172.21.15.83:0/899169753 shutdown_connections 2024-04-24T08:55:06.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.318+0000 7f75def95740 1 --2- 172.21.15.83:0/899169753 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559406295b30 0x559406297f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:06.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.318+0000 7f75def95740 1 --2- 172.21.15.83:0/899169753 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55940628b460 0x5594062955f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:06.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.318+0000 7f75def95740 1 --2- 172.21.15.83:0/899169753 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55940628ab40 0x55940628af20 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:06.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.318+0000 7f75def95740 1 -- 172.21.15.83:0/899169753 >> 172.21.15.83:0/899169753 conn(0x559406286e60 msgr2=0x559406287240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:06.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.318+0000 7f75def95740 1 -- 172.21.15.83:0/899169753 shutdown_connections 2024-04-24T08:55:06.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.318+0000 7f75def95740 1 -- 172.21.15.83:0/899169753 wait complete. 2024-04-24T08:55:06.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.318+0000 7f75def95740 1 -- start start 2024-04-24T08:55:06.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.319+0000 7f75def95740 1 Processor -- start 2024-04-24T08:55:06.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.319+0000 7f75def95740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55940628ab40 0x5594062928f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:06.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.319+0000 7f75def95740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55940628b460 0x559406292e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:06.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.319+0000 7f75def95740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559406295b30 0x559406293370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:06.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.319+0000 7f75def95740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5594062989c0 con 0x559406295b30 2024-04-24T08:55:06.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.319+0000 7f75def95740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5594061a9810 con 0x55940628b460 2024-04-24T08:55:06.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.319+0000 7f75def95740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559406298b40 con 0x55940628ab40 2024-04-24T08:55:06.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.319+0000 7f75def95740 10 client.0 _ll_register_callbacks cb 0x559406177140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:55:06.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.319+0000 7f75da200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559406295b30 0x559406293370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:06.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.319+0000 7f75da200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559406295b30 0x559406293370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34690/0 (socket says 172.21.15.83:34690) 2024-04-24T08:55:06.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.319+0000 7f75da200640 1 -- 172.21.15.83:0/2888620930 learned_addr learned my addr 172.21.15.83:0/2888620930 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:55:06.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.320+0000 7f75da200640 1 --2- 172.21.15.83:0/2888620930 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55940628ab40 0x5594062928f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:06.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.320+0000 7f75d8e00640 1 --2- 172.21.15.83:0/2888620930 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55940628b460 0x559406292e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:06.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.320+0000 7f75da200640 1 -- 172.21.15.83:0/2888620930 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55940628ab40 msgr2=0x5594062928f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:06.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.320+0000 7f75da200640 1 --2- 172.21.15.83:0/2888620930 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55940628ab40 0x5594062928f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:06.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.320+0000 7f75da200640 1 -- 172.21.15.83:0/2888620930 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55940628b460 msgr2=0x559406292e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:06.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.320+0000 7f75da200640 1 --2- 172.21.15.83:0/2888620930 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55940628b460 0x559406292e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:06.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.320+0000 7f75da200640 1 -- 172.21.15.83:0/2888620930 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594061ba3f0 con 0x559406295b30 2024-04-24T08:55:06.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.320+0000 7f75da200640 1 --2- 172.21.15.83:0/2888620930 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559406295b30 0x559406293370 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f75cc004c30 tx=0x7f75cc010430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ff74f832b2f6415 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:06.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.320+0000 7f75bb400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:55:06.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.321+0000 7f75bb400640 1 -- 172.21.15.83:0/2888620930 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75cc014040 con 0x559406295b30 2024-04-24T08:55:06.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.321+0000 7f75bb400640 1 -- 172.21.15.83:0/2888620930 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f75cc003970 con 0x559406295b30 2024-04-24T08:55:06.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.321+0000 7f75def95740 1 -- 172.21.15.83:0/2888620930 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75cc004d20 con 0x559406295b30 2024-04-24T08:55:06.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.321+0000 7f75bb400640 1 -- 172.21.15.83:0/2888620930 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f75cc011e80 con 0x559406295b30 2024-04-24T08:55:06.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.321+0000 7f75def95740 10 client.16881 fetch_fsmap learned FSMap version 30 2024-04-24T08:55:06.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.321+0000 7f75def95740 1 -- 172.21.15.83:0/2888620930 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559405f023f0 con 0x559406295b30 2024-04-24T08:55:06.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.321+0000 7f75bb400640 1 -- 172.21.15.83:0/2888620930 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f75cc004d20 con 0x559406295b30 2024-04-24T08:55:06.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.322+0000 7f75bb400640 1 -- 172.21.15.83:0/2888620930 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7f75cc026040 con 0x559406295b30 2024-04-24T08:55:06.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.322+0000 7f75def95740 10 client.16881 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T08:55:06.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.322+0000 7f75def95740 10 client.16881 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-24T08:55:06.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.322+0000 7f75def95740 20 client.16881 populate_metadata read hostname 'smithi083' 2024-04-24T08:55:06.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.322+0000 7f75def95740 1 --2- 172.21.15.83:0/2888620930 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x5594062a5c60 0x5594062a8050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:06.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.322+0000 7f75def95740 4 client.16881 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:55:06.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.322+0000 7f75def95740 1 -- 172.21.15.83:0/2888620930 --> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5594061fdbf0 con 0x5594062a5c60 2024-04-24T08:55:06.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.323+0000 7f75d8e00640 1 --2- 172.21.15.83:0/2888620930 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x5594062a5c60 0x5594062a8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:06.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.323+0000 7f75d8e00640 1 --2- 172.21.15.83:0/2888620930 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x5594062a5c60 0x5594062a8050 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=cb531a064c5c895 server_cookie=ebe99c7cbcdf56 in_seq=0 out_seq=0 2024-04-24T08:55:06.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.323+0000 7f75bb400640 10 client.16881 ms_handle_connect on v2:172.21.15.189:6838/2548965336 2024-04-24T08:55:06.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.325+0000 7f75bb400640 1 -- 172.21.15.83:0/2888620930 <== mds.4 v2:172.21.15.189:6838/2548965336 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5594061fdbf0 con 0x5594062a5c60 2024-04-24T08:55:06.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.325+0000 7f75bb400640 10 client.16881 handle_command_reply: tid=0 2024-04-24T08:55:06.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.325+0000 7f75def95740 1 client.16881 shutdown 2024-04-24T08:55:06.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.326+0000 7f75def95740 1 -- 172.21.15.83:0/2888620930 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559406295b30 msgr2=0x559406293370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:06.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.326+0000 7f75def95740 1 --2- 172.21.15.83:0/2888620930 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559406295b30 0x559406293370 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f75cc004c30 tx=0x7f75cc010430 comp rx=0 tx=0).stop 2024-04-24T08:55:06.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.326+0000 7f75def95740 1 -- 172.21.15.83:0/2888620930 shutdown_connections 2024-04-24T08:55:06.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.326+0000 7f75def95740 1 --2- 172.21.15.83:0/2888620930 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x5594062a5c60 0x5594062a8050 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:06.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.326+0000 7f75def95740 1 --2- 172.21.15.83:0/2888620930 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559406295b30 0x559406293370 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:06.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.326+0000 7f75def95740 1 --2- 172.21.15.83:0/2888620930 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55940628b460 0x559406292e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:06.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.326+0000 7f75def95740 1 --2- 172.21.15.83:0/2888620930 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55940628ab40 0x5594062928f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:06.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.326+0000 7f75def95740 1 -- 172.21.15.83:0/2888620930 >> 172.21.15.83:0/2888620930 conn(0x559406286e60 msgr2=0x55940628cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:06.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.326+0000 7f75bb400640 0 client.16881 ms_handle_reset on v2:172.21.15.189:6838/2548965336 2024-04-24T08:55:06.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.326+0000 7f75def95740 1 -- 172.21.15.83:0/2888620930 shutdown_connections 2024-04-24T08:55:06.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.326+0000 7f75def95740 1 -- 172.21.15.83:0/2888620930 wait complete. 2024-04-24T08:55:06.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.326+0000 7f75def95740 20 client.16881 trim_cache size 0 max 16384 2024-04-24T08:55:06.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.345+0000 7f75def95740 1 Processor -- start 2024-04-24T08:55:06.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.345+0000 7f75def95740 1 -- start start 2024-04-24T08:55:06.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.346+0000 7f75def95740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55940628ab40 0x559406295130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:06.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.346+0000 7f75def95740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55940628b460 0x5594062a1880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:06.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.346+0000 7f75def95740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559406295b30 0x55940628ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:06.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.346+0000 7f75def95740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f75d41b1540 con 0x55940628ab40 2024-04-24T08:55:06.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.346+0000 7f75def95740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f75d41b3110 con 0x559406295b30 2024-04-24T08:55:06.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.346+0000 7f75def95740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f75d41b1c10 con 0x55940628b460 2024-04-24T08:55:06.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.346+0000 7f75da200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55940628ab40 0x559406295130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:06.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.346+0000 7f75da200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55940628ab40 0x559406295130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34700/0 (socket says 172.21.15.83:34700) 2024-04-24T08:55:06.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.346+0000 7f75da200640 1 -- 172.21.15.83:0/2766534756 learned_addr learned my addr 172.21.15.83:0/2766534756 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:55:06.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.347+0000 7f75d8e00640 1 --2- 172.21.15.83:0/2766534756 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55940628b460 0x5594062a1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:06.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.347+0000 7f75da200640 1 -- 172.21.15.83:0/2766534756 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55940628b460 msgr2=0x5594062a1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:06.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.347+0000 7f75da200640 1 --2- 172.21.15.83:0/2766534756 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55940628b460 0x5594062a1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:06.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.347+0000 7f75da200640 1 -- 172.21.15.83:0/2766534756 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559406295b30 msgr2=0x55940628ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:06.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.347+0000 7f75da200640 1 --2- 172.21.15.83:0/2766534756 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559406295b30 0x55940628ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:06.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.347+0000 7f75da200640 1 -- 172.21.15.83:0/2766534756 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559405fcc7d0 con 0x55940628ab40 2024-04-24T08:55:06.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.347+0000 7f75da200640 1 --2- 172.21.15.83:0/2766534756 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55940628ab40 0x559406295130 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f75cc00e760 tx=0x7f75cc0078e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=859b791f2dee348d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:06.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.347+0000 7f75c6000640 1 -- 172.21.15.83:0/2766534756 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75cc00ec20 con 0x55940628ab40 2024-04-24T08:55:06.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.347+0000 7f75c6000640 1 -- 172.21.15.83:0/2766534756 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f75cc003970 con 0x55940628ab40 2024-04-24T08:55:06.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.348+0000 7f75c6000640 1 -- 172.21.15.83:0/2766534756 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75cc010ba0 con 0x55940628ab40 2024-04-24T08:55:06.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.348+0000 7f75def95740 1 -- 172.21.15.83:0/2766534756 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55940628ab40 msgr2=0x559406295130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:06.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.348+0000 7f75def95740 1 --2- 172.21.15.83:0/2766534756 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55940628ab40 0x559406295130 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f75cc00e760 tx=0x7f75cc0078e0 comp rx=0 tx=0).stop 2024-04-24T08:55:06.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.348+0000 7f75def95740 1 -- 172.21.15.83:0/2766534756 shutdown_connections 2024-04-24T08:55:06.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.348+0000 7f75def95740 1 --2- 172.21.15.83:0/2766534756 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559406295b30 0x55940628ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:06.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.348+0000 7f75def95740 1 --2- 172.21.15.83:0/2766534756 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55940628b460 0x5594062a1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:06.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.348+0000 7f75def95740 1 --2- 172.21.15.83:0/2766534756 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55940628ab40 0x559406295130 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:06.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.348+0000 7f75def95740 1 -- 172.21.15.83:0/2766534756 >> 172.21.15.83:0/2766534756 conn(0x559406286e60 msgr2=0x55940628c380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:06.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.349+0000 7f75def95740 1 -- 172.21.15.83:0/2766534756 shutdown_connections 2024-04-24T08:55:06.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.349+0000 7f75def95740 1 -- 172.21.15.83:0/2766534756 wait complete. 2024-04-24T08:55:06.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.349+0000 7f75def95740 1 -- start start 2024-04-24T08:55:06.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.349+0000 7f75def95740 1 Processor -- start 2024-04-24T08:55:06.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.350+0000 7f75def95740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55940628ab40 0x559406295f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:06.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.350+0000 7f75def95740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55940628b460 0x559406296450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:06.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.350+0000 7f75def95740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559406295b30 0x559406296990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:06.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.350+0000 7f75def95740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f75d41b29b0 con 0x55940628b460 2024-04-24T08:55:06.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.350+0000 7f75def95740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f75d41b22e0 con 0x559406295b30 2024-04-24T08:55:06.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.350+0000 7f75def95740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f75d41af520 con 0x55940628ab40 2024-04-24T08:55:06.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.350+0000 7f75def95740 10 client.0 _ll_register_callbacks cb 0x7f75cc007c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:55:06.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.350+0000 7f75d8e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55940628b460 0x559406296450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:06.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.350+0000 7f75d8e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55940628b460 0x559406296450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34708/0 (socket says 172.21.15.83:34708) 2024-04-24T08:55:06.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.350+0000 7f75d8e00640 1 -- 172.21.15.83:0/2967608468 learned_addr learned my addr 172.21.15.83:0/2967608468 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:55:06.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.350+0000 7f75da200640 1 --2- 172.21.15.83:0/2967608468 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55940628ab40 0x559406295f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:06.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.350+0000 7f75d8e00640 1 -- 172.21.15.83:0/2967608468 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55940628ab40 msgr2=0x559406295f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:06.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.350+0000 7f75d8e00640 1 --2- 172.21.15.83:0/2967608468 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55940628ab40 0x559406295f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:06.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.350+0000 7f75da200640 1 --2- 172.21.15.83:0/2967608468 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559406295b30 0x559406296990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:06.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.350+0000 7f75d8e00640 1 -- 172.21.15.83:0/2967608468 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559406295b30 msgr2=0x559406296990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:06.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.351+0000 7f75d8e00640 1 --2- 172.21.15.83:0/2967608468 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559406295b30 0x559406296990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:06.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.351+0000 7f75d8e00640 1 -- 172.21.15.83:0/2967608468 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55940629ce10 con 0x55940628b460 2024-04-24T08:55:06.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.351+0000 7f75da200640 1 --2- 172.21.15.83:0/2967608468 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559406295b30 0x559406296990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:55:06.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.351+0000 7f75d8e00640 1 --2- 172.21.15.83:0/2967608468 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55940628b460 0x559406296450 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f75d419bf00 tx=0x7f75c001b600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26030d9e8cdaf75 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:06.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.351+0000 7f75bb400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:55:06.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.351+0000 7f75bb400640 1 -- 172.21.15.83:0/2967608468 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c001b940 con 0x55940628b460 2024-04-24T08:55:06.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.351+0000 7f75bb400640 1 -- 172.21.15.83:0/2967608468 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f75c001bdc0 con 0x55940628b460 2024-04-24T08:55:06.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.351+0000 7f75def95740 1 -- 172.21.15.83:0/2967608468 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75cc003970 con 0x55940628b460 2024-04-24T08:55:06.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.352+0000 7f75bb400640 1 -- 172.21.15.83:0/2967608468 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f75c002d050 con 0x55940628b460 2024-04-24T08:55:06.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.352+0000 7f75def95740 10 client.16887 fetch_fsmap learned FSMap version 30 2024-04-24T08:55:06.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.352+0000 7f75def95740 1 -- 172.21.15.83:0/2967608468 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55940629e130 con 0x55940628b460 2024-04-24T08:55:06.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.352+0000 7f75bb400640 1 -- 172.21.15.83:0/2967608468 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f75cc003970 con 0x55940628b460 2024-04-24T08:55:06.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.352+0000 7f75bb400640 1 -- 172.21.15.83:0/2967608468 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7f75c00249a0 con 0x55940628b460 2024-04-24T08:55:06.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.352+0000 7f75def95740 10 client.16887 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T08:55:06.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.352+0000 7f75def95740 10 client.16887 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-24T08:55:06.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.352+0000 7f75def95740 20 client.16887 populate_metadata read hostname 'smithi083' 2024-04-24T08:55:06.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.352+0000 7f75def95740 1 --2- 172.21.15.83:0/2967608468 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x559406314860 0x559406316c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:06.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.352+0000 7f75def95740 4 client.16887 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T08:55:06.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.352+0000 7f75def95740 1 -- 172.21.15.83:0/2967608468 --> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5594061f7a60 con 0x559406314860 2024-04-24T08:55:06.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.353+0000 7f75da200640 1 --2- 172.21.15.83:0/2967608468 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x559406314860 0x559406316c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:06.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.354+0000 7f75da200640 1 --2- 172.21.15.83:0/2967608468 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x559406314860 0x559406316c50 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=dd542a690c0bb284 server_cookie=e8dbc3ca44bab761 in_seq=0 out_seq=0 2024-04-24T08:55:06.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.354+0000 7f75bb400640 10 client.16887 ms_handle_connect on v2:172.21.15.189:6838/2548965336 2024-04-24T08:55:06.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.354+0000 7f75bb400640 1 -- 172.21.15.83:0/2967608468 <== mds.4 v2:172.21.15.189:6838/2548965336 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5594061f7a60 con 0x559406314860 2024-04-24T08:55:06.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.354+0000 7f75bb400640 10 client.16887 handle_command_reply: tid=0 2024-04-24T08:55:06.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.354+0000 7f75def95740 1 client.16887 shutdown 2024-04-24T08:55:06.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.354+0000 7f75def95740 1 -- 172.21.15.83:0/2967608468 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55940628b460 msgr2=0x559406296450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:06.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.354+0000 7f75def95740 1 --2- 172.21.15.83:0/2967608468 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55940628b460 0x559406296450 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f75d419bf00 tx=0x7f75c001b600 comp rx=0 tx=0).stop 2024-04-24T08:55:06.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.355+0000 7f75def95740 1 -- 172.21.15.83:0/2967608468 shutdown_connections 2024-04-24T08:55:06.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.355+0000 7f75def95740 1 --2- 172.21.15.83:0/2967608468 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x559406314860 0x559406316c50 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:06.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.355+0000 7f75def95740 1 --2- 172.21.15.83:0/2967608468 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559406295b30 0x559406296990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:06.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.355+0000 7f75def95740 1 --2- 172.21.15.83:0/2967608468 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55940628b460 0x559406296450 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:06.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.355+0000 7f75def95740 1 --2- 172.21.15.83:0/2967608468 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55940628ab40 0x559406295f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:06.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.355+0000 7f75def95740 1 -- 172.21.15.83:0/2967608468 >> 172.21.15.83:0/2967608468 conn(0x559406286e60 msgr2=0x559406285f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:06.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.355+0000 7f75bb400640 0 client.16887 ms_handle_reset on v2:172.21.15.189:6838/2548965336 2024-04-24T08:55:06.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.355+0000 7f75def95740 1 -- 172.21.15.83:0/2967608468 shutdown_connections 2024-04-24T08:55:06.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.355+0000 7f75def95740 1 -- 172.21.15.83:0/2967608468 wait complete. 2024-04-24T08:55:06.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.355+0000 7f75def95740 20 client.16887 trim_cache size 0 max 16384 2024-04-24T08:55:06.359 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T08:55:06.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.356+0000 7f75dac00640 1 -- 172.21.15.83:0/477772030 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f75bc0a3d20 msgr2=0x7f75bc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:06.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.356+0000 7f75dac00640 1 --2- 172.21.15.83:0/477772030 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f75bc0a3d20 0x7f75bc0a61e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f75c80ee640 tx=0x7f75c802a3f0 comp rx=0 tx=0).stop 2024-04-24T08:55:06.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.356+0000 7f75dac00640 1 -- 172.21.15.83:0/477772030 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75d4104320 msgr2=0x7f75d41a05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:06.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.356+0000 7f75dac00640 1 --2- 172.21.15.83:0/477772030 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75d4104320 0x7f75d41a05f0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f75c8005940 tx=0x7f75c8004270 comp rx=0 tx=0).stop 2024-04-24T08:55:06.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.356+0000 7f75dac00640 1 -- 172.21.15.83:0/477772030 shutdown_connections 2024-04-24T08:55:06.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.356+0000 7f75dac00640 1 --2- 172.21.15.83:0/477772030 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f75bc0a3d20 0x7f75bc0a61e0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:06.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.356+0000 7f75dac00640 1 --2- 172.21.15.83:0/477772030 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f75d419a6b0 0x7f75d419ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:06.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.356+0000 7f75dac00640 1 --2- 172.21.15.83:0/477772030 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f75d41093f0 0x7f75d41a0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:06.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.356+0000 7f75dac00640 1 --2- 172.21.15.83:0/477772030 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75d4104320 0x7f75d41a05f0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:06.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.356+0000 7f75dac00640 1 -- 172.21.15.83:0/477772030 >> 172.21.15.83:0/477772030 conn(0x7f75d40ffb40 msgr2=0x7f75d4100f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:06.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.356+0000 7f75dac00640 1 -- 172.21.15.83:0/477772030 shutdown_connections 2024-04-24T08:55:06.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.356+0000 7f75dac00640 1 -- 172.21.15.83:0/477772030 wait complete. 2024-04-24T08:55:06.374 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-24T08:55:06.565 INFO:tasks.workunit.client.0.smithi083.stdout: 1 12518 0.84 MB/sec warmup 100 sec latency 44.365 ms 2024-04-24T08:55:06.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.570+0000 7fe866400640 1 -- 172.21.15.83:0/354899685 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe86010d500 msgr2=0x7fe86010d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:06.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.570+0000 7fe866400640 1 --2- 172.21.15.83:0/354899685 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe86010d500 0x7fe86010d8e0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fe854009930 tx=0x7fe85402a420 comp rx=0 tx=0).stop 2024-04-24T08:55:06.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.570+0000 7fe866400640 1 -- 172.21.15.83:0/354899685 shutdown_connections 2024-04-24T08:55:06.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.570+0000 7fe866400640 1 --2- 172.21.15.83:0/354899685 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe860108f40 0x7fe8601153b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:06.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.570+0000 7fe866400640 1 --2- 172.21.15.83:0/354899685 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe860108580 0x7fe860108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:06.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.570+0000 7fe866400640 1 --2- 172.21.15.83:0/354899685 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe86010d500 0x7fe86010d8e0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:06.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.570+0000 7fe866400640 1 -- 172.21.15.83:0/354899685 >> 172.21.15.83:0/354899685 conn(0x7fe860077e20 msgr2=0x7fe860106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:06.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.571+0000 7fe866400640 1 -- 172.21.15.83:0/354899685 shutdown_connections 2024-04-24T08:55:06.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.571+0000 7fe866400640 1 -- 172.21.15.83:0/354899685 wait complete. 2024-04-24T08:55:06.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.572+0000 7fe866400640 1 Processor -- start 2024-04-24T08:55:06.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.572+0000 7fe866400640 1 -- start start 2024-04-24T08:55:06.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.572+0000 7fe866400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe860108580 0x7fe8601a4950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:06.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.572+0000 7fe866400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe860108f40 0x7fe8601a4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:06.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.572+0000 7fe866400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe86010d500 0x7fe86019eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:06.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.572+0000 7fe866400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe860118830 con 0x7fe860108580 2024-04-24T08:55:06.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.572+0000 7fe866400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe8601186b0 con 0x7fe86010d500 2024-04-24T08:55:06.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.572+0000 7fe866400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe8601189b0 con 0x7fe860108f40 2024-04-24T08:55:06.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.572+0000 7fe865000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe860108580 0x7fe8601a4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:06.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.573+0000 7fe865000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe860108580 0x7fe8601a4950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34732/0 (socket says 172.21.15.83:34732) 2024-04-24T08:55:06.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.573+0000 7fe865000640 1 -- 172.21.15.83:0/18192662 learned_addr learned my addr 172.21.15.83:0/18192662 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:55:06.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.573+0000 7fe85fe00640 1 --2- 172.21.15.83:0/18192662 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe860108f40 0x7fe8601a4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:06.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.573+0000 7fe865a00640 1 --2- 172.21.15.83:0/18192662 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe86010d500 0x7fe86019eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:06.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.573+0000 7fe865000640 1 -- 172.21.15.83:0/18192662 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe860108f40 msgr2=0x7fe8601a4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:06.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.573+0000 7fe865000640 1 --2- 172.21.15.83:0/18192662 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe860108f40 0x7fe8601a4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:06.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.573+0000 7fe865000640 1 -- 172.21.15.83:0/18192662 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe86010d500 msgr2=0x7fe86019eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:06.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.573+0000 7fe865000640 1 --2- 172.21.15.83:0/18192662 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe86010d500 0x7fe86019eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:06.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.573+0000 7fe865000640 1 -- 172.21.15.83:0/18192662 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe86019f2d0 con 0x7fe860108580 2024-04-24T08:55:06.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.573+0000 7fe865a00640 1 --2- 172.21.15.83:0/18192662 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe86010d500 0x7fe86019eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:55:06.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.573+0000 7fe865000640 1 --2- 172.21.15.83:0/18192662 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe860108580 0x7fe8601a4950 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fe854031440 tx=0x7fe854003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:06.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.574+0000 7fe85d600640 1 -- 172.21.15.83:0/18192662 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe85403f070 con 0x7fe860108580 2024-04-24T08:55:06.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.574+0000 7fe85d600640 1 -- 172.21.15.83:0/18192662 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe854031d60 con 0x7fe860108580 2024-04-24T08:55:06.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.574+0000 7fe866400640 1 -- 172.21.15.83:0/18192662 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe86019f560 con 0x7fe860108580 2024-04-24T08:55:06.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.574+0000 7fe85d600640 1 -- 172.21.15.83:0/18192662 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe854035030 con 0x7fe860108580 2024-04-24T08:55:06.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.574+0000 7fe866400640 1 -- 172.21.15.83:0/18192662 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8601ab810 con 0x7fe860108580 2024-04-24T08:55:06.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.577+0000 7fe86a85c740 1 Processor -- start 2024-04-24T08:55:06.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.578+0000 7fe86a85c740 1 -- start start 2024-04-24T08:55:06.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.578+0000 7fe86a85c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f6d2fc3b40 0x55f6d2fc3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:06.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.578+0000 7fe85d600640 1 -- 172.21.15.83:0/18192662 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe854042070 con 0x7fe860108580 2024-04-24T08:55:06.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.578+0000 7fe86a85c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6d2fc4460 0x55f6d2fce5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:06.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.578+0000 7fe86a85c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6d2fceb30 0x55f6d2fd0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:06.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.578+0000 7fe86a85c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55f6d2f291d0 con 0x55f6d2fc4460 2024-04-24T08:55:06.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.578+0000 7fe86a85c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f6d2ecf450 con 0x55f6d2fceb30 2024-04-24T08:55:06.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.578+0000 7fe86a85c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55f6d2f25190 con 0x55f6d2fc3b40 2024-04-24T08:55:06.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.579+0000 7fe865000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6d2fc4460 0x55f6d2fce5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:06.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.579+0000 7fe865000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6d2fc4460 0x55f6d2fce5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34746/0 (socket says 172.21.15.83:34746) 2024-04-24T08:55:06.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.579+0000 7fe865000640 1 -- 172.21.15.83:0/2383764111 learned_addr learned my addr 172.21.15.83:0/2383764111 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:55:06.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.579+0000 7fe865a00640 1 --2- 172.21.15.83:0/2383764111 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f6d2fc3b40 0x55f6d2fc3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:06.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.579+0000 7fe85fe00640 1 --2- 172.21.15.83:0/2383764111 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6d2fceb30 0x55f6d2fd0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:06.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.579+0000 7fe865000640 1 -- 172.21.15.83:0/2383764111 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f6d2fc3b40 msgr2=0x55f6d2fc3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:06.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.579+0000 7fe865000640 1 --2- 172.21.15.83:0/2383764111 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f6d2fc3b40 0x55f6d2fc3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:06.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.579+0000 7fe865000640 1 -- 172.21.15.83:0/2383764111 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6d2fceb30 msgr2=0x55f6d2fd0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:06.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.579+0000 7fe865000640 1 --2- 172.21.15.83:0/2383764111 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6d2fceb30 0x55f6d2fd0f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:06.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.579+0000 7fe865000640 1 -- 172.21.15.83:0/2383764111 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6d2f8c310 con 0x55f6d2fc4460 2024-04-24T08:55:06.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.580+0000 7fe865000640 1 --2- 172.21.15.83:0/2383764111 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6d2fc4460 0x55f6d2fce5f0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fe8540ee740 tx=0x7fe8540f76c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8c38d3d19d0690e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:06.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.580+0000 7fe84e000640 1 -- 172.21.15.83:0/2383764111 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8540f1cb0 con 0x55f6d2fc4460 2024-04-24T08:55:06.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.580+0000 7fe84e000640 1 -- 172.21.15.83:0/2383764111 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe8540fe030 con 0x55f6d2fc4460 2024-04-24T08:55:06.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.580+0000 7fe84e000640 1 -- 172.21.15.83:0/2383764111 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe854103960 con 0x55f6d2fc4460 2024-04-24T08:55:06.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.580+0000 7fe85d600640 1 --2- 172.21.15.83:0/18192662 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe8480a3da0 0x7fe8480a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:06.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.580+0000 7fe85d600640 1 -- 172.21.15.83:0/18192662 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7fe8540f14b0 con 0x7fe860108580 2024-04-24T08:55:06.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.580+0000 7fe865a00640 1 --2- 172.21.15.83:0/18192662 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe8480a3da0 0x7fe8480a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:06.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.581+0000 7fe86a85c740 1 -- 172.21.15.83:0/2383764111 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6d2fc4460 msgr2=0x55f6d2fce5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:06.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.581+0000 7fe86a85c740 1 --2- 172.21.15.83:0/2383764111 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6d2fc4460 0x55f6d2fce5f0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fe8540ee740 tx=0x7fe8540f76c0 comp rx=0 tx=0).stop 2024-04-24T08:55:06.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.581+0000 7fe86a85c740 1 -- 172.21.15.83:0/2383764111 shutdown_connections 2024-04-24T08:55:06.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.581+0000 7fe86a85c740 1 --2- 172.21.15.83:0/2383764111 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6d2fceb30 0x55f6d2fd0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:06.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.581+0000 7fe86a85c740 1 --2- 172.21.15.83:0/2383764111 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6d2fc4460 0x55f6d2fce5f0 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:06.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.581+0000 7fe86a85c740 1 --2- 172.21.15.83:0/2383764111 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f6d2fc3b40 0x55f6d2fc3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:06.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.581+0000 7fe86a85c740 1 -- 172.21.15.83:0/2383764111 >> 172.21.15.83:0/2383764111 conn(0x55f6d2fbfe60 msgr2=0x55f6d2fc0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:06.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.581+0000 7fe865a00640 1 --2- 172.21.15.83:0/18192662 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe8480a3da0 0x7fe8480a6260 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe858005fd0 tx=0x7fe858008620 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:06.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.581+0000 7fe86a85c740 1 -- 172.21.15.83:0/2383764111 shutdown_connections 2024-04-24T08:55:06.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.581+0000 7fe86a85c740 1 -- 172.21.15.83:0/2383764111 wait complete. 2024-04-24T08:55:06.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.582+0000 7fe86a85c740 1 -- start start 2024-04-24T08:55:06.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.582+0000 7fe86a85c740 1 Processor -- start 2024-04-24T08:55:06.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.582+0000 7fe86a85c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f6d2fc3b40 0x55f6d2fce4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:06.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.582+0000 7fe86a85c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6d2fc4460 0x55f6d2fcb7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:06.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.582+0000 7fe86a85c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6d2fceb30 0x55f6d2fcbd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:06.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.582+0000 7fe86a85c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55f6d2fd16e0 con 0x55f6d2fc4460 2024-04-24T08:55:06.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.582+0000 7fe86a85c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f6d2ee2810 con 0x55f6d2fceb30 2024-04-24T08:55:06.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.582+0000 7fe86a85c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55f6d2fd1860 con 0x55f6d2fc3b40 2024-04-24T08:55:06.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.583+0000 7fe86a85c740 10 client.0 _ll_register_callbacks cb 0x55f6d2eb0140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:55:06.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.583+0000 7fe85fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6d2fc4460 0x55f6d2fcb7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:06.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.583+0000 7fe85fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6d2fc4460 0x55f6d2fcb7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34762/0 (socket says 172.21.15.83:34762) 2024-04-24T08:55:06.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.583+0000 7fe85fe00640 1 -- 172.21.15.83:0/3329968557 learned_addr learned my addr 172.21.15.83:0/3329968557 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:55:06.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.583+0000 7fe865000640 1 --2- 172.21.15.83:0/3329968557 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f6d2fc3b40 0x55f6d2fce4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:06.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.583+0000 7fe865a00640 1 --2- 172.21.15.83:0/3329968557 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6d2fceb30 0x55f6d2fcbd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:06.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.583+0000 7fe85fe00640 1 -- 172.21.15.83:0/3329968557 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f6d2fc3b40 msgr2=0x55f6d2fce4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:06.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.583+0000 7fe85fe00640 1 --2- 172.21.15.83:0/3329968557 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f6d2fc3b40 0x55f6d2fce4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:06.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.583+0000 7fe85fe00640 1 -- 172.21.15.83:0/3329968557 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6d2fceb30 msgr2=0x55f6d2fcbd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:06.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.583+0000 7fe85fe00640 1 --2- 172.21.15.83:0/3329968557 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6d2fceb30 0x55f6d2fcbd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:06.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.583+0000 7fe85fe00640 1 -- 172.21.15.83:0/3329968557 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6d2ef33f0 con 0x55f6d2fc4460 2024-04-24T08:55:06.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.583+0000 7fe865000640 1 --2- 172.21.15.83:0/3329968557 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f6d2fc3b40 0x55f6d2fce4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:55:06.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.583+0000 7fe865a00640 1 --2- 172.21.15.83:0/3329968557 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6d2fceb30 0x55f6d2fcbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:55:06.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.584+0000 7fe85fe00640 1 --2- 172.21.15.83:0/3329968557 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6d2fc4460 0x55f6d2fcb7b0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fe85000bf60 tx=0x7fe85000bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0e91fabae9b0f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:06.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.584+0000 7fe842a00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:55:06.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.584+0000 7fe842a00640 1 -- 172.21.15.83:0/3329968557 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe850018020 con 0x55f6d2fc4460 2024-04-24T08:55:06.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.584+0000 7fe842a00640 1 -- 172.21.15.83:0/3329968557 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7fe850014660 con 0x55f6d2fc4460 2024-04-24T08:55:06.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.584+0000 7fe86a85c740 1 -- 172.21.15.83:0/3329968557 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8540fe030 con 0x55f6d2fc4460 2024-04-24T08:55:06.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.584+0000 7fe842a00640 1 -- 172.21.15.83:0/3329968557 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe850023050 con 0x55f6d2fc4460 2024-04-24T08:55:06.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.584+0000 7fe86a85c740 10 client.16899 fetch_fsmap learned FSMap version 30 2024-04-24T08:55:06.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.584+0000 7fe86a85c740 1 -- 172.21.15.83:0/3329968557 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f6d2c3b3f0 con 0x55f6d2fc4460 2024-04-24T08:55:06.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.585+0000 7fe842a00640 1 -- 172.21.15.83:0/3329968557 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe8540fe030 con 0x55f6d2fc4460 2024-04-24T08:55:06.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.585+0000 7fe842a00640 1 -- 172.21.15.83:0/3329968557 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7fe850013aa0 con 0x55f6d2fc4460 2024-04-24T08:55:06.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.585+0000 7fe86a85c740 10 client.16899 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T08:55:06.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.585+0000 7fe86a85c740 10 client.16899 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-24T08:55:06.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.585+0000 7fe86a85c740 20 client.16899 populate_metadata read hostname 'smithi083' 2024-04-24T08:55:06.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.585+0000 7fe86a85c740 1 --2- 172.21.15.83:0/3329968557 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55f6d2fdeac0 0x55f6d2fe0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:06.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.585+0000 7fe86a85c740 4 client.16899 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:55:06.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.585+0000 7fe86a85c740 1 -- 172.21.15.83:0/3329968557 --> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f6d2f36bf0 con 0x55f6d2fdeac0 2024-04-24T08:55:06.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.586+0000 7fe865000640 1 --2- 172.21.15.83:0/3329968557 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55f6d2fdeac0 0x55f6d2fe0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:06.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.587+0000 7fe865000640 1 --2- 172.21.15.83:0/3329968557 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55f6d2fdeac0 0x55f6d2fe0ef0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a28bd58e8ab65a9d server_cookie=9fa1c2966b57b24a in_seq=0 out_seq=0 2024-04-24T08:55:06.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.587+0000 7fe842a00640 10 client.16899 ms_handle_connect on v2:172.21.15.138:6832/1157967658 2024-04-24T08:55:06.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.589+0000 7fe842a00640 1 -- 172.21.15.83:0/3329968557 <== mds.3 v2:172.21.15.138:6832/1157967658 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f6d2f36bf0 con 0x55f6d2fdeac0 2024-04-24T08:55:06.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.589+0000 7fe842a00640 10 client.16899 handle_command_reply: tid=0 2024-04-24T08:55:06.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.589+0000 7fe86a85c740 1 client.16899 shutdown 2024-04-24T08:55:06.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.589+0000 7fe86a85c740 1 -- 172.21.15.83:0/3329968557 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6d2fc4460 msgr2=0x55f6d2fcb7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:06.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.589+0000 7fe86a85c740 1 --2- 172.21.15.83:0/3329968557 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6d2fc4460 0x55f6d2fcb7b0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fe85000bf60 tx=0x7fe85000bf90 comp rx=0 tx=0).stop 2024-04-24T08:55:06.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.589+0000 7fe86a85c740 1 -- 172.21.15.83:0/3329968557 shutdown_connections 2024-04-24T08:55:06.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.589+0000 7fe86a85c740 1 --2- 172.21.15.83:0/3329968557 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6d2fceb30 0x55f6d2fcbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:06.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.589+0000 7fe86a85c740 1 --2- 172.21.15.83:0/3329968557 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6d2fc4460 0x55f6d2fcb7b0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:06.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.589+0000 7fe86a85c740 1 --2- 172.21.15.83:0/3329968557 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55f6d2fdeac0 0x55f6d2fe0ef0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:06.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.589+0000 7fe86a85c740 1 --2- 172.21.15.83:0/3329968557 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f6d2fc3b40 0x55f6d2fce4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:06.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.589+0000 7fe86a85c740 1 -- 172.21.15.83:0/3329968557 >> 172.21.15.83:0/3329968557 conn(0x55f6d2fbfe60 msgr2=0x55f6d2fc5b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:06.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.589+0000 7fe842a00640 0 client.16899 ms_handle_reset on v2:172.21.15.138:6832/1157967658 2024-04-24T08:55:06.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.589+0000 7fe86a85c740 1 -- 172.21.15.83:0/3329968557 shutdown_connections 2024-04-24T08:55:06.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.590+0000 7fe86a85c740 1 -- 172.21.15.83:0/3329968557 wait complete. 2024-04-24T08:55:06.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.590+0000 7fe86a85c740 20 client.16899 trim_cache size 0 max 16384 2024-04-24T08:55:06.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.604+0000 7fe86a85c740 1 Processor -- start 2024-04-24T08:55:06.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.604+0000 7fe86a85c740 1 -- start start 2024-04-24T08:55:06.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.605+0000 7fe86a85c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f6d2fc3b40 0x55f6d2fda600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:06.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.605+0000 7fe86a85c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6d2fc4460 0x55f6d2fcdff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:06.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.605+0000 7fe86a85c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6d2fceb30 0x55f6d2fe6800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:06.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.605+0000 7fe86a85c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe8601b5960 con 0x55f6d2fc4460 2024-04-24T08:55:06.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.605+0000 7fe86a85c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe8601b7530 con 0x55f6d2fceb30 2024-04-24T08:55:06.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.605+0000 7fe86a85c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe8601b6030 con 0x55f6d2fc3b40 2024-04-24T08:55:06.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.605+0000 7fe85fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6d2fc4460 0x55f6d2fcdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:06.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.605+0000 7fe85fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6d2fc4460 0x55f6d2fcdff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34774/0 (socket says 172.21.15.83:34774) 2024-04-24T08:55:06.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.605+0000 7fe85fe00640 1 -- 172.21.15.83:0/2313344633 learned_addr learned my addr 172.21.15.83:0/2313344633 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:55:06.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.605+0000 7fe865000640 1 --2- 172.21.15.83:0/2313344633 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f6d2fc3b40 0x55f6d2fda600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:06.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.605+0000 7fe865a00640 1 --2- 172.21.15.83:0/2313344633 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6d2fceb30 0x55f6d2fe6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:06.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.606+0000 7fe85fe00640 1 -- 172.21.15.83:0/2313344633 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f6d2fc3b40 msgr2=0x55f6d2fda600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:06.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.606+0000 7fe85fe00640 1 --2- 172.21.15.83:0/2313344633 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f6d2fc3b40 0x55f6d2fda600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:06.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.606+0000 7fe85fe00640 1 -- 172.21.15.83:0/2313344633 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6d2fceb30 msgr2=0x55f6d2fe6800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:06.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.606+0000 7fe85fe00640 1 --2- 172.21.15.83:0/2313344633 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6d2fceb30 0x55f6d2fe6800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:06.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.606+0000 7fe85fe00640 1 -- 172.21.15.83:0/2313344633 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6d2d057d0 con 0x55f6d2fc4460 2024-04-24T08:55:06.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.606+0000 7fe865a00640 1 --2- 172.21.15.83:0/2313344633 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6d2fceb30 0x55f6d2fe6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:55:06.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.606+0000 7fe85fe00640 1 --2- 172.21.15.83:0/2313344633 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6d2fc4460 0x55f6d2fcdff0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fe850006aa0 tx=0x7fe85000fc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe6facff31d2977b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:06.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.606+0000 7fe84d600640 1 -- 172.21.15.83:0/2313344633 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe85002b020 con 0x55f6d2fc4460 2024-04-24T08:55:06.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.606+0000 7fe84d600640 1 -- 172.21.15.83:0/2313344633 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe85001cdc0 con 0x55f6d2fc4460 2024-04-24T08:55:06.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.606+0000 7fe84d600640 1 -- 172.21.15.83:0/2313344633 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe850018040 con 0x55f6d2fc4460 2024-04-24T08:55:06.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.607+0000 7fe86a85c740 1 -- 172.21.15.83:0/2313344633 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6d2fc4460 msgr2=0x55f6d2fcdff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:06.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.607+0000 7fe86a85c740 1 --2- 172.21.15.83:0/2313344633 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6d2fc4460 0x55f6d2fcdff0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fe850006aa0 tx=0x7fe85000fc30 comp rx=0 tx=0).stop 2024-04-24T08:55:06.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.607+0000 7fe86a85c740 1 -- 172.21.15.83:0/2313344633 shutdown_connections 2024-04-24T08:55:06.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.607+0000 7fe86a85c740 1 --2- 172.21.15.83:0/2313344633 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6d2fceb30 0x55f6d2fe6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:06.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.607+0000 7fe86a85c740 1 --2- 172.21.15.83:0/2313344633 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6d2fc4460 0x55f6d2fcdff0 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:06.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.607+0000 7fe86a85c740 1 --2- 172.21.15.83:0/2313344633 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f6d2fc3b40 0x55f6d2fda600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:06.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.607+0000 7fe86a85c740 1 -- 172.21.15.83:0/2313344633 >> 172.21.15.83:0/2313344633 conn(0x55f6d2fbfe60 msgr2=0x55f6d2fc0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:06.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.607+0000 7fe86a85c740 1 -- 172.21.15.83:0/2313344633 shutdown_connections 2024-04-24T08:55:06.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.607+0000 7fe86a85c740 1 -- 172.21.15.83:0/2313344633 wait complete. 2024-04-24T08:55:06.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.608+0000 7fe86a85c740 1 -- start start 2024-04-24T08:55:06.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.608+0000 7fe86a85c740 1 Processor -- start 2024-04-24T08:55:06.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.608+0000 7fe86a85c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6d2fc3b40 0x55f6d2fcf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:06.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.608+0000 7fe86a85c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6d2fc4460 0x55f6d2fcfa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:06.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.608+0000 7fe86a85c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f6d2fceb30 0x55f6d2fd5b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:06.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.608+0000 7fe86a85c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe8601b6dd0 con 0x55f6d2fc4460 2024-04-24T08:55:06.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.608+0000 7fe86a85c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe8601b6700 con 0x55f6d2fc3b40 2024-04-24T08:55:06.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.608+0000 7fe86a85c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe8601b3940 con 0x55f6d2fceb30 2024-04-24T08:55:06.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.608+0000 7fe86a85c740 10 client.0 _ll_register_callbacks cb 0x7fe85000b280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:55:06.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.609+0000 7fe85fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6d2fc4460 0x55f6d2fcfa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:06.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.609+0000 7fe85fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6d2fc4460 0x55f6d2fcfa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34778/0 (socket says 172.21.15.83:34778) 2024-04-24T08:55:06.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.609+0000 7fe85fe00640 1 -- 172.21.15.83:0/3123423384 learned_addr learned my addr 172.21.15.83:0/3123423384 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:55:06.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.609+0000 7fe865000640 1 --2- 172.21.15.83:0/3123423384 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6d2fc3b40 0x55f6d2fcf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:06.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.609+0000 7fe865a00640 1 --2- 172.21.15.83:0/3123423384 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f6d2fceb30 0x55f6d2fd5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:06.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.609+0000 7fe85fe00640 1 -- 172.21.15.83:0/3123423384 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f6d2fceb30 msgr2=0x55f6d2fd5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:06.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.609+0000 7fe85fe00640 1 --2- 172.21.15.83:0/3123423384 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f6d2fceb30 0x55f6d2fd5b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:06.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.609+0000 7fe85fe00640 1 -- 172.21.15.83:0/3123423384 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6d2fc3b40 msgr2=0x55f6d2fcf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:06.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.609+0000 7fe85fe00640 1 --2- 172.21.15.83:0/3123423384 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6d2fc3b40 0x55f6d2fcf510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:06.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.609+0000 7fe85fe00640 1 -- 172.21.15.83:0/3123423384 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6d2fd6050 con 0x55f6d2fc4460 2024-04-24T08:55:06.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.609+0000 7fe865000640 1 --2- 172.21.15.83:0/3123423384 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6d2fc3b40 0x55f6d2fcf510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:55:06.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.609+0000 7fe865a00640 1 --2- 172.21.15.83:0/3123423384 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f6d2fceb30 0x55f6d2fd5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:55:06.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.610+0000 7fe85fe00640 1 --2- 172.21.15.83:0/3123423384 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6d2fc4460 0x55f6d2fcfa50 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fe850004700 tx=0x7fe850013660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32697be00ce225c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:06.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.610+0000 7fe842a00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:55:06.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.610+0000 7fe842a00640 1 -- 172.21.15.83:0/3123423384 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe85002b070 con 0x55f6d2fc4460 2024-04-24T08:55:06.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.610+0000 7fe86a85c740 1 -- 172.21.15.83:0/3123423384 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe85001cdc0 con 0x55f6d2fc4460 2024-04-24T08:55:06.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.610+0000 7fe842a00640 1 -- 172.21.15.83:0/3123423384 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7fe85001c470 con 0x55f6d2fc4460 2024-04-24T08:55:06.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.610+0000 7fe842a00640 1 -- 172.21.15.83:0/3123423384 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe8500059c0 con 0x55f6d2fc4460 2024-04-24T08:55:06.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.610+0000 7fe86a85c740 10 client.16905 fetch_fsmap learned FSMap version 30 2024-04-24T08:55:06.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.610+0000 7fe86a85c740 1 -- 172.21.15.83:0/3123423384 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f6d2fd71f0 con 0x55f6d2fc4460 2024-04-24T08:55:06.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.610+0000 7fe842a00640 1 -- 172.21.15.83:0/3123423384 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe85001cdc0 con 0x55f6d2fc4460 2024-04-24T08:55:06.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.611+0000 7fe842a00640 1 -- 172.21.15.83:0/3123423384 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7fe850006040 con 0x55f6d2fc4460 2024-04-24T08:55:06.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.611+0000 7fe86a85c740 10 client.16905 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T08:55:06.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.611+0000 7fe86a85c740 10 client.16905 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-24T08:55:06.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.611+0000 7fe86a85c740 20 client.16905 populate_metadata read hostname 'smithi083' 2024-04-24T08:55:06.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.611+0000 7fe86a85c740 1 --2- 172.21.15.83:0/3123423384 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55f6d3045400 0x55f6d3047820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:06.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.611+0000 7fe86a85c740 4 client.16905 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T08:55:06.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.611+0000 7fe86a85c740 1 -- 172.21.15.83:0/3123423384 --> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f6d2f30a60 con 0x55f6d3045400 2024-04-24T08:55:06.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.611+0000 7fe865000640 1 --2- 172.21.15.83:0/3123423384 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55f6d3045400 0x55f6d3047820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:06.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.613+0000 7fe842a00640 10 client.16905 ms_handle_connect on v2:172.21.15.138:6832/1157967658 2024-04-24T08:55:06.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.613+0000 7fe865000640 1 --2- 172.21.15.83:0/3123423384 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55f6d3045400 0x55f6d3047820 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=96490fcdd437613f server_cookie=5391cf4fc53d3972 in_seq=0 out_seq=0 2024-04-24T08:55:06.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.613+0000 7fe842a00640 1 -- 172.21.15.83:0/3123423384 <== mds.3 v2:172.21.15.138:6832/1157967658 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f6d2f30a60 con 0x55f6d3045400 2024-04-24T08:55:06.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.613+0000 7fe842a00640 10 client.16905 handle_command_reply: tid=0 2024-04-24T08:55:06.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.613+0000 7fe86a85c740 1 client.16905 shutdown 2024-04-24T08:55:06.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.613+0000 7fe86a85c740 1 -- 172.21.15.83:0/3123423384 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6d2fc4460 msgr2=0x55f6d2fcfa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:06.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.613+0000 7fe86a85c740 1 --2- 172.21.15.83:0/3123423384 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6d2fc4460 0x55f6d2fcfa50 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fe850004700 tx=0x7fe850013660 comp rx=0 tx=0).stop 2024-04-24T08:55:06.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.614+0000 7fe86a85c740 1 -- 172.21.15.83:0/3123423384 shutdown_connections 2024-04-24T08:55:06.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.614+0000 7fe86a85c740 1 --2- 172.21.15.83:0/3123423384 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55f6d3045400 0x55f6d3047820 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:06.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.614+0000 7fe86a85c740 1 --2- 172.21.15.83:0/3123423384 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f6d2fceb30 0x55f6d2fd5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:06.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.614+0000 7fe86a85c740 1 --2- 172.21.15.83:0/3123423384 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6d2fc4460 0x55f6d2fcfa50 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:06.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.614+0000 7fe86a85c740 1 --2- 172.21.15.83:0/3123423384 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6d2fc3b40 0x55f6d2fcf510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:06.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.614+0000 7fe842a00640 0 client.16905 ms_handle_reset on v2:172.21.15.138:6832/1157967658 2024-04-24T08:55:06.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.614+0000 7fe86a85c740 1 -- 172.21.15.83:0/3123423384 >> 172.21.15.83:0/3123423384 conn(0x55f6d2fbfe60 msgr2=0x55f6d2fbe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:06.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.614+0000 7fe86a85c740 1 -- 172.21.15.83:0/3123423384 shutdown_connections 2024-04-24T08:55:06.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.614+0000 7fe86a85c740 1 -- 172.21.15.83:0/3123423384 wait complete. 2024-04-24T08:55:06.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.614+0000 7fe86a85c740 20 client.16905 trim_cache size 0 max 16384 2024-04-24T08:55:06.622 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T08:55:06.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.615+0000 7fe866400640 1 -- 172.21.15.83:0/18192662 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe8480a3da0 msgr2=0x7fe8480a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:06.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.615+0000 7fe866400640 1 --2- 172.21.15.83:0/18192662 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe8480a3da0 0x7fe8480a6260 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe858005fd0 tx=0x7fe858008620 comp rx=0 tx=0).stop 2024-04-24T08:55:06.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.615+0000 7fe866400640 1 -- 172.21.15.83:0/18192662 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe860108580 msgr2=0x7fe8601a4950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:06.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.615+0000 7fe866400640 1 --2- 172.21.15.83:0/18192662 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe860108580 0x7fe8601a4950 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fe854031440 tx=0x7fe854003d50 comp rx=0 tx=0).stop 2024-04-24T08:55:06.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.615+0000 7fe866400640 1 -- 172.21.15.83:0/18192662 shutdown_connections 2024-04-24T08:55:06.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.615+0000 7fe866400640 1 --2- 172.21.15.83:0/18192662 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe8480a3da0 0x7fe8480a6260 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:06.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.615+0000 7fe866400640 1 --2- 172.21.15.83:0/18192662 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe86010d500 0x7fe86019eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:06.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.615+0000 7fe866400640 1 --2- 172.21.15.83:0/18192662 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe860108f40 0x7fe8601a4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:06.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.615+0000 7fe866400640 1 --2- 172.21.15.83:0/18192662 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe860108580 0x7fe8601a4950 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:06.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.615+0000 7fe866400640 1 -- 172.21.15.83:0/18192662 >> 172.21.15.83:0/18192662 conn(0x7fe860077e20 msgr2=0x7fe8600783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:06.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.615+0000 7fe866400640 1 -- 172.21.15.83:0/18192662 shutdown_connections 2024-04-24T08:55:06.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.615+0000 7fe866400640 1 -- 172.21.15.83:0/18192662 wait complete. 2024-04-24T08:55:06.635 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T08:55:06.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.847+0000 7f678c200640 1 -- 172.21.15.83:0/306344653 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6784101af0 msgr2=0x7f6784105b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:06.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.847+0000 7f678c200640 1 --2- 172.21.15.83:0/306344653 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6784101af0 0x7f6784105b30 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f6770009900 tx=0x7f677002a3f0 comp rx=0 tx=0).stop 2024-04-24T08:55:06.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.847+0000 7f678c200640 1 -- 172.21.15.83:0/306344653 shutdown_connections 2024-04-24T08:55:06.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.847+0000 7f678c200640 1 --2- 172.21.15.83:0/306344653 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6784106210 0x7f678410dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:06.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.847+0000 7f678c200640 1 --2- 172.21.15.83:0/306344653 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6784101af0 0x7f6784105b30 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:06.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.847+0000 7f678c200640 1 --2- 172.21.15.83:0/306344653 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6784101140 0x7f6784101520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:06.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.847+0000 7f678c200640 1 -- 172.21.15.83:0/306344653 >> 172.21.15.83:0/306344653 conn(0x7f6784078000 msgr2=0x7f6784078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:06.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.848+0000 7f678c200640 1 -- 172.21.15.83:0/306344653 shutdown_connections 2024-04-24T08:55:06.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.848+0000 7f678c200640 1 -- 172.21.15.83:0/306344653 wait complete. 2024-04-24T08:55:06.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.849+0000 7f678c200640 1 Processor -- start 2024-04-24T08:55:06.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.849+0000 7f678c200640 1 -- start start 2024-04-24T08:55:06.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.849+0000 7f678c200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6784101140 0x7f6784107e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:06.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.849+0000 7f678c200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6784106210 0x7f67841083a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:06.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.849+0000 7f678c200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67841088e0 0x7f67841ab030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:06.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.849+0000 7f678c200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f67841122c0 con 0x7f67841088e0 2024-04-24T08:55:06.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.849+0000 7f678c200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6784112140 con 0x7f6784101140 2024-04-24T08:55:06.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.849+0000 7f678c200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6784112440 con 0x7f6784106210 2024-04-24T08:55:06.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.850+0000 7f678a200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67841088e0 0x7f67841ab030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:06.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.850+0000 7f678a200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67841088e0 0x7f67841ab030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34794/0 (socket says 172.21.15.83:34794) 2024-04-24T08:55:06.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.850+0000 7f678a200640 1 -- 172.21.15.83:0/3697064928 learned_addr learned my addr 172.21.15.83:0/3697064928 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:55:06.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.850+0000 7f6789800640 1 --2- 172.21.15.83:0/3697064928 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6784101140 0x7f6784107e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:06.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.850+0000 7f6788e00640 1 --2- 172.21.15.83:0/3697064928 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6784106210 0x7f67841083a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:06.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.850+0000 7f678a200640 1 -- 172.21.15.83:0/3697064928 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6784106210 msgr2=0x7f67841083a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:06.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.850+0000 7f678a200640 1 --2- 172.21.15.83:0/3697064928 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6784106210 0x7f67841083a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:06.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.850+0000 7f678a200640 1 -- 172.21.15.83:0/3697064928 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6784101140 msgr2=0x7f6784107e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:06.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.850+0000 7f678a200640 1 --2- 172.21.15.83:0/3697064928 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6784101140 0x7f6784107e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:06.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.850+0000 7f678a200640 1 -- 172.21.15.83:0/3697064928 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67841ab570 con 0x7f67841088e0 2024-04-24T08:55:06.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.850+0000 7f678a200640 1 --2- 172.21.15.83:0/3697064928 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67841088e0 0x7f67841ab030 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f677c00cce0 tx=0x7f677c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:06.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.851+0000 7f6782000640 1 -- 172.21.15.83:0/3697064928 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f677c00f450 con 0x7f67841088e0 2024-04-24T08:55:06.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.851+0000 7f6782000640 1 -- 172.21.15.83:0/3697064928 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f677c00faa0 con 0x7f67841088e0 2024-04-24T08:55:06.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.851+0000 7f678c200640 1 -- 172.21.15.83:0/3697064928 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67841ab860 con 0x7f67841088e0 2024-04-24T08:55:06.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.851+0000 7f6782000640 1 -- 172.21.15.83:0/3697064928 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f677c018b70 con 0x7f67841088e0 2024-04-24T08:55:06.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.851+0000 7f678c200640 1 -- 172.21.15.83:0/3697064928 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67841abd80 con 0x7f67841088e0 2024-04-24T08:55:06.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.854+0000 7f678ef8f740 1 Processor -- start 2024-04-24T08:55:06.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.855+0000 7f6782000640 1 -- 172.21.15.83:0/3697064928 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f677c018d10 con 0x7f67841088e0 2024-04-24T08:55:06.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.855+0000 7f678ef8f740 1 -- start start 2024-04-24T08:55:06.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.856+0000 7f678ef8f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562d14b56950 0x562d14b56d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:06.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.856+0000 7f678ef8f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562d14b57270 0x562d14b61400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:06.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.856+0000 7f678ef8f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562d14b61940 0x562d14b63d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:06.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.856+0000 7f678ef8f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562d14abc1d0 con 0x562d14b57270 2024-04-24T08:55:06.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.856+0000 7f678ef8f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562d14a62450 con 0x562d14b61940 2024-04-24T08:55:06.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.856+0000 7f678ef8f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x562d14ab8190 con 0x562d14b56950 2024-04-24T08:55:06.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.856+0000 7f6789800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562d14b57270 0x562d14b61400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:06.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.856+0000 7f6789800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562d14b57270 0x562d14b61400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34808/0 (socket says 172.21.15.83:34808) 2024-04-24T08:55:06.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.856+0000 7f6789800640 1 -- 172.21.15.83:0/1404890698 learned_addr learned my addr 172.21.15.83:0/1404890698 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:55:06.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.856+0000 7f6788e00640 1 --2- 172.21.15.83:0/1404890698 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562d14b56950 0x562d14b56d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:06.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.856+0000 7f6788e00640 1 --2- 172.21.15.83:0/1404890698 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562d14b61940 0x562d14b63d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:06.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.856+0000 7f6789800640 1 -- 172.21.15.83:0/1404890698 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562d14b56950 msgr2=0x562d14b56d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:06.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.856+0000 7f6789800640 1 --2- 172.21.15.83:0/1404890698 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562d14b56950 0x562d14b56d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:06.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.856+0000 7f6789800640 1 -- 172.21.15.83:0/1404890698 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562d14b61940 msgr2=0x562d14b63d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:06.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.856+0000 7f6789800640 1 --2- 172.21.15.83:0/1404890698 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562d14b61940 0x562d14b63d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:06.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.857+0000 7f6789800640 1 -- 172.21.15.83:0/1404890698 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d14b1f310 con 0x562d14b57270 2024-04-24T08:55:06.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.857+0000 7f6788e00640 1 --2- 172.21.15.83:0/1404890698 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562d14b61940 0x562d14b63d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:55:06.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.857+0000 7f6789800640 1 --2- 172.21.15.83:0/1404890698 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562d14b57270 0x562d14b61400 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f677800bce0 tx=0x7f677800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=310f04f3dde10027 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:06.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.857+0000 7f6782000640 1 --2- 172.21.15.83:0/3697064928 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f676c0a3da0 0x7f676c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:06.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.857+0000 7f6782000640 1 -- 172.21.15.83:0/3697064928 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f677c0d0fe0 con 0x7f67841088e0 2024-04-24T08:55:06.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.857+0000 7f6776a00640 1 -- 172.21.15.83:0/1404890698 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6778015070 con 0x562d14b57270 2024-04-24T08:55:06.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.857+0000 7f6788e00640 1 --2- 172.21.15.83:0/3697064928 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f676c0a3da0 0x7f676c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:06.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.857+0000 7f6776a00640 1 -- 172.21.15.83:0/1404890698 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f67780118f0 con 0x562d14b57270 2024-04-24T08:55:06.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.857+0000 7f6776a00640 1 -- 172.21.15.83:0/1404890698 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6778010990 con 0x562d14b57270 2024-04-24T08:55:06.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.858+0000 7f678ef8f740 1 -- 172.21.15.83:0/1404890698 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562d14b57270 msgr2=0x562d14b61400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:06.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.858+0000 7f678ef8f740 1 --2- 172.21.15.83:0/1404890698 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562d14b57270 0x562d14b61400 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f677800bce0 tx=0x7f677800f730 comp rx=0 tx=0).stop 2024-04-24T08:55:06.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.858+0000 7f6788e00640 1 --2- 172.21.15.83:0/3697064928 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f676c0a3da0 0x7f676c0a6260 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f6770006010 tx=0x7f6770031400 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:06.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.859+0000 7f678ef8f740 1 -- 172.21.15.83:0/1404890698 shutdown_connections 2024-04-24T08:55:06.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.859+0000 7f678ef8f740 1 --2- 172.21.15.83:0/1404890698 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562d14b61940 0x562d14b63d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:06.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.859+0000 7f678ef8f740 1 --2- 172.21.15.83:0/1404890698 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562d14b57270 0x562d14b61400 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:06.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.859+0000 7f678ef8f740 1 --2- 172.21.15.83:0/1404890698 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562d14b56950 0x562d14b56d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:06.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.859+0000 7f678ef8f740 1 -- 172.21.15.83:0/1404890698 >> 172.21.15.83:0/1404890698 conn(0x562d14b52c70 msgr2=0x562d14b53050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:06.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.859+0000 7f678ef8f740 1 -- 172.21.15.83:0/1404890698 shutdown_connections 2024-04-24T08:55:06.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.859+0000 7f678ef8f740 1 -- 172.21.15.83:0/1404890698 wait complete. 2024-04-24T08:55:06.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.859+0000 7f678ef8f740 1 -- start start 2024-04-24T08:55:06.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.859+0000 7f678ef8f740 1 Processor -- start 2024-04-24T08:55:06.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.860+0000 7f678ef8f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562d14b56950 0x562d14b613a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:06.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.860+0000 7f678ef8f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562d14b57270 0x562d14b5e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:06.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.860+0000 7f678ef8f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562d14b61940 0x562d14b5edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:06.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.860+0000 7f678ef8f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562d14b647d0 con 0x562d14b61940 2024-04-24T08:55:06.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.860+0000 7f678ef8f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562d14a75810 con 0x562d14b57270 2024-04-24T08:55:06.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.860+0000 7f678ef8f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x562d14b64950 con 0x562d14b56950 2024-04-24T08:55:06.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.860+0000 7f678ef8f740 10 client.0 _ll_register_callbacks cb 0x562d14a43140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:55:06.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.860+0000 7f678a200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562d14b61940 0x562d14b5edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:06.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.860+0000 7f678a200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562d14b61940 0x562d14b5edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34810/0 (socket says 172.21.15.83:34810) 2024-04-24T08:55:06.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.860+0000 7f678a200640 1 -- 172.21.15.83:0/4246569873 learned_addr learned my addr 172.21.15.83:0/4246569873 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:55:06.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.860+0000 7f6789800640 1 --2- 172.21.15.83:0/4246569873 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562d14b56950 0x562d14b613a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:06.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.860+0000 7f6788e00640 1 --2- 172.21.15.83:0/4246569873 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562d14b57270 0x562d14b5e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:06.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.861+0000 7f678a200640 1 -- 172.21.15.83:0/4246569873 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562d14b56950 msgr2=0x562d14b613a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:06.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.861+0000 7f678a200640 1 --2- 172.21.15.83:0/4246569873 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562d14b56950 0x562d14b613a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:06.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.861+0000 7f678a200640 1 -- 172.21.15.83:0/4246569873 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562d14b57270 msgr2=0x562d14b5e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:06.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.861+0000 7f678a200640 1 --2- 172.21.15.83:0/4246569873 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562d14b57270 0x562d14b5e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:06.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.861+0000 7f678a200640 1 -- 172.21.15.83:0/4246569873 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d14a863f0 con 0x562d14b61940 2024-04-24T08:55:06.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.861+0000 7f6788e00640 1 --2- 172.21.15.83:0/4246569873 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562d14b57270 0x562d14b5e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:55:06.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.861+0000 7f678a200640 1 --2- 172.21.15.83:0/4246569873 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562d14b61940 0x562d14b5edb0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f677c0a52f0 tx=0x7f677c09c5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc4d438bf7e9ff67 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:06.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.861+0000 7f676b400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:55:06.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.861+0000 7f676b400640 1 -- 172.21.15.83:0/4246569873 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f677c0b6060 con 0x562d14b61940 2024-04-24T08:55:06.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.861+0000 7f676b400640 1 -- 172.21.15.83:0/4246569873 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f677c09ce10 con 0x562d14b61940 2024-04-24T08:55:06.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.861+0000 7f678ef8f740 1 -- 172.21.15.83:0/4246569873 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67780118f0 con 0x562d14b61940 2024-04-24T08:55:06.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.861+0000 7f676b400640 1 -- 172.21.15.83:0/4246569873 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f677c0a0f60 con 0x562d14b61940 2024-04-24T08:55:06.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.861+0000 7f678ef8f740 10 client.16917 fetch_fsmap learned FSMap version 30 2024-04-24T08:55:06.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.861+0000 7f678ef8f740 1 -- 172.21.15.83:0/4246569873 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562d147ce3f0 con 0x562d14b61940 2024-04-24T08:55:06.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.862+0000 7f676b400640 1 -- 172.21.15.83:0/4246569873 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f67780118f0 con 0x562d14b61940 2024-04-24T08:55:06.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.862+0000 7f676b400640 1 -- 172.21.15.83:0/4246569873 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7f677c09c000 con 0x562d14b61940 2024-04-24T08:55:06.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.862+0000 7f678ef8f740 10 client.16917 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T08:55:06.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.862+0000 7f678ef8f740 10 client.16917 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-24T08:55:06.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.862+0000 7f678ef8f740 20 client.16917 populate_metadata read hostname 'smithi083' 2024-04-24T08:55:06.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.862+0000 7f678ef8f740 1 --2- 172.21.15.83:0/4246569873 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x562d14b71b00 0x562d14b73f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:06.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.862+0000 7f678ef8f740 4 client.16917 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:55:06.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.862+0000 7f678ef8f740 1 -- 172.21.15.83:0/4246569873 --> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562d14ac9bf0 con 0x562d14b71b00 2024-04-24T08:55:06.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.863+0000 7f6789800640 1 --2- 172.21.15.83:0/4246569873 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x562d14b71b00 0x562d14b73f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:06.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.864+0000 7f6789800640 1 --2- 172.21.15.83:0/4246569873 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x562d14b71b00 0x562d14b73f70 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7d447919bfc51841 server_cookie=33fb3a7e958e5caf in_seq=0 out_seq=0 2024-04-24T08:55:06.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.864+0000 7f676b400640 10 client.16917 ms_handle_connect on v2:172.21.15.138:6834/1909219758 2024-04-24T08:55:06.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.867+0000 7f676b400640 1 -- 172.21.15.83:0/4246569873 <== mds.2 v2:172.21.15.138:6834/1909219758 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562d14ac9bf0 con 0x562d14b71b00 2024-04-24T08:55:06.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.867+0000 7f676b400640 10 client.16917 handle_command_reply: tid=0 2024-04-24T08:55:06.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.867+0000 7f678ef8f740 1 client.16917 shutdown 2024-04-24T08:55:06.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.867+0000 7f678ef8f740 1 -- 172.21.15.83:0/4246569873 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562d14b61940 msgr2=0x562d14b5edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:06.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.867+0000 7f678ef8f740 1 --2- 172.21.15.83:0/4246569873 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562d14b61940 0x562d14b5edb0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f677c0a52f0 tx=0x7f677c09c5d0 comp rx=0 tx=0).stop 2024-04-24T08:55:06.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.867+0000 7f678ef8f740 1 -- 172.21.15.83:0/4246569873 shutdown_connections 2024-04-24T08:55:06.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.867+0000 7f678ef8f740 1 --2- 172.21.15.83:0/4246569873 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x562d14b71b00 0x562d14b73f70 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:06.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.868+0000 7f678ef8f740 1 --2- 172.21.15.83:0/4246569873 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562d14b61940 0x562d14b5edb0 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:06.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.868+0000 7f678ef8f740 1 --2- 172.21.15.83:0/4246569873 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562d14b57270 0x562d14b5e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:06.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.868+0000 7f676b400640 0 client.16917 ms_handle_reset on v2:172.21.15.138:6834/1909219758 2024-04-24T08:55:06.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.868+0000 7f678ef8f740 1 --2- 172.21.15.83:0/4246569873 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562d14b56950 0x562d14b613a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:06.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.868+0000 7f678ef8f740 1 -- 172.21.15.83:0/4246569873 >> 172.21.15.83:0/4246569873 conn(0x562d14b52c70 msgr2=0x562d14b58a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:06.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.868+0000 7f678ef8f740 1 -- 172.21.15.83:0/4246569873 shutdown_connections 2024-04-24T08:55:06.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.868+0000 7f678ef8f740 1 -- 172.21.15.83:0/4246569873 wait complete. 2024-04-24T08:55:06.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.868+0000 7f678ef8f740 20 client.16917 trim_cache size 0 max 16384 2024-04-24T08:55:06.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.885+0000 7f678ef8f740 1 Processor -- start 2024-04-24T08:55:06.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.885+0000 7f678ef8f740 1 -- start start 2024-04-24T08:55:06.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.885+0000 7f678ef8f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562d14b56950 0x562d14b6d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:06.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.885+0000 7f678ef8f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562d14b57270 0x562d14b52570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:06.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.885+0000 7f678ef8f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562d14b61940 0x562d14b60ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:06.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.885+0000 7f678ef8f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f67841b5a60 con 0x562d14b56950 2024-04-24T08:55:06.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.885+0000 7f678ef8f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f67841b7630 con 0x562d14b57270 2024-04-24T08:55:06.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.885+0000 7f678ef8f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f67841b6130 con 0x562d14b61940 2024-04-24T08:55:06.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.886+0000 7f6789800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562d14b56950 0x562d14b6d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:06.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.886+0000 7f6789800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562d14b56950 0x562d14b6d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34814/0 (socket says 172.21.15.83:34814) 2024-04-24T08:55:06.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.886+0000 7f6789800640 1 -- 172.21.15.83:0/3865959744 learned_addr learned my addr 172.21.15.83:0/3865959744 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:55:06.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.886+0000 7f6788e00640 1 --2- 172.21.15.83:0/3865959744 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562d14b57270 0x562d14b52570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:06.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.886+0000 7f678a200640 1 --2- 172.21.15.83:0/3865959744 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562d14b61940 0x562d14b60ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:06.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.886+0000 7f6789800640 1 -- 172.21.15.83:0/3865959744 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562d14b61940 msgr2=0x562d14b60ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:06.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.886+0000 7f6789800640 1 --2- 172.21.15.83:0/3865959744 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562d14b61940 0x562d14b60ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:06.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.886+0000 7f6789800640 1 -- 172.21.15.83:0/3865959744 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562d14b57270 msgr2=0x562d14b52570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:06.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.886+0000 7f6789800640 1 --2- 172.21.15.83:0/3865959744 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562d14b57270 0x562d14b52570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:06.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.886+0000 7f6789800640 1 -- 172.21.15.83:0/3865959744 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d148987d0 con 0x562d14b56950 2024-04-24T08:55:06.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.887+0000 7f6789800640 1 --2- 172.21.15.83:0/3865959744 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562d14b56950 0x562d14b6d690 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x562d14b57f10 tx=0x7f6778005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cb9931103e7fe18 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:06.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.888+0000 7f6776000640 1 -- 172.21.15.83:0/3865959744 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6778015040 con 0x562d14b56950 2024-04-24T08:55:06.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.888+0000 7f6776000640 1 -- 172.21.15.83:0/3865959744 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6778010680 con 0x562d14b56950 2024-04-24T08:55:06.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.888+0000 7f6776000640 1 -- 172.21.15.83:0/3865959744 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67780048a0 con 0x562d14b56950 2024-04-24T08:55:06.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.889+0000 7f678ef8f740 1 -- 172.21.15.83:0/3865959744 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562d14b56950 msgr2=0x562d14b6d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:06.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.889+0000 7f678ef8f740 1 --2- 172.21.15.83:0/3865959744 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562d14b56950 0x562d14b6d690 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x562d14b57f10 tx=0x7f6778005e50 comp rx=0 tx=0).stop 2024-04-24T08:55:06.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.889+0000 7f678ef8f740 1 -- 172.21.15.83:0/3865959744 shutdown_connections 2024-04-24T08:55:06.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.889+0000 7f678ef8f740 1 --2- 172.21.15.83:0/3865959744 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562d14b61940 0x562d14b60ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:06.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.889+0000 7f678ef8f740 1 --2- 172.21.15.83:0/3865959744 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562d14b57270 0x562d14b52570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:06.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.889+0000 7f678ef8f740 1 --2- 172.21.15.83:0/3865959744 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562d14b56950 0x562d14b6d690 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:06.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.889+0000 7f678ef8f740 1 -- 172.21.15.83:0/3865959744 >> 172.21.15.83:0/3865959744 conn(0x562d14b52c70 msgr2=0x562d14b5d6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:06.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.889+0000 7f678ef8f740 1 -- 172.21.15.83:0/3865959744 shutdown_connections 2024-04-24T08:55:06.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.889+0000 7f678ef8f740 1 -- 172.21.15.83:0/3865959744 wait complete. 2024-04-24T08:55:06.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.890+0000 7f678ef8f740 1 -- start start 2024-04-24T08:55:06.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.890+0000 7f678ef8f740 1 Processor -- start 2024-04-24T08:55:06.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.890+0000 7f678ef8f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562d14b56950 0x562d14b69f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:06.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.890+0000 7f678ef8f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562d14b57270 0x562d14b5b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:06.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.890+0000 7f678ef8f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562d14b61940 0x562d14b5b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:06.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.890+0000 7f678ef8f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f67841b6ed0 con 0x562d14b57270 2024-04-24T08:55:06.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.890+0000 7f678ef8f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f67841b6800 con 0x562d14b56950 2024-04-24T08:55:06.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.890+0000 7f678ef8f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f67841b3a40 con 0x562d14b61940 2024-04-24T08:55:06.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.890+0000 7f678ef8f740 10 client.0 _ll_register_callbacks cb 0x7f677c0077a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:55:06.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.891+0000 7f6788e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562d14b57270 0x562d14b5b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:06.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.891+0000 7f6788e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562d14b57270 0x562d14b5b020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34822/0 (socket says 172.21.15.83:34822) 2024-04-24T08:55:06.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.891+0000 7f6788e00640 1 -- 172.21.15.83:0/1436475263 learned_addr learned my addr 172.21.15.83:0/1436475263 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:55:06.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.891+0000 7f678a200640 1 --2- 172.21.15.83:0/1436475263 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562d14b61940 0x562d14b5b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:06.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.891+0000 7f6789800640 1 --2- 172.21.15.83:0/1436475263 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562d14b56950 0x562d14b69f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:06.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.891+0000 7f6788e00640 1 -- 172.21.15.83:0/1436475263 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562d14b61940 msgr2=0x562d14b5b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:06.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.891+0000 7f6788e00640 1 --2- 172.21.15.83:0/1436475263 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562d14b61940 0x562d14b5b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:06.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.891+0000 7f6788e00640 1 -- 172.21.15.83:0/1436475263 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562d14b56950 msgr2=0x562d14b69f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:06.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.891+0000 7f6788e00640 1 --2- 172.21.15.83:0/1436475263 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562d14b56950 0x562d14b69f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:06.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.891+0000 7f6788e00640 1 -- 172.21.15.83:0/1436475263 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d14b5bd90 con 0x562d14b57270 2024-04-24T08:55:06.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.891+0000 7f6788e00640 1 --2- 172.21.15.83:0/1436475263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562d14b57270 0x562d14b5b020 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f6770009900 tx=0x7f6770000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5537e0a6ba1dc349 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:06.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.892+0000 7f676b400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:55:06.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.892+0000 7f676b400640 1 -- 172.21.15.83:0/1436475263 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6770035040 con 0x562d14b57270 2024-04-24T08:55:06.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.892+0000 7f678ef8f740 1 -- 172.21.15.83:0/1436475263 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6778010680 con 0x562d14b57270 2024-04-24T08:55:06.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.892+0000 7f676b400640 1 -- 172.21.15.83:0/1436475263 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f67700035f0 con 0x562d14b57270 2024-04-24T08:55:06.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.892+0000 7f676b400640 1 -- 172.21.15.83:0/1436475263 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f677005c050 con 0x562d14b57270 2024-04-24T08:55:06.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.892+0000 7f678ef8f740 10 client.16923 fetch_fsmap learned FSMap version 30 2024-04-24T08:55:06.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.892+0000 7f678ef8f740 1 -- 172.21.15.83:0/1436475263 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562d14bd8110 con 0x562d14b57270 2024-04-24T08:55:06.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.892+0000 7f676b400640 1 -- 172.21.15.83:0/1436475263 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6778010680 con 0x562d14b57270 2024-04-24T08:55:06.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.893+0000 7f676b400640 1 -- 172.21.15.83:0/1436475263 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7f67700328c0 con 0x562d14b57270 2024-04-24T08:55:06.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.893+0000 7f678ef8f740 10 client.16923 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T08:55:06.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.893+0000 7f678ef8f740 10 client.16923 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-24T08:55:06.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.893+0000 7f678ef8f740 20 client.16923 populate_metadata read hostname 'smithi083' 2024-04-24T08:55:06.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.893+0000 7f678ef8f740 1 --2- 172.21.15.83:0/1436475263 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x562d14bd88f0 0x562d14bdad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:06.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.893+0000 7f678ef8f740 4 client.16923 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T08:55:06.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.893+0000 7f678ef8f740 1 -- 172.21.15.83:0/1436475263 --> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562d14ac3a60 con 0x562d14bd88f0 2024-04-24T08:55:06.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.893+0000 7f6789800640 1 --2- 172.21.15.83:0/1436475263 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x562d14bd88f0 0x562d14bdad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:06.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.894+0000 7f6789800640 1 --2- 172.21.15.83:0/1436475263 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x562d14bd88f0 0x562d14bdad10 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f94d6d8c2f7461f9 server_cookie=23adc351bb2dd1f4 in_seq=0 out_seq=0 2024-04-24T08:55:06.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.894+0000 7f676b400640 10 client.16923 ms_handle_connect on v2:172.21.15.138:6834/1909219758 2024-04-24T08:55:06.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.895+0000 7f676b400640 1 -- 172.21.15.83:0/1436475263 <== mds.2 v2:172.21.15.138:6834/1909219758 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562d14ac3a60 con 0x562d14bd88f0 2024-04-24T08:55:06.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.895+0000 7f676b400640 10 client.16923 handle_command_reply: tid=0 2024-04-24T08:55:06.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.895+0000 7f678ef8f740 1 client.16923 shutdown 2024-04-24T08:55:06.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.895+0000 7f678ef8f740 1 -- 172.21.15.83:0/1436475263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562d14b57270 msgr2=0x562d14b5b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:06.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.895+0000 7f678ef8f740 1 --2- 172.21.15.83:0/1436475263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562d14b57270 0x562d14b5b020 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f6770009900 tx=0x7f6770000bc0 comp rx=0 tx=0).stop 2024-04-24T08:55:06.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.895+0000 7f678ef8f740 1 -- 172.21.15.83:0/1436475263 shutdown_connections 2024-04-24T08:55:06.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.895+0000 7f678ef8f740 1 --2- 172.21.15.83:0/1436475263 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x562d14bd88f0 0x562d14bdad10 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:06.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.895+0000 7f678ef8f740 1 --2- 172.21.15.83:0/1436475263 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562d14b61940 0x562d14b5b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:06.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.895+0000 7f678ef8f740 1 --2- 172.21.15.83:0/1436475263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562d14b57270 0x562d14b5b020 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:06.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.895+0000 7f676b400640 0 client.16923 ms_handle_reset on v2:172.21.15.138:6834/1909219758 2024-04-24T08:55:06.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.895+0000 7f678ef8f740 1 --2- 172.21.15.83:0/1436475263 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562d14b56950 0x562d14b69f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:06.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.895+0000 7f678ef8f740 1 -- 172.21.15.83:0/1436475263 >> 172.21.15.83:0/1436475263 conn(0x562d14b52c70 msgr2=0x562d14b6c680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:06.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.895+0000 7f678ef8f740 1 -- 172.21.15.83:0/1436475263 shutdown_connections 2024-04-24T08:55:06.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.895+0000 7f678ef8f740 1 -- 172.21.15.83:0/1436475263 wait complete. 2024-04-24T08:55:06.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.896+0000 7f678ef8f740 20 client.16923 trim_cache size 0 max 16384 2024-04-24T08:55:06.898 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T08:55:06.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.896+0000 7f678c200640 1 -- 172.21.15.83:0/3697064928 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f676c0a3da0 msgr2=0x7f676c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:06.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.896+0000 7f678c200640 1 --2- 172.21.15.83:0/3697064928 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f676c0a3da0 0x7f676c0a6260 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f6770006010 tx=0x7f6770031400 comp rx=0 tx=0).stop 2024-04-24T08:55:06.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.896+0000 7f678c200640 1 -- 172.21.15.83:0/3697064928 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67841088e0 msgr2=0x7f67841ab030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:06.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.896+0000 7f678c200640 1 --2- 172.21.15.83:0/3697064928 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67841088e0 0x7f67841ab030 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f677c00cce0 tx=0x7f677c00a620 comp rx=0 tx=0).stop 2024-04-24T08:55:06.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.896+0000 7f678c200640 1 -- 172.21.15.83:0/3697064928 shutdown_connections 2024-04-24T08:55:06.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.896+0000 7f678c200640 1 --2- 172.21.15.83:0/3697064928 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f676c0a3da0 0x7f676c0a6260 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:06.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.896+0000 7f678c200640 1 --2- 172.21.15.83:0/3697064928 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67841088e0 0x7f67841ab030 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:06.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.896+0000 7f678c200640 1 --2- 172.21.15.83:0/3697064928 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6784106210 0x7f67841083a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:06.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.896+0000 7f678c200640 1 --2- 172.21.15.83:0/3697064928 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6784101140 0x7f6784107e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:06.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.896+0000 7f678c200640 1 -- 172.21.15.83:0/3697064928 >> 172.21.15.83:0/3697064928 conn(0x7f6784078000 msgr2=0x7f678410c580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:06.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.896+0000 7f678c200640 1 -- 172.21.15.83:0/3697064928 shutdown_connections 2024-04-24T08:55:06.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:06.897+0000 7f678c200640 1 -- 172.21.15.83:0/3697064928 wait complete. 2024-04-24T08:55:06.915 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T08:55:07.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.117+0000 7f33ad000640 1 -- 172.21.15.83:0/2816717056 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33a8108610 msgr2=0x7f33a8108a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:07.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.117+0000 7f33ad000640 1 --2- 172.21.15.83:0/2816717056 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33a8108610 0x7f33a8108a90 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f3390009cd0 tx=0x7f339002a8e0 comp rx=0 tx=0).stop 2024-04-24T08:55:07.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.118+0000 7f33ad000640 1 -- 172.21.15.83:0/2816717056 shutdown_connections 2024-04-24T08:55:07.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.118+0000 7f33ad000640 1 --2- 172.21.15.83:0/2816717056 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f33a8108fd0 0x7f33a8115490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:07.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.118+0000 7f33ad000640 1 --2- 172.21.15.83:0/2816717056 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33a8108610 0x7f33a8108a90 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:07.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.118+0000 7f33ad000640 1 --2- 172.21.15.83:0/2816717056 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f33a810e660 0x7f33a810ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:07.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.118+0000 7f33ad000640 1 -- 172.21.15.83:0/2816717056 >> 172.21.15.83:0/2816717056 conn(0x7f33a8076540 msgr2=0x7f33a8078960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:07.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.118+0000 7f33ad000640 1 -- 172.21.15.83:0/2816717056 shutdown_connections 2024-04-24T08:55:07.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.118+0000 7f33ad000640 1 -- 172.21.15.83:0/2816717056 wait complete. 2024-04-24T08:55:07.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.119+0000 7f33ad000640 1 Processor -- start 2024-04-24T08:55:07.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.119+0000 7f33ad000640 1 -- start start 2024-04-24T08:55:07.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.119+0000 7f33ad000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f33a8108610 0x7f33a819eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:07.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.119+0000 7f33ad000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33a8108fd0 0x7f33a819f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:07.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.119+0000 7f33ad000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f33a810e660 0x7f33a81a3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:07.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.119+0000 7f33ad000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f33a81188a0 con 0x7f33a8108fd0 2024-04-24T08:55:07.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.119+0000 7f33ad000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f33a8118720 con 0x7f33a810e660 2024-04-24T08:55:07.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.119+0000 7f33ad000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f33a8118a20 con 0x7f33a8108610 2024-04-24T08:55:07.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.120+0000 7f33a5400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33a8108fd0 0x7f33a819f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:07.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.120+0000 7f33a5400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33a8108fd0 0x7f33a819f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34840/0 (socket says 172.21.15.83:34840) 2024-04-24T08:55:07.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.120+0000 7f33a5400640 1 -- 172.21.15.83:0/782348666 learned_addr learned my addr 172.21.15.83:0/782348666 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:55:07.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.120+0000 7f33a6800640 1 --2- 172.21.15.83:0/782348666 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f33a810e660 0x7f33a81a3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:07.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.120+0000 7f33a5e00640 1 --2- 172.21.15.83:0/782348666 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f33a8108610 0x7f33a819eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:07.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.120+0000 7f33a5400640 1 -- 172.21.15.83:0/782348666 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f33a8108610 msgr2=0x7f33a819eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:07.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.120+0000 7f33a5400640 1 --2- 172.21.15.83:0/782348666 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f33a8108610 0x7f33a819eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:07.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.120+0000 7f33a5400640 1 -- 172.21.15.83:0/782348666 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f33a810e660 msgr2=0x7f33a81a3780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:07.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.120+0000 7f33a5400640 1 --2- 172.21.15.83:0/782348666 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f33a810e660 0x7f33a81a3780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:07.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.120+0000 7f33a5400640 1 -- 172.21.15.83:0/782348666 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33a81a3eb0 con 0x7f33a8108fd0 2024-04-24T08:55:07.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.120+0000 7f33a6800640 1 --2- 172.21.15.83:0/782348666 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f33a810e660 0x7f33a81a3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:55:07.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.121+0000 7f33a5400640 1 --2- 172.21.15.83:0/782348666 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33a8108fd0 0x7f33a819f400 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f3390005e30 tx=0x7f3390003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:07.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.121+0000 7f339ea00640 1 -- 172.21.15.83:0/782348666 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3390004450 con 0x7f33a8108fd0 2024-04-24T08:55:07.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.121+0000 7f339ea00640 1 -- 172.21.15.83:0/782348666 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3390031ce0 con 0x7f33a8108fd0 2024-04-24T08:55:07.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.121+0000 7f33ad000640 1 -- 172.21.15.83:0/782348666 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33a81a4140 con 0x7f33a8108fd0 2024-04-24T08:55:07.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.121+0000 7f339ea00640 1 -- 172.21.15.83:0/782348666 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3390046480 con 0x7f33a8108fd0 2024-04-24T08:55:07.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.121+0000 7f33ad000640 1 -- 172.21.15.83:0/782348666 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33a81aba30 con 0x7f33a8108fd0 2024-04-24T08:55:07.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.125+0000 7f339ea00640 1 -- 172.21.15.83:0/782348666 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f33900305a0 con 0x7f33a8108fd0 2024-04-24T08:55:07.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.125+0000 7f33afd9b740 1 Processor -- start 2024-04-24T08:55:07.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.125+0000 7f33afd9b740 1 -- start start 2024-04-24T08:55:07.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.125+0000 7f33afd9b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562163d40950 0x562163d40d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:07.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.125+0000 7f33afd9b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562163d41270 0x562163d4b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:07.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.125+0000 7f33afd9b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562163d4b940 0x562163d4dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:07.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.125+0000 7f33afd9b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562163ca61d0 con 0x562163d41270 2024-04-24T08:55:07.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.125+0000 7f33afd9b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562163c4c450 con 0x562163d4b940 2024-04-24T08:55:07.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.125+0000 7f33afd9b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x562163ca2190 con 0x562163d40950 2024-04-24T08:55:07.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.126+0000 7f33a5e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562163d41270 0x562163d4b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:07.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.126+0000 7f33a5e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562163d41270 0x562163d4b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34842/0 (socket says 172.21.15.83:34842) 2024-04-24T08:55:07.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.126+0000 7f33a5e00640 1 -- 172.21.15.83:0/2086663215 learned_addr learned my addr 172.21.15.83:0/2086663215 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:55:07.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.126+0000 7f33a6800640 1 --2- 172.21.15.83:0/2086663215 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562163d40950 0x562163d40d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:07.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.126+0000 7f33a5400640 1 --2- 172.21.15.83:0/2086663215 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562163d4b940 0x562163d4dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:07.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.126+0000 7f33a5e00640 1 -- 172.21.15.83:0/2086663215 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562163d40950 msgr2=0x562163d40d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:07.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.126+0000 7f33a5e00640 1 --2- 172.21.15.83:0/2086663215 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562163d40950 0x562163d40d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:07.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.126+0000 7f33a5e00640 1 -- 172.21.15.83:0/2086663215 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562163d4b940 msgr2=0x562163d4dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:07.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.126+0000 7f33a5e00640 1 --2- 172.21.15.83:0/2086663215 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562163d4b940 0x562163d4dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:07.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.126+0000 7f33a5e00640 1 -- 172.21.15.83:0/2086663215 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562163d09310 con 0x562163d41270 2024-04-24T08:55:07.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.127+0000 7f33a5e00640 1 --2- 172.21.15.83:0/2086663215 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562163d41270 0x562163d4b400 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f33a0009af0 tx=0x7f33a000e2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b953f4f388b0df6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:07.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.127+0000 7f339ea00640 1 --2- 172.21.15.83:0/782348666 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f33880a3d20 0x7f33880a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:07.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.127+0000 7f3397400640 1 -- 172.21.15.83:0/2086663215 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a000ed20 con 0x562163d41270 2024-04-24T08:55:07.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.127+0000 7f3397400640 1 -- 172.21.15.83:0/2086663215 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f33a00054d0 con 0x562163d41270 2024-04-24T08:55:07.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.127+0000 7f3397400640 1 -- 172.21.15.83:0/2086663215 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a0011470 con 0x562163d41270 2024-04-24T08:55:07.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.127+0000 7f339ea00640 1 -- 172.21.15.83:0/782348666 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f339002e030 con 0x7f33a8108fd0 2024-04-24T08:55:07.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.127+0000 7f33a6800640 1 --2- 172.21.15.83:0/782348666 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f33880a3d20 0x7f33880a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:07.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.128+0000 7f33afd9b740 1 -- 172.21.15.83:0/2086663215 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562163d41270 msgr2=0x562163d4b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:07.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.128+0000 7f33afd9b740 1 --2- 172.21.15.83:0/2086663215 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562163d41270 0x562163d4b400 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f33a0009af0 tx=0x7f33a000e2f0 comp rx=0 tx=0).stop 2024-04-24T08:55:07.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.128+0000 7f33afd9b740 1 -- 172.21.15.83:0/2086663215 shutdown_connections 2024-04-24T08:55:07.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.128+0000 7f33afd9b740 1 --2- 172.21.15.83:0/2086663215 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562163d4b940 0x562163d4dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:07.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.128+0000 7f33afd9b740 1 --2- 172.21.15.83:0/2086663215 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562163d41270 0x562163d4b400 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:07.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.128+0000 7f33afd9b740 1 --2- 172.21.15.83:0/2086663215 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562163d40950 0x562163d40d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:07.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.128+0000 7f33afd9b740 1 -- 172.21.15.83:0/2086663215 >> 172.21.15.83:0/2086663215 conn(0x562163d3cc70 msgr2=0x562163d3d050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:07.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.128+0000 7f33a6800640 1 --2- 172.21.15.83:0/782348666 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f33880a3d20 0x7f33880a61e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f339800bba0 tx=0x7f3398005e10 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:07.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.128+0000 7f33afd9b740 1 -- 172.21.15.83:0/2086663215 shutdown_connections 2024-04-24T08:55:07.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.128+0000 7f33afd9b740 1 -- 172.21.15.83:0/2086663215 wait complete. 2024-04-24T08:55:07.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.129+0000 7f33afd9b740 1 -- start start 2024-04-24T08:55:07.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.129+0000 7f33afd9b740 1 Processor -- start 2024-04-24T08:55:07.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.129+0000 7f33afd9b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562163d40950 0x562163d4b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:07.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.129+0000 7f33afd9b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562163d41270 0x562163d48630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:07.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.129+0000 7f33afd9b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562163d4b940 0x562163d48c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:07.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.129+0000 7f33afd9b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562163d4e7d0 con 0x562163d4b940 2024-04-24T08:55:07.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.129+0000 7f33afd9b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562163c5f810 con 0x562163d41270 2024-04-24T08:55:07.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.130+0000 7f33afd9b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x562163d4e950 con 0x562163d40950 2024-04-24T08:55:07.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.130+0000 7f33afd9b740 10 client.0 _ll_register_callbacks cb 0x562163c2d140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:55:07.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.130+0000 7f33a6800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562163d4b940 0x562163d48c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:07.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.130+0000 7f33a6800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562163d4b940 0x562163d48c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34848/0 (socket says 172.21.15.83:34848) 2024-04-24T08:55:07.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.130+0000 7f33a6800640 1 -- 172.21.15.83:0/1322383767 learned_addr learned my addr 172.21.15.83:0/1322383767 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:55:07.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.130+0000 7f33a5e00640 1 --2- 172.21.15.83:0/1322383767 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562163d40950 0x562163d4b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:07.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.130+0000 7f33a5400640 1 --2- 172.21.15.83:0/1322383767 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562163d41270 0x562163d48630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:07.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.130+0000 7f33a6800640 1 -- 172.21.15.83:0/1322383767 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562163d40950 msgr2=0x562163d4b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:07.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.130+0000 7f33a6800640 1 --2- 172.21.15.83:0/1322383767 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562163d40950 0x562163d4b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:07.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.130+0000 7f33a6800640 1 -- 172.21.15.83:0/1322383767 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562163d41270 msgr2=0x562163d48630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:07.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.130+0000 7f33a6800640 1 --2- 172.21.15.83:0/1322383767 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562163d41270 0x562163d48630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:07.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.130+0000 7f33a6800640 1 -- 172.21.15.83:0/1322383767 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562163c703f0 con 0x562163d4b940 2024-04-24T08:55:07.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.130+0000 7f33a5e00640 1 --2- 172.21.15.83:0/1322383767 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562163d40950 0x562163d4b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:55:07.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.130+0000 7f33a5400640 1 --2- 172.21.15.83:0/1322383767 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562163d41270 0x562163d48630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:55:07.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.131+0000 7f33a6800640 1 --2- 172.21.15.83:0/1322383767 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562163d4b940 0x562163d48c40 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f3398010a90 tx=0x7f33980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c3f61bcf9585309 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:07.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.131+0000 7f338fe00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:55:07.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.131+0000 7f338fe00640 1 -- 172.21.15.83:0/1322383767 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f339800de00 con 0x562163d4b940 2024-04-24T08:55:07.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.131+0000 7f338fe00640 1 -- 172.21.15.83:0/1322383767 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f339801d420 con 0x562163d4b940 2024-04-24T08:55:07.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.131+0000 7f33afd9b740 1 -- 172.21.15.83:0/1322383767 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33a00054d0 con 0x562163d4b940 2024-04-24T08:55:07.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.131+0000 7f338fe00640 1 -- 172.21.15.83:0/1322383767 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f339801d620 con 0x562163d4b940 2024-04-24T08:55:07.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.132+0000 7f33afd9b740 10 client.16935 fetch_fsmap learned FSMap version 30 2024-04-24T08:55:07.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.132+0000 7f33afd9b740 1 -- 172.21.15.83:0/1322383767 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5621639b83f0 con 0x562163d4b940 2024-04-24T08:55:07.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.132+0000 7f338fe00640 1 -- 172.21.15.83:0/1322383767 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f33a00054d0 con 0x562163d4b940 2024-04-24T08:55:07.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.132+0000 7f338fe00640 1 -- 172.21.15.83:0/1322383767 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7f3398025850 con 0x562163d4b940 2024-04-24T08:55:07.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.132+0000 7f33afd9b740 10 client.16935 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T08:55:07.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.132+0000 7f33afd9b740 10 client.16935 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-24T08:55:07.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.132+0000 7f33afd9b740 20 client.16935 populate_metadata read hostname 'smithi083' 2024-04-24T08:55:07.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.132+0000 7f33afd9b740 1 --2- 172.21.15.83:0/1322383767 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x562163d5bb10 0x562163d5df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:07.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.132+0000 7f33afd9b740 4 client.16935 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:55:07.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.132+0000 7f33afd9b740 1 -- 172.21.15.83:0/1322383767 --> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562163cb3bf0 con 0x562163d5bb10 2024-04-24T08:55:07.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.133+0000 7f33a5e00640 1 --2- 172.21.15.83:0/1322383767 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x562163d5bb10 0x562163d5df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:07.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.134+0000 7f33a5e00640 1 --2- 172.21.15.83:0/1322383767 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x562163d5bb10 0x562163d5df80 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9cd2b33ea6a3799e server_cookie=a7f3e9bb5d1efa85 in_seq=0 out_seq=0 2024-04-24T08:55:07.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.134+0000 7f338fe00640 10 client.16935 ms_handle_connect on v2:172.21.15.138:6836/900566754 2024-04-24T08:55:07.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.136+0000 7f338fe00640 1 -- 172.21.15.83:0/1322383767 <== mds.1 v2:172.21.15.138:6836/900566754 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562163cb3bf0 con 0x562163d5bb10 2024-04-24T08:55:07.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.136+0000 7f338fe00640 10 client.16935 handle_command_reply: tid=0 2024-04-24T08:55:07.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.137+0000 7f33afd9b740 1 client.16935 shutdown 2024-04-24T08:55:07.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.137+0000 7f33afd9b740 1 -- 172.21.15.83:0/1322383767 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562163d4b940 msgr2=0x562163d48c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:07.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.137+0000 7f33afd9b740 1 --2- 172.21.15.83:0/1322383767 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562163d4b940 0x562163d48c40 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f3398010a90 tx=0x7f33980046c0 comp rx=0 tx=0).stop 2024-04-24T08:55:07.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.137+0000 7f33afd9b740 1 -- 172.21.15.83:0/1322383767 shutdown_connections 2024-04-24T08:55:07.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.137+0000 7f33afd9b740 1 --2- 172.21.15.83:0/1322383767 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562163d4b940 0x562163d48c40 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:07.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.137+0000 7f33afd9b740 1 --2- 172.21.15.83:0/1322383767 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562163d41270 0x562163d48630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:07.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.137+0000 7f33afd9b740 1 --2- 172.21.15.83:0/1322383767 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x562163d5bb10 0x562163d5df80 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:07.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.137+0000 7f33afd9b740 1 --2- 172.21.15.83:0/1322383767 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562163d40950 0x562163d4b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:07.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.137+0000 7f338fe00640 0 client.16935 ms_handle_reset on v2:172.21.15.138:6836/900566754 2024-04-24T08:55:07.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.137+0000 7f33afd9b740 1 -- 172.21.15.83:0/1322383767 >> 172.21.15.83:0/1322383767 conn(0x562163d3cc70 msgr2=0x562163d42a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:07.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.137+0000 7f33afd9b740 1 -- 172.21.15.83:0/1322383767 shutdown_connections 2024-04-24T08:55:07.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.137+0000 7f33afd9b740 1 -- 172.21.15.83:0/1322383767 wait complete. 2024-04-24T08:55:07.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.137+0000 7f33afd9b740 20 client.16935 trim_cache size 0 max 16384 2024-04-24T08:55:07.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.155+0000 7f33afd9b740 1 Processor -- start 2024-04-24T08:55:07.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.156+0000 7f33afd9b740 1 -- start start 2024-04-24T08:55:07.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.156+0000 7f33afd9b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562163d40950 0x562163d57690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:07.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.156+0000 7f33afd9b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562163d41270 0x562163d4ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:07.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.156+0000 7f33afd9b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562163d4b940 0x562163d63850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:07.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.156+0000 7f33afd9b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f33a81b5ba0 con 0x562163d40950 2024-04-24T08:55:07.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.156+0000 7f33afd9b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f33a81b7770 con 0x562163d41270 2024-04-24T08:55:07.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.156+0000 7f33afd9b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f33a81b6270 con 0x562163d4b940 2024-04-24T08:55:07.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.156+0000 7f33a5e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562163d40950 0x562163d57690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:07.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.156+0000 7f33a5e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562163d40950 0x562163d57690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34862/0 (socket says 172.21.15.83:34862) 2024-04-24T08:55:07.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.156+0000 7f33a5e00640 1 -- 172.21.15.83:0/2233983208 learned_addr learned my addr 172.21.15.83:0/2233983208 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:55:07.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.158+0000 7f33a5400640 1 --2- 172.21.15.83:0/2233983208 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562163d41270 0x562163d4ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:07.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.158+0000 7f33a6800640 1 --2- 172.21.15.83:0/2233983208 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562163d4b940 0x562163d63850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:07.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.158+0000 7f33a5e00640 1 -- 172.21.15.83:0/2233983208 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562163d4b940 msgr2=0x562163d63850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:07.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.158+0000 7f33a5e00640 1 --2- 172.21.15.83:0/2233983208 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562163d4b940 0x562163d63850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:07.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.158+0000 7f33a5e00640 1 -- 172.21.15.83:0/2233983208 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562163d41270 msgr2=0x562163d4ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:07.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.158+0000 7f33a5e00640 1 --2- 172.21.15.83:0/2233983208 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562163d41270 0x562163d4ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:07.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.158+0000 7f33a5e00640 1 -- 172.21.15.83:0/2233983208 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562163a827d0 con 0x562163d40950 2024-04-24T08:55:07.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.158+0000 7f33a5e00640 1 --2- 172.21.15.83:0/2233983208 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562163d40950 0x562163d57690 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f33a0009af0 tx=0x7f33a0009400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d4de01e5232c407 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:07.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.158+0000 7f3396a00640 1 -- 172.21.15.83:0/2233983208 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a0004a80 con 0x562163d40950 2024-04-24T08:55:07.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.158+0000 7f3396a00640 1 -- 172.21.15.83:0/2233983208 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f33a002a170 con 0x562163d40950 2024-04-24T08:55:07.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.158+0000 7f3396a00640 1 -- 172.21.15.83:0/2233983208 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a0004f70 con 0x562163d40950 2024-04-24T08:55:07.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.158+0000 7f33afd9b740 1 -- 172.21.15.83:0/2233983208 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562163d40950 msgr2=0x562163d57690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:07.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.158+0000 7f33afd9b740 1 --2- 172.21.15.83:0/2233983208 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562163d40950 0x562163d57690 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f33a0009af0 tx=0x7f33a0009400 comp rx=0 tx=0).stop 2024-04-24T08:55:07.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.159+0000 7f33afd9b740 1 -- 172.21.15.83:0/2233983208 shutdown_connections 2024-04-24T08:55:07.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.159+0000 7f33afd9b740 1 --2- 172.21.15.83:0/2233983208 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562163d4b940 0x562163d63850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:07.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.159+0000 7f33afd9b740 1 --2- 172.21.15.83:0/2233983208 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562163d41270 0x562163d4ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:07.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.159+0000 7f33afd9b740 1 --2- 172.21.15.83:0/2233983208 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562163d40950 0x562163d57690 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:07.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.159+0000 7f33afd9b740 1 -- 172.21.15.83:0/2233983208 >> 172.21.15.83:0/2233983208 conn(0x562163d3cc70 msgr2=0x562163d485f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:07.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.159+0000 7f33afd9b740 1 -- 172.21.15.83:0/2233983208 shutdown_connections 2024-04-24T08:55:07.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.159+0000 7f33afd9b740 1 -- 172.21.15.83:0/2233983208 wait complete. 2024-04-24T08:55:07.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.159+0000 7f33afd9b740 1 -- start start 2024-04-24T08:55:07.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.160+0000 7f33afd9b740 1 Processor -- start 2024-04-24T08:55:07.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.160+0000 7f33afd9b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562163d40950 0x562163d4cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:07.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.160+0000 7f33afd9b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562163d41270 0x562163d4d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:07.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.160+0000 7f33afd9b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562163d4b940 0x562163d52260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:07.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.160+0000 7f33afd9b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f33a81b7010 con 0x562163d41270 2024-04-24T08:55:07.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.160+0000 7f33afd9b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f33a81b6940 con 0x562163d40950 2024-04-24T08:55:07.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.160+0000 7f33afd9b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f33a81b3b80 con 0x562163d4b940 2024-04-24T08:55:07.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.160+0000 7f33afd9b740 10 client.0 _ll_register_callbacks cb 0x7f339800b810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:55:07.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.161+0000 7f33a5400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562163d41270 0x562163d4d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:07.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.161+0000 7f33a5400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562163d41270 0x562163d4d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34872/0 (socket says 172.21.15.83:34872) 2024-04-24T08:55:07.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.161+0000 7f33a5400640 1 -- 172.21.15.83:0/221364138 learned_addr learned my addr 172.21.15.83:0/221364138 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:55:07.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.161+0000 7f33a5e00640 1 --2- 172.21.15.83:0/221364138 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562163d40950 0x562163d4cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:07.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.161+0000 7f33a6800640 1 --2- 172.21.15.83:0/221364138 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562163d4b940 0x562163d52260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:07.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.161+0000 7f33a5400640 1 -- 172.21.15.83:0/221364138 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562163d4b940 msgr2=0x562163d52260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:07.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.161+0000 7f33a5400640 1 --2- 172.21.15.83:0/221364138 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562163d4b940 0x562163d52260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:07.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.161+0000 7f33a5400640 1 -- 172.21.15.83:0/221364138 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562163d40950 msgr2=0x562163d4cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:07.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.161+0000 7f33a5400640 1 --2- 172.21.15.83:0/221364138 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562163d40950 0x562163d4cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:07.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.161+0000 7f33a5400640 1 -- 172.21.15.83:0/221364138 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562163d527a0 con 0x562163d41270 2024-04-24T08:55:07.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.162+0000 7f33a5400640 1 --2- 172.21.15.83:0/221364138 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562163d41270 0x562163d4d2a0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f33900bdc00 tx=0x7f33900bd7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14bac8a692ab8a8b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:07.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.162+0000 7f338fe00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:55:07.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.162+0000 7f338fe00640 1 -- 172.21.15.83:0/221364138 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33900be370 con 0x562163d41270 2024-04-24T08:55:07.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.162+0000 7f338fe00640 1 -- 172.21.15.83:0/221364138 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f33900ba2e0 con 0x562163d41270 2024-04-24T08:55:07.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.162+0000 7f33afd9b740 1 -- 172.21.15.83:0/221364138 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33a002a170 con 0x562163d41270 2024-04-24T08:55:07.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.162+0000 7f338fe00640 1 -- 172.21.15.83:0/221364138 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3390044e40 con 0x562163d41270 2024-04-24T08:55:07.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.162+0000 7f33afd9b740 10 client.16941 fetch_fsmap learned FSMap version 30 2024-04-24T08:55:07.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.162+0000 7f33afd9b740 1 -- 172.21.15.83:0/221364138 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562163dc28a0 con 0x562163d41270 2024-04-24T08:55:07.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.163+0000 7f338fe00640 1 -- 172.21.15.83:0/221364138 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f33a002a170 con 0x562163d41270 2024-04-24T08:55:07.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.163+0000 7f338fe00640 1 -- 172.21.15.83:0/221364138 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7f3390030070 con 0x562163d41270 2024-04-24T08:55:07.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.163+0000 7f33afd9b740 10 client.16941 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T08:55:07.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.163+0000 7f33afd9b740 10 client.16941 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-24T08:55:07.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.163+0000 7f33afd9b740 20 client.16941 populate_metadata read hostname 'smithi083' 2024-04-24T08:55:07.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.163+0000 7f33afd9b740 1 --2- 172.21.15.83:0/221364138 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x562163dc2e60 0x562163dc5280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:07.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.163+0000 7f33afd9b740 4 client.16941 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T08:55:07.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.163+0000 7f33afd9b740 1 -- 172.21.15.83:0/221364138 --> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562163cada60 con 0x562163dc2e60 2024-04-24T08:55:07.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.164+0000 7f33a5e00640 1 --2- 172.21.15.83:0/221364138 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x562163dc2e60 0x562163dc5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:07.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.165+0000 7f33a5e00640 1 --2- 172.21.15.83:0/221364138 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x562163dc2e60 0x562163dc5280 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=15e54346a5b2ddfd server_cookie=353e67ba531f2553 in_seq=0 out_seq=0 2024-04-24T08:55:07.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.165+0000 7f338fe00640 10 client.16941 ms_handle_connect on v2:172.21.15.138:6836/900566754 2024-04-24T08:55:07.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.165+0000 7f338fe00640 1 -- 172.21.15.83:0/221364138 <== mds.1 v2:172.21.15.138:6836/900566754 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562163cada60 con 0x562163dc2e60 2024-04-24T08:55:07.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.165+0000 7f338fe00640 10 client.16941 handle_command_reply: tid=0 2024-04-24T08:55:07.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.165+0000 7f33afd9b740 1 client.16941 shutdown 2024-04-24T08:55:07.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.165+0000 7f33afd9b740 1 -- 172.21.15.83:0/221364138 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562163d41270 msgr2=0x562163d4d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:07.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.165+0000 7f33afd9b740 1 --2- 172.21.15.83:0/221364138 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562163d41270 0x562163d4d2a0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f33900bdc00 tx=0x7f33900bd7b0 comp rx=0 tx=0).stop 2024-04-24T08:55:07.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.166+0000 7f33afd9b740 1 -- 172.21.15.83:0/221364138 shutdown_connections 2024-04-24T08:55:07.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.166+0000 7f33afd9b740 1 --2- 172.21.15.83:0/221364138 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x562163dc2e60 0x562163dc5280 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:07.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.166+0000 7f33afd9b740 1 --2- 172.21.15.83:0/221364138 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562163d4b940 0x562163d52260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:07.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.166+0000 7f33afd9b740 1 --2- 172.21.15.83:0/221364138 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562163d41270 0x562163d4d2a0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:07.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.166+0000 7f33afd9b740 1 --2- 172.21.15.83:0/221364138 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562163d40950 0x562163d4cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:07.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.166+0000 7f338fe00640 0 client.16941 ms_handle_reset on v2:172.21.15.138:6836/900566754 2024-04-24T08:55:07.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.166+0000 7f33afd9b740 1 -- 172.21.15.83:0/221364138 >> 172.21.15.83:0/221364138 conn(0x562163d3cc70 msgr2=0x562163d3b560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:07.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.166+0000 7f33afd9b740 1 -- 172.21.15.83:0/221364138 shutdown_connections 2024-04-24T08:55:07.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.166+0000 7f33afd9b740 1 -- 172.21.15.83:0/221364138 wait complete. 2024-04-24T08:55:07.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.166+0000 7f33afd9b740 20 client.16941 trim_cache size 0 max 16384 2024-04-24T08:55:07.167 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T08:55:07.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.167+0000 7f33ad000640 1 -- 172.21.15.83:0/782348666 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f33880a3d20 msgr2=0x7f33880a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:07.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.167+0000 7f33ad000640 1 --2- 172.21.15.83:0/782348666 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f33880a3d20 0x7f33880a61e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f339800bba0 tx=0x7f3398005e10 comp rx=0 tx=0).stop 2024-04-24T08:55:07.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.167+0000 7f33ad000640 1 -- 172.21.15.83:0/782348666 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33a8108fd0 msgr2=0x7f33a819f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:07.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.167+0000 7f33ad000640 1 --2- 172.21.15.83:0/782348666 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33a8108fd0 0x7f33a819f400 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f3390005e30 tx=0x7f3390003c20 comp rx=0 tx=0).stop 2024-04-24T08:55:07.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.167+0000 7f33ad000640 1 -- 172.21.15.83:0/782348666 shutdown_connections 2024-04-24T08:55:07.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.167+0000 7f33ad000640 1 --2- 172.21.15.83:0/782348666 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f33880a3d20 0x7f33880a61e0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:07.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.167+0000 7f33ad000640 1 --2- 172.21.15.83:0/782348666 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f33a810e660 0x7f33a81a3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:07.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.168+0000 7f33ad000640 1 --2- 172.21.15.83:0/782348666 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33a8108fd0 0x7f33a819f400 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:07.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.168+0000 7f33ad000640 1 --2- 172.21.15.83:0/782348666 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f33a8108610 0x7f33a819eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:07.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.168+0000 7f33ad000640 1 -- 172.21.15.83:0/782348666 >> 172.21.15.83:0/782348666 conn(0x7f33a8076540 msgr2=0x7f33a8077db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:07.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.168+0000 7f33ad000640 1 -- 172.21.15.83:0/782348666 shutdown_connections 2024-04-24T08:55:07.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.168+0000 7f33ad000640 1 -- 172.21.15.83:0/782348666 wait complete. 2024-04-24T08:55:07.191 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:55:06 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/1290071886' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:55:07.193 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T08:55:07.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:55:06 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/1290071886' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:55:07.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:55:06 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/1290071886' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:55:07.259 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.258+0000 7fd2d6200640 1 -- 172.21.15.83:0/1223294760 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd2d0074c20 msgr2=0x7fd2d010e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:07.259 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.258+0000 7fd2d6200640 1 --2- 172.21.15.83:0/1223294760 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd2d0074c20 0x7fd2d010e3b0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc009900 tx=0x7fd2bc02a480 comp rx=0 tx=0).stop 2024-04-24T08:55:07.259 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.258+0000 7fd2d6200640 1 -- 172.21.15.83:0/1223294760 shutdown_connections 2024-04-24T08:55:07.259 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.258+0000 7fd2d6200640 1 --2- 172.21.15.83:0/1223294760 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd2d010e8f0 0x7fd2d0110ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:07.259 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.258+0000 7fd2d6200640 1 --2- 172.21.15.83:0/1223294760 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd2d0074c20 0x7fd2d010e3b0 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:07.259 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.258+0000 7fd2d6200640 1 --2- 172.21.15.83:0/1223294760 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd2d0074300 0x7fd2d00746e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:07.259 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.258+0000 7fd2d6200640 1 -- 172.21.15.83:0/1223294760 >> 172.21.15.83:0/1223294760 conn(0x7fd2d0079490 msgr2=0x7fd2d00798a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:07.259 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.259+0000 7fd2d6200640 1 -- 172.21.15.83:0/1223294760 shutdown_connections 2024-04-24T08:55:07.259 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.259+0000 7fd2d6200640 1 -- 172.21.15.83:0/1223294760 wait complete. 2024-04-24T08:55:07.261 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.260+0000 7fd2d6200640 1 Processor -- start 2024-04-24T08:55:07.262 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.260+0000 7fd2d6200640 1 -- start start 2024-04-24T08:55:07.262 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.260+0000 7fd2d6200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd2d0074300 0x7fd2d01a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:07.262 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.260+0000 7fd2d6200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd2d0074c20 0x7fd2d01a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:07.262 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.260+0000 7fd2d6200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd2d010e8f0 0x7fd2d019a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:07.262 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.260+0000 7fd2d6200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd2d01144f0 con 0x7fd2d010e8f0 2024-04-24T08:55:07.262 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.260+0000 7fd2d6200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd2d0114370 con 0x7fd2d0074c20 2024-04-24T08:55:07.262 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.260+0000 7fd2d6200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd2d0114670 con 0x7fd2d0074300 2024-04-24T08:55:07.262 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.261+0000 7fd2cfe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd2d010e8f0 0x7fd2d019a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:07.262 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.261+0000 7fd2cfe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd2d010e8f0 0x7fd2d019a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34900/0 (socket says 172.21.15.83:34900) 2024-04-24T08:55:07.262 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.261+0000 7fd2cfe00640 1 -- 172.21.15.83:0/1070112997 learned_addr learned my addr 172.21.15.83:0/1070112997 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:55:07.262 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.261+0000 7fd2cea00640 1 --2- 172.21.15.83:0/1070112997 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd2d0074c20 0x7fd2d01a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:07.262 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.261+0000 7fd2cf400640 1 --2- 172.21.15.83:0/1070112997 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd2d0074300 0x7fd2d01a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:07.263 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.261+0000 7fd2cfe00640 1 -- 172.21.15.83:0/1070112997 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd2d0074300 msgr2=0x7fd2d01a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:07.263 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.261+0000 7fd2cfe00640 1 --2- 172.21.15.83:0/1070112997 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd2d0074300 0x7fd2d01a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:07.263 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.261+0000 7fd2cfe00640 1 -- 172.21.15.83:0/1070112997 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd2d0074c20 msgr2=0x7fd2d01a0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:07.263 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.261+0000 7fd2cfe00640 1 --2- 172.21.15.83:0/1070112997 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd2d0074c20 0x7fd2d01a0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:07.263 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.261+0000 7fd2cfe00640 1 -- 172.21.15.83:0/1070112997 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2d019aff0 con 0x7fd2d010e8f0 2024-04-24T08:55:07.263 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.262+0000 7fd2cfe00640 1 --2- 172.21.15.83:0/1070112997 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd2d010e8f0 0x7fd2d019a730 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fd2c800e960 tx=0x7fd2c800ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:07.264 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.262+0000 7fd2c7e00640 1 -- 172.21.15.83:0/1070112997 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c8004d20 con 0x7fd2d010e8f0 2024-04-24T08:55:07.264 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.262+0000 7fd2c7e00640 1 -- 172.21.15.83:0/1070112997 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd2c8005020 con 0x7fd2d010e8f0 2024-04-24T08:55:07.264 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.262+0000 7fd2c7e00640 1 -- 172.21.15.83:0/1070112997 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c80116d0 con 0x7fd2d010e8f0 2024-04-24T08:55:07.264 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.262+0000 7fd2d6200640 1 -- 172.21.15.83:0/1070112997 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2d019b2e0 con 0x7fd2d010e8f0 2024-04-24T08:55:07.264 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.263+0000 7fd2d6200640 1 -- 172.21.15.83:0/1070112997 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2d01a75a0 con 0x7fd2d010e8f0 2024-04-24T08:55:07.266 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.266+0000 7fd2d8f84740 1 Processor -- start 2024-04-24T08:55:07.266 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.266+0000 7fd2c7e00640 1 -- 172.21.15.83:0/1070112997 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd2c80051d0 con 0x7fd2d010e8f0 2024-04-24T08:55:07.267 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.267+0000 7fd2d8f84740 1 -- start start 2024-04-24T08:55:07.267 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.267+0000 7fd2d8f84740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556b107571b0 0x556b10757590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:07.268 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.267+0000 7fd2d8f84740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b10757ad0 0x556b10761e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:07.269 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.267+0000 7fd2d8f84740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556b10762390 0x556b10764780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:07.269 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.267+0000 7fd2d8f84740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556b106bd900 con 0x556b10762390 2024-04-24T08:55:07.269 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.267+0000 7fd2d8f84740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556b10663f20 con 0x556b10757ad0 2024-04-24T08:55:07.269 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.267+0000 7fd2d8f84740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x556b106b98c0 con 0x556b107571b0 2024-04-24T08:55:07.269 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.267+0000 7fd2cea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556b10762390 0x556b10764780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:07.269 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.267+0000 7fd2cea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556b10762390 0x556b10764780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34914/0 (socket says 172.21.15.83:34914) 2024-04-24T08:55:07.269 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.267+0000 7fd2cea00640 1 -- 172.21.15.83:0/1458071409 learned_addr learned my addr 172.21.15.83:0/1458071409 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:55:07.270 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.268+0000 7fd2cea00640 1 --2- 172.21.15.83:0/1458071409 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556b107571b0 0x556b10757590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:07.270 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.268+0000 7fd2cf400640 1 --2- 172.21.15.83:0/1458071409 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b10757ad0 0x556b10761e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:07.273 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.268+0000 7fd2cea00640 1 -- 172.21.15.83:0/1458071409 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556b107571b0 msgr2=0x556b10757590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:07.273 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.268+0000 7fd2cea00640 1 --2- 172.21.15.83:0/1458071409 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556b107571b0 0x556b10757590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:07.274 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.268+0000 7fd2cea00640 1 -- 172.21.15.83:0/1458071409 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b10757ad0 msgr2=0x556b10761e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:07.274 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.268+0000 7fd2cea00640 1 --2- 172.21.15.83:0/1458071409 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b10757ad0 0x556b10761e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:07.274 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.268+0000 7fd2cea00640 1 -- 172.21.15.83:0/1458071409 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b10688e10 con 0x556b10762390 2024-04-24T08:55:07.274 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.268+0000 7fd2c7e00640 1 --2- 172.21.15.83:0/1070112997 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fd2b40a3dc0 0x7fd2b40a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:07.274 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.268+0000 7fd2cea00640 1 --2- 172.21.15.83:0/1458071409 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556b10762390 0x556b10764780 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc005c40 tx=0x7fd2bc005bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9afaca3221f8b49d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:07.274 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.269+0000 7fd2c7e00640 1 -- 172.21.15.83:0/1070112997 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7fd2c80d10c0 con 0x7fd2d010e8f0 2024-04-24T08:55:07.274 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.269+0000 7fd2bbe00640 1 -- 172.21.15.83:0/1458071409 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc035040 con 0x556b10762390 2024-04-24T08:55:07.274 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.269+0000 7fd2bbe00640 1 -- 172.21.15.83:0/1458071409 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd2bc0040c0 con 0x556b10762390 2024-04-24T08:55:07.274 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.269+0000 7fd2bbe00640 1 -- 172.21.15.83:0/1458071409 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc031d30 con 0x556b10762390 2024-04-24T08:55:07.274 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.269+0000 7fd2cf400640 1 --2- 172.21.15.83:0/1070112997 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fd2b40a3dc0 0x7fd2b40a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:07.275 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.270+0000 7fd2d8f84740 1 -- 172.21.15.83:0/1458071409 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556b10762390 msgr2=0x556b10764780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:07.275 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.270+0000 7fd2cf400640 1 --2- 172.21.15.83:0/1070112997 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fd2b40a3dc0 0x7fd2b40a6280 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x556b10758750 tx=0x7fd2c000b4d0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:07.275 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.270+0000 7fd2d8f84740 1 --2- 172.21.15.83:0/1458071409 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556b10762390 0x556b10764780 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc005c40 tx=0x7fd2bc005bb0 comp rx=0 tx=0).stop 2024-04-24T08:55:07.275 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.270+0000 7fd2d8f84740 1 -- 172.21.15.83:0/1458071409 shutdown_connections 2024-04-24T08:55:07.275 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.270+0000 7fd2d8f84740 1 --2- 172.21.15.83:0/1458071409 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556b10762390 0x556b10764780 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:07.275 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.270+0000 7fd2d8f84740 1 --2- 172.21.15.83:0/1458071409 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b10757ad0 0x556b10761e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:07.275 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.270+0000 7fd2d8f84740 1 --2- 172.21.15.83:0/1458071409 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556b107571b0 0x556b10757590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:07.275 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.270+0000 7fd2d8f84740 1 -- 172.21.15.83:0/1458071409 >> 172.21.15.83:0/1458071409 conn(0x556b107536c0 msgr2=0x556b10753aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:07.275 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.270+0000 7fd2d8f84740 1 -- 172.21.15.83:0/1458071409 shutdown_connections 2024-04-24T08:55:07.275 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.270+0000 7fd2d8f84740 1 -- 172.21.15.83:0/1458071409 wait complete. 2024-04-24T08:55:07.275 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.271+0000 7fd2d8f84740 1 -- start start 2024-04-24T08:55:07.275 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.271+0000 7fd2d8f84740 1 Processor -- start 2024-04-24T08:55:07.275 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.271+0000 7fd2d8f84740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b107571b0 0x556b1075f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:07.275 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.272+0000 7fd2d8f84740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556b10757ad0 0x556b1075f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:07.276 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.272+0000 7fd2d8f84740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556b10762390 0x556b1075fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:07.276 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.272+0000 7fd2d8f84740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556b10764f60 con 0x556b10757ad0 2024-04-24T08:55:07.276 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.272+0000 7fd2d8f84740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556b106772c0 con 0x556b107571b0 2024-04-24T08:55:07.276 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.272+0000 7fd2d8f84740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x556b107650e0 con 0x556b10762390 2024-04-24T08:55:07.276 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.272+0000 7fd2d8f84740 10 client.0 _ll_register_callbacks cb 0x556b103cfa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:55:07.276 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.272+0000 7fd2cea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556b10757ad0 0x556b1075f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:07.276 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.272+0000 7fd2cea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556b10757ad0 0x556b1075f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34920/0 (socket says 172.21.15.83:34920) 2024-04-24T08:55:07.276 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.272+0000 7fd2cea00640 1 -- 172.21.15.83:0/2566085586 learned_addr learned my addr 172.21.15.83:0/2566085586 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:55:07.276 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.272+0000 7fd2cf400640 1 --2- 172.21.15.83:0/2566085586 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b107571b0 0x556b1075f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:07.276 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.272+0000 7fd2cfe00640 1 --2- 172.21.15.83:0/2566085586 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556b10762390 0x556b1075fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:07.276 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.272+0000 7fd2cea00640 1 -- 172.21.15.83:0/2566085586 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556b10762390 msgr2=0x556b1075fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:07.276 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.272+0000 7fd2cea00640 1 --2- 172.21.15.83:0/2566085586 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556b10762390 0x556b1075fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:07.276 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.272+0000 7fd2cea00640 1 -- 172.21.15.83:0/2566085586 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b107571b0 msgr2=0x556b1075f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:07.276 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.272+0000 7fd2cea00640 1 --2- 172.21.15.83:0/2566085586 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b107571b0 0x556b1075f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:07.276 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.272+0000 7fd2cea00640 1 -- 172.21.15.83:0/2566085586 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b10687fd0 con 0x556b10757ad0 2024-04-24T08:55:07.277 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.272+0000 7fd2cfe00640 1 --2- 172.21.15.83:0/2566085586 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556b10762390 0x556b1075fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:55:07.277 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.273+0000 7fd2cea00640 1 --2- 172.21.15.83:0/2566085586 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556b10757ad0 0x556b1075f850 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc02a9a0 tx=0x7fd2bc009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=857458339cc62dd2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:07.277 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.273+0000 7fd2b8c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:55:07.277 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.273+0000 7fd2b8c00640 1 -- 172.21.15.83:0/2566085586 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc035070 con 0x556b10757ad0 2024-04-24T08:55:07.277 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.273+0000 7fd2d8f84740 1 -- 172.21.15.83:0/2566085586 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd2bc0040c0 con 0x556b10757ad0 2024-04-24T08:55:07.277 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.273+0000 7fd2b8c00640 1 -- 172.21.15.83:0/2566085586 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7fd2bc031d10 con 0x556b10757ad0 2024-04-24T08:55:07.277 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.273+0000 7fd2b8c00640 1 -- 172.21.15.83:0/2566085586 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd2bc05a050 con 0x556b10757ad0 2024-04-24T08:55:07.277 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.273+0000 7fd2d8f84740 10 client.16953 fetch_fsmap learned FSMap version 30 2024-04-24T08:55:07.277 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.273+0000 7fd2d8f84740 1 -- 172.21.15.83:0/2566085586 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556b103d0210 con 0x556b10757ad0 2024-04-24T08:55:07.280 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.274+0000 7fd2b8c00640 1 -- 172.21.15.83:0/2566085586 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd2bc0040c0 con 0x556b10757ad0 2024-04-24T08:55:07.280 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.274+0000 7fd2b8c00640 1 -- 172.21.15.83:0/2566085586 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7fd2bc03a040 con 0x556b10757ad0 2024-04-24T08:55:07.280 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.274+0000 7fd2d8f84740 10 client.16953 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T08:55:07.280 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.274+0000 7fd2d8f84740 10 client.16953 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T08:55:07.280 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.274+0000 7fd2d8f84740 20 client.16953 populate_metadata read hostname 'smithi083' 2024-04-24T08:55:07.280 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.274+0000 7fd2d8f84740 1 --2- 172.21.15.83:0/2566085586 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556b107722e0 0x556b107746d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:07.280 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.274+0000 7fd2d8f84740 4 client.16953 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:55:07.281 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.274+0000 7fd2d8f84740 1 -- 172.21.15.83:0/2566085586 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556b106cb320 con 0x556b107722e0 2024-04-24T08:55:07.281 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.275+0000 7fd2cf400640 1 --2- 172.21.15.83:0/2566085586 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556b107722e0 0x556b107746d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:07.281 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.276+0000 7fd2cf400640 1 --2- 172.21.15.83:0/2566085586 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556b107722e0 0x556b107746d0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b7039f05c183569 server_cookie=53523fa82c5abaa0 in_seq=0 out_seq=0 2024-04-24T08:55:07.281 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.276+0000 7fd2b8c00640 10 client.16953 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:55:07.281 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.278+0000 7fd2b8c00640 1 -- 172.21.15.83:0/2566085586 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556b106cb320 con 0x556b107722e0 2024-04-24T08:55:07.281 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.278+0000 7fd2b8c00640 10 client.16953 handle_command_reply: tid=0 2024-04-24T08:55:07.281 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.278+0000 7fd2d8f84740 1 client.16953 shutdown 2024-04-24T08:55:07.282 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.279+0000 7fd2d8f84740 1 -- 172.21.15.83:0/2566085586 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556b10757ad0 msgr2=0x556b1075f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:07.282 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.279+0000 7fd2d8f84740 1 --2- 172.21.15.83:0/2566085586 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556b10757ad0 0x556b1075f850 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc02a9a0 tx=0x7fd2bc009260 comp rx=0 tx=0).stop 2024-04-24T08:55:07.282 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.279+0000 7fd2d8f84740 1 -- 172.21.15.83:0/2566085586 shutdown_connections 2024-04-24T08:55:07.282 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.279+0000 7fd2d8f84740 1 --2- 172.21.15.83:0/2566085586 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556b107722e0 0x556b107746d0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:07.282 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.279+0000 7fd2d8f84740 1 --2- 172.21.15.83:0/2566085586 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556b10762390 0x556b1075fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:07.282 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.279+0000 7fd2d8f84740 1 --2- 172.21.15.83:0/2566085586 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556b10757ad0 0x556b1075f850 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:07.283 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.279+0000 7fd2d8f84740 1 --2- 172.21.15.83:0/2566085586 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b107571b0 0x556b1075f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:07.283 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.279+0000 7fd2d8f84740 1 -- 172.21.15.83:0/2566085586 >> 172.21.15.83:0/2566085586 conn(0x556b107536c0 msgr2=0x556b107593f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:07.283 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.279+0000 7fd2b8c00640 0 client.16953 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:55:07.283 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.279+0000 7fd2d8f84740 1 -- 172.21.15.83:0/2566085586 shutdown_connections 2024-04-24T08:55:07.283 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.279+0000 7fd2d8f84740 1 -- 172.21.15.83:0/2566085586 wait complete. 2024-04-24T08:55:07.283 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.279+0000 7fd2d8f84740 20 client.16953 trim_cache size 0 max 16384 2024-04-24T08:55:07.299 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.298+0000 7fd2d8f84740 1 Processor -- start 2024-04-24T08:55:07.299 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.298+0000 7fd2d8f84740 1 -- start start 2024-04-24T08:55:07.300 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.299+0000 7fd2d8f84740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556b107571b0 0x556b10759ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:07.300 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.299+0000 7fd2d8f84740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b10757ad0 0x556b1075fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:07.300 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.299+0000 7fd2d8f84740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556b10762390 0x556b10777310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:07.300 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.299+0000 7fd2d8f84740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd2d01b1690 con 0x556b10762390 2024-04-24T08:55:07.300 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.299+0000 7fd2d8f84740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd2d01b3260 con 0x556b10757ad0 2024-04-24T08:55:07.300 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.299+0000 7fd2d8f84740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd2d01b1d60 con 0x556b107571b0 2024-04-24T08:55:07.300 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.299+0000 7fd2cfe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556b10762390 0x556b10777310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:07.300 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.299+0000 7fd2cfe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556b10762390 0x556b10777310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34934/0 (socket says 172.21.15.83:34934) 2024-04-24T08:55:07.300 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.299+0000 7fd2cfe00640 1 -- 172.21.15.83:0/979099463 learned_addr learned my addr 172.21.15.83:0/979099463 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:55:07.300 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.299+0000 7fd2cea00640 1 --2- 172.21.15.83:0/979099463 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b10757ad0 0x556b1075fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:07.300 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.299+0000 7fd2cf400640 1 --2- 172.21.15.83:0/979099463 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556b107571b0 0x556b10759ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:07.300 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.299+0000 7fd2cfe00640 1 -- 172.21.15.83:0/979099463 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556b107571b0 msgr2=0x556b10759ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:07.300 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.299+0000 7fd2cfe00640 1 --2- 172.21.15.83:0/979099463 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556b107571b0 0x556b10759ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:07.300 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.300+0000 7fd2cfe00640 1 -- 172.21.15.83:0/979099463 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b10757ad0 msgr2=0x556b1075fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:07.300 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.300+0000 7fd2cfe00640 1 --2- 172.21.15.83:0/979099463 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b10757ad0 0x556b1075fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:07.300 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.300+0000 7fd2cfe00640 1 -- 172.21.15.83:0/979099463 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b10764af0 con 0x556b10762390 2024-04-24T08:55:07.302 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.300+0000 7fd2cfe00640 1 --2- 172.21.15.83:0/979099463 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556b10762390 0x556b10777310 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fd2c800c640 tx=0x7fd2c809c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d9b80e861eaff33 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:07.302 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.300+0000 7fd2bb400640 1 -- 172.21.15.83:0/979099463 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c801f050 con 0x556b10762390 2024-04-24T08:55:07.302 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.300+0000 7fd2bb400640 1 -- 172.21.15.83:0/979099463 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd2c80a0fe0 con 0x556b10762390 2024-04-24T08:55:07.302 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.300+0000 7fd2bb400640 1 -- 172.21.15.83:0/979099463 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c8087040 con 0x556b10762390 2024-04-24T08:55:07.302 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.301+0000 7fd2d8f84740 1 -- 172.21.15.83:0/979099463 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556b10762390 msgr2=0x556b10777310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:07.303 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.301+0000 7fd2d8f84740 1 --2- 172.21.15.83:0/979099463 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556b10762390 0x556b10777310 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fd2c800c640 tx=0x7fd2c809c650 comp rx=0 tx=0).stop 2024-04-24T08:55:07.303 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.301+0000 7fd2d8f84740 1 -- 172.21.15.83:0/979099463 shutdown_connections 2024-04-24T08:55:07.303 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.301+0000 7fd2d8f84740 1 --2- 172.21.15.83:0/979099463 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556b10762390 0x556b10777310 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:07.303 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.301+0000 7fd2d8f84740 1 --2- 172.21.15.83:0/979099463 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b10757ad0 0x556b1075fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:07.303 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.301+0000 7fd2d8f84740 1 --2- 172.21.15.83:0/979099463 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556b107571b0 0x556b10759ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:07.303 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.301+0000 7fd2d8f84740 1 -- 172.21.15.83:0/979099463 >> 172.21.15.83:0/979099463 conn(0x556b107536c0 msgr2=0x556b1076dbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:07.303 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.301+0000 7fd2d8f84740 1 -- 172.21.15.83:0/979099463 shutdown_connections 2024-04-24T08:55:07.303 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.301+0000 7fd2d8f84740 1 -- 172.21.15.83:0/979099463 wait complete. 2024-04-24T08:55:07.303 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.302+0000 7fd2d8f84740 1 -- start start 2024-04-24T08:55:07.304 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.302+0000 7fd2d8f84740 1 Processor -- start 2024-04-24T08:55:07.307 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.302+0000 7fd2d8f84740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556b107571b0 0x556b10777640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:07.307 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.303+0000 7fd2d8f84740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556b10757ad0 0x556b107749d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:07.308 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.303+0000 7fd2d8f84740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b10762390 0x556b10774f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:07.308 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.303+0000 7fd2d8f84740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd2d01b2b00 con 0x556b107571b0 2024-04-24T08:55:07.308 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.303+0000 7fd2d8f84740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd2d01b2430 con 0x556b10762390 2024-04-24T08:55:07.308 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.303+0000 7fd2d8f84740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd2d01af670 con 0x556b10757ad0 2024-04-24T08:55:07.308 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.303+0000 7fd2d8f84740 10 client.0 _ll_register_callbacks cb 0x7fd2c000b3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:55:07.308 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.303+0000 7fd2cf400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556b107571b0 0x556b10777640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:07.308 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.303+0000 7fd2cf400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556b107571b0 0x556b10777640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34944/0 (socket says 172.21.15.83:34944) 2024-04-24T08:55:07.308 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.303+0000 7fd2cf400640 1 -- 172.21.15.83:0/1385225297 learned_addr learned my addr 172.21.15.83:0/1385225297 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:55:07.309 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.303+0000 7fd2cfe00640 1 --2- 172.21.15.83:0/1385225297 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b10762390 0x556b10774f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:07.309 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.303+0000 7fd2cea00640 1 --2- 172.21.15.83:0/1385225297 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556b10757ad0 0x556b107749d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:07.309 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.303+0000 7fd2cf400640 1 -- 172.21.15.83:0/1385225297 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556b10757ad0 msgr2=0x556b107749d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:07.309 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.303+0000 7fd2cf400640 1 --2- 172.21.15.83:0/1385225297 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556b10757ad0 0x556b107749d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:07.309 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.303+0000 7fd2cf400640 1 -- 172.21.15.83:0/1385225297 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b10762390 msgr2=0x556b10774f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:07.309 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.303+0000 7fd2cf400640 1 --2- 172.21.15.83:0/1385225297 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b10762390 0x556b10774f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:07.309 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.303+0000 7fd2cf400640 1 -- 172.21.15.83:0/1385225297 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b1049c620 con 0x556b107571b0 2024-04-24T08:55:07.309 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.304+0000 7fd2cf400640 1 --2- 172.21.15.83:0/1385225297 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556b107571b0 0x556b10777640 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fd2c000bf20 tx=0x7fd2c00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6ba38a89cd562d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:07.310 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.304+0000 7fd2b8c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:55:07.310 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.304+0000 7fd2b8c00640 1 -- 172.21.15.83:0/1385225297 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c002b070 con 0x556b107571b0 2024-04-24T08:55:07.310 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.304+0000 7fd2b8c00640 1 -- 172.21.15.83:0/1385225297 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7fd2c0027420 con 0x556b107571b0 2024-04-24T08:55:07.310 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.304+0000 7fd2d8f84740 1 -- 172.21.15.83:0/1385225297 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd2c80a0fe0 con 0x556b107571b0 2024-04-24T08:55:07.310 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.304+0000 7fd2b8c00640 1 -- 172.21.15.83:0/1385225297 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd2c0027e40 con 0x556b107571b0 2024-04-24T08:55:07.310 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.304+0000 7fd2d8f84740 10 client.16959 fetch_fsmap learned FSMap version 30 2024-04-24T08:55:07.310 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.304+0000 7fd2d8f84740 1 -- 172.21.15.83:0/1385225297 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556b107d8270 con 0x556b107571b0 2024-04-24T08:55:07.310 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.305+0000 7fd2b8c00640 1 -- 172.21.15.83:0/1385225297 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd2c80a0fe0 con 0x556b107571b0 2024-04-24T08:55:07.312 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.305+0000 7fd2b8c00640 1 -- 172.21.15.83:0/1385225297 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7fd2c00304a0 con 0x556b107571b0 2024-04-24T08:55:07.312 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.305+0000 7fd2d8f84740 10 client.16959 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T08:55:07.312 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.305+0000 7fd2d8f84740 10 client.16959 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T08:55:07.312 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.305+0000 7fd2d8f84740 20 client.16959 populate_metadata read hostname 'smithi083' 2024-04-24T08:55:07.312 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.305+0000 7fd2d8f84740 1 --2- 172.21.15.83:0/1385225297 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556b107d8a60 0x556b107dae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:07.312 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.305+0000 7fd2d8f84740 4 client.16959 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T08:55:07.312 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.305+0000 7fd2d8f84740 1 -- 172.21.15.83:0/1385225297 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556b106c5190 con 0x556b107d8a60 2024-04-24T08:55:07.312 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.306+0000 7fd2cea00640 1 --2- 172.21.15.83:0/1385225297 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556b107d8a60 0x556b107dae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:07.312 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.307+0000 7fd2b8c00640 10 client.16959 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:55:07.312 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.307+0000 7fd2cea00640 1 --2- 172.21.15.83:0/1385225297 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556b107d8a60 0x556b107dae80 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5c4c1a1b9f9ad10 server_cookie=2af37c9d03270bbd in_seq=0 out_seq=0 2024-04-24T08:55:07.370 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.369+0000 7fd2b8c00640 1 -- 172.21.15.83:0/1385225297 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556b106c5190 con 0x556b107d8a60 2024-04-24T08:55:07.370 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.369+0000 7fd2b8c00640 10 client.16959 handle_command_reply: tid=0 2024-04-24T08:55:07.370 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.369+0000 7fd2d8f84740 1 client.16959 shutdown 2024-04-24T08:55:07.370 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.369+0000 7fd2d8f84740 1 -- 172.21.15.83:0/1385225297 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556b107571b0 msgr2=0x556b10777640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:07.371 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.369+0000 7fd2d8f84740 1 --2- 172.21.15.83:0/1385225297 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556b107571b0 0x556b10777640 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fd2c000bf20 tx=0x7fd2c00046c0 comp rx=0 tx=0).stop 2024-04-24T08:55:07.371 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.370+0000 7fd2d8f84740 1 -- 172.21.15.83:0/1385225297 shutdown_connections 2024-04-24T08:55:07.371 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.370+0000 7fd2d8f84740 1 --2- 172.21.15.83:0/1385225297 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556b107d8a60 0x556b107dae80 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:07.371 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.370+0000 7fd2d8f84740 1 --2- 172.21.15.83:0/1385225297 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556b10762390 0x556b10774f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:07.371 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.370+0000 7fd2d8f84740 1 --2- 172.21.15.83:0/1385225297 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556b10757ad0 0x556b107749d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:07.371 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.370+0000 7fd2d8f84740 1 --2- 172.21.15.83:0/1385225297 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556b107571b0 0x556b10777640 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:07.371 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.370+0000 7fd2d8f84740 1 -- 172.21.15.83:0/1385225297 >> 172.21.15.83:0/1385225297 conn(0x556b107536c0 msgr2=0x556b10760b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:07.371 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.370+0000 7fd2b8c00640 0 client.16959 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:55:07.371 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.370+0000 7fd2d8f84740 1 -- 172.21.15.83:0/1385225297 shutdown_connections 2024-04-24T08:55:07.371 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.370+0000 7fd2d8f84740 1 -- 172.21.15.83:0/1385225297 wait complete. 2024-04-24T08:55:07.371 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.370+0000 7fd2d8f84740 20 client.16959 trim_cache size 0 max 16384 2024-04-24T08:55:07.372 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T08:55:07.372 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T08:55:07.372 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T08:55:07.372 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T08:55:07.373 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.371+0000 7fd2d6200640 1 -- 172.21.15.83:0/1070112997 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fd2b40a3dc0 msgr2=0x7fd2b40a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:07.373 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.371+0000 7fd2d6200640 1 --2- 172.21.15.83:0/1070112997 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fd2b40a3dc0 0x7fd2b40a6280 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x556b10758750 tx=0x7fd2c000b4d0 comp rx=0 tx=0).stop 2024-04-24T08:55:07.373 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.371+0000 7fd2d6200640 1 -- 172.21.15.83:0/1070112997 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd2d010e8f0 msgr2=0x7fd2d019a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:07.373 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.371+0000 7fd2d6200640 1 --2- 172.21.15.83:0/1070112997 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd2d010e8f0 0x7fd2d019a730 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fd2c800e960 tx=0x7fd2c800ee20 comp rx=0 tx=0).stop 2024-04-24T08:55:07.373 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.372+0000 7fd2d6200640 1 -- 172.21.15.83:0/1070112997 shutdown_connections 2024-04-24T08:55:07.373 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.372+0000 7fd2d6200640 1 --2- 172.21.15.83:0/1070112997 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fd2b40a3dc0 0x7fd2b40a6280 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:07.373 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.372+0000 7fd2d6200640 1 --2- 172.21.15.83:0/1070112997 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd2d010e8f0 0x7fd2d019a730 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:07.373 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.372+0000 7fd2d6200640 1 --2- 172.21.15.83:0/1070112997 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd2d0074c20 0x7fd2d01a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:07.373 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.372+0000 7fd2d6200640 1 --2- 172.21.15.83:0/1070112997 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd2d0074300 0x7fd2d01a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:07.374 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.372+0000 7fd2d6200640 1 -- 172.21.15.83:0/1070112997 >> 172.21.15.83:0/1070112997 conn(0x7fd2d0079490 msgr2=0x7fd2d00feec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:07.374 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.372+0000 7fd2d6200640 1 -- 172.21.15.83:0/1070112997 shutdown_connections 2024-04-24T08:55:07.374 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:07.372+0000 7fd2d6200640 1 -- 172.21.15.83:0/1070112997 wait complete. 2024-04-24T08:55:07.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.403+0000 7f9eeee00640 1 -- 172.21.15.83:0/1280740292 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ee8102ac0 msgr2=0x7f9ee8106b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:07.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.403+0000 7f9eeee00640 1 --2- 172.21.15.83:0/1280740292 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ee8102ac0 0x7f9ee8106b00 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f9ed4009900 tx=0x7f9ed402a400 comp rx=0 tx=0).stop 2024-04-24T08:55:07.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.403+0000 7f9eeee00640 1 -- 172.21.15.83:0/1280740292 shutdown_connections 2024-04-24T08:55:07.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.403+0000 7f9eeee00640 1 --2- 172.21.15.83:0/1280740292 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9ee81071e0 0x7f9ee810edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:07.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.403+0000 7f9eeee00640 1 --2- 172.21.15.83:0/1280740292 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ee8102ac0 0x7f9ee8106b00 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:07.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.403+0000 7f9eeee00640 1 --2- 172.21.15.83:0/1280740292 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9ee8102110 0x7f9ee81024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:07.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.403+0000 7f9eeee00640 1 -- 172.21.15.83:0/1280740292 >> 172.21.15.83:0/1280740292 conn(0x7f9ee80fd9d0 msgr2=0x7f9ee80ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:07.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.403+0000 7f9eeee00640 1 -- 172.21.15.83:0/1280740292 shutdown_connections 2024-04-24T08:55:07.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.403+0000 7f9eeee00640 1 -- 172.21.15.83:0/1280740292 wait complete. 2024-04-24T08:55:07.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.404+0000 7f9eeee00640 1 Processor -- start 2024-04-24T08:55:07.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.404+0000 7f9eeee00640 1 -- start start 2024-04-24T08:55:07.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.404+0000 7f9eeee00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ee8102110 0x7f9ee81985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:07.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.404+0000 7f9eeee00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9ee81071e0 0x7f9ee8198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:07.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.404+0000 7f9eeee00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9ee819ce80 0x7f9ee819d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:07.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.405+0000 7f9eeee00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee8112100 con 0x7f9ee8102110 2024-04-24T08:55:07.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.405+0000 7f9eeee00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee8111f80 con 0x7f9ee81071e0 2024-04-24T08:55:07.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.405+0000 7f9eeee00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee8112280 con 0x7f9ee819ce80 2024-04-24T08:55:07.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.405+0000 7f9eeda00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ee8102110 0x7f9ee81985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:07.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.405+0000 7f9eeda00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ee8102110 0x7f9ee81985c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34970/0 (socket says 172.21.15.83:34970) 2024-04-24T08:55:07.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.405+0000 7f9eeda00640 1 -- 172.21.15.83:0/2460501855 learned_addr learned my addr 172.21.15.83:0/2460501855 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:55:07.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.405+0000 7f9eed000640 1 --2- 172.21.15.83:0/2460501855 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9ee81071e0 0x7f9ee8198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:07.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.405+0000 7f9eeda00640 1 -- 172.21.15.83:0/2460501855 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9ee819ce80 msgr2=0x7f9ee819d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:07.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.405+0000 7f9eeda00640 1 --2- 172.21.15.83:0/2460501855 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9ee819ce80 0x7f9ee819d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:07.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.405+0000 7f9eeda00640 1 -- 172.21.15.83:0/2460501855 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9ee81071e0 msgr2=0x7f9ee8198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:07.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.405+0000 7f9eeda00640 1 --2- 172.21.15.83:0/2460501855 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9ee81071e0 0x7f9ee8198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:07.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.405+0000 7f9eeda00640 1 -- 172.21.15.83:0/2460501855 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ee819d970 con 0x7f9ee8102110 2024-04-24T08:55:07.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.406+0000 7f9eeda00640 1 --2- 172.21.15.83:0/2460501855 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ee8102110 0x7f9ee81985c0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f9edc00c920 tx=0x7f9edc00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:07.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.406+0000 7f9ee6000640 1 -- 172.21.15.83:0/2460501855 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9edc007c70 con 0x7f9ee8102110 2024-04-24T08:55:07.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.406+0000 7f9ee6000640 1 -- 172.21.15.83:0/2460501855 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9edc007e10 con 0x7f9ee8102110 2024-04-24T08:55:07.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.406+0000 7f9eeee00640 1 -- 172.21.15.83:0/2460501855 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ee819dc60 con 0x7f9ee8102110 2024-04-24T08:55:07.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.406+0000 7f9ee6000640 1 -- 172.21.15.83:0/2460501855 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9edc018820 con 0x7f9ee8102110 2024-04-24T08:55:07.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.406+0000 7f9eeee00640 1 -- 172.21.15.83:0/2460501855 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ee81a5420 con 0x7f9ee8102110 2024-04-24T08:55:07.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.410+0000 7f9ef3226740 1 Processor -- start 2024-04-24T08:55:07.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.410+0000 7f9ee6000640 1 -- 172.21.15.83:0/2460501855 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9edc0189c0 con 0x7f9ee8102110 2024-04-24T08:55:07.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.410+0000 7f9ef3226740 1 -- start start 2024-04-24T08:55:07.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.410+0000 7f9ef3226740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5567fd0cba20 0x5567fd0cbe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:07.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.410+0000 7f9ef3226740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567fd0cc340 0x5567fd0daa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:07.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.410+0000 7f9ef3226740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5567fd0daf40 0x5567fd0dd330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:07.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.411+0000 7f9ef3226740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5567fd0311d0 con 0x5567fd0cc340 2024-04-24T08:55:07.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.411+0000 7f9ef3226740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5567fcfd7450 con 0x5567fd0daf40 2024-04-24T08:55:07.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.411+0000 7f9ef3226740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5567fd02d190 con 0x5567fd0cba20 2024-04-24T08:55:07.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.411+0000 7f9eeda00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567fd0cc340 0x5567fd0daa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:07.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.411+0000 7f9eeda00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567fd0cc340 0x5567fd0daa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34982/0 (socket says 172.21.15.83:34982) 2024-04-24T08:55:07.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.411+0000 7f9eeda00640 1 -- 172.21.15.83:0/2123373154 learned_addr learned my addr 172.21.15.83:0/2123373154 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:55:07.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.411+0000 7f9eed000640 1 --2- 172.21.15.83:0/2123373154 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5567fd0daf40 0x5567fd0dd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:07.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.411+0000 7f9eee400640 1 --2- 172.21.15.83:0/2123373154 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5567fd0cba20 0x5567fd0cbe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:07.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.412+0000 7f9eeda00640 1 -- 172.21.15.83:0/2123373154 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5567fd0cba20 msgr2=0x5567fd0cbe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:07.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.412+0000 7f9eeda00640 1 --2- 172.21.15.83:0/2123373154 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5567fd0cba20 0x5567fd0cbe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:07.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.412+0000 7f9eeda00640 1 -- 172.21.15.83:0/2123373154 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5567fd0daf40 msgr2=0x5567fd0dd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:07.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.412+0000 7f9eeda00640 1 --2- 172.21.15.83:0/2123373154 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5567fd0daf40 0x5567fd0dd330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:07.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.412+0000 7f9eeda00640 1 -- 172.21.15.83:0/2123373154 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567fd094310 con 0x5567fd0cc340 2024-04-24T08:55:07.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.412+0000 7f9eed000640 1 --2- 172.21.15.83:0/2123373154 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5567fd0daf40 0x5567fd0dd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:55:07.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.412+0000 7f9ee6000640 1 --2- 172.21.15.83:0/2460501855 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f9ed00a3d20 0x7f9ed00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:07.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.412+0000 7f9ee6000640 1 -- 172.21.15.83:0/2460501855 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f9edc0d0e20 con 0x7f9ee8102110 2024-04-24T08:55:07.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.412+0000 7f9eee400640 1 --2- 172.21.15.83:0/2460501855 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f9ed00a3d20 0x7f9ed00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:07.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.412+0000 7f9eeda00640 1 --2- 172.21.15.83:0/2123373154 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567fd0cc340 0x5567fd0daa00 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f9edc0ce080 tx=0x7f9edc0d90e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46445d284b6fb43c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:07.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.412+0000 7f9edaa00640 1 -- 172.21.15.83:0/2123373154 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9edc09c760 con 0x5567fd0cc340 2024-04-24T08:55:07.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.412+0000 7f9edaa00640 1 -- 172.21.15.83:0/2123373154 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9edc00fb00 con 0x5567fd0cc340 2024-04-24T08:55:07.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.412+0000 7f9edaa00640 1 -- 172.21.15.83:0/2123373154 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9edc018cb0 con 0x5567fd0cc340 2024-04-24T08:55:07.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.413+0000 7f9ef3226740 1 -- 172.21.15.83:0/2123373154 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567fd0cc340 msgr2=0x5567fd0daa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:07.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.413+0000 7f9ef3226740 1 --2- 172.21.15.83:0/2123373154 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567fd0cc340 0x5567fd0daa00 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f9edc0ce080 tx=0x7f9edc0d90e0 comp rx=0 tx=0).stop 2024-04-24T08:55:07.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.413+0000 7f9eee400640 1 --2- 172.21.15.83:0/2460501855 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f9ed00a3d20 0x7f9ed00a61e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f9ee0005cc0 tx=0x7f9ee000a330 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:07.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.413+0000 7f9ef3226740 1 -- 172.21.15.83:0/2123373154 shutdown_connections 2024-04-24T08:55:07.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.413+0000 7f9ef3226740 1 --2- 172.21.15.83:0/2123373154 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5567fd0daf40 0x5567fd0dd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:07.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.413+0000 7f9ef3226740 1 --2- 172.21.15.83:0/2123373154 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567fd0cc340 0x5567fd0daa00 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:07.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.413+0000 7f9ef3226740 1 --2- 172.21.15.83:0/2123373154 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5567fd0cba20 0x5567fd0cbe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:07.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.413+0000 7f9ef3226740 1 -- 172.21.15.83:0/2123373154 >> 172.21.15.83:0/2123373154 conn(0x5567fd0c7e60 msgr2=0x5567fd0c8240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:07.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.413+0000 7f9ef3226740 1 -- 172.21.15.83:0/2123373154 shutdown_connections 2024-04-24T08:55:07.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.414+0000 7f9ef3226740 1 -- 172.21.15.83:0/2123373154 wait complete. 2024-04-24T08:55:07.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.414+0000 7f9ef3226740 1 -- start start 2024-04-24T08:55:07.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.414+0000 7f9ef3226740 1 Processor -- start 2024-04-24T08:55:07.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.415+0000 7f9ef3226740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5567fd0cba20 0x5567fd0da970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:07.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.415+0000 7f9ef3226740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5567fd0cc340 0x5567fd0d7c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:07.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.415+0000 7f9ef3226740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567fd0daf40 0x5567fd0d83a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:07.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.415+0000 7f9ef3226740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5567fd0dde20 con 0x5567fd0daf40 2024-04-24T08:55:07.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.415+0000 7f9ef3226740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5567fcfea810 con 0x5567fd0cc340 2024-04-24T08:55:07.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.415+0000 7f9ef3226740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5567fd0ddfa0 con 0x5567fd0cba20 2024-04-24T08:55:07.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.415+0000 7f9ef3226740 10 client.0 _ll_register_callbacks cb 0x5567fcfb8140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:55:07.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.415+0000 7f9eee400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567fd0daf40 0x5567fd0d83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:07.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.415+0000 7f9eee400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567fd0daf40 0x5567fd0d83a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34988/0 (socket says 172.21.15.83:34988) 2024-04-24T08:55:07.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.415+0000 7f9eee400640 1 -- 172.21.15.83:0/3558902817 learned_addr learned my addr 172.21.15.83:0/3558902817 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:55:07.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.415+0000 7f9eeda00640 1 --2- 172.21.15.83:0/3558902817 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5567fd0cba20 0x5567fd0da970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:07.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.415+0000 7f9eed000640 1 --2- 172.21.15.83:0/3558902817 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5567fd0cc340 0x5567fd0d7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:07.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.415+0000 7f9eee400640 1 -- 172.21.15.83:0/3558902817 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5567fd0cba20 msgr2=0x5567fd0da970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:07.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.415+0000 7f9eee400640 1 --2- 172.21.15.83:0/3558902817 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5567fd0cba20 0x5567fd0da970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:07.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.415+0000 7f9eee400640 1 -- 172.21.15.83:0/3558902817 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5567fd0cc340 msgr2=0x5567fd0d7c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:07.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.415+0000 7f9eee400640 1 --2- 172.21.15.83:0/3558902817 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5567fd0cc340 0x5567fd0d7c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:07.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.415+0000 7f9eee400640 1 -- 172.21.15.83:0/3558902817 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567fcffb3f0 con 0x5567fd0daf40 2024-04-24T08:55:07.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.416+0000 7f9eee400640 1 --2- 172.21.15.83:0/3558902817 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567fd0daf40 0x5567fd0d83a0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f9ee0010e30 tx=0x7f9ee0005c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1c718e0a54ac587 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:07.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.416+0000 7f9ecf400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:55:07.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.416+0000 7f9ecf400640 1 -- 172.21.15.83:0/3558902817 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ee0021070 con 0x5567fd0daf40 2024-04-24T08:55:07.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.416+0000 7f9ecf400640 1 -- 172.21.15.83:0/3558902817 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f9ee001d450 con 0x5567fd0daf40 2024-04-24T08:55:07.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.416+0000 7f9ef3226740 1 -- 172.21.15.83:0/3558902817 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5567fd03ebf0 con 0x5567fd0daf40 2024-04-24T08:55:07.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.416+0000 7f9ecf400640 1 -- 172.21.15.83:0/3558902817 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9ee0010e60 con 0x5567fd0daf40 2024-04-24T08:55:07.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.416+0000 7f9ef3226740 10 client.16971 fetch_fsmap learned FSMap version 30 2024-04-24T08:55:07.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.416+0000 7f9ef3226740 1 -- 172.21.15.83:0/3558902817 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5567fcd433f0 con 0x5567fd0daf40 2024-04-24T08:55:07.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.417+0000 7f9ecf400640 1 -- 172.21.15.83:0/3558902817 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x5567fd03ebf0 con 0x5567fd0daf40 2024-04-24T08:55:07.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.417+0000 7f9ecf400640 1 -- 172.21.15.83:0/3558902817 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7f9ee0005020 con 0x5567fd0daf40 2024-04-24T08:55:07.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.417+0000 7f9ef3226740 10 client.16971 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T08:55:07.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.417+0000 7f9ef3226740 10 client.16971 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:55:07.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.417+0000 7f9ef3226740 20 client.16971 populate_metadata read hostname 'smithi083' 2024-04-24T08:55:07.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.417+0000 7f9ef3226740 1 --2- 172.21.15.83:0/3558902817 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5567fd0eb1c0 0x5567fd0ed5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:07.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.417+0000 7f9ef3226740 4 client.16971 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:55:07.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.417+0000 7f9ef3226740 1 -- 172.21.15.83:0/3558902817 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5567fd038a60 con 0x5567fd0eb1c0 2024-04-24T08:55:07.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.418+0000 7f9eeda00640 1 --2- 172.21.15.83:0/3558902817 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5567fd0eb1c0 0x5567fd0ed5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:07.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.418+0000 7f9eeda00640 1 --2- 172.21.15.83:0/3558902817 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5567fd0eb1c0 0x5567fd0ed5e0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cca94b13ca5e108e server_cookie=f58509798c51b799 in_seq=0 out_seq=0 2024-04-24T08:55:07.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.418+0000 7f9ecf400640 10 client.16971 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:55:07.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.421+0000 7f9ecf400640 1 -- 172.21.15.83:0/3558902817 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5567fd038a60 con 0x5567fd0eb1c0 2024-04-24T08:55:07.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.421+0000 7f9ecf400640 10 client.16971 handle_command_reply: tid=0 2024-04-24T08:55:07.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.421+0000 7f9ef3226740 1 client.16971 shutdown 2024-04-24T08:55:07.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.422+0000 7f9ef3226740 1 -- 172.21.15.83:0/3558902817 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567fd0daf40 msgr2=0x5567fd0d83a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:07.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.422+0000 7f9ef3226740 1 --2- 172.21.15.83:0/3558902817 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567fd0daf40 0x5567fd0d83a0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f9ee0010e30 tx=0x7f9ee0005c10 comp rx=0 tx=0).stop 2024-04-24T08:55:07.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.422+0000 7f9ef3226740 1 -- 172.21.15.83:0/3558902817 shutdown_connections 2024-04-24T08:55:07.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.422+0000 7f9ef3226740 1 --2- 172.21.15.83:0/3558902817 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5567fd0eb1c0 0x5567fd0ed5e0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:07.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.422+0000 7f9ef3226740 1 --2- 172.21.15.83:0/3558902817 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567fd0daf40 0x5567fd0d83a0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:07.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.422+0000 7f9ef3226740 1 --2- 172.21.15.83:0/3558902817 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5567fd0cc340 0x5567fd0d7c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:07.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.422+0000 7f9ef3226740 1 --2- 172.21.15.83:0/3558902817 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5567fd0cba20 0x5567fd0da970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:07.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.422+0000 7f9ef3226740 1 -- 172.21.15.83:0/3558902817 >> 172.21.15.83:0/3558902817 conn(0x5567fd0c7e60 msgr2=0x5567fd0cdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:07.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.422+0000 7f9ecf400640 0 client.16971 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:55:07.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.422+0000 7f9ef3226740 1 -- 172.21.15.83:0/3558902817 shutdown_connections 2024-04-24T08:55:07.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.422+0000 7f9ef3226740 1 -- 172.21.15.83:0/3558902817 wait complete. 2024-04-24T08:55:07.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.422+0000 7f9ef3226740 20 client.16971 trim_cache size 0 max 16384 2024-04-24T08:55:07.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.440+0000 7f9ef3226740 1 Processor -- start 2024-04-24T08:55:07.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.440+0000 7f9ef3226740 1 -- start start 2024-04-24T08:55:07.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.440+0000 7f9ef3226740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5567fd0cba20 0x5567fd0e6d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:07.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.440+0000 7f9ef3226740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567fd0cc340 0x5567fd0d6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:07.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.440+0000 7f9ef3226740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5567fd0daf40 0x5567fd0da4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:07.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.440+0000 7f9ef3226740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee81acfc0 con 0x5567fd0cc340 2024-04-24T08:55:07.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.440+0000 7f9ef3226740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee81aeb90 con 0x5567fd0daf40 2024-04-24T08:55:07.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.440+0000 7f9ef3226740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee81ad690 con 0x5567fd0cba20 2024-04-24T08:55:07.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.440+0000 7f9eed000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567fd0cc340 0x5567fd0d6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:07.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.441+0000 7f9eed000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567fd0cc340 0x5567fd0d6c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35002/0 (socket says 172.21.15.83:35002) 2024-04-24T08:55:07.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.441+0000 7f9eed000640 1 -- 172.21.15.83:0/1358997467 learned_addr learned my addr 172.21.15.83:0/1358997467 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:55:07.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.441+0000 7f9eeda00640 1 --2- 172.21.15.83:0/1358997467 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5567fd0cba20 0x5567fd0e6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:07.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.441+0000 7f9eee400640 1 --2- 172.21.15.83:0/1358997467 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5567fd0daf40 0x5567fd0da4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:07.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.441+0000 7f9eed000640 1 -- 172.21.15.83:0/1358997467 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5567fd0cba20 msgr2=0x5567fd0e6d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:07.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.441+0000 7f9eed000640 1 --2- 172.21.15.83:0/1358997467 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5567fd0cba20 0x5567fd0e6d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:07.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.441+0000 7f9eed000640 1 -- 172.21.15.83:0/1358997467 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5567fd0daf40 msgr2=0x5567fd0da4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:07.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.441+0000 7f9eed000640 1 --2- 172.21.15.83:0/1358997467 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5567fd0daf40 0x5567fd0da4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:07.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.441+0000 7f9eed000640 1 -- 172.21.15.83:0/1358997467 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567fce0d7d0 con 0x5567fd0cc340 2024-04-24T08:55:07.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.441+0000 7f9eed000640 1 --2- 172.21.15.83:0/1358997467 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567fd0cc340 0x5567fd0d6c40 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f9ed4033000 tx=0x7f9ed4031e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c4981fd8ac61382 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:07.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.442+0000 7f9eda000640 1 -- 172.21.15.83:0/1358997467 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed4035050 con 0x5567fd0cc340 2024-04-24T08:55:07.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.442+0000 7f9eda000640 1 -- 172.21.15.83:0/1358997467 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9ed40316a0 con 0x5567fd0cc340 2024-04-24T08:55:07.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.442+0000 7f9eda000640 1 -- 172.21.15.83:0/1358997467 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed40047a0 con 0x5567fd0cc340 2024-04-24T08:55:07.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.443+0000 7f9ef3226740 1 -- 172.21.15.83:0/1358997467 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567fd0cc340 msgr2=0x5567fd0d6c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:07.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.443+0000 7f9ef3226740 1 --2- 172.21.15.83:0/1358997467 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567fd0cc340 0x5567fd0d6c40 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f9ed4033000 tx=0x7f9ed4031e30 comp rx=0 tx=0).stop 2024-04-24T08:55:07.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.443+0000 7f9ef3226740 1 -- 172.21.15.83:0/1358997467 shutdown_connections 2024-04-24T08:55:07.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.443+0000 7f9ef3226740 1 --2- 172.21.15.83:0/1358997467 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5567fd0daf40 0x5567fd0da4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:07.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.443+0000 7f9ef3226740 1 --2- 172.21.15.83:0/1358997467 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567fd0cc340 0x5567fd0d6c40 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:07.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.443+0000 7f9ef3226740 1 --2- 172.21.15.83:0/1358997467 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5567fd0cba20 0x5567fd0e6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:07.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.443+0000 7f9ef3226740 1 -- 172.21.15.83:0/1358997467 >> 172.21.15.83:0/1358997467 conn(0x5567fd0c7e60 msgr2=0x5567fd0d6530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:07.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.443+0000 7f9ef3226740 1 -- 172.21.15.83:0/1358997467 shutdown_connections 2024-04-24T08:55:07.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.443+0000 7f9ef3226740 1 -- 172.21.15.83:0/1358997467 wait complete. 2024-04-24T08:55:07.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.443+0000 7f9ef3226740 1 -- start start 2024-04-24T08:55:07.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.444+0000 7f9ef3226740 1 Processor -- start 2024-04-24T08:55:07.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.444+0000 7f9ef3226740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5567fd0cba20 0x5567fd0e3510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:07.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.444+0000 7f9ef3226740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567fd0cc340 0x5567fd0db9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:07.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.444+0000 7f9ef3226740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5567fd0daf40 0x5567fd0dbee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:07.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.444+0000 7f9ef3226740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee81ae430 con 0x5567fd0cc340 2024-04-24T08:55:07.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.444+0000 7f9ef3226740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee81add60 con 0x5567fd0cba20 2024-04-24T08:55:07.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.444+0000 7f9ef3226740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee81aafa0 con 0x5567fd0daf40 2024-04-24T08:55:07.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.444+0000 7f9ef3226740 10 client.0 _ll_register_callbacks cb 0x7f9ee0014070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:55:07.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.444+0000 7f9eed000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567fd0cc340 0x5567fd0db9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:07.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.444+0000 7f9eed000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567fd0cc340 0x5567fd0db9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35012/0 (socket says 172.21.15.83:35012) 2024-04-24T08:55:07.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.444+0000 7f9eed000640 1 -- 172.21.15.83:0/3014118709 learned_addr learned my addr 172.21.15.83:0/3014118709 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:55:07.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.445+0000 7f9eeda00640 1 --2- 172.21.15.83:0/3014118709 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5567fd0cba20 0x5567fd0e3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:07.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.445+0000 7f9eed000640 1 -- 172.21.15.83:0/3014118709 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5567fd0daf40 msgr2=0x5567fd0dbee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:07.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.445+0000 7f9eed000640 1 --2- 172.21.15.83:0/3014118709 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5567fd0daf40 0x5567fd0dbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:07.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.445+0000 7f9eed000640 1 -- 172.21.15.83:0/3014118709 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5567fd0cba20 msgr2=0x5567fd0e3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:07.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.445+0000 7f9eed000640 1 --2- 172.21.15.83:0/3014118709 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5567fd0cba20 0x5567fd0e3510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:07.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.445+0000 7f9eed000640 1 -- 172.21.15.83:0/3014118709 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567fd0dc620 con 0x5567fd0cc340 2024-04-24T08:55:07.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.445+0000 7f9eed000640 1 --2- 172.21.15.83:0/3014118709 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567fd0cc340 0x5567fd0db9a0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f9ed4005440 tx=0x7f9ed40051a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59584c4f0293de14 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:07.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.445+0000 7f9ecf400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:55:07.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.445+0000 7f9ecf400640 1 -- 172.21.15.83:0/3014118709 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed40314a0 con 0x5567fd0cc340 2024-04-24T08:55:07.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.445+0000 7f9ecf400640 1 -- 172.21.15.83:0/3014118709 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f9ed4004240 con 0x5567fd0cc340 2024-04-24T08:55:07.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.445+0000 7f9ef3226740 1 -- 172.21.15.83:0/3014118709 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ed40316a0 con 0x5567fd0cc340 2024-04-24T08:55:07.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.446+0000 7f9ecf400640 1 -- 172.21.15.83:0/3014118709 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9ed4052050 con 0x5567fd0cc340 2024-04-24T08:55:07.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.446+0000 7f9ef3226740 10 client.16977 fetch_fsmap learned FSMap version 30 2024-04-24T08:55:07.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.446+0000 7f9ef3226740 1 -- 172.21.15.83:0/3014118709 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5567fd151740 con 0x5567fd0cc340 2024-04-24T08:55:07.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.446+0000 7f9ecf400640 1 -- 172.21.15.83:0/3014118709 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9ed40316a0 con 0x5567fd0cc340 2024-04-24T08:55:07.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.446+0000 7f9ecf400640 1 -- 172.21.15.83:0/3014118709 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7f9ed4003920 con 0x5567fd0cc340 2024-04-24T08:55:07.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.447+0000 7f9ef3226740 10 client.16977 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T08:55:07.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.447+0000 7f9ef3226740 10 client.16977 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:55:07.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.447+0000 7f9ef3226740 20 client.16977 populate_metadata read hostname 'smithi083' 2024-04-24T08:55:07.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.447+0000 7f9ef3226740 1 --2- 172.21.15.83:0/3014118709 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5567fd151ec0 0x5567fd1542e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:07.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.447+0000 7f9ef3226740 4 client.16977 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T08:55:07.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.447+0000 7f9ef3226740 1 -- 172.21.15.83:0/3014118709 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5567fcfb44d0 con 0x5567fd151ec0 2024-04-24T08:55:07.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.447+0000 7f9eeda00640 1 --2- 172.21.15.83:0/3014118709 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5567fd151ec0 0x5567fd1542e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:07.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.448+0000 7f9eeda00640 1 --2- 172.21.15.83:0/3014118709 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5567fd151ec0 0x5567fd1542e0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f31d447840833fd8 server_cookie=23737785558a431b in_seq=0 out_seq=0 2024-04-24T08:55:07.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.448+0000 7f9ecf400640 10 client.16977 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:55:07.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.448+0000 7f9ecf400640 1 -- 172.21.15.83:0/3014118709 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5567fcfb44d0 con 0x5567fd151ec0 2024-04-24T08:55:07.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.448+0000 7f9ecf400640 10 client.16977 handle_command_reply: tid=0 2024-04-24T08:55:07.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.448+0000 7f9ef3226740 1 client.16977 shutdown 2024-04-24T08:55:07.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.449+0000 7f9ef3226740 1 -- 172.21.15.83:0/3014118709 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567fd0cc340 msgr2=0x5567fd0db9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:07.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.449+0000 7f9ef3226740 1 --2- 172.21.15.83:0/3014118709 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567fd0cc340 0x5567fd0db9a0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f9ed4005440 tx=0x7f9ed40051a0 comp rx=0 tx=0).stop 2024-04-24T08:55:07.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.449+0000 7f9ef3226740 1 -- 172.21.15.83:0/3014118709 shutdown_connections 2024-04-24T08:55:07.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.449+0000 7f9ef3226740 1 --2- 172.21.15.83:0/3014118709 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5567fd151ec0 0x5567fd1542e0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:07.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.449+0000 7f9ef3226740 1 --2- 172.21.15.83:0/3014118709 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5567fd0daf40 0x5567fd0dbee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:07.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.449+0000 7f9ef3226740 1 --2- 172.21.15.83:0/3014118709 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567fd0cc340 0x5567fd0db9a0 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:07.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.449+0000 7f9ef3226740 1 --2- 172.21.15.83:0/3014118709 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5567fd0cba20 0x5567fd0e3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:07.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.449+0000 7f9ecf400640 0 client.16977 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:55:07.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.449+0000 7f9ef3226740 1 -- 172.21.15.83:0/3014118709 >> 172.21.15.83:0/3014118709 conn(0x5567fd0c7e60 msgr2=0x5567fd0c6f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:07.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.449+0000 7f9ef3226740 1 -- 172.21.15.83:0/3014118709 shutdown_connections 2024-04-24T08:55:07.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.449+0000 7f9ef3226740 1 -- 172.21.15.83:0/3014118709 wait complete. 2024-04-24T08:55:07.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.449+0000 7f9ef3226740 20 client.16977 trim_cache size 0 max 16384 2024-04-24T08:55:07.453 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T08:55:07.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.450+0000 7f9eeee00640 1 -- 172.21.15.83:0/2460501855 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f9ed00a3d20 msgr2=0x7f9ed00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:07.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.450+0000 7f9eeee00640 1 --2- 172.21.15.83:0/2460501855 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f9ed00a3d20 0x7f9ed00a61e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f9ee0005cc0 tx=0x7f9ee000a330 comp rx=0 tx=0).stop 2024-04-24T08:55:07.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.450+0000 7f9eeee00640 1 -- 172.21.15.83:0/2460501855 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ee8102110 msgr2=0x7f9ee81985c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:07.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.450+0000 7f9eeee00640 1 --2- 172.21.15.83:0/2460501855 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ee8102110 0x7f9ee81985c0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f9edc00c920 tx=0x7f9edc00cde0 comp rx=0 tx=0).stop 2024-04-24T08:55:07.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.450+0000 7f9eeee00640 1 -- 172.21.15.83:0/2460501855 shutdown_connections 2024-04-24T08:55:07.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.450+0000 7f9eeee00640 1 --2- 172.21.15.83:0/2460501855 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f9ed00a3d20 0x7f9ed00a61e0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:07.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.450+0000 7f9eeee00640 1 --2- 172.21.15.83:0/2460501855 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9ee819ce80 0x7f9ee819d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:07.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.450+0000 7f9eeee00640 1 --2- 172.21.15.83:0/2460501855 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9ee81071e0 0x7f9ee8198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:07.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.450+0000 7f9eeee00640 1 --2- 172.21.15.83:0/2460501855 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ee8102110 0x7f9ee81985c0 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:07.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.450+0000 7f9eeee00640 1 -- 172.21.15.83:0/2460501855 >> 172.21.15.83:0/2460501855 conn(0x7f9ee80fd9d0 msgr2=0x7f9ee80ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:07.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.450+0000 7f9eeee00640 1 -- 172.21.15.83:0/2460501855 shutdown_connections 2024-04-24T08:55:07.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:07.450+0000 7f9eeee00640 1 -- 172.21.15.83:0/2460501855 wait complete. 2024-04-24T08:55:07.565 INFO:tasks.workunit.client.0.smithi083.stdout: 1 12730 0.83 MB/sec warmup 101 sec latency 60.015 ms 2024-04-24T08:55:08.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:55:07 smithi138 ceph-mon[31441]: pgmap v108: 129 pgs: 129 active+clean; 28 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 285 KiB/s wr, 70 op/s 2024-04-24T08:55:08.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:55:07 smithi189 ceph-mon[31791]: pgmap v108: 129 pgs: 129 active+clean; 28 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 285 KiB/s wr, 70 op/s 2024-04-24T08:55:08.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:55:07 smithi083 ceph-mon[27907]: pgmap v108: 129 pgs: 129 active+clean; 28 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 285 KiB/s wr, 70 op/s 2024-04-24T08:55:08.470 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T08:55:08.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T08:55:08.470 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T08:55:08.565 INFO:tasks.workunit.client.0.smithi083.stdout: 1 12870 0.82 MB/sec warmup 102 sec latency 52.356 ms 2024-04-24T08:55:08.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.712+0000 7f8be3e00640 1 -- 172.21.15.83:0/1176739480 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8be410d500 msgr2=0x7f8be410d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:08.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.712+0000 7f8be3e00640 1 --2- 172.21.15.83:0/1176739480 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8be410d500 0x7f8be410d8e0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f8bd4009930 tx=0x7f8bd402a420 comp rx=0 tx=0).stop 2024-04-24T08:55:08.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.713+0000 7f8be3e00640 1 -- 172.21.15.83:0/1176739480 shutdown_connections 2024-04-24T08:55:08.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.713+0000 7f8be3e00640 1 --2- 172.21.15.83:0/1176739480 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8be4108f40 0x7f8be41153b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:08.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.713+0000 7f8be3e00640 1 --2- 172.21.15.83:0/1176739480 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8be4108580 0x7f8be4108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:08.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.713+0000 7f8be3e00640 1 --2- 172.21.15.83:0/1176739480 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8be410d500 0x7f8be410d8e0 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:08.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.713+0000 7f8be3e00640 1 -- 172.21.15.83:0/1176739480 >> 172.21.15.83:0/1176739480 conn(0x7f8be4077e20 msgr2=0x7f8be4106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:08.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.713+0000 7f8be3e00640 1 -- 172.21.15.83:0/1176739480 shutdown_connections 2024-04-24T08:55:08.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.713+0000 7f8be3e00640 1 -- 172.21.15.83:0/1176739480 wait complete. 2024-04-24T08:55:08.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.714+0000 7f8be3e00640 1 Processor -- start 2024-04-24T08:55:08.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.714+0000 7f8be3e00640 1 -- start start 2024-04-24T08:55:08.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.714+0000 7f8be3e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8be4108580 0x7f8be41a48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:08.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.714+0000 7f8be3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8be4108f40 0x7f8be41a4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:08.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.714+0000 7f8be3e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8be410d500 0x7f8be419eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:08.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.714+0000 7f8be3e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8be4118760 con 0x7f8be4108f40 2024-04-24T08:55:08.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.714+0000 7f8be3e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8be41185e0 con 0x7f8be4108580 2024-04-24T08:55:08.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.714+0000 7f8be3e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8be41188e0 con 0x7f8be410d500 2024-04-24T08:55:08.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.714+0000 7f8be2000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8be4108f40 0x7f8be41a4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:08.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.715+0000 7f8be2000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8be4108f40 0x7f8be41a4e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35028/0 (socket says 172.21.15.83:35028) 2024-04-24T08:55:08.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.715+0000 7f8be2000640 1 -- 172.21.15.83:0/744191882 learned_addr learned my addr 172.21.15.83:0/744191882 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:55:08.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.715+0000 7f8be2a00640 1 --2- 172.21.15.83:0/744191882 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8be4108580 0x7f8be41a48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:08.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.715+0000 7f8be3400640 1 --2- 172.21.15.83:0/744191882 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8be410d500 0x7f8be419eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:08.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.715+0000 7f8be2000640 1 -- 172.21.15.83:0/744191882 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8be410d500 msgr2=0x7f8be419eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:08.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.715+0000 7f8be2000640 1 --2- 172.21.15.83:0/744191882 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8be410d500 0x7f8be419eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:08.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.715+0000 7f8be2000640 1 -- 172.21.15.83:0/744191882 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8be4108580 msgr2=0x7f8be41a48c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:08.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.715+0000 7f8be2000640 1 --2- 172.21.15.83:0/744191882 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8be4108580 0x7f8be41a48c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:08.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.715+0000 7f8be2000640 1 -- 172.21.15.83:0/744191882 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8be419f240 con 0x7f8be4108f40 2024-04-24T08:55:08.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.715+0000 7f8be2000640 1 --2- 172.21.15.83:0/744191882 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8be4108f40 0x7f8be41a4e00 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f8bd000d950 tx=0x7f8bd000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:08.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.716+0000 7f8bdb400640 1 -- 172.21.15.83:0/744191882 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd0017070 con 0x7f8be4108f40 2024-04-24T08:55:08.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.716+0000 7f8bdb400640 1 -- 172.21.15.83:0/744191882 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8bd0009c30 con 0x7f8be4108f40 2024-04-24T08:55:08.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.716+0000 7f8be3e00640 1 -- 172.21.15.83:0/744191882 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8be419f530 con 0x7f8be4108f40 2024-04-24T08:55:08.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.716+0000 7f8bdb400640 1 -- 172.21.15.83:0/744191882 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd000f620 con 0x7f8be4108f40 2024-04-24T08:55:08.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.716+0000 7f8be3e00640 1 -- 172.21.15.83:0/744191882 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8be41ab780 con 0x7f8be4108f40 2024-04-24T08:55:08.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.720+0000 7f8bec403740 1 Processor -- start 2024-04-24T08:55:08.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.720+0000 7f8bec403740 1 -- start start 2024-04-24T08:55:08.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.720+0000 7f8bec403740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564235366270 0x564235366650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:08.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.720+0000 7f8bec403740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564235366b90 0x564235370d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:08.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.720+0000 7f8bec403740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564235371260 0x564235373650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:08.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.720+0000 7f8bec403740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5642352cbe70 con 0x564235366b90 2024-04-24T08:55:08.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.720+0000 7f8bec403740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x564235272380 con 0x564235366270 2024-04-24T08:55:08.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.720+0000 7f8bec403740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5642352c7e30 con 0x564235371260 2024-04-24T08:55:08.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.720+0000 7f8be2a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564235366b90 0x564235370d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:08.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.720+0000 7f8be2a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564235366b90 0x564235370d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35036/0 (socket says 172.21.15.83:35036) 2024-04-24T08:55:08.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.721+0000 7f8be2a00640 1 -- 172.21.15.83:0/1644222698 learned_addr learned my addr 172.21.15.83:0/1644222698 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:55:08.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.721+0000 7f8be2a00640 1 -- 172.21.15.83:0/1644222698 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564235371260 msgr2=0x564235373650 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T08:55:08.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.721+0000 7f8be2a00640 1 --2- 172.21.15.83:0/1644222698 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564235371260 0x564235373650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:08.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.721+0000 7f8be2a00640 1 -- 172.21.15.83:0/1644222698 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564235366270 msgr2=0x564235366650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:08.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.721+0000 7f8be2a00640 1 --2- 172.21.15.83:0/1644222698 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564235366270 0x564235366650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:08.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.721+0000 7f8be2a00640 1 -- 172.21.15.83:0/1644222698 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642350a67e0 con 0x564235366b90 2024-04-24T08:55:08.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.721+0000 7f8bdb400640 1 -- 172.21.15.83:0/744191882 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8bd00109a0 con 0x7f8be4108f40 2024-04-24T08:55:08.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.721+0000 7f8be2a00640 1 --2- 172.21.15.83:0/1644222698 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564235366b90 0x564235370d20 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f8bd4031440 tx=0x7f8bd4032c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca527e9632a2b961 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:08.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.722+0000 7f8bcf400640 1 -- 172.21.15.83:0/1644222698 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd403f070 con 0x564235366b90 2024-04-24T08:55:08.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.722+0000 7f8bcf400640 1 -- 172.21.15.83:0/1644222698 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8bd40044d0 con 0x564235366b90 2024-04-24T08:55:08.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.722+0000 7f8bcf400640 1 -- 172.21.15.83:0/1644222698 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd4035070 con 0x564235366b90 2024-04-24T08:55:08.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.722+0000 7f8bec403740 1 -- 172.21.15.83:0/1644222698 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564235366b90 msgr2=0x564235370d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:08.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.723+0000 7f8bec403740 1 --2- 172.21.15.83:0/1644222698 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564235366b90 0x564235370d20 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f8bd4031440 tx=0x7f8bd4032c30 comp rx=0 tx=0).stop 2024-04-24T08:55:08.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.723+0000 7f8bec403740 1 -- 172.21.15.83:0/1644222698 shutdown_connections 2024-04-24T08:55:08.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.723+0000 7f8bec403740 1 --2- 172.21.15.83:0/1644222698 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564235371260 0x564235373650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:08.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.723+0000 7f8bec403740 1 --2- 172.21.15.83:0/1644222698 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564235366b90 0x564235370d20 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:08.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.723+0000 7f8bec403740 1 --2- 172.21.15.83:0/1644222698 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564235366270 0x564235366650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:08.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.723+0000 7f8bec403740 1 -- 172.21.15.83:0/1644222698 >> 172.21.15.83:0/1644222698 conn(0x564235362590 msgr2=0x564235362970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:08.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.723+0000 7f8bec403740 1 -- 172.21.15.83:0/1644222698 shutdown_connections 2024-04-24T08:55:08.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.723+0000 7f8bec403740 1 -- 172.21.15.83:0/1644222698 wait complete. 2024-04-24T08:55:08.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.723+0000 7f8bdb400640 1 --2- 172.21.15.83:0/744191882 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f8bc80a3cb0 0x7f8bc80a6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:08.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.723+0000 7f8bec403740 1 -- start start 2024-04-24T08:55:08.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.723+0000 7f8bdb400640 1 -- 172.21.15.83:0/744191882 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f8bd00d0830 con 0x7f8be4108f40 2024-04-24T08:55:08.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.724+0000 7f8be3400640 1 --2- 172.21.15.83:0/744191882 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f8bc80a3cb0 0x7f8bc80a6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:08.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.724+0000 7f8bec403740 1 Processor -- start 2024-04-24T08:55:08.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.724+0000 7f8bec403740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564235366270 0x56423536e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:08.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.724+0000 7f8bec403740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564235366b90 0x56423536e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:08.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.724+0000 7f8bec403740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564235371260 0x56423536eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:08.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.724+0000 7f8bec403740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x564235373ff0 con 0x564235371260 2024-04-24T08:55:08.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.724+0000 7f8be3400640 1 --2- 172.21.15.83:0/744191882 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f8bc80a3cb0 0x7f8bc80a6170 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc005fd0 tx=0x7f8bdc009400 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:08.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.724+0000 7f8bec403740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x564235285660 con 0x564235366270 2024-04-24T08:55:08.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.724+0000 7f8bec403740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x564235374170 con 0x564235366b90 2024-04-24T08:55:08.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.724+0000 7f8bec403740 10 client.0 _ll_register_callbacks cb 0x564235273aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:55:08.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.725+0000 7f8be3400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564235371260 0x56423536eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:08.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.725+0000 7f8be3400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564235371260 0x56423536eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35048/0 (socket says 172.21.15.83:35048) 2024-04-24T08:55:08.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.725+0000 7f8be3400640 1 -- 172.21.15.83:0/4232853863 learned_addr learned my addr 172.21.15.83:0/4232853863 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:55:08.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.725+0000 7f8be2a00640 1 --2- 172.21.15.83:0/4232853863 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564235366270 0x56423536e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:08.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.725+0000 7f8be2000640 1 --2- 172.21.15.83:0/4232853863 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564235366b90 0x56423536e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:08.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.725+0000 7f8be3400640 1 -- 172.21.15.83:0/4232853863 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564235366b90 msgr2=0x56423536e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:08.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.725+0000 7f8be3400640 1 --2- 172.21.15.83:0/4232853863 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564235366b90 0x56423536e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:08.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.725+0000 7f8be3400640 1 -- 172.21.15.83:0/4232853863 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564235366270 msgr2=0x56423536e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:08.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.725+0000 7f8be3400640 1 --2- 172.21.15.83:0/4232853863 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564235366270 0x56423536e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:08.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.725+0000 7f8be3400640 1 -- 172.21.15.83:0/4232853863 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564235106f70 con 0x564235371260 2024-04-24T08:55:08.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.726+0000 7f8be3400640 1 --2- 172.21.15.83:0/4232853863 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564235371260 0x56423536eca0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc011720 tx=0x7f8bdc004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c1224c6a6123e09 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:08.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.726+0000 7f8bc3e00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:55:08.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.726+0000 7f8bc3e00640 1 -- 172.21.15.83:0/4232853863 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bdc022070 con 0x564235371260 2024-04-24T08:55:08.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.726+0000 7f8bc3e00640 1 -- 172.21.15.83:0/4232853863 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f8bdc010920 con 0x564235371260 2024-04-24T08:55:08.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.726+0000 7f8bec403740 1 -- 172.21.15.83:0/4232853863 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8bd40044d0 con 0x564235371260 2024-04-24T08:55:08.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.726+0000 7f8bc3e00640 1 -- 172.21.15.83:0/4232853863 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8bdc02e050 con 0x564235371260 2024-04-24T08:55:08.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.726+0000 7f8bec403740 10 client.16989 fetch_fsmap learned FSMap version 30 2024-04-24T08:55:08.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.726+0000 7f8bec403740 1 -- 172.21.15.83:0/4232853863 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564234f9e170 con 0x564235371260 2024-04-24T08:55:08.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.727+0000 7f8bc3e00640 1 -- 172.21.15.83:0/4232853863 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8bd40044d0 con 0x564235371260 2024-04-24T08:55:08.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.727+0000 7f8bc3e00640 1 -- 172.21.15.83:0/4232853863 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7f8bdc01ecd0 con 0x564235371260 2024-04-24T08:55:08.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.727+0000 7f8bec403740 10 client.16989 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T08:55:08.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.727+0000 7f8bec403740 10 client.16989 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:55:08.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.727+0000 7f8bec403740 20 client.16989 populate_metadata read hostname 'smithi083' 2024-04-24T08:55:08.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.727+0000 7f8bec403740 1 --2- 172.21.15.83:0/4232853863 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x564235381670 0x564235383ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:08.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.727+0000 7f8bec403740 4 client.16989 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:55:08.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.727+0000 7f8bec403740 1 -- 172.21.15.83:0/4232853863 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5642352d9890 con 0x564235381670 2024-04-24T08:55:08.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.728+0000 7f8be2a00640 1 --2- 172.21.15.83:0/4232853863 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x564235381670 0x564235383ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:08.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.729+0000 7f8bc3e00640 10 client.16989 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:55:08.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.729+0000 7f8be2a00640 1 --2- 172.21.15.83:0/4232853863 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x564235381670 0x564235383ab0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26aaa77e010a8f62 server_cookie=6cc1d12b9e1170e0 in_seq=0 out_seq=0 2024-04-24T08:55:08.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.731+0000 7f8bc3e00640 1 -- 172.21.15.83:0/4232853863 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5642352d9890 con 0x564235381670 2024-04-24T08:55:08.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.731+0000 7f8bc3e00640 10 client.16989 handle_command_reply: tid=0 2024-04-24T08:55:08.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.731+0000 7f8bec403740 1 client.16989 shutdown 2024-04-24T08:55:08.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.732+0000 7f8bec403740 1 -- 172.21.15.83:0/4232853863 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564235371260 msgr2=0x56423536eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:08.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.732+0000 7f8bec403740 1 --2- 172.21.15.83:0/4232853863 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564235371260 0x56423536eca0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc011720 tx=0x7f8bdc004ab0 comp rx=0 tx=0).stop 2024-04-24T08:55:08.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.732+0000 7f8bec403740 1 -- 172.21.15.83:0/4232853863 shutdown_connections 2024-04-24T08:55:08.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.732+0000 7f8bec403740 1 --2- 172.21.15.83:0/4232853863 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x564235381670 0x564235383ab0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:08.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.732+0000 7f8bec403740 1 --2- 172.21.15.83:0/4232853863 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564235371260 0x56423536eca0 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:08.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.732+0000 7f8bec403740 1 --2- 172.21.15.83:0/4232853863 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564235366b90 0x56423536e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:08.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.732+0000 7f8bec403740 1 --2- 172.21.15.83:0/4232853863 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564235366270 0x56423536e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:08.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.732+0000 7f8bec403740 1 -- 172.21.15.83:0/4232853863 >> 172.21.15.83:0/4232853863 conn(0x564235362590 msgr2=0x564235368290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:08.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.732+0000 7f8bc3e00640 0 client.16989 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:55:08.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.732+0000 7f8bec403740 1 -- 172.21.15.83:0/4232853863 shutdown_connections 2024-04-24T08:55:08.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.732+0000 7f8bec403740 1 -- 172.21.15.83:0/4232853863 wait complete. 2024-04-24T08:55:08.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.732+0000 7f8bec403740 20 client.16989 trim_cache size 0 max 16384 2024-04-24T08:55:08.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.753+0000 7f8bec403740 1 Processor -- start 2024-04-24T08:55:08.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.753+0000 7f8bec403740 1 -- start start 2024-04-24T08:55:08.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.754+0000 7f8bec403740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564235366270 0x564235361d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:08.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.754+0000 7f8bec403740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564235366b90 0x56423537cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:08.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.754+0000 7f8bec403740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564235371260 0x56423536eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:08.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.754+0000 7f8bec403740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8be41b5840 con 0x564235371260 2024-04-24T08:55:08.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.754+0000 7f8bec403740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8be41b7410 con 0x564235366270 2024-04-24T08:55:08.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.754+0000 7f8bec403740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8be41b5f10 con 0x564235366b90 2024-04-24T08:55:08.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.754+0000 7f8be3400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564235371260 0x56423536eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:08.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.754+0000 7f8be3400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564235371260 0x56423536eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35050/0 (socket says 172.21.15.83:35050) 2024-04-24T08:55:08.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.754+0000 7f8be3400640 1 -- 172.21.15.83:0/3580647200 learned_addr learned my addr 172.21.15.83:0/3580647200 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:55:08.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.754+0000 7f8be2a00640 1 --2- 172.21.15.83:0/3580647200 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564235366270 0x564235361d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:08.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.754+0000 7f8be2000640 1 --2- 172.21.15.83:0/3580647200 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564235366b90 0x56423537cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:08.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.754+0000 7f8be3400640 1 -- 172.21.15.83:0/3580647200 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564235366b90 msgr2=0x56423537cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:08.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.754+0000 7f8be3400640 1 --2- 172.21.15.83:0/3580647200 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564235366b90 0x56423537cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:08.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.755+0000 7f8be3400640 1 -- 172.21.15.83:0/3580647200 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564235366270 msgr2=0x564235361d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:08.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.755+0000 7f8be3400640 1 --2- 172.21.15.83:0/3580647200 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564235366270 0x564235361d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:08.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.755+0000 7f8be3400640 1 -- 172.21.15.83:0/3580647200 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56423535def0 con 0x564235371260 2024-04-24T08:55:08.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.755+0000 7f8be2a00640 1 --2- 172.21.15.83:0/3580647200 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564235366270 0x564235361d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:55:08.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.755+0000 7f8be2000640 1 --2- 172.21.15.83:0/3580647200 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564235366b90 0x56423537cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:55:08.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.755+0000 7f8be3400640 1 --2- 172.21.15.83:0/3580647200 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564235371260 0x56423536eae0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc00fbc0 tx=0x7f8bdc0096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2363f161df695248 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:08.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.755+0000 7f8bcea00640 1 -- 172.21.15.83:0/3580647200 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bdc022070 con 0x564235371260 2024-04-24T08:55:08.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.755+0000 7f8bcea00640 1 -- 172.21.15.83:0/3580647200 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8bdc037070 con 0x564235371260 2024-04-24T08:55:08.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.755+0000 7f8bcea00640 1 -- 172.21.15.83:0/3580647200 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bdc010b50 con 0x564235371260 2024-04-24T08:55:08.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.756+0000 7f8bec403740 1 -- 172.21.15.83:0/3580647200 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564235371260 msgr2=0x56423536eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:08.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.756+0000 7f8bec403740 1 --2- 172.21.15.83:0/3580647200 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564235371260 0x56423536eae0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc00fbc0 tx=0x7f8bdc0096c0 comp rx=0 tx=0).stop 2024-04-24T08:55:08.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.756+0000 7f8bec403740 1 -- 172.21.15.83:0/3580647200 shutdown_connections 2024-04-24T08:55:08.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.756+0000 7f8bec403740 1 --2- 172.21.15.83:0/3580647200 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564235371260 0x56423536eae0 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:08.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.756+0000 7f8bec403740 1 --2- 172.21.15.83:0/3580647200 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564235366b90 0x56423537cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:08.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.756+0000 7f8bec403740 1 --2- 172.21.15.83:0/3580647200 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564235366270 0x564235361d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:08.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.756+0000 7f8bec403740 1 -- 172.21.15.83:0/3580647200 >> 172.21.15.83:0/3580647200 conn(0x564235362590 msgr2=0x56423536fa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:08.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.756+0000 7f8bec403740 1 -- 172.21.15.83:0/3580647200 shutdown_connections 2024-04-24T08:55:08.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.756+0000 7f8bec403740 1 -- 172.21.15.83:0/3580647200 wait complete. 2024-04-24T08:55:08.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.757+0000 7f8bec403740 1 -- start start 2024-04-24T08:55:08.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.757+0000 7f8bec403740 1 Processor -- start 2024-04-24T08:55:08.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.757+0000 7f8bec403740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564235366270 0x56423536b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:08.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.757+0000 7f8bec403740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564235366b90 0x56423536b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:08.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.757+0000 7f8bec403740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564235371260 0x56423536bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:08.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.757+0000 7f8bec403740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8be41b6cb0 con 0x564235366b90 2024-04-24T08:55:08.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.757+0000 7f8bec403740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8be41b65e0 con 0x564235366270 2024-04-24T08:55:08.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.757+0000 7f8bec403740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8be41b3820 con 0x564235371260 2024-04-24T08:55:08.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.757+0000 7f8bec403740 10 client.0 _ll_register_callbacks cb 0x7f8bdc015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:55:08.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.758+0000 7f8be2000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564235366b90 0x56423536b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:08.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.758+0000 7f8be2000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564235366b90 0x56423536b5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35064/0 (socket says 172.21.15.83:35064) 2024-04-24T08:55:08.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.758+0000 7f8be2000640 1 -- 172.21.15.83:0/2481373797 learned_addr learned my addr 172.21.15.83:0/2481373797 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:55:08.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.758+0000 7f8be2a00640 1 --2- 172.21.15.83:0/2481373797 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564235366270 0x56423536b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:08.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.758+0000 7f8be3400640 1 --2- 172.21.15.83:0/2481373797 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564235371260 0x56423536bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:08.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.758+0000 7f8be2000640 1 -- 172.21.15.83:0/2481373797 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564235371260 msgr2=0x56423536bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:08.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.758+0000 7f8be2000640 1 --2- 172.21.15.83:0/2481373797 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564235371260 0x56423536bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:08.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.758+0000 7f8be2000640 1 -- 172.21.15.83:0/2481373797 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564235366270 msgr2=0x56423536b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:08.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.758+0000 7f8be2000640 1 --2- 172.21.15.83:0/2481373797 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564235366270 0x56423536b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:08.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.758+0000 7f8be2000640 1 -- 172.21.15.83:0/2481373797 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642353e6d90 con 0x564235366b90 2024-04-24T08:55:08.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.758+0000 7f8be2000640 1 --2- 172.21.15.83:0/2481373797 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564235366b90 0x56423536b5a0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f8bd0000c00 tx=0x7f8bd0009ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=200a2840758906a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:08.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.758+0000 7f8bc3e00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:55:08.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.759+0000 7f8bc3e00640 1 -- 172.21.15.83:0/2481373797 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd0020050 con 0x564235366b90 2024-04-24T08:55:08.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.759+0000 7f8bc3e00640 1 -- 172.21.15.83:0/2481373797 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f8bd00d0410 con 0x564235366b90 2024-04-24T08:55:08.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.759+0000 7f8bec403740 1 -- 172.21.15.83:0/2481373797 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8bdc037070 con 0x564235366b90 2024-04-24T08:55:08.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.759+0000 7f8bc3e00640 1 -- 172.21.15.83:0/2481373797 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8bd00a07b0 con 0x564235366b90 2024-04-24T08:55:08.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.759+0000 7f8bec403740 10 client.16995 fetch_fsmap learned FSMap version 30 2024-04-24T08:55:08.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.759+0000 7f8bec403740 1 -- 172.21.15.83:0/2481373797 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5642353e8020 con 0x564235366b90 2024-04-24T08:55:08.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.759+0000 7f8bc3e00640 1 -- 172.21.15.83:0/2481373797 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8bdc037070 con 0x564235366b90 2024-04-24T08:55:08.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.759+0000 7f8bc3e00640 1 -- 172.21.15.83:0/2481373797 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7f8bd0010460 con 0x564235366b90 2024-04-24T08:55:08.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.760+0000 7f8bec403740 10 client.16995 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T08:55:08.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.760+0000 7f8bec403740 10 client.16995 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:55:08.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.760+0000 7f8bec403740 20 client.16995 populate_metadata read hostname 'smithi083' 2024-04-24T08:55:08.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.760+0000 7f8bec403740 1 --2- 172.21.15.83:0/2481373797 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5642353e8880 0x5642353eaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:08.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.760+0000 7f8bec403740 4 client.16995 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T08:55:08.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.760+0000 7f8bec403740 1 -- 172.21.15.83:0/2481373797 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5642352d3700 con 0x5642353e8880 2024-04-24T08:55:08.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.760+0000 7f8be2a00640 1 --2- 172.21.15.83:0/2481373797 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5642353e8880 0x5642353eaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:08.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.761+0000 7f8be2a00640 1 --2- 172.21.15.83:0/2481373797 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5642353e8880 0x5642353eaca0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2831c87797f9c592 server_cookie=42e4a4d4d671ae23 in_seq=0 out_seq=0 2024-04-24T08:55:08.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.761+0000 7f8bc3e00640 10 client.16995 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:55:08.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.766+0000 7f8bc3e00640 1 -- 172.21.15.83:0/2481373797 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5642352d3700 con 0x5642353e8880 2024-04-24T08:55:08.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.766+0000 7f8bc3e00640 10 client.16995 handle_command_reply: tid=0 2024-04-24T08:55:08.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.766+0000 7f8bec403740 1 client.16995 shutdown 2024-04-24T08:55:08.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.766+0000 7f8bec403740 1 -- 172.21.15.83:0/2481373797 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564235366b90 msgr2=0x56423536b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:08.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.766+0000 7f8bec403740 1 --2- 172.21.15.83:0/2481373797 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564235366b90 0x56423536b5a0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f8bd0000c00 tx=0x7f8bd0009ea0 comp rx=0 tx=0).stop 2024-04-24T08:55:08.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.766+0000 7f8bec403740 1 -- 172.21.15.83:0/2481373797 shutdown_connections 2024-04-24T08:55:08.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.766+0000 7f8bec403740 1 --2- 172.21.15.83:0/2481373797 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5642353e8880 0x5642353eaca0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:08.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.766+0000 7f8bec403740 1 --2- 172.21.15.83:0/2481373797 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564235371260 0x56423536bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:08.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.766+0000 7f8bec403740 1 --2- 172.21.15.83:0/2481373797 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564235366b90 0x56423536b5a0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:08.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.766+0000 7f8bec403740 1 --2- 172.21.15.83:0/2481373797 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564235366270 0x56423536b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:08.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.766+0000 7f8bec403740 1 -- 172.21.15.83:0/2481373797 >> 172.21.15.83:0/2481373797 conn(0x564235362590 msgr2=0x5642353702e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:08.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.766+0000 7f8bc3e00640 0 client.16995 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:55:08.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.766+0000 7f8bec403740 1 -- 172.21.15.83:0/2481373797 shutdown_connections 2024-04-24T08:55:08.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.766+0000 7f8bec403740 1 -- 172.21.15.83:0/2481373797 wait complete. 2024-04-24T08:55:08.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.766+0000 7f8bec403740 20 client.16995 trim_cache size 0 max 16384 2024-04-24T08:55:08.768 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T08:55:08.768 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-04-24T08:55:08.768 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "d75a3574-ba7e-4796-b98a-22b8e0f4b5f4", 2024-04-24T08:55:08.768 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-04-24T08:55:08.768 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T08:55:08.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.767+0000 7f8be3e00640 1 -- 172.21.15.83:0/744191882 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f8bc80a3cb0 msgr2=0x7f8bc80a6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:08.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.767+0000 7f8be3e00640 1 --2- 172.21.15.83:0/744191882 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f8bc80a3cb0 0x7f8bc80a6170 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc005fd0 tx=0x7f8bdc009400 comp rx=0 tx=0).stop 2024-04-24T08:55:08.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.767+0000 7f8be3e00640 1 -- 172.21.15.83:0/744191882 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8be4108f40 msgr2=0x7f8be41a4e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:08.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.767+0000 7f8be3e00640 1 --2- 172.21.15.83:0/744191882 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8be4108f40 0x7f8be41a4e00 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f8bd000d950 tx=0x7f8bd000de10 comp rx=0 tx=0).stop 2024-04-24T08:55:08.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.767+0000 7f8be3e00640 1 -- 172.21.15.83:0/744191882 shutdown_connections 2024-04-24T08:55:08.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.767+0000 7f8be3e00640 1 --2- 172.21.15.83:0/744191882 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f8bc80a3cb0 0x7f8bc80a6170 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:08.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.767+0000 7f8be3e00640 1 --2- 172.21.15.83:0/744191882 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8be410d500 0x7f8be419eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:08.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.767+0000 7f8be3e00640 1 --2- 172.21.15.83:0/744191882 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8be4108f40 0x7f8be41a4e00 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:08.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.767+0000 7f8be3e00640 1 --2- 172.21.15.83:0/744191882 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8be4108580 0x7f8be41a48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:08.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.767+0000 7f8be3e00640 1 -- 172.21.15.83:0/744191882 >> 172.21.15.83:0/744191882 conn(0x7f8be4077e20 msgr2=0x7f8be40783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:08.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.767+0000 7f8be3e00640 1 -- 172.21.15.83:0/744191882 shutdown_connections 2024-04-24T08:55:08.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:08.767+0000 7f8be3e00640 1 -- 172.21.15.83:0/744191882 wait complete. 2024-04-24T08:55:08.788 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T08:55:08.788 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T08:55:09.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.000+0000 7f4563200640 1 -- 172.21.15.83:0/213630654 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45640743b0 msgr2=0x7f4564074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:09.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.000+0000 7f4563200640 1 --2- 172.21.15.83:0/213630654 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45640743b0 0x7f4564074790 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f4554009900 tx=0x7f455402a480 comp rx=0 tx=0).stop 2024-04-24T08:55:09.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.001+0000 7f4563200640 1 -- 172.21.15.83:0/213630654 shutdown_connections 2024-04-24T08:55:09.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.001+0000 7f4563200640 1 --2- 172.21.15.83:0/213630654 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f456410e760 0x7f4564110b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:09.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.001+0000 7f4563200640 1 --2- 172.21.15.83:0/213630654 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4564074cd0 0x7f456410e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:09.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.001+0000 7f4563200640 1 --2- 172.21.15.83:0/213630654 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45640743b0 0x7f4564074790 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:09.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.001+0000 7f4563200640 1 -- 172.21.15.83:0/213630654 >> 172.21.15.83:0/213630654 conn(0x7f45640fd9d0 msgr2=0x7f45640ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:09.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.001+0000 7f4563200640 1 -- 172.21.15.83:0/213630654 shutdown_connections 2024-04-24T08:55:09.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.001+0000 7f4563200640 1 -- 172.21.15.83:0/213630654 wait complete. 2024-04-24T08:55:09.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.002+0000 7f4563200640 1 Processor -- start 2024-04-24T08:55:09.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.002+0000 7f4563200640 1 -- start start 2024-04-24T08:55:09.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.002+0000 7f4563200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f45640743b0 0x7f45641a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:09.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.002+0000 7f4563200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4564074cd0 0x7f45641a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:09.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.002+0000 7f4563200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f456410e760 0x7f456419a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:09.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.002+0000 7f4563200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4564114370 con 0x7f456410e760 2024-04-24T08:55:09.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.003+0000 7f4563200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f45641141f0 con 0x7f4564074cd0 2024-04-24T08:55:09.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.003+0000 7f4563200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f45641144f0 con 0x7f45640743b0 2024-04-24T08:55:09.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.003+0000 7f4562800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f456410e760 0x7f456419a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:09.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.003+0000 7f4562800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f456410e760 0x7f456419a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46182/0 (socket says 172.21.15.83:46182) 2024-04-24T08:55:09.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.003+0000 7f4562800640 1 -- 172.21.15.83:0/839237358 learned_addr learned my addr 172.21.15.83:0/839237358 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:55:09.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.003+0000 7f4561e00640 1 --2- 172.21.15.83:0/839237358 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f45640743b0 0x7f45641a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:09.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.003+0000 7f4559400640 1 --2- 172.21.15.83:0/839237358 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4564074cd0 0x7f45641a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:09.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.003+0000 7f4562800640 1 -- 172.21.15.83:0/839237358 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f45640743b0 msgr2=0x7f45641a0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:09.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.003+0000 7f4562800640 1 --2- 172.21.15.83:0/839237358 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f45640743b0 0x7f45641a0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:09.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.003+0000 7f4562800640 1 -- 172.21.15.83:0/839237358 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4564074cd0 msgr2=0x7f45641a0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:09.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.003+0000 7f4562800640 1 --2- 172.21.15.83:0/839237358 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4564074cd0 0x7f45641a0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:09.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.003+0000 7f4562800640 1 -- 172.21.15.83:0/839237358 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f456419afd0 con 0x7f456410e760 2024-04-24T08:55:09.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.004+0000 7f4562800640 1 --2- 172.21.15.83:0/839237358 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f456410e760 0x7f456419a710 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f455c00b970 tx=0x7f455c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:09.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.004+0000 7f455b400640 1 -- 172.21.15.83:0/839237358 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f455c019070 con 0x7f456410e760 2024-04-24T08:55:09.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.004+0000 7f455b400640 1 -- 172.21.15.83:0/839237358 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f455c014440 con 0x7f456410e760 2024-04-24T08:55:09.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.004+0000 7f4563200640 1 -- 172.21.15.83:0/839237358 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f456419b2c0 con 0x7f456410e760 2024-04-24T08:55:09.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.004+0000 7f455b400640 1 -- 172.21.15.83:0/839237358 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f455c0075d0 con 0x7f456410e760 2024-04-24T08:55:09.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.004+0000 7f4563200640 1 -- 172.21.15.83:0/839237358 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45641a75d0 con 0x7f456410e760 2024-04-24T08:55:09.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.008+0000 7f455b400640 1 -- 172.21.15.83:0/839237358 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f455c01c070 con 0x7f456410e760 2024-04-24T08:55:09.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.008+0000 7f456c047740 1 Processor -- start 2024-04-24T08:55:09.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.008+0000 7f456c047740 1 -- start start 2024-04-24T08:55:09.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.009+0000 7f456c047740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a1faa67a30 0x55a1faa67e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:09.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.009+0000 7f456c047740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1faa68350 0x55a1faa724e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:09.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.009+0000 7f456c047740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1faa72a20 0x55a1faa74e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:09.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.009+0000 7f456c047740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a1fa9cd1f0 con 0x55a1faa72a20 2024-04-24T08:55:09.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.009+0000 7f456c047740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a1fa973350 con 0x55a1faa68350 2024-04-24T08:55:09.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.009+0000 7f456c047740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a1fa9c91b0 con 0x55a1faa67a30 2024-04-24T08:55:09.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.009+0000 7f4559400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1faa72a20 0x55a1faa74e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:09.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.009+0000 7f4559400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1faa72a20 0x55a1faa74e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46186/0 (socket says 172.21.15.83:46186) 2024-04-24T08:55:09.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.009+0000 7f4559400640 1 -- 172.21.15.83:0/1255600633 learned_addr learned my addr 172.21.15.83:0/1255600633 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:55:09.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.009+0000 7f4561e00640 1 --2- 172.21.15.83:0/1255600633 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1faa68350 0x55a1faa724e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:09.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.010+0000 7f4559400640 1 --2- 172.21.15.83:0/1255600633 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a1faa67a30 0x55a1faa67e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:09.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.010+0000 7f455b400640 1 --2- 172.21.15.83:0/839237358 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f45480a3d20 0x7f45480a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:09.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.010+0000 7f4559400640 1 -- 172.21.15.83:0/1255600633 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a1faa67a30 msgr2=0x55a1faa67e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:09.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.010+0000 7f4559400640 1 --2- 172.21.15.83:0/1255600633 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a1faa67a30 0x55a1faa67e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:09.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.010+0000 7f4559400640 1 -- 172.21.15.83:0/1255600633 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1faa68350 msgr2=0x55a1faa724e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:09.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.010+0000 7f4559400640 1 --2- 172.21.15.83:0/1255600633 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1faa68350 0x55a1faa724e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:09.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.010+0000 7f4559400640 1 -- 172.21.15.83:0/1255600633 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1fa9c7f50 con 0x55a1faa72a20 2024-04-24T08:55:09.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.010+0000 7f455b400640 1 -- 172.21.15.83:0/839237358 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f455c0d0110 con 0x7f456410e760 2024-04-24T08:55:09.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.010+0000 7f4562800640 1 --2- 172.21.15.83:0/839237358 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f45480a3d20 0x7f45480a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:09.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.010+0000 7f4559400640 1 --2- 172.21.15.83:0/1255600633 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1faa72a20 0x55a1faa74e10 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f454c0124f0 tx=0x7f454c00e610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a55a7bba997aea36 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:09.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.010+0000 7f4553400640 1 -- 172.21.15.83:0/1255600633 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f454c018070 con 0x55a1faa72a20 2024-04-24T08:55:09.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.011+0000 7f4553400640 1 -- 172.21.15.83:0/1255600633 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f454c0047a0 con 0x55a1faa72a20 2024-04-24T08:55:09.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.011+0000 7f4553400640 1 -- 172.21.15.83:0/1255600633 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f454c010a70 con 0x55a1faa72a20 2024-04-24T08:55:09.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.011+0000 7f456c047740 1 -- 172.21.15.83:0/1255600633 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1faa72a20 msgr2=0x55a1faa74e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:09.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.011+0000 7f4562800640 1 --2- 172.21.15.83:0/839237358 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f45480a3d20 0x7f45480a61e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f455c0cd5a0 tx=0x7f455c014b50 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:09.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.011+0000 7f456c047740 1 --2- 172.21.15.83:0/1255600633 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1faa72a20 0x55a1faa74e10 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f454c0124f0 tx=0x7f454c00e610 comp rx=0 tx=0).stop 2024-04-24T08:55:09.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.012+0000 7f456c047740 1 -- 172.21.15.83:0/1255600633 shutdown_connections 2024-04-24T08:55:09.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.012+0000 7f456c047740 1 --2- 172.21.15.83:0/1255600633 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1faa72a20 0x55a1faa74e10 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:09.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.012+0000 7f456c047740 1 --2- 172.21.15.83:0/1255600633 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1faa68350 0x55a1faa724e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:09.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.012+0000 7f456c047740 1 --2- 172.21.15.83:0/1255600633 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a1faa67a30 0x55a1faa67e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:09.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.012+0000 7f456c047740 1 -- 172.21.15.83:0/1255600633 >> 172.21.15.83:0/1255600633 conn(0x55a1faa63d50 msgr2=0x55a1faa64130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:09.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.012+0000 7f456c047740 1 -- 172.21.15.83:0/1255600633 shutdown_connections 2024-04-24T08:55:09.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.012+0000 7f456c047740 1 -- 172.21.15.83:0/1255600633 wait complete. 2024-04-24T08:55:09.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.012+0000 7f456c047740 1 -- start start 2024-04-24T08:55:09.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.012+0000 7f456c047740 1 Processor -- start 2024-04-24T08:55:09.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.013+0000 7f456c047740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1faa67a30 0x55a1faa6fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:09.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.013+0000 7f456c047740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1faa68350 0x55a1faa6ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:09.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.013+0000 7f456c047740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a1faa72a20 0x55a1faa70490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:09.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.013+0000 7f456c047740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a1faa75810 con 0x55a1faa67a30 2024-04-24T08:55:09.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.013+0000 7f456c047740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a1fa986690 con 0x55a1faa68350 2024-04-24T08:55:09.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.013+0000 7f456c047740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a1faa75990 con 0x55a1faa72a20 2024-04-24T08:55:09.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.013+0000 7f456c047740 10 client.0 _ll_register_callbacks cb 0x55a1fa9c2650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:55:09.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.013+0000 7f4559400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1faa67a30 0x55a1faa6fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:09.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.013+0000 7f4559400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1faa67a30 0x55a1faa6fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46200/0 (socket says 172.21.15.83:46200) 2024-04-24T08:55:09.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.013+0000 7f4559400640 1 -- 172.21.15.83:0/1148032147 learned_addr learned my addr 172.21.15.83:0/1148032147 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:55:09.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.013+0000 7f4561e00640 1 --2- 172.21.15.83:0/1148032147 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1faa68350 0x55a1faa6ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:09.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.014+0000 7f4559400640 1 -- 172.21.15.83:0/1148032147 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a1faa72a20 msgr2=0x55a1faa70490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:09.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.014+0000 7f4559400640 1 --2- 172.21.15.83:0/1148032147 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a1faa72a20 0x55a1faa70490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:09.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.014+0000 7f4559400640 1 -- 172.21.15.83:0/1148032147 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1faa68350 msgr2=0x55a1faa6ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:09.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.014+0000 7f4559400640 1 --2- 172.21.15.83:0/1148032147 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1faa68350 0x55a1faa6ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:09.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.014+0000 7f4559400640 1 -- 172.21.15.83:0/1148032147 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1fa985920 con 0x55a1faa67a30 2024-04-24T08:55:09.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.014+0000 7f4559400640 1 --2- 172.21.15.83:0/1148032147 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1faa67a30 0x55a1faa6fa10 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f454c001e50 tx=0x7f454c004f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cbc8bc34a05d024 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:09.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.014+0000 7f4547e00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:55:09.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.014+0000 7f4547e00640 1 -- 172.21.15.83:0/1148032147 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f454c018040 con 0x55a1faa67a30 2024-04-24T08:55:09.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.014+0000 7f4547e00640 1 -- 172.21.15.83:0/1148032147 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f454c004020 con 0x55a1faa67a30 2024-04-24T08:55:09.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.014+0000 7f456c047740 1 -- 172.21.15.83:0/1148032147 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f454c0047a0 con 0x55a1faa67a30 2024-04-24T08:55:09.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.015+0000 7f4547e00640 1 -- 172.21.15.83:0/1148032147 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f454c030050 con 0x55a1faa67a30 2024-04-24T08:55:09.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.015+0000 7f456c047740 10 client.17007 fetch_fsmap learned FSMap version 30 2024-04-24T08:55:09.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.015+0000 7f456c047740 1 -- 172.21.15.83:0/1148032147 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a1fa7a97d0 con 0x55a1faa67a30 2024-04-24T08:55:09.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.015+0000 7f4547e00640 1 -- 172.21.15.83:0/1148032147 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f454c0047a0 con 0x55a1faa67a30 2024-04-24T08:55:09.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.015+0000 7f4547e00640 1 -- 172.21.15.83:0/1148032147 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7f454c003680 con 0x55a1faa67a30 2024-04-24T08:55:09.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.015+0000 7f456c047740 10 client.17007 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T08:55:09.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.015+0000 7f456c047740 10 client.17007 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:55:09.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.015+0000 7f456c047740 20 client.17007 populate_metadata read hostname 'smithi083' 2024-04-24T08:55:09.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.015+0000 7f456c047740 1 --2- 172.21.15.83:0/1148032147 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55a1faa82c90 0x55a1faa85080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:09.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.016+0000 7f456c047740 4 client.17007 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:55:09.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.016+0000 7f456c047740 1 -- 172.21.15.83:0/1148032147 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a1fa9dac10 con 0x55a1faa82c90 2024-04-24T08:55:09.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.016+0000 7f4561e00640 1 --2- 172.21.15.83:0/1148032147 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55a1faa82c90 0x55a1faa85080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:09.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.017+0000 7f4547e00640 10 client.17007 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:55:09.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.017+0000 7f4561e00640 1 --2- 172.21.15.83:0/1148032147 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55a1faa82c90 0x55a1faa85080 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f853167e4ca3ab3 server_cookie=a70d0abbf64606f in_seq=0 out_seq=0 2024-04-24T08:55:09.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.019+0000 7f4547e00640 1 -- 172.21.15.83:0/1148032147 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a1fa9dac10 con 0x55a1faa82c90 2024-04-24T08:55:09.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.019+0000 7f4547e00640 10 client.17007 handle_command_reply: tid=0 2024-04-24T08:55:09.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.020+0000 7f456c047740 1 client.17007 shutdown 2024-04-24T08:55:09.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.020+0000 7f456c047740 1 -- 172.21.15.83:0/1148032147 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1faa67a30 msgr2=0x55a1faa6fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:09.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.020+0000 7f456c047740 1 --2- 172.21.15.83:0/1148032147 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1faa67a30 0x55a1faa6fa10 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f454c001e50 tx=0x7f454c004f30 comp rx=0 tx=0).stop 2024-04-24T08:55:09.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.020+0000 7f456c047740 1 -- 172.21.15.83:0/1148032147 shutdown_connections 2024-04-24T08:55:09.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.020+0000 7f456c047740 1 --2- 172.21.15.83:0/1148032147 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55a1faa82c90 0x55a1faa85080 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:09.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.020+0000 7f456c047740 1 --2- 172.21.15.83:0/1148032147 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a1faa72a20 0x55a1faa70490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:09.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.020+0000 7f456c047740 1 --2- 172.21.15.83:0/1148032147 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1faa68350 0x55a1faa6ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:09.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.020+0000 7f4547e00640 0 client.17007 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:55:09.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.020+0000 7f456c047740 1 --2- 172.21.15.83:0/1148032147 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1faa67a30 0x55a1faa6fa10 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:09.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.020+0000 7f456c047740 1 -- 172.21.15.83:0/1148032147 >> 172.21.15.83:0/1148032147 conn(0x55a1faa63d50 msgr2=0x55a1faa69b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:09.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.020+0000 7f456c047740 1 -- 172.21.15.83:0/1148032147 shutdown_connections 2024-04-24T08:55:09.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.020+0000 7f456c047740 1 -- 172.21.15.83:0/1148032147 wait complete. 2024-04-24T08:55:09.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.020+0000 7f456c047740 20 client.17007 trim_cache size 0 max 16384 2024-04-24T08:55:09.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.040+0000 7f456c047740 1 Processor -- start 2024-04-24T08:55:09.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.040+0000 7f456c047740 1 -- start start 2024-04-24T08:55:09.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.040+0000 7f456c047740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a1faa67a30 0x55a1faa7e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:09.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.040+0000 7f456c047740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1faa68350 0x55a1faa6a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:09.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.040+0000 7f456c047740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1faa72a20 0x55a1faa8a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:09.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.040+0000 7f456c047740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f45641b1530 con 0x55a1faa72a20 2024-04-24T08:55:09.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.040+0000 7f456c047740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f45641b3100 con 0x55a1faa68350 2024-04-24T08:55:09.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.040+0000 7f456c047740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f45641b1c00 con 0x55a1faa67a30 2024-04-24T08:55:09.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.041+0000 7f4559400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1faa72a20 0x55a1faa8a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:09.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.041+0000 7f4561e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1faa68350 0x55a1faa6a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:09.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.041+0000 7f4559400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1faa72a20 0x55a1faa8a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46216/0 (socket says 172.21.15.83:46216) 2024-04-24T08:55:09.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.041+0000 7f4559400640 1 -- 172.21.15.83:0/1172878565 learned_addr learned my addr 172.21.15.83:0/1172878565 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:55:09.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.041+0000 7f4559400640 1 --2- 172.21.15.83:0/1172878565 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a1faa67a30 0x55a1faa7e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:09.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.041+0000 7f4559400640 1 -- 172.21.15.83:0/1172878565 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a1faa67a30 msgr2=0x55a1faa7e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:09.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.042+0000 7f4559400640 1 --2- 172.21.15.83:0/1172878565 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a1faa67a30 0x55a1faa7e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:09.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.042+0000 7f4559400640 1 -- 172.21.15.83:0/1172878565 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1faa68350 msgr2=0x55a1faa6a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:09.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.042+0000 7f4559400640 1 --2- 172.21.15.83:0/1172878565 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1faa68350 0x55a1faa6a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:09.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.042+0000 7f4559400640 1 -- 172.21.15.83:0/1172878565 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1fa6df3f0 con 0x55a1faa72a20 2024-04-24T08:55:09.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.042+0000 7f4559400640 1 --2- 172.21.15.83:0/1172878565 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1faa72a20 0x55a1faa8a6e0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f454c004ce0 tx=0x7f454c010650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54cc9c883740768a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:09.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.042+0000 7f4552a00640 1 -- 172.21.15.83:0/1172878565 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f454c03a070 con 0x55a1faa72a20 2024-04-24T08:55:09.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.042+0000 7f4552a00640 1 -- 172.21.15.83:0/1172878565 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f454c023070 con 0x55a1faa72a20 2024-04-24T08:55:09.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.042+0000 7f4552a00640 1 -- 172.21.15.83:0/1172878565 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f454c0042d0 con 0x55a1faa72a20 2024-04-24T08:55:09.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.043+0000 7f456c047740 1 -- 172.21.15.83:0/1172878565 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1faa72a20 msgr2=0x55a1faa8a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:09.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.043+0000 7f456c047740 1 --2- 172.21.15.83:0/1172878565 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1faa72a20 0x55a1faa8a6e0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f454c004ce0 tx=0x7f454c010650 comp rx=0 tx=0).stop 2024-04-24T08:55:09.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.043+0000 7f456c047740 1 -- 172.21.15.83:0/1172878565 shutdown_connections 2024-04-24T08:55:09.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.043+0000 7f456c047740 1 --2- 172.21.15.83:0/1172878565 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1faa72a20 0x55a1faa8a6e0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:09.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.043+0000 7f456c047740 1 --2- 172.21.15.83:0/1172878565 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1faa68350 0x55a1faa6a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:09.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.043+0000 7f456c047740 1 --2- 172.21.15.83:0/1172878565 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a1faa67a30 0x55a1faa7e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:09.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.043+0000 7f456c047740 1 -- 172.21.15.83:0/1172878565 >> 172.21.15.83:0/1172878565 conn(0x55a1faa63d50 msgr2=0x55a1faa70260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:09.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.043+0000 7f456c047740 1 -- 172.21.15.83:0/1172878565 shutdown_connections 2024-04-24T08:55:09.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.043+0000 7f456c047740 1 -- 172.21.15.83:0/1172878565 wait complete. 2024-04-24T08:55:09.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.044+0000 7f456c047740 1 -- start start 2024-04-24T08:55:09.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.044+0000 7f456c047740 1 Processor -- start 2024-04-24T08:55:09.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.044+0000 7f456c047740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1faa67a30 0x55a1faa7ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:09.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.045+0000 7f456c047740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a1faa68350 0x55a1faa76220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:09.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.045+0000 7f456c047740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1faa72a20 0x55a1faa76760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:09.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.045+0000 7f456c047740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f45641b29a0 con 0x55a1faa67a30 2024-04-24T08:55:09.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.045+0000 7f456c047740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f45641b22d0 con 0x55a1faa72a20 2024-04-24T08:55:09.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.045+0000 7f456c047740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f45641af510 con 0x55a1faa68350 2024-04-24T08:55:09.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.045+0000 7f456c047740 10 client.0 _ll_register_callbacks cb 0x7f454c005ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:55:09.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.045+0000 7f4559400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1faa67a30 0x55a1faa7ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:09.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.045+0000 7f4559400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1faa67a30 0x55a1faa7ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46232/0 (socket says 172.21.15.83:46232) 2024-04-24T08:55:09.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.045+0000 7f4559400640 1 -- 172.21.15.83:0/1483513234 learned_addr learned my addr 172.21.15.83:0/1483513234 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:55:09.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.045+0000 7f4561e00640 1 --2- 172.21.15.83:0/1483513234 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a1faa68350 0x55a1faa76220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:09.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.045+0000 7f4559400640 1 -- 172.21.15.83:0/1483513234 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a1faa68350 msgr2=0x55a1faa76220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:09.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.045+0000 7f4559400640 1 --2- 172.21.15.83:0/1483513234 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a1faa68350 0x55a1faa76220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:09.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.045+0000 7f4559400640 1 -- 172.21.15.83:0/1483513234 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1faa72a20 msgr2=0x55a1faa76760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:09.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.045+0000 7f4559400640 1 --2- 172.21.15.83:0/1483513234 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1faa72a20 0x55a1faa76760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:09.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.045+0000 7f4559400640 1 -- 172.21.15.83:0/1483513234 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1fa69f0f0 con 0x55a1faa67a30 2024-04-24T08:55:09.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.046+0000 7f4559400640 1 --2- 172.21.15.83:0/1483513234 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1faa67a30 0x55a1faa7ae60 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f454c000c00 tx=0x7f454c004f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad7a222ea4ce6c13 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:09.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.046+0000 7f4547e00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:55:09.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.046+0000 7f4547e00640 1 -- 172.21.15.83:0/1483513234 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f454c03a070 con 0x55a1faa67a30 2024-04-24T08:55:09.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.046+0000 7f4547e00640 1 -- 172.21.15.83:0/1483513234 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f454c0035f0 con 0x55a1faa67a30 2024-04-24T08:55:09.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.046+0000 7f456c047740 1 -- 172.21.15.83:0/1483513234 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f454c023070 con 0x55a1faa67a30 2024-04-24T08:55:09.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.046+0000 7f4547e00640 1 -- 172.21.15.83:0/1483513234 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f454c049050 con 0x55a1faa67a30 2024-04-24T08:55:09.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.046+0000 7f456c047740 10 client.17013 fetch_fsmap learned FSMap version 30 2024-04-24T08:55:09.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.046+0000 7f456c047740 1 -- 172.21.15.83:0/1483513234 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a1fa992810 con 0x55a1faa67a30 2024-04-24T08:55:09.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.047+0000 7f4547e00640 1 -- 172.21.15.83:0/1483513234 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f454c023070 con 0x55a1faa67a30 2024-04-24T08:55:09.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.047+0000 7f4547e00640 1 -- 172.21.15.83:0/1483513234 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7f454c010950 con 0x55a1faa67a30 2024-04-24T08:55:09.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.047+0000 7f456c047740 10 client.17013 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T08:55:09.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.047+0000 7f456c047740 10 client.17013 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:55:09.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.047+0000 7f456c047740 20 client.17013 populate_metadata read hostname 'smithi083' 2024-04-24T08:55:09.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.047+0000 7f456c047740 1 --2- 172.21.15.83:0/1483513234 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55a1faae90d0 0x55a1faaeb4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:09.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.047+0000 7f456c047740 4 client.17013 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T08:55:09.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.047+0000 7f456c047740 1 -- 172.21.15.83:0/1483513234 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a1fa9d4a80 con 0x55a1faae90d0 2024-04-24T08:55:09.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.050+0000 7f4561e00640 1 --2- 172.21.15.83:0/1483513234 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55a1faae90d0 0x55a1faaeb4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:09.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.051+0000 7f4561e00640 1 --2- 172.21.15.83:0/1483513234 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55a1faae90d0 0x55a1faaeb4f0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=730d67db9e071635 server_cookie=58ac09adaab037da in_seq=0 out_seq=0 2024-04-24T08:55:09.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.051+0000 7f4547e00640 10 client.17013 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:55:09.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.052+0000 7f4547e00640 1 -- 172.21.15.83:0/1483513234 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55a1fa9d4a80 con 0x55a1faae90d0 2024-04-24T08:55:09.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.052+0000 7f4547e00640 10 client.17013 handle_command_reply: tid=0 2024-04-24T08:55:09.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.052+0000 7f456c047740 1 client.17013 shutdown 2024-04-24T08:55:09.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.052+0000 7f456c047740 1 -- 172.21.15.83:0/1483513234 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1faa67a30 msgr2=0x55a1faa7ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:09.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.052+0000 7f456c047740 1 --2- 172.21.15.83:0/1483513234 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1faa67a30 0x55a1faa7ae60 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f454c000c00 tx=0x7f454c004f30 comp rx=0 tx=0).stop 2024-04-24T08:55:09.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.052+0000 7f456c047740 1 -- 172.21.15.83:0/1483513234 shutdown_connections 2024-04-24T08:55:09.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.052+0000 7f456c047740 1 --2- 172.21.15.83:0/1483513234 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55a1faae90d0 0x55a1faaeb4f0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:09.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.052+0000 7f456c047740 1 --2- 172.21.15.83:0/1483513234 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1faa72a20 0x55a1faa76760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:09.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.052+0000 7f456c047740 1 --2- 172.21.15.83:0/1483513234 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a1faa68350 0x55a1faa76220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:09.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.052+0000 7f456c047740 1 --2- 172.21.15.83:0/1483513234 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1faa67a30 0x55a1faa7ae60 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:09.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.052+0000 7f456c047740 1 -- 172.21.15.83:0/1483513234 >> 172.21.15.83:0/1483513234 conn(0x55a1faa63d50 msgr2=0x55a1faa62740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:09.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.052+0000 7f4547e00640 0 client.17013 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:55:09.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.052+0000 7f456c047740 1 -- 172.21.15.83:0/1483513234 shutdown_connections 2024-04-24T08:55:09.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.052+0000 7f456c047740 1 -- 172.21.15.83:0/1483513234 wait complete. 2024-04-24T08:55:09.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.053+0000 7f456c047740 20 client.17013 trim_cache size 0 max 16384 2024-04-24T08:55:09.054 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T08:55:09.054 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T08:55:09.054 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-04-24T08:55:09.054 INFO:teuthology.orchestra.run.smithi083.stdout: "d75a3574-ba7e-4796-b98a-22b8e0f4b5f4": { 2024-04-24T08:55:09.054 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-04-24T08:55:09.054 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "d75a3574-ba7e-4796-b98a-22b8e0f4b5f4", 2024-04-24T08:55:09.054 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-04-24T08:55:09.055 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T08:55:09.055 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T08:55:09.055 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T08:55:09.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.053+0000 7f4563200640 1 -- 172.21.15.83:0/839237358 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f45480a3d20 msgr2=0x7f45480a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:09.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.053+0000 7f4563200640 1 --2- 172.21.15.83:0/839237358 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f45480a3d20 0x7f45480a61e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f455c0cd5a0 tx=0x7f455c014b50 comp rx=0 tx=0).stop 2024-04-24T08:55:09.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.053+0000 7f4563200640 1 -- 172.21.15.83:0/839237358 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f456410e760 msgr2=0x7f456419a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:09.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.053+0000 7f4563200640 1 --2- 172.21.15.83:0/839237358 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f456410e760 0x7f456419a710 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f455c00b970 tx=0x7f455c00be30 comp rx=0 tx=0).stop 2024-04-24T08:55:09.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.054+0000 7f4563200640 1 -- 172.21.15.83:0/839237358 shutdown_connections 2024-04-24T08:55:09.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.054+0000 7f4563200640 1 --2- 172.21.15.83:0/839237358 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f45480a3d20 0x7f45480a61e0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:09.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.054+0000 7f4563200640 1 --2- 172.21.15.83:0/839237358 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f456410e760 0x7f456419a710 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:09.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.054+0000 7f4563200640 1 --2- 172.21.15.83:0/839237358 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4564074cd0 0x7f45641a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:09.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.054+0000 7f4563200640 1 --2- 172.21.15.83:0/839237358 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f45640743b0 0x7f45641a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:09.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.054+0000 7f4563200640 1 -- 172.21.15.83:0/839237358 >> 172.21.15.83:0/839237358 conn(0x7f45640fd9d0 msgr2=0x7f45640fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:09.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.054+0000 7f4563200640 1 -- 172.21.15.83:0/839237358 shutdown_connections 2024-04-24T08:55:09.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:09.054+0000 7f4563200640 1 -- 172.21.15.83:0/839237358 wait complete. 2024-04-24T08:55:09.072 INFO:tasks.cephfs.filesystem:scrub status for tag:d75a3574-ba7e-4796-b98a-22b8e0f4b5f4 - {'path': '/', 'tag': 'd75a3574-ba7e-4796-b98a-22b8e0f4b5f4', 'options': 'recursive,force'} 2024-04-24T08:55:09.206 INFO:journalctl@ceph.mds.b.smithi138.stdout:Apr 24 08:55:08 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-b[61515]: 2024-04-24T08:55:08.800+0000 7f7fbbc00700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000215 [...6,head] /volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/COREL/ auth{2=1} v2420 ap=2 f(v78 m2024-04-24T08:54:56.219545+0000 11=11+0) n(v3 rc2024-04-24T08:54:56.219545+0000 b2861056 12=11+1) old_inodes=2 (inest sync->mix g=2) caps={15603=pAsLsXsFs/-@1804} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x55cf8760d700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000000215:[<0x10000000009/COREL v369>,<0x10000000008/~dmtmp v57>,<0x10000000007/client0 v33>,<0x10000000006/clients v23>,<0x10000000005/tmp v23>,<0x10000000003/client.0 v22>,<0x10000000002/e395bed6-5bba-41b8-a256-3505e4afcca2 v36>,<0x10000000001/sv_0 v55>,<0x10000000000/qa v43>,<0x1/volumes v39>]//[]","memoryvalue":"(2)0x10000000215:[<0x10000000009/COREL v2420>,<0x10000000008/~dmtmp v1806>,<0x10000000007/client0 v248>,<0x10000000006/clients v238>,<0x10000000005/tmp v236>,<0x10000000003/client.0 v233>,<0x10000000002/e395bed6-5bba-41b8-a256-3505e4afcca2 v246>,<0x10000000001/sv_0 v260>,<0x10000000000/qa v244>,<0x1/volumes v231>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-04-24T08:54:56.177355+0000 12=12+0)","ondisk_value.rstat":"n(v0 rc2024-04-24T08:54:56.219545+0000 b2861056 12=11+1)","memory_value.dirstat":"f(v78 m2024-04-24T08:54:56.219545+0000 11=11+0)","memory_value.rstat":"n(v3 rc2024-04-24T08:54:56.219545+0000 b2861056 12=11+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-04-24T08:55:09.206 INFO:journalctl@ceph.mds.b.smithi138.stdout:Apr 24 08:55:08 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-b[61515]: 2024-04-24T08:55:08.803+0000 7f7fbbc00700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000020c [...6,head] /volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/EXCEL/ auth{3=1} v2466 ap=2 f(v90 m2024-04-24T08:54:55.960015+0000 2=2+0) n(v8 rc2024-04-24T08:54:55.960015+0000 b520192 3=2+1) old_inodes=2 (inest sync->mix g=3) caps={15603=pAsLsXsFs/-@1909} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x55cf8760a580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000020c:[<0x10000000009/EXCEL v347>,<0x10000000008/~dmtmp v57>,<0x10000000007/client0 v33>,<0x10000000006/clients v23>,<0x10000000005/tmp v23>,<0x10000000003/client.0 v22>,<0x10000000002/e395bed6-5bba-41b8-a256-3505e4afcca2 v36>,<0x10000000001/sv_0 v55>,<0x10000000000/qa v43>,<0x1/volumes v39>]//[]","memoryvalue":"(2)0x1000000020c:[<0x10000000009/EXCEL v2466>,<0x10000000008/~dmtmp v1806>,<0x10000000007/client0 v248>,<0x10000000006/clients v238>,<0x10000000005/tmp v236>,<0x10000000003/client.0 v233>,<0x10000000002/e395bed6-5bba-41b8-a256-3505e4afcca2 v246>,<0x10000000001/sv_0 v260>,<0x10000000000/qa v244>,<0x1/volumes v231>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-04-24T08:54:55.921796+0000 3=3+0)","ondisk_value.rstat":"n(v0 rc2024-04-24T08:54:55.960015+0000 b520192 3=2+1)","memory_value.dirstat":"f(v90 m2024-04-24T08:54:55.960015+0000 2=2+0)","memory_value.rstat":"n(v8 rc2024-04-24T08:54:55.960015+0000 b520192 3=2+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-04-24T08:55:09.206 INFO:journalctl@ceph.mds.b.smithi138.stdout:Apr 24 08:55:08 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-b[61515]: 2024-04-24T08:55:08.810+0000 7f7fbbc00700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000221 [...6,head] /volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/WORDPRO/ auth{3=1} v2468 ap=2 f(v105 m2024-04-24T08:54:56.454834+0000 3=3+0) n(v6 rc2024-04-24T08:54:56.454834+0000 b780288 4=3+1) old_inodes=2 (inest sync->mix g=3) caps={15603=pAsLsXsFs/-@1860} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x55cf87627700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000000221:[<0x10000000009/WORDPRO v375>,<0x10000000008/~dmtmp v57>,<0x10000000007/client0 v33>,<0x10000000006/clients v23>,<0x10000000005/tmp v23>,<0x10000000003/client.0 v22>,<0x10000000002/e395bed6-5bba-41b8-a256-3505e4afcca2 v36>,<0x10000000001/sv_0 v55>,<0x10000000000/qa v43>,<0x1/volumes v39>]//[]","memoryvalue":"(2)0x10000000221:[<0x10000000009/WORDPRO v2468>,<0x10000000008/~dmtmp v1806>,<0x10000000007/client0 v248>,<0x10000000006/clients v238>,<0x10000000005/tmp v236>,<0x10000000003/client.0 v233>,<0x10000000002/e395bed6-5bba-41b8-a256-3505e4afcca2 v246>,<0x10000000001/sv_0 v260>,<0x10000000000/qa v244>,<0x1/volumes v231>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-04-24T08:54:56.417246+0000 4=4+0)","ondisk_value.rstat":"n(v0 rc2024-04-24T08:54:56.454834+0000 b780288 4=3+1)","memory_value.dirstat":"f(v105 m2024-04-24T08:54:56.454834+0000 3=3+0)","memory_value.rstat":"n(v6 rc2024-04-24T08:54:56.454834+0000 b780288 4=3+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-04-24T08:55:09.206 INFO:journalctl@ceph.mds.b.smithi138.stdout:Apr 24 08:55:08 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-b[61515]: 2024-04-24T08:55:08.817+0000 7f7fbbc00700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000225 [...6,head] /volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/PM/ auth{3=1} v2476 ap=2 f(v222 m2024-04-24T08:54:56.493918+0000 3=3+0) n(v6 rc2024-04-24T08:54:56.493918+0000 b780288 4=3+1) old_inodes=2 (inest sync->mix g=3) caps={15603=pAsLsXsFs/-@2338} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x55cf8762f180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000000225:[<0x10000000009/PM v381>,<0x10000000008/~dmtmp v57>,<0x10000000007/client0 v33>,<0x10000000006/clients v23>,<0x10000000005/tmp v23>,<0x10000000003/client.0 v22>,<0x10000000002/e395bed6-5bba-41b8-a256-3505e4afcca2 v36>,<0x10000000001/sv_0 v55>,<0x10000000000/qa v43>,<0x1/volumes v39>]//[]","memoryvalue":"(2)0x10000000225:[<0x10000000009/PM v2476>,<0x10000000008/~dmtmp v1806>,<0x10000000007/client0 v248>,<0x10000000006/clients v238>,<0x10000000005/tmp v236>,<0x10000000003/client.0 v233>,<0x10000000002/e395bed6-5bba-41b8-a256-3505e4afcca2 v246>,<0x10000000001/sv_0 v260>,<0x10000000000/qa v244>,<0x1/volumes v231>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-04-24T08:54:54.945496+0000 4=4+0)","ondisk_value.rstat":"n(v0 rc2024-04-24T08:54:56.493918+0000 b780288 4=3+1)","memory_value.dirstat":"f(v222 m2024-04-24T08:54:56.493918+0000 3=3+0)","memory_value.rstat":"n(v6 rc2024-04-24T08:54:56.493918+0000 b780288 4=3+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-04-24T08:55:09.206 INFO:journalctl@ceph.mds.b.smithi138.stdout:Apr 24 08:55:08 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-b[61515]: 2024-04-24T08:55:08.819+0000 7f7fbbc00700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000023e [...6,head] /volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/ACCESS/ auth{3=1} v2474 ap=2 f(v42 m2024-04-24T08:54:56.803269+0000 1=1+0) n(v6 rc2024-04-24T08:54:56.803269+0000 b260096 2=1+1) old_inodes=2 (inest sync->mix g=3) caps={15603=pAsLsXsFs/-@1601} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x55cf87666b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000023e:[<0x10000000009/ACCESS v463>,<0x10000000008/~dmtmp v57>,<0x10000000007/client0 v33>,<0x10000000006/clients v23>,<0x10000000005/tmp v23>,<0x10000000003/client.0 v22>,<0x10000000002/e395bed6-5bba-41b8-a256-3505e4afcca2 v36>,<0x10000000001/sv_0 v55>,<0x10000000000/qa v43>,<0x1/volumes v39>]//[]","memoryvalue":"(2)0x1000000023e:[<0x10000000009/ACCESS v2474>,<0x10000000008/~dmtmp v1806>,<0x10000000007/client0 v248>,<0x10000000006/clients v238>,<0x10000000005/tmp v236>,<0x10000000003/client.0 v233>,<0x10000000002/e395bed6-5bba-41b8-a256-3505e4afcca2 v246>,<0x10000000001/sv_0 v260>,<0x10000000000/qa v244>,<0x1/volumes v231>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-04-24T08:54:56.765454+0000 2=2+0)","ondisk_value.rstat":"n(v0 rc2024-04-24T08:54:56.803269+0000 b260096 2=1+1)","memory_value.dirstat":"f(v42 m2024-04-24T08:54:56.803269+0000 1=1+0)","memory_value.rstat":"n(v6 rc2024-04-24T08:54:56.803269+0000 b260096 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-04-24T08:55:09.565 INFO:tasks.workunit.client.0.smithi083.stdout: 1 13021 0.83 MB/sec warmup 103 sec latency 51.740 ms 2024-04-24T08:55:10.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:55:09 smithi138 ceph-mon[31441]: pgmap v109: 129 pgs: 129 active+clean; 28 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 349 KiB/s wr, 95 op/s 2024-04-24T08:55:10.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:55:09 smithi138 ceph-mon[31441]: scrub queued for path: / 2024-04-24T08:55:10.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:55:09 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T08:55:10.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:55:09 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T08:55:10.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:55:09 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T08:55:10.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:55:09 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T08:55:10.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:55:09 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T08:55:10.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:55:09 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T08:55:10.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:55:09 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T08:55:10.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:55:09 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T08:55:10.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:55:09 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T08:55:10.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:55:09 smithi138 ceph-mon[31441]: Scrub error on inode 0x10000000215 (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/COREL) see mds.b log and `damage ls` output for details 2024-04-24T08:55:10.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:55:09 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T08:55:10.207 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:55:09 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T08:55:10.207 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:55:09 smithi138 ceph-mon[31441]: Scrub error on inode 0x1000000020c (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/EXCEL) see mds.b log and `damage ls` output for details 2024-04-24T08:55:10.207 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:55:09 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T08:55:10.207 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:55:09 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T08:55:10.207 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:55:09 smithi138 ceph-mon[31441]: Scrub error on inode 0x10000000221 (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/WORDPRO) see mds.b log and `damage ls` output for details 2024-04-24T08:55:10.207 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:55:09 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T08:55:10.207 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:55:09 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T08:55:10.207 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:55:09 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T08:55:10.207 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:55:09 smithi138 ceph-mon[31441]: Scrub error on inode 0x10000000225 (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/PM) see mds.b log and `damage ls` output for details 2024-04-24T08:55:10.207 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:55:09 smithi138 ceph-mon[31441]: Scrub error on inode 0x1000000023e (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/ACCESS) see mds.b log and `damage ls` output for details 2024-04-24T08:55:10.207 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:55:09 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T08:55:10.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:55:09 smithi189 ceph-mon[31791]: pgmap v109: 129 pgs: 129 active+clean; 28 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 349 KiB/s wr, 95 op/s 2024-04-24T08:55:10.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:55:09 smithi189 ceph-mon[31791]: scrub queued for path: / 2024-04-24T08:55:10.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:55:09 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T08:55:10.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:55:09 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T08:55:10.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:55:09 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T08:55:10.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:55:09 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T08:55:10.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:55:09 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T08:55:10.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:55:09 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T08:55:10.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:55:09 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T08:55:10.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:55:09 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T08:55:10.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:55:09 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T08:55:10.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:55:09 smithi189 ceph-mon[31791]: Scrub error on inode 0x10000000215 (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/COREL) see mds.b log and `damage ls` output for details 2024-04-24T08:55:10.209 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:55:09 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T08:55:10.209 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:55:09 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T08:55:10.209 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:55:09 smithi189 ceph-mon[31791]: Scrub error on inode 0x1000000020c (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/EXCEL) see mds.b log and `damage ls` output for details 2024-04-24T08:55:10.209 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:55:09 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T08:55:10.209 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:55:09 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T08:55:10.209 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:55:09 smithi189 ceph-mon[31791]: Scrub error on inode 0x10000000221 (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/WORDPRO) see mds.b log and `damage ls` output for details 2024-04-24T08:55:10.209 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:55:09 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T08:55:10.209 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:55:09 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T08:55:10.209 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:55:09 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T08:55:10.209 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:55:09 smithi189 ceph-mon[31791]: Scrub error on inode 0x10000000225 (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/PM) see mds.b log and `damage ls` output for details 2024-04-24T08:55:10.209 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:55:09 smithi189 ceph-mon[31791]: Scrub error on inode 0x1000000023e (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/ACCESS) see mds.b log and `damage ls` output for details 2024-04-24T08:55:10.209 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:55:09 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T08:55:10.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:55:09 smithi083 ceph-mon[27907]: pgmap v109: 129 pgs: 129 active+clean; 28 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 349 KiB/s wr, 95 op/s 2024-04-24T08:55:10.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:55:09 smithi083 ceph-mon[27907]: scrub queued for path: / 2024-04-24T08:55:10.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:55:09 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T08:55:10.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:55:09 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T08:55:10.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:55:09 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T08:55:10.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:55:09 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T08:55:10.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:55:09 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T08:55:10.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:55:09 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T08:55:10.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:55:09 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T08:55:10.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:55:09 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T08:55:10.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:55:09 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T08:55:10.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:55:09 smithi083 ceph-mon[27907]: Scrub error on inode 0x10000000215 (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/COREL) see mds.b log and `damage ls` output for details 2024-04-24T08:55:10.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:55:09 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T08:55:10.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:55:09 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T08:55:10.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:55:09 smithi083 ceph-mon[27907]: Scrub error on inode 0x1000000020c (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/EXCEL) see mds.b log and `damage ls` output for details 2024-04-24T08:55:10.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:55:09 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T08:55:10.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:55:09 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T08:55:10.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:55:09 smithi083 ceph-mon[27907]: Scrub error on inode 0x10000000221 (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/WORDPRO) see mds.b log and `damage ls` output for details 2024-04-24T08:55:10.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:55:09 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T08:55:10.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:55:09 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T08:55:10.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:55:09 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T08:55:10.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:55:09 smithi083 ceph-mon[27907]: Scrub error on inode 0x10000000225 (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/PM) see mds.b log and `damage ls` output for details 2024-04-24T08:55:10.306 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:55:09 smithi083 ceph-mon[27907]: Scrub error on inode 0x1000000023e (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/ACCESS) see mds.b log and `damage ls` output for details 2024-04-24T08:55:10.306 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:55:09 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T08:55:10.565 INFO:tasks.workunit.client.0.smithi083.stdout: 1 13102 0.82 MB/sec warmup 104 sec latency 51.669 ms 2024-04-24T08:55:11.565 INFO:tasks.workunit.client.0.smithi083.stdout: 1 13228 0.82 MB/sec warmup 105 sec latency 57.986 ms 2024-04-24T08:55:12.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:55:11 smithi138 ceph-mon[31441]: pgmap v110: 129 pgs: 129 active+clean; 28 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 258 KiB/s wr, 69 op/s 2024-04-24T08:55:12.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:55:11 smithi189 ceph-mon[31791]: pgmap v110: 129 pgs: 129 active+clean; 28 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 258 KiB/s wr, 69 op/s 2024-04-24T08:55:12.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:55:11 smithi083 ceph-mon[27907]: pgmap v110: 129 pgs: 129 active+clean; 28 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 258 KiB/s wr, 69 op/s 2024-04-24T08:55:12.565 INFO:tasks.workunit.client.0.smithi083.stdout: 1 13411 0.84 MB/sec warmup 106 sec latency 49.909 ms 2024-04-24T08:55:13.565 INFO:tasks.workunit.client.0.smithi083.stdout: 1 13588 0.85 MB/sec warmup 107 sec latency 47.904 ms 2024-04-24T08:55:14.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:55:13 smithi138 ceph-mon[31441]: pgmap v111: 129 pgs: 129 active+clean; 30 MiB data, 521 MiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 462 KiB/s wr, 96 op/s 2024-04-24T08:55:14.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:55:13 smithi189 ceph-mon[31791]: pgmap v111: 129 pgs: 129 active+clean; 30 MiB data, 521 MiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 462 KiB/s wr, 96 op/s 2024-04-24T08:55:14.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:55:13 smithi083 ceph-mon[27907]: pgmap v111: 129 pgs: 129 active+clean; 30 MiB data, 521 MiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 462 KiB/s wr, 96 op/s 2024-04-24T08:55:14.566 INFO:tasks.workunit.client.0.smithi083.stdout: 1 13823 0.85 MB/sec warmup 108 sec latency 49.587 ms 2024-04-24T08:55:15.566 INFO:tasks.workunit.client.0.smithi083.stdout: 1 13940 0.85 MB/sec warmup 109 sec latency 50.946 ms 2024-04-24T08:55:16.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:55:15 smithi138 ceph-mon[31441]: pgmap v112: 129 pgs: 129 active+clean; 30 MiB data, 521 MiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 267 KiB/s wr, 52 op/s 2024-04-24T08:55:16.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:55:15 smithi189 ceph-mon[31791]: pgmap v112: 129 pgs: 129 active+clean; 30 MiB data, 521 MiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 267 KiB/s wr, 52 op/s 2024-04-24T08:55:16.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:55:15 smithi083 ceph-mon[27907]: pgmap v112: 129 pgs: 129 active+clean; 30 MiB data, 521 MiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 267 KiB/s wr, 52 op/s 2024-04-24T08:55:16.566 INFO:tasks.workunit.client.0.smithi083.stdout: 1 13968 0.84 MB/sec warmup 110 sec latency 53.129 ms 2024-04-24T08:55:17.566 INFO:tasks.workunit.client.0.smithi083.stdout: 1 14009 0.84 MB/sec warmup 111 sec latency 67.269 ms 2024-04-24T08:55:18.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:55:17 smithi138 ceph-mon[31441]: pgmap v113: 129 pgs: 129 active+clean; 30 MiB data, 521 MiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 267 KiB/s wr, 52 op/s 2024-04-24T08:55:18.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:55:17 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T08:55:18.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:55:17 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T08:55:18.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:55:17 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T08:55:18.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:55:17 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T08:55:18.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:55:17 smithi189 ceph-mon[31791]: pgmap v113: 129 pgs: 129 active+clean; 30 MiB data, 521 MiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 267 KiB/s wr, 52 op/s 2024-04-24T08:55:18.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:55:17 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T08:55:18.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:55:17 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T08:55:18.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:55:17 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T08:55:18.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:55:17 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T08:55:18.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:55:17 smithi083 ceph-mon[27907]: pgmap v113: 129 pgs: 129 active+clean; 30 MiB data, 521 MiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 267 KiB/s wr, 52 op/s 2024-04-24T08:55:18.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:55:17 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T08:55:18.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:55:17 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T08:55:18.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:55:17 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T08:55:18.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:55:17 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T08:55:18.566 INFO:tasks.workunit.client.0.smithi083.stdout: 1 14194 0.86 MB/sec warmup 112 sec latency 51.896 ms 2024-04-24T08:55:19.566 INFO:tasks.workunit.client.0.smithi083.stdout: 1 14313 0.85 MB/sec warmup 113 sec latency 56.229 ms 2024-04-24T08:55:20.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:55:19 smithi138 ceph-mon[31441]: pgmap v114: 129 pgs: 129 active+clean; 31 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 431 KiB/s wr, 70 op/s 2024-04-24T08:55:20.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:55:19 smithi189 ceph-mon[31791]: pgmap v114: 129 pgs: 129 active+clean; 31 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 431 KiB/s wr, 70 op/s 2024-04-24T08:55:20.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:55:19 smithi083 ceph-mon[27907]: pgmap v114: 129 pgs: 129 active+clean; 31 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 431 KiB/s wr, 70 op/s 2024-04-24T08:55:20.522 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.522+0000 7f5da3c00640 1 -- 172.21.15.83:0/494964426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d9c104e20 msgr2=0x7f5d9c108e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:20.522 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.522+0000 7f5da3c00640 1 --2- 172.21.15.83:0/494964426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d9c104e20 0x7f5d9c108e60 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c009900 tx=0x7f5d8c02a410 comp rx=0 tx=0).stop 2024-04-24T08:55:20.522 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.522+0000 7f5da3c00640 1 -- 172.21.15.83:0/494964426 shutdown_connections 2024-04-24T08:55:20.522 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.522+0000 7f5da3c00640 1 --2- 172.21.15.83:0/494964426 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5d9c109540 0x7f5d9c111110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:20.522 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.522+0000 7f5da3c00640 1 --2- 172.21.15.83:0/494964426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d9c104e20 0x7f5d9c108e60 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:20.522 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.522+0000 7f5da3c00640 1 --2- 172.21.15.83:0/494964426 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d9c104470 0x7f5d9c104850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:20.522 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.522+0000 7f5da3c00640 1 -- 172.21.15.83:0/494964426 >> 172.21.15.83:0/494964426 conn(0x7f5d9c0ffd30 msgr2=0x7f5d9c102150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:20.522 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.522+0000 7f5da3c00640 1 -- 172.21.15.83:0/494964426 shutdown_connections 2024-04-24T08:55:20.522 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.522+0000 7f5da3c00640 1 -- 172.21.15.83:0/494964426 wait complete. 2024-04-24T08:55:20.524 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.523+0000 7f5da3c00640 1 Processor -- start 2024-04-24T08:55:20.524 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.524+0000 7f5da3c00640 1 -- start start 2024-04-24T08:55:20.524 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.524+0000 7f5da3c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d9c104470 0x7f5d9c1a06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:20.524 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.524+0000 7f5da3c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5d9c109540 0x7f5d9c1a0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:20.524 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.524+0000 7f5da3c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d9c19a840 0x7f5d9c19acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:20.524 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.524+0000 7f5da3c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5d9c114480 con 0x7f5d9c104470 2024-04-24T08:55:20.525 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.524+0000 7f5da3c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5d9c114300 con 0x7f5d9c19a840 2024-04-24T08:55:20.525 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.524+0000 7f5da3c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5d9c114600 con 0x7f5d9c109540 2024-04-24T08:55:20.525 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.524+0000 7f5da1200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d9c104470 0x7f5d9c1a06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:20.525 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.525+0000 7f5da1200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d9c104470 0x7f5d9c1a06d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33558/0 (socket says 172.21.15.83:33558) 2024-04-24T08:55:20.525 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.525+0000 7f5da1200640 1 -- 172.21.15.83:0/678591341 learned_addr learned my addr 172.21.15.83:0/678591341 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:55:20.525 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.525+0000 7f5d9be00640 1 --2- 172.21.15.83:0/678591341 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5d9c109540 0x7f5d9c1a0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:20.525 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.525+0000 7f5da1c00640 1 --2- 172.21.15.83:0/678591341 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d9c19a840 0x7f5d9c19acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:20.525 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.525+0000 7f5da1200640 1 -- 172.21.15.83:0/678591341 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5d9c109540 msgr2=0x7f5d9c1a0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:20.525 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.525+0000 7f5da1200640 1 --2- 172.21.15.83:0/678591341 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5d9c109540 0x7f5d9c1a0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:20.525 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.525+0000 7f5da1200640 1 -- 172.21.15.83:0/678591341 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d9c19a840 msgr2=0x7f5d9c19acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:20.525 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.525+0000 7f5da1200640 1 --2- 172.21.15.83:0/678591341 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d9c19a840 0x7f5d9c19acf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:20.525 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.525+0000 7f5da1200640 1 -- 172.21.15.83:0/678591341 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d9c19b3d0 con 0x7f5d9c104470 2024-04-24T08:55:20.525 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.525+0000 7f5da1200640 1 --2- 172.21.15.83:0/678591341 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d9c104470 0x7f5d9c1a06d0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f5d9000c890 tx=0x7f5d9000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:20.526 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.526+0000 7f5d99600640 1 -- 172.21.15.83:0/678591341 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d90004220 con 0x7f5d9c104470 2024-04-24T08:55:20.526 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.526+0000 7f5d99600640 1 -- 172.21.15.83:0/678591341 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5d900043c0 con 0x7f5d9c104470 2024-04-24T08:55:20.526 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.526+0000 7f5da3c00640 1 -- 172.21.15.83:0/678591341 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d9c19b6c0 con 0x7f5d9c104470 2024-04-24T08:55:20.526 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.526+0000 7f5d99600640 1 -- 172.21.15.83:0/678591341 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d90004a00 con 0x7f5d9c104470 2024-04-24T08:55:20.526 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.526+0000 7f5da3c00640 1 -- 172.21.15.83:0/678591341 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d9c1a7870 con 0x7f5d9c104470 2024-04-24T08:55:20.528 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.529+0000 7f5da6a34740 1 Processor -- start 2024-04-24T08:55:20.528 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.529+0000 7f5da6a34740 1 -- start start 2024-04-24T08:55:20.529 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.529+0000 7f5d99600640 1 -- 172.21.15.83:0/678591341 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5d90004ba0 con 0x7f5d9c104470 2024-04-24T08:55:20.529 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.529+0000 7f5da6a34740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d962931b0 0x559d96293590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:20.529 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.529+0000 7f5da6a34740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559d96293ad0 0x559d9629de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:20.530 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.529+0000 7f5da6a34740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d9629e390 0x559d962a0780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:20.530 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.529+0000 7f5da6a34740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559d961f9900 con 0x559d9629e390 2024-04-24T08:55:20.530 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.529+0000 7f5da6a34740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559d9619ff20 con 0x559d962931b0 2024-04-24T08:55:20.530 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.529+0000 7f5da6a34740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559d961f58c0 con 0x559d96293ad0 2024-04-24T08:55:20.530 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.530+0000 7f5d9be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d9629e390 0x559d962a0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:20.530 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.530+0000 7f5d9be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d9629e390 0x559d962a0780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33560/0 (socket says 172.21.15.83:33560) 2024-04-24T08:55:20.530 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.530+0000 7f5d9be00640 1 -- 172.21.15.83:0/3313530982 learned_addr learned my addr 172.21.15.83:0/3313530982 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:55:20.530 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.530+0000 7f5da1c00640 1 --2- 172.21.15.83:0/3313530982 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d962931b0 0x559d96293590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:20.530 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.530+0000 7f5da1200640 1 --2- 172.21.15.83:0/3313530982 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559d96293ad0 0x559d9629de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:20.530 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.530+0000 7f5d9be00640 1 -- 172.21.15.83:0/3313530982 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559d96293ad0 msgr2=0x559d9629de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:20.530 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.530+0000 7f5d9be00640 1 --2- 172.21.15.83:0/3313530982 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559d96293ad0 0x559d9629de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:20.530 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.530+0000 7f5d9be00640 1 -- 172.21.15.83:0/3313530982 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d962931b0 msgr2=0x559d96293590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:20.530 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.530+0000 7f5d9be00640 1 --2- 172.21.15.83:0/3313530982 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d962931b0 0x559d96293590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:20.531 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.530+0000 7f5d9be00640 1 -- 172.21.15.83:0/3313530982 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d961c4e10 con 0x559d9629e390 2024-04-24T08:55:20.532 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.531+0000 7f5d9be00640 1 --2- 172.21.15.83:0/3313530982 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d9629e390 0x559d962a0780 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c009900 tx=0x7f5d8c032cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42a4db859ffe100c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:20.532 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.531+0000 7f5d8a000640 1 -- 172.21.15.83:0/3313530982 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d8c03f070 con 0x559d9629e390 2024-04-24T08:55:20.532 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.531+0000 7f5d8a000640 1 -- 172.21.15.83:0/3313530982 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5d8c0316a0 con 0x559d9629e390 2024-04-24T08:55:20.532 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.531+0000 7f5d8a000640 1 -- 172.21.15.83:0/3313530982 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d8c035070 con 0x559d9629e390 2024-04-24T08:55:20.532 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.531+0000 7f5d99600640 1 --2- 172.21.15.83:0/678591341 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f5d840a3dc0 0x7f5d840a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:20.532 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.532+0000 7f5d99600640 1 -- 172.21.15.83:0/678591341 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f5d900d0920 con 0x7f5d9c104470 2024-04-24T08:55:20.533 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.532+0000 7f5da1c00640 1 --2- 172.21.15.83:0/678591341 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f5d840a3dc0 0x7f5d840a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:20.533 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.532+0000 7f5da6a34740 1 -- 172.21.15.83:0/3313530982 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d9629e390 msgr2=0x559d962a0780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:20.533 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.532+0000 7f5da6a34740 1 --2- 172.21.15.83:0/3313530982 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d9629e390 0x559d962a0780 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c009900 tx=0x7f5d8c032cc0 comp rx=0 tx=0).stop 2024-04-24T08:55:20.533 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.532+0000 7f5da6a34740 1 -- 172.21.15.83:0/3313530982 shutdown_connections 2024-04-24T08:55:20.533 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.532+0000 7f5da6a34740 1 --2- 172.21.15.83:0/3313530982 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d9629e390 0x559d962a0780 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:20.533 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.532+0000 7f5da6a34740 1 --2- 172.21.15.83:0/3313530982 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559d96293ad0 0x559d9629de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:20.533 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.532+0000 7f5da6a34740 1 --2- 172.21.15.83:0/3313530982 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d962931b0 0x559d96293590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:20.533 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.532+0000 7f5da6a34740 1 -- 172.21.15.83:0/3313530982 >> 172.21.15.83:0/3313530982 conn(0x559d9628f6c0 msgr2=0x559d9628faa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:20.533 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.533+0000 7f5da6a34740 1 -- 172.21.15.83:0/3313530982 shutdown_connections 2024-04-24T08:55:20.533 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.533+0000 7f5da1c00640 1 --2- 172.21.15.83:0/678591341 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f5d840a3dc0 0x7f5d840a6280 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f5d94005fd0 tx=0x7f5d940095a0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:20.533 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.533+0000 7f5da6a34740 1 -- 172.21.15.83:0/3313530982 wait complete. 2024-04-24T08:55:20.533 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.533+0000 7f5da6a34740 1 -- start start 2024-04-24T08:55:20.533 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.533+0000 7f5da6a34740 1 Processor -- start 2024-04-24T08:55:20.536 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.534+0000 7f5da6a34740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559d962931b0 0x559d9629b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:20.536 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.534+0000 7f5da6a34740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d96293ad0 0x559d9629b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:20.537 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.534+0000 7f5da6a34740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d9629e390 0x559d9629bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:20.537 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.534+0000 7f5da6a34740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559d962a0f60 con 0x559d96293ad0 2024-04-24T08:55:20.537 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.534+0000 7f5da6a34740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559d961b32c0 con 0x559d9629e390 2024-04-24T08:55:20.537 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.534+0000 7f5da6a34740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559d962a10e0 con 0x559d962931b0 2024-04-24T08:55:20.537 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.534+0000 7f5da6a34740 10 client.0 _ll_register_callbacks cb 0x559d95f0ba00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:55:20.537 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.534+0000 7f5d9be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d96293ad0 0x559d9629b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:20.537 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.534+0000 7f5d9be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d96293ad0 0x559d9629b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33570/0 (socket says 172.21.15.83:33570) 2024-04-24T08:55:20.537 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.534+0000 7f5d9be00640 1 -- 172.21.15.83:0/4130158703 learned_addr learned my addr 172.21.15.83:0/4130158703 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:55:20.537 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.534+0000 7f5da1200640 1 --2- 172.21.15.83:0/4130158703 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559d962931b0 0x559d9629b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:20.537 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.534+0000 7f5da1c00640 1 --2- 172.21.15.83:0/4130158703 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d9629e390 0x559d9629bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:20.537 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.534+0000 7f5d9be00640 1 -- 172.21.15.83:0/4130158703 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559d962931b0 msgr2=0x559d9629b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:20.537 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.534+0000 7f5d9be00640 1 --2- 172.21.15.83:0/4130158703 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559d962931b0 0x559d9629b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:20.537 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.534+0000 7f5d9be00640 1 -- 172.21.15.83:0/4130158703 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d9629e390 msgr2=0x559d9629bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:20.538 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.534+0000 7f5d9be00640 1 --2- 172.21.15.83:0/4130158703 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d9629e390 0x559d9629bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:20.538 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.534+0000 7f5d9be00640 1 -- 172.21.15.83:0/4130158703 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d961c3fd0 con 0x559d96293ad0 2024-04-24T08:55:20.538 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.534+0000 7f5da1200640 1 --2- 172.21.15.83:0/4130158703 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559d962931b0 0x559d9629b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:55:20.538 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.535+0000 7f5da1c00640 1 --2- 172.21.15.83:0/4130158703 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d9629e390 0x559d9629bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:55:20.538 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.535+0000 7f5d9be00640 1 --2- 172.21.15.83:0/4130158703 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d96293ad0 0x559d9629b850 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c02a930 tx=0x7f5d8c0314a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=798186816746ead0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:20.538 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.535+0000 7f5d7ea00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:55:20.538 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.535+0000 7f5d7ea00640 1 -- 172.21.15.83:0/4130158703 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d8c04a020 con 0x559d96293ad0 2024-04-24T08:55:20.538 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.535+0000 7f5d7ea00640 1 -- 172.21.15.83:0/4130158703 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f5d8c0035f0 con 0x559d96293ad0 2024-04-24T08:55:20.538 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.535+0000 7f5da6a34740 1 -- 172.21.15.83:0/4130158703 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d8c0316a0 con 0x559d96293ad0 2024-04-24T08:55:20.538 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.535+0000 7f5d7ea00640 1 -- 172.21.15.83:0/4130158703 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5d8c055050 con 0x559d96293ad0 2024-04-24T08:55:20.538 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.535+0000 7f5da6a34740 10 client.17025 fetch_fsmap learned FSMap version 30 2024-04-24T08:55:20.538 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.536+0000 7f5da6a34740 1 -- 172.21.15.83:0/4130158703 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559d95f0c210 con 0x559d96293ad0 2024-04-24T08:55:20.538 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.536+0000 7f5d7ea00640 1 -- 172.21.15.83:0/4130158703 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5d8c0316a0 con 0x559d96293ad0 2024-04-24T08:55:20.538 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.536+0000 7f5d7ea00640 1 -- 172.21.15.83:0/4130158703 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7f5d8c004b00 con 0x559d96293ad0 2024-04-24T08:55:20.539 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.536+0000 7f5da6a34740 10 client.17025 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T08:55:20.539 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.536+0000 7f5da6a34740 10 client.17025 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T08:55:20.539 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.536+0000 7f5da6a34740 20 client.17025 populate_metadata read hostname 'smithi083' 2024-04-24T08:55:20.539 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.536+0000 7f5da6a34740 1 --2- 172.21.15.83:0/4130158703 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x559d962ae2e0 0x559d962b06d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:20.539 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.536+0000 7f5da6a34740 4 client.17025 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:55:20.539 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.536+0000 7f5da6a34740 1 -- 172.21.15.83:0/4130158703 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559d96207320 con 0x559d962ae2e0 2024-04-24T08:55:20.539 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.537+0000 7f5da1200640 1 --2- 172.21.15.83:0/4130158703 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x559d962ae2e0 0x559d962b06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:20.539 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.538+0000 7f5da1200640 1 --2- 172.21.15.83:0/4130158703 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x559d962ae2e0 0x559d962b06d0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66cf8419e898fad server_cookie=7026bb24bcf78155 in_seq=0 out_seq=0 2024-04-24T08:55:20.539 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.538+0000 7f5d7ea00640 10 client.17025 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:55:20.540 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.540+0000 7f5d7ea00640 1 -- 172.21.15.83:0/4130158703 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559d96207320 con 0x559d962ae2e0 2024-04-24T08:55:20.540 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.540+0000 7f5d7ea00640 10 client.17025 handle_command_reply: tid=0 2024-04-24T08:55:20.540 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.540+0000 7f5da6a34740 1 client.17025 shutdown 2024-04-24T08:55:20.540 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.540+0000 7f5da6a34740 1 -- 172.21.15.83:0/4130158703 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d96293ad0 msgr2=0x559d9629b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:20.541 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.540+0000 7f5da6a34740 1 --2- 172.21.15.83:0/4130158703 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d96293ad0 0x559d9629b850 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c02a930 tx=0x7f5d8c0314a0 comp rx=0 tx=0).stop 2024-04-24T08:55:20.541 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.540+0000 7f5da6a34740 1 -- 172.21.15.83:0/4130158703 shutdown_connections 2024-04-24T08:55:20.541 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.540+0000 7f5da6a34740 1 --2- 172.21.15.83:0/4130158703 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x559d962ae2e0 0x559d962b06d0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:20.541 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.540+0000 7f5da6a34740 1 --2- 172.21.15.83:0/4130158703 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d9629e390 0x559d9629bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:20.541 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.540+0000 7f5da6a34740 1 --2- 172.21.15.83:0/4130158703 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d96293ad0 0x559d9629b850 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:20.541 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.540+0000 7f5d7ea00640 0 client.17025 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:55:20.541 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.540+0000 7f5da6a34740 1 --2- 172.21.15.83:0/4130158703 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559d962931b0 0x559d9629b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:20.541 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.540+0000 7f5da6a34740 1 -- 172.21.15.83:0/4130158703 >> 172.21.15.83:0/4130158703 conn(0x559d9628f6c0 msgr2=0x559d9628e840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:20.541 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.540+0000 7f5da6a34740 1 -- 172.21.15.83:0/4130158703 shutdown_connections 2024-04-24T08:55:20.541 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.540+0000 7f5da6a34740 1 -- 172.21.15.83:0/4130158703 wait complete. 2024-04-24T08:55:20.541 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.540+0000 7f5da6a34740 20 client.17025 trim_cache size 0 max 16384 2024-04-24T08:55:20.557 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.557+0000 7f5da6a34740 1 Processor -- start 2024-04-24T08:55:20.557 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.557+0000 7f5da6a34740 1 -- start start 2024-04-24T08:55:20.557 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.558+0000 7f5da6a34740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559d962931b0 0x559d962a9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:20.557 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.558+0000 7f5da6a34740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d96293ad0 0x559d96295b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:20.557 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.558+0000 7f5da6a34740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d9629e390 0x559d962b5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:20.557 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.558+0000 7f5da6a34740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5d9c1b1610 con 0x559d9629e390 2024-04-24T08:55:20.557 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.558+0000 7f5da6a34740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5d9c1b31e0 con 0x559d96293ad0 2024-04-24T08:55:20.558 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.558+0000 7f5da6a34740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5d9c1b1ce0 con 0x559d962931b0 2024-04-24T08:55:20.558 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.558+0000 7f5da1c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d9629e390 0x559d962b5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:20.558 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.558+0000 7f5da1c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d9629e390 0x559d962b5d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33576/0 (socket says 172.21.15.83:33576) 2024-04-24T08:55:20.558 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.558+0000 7f5da1c00640 1 -- 172.21.15.83:0/3022047206 learned_addr learned my addr 172.21.15.83:0/3022047206 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:55:20.559 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.558+0000 7f5da1200640 1 --2- 172.21.15.83:0/3022047206 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559d962931b0 0x559d962a9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:20.559 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.559+0000 7f5d9be00640 1 --2- 172.21.15.83:0/3022047206 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d96293ad0 0x559d96295b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:20.559 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.559+0000 7f5da1c00640 1 -- 172.21.15.83:0/3022047206 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559d962931b0 msgr2=0x559d962a9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:20.559 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.559+0000 7f5da1c00640 1 --2- 172.21.15.83:0/3022047206 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559d962931b0 0x559d962a9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:20.560 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.559+0000 7f5da1c00640 1 -- 172.21.15.83:0/3022047206 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d96293ad0 msgr2=0x559d96295b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:20.560 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.559+0000 7f5da1c00640 1 --2- 172.21.15.83:0/3022047206 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d96293ad0 0x559d96295b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:20.560 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.559+0000 7f5da1c00640 1 -- 172.21.15.83:0/3022047206 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d95fd8620 con 0x559d9629e390 2024-04-24T08:55:20.560 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.559+0000 7f5da1c00640 1 --2- 172.21.15.83:0/3022047206 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d9629e390 0x559d962b5d80 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f5d94003be0 tx=0x7f5d94015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2a66d1b8df40503 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:20.560 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.559+0000 7f5d89600640 1 -- 172.21.15.83:0/3022047206 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d9401f420 con 0x559d9629e390 2024-04-24T08:55:20.560 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.559+0000 7f5d89600640 1 -- 172.21.15.83:0/3022047206 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5d9401fa70 con 0x559d9629e390 2024-04-24T08:55:20.560 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.559+0000 7f5d89600640 1 -- 172.21.15.83:0/3022047206 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d94027b10 con 0x559d9629e390 2024-04-24T08:55:20.560 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.560+0000 7f5da6a34740 1 -- 172.21.15.83:0/3022047206 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d9629e390 msgr2=0x559d962b5d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:20.560 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.560+0000 7f5da6a34740 1 --2- 172.21.15.83:0/3022047206 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d9629e390 0x559d962b5d80 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f5d94003be0 tx=0x7f5d94015000 comp rx=0 tx=0).stop 2024-04-24T08:55:20.560 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.560+0000 7f5da6a34740 1 -- 172.21.15.83:0/3022047206 shutdown_connections 2024-04-24T08:55:20.560 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.560+0000 7f5da6a34740 1 --2- 172.21.15.83:0/3022047206 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d9629e390 0x559d962b5d80 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:20.560 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.560+0000 7f5da6a34740 1 --2- 172.21.15.83:0/3022047206 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d96293ad0 0x559d96295b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:20.560 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.560+0000 7f5da6a34740 1 --2- 172.21.15.83:0/3022047206 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559d962931b0 0x559d962a9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:20.560 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.560+0000 7f5da6a34740 1 -- 172.21.15.83:0/3022047206 >> 172.21.15.83:0/3022047206 conn(0x559d9628f6c0 msgr2=0x559d96293eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:20.561 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.560+0000 7f5da6a34740 1 -- 172.21.15.83:0/3022047206 shutdown_connections 2024-04-24T08:55:20.561 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.560+0000 7f5da6a34740 1 -- 172.21.15.83:0/3022047206 wait complete. 2024-04-24T08:55:20.563 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.561+0000 7f5da6a34740 1 -- start start 2024-04-24T08:55:20.563 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.561+0000 7f5da6a34740 1 Processor -- start 2024-04-24T08:55:20.563 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.561+0000 7f5da6a34740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559d962931b0 0x559d962a67a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:20.563 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.561+0000 7f5da6a34740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d96293ad0 0x559d962a1b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:20.564 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.562+0000 7f5da6a34740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d9629e390 0x559d962a20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:20.564 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.562+0000 7f5da6a34740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5d9c1b2a80 con 0x559d96293ad0 2024-04-24T08:55:20.564 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.562+0000 7f5da6a34740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5d9c1b23b0 con 0x559d9629e390 2024-04-24T08:55:20.564 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.562+0000 7f5da6a34740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5d9c1af5f0 con 0x559d962931b0 2024-04-24T08:55:20.564 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.562+0000 7f5da6a34740 10 client.0 _ll_register_callbacks cb 0x7f5d94009440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:55:20.564 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.562+0000 7f5d9be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d96293ad0 0x559d962a1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:20.564 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.562+0000 7f5d9be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d96293ad0 0x559d962a1b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33592/0 (socket says 172.21.15.83:33592) 2024-04-24T08:55:20.564 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.562+0000 7f5d9be00640 1 -- 172.21.15.83:0/3589482312 learned_addr learned my addr 172.21.15.83:0/3589482312 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:55:20.564 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.562+0000 7f5da1200640 1 --2- 172.21.15.83:0/3589482312 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559d962931b0 0x559d962a67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:20.564 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.562+0000 7f5da1c00640 1 --2- 172.21.15.83:0/3589482312 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d9629e390 0x559d962a20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:20.564 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.562+0000 7f5d9be00640 1 -- 172.21.15.83:0/3589482312 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559d962931b0 msgr2=0x559d962a67a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:20.564 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.562+0000 7f5d9be00640 1 --2- 172.21.15.83:0/3589482312 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559d962931b0 0x559d962a67a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:20.564 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.562+0000 7f5d9be00640 1 -- 172.21.15.83:0/3589482312 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d9629e390 msgr2=0x559d962a20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:20.564 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.562+0000 7f5d9be00640 1 --2- 172.21.15.83:0/3589482312 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d9629e390 0x559d962a20b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:20.565 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.562+0000 7f5d9be00640 1 -- 172.21.15.83:0/3589482312 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d962a28e0 con 0x559d96293ad0 2024-04-24T08:55:20.565 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.562+0000 7f5da1200640 1 --2- 172.21.15.83:0/3589482312 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559d962931b0 0x559d962a67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:55:20.565 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.562+0000 7f5d9be00640 1 --2- 172.21.15.83:0/3589482312 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d96293ad0 0x559d962a1b10 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x559d9629c8e0 tx=0x7f5d8c0317a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc31a2c66ca802d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:20.565 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.563+0000 7f5d7ea00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:55:20.565 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.563+0000 7f5d7ea00640 1 -- 172.21.15.83:0/3589482312 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d8c04a040 con 0x559d96293ad0 2024-04-24T08:55:20.565 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.563+0000 7f5d7ea00640 1 -- 172.21.15.83:0/3589482312 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f5d8c0046c0 con 0x559d96293ad0 2024-04-24T08:55:20.565 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.563+0000 7f5da6a34740 1 -- 172.21.15.83:0/3589482312 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d9401fa70 con 0x559d96293ad0 2024-04-24T08:55:20.565 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.563+0000 7f5d7ea00640 1 -- 172.21.15.83:0/3589482312 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5d8c03f070 con 0x559d96293ad0 2024-04-24T08:55:20.565 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.563+0000 7f5da6a34740 10 client.17031 fetch_fsmap learned FSMap version 30 2024-04-24T08:55:20.565 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.563+0000 7f5da6a34740 1 -- 172.21.15.83:0/3589482312 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559d963146d0 con 0x559d96293ad0 2024-04-24T08:55:20.565 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.563+0000 7f5d7ea00640 1 -- 172.21.15.83:0/3589482312 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5d9401fa70 con 0x559d96293ad0 2024-04-24T08:55:20.565 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.564+0000 7f5d7ea00640 1 -- 172.21.15.83:0/3589482312 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7f5d8c003a90 con 0x559d96293ad0 2024-04-24T08:55:20.565 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.564+0000 7f5da6a34740 10 client.17031 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T08:55:20.565 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.564+0000 7f5da6a34740 10 client.17031 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T08:55:20.566 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.564+0000 7f5da6a34740 20 client.17031 populate_metadata read hostname 'smithi083' 2024-04-24T08:55:20.566 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.564+0000 7f5da6a34740 1 --2- 172.21.15.83:0/3589482312 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x559d96314cd0 0x559d963170f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:20.566 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.564+0000 7f5da6a34740 4 client.17031 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T08:55:20.566 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.564+0000 7f5da6a34740 1 -- 172.21.15.83:0/3589482312 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559d96201190 con 0x559d96314cd0 2024-04-24T08:55:20.566 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.564+0000 7f5da1200640 1 --2- 172.21.15.83:0/3589482312 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x559d96314cd0 0x559d963170f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:20.566 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.565+0000 7f5da1200640 1 --2- 172.21.15.83:0/3589482312 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x559d96314cd0 0x559d963170f0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a533ba96b971a9a6 server_cookie=1431d33dd580204a in_seq=0 out_seq=0 2024-04-24T08:55:20.566 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.565+0000 7f5d7ea00640 10 client.17031 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:55:20.567 INFO:tasks.workunit.client.0.smithi083.stdout: 1 14423 0.85 MB/sec warmup 114 sec latency 50.768 ms 2024-04-24T08:55:20.618 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.619+0000 7f5d7ea00640 1 -- 172.21.15.83:0/3589482312 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559d96201190 con 0x559d96314cd0 2024-04-24T08:55:20.618 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.619+0000 7f5d7ea00640 10 client.17031 handle_command_reply: tid=0 2024-04-24T08:55:20.619 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.619+0000 7f5da6a34740 1 client.17031 shutdown 2024-04-24T08:55:20.619 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.619+0000 7f5da6a34740 1 -- 172.21.15.83:0/3589482312 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d96293ad0 msgr2=0x559d962a1b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:20.619 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.619+0000 7f5da6a34740 1 --2- 172.21.15.83:0/3589482312 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d96293ad0 0x559d962a1b10 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x559d9629c8e0 tx=0x7f5d8c0317a0 comp rx=0 tx=0).stop 2024-04-24T08:55:20.620 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.620+0000 7f5da6a34740 1 -- 172.21.15.83:0/3589482312 shutdown_connections 2024-04-24T08:55:20.620 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.620+0000 7f5da6a34740 1 --2- 172.21.15.83:0/3589482312 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x559d96314cd0 0x559d963170f0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:20.620 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.620+0000 7f5da6a34740 1 --2- 172.21.15.83:0/3589482312 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d9629e390 0x559d962a20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:20.620 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.620+0000 7f5da6a34740 1 --2- 172.21.15.83:0/3589482312 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559d96293ad0 0x559d962a1b10 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:20.620 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.620+0000 7f5da6a34740 1 --2- 172.21.15.83:0/3589482312 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559d962931b0 0x559d962a67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:20.620 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.620+0000 7f5da6a34740 1 -- 172.21.15.83:0/3589482312 >> 172.21.15.83:0/3589482312 conn(0x559d9628f6c0 msgr2=0x559d9628dfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:20.620 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.620+0000 7f5d7ea00640 0 client.17031 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:55:20.620 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.620+0000 7f5da6a34740 1 -- 172.21.15.83:0/3589482312 shutdown_connections 2024-04-24T08:55:20.620 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.620+0000 7f5da6a34740 1 -- 172.21.15.83:0/3589482312 wait complete. 2024-04-24T08:55:20.620 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.620+0000 7f5da6a34740 20 client.17031 trim_cache size 0 max 16384 2024-04-24T08:55:20.621 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T08:55:20.621 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T08:55:20.621 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T08:55:20.621 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T08:55:20.621 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.621+0000 7f5da3c00640 1 -- 172.21.15.83:0/678591341 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f5d840a3dc0 msgr2=0x7f5d840a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:20.622 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.621+0000 7f5da3c00640 1 --2- 172.21.15.83:0/678591341 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f5d840a3dc0 0x7f5d840a6280 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f5d94005fd0 tx=0x7f5d940095a0 comp rx=0 tx=0).stop 2024-04-24T08:55:20.622 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.621+0000 7f5da3c00640 1 -- 172.21.15.83:0/678591341 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d9c104470 msgr2=0x7f5d9c1a06d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:20.622 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.621+0000 7f5da3c00640 1 --2- 172.21.15.83:0/678591341 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d9c104470 0x7f5d9c1a06d0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f5d9000c890 tx=0x7f5d9000cd50 comp rx=0 tx=0).stop 2024-04-24T08:55:20.622 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.621+0000 7f5da3c00640 1 -- 172.21.15.83:0/678591341 shutdown_connections 2024-04-24T08:55:20.622 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.621+0000 7f5da3c00640 1 --2- 172.21.15.83:0/678591341 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f5d840a3dc0 0x7f5d840a6280 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:20.622 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.621+0000 7f5da3c00640 1 --2- 172.21.15.83:0/678591341 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d9c19a840 0x7f5d9c19acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:20.622 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.621+0000 7f5da3c00640 1 --2- 172.21.15.83:0/678591341 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5d9c109540 0x7f5d9c1a0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:20.622 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.621+0000 7f5da3c00640 1 --2- 172.21.15.83:0/678591341 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d9c104470 0x7f5d9c1a06d0 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:20.622 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.621+0000 7f5da3c00640 1 -- 172.21.15.83:0/678591341 >> 172.21.15.83:0/678591341 conn(0x7f5d9c0ffd30 msgr2=0x7f5d9c1010e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:20.622 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.621+0000 7f5da3c00640 1 -- 172.21.15.83:0/678591341 shutdown_connections 2024-04-24T08:55:20.622 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:20.622+0000 7f5da3c00640 1 -- 172.21.15.83:0/678591341 wait complete. 2024-04-24T08:55:21.567 INFO:tasks.workunit.client.0.smithi083.stdout: 1 14478 0.84 MB/sec warmup 115 sec latency 42.964 ms 2024-04-24T08:55:22.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:55:21 smithi138 ceph-mon[31441]: pgmap v115: 129 pgs: 129 active+clean; 31 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 367 KiB/s wr, 45 op/s 2024-04-24T08:55:22.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:55:21 smithi189 ceph-mon[31791]: pgmap v115: 129 pgs: 129 active+clean; 31 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 367 KiB/s wr, 45 op/s 2024-04-24T08:55:22.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:55:21 smithi083 ceph-mon[27907]: pgmap v115: 129 pgs: 129 active+clean; 31 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 367 KiB/s wr, 45 op/s 2024-04-24T08:55:22.566 INFO:tasks.workunit.client.0.smithi083.stdout: 1 14539 0.84 MB/sec warmup 116 sec latency 50.919 ms 2024-04-24T08:55:23.566 INFO:tasks.workunit.client.0.smithi083.stdout: 1 14598 0.83 MB/sec warmup 117 sec latency 45.532 ms 2024-04-24T08:55:24.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:55:23 smithi138 ceph-mon[31441]: pgmap v116: 129 pgs: 129 active+clean; 33 MiB data, 532 MiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 625 KiB/s wr, 71 op/s 2024-04-24T08:55:24.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:55:23 smithi189 ceph-mon[31791]: pgmap v116: 129 pgs: 129 active+clean; 33 MiB data, 532 MiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 625 KiB/s wr, 71 op/s 2024-04-24T08:55:24.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:55:23 smithi083 ceph-mon[27907]: pgmap v116: 129 pgs: 129 active+clean; 33 MiB data, 532 MiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 625 KiB/s wr, 71 op/s 2024-04-24T08:55:24.567 INFO:tasks.workunit.client.0.smithi083.stdout: 1 14660 0.83 MB/sec warmup 118 sec latency 37.612 ms 2024-04-24T08:55:25.567 INFO:tasks.workunit.client.0.smithi083.stdout: 1 14731 0.82 MB/sec warmup 119 sec latency 51.208 ms 2024-04-24T08:55:26.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:55:25 smithi083 ceph-mon[27907]: pgmap v117: 129 pgs: 129 active+clean; 33 MiB data, 532 MiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 421 KiB/s wr, 43 op/s 2024-04-24T08:55:26.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:55:25 smithi138 ceph-mon[31441]: pgmap v117: 129 pgs: 129 active+clean; 33 MiB data, 532 MiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 421 KiB/s wr, 43 op/s 2024-04-24T08:55:26.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:55:25 smithi189 ceph-mon[31791]: pgmap v117: 129 pgs: 129 active+clean; 33 MiB data, 532 MiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 421 KiB/s wr, 43 op/s 2024-04-24T08:55:27.567 INFO:tasks.workunit.client.0.smithi083.stdout: 1 14917 0.23 MB/sec execute 1 sec latency 50.583 ms 2024-04-24T08:55:28.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:55:27 smithi083 ceph-mon[27907]: pgmap v118: 129 pgs: 129 active+clean; 33 MiB data, 532 MiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 421 KiB/s wr, 43 op/s 2024-04-24T08:55:28.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:55:27 smithi138 ceph-mon[31441]: pgmap v118: 129 pgs: 129 active+clean; 33 MiB data, 532 MiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 421 KiB/s wr, 43 op/s 2024-04-24T08:55:28.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:55:27 smithi189 ceph-mon[31791]: pgmap v118: 129 pgs: 129 active+clean; 33 MiB data, 532 MiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 421 KiB/s wr, 43 op/s 2024-04-24T08:55:28.567 INFO:tasks.workunit.client.0.smithi083.stdout: 1 15059 0.30 MB/sec execute 2 sec latency 48.869 ms 2024-04-24T08:55:29.567 INFO:tasks.workunit.client.0.smithi083.stdout: 1 15147 0.27 MB/sec execute 3 sec latency 61.565 ms 2024-04-24T08:55:30.260 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:55:29 smithi189 ceph-mon[31791]: pgmap v119: 129 pgs: 129 active+clean; 33 MiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 493 KiB/s wr, 66 op/s 2024-04-24T08:55:30.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:55:29 smithi083 ceph-mon[27907]: pgmap v119: 129 pgs: 129 active+clean; 33 MiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 493 KiB/s wr, 66 op/s 2024-04-24T08:55:30.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:55:29 smithi138 ceph-mon[31441]: pgmap v119: 129 pgs: 129 active+clean; 33 MiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 493 KiB/s wr, 66 op/s 2024-04-24T08:55:30.567 INFO:tasks.workunit.client.0.smithi083.stdout: 1 15268 0.48 MB/sec execute 4 sec latency 50.231 ms 2024-04-24T08:55:31.567 INFO:tasks.workunit.client.0.smithi083.stdout: 1 15377 0.41 MB/sec execute 5 sec latency 50.836 ms 2024-04-24T08:55:32.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:55:31 smithi083 ceph-mon[27907]: pgmap v120: 129 pgs: 129 active+clean; 33 MiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 329 KiB/s wr, 48 op/s 2024-04-24T08:55:32.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:55:31 smithi138 ceph-mon[31441]: pgmap v120: 129 pgs: 129 active+clean; 33 MiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 329 KiB/s wr, 48 op/s 2024-04-24T08:55:32.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:55:31 smithi189 ceph-mon[31791]: pgmap v120: 129 pgs: 129 active+clean; 33 MiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 329 KiB/s wr, 48 op/s 2024-04-24T08:55:32.568 INFO:tasks.workunit.client.0.smithi083.stdout: 1 15486 0.36 MB/sec execute 6 sec latency 47.781 ms 2024-04-24T08:55:33.568 INFO:tasks.workunit.client.0.smithi083.stdout: 1 15668 0.72 MB/sec execute 7 sec latency 52.762 ms 2024-04-24T08:55:33.772 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.771+0000 7f4f97e00640 1 -- 172.21.15.83:0/1003937334 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f981038e0 msgr2=0x7f4f9810bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:33.772 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.771+0000 7f4f97e00640 1 --2- 172.21.15.83:0/1003937334 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f981038e0 0x7f4f9810bdc0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f4f84009900 tx=0x7f4f8402a480 comp rx=0 tx=0).stop 2024-04-24T08:55:33.772 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.771+0000 7f4f97e00640 1 -- 172.21.15.83:0/1003937334 shutdown_connections 2024-04-24T08:55:33.772 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.771+0000 7f4f97e00640 1 --2- 172.21.15.83:0/1003937334 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4f9810c3e0 0x7f4f9810e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:33.772 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.771+0000 7f4f97e00640 1 --2- 172.21.15.83:0/1003937334 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f981038e0 0x7f4f9810bdc0 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:33.772 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.771+0000 7f4f97e00640 1 --2- 172.21.15.83:0/1003937334 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4f98102fc0 0x7f4f981033a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:33.772 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.771+0000 7f4f97e00640 1 -- 172.21.15.83:0/1003937334 >> 172.21.15.83:0/1003937334 conn(0x7f4f980fc820 msgr2=0x7f4f980fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:33.772 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.771+0000 7f4f97e00640 1 -- 172.21.15.83:0/1003937334 shutdown_connections 2024-04-24T08:55:33.772 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.771+0000 7f4f97e00640 1 -- 172.21.15.83:0/1003937334 wait complete. 2024-04-24T08:55:33.775 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.772+0000 7f4f97e00640 1 Processor -- start 2024-04-24T08:55:33.775 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.772+0000 7f4f97e00640 1 -- start start 2024-04-24T08:55:33.775 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.773+0000 7f4f97e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4f98102fc0 0x7f4f981a0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:33.775 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.773+0000 7f4f97e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4f981038e0 0x7f4f981a0970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:33.775 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.773+0000 7f4f97e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f9810c3e0 0x7f4f9819a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:33.775 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.773+0000 7f4f97e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4f981120a0 con 0x7f4f9810c3e0 2024-04-24T08:55:33.775 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.773+0000 7f4f97e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4f98111f20 con 0x7f4f98102fc0 2024-04-24T08:55:33.775 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.773+0000 7f4f97e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4f98112220 con 0x7f4f981038e0 2024-04-24T08:55:33.775 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.773+0000 7f4f97400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f9810c3e0 0x7f4f9819a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:33.775 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.773+0000 7f4f97400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f9810c3e0 0x7f4f9819a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56692/0 (socket says 172.21.15.83:56692) 2024-04-24T08:55:33.775 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.773+0000 7f4f97400640 1 -- 172.21.15.83:0/449859265 learned_addr learned my addr 172.21.15.83:0/449859265 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:55:33.775 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.773+0000 7f4f96a00640 1 --2- 172.21.15.83:0/449859265 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4f98102fc0 0x7f4f981a0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:33.775 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.773+0000 7f4f96000640 1 --2- 172.21.15.83:0/449859265 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4f981038e0 0x7f4f981a0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:33.776 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.774+0000 7f4f97400640 1 -- 172.21.15.83:0/449859265 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4f981038e0 msgr2=0x7f4f981a0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:33.776 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.774+0000 7f4f97400640 1 --2- 172.21.15.83:0/449859265 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4f981038e0 0x7f4f981a0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:33.776 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.774+0000 7f4f97400640 1 -- 172.21.15.83:0/449859265 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4f98102fc0 msgr2=0x7f4f981a0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:33.776 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.774+0000 7f4f97400640 1 --2- 172.21.15.83:0/449859265 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4f98102fc0 0x7f4f981a0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:33.776 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.774+0000 7f4f97400640 1 -- 172.21.15.83:0/449859265 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f9819ad60 con 0x7f4f9810c3e0 2024-04-24T08:55:33.776 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.774+0000 7f4f96a00640 1 --2- 172.21.15.83:0/449859265 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4f98102fc0 0x7f4f981a0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:55:33.776 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.774+0000 7f4f97400640 1 --2- 172.21.15.83:0/449859265 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f9810c3e0 0x7f4f9819a5e0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f4f9000cce0 tx=0x7f4f9000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:33.776 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.774+0000 7f4f8f400640 1 -- 172.21.15.83:0/449859265 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f9000f450 con 0x7f4f9810c3e0 2024-04-24T08:55:33.776 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.774+0000 7f4f8f400640 1 -- 172.21.15.83:0/449859265 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4f9000faa0 con 0x7f4f9810c3e0 2024-04-24T08:55:33.776 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.774+0000 7f4f97e00640 1 -- 172.21.15.83:0/449859265 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f9819b050 con 0x7f4f9810c3e0 2024-04-24T08:55:33.776 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.774+0000 7f4f8f400640 1 -- 172.21.15.83:0/449859265 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f90018b70 con 0x7f4f9810c3e0 2024-04-24T08:55:33.776 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.775+0000 7f4f97e00640 1 -- 172.21.15.83:0/449859265 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f981a72f0 con 0x7f4f9810c3e0 2024-04-24T08:55:33.778 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.778+0000 7f4fa0b83740 1 Processor -- start 2024-04-24T08:55:33.778 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.778+0000 7f4f8f400640 1 -- 172.21.15.83:0/449859265 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4f90018d10 con 0x7f4f9810c3e0 2024-04-24T08:55:33.778 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.778+0000 7f4fa0b83740 1 -- start start 2024-04-24T08:55:33.780 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.779+0000 7f4fa0b83740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564a6f7a03c0 0x564a6f7a07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:33.780 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.779+0000 7f4fa0b83740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a6f7a0ce0 0x564a6f7ab060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:33.780 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.779+0000 7f4fa0b83740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564a6f7ab5a0 0x564a6f7ad990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:33.780 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.779+0000 7f4fa0b83740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x564a6f706900 con 0x564a6f7a03c0 2024-04-24T08:55:33.780 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.779+0000 7f4fa0b83740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x564a6f6acf20 con 0x564a6f7ab5a0 2024-04-24T08:55:33.780 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.779+0000 7f4fa0b83740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x564a6f7028c0 con 0x564a6f7a0ce0 2024-04-24T08:55:33.780 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.779+0000 7f4f96000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564a6f7a03c0 0x564a6f7a07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:33.781 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.779+0000 7f4f96000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564a6f7a03c0 0x564a6f7a07a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56708/0 (socket says 172.21.15.83:56708) 2024-04-24T08:55:33.781 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.779+0000 7f4f96000640 1 -- 172.21.15.83:0/2188288843 learned_addr learned my addr 172.21.15.83:0/2188288843 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:55:33.781 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.779+0000 7f4f96a00640 1 --2- 172.21.15.83:0/2188288843 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a6f7a0ce0 0x564a6f7ab060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:33.781 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.780+0000 7f4f96000640 1 --2- 172.21.15.83:0/2188288843 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564a6f7ab5a0 0x564a6f7ad990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:33.781 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.780+0000 7f4f96000640 1 -- 172.21.15.83:0/2188288843 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a6f7a0ce0 msgr2=0x564a6f7ab060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:33.781 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.780+0000 7f4f96000640 1 --2- 172.21.15.83:0/2188288843 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a6f7a0ce0 0x564a6f7ab060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:33.781 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.780+0000 7f4f96000640 1 -- 172.21.15.83:0/2188288843 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564a6f7ab5a0 msgr2=0x564a6f7ad990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:33.781 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.780+0000 7f4f96000640 1 --2- 172.21.15.83:0/2188288843 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564a6f7ab5a0 0x564a6f7ad990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:33.781 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.780+0000 7f4f96000640 1 -- 172.21.15.83:0/2188288843 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a6f6d1e10 con 0x564a6f7a03c0 2024-04-24T08:55:33.781 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.780+0000 7f4f8f400640 1 --2- 172.21.15.83:0/449859265 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f4f7c0a3d10 0x7f4f7c0a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:33.781 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.780+0000 7f4f8f400640 1 -- 172.21.15.83:0/449859265 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f4f900d10e0 con 0x7f4f9810c3e0 2024-04-24T08:55:33.781 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.780+0000 7f4f96000640 1 --2- 172.21.15.83:0/2188288843 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564a6f7a03c0 0x564a6f7a07a0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f4f840096c0 tx=0x7f4f84009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34595f5adda1da4d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:33.785 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.780+0000 7f4f96a00640 1 --2- 172.21.15.83:0/449859265 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f4f7c0a3d10 0x7f4f7c0a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:33.785 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.780+0000 7f4f83e00640 1 -- 172.21.15.83:0/2188288843 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f84043030 con 0x564a6f7a03c0 2024-04-24T08:55:33.785 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.781+0000 7f4f83e00640 1 -- 172.21.15.83:0/2188288843 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4f840054d0 con 0x564a6f7a03c0 2024-04-24T08:55:33.785 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.781+0000 7f4f83e00640 1 -- 172.21.15.83:0/2188288843 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f84035050 con 0x564a6f7a03c0 2024-04-24T08:55:33.785 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.781+0000 7f4fa0b83740 1 -- 172.21.15.83:0/2188288843 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564a6f7a03c0 msgr2=0x564a6f7a07a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:33.786 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.781+0000 7f4f96a00640 1 --2- 172.21.15.83:0/449859265 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f4f7c0a3d10 0x7f4f7c0a61d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x564a6f7a1960 tx=0x7f4f8800b470 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:33.786 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.781+0000 7f4fa0b83740 1 --2- 172.21.15.83:0/2188288843 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564a6f7a03c0 0x564a6f7a07a0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f4f840096c0 tx=0x7f4f84009560 comp rx=0 tx=0).stop 2024-04-24T08:55:33.786 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.781+0000 7f4fa0b83740 1 -- 172.21.15.83:0/2188288843 shutdown_connections 2024-04-24T08:55:33.786 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.781+0000 7f4fa0b83740 1 --2- 172.21.15.83:0/2188288843 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564a6f7ab5a0 0x564a6f7ad990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:33.786 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.781+0000 7f4fa0b83740 1 --2- 172.21.15.83:0/2188288843 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a6f7a0ce0 0x564a6f7ab060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:33.786 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.781+0000 7f4fa0b83740 1 --2- 172.21.15.83:0/2188288843 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564a6f7a03c0 0x564a6f7a07a0 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:33.786 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.781+0000 7f4fa0b83740 1 -- 172.21.15.83:0/2188288843 >> 172.21.15.83:0/2188288843 conn(0x564a6f79c8d0 msgr2=0x564a6f79ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:33.786 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.782+0000 7f4fa0b83740 1 -- 172.21.15.83:0/2188288843 shutdown_connections 2024-04-24T08:55:33.786 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.782+0000 7f4fa0b83740 1 -- 172.21.15.83:0/2188288843 wait complete. 2024-04-24T08:55:33.786 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.782+0000 7f4fa0b83740 1 -- start start 2024-04-24T08:55:33.786 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.782+0000 7f4fa0b83740 1 Processor -- start 2024-04-24T08:55:33.786 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.783+0000 7f4fa0b83740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a6f7a03c0 0x564a6f7a8680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:33.786 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.783+0000 7f4fa0b83740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564a6f7a0ce0 0x564a6f7a8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:33.786 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.783+0000 7f4fa0b83740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564a6f7ab5a0 0x564a6f7a9100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:33.787 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.783+0000 7f4fa0b83740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x564a6f7adfc0 con 0x564a6f7ab5a0 2024-04-24T08:55:33.787 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.783+0000 7f4fa0b83740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x564a6f6c02c0 con 0x564a6f7a0ce0 2024-04-24T08:55:33.787 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.783+0000 7f4fa0b83740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x564a6f7ae140 con 0x564a6f7a03c0 2024-04-24T08:55:33.787 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.783+0000 7f4fa0b83740 10 client.0 _ll_register_callbacks cb 0x564a6f418a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:55:33.787 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.783+0000 7f4f97400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564a6f7ab5a0 0x564a6f7a9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:33.787 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.783+0000 7f4f97400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564a6f7ab5a0 0x564a6f7a9100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56716/0 (socket says 172.21.15.83:56716) 2024-04-24T08:55:33.787 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.783+0000 7f4f97400640 1 -- 172.21.15.83:0/4159099554 learned_addr learned my addr 172.21.15.83:0/4159099554 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:55:33.787 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.783+0000 7f4f96000640 1 --2- 172.21.15.83:0/4159099554 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564a6f7a0ce0 0x564a6f7a8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:33.787 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.783+0000 7f4f96a00640 1 --2- 172.21.15.83:0/4159099554 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a6f7a03c0 0x564a6f7a8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:33.787 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.784+0000 7f4f97400640 1 -- 172.21.15.83:0/4159099554 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a6f7a03c0 msgr2=0x564a6f7a8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:33.787 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.784+0000 7f4f97400640 1 --2- 172.21.15.83:0/4159099554 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a6f7a03c0 0x564a6f7a8680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:33.787 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.784+0000 7f4f97400640 1 -- 172.21.15.83:0/4159099554 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564a6f7a0ce0 msgr2=0x564a6f7a8bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:33.787 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.784+0000 7f4f97400640 1 --2- 172.21.15.83:0/4159099554 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564a6f7a0ce0 0x564a6f7a8bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:33.787 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.784+0000 7f4f97400640 1 -- 172.21.15.83:0/4159099554 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a6f6d0fd0 con 0x564a6f7ab5a0 2024-04-24T08:55:33.788 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.784+0000 7f4f96000640 1 --2- 172.21.15.83:0/4159099554 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564a6f7a0ce0 0x564a6f7a8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:55:33.788 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.784+0000 7f4f97400640 1 --2- 172.21.15.83:0/4159099554 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564a6f7ab5a0 0x564a6f7a9100 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f4f900990b0 tx=0x7f4f9009c670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc686fece7796ff5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:33.788 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.784+0000 7f4f80c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:55:33.788 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.784+0000 7f4f80c00640 1 -- 172.21.15.83:0/4159099554 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f900b6060 con 0x564a6f7ab5a0 2024-04-24T08:55:33.788 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.784+0000 7f4f80c00640 1 -- 172.21.15.83:0/4159099554 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f4f9009ceb0 con 0x564a6f7ab5a0 2024-04-24T08:55:33.788 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.784+0000 7f4fa0b83740 1 -- 172.21.15.83:0/4159099554 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f840054d0 con 0x564a6f7ab5a0 2024-04-24T08:55:33.788 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.785+0000 7f4f80c00640 1 -- 172.21.15.83:0/4159099554 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4f900bd2b0 con 0x564a6f7ab5a0 2024-04-24T08:55:33.788 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.785+0000 7f4fa0b83740 10 client.17043 fetch_fsmap learned FSMap version 30 2024-04-24T08:55:33.788 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.785+0000 7f4fa0b83740 1 -- 172.21.15.83:0/4159099554 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564a6f419210 con 0x564a6f7ab5a0 2024-04-24T08:55:33.788 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.785+0000 7f4f80c00640 1 -- 172.21.15.83:0/4159099554 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4f840054d0 con 0x564a6f7ab5a0 2024-04-24T08:55:33.788 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.785+0000 7f4f80c00640 1 -- 172.21.15.83:0/4159099554 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7f4f900a5430 con 0x564a6f7ab5a0 2024-04-24T08:55:33.788 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.785+0000 7f4fa0b83740 10 client.17043 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T08:55:33.788 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.785+0000 7f4fa0b83740 10 client.17043 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T08:55:33.788 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.785+0000 7f4fa0b83740 20 client.17043 populate_metadata read hostname 'smithi083' 2024-04-24T08:55:33.789 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.785+0000 7f4fa0b83740 1 --2- 172.21.15.83:0/4159099554 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x564a6f7bb750 0x564a6f7bdb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:33.789 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.785+0000 7f4fa0b83740 4 client.17043 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:55:33.789 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.785+0000 7f4fa0b83740 1 -- 172.21.15.83:0/4159099554 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564a6f714320 con 0x564a6f7bb750 2024-04-24T08:55:33.789 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.786+0000 7f4f96a00640 1 --2- 172.21.15.83:0/4159099554 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x564a6f7bb750 0x564a6f7bdb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:33.790 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.787+0000 7f4f80c00640 10 client.17043 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:55:33.790 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.787+0000 7f4f96a00640 1 --2- 172.21.15.83:0/4159099554 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x564a6f7bb750 0x564a6f7bdb40 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b2a5c0c67948c server_cookie=5919e6e802a6d90e in_seq=0 out_seq=0 2024-04-24T08:55:33.790 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.789+0000 7f4f80c00640 1 -- 172.21.15.83:0/4159099554 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564a6f714320 con 0x564a6f7bb750 2024-04-24T08:55:33.790 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.789+0000 7f4f80c00640 10 client.17043 handle_command_reply: tid=0 2024-04-24T08:55:33.790 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.790+0000 7f4fa0b83740 1 client.17043 shutdown 2024-04-24T08:55:33.791 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.790+0000 7f4fa0b83740 1 -- 172.21.15.83:0/4159099554 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564a6f7ab5a0 msgr2=0x564a6f7a9100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:33.791 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.790+0000 7f4fa0b83740 1 --2- 172.21.15.83:0/4159099554 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564a6f7ab5a0 0x564a6f7a9100 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f4f900990b0 tx=0x7f4f9009c670 comp rx=0 tx=0).stop 2024-04-24T08:55:33.791 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.790+0000 7f4fa0b83740 1 -- 172.21.15.83:0/4159099554 shutdown_connections 2024-04-24T08:55:33.791 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.790+0000 7f4fa0b83740 1 --2- 172.21.15.83:0/4159099554 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x564a6f7bb750 0x564a6f7bdb40 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:33.791 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.790+0000 7f4fa0b83740 1 --2- 172.21.15.83:0/4159099554 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564a6f7ab5a0 0x564a6f7a9100 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:33.791 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.790+0000 7f4fa0b83740 1 --2- 172.21.15.83:0/4159099554 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564a6f7a0ce0 0x564a6f7a8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:33.791 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.790+0000 7f4fa0b83740 1 --2- 172.21.15.83:0/4159099554 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a6f7a03c0 0x564a6f7a8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:33.791 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.790+0000 7f4f80c00640 0 client.17043 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:55:33.791 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.790+0000 7f4fa0b83740 1 -- 172.21.15.83:0/4159099554 >> 172.21.15.83:0/4159099554 conn(0x564a6f79c8d0 msgr2=0x564a6f7a2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:33.791 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.790+0000 7f4fa0b83740 1 -- 172.21.15.83:0/4159099554 shutdown_connections 2024-04-24T08:55:33.791 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.790+0000 7f4fa0b83740 1 -- 172.21.15.83:0/4159099554 wait complete. 2024-04-24T08:55:33.791 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.790+0000 7f4fa0b83740 20 client.17043 trim_cache size 0 max 16384 2024-04-24T08:55:33.808 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.807+0000 7f4fa0b83740 1 Processor -- start 2024-04-24T08:55:33.808 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.807+0000 7f4fa0b83740 1 -- start start 2024-04-24T08:55:33.808 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.808+0000 7f4fa0b83740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564a6f7a03c0 0x564a6f7b6f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:33.808 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.808+0000 7f4fa0b83740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564a6f7a0ce0 0x564a6f79c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:33.808 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.808+0000 7f4fa0b83740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a6f7ab5a0 0x564a6f7c3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:33.808 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.808+0000 7f4fa0b83740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4f981b1250 con 0x564a6f7a0ce0 2024-04-24T08:55:33.808 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.808+0000 7f4fa0b83740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4f981b2e20 con 0x564a6f7a03c0 2024-04-24T08:55:33.808 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.808+0000 7f4fa0b83740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4f981b1920 con 0x564a6f7ab5a0 2024-04-24T08:55:33.810 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.808+0000 7f4f96000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564a6f7a0ce0 0x564a6f79c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:33.810 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.808+0000 7f4f96000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564a6f7a0ce0 0x564a6f79c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56724/0 (socket says 172.21.15.83:56724) 2024-04-24T08:55:33.810 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.808+0000 7f4f96000640 1 -- 172.21.15.83:0/1714292471 learned_addr learned my addr 172.21.15.83:0/1714292471 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:55:33.810 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.808+0000 7f4f96a00640 1 --2- 172.21.15.83:0/1714292471 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564a6f7a03c0 0x564a6f7b6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:33.810 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.808+0000 7f4f97400640 1 --2- 172.21.15.83:0/1714292471 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a6f7ab5a0 0x564a6f7c3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:33.810 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.809+0000 7f4f96000640 1 -- 172.21.15.83:0/1714292471 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a6f7ab5a0 msgr2=0x564a6f7c3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:33.810 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.809+0000 7f4f96000640 1 --2- 172.21.15.83:0/1714292471 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a6f7ab5a0 0x564a6f7c3450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:33.810 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.809+0000 7f4f96000640 1 -- 172.21.15.83:0/1714292471 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564a6f7a03c0 msgr2=0x564a6f7b6f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:33.810 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.809+0000 7f4f96000640 1 --2- 172.21.15.83:0/1714292471 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564a6f7a03c0 0x564a6f7b6f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:33.810 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.809+0000 7f4f96000640 1 -- 172.21.15.83:0/1714292471 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a6f7a1790 con 0x564a6f7a0ce0 2024-04-24T08:55:33.811 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.809+0000 7f4f97400640 1 --2- 172.21.15.83:0/1714292471 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a6f7ab5a0 0x564a6f7c3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:55:33.811 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.809+0000 7f4f96a00640 1 --2- 172.21.15.83:0/1714292471 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564a6f7a03c0 0x564a6f7b6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:55:33.811 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.809+0000 7f4f96000640 1 --2- 172.21.15.83:0/1714292471 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564a6f7a0ce0 0x564a6f79c210 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f4f84000c00 tx=0x7f4f84005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18a53265a88b1e6b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:33.811 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.810+0000 7f4f83400640 1 -- 172.21.15.83:0/1714292471 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f8404c070 con 0x564a6f7a0ce0 2024-04-24T08:55:33.811 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.810+0000 7f4f83400640 1 -- 172.21.15.83:0/1714292471 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4f84044040 con 0x564a6f7a0ce0 2024-04-24T08:55:33.811 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.810+0000 7f4f83400640 1 -- 172.21.15.83:0/1714292471 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f84035030 con 0x564a6f7a0ce0 2024-04-24T08:55:33.814 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.810+0000 7f4fa0b83740 1 -- 172.21.15.83:0/1714292471 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564a6f7a0ce0 msgr2=0x564a6f79c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:33.814 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.810+0000 7f4fa0b83740 1 --2- 172.21.15.83:0/1714292471 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564a6f7a0ce0 0x564a6f79c210 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f4f84000c00 tx=0x7f4f84005fd0 comp rx=0 tx=0).stop 2024-04-24T08:55:33.814 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.810+0000 7f4fa0b83740 1 -- 172.21.15.83:0/1714292471 shutdown_connections 2024-04-24T08:55:33.814 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.810+0000 7f4fa0b83740 1 --2- 172.21.15.83:0/1714292471 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a6f7ab5a0 0x564a6f7c3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:33.814 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.810+0000 7f4fa0b83740 1 --2- 172.21.15.83:0/1714292471 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564a6f7a0ce0 0x564a6f79c210 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:33.814 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.810+0000 7f4fa0b83740 1 --2- 172.21.15.83:0/1714292471 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564a6f7a03c0 0x564a6f7b6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:33.814 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.810+0000 7f4fa0b83740 1 -- 172.21.15.83:0/1714292471 >> 172.21.15.83:0/1714292471 conn(0x564a6f79c8d0 msgr2=0x564a6f7a3960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:33.814 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.810+0000 7f4fa0b83740 1 -- 172.21.15.83:0/1714292471 shutdown_connections 2024-04-24T08:55:33.814 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.810+0000 7f4fa0b83740 1 -- 172.21.15.83:0/1714292471 wait complete. 2024-04-24T08:55:33.815 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.811+0000 7f4fa0b83740 1 -- start start 2024-04-24T08:55:33.815 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.811+0000 7f4fa0b83740 1 Processor -- start 2024-04-24T08:55:33.815 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.812+0000 7f4fa0b83740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564a6f7a03c0 0x564a6f7b3850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:33.815 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.812+0000 7f4fa0b83740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a6f7a0ce0 0x564a6f7a5080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:33.815 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.812+0000 7f4fa0b83740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564a6f7ab5a0 0x564a6f7a55c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:33.815 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.812+0000 7f4fa0b83740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4f981b26c0 con 0x564a6f7a03c0 2024-04-24T08:55:33.815 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.812+0000 7f4fa0b83740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4f981b1ff0 con 0x564a6f7ab5a0 2024-04-24T08:55:33.815 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.812+0000 7f4fa0b83740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4f981af230 con 0x564a6f7a0ce0 2024-04-24T08:55:33.815 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.812+0000 7f4fa0b83740 10 client.0 _ll_register_callbacks cb 0x7f4f900185b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:55:33.815 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.812+0000 7f4f96a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564a6f7a03c0 0x564a6f7b3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:33.815 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.812+0000 7f4f96a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564a6f7a03c0 0x564a6f7b3850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56736/0 (socket says 172.21.15.83:56736) 2024-04-24T08:55:33.815 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.812+0000 7f4f96a00640 1 -- 172.21.15.83:0/757723984 learned_addr learned my addr 172.21.15.83:0/757723984 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:55:33.815 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.812+0000 7f4f97400640 1 --2- 172.21.15.83:0/757723984 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564a6f7ab5a0 0x564a6f7a55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:33.816 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.812+0000 7f4f96000640 1 --2- 172.21.15.83:0/757723984 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a6f7a0ce0 0x564a6f7a5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:33.816 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.812+0000 7f4f96a00640 1 -- 172.21.15.83:0/757723984 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a6f7a0ce0 msgr2=0x564a6f7a5080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:33.816 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.812+0000 7f4f96a00640 1 --2- 172.21.15.83:0/757723984 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a6f7a0ce0 0x564a6f7a5080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:33.816 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.812+0000 7f4f96a00640 1 -- 172.21.15.83:0/757723984 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564a6f7ab5a0 msgr2=0x564a6f7a55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:33.816 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.812+0000 7f4f96a00640 1 --2- 172.21.15.83:0/757723984 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564a6f7ab5a0 0x564a6f7a55c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:33.816 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.812+0000 7f4f96a00640 1 -- 172.21.15.83:0/757723984 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a6f4e5620 con 0x564a6f7a03c0 2024-04-24T08:55:33.816 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.812+0000 7f4f97400640 1 --2- 172.21.15.83:0/757723984 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564a6f7ab5a0 0x564a6f7a55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:55:33.816 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.813+0000 7f4f96a00640 1 --2- 172.21.15.83:0/757723984 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564a6f7a03c0 0x564a6f7b3850 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f4f8800bb00 tx=0x7f4f880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e64d717e337a06ca server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:33.816 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.813+0000 7f4f80c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:55:33.816 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.813+0000 7f4f80c00640 1 -- 172.21.15.83:0/757723984 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f880257f0 con 0x564a6f7a03c0 2024-04-24T08:55:33.816 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.813+0000 7f4fa0b83740 1 -- 172.21.15.83:0/757723984 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f84044040 con 0x564a6f7a03c0 2024-04-24T08:55:33.816 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.813+0000 7f4f80c00640 1 -- 172.21.15.83:0/757723984 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f4f8801d7c0 con 0x564a6f7a03c0 2024-04-24T08:55:33.816 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.813+0000 7f4f80c00640 1 -- 172.21.15.83:0/757723984 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4f88037050 con 0x564a6f7a03c0 2024-04-24T08:55:33.816 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.813+0000 7f4fa0b83740 10 client.17049 fetch_fsmap learned FSMap version 30 2024-04-24T08:55:33.816 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.813+0000 7f4fa0b83740 1 -- 172.21.15.83:0/757723984 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564a6f821bc0 con 0x564a6f7a03c0 2024-04-24T08:55:33.817 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.814+0000 7f4f80c00640 1 -- 172.21.15.83:0/757723984 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4f84044040 con 0x564a6f7a03c0 2024-04-24T08:55:33.817 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.814+0000 7f4f80c00640 1 -- 172.21.15.83:0/757723984 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7f4f88030a30 con 0x564a6f7a03c0 2024-04-24T08:55:33.817 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.814+0000 7f4fa0b83740 10 client.17049 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T08:55:33.817 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.814+0000 7f4fa0b83740 10 client.17049 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T08:55:33.817 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.814+0000 7f4fa0b83740 20 client.17049 populate_metadata read hostname 'smithi083' 2024-04-24T08:55:33.817 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.814+0000 7f4fa0b83740 1 --2- 172.21.15.83:0/757723984 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x564a6f822370 0x564a6f8247d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:33.817 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.814+0000 7f4fa0b83740 4 client.17049 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T08:55:33.817 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.814+0000 7f4fa0b83740 1 -- 172.21.15.83:0/757723984 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564a6f70e190 con 0x564a6f822370 2024-04-24T08:55:33.817 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.815+0000 7f4f96000640 1 --2- 172.21.15.83:0/757723984 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x564a6f822370 0x564a6f8247d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:33.818 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.815+0000 7f4f96000640 1 --2- 172.21.15.83:0/757723984 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x564a6f822370 0x564a6f8247d0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ebc95a1e31552aaa server_cookie=c2b33cdd7ff848bb in_seq=0 out_seq=0 2024-04-24T08:55:33.818 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.815+0000 7f4f80c00640 10 client.17049 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:55:33.874 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.874+0000 7f4f80c00640 1 -- 172.21.15.83:0/757723984 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564a6f70e190 con 0x564a6f822370 2024-04-24T08:55:33.874 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.874+0000 7f4f80c00640 10 client.17049 handle_command_reply: tid=0 2024-04-24T08:55:33.874 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.874+0000 7f4fa0b83740 1 client.17049 shutdown 2024-04-24T08:55:33.875 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.874+0000 7f4fa0b83740 1 -- 172.21.15.83:0/757723984 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564a6f7a03c0 msgr2=0x564a6f7b3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:33.875 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.874+0000 7f4fa0b83740 1 --2- 172.21.15.83:0/757723984 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564a6f7a03c0 0x564a6f7b3850 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f4f8800bb00 tx=0x7f4f880046c0 comp rx=0 tx=0).stop 2024-04-24T08:55:33.875 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.874+0000 7f4fa0b83740 1 -- 172.21.15.83:0/757723984 shutdown_connections 2024-04-24T08:55:33.876 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.874+0000 7f4fa0b83740 1 --2- 172.21.15.83:0/757723984 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x564a6f822370 0x564a6f8247d0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:33.876 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.874+0000 7f4fa0b83740 1 --2- 172.21.15.83:0/757723984 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564a6f7ab5a0 0x564a6f7a55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:33.876 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.875+0000 7f4fa0b83740 1 --2- 172.21.15.83:0/757723984 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a6f7a0ce0 0x564a6f7a5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:33.876 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.875+0000 7f4fa0b83740 1 --2- 172.21.15.83:0/757723984 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564a6f7a03c0 0x564a6f7b3850 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:33.876 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.875+0000 7f4fa0b83740 1 -- 172.21.15.83:0/757723984 >> 172.21.15.83:0/757723984 conn(0x564a6f79c8d0 msgr2=0x564a6f7a8640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:33.876 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.875+0000 7f4f80c00640 0 client.17049 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:55:33.876 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.875+0000 7f4fa0b83740 1 -- 172.21.15.83:0/757723984 shutdown_connections 2024-04-24T08:55:33.876 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.875+0000 7f4fa0b83740 1 -- 172.21.15.83:0/757723984 wait complete. 2024-04-24T08:55:33.876 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.875+0000 7f4fa0b83740 20 client.17049 trim_cache size 0 max 16384 2024-04-24T08:55:33.876 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T08:55:33.876 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T08:55:33.877 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T08:55:33.877 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T08:55:33.877 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.876+0000 7f4f97e00640 1 -- 172.21.15.83:0/449859265 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f4f7c0a3d10 msgr2=0x7f4f7c0a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:33.878 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.876+0000 7f4f97e00640 1 --2- 172.21.15.83:0/449859265 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f4f7c0a3d10 0x7f4f7c0a61d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x564a6f7a1960 tx=0x7f4f8800b470 comp rx=0 tx=0).stop 2024-04-24T08:55:33.878 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.876+0000 7f4f97e00640 1 -- 172.21.15.83:0/449859265 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f9810c3e0 msgr2=0x7f4f9819a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:33.878 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.876+0000 7f4f97e00640 1 --2- 172.21.15.83:0/449859265 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f9810c3e0 0x7f4f9819a5e0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f4f9000cce0 tx=0x7f4f9000a620 comp rx=0 tx=0).stop 2024-04-24T08:55:33.878 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.876+0000 7f4f97e00640 1 -- 172.21.15.83:0/449859265 shutdown_connections 2024-04-24T08:55:33.878 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.876+0000 7f4f97e00640 1 --2- 172.21.15.83:0/449859265 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f4f7c0a3d10 0x7f4f7c0a61d0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:33.878 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.876+0000 7f4f97e00640 1 --2- 172.21.15.83:0/449859265 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f9810c3e0 0x7f4f9819a5e0 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:33.878 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.876+0000 7f4f97e00640 1 --2- 172.21.15.83:0/449859265 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4f981038e0 0x7f4f981a0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:33.878 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.876+0000 7f4f97e00640 1 --2- 172.21.15.83:0/449859265 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4f98102fc0 0x7f4f981a0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:33.878 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.876+0000 7f4f97e00640 1 -- 172.21.15.83:0/449859265 >> 172.21.15.83:0/449859265 conn(0x7f4f980fc820 msgr2=0x7f4f980fcc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:33.878 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.877+0000 7f4f97e00640 1 -- 172.21.15.83:0/449859265 shutdown_connections 2024-04-24T08:55:33.878 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:33.877+0000 7f4f97e00640 1 -- 172.21.15.83:0/449859265 wait complete. 2024-04-24T08:55:34.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:55:33 smithi083 ceph-mon[27907]: pgmap v121: 129 pgs: 129 active+clean; 31 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 502 KiB/s wr, 76 op/s 2024-04-24T08:55:34.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:55:33 smithi138 ceph-mon[31441]: pgmap v121: 129 pgs: 129 active+clean; 31 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 502 KiB/s wr, 76 op/s 2024-04-24T08:55:34.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:55:33 smithi189 ceph-mon[31791]: pgmap v121: 129 pgs: 129 active+clean; 31 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 502 KiB/s wr, 76 op/s 2024-04-24T08:55:34.568 INFO:tasks.workunit.client.0.smithi083.stdout: 1 15790 0.65 MB/sec execute 8 sec latency 47.881 ms 2024-04-24T08:55:35.568 INFO:tasks.workunit.client.0.smithi083.stdout: 1 15907 0.58 MB/sec execute 9 sec latency 53.345 ms 2024-04-24T08:55:36.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:55:35 smithi083 ceph-mon[27907]: pgmap v122: 129 pgs: 129 active+clean; 31 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 245 KiB/s wr, 50 op/s 2024-04-24T08:55:36.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:55:35 smithi138 ceph-mon[31441]: pgmap v122: 129 pgs: 129 active+clean; 31 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 245 KiB/s wr, 50 op/s 2024-04-24T08:55:36.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:55:35 smithi189 ceph-mon[31791]: pgmap v122: 129 pgs: 129 active+clean; 31 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 245 KiB/s wr, 50 op/s 2024-04-24T08:55:36.568 INFO:tasks.workunit.client.0.smithi083.stdout: 1 16053 0.54 MB/sec execute 10 sec latency 58.189 ms 2024-04-24T08:55:37.568 INFO:tasks.workunit.client.0.smithi083.stdout: 1 16187 0.50 MB/sec execute 11 sec latency 57.122 ms 2024-04-24T08:55:38.256 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:55:38 smithi138 ceph-mon[31441]: pgmap v123: 129 pgs: 129 active+clean; 31 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 245 KiB/s wr, 50 op/s 2024-04-24T08:55:38.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:55:38 smithi083 ceph-mon[27907]: pgmap v123: 129 pgs: 129 active+clean; 31 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 245 KiB/s wr, 50 op/s 2024-04-24T08:55:38.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:55:38 smithi189 ceph-mon[31791]: pgmap v123: 129 pgs: 129 active+clean; 31 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 245 KiB/s wr, 50 op/s 2024-04-24T08:55:38.569 INFO:tasks.workunit.client.0.smithi083.stdout: 1 16329 0.48 MB/sec execute 12 sec latency 53.366 ms 2024-04-24T08:55:39.073 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T08:55:39.073 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T08:55:39.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.297+0000 7f381a800640 1 -- 172.21.15.83:0/1982995016 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f381410d500 msgr2=0x7f381410d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:39.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.297+0000 7f381a800640 1 --2- 172.21.15.83:0/1982995016 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f381410d500 0x7f381410d8e0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f3808009930 tx=0x7f380802a420 comp rx=0 tx=0).stop 2024-04-24T08:55:39.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.297+0000 7f381a800640 1 -- 172.21.15.83:0/1982995016 shutdown_connections 2024-04-24T08:55:39.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.297+0000 7f381a800640 1 --2- 172.21.15.83:0/1982995016 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3814108f40 0x7f38141153b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:39.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.297+0000 7f381a800640 1 --2- 172.21.15.83:0/1982995016 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3814108580 0x7f3814108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:39.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.297+0000 7f381a800640 1 --2- 172.21.15.83:0/1982995016 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f381410d500 0x7f381410d8e0 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:39.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.297+0000 7f381a800640 1 -- 172.21.15.83:0/1982995016 >> 172.21.15.83:0/1982995016 conn(0x7f3814077e20 msgr2=0x7f3814106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:39.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.297+0000 7f381a800640 1 -- 172.21.15.83:0/1982995016 shutdown_connections 2024-04-24T08:55:39.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.297+0000 7f381a800640 1 -- 172.21.15.83:0/1982995016 wait complete. 2024-04-24T08:55:39.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.298+0000 7f381a800640 1 Processor -- start 2024-04-24T08:55:39.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.298+0000 7f381a800640 1 -- start start 2024-04-24T08:55:39.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.299+0000 7f381a800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3814108580 0x7f38141a4a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:39.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.299+0000 7f381a800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3814108f40 0x7f38141a4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:39.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.299+0000 7f381a800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f381410d500 0x7f381419eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:39.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.299+0000 7f381a800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f38141187b0 con 0x7f3814108f40 2024-04-24T08:55:39.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.299+0000 7f381a800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3814118630 con 0x7f381410d500 2024-04-24T08:55:39.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.299+0000 7f381a800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3814118930 con 0x7f3814108580 2024-04-24T08:55:39.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.299+0000 7f3818a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3814108f40 0x7f38141a4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:39.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.299+0000 7f3818a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3814108f40 0x7f38141a4fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55282/0 (socket says 172.21.15.83:55282) 2024-04-24T08:55:39.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.299+0000 7f3818a00640 1 -- 172.21.15.83:0/3048936832 learned_addr learned my addr 172.21.15.83:0/3048936832 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:55:39.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.300+0000 7f3819400640 1 --2- 172.21.15.83:0/3048936832 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3814108580 0x7f38141a4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:39.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.300+0000 7f3819e00640 1 --2- 172.21.15.83:0/3048936832 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f381410d500 0x7f381419eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:39.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.300+0000 7f3818a00640 1 -- 172.21.15.83:0/3048936832 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3814108580 msgr2=0x7f38141a4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:39.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.300+0000 7f3818a00640 1 --2- 172.21.15.83:0/3048936832 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3814108580 0x7f38141a4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:39.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.300+0000 7f3818a00640 1 -- 172.21.15.83:0/3048936832 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f381410d500 msgr2=0x7f381419eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:39.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.300+0000 7f3818a00640 1 --2- 172.21.15.83:0/3048936832 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f381410d500 0x7f381419eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:39.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.300+0000 7f3818a00640 1 -- 172.21.15.83:0/3048936832 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f381419f430 con 0x7f3814108f40 2024-04-24T08:55:39.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.300+0000 7f3818a00640 1 --2- 172.21.15.83:0/3048936832 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3814108f40 0x7f38141a4fa0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f380000d950 tx=0x7f380000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:39.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.300+0000 7f3812000640 1 -- 172.21.15.83:0/3048936832 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3800017030 con 0x7f3814108f40 2024-04-24T08:55:39.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.300+0000 7f3812000640 1 -- 172.21.15.83:0/3048936832 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3800004020 con 0x7f3814108f40 2024-04-24T08:55:39.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.300+0000 7f3812000640 1 -- 172.21.15.83:0/3048936832 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38000047a0 con 0x7f3814108f40 2024-04-24T08:55:39.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.300+0000 7f381a800640 1 -- 172.21.15.83:0/3048936832 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f381419f720 con 0x7f3814108f40 2024-04-24T08:55:39.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.301+0000 7f381a800640 1 -- 172.21.15.83:0/3048936832 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38141ab9e0 con 0x7f3814108f40 2024-04-24T08:55:39.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.304+0000 7f381ec08740 1 Processor -- start 2024-04-24T08:55:39.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.304+0000 7f3812000640 1 -- 172.21.15.83:0/3048936832 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f38000041d0 con 0x7f3814108f40 2024-04-24T08:55:39.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.304+0000 7f381ec08740 1 -- start start 2024-04-24T08:55:39.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.304+0000 7f381ec08740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ae2e04a30 0x558ae2e04e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:39.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.304+0000 7f381ec08740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558ae2e05350 0x558ae2e0f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:39.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.305+0000 7f381ec08740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558ae2e0fa20 0x558ae2e11e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:39.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.305+0000 7f381ec08740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x558ae2d6a1f0 con 0x558ae2e05350 2024-04-24T08:55:39.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.305+0000 7f381ec08740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558ae2d10350 con 0x558ae2e04a30 2024-04-24T08:55:39.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.305+0000 7f381ec08740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x558ae2d661b0 con 0x558ae2e0fa20 2024-04-24T08:55:39.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.305+0000 7f3819400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558ae2e05350 0x558ae2e0f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:39.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.305+0000 7f3819400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558ae2e05350 0x558ae2e0f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55294/0 (socket says 172.21.15.83:55294) 2024-04-24T08:55:39.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.305+0000 7f3819400640 1 -- 172.21.15.83:0/461606112 learned_addr learned my addr 172.21.15.83:0/461606112 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:55:39.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.305+0000 7f3819e00640 1 --2- 172.21.15.83:0/461606112 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ae2e04a30 0x558ae2e04e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:39.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.305+0000 7f3818a00640 1 --2- 172.21.15.83:0/461606112 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558ae2e0fa20 0x558ae2e11e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:39.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.306+0000 7f3819400640 1 -- 172.21.15.83:0/461606112 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558ae2e0fa20 msgr2=0x558ae2e11e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:39.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.306+0000 7f3819400640 1 --2- 172.21.15.83:0/461606112 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558ae2e0fa20 0x558ae2e11e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:39.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.306+0000 7f3819400640 1 -- 172.21.15.83:0/461606112 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ae2e04a30 msgr2=0x558ae2e04e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:39.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.306+0000 7f3819400640 1 --2- 172.21.15.83:0/461606112 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ae2e04a30 0x558ae2e04e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:39.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.306+0000 7f3819400640 1 -- 172.21.15.83:0/461606112 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ae2d64f50 con 0x558ae2e05350 2024-04-24T08:55:39.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.306+0000 7f3819400640 1 --2- 172.21.15.83:0/461606112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558ae2e05350 0x558ae2e0f4e0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f3808005ab0 tx=0x7f3808004170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48158d7ddda36f50 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:39.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.306+0000 7f3812000640 1 --2- 172.21.15.83:0/3048936832 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f37fc0a3d20 0x7f37fc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:39.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.306+0000 7f3806600640 1 -- 172.21.15.83:0/461606112 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3808035030 con 0x558ae2e05350 2024-04-24T08:55:39.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.306+0000 7f3806600640 1 -- 172.21.15.83:0/461606112 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f38080054f0 con 0x558ae2e05350 2024-04-24T08:55:39.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.306+0000 7f3806600640 1 -- 172.21.15.83:0/461606112 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38080035f0 con 0x558ae2e05350 2024-04-24T08:55:39.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.306+0000 7f3812000640 1 -- 172.21.15.83:0/3048936832 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f38000d0760 con 0x7f3814108f40 2024-04-24T08:55:39.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.306+0000 7f3819e00640 1 --2- 172.21.15.83:0/3048936832 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f37fc0a3d20 0x7f37fc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:39.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.307+0000 7f381ec08740 1 -- 172.21.15.83:0/461606112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558ae2e05350 msgr2=0x558ae2e0f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:39.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.307+0000 7f381ec08740 1 --2- 172.21.15.83:0/461606112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558ae2e05350 0x558ae2e0f4e0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f3808005ab0 tx=0x7f3808004170 comp rx=0 tx=0).stop 2024-04-24T08:55:39.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.307+0000 7f3819e00640 1 --2- 172.21.15.83:0/3048936832 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f37fc0a3d20 0x7f37fc0a61e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f380c005fd0 tx=0x7f380c0096d0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:39.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.307+0000 7f381ec08740 1 -- 172.21.15.83:0/461606112 shutdown_connections 2024-04-24T08:55:39.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.307+0000 7f381ec08740 1 --2- 172.21.15.83:0/461606112 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558ae2e0fa20 0x558ae2e11e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:39.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.307+0000 7f381ec08740 1 --2- 172.21.15.83:0/461606112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558ae2e05350 0x558ae2e0f4e0 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:39.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.307+0000 7f381ec08740 1 --2- 172.21.15.83:0/461606112 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ae2e04a30 0x558ae2e04e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:39.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.307+0000 7f381ec08740 1 -- 172.21.15.83:0/461606112 >> 172.21.15.83:0/461606112 conn(0x558ae2e00d50 msgr2=0x558ae2e01130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:39.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.307+0000 7f381ec08740 1 -- 172.21.15.83:0/461606112 shutdown_connections 2024-04-24T08:55:39.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.307+0000 7f381ec08740 1 -- 172.21.15.83:0/461606112 wait complete. 2024-04-24T08:55:39.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.308+0000 7f381ec08740 1 -- start start 2024-04-24T08:55:39.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.308+0000 7f381ec08740 1 Processor -- start 2024-04-24T08:55:39.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.309+0000 7f381ec08740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558ae2e04a30 0x558ae2e0ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:39.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.309+0000 7f381ec08740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558ae2e05350 0x558ae2e0cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:39.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.309+0000 7f381ec08740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ae2e0fa20 0x558ae2e0d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:39.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.309+0000 7f381ec08740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x558ae2e127b0 con 0x558ae2e04a30 2024-04-24T08:55:39.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.309+0000 7f381ec08740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558ae2d23690 con 0x558ae2e0fa20 2024-04-24T08:55:39.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.309+0000 7f381ec08740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x558ae2e12930 con 0x558ae2e05350 2024-04-24T08:55:39.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.309+0000 7f381ec08740 10 client.0 _ll_register_callbacks cb 0x558ae2d5f650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:55:39.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.309+0000 7f3819400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558ae2e04a30 0x558ae2e0ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:39.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.309+0000 7f3819400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558ae2e04a30 0x558ae2e0ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55302/0 (socket says 172.21.15.83:55302) 2024-04-24T08:55:39.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.309+0000 7f3819400640 1 -- 172.21.15.83:0/921062987 learned_addr learned my addr 172.21.15.83:0/921062987 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:55:39.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.309+0000 7f3819400640 1 -- 172.21.15.83:0/921062987 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558ae2e05350 msgr2=0x558ae2e0cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:39.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.309+0000 7f3819400640 1 --2- 172.21.15.83:0/921062987 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558ae2e05350 0x558ae2e0cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:39.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.309+0000 7f3819400640 1 -- 172.21.15.83:0/921062987 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ae2e0fa20 msgr2=0x558ae2e0d480 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-24T08:55:39.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.309+0000 7f3819400640 1 --2- 172.21.15.83:0/921062987 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ae2e0fa20 0x558ae2e0d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:39.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.309+0000 7f3819400640 1 -- 172.21.15.83:0/921062987 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ae2d22920 con 0x558ae2e04a30 2024-04-24T08:55:39.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.310+0000 7f3819400640 1 --2- 172.21.15.83:0/921062987 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558ae2e04a30 0x558ae2e0ca00 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f3808000c00 tx=0x7f38080048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be1ab1c21cb0b9ea server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:39.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.310+0000 7f37faa00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:55:39.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.310+0000 7f37faa00640 1 -- 172.21.15.83:0/921062987 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3808051060 con 0x558ae2e04a30 2024-04-24T08:55:39.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.310+0000 7f37faa00640 1 -- 172.21.15.83:0/921062987 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f3808005100 con 0x558ae2e04a30 2024-04-24T08:55:39.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.310+0000 7f381ec08740 1 -- 172.21.15.83:0/921062987 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38080054f0 con 0x558ae2e04a30 2024-04-24T08:55:39.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.310+0000 7f37faa00640 1 -- 172.21.15.83:0/921062987 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3808058050 con 0x558ae2e04a30 2024-04-24T08:55:39.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.310+0000 7f381ec08740 10 client.17061 fetch_fsmap learned FSMap version 30 2024-04-24T08:55:39.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.310+0000 7f381ec08740 1 -- 172.21.15.83:0/921062987 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558ae2b467d0 con 0x558ae2e04a30 2024-04-24T08:55:39.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.311+0000 7f37faa00640 1 -- 172.21.15.83:0/921062987 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f38080054f0 con 0x558ae2e04a30 2024-04-24T08:55:39.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.311+0000 7f37faa00640 1 -- 172.21.15.83:0/921062987 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7f380804b030 con 0x558ae2e04a30 2024-04-24T08:55:39.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.311+0000 7f381ec08740 10 client.17061 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T08:55:39.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.311+0000 7f381ec08740 10 client.17061 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:55:39.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.311+0000 7f381ec08740 20 client.17061 populate_metadata read hostname 'smithi083' 2024-04-24T08:55:39.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.311+0000 7f381ec08740 1 --2- 172.21.15.83:0/921062987 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x558ae2e1fc50 0x558ae2e22040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:39.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.311+0000 7f381ec08740 4 client.17061 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:55:39.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.311+0000 7f381ec08740 1 -- 172.21.15.83:0/921062987 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558ae2d77c10 con 0x558ae2e1fc50 2024-04-24T08:55:39.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.312+0000 7f3818a00640 1 --2- 172.21.15.83:0/921062987 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x558ae2e1fc50 0x558ae2e22040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:39.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.313+0000 7f3818a00640 1 --2- 172.21.15.83:0/921062987 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x558ae2e1fc50 0x558ae2e22040 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee7385a10ac9ca44 server_cookie=7fde1ad19def021c in_seq=0 out_seq=0 2024-04-24T08:55:39.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.313+0000 7f37faa00640 10 client.17061 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:55:39.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.314+0000 7f37faa00640 1 -- 172.21.15.83:0/921062987 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558ae2d77c10 con 0x558ae2e1fc50 2024-04-24T08:55:39.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.314+0000 7f37faa00640 10 client.17061 handle_command_reply: tid=0 2024-04-24T08:55:39.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.315+0000 7f381ec08740 1 client.17061 shutdown 2024-04-24T08:55:39.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.315+0000 7f381ec08740 1 -- 172.21.15.83:0/921062987 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558ae2e04a30 msgr2=0x558ae2e0ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:39.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.315+0000 7f381ec08740 1 --2- 172.21.15.83:0/921062987 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558ae2e04a30 0x558ae2e0ca00 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f3808000c00 tx=0x7f38080048f0 comp rx=0 tx=0).stop 2024-04-24T08:55:39.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.315+0000 7f381ec08740 1 -- 172.21.15.83:0/921062987 shutdown_connections 2024-04-24T08:55:39.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.315+0000 7f381ec08740 1 --2- 172.21.15.83:0/921062987 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x558ae2e1fc50 0x558ae2e22040 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:39.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.315+0000 7f381ec08740 1 --2- 172.21.15.83:0/921062987 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ae2e0fa20 0x558ae2e0d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:39.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.315+0000 7f381ec08740 1 --2- 172.21.15.83:0/921062987 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558ae2e05350 0x558ae2e0cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:39.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.315+0000 7f381ec08740 1 --2- 172.21.15.83:0/921062987 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558ae2e04a30 0x558ae2e0ca00 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:39.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.315+0000 7f381ec08740 1 -- 172.21.15.83:0/921062987 >> 172.21.15.83:0/921062987 conn(0x558ae2e00d50 msgr2=0x558ae2dfffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:39.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.315+0000 7f37faa00640 0 client.17061 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:55:39.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.315+0000 7f381ec08740 1 -- 172.21.15.83:0/921062987 shutdown_connections 2024-04-24T08:55:39.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.315+0000 7f381ec08740 1 -- 172.21.15.83:0/921062987 wait complete. 2024-04-24T08:55:39.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.315+0000 7f381ec08740 20 client.17061 trim_cache size 0 max 16384 2024-04-24T08:55:39.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.335+0000 7f381ec08740 1 Processor -- start 2024-04-24T08:55:39.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.335+0000 7f381ec08740 1 -- start start 2024-04-24T08:55:39.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.336+0000 7f381ec08740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558ae2e00d50 0x558ae2e1b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:39.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.336+0000 7f381ec08740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ae2e04a30 0x558ae2e27670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:39.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.336+0000 7f381ec08740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558ae2e05350 0x558ae2e27bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:39.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.336+0000 7f381ec08740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f38141b5910 con 0x558ae2e05350 2024-04-24T08:55:39.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.336+0000 7f381ec08740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f38141b74e0 con 0x558ae2e04a30 2024-04-24T08:55:39.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.336+0000 7f381ec08740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f38141b5fe0 con 0x558ae2e00d50 2024-04-24T08:55:39.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.336+0000 7f3819e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558ae2e05350 0x558ae2e27bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:39.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.336+0000 7f3819e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558ae2e05350 0x558ae2e27bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55308/0 (socket says 172.21.15.83:55308) 2024-04-24T08:55:39.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.336+0000 7f3819e00640 1 -- 172.21.15.83:0/1069450038 learned_addr learned my addr 172.21.15.83:0/1069450038 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:55:39.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.336+0000 7f3819400640 1 --2- 172.21.15.83:0/1069450038 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558ae2e00d50 0x558ae2e1b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:39.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.336+0000 7f3818a00640 1 --2- 172.21.15.83:0/1069450038 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ae2e04a30 0x558ae2e27670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:39.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.337+0000 7f3819e00640 1 -- 172.21.15.83:0/1069450038 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558ae2e00d50 msgr2=0x558ae2e1b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:39.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.337+0000 7f3819e00640 1 --2- 172.21.15.83:0/1069450038 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558ae2e00d50 0x558ae2e1b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:39.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.337+0000 7f3819e00640 1 -- 172.21.15.83:0/1069450038 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ae2e04a30 msgr2=0x558ae2e27670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:39.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.337+0000 7f3819e00640 1 --2- 172.21.15.83:0/1069450038 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ae2e04a30 0x558ae2e27670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:39.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.337+0000 7f3819e00640 1 -- 172.21.15.83:0/1069450038 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ae2a7c3f0 con 0x558ae2e05350 2024-04-24T08:55:39.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.337+0000 7f3819e00640 1 --2- 172.21.15.83:0/1069450038 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558ae2e05350 0x558ae2e27bb0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f380c005c40 tx=0x7f380c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d2e8dff221fde7a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:39.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.337+0000 7f3805c00640 1 -- 172.21.15.83:0/1069450038 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f380c0107d0 con 0x558ae2e05350 2024-04-24T08:55:39.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.337+0000 7f3805c00640 1 -- 172.21.15.83:0/1069450038 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f380c010e20 con 0x558ae2e05350 2024-04-24T08:55:39.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.337+0000 7f3805c00640 1 -- 172.21.15.83:0/1069450038 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f380c0253e0 con 0x558ae2e05350 2024-04-24T08:55:39.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.338+0000 7f381ec08740 1 -- 172.21.15.83:0/1069450038 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558ae2e05350 msgr2=0x558ae2e27bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:39.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.338+0000 7f381ec08740 1 --2- 172.21.15.83:0/1069450038 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558ae2e05350 0x558ae2e27bb0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f380c005c40 tx=0x7f380c015000 comp rx=0 tx=0).stop 2024-04-24T08:55:39.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.338+0000 7f381ec08740 1 -- 172.21.15.83:0/1069450038 shutdown_connections 2024-04-24T08:55:39.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.338+0000 7f381ec08740 1 --2- 172.21.15.83:0/1069450038 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558ae2e05350 0x558ae2e27bb0 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:39.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.338+0000 7f381ec08740 1 --2- 172.21.15.83:0/1069450038 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ae2e04a30 0x558ae2e27670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:39.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.338+0000 7f381ec08740 1 --2- 172.21.15.83:0/1069450038 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558ae2e00d50 0x558ae2e1b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:39.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.338+0000 7f381ec08740 1 -- 172.21.15.83:0/1069450038 >> 172.21.15.83:0/1069450038 conn(0x558ae2e0b640 msgr2=0x558ae2dfffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:39.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.338+0000 7f381ec08740 1 -- 172.21.15.83:0/1069450038 shutdown_connections 2024-04-24T08:55:39.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.338+0000 7f381ec08740 1 -- 172.21.15.83:0/1069450038 wait complete. 2024-04-24T08:55:39.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.339+0000 7f381ec08740 1 -- start start 2024-04-24T08:55:39.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.339+0000 7f381ec08740 1 Processor -- start 2024-04-24T08:55:39.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.339+0000 7f381ec08740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558ae2e00d50 0x558ae2e09620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:39.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.339+0000 7f381ec08740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558ae2e04a30 0x558ae2e09b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:39.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.339+0000 7f381ec08740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ae2e05350 0x558ae2e0a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:39.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.339+0000 7f381ec08740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f38141b6d80 con 0x558ae2e00d50 2024-04-24T08:55:39.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.339+0000 7f381ec08740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f38141b66b0 con 0x558ae2e05350 2024-04-24T08:55:39.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.339+0000 7f381ec08740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f38141b38f0 con 0x558ae2e04a30 2024-04-24T08:55:39.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.339+0000 7f381ec08740 10 client.0 _ll_register_callbacks cb 0x7f381406b700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:55:39.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.340+0000 7f3819400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558ae2e00d50 0x558ae2e09620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:39.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.340+0000 7f3819400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558ae2e00d50 0x558ae2e09620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55320/0 (socket says 172.21.15.83:55320) 2024-04-24T08:55:39.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.340+0000 7f3819400640 1 -- 172.21.15.83:0/2360075889 learned_addr learned my addr 172.21.15.83:0/2360075889 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:55:39.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.340+0000 7f3819e00640 1 --2- 172.21.15.83:0/2360075889 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ae2e05350 0x558ae2e0a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:39.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.340+0000 7f3818a00640 1 --2- 172.21.15.83:0/2360075889 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558ae2e04a30 0x558ae2e09b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:39.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.340+0000 7f3819400640 1 -- 172.21.15.83:0/2360075889 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558ae2e04a30 msgr2=0x558ae2e09b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:39.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.340+0000 7f3819400640 1 --2- 172.21.15.83:0/2360075889 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558ae2e04a30 0x558ae2e09b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:39.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.340+0000 7f3819400640 1 -- 172.21.15.83:0/2360075889 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ae2e05350 msgr2=0x558ae2e0a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:39.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.340+0000 7f3819400640 1 --2- 172.21.15.83:0/2360075889 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ae2e05350 0x558ae2e0a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:39.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.340+0000 7f3819400640 1 -- 172.21.15.83:0/2360075889 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ae2a3c0f0 con 0x558ae2e00d50 2024-04-24T08:55:39.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.340+0000 7f3819e00640 1 --2- 172.21.15.83:0/2360075889 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ae2e05350 0x558ae2e0a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:55:39.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.340+0000 7f3819400640 1 --2- 172.21.15.83:0/2360075889 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558ae2e00d50 0x558ae2e09620 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f3808031780 tx=0x7f3808031ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a84f12611d4736d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:39.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.340+0000 7f37faa00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:55:39.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.341+0000 7f37faa00640 1 -- 172.21.15.83:0/2360075889 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3808051040 con 0x558ae2e00d50 2024-04-24T08:55:39.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.341+0000 7f37faa00640 1 -- 172.21.15.83:0/2360075889 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f3808005310 con 0x558ae2e00d50 2024-04-24T08:55:39.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.341+0000 7f381ec08740 1 -- 172.21.15.83:0/2360075889 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f380c010e20 con 0x558ae2e00d50 2024-04-24T08:55:39.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.341+0000 7f37faa00640 1 -- 172.21.15.83:0/2360075889 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3808065050 con 0x558ae2e00d50 2024-04-24T08:55:39.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.341+0000 7f381ec08740 10 client.17067 fetch_fsmap learned FSMap version 30 2024-04-24T08:55:39.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.341+0000 7f381ec08740 1 -- 172.21.15.83:0/2360075889 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558ae2d2f810 con 0x558ae2e00d50 2024-04-24T08:55:39.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.341+0000 7f37faa00640 1 -- 172.21.15.83:0/2360075889 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f380c010e20 con 0x558ae2e00d50 2024-04-24T08:55:39.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.341+0000 7f37faa00640 1 -- 172.21.15.83:0/2360075889 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7f380804b070 con 0x558ae2e00d50 2024-04-24T08:55:39.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.341+0000 7f381ec08740 10 client.17067 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T08:55:39.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.341+0000 7f381ec08740 10 client.17067 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:55:39.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.341+0000 7f381ec08740 20 client.17067 populate_metadata read hostname 'smithi083' 2024-04-24T08:55:39.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.341+0000 7f381ec08740 1 --2- 172.21.15.83:0/2360075889 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x558ae2e0fa20 0x558ae2e88630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:39.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.341+0000 7f381ec08740 4 client.17067 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T08:55:39.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.341+0000 7f381ec08740 1 -- 172.21.15.83:0/2360075889 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558ae2d71a80 con 0x558ae2e0fa20 2024-04-24T08:55:39.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.342+0000 7f3818a00640 1 --2- 172.21.15.83:0/2360075889 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x558ae2e0fa20 0x558ae2e88630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:39.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.343+0000 7f3818a00640 1 --2- 172.21.15.83:0/2360075889 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x558ae2e0fa20 0x558ae2e88630 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ebea77822cf1d5b4 server_cookie=1d72f8ec589d59b9 in_seq=0 out_seq=0 2024-04-24T08:55:39.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.343+0000 7f37faa00640 10 client.17067 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:55:39.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.343+0000 7f37faa00640 1 -- 172.21.15.83:0/2360075889 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x558ae2d71a80 con 0x558ae2e0fa20 2024-04-24T08:55:39.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.343+0000 7f37faa00640 10 client.17067 handle_command_reply: tid=0 2024-04-24T08:55:39.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.344+0000 7f381ec08740 1 client.17067 shutdown 2024-04-24T08:55:39.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.344+0000 7f381ec08740 1 -- 172.21.15.83:0/2360075889 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558ae2e00d50 msgr2=0x558ae2e09620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:39.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.344+0000 7f381ec08740 1 --2- 172.21.15.83:0/2360075889 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558ae2e00d50 0x558ae2e09620 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f3808031780 tx=0x7f3808031ab0 comp rx=0 tx=0).stop 2024-04-24T08:55:39.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.344+0000 7f381ec08740 1 -- 172.21.15.83:0/2360075889 shutdown_connections 2024-04-24T08:55:39.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.344+0000 7f381ec08740 1 --2- 172.21.15.83:0/2360075889 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x558ae2e0fa20 0x558ae2e88630 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:39.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.344+0000 7f381ec08740 1 --2- 172.21.15.83:0/2360075889 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ae2e05350 0x558ae2e0a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:39.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.344+0000 7f381ec08740 1 --2- 172.21.15.83:0/2360075889 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558ae2e04a30 0x558ae2e09b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:39.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.344+0000 7f37faa00640 0 client.17067 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:55:39.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.344+0000 7f381ec08740 1 --2- 172.21.15.83:0/2360075889 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558ae2e00d50 0x558ae2e09620 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:39.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.344+0000 7f381ec08740 1 -- 172.21.15.83:0/2360075889 >> 172.21.15.83:0/2360075889 conn(0x558ae2e0b640 msgr2=0x558ae2e0c9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:39.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.344+0000 7f381ec08740 1 -- 172.21.15.83:0/2360075889 shutdown_connections 2024-04-24T08:55:39.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.344+0000 7f381ec08740 1 -- 172.21.15.83:0/2360075889 wait complete. 2024-04-24T08:55:39.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.344+0000 7f381ec08740 20 client.17067 trim_cache size 0 max 16384 2024-04-24T08:55:39.350 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T08:55:39.350 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-04-24T08:55:39.350 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-04-24T08:55:39.351 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T08:55:39.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.345+0000 7f381a800640 1 -- 172.21.15.83:0/3048936832 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f37fc0a3d20 msgr2=0x7f37fc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:39.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.345+0000 7f381a800640 1 --2- 172.21.15.83:0/3048936832 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f37fc0a3d20 0x7f37fc0a61e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f380c005fd0 tx=0x7f380c0096d0 comp rx=0 tx=0).stop 2024-04-24T08:55:39.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.345+0000 7f381a800640 1 -- 172.21.15.83:0/3048936832 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3814108f40 msgr2=0x7f38141a4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:39.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.345+0000 7f381a800640 1 --2- 172.21.15.83:0/3048936832 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3814108f40 0x7f38141a4fa0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f380000d950 tx=0x7f380000de10 comp rx=0 tx=0).stop 2024-04-24T08:55:39.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.345+0000 7f381a800640 1 -- 172.21.15.83:0/3048936832 shutdown_connections 2024-04-24T08:55:39.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.345+0000 7f381a800640 1 --2- 172.21.15.83:0/3048936832 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f37fc0a3d20 0x7f37fc0a61e0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:39.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.345+0000 7f381a800640 1 --2- 172.21.15.83:0/3048936832 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f381410d500 0x7f381419eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:39.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.345+0000 7f381a800640 1 --2- 172.21.15.83:0/3048936832 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3814108f40 0x7f38141a4fa0 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:39.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.345+0000 7f381a800640 1 --2- 172.21.15.83:0/3048936832 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3814108580 0x7f38141a4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:39.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.345+0000 7f381a800640 1 -- 172.21.15.83:0/3048936832 >> 172.21.15.83:0/3048936832 conn(0x7f3814077e20 msgr2=0x7f3814078480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:39.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.345+0000 7f381a800640 1 -- 172.21.15.83:0/3048936832 shutdown_connections 2024-04-24T08:55:39.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.345+0000 7f381a800640 1 -- 172.21.15.83:0/3048936832 wait complete. 2024-04-24T08:55:39.366 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T08:55:39.366 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T08:55:39.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.565+0000 7fafbd600640 1 -- 172.21.15.83:0/2568386707 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fafb8104440 msgr2=0x7fafb8104820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:39.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.565+0000 7fafbd600640 1 --2- 172.21.15.83:0/2568386707 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fafb8104440 0x7fafb8104820 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fafa8009900 tx=0x7fafa802a410 comp rx=0 tx=0).stop 2024-04-24T08:55:39.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.565+0000 7fafbd600640 1 -- 172.21.15.83:0/2568386707 shutdown_connections 2024-04-24T08:55:39.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.565+0000 7fafbd600640 1 --2- 172.21.15.83:0/2568386707 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fafb8109510 0x7fafb81110e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:39.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.565+0000 7fafbd600640 1 --2- 172.21.15.83:0/2568386707 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fafb8104df0 0x7fafb8108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:39.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.565+0000 7fafbd600640 1 --2- 172.21.15.83:0/2568386707 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fafb8104440 0x7fafb8104820 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:39.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.565+0000 7fafbd600640 1 -- 172.21.15.83:0/2568386707 >> 172.21.15.83:0/2568386707 conn(0x7fafb80ffd00 msgr2=0x7fafb8102120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:39.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.565+0000 7fafbd600640 1 -- 172.21.15.83:0/2568386707 shutdown_connections 2024-04-24T08:55:39.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.565+0000 7fafbd600640 1 -- 172.21.15.83:0/2568386707 wait complete. 2024-04-24T08:55:39.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.566+0000 7fafbd600640 1 Processor -- start 2024-04-24T08:55:39.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.567+0000 7fafbd600640 1 -- start start 2024-04-24T08:55:39.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.567+0000 7fafbd600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fafb8104440 0x7fafb81a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:39.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.567+0000 7fafbd600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fafb8109510 0x7fafb81a0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:39.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.567+0000 7fafbd600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fafb819a880 0x7fafb819ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:39.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.567+0000 7fafbd600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fafb8114480 con 0x7fafb8109510 2024-04-24T08:55:39.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.567+0000 7fafbd600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fafb8114300 con 0x7fafb819a880 2024-04-24T08:55:39.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.567+0000 7fafbd600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fafb8114600 con 0x7fafb8104440 2024-04-24T08:55:39.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.567+0000 7fafb6000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fafb8109510 0x7fafb81a0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:39.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.567+0000 7fafb6000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fafb8109510 0x7fafb81a0c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55348/0 (socket says 172.21.15.83:55348) 2024-04-24T08:55:39.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.567+0000 7fafb6000640 1 -- 172.21.15.83:0/2093663737 learned_addr learned my addr 172.21.15.83:0/2093663737 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:55:39.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.568+0000 7fafb6a00640 1 --2- 172.21.15.83:0/2093663737 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fafb8104440 0x7fafb81a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:39.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.568+0000 7fafb6000640 1 -- 172.21.15.83:0/2093663737 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fafb8104440 msgr2=0x7fafb81a06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:39.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.568+0000 7fafb6000640 1 --2- 172.21.15.83:0/2093663737 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fafb8104440 0x7fafb81a06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:39.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.568+0000 7fafb6000640 1 -- 172.21.15.83:0/2093663737 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fafb819a880 msgr2=0x7fafb819ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:39.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.568+0000 7fafb6000640 1 --2- 172.21.15.83:0/2093663737 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fafb819a880 0x7fafb819ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:39.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.568+0000 7fafb6000640 1 -- 172.21.15.83:0/2093663737 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafb819b410 con 0x7fafb8109510 2024-04-24T08:55:39.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.568+0000 7fafb6000640 1 --2- 172.21.15.83:0/2093663737 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fafb8109510 0x7fafb81a0c00 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fafa400d770 tx=0x7fafa400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:39.570 INFO:tasks.workunit.client.0.smithi083.stdout: 1 16502 0.52 MB/sec execute 13 sec latency 56.724 ms 2024-04-24T08:55:39.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.568+0000 7fafaf400640 1 -- 172.21.15.83:0/2093663737 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa4017070 con 0x7fafb8109510 2024-04-24T08:55:39.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.568+0000 7fafaf400640 1 -- 172.21.15.83:0/2093663737 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fafa4009c30 con 0x7fafb8109510 2024-04-24T08:55:39.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.568+0000 7fafaf400640 1 -- 172.21.15.83:0/2093663737 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa400f620 con 0x7fafb8109510 2024-04-24T08:55:39.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.568+0000 7fafbd600640 1 -- 172.21.15.83:0/2093663737 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafb819b700 con 0x7fafb8109510 2024-04-24T08:55:39.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.569+0000 7fafbd600640 1 -- 172.21.15.83:0/2093663737 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafb81a78e0 con 0x7fafb8109510 2024-04-24T08:55:39.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.570+0000 7fafbd600640 1 -- 172.21.15.83:0/2093663737 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafb8105910 con 0x7fafb8109510 2024-04-24T08:55:39.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.572+0000 7fafaf400640 1 -- 172.21.15.83:0/2093663737 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fafa401a070 con 0x7fafb8109510 2024-04-24T08:55:39.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.573+0000 7fafaf400640 1 --2- 172.21.15.83:0/2093663737 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7faf9c0a3da0 0x7faf9c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:39.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.574+0000 7fafaf400640 1 -- 172.21.15.83:0/2093663737 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7fafa40d0700 con 0x7fafb8109510 2024-04-24T08:55:39.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.574+0000 7fafb6a00640 1 --2- 172.21.15.83:0/2093663737 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7faf9c0a3da0 0x7faf9c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:39.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.574+0000 7fafb6a00640 1 --2- 172.21.15.83:0/2093663737 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7faf9c0a3da0 0x7faf9c0a6260 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fafa8031440 tx=0x7fafa80095f0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:39.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.578+0000 7fafaf400640 1 -- 172.21.15.83:0/2093663737 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fafb819b700 con 0x7fafb8109510 2024-04-24T08:55:39.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.844+0000 7fafbd600640 1 -- 172.21.15.83:0/2093663737 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fafb8104df0 con 0x7fafb8109510 2024-04-24T08:55:39.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.845+0000 7fafaf400640 1 -- 172.21.15.83:0/2093663737 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11949 (secure 0 0 0) 0x7fafa4009de0 con 0x7fafb8109510 2024-04-24T08:55:39.846 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:55:39.846 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/358872257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":358872257},{"type":"v1","addr":"172.21.15.83:6835","nonce":358872257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3157070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3157070851},{"type":"v1","addr":"172.21.15.83:6837","nonce":3157070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T08:48:29.422576+0000","modified":"2024-04-24T08:53:42.044317+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34520,"mds_1":34493,"mds_2":34451,"mds_3":34427,"mds_4":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14658":{"gid":14658,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6839/4085470844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":4085470844},{"type":"v1","addr":"172.21.15.83:6839","nonce":4085470844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14676":{"gid":14676,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6841/2634754210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":2634754210},{"type":"v1","addr":"172.21.15.83:6841","nonce":2634754210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24382":{"gid":24382,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/1859921310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1859921310},{"type":"v1","addr":"172.21.15.189:6833","nonce":1859921310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/2241923197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2241923197},{"type":"v1","addr":"172.21.15.189:6835","nonce":2241923197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6837/3172311220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":3172311220},{"type":"v1","addr":"172.21.15.189:6837","nonce":3172311220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.189:6839/2548965336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":2548965336},{"type":"v1","addr":"172.21.15.189:6839","nonce":2548965336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.138:6833/1157967658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1157967658},{"type":"v1","addr":"172.21.15.138:6833","nonce":1157967658}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.138:6835/1909219758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1909219758},{"type":"v1","addr":"172.21.15.138:6835","nonce":1909219758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34493":{"gid":34493,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.138:6837/900566754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":900566754},{"type":"v1","addr":"172.21.15.138:6837","nonce":900566754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34520":{"gid":34520,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.138:6839/2236932393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2236932393},{"type":"v1","addr":"172.21.15.138:6839","nonce":2236932393}]},"join_fscid":-1,"export_targets":[1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34520,"qdb_cluster":[34493,34451,34427,24454,34520]},"id":1}]} 2024-04-24T08:55:39.846 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 30 2024-04-24T08:55:39.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.848+0000 7fafbd600640 1 -- 172.21.15.83:0/2093663737 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7faf9c0a3da0 msgr2=0x7faf9c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:39.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.848+0000 7fafbd600640 1 --2- 172.21.15.83:0/2093663737 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7faf9c0a3da0 0x7faf9c0a6260 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fafa8031440 tx=0x7fafa80095f0 comp rx=0 tx=0).stop 2024-04-24T08:55:39.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.848+0000 7fafbd600640 1 -- 172.21.15.83:0/2093663737 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fafb8109510 msgr2=0x7fafb81a0c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:39.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.848+0000 7fafbd600640 1 --2- 172.21.15.83:0/2093663737 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fafb8109510 0x7fafb81a0c00 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fafa400d770 tx=0x7fafa400dc30 comp rx=0 tx=0).stop 2024-04-24T08:55:39.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.848+0000 7fafbd600640 1 -- 172.21.15.83:0/2093663737 shutdown_connections 2024-04-24T08:55:39.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.848+0000 7fafbd600640 1 --2- 172.21.15.83:0/2093663737 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7faf9c0a3da0 0x7faf9c0a6260 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:39.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.848+0000 7fafbd600640 1 --2- 172.21.15.83:0/2093663737 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fafb819a880 0x7fafb819ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:39.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.848+0000 7fafbd600640 1 --2- 172.21.15.83:0/2093663737 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fafb8109510 0x7fafb81a0c00 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:39.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.848+0000 7fafbd600640 1 --2- 172.21.15.83:0/2093663737 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fafb8104440 0x7fafb81a06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:39.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.848+0000 7fafbd600640 1 -- 172.21.15.83:0/2093663737 >> 172.21.15.83:0/2093663737 conn(0x7fafb80ffd00 msgr2=0x7fafb81011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:39.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.848+0000 7fafbd600640 1 -- 172.21.15.83:0/2093663737 shutdown_connections 2024-04-24T08:55:39.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:39.848+0000 7fafbd600640 1 -- 172.21.15.83:0/2093663737 wait complete. 2024-04-24T08:55:39.867 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-24T08:55:40.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.080+0000 7f3a26200640 1 -- 172.21.15.83:0/3145713799 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a201093f0 msgr2=0x7f3a20110fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:40.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.080+0000 7f3a26200640 1 --2- 172.21.15.83:0/3145713799 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a201093f0 0x7f3a20110fc0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f3a18009ca0 tx=0x7f3a1802a8c0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.080+0000 7f3a26200640 1 -- 172.21.15.83:0/3145713799 shutdown_connections 2024-04-24T08:55:40.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.080+0000 7f3a26200640 1 --2- 172.21.15.83:0/3145713799 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a201093f0 0x7f3a20110fc0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.080+0000 7f3a26200640 1 --2- 172.21.15.83:0/3145713799 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a20104cd0 0x7f3a20108d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.080+0000 7f3a26200640 1 --2- 172.21.15.83:0/3145713799 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3a20104320 0x7f3a20104700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.080+0000 7f3a26200640 1 -- 172.21.15.83:0/3145713799 >> 172.21.15.83:0/3145713799 conn(0x7f3a200ffb40 msgr2=0x7f3a20101f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:40.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.080+0000 7f3a26200640 1 -- 172.21.15.83:0/3145713799 shutdown_connections 2024-04-24T08:55:40.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.080+0000 7f3a26200640 1 -- 172.21.15.83:0/3145713799 wait complete. 2024-04-24T08:55:40.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.081+0000 7f3a26200640 1 Processor -- start 2024-04-24T08:55:40.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.081+0000 7f3a26200640 1 -- start start 2024-04-24T08:55:40.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.081+0000 7f3a26200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3a20104320 0x7f3a201a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:40.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.081+0000 7f3a26200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a201093f0 0x7f3a201a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:40.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.081+0000 7f3a26200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a2019a6e0 0x7f3a2019ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:40.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.082+0000 7f3a26200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3a20114360 con 0x7f3a201093f0 2024-04-24T08:55:40.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.082+0000 7f3a26200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3a201141e0 con 0x7f3a2019a6e0 2024-04-24T08:55:40.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.082+0000 7f3a26200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3a201144e0 con 0x7f3a20104320 2024-04-24T08:55:40.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.082+0000 7f3a1fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a201093f0 0x7f3a201a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:40.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.082+0000 7f3a1fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a201093f0 0x7f3a201a0b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55352/0 (socket says 172.21.15.83:55352) 2024-04-24T08:55:40.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.082+0000 7f3a1fe00640 1 -- 172.21.15.83:0/151835195 learned_addr learned my addr 172.21.15.83:0/151835195 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:55:40.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.082+0000 7f3a24e00640 1 --2- 172.21.15.83:0/151835195 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3a20104320 0x7f3a201a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:40.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.082+0000 7f3a25800640 1 --2- 172.21.15.83:0/151835195 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a2019a6e0 0x7f3a2019ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:40.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.082+0000 7f3a1fe00640 1 -- 172.21.15.83:0/151835195 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3a20104320 msgr2=0x7f3a201a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:40.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.082+0000 7f3a1fe00640 1 --2- 172.21.15.83:0/151835195 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3a20104320 0x7f3a201a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.082+0000 7f3a1fe00640 1 -- 172.21.15.83:0/151835195 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a2019a6e0 msgr2=0x7f3a2019ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:40.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.082+0000 7f3a1fe00640 1 --2- 172.21.15.83:0/151835195 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a2019a6e0 0x7f3a2019ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.082+0000 7f3a1fe00640 1 -- 172.21.15.83:0/151835195 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a2019b2e0 con 0x7f3a201093f0 2024-04-24T08:55:40.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.082+0000 7f3a25800640 1 --2- 172.21.15.83:0/151835195 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a2019a6e0 0x7f3a2019ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:55:40.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.083+0000 7f3a1fe00640 1 --2- 172.21.15.83:0/151835195 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a201093f0 0x7f3a201a0b60 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c00d770 tx=0x7f3a0c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:40.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.083+0000 7f3a1d600640 1 -- 172.21.15.83:0/151835195 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a0c017070 con 0x7f3a201093f0 2024-04-24T08:55:40.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.083+0000 7f3a1d600640 1 -- 172.21.15.83:0/151835195 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3a0c009c30 con 0x7f3a201093f0 2024-04-24T08:55:40.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.083+0000 7f3a1d600640 1 -- 172.21.15.83:0/151835195 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a0c00f620 con 0x7f3a201093f0 2024-04-24T08:55:40.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.083+0000 7f3a26200640 1 -- 172.21.15.83:0/151835195 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a201a73c0 con 0x7f3a201093f0 2024-04-24T08:55:40.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.083+0000 7f3a26200640 1 -- 172.21.15.83:0/151835195 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a201a77c0 con 0x7f3a201093f0 2024-04-24T08:55:40.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.086+0000 7f3a1d600640 1 -- 172.21.15.83:0/151835195 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3a0c01a070 con 0x7f3a201093f0 2024-04-24T08:55:40.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.087+0000 7f3a2a791740 1 Processor -- start 2024-04-24T08:55:40.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.087+0000 7f3a2a791740 1 -- start start 2024-04-24T08:55:40.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.087+0000 7f3a2a791740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d248c00b40 0x55d248c00f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:40.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.087+0000 7f3a2a791740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d248c01460 0x55d248c0b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:40.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.088+0000 7f3a2a791740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d248c0bb30 0x55d248c0df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:40.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.088+0000 7f3a2a791740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d248b661d0 con 0x55d248c00b40 2024-04-24T08:55:40.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.088+0000 7f3a2a791740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d248b0c450 con 0x55d248c0bb30 2024-04-24T08:55:40.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.088+0000 7f3a2a791740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d248b62190 con 0x55d248c01460 2024-04-24T08:55:40.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.088+0000 7f3a25800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d248c00b40 0x55d248c00f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:40.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.088+0000 7f3a25800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d248c00b40 0x55d248c00f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55360/0 (socket says 172.21.15.83:55360) 2024-04-24T08:55:40.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.088+0000 7f3a25800640 1 -- 172.21.15.83:0/2582288398 learned_addr learned my addr 172.21.15.83:0/2582288398 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:55:40.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.088+0000 7f3a24e00640 1 --2- 172.21.15.83:0/2582288398 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d248c01460 0x55d248c0b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:40.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.088+0000 7f3a1fe00640 1 --2- 172.21.15.83:0/2582288398 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d248c0bb30 0x55d248c0df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:40.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.088+0000 7f3a25800640 1 -- 172.21.15.83:0/2582288398 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d248c01460 msgr2=0x55d248c0b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:40.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.088+0000 7f3a1d600640 1 --2- 172.21.15.83:0/151835195 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f3a080a3d20 0x7f3a080a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:40.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.088+0000 7f3a25800640 1 --2- 172.21.15.83:0/2582288398 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d248c01460 0x55d248c0b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.088+0000 7f3a25800640 1 -- 172.21.15.83:0/2582288398 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d248c0bb30 msgr2=0x55d248c0df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:40.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.088+0000 7f3a25800640 1 --2- 172.21.15.83:0/2582288398 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d248c0bb30 0x55d248c0df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.088+0000 7f3a25800640 1 -- 172.21.15.83:0/2582288398 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d248bc9310 con 0x55d248c00b40 2024-04-24T08:55:40.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.088+0000 7f3a1fe00640 1 --2- 172.21.15.83:0/2582288398 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d248c0bb30 0x55d248c0df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:55:40.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.089+0000 7f3a1d600640 1 -- 172.21.15.83:0/151835195 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f3a0c0d0760 con 0x7f3a201093f0 2024-04-24T08:55:40.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.089+0000 7f3a25800640 1 --2- 172.21.15.83:0/2582288398 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d248c00b40 0x55d248c00f20 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f3a18009400 tx=0x7f3a18009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19c5e7e838e15ecc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:40.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.089+0000 7f3a25800640 1 --2- 172.21.15.83:0/151835195 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f3a080a3d20 0x7f3a080a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:40.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.089+0000 7f3a12000640 1 -- 172.21.15.83:0/2582288398 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a1804c070 con 0x55d248c00b40 2024-04-24T08:55:40.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.090+0000 7f3a12000640 1 -- 172.21.15.83:0/2582288398 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3a18036070 con 0x55d248c00b40 2024-04-24T08:55:40.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.090+0000 7f3a12000640 1 -- 172.21.15.83:0/2582288398 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a18030820 con 0x55d248c00b40 2024-04-24T08:55:40.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.090+0000 7f3a2a791740 1 -- 172.21.15.83:0/2582288398 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d248c00b40 msgr2=0x55d248c00f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:40.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.090+0000 7f3a25800640 1 --2- 172.21.15.83:0/151835195 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f3a080a3d20 0x7f3a080a61e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f3a18049000 tx=0x7f3a1804a000 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:40.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.090+0000 7f3a2a791740 1 --2- 172.21.15.83:0/2582288398 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d248c00b40 0x55d248c00f20 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f3a18009400 tx=0x7f3a18009a20 comp rx=0 tx=0).stop 2024-04-24T08:55:40.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.091+0000 7f3a2a791740 1 -- 172.21.15.83:0/2582288398 shutdown_connections 2024-04-24T08:55:40.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.091+0000 7f3a2a791740 1 --2- 172.21.15.83:0/2582288398 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d248c0bb30 0x55d248c0df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.091+0000 7f3a2a791740 1 --2- 172.21.15.83:0/2582288398 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d248c01460 0x55d248c0b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.091+0000 7f3a2a791740 1 --2- 172.21.15.83:0/2582288398 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d248c00b40 0x55d248c00f20 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.091+0000 7f3a2a791740 1 -- 172.21.15.83:0/2582288398 >> 172.21.15.83:0/2582288398 conn(0x55d248bfce60 msgr2=0x55d248bfd240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:40.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.091+0000 7f3a2a791740 1 -- 172.21.15.83:0/2582288398 shutdown_connections 2024-04-24T08:55:40.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.091+0000 7f3a2a791740 1 -- 172.21.15.83:0/2582288398 wait complete. 2024-04-24T08:55:40.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.091+0000 7f3a2a791740 1 -- start start 2024-04-24T08:55:40.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.091+0000 7f3a2a791740 1 Processor -- start 2024-04-24T08:55:40.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.092+0000 7f3a2a791740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d248c00b40 0x55d248c0b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:40.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.092+0000 7f3a2a791740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d248c01460 0x55d248c087f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:40.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.092+0000 7f3a2a791740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d248c0bb30 0x55d248c08e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:40.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.092+0000 7f3a2a791740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d248c0e9c0 con 0x55d248c00b40 2024-04-24T08:55:40.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.092+0000 7f3a2a791740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d248b1f810 con 0x55d248c0bb30 2024-04-24T08:55:40.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.092+0000 7f3a2a791740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d248c0eb40 con 0x55d248c01460 2024-04-24T08:55:40.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.092+0000 7f3a2a791740 10 client.0 _ll_register_callbacks cb 0x55d248aed140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:55:40.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.092+0000 7f3a24e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d248c00b40 0x55d248c0b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:40.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.092+0000 7f3a24e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d248c00b40 0x55d248c0b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55370/0 (socket says 172.21.15.83:55370) 2024-04-24T08:55:40.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.092+0000 7f3a24e00640 1 -- 172.21.15.83:0/589667787 learned_addr learned my addr 172.21.15.83:0/589667787 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:55:40.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.092+0000 7f3a1fe00640 1 --2- 172.21.15.83:0/589667787 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d248c01460 0x55d248c087f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:40.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.093+0000 7f3a25800640 1 --2- 172.21.15.83:0/589667787 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d248c0bb30 0x55d248c08e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:40.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.093+0000 7f3a24e00640 1 -- 172.21.15.83:0/589667787 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d248c01460 msgr2=0x55d248c087f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:40.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.093+0000 7f3a24e00640 1 --2- 172.21.15.83:0/589667787 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d248c01460 0x55d248c087f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.093+0000 7f3a24e00640 1 -- 172.21.15.83:0/589667787 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d248c0bb30 msgr2=0x55d248c08e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:40.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.093+0000 7f3a24e00640 1 --2- 172.21.15.83:0/589667787 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d248c0bb30 0x55d248c08e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.093+0000 7f3a24e00640 1 -- 172.21.15.83:0/589667787 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d248b303f0 con 0x55d248c00b40 2024-04-24T08:55:40.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.093+0000 7f3a24e00640 1 --2- 172.21.15.83:0/589667787 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d248c00b40 0x55d248c0b500 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f3a1400bed0 tx=0x7f3a140117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c819d0ff0d6c4474 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:40.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.093+0000 7f3a06a00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:55:40.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.093+0000 7f3a06a00640 1 -- 172.21.15.83:0/589667787 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a14012650 con 0x55d248c00b40 2024-04-24T08:55:40.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.093+0000 7f3a06a00640 1 -- 172.21.15.83:0/589667787 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f3a140127f0 con 0x55d248c00b40 2024-04-24T08:55:40.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.093+0000 7f3a2a791740 1 -- 172.21.15.83:0/589667787 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a18036070 con 0x55d248c00b40 2024-04-24T08:55:40.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.094+0000 7f3a06a00640 1 -- 172.21.15.83:0/589667787 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3a140129f0 con 0x55d248c00b40 2024-04-24T08:55:40.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.094+0000 7f3a2a791740 10 client.17085 fetch_fsmap learned FSMap version 30 2024-04-24T08:55:40.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.094+0000 7f3a2a791740 1 -- 172.21.15.83:0/589667787 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d2488783f0 con 0x55d248c00b40 2024-04-24T08:55:40.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.094+0000 7f3a06a00640 1 -- 172.21.15.83:0/589667787 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3a18036070 con 0x55d248c00b40 2024-04-24T08:55:40.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.094+0000 7f3a06a00640 1 -- 172.21.15.83:0/589667787 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7f3a1401a3d0 con 0x55d248c00b40 2024-04-24T08:55:40.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.094+0000 7f3a2a791740 10 client.17085 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T08:55:40.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.094+0000 7f3a2a791740 10 client.17085 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-24T08:55:40.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.094+0000 7f3a2a791740 20 client.17085 populate_metadata read hostname 'smithi083' 2024-04-24T08:55:40.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.094+0000 7f3a2a791740 1 --2- 172.21.15.83:0/589667787 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55d248c1bc80 0x55d248c1e070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:40.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.094+0000 7f3a2a791740 4 client.17085 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:55:40.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.094+0000 7f3a2a791740 1 -- 172.21.15.83:0/589667787 --> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d248b73bf0 con 0x55d248c1bc80 2024-04-24T08:55:40.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.095+0000 7f3a1fe00640 1 --2- 172.21.15.83:0/589667787 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55d248c1bc80 0x55d248c1e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:40.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.096+0000 7f3a1fe00640 1 --2- 172.21.15.83:0/589667787 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55d248c1bc80 0x55d248c1e070 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=85491d87714133ee server_cookie=31fdc6434fd25c3 in_seq=0 out_seq=0 2024-04-24T08:55:40.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.096+0000 7f3a06a00640 10 client.17085 ms_handle_connect on v2:172.21.15.189:6838/2548965336 2024-04-24T08:55:40.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.098+0000 7f3a06a00640 1 -- 172.21.15.83:0/589667787 <== mds.4 v2:172.21.15.189:6838/2548965336 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d248b73bf0 con 0x55d248c1bc80 2024-04-24T08:55:40.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.098+0000 7f3a06a00640 10 client.17085 handle_command_reply: tid=0 2024-04-24T08:55:40.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.098+0000 7f3a2a791740 1 client.17085 shutdown 2024-04-24T08:55:40.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.098+0000 7f3a2a791740 1 -- 172.21.15.83:0/589667787 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d248c00b40 msgr2=0x55d248c0b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:40.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.098+0000 7f3a2a791740 1 --2- 172.21.15.83:0/589667787 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d248c00b40 0x55d248c0b500 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f3a1400bed0 tx=0x7f3a140117b0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.098+0000 7f3a2a791740 1 -- 172.21.15.83:0/589667787 shutdown_connections 2024-04-24T08:55:40.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.098+0000 7f3a2a791740 1 --2- 172.21.15.83:0/589667787 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55d248c1bc80 0x55d248c1e070 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.098+0000 7f3a2a791740 1 --2- 172.21.15.83:0/589667787 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d248c0bb30 0x55d248c08e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.098+0000 7f3a2a791740 1 --2- 172.21.15.83:0/589667787 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d248c01460 0x55d248c087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.098+0000 7f3a2a791740 1 --2- 172.21.15.83:0/589667787 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d248c00b40 0x55d248c0b500 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.098+0000 7f3a06a00640 0 client.17085 ms_handle_reset on v2:172.21.15.189:6838/2548965336 2024-04-24T08:55:40.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.098+0000 7f3a2a791740 1 -- 172.21.15.83:0/589667787 >> 172.21.15.83:0/589667787 conn(0x55d248bfce60 msgr2=0x55d248c02b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:40.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.098+0000 7f3a2a791740 1 -- 172.21.15.83:0/589667787 shutdown_connections 2024-04-24T08:55:40.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.098+0000 7f3a2a791740 1 -- 172.21.15.83:0/589667787 wait complete. 2024-04-24T08:55:40.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.099+0000 7f3a2a791740 20 client.17085 trim_cache size 0 max 16384 2024-04-24T08:55:40.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.115+0000 7f3a2a791740 1 Processor -- start 2024-04-24T08:55:40.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.115+0000 7f3a2a791740 1 -- start start 2024-04-24T08:55:40.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.115+0000 7f3a2a791740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d248c00b40 0x55d248c178b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:40.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.115+0000 7f3a2a791740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d248c01460 0x55d248c0b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:40.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.115+0000 7f3a2a791740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d248c0bb30 0x55d248c238d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:40.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.115+0000 7f3a2a791740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3a201b1590 con 0x55d248c0bb30 2024-04-24T08:55:40.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.115+0000 7f3a2a791740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3a201b3160 con 0x55d248c01460 2024-04-24T08:55:40.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.115+0000 7f3a2a791740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3a201b1c60 con 0x55d248c00b40 2024-04-24T08:55:40.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.116+0000 7f3a25800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d248c0bb30 0x55d248c238d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:40.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.116+0000 7f3a25800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d248c0bb30 0x55d248c238d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55384/0 (socket says 172.21.15.83:55384) 2024-04-24T08:55:40.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.116+0000 7f3a25800640 1 -- 172.21.15.83:0/3067813334 learned_addr learned my addr 172.21.15.83:0/3067813334 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:55:40.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.116+0000 7f3a1fe00640 1 --2- 172.21.15.83:0/3067813334 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d248c01460 0x55d248c0b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:40.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.116+0000 7f3a24e00640 1 --2- 172.21.15.83:0/3067813334 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d248c00b40 0x55d248c178b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:40.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.116+0000 7f3a25800640 1 -- 172.21.15.83:0/3067813334 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d248c00b40 msgr2=0x55d248c178b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:40.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.116+0000 7f3a25800640 1 --2- 172.21.15.83:0/3067813334 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d248c00b40 0x55d248c178b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.116+0000 7f3a25800640 1 -- 172.21.15.83:0/3067813334 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d248c01460 msgr2=0x55d248c0b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:40.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.116+0000 7f3a25800640 1 --2- 172.21.15.83:0/3067813334 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d248c01460 0x55d248c0b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.116+0000 7f3a25800640 1 -- 172.21.15.83:0/3067813334 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2489427d0 con 0x55d248c0bb30 2024-04-24T08:55:40.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.117+0000 7f3a25800640 1 --2- 172.21.15.83:0/3067813334 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d248c0bb30 0x55d248c238d0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f3a180049d0 tx=0x7f3a18050000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b3444121bf38f98 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:40.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.117+0000 7f3a11600640 1 -- 172.21.15.83:0/3067813334 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a18058070 con 0x55d248c0bb30 2024-04-24T08:55:40.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.117+0000 7f3a11600640 1 -- 172.21.15.83:0/3067813334 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3a18036040 con 0x55d248c0bb30 2024-04-24T08:55:40.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.117+0000 7f3a11600640 1 -- 172.21.15.83:0/3067813334 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a1804c030 con 0x55d248c0bb30 2024-04-24T08:55:40.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.118+0000 7f3a2a791740 1 -- 172.21.15.83:0/3067813334 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d248c0bb30 msgr2=0x55d248c238d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:40.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.118+0000 7f3a2a791740 1 --2- 172.21.15.83:0/3067813334 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d248c0bb30 0x55d248c238d0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f3a180049d0 tx=0x7f3a18050000 comp rx=0 tx=0).stop 2024-04-24T08:55:40.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.118+0000 7f3a2a791740 1 -- 172.21.15.83:0/3067813334 shutdown_connections 2024-04-24T08:55:40.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.118+0000 7f3a2a791740 1 --2- 172.21.15.83:0/3067813334 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d248c0bb30 0x55d248c238d0 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.118+0000 7f3a2a791740 1 --2- 172.21.15.83:0/3067813334 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d248c01460 0x55d248c0b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.118+0000 7f3a2a791740 1 --2- 172.21.15.83:0/3067813334 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d248c00b40 0x55d248c178b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.118+0000 7f3a2a791740 1 -- 172.21.15.83:0/3067813334 >> 172.21.15.83:0/3067813334 conn(0x55d248bfce60 msgr2=0x55d248c02290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:40.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.118+0000 7f3a2a791740 1 -- 172.21.15.83:0/3067813334 shutdown_connections 2024-04-24T08:55:40.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.118+0000 7f3a2a791740 1 -- 172.21.15.83:0/3067813334 wait complete. 2024-04-24T08:55:40.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.119+0000 7f3a2a791740 1 -- start start 2024-04-24T08:55:40.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.119+0000 7f3a2a791740 1 Processor -- start 2024-04-24T08:55:40.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.120+0000 7f3a2a791740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d248c00b40 0x55d248c14290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:40.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.120+0000 7f3a2a791740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d248c01460 0x55d248c05e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:40.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.120+0000 7f3a2a791740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d248c0bb30 0x55d248c064b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:40.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.120+0000 7f3a2a791740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3a201b2a00 con 0x55d248c01460 2024-04-24T08:55:40.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.120+0000 7f3a2a791740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3a201b2330 con 0x55d248c0bb30 2024-04-24T08:55:40.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.120+0000 7f3a2a791740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3a201af570 con 0x55d248c00b40 2024-04-24T08:55:40.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.120+0000 7f3a2a791740 10 client.0 _ll_register_callbacks cb 0x7f3a1400b400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:55:40.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.120+0000 7f3a1fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d248c01460 0x55d248c05e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:40.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.120+0000 7f3a1fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d248c01460 0x55d248c05e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55394/0 (socket says 172.21.15.83:55394) 2024-04-24T08:55:40.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.120+0000 7f3a1fe00640 1 -- 172.21.15.83:0/1345283636 learned_addr learned my addr 172.21.15.83:0/1345283636 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:55:40.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.120+0000 7f3a25800640 1 --2- 172.21.15.83:0/1345283636 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d248c0bb30 0x55d248c064b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:40.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.120+0000 7f3a24e00640 1 --2- 172.21.15.83:0/1345283636 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d248c00b40 0x55d248c14290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:40.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.120+0000 7f3a1fe00640 1 -- 172.21.15.83:0/1345283636 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d248c00b40 msgr2=0x55d248c14290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:40.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.120+0000 7f3a1fe00640 1 --2- 172.21.15.83:0/1345283636 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d248c00b40 0x55d248c14290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.120+0000 7f3a1fe00640 1 -- 172.21.15.83:0/1345283636 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d248c0bb30 msgr2=0x55d248c064b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:40.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.120+0000 7f3a1fe00640 1 --2- 172.21.15.83:0/1345283636 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d248c0bb30 0x55d248c064b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.120+0000 7f3a1fe00640 1 -- 172.21.15.83:0/1345283636 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d248c0cca0 con 0x55d248c01460 2024-04-24T08:55:40.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.120+0000 7f3a25800640 1 --2- 172.21.15.83:0/1345283636 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d248c0bb30 0x55d248c064b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:55:40.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.121+0000 7f3a1fe00640 1 --2- 172.21.15.83:0/1345283636 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d248c01460 0x55d248c05e80 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c0cdae0 tx=0x7f3a0c09c1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a67d1f62275ec9f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:40.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.121+0000 7f3a06a00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:55:40.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.121+0000 7f3a06a00640 1 -- 172.21.15.83:0/1345283636 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a0c021040 con 0x55d248c01460 2024-04-24T08:55:40.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.121+0000 7f3a06a00640 1 -- 172.21.15.83:0/1345283636 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f3a0c098940 con 0x55d248c01460 2024-04-24T08:55:40.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.121+0000 7f3a2a791740 1 -- 172.21.15.83:0/1345283636 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a18036040 con 0x55d248c01460 2024-04-24T08:55:40.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.121+0000 7f3a06a00640 1 -- 172.21.15.83:0/1345283636 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3a0c0a4f40 con 0x55d248c01460 2024-04-24T08:55:40.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.121+0000 7f3a2a791740 10 client.17091 fetch_fsmap learned FSMap version 30 2024-04-24T08:55:40.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.121+0000 7f3a2a791740 1 -- 172.21.15.83:0/1345283636 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d248c82670 con 0x55d248c01460 2024-04-24T08:55:40.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.121+0000 7f3a06a00640 1 -- 172.21.15.83:0/1345283636 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3a18036040 con 0x55d248c01460 2024-04-24T08:55:40.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.122+0000 7f3a06a00640 1 -- 172.21.15.83:0/1345283636 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7f3a0c0d02e0 con 0x55d248c01460 2024-04-24T08:55:40.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.122+0000 7f3a2a791740 10 client.17091 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T08:55:40.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.122+0000 7f3a2a791740 10 client.17091 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-24T08:55:40.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.122+0000 7f3a2a791740 20 client.17091 populate_metadata read hostname 'smithi083' 2024-04-24T08:55:40.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.122+0000 7f3a2a791740 1 --2- 172.21.15.83:0/1345283636 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55d248c82de0 0x55d248c85200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:40.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.122+0000 7f3a2a791740 4 client.17091 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T08:55:40.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.122+0000 7f3a2a791740 1 -- 172.21.15.83:0/1345283636 --> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d248b6da60 con 0x55d248c82de0 2024-04-24T08:55:40.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.122+0000 7f3a24e00640 1 --2- 172.21.15.83:0/1345283636 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55d248c82de0 0x55d248c85200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:40.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.123+0000 7f3a24e00640 1 --2- 172.21.15.83:0/1345283636 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55d248c82de0 0x55d248c85200 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=775f5c434dc8f8b4 server_cookie=cc49383a84f39272 in_seq=0 out_seq=0 2024-04-24T08:55:40.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.123+0000 7f3a06a00640 10 client.17091 ms_handle_connect on v2:172.21.15.189:6838/2548965336 2024-04-24T08:55:40.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.124+0000 7f3a06a00640 1 -- 172.21.15.83:0/1345283636 <== mds.4 v2:172.21.15.189:6838/2548965336 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d248b6da60 con 0x55d248c82de0 2024-04-24T08:55:40.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.124+0000 7f3a06a00640 10 client.17091 handle_command_reply: tid=0 2024-04-24T08:55:40.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.124+0000 7f3a2a791740 1 client.17091 shutdown 2024-04-24T08:55:40.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.124+0000 7f3a2a791740 1 -- 172.21.15.83:0/1345283636 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d248c01460 msgr2=0x55d248c05e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:40.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.124+0000 7f3a2a791740 1 --2- 172.21.15.83:0/1345283636 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d248c01460 0x55d248c05e80 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c0cdae0 tx=0x7f3a0c09c1c0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.124+0000 7f3a2a791740 1 -- 172.21.15.83:0/1345283636 shutdown_connections 2024-04-24T08:55:40.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.124+0000 7f3a2a791740 1 --2- 172.21.15.83:0/1345283636 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55d248c82de0 0x55d248c85200 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.124+0000 7f3a2a791740 1 --2- 172.21.15.83:0/1345283636 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d248c0bb30 0x55d248c064b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.124+0000 7f3a2a791740 1 --2- 172.21.15.83:0/1345283636 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d248c01460 0x55d248c05e80 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.124+0000 7f3a2a791740 1 --2- 172.21.15.83:0/1345283636 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d248c00b40 0x55d248c14290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.124+0000 7f3a2a791740 1 -- 172.21.15.83:0/1345283636 >> 172.21.15.83:0/1345283636 conn(0x55d248bfce60 msgr2=0x55d248c02b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:40.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.124+0000 7f3a06a00640 0 client.17091 ms_handle_reset on v2:172.21.15.189:6838/2548965336 2024-04-24T08:55:40.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.124+0000 7f3a2a791740 1 -- 172.21.15.83:0/1345283636 shutdown_connections 2024-04-24T08:55:40.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.124+0000 7f3a2a791740 1 -- 172.21.15.83:0/1345283636 wait complete. 2024-04-24T08:55:40.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.125+0000 7f3a2a791740 20 client.17091 trim_cache size 0 max 16384 2024-04-24T08:55:40.128 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T08:55:40.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.125+0000 7f3a26200640 1 -- 172.21.15.83:0/151835195 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f3a080a3d20 msgr2=0x7f3a080a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:40.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.125+0000 7f3a26200640 1 --2- 172.21.15.83:0/151835195 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f3a080a3d20 0x7f3a080a61e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f3a18049000 tx=0x7f3a1804a000 comp rx=0 tx=0).stop 2024-04-24T08:55:40.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.125+0000 7f3a26200640 1 -- 172.21.15.83:0/151835195 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a201093f0 msgr2=0x7f3a201a0b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:40.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.125+0000 7f3a26200640 1 --2- 172.21.15.83:0/151835195 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a201093f0 0x7f3a201a0b60 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c00d770 tx=0x7f3a0c00dc30 comp rx=0 tx=0).stop 2024-04-24T08:55:40.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.125+0000 7f3a26200640 1 -- 172.21.15.83:0/151835195 shutdown_connections 2024-04-24T08:55:40.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.125+0000 7f3a26200640 1 --2- 172.21.15.83:0/151835195 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f3a080a3d20 0x7f3a080a61e0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.125+0000 7f3a26200640 1 --2- 172.21.15.83:0/151835195 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a2019a6e0 0x7f3a2019ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.125+0000 7f3a26200640 1 --2- 172.21.15.83:0/151835195 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a201093f0 0x7f3a201a0b60 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.125+0000 7f3a26200640 1 --2- 172.21.15.83:0/151835195 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3a20104320 0x7f3a201a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.125+0000 7f3a26200640 1 -- 172.21.15.83:0/151835195 >> 172.21.15.83:0/151835195 conn(0x7f3a200ffb40 msgr2=0x7f3a20101ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:40.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.126+0000 7f3a26200640 1 -- 172.21.15.83:0/151835195 shutdown_connections 2024-04-24T08:55:40.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.126+0000 7f3a26200640 1 -- 172.21.15.83:0/151835195 wait complete. 2024-04-24T08:55:40.142 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:55:40 smithi083 ceph-mon[27907]: pgmap v124: 129 pgs: 129 active+clean; 32 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 302 KiB/s wr, 74 op/s 2024-04-24T08:55:40.142 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:55:40 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/2093663737' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:55:40.145 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-24T08:55:40.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.355+0000 7fe9d1a00640 1 -- 172.21.15.83:0/3730929051 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9cc102ac0 msgr2=0x7fe9cc106b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:40.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.355+0000 7fe9d1a00640 1 --2- 172.21.15.83:0/3730929051 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9cc102ac0 0x7fe9cc106b00 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fe9b8009900 tx=0x7fe9b802a410 comp rx=0 tx=0).stop 2024-04-24T08:55:40.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.355+0000 7fe9d1a00640 1 -- 172.21.15.83:0/3730929051 shutdown_connections 2024-04-24T08:55:40.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.355+0000 7fe9d1a00640 1 --2- 172.21.15.83:0/3730929051 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe9cc1071e0 0x7fe9cc10edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.355+0000 7fe9d1a00640 1 --2- 172.21.15.83:0/3730929051 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9cc102ac0 0x7fe9cc106b00 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.355+0000 7fe9d1a00640 1 --2- 172.21.15.83:0/3730929051 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe9cc102110 0x7fe9cc1024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.355+0000 7fe9d1a00640 1 -- 172.21.15.83:0/3730929051 >> 172.21.15.83:0/3730929051 conn(0x7fe9cc0fd9d0 msgr2=0x7fe9cc0ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:40.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.355+0000 7fe9d1a00640 1 -- 172.21.15.83:0/3730929051 shutdown_connections 2024-04-24T08:55:40.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.355+0000 7fe9d1a00640 1 -- 172.21.15.83:0/3730929051 wait complete. 2024-04-24T08:55:40.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.356+0000 7fe9d1a00640 1 Processor -- start 2024-04-24T08:55:40.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.356+0000 7fe9d1a00640 1 -- start start 2024-04-24T08:55:40.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.357+0000 7fe9d1a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe9cc102110 0x7fe9cc1a05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:40.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.357+0000 7fe9d1a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe9cc1071e0 0x7fe9cc1a0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:40.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.357+0000 7fe9d1a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9cc19a7e0 0x7fe9cc19ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:40.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.357+0000 7fe9d1a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe9cc112150 con 0x7fe9cc19a7e0 2024-04-24T08:55:40.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.357+0000 7fe9d1a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe9cc111fd0 con 0x7fe9cc1071e0 2024-04-24T08:55:40.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.357+0000 7fe9d1a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe9cc1122d0 con 0x7fe9cc102110 2024-04-24T08:55:40.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.357+0000 7fe9d1000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9cc19a7e0 0x7fe9cc19ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:40.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.357+0000 7fe9d1000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9cc19a7e0 0x7fe9cc19ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55418/0 (socket says 172.21.15.83:55418) 2024-04-24T08:55:40.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.357+0000 7fe9d1000640 1 -- 172.21.15.83:0/2387167171 learned_addr learned my addr 172.21.15.83:0/2387167171 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:55:40.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.357+0000 7fe9cb400640 1 --2- 172.21.15.83:0/2387167171 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe9cc1071e0 0x7fe9cc1a0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:40.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.357+0000 7fe9cbe00640 1 --2- 172.21.15.83:0/2387167171 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe9cc102110 0x7fe9cc1a05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:40.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.357+0000 7fe9d1000640 1 -- 172.21.15.83:0/2387167171 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe9cc102110 msgr2=0x7fe9cc1a05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:40.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.357+0000 7fe9d1000640 1 --2- 172.21.15.83:0/2387167171 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe9cc102110 0x7fe9cc1a05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.358+0000 7fe9d1000640 1 -- 172.21.15.83:0/2387167171 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe9cc1071e0 msgr2=0x7fe9cc1a0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:40.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.358+0000 7fe9d1000640 1 --2- 172.21.15.83:0/2387167171 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe9cc1071e0 0x7fe9cc1a0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.358+0000 7fe9d1000640 1 -- 172.21.15.83:0/2387167171 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9cc19b370 con 0x7fe9cc19a7e0 2024-04-24T08:55:40.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.358+0000 7fe9cbe00640 1 --2- 172.21.15.83:0/2387167171 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe9cc102110 0x7fe9cc1a05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:55:40.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.358+0000 7fe9d1000640 1 --2- 172.21.15.83:0/2387167171 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9cc19a7e0 0x7fe9cc19ac90 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fe9c400eaa0 tx=0x7fe9c400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:40.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.358+0000 7fe9c8c00640 1 -- 172.21.15.83:0/2387167171 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c400cdb0 con 0x7fe9cc19a7e0 2024-04-24T08:55:40.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.358+0000 7fe9c8c00640 1 -- 172.21.15.83:0/2387167171 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe9c4005020 con 0x7fe9cc19a7e0 2024-04-24T08:55:40.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.358+0000 7fe9d1a00640 1 -- 172.21.15.83:0/2387167171 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9cc19b660 con 0x7fe9cc19a7e0 2024-04-24T08:55:40.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.358+0000 7fe9c8c00640 1 -- 172.21.15.83:0/2387167171 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c40116d0 con 0x7fe9cc19a7e0 2024-04-24T08:55:40.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.358+0000 7fe9d1a00640 1 -- 172.21.15.83:0/2387167171 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9cc1a7720 con 0x7fe9cc19a7e0 2024-04-24T08:55:40.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.361+0000 7fe9d5fb4740 1 Processor -- start 2024-04-24T08:55:40.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.362+0000 7fe9c8c00640 1 -- 172.21.15.83:0/2387167171 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe9c40051d0 con 0x7fe9cc19a7e0 2024-04-24T08:55:40.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.362+0000 7fe9d5fb4740 1 -- start start 2024-04-24T08:55:40.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.362+0000 7fe9d5fb4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f506ce2b40 0x55f506ce2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:40.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.362+0000 7fe9d5fb4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f506ce3460 0x55f506ced5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:40.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.362+0000 7fe9d5fb4740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f506cedb30 0x55f506ceff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:40.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.362+0000 7fe9d5fb4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55f506c481d0 con 0x55f506ce3460 2024-04-24T08:55:40.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.362+0000 7fe9d5fb4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f506bee450 con 0x55f506ce2b40 2024-04-24T08:55:40.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.362+0000 7fe9d5fb4740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55f506c44190 con 0x55f506cedb30 2024-04-24T08:55:40.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.363+0000 7fe9cbe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f506ce3460 0x55f506ced5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:40.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.363+0000 7fe9cbe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f506ce3460 0x55f506ced5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55424/0 (socket says 172.21.15.83:55424) 2024-04-24T08:55:40.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.363+0000 7fe9cbe00640 1 -- 172.21.15.83:0/2106513731 learned_addr learned my addr 172.21.15.83:0/2106513731 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:55:40.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.363+0000 7fe9cb400640 1 --2- 172.21.15.83:0/2106513731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f506ce2b40 0x55f506ce2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:40.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.363+0000 7fe9cb400640 1 --2- 172.21.15.83:0/2106513731 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f506cedb30 0x55f506ceff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:40.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.363+0000 7fe9cbe00640 1 -- 172.21.15.83:0/2106513731 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f506cedb30 msgr2=0x55f506ceff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:40.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.363+0000 7fe9cbe00640 1 --2- 172.21.15.83:0/2106513731 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f506cedb30 0x55f506ceff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.363+0000 7fe9cbe00640 1 -- 172.21.15.83:0/2106513731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f506ce2b40 msgr2=0x55f506ce2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:40.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.363+0000 7fe9cbe00640 1 --2- 172.21.15.83:0/2106513731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f506ce2b40 0x55f506ce2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.363+0000 7fe9cbe00640 1 -- 172.21.15.83:0/2106513731 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f506cab310 con 0x55f506ce3460 2024-04-24T08:55:40.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.363+0000 7fe9cb400640 1 --2- 172.21.15.83:0/2106513731 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f506cedb30 0x55f506ceff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:55:40.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.364+0000 7fe9cbe00640 1 --2- 172.21.15.83:0/2106513731 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f506ce3460 0x55f506ced5f0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc00bb70 tx=0x7fe9bc00d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d8d69ab87f506b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:40.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.364+0000 7fe9c8c00640 1 --2- 172.21.15.83:0/2387167171 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe9b00a3d20 0x7fe9b00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:40.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.364+0000 7fe9c1600640 1 -- 172.21.15.83:0/2106513731 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9bc017620 con 0x55f506ce3460 2024-04-24T08:55:40.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.364+0000 7fe9c8c00640 1 -- 172.21.15.83:0/2387167171 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7fe9c40d0e00 con 0x7fe9cc19a7e0 2024-04-24T08:55:40.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.364+0000 7fe9c1600640 1 -- 172.21.15.83:0/2106513731 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe9bc017c70 con 0x55f506ce3460 2024-04-24T08:55:40.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.364+0000 7fe9c1600640 1 -- 172.21.15.83:0/2106513731 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9bc00fb40 con 0x55f506ce3460 2024-04-24T08:55:40.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.364+0000 7fe9cb400640 1 --2- 172.21.15.83:0/2387167171 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe9b00a3d20 0x7fe9b00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:40.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.365+0000 7fe9d5fb4740 1 -- 172.21.15.83:0/2106513731 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f506ce3460 msgr2=0x55f506ced5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:40.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.365+0000 7fe9d5fb4740 1 --2- 172.21.15.83:0/2106513731 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f506ce3460 0x55f506ced5f0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc00bb70 tx=0x7fe9bc00d710 comp rx=0 tx=0).stop 2024-04-24T08:55:40.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.365+0000 7fe9cb400640 1 --2- 172.21.15.83:0/2387167171 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe9b00a3d20 0x7fe9b00a61e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe9b8006010 tx=0x7fe9b8000bc0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:40.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.365+0000 7fe9d5fb4740 1 -- 172.21.15.83:0/2106513731 shutdown_connections 2024-04-24T08:55:40.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.365+0000 7fe9d5fb4740 1 --2- 172.21.15.83:0/2106513731 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f506cedb30 0x55f506ceff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.365+0000 7fe9d5fb4740 1 --2- 172.21.15.83:0/2106513731 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f506ce3460 0x55f506ced5f0 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.365+0000 7fe9d5fb4740 1 --2- 172.21.15.83:0/2106513731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f506ce2b40 0x55f506ce2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.365+0000 7fe9d5fb4740 1 -- 172.21.15.83:0/2106513731 >> 172.21.15.83:0/2106513731 conn(0x55f506cdee60 msgr2=0x55f506cdf240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:40.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.365+0000 7fe9d5fb4740 1 -- 172.21.15.83:0/2106513731 shutdown_connections 2024-04-24T08:55:40.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.365+0000 7fe9d5fb4740 1 -- 172.21.15.83:0/2106513731 wait complete. 2024-04-24T08:55:40.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.366+0000 7fe9d5fb4740 1 -- start start 2024-04-24T08:55:40.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.366+0000 7fe9d5fb4740 1 Processor -- start 2024-04-24T08:55:40.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.367+0000 7fe9d5fb4740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f506ce2b40 0x55f506ced5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:40.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.367+0000 7fe9d5fb4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f506ce3460 0x55f506cea8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:40.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.367+0000 7fe9d5fb4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f506cedb30 0x55f506ceb010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:40.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.367+0000 7fe9d5fb4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55f506cf09c0 con 0x55f506cedb30 2024-04-24T08:55:40.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.367+0000 7fe9d5fb4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f506c01810 con 0x55f506ce3460 2024-04-24T08:55:40.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.367+0000 7fe9d5fb4740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55f506cf0b40 con 0x55f506ce2b40 2024-04-24T08:55:40.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.367+0000 7fe9d5fb4740 10 client.0 _ll_register_callbacks cb 0x55f506bcf140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:55:40.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.367+0000 7fe9d1000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f506cedb30 0x55f506ceb010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:40.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.367+0000 7fe9d1000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f506cedb30 0x55f506ceb010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55428/0 (socket says 172.21.15.83:55428) 2024-04-24T08:55:40.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.367+0000 7fe9d1000640 1 -- 172.21.15.83:0/1856420872 learned_addr learned my addr 172.21.15.83:0/1856420872 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:55:40.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.367+0000 7fe9cbe00640 1 --2- 172.21.15.83:0/1856420872 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f506ce2b40 0x55f506ced5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:40.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.367+0000 7fe9cb400640 1 --2- 172.21.15.83:0/1856420872 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f506ce3460 0x55f506cea8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:40.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.367+0000 7fe9d1000640 1 -- 172.21.15.83:0/1856420872 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f506ce2b40 msgr2=0x55f506ced5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:40.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.367+0000 7fe9d1000640 1 --2- 172.21.15.83:0/1856420872 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f506ce2b40 0x55f506ced5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.367+0000 7fe9d1000640 1 -- 172.21.15.83:0/1856420872 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f506ce3460 msgr2=0x55f506cea8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:40.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.367+0000 7fe9d1000640 1 --2- 172.21.15.83:0/1856420872 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f506ce3460 0x55f506cea8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.367+0000 7fe9d1000640 1 -- 172.21.15.83:0/1856420872 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f506c123f0 con 0x55f506cedb30 2024-04-24T08:55:40.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.368+0000 7fe9d1000640 1 --2- 172.21.15.83:0/1856420872 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f506cedb30 0x55f506ceb010 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fe9c40ce780 tx=0x7fe9c409c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26b4f67c4a715ed8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:40.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.368+0000 7fe9b6000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:55:40.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.368+0000 7fe9b6000640 1 -- 172.21.15.83:0/1856420872 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c401f040 con 0x55f506cedb30 2024-04-24T08:55:40.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.368+0000 7fe9b6000640 1 -- 172.21.15.83:0/1856420872 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7fe9c40bbb90 con 0x55f506cedb30 2024-04-24T08:55:40.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.368+0000 7fe9d5fb4740 1 -- 172.21.15.83:0/1856420872 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe9bc017c70 con 0x55f506cedb30 2024-04-24T08:55:40.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.368+0000 7fe9b6000640 1 -- 172.21.15.83:0/1856420872 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe9c40a5640 con 0x55f506cedb30 2024-04-24T08:55:40.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.368+0000 7fe9d5fb4740 10 client.17103 fetch_fsmap learned FSMap version 30 2024-04-24T08:55:40.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.368+0000 7fe9d5fb4740 1 -- 172.21.15.83:0/1856420872 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f50695a3f0 con 0x55f506cedb30 2024-04-24T08:55:40.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.368+0000 7fe9b6000640 1 -- 172.21.15.83:0/1856420872 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe9bc017c70 con 0x55f506cedb30 2024-04-24T08:55:40.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.369+0000 7fe9b6000640 1 -- 172.21.15.83:0/1856420872 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7fe9c400f040 con 0x55f506cedb30 2024-04-24T08:55:40.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.369+0000 7fe9d5fb4740 10 client.17103 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T08:55:40.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.369+0000 7fe9d5fb4740 10 client.17103 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-24T08:55:40.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.369+0000 7fe9d5fb4740 20 client.17103 populate_metadata read hostname 'smithi083' 2024-04-24T08:55:40.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.369+0000 7fe9d5fb4740 1 --2- 172.21.15.83:0/1856420872 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55f506cfdc80 0x55f506d00120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:40.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.369+0000 7fe9d5fb4740 4 client.17103 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:55:40.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.369+0000 7fe9d5fb4740 1 -- 172.21.15.83:0/1856420872 --> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f506c55bf0 con 0x55f506cfdc80 2024-04-24T08:55:40.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.370+0000 7fe9cbe00640 1 --2- 172.21.15.83:0/1856420872 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55f506cfdc80 0x55f506d00120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:40.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.370+0000 7fe9cbe00640 1 --2- 172.21.15.83:0/1856420872 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55f506cfdc80 0x55f506d00120 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=fe08bfd668b60e37 server_cookie=c0a71b49b347d5d9 in_seq=0 out_seq=0 2024-04-24T08:55:40.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.371+0000 7fe9b6000640 10 client.17103 ms_handle_connect on v2:172.21.15.138:6832/1157967658 2024-04-24T08:55:40.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.372+0000 7fe9b6000640 1 -- 172.21.15.83:0/1856420872 <== mds.3 v2:172.21.15.138:6832/1157967658 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f506c55bf0 con 0x55f506cfdc80 2024-04-24T08:55:40.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.372+0000 7fe9b6000640 10 client.17103 handle_command_reply: tid=0 2024-04-24T08:55:40.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.372+0000 7fe9d5fb4740 1 client.17103 shutdown 2024-04-24T08:55:40.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.373+0000 7fe9d5fb4740 1 -- 172.21.15.83:0/1856420872 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f506cedb30 msgr2=0x55f506ceb010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:40.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.373+0000 7fe9d5fb4740 1 --2- 172.21.15.83:0/1856420872 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f506cedb30 0x55f506ceb010 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fe9c40ce780 tx=0x7fe9c409c9d0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.373+0000 7fe9d5fb4740 1 -- 172.21.15.83:0/1856420872 shutdown_connections 2024-04-24T08:55:40.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.373+0000 7fe9d5fb4740 1 --2- 172.21.15.83:0/1856420872 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f506cedb30 0x55f506ceb010 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.373+0000 7fe9d5fb4740 1 --2- 172.21.15.83:0/1856420872 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f506ce3460 0x55f506cea8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.373+0000 7fe9d5fb4740 1 --2- 172.21.15.83:0/1856420872 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55f506cfdc80 0x55f506d00120 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.373+0000 7fe9d5fb4740 1 --2- 172.21.15.83:0/1856420872 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f506ce2b40 0x55f506ced5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.373+0000 7fe9d5fb4740 1 -- 172.21.15.83:0/1856420872 >> 172.21.15.83:0/1856420872 conn(0x55f506cdee60 msgr2=0x55f506ce4c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:40.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.373+0000 7fe9b6000640 0 client.17103 ms_handle_reset on v2:172.21.15.138:6832/1157967658 2024-04-24T08:55:40.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.373+0000 7fe9d5fb4740 1 -- 172.21.15.83:0/1856420872 shutdown_connections 2024-04-24T08:55:40.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.373+0000 7fe9d5fb4740 1 -- 172.21.15.83:0/1856420872 wait complete. 2024-04-24T08:55:40.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.373+0000 7fe9d5fb4740 20 client.17103 trim_cache size 0 max 16384 2024-04-24T08:55:40.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.389+0000 7fe9d5fb4740 1 Processor -- start 2024-04-24T08:55:40.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.389+0000 7fe9d5fb4740 1 -- start start 2024-04-24T08:55:40.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.389+0000 7fe9d5fb4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f506ce2b40 0x55f506cde6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:40.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.389+0000 7fe9d5fb4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f506ce3460 0x55f506ce98b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:40.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.389+0000 7fe9d5fb4740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f506cedb30 0x55f506ced110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:40.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.389+0000 7fe9d5fb4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe9cc1b1550 con 0x55f506ce3460 2024-04-24T08:55:40.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.389+0000 7fe9d5fb4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe9cc1b3120 con 0x55f506ce2b40 2024-04-24T08:55:40.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.389+0000 7fe9d5fb4740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe9cc1b1c20 con 0x55f506cedb30 2024-04-24T08:55:40.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.390+0000 7fe9cb400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f506ce3460 0x55f506ce98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:40.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.390+0000 7fe9cb400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f506ce3460 0x55f506ce98b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55440/0 (socket says 172.21.15.83:55440) 2024-04-24T08:55:40.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.390+0000 7fe9cb400640 1 -- 172.21.15.83:0/3835178211 learned_addr learned my addr 172.21.15.83:0/3835178211 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:55:40.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.390+0000 7fe9d1000640 1 --2- 172.21.15.83:0/3835178211 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f506cedb30 0x55f506ced110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:40.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.390+0000 7fe9cbe00640 1 --2- 172.21.15.83:0/3835178211 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f506ce2b40 0x55f506cde6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:40.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.390+0000 7fe9cb400640 1 -- 172.21.15.83:0/3835178211 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f506cedb30 msgr2=0x55f506ced110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:40.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.390+0000 7fe9cb400640 1 --2- 172.21.15.83:0/3835178211 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f506cedb30 0x55f506ced110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.390+0000 7fe9cb400640 1 -- 172.21.15.83:0/3835178211 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f506ce2b40 msgr2=0x55f506cde6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:40.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.390+0000 7fe9cb400640 1 --2- 172.21.15.83:0/3835178211 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f506ce2b40 0x55f506cde6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.390+0000 7fe9cb400640 1 -- 172.21.15.83:0/3835178211 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f506a247d0 con 0x55f506ce3460 2024-04-24T08:55:40.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.391+0000 7fe9cb400640 1 --2- 172.21.15.83:0/3835178211 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f506ce3460 0x55f506ce98b0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fe9b8009900 tx=0x7fe9b803e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d60db202e226fb0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:40.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.391+0000 7fe9c0c00640 1 -- 172.21.15.83:0/3835178211 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9b8035040 con 0x55f506ce3460 2024-04-24T08:55:40.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.391+0000 7fe9c0c00640 1 -- 172.21.15.83:0/3835178211 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe9b8004ca0 con 0x55f506ce3460 2024-04-24T08:55:40.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.391+0000 7fe9c0c00640 1 -- 172.21.15.83:0/3835178211 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9b803d6b0 con 0x55f506ce3460 2024-04-24T08:55:40.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.392+0000 7fe9d5fb4740 1 -- 172.21.15.83:0/3835178211 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f506ce3460 msgr2=0x55f506ce98b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:40.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.392+0000 7fe9d5fb4740 1 --2- 172.21.15.83:0/3835178211 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f506ce3460 0x55f506ce98b0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fe9b8009900 tx=0x7fe9b803e000 comp rx=0 tx=0).stop 2024-04-24T08:55:40.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.392+0000 7fe9d5fb4740 1 -- 172.21.15.83:0/3835178211 shutdown_connections 2024-04-24T08:55:40.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.392+0000 7fe9d5fb4740 1 --2- 172.21.15.83:0/3835178211 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f506cedb30 0x55f506ced110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.392+0000 7fe9d5fb4740 1 --2- 172.21.15.83:0/3835178211 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f506ce3460 0x55f506ce98b0 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.392+0000 7fe9d5fb4740 1 --2- 172.21.15.83:0/3835178211 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f506ce2b40 0x55f506cde6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.392+0000 7fe9d5fb4740 1 -- 172.21.15.83:0/3835178211 >> 172.21.15.83:0/3835178211 conn(0x55f506cdee60 msgr2=0x55f506ce91a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:40.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.392+0000 7fe9d5fb4740 1 -- 172.21.15.83:0/3835178211 shutdown_connections 2024-04-24T08:55:40.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.392+0000 7fe9d5fb4740 1 -- 172.21.15.83:0/3835178211 wait complete. 2024-04-24T08:55:40.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.392+0000 7fe9d5fb4740 1 -- start start 2024-04-24T08:55:40.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.392+0000 7fe9d5fb4740 1 Processor -- start 2024-04-24T08:55:40.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.393+0000 7fe9d5fb4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f506ce2b40 0x55f506cee340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:40.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.393+0000 7fe9d5fb4740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f506ce3460 0x55f506cee880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:40.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.393+0000 7fe9d5fb4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f506cedb30 0x55f506cf4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:40.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.393+0000 7fe9d5fb4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe9cc1b29c0 con 0x55f506cedb30 2024-04-24T08:55:40.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.393+0000 7fe9d5fb4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe9cc1b22f0 con 0x55f506ce2b40 2024-04-24T08:55:40.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.393+0000 7fe9d5fb4740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe9cc1af530 con 0x55f506ce3460 2024-04-24T08:55:40.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.393+0000 7fe9d5fb4740 10 client.0 _ll_register_callbacks cb 0x7fe9c4005830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:55:40.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.393+0000 7fe9d1000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f506cedb30 0x55f506cf4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:40.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.393+0000 7fe9d1000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f506cedb30 0x55f506cf4d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55454/0 (socket says 172.21.15.83:55454) 2024-04-24T08:55:40.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.393+0000 7fe9d1000640 1 -- 172.21.15.83:0/2715265854 learned_addr learned my addr 172.21.15.83:0/2715265854 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:55:40.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.394+0000 7fe9cbe00640 1 --2- 172.21.15.83:0/2715265854 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f506ce2b40 0x55f506cee340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:40.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.394+0000 7fe9d1000640 1 -- 172.21.15.83:0/2715265854 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f506ce3460 msgr2=0x55f506cee880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:40.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.394+0000 7fe9d1000640 1 --2- 172.21.15.83:0/2715265854 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f506ce3460 0x55f506cee880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.394+0000 7fe9d1000640 1 -- 172.21.15.83:0/2715265854 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f506ce2b40 msgr2=0x55f506cee340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:40.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.394+0000 7fe9d1000640 1 --2- 172.21.15.83:0/2715265854 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f506ce2b40 0x55f506cee340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.394+0000 7fe9d1000640 1 -- 172.21.15.83:0/2715265854 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f506cf52d0 con 0x55f506cedb30 2024-04-24T08:55:40.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.394+0000 7fe9d1000640 1 --2- 172.21.15.83:0/2715265854 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f506cedb30 0x55f506cf4d90 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fe9c40058e0 tx=0x7fe9c409c030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a4ecd67ede3f4b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:40.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.394+0000 7fe9b6000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:55:40.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.394+0000 7fe9b6000640 1 -- 172.21.15.83:0/2715265854 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c401f070 con 0x55f506cedb30 2024-04-24T08:55:40.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.394+0000 7fe9b6000640 1 -- 172.21.15.83:0/2715265854 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7fe9c40bbd90 con 0x55f506cedb30 2024-04-24T08:55:40.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.394+0000 7fe9d5fb4740 1 -- 172.21.15.83:0/2715265854 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe9b8004ca0 con 0x55f506cedb30 2024-04-24T08:55:40.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.394+0000 7fe9b6000640 1 -- 172.21.15.83:0/2715265854 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe9c409c7a0 con 0x55f506cedb30 2024-04-24T08:55:40.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.394+0000 7fe9d5fb4740 10 client.17109 fetch_fsmap learned FSMap version 30 2024-04-24T08:55:40.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.394+0000 7fe9d5fb4740 1 -- 172.21.15.83:0/2715265854 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f506cf65d0 con 0x55f506cedb30 2024-04-24T08:55:40.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.395+0000 7fe9b6000640 1 -- 172.21.15.83:0/2715265854 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe9b8004ca0 con 0x55f506cedb30 2024-04-24T08:55:40.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.395+0000 7fe9b6000640 1 -- 172.21.15.83:0/2715265854 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7fe9c409b9b0 con 0x55f506cedb30 2024-04-24T08:55:40.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.395+0000 7fe9d5fb4740 10 client.17109 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T08:55:40.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.395+0000 7fe9d5fb4740 10 client.17109 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-24T08:55:40.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.395+0000 7fe9d5fb4740 20 client.17109 populate_metadata read hostname 'smithi083' 2024-04-24T08:55:40.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.395+0000 7fe9d5fb4740 1 --2- 172.21.15.83:0/2715265854 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55f506d64670 0x55f506d66a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:40.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.395+0000 7fe9d5fb4740 4 client.17109 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T08:55:40.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.395+0000 7fe9d5fb4740 1 -- 172.21.15.83:0/2715265854 --> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f506c4fa60 con 0x55f506d64670 2024-04-24T08:55:40.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.395+0000 7fe9cbe00640 1 --2- 172.21.15.83:0/2715265854 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55f506d64670 0x55f506d66a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:40.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.397+0000 7fe9cbe00640 1 --2- 172.21.15.83:0/2715265854 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55f506d64670 0x55f506d66a90 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9c8988ea20d3ff66 server_cookie=5aae2b13d6651365 in_seq=0 out_seq=0 2024-04-24T08:55:40.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.397+0000 7fe9b6000640 10 client.17109 ms_handle_connect on v2:172.21.15.138:6832/1157967658 2024-04-24T08:55:40.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.397+0000 7fe9b6000640 1 -- 172.21.15.83:0/2715265854 <== mds.3 v2:172.21.15.138:6832/1157967658 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f506c4fa60 con 0x55f506d64670 2024-04-24T08:55:40.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.397+0000 7fe9b6000640 10 client.17109 handle_command_reply: tid=0 2024-04-24T08:55:40.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.397+0000 7fe9d5fb4740 1 client.17109 shutdown 2024-04-24T08:55:40.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.397+0000 7fe9d5fb4740 1 -- 172.21.15.83:0/2715265854 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f506cedb30 msgr2=0x55f506cf4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:40.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.397+0000 7fe9d5fb4740 1 --2- 172.21.15.83:0/2715265854 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f506cedb30 0x55f506cf4d90 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fe9c40058e0 tx=0x7fe9c409c030 comp rx=0 tx=0).stop 2024-04-24T08:55:40.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.397+0000 7fe9d5fb4740 1 -- 172.21.15.83:0/2715265854 shutdown_connections 2024-04-24T08:55:40.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.397+0000 7fe9d5fb4740 1 --2- 172.21.15.83:0/2715265854 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f506cedb30 0x55f506cf4d90 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.397+0000 7fe9d5fb4740 1 --2- 172.21.15.83:0/2715265854 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55f506d64670 0x55f506d66a90 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.397+0000 7fe9d5fb4740 1 --2- 172.21.15.83:0/2715265854 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f506ce3460 0x55f506cee880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.397+0000 7fe9d5fb4740 1 --2- 172.21.15.83:0/2715265854 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f506ce2b40 0x55f506cee340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.397+0000 7fe9d5fb4740 1 -- 172.21.15.83:0/2715265854 >> 172.21.15.83:0/2715265854 conn(0x55f506cdee60 msgr2=0x55f506ce4c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:40.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.397+0000 7fe9b6000640 0 client.17109 ms_handle_reset on v2:172.21.15.138:6832/1157967658 2024-04-24T08:55:40.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.398+0000 7fe9d5fb4740 1 -- 172.21.15.83:0/2715265854 shutdown_connections 2024-04-24T08:55:40.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.398+0000 7fe9d5fb4740 1 -- 172.21.15.83:0/2715265854 wait complete. 2024-04-24T08:55:40.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.398+0000 7fe9d5fb4740 20 client.17109 trim_cache size 0 max 16384 2024-04-24T08:55:40.401 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T08:55:40.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.398+0000 7fe9d1a00640 1 -- 172.21.15.83:0/2387167171 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe9b00a3d20 msgr2=0x7fe9b00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:40.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.398+0000 7fe9d1a00640 1 --2- 172.21.15.83:0/2387167171 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe9b00a3d20 0x7fe9b00a61e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe9b8006010 tx=0x7fe9b8000bc0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.398+0000 7fe9d1a00640 1 -- 172.21.15.83:0/2387167171 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9cc19a7e0 msgr2=0x7fe9cc19ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:40.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.399+0000 7fe9d1a00640 1 --2- 172.21.15.83:0/2387167171 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9cc19a7e0 0x7fe9cc19ac90 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fe9c400eaa0 tx=0x7fe9c400ef60 comp rx=0 tx=0).stop 2024-04-24T08:55:40.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.399+0000 7fe9d1a00640 1 -- 172.21.15.83:0/2387167171 shutdown_connections 2024-04-24T08:55:40.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.399+0000 7fe9d1a00640 1 --2- 172.21.15.83:0/2387167171 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe9b00a3d20 0x7fe9b00a61e0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.399+0000 7fe9d1a00640 1 --2- 172.21.15.83:0/2387167171 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9cc19a7e0 0x7fe9cc19ac90 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.399+0000 7fe9d1a00640 1 --2- 172.21.15.83:0/2387167171 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe9cc1071e0 0x7fe9cc1a0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.399+0000 7fe9d1a00640 1 --2- 172.21.15.83:0/2387167171 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe9cc102110 0x7fe9cc1a05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.399+0000 7fe9d1a00640 1 -- 172.21.15.83:0/2387167171 >> 172.21.15.83:0/2387167171 conn(0x7fe9cc0fd9d0 msgr2=0x7fe9cc0fede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:40.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.399+0000 7fe9d1a00640 1 -- 172.21.15.83:0/2387167171 shutdown_connections 2024-04-24T08:55:40.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.399+0000 7fe9d1a00640 1 -- 172.21.15.83:0/2387167171 wait complete. 2024-04-24T08:55:40.417 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T08:55:40.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:55:40 smithi138 ceph-mon[31441]: pgmap v124: 129 pgs: 129 active+clean; 32 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 302 KiB/s wr, 74 op/s 2024-04-24T08:55:40.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:55:40 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/2093663737' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:55:40.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:55:40 smithi189 ceph-mon[31791]: pgmap v124: 129 pgs: 129 active+clean; 32 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 302 KiB/s wr, 74 op/s 2024-04-24T08:55:40.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:55:40 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/2093663737' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:55:40.569 INFO:tasks.workunit.client.0.smithi083.stdout: 1 16606 0.50 MB/sec execute 14 sec latency 46.858 ms 2024-04-24T08:55:40.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.629+0000 7f9390200640 1 -- 172.21.15.83:0/3036953189 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9388103ba0 msgr2=0x7f938810c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:40.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.629+0000 7f9390200640 1 --2- 172.21.15.83:0/3036953189 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9388103ba0 0x7f938810c080 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f9374009ca0 tx=0x7f937402a9e0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.629+0000 7f9390200640 1 -- 172.21.15.83:0/3036953189 shutdown_connections 2024-04-24T08:55:40.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.629+0000 7f9390200640 1 --2- 172.21.15.83:0/3036953189 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f938810c5c0 0x7f938810ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.629+0000 7f9390200640 1 --2- 172.21.15.83:0/3036953189 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9388103ba0 0x7f938810c080 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.629+0000 7f9390200640 1 --2- 172.21.15.83:0/3036953189 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9388103280 0x7f9388103660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.629+0000 7f9390200640 1 -- 172.21.15.83:0/3036953189 >> 172.21.15.83:0/3036953189 conn(0x7f9388078000 msgr2=0x7f9388078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:40.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.629+0000 7f9390200640 1 -- 172.21.15.83:0/3036953189 shutdown_connections 2024-04-24T08:55:40.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.629+0000 7f9390200640 1 -- 172.21.15.83:0/3036953189 wait complete. 2024-04-24T08:55:40.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.630+0000 7f9390200640 1 Processor -- start 2024-04-24T08:55:40.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.630+0000 7f9390200640 1 -- start start 2024-04-24T08:55:40.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.630+0000 7f9390200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9388103280 0x7f93881a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:40.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.631+0000 7f9390200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9388103ba0 0x7f93881a0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:40.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.631+0000 7f9390200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f938810c5c0 0x7f938819a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:40.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.631+0000 7f9390200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f93881122a0 con 0x7f9388103ba0 2024-04-24T08:55:40.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.631+0000 7f9390200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9388112120 con 0x7f938810c5c0 2024-04-24T08:55:40.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.631+0000 7f9390200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9388112420 con 0x7f9388103280 2024-04-24T08:55:40.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.631+0000 7f938ce00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9388103ba0 0x7f93881a0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:40.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.631+0000 7f938ce00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9388103ba0 0x7f93881a0c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55484/0 (socket says 172.21.15.83:55484) 2024-04-24T08:55:40.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.631+0000 7f938ce00640 1 -- 172.21.15.83:0/3958150281 learned_addr learned my addr 172.21.15.83:0/3958150281 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:55:40.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.631+0000 7f938d800640 1 --2- 172.21.15.83:0/3958150281 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9388103280 0x7f93881a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:40.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.631+0000 7f938e200640 1 --2- 172.21.15.83:0/3958150281 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f938810c5c0 0x7f938819a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:40.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.631+0000 7f938ce00640 1 -- 172.21.15.83:0/3958150281 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9388103280 msgr2=0x7f93881a06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:40.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.631+0000 7f938ce00640 1 --2- 172.21.15.83:0/3958150281 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9388103280 0x7f93881a06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.631+0000 7f938ce00640 1 -- 172.21.15.83:0/3958150281 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f938810c5c0 msgr2=0x7f938819a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:40.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.631+0000 7f938ce00640 1 --2- 172.21.15.83:0/3958150281 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f938810c5c0 0x7f938819a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.631+0000 7f938ce00640 1 -- 172.21.15.83:0/3958150281 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f938819b050 con 0x7f9388103ba0 2024-04-24T08:55:40.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.631+0000 7f938e200640 1 --2- 172.21.15.83:0/3958150281 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f938810c5c0 0x7f938819a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:55:40.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.632+0000 7f938ce00640 1 --2- 172.21.15.83:0/3958150281 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9388103ba0 0x7f93881a0c00 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f9374004af0 tx=0x7f937402fbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:40.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.632+0000 7f9386000640 1 -- 172.21.15.83:0/3958150281 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9374033040 con 0x7f9388103ba0 2024-04-24T08:55:40.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.632+0000 7f9386000640 1 -- 172.21.15.83:0/3958150281 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f937402e8a0 con 0x7f9388103ba0 2024-04-24T08:55:40.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.632+0000 7f9386000640 1 -- 172.21.15.83:0/3958150281 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93740427c0 con 0x7f9388103ba0 2024-04-24T08:55:40.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.632+0000 7f9390200640 1 -- 172.21.15.83:0/3958150281 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f938819b2e0 con 0x7f9388103ba0 2024-04-24T08:55:40.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.633+0000 7f9390200640 1 -- 172.21.15.83:0/3958150281 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93881a7510 con 0x7f9388103ba0 2024-04-24T08:55:40.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.637+0000 7f9393180740 1 Processor -- start 2024-04-24T08:55:40.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.637+0000 7f9393180740 1 -- start start 2024-04-24T08:55:40.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.637+0000 7f9393180740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5586e99a4950 0x5586e99a4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:40.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.637+0000 7f9393180740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5586e99a5270 0x5586e99af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:40.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.637+0000 7f9386000640 1 -- 172.21.15.83:0/3958150281 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f937404a060 con 0x7f9388103ba0 2024-04-24T08:55:40.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.637+0000 7f9393180740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5586e99af940 0x5586e99b1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:40.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.637+0000 7f9393180740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5586e990a1d0 con 0x5586e99af940 2024-04-24T08:55:40.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.637+0000 7f9393180740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5586e98b0450 con 0x5586e99a5270 2024-04-24T08:55:40.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.637+0000 7f9393180740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5586e9906190 con 0x5586e99a4950 2024-04-24T08:55:40.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.637+0000 7f938ce00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5586e99af940 0x5586e99b1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:40.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.638+0000 7f938ce00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5586e99af940 0x5586e99b1d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55496/0 (socket says 172.21.15.83:55496) 2024-04-24T08:55:40.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.638+0000 7f938ce00640 1 -- 172.21.15.83:0/3060754318 learned_addr learned my addr 172.21.15.83:0/3060754318 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:55:40.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.638+0000 7f938e200640 1 --2- 172.21.15.83:0/3060754318 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5586e99a4950 0x5586e99a4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:40.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.638+0000 7f938d800640 1 --2- 172.21.15.83:0/3060754318 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5586e99a5270 0x5586e99af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:40.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.638+0000 7f938ce00640 1 -- 172.21.15.83:0/3060754318 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5586e99a4950 msgr2=0x5586e99a4d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:40.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.638+0000 7f938ce00640 1 --2- 172.21.15.83:0/3060754318 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5586e99a4950 0x5586e99a4d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.638+0000 7f938ce00640 1 -- 172.21.15.83:0/3060754318 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5586e99a5270 msgr2=0x5586e99af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:40.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.638+0000 7f938ce00640 1 --2- 172.21.15.83:0/3060754318 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5586e99a5270 0x5586e99af400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.638+0000 7f938ce00640 1 -- 172.21.15.83:0/3060754318 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586e996d310 con 0x5586e99af940 2024-04-24T08:55:40.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.639+0000 7f938ce00640 1 --2- 172.21.15.83:0/3060754318 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5586e99af940 0x5586e99b1d30 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f93740eeb80 tx=0x7f93740f7b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4efdd1d0b7e882b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:40.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.639+0000 7f937ea00640 1 -- 172.21.15.83:0/3060754318 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9374100070 con 0x5586e99af940 2024-04-24T08:55:40.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.639+0000 7f937ea00640 1 -- 172.21.15.83:0/3060754318 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f93740fc420 con 0x5586e99af940 2024-04-24T08:55:40.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.639+0000 7f937ea00640 1 -- 172.21.15.83:0/3060754318 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93740f3310 con 0x5586e99af940 2024-04-24T08:55:40.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.639+0000 7f9386000640 1 --2- 172.21.15.83:0/3958150281 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f936c0a3da0 0x7f936c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:40.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.639+0000 7f9386000640 1 -- 172.21.15.83:0/3958150281 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f93740f1920 con 0x7f9388103ba0 2024-04-24T08:55:40.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.639+0000 7f938e200640 1 --2- 172.21.15.83:0/3958150281 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f936c0a3da0 0x7f936c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:40.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.640+0000 7f9393180740 1 -- 172.21.15.83:0/3060754318 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5586e99af940 msgr2=0x5586e99b1d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:40.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.640+0000 7f9393180740 1 --2- 172.21.15.83:0/3060754318 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5586e99af940 0x5586e99b1d30 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f93740eeb80 tx=0x7f93740f7b20 comp rx=0 tx=0).stop 2024-04-24T08:55:40.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.640+0000 7f9393180740 1 -- 172.21.15.83:0/3060754318 shutdown_connections 2024-04-24T08:55:40.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.640+0000 7f9393180740 1 --2- 172.21.15.83:0/3060754318 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5586e99af940 0x5586e99b1d30 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.640+0000 7f9393180740 1 --2- 172.21.15.83:0/3060754318 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5586e99a5270 0x5586e99af400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.640+0000 7f9393180740 1 --2- 172.21.15.83:0/3060754318 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5586e99a4950 0x5586e99a4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.640+0000 7f9393180740 1 -- 172.21.15.83:0/3060754318 >> 172.21.15.83:0/3060754318 conn(0x5586e99a0c70 msgr2=0x5586e99a1050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:40.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.640+0000 7f9393180740 1 -- 172.21.15.83:0/3060754318 shutdown_connections 2024-04-24T08:55:40.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.640+0000 7f938e200640 1 --2- 172.21.15.83:0/3958150281 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f936c0a3da0 0x7f936c0a6260 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9378009c70 tx=0x7f93780093d0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:40.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.640+0000 7f9393180740 1 -- 172.21.15.83:0/3060754318 wait complete. 2024-04-24T08:55:40.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.641+0000 7f9393180740 1 -- start start 2024-04-24T08:55:40.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.641+0000 7f9393180740 1 Processor -- start 2024-04-24T08:55:40.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.641+0000 7f9393180740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5586e99a4950 0x5586e99af2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:40.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.641+0000 7f9393180740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5586e99a5270 0x5586e99ac650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:40.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.641+0000 7f9393180740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5586e99af940 0x5586e99acc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:40.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.641+0000 7f9393180740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5586e990a1d0 con 0x5586e99a5270 2024-04-24T08:55:40.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.641+0000 7f9393180740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5586e98c3810 con 0x5586e99a4950 2024-04-24T08:55:40.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.641+0000 7f9393180740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5586e99b24f0 con 0x5586e99af940 2024-04-24T08:55:40.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.641+0000 7f9393180740 10 client.0 _ll_register_callbacks cb 0x5586e9891140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:55:40.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.642+0000 7f938ce00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5586e99a5270 0x5586e99ac650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:40.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.642+0000 7f938ce00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5586e99a5270 0x5586e99ac650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55498/0 (socket says 172.21.15.83:55498) 2024-04-24T08:55:40.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.642+0000 7f938ce00640 1 -- 172.21.15.83:0/2792954194 learned_addr learned my addr 172.21.15.83:0/2792954194 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:55:40.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.642+0000 7f938d800640 1 --2- 172.21.15.83:0/2792954194 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5586e99a4950 0x5586e99af2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:40.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.642+0000 7f938e200640 1 --2- 172.21.15.83:0/2792954194 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5586e99af940 0x5586e99acc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:40.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.642+0000 7f938ce00640 1 -- 172.21.15.83:0/2792954194 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5586e99af940 msgr2=0x5586e99acc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:40.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.642+0000 7f938ce00640 1 --2- 172.21.15.83:0/2792954194 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5586e99af940 0x5586e99acc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.642+0000 7f938ce00640 1 -- 172.21.15.83:0/2792954194 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5586e99a4950 msgr2=0x5586e99af2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:40.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.642+0000 7f938ce00640 1 --2- 172.21.15.83:0/2792954194 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5586e99a4950 0x5586e99af2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.642+0000 7f938ce00640 1 -- 172.21.15.83:0/2792954194 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586e98d43f0 con 0x5586e99a5270 2024-04-24T08:55:40.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.642+0000 7f938d800640 1 --2- 172.21.15.83:0/2792954194 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5586e99a4950 0x5586e99af2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:55:40.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.642+0000 7f938e200640 1 --2- 172.21.15.83:0/2792954194 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5586e99af940 0x5586e99acc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:55:40.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.642+0000 7f938ce00640 1 --2- 172.21.15.83:0/2792954194 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5586e99a5270 0x5586e99ac650 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f93740b98f0 tx=0x7f93740fcb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=501704a66e09e461 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:40.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.642+0000 7f9373400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:55:40.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.643+0000 7f9373400640 1 -- 172.21.15.83:0/2792954194 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9374100030 con 0x5586e99a5270 2024-04-24T08:55:40.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.643+0000 7f9373400640 1 -- 172.21.15.83:0/2792954194 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f93740bd6f0 con 0x5586e99a5270 2024-04-24T08:55:40.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.643+0000 7f9393180740 1 -- 172.21.15.83:0/2792954194 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93740fc420 con 0x5586e99a5270 2024-04-24T08:55:40.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.643+0000 7f9373400640 1 -- 172.21.15.83:0/2792954194 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f93740b9730 con 0x5586e99a5270 2024-04-24T08:55:40.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.643+0000 7f9393180740 10 client.17121 fetch_fsmap learned FSMap version 30 2024-04-24T08:55:40.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.643+0000 7f9393180740 1 -- 172.21.15.83:0/2792954194 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5586e961c3f0 con 0x5586e99a5270 2024-04-24T08:55:40.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.643+0000 7f9373400640 1 -- 172.21.15.83:0/2792954194 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f93740fc420 con 0x5586e99a5270 2024-04-24T08:55:40.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.643+0000 7f9373400640 1 -- 172.21.15.83:0/2792954194 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7f93740c5c70 con 0x5586e99a5270 2024-04-24T08:55:40.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.643+0000 7f9393180740 10 client.17121 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T08:55:40.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.643+0000 7f9393180740 10 client.17121 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-24T08:55:40.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.643+0000 7f9393180740 20 client.17121 populate_metadata read hostname 'smithi083' 2024-04-24T08:55:40.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.643+0000 7f9393180740 1 --2- 172.21.15.83:0/2792954194 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x5586e99bf5d0 0x5586e99c19c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:40.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.643+0000 7f9393180740 4 client.17121 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:55:40.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.643+0000 7f9393180740 1 -- 172.21.15.83:0/2792954194 --> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5586e9917bf0 con 0x5586e99bf5d0 2024-04-24T08:55:40.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.644+0000 7f938d800640 1 --2- 172.21.15.83:0/2792954194 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x5586e99bf5d0 0x5586e99c19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:40.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.645+0000 7f938d800640 1 --2- 172.21.15.83:0/2792954194 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x5586e99bf5d0 0x5586e99c19c0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b25d5fd5d70afa02 server_cookie=e875040092e32a39 in_seq=0 out_seq=0 2024-04-24T08:55:40.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.645+0000 7f9373400640 10 client.17121 ms_handle_connect on v2:172.21.15.138:6834/1909219758 2024-04-24T08:55:40.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.647+0000 7f9373400640 1 -- 172.21.15.83:0/2792954194 <== mds.2 v2:172.21.15.138:6834/1909219758 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5586e9917bf0 con 0x5586e99bf5d0 2024-04-24T08:55:40.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.647+0000 7f9373400640 10 client.17121 handle_command_reply: tid=0 2024-04-24T08:55:40.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.647+0000 7f9393180740 1 client.17121 shutdown 2024-04-24T08:55:40.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.647+0000 7f9393180740 1 -- 172.21.15.83:0/2792954194 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5586e99a5270 msgr2=0x5586e99ac650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:40.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.647+0000 7f9393180740 1 --2- 172.21.15.83:0/2792954194 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5586e99a5270 0x5586e99ac650 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f93740b98f0 tx=0x7f93740fcb30 comp rx=0 tx=0).stop 2024-04-24T08:55:40.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.647+0000 7f9393180740 1 -- 172.21.15.83:0/2792954194 shutdown_connections 2024-04-24T08:55:40.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.647+0000 7f9393180740 1 --2- 172.21.15.83:0/2792954194 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x5586e99bf5d0 0x5586e99c19c0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.647+0000 7f9393180740 1 --2- 172.21.15.83:0/2792954194 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5586e99af940 0x5586e99acc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.647+0000 7f9373400640 0 client.17121 ms_handle_reset on v2:172.21.15.138:6834/1909219758 2024-04-24T08:55:40.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.647+0000 7f9393180740 1 --2- 172.21.15.83:0/2792954194 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5586e99a5270 0x5586e99ac650 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.647+0000 7f9393180740 1 --2- 172.21.15.83:0/2792954194 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5586e99a4950 0x5586e99af2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.647+0000 7f9393180740 1 -- 172.21.15.83:0/2792954194 >> 172.21.15.83:0/2792954194 conn(0x5586e99a0c70 msgr2=0x5586e99a6a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:40.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.647+0000 7f9393180740 1 -- 172.21.15.83:0/2792954194 shutdown_connections 2024-04-24T08:55:40.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.648+0000 7f9393180740 1 -- 172.21.15.83:0/2792954194 wait complete. 2024-04-24T08:55:40.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.648+0000 7f9393180740 20 client.17121 trim_cache size 0 max 16384 2024-04-24T08:55:40.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.663+0000 7f9393180740 1 Processor -- start 2024-04-24T08:55:40.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.663+0000 7f9393180740 1 -- start start 2024-04-24T08:55:40.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.663+0000 7f9393180740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5586e99a4950 0x5586e99a70b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:40.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.663+0000 7f9393180740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5586e99a5270 0x5586e99aee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:40.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.663+0000 7f9393180740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5586e99af940 0x5586e999fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:40.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.663+0000 7f9393180740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f93881b16e0 con 0x5586e99af940 2024-04-24T08:55:40.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.663+0000 7f9393180740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f93881b32b0 con 0x5586e99a5270 2024-04-24T08:55:40.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.663+0000 7f9393180740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f93881b1db0 con 0x5586e99a4950 2024-04-24T08:55:40.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.664+0000 7f938e200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5586e99af940 0x5586e999fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:40.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.664+0000 7f938e200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5586e99af940 0x5586e999fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55502/0 (socket says 172.21.15.83:55502) 2024-04-24T08:55:40.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.664+0000 7f938e200640 1 -- 172.21.15.83:0/53778974 learned_addr learned my addr 172.21.15.83:0/53778974 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:55:40.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.664+0000 7f938d800640 1 --2- 172.21.15.83:0/53778974 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5586e99a4950 0x5586e99a70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:40.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.664+0000 7f938ce00640 1 --2- 172.21.15.83:0/53778974 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5586e99a5270 0x5586e99aee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:40.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.664+0000 7f938e200640 1 -- 172.21.15.83:0/53778974 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5586e99a4950 msgr2=0x5586e99a70b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:40.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.664+0000 7f938e200640 1 --2- 172.21.15.83:0/53778974 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5586e99a4950 0x5586e99a70b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.664+0000 7f938e200640 1 -- 172.21.15.83:0/53778974 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5586e99a5270 msgr2=0x5586e99aee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:40.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.664+0000 7f938e200640 1 --2- 172.21.15.83:0/53778974 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5586e99a5270 0x5586e99aee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.664+0000 7f938e200640 1 -- 172.21.15.83:0/53778974 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586e99c1d30 con 0x5586e99af940 2024-04-24T08:55:40.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.664+0000 7f938ce00640 1 --2- 172.21.15.83:0/53778974 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5586e99a5270 0x5586e99aee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:55:40.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.665+0000 7f938e200640 1 --2- 172.21.15.83:0/53778974 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5586e99af940 0x5586e999fc70 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f93780125b0 tx=0x7f9378014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e980da76ef0dcd0d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:40.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.665+0000 7f937e000640 1 -- 172.21.15.83:0/53778974 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f937801f6e0 con 0x5586e99af940 2024-04-24T08:55:40.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.665+0000 7f937e000640 1 -- 172.21.15.83:0/53778974 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9378023070 con 0x5586e99af940 2024-04-24T08:55:40.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.665+0000 7f937e000640 1 -- 172.21.15.83:0/53778974 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9378028c70 con 0x5586e99af940 2024-04-24T08:55:40.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.666+0000 7f9393180740 1 -- 172.21.15.83:0/53778974 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5586e99af940 msgr2=0x5586e999fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:40.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.666+0000 7f9393180740 1 --2- 172.21.15.83:0/53778974 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5586e99af940 0x5586e999fc70 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f93780125b0 tx=0x7f9378014000 comp rx=0 tx=0).stop 2024-04-24T08:55:40.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.666+0000 7f9393180740 1 -- 172.21.15.83:0/53778974 shutdown_connections 2024-04-24T08:55:40.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.666+0000 7f9393180740 1 --2- 172.21.15.83:0/53778974 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5586e99af940 0x5586e999fc70 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.666+0000 7f9393180740 1 --2- 172.21.15.83:0/53778974 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5586e99a5270 0x5586e99aee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.666+0000 7f9393180740 1 --2- 172.21.15.83:0/53778974 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5586e99a4950 0x5586e99a70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.666+0000 7f9393180740 1 -- 172.21.15.83:0/53778974 >> 172.21.15.83:0/53778974 conn(0x5586e99a0c70 msgr2=0x5586e999f560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:40.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.666+0000 7f9393180740 1 -- 172.21.15.83:0/53778974 shutdown_connections 2024-04-24T08:55:40.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.666+0000 7f9393180740 1 -- 172.21.15.83:0/53778974 wait complete. 2024-04-24T08:55:40.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.667+0000 7f9393180740 1 -- start start 2024-04-24T08:55:40.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.667+0000 7f9393180740 1 Processor -- start 2024-04-24T08:55:40.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.667+0000 7f9393180740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5586e99a4950 0x5586e99b00f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:40.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.667+0000 7f9393180740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5586e99a5270 0x5586e99b0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:40.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.667+0000 7f9393180740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5586e99af940 0x5586e99b6770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:40.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.667+0000 7f9393180740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f93881b2b50 con 0x5586e99a5270 2024-04-24T08:55:40.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.667+0000 7f9393180740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f93881b2480 con 0x5586e99a4950 2024-04-24T08:55:40.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.667+0000 7f9393180740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f93881af6c0 con 0x5586e99af940 2024-04-24T08:55:40.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.667+0000 7f9393180740 10 client.0 _ll_register_callbacks cb 0x7f93780092a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:55:40.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.668+0000 7f938ce00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5586e99a5270 0x5586e99b0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:40.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.668+0000 7f938ce00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5586e99a5270 0x5586e99b0630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55512/0 (socket says 172.21.15.83:55512) 2024-04-24T08:55:40.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.668+0000 7f938ce00640 1 -- 172.21.15.83:0/3050575897 learned_addr learned my addr 172.21.15.83:0/3050575897 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:55:40.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.668+0000 7f938d800640 1 --2- 172.21.15.83:0/3050575897 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5586e99a4950 0x5586e99b00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:40.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.668+0000 7f938e200640 1 --2- 172.21.15.83:0/3050575897 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5586e99af940 0x5586e99b6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:40.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.668+0000 7f938ce00640 1 -- 172.21.15.83:0/3050575897 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5586e99af940 msgr2=0x5586e99b6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:40.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.668+0000 7f938ce00640 1 --2- 172.21.15.83:0/3050575897 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5586e99af940 0x5586e99b6770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.668+0000 7f938ce00640 1 -- 172.21.15.83:0/3050575897 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5586e99a4950 msgr2=0x5586e99b00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:40.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.668+0000 7f938ce00640 1 --2- 172.21.15.83:0/3050575897 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5586e99a4950 0x5586e99b00f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.668+0000 7f938ce00640 1 -- 172.21.15.83:0/3050575897 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586e96e67d0 con 0x5586e99a5270 2024-04-24T08:55:40.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.668+0000 7f938d800640 1 --2- 172.21.15.83:0/3050575897 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5586e99a4950 0x5586e99b00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:55:40.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.668+0000 7f938ce00640 1 --2- 172.21.15.83:0/3050575897 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5586e99a5270 0x5586e99b0630 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f93740c6070 tx=0x7f93740f1ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c76f739b2e6318dd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:40.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.668+0000 7f9373400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:55:40.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.669+0000 7f9373400640 1 -- 172.21.15.83:0/3050575897 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9374100070 con 0x5586e99a5270 2024-04-24T08:55:40.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.669+0000 7f9373400640 1 -- 172.21.15.83:0/3050575897 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f93740bd6f0 con 0x5586e99a5270 2024-04-24T08:55:40.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.669+0000 7f9393180740 1 -- 172.21.15.83:0/3050575897 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9378023070 con 0x5586e99a5270 2024-04-24T08:55:40.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.669+0000 7f9373400640 1 -- 172.21.15.83:0/3050575897 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f93740bcdf0 con 0x5586e99a5270 2024-04-24T08:55:40.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.669+0000 7f9393180740 10 client.17127 fetch_fsmap learned FSMap version 30 2024-04-24T08:55:40.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.669+0000 7f9393180740 1 -- 172.21.15.83:0/3050575897 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5586e96e67d0 con 0x5586e99a5270 2024-04-24T08:55:40.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.669+0000 7f9373400640 1 -- 172.21.15.83:0/3050575897 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9378023070 con 0x5586e99a5270 2024-04-24T08:55:40.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.669+0000 7f9373400640 1 -- 172.21.15.83:0/3050575897 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7f937402db90 con 0x5586e99a5270 2024-04-24T08:55:40.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.670+0000 7f9393180740 10 client.17127 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T08:55:40.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.670+0000 7f9393180740 10 client.17127 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-24T08:55:40.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.670+0000 7f9393180740 20 client.17127 populate_metadata read hostname 'smithi083' 2024-04-24T08:55:40.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.670+0000 7f9393180740 1 --2- 172.21.15.83:0/3050575897 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x5586e9a25b00 0x5586e9a27ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:40.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.670+0000 7f9393180740 4 client.17127 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T08:55:40.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.670+0000 7f9393180740 1 -- 172.21.15.83:0/3050575897 --> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5586e9911a60 con 0x5586e9a25b00 2024-04-24T08:55:40.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.670+0000 7f938d800640 1 --2- 172.21.15.83:0/3050575897 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x5586e9a25b00 0x5586e9a27ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:40.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.671+0000 7f938d800640 1 --2- 172.21.15.83:0/3050575897 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x5586e9a25b00 0x5586e9a27ef0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=dabc807f3041d437 server_cookie=4d9e8d4f9225ddb2 in_seq=0 out_seq=0 2024-04-24T08:55:40.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.671+0000 7f9373400640 10 client.17127 ms_handle_connect on v2:172.21.15.138:6834/1909219758 2024-04-24T08:55:40.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.672+0000 7f9373400640 1 -- 172.21.15.83:0/3050575897 <== mds.2 v2:172.21.15.138:6834/1909219758 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5586e9911a60 con 0x5586e9a25b00 2024-04-24T08:55:40.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.672+0000 7f9373400640 10 client.17127 handle_command_reply: tid=0 2024-04-24T08:55:40.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.672+0000 7f9393180740 1 client.17127 shutdown 2024-04-24T08:55:40.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.672+0000 7f9393180740 1 -- 172.21.15.83:0/3050575897 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5586e99a5270 msgr2=0x5586e99b0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:40.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.672+0000 7f9393180740 1 --2- 172.21.15.83:0/3050575897 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5586e99a5270 0x5586e99b0630 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f93740c6070 tx=0x7f93740f1ba0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.672+0000 7f9393180740 1 -- 172.21.15.83:0/3050575897 shutdown_connections 2024-04-24T08:55:40.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.672+0000 7f9393180740 1 --2- 172.21.15.83:0/3050575897 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x5586e9a25b00 0x5586e9a27ef0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.672+0000 7f9393180740 1 --2- 172.21.15.83:0/3050575897 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5586e99af940 0x5586e99b6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.672+0000 7f9393180740 1 --2- 172.21.15.83:0/3050575897 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5586e99a5270 0x5586e99b0630 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.672+0000 7f9373400640 0 client.17127 ms_handle_reset on v2:172.21.15.138:6834/1909219758 2024-04-24T08:55:40.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.672+0000 7f9393180740 1 --2- 172.21.15.83:0/3050575897 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5586e99a4950 0x5586e99b00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.672+0000 7f9393180740 1 -- 172.21.15.83:0/3050575897 >> 172.21.15.83:0/3050575897 conn(0x5586e99a0c70 msgr2=0x5586e999fdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:40.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.672+0000 7f9393180740 1 -- 172.21.15.83:0/3050575897 shutdown_connections 2024-04-24T08:55:40.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.672+0000 7f9393180740 1 -- 172.21.15.83:0/3050575897 wait complete. 2024-04-24T08:55:40.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.673+0000 7f9393180740 20 client.17127 trim_cache size 0 max 16384 2024-04-24T08:55:40.675 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T08:55:40.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.673+0000 7f9390200640 1 -- 172.21.15.83:0/3958150281 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f936c0a3da0 msgr2=0x7f936c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:40.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.673+0000 7f9390200640 1 --2- 172.21.15.83:0/3958150281 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f936c0a3da0 0x7f936c0a6260 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9378009c70 tx=0x7f93780093d0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.673+0000 7f9390200640 1 -- 172.21.15.83:0/3958150281 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9388103ba0 msgr2=0x7f93881a0c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:40.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.673+0000 7f9390200640 1 --2- 172.21.15.83:0/3958150281 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9388103ba0 0x7f93881a0c00 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f9374004af0 tx=0x7f937402fbe0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.673+0000 7f9390200640 1 -- 172.21.15.83:0/3958150281 shutdown_connections 2024-04-24T08:55:40.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.673+0000 7f9390200640 1 --2- 172.21.15.83:0/3958150281 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f936c0a3da0 0x7f936c0a6260 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.673+0000 7f9390200640 1 --2- 172.21.15.83:0/3958150281 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f938810c5c0 0x7f938819a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.673+0000 7f9390200640 1 --2- 172.21.15.83:0/3958150281 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9388103ba0 0x7f93881a0c00 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.673+0000 7f9390200640 1 --2- 172.21.15.83:0/3958150281 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9388103280 0x7f93881a06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.673+0000 7f9390200640 1 -- 172.21.15.83:0/3958150281 >> 172.21.15.83:0/3958150281 conn(0x7f9388078000 msgr2=0x7f9388108570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:40.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.674+0000 7f9390200640 1 -- 172.21.15.83:0/3958150281 shutdown_connections 2024-04-24T08:55:40.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.674+0000 7f9390200640 1 -- 172.21.15.83:0/3958150281 wait complete. 2024-04-24T08:55:40.692 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T08:55:40.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.895+0000 7f4862800640 1 -- 172.21.15.83:0/2943885589 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f485c108f40 msgr2=0x7f485c1153b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:40.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.895+0000 7f4862800640 1 --2- 172.21.15.83:0/2943885589 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f485c108f40 0x7f485c1153b0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f4854009cd0 tx=0x7f485402a8e0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.896+0000 7f4862800640 1 -- 172.21.15.83:0/2943885589 shutdown_connections 2024-04-24T08:55:40.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.896+0000 7f4862800640 1 --2- 172.21.15.83:0/2943885589 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f485c108f40 0x7f485c1153b0 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.896+0000 7f4862800640 1 --2- 172.21.15.83:0/2943885589 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f485c108580 0x7f485c108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.896+0000 7f4862800640 1 --2- 172.21.15.83:0/2943885589 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f485c10d500 0x7f485c10d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.896+0000 7f4862800640 1 -- 172.21.15.83:0/2943885589 >> 172.21.15.83:0/2943885589 conn(0x7f485c077e20 msgr2=0x7f485c106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:40.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.896+0000 7f4862800640 1 -- 172.21.15.83:0/2943885589 shutdown_connections 2024-04-24T08:55:40.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.896+0000 7f4862800640 1 -- 172.21.15.83:0/2943885589 wait complete. 2024-04-24T08:55:40.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.897+0000 7f4862800640 1 Processor -- start 2024-04-24T08:55:40.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.897+0000 7f4862800640 1 -- start start 2024-04-24T08:55:40.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.897+0000 7f4862800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f485c108580 0x7f485c1a4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:40.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.897+0000 7f4862800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f485c108f40 0x7f485c1a4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:40.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.897+0000 7f4862800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f485c10d500 0x7f485c19ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:40.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.898+0000 7f4862800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f485c1187d0 con 0x7f485c108580 2024-04-24T08:55:40.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.898+0000 7f4862800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f485c118650 con 0x7f485c10d500 2024-04-24T08:55:40.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.898+0000 7f4862800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f485c118950 con 0x7f485c108f40 2024-04-24T08:55:40.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.898+0000 7f4861400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f485c108580 0x7f485c1a4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:40.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.898+0000 7f4861400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f485c108580 0x7f485c1a4850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55540/0 (socket says 172.21.15.83:55540) 2024-04-24T08:55:40.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.898+0000 7f4861400640 1 -- 172.21.15.83:0/3013318258 learned_addr learned my addr 172.21.15.83:0/3013318258 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:55:40.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.898+0000 7f4861e00640 1 --2- 172.21.15.83:0/3013318258 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f485c10d500 0x7f485c19ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:40.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.898+0000 7f4861400640 1 -- 172.21.15.83:0/3013318258 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f485c108f40 msgr2=0x7f485c1a4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:40.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.898+0000 7f4861400640 1 --2- 172.21.15.83:0/3013318258 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f485c108f40 0x7f485c1a4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.898+0000 7f4861400640 1 -- 172.21.15.83:0/3013318258 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f485c10d500 msgr2=0x7f485c19ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:40.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.898+0000 7f4861400640 1 --2- 172.21.15.83:0/3013318258 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f485c10d500 0x7f485c19ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.898+0000 7f4861400640 1 -- 172.21.15.83:0/3013318258 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f485c19f1d0 con 0x7f485c108580 2024-04-24T08:55:40.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.898+0000 7f4861400640 1 --2- 172.21.15.83:0/3013318258 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f485c108580 0x7f485c1a4850 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f485000d950 tx=0x7f485000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:40.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.899+0000 7f485a000640 1 -- 172.21.15.83:0/3013318258 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4850017070 con 0x7f485c108580 2024-04-24T08:55:40.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.899+0000 7f485a000640 1 -- 172.21.15.83:0/3013318258 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4850009c30 con 0x7f485c108580 2024-04-24T08:55:40.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.899+0000 7f485a000640 1 -- 172.21.15.83:0/3013318258 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f485000f620 con 0x7f485c108580 2024-04-24T08:55:40.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.899+0000 7f4862800640 1 -- 172.21.15.83:0/3013318258 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f485c19f4c0 con 0x7f485c108580 2024-04-24T08:55:40.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.899+0000 7f4862800640 1 -- 172.21.15.83:0/3013318258 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f485c1ab710 con 0x7f485c108580 2024-04-24T08:55:40.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.902+0000 7f485a000640 1 -- 172.21.15.83:0/3013318258 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f485001a070 con 0x7f485c108580 2024-04-24T08:55:40.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.903+0000 7f4866df2740 1 Processor -- start 2024-04-24T08:55:40.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.903+0000 7f4866df2740 1 -- start start 2024-04-24T08:55:40.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.903+0000 7f4866df2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563222c7b40 0x5563222c7f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:40.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.903+0000 7f4866df2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563222c8460 0x5563222d25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:40.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.903+0000 7f4866df2740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563222d2b30 0x5563222d4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:40.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.903+0000 7f4866df2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55632222d1d0 con 0x5563222c8460 2024-04-24T08:55:40.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.903+0000 7f4866df2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5563221d3450 con 0x5563222c7b40 2024-04-24T08:55:40.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.903+0000 7f4866df2740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x556322229190 con 0x5563222d2b30 2024-04-24T08:55:40.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.904+0000 7f4861400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563222c8460 0x5563222d25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:40.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.904+0000 7f4861400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563222c8460 0x5563222d25f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55546/0 (socket says 172.21.15.83:55546) 2024-04-24T08:55:40.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.904+0000 7f4861400640 1 -- 172.21.15.83:0/1935451376 learned_addr learned my addr 172.21.15.83:0/1935451376 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:55:40.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.904+0000 7f4860a00640 1 --2- 172.21.15.83:0/1935451376 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563222d2b30 0x5563222d4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:40.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.904+0000 7f4861e00640 1 --2- 172.21.15.83:0/1935451376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563222c7b40 0x5563222c7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:40.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.904+0000 7f4861400640 1 -- 172.21.15.83:0/1935451376 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563222d2b30 msgr2=0x5563222d4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:40.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.904+0000 7f4861400640 1 --2- 172.21.15.83:0/1935451376 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563222d2b30 0x5563222d4f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.904+0000 7f4861400640 1 -- 172.21.15.83:0/1935451376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563222c7b40 msgr2=0x5563222c7f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:40.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.904+0000 7f4861400640 1 --2- 172.21.15.83:0/1935451376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563222c7b40 0x5563222c7f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.904+0000 7f4861400640 1 -- 172.21.15.83:0/1935451376 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556322290310 con 0x5563222c8460 2024-04-24T08:55:40.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.905+0000 7f485a000640 1 --2- 172.21.15.83:0/3013318258 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f48440a3da0 0x7f48440a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:40.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.905+0000 7f4861400640 1 --2- 172.21.15.83:0/1935451376 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563222c8460 0x5563222d25f0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f48500cd8b0 tx=0x7f48500d6d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dffde4032b1529c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:40.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.905+0000 7f4861e00640 1 --2- 172.21.15.83:0/3013318258 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f48440a3da0 0x7f48440a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:40.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.905+0000 7f485a000640 1 -- 172.21.15.83:0/3013318258 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f48500d0650 con 0x7f485c108580 2024-04-24T08:55:40.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.905+0000 7f484e600640 1 -- 172.21.15.83:0/1935451376 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4850098460 con 0x5563222c8460 2024-04-24T08:55:40.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.905+0000 7f484e600640 1 -- 172.21.15.83:0/1935451376 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f48500dd070 con 0x5563222c8460 2024-04-24T08:55:40.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.905+0000 7f484e600640 1 -- 172.21.15.83:0/1935451376 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f485009bb20 con 0x5563222c8460 2024-04-24T08:55:40.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.906+0000 7f4866df2740 1 -- 172.21.15.83:0/1935451376 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563222c8460 msgr2=0x5563222d25f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:40.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.906+0000 7f4861e00640 1 --2- 172.21.15.83:0/3013318258 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f48440a3da0 0x7f48440a6260 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f485402ae00 tx=0x7f4854009800 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:40.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.906+0000 7f4866df2740 1 --2- 172.21.15.83:0/1935451376 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563222c8460 0x5563222d25f0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f48500cd8b0 tx=0x7f48500d6d90 comp rx=0 tx=0).stop 2024-04-24T08:55:40.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.906+0000 7f4866df2740 1 -- 172.21.15.83:0/1935451376 shutdown_connections 2024-04-24T08:55:40.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.906+0000 7f4866df2740 1 --2- 172.21.15.83:0/1935451376 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563222d2b30 0x5563222d4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.906+0000 7f4866df2740 1 --2- 172.21.15.83:0/1935451376 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563222c8460 0x5563222d25f0 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.906+0000 7f4866df2740 1 --2- 172.21.15.83:0/1935451376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563222c7b40 0x5563222c7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.906+0000 7f4866df2740 1 -- 172.21.15.83:0/1935451376 >> 172.21.15.83:0/1935451376 conn(0x5563222c3e60 msgr2=0x5563222c4240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:40.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.906+0000 7f4866df2740 1 -- 172.21.15.83:0/1935451376 shutdown_connections 2024-04-24T08:55:40.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.907+0000 7f4866df2740 1 -- 172.21.15.83:0/1935451376 wait complete. 2024-04-24T08:55:40.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.907+0000 7f4866df2740 1 -- start start 2024-04-24T08:55:40.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.907+0000 7f4866df2740 1 Processor -- start 2024-04-24T08:55:40.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.908+0000 7f4866df2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563222c7b40 0x5563222d2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:40.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.908+0000 7f4866df2740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563222c8460 0x5563222cf830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:40.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.908+0000 7f4866df2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563222d2b30 0x5563222cfe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:40.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.908+0000 7f4866df2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5563222d59c0 con 0x5563222d2b30 2024-04-24T08:55:40.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.908+0000 7f4866df2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5563221e6810 con 0x5563222c7b40 2024-04-24T08:55:40.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.908+0000 7f4866df2740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5563222d5b40 con 0x5563222c8460 2024-04-24T08:55:40.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.908+0000 7f4866df2740 10 client.0 _ll_register_callbacks cb 0x5563221b4140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:55:40.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.908+0000 7f4861e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563222d2b30 0x5563222cfe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:40.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.908+0000 7f4861e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563222d2b30 0x5563222cfe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55548/0 (socket says 172.21.15.83:55548) 2024-04-24T08:55:40.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.908+0000 7f4861e00640 1 -- 172.21.15.83:0/619515606 learned_addr learned my addr 172.21.15.83:0/619515606 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:55:40.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.908+0000 7f4861400640 1 --2- 172.21.15.83:0/619515606 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563222c7b40 0x5563222d2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:40.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.908+0000 7f4860a00640 1 --2- 172.21.15.83:0/619515606 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563222c8460 0x5563222cf830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:40.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.909+0000 7f4861e00640 1 -- 172.21.15.83:0/619515606 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563222c8460 msgr2=0x5563222cf830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:40.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.909+0000 7f4861e00640 1 --2- 172.21.15.83:0/619515606 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563222c8460 0x5563222cf830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.909+0000 7f4861e00640 1 -- 172.21.15.83:0/619515606 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563222c7b40 msgr2=0x5563222d2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:40.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.909+0000 7f4861e00640 1 --2- 172.21.15.83:0/619515606 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563222c7b40 0x5563222d2540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.909+0000 7f4861e00640 1 -- 172.21.15.83:0/619515606 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563221f73f0 con 0x5563222d2b30 2024-04-24T08:55:40.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.909+0000 7f4861e00640 1 --2- 172.21.15.83:0/619515606 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563222d2b30 0x5563222cfe40 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f4854004af0 tx=0x7f48540095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa5a6b2382f8deb1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:40.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.909+0000 7f4842a00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:55:40.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.909+0000 7f4842a00640 1 -- 172.21.15.83:0/619515606 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4854041070 con 0x5563222d2b30 2024-04-24T08:55:40.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.909+0000 7f4866df2740 1 -- 172.21.15.83:0/619515606 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f48500dd070 con 0x5563222d2b30 2024-04-24T08:55:40.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.909+0000 7f4842a00640 1 -- 172.21.15.83:0/619515606 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f48540045c0 con 0x5563222d2b30 2024-04-24T08:55:40.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.910+0000 7f4842a00640 1 -- 172.21.15.83:0/619515606 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f48540047c0 con 0x5563222d2b30 2024-04-24T08:55:40.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.910+0000 7f4866df2740 10 client.17139 fetch_fsmap learned FSMap version 30 2024-04-24T08:55:40.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.910+0000 7f4866df2740 1 -- 172.21.15.83:0/619515606 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556321f3f3f0 con 0x5563222d2b30 2024-04-24T08:55:40.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.910+0000 7f4842a00640 1 -- 172.21.15.83:0/619515606 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f48500dd070 con 0x5563222d2b30 2024-04-24T08:55:40.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.910+0000 7f4842a00640 1 -- 172.21.15.83:0/619515606 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7f4854030070 con 0x5563222d2b30 2024-04-24T08:55:40.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.910+0000 7f4866df2740 10 client.17139 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T08:55:40.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.910+0000 7f4866df2740 10 client.17139 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-24T08:55:40.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.910+0000 7f4866df2740 20 client.17139 populate_metadata read hostname 'smithi083' 2024-04-24T08:55:40.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.910+0000 7f4866df2740 1 --2- 172.21.15.83:0/619515606 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x5563222e2de0 0x5563222e5240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:40.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.910+0000 7f4866df2740 4 client.17139 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:55:40.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.910+0000 7f4866df2740 1 -- 172.21.15.83:0/619515606 --> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55632223abf0 con 0x5563222e2de0 2024-04-24T08:55:40.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.911+0000 7f4861400640 1 --2- 172.21.15.83:0/619515606 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x5563222e2de0 0x5563222e5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:40.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.912+0000 7f4861400640 1 --2- 172.21.15.83:0/619515606 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x5563222e2de0 0x5563222e5240 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cebf57aaad2efb58 server_cookie=6b190b08c5e4400e in_seq=0 out_seq=0 2024-04-24T08:55:40.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.912+0000 7f4842a00640 10 client.17139 ms_handle_connect on v2:172.21.15.138:6836/900566754 2024-04-24T08:55:40.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.914+0000 7f4842a00640 1 -- 172.21.15.83:0/619515606 <== mds.1 v2:172.21.15.138:6836/900566754 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55632223abf0 con 0x5563222e2de0 2024-04-24T08:55:40.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.914+0000 7f4842a00640 10 client.17139 handle_command_reply: tid=0 2024-04-24T08:55:40.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.914+0000 7f4866df2740 1 client.17139 shutdown 2024-04-24T08:55:40.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.915+0000 7f4866df2740 1 -- 172.21.15.83:0/619515606 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563222d2b30 msgr2=0x5563222cfe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:40.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.915+0000 7f4866df2740 1 --2- 172.21.15.83:0/619515606 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563222d2b30 0x5563222cfe40 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f4854004af0 tx=0x7f48540095c0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.915+0000 7f4866df2740 1 -- 172.21.15.83:0/619515606 shutdown_connections 2024-04-24T08:55:40.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.915+0000 7f4866df2740 1 --2- 172.21.15.83:0/619515606 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563222d2b30 0x5563222cfe40 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.915+0000 7f4866df2740 1 --2- 172.21.15.83:0/619515606 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x5563222e2de0 0x5563222e5240 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.915+0000 7f4866df2740 1 --2- 172.21.15.83:0/619515606 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563222c8460 0x5563222cf830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.915+0000 7f4866df2740 1 --2- 172.21.15.83:0/619515606 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563222c7b40 0x5563222d2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.915+0000 7f4842a00640 0 client.17139 ms_handle_reset on v2:172.21.15.138:6836/900566754 2024-04-24T08:55:40.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.915+0000 7f4866df2740 1 -- 172.21.15.83:0/619515606 >> 172.21.15.83:0/619515606 conn(0x5563222c3e60 msgr2=0x5563222c9b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:40.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.915+0000 7f4866df2740 1 -- 172.21.15.83:0/619515606 shutdown_connections 2024-04-24T08:55:40.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.915+0000 7f4866df2740 1 -- 172.21.15.83:0/619515606 wait complete. 2024-04-24T08:55:40.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.915+0000 7f4866df2740 20 client.17139 trim_cache size 0 max 16384 2024-04-24T08:55:40.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.930+0000 7f4866df2740 1 Processor -- start 2024-04-24T08:55:40.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.931+0000 7f4866df2740 1 -- start start 2024-04-24T08:55:40.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.931+0000 7f4866df2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563222c7b40 0x5563222c2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:40.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.931+0000 7f4866df2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563222c8460 0x5563222d2070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:40.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.931+0000 7f4866df2740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563222d2b30 0x5563222eab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:40.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.931+0000 7f4866df2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f485c1b5880 con 0x5563222c8460 2024-04-24T08:55:40.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.931+0000 7f4866df2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f485c1b7450 con 0x5563222c7b40 2024-04-24T08:55:40.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.931+0000 7f4866df2740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f485c1b5f50 con 0x5563222d2b30 2024-04-24T08:55:40.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.931+0000 7f4860a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563222c8460 0x5563222d2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:40.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.931+0000 7f4860a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563222c8460 0x5563222d2070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55560/0 (socket says 172.21.15.83:55560) 2024-04-24T08:55:40.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.931+0000 7f4860a00640 1 -- 172.21.15.83:0/3886226627 learned_addr learned my addr 172.21.15.83:0/3886226627 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:55:40.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.932+0000 7f4861e00640 1 --2- 172.21.15.83:0/3886226627 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563222d2b30 0x5563222eab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:40.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.932+0000 7f4861400640 1 --2- 172.21.15.83:0/3886226627 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563222c7b40 0x5563222c2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:40.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.932+0000 7f4860a00640 1 -- 172.21.15.83:0/3886226627 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563222d2b30 msgr2=0x5563222eab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:40.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.932+0000 7f4860a00640 1 --2- 172.21.15.83:0/3886226627 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563222d2b30 0x5563222eab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.932+0000 7f4860a00640 1 -- 172.21.15.83:0/3886226627 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563222c7b40 msgr2=0x5563222c2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:40.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.932+0000 7f4860a00640 1 --2- 172.21.15.83:0/3886226627 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563222c7b40 0x5563222c2dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.932+0000 7f4860a00640 1 -- 172.21.15.83:0/3886226627 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563220097d0 con 0x5563222c8460 2024-04-24T08:55:40.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.932+0000 7f4860a00640 1 --2- 172.21.15.83:0/3886226627 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563222c8460 0x5563222d2070 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f485c1a01f0 tx=0x7f484800ef50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e499e868d685fc0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:40.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.932+0000 7f484dc00640 1 -- 172.21.15.83:0/3886226627 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4848004020 con 0x5563222c8460 2024-04-24T08:55:40.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.933+0000 7f484dc00640 1 -- 172.21.15.83:0/3886226627 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4848015070 con 0x5563222c8460 2024-04-24T08:55:40.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.933+0000 7f484dc00640 1 -- 172.21.15.83:0/3886226627 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48480116a0 con 0x5563222c8460 2024-04-24T08:55:40.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.934+0000 7f4866df2740 1 -- 172.21.15.83:0/3886226627 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563222c8460 msgr2=0x5563222d2070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:40.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.934+0000 7f4866df2740 1 --2- 172.21.15.83:0/3886226627 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563222c8460 0x5563222d2070 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f485c1a01f0 tx=0x7f484800ef50 comp rx=0 tx=0).stop 2024-04-24T08:55:40.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.934+0000 7f4866df2740 1 -- 172.21.15.83:0/3886226627 shutdown_connections 2024-04-24T08:55:40.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.934+0000 7f4866df2740 1 --2- 172.21.15.83:0/3886226627 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563222d2b30 0x5563222eab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.934+0000 7f4866df2740 1 --2- 172.21.15.83:0/3886226627 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563222c8460 0x5563222d2070 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.934+0000 7f4866df2740 1 --2- 172.21.15.83:0/3886226627 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563222c7b40 0x5563222c2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.934+0000 7f4866df2740 1 -- 172.21.15.83:0/3886226627 >> 172.21.15.83:0/3886226627 conn(0x5563222c3e60 msgr2=0x5563222c26c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:40.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.934+0000 7f4866df2740 1 -- 172.21.15.83:0/3886226627 shutdown_connections 2024-04-24T08:55:40.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.934+0000 7f4866df2740 1 -- 172.21.15.83:0/3886226627 wait complete. 2024-04-24T08:55:40.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.934+0000 7f4866df2740 1 -- start start 2024-04-24T08:55:40.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.934+0000 7f4866df2740 1 Processor -- start 2024-04-24T08:55:40.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.935+0000 7f4866df2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563222c7b40 0x5563222d3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:40.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.935+0000 7f4866df2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563222c8460 0x5563222d4170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:40.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.935+0000 7f4866df2740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563222d2b30 0x5563222d93d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:40.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.935+0000 7f4866df2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f485c1b6cf0 con 0x5563222c8460 2024-04-24T08:55:40.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.935+0000 7f4866df2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f485c1b6620 con 0x5563222c7b40 2024-04-24T08:55:40.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.935+0000 7f4866df2740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f485c1b3860 con 0x5563222d2b30 2024-04-24T08:55:40.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.935+0000 7f4866df2740 10 client.0 _ll_register_callbacks cb 0x7f4854031f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:55:40.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.935+0000 7f4860a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563222c8460 0x5563222d4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:40.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.935+0000 7f4860a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563222c8460 0x5563222d4170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55570/0 (socket says 172.21.15.83:55570) 2024-04-24T08:55:40.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.935+0000 7f4860a00640 1 -- 172.21.15.83:0/752556127 learned_addr learned my addr 172.21.15.83:0/752556127 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:55:40.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.936+0000 7f4861e00640 1 --2- 172.21.15.83:0/752556127 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563222d2b30 0x5563222d93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:40.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.936+0000 7f4861400640 1 --2- 172.21.15.83:0/752556127 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563222c7b40 0x5563222d3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:40.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.936+0000 7f4860a00640 1 -- 172.21.15.83:0/752556127 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563222d2b30 msgr2=0x5563222d93d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:40.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.936+0000 7f4860a00640 1 --2- 172.21.15.83:0/752556127 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563222d2b30 0x5563222d93d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.936+0000 7f4860a00640 1 -- 172.21.15.83:0/752556127 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563222c7b40 msgr2=0x5563222d3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:40.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.936+0000 7f4860a00640 1 --2- 172.21.15.83:0/752556127 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563222c7b40 0x5563222d3c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.936+0000 7f4860a00640 1 -- 172.21.15.83:0/752556127 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563222d9910 con 0x5563222c8460 2024-04-24T08:55:40.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.936+0000 7f4860a00640 1 --2- 172.21.15.83:0/752556127 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563222c8460 0x5563222d4170 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f485c1a01f0 tx=0x7f4848003af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65e087a85853df4e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:40.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.936+0000 7f4842a00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:55:40.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.936+0000 7f4842a00640 1 -- 172.21.15.83:0/752556127 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4848004000 con 0x5563222c8460 2024-04-24T08:55:40.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.937+0000 7f4842a00640 1 -- 172.21.15.83:0/752556127 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f48480041a0 con 0x5563222c8460 2024-04-24T08:55:40.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.937+0000 7f4866df2740 1 -- 172.21.15.83:0/752556127 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4848015070 con 0x5563222c8460 2024-04-24T08:55:40.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.937+0000 7f4842a00640 1 -- 172.21.15.83:0/752556127 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4848032050 con 0x5563222c8460 2024-04-24T08:55:40.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.937+0000 7f4866df2740 10 client.17145 fetch_fsmap learned FSMap version 30 2024-04-24T08:55:40.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.937+0000 7f4866df2740 1 -- 172.21.15.83:0/752556127 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556322349a80 con 0x5563222c8460 2024-04-24T08:55:40.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.937+0000 7f4842a00640 1 -- 172.21.15.83:0/752556127 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4848015070 con 0x5563222c8460 2024-04-24T08:55:40.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.937+0000 7f4842a00640 1 -- 172.21.15.83:0/752556127 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7f484801a070 con 0x5563222c8460 2024-04-24T08:55:40.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.937+0000 7f4866df2740 10 client.17145 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T08:55:40.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.937+0000 7f4866df2740 10 client.17145 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-24T08:55:40.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.937+0000 7f4866df2740 20 client.17145 populate_metadata read hostname 'smithi083' 2024-04-24T08:55:40.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.937+0000 7f4866df2740 1 --2- 172.21.15.83:0/752556127 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x55632234a1c0 0x55632234c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:40.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.937+0000 7f4866df2740 4 client.17145 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T08:55:40.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.937+0000 7f4866df2740 1 -- 172.21.15.83:0/752556127 --> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556322234a60 con 0x55632234a1c0 2024-04-24T08:55:40.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.938+0000 7f4861400640 1 --2- 172.21.15.83:0/752556127 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x55632234a1c0 0x55632234c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:40.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.939+0000 7f4861400640 1 --2- 172.21.15.83:0/752556127 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x55632234a1c0 0x55632234c5e0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a9c7ec41e1b2cc6c server_cookie=3f823e4a0186a3d5 in_seq=0 out_seq=0 2024-04-24T08:55:40.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.939+0000 7f4842a00640 10 client.17145 ms_handle_connect on v2:172.21.15.138:6836/900566754 2024-04-24T08:55:40.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.940+0000 7f4842a00640 1 -- 172.21.15.83:0/752556127 <== mds.1 v2:172.21.15.138:6836/900566754 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556322234a60 con 0x55632234a1c0 2024-04-24T08:55:40.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.940+0000 7f4842a00640 10 client.17145 handle_command_reply: tid=0 2024-04-24T08:55:40.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.940+0000 7f4866df2740 1 client.17145 shutdown 2024-04-24T08:55:40.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.940+0000 7f4866df2740 1 -- 172.21.15.83:0/752556127 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563222c8460 msgr2=0x5563222d4170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:40.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.940+0000 7f4866df2740 1 --2- 172.21.15.83:0/752556127 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563222c8460 0x5563222d4170 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f485c1a01f0 tx=0x7f4848003af0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.940+0000 7f4866df2740 1 -- 172.21.15.83:0/752556127 shutdown_connections 2024-04-24T08:55:40.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.940+0000 7f4866df2740 1 --2- 172.21.15.83:0/752556127 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x55632234a1c0 0x55632234c5e0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.940+0000 7f4866df2740 1 --2- 172.21.15.83:0/752556127 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563222d2b30 0x5563222d93d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.940+0000 7f4866df2740 1 --2- 172.21.15.83:0/752556127 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563222c8460 0x5563222d4170 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.940+0000 7f4866df2740 1 --2- 172.21.15.83:0/752556127 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563222c7b40 0x5563222d3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.940+0000 7f4866df2740 1 -- 172.21.15.83:0/752556127 >> 172.21.15.83:0/752556127 conn(0x5563222c3e60 msgr2=0x5563222c2f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:40.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.940+0000 7f4842a00640 0 client.17145 ms_handle_reset on v2:172.21.15.138:6836/900566754 2024-04-24T08:55:40.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.940+0000 7f4866df2740 1 -- 172.21.15.83:0/752556127 shutdown_connections 2024-04-24T08:55:40.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.940+0000 7f4866df2740 1 -- 172.21.15.83:0/752556127 wait complete. 2024-04-24T08:55:40.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.940+0000 7f4866df2740 20 client.17145 trim_cache size 0 max 16384 2024-04-24T08:55:40.942 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T08:55:40.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.941+0000 7f4862800640 1 -- 172.21.15.83:0/3013318258 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f48440a3da0 msgr2=0x7f48440a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:40.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.941+0000 7f4862800640 1 --2- 172.21.15.83:0/3013318258 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f48440a3da0 0x7f48440a6260 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f485402ae00 tx=0x7f4854009800 comp rx=0 tx=0).stop 2024-04-24T08:55:40.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.941+0000 7f4862800640 1 -- 172.21.15.83:0/3013318258 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f485c108580 msgr2=0x7f485c1a4850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:40.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.941+0000 7f4862800640 1 --2- 172.21.15.83:0/3013318258 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f485c108580 0x7f485c1a4850 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f485000d950 tx=0x7f485000de10 comp rx=0 tx=0).stop 2024-04-24T08:55:40.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.941+0000 7f4862800640 1 -- 172.21.15.83:0/3013318258 shutdown_connections 2024-04-24T08:55:40.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.941+0000 7f4862800640 1 --2- 172.21.15.83:0/3013318258 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f48440a3da0 0x7f48440a6260 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.941+0000 7f4862800640 1 --2- 172.21.15.83:0/3013318258 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f485c10d500 0x7f485c19ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.941+0000 7f4862800640 1 --2- 172.21.15.83:0/3013318258 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f485c108f40 0x7f485c1a4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.941+0000 7f4862800640 1 --2- 172.21.15.83:0/3013318258 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f485c108580 0x7f485c1a4850 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:40.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.941+0000 7f4862800640 1 -- 172.21.15.83:0/3013318258 >> 172.21.15.83:0/3013318258 conn(0x7f485c077e20 msgr2=0x7f485c078340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:40.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.941+0000 7f4862800640 1 -- 172.21.15.83:0/3013318258 shutdown_connections 2024-04-24T08:55:40.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:40.941+0000 7f4862800640 1 -- 172.21.15.83:0/3013318258 wait complete. 2024-04-24T08:55:40.961 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T08:55:41.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.168+0000 7f891e400640 1 -- 172.21.15.83:0/642069071 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8918109580 msgr2=0x7f8918111150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:41.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.168+0000 7f891e400640 1 --2- 172.21.15.83:0/642069071 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8918109580 0x7f8918111150 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f8910009ca0 tx=0x7f891002a8c0 comp rx=0 tx=0).stop 2024-04-24T08:55:41.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.168+0000 7f891e400640 1 -- 172.21.15.83:0/642069071 shutdown_connections 2024-04-24T08:55:41.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.168+0000 7f891e400640 1 --2- 172.21.15.83:0/642069071 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8918109580 0x7f8918111150 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:41.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.168+0000 7f891e400640 1 --2- 172.21.15.83:0/642069071 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8918104e60 0x7f8918108ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:41.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.168+0000 7f891e400640 1 --2- 172.21.15.83:0/642069071 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f89181044b0 0x7f8918104890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:41.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.168+0000 7f891e400640 1 -- 172.21.15.83:0/642069071 >> 172.21.15.83:0/642069071 conn(0x7f89180ffd90 msgr2=0x7f89181021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:41.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.168+0000 7f891e400640 1 -- 172.21.15.83:0/642069071 shutdown_connections 2024-04-24T08:55:41.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.168+0000 7f891e400640 1 -- 172.21.15.83:0/642069071 wait complete. 2024-04-24T08:55:41.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.169+0000 7f891e400640 1 Processor -- start 2024-04-24T08:55:41.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.170+0000 7f891e400640 1 -- start start 2024-04-24T08:55:41.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.170+0000 7f891e400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f89181044b0 0x7f89181a0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:41.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.170+0000 7f891e400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8918109580 0x7f89181a0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:41.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.170+0000 7f891e400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f891819a900 0x7f891819adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:41.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.170+0000 7f891e400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8918114560 con 0x7f8918109580 2024-04-24T08:55:41.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.170+0000 7f891e400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f89181143e0 con 0x7f891819a900 2024-04-24T08:55:41.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.170+0000 7f891e400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f89181146e0 con 0x7f89181044b0 2024-04-24T08:55:41.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.170+0000 7f8916a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8918109580 0x7f89181a0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:41.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.170+0000 7f8916a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8918109580 0x7f89181a0c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55594/0 (socket says 172.21.15.83:55594) 2024-04-24T08:55:41.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.170+0000 7f8916a00640 1 -- 172.21.15.83:0/3240156291 learned_addr learned my addr 172.21.15.83:0/3240156291 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:55:41.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.171+0000 7f8917400640 1 --2- 172.21.15.83:0/3240156291 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f89181044b0 0x7f89181a0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:41.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.171+0000 7f8917e00640 1 --2- 172.21.15.83:0/3240156291 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f891819a900 0x7f891819adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:41.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.171+0000 7f8916a00640 1 -- 172.21.15.83:0/3240156291 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f89181044b0 msgr2=0x7f89181a0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:41.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.171+0000 7f8916a00640 1 --2- 172.21.15.83:0/3240156291 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f89181044b0 0x7f89181a0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:41.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.171+0000 7f8916a00640 1 -- 172.21.15.83:0/3240156291 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f891819a900 msgr2=0x7f891819adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:41.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.171+0000 7f8916a00640 1 --2- 172.21.15.83:0/3240156291 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f891819a900 0x7f891819adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:41.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.171+0000 7f8916a00640 1 -- 172.21.15.83:0/3240156291 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f891819b420 con 0x7f8918109580 2024-04-24T08:55:41.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.171+0000 7f8916a00640 1 --2- 172.21.15.83:0/3240156291 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8918109580 0x7f89181a0c90 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f890400b450 tx=0x7f890400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:41.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.171+0000 7f890fe00640 1 -- 172.21.15.83:0/3240156291 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8904015020 con 0x7f8918109580 2024-04-24T08:55:41.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.171+0000 7f890fe00640 1 -- 172.21.15.83:0/3240156291 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8904011d40 con 0x7f8918109580 2024-04-24T08:55:41.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.171+0000 7f891e400640 1 -- 172.21.15.83:0/3240156291 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f891819b710 con 0x7f8918109580 2024-04-24T08:55:41.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.171+0000 7f890fe00640 1 -- 172.21.15.83:0/3240156291 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f890400f340 con 0x7f8918109580 2024-04-24T08:55:41.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.172+0000 7f891e400640 1 -- 172.21.15.83:0/3240156291 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89181a7960 con 0x7f8918109580 2024-04-24T08:55:41.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.175+0000 7f8921386740 1 Processor -- start 2024-04-24T08:55:41.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.175+0000 7f8921386740 1 -- start start 2024-04-24T08:55:41.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.175+0000 7f8921386740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d496a92950 0x55d496a92d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:41.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.175+0000 7f8921386740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d496a93270 0x55d496a9d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:41.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.176+0000 7f8921386740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d496a9d940 0x55d496a9fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:41.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.176+0000 7f8921386740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d4969f81d0 con 0x55d496a92950 2024-04-24T08:55:41.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.176+0000 7f8921386740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d49699e450 con 0x55d496a9d940 2024-04-24T08:55:41.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.176+0000 7f8921386740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d4969f4190 con 0x55d496a93270 2024-04-24T08:55:41.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.176+0000 7f890fe00640 1 -- 172.21.15.83:0/3240156291 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8904018020 con 0x7f8918109580 2024-04-24T08:55:41.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.176+0000 7f8917e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d496a92950 0x55d496a92d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:41.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.176+0000 7f8917e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d496a92950 0x55d496a92d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55600/0 (socket says 172.21.15.83:55600) 2024-04-24T08:55:41.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.176+0000 7f8917e00640 1 -- 172.21.15.83:0/2093833830 learned_addr learned my addr 172.21.15.83:0/2093833830 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:55:41.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.176+0000 7f8917400640 1 --2- 172.21.15.83:0/2093833830 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d496a93270 0x55d496a9d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:41.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.176+0000 7f8917e00640 1 -- 172.21.15.83:0/2093833830 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d496a93270 msgr2=0x55d496a9d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:41.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.176+0000 7f8917e00640 1 --2- 172.21.15.83:0/2093833830 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d496a93270 0x55d496a9d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:41.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.176+0000 7f8917e00640 1 -- 172.21.15.83:0/2093833830 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d496a9d940 msgr2=0x55d496a9fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:41.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.176+0000 7f8917e00640 1 --2- 172.21.15.83:0/2093833830 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d496a9d940 0x55d496a9fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:41.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.176+0000 7f8917e00640 1 -- 172.21.15.83:0/2093833830 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d496a5b310 con 0x55d496a92950 2024-04-24T08:55:41.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.176+0000 7f8917400640 1 --2- 172.21.15.83:0/2093833830 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d496a93270 0x55d496a9d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:55:41.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.177+0000 7f8917e00640 1 --2- 172.21.15.83:0/2093833830 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d496a92950 0x55d496a92d30 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f8910005cc0 tx=0x7f8910005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3463e7ae44a17b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:41.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.177+0000 7f890cc00640 1 -- 172.21.15.83:0/2093833830 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8910031450 con 0x55d496a92950 2024-04-24T08:55:41.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.177+0000 7f890cc00640 1 -- 172.21.15.83:0/2093833830 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8910036070 con 0x55d496a92950 2024-04-24T08:55:41.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.177+0000 7f890cc00640 1 -- 172.21.15.83:0/2093833830 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8910004460 con 0x55d496a92950 2024-04-24T08:55:41.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.178+0000 7f890fe00640 1 --2- 172.21.15.83:0/3240156291 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f88fc0a3d20 0x7f88fc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:41.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.178+0000 7f890fe00640 1 -- 172.21.15.83:0/3240156291 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f89040d0600 con 0x7f8918109580 2024-04-24T08:55:41.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.178+0000 7f8917400640 1 --2- 172.21.15.83:0/3240156291 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f88fc0a3d20 0x7f88fc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:41.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.178+0000 7f8921386740 1 -- 172.21.15.83:0/2093833830 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d496a92950 msgr2=0x55d496a92d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:41.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.178+0000 7f8921386740 1 --2- 172.21.15.83:0/2093833830 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d496a92950 0x55d496a92d30 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f8910005cc0 tx=0x7f8910005fd0 comp rx=0 tx=0).stop 2024-04-24T08:55:41.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.178+0000 7f8921386740 1 -- 172.21.15.83:0/2093833830 shutdown_connections 2024-04-24T08:55:41.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.178+0000 7f8921386740 1 --2- 172.21.15.83:0/2093833830 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d496a9d940 0x55d496a9fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:41.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.178+0000 7f8921386740 1 --2- 172.21.15.83:0/2093833830 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d496a93270 0x55d496a9d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:41.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.178+0000 7f8921386740 1 --2- 172.21.15.83:0/2093833830 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d496a92950 0x55d496a92d30 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:41.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.178+0000 7f8921386740 1 -- 172.21.15.83:0/2093833830 >> 172.21.15.83:0/2093833830 conn(0x55d496a8ec70 msgr2=0x55d496a8f050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:41.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.178+0000 7f8921386740 1 -- 172.21.15.83:0/2093833830 shutdown_connections 2024-04-24T08:55:41.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.179+0000 7f8921386740 1 -- 172.21.15.83:0/2093833830 wait complete. 2024-04-24T08:55:41.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.179+0000 7f8917400640 1 --2- 172.21.15.83:0/3240156291 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f88fc0a3d20 0x7f88fc0a61e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x55d496a93f10 tx=0x7f8908005ef0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:41.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.179+0000 7f8921386740 1 -- start start 2024-04-24T08:55:41.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.179+0000 7f8921386740 1 Processor -- start 2024-04-24T08:55:41.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.180+0000 7f8921386740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d496a92950 0x55d496a9d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:41.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.180+0000 7f8921386740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d496a93270 0x55d496a9a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:41.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.180+0000 7f8921386740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d496a9d940 0x55d496a9adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:41.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.180+0000 7f8921386740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d496aa0770 con 0x55d496a9d940 2024-04-24T08:55:41.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.180+0000 7f8921386740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d4969b1810 con 0x55d496a93270 2024-04-24T08:55:41.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.180+0000 7f8921386740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d496aa08f0 con 0x55d496a92950 2024-04-24T08:55:41.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.180+0000 7f8921386740 10 client.0 _ll_register_callbacks cb 0x55d49697f140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:55:41.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.180+0000 7f8917e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d496a9d940 0x55d496a9adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:41.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.180+0000 7f8917e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d496a9d940 0x55d496a9adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55606/0 (socket says 172.21.15.83:55606) 2024-04-24T08:55:41.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.180+0000 7f8917e00640 1 -- 172.21.15.83:0/467578081 learned_addr learned my addr 172.21.15.83:0/467578081 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:55:41.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.180+0000 7f8917400640 1 --2- 172.21.15.83:0/467578081 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d496a92950 0x55d496a9d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:41.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.180+0000 7f8917e00640 1 -- 172.21.15.83:0/467578081 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d496a92950 msgr2=0x55d496a9d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:41.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.180+0000 7f8917e00640 1 --2- 172.21.15.83:0/467578081 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d496a92950 0x55d496a9d3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:41.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.180+0000 7f8917e00640 1 -- 172.21.15.83:0/467578081 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d496a93270 msgr2=0x55d496a9a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:41.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.180+0000 7f8917e00640 1 --2- 172.21.15.83:0/467578081 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d496a93270 0x55d496a9a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:41.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.181+0000 7f8917e00640 1 -- 172.21.15.83:0/467578081 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4969c23f0 con 0x55d496a9d940 2024-04-24T08:55:41.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.181+0000 7f8917e00640 1 --2- 172.21.15.83:0/467578081 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d496a9d940 0x55d496a9adf0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f891002a900 tx=0x7f8910031f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3d83952a1d88f03 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:41.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.181+0000 7f8901600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:55:41.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.181+0000 7f8901600640 1 -- 172.21.15.83:0/467578081 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8910051020 con 0x55d496a9d940 2024-04-24T08:55:41.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.181+0000 7f8901600640 1 -- 172.21.15.83:0/467578081 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f8910003970 con 0x55d496a9d940 2024-04-24T08:55:41.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.181+0000 7f8921386740 1 -- 172.21.15.83:0/467578081 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8910036070 con 0x55d496a9d940 2024-04-24T08:55:41.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.181+0000 7f8901600640 1 -- 172.21.15.83:0/467578081 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f891005c050 con 0x55d496a9d940 2024-04-24T08:55:41.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.181+0000 7f8921386740 10 client.17157 fetch_fsmap learned FSMap version 30 2024-04-24T08:55:41.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.181+0000 7f8921386740 1 -- 172.21.15.83:0/467578081 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d49670a3f0 con 0x55d496a9d940 2024-04-24T08:55:41.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.182+0000 7f8901600640 1 -- 172.21.15.83:0/467578081 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8910036070 con 0x55d496a9d940 2024-04-24T08:55:41.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.182+0000 7f8901600640 1 -- 172.21.15.83:0/467578081 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7f891004c030 con 0x55d496a9d940 2024-04-24T08:55:41.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.182+0000 7f8921386740 10 client.17157 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T08:55:41.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.182+0000 7f8921386740 10 client.17157 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:55:41.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.182+0000 7f8921386740 20 client.17157 populate_metadata read hostname 'smithi083' 2024-04-24T08:55:41.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.182+0000 7f8921386740 1 --2- 172.21.15.83:0/467578081 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d496aad920 0x55d496aafd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:41.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.182+0000 7f8921386740 4 client.17157 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:55:41.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.182+0000 7f8921386740 1 -- 172.21.15.83:0/467578081 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d496a05bf0 con 0x55d496aad920 2024-04-24T08:55:41.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.183+0000 7f8917400640 1 --2- 172.21.15.83:0/467578081 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d496aad920 0x55d496aafd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:41.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.184+0000 7f8917400640 1 --2- 172.21.15.83:0/467578081 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d496aad920 0x55d496aafd10 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=910b5218b84aa4e7 server_cookie=dd5fb9b479b67f3d in_seq=0 out_seq=0 2024-04-24T08:55:41.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.184+0000 7f8901600640 10 client.17157 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:55:41.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.186+0000 7f8901600640 1 -- 172.21.15.83:0/467578081 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d496a05bf0 con 0x55d496aad920 2024-04-24T08:55:41.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.186+0000 7f8901600640 10 client.17157 handle_command_reply: tid=0 2024-04-24T08:55:41.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.186+0000 7f8921386740 1 client.17157 shutdown 2024-04-24T08:55:41.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.186+0000 7f8921386740 1 -- 172.21.15.83:0/467578081 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d496a9d940 msgr2=0x55d496a9adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:41.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.186+0000 7f8921386740 1 --2- 172.21.15.83:0/467578081 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d496a9d940 0x55d496a9adf0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f891002a900 tx=0x7f8910031f60 comp rx=0 tx=0).stop 2024-04-24T08:55:41.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.187+0000 7f8921386740 1 -- 172.21.15.83:0/467578081 shutdown_connections 2024-04-24T08:55:41.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.187+0000 7f8921386740 1 --2- 172.21.15.83:0/467578081 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d496aad920 0x55d496aafd10 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:41.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.187+0000 7f8921386740 1 --2- 172.21.15.83:0/467578081 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d496a9d940 0x55d496a9adf0 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:41.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.187+0000 7f8921386740 1 --2- 172.21.15.83:0/467578081 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d496a93270 0x55d496a9a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:41.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.187+0000 7f8921386740 1 --2- 172.21.15.83:0/467578081 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d496a92950 0x55d496a9d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:41.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.187+0000 7f8921386740 1 -- 172.21.15.83:0/467578081 >> 172.21.15.83:0/467578081 conn(0x55d496a8ec70 msgr2=0x55d496a94970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:41.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.187+0000 7f8901600640 0 client.17157 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:55:41.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.187+0000 7f8921386740 1 -- 172.21.15.83:0/467578081 shutdown_connections 2024-04-24T08:55:41.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.187+0000 7f8921386740 1 -- 172.21.15.83:0/467578081 wait complete. 2024-04-24T08:55:41.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.187+0000 7f8921386740 20 client.17157 trim_cache size 0 max 16384 2024-04-24T08:55:41.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.204+0000 7f8921386740 1 Processor -- start 2024-04-24T08:55:41.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.204+0000 7f8921386740 1 -- start start 2024-04-24T08:55:41.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.205+0000 7f8921386740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d496a92950 0x55d496a950f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:41.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.205+0000 7f8921386740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d496a93270 0x55d496a99850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:41.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.205+0000 7f8921386740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d496a9d940 0x55d496a9cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:41.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.205+0000 7f8921386740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f89181b16a0 con 0x55d496a9d940 2024-04-24T08:55:41.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.205+0000 7f8921386740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f89181b3270 con 0x55d496a93270 2024-04-24T08:55:41.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.205+0000 7f8921386740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f89181b1d70 con 0x55d496a92950 2024-04-24T08:55:41.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.205+0000 7f8917e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d496a9d940 0x55d496a9cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:41.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.205+0000 7f8917e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d496a9d940 0x55d496a9cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55622/0 (socket says 172.21.15.83:55622) 2024-04-24T08:55:41.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.205+0000 7f8917e00640 1 -- 172.21.15.83:0/2124477932 learned_addr learned my addr 172.21.15.83:0/2124477932 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:55:41.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.205+0000 7f8916a00640 1 --2- 172.21.15.83:0/2124477932 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d496a93270 0x55d496a99850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:41.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.205+0000 7f8917e00640 1 -- 172.21.15.83:0/2124477932 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d496a92950 msgr2=0x55d496a950f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:41.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.205+0000 7f8917e00640 1 --2- 172.21.15.83:0/2124477932 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d496a92950 0x55d496a950f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:41.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.205+0000 7f8917e00640 1 -- 172.21.15.83:0/2124477932 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d496a93270 msgr2=0x55d496a99850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:41.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.205+0000 7f8917e00640 1 --2- 172.21.15.83:0/2124477932 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d496a93270 0x55d496a99850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:41.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.205+0000 7f8917e00640 1 -- 172.21.15.83:0/2124477932 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d496a8a450 con 0x55d496a9d940 2024-04-24T08:55:41.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.205+0000 7f8916a00640 1 --2- 172.21.15.83:0/2124477932 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d496a93270 0x55d496a99850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:55:41.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.206+0000 7f8917e00640 1 --2- 172.21.15.83:0/2124477932 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d496a9d940 0x55d496a9cef0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f89100097b0 tx=0x7f8910009860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e0483b5fcb6a23d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:41.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.206+0000 7f8903e00640 1 -- 172.21.15.83:0/2124477932 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8910051040 con 0x55d496a9d940 2024-04-24T08:55:41.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.206+0000 7f8903e00640 1 -- 172.21.15.83:0/2124477932 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8910036040 con 0x55d496a9d940 2024-04-24T08:55:41.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.206+0000 7f8903e00640 1 -- 172.21.15.83:0/2124477932 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8910049040 con 0x55d496a9d940 2024-04-24T08:55:41.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.207+0000 7f8921386740 1 -- 172.21.15.83:0/2124477932 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d496a9d940 msgr2=0x55d496a9cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:41.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.207+0000 7f8921386740 1 --2- 172.21.15.83:0/2124477932 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d496a9d940 0x55d496a9cef0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f89100097b0 tx=0x7f8910009860 comp rx=0 tx=0).stop 2024-04-24T08:55:41.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.207+0000 7f8921386740 1 -- 172.21.15.83:0/2124477932 shutdown_connections 2024-04-24T08:55:41.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.207+0000 7f8921386740 1 --2- 172.21.15.83:0/2124477932 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d496a9d940 0x55d496a9cef0 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:41.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.207+0000 7f8921386740 1 --2- 172.21.15.83:0/2124477932 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d496a93270 0x55d496a99850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:41.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.207+0000 7f8921386740 1 --2- 172.21.15.83:0/2124477932 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d496a92950 0x55d496a950f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:41.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.207+0000 7f8921386740 1 -- 172.21.15.83:0/2124477932 >> 172.21.15.83:0/2124477932 conn(0x55d496a8ec70 msgr2=0x55d496a8d560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:41.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.207+0000 7f8921386740 1 -- 172.21.15.83:0/2124477932 shutdown_connections 2024-04-24T08:55:41.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.207+0000 7f8921386740 1 -- 172.21.15.83:0/2124477932 wait complete. 2024-04-24T08:55:41.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.207+0000 7f8921386740 1 -- start start 2024-04-24T08:55:41.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.208+0000 7f8921386740 1 Processor -- start 2024-04-24T08:55:41.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.208+0000 7f8921386740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d496a92950 0x55d496aa5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:41.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.208+0000 7f8921386740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d496a93270 0x55d496aa60b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:41.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.208+0000 7f8921386740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d496a9d940 0x55d496a97970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:41.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.208+0000 7f8921386740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f89181b2b10 con 0x55d496a92950 2024-04-24T08:55:41.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.208+0000 7f8921386740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f89181b2440 con 0x55d496a93270 2024-04-24T08:55:41.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.208+0000 7f8921386740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f89181af680 con 0x55d496a9d940 2024-04-24T08:55:41.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.208+0000 7f8921386740 10 client.0 _ll_register_callbacks cb 0x7f8910031a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:55:41.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.208+0000 7f8917400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d496a92950 0x55d496aa5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:41.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.208+0000 7f8917400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d496a92950 0x55d496aa5b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55634/0 (socket says 172.21.15.83:55634) 2024-04-24T08:55:41.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.208+0000 7f8917400640 1 -- 172.21.15.83:0/4232729064 learned_addr learned my addr 172.21.15.83:0/4232729064 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:55:41.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.209+0000 7f8916a00640 1 --2- 172.21.15.83:0/4232729064 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d496a93270 0x55d496aa60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:41.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.209+0000 7f8917400640 1 -- 172.21.15.83:0/4232729064 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d496a9d940 msgr2=0x55d496a97970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:41.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.209+0000 7f8917400640 1 --2- 172.21.15.83:0/4232729064 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d496a9d940 0x55d496a97970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:41.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.209+0000 7f8917400640 1 -- 172.21.15.83:0/4232729064 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d496a93270 msgr2=0x55d496aa60b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:41.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.209+0000 7f8917400640 1 --2- 172.21.15.83:0/4232729064 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d496a93270 0x55d496aa60b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:41.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.209+0000 7f8917400640 1 -- 172.21.15.83:0/4232729064 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4967d47d0 con 0x55d496a92950 2024-04-24T08:55:41.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.209+0000 7f8917400640 1 --2- 172.21.15.83:0/4232729064 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d496a92950 0x55d496aa5b70 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f89080058c0 tx=0x7f8908004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cca1988057c33909 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:41.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.209+0000 7f8901600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:55:41.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.209+0000 7f8901600640 1 -- 172.21.15.83:0/4232729064 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8908011d20 con 0x55d496a92950 2024-04-24T08:55:41.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.209+0000 7f8901600640 1 -- 172.21.15.83:0/4232729064 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f8908025070 con 0x55d496a92950 2024-04-24T08:55:41.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.209+0000 7f8921386740 1 -- 172.21.15.83:0/4232729064 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8910036040 con 0x55d496a92950 2024-04-24T08:55:41.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.210+0000 7f8901600640 1 -- 172.21.15.83:0/4232729064 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8908033050 con 0x55d496a92950 2024-04-24T08:55:41.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.210+0000 7f8921386740 10 client.17163 fetch_fsmap learned FSMap version 30 2024-04-24T08:55:41.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.210+0000 7f8921386740 1 -- 172.21.15.83:0/4232729064 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d496a9e750 con 0x55d496a92950 2024-04-24T08:55:41.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.210+0000 7f8901600640 1 -- 172.21.15.83:0/4232729064 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8910036040 con 0x55d496a92950 2024-04-24T08:55:41.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.210+0000 7f8901600640 1 -- 172.21.15.83:0/4232729064 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7f8908004b20 con 0x55d496a92950 2024-04-24T08:55:41.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.210+0000 7f8921386740 10 client.17163 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T08:55:41.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.210+0000 7f8921386740 10 client.17163 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:55:41.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.210+0000 7f8921386740 20 client.17163 populate_metadata read hostname 'smithi083' 2024-04-24T08:55:41.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.210+0000 7f8921386740 1 --2- 172.21.15.83:0/4232729064 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d496b13bf0 0x55d496b16010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:41.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.210+0000 7f8921386740 4 client.17163 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T08:55:41.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.210+0000 7f8921386740 1 -- 172.21.15.83:0/4232729064 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d4969ffa60 con 0x55d496b13bf0 2024-04-24T08:55:41.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.211+0000 7f8916a00640 1 --2- 172.21.15.83:0/4232729064 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d496b13bf0 0x55d496b16010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:41.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.212+0000 7f8916a00640 1 --2- 172.21.15.83:0/4232729064 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d496b13bf0 0x55d496b16010 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4cb2ec38ba9a871a server_cookie=71cd0551cefe3cae in_seq=0 out_seq=0 2024-04-24T08:55:41.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.212+0000 7f8901600640 10 client.17163 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:55:41.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.213+0000 7f8901600640 1 -- 172.21.15.83:0/4232729064 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d4969ffa60 con 0x55d496b13bf0 2024-04-24T08:55:41.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.213+0000 7f8901600640 10 client.17163 handle_command_reply: tid=0 2024-04-24T08:55:41.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.213+0000 7f8921386740 1 client.17163 shutdown 2024-04-24T08:55:41.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.213+0000 7f8921386740 1 -- 172.21.15.83:0/4232729064 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d496a92950 msgr2=0x55d496aa5b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:41.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.213+0000 7f8921386740 1 --2- 172.21.15.83:0/4232729064 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d496a92950 0x55d496aa5b70 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f89080058c0 tx=0x7f8908004730 comp rx=0 tx=0).stop 2024-04-24T08:55:41.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.213+0000 7f8921386740 1 -- 172.21.15.83:0/4232729064 shutdown_connections 2024-04-24T08:55:41.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.213+0000 7f8921386740 1 --2- 172.21.15.83:0/4232729064 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d496b13bf0 0x55d496b16010 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:41.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.213+0000 7f8921386740 1 --2- 172.21.15.83:0/4232729064 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d496a9d940 0x55d496a97970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:41.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.213+0000 7f8921386740 1 --2- 172.21.15.83:0/4232729064 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d496a93270 0x55d496aa60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:41.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.213+0000 7f8921386740 1 --2- 172.21.15.83:0/4232729064 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d496a92950 0x55d496aa5b70 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:41.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.213+0000 7f8901600640 0 client.17163 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:55:41.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.213+0000 7f8921386740 1 -- 172.21.15.83:0/4232729064 >> 172.21.15.83:0/4232729064 conn(0x55d496a8ec70 msgr2=0x55d496a996d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:41.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.213+0000 7f8921386740 1 -- 172.21.15.83:0/4232729064 shutdown_connections 2024-04-24T08:55:41.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.213+0000 7f8921386740 1 -- 172.21.15.83:0/4232729064 wait complete. 2024-04-24T08:55:41.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.213+0000 7f8921386740 20 client.17163 trim_cache size 0 max 16384 2024-04-24T08:55:41.216 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T08:55:41.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.214+0000 7f891e400640 1 -- 172.21.15.83:0/3240156291 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f88fc0a3d20 msgr2=0x7f88fc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:41.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.214+0000 7f891e400640 1 --2- 172.21.15.83:0/3240156291 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f88fc0a3d20 0x7f88fc0a61e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x55d496a93f10 tx=0x7f8908005ef0 comp rx=0 tx=0).stop 2024-04-24T08:55:41.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.214+0000 7f891e400640 1 -- 172.21.15.83:0/3240156291 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8918109580 msgr2=0x7f89181a0c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:41.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.214+0000 7f891e400640 1 --2- 172.21.15.83:0/3240156291 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8918109580 0x7f89181a0c90 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f890400b450 tx=0x7f890400b910 comp rx=0 tx=0).stop 2024-04-24T08:55:41.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.214+0000 7f891e400640 1 -- 172.21.15.83:0/3240156291 shutdown_connections 2024-04-24T08:55:41.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.214+0000 7f891e400640 1 --2- 172.21.15.83:0/3240156291 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f88fc0a3d20 0x7f88fc0a61e0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:41.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.214+0000 7f891e400640 1 --2- 172.21.15.83:0/3240156291 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f891819a900 0x7f891819adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:41.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.214+0000 7f891e400640 1 --2- 172.21.15.83:0/3240156291 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8918109580 0x7f89181a0c90 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:41.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.214+0000 7f891e400640 1 --2- 172.21.15.83:0/3240156291 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f89181044b0 0x7f89181a0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:41.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.214+0000 7f891e400640 1 -- 172.21.15.83:0/3240156291 >> 172.21.15.83:0/3240156291 conn(0x7f89180ffd90 msgr2=0x7f89181011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:41.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.214+0000 7f891e400640 1 -- 172.21.15.83:0/3240156291 shutdown_connections 2024-04-24T08:55:41.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:41.214+0000 7f891e400640 1 -- 172.21.15.83:0/3240156291 wait complete. 2024-04-24T08:55:41.569 INFO:tasks.workunit.client.0.smithi083.stdout: 1 16748 0.53 MB/sec execute 15 sec latency 48.519 ms 2024-04-24T08:55:42.233 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T08:55:42.233 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T08:55:42.233 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T08:55:42.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:55:42 smithi083 ceph-mon[27907]: pgmap v125: 129 pgs: 129 active+clean; 32 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 230 KiB/s wr, 51 op/s 2024-04-24T08:55:42.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:55:42 smithi138 ceph-mon[31441]: pgmap v125: 129 pgs: 129 active+clean; 32 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 230 KiB/s wr, 51 op/s 2024-04-24T08:55:42.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.455+0000 7efe46600640 1 -- 172.21.15.83:0/425326970 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efe40104e00 msgr2=0x7efe40108e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:42.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.455+0000 7efe46600640 1 --2- 172.21.15.83:0/425326970 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efe40104e00 0x7efe40108e40 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7efe28009900 tx=0x7efe2802a3f0 comp rx=0 tx=0).stop 2024-04-24T08:55:42.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.455+0000 7efe46600640 1 -- 172.21.15.83:0/425326970 shutdown_connections 2024-04-24T08:55:42.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.455+0000 7efe46600640 1 --2- 172.21.15.83:0/425326970 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efe40109520 0x7efe401110f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:42.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.455+0000 7efe46600640 1 --2- 172.21.15.83:0/425326970 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efe40104e00 0x7efe40108e40 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:42.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.455+0000 7efe46600640 1 --2- 172.21.15.83:0/425326970 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe40104450 0x7efe40104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:42.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.455+0000 7efe46600640 1 -- 172.21.15.83:0/425326970 >> 172.21.15.83:0/425326970 conn(0x7efe400ffc70 msgr2=0x7efe40102090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:42.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.455+0000 7efe46600640 1 -- 172.21.15.83:0/425326970 shutdown_connections 2024-04-24T08:55:42.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.455+0000 7efe46600640 1 -- 172.21.15.83:0/425326970 wait complete. 2024-04-24T08:55:42.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:55:42 smithi189 ceph-mon[31791]: pgmap v125: 129 pgs: 129 active+clean; 32 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 230 KiB/s wr, 51 op/s 2024-04-24T08:55:42.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.456+0000 7efe46600640 1 Processor -- start 2024-04-24T08:55:42.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.457+0000 7efe46600640 1 -- start start 2024-04-24T08:55:42.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.457+0000 7efe46600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efe40104450 0x7efe401a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:42.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.457+0000 7efe46600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efe40109520 0x7efe401a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:42.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.457+0000 7efe46600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe4019a810 0x7efe4019acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:42.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.457+0000 7efe46600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7efe40114490 con 0x7efe40104450 2024-04-24T08:55:42.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.457+0000 7efe46600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efe40114310 con 0x7efe4019a810 2024-04-24T08:55:42.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.457+0000 7efe46600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7efe40114610 con 0x7efe40109520 2024-04-24T08:55:42.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.457+0000 7efe3f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efe40104450 0x7efe401a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:42.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.457+0000 7efe3f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efe40104450 0x7efe401a06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55664/0 (socket says 172.21.15.83:55664) 2024-04-24T08:55:42.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.457+0000 7efe3f400640 1 -- 172.21.15.83:0/1441939279 learned_addr learned my addr 172.21.15.83:0/1441939279 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:55:42.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.458+0000 7efe3ea00640 1 --2- 172.21.15.83:0/1441939279 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efe40109520 0x7efe401a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:42.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.458+0000 7efe3fe00640 1 --2- 172.21.15.83:0/1441939279 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe4019a810 0x7efe4019acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:42.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.458+0000 7efe3f400640 1 -- 172.21.15.83:0/1441939279 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efe40109520 msgr2=0x7efe401a0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:42.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.458+0000 7efe3f400640 1 --2- 172.21.15.83:0/1441939279 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efe40109520 0x7efe401a0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:42.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.458+0000 7efe3f400640 1 -- 172.21.15.83:0/1441939279 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe4019a810 msgr2=0x7efe4019acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:42.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.458+0000 7efe3f400640 1 --2- 172.21.15.83:0/1441939279 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe4019a810 0x7efe4019acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:42.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.458+0000 7efe3f400640 1 -- 172.21.15.83:0/1441939279 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe4019b3a0 con 0x7efe40104450 2024-04-24T08:55:42.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.458+0000 7efe3f400640 1 --2- 172.21.15.83:0/1441939279 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efe40104450 0x7efe401a06a0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7efe3000c920 tx=0x7efe3000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:42.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.458+0000 7efe37e00640 1 -- 172.21.15.83:0/1441939279 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe30007c70 con 0x7efe40104450 2024-04-24T08:55:42.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.458+0000 7efe37e00640 1 -- 172.21.15.83:0/1441939279 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efe30007e10 con 0x7efe40104450 2024-04-24T08:55:42.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.458+0000 7efe46600640 1 -- 172.21.15.83:0/1441939279 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe4019b690 con 0x7efe40104450 2024-04-24T08:55:42.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.458+0000 7efe37e00640 1 -- 172.21.15.83:0/1441939279 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe30018820 con 0x7efe40104450 2024-04-24T08:55:42.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.459+0000 7efe46600640 1 -- 172.21.15.83:0/1441939279 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe401a7840 con 0x7efe40104450 2024-04-24T08:55:42.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.462+0000 7efe49383740 1 Processor -- start 2024-04-24T08:55:42.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.462+0000 7efe49383740 1 -- start start 2024-04-24T08:55:42.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.462+0000 7efe37e00640 1 -- 172.21.15.83:0/1441939279 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7efe300189c0 con 0x7efe40104450 2024-04-24T08:55:42.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.462+0000 7efe49383740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55595e683090 0x55595e683470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:42.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.462+0000 7efe49383740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55595e6839b0 0x55595e68db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:42.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.462+0000 7efe49383740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55595e68e080 0x55595e690470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:42.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.462+0000 7efe49383740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55595e5e8e70 con 0x55595e683090 2024-04-24T08:55:42.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.462+0000 7efe49383740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55595e58f380 con 0x55595e68e080 2024-04-24T08:55:42.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.463+0000 7efe49383740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55595e5e4e30 con 0x55595e6839b0 2024-04-24T08:55:42.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.463+0000 7efe3fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55595e683090 0x55595e683470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:42.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.463+0000 7efe3fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55595e683090 0x55595e683470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55672/0 (socket says 172.21.15.83:55672) 2024-04-24T08:55:42.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.463+0000 7efe3fe00640 1 -- 172.21.15.83:0/3735207414 learned_addr learned my addr 172.21.15.83:0/3735207414 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:55:42.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.463+0000 7efe3ea00640 1 --2- 172.21.15.83:0/3735207414 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55595e68e080 0x55595e690470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:42.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.463+0000 7efe3f400640 1 --2- 172.21.15.83:0/3735207414 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55595e6839b0 0x55595e68db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:42.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.463+0000 7efe3fe00640 1 -- 172.21.15.83:0/3735207414 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55595e6839b0 msgr2=0x55595e68db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:42.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.463+0000 7efe3fe00640 1 --2- 172.21.15.83:0/3735207414 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55595e6839b0 0x55595e68db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:42.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.463+0000 7efe3fe00640 1 -- 172.21.15.83:0/3735207414 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55595e68e080 msgr2=0x55595e690470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:42.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.463+0000 7efe3fe00640 1 --2- 172.21.15.83:0/3735207414 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55595e68e080 0x55595e690470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:42.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.463+0000 7efe3fe00640 1 -- 172.21.15.83:0/3735207414 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55595e3c37e0 con 0x55595e683090 2024-04-24T08:55:42.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.464+0000 7efe3fe00640 1 --2- 172.21.15.83:0/3735207414 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55595e683090 0x55595e683470 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7efe38008c80 tx=0x7efe3800d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=961c0058b8480a3f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:42.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.464+0000 7efe37e00640 1 --2- 172.21.15.83:0/1441939279 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7efe240a3d20 0x7efe240a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:42.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.464+0000 7efe34c00640 1 -- 172.21.15.83:0/3735207414 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe38018680 con 0x55595e683090 2024-04-24T08:55:42.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.464+0000 7efe34c00640 1 -- 172.21.15.83:0/3735207414 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efe38018cd0 con 0x55595e683090 2024-04-24T08:55:42.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.464+0000 7efe34c00640 1 -- 172.21.15.83:0/3735207414 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe3800fd00 con 0x55595e683090 2024-04-24T08:55:42.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.464+0000 7efe3f400640 1 --2- 172.21.15.83:0/1441939279 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7efe240a3d20 0x7efe240a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:42.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.464+0000 7efe37e00640 1 -- 172.21.15.83:0/1441939279 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7efe300d0e20 con 0x7efe40104450 2024-04-24T08:55:42.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.465+0000 7efe3f400640 1 --2- 172.21.15.83:0/1441939279 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7efe240a3d20 0x7efe240a61e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x55595e684650 tx=0x7efe30000bc0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:42.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.465+0000 7efe49383740 1 -- 172.21.15.83:0/3735207414 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55595e683090 msgr2=0x55595e683470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:42.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.465+0000 7efe49383740 1 --2- 172.21.15.83:0/3735207414 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55595e683090 0x55595e683470 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7efe38008c80 tx=0x7efe3800d710 comp rx=0 tx=0).stop 2024-04-24T08:55:42.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.465+0000 7efe49383740 1 -- 172.21.15.83:0/3735207414 shutdown_connections 2024-04-24T08:55:42.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.465+0000 7efe49383740 1 --2- 172.21.15.83:0/3735207414 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55595e68e080 0x55595e690470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:42.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.465+0000 7efe49383740 1 --2- 172.21.15.83:0/3735207414 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55595e6839b0 0x55595e68db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:42.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.465+0000 7efe49383740 1 --2- 172.21.15.83:0/3735207414 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55595e683090 0x55595e683470 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:42.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.465+0000 7efe49383740 1 -- 172.21.15.83:0/3735207414 >> 172.21.15.83:0/3735207414 conn(0x55595e67f3b0 msgr2=0x55595e67f790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:42.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.465+0000 7efe49383740 1 -- 172.21.15.83:0/3735207414 shutdown_connections 2024-04-24T08:55:42.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.465+0000 7efe49383740 1 -- 172.21.15.83:0/3735207414 wait complete. 2024-04-24T08:55:42.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.466+0000 7efe49383740 1 -- start start 2024-04-24T08:55:42.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.466+0000 7efe49383740 1 Processor -- start 2024-04-24T08:55:42.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.467+0000 7efe49383740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55595e683090 0x55595e68b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:42.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.467+0000 7efe49383740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55595e6839b0 0x55595e68b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:42.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.467+0000 7efe49383740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55595e68e080 0x55595e68bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:42.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.467+0000 7efe49383740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55595e690e70 con 0x55595e68e080 2024-04-24T08:55:42.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.467+0000 7efe49383740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55595e5a2660 con 0x55595e6839b0 2024-04-24T08:55:42.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.467+0000 7efe49383740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55595e690ff0 con 0x55595e683090 2024-04-24T08:55:42.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.467+0000 7efe49383740 10 client.0 _ll_register_callbacks cb 0x55595e590aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:55:42.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.467+0000 7efe3fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55595e68e080 0x55595e68bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:42.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.467+0000 7efe3fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55595e68e080 0x55595e68bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55686/0 (socket says 172.21.15.83:55686) 2024-04-24T08:55:42.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.467+0000 7efe3fe00640 1 -- 172.21.15.83:0/1021996675 learned_addr learned my addr 172.21.15.83:0/1021996675 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:55:42.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.467+0000 7efe3ea00640 1 --2- 172.21.15.83:0/1021996675 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55595e6839b0 0x55595e68b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:42.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.467+0000 7efe3fe00640 1 --2- 172.21.15.83:0/1021996675 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55595e683090 0x55595e68b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:42.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.468+0000 7efe3fe00640 1 -- 172.21.15.83:0/1021996675 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55595e683090 msgr2=0x55595e68b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:42.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.468+0000 7efe3fe00640 1 --2- 172.21.15.83:0/1021996675 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55595e683090 0x55595e68b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:42.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.468+0000 7efe3fe00640 1 -- 172.21.15.83:0/1021996675 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55595e6839b0 msgr2=0x55595e68b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:42.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.468+0000 7efe3fe00640 1 --2- 172.21.15.83:0/1021996675 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55595e6839b0 0x55595e68b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:42.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.468+0000 7efe3fe00640 1 -- 172.21.15.83:0/1021996675 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55595e423f70 con 0x55595e68e080 2024-04-24T08:55:42.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.468+0000 7efe3ea00640 1 --2- 172.21.15.83:0/1021996675 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55595e6839b0 0x55595e68b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:55:42.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.468+0000 7efe3fe00640 1 --2- 172.21.15.83:0/1021996675 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55595e68e080 0x55595e68bac0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7efe3800f960 tx=0x7efe3800f860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df383733ca83067d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:42.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.468+0000 7efe2d600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:55:42.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.468+0000 7efe2d600640 1 -- 172.21.15.83:0/1021996675 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe38004e50 con 0x55595e68e080 2024-04-24T08:55:42.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.468+0000 7efe2d600640 1 -- 172.21.15.83:0/1021996675 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7efe38018480 con 0x55595e68e080 2024-04-24T08:55:42.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.468+0000 7efe49383740 1 -- 172.21.15.83:0/1021996675 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe38018cd0 con 0x55595e68e080 2024-04-24T08:55:42.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.469+0000 7efe2d600640 1 -- 172.21.15.83:0/1021996675 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7efe38018680 con 0x55595e68e080 2024-04-24T08:55:42.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.469+0000 7efe49383740 10 client.17175 fetch_fsmap learned FSMap version 30 2024-04-24T08:55:42.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.469+0000 7efe49383740 1 -- 172.21.15.83:0/1021996675 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55595e2bb170 con 0x55595e68e080 2024-04-24T08:55:42.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.469+0000 7efe2d600640 1 -- 172.21.15.83:0/1021996675 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efe38018cd0 con 0x55595e68e080 2024-04-24T08:55:42.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.469+0000 7efe2d600640 1 -- 172.21.15.83:0/1021996675 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7efe38027400 con 0x55595e68e080 2024-04-24T08:55:42.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.469+0000 7efe49383740 10 client.17175 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T08:55:42.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.469+0000 7efe49383740 10 client.17175 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:55:42.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.469+0000 7efe49383740 20 client.17175 populate_metadata read hostname 'smithi083' 2024-04-24T08:55:42.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.469+0000 7efe49383740 1 --2- 172.21.15.83:0/1021996675 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55595e69e2c0 0x55595e6a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:42.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.469+0000 7efe49383740 4 client.17175 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:55:42.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.469+0000 7efe49383740 1 -- 172.21.15.83:0/1021996675 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55595e5f6890 con 0x55595e69e2c0 2024-04-24T08:55:42.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.470+0000 7efe3ea00640 1 --2- 172.21.15.83:0/1021996675 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55595e69e2c0 0x55595e6a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:42.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.471+0000 7efe3ea00640 1 --2- 172.21.15.83:0/1021996675 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55595e69e2c0 0x55595e6a06b0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=487d2b62816342ed server_cookie=8bc2db661c9af863 in_seq=0 out_seq=0 2024-04-24T08:55:42.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.471+0000 7efe2d600640 10 client.17175 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:55:42.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.475+0000 7efe2d600640 1 -- 172.21.15.83:0/1021996675 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55595e5f6890 con 0x55595e69e2c0 2024-04-24T08:55:42.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.475+0000 7efe2d600640 10 client.17175 handle_command_reply: tid=0 2024-04-24T08:55:42.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.475+0000 7efe49383740 1 client.17175 shutdown 2024-04-24T08:55:42.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.475+0000 7efe49383740 1 -- 172.21.15.83:0/1021996675 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55595e68e080 msgr2=0x55595e68bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:42.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.475+0000 7efe49383740 1 --2- 172.21.15.83:0/1021996675 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55595e68e080 0x55595e68bac0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7efe3800f960 tx=0x7efe3800f860 comp rx=0 tx=0).stop 2024-04-24T08:55:42.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.476+0000 7efe49383740 1 -- 172.21.15.83:0/1021996675 shutdown_connections 2024-04-24T08:55:42.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.476+0000 7efe49383740 1 --2- 172.21.15.83:0/1021996675 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55595e69e2c0 0x55595e6a06b0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:42.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.476+0000 7efe49383740 1 --2- 172.21.15.83:0/1021996675 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55595e68e080 0x55595e68bac0 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:42.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.476+0000 7efe49383740 1 --2- 172.21.15.83:0/1021996675 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55595e6839b0 0x55595e68b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:42.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.476+0000 7efe49383740 1 --2- 172.21.15.83:0/1021996675 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55595e683090 0x55595e68b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:42.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.476+0000 7efe2d600640 0 client.17175 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:55:42.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.476+0000 7efe49383740 1 -- 172.21.15.83:0/1021996675 >> 172.21.15.83:0/1021996675 conn(0x55595e67f3b0 msgr2=0x55595e6851b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:42.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.476+0000 7efe49383740 1 -- 172.21.15.83:0/1021996675 shutdown_connections 2024-04-24T08:55:42.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.476+0000 7efe49383740 1 -- 172.21.15.83:0/1021996675 wait complete. 2024-04-24T08:55:42.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.476+0000 7efe49383740 20 client.17175 trim_cache size 0 max 16384 2024-04-24T08:55:42.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.498+0000 7efe49383740 1 Processor -- start 2024-04-24T08:55:42.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.498+0000 7efe49383740 1 -- start start 2024-04-24T08:55:42.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.498+0000 7efe49383740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55595e683090 0x55595e699d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:42.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.498+0000 7efe49383740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55595e6839b0 0x55595e685760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:42.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.498+0000 7efe49383740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55595e68e080 0x55595e6a5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:42.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.498+0000 7efe49383740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7efe401b1780 con 0x55595e68e080 2024-04-24T08:55:42.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.498+0000 7efe49383740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efe401b3350 con 0x55595e683090 2024-04-24T08:55:42.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.498+0000 7efe49383740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7efe401b1e50 con 0x55595e6839b0 2024-04-24T08:55:42.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.499+0000 7efe3fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55595e68e080 0x55595e6a5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:42.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.499+0000 7efe3fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55595e68e080 0x55595e6a5eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55698/0 (socket says 172.21.15.83:55698) 2024-04-24T08:55:42.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.499+0000 7efe3fe00640 1 -- 172.21.15.83:0/2973012908 learned_addr learned my addr 172.21.15.83:0/2973012908 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:55:42.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.499+0000 7efe3fe00640 1 --2- 172.21.15.83:0/2973012908 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55595e683090 0x55595e699d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:42.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.499+0000 7efe3ea00640 1 --2- 172.21.15.83:0/2973012908 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55595e6839b0 0x55595e685760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:42.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.499+0000 7efe3fe00640 1 -- 172.21.15.83:0/2973012908 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55595e6839b0 msgr2=0x55595e685760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:42.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.499+0000 7efe3fe00640 1 --2- 172.21.15.83:0/2973012908 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55595e6839b0 0x55595e685760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:42.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.499+0000 7efe3fe00640 1 -- 172.21.15.83:0/2973012908 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55595e683090 msgr2=0x55595e699d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:42.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.499+0000 7efe3fe00640 1 --2- 172.21.15.83:0/2973012908 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55595e683090 0x55595e699d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:42.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.499+0000 7efe3fe00640 1 -- 172.21.15.83:0/2973012908 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55595e6a0c40 con 0x55595e68e080 2024-04-24T08:55:42.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.500+0000 7efe3fe00640 1 --2- 172.21.15.83:0/2973012908 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55595e68e080 0x55595e6a5eb0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7efe38028e40 tx=0x7efe3800d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4edcafcfb9ff181 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:42.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.500+0000 7efe2fe00640 1 -- 172.21.15.83:0/2973012908 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe38008790 con 0x55595e68e080 2024-04-24T08:55:42.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.500+0000 7efe2fe00640 1 -- 172.21.15.83:0/2973012908 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efe38027400 con 0x55595e68e080 2024-04-24T08:55:42.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.500+0000 7efe2fe00640 1 -- 172.21.15.83:0/2973012908 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe38018820 con 0x55595e68e080 2024-04-24T08:55:42.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.501+0000 7efe49383740 1 -- 172.21.15.83:0/2973012908 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55595e68e080 msgr2=0x55595e6a5eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:42.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.501+0000 7efe49383740 1 --2- 172.21.15.83:0/2973012908 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55595e68e080 0x55595e6a5eb0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7efe38028e40 tx=0x7efe3800d710 comp rx=0 tx=0).stop 2024-04-24T08:55:42.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.501+0000 7efe49383740 1 -- 172.21.15.83:0/2973012908 shutdown_connections 2024-04-24T08:55:42.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.501+0000 7efe49383740 1 --2- 172.21.15.83:0/2973012908 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55595e68e080 0x55595e6a5eb0 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:42.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.501+0000 7efe49383740 1 --2- 172.21.15.83:0/2973012908 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55595e6839b0 0x55595e685760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:42.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.501+0000 7efe49383740 1 --2- 172.21.15.83:0/2973012908 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55595e683090 0x55595e699d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:42.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.501+0000 7efe49383740 1 -- 172.21.15.83:0/2973012908 >> 172.21.15.83:0/2973012908 conn(0x55595e67f3b0 msgr2=0x55595e68b890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:42.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.501+0000 7efe49383740 1 -- 172.21.15.83:0/2973012908 shutdown_connections 2024-04-24T08:55:42.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.501+0000 7efe49383740 1 -- 172.21.15.83:0/2973012908 wait complete. 2024-04-24T08:55:42.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.502+0000 7efe49383740 1 -- start start 2024-04-24T08:55:42.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.502+0000 7efe49383740 1 Processor -- start 2024-04-24T08:55:42.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.502+0000 7efe49383740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55595e683090 0x55595e6965e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:42.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.502+0000 7efe49383740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55595e6839b0 0x55595e691950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:42.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.502+0000 7efe49383740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55595e68e080 0x55595e691e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:42.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.502+0000 7efe49383740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7efe401b2bf0 con 0x55595e683090 2024-04-24T08:55:42.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.502+0000 7efe49383740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efe401b2520 con 0x55595e6839b0 2024-04-24T08:55:42.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.502+0000 7efe49383740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7efe401af760 con 0x55595e68e080 2024-04-24T08:55:42.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.502+0000 7efe49383740 10 client.0 _ll_register_callbacks cb 0x7efe38008ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:55:42.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.503+0000 7efe3fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55595e683090 0x55595e6965e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:42.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.503+0000 7efe3fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55595e683090 0x55595e6965e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55702/0 (socket says 172.21.15.83:55702) 2024-04-24T08:55:42.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.503+0000 7efe3fe00640 1 -- 172.21.15.83:0/4187798041 learned_addr learned my addr 172.21.15.83:0/4187798041 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:55:42.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.503+0000 7efe3ea00640 1 --2- 172.21.15.83:0/4187798041 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55595e6839b0 0x55595e691950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:42.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.503+0000 7efe3fe00640 1 -- 172.21.15.83:0/4187798041 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55595e68e080 msgr2=0x55595e691e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:42.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.503+0000 7efe3fe00640 1 --2- 172.21.15.83:0/4187798041 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55595e68e080 0x55595e691e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:42.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.503+0000 7efe3fe00640 1 -- 172.21.15.83:0/4187798041 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55595e6839b0 msgr2=0x55595e691950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:42.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.503+0000 7efe3fe00640 1 --2- 172.21.15.83:0/4187798041 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55595e6839b0 0x55595e691950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:42.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.503+0000 7efe3fe00640 1 -- 172.21.15.83:0/4187798041 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55595e6926c0 con 0x55595e683090 2024-04-24T08:55:42.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.504+0000 7efe3fe00640 1 --2- 172.21.15.83:0/4187798041 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55595e683090 0x55595e6965e0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7efe38008ba0 tx=0x7efe3800f9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9656dd21684675f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:42.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.504+0000 7efe2d600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:55:42.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.504+0000 7efe2d600640 1 -- 172.21.15.83:0/4187798041 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe38045020 con 0x55595e683090 2024-04-24T08:55:42.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.504+0000 7efe2d600640 1 -- 172.21.15.83:0/4187798041 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7efe38027a60 con 0x55595e683090 2024-04-24T08:55:42.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.504+0000 7efe49383740 1 -- 172.21.15.83:0/4187798041 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe38027400 con 0x55595e683090 2024-04-24T08:55:42.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.504+0000 7efe2d600640 1 -- 172.21.15.83:0/4187798041 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7efe38050050 con 0x55595e683090 2024-04-24T08:55:42.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.504+0000 7efe49383740 10 client.17181 fetch_fsmap learned FSMap version 30 2024-04-24T08:55:42.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.504+0000 7efe49383740 1 -- 172.21.15.83:0/4187798041 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55595e6926c0 con 0x55595e683090 2024-04-24T08:55:42.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.505+0000 7efe2d600640 1 -- 172.21.15.83:0/4187798041 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efe38027400 con 0x55595e683090 2024-04-24T08:55:42.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.505+0000 7efe2d600640 1 -- 172.21.15.83:0/4187798041 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7efe38004c50 con 0x55595e683090 2024-04-24T08:55:42.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.505+0000 7efe49383740 10 client.17181 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T08:55:42.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.505+0000 7efe49383740 10 client.17181 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:55:42.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.505+0000 7efe49383740 20 client.17181 populate_metadata read hostname 'smithi083' 2024-04-24T08:55:42.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.505+0000 7efe49383740 1 --2- 172.21.15.83:0/4187798041 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55595e704b90 0x55595e706fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:42.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.505+0000 7efe49383740 4 client.17181 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T08:55:42.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.505+0000 7efe49383740 1 -- 172.21.15.83:0/4187798041 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55595e5f0700 con 0x55595e704b90 2024-04-24T08:55:42.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.506+0000 7efe3ea00640 1 --2- 172.21.15.83:0/4187798041 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55595e704b90 0x55595e706fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:42.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.506+0000 7efe3ea00640 1 --2- 172.21.15.83:0/4187798041 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55595e704b90 0x55595e706fb0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6deb38bd24459d5 server_cookie=359c104c5b7495e7 in_seq=0 out_seq=0 2024-04-24T08:55:42.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.506+0000 7efe2d600640 10 client.17181 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:55:42.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.509+0000 7efe2d600640 1 -- 172.21.15.83:0/4187798041 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55595e5f0700 con 0x55595e704b90 2024-04-24T08:55:42.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.509+0000 7efe2d600640 10 client.17181 handle_command_reply: tid=0 2024-04-24T08:55:42.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.509+0000 7efe49383740 1 client.17181 shutdown 2024-04-24T08:55:42.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.509+0000 7efe49383740 1 -- 172.21.15.83:0/4187798041 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55595e683090 msgr2=0x55595e6965e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:42.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.509+0000 7efe49383740 1 --2- 172.21.15.83:0/4187798041 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55595e683090 0x55595e6965e0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7efe38008ba0 tx=0x7efe3800f9f0 comp rx=0 tx=0).stop 2024-04-24T08:55:42.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.509+0000 7efe49383740 1 -- 172.21.15.83:0/4187798041 shutdown_connections 2024-04-24T08:55:42.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.509+0000 7efe49383740 1 --2- 172.21.15.83:0/4187798041 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55595e704b90 0x55595e706fb0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:42.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.509+0000 7efe49383740 1 --2- 172.21.15.83:0/4187798041 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55595e68e080 0x55595e691e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:42.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.509+0000 7efe49383740 1 --2- 172.21.15.83:0/4187798041 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55595e6839b0 0x55595e691950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:42.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.509+0000 7efe2d600640 0 client.17181 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:55:42.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.509+0000 7efe49383740 1 --2- 172.21.15.83:0/4187798041 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55595e683090 0x55595e6965e0 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:42.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.509+0000 7efe49383740 1 -- 172.21.15.83:0/4187798041 >> 172.21.15.83:0/4187798041 conn(0x55595e67f3b0 msgr2=0x55595e67dc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:42.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.510+0000 7efe49383740 1 -- 172.21.15.83:0/4187798041 shutdown_connections 2024-04-24T08:55:42.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.510+0000 7efe49383740 1 -- 172.21.15.83:0/4187798041 wait complete. 2024-04-24T08:55:42.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.510+0000 7efe49383740 20 client.17181 trim_cache size 0 max 16384 2024-04-24T08:55:42.511 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T08:55:42.511 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-04-24T08:55:42.511 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "96970829-67bb-4965-98fe-9390020ea009", 2024-04-24T08:55:42.511 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-04-24T08:55:42.511 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T08:55:42.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.510+0000 7efe46600640 1 -- 172.21.15.83:0/1441939279 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7efe240a3d20 msgr2=0x7efe240a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:42.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.510+0000 7efe46600640 1 --2- 172.21.15.83:0/1441939279 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7efe240a3d20 0x7efe240a61e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x55595e684650 tx=0x7efe30000bc0 comp rx=0 tx=0).stop 2024-04-24T08:55:42.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.510+0000 7efe46600640 1 -- 172.21.15.83:0/1441939279 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efe40104450 msgr2=0x7efe401a06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:42.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.510+0000 7efe46600640 1 --2- 172.21.15.83:0/1441939279 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efe40104450 0x7efe401a06a0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7efe3000c920 tx=0x7efe3000cde0 comp rx=0 tx=0).stop 2024-04-24T08:55:42.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.511+0000 7efe46600640 1 -- 172.21.15.83:0/1441939279 shutdown_connections 2024-04-24T08:55:42.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.511+0000 7efe46600640 1 --2- 172.21.15.83:0/1441939279 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7efe240a3d20 0x7efe240a61e0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:42.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.511+0000 7efe46600640 1 --2- 172.21.15.83:0/1441939279 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe4019a810 0x7efe4019acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:42.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.511+0000 7efe46600640 1 --2- 172.21.15.83:0/1441939279 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efe40109520 0x7efe401a0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:42.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.511+0000 7efe46600640 1 --2- 172.21.15.83:0/1441939279 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efe40104450 0x7efe401a06a0 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:42.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.511+0000 7efe46600640 1 -- 172.21.15.83:0/1441939279 >> 172.21.15.83:0/1441939279 conn(0x7efe400ffc70 msgr2=0x7efe40100f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:42.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.511+0000 7efe46600640 1 -- 172.21.15.83:0/1441939279 shutdown_connections 2024-04-24T08:55:42.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.511+0000 7efe46600640 1 -- 172.21.15.83:0/1441939279 wait complete. 2024-04-24T08:55:42.531 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T08:55:42.531 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T08:55:42.569 INFO:tasks.workunit.client.0.smithi083.stdout: 1 16912 0.68 MB/sec execute 16 sec latency 56.789 ms 2024-04-24T08:55:42.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.748+0000 7f2422800640 1 -- 172.21.15.83:0/4294390546 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f241c103ba0 msgr2=0x7f241c10c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:42.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.748+0000 7f2422800640 1 --2- 172.21.15.83:0/4294390546 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f241c103ba0 0x7f241c10c080 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f2408009900 tx=0x7f240802a510 comp rx=0 tx=0).stop 2024-04-24T08:55:42.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.749+0000 7f2422800640 1 -- 172.21.15.83:0/4294390546 shutdown_connections 2024-04-24T08:55:42.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.749+0000 7f2422800640 1 --2- 172.21.15.83:0/4294390546 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f241c10c5c0 0x7f241c10ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:42.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.749+0000 7f2422800640 1 --2- 172.21.15.83:0/4294390546 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f241c103ba0 0x7f241c10c080 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:42.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.749+0000 7f2422800640 1 --2- 172.21.15.83:0/4294390546 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f241c103280 0x7f241c103660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:42.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.749+0000 7f2422800640 1 -- 172.21.15.83:0/4294390546 >> 172.21.15.83:0/4294390546 conn(0x7f241c078000 msgr2=0x7f241c078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:42.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.749+0000 7f2422800640 1 -- 172.21.15.83:0/4294390546 shutdown_connections 2024-04-24T08:55:42.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.749+0000 7f2422800640 1 -- 172.21.15.83:0/4294390546 wait complete. 2024-04-24T08:55:42.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.750+0000 7f2422800640 1 Processor -- start 2024-04-24T08:55:42.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.750+0000 7f2422800640 1 -- start start 2024-04-24T08:55:42.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.750+0000 7f2422800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f241c103280 0x7f241c1a06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:42.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.750+0000 7f2422800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f241c103ba0 0x7f241c1a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:42.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.750+0000 7f2422800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f241c10c5c0 0x7f241c19a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:42.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.750+0000 7f2422800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f241c1122b0 con 0x7f241c103ba0 2024-04-24T08:55:42.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.750+0000 7f2422800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f241c112130 con 0x7f241c103280 2024-04-24T08:55:42.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.750+0000 7f2422800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f241c112430 con 0x7f241c10c5c0 2024-04-24T08:55:42.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.751+0000 7f241aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f241c103ba0 0x7f241c1a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:42.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.751+0000 7f241aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f241c103ba0 0x7f241c1a0c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55724/0 (socket says 172.21.15.83:55724) 2024-04-24T08:55:42.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.751+0000 7f241aa00640 1 -- 172.21.15.83:0/207859414 learned_addr learned my addr 172.21.15.83:0/207859414 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:55:42.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.751+0000 7f241b400640 1 --2- 172.21.15.83:0/207859414 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f241c103280 0x7f241c1a06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:42.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.751+0000 7f241be00640 1 --2- 172.21.15.83:0/207859414 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f241c10c5c0 0x7f241c19a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:42.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.751+0000 7f241aa00640 1 -- 172.21.15.83:0/207859414 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f241c10c5c0 msgr2=0x7f241c19a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:42.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.751+0000 7f241aa00640 1 --2- 172.21.15.83:0/207859414 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f241c10c5c0 0x7f241c19a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:42.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.751+0000 7f241aa00640 1 -- 172.21.15.83:0/207859414 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f241c103280 msgr2=0x7f241c1a06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:42.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.751+0000 7f241aa00640 1 --2- 172.21.15.83:0/207859414 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f241c103280 0x7f241c1a06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:42.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.751+0000 7f241aa00640 1 -- 172.21.15.83:0/207859414 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f241c19b010 con 0x7f241c103ba0 2024-04-24T08:55:42.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.751+0000 7f241b400640 1 --2- 172.21.15.83:0/207859414 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f241c103280 0x7f241c1a06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:55:42.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.751+0000 7f241aa00640 1 --2- 172.21.15.83:0/207859414 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f241c103ba0 0x7f241c1a0c20 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f2408006010 tx=0x7f2408004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:42.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.752+0000 7f2413e00640 1 -- 172.21.15.83:0/207859414 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f240803f030 con 0x7f241c103ba0 2024-04-24T08:55:42.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.752+0000 7f2413e00640 1 -- 172.21.15.83:0/207859414 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f24080035f0 con 0x7f241c103ba0 2024-04-24T08:55:42.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.752+0000 7f2422800640 1 -- 172.21.15.83:0/207859414 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f241c19b2a0 con 0x7f241c103ba0 2024-04-24T08:55:42.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.752+0000 7f2413e00640 1 -- 172.21.15.83:0/207859414 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2408035070 con 0x7f241c103ba0 2024-04-24T08:55:42.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.752+0000 7f2422800640 1 -- 172.21.15.83:0/207859414 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f241c1a75a0 con 0x7f241c103ba0 2024-04-24T08:55:42.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.756+0000 7f2413e00640 1 -- 172.21.15.83:0/207859414 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2408003b10 con 0x7f241c103ba0 2024-04-24T08:55:42.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.756+0000 7f24255ff740 1 Processor -- start 2024-04-24T08:55:42.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.756+0000 7f24255ff740 1 -- start start 2024-04-24T08:55:42.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.756+0000 7f24255ff740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e12bd69860 0x55e12bd69c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:42.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.756+0000 7f24255ff740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e12bd6a180 0x55e12bd74310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:42.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.756+0000 7f24255ff740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e12bd74850 0x55e12bd76c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:42.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.756+0000 7f24255ff740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e12bccf1f0 con 0x55e12bd74850 2024-04-24T08:55:42.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.756+0000 7f24255ff740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e12bc75350 con 0x55e12bd6a180 2024-04-24T08:55:42.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.756+0000 7f24255ff740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55e12bccb1b0 con 0x55e12bd69860 2024-04-24T08:55:42.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.757+0000 7f241aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e12bd74850 0x55e12bd76c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:42.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.757+0000 7f241aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e12bd74850 0x55e12bd76c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55734/0 (socket says 172.21.15.83:55734) 2024-04-24T08:55:42.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.757+0000 7f241aa00640 1 -- 172.21.15.83:0/3111953068 learned_addr learned my addr 172.21.15.83:0/3111953068 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:55:42.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.757+0000 7f241be00640 1 --2- 172.21.15.83:0/3111953068 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e12bd69860 0x55e12bd69c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:42.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.757+0000 7f241b400640 1 --2- 172.21.15.83:0/3111953068 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e12bd6a180 0x55e12bd74310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:42.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.757+0000 7f241aa00640 1 -- 172.21.15.83:0/3111953068 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e12bd69860 msgr2=0x55e12bd69c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:42.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.757+0000 7f241aa00640 1 --2- 172.21.15.83:0/3111953068 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e12bd69860 0x55e12bd69c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:42.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.757+0000 7f241aa00640 1 -- 172.21.15.83:0/3111953068 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e12bd6a180 msgr2=0x55e12bd74310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:42.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.757+0000 7f241aa00640 1 --2- 172.21.15.83:0/3111953068 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e12bd6a180 0x55e12bd74310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:42.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.757+0000 7f241aa00640 1 -- 172.21.15.83:0/3111953068 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e12bcc9f50 con 0x55e12bd74850 2024-04-24T08:55:42.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.757+0000 7f2413e00640 1 --2- 172.21.15.83:0/207859414 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f24000a3da0 0x7f24000a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:42.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.758+0000 7f2413e00640 1 -- 172.21.15.83:0/207859414 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f24080f1800 con 0x7f241c103ba0 2024-04-24T08:55:42.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.758+0000 7f241aa00640 1 --2- 172.21.15.83:0/3111953068 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e12bd74850 0x55e12bd76c40 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f24080eea60 tx=0x7f24080f7d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc9487e8447eb575 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:42.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.758+0000 7f241be00640 1 --2- 172.21.15.83:0/207859414 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f24000a3da0 0x7f24000a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:42.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.758+0000 7f2410c00640 1 -- 172.21.15.83:0/3111953068 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2408049030 con 0x55e12bd74850 2024-04-24T08:55:42.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.758+0000 7f2410c00640 1 -- 172.21.15.83:0/3111953068 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f24080bda70 con 0x55e12bd74850 2024-04-24T08:55:42.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.758+0000 7f2410c00640 1 -- 172.21.15.83:0/3111953068 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24080bceb0 con 0x55e12bd74850 2024-04-24T08:55:42.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.759+0000 7f24255ff740 1 -- 172.21.15.83:0/3111953068 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e12bd74850 msgr2=0x55e12bd76c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:42.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.759+0000 7f241be00640 1 --2- 172.21.15.83:0/207859414 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f24000a3da0 0x7f24000a6260 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2414007ed0 tx=0x7f24140075c0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:42.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.759+0000 7f24255ff740 1 --2- 172.21.15.83:0/3111953068 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e12bd74850 0x55e12bd76c40 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f24080eea60 tx=0x7f24080f7d90 comp rx=0 tx=0).stop 2024-04-24T08:55:42.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.759+0000 7f24255ff740 1 -- 172.21.15.83:0/3111953068 shutdown_connections 2024-04-24T08:55:42.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.759+0000 7f24255ff740 1 --2- 172.21.15.83:0/3111953068 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e12bd74850 0x55e12bd76c40 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:42.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.759+0000 7f24255ff740 1 --2- 172.21.15.83:0/3111953068 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e12bd6a180 0x55e12bd74310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:42.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.759+0000 7f24255ff740 1 --2- 172.21.15.83:0/3111953068 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e12bd69860 0x55e12bd69c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:42.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.759+0000 7f24255ff740 1 -- 172.21.15.83:0/3111953068 >> 172.21.15.83:0/3111953068 conn(0x55e12bd65b80 msgr2=0x55e12bd65f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:42.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.759+0000 7f24255ff740 1 -- 172.21.15.83:0/3111953068 shutdown_connections 2024-04-24T08:55:42.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.759+0000 7f24255ff740 1 -- 172.21.15.83:0/3111953068 wait complete. 2024-04-24T08:55:42.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.759+0000 7f24255ff740 1 -- start start 2024-04-24T08:55:42.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.760+0000 7f24255ff740 1 Processor -- start 2024-04-24T08:55:42.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.760+0000 7f24255ff740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e12bd69860 0x55e12bd71730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:42.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.760+0000 7f24255ff740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e12bd6a180 0x55e12bd71c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:42.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.760+0000 7f24255ff740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e12bd74850 0x55e12bd721b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:42.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.760+0000 7f24255ff740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e12bd77400 con 0x55e12bd6a180 2024-04-24T08:55:42.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.760+0000 7f24255ff740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e12bc88690 con 0x55e12bd69860 2024-04-24T08:55:42.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.760+0000 7f24255ff740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55e12bd77580 con 0x55e12bd74850 2024-04-24T08:55:42.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.760+0000 7f24255ff740 10 client.0 _ll_register_callbacks cb 0x55e12bcc4650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:55:42.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.760+0000 7f241aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e12bd6a180 0x55e12bd71c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:42.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.761+0000 7f241aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e12bd6a180 0x55e12bd71c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55748/0 (socket says 172.21.15.83:55748) 2024-04-24T08:55:42.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.761+0000 7f241aa00640 1 -- 172.21.15.83:0/851225317 learned_addr learned my addr 172.21.15.83:0/851225317 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:55:42.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.761+0000 7f241be00640 1 --2- 172.21.15.83:0/851225317 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e12bd74850 0x55e12bd721b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:42.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.761+0000 7f241b400640 1 --2- 172.21.15.83:0/851225317 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e12bd69860 0x55e12bd71730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:42.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.761+0000 7f241aa00640 1 -- 172.21.15.83:0/851225317 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e12bd74850 msgr2=0x55e12bd721b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:42.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.761+0000 7f241aa00640 1 --2- 172.21.15.83:0/851225317 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e12bd74850 0x55e12bd721b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:42.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.761+0000 7f241aa00640 1 -- 172.21.15.83:0/851225317 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e12bd69860 msgr2=0x55e12bd71730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:42.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.761+0000 7f241aa00640 1 --2- 172.21.15.83:0/851225317 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e12bd69860 0x55e12bd71730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:42.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.761+0000 7f241aa00640 1 -- 172.21.15.83:0/851225317 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e12bc87920 con 0x55e12bd6a180 2024-04-24T08:55:42.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.761+0000 7f241b400640 1 --2- 172.21.15.83:0/851225317 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e12bd69860 0x55e12bd71730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:55:42.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.761+0000 7f241aa00640 1 --2- 172.21.15.83:0/851225317 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e12bd6a180 0x55e12bd71c70 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f24080bdc20 tx=0x7f24080f79c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29a7be42e0d9f665 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:42.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.761+0000 7f2405600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:55:42.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.761+0000 7f2405600640 1 -- 172.21.15.83:0/851225317 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2408049040 con 0x55e12bd6a180 2024-04-24T08:55:42.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.761+0000 7f2405600640 1 -- 172.21.15.83:0/851225317 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f24080eeda0 con 0x55e12bd6a180 2024-04-24T08:55:42.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.762+0000 7f24255ff740 1 -- 172.21.15.83:0/851225317 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24080bda70 con 0x55e12bd6a180 2024-04-24T08:55:42.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.762+0000 7f2405600640 1 -- 172.21.15.83:0/851225317 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f24080037a0 con 0x55e12bd6a180 2024-04-24T08:55:42.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.762+0000 7f24255ff740 10 client.17193 fetch_fsmap learned FSMap version 30 2024-04-24T08:55:42.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.762+0000 7f24255ff740 1 -- 172.21.15.83:0/851225317 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e12baab7d0 con 0x55e12bd6a180 2024-04-24T08:55:42.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.762+0000 7f2405600640 1 -- 172.21.15.83:0/851225317 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f24080bda70 con 0x55e12bd6a180 2024-04-24T08:55:42.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.762+0000 7f2405600640 1 -- 172.21.15.83:0/851225317 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7f24080dc170 con 0x55e12bd6a180 2024-04-24T08:55:42.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.762+0000 7f24255ff740 10 client.17193 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T08:55:42.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.762+0000 7f24255ff740 10 client.17193 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:55:42.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.762+0000 7f24255ff740 20 client.17193 populate_metadata read hostname 'smithi083' 2024-04-24T08:55:42.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.762+0000 7f24255ff740 1 --2- 172.21.15.83:0/851225317 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55e12bd84690 0x55e12bd86a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:42.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.762+0000 7f24255ff740 4 client.17193 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:55:42.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.762+0000 7f24255ff740 1 -- 172.21.15.83:0/851225317 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e12bcdcc10 con 0x55e12bd84690 2024-04-24T08:55:42.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.763+0000 7f241b400640 1 --2- 172.21.15.83:0/851225317 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55e12bd84690 0x55e12bd86a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:42.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.764+0000 7f241b400640 1 --2- 172.21.15.83:0/851225317 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55e12bd84690 0x55e12bd86a80 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=187292de3246aa8c server_cookie=6c265d6c64e73e0b in_seq=0 out_seq=0 2024-04-24T08:55:42.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.764+0000 7f2405600640 10 client.17193 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:55:42.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.766+0000 7f2405600640 1 -- 172.21.15.83:0/851225317 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e12bcdcc10 con 0x55e12bd84690 2024-04-24T08:55:42.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.766+0000 7f2405600640 10 client.17193 handle_command_reply: tid=0 2024-04-24T08:55:42.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.766+0000 7f24255ff740 1 client.17193 shutdown 2024-04-24T08:55:42.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.766+0000 7f24255ff740 1 -- 172.21.15.83:0/851225317 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e12bd6a180 msgr2=0x55e12bd71c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:42.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.766+0000 7f24255ff740 1 --2- 172.21.15.83:0/851225317 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e12bd6a180 0x55e12bd71c70 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f24080bdc20 tx=0x7f24080f79c0 comp rx=0 tx=0).stop 2024-04-24T08:55:42.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.766+0000 7f24255ff740 1 -- 172.21.15.83:0/851225317 shutdown_connections 2024-04-24T08:55:42.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.766+0000 7f24255ff740 1 --2- 172.21.15.83:0/851225317 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55e12bd84690 0x55e12bd86a80 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:42.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.766+0000 7f24255ff740 1 --2- 172.21.15.83:0/851225317 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e12bd74850 0x55e12bd721b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:42.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.766+0000 7f24255ff740 1 --2- 172.21.15.83:0/851225317 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e12bd6a180 0x55e12bd71c70 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:42.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.766+0000 7f24255ff740 1 --2- 172.21.15.83:0/851225317 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e12bd69860 0x55e12bd71730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:42.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.766+0000 7f24255ff740 1 -- 172.21.15.83:0/851225317 >> 172.21.15.83:0/851225317 conn(0x55e12bd65b80 msgr2=0x55e12bd6b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:42.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.766+0000 7f2405600640 0 client.17193 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:55:42.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.766+0000 7f24255ff740 1 -- 172.21.15.83:0/851225317 shutdown_connections 2024-04-24T08:55:42.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.766+0000 7f24255ff740 1 -- 172.21.15.83:0/851225317 wait complete. 2024-04-24T08:55:42.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.766+0000 7f24255ff740 20 client.17193 trim_cache size 0 max 16384 2024-04-24T08:55:42.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.786+0000 7f24255ff740 1 Processor -- start 2024-04-24T08:55:42.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.786+0000 7f24255ff740 1 -- start start 2024-04-24T08:55:42.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.786+0000 7f24255ff740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e12bd69860 0x55e12bd80230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:42.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.786+0000 7f24255ff740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e12bd6a180 0x55e12bd70670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:42.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.786+0000 7f24255ff740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e12bd74850 0x55e12bd6c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:42.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.786+0000 7f24255ff740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f241c1b1730 con 0x55e12bd6a180 2024-04-24T08:55:42.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.786+0000 7f24255ff740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f241c1b3300 con 0x55e12bd74850 2024-04-24T08:55:42.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.786+0000 7f24255ff740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f241c1b1e00 con 0x55e12bd69860 2024-04-24T08:55:42.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.787+0000 7f241aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e12bd6a180 0x55e12bd70670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:42.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.787+0000 7f241b400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e12bd69860 0x55e12bd80230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:42.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.787+0000 7f241aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e12bd6a180 0x55e12bd70670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55752/0 (socket says 172.21.15.83:55752) 2024-04-24T08:55:42.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.787+0000 7f241aa00640 1 -- 172.21.15.83:0/813300081 learned_addr learned my addr 172.21.15.83:0/813300081 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:55:42.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.787+0000 7f241be00640 1 --2- 172.21.15.83:0/813300081 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e12bd74850 0x55e12bd6c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:42.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.787+0000 7f241aa00640 1 -- 172.21.15.83:0/813300081 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e12bd69860 msgr2=0x55e12bd80230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:42.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.787+0000 7f241aa00640 1 --2- 172.21.15.83:0/813300081 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e12bd69860 0x55e12bd80230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:42.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.787+0000 7f241aa00640 1 -- 172.21.15.83:0/813300081 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e12bd74850 msgr2=0x55e12bd6c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:42.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.787+0000 7f241aa00640 1 --2- 172.21.15.83:0/813300081 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e12bd74850 0x55e12bd6c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:42.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.787+0000 7f241aa00640 1 -- 172.21.15.83:0/813300081 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e12b9e13f0 con 0x55e12bd6a180 2024-04-24T08:55:42.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.787+0000 7f241be00640 1 --2- 172.21.15.83:0/813300081 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e12bd74850 0x55e12bd6c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:55:42.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.787+0000 7f241aa00640 1 --2- 172.21.15.83:0/813300081 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e12bd6a180 0x55e12bd70670 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f2408005ec0 tx=0x7f24080f7d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91c979c9bd91f37a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:42.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.788+0000 7f2407e00640 1 -- 172.21.15.83:0/813300081 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2408049070 con 0x55e12bd6a180 2024-04-24T08:55:42.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.788+0000 7f2407e00640 1 -- 172.21.15.83:0/813300081 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f24080bda70 con 0x55e12bd6a180 2024-04-24T08:55:42.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.788+0000 7f2407e00640 1 -- 172.21.15.83:0/813300081 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24080bccd0 con 0x55e12bd6a180 2024-04-24T08:55:42.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.788+0000 7f24255ff740 1 -- 172.21.15.83:0/813300081 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e12bd6a180 msgr2=0x55e12bd70670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:42.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.788+0000 7f24255ff740 1 --2- 172.21.15.83:0/813300081 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e12bd6a180 0x55e12bd70670 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f2408005ec0 tx=0x7f24080f7d90 comp rx=0 tx=0).stop 2024-04-24T08:55:42.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.788+0000 7f24255ff740 1 -- 172.21.15.83:0/813300081 shutdown_connections 2024-04-24T08:55:42.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.789+0000 7f24255ff740 1 --2- 172.21.15.83:0/813300081 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e12bd74850 0x55e12bd6c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:42.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.789+0000 7f24255ff740 1 --2- 172.21.15.83:0/813300081 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e12bd6a180 0x55e12bd70670 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:42.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.789+0000 7f24255ff740 1 --2- 172.21.15.83:0/813300081 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e12bd69860 0x55e12bd80230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:42.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.789+0000 7f24255ff740 1 -- 172.21.15.83:0/813300081 >> 172.21.15.83:0/813300081 conn(0x55e12bd65b80 msgr2=0x55e12bd64c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:42.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.789+0000 7f24255ff740 1 -- 172.21.15.83:0/813300081 shutdown_connections 2024-04-24T08:55:42.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.789+0000 7f24255ff740 1 -- 172.21.15.83:0/813300081 wait complete. 2024-04-24T08:55:42.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.789+0000 7f24255ff740 1 -- start start 2024-04-24T08:55:42.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.789+0000 7f24255ff740 1 Processor -- start 2024-04-24T08:55:42.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.790+0000 7f24255ff740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e12bd69860 0x55e12bd7cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:42.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.790+0000 7f24255ff740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e12bd6a180 0x55e12bd77fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:42.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.790+0000 7f24255ff740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e12bd74850 0x55e12bd78700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:42.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.790+0000 7f24255ff740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f241c1b2ba0 con 0x55e12bd74850 2024-04-24T08:55:42.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.790+0000 7f24255ff740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f241c1b24d0 con 0x55e12bd6a180 2024-04-24T08:55:42.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.790+0000 7f24255ff740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f241c1af710 con 0x55e12bd69860 2024-04-24T08:55:42.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.790+0000 7f24255ff740 10 client.0 _ll_register_callbacks cb 0x7f2414008040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:55:42.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.790+0000 7f241be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e12bd74850 0x55e12bd78700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:42.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.790+0000 7f241be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e12bd74850 0x55e12bd78700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55756/0 (socket says 172.21.15.83:55756) 2024-04-24T08:55:42.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.790+0000 7f241be00640 1 -- 172.21.15.83:0/3743411357 learned_addr learned my addr 172.21.15.83:0/3743411357 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:55:42.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.790+0000 7f241b400640 1 --2- 172.21.15.83:0/3743411357 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e12bd69860 0x55e12bd7cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:42.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.791+0000 7f241aa00640 1 --2- 172.21.15.83:0/3743411357 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e12bd6a180 0x55e12bd77fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:42.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.791+0000 7f241be00640 1 -- 172.21.15.83:0/3743411357 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e12bd69860 msgr2=0x55e12bd7cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:42.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.791+0000 7f241be00640 1 --2- 172.21.15.83:0/3743411357 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e12bd69860 0x55e12bd7cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:42.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.791+0000 7f241be00640 1 -- 172.21.15.83:0/3743411357 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e12bd6a180 msgr2=0x55e12bd77fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:42.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.791+0000 7f241be00640 1 --2- 172.21.15.83:0/3743411357 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e12bd6a180 0x55e12bd77fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:42.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.791+0000 7f241be00640 1 -- 172.21.15.83:0/3743411357 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e12b9a10f0 con 0x55e12bd74850 2024-04-24T08:55:42.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.791+0000 7f241be00640 1 --2- 172.21.15.83:0/3743411357 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e12bd74850 0x55e12bd78700 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f2414012e50 tx=0x7f2414004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f6071c7ec1a9da6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:42.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.791+0000 7f2405600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:55:42.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.791+0000 7f2405600640 1 -- 172.21.15.83:0/3743411357 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2414029600 con 0x55e12bd74850 2024-04-24T08:55:42.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.791+0000 7f2405600640 1 -- 172.21.15.83:0/3743411357 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f24140297a0 con 0x55e12bd74850 2024-04-24T08:55:42.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.791+0000 7f24255ff740 1 -- 172.21.15.83:0/3743411357 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24080bda70 con 0x55e12bd74850 2024-04-24T08:55:42.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.792+0000 7f2405600640 1 -- 172.21.15.83:0/3743411357 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2414032050 con 0x55e12bd74850 2024-04-24T08:55:42.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.792+0000 7f24255ff740 10 client.17199 fetch_fsmap learned FSMap version 30 2024-04-24T08:55:42.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.792+0000 7f24255ff740 1 -- 172.21.15.83:0/3743411357 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e12bc94810 con 0x55e12bd74850 2024-04-24T08:55:42.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.792+0000 7f2405600640 1 -- 172.21.15.83:0/3743411357 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f24080bda70 con 0x55e12bd74850 2024-04-24T08:55:42.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.792+0000 7f2405600640 1 -- 172.21.15.83:0/3743411357 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7f2414011040 con 0x55e12bd74850 2024-04-24T08:55:42.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.792+0000 7f24255ff740 10 client.17199 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T08:55:42.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.792+0000 7f24255ff740 10 client.17199 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:55:42.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.792+0000 7f24255ff740 20 client.17199 populate_metadata read hostname 'smithi083' 2024-04-24T08:55:42.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.792+0000 7f24255ff740 1 --2- 172.21.15.83:0/3743411357 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55e12bdeb1f0 0x55e12bded610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:42.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.792+0000 7f24255ff740 4 client.17199 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T08:55:42.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.792+0000 7f24255ff740 1 -- 172.21.15.83:0/3743411357 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e12bcd6a80 con 0x55e12bdeb1f0 2024-04-24T08:55:42.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.793+0000 7f241b400640 1 --2- 172.21.15.83:0/3743411357 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55e12bdeb1f0 0x55e12bded610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:42.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.794+0000 7f241b400640 1 --2- 172.21.15.83:0/3743411357 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55e12bdeb1f0 0x55e12bded610 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24eb49d212ca75df server_cookie=af5cbe0a2610cdf5 in_seq=0 out_seq=0 2024-04-24T08:55:42.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.794+0000 7f2405600640 10 client.17199 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:55:42.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.796+0000 7f2405600640 1 -- 172.21.15.83:0/3743411357 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55e12bcd6a80 con 0x55e12bdeb1f0 2024-04-24T08:55:42.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.796+0000 7f2405600640 10 client.17199 handle_command_reply: tid=0 2024-04-24T08:55:42.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.796+0000 7f24255ff740 1 client.17199 shutdown 2024-04-24T08:55:42.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.796+0000 7f24255ff740 1 -- 172.21.15.83:0/3743411357 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e12bd74850 msgr2=0x55e12bd78700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:42.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.796+0000 7f24255ff740 1 --2- 172.21.15.83:0/3743411357 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e12bd74850 0x55e12bd78700 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f2414012e50 tx=0x7f2414004a40 comp rx=0 tx=0).stop 2024-04-24T08:55:42.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.796+0000 7f24255ff740 1 -- 172.21.15.83:0/3743411357 shutdown_connections 2024-04-24T08:55:42.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.796+0000 7f24255ff740 1 --2- 172.21.15.83:0/3743411357 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55e12bdeb1f0 0x55e12bded610 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:42.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.796+0000 7f24255ff740 1 --2- 172.21.15.83:0/3743411357 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e12bd74850 0x55e12bd78700 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:42.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.796+0000 7f24255ff740 1 --2- 172.21.15.83:0/3743411357 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e12bd6a180 0x55e12bd77fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:42.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.796+0000 7f24255ff740 1 --2- 172.21.15.83:0/3743411357 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e12bd69860 0x55e12bd7cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:42.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.796+0000 7f24255ff740 1 -- 172.21.15.83:0/3743411357 >> 172.21.15.83:0/3743411357 conn(0x55e12bd65b80 msgr2=0x55e12bd71520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:42.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.796+0000 7f2405600640 0 client.17199 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:55:42.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.796+0000 7f24255ff740 1 -- 172.21.15.83:0/3743411357 shutdown_connections 2024-04-24T08:55:42.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.796+0000 7f24255ff740 1 -- 172.21.15.83:0/3743411357 wait complete. 2024-04-24T08:55:42.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.797+0000 7f24255ff740 20 client.17199 trim_cache size 0 max 16384 2024-04-24T08:55:42.799 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T08:55:42.799 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T08:55:42.800 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-04-24T08:55:42.800 INFO:teuthology.orchestra.run.smithi083.stdout: "96970829-67bb-4965-98fe-9390020ea009": { 2024-04-24T08:55:42.800 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-04-24T08:55:42.800 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "96970829-67bb-4965-98fe-9390020ea009", 2024-04-24T08:55:42.800 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-04-24T08:55:42.800 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T08:55:42.800 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T08:55:42.800 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T08:55:42.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.797+0000 7f2422800640 1 -- 172.21.15.83:0/207859414 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f24000a3da0 msgr2=0x7f24000a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:42.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.797+0000 7f2422800640 1 --2- 172.21.15.83:0/207859414 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f24000a3da0 0x7f24000a6260 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2414007ed0 tx=0x7f24140075c0 comp rx=0 tx=0).stop 2024-04-24T08:55:42.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.797+0000 7f2422800640 1 -- 172.21.15.83:0/207859414 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f241c103ba0 msgr2=0x7f241c1a0c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:42.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.797+0000 7f2422800640 1 --2- 172.21.15.83:0/207859414 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f241c103ba0 0x7f241c1a0c20 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f2408006010 tx=0x7f2408004270 comp rx=0 tx=0).stop 2024-04-24T08:55:42.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.797+0000 7f2422800640 1 -- 172.21.15.83:0/207859414 shutdown_connections 2024-04-24T08:55:42.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.797+0000 7f2422800640 1 --2- 172.21.15.83:0/207859414 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f24000a3da0 0x7f24000a6260 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:42.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.797+0000 7f2422800640 1 --2- 172.21.15.83:0/207859414 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f241c10c5c0 0x7f241c19a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:42.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.797+0000 7f2422800640 1 --2- 172.21.15.83:0/207859414 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f241c103ba0 0x7f241c1a0c20 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:42.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.797+0000 7f2422800640 1 --2- 172.21.15.83:0/207859414 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f241c103280 0x7f241c1a06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:42.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.797+0000 7f2422800640 1 -- 172.21.15.83:0/207859414 >> 172.21.15.83:0/207859414 conn(0x7f241c078000 msgr2=0x7f241c108530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:42.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.798+0000 7f2422800640 1 -- 172.21.15.83:0/207859414 shutdown_connections 2024-04-24T08:55:42.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:55:42.798+0000 7f2422800640 1 -- 172.21.15.83:0/207859414 wait complete. 2024-04-24T08:55:42.815 INFO:tasks.cephfs.filesystem:scrub status for tag:96970829-67bb-4965-98fe-9390020ea009 - {'path': '/', 'tag': '96970829-67bb-4965-98fe-9390020ea009', 'options': 'recursive,force'} 2024-04-24T08:55:42.955 INFO:journalctl@ceph.mds.b.smithi138.stdout:Apr 24 08:55:42 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-b[61515]: 2024-04-24T08:55:42.538+0000 7f7fbbc00700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000215 [...6,head] /volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/COREL/ auth{2=1} v3010 ap=2 f(v104 m2024-04-24T08:55:26.050891+0000 11=11+0) n(v4 rc2024-04-24T08:55:26.050891+0000 b2861056 12=11+1) old_inodes=2 (inest sync->mix g=2) caps={15603=pAsLsXsFs/-@2383} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x55cf8760d700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000000215:[<0x10000000009/COREL v369>,<0x10000000008/~dmtmp v57>,<0x10000000007/client0 v33>,<0x10000000006/clients v23>,<0x10000000005/tmp v23>,<0x10000000003/client.0 v22>,<0x10000000002/e395bed6-5bba-41b8-a256-3505e4afcca2 v36>,<0x10000000001/sv_0 v55>,<0x10000000000/qa v43>,<0x1/volumes v39>]//[]","memoryvalue":"(2)0x10000000215:[<0x10000000009/COREL v3010>,<0x10000000008/~dmtmp v2459>,<0x10000000007/client0 v321>,<0x10000000006/clients v311>,<0x10000000005/tmp v309>,<0x10000000003/client.0 v306>,<0x10000000002/e395bed6-5bba-41b8-a256-3505e4afcca2 v319>,<0x10000000001/sv_0 v333>,<0x10000000000/qa v317>,<0x1/volumes v304>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-04-24T08:55:26.011531+0000 12=12+0)","ondisk_value.rstat":"n(v0 rc2024-04-24T08:55:26.050891+0000 b2861056 12=11+1)","memory_value.dirstat":"f(v104 m2024-04-24T08:55:26.050891+0000 11=11+0)","memory_value.rstat":"n(v4 rc2024-04-24T08:55:26.050891+0000 b2861056 12=11+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-04-24T08:55:43.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:55:43 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T08:55:43.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:55:43 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T08:55:43.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:55:43 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T08:55:43.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:55:43 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T08:55:43.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:55:43 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T08:55:43.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:55:43 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T08:55:43.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:55:43 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T08:55:43.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:55:43 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T08:55:43.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:55:43 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T08:55:43.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:55:43 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T08:55:43.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:55:43 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T08:55:43.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:55:43 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T08:55:43.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:55:43 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T08:55:43.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:55:43 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T08:55:43.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:55:43 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T08:55:43.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:55:43 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T08:55:43.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:55:43 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T08:55:43.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:55:43 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T08:55:43.569 INFO:tasks.workunit.client.0.smithi083.stdout: 1 17123 0.74 MB/sec execute 17 sec latency 67.670 ms 2024-04-24T08:55:44.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:55:44 smithi083 ceph-mon[27907]: pgmap v126: 129 pgs: 129 active+clean; 33 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 433 KiB/s wr, 72 op/s 2024-04-24T08:55:44.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:55:44 smithi083 ceph-mon[27907]: scrub queued for path: / 2024-04-24T08:55:44.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:55:44 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T08:55:44.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:55:44 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T08:55:44.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:55:44 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T08:55:44.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:55:44 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T08:55:44.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:55:44 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T08:55:44.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:55:44 smithi083 ceph-mon[27907]: Scrub error on inode 0x10000000215 (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/COREL) see mds.b log and `damage ls` output for details 2024-04-24T08:55:44.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:55:44 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T08:55:44.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:55:44 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T08:55:44.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:55:44 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T08:55:44.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:55:44 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T08:55:44.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:55:44 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T08:55:44.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:55:44 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T08:55:44.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:55:44 smithi138 ceph-mon[31441]: pgmap v126: 129 pgs: 129 active+clean; 33 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 433 KiB/s wr, 72 op/s 2024-04-24T08:55:44.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:55:44 smithi138 ceph-mon[31441]: scrub queued for path: / 2024-04-24T08:55:44.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:55:44 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T08:55:44.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:55:44 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T08:55:44.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:55:44 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T08:55:44.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:55:44 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T08:55:44.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:55:44 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T08:55:44.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:55:44 smithi138 ceph-mon[31441]: Scrub error on inode 0x10000000215 (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/COREL) see mds.b log and `damage ls` output for details 2024-04-24T08:55:44.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:55:44 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T08:55:44.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:55:44 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T08:55:44.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:55:44 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T08:55:44.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:55:44 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T08:55:44.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:55:44 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T08:55:44.457 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:55:44 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T08:55:44.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:55:44 smithi189 ceph-mon[31791]: pgmap v126: 129 pgs: 129 active+clean; 33 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 433 KiB/s wr, 72 op/s 2024-04-24T08:55:44.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:55:44 smithi189 ceph-mon[31791]: scrub queued for path: / 2024-04-24T08:55:44.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:55:44 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T08:55:44.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:55:44 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T08:55:44.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:55:44 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T08:55:44.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:55:44 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T08:55:44.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:55:44 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T08:55:44.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:55:44 smithi189 ceph-mon[31791]: Scrub error on inode 0x10000000215 (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/COREL) see mds.b log and `damage ls` output for details 2024-04-24T08:55:44.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:55:44 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T08:55:44.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:55:44 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T08:55:44.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:55:44 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T08:55:44.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:55:44 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T08:55:44.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:55:44 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T08:55:44.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:55:44 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T08:55:44.569 INFO:tasks.workunit.client.0.smithi083.stdout: 1 17223 0.70 MB/sec execute 18 sec latency 49.614 ms 2024-04-24T08:55:45.569 INFO:tasks.workunit.client.0.smithi083.stdout: 1 17478 0.76 MB/sec execute 19 sec latency 52.575 ms 2024-04-24T08:55:46.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:55:46 smithi083 ceph-mon[27907]: pgmap v127: 129 pgs: 129 active+clean; 33 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 259 KiB/s wr, 44 op/s 2024-04-24T08:55:46.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:55:46 smithi138 ceph-mon[31441]: pgmap v127: 129 pgs: 129 active+clean; 33 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 259 KiB/s wr, 44 op/s 2024-04-24T08:55:46.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:55:46 smithi189 ceph-mon[31791]: pgmap v127: 129 pgs: 129 active+clean; 33 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 259 KiB/s wr, 44 op/s 2024-04-24T08:55:46.569 INFO:tasks.workunit.client.0.smithi083.stdout: 1 17508 0.72 MB/sec execute 20 sec latency 57.241 ms 2024-04-24T08:55:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.025+0000 7f77a2a00640 1 -- 172.21.15.83:0/360654489 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f779c1038f0 msgr2=0x7f779c10bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.025+0000 7f77a2a00640 1 --2- 172.21.15.83:0/360654489 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f779c1038f0 0x7f779c10bdd0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f7788009900 tx=0x7f778802a3f0 comp rx=0 tx=0).stop 2024-04-24T08:55:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.026+0000 7f77a2a00640 1 -- 172.21.15.83:0/360654489 shutdown_connections 2024-04-24T08:55:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.026+0000 7f77a2a00640 1 --2- 172.21.15.83:0/360654489 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f779c10c440 0x7f779c10e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.026+0000 7f77a2a00640 1 --2- 172.21.15.83:0/360654489 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f779c1038f0 0x7f779c10bdd0 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.026+0000 7f77a2a00640 1 --2- 172.21.15.83:0/360654489 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f779c102fd0 0x7f779c1033b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.026+0000 7f77a2a00640 1 -- 172.21.15.83:0/360654489 >> 172.21.15.83:0/360654489 conn(0x7f779c0fc810 msgr2=0x7f779c0fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.026+0000 7f77a2a00640 1 -- 172.21.15.83:0/360654489 shutdown_connections 2024-04-24T08:55:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.026+0000 7f77a2a00640 1 -- 172.21.15.83:0/360654489 wait complete. 2024-04-24T08:55:47.029 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.027+0000 7f77a2a00640 1 Processor -- start 2024-04-24T08:55:47.029 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.027+0000 7f77a2a00640 1 -- start start 2024-04-24T08:55:47.029 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.027+0000 7f77a2a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f779c102fd0 0x7f779c100c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:47.029 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.027+0000 7f77a2a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f779c1038f0 0x7f779c101160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:47.029 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.027+0000 7f77a2a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f779c10c440 0x7f779c1016a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:47.029 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.027+0000 7f77a2a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f779c1120e0 con 0x7f779c10c440 2024-04-24T08:55:47.030 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.027+0000 7f77a2a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f779c111f60 con 0x7f779c1038f0 2024-04-24T08:55:47.030 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.027+0000 7f77a2a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f779c112260 con 0x7f779c102fd0 2024-04-24T08:55:47.030 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.028+0000 7f77a2000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f779c10c440 0x7f779c1016a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:47.030 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.028+0000 7f77a2000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f779c10c440 0x7f779c1016a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55768/0 (socket says 172.21.15.83:55768) 2024-04-24T08:55:47.030 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.028+0000 7f77a2000640 1 -- 172.21.15.83:0/3353257423 learned_addr learned my addr 172.21.15.83:0/3353257423 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:55:47.030 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.028+0000 7f77a1600640 1 --2- 172.21.15.83:0/3353257423 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f779c102fd0 0x7f779c100c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:47.030 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.028+0000 7f77a0c00640 1 --2- 172.21.15.83:0/3353257423 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f779c1038f0 0x7f779c101160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:47.030 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.028+0000 7f77a2000640 1 -- 172.21.15.83:0/3353257423 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f779c102fd0 msgr2=0x7f779c100c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:47.030 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.028+0000 7f77a2000640 1 --2- 172.21.15.83:0/3353257423 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f779c102fd0 0x7f779c100c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:47.030 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.028+0000 7f77a2000640 1 -- 172.21.15.83:0/3353257423 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f779c1038f0 msgr2=0x7f779c101160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:47.030 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.028+0000 7f77a2000640 1 --2- 172.21.15.83:0/3353257423 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f779c1038f0 0x7f779c101160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:47.030 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.028+0000 7f77a2000640 1 -- 172.21.15.83:0/3353257423 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f779c1a6b60 con 0x7f779c10c440 2024-04-24T08:55:47.030 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.029+0000 7f77a2000640 1 --2- 172.21.15.83:0/3353257423 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f779c10c440 0x7f779c1016a0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f779400cad0 tx=0x7f779400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:47.030 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.029+0000 7f779a000640 1 -- 172.21.15.83:0/3353257423 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f779400ae20 con 0x7f779c10c440 2024-04-24T08:55:47.031 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.029+0000 7f779a000640 1 -- 172.21.15.83:0/3353257423 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7794005020 con 0x7f779c10c440 2024-04-24T08:55:47.031 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.029+0000 7f779a000640 1 -- 172.21.15.83:0/3353257423 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f779400f6b0 con 0x7f779c10c440 2024-04-24T08:55:47.031 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.029+0000 7f77a2a00640 1 -- 172.21.15.83:0/3353257423 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f779c1a6e50 con 0x7f779c10c440 2024-04-24T08:55:47.031 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.029+0000 7f77a2a00640 1 -- 172.21.15.83:0/3353257423 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f779c1a7320 con 0x7f779c10c440 2024-04-24T08:55:47.033 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.033+0000 7f77a6e42740 1 Processor -- start 2024-04-24T08:55:47.033 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.033+0000 7f779a000640 1 -- 172.21.15.83:0/3353257423 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f77940051d0 con 0x7f779c10c440 2024-04-24T08:55:47.033 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.033+0000 7f77a6e42740 1 -- start start 2024-04-24T08:55:47.035 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.033+0000 7f77a6e42740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561a832c3c0 0x5561a832c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:47.035 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.033+0000 7f77a6e42740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5561a832cce0 0x5561a8337060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:47.035 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.033+0000 7f77a6e42740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5561a83375a0 0x5561a8339990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:47.035 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.033+0000 7f77a6e42740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5561a8292900 con 0x5561a83375a0 2024-04-24T08:55:47.035 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.033+0000 7f77a6e42740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5561a8238f20 con 0x5561a832cce0 2024-04-24T08:55:47.035 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.033+0000 7f77a6e42740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5561a828e8c0 con 0x5561a832c3c0 2024-04-24T08:55:47.035 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.034+0000 7f77a0c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5561a83375a0 0x5561a8339990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:47.035 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.034+0000 7f77a0c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5561a83375a0 0x5561a8339990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55772/0 (socket says 172.21.15.83:55772) 2024-04-24T08:55:47.035 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.034+0000 7f77a0c00640 1 -- 172.21.15.83:0/4189591397 learned_addr learned my addr 172.21.15.83:0/4189591397 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:55:47.035 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.034+0000 7f77a0c00640 1 --2- 172.21.15.83:0/4189591397 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561a832c3c0 0x5561a832c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:47.036 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.034+0000 7f77a1600640 1 --2- 172.21.15.83:0/4189591397 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5561a832cce0 0x5561a8337060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:47.036 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.034+0000 7f77a0c00640 1 -- 172.21.15.83:0/4189591397 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561a832c3c0 msgr2=0x5561a832c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:47.036 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.034+0000 7f77a0c00640 1 --2- 172.21.15.83:0/4189591397 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561a832c3c0 0x5561a832c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:47.036 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.034+0000 7f77a0c00640 1 -- 172.21.15.83:0/4189591397 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5561a832cce0 msgr2=0x5561a8337060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:47.036 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.034+0000 7f77a0c00640 1 --2- 172.21.15.83:0/4189591397 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5561a832cce0 0x5561a8337060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:47.036 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.034+0000 7f77a0c00640 1 -- 172.21.15.83:0/4189591397 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561a825de10 con 0x5561a83375a0 2024-04-24T08:55:47.036 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.035+0000 7f779a000640 1 --2- 172.21.15.83:0/3353257423 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f77840a3d10 0x7f77840a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:47.036 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.035+0000 7f77a0c00640 1 --2- 172.21.15.83:0/4189591397 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5561a83375a0 0x5561a8339990 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f7788009900 tx=0x7f7788000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e59b1142b8abc260 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:47.036 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.035+0000 7f779a000640 1 -- 172.21.15.83:0/3353257423 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f77940d0cb0 con 0x7f779c10c440 2024-04-24T08:55:47.036 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.035+0000 7f77a1600640 1 --2- 172.21.15.83:0/3353257423 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f77840a3d10 0x7f77840a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:47.039 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.035+0000 7f778e600640 1 -- 172.21.15.83:0/4189591397 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7788035040 con 0x5561a83375a0 2024-04-24T08:55:47.039 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.035+0000 7f778e600640 1 -- 172.21.15.83:0/4189591397 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f77880054e0 con 0x5561a83375a0 2024-04-24T08:55:47.039 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.036+0000 7f778e600640 1 -- 172.21.15.83:0/4189591397 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77880037f0 con 0x5561a83375a0 2024-04-24T08:55:47.040 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.036+0000 7f77a6e42740 1 -- 172.21.15.83:0/4189591397 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5561a83375a0 msgr2=0x5561a8339990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:47.040 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.036+0000 7f77a6e42740 1 --2- 172.21.15.83:0/4189591397 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5561a83375a0 0x5561a8339990 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f7788009900 tx=0x7f7788000bc0 comp rx=0 tx=0).stop 2024-04-24T08:55:47.040 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.036+0000 7f77a1600640 1 --2- 172.21.15.83:0/3353257423 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f77840a3d10 0x7f77840a61d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7790005fd0 tx=0x7f779000b650 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:47.040 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.036+0000 7f77a6e42740 1 -- 172.21.15.83:0/4189591397 shutdown_connections 2024-04-24T08:55:47.040 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.036+0000 7f77a6e42740 1 --2- 172.21.15.83:0/4189591397 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5561a83375a0 0x5561a8339990 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:47.040 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.036+0000 7f77a6e42740 1 --2- 172.21.15.83:0/4189591397 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5561a832cce0 0x5561a8337060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:47.040 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.036+0000 7f77a6e42740 1 --2- 172.21.15.83:0/4189591397 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561a832c3c0 0x5561a832c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:47.040 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.036+0000 7f77a6e42740 1 -- 172.21.15.83:0/4189591397 >> 172.21.15.83:0/4189591397 conn(0x5561a83288d0 msgr2=0x5561a8328cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:47.040 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.036+0000 7f77a6e42740 1 -- 172.21.15.83:0/4189591397 shutdown_connections 2024-04-24T08:55:47.040 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.037+0000 7f77a6e42740 1 -- 172.21.15.83:0/4189591397 wait complete. 2024-04-24T08:55:47.040 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.037+0000 7f77a6e42740 1 -- start start 2024-04-24T08:55:47.040 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.037+0000 7f77a6e42740 1 Processor -- start 2024-04-24T08:55:47.040 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.038+0000 7f77a6e42740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561a832c3c0 0x5561a83345c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:47.041 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.038+0000 7f77a6e42740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5561a832cce0 0x5561a8334b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:47.041 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.038+0000 7f77a6e42740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5561a83375a0 0x5561a8335040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:47.041 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.038+0000 7f77a6e42740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5561a833a080 con 0x5561a832cce0 2024-04-24T08:55:47.041 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.038+0000 7f77a6e42740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5561a824c2c0 con 0x5561a83375a0 2024-04-24T08:55:47.041 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.038+0000 7f77a6e42740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5561a833a200 con 0x5561a832c3c0 2024-04-24T08:55:47.041 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.038+0000 7f77a6e42740 10 client.0 _ll_register_callbacks cb 0x5561a7fa4a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:55:47.041 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.038+0000 7f77a0c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5561a832cce0 0x5561a8334b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:47.041 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.038+0000 7f77a0c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5561a832cce0 0x5561a8334b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55788/0 (socket says 172.21.15.83:55788) 2024-04-24T08:55:47.041 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.038+0000 7f77a0c00640 1 -- 172.21.15.83:0/705363550 learned_addr learned my addr 172.21.15.83:0/705363550 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:55:47.041 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.039+0000 7f77a2000640 1 --2- 172.21.15.83:0/705363550 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5561a83375a0 0x5561a8335040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:47.041 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.039+0000 7f77a1600640 1 --2- 172.21.15.83:0/705363550 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561a832c3c0 0x5561a83345c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:47.041 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.039+0000 7f77a0c00640 1 -- 172.21.15.83:0/705363550 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561a832c3c0 msgr2=0x5561a83345c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:47.041 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.039+0000 7f77a0c00640 1 --2- 172.21.15.83:0/705363550 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561a832c3c0 0x5561a83345c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:47.042 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.039+0000 7f77a0c00640 1 -- 172.21.15.83:0/705363550 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5561a83375a0 msgr2=0x5561a8335040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:47.042 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.039+0000 7f77a0c00640 1 --2- 172.21.15.83:0/705363550 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5561a83375a0 0x5561a8335040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:47.042 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.039+0000 7f77a0c00640 1 -- 172.21.15.83:0/705363550 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561a825cfd0 con 0x5561a832cce0 2024-04-24T08:55:47.042 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.039+0000 7f77a1600640 1 --2- 172.21.15.83:0/705363550 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561a832c3c0 0x5561a83345c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:55:47.042 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.039+0000 7f77a0c00640 1 --2- 172.21.15.83:0/705363550 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5561a832cce0 0x5561a8334b00 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f778802a910 tx=0x7f7788031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7d307ba7ff764bf server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:47.042 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.039+0000 7f7782a00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:55:47.044 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.041+0000 7f7782a00640 1 -- 172.21.15.83:0/705363550 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f778804c020 con 0x5561a832cce0 2024-04-24T08:55:47.044 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.041+0000 7f7782a00640 1 -- 172.21.15.83:0/705363550 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f77880048d0 con 0x5561a832cce0 2024-04-24T08:55:47.044 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.041+0000 7f77a6e42740 1 -- 172.21.15.83:0/705363550 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77880054e0 con 0x5561a832cce0 2024-04-24T08:55:47.044 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.041+0000 7f7782a00640 1 -- 172.21.15.83:0/705363550 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7788057050 con 0x5561a832cce0 2024-04-24T08:55:47.044 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.041+0000 7f77a6e42740 10 client.17211 fetch_fsmap learned FSMap version 30 2024-04-24T08:55:47.044 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.041+0000 7f77a6e42740 1 -- 172.21.15.83:0/705363550 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5561a7fa5210 con 0x5561a832cce0 2024-04-24T08:55:47.044 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.041+0000 7f7782a00640 1 -- 172.21.15.83:0/705363550 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f77880054e0 con 0x5561a832cce0 2024-04-24T08:55:47.044 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.042+0000 7f7782a00640 1 -- 172.21.15.83:0/705363550 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7f778803a040 con 0x5561a832cce0 2024-04-24T08:55:47.044 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.042+0000 7f77a6e42740 10 client.17211 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T08:55:47.044 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.042+0000 7f77a6e42740 10 client.17211 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T08:55:47.044 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.042+0000 7f77a6e42740 20 client.17211 populate_metadata read hostname 'smithi083' 2024-04-24T08:55:47.044 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.042+0000 7f77a6e42740 1 --2- 172.21.15.83:0/705363550 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5561a834fd80 0x5561a8352170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:47.044 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.042+0000 7f77a6e42740 4 client.17211 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:55:47.045 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.042+0000 7f77a6e42740 1 -- 172.21.15.83:0/705363550 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5561a82a0320 con 0x5561a834fd80 2024-04-24T08:55:47.045 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.042+0000 7f77a1600640 1 --2- 172.21.15.83:0/705363550 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5561a834fd80 0x5561a8352170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:47.045 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.043+0000 7f77a1600640 1 --2- 172.21.15.83:0/705363550 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5561a834fd80 0x5561a8352170 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ace12607af8d9330 server_cookie=d5d1b3c89da0fd45 in_seq=0 out_seq=0 2024-04-24T08:55:47.045 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.043+0000 7f7782a00640 10 client.17211 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:55:47.048 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.047+0000 7f7782a00640 1 -- 172.21.15.83:0/705363550 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5561a82a0320 con 0x5561a834fd80 2024-04-24T08:55:47.048 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.047+0000 7f7782a00640 10 client.17211 handle_command_reply: tid=0 2024-04-24T08:55:47.048 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.048+0000 7f77a6e42740 1 client.17211 shutdown 2024-04-24T08:55:47.048 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.048+0000 7f77a6e42740 1 -- 172.21.15.83:0/705363550 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5561a832cce0 msgr2=0x5561a8334b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:47.048 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.048+0000 7f77a6e42740 1 --2- 172.21.15.83:0/705363550 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5561a832cce0 0x5561a8334b00 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f778802a910 tx=0x7f7788031400 comp rx=0 tx=0).stop 2024-04-24T08:55:47.048 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.048+0000 7f77a6e42740 1 -- 172.21.15.83:0/705363550 shutdown_connections 2024-04-24T08:55:47.049 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.048+0000 7f77a6e42740 1 --2- 172.21.15.83:0/705363550 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5561a834fd80 0x5561a8352170 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:47.049 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.048+0000 7f77a6e42740 1 --2- 172.21.15.83:0/705363550 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5561a83375a0 0x5561a8335040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:47.049 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.048+0000 7f77a6e42740 1 --2- 172.21.15.83:0/705363550 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5561a832cce0 0x5561a8334b00 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:47.049 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.048+0000 7f77a6e42740 1 --2- 172.21.15.83:0/705363550 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561a832c3c0 0x5561a83345c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:47.049 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.048+0000 7f7782a00640 0 client.17211 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:55:47.049 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.048+0000 7f77a6e42740 1 -- 172.21.15.83:0/705363550 >> 172.21.15.83:0/705363550 conn(0x5561a83288d0 msgr2=0x5561a832e600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:47.049 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.048+0000 7f77a6e42740 1 -- 172.21.15.83:0/705363550 shutdown_connections 2024-04-24T08:55:47.049 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.048+0000 7f77a6e42740 1 -- 172.21.15.83:0/705363550 wait complete. 2024-04-24T08:55:47.049 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.048+0000 7f77a6e42740 20 client.17211 trim_cache size 0 max 16384 2024-04-24T08:55:47.067 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.066+0000 7f77a6e42740 1 Processor -- start 2024-04-24T08:55:47.067 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.066+0000 7f77a6e42740 1 -- start start 2024-04-24T08:55:47.067 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.066+0000 7f77a6e42740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5561a832c3c0 0x5561a834b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:47.067 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.066+0000 7f77a6e42740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5561a832cce0 0x5561a8334e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:47.068 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.066+0000 7f77a6e42740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561a83375a0 0x5561a832ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:47.068 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.066+0000 7f77a6e42740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f779c1ad150 con 0x5561a832c3c0 2024-04-24T08:55:47.068 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.066+0000 7f77a6e42740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f779c1aed20 con 0x5561a832cce0 2024-04-24T08:55:47.068 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.066+0000 7f77a6e42740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f779c1ad820 con 0x5561a83375a0 2024-04-24T08:55:47.068 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.066+0000 7f77a1600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5561a832c3c0 0x5561a834b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:47.068 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.067+0000 7f77a1600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5561a832c3c0 0x5561a834b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55790/0 (socket says 172.21.15.83:55790) 2024-04-24T08:55:47.068 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.067+0000 7f77a1600640 1 -- 172.21.15.83:0/3665275935 learned_addr learned my addr 172.21.15.83:0/3665275935 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:55:47.068 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.067+0000 7f77a1600640 1 -- 172.21.15.83:0/3665275935 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561a83375a0 msgr2=0x5561a832ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:47.068 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.067+0000 7f77a0c00640 1 --2- 172.21.15.83:0/3665275935 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5561a832cce0 0x5561a8334e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:47.068 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.067+0000 7f77a2000640 1 --2- 172.21.15.83:0/3665275935 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561a83375a0 0x5561a832ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:47.068 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.067+0000 7f77a1600640 1 --2- 172.21.15.83:0/3665275935 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561a83375a0 0x5561a832ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:47.068 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.067+0000 7f77a1600640 1 -- 172.21.15.83:0/3665275935 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5561a832cce0 msgr2=0x5561a8334e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:47.068 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.067+0000 7f77a1600640 1 --2- 172.21.15.83:0/3665275935 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5561a832cce0 0x5561a8334e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:47.069 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.067+0000 7f77a1600640 1 -- 172.21.15.83:0/3665275935 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561a8071620 con 0x5561a832c3c0 2024-04-24T08:55:47.069 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.067+0000 7f77a2000640 1 --2- 172.21.15.83:0/3665275935 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561a83375a0 0x5561a832ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:55:47.069 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.067+0000 7f77a0c00640 1 --2- 172.21.15.83:0/3665275935 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5561a832cce0 0x5561a8334e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:55:47.069 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.067+0000 7f77a1600640 1 --2- 172.21.15.83:0/3665275935 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5561a832c3c0 0x5561a834b8c0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x5561a8352710 tx=0x7f7790014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d66b06533f2ffb11 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:47.072 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.068+0000 7f778dc00640 1 -- 172.21.15.83:0/3665275935 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f779002d070 con 0x5561a832c3c0 2024-04-24T08:55:47.072 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.068+0000 7f778dc00640 1 -- 172.21.15.83:0/3665275935 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7790027700 con 0x5561a832c3c0 2024-04-24T08:55:47.072 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.068+0000 7f778dc00640 1 -- 172.21.15.83:0/3665275935 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77900209f0 con 0x5561a832c3c0 2024-04-24T08:55:47.072 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.068+0000 7f77a6e42740 1 -- 172.21.15.83:0/3665275935 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5561a832c3c0 msgr2=0x5561a834b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:47.072 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.068+0000 7f77a6e42740 1 --2- 172.21.15.83:0/3665275935 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5561a832c3c0 0x5561a834b8c0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x5561a8352710 tx=0x7f7790014000 comp rx=0 tx=0).stop 2024-04-24T08:55:47.072 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.068+0000 7f77a6e42740 1 -- 172.21.15.83:0/3665275935 shutdown_connections 2024-04-24T08:55:47.073 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.068+0000 7f77a6e42740 1 --2- 172.21.15.83:0/3665275935 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561a83375a0 0x5561a832ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:47.073 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.068+0000 7f77a6e42740 1 --2- 172.21.15.83:0/3665275935 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5561a832cce0 0x5561a8334e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:47.073 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.068+0000 7f77a6e42740 1 --2- 172.21.15.83:0/3665275935 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5561a832c3c0 0x5561a834b8c0 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:47.073 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.068+0000 7f77a6e42740 1 -- 172.21.15.83:0/3665275935 >> 172.21.15.83:0/3665275935 conn(0x5561a83288d0 msgr2=0x5561a8335ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:47.073 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.069+0000 7f77a6e42740 1 -- 172.21.15.83:0/3665275935 shutdown_connections 2024-04-24T08:55:47.073 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.069+0000 7f77a6e42740 1 -- 172.21.15.83:0/3665275935 wait complete. 2024-04-24T08:55:47.073 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.069+0000 7f77a6e42740 1 -- start start 2024-04-24T08:55:47.073 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.069+0000 7f77a6e42740 1 Processor -- start 2024-04-24T08:55:47.073 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.070+0000 7f77a6e42740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561a832c3c0 0x5561a8331650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:47.073 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.070+0000 7f77a6e42740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5561a832cce0 0x5561a8331b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:47.073 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.070+0000 7f77a6e42740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5561a83375a0 0x5561a83b5010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:47.073 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.070+0000 7f77a6e42740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f779c1ae5c0 con 0x5561a832cce0 2024-04-24T08:55:47.073 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.070+0000 7f77a6e42740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f779c1adef0 con 0x5561a83375a0 2024-04-24T08:55:47.074 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.070+0000 7f77a6e42740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f779c1ab150 con 0x5561a832c3c0 2024-04-24T08:55:47.074 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.070+0000 7f77a6e42740 10 client.0 _ll_register_callbacks cb 0x7f7788004380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:55:47.074 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.070+0000 7f77a0c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5561a832cce0 0x5561a8331b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:47.074 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.070+0000 7f77a0c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5561a832cce0 0x5561a8331b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55796/0 (socket says 172.21.15.83:55796) 2024-04-24T08:55:47.074 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.070+0000 7f77a0c00640 1 -- 172.21.15.83:0/1620349619 learned_addr learned my addr 172.21.15.83:0/1620349619 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:55:47.074 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.070+0000 7f77a1600640 1 --2- 172.21.15.83:0/1620349619 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561a832c3c0 0x5561a8331650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:47.074 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.070+0000 7f77a2000640 1 --2- 172.21.15.83:0/1620349619 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5561a83375a0 0x5561a83b5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:47.074 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.070+0000 7f77a0c00640 1 -- 172.21.15.83:0/1620349619 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561a832c3c0 msgr2=0x5561a8331650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:47.074 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.070+0000 7f77a0c00640 1 --2- 172.21.15.83:0/1620349619 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561a832c3c0 0x5561a8331650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:47.074 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.070+0000 7f77a0c00640 1 -- 172.21.15.83:0/1620349619 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5561a83375a0 msgr2=0x5561a83b5010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:55:47.074 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.070+0000 7f77a0c00640 1 --2- 172.21.15.83:0/1620349619 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5561a83375a0 0x5561a83b5010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:47.074 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.070+0000 7f77a0c00640 1 -- 172.21.15.83:0/1620349619 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561a8332460 con 0x5561a832cce0 2024-04-24T08:55:47.074 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.070+0000 7f77a2000640 1 --2- 172.21.15.83:0/1620349619 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5561a83375a0 0x5561a83b5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:55:47.075 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.071+0000 7f77a0c00640 1 --2- 172.21.15.83:0/1620349619 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5561a832cce0 0x5561a8331b90 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f7788005ec0 tx=0x7f778802aae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3aac6c42313e1235 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:55:47.075 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.071+0000 7f7782a00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:55:47.075 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.071+0000 7f7782a00640 1 -- 172.21.15.83:0/1620349619 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f778804c070 con 0x5561a832cce0 2024-04-24T08:55:47.075 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.071+0000 7f7782a00640 1 -- 172.21.15.83:0/1620349619 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f7788031a60 con 0x5561a832cce0 2024-04-24T08:55:47.075 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.071+0000 7f77a6e42740 1 -- 172.21.15.83:0/1620349619 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7790027700 con 0x5561a832cce0 2024-04-24T08:55:47.075 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.071+0000 7f7782a00640 1 -- 172.21.15.83:0/1620349619 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7788035030 con 0x5561a832cce0 2024-04-24T08:55:47.075 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.071+0000 7f77a6e42740 10 client.17217 fetch_fsmap learned FSMap version 30 2024-04-24T08:55:47.075 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.071+0000 7f77a6e42740 1 -- 172.21.15.83:0/1620349619 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5561a83b65d0 con 0x5561a832cce0 2024-04-24T08:55:47.075 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.072+0000 7f7782a00640 1 -- 172.21.15.83:0/1620349619 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7790027700 con 0x5561a832cce0 2024-04-24T08:55:47.075 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.072+0000 7f7782a00640 1 -- 172.21.15.83:0/1620349619 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7f77880035f0 con 0x5561a832cce0 2024-04-24T08:55:47.075 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.072+0000 7f77a6e42740 10 client.17217 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T08:55:47.075 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.072+0000 7f77a6e42740 10 client.17217 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T08:55:47.075 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.072+0000 7f77a6e42740 20 client.17217 populate_metadata read hostname 'smithi083' 2024-04-24T08:55:47.076 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.072+0000 7f77a6e42740 1 --2- 172.21.15.83:0/1620349619 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5561a83b6ba0 0x5561a83b8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:55:47.076 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.072+0000 7f77a6e42740 4 client.17217 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T08:55:47.076 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.072+0000 7f77a6e42740 1 -- 172.21.15.83:0/1620349619 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5561a829a190 con 0x5561a83b6ba0 2024-04-24T08:55:47.076 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.073+0000 7f77a1600640 1 --2- 172.21.15.83:0/1620349619 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5561a83b6ba0 0x5561a83b8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:55:47.076 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.074+0000 7f77a1600640 1 --2- 172.21.15.83:0/1620349619 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5561a83b6ba0 0x5561a83b8fc0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4434a562a7dc4e2 server_cookie=5d061495804bd9bb in_seq=0 out_seq=0 2024-04-24T08:55:47.076 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.074+0000 7f7782a00640 10 client.17217 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:55:47.135 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.134+0000 7f7782a00640 1 -- 172.21.15.83:0/1620349619 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5561a829a190 con 0x5561a83b6ba0 2024-04-24T08:55:47.135 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.134+0000 7f7782a00640 10 client.17217 handle_command_reply: tid=0 2024-04-24T08:55:47.135 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.135+0000 7f77a6e42740 1 client.17217 shutdown 2024-04-24T08:55:47.136 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.135+0000 7f77a6e42740 1 -- 172.21.15.83:0/1620349619 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5561a832cce0 msgr2=0x5561a8331b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:47.136 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.135+0000 7f77a6e42740 1 --2- 172.21.15.83:0/1620349619 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5561a832cce0 0x5561a8331b90 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f7788005ec0 tx=0x7f778802aae0 comp rx=0 tx=0).stop 2024-04-24T08:55:47.136 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.135+0000 7f77a6e42740 1 -- 172.21.15.83:0/1620349619 shutdown_connections 2024-04-24T08:55:47.136 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.135+0000 7f77a6e42740 1 --2- 172.21.15.83:0/1620349619 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5561a83b6ba0 0x5561a83b8fc0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:47.136 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.135+0000 7f77a6e42740 1 --2- 172.21.15.83:0/1620349619 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5561a83375a0 0x5561a83b5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:47.136 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.135+0000 7f77a6e42740 1 --2- 172.21.15.83:0/1620349619 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5561a832cce0 0x5561a8331b90 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:47.136 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.135+0000 7f77a6e42740 1 --2- 172.21.15.83:0/1620349619 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561a832c3c0 0x5561a8331650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:47.136 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.135+0000 7f77a6e42740 1 -- 172.21.15.83:0/1620349619 >> 172.21.15.83:0/1620349619 conn(0x5561a83288d0 msgr2=0x5561a8334bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:47.136 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.135+0000 7f7782a00640 0 client.17217 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:55:47.136 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.136+0000 7f77a6e42740 1 -- 172.21.15.83:0/1620349619 shutdown_connections 2024-04-24T08:55:47.136 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.136+0000 7f77a6e42740 1 -- 172.21.15.83:0/1620349619 wait complete. 2024-04-24T08:55:47.136 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.136+0000 7f77a6e42740 20 client.17217 trim_cache size 0 max 16384 2024-04-24T08:55:47.137 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T08:55:47.137 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T08:55:47.137 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T08:55:47.137 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T08:55:47.138 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.137+0000 7f77a2a00640 1 -- 172.21.15.83:0/3353257423 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f77840a3d10 msgr2=0x7f77840a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:47.138 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.137+0000 7f77a2a00640 1 --2- 172.21.15.83:0/3353257423 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f77840a3d10 0x7f77840a61d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7790005fd0 tx=0x7f779000b650 comp rx=0 tx=0).stop 2024-04-24T08:55:47.138 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.137+0000 7f77a2a00640 1 -- 172.21.15.83:0/3353257423 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f779c10c440 msgr2=0x7f779c1016a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:55:47.138 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.137+0000 7f77a2a00640 1 --2- 172.21.15.83:0/3353257423 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f779c10c440 0x7f779c1016a0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f779400cad0 tx=0x7f779400cf90 comp rx=0 tx=0).stop 2024-04-24T08:55:47.138 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.137+0000 7f77a2a00640 1 -- 172.21.15.83:0/3353257423 shutdown_connections 2024-04-24T08:55:47.138 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.137+0000 7f77a2a00640 1 --2- 172.21.15.83:0/3353257423 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f77840a3d10 0x7f77840a61d0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:47.138 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.137+0000 7f77a2a00640 1 --2- 172.21.15.83:0/3353257423 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f779c10c440 0x7f779c1016a0 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:47.138 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.137+0000 7f77a2a00640 1 --2- 172.21.15.83:0/3353257423 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f779c1038f0 0x7f779c101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:47.138 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.137+0000 7f77a2a00640 1 --2- 172.21.15.83:0/3353257423 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f779c102fd0 0x7f779c100c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:55:47.138 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.137+0000 7f77a2a00640 1 -- 172.21.15.83:0/3353257423 >> 172.21.15.83:0/3353257423 conn(0x7f779c0fc810 msgr2=0x7f779c0fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:55:47.139 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.137+0000 7f77a2a00640 1 -- 172.21.15.83:0/3353257423 shutdown_connections 2024-04-24T08:55:47.139 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:55:47.137+0000 7f77a2a00640 1 -- 172.21.15.83:0/3353257423 wait complete. 2024-04-24T08:55:47.570 INFO:tasks.workunit.client.0.smithi083.stdout: 1 17544 0.69 MB/sec execute 21 sec latency 55.464 ms 2024-04-24T08:55:48.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:55:48 smithi083 ceph-mon[27907]: pgmap v128: 129 pgs: 129 active+clean; 33 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 259 KiB/s wr, 44 op/s 2024-04-24T08:55:48.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:55:48 smithi138 ceph-mon[31441]: pgmap v128: 129 pgs: 129 active+clean; 33 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 259 KiB/s wr, 44 op/s 2024-04-24T08:55:48.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:55:48 smithi189 ceph-mon[31791]: pgmap v128: 129 pgs: 129 active+clean; 33 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 259 KiB/s wr, 44 op/s 2024-04-24T08:55:48.570 INFO:tasks.workunit.client.0.smithi083.stdout: 1 17725 0.79 MB/sec execute 22 sec latency 48.067 ms 2024-04-24T08:55:49.570 INFO:tasks.workunit.client.0.smithi083.stdout: 1 17852 0.77 MB/sec execute 23 sec latency 47.391 ms 2024-04-24T08:55:50.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:55:50 smithi083 ceph-mon[27907]: pgmap v129: 129 pgs: 129 active+clean; 35 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 431 KiB/s wr, 73 op/s 2024-04-24T08:55:50.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:55:50 smithi138 ceph-mon[31441]: pgmap v129: 129 pgs: 129 active+clean; 35 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 431 KiB/s wr, 73 op/s 2024-04-24T08:55:50.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:55:50 smithi189 ceph-mon[31791]: pgmap v129: 129 pgs: 129 active+clean; 35 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 431 KiB/s wr, 73 op/s 2024-04-24T08:55:50.570 INFO:tasks.workunit.client.0.smithi083.stdout: 1 17963 0.78 MB/sec execute 24 sec latency 49.742 ms 2024-04-24T08:55:51.570 INFO:tasks.workunit.client.0.smithi083.stdout: 1 18029 0.75 MB/sec execute 25 sec latency 51.926 ms 2024-04-24T08:55:52.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:55:52 smithi138 ceph-mon[31441]: pgmap v130: 129 pgs: 129 active+clean; 35 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 374 KiB/s wr, 49 op/s 2024-04-24T08:55:52.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:55:52 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:55:52.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:55:52 smithi189 ceph-mon[31791]: pgmap v130: 129 pgs: 129 active+clean; 35 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 374 KiB/s wr, 49 op/s 2024-04-24T08:55:52.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:55:52 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:55:52.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:55:52 smithi083 ceph-mon[27907]: pgmap v130: 129 pgs: 129 active+clean; 35 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 374 KiB/s wr, 49 op/s 2024-04-24T08:55:52.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:55:52 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:55:52.570 INFO:tasks.workunit.client.0.smithi083.stdout: 1 18115 0.73 MB/sec execute 26 sec latency 42.207 ms 2024-04-24T08:55:53.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:55:53 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T08:55:53.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:55:53 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T08:55:53.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:55:53 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T08:55:53.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:55:53 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T08:55:53.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:55:53 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:55:53.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:55:53 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:55:53.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:55:53 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T08:55:53.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:55:53 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T08:55:53.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:55:53 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T08:55:53.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:55:53 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T08:55:53.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:55:53 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T08:55:53.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:55:53 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:55:53.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:55:53 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:55:53.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:55:53 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T08:55:53.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:55:53 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T08:55:53.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:55:53 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T08:55:53.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:55:53 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T08:55:53.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:55:53 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T08:55:53.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:55:53 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:55:53.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:55:53 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:55:53.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:55:53 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T08:55:53.570 INFO:tasks.workunit.client.0.smithi083.stdout: 1 18174 0.71 MB/sec execute 27 sec latency 48.387 ms 2024-04-24T08:55:54.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:55:54 smithi138 ceph-mon[31441]: pgmap v131: 129 pgs: 129 active+clean; 36 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 627 KiB/s wr, 73 op/s 2024-04-24T08:55:54.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:55:54 smithi189 ceph-mon[31791]: pgmap v131: 129 pgs: 129 active+clean; 36 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 627 KiB/s wr, 73 op/s 2024-04-24T08:55:54.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:55:54 smithi083 ceph-mon[27907]: pgmap v131: 129 pgs: 129 active+clean; 36 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 627 KiB/s wr, 73 op/s 2024-04-24T08:55:54.570 INFO:tasks.workunit.client.0.smithi083.stdout: 1 18230 0.68 MB/sec execute 28 sec latency 57.406 ms 2024-04-24T08:55:55.570 INFO:tasks.workunit.client.0.smithi083.stdout: 1 18317 0.66 MB/sec execute 29 sec latency 54.379 ms 2024-04-24T08:55:56.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:55:56 smithi138 ceph-mon[31441]: pgmap v132: 129 pgs: 129 active+clean; 36 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 425 KiB/s wr, 52 op/s 2024-04-24T08:55:56.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:55:56 smithi189 ceph-mon[31791]: pgmap v132: 129 pgs: 129 active+clean; 36 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 425 KiB/s wr, 52 op/s 2024-04-24T08:55:56.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:55:56 smithi083 ceph-mon[27907]: pgmap v132: 129 pgs: 129 active+clean; 36 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 425 KiB/s wr, 52 op/s 2024-04-24T08:55:56.570 INFO:tasks.workunit.client.0.smithi083.stdout: 1 18407 0.64 MB/sec execute 30 sec latency 52.574 ms 2024-04-24T08:55:57.571 INFO:tasks.workunit.client.0.smithi083.stdout: 1 18533 0.63 MB/sec execute 31 sec latency 50.535 ms 2024-04-24T08:55:58.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:55:58 smithi138 ceph-mon[31441]: pgmap v133: 129 pgs: 129 active+clean; 36 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 425 KiB/s wr, 52 op/s 2024-04-24T08:55:58.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:55:58 smithi189 ceph-mon[31791]: pgmap v133: 129 pgs: 129 active+clean; 36 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 425 KiB/s wr, 52 op/s 2024-04-24T08:55:58.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:55:58 smithi083 ceph-mon[27907]: pgmap v133: 129 pgs: 129 active+clean; 36 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 425 KiB/s wr, 52 op/s 2024-04-24T08:55:58.574 INFO:tasks.workunit.client.0.smithi083.stdout: 1 18648 0.62 MB/sec execute 32 sec latency 57.654 ms 2024-04-24T08:55:59.571 INFO:tasks.workunit.client.0.smithi083.stdout: 1 18748 0.61 MB/sec execute 33 sec latency 49.967 ms 2024-04-24T08:56:00.293 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.293+0000 7f8fc2000640 1 -- 172.21.15.83:0/4181029411 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8fbc101930 msgr2=0x7f8fbc105970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:00.293 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.293+0000 7f8fc2000640 1 --2- 172.21.15.83:0/4181029411 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8fbc101930 0x7f8fbc105970 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f8fa8009900 tx=0x7f8fa802a410 comp rx=0 tx=0).stop 2024-04-24T08:56:00.293 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.293+0000 7f8fc2000640 1 -- 172.21.15.83:0/4181029411 shutdown_connections 2024-04-24T08:56:00.293 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.293+0000 7f8fc2000640 1 --2- 172.21.15.83:0/4181029411 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8fbc106050 0x7f8fbc10dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:00.293 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.293+0000 7f8fc2000640 1 --2- 172.21.15.83:0/4181029411 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8fbc101930 0x7f8fbc105970 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:00.293 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.293+0000 7f8fc2000640 1 --2- 172.21.15.83:0/4181029411 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8fbc100f80 0x7f8fbc101360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:00.293 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.293+0000 7f8fc2000640 1 -- 172.21.15.83:0/4181029411 >> 172.21.15.83:0/4181029411 conn(0x7f8fbc0fc7e0 msgr2=0x7f8fbc0fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:00.293 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.293+0000 7f8fc2000640 1 -- 172.21.15.83:0/4181029411 shutdown_connections 2024-04-24T08:56:00.293 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.293+0000 7f8fc2000640 1 -- 172.21.15.83:0/4181029411 wait complete. 2024-04-24T08:56:00.296 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.294+0000 7f8fc2000640 1 Processor -- start 2024-04-24T08:56:00.296 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.295+0000 7f8fc2000640 1 -- start start 2024-04-24T08:56:00.296 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.295+0000 7f8fc2000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8fbc100f80 0x7f8fbc1a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:00.296 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.295+0000 7f8fc2000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8fbc106050 0x7f8fbc1a0b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:00.296 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.295+0000 7f8fc2000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8fbc19a6b0 0x7f8fbc19ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:00.296 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.295+0000 7f8fc2000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8fbc1120d0 con 0x7f8fbc100f80 2024-04-24T08:56:00.296 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.295+0000 7f8fc2000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8fbc111f50 con 0x7f8fbc106050 2024-04-24T08:56:00.296 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.295+0000 7f8fc2000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8fbc112250 con 0x7f8fbc19a6b0 2024-04-24T08:56:00.296 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.295+0000 7f8fc0c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8fbc100f80 0x7f8fbc1a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:00.296 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.295+0000 7f8fc0c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8fbc100f80 0x7f8fbc1a05f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36906/0 (socket says 172.21.15.83:36906) 2024-04-24T08:56:00.297 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.295+0000 7f8fc0c00640 1 -- 172.21.15.83:0/296999430 learned_addr learned my addr 172.21.15.83:0/296999430 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:56:00.297 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.296+0000 7f8fbbe00640 1 --2- 172.21.15.83:0/296999430 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8fbc106050 0x7f8fbc1a0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:00.297 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.296+0000 7f8fc1600640 1 --2- 172.21.15.83:0/296999430 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8fbc19a6b0 0x7f8fbc19ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:00.297 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.296+0000 7f8fc0c00640 1 -- 172.21.15.83:0/296999430 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8fbc19a6b0 msgr2=0x7f8fbc19ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:00.297 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.296+0000 7f8fc0c00640 1 --2- 172.21.15.83:0/296999430 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8fbc19a6b0 0x7f8fbc19ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:00.297 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.296+0000 7f8fc0c00640 1 -- 172.21.15.83:0/296999430 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8fbc106050 msgr2=0x7f8fbc1a0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:00.297 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.296+0000 7f8fc0c00640 1 --2- 172.21.15.83:0/296999430 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8fbc106050 0x7f8fbc1a0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:00.297 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.296+0000 7f8fc0c00640 1 -- 172.21.15.83:0/296999430 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fbc19b2b0 con 0x7f8fbc100f80 2024-04-24T08:56:00.297 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.296+0000 7f8fc0c00640 1 --2- 172.21.15.83:0/296999430 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8fbc100f80 0x7f8fbc1a05f0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f8fb000d950 tx=0x7f8fb000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:00.297 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.296+0000 7f8fb9600640 1 -- 172.21.15.83:0/296999430 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb0017070 con 0x7f8fbc100f80 2024-04-24T08:56:00.297 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.296+0000 7f8fc2000640 1 -- 172.21.15.83:0/296999430 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fbc1a7390 con 0x7f8fbc100f80 2024-04-24T08:56:00.297 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.296+0000 7f8fb9600640 1 -- 172.21.15.83:0/296999430 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8fb0009c30 con 0x7f8fbc100f80 2024-04-24T08:56:00.297 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.296+0000 7f8fb9600640 1 -- 172.21.15.83:0/296999430 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb0020060 con 0x7f8fbc100f80 2024-04-24T08:56:00.298 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.296+0000 7f8fc2000640 1 -- 172.21.15.83:0/296999430 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fbc1a7740 con 0x7f8fbc100f80 2024-04-24T08:56:00.299 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.300+0000 7f8fb9600640 1 -- 172.21.15.83:0/296999430 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8fb00109a0 con 0x7f8fbc100f80 2024-04-24T08:56:00.299 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.300+0000 7f8fc6593740 1 Processor -- start 2024-04-24T08:56:00.300 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.300+0000 7f8fc6593740 1 -- start start 2024-04-24T08:56:00.300 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.300+0000 7f8fc6593740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df405413c0 0x55df405417a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:00.300 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.300+0000 7f8fc6593740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55df40541ce0 0x55df4054c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:00.300 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.300+0000 7f8fc6593740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df4054c5a0 0x55df4054e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:00.300 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.300+0000 7f8fc6593740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55df404a7900 con 0x55df405413c0 2024-04-24T08:56:00.300 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.300+0000 7f8fc6593740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55df4044df20 con 0x55df4054c5a0 2024-04-24T08:56:00.300 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.300+0000 7f8fc6593740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55df404a38c0 con 0x55df40541ce0 2024-04-24T08:56:00.302 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.301+0000 7f8fc1600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df405413c0 0x55df405417a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:00.302 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.301+0000 7f8fc1600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df405413c0 0x55df405417a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36920/0 (socket says 172.21.15.83:36920) 2024-04-24T08:56:00.302 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.301+0000 7f8fc1600640 1 -- 172.21.15.83:0/2017900515 learned_addr learned my addr 172.21.15.83:0/2017900515 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:56:00.302 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.301+0000 7f8fc0c00640 1 --2- 172.21.15.83:0/2017900515 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55df40541ce0 0x55df4054c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:00.302 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.301+0000 7f8fbbe00640 1 --2- 172.21.15.83:0/2017900515 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df4054c5a0 0x55df4054e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:00.302 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.301+0000 7f8fc1600640 1 -- 172.21.15.83:0/2017900515 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55df40541ce0 msgr2=0x55df4054c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:00.303 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.301+0000 7f8fc1600640 1 --2- 172.21.15.83:0/2017900515 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55df40541ce0 0x55df4054c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:00.303 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.301+0000 7f8fc1600640 1 -- 172.21.15.83:0/2017900515 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df4054c5a0 msgr2=0x55df4054e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:00.303 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.301+0000 7f8fc1600640 1 --2- 172.21.15.83:0/2017900515 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df4054c5a0 0x55df4054e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:00.303 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.301+0000 7f8fc1600640 1 -- 172.21.15.83:0/2017900515 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df40472e10 con 0x55df405413c0 2024-04-24T08:56:00.303 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.302+0000 7f8fb9600640 1 --2- 172.21.15.83:0/296999430 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f8fa40a3d10 0x7f8fa40a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:00.303 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.302+0000 7f8fc1600640 1 --2- 172.21.15.83:0/2017900515 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df405413c0 0x55df405417a0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4009f20 tx=0x7f8fb400f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3c6f6b387c4941b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:00.303 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.302+0000 7f8fb9600640 1 -- 172.21.15.83:0/296999430 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f8fb00d09e0 con 0x7f8fbc100f80 2024-04-24T08:56:00.303 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.302+0000 7f8fadc00640 1 -- 172.21.15.83:0/2017900515 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb4015070 con 0x55df405413c0 2024-04-24T08:56:00.303 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.302+0000 7f8fc0c00640 1 --2- 172.21.15.83:0/296999430 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f8fa40a3d10 0x7f8fa40a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:00.303 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.302+0000 7f8fadc00640 1 -- 172.21.15.83:0/2017900515 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8fb401ab10 con 0x55df405413c0 2024-04-24T08:56:00.303 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.302+0000 7f8fadc00640 1 -- 172.21.15.83:0/2017900515 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb4011b60 con 0x55df405413c0 2024-04-24T08:56:00.303 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.303+0000 7f8fc6593740 1 -- 172.21.15.83:0/2017900515 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df405413c0 msgr2=0x55df405417a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:00.303 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.303+0000 7f8fc0c00640 1 --2- 172.21.15.83:0/296999430 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f8fa40a3d10 0x7f8fa40a61d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x55df40542960 tx=0x7f8fb000bf60 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:00.303 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.303+0000 7f8fc6593740 1 --2- 172.21.15.83:0/2017900515 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df405413c0 0x55df405417a0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4009f20 tx=0x7f8fb400f7f0 comp rx=0 tx=0).stop 2024-04-24T08:56:00.303 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.303+0000 7f8fc6593740 1 -- 172.21.15.83:0/2017900515 shutdown_connections 2024-04-24T08:56:00.304 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.303+0000 7f8fc6593740 1 --2- 172.21.15.83:0/2017900515 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df4054c5a0 0x55df4054e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:00.304 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.303+0000 7f8fc6593740 1 --2- 172.21.15.83:0/2017900515 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55df40541ce0 0x55df4054c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:00.304 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.303+0000 7f8fc6593740 1 --2- 172.21.15.83:0/2017900515 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df405413c0 0x55df405417a0 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:00.304 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.303+0000 7f8fc6593740 1 -- 172.21.15.83:0/2017900515 >> 172.21.15.83:0/2017900515 conn(0x55df4053d8d0 msgr2=0x55df4053dcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:00.304 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.303+0000 7f8fc6593740 1 -- 172.21.15.83:0/2017900515 shutdown_connections 2024-04-24T08:56:00.304 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.303+0000 7f8fc6593740 1 -- 172.21.15.83:0/2017900515 wait complete. 2024-04-24T08:56:00.307 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.304+0000 7f8fc6593740 1 -- start start 2024-04-24T08:56:00.307 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.304+0000 7f8fc6593740 1 Processor -- start 2024-04-24T08:56:00.307 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.304+0000 7f8fc6593740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55df405413c0 0x55df405495d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:00.307 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.304+0000 7f8fc6593740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df40541ce0 0x55df40549b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:00.307 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.304+0000 7f8fc6593740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df4054c5a0 0x55df4054a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:00.307 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.304+0000 7f8fc6593740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55df4054f0e0 con 0x55df4054c5a0 2024-04-24T08:56:00.307 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.304+0000 7f8fc6593740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55df404612c0 con 0x55df40541ce0 2024-04-24T08:56:00.307 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.304+0000 7f8fc6593740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55df4054f260 con 0x55df405413c0 2024-04-24T08:56:00.307 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.304+0000 7f8fc6593740 10 client.0 _ll_register_callbacks cb 0x55df401b9a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:56:00.307 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.305+0000 7f8fc1600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df4054c5a0 0x55df4054a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:00.308 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.305+0000 7f8fc1600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df4054c5a0 0x55df4054a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36936/0 (socket says 172.21.15.83:36936) 2024-04-24T08:56:00.308 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.305+0000 7f8fc1600640 1 -- 172.21.15.83:0/1036031883 learned_addr learned my addr 172.21.15.83:0/1036031883 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:56:00.308 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.305+0000 7f8fc1600640 1 --2- 172.21.15.83:0/1036031883 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55df405413c0 0x55df405495d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:00.308 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.305+0000 7f8fbbe00640 1 --2- 172.21.15.83:0/1036031883 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df40541ce0 0x55df40549b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:00.308 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.305+0000 7f8fc1600640 1 -- 172.21.15.83:0/1036031883 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55df405413c0 msgr2=0x55df405495d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:00.308 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.305+0000 7f8fc1600640 1 --2- 172.21.15.83:0/1036031883 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55df405413c0 0x55df405495d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:00.308 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.305+0000 7f8fc1600640 1 -- 172.21.15.83:0/1036031883 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df40541ce0 msgr2=0x55df40549b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:00.308 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.305+0000 7f8fc1600640 1 --2- 172.21.15.83:0/1036031883 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df40541ce0 0x55df40549b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:00.308 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.305+0000 7f8fc1600640 1 -- 172.21.15.83:0/1036031883 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df40471fd0 con 0x55df4054c5a0 2024-04-24T08:56:00.308 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.306+0000 7f8fc1600640 1 --2- 172.21.15.83:0/1036031883 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df4054c5a0 0x55df4054a050 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f8fb401af60 tx=0x7f8fb4011870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b05bf9738b3ef7fa server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:00.308 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.306+0000 7f8fa2000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:56:00.308 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.306+0000 7f8fa2000640 1 -- 172.21.15.83:0/1036031883 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb4022070 con 0x55df4054c5a0 2024-04-24T08:56:00.308 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.306+0000 7f8fa2000640 1 -- 172.21.15.83:0/1036031883 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f8fb4004a40 con 0x55df4054c5a0 2024-04-24T08:56:00.308 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.306+0000 7f8fc6593740 1 -- 172.21.15.83:0/1036031883 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8fb401ab10 con 0x55df4054c5a0 2024-04-24T08:56:00.308 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.306+0000 7f8fa2000640 1 -- 172.21.15.83:0/1036031883 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8fb4032050 con 0x55df4054c5a0 2024-04-24T08:56:00.309 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.306+0000 7f8fc6593740 10 client.17229 fetch_fsmap learned FSMap version 30 2024-04-24T08:56:00.309 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.306+0000 7f8fc6593740 1 -- 172.21.15.83:0/1036031883 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55df401ba210 con 0x55df4054c5a0 2024-04-24T08:56:00.309 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.306+0000 7f8fa2000640 1 -- 172.21.15.83:0/1036031883 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8fb401ab10 con 0x55df4054c5a0 2024-04-24T08:56:00.309 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.307+0000 7f8fa2000640 1 -- 172.21.15.83:0/1036031883 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7f8fb40296f0 con 0x55df4054c5a0 2024-04-24T08:56:00.309 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.307+0000 7f8fc6593740 10 client.17229 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T08:56:00.309 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.307+0000 7f8fc6593740 10 client.17229 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T08:56:00.309 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.307+0000 7f8fc6593740 20 client.17229 populate_metadata read hostname 'smithi083' 2024-04-24T08:56:00.309 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.307+0000 7f8fc6593740 1 --2- 172.21.15.83:0/1036031883 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55df4055c6a0 0x55df4055ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:00.309 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.307+0000 7f8fc6593740 4 client.17229 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:56:00.309 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.307+0000 7f8fc6593740 1 -- 172.21.15.83:0/1036031883 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55df404b5320 con 0x55df4055c6a0 2024-04-24T08:56:00.309 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.307+0000 7f8fbbe00640 1 --2- 172.21.15.83:0/1036031883 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55df4055c6a0 0x55df4055ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:00.310 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.308+0000 7f8fbbe00640 1 --2- 172.21.15.83:0/1036031883 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55df4055c6a0 0x55df4055ea90 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c11aa50a7441374 server_cookie=fe93ca25372d769c in_seq=0 out_seq=0 2024-04-24T08:56:00.310 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.308+0000 7f8fa2000640 10 client.17229 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:56:00.311 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.311+0000 7f8fa2000640 1 -- 172.21.15.83:0/1036031883 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55df404b5320 con 0x55df4055c6a0 2024-04-24T08:56:00.311 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.311+0000 7f8fa2000640 10 client.17229 handle_command_reply: tid=0 2024-04-24T08:56:00.311 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.312+0000 7f8fc6593740 1 client.17229 shutdown 2024-04-24T08:56:00.311 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.312+0000 7f8fc6593740 1 -- 172.21.15.83:0/1036031883 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df4054c5a0 msgr2=0x55df4054a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:00.311 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.312+0000 7f8fc6593740 1 --2- 172.21.15.83:0/1036031883 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df4054c5a0 0x55df4054a050 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f8fb401af60 tx=0x7f8fb4011870 comp rx=0 tx=0).stop 2024-04-24T08:56:00.312 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.312+0000 7f8fc6593740 1 -- 172.21.15.83:0/1036031883 shutdown_connections 2024-04-24T08:56:00.312 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.312+0000 7f8fc6593740 1 --2- 172.21.15.83:0/1036031883 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55df4055c6a0 0x55df4055ea90 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:00.312 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.312+0000 7f8fc6593740 1 --2- 172.21.15.83:0/1036031883 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df4054c5a0 0x55df4054a050 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:00.312 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.312+0000 7f8fc6593740 1 --2- 172.21.15.83:0/1036031883 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df40541ce0 0x55df40549b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:00.312 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.312+0000 7f8fc6593740 1 --2- 172.21.15.83:0/1036031883 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55df405413c0 0x55df405495d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:00.312 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.312+0000 7f8fc6593740 1 -- 172.21.15.83:0/1036031883 >> 172.21.15.83:0/1036031883 conn(0x55df4053d8d0 msgr2=0x55df40543600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:00.312 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.312+0000 7f8fa2000640 0 client.17229 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:56:00.312 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.312+0000 7f8fc6593740 1 -- 172.21.15.83:0/1036031883 shutdown_connections 2024-04-24T08:56:00.312 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.312+0000 7f8fc6593740 1 -- 172.21.15.83:0/1036031883 wait complete. 2024-04-24T08:56:00.312 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.312+0000 7f8fc6593740 20 client.17229 trim_cache size 0 max 16384 2024-04-24T08:56:00.332 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.332+0000 7f8fc6593740 1 Processor -- start 2024-04-24T08:56:00.332 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.332+0000 7f8fc6593740 1 -- start start 2024-04-24T08:56:00.332 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.333+0000 7f8fc6593740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55df405413c0 0x55df4053d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:00.332 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.333+0000 7f8fc6593740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df40541ce0 0x55df40558080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:00.332 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.333+0000 7f8fc6593740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df4054c5a0 0x55df40549e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:00.332 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.333+0000 7f8fc6593740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8fbc1b1600 con 0x55df4054c5a0 2024-04-24T08:56:00.332 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.333+0000 7f8fc6593740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8fbc1b31d0 con 0x55df40541ce0 2024-04-24T08:56:00.332 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.333+0000 7f8fc6593740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8fbc1b1cd0 con 0x55df405413c0 2024-04-24T08:56:00.334 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.333+0000 7f8fc1600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df4054c5a0 0x55df40549e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:00.334 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.333+0000 7f8fc1600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df4054c5a0 0x55df40549e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36952/0 (socket says 172.21.15.83:36952) 2024-04-24T08:56:00.334 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.333+0000 7f8fc1600640 1 -- 172.21.15.83:0/3107158501 learned_addr learned my addr 172.21.15.83:0/3107158501 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:56:00.334 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.333+0000 7f8fbbe00640 1 --2- 172.21.15.83:0/3107158501 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df40541ce0 0x55df40558080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:00.334 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.334+0000 7f8fc1600640 1 --2- 172.21.15.83:0/3107158501 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55df405413c0 0x55df4053d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:00.334 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.334+0000 7f8fc1600640 1 -- 172.21.15.83:0/3107158501 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55df405413c0 msgr2=0x55df4053d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:00.334 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.334+0000 7f8fc1600640 1 --2- 172.21.15.83:0/3107158501 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55df405413c0 0x55df4053d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:00.334 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.334+0000 7f8fc1600640 1 -- 172.21.15.83:0/3107158501 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df40541ce0 msgr2=0x55df40558080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:00.334 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.334+0000 7f8fc1600640 1 --2- 172.21.15.83:0/3107158501 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df40541ce0 0x55df40558080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:00.334 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.334+0000 7f8fc1600640 1 -- 172.21.15.83:0/3107158501 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df405485a0 con 0x55df4054c5a0 2024-04-24T08:56:00.334 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.334+0000 7f8fc1600640 1 --2- 172.21.15.83:0/3107158501 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df4054c5a0 0x55df40549e20 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4006010 tx=0x7f8fb400f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ba595fd799ed442 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:00.335 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.334+0000 7f8fad200640 1 -- 172.21.15.83:0/3107158501 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb401e070 con 0x55df4054c5a0 2024-04-24T08:56:00.335 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.334+0000 7f8fad200640 1 -- 172.21.15.83:0/3107158501 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8fb4003e20 con 0x55df4054c5a0 2024-04-24T08:56:00.335 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.334+0000 7f8fad200640 1 -- 172.21.15.83:0/3107158501 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb4022040 con 0x55df4054c5a0 2024-04-24T08:56:00.338 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.335+0000 7f8fc6593740 1 -- 172.21.15.83:0/3107158501 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df4054c5a0 msgr2=0x55df40549e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:00.338 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.335+0000 7f8fc6593740 1 --2- 172.21.15.83:0/3107158501 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df4054c5a0 0x55df40549e20 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4006010 tx=0x7f8fb400f7f0 comp rx=0 tx=0).stop 2024-04-24T08:56:00.338 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.335+0000 7f8fc6593740 1 -- 172.21.15.83:0/3107158501 shutdown_connections 2024-04-24T08:56:00.338 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.335+0000 7f8fc6593740 1 --2- 172.21.15.83:0/3107158501 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df4054c5a0 0x55df40549e20 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:00.338 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.335+0000 7f8fc6593740 1 --2- 172.21.15.83:0/3107158501 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df40541ce0 0x55df40558080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:00.338 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.335+0000 7f8fc6593740 1 --2- 172.21.15.83:0/3107158501 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55df405413c0 0x55df4053d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:00.338 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.335+0000 7f8fc6593740 1 -- 172.21.15.83:0/3107158501 >> 172.21.15.83:0/3107158501 conn(0x55df4053d8d0 msgr2=0x55df4053dcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:00.338 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.335+0000 7f8fc6593740 1 -- 172.21.15.83:0/3107158501 shutdown_connections 2024-04-24T08:56:00.339 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.335+0000 7f8fc6593740 1 -- 172.21.15.83:0/3107158501 wait complete. 2024-04-24T08:56:00.339 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.336+0000 7f8fc6593740 1 -- start start 2024-04-24T08:56:00.339 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.336+0000 7f8fc6593740 1 Processor -- start 2024-04-24T08:56:00.339 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.336+0000 7f8fc6593740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55df405413c0 0x55df40546580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:00.339 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.337+0000 7f8fc6593740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df40541ce0 0x55df40546ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:00.339 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.337+0000 7f8fc6593740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df4054c5a0 0x55df40547020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:00.339 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.337+0000 7f8fc6593740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8fbc1b2a70 con 0x55df4054c5a0 2024-04-24T08:56:00.339 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.337+0000 7f8fc6593740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8fbc1b23a0 con 0x55df40541ce0 2024-04-24T08:56:00.339 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.337+0000 7f8fc6593740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8fbc1af5e0 con 0x55df405413c0 2024-04-24T08:56:00.339 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.337+0000 7f8fc6593740 10 client.0 _ll_register_callbacks cb 0x7f8fb4009da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:56:00.339 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.337+0000 7f8fc1600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df4054c5a0 0x55df40547020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:00.339 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.337+0000 7f8fc1600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df4054c5a0 0x55df40547020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36968/0 (socket says 172.21.15.83:36968) 2024-04-24T08:56:00.340 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.337+0000 7f8fc1600640 1 -- 172.21.15.83:0/3117790777 learned_addr learned my addr 172.21.15.83:0/3117790777 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:56:00.340 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.337+0000 7f8fbbe00640 1 --2- 172.21.15.83:0/3117790777 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df40541ce0 0x55df40546ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:00.340 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.337+0000 7f8fc1600640 1 -- 172.21.15.83:0/3117790777 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55df405413c0 msgr2=0x55df40546580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:00.340 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.337+0000 7f8fc1600640 1 --2- 172.21.15.83:0/3117790777 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55df405413c0 0x55df40546580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:00.340 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.337+0000 7f8fc1600640 1 -- 172.21.15.83:0/3117790777 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df40541ce0 msgr2=0x55df40546ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:00.340 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.337+0000 7f8fc1600640 1 --2- 172.21.15.83:0/3117790777 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df40541ce0 0x55df40546ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:00.340 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.337+0000 7f8fc1600640 1 -- 172.21.15.83:0/3117790777 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df4054ed00 con 0x55df4054c5a0 2024-04-24T08:56:00.340 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.338+0000 7f8fc1600640 1 --2- 172.21.15.83:0/3117790777 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df4054c5a0 0x55df40547020 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f8fb401c000 tx=0x7f8fb4009830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54b740adc31319cb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:00.340 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.338+0000 7f8fa2000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:56:00.340 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.338+0000 7f8fa2000640 1 -- 172.21.15.83:0/3117790777 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb401e070 con 0x55df4054c5a0 2024-04-24T08:56:00.340 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.338+0000 7f8fc6593740 1 -- 172.21.15.83:0/3117790777 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8fb4003e20 con 0x55df4054c5a0 2024-04-24T08:56:00.340 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.338+0000 7f8fa2000640 1 -- 172.21.15.83:0/3117790777 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f8fb400fcf0 con 0x55df4054c5a0 2024-04-24T08:56:00.340 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.338+0000 7f8fa2000640 1 -- 172.21.15.83:0/3117790777 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8fb4044050 con 0x55df4054c5a0 2024-04-24T08:56:00.340 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.338+0000 7f8fc6593740 10 client.17235 fetch_fsmap learned FSMap version 30 2024-04-24T08:56:00.341 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.338+0000 7f8fc6593740 1 -- 172.21.15.83:0/3117790777 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55df40286620 con 0x55df4054c5a0 2024-04-24T08:56:00.341 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.338+0000 7f8fa2000640 1 -- 172.21.15.83:0/3117790777 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8fb4003e20 con 0x55df4054c5a0 2024-04-24T08:56:00.341 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.339+0000 7f8fa2000640 1 -- 172.21.15.83:0/3117790777 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7f8fb4029430 con 0x55df4054c5a0 2024-04-24T08:56:00.341 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.339+0000 7f8fc6593740 10 client.17235 fetch_fsmap finished waiting for FSMap version 30 2024-04-24T08:56:00.341 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.339+0000 7f8fc6593740 10 client.17235 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T08:56:00.341 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.339+0000 7f8fc6593740 20 client.17235 populate_metadata read hostname 'smithi083' 2024-04-24T08:56:00.341 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.339+0000 7f8fc6593740 1 --2- 172.21.15.83:0/3117790777 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55df405c31a0 0x55df405c55c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:00.341 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.339+0000 7f8fc6593740 4 client.17235 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T08:56:00.341 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.339+0000 7f8fc6593740 1 -- 172.21.15.83:0/3117790777 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55df404af190 con 0x55df405c31a0 2024-04-24T08:56:00.342 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.339+0000 7f8fbbe00640 1 --2- 172.21.15.83:0/3117790777 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55df405c31a0 0x55df405c55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:00.342 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.340+0000 7f8fbbe00640 1 --2- 172.21.15.83:0/3117790777 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55df405c31a0 0x55df405c55c0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3543869373631b12 server_cookie=526dcbc88305dc9b in_seq=0 out_seq=0 2024-04-24T08:56:00.342 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.340+0000 7f8fa2000640 10 client.17235 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:56:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.392+0000 7f8fa2000640 1 -- 172.21.15.83:0/3117790777 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55df404af190 con 0x55df405c31a0 2024-04-24T08:56:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.392+0000 7f8fa2000640 10 client.17235 handle_command_reply: tid=0 2024-04-24T08:56:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.392+0000 7f8fc6593740 1 client.17235 shutdown 2024-04-24T08:56:00.393 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.393+0000 7f8fc6593740 1 -- 172.21.15.83:0/3117790777 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df4054c5a0 msgr2=0x55df40547020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:00.393 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.393+0000 7f8fc6593740 1 --2- 172.21.15.83:0/3117790777 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df4054c5a0 0x55df40547020 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f8fb401c000 tx=0x7f8fb4009830 comp rx=0 tx=0).stop 2024-04-24T08:56:00.393 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.393+0000 7f8fc6593740 1 -- 172.21.15.83:0/3117790777 shutdown_connections 2024-04-24T08:56:00.393 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.393+0000 7f8fc6593740 1 --2- 172.21.15.83:0/3117790777 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55df405c31a0 0x55df405c55c0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:00.393 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.393+0000 7f8fc6593740 1 --2- 172.21.15.83:0/3117790777 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df4054c5a0 0x55df40547020 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:00.393 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.393+0000 7f8fa2000640 0 client.17235 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:56:00.393 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.393+0000 7f8fc6593740 1 --2- 172.21.15.83:0/3117790777 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df40541ce0 0x55df40546ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:00.393 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.393+0000 7f8fc6593740 1 --2- 172.21.15.83:0/3117790777 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55df405413c0 0x55df40546580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:00.393 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.393+0000 7f8fc6593740 1 -- 172.21.15.83:0/3117790777 >> 172.21.15.83:0/3117790777 conn(0x55df4053d8d0 msgr2=0x55df40549e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:00.393 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.393+0000 7f8fc6593740 1 -- 172.21.15.83:0/3117790777 shutdown_connections 2024-04-24T08:56:00.393 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.393+0000 7f8fc6593740 1 -- 172.21.15.83:0/3117790777 wait complete. 2024-04-24T08:56:00.393 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.394+0000 7f8fc6593740 20 client.17235 trim_cache size 0 max 16384 2024-04-24T08:56:00.394 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T08:56:00.394 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T08:56:00.394 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T08:56:00.394 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T08:56:00.395 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.395+0000 7f8fc2000640 1 -- 172.21.15.83:0/296999430 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f8fa40a3d10 msgr2=0x7f8fa40a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:00.395 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.395+0000 7f8fc2000640 1 --2- 172.21.15.83:0/296999430 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f8fa40a3d10 0x7f8fa40a61d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x55df40542960 tx=0x7f8fb000bf60 comp rx=0 tx=0).stop 2024-04-24T08:56:00.395 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.395+0000 7f8fc2000640 1 -- 172.21.15.83:0/296999430 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8fbc100f80 msgr2=0x7f8fbc1a05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:00.395 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.395+0000 7f8fc2000640 1 --2- 172.21.15.83:0/296999430 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8fbc100f80 0x7f8fbc1a05f0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f8fb000d950 tx=0x7f8fb000de10 comp rx=0 tx=0).stop 2024-04-24T08:56:00.395 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.395+0000 7f8fc2000640 1 -- 172.21.15.83:0/296999430 shutdown_connections 2024-04-24T08:56:00.395 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.395+0000 7f8fc2000640 1 --2- 172.21.15.83:0/296999430 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f8fa40a3d10 0x7f8fa40a61d0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:00.395 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.395+0000 7f8fc2000640 1 --2- 172.21.15.83:0/296999430 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8fbc19a6b0 0x7f8fbc19ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:00.395 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.395+0000 7f8fc2000640 1 --2- 172.21.15.83:0/296999430 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8fbc106050 0x7f8fbc1a0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:00.395 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.395+0000 7f8fc2000640 1 --2- 172.21.15.83:0/296999430 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8fbc100f80 0x7f8fbc1a05f0 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:00.395 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.395+0000 7f8fc2000640 1 -- 172.21.15.83:0/296999430 >> 172.21.15.83:0/296999430 conn(0x7f8fbc0fc7e0 msgr2=0x7f8fbc0fdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:00.396 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.395+0000 7f8fc2000640 1 -- 172.21.15.83:0/296999430 shutdown_connections 2024-04-24T08:56:00.396 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:00.395+0000 7f8fc2000640 1 -- 172.21.15.83:0/296999430 wait complete. 2024-04-24T08:56:00.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:56:00 smithi138 ceph-mon[31441]: pgmap v134: 129 pgs: 129 active+clean; 36 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 506 KiB/s wr, 77 op/s 2024-04-24T08:56:00.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:56:00 smithi189 ceph-mon[31791]: pgmap v134: 129 pgs: 129 active+clean; 36 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 506 KiB/s wr, 77 op/s 2024-04-24T08:56:00.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:56:00 smithi083 ceph-mon[27907]: pgmap v134: 129 pgs: 129 active+clean; 36 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 506 KiB/s wr, 77 op/s 2024-04-24T08:56:00.571 INFO:tasks.workunit.client.0.smithi083.stdout: 1 18869 0.62 MB/sec execute 34 sec latency 62.909 ms 2024-04-24T08:56:01.571 INFO:tasks.workunit.client.0.smithi083.stdout: 1 18979 0.61 MB/sec execute 35 sec latency 49.049 ms 2024-04-24T08:56:02.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:56:02 smithi138 ceph-mon[31441]: pgmap v135: 129 pgs: 129 active+clean; 36 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 335 KiB/s wr, 48 op/s 2024-04-24T08:56:02.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:56:02 smithi189 ceph-mon[31791]: pgmap v135: 129 pgs: 129 active+clean; 36 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 335 KiB/s wr, 48 op/s 2024-04-24T08:56:02.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:56:02 smithi083 ceph-mon[27907]: pgmap v135: 129 pgs: 129 active+clean; 36 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 335 KiB/s wr, 48 op/s 2024-04-24T08:56:02.571 INFO:tasks.workunit.client.0.smithi083.stdout: 1 19148 0.67 MB/sec execute 36 sec latency 56.143 ms 2024-04-24T08:56:03.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:56:03 smithi138 ceph-mon[31441]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T08:56:03.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:56:03 smithi189 ceph-mon[31791]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T08:56:03.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:56:03 smithi083 ceph-mon[27907]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T08:56:03.571 INFO:tasks.workunit.client.0.smithi083.stdout: 1 19255 0.65 MB/sec execute 37 sec latency 54.251 ms 2024-04-24T08:56:04.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:56:04 smithi138 ceph-mon[31441]: pgmap v136: 129 pgs: 129 active+clean; 34 MiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 531 KiB/s wr, 93 op/s 2024-04-24T08:56:04.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:56:04 smithi189 ceph-mon[31791]: pgmap v136: 129 pgs: 129 active+clean; 34 MiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 531 KiB/s wr, 93 op/s 2024-04-24T08:56:04.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:56:04 smithi083 ceph-mon[27907]: pgmap v136: 129 pgs: 129 active+clean; 34 MiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 531 KiB/s wr, 93 op/s 2024-04-24T08:56:04.571 INFO:tasks.workunit.client.0.smithi083.stdout: 1 19367 0.64 MB/sec execute 38 sec latency 56.736 ms 2024-04-24T08:56:05.571 INFO:tasks.workunit.client.0.smithi083.stdout: 1 19522 0.63 MB/sec execute 39 sec latency 48.761 ms 2024-04-24T08:56:05.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:56:05 smithi138 ceph-mon[31441]: pgmap v137: 129 pgs: 129 active+clean; 34 MiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 279 KiB/s wr, 69 op/s 2024-04-24T08:56:05.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:56:05 smithi189 ceph-mon[31791]: pgmap v137: 129 pgs: 129 active+clean; 34 MiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 279 KiB/s wr, 69 op/s 2024-04-24T08:56:05.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:56:05 smithi083 ceph-mon[27907]: pgmap v137: 129 pgs: 129 active+clean; 34 MiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 279 KiB/s wr, 69 op/s 2024-04-24T08:56:06.572 INFO:tasks.workunit.client.0.smithi083.stdout: 1 19649 0.61 MB/sec execute 40 sec latency 56.123 ms 2024-04-24T08:56:07.572 INFO:tasks.workunit.client.0.smithi083.stdout: 1 19794 0.60 MB/sec execute 41 sec latency 57.547 ms 2024-04-24T08:56:07.954 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:56:07 smithi138 ceph-mon[31441]: pgmap v138: 129 pgs: 129 active+clean; 34 MiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 279 KiB/s wr, 69 op/s 2024-04-24T08:56:07.956 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:56:07 smithi189 ceph-mon[31791]: pgmap v138: 129 pgs: 129 active+clean; 34 MiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 279 KiB/s wr, 69 op/s 2024-04-24T08:56:08.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:56:07 smithi083 ceph-mon[27907]: pgmap v138: 129 pgs: 129 active+clean; 34 MiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 279 KiB/s wr, 69 op/s 2024-04-24T08:56:08.572 INFO:tasks.workunit.client.0.smithi083.stdout: 1 19929 0.59 MB/sec execute 42 sec latency 61.725 ms 2024-04-24T08:56:08.857 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:56:08 smithi138 ceph-mon[31441]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T08:56:08.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:56:08 smithi189 ceph-mon[31791]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T08:56:09.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:56:08 smithi083 ceph-mon[27907]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T08:56:09.572 INFO:tasks.workunit.client.0.smithi083.stdout: 1 20106 0.61 MB/sec execute 43 sec latency 50.032 ms 2024-04-24T08:56:09.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:56:09 smithi138 ceph-mon[31441]: pgmap v139: 129 pgs: 129 active+clean; 35 MiB data, 511 MiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 334 KiB/s wr, 90 op/s 2024-04-24T08:56:09.956 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:56:09 smithi189 ceph-mon[31791]: pgmap v139: 129 pgs: 129 active+clean; 35 MiB data, 511 MiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 334 KiB/s wr, 90 op/s 2024-04-24T08:56:10.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:56:09 smithi083 ceph-mon[27907]: pgmap v139: 129 pgs: 129 active+clean; 35 MiB data, 511 MiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 334 KiB/s wr, 90 op/s 2024-04-24T08:56:10.572 INFO:tasks.workunit.client.0.smithi083.stdout: 1 20183 0.59 MB/sec execute 44 sec latency 48.612 ms 2024-04-24T08:56:11.572 INFO:tasks.workunit.client.0.smithi083.stdout: 1 20274 0.60 MB/sec execute 45 sec latency 55.135 ms 2024-04-24T08:56:11.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:56:11 smithi138 ceph-mon[31441]: pgmap v140: 129 pgs: 129 active+clean; 35 MiB data, 511 MiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 253 KiB/s wr, 65 op/s 2024-04-24T08:56:11.956 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:56:11 smithi189 ceph-mon[31791]: pgmap v140: 129 pgs: 129 active+clean; 35 MiB data, 511 MiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 253 KiB/s wr, 65 op/s 2024-04-24T08:56:12.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:56:11 smithi083 ceph-mon[27907]: pgmap v140: 129 pgs: 129 active+clean; 35 MiB data, 511 MiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 253 KiB/s wr, 65 op/s 2024-04-24T08:56:12.572 INFO:tasks.workunit.client.0.smithi083.stdout: 1 20383 0.59 MB/sec execute 46 sec latency 51.604 ms 2024-04-24T08:56:12.817 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T08:56:12.817 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T08:56:13.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.034+0000 7f7f00e00640 1 -- 172.21.15.83:0/1376429111 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7efc104cd0 msgr2=0x7f7efc108d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:13.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.034+0000 7f7f00e00640 1 --2- 172.21.15.83:0/1376429111 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7efc104cd0 0x7f7efc108d10 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f7ee8009900 tx=0x7f7ee802a3f0 comp rx=0 tx=0).stop 2024-04-24T08:56:13.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.034+0000 7f7f00e00640 1 -- 172.21.15.83:0/1376429111 shutdown_connections 2024-04-24T08:56:13.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.034+0000 7f7f00e00640 1 --2- 172.21.15.83:0/1376429111 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7efc1093f0 0x7f7efc110fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:13.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.034+0000 7f7f00e00640 1 --2- 172.21.15.83:0/1376429111 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7efc104cd0 0x7f7efc108d10 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:13.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.034+0000 7f7f00e00640 1 --2- 172.21.15.83:0/1376429111 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7efc104320 0x7f7efc104700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:13.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.034+0000 7f7f00e00640 1 -- 172.21.15.83:0/1376429111 >> 172.21.15.83:0/1376429111 conn(0x7f7efc0ffb40 msgr2=0x7f7efc101f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:13.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.034+0000 7f7f00e00640 1 -- 172.21.15.83:0/1376429111 shutdown_connections 2024-04-24T08:56:13.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.034+0000 7f7f00e00640 1 -- 172.21.15.83:0/1376429111 wait complete. 2024-04-24T08:56:13.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.035+0000 7f7f00e00640 1 Processor -- start 2024-04-24T08:56:13.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.035+0000 7f7f00e00640 1 -- start start 2024-04-24T08:56:13.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.036+0000 7f7f00e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7efc104320 0x7f7efc1a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:13.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.036+0000 7f7f00e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7efc1093f0 0x7f7efc1a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:13.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.036+0000 7f7f00e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7efc19a760 0x7f7efc19abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:13.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.036+0000 7f7f00e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7efc114380 con 0x7f7efc104320 2024-04-24T08:56:13.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.036+0000 7f7f00e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7efc114200 con 0x7f7efc19a760 2024-04-24T08:56:13.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.036+0000 7f7f00e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7efc114500 con 0x7f7efc1093f0 2024-04-24T08:56:13.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.036+0000 7f7efb400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7efc104320 0x7f7efc1a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:13.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.036+0000 7f7efb400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7efc104320 0x7f7efc1a0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45512/0 (socket says 172.21.15.83:45512) 2024-04-24T08:56:13.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.036+0000 7f7efb400640 1 -- 172.21.15.83:0/2828888033 learned_addr learned my addr 172.21.15.83:0/2828888033 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:56:13.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.036+0000 7f7efaa00640 1 --2- 172.21.15.83:0/2828888033 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7efc1093f0 0x7f7efc1a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:13.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.037+0000 7f7efbe00640 1 --2- 172.21.15.83:0/2828888033 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7efc19a760 0x7f7efc19abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:13.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.037+0000 7f7efb400640 1 -- 172.21.15.83:0/2828888033 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7efc1093f0 msgr2=0x7f7efc1a0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:13.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.037+0000 7f7efb400640 1 --2- 172.21.15.83:0/2828888033 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7efc1093f0 0x7f7efc1a0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:13.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.037+0000 7f7efb400640 1 -- 172.21.15.83:0/2828888033 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7efc19a760 msgr2=0x7f7efc19abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:13.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.037+0000 7f7efb400640 1 --2- 172.21.15.83:0/2828888033 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7efc19a760 0x7f7efc19abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:13.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.037+0000 7f7efb400640 1 -- 172.21.15.83:0/2828888033 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7efc19b360 con 0x7f7efc104320 2024-04-24T08:56:13.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.037+0000 7f7efb400640 1 --2- 172.21.15.83:0/2828888033 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7efc104320 0x7f7efc1a0650 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f7eec00d950 tx=0x7f7eec00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:13.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.037+0000 7f7ef3e00640 1 -- 172.21.15.83:0/2828888033 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eec017070 con 0x7f7efc104320 2024-04-24T08:56:13.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.037+0000 7f7ef3e00640 1 -- 172.21.15.83:0/2828888033 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7eec009c30 con 0x7f7efc104320 2024-04-24T08:56:13.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.037+0000 7f7f00e00640 1 -- 172.21.15.83:0/2828888033 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7efc1a73f0 con 0x7f7efc104320 2024-04-24T08:56:13.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.037+0000 7f7ef3e00640 1 -- 172.21.15.83:0/2828888033 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eec00f620 con 0x7f7efc104320 2024-04-24T08:56:13.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.038+0000 7f7f00e00640 1 -- 172.21.15.83:0/2828888033 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7efc1a77a0 con 0x7f7efc104320 2024-04-24T08:56:13.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.041+0000 7f7f051f6740 1 Processor -- start 2024-04-24T08:56:13.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.041+0000 7f7ef3e00640 1 -- 172.21.15.83:0/2828888033 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7eec0109a0 con 0x7f7efc104320 2024-04-24T08:56:13.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.041+0000 7f7f051f6740 1 -- start start 2024-04-24T08:56:13.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.041+0000 7f7f051f6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ecf86b0a30 0x55ecf86b0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:13.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.041+0000 7f7f051f6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ecf86b1350 0x55ecf86bb4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:13.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.042+0000 7f7f051f6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ecf86bba20 0x55ecf86bde10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:13.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.042+0000 7f7f051f6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ecf86161f0 con 0x55ecf86b1350 2024-04-24T08:56:13.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.042+0000 7f7f051f6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ecf85bc350 con 0x55ecf86b0a30 2024-04-24T08:56:13.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.042+0000 7f7f051f6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55ecf86121b0 con 0x55ecf86bba20 2024-04-24T08:56:13.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.042+0000 7f7efb400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ecf86b1350 0x55ecf86bb4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:13.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.042+0000 7f7efb400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ecf86b1350 0x55ecf86bb4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45526/0 (socket says 172.21.15.83:45526) 2024-04-24T08:56:13.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.042+0000 7f7efb400640 1 -- 172.21.15.83:0/1964053479 learned_addr learned my addr 172.21.15.83:0/1964053479 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:56:13.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.042+0000 7f7efbe00640 1 --2- 172.21.15.83:0/1964053479 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ecf86b0a30 0x55ecf86b0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:13.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.042+0000 7f7efaa00640 1 --2- 172.21.15.83:0/1964053479 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ecf86bba20 0x55ecf86bde10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:13.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.042+0000 7f7efb400640 1 -- 172.21.15.83:0/1964053479 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ecf86bba20 msgr2=0x55ecf86bde10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:13.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.042+0000 7f7efb400640 1 --2- 172.21.15.83:0/1964053479 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ecf86bba20 0x55ecf86bde10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:13.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.043+0000 7f7efb400640 1 -- 172.21.15.83:0/1964053479 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ecf86b0a30 msgr2=0x55ecf86b0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:13.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.043+0000 7f7efb400640 1 --2- 172.21.15.83:0/1964053479 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ecf86b0a30 0x55ecf86b0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:13.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.043+0000 7f7efb400640 1 -- 172.21.15.83:0/1964053479 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ecf8610f50 con 0x55ecf86b1350 2024-04-24T08:56:13.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.043+0000 7f7efb400640 1 --2- 172.21.15.83:0/1964053479 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ecf86b1350 0x55ecf86bb4e0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f7eec0cdb40 tx=0x7f7eec0d8670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d949cd15934ab816 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:13.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.043+0000 7f7ef3e00640 1 --2- 172.21.15.83:0/2828888033 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f7ee00a3d20 0x7f7ee00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:13.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.043+0000 7f7ef0c00640 1 -- 172.21.15.83:0/1964053479 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eec0df070 con 0x55ecf86b1350 2024-04-24T08:56:13.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.044+0000 7f7ef0c00640 1 -- 172.21.15.83:0/1964053479 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7eec09cb50 con 0x55ecf86b1350 2024-04-24T08:56:13.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.044+0000 7f7ef0c00640 1 -- 172.21.15.83:0/1964053479 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eec09bdb0 con 0x55ecf86b1350 2024-04-24T08:56:13.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.044+0000 7f7ef3e00640 1 -- 172.21.15.83:0/2828888033 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f7eec0d08a0 con 0x7f7efc104320 2024-04-24T08:56:13.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.044+0000 7f7efbe00640 1 --2- 172.21.15.83:0/2828888033 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f7ee00a3d20 0x7f7ee00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:13.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.044+0000 7f7f051f6740 1 -- 172.21.15.83:0/1964053479 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ecf86b1350 msgr2=0x55ecf86bb4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:13.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.045+0000 7f7f051f6740 1 --2- 172.21.15.83:0/1964053479 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ecf86b1350 0x55ecf86bb4e0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f7eec0cdb40 tx=0x7f7eec0d8670 comp rx=0 tx=0).stop 2024-04-24T08:56:13.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.045+0000 7f7f051f6740 1 -- 172.21.15.83:0/1964053479 shutdown_connections 2024-04-24T08:56:13.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.045+0000 7f7f051f6740 1 --2- 172.21.15.83:0/1964053479 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ecf86bba20 0x55ecf86bde10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:13.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.045+0000 7f7f051f6740 1 --2- 172.21.15.83:0/1964053479 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ecf86b1350 0x55ecf86bb4e0 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:13.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.045+0000 7f7f051f6740 1 --2- 172.21.15.83:0/1964053479 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ecf86b0a30 0x55ecf86b0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:13.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.045+0000 7f7f051f6740 1 -- 172.21.15.83:0/1964053479 >> 172.21.15.83:0/1964053479 conn(0x55ecf86acd50 msgr2=0x55ecf86ad130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:13.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.045+0000 7f7efbe00640 1 --2- 172.21.15.83:0/2828888033 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f7ee00a3d20 0x7f7ee00a61e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7ef4005fd0 tx=0x7f7ef4009500 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:13.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.045+0000 7f7f051f6740 1 -- 172.21.15.83:0/1964053479 shutdown_connections 2024-04-24T08:56:13.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.045+0000 7f7f051f6740 1 -- 172.21.15.83:0/1964053479 wait complete. 2024-04-24T08:56:13.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.046+0000 7f7f051f6740 1 -- start start 2024-04-24T08:56:13.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.046+0000 7f7f051f6740 1 Processor -- start 2024-04-24T08:56:13.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.046+0000 7f7f051f6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ecf86b0a30 0x55ecf86b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:13.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.046+0000 7f7f051f6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ecf86bba20 0x55ecf86b9150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:13.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.046+0000 7f7f051f6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ecf86b96e0 0x55ecf86c7c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:13.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.046+0000 7f7f051f6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ecf86be810 con 0x55ecf86b96e0 2024-04-24T08:56:13.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.046+0000 7f7f051f6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ecf85cf690 con 0x55ecf86bba20 2024-04-24T08:56:13.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.046+0000 7f7f051f6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55ecf86be990 con 0x55ecf86b0a30 2024-04-24T08:56:13.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.046+0000 7f7f051f6740 10 client.0 _ll_register_callbacks cb 0x55ecf860b650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:56:13.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.047+0000 7f7efbe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ecf86b96e0 0x55ecf86c7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:13.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.047+0000 7f7efbe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ecf86b96e0 0x55ecf86c7c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45530/0 (socket says 172.21.15.83:45530) 2024-04-24T08:56:13.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.047+0000 7f7efbe00640 1 -- 172.21.15.83:0/2784986258 learned_addr learned my addr 172.21.15.83:0/2784986258 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:56:13.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.047+0000 7f7efaa00640 1 --2- 172.21.15.83:0/2784986258 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ecf86bba20 0x55ecf86b9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:13.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.047+0000 7f7efb400640 1 --2- 172.21.15.83:0/2784986258 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ecf86b0a30 0x55ecf86b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:13.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.047+0000 7f7efbe00640 1 -- 172.21.15.83:0/2784986258 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ecf86b0a30 msgr2=0x55ecf86b8c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:13.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.047+0000 7f7efbe00640 1 --2- 172.21.15.83:0/2784986258 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ecf86b0a30 0x55ecf86b8c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:13.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.047+0000 7f7efbe00640 1 -- 172.21.15.83:0/2784986258 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ecf86bba20 msgr2=0x55ecf86b9150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:13.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.047+0000 7f7efbe00640 1 --2- 172.21.15.83:0/2784986258 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ecf86bba20 0x55ecf86b9150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:13.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.047+0000 7f7efbe00640 1 -- 172.21.15.83:0/2784986258 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ecf85ce920 con 0x55ecf86b96e0 2024-04-24T08:56:13.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.048+0000 7f7efbe00640 1 --2- 172.21.15.83:0/2784986258 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ecf86b96e0 0x55ecf86c7c80 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f7ef40049d0 tx=0x7f7ef4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fa38162d07b7ee9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:13.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.048+0000 7f7ee5600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:56:13.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.048+0000 7f7ee5600640 1 -- 172.21.15.83:0/2784986258 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ef402c060 con 0x55ecf86b96e0 2024-04-24T08:56:13.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.048+0000 7f7ee5600640 1 -- 172.21.15.83:0/2784986258 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f7ef4017030 con 0x55ecf86b96e0 2024-04-24T08:56:13.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.048+0000 7f7f051f6740 1 -- 172.21.15.83:0/2784986258 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7eec09cb50 con 0x55ecf86b96e0 2024-04-24T08:56:13.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.048+0000 7f7ee5600640 1 -- 172.21.15.83:0/2784986258 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f7ef4033050 con 0x55ecf86b96e0 2024-04-24T08:56:13.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.048+0000 7f7f051f6740 10 client.17247 fetch_fsmap learned FSMap version 32 2024-04-24T08:56:13.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.048+0000 7f7f051f6740 1 -- 172.21.15.83:0/2784986258 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55ecf83f27d0 con 0x55ecf86b96e0 2024-04-24T08:56:13.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.049+0000 7f7ee5600640 1 -- 172.21.15.83:0/2784986258 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7eec09cb50 con 0x55ecf86b96e0 2024-04-24T08:56:13.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.049+0000 7f7ee5600640 1 -- 172.21.15.83:0/2784986258 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7f7ef4027960 con 0x55ecf86b96e0 2024-04-24T08:56:13.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.049+0000 7f7f051f6740 10 client.17247 fetch_fsmap finished waiting for FSMap version 32 2024-04-24T08:56:13.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.049+0000 7f7f051f6740 10 client.17247 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:56:13.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.049+0000 7f7f051f6740 20 client.17247 populate_metadata read hostname 'smithi083' 2024-04-24T08:56:13.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.049+0000 7f7f051f6740 1 --2- 172.21.15.83:0/2784986258 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ecf86cc460 0x55ecf86ce8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:13.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.049+0000 7f7f051f6740 4 client.17247 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:56:13.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.049+0000 7f7f051f6740 1 -- 172.21.15.83:0/2784986258 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ecf8623c10 con 0x55ecf86cc460 2024-04-24T08:56:13.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.050+0000 7f7efb400640 1 --2- 172.21.15.83:0/2784986258 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ecf86cc460 0x55ecf86ce8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:13.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.051+0000 7f7efb400640 1 --2- 172.21.15.83:0/2784986258 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ecf86cc460 0x55ecf86ce8a0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18ee33b0194e5c19 server_cookie=7edc710f7d235 in_seq=0 out_seq=0 2024-04-24T08:56:13.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.051+0000 7f7ee5600640 10 client.17247 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:56:13.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.053+0000 7f7ee5600640 1 -- 172.21.15.83:0/2784986258 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ecf8623c10 con 0x55ecf86cc460 2024-04-24T08:56:13.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.053+0000 7f7ee5600640 10 client.17247 handle_command_reply: tid=0 2024-04-24T08:56:13.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.053+0000 7f7f051f6740 1 client.17247 shutdown 2024-04-24T08:56:13.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.054+0000 7f7f051f6740 1 -- 172.21.15.83:0/2784986258 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ecf86b96e0 msgr2=0x55ecf86c7c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:13.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.054+0000 7f7f051f6740 1 --2- 172.21.15.83:0/2784986258 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ecf86b96e0 0x55ecf86c7c80 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f7ef40049d0 tx=0x7f7ef4004ab0 comp rx=0 tx=0).stop 2024-04-24T08:56:13.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.054+0000 7f7f051f6740 1 -- 172.21.15.83:0/2784986258 shutdown_connections 2024-04-24T08:56:13.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.054+0000 7f7f051f6740 1 --2- 172.21.15.83:0/2784986258 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ecf86cc460 0x55ecf86ce8a0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:13.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.054+0000 7f7f051f6740 1 --2- 172.21.15.83:0/2784986258 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ecf86b96e0 0x55ecf86c7c80 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:13.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.054+0000 7f7ee5600640 0 client.17247 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:56:13.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.054+0000 7f7f051f6740 1 --2- 172.21.15.83:0/2784986258 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ecf86bba20 0x55ecf86b9150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:13.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.054+0000 7f7f051f6740 1 --2- 172.21.15.83:0/2784986258 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ecf86b0a30 0x55ecf86b8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:13.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.054+0000 7f7f051f6740 1 -- 172.21.15.83:0/2784986258 >> 172.21.15.83:0/2784986258 conn(0x55ecf86acd50 msgr2=0x55ecf86abfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:13.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.054+0000 7f7f051f6740 1 -- 172.21.15.83:0/2784986258 shutdown_connections 2024-04-24T08:56:13.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.054+0000 7f7f051f6740 1 -- 172.21.15.83:0/2784986258 wait complete. 2024-04-24T08:56:13.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.054+0000 7f7f051f6740 20 client.17247 trim_cache size 0 max 16384 2024-04-24T08:56:13.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.075+0000 7f7f051f6740 1 Processor -- start 2024-04-24T08:56:13.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.075+0000 7f7f051f6740 1 -- start start 2024-04-24T08:56:13.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.075+0000 7f7f051f6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ecf86b0a30 0x55ecf86ac750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:13.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.075+0000 7f7f051f6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ecf86b96e0 0x55ecf86d4170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:13.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.075+0000 7f7f051f6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ecf86bba20 0x55ecf86d46b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:13.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.075+0000 7f7f051f6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7efc1b15d0 con 0x55ecf86b96e0 2024-04-24T08:56:13.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.075+0000 7f7f051f6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7efc1b3140 con 0x55ecf86bba20 2024-04-24T08:56:13.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.075+0000 7f7f051f6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7efc1b1ca0 con 0x55ecf86b0a30 2024-04-24T08:56:13.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.075+0000 7f7efaa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ecf86b96e0 0x55ecf86d4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:13.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.075+0000 7f7efaa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ecf86b96e0 0x55ecf86d4170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45538/0 (socket says 172.21.15.83:45538) 2024-04-24T08:56:13.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.075+0000 7f7efaa00640 1 -- 172.21.15.83:0/2638777897 learned_addr learned my addr 172.21.15.83:0/2638777897 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:56:13.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.076+0000 7f7efb400640 1 --2- 172.21.15.83:0/2638777897 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ecf86b0a30 0x55ecf86ac750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:13.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.076+0000 7f7efbe00640 1 --2- 172.21.15.83:0/2638777897 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ecf86bba20 0x55ecf86d46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:13.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.076+0000 7f7efaa00640 1 -- 172.21.15.83:0/2638777897 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ecf86b0a30 msgr2=0x55ecf86ac750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:13.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.076+0000 7f7efaa00640 1 --2- 172.21.15.83:0/2638777897 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ecf86b0a30 0x55ecf86ac750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:13.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.076+0000 7f7efaa00640 1 -- 172.21.15.83:0/2638777897 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ecf86bba20 msgr2=0x55ecf86d46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:13.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.076+0000 7f7efaa00640 1 --2- 172.21.15.83:0/2638777897 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ecf86bba20 0x55ecf86d46b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:13.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.076+0000 7f7efaa00640 1 -- 172.21.15.83:0/2638777897 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ecf83283f0 con 0x55ecf86b96e0 2024-04-24T08:56:13.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.076+0000 7f7efaa00640 1 --2- 172.21.15.83:0/2638777897 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ecf86b96e0 0x55ecf86d4170 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f7efc06b560 tx=0x7f7ee8004260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=302e5368a2134a6d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:13.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.077+0000 7f7ee7e00640 1 -- 172.21.15.83:0/2638777897 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee8035070 con 0x55ecf86b96e0 2024-04-24T08:56:13.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.077+0000 7f7ee7e00640 1 -- 172.21.15.83:0/2638777897 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7ee80054f0 con 0x55ecf86b96e0 2024-04-24T08:56:13.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.077+0000 7f7ee7e00640 1 -- 172.21.15.83:0/2638777897 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee80035f0 con 0x55ecf86b96e0 2024-04-24T08:56:13.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.077+0000 7f7f051f6740 1 -- 172.21.15.83:0/2638777897 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ecf86b96e0 msgr2=0x55ecf86d4170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:13.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.077+0000 7f7f051f6740 1 --2- 172.21.15.83:0/2638777897 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ecf86b96e0 0x55ecf86d4170 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f7efc06b560 tx=0x7f7ee8004260 comp rx=0 tx=0).stop 2024-04-24T08:56:13.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.077+0000 7f7f051f6740 1 -- 172.21.15.83:0/2638777897 shutdown_connections 2024-04-24T08:56:13.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.077+0000 7f7f051f6740 1 --2- 172.21.15.83:0/2638777897 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ecf86bba20 0x55ecf86d46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:13.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.077+0000 7f7f051f6740 1 --2- 172.21.15.83:0/2638777897 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ecf86b96e0 0x55ecf86d4170 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:13.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.077+0000 7f7f051f6740 1 --2- 172.21.15.83:0/2638777897 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ecf86b0a30 0x55ecf86ac750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:13.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.077+0000 7f7f051f6740 1 -- 172.21.15.83:0/2638777897 >> 172.21.15.83:0/2638777897 conn(0x55ecf86acd50 msgr2=0x55ecf86b74e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:13.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.078+0000 7f7f051f6740 1 -- 172.21.15.83:0/2638777897 shutdown_connections 2024-04-24T08:56:13.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.078+0000 7f7f051f6740 1 -- 172.21.15.83:0/2638777897 wait complete. 2024-04-24T08:56:13.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.078+0000 7f7f051f6740 1 -- start start 2024-04-24T08:56:13.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.078+0000 7f7f051f6740 1 Processor -- start 2024-04-24T08:56:13.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.079+0000 7f7f051f6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ecf86b0a30 0x55ecf86c3e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:13.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.079+0000 7f7f051f6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ecf86b96e0 0x55ecf86b5460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:13.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.079+0000 7f7f051f6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ecf86bba20 0x55ecf86b59a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:13.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.079+0000 7f7f051f6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7efc1b2a40 con 0x55ecf86bba20 2024-04-24T08:56:13.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.079+0000 7f7f051f6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7efc1b2370 con 0x55ecf86b96e0 2024-04-24T08:56:13.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.079+0000 7f7f051f6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7efc1af5b0 con 0x55ecf86b0a30 2024-04-24T08:56:13.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.079+0000 7f7f051f6740 10 client.0 _ll_register_callbacks cb 0x7f7ef4009d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:56:13.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.079+0000 7f7efbe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ecf86bba20 0x55ecf86b59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:13.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.079+0000 7f7efbe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ecf86bba20 0x55ecf86b59a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45546/0 (socket says 172.21.15.83:45546) 2024-04-24T08:56:13.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.079+0000 7f7efbe00640 1 -- 172.21.15.83:0/1711547125 learned_addr learned my addr 172.21.15.83:0/1711547125 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:56:13.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.079+0000 7f7efaa00640 1 --2- 172.21.15.83:0/1711547125 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ecf86b96e0 0x55ecf86b5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:13.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.079+0000 7f7efb400640 1 --2- 172.21.15.83:0/1711547125 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ecf86b0a30 0x55ecf86c3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:13.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.079+0000 7f7efbe00640 1 -- 172.21.15.83:0/1711547125 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ecf86b0a30 msgr2=0x55ecf86c3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:13.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.079+0000 7f7efbe00640 1 --2- 172.21.15.83:0/1711547125 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ecf86b0a30 0x55ecf86c3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:13.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.079+0000 7f7efbe00640 1 -- 172.21.15.83:0/1711547125 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ecf86b96e0 msgr2=0x55ecf86b5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:13.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.079+0000 7f7efbe00640 1 --2- 172.21.15.83:0/1711547125 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ecf86b96e0 0x55ecf86b5460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:13.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.079+0000 7f7efbe00640 1 -- 172.21.15.83:0/1711547125 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ecf82e80f0 con 0x55ecf86bba20 2024-04-24T08:56:13.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.079+0000 7f7efb400640 1 --2- 172.21.15.83:0/1711547125 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ecf86b0a30 0x55ecf86c3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:56:13.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.080+0000 7f7efbe00640 1 --2- 172.21.15.83:0/1711547125 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ecf86bba20 0x55ecf86b59a0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f7ef4004010 tx=0x7f7ef4011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fd7001cf936aac2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:13.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.080+0000 7f7ee5600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:56:13.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.080+0000 7f7ee5600640 1 -- 172.21.15.83:0/1711547125 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ef403c060 con 0x55ecf86bba20 2024-04-24T08:56:13.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.080+0000 7f7f051f6740 1 -- 172.21.15.83:0/1711547125 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ee80054f0 con 0x55ecf86bba20 2024-04-24T08:56:13.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.080+0000 7f7ee5600640 1 -- 172.21.15.83:0/1711547125 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f7ef40275e0 con 0x55ecf86bba20 2024-04-24T08:56:13.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.080+0000 7f7ee5600640 1 -- 172.21.15.83:0/1711547125 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f7ef4043050 con 0x55ecf86bba20 2024-04-24T08:56:13.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.080+0000 7f7f051f6740 10 client.17253 fetch_fsmap learned FSMap version 32 2024-04-24T08:56:13.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.080+0000 7f7f051f6740 1 -- 172.21.15.83:0/1711547125 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55ecf85db810 con 0x55ecf86bba20 2024-04-24T08:56:13.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.081+0000 7f7ee5600640 1 -- 172.21.15.83:0/1711547125 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7ee80054f0 con 0x55ecf86bba20 2024-04-24T08:56:13.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.081+0000 7f7ee5600640 1 -- 172.21.15.83:0/1711547125 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7f7ef4037a50 con 0x55ecf86bba20 2024-04-24T08:56:13.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.081+0000 7f7f051f6740 10 client.17253 fetch_fsmap finished waiting for FSMap version 32 2024-04-24T08:56:13.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.081+0000 7f7f051f6740 10 client.17253 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:56:13.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.081+0000 7f7f051f6740 20 client.17253 populate_metadata read hostname 'smithi083' 2024-04-24T08:56:13.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.081+0000 7f7f051f6740 1 --2- 172.21.15.83:0/1711547125 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ecf86cc460 0x55ecf8734bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:13.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.081+0000 7f7f051f6740 4 client.17253 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T08:56:13.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.081+0000 7f7f051f6740 1 -- 172.21.15.83:0/1711547125 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ecf861da80 con 0x55ecf86cc460 2024-04-24T08:56:13.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.082+0000 7f7efb400640 1 --2- 172.21.15.83:0/1711547125 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ecf86cc460 0x55ecf8734bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:13.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.082+0000 7f7efb400640 1 --2- 172.21.15.83:0/1711547125 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ecf86cc460 0x55ecf8734bd0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2a6539220db98d3 server_cookie=641d15ced292a820 in_seq=0 out_seq=0 2024-04-24T08:56:13.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.082+0000 7f7ee5600640 10 client.17253 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:56:13.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.083+0000 7f7ee5600640 1 -- 172.21.15.83:0/1711547125 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55ecf861da80 con 0x55ecf86cc460 2024-04-24T08:56:13.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.083+0000 7f7ee5600640 10 client.17253 handle_command_reply: tid=0 2024-04-24T08:56:13.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.083+0000 7f7f051f6740 1 client.17253 shutdown 2024-04-24T08:56:13.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.083+0000 7f7f051f6740 1 -- 172.21.15.83:0/1711547125 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ecf86bba20 msgr2=0x55ecf86b59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:13.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.083+0000 7f7f051f6740 1 --2- 172.21.15.83:0/1711547125 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ecf86bba20 0x55ecf86b59a0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f7ef4004010 tx=0x7f7ef4011ac0 comp rx=0 tx=0).stop 2024-04-24T08:56:13.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.083+0000 7f7f051f6740 1 -- 172.21.15.83:0/1711547125 shutdown_connections 2024-04-24T08:56:13.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.083+0000 7f7f051f6740 1 --2- 172.21.15.83:0/1711547125 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ecf86cc460 0x55ecf8734bd0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:13.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.084+0000 7f7f051f6740 1 --2- 172.21.15.83:0/1711547125 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ecf86bba20 0x55ecf86b59a0 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:13.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.084+0000 7f7f051f6740 1 --2- 172.21.15.83:0/1711547125 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ecf86b96e0 0x55ecf86b5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:13.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.084+0000 7f7f051f6740 1 --2- 172.21.15.83:0/1711547125 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ecf86b0a30 0x55ecf86c3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:13.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.084+0000 7f7ee5600640 0 client.17253 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:56:13.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.084+0000 7f7f051f6740 1 -- 172.21.15.83:0/1711547125 >> 172.21.15.83:0/1711547125 conn(0x55ecf86acd50 msgr2=0x55ecf86b2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:13.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.084+0000 7f7f051f6740 1 -- 172.21.15.83:0/1711547125 shutdown_connections 2024-04-24T08:56:13.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.084+0000 7f7f051f6740 1 -- 172.21.15.83:0/1711547125 wait complete. 2024-04-24T08:56:13.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.084+0000 7f7f051f6740 20 client.17253 trim_cache size 0 max 16384 2024-04-24T08:56:13.087 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T08:56:13.087 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-04-24T08:56:13.087 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-04-24T08:56:13.087 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T08:56:13.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.084+0000 7f7f00e00640 1 -- 172.21.15.83:0/2828888033 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f7ee00a3d20 msgr2=0x7f7ee00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:13.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.084+0000 7f7f00e00640 1 --2- 172.21.15.83:0/2828888033 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f7ee00a3d20 0x7f7ee00a61e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7ef4005fd0 tx=0x7f7ef4009500 comp rx=0 tx=0).stop 2024-04-24T08:56:13.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.084+0000 7f7f00e00640 1 -- 172.21.15.83:0/2828888033 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7efc104320 msgr2=0x7f7efc1a0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:13.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.084+0000 7f7f00e00640 1 --2- 172.21.15.83:0/2828888033 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7efc104320 0x7f7efc1a0650 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f7eec00d950 tx=0x7f7eec00de10 comp rx=0 tx=0).stop 2024-04-24T08:56:13.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.085+0000 7f7f00e00640 1 -- 172.21.15.83:0/2828888033 shutdown_connections 2024-04-24T08:56:13.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.085+0000 7f7f00e00640 1 --2- 172.21.15.83:0/2828888033 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f7ee00a3d20 0x7f7ee00a61e0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:13.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.085+0000 7f7f00e00640 1 --2- 172.21.15.83:0/2828888033 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7efc19a760 0x7f7efc19abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:13.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.085+0000 7f7f00e00640 1 --2- 172.21.15.83:0/2828888033 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7efc1093f0 0x7f7efc1a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:13.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.085+0000 7f7f00e00640 1 --2- 172.21.15.83:0/2828888033 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7efc104320 0x7f7efc1a0650 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:13.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.085+0000 7f7f00e00640 1 -- 172.21.15.83:0/2828888033 >> 172.21.15.83:0/2828888033 conn(0x7f7efc0ffb40 msgr2=0x7f7efc100e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:13.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.085+0000 7f7f00e00640 1 -- 172.21.15.83:0/2828888033 shutdown_connections 2024-04-24T08:56:13.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.085+0000 7f7f00e00640 1 -- 172.21.15.83:0/2828888033 wait complete. 2024-04-24T08:56:13.103 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T08:56:13.104 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T08:56:13.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.309+0000 7efda9800640 1 -- 172.21.15.83:0/2422209693 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efda4074cd0 msgr2=0x7efda410e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:13.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.309+0000 7efda9800640 1 --2- 172.21.15.83:0/2422209693 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efda4074cd0 0x7efda410e1a0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7efd8c009900 tx=0x7efd8c02a480 comp rx=0 tx=0).stop 2024-04-24T08:56:13.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.310+0000 7efda9800640 1 -- 172.21.15.83:0/2422209693 shutdown_connections 2024-04-24T08:56:13.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.310+0000 7efda9800640 1 --2- 172.21.15.83:0/2422209693 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efda410e6e0 0x7efda4110ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:13.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.310+0000 7efda9800640 1 --2- 172.21.15.83:0/2422209693 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efda4074cd0 0x7efda410e1a0 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:13.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.310+0000 7efda9800640 1 --2- 172.21.15.83:0/2422209693 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efda40743b0 0x7efda4074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:13.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.310+0000 7efda9800640 1 -- 172.21.15.83:0/2422209693 >> 172.21.15.83:0/2422209693 conn(0x7efda40fd930 msgr2=0x7efda40ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:13.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.310+0000 7efda9800640 1 -- 172.21.15.83:0/2422209693 shutdown_connections 2024-04-24T08:56:13.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.310+0000 7efda9800640 1 -- 172.21.15.83:0/2422209693 wait complete. 2024-04-24T08:56:13.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.311+0000 7efda9800640 1 Processor -- start 2024-04-24T08:56:13.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.311+0000 7efda9800640 1 -- start start 2024-04-24T08:56:13.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.311+0000 7efda9800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efda40743b0 0x7efda41a05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:13.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.311+0000 7efda9800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efda4074cd0 0x7efda41a0b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:13.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.311+0000 7efda9800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efda410e6e0 0x7efda419a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:13.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.311+0000 7efda9800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7efda41142f0 con 0x7efda410e6e0 2024-04-24T08:56:13.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.311+0000 7efda9800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efda4114170 con 0x7efda40743b0 2024-04-24T08:56:13.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.311+0000 7efda9800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7efda4114470 con 0x7efda4074cd0 2024-04-24T08:56:13.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.312+0000 7efda8e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efda410e6e0 0x7efda419a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:13.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.312+0000 7efda8e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efda410e6e0 0x7efda419a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45574/0 (socket says 172.21.15.83:45574) 2024-04-24T08:56:13.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.312+0000 7efda8e00640 1 -- 172.21.15.83:0/4277759247 learned_addr learned my addr 172.21.15.83:0/4277759247 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:56:13.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.312+0000 7efda3e00640 1 --2- 172.21.15.83:0/4277759247 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efda40743b0 0x7efda41a05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:13.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.312+0000 7efda8e00640 1 -- 172.21.15.83:0/4277759247 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efda4074cd0 msgr2=0x7efda41a0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:13.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.312+0000 7efda8e00640 1 --2- 172.21.15.83:0/4277759247 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efda4074cd0 0x7efda41a0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:13.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.312+0000 7efda8e00640 1 -- 172.21.15.83:0/4277759247 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efda40743b0 msgr2=0x7efda41a05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:13.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.312+0000 7efda8e00640 1 --2- 172.21.15.83:0/4277759247 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efda40743b0 0x7efda41a05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:13.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.312+0000 7efda8e00640 1 -- 172.21.15.83:0/4277759247 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efda419afb0 con 0x7efda410e6e0 2024-04-24T08:56:13.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.312+0000 7efda3e00640 1 --2- 172.21.15.83:0/4277759247 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efda40743b0 0x7efda41a05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:56:13.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.313+0000 7efda8e00640 1 --2- 172.21.15.83:0/4277759247 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efda410e6e0 0x7efda419a6f0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7efd9c00cad0 tx=0x7efd9c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:13.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.313+0000 7efda0c00640 1 -- 172.21.15.83:0/4277759247 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd9c00ae20 con 0x7efda410e6e0 2024-04-24T08:56:13.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.313+0000 7efda0c00640 1 -- 172.21.15.83:0/4277759247 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efd9c005020 con 0x7efda410e6e0 2024-04-24T08:56:13.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.313+0000 7efda0c00640 1 -- 172.21.15.83:0/4277759247 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd9c00f6b0 con 0x7efda410e6e0 2024-04-24T08:56:13.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.313+0000 7efda9800640 1 -- 172.21.15.83:0/4277759247 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efda419b2a0 con 0x7efda410e6e0 2024-04-24T08:56:13.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.313+0000 7efda9800640 1 -- 172.21.15.83:0/4277759247 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efda41a7560 con 0x7efda410e6e0 2024-04-24T08:56:13.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.314+0000 7efda9800640 1 -- 172.21.15.83:0/4277759247 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efda4075a60 con 0x7efda410e6e0 2024-04-24T08:56:13.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.317+0000 7efda0c00640 1 -- 172.21.15.83:0/4277759247 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7efd9c0051d0 con 0x7efda410e6e0 2024-04-24T08:56:13.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.319+0000 7efda0c00640 1 --2- 172.21.15.83:0/4277759247 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7efd880a3da0 0x7efd880a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:13.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.319+0000 7efda0c00640 1 -- 172.21.15.83:0/4277759247 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7efd9c0d0cb0 con 0x7efda410e6e0 2024-04-24T08:56:13.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.319+0000 7efda3e00640 1 --2- 172.21.15.83:0/4277759247 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7efd880a3da0 0x7efd880a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:13.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.320+0000 7efda3e00640 1 --2- 172.21.15.83:0/4277759247 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7efd880a3da0 0x7efd880a6260 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7efd94009b40 tx=0x7efd94005e50 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:13.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.324+0000 7efda0c00640 1 -- 172.21.15.83:0/4277759247 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efd9c014030 con 0x7efda410e6e0 2024-04-24T08:56:13.550 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.549+0000 7fee81c00640 1 -- 172.21.15.83:0/427233043 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee7c101930 msgr2=0x7fee7c105970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:13.550 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.549+0000 7fee81c00640 1 --2- 172.21.15.83:0/427233043 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee7c101930 0x7fee7c105970 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fee68009900 tx=0x7fee6802a410 comp rx=0 tx=0).stop 2024-04-24T08:56:13.550 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.549+0000 7fee81c00640 1 -- 172.21.15.83:0/427233043 shutdown_connections 2024-04-24T08:56:13.550 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.549+0000 7fee81c00640 1 --2- 172.21.15.83:0/427233043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee7c106050 0x7fee7c10dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:13.550 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.549+0000 7fee81c00640 1 --2- 172.21.15.83:0/427233043 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee7c101930 0x7fee7c105970 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:13.550 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.549+0000 7fee81c00640 1 --2- 172.21.15.83:0/427233043 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fee7c100f80 0x7fee7c101360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:13.550 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.549+0000 7fee81c00640 1 -- 172.21.15.83:0/427233043 >> 172.21.15.83:0/427233043 conn(0x7fee7c0fc7e0 msgr2=0x7fee7c0fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:13.550 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.549+0000 7fee81c00640 1 -- 172.21.15.83:0/427233043 shutdown_connections 2024-04-24T08:56:13.550 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.549+0000 7fee81c00640 1 -- 172.21.15.83:0/427233043 wait complete. 2024-04-24T08:56:13.552 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.550+0000 7fee81c00640 1 Processor -- start 2024-04-24T08:56:13.552 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.551+0000 7fee81c00640 1 -- start start 2024-04-24T08:56:13.552 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.551+0000 7fee81c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fee7c100f80 0x7fee7c1986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:13.552 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.551+0000 7fee81c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee7c106050 0x7fee7c198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:13.552 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.551+0000 7fee81c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee7c19cf10 0x7fee7c19d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:13.552 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.551+0000 7fee81c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fee7c1120d0 con 0x7fee7c106050 2024-04-24T08:56:13.552 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.551+0000 7fee81c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fee7c111f50 con 0x7fee7c19cf10 2024-04-24T08:56:13.552 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.551+0000 7fee81c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fee7c112250 con 0x7fee7c100f80 2024-04-24T08:56:13.552 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.551+0000 7fee7b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee7c106050 0x7fee7c198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:13.552 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.551+0000 7fee7b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee7c106050 0x7fee7c198be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45592/0 (socket says 172.21.15.83:45592) 2024-04-24T08:56:13.553 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.551+0000 7fee7b400640 1 -- 172.21.15.83:0/1186111628 learned_addr learned my addr 172.21.15.83:0/1186111628 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:56:13.553 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.551+0000 7fee7be00640 1 --2- 172.21.15.83:0/1186111628 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fee7c100f80 0x7fee7c1986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:13.553 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.552+0000 7fee81200640 1 --2- 172.21.15.83:0/1186111628 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee7c19cf10 0x7fee7c19d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:13.553 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.552+0000 7fee7b400640 1 -- 172.21.15.83:0/1186111628 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fee7c100f80 msgr2=0x7fee7c1986a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:13.553 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.552+0000 7fee7b400640 1 --2- 172.21.15.83:0/1186111628 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fee7c100f80 0x7fee7c1986a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:13.553 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.552+0000 7fee7b400640 1 -- 172.21.15.83:0/1186111628 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee7c19cf10 msgr2=0x7fee7c19d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:13.553 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.552+0000 7fee7b400640 1 --2- 172.21.15.83:0/1186111628 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee7c19cf10 0x7fee7c19d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:13.553 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.552+0000 7fee7b400640 1 -- 172.21.15.83:0/1186111628 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee7c19da00 con 0x7fee7c106050 2024-04-24T08:56:13.554 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.552+0000 7fee7b400640 1 --2- 172.21.15.83:0/1186111628 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee7c106050 0x7fee7c198be0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fee68004040 tx=0x7fee68004070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:13.554 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.553+0000 7fee78c00640 1 -- 172.21.15.83:0/1186111628 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee6803f030 con 0x7fee7c106050 2024-04-24T08:56:13.554 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.553+0000 7fee81c00640 1 -- 172.21.15.83:0/1186111628 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee7c19dc90 con 0x7fee7c106050 2024-04-24T08:56:13.554 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.553+0000 7fee78c00640 1 -- 172.21.15.83:0/1186111628 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fee68003aa0 con 0x7fee7c106050 2024-04-24T08:56:13.554 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.553+0000 7fee78c00640 1 -- 172.21.15.83:0/1186111628 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee68035070 con 0x7fee7c106050 2024-04-24T08:56:13.554 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.553+0000 7fee81c00640 1 -- 172.21.15.83:0/1186111628 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee7c1a54b0 con 0x7fee7c106050 2024-04-24T08:56:13.557 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.557+0000 7fee86019740 1 Processor -- start 2024-04-24T08:56:13.557 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.557+0000 7fee78c00640 1 -- 172.21.15.83:0/1186111628 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fee68042030 con 0x7fee7c106050 2024-04-24T08:56:13.557 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.557+0000 7fee86019740 1 -- start start 2024-04-24T08:56:13.558 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.557+0000 7fee86019740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562313e98280 0x562313e98660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:13.558 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.557+0000 7fee86019740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562313e98ba0 0x562313ea7470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:13.558 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.557+0000 7fee86019740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562313ea79b0 0x562313ea9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:13.559 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.557+0000 7fee86019740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562313dfe900 con 0x562313e98280 2024-04-24T08:56:13.559 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.557+0000 7fee86019740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562313da4f20 con 0x562313ea79b0 2024-04-24T08:56:13.559 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.557+0000 7fee86019740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x562313dfa8c0 con 0x562313e98ba0 2024-04-24T08:56:13.559 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.558+0000 7fee81200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562313e98280 0x562313e98660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:13.559 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.558+0000 7fee81200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562313e98280 0x562313e98660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45596/0 (socket says 172.21.15.83:45596) 2024-04-24T08:56:13.559 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.558+0000 7fee81200640 1 -- 172.21.15.83:0/2119991673 learned_addr learned my addr 172.21.15.83:0/2119991673 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:56:13.559 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.558+0000 7fee7be00640 1 --2- 172.21.15.83:0/2119991673 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562313e98ba0 0x562313ea7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:13.559 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.558+0000 7fee7b400640 1 --2- 172.21.15.83:0/2119991673 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562313ea79b0 0x562313ea9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:13.559 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.558+0000 7fee81200640 1 -- 172.21.15.83:0/2119991673 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562313e98ba0 msgr2=0x562313ea7470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:13.559 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.558+0000 7fee81200640 1 --2- 172.21.15.83:0/2119991673 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562313e98ba0 0x562313ea7470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:13.559 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.558+0000 7fee81200640 1 -- 172.21.15.83:0/2119991673 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562313ea79b0 msgr2=0x562313ea9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:13.559 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.558+0000 7fee81200640 1 --2- 172.21.15.83:0/2119991673 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562313ea79b0 0x562313ea9da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:13.559 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.558+0000 7fee81200640 1 -- 172.21.15.83:0/2119991673 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562313dc9e10 con 0x562313e98280 2024-04-24T08:56:13.561 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.559+0000 7fee81200640 1 --2- 172.21.15.83:0/2119991673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562313e98280 0x562313e98660 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fee74009e60 tx=0x7fee74010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30321e2093e80ba1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:13.561 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.559+0000 7fee78c00640 1 --2- 172.21.15.83:0/1186111628 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fee600a3d10 0x7fee600a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:13.561 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.559+0000 7fee71600640 1 -- 172.21.15.83:0/2119991673 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee74016070 con 0x562313e98280 2024-04-24T08:56:13.561 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.559+0000 7fee71600640 1 -- 172.21.15.83:0/2119991673 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fee74004d20 con 0x562313e98280 2024-04-24T08:56:13.561 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.559+0000 7fee71600640 1 -- 172.21.15.83:0/2119991673 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee740126e0 con 0x562313e98280 2024-04-24T08:56:13.561 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.559+0000 7fee78c00640 1 -- 172.21.15.83:0/1186111628 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7fee680f0d20 con 0x7fee7c106050 2024-04-24T08:56:13.561 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.559+0000 7fee7be00640 1 --2- 172.21.15.83:0/1186111628 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fee600a3d10 0x7fee600a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:13.561 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.560+0000 7fee86019740 1 -- 172.21.15.83:0/2119991673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562313e98280 msgr2=0x562313e98660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:13.561 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.560+0000 7fee86019740 1 --2- 172.21.15.83:0/2119991673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562313e98280 0x562313e98660 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fee74009e60 tx=0x7fee74010740 comp rx=0 tx=0).stop 2024-04-24T08:56:13.561 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.560+0000 7fee86019740 1 -- 172.21.15.83:0/2119991673 shutdown_connections 2024-04-24T08:56:13.562 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.560+0000 7fee86019740 1 --2- 172.21.15.83:0/2119991673 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562313ea79b0 0x562313ea9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:13.562 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.560+0000 7fee86019740 1 --2- 172.21.15.83:0/2119991673 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562313e98ba0 0x562313ea7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:13.562 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.560+0000 7fee86019740 1 --2- 172.21.15.83:0/2119991673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562313e98280 0x562313e98660 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:13.562 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.560+0000 7fee86019740 1 -- 172.21.15.83:0/2119991673 >> 172.21.15.83:0/2119991673 conn(0x562313e948d0 msgr2=0x562313e94cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:13.562 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.560+0000 7fee7be00640 1 --2- 172.21.15.83:0/1186111628 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fee600a3d10 0x7fee600a61d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x562313e9afb0 tx=0x7fee6c005f90 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:13.562 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.560+0000 7fee86019740 1 -- 172.21.15.83:0/2119991673 shutdown_connections 2024-04-24T08:56:13.562 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.560+0000 7fee86019740 1 -- 172.21.15.83:0/2119991673 wait complete. 2024-04-24T08:56:13.562 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.561+0000 7fee86019740 1 -- start start 2024-04-24T08:56:13.562 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.561+0000 7fee86019740 1 Processor -- start 2024-04-24T08:56:13.565 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.562+0000 7fee86019740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562313e98280 0x562313ea72e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:13.565 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.562+0000 7fee86019740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562313e98ba0 0x562313ea0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:13.565 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.562+0000 7fee86019740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562313ea79b0 0x562313ea0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:13.565 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.562+0000 7fee86019740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562313eaa580 con 0x562313e98280 2024-04-24T08:56:13.565 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.562+0000 7fee86019740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562313db82c0 con 0x562313e98ba0 2024-04-24T08:56:13.565 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.562+0000 7fee86019740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x562313eaa700 con 0x562313ea79b0 2024-04-24T08:56:13.565 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.562+0000 7fee86019740 10 client.0 _ll_register_callbacks cb 0x562313b10a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:56:13.565 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.562+0000 7fee7be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562313e98280 0x562313ea72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:13.565 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.562+0000 7fee7be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562313e98280 0x562313ea72e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45604/0 (socket says 172.21.15.83:45604) 2024-04-24T08:56:13.565 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.562+0000 7fee7be00640 1 -- 172.21.15.83:0/3327478708 learned_addr learned my addr 172.21.15.83:0/3327478708 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:56:13.565 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.562+0000 7fee7b400640 1 --2- 172.21.15.83:0/3327478708 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562313e98ba0 0x562313ea0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:13.565 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.562+0000 7fee81200640 1 --2- 172.21.15.83:0/3327478708 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562313ea79b0 0x562313ea0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:13.566 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.563+0000 7fee7be00640 1 -- 172.21.15.83:0/3327478708 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562313ea79b0 msgr2=0x562313ea0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:13.566 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.563+0000 7fee7be00640 1 --2- 172.21.15.83:0/3327478708 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562313ea79b0 0x562313ea0ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:13.566 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.563+0000 7fee7be00640 1 -- 172.21.15.83:0/3327478708 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562313e98ba0 msgr2=0x562313ea0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:13.566 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.563+0000 7fee7be00640 1 --2- 172.21.15.83:0/3327478708 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562313e98ba0 0x562313ea0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:13.566 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.563+0000 7fee7be00640 1 -- 172.21.15.83:0/3327478708 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562313dc8fd0 con 0x562313e98280 2024-04-24T08:56:13.566 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.563+0000 7fee7be00640 1 --2- 172.21.15.83:0/3327478708 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562313e98280 0x562313ea72e0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7fee6c004650 tx=0x7fee6c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb76ee4cac5568ce server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:13.566 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.563+0000 7fee66000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:56:13.566 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.563+0000 7fee66000640 1 -- 172.21.15.83:0/3327478708 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee6c009db0 con 0x562313e98280 2024-04-24T08:56:13.566 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.564+0000 7fee86019740 1 -- 172.21.15.83:0/3327478708 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee74004d20 con 0x562313e98280 2024-04-24T08:56:13.566 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.564+0000 7fee66000640 1 -- 172.21.15.83:0/3327478708 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7fee6c018030 con 0x562313e98280 2024-04-24T08:56:13.566 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.564+0000 7fee66000640 1 -- 172.21.15.83:0/3327478708 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fee6c034050 con 0x562313e98280 2024-04-24T08:56:13.566 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.564+0000 7fee86019740 10 client.17271 fetch_fsmap learned FSMap version 32 2024-04-24T08:56:13.566 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.564+0000 7fee86019740 1 -- 172.21.15.83:0/3327478708 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562313b11210 con 0x562313e98280 2024-04-24T08:56:13.566 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.564+0000 7fee66000640 1 -- 172.21.15.83:0/3327478708 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fee74004d20 con 0x562313e98280 2024-04-24T08:56:13.567 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.564+0000 7fee66000640 1 -- 172.21.15.83:0/3327478708 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7fee6c00f4c0 con 0x562313e98280 2024-04-24T08:56:13.567 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.565+0000 7fee86019740 10 client.17271 fetch_fsmap finished waiting for FSMap version 32 2024-04-24T08:56:13.567 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.565+0000 7fee86019740 10 client.17271 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T08:56:13.567 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.565+0000 7fee86019740 20 client.17271 populate_metadata read hostname 'smithi083' 2024-04-24T08:56:13.567 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.565+0000 7fee86019740 1 --2- 172.21.15.83:0/3327478708 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x562313ebc0d0 0x562313ebe4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:13.567 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.565+0000 7fee86019740 4 client.17271 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:56:13.567 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.565+0000 7fee86019740 1 -- 172.21.15.83:0/3327478708 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562313e0c320 con 0x562313ebc0d0 2024-04-24T08:56:13.568 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.565+0000 7fee7b400640 1 --2- 172.21.15.83:0/3327478708 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x562313ebc0d0 0x562313ebe4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:13.568 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.566+0000 7fee7b400640 1 --2- 172.21.15.83:0/3327478708 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x562313ebc0d0 0x562313ebe4c0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4792b956c5422684 server_cookie=eb5c63454f84c480 in_seq=0 out_seq=0 2024-04-24T08:56:13.568 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.566+0000 7fee66000640 10 client.17271 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:56:13.568 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.568+0000 7fee66000640 1 -- 172.21.15.83:0/3327478708 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562313e0c320 con 0x562313ebc0d0 2024-04-24T08:56:13.568 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.568+0000 7fee66000640 10 client.17271 handle_command_reply: tid=0 2024-04-24T08:56:13.568 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.568+0000 7fee86019740 1 client.17271 shutdown 2024-04-24T08:56:13.569 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.568+0000 7fee86019740 1 -- 172.21.15.83:0/3327478708 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562313e98280 msgr2=0x562313ea72e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:13.569 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.568+0000 7fee86019740 1 --2- 172.21.15.83:0/3327478708 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562313e98280 0x562313ea72e0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7fee6c004650 tx=0x7fee6c004730 comp rx=0 tx=0).stop 2024-04-24T08:56:13.569 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.568+0000 7fee86019740 1 -- 172.21.15.83:0/3327478708 shutdown_connections 2024-04-24T08:56:13.569 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.568+0000 7fee86019740 1 --2- 172.21.15.83:0/3327478708 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x562313ebc0d0 0x562313ebe4c0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:13.569 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.568+0000 7fee86019740 1 --2- 172.21.15.83:0/3327478708 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562313ea79b0 0x562313ea0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:13.569 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.568+0000 7fee86019740 1 --2- 172.21.15.83:0/3327478708 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562313e98ba0 0x562313ea0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:13.569 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.568+0000 7fee86019740 1 --2- 172.21.15.83:0/3327478708 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562313e98280 0x562313ea72e0 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:13.570 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.568+0000 7fee66000640 0 client.17271 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:56:13.570 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.568+0000 7fee86019740 1 -- 172.21.15.83:0/3327478708 >> 172.21.15.83:0/3327478708 conn(0x562313e948d0 msgr2=0x562313e9a670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:13.570 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.569+0000 7fee86019740 1 -- 172.21.15.83:0/3327478708 shutdown_connections 2024-04-24T08:56:13.570 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.569+0000 7fee86019740 1 -- 172.21.15.83:0/3327478708 wait complete. 2024-04-24T08:56:13.570 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.569+0000 7fee86019740 20 client.17271 trim_cache size 0 max 16384 2024-04-24T08:56:13.572 INFO:tasks.workunit.client.0.smithi083.stdout: 1 20678 0.66 MB/sec execute 47 sec latency 46.829 ms 2024-04-24T08:56:13.587 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.586+0000 7fee86019740 1 Processor -- start 2024-04-24T08:56:13.587 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.586+0000 7fee86019740 1 -- start start 2024-04-24T08:56:13.587 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.586+0000 7fee86019740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562313e98280 0x562313ea6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:13.587 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.586+0000 7fee86019740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562313e98ba0 0x562313ec3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:13.587 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.586+0000 7fee86019740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562313ea79b0 0x562313ec42f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:13.587 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.586+0000 7fee86019740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fee7c1ad130 con 0x562313e98280 2024-04-24T08:56:13.587 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.586+0000 7fee86019740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fee7c1aeca0 con 0x562313e98ba0 2024-04-24T08:56:13.587 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.586+0000 7fee86019740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fee7c1ad800 con 0x562313ea79b0 2024-04-24T08:56:13.587 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.587+0000 7fee7be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562313e98280 0x562313ea6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:13.587 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.587+0000 7fee7be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562313e98280 0x562313ea6e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45618/0 (socket says 172.21.15.83:45618) 2024-04-24T08:56:13.587 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.587+0000 7fee7be00640 1 -- 172.21.15.83:0/3601283655 learned_addr learned my addr 172.21.15.83:0/3601283655 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:56:13.588 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.587+0000 7fee81200640 1 --2- 172.21.15.83:0/3601283655 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562313ea79b0 0x562313ec42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:13.588 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.587+0000 7fee7b400640 1 --2- 172.21.15.83:0/3601283655 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562313e98ba0 0x562313ec3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:13.588 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.587+0000 7fee7be00640 1 -- 172.21.15.83:0/3601283655 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562313ea79b0 msgr2=0x562313ec42f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:13.588 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.587+0000 7fee7be00640 1 --2- 172.21.15.83:0/3601283655 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562313ea79b0 0x562313ec42f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:13.588 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.587+0000 7fee7be00640 1 -- 172.21.15.83:0/3601283655 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562313e98ba0 msgr2=0x562313ec3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:13.588 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.587+0000 7fee7be00640 1 --2- 172.21.15.83:0/3601283655 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562313e98ba0 0x562313ec3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:13.588 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.587+0000 7fee7be00640 1 -- 172.21.15.83:0/3601283655 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562313bdd620 con 0x562313e98280 2024-04-24T08:56:13.589 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.587+0000 7fee7be00640 1 --2- 172.21.15.83:0/3601283655 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562313e98280 0x562313ea6e10 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fee6c004700 tx=0x7fee6c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28518024dc2264b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:13.589 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.588+0000 7fee70c00640 1 -- 172.21.15.83:0/3601283655 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee6c00fb50 con 0x562313e98280 2024-04-24T08:56:13.589 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.588+0000 7fee70c00640 1 -- 172.21.15.83:0/3601283655 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fee6c00fcf0 con 0x562313e98280 2024-04-24T08:56:13.589 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.588+0000 7fee70c00640 1 -- 172.21.15.83:0/3601283655 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee6c004e60 con 0x562313e98280 2024-04-24T08:56:13.589 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.588+0000 7fee86019740 1 -- 172.21.15.83:0/3601283655 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562313e98280 msgr2=0x562313ea6e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:13.589 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.589+0000 7fee86019740 1 --2- 172.21.15.83:0/3601283655 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562313e98280 0x562313ea6e10 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fee6c004700 tx=0x7fee6c009b50 comp rx=0 tx=0).stop 2024-04-24T08:56:13.589 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.589+0000 7fee86019740 1 -- 172.21.15.83:0/3601283655 shutdown_connections 2024-04-24T08:56:13.589 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.589+0000 7fee86019740 1 --2- 172.21.15.83:0/3601283655 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562313ea79b0 0x562313ec42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:13.590 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.589+0000 7fee86019740 1 --2- 172.21.15.83:0/3601283655 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562313e98ba0 0x562313ec3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:13.590 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.589+0000 7fee86019740 1 --2- 172.21.15.83:0/3601283655 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562313e98280 0x562313ea6e10 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:13.590 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.589+0000 7fee86019740 1 -- 172.21.15.83:0/3601283655 >> 172.21.15.83:0/3601283655 conn(0x562313e948d0 msgr2=0x562313ea4660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:13.590 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.589+0000 7fee86019740 1 -- 172.21.15.83:0/3601283655 shutdown_connections 2024-04-24T08:56:13.590 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.589+0000 7fee86019740 1 -- 172.21.15.83:0/3601283655 wait complete. 2024-04-24T08:56:13.590 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.589+0000 7fee86019740 1 -- start start 2024-04-24T08:56:13.592 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.589+0000 7fee86019740 1 Processor -- start 2024-04-24T08:56:13.592 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.590+0000 7fee86019740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562313e98280 0x562313e9d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:13.592 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.590+0000 7fee86019740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562313e98ba0 0x562313e9dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:13.592 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.590+0000 7fee86019740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562313ea79b0 0x562313e9e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:13.593 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.590+0000 7fee86019740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fee7c1ae5a0 con 0x562313ea79b0 2024-04-24T08:56:13.593 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.590+0000 7fee86019740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fee7c1aded0 con 0x562313e98280 2024-04-24T08:56:13.593 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.590+0000 7fee86019740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fee7c1ab110 con 0x562313e98ba0 2024-04-24T08:56:13.593 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.590+0000 7fee86019740 10 client.0 _ll_register_callbacks cb 0x7fee6c009a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:56:13.593 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.590+0000 7fee81200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562313ea79b0 0x562313e9e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:13.593 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.590+0000 7fee81200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562313ea79b0 0x562313e9e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45634/0 (socket says 172.21.15.83:45634) 2024-04-24T08:56:13.593 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.590+0000 7fee81200640 1 -- 172.21.15.83:0/778026306 learned_addr learned my addr 172.21.15.83:0/778026306 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:56:13.593 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.591+0000 7fee7be00640 1 --2- 172.21.15.83:0/778026306 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562313e98280 0x562313e9d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:13.593 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.591+0000 7fee81200640 1 -- 172.21.15.83:0/778026306 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562313e98ba0 msgr2=0x562313e9dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:13.593 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.591+0000 7fee81200640 1 --2- 172.21.15.83:0/778026306 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562313e98ba0 0x562313e9dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:13.593 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.591+0000 7fee81200640 1 -- 172.21.15.83:0/778026306 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562313e98280 msgr2=0x562313e9d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:13.593 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.591+0000 7fee81200640 1 --2- 172.21.15.83:0/778026306 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562313e98280 0x562313e9d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:13.593 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.591+0000 7fee81200640 1 -- 172.21.15.83:0/778026306 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562313f21620 con 0x562313ea79b0 2024-04-24T08:56:13.594 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.591+0000 7fee7be00640 1 --2- 172.21.15.83:0/778026306 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562313e98280 0x562313e9d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:56:13.594 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.591+0000 7fee81200640 1 --2- 172.21.15.83:0/778026306 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562313ea79b0 0x562313e9e140 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fee74009db0 tx=0x7fee74009bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e269c79b93d72e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:13.594 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.591+0000 7fee66000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:56:13.594 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.591+0000 7fee66000640 1 -- 172.21.15.83:0/778026306 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee74028060 con 0x562313ea79b0 2024-04-24T08:56:13.594 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.591+0000 7fee66000640 1 -- 172.21.15.83:0/778026306 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7fee74012450 con 0x562313ea79b0 2024-04-24T08:56:13.594 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.591+0000 7fee86019740 1 -- 172.21.15.83:0/778026306 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee6c00fcf0 con 0x562313ea79b0 2024-04-24T08:56:13.594 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.592+0000 7fee66000640 1 -- 172.21.15.83:0/778026306 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fee7402f050 con 0x562313ea79b0 2024-04-24T08:56:13.594 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.592+0000 7fee86019740 10 client.17277 fetch_fsmap learned FSMap version 32 2024-04-24T08:56:13.594 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.592+0000 7fee86019740 1 -- 172.21.15.83:0/778026306 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562313f22840 con 0x562313ea79b0 2024-04-24T08:56:13.594 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.592+0000 7fee66000640 1 -- 172.21.15.83:0/778026306 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fee6c00fcf0 con 0x562313ea79b0 2024-04-24T08:56:13.594 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.592+0000 7fee66000640 1 -- 172.21.15.83:0/778026306 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7fee7400f040 con 0x562313ea79b0 2024-04-24T08:56:13.594 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.592+0000 7fee86019740 10 client.17277 fetch_fsmap finished waiting for FSMap version 32 2024-04-24T08:56:13.594 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.592+0000 7fee86019740 10 client.17277 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T08:56:13.594 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.592+0000 7fee86019740 20 client.17277 populate_metadata read hostname 'smithi083' 2024-04-24T08:56:13.595 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.592+0000 7fee86019740 1 --2- 172.21.15.83:0/778026306 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x562313f22cd0 0x562313f250f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:13.595 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.592+0000 7fee86019740 4 client.17277 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T08:56:13.595 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.592+0000 7fee86019740 1 -- 172.21.15.83:0/778026306 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562313e06190 con 0x562313f22cd0 2024-04-24T08:56:13.595 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.593+0000 7fee7be00640 1 --2- 172.21.15.83:0/778026306 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x562313f22cd0 0x562313f250f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:13.595 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.594+0000 7fee7be00640 1 --2- 172.21.15.83:0/778026306 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x562313f22cd0 0x562313f250f0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8744cd554f34734 server_cookie=301ebe87a4631e53 in_seq=0 out_seq=0 2024-04-24T08:56:13.595 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.594+0000 7fee66000640 10 client.17277 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:56:13.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.599+0000 7efda9800640 1 -- 172.21.15.83:0/4277759247 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efda4068d40 con 0x7efda410e6e0 2024-04-24T08:56:13.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.600+0000 7efda0c00640 1 -- 172.21.15.83:0/4277759247 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11945 (secure 0 0 0) 0x7efd9c09cf20 con 0x7efda410e6e0 2024-04-24T08:56:13.600 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:56:13.601 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/358872257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":358872257},{"type":"v1","addr":"172.21.15.83:6835","nonce":358872257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3157070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3157070851},{"type":"v1","addr":"172.21.15.83:6837","nonce":3157070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T08:48:29.422576+0000","modified":"2024-04-24T08:56:07.052757+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34520,"mds_1":34493,"mds_2":34451,"mds_3":34427,"mds_4":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14658":{"gid":14658,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6839/4085470844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":4085470844},{"type":"v1","addr":"172.21.15.83:6839","nonce":4085470844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14676":{"gid":14676,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6841/2634754210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":2634754210},{"type":"v1","addr":"172.21.15.83:6841","nonce":2634754210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24382":{"gid":24382,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/1859921310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1859921310},{"type":"v1","addr":"172.21.15.189:6833","nonce":1859921310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/2241923197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2241923197},{"type":"v1","addr":"172.21.15.189:6835","nonce":2241923197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6837/3172311220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":3172311220},{"type":"v1","addr":"172.21.15.189:6837","nonce":3172311220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.189:6839/2548965336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":2548965336},{"type":"v1","addr":"172.21.15.189:6839","nonce":2548965336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.138:6833/1157967658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1157967658},{"type":"v1","addr":"172.21.15.138:6833","nonce":1157967658}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.138:6835/1909219758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1909219758},{"type":"v1","addr":"172.21.15.138:6835","nonce":1909219758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34493":{"gid":34493,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.138:6837/900566754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":900566754},{"type":"v1","addr":"172.21.15.138:6837","nonce":900566754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34520":{"gid":34520,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.138:6839/2236932393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2236932393},{"type":"v1","addr":"172.21.15.138:6839","nonce":2236932393}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34520,"qdb_cluster":[34451,34493,34427,34520,24454]},"id":1}]} 2024-04-24T08:56:13.601 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 32 2024-04-24T08:56:13.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.603+0000 7efda9800640 1 -- 172.21.15.83:0/4277759247 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7efd880a3da0 msgr2=0x7efd880a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:13.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.603+0000 7efda9800640 1 --2- 172.21.15.83:0/4277759247 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7efd880a3da0 0x7efd880a6260 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7efd94009b40 tx=0x7efd94005e50 comp rx=0 tx=0).stop 2024-04-24T08:56:13.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.603+0000 7efda9800640 1 -- 172.21.15.83:0/4277759247 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efda410e6e0 msgr2=0x7efda419a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:13.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.603+0000 7efda9800640 1 --2- 172.21.15.83:0/4277759247 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efda410e6e0 0x7efda419a6f0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7efd9c00cad0 tx=0x7efd9c00cf90 comp rx=0 tx=0).stop 2024-04-24T08:56:13.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.603+0000 7efda9800640 1 -- 172.21.15.83:0/4277759247 shutdown_connections 2024-04-24T08:56:13.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.603+0000 7efda9800640 1 --2- 172.21.15.83:0/4277759247 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7efd880a3da0 0x7efd880a6260 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:13.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.603+0000 7efda9800640 1 --2- 172.21.15.83:0/4277759247 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efda410e6e0 0x7efda419a6f0 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:13.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.603+0000 7efda9800640 1 --2- 172.21.15.83:0/4277759247 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efda4074cd0 0x7efda41a0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:13.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.603+0000 7efda9800640 1 --2- 172.21.15.83:0/4277759247 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efda40743b0 0x7efda41a05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:13.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.603+0000 7efda9800640 1 -- 172.21.15.83:0/4277759247 >> 172.21.15.83:0/4277759247 conn(0x7efda40fd930 msgr2=0x7efda40fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:13.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.604+0000 7efda9800640 1 -- 172.21.15.83:0/4277759247 shutdown_connections 2024-04-24T08:56:13.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.604+0000 7efda9800640 1 -- 172.21.15.83:0/4277759247 wait complete. 2024-04-24T08:56:13.623 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-24T08:56:13.651 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.650+0000 7fee66000640 1 -- 172.21.15.83:0/778026306 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562313e06190 con 0x562313f22cd0 2024-04-24T08:56:13.651 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.650+0000 7fee66000640 10 client.17277 handle_command_reply: tid=0 2024-04-24T08:56:13.651 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.650+0000 7fee86019740 1 client.17277 shutdown 2024-04-24T08:56:13.651 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.650+0000 7fee86019740 1 -- 172.21.15.83:0/778026306 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562313ea79b0 msgr2=0x562313e9e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:13.651 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.651+0000 7fee86019740 1 --2- 172.21.15.83:0/778026306 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562313ea79b0 0x562313e9e140 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fee74009db0 tx=0x7fee74009bc0 comp rx=0 tx=0).stop 2024-04-24T08:56:13.651 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.651+0000 7fee86019740 1 -- 172.21.15.83:0/778026306 shutdown_connections 2024-04-24T08:56:13.651 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.651+0000 7fee86019740 1 --2- 172.21.15.83:0/778026306 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x562313f22cd0 0x562313f250f0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:13.651 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.651+0000 7fee86019740 1 --2- 172.21.15.83:0/778026306 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562313ea79b0 0x562313e9e140 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:13.651 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.651+0000 7fee66000640 0 client.17277 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:56:13.652 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.651+0000 7fee86019740 1 --2- 172.21.15.83:0/778026306 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562313e98ba0 0x562313e9dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:13.652 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.651+0000 7fee86019740 1 --2- 172.21.15.83:0/778026306 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562313e98280 0x562313e9d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:13.652 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.651+0000 7fee86019740 1 -- 172.21.15.83:0/778026306 >> 172.21.15.83:0/778026306 conn(0x562313e948d0 msgr2=0x562313e93a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:13.656 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T08:56:13.656 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T08:56:13.656 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T08:56:13.657 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T08:56:13.657 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.651+0000 7fee86019740 1 -- 172.21.15.83:0/778026306 shutdown_connections 2024-04-24T08:56:13.657 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.651+0000 7fee86019740 1 -- 172.21.15.83:0/778026306 wait complete. 2024-04-24T08:56:13.657 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.652+0000 7fee86019740 20 client.17277 trim_cache size 0 max 16384 2024-04-24T08:56:13.657 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.653+0000 7fee81c00640 1 -- 172.21.15.83:0/1186111628 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fee600a3d10 msgr2=0x7fee600a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:13.657 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.653+0000 7fee81c00640 1 --2- 172.21.15.83:0/1186111628 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fee600a3d10 0x7fee600a61d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x562313e9afb0 tx=0x7fee6c005f90 comp rx=0 tx=0).stop 2024-04-24T08:56:13.657 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.653+0000 7fee81c00640 1 -- 172.21.15.83:0/1186111628 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee7c106050 msgr2=0x7fee7c198be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:13.657 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.653+0000 7fee81c00640 1 --2- 172.21.15.83:0/1186111628 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee7c106050 0x7fee7c198be0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fee68004040 tx=0x7fee68004070 comp rx=0 tx=0).stop 2024-04-24T08:56:13.657 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.653+0000 7fee81c00640 1 -- 172.21.15.83:0/1186111628 shutdown_connections 2024-04-24T08:56:13.657 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.653+0000 7fee81c00640 1 --2- 172.21.15.83:0/1186111628 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fee600a3d10 0x7fee600a61d0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:13.657 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.653+0000 7fee81c00640 1 --2- 172.21.15.83:0/1186111628 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee7c19cf10 0x7fee7c19d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:13.657 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.653+0000 7fee81c00640 1 --2- 172.21.15.83:0/1186111628 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee7c106050 0x7fee7c198be0 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:13.657 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.653+0000 7fee81c00640 1 --2- 172.21.15.83:0/1186111628 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fee7c100f80 0x7fee7c1986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:13.658 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.653+0000 7fee81c00640 1 -- 172.21.15.83:0/1186111628 >> 172.21.15.83:0/1186111628 conn(0x7fee7c0fc7e0 msgr2=0x7fee7c0fdbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:13.658 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.653+0000 7fee81c00640 1 -- 172.21.15.83:0/1186111628 shutdown_connections 2024-04-24T08:56:13.658 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:13.653+0000 7fee81c00640 1 -- 172.21.15.83:0/1186111628 wait complete. 2024-04-24T08:56:13.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.852+0000 7f6f89e00640 1 -- 172.21.15.83:0/2899197566 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f84104e00 msgr2=0x7f6f84108e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:13.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.852+0000 7f6f89e00640 1 --2- 172.21.15.83:0/2899197566 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f84104e00 0x7f6f84108e40 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f6f70009900 tx=0x7f6f7002a3f0 comp rx=0 tx=0).stop 2024-04-24T08:56:13.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.852+0000 7f6f89e00640 1 -- 172.21.15.83:0/2899197566 shutdown_connections 2024-04-24T08:56:13.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.852+0000 7f6f89e00640 1 --2- 172.21.15.83:0/2899197566 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6f84109520 0x7f6f841110f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:13.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.852+0000 7f6f89e00640 1 --2- 172.21.15.83:0/2899197566 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f84104e00 0x7f6f84108e40 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:13.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.852+0000 7f6f89e00640 1 --2- 172.21.15.83:0/2899197566 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6f84104450 0x7f6f84104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:13.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.852+0000 7f6f89e00640 1 -- 172.21.15.83:0/2899197566 >> 172.21.15.83:0/2899197566 conn(0x7f6f840ffc70 msgr2=0x7f6f84102090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:13.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.852+0000 7f6f89e00640 1 -- 172.21.15.83:0/2899197566 shutdown_connections 2024-04-24T08:56:13.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.852+0000 7f6f89e00640 1 -- 172.21.15.83:0/2899197566 wait complete. 2024-04-24T08:56:13.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.853+0000 7f6f89e00640 1 Processor -- start 2024-04-24T08:56:13.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.853+0000 7f6f89e00640 1 -- start start 2024-04-24T08:56:13.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.853+0000 7f6f89e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6f84104450 0x7f6f841a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:13.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.853+0000 7f6f89e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f84109520 0x7f6f841a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:13.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.853+0000 7f6f89e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6f8419a810 0x7f6f8419acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:13.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.853+0000 7f6f89e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6f841144b0 con 0x7f6f84109520 2024-04-24T08:56:13.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.853+0000 7f6f89e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6f84114330 con 0x7f6f8419a810 2024-04-24T08:56:13.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.853+0000 7f6f89e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6f84114630 con 0x7f6f84104450 2024-04-24T08:56:13.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.854+0000 7f6f82000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f84109520 0x7f6f841a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:13.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.854+0000 7f6f82000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f84109520 0x7f6f841a0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45664/0 (socket says 172.21.15.83:45664) 2024-04-24T08:56:13.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.854+0000 7f6f82000640 1 -- 172.21.15.83:0/3061515715 learned_addr learned my addr 172.21.15.83:0/3061515715 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:56:13.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.854+0000 7f6f82a00640 1 --2- 172.21.15.83:0/3061515715 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6f84104450 0x7f6f841a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:13.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.854+0000 7f6f83400640 1 --2- 172.21.15.83:0/3061515715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6f8419a810 0x7f6f8419acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:13.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.854+0000 7f6f82000640 1 -- 172.21.15.83:0/3061515715 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6f84104450 msgr2=0x7f6f841a06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:13.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.854+0000 7f6f82000640 1 --2- 172.21.15.83:0/3061515715 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6f84104450 0x7f6f841a06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:13.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.854+0000 7f6f82000640 1 -- 172.21.15.83:0/3061515715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6f8419a810 msgr2=0x7f6f8419acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:13.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.854+0000 7f6f82000640 1 --2- 172.21.15.83:0/3061515715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6f8419a810 0x7f6f8419acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:13.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.854+0000 7f6f82000640 1 -- 172.21.15.83:0/3061515715 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f8419b3a0 con 0x7f6f84109520 2024-04-24T08:56:13.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.854+0000 7f6f83400640 1 --2- 172.21.15.83:0/3061515715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6f8419a810 0x7f6f8419acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:56:13.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.855+0000 7f6f82000640 1 --2- 172.21.15.83:0/3061515715 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f84109520 0x7f6f841a0be0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f6f70031440 tx=0x7f6f70003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:13.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.855+0000 7f6f88a00640 1 -- 172.21.15.83:0/3061515715 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f7003f070 con 0x7f6f84109520 2024-04-24T08:56:13.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.855+0000 7f6f88a00640 1 -- 172.21.15.83:0/3061515715 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6f70031d60 con 0x7f6f84109520 2024-04-24T08:56:13.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.855+0000 7f6f89e00640 1 -- 172.21.15.83:0/3061515715 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f8419b630 con 0x7f6f84109520 2024-04-24T08:56:13.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.855+0000 7f6f88a00640 1 -- 172.21.15.83:0/3061515715 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f70035030 con 0x7f6f84109520 2024-04-24T08:56:13.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.855+0000 7f6f89e00640 1 -- 172.21.15.83:0/3061515715 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f841a77e0 con 0x7f6f84109520 2024-04-24T08:56:13.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.859+0000 7f6f88a00640 1 -- 172.21.15.83:0/3061515715 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6f70042070 con 0x7f6f84109520 2024-04-24T08:56:13.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.859+0000 7f6f8cdb8740 1 Processor -- start 2024-04-24T08:56:13.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.859+0000 7f6f8cdb8740 1 -- start start 2024-04-24T08:56:13.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.860+0000 7f6f8cdb8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562880c94950 0x562880c94d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:13.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.860+0000 7f6f8cdb8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562880c95270 0x562880c9f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:13.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.860+0000 7f6f8cdb8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562880c9f940 0x562880ca1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:13.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.860+0000 7f6f8cdb8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562880bfa1d0 con 0x562880c94950 2024-04-24T08:56:13.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.860+0000 7f6f8cdb8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562880ba0450 con 0x562880c95270 2024-04-24T08:56:13.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.860+0000 7f6f8cdb8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x562880bf6190 con 0x562880c9f940 2024-04-24T08:56:13.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.860+0000 7f6f83400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562880c94950 0x562880c94d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:13.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.860+0000 7f6f83400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562880c94950 0x562880c94d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45674/0 (socket says 172.21.15.83:45674) 2024-04-24T08:56:13.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.860+0000 7f6f83400640 1 -- 172.21.15.83:0/4275657500 learned_addr learned my addr 172.21.15.83:0/4275657500 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:56:13.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.861+0000 7f6f82a00640 1 --2- 172.21.15.83:0/4275657500 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562880c95270 0x562880c9f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:13.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.861+0000 7f6f82000640 1 --2- 172.21.15.83:0/4275657500 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562880c9f940 0x562880ca1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:13.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.861+0000 7f6f83400640 1 -- 172.21.15.83:0/4275657500 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562880c9f940 msgr2=0x562880ca1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:13.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.861+0000 7f6f83400640 1 --2- 172.21.15.83:0/4275657500 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562880c9f940 0x562880ca1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:13.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.861+0000 7f6f83400640 1 -- 172.21.15.83:0/4275657500 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562880c95270 msgr2=0x562880c9f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:13.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.861+0000 7f6f83400640 1 --2- 172.21.15.83:0/4275657500 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562880c95270 0x562880c9f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:13.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.861+0000 7f6f83400640 1 -- 172.21.15.83:0/4275657500 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562880c5d310 con 0x562880c94950 2024-04-24T08:56:13.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.861+0000 7f6f82000640 1 --2- 172.21.15.83:0/4275657500 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562880c9f940 0x562880ca1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:56:13.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.861+0000 7f6f88a00640 1 --2- 172.21.15.83:0/3061515715 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f6f680a3d20 0x7f6f680a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:13.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.861+0000 7f6f88a00640 1 -- 172.21.15.83:0/3061515715 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f6f700f1570 con 0x7f6f84109520 2024-04-24T08:56:13.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.861+0000 7f6f82a00640 1 --2- 172.21.15.83:0/3061515715 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f6f680a3d20 0x7f6f680a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:13.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.861+0000 7f6f83400640 1 --2- 172.21.15.83:0/4275657500 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562880c94950 0x562880c94d30 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c008de0 tx=0x7f6f7c00d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58a6bcca72cf9391 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:13.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.862+0000 7f6f6fe00640 1 -- 172.21.15.83:0/4275657500 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f7c00f830 con 0x562880c94950 2024-04-24T08:56:13.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.862+0000 7f6f6fe00640 1 -- 172.21.15.83:0/4275657500 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6f7c00de20 con 0x562880c94950 2024-04-24T08:56:13.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.862+0000 7f6f6fe00640 1 -- 172.21.15.83:0/4275657500 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f7c00ee10 con 0x562880c94950 2024-04-24T08:56:13.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.863+0000 7f6f8cdb8740 1 -- 172.21.15.83:0/4275657500 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562880c94950 msgr2=0x562880c94d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:13.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.863+0000 7f6f82a00640 1 --2- 172.21.15.83:0/3061515715 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f6f680a3d20 0x7f6f680a61e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f6f74005fd0 tx=0x7f6f74005f00 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:13.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.863+0000 7f6f8cdb8740 1 --2- 172.21.15.83:0/4275657500 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562880c94950 0x562880c94d30 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c008de0 tx=0x7f6f7c00d710 comp rx=0 tx=0).stop 2024-04-24T08:56:13.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.863+0000 7f6f8cdb8740 1 -- 172.21.15.83:0/4275657500 shutdown_connections 2024-04-24T08:56:13.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.863+0000 7f6f8cdb8740 1 --2- 172.21.15.83:0/4275657500 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562880c9f940 0x562880ca1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:13.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.863+0000 7f6f8cdb8740 1 --2- 172.21.15.83:0/4275657500 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562880c95270 0x562880c9f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:13.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.863+0000 7f6f8cdb8740 1 --2- 172.21.15.83:0/4275657500 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562880c94950 0x562880c94d30 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:13.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.863+0000 7f6f8cdb8740 1 -- 172.21.15.83:0/4275657500 >> 172.21.15.83:0/4275657500 conn(0x562880c90c70 msgr2=0x562880c91050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:13.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.863+0000 7f6f8cdb8740 1 -- 172.21.15.83:0/4275657500 shutdown_connections 2024-04-24T08:56:13.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.863+0000 7f6f8cdb8740 1 -- 172.21.15.83:0/4275657500 wait complete. 2024-04-24T08:56:13.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.864+0000 7f6f8cdb8740 1 -- start start 2024-04-24T08:56:13.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.864+0000 7f6f8cdb8740 1 Processor -- start 2024-04-24T08:56:13.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.865+0000 7f6f8cdb8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562880c94950 0x562880c9f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:13.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.865+0000 7f6f8cdb8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562880c95270 0x562880c9c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:13.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.865+0000 7f6f8cdb8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562880c9f940 0x562880c9cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:13.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.865+0000 7f6f8cdb8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562880ca27d0 con 0x562880c95270 2024-04-24T08:56:13.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.865+0000 7f6f8cdb8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562880bb3810 con 0x562880c94950 2024-04-24T08:56:13.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.865+0000 7f6f8cdb8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x562880ca2950 con 0x562880c9f940 2024-04-24T08:56:13.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.865+0000 7f6f8cdb8740 10 client.0 _ll_register_callbacks cb 0x562880b81140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:56:13.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.865+0000 7f6f82000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562880c95270 0x562880c9c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:13.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.865+0000 7f6f82000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562880c95270 0x562880c9c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45676/0 (socket says 172.21.15.83:45676) 2024-04-24T08:56:13.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.865+0000 7f6f82000640 1 -- 172.21.15.83:0/3470965611 learned_addr learned my addr 172.21.15.83:0/3470965611 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:56:13.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.865+0000 7f6f83400640 1 --2- 172.21.15.83:0/3470965611 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562880c9f940 0x562880c9cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:13.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.865+0000 7f6f82a00640 1 --2- 172.21.15.83:0/3470965611 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562880c94950 0x562880c9f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:13.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.865+0000 7f6f82000640 1 -- 172.21.15.83:0/3470965611 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562880c9f940 msgr2=0x562880c9cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:13.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.865+0000 7f6f82000640 1 --2- 172.21.15.83:0/3470965611 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562880c9f940 0x562880c9cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:13.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.865+0000 7f6f82000640 1 -- 172.21.15.83:0/3470965611 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562880c94950 msgr2=0x562880c9f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:13.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.865+0000 7f6f82000640 1 --2- 172.21.15.83:0/3470965611 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562880c94950 0x562880c9f3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:13.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.865+0000 7f6f82000640 1 -- 172.21.15.83:0/3470965611 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562880bc43f0 con 0x562880c95270 2024-04-24T08:56:13.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.866+0000 7f6f82000640 1 --2- 172.21.15.83:0/3470965611 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562880c95270 0x562880c9c6d0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f6f700bd0d0 tx=0x7f6f7002a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b48314a741787545 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:13.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.866+0000 7f6f6cc00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:56:13.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.866+0000 7f6f6cc00640 1 -- 172.21.15.83:0/3470965611 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f70049070 con 0x562880c95270 2024-04-24T08:56:13.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.866+0000 7f6f8cdb8740 1 -- 172.21.15.83:0/3470965611 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f7c00de20 con 0x562880c95270 2024-04-24T08:56:13.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.866+0000 7f6f6cc00640 1 -- 172.21.15.83:0/3470965611 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f6f700b97b0 con 0x562880c95270 2024-04-24T08:56:13.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.866+0000 7f6f6cc00640 1 -- 172.21.15.83:0/3470965611 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6f700bd840 con 0x562880c95270 2024-04-24T08:56:13.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.866+0000 7f6f8cdb8740 10 client.17289 fetch_fsmap learned FSMap version 32 2024-04-24T08:56:13.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.866+0000 7f6f8cdb8740 1 -- 172.21.15.83:0/3470965611 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56288090c3f0 con 0x562880c95270 2024-04-24T08:56:13.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.866+0000 7f6f6cc00640 1 -- 172.21.15.83:0/3470965611 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6f7c00de20 con 0x562880c95270 2024-04-24T08:56:13.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.867+0000 7f6f6cc00640 1 -- 172.21.15.83:0/3470965611 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7f6f700c1160 con 0x562880c95270 2024-04-24T08:56:13.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.867+0000 7f6f8cdb8740 10 client.17289 fetch_fsmap finished waiting for FSMap version 32 2024-04-24T08:56:13.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.867+0000 7f6f8cdb8740 10 client.17289 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-24T08:56:13.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.867+0000 7f6f8cdb8740 20 client.17289 populate_metadata read hostname 'smithi083' 2024-04-24T08:56:13.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.867+0000 7f6f8cdb8740 1 --2- 172.21.15.83:0/3470965611 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x562880caf9e0 0x562880cb1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:13.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.867+0000 7f6f8cdb8740 4 client.17289 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:56:13.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.867+0000 7f6f8cdb8740 1 -- 172.21.15.83:0/3470965611 --> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562880c07bf0 con 0x562880caf9e0 2024-04-24T08:56:13.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.868+0000 7f6f82a00640 1 --2- 172.21.15.83:0/3470965611 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x562880caf9e0 0x562880cb1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:13.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.869+0000 7f6f82a00640 1 --2- 172.21.15.83:0/3470965611 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x562880caf9e0 0x562880cb1dd0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=db336506a9913a5c server_cookie=9ad47c087ce69e51 in_seq=0 out_seq=0 2024-04-24T08:56:13.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.869+0000 7f6f6cc00640 10 client.17289 ms_handle_connect on v2:172.21.15.189:6838/2548965336 2024-04-24T08:56:13.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.870+0000 7f6f6cc00640 1 -- 172.21.15.83:0/3470965611 <== mds.4 v2:172.21.15.189:6838/2548965336 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562880c07bf0 con 0x562880caf9e0 2024-04-24T08:56:13.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.870+0000 7f6f6cc00640 10 client.17289 handle_command_reply: tid=0 2024-04-24T08:56:13.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.870+0000 7f6f8cdb8740 1 client.17289 shutdown 2024-04-24T08:56:13.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.871+0000 7f6f8cdb8740 1 -- 172.21.15.83:0/3470965611 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562880c95270 msgr2=0x562880c9c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:13.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.871+0000 7f6f8cdb8740 1 --2- 172.21.15.83:0/3470965611 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562880c95270 0x562880c9c6d0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f6f700bd0d0 tx=0x7f6f7002a3f0 comp rx=0 tx=0).stop 2024-04-24T08:56:13.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.871+0000 7f6f8cdb8740 1 -- 172.21.15.83:0/3470965611 shutdown_connections 2024-04-24T08:56:13.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.871+0000 7f6f8cdb8740 1 --2- 172.21.15.83:0/3470965611 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x562880caf9e0 0x562880cb1dd0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:13.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.871+0000 7f6f8cdb8740 1 --2- 172.21.15.83:0/3470965611 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562880c9f940 0x562880c9cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:13.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.871+0000 7f6f8cdb8740 1 --2- 172.21.15.83:0/3470965611 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562880c95270 0x562880c9c6d0 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:13.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.871+0000 7f6f8cdb8740 1 --2- 172.21.15.83:0/3470965611 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562880c94950 0x562880c9f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:13.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.871+0000 7f6f6cc00640 0 client.17289 ms_handle_reset on v2:172.21.15.189:6838/2548965336 2024-04-24T08:56:13.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.871+0000 7f6f8cdb8740 1 -- 172.21.15.83:0/3470965611 >> 172.21.15.83:0/3470965611 conn(0x562880c90c70 msgr2=0x562880c96a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:13.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.871+0000 7f6f8cdb8740 1 -- 172.21.15.83:0/3470965611 shutdown_connections 2024-04-24T08:56:13.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.871+0000 7f6f8cdb8740 1 -- 172.21.15.83:0/3470965611 wait complete. 2024-04-24T08:56:13.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.871+0000 7f6f8cdb8740 20 client.17289 trim_cache size 0 max 16384 2024-04-24T08:56:13.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.886+0000 7f6f8cdb8740 1 Processor -- start 2024-04-24T08:56:13.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.886+0000 7f6f8cdb8740 1 -- start start 2024-04-24T08:56:13.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.886+0000 7f6f8cdb8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562880c94950 0x562880cab690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:13.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.886+0000 7f6f8cdb8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562880c95270 0x562880c9ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:13.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.886+0000 7f6f8cdb8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562880c9f940 0x562880cb7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:13.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.886+0000 7f6f8cdb8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6f841b16d0 con 0x562880c9f940 2024-04-24T08:56:13.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.886+0000 7f6f8cdb8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6f841b3240 con 0x562880c95270 2024-04-24T08:56:13.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.886+0000 7f6f8cdb8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6f841b1da0 con 0x562880c94950 2024-04-24T08:56:13.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.887+0000 7f6f83400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562880c9f940 0x562880cb7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:13.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.887+0000 7f6f83400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562880c9f940 0x562880cb7650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45682/0 (socket says 172.21.15.83:45682) 2024-04-24T08:56:13.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.887+0000 7f6f83400640 1 -- 172.21.15.83:0/3351619393 learned_addr learned my addr 172.21.15.83:0/3351619393 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:56:13.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.887+0000 7f6f82000640 1 --2- 172.21.15.83:0/3351619393 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562880c95270 0x562880c9ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:13.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.887+0000 7f6f82a00640 1 --2- 172.21.15.83:0/3351619393 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562880c94950 0x562880cab690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:13.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.887+0000 7f6f83400640 1 -- 172.21.15.83:0/3351619393 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562880c94950 msgr2=0x562880cab690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:13.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.887+0000 7f6f83400640 1 --2- 172.21.15.83:0/3351619393 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562880c94950 0x562880cab690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:13.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.887+0000 7f6f83400640 1 -- 172.21.15.83:0/3351619393 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562880c95270 msgr2=0x562880c9ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:13.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.887+0000 7f6f83400640 1 --2- 172.21.15.83:0/3351619393 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562880c95270 0x562880c9ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:13.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.887+0000 7f6f83400640 1 -- 172.21.15.83:0/3351619393 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562880c9d130 con 0x562880c9f940 2024-04-24T08:56:13.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.887+0000 7f6f82000640 1 --2- 172.21.15.83:0/3351619393 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562880c95270 0x562880c9ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:56:13.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.887+0000 7f6f83400640 1 --2- 172.21.15.83:0/3351619393 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562880c9f940 0x562880cb7650 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c017000 tx=0x7f6f7c008a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d3ddd631c2e5619 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:13.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.888+0000 7f6f6f400640 1 -- 172.21.15.83:0/3351619393 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f7c004d00 con 0x562880c9f940 2024-04-24T08:56:13.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.888+0000 7f6f6f400640 1 -- 172.21.15.83:0/3351619393 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6f7c00fae0 con 0x562880c9f940 2024-04-24T08:56:13.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.888+0000 7f6f6f400640 1 -- 172.21.15.83:0/3351619393 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f7c0056a0 con 0x562880c9f940 2024-04-24T08:56:13.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.888+0000 7f6f8cdb8740 1 -- 172.21.15.83:0/3351619393 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562880c9f940 msgr2=0x562880cb7650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:13.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.888+0000 7f6f8cdb8740 1 --2- 172.21.15.83:0/3351619393 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562880c9f940 0x562880cb7650 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c017000 tx=0x7f6f7c008a60 comp rx=0 tx=0).stop 2024-04-24T08:56:13.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.888+0000 7f6f8cdb8740 1 -- 172.21.15.83:0/3351619393 shutdown_connections 2024-04-24T08:56:13.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.888+0000 7f6f8cdb8740 1 --2- 172.21.15.83:0/3351619393 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562880c9f940 0x562880cb7650 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:13.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.888+0000 7f6f8cdb8740 1 --2- 172.21.15.83:0/3351619393 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562880c95270 0x562880c9ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:13.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.888+0000 7f6f8cdb8740 1 --2- 172.21.15.83:0/3351619393 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562880c94950 0x562880cab690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:13.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.889+0000 7f6f8cdb8740 1 -- 172.21.15.83:0/3351619393 >> 172.21.15.83:0/3351619393 conn(0x562880c90c70 msgr2=0x562880c960e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:13.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.889+0000 7f6f8cdb8740 1 -- 172.21.15.83:0/3351619393 shutdown_connections 2024-04-24T08:56:13.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.889+0000 7f6f8cdb8740 1 -- 172.21.15.83:0/3351619393 wait complete. 2024-04-24T08:56:13.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.889+0000 7f6f8cdb8740 1 -- start start 2024-04-24T08:56:13.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.889+0000 7f6f8cdb8740 1 Processor -- start 2024-04-24T08:56:13.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.890+0000 7f6f8cdb8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562880c94950 0x562880ca0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:13.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.890+0000 7f6f8cdb8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562880c95270 0x562880ca1390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:13.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.890+0000 7f6f8cdb8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562880c9f940 0x562880ca62a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:13.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.890+0000 7f6f8cdb8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6f841b2b40 con 0x562880c94950 2024-04-24T08:56:13.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.890+0000 7f6f8cdb8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6f841b2470 con 0x562880c9f940 2024-04-24T08:56:13.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.890+0000 7f6f8cdb8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6f841af6b0 con 0x562880c95270 2024-04-24T08:56:13.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.890+0000 7f6f8cdb8740 10 client.0 _ll_register_callbacks cb 0x7f6f70009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:56:13.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.890+0000 7f6f82a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562880c94950 0x562880ca0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:13.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.890+0000 7f6f82a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562880c94950 0x562880ca0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45690/0 (socket says 172.21.15.83:45690) 2024-04-24T08:56:13.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.890+0000 7f6f82a00640 1 -- 172.21.15.83:0/3725009126 learned_addr learned my addr 172.21.15.83:0/3725009126 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:56:13.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.890+0000 7f6f83400640 1 --2- 172.21.15.83:0/3725009126 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562880c9f940 0x562880ca62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:13.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.891+0000 7f6f82a00640 1 -- 172.21.15.83:0/3725009126 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562880c95270 msgr2=0x562880ca1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:13.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.891+0000 7f6f82000640 1 --2- 172.21.15.83:0/3725009126 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562880c95270 0x562880ca1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:13.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.891+0000 7f6f82a00640 1 --2- 172.21.15.83:0/3725009126 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562880c95270 0x562880ca1390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:13.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.891+0000 7f6f82a00640 1 -- 172.21.15.83:0/3725009126 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562880c9f940 msgr2=0x562880ca62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:13.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.891+0000 7f6f82a00640 1 --2- 172.21.15.83:0/3725009126 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562880c9f940 0x562880ca62a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:13.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.891+0000 7f6f82a00640 1 -- 172.21.15.83:0/3725009126 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628809d67d0 con 0x562880c94950 2024-04-24T08:56:13.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.891+0000 7f6f83400640 1 --2- 172.21.15.83:0/3725009126 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562880c9f940 0x562880ca62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:56:13.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.891+0000 7f6f82a00640 1 --2- 172.21.15.83:0/3725009126 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562880c94950 0x562880ca0e50 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x562880cb25c0 tx=0x7f6f740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=227a248c78031cc7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:13.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.891+0000 7f6f6cc00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:56:13.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.891+0000 7f6f6cc00640 1 -- 172.21.15.83:0/3725009126 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f74011d00 con 0x562880c94950 2024-04-24T08:56:13.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.891+0000 7f6f6cc00640 1 -- 172.21.15.83:0/3725009126 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f6f74004020 con 0x562880c94950 2024-04-24T08:56:13.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.891+0000 7f6f8cdb8740 1 -- 172.21.15.83:0/3725009126 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f7c00fae0 con 0x562880c94950 2024-04-24T08:56:13.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.892+0000 7f6f6cc00640 1 -- 172.21.15.83:0/3725009126 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6f74037050 con 0x562880c94950 2024-04-24T08:56:13.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.892+0000 7f6f8cdb8740 10 client.17295 fetch_fsmap learned FSMap version 32 2024-04-24T08:56:13.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.892+0000 7f6f8cdb8740 1 -- 172.21.15.83:0/3725009126 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562880d164d0 con 0x562880c94950 2024-04-24T08:56:13.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.892+0000 7f6f6cc00640 1 -- 172.21.15.83:0/3725009126 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6f7c00fae0 con 0x562880c94950 2024-04-24T08:56:13.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.892+0000 7f6f6cc00640 1 -- 172.21.15.83:0/3725009126 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7f6f74003710 con 0x562880c94950 2024-04-24T08:56:13.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.892+0000 7f6f8cdb8740 10 client.17295 fetch_fsmap finished waiting for FSMap version 32 2024-04-24T08:56:13.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.892+0000 7f6f8cdb8740 10 client.17295 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-24T08:56:13.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.892+0000 7f6f8cdb8740 20 client.17295 populate_metadata read hostname 'smithi083' 2024-04-24T08:56:13.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.892+0000 7f6f8cdb8740 1 --2- 172.21.15.83:0/3725009126 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x562880d16c00 0x562880d19020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:13.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.892+0000 7f6f8cdb8740 4 client.17295 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T08:56:13.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.892+0000 7f6f8cdb8740 1 -- 172.21.15.83:0/3725009126 --> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562880c01a60 con 0x562880d16c00 2024-04-24T08:56:13.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.893+0000 7f6f82000640 1 --2- 172.21.15.83:0/3725009126 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x562880d16c00 0x562880d19020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:13.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.894+0000 7f6f82000640 1 --2- 172.21.15.83:0/3725009126 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x562880d16c00 0x562880d19020 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4156578911c1b238 server_cookie=e306c00c143d627a in_seq=0 out_seq=0 2024-04-24T08:56:13.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.894+0000 7f6f6cc00640 10 client.17295 ms_handle_connect on v2:172.21.15.189:6838/2548965336 2024-04-24T08:56:13.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.894+0000 7f6f6cc00640 1 -- 172.21.15.83:0/3725009126 <== mds.4 v2:172.21.15.189:6838/2548965336 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562880c01a60 con 0x562880d16c00 2024-04-24T08:56:13.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.894+0000 7f6f6cc00640 10 client.17295 handle_command_reply: tid=0 2024-04-24T08:56:13.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.894+0000 7f6f8cdb8740 1 client.17295 shutdown 2024-04-24T08:56:13.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.894+0000 7f6f8cdb8740 1 -- 172.21.15.83:0/3725009126 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562880c94950 msgr2=0x562880ca0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:13.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.894+0000 7f6f8cdb8740 1 --2- 172.21.15.83:0/3725009126 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562880c94950 0x562880ca0e50 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x562880cb25c0 tx=0x7f6f740046c0 comp rx=0 tx=0).stop 2024-04-24T08:56:13.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.894+0000 7f6f8cdb8740 1 -- 172.21.15.83:0/3725009126 shutdown_connections 2024-04-24T08:56:13.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.894+0000 7f6f8cdb8740 1 --2- 172.21.15.83:0/3725009126 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x562880d16c00 0x562880d19020 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:13.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.894+0000 7f6f8cdb8740 1 --2- 172.21.15.83:0/3725009126 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562880c9f940 0x562880ca62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:13.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.894+0000 7f6f8cdb8740 1 --2- 172.21.15.83:0/3725009126 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562880c95270 0x562880ca1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:13.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.894+0000 7f6f6cc00640 0 client.17295 ms_handle_reset on v2:172.21.15.189:6838/2548965336 2024-04-24T08:56:13.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.894+0000 7f6f8cdb8740 1 --2- 172.21.15.83:0/3725009126 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562880c94950 0x562880ca0e50 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:13.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.894+0000 7f6f8cdb8740 1 -- 172.21.15.83:0/3725009126 >> 172.21.15.83:0/3725009126 conn(0x562880c90c70 msgr2=0x562880caaf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:13.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.895+0000 7f6f8cdb8740 1 -- 172.21.15.83:0/3725009126 shutdown_connections 2024-04-24T08:56:13.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.895+0000 7f6f8cdb8740 1 -- 172.21.15.83:0/3725009126 wait complete. 2024-04-24T08:56:13.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.895+0000 7f6f8cdb8740 20 client.17295 trim_cache size 0 max 16384 2024-04-24T08:56:13.898 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T08:56:13.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.895+0000 7f6f89e00640 1 -- 172.21.15.83:0/3061515715 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f6f680a3d20 msgr2=0x7f6f680a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:13.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.895+0000 7f6f89e00640 1 --2- 172.21.15.83:0/3061515715 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f6f680a3d20 0x7f6f680a61e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f6f74005fd0 tx=0x7f6f74005f00 comp rx=0 tx=0).stop 2024-04-24T08:56:13.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.895+0000 7f6f89e00640 1 -- 172.21.15.83:0/3061515715 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f84109520 msgr2=0x7f6f841a0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:13.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.895+0000 7f6f89e00640 1 --2- 172.21.15.83:0/3061515715 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f84109520 0x7f6f841a0be0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f6f70031440 tx=0x7f6f70003d50 comp rx=0 tx=0).stop 2024-04-24T08:56:13.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.896+0000 7f6f89e00640 1 -- 172.21.15.83:0/3061515715 shutdown_connections 2024-04-24T08:56:13.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.896+0000 7f6f89e00640 1 --2- 172.21.15.83:0/3061515715 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f6f680a3d20 0x7f6f680a61e0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:13.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.896+0000 7f6f89e00640 1 --2- 172.21.15.83:0/3061515715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6f8419a810 0x7f6f8419acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:13.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.896+0000 7f6f89e00640 1 --2- 172.21.15.83:0/3061515715 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f84109520 0x7f6f841a0be0 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:13.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.896+0000 7f6f89e00640 1 --2- 172.21.15.83:0/3061515715 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6f84104450 0x7f6f841a06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:13.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.896+0000 7f6f89e00640 1 -- 172.21.15.83:0/3061515715 >> 172.21.15.83:0/3061515715 conn(0x7f6f840ffc70 msgr2=0x7f6f84100f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:13.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.896+0000 7f6f89e00640 1 -- 172.21.15.83:0/3061515715 shutdown_connections 2024-04-24T08:56:13.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:13.896+0000 7f6f89e00640 1 -- 172.21.15.83:0/3061515715 wait complete. 2024-04-24T08:56:13.911 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:56:13 smithi083 ceph-mon[27907]: pgmap v141: 129 pgs: 129 active+clean; 37 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 451 KiB/s wr, 84 op/s 2024-04-24T08:56:13.914 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-24T08:56:13.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:56:13 smithi138 ceph-mon[31441]: pgmap v141: 129 pgs: 129 active+clean; 37 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 451 KiB/s wr, 84 op/s 2024-04-24T08:56:13.956 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:56:13 smithi189 ceph-mon[31791]: pgmap v141: 129 pgs: 129 active+clean; 37 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 451 KiB/s wr, 84 op/s 2024-04-24T08:56:14.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.126+0000 7f857f200640 1 -- 172.21.15.83:0/3361728146 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f858010d500 msgr2=0x7f858010d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:14.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.126+0000 7f857f200640 1 --2- 172.21.15.83:0/3361728146 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f858010d500 0x7f858010d8e0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f8570009930 tx=0x7f857002a420 comp rx=0 tx=0).stop 2024-04-24T08:56:14.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.126+0000 7f857f200640 1 -- 172.21.15.83:0/3361728146 shutdown_connections 2024-04-24T08:56:14.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.126+0000 7f857f200640 1 --2- 172.21.15.83:0/3361728146 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8580108f40 0x7f85801153b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.126+0000 7f857f200640 1 --2- 172.21.15.83:0/3361728146 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8580108580 0x7f8580108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.126+0000 7f857f200640 1 --2- 172.21.15.83:0/3361728146 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f858010d500 0x7f858010d8e0 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.126+0000 7f857f200640 1 -- 172.21.15.83:0/3361728146 >> 172.21.15.83:0/3361728146 conn(0x7f8580077e20 msgr2=0x7f8580106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:14.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.126+0000 7f857f200640 1 -- 172.21.15.83:0/3361728146 shutdown_connections 2024-04-24T08:56:14.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.126+0000 7f857f200640 1 -- 172.21.15.83:0/3361728146 wait complete. 2024-04-24T08:56:14.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.127+0000 7f857f200640 1 Processor -- start 2024-04-24T08:56:14.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.127+0000 7f857f200640 1 -- start start 2024-04-24T08:56:14.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.128+0000 7f857f200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8580108580 0x7f85801a4910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:14.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.128+0000 7f857f200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8580108f40 0x7f85801a4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:14.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.128+0000 7f857f200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f858010d500 0x7f858019eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:14.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.128+0000 7f857f200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f85801187c0 con 0x7f8580108f40 2024-04-24T08:56:14.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.128+0000 7f857f200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8580118640 con 0x7f8580108580 2024-04-24T08:56:14.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.128+0000 7f857f200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8580118940 con 0x7f858010d500 2024-04-24T08:56:14.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.128+0000 7f857d400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8580108f40 0x7f85801a4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:14.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.128+0000 7f857d400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8580108f40 0x7f85801a4e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45704/0 (socket says 172.21.15.83:45704) 2024-04-24T08:56:14.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.128+0000 7f857d400640 1 -- 172.21.15.83:0/3390411020 learned_addr learned my addr 172.21.15.83:0/3390411020 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:56:14.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.128+0000 7f857de00640 1 --2- 172.21.15.83:0/3390411020 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8580108580 0x7f85801a4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:14.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.128+0000 7f857e800640 1 --2- 172.21.15.83:0/3390411020 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f858010d500 0x7f858019eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:14.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.129+0000 7f857d400640 1 -- 172.21.15.83:0/3390411020 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f858010d500 msgr2=0x7f858019eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:14.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.129+0000 7f857d400640 1 --2- 172.21.15.83:0/3390411020 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f858010d500 0x7f858019eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.129+0000 7f857d400640 1 -- 172.21.15.83:0/3390411020 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8580108580 msgr2=0x7f85801a4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:14.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.129+0000 7f857d400640 1 --2- 172.21.15.83:0/3390411020 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8580108580 0x7f85801a4910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.129+0000 7f857d400640 1 -- 172.21.15.83:0/3390411020 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f858019f240 con 0x7f8580108f40 2024-04-24T08:56:14.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.129+0000 7f857d400640 1 --2- 172.21.15.83:0/3390411020 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8580108f40 0x7f85801a4e50 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f856c00d950 tx=0x7f856c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:14.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.129+0000 7f8576a00640 1 -- 172.21.15.83:0/3390411020 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f856c017070 con 0x7f8580108f40 2024-04-24T08:56:14.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.129+0000 7f8576a00640 1 -- 172.21.15.83:0/3390411020 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f856c009c30 con 0x7f8580108f40 2024-04-24T08:56:14.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.129+0000 7f857f200640 1 -- 172.21.15.83:0/3390411020 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f858019f530 con 0x7f8580108f40 2024-04-24T08:56:14.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.129+0000 7f8576a00640 1 -- 172.21.15.83:0/3390411020 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f856c00f620 con 0x7f8580108f40 2024-04-24T08:56:14.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.129+0000 7f857f200640 1 -- 172.21.15.83:0/3390411020 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85801ab7d0 con 0x7f8580108f40 2024-04-24T08:56:14.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.133+0000 7f8576a00640 1 -- 172.21.15.83:0/3390411020 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f856c0109a0 con 0x7f8580108f40 2024-04-24T08:56:14.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.133+0000 7f858802e740 1 Processor -- start 2024-04-24T08:56:14.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.133+0000 7f858802e740 1 -- start start 2024-04-24T08:56:14.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.134+0000 7f858802e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f64b9bb40 0x555f64b9bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:14.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.134+0000 7f858802e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555f64b9c460 0x555f64ba65f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:14.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.134+0000 7f858802e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f64ba6b30 0x555f64ba8f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:14.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.134+0000 7f858802e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x555f64b011d0 con 0x555f64b9bb40 2024-04-24T08:56:14.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.134+0000 7f858802e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555f64aa7450 con 0x555f64ba6b30 2024-04-24T08:56:14.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.134+0000 7f858802e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x555f64afd190 con 0x555f64b9c460 2024-04-24T08:56:14.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.134+0000 7f857e800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f64b9bb40 0x555f64b9bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:14.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.134+0000 7f857e800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f64b9bb40 0x555f64b9bf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45718/0 (socket says 172.21.15.83:45718) 2024-04-24T08:56:14.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.134+0000 7f857e800640 1 -- 172.21.15.83:0/4161351632 learned_addr learned my addr 172.21.15.83:0/4161351632 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:56:14.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.134+0000 7f857d400640 1 --2- 172.21.15.83:0/4161351632 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f64ba6b30 0x555f64ba8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:14.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.134+0000 7f857de00640 1 --2- 172.21.15.83:0/4161351632 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555f64b9c460 0x555f64ba65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:14.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.134+0000 7f857e800640 1 -- 172.21.15.83:0/4161351632 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555f64b9c460 msgr2=0x555f64ba65f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:14.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.134+0000 7f857e800640 1 --2- 172.21.15.83:0/4161351632 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555f64b9c460 0x555f64ba65f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.134+0000 7f857e800640 1 -- 172.21.15.83:0/4161351632 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f64ba6b30 msgr2=0x555f64ba8f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:14.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.134+0000 7f857e800640 1 --2- 172.21.15.83:0/4161351632 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f64ba6b30 0x555f64ba8f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.134+0000 7f857e800640 1 -- 172.21.15.83:0/4161351632 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f64b64310 con 0x555f64b9bb40 2024-04-24T08:56:14.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.134+0000 7f857d400640 1 --2- 172.21.15.83:0/4161351632 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f64ba6b30 0x555f64ba8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:56:14.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.135+0000 7f8576a00640 1 --2- 172.21.15.83:0/3390411020 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f85640a3de0 0x7f85640a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:14.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.135+0000 7f857e800640 1 --2- 172.21.15.83:0/4161351632 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f64b9bb40 0x555f64b9bf20 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f857800ae30 tx=0x7f857800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffa4ac1829ae20f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:14.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.135+0000 7f8576a00640 1 -- 172.21.15.83:0/3390411020 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f856c0d08a0 con 0x7f8580108f40 2024-04-24T08:56:14.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.135+0000 7f857de00640 1 --2- 172.21.15.83:0/3390411020 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f85640a3de0 0x7f85640a62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:14.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.135+0000 7f856b400640 1 -- 172.21.15.83:0/4161351632 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8578003b70 con 0x555f64b9bb40 2024-04-24T08:56:14.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.135+0000 7f856b400640 1 -- 172.21.15.83:0/4161351632 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8578003d10 con 0x555f64b9bb40 2024-04-24T08:56:14.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.135+0000 7f856b400640 1 -- 172.21.15.83:0/4161351632 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8578010ae0 con 0x555f64b9bb40 2024-04-24T08:56:14.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.136+0000 7f858802e740 1 -- 172.21.15.83:0/4161351632 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f64b9bb40 msgr2=0x555f64b9bf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:14.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.136+0000 7f857de00640 1 --2- 172.21.15.83:0/3390411020 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f85640a3de0 0x7f85640a62a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8570009900 tx=0x7f8570000bc0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:14.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.136+0000 7f858802e740 1 --2- 172.21.15.83:0/4161351632 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f64b9bb40 0x555f64b9bf20 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f857800ae30 tx=0x7f857800e720 comp rx=0 tx=0).stop 2024-04-24T08:56:14.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.136+0000 7f858802e740 1 -- 172.21.15.83:0/4161351632 shutdown_connections 2024-04-24T08:56:14.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.136+0000 7f858802e740 1 --2- 172.21.15.83:0/4161351632 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f64ba6b30 0x555f64ba8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.136+0000 7f858802e740 1 --2- 172.21.15.83:0/4161351632 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555f64b9c460 0x555f64ba65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.136+0000 7f858802e740 1 --2- 172.21.15.83:0/4161351632 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f64b9bb40 0x555f64b9bf20 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.136+0000 7f858802e740 1 -- 172.21.15.83:0/4161351632 >> 172.21.15.83:0/4161351632 conn(0x555f64b97e60 msgr2=0x555f64b98240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:14.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.136+0000 7f858802e740 1 -- 172.21.15.83:0/4161351632 shutdown_connections 2024-04-24T08:56:14.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.136+0000 7f858802e740 1 -- 172.21.15.83:0/4161351632 wait complete. 2024-04-24T08:56:14.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.137+0000 7f858802e740 1 -- start start 2024-04-24T08:56:14.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.137+0000 7f858802e740 1 Processor -- start 2024-04-24T08:56:14.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.137+0000 7f858802e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f64b9bb40 0x555f64ba65e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:14.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.137+0000 7f858802e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555f64b9c460 0x555f64ba38d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:14.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.137+0000 7f858802e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f64ba6b30 0x555f64ba4010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:14.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.137+0000 7f858802e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x555f64ba99c0 con 0x555f64ba6b30 2024-04-24T08:56:14.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.137+0000 7f858802e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555f64aba810 con 0x555f64b9bb40 2024-04-24T08:56:14.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.137+0000 7f858802e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x555f64ba9b40 con 0x555f64b9c460 2024-04-24T08:56:14.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.138+0000 7f858802e740 10 client.0 _ll_register_callbacks cb 0x555f64a88140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:56:14.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.138+0000 7f857e800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f64ba6b30 0x555f64ba4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:14.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.138+0000 7f857e800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f64ba6b30 0x555f64ba4010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45726/0 (socket says 172.21.15.83:45726) 2024-04-24T08:56:14.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.138+0000 7f857e800640 1 -- 172.21.15.83:0/3381217702 learned_addr learned my addr 172.21.15.83:0/3381217702 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:56:14.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.138+0000 7f857de00640 1 --2- 172.21.15.83:0/3381217702 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f64b9bb40 0x555f64ba65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:14.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.138+0000 7f857e800640 1 -- 172.21.15.83:0/3381217702 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555f64b9c460 msgr2=0x555f64ba38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:14.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.138+0000 7f857e800640 1 --2- 172.21.15.83:0/3381217702 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555f64b9c460 0x555f64ba38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.138+0000 7f857e800640 1 -- 172.21.15.83:0/3381217702 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f64b9bb40 msgr2=0x555f64ba65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:14.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.138+0000 7f857e800640 1 --2- 172.21.15.83:0/3381217702 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f64b9bb40 0x555f64ba65e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.138+0000 7f857e800640 1 -- 172.21.15.83:0/3381217702 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f64acb3f0 con 0x555f64ba6b30 2024-04-24T08:56:14.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.138+0000 7f857e800640 1 --2- 172.21.15.83:0/3381217702 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f64ba6b30 0x555f64ba4010 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f857800ab70 tx=0x7f8578004840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7d1e7f2ec62252b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:14.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.138+0000 7f855fe00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:56:14.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.139+0000 7f855fe00640 1 -- 172.21.15.83:0/3381217702 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8578004d10 con 0x555f64ba6b30 2024-04-24T08:56:14.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.139+0000 7f855fe00640 1 -- 172.21.15.83:0/3381217702 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f8578004040 con 0x555f64ba6b30 2024-04-24T08:56:14.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.139+0000 7f858802e740 1 -- 172.21.15.83:0/3381217702 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8578003d10 con 0x555f64ba6b30 2024-04-24T08:56:14.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.139+0000 7f855fe00640 1 -- 172.21.15.83:0/3381217702 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8578004240 con 0x555f64ba6b30 2024-04-24T08:56:14.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.139+0000 7f858802e740 10 client.17307 fetch_fsmap learned FSMap version 32 2024-04-24T08:56:14.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.139+0000 7f858802e740 1 -- 172.21.15.83:0/3381217702 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x555f648133f0 con 0x555f64ba6b30 2024-04-24T08:56:14.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.139+0000 7f855fe00640 1 -- 172.21.15.83:0/3381217702 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8578003d10 con 0x555f64ba6b30 2024-04-24T08:56:14.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.140+0000 7f855fe00640 1 -- 172.21.15.83:0/3381217702 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7f85780052f0 con 0x555f64ba6b30 2024-04-24T08:56:14.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.140+0000 7f858802e740 10 client.17307 fetch_fsmap finished waiting for FSMap version 32 2024-04-24T08:56:14.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.140+0000 7f858802e740 10 client.17307 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-24T08:56:14.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.140+0000 7f858802e740 20 client.17307 populate_metadata read hostname 'smithi083' 2024-04-24T08:56:14.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.140+0000 7f858802e740 1 --2- 172.21.15.83:0/3381217702 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x555f64bb6c60 0x555f64bb9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:14.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.140+0000 7f858802e740 4 client.17307 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:56:14.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.140+0000 7f858802e740 1 -- 172.21.15.83:0/3381217702 --> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555f64b0ebf0 con 0x555f64bb6c60 2024-04-24T08:56:14.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.140+0000 7f857de00640 1 --2- 172.21.15.83:0/3381217702 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x555f64bb6c60 0x555f64bb9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:14.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.142+0000 7f857de00640 1 --2- 172.21.15.83:0/3381217702 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x555f64bb6c60 0x555f64bb9090 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=129ed158a70921d5 server_cookie=33d40d6afe2a5a64 in_seq=0 out_seq=0 2024-04-24T08:56:14.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.142+0000 7f855fe00640 10 client.17307 ms_handle_connect on v2:172.21.15.138:6832/1157967658 2024-04-24T08:56:14.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.144+0000 7f855fe00640 1 -- 172.21.15.83:0/3381217702 <== mds.3 v2:172.21.15.138:6832/1157967658 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555f64b0ebf0 con 0x555f64bb6c60 2024-04-24T08:56:14.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.144+0000 7f855fe00640 10 client.17307 handle_command_reply: tid=0 2024-04-24T08:56:14.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.144+0000 7f858802e740 1 client.17307 shutdown 2024-04-24T08:56:14.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.144+0000 7f858802e740 1 -- 172.21.15.83:0/3381217702 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f64ba6b30 msgr2=0x555f64ba4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:14.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.144+0000 7f858802e740 1 --2- 172.21.15.83:0/3381217702 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f64ba6b30 0x555f64ba4010 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f857800ab70 tx=0x7f8578004840 comp rx=0 tx=0).stop 2024-04-24T08:56:14.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.144+0000 7f858802e740 1 -- 172.21.15.83:0/3381217702 shutdown_connections 2024-04-24T08:56:14.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.144+0000 7f858802e740 1 --2- 172.21.15.83:0/3381217702 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f64ba6b30 0x555f64ba4010 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.144+0000 7f858802e740 1 --2- 172.21.15.83:0/3381217702 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x555f64bb6c60 0x555f64bb9090 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.144+0000 7f858802e740 1 --2- 172.21.15.83:0/3381217702 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555f64b9c460 0x555f64ba38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.144+0000 7f858802e740 1 --2- 172.21.15.83:0/3381217702 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f64b9bb40 0x555f64ba65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.144+0000 7f855fe00640 0 client.17307 ms_handle_reset on v2:172.21.15.138:6832/1157967658 2024-04-24T08:56:14.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.144+0000 7f858802e740 1 -- 172.21.15.83:0/3381217702 >> 172.21.15.83:0/3381217702 conn(0x555f64b97e60 msgr2=0x555f64b9dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:14.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.145+0000 7f858802e740 1 -- 172.21.15.83:0/3381217702 shutdown_connections 2024-04-24T08:56:14.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.145+0000 7f858802e740 1 -- 172.21.15.83:0/3381217702 wait complete. 2024-04-24T08:56:14.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.145+0000 7f858802e740 20 client.17307 trim_cache size 0 max 16384 2024-04-24T08:56:14.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.161+0000 7f858802e740 1 Processor -- start 2024-04-24T08:56:14.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.161+0000 7f858802e740 1 -- start start 2024-04-24T08:56:14.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.161+0000 7f858802e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f64b9bb40 0x555f64bb2880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:14.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.161+0000 7f858802e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555f64b9c460 0x555f64ba6110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:14.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.161+0000 7f858802e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f64ba6b30 0x555f64bbe960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:14.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.161+0000 7f858802e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f85801b5900 con 0x555f64b9bb40 2024-04-24T08:56:14.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.161+0000 7f858802e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f85801b7470 con 0x555f64ba6b30 2024-04-24T08:56:14.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.161+0000 7f858802e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f85801b5fd0 con 0x555f64b9c460 2024-04-24T08:56:14.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.161+0000 7f857de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f64b9bb40 0x555f64bb2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:14.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.162+0000 7f857de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f64b9bb40 0x555f64bb2880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45740/0 (socket says 172.21.15.83:45740) 2024-04-24T08:56:14.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.162+0000 7f857de00640 1 -- 172.21.15.83:0/1257054312 learned_addr learned my addr 172.21.15.83:0/1257054312 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:56:14.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.162+0000 7f857d400640 1 --2- 172.21.15.83:0/1257054312 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555f64b9c460 0x555f64ba6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:14.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.162+0000 7f857de00640 1 -- 172.21.15.83:0/1257054312 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555f64b9c460 msgr2=0x555f64ba6110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:14.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.162+0000 7f857e800640 1 --2- 172.21.15.83:0/1257054312 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f64ba6b30 0x555f64bbe960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:14.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.162+0000 7f857de00640 1 --2- 172.21.15.83:0/1257054312 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555f64b9c460 0x555f64ba6110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.162+0000 7f857de00640 1 -- 172.21.15.83:0/1257054312 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f64ba6b30 msgr2=0x555f64bbe960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:14.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.162+0000 7f857de00640 1 --2- 172.21.15.83:0/1257054312 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f64ba6b30 0x555f64bbe960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.162+0000 7f857de00640 1 -- 172.21.15.83:0/1257054312 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f648dd7d0 con 0x555f64b9bb40 2024-04-24T08:56:14.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.162+0000 7f857de00640 1 --2- 172.21.15.83:0/1257054312 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f64b9bb40 0x555f64bb2880 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f8570005c40 tx=0x7f857003f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e275de7b46eefafb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:14.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.163+0000 7f856aa00640 1 -- 172.21.15.83:0/1257054312 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8570055070 con 0x555f64b9bb40 2024-04-24T08:56:14.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.163+0000 7f856aa00640 1 -- 172.21.15.83:0/1257054312 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f857004f700 con 0x555f64b9bb40 2024-04-24T08:56:14.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.163+0000 7f856aa00640 1 -- 172.21.15.83:0/1257054312 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8570035050 con 0x555f64b9bb40 2024-04-24T08:56:14.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.163+0000 7f858802e740 1 -- 172.21.15.83:0/1257054312 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f64b9bb40 msgr2=0x555f64bb2880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:14.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.163+0000 7f858802e740 1 --2- 172.21.15.83:0/1257054312 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f64b9bb40 0x555f64bb2880 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f8570005c40 tx=0x7f857003f000 comp rx=0 tx=0).stop 2024-04-24T08:56:14.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.163+0000 7f858802e740 1 -- 172.21.15.83:0/1257054312 shutdown_connections 2024-04-24T08:56:14.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.163+0000 7f858802e740 1 --2- 172.21.15.83:0/1257054312 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f64ba6b30 0x555f64bbe960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.163+0000 7f858802e740 1 --2- 172.21.15.83:0/1257054312 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555f64b9c460 0x555f64ba6110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.163+0000 7f858802e740 1 --2- 172.21.15.83:0/1257054312 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f64b9bb40 0x555f64bb2880 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.163+0000 7f858802e740 1 -- 172.21.15.83:0/1257054312 >> 172.21.15.83:0/1257054312 conn(0x555f64b97e60 msgr2=0x555f64b98240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:14.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.163+0000 7f858802e740 1 -- 172.21.15.83:0/1257054312 shutdown_connections 2024-04-24T08:56:14.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.164+0000 7f858802e740 1 -- 172.21.15.83:0/1257054312 wait complete. 2024-04-24T08:56:14.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.164+0000 7f858802e740 1 -- start start 2024-04-24T08:56:14.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.164+0000 7f858802e740 1 Processor -- start 2024-04-24T08:56:14.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.165+0000 7f858802e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555f64b9bb40 0x555f64baed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:14.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.165+0000 7f858802e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f64b9c460 0x555f64baf2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:14.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.165+0000 7f858802e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f64ba6b30 0x555f64ba0b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:14.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.165+0000 7f858802e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f85801b6d70 con 0x555f64b9c460 2024-04-24T08:56:14.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.165+0000 7f858802e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f85801b66a0 con 0x555f64ba6b30 2024-04-24T08:56:14.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.165+0000 7f858802e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f85801b38e0 con 0x555f64b9bb40 2024-04-24T08:56:14.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.165+0000 7f858802e740 10 client.0 _ll_register_callbacks cb 0x7f8578014070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:56:14.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.165+0000 7f857d400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f64b9c460 0x555f64baf2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:14.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.165+0000 7f857d400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f64b9c460 0x555f64baf2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45750/0 (socket says 172.21.15.83:45750) 2024-04-24T08:56:14.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.165+0000 7f857d400640 1 -- 172.21.15.83:0/4084049811 learned_addr learned my addr 172.21.15.83:0/4084049811 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:56:14.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.165+0000 7f857d400640 1 -- 172.21.15.83:0/4084049811 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555f64b9bb40 msgr2=0x555f64baed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:14.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.165+0000 7f857d400640 1 --2- 172.21.15.83:0/4084049811 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555f64b9bb40 0x555f64baed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.165+0000 7f857d400640 1 -- 172.21.15.83:0/4084049811 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f64ba6b30 msgr2=0x555f64ba0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:14.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.165+0000 7f857e800640 1 --2- 172.21.15.83:0/4084049811 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f64ba6b30 0x555f64ba0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:14.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.165+0000 7f857d400640 1 --2- 172.21.15.83:0/4084049811 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f64ba6b30 0x555f64ba0b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.165+0000 7f857d400640 1 -- 172.21.15.83:0/4084049811 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f64ba13a0 con 0x555f64b9c460 2024-04-24T08:56:14.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.166+0000 7f857d400640 1 --2- 172.21.15.83:0/4084049811 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f64b9c460 0x555f64baf2d0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f856c0cdc20 tx=0x7f856c09c010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec8b23eb939da958 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:14.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.166+0000 7f855fe00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:56:14.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.166+0000 7f855fe00640 1 -- 172.21.15.83:0/4084049811 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f856c020040 con 0x555f64b9c460 2024-04-24T08:56:14.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.166+0000 7f855fe00640 1 -- 172.21.15.83:0/4084049811 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f856c0d0410 con 0x555f64b9c460 2024-04-24T08:56:14.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.166+0000 7f858802e740 1 -- 172.21.15.83:0/4084049811 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f857004f700 con 0x555f64b9c460 2024-04-24T08:56:14.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.166+0000 7f855fe00640 1 -- 172.21.15.83:0/4084049811 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f856c0ce040 con 0x555f64b9c460 2024-04-24T08:56:14.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.166+0000 7f858802e740 10 client.17313 fetch_fsmap learned FSMap version 32 2024-04-24T08:56:14.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.166+0000 7f858802e740 1 -- 172.21.15.83:0/4084049811 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x555f64ba7cb0 con 0x555f64b9c460 2024-04-24T08:56:14.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.166+0000 7f855fe00640 1 -- 172.21.15.83:0/4084049811 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f857004f700 con 0x555f64b9c460 2024-04-24T08:56:14.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.166+0000 7f855fe00640 1 -- 172.21.15.83:0/4084049811 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7f856c0b5b90 con 0x555f64b9c460 2024-04-24T08:56:14.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.167+0000 7f858802e740 10 client.17313 fetch_fsmap finished waiting for FSMap version 32 2024-04-24T08:56:14.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.167+0000 7f858802e740 10 client.17313 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-24T08:56:14.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.167+0000 7f858802e740 20 client.17313 populate_metadata read hostname 'smithi083' 2024-04-24T08:56:14.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.167+0000 7f858802e740 1 --2- 172.21.15.83:0/4084049811 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x555f64c1d520 0x555f64c1f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:14.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.167+0000 7f858802e740 4 client.17313 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T08:56:14.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.167+0000 7f858802e740 1 -- 172.21.15.83:0/4084049811 --> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555f64b08a60 con 0x555f64c1d520 2024-04-24T08:56:14.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.167+0000 7f857de00640 1 --2- 172.21.15.83:0/4084049811 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x555f64c1d520 0x555f64c1f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:14.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.169+0000 7f857de00640 1 --2- 172.21.15.83:0/4084049811 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x555f64c1d520 0x555f64c1f940 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1060ce1fc57647b4 server_cookie=672159c26093ea1b in_seq=0 out_seq=0 2024-04-24T08:56:14.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.169+0000 7f855fe00640 10 client.17313 ms_handle_connect on v2:172.21.15.138:6832/1157967658 2024-04-24T08:56:14.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.169+0000 7f855fe00640 1 -- 172.21.15.83:0/4084049811 <== mds.3 v2:172.21.15.138:6832/1157967658 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555f64b08a60 con 0x555f64c1d520 2024-04-24T08:56:14.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.169+0000 7f855fe00640 10 client.17313 handle_command_reply: tid=0 2024-04-24T08:56:14.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.169+0000 7f858802e740 1 client.17313 shutdown 2024-04-24T08:56:14.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.169+0000 7f858802e740 1 -- 172.21.15.83:0/4084049811 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f64b9c460 msgr2=0x555f64baf2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:14.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.169+0000 7f858802e740 1 --2- 172.21.15.83:0/4084049811 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f64b9c460 0x555f64baf2d0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f856c0cdc20 tx=0x7f856c09c010 comp rx=0 tx=0).stop 2024-04-24T08:56:14.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.169+0000 7f858802e740 1 -- 172.21.15.83:0/4084049811 shutdown_connections 2024-04-24T08:56:14.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.169+0000 7f858802e740 1 --2- 172.21.15.83:0/4084049811 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f64ba6b30 0x555f64ba0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.169+0000 7f858802e740 1 --2- 172.21.15.83:0/4084049811 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f64b9c460 0x555f64baf2d0 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.169+0000 7f858802e740 1 --2- 172.21.15.83:0/4084049811 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x555f64c1d520 0x555f64c1f940 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.169+0000 7f858802e740 1 --2- 172.21.15.83:0/4084049811 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555f64b9bb40 0x555f64baed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.169+0000 7f855fe00640 0 client.17313 ms_handle_reset on v2:172.21.15.138:6832/1157967658 2024-04-24T08:56:14.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.169+0000 7f858802e740 1 -- 172.21.15.83:0/4084049811 >> 172.21.15.83:0/4084049811 conn(0x555f64b97e60 msgr2=0x555f64ba2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:14.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.170+0000 7f858802e740 1 -- 172.21.15.83:0/4084049811 shutdown_connections 2024-04-24T08:56:14.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.170+0000 7f858802e740 1 -- 172.21.15.83:0/4084049811 wait complete. 2024-04-24T08:56:14.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.170+0000 7f858802e740 20 client.17313 trim_cache size 0 max 16384 2024-04-24T08:56:14.172 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T08:56:14.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.170+0000 7f857f200640 1 -- 172.21.15.83:0/3390411020 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f85640a3de0 msgr2=0x7f85640a62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:14.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.170+0000 7f857f200640 1 --2- 172.21.15.83:0/3390411020 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f85640a3de0 0x7f85640a62a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8570009900 tx=0x7f8570000bc0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.170+0000 7f857f200640 1 -- 172.21.15.83:0/3390411020 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8580108f40 msgr2=0x7f85801a4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:14.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.170+0000 7f857f200640 1 --2- 172.21.15.83:0/3390411020 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8580108f40 0x7f85801a4e50 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f856c00d950 tx=0x7f856c00de10 comp rx=0 tx=0).stop 2024-04-24T08:56:14.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.171+0000 7f857f200640 1 -- 172.21.15.83:0/3390411020 shutdown_connections 2024-04-24T08:56:14.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.171+0000 7f857f200640 1 --2- 172.21.15.83:0/3390411020 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f85640a3de0 0x7f85640a62a0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.171+0000 7f857f200640 1 --2- 172.21.15.83:0/3390411020 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f858010d500 0x7f858019eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.171+0000 7f857f200640 1 --2- 172.21.15.83:0/3390411020 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8580108f40 0x7f85801a4e50 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.171+0000 7f857f200640 1 --2- 172.21.15.83:0/3390411020 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8580108580 0x7f85801a4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.171+0000 7f857f200640 1 -- 172.21.15.83:0/3390411020 >> 172.21.15.83:0/3390411020 conn(0x7f8580077e20 msgr2=0x7f8580078440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:14.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.171+0000 7f857f200640 1 -- 172.21.15.83:0/3390411020 shutdown_connections 2024-04-24T08:56:14.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.171+0000 7f857f200640 1 -- 172.21.15.83:0/3390411020 wait complete. 2024-04-24T08:56:14.189 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T08:56:14.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.400+0000 7f4edee00640 1 -- 172.21.15.83:0/2931697922 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ed8108580 msgr2=0x7f4ed8108a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:14.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.400+0000 7f4edee00640 1 --2- 172.21.15.83:0/2931697922 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ed8108580 0x7f4ed8108a00 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f4ec4009930 tx=0x7f4ec402a420 comp rx=0 tx=0).stop 2024-04-24T08:56:14.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.400+0000 7f4edee00640 1 -- 172.21.15.83:0/2931697922 shutdown_connections 2024-04-24T08:56:14.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.400+0000 7f4edee00640 1 --2- 172.21.15.83:0/2931697922 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ed8108f40 0x7f4ed81153b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.400+0000 7f4edee00640 1 --2- 172.21.15.83:0/2931697922 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ed8108580 0x7f4ed8108a00 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.400+0000 7f4edee00640 1 --2- 172.21.15.83:0/2931697922 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4ed810d500 0x7f4ed810d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.400+0000 7f4edee00640 1 -- 172.21.15.83:0/2931697922 >> 172.21.15.83:0/2931697922 conn(0x7f4ed8077e20 msgr2=0x7f4ed8106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:14.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.400+0000 7f4edee00640 1 -- 172.21.15.83:0/2931697922 shutdown_connections 2024-04-24T08:56:14.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.400+0000 7f4edee00640 1 -- 172.21.15.83:0/2931697922 wait complete. 2024-04-24T08:56:14.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.401+0000 7f4edee00640 1 Processor -- start 2024-04-24T08:56:14.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.401+0000 7f4edee00640 1 -- start start 2024-04-24T08:56:14.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.402+0000 7f4edee00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ed8108580 0x7f4ed81a4910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:14.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.402+0000 7f4edee00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4ed8108f40 0x7f4ed81a4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:14.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.402+0000 7f4edee00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ed810d500 0x7f4ed819eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:14.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.402+0000 7f4edee00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4ed81187b0 con 0x7f4ed8108580 2024-04-24T08:56:14.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.402+0000 7f4edee00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4ed8118630 con 0x7f4ed810d500 2024-04-24T08:56:14.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.402+0000 7f4edee00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4ed8118930 con 0x7f4ed8108f40 2024-04-24T08:56:14.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.402+0000 7f4edda00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ed8108580 0x7f4ed81a4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:14.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.402+0000 7f4edda00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ed8108580 0x7f4ed81a4910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45764/0 (socket says 172.21.15.83:45764) 2024-04-24T08:56:14.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.402+0000 7f4edda00640 1 -- 172.21.15.83:0/3087502345 learned_addr learned my addr 172.21.15.83:0/3087502345 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:56:14.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.402+0000 7f4edd000640 1 --2- 172.21.15.83:0/3087502345 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4ed8108f40 0x7f4ed81a4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:14.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.403+0000 7f4edda00640 1 -- 172.21.15.83:0/3087502345 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4ed8108f40 msgr2=0x7f4ed81a4e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:14.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.403+0000 7f4ede400640 1 --2- 172.21.15.83:0/3087502345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ed810d500 0x7f4ed819eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:14.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.403+0000 7f4edda00640 1 --2- 172.21.15.83:0/3087502345 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4ed8108f40 0x7f4ed81a4e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.403+0000 7f4edda00640 1 -- 172.21.15.83:0/3087502345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ed810d500 msgr2=0x7f4ed819eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:14.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.403+0000 7f4edda00640 1 --2- 172.21.15.83:0/3087502345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ed810d500 0x7f4ed819eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.403+0000 7f4edda00640 1 -- 172.21.15.83:0/3087502345 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ed819f290 con 0x7f4ed8108580 2024-04-24T08:56:14.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.403+0000 7f4edda00640 1 --2- 172.21.15.83:0/3087502345 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ed8108580 0x7f4ed81a4910 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc00c970 tx=0x7f4ecc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:14.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.403+0000 7f4ed6000640 1 -- 172.21.15.83:0/3087502345 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ecc007c10 con 0x7f4ed8108580 2024-04-24T08:56:14.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.403+0000 7f4ed6000640 1 -- 172.21.15.83:0/3087502345 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4ecc007db0 con 0x7f4ed8108580 2024-04-24T08:56:14.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.403+0000 7f4ed6000640 1 -- 172.21.15.83:0/3087502345 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ecc00f870 con 0x7f4ed8108580 2024-04-24T08:56:14.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.403+0000 7f4edee00640 1 -- 172.21.15.83:0/3087502345 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ed819f580 con 0x7f4ed8108580 2024-04-24T08:56:14.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.403+0000 7f4edee00640 1 -- 172.21.15.83:0/3087502345 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ed81ab7d0 con 0x7f4ed8108580 2024-04-24T08:56:14.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.407+0000 7f4ee318e740 1 Processor -- start 2024-04-24T08:56:14.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.407+0000 7f4ee318e740 1 -- start start 2024-04-24T08:56:14.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.407+0000 7f4ed6000640 1 -- 172.21.15.83:0/3087502345 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4ecc00fa10 con 0x7f4ed8108580 2024-04-24T08:56:14.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.407+0000 7f4ee318e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55561e234b40 0x55561e234f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:14.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.407+0000 7f4ee318e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55561e235460 0x55561e23f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:14.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.407+0000 7f4ee318e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55561e23fb30 0x55561e241f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:14.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.408+0000 7f4ee318e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55561e19a1d0 con 0x55561e235460 2024-04-24T08:56:14.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.408+0000 7f4ee318e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55561e140450 con 0x55561e23fb30 2024-04-24T08:56:14.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.408+0000 7f4ee318e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55561e196190 con 0x55561e234b40 2024-04-24T08:56:14.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.408+0000 7f4edda00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55561e235460 0x55561e23f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:14.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.408+0000 7f4edda00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55561e235460 0x55561e23f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45770/0 (socket says 172.21.15.83:45770) 2024-04-24T08:56:14.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.408+0000 7f4edda00640 1 -- 172.21.15.83:0/2670588492 learned_addr learned my addr 172.21.15.83:0/2670588492 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:56:14.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.408+0000 7f4edd000640 1 --2- 172.21.15.83:0/2670588492 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55561e23fb30 0x55561e241f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:14.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.408+0000 7f4ede400640 1 --2- 172.21.15.83:0/2670588492 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55561e234b40 0x55561e234f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:14.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.408+0000 7f4edda00640 1 -- 172.21.15.83:0/2670588492 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55561e234b40 msgr2=0x55561e234f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:14.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.408+0000 7f4edda00640 1 --2- 172.21.15.83:0/2670588492 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55561e234b40 0x55561e234f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.408+0000 7f4edda00640 1 -- 172.21.15.83:0/2670588492 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55561e23fb30 msgr2=0x55561e241f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:14.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.408+0000 7f4edda00640 1 --2- 172.21.15.83:0/2670588492 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55561e23fb30 0x55561e241f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.409+0000 7f4edda00640 1 -- 172.21.15.83:0/2670588492 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55561e1fd310 con 0x55561e235460 2024-04-24T08:56:14.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.409+0000 7f4edd000640 1 --2- 172.21.15.83:0/2670588492 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55561e23fb30 0x55561e241f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:56:14.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.409+0000 7f4edda00640 1 --2- 172.21.15.83:0/2670588492 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55561e235460 0x55561e23f5f0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc0ce280 tx=0x7f4ecc0d92a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81fe120796ad5d8d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:14.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.409+0000 7f4ed6000640 1 --2- 172.21.15.83:0/3087502345 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f4ec00a3d20 0x7f4ec00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:14.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.409+0000 7f4ecaa00640 1 -- 172.21.15.83:0/2670588492 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ecc0e4070 con 0x55561e235460 2024-04-24T08:56:14.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.409+0000 7f4ecaa00640 1 -- 172.21.15.83:0/2670588492 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4ecc018b50 con 0x55561e235460 2024-04-24T08:56:14.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.409+0000 7f4ecaa00640 1 -- 172.21.15.83:0/2670588492 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ecc010050 con 0x55561e235460 2024-04-24T08:56:14.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.410+0000 7f4ede400640 1 --2- 172.21.15.83:0/3087502345 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f4ec00a3d20 0x7f4ec00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:14.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.410+0000 7f4ed6000640 1 -- 172.21.15.83:0/3087502345 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f4ecc0d0fe0 con 0x7f4ed8108580 2024-04-24T08:56:14.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.410+0000 7f4ee318e740 1 -- 172.21.15.83:0/2670588492 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55561e235460 msgr2=0x55561e23f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:14.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.410+0000 7f4ede400640 1 --2- 172.21.15.83:0/3087502345 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f4ec00a3d20 0x7f4ec00a61e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4ed0008e00 tx=0x7f4ed0008390 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:14.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.410+0000 7f4ee318e740 1 --2- 172.21.15.83:0/2670588492 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55561e235460 0x55561e23f5f0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc0ce280 tx=0x7f4ecc0d92a0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.411+0000 7f4ee318e740 1 -- 172.21.15.83:0/2670588492 shutdown_connections 2024-04-24T08:56:14.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.411+0000 7f4ee318e740 1 --2- 172.21.15.83:0/2670588492 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55561e23fb30 0x55561e241f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.411+0000 7f4ee318e740 1 --2- 172.21.15.83:0/2670588492 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55561e235460 0x55561e23f5f0 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.411+0000 7f4ee318e740 1 --2- 172.21.15.83:0/2670588492 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55561e234b40 0x55561e234f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.411+0000 7f4ee318e740 1 -- 172.21.15.83:0/2670588492 >> 172.21.15.83:0/2670588492 conn(0x55561e230e60 msgr2=0x55561e231240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:14.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.411+0000 7f4ee318e740 1 -- 172.21.15.83:0/2670588492 shutdown_connections 2024-04-24T08:56:14.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.411+0000 7f4ee318e740 1 -- 172.21.15.83:0/2670588492 wait complete. 2024-04-24T08:56:14.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.411+0000 7f4ee318e740 1 -- start start 2024-04-24T08:56:14.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.412+0000 7f4ee318e740 1 Processor -- start 2024-04-24T08:56:14.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.412+0000 7f4ee318e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55561e234b40 0x55561e23c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:14.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.412+0000 7f4ee318e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55561e235460 0x55561e23ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:14.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.412+0000 7f4ee318e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55561e23fb30 0x55561e23d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:14.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.412+0000 7f4ee318e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55561e2429c0 con 0x55561e235460 2024-04-24T08:56:14.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.412+0000 7f4ee318e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55561e153810 con 0x55561e23fb30 2024-04-24T08:56:14.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.412+0000 7f4ee318e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55561e242b40 con 0x55561e234b40 2024-04-24T08:56:14.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.412+0000 7f4ee318e740 10 client.0 _ll_register_callbacks cb 0x55561e121140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:56:14.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.412+0000 7f4edd000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55561e235460 0x55561e23ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:14.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.412+0000 7f4edd000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55561e235460 0x55561e23ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45784/0 (socket says 172.21.15.83:45784) 2024-04-24T08:56:14.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.412+0000 7f4edd000640 1 -- 172.21.15.83:0/3514033528 learned_addr learned my addr 172.21.15.83:0/3514033528 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:56:14.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.413+0000 7f4ede400640 1 --2- 172.21.15.83:0/3514033528 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55561e23fb30 0x55561e23d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:14.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.413+0000 7f4edda00640 1 --2- 172.21.15.83:0/3514033528 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55561e234b40 0x55561e23c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:14.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.413+0000 7f4edd000640 1 -- 172.21.15.83:0/3514033528 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55561e234b40 msgr2=0x55561e23c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:14.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.413+0000 7f4edd000640 1 --2- 172.21.15.83:0/3514033528 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55561e234b40 0x55561e23c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.413+0000 7f4edd000640 1 -- 172.21.15.83:0/3514033528 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55561e23fb30 msgr2=0x55561e23d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:14.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.413+0000 7f4edd000640 1 --2- 172.21.15.83:0/3514033528 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55561e23fb30 0x55561e23d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.413+0000 7f4edd000640 1 -- 172.21.15.83:0/3514033528 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55561e1643f0 con 0x55561e235460 2024-04-24T08:56:14.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.413+0000 7f4edd000640 1 --2- 172.21.15.83:0/3514033528 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55561e235460 0x55561e23ce50 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f4ec4009930 tx=0x7f4ec4003880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41214f15df1a261e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:14.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.413+0000 7f4ebf400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:56:14.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.413+0000 7f4ebf400640 1 -- 172.21.15.83:0/3514033528 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ec4035070 con 0x55561e235460 2024-04-24T08:56:14.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.414+0000 7f4ebf400640 1 -- 172.21.15.83:0/3514033528 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f4ec4003c70 con 0x55561e235460 2024-04-24T08:56:14.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.414+0000 7f4ee318e740 1 -- 172.21.15.83:0/3514033528 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55561e1a7bf0 con 0x55561e235460 2024-04-24T08:56:14.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.414+0000 7f4ebf400640 1 -- 172.21.15.83:0/3514033528 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f4ec404d050 con 0x55561e235460 2024-04-24T08:56:14.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.414+0000 7f4ee318e740 10 client.17325 fetch_fsmap learned FSMap version 32 2024-04-24T08:56:14.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.414+0000 7f4ee318e740 1 -- 172.21.15.83:0/3514033528 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55561deac3f0 con 0x55561e235460 2024-04-24T08:56:14.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.414+0000 7f4ebf400640 1 -- 172.21.15.83:0/3514033528 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55561e1a7bf0 con 0x55561e235460 2024-04-24T08:56:14.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.414+0000 7f4ebf400640 1 -- 172.21.15.83:0/3514033528 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7f4ec4044470 con 0x55561e235460 2024-04-24T08:56:14.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.415+0000 7f4ee318e740 10 client.17325 fetch_fsmap finished waiting for FSMap version 32 2024-04-24T08:56:14.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.415+0000 7f4ee318e740 10 client.17325 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-24T08:56:14.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.415+0000 7f4ee318e740 20 client.17325 populate_metadata read hostname 'smithi083' 2024-04-24T08:56:14.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.415+0000 7f4ee318e740 1 --2- 172.21.15.83:0/3514033528 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55561e24fdd0 0x55561e2521c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:14.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.415+0000 7f4ee318e740 4 client.17325 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:56:14.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.415+0000 7f4ee318e740 1 -- 172.21.15.83:0/3514033528 --> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55561e1a1a60 con 0x55561e24fdd0 2024-04-24T08:56:14.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.415+0000 7f4edda00640 1 --2- 172.21.15.83:0/3514033528 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55561e24fdd0 0x55561e2521c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:14.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.417+0000 7f4edda00640 1 --2- 172.21.15.83:0/3514033528 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55561e24fdd0 0x55561e2521c0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c92c5a33d8fd8aa server_cookie=22415790d6b2e7b4 in_seq=0 out_seq=0 2024-04-24T08:56:14.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.417+0000 7f4ebf400640 10 client.17325 ms_handle_connect on v2:172.21.15.138:6834/1909219758 2024-04-24T08:56:14.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.419+0000 7f4ebf400640 1 -- 172.21.15.83:0/3514033528 <== mds.2 v2:172.21.15.138:6834/1909219758 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55561e1a1a60 con 0x55561e24fdd0 2024-04-24T08:56:14.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.419+0000 7f4ebf400640 10 client.17325 handle_command_reply: tid=0 2024-04-24T08:56:14.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.419+0000 7f4ee318e740 1 client.17325 shutdown 2024-04-24T08:56:14.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.419+0000 7f4ee318e740 1 -- 172.21.15.83:0/3514033528 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55561e235460 msgr2=0x55561e23ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:14.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.419+0000 7f4ee318e740 1 --2- 172.21.15.83:0/3514033528 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55561e235460 0x55561e23ce50 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f4ec4009930 tx=0x7f4ec4003880 comp rx=0 tx=0).stop 2024-04-24T08:56:14.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.419+0000 7f4ee318e740 1 -- 172.21.15.83:0/3514033528 shutdown_connections 2024-04-24T08:56:14.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.419+0000 7f4ee318e740 1 --2- 172.21.15.83:0/3514033528 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55561e24fdd0 0x55561e2521c0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.419+0000 7f4ee318e740 1 --2- 172.21.15.83:0/3514033528 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55561e23fb30 0x55561e23d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.419+0000 7f4ee318e740 1 --2- 172.21.15.83:0/3514033528 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55561e235460 0x55561e23ce50 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.419+0000 7f4ee318e740 1 --2- 172.21.15.83:0/3514033528 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55561e234b40 0x55561e23c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.419+0000 7f4ebf400640 0 client.17325 ms_handle_reset on v2:172.21.15.138:6834/1909219758 2024-04-24T08:56:14.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.419+0000 7f4ee318e740 1 -- 172.21.15.83:0/3514033528 >> 172.21.15.83:0/3514033528 conn(0x55561e230e60 msgr2=0x55561e236b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:14.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.419+0000 7f4ee318e740 1 -- 172.21.15.83:0/3514033528 shutdown_connections 2024-04-24T08:56:14.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.419+0000 7f4ee318e740 1 -- 172.21.15.83:0/3514033528 wait complete. 2024-04-24T08:56:14.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.419+0000 7f4ee318e740 20 client.17325 trim_cache size 0 max 16384 2024-04-24T08:56:14.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.435+0000 7f4ee318e740 1 Processor -- start 2024-04-24T08:56:14.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.435+0000 7f4ee318e740 1 -- start start 2024-04-24T08:56:14.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.435+0000 7f4ee318e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55561e234b40 0x55561e2372a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:14.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.435+0000 7f4ee318e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55561e235460 0x55561e24b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:14.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.435+0000 7f4ee318e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55561e23fb30 0x55561e257ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:14.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.435+0000 7f4ee318e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4ed81b5930 con 0x55561e234b40 2024-04-24T08:56:14.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.435+0000 7f4ee318e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4ed81b74a0 con 0x55561e235460 2024-04-24T08:56:14.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.435+0000 7f4ee318e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4ed81b6000 con 0x55561e23fb30 2024-04-24T08:56:14.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.436+0000 7f4edda00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55561e234b40 0x55561e2372a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:14.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.436+0000 7f4edda00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55561e234b40 0x55561e2372a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45786/0 (socket says 172.21.15.83:45786) 2024-04-24T08:56:14.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.436+0000 7f4edda00640 1 -- 172.21.15.83:0/55531753 learned_addr learned my addr 172.21.15.83:0/55531753 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:56:14.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.436+0000 7f4edd000640 1 --2- 172.21.15.83:0/55531753 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55561e235460 0x55561e24b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:14.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.436+0000 7f4edda00640 1 -- 172.21.15.83:0/55531753 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55561e23fb30 msgr2=0x55561e257ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:14.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.436+0000 7f4edda00640 1 --2- 172.21.15.83:0/55531753 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55561e23fb30 0x55561e257ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.436+0000 7f4edda00640 1 -- 172.21.15.83:0/55531753 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55561e235460 msgr2=0x55561e24b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:14.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.436+0000 7f4edda00640 1 --2- 172.21.15.83:0/55531753 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55561e235460 0x55561e24b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.436+0000 7f4edda00640 1 -- 172.21.15.83:0/55531753 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55561df767d0 con 0x55561e234b40 2024-04-24T08:56:14.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.436+0000 7f4edda00640 1 --2- 172.21.15.83:0/55531753 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55561e234b40 0x55561e2372a0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc0d97a0 tx=0x7f4ecc0d4ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=558d6a54184b16a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:14.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.437+0000 7f4eca000640 1 -- 172.21.15.83:0/55531753 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ecc0d4bc0 con 0x55561e234b40 2024-04-24T08:56:14.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.437+0000 7f4eca000640 1 -- 172.21.15.83:0/55531753 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4ecc0d4920 con 0x55561e234b40 2024-04-24T08:56:14.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.437+0000 7f4eca000640 1 -- 172.21.15.83:0/55531753 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ecc09c580 con 0x55561e234b40 2024-04-24T08:56:14.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.437+0000 7f4ee318e740 1 -- 172.21.15.83:0/55531753 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55561e234b40 msgr2=0x55561e2372a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:14.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.437+0000 7f4ee318e740 1 --2- 172.21.15.83:0/55531753 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55561e234b40 0x55561e2372a0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc0d97a0 tx=0x7f4ecc0d4ad0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.437+0000 7f4ee318e740 1 -- 172.21.15.83:0/55531753 shutdown_connections 2024-04-24T08:56:14.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.437+0000 7f4ee318e740 1 --2- 172.21.15.83:0/55531753 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55561e23fb30 0x55561e257ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.437+0000 7f4ee318e740 1 --2- 172.21.15.83:0/55531753 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55561e235460 0x55561e24b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.437+0000 7f4ee318e740 1 --2- 172.21.15.83:0/55531753 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55561e234b40 0x55561e2372a0 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.437+0000 7f4ee318e740 1 -- 172.21.15.83:0/55531753 >> 172.21.15.83:0/55531753 conn(0x55561e230e60 msgr2=0x55561e23d160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:14.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.437+0000 7f4ee318e740 1 -- 172.21.15.83:0/55531753 shutdown_connections 2024-04-24T08:56:14.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.438+0000 7f4ee318e740 1 -- 172.21.15.83:0/55531753 wait complete. 2024-04-24T08:56:14.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.438+0000 7f4ee318e740 1 -- start start 2024-04-24T08:56:14.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.438+0000 7f4ee318e740 1 Processor -- start 2024-04-24T08:56:14.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.438+0000 7f4ee318e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55561e234b40 0x55561e240a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:14.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.438+0000 7f4ee318e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55561e235460 0x55561e240fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:14.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.438+0000 7f4ee318e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55561e23fb30 0x55561e241500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:14.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.438+0000 7f4ee318e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4ed81b6da0 con 0x55561e23fb30 2024-04-24T08:56:14.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.438+0000 7f4ee318e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4ed81b66d0 con 0x55561e235460 2024-04-24T08:56:14.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.438+0000 7f4ee318e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4ed81b3910 con 0x55561e234b40 2024-04-24T08:56:14.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.439+0000 7f4ee318e740 10 client.0 _ll_register_callbacks cb 0x7f4ec4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:56:14.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.439+0000 7f4ede400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55561e23fb30 0x55561e241500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:14.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.439+0000 7f4ede400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55561e23fb30 0x55561e241500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45792/0 (socket says 172.21.15.83:45792) 2024-04-24T08:56:14.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.439+0000 7f4ede400640 1 -- 172.21.15.83:0/1254104832 learned_addr learned my addr 172.21.15.83:0/1254104832 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:56:14.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.439+0000 7f4edd000640 1 --2- 172.21.15.83:0/1254104832 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55561e235460 0x55561e240fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:14.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.439+0000 7f4ede400640 1 -- 172.21.15.83:0/1254104832 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55561e234b40 msgr2=0x55561e240a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:14.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.439+0000 7f4ede400640 1 --2- 172.21.15.83:0/1254104832 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55561e234b40 0x55561e240a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.439+0000 7f4ede400640 1 -- 172.21.15.83:0/1254104832 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55561e235460 msgr2=0x55561e240fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:14.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.439+0000 7f4ede400640 1 --2- 172.21.15.83:0/1254104832 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55561e235460 0x55561e240fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.439+0000 7f4ede400640 1 -- 172.21.15.83:0/1254104832 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55561e246370 con 0x55561e23fb30 2024-04-24T08:56:14.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.439+0000 7f4ede400640 1 --2- 172.21.15.83:0/1254104832 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55561e23fb30 0x55561e241500 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f4ed0008fd0 tx=0x7f4ed0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5ce524881c2a051 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:14.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.440+0000 7f4ebf400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:56:14.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.440+0000 7f4ebf400640 1 -- 172.21.15.83:0/1254104832 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ed0010e20 con 0x55561e23fb30 2024-04-24T08:56:14.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.440+0000 7f4ebf400640 1 -- 172.21.15.83:0/1254104832 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f4ed001e430 con 0x55561e23fb30 2024-04-24T08:56:14.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.440+0000 7f4ee318e740 1 -- 172.21.15.83:0/1254104832 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ecc0d4920 con 0x55561e23fb30 2024-04-24T08:56:14.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.440+0000 7f4ebf400640 1 -- 172.21.15.83:0/1254104832 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f4ed001e630 con 0x55561e23fb30 2024-04-24T08:56:14.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.440+0000 7f4ee318e740 10 client.17331 fetch_fsmap learned FSMap version 32 2024-04-24T08:56:14.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.440+0000 7f4ee318e740 1 -- 172.21.15.83:0/1254104832 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55561e2b6880 con 0x55561e23fb30 2024-04-24T08:56:14.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.440+0000 7f4ebf400640 1 -- 172.21.15.83:0/1254104832 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4ecc0d4920 con 0x55561e23fb30 2024-04-24T08:56:14.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.441+0000 7f4ebf400640 1 -- 172.21.15.83:0/1254104832 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7f4ed00278b0 con 0x55561e23fb30 2024-04-24T08:56:14.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.441+0000 7f4ee318e740 10 client.17331 fetch_fsmap finished waiting for FSMap version 32 2024-04-24T08:56:14.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.441+0000 7f4ee318e740 10 client.17331 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-24T08:56:14.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.441+0000 7f4ee318e740 20 client.17331 populate_metadata read hostname 'smithi083' 2024-04-24T08:56:14.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.441+0000 7f4ee318e740 1 --2- 172.21.15.83:0/1254104832 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55561e2b7030 0x55561e2b9450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:14.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.441+0000 7f4ee318e740 4 client.17331 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T08:56:14.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.441+0000 7f4ee318e740 1 -- 172.21.15.83:0/1254104832 --> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55561e11d4d0 con 0x55561e2b7030 2024-04-24T08:56:14.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.441+0000 7f4edda00640 1 --2- 172.21.15.83:0/1254104832 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55561e2b7030 0x55561e2b9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:14.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.442+0000 7f4edda00640 1 --2- 172.21.15.83:0/1254104832 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55561e2b7030 0x55561e2b9450 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8c6fbe57c6b62763 server_cookie=b31da5aa725ae7b4 in_seq=0 out_seq=0 2024-04-24T08:56:14.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.442+0000 7f4ebf400640 10 client.17331 ms_handle_connect on v2:172.21.15.138:6834/1909219758 2024-04-24T08:56:14.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.443+0000 7f4ebf400640 1 -- 172.21.15.83:0/1254104832 <== mds.2 v2:172.21.15.138:6834/1909219758 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55561e11d4d0 con 0x55561e2b7030 2024-04-24T08:56:14.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.443+0000 7f4ebf400640 10 client.17331 handle_command_reply: tid=0 2024-04-24T08:56:14.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.443+0000 7f4ee318e740 1 client.17331 shutdown 2024-04-24T08:56:14.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.443+0000 7f4ee318e740 1 -- 172.21.15.83:0/1254104832 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55561e23fb30 msgr2=0x55561e241500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:14.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.443+0000 7f4ee318e740 1 --2- 172.21.15.83:0/1254104832 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55561e23fb30 0x55561e241500 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f4ed0008fd0 tx=0x7f4ed0004ab0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.443+0000 7f4ee318e740 1 -- 172.21.15.83:0/1254104832 shutdown_connections 2024-04-24T08:56:14.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.443+0000 7f4ee318e740 1 --2- 172.21.15.83:0/1254104832 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55561e2b7030 0x55561e2b9450 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.443+0000 7f4ee318e740 1 --2- 172.21.15.83:0/1254104832 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55561e23fb30 0x55561e241500 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.443+0000 7f4ee318e740 1 --2- 172.21.15.83:0/1254104832 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55561e235460 0x55561e240fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.443+0000 7f4ee318e740 1 --2- 172.21.15.83:0/1254104832 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55561e234b40 0x55561e240a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.443+0000 7f4ebf400640 0 client.17331 ms_handle_reset on v2:172.21.15.138:6834/1909219758 2024-04-24T08:56:14.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.443+0000 7f4ee318e740 1 -- 172.21.15.83:0/1254104832 >> 172.21.15.83:0/1254104832 conn(0x55561e230e60 msgr2=0x55561e22f6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:14.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.443+0000 7f4ee318e740 1 -- 172.21.15.83:0/1254104832 shutdown_connections 2024-04-24T08:56:14.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.443+0000 7f4ee318e740 1 -- 172.21.15.83:0/1254104832 wait complete. 2024-04-24T08:56:14.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.443+0000 7f4ee318e740 20 client.17331 trim_cache size 0 max 16384 2024-04-24T08:56:14.447 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T08:56:14.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.444+0000 7f4edee00640 1 -- 172.21.15.83:0/3087502345 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f4ec00a3d20 msgr2=0x7f4ec00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:14.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.444+0000 7f4edee00640 1 --2- 172.21.15.83:0/3087502345 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f4ec00a3d20 0x7f4ec00a61e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4ed0008e00 tx=0x7f4ed0008390 comp rx=0 tx=0).stop 2024-04-24T08:56:14.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.444+0000 7f4edee00640 1 -- 172.21.15.83:0/3087502345 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ed8108580 msgr2=0x7f4ed81a4910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:14.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.444+0000 7f4edee00640 1 --2- 172.21.15.83:0/3087502345 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ed8108580 0x7f4ed81a4910 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc00c970 tx=0x7f4ecc00ce30 comp rx=0 tx=0).stop 2024-04-24T08:56:14.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.444+0000 7f4edee00640 1 -- 172.21.15.83:0/3087502345 shutdown_connections 2024-04-24T08:56:14.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.444+0000 7f4edee00640 1 --2- 172.21.15.83:0/3087502345 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f4ec00a3d20 0x7f4ec00a61e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.444+0000 7f4edee00640 1 --2- 172.21.15.83:0/3087502345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ed810d500 0x7f4ed819eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.444+0000 7f4edee00640 1 --2- 172.21.15.83:0/3087502345 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4ed8108f40 0x7f4ed81a4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.444+0000 7f4edee00640 1 --2- 172.21.15.83:0/3087502345 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ed8108580 0x7f4ed81a4910 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.444+0000 7f4edee00640 1 -- 172.21.15.83:0/3087502345 >> 172.21.15.83:0/3087502345 conn(0x7f4ed8077e20 msgr2=0x7f4ed8078480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:14.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.444+0000 7f4edee00640 1 -- 172.21.15.83:0/3087502345 shutdown_connections 2024-04-24T08:56:14.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.444+0000 7f4edee00640 1 -- 172.21.15.83:0/3087502345 wait complete. 2024-04-24T08:56:14.461 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T08:56:14.572 INFO:tasks.workunit.client.0.smithi083.stdout: 1 20947 0.67 MB/sec execute 48 sec latency 50.631 ms 2024-04-24T08:56:14.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.666+0000 7f9128e00640 1 -- 172.21.15.83:0/2802351711 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9124100c40 msgr2=0x7f9124101020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:14.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.666+0000 7f9128e00640 1 --2- 172.21.15.83:0/2802351711 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9124100c40 0x7f9124101020 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f9114009900 tx=0x7f911402a410 comp rx=0 tx=0).stop 2024-04-24T08:56:14.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.666+0000 7f9128e00640 1 -- 172.21.15.83:0/2802351711 shutdown_connections 2024-04-24T08:56:14.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.666+0000 7f9128e00640 1 --2- 172.21.15.83:0/2802351711 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f912410e7f0 0x7f9124110be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.666+0000 7f9128e00640 1 --2- 172.21.15.83:0/2802351711 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f91241015f0 0x7f912410e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.666+0000 7f9128e00640 1 --2- 172.21.15.83:0/2802351711 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9124100c40 0x7f9124101020 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.666+0000 7f9128e00640 1 -- 172.21.15.83:0/2802351711 >> 172.21.15.83:0/2802351711 conn(0x7f91240fc880 msgr2=0x7f91240feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:14.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.667+0000 7f9128e00640 1 -- 172.21.15.83:0/2802351711 shutdown_connections 2024-04-24T08:56:14.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.667+0000 7f9128e00640 1 -- 172.21.15.83:0/2802351711 wait complete. 2024-04-24T08:56:14.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.667+0000 7f9128e00640 1 Processor -- start 2024-04-24T08:56:14.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.668+0000 7f9128e00640 1 -- start start 2024-04-24T08:56:14.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.668+0000 7f9128e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9124100c40 0x7f91241a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:14.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.668+0000 7f9128e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f91241015f0 0x7f91241a0b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:14.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.668+0000 7f9128e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f912410e7f0 0x7f912419a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:14.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.668+0000 7f9128e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9124114340 con 0x7f91241015f0 2024-04-24T08:56:14.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.668+0000 7f9128e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f91241141c0 con 0x7f9124100c40 2024-04-24T08:56:14.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.668+0000 7f9128e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f91241144c0 con 0x7f912410e7f0 2024-04-24T08:56:14.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.668+0000 7f9122a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f91241015f0 0x7f91241a0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:14.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.668+0000 7f9122a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f91241015f0 0x7f91241a0b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45810/0 (socket says 172.21.15.83:45810) 2024-04-24T08:56:14.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.668+0000 7f9122a00640 1 -- 172.21.15.83:0/343334989 learned_addr learned my addr 172.21.15.83:0/343334989 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:56:14.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.669+0000 7f9123400640 1 --2- 172.21.15.83:0/343334989 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9124100c40 0x7f91241a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:14.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.669+0000 7f9122a00640 1 -- 172.21.15.83:0/343334989 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f912410e7f0 msgr2=0x7f912419a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:14.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.669+0000 7f9122a00640 1 --2- 172.21.15.83:0/343334989 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f912410e7f0 0x7f912419a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.669+0000 7f9122a00640 1 -- 172.21.15.83:0/343334989 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9124100c40 msgr2=0x7f91241a05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:14.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.669+0000 7f9122a00640 1 --2- 172.21.15.83:0/343334989 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9124100c40 0x7f91241a05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.669+0000 7f9122a00640 1 -- 172.21.15.83:0/343334989 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f912419afc0 con 0x7f91241015f0 2024-04-24T08:56:14.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.669+0000 7f9123400640 1 --2- 172.21.15.83:0/343334989 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9124100c40 0x7f91241a05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:56:14.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.669+0000 7f9122a00640 1 --2- 172.21.15.83:0/343334989 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f91241015f0 0x7f91241a0b30 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f911000d770 tx=0x7f911000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:14.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.669+0000 7f911be00640 1 -- 172.21.15.83:0/343334989 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9110017070 con 0x7f91241015f0 2024-04-24T08:56:14.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.669+0000 7f9128e00640 1 -- 172.21.15.83:0/343334989 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f912419b2b0 con 0x7f91241015f0 2024-04-24T08:56:14.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.669+0000 7f911be00640 1 -- 172.21.15.83:0/343334989 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f91100054d0 con 0x7f91241015f0 2024-04-24T08:56:14.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.669+0000 7f911be00640 1 -- 172.21.15.83:0/343334989 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9110004cf0 con 0x7f91241015f0 2024-04-24T08:56:14.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.669+0000 7f9128e00640 1 -- 172.21.15.83:0/343334989 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91241a7570 con 0x7f91241015f0 2024-04-24T08:56:14.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.673+0000 7f912d3dd740 1 Processor -- start 2024-04-24T08:56:14.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.673+0000 7f912d3dd740 1 -- start start 2024-04-24T08:56:14.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.673+0000 7f912d3dd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e9b9fc4b40 0x55e9b9fc4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:14.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.673+0000 7f912d3dd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9b9fc5460 0x55e9b9fcf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:14.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.673+0000 7f911be00640 1 -- 172.21.15.83:0/343334989 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9110004e90 con 0x7f91241015f0 2024-04-24T08:56:14.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.673+0000 7f912d3dd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9b9fcfb30 0x55e9b9fd1f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:14.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.673+0000 7f912d3dd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e9b9f2a1d0 con 0x55e9b9fcfb30 2024-04-24T08:56:14.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.673+0000 7f912d3dd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e9b9ed0450 con 0x55e9b9fc5460 2024-04-24T08:56:14.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.673+0000 7f912d3dd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55e9b9f26190 con 0x55e9b9fc4b40 2024-04-24T08:56:14.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.674+0000 7f9122a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9b9fcfb30 0x55e9b9fd1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:14.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.674+0000 7f9122a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9b9fcfb30 0x55e9b9fd1f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45812/0 (socket says 172.21.15.83:45812) 2024-04-24T08:56:14.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.674+0000 7f9123e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e9b9fc4b40 0x55e9b9fc4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:14.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.674+0000 7f9122a00640 1 -- 172.21.15.83:0/3931658193 learned_addr learned my addr 172.21.15.83:0/3931658193 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:56:14.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.674+0000 7f9123400640 1 --2- 172.21.15.83:0/3931658193 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9b9fc5460 0x55e9b9fcf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:14.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.674+0000 7f9122a00640 1 -- 172.21.15.83:0/3931658193 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e9b9fc4b40 msgr2=0x55e9b9fc4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:14.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.674+0000 7f9122a00640 1 --2- 172.21.15.83:0/3931658193 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e9b9fc4b40 0x55e9b9fc4f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.674+0000 7f9122a00640 1 -- 172.21.15.83:0/3931658193 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9b9fc5460 msgr2=0x55e9b9fcf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:14.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.674+0000 7f9122a00640 1 --2- 172.21.15.83:0/3931658193 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9b9fc5460 0x55e9b9fcf5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.674+0000 7f9122a00640 1 -- 172.21.15.83:0/3931658193 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9b9f8d310 con 0x55e9b9fcfb30 2024-04-24T08:56:14.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.675+0000 7f9122a00640 1 --2- 172.21.15.83:0/3931658193 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9b9fcfb30 0x55e9b9fd1f20 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f91100cd940 tx=0x7f91100d8750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd91e04211effc6e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:14.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.675+0000 7f9118c00640 1 -- 172.21.15.83:0/3931658193 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91100d0cd0 con 0x55e9b9fcfb30 2024-04-24T08:56:14.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.675+0000 7f911be00640 1 --2- 172.21.15.83:0/343334989 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f91080a3da0 0x7f91080a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:14.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.675+0000 7f9118c00640 1 -- 172.21.15.83:0/3931658193 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f91100dd070 con 0x55e9b9fcfb30 2024-04-24T08:56:14.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.675+0000 7f9118c00640 1 -- 172.21.15.83:0/3931658193 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91100d9760 con 0x55e9b9fcfb30 2024-04-24T08:56:14.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.675+0000 7f911be00640 1 -- 172.21.15.83:0/343334989 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f91100d06e0 con 0x7f91241015f0 2024-04-24T08:56:14.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.676+0000 7f9123e00640 1 --2- 172.21.15.83:0/343334989 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f91080a3da0 0x7f91080a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:14.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.676+0000 7f912d3dd740 1 -- 172.21.15.83:0/3931658193 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9b9fcfb30 msgr2=0x55e9b9fd1f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:14.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.676+0000 7f912d3dd740 1 --2- 172.21.15.83:0/3931658193 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9b9fcfb30 0x55e9b9fd1f20 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f91100cd940 tx=0x7f91100d8750 comp rx=0 tx=0).stop 2024-04-24T08:56:14.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.676+0000 7f912d3dd740 1 -- 172.21.15.83:0/3931658193 shutdown_connections 2024-04-24T08:56:14.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.676+0000 7f912d3dd740 1 --2- 172.21.15.83:0/3931658193 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9b9fcfb30 0x55e9b9fd1f20 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.676+0000 7f912d3dd740 1 --2- 172.21.15.83:0/3931658193 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9b9fc5460 0x55e9b9fcf5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.676+0000 7f912d3dd740 1 --2- 172.21.15.83:0/3931658193 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e9b9fc4b40 0x55e9b9fc4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.676+0000 7f912d3dd740 1 -- 172.21.15.83:0/3931658193 >> 172.21.15.83:0/3931658193 conn(0x55e9b9fc0e60 msgr2=0x55e9b9fc1240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:14.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.676+0000 7f9123e00640 1 --2- 172.21.15.83:0/343334989 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f91080a3da0 0x7f91080a6260 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f911c005c40 tx=0x7f911c009560 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:14.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.676+0000 7f912d3dd740 1 -- 172.21.15.83:0/3931658193 shutdown_connections 2024-04-24T08:56:14.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.676+0000 7f912d3dd740 1 -- 172.21.15.83:0/3931658193 wait complete. 2024-04-24T08:56:14.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.677+0000 7f912d3dd740 1 -- start start 2024-04-24T08:56:14.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.677+0000 7f912d3dd740 1 Processor -- start 2024-04-24T08:56:14.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.677+0000 7f912d3dd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9b9fc4b40 0x55e9b9fcf5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:14.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.677+0000 7f912d3dd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e9b9fc5460 0x55e9b9fcc8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:14.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.677+0000 7f912d3dd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9b9fcfb30 0x55e9b9fccfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:14.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.677+0000 7f912d3dd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e9b9fd29c0 con 0x55e9b9fc4b40 2024-04-24T08:56:14.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.678+0000 7f912d3dd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e9b9ee3810 con 0x55e9b9fcfb30 2024-04-24T08:56:14.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.678+0000 7f912d3dd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55e9b9fd2b40 con 0x55e9b9fc5460 2024-04-24T08:56:14.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.678+0000 7f912d3dd740 10 client.0 _ll_register_callbacks cb 0x55e9b9eb1140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:56:14.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.678+0000 7f9123400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9b9fc4b40 0x55e9b9fcf5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:14.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.678+0000 7f9123400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9b9fc4b40 0x55e9b9fcf5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45828/0 (socket says 172.21.15.83:45828) 2024-04-24T08:56:14.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.678+0000 7f9123400640 1 -- 172.21.15.83:0/223408362 learned_addr learned my addr 172.21.15.83:0/223408362 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:56:14.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.678+0000 7f9123e00640 1 --2- 172.21.15.83:0/223408362 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9b9fcfb30 0x55e9b9fccfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:14.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.678+0000 7f9123400640 1 -- 172.21.15.83:0/223408362 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e9b9fc5460 msgr2=0x55e9b9fcc8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:14.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.678+0000 7f9123400640 1 --2- 172.21.15.83:0/223408362 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e9b9fc5460 0x55e9b9fcc8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.678+0000 7f9123400640 1 -- 172.21.15.83:0/223408362 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9b9fcfb30 msgr2=0x55e9b9fccfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:14.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.678+0000 7f9123400640 1 --2- 172.21.15.83:0/223408362 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9b9fcfb30 0x55e9b9fccfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.678+0000 7f9123400640 1 -- 172.21.15.83:0/223408362 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9b9ef43f0 con 0x55e9b9fc4b40 2024-04-24T08:56:14.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.678+0000 7f9123e00640 1 --2- 172.21.15.83:0/223408362 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9b9fcfb30 0x55e9b9fccfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:56:14.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.679+0000 7f9123400640 1 --2- 172.21.15.83:0/223408362 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9b9fc4b40 0x55e9b9fcf5c0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f9114005ab0 tx=0x7f91140316d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e59dbe4417fde56 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:14.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.679+0000 7f910d600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:56:14.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.679+0000 7f910d600640 1 -- 172.21.15.83:0/223408362 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f911403f070 con 0x55e9b9fc4b40 2024-04-24T08:56:14.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.679+0000 7f910d600640 1 -- 172.21.15.83:0/223408362 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f9114004450 con 0x55e9b9fc4b40 2024-04-24T08:56:14.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.679+0000 7f912d3dd740 1 -- 172.21.15.83:0/223408362 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f91100dd070 con 0x55e9b9fc4b40 2024-04-24T08:56:14.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.679+0000 7f910d600640 1 -- 172.21.15.83:0/223408362 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9114035070 con 0x55e9b9fc4b40 2024-04-24T08:56:14.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.679+0000 7f912d3dd740 10 client.17343 fetch_fsmap learned FSMap version 32 2024-04-24T08:56:14.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.679+0000 7f912d3dd740 1 -- 172.21.15.83:0/223408362 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55e9b9c3c3f0 con 0x55e9b9fc4b40 2024-04-24T08:56:14.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.679+0000 7f910d600640 1 -- 172.21.15.83:0/223408362 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f91100dd070 con 0x55e9b9fc4b40 2024-04-24T08:56:14.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.680+0000 7f910d600640 1 -- 172.21.15.83:0/223408362 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7f9114004ae0 con 0x55e9b9fc4b40 2024-04-24T08:56:14.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.680+0000 7f912d3dd740 10 client.17343 fetch_fsmap finished waiting for FSMap version 32 2024-04-24T08:56:14.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.680+0000 7f912d3dd740 10 client.17343 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-24T08:56:14.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.680+0000 7f912d3dd740 20 client.17343 populate_metadata read hostname 'smithi083' 2024-04-24T08:56:14.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.680+0000 7f912d3dd740 1 --2- 172.21.15.83:0/223408362 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x55e9b9fdfd70 0x55e9b9fe2230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:14.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.680+0000 7f912d3dd740 4 client.17343 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:56:14.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.680+0000 7f912d3dd740 1 -- 172.21.15.83:0/223408362 --> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e9b9f37bf0 con 0x55e9b9fdfd70 2024-04-24T08:56:14.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.681+0000 7f9122a00640 1 --2- 172.21.15.83:0/223408362 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x55e9b9fdfd70 0x55e9b9fe2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:14.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.681+0000 7f9122a00640 1 --2- 172.21.15.83:0/223408362 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x55e9b9fdfd70 0x55e9b9fe2230 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=516c3489e82b2802 server_cookie=6edb02401d772074 in_seq=0 out_seq=0 2024-04-24T08:56:14.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.681+0000 7f910d600640 10 client.17343 ms_handle_connect on v2:172.21.15.138:6836/900566754 2024-04-24T08:56:14.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.683+0000 7f910d600640 1 -- 172.21.15.83:0/223408362 <== mds.1 v2:172.21.15.138:6836/900566754 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e9b9f37bf0 con 0x55e9b9fdfd70 2024-04-24T08:56:14.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.683+0000 7f910d600640 10 client.17343 handle_command_reply: tid=0 2024-04-24T08:56:14.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.683+0000 7f912d3dd740 1 client.17343 shutdown 2024-04-24T08:56:14.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.684+0000 7f912d3dd740 1 -- 172.21.15.83:0/223408362 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9b9fc4b40 msgr2=0x55e9b9fcf5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:14.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.684+0000 7f912d3dd740 1 --2- 172.21.15.83:0/223408362 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9b9fc4b40 0x55e9b9fcf5c0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f9114005ab0 tx=0x7f91140316d0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.684+0000 7f912d3dd740 1 -- 172.21.15.83:0/223408362 shutdown_connections 2024-04-24T08:56:14.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.684+0000 7f912d3dd740 1 --2- 172.21.15.83:0/223408362 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9b9fcfb30 0x55e9b9fccfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.684+0000 7f912d3dd740 1 --2- 172.21.15.83:0/223408362 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x55e9b9fdfd70 0x55e9b9fe2230 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.684+0000 7f912d3dd740 1 --2- 172.21.15.83:0/223408362 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e9b9fc5460 0x55e9b9fcc8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.684+0000 7f912d3dd740 1 --2- 172.21.15.83:0/223408362 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9b9fc4b40 0x55e9b9fcf5c0 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.684+0000 7f910d600640 0 client.17343 ms_handle_reset on v2:172.21.15.138:6836/900566754 2024-04-24T08:56:14.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.684+0000 7f912d3dd740 1 -- 172.21.15.83:0/223408362 >> 172.21.15.83:0/223408362 conn(0x55e9b9fc0e60 msgr2=0x55e9b9fc6c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:14.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.684+0000 7f912d3dd740 1 -- 172.21.15.83:0/223408362 shutdown_connections 2024-04-24T08:56:14.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.684+0000 7f912d3dd740 1 -- 172.21.15.83:0/223408362 wait complete. 2024-04-24T08:56:14.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.684+0000 7f912d3dd740 20 client.17343 trim_cache size 0 max 16384 2024-04-24T08:56:14.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.700+0000 7f912d3dd740 1 Processor -- start 2024-04-24T08:56:14.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.700+0000 7f912d3dd740 1 -- start start 2024-04-24T08:56:14.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.700+0000 7f912d3dd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9b9fc4b40 0x55e9b9fdb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:14.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.700+0000 7f912d3dd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e9b9fc5460 0x55e9b9fd1e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:14.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.700+0000 7f912d3dd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9b9fcfb30 0x55e9b9fcf0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:14.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.700+0000 7f912d3dd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f91241b1560 con 0x55e9b9fc4b40 2024-04-24T08:56:14.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.700+0000 7f912d3dd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f91241b30d0 con 0x55e9b9fcfb30 2024-04-24T08:56:14.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.700+0000 7f912d3dd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f91241b1c30 con 0x55e9b9fc5460 2024-04-24T08:56:14.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.701+0000 7f9123400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9b9fc4b40 0x55e9b9fdb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:14.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.701+0000 7f9123400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9b9fc4b40 0x55e9b9fdb8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45840/0 (socket says 172.21.15.83:45840) 2024-04-24T08:56:14.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.701+0000 7f9123400640 1 -- 172.21.15.83:0/1705147978 learned_addr learned my addr 172.21.15.83:0/1705147978 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:56:14.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.701+0000 7f9123400640 1 -- 172.21.15.83:0/1705147978 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e9b9fc5460 msgr2=0x55e9b9fd1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:14.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.701+0000 7f9123400640 1 --2- 172.21.15.83:0/1705147978 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e9b9fc5460 0x55e9b9fd1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.701+0000 7f9123400640 1 -- 172.21.15.83:0/1705147978 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9b9fcfb30 msgr2=0x55e9b9fcf0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:14.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.701+0000 7f9123400640 1 --2- 172.21.15.83:0/1705147978 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9b9fcfb30 0x55e9b9fcf0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.701+0000 7f9123400640 1 -- 172.21.15.83:0/1705147978 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9b9fcd310 con 0x55e9b9fc4b40 2024-04-24T08:56:14.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.701+0000 7f9123400640 1 --2- 172.21.15.83:0/1705147978 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9b9fc4b40 0x55e9b9fdb8b0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f9114009850 tx=0x7f911402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b57815136781d978 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:14.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.701+0000 7f910fe00640 1 -- 172.21.15.83:0/1705147978 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9114056070 con 0x55e9b9fc4b40 2024-04-24T08:56:14.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.701+0000 7f910fe00640 1 -- 172.21.15.83:0/1705147978 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9114003910 con 0x55e9b9fc4b40 2024-04-24T08:56:14.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.701+0000 7f910fe00640 1 -- 172.21.15.83:0/1705147978 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f911403f070 con 0x55e9b9fc4b40 2024-04-24T08:56:14.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.702+0000 7f912d3dd740 1 -- 172.21.15.83:0/1705147978 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9b9fc4b40 msgr2=0x55e9b9fdb8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:14.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.702+0000 7f912d3dd740 1 --2- 172.21.15.83:0/1705147978 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9b9fc4b40 0x55e9b9fdb8b0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f9114009850 tx=0x7f911402a410 comp rx=0 tx=0).stop 2024-04-24T08:56:14.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.702+0000 7f912d3dd740 1 -- 172.21.15.83:0/1705147978 shutdown_connections 2024-04-24T08:56:14.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.702+0000 7f912d3dd740 1 --2- 172.21.15.83:0/1705147978 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9b9fcfb30 0x55e9b9fcf0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.702+0000 7f912d3dd740 1 --2- 172.21.15.83:0/1705147978 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e9b9fc5460 0x55e9b9fd1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.702+0000 7f912d3dd740 1 --2- 172.21.15.83:0/1705147978 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9b9fc4b40 0x55e9b9fdb8b0 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.702+0000 7f912d3dd740 1 -- 172.21.15.83:0/1705147978 >> 172.21.15.83:0/1705147978 conn(0x55e9b9fc0e60 msgr2=0x55e9b9fcb180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:14.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.702+0000 7f912d3dd740 1 -- 172.21.15.83:0/1705147978 shutdown_connections 2024-04-24T08:56:14.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.702+0000 7f912d3dd740 1 -- 172.21.15.83:0/1705147978 wait complete. 2024-04-24T08:56:14.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.703+0000 7f912d3dd740 1 -- start start 2024-04-24T08:56:14.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.703+0000 7f912d3dd740 1 Processor -- start 2024-04-24T08:56:14.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.703+0000 7f912d3dd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e9b9fc4b40 0x55e9b9fd0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:14.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.703+0000 7f912d3dd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9b9fc5460 0x55e9b9fd0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:14.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.703+0000 7f912d3dd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9b9fcfb30 0x55e9b9fd6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:14.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.703+0000 7f912d3dd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f91241b29d0 con 0x55e9b9fcfb30 2024-04-24T08:56:14.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.703+0000 7f912d3dd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f91241b2300 con 0x55e9b9fc5460 2024-04-24T08:56:14.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.703+0000 7f912d3dd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f91241af540 con 0x55e9b9fc4b40 2024-04-24T08:56:14.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.703+0000 7f912d3dd740 10 client.0 _ll_register_callbacks cb 0x7f9114009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:56:14.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.704+0000 7f9123e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9b9fcfb30 0x55e9b9fd6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:14.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.704+0000 7f9123e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9b9fcfb30 0x55e9b9fd6dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45854/0 (socket says 172.21.15.83:45854) 2024-04-24T08:56:14.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.704+0000 7f9123e00640 1 -- 172.21.15.83:0/4236597828 learned_addr learned my addr 172.21.15.83:0/4236597828 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:56:14.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.704+0000 7f9123400640 1 --2- 172.21.15.83:0/4236597828 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e9b9fc4b40 0x55e9b9fd0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:14.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.704+0000 7f9122a00640 1 --2- 172.21.15.83:0/4236597828 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9b9fc5460 0x55e9b9fd0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:14.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.704+0000 7f9123e00640 1 -- 172.21.15.83:0/4236597828 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e9b9fc4b40 msgr2=0x55e9b9fd0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:14.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.704+0000 7f9123e00640 1 --2- 172.21.15.83:0/4236597828 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e9b9fc4b40 0x55e9b9fd0770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.704+0000 7f9123e00640 1 -- 172.21.15.83:0/4236597828 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9b9fc5460 msgr2=0x55e9b9fd0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:14.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.704+0000 7f9123e00640 1 --2- 172.21.15.83:0/4236597828 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9b9fc5460 0x55e9b9fd0cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.704+0000 7f9123e00640 1 -- 172.21.15.83:0/4236597828 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9b9d067d0 con 0x55e9b9fcfb30 2024-04-24T08:56:14.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.705+0000 7f9123e00640 1 --2- 172.21.15.83:0/4236597828 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9b9fcfb30 0x55e9b9fd6dc0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f911c0127b0 tx=0x7f911c005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fdca793df8af82e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:14.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.705+0000 7f910d600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:56:14.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.705+0000 7f910d600640 1 -- 172.21.15.83:0/4236597828 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f911c01eab0 con 0x55e9b9fcfb30 2024-04-24T08:56:14.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.705+0000 7f910d600640 1 -- 172.21.15.83:0/4236597828 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f911c01ec50 con 0x55e9b9fcfb30 2024-04-24T08:56:14.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.705+0000 7f912d3dd740 1 -- 172.21.15.83:0/4236597828 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9114003910 con 0x55e9b9fcfb30 2024-04-24T08:56:14.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.705+0000 7f910d600640 1 -- 172.21.15.83:0/4236597828 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f911c032050 con 0x55e9b9fcfb30 2024-04-24T08:56:14.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.705+0000 7f912d3dd740 10 client.17349 fetch_fsmap learned FSMap version 32 2024-04-24T08:56:14.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.705+0000 7f912d3dd740 1 -- 172.21.15.83:0/4236597828 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55e9b9fd8390 con 0x55e9b9fcfb30 2024-04-24T08:56:14.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.706+0000 7f910d600640 1 -- 172.21.15.83:0/4236597828 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9114003910 con 0x55e9b9fcfb30 2024-04-24T08:56:14.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.706+0000 7f910d600640 1 -- 172.21.15.83:0/4236597828 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7f911c00f040 con 0x55e9b9fcfb30 2024-04-24T08:56:14.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.706+0000 7f912d3dd740 10 client.17349 fetch_fsmap finished waiting for FSMap version 32 2024-04-24T08:56:14.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.706+0000 7f912d3dd740 10 client.17349 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-24T08:56:14.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.706+0000 7f912d3dd740 20 client.17349 populate_metadata read hostname 'smithi083' 2024-04-24T08:56:14.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.706+0000 7f912d3dd740 1 --2- 172.21.15.83:0/4236597828 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x55e9ba046640 0x55e9ba048a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:14.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.706+0000 7f912d3dd740 4 client.17349 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T08:56:14.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.706+0000 7f912d3dd740 1 -- 172.21.15.83:0/4236597828 --> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e9b9f31a60 con 0x55e9ba046640 2024-04-24T08:56:14.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.706+0000 7f9123400640 1 --2- 172.21.15.83:0/4236597828 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x55e9ba046640 0x55e9ba048a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:14.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.707+0000 7f9123400640 1 --2- 172.21.15.83:0/4236597828 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x55e9ba046640 0x55e9ba048a60 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3803d703a823efea server_cookie=7e766cdca7f258d9 in_seq=0 out_seq=0 2024-04-24T08:56:14.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.707+0000 7f910d600640 10 client.17349 ms_handle_connect on v2:172.21.15.138:6836/900566754 2024-04-24T08:56:14.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.708+0000 7f910d600640 1 -- 172.21.15.83:0/4236597828 <== mds.1 v2:172.21.15.138:6836/900566754 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e9b9f31a60 con 0x55e9ba046640 2024-04-24T08:56:14.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.708+0000 7f910d600640 10 client.17349 handle_command_reply: tid=0 2024-04-24T08:56:14.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.708+0000 7f912d3dd740 1 client.17349 shutdown 2024-04-24T08:56:14.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.708+0000 7f912d3dd740 1 -- 172.21.15.83:0/4236597828 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9b9fcfb30 msgr2=0x55e9b9fd6dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:14.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.708+0000 7f912d3dd740 1 --2- 172.21.15.83:0/4236597828 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9b9fcfb30 0x55e9b9fd6dc0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f911c0127b0 tx=0x7f911c005bd0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.708+0000 7f912d3dd740 1 -- 172.21.15.83:0/4236597828 shutdown_connections 2024-04-24T08:56:14.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.708+0000 7f912d3dd740 1 --2- 172.21.15.83:0/4236597828 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9b9fcfb30 0x55e9b9fd6dc0 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.708+0000 7f912d3dd740 1 --2- 172.21.15.83:0/4236597828 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9b9fc5460 0x55e9b9fd0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.708+0000 7f912d3dd740 1 --2- 172.21.15.83:0/4236597828 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x55e9ba046640 0x55e9ba048a60 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.708+0000 7f912d3dd740 1 --2- 172.21.15.83:0/4236597828 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e9b9fc4b40 0x55e9b9fd0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.708+0000 7f912d3dd740 1 -- 172.21.15.83:0/4236597828 >> 172.21.15.83:0/4236597828 conn(0x55e9b9fc0e60 msgr2=0x55e9b9fbf6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:14.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.708+0000 7f910d600640 0 client.17349 ms_handle_reset on v2:172.21.15.138:6836/900566754 2024-04-24T08:56:14.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.709+0000 7f912d3dd740 1 -- 172.21.15.83:0/4236597828 shutdown_connections 2024-04-24T08:56:14.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.709+0000 7f912d3dd740 1 -- 172.21.15.83:0/4236597828 wait complete. 2024-04-24T08:56:14.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.709+0000 7f912d3dd740 20 client.17349 trim_cache size 0 max 16384 2024-04-24T08:56:14.712 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T08:56:14.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.709+0000 7f9128e00640 1 -- 172.21.15.83:0/343334989 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f91080a3da0 msgr2=0x7f91080a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:14.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.709+0000 7f9128e00640 1 --2- 172.21.15.83:0/343334989 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f91080a3da0 0x7f91080a6260 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f911c005c40 tx=0x7f911c009560 comp rx=0 tx=0).stop 2024-04-24T08:56:14.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.709+0000 7f9128e00640 1 -- 172.21.15.83:0/343334989 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f91241015f0 msgr2=0x7f91241a0b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:14.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.709+0000 7f9128e00640 1 --2- 172.21.15.83:0/343334989 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f91241015f0 0x7f91241a0b30 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f911000d770 tx=0x7f911000dc30 comp rx=0 tx=0).stop 2024-04-24T08:56:14.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.710+0000 7f9128e00640 1 -- 172.21.15.83:0/343334989 shutdown_connections 2024-04-24T08:56:14.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.710+0000 7f9128e00640 1 --2- 172.21.15.83:0/343334989 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f91080a3da0 0x7f91080a6260 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.710+0000 7f9128e00640 1 --2- 172.21.15.83:0/343334989 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f912410e7f0 0x7f912419a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.710+0000 7f9128e00640 1 --2- 172.21.15.83:0/343334989 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f91241015f0 0x7f91241a0b30 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.710+0000 7f9128e00640 1 --2- 172.21.15.83:0/343334989 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9124100c40 0x7f91241a05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.710+0000 7f9128e00640 1 -- 172.21.15.83:0/343334989 >> 172.21.15.83:0/343334989 conn(0x7f91240fc880 msgr2=0x7f91240fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:14.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.710+0000 7f9128e00640 1 -- 172.21.15.83:0/343334989 shutdown_connections 2024-04-24T08:56:14.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.710+0000 7f9128e00640 1 -- 172.21.15.83:0/343334989 wait complete. 2024-04-24T08:56:14.727 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:56:14 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/4277759247' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:56:14.729 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T08:56:14.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.933+0000 7f32dde00640 1 -- 172.21.15.83:0/3892619551 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32d8103280 msgr2=0x7f32d8103660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:14.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.933+0000 7f32dde00640 1 --2- 172.21.15.83:0/3892619551 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32d8103280 0x7f32d8103660 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f32c8009ca0 tx=0x7f32c802a9e0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.934+0000 7f32dde00640 1 -- 172.21.15.83:0/3892619551 shutdown_connections 2024-04-24T08:56:14.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.934+0000 7f32dde00640 1 --2- 172.21.15.83:0/3892619551 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f32d810c5c0 0x7f32d810ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.934+0000 7f32dde00640 1 --2- 172.21.15.83:0/3892619551 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32d8103ba0 0x7f32d810c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.934+0000 7f32dde00640 1 --2- 172.21.15.83:0/3892619551 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32d8103280 0x7f32d8103660 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.934+0000 7f32dde00640 1 -- 172.21.15.83:0/3892619551 >> 172.21.15.83:0/3892619551 conn(0x7f32d8078000 msgr2=0x7f32d8078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:14.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.934+0000 7f32dde00640 1 -- 172.21.15.83:0/3892619551 shutdown_connections 2024-04-24T08:56:14.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.934+0000 7f32dde00640 1 -- 172.21.15.83:0/3892619551 wait complete. 2024-04-24T08:56:14.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.935+0000 7f32dde00640 1 Processor -- start 2024-04-24T08:56:14.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.935+0000 7f32dde00640 1 -- start start 2024-04-24T08:56:14.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.935+0000 7f32dde00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f32d8103280 0x7f32d81a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:14.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.935+0000 7f32dde00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32d8103ba0 0x7f32d81a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:14.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.935+0000 7f32dde00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32d810c5c0 0x7f32d819a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:14.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.935+0000 7f32dde00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f32d81122a0 con 0x7f32d810c5c0 2024-04-24T08:56:14.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.935+0000 7f32dde00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f32d8112120 con 0x7f32d8103ba0 2024-04-24T08:56:14.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.935+0000 7f32dde00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f32d8112420 con 0x7f32d8103280 2024-04-24T08:56:14.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.935+0000 7f32d7400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32d810c5c0 0x7f32d819a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:14.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.936+0000 7f32d7400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32d810c5c0 0x7f32d819a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45876/0 (socket says 172.21.15.83:45876) 2024-04-24T08:56:14.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.936+0000 7f32d7400640 1 -- 172.21.15.83:0/3043701762 learned_addr learned my addr 172.21.15.83:0/3043701762 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:56:14.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.936+0000 7f32d6a00640 1 --2- 172.21.15.83:0/3043701762 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f32d8103280 0x7f32d81a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:14.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.936+0000 7f32ce000640 1 --2- 172.21.15.83:0/3043701762 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32d8103ba0 0x7f32d81a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:14.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.936+0000 7f32d7400640 1 -- 172.21.15.83:0/3043701762 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f32d8103280 msgr2=0x7f32d81a0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:14.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.936+0000 7f32d7400640 1 --2- 172.21.15.83:0/3043701762 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f32d8103280 0x7f32d81a0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.936+0000 7f32d7400640 1 -- 172.21.15.83:0/3043701762 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32d8103ba0 msgr2=0x7f32d81a0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:14.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.936+0000 7f32d7400640 1 --2- 172.21.15.83:0/3043701762 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32d8103ba0 0x7f32d81a0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.936+0000 7f32d7400640 1 -- 172.21.15.83:0/3043701762 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32d819b010 con 0x7f32d810c5c0 2024-04-24T08:56:14.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.936+0000 7f32ce000640 1 --2- 172.21.15.83:0/3043701762 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32d8103ba0 0x7f32d81a0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:56:14.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.936+0000 7f32d7400640 1 --2- 172.21.15.83:0/3043701762 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32d810c5c0 0x7f32d819a840 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f32d000d750 tx=0x7f32d000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:14.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.937+0000 7f32dca00640 1 -- 172.21.15.83:0/3043701762 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d0017070 con 0x7f32d810c5c0 2024-04-24T08:56:14.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.937+0000 7f32dca00640 1 -- 172.21.15.83:0/3043701762 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f32d00054d0 con 0x7f32d810c5c0 2024-04-24T08:56:14.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.937+0000 7f32dca00640 1 -- 172.21.15.83:0/3043701762 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d0004c60 con 0x7f32d810c5c0 2024-04-24T08:56:14.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.937+0000 7f32dde00640 1 -- 172.21.15.83:0/3043701762 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32d819b300 con 0x7f32d810c5c0 2024-04-24T08:56:14.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.937+0000 7f32dde00640 1 -- 172.21.15.83:0/3043701762 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32d81a74d0 con 0x7f32d810c5c0 2024-04-24T08:56:14.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.940+0000 7f32dca00640 1 -- 172.21.15.83:0/3043701762 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f32d0004e00 con 0x7f32d810c5c0 2024-04-24T08:56:14.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.940+0000 7f32e0be7740 1 Processor -- start 2024-04-24T08:56:14.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.940+0000 7f32e0be7740 1 -- start start 2024-04-24T08:56:14.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.940+0000 7f32e0be7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5598f4ab5950 0x5598f4ab5d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:14.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.941+0000 7f32e0be7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5598f4ab6270 0x5598f4ac0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:14.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.941+0000 7f32e0be7740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5598f4ac0940 0x5598f4ac2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:14.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.941+0000 7f32e0be7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5598f4a1b1d0 con 0x5598f4ab6270 2024-04-24T08:56:14.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.941+0000 7f32e0be7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5598f49c1450 con 0x5598f4ab5950 2024-04-24T08:56:14.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.941+0000 7f32e0be7740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5598f4a17190 con 0x5598f4ac0940 2024-04-24T08:56:14.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.941+0000 7f32ce000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5598f4ab5950 0x5598f4ab5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:14.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.941+0000 7f32d6a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5598f4ab6270 0x5598f4ac0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:14.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.941+0000 7f32ce000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5598f4ab5950 0x5598f4ab5d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:42634/0 (socket says 172.21.15.83:42634) 2024-04-24T08:56:14.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.941+0000 7f32d6a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5598f4ab6270 0x5598f4ac0400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45890/0 (socket says 172.21.15.83:45890) 2024-04-24T08:56:14.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.941+0000 7f32ce000640 1 -- 172.21.15.83:0/2035768178 learned_addr learned my addr 172.21.15.83:0/2035768178 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:56:14.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.941+0000 7f32ce000640 1 --2- 172.21.15.83:0/2035768178 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5598f4ac0940 0x5598f4ac2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:14.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.942+0000 7f32dca00640 1 --2- 172.21.15.83:0/3043701762 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f32bc0a3d20 0x7f32bc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:14.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.942+0000 7f32d6a00640 1 -- 172.21.15.83:0/2035768178 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5598f4ac0940 msgr2=0x5598f4ac2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:14.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.942+0000 7f32d6a00640 1 --2- 172.21.15.83:0/2035768178 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5598f4ac0940 0x5598f4ac2d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.942+0000 7f32d6a00640 1 -- 172.21.15.83:0/2035768178 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5598f4ab5950 msgr2=0x5598f4ab5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:14.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.942+0000 7f32d6a00640 1 --2- 172.21.15.83:0/2035768178 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5598f4ab5950 0x5598f4ab5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.942+0000 7f32d6a00640 1 -- 172.21.15.83:0/2035768178 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598f4a7e310 con 0x5598f4ab6270 2024-04-24T08:56:14.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.942+0000 7f32dca00640 1 -- 172.21.15.83:0/3043701762 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f32d00d0600 con 0x7f32d810c5c0 2024-04-24T08:56:14.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.942+0000 7f32d7400640 1 --2- 172.21.15.83:0/3043701762 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f32bc0a3d20 0x7f32bc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:14.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.942+0000 7f32d6a00640 1 --2- 172.21.15.83:0/2035768178 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5598f4ab6270 0x5598f4ac0400 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f32c8006010 tx=0x7f32c802ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2b65f581b84efa7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:14.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.942+0000 7f32c7e00640 1 -- 172.21.15.83:0/2035768178 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c8047070 con 0x5598f4ab6270 2024-04-24T08:56:14.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.942+0000 7f32c7e00640 1 -- 172.21.15.83:0/2035768178 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f32c80098d0 con 0x5598f4ab6270 2024-04-24T08:56:14.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.942+0000 7f32c7e00640 1 -- 172.21.15.83:0/2035768178 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c8033040 con 0x5598f4ab6270 2024-04-24T08:56:14.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.943+0000 7f32e0be7740 1 -- 172.21.15.83:0/2035768178 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5598f4ab6270 msgr2=0x5598f4ac0400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:14.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.943+0000 7f32e0be7740 1 --2- 172.21.15.83:0/2035768178 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5598f4ab6270 0x5598f4ac0400 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f32c8006010 tx=0x7f32c802ed90 comp rx=0 tx=0).stop 2024-04-24T08:56:14.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.943+0000 7f32d7400640 1 --2- 172.21.15.83:0/3043701762 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f32bc0a3d20 0x7f32bc0a61e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f32d0009f50 tx=0x7f32d0009ea0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:14.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.943+0000 7f32e0be7740 1 -- 172.21.15.83:0/2035768178 shutdown_connections 2024-04-24T08:56:14.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.943+0000 7f32e0be7740 1 --2- 172.21.15.83:0/2035768178 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5598f4ac0940 0x5598f4ac2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.943+0000 7f32e0be7740 1 --2- 172.21.15.83:0/2035768178 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5598f4ab6270 0x5598f4ac0400 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.943+0000 7f32e0be7740 1 --2- 172.21.15.83:0/2035768178 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5598f4ab5950 0x5598f4ab5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.943+0000 7f32e0be7740 1 -- 172.21.15.83:0/2035768178 >> 172.21.15.83:0/2035768178 conn(0x5598f4ab1c70 msgr2=0x5598f4ab2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:14.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.943+0000 7f32e0be7740 1 -- 172.21.15.83:0/2035768178 shutdown_connections 2024-04-24T08:56:14.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.943+0000 7f32e0be7740 1 -- 172.21.15.83:0/2035768178 wait complete. 2024-04-24T08:56:14.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.944+0000 7f32e0be7740 1 -- start start 2024-04-24T08:56:14.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.944+0000 7f32e0be7740 1 Processor -- start 2024-04-24T08:56:14.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.945+0000 7f32e0be7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5598f4ab5950 0x5598f4ac03b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:14.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.945+0000 7f32e0be7740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5598f4ab6270 0x5598f4abd6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:14.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.945+0000 7f32e0be7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5598f4ac0940 0x5598f4abdde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:14.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.945+0000 7f32e0be7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5598f4ac3770 con 0x5598f4ac0940 2024-04-24T08:56:14.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.945+0000 7f32e0be7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5598f49d4810 con 0x5598f4ab5950 2024-04-24T08:56:14.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.945+0000 7f32e0be7740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5598f4ac38f0 con 0x5598f4ab6270 2024-04-24T08:56:14.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.945+0000 7f32e0be7740 10 client.0 _ll_register_callbacks cb 0x5598f49a2140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:56:14.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.945+0000 7f32ce000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5598f4ac0940 0x5598f4abdde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:14.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.945+0000 7f32ce000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5598f4ac0940 0x5598f4abdde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45892/0 (socket says 172.21.15.83:45892) 2024-04-24T08:56:14.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.945+0000 7f32ce000640 1 -- 172.21.15.83:0/3737538975 learned_addr learned my addr 172.21.15.83:0/3737538975 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:56:14.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.945+0000 7f32ce000640 1 --2- 172.21.15.83:0/3737538975 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5598f4ab5950 0x5598f4ac03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:14.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.945+0000 7f32d6a00640 1 --2- 172.21.15.83:0/3737538975 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5598f4ab6270 0x5598f4abd6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:14.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.945+0000 7f32ce000640 1 -- 172.21.15.83:0/3737538975 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5598f4ab6270 msgr2=0x5598f4abd6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:14.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.945+0000 7f32ce000640 1 --2- 172.21.15.83:0/3737538975 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5598f4ab6270 0x5598f4abd6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.945+0000 7f32ce000640 1 -- 172.21.15.83:0/3737538975 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5598f4ab5950 msgr2=0x5598f4ac03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:14.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.945+0000 7f32ce000640 1 --2- 172.21.15.83:0/3737538975 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5598f4ab5950 0x5598f4ac03b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.945+0000 7f32ce000640 1 -- 172.21.15.83:0/3737538975 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598f49e53f0 con 0x5598f4ac0940 2024-04-24T08:56:14.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.946+0000 7f32ce000640 1 --2- 172.21.15.83:0/3737538975 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5598f4ac0940 0x5598f4abdde0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f32c0012cb0 tx=0x7f32c0014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a37c5968e3b50697 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:14.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.946+0000 7f32c4c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:56:14.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.946+0000 7f32c4c00640 1 -- 172.21.15.83:0/3737538975 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c001f420 con 0x5598f4ac0940 2024-04-24T08:56:14.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.946+0000 7f32c4c00640 1 -- 172.21.15.83:0/3737538975 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f32c001e3c0 con 0x5598f4ac0940 2024-04-24T08:56:14.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.946+0000 7f32e0be7740 1 -- 172.21.15.83:0/3737538975 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5598f4a28bf0 con 0x5598f4ac0940 2024-04-24T08:56:14.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.946+0000 7f32c4c00640 1 -- 172.21.15.83:0/3737538975 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f32c0005020 con 0x5598f4ac0940 2024-04-24T08:56:14.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.946+0000 7f32e0be7740 10 client.17361 fetch_fsmap learned FSMap version 32 2024-04-24T08:56:14.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.946+0000 7f32e0be7740 1 -- 172.21.15.83:0/3737538975 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5598f472d3f0 con 0x5598f4ac0940 2024-04-24T08:56:14.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.947+0000 7f32c4c00640 1 -- 172.21.15.83:0/3737538975 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x5598f4a28bf0 con 0x5598f4ac0940 2024-04-24T08:56:14.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.947+0000 7f32c4c00640 1 -- 172.21.15.83:0/3737538975 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7f32c0004bb0 con 0x5598f4ac0940 2024-04-24T08:56:14.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.947+0000 7f32e0be7740 10 client.17361 fetch_fsmap finished waiting for FSMap version 32 2024-04-24T08:56:14.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.947+0000 7f32e0be7740 10 client.17361 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:56:14.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.947+0000 7f32e0be7740 20 client.17361 populate_metadata read hostname 'smithi083' 2024-04-24T08:56:14.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.947+0000 7f32e0be7740 1 --2- 172.21.15.83:0/3737538975 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5598f4ad0b50 0x5598f4ad2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:14.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.947+0000 7f32e0be7740 4 client.17361 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:56:14.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.947+0000 7f32e0be7740 1 -- 172.21.15.83:0/3737538975 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5598f4a22a60 con 0x5598f4ad0b50 2024-04-24T08:56:14.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.948+0000 7f32d6a00640 1 --2- 172.21.15.83:0/3737538975 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5598f4ad0b50 0x5598f4ad2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:14.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.949+0000 7f32d6a00640 1 --2- 172.21.15.83:0/3737538975 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5598f4ad0b50 0x5598f4ad2fd0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33448105dfd4159f server_cookie=7a02d386b710ff6c in_seq=0 out_seq=0 2024-04-24T08:56:14.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.949+0000 7f32c4c00640 10 client.17361 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:56:14.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.951+0000 7f32c4c00640 1 -- 172.21.15.83:0/3737538975 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5598f4a22a60 con 0x5598f4ad0b50 2024-04-24T08:56:14.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.951+0000 7f32c4c00640 10 client.17361 handle_command_reply: tid=0 2024-04-24T08:56:14.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.951+0000 7f32e0be7740 1 client.17361 shutdown 2024-04-24T08:56:14.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.951+0000 7f32e0be7740 1 -- 172.21.15.83:0/3737538975 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5598f4ac0940 msgr2=0x5598f4abdde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:14.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.951+0000 7f32e0be7740 1 --2- 172.21.15.83:0/3737538975 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5598f4ac0940 0x5598f4abdde0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f32c0012cb0 tx=0x7f32c0014040 comp rx=0 tx=0).stop 2024-04-24T08:56:14.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.951+0000 7f32e0be7740 1 -- 172.21.15.83:0/3737538975 shutdown_connections 2024-04-24T08:56:14.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.951+0000 7f32e0be7740 1 --2- 172.21.15.83:0/3737538975 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5598f4ad0b50 0x5598f4ad2fd0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.951+0000 7f32e0be7740 1 --2- 172.21.15.83:0/3737538975 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5598f4ac0940 0x5598f4abdde0 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.951+0000 7f32e0be7740 1 --2- 172.21.15.83:0/3737538975 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5598f4ab6270 0x5598f4abd6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.951+0000 7f32e0be7740 1 --2- 172.21.15.83:0/3737538975 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5598f4ab5950 0x5598f4ac03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.951+0000 7f32e0be7740 1 -- 172.21.15.83:0/3737538975 >> 172.21.15.83:0/3737538975 conn(0x5598f4ab1c70 msgr2=0x5598f4ab7930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:14.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.951+0000 7f32c4c00640 0 client.17361 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:56:14.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.951+0000 7f32e0be7740 1 -- 172.21.15.83:0/3737538975 shutdown_connections 2024-04-24T08:56:14.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.951+0000 7f32e0be7740 1 -- 172.21.15.83:0/3737538975 wait complete. 2024-04-24T08:56:14.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.951+0000 7f32e0be7740 20 client.17361 trim_cache size 0 max 16384 2024-04-24T08:56:14.954 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:56:14 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/4277759247' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:56:14.956 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:56:14 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/4277759247' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:56:14.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.967+0000 7f32e0be7740 1 Processor -- start 2024-04-24T08:56:14.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.967+0000 7f32e0be7740 1 -- start start 2024-04-24T08:56:14.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.967+0000 7f32e0be7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5598f4ab5950 0x5598f4ab80b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:14.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.967+0000 7f32e0be7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5598f4ab6270 0x5598f4abcf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:14.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.967+0000 7f32e0be7740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5598f4ac0940 0x5598f4ad2b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:14.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.968+0000 7f32e0be7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f32d81b1740 con 0x5598f4ab5950 2024-04-24T08:56:14.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.968+0000 7f32e0be7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f32d81b32b0 con 0x5598f4ab6270 2024-04-24T08:56:14.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.968+0000 7f32e0be7740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f32d81b1e10 con 0x5598f4ac0940 2024-04-24T08:56:14.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.968+0000 7f32ce000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5598f4ab5950 0x5598f4ab80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:14.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.968+0000 7f32ce000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5598f4ab5950 0x5598f4ab80b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45902/0 (socket says 172.21.15.83:45902) 2024-04-24T08:56:14.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.968+0000 7f32ce000640 1 -- 172.21.15.83:0/856734202 learned_addr learned my addr 172.21.15.83:0/856734202 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:56:14.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.968+0000 7f32d6a00640 1 --2- 172.21.15.83:0/856734202 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5598f4ab6270 0x5598f4abcf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:14.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.968+0000 7f32ce000640 1 -- 172.21.15.83:0/856734202 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5598f4ac0940 msgr2=0x5598f4ad2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:14.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.968+0000 7f32ce000640 1 --2- 172.21.15.83:0/856734202 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5598f4ac0940 0x5598f4ad2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.968+0000 7f32ce000640 1 -- 172.21.15.83:0/856734202 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5598f4ab6270 msgr2=0x5598f4abcf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:14.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.968+0000 7f32ce000640 1 --2- 172.21.15.83:0/856734202 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5598f4ab6270 0x5598f4abcf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.968+0000 7f32ce000640 1 -- 172.21.15.83:0/856734202 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598f47f77d0 con 0x5598f4ab5950 2024-04-24T08:56:14.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.969+0000 7f32ce000640 1 --2- 172.21.15.83:0/856734202 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5598f4ab5950 0x5598f4ab80b0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f32c0004f50 tx=0x7f32c0012530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7a336901a1d45c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:14.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.969+0000 7f32c7400640 1 -- 172.21.15.83:0/856734202 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c0034070 con 0x5598f4ab5950 2024-04-24T08:56:14.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.969+0000 7f32c7400640 1 -- 172.21.15.83:0/856734202 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f32c0005be0 con 0x5598f4ab5950 2024-04-24T08:56:14.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.969+0000 7f32c7400640 1 -- 172.21.15.83:0/856734202 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c0004be0 con 0x5598f4ab5950 2024-04-24T08:56:14.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.970+0000 7f32e0be7740 1 -- 172.21.15.83:0/856734202 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5598f4ab5950 msgr2=0x5598f4ab80b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:14.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.970+0000 7f32e0be7740 1 --2- 172.21.15.83:0/856734202 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5598f4ab5950 0x5598f4ab80b0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f32c0004f50 tx=0x7f32c0012530 comp rx=0 tx=0).stop 2024-04-24T08:56:14.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.970+0000 7f32e0be7740 1 -- 172.21.15.83:0/856734202 shutdown_connections 2024-04-24T08:56:14.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.970+0000 7f32e0be7740 1 --2- 172.21.15.83:0/856734202 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5598f4ac0940 0x5598f4ad2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.970+0000 7f32e0be7740 1 --2- 172.21.15.83:0/856734202 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5598f4ab6270 0x5598f4abcf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.970+0000 7f32e0be7740 1 --2- 172.21.15.83:0/856734202 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5598f4ab5950 0x5598f4ab80b0 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.970+0000 7f32e0be7740 1 -- 172.21.15.83:0/856734202 >> 172.21.15.83:0/856734202 conn(0x5598f4ab1c70 msgr2=0x5598f4ab0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:14.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.970+0000 7f32e0be7740 1 -- 172.21.15.83:0/856734202 shutdown_connections 2024-04-24T08:56:14.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.970+0000 7f32e0be7740 1 -- 172.21.15.83:0/856734202 wait complete. 2024-04-24T08:56:14.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.971+0000 7f32e0be7740 1 -- start start 2024-04-24T08:56:14.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.971+0000 7f32e0be7740 1 Processor -- start 2024-04-24T08:56:14.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.971+0000 7f32e0be7740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5598f4ab5950 0x5598f4ac1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:14.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.971+0000 7f32e0be7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5598f4ab6270 0x5598f4ac1880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:14.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.971+0000 7f32e0be7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5598f4ac0940 0x5598f4ac7b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:14.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.971+0000 7f32e0be7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f32d81b2bb0 con 0x5598f4ac0940 2024-04-24T08:56:14.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.971+0000 7f32e0be7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f32d81b24e0 con 0x5598f4ab6270 2024-04-24T08:56:14.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.971+0000 7f32e0be7740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f32d81af720 con 0x5598f4ab5950 2024-04-24T08:56:14.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.971+0000 7f32e0be7740 10 client.0 _ll_register_callbacks cb 0x7f32c00128f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:56:14.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.972+0000 7f32ce000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5598f4ac0940 0x5598f4ac7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:14.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.972+0000 7f32ce000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5598f4ac0940 0x5598f4ac7b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45916/0 (socket says 172.21.15.83:45916) 2024-04-24T08:56:14.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.972+0000 7f32ce000640 1 -- 172.21.15.83:0/4208398086 learned_addr learned my addr 172.21.15.83:0/4208398086 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:56:14.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.972+0000 7f32d6a00640 1 --2- 172.21.15.83:0/4208398086 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5598f4ab6270 0x5598f4ac1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:14.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.972+0000 7f32ce000640 1 --2- 172.21.15.83:0/4208398086 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5598f4ab5950 0x5598f4ac1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:14.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.972+0000 7f32ce000640 1 -- 172.21.15.83:0/4208398086 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5598f4ab5950 msgr2=0x5598f4ac1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:14.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.972+0000 7f32ce000640 1 --2- 172.21.15.83:0/4208398086 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5598f4ab5950 0x5598f4ac1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.972+0000 7f32ce000640 1 -- 172.21.15.83:0/4208398086 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5598f4ab6270 msgr2=0x5598f4ac1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:14.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.972+0000 7f32ce000640 1 --2- 172.21.15.83:0/4208398086 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5598f4ab6270 0x5598f4ac1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.972+0000 7f32ce000640 1 -- 172.21.15.83:0/4208398086 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598f4ac8080 con 0x5598f4ac0940 2024-04-24T08:56:14.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.972+0000 7f32d6a00640 1 --2- 172.21.15.83:0/4208398086 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5598f4ab6270 0x5598f4ac1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:56:14.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.973+0000 7f32ce000640 1 --2- 172.21.15.83:0/4208398086 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5598f4ac0940 0x5598f4ac7b40 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x5598f4ac2300 tx=0x7f32c00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65b3afd458ebffd8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:14.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.973+0000 7f32c4c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:56:14.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.973+0000 7f32c4c00640 1 -- 172.21.15.83:0/4208398086 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c0034040 con 0x5598f4ac0940 2024-04-24T08:56:14.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.973+0000 7f32c4c00640 1 -- 172.21.15.83:0/4208398086 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f32c00051c0 con 0x5598f4ac0940 2024-04-24T08:56:14.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.973+0000 7f32e0be7740 1 -- 172.21.15.83:0/4208398086 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5598f499e4d0 con 0x5598f4ac0940 2024-04-24T08:56:14.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.973+0000 7f32c4c00640 1 -- 172.21.15.83:0/4208398086 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f32c00053c0 con 0x5598f4ac0940 2024-04-24T08:56:14.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.973+0000 7f32e0be7740 10 client.17367 fetch_fsmap learned FSMap version 32 2024-04-24T08:56:14.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.973+0000 7f32e0be7740 1 -- 172.21.15.83:0/4208398086 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5598f4ac92f0 con 0x5598f4ac0940 2024-04-24T08:56:14.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.974+0000 7f32c4c00640 1 -- 172.21.15.83:0/4208398086 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x5598f499e4d0 con 0x5598f4ac0940 2024-04-24T08:56:14.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.974+0000 7f32c4c00640 1 -- 172.21.15.83:0/4208398086 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7f32c003d470 con 0x5598f4ac0940 2024-04-24T08:56:14.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.974+0000 7f32e0be7740 10 client.17367 fetch_fsmap finished waiting for FSMap version 32 2024-04-24T08:56:14.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.974+0000 7f32e0be7740 10 client.17367 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:56:14.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.974+0000 7f32e0be7740 20 client.17367 populate_metadata read hostname 'smithi083' 2024-04-24T08:56:14.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.974+0000 7f32e0be7740 1 --2- 172.21.15.83:0/4208398086 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5598f4b374e0 0x5598f4b39900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:14.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.974+0000 7f32e0be7740 4 client.17367 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T08:56:14.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.974+0000 7f32e0be7740 1 -- 172.21.15.83:0/4208398086 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5598f49eb9f0 con 0x5598f4b374e0 2024-04-24T08:56:14.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.974+0000 7f32d6a00640 1 --2- 172.21.15.83:0/4208398086 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5598f4b374e0 0x5598f4b39900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:14.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.975+0000 7f32d6a00640 1 --2- 172.21.15.83:0/4208398086 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5598f4b374e0 0x5598f4b39900 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be8004da13800742 server_cookie=7ef91ab1acac97f1 in_seq=0 out_seq=0 2024-04-24T08:56:14.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.975+0000 7f32c4c00640 10 client.17367 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:56:14.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.976+0000 7f32c4c00640 1 -- 172.21.15.83:0/4208398086 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5598f49eb9f0 con 0x5598f4b374e0 2024-04-24T08:56:14.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.976+0000 7f32c4c00640 10 client.17367 handle_command_reply: tid=0 2024-04-24T08:56:14.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.976+0000 7f32e0be7740 1 client.17367 shutdown 2024-04-24T08:56:14.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.976+0000 7f32e0be7740 1 -- 172.21.15.83:0/4208398086 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5598f4ac0940 msgr2=0x5598f4ac7b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:14.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.976+0000 7f32e0be7740 1 --2- 172.21.15.83:0/4208398086 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5598f4ac0940 0x5598f4ac7b40 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x5598f4ac2300 tx=0x7f32c00047c0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.976+0000 7f32e0be7740 1 -- 172.21.15.83:0/4208398086 shutdown_connections 2024-04-24T08:56:14.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.976+0000 7f32e0be7740 1 --2- 172.21.15.83:0/4208398086 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5598f4b374e0 0x5598f4b39900 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.976+0000 7f32e0be7740 1 --2- 172.21.15.83:0/4208398086 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5598f4ac0940 0x5598f4ac7b40 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.976+0000 7f32e0be7740 1 --2- 172.21.15.83:0/4208398086 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5598f4ab6270 0x5598f4ac1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.976+0000 7f32e0be7740 1 --2- 172.21.15.83:0/4208398086 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5598f4ab5950 0x5598f4ac1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.976+0000 7f32c4c00640 0 client.17367 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:56:14.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.976+0000 7f32e0be7740 1 -- 172.21.15.83:0/4208398086 >> 172.21.15.83:0/4208398086 conn(0x5598f4ab1c70 msgr2=0x5598f4ab70a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:14.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.976+0000 7f32e0be7740 1 -- 172.21.15.83:0/4208398086 shutdown_connections 2024-04-24T08:56:14.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.976+0000 7f32e0be7740 1 -- 172.21.15.83:0/4208398086 wait complete. 2024-04-24T08:56:14.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.976+0000 7f32e0be7740 20 client.17367 trim_cache size 0 max 16384 2024-04-24T08:56:14.979 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T08:56:14.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.977+0000 7f32dde00640 1 -- 172.21.15.83:0/3043701762 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f32bc0a3d20 msgr2=0x7f32bc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:14.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.977+0000 7f32dde00640 1 --2- 172.21.15.83:0/3043701762 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f32bc0a3d20 0x7f32bc0a61e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f32d0009f50 tx=0x7f32d0009ea0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.977+0000 7f32dde00640 1 -- 172.21.15.83:0/3043701762 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32d810c5c0 msgr2=0x7f32d819a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:14.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.977+0000 7f32dde00640 1 --2- 172.21.15.83:0/3043701762 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32d810c5c0 0x7f32d819a840 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f32d000d750 tx=0x7f32d000dc10 comp rx=0 tx=0).stop 2024-04-24T08:56:14.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.977+0000 7f32dde00640 1 -- 172.21.15.83:0/3043701762 shutdown_connections 2024-04-24T08:56:14.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.977+0000 7f32dde00640 1 --2- 172.21.15.83:0/3043701762 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f32bc0a3d20 0x7f32bc0a61e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.977+0000 7f32dde00640 1 --2- 172.21.15.83:0/3043701762 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32d810c5c0 0x7f32d819a840 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.977+0000 7f32dde00640 1 --2- 172.21.15.83:0/3043701762 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32d8103ba0 0x7f32d81a0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.977+0000 7f32dde00640 1 --2- 172.21.15.83:0/3043701762 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f32d8103280 0x7f32d81a0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:14.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.977+0000 7f32dde00640 1 -- 172.21.15.83:0/3043701762 >> 172.21.15.83:0/3043701762 conn(0x7f32d8078000 msgr2=0x7f32d80fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:14.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.977+0000 7f32dde00640 1 -- 172.21.15.83:0/3043701762 shutdown_connections 2024-04-24T08:56:14.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:14.977+0000 7f32dde00640 1 -- 172.21.15.83:0/3043701762 wait complete. 2024-04-24T08:56:15.572 INFO:tasks.workunit.client.0.smithi083.stdout: 1 21070 0.68 MB/sec execute 49 sec latency 56.502 ms 2024-04-24T08:56:15.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:56:15 smithi138 ceph-mon[31441]: pgmap v142: 129 pgs: 129 active+clean; 37 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 255 KiB/s wr, 39 op/s 2024-04-24T08:56:15.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:56:15 smithi189 ceph-mon[31791]: pgmap v142: 129 pgs: 129 active+clean; 37 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 255 KiB/s wr, 39 op/s 2024-04-24T08:56:15.996 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T08:56:15.996 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T08:56:15.997 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T08:56:16.052 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:56:15 smithi083 ceph-mon[27907]: pgmap v142: 129 pgs: 129 active+clean; 37 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 255 KiB/s wr, 39 op/s 2024-04-24T08:56:16.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.221+0000 7f3d2ac00640 1 -- 172.21.15.83:0/3088704757 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d24104cd0 msgr2=0x7f3d24108d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:16.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.221+0000 7f3d2ac00640 1 --2- 172.21.15.83:0/3088704757 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d24104cd0 0x7f3d24108d10 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f3d10009900 tx=0x7f3d1002a3f0 comp rx=0 tx=0).stop 2024-04-24T08:56:16.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.221+0000 7f3d2ac00640 1 -- 172.21.15.83:0/3088704757 shutdown_connections 2024-04-24T08:56:16.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.221+0000 7f3d2ac00640 1 --2- 172.21.15.83:0/3088704757 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d241093f0 0x7f3d24110fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:16.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.221+0000 7f3d2ac00640 1 --2- 172.21.15.83:0/3088704757 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d24104cd0 0x7f3d24108d10 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:16.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.221+0000 7f3d2ac00640 1 --2- 172.21.15.83:0/3088704757 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3d24104320 0x7f3d24104700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:16.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.221+0000 7f3d2ac00640 1 -- 172.21.15.83:0/3088704757 >> 172.21.15.83:0/3088704757 conn(0x7f3d240ffb40 msgr2=0x7f3d24101f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:16.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.221+0000 7f3d2ac00640 1 -- 172.21.15.83:0/3088704757 shutdown_connections 2024-04-24T08:56:16.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.221+0000 7f3d2ac00640 1 -- 172.21.15.83:0/3088704757 wait complete. 2024-04-24T08:56:16.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.222+0000 7f3d2ac00640 1 Processor -- start 2024-04-24T08:56:16.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.222+0000 7f3d2ac00640 1 -- start start 2024-04-24T08:56:16.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.223+0000 7f3d2ac00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d24104320 0x7f3d241a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:16.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.223+0000 7f3d2ac00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3d241093f0 0x7f3d241a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:16.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.223+0000 7f3d2ac00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d2419a730 0x7f3d2419abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:16.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.223+0000 7f3d2ac00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3d24114330 con 0x7f3d24104320 2024-04-24T08:56:16.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.223+0000 7f3d2ac00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3d241141b0 con 0x7f3d2419a730 2024-04-24T08:56:16.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.223+0000 7f3d2ac00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3d241144b0 con 0x7f3d241093f0 2024-04-24T08:56:16.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.223+0000 7f3d29800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d24104320 0x7f3d241a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:16.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.223+0000 7f3d29800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d24104320 0x7f3d241a0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45936/0 (socket says 172.21.15.83:45936) 2024-04-24T08:56:16.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.223+0000 7f3d29800640 1 -- 172.21.15.83:0/3793869741 learned_addr learned my addr 172.21.15.83:0/3793869741 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:56:16.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.224+0000 7f3d2a200640 1 --2- 172.21.15.83:0/3793869741 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d2419a730 0x7f3d2419abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:16.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.224+0000 7f3d28e00640 1 --2- 172.21.15.83:0/3793869741 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3d241093f0 0x7f3d241a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:16.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.224+0000 7f3d29800640 1 -- 172.21.15.83:0/3793869741 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3d241093f0 msgr2=0x7f3d241a0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:16.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.224+0000 7f3d29800640 1 --2- 172.21.15.83:0/3793869741 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3d241093f0 0x7f3d241a0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:16.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.224+0000 7f3d29800640 1 -- 172.21.15.83:0/3793869741 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d2419a730 msgr2=0x7f3d2419abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:16.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.224+0000 7f3d29800640 1 --2- 172.21.15.83:0/3793869741 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d2419a730 0x7f3d2419abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:16.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.224+0000 7f3d29800640 1 -- 172.21.15.83:0/3793869741 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d2419b330 con 0x7f3d24104320 2024-04-24T08:56:16.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.224+0000 7f3d29800640 1 --2- 172.21.15.83:0/3793869741 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d24104320 0x7f3d241a0670 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f3d1800d770 tx=0x7f3d1800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:16.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.224+0000 7f3d22000640 1 -- 172.21.15.83:0/3793869741 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d18017070 con 0x7f3d24104320 2024-04-24T08:56:16.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.224+0000 7f3d22000640 1 -- 172.21.15.83:0/3793869741 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3d180054d0 con 0x7f3d24104320 2024-04-24T08:56:16.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.224+0000 7f3d2ac00640 1 -- 172.21.15.83:0/3793869741 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d241a7410 con 0x7f3d24104320 2024-04-24T08:56:16.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.225+0000 7f3d22000640 1 -- 172.21.15.83:0/3793869741 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d18004cf0 con 0x7f3d24104320 2024-04-24T08:56:16.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.225+0000 7f3d2ac00640 1 -- 172.21.15.83:0/3793869741 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d241a77c0 con 0x7f3d24104320 2024-04-24T08:56:16.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.228+0000 7f3d2f167740 1 Processor -- start 2024-04-24T08:56:16.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.228+0000 7f3d2f167740 1 -- start start 2024-04-24T08:56:16.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.228+0000 7f3d22000640 1 -- 172.21.15.83:0/3793869741 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3d18004e90 con 0x7f3d24104320 2024-04-24T08:56:16.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.228+0000 7f3d2f167740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c88a13270 0x561c88a13650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:16.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.228+0000 7f3d2f167740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c88a13b90 0x561c88a1dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:16.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.228+0000 7f3d2f167740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c88a1e260 0x561c88a20650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:16.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.228+0000 7f3d2f167740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561c88978e70 con 0x561c88a13b90 2024-04-24T08:56:16.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.229+0000 7f3d2f167740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x561c8891f380 con 0x561c88a1e260 2024-04-24T08:56:16.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.229+0000 7f3d2f167740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x561c88974e30 con 0x561c88a13270 2024-04-24T08:56:16.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.229+0000 7f3d29800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c88a13b90 0x561c88a1dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:16.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.229+0000 7f3d29800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c88a13b90 0x561c88a1dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45946/0 (socket says 172.21.15.83:45946) 2024-04-24T08:56:16.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.229+0000 7f3d29800640 1 -- 172.21.15.83:0/3057852099 learned_addr learned my addr 172.21.15.83:0/3057852099 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:56:16.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.229+0000 7f3d28e00640 1 --2- 172.21.15.83:0/3057852099 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c88a1e260 0x561c88a20650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:16.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.229+0000 7f3d2a200640 1 --2- 172.21.15.83:0/3057852099 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c88a13270 0x561c88a13650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:16.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.229+0000 7f3d29800640 1 -- 172.21.15.83:0/3057852099 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c88a13270 msgr2=0x561c88a13650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:16.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.229+0000 7f3d29800640 1 --2- 172.21.15.83:0/3057852099 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c88a13270 0x561c88a13650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:16.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.229+0000 7f3d29800640 1 -- 172.21.15.83:0/3057852099 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c88a1e260 msgr2=0x561c88a20650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:16.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.229+0000 7f3d29800640 1 --2- 172.21.15.83:0/3057852099 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c88a1e260 0x561c88a20650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:16.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.229+0000 7f3d29800640 1 -- 172.21.15.83:0/3057852099 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c887537e0 con 0x561c88a13b90 2024-04-24T08:56:16.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.230+0000 7f3d29800640 1 --2- 172.21.15.83:0/3057852099 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c88a13b90 0x561c88a1dd20 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f3d180cd9f0 tx=0x7f3d180d8690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=129e2dc95327ad7f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:16.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.230+0000 7f3d16a00640 1 -- 172.21.15.83:0/3057852099 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d180df070 con 0x561c88a13b90 2024-04-24T08:56:16.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.230+0000 7f3d16a00640 1 -- 172.21.15.83:0/3057852099 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3d180d0db0 con 0x561c88a13b90 2024-04-24T08:56:16.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.230+0000 7f3d16a00640 1 -- 172.21.15.83:0/3057852099 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d180db680 con 0x561c88a13b90 2024-04-24T08:56:16.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.231+0000 7f3d22000640 1 --2- 172.21.15.83:0/3793869741 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f3d0c0a3d20 0x7f3d0c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:16.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.231+0000 7f3d22000640 1 -- 172.21.15.83:0/3793869741 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f3d180d0700 con 0x7f3d24104320 2024-04-24T08:56:16.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.231+0000 7f3d2a200640 1 --2- 172.21.15.83:0/3793869741 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f3d0c0a3d20 0x7f3d0c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:16.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.231+0000 7f3d2f167740 1 -- 172.21.15.83:0/3057852099 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c88a13b90 msgr2=0x561c88a1dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:16.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.231+0000 7f3d2f167740 1 --2- 172.21.15.83:0/3057852099 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c88a13b90 0x561c88a1dd20 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f3d180cd9f0 tx=0x7f3d180d8690 comp rx=0 tx=0).stop 2024-04-24T08:56:16.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.232+0000 7f3d2f167740 1 -- 172.21.15.83:0/3057852099 shutdown_connections 2024-04-24T08:56:16.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.232+0000 7f3d2f167740 1 --2- 172.21.15.83:0/3057852099 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c88a1e260 0x561c88a20650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:16.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.232+0000 7f3d2f167740 1 --2- 172.21.15.83:0/3057852099 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c88a13b90 0x561c88a1dd20 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:16.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.232+0000 7f3d2f167740 1 --2- 172.21.15.83:0/3057852099 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c88a13270 0x561c88a13650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:16.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.232+0000 7f3d2f167740 1 -- 172.21.15.83:0/3057852099 >> 172.21.15.83:0/3057852099 conn(0x561c88a0f590 msgr2=0x561c88a0f970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:16.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.232+0000 7f3d2a200640 1 --2- 172.21.15.83:0/3793869741 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f3d0c0a3d20 0x7f3d0c0a61e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c0092f0 tx=0x7f3d1c009280 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:16.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.232+0000 7f3d2f167740 1 -- 172.21.15.83:0/3057852099 shutdown_connections 2024-04-24T08:56:16.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.232+0000 7f3d2f167740 1 -- 172.21.15.83:0/3057852099 wait complete. 2024-04-24T08:56:16.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.232+0000 7f3d2f167740 1 -- start start 2024-04-24T08:56:16.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.232+0000 7f3d2f167740 1 Processor -- start 2024-04-24T08:56:16.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.233+0000 7f3d2f167740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c88a13270 0x561c88a1b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:16.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.233+0000 7f3d2f167740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c88a13b90 0x561c88a1b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:16.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.233+0000 7f3d2f167740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c88a1e260 0x561c88a1bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:16.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.233+0000 7f3d2f167740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561c88a21050 con 0x561c88a13270 2024-04-24T08:56:16.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.233+0000 7f3d2f167740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x561c88932660 con 0x561c88a13b90 2024-04-24T08:56:16.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.233+0000 7f3d2f167740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x561c88a211d0 con 0x561c88a1e260 2024-04-24T08:56:16.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.233+0000 7f3d2f167740 10 client.0 _ll_register_callbacks cb 0x561c88920aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:56:16.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.233+0000 7f3d29800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c88a13270 0x561c88a1b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:16.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.233+0000 7f3d29800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c88a13270 0x561c88a1b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45956/0 (socket says 172.21.15.83:45956) 2024-04-24T08:56:16.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.233+0000 7f3d29800640 1 -- 172.21.15.83:0/3437973719 learned_addr learned my addr 172.21.15.83:0/3437973719 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:56:16.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.234+0000 7f3d28e00640 1 --2- 172.21.15.83:0/3437973719 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c88a13b90 0x561c88a1b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:16.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.234+0000 7f3d2a200640 1 --2- 172.21.15.83:0/3437973719 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c88a1e260 0x561c88a1bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:16.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.234+0000 7f3d29800640 1 -- 172.21.15.83:0/3437973719 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c88a1e260 msgr2=0x561c88a1bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:16.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.234+0000 7f3d29800640 1 --2- 172.21.15.83:0/3437973719 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c88a1e260 0x561c88a1bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:16.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.234+0000 7f3d29800640 1 -- 172.21.15.83:0/3437973719 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c88a13b90 msgr2=0x561c88a1b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:16.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.234+0000 7f3d29800640 1 --2- 172.21.15.83:0/3437973719 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c88a13b90 0x561c88a1b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:16.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.234+0000 7f3d29800640 1 -- 172.21.15.83:0/3437973719 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c887b3f70 con 0x561c88a13270 2024-04-24T08:56:16.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.234+0000 7f3d28e00640 1 --2- 172.21.15.83:0/3437973719 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c88a13b90 0x561c88a1b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:56:16.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.234+0000 7f3d2a200640 1 --2- 172.21.15.83:0/3437973719 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c88a1e260 0x561c88a1bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:56:16.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.234+0000 7f3d29800640 1 --2- 172.21.15.83:0/3437973719 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c88a13270 0x561c88a1b1f0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f3d18098730 tx=0x7f3d18005bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66e31f9e76d6f119 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:16.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.234+0000 7f3d0b400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:56:16.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.234+0000 7f3d0b400640 1 -- 172.21.15.83:0/3437973719 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d18020030 con 0x561c88a13270 2024-04-24T08:56:16.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.234+0000 7f3d0b400640 1 -- 172.21.15.83:0/3437973719 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f3d180d0bb0 con 0x561c88a13270 2024-04-24T08:56:16.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.234+0000 7f3d2f167740 1 -- 172.21.15.83:0/3437973719 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d180d0db0 con 0x561c88a13270 2024-04-24T08:56:16.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.235+0000 7f3d0b400640 1 -- 172.21.15.83:0/3437973719 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f3d180df070 con 0x561c88a13270 2024-04-24T08:56:16.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.235+0000 7f3d2f167740 10 client.17379 fetch_fsmap learned FSMap version 32 2024-04-24T08:56:16.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.235+0000 7f3d2f167740 1 -- 172.21.15.83:0/3437973719 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x561c8864b170 con 0x561c88a13270 2024-04-24T08:56:16.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.235+0000 7f3d0b400640 1 -- 172.21.15.83:0/3437973719 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3d180d0db0 con 0x561c88a13270 2024-04-24T08:56:16.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.235+0000 7f3d0b400640 1 -- 172.21.15.83:0/3437973719 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7f3d180cdd00 con 0x561c88a13270 2024-04-24T08:56:16.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.235+0000 7f3d2f167740 10 client.17379 fetch_fsmap finished waiting for FSMap version 32 2024-04-24T08:56:16.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.235+0000 7f3d2f167740 10 client.17379 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:56:16.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.235+0000 7f3d2f167740 20 client.17379 populate_metadata read hostname 'smithi083' 2024-04-24T08:56:16.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.236+0000 7f3d2f167740 1 --2- 172.21.15.83:0/3437973719 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x561c88a2e470 0x561c88a30860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:16.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.236+0000 7f3d2f167740 4 client.17379 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:56:16.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.236+0000 7f3d2f167740 1 -- 172.21.15.83:0/3437973719 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561c88986890 con 0x561c88a2e470 2024-04-24T08:56:16.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.236+0000 7f3d28e00640 1 --2- 172.21.15.83:0/3437973719 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x561c88a2e470 0x561c88a30860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:16.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.237+0000 7f3d0b400640 10 client.17379 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:56:16.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.237+0000 7f3d28e00640 1 --2- 172.21.15.83:0/3437973719 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x561c88a2e470 0x561c88a30860 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75d3b4f464cd2d48 server_cookie=a611709d6daddd90 in_seq=0 out_seq=0 2024-04-24T08:56:16.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.239+0000 7f3d0b400640 1 -- 172.21.15.83:0/3437973719 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561c88986890 con 0x561c88a2e470 2024-04-24T08:56:16.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.239+0000 7f3d0b400640 10 client.17379 handle_command_reply: tid=0 2024-04-24T08:56:16.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.239+0000 7f3d2f167740 1 client.17379 shutdown 2024-04-24T08:56:16.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.240+0000 7f3d2f167740 1 -- 172.21.15.83:0/3437973719 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c88a13270 msgr2=0x561c88a1b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:16.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.240+0000 7f3d2f167740 1 --2- 172.21.15.83:0/3437973719 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c88a13270 0x561c88a1b1f0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f3d18098730 tx=0x7f3d18005bb0 comp rx=0 tx=0).stop 2024-04-24T08:56:16.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.240+0000 7f3d2f167740 1 -- 172.21.15.83:0/3437973719 shutdown_connections 2024-04-24T08:56:16.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.240+0000 7f3d2f167740 1 --2- 172.21.15.83:0/3437973719 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x561c88a2e470 0x561c88a30860 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:16.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.240+0000 7f3d2f167740 1 --2- 172.21.15.83:0/3437973719 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c88a1e260 0x561c88a1bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:16.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.240+0000 7f3d2f167740 1 --2- 172.21.15.83:0/3437973719 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c88a13b90 0x561c88a1b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:16.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.240+0000 7f3d2f167740 1 --2- 172.21.15.83:0/3437973719 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c88a13270 0x561c88a1b1f0 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:16.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.240+0000 7f3d2f167740 1 -- 172.21.15.83:0/3437973719 >> 172.21.15.83:0/3437973719 conn(0x561c88a0f590 msgr2=0x561c88a15390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:16.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.240+0000 7f3d0b400640 0 client.17379 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:56:16.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.240+0000 7f3d2f167740 1 -- 172.21.15.83:0/3437973719 shutdown_connections 2024-04-24T08:56:16.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.240+0000 7f3d2f167740 1 -- 172.21.15.83:0/3437973719 wait complete. 2024-04-24T08:56:16.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.240+0000 7f3d2f167740 20 client.17379 trim_cache size 0 max 16384 2024-04-24T08:56:16.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.262+0000 7f3d2f167740 1 Processor -- start 2024-04-24T08:56:16.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.262+0000 7f3d2f167740 1 -- start start 2024-04-24T08:56:16.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.262+0000 7f3d2f167740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c88a13270 0x561c88a29f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:16.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.262+0000 7f3d2f167740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c88a13b90 0x561c88a159d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:16.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.262+0000 7f3d2f167740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c88a1e260 0x561c88a1b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:16.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.262+0000 7f3d2f167740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3d241b15d0 con 0x561c88a13b90 2024-04-24T08:56:16.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.262+0000 7f3d2f167740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3d241b3140 con 0x561c88a13270 2024-04-24T08:56:16.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.262+0000 7f3d2f167740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3d241b1ca0 con 0x561c88a1e260 2024-04-24T08:56:16.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.263+0000 7f3d28e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c88a13b90 0x561c88a159d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:16.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.263+0000 7f3d28e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c88a13b90 0x561c88a159d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45958/0 (socket says 172.21.15.83:45958) 2024-04-24T08:56:16.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.263+0000 7f3d28e00640 1 -- 172.21.15.83:0/807959931 learned_addr learned my addr 172.21.15.83:0/807959931 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:56:16.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.263+0000 7f3d29800640 1 --2- 172.21.15.83:0/807959931 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c88a13270 0x561c88a29f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:16.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.263+0000 7f3d28e00640 1 -- 172.21.15.83:0/807959931 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c88a1e260 msgr2=0x561c88a1b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:16.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.263+0000 7f3d28e00640 1 --2- 172.21.15.83:0/807959931 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c88a1e260 0x561c88a1b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:16.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.263+0000 7f3d28e00640 1 -- 172.21.15.83:0/807959931 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c88a13270 msgr2=0x561c88a29f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:16.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.263+0000 7f3d28e00640 1 --2- 172.21.15.83:0/807959931 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c88a13270 0x561c88a29f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:16.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.263+0000 7f3d28e00640 1 -- 172.21.15.83:0/807959931 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c88a30bd0 con 0x561c88a13b90 2024-04-24T08:56:16.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.263+0000 7f3d28e00640 1 --2- 172.21.15.83:0/807959931 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c88a13b90 0x561c88a159d0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f3d10005ab0 tx=0x7f3d10005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f11655d81e6621de server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:16.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.264+0000 7f3d16000640 1 -- 172.21.15.83:0/807959931 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d10035040 con 0x561c88a13b90 2024-04-24T08:56:16.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.264+0000 7f3d16000640 1 -- 172.21.15.83:0/807959931 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3d100041b0 con 0x561c88a13b90 2024-04-24T08:56:16.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.264+0000 7f3d16000640 1 -- 172.21.15.83:0/807959931 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d100422d0 con 0x561c88a13b90 2024-04-24T08:56:16.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.264+0000 7f3d2f167740 1 -- 172.21.15.83:0/807959931 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c88a13b90 msgr2=0x561c88a159d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:16.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.264+0000 7f3d2f167740 1 --2- 172.21.15.83:0/807959931 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c88a13b90 0x561c88a159d0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f3d10005ab0 tx=0x7f3d10005a20 comp rx=0 tx=0).stop 2024-04-24T08:56:16.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.265+0000 7f3d2f167740 1 -- 172.21.15.83:0/807959931 shutdown_connections 2024-04-24T08:56:16.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.265+0000 7f3d2f167740 1 --2- 172.21.15.83:0/807959931 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c88a1e260 0x561c88a1b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:16.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.265+0000 7f3d2f167740 1 --2- 172.21.15.83:0/807959931 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c88a13b90 0x561c88a159d0 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:16.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.265+0000 7f3d2f167740 1 --2- 172.21.15.83:0/807959931 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c88a13270 0x561c88a29f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:16.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.265+0000 7f3d2f167740 1 -- 172.21.15.83:0/807959931 >> 172.21.15.83:0/807959931 conn(0x561c88a0f590 msgr2=0x561c88a1b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:16.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.265+0000 7f3d2f167740 1 -- 172.21.15.83:0/807959931 shutdown_connections 2024-04-24T08:56:16.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.265+0000 7f3d2f167740 1 -- 172.21.15.83:0/807959931 wait complete. 2024-04-24T08:56:16.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.265+0000 7f3d2f167740 1 -- start start 2024-04-24T08:56:16.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.265+0000 7f3d2f167740 1 Processor -- start 2024-04-24T08:56:16.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.266+0000 7f3d2f167740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c88a13270 0x561c88a18050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:16.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.266+0000 7f3d2f167740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c88a13b90 0x561c88a18590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:16.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.266+0000 7f3d2f167740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c88a1e260 0x561c88a18ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:16.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.266+0000 7f3d2f167740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3d241b2a40 con 0x561c88a13b90 2024-04-24T08:56:16.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.266+0000 7f3d2f167740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3d241b2370 con 0x561c88a13270 2024-04-24T08:56:16.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.266+0000 7f3d2f167740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3d241af5b0 con 0x561c88a1e260 2024-04-24T08:56:16.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.266+0000 7f3d2f167740 10 client.0 _ll_register_callbacks cb 0x7f3d180d6a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:56:16.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.266+0000 7f3d28e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c88a13b90 0x561c88a18590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:16.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.266+0000 7f3d28e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c88a13b90 0x561c88a18590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45962/0 (socket says 172.21.15.83:45962) 2024-04-24T08:56:16.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.266+0000 7f3d28e00640 1 -- 172.21.15.83:0/1015160997 learned_addr learned my addr 172.21.15.83:0/1015160997 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:56:16.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.266+0000 7f3d29800640 1 --2- 172.21.15.83:0/1015160997 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c88a13270 0x561c88a18050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:16.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.267+0000 7f3d2a200640 1 --2- 172.21.15.83:0/1015160997 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c88a1e260 0x561c88a18ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:16.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.267+0000 7f3d28e00640 1 -- 172.21.15.83:0/1015160997 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c88a1e260 msgr2=0x561c88a18ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:16.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.267+0000 7f3d28e00640 1 --2- 172.21.15.83:0/1015160997 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c88a1e260 0x561c88a18ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:16.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.267+0000 7f3d28e00640 1 -- 172.21.15.83:0/1015160997 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c88a13270 msgr2=0x561c88a18050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:16.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.267+0000 7f3d28e00640 1 --2- 172.21.15.83:0/1015160997 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c88a13270 0x561c88a18050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:16.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.267+0000 7f3d28e00640 1 -- 172.21.15.83:0/1015160997 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c88a93820 con 0x561c88a13b90 2024-04-24T08:56:16.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.267+0000 7f3d2a200640 1 --2- 172.21.15.83:0/1015160997 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c88a1e260 0x561c88a18ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:56:16.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.267+0000 7f3d28e00640 1 --2- 172.21.15.83:0/1015160997 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c88a13b90 0x561c88a18590 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f3d10000c00 tx=0x7f3d100055c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d08c7fed2e16a7ea server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:16.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.267+0000 7f3d0b400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:56:16.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.267+0000 7f3d0b400640 1 -- 172.21.15.83:0/1015160997 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d1004f070 con 0x561c88a13b90 2024-04-24T08:56:16.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.268+0000 7f3d0b400640 1 -- 172.21.15.83:0/1015160997 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f3d100035f0 con 0x561c88a13b90 2024-04-24T08:56:16.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.268+0000 7f3d2f167740 1 -- 172.21.15.83:0/1015160997 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d100041b0 con 0x561c88a13b90 2024-04-24T08:56:16.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.268+0000 7f3d0b400640 1 -- 172.21.15.83:0/1015160997 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f3d10035040 con 0x561c88a13b90 2024-04-24T08:56:16.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.268+0000 7f3d2f167740 10 client.17385 fetch_fsmap learned FSMap version 32 2024-04-24T08:56:16.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.268+0000 7f3d2f167740 1 -- 172.21.15.83:0/1015160997 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x561c88a94b20 con 0x561c88a13b90 2024-04-24T08:56:16.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.268+0000 7f3d0b400640 1 -- 172.21.15.83:0/1015160997 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3d100041b0 con 0x561c88a13b90 2024-04-24T08:56:16.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.268+0000 7f3d0b400640 1 -- 172.21.15.83:0/1015160997 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7f3d10054970 con 0x561c88a13b90 2024-04-24T08:56:16.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.269+0000 7f3d2f167740 10 client.17385 fetch_fsmap finished waiting for FSMap version 32 2024-04-24T08:56:16.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.269+0000 7f3d2f167740 10 client.17385 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:56:16.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.269+0000 7f3d2f167740 20 client.17385 populate_metadata read hostname 'smithi083' 2024-04-24T08:56:16.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.269+0000 7f3d2f167740 1 --2- 172.21.15.83:0/1015160997 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x561c88a952a0 0x561c88a976c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:16.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.269+0000 7f3d2f167740 4 client.17385 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T08:56:16.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.269+0000 7f3d2f167740 1 -- 172.21.15.83:0/1015160997 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x561c88980700 con 0x561c88a952a0 2024-04-24T08:56:16.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.269+0000 7f3d29800640 1 --2- 172.21.15.83:0/1015160997 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x561c88a952a0 0x561c88a976c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:16.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.270+0000 7f3d29800640 1 --2- 172.21.15.83:0/1015160997 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x561c88a952a0 0x561c88a976c0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=999af6d4331c8008 server_cookie=8d62dc8af5713dc0 in_seq=0 out_seq=0 2024-04-24T08:56:16.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.270+0000 7f3d0b400640 10 client.17385 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:56:16.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.274+0000 7f3d0b400640 1 -- 172.21.15.83:0/1015160997 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x561c88980700 con 0x561c88a952a0 2024-04-24T08:56:16.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.274+0000 7f3d0b400640 10 client.17385 handle_command_reply: tid=0 2024-04-24T08:56:16.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.274+0000 7f3d2f167740 1 client.17385 shutdown 2024-04-24T08:56:16.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.274+0000 7f3d2f167740 1 -- 172.21.15.83:0/1015160997 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c88a13b90 msgr2=0x561c88a18590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:16.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.274+0000 7f3d2f167740 1 --2- 172.21.15.83:0/1015160997 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c88a13b90 0x561c88a18590 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f3d10000c00 tx=0x7f3d100055c0 comp rx=0 tx=0).stop 2024-04-24T08:56:16.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.274+0000 7f3d2f167740 1 -- 172.21.15.83:0/1015160997 shutdown_connections 2024-04-24T08:56:16.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.274+0000 7f3d2f167740 1 --2- 172.21.15.83:0/1015160997 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x561c88a952a0 0x561c88a976c0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:16.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.274+0000 7f3d2f167740 1 --2- 172.21.15.83:0/1015160997 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c88a1e260 0x561c88a18ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:16.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.274+0000 7f3d2f167740 1 --2- 172.21.15.83:0/1015160997 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c88a13b90 0x561c88a18590 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:16.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.274+0000 7f3d2f167740 1 --2- 172.21.15.83:0/1015160997 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c88a13270 0x561c88a18050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:16.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.274+0000 7f3d0b400640 0 client.17385 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:56:16.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.274+0000 7f3d2f167740 1 -- 172.21.15.83:0/1015160997 >> 172.21.15.83:0/1015160997 conn(0x561c88a0f590 msgr2=0x561c88a1b9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:16.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.274+0000 7f3d2f167740 1 -- 172.21.15.83:0/1015160997 shutdown_connections 2024-04-24T08:56:16.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.274+0000 7f3d2f167740 1 -- 172.21.15.83:0/1015160997 wait complete. 2024-04-24T08:56:16.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.274+0000 7f3d2f167740 20 client.17385 trim_cache size 0 max 16384 2024-04-24T08:56:16.276 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T08:56:16.276 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-04-24T08:56:16.276 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "fefe782a-33f3-4c85-b825-61330ae6c8ed", 2024-04-24T08:56:16.276 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-04-24T08:56:16.276 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T08:56:16.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.275+0000 7f3d2ac00640 1 -- 172.21.15.83:0/3793869741 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f3d0c0a3d20 msgr2=0x7f3d0c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:16.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.275+0000 7f3d2ac00640 1 --2- 172.21.15.83:0/3793869741 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f3d0c0a3d20 0x7f3d0c0a61e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c0092f0 tx=0x7f3d1c009280 comp rx=0 tx=0).stop 2024-04-24T08:56:16.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.275+0000 7f3d2ac00640 1 -- 172.21.15.83:0/3793869741 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d24104320 msgr2=0x7f3d241a0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:16.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.275+0000 7f3d2ac00640 1 --2- 172.21.15.83:0/3793869741 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d24104320 0x7f3d241a0670 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f3d1800d770 tx=0x7f3d1800dc30 comp rx=0 tx=0).stop 2024-04-24T08:56:16.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.275+0000 7f3d2ac00640 1 -- 172.21.15.83:0/3793869741 shutdown_connections 2024-04-24T08:56:16.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.275+0000 7f3d2ac00640 1 --2- 172.21.15.83:0/3793869741 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f3d0c0a3d20 0x7f3d0c0a61e0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:16.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.275+0000 7f3d2ac00640 1 --2- 172.21.15.83:0/3793869741 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d2419a730 0x7f3d2419abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:16.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.275+0000 7f3d2ac00640 1 --2- 172.21.15.83:0/3793869741 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3d241093f0 0x7f3d241a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:16.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.275+0000 7f3d2ac00640 1 --2- 172.21.15.83:0/3793869741 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3d24104320 0x7f3d241a0670 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:16.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.275+0000 7f3d2ac00640 1 -- 172.21.15.83:0/3793869741 >> 172.21.15.83:0/3793869741 conn(0x7f3d240ffb40 msgr2=0x7f3d24100f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:16.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.275+0000 7f3d2ac00640 1 -- 172.21.15.83:0/3793869741 shutdown_connections 2024-04-24T08:56:16.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.275+0000 7f3d2ac00640 1 -- 172.21.15.83:0/3793869741 wait complete. 2024-04-24T08:56:16.293 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T08:56:16.294 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T08:56:16.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.514+0000 7f1169600640 1 -- 172.21.15.83:0/2181873111 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1164104cd0 msgr2=0x7f1164108d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:16.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.514+0000 7f1169600640 1 --2- 172.21.15.83:0/2181873111 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1164104cd0 0x7f1164108d10 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f114c009900 tx=0x7f114c02a410 comp rx=0 tx=0).stop 2024-04-24T08:56:16.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.514+0000 7f1169600640 1 -- 172.21.15.83:0/2181873111 shutdown_connections 2024-04-24T08:56:16.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.514+0000 7f1169600640 1 --2- 172.21.15.83:0/2181873111 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11641093f0 0x7f1164110fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:16.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.514+0000 7f1169600640 1 --2- 172.21.15.83:0/2181873111 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1164104cd0 0x7f1164108d10 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:16.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.514+0000 7f1169600640 1 --2- 172.21.15.83:0/2181873111 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1164104320 0x7f1164104700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:16.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.514+0000 7f1169600640 1 -- 172.21.15.83:0/2181873111 >> 172.21.15.83:0/2181873111 conn(0x7f11640ffb40 msgr2=0x7f1164101f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:16.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.514+0000 7f1169600640 1 -- 172.21.15.83:0/2181873111 shutdown_connections 2024-04-24T08:56:16.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.514+0000 7f1169600640 1 -- 172.21.15.83:0/2181873111 wait complete. 2024-04-24T08:56:16.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.515+0000 7f1169600640 1 Processor -- start 2024-04-24T08:56:16.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.516+0000 7f1169600640 1 -- start start 2024-04-24T08:56:16.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.516+0000 7f1169600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1164104320 0x7f11641a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:16.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.516+0000 7f1169600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11641093f0 0x7f11641a0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:16.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.516+0000 7f1169600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f116419a700 0x7f116419ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:16.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.516+0000 7f1169600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1164114360 con 0x7f116419a700 2024-04-24T08:56:16.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.516+0000 7f1169600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f11641141e0 con 0x7f11641093f0 2024-04-24T08:56:16.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.516+0000 7f1169600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f11641144e0 con 0x7f1164104320 2024-04-24T08:56:16.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.516+0000 7f1168c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f116419a700 0x7f116419ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:16.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.516+0000 7f1168c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f116419a700 0x7f116419ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45990/0 (socket says 172.21.15.83:45990) 2024-04-24T08:56:16.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.517+0000 7f1168c00640 1 -- 172.21.15.83:0/3777463538 learned_addr learned my addr 172.21.15.83:0/3777463538 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:56:16.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.517+0000 7f115b400640 1 --2- 172.21.15.83:0/3777463538 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11641093f0 0x7f11641a0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:16.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.517+0000 7f1163e00640 1 --2- 172.21.15.83:0/3777463538 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1164104320 0x7f11641a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:16.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.517+0000 7f1168c00640 1 -- 172.21.15.83:0/3777463538 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1164104320 msgr2=0x7f11641a0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:16.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.517+0000 7f1168c00640 1 --2- 172.21.15.83:0/3777463538 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1164104320 0x7f11641a0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:16.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.517+0000 7f1168c00640 1 -- 172.21.15.83:0/3777463538 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11641093f0 msgr2=0x7f11641a0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:16.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.517+0000 7f1168c00640 1 --2- 172.21.15.83:0/3777463538 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11641093f0 0x7f11641a0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:16.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.517+0000 7f1168c00640 1 -- 172.21.15.83:0/3777463538 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f116419b300 con 0x7f116419a700 2024-04-24T08:56:16.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.517+0000 7f1168c00640 1 --2- 172.21.15.83:0/3777463538 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f116419a700 0x7f116419ab80 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f115c00eaa0 tx=0x7f115c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:16.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.518+0000 7f1161600640 1 -- 172.21.15.83:0/3777463538 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f115c00cdb0 con 0x7f116419a700 2024-04-24T08:56:16.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.518+0000 7f1161600640 1 -- 172.21.15.83:0/3777463538 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f115c005020 con 0x7f116419a700 2024-04-24T08:56:16.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.518+0000 7f1161600640 1 -- 172.21.15.83:0/3777463538 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f115c0116d0 con 0x7f116419a700 2024-04-24T08:56:16.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.518+0000 7f1169600640 1 -- 172.21.15.83:0/3777463538 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11641a73e0 con 0x7f116419a700 2024-04-24T08:56:16.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.518+0000 7f1169600640 1 -- 172.21.15.83:0/3777463538 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11641a7790 con 0x7f116419a700 2024-04-24T08:56:16.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.521+0000 7f1161600640 1 -- 172.21.15.83:0/3777463538 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f115c0051d0 con 0x7f116419a700 2024-04-24T08:56:16.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.521+0000 7f116dbde740 1 Processor -- start 2024-04-24T08:56:16.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.521+0000 7f116dbde740 1 -- start start 2024-04-24T08:56:16.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.522+0000 7f116dbde740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c1b738a30 0x561c1b738e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:16.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.522+0000 7f116dbde740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c1b739350 0x561c1b7434e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:16.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.522+0000 7f116dbde740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c1b743a20 0x561c1b745e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:16.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.522+0000 7f116dbde740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561c1b69e1f0 con 0x561c1b739350 2024-04-24T08:56:16.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.522+0000 7f116dbde740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x561c1b644350 con 0x561c1b738a30 2024-04-24T08:56:16.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.522+0000 7f116dbde740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x561c1b69a1b0 con 0x561c1b743a20 2024-04-24T08:56:16.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.522+0000 7f1163e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c1b739350 0x561c1b7434e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:16.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.522+0000 7f1163e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c1b739350 0x561c1b7434e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45998/0 (socket says 172.21.15.83:45998) 2024-04-24T08:56:16.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.522+0000 7f1163e00640 1 -- 172.21.15.83:0/2796474732 learned_addr learned my addr 172.21.15.83:0/2796474732 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:56:16.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.522+0000 7f115b400640 1 --2- 172.21.15.83:0/2796474732 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c1b738a30 0x561c1b738e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:16.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.523+0000 7f1163e00640 1 -- 172.21.15.83:0/2796474732 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c1b743a20 msgr2=0x561c1b745e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:16.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.523+0000 7f1163e00640 1 --2- 172.21.15.83:0/2796474732 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c1b743a20 0x561c1b745e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:16.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.523+0000 7f1163e00640 1 -- 172.21.15.83:0/2796474732 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c1b738a30 msgr2=0x561c1b738e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:16.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.523+0000 7f1163e00640 1 --2- 172.21.15.83:0/2796474732 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c1b738a30 0x561c1b738e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:16.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.523+0000 7f1163e00640 1 -- 172.21.15.83:0/2796474732 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c1b698f50 con 0x561c1b739350 2024-04-24T08:56:16.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.523+0000 7f1161600640 1 --2- 172.21.15.83:0/3777463538 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f11440a3d20 0x7f11440a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:16.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.523+0000 7f1163e00640 1 --2- 172.21.15.83:0/2796474732 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c1b739350 0x561c1b7434e0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f115400bcb0 tx=0x7f115400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=120584660858cb9f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:16.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.523+0000 7f1161600640 1 -- 172.21.15.83:0/3777463538 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f115c0d0e80 con 0x7f116419a700 2024-04-24T08:56:16.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.523+0000 7f115b400640 1 --2- 172.21.15.83:0/3777463538 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f11440a3d20 0x7f11440a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:16.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.523+0000 7f1159200640 1 -- 172.21.15.83:0/2796474732 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1154015070 con 0x561c1b739350 2024-04-24T08:56:16.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.524+0000 7f1159200640 1 -- 172.21.15.83:0/2796474732 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f115401a910 con 0x561c1b739350 2024-04-24T08:56:16.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.524+0000 7f1159200640 1 -- 172.21.15.83:0/2796474732 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1154011990 con 0x561c1b739350 2024-04-24T08:56:16.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.524+0000 7f115b400640 1 --2- 172.21.15.83:0/3777463538 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f11440a3d20 0x7f11440a61e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f114c006010 tx=0x7f114c000bc0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:16.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.524+0000 7f116dbde740 1 -- 172.21.15.83:0/2796474732 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c1b739350 msgr2=0x561c1b7434e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:16.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.524+0000 7f116dbde740 1 --2- 172.21.15.83:0/2796474732 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c1b739350 0x561c1b7434e0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f115400bcb0 tx=0x7f115400f730 comp rx=0 tx=0).stop 2024-04-24T08:56:16.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.525+0000 7f116dbde740 1 -- 172.21.15.83:0/2796474732 shutdown_connections 2024-04-24T08:56:16.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.525+0000 7f116dbde740 1 --2- 172.21.15.83:0/2796474732 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c1b743a20 0x561c1b745e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:16.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.525+0000 7f116dbde740 1 --2- 172.21.15.83:0/2796474732 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c1b739350 0x561c1b7434e0 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:16.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.525+0000 7f116dbde740 1 --2- 172.21.15.83:0/2796474732 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c1b738a30 0x561c1b738e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:16.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.525+0000 7f116dbde740 1 -- 172.21.15.83:0/2796474732 >> 172.21.15.83:0/2796474732 conn(0x561c1b734d50 msgr2=0x561c1b735130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:16.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.525+0000 7f116dbde740 1 -- 172.21.15.83:0/2796474732 shutdown_connections 2024-04-24T08:56:16.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.525+0000 7f116dbde740 1 -- 172.21.15.83:0/2796474732 wait complete. 2024-04-24T08:56:16.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.525+0000 7f116dbde740 1 -- start start 2024-04-24T08:56:16.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.526+0000 7f116dbde740 1 Processor -- start 2024-04-24T08:56:16.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.526+0000 7f116dbde740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c1b738a30 0x561c1b740980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:16.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.526+0000 7f116dbde740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c1b739350 0x561c1b740ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:16.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.526+0000 7f116dbde740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c1b743a20 0x561c1b741400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:16.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.526+0000 7f116dbde740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561c1b746810 con 0x561c1b743a20 2024-04-24T08:56:16.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.526+0000 7f116dbde740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x561c1b657690 con 0x561c1b739350 2024-04-24T08:56:16.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.526+0000 7f116dbde740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x561c1b746990 con 0x561c1b738a30 2024-04-24T08:56:16.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.526+0000 7f116dbde740 10 client.0 _ll_register_callbacks cb 0x561c1b693650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:56:16.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.526+0000 7f1168c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c1b743a20 0x561c1b741400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:16.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.526+0000 7f1168c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c1b743a20 0x561c1b741400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46002/0 (socket says 172.21.15.83:46002) 2024-04-24T08:56:16.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.527+0000 7f1168c00640 1 -- 172.21.15.83:0/3326558177 learned_addr learned my addr 172.21.15.83:0/3326558177 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:56:16.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.527+0000 7f1163e00640 1 --2- 172.21.15.83:0/3326558177 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c1b738a30 0x561c1b740980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:16.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.527+0000 7f115b400640 1 --2- 172.21.15.83:0/3326558177 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c1b739350 0x561c1b740ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:16.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.527+0000 7f1168c00640 1 -- 172.21.15.83:0/3326558177 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c1b738a30 msgr2=0x561c1b740980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:16.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.527+0000 7f1168c00640 1 --2- 172.21.15.83:0/3326558177 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c1b738a30 0x561c1b740980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:16.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.527+0000 7f1168c00640 1 -- 172.21.15.83:0/3326558177 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c1b739350 msgr2=0x561c1b740ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:16.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.527+0000 7f1168c00640 1 --2- 172.21.15.83:0/3326558177 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c1b739350 0x561c1b740ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:16.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.527+0000 7f1168c00640 1 -- 172.21.15.83:0/3326558177 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c1b656920 con 0x561c1b743a20 2024-04-24T08:56:16.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.527+0000 7f1168c00640 1 --2- 172.21.15.83:0/3326558177 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c1b743a20 0x561c1b741400 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f115c09c6f0 tx=0x7f115c09cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a9b75f63e4fa6e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:16.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.527+0000 7f1151600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:56:16.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.527+0000 7f1151600640 1 -- 172.21.15.83:0/3326558177 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f115c01f040 con 0x561c1b743a20 2024-04-24T08:56:16.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.527+0000 7f1151600640 1 -- 172.21.15.83:0/3326558177 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f115c0bbbb0 con 0x561c1b743a20 2024-04-24T08:56:16.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.528+0000 7f116dbde740 1 -- 172.21.15.83:0/3326558177 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f115401a910 con 0x561c1b743a20 2024-04-24T08:56:16.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.528+0000 7f1151600640 1 -- 172.21.15.83:0/3326558177 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f115c0bbdb0 con 0x561c1b743a20 2024-04-24T08:56:16.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.528+0000 7f116dbde740 10 client.17397 fetch_fsmap learned FSMap version 32 2024-04-24T08:56:16.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.528+0000 7f116dbde740 1 -- 172.21.15.83:0/3326558177 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x561c1b47a7d0 con 0x561c1b743a20 2024-04-24T08:56:16.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.528+0000 7f1151600640 1 -- 172.21.15.83:0/3326558177 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f115401a910 con 0x561c1b743a20 2024-04-24T08:56:16.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.528+0000 7f1151600640 1 -- 172.21.15.83:0/3326558177 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7f115c00f040 con 0x561c1b743a20 2024-04-24T08:56:16.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.528+0000 7f116dbde740 10 client.17397 fetch_fsmap finished waiting for FSMap version 32 2024-04-24T08:56:16.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.528+0000 7f116dbde740 10 client.17397 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:56:16.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.528+0000 7f116dbde740 20 client.17397 populate_metadata read hostname 'smithi083' 2024-04-24T08:56:16.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.528+0000 7f116dbde740 1 --2- 172.21.15.83:0/3326558177 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x561c1b753e20 0x561c1b756250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:16.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.528+0000 7f116dbde740 4 client.17397 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:56:16.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.528+0000 7f116dbde740 1 -- 172.21.15.83:0/3326558177 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561c1b6abc10 con 0x561c1b753e20 2024-04-24T08:56:16.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.529+0000 7f1163e00640 1 --2- 172.21.15.83:0/3326558177 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x561c1b753e20 0x561c1b756250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:16.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.530+0000 7f1163e00640 1 --2- 172.21.15.83:0/3326558177 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x561c1b753e20 0x561c1b756250 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e859865af9ba70a server_cookie=89af3a7577935f3d in_seq=0 out_seq=0 2024-04-24T08:56:16.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.530+0000 7f1151600640 10 client.17397 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:56:16.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.532+0000 7f1151600640 1 -- 172.21.15.83:0/3326558177 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561c1b6abc10 con 0x561c1b753e20 2024-04-24T08:56:16.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.532+0000 7f1151600640 10 client.17397 handle_command_reply: tid=0 2024-04-24T08:56:16.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.532+0000 7f116dbde740 1 client.17397 shutdown 2024-04-24T08:56:16.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.532+0000 7f116dbde740 1 -- 172.21.15.83:0/3326558177 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c1b743a20 msgr2=0x561c1b741400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:16.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.532+0000 7f116dbde740 1 --2- 172.21.15.83:0/3326558177 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c1b743a20 0x561c1b741400 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f115c09c6f0 tx=0x7f115c09cab0 comp rx=0 tx=0).stop 2024-04-24T08:56:16.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.532+0000 7f116dbde740 1 -- 172.21.15.83:0/3326558177 shutdown_connections 2024-04-24T08:56:16.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.532+0000 7f116dbde740 1 --2- 172.21.15.83:0/3326558177 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x561c1b753e20 0x561c1b756250 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:16.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.532+0000 7f116dbde740 1 --2- 172.21.15.83:0/3326558177 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c1b743a20 0x561c1b741400 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:16.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.532+0000 7f116dbde740 1 --2- 172.21.15.83:0/3326558177 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c1b739350 0x561c1b740ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:16.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.532+0000 7f116dbde740 1 --2- 172.21.15.83:0/3326558177 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c1b738a30 0x561c1b740980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:16.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.532+0000 7f1151600640 0 client.17397 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:56:16.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.532+0000 7f116dbde740 1 -- 172.21.15.83:0/3326558177 >> 172.21.15.83:0/3326558177 conn(0x561c1b734d50 msgr2=0x561c1b733fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:16.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.533+0000 7f116dbde740 1 -- 172.21.15.83:0/3326558177 shutdown_connections 2024-04-24T08:56:16.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.533+0000 7f116dbde740 1 -- 172.21.15.83:0/3326558177 wait complete. 2024-04-24T08:56:16.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.533+0000 7f116dbde740 20 client.17397 trim_cache size 0 max 16384 2024-04-24T08:56:16.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.555+0000 7f116dbde740 1 Processor -- start 2024-04-24T08:56:16.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.556+0000 7f116dbde740 1 -- start start 2024-04-24T08:56:16.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.556+0000 7f116dbde740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c1b738a30 0x561c1b74f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:16.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.556+0000 7f116dbde740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c1b739350 0x561c1b735130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:16.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.556+0000 7f116dbde740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c1b743a20 0x561c1b73a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:16.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.556+0000 7f116dbde740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f11641b14f0 con 0x561c1b743a20 2024-04-24T08:56:16.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.556+0000 7f116dbde740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f11641b3060 con 0x561c1b739350 2024-04-24T08:56:16.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.556+0000 7f116dbde740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f11641b1bc0 con 0x561c1b738a30 2024-04-24T08:56:16.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.556+0000 7f1168c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c1b743a20 0x561c1b73a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:16.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.556+0000 7f1168c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c1b743a20 0x561c1b73a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46016/0 (socket says 172.21.15.83:46016) 2024-04-24T08:56:16.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.556+0000 7f1168c00640 1 -- 172.21.15.83:0/1334439318 learned_addr learned my addr 172.21.15.83:0/1334439318 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:56:16.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.557+0000 7f115b400640 1 --2- 172.21.15.83:0/1334439318 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c1b739350 0x561c1b735130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:16.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.557+0000 7f1163e00640 1 --2- 172.21.15.83:0/1334439318 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c1b738a30 0x561c1b74f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:16.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.557+0000 7f1168c00640 1 -- 172.21.15.83:0/1334439318 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c1b738a30 msgr2=0x561c1b74f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:16.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.557+0000 7f1168c00640 1 --2- 172.21.15.83:0/1334439318 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c1b738a30 0x561c1b74f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:16.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.557+0000 7f1168c00640 1 -- 172.21.15.83:0/1334439318 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c1b739350 msgr2=0x561c1b735130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:16.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.557+0000 7f1168c00640 1 --2- 172.21.15.83:0/1334439318 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c1b739350 0x561c1b735130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:16.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.557+0000 7f1168c00640 1 -- 172.21.15.83:0/1334439318 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c1b3b03f0 con 0x561c1b743a20 2024-04-24T08:56:16.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.557+0000 7f1168c00640 1 --2- 172.21.15.83:0/1334439318 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c1b743a20 0x561c1b73a890 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f115c098c30 tx=0x7f115c0bd0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8afbf17f196a25d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:16.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.557+0000 7f1153e00640 1 -- 172.21.15.83:0/1334439318 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f115c01f070 con 0x561c1b743a20 2024-04-24T08:56:16.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.558+0000 7f1153e00640 1 -- 172.21.15.83:0/1334439318 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f115c098910 con 0x561c1b743a20 2024-04-24T08:56:16.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.558+0000 7f1153e00640 1 -- 172.21.15.83:0/1334439318 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f115c0b7020 con 0x561c1b743a20 2024-04-24T08:56:16.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.558+0000 7f116dbde740 1 -- 172.21.15.83:0/1334439318 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c1b743a20 msgr2=0x561c1b73a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:16.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.558+0000 7f116dbde740 1 --2- 172.21.15.83:0/1334439318 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c1b743a20 0x561c1b73a890 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f115c098c30 tx=0x7f115c0bd0d0 comp rx=0 tx=0).stop 2024-04-24T08:56:16.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.558+0000 7f116dbde740 1 -- 172.21.15.83:0/1334439318 shutdown_connections 2024-04-24T08:56:16.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.558+0000 7f116dbde740 1 --2- 172.21.15.83:0/1334439318 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c1b743a20 0x561c1b73a890 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:16.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.558+0000 7f116dbde740 1 --2- 172.21.15.83:0/1334439318 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c1b739350 0x561c1b735130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:16.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.558+0000 7f116dbde740 1 --2- 172.21.15.83:0/1334439318 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c1b738a30 0x561c1b74f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:16.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.558+0000 7f116dbde740 1 -- 172.21.15.83:0/1334439318 >> 172.21.15.83:0/1334439318 conn(0x561c1b734d50 msgr2=0x561c1b73a180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:16.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.558+0000 7f116dbde740 1 -- 172.21.15.83:0/1334439318 shutdown_connections 2024-04-24T08:56:16.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.559+0000 7f116dbde740 1 -- 172.21.15.83:0/1334439318 wait complete. 2024-04-24T08:56:16.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.559+0000 7f116dbde740 1 -- start start 2024-04-24T08:56:16.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.559+0000 7f116dbde740 1 Processor -- start 2024-04-24T08:56:16.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.560+0000 7f116dbde740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c1b738a30 0x561c1b74c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:16.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.560+0000 7f116dbde740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c1b739350 0x561c1b747440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:16.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.560+0000 7f116dbde740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c1b743a20 0x561c1b747a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:16.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.560+0000 7f116dbde740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f11641b2960 con 0x561c1b743a20 2024-04-24T08:56:16.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.560+0000 7f116dbde740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f11641b2290 con 0x561c1b738a30 2024-04-24T08:56:16.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.560+0000 7f116dbde740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f11641af4d0 con 0x561c1b739350 2024-04-24T08:56:16.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.560+0000 7f116dbde740 10 client.0 _ll_register_callbacks cb 0x7f115c098c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:56:16.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.560+0000 7f1168c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c1b743a20 0x561c1b747a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:16.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.560+0000 7f1168c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c1b743a20 0x561c1b747a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46030/0 (socket says 172.21.15.83:46030) 2024-04-24T08:56:16.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.560+0000 7f1168c00640 1 -- 172.21.15.83:0/2729574520 learned_addr learned my addr 172.21.15.83:0/2729574520 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:56:16.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.560+0000 7f1163e00640 1 --2- 172.21.15.83:0/2729574520 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c1b738a30 0x561c1b74c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:16.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.560+0000 7f115b400640 1 --2- 172.21.15.83:0/2729574520 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c1b739350 0x561c1b747440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:16.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.560+0000 7f1168c00640 1 -- 172.21.15.83:0/2729574520 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c1b739350 msgr2=0x561c1b747440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:16.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.560+0000 7f1168c00640 1 --2- 172.21.15.83:0/2729574520 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c1b739350 0x561c1b747440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:16.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.560+0000 7f1168c00640 1 -- 172.21.15.83:0/2729574520 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c1b738a30 msgr2=0x561c1b74c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:16.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.560+0000 7f1168c00640 1 --2- 172.21.15.83:0/2729574520 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c1b738a30 0x561c1b74c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:16.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.560+0000 7f1168c00640 1 -- 172.21.15.83:0/2729574520 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c1b3700f0 con 0x561c1b743a20 2024-04-24T08:56:16.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.561+0000 7f1168c00640 1 --2- 172.21.15.83:0/2729574520 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c1b743a20 0x561c1b747a50 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f115c014040 tx=0x7f115c09cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adfb5d20eb538e31 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:16.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.561+0000 7f1151600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:56:16.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.561+0000 7f1151600640 1 -- 172.21.15.83:0/2729574520 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f115c082060 con 0x561c1b743a20 2024-04-24T08:56:16.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.561+0000 7f1151600640 1 -- 172.21.15.83:0/2729574520 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f115c0872c0 con 0x561c1b743a20 2024-04-24T08:56:16.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.561+0000 7f116dbde740 1 -- 172.21.15.83:0/2729574520 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f115c098910 con 0x561c1b743a20 2024-04-24T08:56:16.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.561+0000 7f1151600640 1 -- 172.21.15.83:0/2729574520 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f115c0b7070 con 0x561c1b743a20 2024-04-24T08:56:16.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.561+0000 7f116dbde740 10 client.17403 fetch_fsmap learned FSMap version 32 2024-04-24T08:56:16.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.561+0000 7f116dbde740 1 -- 172.21.15.83:0/2729574520 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x561c1b663810 con 0x561c1b743a20 2024-04-24T08:56:16.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.561+0000 7f1151600640 1 -- 172.21.15.83:0/2729574520 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f115c098910 con 0x561c1b743a20 2024-04-24T08:56:16.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.562+0000 7f1151600640 1 -- 172.21.15.83:0/2729574520 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7f115c0a1f30 con 0x561c1b743a20 2024-04-24T08:56:16.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.562+0000 7f116dbde740 10 client.17403 fetch_fsmap finished waiting for FSMap version 32 2024-04-24T08:56:16.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.562+0000 7f116dbde740 10 client.17403 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:56:16.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.562+0000 7f116dbde740 20 client.17403 populate_metadata read hostname 'smithi083' 2024-04-24T08:56:16.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.562+0000 7f116dbde740 1 --2- 172.21.15.83:0/2729574520 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x561c1b7ba5c0 0x561c1b7bc9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:16.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.562+0000 7f116dbde740 4 client.17403 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T08:56:16.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.562+0000 7f116dbde740 1 -- 172.21.15.83:0/2729574520 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561c1b6a5a80 con 0x561c1b7ba5c0 2024-04-24T08:56:16.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.562+0000 7f1163e00640 1 --2- 172.21.15.83:0/2729574520 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x561c1b7ba5c0 0x561c1b7bc9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:16.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.564+0000 7f1163e00640 1 --2- 172.21.15.83:0/2729574520 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x561c1b7ba5c0 0x561c1b7bc9b0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1bc3f9bd4f9462 server_cookie=4559dadc86f966f9 in_seq=0 out_seq=0 2024-04-24T08:56:16.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.564+0000 7f1151600640 10 client.17403 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:56:16.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.565+0000 7f1151600640 1 -- 172.21.15.83:0/2729574520 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x561c1b6a5a80 con 0x561c1b7ba5c0 2024-04-24T08:56:16.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.565+0000 7f1151600640 10 client.17403 handle_command_reply: tid=0 2024-04-24T08:56:16.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.565+0000 7f116dbde740 1 client.17403 shutdown 2024-04-24T08:56:16.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.565+0000 7f116dbde740 1 -- 172.21.15.83:0/2729574520 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c1b743a20 msgr2=0x561c1b747a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:16.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.565+0000 7f116dbde740 1 --2- 172.21.15.83:0/2729574520 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c1b743a20 0x561c1b747a50 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f115c014040 tx=0x7f115c09cab0 comp rx=0 tx=0).stop 2024-04-24T08:56:16.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.565+0000 7f116dbde740 1 -- 172.21.15.83:0/2729574520 shutdown_connections 2024-04-24T08:56:16.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.565+0000 7f116dbde740 1 --2- 172.21.15.83:0/2729574520 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x561c1b7ba5c0 0x561c1b7bc9b0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:16.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.565+0000 7f116dbde740 1 --2- 172.21.15.83:0/2729574520 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c1b743a20 0x561c1b747a50 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:16.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.565+0000 7f116dbde740 1 --2- 172.21.15.83:0/2729574520 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c1b739350 0x561c1b747440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:16.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.565+0000 7f116dbde740 1 --2- 172.21.15.83:0/2729574520 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c1b738a30 0x561c1b74c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:16.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.565+0000 7f1151600640 0 client.17403 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:56:16.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.565+0000 7f116dbde740 1 -- 172.21.15.83:0/2729574520 >> 172.21.15.83:0/2729574520 conn(0x561c1b734d50 msgr2=0x561c1b73aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:16.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.565+0000 7f116dbde740 1 -- 172.21.15.83:0/2729574520 shutdown_connections 2024-04-24T08:56:16.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.565+0000 7f116dbde740 1 -- 172.21.15.83:0/2729574520 wait complete. 2024-04-24T08:56:16.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.566+0000 7f116dbde740 20 client.17403 trim_cache size 0 max 16384 2024-04-24T08:56:16.568 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T08:56:16.568 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T08:56:16.569 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-04-24T08:56:16.569 INFO:teuthology.orchestra.run.smithi083.stdout: "fefe782a-33f3-4c85-b825-61330ae6c8ed": { 2024-04-24T08:56:16.569 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-04-24T08:56:16.569 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "fefe782a-33f3-4c85-b825-61330ae6c8ed", 2024-04-24T08:56:16.569 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-04-24T08:56:16.569 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T08:56:16.569 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T08:56:16.569 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T08:56:16.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.566+0000 7f1169600640 1 -- 172.21.15.83:0/3777463538 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f11440a3d20 msgr2=0x7f11440a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:16.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.566+0000 7f1169600640 1 --2- 172.21.15.83:0/3777463538 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f11440a3d20 0x7f11440a61e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f114c006010 tx=0x7f114c000bc0 comp rx=0 tx=0).stop 2024-04-24T08:56:16.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.566+0000 7f1169600640 1 -- 172.21.15.83:0/3777463538 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f116419a700 msgr2=0x7f116419ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:16.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.566+0000 7f1169600640 1 --2- 172.21.15.83:0/3777463538 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f116419a700 0x7f116419ab80 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f115c00eaa0 tx=0x7f115c00ef60 comp rx=0 tx=0).stop 2024-04-24T08:56:16.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.566+0000 7f1169600640 1 -- 172.21.15.83:0/3777463538 shutdown_connections 2024-04-24T08:56:16.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.566+0000 7f1169600640 1 --2- 172.21.15.83:0/3777463538 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f11440a3d20 0x7f11440a61e0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:16.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.566+0000 7f1169600640 1 --2- 172.21.15.83:0/3777463538 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f116419a700 0x7f116419ab80 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:16.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.566+0000 7f1169600640 1 --2- 172.21.15.83:0/3777463538 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11641093f0 0x7f11641a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:16.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.566+0000 7f1169600640 1 --2- 172.21.15.83:0/3777463538 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1164104320 0x7f11641a0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:16.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.566+0000 7f1169600640 1 -- 172.21.15.83:0/3777463538 >> 172.21.15.83:0/3777463538 conn(0x7f11640ffb40 msgr2=0x7f1164100f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:16.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.567+0000 7f1169600640 1 -- 172.21.15.83:0/3777463538 shutdown_connections 2024-04-24T08:56:16.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:16.567+0000 7f1169600640 1 -- 172.21.15.83:0/3777463538 wait complete. 2024-04-24T08:56:16.572 INFO:tasks.workunit.client.0.smithi083.stdout: 1 21103 0.67 MB/sec execute 50 sec latency 65.954 ms 2024-04-24T08:56:16.586 INFO:tasks.cephfs.filesystem:scrub status for tag:fefe782a-33f3-4c85-b825-61330ae6c8ed - {'path': '/', 'tag': 'fefe782a-33f3-4c85-b825-61330ae6c8ed', 'options': 'recursive,force'} 2024-04-24T08:56:16.631 INFO:journalctl@ceph.mds.b.smithi138.stdout:Apr 24 08:56:16 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-b[61515]: 2024-04-24T08:56:16.310+0000 7f7fbbc00700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000215 [...7,head] /volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/COREL/ auth{2=1} v3614 ap=2 f(v130 m2024-04-24T08:55:55.537694+0000 11=11+0) n(v5 rc2024-04-24T08:55:55.537694+0000 b2861056 12=11+1) old_inodes=3 (inest sync->mix g=2) caps={15603=pAsLsXsFs/-@2978} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x55cf8760d700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000000215:[<0x10000000009/COREL v369>,<0x10000000008/~dmtmp v57>,<0x10000000007/client0 v33>,<0x10000000006/clients v23>,<0x10000000005/tmp v23>,<0x10000000003/client.0 v22>,<0x10000000002/e395bed6-5bba-41b8-a256-3505e4afcca2 v36>,<0x10000000001/sv_0 v55>,<0x10000000000/qa v43>,<0x1/volumes v39>]//[]","memoryvalue":"(2)0x10000000215:[<0x10000000009/COREL v3614>,<0x10000000008/~dmtmp v3126>,<0x10000000007/client0 v396>,<0x10000000006/clients v386>,<0x10000000005/tmp v384>,<0x10000000003/client.0 v381>,<0x10000000002/e395bed6-5bba-41b8-a256-3505e4afcca2 v394>,<0x10000000001/sv_0 v408>,<0x10000000000/qa v392>,<0x1/volumes v378>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-04-24T08:55:55.497929+0000 12=12+0)","ondisk_value.rstat":"n(v0 rc2024-04-24T08:55:55.537694+0000 b2861056 12=11+1)","memory_value.dirstat":"f(v130 m2024-04-24T08:55:55.537694+0000 11=11+0)","memory_value.rstat":"n(v5 rc2024-04-24T08:55:55.537694+0000 b2861056 12=11+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-04-24T08:56:16.631 INFO:journalctl@ceph.mds.b.smithi138.stdout:Apr 24 08:56:16 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-b[61515]: 2024-04-24T08:56:16.316+0000 7f7fbbc00700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000208 [...7,head] /volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/WORD/ auth{2=1} v3748 ap=2 f(v120 m2024-04-24T08:56:01.985465+0000 5=5+0) n(v11 rc2024-04-24T08:56:01.985465+0000 b814326 6=5+1) old_inodes=3 (inest sync->mix g=2) caps={15603=pAsLsXsFs/-@3011} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x55cf87274c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000000208:[<0x10000000009/WORD v427>,<0x10000000008/~dmtmp v57>,<0x10000000007/client0 v33>,<0x10000000006/clients v23>,<0x10000000005/tmp v23>,<0x10000000003/client.0 v22>,<0x10000000002/e395bed6-5bba-41b8-a256-3505e4afcca2 v36>,<0x10000000001/sv_0 v55>,<0x10000000000/qa v43>,<0x1/volumes v39>]//[]","memoryvalue":"(2)0x10000000208:[<0x10000000009/WORD v3748>,<0x10000000008/~dmtmp v3126>,<0x10000000007/client0 v396>,<0x10000000006/clients v386>,<0x10000000005/tmp v384>,<0x10000000003/client.0 v381>,<0x10000000002/e395bed6-5bba-41b8-a256-3505e4afcca2 v394>,<0x10000000001/sv_0 v408>,<0x10000000000/qa v392>,<0x1/volumes v378>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-04-24T08:56:01.939599+0000 6=6+0)","ondisk_value.rstat":"n(v0 rc2024-04-24T08:56:01.985465+0000 b814326 6=5+1)","memory_value.dirstat":"f(v120 m2024-04-24T08:56:01.985465+0000 5=5+0)","memory_value.rstat":"n(v11 rc2024-04-24T08:56:01.985465+0000 b814326 6=5+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-04-24T08:56:16.631 INFO:journalctl@ceph.mds.b.smithi138.stdout:Apr 24 08:56:16 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-b[61515]: 2024-04-24T08:56:16.332+0000 7f7fbbc00700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000020f [...7,head] /volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/PWRPNT/ auth{1=1} v3696 ap=2 f(v140 m2024-04-24T08:55:59.508550+0000 7=7+0) n(v10 rc2024-04-24T08:55:59.508550+0000 b1462582 8=7+1) old_inodes=3 (inest sync->mix g=1) caps={15603=pAsLsXsFs/-@3001} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x55cf8760b600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000020f:[<0x10000000009/PWRPNT v274>,<0x10000000008/~dmtmp v57>,<0x10000000007/client0 v33>,<0x10000000006/clients v23>,<0x10000000005/tmp v23>,<0x10000000003/client.0 v22>,<0x10000000002/e395bed6-5bba-41b8-a256-3505e4afcca2 v36>,<0x10000000001/sv_0 v55>,<0x10000000000/qa v43>,<0x1/volumes v39>]//[]","memoryvalue":"(2)0x1000000020f:[<0x10000000009/PWRPNT v3696>,<0x10000000008/~dmtmp v3126>,<0x10000000007/client0 v396>,<0x10000000006/clients v386>,<0x10000000005/tmp v384>,<0x10000000003/client.0 v381>,<0x10000000002/e395bed6-5bba-41b8-a256-3505e4afcca2 v394>,<0x10000000001/sv_0 v408>,<0x10000000000/qa v392>,<0x1/volumes v378>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-04-24T08:55:59.409326+0000 8=8+0)","ondisk_value.rstat":"n(v0 rc2024-04-24T08:55:59.508550+0000 b1462582 8=7+1)","memory_value.dirstat":"f(v140 m2024-04-24T08:55:59.508550+0000 7=7+0)","memory_value.rstat":"n(v10 rc2024-04-24T08:55:59.508550+0000 b1462582 8=7+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-04-24T08:56:16.632 INFO:journalctl@ceph.mds.b.smithi138.stdout:Apr 24 08:56:16 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-b[61515]: 2024-04-24T08:56:16.333+0000 7f7fbbc00700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000225 [...7,head] /volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/PM/ auth{3=1} v3654 ap=2 f(v370 m2024-04-24T08:55:55.823974+0000 3=3+0) n(v10 rc2024-04-24T08:55:55.823974+0000 b780288 4=3+1) old_inodes=3 (inest sync->mix g=3) caps={15603=pAsLsXsFs/-@3896} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x55cf8762f180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000000225:[<0x10000000009/PM v381>,<0x10000000008/~dmtmp v57>,<0x10000000007/client0 v33>,<0x10000000006/clients v23>,<0x10000000005/tmp v23>,<0x10000000003/client.0 v22>,<0x10000000002/e395bed6-5bba-41b8-a256-3505e4afcca2 v36>,<0x10000000001/sv_0 v55>,<0x10000000000/qa v43>,<0x1/volumes v39>]//[]","memoryvalue":"(2)0x10000000225:[<0x10000000009/PM v3654>,<0x10000000008/~dmtmp v3126>,<0x10000000007/client0 v396>,<0x10000000006/clients v386>,<0x10000000005/tmp v384>,<0x10000000003/client.0 v381>,<0x10000000002/e395bed6-5bba-41b8-a256-3505e4afcca2 v394>,<0x10000000001/sv_0 v408>,<0x10000000000/qa v392>,<0x1/volumes v378>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-04-24T08:55:54.287168+0000 4=4+0)","ondisk_value.rstat":"n(v0 rc2024-04-24T08:55:55.823974+0000 b780288 4=3+1)","memory_value.dirstat":"f(v370 m2024-04-24T08:55:55.823974+0000 3=3+0)","memory_value.rstat":"n(v10 rc2024-04-24T08:55:55.823974+0000 b780288 4=3+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-04-24T08:56:16.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:56:16 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T08:56:16.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:56:16 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T08:56:16.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:56:16 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T08:56:16.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:56:16 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T08:56:17.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:56:16 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T08:56:17.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:56:16 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T08:56:17.573 INFO:tasks.workunit.client.0.smithi083.stdout: 1 21157 0.66 MB/sec execute 51 sec latency 56.402 ms 2024-04-24T08:56:17.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:56:17 smithi138 ceph-mon[31441]: scrub queued for path: / 2024-04-24T08:56:17.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:56:17 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T08:56:17.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:56:17 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T08:56:17.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:56:17 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T08:56:17.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:56:17 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T08:56:17.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:56:17 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T08:56:17.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:56:17 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T08:56:17.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:56:17 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T08:56:17.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:56:17 smithi138 ceph-mon[31441]: Scrub error on inode 0x10000000215 (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/COREL) see mds.b log and `damage ls` output for details 2024-04-24T08:56:17.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:56:17 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T08:56:17.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:56:17 smithi138 ceph-mon[31441]: Scrub error on inode 0x10000000208 (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/WORD) see mds.b log and `damage ls` output for details 2024-04-24T08:56:17.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:56:17 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T08:56:17.957 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:56:17 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T08:56:17.957 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:56:17 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T08:56:17.957 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:56:17 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T08:56:17.957 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:56:17 smithi138 ceph-mon[31441]: Scrub error on inode 0x1000000020f (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/PWRPNT) see mds.b log and `damage ls` output for details 2024-04-24T08:56:17.957 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:56:17 smithi138 ceph-mon[31441]: Scrub error on inode 0x10000000225 (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/PM) see mds.b log and `damage ls` output for details 2024-04-24T08:56:17.957 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:56:17 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T08:56:17.957 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:56:17 smithi138 ceph-mon[31441]: pgmap v143: 129 pgs: 129 active+clean; 37 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 255 KiB/s wr, 39 op/s 2024-04-24T08:56:17.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:56:17 smithi189 ceph-mon[31791]: scrub queued for path: / 2024-04-24T08:56:17.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:56:17 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T08:56:17.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:56:17 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T08:56:17.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:56:17 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T08:56:17.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:56:17 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T08:56:17.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:56:17 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T08:56:17.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:56:17 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T08:56:17.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:56:17 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T08:56:17.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:56:17 smithi189 ceph-mon[31791]: Scrub error on inode 0x10000000215 (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/COREL) see mds.b log and `damage ls` output for details 2024-04-24T08:56:17.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:56:17 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T08:56:17.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:56:17 smithi189 ceph-mon[31791]: Scrub error on inode 0x10000000208 (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/WORD) see mds.b log and `damage ls` output for details 2024-04-24T08:56:17.959 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:56:17 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T08:56:17.959 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:56:17 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T08:56:17.959 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:56:17 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T08:56:17.959 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:56:17 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T08:56:17.959 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:56:17 smithi189 ceph-mon[31791]: Scrub error on inode 0x1000000020f (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/PWRPNT) see mds.b log and `damage ls` output for details 2024-04-24T08:56:17.959 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:56:17 smithi189 ceph-mon[31791]: Scrub error on inode 0x10000000225 (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/PM) see mds.b log and `damage ls` output for details 2024-04-24T08:56:17.959 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:56:17 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T08:56:17.959 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:56:17 smithi189 ceph-mon[31791]: pgmap v143: 129 pgs: 129 active+clean; 37 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 255 KiB/s wr, 39 op/s 2024-04-24T08:56:18.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:56:17 smithi083 ceph-mon[27907]: scrub queued for path: / 2024-04-24T08:56:18.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:56:17 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T08:56:18.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:56:17 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T08:56:18.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:56:17 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T08:56:18.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:56:17 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T08:56:18.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:56:17 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T08:56:18.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:56:17 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T08:56:18.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:56:17 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T08:56:18.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:56:17 smithi083 ceph-mon[27907]: Scrub error on inode 0x10000000215 (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/COREL) see mds.b log and `damage ls` output for details 2024-04-24T08:56:18.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:56:17 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T08:56:18.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:56:17 smithi083 ceph-mon[27907]: Scrub error on inode 0x10000000208 (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/WORD) see mds.b log and `damage ls` output for details 2024-04-24T08:56:18.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:56:17 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T08:56:18.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:56:17 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T08:56:18.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:56:17 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T08:56:18.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:56:17 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T08:56:18.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:56:17 smithi083 ceph-mon[27907]: Scrub error on inode 0x1000000020f (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/PWRPNT) see mds.b log and `damage ls` output for details 2024-04-24T08:56:18.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:56:17 smithi083 ceph-mon[27907]: Scrub error on inode 0x10000000225 (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/PM) see mds.b log and `damage ls` output for details 2024-04-24T08:56:18.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:56:17 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T08:56:18.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:56:17 smithi083 ceph-mon[27907]: pgmap v143: 129 pgs: 129 active+clean; 37 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 255 KiB/s wr, 39 op/s 2024-04-24T08:56:18.573 INFO:tasks.workunit.client.0.smithi083.stdout: 1 21337 0.70 MB/sec execute 52 sec latency 56.456 ms 2024-04-24T08:56:18.954 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:56:18 smithi138 ceph-mon[31441]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T08:56:18.956 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:56:18 smithi189 ceph-mon[31791]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T08:56:19.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:56:18 smithi083 ceph-mon[27907]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T08:56:19.573 INFO:tasks.workunit.client.0.smithi083.stdout: 1 21459 0.70 MB/sec execute 53 sec latency 54.485 ms 2024-04-24T08:56:19.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:56:19 smithi138 ceph-mon[31441]: pgmap v144: 129 pgs: 129 active+clean; 35 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 528 KiB/s wr, 70 op/s 2024-04-24T08:56:19.956 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:56:19 smithi189 ceph-mon[31791]: pgmap v144: 129 pgs: 129 active+clean; 35 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 528 KiB/s wr, 70 op/s 2024-04-24T08:56:20.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:56:19 smithi083 ceph-mon[27907]: pgmap v144: 129 pgs: 129 active+clean; 35 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 528 KiB/s wr, 70 op/s 2024-04-24T08:56:20.573 INFO:tasks.workunit.client.0.smithi083.stdout: 1 21563 0.70 MB/sec execute 54 sec latency 47.794 ms 2024-04-24T08:56:21.573 INFO:tasks.workunit.client.0.smithi083.stdout: 1 21621 0.69 MB/sec execute 55 sec latency 42.440 ms 2024-04-24T08:56:21.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:56:21 smithi138 ceph-mon[31441]: pgmap v145: 129 pgs: 129 active+clean; 35 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 472 KiB/s wr, 50 op/s 2024-04-24T08:56:21.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:56:21 smithi189 ceph-mon[31791]: pgmap v145: 129 pgs: 129 active+clean; 35 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 472 KiB/s wr, 50 op/s 2024-04-24T08:56:22.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:56:21 smithi083 ceph-mon[27907]: pgmap v145: 129 pgs: 129 active+clean; 35 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 472 KiB/s wr, 50 op/s 2024-04-24T08:56:22.573 INFO:tasks.workunit.client.0.smithi083.stdout: 1 21689 0.68 MB/sec execute 56 sec latency 54.705 ms 2024-04-24T08:56:22.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:56:22 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T08:56:22.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:56:22 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T08:56:22.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:56:22 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T08:56:22.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:56:22 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T08:56:22.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:56:22 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T08:56:22.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:56:22 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T08:56:23.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:56:22 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T08:56:23.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:56:22 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T08:56:23.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:56:22 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T08:56:23.573 INFO:tasks.workunit.client.0.smithi083.stdout: 1 21742 0.67 MB/sec execute 57 sec latency 46.409 ms 2024-04-24T08:56:23.954 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:56:23 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T08:56:23.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:56:23 smithi138 ceph-mon[31441]: pgmap v146: 129 pgs: 129 active+clean; 36 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 647 KiB/s wr, 73 op/s 2024-04-24T08:56:23.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:56:23 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T08:56:23.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:56:23 smithi189 ceph-mon[31791]: pgmap v146: 129 pgs: 129 active+clean; 36 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 647 KiB/s wr, 73 op/s 2024-04-24T08:56:24.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:56:23 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T08:56:24.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:56:23 smithi083 ceph-mon[27907]: pgmap v146: 129 pgs: 129 active+clean; 36 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 647 KiB/s wr, 73 op/s 2024-04-24T08:56:24.573 INFO:tasks.workunit.client.0.smithi083.stdout: 1 21799 0.66 MB/sec execute 58 sec latency 48.330 ms 2024-04-24T08:56:25.574 INFO:tasks.workunit.client.0.smithi083.stdout: 1 21876 0.65 MB/sec execute 59 sec latency 58.208 ms 2024-04-24T08:56:25.954 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:56:25 smithi138 ceph-mon[31441]: pgmap v147: 129 pgs: 129 active+clean; 36 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 448 KiB/s wr, 54 op/s 2024-04-24T08:56:25.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:56:25 smithi189 ceph-mon[31791]: pgmap v147: 129 pgs: 129 active+clean; 36 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 448 KiB/s wr, 54 op/s 2024-04-24T08:56:26.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:56:25 smithi083 ceph-mon[27907]: pgmap v147: 129 pgs: 129 active+clean; 36 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 448 KiB/s wr, 54 op/s 2024-04-24T08:56:26.574 INFO:tasks.workunit.client.0.smithi083.stdout: 1 21944 0.64 MB/sec execute 60 sec latency 49.318 ms 2024-04-24T08:56:26.813 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.812+0000 7f1e0a000640 1 -- 172.21.15.83:0/4274818399 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e04103900 msgr2=0x7f1e0410bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:26.813 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.812+0000 7f1e0a000640 1 --2- 172.21.15.83:0/4274818399 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e04103900 0x7f1e0410bde0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f1df0009900 tx=0x7f1df002a3f0 comp rx=0 tx=0).stop 2024-04-24T08:56:26.813 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.813+0000 7f1e0a000640 1 -- 172.21.15.83:0/4274818399 shutdown_connections 2024-04-24T08:56:26.813 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.813+0000 7f1e0a000640 1 --2- 172.21.15.83:0/4274818399 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1e0410c450 0x7f1e0410e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:26.814 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.813+0000 7f1e0a000640 1 --2- 172.21.15.83:0/4274818399 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e04103900 0x7f1e0410bde0 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:26.814 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.813+0000 7f1e0a000640 1 --2- 172.21.15.83:0/4274818399 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e04102fe0 0x7f1e041033c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:26.814 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.813+0000 7f1e0a000640 1 -- 172.21.15.83:0/4274818399 >> 172.21.15.83:0/4274818399 conn(0x7f1e040fc7e0 msgr2=0x7f1e040fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:26.814 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.813+0000 7f1e0a000640 1 -- 172.21.15.83:0/4274818399 shutdown_connections 2024-04-24T08:56:26.814 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.813+0000 7f1e0a000640 1 -- 172.21.15.83:0/4274818399 wait complete. 2024-04-24T08:56:26.816 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.814+0000 7f1e0a000640 1 Processor -- start 2024-04-24T08:56:26.816 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.814+0000 7f1e0a000640 1 -- start start 2024-04-24T08:56:26.816 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.814+0000 7f1e0a000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e04102fe0 0x7f1e041a0590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:26.817 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.814+0000 7f1e0a000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e04103900 0x7f1e041a0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:26.817 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.814+0000 7f1e0a000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1e0410c450 0x7f1e0419a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:26.817 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.814+0000 7f1e0a000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1e041120f0 con 0x7f1e04102fe0 2024-04-24T08:56:26.817 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.814+0000 7f1e0a000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1e04111f70 con 0x7f1e04103900 2024-04-24T08:56:26.817 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.814+0000 7f1e0a000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1e04112270 con 0x7f1e0410c450 2024-04-24T08:56:26.817 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.815+0000 7f1e08c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e04102fe0 0x7f1e041a0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:26.817 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.815+0000 7f1e08c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e04102fe0 0x7f1e041a0590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40058/0 (socket says 172.21.15.83:40058) 2024-04-24T08:56:26.817 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.815+0000 7f1e08c00640 1 -- 172.21.15.83:0/3509819881 learned_addr learned my addr 172.21.15.83:0/3509819881 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:56:26.817 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.815+0000 7f1e08c00640 1 -- 172.21.15.83:0/3509819881 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1e0410c450 msgr2=0x7f1e0419a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:26.817 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.815+0000 7f1e09600640 1 --2- 172.21.15.83:0/3509819881 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1e0410c450 0x7f1e0419a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:26.817 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.815+0000 7f1e03e00640 1 --2- 172.21.15.83:0/3509819881 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e04103900 0x7f1e041a0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:26.817 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.815+0000 7f1e08c00640 1 --2- 172.21.15.83:0/3509819881 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1e0410c450 0x7f1e0419a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:26.817 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.815+0000 7f1e08c00640 1 -- 172.21.15.83:0/3509819881 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e04103900 msgr2=0x7f1e041a0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:26.817 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.815+0000 7f1e08c00640 1 --2- 172.21.15.83:0/3509819881 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e04103900 0x7f1e041a0ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:26.818 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.815+0000 7f1e08c00640 1 -- 172.21.15.83:0/3509819881 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e0419af10 con 0x7f1e04102fe0 2024-04-24T08:56:26.818 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.815+0000 7f1e03e00640 1 --2- 172.21.15.83:0/3509819881 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e04103900 0x7f1e041a0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:56:26.818 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.815+0000 7f1e09600640 1 --2- 172.21.15.83:0/3509819881 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1e0410c450 0x7f1e0419a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:56:26.818 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.815+0000 7f1e08c00640 1 --2- 172.21.15.83:0/3509819881 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e04102fe0 0x7f1e041a0590 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f1df800d950 tx=0x7f1df800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:26.818 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.816+0000 7f1e01600640 1 -- 172.21.15.83:0/3509819881 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df8017070 con 0x7f1e04102fe0 2024-04-24T08:56:26.818 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.816+0000 7f1e01600640 1 -- 172.21.15.83:0/3509819881 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1df8009c30 con 0x7f1e04102fe0 2024-04-24T08:56:26.818 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.816+0000 7f1e0a000640 1 -- 172.21.15.83:0/3509819881 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e0419b200 con 0x7f1e04102fe0 2024-04-24T08:56:26.818 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.816+0000 7f1e01600640 1 -- 172.21.15.83:0/3509819881 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df800f620 con 0x7f1e04102fe0 2024-04-24T08:56:26.818 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.816+0000 7f1e0a000640 1 -- 172.21.15.83:0/3509819881 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e041a7510 con 0x7f1e04102fe0 2024-04-24T08:56:26.820 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.820+0000 7f1e0e425740 1 Processor -- start 2024-04-24T08:56:26.820 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.821+0000 7f1e0e425740 1 -- start start 2024-04-24T08:56:26.822 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.821+0000 7f1e0e425740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbcb9853c0 0x55bbcb9857a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:26.822 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.821+0000 7f1e0e425740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bbcb985ce0 0x55bbcb990060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:26.822 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.821+0000 7f1e0e425740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bbcb9905a0 0x55bbcb992990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:26.822 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.821+0000 7f1e0e425740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55bbcb8eb900 con 0x55bbcb9905a0 2024-04-24T08:56:26.822 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.821+0000 7f1e0e425740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55bbcb891f20 con 0x55bbcb9853c0 2024-04-24T08:56:26.822 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.821+0000 7f1e01600640 1 -- 172.21.15.83:0/3509819881 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1df80109a0 con 0x7f1e04102fe0 2024-04-24T08:56:26.822 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.821+0000 7f1e0e425740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55bbcb8e78c0 con 0x55bbcb985ce0 2024-04-24T08:56:26.822 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.821+0000 7f1e03e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bbcb9905a0 0x55bbcb992990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:26.822 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.821+0000 7f1e03e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bbcb9905a0 0x55bbcb992990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40064/0 (socket says 172.21.15.83:40064) 2024-04-24T08:56:26.822 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.821+0000 7f1e03e00640 1 -- 172.21.15.83:0/1827651047 learned_addr learned my addr 172.21.15.83:0/1827651047 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:56:26.823 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.822+0000 7f1e09600640 1 --2- 172.21.15.83:0/1827651047 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbcb9853c0 0x55bbcb9857a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:26.823 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.822+0000 7f1e08c00640 1 --2- 172.21.15.83:0/1827651047 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bbcb985ce0 0x55bbcb990060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:26.823 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.822+0000 7f1e03e00640 1 -- 172.21.15.83:0/1827651047 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bbcb985ce0 msgr2=0x55bbcb990060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:26.823 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.822+0000 7f1e03e00640 1 --2- 172.21.15.83:0/1827651047 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bbcb985ce0 0x55bbcb990060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:26.823 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.822+0000 7f1e03e00640 1 -- 172.21.15.83:0/1827651047 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbcb9853c0 msgr2=0x55bbcb9857a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:26.823 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.822+0000 7f1e03e00640 1 --2- 172.21.15.83:0/1827651047 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbcb9853c0 0x55bbcb9857a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:26.823 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.822+0000 7f1e03e00640 1 -- 172.21.15.83:0/1827651047 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbcb8b6e10 con 0x55bbcb9905a0 2024-04-24T08:56:26.827 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.822+0000 7f1e03e00640 1 --2- 172.21.15.83:0/1827651047 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bbcb9905a0 0x55bbcb992990 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f1df002a910 tx=0x7f1df0004220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a44ec755cec914f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:26.827 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.823+0000 7f1df5c00640 1 -- 172.21.15.83:0/1827651047 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df003f070 con 0x55bbcb9905a0 2024-04-24T08:56:26.827 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.823+0000 7f1df5c00640 1 -- 172.21.15.83:0/1827651047 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1df00054f0 con 0x55bbcb9905a0 2024-04-24T08:56:26.827 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.823+0000 7f1df5c00640 1 -- 172.21.15.83:0/1827651047 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df0035030 con 0x55bbcb9905a0 2024-04-24T08:56:26.827 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.823+0000 7f1e0e425740 1 -- 172.21.15.83:0/1827651047 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bbcb9905a0 msgr2=0x55bbcb992990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:26.828 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.823+0000 7f1e0e425740 1 --2- 172.21.15.83:0/1827651047 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bbcb9905a0 0x55bbcb992990 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f1df002a910 tx=0x7f1df0004220 comp rx=0 tx=0).stop 2024-04-24T08:56:26.828 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.824+0000 7f1e0e425740 1 -- 172.21.15.83:0/1827651047 shutdown_connections 2024-04-24T08:56:26.828 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.824+0000 7f1e0e425740 1 --2- 172.21.15.83:0/1827651047 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bbcb9905a0 0x55bbcb992990 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:26.828 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.824+0000 7f1e0e425740 1 --2- 172.21.15.83:0/1827651047 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bbcb985ce0 0x55bbcb990060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:26.828 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.824+0000 7f1e0e425740 1 --2- 172.21.15.83:0/1827651047 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbcb9853c0 0x55bbcb9857a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:26.828 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.824+0000 7f1e0e425740 1 -- 172.21.15.83:0/1827651047 >> 172.21.15.83:0/1827651047 conn(0x55bbcb9818d0 msgr2=0x55bbcb981cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:26.828 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.824+0000 7f1e0e425740 1 -- 172.21.15.83:0/1827651047 shutdown_connections 2024-04-24T08:56:26.828 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.824+0000 7f1e0e425740 1 -- 172.21.15.83:0/1827651047 wait complete. 2024-04-24T08:56:26.828 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.824+0000 7f1e01600640 1 --2- 172.21.15.83:0/3509819881 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f1dec0a3c20 0x7f1dec0a60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:26.828 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.824+0000 7f1e0e425740 1 -- start start 2024-04-24T08:56:26.828 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.824+0000 7f1e01600640 1 -- 172.21.15.83:0/3509819881 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f1df80d08e0 con 0x7f1e04102fe0 2024-04-24T08:56:26.828 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.824+0000 7f1e03e00640 1 --2- 172.21.15.83:0/3509819881 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f1dec0a3c20 0x7f1dec0a60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:26.828 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.824+0000 7f1e0e425740 1 Processor -- start 2024-04-24T08:56:26.829 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.825+0000 7f1e0e425740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bbcb9853c0 0x55bbcb98d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:26.829 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.825+0000 7f1e0e425740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbcb985ce0 0x55bbcb98dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:26.829 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.825+0000 7f1e0e425740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bbcb98e2c0 0x55bbcb99c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:26.829 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.825+0000 7f1e0e425740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55bbcb993080 con 0x55bbcb9853c0 2024-04-24T08:56:26.829 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.825+0000 7f1e0e425740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55bbcb8a52c0 con 0x55bbcb985ce0 2024-04-24T08:56:26.829 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.825+0000 7f1e0e425740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55bbcb993200 con 0x55bbcb98e2c0 2024-04-24T08:56:26.829 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.825+0000 7f1e0e425740 10 client.0 _ll_register_callbacks cb 0x55bbcb5fda00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:56:26.829 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.825+0000 7f1e03e00640 1 --2- 172.21.15.83:0/3509819881 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f1dec0a3c20 0x7f1dec0a60e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1df0035000 tx=0x7f1df002a3f0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:26.829 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.825+0000 7f1e08c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bbcb9853c0 0x55bbcb98d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:26.829 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.825+0000 7f1e08c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bbcb9853c0 0x55bbcb98d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40070/0 (socket says 172.21.15.83:40070) 2024-04-24T08:56:26.829 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.825+0000 7f1e08c00640 1 -- 172.21.15.83:0/3639754753 learned_addr learned my addr 172.21.15.83:0/3639754753 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:56:26.829 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.826+0000 7f1e09600640 1 --2- 172.21.15.83:0/3639754753 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bbcb98e2c0 0x55bbcb99c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:26.829 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.826+0000 7f1e08c00640 1 -- 172.21.15.83:0/3639754753 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bbcb98e2c0 msgr2=0x55bbcb99c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:26.829 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.826+0000 7f1e03e00640 1 --2- 172.21.15.83:0/3639754753 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbcb985ce0 0x55bbcb98dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:26.830 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.826+0000 7f1e08c00640 1 --2- 172.21.15.83:0/3639754753 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bbcb98e2c0 0x55bbcb99c5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:26.830 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.826+0000 7f1e08c00640 1 -- 172.21.15.83:0/3639754753 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbcb985ce0 msgr2=0x55bbcb98dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:26.830 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.826+0000 7f1e08c00640 1 --2- 172.21.15.83:0/3639754753 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbcb985ce0 0x55bbcb98dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:26.830 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.826+0000 7f1e08c00640 1 -- 172.21.15.83:0/3639754753 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bbcb8b5fd0 con 0x55bbcb9853c0 2024-04-24T08:56:26.830 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.826+0000 7f1e08c00640 1 --2- 172.21.15.83:0/3639754753 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bbcb9853c0 0x55bbcb98d7f0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f1df8000c00 tx=0x7f1df809c320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14eae3a6be649a62 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:26.830 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.826+0000 7f1dea000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:56:26.830 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.826+0000 7f1dea000640 1 -- 172.21.15.83:0/3639754753 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df8020050 con 0x55bbcb9853c0 2024-04-24T08:56:26.830 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.826+0000 7f1dea000640 1 -- 172.21.15.83:0/3639754753 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f1df80a0590 con 0x55bbcb9853c0 2024-04-24T08:56:26.830 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.826+0000 7f1e0e425740 1 -- 172.21.15.83:0/3639754753 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1df00054f0 con 0x55bbcb9853c0 2024-04-24T08:56:26.830 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.827+0000 7f1dea000640 1 -- 172.21.15.83:0/3639754753 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f1df80ce080 con 0x55bbcb9853c0 2024-04-24T08:56:26.830 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.827+0000 7f1e0e425740 10 client.17415 fetch_fsmap learned FSMap version 33 2024-04-24T08:56:26.830 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.827+0000 7f1e0e425740 1 -- 172.21.15.83:0/3639754753 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55bbcb5fe210 con 0x55bbcb9853c0 2024-04-24T08:56:26.831 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.827+0000 7f1dea000640 1 -- 172.21.15.83:0/3639754753 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1df00054f0 con 0x55bbcb9853c0 2024-04-24T08:56:26.831 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.827+0000 7f1dea000640 1 -- 172.21.15.83:0/3639754753 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f1df80a01b0 con 0x55bbcb9853c0 2024-04-24T08:56:26.831 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.827+0000 7f1e0e425740 10 client.17415 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T08:56:26.831 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.827+0000 7f1e0e425740 10 client.17415 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T08:56:26.831 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.827+0000 7f1e0e425740 20 client.17415 populate_metadata read hostname 'smithi083' 2024-04-24T08:56:26.831 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.827+0000 7f1e0e425740 1 --2- 172.21.15.83:0/3639754753 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55bbcb9a0d90 0x55bbcb9a3180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:26.831 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.827+0000 7f1e0e425740 4 client.17415 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:56:26.831 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.827+0000 7f1e0e425740 1 -- 172.21.15.83:0/3639754753 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bbcb8f9320 con 0x55bbcb9a0d90 2024-04-24T08:56:26.832 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.828+0000 7f1e03e00640 1 --2- 172.21.15.83:0/3639754753 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55bbcb9a0d90 0x55bbcb9a3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:26.833 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.829+0000 7f1e03e00640 1 --2- 172.21.15.83:0/3639754753 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55bbcb9a0d90 0x55bbcb9a3180 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81d878dece6f1c8c server_cookie=1ac6cdeb78d972f in_seq=0 out_seq=0 2024-04-24T08:56:26.833 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.829+0000 7f1dea000640 10 client.17415 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:56:26.833 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.831+0000 7f1dea000640 1 -- 172.21.15.83:0/3639754753 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bbcb8f9320 con 0x55bbcb9a0d90 2024-04-24T08:56:26.833 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.831+0000 7f1dea000640 10 client.17415 handle_command_reply: tid=0 2024-04-24T08:56:26.833 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.831+0000 7f1e0e425740 1 client.17415 shutdown 2024-04-24T08:56:26.833 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.832+0000 7f1e0e425740 1 -- 172.21.15.83:0/3639754753 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bbcb9853c0 msgr2=0x55bbcb98d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:26.833 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.832+0000 7f1e0e425740 1 --2- 172.21.15.83:0/3639754753 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bbcb9853c0 0x55bbcb98d7f0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f1df8000c00 tx=0x7f1df809c320 comp rx=0 tx=0).stop 2024-04-24T08:56:26.833 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.832+0000 7f1e0e425740 1 -- 172.21.15.83:0/3639754753 shutdown_connections 2024-04-24T08:56:26.833 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.832+0000 7f1e0e425740 1 --2- 172.21.15.83:0/3639754753 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55bbcb9a0d90 0x55bbcb9a3180 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:26.833 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.832+0000 7f1e0e425740 1 --2- 172.21.15.83:0/3639754753 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bbcb98e2c0 0x55bbcb99c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:26.833 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.832+0000 7f1e0e425740 1 --2- 172.21.15.83:0/3639754753 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbcb985ce0 0x55bbcb98dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:26.833 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.832+0000 7f1e0e425740 1 --2- 172.21.15.83:0/3639754753 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bbcb9853c0 0x55bbcb98d7f0 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:26.833 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.832+0000 7f1e0e425740 1 -- 172.21.15.83:0/3639754753 >> 172.21.15.83:0/3639754753 conn(0x55bbcb9818d0 msgr2=0x55bbcb9876d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:26.833 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.832+0000 7f1dea000640 0 client.17415 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:56:26.834 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.832+0000 7f1e0e425740 1 -- 172.21.15.83:0/3639754753 shutdown_connections 2024-04-24T08:56:26.834 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.832+0000 7f1e0e425740 1 -- 172.21.15.83:0/3639754753 wait complete. 2024-04-24T08:56:26.834 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.832+0000 7f1e0e425740 20 client.17415 trim_cache size 0 max 16384 2024-04-24T08:56:26.850 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.850+0000 7f1e0e425740 1 Processor -- start 2024-04-24T08:56:26.850 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.850+0000 7f1e0e425740 1 -- start start 2024-04-24T08:56:26.850 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.850+0000 7f1e0e425740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbcb9853c0 0x55bbcb987550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:26.850 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.850+0000 7f1e0e425740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bbcb985ce0 0x55bbcb9a89f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:26.850 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.850+0000 7f1e0e425740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bbcb98e2c0 0x55bbcb9a8f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:26.850 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.850+0000 7f1e0e425740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1e041b15c0 con 0x55bbcb98e2c0 2024-04-24T08:56:26.851 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.850+0000 7f1e0e425740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1e041b3100 con 0x55bbcb9853c0 2024-04-24T08:56:26.851 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.850+0000 7f1e0e425740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1e041b1c90 con 0x55bbcb985ce0 2024-04-24T08:56:26.852 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.850+0000 7f1e09600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bbcb98e2c0 0x55bbcb9a8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:26.852 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.851+0000 7f1e09600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bbcb98e2c0 0x55bbcb9a8f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40084/0 (socket says 172.21.15.83:40084) 2024-04-24T08:56:26.852 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.851+0000 7f1e09600640 1 -- 172.21.15.83:0/1780365510 learned_addr learned my addr 172.21.15.83:0/1780365510 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:56:26.853 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.851+0000 7f1e08c00640 1 --2- 172.21.15.83:0/1780365510 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbcb9853c0 0x55bbcb987550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:26.853 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.851+0000 7f1e03e00640 1 --2- 172.21.15.83:0/1780365510 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bbcb985ce0 0x55bbcb9a89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:26.853 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.851+0000 7f1e09600640 1 -- 172.21.15.83:0/1780365510 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bbcb985ce0 msgr2=0x55bbcb9a89f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:26.853 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.851+0000 7f1e09600640 1 --2- 172.21.15.83:0/1780365510 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bbcb985ce0 0x55bbcb9a89f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:26.853 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.851+0000 7f1e09600640 1 -- 172.21.15.83:0/1780365510 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbcb9853c0 msgr2=0x55bbcb987550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:26.853 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.851+0000 7f1e09600640 1 --2- 172.21.15.83:0/1780365510 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbcb9853c0 0x55bbcb987550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:26.853 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.851+0000 7f1e09600640 1 -- 172.21.15.83:0/1780365510 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbcb6ca620 con 0x55bbcb98e2c0 2024-04-24T08:56:26.853 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.851+0000 7f1e03e00640 1 --2- 172.21.15.83:0/1780365510 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bbcb985ce0 0x55bbcb9a89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:56:26.853 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.851+0000 7f1e09600640 1 --2- 172.21.15.83:0/1780365510 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bbcb98e2c0 0x55bbcb9a8f30 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc009f20 tx=0x7f1dfc011870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd1f321861aa77e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:26.853 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.852+0000 7f1df5200640 1 -- 172.21.15.83:0/1780365510 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dfc012710 con 0x55bbcb98e2c0 2024-04-24T08:56:26.853 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.852+0000 7f1df5200640 1 -- 172.21.15.83:0/1780365510 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1dfc012d60 con 0x55bbcb98e2c0 2024-04-24T08:56:26.853 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.852+0000 7f1df5200640 1 -- 172.21.15.83:0/1780365510 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dfc01ae50 con 0x55bbcb98e2c0 2024-04-24T08:56:26.853 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.852+0000 7f1e0e425740 1 -- 172.21.15.83:0/1780365510 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bbcb98e2c0 msgr2=0x55bbcb9a8f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:26.854 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.852+0000 7f1e0e425740 1 --2- 172.21.15.83:0/1780365510 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bbcb98e2c0 0x55bbcb9a8f30 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc009f20 tx=0x7f1dfc011870 comp rx=0 tx=0).stop 2024-04-24T08:56:26.854 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.852+0000 7f1e0e425740 1 -- 172.21.15.83:0/1780365510 shutdown_connections 2024-04-24T08:56:26.854 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.852+0000 7f1e0e425740 1 --2- 172.21.15.83:0/1780365510 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bbcb98e2c0 0x55bbcb9a8f30 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:26.854 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.852+0000 7f1e0e425740 1 --2- 172.21.15.83:0/1780365510 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bbcb985ce0 0x55bbcb9a89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:26.854 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.852+0000 7f1e0e425740 1 --2- 172.21.15.83:0/1780365510 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbcb9853c0 0x55bbcb987550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:26.854 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.852+0000 7f1e0e425740 1 -- 172.21.15.83:0/1780365510 >> 172.21.15.83:0/1780365510 conn(0x55bbcb9818d0 msgr2=0x55bbcb986e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:26.854 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.853+0000 7f1e0e425740 1 -- 172.21.15.83:0/1780365510 shutdown_connections 2024-04-24T08:56:26.854 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.853+0000 7f1e0e425740 1 -- 172.21.15.83:0/1780365510 wait complete. 2024-04-24T08:56:26.857 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.853+0000 7f1e0e425740 1 -- start start 2024-04-24T08:56:26.857 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.853+0000 7f1e0e425740 1 Processor -- start 2024-04-24T08:56:26.857 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.854+0000 7f1e0e425740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbcb9853c0 0x55bbcb98a400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:26.857 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.854+0000 7f1e0e425740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bbcb985ce0 0x55bbcb98a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:26.857 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.854+0000 7f1e0e425740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bbcb98e2c0 0x55bbcb98ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:26.857 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.854+0000 7f1e0e425740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1e041b2a30 con 0x55bbcb98e2c0 2024-04-24T08:56:26.857 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.854+0000 7f1e0e425740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1e041b2360 con 0x55bbcb9853c0 2024-04-24T08:56:26.857 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.854+0000 7f1e0e425740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1e041af5a0 con 0x55bbcb985ce0 2024-04-24T08:56:26.857 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.854+0000 7f1e0e425740 10 client.0 _ll_register_callbacks cb 0x7f1df800df40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:56:26.857 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.854+0000 7f1e09600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bbcb98e2c0 0x55bbcb98ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:26.857 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.854+0000 7f1e09600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bbcb98e2c0 0x55bbcb98ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40090/0 (socket says 172.21.15.83:40090) 2024-04-24T08:56:26.857 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.854+0000 7f1e09600640 1 -- 172.21.15.83:0/590220247 learned_addr learned my addr 172.21.15.83:0/590220247 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:56:26.858 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.854+0000 7f1e08c00640 1 --2- 172.21.15.83:0/590220247 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbcb9853c0 0x55bbcb98a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:26.858 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.854+0000 7f1e09600640 1 -- 172.21.15.83:0/590220247 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bbcb985ce0 msgr2=0x55bbcb98a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:26.858 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.854+0000 7f1e09600640 1 --2- 172.21.15.83:0/590220247 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bbcb985ce0 0x55bbcb98a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:26.858 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.854+0000 7f1e09600640 1 -- 172.21.15.83:0/590220247 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbcb9853c0 msgr2=0x55bbcb98a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:26.858 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.854+0000 7f1e09600640 1 --2- 172.21.15.83:0/590220247 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbcb9853c0 0x55bbcb98a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:26.858 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.854+0000 7f1e09600640 1 -- 172.21.15.83:0/590220247 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bbcba06260 con 0x55bbcb98e2c0 2024-04-24T08:56:26.858 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.855+0000 7f1e09600640 1 --2- 172.21.15.83:0/590220247 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bbcb98e2c0 0x55bbcb98ae80 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc01a720 tx=0x7f1dfc009c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5136e382104aa37b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:26.858 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.855+0000 7f1dea000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:56:26.858 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.855+0000 7f1dea000640 1 -- 172.21.15.83:0/590220247 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dfc028020 con 0x55bbcb98e2c0 2024-04-24T08:56:26.858 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.855+0000 7f1dea000640 1 -- 172.21.15.83:0/590220247 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f1dfc01a3d0 con 0x55bbcb98e2c0 2024-04-24T08:56:26.858 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.855+0000 7f1e0e425740 1 -- 172.21.15.83:0/590220247 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1dfc012d60 con 0x55bbcb98e2c0 2024-04-24T08:56:26.858 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.855+0000 7f1dea000640 1 -- 172.21.15.83:0/590220247 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f1dfc033050 con 0x55bbcb98e2c0 2024-04-24T08:56:26.858 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.855+0000 7f1e0e425740 10 client.17421 fetch_fsmap learned FSMap version 33 2024-04-24T08:56:26.858 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.855+0000 7f1e0e425740 1 -- 172.21.15.83:0/590220247 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55bbcba073e0 con 0x55bbcb98e2c0 2024-04-24T08:56:26.859 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.855+0000 7f1dea000640 1 -- 172.21.15.83:0/590220247 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1dfc012d60 con 0x55bbcb98e2c0 2024-04-24T08:56:26.859 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.856+0000 7f1dea000640 1 -- 172.21.15.83:0/590220247 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f1dfc012760 con 0x55bbcb98e2c0 2024-04-24T08:56:26.859 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.856+0000 7f1e0e425740 10 client.17421 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T08:56:26.859 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.856+0000 7f1e0e425740 10 client.17421 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T08:56:26.859 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.856+0000 7f1e0e425740 20 client.17421 populate_metadata read hostname 'smithi083' 2024-04-24T08:56:26.859 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.856+0000 7f1e0e425740 1 --2- 172.21.15.83:0/590220247 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55bbcba07af0 0x55bbcba09f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:26.859 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.856+0000 7f1e0e425740 4 client.17421 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T08:56:26.859 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.856+0000 7f1e0e425740 1 -- 172.21.15.83:0/590220247 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bbcb8f3190 con 0x55bbcba07af0 2024-04-24T08:56:26.859 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.856+0000 7f1e08c00640 1 --2- 172.21.15.83:0/590220247 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55bbcba07af0 0x55bbcba09f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:26.859 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.858+0000 7f1e08c00640 1 --2- 172.21.15.83:0/590220247 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55bbcba07af0 0x55bbcba09f10 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e89a31c2a40caa73 server_cookie=78fad570d3bc4a01 in_seq=0 out_seq=0 2024-04-24T08:56:26.860 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.858+0000 7f1dea000640 10 client.17421 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:56:26.914 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.913+0000 7f1dea000640 1 -- 172.21.15.83:0/590220247 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bbcb8f3190 con 0x55bbcba07af0 2024-04-24T08:56:26.914 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.913+0000 7f1dea000640 10 client.17421 handle_command_reply: tid=0 2024-04-24T08:56:26.914 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.914+0000 7f1e0e425740 1 client.17421 shutdown 2024-04-24T08:56:26.915 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.914+0000 7f1e0e425740 1 -- 172.21.15.83:0/590220247 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bbcb98e2c0 msgr2=0x55bbcb98ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:26.915 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.914+0000 7f1e0e425740 1 --2- 172.21.15.83:0/590220247 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bbcb98e2c0 0x55bbcb98ae80 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc01a720 tx=0x7f1dfc009c70 comp rx=0 tx=0).stop 2024-04-24T08:56:26.915 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.914+0000 7f1e0e425740 1 -- 172.21.15.83:0/590220247 shutdown_connections 2024-04-24T08:56:26.915 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.914+0000 7f1e0e425740 1 --2- 172.21.15.83:0/590220247 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55bbcba07af0 0x55bbcba09f10 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:26.915 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.914+0000 7f1e0e425740 1 --2- 172.21.15.83:0/590220247 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bbcb98e2c0 0x55bbcb98ae80 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:26.915 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.914+0000 7f1e0e425740 1 --2- 172.21.15.83:0/590220247 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bbcb985ce0 0x55bbcb98a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:26.915 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.914+0000 7f1e0e425740 1 --2- 172.21.15.83:0/590220247 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbcb9853c0 0x55bbcb98a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:26.915 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.914+0000 7f1dea000640 0 client.17421 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:56:26.915 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.914+0000 7f1e0e425740 1 -- 172.21.15.83:0/590220247 >> 172.21.15.83:0/590220247 conn(0x55bbcb9818d0 msgr2=0x55bbcb9876d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:26.916 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.915+0000 7f1e0e425740 1 -- 172.21.15.83:0/590220247 shutdown_connections 2024-04-24T08:56:26.916 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.915+0000 7f1e0e425740 1 -- 172.21.15.83:0/590220247 wait complete. 2024-04-24T08:56:26.916 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.915+0000 7f1e0e425740 20 client.17421 trim_cache size 0 max 16384 2024-04-24T08:56:26.916 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T08:56:26.916 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T08:56:26.916 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T08:56:26.916 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T08:56:26.917 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.916+0000 7f1e0a000640 1 -- 172.21.15.83:0/3509819881 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f1dec0a3c20 msgr2=0x7f1dec0a60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:26.917 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.916+0000 7f1e0a000640 1 --2- 172.21.15.83:0/3509819881 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f1dec0a3c20 0x7f1dec0a60e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1df0035000 tx=0x7f1df002a3f0 comp rx=0 tx=0).stop 2024-04-24T08:56:26.917 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.916+0000 7f1e0a000640 1 -- 172.21.15.83:0/3509819881 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e04102fe0 msgr2=0x7f1e041a0590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:26.917 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.916+0000 7f1e0a000640 1 --2- 172.21.15.83:0/3509819881 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e04102fe0 0x7f1e041a0590 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f1df800d950 tx=0x7f1df800de10 comp rx=0 tx=0).stop 2024-04-24T08:56:26.917 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.916+0000 7f1e0a000640 1 -- 172.21.15.83:0/3509819881 shutdown_connections 2024-04-24T08:56:26.918 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.916+0000 7f1e0a000640 1 --2- 172.21.15.83:0/3509819881 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f1dec0a3c20 0x7f1dec0a60e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:26.918 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.916+0000 7f1e0a000640 1 --2- 172.21.15.83:0/3509819881 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1e0410c450 0x7f1e0419a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:26.918 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.916+0000 7f1e0a000640 1 --2- 172.21.15.83:0/3509819881 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e04103900 0x7f1e041a0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:26.918 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.916+0000 7f1e0a000640 1 --2- 172.21.15.83:0/3509819881 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e04102fe0 0x7f1e041a0590 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:26.918 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.916+0000 7f1e0a000640 1 -- 172.21.15.83:0/3509819881 >> 172.21.15.83:0/3509819881 conn(0x7f1e040fc7e0 msgr2=0x7f1e04101d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:26.918 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.917+0000 7f1e0a000640 1 -- 172.21.15.83:0/3509819881 shutdown_connections 2024-04-24T08:56:26.918 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:26.917+0000 7f1e0a000640 1 -- 172.21.15.83:0/3509819881 wait complete. 2024-04-24T08:56:27.574 INFO:tasks.workunit.client.0.smithi083.stdout: 1 22080 0.63 MB/sec execute 61 sec latency 54.903 ms 2024-04-24T08:56:27.954 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:56:27 smithi138 ceph-mon[31441]: pgmap v148: 129 pgs: 129 active+clean; 36 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 448 KiB/s wr, 54 op/s 2024-04-24T08:56:27.956 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:56:27 smithi189 ceph-mon[31791]: pgmap v148: 129 pgs: 129 active+clean; 36 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 448 KiB/s wr, 54 op/s 2024-04-24T08:56:28.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:56:27 smithi083 ceph-mon[27907]: pgmap v148: 129 pgs: 129 active+clean; 36 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 448 KiB/s wr, 54 op/s 2024-04-24T08:56:28.574 INFO:tasks.workunit.client.0.smithi083.stdout: 1 22198 0.63 MB/sec execute 62 sec latency 54.508 ms 2024-04-24T08:56:29.574 INFO:tasks.workunit.client.0.smithi083.stdout: 1 22303 0.62 MB/sec execute 63 sec latency 61.368 ms 2024-04-24T08:56:30.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:56:29 smithi083 ceph-mon[27907]: pgmap v149: 129 pgs: 129 active+clean; 37 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 543 KiB/s wr, 78 op/s 2024-04-24T08:56:30.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:56:29 smithi138 ceph-mon[31441]: pgmap v149: 129 pgs: 129 active+clean; 37 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 543 KiB/s wr, 78 op/s 2024-04-24T08:56:30.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:56:29 smithi189 ceph-mon[31791]: pgmap v149: 129 pgs: 129 active+clean; 37 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 543 KiB/s wr, 78 op/s 2024-04-24T08:56:30.574 INFO:tasks.workunit.client.0.smithi083.stdout: 1 22421 0.63 MB/sec execute 64 sec latency 56.353 ms 2024-04-24T08:56:31.574 INFO:tasks.workunit.client.0.smithi083.stdout: 1 22531 0.62 MB/sec execute 65 sec latency 55.192 ms 2024-04-24T08:56:32.010 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:56:31 smithi189 ceph-mon[31791]: pgmap v150: 129 pgs: 129 active+clean; 37 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 269 KiB/s wr, 47 op/s 2024-04-24T08:56:32.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:56:31 smithi083 ceph-mon[27907]: pgmap v150: 129 pgs: 129 active+clean; 37 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 269 KiB/s wr, 47 op/s 2024-04-24T08:56:32.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:56:31 smithi138 ceph-mon[31441]: pgmap v150: 129 pgs: 129 active+clean; 37 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 269 KiB/s wr, 47 op/s 2024-04-24T08:56:32.575 INFO:tasks.workunit.client.0.smithi083.stdout: 1 22707 0.66 MB/sec execute 66 sec latency 74.560 ms 2024-04-24T08:56:33.575 INFO:tasks.workunit.client.0.smithi083.stdout: 1 22805 0.65 MB/sec execute 67 sec latency 61.655 ms 2024-04-24T08:56:34.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:56:33 smithi083 ceph-mon[27907]: pgmap v151: 129 pgs: 129 active+clean; 35 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 441 KiB/s wr, 75 op/s 2024-04-24T08:56:34.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:56:33 smithi138 ceph-mon[31441]: pgmap v151: 129 pgs: 129 active+clean; 35 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 441 KiB/s wr, 75 op/s 2024-04-24T08:56:34.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:56:33 smithi189 ceph-mon[31791]: pgmap v151: 129 pgs: 129 active+clean; 35 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 441 KiB/s wr, 75 op/s 2024-04-24T08:56:34.574 INFO:tasks.workunit.client.0.smithi083.stdout: 1 22919 0.64 MB/sec execute 68 sec latency 47.172 ms 2024-04-24T08:56:35.575 INFO:tasks.workunit.client.0.smithi083.stdout: 1 23037 0.63 MB/sec execute 69 sec latency 48.292 ms 2024-04-24T08:56:36.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:56:35 smithi083 ceph-mon[27907]: pgmap v152: 129 pgs: 129 active+clean; 35 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 266 KiB/s wr, 51 op/s 2024-04-24T08:56:36.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:56:35 smithi138 ceph-mon[31441]: pgmap v152: 129 pgs: 129 active+clean; 35 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 266 KiB/s wr, 51 op/s 2024-04-24T08:56:36.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:56:35 smithi189 ceph-mon[31791]: pgmap v152: 129 pgs: 129 active+clean; 35 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 266 KiB/s wr, 51 op/s 2024-04-24T08:56:36.575 INFO:tasks.workunit.client.0.smithi083.stdout: 1 23188 0.62 MB/sec execute 70 sec latency 51.251 ms 2024-04-24T08:56:37.575 INFO:tasks.workunit.client.0.smithi083.stdout: 1 23327 0.62 MB/sec execute 71 sec latency 58.703 ms 2024-04-24T08:56:38.008 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:56:37 smithi138 ceph-mon[31441]: pgmap v153: 129 pgs: 129 active+clean; 35 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 266 KiB/s wr, 51 op/s 2024-04-24T08:56:38.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:56:37 smithi083 ceph-mon[27907]: pgmap v153: 129 pgs: 129 active+clean; 35 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 266 KiB/s wr, 51 op/s 2024-04-24T08:56:38.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:56:37 smithi189 ceph-mon[31791]: pgmap v153: 129 pgs: 129 active+clean; 35 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 266 KiB/s wr, 51 op/s 2024-04-24T08:56:38.575 INFO:tasks.workunit.client.0.smithi083.stdout: 1 23465 0.61 MB/sec execute 72 sec latency 61.597 ms 2024-04-24T08:56:39.575 INFO:tasks.workunit.client.0.smithi083.stdout: 1 23627 0.62 MB/sec execute 73 sec latency 48.650 ms 2024-04-24T08:56:40.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:56:39 smithi083 ceph-mon[27907]: pgmap v154: 129 pgs: 129 active+clean; 35 MiB data, 550 MiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 319 KiB/s wr, 71 op/s 2024-04-24T08:56:40.071 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.072+0000 7f1133e00640 1 -- 172.21.15.83:0/3482248467 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1134102090 msgr2=0x7f1134102470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:40.071 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.072+0000 7f1133e00640 1 --2- 172.21.15.83:0/3482248467 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1134102090 0x7f1134102470 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f112c009900 tx=0x7f112c02a3f0 comp rx=0 tx=0).stop 2024-04-24T08:56:40.071 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.072+0000 7f1133e00640 1 -- 172.21.15.83:0/3482248467 shutdown_connections 2024-04-24T08:56:40.072 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.072+0000 7f1133e00640 1 --2- 172.21.15.83:0/3482248467 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1134107160 0x7f113410ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:40.072 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.072+0000 7f1133e00640 1 --2- 172.21.15.83:0/3482248467 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1134102a40 0x7f1134106a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:40.072 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.072+0000 7f1133e00640 1 --2- 172.21.15.83:0/3482248467 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1134102090 0x7f1134102470 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:40.072 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.072+0000 7f1133e00640 1 -- 172.21.15.83:0/3482248467 >> 172.21.15.83:0/3482248467 conn(0x7f11340fd950 msgr2=0x7f11340ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:40.072 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.072+0000 7f1133e00640 1 -- 172.21.15.83:0/3482248467 shutdown_connections 2024-04-24T08:56:40.072 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.072+0000 7f1133e00640 1 -- 172.21.15.83:0/3482248467 wait complete. 2024-04-24T08:56:40.074 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.073+0000 7f1133e00640 1 Processor -- start 2024-04-24T08:56:40.075 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.073+0000 7f1133e00640 1 -- start start 2024-04-24T08:56:40.075 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.074+0000 7f1133e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1134102090 0x7f11341a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:40.075 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.074+0000 7f1133e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1134107160 0x7f11341a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:40.075 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.074+0000 7f1133e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f113419a720 0x7f113419aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:40.075 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.074+0000 7f1133e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1134112120 con 0x7f113419a720 2024-04-24T08:56:40.075 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.074+0000 7f1133e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1134111fa0 con 0x7f1134102090 2024-04-24T08:56:40.075 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.074+0000 7f1133e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f11341122a0 con 0x7f1134107160 2024-04-24T08:56:40.075 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.074+0000 7f1133400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f113419a720 0x7f113419aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:40.075 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.074+0000 7f1133400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f113419a720 0x7f113419aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53296/0 (socket says 172.21.15.83:53296) 2024-04-24T08:56:40.075 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.074+0000 7f1133400640 1 -- 172.21.15.83:0/3444305146 learned_addr learned my addr 172.21.15.83:0/3444305146 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:56:40.075 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.074+0000 7f1132a00640 1 --2- 172.21.15.83:0/3444305146 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1134102090 0x7f11341a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:40.075 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.074+0000 7f1132000640 1 --2- 172.21.15.83:0/3444305146 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1134107160 0x7f11341a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:40.075 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.074+0000 7f1133400640 1 -- 172.21.15.83:0/3444305146 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1134107160 msgr2=0x7f11341a0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:40.075 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.074+0000 7f1133400640 1 --2- 172.21.15.83:0/3444305146 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1134107160 0x7f11341a0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:40.076 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.075+0000 7f1133400640 1 -- 172.21.15.83:0/3444305146 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1134102090 msgr2=0x7f11341a0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:40.076 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.075+0000 7f1133400640 1 --2- 172.21.15.83:0/3444305146 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1134102090 0x7f11341a0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:40.076 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.075+0000 7f1133400640 1 -- 172.21.15.83:0/3444305146 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f113419b320 con 0x7f113419a720 2024-04-24T08:56:40.076 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.075+0000 7f1132a00640 1 --2- 172.21.15.83:0/3444305146 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1134102090 0x7f11341a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:56:40.076 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.075+0000 7f1133400640 1 --2- 172.21.15.83:0/3444305146 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f113419a720 0x7f113419aba0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f1124009fd0 tx=0x7f112400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:40.076 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.075+0000 7f1138a00640 1 -- 172.21.15.83:0/3444305146 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1124011830 con 0x7f113419a720 2024-04-24T08:56:40.076 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.075+0000 7f1133e00640 1 -- 172.21.15.83:0/3444305146 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11341a7400 con 0x7f113419a720 2024-04-24T08:56:40.076 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.075+0000 7f1138a00640 1 -- 172.21.15.83:0/3444305146 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1124016070 con 0x7f113419a720 2024-04-24T08:56:40.076 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.075+0000 7f1138a00640 1 -- 172.21.15.83:0/3444305146 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1124010de0 con 0x7f113419a720 2024-04-24T08:56:40.076 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.075+0000 7f1133e00640 1 -- 172.21.15.83:0/3444305146 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11341a77b0 con 0x7f113419a720 2024-04-24T08:56:40.077 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.078+0000 7f113cbec740 1 Processor -- start 2024-04-24T08:56:40.079 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.079+0000 7f1138a00640 1 -- 172.21.15.83:0/3444305146 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f11240119d0 con 0x7f113419a720 2024-04-24T08:56:40.079 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.079+0000 7f113cbec740 1 -- start start 2024-04-24T08:56:40.080 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.079+0000 7f113cbec740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c423303c0 0x557c423307a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:40.080 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.079+0000 7f113cbec740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c42330ce0 0x557c4233b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:40.080 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.079+0000 7f113cbec740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557c4233b5a0 0x557c4233d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:40.080 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.079+0000 7f113cbec740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x557c42296900 con 0x557c4233b5a0 2024-04-24T08:56:40.080 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.079+0000 7f113cbec740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x557c4223cf20 con 0x557c42330ce0 2024-04-24T08:56:40.080 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.079+0000 7f113cbec740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x557c422928c0 con 0x557c423303c0 2024-04-24T08:56:40.080 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.080+0000 7f1132000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557c4233b5a0 0x557c4233d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:40.080 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.080+0000 7f1132000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557c4233b5a0 0x557c4233d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53310/0 (socket says 172.21.15.83:53310) 2024-04-24T08:56:40.080 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.080+0000 7f1132000640 1 -- 172.21.15.83:0/3043464881 learned_addr learned my addr 172.21.15.83:0/3043464881 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:56:40.080 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.080+0000 7f1132a00640 1 --2- 172.21.15.83:0/3043464881 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c42330ce0 0x557c4233b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:40.080 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.080+0000 7f1132000640 1 --2- 172.21.15.83:0/3043464881 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c423303c0 0x557c423307a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:40.081 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.080+0000 7f1132000640 1 -- 172.21.15.83:0/3043464881 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c423303c0 msgr2=0x557c423307a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:40.081 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.080+0000 7f1132000640 1 --2- 172.21.15.83:0/3043464881 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c423303c0 0x557c423307a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:40.081 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.080+0000 7f1132000640 1 -- 172.21.15.83:0/3043464881 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c42330ce0 msgr2=0x557c4233b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:40.081 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.080+0000 7f1132000640 1 --2- 172.21.15.83:0/3043464881 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c42330ce0 0x557c4233b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:40.081 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.080+0000 7f1132000640 1 -- 172.21.15.83:0/3043464881 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c42261e10 con 0x557c4233b5a0 2024-04-24T08:56:40.081 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.080+0000 7f1132a00640 1 --2- 172.21.15.83:0/3043464881 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c42330ce0 0x557c4233b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:56:40.081 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.081+0000 7f1132000640 1 --2- 172.21.15.83:0/3043464881 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557c4233b5a0 0x557c4233d990 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f111c007fd0 tx=0x7f111c007890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7fe35cb45067fd7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:40.086 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.081+0000 7f1138a00640 1 --2- 172.21.15.83:0/3444305146 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f11180a3d10 0x7f11180a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:40.086 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.081+0000 7f1123e00640 1 -- 172.21.15.83:0/3043464881 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f111c018030 con 0x557c4233b5a0 2024-04-24T08:56:40.086 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.081+0000 7f1123e00640 1 -- 172.21.15.83:0/3043464881 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f111c010e00 con 0x557c4233b5a0 2024-04-24T08:56:40.086 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.081+0000 7f1123e00640 1 -- 172.21.15.83:0/3043464881 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f111c00ec70 con 0x557c4233b5a0 2024-04-24T08:56:40.086 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.081+0000 7f1138a00640 1 -- 172.21.15.83:0/3444305146 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f11240d0330 con 0x7f113419a720 2024-04-24T08:56:40.086 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.081+0000 7f1132a00640 1 --2- 172.21.15.83:0/3444305146 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f11180a3d10 0x7f11180a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:40.086 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.082+0000 7f113cbec740 1 -- 172.21.15.83:0/3043464881 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557c4233b5a0 msgr2=0x557c4233d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:40.086 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.082+0000 7f113cbec740 1 --2- 172.21.15.83:0/3043464881 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557c4233b5a0 0x557c4233d990 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f111c007fd0 tx=0x7f111c007890 comp rx=0 tx=0).stop 2024-04-24T08:56:40.087 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.082+0000 7f113cbec740 1 -- 172.21.15.83:0/3043464881 shutdown_connections 2024-04-24T08:56:40.087 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.082+0000 7f113cbec740 1 --2- 172.21.15.83:0/3043464881 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557c4233b5a0 0x557c4233d990 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:40.087 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.082+0000 7f113cbec740 1 --2- 172.21.15.83:0/3043464881 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c42330ce0 0x557c4233b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:40.087 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.082+0000 7f113cbec740 1 --2- 172.21.15.83:0/3043464881 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c423303c0 0x557c423307a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:40.087 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.082+0000 7f113cbec740 1 -- 172.21.15.83:0/3043464881 >> 172.21.15.83:0/3043464881 conn(0x557c4232c8d0 msgr2=0x557c4232ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:40.087 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.082+0000 7f113cbec740 1 -- 172.21.15.83:0/3043464881 shutdown_connections 2024-04-24T08:56:40.087 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.082+0000 7f113cbec740 1 -- 172.21.15.83:0/3043464881 wait complete. 2024-04-24T08:56:40.087 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.082+0000 7f1132a00640 1 --2- 172.21.15.83:0/3444305146 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f11180a3d10 0x7f11180a61d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f112c031440 tx=0x7f112c005a20 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:40.087 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.082+0000 7f113cbec740 1 -- start start 2024-04-24T08:56:40.087 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.083+0000 7f113cbec740 1 Processor -- start 2024-04-24T08:56:40.087 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.083+0000 7f113cbec740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c423303c0 0x557c42338620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:40.087 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.083+0000 7f113cbec740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c42330ce0 0x557c42338b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:40.087 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.083+0000 7f113cbec740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557c4233b5a0 0x557c423390a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:40.087 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.083+0000 7f113cbec740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x557c4233e0e0 con 0x557c4233b5a0 2024-04-24T08:56:40.088 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.083+0000 7f113cbec740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x557c422502c0 con 0x557c42330ce0 2024-04-24T08:56:40.088 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.083+0000 7f113cbec740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x557c4233e260 con 0x557c423303c0 2024-04-24T08:56:40.088 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.083+0000 7f113cbec740 10 client.0 _ll_register_callbacks cb 0x557c41fa8a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:56:40.088 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.083+0000 7f1133400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557c4233b5a0 0x557c423390a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:40.088 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.084+0000 7f1133400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557c4233b5a0 0x557c423390a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53320/0 (socket says 172.21.15.83:53320) 2024-04-24T08:56:40.088 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.084+0000 7f1133400640 1 -- 172.21.15.83:0/4096514704 learned_addr learned my addr 172.21.15.83:0/4096514704 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:56:40.088 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.084+0000 7f1132a00640 1 --2- 172.21.15.83:0/4096514704 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c423303c0 0x557c42338620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:40.088 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.084+0000 7f1132000640 1 --2- 172.21.15.83:0/4096514704 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c42330ce0 0x557c42338b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:40.088 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.084+0000 7f1133400640 1 -- 172.21.15.83:0/4096514704 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c423303c0 msgr2=0x557c42338620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:40.088 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.084+0000 7f1133400640 1 --2- 172.21.15.83:0/4096514704 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c423303c0 0x557c42338620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:40.088 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.084+0000 7f1133400640 1 -- 172.21.15.83:0/4096514704 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c42330ce0 msgr2=0x557c42338b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:40.088 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.084+0000 7f1133400640 1 --2- 172.21.15.83:0/4096514704 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c42330ce0 0x557c42338b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:40.088 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.084+0000 7f1133400640 1 -- 172.21.15.83:0/4096514704 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c42260fd0 con 0x557c4233b5a0 2024-04-24T08:56:40.088 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.084+0000 7f1133400640 1 --2- 172.21.15.83:0/4096514704 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557c4233b5a0 0x557c423390a0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f11240a46f0 tx=0x7f112409ba00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9d11ec0b933792d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:40.089 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.084+0000 7f1120c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:56:40.089 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.085+0000 7f1120c00640 1 -- 172.21.15.83:0/4096514704 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11240bc050 con 0x557c4233b5a0 2024-04-24T08:56:40.089 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.085+0000 7f1120c00640 1 -- 172.21.15.83:0/4096514704 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f1124098620 con 0x557c4233b5a0 2024-04-24T08:56:40.089 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.085+0000 7f113cbec740 1 -- 172.21.15.83:0/4096514704 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f111c010e00 con 0x557c4233b5a0 2024-04-24T08:56:40.089 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.085+0000 7f1120c00640 1 -- 172.21.15.83:0/4096514704 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f11240a0360 con 0x557c4233b5a0 2024-04-24T08:56:40.089 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.085+0000 7f113cbec740 10 client.17433 fetch_fsmap learned FSMap version 33 2024-04-24T08:56:40.089 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.085+0000 7f113cbec740 1 -- 172.21.15.83:0/4096514704 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x557c41fa9210 con 0x557c4233b5a0 2024-04-24T08:56:40.089 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.085+0000 7f1120c00640 1 -- 172.21.15.83:0/4096514704 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f111c010e00 con 0x557c4233b5a0 2024-04-24T08:56:40.089 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.086+0000 7f1120c00640 1 -- 172.21.15.83:0/4096514704 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f1124097d00 con 0x557c4233b5a0 2024-04-24T08:56:40.089 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.086+0000 7f113cbec740 10 client.17433 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T08:56:40.089 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.086+0000 7f113cbec740 10 client.17433 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T08:56:40.089 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.086+0000 7f113cbec740 20 client.17433 populate_metadata read hostname 'smithi083' 2024-04-24T08:56:40.090 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.086+0000 7f113cbec740 1 --2- 172.21.15.83:0/4096514704 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x557c4234b800 0x557c4234dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:40.090 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.086+0000 7f113cbec740 4 client.17433 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:56:40.090 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.086+0000 7f113cbec740 1 -- 172.21.15.83:0/4096514704 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557c422a4320 con 0x557c4234b800 2024-04-24T08:56:40.090 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.087+0000 7f1132a00640 1 --2- 172.21.15.83:0/4096514704 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x557c4234b800 0x557c4234dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:40.091 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.088+0000 7f1132a00640 1 --2- 172.21.15.83:0/4096514704 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x557c4234b800 0x557c4234dbf0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51dd98950f23b4a4 server_cookie=d502c2f37c925fdf in_seq=0 out_seq=0 2024-04-24T08:56:40.091 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.088+0000 7f1120c00640 10 client.17433 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:56:40.091 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.091+0000 7f1120c00640 1 -- 172.21.15.83:0/4096514704 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557c422a4320 con 0x557c4234b800 2024-04-24T08:56:40.091 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.091+0000 7f1120c00640 10 client.17433 handle_command_reply: tid=0 2024-04-24T08:56:40.091 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.091+0000 7f113cbec740 1 client.17433 shutdown 2024-04-24T08:56:40.091 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.091+0000 7f113cbec740 1 -- 172.21.15.83:0/4096514704 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557c4233b5a0 msgr2=0x557c423390a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:40.091 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.091+0000 7f113cbec740 1 --2- 172.21.15.83:0/4096514704 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557c4233b5a0 0x557c423390a0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f11240a46f0 tx=0x7f112409ba00 comp rx=0 tx=0).stop 2024-04-24T08:56:40.092 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.091+0000 7f113cbec740 1 -- 172.21.15.83:0/4096514704 shutdown_connections 2024-04-24T08:56:40.092 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.091+0000 7f113cbec740 1 --2- 172.21.15.83:0/4096514704 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x557c4234b800 0x557c4234dbf0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:40.092 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.091+0000 7f113cbec740 1 --2- 172.21.15.83:0/4096514704 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557c4233b5a0 0x557c423390a0 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:40.092 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.091+0000 7f113cbec740 1 --2- 172.21.15.83:0/4096514704 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c42330ce0 0x557c42338b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:40.092 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.091+0000 7f113cbec740 1 --2- 172.21.15.83:0/4096514704 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c423303c0 0x557c42338620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:40.092 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.091+0000 7f1120c00640 0 client.17433 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:56:40.092 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.091+0000 7f113cbec740 1 -- 172.21.15.83:0/4096514704 >> 172.21.15.83:0/4096514704 conn(0x557c4232c8d0 msgr2=0x557c42332600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:40.092 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.091+0000 7f113cbec740 1 -- 172.21.15.83:0/4096514704 shutdown_connections 2024-04-24T08:56:40.092 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.091+0000 7f113cbec740 1 -- 172.21.15.83:0/4096514704 wait complete. 2024-04-24T08:56:40.092 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.091+0000 7f113cbec740 20 client.17433 trim_cache size 0 max 16384 2024-04-24T08:56:40.110 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.111+0000 7f113cbec740 1 Processor -- start 2024-04-24T08:56:40.110 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.111+0000 7f113cbec740 1 -- start start 2024-04-24T08:56:40.110 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.111+0000 7f113cbec740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c423303c0 0x557c423470e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:40.111 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.111+0000 7f113cbec740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c42330ce0 0x557c42332d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:40.111 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.111+0000 7f113cbec740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557c4233b5a0 0x557c4232b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:40.111 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.111+0000 7f113cbec740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f11341b1480 con 0x557c4233b5a0 2024-04-24T08:56:40.111 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.111+0000 7f113cbec740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f11341b2fc0 con 0x557c42330ce0 2024-04-24T08:56:40.111 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.111+0000 7f113cbec740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f11341b1b50 con 0x557c423303c0 2024-04-24T08:56:40.112 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.112+0000 7f1133400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557c4233b5a0 0x557c4232b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:40.112 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.112+0000 7f1133400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557c4233b5a0 0x557c4232b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53324/0 (socket says 172.21.15.83:53324) 2024-04-24T08:56:40.112 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.112+0000 7f1133400640 1 -- 172.21.15.83:0/984688552 learned_addr learned my addr 172.21.15.83:0/984688552 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:56:40.112 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.112+0000 7f1132a00640 1 --2- 172.21.15.83:0/984688552 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c423303c0 0x557c423470e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:40.112 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.112+0000 7f1132000640 1 --2- 172.21.15.83:0/984688552 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c42330ce0 0x557c42332d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:40.112 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.112+0000 7f1133400640 1 -- 172.21.15.83:0/984688552 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c423303c0 msgr2=0x557c423470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:40.112 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.112+0000 7f1133400640 1 --2- 172.21.15.83:0/984688552 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c423303c0 0x557c423470e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:40.113 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.112+0000 7f1133400640 1 -- 172.21.15.83:0/984688552 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c42330ce0 msgr2=0x557c42332d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:40.113 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.112+0000 7f1133400640 1 --2- 172.21.15.83:0/984688552 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c42330ce0 0x557c42332d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:40.113 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.112+0000 7f1133400640 1 -- 172.21.15.83:0/984688552 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c42075620 con 0x557c4233b5a0 2024-04-24T08:56:40.113 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.113+0000 7f1133400640 1 --2- 172.21.15.83:0/984688552 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557c4233b5a0 0x557c4232b910 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f1124010450 tx=0x7f1124009200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57b987c04c1a1fba server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:40.113 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.113+0000 7f1123400640 1 -- 172.21.15.83:0/984688552 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11240bc070 con 0x557c4233b5a0 2024-04-24T08:56:40.113 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.113+0000 7f1123400640 1 -- 172.21.15.83:0/984688552 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f112409c6b0 con 0x557c4233b5a0 2024-04-24T08:56:40.113 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.113+0000 7f1123400640 1 -- 172.21.15.83:0/984688552 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11240cdaa0 con 0x557c4233b5a0 2024-04-24T08:56:40.114 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.114+0000 7f113cbec740 1 -- 172.21.15.83:0/984688552 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557c4233b5a0 msgr2=0x557c4232b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:40.115 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.114+0000 7f113cbec740 1 --2- 172.21.15.83:0/984688552 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557c4233b5a0 0x557c4232b910 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f1124010450 tx=0x7f1124009200 comp rx=0 tx=0).stop 2024-04-24T08:56:40.115 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.114+0000 7f113cbec740 1 -- 172.21.15.83:0/984688552 shutdown_connections 2024-04-24T08:56:40.115 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.114+0000 7f113cbec740 1 --2- 172.21.15.83:0/984688552 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557c4233b5a0 0x557c4232b910 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:40.115 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.114+0000 7f113cbec740 1 --2- 172.21.15.83:0/984688552 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c42330ce0 0x557c42332d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:40.115 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.114+0000 7f113cbec740 1 --2- 172.21.15.83:0/984688552 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c423303c0 0x557c423470e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:40.115 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.114+0000 7f113cbec740 1 -- 172.21.15.83:0/984688552 >> 172.21.15.83:0/984688552 conn(0x557c4232c8d0 msgr2=0x557c4232b200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:40.115 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.114+0000 7f113cbec740 1 -- 172.21.15.83:0/984688552 shutdown_connections 2024-04-24T08:56:40.115 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.114+0000 7f113cbec740 1 -- 172.21.15.83:0/984688552 wait complete. 2024-04-24T08:56:40.115 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.114+0000 7f113cbec740 1 -- start start 2024-04-24T08:56:40.115 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.115+0000 7f113cbec740 1 Processor -- start 2024-04-24T08:56:40.115 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.115+0000 7f113cbec740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c423303c0 0x557c4233f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:40.115 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.115+0000 7f113cbec740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c42330ce0 0x557c4233f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:40.115 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.115+0000 7f113cbec740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557c4233b5a0 0x557c4233fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:40.116 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.115+0000 7f113cbec740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f11341b28f0 con 0x557c4233b5a0 2024-04-24T08:56:40.116 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.115+0000 7f113cbec740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f11341b2220 con 0x557c423303c0 2024-04-24T08:56:40.116 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.115+0000 7f113cbec740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f11341af460 con 0x557c42330ce0 2024-04-24T08:56:40.116 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.115+0000 7f113cbec740 10 client.0 _ll_register_callbacks cb 0x7f1124010380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:56:40.116 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.116+0000 7f1133400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557c4233b5a0 0x557c4233fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:40.116 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.116+0000 7f1133400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557c4233b5a0 0x557c4233fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53336/0 (socket says 172.21.15.83:53336) 2024-04-24T08:56:40.116 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.116+0000 7f1133400640 1 -- 172.21.15.83:0/3210261844 learned_addr learned my addr 172.21.15.83:0/3210261844 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:56:40.118 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.116+0000 7f1132a00640 1 --2- 172.21.15.83:0/3210261844 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c423303c0 0x557c4233f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:40.118 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.116+0000 7f1132000640 1 --2- 172.21.15.83:0/3210261844 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c42330ce0 0x557c4233f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:40.118 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.116+0000 7f1133400640 1 -- 172.21.15.83:0/3210261844 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c42330ce0 msgr2=0x557c4233f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:40.118 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.116+0000 7f1133400640 1 --2- 172.21.15.83:0/3210261844 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c42330ce0 0x557c4233f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:40.118 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.116+0000 7f1133400640 1 -- 172.21.15.83:0/3210261844 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c423303c0 msgr2=0x557c4233f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:40.118 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.116+0000 7f1133400640 1 --2- 172.21.15.83:0/3210261844 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c423303c0 0x557c4233f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:40.118 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.116+0000 7f1133400640 1 -- 172.21.15.83:0/3210261844 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c423b9830 con 0x557c4233b5a0 2024-04-24T08:56:40.119 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.116+0000 7f1132000640 1 --2- 172.21.15.83:0/3210261844 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c42330ce0 0x557c4233f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:56:40.119 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.116+0000 7f1132a00640 1 --2- 172.21.15.83:0/3210261844 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c423303c0 0x557c4233f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:56:40.119 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.116+0000 7f1133400640 1 --2- 172.21.15.83:0/3210261844 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557c4233b5a0 0x557c4233fdc0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f112409a260 tx=0x7f1124098850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e51f9499642fc16 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:40.119 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.116+0000 7f1120c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:56:40.119 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.117+0000 7f1120c00640 1 -- 172.21.15.83:0/3210261844 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11240bc040 con 0x557c4233b5a0 2024-04-24T08:56:40.119 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.117+0000 7f1120c00640 1 -- 172.21.15.83:0/3210261844 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f11240cd840 con 0x557c4233b5a0 2024-04-24T08:56:40.119 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.117+0000 7f113cbec740 1 -- 172.21.15.83:0/3210261844 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f112409c6b0 con 0x557c4233b5a0 2024-04-24T08:56:40.119 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.117+0000 7f1120c00640 1 -- 172.21.15.83:0/3210261844 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f1124097f10 con 0x557c4233b5a0 2024-04-24T08:56:40.119 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.117+0000 7f113cbec740 10 client.17439 fetch_fsmap learned FSMap version 33 2024-04-24T08:56:40.119 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.117+0000 7f113cbec740 1 -- 172.21.15.83:0/3210261844 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x557c423b9830 con 0x557c4233b5a0 2024-04-24T08:56:40.119 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.117+0000 7f1120c00640 1 -- 172.21.15.83:0/3210261844 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f112409c6b0 con 0x557c4233b5a0 2024-04-24T08:56:40.119 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.117+0000 7f1120c00640 1 -- 172.21.15.83:0/3210261844 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f1124096b20 con 0x557c4233b5a0 2024-04-24T08:56:40.119 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.117+0000 7f113cbec740 10 client.17439 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T08:56:40.120 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.117+0000 7f113cbec740 10 client.17439 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T08:56:40.120 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.117+0000 7f113cbec740 20 client.17439 populate_metadata read hostname 'smithi083' 2024-04-24T08:56:40.120 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.118+0000 7f113cbec740 1 --2- 172.21.15.83:0/3210261844 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x557c423bada0 0x557c423bd190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:40.120 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.118+0000 7f113cbec740 4 client.17439 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T08:56:40.120 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.118+0000 7f113cbec740 1 -- 172.21.15.83:0/3210261844 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557c4229e190 con 0x557c423bada0 2024-04-24T08:56:40.120 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.118+0000 7f1132a00640 1 --2- 172.21.15.83:0/3210261844 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x557c423bada0 0x557c423bd190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:40.120 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.119+0000 7f1132a00640 1 --2- 172.21.15.83:0/3210261844 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x557c423bada0 0x557c423bd190 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=792877c9bb00c2ea server_cookie=1aed7676cc4380ac in_seq=0 out_seq=0 2024-04-24T08:56:40.120 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.119+0000 7f1120c00640 10 client.17439 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:56:40.177 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.178+0000 7f1120c00640 1 -- 172.21.15.83:0/3210261844 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557c4229e190 con 0x557c423bada0 2024-04-24T08:56:40.177 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.178+0000 7f1120c00640 10 client.17439 handle_command_reply: tid=0 2024-04-24T08:56:40.178 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.178+0000 7f113cbec740 1 client.17439 shutdown 2024-04-24T08:56:40.179 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.179+0000 7f113cbec740 1 -- 172.21.15.83:0/3210261844 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557c4233b5a0 msgr2=0x557c4233fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:40.179 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.179+0000 7f113cbec740 1 --2- 172.21.15.83:0/3210261844 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557c4233b5a0 0x557c4233fdc0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f112409a260 tx=0x7f1124098850 comp rx=0 tx=0).stop 2024-04-24T08:56:40.179 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.179+0000 7f113cbec740 1 -- 172.21.15.83:0/3210261844 shutdown_connections 2024-04-24T08:56:40.179 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.179+0000 7f113cbec740 1 --2- 172.21.15.83:0/3210261844 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x557c423bada0 0x557c423bd190 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:40.179 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.179+0000 7f113cbec740 1 --2- 172.21.15.83:0/3210261844 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557c4233b5a0 0x557c4233fdc0 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:40.179 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.179+0000 7f113cbec740 1 --2- 172.21.15.83:0/3210261844 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c42330ce0 0x557c4233f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:40.179 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.179+0000 7f113cbec740 1 --2- 172.21.15.83:0/3210261844 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c423303c0 0x557c4233f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:40.179 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.179+0000 7f113cbec740 1 -- 172.21.15.83:0/3210261844 >> 172.21.15.83:0/3210261844 conn(0x557c4232c8d0 msgr2=0x557c4232ba90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:40.179 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.179+0000 7f1120c00640 0 client.17439 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:56:40.179 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.179+0000 7f113cbec740 1 -- 172.21.15.83:0/3210261844 shutdown_connections 2024-04-24T08:56:40.180 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.179+0000 7f113cbec740 1 -- 172.21.15.83:0/3210261844 wait complete. 2024-04-24T08:56:40.180 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.180+0000 7f113cbec740 20 client.17439 trim_cache size 0 max 16384 2024-04-24T08:56:40.180 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T08:56:40.180 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T08:56:40.180 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T08:56:40.180 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T08:56:40.181 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.181+0000 7f1133e00640 1 -- 172.21.15.83:0/3444305146 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f11180a3d10 msgr2=0x7f11180a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:40.181 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.181+0000 7f1133e00640 1 --2- 172.21.15.83:0/3444305146 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f11180a3d10 0x7f11180a61d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f112c031440 tx=0x7f112c005a20 comp rx=0 tx=0).stop 2024-04-24T08:56:40.181 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.181+0000 7f1133e00640 1 -- 172.21.15.83:0/3444305146 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f113419a720 msgr2=0x7f113419aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:40.181 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.181+0000 7f1133e00640 1 --2- 172.21.15.83:0/3444305146 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f113419a720 0x7f113419aba0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f1124009fd0 tx=0x7f112400b8e0 comp rx=0 tx=0).stop 2024-04-24T08:56:40.181 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.181+0000 7f1133e00640 1 -- 172.21.15.83:0/3444305146 shutdown_connections 2024-04-24T08:56:40.182 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.181+0000 7f1133e00640 1 --2- 172.21.15.83:0/3444305146 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f11180a3d10 0x7f11180a61d0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:40.182 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.181+0000 7f1133e00640 1 --2- 172.21.15.83:0/3444305146 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f113419a720 0x7f113419aba0 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:40.182 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.181+0000 7f1133e00640 1 --2- 172.21.15.83:0/3444305146 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1134107160 0x7f11341a0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:40.182 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.181+0000 7f1133e00640 1 --2- 172.21.15.83:0/3444305146 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1134102090 0x7f11341a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:40.182 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.181+0000 7f1133e00640 1 -- 172.21.15.83:0/3444305146 >> 172.21.15.83:0/3444305146 conn(0x7f11340fd950 msgr2=0x7f11340fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:40.182 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.182+0000 7f1133e00640 1 -- 172.21.15.83:0/3444305146 shutdown_connections 2024-04-24T08:56:40.182 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:40.182+0000 7f1133e00640 1 -- 172.21.15.83:0/3444305146 wait complete. 2024-04-24T08:56:40.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:56:39 smithi138 ceph-mon[31441]: pgmap v154: 129 pgs: 129 active+clean; 35 MiB data, 550 MiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 319 KiB/s wr, 71 op/s 2024-04-24T08:56:40.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:56:39 smithi189 ceph-mon[31791]: pgmap v154: 129 pgs: 129 active+clean; 35 MiB data, 550 MiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 319 KiB/s wr, 71 op/s 2024-04-24T08:56:40.575 INFO:tasks.workunit.client.0.smithi083.stdout: 1 23715 0.61 MB/sec execute 74 sec latency 56.514 ms 2024-04-24T08:56:41.575 INFO:tasks.workunit.client.0.smithi083.stdout: 1 23796 0.61 MB/sec execute 75 sec latency 48.706 ms 2024-04-24T08:56:42.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:56:41 smithi083 ceph-mon[27907]: pgmap v155: 129 pgs: 129 active+clean; 35 MiB data, 550 MiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 225 KiB/s wr, 48 op/s 2024-04-24T08:56:42.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:56:41 smithi138 ceph-mon[31441]: pgmap v155: 129 pgs: 129 active+clean; 35 MiB data, 550 MiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 225 KiB/s wr, 48 op/s 2024-04-24T08:56:42.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:56:41 smithi189 ceph-mon[31791]: pgmap v155: 129 pgs: 129 active+clean; 35 MiB data, 550 MiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 225 KiB/s wr, 48 op/s 2024-04-24T08:56:42.575 INFO:tasks.workunit.client.0.smithi083.stdout: 1 23916 0.61 MB/sec execute 76 sec latency 53.502 ms 2024-04-24T08:56:43.576 INFO:tasks.workunit.client.0.smithi083.stdout: 1 24100 0.64 MB/sec execute 77 sec latency 54.068 ms 2024-04-24T08:56:44.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:56:43 smithi083 ceph-mon[27907]: pgmap v156: 129 pgs: 129 active+clean; 37 MiB data, 557 MiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 427 KiB/s wr, 71 op/s 2024-04-24T08:56:44.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:56:43 smithi138 ceph-mon[31441]: pgmap v156: 129 pgs: 129 active+clean; 37 MiB data, 557 MiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 427 KiB/s wr, 71 op/s 2024-04-24T08:56:44.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:56:43 smithi189 ceph-mon[31791]: pgmap v156: 129 pgs: 129 active+clean; 37 MiB data, 557 MiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 427 KiB/s wr, 71 op/s 2024-04-24T08:56:44.576 INFO:tasks.workunit.client.0.smithi083.stdout: 1 24298 0.65 MB/sec execute 78 sec latency 54.566 ms 2024-04-24T08:56:45.576 INFO:tasks.workunit.client.0.smithi083.stdout: 1 24522 0.65 MB/sec execute 79 sec latency 53.868 ms 2024-04-24T08:56:46.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:56:45 smithi083 ceph-mon[27907]: pgmap v157: 129 pgs: 129 active+clean; 37 MiB data, 557 MiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 255 KiB/s wr, 43 op/s 2024-04-24T08:56:46.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:56:45 smithi138 ceph-mon[31441]: pgmap v157: 129 pgs: 129 active+clean; 37 MiB data, 557 MiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 255 KiB/s wr, 43 op/s 2024-04-24T08:56:46.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:56:45 smithi189 ceph-mon[31791]: pgmap v157: 129 pgs: 129 active+clean; 37 MiB data, 557 MiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 255 KiB/s wr, 43 op/s 2024-04-24T08:56:46.576 INFO:tasks.workunit.client.0.smithi083.stdout: 1 24618 0.66 MB/sec execute 80 sec latency 57.113 ms 2024-04-24T08:56:46.587 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T08:56:46.587 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T08:56:46.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.815+0000 7fdf00a00640 1 -- 172.21.15.83:0/975912046 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdefc104cc0 msgr2=0x7fdefc108d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:46.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.815+0000 7fdf00a00640 1 --2- 172.21.15.83:0/975912046 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdefc104cc0 0x7fdefc108d00 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fdee8009900 tx=0x7fdee802a3f0 comp rx=0 tx=0).stop 2024-04-24T08:56:46.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.815+0000 7fdf00a00640 1 -- 172.21.15.83:0/975912046 shutdown_connections 2024-04-24T08:56:46.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.815+0000 7fdf00a00640 1 --2- 172.21.15.83:0/975912046 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdefc1093e0 0x7fdefc110fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:46.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.816+0000 7fdf00a00640 1 --2- 172.21.15.83:0/975912046 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdefc104cc0 0x7fdefc108d00 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:46.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.816+0000 7fdf00a00640 1 --2- 172.21.15.83:0/975912046 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdefc104310 0x7fdefc1046f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:46.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.816+0000 7fdf00a00640 1 -- 172.21.15.83:0/975912046 >> 172.21.15.83:0/975912046 conn(0x7fdefc0ffb10 msgr2=0x7fdefc101f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:46.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.816+0000 7fdf00a00640 1 -- 172.21.15.83:0/975912046 shutdown_connections 2024-04-24T08:56:46.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.816+0000 7fdf00a00640 1 -- 172.21.15.83:0/975912046 wait complete. 2024-04-24T08:56:46.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.817+0000 7fdf00a00640 1 Processor -- start 2024-04-24T08:56:46.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.817+0000 7fdf00a00640 1 -- start start 2024-04-24T08:56:46.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.817+0000 7fdf00a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdefc104310 0x7fdefc1a0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:46.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.817+0000 7fdf00a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdefc1093e0 0x7fdefc1a0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:46.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.817+0000 7fdf00a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdefc19a680 0x7fdefc19ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:46.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.817+0000 7fdf00a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdefc114300 con 0x7fdefc1093e0 2024-04-24T08:56:46.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.817+0000 7fdf00a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdefc114180 con 0x7fdefc104310 2024-04-24T08:56:46.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.817+0000 7fdf00a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdefc114480 con 0x7fdefc19a680 2024-04-24T08:56:46.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.818+0000 7fdef9e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdefc1093e0 0x7fdefc1a0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:46.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.818+0000 7fdef9e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdefc1093e0 0x7fdefc1a0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53358/0 (socket says 172.21.15.83:53358) 2024-04-24T08:56:46.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.818+0000 7fdef9e00640 1 -- 172.21.15.83:0/946253124 learned_addr learned my addr 172.21.15.83:0/946253124 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:56:46.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.818+0000 7fdefa800640 1 --2- 172.21.15.83:0/946253124 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdefc104310 0x7fdefc1a0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:46.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.818+0000 7fdef9e00640 1 -- 172.21.15.83:0/946253124 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdefc19a680 msgr2=0x7fdefc19ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:46.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.818+0000 7fdef9e00640 1 --2- 172.21.15.83:0/946253124 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdefc19a680 0x7fdefc19ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:46.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.818+0000 7fdef9e00640 1 -- 172.21.15.83:0/946253124 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdefc104310 msgr2=0x7fdefc1a0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:46.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.818+0000 7fdef9e00640 1 --2- 172.21.15.83:0/946253124 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdefc104310 0x7fdefc1a0570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:46.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.818+0000 7fdef9e00640 1 -- 172.21.15.83:0/946253124 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdefc19b280 con 0x7fdefc1093e0 2024-04-24T08:56:46.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.818+0000 7fdef9e00640 1 --2- 172.21.15.83:0/946253124 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdefc1093e0 0x7fdefc1a0ab0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fdee8004270 tx=0x7fdee80042a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:46.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.819+0000 7fdef3400640 1 -- 172.21.15.83:0/946253124 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdee803f030 con 0x7fdefc1093e0 2024-04-24T08:56:46.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.819+0000 7fdef3400640 1 -- 172.21.15.83:0/946253124 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdee80044c0 con 0x7fdefc1093e0 2024-04-24T08:56:46.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.819+0000 7fdef3400640 1 -- 172.21.15.83:0/946253124 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdee8035070 con 0x7fdefc1093e0 2024-04-24T08:56:46.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.819+0000 7fdf00a00640 1 -- 172.21.15.83:0/946253124 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdefc19b510 con 0x7fdefc1093e0 2024-04-24T08:56:46.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.819+0000 7fdf00a00640 1 -- 172.21.15.83:0/946253124 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdefc1a75b0 con 0x7fdefc1093e0 2024-04-24T08:56:46.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.823+0000 7fdf0439a740 1 Processor -- start 2024-04-24T08:56:46.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.823+0000 7fdf0439a740 1 -- start start 2024-04-24T08:56:46.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.823+0000 7fdef3400640 1 -- 172.21.15.83:0/946253124 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fdee8042030 con 0x7fdefc1093e0 2024-04-24T08:56:46.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.823+0000 7fdf0439a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556402bb0a30 0x556402bb0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:46.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.823+0000 7fdf0439a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556402bb1350 0x556402bbb4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:46.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.823+0000 7fdf0439a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556402bbba20 0x556402bbde10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:46.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.823+0000 7fdf0439a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556402b161f0 con 0x556402bb1350 2024-04-24T08:56:46.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.823+0000 7fdf0439a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556402abc350 con 0x556402bb0a30 2024-04-24T08:56:46.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.823+0000 7fdf0439a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x556402b121b0 con 0x556402bbba20 2024-04-24T08:56:46.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.823+0000 7fdefa800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556402bb1350 0x556402bbb4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:46.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.824+0000 7fdefa800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556402bb1350 0x556402bbb4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53374/0 (socket says 172.21.15.83:53374) 2024-04-24T08:56:46.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.824+0000 7fdefa800640 1 -- 172.21.15.83:0/698552985 learned_addr learned my addr 172.21.15.83:0/698552985 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:56:46.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.824+0000 7fdefb200640 1 --2- 172.21.15.83:0/698552985 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556402bb0a30 0x556402bb0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:46.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.824+0000 7fdef9e00640 1 --2- 172.21.15.83:0/698552985 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556402bbba20 0x556402bbde10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:46.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.824+0000 7fdefa800640 1 -- 172.21.15.83:0/698552985 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556402bbba20 msgr2=0x556402bbde10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:46.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.824+0000 7fdefa800640 1 --2- 172.21.15.83:0/698552985 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556402bbba20 0x556402bbde10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:46.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.824+0000 7fdefa800640 1 -- 172.21.15.83:0/698552985 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556402bb0a30 msgr2=0x556402bb0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:46.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.824+0000 7fdefa800640 1 --2- 172.21.15.83:0/698552985 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556402bb0a30 0x556402bb0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:46.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.824+0000 7fdefa800640 1 -- 172.21.15.83:0/698552985 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556402b10f50 con 0x556402bb1350 2024-04-24T08:56:46.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.824+0000 7fdefa800640 1 --2- 172.21.15.83:0/698552985 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556402bb1350 0x556402bbb4e0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fdeec00bcb0 tx=0x7fdeec00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3200aa254b4315ee server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:46.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.825+0000 7fdef3400640 1 --2- 172.21.15.83:0/946253124 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fdee00a3da0 0x7fdee00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:46.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.825+0000 7fdef3400640 1 -- 172.21.15.83:0/946253124 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7fdee80f1120 con 0x7fdefc1093e0 2024-04-24T08:56:46.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.825+0000 7fdee7400640 1 -- 172.21.15.83:0/698552985 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeec015070 con 0x556402bb1350 2024-04-24T08:56:46.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.825+0000 7fdee7400640 1 -- 172.21.15.83:0/698552985 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdeec01a910 con 0x556402bb1350 2024-04-24T08:56:46.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.825+0000 7fdee7400640 1 -- 172.21.15.83:0/698552985 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeec011990 con 0x556402bb1350 2024-04-24T08:56:46.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.825+0000 7fdefb200640 1 --2- 172.21.15.83:0/946253124 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fdee00a3da0 0x7fdee00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:46.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.826+0000 7fdf0439a740 1 -- 172.21.15.83:0/698552985 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556402bb1350 msgr2=0x556402bbb4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:46.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.826+0000 7fdefb200640 1 --2- 172.21.15.83:0/946253124 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fdee00a3da0 0x7fdee00a6260 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fdef4005cc0 tx=0x7fdef400a330 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:46.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.826+0000 7fdf0439a740 1 --2- 172.21.15.83:0/698552985 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556402bb1350 0x556402bbb4e0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fdeec00bcb0 tx=0x7fdeec00f730 comp rx=0 tx=0).stop 2024-04-24T08:56:46.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.826+0000 7fdf0439a740 1 -- 172.21.15.83:0/698552985 shutdown_connections 2024-04-24T08:56:46.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.826+0000 7fdf0439a740 1 --2- 172.21.15.83:0/698552985 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556402bbba20 0x556402bbde10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:46.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.826+0000 7fdf0439a740 1 --2- 172.21.15.83:0/698552985 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556402bb1350 0x556402bbb4e0 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:46.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.826+0000 7fdf0439a740 1 --2- 172.21.15.83:0/698552985 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556402bb0a30 0x556402bb0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:46.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.826+0000 7fdf0439a740 1 -- 172.21.15.83:0/698552985 >> 172.21.15.83:0/698552985 conn(0x556402bacd50 msgr2=0x556402bad130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:46.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.826+0000 7fdf0439a740 1 -- 172.21.15.83:0/698552985 shutdown_connections 2024-04-24T08:56:46.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.826+0000 7fdf0439a740 1 -- 172.21.15.83:0/698552985 wait complete. 2024-04-24T08:56:46.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.827+0000 7fdf0439a740 1 -- start start 2024-04-24T08:56:46.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.827+0000 7fdf0439a740 1 Processor -- start 2024-04-24T08:56:46.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.827+0000 7fdf0439a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556402bb0a30 0x556402bb89f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:46.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.827+0000 7fdf0439a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556402bb1350 0x556402bb8f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:46.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.827+0000 7fdf0439a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556402bbba20 0x556402bb9470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:46.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.827+0000 7fdf0439a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556402bbe810 con 0x556402bb1350 2024-04-24T08:56:46.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.827+0000 7fdf0439a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556402acf690 con 0x556402bb0a30 2024-04-24T08:56:46.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.827+0000 7fdf0439a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x556402bbe990 con 0x556402bbba20 2024-04-24T08:56:46.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.827+0000 7fdf0439a740 10 client.0 _ll_register_callbacks cb 0x556402b0b650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:56:46.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.828+0000 7fdef9e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556402bb1350 0x556402bb8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:46.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.828+0000 7fdef9e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556402bb1350 0x556402bb8f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53384/0 (socket says 172.21.15.83:53384) 2024-04-24T08:56:46.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.828+0000 7fdef9e00640 1 -- 172.21.15.83:0/1012983601 learned_addr learned my addr 172.21.15.83:0/1012983601 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:56:46.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.828+0000 7fdefa800640 1 --2- 172.21.15.83:0/1012983601 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556402bb0a30 0x556402bb89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:46.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.828+0000 7fdefb200640 1 --2- 172.21.15.83:0/1012983601 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556402bbba20 0x556402bb9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:46.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.828+0000 7fdef9e00640 1 -- 172.21.15.83:0/1012983601 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556402bbba20 msgr2=0x556402bb9470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:46.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.828+0000 7fdef9e00640 1 --2- 172.21.15.83:0/1012983601 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556402bbba20 0x556402bb9470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:46.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.828+0000 7fdef9e00640 1 -- 172.21.15.83:0/1012983601 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556402bb0a30 msgr2=0x556402bb89f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:46.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.828+0000 7fdef9e00640 1 --2- 172.21.15.83:0/1012983601 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556402bb0a30 0x556402bb89f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:46.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.828+0000 7fdef9e00640 1 -- 172.21.15.83:0/1012983601 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556402ace920 con 0x556402bb1350 2024-04-24T08:56:46.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.828+0000 7fdef9e00640 1 --2- 172.21.15.83:0/1012983601 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556402bb1350 0x556402bb8f30 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fdee8031440 tx=0x7fdee80bc890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b85e3bf503ee7e20 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:46.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.828+0000 7fdedbe00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:56:46.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.829+0000 7fdedbe00640 1 -- 172.21.15.83:0/1012983601 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdee8049040 con 0x556402bb1350 2024-04-24T08:56:46.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.829+0000 7fdedbe00640 1 -- 172.21.15.83:0/1012983601 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7fdee80b9300 con 0x556402bb1350 2024-04-24T08:56:46.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.829+0000 7fdf0439a740 1 -- 172.21.15.83:0/1012983601 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdeec01a910 con 0x556402bb1350 2024-04-24T08:56:46.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.829+0000 7fdedbe00640 1 -- 172.21.15.83:0/1012983601 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fdee80c1040 con 0x556402bb1350 2024-04-24T08:56:46.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.829+0000 7fdf0439a740 10 client.17451 fetch_fsmap learned FSMap version 33 2024-04-24T08:56:46.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.829+0000 7fdf0439a740 1 -- 172.21.15.83:0/1012983601 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5564028f27d0 con 0x556402bb1350 2024-04-24T08:56:46.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.829+0000 7fdedbe00640 1 -- 172.21.15.83:0/1012983601 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdeec01a910 con 0x556402bb1350 2024-04-24T08:56:46.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.829+0000 7fdedbe00640 1 -- 172.21.15.83:0/1012983601 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fdee80dd0d0 con 0x556402bb1350 2024-04-24T08:56:46.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.830+0000 7fdf0439a740 10 client.17451 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T08:56:46.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.830+0000 7fdf0439a740 10 client.17451 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:56:46.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.830+0000 7fdf0439a740 20 client.17451 populate_metadata read hostname 'smithi083' 2024-04-24T08:56:46.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.830+0000 7fdf0439a740 1 --2- 172.21.15.83:0/1012983601 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556402bcbde0 0x556402bce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:46.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.830+0000 7fdf0439a740 4 client.17451 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:56:46.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.830+0000 7fdf0439a740 1 -- 172.21.15.83:0/1012983601 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556402b23c10 con 0x556402bcbde0 2024-04-24T08:56:46.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.830+0000 7fdefa800640 1 --2- 172.21.15.83:0/1012983601 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556402bcbde0 0x556402bce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:46.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.831+0000 7fdefa800640 1 --2- 172.21.15.83:0/1012983601 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556402bcbde0 0x556402bce240 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c2fd8b401bb1b5f server_cookie=f2a6f1fa1218ab4d in_seq=0 out_seq=0 2024-04-24T08:56:46.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.831+0000 7fdedbe00640 10 client.17451 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:56:46.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.833+0000 7fdedbe00640 1 -- 172.21.15.83:0/1012983601 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556402b23c10 con 0x556402bcbde0 2024-04-24T08:56:46.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.833+0000 7fdedbe00640 10 client.17451 handle_command_reply: tid=0 2024-04-24T08:56:46.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.833+0000 7fdf0439a740 1 client.17451 shutdown 2024-04-24T08:56:46.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.834+0000 7fdf0439a740 1 -- 172.21.15.83:0/1012983601 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556402bb1350 msgr2=0x556402bb8f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:46.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.834+0000 7fdf0439a740 1 --2- 172.21.15.83:0/1012983601 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556402bb1350 0x556402bb8f30 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fdee8031440 tx=0x7fdee80bc890 comp rx=0 tx=0).stop 2024-04-24T08:56:46.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.834+0000 7fdf0439a740 1 -- 172.21.15.83:0/1012983601 shutdown_connections 2024-04-24T08:56:46.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.834+0000 7fdf0439a740 1 --2- 172.21.15.83:0/1012983601 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556402bcbde0 0x556402bce240 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:46.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.834+0000 7fdf0439a740 1 --2- 172.21.15.83:0/1012983601 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556402bbba20 0x556402bb9470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:46.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.834+0000 7fdedbe00640 0 client.17451 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:56:46.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.834+0000 7fdf0439a740 1 --2- 172.21.15.83:0/1012983601 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556402bb1350 0x556402bb8f30 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:46.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.834+0000 7fdf0439a740 1 --2- 172.21.15.83:0/1012983601 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556402bb0a30 0x556402bb89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:46.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.834+0000 7fdf0439a740 1 -- 172.21.15.83:0/1012983601 >> 172.21.15.83:0/1012983601 conn(0x556402bacd50 msgr2=0x556402babfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:46.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.834+0000 7fdf0439a740 1 -- 172.21.15.83:0/1012983601 shutdown_connections 2024-04-24T08:56:46.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.834+0000 7fdf0439a740 1 -- 172.21.15.83:0/1012983601 wait complete. 2024-04-24T08:56:46.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.834+0000 7fdf0439a740 20 client.17451 trim_cache size 0 max 16384 2024-04-24T08:56:46.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.856+0000 7fdf0439a740 1 Processor -- start 2024-04-24T08:56:46.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.856+0000 7fdf0439a740 1 -- start start 2024-04-24T08:56:46.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.856+0000 7fdf0439a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556402bb0a30 0x556402bb9240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:46.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.856+0000 7fdf0439a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556402bb1350 0x556402bb31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:46.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.856+0000 7fdf0439a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556402bbba20 0x556402bd3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:46.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.856+0000 7fdf0439a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdefc1b1490 con 0x556402bb0a30 2024-04-24T08:56:46.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.856+0000 7fdf0439a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdefc1b2fd0 con 0x556402bbba20 2024-04-24T08:56:46.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.856+0000 7fdf0439a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdefc1b1b60 con 0x556402bb1350 2024-04-24T08:56:46.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.856+0000 7fdefa800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556402bb0a30 0x556402bb9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:46.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.856+0000 7fdefa800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556402bb0a30 0x556402bb9240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53400/0 (socket says 172.21.15.83:53400) 2024-04-24T08:56:46.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.856+0000 7fdefa800640 1 -- 172.21.15.83:0/1504697435 learned_addr learned my addr 172.21.15.83:0/1504697435 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:56:46.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.857+0000 7fdef9e00640 1 --2- 172.21.15.83:0/1504697435 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556402bb1350 0x556402bb31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:46.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.857+0000 7fdefb200640 1 --2- 172.21.15.83:0/1504697435 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556402bbba20 0x556402bd3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:46.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.857+0000 7fdefa800640 1 -- 172.21.15.83:0/1504697435 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556402bb1350 msgr2=0x556402bb31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:46.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.857+0000 7fdefa800640 1 --2- 172.21.15.83:0/1504697435 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556402bb1350 0x556402bb31d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:46.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.857+0000 7fdefa800640 1 -- 172.21.15.83:0/1504697435 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556402bbba20 msgr2=0x556402bd3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:46.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.857+0000 7fdefa800640 1 --2- 172.21.15.83:0/1504697435 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556402bbba20 0x556402bd3b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:46.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.857+0000 7fdefa800640 1 -- 172.21.15.83:0/1504697435 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556402bbe180 con 0x556402bb0a30 2024-04-24T08:56:46.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.857+0000 7fdefb200640 1 --2- 172.21.15.83:0/1504697435 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556402bbba20 0x556402bd3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:56:46.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.857+0000 7fdefa800640 1 --2- 172.21.15.83:0/1504697435 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556402bb0a30 0x556402bb9240 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x556402bb1ff0 tx=0x7fdeec00b3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8ec8b989fd2c22b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:46.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.858+0000 7fdee6a00640 1 -- 172.21.15.83:0/1504697435 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeec02e070 con 0x556402bb0a30 2024-04-24T08:56:46.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.858+0000 7fdee6a00640 1 -- 172.21.15.83:0/1504697435 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdeec00be60 con 0x556402bb0a30 2024-04-24T08:56:46.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.858+0000 7fdee6a00640 1 -- 172.21.15.83:0/1504697435 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeec015040 con 0x556402bb0a30 2024-04-24T08:56:46.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.858+0000 7fdf0439a740 1 -- 172.21.15.83:0/1504697435 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556402bb0a30 msgr2=0x556402bb9240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:46.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.858+0000 7fdf0439a740 1 --2- 172.21.15.83:0/1504697435 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556402bb0a30 0x556402bb9240 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x556402bb1ff0 tx=0x7fdeec00b3e0 comp rx=0 tx=0).stop 2024-04-24T08:56:46.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.858+0000 7fdf0439a740 1 -- 172.21.15.83:0/1504697435 shutdown_connections 2024-04-24T08:56:46.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.858+0000 7fdf0439a740 1 --2- 172.21.15.83:0/1504697435 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556402bbba20 0x556402bd3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:46.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.858+0000 7fdf0439a740 1 --2- 172.21.15.83:0/1504697435 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556402bb1350 0x556402bb31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:46.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.858+0000 7fdf0439a740 1 --2- 172.21.15.83:0/1504697435 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556402bb0a30 0x556402bb9240 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:46.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.858+0000 7fdf0439a740 1 -- 172.21.15.83:0/1504697435 >> 172.21.15.83:0/1504697435 conn(0x556402bacd50 msgr2=0x556402bba220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:46.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.858+0000 7fdf0439a740 1 -- 172.21.15.83:0/1504697435 shutdown_connections 2024-04-24T08:56:46.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.859+0000 7fdf0439a740 1 -- 172.21.15.83:0/1504697435 wait complete. 2024-04-24T08:56:46.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.859+0000 7fdf0439a740 1 -- start start 2024-04-24T08:56:46.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.859+0000 7fdf0439a740 1 Processor -- start 2024-04-24T08:56:46.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.860+0000 7fdf0439a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556402bb0a30 0x556402bc3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:46.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.860+0000 7fdf0439a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556402bb1350 0x556402bc4170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:46.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.860+0000 7fdf0439a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556402bbba20 0x556402bc0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:46.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.860+0000 7fdf0439a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdefc1b2900 con 0x556402bbba20 2024-04-24T08:56:46.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.860+0000 7fdf0439a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdefc1b2230 con 0x556402bb1350 2024-04-24T08:56:46.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.860+0000 7fdf0439a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdefc1af470 con 0x556402bb0a30 2024-04-24T08:56:46.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.860+0000 7fdf0439a740 10 client.0 _ll_register_callbacks cb 0x7fdee8005760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:56:46.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.860+0000 7fdefb200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556402bbba20 0x556402bc0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:46.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.860+0000 7fdefb200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556402bbba20 0x556402bc0c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53406/0 (socket says 172.21.15.83:53406) 2024-04-24T08:56:46.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.860+0000 7fdefb200640 1 -- 172.21.15.83:0/2421042445 learned_addr learned my addr 172.21.15.83:0/2421042445 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:56:46.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.860+0000 7fdefa800640 1 --2- 172.21.15.83:0/2421042445 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556402bb0a30 0x556402bc3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:46.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.860+0000 7fdef9e00640 1 --2- 172.21.15.83:0/2421042445 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556402bb1350 0x556402bc4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:46.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.860+0000 7fdefb200640 1 -- 172.21.15.83:0/2421042445 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556402bb0a30 msgr2=0x556402bc3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:46.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.860+0000 7fdefb200640 1 --2- 172.21.15.83:0/2421042445 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556402bb0a30 0x556402bc3c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:46.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.860+0000 7fdefb200640 1 -- 172.21.15.83:0/2421042445 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556402bb1350 msgr2=0x556402bc4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:46.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.860+0000 7fdefb200640 1 --2- 172.21.15.83:0/2421042445 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556402bb1350 0x556402bc4170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:46.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.860+0000 7fdefb200640 1 -- 172.21.15.83:0/2421042445 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564028283f0 con 0x556402bbba20 2024-04-24T08:56:46.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.861+0000 7fdefb200640 1 --2- 172.21.15.83:0/2421042445 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556402bbba20 0x556402bc0c00 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fdef40109e0 tx=0x7fdef4005c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ed4889a2b57f408 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:46.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.861+0000 7fdedbe00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:56:46.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.861+0000 7fdedbe00640 1 -- 172.21.15.83:0/2421042445 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdef4004d20 con 0x556402bbba20 2024-04-24T08:56:46.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.861+0000 7fdedbe00640 1 -- 172.21.15.83:0/2421042445 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7fdef400fb50 con 0x556402bbba20 2024-04-24T08:56:46.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.861+0000 7fdf0439a740 1 -- 172.21.15.83:0/2421042445 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdeec00be60 con 0x556402bbba20 2024-04-24T08:56:46.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.861+0000 7fdedbe00640 1 -- 172.21.15.83:0/2421042445 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fdef402f050 con 0x556402bbba20 2024-04-24T08:56:46.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.861+0000 7fdf0439a740 10 client.17457 fetch_fsmap learned FSMap version 33 2024-04-24T08:56:46.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.861+0000 7fdf0439a740 1 -- 172.21.15.83:0/2421042445 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5564027e80f0 con 0x556402bbba20 2024-04-24T08:56:46.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.862+0000 7fdedbe00640 1 -- 172.21.15.83:0/2421042445 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdeec00be60 con 0x556402bbba20 2024-04-24T08:56:46.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.862+0000 7fdedbe00640 1 -- 172.21.15.83:0/2421042445 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fdef401f440 con 0x556402bbba20 2024-04-24T08:56:46.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.862+0000 7fdf0439a740 10 client.17457 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T08:56:46.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.862+0000 7fdf0439a740 10 client.17457 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:56:46.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.862+0000 7fdf0439a740 20 client.17457 populate_metadata read hostname 'smithi083' 2024-04-24T08:56:46.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.862+0000 7fdf0439a740 1 --2- 172.21.15.83:0/2421042445 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556402c33140 0x556402c35560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:46.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.862+0000 7fdf0439a740 4 client.17457 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T08:56:46.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.862+0000 7fdf0439a740 1 -- 172.21.15.83:0/2421042445 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556402b1da80 con 0x556402c33140 2024-04-24T08:56:46.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.863+0000 7fdefa800640 1 --2- 172.21.15.83:0/2421042445 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556402c33140 0x556402c35560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:46.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.864+0000 7fdefa800640 1 --2- 172.21.15.83:0/2421042445 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556402c33140 0x556402c35560 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3b864b34ed8c285 server_cookie=9f18b36920f547bc in_seq=0 out_seq=0 2024-04-24T08:56:46.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.864+0000 7fdedbe00640 10 client.17457 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:56:46.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.865+0000 7fdedbe00640 1 -- 172.21.15.83:0/2421042445 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x556402b1da80 con 0x556402c33140 2024-04-24T08:56:46.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.865+0000 7fdedbe00640 10 client.17457 handle_command_reply: tid=0 2024-04-24T08:56:46.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.865+0000 7fdf0439a740 1 client.17457 shutdown 2024-04-24T08:56:46.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.866+0000 7fdf0439a740 1 -- 172.21.15.83:0/2421042445 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556402bbba20 msgr2=0x556402bc0c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:46.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.866+0000 7fdf0439a740 1 --2- 172.21.15.83:0/2421042445 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556402bbba20 0x556402bc0c00 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fdef40109e0 tx=0x7fdef4005c10 comp rx=0 tx=0).stop 2024-04-24T08:56:46.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.866+0000 7fdf0439a740 1 -- 172.21.15.83:0/2421042445 shutdown_connections 2024-04-24T08:56:46.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.866+0000 7fdf0439a740 1 --2- 172.21.15.83:0/2421042445 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556402c33140 0x556402c35560 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:46.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.866+0000 7fdf0439a740 1 --2- 172.21.15.83:0/2421042445 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556402bbba20 0x556402bc0c00 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:46.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.866+0000 7fdf0439a740 1 --2- 172.21.15.83:0/2421042445 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556402bb1350 0x556402bc4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:46.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.866+0000 7fdf0439a740 1 --2- 172.21.15.83:0/2421042445 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556402bb0a30 0x556402bc3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:46.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.866+0000 7fdedbe00640 0 client.17457 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:56:46.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.866+0000 7fdf0439a740 1 -- 172.21.15.83:0/2421042445 >> 172.21.15.83:0/2421042445 conn(0x556402bacd50 msgr2=0x556402bbaab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:46.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.866+0000 7fdf0439a740 1 -- 172.21.15.83:0/2421042445 shutdown_connections 2024-04-24T08:56:46.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.866+0000 7fdf0439a740 1 -- 172.21.15.83:0/2421042445 wait complete. 2024-04-24T08:56:46.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.866+0000 7fdf0439a740 20 client.17457 trim_cache size 0 max 16384 2024-04-24T08:56:46.867 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T08:56:46.867 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-04-24T08:56:46.867 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-04-24T08:56:46.867 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T08:56:46.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.867+0000 7fdf00a00640 1 -- 172.21.15.83:0/946253124 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fdee00a3da0 msgr2=0x7fdee00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:46.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.867+0000 7fdf00a00640 1 --2- 172.21.15.83:0/946253124 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fdee00a3da0 0x7fdee00a6260 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fdef4005cc0 tx=0x7fdef400a330 comp rx=0 tx=0).stop 2024-04-24T08:56:46.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.867+0000 7fdf00a00640 1 -- 172.21.15.83:0/946253124 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdefc1093e0 msgr2=0x7fdefc1a0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:46.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.867+0000 7fdf00a00640 1 --2- 172.21.15.83:0/946253124 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdefc1093e0 0x7fdefc1a0ab0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fdee8004270 tx=0x7fdee80042a0 comp rx=0 tx=0).stop 2024-04-24T08:56:46.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.867+0000 7fdf00a00640 1 -- 172.21.15.83:0/946253124 shutdown_connections 2024-04-24T08:56:46.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.867+0000 7fdf00a00640 1 --2- 172.21.15.83:0/946253124 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fdee00a3da0 0x7fdee00a6260 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:46.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.867+0000 7fdf00a00640 1 --2- 172.21.15.83:0/946253124 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdefc19a680 0x7fdefc19ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:46.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.867+0000 7fdf00a00640 1 --2- 172.21.15.83:0/946253124 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdefc1093e0 0x7fdefc1a0ab0 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:46.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.867+0000 7fdf00a00640 1 --2- 172.21.15.83:0/946253124 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdefc104310 0x7fdefc1a0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:46.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.867+0000 7fdf00a00640 1 -- 172.21.15.83:0/946253124 >> 172.21.15.83:0/946253124 conn(0x7fdefc0ffb10 msgr2=0x7fdefc101e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:46.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.867+0000 7fdf00a00640 1 -- 172.21.15.83:0/946253124 shutdown_connections 2024-04-24T08:56:46.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:46.867+0000 7fdf00a00640 1 -- 172.21.15.83:0/946253124 wait complete. 2024-04-24T08:56:46.886 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T08:56:46.886 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T08:56:47.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.098+0000 7f604aa00640 1 -- 172.21.15.83:0/3708659408 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6044109510 msgr2=0x7f60441110e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:47.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.098+0000 7f604aa00640 1 --2- 172.21.15.83:0/3708659408 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6044109510 0x7f60441110e0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f603c009ca0 tx=0x7f603c02a8c0 comp rx=0 tx=0).stop 2024-04-24T08:56:47.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.098+0000 7f604aa00640 1 -- 172.21.15.83:0/3708659408 shutdown_connections 2024-04-24T08:56:47.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.098+0000 7f604aa00640 1 --2- 172.21.15.83:0/3708659408 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6044109510 0x7f60441110e0 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:47.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.098+0000 7f604aa00640 1 --2- 172.21.15.83:0/3708659408 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6044104df0 0x7f6044108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:47.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.098+0000 7f604aa00640 1 --2- 172.21.15.83:0/3708659408 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6044104440 0x7f6044104820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:47.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.098+0000 7f604aa00640 1 -- 172.21.15.83:0/3708659408 >> 172.21.15.83:0/3708659408 conn(0x7f60440ffd00 msgr2=0x7f6044102120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:47.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.098+0000 7f604aa00640 1 -- 172.21.15.83:0/3708659408 shutdown_connections 2024-04-24T08:56:47.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.098+0000 7f604aa00640 1 -- 172.21.15.83:0/3708659408 wait complete. 2024-04-24T08:56:47.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.099+0000 7f604aa00640 1 Processor -- start 2024-04-24T08:56:47.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.099+0000 7f604aa00640 1 -- start start 2024-04-24T08:56:47.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.100+0000 7f604aa00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6044104440 0x7f60441a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:47.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.100+0000 7f604aa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6044109510 0x7f60441a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:47.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.100+0000 7f604aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f604419a850 0x7f604419ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:47.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.100+0000 7f604aa00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f60441144b0 con 0x7f604419a850 2024-04-24T08:56:47.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.100+0000 7f604aa00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6044114330 con 0x7f6044109510 2024-04-24T08:56:47.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.100+0000 7f604aa00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6044114630 con 0x7f6044104440 2024-04-24T08:56:47.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.100+0000 7f6048a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f604419a850 0x7f604419ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:47.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.100+0000 7f6048a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f604419a850 0x7f604419ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53422/0 (socket says 172.21.15.83:53422) 2024-04-24T08:56:47.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.100+0000 7f6048a00640 1 -- 172.21.15.83:0/724621047 learned_addr learned my addr 172.21.15.83:0/724621047 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:56:47.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.101+0000 7f6043e00640 1 --2- 172.21.15.83:0/724621047 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6044104440 0x7f60441a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:47.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.101+0000 7f6043400640 1 --2- 172.21.15.83:0/724621047 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6044109510 0x7f60441a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:47.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.101+0000 7f6048a00640 1 -- 172.21.15.83:0/724621047 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6044104440 msgr2=0x7f60441a0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:47.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.101+0000 7f6048a00640 1 --2- 172.21.15.83:0/724621047 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6044104440 0x7f60441a0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:47.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.101+0000 7f6048a00640 1 -- 172.21.15.83:0/724621047 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6044109510 msgr2=0x7f60441a0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:47.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.101+0000 7f6048a00640 1 --2- 172.21.15.83:0/724621047 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6044109510 0x7f60441a0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:47.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.101+0000 7f6048a00640 1 -- 172.21.15.83:0/724621047 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f604419b3e0 con 0x7f604419a850 2024-04-24T08:56:47.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.101+0000 7f6043e00640 1 --2- 172.21.15.83:0/724621047 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6044104440 0x7f60441a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:56:47.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.101+0000 7f6043400640 1 --2- 172.21.15.83:0/724621047 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6044109510 0x7f60441a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:56:47.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.101+0000 7f6048a00640 1 --2- 172.21.15.83:0/724621047 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f604419a850 0x7f604419ad00 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f603c031040 tx=0x7f603c003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:47.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.102+0000 7f6040c00640 1 -- 172.21.15.83:0/724621047 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f603c003ce0 con 0x7f604419a850 2024-04-24T08:56:47.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.102+0000 7f604aa00640 1 -- 172.21.15.83:0/724621047 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f604419b670 con 0x7f604419a850 2024-04-24T08:56:47.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.102+0000 7f6040c00640 1 -- 172.21.15.83:0/724621047 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f603c0044f0 con 0x7f604419a850 2024-04-24T08:56:47.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.102+0000 7f604aa00640 1 -- 172.21.15.83:0/724621047 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60441a7800 con 0x7f604419a850 2024-04-24T08:56:47.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.102+0000 7f6040c00640 1 -- 172.21.15.83:0/724621047 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f603c030c30 con 0x7f604419a850 2024-04-24T08:56:47.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.103+0000 7f604aa00640 1 -- 172.21.15.83:0/724621047 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6044105910 con 0x7f604419a850 2024-04-24T08:56:47.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.106+0000 7f6040c00640 1 -- 172.21.15.83:0/724621047 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f603c0046a0 con 0x7f604419a850 2024-04-24T08:56:47.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.107+0000 7f6040c00640 1 --2- 172.21.15.83:0/724621047 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f60280a3d20 0x7f60280a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:47.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.107+0000 7f6040c00640 1 -- 172.21.15.83:0/724621047 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f603c0f2140 con 0x7f604419a850 2024-04-24T08:56:47.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.107+0000 7f6043e00640 1 --2- 172.21.15.83:0/724621047 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f60280a3d20 0x7f60280a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:47.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.108+0000 7f6043e00640 1 --2- 172.21.15.83:0/724621047 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f60280a3d20 0x7f60280a61e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f60340058c0 tx=0x7f6034005850 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:47.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.112+0000 7f6040c00640 1 -- 172.21.15.83:0/724621047 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f603c0be5e0 con 0x7f604419a850 2024-04-24T08:56:47.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.382+0000 7f604aa00640 1 -- 172.21.15.83:0/724621047 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f604419bd30 con 0x7f604419a850 2024-04-24T08:56:47.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.384+0000 7f6040c00640 1 -- 172.21.15.83:0/724621047 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11944 (secure 0 0 0) 0x7f604419bd30 con 0x7f604419a850 2024-04-24T08:56:47.384 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:56:47.384 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/358872257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":358872257},{"type":"v1","addr":"172.21.15.83:6835","nonce":358872257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3157070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3157070851},{"type":"v1","addr":"172.21.15.83:6837","nonce":3157070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":33,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T08:48:29.422576+0000","modified":"2024-04-24T08:56:17.048967+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34520,"mds_1":34493,"mds_2":34451,"mds_3":34427,"mds_4":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14658":{"gid":14658,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6839/4085470844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":4085470844},{"type":"v1","addr":"172.21.15.83:6839","nonce":4085470844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14676":{"gid":14676,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6841/2634754210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":2634754210},{"type":"v1","addr":"172.21.15.83:6841","nonce":2634754210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24382":{"gid":24382,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/1859921310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1859921310},{"type":"v1","addr":"172.21.15.189:6833","nonce":1859921310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/2241923197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2241923197},{"type":"v1","addr":"172.21.15.189:6835","nonce":2241923197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6837/3172311220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":3172311220},{"type":"v1","addr":"172.21.15.189:6837","nonce":3172311220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.189:6839/2548965336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":2548965336},{"type":"v1","addr":"172.21.15.189:6839","nonce":2548965336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.138:6833/1157967658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1157967658},{"type":"v1","addr":"172.21.15.138:6833","nonce":1157967658}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.138:6835/1909219758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1909219758},{"type":"v1","addr":"172.21.15.138:6835","nonce":1909219758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34493":{"gid":34493,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.138:6837/900566754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":900566754},{"type":"v1","addr":"172.21.15.138:6837","nonce":900566754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34520":{"gid":34520,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.138:6839/2236932393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2236932393},{"type":"v1","addr":"172.21.15.138:6839","nonce":2236932393}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34520,"qdb_cluster":[24454,34520,34427,34451,34493]},"id":1}]} 2024-04-24T08:56:47.384 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 33 2024-04-24T08:56:47.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.387+0000 7f604aa00640 1 -- 172.21.15.83:0/724621047 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f60280a3d20 msgr2=0x7f60280a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:47.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.387+0000 7f604aa00640 1 --2- 172.21.15.83:0/724621047 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f60280a3d20 0x7f60280a61e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f60340058c0 tx=0x7f6034005850 comp rx=0 tx=0).stop 2024-04-24T08:56:47.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.387+0000 7f604aa00640 1 -- 172.21.15.83:0/724621047 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f604419a850 msgr2=0x7f604419ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:47.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.387+0000 7f604aa00640 1 --2- 172.21.15.83:0/724621047 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f604419a850 0x7f604419ad00 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f603c031040 tx=0x7f603c003970 comp rx=0 tx=0).stop 2024-04-24T08:56:47.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.387+0000 7f604aa00640 1 -- 172.21.15.83:0/724621047 shutdown_connections 2024-04-24T08:56:47.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.387+0000 7f604aa00640 1 --2- 172.21.15.83:0/724621047 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f60280a3d20 0x7f60280a61e0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:47.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.387+0000 7f604aa00640 1 --2- 172.21.15.83:0/724621047 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f604419a850 0x7f604419ad00 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:47.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.387+0000 7f604aa00640 1 --2- 172.21.15.83:0/724621047 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6044109510 0x7f60441a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:47.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.387+0000 7f604aa00640 1 --2- 172.21.15.83:0/724621047 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6044104440 0x7f60441a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:47.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.387+0000 7f604aa00640 1 -- 172.21.15.83:0/724621047 >> 172.21.15.83:0/724621047 conn(0x7f60440ffd00 msgr2=0x7f60441020a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:47.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.387+0000 7f604aa00640 1 -- 172.21.15.83:0/724621047 shutdown_connections 2024-04-24T08:56:47.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.388+0000 7f604aa00640 1 -- 172.21.15.83:0/724621047 wait complete. 2024-04-24T08:56:47.407 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-24T08:56:47.576 INFO:tasks.workunit.client.0.smithi083.stdout: 1 24655 0.65 MB/sec execute 81 sec latency 54.105 ms 2024-04-24T08:56:47.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.626+0000 7f5925400640 1 -- 172.21.15.83:0/1091086572 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5920103980 msgr2=0x7f592010be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:47.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.626+0000 7f5925400640 1 --2- 172.21.15.83:0/1091086572 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5920103980 0x7f592010be60 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f590c009900 tx=0x7f590c02a480 comp rx=0 tx=0).stop 2024-04-24T08:56:47.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.626+0000 7f5925400640 1 -- 172.21.15.83:0/1091086572 shutdown_connections 2024-04-24T08:56:47.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.626+0000 7f5925400640 1 --2- 172.21.15.83:0/1091086572 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f592010c480 0x7f592010e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:47.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.626+0000 7f5925400640 1 --2- 172.21.15.83:0/1091086572 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5920103980 0x7f592010be60 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:47.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.626+0000 7f5925400640 1 --2- 172.21.15.83:0/1091086572 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5920103060 0x7f5920103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:47.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.626+0000 7f5925400640 1 -- 172.21.15.83:0/1091086572 >> 172.21.15.83:0/1091086572 conn(0x7f59200fc880 msgr2=0x7f59200feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:47.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.626+0000 7f5925400640 1 -- 172.21.15.83:0/1091086572 shutdown_connections 2024-04-24T08:56:47.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.626+0000 7f5925400640 1 -- 172.21.15.83:0/1091086572 wait complete. 2024-04-24T08:56:47.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.627+0000 7f5925400640 1 Processor -- start 2024-04-24T08:56:47.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.627+0000 7f5925400640 1 -- start start 2024-04-24T08:56:47.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.627+0000 7f5925400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5920103060 0x7f59201a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:47.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.627+0000 7f5925400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5920103980 0x7f59201a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:47.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.627+0000 7f5925400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f592010c480 0x7f592019a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:47.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.627+0000 7f5925400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5920112160 con 0x7f5920103980 2024-04-24T08:56:47.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.627+0000 7f5925400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5920111fe0 con 0x7f5920103060 2024-04-24T08:56:47.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.627+0000 7f5925400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f59201122e0 con 0x7f592010c480 2024-04-24T08:56:47.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.628+0000 7f591f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5920103980 0x7f59201a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:47.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.628+0000 7f591f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5920103980 0x7f59201a0b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53442/0 (socket says 172.21.15.83:53442) 2024-04-24T08:56:47.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.628+0000 7f591f400640 1 -- 172.21.15.83:0/2577538325 learned_addr learned my addr 172.21.15.83:0/2577538325 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:56:47.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.628+0000 7f591fe00640 1 --2- 172.21.15.83:0/2577538325 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5920103060 0x7f59201a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:47.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.628+0000 7f5924a00640 1 --2- 172.21.15.83:0/2577538325 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f592010c480 0x7f592019a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:47.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.628+0000 7f591f400640 1 -- 172.21.15.83:0/2577538325 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f592010c480 msgr2=0x7f592019a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:47.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.628+0000 7f591f400640 1 --2- 172.21.15.83:0/2577538325 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f592010c480 0x7f592019a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:47.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.628+0000 7f591f400640 1 -- 172.21.15.83:0/2577538325 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5920103060 msgr2=0x7f59201a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:47.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.628+0000 7f591f400640 1 --2- 172.21.15.83:0/2577538325 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5920103060 0x7f59201a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:47.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.628+0000 7f591f400640 1 -- 172.21.15.83:0/2577538325 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f592019aff0 con 0x7f5920103980 2024-04-24T08:56:47.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.629+0000 7f591f400640 1 --2- 172.21.15.83:0/2577538325 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5920103980 0x7f59201a0b60 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f590c005ab0 tx=0x7f590c003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:47.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.629+0000 7f591cc00640 1 -- 172.21.15.83:0/2577538325 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f590c03f070 con 0x7f5920103980 2024-04-24T08:56:47.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.629+0000 7f5925400640 1 -- 172.21.15.83:0/2577538325 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f592019b280 con 0x7f5920103980 2024-04-24T08:56:47.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.629+0000 7f591cc00640 1 -- 172.21.15.83:0/2577538325 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f590c031d60 con 0x7f5920103980 2024-04-24T08:56:47.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.629+0000 7f591cc00640 1 -- 172.21.15.83:0/2577538325 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f590c035030 con 0x7f5920103980 2024-04-24T08:56:47.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.629+0000 7f5925400640 1 -- 172.21.15.83:0/2577538325 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59201a75a0 con 0x7f5920103980 2024-04-24T08:56:47.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.632+0000 7f59297c5740 1 Processor -- start 2024-04-24T08:56:47.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.632+0000 7f59297c5740 1 -- start start 2024-04-24T08:56:47.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.633+0000 7f591cc00640 1 -- 172.21.15.83:0/2577538325 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f590c042070 con 0x7f5920103980 2024-04-24T08:56:47.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.633+0000 7f59297c5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3d5d8eb40 0x55d3d5d8ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:47.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.633+0000 7f59297c5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d3d5d8f460 0x55d3d5d995f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:47.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.633+0000 7f59297c5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d3d5d99b30 0x55d3d5d9bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:47.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.633+0000 7f59297c5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d3d5cf41d0 con 0x55d3d5d8eb40 2024-04-24T08:56:47.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.633+0000 7f59297c5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d3d5c9a450 con 0x55d3d5d99b30 2024-04-24T08:56:47.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.633+0000 7f59297c5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d3d5cf0190 con 0x55d3d5d8f460 2024-04-24T08:56:47.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.633+0000 7f5924a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3d5d8eb40 0x55d3d5d8ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:47.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.633+0000 7f5924a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3d5d8eb40 0x55d3d5d8ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53454/0 (socket says 172.21.15.83:53454) 2024-04-24T08:56:47.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.633+0000 7f5924a00640 1 -- 172.21.15.83:0/3049532196 learned_addr learned my addr 172.21.15.83:0/3049532196 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:56:47.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.634+0000 7f591fe00640 1 --2- 172.21.15.83:0/3049532196 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d3d5d8f460 0x55d3d5d995f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:47.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.634+0000 7f591f400640 1 --2- 172.21.15.83:0/3049532196 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d3d5d99b30 0x55d3d5d9bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:47.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.634+0000 7f5924a00640 1 -- 172.21.15.83:0/3049532196 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d3d5d8f460 msgr2=0x55d3d5d995f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:47.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.634+0000 7f5924a00640 1 --2- 172.21.15.83:0/3049532196 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d3d5d8f460 0x55d3d5d995f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:47.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.634+0000 7f5924a00640 1 -- 172.21.15.83:0/3049532196 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d3d5d99b30 msgr2=0x55d3d5d9bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:47.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.634+0000 7f5924a00640 1 --2- 172.21.15.83:0/3049532196 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d3d5d99b30 0x55d3d5d9bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:47.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.634+0000 7f5924a00640 1 -- 172.21.15.83:0/3049532196 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3d5d57310 con 0x55d3d5d8eb40 2024-04-24T08:56:47.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.634+0000 7f591f400640 1 --2- 172.21.15.83:0/3049532196 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d3d5d99b30 0x55d3d5d9bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:56:47.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.634+0000 7f5924a00640 1 --2- 172.21.15.83:0/3049532196 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3d5d8eb40 0x55d3d5d8ef20 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f5918009e60 tx=0x7f5918010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ef71b40066053f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:47.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.634+0000 7f591cc00640 1 --2- 172.21.15.83:0/2577538325 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f59040a3d20 0x7f59040a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:47.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.634+0000 7f5915200640 1 -- 172.21.15.83:0/3049532196 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5918016070 con 0x55d3d5d8eb40 2024-04-24T08:56:47.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.635+0000 7f5915200640 1 -- 172.21.15.83:0/3049532196 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5918004d20 con 0x55d3d5d8eb40 2024-04-24T08:56:47.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.635+0000 7f5915200640 1 -- 172.21.15.83:0/3049532196 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5918012740 con 0x55d3d5d8eb40 2024-04-24T08:56:47.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.635+0000 7f591cc00640 1 -- 172.21.15.83:0/2577538325 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f590c0f1110 con 0x7f5920103980 2024-04-24T08:56:47.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.635+0000 7f591fe00640 1 --2- 172.21.15.83:0/2577538325 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f59040a3d20 0x7f59040a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:47.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.636+0000 7f591fe00640 1 --2- 172.21.15.83:0/2577538325 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f59040a3d20 0x7f59040a61e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5910005fd0 tx=0x7f591000b650 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:47.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.636+0000 7f59297c5740 1 -- 172.21.15.83:0/3049532196 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3d5d8eb40 msgr2=0x55d3d5d8ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:47.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.636+0000 7f59297c5740 1 --2- 172.21.15.83:0/3049532196 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3d5d8eb40 0x55d3d5d8ef20 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f5918009e60 tx=0x7f5918010740 comp rx=0 tx=0).stop 2024-04-24T08:56:47.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.636+0000 7f59297c5740 1 -- 172.21.15.83:0/3049532196 shutdown_connections 2024-04-24T08:56:47.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.636+0000 7f59297c5740 1 --2- 172.21.15.83:0/3049532196 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d3d5d99b30 0x55d3d5d9bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:47.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.636+0000 7f59297c5740 1 --2- 172.21.15.83:0/3049532196 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d3d5d8f460 0x55d3d5d995f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:47.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.636+0000 7f59297c5740 1 --2- 172.21.15.83:0/3049532196 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3d5d8eb40 0x55d3d5d8ef20 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:47.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.636+0000 7f59297c5740 1 -- 172.21.15.83:0/3049532196 >> 172.21.15.83:0/3049532196 conn(0x55d3d5d8ae60 msgr2=0x55d3d5d8b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:47.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.636+0000 7f59297c5740 1 -- 172.21.15.83:0/3049532196 shutdown_connections 2024-04-24T08:56:47.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.636+0000 7f59297c5740 1 -- 172.21.15.83:0/3049532196 wait complete. 2024-04-24T08:56:47.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.636+0000 7f59297c5740 1 -- start start 2024-04-24T08:56:47.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.636+0000 7f59297c5740 1 Processor -- start 2024-04-24T08:56:47.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.637+0000 7f59297c5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d3d5d8eb40 0x55d3d5d995b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:47.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.637+0000 7f59297c5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3d5d8f460 0x55d3d5d968a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:47.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.637+0000 7f59297c5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d3d5d99b30 0x55d3d5d96fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:47.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.637+0000 7f59297c5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d3d5d9c9c0 con 0x55d3d5d8f460 2024-04-24T08:56:47.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.637+0000 7f59297c5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d3d5cad810 con 0x55d3d5d99b30 2024-04-24T08:56:47.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.637+0000 7f59297c5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d3d5d9cb40 con 0x55d3d5d8eb40 2024-04-24T08:56:47.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.637+0000 7f59297c5740 10 client.0 _ll_register_callbacks cb 0x55d3d5c7b140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:56:47.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.637+0000 7f591f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3d5d8f460 0x55d3d5d968a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:47.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.637+0000 7f591f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3d5d8f460 0x55d3d5d968a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53470/0 (socket says 172.21.15.83:53470) 2024-04-24T08:56:47.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.637+0000 7f591f400640 1 -- 172.21.15.83:0/2654056262 learned_addr learned my addr 172.21.15.83:0/2654056262 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:56:47.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.638+0000 7f591fe00640 1 --2- 172.21.15.83:0/2654056262 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d3d5d8eb40 0x55d3d5d995b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:47.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.638+0000 7f5924a00640 1 --2- 172.21.15.83:0/2654056262 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d3d5d99b30 0x55d3d5d96fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:47.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.638+0000 7f591f400640 1 -- 172.21.15.83:0/2654056262 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d3d5d8eb40 msgr2=0x55d3d5d995b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:47.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.638+0000 7f591f400640 1 --2- 172.21.15.83:0/2654056262 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d3d5d8eb40 0x55d3d5d995b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:47.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.638+0000 7f591f400640 1 -- 172.21.15.83:0/2654056262 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d3d5d99b30 msgr2=0x55d3d5d96fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:47.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.638+0000 7f591f400640 1 --2- 172.21.15.83:0/2654056262 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d3d5d99b30 0x55d3d5d96fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:47.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.638+0000 7f591f400640 1 -- 172.21.15.83:0/2654056262 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3d5cbe3f0 con 0x55d3d5d8f460 2024-04-24T08:56:47.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.638+0000 7f591fe00640 1 --2- 172.21.15.83:0/2654056262 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d3d5d8eb40 0x55d3d5d995b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:56:47.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.638+0000 7f591f400640 1 --2- 172.21.15.83:0/2654056262 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3d5d8f460 0x55d3d5d968a0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f590c0bcc70 tx=0x7f590c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f840a84858d18f3d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:47.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.638+0000 7f5909600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:56:47.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.638+0000 7f5909600640 1 -- 172.21.15.83:0/2654056262 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f590c049070 con 0x55d3d5d8f460 2024-04-24T08:56:47.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.638+0000 7f5909600640 1 -- 172.21.15.83:0/2654056262 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f590c03b560 con 0x55d3d5d8f460 2024-04-24T08:56:47.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.638+0000 7f59297c5740 1 -- 172.21.15.83:0/2654056262 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5918004d20 con 0x55d3d5d8f460 2024-04-24T08:56:47.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.639+0000 7f5909600640 1 -- 172.21.15.83:0/2654056262 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f590c0bd3e0 con 0x55d3d5d8f460 2024-04-24T08:56:47.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.639+0000 7f59297c5740 10 client.17475 fetch_fsmap learned FSMap version 33 2024-04-24T08:56:47.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.639+0000 7f59297c5740 1 -- 172.21.15.83:0/2654056262 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55d3d5a063f0 con 0x55d3d5d8f460 2024-04-24T08:56:47.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.639+0000 7f5909600640 1 -- 172.21.15.83:0/2654056262 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5918004d20 con 0x55d3d5d8f460 2024-04-24T08:56:47.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.639+0000 7f5909600640 1 -- 172.21.15.83:0/2654056262 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f590c04c060 con 0x55d3d5d8f460 2024-04-24T08:56:47.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.639+0000 7f59297c5740 10 client.17475 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T08:56:47.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.639+0000 7f59297c5740 10 client.17475 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-24T08:56:47.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.639+0000 7f59297c5740 20 client.17475 populate_metadata read hostname 'smithi083' 2024-04-24T08:56:47.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.639+0000 7f59297c5740 1 --2- 172.21.15.83:0/2654056262 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55d3d5da9b70 0x55d3d5dabfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:47.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.639+0000 7f59297c5740 4 client.17475 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:56:47.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.639+0000 7f59297c5740 1 -- 172.21.15.83:0/2654056262 --> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d3d5d01bf0 con 0x55d3d5da9b70 2024-04-24T08:56:47.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.640+0000 7f591fe00640 1 --2- 172.21.15.83:0/2654056262 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55d3d5da9b70 0x55d3d5dabfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:47.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.641+0000 7f591fe00640 1 --2- 172.21.15.83:0/2654056262 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55d3d5da9b70 0x55d3d5dabfa0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b0c98968bb94f43c server_cookie=1d1e71c6ff0659c in_seq=0 out_seq=0 2024-04-24T08:56:47.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.641+0000 7f5909600640 10 client.17475 ms_handle_connect on v2:172.21.15.189:6838/2548965336 2024-04-24T08:56:47.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.643+0000 7f5909600640 1 -- 172.21.15.83:0/2654056262 <== mds.4 v2:172.21.15.189:6838/2548965336 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d3d5d01bf0 con 0x55d3d5da9b70 2024-04-24T08:56:47.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.643+0000 7f5909600640 10 client.17475 handle_command_reply: tid=0 2024-04-24T08:56:47.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.643+0000 7f59297c5740 1 client.17475 shutdown 2024-04-24T08:56:47.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.644+0000 7f59297c5740 1 -- 172.21.15.83:0/2654056262 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3d5d8f460 msgr2=0x55d3d5d968a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:47.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.644+0000 7f59297c5740 1 --2- 172.21.15.83:0/2654056262 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3d5d8f460 0x55d3d5d968a0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f590c0bcc70 tx=0x7f590c009260 comp rx=0 tx=0).stop 2024-04-24T08:56:47.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.644+0000 7f59297c5740 1 -- 172.21.15.83:0/2654056262 shutdown_connections 2024-04-24T08:56:47.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.644+0000 7f59297c5740 1 --2- 172.21.15.83:0/2654056262 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55d3d5da9b70 0x55d3d5dabfa0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:47.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.644+0000 7f59297c5740 1 --2- 172.21.15.83:0/2654056262 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d3d5d99b30 0x55d3d5d96fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:47.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.644+0000 7f59297c5740 1 --2- 172.21.15.83:0/2654056262 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3d5d8f460 0x55d3d5d968a0 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:47.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.644+0000 7f59297c5740 1 --2- 172.21.15.83:0/2654056262 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d3d5d8eb40 0x55d3d5d995b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:47.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.644+0000 7f5909600640 0 client.17475 ms_handle_reset on v2:172.21.15.189:6838/2548965336 2024-04-24T08:56:47.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.644+0000 7f59297c5740 1 -- 172.21.15.83:0/2654056262 >> 172.21.15.83:0/2654056262 conn(0x55d3d5d8ae60 msgr2=0x55d3d5d90b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:47.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.644+0000 7f59297c5740 1 -- 172.21.15.83:0/2654056262 shutdown_connections 2024-04-24T08:56:47.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.644+0000 7f59297c5740 1 -- 172.21.15.83:0/2654056262 wait complete. 2024-04-24T08:56:47.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.644+0000 7f59297c5740 20 client.17475 trim_cache size 0 max 16384 2024-04-24T08:56:47.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.661+0000 7f59297c5740 1 Processor -- start 2024-04-24T08:56:47.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.661+0000 7f59297c5740 1 -- start start 2024-04-24T08:56:47.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.661+0000 7f59297c5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d3d5d8eb40 0x55d3d5d95a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:47.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.661+0000 7f59297c5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d3d5d8f460 0x55d3d5d990e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:47.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.661+0000 7f59297c5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3d5d99b30 0x55d3d5db18b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:47.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.661+0000 7f59297c5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f59201b1560 con 0x55d3d5d99b30 2024-04-24T08:56:47.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.661+0000 7f59297c5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f59201b30a0 con 0x55d3d5d8f460 2024-04-24T08:56:47.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.661+0000 7f59297c5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f59201b1c30 con 0x55d3d5d8eb40 2024-04-24T08:56:47.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.662+0000 7f5924a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3d5d99b30 0x55d3d5db18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:47.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.662+0000 7f5924a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3d5d99b30 0x55d3d5db18b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53478/0 (socket says 172.21.15.83:53478) 2024-04-24T08:56:47.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.662+0000 7f5924a00640 1 -- 172.21.15.83:0/1418621220 learned_addr learned my addr 172.21.15.83:0/1418621220 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:56:47.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.662+0000 7f591fe00640 1 --2- 172.21.15.83:0/1418621220 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d3d5d8eb40 0x55d3d5d95a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:47.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.662+0000 7f5924a00640 1 -- 172.21.15.83:0/1418621220 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d3d5d8eb40 msgr2=0x55d3d5d95a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:47.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.662+0000 7f5924a00640 1 --2- 172.21.15.83:0/1418621220 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d3d5d8eb40 0x55d3d5d95a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:47.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.662+0000 7f5924a00640 1 -- 172.21.15.83:0/1418621220 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d3d5d8f460 msgr2=0x55d3d5d990e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:47.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.662+0000 7f591f400640 1 --2- 172.21.15.83:0/1418621220 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d3d5d8f460 0x55d3d5d990e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:47.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.662+0000 7f5924a00640 1 --2- 172.21.15.83:0/1418621220 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d3d5d8f460 0x55d3d5d990e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:47.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.662+0000 7f5924a00640 1 -- 172.21.15.83:0/1418621220 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3d5ad07d0 con 0x55d3d5d99b30 2024-04-24T08:56:47.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.662+0000 7f591fe00640 1 --2- 172.21.15.83:0/1418621220 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d3d5d8eb40 0x55d3d5d95a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:56:47.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.663+0000 7f5924a00640 1 --2- 172.21.15.83:0/1418621220 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3d5d99b30 0x55d3d5db18b0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f5918009bf0 tx=0x7f5918010f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6811369663ea7dff server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:47.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.663+0000 7f590be00640 1 -- 172.21.15.83:0/1418621220 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5918027070 con 0x55d3d5d99b30 2024-04-24T08:56:47.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.663+0000 7f590be00640 1 -- 172.21.15.83:0/1418621220 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5918012bc0 con 0x55d3d5d99b30 2024-04-24T08:56:47.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.663+0000 7f590be00640 1 -- 172.21.15.83:0/1418621220 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5918016040 con 0x55d3d5d99b30 2024-04-24T08:56:47.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.664+0000 7f59297c5740 1 -- 172.21.15.83:0/1418621220 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3d5d99b30 msgr2=0x55d3d5db18b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:47.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.664+0000 7f59297c5740 1 --2- 172.21.15.83:0/1418621220 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3d5d99b30 0x55d3d5db18b0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f5918009bf0 tx=0x7f5918010f10 comp rx=0 tx=0).stop 2024-04-24T08:56:47.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.664+0000 7f59297c5740 1 -- 172.21.15.83:0/1418621220 shutdown_connections 2024-04-24T08:56:47.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.664+0000 7f59297c5740 1 --2- 172.21.15.83:0/1418621220 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3d5d99b30 0x55d3d5db18b0 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:47.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.664+0000 7f59297c5740 1 --2- 172.21.15.83:0/1418621220 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d3d5d8f460 0x55d3d5d990e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:47.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.664+0000 7f59297c5740 1 --2- 172.21.15.83:0/1418621220 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d3d5d8eb40 0x55d3d5d95a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:47.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.664+0000 7f59297c5740 1 -- 172.21.15.83:0/1418621220 >> 172.21.15.83:0/1418621220 conn(0x55d3d5d8ae60 msgr2=0x55d3d5d896c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:47.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.664+0000 7f59297c5740 1 -- 172.21.15.83:0/1418621220 shutdown_connections 2024-04-24T08:56:47.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.664+0000 7f59297c5740 1 -- 172.21.15.83:0/1418621220 wait complete. 2024-04-24T08:56:47.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.664+0000 7f59297c5740 1 -- start start 2024-04-24T08:56:47.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.665+0000 7f59297c5740 1 Processor -- start 2024-04-24T08:56:47.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.665+0000 7f59297c5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d3d5d8eb40 0x55d3d5d9aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:47.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.665+0000 7f59297c5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3d5d8f460 0x55d3d5d9b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:47.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.665+0000 7f59297c5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d3d5d99b30 0x55d3d5da04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:47.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.665+0000 7f59297c5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f59201b29d0 con 0x55d3d5d8f460 2024-04-24T08:56:47.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.665+0000 7f59297c5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f59201b2300 con 0x55d3d5d99b30 2024-04-24T08:56:47.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.665+0000 7f59297c5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f59201af540 con 0x55d3d5d8eb40 2024-04-24T08:56:47.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.665+0000 7f59297c5740 10 client.0 _ll_register_callbacks cb 0x7f590c03b460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:56:47.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.665+0000 7f591f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3d5d8f460 0x55d3d5d9b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:47.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.665+0000 7f591f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3d5d8f460 0x55d3d5d9b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53492/0 (socket says 172.21.15.83:53492) 2024-04-24T08:56:47.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.665+0000 7f591f400640 1 -- 172.21.15.83:0/403602218 learned_addr learned my addr 172.21.15.83:0/403602218 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:56:47.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.666+0000 7f591fe00640 1 --2- 172.21.15.83:0/403602218 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d3d5d8eb40 0x55d3d5d9aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:47.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.666+0000 7f5924a00640 1 --2- 172.21.15.83:0/403602218 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d3d5d99b30 0x55d3d5da04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:47.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.666+0000 7f591f400640 1 -- 172.21.15.83:0/403602218 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d3d5d8eb40 msgr2=0x55d3d5d9aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:47.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.666+0000 7f591f400640 1 --2- 172.21.15.83:0/403602218 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d3d5d8eb40 0x55d3d5d9aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:47.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.666+0000 7f591f400640 1 -- 172.21.15.83:0/403602218 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d3d5d99b30 msgr2=0x55d3d5da04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:47.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.666+0000 7f591f400640 1 --2- 172.21.15.83:0/403602218 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d3d5d99b30 0x55d3d5da04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:47.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.666+0000 7f591f400640 1 -- 172.21.15.83:0/403602218 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3d5da0a00 con 0x55d3d5d8f460 2024-04-24T08:56:47.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.666+0000 7f5924a00640 1 --2- 172.21.15.83:0/403602218 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d3d5d99b30 0x55d3d5da04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:56:47.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.666+0000 7f591f400640 1 --2- 172.21.15.83:0/403602218 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3d5d8f460 0x55d3d5d9b530 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f590c0eea90 tx=0x7f590c03b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77a47451bd1158ac server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:47.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.666+0000 7f5909600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:56:47.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.666+0000 7f5909600640 1 -- 172.21.15.83:0/403602218 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f590c049070 con 0x55d3d5d8f460 2024-04-24T08:56:47.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.666+0000 7f5909600640 1 -- 172.21.15.83:0/403602218 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f590c0dd0b0 con 0x55d3d5d8f460 2024-04-24T08:56:47.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.666+0000 7f59297c5740 1 -- 172.21.15.83:0/403602218 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5918012bc0 con 0x55d3d5d8f460 2024-04-24T08:56:47.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.667+0000 7f5909600640 1 -- 172.21.15.83:0/403602218 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f590c0bcce0 con 0x55d3d5d8f460 2024-04-24T08:56:47.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.667+0000 7f59297c5740 10 client.17481 fetch_fsmap learned FSMap version 33 2024-04-24T08:56:47.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.667+0000 7f59297c5740 1 -- 172.21.15.83:0/403602218 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55d3d5e10890 con 0x55d3d5d8f460 2024-04-24T08:56:47.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.667+0000 7f5909600640 1 -- 172.21.15.83:0/403602218 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5918012bc0 con 0x55d3d5d8f460 2024-04-24T08:56:47.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.667+0000 7f5909600640 1 -- 172.21.15.83:0/403602218 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f590c0bc130 con 0x55d3d5d8f460 2024-04-24T08:56:47.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.667+0000 7f59297c5740 10 client.17481 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T08:56:47.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.667+0000 7f59297c5740 10 client.17481 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-24T08:56:47.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.667+0000 7f59297c5740 20 client.17481 populate_metadata read hostname 'smithi083' 2024-04-24T08:56:47.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.667+0000 7f59297c5740 1 --2- 172.21.15.83:0/403602218 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55d3d5e10e00 0x55d3d5e13220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:47.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.667+0000 7f59297c5740 4 client.17481 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T08:56:47.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.667+0000 7f59297c5740 1 -- 172.21.15.83:0/403602218 --> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d3d5cfba60 con 0x55d3d5e10e00 2024-04-24T08:56:47.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.668+0000 7f591fe00640 1 --2- 172.21.15.83:0/403602218 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55d3d5e10e00 0x55d3d5e13220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:47.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.669+0000 7f591fe00640 1 --2- 172.21.15.83:0/403602218 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55d3d5e10e00 0x55d3d5e13220 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=754eb8e431602020 server_cookie=83973b646d1deb2 in_seq=0 out_seq=0 2024-04-24T08:56:47.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.669+0000 7f5909600640 10 client.17481 ms_handle_connect on v2:172.21.15.189:6838/2548965336 2024-04-24T08:56:47.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.669+0000 7f5909600640 1 -- 172.21.15.83:0/403602218 <== mds.4 v2:172.21.15.189:6838/2548965336 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d3d5cfba60 con 0x55d3d5e10e00 2024-04-24T08:56:47.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.669+0000 7f5909600640 10 client.17481 handle_command_reply: tid=0 2024-04-24T08:56:47.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.669+0000 7f59297c5740 1 client.17481 shutdown 2024-04-24T08:56:47.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.670+0000 7f59297c5740 1 -- 172.21.15.83:0/403602218 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3d5d8f460 msgr2=0x55d3d5d9b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:47.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.670+0000 7f59297c5740 1 --2- 172.21.15.83:0/403602218 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3d5d8f460 0x55d3d5d9b530 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f590c0eea90 tx=0x7f590c03b790 comp rx=0 tx=0).stop 2024-04-24T08:56:47.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.670+0000 7f59297c5740 1 -- 172.21.15.83:0/403602218 shutdown_connections 2024-04-24T08:56:47.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.670+0000 7f59297c5740 1 --2- 172.21.15.83:0/403602218 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55d3d5e10e00 0x55d3d5e13220 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:47.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.670+0000 7f59297c5740 1 --2- 172.21.15.83:0/403602218 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d3d5d99b30 0x55d3d5da04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:47.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.670+0000 7f59297c5740 1 --2- 172.21.15.83:0/403602218 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3d5d8f460 0x55d3d5d9b530 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:47.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.670+0000 7f5909600640 0 client.17481 ms_handle_reset on v2:172.21.15.189:6838/2548965336 2024-04-24T08:56:47.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.670+0000 7f59297c5740 1 --2- 172.21.15.83:0/403602218 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d3d5d8eb40 0x55d3d5d9aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:47.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.670+0000 7f59297c5740 1 -- 172.21.15.83:0/403602218 >> 172.21.15.83:0/403602218 conn(0x55d3d5d8ae60 msgr2=0x55d3d5d95880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:47.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.670+0000 7f59297c5740 1 -- 172.21.15.83:0/403602218 shutdown_connections 2024-04-24T08:56:47.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.670+0000 7f59297c5740 1 -- 172.21.15.83:0/403602218 wait complete. 2024-04-24T08:56:47.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.670+0000 7f59297c5740 20 client.17481 trim_cache size 0 max 16384 2024-04-24T08:56:47.672 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T08:56:47.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.671+0000 7f5925400640 1 -- 172.21.15.83:0/2577538325 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f59040a3d20 msgr2=0x7f59040a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:47.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.671+0000 7f5925400640 1 --2- 172.21.15.83:0/2577538325 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f59040a3d20 0x7f59040a61e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5910005fd0 tx=0x7f591000b650 comp rx=0 tx=0).stop 2024-04-24T08:56:47.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.671+0000 7f5925400640 1 -- 172.21.15.83:0/2577538325 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5920103980 msgr2=0x7f59201a0b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:47.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.671+0000 7f5925400640 1 --2- 172.21.15.83:0/2577538325 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5920103980 0x7f59201a0b60 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f590c005ab0 tx=0x7f590c003c80 comp rx=0 tx=0).stop 2024-04-24T08:56:47.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.671+0000 7f5925400640 1 -- 172.21.15.83:0/2577538325 shutdown_connections 2024-04-24T08:56:47.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.671+0000 7f5925400640 1 --2- 172.21.15.83:0/2577538325 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f59040a3d20 0x7f59040a61e0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:47.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.671+0000 7f5925400640 1 --2- 172.21.15.83:0/2577538325 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f592010c480 0x7f592019a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:47.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.671+0000 7f5925400640 1 --2- 172.21.15.83:0/2577538325 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5920103980 0x7f59201a0b60 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:47.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.671+0000 7f5925400640 1 --2- 172.21.15.83:0/2577538325 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5920103060 0x7f59201a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:47.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.671+0000 7f5925400640 1 -- 172.21.15.83:0/2577538325 >> 172.21.15.83:0/2577538325 conn(0x7f59200fc880 msgr2=0x7f5920100c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:47.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.671+0000 7f5925400640 1 -- 172.21.15.83:0/2577538325 shutdown_connections 2024-04-24T08:56:47.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.671+0000 7f5925400640 1 -- 172.21.15.83:0/2577538325 wait complete. 2024-04-24T08:56:47.688 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-24T08:56:47.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.901+0000 7fc391e00640 1 -- 172.21.15.83:0/2314108232 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc384031880 con 0x7fc39410d500 2024-04-24T08:56:47.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.902+0000 7fc398a00640 1 -- 172.21.15.83:0/2314108232 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc39410d500 msgr2=0x7fc39410d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:47.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.902+0000 7fc398a00640 1 --2- 172.21.15.83:0/2314108232 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc39410d500 0x7fc39410d8e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc384009930 tx=0x7fc38402a420 comp rx=0 tx=0).stop 2024-04-24T08:56:47.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.902+0000 7fc398a00640 1 -- 172.21.15.83:0/2314108232 shutdown_connections 2024-04-24T08:56:47.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.902+0000 7fc398a00640 1 --2- 172.21.15.83:0/2314108232 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc394108f40 0x7fc3941153b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:47.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.902+0000 7fc398a00640 1 --2- 172.21.15.83:0/2314108232 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc394108580 0x7fc394108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:47.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.902+0000 7fc398a00640 1 --2- 172.21.15.83:0/2314108232 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc39410d500 0x7fc39410d8e0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:47.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.902+0000 7fc398a00640 1 -- 172.21.15.83:0/2314108232 >> 172.21.15.83:0/2314108232 conn(0x7fc394077e20 msgr2=0x7fc394106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:47.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.902+0000 7fc398a00640 1 -- 172.21.15.83:0/2314108232 shutdown_connections 2024-04-24T08:56:47.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.902+0000 7fc398a00640 1 -- 172.21.15.83:0/2314108232 wait complete. 2024-04-24T08:56:47.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.903+0000 7fc398a00640 1 Processor -- start 2024-04-24T08:56:47.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.903+0000 7fc398a00640 1 -- start start 2024-04-24T08:56:47.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.903+0000 7fc398a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc394108580 0x7fc3941a4910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:47.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.903+0000 7fc398a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc394108f40 0x7fc3941a4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:47.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.903+0000 7fc398a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc39410d500 0x7fc39419eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:47.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.903+0000 7fc398a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc394118770 con 0x7fc394108f40 2024-04-24T08:56:47.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.903+0000 7fc398a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc3941185f0 con 0x7fc394108580 2024-04-24T08:56:47.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.903+0000 7fc398a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc3941188f0 con 0x7fc39410d500 2024-04-24T08:56:47.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.904+0000 7fc389e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc394108f40 0x7fc3941a4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:47.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.904+0000 7fc389e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc394108f40 0x7fc3941a4e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53506/0 (socket says 172.21.15.83:53506) 2024-04-24T08:56:47.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.904+0000 7fc389e00640 1 -- 172.21.15.83:0/4177503053 learned_addr learned my addr 172.21.15.83:0/4177503053 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:56:47.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.904+0000 7fc392800640 1 --2- 172.21.15.83:0/4177503053 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc394108580 0x7fc3941a4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:47.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.904+0000 7fc393200640 1 --2- 172.21.15.83:0/4177503053 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc39410d500 0x7fc39419eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:47.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.904+0000 7fc389e00640 1 -- 172.21.15.83:0/4177503053 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc39410d500 msgr2=0x7fc39419eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:47.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.904+0000 7fc389e00640 1 --2- 172.21.15.83:0/4177503053 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc39410d500 0x7fc39419eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:47.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.904+0000 7fc389e00640 1 -- 172.21.15.83:0/4177503053 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc394108580 msgr2=0x7fc3941a4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:47.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.904+0000 7fc389e00640 1 --2- 172.21.15.83:0/4177503053 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc394108580 0x7fc3941a4910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:47.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.904+0000 7fc389e00640 1 -- 172.21.15.83:0/4177503053 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc39419f290 con 0x7fc394108f40 2024-04-24T08:56:47.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.905+0000 7fc389e00640 1 --2- 172.21.15.83:0/4177503053 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc394108f40 0x7fc3941a4e50 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fc37c00e9f0 tx=0x7fc37c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:47.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.905+0000 7fc38be00640 1 -- 172.21.15.83:0/4177503053 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc37c00cd80 con 0x7fc394108f40 2024-04-24T08:56:47.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.905+0000 7fc38be00640 1 -- 172.21.15.83:0/4177503053 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc37c0054d0 con 0x7fc394108f40 2024-04-24T08:56:47.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.905+0000 7fc398a00640 1 -- 172.21.15.83:0/4177503053 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc39419f580 con 0x7fc394108f40 2024-04-24T08:56:47.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.905+0000 7fc38be00640 1 -- 172.21.15.83:0/4177503053 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc37c010490 con 0x7fc394108f40 2024-04-24T08:56:47.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.905+0000 7fc398a00640 1 -- 172.21.15.83:0/4177503053 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3941ab7d0 con 0x7fc394108f40 2024-04-24T08:56:47.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.908+0000 7fc39c214740 1 Processor -- start 2024-04-24T08:56:47.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.908+0000 7fc38be00640 1 -- 172.21.15.83:0/4177503053 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc37c005020 con 0x7fc394108f40 2024-04-24T08:56:47.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.909+0000 7fc39c214740 1 -- start start 2024-04-24T08:56:47.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.909+0000 7fc39c214740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a378bf4b40 0x55a378bf4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:47.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.909+0000 7fc39c214740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a378bf5460 0x55a378bff5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:47.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.909+0000 7fc39c214740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a378bffb30 0x55a378c01f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:47.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.909+0000 7fc39c214740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a378b5a1d0 con 0x55a378bf4b40 2024-04-24T08:56:47.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.909+0000 7fc39c214740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a378b00450 con 0x55a378bffb30 2024-04-24T08:56:47.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.909+0000 7fc39c214740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a378b56190 con 0x55a378bf5460 2024-04-24T08:56:47.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.909+0000 7fc393200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a378bf4b40 0x55a378bf4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:47.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.909+0000 7fc393200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a378bf4b40 0x55a378bf4f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53508/0 (socket says 172.21.15.83:53508) 2024-04-24T08:56:47.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.909+0000 7fc393200640 1 -- 172.21.15.83:0/377386236 learned_addr learned my addr 172.21.15.83:0/377386236 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:56:47.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.910+0000 7fc392800640 1 --2- 172.21.15.83:0/377386236 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a378bf5460 0x55a378bff5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:47.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.910+0000 7fc389e00640 1 --2- 172.21.15.83:0/377386236 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a378bffb30 0x55a378c01f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:47.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.910+0000 7fc393200640 1 -- 172.21.15.83:0/377386236 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a378bf5460 msgr2=0x55a378bff5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:47.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.910+0000 7fc393200640 1 --2- 172.21.15.83:0/377386236 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a378bf5460 0x55a378bff5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:47.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.910+0000 7fc393200640 1 -- 172.21.15.83:0/377386236 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a378bffb30 msgr2=0x55a378c01f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:47.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.910+0000 7fc393200640 1 --2- 172.21.15.83:0/377386236 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a378bffb30 0x55a378c01f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:47.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.910+0000 7fc393200640 1 -- 172.21.15.83:0/377386236 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a378bbd310 con 0x55a378bf4b40 2024-04-24T08:56:47.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.910+0000 7fc389e00640 1 --2- 172.21.15.83:0/377386236 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a378bffb30 0x55a378c01f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:56:47.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.910+0000 7fc393200640 1 --2- 172.21.15.83:0/377386236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a378bf4b40 0x55a378bf4f20 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fc38c009ed0 tx=0x7fc38c00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc789b0e32380c35 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:47.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.910+0000 7fc38be00640 1 --2- 172.21.15.83:0/4177503053 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fc3780a3d20 0x7fc3780a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:47.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.910+0000 7fc38be00640 1 -- 172.21.15.83:0/4177503053 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7fc37c0d1090 con 0x7fc394108f40 2024-04-24T08:56:47.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.910+0000 7fc383400640 1 -- 172.21.15.83:0/377386236 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc38c011640 con 0x55a378bf4b40 2024-04-24T08:56:47.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.910+0000 7fc392800640 1 --2- 172.21.15.83:0/4177503053 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fc3780a3d20 0x7fc3780a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:47.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.911+0000 7fc383400640 1 -- 172.21.15.83:0/377386236 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc38c0099c0 con 0x55a378bf4b40 2024-04-24T08:56:47.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.911+0000 7fc383400640 1 -- 172.21.15.83:0/377386236 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc38c01bca0 con 0x55a378bf4b40 2024-04-24T08:56:47.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.911+0000 7fc39c214740 1 -- 172.21.15.83:0/377386236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a378bf4b40 msgr2=0x55a378bf4f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:47.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.911+0000 7fc392800640 1 --2- 172.21.15.83:0/4177503053 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fc3780a3d20 0x7fc3780a61e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc384009900 tx=0x7fc384009610 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:47.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.911+0000 7fc39c214740 1 --2- 172.21.15.83:0/377386236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a378bf4b40 0x55a378bf4f20 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fc38c009ed0 tx=0x7fc38c00f730 comp rx=0 tx=0).stop 2024-04-24T08:56:47.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.911+0000 7fc39c214740 1 -- 172.21.15.83:0/377386236 shutdown_connections 2024-04-24T08:56:47.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.911+0000 7fc39c214740 1 --2- 172.21.15.83:0/377386236 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a378bffb30 0x55a378c01f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:47.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.911+0000 7fc39c214740 1 --2- 172.21.15.83:0/377386236 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a378bf5460 0x55a378bff5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:47.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.911+0000 7fc39c214740 1 --2- 172.21.15.83:0/377386236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a378bf4b40 0x55a378bf4f20 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:47.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.912+0000 7fc39c214740 1 -- 172.21.15.83:0/377386236 >> 172.21.15.83:0/377386236 conn(0x55a378bf0e60 msgr2=0x55a378bf1240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:47.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.912+0000 7fc39c214740 1 -- 172.21.15.83:0/377386236 shutdown_connections 2024-04-24T08:56:47.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.912+0000 7fc39c214740 1 -- 172.21.15.83:0/377386236 wait complete. 2024-04-24T08:56:47.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.912+0000 7fc39c214740 1 -- start start 2024-04-24T08:56:47.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.912+0000 7fc39c214740 1 Processor -- start 2024-04-24T08:56:47.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.913+0000 7fc39c214740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a378bf4b40 0x55a378bff5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:47.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.913+0000 7fc39c214740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a378bf5460 0x55a378bfc8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:47.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.913+0000 7fc39c214740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a378bffb30 0x55a378bfd020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:47.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.913+0000 7fc39c214740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a378c029c0 con 0x55a378bffb30 2024-04-24T08:56:47.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.913+0000 7fc39c214740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a378b13810 con 0x55a378bf5460 2024-04-24T08:56:47.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.913+0000 7fc39c214740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a378c02b40 con 0x55a378bf4b40 2024-04-24T08:56:47.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.913+0000 7fc39c214740 10 client.0 _ll_register_callbacks cb 0x55a378ae1140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:56:47.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.913+0000 7fc393200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a378bffb30 0x55a378bfd020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:47.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.913+0000 7fc393200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a378bffb30 0x55a378bfd020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53518/0 (socket says 172.21.15.83:53518) 2024-04-24T08:56:47.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.913+0000 7fc393200640 1 -- 172.21.15.83:0/4055947281 learned_addr learned my addr 172.21.15.83:0/4055947281 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:56:47.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.913+0000 7fc392800640 1 --2- 172.21.15.83:0/4055947281 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a378bf4b40 0x55a378bff5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:47.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.914+0000 7fc389e00640 1 --2- 172.21.15.83:0/4055947281 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a378bf5460 0x55a378bfc8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:47.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.914+0000 7fc393200640 1 -- 172.21.15.83:0/4055947281 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a378bf4b40 msgr2=0x55a378bff5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:47.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.914+0000 7fc393200640 1 --2- 172.21.15.83:0/4055947281 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a378bf4b40 0x55a378bff5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:47.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.914+0000 7fc393200640 1 -- 172.21.15.83:0/4055947281 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a378bf5460 msgr2=0x55a378bfc8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:47.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.914+0000 7fc393200640 1 --2- 172.21.15.83:0/4055947281 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a378bf5460 0x55a378bfc8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:47.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.914+0000 7fc393200640 1 -- 172.21.15.83:0/4055947281 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a378b243f0 con 0x55a378bffb30 2024-04-24T08:56:47.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.914+0000 7fc389e00640 1 --2- 172.21.15.83:0/4055947281 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a378bf5460 0x55a378bfc8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:56:47.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.914+0000 7fc393200640 1 --2- 172.21.15.83:0/4055947281 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a378bffb30 0x55a378bfd020 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fc38c009810 tx=0x7fc38c01b640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3fa612546195d35 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:47.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.914+0000 7fc377e00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:56:47.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.914+0000 7fc377e00640 1 -- 172.21.15.83:0/4055947281 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc38c004a40 con 0x55a378bffb30 2024-04-24T08:56:47.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.914+0000 7fc377e00640 1 -- 172.21.15.83:0/4055947281 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7fc38c004be0 con 0x55a378bffb30 2024-04-24T08:56:47.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.914+0000 7fc39c214740 1 -- 172.21.15.83:0/4055947281 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a378b67bf0 con 0x55a378bffb30 2024-04-24T08:56:47.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.915+0000 7fc377e00640 1 -- 172.21.15.83:0/4055947281 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fc38c004de0 con 0x55a378bffb30 2024-04-24T08:56:47.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.915+0000 7fc39c214740 10 client.17493 fetch_fsmap learned FSMap version 33 2024-04-24T08:56:47.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.915+0000 7fc39c214740 1 -- 172.21.15.83:0/4055947281 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55a37886c3f0 con 0x55a378bffb30 2024-04-24T08:56:47.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.915+0000 7fc377e00640 1 -- 172.21.15.83:0/4055947281 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55a378b67bf0 con 0x55a378bffb30 2024-04-24T08:56:47.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.915+0000 7fc377e00640 1 -- 172.21.15.83:0/4055947281 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fc38c004620 con 0x55a378bffb30 2024-04-24T08:56:47.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.915+0000 7fc39c214740 10 client.17493 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T08:56:47.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.915+0000 7fc39c214740 10 client.17493 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-24T08:56:47.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.915+0000 7fc39c214740 20 client.17493 populate_metadata read hostname 'smithi083' 2024-04-24T08:56:47.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.916+0000 7fc39c214740 1 --2- 172.21.15.83:0/4055947281 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55a378c0fc10 0x55a378c12000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:47.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.916+0000 7fc39c214740 4 client.17493 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:56:47.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.916+0000 7fc39c214740 1 -- 172.21.15.83:0/4055947281 --> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a378b61a60 con 0x55a378c0fc10 2024-04-24T08:56:47.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.916+0000 7fc392800640 1 --2- 172.21.15.83:0/4055947281 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55a378c0fc10 0x55a378c12000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:47.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.917+0000 7fc392800640 1 --2- 172.21.15.83:0/4055947281 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55a378c0fc10 0x55a378c12000 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a92174ff828d4b08 server_cookie=3cf99dd4dbec606a in_seq=0 out_seq=0 2024-04-24T08:56:47.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.917+0000 7fc377e00640 10 client.17493 ms_handle_connect on v2:172.21.15.138:6832/1157967658 2024-04-24T08:56:47.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.919+0000 7fc377e00640 1 -- 172.21.15.83:0/4055947281 <== mds.3 v2:172.21.15.138:6832/1157967658 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a378b61a60 con 0x55a378c0fc10 2024-04-24T08:56:47.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.919+0000 7fc377e00640 10 client.17493 handle_command_reply: tid=0 2024-04-24T08:56:47.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.919+0000 7fc39c214740 1 client.17493 shutdown 2024-04-24T08:56:47.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.920+0000 7fc39c214740 1 -- 172.21.15.83:0/4055947281 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a378bffb30 msgr2=0x55a378bfd020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:47.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.920+0000 7fc39c214740 1 --2- 172.21.15.83:0/4055947281 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a378bffb30 0x55a378bfd020 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fc38c009810 tx=0x7fc38c01b640 comp rx=0 tx=0).stop 2024-04-24T08:56:47.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.920+0000 7fc39c214740 1 -- 172.21.15.83:0/4055947281 shutdown_connections 2024-04-24T08:56:47.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.920+0000 7fc39c214740 1 --2- 172.21.15.83:0/4055947281 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a378bffb30 0x55a378bfd020 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:47.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.920+0000 7fc39c214740 1 --2- 172.21.15.83:0/4055947281 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a378bf5460 0x55a378bfc8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:47.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.920+0000 7fc39c214740 1 --2- 172.21.15.83:0/4055947281 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55a378c0fc10 0x55a378c12000 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:47.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.920+0000 7fc39c214740 1 --2- 172.21.15.83:0/4055947281 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a378bf4b40 0x55a378bff5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:47.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.920+0000 7fc39c214740 1 -- 172.21.15.83:0/4055947281 >> 172.21.15.83:0/4055947281 conn(0x55a378bf0e60 msgr2=0x55a378bf6c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:47.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.920+0000 7fc377e00640 0 client.17493 ms_handle_reset on v2:172.21.15.138:6832/1157967658 2024-04-24T08:56:47.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.920+0000 7fc39c214740 1 -- 172.21.15.83:0/4055947281 shutdown_connections 2024-04-24T08:56:47.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.920+0000 7fc39c214740 1 -- 172.21.15.83:0/4055947281 wait complete. 2024-04-24T08:56:47.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.920+0000 7fc39c214740 20 client.17493 trim_cache size 0 max 16384 2024-04-24T08:56:47.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.935+0000 7fc39c214740 1 Processor -- start 2024-04-24T08:56:47.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.935+0000 7fc39c214740 1 -- start start 2024-04-24T08:56:47.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.936+0000 7fc39c214740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a378bf4b40 0x55a378c0b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:47.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.936+0000 7fc39c214740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a378bf5460 0x55a378bfb820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:47.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.936+0000 7fc39c214740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a378bffb30 0x55a378bf8030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:47.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.936+0000 7fc39c214740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc3941b5990 con 0x55a378bffb30 2024-04-24T08:56:47.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.936+0000 7fc39c214740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc3941b74d0 con 0x55a378bf4b40 2024-04-24T08:56:47.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.936+0000 7fc39c214740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc3941b6060 con 0x55a378bf5460 2024-04-24T08:56:47.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.936+0000 7fc393200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a378bffb30 0x55a378bf8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:47.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.936+0000 7fc393200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a378bffb30 0x55a378bf8030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53532/0 (socket says 172.21.15.83:53532) 2024-04-24T08:56:47.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.936+0000 7fc393200640 1 -- 172.21.15.83:0/4244262662 learned_addr learned my addr 172.21.15.83:0/4244262662 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:56:47.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.936+0000 7fc392800640 1 --2- 172.21.15.83:0/4244262662 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a378bf4b40 0x55a378c0b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:47.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.936+0000 7fc389e00640 1 --2- 172.21.15.83:0/4244262662 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a378bf5460 0x55a378bfb820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:47.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.937+0000 7fc393200640 1 -- 172.21.15.83:0/4244262662 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a378bf5460 msgr2=0x55a378bfb820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:47.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.937+0000 7fc393200640 1 --2- 172.21.15.83:0/4244262662 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a378bf5460 0x55a378bfb820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:47.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.937+0000 7fc393200640 1 -- 172.21.15.83:0/4244262662 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a378bf4b40 msgr2=0x55a378c0b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:47.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.937+0000 7fc393200640 1 --2- 172.21.15.83:0/4244262662 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a378bf4b40 0x55a378c0b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:47.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.937+0000 7fc393200640 1 -- 172.21.15.83:0/4244262662 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3789367d0 con 0x55a378bffb30 2024-04-24T08:56:47.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.937+0000 7fc392800640 1 --2- 172.21.15.83:0/4244262662 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a378bf4b40 0x55a378c0b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:56:47.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.937+0000 7fc393200640 1 --2- 172.21.15.83:0/4244262662 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a378bffb30 0x55a378bf8030 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fc38c009540 tx=0x7fc38c005290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc3ef0bfdf6749e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:47.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.937+0000 7fc382a00640 1 -- 172.21.15.83:0/4244262662 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc38c01b6b0 con 0x55a378bffb30 2024-04-24T08:56:47.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.937+0000 7fc382a00640 1 -- 172.21.15.83:0/4244262662 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc38c005360 con 0x55a378bffb30 2024-04-24T08:56:47.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.937+0000 7fc382a00640 1 -- 172.21.15.83:0/4244262662 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc38c01ba80 con 0x55a378bffb30 2024-04-24T08:56:47.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.938+0000 7fc39c214740 1 -- 172.21.15.83:0/4244262662 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a378bffb30 msgr2=0x55a378bf8030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:47.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.938+0000 7fc39c214740 1 --2- 172.21.15.83:0/4244262662 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a378bffb30 0x55a378bf8030 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fc38c009540 tx=0x7fc38c005290 comp rx=0 tx=0).stop 2024-04-24T08:56:47.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.938+0000 7fc39c214740 1 -- 172.21.15.83:0/4244262662 shutdown_connections 2024-04-24T08:56:47.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.938+0000 7fc39c214740 1 --2- 172.21.15.83:0/4244262662 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a378bffb30 0x55a378bf8030 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:47.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.938+0000 7fc39c214740 1 --2- 172.21.15.83:0/4244262662 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a378bf5460 0x55a378bfb820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:47.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.938+0000 7fc39c214740 1 --2- 172.21.15.83:0/4244262662 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a378bf4b40 0x55a378c0b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:47.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.938+0000 7fc39c214740 1 -- 172.21.15.83:0/4244262662 >> 172.21.15.83:0/4244262662 conn(0x55a378bf0e60 msgr2=0x55a378bfc6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:47.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.938+0000 7fc39c214740 1 -- 172.21.15.83:0/4244262662 shutdown_connections 2024-04-24T08:56:47.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.938+0000 7fc39c214740 1 -- 172.21.15.83:0/4244262662 wait complete. 2024-04-24T08:56:47.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.939+0000 7fc39c214740 1 -- start start 2024-04-24T08:56:47.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.939+0000 7fc39c214740 1 Processor -- start 2024-04-24T08:56:47.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.939+0000 7fc39c214740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a378bf4b40 0x55a378c00700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:47.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.939+0000 7fc39c214740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a378bf5460 0x55a378c00c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:47.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.939+0000 7fc39c214740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a378bffb30 0x55a378c06d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:47.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.939+0000 7fc39c214740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc3941b6e00 con 0x55a378bf5460 2024-04-24T08:56:47.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.939+0000 7fc39c214740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc3941b6730 con 0x55a378bf4b40 2024-04-24T08:56:47.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.939+0000 7fc39c214740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc3941b3970 con 0x55a378bffb30 2024-04-24T08:56:47.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.940+0000 7fc39c214740 10 client.0 _ll_register_callbacks cb 0x7fc38c015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:56:47.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.940+0000 7fc389e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a378bf5460 0x55a378c00c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:47.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.940+0000 7fc389e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a378bf5460 0x55a378c00c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53548/0 (socket says 172.21.15.83:53548) 2024-04-24T08:56:47.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.940+0000 7fc389e00640 1 -- 172.21.15.83:0/3622928651 learned_addr learned my addr 172.21.15.83:0/3622928651 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:56:47.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.940+0000 7fc389e00640 1 -- 172.21.15.83:0/3622928651 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a378bffb30 msgr2=0x55a378c06d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:47.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.940+0000 7fc389e00640 1 --2- 172.21.15.83:0/3622928651 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a378bffb30 0x55a378c06d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:47.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.940+0000 7fc389e00640 1 -- 172.21.15.83:0/3622928651 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a378bf4b40 msgr2=0x55a378c00700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:47.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.940+0000 7fc389e00640 1 --2- 172.21.15.83:0/3622928651 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a378bf4b40 0x55a378c00700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:47.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.940+0000 7fc389e00640 1 -- 172.21.15.83:0/3622928651 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a378c072d0 con 0x55a378bf5460 2024-04-24T08:56:47.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.940+0000 7fc389e00640 1 --2- 172.21.15.83:0/3622928651 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a378bf5460 0x55a378c00c40 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fc37c0990c0 tx=0x7fc37c005310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67c4e669c0d449ae server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:47.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.940+0000 7fc377e00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:56:47.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.941+0000 7fc377e00640 1 -- 172.21.15.83:0/3622928651 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc37c09cc60 con 0x55a378bf5460 2024-04-24T08:56:47.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.941+0000 7fc377e00640 1 -- 172.21.15.83:0/3622928651 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7fc37c0ce690 con 0x55a378bf5460 2024-04-24T08:56:47.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.941+0000 7fc39c214740 1 -- 172.21.15.83:0/3622928651 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc38c005360 con 0x55a378bf5460 2024-04-24T08:56:47.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.941+0000 7fc377e00640 1 -- 172.21.15.83:0/3622928651 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fc37c0ce890 con 0x55a378bf5460 2024-04-24T08:56:47.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.941+0000 7fc39c214740 10 client.17499 fetch_fsmap learned FSMap version 33 2024-04-24T08:56:47.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.941+0000 7fc39c214740 1 -- 172.21.15.83:0/3622928651 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55a378c08590 con 0x55a378bf5460 2024-04-24T08:56:47.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.941+0000 7fc377e00640 1 -- 172.21.15.83:0/3622928651 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc38c005360 con 0x55a378bf5460 2024-04-24T08:56:47.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.941+0000 7fc377e00640 1 -- 172.21.15.83:0/3622928651 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fc37c0a5440 con 0x55a378bf5460 2024-04-24T08:56:47.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.941+0000 7fc39c214740 10 client.17499 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T08:56:47.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.941+0000 7fc39c214740 10 client.17499 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-24T08:56:47.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.941+0000 7fc39c214740 20 client.17499 populate_metadata read hostname 'smithi083' 2024-04-24T08:56:47.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.941+0000 7fc39c214740 1 --2- 172.21.15.83:0/3622928651 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55a378c763d0 0x55a378c787e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:47.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.941+0000 7fc39c214740 4 client.17499 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T08:56:47.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.941+0000 7fc39c214740 1 -- 172.21.15.83:0/3622928651 --> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a378add4d0 con 0x55a378c763d0 2024-04-24T08:56:47.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.942+0000 7fc392800640 1 --2- 172.21.15.83:0/3622928651 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55a378c763d0 0x55a378c787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:47.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.943+0000 7fc392800640 1 --2- 172.21.15.83:0/3622928651 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55a378c763d0 0x55a378c787e0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=cd46d3ed94ef2e34 server_cookie=67af47fa2a816edd in_seq=0 out_seq=0 2024-04-24T08:56:47.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.943+0000 7fc377e00640 10 client.17499 ms_handle_connect on v2:172.21.15.138:6832/1157967658 2024-04-24T08:56:47.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.943+0000 7fc377e00640 1 -- 172.21.15.83:0/3622928651 <== mds.3 v2:172.21.15.138:6832/1157967658 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a378add4d0 con 0x55a378c763d0 2024-04-24T08:56:47.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.943+0000 7fc377e00640 10 client.17499 handle_command_reply: tid=0 2024-04-24T08:56:47.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.944+0000 7fc39c214740 1 client.17499 shutdown 2024-04-24T08:56:47.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.944+0000 7fc39c214740 1 -- 172.21.15.83:0/3622928651 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a378bf5460 msgr2=0x55a378c00c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:47.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.944+0000 7fc39c214740 1 --2- 172.21.15.83:0/3622928651 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a378bf5460 0x55a378c00c40 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fc37c0990c0 tx=0x7fc37c005310 comp rx=0 tx=0).stop 2024-04-24T08:56:47.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.944+0000 7fc39c214740 1 -- 172.21.15.83:0/3622928651 shutdown_connections 2024-04-24T08:56:47.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.944+0000 7fc39c214740 1 --2- 172.21.15.83:0/3622928651 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55a378c763d0 0x55a378c787e0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:47.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.944+0000 7fc39c214740 1 --2- 172.21.15.83:0/3622928651 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a378bffb30 0x55a378c06d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:47.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.944+0000 7fc39c214740 1 --2- 172.21.15.83:0/3622928651 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a378bf5460 0x55a378c00c40 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:47.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.944+0000 7fc39c214740 1 --2- 172.21.15.83:0/3622928651 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a378bf4b40 0x55a378c00700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:47.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.944+0000 7fc377e00640 0 client.17499 ms_handle_reset on v2:172.21.15.138:6832/1157967658 2024-04-24T08:56:47.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.944+0000 7fc39c214740 1 -- 172.21.15.83:0/3622928651 >> 172.21.15.83:0/3622928651 conn(0x55a378bf0e60 msgr2=0x55a378bef6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:47.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.944+0000 7fc39c214740 1 -- 172.21.15.83:0/3622928651 shutdown_connections 2024-04-24T08:56:47.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.944+0000 7fc39c214740 1 -- 172.21.15.83:0/3622928651 wait complete. 2024-04-24T08:56:47.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.944+0000 7fc39c214740 20 client.17499 trim_cache size 0 max 16384 2024-04-24T08:56:47.945 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T08:56:47.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.945+0000 7fc398a00640 1 -- 172.21.15.83:0/4177503053 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fc3780a3d20 msgr2=0x7fc3780a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:47.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.945+0000 7fc398a00640 1 --2- 172.21.15.83:0/4177503053 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fc3780a3d20 0x7fc3780a61e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc384009900 tx=0x7fc384009610 comp rx=0 tx=0).stop 2024-04-24T08:56:47.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.945+0000 7fc398a00640 1 -- 172.21.15.83:0/4177503053 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc394108f40 msgr2=0x7fc3941a4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:47.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.945+0000 7fc398a00640 1 --2- 172.21.15.83:0/4177503053 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc394108f40 0x7fc3941a4e50 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fc37c00e9f0 tx=0x7fc37c00edf0 comp rx=0 tx=0).stop 2024-04-24T08:56:47.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.945+0000 7fc398a00640 1 -- 172.21.15.83:0/4177503053 shutdown_connections 2024-04-24T08:56:47.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.945+0000 7fc398a00640 1 --2- 172.21.15.83:0/4177503053 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fc3780a3d20 0x7fc3780a61e0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:47.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.945+0000 7fc398a00640 1 --2- 172.21.15.83:0/4177503053 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc39410d500 0x7fc39419eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:47.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.945+0000 7fc398a00640 1 --2- 172.21.15.83:0/4177503053 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc394108f40 0x7fc3941a4e50 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:47.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.945+0000 7fc398a00640 1 --2- 172.21.15.83:0/4177503053 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc394108580 0x7fc3941a4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:47.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.945+0000 7fc398a00640 1 -- 172.21.15.83:0/4177503053 >> 172.21.15.83:0/4177503053 conn(0x7fc394077e20 msgr2=0x7fc394078480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:47.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.945+0000 7fc398a00640 1 -- 172.21.15.83:0/4177503053 shutdown_connections 2024-04-24T08:56:47.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:47.945+0000 7fc398a00640 1 -- 172.21.15.83:0/4177503053 wait complete. 2024-04-24T08:56:47.959 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:56:47 smithi083 ceph-mon[27907]: pgmap v158: 129 pgs: 129 active+clean; 37 MiB data, 557 MiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 255 KiB/s wr, 43 op/s 2024-04-24T08:56:47.959 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:56:47 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/724621047' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:56:47.962 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T08:56:48.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.168+0000 7fc436400640 1 -- 172.21.15.83:0/2713323859 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc430104430 msgr2=0x7fc430104810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:48.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.168+0000 7fc436400640 1 --2- 172.21.15.83:0/2713323859 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc430104430 0x7fc430104810 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fc420009900 tx=0x7fc42002a410 comp rx=0 tx=0).stop 2024-04-24T08:56:48.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.169+0000 7fc436400640 1 -- 172.21.15.83:0/2713323859 shutdown_connections 2024-04-24T08:56:48.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.169+0000 7fc436400640 1 --2- 172.21.15.83:0/2713323859 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc430109500 0x7fc4301110d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:48.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.169+0000 7fc436400640 1 --2- 172.21.15.83:0/2713323859 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc430104de0 0x7fc430108e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:48.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.169+0000 7fc436400640 1 --2- 172.21.15.83:0/2713323859 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc430104430 0x7fc430104810 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:48.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.169+0000 7fc436400640 1 -- 172.21.15.83:0/2713323859 >> 172.21.15.83:0/2713323859 conn(0x7fc4300ffcb0 msgr2=0x7fc4301020d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:48.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.169+0000 7fc436400640 1 -- 172.21.15.83:0/2713323859 shutdown_connections 2024-04-24T08:56:48.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.169+0000 7fc436400640 1 -- 172.21.15.83:0/2713323859 wait complete. 2024-04-24T08:56:48.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.170+0000 7fc436400640 1 Processor -- start 2024-04-24T08:56:48.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.170+0000 7fc436400640 1 -- start start 2024-04-24T08:56:48.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.170+0000 7fc436400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc430104430 0x7fc4301a0800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:48.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.170+0000 7fc436400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc430109500 0x7fc4301a0d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:48.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.170+0000 7fc436400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc43019aa00 0x7fc43019aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:48.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.170+0000 7fc436400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc430114540 con 0x7fc430104430 2024-04-24T08:56:48.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.170+0000 7fc436400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc4301143c0 con 0x7fc430109500 2024-04-24T08:56:48.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.170+0000 7fc436400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc4301146c0 con 0x7fc43019aa00 2024-04-24T08:56:48.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.171+0000 7fc42f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc430104430 0x7fc4301a0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:48.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.171+0000 7fc42f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc430104430 0x7fc4301a0800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53568/0 (socket says 172.21.15.83:53568) 2024-04-24T08:56:48.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.171+0000 7fc42f400640 1 -- 172.21.15.83:0/2979649377 learned_addr learned my addr 172.21.15.83:0/2979649377 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:56:48.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.171+0000 7fc427e00640 1 --2- 172.21.15.83:0/2979649377 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc430109500 0x7fc4301a0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:48.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.171+0000 7fc42fe00640 1 --2- 172.21.15.83:0/2979649377 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc43019aa00 0x7fc43019aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:48.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.171+0000 7fc42f400640 1 -- 172.21.15.83:0/2979649377 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc43019aa00 msgr2=0x7fc43019aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:48.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.171+0000 7fc42f400640 1 --2- 172.21.15.83:0/2979649377 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc43019aa00 0x7fc43019aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:48.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.171+0000 7fc42f400640 1 -- 172.21.15.83:0/2979649377 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc430109500 msgr2=0x7fc4301a0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:48.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.171+0000 7fc42f400640 1 --2- 172.21.15.83:0/2979649377 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc430109500 0x7fc4301a0d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:48.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.171+0000 7fc42f400640 1 -- 172.21.15.83:0/2979649377 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc43019b520 con 0x7fc430104430 2024-04-24T08:56:48.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.171+0000 7fc427e00640 1 --2- 172.21.15.83:0/2979649377 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc430109500 0x7fc4301a0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:56:48.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.172+0000 7fc42f400640 1 --2- 172.21.15.83:0/2979649377 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc430104430 0x7fc4301a0800 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fc420006010 tx=0x7fc420004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:48.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.172+0000 7fc42cc00640 1 -- 172.21.15.83:0/2979649377 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc42003f030 con 0x7fc430104430 2024-04-24T08:56:48.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.172+0000 7fc42cc00640 1 -- 172.21.15.83:0/2979649377 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc4200035f0 con 0x7fc430104430 2024-04-24T08:56:48.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.172+0000 7fc436400640 1 -- 172.21.15.83:0/2979649377 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc43019b7b0 con 0x7fc430104430 2024-04-24T08:56:48.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.172+0000 7fc42cc00640 1 -- 172.21.15.83:0/2979649377 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc420035070 con 0x7fc430104430 2024-04-24T08:56:48.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.172+0000 7fc436400640 1 -- 172.21.15.83:0/2979649377 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4301a7950 con 0x7fc430104430 2024-04-24T08:56:48.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.176+0000 7fc439283740 1 Processor -- start 2024-04-24T08:56:48.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.176+0000 7fc439283740 1 -- start start 2024-04-24T08:56:48.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.176+0000 7fc42cc00640 1 -- 172.21.15.83:0/2979649377 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc420003b10 con 0x7fc430104430 2024-04-24T08:56:48.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.177+0000 7fc439283740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56286877d950 0x56286877dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:48.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.177+0000 7fc439283740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56286877e270 0x562868788400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:48.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.177+0000 7fc439283740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562868788940 0x56286878ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:48.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.177+0000 7fc439283740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5628686e31d0 con 0x562868788940 2024-04-24T08:56:48.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.177+0000 7fc439283740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562868689450 con 0x56286877e270 2024-04-24T08:56:48.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.177+0000 7fc439283740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5628686df190 con 0x56286877d950 2024-04-24T08:56:48.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.177+0000 7fc427e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562868788940 0x56286878ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:48.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.177+0000 7fc427e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562868788940 0x56286878ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53576/0 (socket says 172.21.15.83:53576) 2024-04-24T08:56:48.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.177+0000 7fc427e00640 1 -- 172.21.15.83:0/562796547 learned_addr learned my addr 172.21.15.83:0/562796547 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:56:48.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.177+0000 7fc42fe00640 1 --2- 172.21.15.83:0/562796547 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56286877d950 0x56286877dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:48.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.178+0000 7fc427e00640 1 -- 172.21.15.83:0/562796547 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56286877d950 msgr2=0x56286877dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:48.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.178+0000 7fc427e00640 1 --2- 172.21.15.83:0/562796547 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56286877d950 0x56286877dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:48.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.178+0000 7fc427e00640 1 -- 172.21.15.83:0/562796547 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56286877e270 msgr2=0x562868788400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:48.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.178+0000 7fc427e00640 1 --2- 172.21.15.83:0/562796547 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56286877e270 0x562868788400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:48.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.178+0000 7fc427e00640 1 -- 172.21.15.83:0/562796547 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562868746310 con 0x562868788940 2024-04-24T08:56:48.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.178+0000 7fc42cc00640 1 --2- 172.21.15.83:0/2979649377 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fc4140a3d20 0x7fc4140a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:48.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.178+0000 7fc42cc00640 1 -- 172.21.15.83:0/2979649377 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7fc4200f12b0 con 0x7fc430104430 2024-04-24T08:56:48.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.178+0000 7fc42fe00640 1 --2- 172.21.15.83:0/2979649377 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fc4140a3d20 0x7fc4140a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:48.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.178+0000 7fc427e00640 1 --2- 172.21.15.83:0/562796547 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562868788940 0x56286878ad30 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7fc41800a970 tx=0x7fc41800ae30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f602674f579359f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:48.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.178+0000 7fc424c00640 1 -- 172.21.15.83:0/562796547 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc418015070 con 0x562868788940 2024-04-24T08:56:48.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.178+0000 7fc424c00640 1 -- 172.21.15.83:0/562796547 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc41800cc00 con 0x562868788940 2024-04-24T08:56:48.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.178+0000 7fc424c00640 1 -- 172.21.15.83:0/562796547 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4180105b0 con 0x562868788940 2024-04-24T08:56:48.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.179+0000 7fc439283740 1 -- 172.21.15.83:0/562796547 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562868788940 msgr2=0x56286878ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:48.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.179+0000 7fc439283740 1 --2- 172.21.15.83:0/562796547 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562868788940 0x56286878ad30 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7fc41800a970 tx=0x7fc41800ae30 comp rx=0 tx=0).stop 2024-04-24T08:56:48.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.179+0000 7fc42fe00640 1 --2- 172.21.15.83:0/2979649377 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fc4140a3d20 0x7fc4140a61e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc428007d80 tx=0x7fc428007430 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:48.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.179+0000 7fc439283740 1 -- 172.21.15.83:0/562796547 shutdown_connections 2024-04-24T08:56:48.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.179+0000 7fc439283740 1 --2- 172.21.15.83:0/562796547 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562868788940 0x56286878ad30 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:48.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.179+0000 7fc439283740 1 --2- 172.21.15.83:0/562796547 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56286877e270 0x562868788400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:48.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.179+0000 7fc439283740 1 --2- 172.21.15.83:0/562796547 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56286877d950 0x56286877dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:48.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.179+0000 7fc439283740 1 -- 172.21.15.83:0/562796547 >> 172.21.15.83:0/562796547 conn(0x562868779c70 msgr2=0x56286877a050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:48.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.180+0000 7fc439283740 1 -- 172.21.15.83:0/562796547 shutdown_connections 2024-04-24T08:56:48.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.180+0000 7fc439283740 1 -- 172.21.15.83:0/562796547 wait complete. 2024-04-24T08:56:48.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.180+0000 7fc439283740 1 -- start start 2024-04-24T08:56:48.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.180+0000 7fc439283740 1 Processor -- start 2024-04-24T08:56:48.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.181+0000 7fc439283740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56286877d950 0x562868788400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:48.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.181+0000 7fc439283740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56286877e270 0x5628687856f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:48.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.181+0000 7fc439283740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562868788940 0x562868785e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:48.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.181+0000 7fc439283740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56286878b7d0 con 0x56286877d950 2024-04-24T08:56:48.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.181+0000 7fc439283740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56286869c810 con 0x56286877e270 2024-04-24T08:56:48.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.181+0000 7fc439283740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56286878b950 con 0x562868788940 2024-04-24T08:56:48.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.181+0000 7fc439283740 10 client.0 _ll_register_callbacks cb 0x56286866a140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:56:48.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.181+0000 7fc42f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56286877d950 0x562868788400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:48.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.181+0000 7fc42f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56286877d950 0x562868788400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53586/0 (socket says 172.21.15.83:53586) 2024-04-24T08:56:48.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.181+0000 7fc42f400640 1 -- 172.21.15.83:0/3682884039 learned_addr learned my addr 172.21.15.83:0/3682884039 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:56:48.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.181+0000 7fc42f400640 1 -- 172.21.15.83:0/3682884039 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562868788940 msgr2=0x562868785e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:48.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.181+0000 7fc42f400640 1 --2- 172.21.15.83:0/3682884039 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562868788940 0x562868785e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:48.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.181+0000 7fc42f400640 1 -- 172.21.15.83:0/3682884039 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56286877e270 msgr2=0x5628687856f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:48.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.181+0000 7fc42f400640 1 --2- 172.21.15.83:0/3682884039 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56286877e270 0x5628687856f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:48.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.182+0000 7fc42f400640 1 -- 172.21.15.83:0/3682884039 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628686ad3f0 con 0x56286877d950 2024-04-24T08:56:48.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.182+0000 7fc42f400640 1 --2- 172.21.15.83:0/3682884039 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56286877d950 0x562868788400 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fc4200ee690 tx=0x7fc4200bc8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f7a3763a3ad9771 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:48.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.182+0000 7fc41d600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:56:48.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.182+0000 7fc41d600640 1 -- 172.21.15.83:0/3682884039 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc420049040 con 0x56286877d950 2024-04-24T08:56:48.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.182+0000 7fc41d600640 1 -- 172.21.15.83:0/3682884039 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7fc4200bd0e0 con 0x56286877d950 2024-04-24T08:56:48.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.182+0000 7fc439283740 1 -- 172.21.15.83:0/3682884039 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc41800cc00 con 0x56286877d950 2024-04-24T08:56:48.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.182+0000 7fc41d600640 1 -- 172.21.15.83:0/3682884039 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fc4200c1230 con 0x56286877d950 2024-04-24T08:56:48.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.182+0000 7fc439283740 10 client.17511 fetch_fsmap learned FSMap version 33 2024-04-24T08:56:48.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.182+0000 7fc439283740 1 -- 172.21.15.83:0/3682884039 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5628683f53f0 con 0x56286877d950 2024-04-24T08:56:48.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.183+0000 7fc41d600640 1 -- 172.21.15.83:0/3682884039 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc41800cc00 con 0x56286877d950 2024-04-24T08:56:48.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.183+0000 7fc41d600640 1 -- 172.21.15.83:0/3682884039 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fc42004c060 con 0x56286877d950 2024-04-24T08:56:48.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.183+0000 7fc439283740 10 client.17511 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T08:56:48.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.183+0000 7fc439283740 10 client.17511 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-24T08:56:48.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.183+0000 7fc439283740 20 client.17511 populate_metadata read hostname 'smithi083' 2024-04-24T08:56:48.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.183+0000 7fc439283740 1 --2- 172.21.15.83:0/3682884039 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x562868798ac0 0x56286879aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:48.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.183+0000 7fc439283740 4 client.17511 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:56:48.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.183+0000 7fc439283740 1 -- 172.21.15.83:0/3682884039 --> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5628686f0bf0 con 0x562868798ac0 2024-04-24T08:56:48.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.184+0000 7fc427e00640 1 --2- 172.21.15.83:0/3682884039 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x562868798ac0 0x56286879aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:48.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.185+0000 7fc427e00640 1 --2- 172.21.15.83:0/3682884039 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x562868798ac0 0x56286879aed0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f594212a49ce23d3 server_cookie=6687716d72338391 in_seq=0 out_seq=0 2024-04-24T08:56:48.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.185+0000 7fc41d600640 10 client.17511 ms_handle_connect on v2:172.21.15.138:6834/1909219758 2024-04-24T08:56:48.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.187+0000 7fc41d600640 1 -- 172.21.15.83:0/3682884039 <== mds.2 v2:172.21.15.138:6834/1909219758 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5628686f0bf0 con 0x562868798ac0 2024-04-24T08:56:48.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.187+0000 7fc41d600640 10 client.17511 handle_command_reply: tid=0 2024-04-24T08:56:48.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.188+0000 7fc439283740 1 client.17511 shutdown 2024-04-24T08:56:48.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.188+0000 7fc439283740 1 -- 172.21.15.83:0/3682884039 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56286877d950 msgr2=0x562868788400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:48.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.188+0000 7fc439283740 1 --2- 172.21.15.83:0/3682884039 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56286877d950 0x562868788400 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fc4200ee690 tx=0x7fc4200bc8a0 comp rx=0 tx=0).stop 2024-04-24T08:56:48.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.188+0000 7fc439283740 1 -- 172.21.15.83:0/3682884039 shutdown_connections 2024-04-24T08:56:48.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.188+0000 7fc439283740 1 --2- 172.21.15.83:0/3682884039 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x562868798ac0 0x56286879aed0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:48.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.188+0000 7fc439283740 1 --2- 172.21.15.83:0/3682884039 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562868788940 0x562868785e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:48.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.188+0000 7fc439283740 1 --2- 172.21.15.83:0/3682884039 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56286877e270 0x5628687856f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:48.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.188+0000 7fc439283740 1 --2- 172.21.15.83:0/3682884039 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56286877d950 0x562868788400 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:48.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.188+0000 7fc41d600640 0 client.17511 ms_handle_reset on v2:172.21.15.138:6834/1909219758 2024-04-24T08:56:48.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.188+0000 7fc439283740 1 -- 172.21.15.83:0/3682884039 >> 172.21.15.83:0/3682884039 conn(0x562868779c70 msgr2=0x56286877fa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:48.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.188+0000 7fc439283740 1 -- 172.21.15.83:0/3682884039 shutdown_connections 2024-04-24T08:56:48.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.188+0000 7fc439283740 1 -- 172.21.15.83:0/3682884039 wait complete. 2024-04-24T08:56:48.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.188+0000 7fc439283740 20 client.17511 trim_cache size 0 max 16384 2024-04-24T08:56:48.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.205+0000 7fc439283740 1 Processor -- start 2024-04-24T08:56:48.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.205+0000 7fc439283740 1 -- start start 2024-04-24T08:56:48.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.205+0000 7fc439283740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56286877d950 0x56286879aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:48.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.205+0000 7fc439283740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56286877e270 0x562868794690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:48.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.205+0000 7fc439283740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562868788940 0x562868787f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:48.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.205+0000 7fc439283740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc4301b1780 con 0x562868788940 2024-04-24T08:56:48.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.205+0000 7fc439283740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc4301b32c0 con 0x56286877e270 2024-04-24T08:56:48.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.205+0000 7fc439283740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc4301b1e50 con 0x56286877d950 2024-04-24T08:56:48.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:56:47 smithi138 ceph-mon[31441]: pgmap v158: 129 pgs: 129 active+clean; 37 MiB data, 557 MiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 255 KiB/s wr, 43 op/s 2024-04-24T08:56:48.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:56:47 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/724621047' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:56:48.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.206+0000 7fc42fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562868788940 0x562868787f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:48.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.206+0000 7fc42fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562868788940 0x562868787f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53590/0 (socket says 172.21.15.83:53590) 2024-04-24T08:56:48.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.206+0000 7fc42fe00640 1 -- 172.21.15.83:0/1205727211 learned_addr learned my addr 172.21.15.83:0/1205727211 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:56:48.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.206+0000 7fc42f400640 1 --2- 172.21.15.83:0/1205727211 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56286877d950 0x56286879aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:48.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.206+0000 7fc427e00640 1 --2- 172.21.15.83:0/1205727211 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56286877e270 0x562868794690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:48.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.206+0000 7fc42fe00640 1 -- 172.21.15.83:0/1205727211 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56286877d950 msgr2=0x56286879aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:48.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.206+0000 7fc42fe00640 1 --2- 172.21.15.83:0/1205727211 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56286877d950 0x56286879aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:48.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.206+0000 7fc42fe00640 1 -- 172.21.15.83:0/1205727211 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56286877e270 msgr2=0x562868794690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:48.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.206+0000 7fc42fe00640 1 --2- 172.21.15.83:0/1205727211 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56286877e270 0x562868794690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:48.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.206+0000 7fc42fe00640 1 -- 172.21.15.83:0/1205727211 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628684bf7d0 con 0x562868788940 2024-04-24T08:56:48.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.206+0000 7fc42f400640 1 --2- 172.21.15.83:0/1205727211 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56286877d950 0x56286879aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:56:48.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.206+0000 7fc427e00640 1 --2- 172.21.15.83:0/1205727211 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56286877e270 0x562868794690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:56:48.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.207+0000 7fc42fe00640 1 --2- 172.21.15.83:0/1205727211 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562868788940 0x562868787f30 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fc428007fd0 tx=0x7fc428004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b9b4c5f32f9b28c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:48.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.207+0000 7fc41fe00640 1 -- 172.21.15.83:0/1205727211 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc42801a050 con 0x562868788940 2024-04-24T08:56:48.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.207+0000 7fc41fe00640 1 -- 172.21.15.83:0/1205727211 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc428024070 con 0x562868788940 2024-04-24T08:56:48.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.207+0000 7fc41fe00640 1 -- 172.21.15.83:0/1205727211 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc428012700 con 0x562868788940 2024-04-24T08:56:48.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.208+0000 7fc439283740 1 -- 172.21.15.83:0/1205727211 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562868788940 msgr2=0x562868787f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:48.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.208+0000 7fc439283740 1 --2- 172.21.15.83:0/1205727211 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562868788940 0x562868787f30 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fc428007fd0 tx=0x7fc428004b50 comp rx=0 tx=0).stop 2024-04-24T08:56:48.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.208+0000 7fc439283740 1 -- 172.21.15.83:0/1205727211 shutdown_connections 2024-04-24T08:56:48.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.208+0000 7fc439283740 1 --2- 172.21.15.83:0/1205727211 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562868788940 0x562868787f30 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:48.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.208+0000 7fc439283740 1 --2- 172.21.15.83:0/1205727211 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56286877e270 0x562868794690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:48.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.208+0000 7fc439283740 1 --2- 172.21.15.83:0/1205727211 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56286877d950 0x56286879aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:48.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.208+0000 7fc439283740 1 -- 172.21.15.83:0/1205727211 >> 172.21.15.83:0/1205727211 conn(0x562868779c70 msgr2=0x562868783fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:48.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.208+0000 7fc439283740 1 -- 172.21.15.83:0/1205727211 shutdown_connections 2024-04-24T08:56:48.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.208+0000 7fc439283740 1 -- 172.21.15.83:0/1205727211 wait complete. 2024-04-24T08:56:48.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.208+0000 7fc439283740 1 -- start start 2024-04-24T08:56:48.209 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:56:47 smithi189 ceph-mon[31791]: pgmap v158: 129 pgs: 129 active+clean; 37 MiB data, 557 MiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 255 KiB/s wr, 43 op/s 2024-04-24T08:56:48.209 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:56:47 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/724621047' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:56:48.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.208+0000 7fc439283740 1 Processor -- start 2024-04-24T08:56:48.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.209+0000 7fc439283740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56286877d950 0x562868789510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:48.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.209+0000 7fc439283740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56286877e270 0x562868789a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:48.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.209+0000 7fc439283740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562868788940 0x56286878fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:48.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.209+0000 7fc439283740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc4301b2bf0 con 0x56286877e270 2024-04-24T08:56:48.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.209+0000 7fc439283740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc4301b2520 con 0x56286877d950 2024-04-24T08:56:48.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.209+0000 7fc439283740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc4301af760 con 0x562868788940 2024-04-24T08:56:48.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.209+0000 7fc439283740 10 client.0 _ll_register_callbacks cb 0x7fc420009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:56:48.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.209+0000 7fc427e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56286877e270 0x562868789a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:48.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.209+0000 7fc427e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56286877e270 0x562868789a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53606/0 (socket says 172.21.15.83:53606) 2024-04-24T08:56:48.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.209+0000 7fc427e00640 1 -- 172.21.15.83:0/2653781195 learned_addr learned my addr 172.21.15.83:0/2653781195 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:56:48.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.210+0000 7fc42f400640 1 --2- 172.21.15.83:0/2653781195 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56286877d950 0x562868789510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:48.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.210+0000 7fc42fe00640 1 --2- 172.21.15.83:0/2653781195 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562868788940 0x56286878fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:48.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.210+0000 7fc427e00640 1 -- 172.21.15.83:0/2653781195 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562868788940 msgr2=0x56286878fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:48.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.210+0000 7fc427e00640 1 --2- 172.21.15.83:0/2653781195 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562868788940 0x56286878fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:48.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.210+0000 7fc427e00640 1 -- 172.21.15.83:0/2653781195 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56286877d950 msgr2=0x562868789510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:48.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.210+0000 7fc427e00640 1 --2- 172.21.15.83:0/2653781195 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56286877d950 0x562868789510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:48.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.210+0000 7fc427e00640 1 -- 172.21.15.83:0/2653781195 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628687900e0 con 0x56286877e270 2024-04-24T08:56:48.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.210+0000 7fc42fe00640 1 --2- 172.21.15.83:0/2653781195 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562868788940 0x56286878fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:56:48.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.210+0000 7fc427e00640 1 --2- 172.21.15.83:0/2653781195 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56286877e270 0x562868789a50 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fc41800a970 tx=0x7fc418011cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4ca92162b52e71e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:48.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.210+0000 7fc41d600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:56:48.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.210+0000 7fc41d600640 1 -- 172.21.15.83:0/2653781195 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc41802a060 con 0x56286877e270 2024-04-24T08:56:48.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.210+0000 7fc41d600640 1 -- 172.21.15.83:0/2653781195 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7fc418024bd0 con 0x56286877e270 2024-04-24T08:56:48.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.210+0000 7fc439283740 1 -- 172.21.15.83:0/2653781195 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc428024070 con 0x56286877e270 2024-04-24T08:56:48.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.211+0000 7fc41d600640 1 -- 172.21.15.83:0/2653781195 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fc418031050 con 0x56286877e270 2024-04-24T08:56:48.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.211+0000 7fc439283740 10 client.17517 fetch_fsmap learned FSMap version 33 2024-04-24T08:56:48.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.211+0000 7fc439283740 1 -- 172.21.15.83:0/2653781195 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5628687913b0 con 0x56286877e270 2024-04-24T08:56:48.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.211+0000 7fc41d600640 1 -- 172.21.15.83:0/2653781195 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc428024070 con 0x56286877e270 2024-04-24T08:56:48.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.211+0000 7fc41d600640 1 -- 172.21.15.83:0/2653781195 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fc418011480 con 0x56286877e270 2024-04-24T08:56:48.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.211+0000 7fc439283740 10 client.17517 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T08:56:48.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.211+0000 7fc439283740 10 client.17517 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-24T08:56:48.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.211+0000 7fc439283740 20 client.17517 populate_metadata read hostname 'smithi083' 2024-04-24T08:56:48.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.211+0000 7fc439283740 1 --2- 172.21.15.83:0/2653781195 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x5628687ff2b0 0x5628688016d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:48.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.211+0000 7fc439283740 4 client.17517 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T08:56:48.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.211+0000 7fc439283740 1 -- 172.21.15.83:0/2653781195 --> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5628686eaa60 con 0x5628687ff2b0 2024-04-24T08:56:48.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.212+0000 7fc42f400640 1 --2- 172.21.15.83:0/2653781195 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x5628687ff2b0 0x5628688016d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:48.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.213+0000 7fc42f400640 1 --2- 172.21.15.83:0/2653781195 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x5628687ff2b0 0x5628688016d0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6de028a8f79bf4cd server_cookie=4d8f1d1a27440dab in_seq=0 out_seq=0 2024-04-24T08:56:48.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.213+0000 7fc41d600640 10 client.17517 ms_handle_connect on v2:172.21.15.138:6834/1909219758 2024-04-24T08:56:48.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.213+0000 7fc41d600640 1 -- 172.21.15.83:0/2653781195 <== mds.2 v2:172.21.15.138:6834/1909219758 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5628686eaa60 con 0x5628687ff2b0 2024-04-24T08:56:48.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.213+0000 7fc41d600640 10 client.17517 handle_command_reply: tid=0 2024-04-24T08:56:48.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.213+0000 7fc439283740 1 client.17517 shutdown 2024-04-24T08:56:48.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.213+0000 7fc439283740 1 -- 172.21.15.83:0/2653781195 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56286877e270 msgr2=0x562868789a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:48.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.213+0000 7fc439283740 1 --2- 172.21.15.83:0/2653781195 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56286877e270 0x562868789a50 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fc41800a970 tx=0x7fc418011cb0 comp rx=0 tx=0).stop 2024-04-24T08:56:48.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.213+0000 7fc439283740 1 -- 172.21.15.83:0/2653781195 shutdown_connections 2024-04-24T08:56:48.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.213+0000 7fc439283740 1 --2- 172.21.15.83:0/2653781195 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x5628687ff2b0 0x5628688016d0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:48.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.213+0000 7fc439283740 1 --2- 172.21.15.83:0/2653781195 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562868788940 0x56286878fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:48.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.213+0000 7fc439283740 1 --2- 172.21.15.83:0/2653781195 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56286877e270 0x562868789a50 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:48.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.213+0000 7fc439283740 1 --2- 172.21.15.83:0/2653781195 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56286877d950 0x562868789510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:48.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.213+0000 7fc439283740 1 -- 172.21.15.83:0/2653781195 >> 172.21.15.83:0/2653781195 conn(0x562868779c70 msgr2=0x56286877f0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:48.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.213+0000 7fc41d600640 0 client.17517 ms_handle_reset on v2:172.21.15.138:6834/1909219758 2024-04-24T08:56:48.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.214+0000 7fc439283740 1 -- 172.21.15.83:0/2653781195 shutdown_connections 2024-04-24T08:56:48.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.214+0000 7fc439283740 1 -- 172.21.15.83:0/2653781195 wait complete. 2024-04-24T08:56:48.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.214+0000 7fc439283740 20 client.17517 trim_cache size 0 max 16384 2024-04-24T08:56:48.217 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T08:56:48.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.215+0000 7fc436400640 1 -- 172.21.15.83:0/2979649377 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fc4140a3d20 msgr2=0x7fc4140a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:48.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.215+0000 7fc436400640 1 --2- 172.21.15.83:0/2979649377 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fc4140a3d20 0x7fc4140a61e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc428007d80 tx=0x7fc428007430 comp rx=0 tx=0).stop 2024-04-24T08:56:48.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.215+0000 7fc436400640 1 -- 172.21.15.83:0/2979649377 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc430104430 msgr2=0x7fc4301a0800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:48.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.215+0000 7fc436400640 1 --2- 172.21.15.83:0/2979649377 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc430104430 0x7fc4301a0800 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fc420006010 tx=0x7fc420004270 comp rx=0 tx=0).stop 2024-04-24T08:56:48.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.215+0000 7fc436400640 1 -- 172.21.15.83:0/2979649377 shutdown_connections 2024-04-24T08:56:48.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.215+0000 7fc436400640 1 --2- 172.21.15.83:0/2979649377 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fc4140a3d20 0x7fc4140a61e0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:48.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.215+0000 7fc436400640 1 --2- 172.21.15.83:0/2979649377 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc43019aa00 0x7fc43019aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:48.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.215+0000 7fc436400640 1 --2- 172.21.15.83:0/2979649377 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc430109500 0x7fc4301a0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:48.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.215+0000 7fc436400640 1 --2- 172.21.15.83:0/2979649377 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc430104430 0x7fc4301a0800 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:48.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.215+0000 7fc436400640 1 -- 172.21.15.83:0/2979649377 >> 172.21.15.83:0/2979649377 conn(0x7fc4300ffcb0 msgr2=0x7fc430100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:48.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.215+0000 7fc436400640 1 -- 172.21.15.83:0/2979649377 shutdown_connections 2024-04-24T08:56:48.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.215+0000 7fc436400640 1 -- 172.21.15.83:0/2979649377 wait complete. 2024-04-24T08:56:48.233 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T08:56:48.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.440+0000 7fa4dbe00640 1 -- 172.21.15.83:0/2049617513 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4dc104cc0 msgr2=0x7fa4dc108d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:48.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.440+0000 7fa4dbe00640 1 --2- 172.21.15.83:0/2049617513 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4dc104cc0 0x7fa4dc108d00 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fa4c4009900 tx=0x7fa4c402a410 comp rx=0 tx=0).stop 2024-04-24T08:56:48.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.441+0000 7fa4dbe00640 1 -- 172.21.15.83:0/2049617513 shutdown_connections 2024-04-24T08:56:48.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.441+0000 7fa4dbe00640 1 --2- 172.21.15.83:0/2049617513 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4dc1093e0 0x7fa4dc110fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:48.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.441+0000 7fa4dbe00640 1 --2- 172.21.15.83:0/2049617513 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4dc104cc0 0x7fa4dc108d00 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:48.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.441+0000 7fa4dbe00640 1 --2- 172.21.15.83:0/2049617513 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa4dc104310 0x7fa4dc1046f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:48.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.441+0000 7fa4dbe00640 1 -- 172.21.15.83:0/2049617513 >> 172.21.15.83:0/2049617513 conn(0x7fa4dc0ffb10 msgr2=0x7fa4dc101f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:48.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.441+0000 7fa4dbe00640 1 -- 172.21.15.83:0/2049617513 shutdown_connections 2024-04-24T08:56:48.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.441+0000 7fa4dbe00640 1 -- 172.21.15.83:0/2049617513 wait complete. 2024-04-24T08:56:48.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.442+0000 7fa4dbe00640 1 Processor -- start 2024-04-24T08:56:48.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.442+0000 7fa4dbe00640 1 -- start start 2024-04-24T08:56:48.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.442+0000 7fa4dbe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4dc104310 0x7fa4dc1a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:48.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.442+0000 7fa4dbe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa4dc1093e0 0x7fa4dc1a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:48.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.442+0000 7fa4dbe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4dc19a6e0 0x7fa4dc19ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:48.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.442+0000 7fa4dbe00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa4dc114300 con 0x7fa4dc19a6e0 2024-04-24T08:56:48.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.442+0000 7fa4dbe00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa4dc114180 con 0x7fa4dc104310 2024-04-24T08:56:48.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.442+0000 7fa4dbe00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa4dc114480 con 0x7fa4dc1093e0 2024-04-24T08:56:48.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.443+0000 7fa4db400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4dc19a6e0 0x7fa4dc19ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:48.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.443+0000 7fa4db400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4dc19a6e0 0x7fa4dc19ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53618/0 (socket says 172.21.15.83:53618) 2024-04-24T08:56:48.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.443+0000 7fa4db400640 1 -- 172.21.15.83:0/345964509 learned_addr learned my addr 172.21.15.83:0/345964509 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:56:48.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.443+0000 7fa4d2000640 1 --2- 172.21.15.83:0/345964509 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa4dc1093e0 0x7fa4dc1a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:48.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.443+0000 7fa4daa00640 1 --2- 172.21.15.83:0/345964509 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4dc104310 0x7fa4dc1a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:48.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.443+0000 7fa4db400640 1 -- 172.21.15.83:0/345964509 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa4dc1093e0 msgr2=0x7fa4dc1a0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:48.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.443+0000 7fa4db400640 1 --2- 172.21.15.83:0/345964509 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa4dc1093e0 0x7fa4dc1a0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:48.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.443+0000 7fa4db400640 1 -- 172.21.15.83:0/345964509 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4dc104310 msgr2=0x7fa4dc1a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:48.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.443+0000 7fa4db400640 1 --2- 172.21.15.83:0/345964509 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4dc104310 0x7fa4dc1a0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:48.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.443+0000 7fa4db400640 1 -- 172.21.15.83:0/345964509 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4dc19b2e0 con 0x7fa4dc19a6e0 2024-04-24T08:56:48.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.444+0000 7fa4db400640 1 --2- 172.21.15.83:0/345964509 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4dc19a6e0 0x7fa4dc19ab60 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fa4d400eaa0 tx=0x7fa4d400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:48.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.444+0000 7fa4d3e00640 1 -- 172.21.15.83:0/345964509 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d400cdb0 con 0x7fa4dc19a6e0 2024-04-24T08:56:48.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.444+0000 7fa4d3e00640 1 -- 172.21.15.83:0/345964509 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa4d4005020 con 0x7fa4dc19a6e0 2024-04-24T08:56:48.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.444+0000 7fa4d3e00640 1 -- 172.21.15.83:0/345964509 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d40116d0 con 0x7fa4dc19a6e0 2024-04-24T08:56:48.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.444+0000 7fa4dbe00640 1 -- 172.21.15.83:0/345964509 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4dc1a73c0 con 0x7fa4dc19a6e0 2024-04-24T08:56:48.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.444+0000 7fa4dbe00640 1 -- 172.21.15.83:0/345964509 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4dc1a7770 con 0x7fa4dc19a6e0 2024-04-24T08:56:48.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.447+0000 7fa4e4a35740 1 Processor -- start 2024-04-24T08:56:48.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.448+0000 7fa4d3e00640 1 -- 172.21.15.83:0/345964509 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa4d40051d0 con 0x7fa4dc19a6e0 2024-04-24T08:56:48.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.448+0000 7fa4e4a35740 1 -- start start 2024-04-24T08:56:48.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.448+0000 7fa4e4a35740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba67069b40 0x55ba67069f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:48.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.448+0000 7fa4e4a35740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ba6706a460 0x55ba670745f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:48.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.448+0000 7fa4e4a35740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba67074b30 0x55ba67076f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:48.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.448+0000 7fa4e4a35740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ba66fcf1d0 con 0x55ba67069b40 2024-04-24T08:56:48.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.448+0000 7fa4e4a35740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ba66f75450 con 0x55ba67074b30 2024-04-24T08:56:48.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.448+0000 7fa4e4a35740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55ba66fcb190 con 0x55ba6706a460 2024-04-24T08:56:48.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.448+0000 7fa4d2000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba67069b40 0x55ba67069f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:48.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.449+0000 7fa4d2000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba67069b40 0x55ba67069f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53622/0 (socket says 172.21.15.83:53622) 2024-04-24T08:56:48.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.449+0000 7fa4d2000640 1 -- 172.21.15.83:0/999773425 learned_addr learned my addr 172.21.15.83:0/999773425 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:56:48.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.449+0000 7fa4daa00640 1 --2- 172.21.15.83:0/999773425 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ba6706a460 0x55ba670745f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:48.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.449+0000 7fa4d2000640 1 -- 172.21.15.83:0/999773425 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ba6706a460 msgr2=0x55ba670745f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:48.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.449+0000 7fa4d2000640 1 --2- 172.21.15.83:0/999773425 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ba6706a460 0x55ba670745f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:48.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.449+0000 7fa4d2000640 1 -- 172.21.15.83:0/999773425 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba67074b30 msgr2=0x55ba67076f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:48.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.449+0000 7fa4d2000640 1 --2- 172.21.15.83:0/999773425 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba67074b30 0x55ba67076f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:48.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.449+0000 7fa4d2000640 1 -- 172.21.15.83:0/999773425 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba67032310 con 0x55ba67069b40 2024-04-24T08:56:48.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.449+0000 7fa4d3e00640 1 --2- 172.21.15.83:0/345964509 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa4c00a3da0 0x7fa4c00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:48.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.450+0000 7fa4d2000640 1 --2- 172.21.15.83:0/999773425 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba67069b40 0x55ba67069f20 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc06b6d0 tx=0x7fa4c4009670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f566198e2e9b1c8c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:48.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.450+0000 7fa4d3e00640 1 -- 172.21.15.83:0/345964509 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7fa4d40d0dc0 con 0x7fa4dc19a6e0 2024-04-24T08:56:48.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.450+0000 7fa4daa00640 1 --2- 172.21.15.83:0/345964509 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa4c00a3da0 0x7fa4c00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:48.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.450+0000 7fa4cbe00640 1 -- 172.21.15.83:0/999773425 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4c4035040 con 0x55ba67069b40 2024-04-24T08:56:48.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.450+0000 7fa4cbe00640 1 -- 172.21.15.83:0/999773425 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa4c40316a0 con 0x55ba67069b40 2024-04-24T08:56:48.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.450+0000 7fa4cbe00640 1 -- 172.21.15.83:0/999773425 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4c40055a0 con 0x55ba67069b40 2024-04-24T08:56:48.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.451+0000 7fa4e4a35740 1 -- 172.21.15.83:0/999773425 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba67069b40 msgr2=0x55ba67069f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:48.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.451+0000 7fa4e4a35740 1 --2- 172.21.15.83:0/999773425 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba67069b40 0x55ba67069f20 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc06b6d0 tx=0x7fa4c4009670 comp rx=0 tx=0).stop 2024-04-24T08:56:48.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.451+0000 7fa4daa00640 1 --2- 172.21.15.83:0/345964509 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa4c00a3da0 0x7fa4c00a6260 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc005fd0 tx=0x7fa4cc005850 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:48.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.451+0000 7fa4e4a35740 1 -- 172.21.15.83:0/999773425 shutdown_connections 2024-04-24T08:56:48.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.451+0000 7fa4e4a35740 1 --2- 172.21.15.83:0/999773425 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba67074b30 0x55ba67076f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:48.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.451+0000 7fa4e4a35740 1 --2- 172.21.15.83:0/999773425 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ba6706a460 0x55ba670745f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:48.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.451+0000 7fa4e4a35740 1 --2- 172.21.15.83:0/999773425 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba67069b40 0x55ba67069f20 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:48.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.451+0000 7fa4e4a35740 1 -- 172.21.15.83:0/999773425 >> 172.21.15.83:0/999773425 conn(0x55ba67065e60 msgr2=0x55ba67066240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:48.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.451+0000 7fa4e4a35740 1 -- 172.21.15.83:0/999773425 shutdown_connections 2024-04-24T08:56:48.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.451+0000 7fa4e4a35740 1 -- 172.21.15.83:0/999773425 wait complete. 2024-04-24T08:56:48.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.452+0000 7fa4e4a35740 1 -- start start 2024-04-24T08:56:48.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.452+0000 7fa4e4a35740 1 Processor -- start 2024-04-24T08:56:48.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.452+0000 7fa4e4a35740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba67069b40 0x55ba67071910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:48.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.452+0000 7fa4e4a35740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba6706a460 0x55ba67071e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:48.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.452+0000 7fa4e4a35740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ba67074b30 0x55ba67072390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:48.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.452+0000 7fa4e4a35740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ba67077960 con 0x55ba6706a460 2024-04-24T08:56:48.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.452+0000 7fa4e4a35740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ba66f88810 con 0x55ba67069b40 2024-04-24T08:56:48.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.452+0000 7fa4e4a35740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55ba67077ae0 con 0x55ba67074b30 2024-04-24T08:56:48.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.452+0000 7fa4e4a35740 10 client.0 _ll_register_callbacks cb 0x55ba66f56140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:56:48.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.453+0000 7fa4d2000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba6706a460 0x55ba67071e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:48.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.453+0000 7fa4d2000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba6706a460 0x55ba67071e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53632/0 (socket says 172.21.15.83:53632) 2024-04-24T08:56:48.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.453+0000 7fa4d2000640 1 -- 172.21.15.83:0/2879093495 learned_addr learned my addr 172.21.15.83:0/2879093495 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:56:48.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.453+0000 7fa4daa00640 1 --2- 172.21.15.83:0/2879093495 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba67069b40 0x55ba67071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:48.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.453+0000 7fa4db400640 1 --2- 172.21.15.83:0/2879093495 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ba67074b30 0x55ba67072390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:48.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.453+0000 7fa4d2000640 1 -- 172.21.15.83:0/2879093495 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ba67074b30 msgr2=0x55ba67072390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:48.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.453+0000 7fa4d2000640 1 --2- 172.21.15.83:0/2879093495 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ba67074b30 0x55ba67072390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:48.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.453+0000 7fa4d2000640 1 -- 172.21.15.83:0/2879093495 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba67069b40 msgr2=0x55ba67071910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:48.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.453+0000 7fa4d2000640 1 --2- 172.21.15.83:0/2879093495 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba67069b40 0x55ba67071910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:48.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.453+0000 7fa4d2000640 1 -- 172.21.15.83:0/2879093495 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba66f993f0 con 0x55ba6706a460 2024-04-24T08:56:48.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.453+0000 7fa4d2000640 1 --2- 172.21.15.83:0/2879093495 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba6706a460 0x55ba67071e50 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7fa4c4031a60 tx=0x7fa4c4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68bf7d70448c4610 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:48.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.454+0000 7fa4c8c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:56:48.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.454+0000 7fa4c8c00640 1 -- 172.21.15.83:0/2879093495 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4c4035040 con 0x55ba6706a460 2024-04-24T08:56:48.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.454+0000 7fa4c8c00640 1 -- 172.21.15.83:0/2879093495 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7fa4c4004440 con 0x55ba6706a460 2024-04-24T08:56:48.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.454+0000 7fa4e4a35740 1 -- 172.21.15.83:0/2879093495 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4c40316a0 con 0x55ba6706a460 2024-04-24T08:56:48.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.454+0000 7fa4c8c00640 1 -- 172.21.15.83:0/2879093495 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fa4c4057050 con 0x55ba6706a460 2024-04-24T08:56:48.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.454+0000 7fa4e4a35740 10 client.17529 fetch_fsmap learned FSMap version 33 2024-04-24T08:56:48.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.454+0000 7fa4e4a35740 1 -- 172.21.15.83:0/2879093495 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55ba66ce13f0 con 0x55ba6706a460 2024-04-24T08:56:48.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.454+0000 7fa4c8c00640 1 -- 172.21.15.83:0/2879093495 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa4c40316a0 con 0x55ba6706a460 2024-04-24T08:56:48.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.455+0000 7fa4c8c00640 1 -- 172.21.15.83:0/2879093495 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fa4c403a040 con 0x55ba6706a460 2024-04-24T08:56:48.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.455+0000 7fa4e4a35740 10 client.17529 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T08:56:48.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.455+0000 7fa4e4a35740 10 client.17529 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-24T08:56:48.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.455+0000 7fa4e4a35740 20 client.17529 populate_metadata read hostname 'smithi083' 2024-04-24T08:56:48.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.455+0000 7fa4e4a35740 1 --2- 172.21.15.83:0/2879093495 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x55ba67084c00 0x55ba67086ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:48.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.455+0000 7fa4e4a35740 4 client.17529 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:56:48.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.455+0000 7fa4e4a35740 1 -- 172.21.15.83:0/2879093495 --> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ba66fdcbf0 con 0x55ba67084c00 2024-04-24T08:56:48.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.456+0000 7fa4daa00640 1 --2- 172.21.15.83:0/2879093495 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x55ba67084c00 0x55ba67086ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:48.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.457+0000 7fa4c8c00640 10 client.17529 ms_handle_connect on v2:172.21.15.138:6836/900566754 2024-04-24T08:56:48.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.457+0000 7fa4daa00640 1 --2- 172.21.15.83:0/2879093495 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x55ba67084c00 0x55ba67086ff0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2d25d5eee969d5b2 server_cookie=c6db3d2aabafca40 in_seq=0 out_seq=0 2024-04-24T08:56:48.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.460+0000 7fa4c8c00640 1 -- 172.21.15.83:0/2879093495 <== mds.1 v2:172.21.15.138:6836/900566754 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ba66fdcbf0 con 0x55ba67084c00 2024-04-24T08:56:48.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.460+0000 7fa4c8c00640 10 client.17529 handle_command_reply: tid=0 2024-04-24T08:56:48.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.460+0000 7fa4e4a35740 1 client.17529 shutdown 2024-04-24T08:56:48.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.460+0000 7fa4e4a35740 1 -- 172.21.15.83:0/2879093495 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba6706a460 msgr2=0x55ba67071e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:48.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.460+0000 7fa4e4a35740 1 --2- 172.21.15.83:0/2879093495 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba6706a460 0x55ba67071e50 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7fa4c4031a60 tx=0x7fa4c4000bc0 comp rx=0 tx=0).stop 2024-04-24T08:56:48.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.460+0000 7fa4e4a35740 1 -- 172.21.15.83:0/2879093495 shutdown_connections 2024-04-24T08:56:48.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.460+0000 7fa4e4a35740 1 --2- 172.21.15.83:0/2879093495 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x55ba67084c00 0x55ba67086ff0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:48.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.460+0000 7fa4e4a35740 1 --2- 172.21.15.83:0/2879093495 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ba67074b30 0x55ba67072390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:48.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.460+0000 7fa4e4a35740 1 --2- 172.21.15.83:0/2879093495 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba6706a460 0x55ba67071e50 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:48.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.460+0000 7fa4e4a35740 1 --2- 172.21.15.83:0/2879093495 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba67069b40 0x55ba67071910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:48.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.460+0000 7fa4c8c00640 0 client.17529 ms_handle_reset on v2:172.21.15.138:6836/900566754 2024-04-24T08:56:48.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.460+0000 7fa4e4a35740 1 -- 172.21.15.83:0/2879093495 >> 172.21.15.83:0/2879093495 conn(0x55ba67065e60 msgr2=0x55ba6706bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:48.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.460+0000 7fa4e4a35740 1 -- 172.21.15.83:0/2879093495 shutdown_connections 2024-04-24T08:56:48.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.460+0000 7fa4e4a35740 1 -- 172.21.15.83:0/2879093495 wait complete. 2024-04-24T08:56:48.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.460+0000 7fa4e4a35740 20 client.17529 trim_cache size 0 max 16384 2024-04-24T08:56:48.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.476+0000 7fa4e4a35740 1 Processor -- start 2024-04-24T08:56:48.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.476+0000 7fa4e4a35740 1 -- start start 2024-04-24T08:56:48.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.476+0000 7fa4e4a35740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba67069b40 0x55ba6706c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:48.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.476+0000 7fa4e4a35740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba6706a460 0x55ba670721b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:48.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.476+0000 7fa4e4a35740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ba67074b30 0x55ba6708c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:48.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.476+0000 7fa4e4a35740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa4dc1b13e0 con 0x55ba6706a460 2024-04-24T08:56:48.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.476+0000 7fa4e4a35740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa4dc1b2f20 con 0x55ba67069b40 2024-04-24T08:56:48.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.476+0000 7fa4e4a35740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa4dc1b1ab0 con 0x55ba67074b30 2024-04-24T08:56:48.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.477+0000 7fa4d2000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba6706a460 0x55ba670721b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:48.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.477+0000 7fa4d2000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba6706a460 0x55ba670721b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53634/0 (socket says 172.21.15.83:53634) 2024-04-24T08:56:48.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.477+0000 7fa4d2000640 1 -- 172.21.15.83:0/1256319717 learned_addr learned my addr 172.21.15.83:0/1256319717 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:56:48.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.477+0000 7fa4daa00640 1 --2- 172.21.15.83:0/1256319717 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba67069b40 0x55ba6706c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:48.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.477+0000 7fa4d2000640 1 -- 172.21.15.83:0/1256319717 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ba67074b30 msgr2=0x55ba6708c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:48.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.477+0000 7fa4db400640 1 --2- 172.21.15.83:0/1256319717 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ba67074b30 0x55ba6708c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:48.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.477+0000 7fa4d2000640 1 --2- 172.21.15.83:0/1256319717 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ba67074b30 0x55ba6708c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:48.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.477+0000 7fa4d2000640 1 -- 172.21.15.83:0/1256319717 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba67069b40 msgr2=0x55ba6706c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:48.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.477+0000 7fa4d2000640 1 --2- 172.21.15.83:0/1256319717 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba67069b40 0x55ba6706c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:48.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.477+0000 7fa4d2000640 1 -- 172.21.15.83:0/1256319717 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba67087360 con 0x55ba6706a460 2024-04-24T08:56:48.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.477+0000 7fa4daa00640 1 --2- 172.21.15.83:0/1256319717 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba67069b40 0x55ba6706c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:56:48.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.477+0000 7fa4db400640 1 --2- 172.21.15.83:0/1256319717 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ba67074b30 0x55ba6708c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:56:48.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.478+0000 7fa4d2000640 1 --2- 172.21.15.83:0/1256319717 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba6706a460 0x55ba670721b0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7fa4c403d8b0 tx=0x7fa4c4005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ef015274c39b54e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:48.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.478+0000 7fa4cb400640 1 -- 172.21.15.83:0/1256319717 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4c4035070 con 0x55ba6706a460 2024-04-24T08:56:48.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.478+0000 7fa4cb400640 1 -- 172.21.15.83:0/1256319717 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa4c404e070 con 0x55ba6706a460 2024-04-24T08:56:48.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.478+0000 7fa4cb400640 1 -- 172.21.15.83:0/1256319717 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4c4004240 con 0x55ba6706a460 2024-04-24T08:56:48.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.479+0000 7fa4e4a35740 1 -- 172.21.15.83:0/1256319717 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba6706a460 msgr2=0x55ba670721b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:48.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.479+0000 7fa4e4a35740 1 --2- 172.21.15.83:0/1256319717 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba6706a460 0x55ba670721b0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7fa4c403d8b0 tx=0x7fa4c4005fd0 comp rx=0 tx=0).stop 2024-04-24T08:56:48.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.479+0000 7fa4e4a35740 1 -- 172.21.15.83:0/1256319717 shutdown_connections 2024-04-24T08:56:48.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.479+0000 7fa4e4a35740 1 --2- 172.21.15.83:0/1256319717 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ba67074b30 0x55ba6708c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:48.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.479+0000 7fa4e4a35740 1 --2- 172.21.15.83:0/1256319717 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba6706a460 0x55ba670721b0 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:48.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.479+0000 7fa4e4a35740 1 --2- 172.21.15.83:0/1256319717 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba67069b40 0x55ba6706c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:48.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.479+0000 7fa4e4a35740 1 -- 172.21.15.83:0/1256319717 >> 172.21.15.83:0/1256319717 conn(0x55ba67065e60 msgr2=0x55ba67064f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:48.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.479+0000 7fa4e4a35740 1 -- 172.21.15.83:0/1256319717 shutdown_connections 2024-04-24T08:56:48.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.479+0000 7fa4e4a35740 1 -- 172.21.15.83:0/1256319717 wait complete. 2024-04-24T08:56:48.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.479+0000 7fa4e4a35740 1 -- start start 2024-04-24T08:56:48.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.479+0000 7fa4e4a35740 1 Processor -- start 2024-04-24T08:56:48.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.480+0000 7fa4e4a35740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba67069b40 0x55ba6707d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:48.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.480+0000 7fa4e4a35740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba6706a460 0x55ba670757a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:48.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.480+0000 7fa4e4a35740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ba67074b30 0x55ba67075ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:48.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.480+0000 7fa4e4a35740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa4dc1b2850 con 0x55ba6706a460 2024-04-24T08:56:48.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.480+0000 7fa4e4a35740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa4dc1b2180 con 0x55ba67069b40 2024-04-24T08:56:48.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.480+0000 7fa4e4a35740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa4dc1af3c0 con 0x55ba67074b30 2024-04-24T08:56:48.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.480+0000 7fa4e4a35740 10 client.0 _ll_register_callbacks cb 0x7fa4c4032b90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:56:48.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.480+0000 7fa4d2000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba6706a460 0x55ba670757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:48.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.480+0000 7fa4d2000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba6706a460 0x55ba670757a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53640/0 (socket says 172.21.15.83:53640) 2024-04-24T08:56:48.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.480+0000 7fa4d2000640 1 -- 172.21.15.83:0/1596246211 learned_addr learned my addr 172.21.15.83:0/1596246211 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:56:48.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.480+0000 7fa4d2000640 1 -- 172.21.15.83:0/1596246211 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ba67074b30 msgr2=0x55ba67075ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:48.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.480+0000 7fa4d2000640 1 --2- 172.21.15.83:0/1596246211 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ba67074b30 0x55ba67075ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:48.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.480+0000 7fa4d2000640 1 -- 172.21.15.83:0/1596246211 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba67069b40 msgr2=0x55ba6707d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:48.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.480+0000 7fa4d2000640 1 --2- 172.21.15.83:0/1596246211 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba67069b40 0x55ba6707d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:48.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.480+0000 7fa4d2000640 1 -- 172.21.15.83:0/1596246211 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba66dab7d0 con 0x55ba6706a460 2024-04-24T08:56:48.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.481+0000 7fa4d2000640 1 --2- 172.21.15.83:0/1596246211 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba6706a460 0x55ba670757a0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fa4c4009900 tx=0x7fa4c4032e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94bc0fd23a77ae75 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:48.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.481+0000 7fa4c8c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:56:48.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.481+0000 7fa4c8c00640 1 -- 172.21.15.83:0/1596246211 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4c4035070 con 0x55ba6706a460 2024-04-24T08:56:48.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.481+0000 7fa4c8c00640 1 -- 172.21.15.83:0/1596246211 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7fa4c4004240 con 0x55ba6706a460 2024-04-24T08:56:48.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.481+0000 7fa4e4a35740 1 -- 172.21.15.83:0/1596246211 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4c404e070 con 0x55ba6706a460 2024-04-24T08:56:48.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.481+0000 7fa4c8c00640 1 -- 172.21.15.83:0/1596246211 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fa4c403dd20 con 0x55ba6706a460 2024-04-24T08:56:48.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.481+0000 7fa4e4a35740 10 client.17535 fetch_fsmap learned FSMap version 33 2024-04-24T08:56:48.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.481+0000 7fa4e4a35740 1 -- 172.21.15.83:0/1596246211 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55ba670eae60 con 0x55ba6706a460 2024-04-24T08:56:48.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.482+0000 7fa4c8c00640 1 -- 172.21.15.83:0/1596246211 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa4c404e070 con 0x55ba6706a460 2024-04-24T08:56:48.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.482+0000 7fa4c8c00640 1 -- 172.21.15.83:0/1596246211 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fa4c4003ba0 con 0x55ba6706a460 2024-04-24T08:56:48.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.482+0000 7fa4e4a35740 10 client.17535 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T08:56:48.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.482+0000 7fa4e4a35740 10 client.17535 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-24T08:56:48.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.482+0000 7fa4e4a35740 20 client.17535 populate_metadata read hostname 'smithi083' 2024-04-24T08:56:48.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.482+0000 7fa4e4a35740 1 --2- 172.21.15.83:0/1596246211 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x55ba670eb2a0 0x55ba670ed700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:48.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.482+0000 7fa4e4a35740 4 client.17535 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T08:56:48.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.482+0000 7fa4e4a35740 1 -- 172.21.15.83:0/1596246211 --> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ba66fd6a60 con 0x55ba670eb2a0 2024-04-24T08:56:48.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.482+0000 7fa4daa00640 1 --2- 172.21.15.83:0/1596246211 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x55ba670eb2a0 0x55ba670ed700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:48.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.483+0000 7fa4daa00640 1 --2- 172.21.15.83:0/1596246211 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x55ba670eb2a0 0x55ba670ed700 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c17cc1ed454fac77 server_cookie=40290f1f0e9c845f in_seq=0 out_seq=0 2024-04-24T08:56:48.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.483+0000 7fa4c8c00640 10 client.17535 ms_handle_connect on v2:172.21.15.138:6836/900566754 2024-04-24T08:56:48.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.484+0000 7fa4c8c00640 1 -- 172.21.15.83:0/1596246211 <== mds.1 v2:172.21.15.138:6836/900566754 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ba66fd6a60 con 0x55ba670eb2a0 2024-04-24T08:56:48.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.484+0000 7fa4c8c00640 10 client.17535 handle_command_reply: tid=0 2024-04-24T08:56:48.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.484+0000 7fa4e4a35740 1 client.17535 shutdown 2024-04-24T08:56:48.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.484+0000 7fa4e4a35740 1 -- 172.21.15.83:0/1596246211 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba6706a460 msgr2=0x55ba670757a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:48.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.484+0000 7fa4e4a35740 1 --2- 172.21.15.83:0/1596246211 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba6706a460 0x55ba670757a0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fa4c4009900 tx=0x7fa4c4032e80 comp rx=0 tx=0).stop 2024-04-24T08:56:48.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.484+0000 7fa4e4a35740 1 -- 172.21.15.83:0/1596246211 shutdown_connections 2024-04-24T08:56:48.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.484+0000 7fa4e4a35740 1 --2- 172.21.15.83:0/1596246211 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x55ba670eb2a0 0x55ba670ed700 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:48.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.484+0000 7fa4e4a35740 1 --2- 172.21.15.83:0/1596246211 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ba67074b30 0x55ba67075ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:48.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.484+0000 7fa4e4a35740 1 --2- 172.21.15.83:0/1596246211 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba6706a460 0x55ba670757a0 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:48.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.484+0000 7fa4c8c00640 0 client.17535 ms_handle_reset on v2:172.21.15.138:6836/900566754 2024-04-24T08:56:48.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.484+0000 7fa4e4a35740 1 --2- 172.21.15.83:0/1596246211 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba67069b40 0x55ba6707d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:48.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.484+0000 7fa4e4a35740 1 -- 172.21.15.83:0/1596246211 >> 172.21.15.83:0/1596246211 conn(0x55ba67065e60 msgr2=0x55ba67073140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:48.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.484+0000 7fa4e4a35740 1 -- 172.21.15.83:0/1596246211 shutdown_connections 2024-04-24T08:56:48.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.484+0000 7fa4e4a35740 1 -- 172.21.15.83:0/1596246211 wait complete. 2024-04-24T08:56:48.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.485+0000 7fa4e4a35740 20 client.17535 trim_cache size 0 max 16384 2024-04-24T08:56:48.487 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T08:56:48.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.485+0000 7fa4dbe00640 1 -- 172.21.15.83:0/345964509 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa4c00a3da0 msgr2=0x7fa4c00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:48.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.485+0000 7fa4dbe00640 1 --2- 172.21.15.83:0/345964509 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa4c00a3da0 0x7fa4c00a6260 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc005fd0 tx=0x7fa4cc005850 comp rx=0 tx=0).stop 2024-04-24T08:56:48.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.485+0000 7fa4dbe00640 1 -- 172.21.15.83:0/345964509 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4dc19a6e0 msgr2=0x7fa4dc19ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:48.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.485+0000 7fa4dbe00640 1 --2- 172.21.15.83:0/345964509 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4dc19a6e0 0x7fa4dc19ab60 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fa4d400eaa0 tx=0x7fa4d400ef60 comp rx=0 tx=0).stop 2024-04-24T08:56:48.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.485+0000 7fa4dbe00640 1 -- 172.21.15.83:0/345964509 shutdown_connections 2024-04-24T08:56:48.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.485+0000 7fa4dbe00640 1 --2- 172.21.15.83:0/345964509 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa4c00a3da0 0x7fa4c00a6260 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:48.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.485+0000 7fa4dbe00640 1 --2- 172.21.15.83:0/345964509 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4dc19a6e0 0x7fa4dc19ab60 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:48.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.485+0000 7fa4dbe00640 1 --2- 172.21.15.83:0/345964509 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa4dc1093e0 0x7fa4dc1a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:48.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.485+0000 7fa4dbe00640 1 --2- 172.21.15.83:0/345964509 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4dc104310 0x7fa4dc1a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:48.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.485+0000 7fa4dbe00640 1 -- 172.21.15.83:0/345964509 >> 172.21.15.83:0/345964509 conn(0x7fa4dc0ffb10 msgr2=0x7fa4dc100ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:48.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.485+0000 7fa4dbe00640 1 -- 172.21.15.83:0/345964509 shutdown_connections 2024-04-24T08:56:48.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.485+0000 7fa4dbe00640 1 -- 172.21.15.83:0/345964509 wait complete. 2024-04-24T08:56:48.501 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T08:56:48.576 INFO:tasks.workunit.client.0.smithi083.stdout: 1 24852 0.68 MB/sec execute 82 sec latency 45.189 ms 2024-04-24T08:56:48.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.710+0000 7f457b200640 1 -- 172.21.15.83:0/963050266 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f457c102110 msgr2=0x7f457c1024f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:48.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.710+0000 7f457b200640 1 --2- 172.21.15.83:0/963050266 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f457c102110 0x7f457c1024f0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f456c009900 tx=0x7f456c02a400 comp rx=0 tx=0).stop 2024-04-24T08:56:48.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.710+0000 7f457b200640 1 -- 172.21.15.83:0/963050266 shutdown_connections 2024-04-24T08:56:48.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.710+0000 7f457b200640 1 --2- 172.21.15.83:0/963050266 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f457c1071e0 0x7f457c10edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:48.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.710+0000 7f457b200640 1 --2- 172.21.15.83:0/963050266 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f457c102ac0 0x7f457c106b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:48.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.710+0000 7f457b200640 1 --2- 172.21.15.83:0/963050266 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f457c102110 0x7f457c1024f0 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:48.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.710+0000 7f457b200640 1 -- 172.21.15.83:0/963050266 >> 172.21.15.83:0/963050266 conn(0x7f457c0fd9d0 msgr2=0x7f457c0ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:48.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.710+0000 7f457b200640 1 -- 172.21.15.83:0/963050266 shutdown_connections 2024-04-24T08:56:48.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.710+0000 7f457b200640 1 -- 172.21.15.83:0/963050266 wait complete. 2024-04-24T08:56:48.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.711+0000 7f457b200640 1 Processor -- start 2024-04-24T08:56:48.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.711+0000 7f457b200640 1 -- start start 2024-04-24T08:56:48.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.712+0000 7f457b200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f457c102110 0x7f457c198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:48.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.712+0000 7f457b200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f457c1071e0 0x7f457c198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:48.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.712+0000 7f457b200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f457c19cf30 0x7f457c19d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:48.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.712+0000 7f457b200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f457c112170 con 0x7f457c19cf30 2024-04-24T08:56:48.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.712+0000 7f457b200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f457c111ff0 con 0x7f457c102110 2024-04-24T08:56:48.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.712+0000 7f457b200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f457c1122f0 con 0x7f457c1071e0 2024-04-24T08:56:48.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.712+0000 7f457a800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f457c19cf30 0x7f457c19d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:48.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.712+0000 7f457a800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f457c19cf30 0x7f457c19d3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53656/0 (socket says 172.21.15.83:53656) 2024-04-24T08:56:48.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.712+0000 7f457a800640 1 -- 172.21.15.83:0/4192892993 learned_addr learned my addr 172.21.15.83:0/4192892993 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:56:48.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.712+0000 7f4579e00640 1 --2- 172.21.15.83:0/4192892993 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f457c102110 0x7f457c198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:48.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.712+0000 7f4573e00640 1 --2- 172.21.15.83:0/4192892993 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f457c1071e0 0x7f457c198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:48.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.712+0000 7f457a800640 1 -- 172.21.15.83:0/4192892993 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f457c1071e0 msgr2=0x7f457c198bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:48.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.712+0000 7f457a800640 1 --2- 172.21.15.83:0/4192892993 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f457c1071e0 0x7f457c198bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:48.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.712+0000 7f457a800640 1 -- 172.21.15.83:0/4192892993 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f457c102110 msgr2=0x7f457c198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:48.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.712+0000 7f457a800640 1 --2- 172.21.15.83:0/4192892993 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f457c102110 0x7f457c198670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:48.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.713+0000 7f457a800640 1 -- 172.21.15.83:0/4192892993 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f457c19da20 con 0x7f457c19cf30 2024-04-24T08:56:48.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.713+0000 7f457a800640 1 --2- 172.21.15.83:0/4192892993 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f457c19cf30 0x7f457c19d3e0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f457400cce0 tx=0x7f457400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:48.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.713+0000 7f4572a00640 1 -- 172.21.15.83:0/4192892993 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f457400f450 con 0x7f457c19cf30 2024-04-24T08:56:48.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.713+0000 7f4572a00640 1 -- 172.21.15.83:0/4192892993 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f457400faa0 con 0x7f457c19cf30 2024-04-24T08:56:48.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.713+0000 7f457b200640 1 -- 172.21.15.83:0/4192892993 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f457c19dd10 con 0x7f457c19cf30 2024-04-24T08:56:48.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.713+0000 7f4572a00640 1 -- 172.21.15.83:0/4192892993 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4574018b70 con 0x7f457c19cf30 2024-04-24T08:56:48.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.713+0000 7f457b200640 1 -- 172.21.15.83:0/4192892993 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f457c1a54b0 con 0x7f457c19cf30 2024-04-24T08:56:48.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.718+0000 7f4583b6c740 1 Processor -- start 2024-04-24T08:56:48.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.718+0000 7f4572a00640 1 -- 172.21.15.83:0/4192892993 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4574018d10 con 0x7f457c19cf30 2024-04-24T08:56:48.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.718+0000 7f4583b6c740 1 -- start start 2024-04-24T08:56:48.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.718+0000 7f4583b6c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622cee68b40 0x5622cee68f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:48.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.718+0000 7f4583b6c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622cee69460 0x5622cee735f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:48.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.718+0000 7f4583b6c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622cee73b30 0x5622cee75f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:48.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.718+0000 7f4583b6c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5622cedce1d0 con 0x5622cee68b40 2024-04-24T08:56:48.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.718+0000 7f4583b6c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5622ced74450 con 0x5622cee73b30 2024-04-24T08:56:48.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.718+0000 7f4583b6c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5622cedca190 con 0x5622cee69460 2024-04-24T08:56:48.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.719+0000 7f4573e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622cee68b40 0x5622cee68f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:48.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.719+0000 7f4573e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622cee68b40 0x5622cee68f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53672/0 (socket says 172.21.15.83:53672) 2024-04-24T08:56:48.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.719+0000 7f4573e00640 1 -- 172.21.15.83:0/3885503625 learned_addr learned my addr 172.21.15.83:0/3885503625 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:56:48.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.719+0000 7f4579e00640 1 --2- 172.21.15.83:0/3885503625 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622cee69460 0x5622cee735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:48.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.719+0000 7f4573e00640 1 --2- 172.21.15.83:0/3885503625 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622cee73b30 0x5622cee75f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:48.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.719+0000 7f4573e00640 1 -- 172.21.15.83:0/3885503625 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622cee69460 msgr2=0x5622cee735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:48.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.719+0000 7f4573e00640 1 --2- 172.21.15.83:0/3885503625 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622cee69460 0x5622cee735f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:48.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.719+0000 7f4573e00640 1 -- 172.21.15.83:0/3885503625 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622cee73b30 msgr2=0x5622cee75f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:48.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.719+0000 7f4573e00640 1 --2- 172.21.15.83:0/3885503625 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622cee73b30 0x5622cee75f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:48.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.719+0000 7f4573e00640 1 -- 172.21.15.83:0/3885503625 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622cee31310 con 0x5622cee68b40 2024-04-24T08:56:48.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.720+0000 7f4573e00640 1 --2- 172.21.15.83:0/3885503625 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622cee68b40 0x5622cee68f20 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f456400bed0 tx=0x7f456400b650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6ad8b4d9c76bc11 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:48.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.720+0000 7f4572a00640 1 --2- 172.21.15.83:0/4192892993 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f45600a3d20 0x7f45600a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:48.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.720+0000 7f456b400640 1 -- 172.21.15.83:0/3885503625 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f456400f3f0 con 0x5622cee68b40 2024-04-24T08:56:48.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.720+0000 7f4572a00640 1 -- 172.21.15.83:0/4192892993 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f45740d1080 con 0x7f457c19cf30 2024-04-24T08:56:48.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.720+0000 7f456b400640 1 -- 172.21.15.83:0/3885503625 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f456400fa40 con 0x5622cee68b40 2024-04-24T08:56:48.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.720+0000 7f456b400640 1 -- 172.21.15.83:0/3885503625 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4564012750 con 0x5622cee68b40 2024-04-24T08:56:48.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.720+0000 7f4579e00640 1 --2- 172.21.15.83:0/4192892993 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f45600a3d20 0x7f45600a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:48.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.721+0000 7f4583b6c740 1 -- 172.21.15.83:0/3885503625 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622cee68b40 msgr2=0x5622cee68f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:48.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.721+0000 7f4583b6c740 1 --2- 172.21.15.83:0/3885503625 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622cee68b40 0x5622cee68f20 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f456400bed0 tx=0x7f456400b650 comp rx=0 tx=0).stop 2024-04-24T08:56:48.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.721+0000 7f4579e00640 1 --2- 172.21.15.83:0/4192892993 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f45600a3d20 0x7f45600a61e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f456c000c00 tx=0x7f456c005a20 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:48.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.721+0000 7f4583b6c740 1 -- 172.21.15.83:0/3885503625 shutdown_connections 2024-04-24T08:56:48.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.721+0000 7f4583b6c740 1 --2- 172.21.15.83:0/3885503625 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622cee73b30 0x5622cee75f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:48.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.721+0000 7f4583b6c740 1 --2- 172.21.15.83:0/3885503625 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622cee69460 0x5622cee735f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:48.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.721+0000 7f4583b6c740 1 --2- 172.21.15.83:0/3885503625 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622cee68b40 0x5622cee68f20 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:48.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.721+0000 7f4583b6c740 1 -- 172.21.15.83:0/3885503625 >> 172.21.15.83:0/3885503625 conn(0x5622cee64e60 msgr2=0x5622cee65240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:48.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.721+0000 7f4583b6c740 1 -- 172.21.15.83:0/3885503625 shutdown_connections 2024-04-24T08:56:48.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.721+0000 7f4583b6c740 1 -- 172.21.15.83:0/3885503625 wait complete. 2024-04-24T08:56:48.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.721+0000 7f4583b6c740 1 -- start start 2024-04-24T08:56:48.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.722+0000 7f4583b6c740 1 Processor -- start 2024-04-24T08:56:48.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.722+0000 7f4583b6c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622cee68b40 0x5622cee73340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:48.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.722+0000 7f4583b6c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622cee69460 0x5622cee70890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:48.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.722+0000 7f4583b6c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622cee73b30 0x5622cee70dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:48.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.722+0000 7f4583b6c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5622cee76960 con 0x5622cee73b30 2024-04-24T08:56:48.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.722+0000 7f4583b6c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5622ced87810 con 0x5622cee69460 2024-04-24T08:56:48.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.722+0000 7f4583b6c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5622cee76ae0 con 0x5622cee68b40 2024-04-24T08:56:48.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.723+0000 7f4583b6c740 10 client.0 _ll_register_callbacks cb 0x5622ced55140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:56:48.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.723+0000 7f457a800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622cee73b30 0x5622cee70dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:48.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.723+0000 7f457a800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622cee73b30 0x5622cee70dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53676/0 (socket says 172.21.15.83:53676) 2024-04-24T08:56:48.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.723+0000 7f457a800640 1 -- 172.21.15.83:0/2558251316 learned_addr learned my addr 172.21.15.83:0/2558251316 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:56:48.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.723+0000 7f4573e00640 1 --2- 172.21.15.83:0/2558251316 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622cee69460 0x5622cee70890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:48.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.723+0000 7f4579e00640 1 --2- 172.21.15.83:0/2558251316 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622cee68b40 0x5622cee73340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:48.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.723+0000 7f457a800640 1 -- 172.21.15.83:0/2558251316 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622cee68b40 msgr2=0x5622cee73340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:48.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.723+0000 7f457a800640 1 --2- 172.21.15.83:0/2558251316 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622cee68b40 0x5622cee73340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:48.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.723+0000 7f457a800640 1 -- 172.21.15.83:0/2558251316 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622cee69460 msgr2=0x5622cee70890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:48.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.723+0000 7f457a800640 1 --2- 172.21.15.83:0/2558251316 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622cee69460 0x5622cee70890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:48.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.723+0000 7f457a800640 1 -- 172.21.15.83:0/2558251316 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622ced983f0 con 0x5622cee73b30 2024-04-24T08:56:48.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.724+0000 7f457a800640 1 --2- 172.21.15.83:0/2558251316 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622cee73b30 0x5622cee70dd0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f45740990b0 tx=0x7f457409c670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee4486b970c3a4a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:48.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.724+0000 7f455fe00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:56:48.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.724+0000 7f455fe00640 1 -- 172.21.15.83:0/2558251316 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45740b6060 con 0x5622cee73b30 2024-04-24T08:56:48.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.724+0000 7f455fe00640 1 -- 172.21.15.83:0/2558251316 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f457409ceb0 con 0x5622cee73b30 2024-04-24T08:56:48.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.724+0000 7f4583b6c740 1 -- 172.21.15.83:0/2558251316 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f456400fa40 con 0x5622cee73b30 2024-04-24T08:56:48.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.724+0000 7f455fe00640 1 -- 172.21.15.83:0/2558251316 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f45740bd2b0 con 0x5622cee73b30 2024-04-24T08:56:48.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.724+0000 7f4583b6c740 10 client.17547 fetch_fsmap learned FSMap version 33 2024-04-24T08:56:48.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.724+0000 7f4583b6c740 1 -- 172.21.15.83:0/2558251316 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5622ceae03f0 con 0x5622cee73b30 2024-04-24T08:56:48.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.724+0000 7f455fe00640 1 -- 172.21.15.83:0/2558251316 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f456400fa40 con 0x5622cee73b30 2024-04-24T08:56:48.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.724+0000 7f455fe00640 1 -- 172.21.15.83:0/2558251316 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f4574061bd0 con 0x5622cee73b30 2024-04-24T08:56:48.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.725+0000 7f4583b6c740 10 client.17547 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T08:56:48.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.725+0000 7f4583b6c740 10 client.17547 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:56:48.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.725+0000 7f4583b6c740 20 client.17547 populate_metadata read hostname 'smithi083' 2024-04-24T08:56:48.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.725+0000 7f4583b6c740 1 --2- 172.21.15.83:0/2558251316 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5622cee8c540 0x5622cee8e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:48.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.725+0000 7f4583b6c740 4 client.17547 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:56:48.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.725+0000 7f4583b6c740 1 -- 172.21.15.83:0/2558251316 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5622ceddbbf0 con 0x5622cee8c540 2024-04-24T08:56:48.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.725+0000 7f4579e00640 1 --2- 172.21.15.83:0/2558251316 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5622cee8c540 0x5622cee8e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:48.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.727+0000 7f4579e00640 1 --2- 172.21.15.83:0/2558251316 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5622cee8c540 0x5622cee8e970 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b21bbcadadaf03a server_cookie=f9f261e7482f1fe in_seq=0 out_seq=0 2024-04-24T08:56:48.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.727+0000 7f455fe00640 10 client.17547 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:56:48.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.729+0000 7f455fe00640 1 -- 172.21.15.83:0/2558251316 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5622ceddbbf0 con 0x5622cee8c540 2024-04-24T08:56:48.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.729+0000 7f455fe00640 10 client.17547 handle_command_reply: tid=0 2024-04-24T08:56:48.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.729+0000 7f4583b6c740 1 client.17547 shutdown 2024-04-24T08:56:48.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.729+0000 7f4583b6c740 1 -- 172.21.15.83:0/2558251316 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622cee73b30 msgr2=0x5622cee70dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:48.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.729+0000 7f4583b6c740 1 --2- 172.21.15.83:0/2558251316 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622cee73b30 0x5622cee70dd0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f45740990b0 tx=0x7f457409c670 comp rx=0 tx=0).stop 2024-04-24T08:56:48.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.729+0000 7f4583b6c740 1 -- 172.21.15.83:0/2558251316 shutdown_connections 2024-04-24T08:56:48.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.729+0000 7f4583b6c740 1 --2- 172.21.15.83:0/2558251316 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5622cee8c540 0x5622cee8e970 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:48.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.729+0000 7f4583b6c740 1 --2- 172.21.15.83:0/2558251316 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622cee73b30 0x5622cee70dd0 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:48.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.729+0000 7f4583b6c740 1 --2- 172.21.15.83:0/2558251316 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622cee69460 0x5622cee70890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:48.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.729+0000 7f4583b6c740 1 --2- 172.21.15.83:0/2558251316 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622cee68b40 0x5622cee73340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:48.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.729+0000 7f455fe00640 0 client.17547 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:56:48.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.729+0000 7f4583b6c740 1 -- 172.21.15.83:0/2558251316 >> 172.21.15.83:0/2558251316 conn(0x5622cee64e60 msgr2=0x5622cee6ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:48.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.729+0000 7f4583b6c740 1 -- 172.21.15.83:0/2558251316 shutdown_connections 2024-04-24T08:56:48.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.729+0000 7f4583b6c740 1 -- 172.21.15.83:0/2558251316 wait complete. 2024-04-24T08:56:48.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.730+0000 7f4583b6c740 20 client.17547 trim_cache size 0 max 16384 2024-04-24T08:56:48.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.744+0000 7f4583b6c740 1 Processor -- start 2024-04-24T08:56:48.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.744+0000 7f4583b6c740 1 -- start start 2024-04-24T08:56:48.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.745+0000 7f4583b6c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622cee68b40 0x5622cee79d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:48.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.745+0000 7f4583b6c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622cee69460 0x5622cee730d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:48.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.745+0000 7f4583b6c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622cee73b30 0x5622cee94240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:48.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.745+0000 7f4583b6c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f457c1ad020 con 0x5622cee68b40 2024-04-24T08:56:48.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.745+0000 7f4583b6c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f457c1aeb60 con 0x5622cee73b30 2024-04-24T08:56:48.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.745+0000 7f4583b6c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f457c1ad6f0 con 0x5622cee69460 2024-04-24T08:56:48.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.745+0000 7f4579e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622cee68b40 0x5622cee79d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:48.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.745+0000 7f4579e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622cee68b40 0x5622cee79d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53686/0 (socket says 172.21.15.83:53686) 2024-04-24T08:56:48.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.745+0000 7f4579e00640 1 -- 172.21.15.83:0/1176752172 learned_addr learned my addr 172.21.15.83:0/1176752172 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:56:48.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.745+0000 7f4573e00640 1 --2- 172.21.15.83:0/1176752172 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622cee69460 0x5622cee730d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:48.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.745+0000 7f457a800640 1 --2- 172.21.15.83:0/1176752172 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622cee73b30 0x5622cee94240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:48.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.745+0000 7f4579e00640 1 -- 172.21.15.83:0/1176752172 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622cee69460 msgr2=0x5622cee730d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:48.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.745+0000 7f4579e00640 1 --2- 172.21.15.83:0/1176752172 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622cee69460 0x5622cee730d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:48.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.745+0000 7f4579e00640 1 -- 172.21.15.83:0/1176752172 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622cee73b30 msgr2=0x5622cee94240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:48.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.745+0000 7f4579e00640 1 --2- 172.21.15.83:0/1176752172 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622cee73b30 0x5622cee94240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:48.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.745+0000 7f4579e00640 1 -- 172.21.15.83:0/1176752172 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622cebaa7d0 con 0x5622cee68b40 2024-04-24T08:56:48.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.746+0000 7f4579e00640 1 --2- 172.21.15.83:0/1176752172 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622cee68b40 0x5622cee79d10 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f456c031440 tx=0x7f456c03d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b67d0d84be13bfd3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:48.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.746+0000 7f456aa00640 1 -- 172.21.15.83:0/1176752172 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f456c035050 con 0x5622cee68b40 2024-04-24T08:56:48.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.746+0000 7f456aa00640 1 -- 172.21.15.83:0/1176752172 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f456c050700 con 0x5622cee68b40 2024-04-24T08:56:48.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.746+0000 7f456aa00640 1 -- 172.21.15.83:0/1176752172 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f456c004d80 con 0x5622cee68b40 2024-04-24T08:56:48.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.747+0000 7f4583b6c740 1 -- 172.21.15.83:0/1176752172 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622cee68b40 msgr2=0x5622cee79d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:48.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.747+0000 7f4583b6c740 1 --2- 172.21.15.83:0/1176752172 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622cee68b40 0x5622cee79d10 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f456c031440 tx=0x7f456c03d000 comp rx=0 tx=0).stop 2024-04-24T08:56:48.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.747+0000 7f4583b6c740 1 -- 172.21.15.83:0/1176752172 shutdown_connections 2024-04-24T08:56:48.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.747+0000 7f4583b6c740 1 --2- 172.21.15.83:0/1176752172 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622cee73b30 0x5622cee94240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:48.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.747+0000 7f4583b6c740 1 --2- 172.21.15.83:0/1176752172 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622cee69460 0x5622cee730d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:48.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.747+0000 7f4583b6c740 1 --2- 172.21.15.83:0/1176752172 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622cee68b40 0x5622cee79d10 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:48.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.747+0000 7f4583b6c740 1 -- 172.21.15.83:0/1176752172 >> 172.21.15.83:0/1176752172 conn(0x5622cee64e60 msgr2=0x5622cee70850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:48.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.747+0000 7f4583b6c740 1 -- 172.21.15.83:0/1176752172 shutdown_connections 2024-04-24T08:56:48.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.747+0000 7f4583b6c740 1 -- 172.21.15.83:0/1176752172 wait complete. 2024-04-24T08:56:48.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.747+0000 7f4583b6c740 1 -- start start 2024-04-24T08:56:48.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.748+0000 7f4583b6c740 1 Processor -- start 2024-04-24T08:56:48.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.748+0000 7f4583b6c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622cee68b40 0x5622cee88bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:48.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.748+0000 7f4583b6c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622cee69460 0x5622cee89110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:48.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.748+0000 7f4583b6c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622cee73b30 0x5622cee83990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:48.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.748+0000 7f4583b6c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f457c1ae490 con 0x5622cee69460 2024-04-24T08:56:48.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.748+0000 7f4583b6c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f457c1addc0 con 0x5622cee73b30 2024-04-24T08:56:48.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.748+0000 7f4583b6c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f457c1ab000 con 0x5622cee68b40 2024-04-24T08:56:48.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.748+0000 7f4583b6c740 10 client.0 _ll_register_callbacks cb 0x7f45740077a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:56:48.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.749+0000 7f4573e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622cee69460 0x5622cee89110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:48.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.749+0000 7f4573e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622cee69460 0x5622cee89110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53696/0 (socket says 172.21.15.83:53696) 2024-04-24T08:56:48.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.749+0000 7f4573e00640 1 -- 172.21.15.83:0/2769692453 learned_addr learned my addr 172.21.15.83:0/2769692453 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:56:48.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.749+0000 7f4579e00640 1 --2- 172.21.15.83:0/2769692453 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622cee68b40 0x5622cee88bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:48.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.749+0000 7f457a800640 1 --2- 172.21.15.83:0/2769692453 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622cee73b30 0x5622cee83990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:48.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.749+0000 7f4573e00640 1 -- 172.21.15.83:0/2769692453 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622cee68b40 msgr2=0x5622cee88bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:48.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.749+0000 7f4573e00640 1 --2- 172.21.15.83:0/2769692453 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622cee68b40 0x5622cee88bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:48.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.749+0000 7f4573e00640 1 -- 172.21.15.83:0/2769692453 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622cee73b30 msgr2=0x5622cee83990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:48.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.749+0000 7f4573e00640 1 --2- 172.21.15.83:0/2769692453 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622cee73b30 0x5622cee83990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:48.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.749+0000 7f4573e00640 1 -- 172.21.15.83:0/2769692453 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622cee83ed0 con 0x5622cee69460 2024-04-24T08:56:48.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.749+0000 7f4573e00640 1 --2- 172.21.15.83:0/2769692453 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622cee69460 0x5622cee89110 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f4564005fd0 tx=0x7f456400b3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e2eb3137f2414ee server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:48.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.749+0000 7f455fe00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:56:48.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.750+0000 7f455fe00640 1 -- 172.21.15.83:0/2769692453 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4564005040 con 0x5622cee69460 2024-04-24T08:56:48.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.750+0000 7f455fe00640 1 -- 172.21.15.83:0/2769692453 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f45640051e0 con 0x5622cee69460 2024-04-24T08:56:48.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.750+0000 7f4583b6c740 1 -- 172.21.15.83:0/2769692453 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f456c050700 con 0x5622cee69460 2024-04-24T08:56:48.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.750+0000 7f455fe00640 1 -- 172.21.15.83:0/2769692453 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f45640053e0 con 0x5622cee69460 2024-04-24T08:56:48.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.750+0000 7f4583b6c740 10 client.17553 fetch_fsmap learned FSMap version 33 2024-04-24T08:56:48.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.750+0000 7f4583b6c740 1 -- 172.21.15.83:0/2769692453 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5622ceef3270 con 0x5622cee69460 2024-04-24T08:56:48.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.750+0000 7f455fe00640 1 -- 172.21.15.83:0/2769692453 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f456c050700 con 0x5622cee69460 2024-04-24T08:56:48.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.751+0000 7f455fe00640 1 -- 172.21.15.83:0/2769692453 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f456401c4b0 con 0x5622cee69460 2024-04-24T08:56:48.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.751+0000 7f4583b6c740 10 client.17553 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T08:56:48.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.751+0000 7f4583b6c740 10 client.17553 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:56:48.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.751+0000 7f4583b6c740 20 client.17553 populate_metadata read hostname 'smithi083' 2024-04-24T08:56:48.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.751+0000 7f4583b6c740 1 --2- 172.21.15.83:0/2769692453 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5622ceef3720 0x5622ceef5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:48.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.751+0000 7f4583b6c740 4 client.17553 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T08:56:48.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.751+0000 7f4583b6c740 1 -- 172.21.15.83:0/2769692453 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5622cedd5a60 con 0x5622ceef3720 2024-04-24T08:56:48.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.751+0000 7f4579e00640 1 --2- 172.21.15.83:0/2769692453 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5622ceef3720 0x5622ceef5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:48.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.752+0000 7f455fe00640 10 client.17553 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:56:48.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.752+0000 7f4579e00640 1 --2- 172.21.15.83:0/2769692453 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5622ceef3720 0x5622ceef5b40 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4beb2c7d49c0279a server_cookie=577aeb7c300a27d2 in_seq=0 out_seq=0 2024-04-24T08:56:48.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.753+0000 7f455fe00640 1 -- 172.21.15.83:0/2769692453 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5622cedd5a60 con 0x5622ceef3720 2024-04-24T08:56:48.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.753+0000 7f455fe00640 10 client.17553 handle_command_reply: tid=0 2024-04-24T08:56:48.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.753+0000 7f4583b6c740 1 client.17553 shutdown 2024-04-24T08:56:48.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.753+0000 7f4583b6c740 1 -- 172.21.15.83:0/2769692453 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622cee69460 msgr2=0x5622cee89110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:48.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.753+0000 7f4583b6c740 1 --2- 172.21.15.83:0/2769692453 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622cee69460 0x5622cee89110 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f4564005fd0 tx=0x7f456400b3a0 comp rx=0 tx=0).stop 2024-04-24T08:56:48.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.753+0000 7f4583b6c740 1 -- 172.21.15.83:0/2769692453 shutdown_connections 2024-04-24T08:56:48.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.753+0000 7f4583b6c740 1 --2- 172.21.15.83:0/2769692453 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5622ceef3720 0x5622ceef5b40 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:48.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.753+0000 7f4583b6c740 1 --2- 172.21.15.83:0/2769692453 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622cee73b30 0x5622cee83990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:48.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.753+0000 7f4583b6c740 1 --2- 172.21.15.83:0/2769692453 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622cee69460 0x5622cee89110 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:48.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.753+0000 7f4583b6c740 1 --2- 172.21.15.83:0/2769692453 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622cee68b40 0x5622cee88bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:48.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.753+0000 7f455fe00640 0 client.17553 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:56:48.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.753+0000 7f4583b6c740 1 -- 172.21.15.83:0/2769692453 >> 172.21.15.83:0/2769692453 conn(0x5622cee64e60 msgr2=0x5622cee636c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:48.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.753+0000 7f4583b6c740 1 -- 172.21.15.83:0/2769692453 shutdown_connections 2024-04-24T08:56:48.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.754+0000 7f4583b6c740 1 -- 172.21.15.83:0/2769692453 wait complete. 2024-04-24T08:56:48.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.754+0000 7f4583b6c740 20 client.17553 trim_cache size 0 max 16384 2024-04-24T08:56:48.756 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T08:56:48.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.754+0000 7f457b200640 1 -- 172.21.15.83:0/4192892993 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f45600a3d20 msgr2=0x7f45600a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:48.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.754+0000 7f457b200640 1 --2- 172.21.15.83:0/4192892993 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f45600a3d20 0x7f45600a61e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f456c000c00 tx=0x7f456c005a20 comp rx=0 tx=0).stop 2024-04-24T08:56:48.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.754+0000 7f457b200640 1 -- 172.21.15.83:0/4192892993 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f457c19cf30 msgr2=0x7f457c19d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:48.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.754+0000 7f457b200640 1 --2- 172.21.15.83:0/4192892993 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f457c19cf30 0x7f457c19d3e0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f457400cce0 tx=0x7f457400a620 comp rx=0 tx=0).stop 2024-04-24T08:56:48.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.754+0000 7f457b200640 1 -- 172.21.15.83:0/4192892993 shutdown_connections 2024-04-24T08:56:48.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.754+0000 7f457b200640 1 --2- 172.21.15.83:0/4192892993 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f45600a3d20 0x7f45600a61e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:48.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.754+0000 7f457b200640 1 --2- 172.21.15.83:0/4192892993 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f457c19cf30 0x7f457c19d3e0 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:48.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.754+0000 7f457b200640 1 --2- 172.21.15.83:0/4192892993 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f457c1071e0 0x7f457c198bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:48.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.754+0000 7f457b200640 1 --2- 172.21.15.83:0/4192892993 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f457c102110 0x7f457c198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:48.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.754+0000 7f457b200640 1 -- 172.21.15.83:0/4192892993 >> 172.21.15.83:0/4192892993 conn(0x7f457c0fd9d0 msgr2=0x7f457c0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:48.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.755+0000 7f457b200640 1 -- 172.21.15.83:0/4192892993 shutdown_connections 2024-04-24T08:56:48.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:48.755+0000 7f457b200640 1 -- 172.21.15.83:0/4192892993 wait complete. 2024-04-24T08:56:49.576 INFO:tasks.workunit.client.0.smithi083.stdout: 1 24981 0.68 MB/sec execute 83 sec latency 53.580 ms 2024-04-24T08:56:49.772 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T08:56:49.772 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T08:56:49.772 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T08:56:50.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.001+0000 7f28f5a00640 1 -- 172.21.15.83:0/1003261128 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f28f010e740 msgr2=0x7f28f0110b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:50.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.001+0000 7f28f5a00640 1 --2- 172.21.15.83:0/1003261128 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f28f010e740 0x7f28f0110b50 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f28e8009ca0 tx=0x7f28e802a950 comp rx=0 tx=0).stop 2024-04-24T08:56:50.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.001+0000 7f28f5a00640 1 -- 172.21.15.83:0/1003261128 shutdown_connections 2024-04-24T08:56:50.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.001+0000 7f28f5a00640 1 --2- 172.21.15.83:0/1003261128 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f28f010e740 0x7f28f0110b50 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:50.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.001+0000 7f28f5a00640 1 --2- 172.21.15.83:0/1003261128 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f28f0074cf0 0x7f28f010e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:50.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.001+0000 7f28f5a00640 1 --2- 172.21.15.83:0/1003261128 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f28f00743d0 0x7f28f00747b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:50.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.001+0000 7f28f5a00640 1 -- 172.21.15.83:0/1003261128 >> 172.21.15.83:0/1003261128 conn(0x7f28f00fd9d0 msgr2=0x7f28f00ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:50.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.001+0000 7f28f5a00640 1 -- 172.21.15.83:0/1003261128 shutdown_connections 2024-04-24T08:56:50.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.001+0000 7f28f5a00640 1 -- 172.21.15.83:0/1003261128 wait complete. 2024-04-24T08:56:50.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.002+0000 7f28f5a00640 1 Processor -- start 2024-04-24T08:56:50.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.003+0000 7f28f5a00640 1 -- start start 2024-04-24T08:56:50.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.003+0000 7f28f5a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f28f00743d0 0x7f28f01a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:50.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.003+0000 7f28f5a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f28f0074cf0 0x7f28f01a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:50.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.003+0000 7f28f5a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f28f010e740 0x7f28f019a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:50.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.003+0000 7f28f5a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f28f0114330 con 0x7f28f0074cf0 2024-04-24T08:56:50.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.003+0000 7f28f5a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f28f01141b0 con 0x7f28f010e740 2024-04-24T08:56:50.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.003+0000 7f28f5a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f28f01144b0 con 0x7f28f00743d0 2024-04-24T08:56:50.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.003+0000 7f28ef400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f28f0074cf0 0x7f28f01a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:50.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.003+0000 7f28ef400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f28f0074cf0 0x7f28f01a0b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40690/0 (socket says 172.21.15.83:40690) 2024-04-24T08:56:50.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.003+0000 7f28ef400640 1 -- 172.21.15.83:0/145480370 learned_addr learned my addr 172.21.15.83:0/145480370 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:56:50.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.004+0000 7f28efe00640 1 --2- 172.21.15.83:0/145480370 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f28f00743d0 0x7f28f01a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:50.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.004+0000 7f28f5000640 1 --2- 172.21.15.83:0/145480370 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f28f010e740 0x7f28f019a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:50.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.004+0000 7f28ef400640 1 -- 172.21.15.83:0/145480370 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f28f00743d0 msgr2=0x7f28f01a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:50.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.004+0000 7f28ef400640 1 --2- 172.21.15.83:0/145480370 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f28f00743d0 0x7f28f01a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:50.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.004+0000 7f28ef400640 1 -- 172.21.15.83:0/145480370 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f28f010e740 msgr2=0x7f28f019a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:50.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.004+0000 7f28ef400640 1 --2- 172.21.15.83:0/145480370 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f28f010e740 0x7f28f019a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:50.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.004+0000 7f28ef400640 1 -- 172.21.15.83:0/145480370 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28f019aff0 con 0x7f28f0074cf0 2024-04-24T08:56:50.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.004+0000 7f28ef400640 1 --2- 172.21.15.83:0/145480370 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f28f0074cf0 0x7f28f01a0b60 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f28dc00d770 tx=0x7f28dc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:50.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.004+0000 7f28ecc00640 1 -- 172.21.15.83:0/145480370 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28dc017070 con 0x7f28f0074cf0 2024-04-24T08:56:50.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.004+0000 7f28ecc00640 1 -- 172.21.15.83:0/145480370 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f28dc0054d0 con 0x7f28f0074cf0 2024-04-24T08:56:50.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.004+0000 7f28f5a00640 1 -- 172.21.15.83:0/145480370 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28f019b2e0 con 0x7f28f0074cf0 2024-04-24T08:56:50.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.005+0000 7f28ecc00640 1 -- 172.21.15.83:0/145480370 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28dc004cf0 con 0x7f28f0074cf0 2024-04-24T08:56:50.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.005+0000 7f28f5a00640 1 -- 172.21.15.83:0/145480370 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28f01a75a0 con 0x7f28f0074cf0 2024-04-24T08:56:50.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.008+0000 7f28f9f66740 1 Processor -- start 2024-04-24T08:56:50.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.008+0000 7f28f9f66740 1 -- start start 2024-04-24T08:56:50.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.009+0000 7f28f9f66740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a2c061a270 0x55a2c061a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:50.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.009+0000 7f28f9f66740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2c061ab90 0x55a2c0624d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:50.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.009+0000 7f28ecc00640 1 -- 172.21.15.83:0/145480370 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f28dc004e90 con 0x7f28f0074cf0 2024-04-24T08:56:50.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.009+0000 7f28f9f66740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2c0625260 0x55a2c0627650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:50.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.009+0000 7f28f9f66740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a2c057fe70 con 0x55a2c061ab90 2024-04-24T08:56:50.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.009+0000 7f28f9f66740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a2c0526380 con 0x55a2c0625260 2024-04-24T08:56:50.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.009+0000 7f28f9f66740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a2c057be30 con 0x55a2c061a270 2024-04-24T08:56:50.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.009+0000 7f28efe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2c061ab90 0x55a2c0624d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:50.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.009+0000 7f28efe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2c061ab90 0x55a2c0624d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40694/0 (socket says 172.21.15.83:40694) 2024-04-24T08:56:50.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.009+0000 7f28efe00640 1 -- 172.21.15.83:0/87587183 learned_addr learned my addr 172.21.15.83:0/87587183 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:56:50.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.009+0000 7f28ef400640 1 --2- 172.21.15.83:0/87587183 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2c0625260 0x55a2c0627650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:50.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.009+0000 7f28f5000640 1 --2- 172.21.15.83:0/87587183 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a2c061a270 0x55a2c061a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:50.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.009+0000 7f28efe00640 1 -- 172.21.15.83:0/87587183 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a2c061a270 msgr2=0x55a2c061a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:50.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.010+0000 7f28efe00640 1 --2- 172.21.15.83:0/87587183 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a2c061a270 0x55a2c061a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:50.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.010+0000 7f28efe00640 1 -- 172.21.15.83:0/87587183 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2c0625260 msgr2=0x55a2c0627650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:50.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.010+0000 7f28efe00640 1 --2- 172.21.15.83:0/87587183 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2c0625260 0x55a2c0627650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:50.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.010+0000 7f28efe00640 1 -- 172.21.15.83:0/87587183 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2c035a7e0 con 0x55a2c061ab90 2024-04-24T08:56:50.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.010+0000 7f28efe00640 1 --2- 172.21.15.83:0/87587183 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2c061ab90 0x55a2c0624d20 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f28e000bcb0 tx=0x7f28e000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65aa68ce96e5a7b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:50.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.010+0000 7f28e5600640 1 -- 172.21.15.83:0/87587183 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e0015070 con 0x55a2c061ab90 2024-04-24T08:56:50.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.010+0000 7f28ecc00640 1 --2- 172.21.15.83:0/145480370 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f28d40a3d20 0x7f28d40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:50.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.010+0000 7f28e5600640 1 -- 172.21.15.83:0/87587183 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f28e00118f0 con 0x55a2c061ab90 2024-04-24T08:56:50.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.010+0000 7f28e5600640 1 -- 172.21.15.83:0/87587183 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e00108b0 con 0x55a2c061ab90 2024-04-24T08:56:50.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.010+0000 7f28ecc00640 1 -- 172.21.15.83:0/145480370 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f28dc0d0630 con 0x7f28f0074cf0 2024-04-24T08:56:50.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.011+0000 7f28f5000640 1 --2- 172.21.15.83:0/145480370 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f28d40a3d20 0x7f28d40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:50.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.011+0000 7f28f9f66740 1 -- 172.21.15.83:0/87587183 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2c061ab90 msgr2=0x55a2c0624d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:50.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.011+0000 7f28f9f66740 1 --2- 172.21.15.83:0/87587183 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2c061ab90 0x55a2c0624d20 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f28e000bcb0 tx=0x7f28e000f730 comp rx=0 tx=0).stop 2024-04-24T08:56:50.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.011+0000 7f28f5000640 1 --2- 172.21.15.83:0/145480370 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f28d40a3d20 0x7f28d40a61e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f28e8009a60 tx=0x7f28e8005c30 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:50.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.011+0000 7f28f9f66740 1 -- 172.21.15.83:0/87587183 shutdown_connections 2024-04-24T08:56:50.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.011+0000 7f28f9f66740 1 --2- 172.21.15.83:0/87587183 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2c0625260 0x55a2c0627650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:50.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.011+0000 7f28f9f66740 1 --2- 172.21.15.83:0/87587183 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2c061ab90 0x55a2c0624d20 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:50.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.011+0000 7f28f9f66740 1 --2- 172.21.15.83:0/87587183 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a2c061a270 0x55a2c061a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:50.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.011+0000 7f28f9f66740 1 -- 172.21.15.83:0/87587183 >> 172.21.15.83:0/87587183 conn(0x55a2c0616590 msgr2=0x55a2c0616970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:50.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.012+0000 7f28f9f66740 1 -- 172.21.15.83:0/87587183 shutdown_connections 2024-04-24T08:56:50.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.012+0000 7f28f9f66740 1 -- 172.21.15.83:0/87587183 wait complete. 2024-04-24T08:56:50.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.012+0000 7f28f9f66740 1 -- start start 2024-04-24T08:56:50.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.012+0000 7f28f9f66740 1 Processor -- start 2024-04-24T08:56:50.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.013+0000 7f28f9f66740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2c061a270 0x55a2c06221c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:50.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.013+0000 7f28f9f66740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2c061ab90 0x55a2c0622700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:50.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.013+0000 7f28f9f66740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a2c0625260 0x55a2c0622c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:50.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.013+0000 7f28f9f66740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a2c0628050 con 0x55a2c061ab90 2024-04-24T08:56:50.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.013+0000 7f28f9f66740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a2c0539660 con 0x55a2c061a270 2024-04-24T08:56:50.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.013+0000 7f28f9f66740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a2c06281d0 con 0x55a2c0625260 2024-04-24T08:56:50.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.013+0000 7f28f9f66740 10 client.0 _ll_register_callbacks cb 0x55a2c0527aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:56:50.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.013+0000 7f28ef400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2c061ab90 0x55a2c0622700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:50.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.013+0000 7f28ef400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2c061ab90 0x55a2c0622700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40698/0 (socket says 172.21.15.83:40698) 2024-04-24T08:56:50.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.013+0000 7f28ef400640 1 -- 172.21.15.83:0/3930802254 learned_addr learned my addr 172.21.15.83:0/3930802254 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:56:50.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.013+0000 7f28efe00640 1 --2- 172.21.15.83:0/3930802254 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2c061a270 0x55a2c06221c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:50.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.013+0000 7f28f5000640 1 --2- 172.21.15.83:0/3930802254 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a2c0625260 0x55a2c0622c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:50.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.013+0000 7f28ef400640 1 -- 172.21.15.83:0/3930802254 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a2c0625260 msgr2=0x55a2c0622c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:50.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.014+0000 7f28ef400640 1 --2- 172.21.15.83:0/3930802254 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a2c0625260 0x55a2c0622c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:50.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.014+0000 7f28ef400640 1 -- 172.21.15.83:0/3930802254 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2c061a270 msgr2=0x55a2c06221c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:50.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.014+0000 7f28ef400640 1 --2- 172.21.15.83:0/3930802254 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2c061a270 0x55a2c06221c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:50.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.014+0000 7f28ef400640 1 -- 172.21.15.83:0/3930802254 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2c03baf70 con 0x55a2c061ab90 2024-04-24T08:56:50.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.014+0000 7f28f5000640 1 --2- 172.21.15.83:0/3930802254 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a2c0625260 0x55a2c0622c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:56:50.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.014+0000 7f28ef400640 1 --2- 172.21.15.83:0/3930802254 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2c061ab90 0x55a2c0622700 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f28dc021000 tx=0x7f28dc098590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cf20606570cc960 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:50.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.014+0000 7f28da000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:56:50.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.014+0000 7f28da000640 1 -- 172.21.15.83:0/3930802254 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28dc020040 con 0x55a2c061ab90 2024-04-24T08:56:50.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.014+0000 7f28da000640 1 -- 172.21.15.83:0/3930802254 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f28dc0d01f0 con 0x55a2c061ab90 2024-04-24T08:56:50.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.014+0000 7f28f9f66740 1 -- 172.21.15.83:0/3930802254 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28e00118f0 con 0x55a2c061ab90 2024-04-24T08:56:50.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.014+0000 7f28da000640 1 -- 172.21.15.83:0/3930802254 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f28dc005180 con 0x55a2c061ab90 2024-04-24T08:56:50.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.014+0000 7f28f9f66740 10 client.17565 fetch_fsmap learned FSMap version 33 2024-04-24T08:56:50.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.014+0000 7f28f9f66740 1 -- 172.21.15.83:0/3930802254 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55a2c0252170 con 0x55a2c061ab90 2024-04-24T08:56:50.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.015+0000 7f28da000640 1 -- 172.21.15.83:0/3930802254 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f28e00118f0 con 0x55a2c061ab90 2024-04-24T08:56:50.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.015+0000 7f28da000640 1 -- 172.21.15.83:0/3930802254 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f28dc0a0110 con 0x55a2c061ab90 2024-04-24T08:56:50.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.015+0000 7f28f9f66740 10 client.17565 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T08:56:50.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.015+0000 7f28f9f66740 10 client.17565 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:56:50.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.015+0000 7f28f9f66740 20 client.17565 populate_metadata read hostname 'smithi083' 2024-04-24T08:56:50.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.015+0000 7f28f9f66740 1 --2- 172.21.15.83:0/3930802254 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55a2c06354f0 0x55a2c06378e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:50.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.015+0000 7f28f9f66740 4 client.17565 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:56:50.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.015+0000 7f28f9f66740 1 -- 172.21.15.83:0/3930802254 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a2c058d890 con 0x55a2c06354f0 2024-04-24T08:56:50.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.016+0000 7f28efe00640 1 --2- 172.21.15.83:0/3930802254 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55a2c06354f0 0x55a2c06378e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:50.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.017+0000 7f28efe00640 1 --2- 172.21.15.83:0/3930802254 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55a2c06354f0 0x55a2c06378e0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4617f6af062a2a15 server_cookie=6f1d4b7c1c1ac51 in_seq=0 out_seq=0 2024-04-24T08:56:50.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.017+0000 7f28da000640 10 client.17565 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:56:50.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.020+0000 7f28da000640 1 -- 172.21.15.83:0/3930802254 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a2c058d890 con 0x55a2c06354f0 2024-04-24T08:56:50.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.020+0000 7f28da000640 10 client.17565 handle_command_reply: tid=0 2024-04-24T08:56:50.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.020+0000 7f28f9f66740 1 client.17565 shutdown 2024-04-24T08:56:50.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.020+0000 7f28f9f66740 1 -- 172.21.15.83:0/3930802254 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2c061ab90 msgr2=0x55a2c0622700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:50.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.020+0000 7f28f9f66740 1 --2- 172.21.15.83:0/3930802254 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2c061ab90 0x55a2c0622700 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f28dc021000 tx=0x7f28dc098590 comp rx=0 tx=0).stop 2024-04-24T08:56:50.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.020+0000 7f28f9f66740 1 -- 172.21.15.83:0/3930802254 shutdown_connections 2024-04-24T08:56:50.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.020+0000 7f28f9f66740 1 --2- 172.21.15.83:0/3930802254 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55a2c06354f0 0x55a2c06378e0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:50.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.020+0000 7f28f9f66740 1 --2- 172.21.15.83:0/3930802254 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a2c0625260 0x55a2c0622c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:50.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.020+0000 7f28f9f66740 1 --2- 172.21.15.83:0/3930802254 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2c061ab90 0x55a2c0622700 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:50.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.020+0000 7f28da000640 0 client.17565 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:56:50.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.020+0000 7f28f9f66740 1 --2- 172.21.15.83:0/3930802254 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2c061a270 0x55a2c06221c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:50.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.020+0000 7f28f9f66740 1 -- 172.21.15.83:0/3930802254 >> 172.21.15.83:0/3930802254 conn(0x55a2c0616590 msgr2=0x55a2c061c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:50.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.020+0000 7f28f9f66740 1 -- 172.21.15.83:0/3930802254 shutdown_connections 2024-04-24T08:56:50.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.020+0000 7f28f9f66740 1 -- 172.21.15.83:0/3930802254 wait complete. 2024-04-24T08:56:50.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.021+0000 7f28f9f66740 20 client.17565 trim_cache size 0 max 16384 2024-04-24T08:56:50.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.040+0000 7f28f9f66740 1 Processor -- start 2024-04-24T08:56:50.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.041+0000 7f28f9f66740 1 -- start start 2024-04-24T08:56:50.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.041+0000 7f28f9f66740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2c061a270 0x55a2c0630f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:50.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.041+0000 7f28f9f66740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2c061ab90 0x55a2c061ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:50.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.041+0000 7f28f9f66740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a2c0625260 0x55a2c063d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:50.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.041+0000 7f28f9f66740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f28f01b1530 con 0x55a2c061ab90 2024-04-24T08:56:50.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.041+0000 7f28f9f66740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f28f01b3070 con 0x55a2c061a270 2024-04-24T08:56:50.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.041+0000 7f28f9f66740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f28f01b1c00 con 0x55a2c0625260 2024-04-24T08:56:50.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.042+0000 7f28ef400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2c061ab90 0x55a2c061ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:50.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.042+0000 7f28ef400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2c061ab90 0x55a2c061ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40706/0 (socket says 172.21.15.83:40706) 2024-04-24T08:56:50.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.042+0000 7f28ef400640 1 -- 172.21.15.83:0/755520987 learned_addr learned my addr 172.21.15.83:0/755520987 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:56:50.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.042+0000 7f28f5000640 1 --2- 172.21.15.83:0/755520987 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a2c0625260 0x55a2c063d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:50.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.042+0000 7f28efe00640 1 --2- 172.21.15.83:0/755520987 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2c061a270 0x55a2c0630f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:50.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.042+0000 7f28ef400640 1 -- 172.21.15.83:0/755520987 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a2c0625260 msgr2=0x55a2c063d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:50.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.042+0000 7f28ef400640 1 --2- 172.21.15.83:0/755520987 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a2c0625260 0x55a2c063d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:50.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.042+0000 7f28ef400640 1 -- 172.21.15.83:0/755520987 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2c061a270 msgr2=0x55a2c0630f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:50.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.042+0000 7f28ef400640 1 --2- 172.21.15.83:0/755520987 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2c061a270 0x55a2c0630f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:50.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.042+0000 7f28ef400640 1 -- 172.21.15.83:0/755520987 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2c061a9c0 con 0x55a2c061ab90 2024-04-24T08:56:50.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.043+0000 7f28ef400640 1 --2- 172.21.15.83:0/755520987 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2c061ab90 0x55a2c061ca10 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f28dc098660 tx=0x7f28dc005680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae4927e35091b23a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:50.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.043+0000 7f28e4c00640 1 -- 172.21.15.83:0/755520987 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28dc020040 con 0x55a2c061ab90 2024-04-24T08:56:50.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.043+0000 7f28e4c00640 1 -- 172.21.15.83:0/755520987 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f28dc009c30 con 0x55a2c061ab90 2024-04-24T08:56:50.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.043+0000 7f28e4c00640 1 -- 172.21.15.83:0/755520987 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28dc0d0280 con 0x55a2c061ab90 2024-04-24T08:56:50.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.044+0000 7f28f9f66740 1 -- 172.21.15.83:0/755520987 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2c061ab90 msgr2=0x55a2c061ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:50.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.044+0000 7f28f9f66740 1 --2- 172.21.15.83:0/755520987 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2c061ab90 0x55a2c061ca10 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f28dc098660 tx=0x7f28dc005680 comp rx=0 tx=0).stop 2024-04-24T08:56:50.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.044+0000 7f28f9f66740 1 -- 172.21.15.83:0/755520987 shutdown_connections 2024-04-24T08:56:50.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.044+0000 7f28f9f66740 1 --2- 172.21.15.83:0/755520987 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a2c0625260 0x55a2c063d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:50.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.044+0000 7f28f9f66740 1 --2- 172.21.15.83:0/755520987 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2c061ab90 0x55a2c061ca10 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:50.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.044+0000 7f28f9f66740 1 --2- 172.21.15.83:0/755520987 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2c061a270 0x55a2c0630f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:50.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.044+0000 7f28f9f66740 1 -- 172.21.15.83:0/755520987 >> 172.21.15.83:0/755520987 conn(0x55a2c0616590 msgr2=0x55a2c0622a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:50.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.044+0000 7f28f9f66740 1 -- 172.21.15.83:0/755520987 shutdown_connections 2024-04-24T08:56:50.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.045+0000 7f28f9f66740 1 -- 172.21.15.83:0/755520987 wait complete. 2024-04-24T08:56:50.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.045+0000 7f28f9f66740 1 -- start start 2024-04-24T08:56:50.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.045+0000 7f28f9f66740 1 Processor -- start 2024-04-24T08:56:50.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.046+0000 7f28f9f66740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2c061a270 0x55a2c0629200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:50.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.046+0000 7f28f9f66740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a2c061ab90 0x55a2c0629740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:50.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.046+0000 7f28f9f66740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2c0625260 0x55a2c069aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:50.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.046+0000 7f28f9f66740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f28f01b29a0 con 0x55a2c0625260 2024-04-24T08:56:50.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.046+0000 7f28f9f66740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f28f01b22d0 con 0x55a2c061a270 2024-04-24T08:56:50.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.046+0000 7f28f9f66740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f28f01af510 con 0x55a2c061ab90 2024-04-24T08:56:50.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.046+0000 7f28f9f66740 10 client.0 _ll_register_callbacks cb 0x7f28dc009f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:56:50.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.046+0000 7f28f5000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2c0625260 0x55a2c069aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:50.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.046+0000 7f28f5000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2c0625260 0x55a2c069aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40716/0 (socket says 172.21.15.83:40716) 2024-04-24T08:56:50.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.046+0000 7f28f5000640 1 -- 172.21.15.83:0/1064262764 learned_addr learned my addr 172.21.15.83:0/1064262764 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:56:50.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.046+0000 7f28ef400640 1 --2- 172.21.15.83:0/1064262764 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a2c061ab90 0x55a2c0629740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:50.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.047+0000 7f28efe00640 1 --2- 172.21.15.83:0/1064262764 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2c061a270 0x55a2c0629200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:50.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.047+0000 7f28f5000640 1 -- 172.21.15.83:0/1064262764 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a2c061ab90 msgr2=0x55a2c0629740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:50.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.047+0000 7f28f5000640 1 --2- 172.21.15.83:0/1064262764 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a2c061ab90 0x55a2c0629740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:50.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.047+0000 7f28f5000640 1 -- 172.21.15.83:0/1064262764 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2c061a270 msgr2=0x55a2c0629200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:50.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.047+0000 7f28f5000640 1 --2- 172.21.15.83:0/1064262764 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2c061a270 0x55a2c0629200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:50.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.047+0000 7f28f5000640 1 -- 172.21.15.83:0/1064262764 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2c069af40 con 0x55a2c0625260 2024-04-24T08:56:50.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.047+0000 7f28ef400640 1 --2- 172.21.15.83:0/1064262764 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a2c061ab90 0x55a2c0629740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:56:50.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.047+0000 7f28efe00640 1 --2- 172.21.15.83:0/1064262764 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2c061a270 0x55a2c0629200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:56:50.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.047+0000 7f28f5000640 1 --2- 172.21.15.83:0/1064262764 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2c0625260 0x55a2c069aa00 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f28f0075320 tx=0x7f28e80095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e9d36900c5f8164 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:50.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.047+0000 7f28da000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:56:50.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.047+0000 7f28da000640 1 -- 172.21.15.83:0/1064262764 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e8034040 con 0x55a2c0625260 2024-04-24T08:56:50.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.048+0000 7f28da000640 1 -- 172.21.15.83:0/1064262764 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f28e8003e00 con 0x55a2c0625260 2024-04-24T08:56:50.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.048+0000 7f28f9f66740 1 -- 172.21.15.83:0/1064262764 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a2c0587700 con 0x55a2c0625260 2024-04-24T08:56:50.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.048+0000 7f28da000640 1 -- 172.21.15.83:0/1064262764 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f28e8004040 con 0x55a2c0625260 2024-04-24T08:56:50.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.048+0000 7f28f9f66740 10 client.17571 fetch_fsmap learned FSMap version 33 2024-04-24T08:56:50.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.048+0000 7f28f9f66740 1 -- 172.21.15.83:0/1064262764 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55a2c069c100 con 0x55a2c0625260 2024-04-24T08:56:50.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.048+0000 7f28da000640 1 -- 172.21.15.83:0/1064262764 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55a2c0587700 con 0x55a2c0625260 2024-04-24T08:56:50.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.048+0000 7f28da000640 1 -- 172.21.15.83:0/1064262764 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f28e8052990 con 0x55a2c0625260 2024-04-24T08:56:50.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.048+0000 7f28f9f66740 10 client.17571 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T08:56:50.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.048+0000 7f28f9f66740 10 client.17571 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:56:50.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.048+0000 7f28f9f66740 20 client.17571 populate_metadata read hostname 'smithi083' 2024-04-24T08:56:50.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.048+0000 7f28f9f66740 1 --2- 172.21.15.83:0/1064262764 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55a2c069c880 0x55a2c069eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:50.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.048+0000 7f28f9f66740 4 client.17571 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T08:56:50.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.048+0000 7f28f9f66740 1 -- 172.21.15.83:0/1064262764 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a2c0503980 con 0x55a2c069c880 2024-04-24T08:56:50.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.049+0000 7f28efe00640 1 --2- 172.21.15.83:0/1064262764 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55a2c069c880 0x55a2c069eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:50.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.050+0000 7f28efe00640 1 --2- 172.21.15.83:0/1064262764 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55a2c069c880 0x55a2c069eca0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f55d82adc6ea9e1e server_cookie=f2b046995b18bff2 in_seq=0 out_seq=0 2024-04-24T08:56:50.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.050+0000 7f28da000640 10 client.17571 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:56:50.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.053+0000 7f28da000640 1 -- 172.21.15.83:0/1064262764 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a2c0503980 con 0x55a2c069c880 2024-04-24T08:56:50.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.053+0000 7f28da000640 10 client.17571 handle_command_reply: tid=0 2024-04-24T08:56:50.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.053+0000 7f28f9f66740 1 client.17571 shutdown 2024-04-24T08:56:50.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.053+0000 7f28f9f66740 1 -- 172.21.15.83:0/1064262764 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2c0625260 msgr2=0x55a2c069aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:50.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.053+0000 7f28f9f66740 1 --2- 172.21.15.83:0/1064262764 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2c0625260 0x55a2c069aa00 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f28f0075320 tx=0x7f28e80095c0 comp rx=0 tx=0).stop 2024-04-24T08:56:50.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:56:49 smithi083 ceph-mon[27907]: pgmap v159: 129 pgs: 129 active+clean; 39 MiB data, 566 MiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 438 KiB/s wr, 62 op/s 2024-04-24T08:56:50.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.053+0000 7f28f9f66740 1 -- 172.21.15.83:0/1064262764 shutdown_connections 2024-04-24T08:56:50.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.053+0000 7f28f9f66740 1 --2- 172.21.15.83:0/1064262764 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55a2c069c880 0x55a2c069eca0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:50.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.053+0000 7f28f9f66740 1 --2- 172.21.15.83:0/1064262764 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a2c0625260 0x55a2c069aa00 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:50.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.053+0000 7f28f9f66740 1 --2- 172.21.15.83:0/1064262764 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a2c061ab90 0x55a2c0629740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:50.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.053+0000 7f28f9f66740 1 --2- 172.21.15.83:0/1064262764 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2c061a270 0x55a2c0629200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:50.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.053+0000 7f28f9f66740 1 -- 172.21.15.83:0/1064262764 >> 172.21.15.83:0/1064262764 conn(0x55a2c0616590 msgr2=0x55a2c0614e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:50.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.053+0000 7f28da000640 0 client.17571 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:56:50.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.054+0000 7f28f9f66740 1 -- 172.21.15.83:0/1064262764 shutdown_connections 2024-04-24T08:56:50.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.054+0000 7f28f9f66740 1 -- 172.21.15.83:0/1064262764 wait complete. 2024-04-24T08:56:50.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.054+0000 7f28f9f66740 20 client.17571 trim_cache size 0 max 16384 2024-04-24T08:56:50.055 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T08:56:50.055 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-04-24T08:56:50.055 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "b2bb71b2-204f-4546-bd19-e9d18cbb15db", 2024-04-24T08:56:50.055 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-04-24T08:56:50.055 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T08:56:50.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.055+0000 7f28f5a00640 1 -- 172.21.15.83:0/145480370 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f28d40a3d20 msgr2=0x7f28d40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:50.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.055+0000 7f28f5a00640 1 --2- 172.21.15.83:0/145480370 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f28d40a3d20 0x7f28d40a61e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f28e8009a60 tx=0x7f28e8005c30 comp rx=0 tx=0).stop 2024-04-24T08:56:50.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.055+0000 7f28f5a00640 1 -- 172.21.15.83:0/145480370 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f28f0074cf0 msgr2=0x7f28f01a0b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:50.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.055+0000 7f28f5a00640 1 --2- 172.21.15.83:0/145480370 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f28f0074cf0 0x7f28f01a0b60 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f28dc00d770 tx=0x7f28dc00dc30 comp rx=0 tx=0).stop 2024-04-24T08:56:50.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.055+0000 7f28f5a00640 1 -- 172.21.15.83:0/145480370 shutdown_connections 2024-04-24T08:56:50.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.055+0000 7f28f5a00640 1 --2- 172.21.15.83:0/145480370 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f28d40a3d20 0x7f28d40a61e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:50.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.055+0000 7f28f5a00640 1 --2- 172.21.15.83:0/145480370 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f28f010e740 0x7f28f019a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:50.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.055+0000 7f28f5a00640 1 --2- 172.21.15.83:0/145480370 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f28f0074cf0 0x7f28f01a0b60 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:50.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.055+0000 7f28f5a00640 1 --2- 172.21.15.83:0/145480370 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f28f00743d0 0x7f28f01a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:50.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.055+0000 7f28f5a00640 1 -- 172.21.15.83:0/145480370 >> 172.21.15.83:0/145480370 conn(0x7f28f00fd9d0 msgr2=0x7f28f00fe780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:50.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.055+0000 7f28f5a00640 1 -- 172.21.15.83:0/145480370 shutdown_connections 2024-04-24T08:56:50.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.055+0000 7f28f5a00640 1 -- 172.21.15.83:0/145480370 wait complete. 2024-04-24T08:56:50.072 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T08:56:50.072 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T08:56:50.112 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:56:49 smithi138 ceph-mon[31441]: pgmap v159: 129 pgs: 129 active+clean; 39 MiB data, 566 MiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 438 KiB/s wr, 62 op/s 2024-04-24T08:56:50.114 INFO:journalctl@ceph.mds.b.smithi138.stdout:Apr 24 08:56:50 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-b[61515]: 2024-04-24T08:56:50.113+0000 7f7fbbc00700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000225 [...7,head] /volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/PM/ auth{3=1} v4244 ap=2 f(v444 m2024-04-24T08:56:26.137809+0000 3=3+0) n(v12 rc2024-04-24T08:56:26.137809+0000 b780288 4=3+1) old_inodes=3 (inest sync->mix g=3) caps={15603=pAsLsXsFs/-@4666} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x55cf8762f180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000000225:[<0x10000000009/PM v381>,<0x10000000008/~dmtmp v57>,<0x10000000007/client0 v33>,<0x10000000006/clients v23>,<0x10000000005/tmp v23>,<0x10000000003/client.0 v22>,<0x10000000002/e395bed6-5bba-41b8-a256-3505e4afcca2 v36>,<0x10000000001/sv_0 v55>,<0x10000000000/qa v43>,<0x1/volumes v39>]//[]","memoryvalue":"(2)0x10000000225:[<0x10000000009/PM v4244>,<0x10000000008/~dmtmp v3786>,<0x10000000007/client0 v471>,<0x10000000006/clients v461>,<0x10000000005/tmp v459>,<0x10000000003/client.0 v456>,<0x10000000002/e395bed6-5bba-41b8-a256-3505e4afcca2 v469>,<0x10000000001/sv_0 v483>,<0x10000000000/qa v467>,<0x1/volumes v453>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-04-24T08:56:24.586815+0000 4=4+0)","ondisk_value.rstat":"n(v0 rc2024-04-24T08:56:26.137809+0000 b780288 4=3+1)","memory_value.dirstat":"f(v444 m2024-04-24T08:56:26.137809+0000 3=3+0)","memory_value.rstat":"n(v12 rc2024-04-24T08:56:26.137809+0000 b780288 4=3+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-04-24T08:56:50.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:56:49 smithi189 ceph-mon[31791]: pgmap v159: 129 pgs: 129 active+clean; 39 MiB data, 566 MiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 438 KiB/s wr, 62 op/s 2024-04-24T08:56:50.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.281+0000 7f3cf7e00640 1 -- 172.21.15.83:0/4107941637 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3cf81093f0 msgr2=0x7f3cf8110fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:50.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.281+0000 7f3cf7e00640 1 --2- 172.21.15.83:0/4107941637 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3cf81093f0 0x7f3cf8110fc0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f3cf0009ca0 tx=0x7f3cf002a8c0 comp rx=0 tx=0).stop 2024-04-24T08:56:50.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.281+0000 7f3cf7e00640 1 -- 172.21.15.83:0/4107941637 shutdown_connections 2024-04-24T08:56:50.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.281+0000 7f3cf7e00640 1 --2- 172.21.15.83:0/4107941637 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3cf81093f0 0x7f3cf8110fc0 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:50.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.281+0000 7f3cf7e00640 1 --2- 172.21.15.83:0/4107941637 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3cf8104cd0 0x7f3cf8108d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:50.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.281+0000 7f3cf7e00640 1 --2- 172.21.15.83:0/4107941637 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3cf8104320 0x7f3cf8104700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:50.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.281+0000 7f3cf7e00640 1 -- 172.21.15.83:0/4107941637 >> 172.21.15.83:0/4107941637 conn(0x7f3cf80ffb40 msgr2=0x7f3cf8101f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:50.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.281+0000 7f3cf7e00640 1 -- 172.21.15.83:0/4107941637 shutdown_connections 2024-04-24T08:56:50.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.281+0000 7f3cf7e00640 1 -- 172.21.15.83:0/4107941637 wait complete. 2024-04-24T08:56:50.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.282+0000 7f3cf7e00640 1 Processor -- start 2024-04-24T08:56:50.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.282+0000 7f3cf7e00640 1 -- start start 2024-04-24T08:56:50.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.282+0000 7f3cf7e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3cf8104320 0x7f3cf81a05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:50.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.283+0000 7f3cf7e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3cf81093f0 0x7f3cf81a0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:50.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.283+0000 7f3cf7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3cf819a690 0x7f3cf819ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:50.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.283+0000 7f3cf7e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3cf8114390 con 0x7f3cf819a690 2024-04-24T08:56:50.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.283+0000 7f3cf7e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3cf8114210 con 0x7f3cf81093f0 2024-04-24T08:56:50.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.283+0000 7f3cf7e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3cf8114510 con 0x7f3cf8104320 2024-04-24T08:56:50.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.283+0000 7f3cf7400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3cf819a690 0x7f3cf819ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:50.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.283+0000 7f3cf7400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3cf819a690 0x7f3cf819ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40720/0 (socket says 172.21.15.83:40720) 2024-04-24T08:56:50.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.283+0000 7f3cf7400640 1 -- 172.21.15.83:0/202061199 learned_addr learned my addr 172.21.15.83:0/202061199 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:56:50.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.283+0000 7f3cee000640 1 --2- 172.21.15.83:0/202061199 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3cf81093f0 0x7f3cf81a0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:50.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.283+0000 7f3cf6a00640 1 --2- 172.21.15.83:0/202061199 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3cf8104320 0x7f3cf81a05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:50.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.283+0000 7f3cf7400640 1 -- 172.21.15.83:0/202061199 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3cf8104320 msgr2=0x7f3cf81a05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:50.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.283+0000 7f3cf7400640 1 --2- 172.21.15.83:0/202061199 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3cf8104320 0x7f3cf81a05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:50.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.283+0000 7f3cf7400640 1 -- 172.21.15.83:0/202061199 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3cf81093f0 msgr2=0x7f3cf81a0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:50.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.283+0000 7f3cf7400640 1 --2- 172.21.15.83:0/202061199 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3cf81093f0 0x7f3cf81a0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:50.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.283+0000 7f3cf7400640 1 -- 172.21.15.83:0/202061199 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cf819b290 con 0x7f3cf819a690 2024-04-24T08:56:50.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.284+0000 7f3cf7400640 1 --2- 172.21.15.83:0/202061199 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3cf819a690 0x7f3cf819ab10 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f3cf0031040 tx=0x7f3cf0003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:50.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.284+0000 7f3cefe00640 1 -- 172.21.15.83:0/202061199 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cf0003ce0 con 0x7f3cf819a690 2024-04-24T08:56:50.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.284+0000 7f3cefe00640 1 -- 172.21.15.83:0/202061199 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3cf00044f0 con 0x7f3cf819a690 2024-04-24T08:56:50.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.284+0000 7f3cefe00640 1 -- 172.21.15.83:0/202061199 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cf0030c90 con 0x7f3cf819a690 2024-04-24T08:56:50.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.284+0000 7f3cf7e00640 1 -- 172.21.15.83:0/202061199 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cf819b520 con 0x7f3cf819a690 2024-04-24T08:56:50.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.284+0000 7f3cf7e00640 1 -- 172.21.15.83:0/202061199 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cf81a7670 con 0x7f3cf819a690 2024-04-24T08:56:50.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.288+0000 7f3cefe00640 1 -- 172.21.15.83:0/202061199 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3cf00046a0 con 0x7f3cf819a690 2024-04-24T08:56:50.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.288+0000 7f3d0058a740 1 Processor -- start 2024-04-24T08:56:50.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.288+0000 7f3d0058a740 1 -- start start 2024-04-24T08:56:50.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.289+0000 7f3d0058a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a575131a30 0x55a575131e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:50.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.289+0000 7f3d0058a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a575132350 0x55a57513c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:50.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.289+0000 7f3d0058a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a57513ca20 0x55a57513ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:50.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.289+0000 7f3d0058a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a5750971f0 con 0x55a575132350 2024-04-24T08:56:50.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.289+0000 7f3d0058a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a57503d350 con 0x55a57513ca20 2024-04-24T08:56:50.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.289+0000 7f3d0058a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a5750931b0 con 0x55a575131a30 2024-04-24T08:56:50.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.289+0000 7f3cf6a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a575132350 0x55a57513c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:50.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.289+0000 7f3cf6a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a575132350 0x55a57513c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40722/0 (socket says 172.21.15.83:40722) 2024-04-24T08:56:50.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.289+0000 7f3cf6a00640 1 -- 172.21.15.83:0/590290865 learned_addr learned my addr 172.21.15.83:0/590290865 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:56:50.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.289+0000 7f3cee000640 1 --2- 172.21.15.83:0/590290865 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a57513ca20 0x55a57513ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:50.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.289+0000 7f3cee000640 1 --2- 172.21.15.83:0/590290865 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a575131a30 0x55a575131e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:50.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.289+0000 7f3cf6a00640 1 -- 172.21.15.83:0/590290865 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a575131a30 msgr2=0x55a575131e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:50.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.289+0000 7f3cf6a00640 1 --2- 172.21.15.83:0/590290865 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a575131a30 0x55a575131e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:50.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.289+0000 7f3cf6a00640 1 -- 172.21.15.83:0/590290865 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a57513ca20 msgr2=0x55a57513ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:50.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.289+0000 7f3cf6a00640 1 --2- 172.21.15.83:0/590290865 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a57513ca20 0x55a57513ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:50.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.289+0000 7f3cf6a00640 1 -- 172.21.15.83:0/590290865 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a575091f50 con 0x55a575132350 2024-04-24T08:56:50.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.289+0000 7f3cee000640 1 --2- 172.21.15.83:0/590290865 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a57513ca20 0x55a57513ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:56:50.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.289+0000 7f3cefe00640 1 --2- 172.21.15.83:0/202061199 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f3cd80a3d20 0x7f3cd80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:50.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.290+0000 7f3cefe00640 1 -- 172.21.15.83:0/202061199 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f3cf00f2180 con 0x7f3cf819a690 2024-04-24T08:56:50.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.290+0000 7f3cf6a00640 1 --2- 172.21.15.83:0/590290865 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a575132350 0x55a57513c4e0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f3ce800bcb0 tx=0x7f3ce800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d339d28da57c875 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:50.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.290+0000 7f3cee000640 1 --2- 172.21.15.83:0/202061199 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f3cd80a3d20 0x7f3cd80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:50.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.290+0000 7f3ce7e00640 1 -- 172.21.15.83:0/590290865 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ce8015070 con 0x55a575132350 2024-04-24T08:56:50.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.290+0000 7f3ce7e00640 1 -- 172.21.15.83:0/590290865 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3ce801a910 con 0x55a575132350 2024-04-24T08:56:50.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.290+0000 7f3ce7e00640 1 -- 172.21.15.83:0/590290865 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ce8011930 con 0x55a575132350 2024-04-24T08:56:50.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.291+0000 7f3d0058a740 1 -- 172.21.15.83:0/590290865 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a575132350 msgr2=0x55a57513c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:50.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.291+0000 7f3d0058a740 1 --2- 172.21.15.83:0/590290865 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a575132350 0x55a57513c4e0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f3ce800bcb0 tx=0x7f3ce800f730 comp rx=0 tx=0).stop 2024-04-24T08:56:50.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.291+0000 7f3d0058a740 1 -- 172.21.15.83:0/590290865 shutdown_connections 2024-04-24T08:56:50.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.291+0000 7f3d0058a740 1 --2- 172.21.15.83:0/590290865 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a57513ca20 0x55a57513ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:50.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.291+0000 7f3cee000640 1 --2- 172.21.15.83:0/202061199 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f3cd80a3d20 0x7f3cd80a61e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x55a57512eee0 tx=0x7f3ce00046c0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:50.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.291+0000 7f3d0058a740 1 --2- 172.21.15.83:0/590290865 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a575132350 0x55a57513c4e0 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:50.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.291+0000 7f3d0058a740 1 --2- 172.21.15.83:0/590290865 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a575131a30 0x55a575131e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:50.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.291+0000 7f3d0058a740 1 -- 172.21.15.83:0/590290865 >> 172.21.15.83:0/590290865 conn(0x55a57512dd50 msgr2=0x55a57512e130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:50.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.291+0000 7f3d0058a740 1 -- 172.21.15.83:0/590290865 shutdown_connections 2024-04-24T08:56:50.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.291+0000 7f3d0058a740 1 -- 172.21.15.83:0/590290865 wait complete. 2024-04-24T08:56:50.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.292+0000 7f3d0058a740 1 -- start start 2024-04-24T08:56:50.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.292+0000 7f3d0058a740 1 Processor -- start 2024-04-24T08:56:50.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.293+0000 7f3d0058a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a575131a30 0x55a575139980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:50.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.293+0000 7f3d0058a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a575132350 0x55a575139ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:50.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.293+0000 7f3d0058a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a57513ca20 0x55a57513a400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:50.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.293+0000 7f3d0058a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a57513f810 con 0x55a575131a30 2024-04-24T08:56:50.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.293+0000 7f3d0058a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a575050690 con 0x55a57513ca20 2024-04-24T08:56:50.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.293+0000 7f3d0058a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a57513f990 con 0x55a575132350 2024-04-24T08:56:50.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.293+0000 7f3d0058a740 10 client.0 _ll_register_callbacks cb 0x55a57508c650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:56:50.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.293+0000 7f3cf6a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a575131a30 0x55a575139980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:50.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.293+0000 7f3cf6a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a575131a30 0x55a575139980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40730/0 (socket says 172.21.15.83:40730) 2024-04-24T08:56:50.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.293+0000 7f3cf6a00640 1 -- 172.21.15.83:0/3658191003 learned_addr learned my addr 172.21.15.83:0/3658191003 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:56:50.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.293+0000 7f3cf7400640 1 --2- 172.21.15.83:0/3658191003 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a57513ca20 0x55a57513a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:50.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.293+0000 7f3cee000640 1 --2- 172.21.15.83:0/3658191003 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a575132350 0x55a575139ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:50.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.293+0000 7f3cf6a00640 1 -- 172.21.15.83:0/3658191003 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a575132350 msgr2=0x55a575139ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:50.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.293+0000 7f3cf6a00640 1 --2- 172.21.15.83:0/3658191003 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a575132350 0x55a575139ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:50.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.293+0000 7f3cf6a00640 1 -- 172.21.15.83:0/3658191003 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a57513ca20 msgr2=0x55a57513a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:50.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.293+0000 7f3cf6a00640 1 --2- 172.21.15.83:0/3658191003 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a57513ca20 0x55a57513a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:50.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.293+0000 7f3cf6a00640 1 -- 172.21.15.83:0/3658191003 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a57504f920 con 0x55a575131a30 2024-04-24T08:56:50.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.294+0000 7f3cee000640 1 --2- 172.21.15.83:0/3658191003 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a575132350 0x55a575139ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:56:50.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.294+0000 7f3cf6a00640 1 --2- 172.21.15.83:0/3658191003 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a575131a30 0x55a575139980 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f3ce8000c00 tx=0x7f3ce800bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8385fa1d93b9b77b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:50.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.294+0000 7f3ce4c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:56:50.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.294+0000 7f3ce4c00640 1 -- 172.21.15.83:0/3658191003 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ce8028060 con 0x55a575131a30 2024-04-24T08:56:50.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.294+0000 7f3ce4c00640 1 -- 172.21.15.83:0/3658191003 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f3ce801aac0 con 0x55a575131a30 2024-04-24T08:56:50.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.294+0000 7f3d0058a740 1 -- 172.21.15.83:0/3658191003 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ce801a910 con 0x55a575131a30 2024-04-24T08:56:50.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.295+0000 7f3ce4c00640 1 -- 172.21.15.83:0/3658191003 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f3ce802f050 con 0x55a575131a30 2024-04-24T08:56:50.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.295+0000 7f3d0058a740 10 client.17583 fetch_fsmap learned FSMap version 33 2024-04-24T08:56:50.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.295+0000 7f3d0058a740 1 -- 172.21.15.83:0/3658191003 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55a574e737d0 con 0x55a575131a30 2024-04-24T08:56:50.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.295+0000 7f3ce4c00640 1 -- 172.21.15.83:0/3658191003 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3ce801a910 con 0x55a575131a30 2024-04-24T08:56:50.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.295+0000 7f3ce4c00640 1 -- 172.21.15.83:0/3658191003 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f3ce80119e0 con 0x55a575131a30 2024-04-24T08:56:50.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.295+0000 7f3d0058a740 10 client.17583 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T08:56:50.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.295+0000 7f3d0058a740 10 client.17583 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:56:50.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.295+0000 7f3d0058a740 20 client.17583 populate_metadata read hostname 'smithi083' 2024-04-24T08:56:50.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.295+0000 7f3d0058a740 1 --2- 172.21.15.83:0/3658191003 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55a57514cc70 0x55a57514f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:50.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.295+0000 7f3d0058a740 4 client.17583 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:56:50.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.295+0000 7f3d0058a740 1 -- 172.21.15.83:0/3658191003 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a5750a4c10 con 0x55a57514cc70 2024-04-24T08:56:50.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.296+0000 7f3cee000640 1 --2- 172.21.15.83:0/3658191003 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55a57514cc70 0x55a57514f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:50.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.297+0000 7f3cee000640 1 --2- 172.21.15.83:0/3658191003 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55a57514cc70 0x55a57514f060 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62fdeb440ccb7b71 server_cookie=39e8257b8849c6e5 in_seq=0 out_seq=0 2024-04-24T08:56:50.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.297+0000 7f3ce4c00640 10 client.17583 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:56:50.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.300+0000 7f3ce4c00640 1 -- 172.21.15.83:0/3658191003 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a5750a4c10 con 0x55a57514cc70 2024-04-24T08:56:50.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.300+0000 7f3ce4c00640 10 client.17583 handle_command_reply: tid=0 2024-04-24T08:56:50.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.300+0000 7f3d0058a740 1 client.17583 shutdown 2024-04-24T08:56:50.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.300+0000 7f3d0058a740 1 -- 172.21.15.83:0/3658191003 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a575131a30 msgr2=0x55a575139980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:50.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.300+0000 7f3d0058a740 1 --2- 172.21.15.83:0/3658191003 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a575131a30 0x55a575139980 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f3ce8000c00 tx=0x7f3ce800bfd0 comp rx=0 tx=0).stop 2024-04-24T08:56:50.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.300+0000 7f3d0058a740 1 -- 172.21.15.83:0/3658191003 shutdown_connections 2024-04-24T08:56:50.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.300+0000 7f3d0058a740 1 --2- 172.21.15.83:0/3658191003 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55a57514cc70 0x55a57514f060 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:50.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.300+0000 7f3d0058a740 1 --2- 172.21.15.83:0/3658191003 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a57513ca20 0x55a57513a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:50.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.300+0000 7f3d0058a740 1 --2- 172.21.15.83:0/3658191003 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a575132350 0x55a575139ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:50.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.300+0000 7f3d0058a740 1 --2- 172.21.15.83:0/3658191003 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a575131a30 0x55a575139980 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:50.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.300+0000 7f3d0058a740 1 -- 172.21.15.83:0/3658191003 >> 172.21.15.83:0/3658191003 conn(0x55a57512dd50 msgr2=0x55a57512cfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:50.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.300+0000 7f3ce4c00640 0 client.17583 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:56:50.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.301+0000 7f3d0058a740 1 -- 172.21.15.83:0/3658191003 shutdown_connections 2024-04-24T08:56:50.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.301+0000 7f3d0058a740 1 -- 172.21.15.83:0/3658191003 wait complete. 2024-04-24T08:56:50.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.301+0000 7f3d0058a740 20 client.17583 trim_cache size 0 max 16384 2024-04-24T08:56:50.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.320+0000 7f3d0058a740 1 Processor -- start 2024-04-24T08:56:50.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.321+0000 7f3d0058a740 1 -- start start 2024-04-24T08:56:50.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.321+0000 7f3d0058a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a575131a30 0x55a5751486d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:50.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.321+0000 7f3d0058a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a575132350 0x55a575154660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:50.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.321+0000 7f3d0058a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a57513ca20 0x55a575154ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:50.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.321+0000 7f3d0058a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3cf81b1550 con 0x55a57513ca20 2024-04-24T08:56:50.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.321+0000 7f3d0058a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3cf81b3090 con 0x55a575131a30 2024-04-24T08:56:50.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.321+0000 7f3d0058a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3cf81b1c20 con 0x55a575132350 2024-04-24T08:56:50.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.321+0000 7f3cf7400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a57513ca20 0x55a575154ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:50.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.321+0000 7f3cf7400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a57513ca20 0x55a575154ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40740/0 (socket says 172.21.15.83:40740) 2024-04-24T08:56:50.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.321+0000 7f3cf7400640 1 -- 172.21.15.83:0/4005039360 learned_addr learned my addr 172.21.15.83:0/4005039360 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:56:50.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.322+0000 7f3cf6a00640 1 --2- 172.21.15.83:0/4005039360 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a575131a30 0x55a5751486d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:50.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.322+0000 7f3cee000640 1 --2- 172.21.15.83:0/4005039360 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a575132350 0x55a575154660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:50.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.322+0000 7f3cf7400640 1 -- 172.21.15.83:0/4005039360 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a575132350 msgr2=0x55a575154660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:50.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.322+0000 7f3cf7400640 1 --2- 172.21.15.83:0/4005039360 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a575132350 0x55a575154660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:50.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.322+0000 7f3cf7400640 1 -- 172.21.15.83:0/4005039360 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a575131a30 msgr2=0x55a5751486d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:50.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.322+0000 7f3cf7400640 1 --2- 172.21.15.83:0/4005039360 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a575131a30 0x55a5751486d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:50.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.322+0000 7f3cf7400640 1 -- 172.21.15.83:0/4005039360 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a574da93f0 con 0x55a57513ca20 2024-04-24T08:56:50.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.322+0000 7f3cf7400640 1 --2- 172.21.15.83:0/4005039360 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a57513ca20 0x55a575154ba0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f3cf0005e30 tx=0x7f3cf00095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3abad1389346bbf server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:50.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.322+0000 7f3ce7400640 1 -- 172.21.15.83:0/4005039360 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cf0044e70 con 0x55a57513ca20 2024-04-24T08:56:50.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.322+0000 7f3ce7400640 1 -- 172.21.15.83:0/4005039360 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3cf0036030 con 0x55a57513ca20 2024-04-24T08:56:50.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.322+0000 7f3ce7400640 1 -- 172.21.15.83:0/4005039360 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cf00bc160 con 0x55a57513ca20 2024-04-24T08:56:50.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.323+0000 7f3d0058a740 1 -- 172.21.15.83:0/4005039360 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a57513ca20 msgr2=0x55a575154ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:50.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.323+0000 7f3d0058a740 1 --2- 172.21.15.83:0/4005039360 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a57513ca20 0x55a575154ba0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f3cf0005e30 tx=0x7f3cf00095c0 comp rx=0 tx=0).stop 2024-04-24T08:56:50.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.323+0000 7f3d0058a740 1 -- 172.21.15.83:0/4005039360 shutdown_connections 2024-04-24T08:56:50.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.323+0000 7f3d0058a740 1 --2- 172.21.15.83:0/4005039360 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a57513ca20 0x55a575154ba0 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:50.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.323+0000 7f3d0058a740 1 --2- 172.21.15.83:0/4005039360 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a575132350 0x55a575154660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:50.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.323+0000 7f3d0058a740 1 --2- 172.21.15.83:0/4005039360 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a575131a30 0x55a5751486d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:50.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.323+0000 7f3d0058a740 1 -- 172.21.15.83:0/4005039360 >> 172.21.15.83:0/4005039360 conn(0x55a57512dd50 msgr2=0x55a57512cfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:50.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.323+0000 7f3d0058a740 1 -- 172.21.15.83:0/4005039360 shutdown_connections 2024-04-24T08:56:50.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.323+0000 7f3d0058a740 1 -- 172.21.15.83:0/4005039360 wait complete. 2024-04-24T08:56:50.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.324+0000 7f3d0058a740 1 -- start start 2024-04-24T08:56:50.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.324+0000 7f3d0058a740 1 Processor -- start 2024-04-24T08:56:50.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.324+0000 7f3d0058a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a575131a30 0x55a575144e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:50.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.324+0000 7f3d0058a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a575132350 0x55a575140280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:50.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.324+0000 7f3d0058a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a57513ca20 0x55a5751407c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:50.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.324+0000 7f3d0058a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3cf81b29c0 con 0x55a575131a30 2024-04-24T08:56:50.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.324+0000 7f3d0058a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3cf81b22f0 con 0x55a57513ca20 2024-04-24T08:56:50.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.324+0000 7f3d0058a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3cf81af530 con 0x55a575132350 2024-04-24T08:56:50.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.325+0000 7f3d0058a740 10 client.0 _ll_register_callbacks cb 0x7f3ce800bb00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:56:50.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.325+0000 7f3cf6a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a575131a30 0x55a575144e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:50.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.325+0000 7f3cf6a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a575131a30 0x55a575144e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40746/0 (socket says 172.21.15.83:40746) 2024-04-24T08:56:50.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.325+0000 7f3cf6a00640 1 -- 172.21.15.83:0/3757610420 learned_addr learned my addr 172.21.15.83:0/3757610420 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:56:50.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.325+0000 7f3cee000640 1 --2- 172.21.15.83:0/3757610420 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a575132350 0x55a575140280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:50.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.325+0000 7f3cf6a00640 1 -- 172.21.15.83:0/3757610420 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a575132350 msgr2=0x55a575140280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:50.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.325+0000 7f3cf6a00640 1 --2- 172.21.15.83:0/3757610420 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a575132350 0x55a575140280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:50.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.325+0000 7f3cf6a00640 1 -- 172.21.15.83:0/3757610420 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a57513ca20 msgr2=0x55a5751407c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:50.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.325+0000 7f3cf6a00640 1 --2- 172.21.15.83:0/3757610420 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a57513ca20 0x55a5751407c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:50.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.325+0000 7f3cf6a00640 1 -- 172.21.15.83:0/3757610420 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a574d690f0 con 0x55a575131a30 2024-04-24T08:56:50.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.325+0000 7f3cf6a00640 1 --2- 172.21.15.83:0/3757610420 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a575131a30 0x55a575144e40 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f3ce800fe00 tx=0x7f3ce8011440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b647d686f38de030 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:50.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.325+0000 7f3ce4c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:56:50.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.326+0000 7f3ce4c00640 1 -- 172.21.15.83:0/3757610420 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ce8028070 con 0x55a575131a30 2024-04-24T08:56:50.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.326+0000 7f3ce4c00640 1 -- 172.21.15.83:0/3757610420 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f3ce8004890 con 0x55a575131a30 2024-04-24T08:56:50.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.326+0000 7f3d0058a740 1 -- 172.21.15.83:0/3757610420 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3cf0036030 con 0x55a575131a30 2024-04-24T08:56:50.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.326+0000 7f3ce4c00640 1 -- 172.21.15.83:0/3757610420 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f3ce8015070 con 0x55a575131a30 2024-04-24T08:56:50.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.326+0000 7f3d0058a740 10 client.17589 fetch_fsmap learned FSMap version 33 2024-04-24T08:56:50.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.326+0000 7f3d0058a740 1 -- 172.21.15.83:0/3757610420 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55a57505c810 con 0x55a575131a30 2024-04-24T08:56:50.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.326+0000 7f3ce4c00640 1 -- 172.21.15.83:0/3757610420 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3cf0036030 con 0x55a575131a30 2024-04-24T08:56:50.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.327+0000 7f3ce4c00640 1 -- 172.21.15.83:0/3757610420 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f3ce801a830 con 0x55a575131a30 2024-04-24T08:56:50.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.327+0000 7f3d0058a740 10 client.17589 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T08:56:50.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.327+0000 7f3d0058a740 10 client.17589 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:56:50.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.327+0000 7f3d0058a740 20 client.17589 populate_metadata read hostname 'smithi083' 2024-04-24T08:56:50.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.327+0000 7f3d0058a740 1 --2- 172.21.15.83:0/3757610420 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55a5751b3200 0x55a5751b5640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:50.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.327+0000 7f3d0058a740 4 client.17589 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T08:56:50.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.327+0000 7f3d0058a740 1 -- 172.21.15.83:0/3757610420 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a57509ea80 con 0x55a5751b3200 2024-04-24T08:56:50.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.327+0000 7f3cee000640 1 --2- 172.21.15.83:0/3757610420 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55a5751b3200 0x55a5751b5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:50.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.328+0000 7f3cee000640 1 --2- 172.21.15.83:0/3757610420 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55a5751b3200 0x55a5751b5640 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad81c9e611d7cda6 server_cookie=36b820bc89faa76a in_seq=0 out_seq=0 2024-04-24T08:56:50.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.328+0000 7f3ce4c00640 10 client.17589 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:56:50.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.329+0000 7f3ce4c00640 1 -- 172.21.15.83:0/3757610420 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55a57509ea80 con 0x55a5751b3200 2024-04-24T08:56:50.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.329+0000 7f3ce4c00640 10 client.17589 handle_command_reply: tid=0 2024-04-24T08:56:50.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.329+0000 7f3d0058a740 1 client.17589 shutdown 2024-04-24T08:56:50.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.329+0000 7f3d0058a740 1 -- 172.21.15.83:0/3757610420 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a575131a30 msgr2=0x55a575144e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:50.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.329+0000 7f3d0058a740 1 --2- 172.21.15.83:0/3757610420 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a575131a30 0x55a575144e40 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f3ce800fe00 tx=0x7f3ce8011440 comp rx=0 tx=0).stop 2024-04-24T08:56:50.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.329+0000 7f3d0058a740 1 -- 172.21.15.83:0/3757610420 shutdown_connections 2024-04-24T08:56:50.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.329+0000 7f3d0058a740 1 --2- 172.21.15.83:0/3757610420 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55a5751b3200 0x55a5751b5640 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:50.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.329+0000 7f3d0058a740 1 --2- 172.21.15.83:0/3757610420 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a57513ca20 0x55a5751407c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:50.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.329+0000 7f3d0058a740 1 --2- 172.21.15.83:0/3757610420 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a575132350 0x55a575140280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:50.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.329+0000 7f3ce4c00640 0 client.17589 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:56:50.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.329+0000 7f3d0058a740 1 --2- 172.21.15.83:0/3757610420 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a575131a30 0x55a575144e40 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:50.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.329+0000 7f3d0058a740 1 -- 172.21.15.83:0/3757610420 >> 172.21.15.83:0/3757610420 conn(0x55a57512dd50 msgr2=0x55a575139920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:50.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.329+0000 7f3d0058a740 1 -- 172.21.15.83:0/3757610420 shutdown_connections 2024-04-24T08:56:50.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.329+0000 7f3d0058a740 1 -- 172.21.15.83:0/3757610420 wait complete. 2024-04-24T08:56:50.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.330+0000 7f3d0058a740 20 client.17589 trim_cache size 0 max 16384 2024-04-24T08:56:50.332 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T08:56:50.332 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T08:56:50.332 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-04-24T08:56:50.332 INFO:teuthology.orchestra.run.smithi083.stdout: "b2bb71b2-204f-4546-bd19-e9d18cbb15db": { 2024-04-24T08:56:50.332 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-04-24T08:56:50.332 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "b2bb71b2-204f-4546-bd19-e9d18cbb15db", 2024-04-24T08:56:50.332 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-04-24T08:56:50.333 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T08:56:50.333 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T08:56:50.333 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T08:56:50.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.330+0000 7f3cf7e00640 1 -- 172.21.15.83:0/202061199 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f3cd80a3d20 msgr2=0x7f3cd80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:50.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.330+0000 7f3cf7e00640 1 --2- 172.21.15.83:0/202061199 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f3cd80a3d20 0x7f3cd80a61e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x55a57512eee0 tx=0x7f3ce00046c0 comp rx=0 tx=0).stop 2024-04-24T08:56:50.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.330+0000 7f3cf7e00640 1 -- 172.21.15.83:0/202061199 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3cf819a690 msgr2=0x7f3cf819ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:50.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.330+0000 7f3cf7e00640 1 --2- 172.21.15.83:0/202061199 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3cf819a690 0x7f3cf819ab10 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f3cf0031040 tx=0x7f3cf0003970 comp rx=0 tx=0).stop 2024-04-24T08:56:50.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.330+0000 7f3cf7e00640 1 -- 172.21.15.83:0/202061199 shutdown_connections 2024-04-24T08:56:50.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.330+0000 7f3cf7e00640 1 --2- 172.21.15.83:0/202061199 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f3cd80a3d20 0x7f3cd80a61e0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:50.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.330+0000 7f3cf7e00640 1 --2- 172.21.15.83:0/202061199 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3cf819a690 0x7f3cf819ab10 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:50.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.330+0000 7f3cf7e00640 1 --2- 172.21.15.83:0/202061199 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3cf81093f0 0x7f3cf81a0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:50.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.330+0000 7f3cf7e00640 1 --2- 172.21.15.83:0/202061199 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3cf8104320 0x7f3cf81a05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:50.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.330+0000 7f3cf7e00640 1 -- 172.21.15.83:0/202061199 >> 172.21.15.83:0/202061199 conn(0x7f3cf80ffb40 msgr2=0x7f3cf8101ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:50.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.330+0000 7f3cf7e00640 1 -- 172.21.15.83:0/202061199 shutdown_connections 2024-04-24T08:56:50.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:56:50.331+0000 7f3cf7e00640 1 -- 172.21.15.83:0/202061199 wait complete. 2024-04-24T08:56:50.351 INFO:tasks.cephfs.filesystem:scrub status for tag:b2bb71b2-204f-4546-bd19-e9d18cbb15db - {'path': '/', 'tag': 'b2bb71b2-204f-4546-bd19-e9d18cbb15db', 'options': 'recursive,force'} 2024-04-24T08:56:50.576 INFO:tasks.workunit.client.0.smithi083.stdout: 1 25100 0.68 MB/sec execute 84 sec latency 47.831 ms 2024-04-24T08:56:51.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:56:50 smithi083 ceph-mon[27907]: scrub queued for path: / 2024-04-24T08:56:51.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:56:50 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T08:56:51.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:56:50 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T08:56:51.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:56:50 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T08:56:51.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:56:50 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T08:56:51.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:56:50 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T08:56:51.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:56:50 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T08:56:51.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:56:50 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T08:56:51.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:56:50 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T08:56:51.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:56:50 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T08:56:51.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:56:50 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T08:56:51.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:56:50 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T08:56:51.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:56:50 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T08:56:51.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:56:50 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T08:56:51.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:56:50 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T08:56:51.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:56:50 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T08:56:51.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:56:50 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T08:56:51.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:56:50 smithi138 ceph-mon[31441]: scrub queued for path: / 2024-04-24T08:56:51.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:56:50 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T08:56:51.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:56:50 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T08:56:51.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:56:50 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T08:56:51.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:56:50 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T08:56:51.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:56:50 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T08:56:51.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:56:50 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T08:56:51.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:56:50 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T08:56:51.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:56:50 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T08:56:51.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:56:50 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T08:56:51.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:56:50 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T08:56:51.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:56:50 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T08:56:51.207 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:56:50 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T08:56:51.207 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:56:50 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T08:56:51.207 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:56:50 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T08:56:51.207 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:56:50 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T08:56:51.207 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:56:50 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T08:56:51.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:56:50 smithi189 ceph-mon[31791]: scrub queued for path: / 2024-04-24T08:56:51.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:56:50 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T08:56:51.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:56:50 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T08:56:51.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:56:50 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T08:56:51.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:56:50 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T08:56:51.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:56:50 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T08:56:51.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:56:50 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T08:56:51.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:56:50 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T08:56:51.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:56:50 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T08:56:51.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:56:50 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T08:56:51.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:56:50 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T08:56:51.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:56:50 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T08:56:51.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:56:50 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T08:56:51.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:56:50 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T08:56:51.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:56:50 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T08:56:51.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:56:50 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T08:56:51.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:56:50 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T08:56:51.576 INFO:tasks.workunit.client.0.smithi083.stdout: 1 25157 0.67 MB/sec execute 85 sec latency 44.448 ms 2024-04-24T08:56:52.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:56:51 smithi138 ceph-mon[31441]: Scrub error on inode 0x10000000225 (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/PM) see mds.b log and `damage ls` output for details 2024-04-24T08:56:52.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:56:51 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T08:56:52.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:56:51 smithi138 ceph-mon[31441]: pgmap v160: 129 pgs: 129 active+clean; 39 MiB data, 566 MiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 385 KiB/s wr, 42 op/s 2024-04-24T08:56:52.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:56:51 smithi189 ceph-mon[31791]: Scrub error on inode 0x10000000225 (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/PM) see mds.b log and `damage ls` output for details 2024-04-24T08:56:52.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:56:51 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T08:56:52.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:56:51 smithi189 ceph-mon[31791]: pgmap v160: 129 pgs: 129 active+clean; 39 MiB data, 566 MiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 385 KiB/s wr, 42 op/s 2024-04-24T08:56:52.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:56:51 smithi083 ceph-mon[27907]: Scrub error on inode 0x10000000225 (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/PM) see mds.b log and `damage ls` output for details 2024-04-24T08:56:52.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:56:51 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T08:56:52.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:56:51 smithi083 ceph-mon[27907]: pgmap v160: 129 pgs: 129 active+clean; 39 MiB data, 566 MiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 385 KiB/s wr, 42 op/s 2024-04-24T08:56:52.577 INFO:tasks.workunit.client.0.smithi083.stdout: 1 25220 0.67 MB/sec execute 86 sec latency 51.951 ms 2024-04-24T08:56:53.327 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.326+0000 7f98aca00640 1 -- 172.21.15.83:0/557218963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f98a81038e0 msgr2=0x7f98a810bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:53.327 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.326+0000 7f98aca00640 1 --2- 172.21.15.83:0/557218963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f98a81038e0 0x7f98a810bdc0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f9894009900 tx=0x7f989402a480 comp rx=0 tx=0).stop 2024-04-24T08:56:53.327 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.327+0000 7f98aca00640 1 -- 172.21.15.83:0/557218963 shutdown_connections 2024-04-24T08:56:53.327 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.327+0000 7f98aca00640 1 --2- 172.21.15.83:0/557218963 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f98a810c3e0 0x7f98a810e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:53.327 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.327+0000 7f98aca00640 1 --2- 172.21.15.83:0/557218963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f98a81038e0 0x7f98a810bdc0 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:53.327 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.327+0000 7f98aca00640 1 --2- 172.21.15.83:0/557218963 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98a8102fc0 0x7f98a81033a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:53.328 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.327+0000 7f98aca00640 1 -- 172.21.15.83:0/557218963 >> 172.21.15.83:0/557218963 conn(0x7f98a80fc820 msgr2=0x7f98a80fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:53.328 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.327+0000 7f98aca00640 1 -- 172.21.15.83:0/557218963 shutdown_connections 2024-04-24T08:56:53.328 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.327+0000 7f98aca00640 1 -- 172.21.15.83:0/557218963 wait complete. 2024-04-24T08:56:53.329 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.328+0000 7f98aca00640 1 Processor -- start 2024-04-24T08:56:53.329 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.328+0000 7f98aca00640 1 -- start start 2024-04-24T08:56:53.329 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.328+0000 7f98aca00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f98a8102fc0 0x7f98a8198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:53.329 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.328+0000 7f98aca00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98a81038e0 0x7f98a8198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:53.329 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.328+0000 7f98aca00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f98a810c3e0 0x7f98a819ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:53.329 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.328+0000 7f98aca00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f98a8112070 con 0x7f98a8102fc0 2024-04-24T08:56:53.330 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.328+0000 7f98aca00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f98a8111ef0 con 0x7f98a81038e0 2024-04-24T08:56:53.330 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.328+0000 7f98aca00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f98a81121f0 con 0x7f98a810c3e0 2024-04-24T08:56:53.330 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.329+0000 7f98a6800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f98a8102fc0 0x7f98a8198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:53.330 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.329+0000 7f98a6800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f98a8102fc0 0x7f98a8198610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40762/0 (socket says 172.21.15.83:40762) 2024-04-24T08:56:53.330 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.329+0000 7f98a6800640 1 -- 172.21.15.83:0/3190407497 learned_addr learned my addr 172.21.15.83:0/3190407497 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:56:53.330 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.329+0000 7f98a7200640 1 --2- 172.21.15.83:0/3190407497 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f98a810c3e0 0x7f98a819ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:53.330 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.329+0000 7f98a5e00640 1 --2- 172.21.15.83:0/3190407497 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98a81038e0 0x7f98a8198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:53.330 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.329+0000 7f98a6800640 1 -- 172.21.15.83:0/3190407497 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f98a810c3e0 msgr2=0x7f98a819ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:53.331 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.329+0000 7f98a6800640 1 --2- 172.21.15.83:0/3190407497 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f98a810c3e0 0x7f98a819ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:53.331 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.329+0000 7f98a6800640 1 -- 172.21.15.83:0/3190407497 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98a81038e0 msgr2=0x7f98a8198b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:53.331 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.329+0000 7f98a6800640 1 --2- 172.21.15.83:0/3190407497 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98a81038e0 0x7f98a8198b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:53.331 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.329+0000 7f98a6800640 1 -- 172.21.15.83:0/3190407497 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98a819d600 con 0x7f98a8102fc0 2024-04-24T08:56:53.331 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.329+0000 7f98a7200640 1 --2- 172.21.15.83:0/3190407497 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f98a810c3e0 0x7f98a819ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:56:53.332 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.330+0000 7f98a6800640 1 --2- 172.21.15.83:0/3190407497 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f98a8102fc0 0x7f98a8198610 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f989800d950 tx=0x7f989800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:53.332 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.330+0000 7f989f400640 1 -- 172.21.15.83:0/3190407497 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9898017030 con 0x7f98a8102fc0 2024-04-24T08:56:53.332 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.330+0000 7f98aca00640 1 -- 172.21.15.83:0/3190407497 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98a819d8f0 con 0x7f98a8102fc0 2024-04-24T08:56:53.332 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.330+0000 7f989f400640 1 -- 172.21.15.83:0/3190407497 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9898004020 con 0x7f98a8102fc0 2024-04-24T08:56:53.332 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.330+0000 7f989f400640 1 -- 172.21.15.83:0/3190407497 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9898005640 con 0x7f98a8102fc0 2024-04-24T08:56:53.332 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.330+0000 7f98aca00640 1 -- 172.21.15.83:0/3190407497 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98a81a5180 con 0x7f98a8102fc0 2024-04-24T08:56:53.335 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.335+0000 7f989f400640 1 -- 172.21.15.83:0/3190407497 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f98980041d0 con 0x7f98a8102fc0 2024-04-24T08:56:53.335 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.335+0000 7f98b0372740 1 Processor -- start 2024-04-24T08:56:53.335 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.335+0000 7f98b0372740 1 -- start start 2024-04-24T08:56:53.335 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.335+0000 7f98b0372740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d09f8a03c0 0x55d09f8a07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:53.336 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.335+0000 7f98b0372740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d09f8a0ce0 0x55d09f8ab060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:53.336 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.335+0000 7f98b0372740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d09f8ab5a0 0x55d09f8ad990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:53.336 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.335+0000 7f98b0372740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d09f806900 con 0x55d09f8a0ce0 2024-04-24T08:56:53.336 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.335+0000 7f98b0372740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d09f7acf20 con 0x55d09f8ab5a0 2024-04-24T08:56:53.336 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.335+0000 7f98b0372740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d09f8028c0 con 0x55d09f8a03c0 2024-04-24T08:56:53.337 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.336+0000 7f98a6800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d09f8a0ce0 0x55d09f8ab060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:53.337 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.336+0000 7f98a6800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d09f8a0ce0 0x55d09f8ab060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40774/0 (socket says 172.21.15.83:40774) 2024-04-24T08:56:53.338 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.336+0000 7f98a6800640 1 -- 172.21.15.83:0/4000218185 learned_addr learned my addr 172.21.15.83:0/4000218185 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:56:53.338 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.336+0000 7f98a7200640 1 --2- 172.21.15.83:0/4000218185 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d09f8a03c0 0x55d09f8a07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:53.338 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.336+0000 7f98a5e00640 1 --2- 172.21.15.83:0/4000218185 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d09f8ab5a0 0x55d09f8ad990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:53.338 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.336+0000 7f98a6800640 1 -- 172.21.15.83:0/4000218185 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d09f8a03c0 msgr2=0x55d09f8a07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:53.338 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.336+0000 7f98a6800640 1 --2- 172.21.15.83:0/4000218185 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d09f8a03c0 0x55d09f8a07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:53.338 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.336+0000 7f98a6800640 1 -- 172.21.15.83:0/4000218185 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d09f8ab5a0 msgr2=0x55d09f8ad990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:53.338 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.336+0000 7f98a6800640 1 --2- 172.21.15.83:0/4000218185 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d09f8ab5a0 0x55d09f8ad990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:53.338 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.336+0000 7f98a6800640 1 -- 172.21.15.83:0/4000218185 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d09f7d1e10 con 0x55d09f8a0ce0 2024-04-24T08:56:53.338 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.336+0000 7f98a5e00640 1 --2- 172.21.15.83:0/4000218185 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d09f8ab5a0 0x55d09f8ad990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:56:53.338 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.337+0000 7f989f400640 1 --2- 172.21.15.83:0/3190407497 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f988c0a3eb0 0x7f988c0a6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:53.338 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.337+0000 7f98a6800640 1 --2- 172.21.15.83:0/4000218185 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d09f8a0ce0 0x55d09f8ab060 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f98980cd9d0 tx=0x7f98980d6f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9773b43dfcb03ad1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:53.338 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.337+0000 7f989f400640 1 -- 172.21.15.83:0/3190407497 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f98980d06e0 con 0x7f98a8102fc0 2024-04-24T08:56:53.338 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.337+0000 7f98a7200640 1 --2- 172.21.15.83:0/3190407497 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f988c0a3eb0 0x7f988c0a6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:53.339 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.337+0000 7f9893400640 1 -- 172.21.15.83:0/4000218185 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9898098550 con 0x55d09f8a0ce0 2024-04-24T08:56:53.339 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.337+0000 7f9893400640 1 -- 172.21.15.83:0/4000218185 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f989809c9b0 con 0x55d09f8a0ce0 2024-04-24T08:56:53.339 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.337+0000 7f9893400640 1 -- 172.21.15.83:0/4000218185 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f989809bc10 con 0x55d09f8a0ce0 2024-04-24T08:56:53.341 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.338+0000 7f98b0372740 1 -- 172.21.15.83:0/4000218185 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d09f8a0ce0 msgr2=0x55d09f8ab060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:53.341 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.338+0000 7f98b0372740 1 --2- 172.21.15.83:0/4000218185 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d09f8a0ce0 0x55d09f8ab060 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f98980cd9d0 tx=0x7f98980d6f10 comp rx=0 tx=0).stop 2024-04-24T08:56:53.341 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.338+0000 7f98a7200640 1 --2- 172.21.15.83:0/3190407497 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f988c0a3eb0 0x7f988c0a6370 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f98a0005fd0 tx=0x7f98a0009680 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:53.341 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.338+0000 7f98b0372740 1 -- 172.21.15.83:0/4000218185 shutdown_connections 2024-04-24T08:56:53.342 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.338+0000 7f98b0372740 1 --2- 172.21.15.83:0/4000218185 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d09f8ab5a0 0x55d09f8ad990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:53.342 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.338+0000 7f98b0372740 1 --2- 172.21.15.83:0/4000218185 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d09f8a0ce0 0x55d09f8ab060 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:53.342 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.338+0000 7f98b0372740 1 --2- 172.21.15.83:0/4000218185 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d09f8a03c0 0x55d09f8a07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:53.342 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.338+0000 7f98b0372740 1 -- 172.21.15.83:0/4000218185 >> 172.21.15.83:0/4000218185 conn(0x55d09f89c8d0 msgr2=0x55d09f89ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:53.342 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.339+0000 7f98b0372740 1 -- 172.21.15.83:0/4000218185 shutdown_connections 2024-04-24T08:56:53.342 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.339+0000 7f98b0372740 1 -- 172.21.15.83:0/4000218185 wait complete. 2024-04-24T08:56:53.342 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.339+0000 7f98b0372740 1 -- start start 2024-04-24T08:56:53.342 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.339+0000 7f98b0372740 1 Processor -- start 2024-04-24T08:56:53.342 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.340+0000 7f98b0372740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d09f8a03c0 0x55d09f8a8530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:53.342 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.340+0000 7f98b0372740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d09f8a0ce0 0x55d09f8a8a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:53.342 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.340+0000 7f98b0372740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d09f8ab5a0 0x55d09f8a8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:53.342 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.340+0000 7f98b0372740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d09f8ae1d0 con 0x55d09f8ab5a0 2024-04-24T08:56:53.343 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.340+0000 7f98b0372740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d09f7c02c0 con 0x55d09f8a03c0 2024-04-24T08:56:53.343 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.340+0000 7f98b0372740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d09f8ae350 con 0x55d09f8a0ce0 2024-04-24T08:56:53.343 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.340+0000 7f98b0372740 10 client.0 _ll_register_callbacks cb 0x55d09f518a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:56:53.343 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.340+0000 7f98a7200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d09f8ab5a0 0x55d09f8a8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:53.343 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.340+0000 7f98a7200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d09f8ab5a0 0x55d09f8a8fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40786/0 (socket says 172.21.15.83:40786) 2024-04-24T08:56:53.343 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.340+0000 7f98a7200640 1 -- 172.21.15.83:0/3183673415 learned_addr learned my addr 172.21.15.83:0/3183673415 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:56:53.343 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.340+0000 7f98a6800640 1 --2- 172.21.15.83:0/3183673415 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d09f8a03c0 0x55d09f8a8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:53.343 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.341+0000 7f98a5e00640 1 --2- 172.21.15.83:0/3183673415 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d09f8a0ce0 0x55d09f8a8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:53.343 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.341+0000 7f98a7200640 1 -- 172.21.15.83:0/3183673415 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d09f8a0ce0 msgr2=0x55d09f8a8a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:53.343 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.341+0000 7f98a7200640 1 --2- 172.21.15.83:0/3183673415 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d09f8a0ce0 0x55d09f8a8a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:53.343 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.341+0000 7f98a7200640 1 -- 172.21.15.83:0/3183673415 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d09f8a03c0 msgr2=0x55d09f8a8530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:53.343 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.341+0000 7f98a7200640 1 --2- 172.21.15.83:0/3183673415 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d09f8a03c0 0x55d09f8a8530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:53.343 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.341+0000 7f98a7200640 1 -- 172.21.15.83:0/3183673415 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d09f7d0fd0 con 0x55d09f8ab5a0 2024-04-24T08:56:53.343 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.341+0000 7f98a7200640 1 --2- 172.21.15.83:0/3183673415 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d09f8ab5a0 0x55d09f8a8fb0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f98a0012960 tx=0x7f98a0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5dd5b6637c55b67 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:53.344 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.341+0000 7f9887e00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:56:53.344 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.341+0000 7f9887e00640 1 -- 172.21.15.83:0/3183673415 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98a000fcf0 con 0x55d09f8ab5a0 2024-04-24T08:56:53.344 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.341+0000 7f9887e00640 1 -- 172.21.15.83:0/3183673415 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f98a001f420 con 0x55d09f8ab5a0 2024-04-24T08:56:53.344 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.341+0000 7f98b0372740 1 -- 172.21.15.83:0/3183673415 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f989809c9b0 con 0x55d09f8ab5a0 2024-04-24T08:56:53.345 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.342+0000 7f9887e00640 1 -- 172.21.15.83:0/3183673415 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f98a0030050 con 0x55d09f8ab5a0 2024-04-24T08:56:53.345 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.342+0000 7f98b0372740 10 client.17601 fetch_fsmap learned FSMap version 33 2024-04-24T08:56:53.345 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.342+0000 7f98b0372740 1 -- 172.21.15.83:0/3183673415 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55d09f519210 con 0x55d09f8ab5a0 2024-04-24T08:56:53.346 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.342+0000 7f9887e00640 1 -- 172.21.15.83:0/3183673415 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f989809c9b0 con 0x55d09f8ab5a0 2024-04-24T08:56:53.346 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.342+0000 7f9887e00640 1 -- 172.21.15.83:0/3183673415 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f98a0027630 con 0x55d09f8ab5a0 2024-04-24T08:56:53.346 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.343+0000 7f98b0372740 10 client.17601 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T08:56:53.346 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.343+0000 7f98b0372740 10 client.17601 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T08:56:53.346 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.343+0000 7f98b0372740 20 client.17601 populate_metadata read hostname 'smithi083' 2024-04-24T08:56:53.346 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.343+0000 7f98b0372740 1 --2- 172.21.15.83:0/3183673415 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d09f8bfb90 0x55d09f8c1f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:53.346 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.343+0000 7f98b0372740 4 client.17601 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:56:53.346 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.343+0000 7f98b0372740 1 -- 172.21.15.83:0/3183673415 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d09f814320 con 0x55d09f8bfb90 2024-04-24T08:56:53.346 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.344+0000 7f98a6800640 1 --2- 172.21.15.83:0/3183673415 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d09f8bfb90 0x55d09f8c1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:53.346 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.345+0000 7f98a6800640 1 --2- 172.21.15.83:0/3183673415 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d09f8bfb90 0x55d09f8c1f80 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb7268a1ba9ff24b server_cookie=2054a90e2afbac82 in_seq=0 out_seq=0 2024-04-24T08:56:53.346 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.345+0000 7f9887e00640 10 client.17601 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:56:53.350 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.349+0000 7f9887e00640 1 -- 172.21.15.83:0/3183673415 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d09f814320 con 0x55d09f8bfb90 2024-04-24T08:56:53.350 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.349+0000 7f9887e00640 10 client.17601 handle_command_reply: tid=0 2024-04-24T08:56:53.350 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.349+0000 7f98b0372740 1 client.17601 shutdown 2024-04-24T08:56:53.350 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.349+0000 7f98b0372740 1 -- 172.21.15.83:0/3183673415 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d09f8ab5a0 msgr2=0x55d09f8a8fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:53.350 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.349+0000 7f98b0372740 1 --2- 172.21.15.83:0/3183673415 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d09f8ab5a0 0x55d09f8a8fb0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f98a0012960 tx=0x7f98a0004a40 comp rx=0 tx=0).stop 2024-04-24T08:56:53.350 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.349+0000 7f98b0372740 1 -- 172.21.15.83:0/3183673415 shutdown_connections 2024-04-24T08:56:53.350 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.349+0000 7f98b0372740 1 --2- 172.21.15.83:0/3183673415 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d09f8bfb90 0x55d09f8c1f80 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:53.350 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.349+0000 7f98b0372740 1 --2- 172.21.15.83:0/3183673415 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d09f8ab5a0 0x55d09f8a8fb0 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:53.350 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.349+0000 7f98b0372740 1 --2- 172.21.15.83:0/3183673415 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d09f8a0ce0 0x55d09f8a8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:53.350 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.349+0000 7f98b0372740 1 --2- 172.21.15.83:0/3183673415 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d09f8a03c0 0x55d09f8a8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:53.350 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.349+0000 7f9887e00640 0 client.17601 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:56:53.350 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.349+0000 7f98b0372740 1 -- 172.21.15.83:0/3183673415 >> 172.21.15.83:0/3183673415 conn(0x55d09f89c8d0 msgr2=0x55d09f89ba90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:53.350 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.350+0000 7f98b0372740 1 -- 172.21.15.83:0/3183673415 shutdown_connections 2024-04-24T08:56:53.351 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.350+0000 7f98b0372740 1 -- 172.21.15.83:0/3183673415 wait complete. 2024-04-24T08:56:53.351 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.350+0000 7f98b0372740 20 client.17601 trim_cache size 0 max 16384 2024-04-24T08:56:53.383 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.383+0000 7f98b0372740 1 Processor -- start 2024-04-24T08:56:53.383 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.383+0000 7f98b0372740 1 -- start start 2024-04-24T08:56:53.385 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.384+0000 7f98b0372740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d09f8a03c0 0x55d09f8bb4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:53.385 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.384+0000 7f98b0372740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d09f8a0ce0 0x55d09f8a2480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:53.385 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.384+0000 7f98b0372740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d09f8ab5a0 0x55d09f8c7870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:53.385 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.384+0000 7f98b0372740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f98a81ad190 con 0x55d09f8a03c0 2024-04-24T08:56:53.385 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.384+0000 7f98b0372740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f98a81aecd0 con 0x55d09f8a0ce0 2024-04-24T08:56:53.386 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.384+0000 7f98b0372740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f98a81ad860 con 0x55d09f8ab5a0 2024-04-24T08:56:53.386 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.384+0000 7f98a6800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d09f8a03c0 0x55d09f8bb4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:53.386 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.384+0000 7f98a6800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d09f8a03c0 0x55d09f8bb4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40798/0 (socket says 172.21.15.83:40798) 2024-04-24T08:56:53.386 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.384+0000 7f98a6800640 1 -- 172.21.15.83:0/366697646 learned_addr learned my addr 172.21.15.83:0/366697646 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:56:53.386 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.385+0000 7f98a7200640 1 --2- 172.21.15.83:0/366697646 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d09f8ab5a0 0x55d09f8c7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:53.386 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.385+0000 7f98a6800640 1 -- 172.21.15.83:0/366697646 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d09f8ab5a0 msgr2=0x55d09f8c7870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:53.386 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.385+0000 7f98a6800640 1 --2- 172.21.15.83:0/366697646 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d09f8ab5a0 0x55d09f8c7870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:53.386 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.385+0000 7f98a6800640 1 -- 172.21.15.83:0/366697646 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d09f8a0ce0 msgr2=0x55d09f8a2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:53.386 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.385+0000 7f98a6800640 1 --2- 172.21.15.83:0/366697646 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d09f8a0ce0 0x55d09f8a2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:53.386 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.385+0000 7f98a6800640 1 -- 172.21.15.83:0/366697646 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d09f5e5620 con 0x55d09f8a03c0 2024-04-24T08:56:53.386 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.385+0000 7f98a6800640 1 --2- 172.21.15.83:0/366697646 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d09f8a03c0 0x55d09f8bb4e0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f9898098c80 tx=0x7f989809c2b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60eff252ce5def06 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:53.386 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.385+0000 7f9892a00640 1 -- 172.21.15.83:0/366697646 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f989809bdf0 con 0x55d09f8a03c0 2024-04-24T08:56:53.386 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.385+0000 7f9892a00640 1 -- 172.21.15.83:0/366697646 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f989809c9b0 con 0x55d09f8a03c0 2024-04-24T08:56:53.387 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.385+0000 7f9892a00640 1 -- 172.21.15.83:0/366697646 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f989809c080 con 0x55d09f8a03c0 2024-04-24T08:56:53.389 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.386+0000 7f98b0372740 1 -- 172.21.15.83:0/366697646 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d09f8a03c0 msgr2=0x55d09f8bb4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:53.389 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.386+0000 7f98b0372740 1 --2- 172.21.15.83:0/366697646 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d09f8a03c0 0x55d09f8bb4e0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f9898098c80 tx=0x7f989809c2b0 comp rx=0 tx=0).stop 2024-04-24T08:56:53.389 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.386+0000 7f98b0372740 1 -- 172.21.15.83:0/366697646 shutdown_connections 2024-04-24T08:56:53.389 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.386+0000 7f98b0372740 1 --2- 172.21.15.83:0/366697646 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d09f8ab5a0 0x55d09f8c7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:53.389 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.386+0000 7f98b0372740 1 --2- 172.21.15.83:0/366697646 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d09f8a0ce0 0x55d09f8a2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:53.389 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.386+0000 7f98b0372740 1 --2- 172.21.15.83:0/366697646 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d09f8a03c0 0x55d09f8bb4e0 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:53.390 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.386+0000 7f98b0372740 1 -- 172.21.15.83:0/366697646 >> 172.21.15.83:0/366697646 conn(0x55d09f89c8d0 msgr2=0x55d09f8a1d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:53.390 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.386+0000 7f98b0372740 1 -- 172.21.15.83:0/366697646 shutdown_connections 2024-04-24T08:56:53.390 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.386+0000 7f98b0372740 1 -- 172.21.15.83:0/366697646 wait complete. 2024-04-24T08:56:53.390 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.387+0000 7f98b0372740 1 -- start start 2024-04-24T08:56:53.390 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.387+0000 7f98b0372740 1 Processor -- start 2024-04-24T08:56:53.390 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.388+0000 7f98b0372740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d09f8a03c0 0x55d09f8b7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:53.390 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.388+0000 7f98b0372740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d09f8a0ce0 0x55d09f8b49c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:53.390 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.388+0000 7f98b0372740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d09f8ab5a0 0x55d09f8b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:53.390 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.388+0000 7f98b0372740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f98a81ae600 con 0x55d09f8a0ce0 2024-04-24T08:56:53.390 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.388+0000 7f98b0372740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f98a81adf30 con 0x55d09f8ab5a0 2024-04-24T08:56:53.390 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.388+0000 7f98b0372740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f98a81ab170 con 0x55d09f8a03c0 2024-04-24T08:56:53.390 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.388+0000 7f98b0372740 10 client.0 _ll_register_callbacks cb 0x7f98a00125e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:56:53.390 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.388+0000 7f98a5e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d09f8a0ce0 0x55d09f8b49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:53.390 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.388+0000 7f98a5e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d09f8a0ce0 0x55d09f8b49c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40812/0 (socket says 172.21.15.83:40812) 2024-04-24T08:56:53.391 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.388+0000 7f98a5e00640 1 -- 172.21.15.83:0/2573512020 learned_addr learned my addr 172.21.15.83:0/2573512020 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:56:53.391 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.388+0000 7f98a7200640 1 --2- 172.21.15.83:0/2573512020 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d09f8ab5a0 0x55d09f8b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:53.391 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.388+0000 7f98a6800640 1 --2- 172.21.15.83:0/2573512020 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d09f8a03c0 0x55d09f8b7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:53.391 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.389+0000 7f98a5e00640 1 -- 172.21.15.83:0/2573512020 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d09f8a03c0 msgr2=0x55d09f8b7cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:53.391 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.389+0000 7f98a5e00640 1 --2- 172.21.15.83:0/2573512020 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d09f8a03c0 0x55d09f8b7cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:53.391 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.389+0000 7f98a5e00640 1 -- 172.21.15.83:0/2573512020 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d09f8ab5a0 msgr2=0x55d09f8b4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:56:53.391 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.389+0000 7f98a5e00640 1 --2- 172.21.15.83:0/2573512020 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d09f8ab5a0 0x55d09f8b4f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:53.391 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.389+0000 7f98a5e00640 1 -- 172.21.15.83:0/2573512020 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d09f8b5640 con 0x55d09f8a0ce0 2024-04-24T08:56:53.391 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.389+0000 7f98a6800640 1 --2- 172.21.15.83:0/2573512020 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d09f8a03c0 0x55d09f8b7cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:56:53.391 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.389+0000 7f98a5e00640 1 --2- 172.21.15.83:0/2573512020 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d09f8a0ce0 0x55d09f8b49c0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f9894005940 tx=0x7f98940316d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6364deb859223762 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:56:53.391 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.389+0000 7f9887e00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:56:53.391 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.389+0000 7f9887e00640 1 -- 172.21.15.83:0/2573512020 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9894035050 con 0x55d09f8a0ce0 2024-04-24T08:56:53.391 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.389+0000 7f98b0372740 1 -- 172.21.15.83:0/2573512020 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f989809c9b0 con 0x55d09f8a0ce0 2024-04-24T08:56:53.391 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.389+0000 7f9887e00640 1 -- 172.21.15.83:0/2573512020 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f9894004240 con 0x55d09f8a0ce0 2024-04-24T08:56:53.392 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.390+0000 7f9887e00640 1 -- 172.21.15.83:0/2573512020 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f9894004440 con 0x55d09f8a0ce0 2024-04-24T08:56:53.393 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.390+0000 7f98b0372740 10 client.17607 fetch_fsmap learned FSMap version 33 2024-04-24T08:56:53.393 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.390+0000 7f98b0372740 1 -- 172.21.15.83:0/2573512020 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55d09f927080 con 0x55d09f8a0ce0 2024-04-24T08:56:53.393 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.390+0000 7f9887e00640 1 -- 172.21.15.83:0/2573512020 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f989809c9b0 con 0x55d09f8a0ce0 2024-04-24T08:56:53.393 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.390+0000 7f9887e00640 1 -- 172.21.15.83:0/2573512020 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f9894042070 con 0x55d09f8a0ce0 2024-04-24T08:56:53.393 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.390+0000 7f98b0372740 10 client.17607 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T08:56:53.393 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.390+0000 7f98b0372740 10 client.17607 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T08:56:53.393 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.390+0000 7f98b0372740 20 client.17607 populate_metadata read hostname 'smithi083' 2024-04-24T08:56:53.393 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.390+0000 7f98b0372740 1 --2- 172.21.15.83:0/2573512020 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d09f927830 0x55d09f929c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:56:53.393 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.390+0000 7f98b0372740 4 client.17607 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T08:56:53.393 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.390+0000 7f98b0372740 1 -- 172.21.15.83:0/2573512020 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d09f80e190 con 0x55d09f927830 2024-04-24T08:56:53.393 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.391+0000 7f98a6800640 1 --2- 172.21.15.83:0/2573512020 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d09f927830 0x55d09f929c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:56:53.393 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.392+0000 7f9887e00640 10 client.17607 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:56:53.393 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.392+0000 7f98a6800640 1 --2- 172.21.15.83:0/2573512020 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d09f927830 0x55d09f929c50 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=463795be3db2d18b server_cookie=9b80452f967dfaed in_seq=0 out_seq=0 2024-04-24T08:56:53.443 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.443+0000 7f9887e00640 1 -- 172.21.15.83:0/2573512020 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d09f80e190 con 0x55d09f927830 2024-04-24T08:56:53.443 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.443+0000 7f9887e00640 10 client.17607 handle_command_reply: tid=0 2024-04-24T08:56:53.443 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.443+0000 7f98b0372740 1 client.17607 shutdown 2024-04-24T08:56:53.444 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.444+0000 7f98b0372740 1 -- 172.21.15.83:0/2573512020 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d09f8a0ce0 msgr2=0x55d09f8b49c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:53.445 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.444+0000 7f98b0372740 1 --2- 172.21.15.83:0/2573512020 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d09f8a0ce0 0x55d09f8b49c0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f9894005940 tx=0x7f98940316d0 comp rx=0 tx=0).stop 2024-04-24T08:56:53.445 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.444+0000 7f98b0372740 1 -- 172.21.15.83:0/2573512020 shutdown_connections 2024-04-24T08:56:53.445 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.444+0000 7f98b0372740 1 --2- 172.21.15.83:0/2573512020 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d09f927830 0x55d09f929c50 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:53.445 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.444+0000 7f98b0372740 1 --2- 172.21.15.83:0/2573512020 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d09f8ab5a0 0x55d09f8b4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:53.445 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.444+0000 7f98b0372740 1 --2- 172.21.15.83:0/2573512020 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d09f8a0ce0 0x55d09f8b49c0 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:53.445 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.444+0000 7f9887e00640 0 client.17607 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:56:53.445 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.444+0000 7f98b0372740 1 --2- 172.21.15.83:0/2573512020 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d09f8a03c0 0x55d09f8b7cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:53.445 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.444+0000 7f98b0372740 1 -- 172.21.15.83:0/2573512020 >> 172.21.15.83:0/2573512020 conn(0x55d09f89c8d0 msgr2=0x55d09f8a8d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:53.445 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.444+0000 7f98b0372740 1 -- 172.21.15.83:0/2573512020 shutdown_connections 2024-04-24T08:56:53.445 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.444+0000 7f98b0372740 1 -- 172.21.15.83:0/2573512020 wait complete. 2024-04-24T08:56:53.445 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.444+0000 7f98b0372740 20 client.17607 trim_cache size 0 max 16384 2024-04-24T08:56:53.446 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T08:56:53.446 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T08:56:53.446 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T08:56:53.446 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T08:56:53.447 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.445+0000 7f98aca00640 1 -- 172.21.15.83:0/3190407497 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f988c0a3eb0 msgr2=0x7f988c0a6370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:53.447 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.445+0000 7f98aca00640 1 --2- 172.21.15.83:0/3190407497 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f988c0a3eb0 0x7f988c0a6370 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f98a0005fd0 tx=0x7f98a0009680 comp rx=0 tx=0).stop 2024-04-24T08:56:53.447 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.445+0000 7f98aca00640 1 -- 172.21.15.83:0/3190407497 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f98a8102fc0 msgr2=0x7f98a8198610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:56:53.447 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.445+0000 7f98aca00640 1 --2- 172.21.15.83:0/3190407497 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f98a8102fc0 0x7f98a8198610 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f989800d950 tx=0x7f989800de10 comp rx=0 tx=0).stop 2024-04-24T08:56:53.447 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.446+0000 7f98aca00640 1 -- 172.21.15.83:0/3190407497 shutdown_connections 2024-04-24T08:56:53.447 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.446+0000 7f98aca00640 1 --2- 172.21.15.83:0/3190407497 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f988c0a3eb0 0x7f988c0a6370 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:53.447 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.446+0000 7f98aca00640 1 --2- 172.21.15.83:0/3190407497 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f98a810c3e0 0x7f98a819ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:53.447 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.446+0000 7f98aca00640 1 --2- 172.21.15.83:0/3190407497 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98a81038e0 0x7f98a8198b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:53.447 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.446+0000 7f98aca00640 1 --2- 172.21.15.83:0/3190407497 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f98a8102fc0 0x7f98a8198610 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:56:53.447 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.446+0000 7f98aca00640 1 -- 172.21.15.83:0/3190407497 >> 172.21.15.83:0/3190407497 conn(0x7f98a80fc820 msgr2=0x7f98a80fe1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:56:53.447 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.446+0000 7f98aca00640 1 -- 172.21.15.83:0/3190407497 shutdown_connections 2024-04-24T08:56:53.447 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:56:53.446+0000 7f98aca00640 1 -- 172.21.15.83:0/3190407497 wait complete. 2024-04-24T08:56:53.577 INFO:tasks.workunit.client.0.smithi083.stdout: 1 25294 0.66 MB/sec execute 87 sec latency 49.434 ms 2024-04-24T08:56:54.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:56:53 smithi138 ceph-mon[31441]: pgmap v161: 129 pgs: 129 active+clean; 40 MiB data, 568 MiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 646 KiB/s wr, 74 op/s 2024-04-24T08:56:54.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:56:53 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:56:54.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:56:53 smithi189 ceph-mon[31791]: pgmap v161: 129 pgs: 129 active+clean; 40 MiB data, 568 MiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 646 KiB/s wr, 74 op/s 2024-04-24T08:56:54.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:56:53 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:56:54.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:56:53 smithi083 ceph-mon[27907]: pgmap v161: 129 pgs: 129 active+clean; 40 MiB data, 568 MiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 646 KiB/s wr, 74 op/s 2024-04-24T08:56:54.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:56:53 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:56:54.577 INFO:tasks.workunit.client.0.smithi083.stdout: 1 25374 0.65 MB/sec execute 88 sec latency 50.479 ms 2024-04-24T08:56:55.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:56:54 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:56:55.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:56:54 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:56:55.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:56:54 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T08:56:55.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:56:54 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:56:55.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:56:54 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:56:55.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:56:54 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T08:56:55.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:56:54 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:56:55.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:56:54 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:56:55.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:56:54 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T08:56:55.577 INFO:tasks.workunit.client.0.smithi083.stdout: 1 25455 0.65 MB/sec execute 89 sec latency 53.662 ms 2024-04-24T08:56:56.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:56:55 smithi138 ceph-mon[31441]: pgmap v162: 129 pgs: 129 active+clean; 40 MiB data, 568 MiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 444 KiB/s wr, 51 op/s 2024-04-24T08:56:56.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:56:55 smithi189 ceph-mon[31791]: pgmap v162: 129 pgs: 129 active+clean; 40 MiB data, 568 MiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 444 KiB/s wr, 51 op/s 2024-04-24T08:56:56.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:56:55 smithi083 ceph-mon[27907]: pgmap v162: 129 pgs: 129 active+clean; 40 MiB data, 568 MiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 444 KiB/s wr, 51 op/s 2024-04-24T08:56:56.577 INFO:tasks.workunit.client.0.smithi083.stdout: 1 25542 0.64 MB/sec execute 90 sec latency 51.490 ms 2024-04-24T08:56:57.577 INFO:tasks.workunit.client.0.smithi083.stdout: 1 25684 0.64 MB/sec execute 91 sec latency 53.350 ms 2024-04-24T08:56:58.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:56:57 smithi138 ceph-mon[31441]: pgmap v163: 129 pgs: 129 active+clean; 40 MiB data, 568 MiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 444 KiB/s wr, 51 op/s 2024-04-24T08:56:58.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:56:57 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T08:56:58.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:56:57 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T08:56:58.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:56:57 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T08:56:58.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:56:57 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T08:56:58.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:56:57 smithi189 ceph-mon[31791]: pgmap v163: 129 pgs: 129 active+clean; 40 MiB data, 568 MiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 444 KiB/s wr, 51 op/s 2024-04-24T08:56:58.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:56:57 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T08:56:58.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:56:57 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T08:56:58.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:56:57 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T08:56:58.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:56:57 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T08:56:58.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:56:57 smithi083 ceph-mon[27907]: pgmap v163: 129 pgs: 129 active+clean; 40 MiB data, 568 MiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 444 KiB/s wr, 51 op/s 2024-04-24T08:56:58.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:56:57 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T08:56:58.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:56:57 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T08:56:58.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:56:57 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T08:56:58.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:56:57 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T08:56:58.577 INFO:tasks.workunit.client.0.smithi083.stdout: 1 25801 0.63 MB/sec execute 92 sec latency 49.597 ms 2024-04-24T08:56:59.577 INFO:tasks.workunit.client.0.smithi083.stdout: 1 25917 0.64 MB/sec execute 93 sec latency 53.336 ms 2024-04-24T08:57:00.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:56:59 smithi138 ceph-mon[31441]: pgmap v164: 129 pgs: 129 active+clean; 40 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 545 KiB/s wr, 80 op/s 2024-04-24T08:57:00.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:56:59 smithi189 ceph-mon[31791]: pgmap v164: 129 pgs: 129 active+clean; 40 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 545 KiB/s wr, 80 op/s 2024-04-24T08:57:00.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:56:59 smithi083 ceph-mon[27907]: pgmap v164: 129 pgs: 129 active+clean; 40 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 545 KiB/s wr, 80 op/s 2024-04-24T08:57:00.577 INFO:tasks.workunit.client.0.smithi083.stdout: 1 26029 0.63 MB/sec execute 94 sec latency 55.549 ms 2024-04-24T08:57:01.577 INFO:tasks.workunit.client.0.smithi083.stdout: 1 26129 0.63 MB/sec execute 95 sec latency 57.360 ms 2024-04-24T08:57:02.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:57:01 smithi138 ceph-mon[31441]: pgmap v165: 129 pgs: 129 active+clean; 40 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 363 KiB/s wr, 61 op/s 2024-04-24T08:57:02.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:57:01 smithi189 ceph-mon[31791]: pgmap v165: 129 pgs: 129 active+clean; 40 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 363 KiB/s wr, 61 op/s 2024-04-24T08:57:02.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:57:01 smithi083 ceph-mon[27907]: pgmap v165: 129 pgs: 129 active+clean; 40 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 363 KiB/s wr, 61 op/s 2024-04-24T08:57:02.578 INFO:tasks.workunit.client.0.smithi083.stdout: 1 26298 0.65 MB/sec execute 96 sec latency 59.566 ms 2024-04-24T08:57:03.578 INFO:tasks.workunit.client.0.smithi083.stdout: 1 26408 0.64 MB/sec execute 97 sec latency 49.370 ms 2024-04-24T08:57:04.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:57:03 smithi138 ceph-mon[31441]: pgmap v166: 129 pgs: 129 active+clean; 39 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 558 KiB/s wr, 103 op/s 2024-04-24T08:57:04.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:57:03 smithi189 ceph-mon[31791]: pgmap v166: 129 pgs: 129 active+clean; 39 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 558 KiB/s wr, 103 op/s 2024-04-24T08:57:04.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:57:03 smithi083 ceph-mon[27907]: pgmap v166: 129 pgs: 129 active+clean; 39 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 558 KiB/s wr, 103 op/s 2024-04-24T08:57:04.578 INFO:tasks.workunit.client.0.smithi083.stdout: 1 26525 0.64 MB/sec execute 98 sec latency 52.156 ms 2024-04-24T08:57:05.578 INFO:tasks.workunit.client.0.smithi083.stdout: 1 26687 0.63 MB/sec execute 99 sec latency 52.759 ms 2024-04-24T08:57:06.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:57:05 smithi138 ceph-mon[31441]: pgmap v167: 129 pgs: 129 active+clean; 39 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 296 KiB/s wr, 71 op/s 2024-04-24T08:57:06.210 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:57:05 smithi189 ceph-mon[31791]: pgmap v167: 129 pgs: 129 active+clean; 39 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 296 KiB/s wr, 71 op/s 2024-04-24T08:57:06.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:57:05 smithi083 ceph-mon[27907]: pgmap v167: 129 pgs: 129 active+clean; 39 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 296 KiB/s wr, 71 op/s 2024-04-24T08:57:06.578 INFO:tasks.workunit.client.0.smithi083.stdout: 1 26793 0.63 MB/sec execute 100 sec latency 51.861 ms 2024-04-24T08:57:06.603 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.602+0000 7f595d800640 1 -- 172.21.15.83:0/3017921166 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5958101910 msgr2=0x7f5958105950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:06.603 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.602+0000 7f595d800640 1 --2- 172.21.15.83:0/3017921166 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5958101910 0x7f5958105950 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f5940009900 tx=0x7f594002a3f0 comp rx=0 tx=0).stop 2024-04-24T08:57:06.603 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.602+0000 7f595d800640 1 -- 172.21.15.83:0/3017921166 shutdown_connections 2024-04-24T08:57:06.603 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.602+0000 7f595d800640 1 --2- 172.21.15.83:0/3017921166 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5958106030 0x7f595810dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:06.603 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.602+0000 7f595d800640 1 --2- 172.21.15.83:0/3017921166 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5958101910 0x7f5958105950 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:06.603 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.602+0000 7f595d800640 1 --2- 172.21.15.83:0/3017921166 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5958100f60 0x7f5958101340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:06.603 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.602+0000 7f595d800640 1 -- 172.21.15.83:0/3017921166 >> 172.21.15.83:0/3017921166 conn(0x7f59580fc820 msgr2=0x7f59580fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:06.603 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.602+0000 7f595d800640 1 -- 172.21.15.83:0/3017921166 shutdown_connections 2024-04-24T08:57:06.603 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.602+0000 7f595d800640 1 -- 172.21.15.83:0/3017921166 wait complete. 2024-04-24T08:57:06.604 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.604+0000 7f595d800640 1 Processor -- start 2024-04-24T08:57:06.604 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.604+0000 7f595d800640 1 -- start start 2024-04-24T08:57:06.605 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.604+0000 7f595d800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5958100f60 0x7f59581a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:06.605 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.604+0000 7f595d800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5958106030 0x7f59581a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:06.605 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.604+0000 7f595d800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f595819a710 0x7f595819ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:06.605 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.604+0000 7f595d800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5958112070 con 0x7f5958100f60 2024-04-24T08:57:06.605 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.604+0000 7f595d800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5958111ef0 con 0x7f595819a710 2024-04-24T08:57:06.605 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.604+0000 7f595d800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f59581121f0 con 0x7f5958106030 2024-04-24T08:57:06.605 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.605+0000 7f5957e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5958100f60 0x7f59581a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:06.605 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.605+0000 7f5957e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5958100f60 0x7f59581a0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43700/0 (socket says 172.21.15.83:43700) 2024-04-24T08:57:06.605 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.605+0000 7f5957e00640 1 -- 172.21.15.83:0/990008056 learned_addr learned my addr 172.21.15.83:0/990008056 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:57:06.606 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.605+0000 7f594fe00640 1 --2- 172.21.15.83:0/990008056 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5958106030 0x7f59581a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:06.607 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.605+0000 7f595ce00640 1 --2- 172.21.15.83:0/990008056 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f595819a710 0x7f595819ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:06.607 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.605+0000 7f5957e00640 1 -- 172.21.15.83:0/990008056 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5958106030 msgr2=0x7f59581a0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:06.607 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.605+0000 7f5957e00640 1 --2- 172.21.15.83:0/990008056 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5958106030 0x7f59581a0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:06.607 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.605+0000 7f5957e00640 1 -- 172.21.15.83:0/990008056 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f595819a710 msgr2=0x7f595819ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:06.607 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.605+0000 7f5957e00640 1 --2- 172.21.15.83:0/990008056 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f595819a710 0x7f595819ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:06.607 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.605+0000 7f5957e00640 1 -- 172.21.15.83:0/990008056 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f595819b310 con 0x7f5958100f60 2024-04-24T08:57:06.607 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.605+0000 7f595ce00640 1 --2- 172.21.15.83:0/990008056 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f595819a710 0x7f595819ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:57:06.607 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.606+0000 7f5957e00640 1 --2- 172.21.15.83:0/990008056 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5958100f60 0x7f59581a0650 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f594800c890 tx=0x7f594800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:06.607 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.606+0000 7f5955600640 1 -- 172.21.15.83:0/990008056 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5948004220 con 0x7f5958100f60 2024-04-24T08:57:06.607 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.606+0000 7f5955600640 1 -- 172.21.15.83:0/990008056 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f59480043c0 con 0x7f5958100f60 2024-04-24T08:57:06.607 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.606+0000 7f5955600640 1 -- 172.21.15.83:0/990008056 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5948004a00 con 0x7f5958100f60 2024-04-24T08:57:06.607 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.606+0000 7f595d800640 1 -- 172.21.15.83:0/990008056 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59581a73f0 con 0x7f5958100f60 2024-04-24T08:57:06.607 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.606+0000 7f595d800640 1 -- 172.21.15.83:0/990008056 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59581a77a0 con 0x7f5958100f60 2024-04-24T08:57:06.610 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.610+0000 7f5961c5a740 1 Processor -- start 2024-04-24T08:57:06.610 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.610+0000 7f5961c5a740 1 -- start start 2024-04-24T08:57:06.610 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.610+0000 7f5955600640 1 -- 172.21.15.83:0/990008056 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5948004ba0 con 0x7f5958100f60 2024-04-24T08:57:06.612 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.610+0000 7f5961c5a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56203c3ff3c0 0x56203c3ff7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:06.612 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.610+0000 7f5961c5a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56203c3ffce0 0x56203c40a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:06.612 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.610+0000 7f5961c5a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56203c40a5a0 0x56203c40c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:06.612 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.610+0000 7f5961c5a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56203c365900 con 0x56203c3ffce0 2024-04-24T08:57:06.612 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.610+0000 7f5961c5a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56203c30bf20 con 0x56203c3ff3c0 2024-04-24T08:57:06.612 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.610+0000 7f5961c5a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56203c3618c0 con 0x56203c40a5a0 2024-04-24T08:57:06.612 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.611+0000 7f5957e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56203c3ffce0 0x56203c40a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:06.612 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.611+0000 7f5957e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56203c3ffce0 0x56203c40a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43702/0 (socket says 172.21.15.83:43702) 2024-04-24T08:57:06.612 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.611+0000 7f5957e00640 1 -- 172.21.15.83:0/1767568339 learned_addr learned my addr 172.21.15.83:0/1767568339 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:57:06.612 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.611+0000 7f595ce00640 1 --2- 172.21.15.83:0/1767568339 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56203c3ff3c0 0x56203c3ff7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:06.612 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.611+0000 7f594fe00640 1 --2- 172.21.15.83:0/1767568339 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56203c40a5a0 0x56203c40c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:06.612 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.611+0000 7f5957e00640 1 -- 172.21.15.83:0/1767568339 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56203c40a5a0 msgr2=0x56203c40c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:06.613 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.611+0000 7f5957e00640 1 --2- 172.21.15.83:0/1767568339 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56203c40a5a0 0x56203c40c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:06.613 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.611+0000 7f5957e00640 1 -- 172.21.15.83:0/1767568339 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56203c3ff3c0 msgr2=0x56203c3ff7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:06.613 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.611+0000 7f5957e00640 1 --2- 172.21.15.83:0/1767568339 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56203c3ff3c0 0x56203c3ff7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:06.613 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.611+0000 7f5957e00640 1 -- 172.21.15.83:0/1767568339 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56203c330e10 con 0x56203c3ffce0 2024-04-24T08:57:06.613 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.612+0000 7f5957e00640 1 --2- 172.21.15.83:0/1767568339 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56203c3ffce0 0x56203c40a060 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f59480cdb90 tx=0x7f59480d8690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1dafd2658db58ca server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:06.613 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.612+0000 7f5955600640 1 --2- 172.21.15.83:0/990008056 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f593c0a3d90 0x7f593c0a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:06.615 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.612+0000 7f594d600640 1 -- 172.21.15.83:0/1767568339 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59480df070 con 0x56203c3ffce0 2024-04-24T08:57:06.615 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.612+0000 7f595ce00640 1 --2- 172.21.15.83:0/990008056 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f593c0a3d90 0x7f593c0a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:06.615 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.612+0000 7f5955600640 1 -- 172.21.15.83:0/990008056 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f59480d08a0 con 0x7f5958100f60 2024-04-24T08:57:06.615 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.612+0000 7f594d600640 1 -- 172.21.15.83:0/1767568339 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5948052d20 con 0x56203c3ffce0 2024-04-24T08:57:06.615 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.612+0000 7f594d600640 1 -- 172.21.15.83:0/1767568339 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5948098710 con 0x56203c3ffce0 2024-04-24T08:57:06.615 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.613+0000 7f5961c5a740 1 -- 172.21.15.83:0/1767568339 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56203c3ffce0 msgr2=0x56203c40a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:06.616 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.613+0000 7f595ce00640 1 --2- 172.21.15.83:0/990008056 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f593c0a3d90 0x7f593c0a6250 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5950005cc0 tx=0x7f59500083f0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:06.616 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.613+0000 7f5961c5a740 1 --2- 172.21.15.83:0/1767568339 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56203c3ffce0 0x56203c40a060 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f59480cdb90 tx=0x7f59480d8690 comp rx=0 tx=0).stop 2024-04-24T08:57:06.616 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.613+0000 7f5961c5a740 1 -- 172.21.15.83:0/1767568339 shutdown_connections 2024-04-24T08:57:06.616 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.613+0000 7f5961c5a740 1 --2- 172.21.15.83:0/1767568339 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56203c40a5a0 0x56203c40c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:06.616 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.613+0000 7f5961c5a740 1 --2- 172.21.15.83:0/1767568339 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56203c3ffce0 0x56203c40a060 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:06.616 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.613+0000 7f5961c5a740 1 --2- 172.21.15.83:0/1767568339 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56203c3ff3c0 0x56203c3ff7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:06.616 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.613+0000 7f5961c5a740 1 -- 172.21.15.83:0/1767568339 >> 172.21.15.83:0/1767568339 conn(0x56203c3fb8d0 msgr2=0x56203c3fbcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:06.616 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.613+0000 7f5961c5a740 1 -- 172.21.15.83:0/1767568339 shutdown_connections 2024-04-24T08:57:06.616 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.613+0000 7f5961c5a740 1 -- 172.21.15.83:0/1767568339 wait complete. 2024-04-24T08:57:06.616 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.614+0000 7f5961c5a740 1 -- start start 2024-04-24T08:57:06.616 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.614+0000 7f5961c5a740 1 Processor -- start 2024-04-24T08:57:06.616 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.615+0000 7f5961c5a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56203c3ff3c0 0x56203c4075d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:06.616 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.615+0000 7f5961c5a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56203c3ffce0 0x56203c407b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:06.616 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.615+0000 7f5961c5a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56203c40a5a0 0x56203c408050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:06.616 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.615+0000 7f5961c5a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56203c40d140 con 0x56203c40a5a0 2024-04-24T08:57:06.617 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.615+0000 7f5961c5a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56203c31f2c0 con 0x56203c3ffce0 2024-04-24T08:57:06.617 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.615+0000 7f5961c5a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56203c40d2c0 con 0x56203c3ff3c0 2024-04-24T08:57:06.617 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.615+0000 7f5961c5a740 10 client.0 _ll_register_callbacks cb 0x56203c077a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:57:06.617 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.615+0000 7f595ce00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56203c40a5a0 0x56203c408050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:06.617 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.615+0000 7f595ce00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56203c40a5a0 0x56203c408050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43704/0 (socket says 172.21.15.83:43704) 2024-04-24T08:57:06.617 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.615+0000 7f595ce00640 1 -- 172.21.15.83:0/53727355 learned_addr learned my addr 172.21.15.83:0/53727355 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:57:06.617 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.615+0000 7f5957e00640 1 --2- 172.21.15.83:0/53727355 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56203c3ff3c0 0x56203c4075d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:06.617 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.615+0000 7f594fe00640 1 --2- 172.21.15.83:0/53727355 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56203c3ffce0 0x56203c407b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:06.619 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.616+0000 7f595ce00640 1 -- 172.21.15.83:0/53727355 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56203c3ff3c0 msgr2=0x56203c4075d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:06.619 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.616+0000 7f595ce00640 1 --2- 172.21.15.83:0/53727355 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56203c3ff3c0 0x56203c4075d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:06.619 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.616+0000 7f595ce00640 1 -- 172.21.15.83:0/53727355 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56203c3ffce0 msgr2=0x56203c407b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:06.619 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.616+0000 7f595ce00640 1 --2- 172.21.15.83:0/53727355 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56203c3ffce0 0x56203c407b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:06.619 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.616+0000 7f595ce00640 1 -- 172.21.15.83:0/53727355 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56203c32ffd0 con 0x56203c40a5a0 2024-04-24T08:57:06.619 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.616+0000 7f595ce00640 1 --2- 172.21.15.83:0/53727355 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56203c40a5a0 0x56203c408050 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f5950010dc0 tx=0x7f5950004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adfd788f7ff907a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:06.619 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.616+0000 7f5946000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:57:06.620 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.616+0000 7f5946000640 1 -- 172.21.15.83:0/53727355 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f595001d620 con 0x56203c40a5a0 2024-04-24T08:57:06.620 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.616+0000 7f5961c5a740 1 -- 172.21.15.83:0/53727355 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5948052d20 con 0x56203c40a5a0 2024-04-24T08:57:06.620 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.616+0000 7f5946000640 1 -- 172.21.15.83:0/53727355 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f595001d7c0 con 0x56203c40a5a0 2024-04-24T08:57:06.620 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.617+0000 7f5946000640 1 -- 172.21.15.83:0/53727355 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f595002e050 con 0x56203c40a5a0 2024-04-24T08:57:06.620 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.617+0000 7f5961c5a740 10 client.17619 fetch_fsmap learned FSMap version 33 2024-04-24T08:57:06.620 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.617+0000 7f5961c5a740 1 -- 172.21.15.83:0/53727355 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56203c078210 con 0x56203c40a5a0 2024-04-24T08:57:06.620 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.617+0000 7f5946000640 1 -- 172.21.15.83:0/53727355 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5948052d20 con 0x56203c40a5a0 2024-04-24T08:57:06.620 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.617+0000 7f5946000640 1 -- 172.21.15.83:0/53727355 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f5950025960 con 0x56203c40a5a0 2024-04-24T08:57:06.620 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.617+0000 7f5961c5a740 10 client.17619 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T08:57:06.620 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.617+0000 7f5961c5a740 10 client.17619 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T08:57:06.620 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.617+0000 7f5961c5a740 20 client.17619 populate_metadata read hostname 'smithi083' 2024-04-24T08:57:06.620 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.617+0000 7f5961c5a740 1 --2- 172.21.15.83:0/53727355 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56203c41a820 0x56203c41cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:06.621 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.617+0000 7f5961c5a740 4 client.17619 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:57:06.621 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.617+0000 7f5961c5a740 1 -- 172.21.15.83:0/53727355 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56203c373320 con 0x56203c41a820 2024-04-24T08:57:06.621 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.618+0000 7f5957e00640 1 --2- 172.21.15.83:0/53727355 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56203c41a820 0x56203c41cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:06.621 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.619+0000 7f5957e00640 1 --2- 172.21.15.83:0/53727355 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56203c41a820 0x56203c41cc10 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=386a7e2c4d59fb88 server_cookie=7b8919ea5de6443f in_seq=0 out_seq=0 2024-04-24T08:57:06.621 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.619+0000 7f5946000640 10 client.17619 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:57:06.623 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.622+0000 7f5946000640 1 -- 172.21.15.83:0/53727355 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56203c373320 con 0x56203c41a820 2024-04-24T08:57:06.623 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.622+0000 7f5946000640 10 client.17619 handle_command_reply: tid=0 2024-04-24T08:57:06.623 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.622+0000 7f5961c5a740 1 client.17619 shutdown 2024-04-24T08:57:06.623 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.623+0000 7f5961c5a740 1 -- 172.21.15.83:0/53727355 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56203c40a5a0 msgr2=0x56203c408050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:06.623 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.623+0000 7f5961c5a740 1 --2- 172.21.15.83:0/53727355 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56203c40a5a0 0x56203c408050 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f5950010dc0 tx=0x7f5950004a40 comp rx=0 tx=0).stop 2024-04-24T08:57:06.623 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.623+0000 7f5961c5a740 1 -- 172.21.15.83:0/53727355 shutdown_connections 2024-04-24T08:57:06.623 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.623+0000 7f5961c5a740 1 --2- 172.21.15.83:0/53727355 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56203c41a820 0x56203c41cc10 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:06.623 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.623+0000 7f5961c5a740 1 --2- 172.21.15.83:0/53727355 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56203c40a5a0 0x56203c408050 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:06.623 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.623+0000 7f5946000640 0 client.17619 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:57:06.623 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.623+0000 7f5961c5a740 1 --2- 172.21.15.83:0/53727355 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56203c3ffce0 0x56203c407b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:06.624 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.623+0000 7f5961c5a740 1 --2- 172.21.15.83:0/53727355 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56203c3ff3c0 0x56203c4075d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:06.624 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.623+0000 7f5961c5a740 1 -- 172.21.15.83:0/53727355 >> 172.21.15.83:0/53727355 conn(0x56203c3fb8d0 msgr2=0x56203c401650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:06.624 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.623+0000 7f5961c5a740 1 -- 172.21.15.83:0/53727355 shutdown_connections 2024-04-24T08:57:06.624 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.623+0000 7f5961c5a740 1 -- 172.21.15.83:0/53727355 wait complete. 2024-04-24T08:57:06.624 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.623+0000 7f5961c5a740 20 client.17619 trim_cache size 0 max 16384 2024-04-24T08:57:06.642 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.642+0000 7f5961c5a740 1 Processor -- start 2024-04-24T08:57:06.642 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.642+0000 7f5961c5a740 1 -- start start 2024-04-24T08:57:06.642 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.642+0000 7f5961c5a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56203c3ff3c0 0x56203c409d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:06.642 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.642+0000 7f5961c5a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56203c3ffce0 0x56203c4160f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:06.644 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.643+0000 7f5961c5a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56203c40a5a0 0x56203c4224e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:06.644 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.643+0000 7f5961c5a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f59581b1550 con 0x56203c3ff3c0 2024-04-24T08:57:06.644 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.643+0000 7f5961c5a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f59581b3090 con 0x56203c40a5a0 2024-04-24T08:57:06.644 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.643+0000 7f5961c5a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f59581b1c20 con 0x56203c3ffce0 2024-04-24T08:57:06.644 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.643+0000 7f5957e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56203c3ff3c0 0x56203c409d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:06.644 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.643+0000 7f5957e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56203c3ff3c0 0x56203c409d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43712/0 (socket says 172.21.15.83:43712) 2024-04-24T08:57:06.644 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.643+0000 7f5957e00640 1 -- 172.21.15.83:0/1608787035 learned_addr learned my addr 172.21.15.83:0/1608787035 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:57:06.644 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.643+0000 7f595ce00640 1 --2- 172.21.15.83:0/1608787035 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56203c40a5a0 0x56203c4224e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:06.644 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.643+0000 7f594fe00640 1 --2- 172.21.15.83:0/1608787035 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56203c3ffce0 0x56203c4160f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:06.644 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.644+0000 7f5957e00640 1 -- 172.21.15.83:0/1608787035 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56203c3ffce0 msgr2=0x56203c4160f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:06.645 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.644+0000 7f5957e00640 1 --2- 172.21.15.83:0/1608787035 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56203c3ffce0 0x56203c4160f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:06.645 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.644+0000 7f5957e00640 1 -- 172.21.15.83:0/1608787035 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56203c40a5a0 msgr2=0x56203c4224e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:06.645 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.644+0000 7f5957e00640 1 --2- 172.21.15.83:0/1608787035 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56203c40a5a0 0x56203c4224e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:06.645 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.644+0000 7f5957e00640 1 -- 172.21.15.83:0/1608787035 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56203c144620 con 0x56203c3ff3c0 2024-04-24T08:57:06.645 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.644+0000 7f595ce00640 1 --2- 172.21.15.83:0/1608787035 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56203c40a5a0 0x56203c4224e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:57:06.649 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.644+0000 7f5957e00640 1 --2- 172.21.15.83:0/1608787035 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56203c3ff3c0 0x56203c409d00 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f5948005fd0 tx=0x7f5948098040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be10a38bf482ba9e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:06.649 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.644+0000 7f594cc00640 1 -- 172.21.15.83:0/1608787035 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f594801d040 con 0x56203c3ff3c0 2024-04-24T08:57:06.649 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.644+0000 7f594cc00640 1 -- 172.21.15.83:0/1608787035 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f59480d8e90 con 0x56203c3ff3c0 2024-04-24T08:57:06.649 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.644+0000 7f594cc00640 1 -- 172.21.15.83:0/1608787035 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59480df040 con 0x56203c3ff3c0 2024-04-24T08:57:06.649 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.645+0000 7f5961c5a740 1 -- 172.21.15.83:0/1608787035 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56203c3ff3c0 msgr2=0x56203c409d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:06.649 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.645+0000 7f5961c5a740 1 --2- 172.21.15.83:0/1608787035 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56203c3ff3c0 0x56203c409d00 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f5948005fd0 tx=0x7f5948098040 comp rx=0 tx=0).stop 2024-04-24T08:57:06.649 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.646+0000 7f5961c5a740 1 -- 172.21.15.83:0/1608787035 shutdown_connections 2024-04-24T08:57:06.649 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.646+0000 7f5961c5a740 1 --2- 172.21.15.83:0/1608787035 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56203c40a5a0 0x56203c4224e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:06.650 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.646+0000 7f5961c5a740 1 --2- 172.21.15.83:0/1608787035 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56203c3ffce0 0x56203c4160f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:06.650 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.646+0000 7f5961c5a740 1 --2- 172.21.15.83:0/1608787035 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56203c3ff3c0 0x56203c409d00 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:06.650 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.646+0000 7f5961c5a740 1 -- 172.21.15.83:0/1608787035 >> 172.21.15.83:0/1608787035 conn(0x56203c3fb8d0 msgr2=0x56203c3faa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:06.650 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.646+0000 7f5961c5a740 1 -- 172.21.15.83:0/1608787035 shutdown_connections 2024-04-24T08:57:06.650 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.646+0000 7f5961c5a740 1 -- 172.21.15.83:0/1608787035 wait complete. 2024-04-24T08:57:06.650 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.646+0000 7f5961c5a740 1 -- start start 2024-04-24T08:57:06.650 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.647+0000 7f5961c5a740 1 Processor -- start 2024-04-24T08:57:06.650 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.647+0000 7f5961c5a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56203c3ff3c0 0x56203c412b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:06.650 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.647+0000 7f5961c5a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56203c3ffce0 0x56203c40eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:06.650 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.647+0000 7f5961c5a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56203c40a5a0 0x56203c40f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:06.650 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.647+0000 7f5961c5a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f59581b29c0 con 0x56203c3ff3c0 2024-04-24T08:57:06.650 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.647+0000 7f5961c5a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f59581b22f0 con 0x56203c40a5a0 2024-04-24T08:57:06.651 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.647+0000 7f5961c5a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f59581af530 con 0x56203c3ffce0 2024-04-24T08:57:06.651 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.647+0000 7f5961c5a740 10 client.0 _ll_register_callbacks cb 0x7f5950008a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:57:06.651 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.647+0000 7f5957e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56203c3ff3c0 0x56203c412b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:06.651 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.647+0000 7f5957e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56203c3ff3c0 0x56203c412b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43726/0 (socket says 172.21.15.83:43726) 2024-04-24T08:57:06.651 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.647+0000 7f5957e00640 1 -- 172.21.15.83:0/634561467 learned_addr learned my addr 172.21.15.83:0/634561467 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:57:06.651 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.648+0000 7f594fe00640 1 --2- 172.21.15.83:0/634561467 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56203c3ffce0 0x56203c40eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:06.651 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.648+0000 7f595ce00640 1 --2- 172.21.15.83:0/634561467 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56203c40a5a0 0x56203c40f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:06.651 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.648+0000 7f5957e00640 1 -- 172.21.15.83:0/634561467 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56203c3ffce0 msgr2=0x56203c40eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:06.651 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.648+0000 7f5957e00640 1 --2- 172.21.15.83:0/634561467 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56203c3ffce0 0x56203c40eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:06.651 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.648+0000 7f5957e00640 1 -- 172.21.15.83:0/634561467 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56203c40a5a0 msgr2=0x56203c40f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:06.651 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.648+0000 7f5957e00640 1 --2- 172.21.15.83:0/634561467 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56203c40a5a0 0x56203c40f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:06.651 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.648+0000 7f5957e00640 1 -- 172.21.15.83:0/634561467 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56203c480d60 con 0x56203c3ff3c0 2024-04-24T08:57:06.651 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.648+0000 7f5957e00640 1 --2- 172.21.15.83:0/634561467 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56203c3ff3c0 0x56203c412b30 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f59480d86d0 tx=0x7f5948005380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a74639e7c9f0ec30 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:06.652 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.648+0000 7f5946000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:57:06.652 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.648+0000 7f5946000640 1 -- 172.21.15.83:0/634561467 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f594801d050 con 0x56203c3ff3c0 2024-04-24T08:57:06.652 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.648+0000 7f5946000640 1 -- 172.21.15.83:0/634561467 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f594809a510 con 0x56203c3ff3c0 2024-04-24T08:57:06.652 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.648+0000 7f5961c5a740 1 -- 172.21.15.83:0/634561467 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59480d8e90 con 0x56203c3ff3c0 2024-04-24T08:57:06.652 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.649+0000 7f5946000640 1 -- 172.21.15.83:0/634561467 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f59480df070 con 0x56203c3ff3c0 2024-04-24T08:57:06.652 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.649+0000 7f5961c5a740 10 client.17625 fetch_fsmap learned FSMap version 33 2024-04-24T08:57:06.652 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.649+0000 7f5961c5a740 1 -- 172.21.15.83:0/634561467 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56203c481ef0 con 0x56203c3ff3c0 2024-04-24T08:57:06.652 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.649+0000 7f5946000640 1 -- 172.21.15.83:0/634561467 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f59480d8e90 con 0x56203c3ff3c0 2024-04-24T08:57:06.652 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.649+0000 7f5946000640 1 -- 172.21.15.83:0/634561467 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f59480cdea0 con 0x56203c3ff3c0 2024-04-24T08:57:06.653 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.649+0000 7f5961c5a740 10 client.17625 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T08:57:06.653 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.649+0000 7f5961c5a740 10 client.17625 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T08:57:06.653 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.649+0000 7f5961c5a740 20 client.17625 populate_metadata read hostname 'smithi083' 2024-04-24T08:57:06.653 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.649+0000 7f5961c5a740 1 --2- 172.21.15.83:0/634561467 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56203c4820c0 0x56203c4844b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:06.653 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.649+0000 7f5961c5a740 4 client.17625 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T08:57:06.653 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.649+0000 7f5961c5a740 1 -- 172.21.15.83:0/634561467 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56203c36d190 con 0x56203c4820c0 2024-04-24T08:57:06.653 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.650+0000 7f594fe00640 1 --2- 172.21.15.83:0/634561467 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56203c4820c0 0x56203c4844b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:06.654 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.651+0000 7f594fe00640 1 --2- 172.21.15.83:0/634561467 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56203c4820c0 0x56203c4844b0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4f66207a365979a server_cookie=da654e25784937d9 in_seq=0 out_seq=0 2024-04-24T08:57:06.654 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.651+0000 7f5946000640 10 client.17625 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:57:06.707 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.706+0000 7f5946000640 1 -- 172.21.15.83:0/634561467 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56203c36d190 con 0x56203c4820c0 2024-04-24T08:57:06.707 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.706+0000 7f5946000640 10 client.17625 handle_command_reply: tid=0 2024-04-24T08:57:06.707 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.706+0000 7f5961c5a740 1 client.17625 shutdown 2024-04-24T08:57:06.707 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.707+0000 7f5961c5a740 1 -- 172.21.15.83:0/634561467 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56203c3ff3c0 msgr2=0x56203c412b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:06.707 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.707+0000 7f5961c5a740 1 --2- 172.21.15.83:0/634561467 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56203c3ff3c0 0x56203c412b30 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f59480d86d0 tx=0x7f5948005380 comp rx=0 tx=0).stop 2024-04-24T08:57:06.708 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.707+0000 7f5961c5a740 1 -- 172.21.15.83:0/634561467 shutdown_connections 2024-04-24T08:57:06.708 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.707+0000 7f5961c5a740 1 --2- 172.21.15.83:0/634561467 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56203c4820c0 0x56203c4844b0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:06.708 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.707+0000 7f5961c5a740 1 --2- 172.21.15.83:0/634561467 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56203c40a5a0 0x56203c40f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:06.708 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.707+0000 7f5961c5a740 1 --2- 172.21.15.83:0/634561467 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56203c3ffce0 0x56203c40eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:06.708 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.707+0000 7f5961c5a740 1 --2- 172.21.15.83:0/634561467 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56203c3ff3c0 0x56203c412b30 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:06.708 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.707+0000 7f5961c5a740 1 -- 172.21.15.83:0/634561467 >> 172.21.15.83:0/634561467 conn(0x56203c3fb8d0 msgr2=0x56203c3fa200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:06.709 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.707+0000 7f5946000640 0 client.17625 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:57:06.709 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.707+0000 7f5961c5a740 1 -- 172.21.15.83:0/634561467 shutdown_connections 2024-04-24T08:57:06.709 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.707+0000 7f5961c5a740 1 -- 172.21.15.83:0/634561467 wait complete. 2024-04-24T08:57:06.709 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.708+0000 7f5961c5a740 20 client.17625 trim_cache size 0 max 16384 2024-04-24T08:57:06.709 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T08:57:06.709 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T08:57:06.709 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T08:57:06.709 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T08:57:06.710 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.709+0000 7f595d800640 1 -- 172.21.15.83:0/990008056 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f593c0a3d90 msgr2=0x7f593c0a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:06.710 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.709+0000 7f595d800640 1 --2- 172.21.15.83:0/990008056 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f593c0a3d90 0x7f593c0a6250 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5950005cc0 tx=0x7f59500083f0 comp rx=0 tx=0).stop 2024-04-24T08:57:06.710 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.709+0000 7f595d800640 1 -- 172.21.15.83:0/990008056 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5958100f60 msgr2=0x7f59581a0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:06.710 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.709+0000 7f595d800640 1 --2- 172.21.15.83:0/990008056 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5958100f60 0x7f59581a0650 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f594800c890 tx=0x7f594800cd50 comp rx=0 tx=0).stop 2024-04-24T08:57:06.710 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.709+0000 7f595d800640 1 -- 172.21.15.83:0/990008056 shutdown_connections 2024-04-24T08:57:06.710 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.709+0000 7f595d800640 1 --2- 172.21.15.83:0/990008056 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f593c0a3d90 0x7f593c0a6250 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:06.710 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.709+0000 7f595d800640 1 --2- 172.21.15.83:0/990008056 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f595819a710 0x7f595819ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:06.710 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.709+0000 7f595d800640 1 --2- 172.21.15.83:0/990008056 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5958106030 0x7f59581a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:06.711 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.709+0000 7f595d800640 1 --2- 172.21.15.83:0/990008056 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5958100f60 0x7f59581a0650 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:06.711 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.709+0000 7f595d800640 1 -- 172.21.15.83:0/990008056 >> 172.21.15.83:0/990008056 conn(0x7f59580fc820 msgr2=0x7f59580fdbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:06.711 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.709+0000 7f595d800640 1 -- 172.21.15.83:0/990008056 shutdown_connections 2024-04-24T08:57:06.711 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:06.710+0000 7f595d800640 1 -- 172.21.15.83:0/990008056 wait complete. 2024-04-24T08:57:07.579 INFO:tasks.workunit.client.0.smithi083.stdout: 1 26947 0.62 MB/sec execute 101 sec latency 56.317 ms 2024-04-24T08:57:08.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:57:07 smithi138 ceph-mon[31441]: pgmap v168: 129 pgs: 129 active+clean; 39 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 296 KiB/s wr, 71 op/s 2024-04-24T08:57:08.226 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:57:07 smithi189 ceph-mon[31791]: pgmap v168: 129 pgs: 129 active+clean; 39 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 296 KiB/s wr, 71 op/s 2024-04-24T08:57:08.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:57:07 smithi083 ceph-mon[27907]: pgmap v168: 129 pgs: 129 active+clean; 39 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 296 KiB/s wr, 71 op/s 2024-04-24T08:57:08.578 INFO:tasks.workunit.client.0.smithi083.stdout: 1 27079 0.62 MB/sec execute 102 sec latency 56.818 ms 2024-04-24T08:57:09.578 INFO:tasks.workunit.client.0.smithi083.stdout: 1 27231 0.62 MB/sec execute 103 sec latency 52.794 ms 2024-04-24T08:57:10.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:57:09 smithi138 ceph-mon[31441]: pgmap v169: 129 pgs: 129 active+clean; 35 MiB data, 517 MiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 363 KiB/s wr, 95 op/s 2024-04-24T08:57:10.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:57:09 smithi189 ceph-mon[31791]: pgmap v169: 129 pgs: 129 active+clean; 35 MiB data, 517 MiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 363 KiB/s wr, 95 op/s 2024-04-24T08:57:10.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:57:09 smithi083 ceph-mon[27907]: pgmap v169: 129 pgs: 129 active+clean; 35 MiB data, 517 MiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 363 KiB/s wr, 95 op/s 2024-04-24T08:57:10.578 INFO:tasks.workunit.client.0.smithi083.stdout: 1 27307 0.62 MB/sec execute 104 sec latency 57.582 ms 2024-04-24T08:57:11.579 INFO:tasks.workunit.client.0.smithi083.stdout: 1 27428 0.62 MB/sec execute 105 sec latency 53.638 ms 2024-04-24T08:57:12.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:57:11 smithi138 ceph-mon[31441]: pgmap v170: 129 pgs: 129 active+clean; 35 MiB data, 517 MiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 262 KiB/s wr, 66 op/s 2024-04-24T08:57:12.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:57:11 smithi189 ceph-mon[31791]: pgmap v170: 129 pgs: 129 active+clean; 35 MiB data, 517 MiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 262 KiB/s wr, 66 op/s 2024-04-24T08:57:12.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:57:11 smithi083 ceph-mon[27907]: pgmap v170: 129 pgs: 129 active+clean; 35 MiB data, 517 MiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 262 KiB/s wr, 66 op/s 2024-04-24T08:57:12.579 INFO:tasks.workunit.client.0.smithi083.stdout: 1 27507 0.62 MB/sec execute 106 sec latency 53.803 ms 2024-04-24T08:57:13.579 INFO:tasks.workunit.client.0.smithi083.stdout: 1 27788 0.65 MB/sec execute 107 sec latency 54.740 ms 2024-04-24T08:57:14.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:57:13 smithi138 ceph-mon[31441]: pgmap v171: 129 pgs: 129 active+clean; 37 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 482 KiB/s wr, 89 op/s 2024-04-24T08:57:14.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:57:13 smithi189 ceph-mon[31791]: pgmap v171: 129 pgs: 129 active+clean; 37 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 482 KiB/s wr, 89 op/s 2024-04-24T08:57:14.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:57:13 smithi083 ceph-mon[27907]: pgmap v171: 129 pgs: 129 active+clean; 37 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 482 KiB/s wr, 89 op/s 2024-04-24T08:57:14.579 INFO:tasks.workunit.client.0.smithi083.stdout: 1 27890 0.65 MB/sec execute 108 sec latency 58.002 ms 2024-04-24T08:57:15.579 INFO:tasks.workunit.client.0.smithi083.stdout: 1 28152 0.66 MB/sec execute 109 sec latency 55.320 ms 2024-04-24T08:57:16.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:57:15 smithi138 ceph-mon[31441]: pgmap v172: 129 pgs: 129 active+clean; 37 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 288 KiB/s wr, 46 op/s 2024-04-24T08:57:16.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:57:15 smithi189 ceph-mon[31791]: pgmap v172: 129 pgs: 129 active+clean; 37 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 288 KiB/s wr, 46 op/s 2024-04-24T08:57:16.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:57:15 smithi083 ceph-mon[27907]: pgmap v172: 129 pgs: 129 active+clean; 37 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 288 KiB/s wr, 46 op/s 2024-04-24T08:57:16.579 INFO:tasks.workunit.client.0.smithi083.stdout: 1 28182 0.65 MB/sec execute 110 sec latency 51.241 ms 2024-04-24T08:57:17.579 INFO:tasks.workunit.client.0.smithi083.stdout: 1 28206 0.65 MB/sec execute 111 sec latency 53.653 ms 2024-04-24T08:57:18.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:57:17 smithi138 ceph-mon[31441]: pgmap v173: 129 pgs: 129 active+clean; 37 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 288 KiB/s wr, 46 op/s 2024-04-24T08:57:18.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:57:17 smithi189 ceph-mon[31791]: pgmap v173: 129 pgs: 129 active+clean; 37 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 288 KiB/s wr, 46 op/s 2024-04-24T08:57:18.306 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:57:17 smithi083 ceph-mon[27907]: pgmap v173: 129 pgs: 129 active+clean; 37 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 288 KiB/s wr, 46 op/s 2024-04-24T08:57:18.579 INFO:tasks.workunit.client.0.smithi083.stdout: 1 28386 0.67 MB/sec execute 112 sec latency 54.004 ms 2024-04-24T08:57:19.579 INFO:tasks.workunit.client.0.smithi083.stdout: 1 28503 0.66 MB/sec execute 113 sec latency 55.032 ms 2024-04-24T08:57:19.861 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.862+0000 7f5e68600640 1 -- 172.21.15.83:0/956241840 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5e60104470 msgr2=0x7f5e60104850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:19.862 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.862+0000 7f5e68600640 1 --2- 172.21.15.83:0/956241840 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5e60104470 0x7f5e60104850 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f5e54009900 tx=0x7f5e5402a410 comp rx=0 tx=0).stop 2024-04-24T08:57:19.862 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.862+0000 7f5e68600640 1 -- 172.21.15.83:0/956241840 shutdown_connections 2024-04-24T08:57:19.862 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.862+0000 7f5e68600640 1 --2- 172.21.15.83:0/956241840 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5e60109540 0x7f5e60111110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:19.862 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.862+0000 7f5e68600640 1 --2- 172.21.15.83:0/956241840 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e60104e20 0x7f5e60108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:19.862 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.862+0000 7f5e68600640 1 --2- 172.21.15.83:0/956241840 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5e60104470 0x7f5e60104850 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:19.862 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.862+0000 7f5e68600640 1 -- 172.21.15.83:0/956241840 >> 172.21.15.83:0/956241840 conn(0x7f5e600ffd30 msgr2=0x7f5e60102150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:19.862 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.862+0000 7f5e68600640 1 -- 172.21.15.83:0/956241840 shutdown_connections 2024-04-24T08:57:19.862 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.862+0000 7f5e68600640 1 -- 172.21.15.83:0/956241840 wait complete. 2024-04-24T08:57:19.864 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.863+0000 7f5e68600640 1 Processor -- start 2024-04-24T08:57:19.864 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.863+0000 7f5e68600640 1 -- start start 2024-04-24T08:57:19.864 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.864+0000 7f5e68600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5e60104470 0x7f5e601a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:19.864 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.864+0000 7f5e68600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e60109540 0x7f5e601a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:19.864 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.864+0000 7f5e68600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5e6019a7a0 0x7f5e6019ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:19.864 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.864+0000 7f5e68600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5e60114500 con 0x7f5e6019a7a0 2024-04-24T08:57:19.864 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.864+0000 7f5e68600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5e60114380 con 0x7f5e60109540 2024-04-24T08:57:19.864 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.864+0000 7f5e68600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5e60114680 con 0x7f5e60104470 2024-04-24T08:57:19.864 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.864+0000 7f5e66600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5e6019a7a0 0x7f5e6019ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:19.864 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.864+0000 7f5e66600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5e6019a7a0 0x7f5e6019ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58784/0 (socket says 172.21.15.83:58784) 2024-04-24T08:57:19.864 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.864+0000 7f5e66600640 1 -- 172.21.15.83:0/591613674 learned_addr learned my addr 172.21.15.83:0/591613674 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:57:19.864 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.864+0000 7f5e65c00640 1 --2- 172.21.15.83:0/591613674 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5e60104470 0x7f5e601a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:19.865 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.864+0000 7f5e66600640 1 -- 172.21.15.83:0/591613674 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5e60104470 msgr2=0x7f5e601a0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:19.865 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.864+0000 7f5e65200640 1 --2- 172.21.15.83:0/591613674 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e60109540 0x7f5e601a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:19.865 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.865+0000 7f5e66600640 1 --2- 172.21.15.83:0/591613674 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5e60104470 0x7f5e601a0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:19.865 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.865+0000 7f5e66600640 1 -- 172.21.15.83:0/591613674 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e60109540 msgr2=0x7f5e601a0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:19.865 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.865+0000 7f5e66600640 1 --2- 172.21.15.83:0/591613674 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e60109540 0x7f5e601a0b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:19.865 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.865+0000 7f5e66600640 1 -- 172.21.15.83:0/591613674 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e6019b330 con 0x7f5e6019a7a0 2024-04-24T08:57:19.865 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.865+0000 7f5e66600640 1 --2- 172.21.15.83:0/591613674 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5e6019a7a0 0x7f5e6019ac50 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f5e5800eaa0 tx=0x7f5e5800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:19.866 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.865+0000 7f5e5e000640 1 -- 172.21.15.83:0/591613674 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e5800cdb0 con 0x7f5e6019a7a0 2024-04-24T08:57:19.866 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.865+0000 7f5e5e000640 1 -- 172.21.15.83:0/591613674 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5e58005020 con 0x7f5e6019a7a0 2024-04-24T08:57:19.866 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.865+0000 7f5e5e000640 1 -- 172.21.15.83:0/591613674 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e580116d0 con 0x7f5e6019a7a0 2024-04-24T08:57:19.866 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.865+0000 7f5e68600640 1 -- 172.21.15.83:0/591613674 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e6019b620 con 0x7f5e6019a7a0 2024-04-24T08:57:19.866 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.865+0000 7f5e68600640 1 -- 172.21.15.83:0/591613674 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e601a77b0 con 0x7f5e6019a7a0 2024-04-24T08:57:19.867 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.868+0000 7f5e6b40a740 1 Processor -- start 2024-04-24T08:57:19.868 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.869+0000 7f5e5e000640 1 -- 172.21.15.83:0/591613674 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5e580051d0 con 0x7f5e6019a7a0 2024-04-24T08:57:19.868 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.869+0000 7f5e6b40a740 1 -- start start 2024-04-24T08:57:19.869 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.869+0000 7f5e6b40a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558bc78c01b0 0x558bc78c0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:19.869 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.870+0000 7f5e6b40a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558bc78c0ad0 0x558bc78cae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:19.869 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.870+0000 7f5e6b40a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558bc78cb390 0x558bc78cd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:19.870 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.870+0000 7f5e6b40a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x558bc7826900 con 0x558bc78cb390 2024-04-24T08:57:19.870 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.870+0000 7f5e6b40a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558bc77ccf20 con 0x558bc78c01b0 2024-04-24T08:57:19.870 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.870+0000 7f5e6b40a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x558bc78228c0 con 0x558bc78c0ad0 2024-04-24T08:57:19.870 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.870+0000 7f5e65200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558bc78cb390 0x558bc78cd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:19.870 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.870+0000 7f5e65200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558bc78cb390 0x558bc78cd780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58794/0 (socket says 172.21.15.83:58794) 2024-04-24T08:57:19.870 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.870+0000 7f5e65200640 1 -- 172.21.15.83:0/3999550488 learned_addr learned my addr 172.21.15.83:0/3999550488 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:57:19.870 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.870+0000 7f5e65200640 1 --2- 172.21.15.83:0/3999550488 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558bc78c01b0 0x558bc78c0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:19.870 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.870+0000 7f5e65c00640 1 --2- 172.21.15.83:0/3999550488 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558bc78c0ad0 0x558bc78cae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:19.870 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.870+0000 7f5e65200640 1 -- 172.21.15.83:0/3999550488 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558bc78c0ad0 msgr2=0x558bc78cae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:19.870 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.870+0000 7f5e65200640 1 --2- 172.21.15.83:0/3999550488 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558bc78c0ad0 0x558bc78cae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:19.870 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.870+0000 7f5e65200640 1 -- 172.21.15.83:0/3999550488 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558bc78c01b0 msgr2=0x558bc78c0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:19.870 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.870+0000 7f5e65200640 1 --2- 172.21.15.83:0/3999550488 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558bc78c01b0 0x558bc78c0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:19.870 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.870+0000 7f5e65200640 1 -- 172.21.15.83:0/3999550488 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558bc77f1e10 con 0x558bc78cb390 2024-04-24T08:57:19.872 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.871+0000 7f5e5e000640 1 --2- 172.21.15.83:0/591613674 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f5e480a3e40 0x7f5e480a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:19.872 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.871+0000 7f5e65200640 1 --2- 172.21.15.83:0/3999550488 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558bc78cb390 0x558bc78cd780 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c00dfd0 tx=0x7f5e4c00d490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f155e852b8bb2908 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:19.872 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.871+0000 7f5e5e000640 1 -- 172.21.15.83:0/591613674 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f5e580d0d90 con 0x7f5e6019a7a0 2024-04-24T08:57:19.872 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.871+0000 7f5e52a00640 1 -- 172.21.15.83:0/3999550488 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e4c012860 con 0x558bc78cb390 2024-04-24T08:57:19.872 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.871+0000 7f5e65c00640 1 --2- 172.21.15.83:0/591613674 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f5e480a3e40 0x7f5e480a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:19.872 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.871+0000 7f5e52a00640 1 -- 172.21.15.83:0/3999550488 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5e4c011440 con 0x558bc78cb390 2024-04-24T08:57:19.872 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.871+0000 7f5e52a00640 1 -- 172.21.15.83:0/3999550488 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e4c010630 con 0x558bc78cb390 2024-04-24T08:57:19.872 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.872+0000 7f5e6b40a740 1 -- 172.21.15.83:0/3999550488 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558bc78cb390 msgr2=0x558bc78cd780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:19.872 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.872+0000 7f5e65c00640 1 --2- 172.21.15.83:0/591613674 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f5e480a3e40 0x7f5e480a6300 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5e54000c00 tx=0x7f5e54005a20 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:19.873 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.872+0000 7f5e6b40a740 1 --2- 172.21.15.83:0/3999550488 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558bc78cb390 0x558bc78cd780 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c00dfd0 tx=0x7f5e4c00d490 comp rx=0 tx=0).stop 2024-04-24T08:57:19.873 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.872+0000 7f5e6b40a740 1 -- 172.21.15.83:0/3999550488 shutdown_connections 2024-04-24T08:57:19.873 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.872+0000 7f5e6b40a740 1 --2- 172.21.15.83:0/3999550488 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558bc78cb390 0x558bc78cd780 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:19.873 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.872+0000 7f5e6b40a740 1 --2- 172.21.15.83:0/3999550488 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558bc78c0ad0 0x558bc78cae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:19.873 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.872+0000 7f5e6b40a740 1 --2- 172.21.15.83:0/3999550488 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558bc78c01b0 0x558bc78c0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:19.873 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.872+0000 7f5e6b40a740 1 -- 172.21.15.83:0/3999550488 >> 172.21.15.83:0/3999550488 conn(0x558bc78bc6c0 msgr2=0x558bc78bcaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:19.873 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.872+0000 7f5e6b40a740 1 -- 172.21.15.83:0/3999550488 shutdown_connections 2024-04-24T08:57:19.873 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.872+0000 7f5e6b40a740 1 -- 172.21.15.83:0/3999550488 wait complete. 2024-04-24T08:57:19.873 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.873+0000 7f5e6b40a740 1 -- start start 2024-04-24T08:57:19.873 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.873+0000 7f5e6b40a740 1 Processor -- start 2024-04-24T08:57:19.876 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.874+0000 7f5e6b40a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558bc78c01b0 0x558bc78c83c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:19.876 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.874+0000 7f5e6b40a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558bc78c0ad0 0x558bc78c8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:19.876 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.874+0000 7f5e6b40a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558bc78cb390 0x558bc78c8e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:19.876 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.874+0000 7f5e6b40a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x558bc78cdfc0 con 0x558bc78c01b0 2024-04-24T08:57:19.876 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.874+0000 7f5e6b40a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558bc77e02c0 con 0x558bc78c0ad0 2024-04-24T08:57:19.876 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.874+0000 7f5e6b40a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x558bc78ce140 con 0x558bc78cb390 2024-04-24T08:57:19.876 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.874+0000 7f5e6b40a740 10 client.0 _ll_register_callbacks cb 0x558bc7538a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:57:19.877 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.874+0000 7f5e65c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558bc78c01b0 0x558bc78c83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:19.877 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.874+0000 7f5e65c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558bc78c01b0 0x558bc78c83c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58800/0 (socket says 172.21.15.83:58800) 2024-04-24T08:57:19.877 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.874+0000 7f5e65c00640 1 -- 172.21.15.83:0/162564087 learned_addr learned my addr 172.21.15.83:0/162564087 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:57:19.877 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.874+0000 7f5e65200640 1 --2- 172.21.15.83:0/162564087 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558bc78c0ad0 0x558bc78c8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:19.877 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.874+0000 7f5e66600640 1 --2- 172.21.15.83:0/162564087 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558bc78cb390 0x558bc78c8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:19.877 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.874+0000 7f5e65c00640 1 -- 172.21.15.83:0/162564087 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558bc78cb390 msgr2=0x558bc78c8e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:19.877 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.874+0000 7f5e65c00640 1 --2- 172.21.15.83:0/162564087 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558bc78cb390 0x558bc78c8e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:19.877 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.874+0000 7f5e65c00640 1 -- 172.21.15.83:0/162564087 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558bc78c0ad0 msgr2=0x558bc78c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:19.877 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.874+0000 7f5e65c00640 1 --2- 172.21.15.83:0/162564087 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558bc78c0ad0 0x558bc78c8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:19.877 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.874+0000 7f5e65c00640 1 -- 172.21.15.83:0/162564087 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558bc77f0fd0 con 0x558bc78c01b0 2024-04-24T08:57:19.877 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.874+0000 7f5e66600640 1 --2- 172.21.15.83:0/162564087 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558bc78cb390 0x558bc78c8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:57:19.877 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.875+0000 7f5e65c00640 1 --2- 172.21.15.83:0/162564087 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558bc78c01b0 0x558bc78c83c0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f5e54031440 tx=0x7f5e54009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99f61c65f1aed0f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:19.877 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.875+0000 7f5e47400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:57:19.877 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.875+0000 7f5e47400640 1 -- 172.21.15.83:0/162564087 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e5404c070 con 0x558bc78c01b0 2024-04-24T08:57:19.877 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.875+0000 7f5e47400640 1 -- 172.21.15.83:0/162564087 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f5e54004d50 con 0x558bc78c01b0 2024-04-24T08:57:19.878 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.875+0000 7f5e6b40a740 1 -- 172.21.15.83:0/162564087 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e4c011440 con 0x558bc78c01b0 2024-04-24T08:57:19.878 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.875+0000 7f5e47400640 1 -- 172.21.15.83:0/162564087 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f5e54058050 con 0x558bc78c01b0 2024-04-24T08:57:19.878 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.876+0000 7f5e6b40a740 10 client.17637 fetch_fsmap learned FSMap version 33 2024-04-24T08:57:19.878 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.876+0000 7f5e6b40a740 1 -- 172.21.15.83:0/162564087 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x558bc7539210 con 0x558bc78c01b0 2024-04-24T08:57:19.878 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.876+0000 7f5e47400640 1 -- 172.21.15.83:0/162564087 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5e4c011440 con 0x558bc78c01b0 2024-04-24T08:57:19.878 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.876+0000 7f5e47400640 1 -- 172.21.15.83:0/162564087 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f5e540327a0 con 0x558bc78c01b0 2024-04-24T08:57:19.878 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.876+0000 7f5e6b40a740 10 client.17637 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T08:57:19.878 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.876+0000 7f5e6b40a740 10 client.17637 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T08:57:19.878 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.876+0000 7f5e6b40a740 20 client.17637 populate_metadata read hostname 'smithi083' 2024-04-24T08:57:19.878 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.876+0000 7f5e6b40a740 1 --2- 172.21.15.83:0/162564087 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x558bc78db560 0x558bc78dd950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:19.878 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.876+0000 7f5e6b40a740 4 client.17637 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:57:19.878 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.876+0000 7f5e6b40a740 1 -- 172.21.15.83:0/162564087 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558bc7834320 con 0x558bc78db560 2024-04-24T08:57:19.878 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.877+0000 7f5e65200640 1 --2- 172.21.15.83:0/162564087 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x558bc78db560 0x558bc78dd950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:19.879 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.878+0000 7f5e65200640 1 --2- 172.21.15.83:0/162564087 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x558bc78db560 0x558bc78dd950 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=172fcf912b09cdff server_cookie=e5caae60a3d5873d in_seq=0 out_seq=0 2024-04-24T08:57:19.879 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.878+0000 7f5e47400640 10 client.17637 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:57:19.880 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.881+0000 7f5e47400640 1 -- 172.21.15.83:0/162564087 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558bc7834320 con 0x558bc78db560 2024-04-24T08:57:19.881 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.881+0000 7f5e47400640 10 client.17637 handle_command_reply: tid=0 2024-04-24T08:57:19.881 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.881+0000 7f5e6b40a740 1 client.17637 shutdown 2024-04-24T08:57:19.881 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.881+0000 7f5e6b40a740 1 -- 172.21.15.83:0/162564087 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558bc78c01b0 msgr2=0x558bc78c83c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:19.881 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.881+0000 7f5e6b40a740 1 --2- 172.21.15.83:0/162564087 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558bc78c01b0 0x558bc78c83c0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f5e54031440 tx=0x7f5e54009260 comp rx=0 tx=0).stop 2024-04-24T08:57:19.881 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.881+0000 7f5e6b40a740 1 -- 172.21.15.83:0/162564087 shutdown_connections 2024-04-24T08:57:19.881 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.881+0000 7f5e6b40a740 1 --2- 172.21.15.83:0/162564087 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x558bc78db560 0x558bc78dd950 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:19.881 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.881+0000 7f5e6b40a740 1 --2- 172.21.15.83:0/162564087 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558bc78cb390 0x558bc78c8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:19.881 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.881+0000 7f5e6b40a740 1 --2- 172.21.15.83:0/162564087 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558bc78c0ad0 0x558bc78c8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:19.881 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.881+0000 7f5e6b40a740 1 --2- 172.21.15.83:0/162564087 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558bc78c01b0 0x558bc78c83c0 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:19.881 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.881+0000 7f5e6b40a740 1 -- 172.21.15.83:0/162564087 >> 172.21.15.83:0/162564087 conn(0x558bc78bc6c0 msgr2=0x558bc78c23f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:19.881 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.881+0000 7f5e47400640 0 client.17637 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:57:19.881 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.882+0000 7f5e6b40a740 1 -- 172.21.15.83:0/162564087 shutdown_connections 2024-04-24T08:57:19.881 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.882+0000 7f5e6b40a740 1 -- 172.21.15.83:0/162564087 wait complete. 2024-04-24T08:57:19.881 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.882+0000 7f5e6b40a740 20 client.17637 trim_cache size 0 max 16384 2024-04-24T08:57:19.899 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.900+0000 7f5e6b40a740 1 Processor -- start 2024-04-24T08:57:19.900 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.900+0000 7f5e6b40a740 1 -- start start 2024-04-24T08:57:19.900 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.900+0000 7f5e6b40a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558bc78c01b0 0x558bc78bbfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:19.900 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.900+0000 7f5e6b40a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558bc78c0ad0 0x558bc78c2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:19.900 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.900+0000 7f5e6b40a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558bc78cb390 0x558bc78e3220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:19.900 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.900+0000 7f5e6b40a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5e601b16b0 con 0x558bc78cb390 2024-04-24T08:57:19.900 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.900+0000 7f5e6b40a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5e601b31f0 con 0x558bc78c0ad0 2024-04-24T08:57:19.900 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.900+0000 7f5e6b40a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5e601b1d80 con 0x558bc78c01b0 2024-04-24T08:57:19.900 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.900+0000 7f5e66600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558bc78cb390 0x558bc78e3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:19.900 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.901+0000 7f5e66600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558bc78cb390 0x558bc78e3220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58808/0 (socket says 172.21.15.83:58808) 2024-04-24T08:57:19.900 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.901+0000 7f5e66600640 1 -- 172.21.15.83:0/1672724820 learned_addr learned my addr 172.21.15.83:0/1672724820 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:57:19.900 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.901+0000 7f5e65c00640 1 --2- 172.21.15.83:0/1672724820 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558bc78c01b0 0x558bc78bbfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:19.901 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.901+0000 7f5e66600640 1 -- 172.21.15.83:0/1672724820 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558bc78c01b0 msgr2=0x558bc78bbfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:19.902 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.901+0000 7f5e66600640 1 --2- 172.21.15.83:0/1672724820 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558bc78c01b0 0x558bc78bbfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:19.902 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.901+0000 7f5e66600640 1 -- 172.21.15.83:0/1672724820 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558bc78c0ad0 msgr2=0x558bc78c2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:19.902 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.901+0000 7f5e66600640 1 --2- 172.21.15.83:0/1672724820 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558bc78c0ad0 0x558bc78c2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:19.902 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.901+0000 7f5e66600640 1 -- 172.21.15.83:0/1672724820 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558bc7605620 con 0x558bc78cb390 2024-04-24T08:57:19.902 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.901+0000 7f5e66600640 1 --2- 172.21.15.83:0/1672724820 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558bc78cb390 0x558bc78e3220 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f5e58098d60 tx=0x7f5e5809c550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccd973cc7d20813a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:19.902 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.901+0000 7f5e52000640 1 -- 172.21.15.83:0/1672724820 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e5801f040 con 0x558bc78cb390 2024-04-24T08:57:19.902 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.902+0000 7f5e52000640 1 -- 172.21.15.83:0/1672724820 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5e58005560 con 0x558bc78cb390 2024-04-24T08:57:19.902 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.902+0000 7f5e52000640 1 -- 172.21.15.83:0/1672724820 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e580870c0 con 0x558bc78cb390 2024-04-24T08:57:19.902 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.902+0000 7f5e6b40a740 1 -- 172.21.15.83:0/1672724820 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558bc78cb390 msgr2=0x558bc78e3220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:19.902 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.902+0000 7f5e6b40a740 1 --2- 172.21.15.83:0/1672724820 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558bc78cb390 0x558bc78e3220 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f5e58098d60 tx=0x7f5e5809c550 comp rx=0 tx=0).stop 2024-04-24T08:57:19.902 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.903+0000 7f5e6b40a740 1 -- 172.21.15.83:0/1672724820 shutdown_connections 2024-04-24T08:57:19.902 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.903+0000 7f5e6b40a740 1 --2- 172.21.15.83:0/1672724820 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558bc78cb390 0x558bc78e3220 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:19.902 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.903+0000 7f5e6b40a740 1 --2- 172.21.15.83:0/1672724820 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558bc78c0ad0 0x558bc78c2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:19.903 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.903+0000 7f5e6b40a740 1 --2- 172.21.15.83:0/1672724820 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558bc78c01b0 0x558bc78bbfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:19.903 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.903+0000 7f5e6b40a740 1 -- 172.21.15.83:0/1672724820 >> 172.21.15.83:0/1672724820 conn(0x558bc78bc6c0 msgr2=0x558bc78c8c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:19.903 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.903+0000 7f5e6b40a740 1 -- 172.21.15.83:0/1672724820 shutdown_connections 2024-04-24T08:57:19.903 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.903+0000 7f5e6b40a740 1 -- 172.21.15.83:0/1672724820 wait complete. 2024-04-24T08:57:19.904 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.903+0000 7f5e6b40a740 1 -- start start 2024-04-24T08:57:19.904 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.903+0000 7f5e6b40a740 1 Processor -- start 2024-04-24T08:57:19.904 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.904+0000 7f5e6b40a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558bc78c01b0 0x558bc78c57c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:19.904 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.904+0000 7f5e6b40a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558bc78c0ad0 0x558bc78c5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:19.904 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.904+0000 7f5e6b40a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558bc78cb390 0x558bc7940a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:19.905 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.904+0000 7f5e6b40a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5e601b2b20 con 0x558bc78cb390 2024-04-24T08:57:19.905 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.904+0000 7f5e6b40a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5e601b2450 con 0x558bc78c0ad0 2024-04-24T08:57:19.905 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.904+0000 7f5e6b40a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5e601af690 con 0x558bc78c01b0 2024-04-24T08:57:19.905 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.904+0000 7f5e6b40a740 10 client.0 _ll_register_callbacks cb 0x7f5e54009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:57:19.905 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.904+0000 7f5e66600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558bc78cb390 0x558bc7940a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:19.905 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.904+0000 7f5e66600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558bc78cb390 0x558bc7940a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58820/0 (socket says 172.21.15.83:58820) 2024-04-24T08:57:19.905 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.904+0000 7f5e66600640 1 -- 172.21.15.83:0/56042940 learned_addr learned my addr 172.21.15.83:0/56042940 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:57:19.905 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.904+0000 7f5e65c00640 1 --2- 172.21.15.83:0/56042940 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558bc78c01b0 0x558bc78c57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:19.905 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.904+0000 7f5e65200640 1 --2- 172.21.15.83:0/56042940 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558bc78c0ad0 0x558bc78c5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:19.905 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.905+0000 7f5e66600640 1 -- 172.21.15.83:0/56042940 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558bc78c01b0 msgr2=0x558bc78c57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:19.905 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.905+0000 7f5e66600640 1 --2- 172.21.15.83:0/56042940 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558bc78c01b0 0x558bc78c57c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:19.905 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.905+0000 7f5e66600640 1 -- 172.21.15.83:0/56042940 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558bc78c0ad0 msgr2=0x558bc78c5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:19.905 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.905+0000 7f5e66600640 1 --2- 172.21.15.83:0/56042940 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558bc78c0ad0 0x558bc78c5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:19.905 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.905+0000 7f5e66600640 1 -- 172.21.15.83:0/56042940 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558bc7940fd0 con 0x558bc78cb390 2024-04-24T08:57:19.906 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.905+0000 7f5e65c00640 1 --2- 172.21.15.83:0/56042940 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558bc78c01b0 0x558bc78c57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:57:19.906 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.905+0000 7f5e66600640 1 --2- 172.21.15.83:0/56042940 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558bc78cb390 0x558bc7940a90 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f5e580992d0 tx=0x7f5e58000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb4b6e2e9d33686e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:19.906 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.905+0000 7f5e47400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:57:19.909 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.909+0000 7f5e47400640 1 -- 172.21.15.83:0/56042940 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e58080060 con 0x558bc78cb390 2024-04-24T08:57:19.909 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.909+0000 7f5e47400640 1 -- 172.21.15.83:0/56042940 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f5e58087250 con 0x558bc78cb390 2024-04-24T08:57:19.909 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.909+0000 7f5e6b40a740 1 -- 172.21.15.83:0/56042940 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e58005560 con 0x558bc78cb390 2024-04-24T08:57:19.909 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.910+0000 7f5e47400640 1 -- 172.21.15.83:0/56042940 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f5e5801f050 con 0x558bc78cb390 2024-04-24T08:57:19.909 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.910+0000 7f5e6b40a740 10 client.17643 fetch_fsmap learned FSMap version 33 2024-04-24T08:57:19.909 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.910+0000 7f5e6b40a740 1 -- 172.21.15.83:0/56042940 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x558bc79422e0 con 0x558bc78cb390 2024-04-24T08:57:19.909 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.910+0000 7f5e47400640 1 -- 172.21.15.83:0/56042940 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5e58005560 con 0x558bc78cb390 2024-04-24T08:57:19.910 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.910+0000 7f5e47400640 1 -- 172.21.15.83:0/56042940 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f5e5800f040 con 0x558bc78cb390 2024-04-24T08:57:19.910 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.910+0000 7f5e6b40a740 10 client.17643 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T08:57:19.910 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.910+0000 7f5e6b40a740 10 client.17643 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T08:57:19.910 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.910+0000 7f5e6b40a740 20 client.17643 populate_metadata read hostname 'smithi083' 2024-04-24T08:57:19.910 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.910+0000 7f5e6b40a740 1 --2- 172.21.15.83:0/56042940 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x558bc7942990 0x558bc7944db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:19.910 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.910+0000 7f5e6b40a740 4 client.17643 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T08:57:19.910 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.910+0000 7f5e6b40a740 1 -- 172.21.15.83:0/56042940 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558bc782e190 con 0x558bc7942990 2024-04-24T08:57:19.910 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.911+0000 7f5e65c00640 1 --2- 172.21.15.83:0/56042940 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x558bc7942990 0x558bc7944db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:19.911 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.912+0000 7f5e65c00640 1 --2- 172.21.15.83:0/56042940 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x558bc7942990 0x558bc7944db0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b074fa7c5e00dfae server_cookie=e861d5653c80f0e in_seq=0 out_seq=0 2024-04-24T08:57:19.911 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.912+0000 7f5e47400640 10 client.17643 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:57:19.960 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.960+0000 7f5e47400640 1 -- 172.21.15.83:0/56042940 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558bc782e190 con 0x558bc7942990 2024-04-24T08:57:19.961 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.960+0000 7f5e47400640 10 client.17643 handle_command_reply: tid=0 2024-04-24T08:57:19.961 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.960+0000 7f5e6b40a740 1 client.17643 shutdown 2024-04-24T08:57:19.961 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.961+0000 7f5e6b40a740 1 -- 172.21.15.83:0/56042940 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558bc78cb390 msgr2=0x558bc7940a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:19.961 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.961+0000 7f5e6b40a740 1 --2- 172.21.15.83:0/56042940 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558bc78cb390 0x558bc7940a90 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f5e580992d0 tx=0x7f5e58000f40 comp rx=0 tx=0).stop 2024-04-24T08:57:19.962 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.961+0000 7f5e6b40a740 1 -- 172.21.15.83:0/56042940 shutdown_connections 2024-04-24T08:57:19.962 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.961+0000 7f5e6b40a740 1 --2- 172.21.15.83:0/56042940 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x558bc7942990 0x558bc7944db0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:19.962 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.961+0000 7f5e6b40a740 1 --2- 172.21.15.83:0/56042940 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558bc78cb390 0x558bc7940a90 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:19.962 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.961+0000 7f5e6b40a740 1 --2- 172.21.15.83:0/56042940 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558bc78c0ad0 0x558bc78c5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:19.962 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.961+0000 7f5e6b40a740 1 --2- 172.21.15.83:0/56042940 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558bc78c01b0 0x558bc78c57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:19.962 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.961+0000 7f5e47400640 0 client.17643 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:57:19.962 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.961+0000 7f5e6b40a740 1 -- 172.21.15.83:0/56042940 >> 172.21.15.83:0/56042940 conn(0x558bc78bc6c0 msgr2=0x558bc78c9bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:19.962 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.961+0000 7f5e6b40a740 1 -- 172.21.15.83:0/56042940 shutdown_connections 2024-04-24T08:57:19.962 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.961+0000 7f5e6b40a740 1 -- 172.21.15.83:0/56042940 wait complete. 2024-04-24T08:57:19.962 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.962+0000 7f5e6b40a740 20 client.17643 trim_cache size 0 max 16384 2024-04-24T08:57:19.963 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T08:57:19.963 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T08:57:19.963 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T08:57:19.963 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T08:57:19.964 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.962+0000 7f5e68600640 1 -- 172.21.15.83:0/591613674 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f5e480a3e40 msgr2=0x7f5e480a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:19.964 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.962+0000 7f5e68600640 1 --2- 172.21.15.83:0/591613674 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f5e480a3e40 0x7f5e480a6300 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5e54000c00 tx=0x7f5e54005a20 comp rx=0 tx=0).stop 2024-04-24T08:57:19.964 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.962+0000 7f5e68600640 1 -- 172.21.15.83:0/591613674 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5e6019a7a0 msgr2=0x7f5e6019ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:19.964 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.962+0000 7f5e68600640 1 --2- 172.21.15.83:0/591613674 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5e6019a7a0 0x7f5e6019ac50 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f5e5800eaa0 tx=0x7f5e5800ef60 comp rx=0 tx=0).stop 2024-04-24T08:57:19.964 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.963+0000 7f5e68600640 1 -- 172.21.15.83:0/591613674 shutdown_connections 2024-04-24T08:57:19.964 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.963+0000 7f5e68600640 1 --2- 172.21.15.83:0/591613674 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f5e480a3e40 0x7f5e480a6300 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:19.965 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.963+0000 7f5e68600640 1 --2- 172.21.15.83:0/591613674 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5e6019a7a0 0x7f5e6019ac50 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:19.965 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.963+0000 7f5e68600640 1 --2- 172.21.15.83:0/591613674 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e60109540 0x7f5e601a0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:19.965 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.963+0000 7f5e68600640 1 --2- 172.21.15.83:0/591613674 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5e60104470 0x7f5e601a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:19.965 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.963+0000 7f5e68600640 1 -- 172.21.15.83:0/591613674 >> 172.21.15.83:0/591613674 conn(0x7f5e600ffd30 msgr2=0x7f5e60102050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:19.965 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.963+0000 7f5e68600640 1 -- 172.21.15.83:0/591613674 shutdown_connections 2024-04-24T08:57:19.965 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:19.963+0000 7f5e68600640 1 -- 172.21.15.83:0/591613674 wait complete. 2024-04-24T08:57:20.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:57:19 smithi138 ceph-mon[31441]: pgmap v174: 129 pgs: 129 active+clean; 39 MiB data, 530 MiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 458 KiB/s wr, 63 op/s 2024-04-24T08:57:20.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:57:19 smithi189 ceph-mon[31791]: pgmap v174: 129 pgs: 129 active+clean; 39 MiB data, 530 MiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 458 KiB/s wr, 63 op/s 2024-04-24T08:57:20.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:57:19 smithi083 ceph-mon[27907]: pgmap v174: 129 pgs: 129 active+clean; 39 MiB data, 530 MiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 458 KiB/s wr, 63 op/s 2024-04-24T08:57:20.353 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T08:57:20.353 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T08:57:20.579 INFO:tasks.workunit.client.0.smithi083.stdout: 1 28632 0.67 MB/sec execute 114 sec latency 48.386 ms 2024-04-24T08:57:20.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.586+0000 7fe98ac00640 1 -- 172.21.15.83:0/846300032 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe984108f40 msgr2=0x7fe9841153b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:20.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.586+0000 7fe98ac00640 1 --2- 172.21.15.83:0/846300032 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe984108f40 0x7fe9841153b0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fe97c009cd0 tx=0x7fe97c02a8e0 comp rx=0 tx=0).stop 2024-04-24T08:57:20.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.586+0000 7fe98ac00640 1 -- 172.21.15.83:0/846300032 shutdown_connections 2024-04-24T08:57:20.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.586+0000 7fe98ac00640 1 --2- 172.21.15.83:0/846300032 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe984108f40 0x7fe9841153b0 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:20.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.586+0000 7fe98ac00640 1 --2- 172.21.15.83:0/846300032 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe984108580 0x7fe984108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:20.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.586+0000 7fe98ac00640 1 --2- 172.21.15.83:0/846300032 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe98410d500 0x7fe98410d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:20.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.586+0000 7fe98ac00640 1 -- 172.21.15.83:0/846300032 >> 172.21.15.83:0/846300032 conn(0x7fe984077e20 msgr2=0x7fe984106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:20.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.586+0000 7fe98ac00640 1 -- 172.21.15.83:0/846300032 shutdown_connections 2024-04-24T08:57:20.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.586+0000 7fe98ac00640 1 -- 172.21.15.83:0/846300032 wait complete. 2024-04-24T08:57:20.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.587+0000 7fe98ac00640 1 Processor -- start 2024-04-24T08:57:20.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.587+0000 7fe98ac00640 1 -- start start 2024-04-24T08:57:20.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.588+0000 7fe98ac00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe984108580 0x7fe9841a4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:20.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.588+0000 7fe98ac00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe984108f40 0x7fe9841a4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:20.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.588+0000 7fe98ac00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe98410d500 0x7fe98419ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:20.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.588+0000 7fe98ac00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe984118780 con 0x7fe984108f40 2024-04-24T08:57:20.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.588+0000 7fe98ac00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe984118600 con 0x7fe984108580 2024-04-24T08:57:20.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.588+0000 7fe98ac00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe984118900 con 0x7fe98410d500 2024-04-24T08:57:20.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.588+0000 7fe988e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe984108f40 0x7fe9841a4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:20.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.588+0000 7fe988e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe984108f40 0x7fe9841a4dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58840/0 (socket says 172.21.15.83:58840) 2024-04-24T08:57:20.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.588+0000 7fe988e00640 1 -- 172.21.15.83:0/1711488013 learned_addr learned my addr 172.21.15.83:0/1711488013 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:57:20.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.589+0000 7fe989800640 1 --2- 172.21.15.83:0/1711488013 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe984108580 0x7fe9841a4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:20.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.589+0000 7fe98a200640 1 --2- 172.21.15.83:0/1711488013 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe98410d500 0x7fe98419ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:20.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.589+0000 7fe988e00640 1 -- 172.21.15.83:0/1711488013 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe98410d500 msgr2=0x7fe98419ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:20.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.589+0000 7fe988e00640 1 --2- 172.21.15.83:0/1711488013 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe98410d500 0x7fe98419ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:20.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.589+0000 7fe988e00640 1 -- 172.21.15.83:0/1711488013 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe984108580 msgr2=0x7fe9841a4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:20.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.589+0000 7fe988e00640 1 --2- 172.21.15.83:0/1711488013 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe984108580 0x7fe9841a4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:20.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.589+0000 7fe988e00640 1 -- 172.21.15.83:0/1711488013 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe98419f200 con 0x7fe984108f40 2024-04-24T08:57:20.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.589+0000 7fe988e00640 1 --2- 172.21.15.83:0/1711488013 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe984108f40 0x7fe9841a4dc0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fe97000d950 tx=0x7fe97000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:20.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.589+0000 7fe982000640 1 -- 172.21.15.83:0/1711488013 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe970017070 con 0x7fe984108f40 2024-04-24T08:57:20.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.589+0000 7fe982000640 1 -- 172.21.15.83:0/1711488013 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe970009c30 con 0x7fe984108f40 2024-04-24T08:57:20.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.589+0000 7fe98ac00640 1 -- 172.21.15.83:0/1711488013 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe98419f4f0 con 0x7fe984108f40 2024-04-24T08:57:20.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.589+0000 7fe982000640 1 -- 172.21.15.83:0/1711488013 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe97000f620 con 0x7fe984108f40 2024-04-24T08:57:20.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.590+0000 7fe98ac00640 1 -- 172.21.15.83:0/1711488013 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9841ab740 con 0x7fe984108f40 2024-04-24T08:57:20.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.594+0000 7fe98f006740 1 Processor -- start 2024-04-24T08:57:20.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.594+0000 7fe98f006740 1 -- start start 2024-04-24T08:57:20.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.594+0000 7fe98f006740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5645d3079a30 0x5645d3079e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:20.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.594+0000 7fe98f006740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645d307a350 0x5645d30844e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:20.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.595+0000 7fe98f006740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5645d3084a20 0x5645d3086e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:20.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.595+0000 7fe98f006740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5645d2fdf1f0 con 0x5645d3079a30 2024-04-24T08:57:20.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.595+0000 7fe98f006740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5645d2f85350 con 0x5645d307a350 2024-04-24T08:57:20.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.595+0000 7fe98f006740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5645d2fdb1b0 con 0x5645d3084a20 2024-04-24T08:57:20.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.595+0000 7fe98a200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5645d3079a30 0x5645d3079e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:20.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.595+0000 7fe982000640 1 -- 172.21.15.83:0/1711488013 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe9700109a0 con 0x7fe984108f40 2024-04-24T08:57:20.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.595+0000 7fe98a200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5645d3079a30 0x5645d3079e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58850/0 (socket says 172.21.15.83:58850) 2024-04-24T08:57:20.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.595+0000 7fe98a200640 1 -- 172.21.15.83:0/2715128641 learned_addr learned my addr 172.21.15.83:0/2715128641 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:57:20.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.595+0000 7fe989800640 1 --2- 172.21.15.83:0/2715128641 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645d307a350 0x5645d30844e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:20.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.595+0000 7fe98a200640 1 -- 172.21.15.83:0/2715128641 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5645d3084a20 msgr2=0x5645d3086e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:20.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.595+0000 7fe98a200640 1 --2- 172.21.15.83:0/2715128641 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5645d3084a20 0x5645d3086e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:20.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.595+0000 7fe98a200640 1 -- 172.21.15.83:0/2715128641 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645d307a350 msgr2=0x5645d30844e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:20.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.595+0000 7fe98a200640 1 --2- 172.21.15.83:0/2715128641 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645d307a350 0x5645d30844e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:20.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.595+0000 7fe98a200640 1 -- 172.21.15.83:0/2715128641 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645d2fd9f50 con 0x5645d3079a30 2024-04-24T08:57:20.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.595+0000 7fe989800640 1 --2- 172.21.15.83:0/2715128641 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645d307a350 0x5645d30844e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:57:20.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.596+0000 7fe98a200640 1 --2- 172.21.15.83:0/2715128641 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5645d3079a30 0x5645d3079e10 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fe97c035000 tx=0x7fe97c009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5b2550c2714709f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:20.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.596+0000 7fe976a00640 1 -- 172.21.15.83:0/2715128641 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe97c031850 con 0x5645d3079a30 2024-04-24T08:57:20.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.596+0000 7fe976a00640 1 -- 172.21.15.83:0/2715128641 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe97c036040 con 0x5645d3079a30 2024-04-24T08:57:20.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.596+0000 7fe976a00640 1 -- 172.21.15.83:0/2715128641 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe97c004440 con 0x5645d3079a30 2024-04-24T08:57:20.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.597+0000 7fe982000640 1 --2- 172.21.15.83:0/1711488013 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe96c0a3da0 0x7fe96c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:20.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.597+0000 7fe982000640 1 -- 172.21.15.83:0/1711488013 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7fe9700d0890 con 0x7fe984108f40 2024-04-24T08:57:20.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.597+0000 7fe989800640 1 --2- 172.21.15.83:0/1711488013 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe96c0a3da0 0x7fe96c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:20.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.597+0000 7fe98f006740 1 -- 172.21.15.83:0/2715128641 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5645d3079a30 msgr2=0x5645d3079e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:20.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.597+0000 7fe98f006740 1 --2- 172.21.15.83:0/2715128641 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5645d3079a30 0x5645d3079e10 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fe97c035000 tx=0x7fe97c009a20 comp rx=0 tx=0).stop 2024-04-24T08:57:20.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.597+0000 7fe98f006740 1 -- 172.21.15.83:0/2715128641 shutdown_connections 2024-04-24T08:57:20.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.597+0000 7fe98f006740 1 --2- 172.21.15.83:0/2715128641 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5645d3084a20 0x5645d3086e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:20.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.597+0000 7fe98f006740 1 --2- 172.21.15.83:0/2715128641 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645d307a350 0x5645d30844e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:20.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.597+0000 7fe98f006740 1 --2- 172.21.15.83:0/2715128641 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5645d3079a30 0x5645d3079e10 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:20.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.597+0000 7fe98f006740 1 -- 172.21.15.83:0/2715128641 >> 172.21.15.83:0/2715128641 conn(0x5645d3075d50 msgr2=0x5645d3076130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:20.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.597+0000 7fe98f006740 1 -- 172.21.15.83:0/2715128641 shutdown_connections 2024-04-24T08:57:20.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.597+0000 7fe98f006740 1 -- 172.21.15.83:0/2715128641 wait complete. 2024-04-24T08:57:20.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.598+0000 7fe989800640 1 --2- 172.21.15.83:0/1711488013 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe96c0a3da0 0x7fe96c0a6260 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x5645d307aff0 tx=0x7fe978005f70 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:20.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.598+0000 7fe98f006740 1 -- start start 2024-04-24T08:57:20.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.598+0000 7fe98f006740 1 Processor -- start 2024-04-24T08:57:20.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.599+0000 7fe98f006740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5645d3079a30 0x5645d3081970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:20.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.599+0000 7fe98f006740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645d307a350 0x5645d3081eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:20.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.599+0000 7fe98f006740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5645d3084a20 0x5645d30823f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:20.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.599+0000 7fe98f006740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5645d30877b0 con 0x5645d3084a20 2024-04-24T08:57:20.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.599+0000 7fe98f006740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5645d2f98690 con 0x5645d307a350 2024-04-24T08:57:20.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.599+0000 7fe98f006740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5645d3087930 con 0x5645d3079a30 2024-04-24T08:57:20.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.599+0000 7fe98f006740 10 client.0 _ll_register_callbacks cb 0x5645d2fd4650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:57:20.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.599+0000 7fe98a200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5645d3084a20 0x5645d30823f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:20.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.599+0000 7fe98a200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5645d3084a20 0x5645d30823f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58852/0 (socket says 172.21.15.83:58852) 2024-04-24T08:57:20.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.599+0000 7fe98a200640 1 -- 172.21.15.83:0/584349978 learned_addr learned my addr 172.21.15.83:0/584349978 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:57:20.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.599+0000 7fe989800640 1 --2- 172.21.15.83:0/584349978 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5645d3079a30 0x5645d3081970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:20.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.599+0000 7fe988e00640 1 --2- 172.21.15.83:0/584349978 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645d307a350 0x5645d3081eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:20.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.600+0000 7fe98a200640 1 -- 172.21.15.83:0/584349978 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5645d3079a30 msgr2=0x5645d3081970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:20.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.600+0000 7fe98a200640 1 --2- 172.21.15.83:0/584349978 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5645d3079a30 0x5645d3081970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:20.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.600+0000 7fe98a200640 1 -- 172.21.15.83:0/584349978 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645d307a350 msgr2=0x5645d3081eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:20.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.600+0000 7fe98a200640 1 --2- 172.21.15.83:0/584349978 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645d307a350 0x5645d3081eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:20.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.600+0000 7fe98a200640 1 -- 172.21.15.83:0/584349978 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645d2f97920 con 0x5645d3084a20 2024-04-24T08:57:20.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.600+0000 7fe989800640 1 --2- 172.21.15.83:0/584349978 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5645d3079a30 0x5645d3081970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:57:20.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.600+0000 7fe98a200640 1 --2- 172.21.15.83:0/584349978 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5645d3084a20 0x5645d30823f0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fe97c009bf0 tx=0x7fe97c003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13ba45028528901c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:20.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.600+0000 7fe96b400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:57:20.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.600+0000 7fe96b400640 1 -- 172.21.15.83:0/584349978 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe97c004140 con 0x5645d3084a20 2024-04-24T08:57:20.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.600+0000 7fe98f006740 1 -- 172.21.15.83:0/584349978 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe97c036040 con 0x5645d3084a20 2024-04-24T08:57:20.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.600+0000 7fe96b400640 1 -- 172.21.15.83:0/584349978 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7fe97c0042e0 con 0x5645d3084a20 2024-04-24T08:57:20.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.601+0000 7fe96b400640 1 -- 172.21.15.83:0/584349978 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fe97c0044e0 con 0x5645d3084a20 2024-04-24T08:57:20.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.601+0000 7fe98f006740 10 client.17655 fetch_fsmap learned FSMap version 33 2024-04-24T08:57:20.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.601+0000 7fe98f006740 1 -- 172.21.15.83:0/584349978 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5645d2dbb7d0 con 0x5645d3084a20 2024-04-24T08:57:20.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.601+0000 7fe96b400640 1 -- 172.21.15.83:0/584349978 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe97c036040 con 0x5645d3084a20 2024-04-24T08:57:20.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.601+0000 7fe96b400640 1 -- 172.21.15.83:0/584349978 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fe97c030070 con 0x5645d3084a20 2024-04-24T08:57:20.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.601+0000 7fe98f006740 10 client.17655 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T08:57:20.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.601+0000 7fe98f006740 10 client.17655 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:57:20.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.601+0000 7fe98f006740 20 client.17655 populate_metadata read hostname 'smithi083' 2024-04-24T08:57:20.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.601+0000 7fe98f006740 1 --2- 172.21.15.83:0/584349978 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5645d3094b50 0x5645d3096f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:20.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.601+0000 7fe98f006740 4 client.17655 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:57:20.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.601+0000 7fe98f006740 1 -- 172.21.15.83:0/584349978 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5645d2fecc10 con 0x5645d3094b50 2024-04-24T08:57:20.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.602+0000 7fe989800640 1 --2- 172.21.15.83:0/584349978 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5645d3094b50 0x5645d3096f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:20.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.603+0000 7fe989800640 1 --2- 172.21.15.83:0/584349978 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5645d3094b50 0x5645d3096f40 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3bdb609a8fdc0bc server_cookie=f8fc01a8edb68c6 in_seq=0 out_seq=0 2024-04-24T08:57:20.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.603+0000 7fe96b400640 10 client.17655 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:57:20.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.605+0000 7fe96b400640 1 -- 172.21.15.83:0/584349978 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5645d2fecc10 con 0x5645d3094b50 2024-04-24T08:57:20.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.605+0000 7fe96b400640 10 client.17655 handle_command_reply: tid=0 2024-04-24T08:57:20.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.605+0000 7fe98f006740 1 client.17655 shutdown 2024-04-24T08:57:20.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.605+0000 7fe98f006740 1 -- 172.21.15.83:0/584349978 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5645d3084a20 msgr2=0x5645d30823f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:20.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.605+0000 7fe98f006740 1 --2- 172.21.15.83:0/584349978 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5645d3084a20 0x5645d30823f0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fe97c009bf0 tx=0x7fe97c003c20 comp rx=0 tx=0).stop 2024-04-24T08:57:20.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.605+0000 7fe98f006740 1 -- 172.21.15.83:0/584349978 shutdown_connections 2024-04-24T08:57:20.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.605+0000 7fe98f006740 1 --2- 172.21.15.83:0/584349978 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5645d3094b50 0x5645d3096f40 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:20.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.605+0000 7fe98f006740 1 --2- 172.21.15.83:0/584349978 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5645d3084a20 0x5645d30823f0 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:20.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.605+0000 7fe98f006740 1 --2- 172.21.15.83:0/584349978 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645d307a350 0x5645d3081eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:20.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.605+0000 7fe98f006740 1 --2- 172.21.15.83:0/584349978 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5645d3079a30 0x5645d3081970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:20.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.605+0000 7fe98f006740 1 -- 172.21.15.83:0/584349978 >> 172.21.15.83:0/584349978 conn(0x5645d3075d50 msgr2=0x5645d3074fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:20.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.605+0000 7fe96b400640 0 client.17655 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:57:20.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.606+0000 7fe98f006740 1 -- 172.21.15.83:0/584349978 shutdown_connections 2024-04-24T08:57:20.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.606+0000 7fe98f006740 1 -- 172.21.15.83:0/584349978 wait complete. 2024-04-24T08:57:20.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.606+0000 7fe98f006740 20 client.17655 trim_cache size 0 max 16384 2024-04-24T08:57:20.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.626+0000 7fe98f006740 1 Processor -- start 2024-04-24T08:57:20.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.626+0000 7fe98f006740 1 -- start start 2024-04-24T08:57:20.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.626+0000 7fe98f006740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5645d3079a30 0x5645d30906d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:20.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.626+0000 7fe98f006740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645d307a350 0x5645d309c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:20.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.626+0000 7fe98f006740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5645d3084a20 0x5645d309ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:20.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.626+0000 7fe98f006740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe9841b58f0 con 0x5645d3084a20 2024-04-24T08:57:20.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.626+0000 7fe98f006740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe9841b7430 con 0x5645d307a350 2024-04-24T08:57:20.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.626+0000 7fe98f006740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe9841b5fc0 con 0x5645d3079a30 2024-04-24T08:57:20.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.627+0000 7fe98a200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5645d3084a20 0x5645d309ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:20.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.627+0000 7fe98a200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5645d3084a20 0x5645d309ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58862/0 (socket says 172.21.15.83:58862) 2024-04-24T08:57:20.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.627+0000 7fe98a200640 1 -- 172.21.15.83:0/1290428678 learned_addr learned my addr 172.21.15.83:0/1290428678 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:57:20.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.627+0000 7fe988e00640 1 --2- 172.21.15.83:0/1290428678 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645d307a350 0x5645d309c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:20.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.627+0000 7fe989800640 1 --2- 172.21.15.83:0/1290428678 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5645d3079a30 0x5645d30906d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:20.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.627+0000 7fe98a200640 1 -- 172.21.15.83:0/1290428678 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5645d3079a30 msgr2=0x5645d30906d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:20.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.627+0000 7fe98a200640 1 --2- 172.21.15.83:0/1290428678 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5645d3079a30 0x5645d30906d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:20.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.627+0000 7fe98a200640 1 -- 172.21.15.83:0/1290428678 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645d307a350 msgr2=0x5645d309c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:20.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.627+0000 7fe98a200640 1 --2- 172.21.15.83:0/1290428678 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645d307a350 0x5645d309c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:20.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.627+0000 7fe98a200640 1 -- 172.21.15.83:0/1290428678 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645d2cf13f0 con 0x5645d3084a20 2024-04-24T08:57:20.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.627+0000 7fe988e00640 1 --2- 172.21.15.83:0/1290428678 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645d307a350 0x5645d309c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:57:20.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.627+0000 7fe98a200640 1 --2- 172.21.15.83:0/1290428678 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5645d3084a20 0x5645d309ca20 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7fe97c02a920 tx=0x7fe97c009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa51071cb2125fa1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:20.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.628+0000 7fe976000640 1 -- 172.21.15.83:0/1290428678 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe97c031b60 con 0x5645d3084a20 2024-04-24T08:57:20.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.628+0000 7fe976000640 1 -- 172.21.15.83:0/1290428678 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe97c036050 con 0x5645d3084a20 2024-04-24T08:57:20.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.628+0000 7fe976000640 1 -- 172.21.15.83:0/1290428678 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe97c004280 con 0x5645d3084a20 2024-04-24T08:57:20.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.628+0000 7fe98f006740 1 -- 172.21.15.83:0/1290428678 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5645d3084a20 msgr2=0x5645d309ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:20.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.628+0000 7fe98f006740 1 --2- 172.21.15.83:0/1290428678 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5645d3084a20 0x5645d309ca20 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7fe97c02a920 tx=0x7fe97c009a20 comp rx=0 tx=0).stop 2024-04-24T08:57:20.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.629+0000 7fe98f006740 1 -- 172.21.15.83:0/1290428678 shutdown_connections 2024-04-24T08:57:20.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.629+0000 7fe98f006740 1 --2- 172.21.15.83:0/1290428678 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5645d3084a20 0x5645d309ca20 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:20.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.629+0000 7fe98f006740 1 --2- 172.21.15.83:0/1290428678 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645d307a350 0x5645d309c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:20.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.629+0000 7fe98f006740 1 --2- 172.21.15.83:0/1290428678 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5645d3079a30 0x5645d30906d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:20.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.629+0000 7fe98f006740 1 -- 172.21.15.83:0/1290428678 >> 172.21.15.83:0/1290428678 conn(0x5645d3075d50 msgr2=0x5645d307cc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:20.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.629+0000 7fe98f006740 1 -- 172.21.15.83:0/1290428678 shutdown_connections 2024-04-24T08:57:20.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.629+0000 7fe98f006740 1 -- 172.21.15.83:0/1290428678 wait complete. 2024-04-24T08:57:20.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.629+0000 7fe98f006740 1 -- start start 2024-04-24T08:57:20.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.629+0000 7fe98f006740 1 Processor -- start 2024-04-24T08:57:20.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.630+0000 7fe98f006740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5645d3079a30 0x5645d308cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:20.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.630+0000 7fe98f006740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5645d307a350 0x5645d308d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:20.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.630+0000 7fe98f006740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645d3084a20 0x5645d3088190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:20.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.630+0000 7fe98f006740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe9841b6d60 con 0x5645d3079a30 2024-04-24T08:57:20.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.630+0000 7fe98f006740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe9841b6690 con 0x5645d3084a20 2024-04-24T08:57:20.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.630+0000 7fe98f006740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe9841b38d0 con 0x5645d307a350 2024-04-24T08:57:20.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.630+0000 7fe98f006740 10 client.0 _ll_register_callbacks cb 0x7fe97c009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:57:20.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.630+0000 7fe989800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5645d3079a30 0x5645d308cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:20.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.630+0000 7fe989800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5645d3079a30 0x5645d308cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58872/0 (socket says 172.21.15.83:58872) 2024-04-24T08:57:20.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.630+0000 7fe989800640 1 -- 172.21.15.83:0/3743246155 learned_addr learned my addr 172.21.15.83:0/3743246155 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:57:20.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.630+0000 7fe98a200640 1 --2- 172.21.15.83:0/3743246155 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645d3084a20 0x5645d3088190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:20.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.630+0000 7fe988e00640 1 --2- 172.21.15.83:0/3743246155 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5645d307a350 0x5645d308d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:20.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.630+0000 7fe989800640 1 -- 172.21.15.83:0/3743246155 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5645d307a350 msgr2=0x5645d308d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:20.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.630+0000 7fe989800640 1 --2- 172.21.15.83:0/3743246155 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5645d307a350 0x5645d308d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:20.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.630+0000 7fe989800640 1 -- 172.21.15.83:0/3743246155 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645d3084a20 msgr2=0x5645d3088190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:20.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.630+0000 7fe989800640 1 --2- 172.21.15.83:0/3743246155 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645d3084a20 0x5645d3088190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:20.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.630+0000 7fe989800640 1 -- 172.21.15.83:0/3743246155 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645d2cb10f0 con 0x5645d3079a30 2024-04-24T08:57:20.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.630+0000 7fe988e00640 1 --2- 172.21.15.83:0/3743246155 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5645d307a350 0x5645d308d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:57:20.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.631+0000 7fe98a200640 1 --2- 172.21.15.83:0/3743246155 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645d3084a20 0x5645d3088190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:57:20.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.631+0000 7fe989800640 1 --2- 172.21.15.83:0/3743246155 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5645d3079a30 0x5645d308cbf0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7fe978027cc0 tx=0x7fe9780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16486eeb01b2a970 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:20.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.631+0000 7fe96b400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:57:20.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.631+0000 7fe96b400640 1 -- 172.21.15.83:0/3743246155 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9780273d0 con 0x5645d3079a30 2024-04-24T08:57:20.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.631+0000 7fe96b400640 1 -- 172.21.15.83:0/3743246155 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7fe978027570 con 0x5645d3079a30 2024-04-24T08:57:20.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.631+0000 7fe98f006740 1 -- 172.21.15.83:0/3743246155 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe97c036050 con 0x5645d3079a30 2024-04-24T08:57:20.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.631+0000 7fe96b400640 1 -- 172.21.15.83:0/3743246155 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fe978037050 con 0x5645d3079a30 2024-04-24T08:57:20.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.631+0000 7fe98f006740 10 client.17661 fetch_fsmap learned FSMap version 33 2024-04-24T08:57:20.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.631+0000 7fe98f006740 1 -- 172.21.15.83:0/3743246155 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5645d2fa4810 con 0x5645d3079a30 2024-04-24T08:57:20.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.632+0000 7fe96b400640 1 -- 172.21.15.83:0/3743246155 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe97c036050 con 0x5645d3079a30 2024-04-24T08:57:20.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.632+0000 7fe96b400640 1 -- 172.21.15.83:0/3743246155 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fe97800f040 con 0x5645d3079a30 2024-04-24T08:57:20.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.632+0000 7fe98f006740 10 client.17661 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T08:57:20.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.632+0000 7fe98f006740 10 client.17661 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:57:20.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.632+0000 7fe98f006740 20 client.17661 populate_metadata read hostname 'smithi083' 2024-04-24T08:57:20.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.632+0000 7fe98f006740 1 --2- 172.21.15.83:0/3743246155 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5645d30fad40 0x5645d30fd160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:20.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.632+0000 7fe98f006740 4 client.17661 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T08:57:20.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.632+0000 7fe98f006740 1 -- 172.21.15.83:0/3743246155 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5645d2fe6a80 con 0x5645d30fad40 2024-04-24T08:57:20.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.633+0000 7fe988e00640 1 --2- 172.21.15.83:0/3743246155 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5645d30fad40 0x5645d30fd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:20.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.634+0000 7fe988e00640 1 --2- 172.21.15.83:0/3743246155 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5645d30fad40 0x5645d30fd160 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98758f30ba43df18 server_cookie=4fb1f7ef1b55e79 in_seq=0 out_seq=0 2024-04-24T08:57:20.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.634+0000 7fe96b400640 10 client.17661 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:57:20.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.634+0000 7fe96b400640 1 -- 172.21.15.83:0/3743246155 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5645d2fe6a80 con 0x5645d30fad40 2024-04-24T08:57:20.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.634+0000 7fe96b400640 10 client.17661 handle_command_reply: tid=0 2024-04-24T08:57:20.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.634+0000 7fe98f006740 1 client.17661 shutdown 2024-04-24T08:57:20.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.634+0000 7fe98f006740 1 -- 172.21.15.83:0/3743246155 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5645d3079a30 msgr2=0x5645d308cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:20.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.634+0000 7fe98f006740 1 --2- 172.21.15.83:0/3743246155 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5645d3079a30 0x5645d308cbf0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7fe978027cc0 tx=0x7fe9780046c0 comp rx=0 tx=0).stop 2024-04-24T08:57:20.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.634+0000 7fe98f006740 1 -- 172.21.15.83:0/3743246155 shutdown_connections 2024-04-24T08:57:20.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.634+0000 7fe98f006740 1 --2- 172.21.15.83:0/3743246155 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5645d30fad40 0x5645d30fd160 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:20.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.634+0000 7fe98f006740 1 --2- 172.21.15.83:0/3743246155 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645d3084a20 0x5645d3088190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:20.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.634+0000 7fe98f006740 1 --2- 172.21.15.83:0/3743246155 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5645d307a350 0x5645d308d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:20.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.634+0000 7fe96b400640 0 client.17661 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:57:20.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.634+0000 7fe98f006740 1 --2- 172.21.15.83:0/3743246155 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5645d3079a30 0x5645d308cbf0 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:20.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.634+0000 7fe98f006740 1 -- 172.21.15.83:0/3743246155 >> 172.21.15.83:0/3743246155 conn(0x5645d3075d50 msgr2=0x5645d30831a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:20.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.635+0000 7fe98f006740 1 -- 172.21.15.83:0/3743246155 shutdown_connections 2024-04-24T08:57:20.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.635+0000 7fe98f006740 1 -- 172.21.15.83:0/3743246155 wait complete. 2024-04-24T08:57:20.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.635+0000 7fe98f006740 20 client.17661 trim_cache size 0 max 16384 2024-04-24T08:57:20.641 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T08:57:20.641 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-04-24T08:57:20.641 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-04-24T08:57:20.642 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T08:57:20.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.635+0000 7fe98ac00640 1 -- 172.21.15.83:0/1711488013 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe96c0a3da0 msgr2=0x7fe96c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:20.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.635+0000 7fe98ac00640 1 --2- 172.21.15.83:0/1711488013 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe96c0a3da0 0x7fe96c0a6260 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x5645d307aff0 tx=0x7fe978005f70 comp rx=0 tx=0).stop 2024-04-24T08:57:20.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.635+0000 7fe98ac00640 1 -- 172.21.15.83:0/1711488013 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe984108f40 msgr2=0x7fe9841a4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:20.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.635+0000 7fe98ac00640 1 --2- 172.21.15.83:0/1711488013 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe984108f40 0x7fe9841a4dc0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fe97000d950 tx=0x7fe97000de10 comp rx=0 tx=0).stop 2024-04-24T08:57:20.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.635+0000 7fe98ac00640 1 -- 172.21.15.83:0/1711488013 shutdown_connections 2024-04-24T08:57:20.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.635+0000 7fe98ac00640 1 --2- 172.21.15.83:0/1711488013 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe96c0a3da0 0x7fe96c0a6260 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:20.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.635+0000 7fe98ac00640 1 --2- 172.21.15.83:0/1711488013 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe98410d500 0x7fe98419ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:20.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.635+0000 7fe98ac00640 1 --2- 172.21.15.83:0/1711488013 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe984108f40 0x7fe9841a4dc0 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:20.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.635+0000 7fe98ac00640 1 --2- 172.21.15.83:0/1711488013 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe984108580 0x7fe9841a4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:20.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.636+0000 7fe98ac00640 1 -- 172.21.15.83:0/1711488013 >> 172.21.15.83:0/1711488013 conn(0x7fe984077e20 msgr2=0x7fe984078480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:20.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.636+0000 7fe98ac00640 1 -- 172.21.15.83:0/1711488013 shutdown_connections 2024-04-24T08:57:20.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.636+0000 7fe98ac00640 1 -- 172.21.15.83:0/1711488013 wait complete. 2024-04-24T08:57:20.652 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T08:57:20.652 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T08:57:20.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.862+0000 7fb2bee00640 1 -- 172.21.15.83:0/2703111719 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb2b8103bb0 msgr2=0x7fb2b810c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:20.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.862+0000 7fb2bee00640 1 --2- 172.21.15.83:0/2703111719 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb2b8103bb0 0x7fb2b810c090 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fb2a0009900 tx=0x7fb2a002a510 comp rx=0 tx=0).stop 2024-04-24T08:57:20.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.862+0000 7fb2bee00640 1 -- 172.21.15.83:0/2703111719 shutdown_connections 2024-04-24T08:57:20.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.862+0000 7fb2bee00640 1 --2- 172.21.15.83:0/2703111719 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb2b810c5d0 0x7fb2b810ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:20.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.862+0000 7fb2bee00640 1 --2- 172.21.15.83:0/2703111719 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb2b8103bb0 0x7fb2b810c090 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:20.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.862+0000 7fb2bee00640 1 --2- 172.21.15.83:0/2703111719 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2b8103290 0x7fb2b8103670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:20.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.862+0000 7fb2bee00640 1 -- 172.21.15.83:0/2703111719 >> 172.21.15.83:0/2703111719 conn(0x7fb2b8078000 msgr2=0x7fb2b8078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:20.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.862+0000 7fb2bee00640 1 -- 172.21.15.83:0/2703111719 shutdown_connections 2024-04-24T08:57:20.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.862+0000 7fb2bee00640 1 -- 172.21.15.83:0/2703111719 wait complete. 2024-04-24T08:57:20.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.863+0000 7fb2bee00640 1 Processor -- start 2024-04-24T08:57:20.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.864+0000 7fb2bee00640 1 -- start start 2024-04-24T08:57:20.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.864+0000 7fb2bee00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb2b8103290 0x7fb2b81a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:20.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.864+0000 7fb2bee00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2b8103bb0 0x7fb2b81a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:20.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.864+0000 7fb2bee00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb2b810c5d0 0x7fb2b819a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:20.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.864+0000 7fb2bee00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b8112260 con 0x7fb2b8103290 2024-04-24T08:57:20.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.864+0000 7fb2bee00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b81120e0 con 0x7fb2b8103bb0 2024-04-24T08:57:20.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.864+0000 7fb2bee00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b81123e0 con 0x7fb2b810c5d0 2024-04-24T08:57:20.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.864+0000 7fb2b7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb2b8103290 0x7fb2b81a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:20.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.864+0000 7fb2b7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb2b8103290 0x7fb2b81a0690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58890/0 (socket says 172.21.15.83:58890) 2024-04-24T08:57:20.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.864+0000 7fb2b7e00640 1 -- 172.21.15.83:0/544681650 learned_addr learned my addr 172.21.15.83:0/544681650 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:57:20.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.865+0000 7fb2bce00640 1 --2- 172.21.15.83:0/544681650 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb2b810c5d0 0x7fb2b819a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:20.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.865+0000 7fb2af400640 1 --2- 172.21.15.83:0/544681650 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2b8103bb0 0x7fb2b81a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:20.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.865+0000 7fb2b7e00640 1 -- 172.21.15.83:0/544681650 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb2b810c5d0 msgr2=0x7fb2b819a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:20.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.865+0000 7fb2b7e00640 1 --2- 172.21.15.83:0/544681650 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb2b810c5d0 0x7fb2b819a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:20.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.865+0000 7fb2b7e00640 1 -- 172.21.15.83:0/544681650 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2b8103bb0 msgr2=0x7fb2b81a0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:20.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.865+0000 7fb2b7e00640 1 --2- 172.21.15.83:0/544681650 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2b8103bb0 0x7fb2b81a0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:20.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.865+0000 7fb2b7e00640 1 -- 172.21.15.83:0/544681650 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2b819afd0 con 0x7fb2b8103290 2024-04-24T08:57:20.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.865+0000 7fb2b7e00640 1 --2- 172.21.15.83:0/544681650 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb2b8103290 0x7fb2b81a0690 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7fb2a800d950 tx=0x7fb2a800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:20.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.865+0000 7fb2b5600640 1 -- 172.21.15.83:0/544681650 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2a8017070 con 0x7fb2b8103290 2024-04-24T08:57:20.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.865+0000 7fb2b5600640 1 -- 172.21.15.83:0/544681650 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb2a8009c30 con 0x7fb2b8103290 2024-04-24T08:57:20.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.865+0000 7fb2bee00640 1 -- 172.21.15.83:0/544681650 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2b819b2c0 con 0x7fb2b8103290 2024-04-24T08:57:20.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.865+0000 7fb2b5600640 1 -- 172.21.15.83:0/544681650 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2a800f620 con 0x7fb2b8103290 2024-04-24T08:57:20.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.866+0000 7fb2bee00640 1 -- 172.21.15.83:0/544681650 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2b81a74e0 con 0x7fb2b8103290 2024-04-24T08:57:20.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.867+0000 7fb2bee00640 1 -- 172.21.15.83:0/544681650 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2b8069990 con 0x7fb2b8103290 2024-04-24T08:57:20.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.869+0000 7fb2b5600640 1 -- 172.21.15.83:0/544681650 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb2a80109a0 con 0x7fb2b8103290 2024-04-24T08:57:20.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.870+0000 7fb2b5600640 1 --2- 172.21.15.83:0/544681650 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb29c0a3d20 0x7fb29c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:20.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.870+0000 7fb2b5600640 1 -- 172.21.15.83:0/544681650 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7fb2a80d08c0 con 0x7fb2b8103290 2024-04-24T08:57:20.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.871+0000 7fb2af400640 1 --2- 172.21.15.83:0/544681650 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb29c0a3d20 0x7fb29c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:20.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.871+0000 7fb2af400640 1 --2- 172.21.15.83:0/544681650 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb29c0a3d20 0x7fb29c0a61e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb2a0006010 tx=0x7fb2a0000bc0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:20.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:20.875+0000 7fb2b5600640 1 -- 172.21.15.83:0/544681650 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb2a8009de0 con 0x7fb2b8103290 2024-04-24T08:57:21.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.147+0000 7fb2bee00640 1 -- 172.21.15.83:0/544681650 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb2b8068e20 con 0x7fb2b8103290 2024-04-24T08:57:21.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.149+0000 7fb2b5600640 1 -- 172.21.15.83:0/544681650 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11944 (secure 0 0 0) 0x7fb2a809cb90 con 0x7fb2b8103290 2024-04-24T08:57:21.149 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:57:21.149 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/358872257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":358872257},{"type":"v1","addr":"172.21.15.83:6835","nonce":358872257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3157070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3157070851},{"type":"v1","addr":"172.21.15.83:6837","nonce":3157070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":33,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T08:48:29.422576+0000","modified":"2024-04-24T08:56:17.048967+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34520,"mds_1":34493,"mds_2":34451,"mds_3":34427,"mds_4":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14658":{"gid":14658,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6839/4085470844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":4085470844},{"type":"v1","addr":"172.21.15.83:6839","nonce":4085470844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14676":{"gid":14676,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6841/2634754210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":2634754210},{"type":"v1","addr":"172.21.15.83:6841","nonce":2634754210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24382":{"gid":24382,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/1859921310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1859921310},{"type":"v1","addr":"172.21.15.189:6833","nonce":1859921310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/2241923197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2241923197},{"type":"v1","addr":"172.21.15.189:6835","nonce":2241923197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6837/3172311220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":3172311220},{"type":"v1","addr":"172.21.15.189:6837","nonce":3172311220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.189:6839/2548965336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":2548965336},{"type":"v1","addr":"172.21.15.189:6839","nonce":2548965336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.138:6833/1157967658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1157967658},{"type":"v1","addr":"172.21.15.138:6833","nonce":1157967658}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.138:6835/1909219758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1909219758},{"type":"v1","addr":"172.21.15.138:6835","nonce":1909219758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34493":{"gid":34493,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.138:6837/900566754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":900566754},{"type":"v1","addr":"172.21.15.138:6837","nonce":900566754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34520":{"gid":34520,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.138:6839/2236932393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2236932393},{"type":"v1","addr":"172.21.15.138:6839","nonce":2236932393}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34520,"qdb_cluster":[24454,34520,34427,34451,34493]},"id":1}]} 2024-04-24T08:57:21.149 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 33 2024-04-24T08:57:21.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.152+0000 7fb2bee00640 1 -- 172.21.15.83:0/544681650 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb29c0a3d20 msgr2=0x7fb29c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:21.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.152+0000 7fb2bee00640 1 --2- 172.21.15.83:0/544681650 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb29c0a3d20 0x7fb29c0a61e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb2a0006010 tx=0x7fb2a0000bc0 comp rx=0 tx=0).stop 2024-04-24T08:57:21.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.152+0000 7fb2bee00640 1 -- 172.21.15.83:0/544681650 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb2b8103290 msgr2=0x7fb2b81a0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:21.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.152+0000 7fb2bee00640 1 --2- 172.21.15.83:0/544681650 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb2b8103290 0x7fb2b81a0690 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7fb2a800d950 tx=0x7fb2a800de10 comp rx=0 tx=0).stop 2024-04-24T08:57:21.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.152+0000 7fb2bee00640 1 -- 172.21.15.83:0/544681650 shutdown_connections 2024-04-24T08:57:21.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.152+0000 7fb2bee00640 1 --2- 172.21.15.83:0/544681650 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb29c0a3d20 0x7fb29c0a61e0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:21.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.152+0000 7fb2bee00640 1 --2- 172.21.15.83:0/544681650 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb2b810c5d0 0x7fb2b819a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:21.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.152+0000 7fb2bee00640 1 --2- 172.21.15.83:0/544681650 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2b8103bb0 0x7fb2b81a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:21.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.152+0000 7fb2bee00640 1 --2- 172.21.15.83:0/544681650 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb2b8103290 0x7fb2b81a0690 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:21.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.152+0000 7fb2bee00640 1 -- 172.21.15.83:0/544681650 >> 172.21.15.83:0/544681650 conn(0x7fb2b8078000 msgr2=0x7fb2b8108540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:21.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.153+0000 7fb2bee00640 1 -- 172.21.15.83:0/544681650 shutdown_connections 2024-04-24T08:57:21.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.153+0000 7fb2bee00640 1 -- 172.21.15.83:0/544681650 wait complete. 2024-04-24T08:57:21.173 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-24T08:57:21.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.380+0000 7f91ad800640 1 -- 172.21.15.83:0/1981591286 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f91a810e820 msgr2=0x7f91a8110c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:21.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.380+0000 7f91ad800640 1 --2- 172.21.15.83:0/1981591286 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f91a810e820 0x7f91a8110c10 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f91a0009ca0 tx=0x7f91a002a8c0 comp rx=0 tx=0).stop 2024-04-24T08:57:21.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.381+0000 7f91ad800640 1 -- 172.21.15.83:0/1981591286 shutdown_connections 2024-04-24T08:57:21.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.381+0000 7f91ad800640 1 --2- 172.21.15.83:0/1981591286 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f91a810e820 0x7f91a8110c10 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:21.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.381+0000 7f91ad800640 1 --2- 172.21.15.83:0/1981591286 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91a8101620 0x7f91a810e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:21.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.381+0000 7f91ad800640 1 --2- 172.21.15.83:0/1981591286 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f91a8100c70 0x7f91a8101050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:21.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.381+0000 7f91ad800640 1 -- 172.21.15.83:0/1981591286 >> 172.21.15.83:0/1981591286 conn(0x7f91a80fc880 msgr2=0x7f91a80feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:21.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.381+0000 7f91ad800640 1 -- 172.21.15.83:0/1981591286 shutdown_connections 2024-04-24T08:57:21.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.381+0000 7f91ad800640 1 -- 172.21.15.83:0/1981591286 wait complete. 2024-04-24T08:57:21.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.382+0000 7f91ad800640 1 Processor -- start 2024-04-24T08:57:21.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.382+0000 7f91ad800640 1 -- start start 2024-04-24T08:57:21.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.382+0000 7f91ad800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f91a8100c70 0x7f91a81a05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:21.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.382+0000 7f91ad800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91a8101620 0x7f91a81a0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:21.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.382+0000 7f91ad800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f91a810e820 0x7f91a819a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:21.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.382+0000 7f91ad800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f91a8114370 con 0x7f91a8100c70 2024-04-24T08:57:21.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.382+0000 7f91ad800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f91a81141f0 con 0x7f91a8101620 2024-04-24T08:57:21.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.382+0000 7f91ad800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f91a81144f0 con 0x7f91a810e820 2024-04-24T08:57:21.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.382+0000 7f91a7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f91a8100c70 0x7f91a81a05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:21.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.383+0000 7f91a7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f91a8100c70 0x7f91a81a05d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58908/0 (socket says 172.21.15.83:58908) 2024-04-24T08:57:21.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.383+0000 7f91a7e00640 1 -- 172.21.15.83:0/3351893443 learned_addr learned my addr 172.21.15.83:0/3351893443 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:57:21.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.383+0000 7f919fe00640 1 --2- 172.21.15.83:0/3351893443 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91a8101620 0x7f91a81a0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:21.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.383+0000 7f91ace00640 1 --2- 172.21.15.83:0/3351893443 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f91a810e820 0x7f91a819a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:21.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.383+0000 7f91a7e00640 1 -- 172.21.15.83:0/3351893443 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f91a810e820 msgr2=0x7f91a819a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:21.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.383+0000 7f91a7e00640 1 --2- 172.21.15.83:0/3351893443 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f91a810e820 0x7f91a819a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:21.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.383+0000 7f91a7e00640 1 -- 172.21.15.83:0/3351893443 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91a8101620 msgr2=0x7f91a81a0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:21.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.383+0000 7f91a7e00640 1 --2- 172.21.15.83:0/3351893443 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91a8101620 0x7f91a81a0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:21.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.383+0000 7f91a7e00640 1 -- 172.21.15.83:0/3351893443 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91a819af50 con 0x7f91a8100c70 2024-04-24T08:57:21.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.383+0000 7f919fe00640 1 --2- 172.21.15.83:0/3351893443 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91a8101620 0x7f91a81a0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:57:21.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.383+0000 7f91a7e00640 1 --2- 172.21.15.83:0/3351893443 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f91a8100c70 0x7f91a81a05d0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f919800d770 tx=0x7f919800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:21.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.384+0000 7f91a5600640 1 -- 172.21.15.83:0/3351893443 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9198017070 con 0x7f91a8100c70 2024-04-24T08:57:21.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.384+0000 7f91a5600640 1 -- 172.21.15.83:0/3351893443 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f91980054d0 con 0x7f91a8100c70 2024-04-24T08:57:21.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.384+0000 7f91a5600640 1 -- 172.21.15.83:0/3351893443 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9198004cf0 con 0x7f91a8100c70 2024-04-24T08:57:21.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.384+0000 7f91ad800640 1 -- 172.21.15.83:0/3351893443 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91a819b240 con 0x7f91a8100c70 2024-04-24T08:57:21.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.384+0000 7f91ad800640 1 -- 172.21.15.83:0/3351893443 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91a81a7550 con 0x7f91a8100c70 2024-04-24T08:57:21.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.387+0000 7f91b1c5c740 1 Processor -- start 2024-04-24T08:57:21.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.387+0000 7f91b1c5c740 1 -- start start 2024-04-24T08:57:21.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.387+0000 7f91a5600640 1 -- 172.21.15.83:0/3351893443 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9198004e90 con 0x7f91a8100c70 2024-04-24T08:57:21.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.388+0000 7f91b1c5c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5591f9256b40 0x5591f9256f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:21.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.388+0000 7f91b1c5c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5591f9257460 0x5591f92615f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:21.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.388+0000 7f91b1c5c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5591f9261b30 0x5591f9263f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:21.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.388+0000 7f91b1c5c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5591f91bc1d0 con 0x5591f9257460 2024-04-24T08:57:21.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.388+0000 7f91b1c5c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5591f9162450 con 0x5591f9261b30 2024-04-24T08:57:21.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.388+0000 7f91b1c5c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5591f91b8190 con 0x5591f9256b40 2024-04-24T08:57:21.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.388+0000 7f91a7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5591f9257460 0x5591f92615f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:21.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.388+0000 7f91a7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5591f9257460 0x5591f92615f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58918/0 (socket says 172.21.15.83:58918) 2024-04-24T08:57:21.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.388+0000 7f91a7e00640 1 -- 172.21.15.83:0/734539416 learned_addr learned my addr 172.21.15.83:0/734539416 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:57:21.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.388+0000 7f919fe00640 1 --2- 172.21.15.83:0/734539416 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5591f9261b30 0x5591f9263f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:21.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.388+0000 7f91ace00640 1 --2- 172.21.15.83:0/734539416 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5591f9256b40 0x5591f9256f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:21.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.389+0000 7f91a7e00640 1 -- 172.21.15.83:0/734539416 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5591f9256b40 msgr2=0x5591f9256f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:21.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.389+0000 7f91a7e00640 1 --2- 172.21.15.83:0/734539416 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5591f9256b40 0x5591f9256f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:21.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.389+0000 7f91a7e00640 1 -- 172.21.15.83:0/734539416 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5591f9261b30 msgr2=0x5591f9263f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:21.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.389+0000 7f91a7e00640 1 --2- 172.21.15.83:0/734539416 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5591f9261b30 0x5591f9263f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:21.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.389+0000 7f91a7e00640 1 -- 172.21.15.83:0/734539416 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591f921f310 con 0x5591f9257460 2024-04-24T08:57:21.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.389+0000 7f919fe00640 1 --2- 172.21.15.83:0/734539416 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5591f9261b30 0x5591f9263f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:57:21.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.389+0000 7f91a7e00640 1 --2- 172.21.15.83:0/734539416 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5591f9257460 0x5591f92615f0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f91980cd940 tx=0x7f91980d8670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1489f3c0879c4ddd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:21.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.389+0000 7f91a5600640 1 --2- 172.21.15.83:0/3351893443 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f918c0a3da0 0x7f918c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:21.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.390+0000 7f919d600640 1 -- 172.21.15.83:0/734539416 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91980df070 con 0x5591f9257460 2024-04-24T08:57:21.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.390+0000 7f919d600640 1 -- 172.21.15.83:0/734539416 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f919809c950 con 0x5591f9257460 2024-04-24T08:57:21.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.390+0000 7f919d600640 1 -- 172.21.15.83:0/734539416 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f919809bd90 con 0x5591f9257460 2024-04-24T08:57:21.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.390+0000 7f91ace00640 1 --2- 172.21.15.83:0/3351893443 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f918c0a3da0 0x7f918c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:21.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.390+0000 7f91a5600640 1 -- 172.21.15.83:0/3351893443 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f91980d06e0 con 0x7f91a8100c70 2024-04-24T08:57:21.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.391+0000 7f91b1c5c740 1 -- 172.21.15.83:0/734539416 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5591f9257460 msgr2=0x5591f92615f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:21.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.391+0000 7f91b1c5c740 1 --2- 172.21.15.83:0/734539416 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5591f9257460 0x5591f92615f0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f91980cd940 tx=0x7f91980d8670 comp rx=0 tx=0).stop 2024-04-24T08:57:21.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.391+0000 7f91ace00640 1 --2- 172.21.15.83:0/3351893443 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f918c0a3da0 0x7f918c0a6260 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f91a0006010 tx=0x7f91a0005d80 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:21.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.391+0000 7f91b1c5c740 1 -- 172.21.15.83:0/734539416 shutdown_connections 2024-04-24T08:57:21.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.391+0000 7f91b1c5c740 1 --2- 172.21.15.83:0/734539416 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5591f9261b30 0x5591f9263f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:21.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.391+0000 7f91b1c5c740 1 --2- 172.21.15.83:0/734539416 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5591f9257460 0x5591f92615f0 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:21.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.391+0000 7f91b1c5c740 1 --2- 172.21.15.83:0/734539416 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5591f9256b40 0x5591f9256f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:21.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.391+0000 7f91b1c5c740 1 -- 172.21.15.83:0/734539416 >> 172.21.15.83:0/734539416 conn(0x5591f9252e60 msgr2=0x5591f9253240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:21.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.391+0000 7f91b1c5c740 1 -- 172.21.15.83:0/734539416 shutdown_connections 2024-04-24T08:57:21.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.391+0000 7f91b1c5c740 1 -- 172.21.15.83:0/734539416 wait complete. 2024-04-24T08:57:21.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.391+0000 7f91b1c5c740 1 -- start start 2024-04-24T08:57:21.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.392+0000 7f91b1c5c740 1 Processor -- start 2024-04-24T08:57:21.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.392+0000 7f91b1c5c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5591f9256b40 0x5591f9261510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:21.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.392+0000 7f91b1c5c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5591f9257460 0x5591f925e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:21.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.392+0000 7f91b1c5c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5591f9261b30 0x5591f925ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:21.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.392+0000 7f91b1c5c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5591f92649c0 con 0x5591f9256b40 2024-04-24T08:57:21.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.392+0000 7f91b1c5c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5591f9175810 con 0x5591f9257460 2024-04-24T08:57:21.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.392+0000 7f91b1c5c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5591f9264b40 con 0x5591f9261b30 2024-04-24T08:57:21.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.392+0000 7f91b1c5c740 10 client.0 _ll_register_callbacks cb 0x5591f9143140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:57:21.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.392+0000 7f91a7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5591f9256b40 0x5591f9261510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:21.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.392+0000 7f91a7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5591f9256b40 0x5591f9261510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58932/0 (socket says 172.21.15.83:58932) 2024-04-24T08:57:21.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.392+0000 7f91a7e00640 1 -- 172.21.15.83:0/2820945125 learned_addr learned my addr 172.21.15.83:0/2820945125 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:57:21.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.393+0000 7f919fe00640 1 --2- 172.21.15.83:0/2820945125 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5591f9257460 0x5591f925e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:21.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.393+0000 7f91ace00640 1 --2- 172.21.15.83:0/2820945125 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5591f9261b30 0x5591f925ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:21.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.393+0000 7f91a7e00640 1 -- 172.21.15.83:0/2820945125 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5591f9261b30 msgr2=0x5591f925ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:21.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.393+0000 7f91a7e00640 1 --2- 172.21.15.83:0/2820945125 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5591f9261b30 0x5591f925ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:21.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.393+0000 7f91a7e00640 1 -- 172.21.15.83:0/2820945125 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5591f9257460 msgr2=0x5591f925e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:21.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.393+0000 7f91a7e00640 1 --2- 172.21.15.83:0/2820945125 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5591f9257460 0x5591f925e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:21.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.393+0000 7f91a7e00640 1 -- 172.21.15.83:0/2820945125 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591f91863f0 con 0x5591f9256b40 2024-04-24T08:57:21.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.393+0000 7f91ace00640 1 --2- 172.21.15.83:0/2820945125 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5591f9261b30 0x5591f925ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:57:21.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.393+0000 7f919fe00640 1 --2- 172.21.15.83:0/2820945125 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5591f9257460 0x5591f925e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:57:21.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.393+0000 7f91a7e00640 1 --2- 172.21.15.83:0/2820945125 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5591f9256b40 0x5591f9261510 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f919809c190 tx=0x7f9198098560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c5e0fabd535ea78 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:21.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.393+0000 7f9196000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:57:21.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.393+0000 7f9196000640 1 -- 172.21.15.83:0/2820945125 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91980c73d0 con 0x5591f9256b40 2024-04-24T08:57:21.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.393+0000 7f9196000640 1 -- 172.21.15.83:0/2820945125 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f91980d66d0 con 0x5591f9256b40 2024-04-24T08:57:21.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.393+0000 7f91b1c5c740 1 -- 172.21.15.83:0/2820945125 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f919809c950 con 0x5591f9256b40 2024-04-24T08:57:21.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.394+0000 7f9196000640 1 -- 172.21.15.83:0/2820945125 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f91980c75c0 con 0x5591f9256b40 2024-04-24T08:57:21.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.394+0000 7f91b1c5c740 10 client.17679 fetch_fsmap learned FSMap version 33 2024-04-24T08:57:21.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.394+0000 7f91b1c5c740 1 -- 172.21.15.83:0/2820945125 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5591f8ece3f0 con 0x5591f9256b40 2024-04-24T08:57:21.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.394+0000 7f9196000640 1 -- 172.21.15.83:0/2820945125 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f919809c950 con 0x5591f9256b40 2024-04-24T08:57:21.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.394+0000 7f9196000640 1 -- 172.21.15.83:0/2820945125 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f91980988c0 con 0x5591f9256b40 2024-04-24T08:57:21.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.394+0000 7f91b1c5c740 10 client.17679 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T08:57:21.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.394+0000 7f91b1c5c740 10 client.17679 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-24T08:57:21.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.394+0000 7f91b1c5c740 20 client.17679 populate_metadata read hostname 'smithi083' 2024-04-24T08:57:21.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.394+0000 7f91b1c5c740 1 --2- 172.21.15.83:0/2820945125 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x5591f9271b60 0x5591f9273f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:21.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.394+0000 7f91b1c5c740 4 client.17679 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:57:21.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.394+0000 7f91b1c5c740 1 -- 172.21.15.83:0/2820945125 --> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5591f91c9bf0 con 0x5591f9271b60 2024-04-24T08:57:21.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.395+0000 7f919fe00640 1 --2- 172.21.15.83:0/2820945125 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x5591f9271b60 0x5591f9273f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:21.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.396+0000 7f919fe00640 1 --2- 172.21.15.83:0/2820945125 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x5591f9271b60 0x5591f9273f50 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8e75fec98f7f206b server_cookie=801e17b5eed709ec in_seq=0 out_seq=0 2024-04-24T08:57:21.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.396+0000 7f9196000640 10 client.17679 ms_handle_connect on v2:172.21.15.189:6838/2548965336 2024-04-24T08:57:21.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.398+0000 7f9196000640 1 -- 172.21.15.83:0/2820945125 <== mds.4 v2:172.21.15.189:6838/2548965336 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5591f91c9bf0 con 0x5591f9271b60 2024-04-24T08:57:21.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.398+0000 7f9196000640 10 client.17679 handle_command_reply: tid=0 2024-04-24T08:57:21.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.398+0000 7f91b1c5c740 1 client.17679 shutdown 2024-04-24T08:57:21.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.399+0000 7f91b1c5c740 1 -- 172.21.15.83:0/2820945125 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5591f9256b40 msgr2=0x5591f9261510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:21.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.399+0000 7f91b1c5c740 1 --2- 172.21.15.83:0/2820945125 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5591f9256b40 0x5591f9261510 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f919809c190 tx=0x7f9198098560 comp rx=0 tx=0).stop 2024-04-24T08:57:21.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.399+0000 7f91b1c5c740 1 -- 172.21.15.83:0/2820945125 shutdown_connections 2024-04-24T08:57:21.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.399+0000 7f91b1c5c740 1 --2- 172.21.15.83:0/2820945125 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x5591f9271b60 0x5591f9273f50 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:21.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.399+0000 7f91b1c5c740 1 --2- 172.21.15.83:0/2820945125 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5591f9261b30 0x5591f925ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:21.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.399+0000 7f91b1c5c740 1 --2- 172.21.15.83:0/2820945125 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5591f9257460 0x5591f925e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:21.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.399+0000 7f91b1c5c740 1 --2- 172.21.15.83:0/2820945125 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5591f9256b40 0x5591f9261510 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:21.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.399+0000 7f9196000640 0 client.17679 ms_handle_reset on v2:172.21.15.189:6838/2548965336 2024-04-24T08:57:21.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.399+0000 7f91b1c5c740 1 -- 172.21.15.83:0/2820945125 >> 172.21.15.83:0/2820945125 conn(0x5591f9252e60 msgr2=0x5591f9258b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:21.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.399+0000 7f91b1c5c740 1 -- 172.21.15.83:0/2820945125 shutdown_connections 2024-04-24T08:57:21.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.399+0000 7f91b1c5c740 1 -- 172.21.15.83:0/2820945125 wait complete. 2024-04-24T08:57:21.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.399+0000 7f91b1c5c740 20 client.17679 trim_cache size 0 max 16384 2024-04-24T08:57:21.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.415+0000 7f91b1c5c740 1 Processor -- start 2024-04-24T08:57:21.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.416+0000 7f91b1c5c740 1 -- start start 2024-04-24T08:57:21.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.416+0000 7f91b1c5c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5591f9256b40 0x5591f926d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:21.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.416+0000 7f91b1c5c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5591f9257460 0x5591f9261040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:21.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.416+0000 7f91b1c5c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5591f9261b30 0x5591f9262f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:21.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.416+0000 7f91b1c5c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f91a81b1520 con 0x5591f9256b40 2024-04-24T08:57:21.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.416+0000 7f91b1c5c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f91a81b3060 con 0x5591f9257460 2024-04-24T08:57:21.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.416+0000 7f91b1c5c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f91a81b1bf0 con 0x5591f9261b30 2024-04-24T08:57:21.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.416+0000 7f91a7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5591f9256b40 0x5591f926d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:21.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.416+0000 7f91a7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5591f9256b40 0x5591f926d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58938/0 (socket says 172.21.15.83:58938) 2024-04-24T08:57:21.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.416+0000 7f91a7e00640 1 -- 172.21.15.83:0/1628812109 learned_addr learned my addr 172.21.15.83:0/1628812109 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:57:21.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.417+0000 7f91ace00640 1 --2- 172.21.15.83:0/1628812109 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5591f9261b30 0x5591f9262f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:21.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.417+0000 7f91a7e00640 1 -- 172.21.15.83:0/1628812109 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5591f9261b30 msgr2=0x5591f9262f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:21.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.417+0000 7f919fe00640 1 --2- 172.21.15.83:0/1628812109 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5591f9257460 0x5591f9261040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:21.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.417+0000 7f91a7e00640 1 --2- 172.21.15.83:0/1628812109 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5591f9261b30 0x5591f9262f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:21.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.417+0000 7f91a7e00640 1 -- 172.21.15.83:0/1628812109 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5591f9257460 msgr2=0x5591f9261040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:21.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.417+0000 7f91a7e00640 1 --2- 172.21.15.83:0/1628812109 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5591f9257460 0x5591f9261040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:21.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.417+0000 7f91a7e00640 1 -- 172.21.15.83:0/1628812109 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591f925f180 con 0x5591f9256b40 2024-04-24T08:57:21.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.417+0000 7f91ace00640 1 --2- 172.21.15.83:0/1628812109 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5591f9261b30 0x5591f9262f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:57:21.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.417+0000 7f919fe00640 1 --2- 172.21.15.83:0/1628812109 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5591f9257460 0x5591f9261040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:57:21.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.417+0000 7f91a7e00640 1 --2- 172.21.15.83:0/1628812109 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5591f9256b40 0x5591f926d4d0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f919809a500 tx=0x7f91980d8670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5163c16d92d28df server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:21.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.417+0000 7f919cc00640 1 -- 172.21.15.83:0/1628812109 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9198020070 con 0x5591f9256b40 2024-04-24T08:57:21.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.417+0000 7f919cc00640 1 -- 172.21.15.83:0/1628812109 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f91980d8b70 con 0x5591f9256b40 2024-04-24T08:57:21.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.417+0000 7f919cc00640 1 -- 172.21.15.83:0/1628812109 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91980df070 con 0x5591f9256b40 2024-04-24T08:57:21.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.418+0000 7f91b1c5c740 1 -- 172.21.15.83:0/1628812109 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5591f9256b40 msgr2=0x5591f926d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:21.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.418+0000 7f91b1c5c740 1 --2- 172.21.15.83:0/1628812109 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5591f9256b40 0x5591f926d4d0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f919809a500 tx=0x7f91980d8670 comp rx=0 tx=0).stop 2024-04-24T08:57:21.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.418+0000 7f91b1c5c740 1 -- 172.21.15.83:0/1628812109 shutdown_connections 2024-04-24T08:57:21.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.418+0000 7f91b1c5c740 1 --2- 172.21.15.83:0/1628812109 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5591f9261b30 0x5591f9262f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:21.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.418+0000 7f91b1c5c740 1 --2- 172.21.15.83:0/1628812109 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5591f9257460 0x5591f9261040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:21.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.418+0000 7f91b1c5c740 1 --2- 172.21.15.83:0/1628812109 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5591f9256b40 0x5591f926d4d0 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:21.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.418+0000 7f91b1c5c740 1 -- 172.21.15.83:0/1628812109 >> 172.21.15.83:0/1628812109 conn(0x5591f9252e60 msgr2=0x5591f926cdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:21.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.418+0000 7f91b1c5c740 1 -- 172.21.15.83:0/1628812109 shutdown_connections 2024-04-24T08:57:21.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.418+0000 7f91b1c5c740 1 -- 172.21.15.83:0/1628812109 wait complete. 2024-04-24T08:57:21.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.419+0000 7f91b1c5c740 1 -- start start 2024-04-24T08:57:21.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.419+0000 7f91b1c5c740 1 Processor -- start 2024-04-24T08:57:21.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.419+0000 7f91b1c5c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5591f9256b40 0x5591f9277250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:21.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.419+0000 7f91b1c5c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5591f9257460 0x5591f92725c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:21.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.419+0000 7f91b1c5c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5591f9261b30 0x5591f9272b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:21.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.419+0000 7f91b1c5c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f91a81b2990 con 0x5591f9257460 2024-04-24T08:57:21.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.419+0000 7f91b1c5c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f91a81b22c0 con 0x5591f9261b30 2024-04-24T08:57:21.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.419+0000 7f91b1c5c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f91a81af500 con 0x5591f9256b40 2024-04-24T08:57:21.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.419+0000 7f91b1c5c740 10 client.0 _ll_register_callbacks cb 0x7f91a0009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:57:21.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.420+0000 7f919fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5591f9257460 0x5591f92725c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:21.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.420+0000 7f919fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5591f9257460 0x5591f92725c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58944/0 (socket says 172.21.15.83:58944) 2024-04-24T08:57:21.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.420+0000 7f919fe00640 1 -- 172.21.15.83:0/3946306309 learned_addr learned my addr 172.21.15.83:0/3946306309 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:57:21.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.420+0000 7f91a7e00640 1 --2- 172.21.15.83:0/3946306309 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5591f9256b40 0x5591f9277250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:21.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.420+0000 7f919fe00640 1 -- 172.21.15.83:0/3946306309 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5591f9256b40 msgr2=0x5591f9277250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:21.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.420+0000 7f919fe00640 1 --2- 172.21.15.83:0/3946306309 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5591f9256b40 0x5591f9277250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:21.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.420+0000 7f919fe00640 1 -- 172.21.15.83:0/3946306309 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5591f9261b30 msgr2=0x5591f9272b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:21.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.420+0000 7f919fe00640 1 --2- 172.21.15.83:0/3946306309 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5591f9261b30 0x5591f9272b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:21.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.420+0000 7f919fe00640 1 -- 172.21.15.83:0/3946306309 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591f8f987d0 con 0x5591f9257460 2024-04-24T08:57:21.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.420+0000 7f919fe00640 1 --2- 172.21.15.83:0/3946306309 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5591f9257460 0x5591f92725c0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f9190005d20 tx=0x7f919000ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cf51681095e4cb5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:21.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.420+0000 7f9196000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:57:21.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.421+0000 7f9196000640 1 -- 172.21.15.83:0/3946306309 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9190014be0 con 0x5591f9257460 2024-04-24T08:57:21.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.421+0000 7f9196000640 1 -- 172.21.15.83:0/3946306309 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f9190014d80 con 0x5591f9257460 2024-04-24T08:57:21.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.421+0000 7f91b1c5c740 1 -- 172.21.15.83:0/3946306309 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f91980d8b70 con 0x5591f9257460 2024-04-24T08:57:21.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.421+0000 7f9196000640 1 -- 172.21.15.83:0/3946306309 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f9190029050 con 0x5591f9257460 2024-04-24T08:57:21.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.421+0000 7f91b1c5c740 10 client.17685 fetch_fsmap learned FSMap version 33 2024-04-24T08:57:21.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.421+0000 7f91b1c5c740 1 -- 172.21.15.83:0/3946306309 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5591f92d7a60 con 0x5591f9257460 2024-04-24T08:57:21.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.421+0000 7f9196000640 1 -- 172.21.15.83:0/3946306309 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f91980d8b70 con 0x5591f9257460 2024-04-24T08:57:21.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.421+0000 7f9196000640 1 -- 172.21.15.83:0/3946306309 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f919000f040 con 0x5591f9257460 2024-04-24T08:57:21.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.422+0000 7f91b1c5c740 10 client.17685 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T08:57:21.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.422+0000 7f91b1c5c740 10 client.17685 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-24T08:57:21.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.422+0000 7f91b1c5c740 20 client.17685 populate_metadata read hostname 'smithi083' 2024-04-24T08:57:21.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.422+0000 7f91b1c5c740 1 --2- 172.21.15.83:0/3946306309 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x5591f92d8230 0x5591f92da650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:21.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.422+0000 7f91b1c5c740 4 client.17685 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T08:57:21.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.422+0000 7f91b1c5c740 1 -- 172.21.15.83:0/3946306309 --> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5591f91c3a60 con 0x5591f92d8230 2024-04-24T08:57:21.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.422+0000 7f91a7e00640 1 --2- 172.21.15.83:0/3946306309 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x5591f92d8230 0x5591f92da650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:21.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.423+0000 7f91a7e00640 1 --2- 172.21.15.83:0/3946306309 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x5591f92d8230 0x5591f92da650 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=595bbe6546f45c19 server_cookie=877fd19f93398d23 in_seq=0 out_seq=0 2024-04-24T08:57:21.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.423+0000 7f9196000640 10 client.17685 ms_handle_connect on v2:172.21.15.189:6838/2548965336 2024-04-24T08:57:21.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.424+0000 7f9196000640 1 -- 172.21.15.83:0/3946306309 <== mds.4 v2:172.21.15.189:6838/2548965336 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5591f91c3a60 con 0x5591f92d8230 2024-04-24T08:57:21.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.424+0000 7f9196000640 10 client.17685 handle_command_reply: tid=0 2024-04-24T08:57:21.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.424+0000 7f91b1c5c740 1 client.17685 shutdown 2024-04-24T08:57:21.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.424+0000 7f91b1c5c740 1 -- 172.21.15.83:0/3946306309 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5591f9257460 msgr2=0x5591f92725c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:21.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.424+0000 7f91b1c5c740 1 --2- 172.21.15.83:0/3946306309 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5591f9257460 0x5591f92725c0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f9190005d20 tx=0x7f919000ce90 comp rx=0 tx=0).stop 2024-04-24T08:57:21.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.424+0000 7f91b1c5c740 1 -- 172.21.15.83:0/3946306309 shutdown_connections 2024-04-24T08:57:21.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.424+0000 7f91b1c5c740 1 --2- 172.21.15.83:0/3946306309 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x5591f92d8230 0x5591f92da650 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:21.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.424+0000 7f91b1c5c740 1 --2- 172.21.15.83:0/3946306309 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5591f9261b30 0x5591f9272b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:21.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.424+0000 7f91b1c5c740 1 --2- 172.21.15.83:0/3946306309 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5591f9257460 0x5591f92725c0 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:21.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.424+0000 7f91b1c5c740 1 --2- 172.21.15.83:0/3946306309 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5591f9256b40 0x5591f9277250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:21.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.424+0000 7f91b1c5c740 1 -- 172.21.15.83:0/3946306309 >> 172.21.15.83:0/3946306309 conn(0x5591f9252e60 msgr2=0x5591f926d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:21.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.424+0000 7f9196000640 0 client.17685 ms_handle_reset on v2:172.21.15.189:6838/2548965336 2024-04-24T08:57:21.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.424+0000 7f91b1c5c740 1 -- 172.21.15.83:0/3946306309 shutdown_connections 2024-04-24T08:57:21.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.424+0000 7f91b1c5c740 1 -- 172.21.15.83:0/3946306309 wait complete. 2024-04-24T08:57:21.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.424+0000 7f91b1c5c740 20 client.17685 trim_cache size 0 max 16384 2024-04-24T08:57:21.427 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T08:57:21.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.425+0000 7f91ad800640 1 -- 172.21.15.83:0/3351893443 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f918c0a3da0 msgr2=0x7f918c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:21.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.425+0000 7f91ad800640 1 --2- 172.21.15.83:0/3351893443 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f918c0a3da0 0x7f918c0a6260 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f91a0006010 tx=0x7f91a0005d80 comp rx=0 tx=0).stop 2024-04-24T08:57:21.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.425+0000 7f91ad800640 1 -- 172.21.15.83:0/3351893443 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f91a8100c70 msgr2=0x7f91a81a05d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:21.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.425+0000 7f91ad800640 1 --2- 172.21.15.83:0/3351893443 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f91a8100c70 0x7f91a81a05d0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f919800d770 tx=0x7f919800dc30 comp rx=0 tx=0).stop 2024-04-24T08:57:21.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.425+0000 7f91ad800640 1 -- 172.21.15.83:0/3351893443 shutdown_connections 2024-04-24T08:57:21.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.425+0000 7f91ad800640 1 --2- 172.21.15.83:0/3351893443 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f918c0a3da0 0x7f918c0a6260 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:21.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.425+0000 7f91ad800640 1 --2- 172.21.15.83:0/3351893443 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f91a810e820 0x7f91a819a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:21.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.425+0000 7f91ad800640 1 --2- 172.21.15.83:0/3351893443 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91a8101620 0x7f91a81a0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:21.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.425+0000 7f91ad800640 1 --2- 172.21.15.83:0/3351893443 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f91a8100c70 0x7f91a81a05d0 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:21.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.425+0000 7f91ad800640 1 -- 172.21.15.83:0/3351893443 >> 172.21.15.83:0/3351893443 conn(0x7f91a80fc880 msgr2=0x7f91a80fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:21.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.425+0000 7f91ad800640 1 -- 172.21.15.83:0/3351893443 shutdown_connections 2024-04-24T08:57:21.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.425+0000 7f91ad800640 1 -- 172.21.15.83:0/3351893443 wait complete. 2024-04-24T08:57:21.442 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-24T08:57:21.579 INFO:tasks.workunit.client.0.smithi083.stdout: 1 28699 0.66 MB/sec execute 115 sec latency 39.654 ms 2024-04-24T08:57:21.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.631+0000 7f40fca00640 1 -- 172.21.15.83:0/3377587826 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f40f81030c0 msgr2=0x7f40f81034a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:21.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.631+0000 7f40fca00640 1 --2- 172.21.15.83:0/3377587826 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f40f81030c0 0x7f40f81034a0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f40e8009900 tx=0x7f40e802a480 comp rx=0 tx=0).stop 2024-04-24T08:57:21.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.631+0000 7f40fca00640 1 -- 172.21.15.83:0/3377587826 shutdown_connections 2024-04-24T08:57:21.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.631+0000 7f40fca00640 1 --2- 172.21.15.83:0/3377587826 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f40f810c490 0x7f40f810e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:21.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.631+0000 7f40fca00640 1 --2- 172.21.15.83:0/3377587826 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f40f81039e0 0x7f40f810bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:21.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.631+0000 7f40fca00640 1 --2- 172.21.15.83:0/3377587826 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f40f81030c0 0x7f40f81034a0 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:21.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.631+0000 7f40fca00640 1 -- 172.21.15.83:0/3377587826 >> 172.21.15.83:0/3377587826 conn(0x7f40f80fc880 msgr2=0x7f40f80feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:21.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.632+0000 7f40fca00640 1 -- 172.21.15.83:0/3377587826 shutdown_connections 2024-04-24T08:57:21.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.632+0000 7f40fca00640 1 -- 172.21.15.83:0/3377587826 wait complete. 2024-04-24T08:57:21.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.633+0000 7f40fca00640 1 Processor -- start 2024-04-24T08:57:21.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.633+0000 7f40fca00640 1 -- start start 2024-04-24T08:57:21.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.633+0000 7f40fca00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f40f81030c0 0x7f40f81a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:21.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.633+0000 7f40fca00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f40f81039e0 0x7f40f81a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:21.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.633+0000 7f40fca00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f40f810c490 0x7f40f819a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:21.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.633+0000 7f40fca00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f40f81121e0 con 0x7f40f81039e0 2024-04-24T08:57:21.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.633+0000 7f40fca00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f40f8112060 con 0x7f40f81030c0 2024-04-24T08:57:21.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.633+0000 7f40fca00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f40f8112360 con 0x7f40f810c490 2024-04-24T08:57:21.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.634+0000 7f40f6a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f40f81039e0 0x7f40f81a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:21.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.634+0000 7f40f6a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f40f81039e0 0x7f40f81a0ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58954/0 (socket says 172.21.15.83:58954) 2024-04-24T08:57:21.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.634+0000 7f40f6a00640 1 -- 172.21.15.83:0/4285392069 learned_addr learned my addr 172.21.15.83:0/4285392069 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:57:21.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.634+0000 7f40f7e00640 1 --2- 172.21.15.83:0/4285392069 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f40f810c490 0x7f40f819a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:21.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.634+0000 7f40f7400640 1 --2- 172.21.15.83:0/4285392069 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f40f81030c0 0x7f40f81a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:21.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.634+0000 7f40f6a00640 1 -- 172.21.15.83:0/4285392069 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f40f810c490 msgr2=0x7f40f819a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:21.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.634+0000 7f40f6a00640 1 --2- 172.21.15.83:0/4285392069 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f40f810c490 0x7f40f819a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:21.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.634+0000 7f40f6a00640 1 -- 172.21.15.83:0/4285392069 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f40f81030c0 msgr2=0x7f40f81a0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:21.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.634+0000 7f40f6a00640 1 --2- 172.21.15.83:0/4285392069 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f40f81030c0 0x7f40f81a0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:21.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.634+0000 7f40f6a00640 1 -- 172.21.15.83:0/4285392069 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40f819afe0 con 0x7f40f81039e0 2024-04-24T08:57:21.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.634+0000 7f40f6a00640 1 --2- 172.21.15.83:0/4285392069 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f40f81039e0 0x7f40f81a0ba0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f40e000ba50 tx=0x7f40e000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:21.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.635+0000 7f40efe00640 1 -- 172.21.15.83:0/4285392069 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e0019070 con 0x7f40f81039e0 2024-04-24T08:57:21.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.635+0000 7f40efe00640 1 -- 172.21.15.83:0/4285392069 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f40e000cbe0 con 0x7f40f81039e0 2024-04-24T08:57:21.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.635+0000 7f40fca00640 1 -- 172.21.15.83:0/4285392069 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40f819b2d0 con 0x7f40f81039e0 2024-04-24T08:57:21.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.635+0000 7f40efe00640 1 -- 172.21.15.83:0/4285392069 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e00075d0 con 0x7f40f81039e0 2024-04-24T08:57:21.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.635+0000 7f40fca00640 1 -- 172.21.15.83:0/4285392069 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40f81a75e0 con 0x7f40f81039e0 2024-04-24T08:57:21.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.639+0000 7f4100fee740 1 Processor -- start 2024-04-24T08:57:21.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.639+0000 7f4100fee740 1 -- start start 2024-04-24T08:57:21.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.640+0000 7f4100fee740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555edea10b40 0x555edea10f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:21.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.640+0000 7f40efe00640 1 -- 172.21.15.83:0/4285392069 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f40e001c070 con 0x7f40f81039e0 2024-04-24T08:57:21.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.640+0000 7f4100fee740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555edea11460 0x555edea1b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:21.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.640+0000 7f4100fee740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555edea1bb30 0x555edea1df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:21.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.640+0000 7f4100fee740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x555ede9761d0 con 0x555edea1bb30 2024-04-24T08:57:21.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.640+0000 7f4100fee740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555ede91c450 con 0x555edea11460 2024-04-24T08:57:21.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.640+0000 7f4100fee740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x555ede972190 con 0x555edea10b40 2024-04-24T08:57:21.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.640+0000 7f40f6a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555edea1bb30 0x555edea1df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:21.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.640+0000 7f40f6a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555edea1bb30 0x555edea1df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58964/0 (socket says 172.21.15.83:58964) 2024-04-24T08:57:21.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.640+0000 7f40f6a00640 1 -- 172.21.15.83:0/1261920315 learned_addr learned my addr 172.21.15.83:0/1261920315 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:57:21.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.641+0000 7f40f7e00640 1 --2- 172.21.15.83:0/1261920315 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555edea10b40 0x555edea10f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:21.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.641+0000 7f40f7400640 1 --2- 172.21.15.83:0/1261920315 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555edea11460 0x555edea1b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:21.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.641+0000 7f40f6a00640 1 -- 172.21.15.83:0/1261920315 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555edea10b40 msgr2=0x555edea10f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:21.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.641+0000 7f40f6a00640 1 --2- 172.21.15.83:0/1261920315 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555edea10b40 0x555edea10f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:21.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.641+0000 7f40f6a00640 1 -- 172.21.15.83:0/1261920315 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555edea11460 msgr2=0x555edea1b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:21.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.641+0000 7f40f6a00640 1 --2- 172.21.15.83:0/1261920315 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555edea11460 0x555edea1b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:21.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.641+0000 7f40f6a00640 1 -- 172.21.15.83:0/1261920315 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ede9d9310 con 0x555edea1bb30 2024-04-24T08:57:21.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.641+0000 7f40f6a00640 1 --2- 172.21.15.83:0/1261920315 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555edea1bb30 0x555edea1df20 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f40e00cd4c0 tx=0x7f40e00d8270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adcd50b8cdaabff2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:21.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.641+0000 7f40e7e00640 1 -- 172.21.15.83:0/1261920315 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e00e3070 con 0x555edea1bb30 2024-04-24T08:57:21.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.642+0000 7f40e7e00640 1 -- 172.21.15.83:0/1261920315 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f40e00d0570 con 0x555edea1bb30 2024-04-24T08:57:21.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.642+0000 7f40e7e00640 1 -- 172.21.15.83:0/1261920315 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e00de620 con 0x555edea1bb30 2024-04-24T08:57:21.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.642+0000 7f40efe00640 1 --2- 172.21.15.83:0/4285392069 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f40d80a3da0 0x7f40d80a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:21.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.642+0000 7f40efe00640 1 -- 172.21.15.83:0/4285392069 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f40e00d00c0 con 0x7f40f81039e0 2024-04-24T08:57:21.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.642+0000 7f40f7e00640 1 --2- 172.21.15.83:0/4285392069 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f40d80a3da0 0x7f40d80a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:21.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.642+0000 7f4100fee740 1 -- 172.21.15.83:0/1261920315 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555edea1bb30 msgr2=0x555edea1df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:21.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.642+0000 7f4100fee740 1 --2- 172.21.15.83:0/1261920315 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555edea1bb30 0x555edea1df20 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f40e00cd4c0 tx=0x7f40e00d8270 comp rx=0 tx=0).stop 2024-04-24T08:57:21.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.642+0000 7f4100fee740 1 -- 172.21.15.83:0/1261920315 shutdown_connections 2024-04-24T08:57:21.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.643+0000 7f4100fee740 1 --2- 172.21.15.83:0/1261920315 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555edea1bb30 0x555edea1df20 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:21.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.643+0000 7f4100fee740 1 --2- 172.21.15.83:0/1261920315 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555edea11460 0x555edea1b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:21.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.643+0000 7f4100fee740 1 --2- 172.21.15.83:0/1261920315 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555edea10b40 0x555edea10f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:21.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.643+0000 7f4100fee740 1 -- 172.21.15.83:0/1261920315 >> 172.21.15.83:0/1261920315 conn(0x555edea0ce60 msgr2=0x555edea0d240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:21.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.643+0000 7f4100fee740 1 -- 172.21.15.83:0/1261920315 shutdown_connections 2024-04-24T08:57:21.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.643+0000 7f4100fee740 1 -- 172.21.15.83:0/1261920315 wait complete. 2024-04-24T08:57:21.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.643+0000 7f40f7e00640 1 --2- 172.21.15.83:0/4285392069 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f40d80a3da0 0x7f40d80a6260 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f40f0005fd0 tx=0x7f40f000b5b0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:21.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.643+0000 7f4100fee740 1 -- start start 2024-04-24T08:57:21.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.643+0000 7f4100fee740 1 Processor -- start 2024-04-24T08:57:21.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.644+0000 7f4100fee740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555edea10b40 0x555edea1b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:21.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.644+0000 7f4100fee740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555edea11460 0x555edea188b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:21.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.644+0000 7f4100fee740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555edea1bb30 0x555edea18f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:21.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.644+0000 7f4100fee740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x555edea1e9c0 con 0x555edea1bb30 2024-04-24T08:57:21.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.644+0000 7f4100fee740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555ede92f810 con 0x555edea10b40 2024-04-24T08:57:21.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.644+0000 7f4100fee740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x555edea1eb40 con 0x555edea11460 2024-04-24T08:57:21.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.644+0000 7f4100fee740 10 client.0 _ll_register_callbacks cb 0x555ede8fd140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:57:21.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.644+0000 7f40f7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555edea1bb30 0x555edea18f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:21.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.644+0000 7f40f7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555edea1bb30 0x555edea18f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58980/0 (socket says 172.21.15.83:58980) 2024-04-24T08:57:21.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.644+0000 7f40f7e00640 1 -- 172.21.15.83:0/3482269963 learned_addr learned my addr 172.21.15.83:0/3482269963 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:57:21.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.644+0000 7f40f7400640 1 --2- 172.21.15.83:0/3482269963 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555edea10b40 0x555edea1b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:21.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.645+0000 7f40f6a00640 1 --2- 172.21.15.83:0/3482269963 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555edea11460 0x555edea188b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:21.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.645+0000 7f40f7e00640 1 -- 172.21.15.83:0/3482269963 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555edea11460 msgr2=0x555edea188b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:21.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.645+0000 7f40f7e00640 1 --2- 172.21.15.83:0/3482269963 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555edea11460 0x555edea188b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:21.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.645+0000 7f40f7e00640 1 -- 172.21.15.83:0/3482269963 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555edea10b40 msgr2=0x555edea1b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:21.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.645+0000 7f40f7e00640 1 --2- 172.21.15.83:0/3482269963 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555edea10b40 0x555edea1b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:21.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.645+0000 7f40f7e00640 1 -- 172.21.15.83:0/3482269963 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ede9403f0 con 0x555edea1bb30 2024-04-24T08:57:21.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.645+0000 7f40f7e00640 1 --2- 172.21.15.83:0/3482269963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555edea1bb30 0x555edea18f30 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f40f0012700 tx=0x7f40f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a0bb36a8432bf3f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:21.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.645+0000 7f40e4c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:57:21.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.645+0000 7f40e4c00640 1 -- 172.21.15.83:0/3482269963 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f0005220 con 0x555edea1bb30 2024-04-24T08:57:21.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.645+0000 7f40e4c00640 1 -- 172.21.15.83:0/3482269963 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f40f00053c0 con 0x555edea1bb30 2024-04-24T08:57:21.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.645+0000 7f4100fee740 1 -- 172.21.15.83:0/3482269963 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40e00d0570 con 0x555edea1bb30 2024-04-24T08:57:21.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.646+0000 7f40e4c00640 1 -- 172.21.15.83:0/3482269963 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f40f002f050 con 0x555edea1bb30 2024-04-24T08:57:21.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.646+0000 7f4100fee740 10 client.17697 fetch_fsmap learned FSMap version 33 2024-04-24T08:57:21.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.646+0000 7f4100fee740 1 -- 172.21.15.83:0/3482269963 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x555ede6883f0 con 0x555edea1bb30 2024-04-24T08:57:21.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.646+0000 7f40e4c00640 1 -- 172.21.15.83:0/3482269963 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f40e00d0570 con 0x555edea1bb30 2024-04-24T08:57:21.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.646+0000 7f40e4c00640 1 -- 172.21.15.83:0/3482269963 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f40f001fa50 con 0x555edea1bb30 2024-04-24T08:57:21.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.646+0000 7f4100fee740 10 client.17697 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T08:57:21.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.646+0000 7f4100fee740 10 client.17697 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-24T08:57:21.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.646+0000 7f4100fee740 20 client.17697 populate_metadata read hostname 'smithi083' 2024-04-24T08:57:21.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.646+0000 7f4100fee740 1 --2- 172.21.15.83:0/3482269963 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x555edea2bd40 0x555edea2e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:21.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.647+0000 7f4100fee740 4 client.17697 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:57:21.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.647+0000 7f4100fee740 1 -- 172.21.15.83:0/3482269963 --> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555ede983bf0 con 0x555edea2bd40 2024-04-24T08:57:21.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.647+0000 7f40f7400640 1 --2- 172.21.15.83:0/3482269963 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x555edea2bd40 0x555edea2e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:21.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.648+0000 7f40f7400640 1 --2- 172.21.15.83:0/3482269963 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x555edea2bd40 0x555edea2e1e0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4043947b28852512 server_cookie=5bc9699f9946cd00 in_seq=0 out_seq=0 2024-04-24T08:57:21.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.648+0000 7f40e4c00640 10 client.17697 ms_handle_connect on v2:172.21.15.138:6832/1157967658 2024-04-24T08:57:21.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.650+0000 7f40e4c00640 1 -- 172.21.15.83:0/3482269963 <== mds.3 v2:172.21.15.138:6832/1157967658 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555ede983bf0 con 0x555edea2bd40 2024-04-24T08:57:21.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.650+0000 7f40e4c00640 10 client.17697 handle_command_reply: tid=0 2024-04-24T08:57:21.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.650+0000 7f4100fee740 1 client.17697 shutdown 2024-04-24T08:57:21.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.651+0000 7f4100fee740 1 -- 172.21.15.83:0/3482269963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555edea1bb30 msgr2=0x555edea18f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:21.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.651+0000 7f4100fee740 1 --2- 172.21.15.83:0/3482269963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555edea1bb30 0x555edea18f30 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f40f0012700 tx=0x7f40f00046c0 comp rx=0 tx=0).stop 2024-04-24T08:57:21.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.651+0000 7f4100fee740 1 -- 172.21.15.83:0/3482269963 shutdown_connections 2024-04-24T08:57:21.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.651+0000 7f4100fee740 1 --2- 172.21.15.83:0/3482269963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555edea1bb30 0x555edea18f30 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:21.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.651+0000 7f4100fee740 1 --2- 172.21.15.83:0/3482269963 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x555edea2bd40 0x555edea2e1e0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:21.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.651+0000 7f4100fee740 1 --2- 172.21.15.83:0/3482269963 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555edea11460 0x555edea188b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:21.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.651+0000 7f4100fee740 1 --2- 172.21.15.83:0/3482269963 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555edea10b40 0x555edea1b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:21.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.651+0000 7f4100fee740 1 -- 172.21.15.83:0/3482269963 >> 172.21.15.83:0/3482269963 conn(0x555edea0ce60 msgr2=0x555edea12c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:21.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.651+0000 7f40e4c00640 0 client.17697 ms_handle_reset on v2:172.21.15.138:6832/1157967658 2024-04-24T08:57:21.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.651+0000 7f4100fee740 1 -- 172.21.15.83:0/3482269963 shutdown_connections 2024-04-24T08:57:21.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.651+0000 7f4100fee740 1 -- 172.21.15.83:0/3482269963 wait complete. 2024-04-24T08:57:21.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.651+0000 7f4100fee740 20 client.17697 trim_cache size 0 max 16384 2024-04-24T08:57:21.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.666+0000 7f4100fee740 1 Processor -- start 2024-04-24T08:57:21.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.666+0000 7f4100fee740 1 -- start start 2024-04-24T08:57:21.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.667+0000 7f4100fee740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555edea10b40 0x555edea27880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:21.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.667+0000 7f4100fee740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555edea11460 0x555edea181b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:21.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.667+0000 7f4100fee740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555edea1bb30 0x555edea07de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:21.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.667+0000 7f4100fee740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f40f81b1530 con 0x555edea11460 2024-04-24T08:57:21.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.667+0000 7f4100fee740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f40f81b3070 con 0x555edea1bb30 2024-04-24T08:57:21.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.667+0000 7f4100fee740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f40f81b1c00 con 0x555edea10b40 2024-04-24T08:57:21.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.667+0000 7f40f6a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555edea11460 0x555edea181b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:21.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.667+0000 7f40f6a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555edea11460 0x555edea181b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58982/0 (socket says 172.21.15.83:58982) 2024-04-24T08:57:21.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.667+0000 7f40f6a00640 1 -- 172.21.15.83:0/682596703 learned_addr learned my addr 172.21.15.83:0/682596703 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:57:21.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.667+0000 7f40f6a00640 1 -- 172.21.15.83:0/682596703 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555edea10b40 msgr2=0x555edea27880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:21.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.667+0000 7f40f6a00640 1 --2- 172.21.15.83:0/682596703 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555edea10b40 0x555edea27880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:21.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.667+0000 7f40f6a00640 1 -- 172.21.15.83:0/682596703 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555edea1bb30 msgr2=0x555edea07de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:21.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.667+0000 7f40f6a00640 1 --2- 172.21.15.83:0/682596703 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555edea1bb30 0x555edea07de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:21.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.667+0000 7f40f6a00640 1 -- 172.21.15.83:0/682596703 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ede7527d0 con 0x555edea11460 2024-04-24T08:57:21.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.668+0000 7f40f6a00640 1 --2- 172.21.15.83:0/682596703 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555edea11460 0x555edea181b0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f40e0098770 tx=0x7f40e00d03c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15a43e01cd354a6e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:21.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.668+0000 7f40e7400640 1 -- 172.21.15.83:0/682596703 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e009bb70 con 0x555edea11460 2024-04-24T08:57:21.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.668+0000 7f40e7400640 1 -- 172.21.15.83:0/682596703 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f40e009c4a0 con 0x555edea11460 2024-04-24T08:57:21.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.668+0000 7f40e7400640 1 -- 172.21.15.83:0/682596703 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e009b700 con 0x555edea11460 2024-04-24T08:57:21.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.669+0000 7f4100fee740 1 -- 172.21.15.83:0/682596703 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555edea11460 msgr2=0x555edea181b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:21.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.669+0000 7f4100fee740 1 --2- 172.21.15.83:0/682596703 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555edea11460 0x555edea181b0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f40e0098770 tx=0x7f40e00d03c0 comp rx=0 tx=0).stop 2024-04-24T08:57:21.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.669+0000 7f4100fee740 1 -- 172.21.15.83:0/682596703 shutdown_connections 2024-04-24T08:57:21.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.669+0000 7f4100fee740 1 --2- 172.21.15.83:0/682596703 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555edea1bb30 0x555edea07de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:21.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.669+0000 7f4100fee740 1 --2- 172.21.15.83:0/682596703 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555edea11460 0x555edea181b0 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:21.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.669+0000 7f4100fee740 1 --2- 172.21.15.83:0/682596703 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555edea10b40 0x555edea27880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:21.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.669+0000 7f4100fee740 1 -- 172.21.15.83:0/682596703 >> 172.21.15.83:0/682596703 conn(0x555edea0ce60 msgr2=0x555edea1a970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:21.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.669+0000 7f4100fee740 1 -- 172.21.15.83:0/682596703 shutdown_connections 2024-04-24T08:57:21.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.669+0000 7f4100fee740 1 -- 172.21.15.83:0/682596703 wait complete. 2024-04-24T08:57:21.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.669+0000 7f4100fee740 1 -- start start 2024-04-24T08:57:21.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.669+0000 7f4100fee740 1 Processor -- start 2024-04-24T08:57:21.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.670+0000 7f4100fee740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555edea10b40 0x555edea23f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:21.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.670+0000 7f4100fee740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555edea11460 0x555edea14b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:21.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.670+0000 7f4100fee740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555edea1bb30 0x555edea15080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:21.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.670+0000 7f4100fee740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f40f81b29a0 con 0x555edea11460 2024-04-24T08:57:21.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.670+0000 7f4100fee740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f40f81b22d0 con 0x555edea10b40 2024-04-24T08:57:21.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.670+0000 7f4100fee740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f40f81af510 con 0x555edea1bb30 2024-04-24T08:57:21.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.670+0000 7f4100fee740 10 client.0 _ll_register_callbacks cb 0x7f40f000b3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:57:21.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.670+0000 7f40f6a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555edea11460 0x555edea14b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:21.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.670+0000 7f40f6a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555edea11460 0x555edea14b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58994/0 (socket says 172.21.15.83:58994) 2024-04-24T08:57:21.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.670+0000 7f40f6a00640 1 -- 172.21.15.83:0/1979025998 learned_addr learned my addr 172.21.15.83:0/1979025998 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:57:21.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.671+0000 7f40f7e00640 1 --2- 172.21.15.83:0/1979025998 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555edea1bb30 0x555edea15080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:21.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.671+0000 7f40f6a00640 1 -- 172.21.15.83:0/1979025998 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555edea1bb30 msgr2=0x555edea15080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:21.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.671+0000 7f40f6a00640 1 --2- 172.21.15.83:0/1979025998 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555edea1bb30 0x555edea15080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:21.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.671+0000 7f40f6a00640 1 -- 172.21.15.83:0/1979025998 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555edea10b40 msgr2=0x555edea23f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:21.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.671+0000 7f40f6a00640 1 --2- 172.21.15.83:0/1979025998 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555edea10b40 0x555edea23f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:21.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.671+0000 7f40f6a00640 1 -- 172.21.15.83:0/1979025998 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555edea155f0 con 0x555edea11460 2024-04-24T08:57:21.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.671+0000 7f40f6a00640 1 --2- 172.21.15.83:0/1979025998 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555edea11460 0x555edea14b40 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f40e0004a80 tx=0x7f40e00a4b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d113027329e8e93 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:21.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.671+0000 7f40e4c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:57:21.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.671+0000 7f40e4c00640 1 -- 172.21.15.83:0/1979025998 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e009b8e0 con 0x555edea11460 2024-04-24T08:57:21.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.671+0000 7f40e4c00640 1 -- 172.21.15.83:0/1979025998 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f40e0098410 con 0x555edea11460 2024-04-24T08:57:21.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.671+0000 7f4100fee740 1 -- 172.21.15.83:0/1979025998 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40e009c4a0 con 0x555edea11460 2024-04-24T08:57:21.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.671+0000 7f40e4c00640 1 -- 172.21.15.83:0/1979025998 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f40e009bb70 con 0x555edea11460 2024-04-24T08:57:21.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.671+0000 7f4100fee740 10 client.17703 fetch_fsmap learned FSMap version 33 2024-04-24T08:57:21.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.671+0000 7f4100fee740 1 -- 172.21.15.83:0/1979025998 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x555edea168f0 con 0x555edea11460 2024-04-24T08:57:21.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.672+0000 7f40e4c00640 1 -- 172.21.15.83:0/1979025998 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f40e009c4a0 con 0x555edea11460 2024-04-24T08:57:21.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.672+0000 7f40e4c00640 1 -- 172.21.15.83:0/1979025998 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f40e00cd630 con 0x555edea11460 2024-04-24T08:57:21.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.672+0000 7f4100fee740 10 client.17703 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T08:57:21.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.672+0000 7f4100fee740 10 client.17703 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-24T08:57:21.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.672+0000 7f4100fee740 20 client.17703 populate_metadata read hostname 'smithi083' 2024-04-24T08:57:21.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.672+0000 7f4100fee740 1 --2- 172.21.15.83:0/1979025998 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x555edea9bb40 0x555edea9df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:21.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.672+0000 7f4100fee740 4 client.17703 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T08:57:21.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.672+0000 7f4100fee740 1 -- 172.21.15.83:0/1979025998 --> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555ede97da60 con 0x555edea9bb40 2024-04-24T08:57:21.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.673+0000 7f40f7400640 1 --2- 172.21.15.83:0/1979025998 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x555edea9bb40 0x555edea9df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:21.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.674+0000 7f40f7400640 1 --2- 172.21.15.83:0/1979025998 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x555edea9bb40 0x555edea9df30 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=16f6c0b867f2780a server_cookie=28dc2771f7d36a2a in_seq=0 out_seq=0 2024-04-24T08:57:21.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.674+0000 7f40e4c00640 10 client.17703 ms_handle_connect on v2:172.21.15.138:6832/1157967658 2024-04-24T08:57:21.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.674+0000 7f40e4c00640 1 -- 172.21.15.83:0/1979025998 <== mds.3 v2:172.21.15.138:6832/1157967658 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555ede97da60 con 0x555edea9bb40 2024-04-24T08:57:21.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.674+0000 7f40e4c00640 10 client.17703 handle_command_reply: tid=0 2024-04-24T08:57:21.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.674+0000 7f4100fee740 1 client.17703 shutdown 2024-04-24T08:57:21.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.674+0000 7f4100fee740 1 -- 172.21.15.83:0/1979025998 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555edea11460 msgr2=0x555edea14b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:21.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.674+0000 7f4100fee740 1 --2- 172.21.15.83:0/1979025998 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555edea11460 0x555edea14b40 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f40e0004a80 tx=0x7f40e00a4b20 comp rx=0 tx=0).stop 2024-04-24T08:57:21.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.674+0000 7f4100fee740 1 -- 172.21.15.83:0/1979025998 shutdown_connections 2024-04-24T08:57:21.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.674+0000 7f4100fee740 1 --2- 172.21.15.83:0/1979025998 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x555edea9bb40 0x555edea9df30 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:21.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.674+0000 7f4100fee740 1 --2- 172.21.15.83:0/1979025998 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555edea1bb30 0x555edea15080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:21.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.674+0000 7f40e4c00640 0 client.17703 ms_handle_reset on v2:172.21.15.138:6832/1157967658 2024-04-24T08:57:21.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.674+0000 7f4100fee740 1 --2- 172.21.15.83:0/1979025998 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555edea11460 0x555edea14b40 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:21.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.674+0000 7f4100fee740 1 --2- 172.21.15.83:0/1979025998 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555edea10b40 0x555edea23f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:21.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.675+0000 7f4100fee740 1 -- 172.21.15.83:0/1979025998 >> 172.21.15.83:0/1979025998 conn(0x555edea0ce60 msgr2=0x555edea0b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:21.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.675+0000 7f4100fee740 1 -- 172.21.15.83:0/1979025998 shutdown_connections 2024-04-24T08:57:21.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.675+0000 7f4100fee740 1 -- 172.21.15.83:0/1979025998 wait complete. 2024-04-24T08:57:21.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.675+0000 7f4100fee740 20 client.17703 trim_cache size 0 max 16384 2024-04-24T08:57:21.678 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T08:57:21.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.675+0000 7f40fca00640 1 -- 172.21.15.83:0/4285392069 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f40d80a3da0 msgr2=0x7f40d80a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:21.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.675+0000 7f40fca00640 1 --2- 172.21.15.83:0/4285392069 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f40d80a3da0 0x7f40d80a6260 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f40f0005fd0 tx=0x7f40f000b5b0 comp rx=0 tx=0).stop 2024-04-24T08:57:21.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.675+0000 7f40fca00640 1 -- 172.21.15.83:0/4285392069 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f40f81039e0 msgr2=0x7f40f81a0ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:21.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.675+0000 7f40fca00640 1 --2- 172.21.15.83:0/4285392069 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f40f81039e0 0x7f40f81a0ba0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f40e000ba50 tx=0x7f40e000bf10 comp rx=0 tx=0).stop 2024-04-24T08:57:21.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.676+0000 7f40fca00640 1 -- 172.21.15.83:0/4285392069 shutdown_connections 2024-04-24T08:57:21.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.676+0000 7f40fca00640 1 --2- 172.21.15.83:0/4285392069 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f40d80a3da0 0x7f40d80a6260 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:21.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.676+0000 7f40fca00640 1 --2- 172.21.15.83:0/4285392069 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f40f810c490 0x7f40f819a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:21.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.676+0000 7f40fca00640 1 --2- 172.21.15.83:0/4285392069 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f40f81039e0 0x7f40f81a0ba0 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:21.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.676+0000 7f40fca00640 1 --2- 172.21.15.83:0/4285392069 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f40f81030c0 0x7f40f81a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:21.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.676+0000 7f40fca00640 1 -- 172.21.15.83:0/4285392069 >> 172.21.15.83:0/4285392069 conn(0x7f40f80fc880 msgr2=0x7f40f8100c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:21.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.676+0000 7f40fca00640 1 -- 172.21.15.83:0/4285392069 shutdown_connections 2024-04-24T08:57:21.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.676+0000 7f40fca00640 1 -- 172.21.15.83:0/4285392069 wait complete. 2024-04-24T08:57:21.692 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T08:57:21.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.902+0000 7f5d98a00640 1 -- 172.21.15.83:0/2556920723 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d94103280 msgr2=0x7f5d94103660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:21.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.902+0000 7f5d98a00640 1 --2- 172.21.15.83:0/2556920723 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d94103280 0x7f5d94103660 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f5d84009900 tx=0x7f5d8402a510 comp rx=0 tx=0).stop 2024-04-24T08:57:21.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.903+0000 7f5d98a00640 1 -- 172.21.15.83:0/2556920723 shutdown_connections 2024-04-24T08:57:21.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.903+0000 7f5d98a00640 1 --2- 172.21.15.83:0/2556920723 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5d9410c5c0 0x7f5d9410ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:21.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.903+0000 7f5d98a00640 1 --2- 172.21.15.83:0/2556920723 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d94103ba0 0x7f5d9410c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:21.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.903+0000 7f5d98a00640 1 --2- 172.21.15.83:0/2556920723 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d94103280 0x7f5d94103660 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:21.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.903+0000 7f5d98a00640 1 -- 172.21.15.83:0/2556920723 >> 172.21.15.83:0/2556920723 conn(0x7f5d94078000 msgr2=0x7f5d94078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:21.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.903+0000 7f5d98a00640 1 -- 172.21.15.83:0/2556920723 shutdown_connections 2024-04-24T08:57:21.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.903+0000 7f5d98a00640 1 -- 172.21.15.83:0/2556920723 wait complete. 2024-04-24T08:57:21.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.904+0000 7f5d98a00640 1 Processor -- start 2024-04-24T08:57:21.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.904+0000 7f5d98a00640 1 -- start start 2024-04-24T08:57:21.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.904+0000 7f5d98a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5d94103280 0x7f5d94100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:21.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.904+0000 7f5d98a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d94103ba0 0x7f5d94101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:21.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.904+0000 7f5d98a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d9410c5c0 0x7f5d94101950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:21.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.904+0000 7f5d98a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5d941122a0 con 0x7f5d9410c5c0 2024-04-24T08:57:21.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.904+0000 7f5d98a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5d94112120 con 0x7f5d94103ba0 2024-04-24T08:57:21.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.904+0000 7f5d98a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5d94112420 con 0x7f5d94103280 2024-04-24T08:57:21.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.905+0000 7f5d92800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d9410c5c0 0x7f5d94101950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:21.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.905+0000 7f5d92800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d9410c5c0 0x7f5d94101950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:59010/0 (socket says 172.21.15.83:59010) 2024-04-24T08:57:21.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.905+0000 7f5d92800640 1 -- 172.21.15.83:0/3093095984 learned_addr learned my addr 172.21.15.83:0/3093095984 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:57:21.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.905+0000 7f5d91e00640 1 --2- 172.21.15.83:0/3093095984 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5d94103280 0x7f5d94100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:21.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.905+0000 7f5d91400640 1 --2- 172.21.15.83:0/3093095984 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d94103ba0 0x7f5d94101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:21.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.905+0000 7f5d92800640 1 -- 172.21.15.83:0/3093095984 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5d94103280 msgr2=0x7f5d94100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:21.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.905+0000 7f5d92800640 1 --2- 172.21.15.83:0/3093095984 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5d94103280 0x7f5d94100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:21.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.905+0000 7f5d92800640 1 -- 172.21.15.83:0/3093095984 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d94103ba0 msgr2=0x7f5d94101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:21.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.905+0000 7f5d92800640 1 --2- 172.21.15.83:0/3093095984 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d94103ba0 0x7f5d94101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:21.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.905+0000 7f5d92800640 1 -- 172.21.15.83:0/3093095984 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d941a6ca0 con 0x7f5d9410c5c0 2024-04-24T08:57:21.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.906+0000 7f5d92800640 1 --2- 172.21.15.83:0/3093095984 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d9410c5c0 0x7f5d94101950 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c00eaa0 tx=0x7f5d8c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:21.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.906+0000 7f5d8aa00640 1 -- 172.21.15.83:0/3093095984 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d8c00cdb0 con 0x7f5d9410c5c0 2024-04-24T08:57:21.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.906+0000 7f5d8aa00640 1 -- 172.21.15.83:0/3093095984 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5d8c005020 con 0x7f5d9410c5c0 2024-04-24T08:57:21.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.906+0000 7f5d98a00640 1 -- 172.21.15.83:0/3093095984 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d941a6f90 con 0x7f5d9410c5c0 2024-04-24T08:57:21.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.906+0000 7f5d8aa00640 1 -- 172.21.15.83:0/3093095984 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d8c0116d0 con 0x7f5d9410c5c0 2024-04-24T08:57:21.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.906+0000 7f5d98a00640 1 -- 172.21.15.83:0/3093095984 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d941a74b0 con 0x7f5d9410c5c0 2024-04-24T08:57:21.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.910+0000 7f5d8aa00640 1 -- 172.21.15.83:0/3093095984 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5d8c0051d0 con 0x7f5d9410c5c0 2024-04-24T08:57:21.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.910+0000 7f5d9b831740 1 Processor -- start 2024-04-24T08:57:21.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.910+0000 7f5d9b831740 1 -- start start 2024-04-24T08:57:21.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.911+0000 7f5d9b831740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fd2a563950 0x55fd2a563d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:21.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.911+0000 7f5d9b831740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd2a564270 0x55fd2a56e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:21.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.911+0000 7f5d9b831740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd2a56e940 0x55fd2a570d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:21.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.911+0000 7f5d9b831740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55fd2a4c91d0 con 0x55fd2a564270 2024-04-24T08:57:21.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.911+0000 7f5d9b831740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55fd2a46f450 con 0x55fd2a563950 2024-04-24T08:57:21.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.911+0000 7f5d9b831740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55fd2a4c5190 con 0x55fd2a56e940 2024-04-24T08:57:21.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.911+0000 7f5d91e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd2a564270 0x55fd2a56e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:21.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.911+0000 7f5d91e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd2a564270 0x55fd2a56e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:59022/0 (socket says 172.21.15.83:59022) 2024-04-24T08:57:21.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.911+0000 7f5d91e00640 1 -- 172.21.15.83:0/3048612787 learned_addr learned my addr 172.21.15.83:0/3048612787 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:57:21.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.911+0000 7f5d8aa00640 1 --2- 172.21.15.83:0/3093095984 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f5d780a3d50 0x7f5d780a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:21.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.911+0000 7f5d91400640 1 --2- 172.21.15.83:0/3048612787 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fd2a563950 0x55fd2a563d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:21.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.912+0000 7f5d8aa00640 1 -- 172.21.15.83:0/3093095984 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f5d8c0d0d70 con 0x7f5d9410c5c0 2024-04-24T08:57:21.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.912+0000 7f5d91400640 1 --2- 172.21.15.83:0/3048612787 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd2a56e940 0x55fd2a570d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:21.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.912+0000 7f5d91e00640 1 -- 172.21.15.83:0/3048612787 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd2a56e940 msgr2=0x55fd2a570d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:21.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.912+0000 7f5d91e00640 1 --2- 172.21.15.83:0/3048612787 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd2a56e940 0x55fd2a570d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:21.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.912+0000 7f5d91e00640 1 -- 172.21.15.83:0/3048612787 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fd2a563950 msgr2=0x55fd2a563d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:21.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.912+0000 7f5d91e00640 1 --2- 172.21.15.83:0/3048612787 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fd2a563950 0x55fd2a563d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:21.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.912+0000 7f5d91e00640 1 -- 172.21.15.83:0/3048612787 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd2a52c310 con 0x55fd2a564270 2024-04-24T08:57:21.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.912+0000 7f5d91400640 1 --2- 172.21.15.83:0/3048612787 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fd2a563950 0x55fd2a563d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:57:21.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.912+0000 7f5d91e00640 1 --2- 172.21.15.83:0/3048612787 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd2a564270 0x55fd2a56e400 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f5d84009900 tx=0x7f5d84004170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=955f613dfc573aaf server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:21.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.912+0000 7f5d92800640 1 --2- 172.21.15.83:0/3093095984 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f5d780a3d50 0x7f5d780a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:21.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.913+0000 7f5d92800640 1 --2- 172.21.15.83:0/3093095984 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f5d780a3d50 0x7f5d780a6210 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c099310 tx=0x7f5d8c00cf50 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:21.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.915+0000 7f5d7f000640 1 -- 172.21.15.83:0/3048612787 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d84035030 con 0x55fd2a564270 2024-04-24T08:57:21.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.915+0000 7f5d7f000640 1 -- 172.21.15.83:0/3048612787 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5d840054f0 con 0x55fd2a564270 2024-04-24T08:57:21.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.915+0000 7f5d7f000640 1 -- 172.21.15.83:0/3048612787 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d840035f0 con 0x55fd2a564270 2024-04-24T08:57:21.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.916+0000 7f5d9b831740 1 -- 172.21.15.83:0/3048612787 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd2a564270 msgr2=0x55fd2a56e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:21.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.916+0000 7f5d9b831740 1 --2- 172.21.15.83:0/3048612787 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd2a564270 0x55fd2a56e400 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f5d84009900 tx=0x7f5d84004170 comp rx=0 tx=0).stop 2024-04-24T08:57:21.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.916+0000 7f5d9b831740 1 -- 172.21.15.83:0/3048612787 shutdown_connections 2024-04-24T08:57:21.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.916+0000 7f5d9b831740 1 --2- 172.21.15.83:0/3048612787 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd2a56e940 0x55fd2a570d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:21.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.916+0000 7f5d9b831740 1 --2- 172.21.15.83:0/3048612787 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd2a564270 0x55fd2a56e400 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:21.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.916+0000 7f5d9b831740 1 --2- 172.21.15.83:0/3048612787 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fd2a563950 0x55fd2a563d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:21.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.916+0000 7f5d9b831740 1 -- 172.21.15.83:0/3048612787 >> 172.21.15.83:0/3048612787 conn(0x55fd2a55fc70 msgr2=0x55fd2a560050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:21.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.916+0000 7f5d9b831740 1 -- 172.21.15.83:0/3048612787 shutdown_connections 2024-04-24T08:57:21.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.916+0000 7f5d9b831740 1 -- 172.21.15.83:0/3048612787 wait complete. 2024-04-24T08:57:21.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.917+0000 7f5d9b831740 1 -- start start 2024-04-24T08:57:21.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.917+0000 7f5d9b831740 1 Processor -- start 2024-04-24T08:57:21.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.917+0000 7f5d9b831740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fd2a563950 0x55fd2a56e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:21.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.917+0000 7f5d9b831740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd2a564270 0x55fd2a56b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:21.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.917+0000 7f5d9b831740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd2a56e940 0x55fd2a56bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:21.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.917+0000 7f5d9b831740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55fd2a571770 con 0x55fd2a564270 2024-04-24T08:57:21.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.917+0000 7f5d9b831740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55fd2a482810 con 0x55fd2a563950 2024-04-24T08:57:21.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.917+0000 7f5d9b831740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55fd2a5718f0 con 0x55fd2a56e940 2024-04-24T08:57:21.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.918+0000 7f5d9b831740 10 client.0 _ll_register_callbacks cb 0x55fd2a450140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:57:21.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.918+0000 7f5d91e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd2a564270 0x55fd2a56b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:21.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.918+0000 7f5d91e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd2a564270 0x55fd2a56b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:59036/0 (socket says 172.21.15.83:59036) 2024-04-24T08:57:21.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.918+0000 7f5d91e00640 1 -- 172.21.15.83:0/3988681712 learned_addr learned my addr 172.21.15.83:0/3988681712 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:57:21.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.918+0000 7f5d91400640 1 --2- 172.21.15.83:0/3988681712 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fd2a563950 0x55fd2a56e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:21.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.918+0000 7f5d91400640 1 --2- 172.21.15.83:0/3988681712 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd2a56e940 0x55fd2a56bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:21.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.918+0000 7f5d91e00640 1 -- 172.21.15.83:0/3988681712 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd2a56e940 msgr2=0x55fd2a56bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:21.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.918+0000 7f5d91e00640 1 --2- 172.21.15.83:0/3988681712 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd2a56e940 0x55fd2a56bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:21.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.918+0000 7f5d91e00640 1 -- 172.21.15.83:0/3988681712 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fd2a563950 msgr2=0x55fd2a56e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:21.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.918+0000 7f5d91e00640 1 --2- 172.21.15.83:0/3988681712 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fd2a563950 0x55fd2a56e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:21.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.918+0000 7f5d91e00640 1 -- 172.21.15.83:0/3988681712 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd2a4933f0 con 0x55fd2a564270 2024-04-24T08:57:21.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.918+0000 7f5d91400640 1 --2- 172.21.15.83:0/3988681712 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fd2a563950 0x55fd2a56e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:57:21.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.919+0000 7f5d91e00640 1 --2- 172.21.15.83:0/3988681712 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd2a564270 0x55fd2a56b650 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f5d8402a550 tx=0x7f5d84004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f50caa1b50b90bf2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:21.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.919+0000 7f5d73400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:57:21.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.919+0000 7f5d73400640 1 -- 172.21.15.83:0/3988681712 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d84035070 con 0x55fd2a564270 2024-04-24T08:57:21.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.919+0000 7f5d73400640 1 -- 172.21.15.83:0/3988681712 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f5d840052c0 con 0x55fd2a564270 2024-04-24T08:57:21.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.919+0000 7f5d9b831740 1 -- 172.21.15.83:0/3988681712 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d840054f0 con 0x55fd2a564270 2024-04-24T08:57:21.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.919+0000 7f5d73400640 1 -- 172.21.15.83:0/3988681712 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f5d840035f0 con 0x55fd2a564270 2024-04-24T08:57:21.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.919+0000 7f5d9b831740 10 client.17715 fetch_fsmap learned FSMap version 33 2024-04-24T08:57:21.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.919+0000 7f5d9b831740 1 -- 172.21.15.83:0/3988681712 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55fd2a1db3f0 con 0x55fd2a564270 2024-04-24T08:57:21.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.920+0000 7f5d73400640 1 -- 172.21.15.83:0/3988681712 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5d840054f0 con 0x55fd2a564270 2024-04-24T08:57:21.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.920+0000 7f5d73400640 1 -- 172.21.15.83:0/3988681712 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f5d84031920 con 0x55fd2a564270 2024-04-24T08:57:21.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.920+0000 7f5d9b831740 10 client.17715 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T08:57:21.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.920+0000 7f5d9b831740 10 client.17715 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-24T08:57:21.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.920+0000 7f5d9b831740 20 client.17715 populate_metadata read hostname 'smithi083' 2024-04-24T08:57:21.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.920+0000 7f5d9b831740 1 --2- 172.21.15.83:0/3988681712 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55fd2a57e950 0x55fd2a580d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:21.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.920+0000 7f5d9b831740 4 client.17715 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:57:21.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.920+0000 7f5d9b831740 1 -- 172.21.15.83:0/3988681712 --> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fd2a4d6bf0 con 0x55fd2a57e950 2024-04-24T08:57:21.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.921+0000 7f5d91400640 1 --2- 172.21.15.83:0/3988681712 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55fd2a57e950 0x55fd2a580d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:21.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.922+0000 7f5d91400640 1 --2- 172.21.15.83:0/3988681712 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55fd2a57e950 0x55fd2a580d40 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=432b6eb424356787 server_cookie=6f7d1de5c078cf71 in_seq=0 out_seq=0 2024-04-24T08:57:21.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.922+0000 7f5d73400640 10 client.17715 ms_handle_connect on v2:172.21.15.138:6834/1909219758 2024-04-24T08:57:21.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.924+0000 7f5d73400640 1 -- 172.21.15.83:0/3988681712 <== mds.2 v2:172.21.15.138:6834/1909219758 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fd2a4d6bf0 con 0x55fd2a57e950 2024-04-24T08:57:21.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.924+0000 7f5d73400640 10 client.17715 handle_command_reply: tid=0 2024-04-24T08:57:21.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.924+0000 7f5d9b831740 1 client.17715 shutdown 2024-04-24T08:57:21.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.924+0000 7f5d9b831740 1 -- 172.21.15.83:0/3988681712 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd2a564270 msgr2=0x55fd2a56b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:21.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.924+0000 7f5d9b831740 1 --2- 172.21.15.83:0/3988681712 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd2a564270 0x55fd2a56b650 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f5d8402a550 tx=0x7f5d84004ab0 comp rx=0 tx=0).stop 2024-04-24T08:57:21.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.924+0000 7f5d9b831740 1 -- 172.21.15.83:0/3988681712 shutdown_connections 2024-04-24T08:57:21.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.924+0000 7f5d9b831740 1 --2- 172.21.15.83:0/3988681712 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55fd2a57e950 0x55fd2a580d40 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:21.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.925+0000 7f5d73400640 0 client.17715 ms_handle_reset on v2:172.21.15.138:6834/1909219758 2024-04-24T08:57:21.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.924+0000 7f5d9b831740 1 --2- 172.21.15.83:0/3988681712 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd2a56e940 0x55fd2a56bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:21.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.925+0000 7f5d9b831740 1 --2- 172.21.15.83:0/3988681712 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd2a564270 0x55fd2a56b650 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:21.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.925+0000 7f5d9b831740 1 --2- 172.21.15.83:0/3988681712 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fd2a563950 0x55fd2a56e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:21.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.925+0000 7f5d9b831740 1 -- 172.21.15.83:0/3988681712 >> 172.21.15.83:0/3988681712 conn(0x55fd2a55fc70 msgr2=0x55fd2a565970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:21.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.925+0000 7f5d9b831740 1 -- 172.21.15.83:0/3988681712 shutdown_connections 2024-04-24T08:57:21.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.925+0000 7f5d9b831740 1 -- 172.21.15.83:0/3988681712 wait complete. 2024-04-24T08:57:21.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.925+0000 7f5d9b831740 20 client.17715 trim_cache size 0 max 16384 2024-04-24T08:57:21.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.943+0000 7f5d9b831740 1 Processor -- start 2024-04-24T08:57:21.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.943+0000 7f5d9b831740 1 -- start start 2024-04-24T08:57:21.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.944+0000 7f5d9b831740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fd2a563950 0x55fd2a57a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:21.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.944+0000 7f5d9b831740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd2a564270 0x55fd2a56de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:21.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.944+0000 7f5d9b831740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd2a56e940 0x55fd2a583d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:21.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.944+0000 7f5d9b831740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5d941b1710 con 0x55fd2a56e940 2024-04-24T08:57:21.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.944+0000 7f5d9b831740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5d941b3250 con 0x55fd2a563950 2024-04-24T08:57:21.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.944+0000 7f5d9b831740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5d941b1de0 con 0x55fd2a564270 2024-04-24T08:57:21.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.944+0000 7f5d91400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd2a56e940 0x55fd2a583d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:21.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.944+0000 7f5d91400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd2a56e940 0x55fd2a583d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:59040/0 (socket says 172.21.15.83:59040) 2024-04-24T08:57:21.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.944+0000 7f5d91400640 1 -- 172.21.15.83:0/3999432246 learned_addr learned my addr 172.21.15.83:0/3999432246 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:57:21.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.944+0000 7f5d91400640 1 --2- 172.21.15.83:0/3999432246 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fd2a563950 0x55fd2a57a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:21.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.944+0000 7f5d91e00640 1 --2- 172.21.15.83:0/3999432246 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd2a564270 0x55fd2a56de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:21.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.945+0000 7f5d91400640 1 -- 172.21.15.83:0/3999432246 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd2a564270 msgr2=0x55fd2a56de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:21.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.945+0000 7f5d91400640 1 --2- 172.21.15.83:0/3999432246 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd2a564270 0x55fd2a56de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:21.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.945+0000 7f5d91400640 1 -- 172.21.15.83:0/3999432246 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fd2a563950 msgr2=0x55fd2a57a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:21.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.945+0000 7f5d91400640 1 --2- 172.21.15.83:0/3999432246 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fd2a563950 0x55fd2a57a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:21.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.945+0000 7f5d91400640 1 -- 172.21.15.83:0/3999432246 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd2a56bfd0 con 0x55fd2a56e940 2024-04-24T08:57:21.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.945+0000 7f5d91400640 1 --2- 172.21.15.83:0/3999432246 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd2a56e940 0x55fd2a583d60 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f5d80000c00 tx=0x7f5d80005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=452e933dd4d055d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:21.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.945+0000 7f5d7e600640 1 -- 172.21.15.83:0/3999432246 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d8001e920 con 0x55fd2a56e940 2024-04-24T08:57:21.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.945+0000 7f5d7e600640 1 -- 172.21.15.83:0/3999432246 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5d800035f0 con 0x55fd2a56e940 2024-04-24T08:57:21.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.945+0000 7f5d7e600640 1 -- 172.21.15.83:0/3999432246 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d800116a0 con 0x55fd2a56e940 2024-04-24T08:57:21.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.946+0000 7f5d9b831740 1 -- 172.21.15.83:0/3999432246 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd2a56e940 msgr2=0x55fd2a583d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:21.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.946+0000 7f5d9b831740 1 --2- 172.21.15.83:0/3999432246 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd2a56e940 0x55fd2a583d60 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f5d80000c00 tx=0x7f5d80005740 comp rx=0 tx=0).stop 2024-04-24T08:57:21.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.946+0000 7f5d9b831740 1 -- 172.21.15.83:0/3999432246 shutdown_connections 2024-04-24T08:57:21.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.946+0000 7f5d9b831740 1 --2- 172.21.15.83:0/3999432246 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd2a56e940 0x55fd2a583d60 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:21.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.946+0000 7f5d9b831740 1 --2- 172.21.15.83:0/3999432246 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd2a564270 0x55fd2a56de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:21.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.946+0000 7f5d9b831740 1 --2- 172.21.15.83:0/3999432246 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fd2a563950 0x55fd2a57a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:21.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.946+0000 7f5d9b831740 1 -- 172.21.15.83:0/3999432246 >> 172.21.15.83:0/3999432246 conn(0x55fd2a55fc70 msgr2=0x55fd2a579b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:21.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.946+0000 7f5d9b831740 1 -- 172.21.15.83:0/3999432246 shutdown_connections 2024-04-24T08:57:21.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.946+0000 7f5d9b831740 1 -- 172.21.15.83:0/3999432246 wait complete. 2024-04-24T08:57:21.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.947+0000 7f5d9b831740 1 -- start start 2024-04-24T08:57:21.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.947+0000 7f5d9b831740 1 Processor -- start 2024-04-24T08:57:21.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.947+0000 7f5d9b831740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fd2a563950 0x55fd2a5689b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:21.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.947+0000 7f5d9b831740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd2a564270 0x55fd2a568ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:21.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.947+0000 7f5d9b831740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd2a56e940 0x55fd2a5e3b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:21.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.947+0000 7f5d9b831740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5d941b2b80 con 0x55fd2a564270 2024-04-24T08:57:21.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.947+0000 7f5d9b831740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5d941b24b0 con 0x55fd2a563950 2024-04-24T08:57:21.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.947+0000 7f5d9b831740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5d941af6f0 con 0x55fd2a56e940 2024-04-24T08:57:21.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.947+0000 7f5d9b831740 10 client.0 _ll_register_callbacks cb 0x7f5d84004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:57:21.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.948+0000 7f5d91e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd2a564270 0x55fd2a568ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:21.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.948+0000 7f5d91e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd2a564270 0x55fd2a568ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:59048/0 (socket says 172.21.15.83:59048) 2024-04-24T08:57:21.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.948+0000 7f5d91e00640 1 -- 172.21.15.83:0/3484483093 learned_addr learned my addr 172.21.15.83:0/3484483093 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:57:21.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.948+0000 7f5d91400640 1 --2- 172.21.15.83:0/3484483093 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fd2a563950 0x55fd2a5689b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:21.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.948+0000 7f5d91e00640 1 -- 172.21.15.83:0/3484483093 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd2a56e940 msgr2=0x55fd2a5e3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:21.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.948+0000 7f5d91e00640 1 --2- 172.21.15.83:0/3484483093 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd2a56e940 0x55fd2a5e3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:21.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.948+0000 7f5d91e00640 1 -- 172.21.15.83:0/3484483093 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fd2a563950 msgr2=0x55fd2a5689b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:21.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.948+0000 7f5d91e00640 1 --2- 172.21.15.83:0/3484483093 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fd2a563950 0x55fd2a5689b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:21.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.948+0000 7f5d91e00640 1 -- 172.21.15.83:0/3484483093 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd2a2a57d0 con 0x55fd2a564270 2024-04-24T08:57:21.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.948+0000 7f5d91e00640 1 --2- 172.21.15.83:0/3484483093 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd2a564270 0x55fd2a568ef0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f5d84004b60 tx=0x7f5d84009230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0a77688221393e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:21.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.948+0000 7f5d73400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:57:21.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.949+0000 7f5d73400640 1 -- 172.21.15.83:0/3484483093 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d84035070 con 0x55fd2a564270 2024-04-24T08:57:21.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.949+0000 7f5d73400640 1 -- 172.21.15.83:0/3484483093 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f5d840318b0 con 0x55fd2a564270 2024-04-24T08:57:21.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.949+0000 7f5d9b831740 1 -- 172.21.15.83:0/3484483093 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d800035f0 con 0x55fd2a564270 2024-04-24T08:57:21.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.949+0000 7f5d73400640 1 -- 172.21.15.83:0/3484483093 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f5d84031ad0 con 0x55fd2a564270 2024-04-24T08:57:21.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.949+0000 7f5d9b831740 10 client.17721 fetch_fsmap learned FSMap version 33 2024-04-24T08:57:21.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.949+0000 7f5d9b831740 1 -- 172.21.15.83:0/3484483093 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55fd2a5e50f0 con 0x55fd2a564270 2024-04-24T08:57:21.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.949+0000 7f5d73400640 1 -- 172.21.15.83:0/3484483093 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5d800035f0 con 0x55fd2a564270 2024-04-24T08:57:21.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.949+0000 7f5d73400640 1 -- 172.21.15.83:0/3484483093 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f5d8403b4c0 con 0x55fd2a564270 2024-04-24T08:57:21.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.949+0000 7f5d9b831740 10 client.17721 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T08:57:21.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.949+0000 7f5d9b831740 10 client.17721 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-24T08:57:21.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.949+0000 7f5d9b831740 20 client.17721 populate_metadata read hostname 'smithi083' 2024-04-24T08:57:21.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.950+0000 7f5d9b831740 1 --2- 172.21.15.83:0/3484483093 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55fd2a5e56b0 0x55fd2a5e7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:21.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.950+0000 7f5d9b831740 4 client.17721 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T08:57:21.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.950+0000 7f5d9b831740 1 -- 172.21.15.83:0/3484483093 --> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fd2a4d0a60 con 0x55fd2a5e56b0 2024-04-24T08:57:21.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.950+0000 7f5d91400640 1 --2- 172.21.15.83:0/3484483093 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55fd2a5e56b0 0x55fd2a5e7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:21.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.951+0000 7f5d73400640 10 client.17721 ms_handle_connect on v2:172.21.15.138:6834/1909219758 2024-04-24T08:57:21.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.951+0000 7f5d91400640 1 --2- 172.21.15.83:0/3484483093 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55fd2a5e56b0 0x55fd2a5e7ad0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=36142f4476067212 server_cookie=863c0b800740283c in_seq=0 out_seq=0 2024-04-24T08:57:21.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.952+0000 7f5d73400640 1 -- 172.21.15.83:0/3484483093 <== mds.2 v2:172.21.15.138:6834/1909219758 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fd2a4d0a60 con 0x55fd2a5e56b0 2024-04-24T08:57:21.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.952+0000 7f5d73400640 10 client.17721 handle_command_reply: tid=0 2024-04-24T08:57:21.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.952+0000 7f5d9b831740 1 client.17721 shutdown 2024-04-24T08:57:21.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.952+0000 7f5d9b831740 1 -- 172.21.15.83:0/3484483093 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd2a564270 msgr2=0x55fd2a568ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:21.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.952+0000 7f5d9b831740 1 --2- 172.21.15.83:0/3484483093 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd2a564270 0x55fd2a568ef0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f5d84004b60 tx=0x7f5d84009230 comp rx=0 tx=0).stop 2024-04-24T08:57:21.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.952+0000 7f5d9b831740 1 -- 172.21.15.83:0/3484483093 shutdown_connections 2024-04-24T08:57:21.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.952+0000 7f5d9b831740 1 --2- 172.21.15.83:0/3484483093 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55fd2a5e56b0 0x55fd2a5e7ad0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:21.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.952+0000 7f5d9b831740 1 --2- 172.21.15.83:0/3484483093 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd2a56e940 0x55fd2a5e3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:21.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.952+0000 7f5d9b831740 1 --2- 172.21.15.83:0/3484483093 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd2a564270 0x55fd2a568ef0 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:21.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.952+0000 7f5d9b831740 1 --2- 172.21.15.83:0/3484483093 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fd2a563950 0x55fd2a5689b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:21.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.952+0000 7f5d73400640 0 client.17721 ms_handle_reset on v2:172.21.15.138:6834/1909219758 2024-04-24T08:57:21.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.952+0000 7f5d9b831740 1 -- 172.21.15.83:0/3484483093 >> 172.21.15.83:0/3484483093 conn(0x55fd2a55fc70 msgr2=0x55fd2a579b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:21.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.952+0000 7f5d9b831740 1 -- 172.21.15.83:0/3484483093 shutdown_connections 2024-04-24T08:57:21.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.952+0000 7f5d9b831740 1 -- 172.21.15.83:0/3484483093 wait complete. 2024-04-24T08:57:21.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.952+0000 7f5d9b831740 20 client.17721 trim_cache size 0 max 16384 2024-04-24T08:57:21.954 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T08:57:21.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.953+0000 7f5d98a00640 1 -- 172.21.15.83:0/3093095984 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f5d780a3d50 msgr2=0x7f5d780a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:21.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.953+0000 7f5d98a00640 1 --2- 172.21.15.83:0/3093095984 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f5d780a3d50 0x7f5d780a6210 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c099310 tx=0x7f5d8c00cf50 comp rx=0 tx=0).stop 2024-04-24T08:57:21.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.953+0000 7f5d98a00640 1 -- 172.21.15.83:0/3093095984 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d9410c5c0 msgr2=0x7f5d94101950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:21.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.953+0000 7f5d98a00640 1 --2- 172.21.15.83:0/3093095984 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d9410c5c0 0x7f5d94101950 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c00eaa0 tx=0x7f5d8c00ef60 comp rx=0 tx=0).stop 2024-04-24T08:57:21.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.953+0000 7f5d98a00640 1 -- 172.21.15.83:0/3093095984 shutdown_connections 2024-04-24T08:57:21.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.953+0000 7f5d98a00640 1 --2- 172.21.15.83:0/3093095984 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f5d780a3d50 0x7f5d780a6210 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:21.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.953+0000 7f5d98a00640 1 --2- 172.21.15.83:0/3093095984 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d9410c5c0 0x7f5d94101950 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:21.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.953+0000 7f5d98a00640 1 --2- 172.21.15.83:0/3093095984 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d94103ba0 0x7f5d94101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:21.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.953+0000 7f5d98a00640 1 --2- 172.21.15.83:0/3093095984 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5d94103280 0x7f5d94100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:21.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.953+0000 7f5d98a00640 1 -- 172.21.15.83:0/3093095984 >> 172.21.15.83:0/3093095984 conn(0x7f5d94078000 msgr2=0x7f5d940fe000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:21.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.953+0000 7f5d98a00640 1 -- 172.21.15.83:0/3093095984 shutdown_connections 2024-04-24T08:57:21.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:21.953+0000 7f5d98a00640 1 -- 172.21.15.83:0/3093095984 wait complete. 2024-04-24T08:57:21.968 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:57:21 smithi083 ceph-mon[27907]: pgmap v175: 129 pgs: 129 active+clean; 39 MiB data, 530 MiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 391 KiB/s wr, 39 op/s 2024-04-24T08:57:21.968 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:57:21 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/544681650' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:57:21.971 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T08:57:22.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.173+0000 7f7d58600640 1 -- 172.21.15.83:0/1780374245 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d50107340 msgr2=0x7f7d5010ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:22.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.173+0000 7f7d58600640 1 --2- 172.21.15.83:0/1780374245 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d50107340 0x7f7d5010ef10 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f7d48009ca0 tx=0x7f7d4802a8c0 comp rx=0 tx=0).stop 2024-04-24T08:57:22.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.173+0000 7f7d58600640 1 -- 172.21.15.83:0/1780374245 shutdown_connections 2024-04-24T08:57:22.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.173+0000 7f7d58600640 1 --2- 172.21.15.83:0/1780374245 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d50107340 0x7f7d5010ef10 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:22.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.173+0000 7f7d58600640 1 --2- 172.21.15.83:0/1780374245 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d50102c20 0x7f7d50106c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:22.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.173+0000 7f7d58600640 1 --2- 172.21.15.83:0/1780374245 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7d50102270 0x7f7d50102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:22.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.173+0000 7f7d58600640 1 -- 172.21.15.83:0/1780374245 >> 172.21.15.83:0/1780374245 conn(0x7f7d50079490 msgr2=0x7f7d500798a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:22.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.173+0000 7f7d58600640 1 -- 172.21.15.83:0/1780374245 shutdown_connections 2024-04-24T08:57:22.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.173+0000 7f7d58600640 1 -- 172.21.15.83:0/1780374245 wait complete. 2024-04-24T08:57:22.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.174+0000 7f7d58600640 1 Processor -- start 2024-04-24T08:57:22.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.174+0000 7f7d58600640 1 -- start start 2024-04-24T08:57:22.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.175+0000 7f7d58600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7d50102270 0x7f7d50074300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:22.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.175+0000 7f7d58600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d50107340 0x7f7d50074840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:22.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.175+0000 7f7d58600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d50074d80 0x7f7d501a6a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:22.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.175+0000 7f7d58600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7d50112370 con 0x7f7d50107340 2024-04-24T08:57:22.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.175+0000 7f7d58600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7d501121f0 con 0x7f7d50074d80 2024-04-24T08:57:22.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.175+0000 7f7d58600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7d501124f0 con 0x7f7d50102270 2024-04-24T08:57:22.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.175+0000 7f7d55200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d50107340 0x7f7d50074840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:22.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.175+0000 7f7d55200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d50107340 0x7f7d50074840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:59056/0 (socket says 172.21.15.83:59056) 2024-04-24T08:57:22.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.175+0000 7f7d55200640 1 -- 172.21.15.83:0/521704136 learned_addr learned my addr 172.21.15.83:0/521704136 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:57:22.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.175+0000 7f7d56600640 1 --2- 172.21.15.83:0/521704136 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d50074d80 0x7f7d501a6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:22.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.175+0000 7f7d55c00640 1 --2- 172.21.15.83:0/521704136 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7d50102270 0x7f7d50074300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:22.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.176+0000 7f7d55200640 1 -- 172.21.15.83:0/521704136 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7d50102270 msgr2=0x7f7d50074300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:22.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.176+0000 7f7d55200640 1 --2- 172.21.15.83:0/521704136 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7d50102270 0x7f7d50074300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:22.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.176+0000 7f7d55200640 1 -- 172.21.15.83:0/521704136 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d50074d80 msgr2=0x7f7d501a6a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:22.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.176+0000 7f7d55200640 1 --2- 172.21.15.83:0/521704136 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d50074d80 0x7f7d501a6a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:22.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.176+0000 7f7d55200640 1 -- 172.21.15.83:0/521704136 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d501a7120 con 0x7f7d50107340 2024-04-24T08:57:22.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.176+0000 7f7d55200640 1 --2- 172.21.15.83:0/521704136 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d50107340 0x7f7d50074840 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f7d3c00b450 tx=0x7f7d3c00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:22.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.176+0000 7f7d4e000640 1 -- 172.21.15.83:0/521704136 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d3c015020 con 0x7f7d50107340 2024-04-24T08:57:22.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.176+0000 7f7d58600640 1 -- 172.21.15.83:0/521704136 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d501a7410 con 0x7f7d50107340 2024-04-24T08:57:22.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.176+0000 7f7d4e000640 1 -- 172.21.15.83:0/521704136 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7d3c011d40 con 0x7f7d50107340 2024-04-24T08:57:22.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.177+0000 7f7d58600640 1 -- 172.21.15.83:0/521704136 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d501a79b0 con 0x7f7d50107340 2024-04-24T08:57:22.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.177+0000 7f7d4e000640 1 -- 172.21.15.83:0/521704136 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d3c010e20 con 0x7f7d50107340 2024-04-24T08:57:22.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.180+0000 7f7d5b3bf740 1 Processor -- start 2024-04-24T08:57:22.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.180+0000 7f7d5b3bf740 1 -- start start 2024-04-24T08:57:22.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.180+0000 7f7d4e000640 1 -- 172.21.15.83:0/521704136 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7d3c018020 con 0x7f7d50107340 2024-04-24T08:57:22.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.180+0000 7f7d5b3bf740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fe4fa74950 0x55fe4fa74d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:22.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.180+0000 7f7d5b3bf740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe4fa75270 0x55fe4fa7f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:22.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.180+0000 7f7d5b3bf740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe4fa7f940 0x55fe4fa81d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:22.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.180+0000 7f7d5b3bf740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55fe4f9da1d0 con 0x55fe4fa75270 2024-04-24T08:57:22.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.180+0000 7f7d5b3bf740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55fe4f980450 con 0x55fe4fa7f940 2024-04-24T08:57:22.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.180+0000 7f7d5b3bf740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55fe4f9d6190 con 0x55fe4fa74950 2024-04-24T08:57:22.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.181+0000 7f7d55c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe4fa75270 0x55fe4fa7f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:22.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.181+0000 7f7d55c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe4fa75270 0x55fe4fa7f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:59070/0 (socket says 172.21.15.83:59070) 2024-04-24T08:57:22.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.181+0000 7f7d55c00640 1 -- 172.21.15.83:0/2739481704 learned_addr learned my addr 172.21.15.83:0/2739481704 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:57:22.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.181+0000 7f7d56600640 1 --2- 172.21.15.83:0/2739481704 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fe4fa74950 0x55fe4fa74d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:22.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.181+0000 7f7d55200640 1 --2- 172.21.15.83:0/2739481704 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe4fa7f940 0x55fe4fa81d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:22.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.181+0000 7f7d55c00640 1 -- 172.21.15.83:0/2739481704 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fe4fa74950 msgr2=0x55fe4fa74d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:22.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.181+0000 7f7d55c00640 1 --2- 172.21.15.83:0/2739481704 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fe4fa74950 0x55fe4fa74d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:22.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.181+0000 7f7d55c00640 1 -- 172.21.15.83:0/2739481704 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe4fa7f940 msgr2=0x55fe4fa81d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:22.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.181+0000 7f7d55c00640 1 --2- 172.21.15.83:0/2739481704 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe4fa7f940 0x55fe4fa81d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:22.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.181+0000 7f7d55c00640 1 -- 172.21.15.83:0/2739481704 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe4fa3d310 con 0x55fe4fa75270 2024-04-24T08:57:22.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.181+0000 7f7d55200640 1 --2- 172.21.15.83:0/2739481704 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe4fa7f940 0x55fe4fa81d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:57:22.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.182+0000 7f7d55c00640 1 --2- 172.21.15.83:0/2739481704 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe4fa75270 0x55fe4fa7f400 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f7d4400b760 tx=0x7f7d4400bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd297074e289300b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:22.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.182+0000 7f7d4e000640 1 --2- 172.21.15.83:0/521704136 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f7d380a3d50 0x7f7d380a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:22.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.182+0000 7f7d42a00640 1 -- 172.21.15.83:0/2739481704 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d44003dd0 con 0x55fe4fa75270 2024-04-24T08:57:22.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.182+0000 7f7d42a00640 1 -- 172.21.15.83:0/2739481704 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7d44015070 con 0x55fe4fa75270 2024-04-24T08:57:22.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.182+0000 7f7d42a00640 1 -- 172.21.15.83:0/2739481704 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d44004d50 con 0x55fe4fa75270 2024-04-24T08:57:22.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.182+0000 7f7d4e000640 1 -- 172.21.15.83:0/521704136 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f7d3c0d0450 con 0x7f7d50107340 2024-04-24T08:57:22.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.182+0000 7f7d56600640 1 --2- 172.21.15.83:0/521704136 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f7d380a3d50 0x7f7d380a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:22.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.183+0000 7f7d5b3bf740 1 -- 172.21.15.83:0/2739481704 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe4fa75270 msgr2=0x55fe4fa7f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:22.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.183+0000 7f7d5b3bf740 1 --2- 172.21.15.83:0/2739481704 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe4fa75270 0x55fe4fa7f400 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f7d4400b760 tx=0x7f7d4400bc20 comp rx=0 tx=0).stop 2024-04-24T08:57:22.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.183+0000 7f7d56600640 1 --2- 172.21.15.83:0/521704136 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f7d380a3d50 0x7f7d380a6210 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f7d48006010 tx=0x7f7d48004a40 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:22.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.183+0000 7f7d5b3bf740 1 -- 172.21.15.83:0/2739481704 shutdown_connections 2024-04-24T08:57:22.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.183+0000 7f7d5b3bf740 1 --2- 172.21.15.83:0/2739481704 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe4fa7f940 0x55fe4fa81d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:22.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.183+0000 7f7d5b3bf740 1 --2- 172.21.15.83:0/2739481704 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe4fa75270 0x55fe4fa7f400 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:22.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.183+0000 7f7d5b3bf740 1 --2- 172.21.15.83:0/2739481704 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fe4fa74950 0x55fe4fa74d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:22.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.183+0000 7f7d5b3bf740 1 -- 172.21.15.83:0/2739481704 >> 172.21.15.83:0/2739481704 conn(0x55fe4fa70c70 msgr2=0x55fe4fa71050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:22.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.183+0000 7f7d5b3bf740 1 -- 172.21.15.83:0/2739481704 shutdown_connections 2024-04-24T08:57:22.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.183+0000 7f7d5b3bf740 1 -- 172.21.15.83:0/2739481704 wait complete. 2024-04-24T08:57:22.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.184+0000 7f7d5b3bf740 1 -- start start 2024-04-24T08:57:22.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.184+0000 7f7d5b3bf740 1 Processor -- start 2024-04-24T08:57:22.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.185+0000 7f7d5b3bf740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fe4fa74950 0x55fe4fa7f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:22.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.185+0000 7f7d5b3bf740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe4fa75270 0x55fe4fa7c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:22.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.185+0000 7f7d5b3bf740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe4fa7f940 0x55fe4fa7cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:22.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.185+0000 7f7d5b3bf740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55fe4fa827d0 con 0x55fe4fa7f940 2024-04-24T08:57:22.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.185+0000 7f7d5b3bf740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55fe4f993810 con 0x55fe4fa75270 2024-04-24T08:57:22.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.185+0000 7f7d5b3bf740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55fe4fa82950 con 0x55fe4fa74950 2024-04-24T08:57:22.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.185+0000 7f7d5b3bf740 10 client.0 _ll_register_callbacks cb 0x55fe4f961140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:57:22.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.185+0000 7f7d56600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe4fa7f940 0x55fe4fa7cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:22.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.185+0000 7f7d56600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe4fa7f940 0x55fe4fa7cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:59076/0 (socket says 172.21.15.83:59076) 2024-04-24T08:57:22.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.185+0000 7f7d56600640 1 -- 172.21.15.83:0/2859380236 learned_addr learned my addr 172.21.15.83:0/2859380236 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:57:22.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.185+0000 7f7d55200640 1 --2- 172.21.15.83:0/2859380236 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe4fa75270 0x55fe4fa7c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:22.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.185+0000 7f7d55c00640 1 --2- 172.21.15.83:0/2859380236 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fe4fa74950 0x55fe4fa7f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:22.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.185+0000 7f7d56600640 1 -- 172.21.15.83:0/2859380236 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fe4fa74950 msgr2=0x55fe4fa7f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:22.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.185+0000 7f7d56600640 1 --2- 172.21.15.83:0/2859380236 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fe4fa74950 0x55fe4fa7f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:22.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.185+0000 7f7d56600640 1 -- 172.21.15.83:0/2859380236 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe4fa75270 msgr2=0x55fe4fa7c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:22.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.185+0000 7f7d56600640 1 --2- 172.21.15.83:0/2859380236 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe4fa75270 0x55fe4fa7c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:22.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.185+0000 7f7d56600640 1 -- 172.21.15.83:0/2859380236 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe4f9a43f0 con 0x55fe4fa7f940 2024-04-24T08:57:22.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.186+0000 7f7d55200640 1 --2- 172.21.15.83:0/2859380236 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe4fa75270 0x55fe4fa7c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:57:22.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.186+0000 7f7d56600640 1 --2- 172.21.15.83:0/2859380236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe4fa7f940 0x55fe4fa7cdb0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f7d48035000 tx=0x7f7d4802a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c84bab419e856360 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:22.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.186+0000 7f7d37400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:57:22.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.186+0000 7f7d37400640 1 -- 172.21.15.83:0/2859380236 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d480305c0 con 0x55fe4fa7f940 2024-04-24T08:57:22.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.186+0000 7f7d37400640 1 -- 172.21.15.83:0/2859380236 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f7d48030760 con 0x55fe4fa7f940 2024-04-24T08:57:22.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.186+0000 7f7d5b3bf740 1 -- 172.21.15.83:0/2859380236 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d44015070 con 0x55fe4fa7f940 2024-04-24T08:57:22.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.187+0000 7f7d37400640 1 -- 172.21.15.83:0/2859380236 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f7d48030960 con 0x55fe4fa7f940 2024-04-24T08:57:22.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.187+0000 7f7d5b3bf740 10 client.17733 fetch_fsmap learned FSMap version 33 2024-04-24T08:57:22.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.187+0000 7f7d5b3bf740 1 -- 172.21.15.83:0/2859380236 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55fe4f6ec3f0 con 0x55fe4fa7f940 2024-04-24T08:57:22.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.187+0000 7f7d37400640 1 -- 172.21.15.83:0/2859380236 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7d44015070 con 0x55fe4fa7f940 2024-04-24T08:57:22.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.187+0000 7f7d37400640 1 -- 172.21.15.83:0/2859380236 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f7d48051070 con 0x55fe4fa7f940 2024-04-24T08:57:22.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.187+0000 7f7d5b3bf740 10 client.17733 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T08:57:22.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.187+0000 7f7d5b3bf740 10 client.17733 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-24T08:57:22.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.187+0000 7f7d5b3bf740 20 client.17733 populate_metadata read hostname 'smithi083' 2024-04-24T08:57:22.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.188+0000 7f7d5b3bf740 1 --2- 172.21.15.83:0/2859380236 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x55fe4fa8fa40 0x55fe4fa91e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:22.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.188+0000 7f7d5b3bf740 4 client.17733 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:57:22.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.188+0000 7f7d5b3bf740 1 -- 172.21.15.83:0/2859380236 --> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fe4f9e7bf0 con 0x55fe4fa8fa40 2024-04-24T08:57:22.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.188+0000 7f7d55c00640 1 --2- 172.21.15.83:0/2859380236 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x55fe4fa8fa40 0x55fe4fa91e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:22.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.189+0000 7f7d37400640 10 client.17733 ms_handle_connect on v2:172.21.15.138:6836/900566754 2024-04-24T08:57:22.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.189+0000 7f7d55c00640 1 --2- 172.21.15.83:0/2859380236 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x55fe4fa8fa40 0x55fe4fa91e90 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=851f27976c464c63 server_cookie=da09dba312415642 in_seq=0 out_seq=0 2024-04-24T08:57:22.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.192+0000 7f7d37400640 1 -- 172.21.15.83:0/2859380236 <== mds.1 v2:172.21.15.138:6836/900566754 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fe4f9e7bf0 con 0x55fe4fa8fa40 2024-04-24T08:57:22.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.192+0000 7f7d37400640 10 client.17733 handle_command_reply: tid=0 2024-04-24T08:57:22.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.192+0000 7f7d5b3bf740 1 client.17733 shutdown 2024-04-24T08:57:22.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.192+0000 7f7d5b3bf740 1 -- 172.21.15.83:0/2859380236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe4fa7f940 msgr2=0x55fe4fa7cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:22.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.192+0000 7f7d5b3bf740 1 --2- 172.21.15.83:0/2859380236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe4fa7f940 0x55fe4fa7cdb0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f7d48035000 tx=0x7f7d4802a8c0 comp rx=0 tx=0).stop 2024-04-24T08:57:22.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.192+0000 7f7d5b3bf740 1 -- 172.21.15.83:0/2859380236 shutdown_connections 2024-04-24T08:57:22.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.192+0000 7f7d5b3bf740 1 --2- 172.21.15.83:0/2859380236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe4fa7f940 0x55fe4fa7cdb0 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:22.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.192+0000 7f7d5b3bf740 1 --2- 172.21.15.83:0/2859380236 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe4fa75270 0x55fe4fa7c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:22.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.192+0000 7f7d5b3bf740 1 --2- 172.21.15.83:0/2859380236 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x55fe4fa8fa40 0x55fe4fa91e90 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:22.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.192+0000 7f7d5b3bf740 1 --2- 172.21.15.83:0/2859380236 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fe4fa74950 0x55fe4fa7f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:22.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.192+0000 7f7d5b3bf740 1 -- 172.21.15.83:0/2859380236 >> 172.21.15.83:0/2859380236 conn(0x55fe4fa70c70 msgr2=0x55fe4fa76a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:22.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.192+0000 7f7d37400640 0 client.17733 ms_handle_reset on v2:172.21.15.138:6836/900566754 2024-04-24T08:57:22.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.193+0000 7f7d5b3bf740 1 -- 172.21.15.83:0/2859380236 shutdown_connections 2024-04-24T08:57:22.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.193+0000 7f7d5b3bf740 1 -- 172.21.15.83:0/2859380236 wait complete. 2024-04-24T08:57:22.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.193+0000 7f7d5b3bf740 20 client.17733 trim_cache size 0 max 16384 2024-04-24T08:57:22.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:57:21 smithi138 ceph-mon[31441]: pgmap v175: 129 pgs: 129 active+clean; 39 MiB data, 530 MiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 391 KiB/s wr, 39 op/s 2024-04-24T08:57:22.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:57:21 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/544681650' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:57:22.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:57:21 smithi189 ceph-mon[31791]: pgmap v175: 129 pgs: 129 active+clean; 39 MiB data, 530 MiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 391 KiB/s wr, 39 op/s 2024-04-24T08:57:22.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:57:21 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/544681650' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:57:22.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.209+0000 7f7d5b3bf740 1 Processor -- start 2024-04-24T08:57:22.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.209+0000 7f7d5b3bf740 1 -- start start 2024-04-24T08:57:22.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.210+0000 7f7d5b3bf740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fe4fa74950 0x55fe4fa770f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:22.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.210+0000 7f7d5b3bf740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe4fa75270 0x55fe4fa7b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:22.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.210+0000 7f7d5b3bf740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe4fa7f940 0x55fe4fa7eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:22.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.210+0000 7f7d5b3bf740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7d501b17b0 con 0x55fe4fa7f940 2024-04-24T08:57:22.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.210+0000 7f7d5b3bf740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7d501b32f0 con 0x55fe4fa75270 2024-04-24T08:57:22.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.210+0000 7f7d5b3bf740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7d501b1e80 con 0x55fe4fa74950 2024-04-24T08:57:22.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.210+0000 7f7d56600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe4fa7f940 0x55fe4fa7eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:22.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.210+0000 7f7d56600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe4fa7f940 0x55fe4fa7eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:59084/0 (socket says 172.21.15.83:59084) 2024-04-24T08:57:22.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.210+0000 7f7d56600640 1 -- 172.21.15.83:0/454521137 learned_addr learned my addr 172.21.15.83:0/454521137 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:57:22.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.210+0000 7f7d55200640 1 --2- 172.21.15.83:0/454521137 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe4fa75270 0x55fe4fa7b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:22.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.210+0000 7f7d55c00640 1 --2- 172.21.15.83:0/454521137 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fe4fa74950 0x55fe4fa770f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:22.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.211+0000 7f7d56600640 1 -- 172.21.15.83:0/454521137 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fe4fa74950 msgr2=0x55fe4fa770f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:22.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.211+0000 7f7d56600640 1 --2- 172.21.15.83:0/454521137 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fe4fa74950 0x55fe4fa770f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:22.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.211+0000 7f7d56600640 1 -- 172.21.15.83:0/454521137 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe4fa75270 msgr2=0x55fe4fa7b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:22.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.211+0000 7f7d56600640 1 --2- 172.21.15.83:0/454521137 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe4fa75270 0x55fe4fa7b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:22.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.211+0000 7f7d56600640 1 -- 172.21.15.83:0/454521137 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe4f7b67d0 con 0x55fe4fa7f940 2024-04-24T08:57:22.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.211+0000 7f7d55200640 1 --2- 172.21.15.83:0/454521137 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe4fa75270 0x55fe4fa7b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:57:22.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.211+0000 7f7d56600640 1 --2- 172.21.15.83:0/454521137 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe4fa7f940 0x55fe4fa7eed0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f7d48053cd0 tx=0x7f7d48054f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11232889f1e8a91f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:22.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.211+0000 7f7d42000640 1 -- 172.21.15.83:0/454521137 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d480305c0 con 0x55fe4fa7f940 2024-04-24T08:57:22.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.211+0000 7f7d42000640 1 -- 172.21.15.83:0/454521137 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7d48036050 con 0x55fe4fa7f940 2024-04-24T08:57:22.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.211+0000 7f7d42000640 1 -- 172.21.15.83:0/454521137 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d48053680 con 0x55fe4fa7f940 2024-04-24T08:57:22.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.212+0000 7f7d5b3bf740 1 -- 172.21.15.83:0/454521137 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe4fa7f940 msgr2=0x55fe4fa7eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:22.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.212+0000 7f7d5b3bf740 1 --2- 172.21.15.83:0/454521137 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe4fa7f940 0x55fe4fa7eed0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f7d48053cd0 tx=0x7f7d48054f50 comp rx=0 tx=0).stop 2024-04-24T08:57:22.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.212+0000 7f7d5b3bf740 1 -- 172.21.15.83:0/454521137 shutdown_connections 2024-04-24T08:57:22.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.212+0000 7f7d5b3bf740 1 --2- 172.21.15.83:0/454521137 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe4fa7f940 0x55fe4fa7eed0 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:22.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.212+0000 7f7d5b3bf740 1 --2- 172.21.15.83:0/454521137 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe4fa75270 0x55fe4fa7b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:22.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.212+0000 7f7d5b3bf740 1 --2- 172.21.15.83:0/454521137 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fe4fa74950 0x55fe4fa770f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:22.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.212+0000 7f7d5b3bf740 1 -- 172.21.15.83:0/454521137 >> 172.21.15.83:0/454521137 conn(0x55fe4fa70c70 msgr2=0x55fe4fa8a680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:22.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.212+0000 7f7d5b3bf740 1 -- 172.21.15.83:0/454521137 shutdown_connections 2024-04-24T08:57:22.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.212+0000 7f7d5b3bf740 1 -- 172.21.15.83:0/454521137 wait complete. 2024-04-24T08:57:22.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.213+0000 7f7d5b3bf740 1 -- start start 2024-04-24T08:57:22.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.213+0000 7f7d5b3bf740 1 Processor -- start 2024-04-24T08:57:22.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.213+0000 7f7d5b3bf740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe4fa74950 0x55fe4fa87f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:22.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.213+0000 7f7d5b3bf740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fe4fa75270 0x55fe4fa79060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:22.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.213+0000 7f7d5b3bf740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe4fa7f940 0x55fe4fa795a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:22.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.213+0000 7f7d5b3bf740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7d501b2c20 con 0x55fe4fa74950 2024-04-24T08:57:22.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.213+0000 7f7d5b3bf740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7d501b2550 con 0x55fe4fa7f940 2024-04-24T08:57:22.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.213+0000 7f7d5b3bf740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7d501af790 con 0x55fe4fa75270 2024-04-24T08:57:22.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.214+0000 7f7d5b3bf740 10 client.0 _ll_register_callbacks cb 0x7f7d48009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:57:22.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.214+0000 7f7d55c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe4fa74950 0x55fe4fa87f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:22.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.214+0000 7f7d55c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe4fa74950 0x55fe4fa87f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:59088/0 (socket says 172.21.15.83:59088) 2024-04-24T08:57:22.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.214+0000 7f7d55c00640 1 -- 172.21.15.83:0/739740806 learned_addr learned my addr 172.21.15.83:0/739740806 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:57:22.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.214+0000 7f7d56600640 1 --2- 172.21.15.83:0/739740806 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe4fa7f940 0x55fe4fa795a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:22.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.214+0000 7f7d55c00640 1 -- 172.21.15.83:0/739740806 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fe4fa75270 msgr2=0x55fe4fa79060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:22.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.214+0000 7f7d55c00640 1 --2- 172.21.15.83:0/739740806 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fe4fa75270 0x55fe4fa79060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:22.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.214+0000 7f7d55c00640 1 -- 172.21.15.83:0/739740806 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe4fa7f940 msgr2=0x55fe4fa795a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:22.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.214+0000 7f7d55c00640 1 --2- 172.21.15.83:0/739740806 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe4fa7f940 0x55fe4fa795a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:22.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.214+0000 7f7d55c00640 1 -- 172.21.15.83:0/739740806 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe4fa79dd0 con 0x55fe4fa74950 2024-04-24T08:57:22.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.214+0000 7f7d56600640 1 --2- 172.21.15.83:0/739740806 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe4fa7f940 0x55fe4fa795a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:57:22.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.214+0000 7f7d55c00640 1 --2- 172.21.15.83:0/739740806 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe4fa74950 0x55fe4fa87f80 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f7d440049c0 tx=0x7f7d440049f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=753e1878aa7d0d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:22.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.214+0000 7f7d37400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:57:22.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.215+0000 7f7d37400640 1 -- 172.21.15.83:0/739740806 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d440270d0 con 0x55fe4fa74950 2024-04-24T08:57:22.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.215+0000 7f7d37400640 1 -- 172.21.15.83:0/739740806 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f7d44027270 con 0x55fe4fa74950 2024-04-24T08:57:22.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.215+0000 7f7d5b3bf740 1 -- 172.21.15.83:0/739740806 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d48036050 con 0x55fe4fa74950 2024-04-24T08:57:22.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.215+0000 7f7d37400640 1 -- 172.21.15.83:0/739740806 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f7d44027470 con 0x55fe4fa74950 2024-04-24T08:57:22.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.215+0000 7f7d5b3bf740 10 client.17739 fetch_fsmap learned FSMap version 33 2024-04-24T08:57:22.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.215+0000 7f7d5b3bf740 1 -- 172.21.15.83:0/739740806 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55fe4faf5fd0 con 0x55fe4fa74950 2024-04-24T08:57:22.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.215+0000 7f7d37400640 1 -- 172.21.15.83:0/739740806 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7d48036050 con 0x55fe4fa74950 2024-04-24T08:57:22.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.216+0000 7f7d37400640 1 -- 172.21.15.83:0/739740806 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f7d44011b10 con 0x55fe4fa74950 2024-04-24T08:57:22.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.216+0000 7f7d5b3bf740 10 client.17739 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T08:57:22.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.216+0000 7f7d5b3bf740 10 client.17739 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-24T08:57:22.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.216+0000 7f7d5b3bf740 20 client.17739 populate_metadata read hostname 'smithi083' 2024-04-24T08:57:22.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.216+0000 7f7d5b3bf740 1 --2- 172.21.15.83:0/739740806 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x55fe4faf6520 0x55fe4faf8960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:22.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.216+0000 7f7d5b3bf740 4 client.17739 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T08:57:22.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.216+0000 7f7d5b3bf740 1 -- 172.21.15.83:0/739740806 --> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fe4f9e1a60 con 0x55fe4faf6520 2024-04-24T08:57:22.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.216+0000 7f7d55200640 1 --2- 172.21.15.83:0/739740806 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x55fe4faf6520 0x55fe4faf8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:22.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.217+0000 7f7d55200640 1 --2- 172.21.15.83:0/739740806 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x55fe4faf6520 0x55fe4faf8960 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=53e349fb0e903033 server_cookie=f4b273ba0cd16ee6 in_seq=0 out_seq=0 2024-04-24T08:57:22.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.217+0000 7f7d37400640 10 client.17739 ms_handle_connect on v2:172.21.15.138:6836/900566754 2024-04-24T08:57:22.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.218+0000 7f7d37400640 1 -- 172.21.15.83:0/739740806 <== mds.1 v2:172.21.15.138:6836/900566754 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fe4f9e1a60 con 0x55fe4faf6520 2024-04-24T08:57:22.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.218+0000 7f7d37400640 10 client.17739 handle_command_reply: tid=0 2024-04-24T08:57:22.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.218+0000 7f7d5b3bf740 1 client.17739 shutdown 2024-04-24T08:57:22.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.218+0000 7f7d5b3bf740 1 -- 172.21.15.83:0/739740806 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe4fa74950 msgr2=0x55fe4fa87f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:22.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.218+0000 7f7d5b3bf740 1 --2- 172.21.15.83:0/739740806 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe4fa74950 0x55fe4fa87f80 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f7d440049c0 tx=0x7f7d440049f0 comp rx=0 tx=0).stop 2024-04-24T08:57:22.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.218+0000 7f7d5b3bf740 1 -- 172.21.15.83:0/739740806 shutdown_connections 2024-04-24T08:57:22.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.218+0000 7f7d5b3bf740 1 --2- 172.21.15.83:0/739740806 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe4fa7f940 0x55fe4fa795a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:22.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.218+0000 7f7d5b3bf740 1 --2- 172.21.15.83:0/739740806 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x55fe4faf6520 0x55fe4faf8960 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:22.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.218+0000 7f7d5b3bf740 1 --2- 172.21.15.83:0/739740806 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fe4fa75270 0x55fe4fa79060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:22.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.218+0000 7f7d5b3bf740 1 --2- 172.21.15.83:0/739740806 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe4fa74950 0x55fe4fa87f80 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:22.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.218+0000 7f7d37400640 0 client.17739 ms_handle_reset on v2:172.21.15.138:6836/900566754 2024-04-24T08:57:22.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.218+0000 7f7d5b3bf740 1 -- 172.21.15.83:0/739740806 >> 172.21.15.83:0/739740806 conn(0x55fe4fa70c70 msgr2=0x55fe4fa76970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:22.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.218+0000 7f7d5b3bf740 1 -- 172.21.15.83:0/739740806 shutdown_connections 2024-04-24T08:57:22.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.218+0000 7f7d5b3bf740 1 -- 172.21.15.83:0/739740806 wait complete. 2024-04-24T08:57:22.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.218+0000 7f7d5b3bf740 20 client.17739 trim_cache size 0 max 16384 2024-04-24T08:57:22.220 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T08:57:22.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.219+0000 7f7d58600640 1 -- 172.21.15.83:0/521704136 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f7d380a3d50 msgr2=0x7f7d380a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:22.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.219+0000 7f7d58600640 1 --2- 172.21.15.83:0/521704136 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f7d380a3d50 0x7f7d380a6210 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f7d48006010 tx=0x7f7d48004a40 comp rx=0 tx=0).stop 2024-04-24T08:57:22.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.219+0000 7f7d58600640 1 -- 172.21.15.83:0/521704136 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d50107340 msgr2=0x7f7d50074840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:22.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.219+0000 7f7d58600640 1 --2- 172.21.15.83:0/521704136 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d50107340 0x7f7d50074840 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f7d3c00b450 tx=0x7f7d3c00b910 comp rx=0 tx=0).stop 2024-04-24T08:57:22.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.219+0000 7f7d58600640 1 -- 172.21.15.83:0/521704136 shutdown_connections 2024-04-24T08:57:22.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.219+0000 7f7d58600640 1 --2- 172.21.15.83:0/521704136 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f7d380a3d50 0x7f7d380a6210 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:22.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.219+0000 7f7d58600640 1 --2- 172.21.15.83:0/521704136 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d50074d80 0x7f7d501a6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:22.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.219+0000 7f7d58600640 1 --2- 172.21.15.83:0/521704136 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d50107340 0x7f7d50074840 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:22.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.219+0000 7f7d58600640 1 --2- 172.21.15.83:0/521704136 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7d50102270 0x7f7d50074300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:22.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.219+0000 7f7d58600640 1 -- 172.21.15.83:0/521704136 >> 172.21.15.83:0/521704136 conn(0x7f7d50079490 msgr2=0x7f7d50108510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:22.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.219+0000 7f7d58600640 1 -- 172.21.15.83:0/521704136 shutdown_connections 2024-04-24T08:57:22.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.219+0000 7f7d58600640 1 -- 172.21.15.83:0/521704136 wait complete. 2024-04-24T08:57:22.235 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T08:57:22.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.454+0000 7faafe600640 1 -- 172.21.15.83:0/739789294 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faaf80743a0 msgr2=0x7faaf8074780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:22.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.454+0000 7faafe600640 1 --2- 172.21.15.83:0/739789294 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faaf80743a0 0x7faaf8074780 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7faaec009900 tx=0x7faaec02a480 comp rx=0 tx=0).stop 2024-04-24T08:57:22.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.454+0000 7faafe600640 1 -- 172.21.15.83:0/739789294 shutdown_connections 2024-04-24T08:57:22.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.454+0000 7faafe600640 1 --2- 172.21.15.83:0/739789294 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faaf8112ed0 0x7faaf81152c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:22.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.454+0000 7faafe600640 1 --2- 172.21.15.83:0/739789294 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faaf8074cc0 0x7faaf8112990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:22.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.454+0000 7faafe600640 1 --2- 172.21.15.83:0/739789294 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faaf80743a0 0x7faaf8074780 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:22.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.454+0000 7faafe600640 1 -- 172.21.15.83:0/739789294 >> 172.21.15.83:0/739789294 conn(0x7faaf80fd9d0 msgr2=0x7faaf80ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:22.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.454+0000 7faafe600640 1 -- 172.21.15.83:0/739789294 shutdown_connections 2024-04-24T08:57:22.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.454+0000 7faafe600640 1 -- 172.21.15.83:0/739789294 wait complete. 2024-04-24T08:57:22.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.455+0000 7faafe600640 1 Processor -- start 2024-04-24T08:57:22.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.455+0000 7faafe600640 1 -- start start 2024-04-24T08:57:22.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.456+0000 7faafe600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faaf80743a0 0x7faaf819ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:22.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.456+0000 7faafe600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faaf8074cc0 0x7faaf819f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:22.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.456+0000 7faafe600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faaf8112ed0 0x7faaf81a3620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:22.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.456+0000 7faafe600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7faaf81187a0 con 0x7faaf80743a0 2024-04-24T08:57:22.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.456+0000 7faafe600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7faaf8118620 con 0x7faaf8074cc0 2024-04-24T08:57:22.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.456+0000 7faafe600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7faaf8118920 con 0x7faaf8112ed0 2024-04-24T08:57:22.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.456+0000 7faafd200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faaf80743a0 0x7faaf819ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:22.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.456+0000 7faafd200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faaf80743a0 0x7faaf819ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:59106/0 (socket says 172.21.15.83:59106) 2024-04-24T08:57:22.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.456+0000 7faafd200640 1 -- 172.21.15.83:0/2729090204 learned_addr learned my addr 172.21.15.83:0/2729090204 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:57:22.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.456+0000 7faaf7e00640 1 --2- 172.21.15.83:0/2729090204 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faaf8074cc0 0x7faaf819f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:22.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.457+0000 7faafdc00640 1 --2- 172.21.15.83:0/2729090204 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faaf8112ed0 0x7faaf81a3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:22.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.457+0000 7faafd200640 1 -- 172.21.15.83:0/2729090204 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faaf8112ed0 msgr2=0x7faaf81a3620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:22.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.457+0000 7faafd200640 1 --2- 172.21.15.83:0/2729090204 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faaf8112ed0 0x7faaf81a3620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:22.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.457+0000 7faafd200640 1 -- 172.21.15.83:0/2729090204 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faaf8074cc0 msgr2=0x7faaf819f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:22.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.457+0000 7faafd200640 1 --2- 172.21.15.83:0/2729090204 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faaf8074cc0 0x7faaf819f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:22.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.457+0000 7faafd200640 1 -- 172.21.15.83:0/2729090204 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faaf81a3d50 con 0x7faaf80743a0 2024-04-24T08:57:22.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.457+0000 7faafd200640 1 --2- 172.21.15.83:0/2729090204 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faaf80743a0 0x7faaf819ed60 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7faaec005ab0 tx=0x7faaec003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:22.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.457+0000 7faaf5600640 1 -- 172.21.15.83:0/2729090204 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaec03f070 con 0x7faaf80743a0 2024-04-24T08:57:22.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.458+0000 7faafe600640 1 -- 172.21.15.83:0/2729090204 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faaf81a3fe0 con 0x7faaf80743a0 2024-04-24T08:57:22.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.458+0000 7faaf5600640 1 -- 172.21.15.83:0/2729090204 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faaec031d60 con 0x7faaf80743a0 2024-04-24T08:57:22.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.458+0000 7faaf5600640 1 -- 172.21.15.83:0/2729090204 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaec035030 con 0x7faaf80743a0 2024-04-24T08:57:22.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.458+0000 7faafe600640 1 -- 172.21.15.83:0/2729090204 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faaf81ab8d0 con 0x7faaf80743a0 2024-04-24T08:57:22.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.461+0000 7faaf5600640 1 -- 172.21.15.83:0/2729090204 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7faaec042070 con 0x7faaf80743a0 2024-04-24T08:57:22.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.461+0000 7fab02a27740 1 Processor -- start 2024-04-24T08:57:22.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.461+0000 7fab02a27740 1 -- start start 2024-04-24T08:57:22.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.462+0000 7fab02a27740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f461b7b40 0x556f461b7f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:22.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.462+0000 7fab02a27740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f461b8460 0x556f461c25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:22.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.462+0000 7fab02a27740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f461c2b30 0x556f461c4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:22.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.462+0000 7fab02a27740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556f4611d1d0 con 0x556f461b7b40 2024-04-24T08:57:22.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.462+0000 7fab02a27740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556f460c3450 con 0x556f461c2b30 2024-04-24T08:57:22.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.462+0000 7fab02a27740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x556f46119190 con 0x556f461b8460 2024-04-24T08:57:22.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.462+0000 7faafdc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f461b7b40 0x556f461b7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:22.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.462+0000 7faafdc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f461b7b40 0x556f461b7f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:59114/0 (socket says 172.21.15.83:59114) 2024-04-24T08:57:22.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.462+0000 7faafdc00640 1 -- 172.21.15.83:0/2059193771 learned_addr learned my addr 172.21.15.83:0/2059193771 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:57:22.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.462+0000 7faafd200640 1 --2- 172.21.15.83:0/2059193771 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f461b8460 0x556f461c25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:22.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.462+0000 7faaf7e00640 1 --2- 172.21.15.83:0/2059193771 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f461c2b30 0x556f461c4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:22.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.462+0000 7faafdc00640 1 -- 172.21.15.83:0/2059193771 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f461b8460 msgr2=0x556f461c25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:22.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.462+0000 7faafdc00640 1 --2- 172.21.15.83:0/2059193771 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f461b8460 0x556f461c25f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:22.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.462+0000 7faafdc00640 1 -- 172.21.15.83:0/2059193771 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f461c2b30 msgr2=0x556f461c4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:22.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.462+0000 7faafdc00640 1 --2- 172.21.15.83:0/2059193771 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f461c2b30 0x556f461c4f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:22.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.463+0000 7faafdc00640 1 -- 172.21.15.83:0/2059193771 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f46180310 con 0x556f461b7b40 2024-04-24T08:57:22.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.463+0000 7faaf7e00640 1 --2- 172.21.15.83:0/2059193771 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f461c2b30 0x556f461c4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:57:22.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.463+0000 7faafdc00640 1 --2- 172.21.15.83:0/2059193771 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f461b7b40 0x556f461b7f20 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7faaf0009ef0 tx=0x7faaf000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9e0d20026756f71 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:22.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.463+0000 7faaf5600640 1 --2- 172.21.15.83:0/2729090204 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7faae00a3d20 0x7faae00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:22.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.463+0000 7faaea000640 1 -- 172.21.15.83:0/2059193771 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf0015070 con 0x556f461b7b40 2024-04-24T08:57:22.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.463+0000 7faaea000640 1 -- 172.21.15.83:0/2059193771 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faaf00118f0 con 0x556f461b7b40 2024-04-24T08:57:22.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.463+0000 7faaea000640 1 -- 172.21.15.83:0/2059193771 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf00108c0 con 0x556f461b7b40 2024-04-24T08:57:22.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.463+0000 7faaf5600640 1 -- 172.21.15.83:0/2729090204 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7faaec0f1540 con 0x7faaf80743a0 2024-04-24T08:57:22.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.463+0000 7faafd200640 1 --2- 172.21.15.83:0/2729090204 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7faae00a3d20 0x7faae00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:22.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.464+0000 7fab02a27740 1 -- 172.21.15.83:0/2059193771 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f461b7b40 msgr2=0x556f461b7f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:22.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.464+0000 7fab02a27740 1 --2- 172.21.15.83:0/2059193771 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f461b7b40 0x556f461b7f20 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7faaf0009ef0 tx=0x7faaf000f730 comp rx=0 tx=0).stop 2024-04-24T08:57:22.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.464+0000 7faafd200640 1 --2- 172.21.15.83:0/2729090204 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7faae00a3d20 0x7faae00a61e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7faaec0ee970 tx=0x7faaec031400 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:22.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.464+0000 7fab02a27740 1 -- 172.21.15.83:0/2059193771 shutdown_connections 2024-04-24T08:57:22.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.464+0000 7fab02a27740 1 --2- 172.21.15.83:0/2059193771 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f461c2b30 0x556f461c4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:22.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.464+0000 7fab02a27740 1 --2- 172.21.15.83:0/2059193771 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f461b8460 0x556f461c25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:22.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.464+0000 7fab02a27740 1 --2- 172.21.15.83:0/2059193771 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f461b7b40 0x556f461b7f20 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:22.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.464+0000 7fab02a27740 1 -- 172.21.15.83:0/2059193771 >> 172.21.15.83:0/2059193771 conn(0x556f461b3e60 msgr2=0x556f461b4240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:22.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.464+0000 7fab02a27740 1 -- 172.21.15.83:0/2059193771 shutdown_connections 2024-04-24T08:57:22.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.464+0000 7fab02a27740 1 -- 172.21.15.83:0/2059193771 wait complete. 2024-04-24T08:57:22.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.465+0000 7fab02a27740 1 -- start start 2024-04-24T08:57:22.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.465+0000 7fab02a27740 1 Processor -- start 2024-04-24T08:57:22.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.465+0000 7fab02a27740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f461b7b40 0x556f461bf910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:22.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.465+0000 7fab02a27740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f461b8460 0x556f461bfe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:22.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.465+0000 7fab02a27740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f461c2b30 0x556f461c0390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:22.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.466+0000 7fab02a27740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556f461c59c0 con 0x556f461b8460 2024-04-24T08:57:22.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.466+0000 7fab02a27740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556f460d6810 con 0x556f461c2b30 2024-04-24T08:57:22.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.466+0000 7fab02a27740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x556f461c5b40 con 0x556f461b7b40 2024-04-24T08:57:22.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.466+0000 7fab02a27740 10 client.0 _ll_register_callbacks cb 0x556f460a4140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:57:22.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.466+0000 7faaf7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f461b8460 0x556f461bfe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:22.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.466+0000 7faaf7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f461b8460 0x556f461bfe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:59116/0 (socket says 172.21.15.83:59116) 2024-04-24T08:57:22.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.466+0000 7faaf7e00640 1 -- 172.21.15.83:0/458006073 learned_addr learned my addr 172.21.15.83:0/458006073 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:57:22.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.466+0000 7faafdc00640 1 --2- 172.21.15.83:0/458006073 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f461b7b40 0x556f461bf910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:22.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.466+0000 7faafdc00640 1 --2- 172.21.15.83:0/458006073 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f461c2b30 0x556f461c0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:22.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.466+0000 7faaf7e00640 1 -- 172.21.15.83:0/458006073 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f461b7b40 msgr2=0x556f461bf910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:22.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.466+0000 7faaf7e00640 1 --2- 172.21.15.83:0/458006073 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f461b7b40 0x556f461bf910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:22.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.466+0000 7faaf7e00640 1 -- 172.21.15.83:0/458006073 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f461c2b30 msgr2=0x556f461c0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:22.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.466+0000 7faaf7e00640 1 --2- 172.21.15.83:0/458006073 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f461c2b30 0x556f461c0390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:22.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.466+0000 7faaf7e00640 1 -- 172.21.15.83:0/458006073 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f460e73f0 con 0x556f461b8460 2024-04-24T08:57:22.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.467+0000 7faaf7e00640 1 --2- 172.21.15.83:0/458006073 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f461b8460 0x556f461bfe50 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7faae4010470 tx=0x7faae4010930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e626cbd661a3129 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:22.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.467+0000 7faadea00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:57:22.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.467+0000 7faadea00640 1 -- 172.21.15.83:0/458006073 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae4012920 con 0x556f461b8460 2024-04-24T08:57:22.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.467+0000 7faadea00640 1 -- 172.21.15.83:0/458006073 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7faae4012ac0 con 0x556f461b8460 2024-04-24T08:57:22.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.467+0000 7fab02a27740 1 -- 172.21.15.83:0/458006073 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faaf00118f0 con 0x556f461b8460 2024-04-24T08:57:22.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.467+0000 7faadea00640 1 -- 172.21.15.83:0/458006073 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7faae4025050 con 0x556f461b8460 2024-04-24T08:57:22.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.467+0000 7fab02a27740 10 client.17751 fetch_fsmap learned FSMap version 33 2024-04-24T08:57:22.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.467+0000 7fab02a27740 1 -- 172.21.15.83:0/458006073 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x556f45e2f3f0 con 0x556f461b8460 2024-04-24T08:57:22.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.468+0000 7faadea00640 1 -- 172.21.15.83:0/458006073 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faaf00118f0 con 0x556f461b8460 2024-04-24T08:57:22.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.468+0000 7faadea00640 1 -- 172.21.15.83:0/458006073 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7faae400f040 con 0x556f461b8460 2024-04-24T08:57:22.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.468+0000 7fab02a27740 10 client.17751 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T08:57:22.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.468+0000 7fab02a27740 10 client.17751 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:57:22.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.468+0000 7fab02a27740 20 client.17751 populate_metadata read hostname 'smithi083' 2024-04-24T08:57:22.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.468+0000 7fab02a27740 1 --2- 172.21.15.83:0/458006073 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556f461d2e40 0x556f461d5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:22.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.468+0000 7fab02a27740 4 client.17751 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:57:22.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.468+0000 7fab02a27740 1 -- 172.21.15.83:0/458006073 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556f4612abf0 con 0x556f461d2e40 2024-04-24T08:57:22.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.469+0000 7faafdc00640 1 --2- 172.21.15.83:0/458006073 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556f461d2e40 0x556f461d5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:22.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.470+0000 7faafdc00640 1 --2- 172.21.15.83:0/458006073 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556f461d2e40 0x556f461d5260 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2de517b8ff5d38da server_cookie=89ef5a6f5a798bc3 in_seq=0 out_seq=0 2024-04-24T08:57:22.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.470+0000 7faadea00640 10 client.17751 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:57:22.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.471+0000 7faadea00640 1 -- 172.21.15.83:0/458006073 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556f4612abf0 con 0x556f461d2e40 2024-04-24T08:57:22.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.471+0000 7faadea00640 10 client.17751 handle_command_reply: tid=0 2024-04-24T08:57:22.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.471+0000 7fab02a27740 1 client.17751 shutdown 2024-04-24T08:57:22.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.472+0000 7fab02a27740 1 -- 172.21.15.83:0/458006073 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f461b8460 msgr2=0x556f461bfe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:22.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.472+0000 7fab02a27740 1 --2- 172.21.15.83:0/458006073 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f461b8460 0x556f461bfe50 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7faae4010470 tx=0x7faae4010930 comp rx=0 tx=0).stop 2024-04-24T08:57:22.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.472+0000 7fab02a27740 1 -- 172.21.15.83:0/458006073 shutdown_connections 2024-04-24T08:57:22.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.472+0000 7fab02a27740 1 --2- 172.21.15.83:0/458006073 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556f461d2e40 0x556f461d5260 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:22.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.472+0000 7fab02a27740 1 --2- 172.21.15.83:0/458006073 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f461c2b30 0x556f461c0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:22.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.472+0000 7fab02a27740 1 --2- 172.21.15.83:0/458006073 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f461b8460 0x556f461bfe50 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:22.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.472+0000 7fab02a27740 1 --2- 172.21.15.83:0/458006073 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f461b7b40 0x556f461bf910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:22.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.472+0000 7fab02a27740 1 -- 172.21.15.83:0/458006073 >> 172.21.15.83:0/458006073 conn(0x556f461b3e60 msgr2=0x556f461b9b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:22.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.472+0000 7faadea00640 0 client.17751 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:57:22.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.472+0000 7fab02a27740 1 -- 172.21.15.83:0/458006073 shutdown_connections 2024-04-24T08:57:22.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.472+0000 7fab02a27740 1 -- 172.21.15.83:0/458006073 wait complete. 2024-04-24T08:57:22.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.472+0000 7fab02a27740 20 client.17751 trim_cache size 0 max 16384 2024-04-24T08:57:22.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.487+0000 7fab02a27740 1 Processor -- start 2024-04-24T08:57:22.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.487+0000 7fab02a27740 1 -- start start 2024-04-24T08:57:22.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.488+0000 7fab02a27740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f461b7b40 0x556f461ce8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:22.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.488+0000 7fab02a27740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f461b8460 0x556f461c01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:22.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.488+0000 7fab02a27740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f461c2b30 0x556f461dab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:22.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.488+0000 7fab02a27740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7faaf81b5980 con 0x556f461b7b40 2024-04-24T08:57:22.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.488+0000 7fab02a27740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7faaf81b74c0 con 0x556f461b8460 2024-04-24T08:57:22.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.488+0000 7fab02a27740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7faaf81b6050 con 0x556f461c2b30 2024-04-24T08:57:22.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.488+0000 7faafdc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f461b7b40 0x556f461ce8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:22.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.488+0000 7faafdc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f461b7b40 0x556f461ce8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:59124/0 (socket says 172.21.15.83:59124) 2024-04-24T08:57:22.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.488+0000 7faafdc00640 1 -- 172.21.15.83:0/1766981710 learned_addr learned my addr 172.21.15.83:0/1766981710 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:57:22.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.488+0000 7faaf7e00640 1 --2- 172.21.15.83:0/1766981710 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f461b8460 0x556f461c01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:22.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.488+0000 7faafdc00640 1 -- 172.21.15.83:0/1766981710 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f461c2b30 msgr2=0x556f461dab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:22.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.488+0000 7faafdc00640 1 --2- 172.21.15.83:0/1766981710 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f461c2b30 0x556f461dab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:22.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.488+0000 7faafdc00640 1 -- 172.21.15.83:0/1766981710 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f461b8460 msgr2=0x556f461c01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:22.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.488+0000 7faafdc00640 1 --2- 172.21.15.83:0/1766981710 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f461b8460 0x556f461c01b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:22.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.488+0000 7faafdc00640 1 -- 172.21.15.83:0/1766981710 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f461b8290 con 0x556f461b7b40 2024-04-24T08:57:22.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.489+0000 7faafdc00640 1 --2- 172.21.15.83:0/1766981710 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f461b7b40 0x556f461ce8b0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7faaf000fc30 tx=0x7faaf0011560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1977faabc21d9ba server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:22.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.489+0000 7faae9600640 1 -- 172.21.15.83:0/1766981710 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf0015040 con 0x556f461b7b40 2024-04-24T08:57:22.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.489+0000 7faae9600640 1 -- 172.21.15.83:0/1766981710 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faaf00288c0 con 0x556f461b7b40 2024-04-24T08:57:22.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.489+0000 7faae9600640 1 -- 172.21.15.83:0/1766981710 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf0011dc0 con 0x556f461b7b40 2024-04-24T08:57:22.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.490+0000 7fab02a27740 1 -- 172.21.15.83:0/1766981710 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f461b7b40 msgr2=0x556f461ce8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:22.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.490+0000 7fab02a27740 1 --2- 172.21.15.83:0/1766981710 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f461b7b40 0x556f461ce8b0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7faaf000fc30 tx=0x7faaf0011560 comp rx=0 tx=0).stop 2024-04-24T08:57:22.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.490+0000 7fab02a27740 1 -- 172.21.15.83:0/1766981710 shutdown_connections 2024-04-24T08:57:22.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.490+0000 7fab02a27740 1 --2- 172.21.15.83:0/1766981710 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f461c2b30 0x556f461dab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:22.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.490+0000 7fab02a27740 1 --2- 172.21.15.83:0/1766981710 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f461b8460 0x556f461c01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:22.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.490+0000 7fab02a27740 1 --2- 172.21.15.83:0/1766981710 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f461b7b40 0x556f461ce8b0 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:22.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.490+0000 7fab02a27740 1 -- 172.21.15.83:0/1766981710 >> 172.21.15.83:0/1766981710 conn(0x556f461b3e60 msgr2=0x556f461b9290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:22.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.490+0000 7fab02a27740 1 -- 172.21.15.83:0/1766981710 shutdown_connections 2024-04-24T08:57:22.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.490+0000 7fab02a27740 1 -- 172.21.15.83:0/1766981710 wait complete. 2024-04-24T08:57:22.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.491+0000 7fab02a27740 1 -- start start 2024-04-24T08:57:22.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.491+0000 7fab02a27740 1 Processor -- start 2024-04-24T08:57:22.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.491+0000 7fab02a27740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f461b7b40 0x556f461cb1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:22.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.491+0000 7fab02a27740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f461b8460 0x556f461c3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:22.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.491+0000 7fab02a27740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f461c2b30 0x556f461c3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:22.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.491+0000 7fab02a27740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7faaf81b6df0 con 0x556f461b7b40 2024-04-24T08:57:22.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.491+0000 7fab02a27740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7faaf81b6720 con 0x556f461b8460 2024-04-24T08:57:22.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.491+0000 7fab02a27740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7faaf81b3960 con 0x556f461c2b30 2024-04-24T08:57:22.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.491+0000 7fab02a27740 10 client.0 _ll_register_callbacks cb 0x7faae4005f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:57:22.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.492+0000 7faafdc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f461b7b40 0x556f461cb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:22.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.492+0000 7faafdc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f461b7b40 0x556f461cb1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:59128/0 (socket says 172.21.15.83:59128) 2024-04-24T08:57:22.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.492+0000 7faafdc00640 1 -- 172.21.15.83:0/1639923393 learned_addr learned my addr 172.21.15.83:0/1639923393 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:57:22.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.492+0000 7faafdc00640 1 -- 172.21.15.83:0/1639923393 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f461c2b30 msgr2=0x556f461c3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:22.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.492+0000 7faafdc00640 1 --2- 172.21.15.83:0/1639923393 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f461c2b30 0x556f461c3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:22.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.492+0000 7faafdc00640 1 -- 172.21.15.83:0/1639923393 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f461b8460 msgr2=0x556f461c3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:22.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.492+0000 7faafdc00640 1 --2- 172.21.15.83:0/1639923393 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f461b8460 0x556f461c3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:22.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.492+0000 7faafdc00640 1 -- 172.21.15.83:0/1639923393 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f45ef97d0 con 0x556f461b7b40 2024-04-24T08:57:22.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.492+0000 7faafdc00640 1 --2- 172.21.15.83:0/1639923393 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f461b7b40 0x556f461cb1e0 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7faaf0005fd0 tx=0x7faaf0010460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8e409b5a2e5bb74 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:22.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.492+0000 7faadea00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:57:22.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.492+0000 7faadea00640 1 -- 172.21.15.83:0/1639923393 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf0015070 con 0x556f461b7b40 2024-04-24T08:57:22.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.493+0000 7faadea00640 1 -- 172.21.15.83:0/1639923393 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7faaf0003bb0 con 0x556f461b7b40 2024-04-24T08:57:22.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.493+0000 7fab02a27740 1 -- 172.21.15.83:0/1639923393 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faaf00288c0 con 0x556f461b7b40 2024-04-24T08:57:22.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.493+0000 7faadea00640 1 -- 172.21.15.83:0/1639923393 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7faaf0003db0 con 0x556f461b7b40 2024-04-24T08:57:22.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.493+0000 7fab02a27740 10 client.17757 fetch_fsmap learned FSMap version 33 2024-04-24T08:57:22.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.493+0000 7fab02a27740 1 -- 172.21.15.83:0/1639923393 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x556f46239380 con 0x556f461b7b40 2024-04-24T08:57:22.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.493+0000 7faadea00640 1 -- 172.21.15.83:0/1639923393 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faaf00288c0 con 0x556f461b7b40 2024-04-24T08:57:22.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.493+0000 7faadea00640 1 -- 172.21.15.83:0/1639923393 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7faaf0010a10 con 0x556f461b7b40 2024-04-24T08:57:22.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.493+0000 7fab02a27740 10 client.17757 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T08:57:22.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.493+0000 7fab02a27740 10 client.17757 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:57:22.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.493+0000 7fab02a27740 20 client.17757 populate_metadata read hostname 'smithi083' 2024-04-24T08:57:22.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.493+0000 7fab02a27740 1 --2- 172.21.15.83:0/1639923393 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556f462397b0 0x556f4623bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:22.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.493+0000 7fab02a27740 4 client.17757 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T08:57:22.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.493+0000 7fab02a27740 1 -- 172.21.15.83:0/1639923393 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556f46124a60 con 0x556f462397b0 2024-04-24T08:57:22.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.494+0000 7faaf7e00640 1 --2- 172.21.15.83:0/1639923393 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556f462397b0 0x556f4623bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:22.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.495+0000 7faaf7e00640 1 --2- 172.21.15.83:0/1639923393 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556f462397b0 0x556f4623bbd0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f3e5bb1f0f2f927 server_cookie=ed1761c17fcc0bc4 in_seq=0 out_seq=0 2024-04-24T08:57:22.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.495+0000 7faadea00640 10 client.17757 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:57:22.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.496+0000 7faadea00640 1 -- 172.21.15.83:0/1639923393 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556f46124a60 con 0x556f462397b0 2024-04-24T08:57:22.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.496+0000 7faadea00640 10 client.17757 handle_command_reply: tid=0 2024-04-24T08:57:22.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.496+0000 7fab02a27740 1 client.17757 shutdown 2024-04-24T08:57:22.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.497+0000 7fab02a27740 1 -- 172.21.15.83:0/1639923393 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f461b7b40 msgr2=0x556f461cb1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:22.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.497+0000 7fab02a27740 1 --2- 172.21.15.83:0/1639923393 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f461b7b40 0x556f461cb1e0 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7faaf0005fd0 tx=0x7faaf0010460 comp rx=0 tx=0).stop 2024-04-24T08:57:22.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.497+0000 7fab02a27740 1 -- 172.21.15.83:0/1639923393 shutdown_connections 2024-04-24T08:57:22.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.497+0000 7fab02a27740 1 --2- 172.21.15.83:0/1639923393 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556f462397b0 0x556f4623bbd0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:22.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.497+0000 7fab02a27740 1 --2- 172.21.15.83:0/1639923393 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f461c2b30 0x556f461c3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:22.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.497+0000 7fab02a27740 1 --2- 172.21.15.83:0/1639923393 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f461b8460 0x556f461c3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:22.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.497+0000 7fab02a27740 1 --2- 172.21.15.83:0/1639923393 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f461b7b40 0x556f461cb1e0 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:22.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.497+0000 7faadea00640 0 client.17757 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:57:22.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.497+0000 7fab02a27740 1 -- 172.21.15.83:0/1639923393 >> 172.21.15.83:0/1639923393 conn(0x556f461b3e60 msgr2=0x556f461b2f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:22.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.497+0000 7fab02a27740 1 -- 172.21.15.83:0/1639923393 shutdown_connections 2024-04-24T08:57:22.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.497+0000 7fab02a27740 1 -- 172.21.15.83:0/1639923393 wait complete. 2024-04-24T08:57:22.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.497+0000 7fab02a27740 20 client.17757 trim_cache size 0 max 16384 2024-04-24T08:57:22.497 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T08:57:22.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.498+0000 7faafe600640 1 -- 172.21.15.83:0/2729090204 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7faae00a3d20 msgr2=0x7faae00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:22.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.498+0000 7faafe600640 1 --2- 172.21.15.83:0/2729090204 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7faae00a3d20 0x7faae00a61e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7faaec0ee970 tx=0x7faaec031400 comp rx=0 tx=0).stop 2024-04-24T08:57:22.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.498+0000 7faafe600640 1 -- 172.21.15.83:0/2729090204 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faaf80743a0 msgr2=0x7faaf819ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:22.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.498+0000 7faafe600640 1 --2- 172.21.15.83:0/2729090204 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faaf80743a0 0x7faaf819ed60 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7faaec005ab0 tx=0x7faaec003cd0 comp rx=0 tx=0).stop 2024-04-24T08:57:22.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.498+0000 7faafe600640 1 -- 172.21.15.83:0/2729090204 shutdown_connections 2024-04-24T08:57:22.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.498+0000 7faafe600640 1 --2- 172.21.15.83:0/2729090204 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7faae00a3d20 0x7faae00a61e0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:22.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.498+0000 7faafe600640 1 --2- 172.21.15.83:0/2729090204 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faaf8112ed0 0x7faaf81a3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:22.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.498+0000 7faafe600640 1 --2- 172.21.15.83:0/2729090204 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faaf8074cc0 0x7faaf819f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:22.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.498+0000 7faafe600640 1 --2- 172.21.15.83:0/2729090204 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faaf80743a0 0x7faaf819ed60 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:22.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.498+0000 7faafe600640 1 -- 172.21.15.83:0/2729090204 >> 172.21.15.83:0/2729090204 conn(0x7faaf80fd9d0 msgr2=0x7faaf80ff2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:22.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.498+0000 7faafe600640 1 -- 172.21.15.83:0/2729090204 shutdown_connections 2024-04-24T08:57:22.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:22.498+0000 7faafe600640 1 -- 172.21.15.83:0/2729090204 wait complete. 2024-04-24T08:57:22.580 INFO:tasks.workunit.client.0.smithi083.stdout: 1 28786 0.66 MB/sec execute 116 sec latency 46.243 ms 2024-04-24T08:57:23.517 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T08:57:23.517 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T08:57:23.517 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T08:57:23.579 INFO:tasks.workunit.client.0.smithi083.stdout: 1 28842 0.65 MB/sec execute 117 sec latency 52.564 ms 2024-04-24T08:57:23.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.733+0000 7f93c3200640 1 -- 172.21.15.83:0/2064231755 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f93c4104320 msgr2=0x7f93c4104700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:23.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.733+0000 7f93c3200640 1 --2- 172.21.15.83:0/2064231755 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f93c4104320 0x7f93c4104700 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f93b4009900 tx=0x7f93b402a3f0 comp rx=0 tx=0).stop 2024-04-24T08:57:23.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.734+0000 7f93c3200640 1 -- 172.21.15.83:0/2064231755 shutdown_connections 2024-04-24T08:57:23.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.734+0000 7f93c3200640 1 --2- 172.21.15.83:0/2064231755 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f93c41093f0 0x7f93c4110fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:23.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.734+0000 7f93c3200640 1 --2- 172.21.15.83:0/2064231755 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f93c4104cd0 0x7f93c4108d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:23.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.734+0000 7f93c3200640 1 --2- 172.21.15.83:0/2064231755 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f93c4104320 0x7f93c4104700 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:23.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.734+0000 7f93c3200640 1 -- 172.21.15.83:0/2064231755 >> 172.21.15.83:0/2064231755 conn(0x7f93c40ffb40 msgr2=0x7f93c4101f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:23.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.734+0000 7f93c3200640 1 -- 172.21.15.83:0/2064231755 shutdown_connections 2024-04-24T08:57:23.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.734+0000 7f93c3200640 1 -- 172.21.15.83:0/2064231755 wait complete. 2024-04-24T08:57:23.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.736+0000 7f93c3200640 1 Processor -- start 2024-04-24T08:57:23.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.736+0000 7f93c3200640 1 -- start start 2024-04-24T08:57:23.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.736+0000 7f93c3200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f93c4104320 0x7f93c41a05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:23.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.736+0000 7f93c3200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f93c41093f0 0x7f93c41a0b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:23.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.736+0000 7f93c3200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f93c419a6a0 0x7f93c419ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:23.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.736+0000 7f93c3200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f93c4114380 con 0x7f93c41093f0 2024-04-24T08:57:23.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.737+0000 7f93c3200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f93c4114200 con 0x7f93c4104320 2024-04-24T08:57:23.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.737+0000 7f93c3200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f93c4114500 con 0x7f93c419a6a0 2024-04-24T08:57:23.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.737+0000 7f93c1400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f93c41093f0 0x7f93c41a0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:23.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.737+0000 7f93c1400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f93c41093f0 0x7f93c41a0b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:59136/0 (socket says 172.21.15.83:59136) 2024-04-24T08:57:23.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.737+0000 7f93c1400640 1 -- 172.21.15.83:0/22195001 learned_addr learned my addr 172.21.15.83:0/22195001 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:57:23.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.737+0000 7f93c1e00640 1 --2- 172.21.15.83:0/22195001 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f93c4104320 0x7f93c41a05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:23.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.737+0000 7f93c2800640 1 --2- 172.21.15.83:0/22195001 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f93c419a6a0 0x7f93c419ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:23.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.737+0000 7f93c1400640 1 -- 172.21.15.83:0/22195001 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f93c419a6a0 msgr2=0x7f93c419ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:23.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.737+0000 7f93c1400640 1 --2- 172.21.15.83:0/22195001 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f93c419a6a0 0x7f93c419ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:23.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.737+0000 7f93c1400640 1 -- 172.21.15.83:0/22195001 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f93c4104320 msgr2=0x7f93c41a05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:23.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.737+0000 7f93c1400640 1 --2- 172.21.15.83:0/22195001 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f93c4104320 0x7f93c41a05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:23.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.737+0000 7f93c1400640 1 -- 172.21.15.83:0/22195001 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93c419b2a0 con 0x7f93c41093f0 2024-04-24T08:57:23.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.737+0000 7f93c2800640 1 --2- 172.21.15.83:0/22195001 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f93c419a6a0 0x7f93c419ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:57:23.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.738+0000 7f93c1400640 1 --2- 172.21.15.83:0/22195001 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f93c41093f0 0x7f93c41a0b20 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f93b000c970 tx=0x7f93b000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:23.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.738+0000 7f93baa00640 1 -- 172.21.15.83:0/22195001 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b0014070 con 0x7f93c41093f0 2024-04-24T08:57:23.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.738+0000 7f93baa00640 1 -- 172.21.15.83:0/22195001 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f93b0004440 con 0x7f93c41093f0 2024-04-24T08:57:23.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.738+0000 7f93c3200640 1 -- 172.21.15.83:0/22195001 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93c41a7380 con 0x7f93c41093f0 2024-04-24T08:57:23.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.738+0000 7f93baa00640 1 -- 172.21.15.83:0/22195001 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b0003e50 con 0x7f93c41093f0 2024-04-24T08:57:23.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.738+0000 7f93c3200640 1 -- 172.21.15.83:0/22195001 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93c41a7780 con 0x7f93c41093f0 2024-04-24T08:57:23.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.742+0000 7f93cb99a740 1 Processor -- start 2024-04-24T08:57:23.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.742+0000 7f93baa00640 1 -- 172.21.15.83:0/22195001 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f93b0017070 con 0x7f93c41093f0 2024-04-24T08:57:23.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.742+0000 7f93cb99a740 1 -- start start 2024-04-24T08:57:23.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.742+0000 7f93cb99a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56533e80d270 0x56533e80d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:23.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.743+0000 7f93cb99a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56533e80db90 0x56533e817d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:23.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.743+0000 7f93cb99a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56533e818260 0x56533e81a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:23.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.743+0000 7f93cb99a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56533e772e70 con 0x56533e80db90 2024-04-24T08:57:23.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.743+0000 7f93cb99a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56533e719380 con 0x56533e80d270 2024-04-24T08:57:23.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.743+0000 7f93cb99a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56533e76ee30 con 0x56533e818260 2024-04-24T08:57:23.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.743+0000 7f93c1e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56533e80db90 0x56533e817d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:23.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.743+0000 7f93c1e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56533e80db90 0x56533e817d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:59138/0 (socket says 172.21.15.83:59138) 2024-04-24T08:57:23.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.743+0000 7f93c1e00640 1 -- 172.21.15.83:0/1841716867 learned_addr learned my addr 172.21.15.83:0/1841716867 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:57:23.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.743+0000 7f93c2800640 1 --2- 172.21.15.83:0/1841716867 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56533e80d270 0x56533e80d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:23.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.743+0000 7f93c1400640 1 --2- 172.21.15.83:0/1841716867 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56533e818260 0x56533e81a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:23.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.743+0000 7f93c1e00640 1 -- 172.21.15.83:0/1841716867 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56533e818260 msgr2=0x56533e81a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:23.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.743+0000 7f93c1e00640 1 --2- 172.21.15.83:0/1841716867 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56533e818260 0x56533e81a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:23.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.743+0000 7f93c1e00640 1 -- 172.21.15.83:0/1841716867 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56533e80d270 msgr2=0x56533e80d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:23.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.743+0000 7f93c1e00640 1 --2- 172.21.15.83:0/1841716867 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56533e80d270 0x56533e80d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:23.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.743+0000 7f93c1e00640 1 -- 172.21.15.83:0/1841716867 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56533e54d7e0 con 0x56533e80db90 2024-04-24T08:57:23.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.743+0000 7f93c1400640 1 --2- 172.21.15.83:0/1841716867 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56533e818260 0x56533e81a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:57:23.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.743+0000 7f93c2800640 1 --2- 172.21.15.83:0/1841716867 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56533e80d270 0x56533e80d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:57:23.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.744+0000 7f93baa00640 1 --2- 172.21.15.83:0/22195001 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f93a80a3ec0 0x7f93a80a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:23.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.744+0000 7f93c1e00640 1 --2- 172.21.15.83:0/1841716867 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56533e80db90 0x56533e817d20 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f93b4009230 tx=0x7f93b4004390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a47c4c17a6e0bb34 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:23.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.744+0000 7f93baa00640 1 -- 172.21.15.83:0/22195001 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f93b00d07f0 con 0x7f93c41093f0 2024-04-24T08:57:23.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.744+0000 7f93af000640 1 -- 172.21.15.83:0/1841716867 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b4035030 con 0x56533e80db90 2024-04-24T08:57:23.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.744+0000 7f93c2800640 1 --2- 172.21.15.83:0/22195001 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f93a80a3ec0 0x7f93a80a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:23.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.744+0000 7f93af000640 1 -- 172.21.15.83:0/1841716867 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f93b40046c0 con 0x56533e80db90 2024-04-24T08:57:23.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.744+0000 7f93af000640 1 -- 172.21.15.83:0/1841716867 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b40037f0 con 0x56533e80db90 2024-04-24T08:57:23.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.745+0000 7f93cb99a740 1 -- 172.21.15.83:0/1841716867 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56533e80db90 msgr2=0x56533e817d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:23.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.745+0000 7f93c2800640 1 --2- 172.21.15.83:0/22195001 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f93a80a3ec0 0x7f93a80a6380 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f93bc005fd0 tx=0x7f93bc009530 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:23.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.745+0000 7f93cb99a740 1 --2- 172.21.15.83:0/1841716867 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56533e80db90 0x56533e817d20 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f93b4009230 tx=0x7f93b4004390 comp rx=0 tx=0).stop 2024-04-24T08:57:23.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.745+0000 7f93cb99a740 1 -- 172.21.15.83:0/1841716867 shutdown_connections 2024-04-24T08:57:23.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.745+0000 7f93cb99a740 1 --2- 172.21.15.83:0/1841716867 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56533e818260 0x56533e81a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:23.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.745+0000 7f93cb99a740 1 --2- 172.21.15.83:0/1841716867 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56533e80db90 0x56533e817d20 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:23.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.745+0000 7f93cb99a740 1 --2- 172.21.15.83:0/1841716867 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56533e80d270 0x56533e80d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:23.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.745+0000 7f93cb99a740 1 -- 172.21.15.83:0/1841716867 >> 172.21.15.83:0/1841716867 conn(0x56533e809590 msgr2=0x56533e809970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:23.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.745+0000 7f93cb99a740 1 -- 172.21.15.83:0/1841716867 shutdown_connections 2024-04-24T08:57:23.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.745+0000 7f93cb99a740 1 -- 172.21.15.83:0/1841716867 wait complete. 2024-04-24T08:57:23.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.746+0000 7f93cb99a740 1 -- start start 2024-04-24T08:57:23.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.746+0000 7f93cb99a740 1 Processor -- start 2024-04-24T08:57:23.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.746+0000 7f93cb99a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56533e80d270 0x56533e8151e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:23.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.746+0000 7f93cb99a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56533e80db90 0x56533e815720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:23.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.746+0000 7f93cb99a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56533e818260 0x56533e815c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:23.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.746+0000 7f93cb99a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56533e81aff0 con 0x56533e80db90 2024-04-24T08:57:23.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.746+0000 7f93cb99a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56533e72c660 con 0x56533e80d270 2024-04-24T08:57:23.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.746+0000 7f93cb99a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56533e81b170 con 0x56533e818260 2024-04-24T08:57:23.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.747+0000 7f93cb99a740 10 client.0 _ll_register_callbacks cb 0x56533e71aaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:57:23.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.747+0000 7f93c1400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56533e80db90 0x56533e815720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:23.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.747+0000 7f93c1400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56533e80db90 0x56533e815720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:59150/0 (socket says 172.21.15.83:59150) 2024-04-24T08:57:23.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.747+0000 7f93c1400640 1 -- 172.21.15.83:0/2893696268 learned_addr learned my addr 172.21.15.83:0/2893696268 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:57:23.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.747+0000 7f93c1400640 1 -- 172.21.15.83:0/2893696268 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56533e818260 msgr2=0x56533e815c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:23.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.747+0000 7f93c1400640 1 --2- 172.21.15.83:0/2893696268 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56533e818260 0x56533e815c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:23.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.747+0000 7f93c1400640 1 -- 172.21.15.83:0/2893696268 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56533e80d270 msgr2=0x56533e8151e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:23.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.747+0000 7f93c1e00640 1 --2- 172.21.15.83:0/2893696268 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56533e80d270 0x56533e8151e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:23.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.747+0000 7f93c1400640 1 --2- 172.21.15.83:0/2893696268 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56533e80d270 0x56533e8151e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:23.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.747+0000 7f93c1400640 1 -- 172.21.15.83:0/2893696268 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56533e5adf70 con 0x56533e80db90 2024-04-24T08:57:23.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.747+0000 7f93c1400640 1 --2- 172.21.15.83:0/2893696268 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56533e80db90 0x56533e815720 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f93b0098820 tx=0x7f93b009c280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=392687a0bc8b5576 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:23.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.747+0000 7f93a3400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:57:23.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.748+0000 7f93a3400640 1 -- 172.21.15.83:0/2893696268 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b00b0e60 con 0x56533e80db90 2024-04-24T08:57:23.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.748+0000 7f93a3400640 1 -- 172.21.15.83:0/2893696268 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f93b00d03c0 con 0x56533e80db90 2024-04-24T08:57:23.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.748+0000 7f93cb99a740 1 -- 172.21.15.83:0/2893696268 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93b40046c0 con 0x56533e80db90 2024-04-24T08:57:23.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.748+0000 7f93a3400640 1 -- 172.21.15.83:0/2893696268 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f93b009c3c0 con 0x56533e80db90 2024-04-24T08:57:23.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.748+0000 7f93cb99a740 10 client.17769 fetch_fsmap learned FSMap version 33 2024-04-24T08:57:23.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.748+0000 7f93cb99a740 1 -- 172.21.15.83:0/2893696268 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56533e445170 con 0x56533e80db90 2024-04-24T08:57:23.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.748+0000 7f93a3400640 1 -- 172.21.15.83:0/2893696268 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f93b40046c0 con 0x56533e80db90 2024-04-24T08:57:23.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.748+0000 7f93a3400640 1 -- 172.21.15.83:0/2893696268 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f93b0098110 con 0x56533e80db90 2024-04-24T08:57:23.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.749+0000 7f93cb99a740 10 client.17769 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T08:57:23.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.749+0000 7f93cb99a740 10 client.17769 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:57:23.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.749+0000 7f93cb99a740 20 client.17769 populate_metadata read hostname 'smithi083' 2024-04-24T08:57:23.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.749+0000 7f93cb99a740 1 --2- 172.21.15.83:0/2893696268 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56533e828580 0x56533e82a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:23.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.749+0000 7f93cb99a740 4 client.17769 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:57:23.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.749+0000 7f93cb99a740 1 -- 172.21.15.83:0/2893696268 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56533e780890 con 0x56533e828580 2024-04-24T08:57:23.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.749+0000 7f93c1e00640 1 --2- 172.21.15.83:0/2893696268 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56533e828580 0x56533e82a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:23.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.750+0000 7f93c1e00640 1 --2- 172.21.15.83:0/2893696268 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56533e828580 0x56533e82a9b0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e470be30b9ed87fa server_cookie=26ea569baa0f46aa in_seq=0 out_seq=0 2024-04-24T08:57:23.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.750+0000 7f93a3400640 10 client.17769 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:57:23.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.752+0000 7f93a3400640 1 -- 172.21.15.83:0/2893696268 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56533e780890 con 0x56533e828580 2024-04-24T08:57:23.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.752+0000 7f93a3400640 10 client.17769 handle_command_reply: tid=0 2024-04-24T08:57:23.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.752+0000 7f93cb99a740 1 client.17769 shutdown 2024-04-24T08:57:23.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.752+0000 7f93cb99a740 1 -- 172.21.15.83:0/2893696268 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56533e80db90 msgr2=0x56533e815720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:23.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.752+0000 7f93cb99a740 1 --2- 172.21.15.83:0/2893696268 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56533e80db90 0x56533e815720 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f93b0098820 tx=0x7f93b009c280 comp rx=0 tx=0).stop 2024-04-24T08:57:23.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.752+0000 7f93cb99a740 1 -- 172.21.15.83:0/2893696268 shutdown_connections 2024-04-24T08:57:23.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.752+0000 7f93cb99a740 1 --2- 172.21.15.83:0/2893696268 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56533e828580 0x56533e82a9b0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:23.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.752+0000 7f93cb99a740 1 --2- 172.21.15.83:0/2893696268 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56533e818260 0x56533e815c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:23.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.752+0000 7f93cb99a740 1 --2- 172.21.15.83:0/2893696268 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56533e80db90 0x56533e815720 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:23.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.752+0000 7f93cb99a740 1 --2- 172.21.15.83:0/2893696268 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56533e80d270 0x56533e8151e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:23.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.752+0000 7f93cb99a740 1 -- 172.21.15.83:0/2893696268 >> 172.21.15.83:0/2893696268 conn(0x56533e809590 msgr2=0x56533e80f290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:23.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.752+0000 7f93a3400640 0 client.17769 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:57:23.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.752+0000 7f93cb99a740 1 -- 172.21.15.83:0/2893696268 shutdown_connections 2024-04-24T08:57:23.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.752+0000 7f93cb99a740 1 -- 172.21.15.83:0/2893696268 wait complete. 2024-04-24T08:57:23.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.753+0000 7f93cb99a740 20 client.17769 trim_cache size 0 max 16384 2024-04-24T08:57:23.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.773+0000 7f93cb99a740 1 Processor -- start 2024-04-24T08:57:23.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.773+0000 7f93cb99a740 1 -- start start 2024-04-24T08:57:23.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.774+0000 7f93cb99a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56533e80d270 0x56533e814120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:23.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.774+0000 7f93cb99a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56533e80db90 0x56533e823eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:23.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.774+0000 7f93cb99a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56533e818260 0x56533e80fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:23.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.774+0000 7f93cb99a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f93c41b1500 con 0x56533e80d270 2024-04-24T08:57:23.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.774+0000 7f93cb99a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f93c41b3040 con 0x56533e818260 2024-04-24T08:57:23.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.774+0000 7f93cb99a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f93c41b1bd0 con 0x56533e80db90 2024-04-24T08:57:23.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.774+0000 7f93c1e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56533e80d270 0x56533e814120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:23.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.774+0000 7f93c1e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56533e80d270 0x56533e814120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:59160/0 (socket says 172.21.15.83:59160) 2024-04-24T08:57:23.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.774+0000 7f93c1e00640 1 -- 172.21.15.83:0/2921977926 learned_addr learned my addr 172.21.15.83:0/2921977926 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:57:23.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.774+0000 7f93c1e00640 1 -- 172.21.15.83:0/2921977926 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56533e80db90 msgr2=0x56533e823eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:23.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.774+0000 7f93c1e00640 1 --2- 172.21.15.83:0/2921977926 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56533e80db90 0x56533e823eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:23.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.774+0000 7f93c1e00640 1 -- 172.21.15.83:0/2921977926 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56533e818260 msgr2=0x56533e80fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:23.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.774+0000 7f93c1e00640 1 --2- 172.21.15.83:0/2921977926 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56533e818260 0x56533e80fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:23.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.774+0000 7f93c1e00640 1 -- 172.21.15.83:0/2921977926 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56533e827120 con 0x56533e80d270 2024-04-24T08:57:23.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.775+0000 7f93c1e00640 1 --2- 172.21.15.83:0/2921977926 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56533e80d270 0x56533e814120 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f93b4005a80 tx=0x7f93b4005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49577b8777815d7f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:23.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.775+0000 7f93ae600640 1 -- 172.21.15.83:0/2921977926 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b4035040 con 0x56533e80d270 2024-04-24T08:57:23.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.775+0000 7f93ae600640 1 -- 172.21.15.83:0/2921977926 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f93b4009780 con 0x56533e80d270 2024-04-24T08:57:23.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.775+0000 7f93ae600640 1 -- 172.21.15.83:0/2921977926 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b4004920 con 0x56533e80d270 2024-04-24T08:57:23.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.776+0000 7f93cb99a740 1 -- 172.21.15.83:0/2921977926 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56533e80d270 msgr2=0x56533e814120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:23.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.776+0000 7f93cb99a740 1 --2- 172.21.15.83:0/2921977926 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56533e80d270 0x56533e814120 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f93b4005a80 tx=0x7f93b4005fd0 comp rx=0 tx=0).stop 2024-04-24T08:57:23.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.776+0000 7f93cb99a740 1 -- 172.21.15.83:0/2921977926 shutdown_connections 2024-04-24T08:57:23.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.776+0000 7f93cb99a740 1 --2- 172.21.15.83:0/2921977926 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56533e818260 0x56533e80fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:23.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.776+0000 7f93cb99a740 1 --2- 172.21.15.83:0/2921977926 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56533e80db90 0x56533e823eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:23.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.776+0000 7f93cb99a740 1 --2- 172.21.15.83:0/2921977926 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56533e80d270 0x56533e814120 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:23.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.776+0000 7f93cb99a740 1 -- 172.21.15.83:0/2921977926 >> 172.21.15.83:0/2921977926 conn(0x56533e809590 msgr2=0x56533e815a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:23.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.776+0000 7f93cb99a740 1 -- 172.21.15.83:0/2921977926 shutdown_connections 2024-04-24T08:57:23.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.776+0000 7f93cb99a740 1 -- 172.21.15.83:0/2921977926 wait complete. 2024-04-24T08:57:23.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.777+0000 7f93cb99a740 1 -- start start 2024-04-24T08:57:23.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.777+0000 7f93cb99a740 1 Processor -- start 2024-04-24T08:57:23.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.777+0000 7f93cb99a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56533e80d270 0x56533e811e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:23.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.777+0000 7f93cb99a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56533e80db90 0x56533e8123d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:23.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.777+0000 7f93cb99a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56533e818260 0x56533e812910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:23.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.777+0000 7f93cb99a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f93c41b2970 con 0x56533e80d270 2024-04-24T08:57:23.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.777+0000 7f93cb99a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f93c41b22a0 con 0x56533e80db90 2024-04-24T08:57:23.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.777+0000 7f93cb99a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f93c41af4e0 con 0x56533e818260 2024-04-24T08:57:23.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.777+0000 7f93cb99a740 10 client.0 _ll_register_callbacks cb 0x7f93b000c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:57:23.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.778+0000 7f93c1e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56533e80d270 0x56533e811e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:23.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.778+0000 7f93c1e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56533e80d270 0x56533e811e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:59164/0 (socket says 172.21.15.83:59164) 2024-04-24T08:57:23.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.778+0000 7f93c1e00640 1 -- 172.21.15.83:0/2431638267 learned_addr learned my addr 172.21.15.83:0/2431638267 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:57:23.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.778+0000 7f93c1e00640 1 -- 172.21.15.83:0/2431638267 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56533e818260 msgr2=0x56533e812910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:23.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.778+0000 7f93c1e00640 1 --2- 172.21.15.83:0/2431638267 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56533e818260 0x56533e812910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:23.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.778+0000 7f93c1e00640 1 -- 172.21.15.83:0/2431638267 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56533e80db90 msgr2=0x56533e8123d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:23.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.778+0000 7f93c1e00640 1 --2- 172.21.15.83:0/2431638267 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56533e80db90 0x56533e8123d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:23.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.778+0000 7f93c1e00640 1 -- 172.21.15.83:0/2431638267 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56533e88daf0 con 0x56533e80d270 2024-04-24T08:57:23.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.778+0000 7f93c1e00640 1 --2- 172.21.15.83:0/2431638267 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56533e80d270 0x56533e811e90 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f93b4005ab0 tx=0x7f93b4031f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ad3aae2f1a3e890 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:23.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.778+0000 7f93a3400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:57:23.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.778+0000 7f93a3400640 1 -- 172.21.15.83:0/2431638267 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b4035070 con 0x56533e80d270 2024-04-24T08:57:23.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.779+0000 7f93a3400640 1 -- 172.21.15.83:0/2431638267 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f93b4005040 con 0x56533e80d270 2024-04-24T08:57:23.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.779+0000 7f93cb99a740 1 -- 172.21.15.83:0/2431638267 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93b4009780 con 0x56533e80d270 2024-04-24T08:57:23.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.779+0000 7f93a3400640 1 -- 172.21.15.83:0/2431638267 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f93b4060050 con 0x56533e80d270 2024-04-24T08:57:23.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.779+0000 7f93cb99a740 10 client.17775 fetch_fsmap learned FSMap version 33 2024-04-24T08:57:23.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.779+0000 7f93cb99a740 1 -- 172.21.15.83:0/2431638267 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56533e88ee10 con 0x56533e80d270 2024-04-24T08:57:23.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.779+0000 7f93a3400640 1 -- 172.21.15.83:0/2431638267 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f93b4009780 con 0x56533e80d270 2024-04-24T08:57:23.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.779+0000 7f93a3400640 1 -- 172.21.15.83:0/2431638267 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f93b404f090 con 0x56533e80d270 2024-04-24T08:57:23.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.779+0000 7f93cb99a740 10 client.17775 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T08:57:23.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.779+0000 7f93cb99a740 10 client.17775 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:57:23.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.779+0000 7f93cb99a740 20 client.17775 populate_metadata read hostname 'smithi083' 2024-04-24T08:57:23.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.779+0000 7f93cb99a740 1 --2- 172.21.15.83:0/2431638267 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56533e88f0e0 0x56533e891520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:23.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.779+0000 7f93cb99a740 4 client.17775 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T08:57:23.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.779+0000 7f93cb99a740 1 -- 172.21.15.83:0/2431638267 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56533e77a700 con 0x56533e88f0e0 2024-04-24T08:57:23.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.780+0000 7f93c1400640 1 --2- 172.21.15.83:0/2431638267 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56533e88f0e0 0x56533e891520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:23.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.781+0000 7f93c1400640 1 --2- 172.21.15.83:0/2431638267 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56533e88f0e0 0x56533e891520 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74c6b84af1c166b5 server_cookie=e61ba92bab93e73e in_seq=0 out_seq=0 2024-04-24T08:57:23.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.781+0000 7f93a3400640 10 client.17775 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:57:23.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.784+0000 7f93a3400640 1 -- 172.21.15.83:0/2431638267 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56533e77a700 con 0x56533e88f0e0 2024-04-24T08:57:23.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.784+0000 7f93a3400640 10 client.17775 handle_command_reply: tid=0 2024-04-24T08:57:23.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.784+0000 7f93cb99a740 1 client.17775 shutdown 2024-04-24T08:57:23.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.785+0000 7f93cb99a740 1 -- 172.21.15.83:0/2431638267 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56533e80d270 msgr2=0x56533e811e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:23.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.785+0000 7f93cb99a740 1 --2- 172.21.15.83:0/2431638267 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56533e80d270 0x56533e811e90 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f93b4005ab0 tx=0x7f93b4031f90 comp rx=0 tx=0).stop 2024-04-24T08:57:23.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.785+0000 7f93cb99a740 1 -- 172.21.15.83:0/2431638267 shutdown_connections 2024-04-24T08:57:23.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.785+0000 7f93cb99a740 1 --2- 172.21.15.83:0/2431638267 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56533e88f0e0 0x56533e891520 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:23.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.785+0000 7f93cb99a740 1 --2- 172.21.15.83:0/2431638267 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56533e818260 0x56533e812910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:23.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.785+0000 7f93cb99a740 1 --2- 172.21.15.83:0/2431638267 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56533e80db90 0x56533e8123d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:23.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.785+0000 7f93cb99a740 1 --2- 172.21.15.83:0/2431638267 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56533e80d270 0x56533e811e90 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:23.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.785+0000 7f93a3400640 0 client.17775 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:57:23.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.785+0000 7f93cb99a740 1 -- 172.21.15.83:0/2431638267 >> 172.21.15.83:0/2431638267 conn(0x56533e809590 msgr2=0x56533e807e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:23.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.785+0000 7f93cb99a740 1 -- 172.21.15.83:0/2431638267 shutdown_connections 2024-04-24T08:57:23.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.785+0000 7f93cb99a740 1 -- 172.21.15.83:0/2431638267 wait complete. 2024-04-24T08:57:23.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.785+0000 7f93cb99a740 20 client.17775 trim_cache size 0 max 16384 2024-04-24T08:57:23.785 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T08:57:23.785 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-04-24T08:57:23.785 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "34823e3c-b311-4121-9bf1-3dc7dafc47e0", 2024-04-24T08:57:23.786 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-04-24T08:57:23.786 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T08:57:23.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.786+0000 7f93c3200640 1 -- 172.21.15.83:0/22195001 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f93a80a3ec0 msgr2=0x7f93a80a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:23.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.786+0000 7f93c3200640 1 --2- 172.21.15.83:0/22195001 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f93a80a3ec0 0x7f93a80a6380 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f93bc005fd0 tx=0x7f93bc009530 comp rx=0 tx=0).stop 2024-04-24T08:57:23.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.786+0000 7f93c3200640 1 -- 172.21.15.83:0/22195001 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f93c41093f0 msgr2=0x7f93c41a0b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:23.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.786+0000 7f93c3200640 1 --2- 172.21.15.83:0/22195001 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f93c41093f0 0x7f93c41a0b20 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f93b000c970 tx=0x7f93b000ce30 comp rx=0 tx=0).stop 2024-04-24T08:57:23.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.786+0000 7f93c3200640 1 -- 172.21.15.83:0/22195001 shutdown_connections 2024-04-24T08:57:23.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.786+0000 7f93c3200640 1 --2- 172.21.15.83:0/22195001 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f93a80a3ec0 0x7f93a80a6380 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:23.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.786+0000 7f93c3200640 1 --2- 172.21.15.83:0/22195001 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f93c419a6a0 0x7f93c419ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:23.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.786+0000 7f93c3200640 1 --2- 172.21.15.83:0/22195001 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f93c41093f0 0x7f93c41a0b20 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:23.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.786+0000 7f93c3200640 1 --2- 172.21.15.83:0/22195001 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f93c4104320 0x7f93c41a05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:23.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.786+0000 7f93c3200640 1 -- 172.21.15.83:0/22195001 >> 172.21.15.83:0/22195001 conn(0x7f93c40ffb40 msgr2=0x7f93c4101ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:23.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.786+0000 7f93c3200640 1 -- 172.21.15.83:0/22195001 shutdown_connections 2024-04-24T08:57:23.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:23.786+0000 7f93c3200640 1 -- 172.21.15.83:0/22195001 wait complete. 2024-04-24T08:57:23.802 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T08:57:23.802 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T08:57:24.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.004+0000 7ff89a400640 1 -- 172.21.15.83:0/4189243407 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8941044b0 msgr2=0x7ff894104890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:24.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.004+0000 7ff89a400640 1 --2- 172.21.15.83:0/4189243407 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8941044b0 0x7ff894104890 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7ff884009900 tx=0x7ff88402a3f0 comp rx=0 tx=0).stop 2024-04-24T08:57:24.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.004+0000 7ff89a400640 1 -- 172.21.15.83:0/4189243407 shutdown_connections 2024-04-24T08:57:24.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.004+0000 7ff89a400640 1 --2- 172.21.15.83:0/4189243407 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff894109580 0x7ff894111150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:24.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.004+0000 7ff89a400640 1 --2- 172.21.15.83:0/4189243407 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff894104e60 0x7ff894108ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:24.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.004+0000 7ff89a400640 1 --2- 172.21.15.83:0/4189243407 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8941044b0 0x7ff894104890 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:24.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.004+0000 7ff89a400640 1 -- 172.21.15.83:0/4189243407 >> 172.21.15.83:0/4189243407 conn(0x7ff8940ffd90 msgr2=0x7ff8941021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:24.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.004+0000 7ff89a400640 1 -- 172.21.15.83:0/4189243407 shutdown_connections 2024-04-24T08:57:24.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.004+0000 7ff89a400640 1 -- 172.21.15.83:0/4189243407 wait complete. 2024-04-24T08:57:24.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.005+0000 7ff89a400640 1 Processor -- start 2024-04-24T08:57:24.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.005+0000 7ff89a400640 1 -- start start 2024-04-24T08:57:24.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.006+0000 7ff89a400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff8941044b0 0x7ff8941a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:24.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.006+0000 7ff89a400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff894109580 0x7ff8941a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:24.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.006+0000 7ff89a400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff89419a800 0x7ff89419acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:24.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.006+0000 7ff89a400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff8941144c0 con 0x7ff894109580 2024-04-24T08:57:24.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.006+0000 7ff89a400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff894114340 con 0x7ff89419a800 2024-04-24T08:57:24.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.006+0000 7ff89a400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff894114640 con 0x7ff8941044b0 2024-04-24T08:57:24.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.006+0000 7ff892a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff894109580 0x7ff8941a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:24.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.006+0000 7ff892a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff894109580 0x7ff8941a0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:59182/0 (socket says 172.21.15.83:59182) 2024-04-24T08:57:24.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.006+0000 7ff892a00640 1 -- 172.21.15.83:0/580374317 learned_addr learned my addr 172.21.15.83:0/580374317 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:57:24.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.006+0000 7ff892a00640 1 -- 172.21.15.83:0/580374317 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff8941044b0 msgr2=0x7ff8941a0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:24.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.006+0000 7ff893e00640 1 --2- 172.21.15.83:0/580374317 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff89419a800 0x7ff89419acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:24.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.006+0000 7ff893400640 1 --2- 172.21.15.83:0/580374317 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff8941044b0 0x7ff8941a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:24.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.007+0000 7ff892a00640 1 --2- 172.21.15.83:0/580374317 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff8941044b0 0x7ff8941a0690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:24.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.007+0000 7ff892a00640 1 -- 172.21.15.83:0/580374317 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff89419a800 msgr2=0x7ff89419acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:24.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.007+0000 7ff892a00640 1 --2- 172.21.15.83:0/580374317 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff89419a800 0x7ff89419acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:24.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.007+0000 7ff892a00640 1 -- 172.21.15.83:0/580374317 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff89419b390 con 0x7ff894109580 2024-04-24T08:57:24.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.007+0000 7ff893e00640 1 --2- 172.21.15.83:0/580374317 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff89419a800 0x7ff89419acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:57:24.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.007+0000 7ff892a00640 1 --2- 172.21.15.83:0/580374317 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff894109580 0x7ff8941a0bd0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7ff88000c970 tx=0x7ff88000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:24.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.007+0000 7ff88be00640 1 -- 172.21.15.83:0/580374317 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff880014070 con 0x7ff894109580 2024-04-24T08:57:24.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.007+0000 7ff88be00640 1 -- 172.21.15.83:0/580374317 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff880004440 con 0x7ff894109580 2024-04-24T08:57:24.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.007+0000 7ff88be00640 1 -- 172.21.15.83:0/580374317 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff880003e50 con 0x7ff894109580 2024-04-24T08:57:24.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.007+0000 7ff89a400640 1 -- 172.21.15.83:0/580374317 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff89419b680 con 0x7ff894109580 2024-04-24T08:57:24.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.007+0000 7ff89a400640 1 -- 172.21.15.83:0/580374317 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8941a7860 con 0x7ff894109580 2024-04-24T08:57:24.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.011+0000 7ff89d3d3740 1 Processor -- start 2024-04-24T08:57:24.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.011+0000 7ff89d3d3740 1 -- start start 2024-04-24T08:57:24.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.011+0000 7ff89d3d3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbb3927860 0x55fbb3927c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:24.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.011+0000 7ff89d3d3740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fbb3928180 0x55fbb3932310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:24.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.011+0000 7ff89d3d3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fbb3932850 0x55fbb3934c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:24.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.011+0000 7ff89d3d3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55fbb388d1f0 con 0x55fbb3927860 2024-04-24T08:57:24.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.011+0000 7ff89d3d3740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55fbb3833350 con 0x55fbb3928180 2024-04-24T08:57:24.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.011+0000 7ff89d3d3740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55fbb38891b0 con 0x55fbb3932850 2024-04-24T08:57:24.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.012+0000 7ff893e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbb3927860 0x55fbb3927c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:24.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.012+0000 7ff893e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbb3927860 0x55fbb3927c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:59186/0 (socket says 172.21.15.83:59186) 2024-04-24T08:57:24.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.012+0000 7ff893e00640 1 -- 172.21.15.83:0/1573755919 learned_addr learned my addr 172.21.15.83:0/1573755919 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:57:24.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.012+0000 7ff88be00640 1 -- 172.21.15.83:0/580374317 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff880017070 con 0x7ff894109580 2024-04-24T08:57:24.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.012+0000 7ff893400640 1 --2- 172.21.15.83:0/1573755919 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fbb3928180 0x55fbb3932310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:24.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.012+0000 7ff893e00640 1 -- 172.21.15.83:0/1573755919 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fbb3932850 msgr2=0x55fbb3934c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:24.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.012+0000 7ff893e00640 1 --2- 172.21.15.83:0/1573755919 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fbb3932850 0x55fbb3934c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:24.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.012+0000 7ff893e00640 1 -- 172.21.15.83:0/1573755919 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fbb3928180 msgr2=0x55fbb3932310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:24.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.012+0000 7ff893e00640 1 --2- 172.21.15.83:0/1573755919 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fbb3928180 0x55fbb3932310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:24.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.012+0000 7ff893e00640 1 -- 172.21.15.83:0/1573755919 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fbb3887f50 con 0x55fbb3927860 2024-04-24T08:57:24.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.012+0000 7ff893400640 1 --2- 172.21.15.83:0/1573755919 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fbb3928180 0x55fbb3932310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:57:24.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.013+0000 7ff893e00640 1 --2- 172.21.15.83:0/1573755919 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbb3927860 0x55fbb3927c40 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7ff88c008de0 tx=0x7ff88c00d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb38fc52479afb2d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:24.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.013+0000 7ff888c00640 1 -- 172.21.15.83:0/1573755919 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff88c00f830 con 0x55fbb3927860 2024-04-24T08:57:24.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.013+0000 7ff888c00640 1 -- 172.21.15.83:0/1573755919 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff88c00de20 con 0x55fbb3927860 2024-04-24T08:57:24.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.013+0000 7ff888c00640 1 -- 172.21.15.83:0/1573755919 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff88c00e3b0 con 0x55fbb3927860 2024-04-24T08:57:24.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.014+0000 7ff88be00640 1 --2- 172.21.15.83:0/580374317 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff8780a3ec0 0x7ff8780a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:24.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.014+0000 7ff89d3d3740 1 -- 172.21.15.83:0/1573755919 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbb3927860 msgr2=0x55fbb3927c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:24.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.014+0000 7ff89d3d3740 1 --2- 172.21.15.83:0/1573755919 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbb3927860 0x55fbb3927c40 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7ff88c008de0 tx=0x7ff88c00d710 comp rx=0 tx=0).stop 2024-04-24T08:57:24.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.014+0000 7ff88be00640 1 -- 172.21.15.83:0/580374317 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7ff8800d0ab0 con 0x7ff894109580 2024-04-24T08:57:24.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.014+0000 7ff893400640 1 --2- 172.21.15.83:0/580374317 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff8780a3ec0 0x7ff8780a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:24.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.014+0000 7ff89d3d3740 1 -- 172.21.15.83:0/1573755919 shutdown_connections 2024-04-24T08:57:24.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.014+0000 7ff89d3d3740 1 --2- 172.21.15.83:0/1573755919 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fbb3932850 0x55fbb3934c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:24.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.014+0000 7ff89d3d3740 1 --2- 172.21.15.83:0/1573755919 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fbb3928180 0x55fbb3932310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:24.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.014+0000 7ff89d3d3740 1 --2- 172.21.15.83:0/1573755919 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbb3927860 0x55fbb3927c40 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:24.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.014+0000 7ff89d3d3740 1 -- 172.21.15.83:0/1573755919 >> 172.21.15.83:0/1573755919 conn(0x55fbb3923b80 msgr2=0x55fbb3923f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:24.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.014+0000 7ff89d3d3740 1 -- 172.21.15.83:0/1573755919 shutdown_connections 2024-04-24T08:57:24.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.014+0000 7ff89d3d3740 1 -- 172.21.15.83:0/1573755919 wait complete. 2024-04-24T08:57:24.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.015+0000 7ff89d3d3740 1 -- start start 2024-04-24T08:57:24.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.015+0000 7ff893400640 1 --2- 172.21.15.83:0/580374317 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff8780a3ec0 0x7ff8780a6380 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff884005ab0 tx=0x7ff884032de0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:24.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.015+0000 7ff89d3d3740 1 Processor -- start 2024-04-24T08:57:24.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.016+0000 7ff89d3d3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbb3927860 0x55fbb392f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:24.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.016+0000 7ff89d3d3740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fbb3928180 0x55fbb392fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:24.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.016+0000 7ff89d3d3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fbb3932850 0x55fbb3930300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:24.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.016+0000 7ff89d3d3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55fbb3935640 con 0x55fbb3927860 2024-04-24T08:57:24.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.016+0000 7ff89d3d3740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55fbb3846690 con 0x55fbb3928180 2024-04-24T08:57:24.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.016+0000 7ff89d3d3740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55fbb39357c0 con 0x55fbb3932850 2024-04-24T08:57:24.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.016+0000 7ff89d3d3740 10 client.0 _ll_register_callbacks cb 0x55fbb3882650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:57:24.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.016+0000 7ff893400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbb3927860 0x55fbb392f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:24.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.016+0000 7ff893400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbb3927860 0x55fbb392f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:59190/0 (socket says 172.21.15.83:59190) 2024-04-24T08:57:24.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.016+0000 7ff893400640 1 -- 172.21.15.83:0/4220921311 learned_addr learned my addr 172.21.15.83:0/4220921311 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:57:24.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.016+0000 7ff892a00640 1 --2- 172.21.15.83:0/4220921311 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fbb3928180 0x55fbb392fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:24.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.016+0000 7ff893e00640 1 --2- 172.21.15.83:0/4220921311 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fbb3932850 0x55fbb3930300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:24.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.016+0000 7ff893400640 1 -- 172.21.15.83:0/4220921311 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fbb3932850 msgr2=0x55fbb3930300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:24.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.016+0000 7ff893400640 1 --2- 172.21.15.83:0/4220921311 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fbb3932850 0x55fbb3930300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:24.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.016+0000 7ff893400640 1 -- 172.21.15.83:0/4220921311 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fbb3928180 msgr2=0x55fbb392fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:24.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.016+0000 7ff893400640 1 --2- 172.21.15.83:0/4220921311 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fbb3928180 0x55fbb392fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:24.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.017+0000 7ff893400640 1 -- 172.21.15.83:0/4220921311 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fbb3845920 con 0x55fbb3927860 2024-04-24T08:57:24.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.017+0000 7ff893e00640 1 --2- 172.21.15.83:0/4220921311 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fbb3932850 0x55fbb3930300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:57:24.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.017+0000 7ff893400640 1 --2- 172.21.15.83:0/4220921311 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbb3927860 0x55fbb392f880 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7ff884033000 tx=0x7ff88402a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c86f781ffab47756 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:24.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.017+0000 7ff87d600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:57:24.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.017+0000 7ff87d600640 1 -- 172.21.15.83:0/4220921311 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff88404a020 con 0x55fbb3927860 2024-04-24T08:57:24.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.017+0000 7ff89d3d3740 1 -- 172.21.15.83:0/4220921311 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff88c00de20 con 0x55fbb3927860 2024-04-24T08:57:24.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.017+0000 7ff87d600640 1 -- 172.21.15.83:0/4220921311 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7ff884003a00 con 0x55fbb3927860 2024-04-24T08:57:24.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.018+0000 7ff87d600640 1 -- 172.21.15.83:0/4220921311 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7ff884055050 con 0x55fbb3927860 2024-04-24T08:57:24.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.018+0000 7ff89d3d3740 10 client.17787 fetch_fsmap learned FSMap version 33 2024-04-24T08:57:24.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.018+0000 7ff89d3d3740 1 -- 172.21.15.83:0/4220921311 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55fbb36697d0 con 0x55fbb3927860 2024-04-24T08:57:24.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.018+0000 7ff87d600640 1 -- 172.21.15.83:0/4220921311 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff88c00de20 con 0x55fbb3927860 2024-04-24T08:57:24.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.018+0000 7ff87d600640 1 -- 172.21.15.83:0/4220921311 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7ff88403ba80 con 0x55fbb3927860 2024-04-24T08:57:24.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.018+0000 7ff89d3d3740 10 client.17787 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T08:57:24.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.018+0000 7ff89d3d3740 10 client.17787 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:57:24.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.018+0000 7ff89d3d3740 20 client.17787 populate_metadata read hostname 'smithi083' 2024-04-24T08:57:24.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.018+0000 7ff89d3d3740 1 --2- 172.21.15.83:0/4220921311 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55fbb3942b70 0x55fbb3945000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:24.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.018+0000 7ff89d3d3740 4 client.17787 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:57:24.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.018+0000 7ff89d3d3740 1 -- 172.21.15.83:0/4220921311 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fbb389ac10 con 0x55fbb3942b70 2024-04-24T08:57:24.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.019+0000 7ff892a00640 1 --2- 172.21.15.83:0/4220921311 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55fbb3942b70 0x55fbb3945000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:24.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.020+0000 7ff87d600640 10 client.17787 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:57:24.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.020+0000 7ff892a00640 1 --2- 172.21.15.83:0/4220921311 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55fbb3942b70 0x55fbb3945000 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49a4857879dc9078 server_cookie=230547b55339008a in_seq=0 out_seq=0 2024-04-24T08:57:24.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.022+0000 7ff87d600640 1 -- 172.21.15.83:0/4220921311 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fbb389ac10 con 0x55fbb3942b70 2024-04-24T08:57:24.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.022+0000 7ff87d600640 10 client.17787 handle_command_reply: tid=0 2024-04-24T08:57:24.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.022+0000 7ff89d3d3740 1 client.17787 shutdown 2024-04-24T08:57:24.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.022+0000 7ff89d3d3740 1 -- 172.21.15.83:0/4220921311 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbb3927860 msgr2=0x55fbb392f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:24.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.022+0000 7ff89d3d3740 1 --2- 172.21.15.83:0/4220921311 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbb3927860 0x55fbb392f880 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7ff884033000 tx=0x7ff88402a3f0 comp rx=0 tx=0).stop 2024-04-24T08:57:24.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.022+0000 7ff89d3d3740 1 -- 172.21.15.83:0/4220921311 shutdown_connections 2024-04-24T08:57:24.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.022+0000 7ff89d3d3740 1 --2- 172.21.15.83:0/4220921311 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55fbb3942b70 0x55fbb3945000 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:24.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.022+0000 7ff89d3d3740 1 --2- 172.21.15.83:0/4220921311 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fbb3932850 0x55fbb3930300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:24.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.022+0000 7ff89d3d3740 1 --2- 172.21.15.83:0/4220921311 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fbb3928180 0x55fbb392fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:24.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.022+0000 7ff89d3d3740 1 --2- 172.21.15.83:0/4220921311 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbb3927860 0x55fbb392f880 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:24.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.022+0000 7ff89d3d3740 1 -- 172.21.15.83:0/4220921311 >> 172.21.15.83:0/4220921311 conn(0x55fbb3923b80 msgr2=0x55fbb3929880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:24.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.022+0000 7ff87d600640 0 client.17787 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:57:24.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.022+0000 7ff89d3d3740 1 -- 172.21.15.83:0/4220921311 shutdown_connections 2024-04-24T08:57:24.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.023+0000 7ff89d3d3740 1 -- 172.21.15.83:0/4220921311 wait complete. 2024-04-24T08:57:24.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.023+0000 7ff89d3d3740 20 client.17787 trim_cache size 0 max 16384 2024-04-24T08:57:24.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.042+0000 7ff89d3d3740 1 Processor -- start 2024-04-24T08:57:24.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.042+0000 7ff89d3d3740 1 -- start start 2024-04-24T08:57:24.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.042+0000 7ff89d3d3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbb3927860 0x55fbb393e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:24.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.042+0000 7ff89d3d3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fbb3928180 0x55fbb394a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:24.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.042+0000 7ff89d3d3740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fbb3932850 0x55fbb394ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:24.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.042+0000 7ff89d3d3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff8941b1760 con 0x55fbb3927860 2024-04-24T08:57:24.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.042+0000 7ff89d3d3740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff8941b32a0 con 0x55fbb3932850 2024-04-24T08:57:24.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.042+0000 7ff89d3d3740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff8941b1e30 con 0x55fbb3928180 2024-04-24T08:57:24.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.043+0000 7ff893400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbb3927860 0x55fbb393e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:24.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.043+0000 7ff893400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbb3927860 0x55fbb393e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:59196/0 (socket says 172.21.15.83:59196) 2024-04-24T08:57:24.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.043+0000 7ff893400640 1 -- 172.21.15.83:0/719797754 learned_addr learned my addr 172.21.15.83:0/719797754 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:57:24.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.043+0000 7ff893400640 1 -- 172.21.15.83:0/719797754 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fbb3928180 msgr2=0x55fbb394a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:24.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.043+0000 7ff893400640 1 --2- 172.21.15.83:0/719797754 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fbb3928180 0x55fbb394a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:24.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.043+0000 7ff893400640 1 -- 172.21.15.83:0/719797754 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fbb3932850 msgr2=0x55fbb394ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:24.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.043+0000 7ff893e00640 1 --2- 172.21.15.83:0/719797754 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fbb3932850 0x55fbb394ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:24.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.043+0000 7ff893400640 1 --2- 172.21.15.83:0/719797754 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fbb3932850 0x55fbb394ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:24.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.043+0000 7ff893400640 1 -- 172.21.15.83:0/719797754 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fbb359f3f0 con 0x55fbb3927860 2024-04-24T08:57:24.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.043+0000 7ff893400640 1 --2- 172.21.15.83:0/719797754 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbb3927860 0x55fbb393e500 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7ff88403bea0 tx=0x7ff884005e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8fd0d7af70fa817 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:24.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.044+0000 7ff87fe00640 1 -- 172.21.15.83:0/719797754 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff88405e070 con 0x55fbb3927860 2024-04-24T08:57:24.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.044+0000 7ff87fe00640 1 -- 172.21.15.83:0/719797754 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff884041070 con 0x55fbb3927860 2024-04-24T08:57:24.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.044+0000 7ff87fe00640 1 -- 172.21.15.83:0/719797754 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff88404a040 con 0x55fbb3927860 2024-04-24T08:57:24.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.045+0000 7ff89d3d3740 1 -- 172.21.15.83:0/719797754 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbb3927860 msgr2=0x55fbb393e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:24.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.045+0000 7ff89d3d3740 1 --2- 172.21.15.83:0/719797754 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbb3927860 0x55fbb393e500 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7ff88403bea0 tx=0x7ff884005e70 comp rx=0 tx=0).stop 2024-04-24T08:57:24.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.045+0000 7ff89d3d3740 1 -- 172.21.15.83:0/719797754 shutdown_connections 2024-04-24T08:57:24.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.045+0000 7ff89d3d3740 1 --2- 172.21.15.83:0/719797754 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fbb3932850 0x55fbb394ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:24.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.045+0000 7ff89d3d3740 1 --2- 172.21.15.83:0/719797754 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fbb3928180 0x55fbb394a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:24.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.045+0000 7ff89d3d3740 1 --2- 172.21.15.83:0/719797754 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbb3927860 0x55fbb393e500 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:24.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.045+0000 7ff89d3d3740 1 -- 172.21.15.83:0/719797754 >> 172.21.15.83:0/719797754 conn(0x55fbb3923b80 msgr2=0x55fbb392e150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:24.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.045+0000 7ff89d3d3740 1 -- 172.21.15.83:0/719797754 shutdown_connections 2024-04-24T08:57:24.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.045+0000 7ff89d3d3740 1 -- 172.21.15.83:0/719797754 wait complete. 2024-04-24T08:57:24.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.045+0000 7ff89d3d3740 1 -- start start 2024-04-24T08:57:24.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.046+0000 7ff89d3d3740 1 Processor -- start 2024-04-24T08:57:24.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.046+0000 7ff89d3d3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fbb3927860 0x55fbb393ab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:24.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.046+0000 7ff89d3d3740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fbb3928180 0x55fbb3935fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:24.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.046+0000 7ff89d3d3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbb3932850 0x55fbb3936510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:24.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.046+0000 7ff89d3d3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff8941b2bd0 con 0x55fbb3932850 2024-04-24T08:57:24.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.046+0000 7ff89d3d3740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff8941b2500 con 0x55fbb3928180 2024-04-24T08:57:24.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.046+0000 7ff89d3d3740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff8941af740 con 0x55fbb3927860 2024-04-24T08:57:24.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.046+0000 7ff89d3d3740 10 client.0 _ll_register_callbacks cb 0x7ff884009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:57:24.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.046+0000 7ff893e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbb3932850 0x55fbb3936510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:24.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.046+0000 7ff893e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbb3932850 0x55fbb3936510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:59204/0 (socket says 172.21.15.83:59204) 2024-04-24T08:57:24.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.046+0000 7ff893e00640 1 -- 172.21.15.83:0/532483084 learned_addr learned my addr 172.21.15.83:0/532483084 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:57:24.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.047+0000 7ff892a00640 1 --2- 172.21.15.83:0/532483084 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fbb3928180 0x55fbb3935fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:24.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.047+0000 7ff893400640 1 --2- 172.21.15.83:0/532483084 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fbb3927860 0x55fbb393ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:24.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.047+0000 7ff893e00640 1 -- 172.21.15.83:0/532483084 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fbb3927860 msgr2=0x55fbb393ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:24.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.047+0000 7ff893e00640 1 --2- 172.21.15.83:0/532483084 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fbb3927860 0x55fbb393ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:24.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.047+0000 7ff893e00640 1 -- 172.21.15.83:0/532483084 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fbb3928180 msgr2=0x55fbb3935fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:24.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.047+0000 7ff893e00640 1 --2- 172.21.15.83:0/532483084 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fbb3928180 0x55fbb3935fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:24.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.047+0000 7ff893e00640 1 -- 172.21.15.83:0/532483084 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fbb355f0f0 con 0x55fbb3932850 2024-04-24T08:57:24.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.047+0000 7ff893e00640 1 --2- 172.21.15.83:0/532483084 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbb3932850 0x55fbb3936510 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7ff88c005fd0 tx=0x7ff88c00f860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8ad48670e266d65 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:24.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.047+0000 7ff87d600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:57:24.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.047+0000 7ff87d600640 1 -- 172.21.15.83:0/532483084 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff88c027060 con 0x55fbb3932850 2024-04-24T08:57:24.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.047+0000 7ff87d600640 1 -- 172.21.15.83:0/532483084 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7ff88c00fd60 con 0x55fbb3932850 2024-04-24T08:57:24.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.048+0000 7ff89d3d3740 1 -- 172.21.15.83:0/532483084 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff884041070 con 0x55fbb3932850 2024-04-24T08:57:24.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.048+0000 7ff87d600640 1 -- 172.21.15.83:0/532483084 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7ff88c0089b0 con 0x55fbb3932850 2024-04-24T08:57:24.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.048+0000 7ff89d3d3740 10 client.17793 fetch_fsmap learned FSMap version 33 2024-04-24T08:57:24.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.048+0000 7ff89d3d3740 1 -- 172.21.15.83:0/532483084 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55fbb3852810 con 0x55fbb3932850 2024-04-24T08:57:24.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.048+0000 7ff87d600640 1 -- 172.21.15.83:0/532483084 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff884041070 con 0x55fbb3932850 2024-04-24T08:57:24.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.048+0000 7ff87d600640 1 -- 172.21.15.83:0/532483084 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7ff88c0185a0 con 0x55fbb3932850 2024-04-24T08:57:24.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.048+0000 7ff89d3d3740 10 client.17793 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T08:57:24.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.048+0000 7ff89d3d3740 10 client.17793 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:57:24.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.048+0000 7ff89d3d3740 20 client.17793 populate_metadata read hostname 'smithi083' 2024-04-24T08:57:24.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.048+0000 7ff89d3d3740 1 --2- 172.21.15.83:0/532483084 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55fbb39a9000 0x55fbb39ab420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:24.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.048+0000 7ff89d3d3740 4 client.17793 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T08:57:24.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.048+0000 7ff89d3d3740 1 -- 172.21.15.83:0/532483084 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55fbb3894a80 con 0x55fbb39a9000 2024-04-24T08:57:24.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.049+0000 7ff893400640 1 --2- 172.21.15.83:0/532483084 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55fbb39a9000 0x55fbb39ab420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:24.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.050+0000 7ff893400640 1 --2- 172.21.15.83:0/532483084 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55fbb39a9000 0x55fbb39ab420 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6915c0a8821b92d server_cookie=193847e5170f52b7 in_seq=0 out_seq=0 2024-04-24T08:57:24.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.050+0000 7ff87d600640 10 client.17793 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:57:24.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.050+0000 7ff87d600640 1 -- 172.21.15.83:0/532483084 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55fbb3894a80 con 0x55fbb39a9000 2024-04-24T08:57:24.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.050+0000 7ff87d600640 10 client.17793 handle_command_reply: tid=0 2024-04-24T08:57:24.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.050+0000 7ff89d3d3740 1 client.17793 shutdown 2024-04-24T08:57:24.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.051+0000 7ff89d3d3740 1 -- 172.21.15.83:0/532483084 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbb3932850 msgr2=0x55fbb3936510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:24.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.051+0000 7ff89d3d3740 1 --2- 172.21.15.83:0/532483084 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbb3932850 0x55fbb3936510 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7ff88c005fd0 tx=0x7ff88c00f860 comp rx=0 tx=0).stop 2024-04-24T08:57:24.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.051+0000 7ff89d3d3740 1 -- 172.21.15.83:0/532483084 shutdown_connections 2024-04-24T08:57:24.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.051+0000 7ff89d3d3740 1 --2- 172.21.15.83:0/532483084 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55fbb39a9000 0x55fbb39ab420 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:24.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.051+0000 7ff89d3d3740 1 --2- 172.21.15.83:0/532483084 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbb3932850 0x55fbb3936510 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:24.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.051+0000 7ff89d3d3740 1 --2- 172.21.15.83:0/532483084 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fbb3928180 0x55fbb3935fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:24.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.051+0000 7ff89d3d3740 1 --2- 172.21.15.83:0/532483084 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fbb3927860 0x55fbb393ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:24.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.051+0000 7ff89d3d3740 1 -- 172.21.15.83:0/532483084 >> 172.21.15.83:0/532483084 conn(0x55fbb3923b80 msgr2=0x55fbb392f820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:24.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.051+0000 7ff87d600640 0 client.17793 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:57:24.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.051+0000 7ff89d3d3740 1 -- 172.21.15.83:0/532483084 shutdown_connections 2024-04-24T08:57:24.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.051+0000 7ff89d3d3740 1 -- 172.21.15.83:0/532483084 wait complete. 2024-04-24T08:57:24.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.051+0000 7ff89d3d3740 20 client.17793 trim_cache size 0 max 16384 2024-04-24T08:57:24.053 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T08:57:24.053 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T08:57:24.053 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-04-24T08:57:24.053 INFO:teuthology.orchestra.run.smithi083.stdout: "34823e3c-b311-4121-9bf1-3dc7dafc47e0": { 2024-04-24T08:57:24.053 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-04-24T08:57:24.053 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "34823e3c-b311-4121-9bf1-3dc7dafc47e0", 2024-04-24T08:57:24.053 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-04-24T08:57:24.053 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T08:57:24.053 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T08:57:24.054 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T08:57:24.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.052+0000 7ff89a400640 1 -- 172.21.15.83:0/580374317 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff8780a3ec0 msgr2=0x7ff8780a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:24.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.052+0000 7ff89a400640 1 --2- 172.21.15.83:0/580374317 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff8780a3ec0 0x7ff8780a6380 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff884005ab0 tx=0x7ff884032de0 comp rx=0 tx=0).stop 2024-04-24T08:57:24.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.052+0000 7ff89a400640 1 -- 172.21.15.83:0/580374317 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff894109580 msgr2=0x7ff8941a0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:24.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.052+0000 7ff89a400640 1 --2- 172.21.15.83:0/580374317 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff894109580 0x7ff8941a0bd0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7ff88000c970 tx=0x7ff88000ce30 comp rx=0 tx=0).stop 2024-04-24T08:57:24.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.052+0000 7ff89a400640 1 -- 172.21.15.83:0/580374317 shutdown_connections 2024-04-24T08:57:24.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.052+0000 7ff89a400640 1 --2- 172.21.15.83:0/580374317 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff8780a3ec0 0x7ff8780a6380 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:24.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.052+0000 7ff89a400640 1 --2- 172.21.15.83:0/580374317 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff89419a800 0x7ff89419acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:24.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.052+0000 7ff89a400640 1 --2- 172.21.15.83:0/580374317 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff894109580 0x7ff8941a0bd0 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:24.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.052+0000 7ff89a400640 1 --2- 172.21.15.83:0/580374317 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff8941044b0 0x7ff8941a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:24.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.052+0000 7ff89a400640 1 -- 172.21.15.83:0/580374317 >> 172.21.15.83:0/580374317 conn(0x7ff8940ffd90 msgr2=0x7ff894102120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:24.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.052+0000 7ff89a400640 1 -- 172.21.15.83:0/580374317 shutdown_connections 2024-04-24T08:57:24.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:24.052+0000 7ff89a400640 1 -- 172.21.15.83:0/580374317 wait complete. 2024-04-24T08:57:24.069 INFO:tasks.cephfs.filesystem:scrub status for tag:34823e3c-b311-4121-9bf1-3dc7dafc47e0 - {'path': '/', 'tag': '34823e3c-b311-4121-9bf1-3dc7dafc47e0', 'options': 'recursive,force'} 2024-04-24T08:57:24.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:57:23 smithi138 ceph-mon[31441]: pgmap v176: 129 pgs: 129 active+clean; 41 MiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 672 KiB/s wr, 66 op/s 2024-04-24T08:57:24.205 INFO:journalctl@ceph.mds.b.smithi138.stdout:Apr 24 08:57:23 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-b[61515]: 2024-04-24T08:57:23.820+0000 7f7fbbc00700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000208 [...8,head] /volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/WORD/ auth{2=1} v4938 ap=2 f(v168 m2024-04-24T08:57:01.824004+0000 5=5+0) n(v15 rc2024-04-24T08:57:01.824004+0000 b814326 6=5+1) old_inodes=4 (inest sync->mix g=2) caps={15603=pAsLsXsFs/-@4113} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x55cf87274c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000000208:[<0x10000000009/WORD v427>,<0x10000000008/~dmtmp v57>,<0x10000000007/client0 v33>,<0x10000000006/clients v23>,<0x10000000005/tmp v23>,<0x10000000003/client.0 v22>,<0x10000000002/e395bed6-5bba-41b8-a256-3505e4afcca2 v36>,<0x10000000001/sv_0 v55>,<0x10000000000/qa v43>,<0x1/volumes v39>]//[]","memoryvalue":"(2)0x10000000208:[<0x10000000009/WORD v4938>,<0x10000000008/~dmtmp v4516>,<0x10000000007/client0 v544>,<0x10000000006/clients v534>,<0x10000000005/tmp v532>,<0x10000000003/client.0 v529>,<0x10000000002/e395bed6-5bba-41b8-a256-3505e4afcca2 v542>,<0x10000000001/sv_0 v556>,<0x10000000000/qa v540>,<0x1/volumes v525>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-04-24T08:57:01.772589+0000 6=6+0)","ondisk_value.rstat":"n(v0 rc2024-04-24T08:57:01.824004+0000 b814326 6=5+1)","memory_value.dirstat":"f(v168 m2024-04-24T08:57:01.824004+0000 5=5+0)","memory_value.rstat":"n(v15 rc2024-04-24T08:57:01.824004+0000 b814326 6=5+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-04-24T08:57:24.205 INFO:journalctl@ceph.mds.b.smithi138.stdout:Apr 24 08:57:23 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-b[61515]: 2024-04-24T08:57:23.838+0000 7f7fbbc00700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000229 [...8,head] /volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/PARADOX/ auth{1=1,2=1} fragtree_t(*^3) v4994 ap=2 f(v245 m2024-04-24T08:57:08.744739+0000 24=24+0) n(v18 rc2024-04-24T08:57:07.878690+0000 b5230239 25=24+1) old_inodes=4 (inest sync->mix g=1,2) caps={15603=pAsLsXsFs/-@4953} | dirtyscattered=0 request=0 lock=0 dirfrag=8 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x55cf87642b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000000229:[<0x10000000009/PARADOX v481>,<0x10000000008/~dmtmp v57>,<0x10000000007/client0 v33>,<0x10000000006/clients v23>,<0x10000000005/tmp v23>,<0x10000000003/client.0 v22>,<0x10000000002/e395bed6-5bba-41b8-a256-3505e4afcca2 v36>,<0x10000000001/sv_0 v55>,<0x10000000000/qa v43>,<0x1/volumes v39>]//[]","memoryvalue":"(2)0x10000000229:[<0x10000000009/PARADOX v4994>,<0x10000000008/~dmtmp v4522>,<0x10000000007/client0 v544>,<0x10000000006/clients v534>,<0x10000000005/tmp v532>,<0x10000000003/client.0 v529>,<0x10000000002/e395bed6-5bba-41b8-a256-3505e4afcca2 v542>,<0x10000000001/sv_0 v556>,<0x10000000000/qa v540>,<0x1/volumes v525>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-04-24T08:57:08.744739+0000 25=25+0)","ondisk_value.rstat":"n(v0 rc2024-04-24T08:57:07.878690+0000 b5230239 25=24+1)","memory_value.dirstat":"f(v245 m2024-04-24T08:57:08.744739+0000 24=24+0)","memory_value.rstat":"n(v18 rc2024-04-24T08:57:07.878690+0000 b5230239 25=24+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-04-24T08:57:24.205 INFO:journalctl@ceph.mds.b.smithi138.stdout:Apr 24 08:57:23 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-b[61515]: 2024-04-24T08:57:23.840+0000 7f7fbbc00700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000020f [...8,head] /volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/PWRPNT/ auth{1=1} v4884 ap=2 f(v196 m2024-04-24T08:56:59.316427+0000 7=7+0) n(v14 rc2024-04-24T08:56:59.316427+0000 b1462582 8=7+1) old_inodes=4 (inest sync->mix g=1) caps={15603=pAsLsXsFs/-@4097} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x55cf8760b600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000020f:[<0x10000000009/PWRPNT v274>,<0x10000000008/~dmtmp v57>,<0x10000000007/client0 v33>,<0x10000000006/clients v23>,<0x10000000005/tmp v23>,<0x10000000003/client.0 v22>,<0x10000000002/e395bed6-5bba-41b8-a256-3505e4afcca2 v36>,<0x10000000001/sv_0 v55>,<0x10000000000/qa v43>,<0x1/volumes v39>]//[]","memoryvalue":"(2)0x1000000020f:[<0x10000000009/PWRPNT v4884>,<0x10000000008/~dmtmp v4522>,<0x10000000007/client0 v544>,<0x10000000006/clients v534>,<0x10000000005/tmp v532>,<0x10000000003/client.0 v529>,<0x10000000002/e395bed6-5bba-41b8-a256-3505e4afcca2 v542>,<0x10000000001/sv_0 v556>,<0x10000000000/qa v540>,<0x1/volumes v525>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-04-24T08:56:59.211896+0000 8=8+0)","ondisk_value.rstat":"n(v0 rc2024-04-24T08:56:59.316427+0000 b1462582 8=7+1)","memory_value.dirstat":"f(v196 m2024-04-24T08:56:59.316427+0000 7=7+0)","memory_value.rstat":"n(v14 rc2024-04-24T08:56:59.316427+0000 b1462582 8=7+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-04-24T08:57:24.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:57:23 smithi189 ceph-mon[31791]: pgmap v176: 129 pgs: 129 active+clean; 41 MiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 672 KiB/s wr, 66 op/s 2024-04-24T08:57:24.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:57:23 smithi083 ceph-mon[27907]: pgmap v176: 129 pgs: 129 active+clean; 41 MiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 672 KiB/s wr, 66 op/s 2024-04-24T08:57:24.580 INFO:tasks.workunit.client.0.smithi083.stdout: 1 28914 0.65 MB/sec execute 118 sec latency 51.453 ms 2024-04-24T08:57:25.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:57:24 smithi138 ceph-mon[31441]: scrub queued for path: / 2024-04-24T08:57:25.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:57:24 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T08:57:25.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:57:24 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T08:57:25.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:57:24 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T08:57:25.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:57:24 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T08:57:25.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:57:24 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T08:57:25.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:57:24 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T08:57:25.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:57:24 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T08:57:25.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:57:24 smithi138 ceph-mon[31441]: Scrub error on inode 0x10000000208 (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/WORD) see mds.b log and `damage ls` output for details 2024-04-24T08:57:25.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:57:24 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T08:57:25.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:57:24 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T08:57:25.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:57:24 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T08:57:25.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:57:24 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T08:57:25.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:57:24 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T08:57:25.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:57:24 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T08:57:25.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:57:24 smithi138 ceph-mon[31441]: Scrub error on inode 0x10000000229 (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/PARADOX) see mds.b log and `damage ls` output for details 2024-04-24T08:57:25.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:57:24 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T08:57:25.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:57:24 smithi138 ceph-mon[31441]: Scrub error on inode 0x1000000020f (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/PWRPNT) see mds.b log and `damage ls` output for details 2024-04-24T08:57:25.207 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:57:24 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T08:57:25.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:57:24 smithi189 ceph-mon[31791]: scrub queued for path: / 2024-04-24T08:57:25.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:57:24 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T08:57:25.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:57:24 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T08:57:25.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:57:24 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T08:57:25.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:57:24 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T08:57:25.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:57:24 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T08:57:25.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:57:24 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T08:57:25.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:57:24 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T08:57:25.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:57:24 smithi189 ceph-mon[31791]: Scrub error on inode 0x10000000208 (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/WORD) see mds.b log and `damage ls` output for details 2024-04-24T08:57:25.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:57:24 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T08:57:25.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:57:24 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T08:57:25.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:57:24 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T08:57:25.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:57:24 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T08:57:25.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:57:24 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T08:57:25.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:57:24 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T08:57:25.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:57:24 smithi189 ceph-mon[31791]: Scrub error on inode 0x10000000229 (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/PARADOX) see mds.b log and `damage ls` output for details 2024-04-24T08:57:25.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:57:24 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T08:57:25.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:57:24 smithi189 ceph-mon[31791]: Scrub error on inode 0x1000000020f (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/PWRPNT) see mds.b log and `damage ls` output for details 2024-04-24T08:57:25.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:57:24 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T08:57:25.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:57:24 smithi083 ceph-mon[27907]: scrub queued for path: / 2024-04-24T08:57:25.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:57:24 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T08:57:25.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:57:24 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T08:57:25.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:57:24 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T08:57:25.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:57:24 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T08:57:25.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:57:24 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T08:57:25.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:57:24 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T08:57:25.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:57:24 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T08:57:25.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:57:24 smithi083 ceph-mon[27907]: Scrub error on inode 0x10000000208 (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/WORD) see mds.b log and `damage ls` output for details 2024-04-24T08:57:25.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:57:24 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T08:57:25.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:57:24 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T08:57:25.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:57:24 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T08:57:25.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:57:24 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T08:57:25.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:57:24 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T08:57:25.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:57:24 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T08:57:25.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:57:24 smithi083 ceph-mon[27907]: Scrub error on inode 0x10000000229 (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/PARADOX) see mds.b log and `damage ls` output for details 2024-04-24T08:57:25.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:57:24 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T08:57:25.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:57:24 smithi083 ceph-mon[27907]: Scrub error on inode 0x1000000020f (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/PWRPNT) see mds.b log and `damage ls` output for details 2024-04-24T08:57:25.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:57:24 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T08:57:25.580 INFO:tasks.workunit.client.0.smithi083.stdout: 1 28992 0.64 MB/sec execute 119 sec latency 53.028 ms 2024-04-24T08:57:26.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:57:25 smithi138 ceph-mon[31441]: pgmap v177: 129 pgs: 129 active+clean; 41 MiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 451 KiB/s wr, 44 op/s 2024-04-24T08:57:26.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:57:25 smithi189 ceph-mon[31791]: pgmap v177: 129 pgs: 129 active+clean; 41 MiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 451 KiB/s wr, 44 op/s 2024-04-24T08:57:26.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:57:25 smithi083 ceph-mon[27907]: pgmap v177: 129 pgs: 129 active+clean; 41 MiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 451 KiB/s wr, 44 op/s 2024-04-24T08:57:26.580 INFO:tasks.workunit.client.0.smithi083.stdout: 1 29081 0.64 MB/sec execute 120 sec latency 61.512 ms 2024-04-24T08:57:27.580 INFO:tasks.workunit.client.0.smithi083.stdout: 1 29206 0.64 MB/sec execute 121 sec latency 58.205 ms 2024-04-24T08:57:28.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:57:27 smithi138 ceph-mon[31441]: pgmap v178: 129 pgs: 129 active+clean; 41 MiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 451 KiB/s wr, 44 op/s 2024-04-24T08:57:28.210 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:57:27 smithi189 ceph-mon[31791]: pgmap v178: 129 pgs: 129 active+clean; 41 MiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 451 KiB/s wr, 44 op/s 2024-04-24T08:57:28.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:57:27 smithi083 ceph-mon[27907]: pgmap v178: 129 pgs: 129 active+clean; 41 MiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 451 KiB/s wr, 44 op/s 2024-04-24T08:57:28.580 INFO:tasks.workunit.client.0.smithi083.stdout: 1 29325 0.63 MB/sec execute 122 sec latency 56.615 ms 2024-04-24T08:57:29.580 INFO:tasks.workunit.client.0.smithi083.stdout: 1 29426 0.63 MB/sec execute 123 sec latency 54.879 ms 2024-04-24T08:57:30.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:57:29 smithi138 ceph-mon[31441]: pgmap v179: 129 pgs: 129 active+clean; 41 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 555 KiB/s wr, 77 op/s 2024-04-24T08:57:30.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:57:29 smithi189 ceph-mon[31791]: pgmap v179: 129 pgs: 129 active+clean; 41 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 555 KiB/s wr, 77 op/s 2024-04-24T08:57:30.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:57:29 smithi083 ceph-mon[27907]: pgmap v179: 129 pgs: 129 active+clean; 41 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 555 KiB/s wr, 77 op/s 2024-04-24T08:57:30.580 INFO:tasks.workunit.client.0.smithi083.stdout: 1 29542 0.63 MB/sec execute 124 sec latency 54.777 ms 2024-04-24T08:57:31.580 INFO:tasks.workunit.client.0.smithi083.stdout: 1 29648 0.63 MB/sec execute 125 sec latency 51.554 ms 2024-04-24T08:57:32.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:57:31 smithi138 ceph-mon[31441]: pgmap v180: 129 pgs: 129 active+clean; 41 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 385 KiB/s wr, 60 op/s 2024-04-24T08:57:32.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:57:31 smithi189 ceph-mon[31791]: pgmap v180: 129 pgs: 129 active+clean; 41 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 385 KiB/s wr, 60 op/s 2024-04-24T08:57:32.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:57:31 smithi083 ceph-mon[27907]: pgmap v180: 129 pgs: 129 active+clean; 41 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 385 KiB/s wr, 60 op/s 2024-04-24T08:57:32.581 INFO:tasks.workunit.client.0.smithi083.stdout: 1 29817 0.65 MB/sec execute 126 sec latency 72.624 ms 2024-04-24T08:57:33.119 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.119+0000 7fb533c00640 1 -- 172.21.15.83:0/3628051148 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb52c104470 msgr2=0x7fb52c104850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:33.119 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.119+0000 7fb533c00640 1 --2- 172.21.15.83:0/3628051148 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb52c104470 0x7fb52c104850 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7fb520009900 tx=0x7fb52002a3f0 comp rx=0 tx=0).stop 2024-04-24T08:57:33.119 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.119+0000 7fb533c00640 1 -- 172.21.15.83:0/3628051148 shutdown_connections 2024-04-24T08:57:33.120 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.119+0000 7fb533c00640 1 --2- 172.21.15.83:0/3628051148 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb52c109540 0x7fb52c111110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:33.120 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.119+0000 7fb533c00640 1 --2- 172.21.15.83:0/3628051148 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb52c104e20 0x7fb52c108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:33.120 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.119+0000 7fb533c00640 1 --2- 172.21.15.83:0/3628051148 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb52c104470 0x7fb52c104850 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:33.120 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.119+0000 7fb533c00640 1 -- 172.21.15.83:0/3628051148 >> 172.21.15.83:0/3628051148 conn(0x7fb52c0ffd30 msgr2=0x7fb52c102150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:33.120 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.119+0000 7fb533c00640 1 -- 172.21.15.83:0/3628051148 shutdown_connections 2024-04-24T08:57:33.120 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.119+0000 7fb533c00640 1 -- 172.21.15.83:0/3628051148 wait complete. 2024-04-24T08:57:33.121 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.120+0000 7fb533c00640 1 Processor -- start 2024-04-24T08:57:33.121 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.120+0000 7fb533c00640 1 -- start start 2024-04-24T08:57:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.121+0000 7fb533c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb52c104470 0x7fb52c1a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.121+0000 7fb533c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb52c109540 0x7fb52c1a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.121+0000 7fb533c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb52c19a810 0x7fb52c19acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.121+0000 7fb533c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb52c114550 con 0x7fb52c104470 2024-04-24T08:57:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.121+0000 7fb533c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb52c1143d0 con 0x7fb52c19a810 2024-04-24T08:57:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.121+0000 7fb533c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb52c1146d0 con 0x7fb52c109540 2024-04-24T08:57:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.121+0000 7fb531200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb52c104470 0x7fb52c1a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.121+0000 7fb531200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb52c104470 0x7fb52c1a0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33472/0 (socket says 172.21.15.83:33472) 2024-04-24T08:57:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.121+0000 7fb531200640 1 -- 172.21.15.83:0/1828083617 learned_addr learned my addr 172.21.15.83:0/1828083617 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:57:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.121+0000 7fb52be00640 1 --2- 172.21.15.83:0/1828083617 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb52c109540 0x7fb52c1a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.121+0000 7fb531200640 1 -- 172.21.15.83:0/1828083617 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb52c109540 msgr2=0x7fb52c1a0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.121+0000 7fb531200640 1 --2- 172.21.15.83:0/1828083617 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb52c109540 0x7fb52c1a0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.121+0000 7fb531200640 1 -- 172.21.15.83:0/1828083617 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb52c19a810 msgr2=0x7fb52c19acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.121+0000 7fb531200640 1 --2- 172.21.15.83:0/1828083617 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb52c19a810 0x7fb52c19acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.121+0000 7fb531200640 1 -- 172.21.15.83:0/1828083617 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb52c19b3a0 con 0x7fb52c104470 2024-04-24T08:57:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.121+0000 7fb52be00640 1 --2- 172.21.15.83:0/1828083617 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb52c109540 0x7fb52c1a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:57:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.122+0000 7fb531200640 1 --2- 172.21.15.83:0/1828083617 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb52c104470 0x7fb52c1a0650 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7fb520005ab0 tx=0x7fb520003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.122+0000 7fb529600640 1 -- 172.21.15.83:0/1828083617 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb52003f070 con 0x7fb52c104470 2024-04-24T08:57:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.122+0000 7fb529600640 1 -- 172.21.15.83:0/1828083617 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb520031d60 con 0x7fb52c104470 2024-04-24T08:57:33.124 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.122+0000 7fb533c00640 1 -- 172.21.15.83:0/1828083617 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb52c19b630 con 0x7fb52c104470 2024-04-24T08:57:33.124 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.122+0000 7fb529600640 1 -- 172.21.15.83:0/1828083617 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb520035030 con 0x7fb52c104470 2024-04-24T08:57:33.124 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.122+0000 7fb533c00640 1 -- 172.21.15.83:0/1828083617 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb52c1a77c0 con 0x7fb52c104470 2024-04-24T08:57:33.126 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.125+0000 7fb536a09740 1 Processor -- start 2024-04-24T08:57:33.126 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.126+0000 7fb536a09740 1 -- start start 2024-04-24T08:57:33.126 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.126+0000 7fb529600640 1 -- 172.21.15.83:0/1828083617 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb520042070 con 0x7fb52c104470 2024-04-24T08:57:33.126 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.126+0000 7fb536a09740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555cf57d91b0 0x555cf57d9590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:33.126 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.126+0000 7fb536a09740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555cf57d9ad0 0x555cf57e3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:33.126 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.126+0000 7fb536a09740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555cf57e4390 0x555cf57e6780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:33.126 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.126+0000 7fb536a09740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x555cf573f900 con 0x555cf57d91b0 2024-04-24T08:57:33.127 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.126+0000 7fb536a09740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555cf56e5f20 con 0x555cf57d9ad0 2024-04-24T08:57:33.127 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.126+0000 7fb536a09740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x555cf573b8c0 con 0x555cf57e4390 2024-04-24T08:57:33.129 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.126+0000 7fb531c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555cf57d91b0 0x555cf57d9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:33.129 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.126+0000 7fb531c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555cf57d91b0 0x555cf57d9590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33476/0 (socket says 172.21.15.83:33476) 2024-04-24T08:57:33.129 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.126+0000 7fb531c00640 1 -- 172.21.15.83:0/1055709320 learned_addr learned my addr 172.21.15.83:0/1055709320 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:57:33.129 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.127+0000 7fb531200640 1 --2- 172.21.15.83:0/1055709320 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555cf57d9ad0 0x555cf57e3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:33.129 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.127+0000 7fb52be00640 1 --2- 172.21.15.83:0/1055709320 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555cf57e4390 0x555cf57e6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:33.129 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.127+0000 7fb531c00640 1 -- 172.21.15.83:0/1055709320 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555cf57e4390 msgr2=0x555cf57e6780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:33.129 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.127+0000 7fb531c00640 1 --2- 172.21.15.83:0/1055709320 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555cf57e4390 0x555cf57e6780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:33.129 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.127+0000 7fb531c00640 1 -- 172.21.15.83:0/1055709320 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555cf57d9ad0 msgr2=0x555cf57e3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:33.129 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.127+0000 7fb531c00640 1 --2- 172.21.15.83:0/1055709320 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555cf57d9ad0 0x555cf57e3e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:33.130 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.127+0000 7fb531c00640 1 -- 172.21.15.83:0/1055709320 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555cf570ae10 con 0x555cf57d91b0 2024-04-24T08:57:33.130 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.127+0000 7fb531200640 1 --2- 172.21.15.83:0/1055709320 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555cf57d9ad0 0x555cf57e3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:57:33.130 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.127+0000 7fb531c00640 1 --2- 172.21.15.83:0/1055709320 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555cf57d91b0 0x555cf57d9590 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7fb524007cf0 tx=0x7fb52400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e7eceae146c1f69 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:33.130 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.127+0000 7fb51e000640 1 -- 172.21.15.83:0/1055709320 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb524014070 con 0x555cf57d91b0 2024-04-24T08:57:33.130 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.128+0000 7fb51e000640 1 -- 172.21.15.83:0/1055709320 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb52400fb00 con 0x555cf57d91b0 2024-04-24T08:57:33.130 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.128+0000 7fb51e000640 1 -- 172.21.15.83:0/1055709320 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb52400eac0 con 0x555cf57d91b0 2024-04-24T08:57:33.130 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.128+0000 7fb529600640 1 --2- 172.21.15.83:0/1828083617 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb5140a3e40 0x7fb5140a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:33.130 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.128+0000 7fb529600640 1 -- 172.21.15.83:0/1828083617 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7fb5200f1650 con 0x7fb52c104470 2024-04-24T08:57:33.130 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.128+0000 7fb531200640 1 --2- 172.21.15.83:0/1828083617 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb5140a3e40 0x7fb5140a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:33.130 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.129+0000 7fb536a09740 1 -- 172.21.15.83:0/1055709320 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555cf57d91b0 msgr2=0x555cf57d9590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:33.130 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.129+0000 7fb536a09740 1 --2- 172.21.15.83:0/1055709320 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555cf57d91b0 0x555cf57d9590 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7fb524007cf0 tx=0x7fb52400c6a0 comp rx=0 tx=0).stop 2024-04-24T08:57:33.130 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.129+0000 7fb536a09740 1 -- 172.21.15.83:0/1055709320 shutdown_connections 2024-04-24T08:57:33.131 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.129+0000 7fb536a09740 1 --2- 172.21.15.83:0/1055709320 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555cf57e4390 0x555cf57e6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:33.131 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.129+0000 7fb536a09740 1 --2- 172.21.15.83:0/1055709320 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555cf57d9ad0 0x555cf57e3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:33.131 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.129+0000 7fb536a09740 1 --2- 172.21.15.83:0/1055709320 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555cf57d91b0 0x555cf57d9590 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:33.131 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.129+0000 7fb536a09740 1 -- 172.21.15.83:0/1055709320 >> 172.21.15.83:0/1055709320 conn(0x555cf57d56c0 msgr2=0x555cf57d5aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:33.131 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.129+0000 7fb536a09740 1 -- 172.21.15.83:0/1055709320 shutdown_connections 2024-04-24T08:57:33.131 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.129+0000 7fb531200640 1 --2- 172.21.15.83:0/1828083617 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb5140a3e40 0x7fb5140a6300 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x555cf57da750 tx=0x7fb520009260 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:33.131 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.129+0000 7fb536a09740 1 -- 172.21.15.83:0/1055709320 wait complete. 2024-04-24T08:57:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.129+0000 7fb536a09740 1 -- start start 2024-04-24T08:57:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.130+0000 7fb536a09740 1 Processor -- start 2024-04-24T08:57:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.130+0000 7fb536a09740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555cf57d91b0 0x555cf57e1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.130+0000 7fb536a09740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555cf57d9ad0 0x555cf57e1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.130+0000 7fb536a09740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555cf57e4390 0x555cf57e1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.130+0000 7fb536a09740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x555cf57e6fc0 con 0x555cf57d9ad0 2024-04-24T08:57:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.130+0000 7fb536a09740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555cf56f92c0 con 0x555cf57e4390 2024-04-24T08:57:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.130+0000 7fb536a09740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x555cf57e7140 con 0x555cf57d91b0 2024-04-24T08:57:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.130+0000 7fb536a09740 10 client.0 _ll_register_callbacks cb 0x555cf5451a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:57:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.130+0000 7fb52be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555cf57d9ad0 0x555cf57e1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.131+0000 7fb52be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555cf57d9ad0 0x555cf57e1790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33482/0 (socket says 172.21.15.83:33482) 2024-04-24T08:57:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.131+0000 7fb52be00640 1 -- 172.21.15.83:0/2420845677 learned_addr learned my addr 172.21.15.83:0/2420845677 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:57:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.131+0000 7fb531c00640 1 --2- 172.21.15.83:0/2420845677 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555cf57d91b0 0x555cf57e1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.131+0000 7fb531c00640 1 --2- 172.21.15.83:0/2420845677 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555cf57e4390 0x555cf57e1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.131+0000 7fb52be00640 1 -- 172.21.15.83:0/2420845677 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555cf57d91b0 msgr2=0x555cf57e1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.131+0000 7fb52be00640 1 --2- 172.21.15.83:0/2420845677 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555cf57d91b0 0x555cf57e1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.131+0000 7fb52be00640 1 -- 172.21.15.83:0/2420845677 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555cf57e4390 msgr2=0x555cf57e1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.131+0000 7fb52be00640 1 --2- 172.21.15.83:0/2420845677 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555cf57e4390 0x555cf57e1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.131+0000 7fb52be00640 1 -- 172.21.15.83:0/2420845677 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555cf5709fd0 con 0x555cf57d9ad0 2024-04-24T08:57:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.131+0000 7fb531c00640 1 --2- 172.21.15.83:0/2420845677 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555cf57e4390 0x555cf57e1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:57:33.136 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.131+0000 7fb52be00640 1 --2- 172.21.15.83:0/2420845677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555cf57d9ad0 0x555cf57e1790 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7fb51800e2a0 tx=0x7fb51800e760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de97dd338e1b01d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:33.136 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.131+0000 7fb512a00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:57:33.136 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.132+0000 7fb512a00640 1 -- 172.21.15.83:0/2420845677 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb518018030 con 0x555cf57d9ad0 2024-04-24T08:57:33.136 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.132+0000 7fb512a00640 1 -- 172.21.15.83:0/2420845677 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7fb51800f040 con 0x555cf57d9ad0 2024-04-24T08:57:33.136 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.132+0000 7fb536a09740 1 -- 172.21.15.83:0/2420845677 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb52400fb00 con 0x555cf57d9ad0 2024-04-24T08:57:33.136 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.132+0000 7fb512a00640 1 -- 172.21.15.83:0/2420845677 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fb518022050 con 0x555cf57d9ad0 2024-04-24T08:57:33.136 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.132+0000 7fb536a09740 10 client.17805 fetch_fsmap learned FSMap version 33 2024-04-24T08:57:33.136 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.132+0000 7fb536a09740 1 -- 172.21.15.83:0/2420845677 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x555cf5452210 con 0x555cf57d9ad0 2024-04-24T08:57:33.136 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.132+0000 7fb512a00640 1 -- 172.21.15.83:0/2420845677 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb52400fb00 con 0x555cf57d9ad0 2024-04-24T08:57:33.136 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.132+0000 7fb512a00640 1 -- 172.21.15.83:0/2420845677 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fb51801d7d0 con 0x555cf57d9ad0 2024-04-24T08:57:33.136 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.133+0000 7fb536a09740 10 client.17805 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T08:57:33.136 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.133+0000 7fb536a09740 10 client.17805 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T08:57:33.136 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.133+0000 7fb536a09740 20 client.17805 populate_metadata read hostname 'smithi083' 2024-04-24T08:57:33.137 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.133+0000 7fb536a09740 1 --2- 172.21.15.83:0/2420845677 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x555cf57f4500 0x555cf57f6930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:33.137 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.133+0000 7fb536a09740 4 client.17805 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:57:33.137 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.133+0000 7fb536a09740 1 -- 172.21.15.83:0/2420845677 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555cf574d320 con 0x555cf57f4500 2024-04-24T08:57:33.137 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.133+0000 7fb531c00640 1 --2- 172.21.15.83:0/2420845677 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x555cf57f4500 0x555cf57f6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:33.137 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.134+0000 7fb531c00640 1 --2- 172.21.15.83:0/2420845677 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x555cf57f4500 0x555cf57f6930 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f77b317813f084ad server_cookie=d056c2ea1ed57bd5 in_seq=0 out_seq=0 2024-04-24T08:57:33.137 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.134+0000 7fb512a00640 10 client.17805 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:57:33.138 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.137+0000 7fb512a00640 1 -- 172.21.15.83:0/2420845677 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555cf574d320 con 0x555cf57f4500 2024-04-24T08:57:33.138 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.138+0000 7fb512a00640 10 client.17805 handle_command_reply: tid=0 2024-04-24T08:57:33.139 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.138+0000 7fb536a09740 1 client.17805 shutdown 2024-04-24T08:57:33.139 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.138+0000 7fb536a09740 1 -- 172.21.15.83:0/2420845677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555cf57d9ad0 msgr2=0x555cf57e1790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:33.140 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.138+0000 7fb536a09740 1 --2- 172.21.15.83:0/2420845677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555cf57d9ad0 0x555cf57e1790 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7fb51800e2a0 tx=0x7fb51800e760 comp rx=0 tx=0).stop 2024-04-24T08:57:33.140 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.138+0000 7fb536a09740 1 -- 172.21.15.83:0/2420845677 shutdown_connections 2024-04-24T08:57:33.140 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.138+0000 7fb536a09740 1 --2- 172.21.15.83:0/2420845677 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x555cf57f4500 0x555cf57f6930 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:33.140 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.138+0000 7fb536a09740 1 --2- 172.21.15.83:0/2420845677 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555cf57e4390 0x555cf57e1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:33.140 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.138+0000 7fb536a09740 1 --2- 172.21.15.83:0/2420845677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555cf57d9ad0 0x555cf57e1790 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:33.140 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.138+0000 7fb536a09740 1 --2- 172.21.15.83:0/2420845677 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555cf57d91b0 0x555cf57e1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:33.140 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.138+0000 7fb512a00640 0 client.17805 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:57:33.140 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.138+0000 7fb536a09740 1 -- 172.21.15.83:0/2420845677 >> 172.21.15.83:0/2420845677 conn(0x555cf57d56c0 msgr2=0x555cf57db440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:33.140 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.138+0000 7fb536a09740 1 -- 172.21.15.83:0/2420845677 shutdown_connections 2024-04-24T08:57:33.140 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.138+0000 7fb536a09740 1 -- 172.21.15.83:0/2420845677 wait complete. 2024-04-24T08:57:33.140 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.139+0000 7fb536a09740 20 client.17805 trim_cache size 0 max 16384 2024-04-24T08:57:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.155+0000 7fb536a09740 1 Processor -- start 2024-04-24T08:57:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.155+0000 7fb536a09740 1 -- start start 2024-04-24T08:57:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.155+0000 7fb536a09740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555cf57d91b0 0x555cf57d46c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.155+0000 7fb536a09740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555cf57d9ad0 0x555cf57fc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.155+0000 7fb536a09740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555cf57e4390 0x555cf57fc780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.155+0000 7fb536a09740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb52c1b1650 con 0x555cf57d91b0 2024-04-24T08:57:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.155+0000 7fb536a09740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb52c1b3190 con 0x555cf57d9ad0 2024-04-24T08:57:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.155+0000 7fb536a09740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb52c1b1d20 con 0x555cf57e4390 2024-04-24T08:57:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.156+0000 7fb531c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555cf57d91b0 0x555cf57d46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.156+0000 7fb531c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555cf57d91b0 0x555cf57d46c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33494/0 (socket says 172.21.15.83:33494) 2024-04-24T08:57:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.156+0000 7fb531c00640 1 -- 172.21.15.83:0/2014727966 learned_addr learned my addr 172.21.15.83:0/2014727966 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:57:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.156+0000 7fb531c00640 1 -- 172.21.15.83:0/2014727966 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555cf57e4390 msgr2=0x555cf57fc780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.156+0000 7fb531c00640 1 --2- 172.21.15.83:0/2014727966 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555cf57e4390 0x555cf57fc780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:33.159 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.156+0000 7fb531c00640 1 -- 172.21.15.83:0/2014727966 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555cf57d9ad0 msgr2=0x555cf57fc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:33.159 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.156+0000 7fb531c00640 1 --2- 172.21.15.83:0/2014727966 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555cf57d9ad0 0x555cf57fc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:33.159 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.156+0000 7fb531c00640 1 -- 172.21.15.83:0/2014727966 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555cf57d9900 con 0x555cf57d91b0 2024-04-24T08:57:33.159 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.156+0000 7fb531c00640 1 --2- 172.21.15.83:0/2014727966 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555cf57d91b0 0x555cf57d46c0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fb52400e650 tx=0x7fb52400e3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf0b7ee5bb4583b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:33.159 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.156+0000 7fb51d600640 1 -- 172.21.15.83:0/2014727966 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb524014040 con 0x555cf57d91b0 2024-04-24T08:57:33.159 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.157+0000 7fb51d600640 1 -- 172.21.15.83:0/2014727966 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb524003970 con 0x555cf57d91b0 2024-04-24T08:57:33.159 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.157+0000 7fb51d600640 1 -- 172.21.15.83:0/2014727966 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5240268a0 con 0x555cf57d91b0 2024-04-24T08:57:33.159 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.157+0000 7fb536a09740 1 -- 172.21.15.83:0/2014727966 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555cf57d91b0 msgr2=0x555cf57d46c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:33.159 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.157+0000 7fb536a09740 1 --2- 172.21.15.83:0/2014727966 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555cf57d91b0 0x555cf57d46c0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fb52400e650 tx=0x7fb52400e3b0 comp rx=0 tx=0).stop 2024-04-24T08:57:33.159 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.157+0000 7fb536a09740 1 -- 172.21.15.83:0/2014727966 shutdown_connections 2024-04-24T08:57:33.159 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.157+0000 7fb536a09740 1 --2- 172.21.15.83:0/2014727966 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555cf57e4390 0x555cf57fc780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:33.159 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.157+0000 7fb536a09740 1 --2- 172.21.15.83:0/2014727966 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555cf57d9ad0 0x555cf57fc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:33.159 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.157+0000 7fb536a09740 1 --2- 172.21.15.83:0/2014727966 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555cf57d91b0 0x555cf57d46c0 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:33.159 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.157+0000 7fb536a09740 1 -- 172.21.15.83:0/2014727966 >> 172.21.15.83:0/2014727966 conn(0x555cf57d56c0 msgr2=0x555cf57d3fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:33.160 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.157+0000 7fb536a09740 1 -- 172.21.15.83:0/2014727966 shutdown_connections 2024-04-24T08:57:33.160 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.157+0000 7fb536a09740 1 -- 172.21.15.83:0/2014727966 wait complete. 2024-04-24T08:57:33.160 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.158+0000 7fb536a09740 1 -- start start 2024-04-24T08:57:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.158+0000 7fb536a09740 1 Processor -- start 2024-04-24T08:57:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.158+0000 7fb536a09740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555cf57d91b0 0x555cf57e86b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.158+0000 7fb536a09740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555cf57d9ad0 0x555cf57e8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.158+0000 7fb536a09740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555cf57e4390 0x555cf5853af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.158+0000 7fb536a09740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb52c1b2ac0 con 0x555cf57d9ad0 2024-04-24T08:57:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.158+0000 7fb536a09740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb52c1b23f0 con 0x555cf57d91b0 2024-04-24T08:57:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.158+0000 7fb536a09740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb52c1af630 con 0x555cf57e4390 2024-04-24T08:57:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.158+0000 7fb536a09740 10 client.0 _ll_register_callbacks cb 0x7fb51800c190 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:57:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.159+0000 7fb52be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555cf57d9ad0 0x555cf57e8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.159+0000 7fb52be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555cf57d9ad0 0x555cf57e8bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33498/0 (socket says 172.21.15.83:33498) 2024-04-24T08:57:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.159+0000 7fb52be00640 1 -- 172.21.15.83:0/3558132828 learned_addr learned my addr 172.21.15.83:0/3558132828 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:57:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.159+0000 7fb531c00640 1 --2- 172.21.15.83:0/3558132828 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555cf57d91b0 0x555cf57e86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.159+0000 7fb52be00640 1 -- 172.21.15.83:0/3558132828 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555cf57e4390 msgr2=0x555cf5853af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.159+0000 7fb52be00640 1 --2- 172.21.15.83:0/3558132828 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555cf57e4390 0x555cf5853af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.159+0000 7fb52be00640 1 -- 172.21.15.83:0/3558132828 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555cf57d91b0 msgr2=0x555cf57e86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.159+0000 7fb52be00640 1 --2- 172.21.15.83:0/3558132828 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555cf57d91b0 0x555cf57e86b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.159+0000 7fb52be00640 1 -- 172.21.15.83:0/3558132828 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555cf551e620 con 0x555cf57d9ad0 2024-04-24T08:57:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.159+0000 7fb52be00640 1 --2- 172.21.15.83:0/3558132828 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555cf57d9ad0 0x555cf57e8bf0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7fb518001eb0 tx=0x7fb51800ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=151a82812a721396 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.159+0000 7fb512a00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:57:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.159+0000 7fb512a00640 1 -- 172.21.15.83:0/3558132828 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb518018070 con 0x555cf57d9ad0 2024-04-24T08:57:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.160+0000 7fb512a00640 1 -- 172.21.15.83:0/3558132828 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7fb51801d400 con 0x555cf57d9ad0 2024-04-24T08:57:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.160+0000 7fb536a09740 1 -- 172.21.15.83:0/3558132828 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb524003970 con 0x555cf57d9ad0 2024-04-24T08:57:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.160+0000 7fb512a00640 1 -- 172.21.15.83:0/3558132828 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fb518032050 con 0x555cf57d9ad0 2024-04-24T08:57:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.160+0000 7fb536a09740 10 client.17811 fetch_fsmap learned FSMap version 33 2024-04-24T08:57:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.160+0000 7fb536a09740 1 -- 172.21.15.83:0/3558132828 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x555cf585b290 con 0x555cf57d9ad0 2024-04-24T08:57:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.160+0000 7fb512a00640 1 -- 172.21.15.83:0/3558132828 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb524003970 con 0x555cf57d9ad0 2024-04-24T08:57:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.160+0000 7fb512a00640 1 -- 172.21.15.83:0/3558132828 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fb518014ca0 con 0x555cf57d9ad0 2024-04-24T08:57:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.161+0000 7fb536a09740 10 client.17811 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T08:57:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.161+0000 7fb536a09740 10 client.17811 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T08:57:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.161+0000 7fb536a09740 20 client.17811 populate_metadata read hostname 'smithi083' 2024-04-24T08:57:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.161+0000 7fb536a09740 1 --2- 172.21.15.83:0/3558132828 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x555cf585b950 0x555cf585dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.161+0000 7fb536a09740 4 client.17811 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T08:57:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.161+0000 7fb536a09740 1 -- 172.21.15.83:0/3558132828 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555cf5747190 con 0x555cf585b950 2024-04-24T08:57:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.161+0000 7fb531c00640 1 --2- 172.21.15.83:0/3558132828 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x555cf585b950 0x555cf585dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.162+0000 7fb531c00640 1 --2- 172.21.15.83:0/3558132828 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x555cf585b950 0x555cf585dd40 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80175bf97ad34fe8 server_cookie=3ac8d82a44acfb62 in_seq=0 out_seq=0 2024-04-24T08:57:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.162+0000 7fb512a00640 10 client.17811 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:57:33.227 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.225+0000 7fb512a00640 1 -- 172.21.15.83:0/3558132828 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555cf5747190 con 0x555cf585b950 2024-04-24T08:57:33.228 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.225+0000 7fb512a00640 10 client.17811 handle_command_reply: tid=0 2024-04-24T08:57:33.228 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.225+0000 7fb536a09740 1 client.17811 shutdown 2024-04-24T08:57:33.228 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.226+0000 7fb536a09740 1 -- 172.21.15.83:0/3558132828 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555cf57d9ad0 msgr2=0x555cf57e8bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:33.229 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.226+0000 7fb536a09740 1 --2- 172.21.15.83:0/3558132828 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555cf57d9ad0 0x555cf57e8bf0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7fb518001eb0 tx=0x7fb51800ccf0 comp rx=0 tx=0).stop 2024-04-24T08:57:33.229 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.226+0000 7fb536a09740 1 -- 172.21.15.83:0/3558132828 shutdown_connections 2024-04-24T08:57:33.229 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.226+0000 7fb536a09740 1 --2- 172.21.15.83:0/3558132828 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x555cf585b950 0x555cf585dd40 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:33.229 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.226+0000 7fb536a09740 1 --2- 172.21.15.83:0/3558132828 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555cf57e4390 0x555cf5853af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:33.229 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.226+0000 7fb536a09740 1 --2- 172.21.15.83:0/3558132828 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555cf57d9ad0 0x555cf57e8bf0 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:33.229 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.226+0000 7fb536a09740 1 --2- 172.21.15.83:0/3558132828 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555cf57d91b0 0x555cf57e86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:33.230 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.226+0000 7fb536a09740 1 -- 172.21.15.83:0/3558132828 >> 172.21.15.83:0/3558132828 conn(0x555cf57d56c0 msgr2=0x555cf57e1a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:33.230 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.226+0000 7fb512a00640 0 client.17811 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:57:33.230 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.226+0000 7fb536a09740 1 -- 172.21.15.83:0/3558132828 shutdown_connections 2024-04-24T08:57:33.230 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.226+0000 7fb536a09740 1 -- 172.21.15.83:0/3558132828 wait complete. 2024-04-24T08:57:33.230 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.227+0000 7fb536a09740 20 client.17811 trim_cache size 0 max 16384 2024-04-24T08:57:33.231 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T08:57:33.231 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T08:57:33.232 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T08:57:33.232 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T08:57:33.233 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.228+0000 7fb533c00640 1 -- 172.21.15.83:0/1828083617 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb5140a3e40 msgr2=0x7fb5140a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:33.233 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.228+0000 7fb533c00640 1 --2- 172.21.15.83:0/1828083617 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb5140a3e40 0x7fb5140a6300 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x555cf57da750 tx=0x7fb520009260 comp rx=0 tx=0).stop 2024-04-24T08:57:33.233 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.228+0000 7fb533c00640 1 -- 172.21.15.83:0/1828083617 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb52c104470 msgr2=0x7fb52c1a0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:33.233 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.228+0000 7fb533c00640 1 --2- 172.21.15.83:0/1828083617 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb52c104470 0x7fb52c1a0650 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7fb520005ab0 tx=0x7fb520003d50 comp rx=0 tx=0).stop 2024-04-24T08:57:33.233 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.228+0000 7fb533c00640 1 -- 172.21.15.83:0/1828083617 shutdown_connections 2024-04-24T08:57:33.233 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.228+0000 7fb533c00640 1 --2- 172.21.15.83:0/1828083617 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb5140a3e40 0x7fb5140a6300 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:33.233 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.228+0000 7fb533c00640 1 --2- 172.21.15.83:0/1828083617 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb52c19a810 0x7fb52c19acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:33.233 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.228+0000 7fb533c00640 1 --2- 172.21.15.83:0/1828083617 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb52c109540 0x7fb52c1a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:33.233 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.228+0000 7fb533c00640 1 --2- 172.21.15.83:0/1828083617 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb52c104470 0x7fb52c1a0650 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:33.234 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.228+0000 7fb533c00640 1 -- 172.21.15.83:0/1828083617 >> 172.21.15.83:0/1828083617 conn(0x7fb52c0ffd30 msgr2=0x7fb52c102030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:33.234 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.228+0000 7fb533c00640 1 -- 172.21.15.83:0/1828083617 shutdown_connections 2024-04-24T08:57:33.234 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:33.228+0000 7fb533c00640 1 -- 172.21.15.83:0/1828083617 wait complete. 2024-04-24T08:57:33.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:57:32 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T08:57:33.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:57:32 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T08:57:33.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:57:32 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T08:57:33.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:57:32 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T08:57:33.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:57:32 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T08:57:33.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:57:32 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T08:57:33.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:57:32 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T08:57:33.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:57:32 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T08:57:33.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:57:32 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T08:57:33.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:57:32 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T08:57:33.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:57:32 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T08:57:33.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:57:32 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T08:57:33.581 INFO:tasks.workunit.client.0.smithi083.stdout: 1 29913 0.64 MB/sec execute 127 sec latency 62.531 ms 2024-04-24T08:57:34.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:57:33 smithi083 ceph-mon[27907]: pgmap v181: 129 pgs: 129 active+clean; 40 MiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 559 KiB/s wr, 86 op/s 2024-04-24T08:57:34.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:57:33 smithi138 ceph-mon[31441]: pgmap v181: 129 pgs: 129 active+clean; 40 MiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 559 KiB/s wr, 86 op/s 2024-04-24T08:57:34.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:57:33 smithi189 ceph-mon[31791]: pgmap v181: 129 pgs: 129 active+clean; 40 MiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 559 KiB/s wr, 86 op/s 2024-04-24T08:57:34.581 INFO:tasks.workunit.client.0.smithi083.stdout: 1 30020 0.64 MB/sec execute 128 sec latency 64.178 ms 2024-04-24T08:57:35.581 INFO:tasks.workunit.client.0.smithi083.stdout: 1 30121 0.63 MB/sec execute 129 sec latency 71.449 ms 2024-04-24T08:57:35.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:57:35 smithi138 ceph-mon[31441]: pgmap v182: 129 pgs: 129 active+clean; 40 MiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 278 KiB/s wr, 58 op/s 2024-04-24T08:57:35.706 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:57:35 smithi189 ceph-mon[31791]: pgmap v182: 129 pgs: 129 active+clean; 40 MiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 278 KiB/s wr, 58 op/s 2024-04-24T08:57:35.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:57:35 smithi083 ceph-mon[27907]: pgmap v182: 129 pgs: 129 active+clean; 40 MiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 278 KiB/s wr, 58 op/s 2024-04-24T08:57:36.581 INFO:tasks.workunit.client.0.smithi083.stdout: 1 30259 0.63 MB/sec execute 130 sec latency 66.687 ms 2024-04-24T08:57:37.581 INFO:tasks.workunit.client.0.smithi083.stdout: 1 30384 0.63 MB/sec execute 131 sec latency 50.047 ms 2024-04-24T08:57:37.954 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:57:37 smithi138 ceph-mon[31441]: pgmap v183: 129 pgs: 129 active+clean; 40 MiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 278 KiB/s wr, 58 op/s 2024-04-24T08:57:37.956 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:57:37 smithi189 ceph-mon[31791]: pgmap v183: 129 pgs: 129 active+clean; 40 MiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 278 KiB/s wr, 58 op/s 2024-04-24T08:57:38.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:57:37 smithi083 ceph-mon[27907]: pgmap v183: 129 pgs: 129 active+clean; 40 MiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 278 KiB/s wr, 58 op/s 2024-04-24T08:57:38.581 INFO:tasks.workunit.client.0.smithi083.stdout: 1 30502 0.62 MB/sec execute 132 sec latency 54.973 ms 2024-04-24T08:57:39.582 INFO:tasks.workunit.client.0.smithi083.stdout: 1 30624 0.62 MB/sec execute 133 sec latency 62.438 ms 2024-04-24T08:57:39.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:57:39 smithi138 ceph-mon[31441]: pgmap v184: 129 pgs: 129 active+clean; 40 MiB data, 595 MiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 348 KiB/s wr, 81 op/s 2024-04-24T08:57:39.956 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:57:39 smithi189 ceph-mon[31791]: pgmap v184: 129 pgs: 129 active+clean; 40 MiB data, 595 MiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 348 KiB/s wr, 81 op/s 2024-04-24T08:57:40.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:57:39 smithi083 ceph-mon[27907]: pgmap v184: 129 pgs: 129 active+clean; 40 MiB data, 595 MiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 348 KiB/s wr, 81 op/s 2024-04-24T08:57:40.582 INFO:tasks.workunit.client.0.smithi083.stdout: 1 30777 0.62 MB/sec execute 134 sec latency 54.315 ms 2024-04-24T08:57:41.582 INFO:tasks.workunit.client.0.smithi083.stdout: 1 30850 0.62 MB/sec execute 135 sec latency 53.261 ms 2024-04-24T08:57:41.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:57:41 smithi138 ceph-mon[31441]: pgmap v185: 129 pgs: 129 active+clean; 40 MiB data, 595 MiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 245 KiB/s wr, 48 op/s 2024-04-24T08:57:41.956 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:57:41 smithi189 ceph-mon[31791]: pgmap v185: 129 pgs: 129 active+clean; 40 MiB data, 595 MiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 245 KiB/s wr, 48 op/s 2024-04-24T08:57:42.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:57:41 smithi083 ceph-mon[27907]: pgmap v185: 129 pgs: 129 active+clean; 40 MiB data, 595 MiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 245 KiB/s wr, 48 op/s 2024-04-24T08:57:42.582 INFO:tasks.workunit.client.0.smithi083.stdout: 1 30971 0.62 MB/sec execute 136 sec latency 68.945 ms 2024-04-24T08:57:43.582 INFO:tasks.workunit.client.0.smithi083.stdout: 1 31133 0.64 MB/sec execute 137 sec latency 49.240 ms 2024-04-24T08:57:43.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:57:43 smithi138 ceph-mon[31441]: pgmap v186: 129 pgs: 129 active+clean; 42 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 445 KiB/s wr, 67 op/s 2024-04-24T08:57:43.956 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:57:43 smithi189 ceph-mon[31791]: pgmap v186: 129 pgs: 129 active+clean; 42 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 445 KiB/s wr, 67 op/s 2024-04-24T08:57:44.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:57:43 smithi083 ceph-mon[27907]: pgmap v186: 129 pgs: 129 active+clean; 42 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 445 KiB/s wr, 67 op/s 2024-04-24T08:57:44.582 INFO:tasks.workunit.client.0.smithi083.stdout: 1 31332 0.65 MB/sec execute 138 sec latency 50.264 ms 2024-04-24T08:57:45.582 INFO:tasks.workunit.client.0.smithi083.stdout: 1 31438 0.64 MB/sec execute 139 sec latency 56.887 ms 2024-04-24T08:57:45.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:57:45 smithi138 ceph-mon[31441]: pgmap v187: 129 pgs: 129 active+clean; 42 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 271 KiB/s wr, 42 op/s 2024-04-24T08:57:45.956 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:57:45 smithi189 ceph-mon[31791]: pgmap v187: 129 pgs: 129 active+clean; 42 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 271 KiB/s wr, 42 op/s 2024-04-24T08:57:46.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:57:45 smithi083 ceph-mon[27907]: pgmap v187: 129 pgs: 129 active+clean; 42 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 271 KiB/s wr, 42 op/s 2024-04-24T08:57:46.383 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.383+0000 7fec6e000640 1 -- 172.21.15.83:0/2142837217 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec68102070 msgr2=0x7fec68102450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:46.383 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.383+0000 7fec6e000640 1 --2- 172.21.15.83:0/2142837217 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec68102070 0x7fec68102450 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7fec5c009900 tx=0x7fec5c02a3f0 comp rx=0 tx=0).stop 2024-04-24T08:57:46.383 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.383+0000 7fec6e000640 1 -- 172.21.15.83:0/2142837217 shutdown_connections 2024-04-24T08:57:46.383 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.383+0000 7fec6e000640 1 --2- 172.21.15.83:0/2142837217 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fec68107140 0x7fec6810ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:46.383 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.383+0000 7fec6e000640 1 --2- 172.21.15.83:0/2142837217 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec68102a20 0x7fec68106a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:46.384 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.383+0000 7fec6e000640 1 --2- 172.21.15.83:0/2142837217 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec68102070 0x7fec68102450 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:46.384 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.383+0000 7fec6e000640 1 -- 172.21.15.83:0/2142837217 >> 172.21.15.83:0/2142837217 conn(0x7fec680fd950 msgr2=0x7fec680ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:46.384 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.383+0000 7fec6e000640 1 -- 172.21.15.83:0/2142837217 shutdown_connections 2024-04-24T08:57:46.384 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.383+0000 7fec6e000640 1 -- 172.21.15.83:0/2142837217 wait complete. 2024-04-24T08:57:46.385 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.384+0000 7fec6e000640 1 Processor -- start 2024-04-24T08:57:46.385 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.384+0000 7fec6e000640 1 -- start start 2024-04-24T08:57:46.386 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.385+0000 7fec6e000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec68102070 0x7fec681a05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:46.386 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.385+0000 7fec6e000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec68107140 0x7fec681a0b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:46.386 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.385+0000 7fec6e000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fec6819a6a0 0x7fec6819ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:46.386 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.385+0000 7fec6e000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fec68112100 con 0x7fec68107140 2024-04-24T08:57:46.386 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.385+0000 7fec6e000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fec68111f80 con 0x7fec68102070 2024-04-24T08:57:46.386 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.385+0000 7fec6e000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fec68112280 con 0x7fec6819a6a0 2024-04-24T08:57:46.386 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.385+0000 7fec67e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec68107140 0x7fec681a0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:46.386 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.385+0000 7fec67e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec68107140 0x7fec681a0b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39552/0 (socket says 172.21.15.83:39552) 2024-04-24T08:57:46.386 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.385+0000 7fec67e00640 1 -- 172.21.15.83:0/3068138877 learned_addr learned my addr 172.21.15.83:0/3068138877 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:57:46.386 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.385+0000 7fec6cc00640 1 --2- 172.21.15.83:0/3068138877 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec68102070 0x7fec681a05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:46.386 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.385+0000 7fec6d600640 1 --2- 172.21.15.83:0/3068138877 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fec6819a6a0 0x7fec6819ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:46.386 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.386+0000 7fec67e00640 1 -- 172.21.15.83:0/3068138877 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fec6819a6a0 msgr2=0x7fec6819ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:46.387 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.386+0000 7fec67e00640 1 --2- 172.21.15.83:0/3068138877 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fec6819a6a0 0x7fec6819ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:46.387 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.386+0000 7fec67e00640 1 -- 172.21.15.83:0/3068138877 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec68102070 msgr2=0x7fec681a05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:46.387 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.386+0000 7fec67e00640 1 --2- 172.21.15.83:0/3068138877 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec68102070 0x7fec681a05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:46.387 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.386+0000 7fec67e00640 1 -- 172.21.15.83:0/3068138877 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec6819b2a0 con 0x7fec68107140 2024-04-24T08:57:46.387 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.386+0000 7fec6cc00640 1 --2- 172.21.15.83:0/3068138877 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec68102070 0x7fec681a05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:57:46.388 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.386+0000 7fec67e00640 1 --2- 172.21.15.83:0/3068138877 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec68107140 0x7fec681a0b20 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7fec5400e9c0 tx=0x7fec5400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:46.388 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.386+0000 7fec65600640 1 -- 172.21.15.83:0/3068138877 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec5400cd80 con 0x7fec68107140 2024-04-24T08:57:46.388 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.386+0000 7fec65600640 1 -- 172.21.15.83:0/3068138877 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fec540054d0 con 0x7fec68107140 2024-04-24T08:57:46.388 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.386+0000 7fec6e000640 1 -- 172.21.15.83:0/3068138877 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec681a7380 con 0x7fec68107140 2024-04-24T08:57:46.388 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.386+0000 7fec65600640 1 -- 172.21.15.83:0/3068138877 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec54010430 con 0x7fec68107140 2024-04-24T08:57:46.388 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.386+0000 7fec6e000640 1 -- 172.21.15.83:0/3068138877 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec681a7780 con 0x7fec68107140 2024-04-24T08:57:46.390 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.390+0000 7fec723fc740 1 Processor -- start 2024-04-24T08:57:46.391 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.390+0000 7fec723fc740 1 -- start start 2024-04-24T08:57:46.391 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.390+0000 7fec65600640 1 -- 172.21.15.83:0/3068138877 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fec54004fd0 con 0x7fec68107140 2024-04-24T08:57:46.391 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.390+0000 7fec723fc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55de128b63c0 0x55de128b67a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:46.391 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.390+0000 7fec723fc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de128b6ce0 0x55de128c1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:46.391 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.390+0000 7fec723fc740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55de128c15a0 0x55de128c3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:46.391 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.390+0000 7fec723fc740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55de1281c900 con 0x55de128b63c0 2024-04-24T08:57:46.391 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.390+0000 7fec723fc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55de127c2f20 con 0x55de128b6ce0 2024-04-24T08:57:46.391 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.390+0000 7fec723fc740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55de128188c0 con 0x55de128c15a0 2024-04-24T08:57:46.391 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.391+0000 7fec6d600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55de128b63c0 0x55de128b67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:46.391 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.391+0000 7fec6d600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55de128b63c0 0x55de128b67a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39560/0 (socket says 172.21.15.83:39560) 2024-04-24T08:57:46.391 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.391+0000 7fec6d600640 1 -- 172.21.15.83:0/99503786 learned_addr learned my addr 172.21.15.83:0/99503786 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:57:46.392 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.391+0000 7fec6cc00640 1 --2- 172.21.15.83:0/99503786 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de128b6ce0 0x55de128c1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:46.392 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.391+0000 7fec67e00640 1 --2- 172.21.15.83:0/99503786 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55de128c15a0 0x55de128c3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:46.393 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.391+0000 7fec6d600640 1 -- 172.21.15.83:0/99503786 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55de128c15a0 msgr2=0x55de128c3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:46.393 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.391+0000 7fec6d600640 1 --2- 172.21.15.83:0/99503786 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55de128c15a0 0x55de128c3990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:46.393 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.391+0000 7fec6d600640 1 -- 172.21.15.83:0/99503786 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de128b6ce0 msgr2=0x55de128c1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:46.393 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.391+0000 7fec6d600640 1 --2- 172.21.15.83:0/99503786 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de128b6ce0 0x55de128c1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:46.393 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.391+0000 7fec6d600640 1 -- 172.21.15.83:0/99503786 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de127e7e10 con 0x55de128b63c0 2024-04-24T08:57:46.393 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.392+0000 7fec6d600640 1 --2- 172.21.15.83:0/99503786 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55de128b63c0 0x55de128b67a0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7fec60009f40 tx=0x7fec60010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9d373d76b698e98 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:46.393 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.392+0000 7fec65600640 1 --2- 172.21.15.83:0/3068138877 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fec500a3d10 0x7fec500a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:46.393 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.392+0000 7fec59c00640 1 -- 172.21.15.83:0/99503786 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec60003b70 con 0x55de128b63c0 2024-04-24T08:57:46.393 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.392+0000 7fec59c00640 1 -- 172.21.15.83:0/99503786 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fec60003d10 con 0x55de128b63c0 2024-04-24T08:57:46.393 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.392+0000 7fec59c00640 1 -- 172.21.15.83:0/99503786 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec600119c0 con 0x55de128b63c0 2024-04-24T08:57:46.393 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.392+0000 7fec65600640 1 -- 172.21.15.83:0/3068138877 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7fec540d1170 con 0x7fec68107140 2024-04-24T08:57:46.394 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.392+0000 7fec6cc00640 1 --2- 172.21.15.83:0/3068138877 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fec500a3d10 0x7fec500a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:46.397 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.393+0000 7fec723fc740 1 -- 172.21.15.83:0/99503786 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55de128b63c0 msgr2=0x55de128b67a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:46.397 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.393+0000 7fec723fc740 1 --2- 172.21.15.83:0/99503786 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55de128b63c0 0x55de128b67a0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7fec60009f40 tx=0x7fec60010860 comp rx=0 tx=0).stop 2024-04-24T08:57:46.397 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.393+0000 7fec6cc00640 1 --2- 172.21.15.83:0/3068138877 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fec500a3d10 0x7fec500a61d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fec5c000c00 tx=0x7fec5c005a40 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:46.397 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.393+0000 7fec723fc740 1 -- 172.21.15.83:0/99503786 shutdown_connections 2024-04-24T08:57:46.397 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.393+0000 7fec723fc740 1 --2- 172.21.15.83:0/99503786 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55de128c15a0 0x55de128c3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:46.397 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.393+0000 7fec723fc740 1 --2- 172.21.15.83:0/99503786 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de128b6ce0 0x55de128c1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:46.397 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.393+0000 7fec723fc740 1 --2- 172.21.15.83:0/99503786 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55de128b63c0 0x55de128b67a0 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:46.398 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.393+0000 7fec723fc740 1 -- 172.21.15.83:0/99503786 >> 172.21.15.83:0/99503786 conn(0x55de128b28d0 msgr2=0x55de128b2cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:46.398 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.393+0000 7fec723fc740 1 -- 172.21.15.83:0/99503786 shutdown_connections 2024-04-24T08:57:46.398 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.393+0000 7fec723fc740 1 -- 172.21.15.83:0/99503786 wait complete. 2024-04-24T08:57:46.398 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.394+0000 7fec723fc740 1 -- start start 2024-04-24T08:57:46.398 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.394+0000 7fec723fc740 1 Processor -- start 2024-04-24T08:57:46.398 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.394+0000 7fec723fc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de128b63c0 0x55de128be5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:46.398 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.394+0000 7fec723fc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55de128b6ce0 0x55de128beae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:46.398 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.394+0000 7fec723fc740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55de128c15a0 0x55de128bf020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:46.398 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.394+0000 7fec723fc740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55de128c40e0 con 0x55de128b6ce0 2024-04-24T08:57:46.398 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.394+0000 7fec723fc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55de127d62c0 con 0x55de128b63c0 2024-04-24T08:57:46.398 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.394+0000 7fec723fc740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55de128c4260 con 0x55de128c15a0 2024-04-24T08:57:46.398 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.394+0000 7fec723fc740 10 client.0 _ll_register_callbacks cb 0x55de1252ea00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:57:46.398 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.395+0000 7fec67e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55de128b6ce0 0x55de128beae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:46.399 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.395+0000 7fec67e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55de128b6ce0 0x55de128beae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39574/0 (socket says 172.21.15.83:39574) 2024-04-24T08:57:46.399 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.395+0000 7fec67e00640 1 -- 172.21.15.83:0/1651296799 learned_addr learned my addr 172.21.15.83:0/1651296799 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:57:46.399 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.395+0000 7fec6d600640 1 --2- 172.21.15.83:0/1651296799 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55de128c15a0 0x55de128bf020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:46.399 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.395+0000 7fec6cc00640 1 --2- 172.21.15.83:0/1651296799 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de128b63c0 0x55de128be5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:46.399 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.395+0000 7fec67e00640 1 -- 172.21.15.83:0/1651296799 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55de128c15a0 msgr2=0x55de128bf020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:46.399 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.395+0000 7fec67e00640 1 --2- 172.21.15.83:0/1651296799 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55de128c15a0 0x55de128bf020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:46.399 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.395+0000 7fec67e00640 1 -- 172.21.15.83:0/1651296799 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de128b63c0 msgr2=0x55de128be5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:46.399 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.395+0000 7fec67e00640 1 --2- 172.21.15.83:0/1651296799 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de128b63c0 0x55de128be5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:46.399 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.395+0000 7fec67e00640 1 -- 172.21.15.83:0/1651296799 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de127e6fd0 con 0x55de128b6ce0 2024-04-24T08:57:46.399 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.395+0000 7fec67e00640 1 --2- 172.21.15.83:0/1651296799 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55de128b6ce0 0x55de128beae0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7fec5409c9e0 tx=0x7fec54099040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b65e3f50f1ba82b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:46.399 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.395+0000 7fec4e000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:57:46.399 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.395+0000 7fec4e000640 1 -- 172.21.15.83:0/1651296799 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec5409d3e0 con 0x55de128b6ce0 2024-04-24T08:57:46.399 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.395+0000 7fec4e000640 1 -- 172.21.15.83:0/1651296799 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7fec540ce710 con 0x55de128b6ce0 2024-04-24T08:57:46.399 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.395+0000 7fec723fc740 1 -- 172.21.15.83:0/1651296799 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec60003d10 con 0x55de128b6ce0 2024-04-24T08:57:46.400 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.396+0000 7fec4e000640 1 -- 172.21.15.83:0/1651296799 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fec540ce910 con 0x55de128b6ce0 2024-04-24T08:57:46.400 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.396+0000 7fec723fc740 10 client.17823 fetch_fsmap learned FSMap version 33 2024-04-24T08:57:46.400 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.396+0000 7fec723fc740 1 -- 172.21.15.83:0/1651296799 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55de1252f210 con 0x55de128b6ce0 2024-04-24T08:57:46.400 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.396+0000 7fec4e000640 1 -- 172.21.15.83:0/1651296799 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fec60003d10 con 0x55de128b6ce0 2024-04-24T08:57:46.400 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.396+0000 7fec4e000640 1 -- 172.21.15.83:0/1651296799 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fec540cebe0 con 0x55de128b6ce0 2024-04-24T08:57:46.400 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.396+0000 7fec723fc740 10 client.17823 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T08:57:46.400 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.396+0000 7fec723fc740 10 client.17823 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T08:57:46.400 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.396+0000 7fec723fc740 20 client.17823 populate_metadata read hostname 'smithi083' 2024-04-24T08:57:46.400 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.396+0000 7fec723fc740 1 --2- 172.21.15.83:0/1651296799 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55de128d16e0 0x55de128d3b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:46.400 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.396+0000 7fec723fc740 4 client.17823 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:57:46.400 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.396+0000 7fec723fc740 1 -- 172.21.15.83:0/1651296799 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55de1282a320 con 0x55de128d16e0 2024-04-24T08:57:46.400 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.397+0000 7fec6cc00640 1 --2- 172.21.15.83:0/1651296799 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55de128d16e0 0x55de128d3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:46.402 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.398+0000 7fec6cc00640 1 --2- 172.21.15.83:0/1651296799 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55de128d16e0 0x55de128d3b30 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92d90fc785d698e server_cookie=8fff386939a8343 in_seq=0 out_seq=0 2024-04-24T08:57:46.402 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.398+0000 7fec4e000640 10 client.17823 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:57:46.402 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.400+0000 7fec4e000640 1 -- 172.21.15.83:0/1651296799 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55de1282a320 con 0x55de128d16e0 2024-04-24T08:57:46.402 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.400+0000 7fec4e000640 10 client.17823 handle_command_reply: tid=0 2024-04-24T08:57:46.402 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.400+0000 7fec723fc740 1 client.17823 shutdown 2024-04-24T08:57:46.402 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.401+0000 7fec723fc740 1 -- 172.21.15.83:0/1651296799 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55de128b6ce0 msgr2=0x55de128beae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:46.402 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.401+0000 7fec723fc740 1 --2- 172.21.15.83:0/1651296799 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55de128b6ce0 0x55de128beae0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7fec5409c9e0 tx=0x7fec54099040 comp rx=0 tx=0).stop 2024-04-24T08:57:46.402 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.401+0000 7fec723fc740 1 -- 172.21.15.83:0/1651296799 shutdown_connections 2024-04-24T08:57:46.402 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.401+0000 7fec723fc740 1 --2- 172.21.15.83:0/1651296799 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55de128d16e0 0x55de128d3b30 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:46.402 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.401+0000 7fec723fc740 1 --2- 172.21.15.83:0/1651296799 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55de128c15a0 0x55de128bf020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:46.402 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.401+0000 7fec723fc740 1 --2- 172.21.15.83:0/1651296799 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55de128b6ce0 0x55de128beae0 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:46.402 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.401+0000 7fec4e000640 0 client.17823 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:57:46.403 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.401+0000 7fec723fc740 1 --2- 172.21.15.83:0/1651296799 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de128b63c0 0x55de128be5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:46.403 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.401+0000 7fec723fc740 1 -- 172.21.15.83:0/1651296799 >> 172.21.15.83:0/1651296799 conn(0x55de128b28d0 msgr2=0x55de128b8650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:46.403 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.401+0000 7fec723fc740 1 -- 172.21.15.83:0/1651296799 shutdown_connections 2024-04-24T08:57:46.403 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.401+0000 7fec723fc740 1 -- 172.21.15.83:0/1651296799 wait complete. 2024-04-24T08:57:46.403 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.401+0000 7fec723fc740 20 client.17823 trim_cache size 0 max 16384 2024-04-24T08:57:46.419 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.419+0000 7fec723fc740 1 Processor -- start 2024-04-24T08:57:46.419 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.419+0000 7fec723fc740 1 -- start start 2024-04-24T08:57:46.420 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.420+0000 7fec723fc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55de128b63c0 0x55de128c0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:46.421 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.420+0000 7fec723fc740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55de128b6ce0 0x55de128cd0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:46.421 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.420+0000 7fec723fc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de128c15a0 0x55de128b8650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:46.421 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.420+0000 7fec723fc740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fec681b1510 con 0x55de128b63c0 2024-04-24T08:57:46.421 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.420+0000 7fec723fc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fec681b3050 con 0x55de128c15a0 2024-04-24T08:57:46.421 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.420+0000 7fec723fc740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fec681b1be0 con 0x55de128b6ce0 2024-04-24T08:57:46.421 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.420+0000 7fec6cc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55de128b63c0 0x55de128c0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:46.421 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.420+0000 7fec6cc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55de128b63c0 0x55de128c0cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39584/0 (socket says 172.21.15.83:39584) 2024-04-24T08:57:46.421 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.420+0000 7fec6cc00640 1 -- 172.21.15.83:0/2474945626 learned_addr learned my addr 172.21.15.83:0/2474945626 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:57:46.421 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.420+0000 7fec6d600640 1 --2- 172.21.15.83:0/2474945626 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de128c15a0 0x55de128b8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:46.421 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.420+0000 7fec67e00640 1 --2- 172.21.15.83:0/2474945626 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55de128b6ce0 0x55de128cd0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:46.421 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.421+0000 7fec6cc00640 1 -- 172.21.15.83:0/2474945626 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55de128b6ce0 msgr2=0x55de128cd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:46.421 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.421+0000 7fec6cc00640 1 --2- 172.21.15.83:0/2474945626 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55de128b6ce0 0x55de128cd0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:46.421 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.421+0000 7fec6cc00640 1 -- 172.21.15.83:0/2474945626 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de128c15a0 msgr2=0x55de128b8650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:46.422 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.421+0000 7fec6cc00640 1 --2- 172.21.15.83:0/2474945626 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de128c15a0 0x55de128b8650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:46.422 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.421+0000 7fec6cc00640 1 -- 172.21.15.83:0/2474945626 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de128c3d00 con 0x55de128b63c0 2024-04-24T08:57:46.423 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.421+0000 7fec6cc00640 1 --2- 172.21.15.83:0/2474945626 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55de128b63c0 0x55de128c0cd0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7fec5c005ec0 tx=0x7fec5c03d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fea0af8d544bfeb9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:46.423 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.421+0000 7fec59200640 1 -- 172.21.15.83:0/2474945626 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec5c055070 con 0x55de128b63c0 2024-04-24T08:57:46.423 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.421+0000 7fec59200640 1 -- 172.21.15.83:0/2474945626 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fec5c04f700 con 0x55de128b63c0 2024-04-24T08:57:46.423 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.421+0000 7fec59200640 1 -- 172.21.15.83:0/2474945626 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec5c035050 con 0x55de128b63c0 2024-04-24T08:57:46.423 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.422+0000 7fec723fc740 1 -- 172.21.15.83:0/2474945626 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55de128b63c0 msgr2=0x55de128c0cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:46.423 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.422+0000 7fec723fc740 1 --2- 172.21.15.83:0/2474945626 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55de128b63c0 0x55de128c0cd0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7fec5c005ec0 tx=0x7fec5c03d000 comp rx=0 tx=0).stop 2024-04-24T08:57:46.423 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.422+0000 7fec723fc740 1 -- 172.21.15.83:0/2474945626 shutdown_connections 2024-04-24T08:57:46.423 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.422+0000 7fec723fc740 1 --2- 172.21.15.83:0/2474945626 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de128c15a0 0x55de128b8650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:46.423 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.422+0000 7fec723fc740 1 --2- 172.21.15.83:0/2474945626 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55de128b6ce0 0x55de128cd0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:46.423 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.422+0000 7fec723fc740 1 --2- 172.21.15.83:0/2474945626 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55de128b63c0 0x55de128c0cd0 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:46.423 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.422+0000 7fec723fc740 1 -- 172.21.15.83:0/2474945626 >> 172.21.15.83:0/2474945626 conn(0x55de128b28d0 msgr2=0x55de128be560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:46.423 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.422+0000 7fec723fc740 1 -- 172.21.15.83:0/2474945626 shutdown_connections 2024-04-24T08:57:46.424 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.422+0000 7fec723fc740 1 -- 172.21.15.83:0/2474945626 wait complete. 2024-04-24T08:57:46.424 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.423+0000 7fec723fc740 1 -- start start 2024-04-24T08:57:46.424 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.423+0000 7fec723fc740 1 Processor -- start 2024-04-24T08:57:46.426 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.423+0000 7fec723fc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55de128b63c0 0x55de128c9a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:46.426 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.423+0000 7fec723fc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de128b6ce0 0x55de128c65c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:46.426 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.423+0000 7fec723fc740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55de128c15a0 0x55de128c6b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:46.427 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.423+0000 7fec723fc740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fec681b2980 con 0x55de128b63c0 2024-04-24T08:57:46.427 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.423+0000 7fec723fc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fec681b22b0 con 0x55de128b6ce0 2024-04-24T08:57:46.427 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.423+0000 7fec723fc740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fec681af4f0 con 0x55de128c15a0 2024-04-24T08:57:46.427 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.423+0000 7fec723fc740 10 client.0 _ll_register_callbacks cb 0x7fec5400cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:57:46.427 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.424+0000 7fec6cc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55de128b63c0 0x55de128c9a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:46.427 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.424+0000 7fec6cc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55de128b63c0 0x55de128c9a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39586/0 (socket says 172.21.15.83:39586) 2024-04-24T08:57:46.427 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.424+0000 7fec6cc00640 1 -- 172.21.15.83:0/1894300137 learned_addr learned my addr 172.21.15.83:0/1894300137 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:57:46.427 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.424+0000 7fec6cc00640 1 -- 172.21.15.83:0/1894300137 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55de128c15a0 msgr2=0x55de128c6b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:46.427 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.424+0000 7fec6cc00640 1 --2- 172.21.15.83:0/1894300137 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55de128c15a0 0x55de128c6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:46.427 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.424+0000 7fec6cc00640 1 -- 172.21.15.83:0/1894300137 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de128b6ce0 msgr2=0x55de128c65c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:46.427 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.424+0000 7fec6cc00640 1 --2- 172.21.15.83:0/1894300137 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de128b6ce0 0x55de128c65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:46.427 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.424+0000 7fec6cc00640 1 -- 172.21.15.83:0/1894300137 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de125fb620 con 0x55de128b63c0 2024-04-24T08:57:46.427 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.424+0000 7fec6cc00640 1 --2- 172.21.15.83:0/1894300137 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55de128b63c0 0x55de128c9a50 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7fec5c006010 tx=0x7fec5c004160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=963ec4de0ce4e7d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:46.427 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.424+0000 7fec4e000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:57:46.428 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.425+0000 7fec4e000640 1 -- 172.21.15.83:0/1894300137 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec5c055040 con 0x55de128b63c0 2024-04-24T08:57:46.428 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.425+0000 7fec723fc740 1 -- 172.21.15.83:0/1894300137 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec5c04f700 con 0x55de128b63c0 2024-04-24T08:57:46.428 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.425+0000 7fec4e000640 1 -- 172.21.15.83:0/1894300137 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7fec5c04fab0 con 0x55de128b63c0 2024-04-24T08:57:46.428 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.425+0000 7fec4e000640 1 -- 172.21.15.83:0/1894300137 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fec5c035040 con 0x55de128b63c0 2024-04-24T08:57:46.428 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.425+0000 7fec723fc740 10 client.17829 fetch_fsmap learned FSMap version 33 2024-04-24T08:57:46.428 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.425+0000 7fec723fc740 1 -- 172.21.15.83:0/1894300137 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55de12938ba0 con 0x55de128b63c0 2024-04-24T08:57:46.428 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.425+0000 7fec4e000640 1 -- 172.21.15.83:0/1894300137 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fec5c04f700 con 0x55de128b63c0 2024-04-24T08:57:46.428 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.425+0000 7fec4e000640 1 -- 172.21.15.83:0/1894300137 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fec5c0046c0 con 0x55de128b63c0 2024-04-24T08:57:46.428 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.425+0000 7fec723fc740 10 client.17829 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T08:57:46.428 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.425+0000 7fec723fc740 10 client.17829 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T08:57:46.428 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.426+0000 7fec723fc740 20 client.17829 populate_metadata read hostname 'smithi083' 2024-04-24T08:57:46.428 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.426+0000 7fec723fc740 1 --2- 172.21.15.83:0/1894300137 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55de12939200 0x55de1293b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:46.428 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.426+0000 7fec723fc740 4 client.17829 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T08:57:46.428 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.426+0000 7fec723fc740 1 -- 172.21.15.83:0/1894300137 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55de12824190 con 0x55de12939200 2024-04-24T08:57:46.428 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.426+0000 7fec67e00640 1 --2- 172.21.15.83:0/1894300137 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55de12939200 0x55de1293b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:46.429 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.427+0000 7fec4e000640 10 client.17829 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:57:46.429 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.427+0000 7fec67e00640 1 --2- 172.21.15.83:0/1894300137 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55de12939200 0x55de1293b650 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c928c26a0147cf78 server_cookie=6b1a3cfd62cd0ff in_seq=0 out_seq=0 2024-04-24T08:57:46.493 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.493+0000 7fec4e000640 1 -- 172.21.15.83:0/1894300137 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55de12824190 con 0x55de12939200 2024-04-24T08:57:46.494 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.493+0000 7fec4e000640 10 client.17829 handle_command_reply: tid=0 2024-04-24T08:57:46.494 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.493+0000 7fec723fc740 1 client.17829 shutdown 2024-04-24T08:57:46.494 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.494+0000 7fec723fc740 1 -- 172.21.15.83:0/1894300137 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55de128b63c0 msgr2=0x55de128c9a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:46.494 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.494+0000 7fec723fc740 1 --2- 172.21.15.83:0/1894300137 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55de128b63c0 0x55de128c9a50 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7fec5c006010 tx=0x7fec5c004160 comp rx=0 tx=0).stop 2024-04-24T08:57:46.494 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.494+0000 7fec723fc740 1 -- 172.21.15.83:0/1894300137 shutdown_connections 2024-04-24T08:57:46.495 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.494+0000 7fec723fc740 1 --2- 172.21.15.83:0/1894300137 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55de12939200 0x55de1293b650 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:46.495 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.494+0000 7fec723fc740 1 --2- 172.21.15.83:0/1894300137 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55de128c15a0 0x55de128c6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:46.495 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.494+0000 7fec723fc740 1 --2- 172.21.15.83:0/1894300137 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de128b6ce0 0x55de128c65c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:46.495 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.494+0000 7fec723fc740 1 --2- 172.21.15.83:0/1894300137 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55de128b63c0 0x55de128c9a50 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:46.495 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.494+0000 7fec723fc740 1 -- 172.21.15.83:0/1894300137 >> 172.21.15.83:0/1894300137 conn(0x55de128b28d0 msgr2=0x55de128b84d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:46.495 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.494+0000 7fec4e000640 0 client.17829 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:57:46.496 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.495+0000 7fec723fc740 1 -- 172.21.15.83:0/1894300137 shutdown_connections 2024-04-24T08:57:46.496 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.495+0000 7fec723fc740 1 -- 172.21.15.83:0/1894300137 wait complete. 2024-04-24T08:57:46.496 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.495+0000 7fec723fc740 20 client.17829 trim_cache size 0 max 16384 2024-04-24T08:57:46.496 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T08:57:46.496 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T08:57:46.496 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T08:57:46.496 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T08:57:46.497 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.497+0000 7fec6e000640 1 -- 172.21.15.83:0/3068138877 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fec500a3d10 msgr2=0x7fec500a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:46.497 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.497+0000 7fec6e000640 1 --2- 172.21.15.83:0/3068138877 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fec500a3d10 0x7fec500a61d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fec5c000c00 tx=0x7fec5c005a40 comp rx=0 tx=0).stop 2024-04-24T08:57:46.497 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.497+0000 7fec6e000640 1 -- 172.21.15.83:0/3068138877 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec68107140 msgr2=0x7fec681a0b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:46.497 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.497+0000 7fec6e000640 1 --2- 172.21.15.83:0/3068138877 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec68107140 0x7fec681a0b20 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7fec5400e9c0 tx=0x7fec5400ee80 comp rx=0 tx=0).stop 2024-04-24T08:57:46.497 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.497+0000 7fec6e000640 1 -- 172.21.15.83:0/3068138877 shutdown_connections 2024-04-24T08:57:46.497 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.497+0000 7fec6e000640 1 --2- 172.21.15.83:0/3068138877 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fec500a3d10 0x7fec500a61d0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:46.497 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.497+0000 7fec6e000640 1 --2- 172.21.15.83:0/3068138877 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fec6819a6a0 0x7fec6819ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:46.498 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.497+0000 7fec6e000640 1 --2- 172.21.15.83:0/3068138877 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec68107140 0x7fec681a0b20 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:46.498 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.497+0000 7fec6e000640 1 --2- 172.21.15.83:0/3068138877 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec68102070 0x7fec681a05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:46.498 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.497+0000 7fec6e000640 1 -- 172.21.15.83:0/3068138877 >> 172.21.15.83:0/3068138877 conn(0x7fec680fd950 msgr2=0x7fec680ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:46.498 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.497+0000 7fec6e000640 1 -- 172.21.15.83:0/3068138877 shutdown_connections 2024-04-24T08:57:46.498 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:46.497+0000 7fec6e000640 1 -- 172.21.15.83:0/3068138877 wait complete. 2024-04-24T08:57:46.582 INFO:tasks.workunit.client.0.smithi083.stdout: 1 31700 0.65 MB/sec execute 140 sec latency 55.176 ms 2024-04-24T08:57:47.582 INFO:tasks.workunit.client.0.smithi083.stdout: 1 31729 0.64 MB/sec execute 141 sec latency 53.362 ms 2024-04-24T08:57:47.954 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:57:47 smithi138 ceph-mon[31441]: pgmap v188: 129 pgs: 129 active+clean; 42 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 271 KiB/s wr, 42 op/s 2024-04-24T08:57:47.956 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:57:47 smithi189 ceph-mon[31791]: pgmap v188: 129 pgs: 129 active+clean; 42 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 271 KiB/s wr, 42 op/s 2024-04-24T08:57:48.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:57:47 smithi083 ceph-mon[27907]: pgmap v188: 129 pgs: 129 active+clean; 42 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 271 KiB/s wr, 42 op/s 2024-04-24T08:57:48.583 INFO:tasks.workunit.client.0.smithi083.stdout: 1 31756 0.64 MB/sec execute 142 sec latency 53.348 ms 2024-04-24T08:57:49.583 INFO:tasks.workunit.client.0.smithi083.stdout: 1 31938 0.66 MB/sec execute 143 sec latency 55.824 ms 2024-04-24T08:57:49.954 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:57:49 smithi138 ceph-mon[31441]: pgmap v189: 129 pgs: 129 active+clean; 40 MiB data, 603 MiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 476 KiB/s wr, 65 op/s 2024-04-24T08:57:49.956 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:57:49 smithi189 ceph-mon[31791]: pgmap v189: 129 pgs: 129 active+clean; 40 MiB data, 603 MiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 476 KiB/s wr, 65 op/s 2024-04-24T08:57:50.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:57:49 smithi083 ceph-mon[27907]: pgmap v189: 129 pgs: 129 active+clean; 40 MiB data, 603 MiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 476 KiB/s wr, 65 op/s 2024-04-24T08:57:50.583 INFO:tasks.workunit.client.0.smithi083.stdout: 1 32054 0.65 MB/sec execute 144 sec latency 50.191 ms 2024-04-24T08:57:51.583 INFO:tasks.workunit.client.0.smithi083.stdout: 1 32176 0.66 MB/sec execute 145 sec latency 58.977 ms 2024-04-24T08:57:51.954 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:57:51 smithi138 ceph-mon[31441]: pgmap v190: 129 pgs: 129 active+clean; 40 MiB data, 603 MiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 406 KiB/s wr, 42 op/s 2024-04-24T08:57:51.956 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:57:51 smithi189 ceph-mon[31791]: pgmap v190: 129 pgs: 129 active+clean; 40 MiB data, 603 MiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 406 KiB/s wr, 42 op/s 2024-04-24T08:57:52.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:57:51 smithi083 ceph-mon[27907]: pgmap v190: 129 pgs: 129 active+clean; 40 MiB data, 603 MiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 406 KiB/s wr, 42 op/s 2024-04-24T08:57:52.583 INFO:tasks.workunit.client.0.smithi083.stdout: 1 32228 0.65 MB/sec execute 146 sec latency 46.793 ms 2024-04-24T08:57:53.583 INFO:tasks.workunit.client.0.smithi083.stdout: 1 32288 0.65 MB/sec execute 147 sec latency 54.235 ms 2024-04-24T08:57:53.954 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:57:53 smithi138 ceph-mon[31441]: pgmap v191: 129 pgs: 129 active+clean; 40 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 634 KiB/s wr, 63 op/s 2024-04-24T08:57:53.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:57:53 smithi189 ceph-mon[31791]: pgmap v191: 129 pgs: 129 active+clean; 40 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 634 KiB/s wr, 63 op/s 2024-04-24T08:57:54.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:57:53 smithi083 ceph-mon[27907]: pgmap v191: 129 pgs: 129 active+clean; 40 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 634 KiB/s wr, 63 op/s 2024-04-24T08:57:54.070 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T08:57:54.070 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T08:57:54.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.282+0000 7f1f07200640 1 -- 172.21.15.83:0/1402196250 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f00112eb0 msgr2=0x7f1f001152a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:54.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.282+0000 7f1f07200640 1 --2- 172.21.15.83:0/1402196250 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f00112eb0 0x7f1f001152a0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f1efc009ca0 tx=0x7f1efc02a950 comp rx=0 tx=0).stop 2024-04-24T08:57:54.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.282+0000 7f1f07200640 1 -- 172.21.15.83:0/1402196250 shutdown_connections 2024-04-24T08:57:54.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.282+0000 7f1f07200640 1 --2- 172.21.15.83:0/1402196250 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f00112eb0 0x7f1f001152a0 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:54.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.282+0000 7f1f07200640 1 --2- 172.21.15.83:0/1402196250 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1f00074cc0 0x7f1f00112970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:54.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.282+0000 7f1f07200640 1 --2- 172.21.15.83:0/1402196250 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f000743a0 0x7f1f00074780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:54.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.282+0000 7f1f07200640 1 -- 172.21.15.83:0/1402196250 >> 172.21.15.83:0/1402196250 conn(0x7f1f000fd9d0 msgr2=0x7f1f000ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:54.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.282+0000 7f1f07200640 1 -- 172.21.15.83:0/1402196250 shutdown_connections 2024-04-24T08:57:54.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.282+0000 7f1f07200640 1 -- 172.21.15.83:0/1402196250 wait complete. 2024-04-24T08:57:54.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.283+0000 7f1f07200640 1 Processor -- start 2024-04-24T08:57:54.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.283+0000 7f1f07200640 1 -- start start 2024-04-24T08:57:54.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.284+0000 7f1f07200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1f000743a0 0x7f1f001a4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:54.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.284+0000 7f1f07200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f00074cc0 0x7f1f001a4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:54.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.284+0000 7f1f07200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f00112eb0 0x7f1f0019eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:54.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.284+0000 7f1f07200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1f00118770 con 0x7f1f00112eb0 2024-04-24T08:57:54.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.284+0000 7f1f07200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1f001185f0 con 0x7f1f00074cc0 2024-04-24T08:57:54.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.284+0000 7f1f07200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1f001188f0 con 0x7f1f000743a0 2024-04-24T08:57:54.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.284+0000 7f1f06800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f00112eb0 0x7f1f0019eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:54.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.284+0000 7f1f06800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f00112eb0 0x7f1f0019eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41528/0 (socket says 172.21.15.83:41528) 2024-04-24T08:57:54.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.284+0000 7f1f06800640 1 -- 172.21.15.83:0/2913424315 learned_addr learned my addr 172.21.15.83:0/2913424315 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:57:54.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.284+0000 7f1f05400640 1 --2- 172.21.15.83:0/2913424315 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f00074cc0 0x7f1f001a4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:54.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.284+0000 7f1f05e00640 1 --2- 172.21.15.83:0/2913424315 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1f000743a0 0x7f1f001a4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:54.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.285+0000 7f1f06800640 1 -- 172.21.15.83:0/2913424315 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1f000743a0 msgr2=0x7f1f001a4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:54.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.285+0000 7f1f06800640 1 --2- 172.21.15.83:0/2913424315 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1f000743a0 0x7f1f001a4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:54.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.285+0000 7f1f06800640 1 -- 172.21.15.83:0/2913424315 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f00074cc0 msgr2=0x7f1f001a4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:54.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.285+0000 7f1f06800640 1 --2- 172.21.15.83:0/2913424315 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f00074cc0 0x7f1f001a4e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:54.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.285+0000 7f1f06800640 1 -- 172.21.15.83:0/2913424315 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f0019f230 con 0x7f1f00112eb0 2024-04-24T08:57:54.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.285+0000 7f1f05400640 1 --2- 172.21.15.83:0/2913424315 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f00074cc0 0x7f1f001a4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:57:54.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.285+0000 7f1f05e00640 1 --2- 172.21.15.83:0/2913424315 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1f000743a0 0x7f1f001a4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:57:54.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.285+0000 7f1f06800640 1 --2- 172.21.15.83:0/2913424315 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f00112eb0 0x7f1f0019eab0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f1efc005e30 tx=0x7f1efc003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:54.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.285+0000 7f1efaa00640 1 -- 172.21.15.83:0/2913424315 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1efc034040 con 0x7f1f00112eb0 2024-04-24T08:57:54.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.285+0000 7f1efaa00640 1 -- 172.21.15.83:0/2913424315 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1efc030850 con 0x7f1f00112eb0 2024-04-24T08:57:54.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.285+0000 7f1f07200640 1 -- 172.21.15.83:0/2913424315 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f0019f4c0 con 0x7f1f00112eb0 2024-04-24T08:57:54.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.285+0000 7f1efaa00640 1 -- 172.21.15.83:0/2913424315 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1efc0046f0 con 0x7f1f00112eb0 2024-04-24T08:57:54.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.285+0000 7f1f07200640 1 -- 172.21.15.83:0/2913424315 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f001ab7c0 con 0x7f1f00112eb0 2024-04-24T08:57:54.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.289+0000 7f1efaa00640 1 -- 172.21.15.83:0/2913424315 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1efc037020 con 0x7f1f00112eb0 2024-04-24T08:57:54.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.289+0000 7f1f0b608740 1 Processor -- start 2024-04-24T08:57:54.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.289+0000 7f1f0b608740 1 -- start start 2024-04-24T08:57:54.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.290+0000 7f1f0b608740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56009e304a30 0x56009e304e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:54.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.290+0000 7f1f0b608740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56009e305350 0x56009e30f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:54.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.290+0000 7f1f0b608740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56009e30fa20 0x56009e311e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:54.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.290+0000 7f1f0b608740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56009e26a1f0 con 0x56009e30fa20 2024-04-24T08:57:54.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.290+0000 7f1f0b608740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56009e210350 con 0x56009e304a30 2024-04-24T08:57:54.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.290+0000 7f1f0b608740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56009e2661b0 con 0x56009e305350 2024-04-24T08:57:54.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.290+0000 7f1f05400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56009e30fa20 0x56009e311e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:54.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.290+0000 7f1f05400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56009e30fa20 0x56009e311e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41542/0 (socket says 172.21.15.83:41542) 2024-04-24T08:57:54.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.290+0000 7f1f05400640 1 -- 172.21.15.83:0/2423831836 learned_addr learned my addr 172.21.15.83:0/2423831836 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:57:54.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.290+0000 7f1f05e00640 1 --2- 172.21.15.83:0/2423831836 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56009e305350 0x56009e30f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:54.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.290+0000 7f1f05400640 1 --2- 172.21.15.83:0/2423831836 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56009e304a30 0x56009e304e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:54.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.290+0000 7f1f05400640 1 -- 172.21.15.83:0/2423831836 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56009e305350 msgr2=0x56009e30f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:54.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.290+0000 7f1f05400640 1 --2- 172.21.15.83:0/2423831836 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56009e305350 0x56009e30f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:54.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.290+0000 7f1f05400640 1 -- 172.21.15.83:0/2423831836 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56009e304a30 msgr2=0x56009e304e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:54.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.290+0000 7f1f05400640 1 --2- 172.21.15.83:0/2423831836 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56009e304a30 0x56009e304e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:54.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.291+0000 7f1f05400640 1 -- 172.21.15.83:0/2423831836 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56009e264f50 con 0x56009e30fa20 2024-04-24T08:57:54.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.291+0000 7f1efaa00640 1 --2- 172.21.15.83:0/2913424315 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f1ee80a3d20 0x7f1ee80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:54.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.291+0000 7f1f05400640 1 --2- 172.21.15.83:0/2423831836 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56009e30fa20 0x56009e311e10 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f1ef00124a0 tx=0x7f1ef000e5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5533ddbe2ba6e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:54.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.291+0000 7f1efaa00640 1 -- 172.21.15.83:0/2913424315 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f1efc0f1bb0 con 0x7f1f00112eb0 2024-04-24T08:57:54.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.291+0000 7f1eef000640 1 -- 172.21.15.83:0/2423831836 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef0018070 con 0x56009e30fa20 2024-04-24T08:57:54.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.291+0000 7f1f05e00640 1 --2- 172.21.15.83:0/2913424315 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f1ee80a3d20 0x7f1ee80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:54.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.291+0000 7f1eef000640 1 -- 172.21.15.83:0/2423831836 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1ef001ca60 con 0x56009e30fa20 2024-04-24T08:57:54.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.291+0000 7f1eef000640 1 -- 172.21.15.83:0/2423831836 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef0010ab0 con 0x56009e30fa20 2024-04-24T08:57:54.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.292+0000 7f1f0b608740 1 -- 172.21.15.83:0/2423831836 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56009e30fa20 msgr2=0x56009e311e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:54.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.292+0000 7f1f0b608740 1 --2- 172.21.15.83:0/2423831836 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56009e30fa20 0x56009e311e10 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f1ef00124a0 tx=0x7f1ef000e5b0 comp rx=0 tx=0).stop 2024-04-24T08:57:54.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.292+0000 7f1f05e00640 1 --2- 172.21.15.83:0/2913424315 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f1ee80a3d20 0x7f1ee80a61e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1ef4005fd0 tx=0x7f1ef4009350 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:54.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.292+0000 7f1f0b608740 1 -- 172.21.15.83:0/2423831836 shutdown_connections 2024-04-24T08:57:54.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.292+0000 7f1f0b608740 1 --2- 172.21.15.83:0/2423831836 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56009e30fa20 0x56009e311e10 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:54.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.292+0000 7f1f0b608740 1 --2- 172.21.15.83:0/2423831836 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56009e305350 0x56009e30f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:54.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.292+0000 7f1f0b608740 1 --2- 172.21.15.83:0/2423831836 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56009e304a30 0x56009e304e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:54.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.292+0000 7f1f0b608740 1 -- 172.21.15.83:0/2423831836 >> 172.21.15.83:0/2423831836 conn(0x56009e300d50 msgr2=0x56009e301130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:54.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.293+0000 7f1f0b608740 1 -- 172.21.15.83:0/2423831836 shutdown_connections 2024-04-24T08:57:54.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.293+0000 7f1f0b608740 1 -- 172.21.15.83:0/2423831836 wait complete. 2024-04-24T08:57:54.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.293+0000 7f1f0b608740 1 -- start start 2024-04-24T08:57:54.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.293+0000 7f1f0b608740 1 Processor -- start 2024-04-24T08:57:54.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.294+0000 7f1f0b608740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56009e304a30 0x56009e30ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:54.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.294+0000 7f1f0b608740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56009e305350 0x56009e30cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:54.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.294+0000 7f1f0b608740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56009e30fa20 0x56009e30d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:54.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.294+0000 7f1f0b608740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56009e312810 con 0x56009e304a30 2024-04-24T08:57:54.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.294+0000 7f1f0b608740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56009e223690 con 0x56009e305350 2024-04-24T08:57:54.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.294+0000 7f1f0b608740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56009e312990 con 0x56009e30fa20 2024-04-24T08:57:54.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.294+0000 7f1f0b608740 10 client.0 _ll_register_callbacks cb 0x56009e25f650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:57:54.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.294+0000 7f1f05e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56009e304a30 0x56009e30ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:54.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.294+0000 7f1f05e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56009e304a30 0x56009e30ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41558/0 (socket says 172.21.15.83:41558) 2024-04-24T08:57:54.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.294+0000 7f1f05e00640 1 -- 172.21.15.83:0/4028997786 learned_addr learned my addr 172.21.15.83:0/4028997786 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:57:54.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.294+0000 7f1f05400640 1 --2- 172.21.15.83:0/4028997786 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56009e305350 0x56009e30cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:54.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.295+0000 7f1f06800640 1 --2- 172.21.15.83:0/4028997786 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56009e30fa20 0x56009e30d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:54.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.295+0000 7f1f05e00640 1 -- 172.21.15.83:0/4028997786 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56009e30fa20 msgr2=0x56009e30d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:54.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.295+0000 7f1f05e00640 1 --2- 172.21.15.83:0/4028997786 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56009e30fa20 0x56009e30d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:54.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.295+0000 7f1f05e00640 1 -- 172.21.15.83:0/4028997786 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56009e305350 msgr2=0x56009e30cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:54.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.295+0000 7f1f05e00640 1 --2- 172.21.15.83:0/4028997786 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56009e305350 0x56009e30cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:54.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.295+0000 7f1f05e00640 1 -- 172.21.15.83:0/4028997786 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56009e222920 con 0x56009e304a30 2024-04-24T08:57:54.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.295+0000 7f1f06800640 1 --2- 172.21.15.83:0/4028997786 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56009e30fa20 0x56009e30d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:57:54.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.295+0000 7f1f05e00640 1 --2- 172.21.15.83:0/4028997786 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56009e304a30 0x56009e30ca50 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f1ef40126d0 tx=0x7f1ef4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=675cf23a70d3a6ae server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:54.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.295+0000 7f1ee3400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:57:54.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.295+0000 7f1ee3400640 1 -- 172.21.15.83:0/4028997786 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef4023070 con 0x56009e304a30 2024-04-24T08:57:54.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.295+0000 7f1ee3400640 1 -- 172.21.15.83:0/4028997786 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f1ef4005220 con 0x56009e304a30 2024-04-24T08:57:54.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.295+0000 7f1f0b608740 1 -- 172.21.15.83:0/4028997786 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ef001ca60 con 0x56009e304a30 2024-04-24T08:57:54.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.296+0000 7f1ee3400640 1 -- 172.21.15.83:0/4028997786 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f1ef402e050 con 0x56009e304a30 2024-04-24T08:57:54.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.296+0000 7f1f0b608740 10 client.17841 fetch_fsmap learned FSMap version 33 2024-04-24T08:57:54.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.296+0000 7f1f0b608740 1 -- 172.21.15.83:0/4028997786 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56009e0467d0 con 0x56009e304a30 2024-04-24T08:57:54.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.296+0000 7f1ee3400640 1 -- 172.21.15.83:0/4028997786 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1ef001ca60 con 0x56009e304a30 2024-04-24T08:57:54.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.296+0000 7f1ee3400640 1 -- 172.21.15.83:0/4028997786 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f1ef400f040 con 0x56009e304a30 2024-04-24T08:57:54.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.296+0000 7f1f0b608740 10 client.17841 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T08:57:54.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.296+0000 7f1f0b608740 10 client.17841 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:57:54.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.296+0000 7f1f0b608740 20 client.17841 populate_metadata read hostname 'smithi083' 2024-04-24T08:57:54.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.296+0000 7f1f0b608740 1 --2- 172.21.15.83:0/4028997786 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56009e31fe90 0x56009e322310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:54.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.296+0000 7f1f0b608740 4 client.17841 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:57:54.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.296+0000 7f1f0b608740 1 -- 172.21.15.83:0/4028997786 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56009e277c10 con 0x56009e31fe90 2024-04-24T08:57:54.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.297+0000 7f1f05400640 1 --2- 172.21.15.83:0/4028997786 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56009e31fe90 0x56009e322310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:54.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.298+0000 7f1f05400640 1 --2- 172.21.15.83:0/4028997786 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56009e31fe90 0x56009e322310 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=398973f1f813a8d server_cookie=80afd83bae7d433e in_seq=0 out_seq=0 2024-04-24T08:57:54.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.298+0000 7f1ee3400640 10 client.17841 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:57:54.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.301+0000 7f1ee3400640 1 -- 172.21.15.83:0/4028997786 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56009e277c10 con 0x56009e31fe90 2024-04-24T08:57:54.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.301+0000 7f1ee3400640 10 client.17841 handle_command_reply: tid=0 2024-04-24T08:57:54.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.302+0000 7f1f0b608740 1 client.17841 shutdown 2024-04-24T08:57:54.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.302+0000 7f1f0b608740 1 -- 172.21.15.83:0/4028997786 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56009e304a30 msgr2=0x56009e30ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:54.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.302+0000 7f1f0b608740 1 --2- 172.21.15.83:0/4028997786 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56009e304a30 0x56009e30ca50 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f1ef40126d0 tx=0x7f1ef4004730 comp rx=0 tx=0).stop 2024-04-24T08:57:54.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.302+0000 7f1f0b608740 1 -- 172.21.15.83:0/4028997786 shutdown_connections 2024-04-24T08:57:54.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.302+0000 7f1f0b608740 1 --2- 172.21.15.83:0/4028997786 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56009e31fe90 0x56009e322310 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:54.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.302+0000 7f1f0b608740 1 --2- 172.21.15.83:0/4028997786 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56009e30fa20 0x56009e30d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:54.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.302+0000 7f1f0b608740 1 --2- 172.21.15.83:0/4028997786 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56009e305350 0x56009e30cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:54.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.302+0000 7f1f0b608740 1 --2- 172.21.15.83:0/4028997786 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56009e304a30 0x56009e30ca50 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:54.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.302+0000 7f1ee3400640 0 client.17841 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:57:54.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.302+0000 7f1f0b608740 1 -- 172.21.15.83:0/4028997786 >> 172.21.15.83:0/4028997786 conn(0x56009e300d50 msgr2=0x56009e306b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:54.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.302+0000 7f1f0b608740 1 -- 172.21.15.83:0/4028997786 shutdown_connections 2024-04-24T08:57:54.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.302+0000 7f1f0b608740 1 -- 172.21.15.83:0/4028997786 wait complete. 2024-04-24T08:57:54.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.302+0000 7f1f0b608740 20 client.17841 trim_cache size 0 max 16384 2024-04-24T08:57:54.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.323+0000 7f1f0b608740 1 Processor -- start 2024-04-24T08:57:54.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.323+0000 7f1f0b608740 1 -- start start 2024-04-24T08:57:54.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.324+0000 7f1f0b608740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56009e304a30 0x56009e31b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:54.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.324+0000 7f1f0b608740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56009e305350 0x56009e307190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:54.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.324+0000 7f1f0b608740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56009e30fa20 0x56009e30d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:54.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.324+0000 7f1f0b608740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1f001b58c0 con 0x56009e304a30 2024-04-24T08:57:54.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.324+0000 7f1f0b608740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1f001b7400 con 0x56009e30fa20 2024-04-24T08:57:54.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.324+0000 7f1f0b608740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1f001b5f90 con 0x56009e305350 2024-04-24T08:57:54.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.324+0000 7f1f05e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56009e304a30 0x56009e31b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:54.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.324+0000 7f1f05e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56009e304a30 0x56009e31b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41566/0 (socket says 172.21.15.83:41566) 2024-04-24T08:57:54.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.324+0000 7f1f05e00640 1 -- 172.21.15.83:0/362415688 learned_addr learned my addr 172.21.15.83:0/362415688 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:57:54.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.324+0000 7f1f05e00640 1 -- 172.21.15.83:0/362415688 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56009e305350 msgr2=0x56009e307190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:54.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.324+0000 7f1f05e00640 1 --2- 172.21.15.83:0/362415688 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56009e305350 0x56009e307190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:54.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.324+0000 7f1f05e00640 1 -- 172.21.15.83:0/362415688 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56009e30fa20 msgr2=0x56009e30d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:54.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.324+0000 7f1f05e00640 1 --2- 172.21.15.83:0/362415688 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56009e30fa20 0x56009e30d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:54.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.324+0000 7f1f05e00640 1 -- 172.21.15.83:0/362415688 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56009df7c3f0 con 0x56009e304a30 2024-04-24T08:57:54.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.325+0000 7f1f05e00640 1 --2- 172.21.15.83:0/362415688 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56009e304a30 0x56009e31b730 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f1ef4009ae0 tx=0x7f1ef4009b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c778e49c05f14631 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:54.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.325+0000 7f1eee600640 1 -- 172.21.15.83:0/362415688 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef4036030 con 0x56009e304a30 2024-04-24T08:57:54.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.325+0000 7f1eee600640 1 -- 172.21.15.83:0/362415688 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1ef4010c20 con 0x56009e304a30 2024-04-24T08:57:54.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.325+0000 7f1eee600640 1 -- 172.21.15.83:0/362415688 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef4023050 con 0x56009e304a30 2024-04-24T08:57:54.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.326+0000 7f1f0b608740 1 -- 172.21.15.83:0/362415688 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56009e304a30 msgr2=0x56009e31b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:54.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.326+0000 7f1f0b608740 1 --2- 172.21.15.83:0/362415688 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56009e304a30 0x56009e31b730 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f1ef4009ae0 tx=0x7f1ef4009b10 comp rx=0 tx=0).stop 2024-04-24T08:57:54.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.326+0000 7f1f0b608740 1 -- 172.21.15.83:0/362415688 shutdown_connections 2024-04-24T08:57:54.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.326+0000 7f1f0b608740 1 --2- 172.21.15.83:0/362415688 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56009e30fa20 0x56009e30d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:54.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.326+0000 7f1f0b608740 1 --2- 172.21.15.83:0/362415688 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56009e305350 0x56009e307190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:54.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.326+0000 7f1f0b608740 1 --2- 172.21.15.83:0/362415688 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56009e304a30 0x56009e31b730 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:54.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.326+0000 7f1f0b608740 1 -- 172.21.15.83:0/362415688 >> 172.21.15.83:0/362415688 conn(0x56009e300d50 msgr2=0x56009e30ca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:54.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.326+0000 7f1f0b608740 1 -- 172.21.15.83:0/362415688 shutdown_connections 2024-04-24T08:57:54.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.326+0000 7f1f0b608740 1 -- 172.21.15.83:0/362415688 wait complete. 2024-04-24T08:57:54.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.327+0000 7f1f0b608740 1 -- start start 2024-04-24T08:57:54.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.327+0000 7f1f0b608740 1 Processor -- start 2024-04-24T08:57:54.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.327+0000 7f1f0b608740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56009e304a30 0x56009e317c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:54.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.327+0000 7f1f0b608740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56009e305350 0x56009e318180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:54.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.327+0000 7f1f0b608740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56009e30fa20 0x56009e3090f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:54.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.327+0000 7f1f0b608740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1f001b6d30 con 0x56009e305350 2024-04-24T08:57:54.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.327+0000 7f1f0b608740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1f001b6660 con 0x56009e304a30 2024-04-24T08:57:54.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.327+0000 7f1f0b608740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1f001b38a0 con 0x56009e30fa20 2024-04-24T08:57:54.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.327+0000 7f1f0b608740 10 client.0 _ll_register_callbacks cb 0x7f1ef4016070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:57:54.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.328+0000 7f1f05400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56009e305350 0x56009e318180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:54.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.328+0000 7f1f05400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56009e305350 0x56009e318180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41572/0 (socket says 172.21.15.83:41572) 2024-04-24T08:57:54.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.328+0000 7f1f05400640 1 -- 172.21.15.83:0/404938101 learned_addr learned my addr 172.21.15.83:0/404938101 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:57:54.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.328+0000 7f1f06800640 1 --2- 172.21.15.83:0/404938101 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56009e30fa20 0x56009e3090f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:54.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.328+0000 7f1f05e00640 1 --2- 172.21.15.83:0/404938101 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56009e304a30 0x56009e317c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:54.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.328+0000 7f1f05400640 1 -- 172.21.15.83:0/404938101 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56009e30fa20 msgr2=0x56009e3090f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:54.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.328+0000 7f1f05400640 1 --2- 172.21.15.83:0/404938101 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56009e30fa20 0x56009e3090f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:54.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.328+0000 7f1f05400640 1 -- 172.21.15.83:0/404938101 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56009e304a30 msgr2=0x56009e317c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:54.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.328+0000 7f1f05400640 1 --2- 172.21.15.83:0/404938101 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56009e304a30 0x56009e317c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:54.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.328+0000 7f1f05400640 1 -- 172.21.15.83:0/404938101 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56009df3c0f0 con 0x56009e305350 2024-04-24T08:57:54.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.328+0000 7f1f05e00640 1 --2- 172.21.15.83:0/404938101 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56009e304a30 0x56009e317c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:57:54.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.328+0000 7f1f05400640 1 --2- 172.21.15.83:0/404938101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56009e305350 0x56009e318180 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f1ef0015000 tx=0x7f1ef000e3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91f5c16173012bf2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:54.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.328+0000 7f1ee3400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:57:54.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.329+0000 7f1ee3400640 1 -- 172.21.15.83:0/404938101 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef0030060 con 0x56009e305350 2024-04-24T08:57:54.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.329+0000 7f1ee3400640 1 -- 172.21.15.83:0/404938101 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f1ef0005230 con 0x56009e305350 2024-04-24T08:57:54.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.329+0000 7f1f0b608740 1 -- 172.21.15.83:0/404938101 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ef4010c20 con 0x56009e305350 2024-04-24T08:57:54.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.329+0000 7f1ee3400640 1 -- 172.21.15.83:0/404938101 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f1ef0037050 con 0x56009e305350 2024-04-24T08:57:54.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.329+0000 7f1f0b608740 10 client.17847 fetch_fsmap learned FSMap version 33 2024-04-24T08:57:54.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.329+0000 7f1f0b608740 1 -- 172.21.15.83:0/404938101 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56009e22f810 con 0x56009e305350 2024-04-24T08:57:54.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.329+0000 7f1ee3400640 1 -- 172.21.15.83:0/404938101 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1ef4010c20 con 0x56009e305350 2024-04-24T08:57:54.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.330+0000 7f1ee3400640 1 -- 172.21.15.83:0/404938101 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f1ef000f040 con 0x56009e305350 2024-04-24T08:57:54.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.330+0000 7f1f0b608740 10 client.17847 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T08:57:54.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.330+0000 7f1f0b608740 10 client.17847 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:57:54.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.330+0000 7f1f0b608740 20 client.17847 populate_metadata read hostname 'smithi083' 2024-04-24T08:57:54.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.330+0000 7f1f0b608740 1 --2- 172.21.15.83:0/404938101 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56009e386090 0x56009e388480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:54.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.330+0000 7f1f0b608740 4 client.17847 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T08:57:54.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.330+0000 7f1f0b608740 1 -- 172.21.15.83:0/404938101 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56009e271a80 con 0x56009e386090 2024-04-24T08:57:54.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.330+0000 7f1f05e00640 1 --2- 172.21.15.83:0/404938101 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56009e386090 0x56009e388480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:54.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.332+0000 7f1f05e00640 1 --2- 172.21.15.83:0/404938101 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56009e386090 0x56009e388480 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7e9e47439861ef5 server_cookie=109e2cc3bea18f5b in_seq=0 out_seq=0 2024-04-24T08:57:54.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.332+0000 7f1ee3400640 10 client.17847 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:57:54.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.332+0000 7f1ee3400640 1 -- 172.21.15.83:0/404938101 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56009e271a80 con 0x56009e386090 2024-04-24T08:57:54.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.332+0000 7f1ee3400640 10 client.17847 handle_command_reply: tid=0 2024-04-24T08:57:54.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.332+0000 7f1f0b608740 1 client.17847 shutdown 2024-04-24T08:57:54.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.333+0000 7f1f0b608740 1 -- 172.21.15.83:0/404938101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56009e305350 msgr2=0x56009e318180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:54.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.333+0000 7f1f0b608740 1 --2- 172.21.15.83:0/404938101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56009e305350 0x56009e318180 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f1ef0015000 tx=0x7f1ef000e3d0 comp rx=0 tx=0).stop 2024-04-24T08:57:54.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.333+0000 7f1f0b608740 1 -- 172.21.15.83:0/404938101 shutdown_connections 2024-04-24T08:57:54.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.333+0000 7f1f0b608740 1 --2- 172.21.15.83:0/404938101 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56009e386090 0x56009e388480 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:54.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.333+0000 7f1f0b608740 1 --2- 172.21.15.83:0/404938101 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56009e30fa20 0x56009e3090f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:54.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.333+0000 7f1f0b608740 1 --2- 172.21.15.83:0/404938101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56009e305350 0x56009e318180 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:54.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.333+0000 7f1f0b608740 1 --2- 172.21.15.83:0/404938101 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56009e304a30 0x56009e317c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:54.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.333+0000 7f1ee3400640 0 client.17847 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:57:54.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.333+0000 7f1f0b608740 1 -- 172.21.15.83:0/404938101 >> 172.21.15.83:0/404938101 conn(0x56009e300d50 msgr2=0x56009e2fffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:54.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.333+0000 7f1f0b608740 1 -- 172.21.15.83:0/404938101 shutdown_connections 2024-04-24T08:57:54.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.333+0000 7f1f0b608740 1 -- 172.21.15.83:0/404938101 wait complete. 2024-04-24T08:57:54.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.333+0000 7f1f0b608740 20 client.17847 trim_cache size 0 max 16384 2024-04-24T08:57:54.335 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T08:57:54.335 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-04-24T08:57:54.336 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-04-24T08:57:54.336 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T08:57:54.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.334+0000 7f1f07200640 1 -- 172.21.15.83:0/2913424315 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f1ee80a3d20 msgr2=0x7f1ee80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:54.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.334+0000 7f1f07200640 1 --2- 172.21.15.83:0/2913424315 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f1ee80a3d20 0x7f1ee80a61e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1ef4005fd0 tx=0x7f1ef4009350 comp rx=0 tx=0).stop 2024-04-24T08:57:54.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.334+0000 7f1f07200640 1 -- 172.21.15.83:0/2913424315 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f00112eb0 msgr2=0x7f1f0019eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:54.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.334+0000 7f1f07200640 1 --2- 172.21.15.83:0/2913424315 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f00112eb0 0x7f1f0019eab0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f1efc005e30 tx=0x7f1efc003f10 comp rx=0 tx=0).stop 2024-04-24T08:57:54.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.334+0000 7f1f07200640 1 -- 172.21.15.83:0/2913424315 shutdown_connections 2024-04-24T08:57:54.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.334+0000 7f1f07200640 1 --2- 172.21.15.83:0/2913424315 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f1ee80a3d20 0x7f1ee80a61e0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:54.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.334+0000 7f1f07200640 1 --2- 172.21.15.83:0/2913424315 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f00112eb0 0x7f1f0019eab0 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:54.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.334+0000 7f1f07200640 1 --2- 172.21.15.83:0/2913424315 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f00074cc0 0x7f1f001a4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:54.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.334+0000 7f1f07200640 1 --2- 172.21.15.83:0/2913424315 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1f000743a0 0x7f1f001a4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:54.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.334+0000 7f1f07200640 1 -- 172.21.15.83:0/2913424315 >> 172.21.15.83:0/2913424315 conn(0x7f1f000fd9d0 msgr2=0x7f1f000ff230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:54.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.334+0000 7f1f07200640 1 -- 172.21.15.83:0/2913424315 shutdown_connections 2024-04-24T08:57:54.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.334+0000 7f1f07200640 1 -- 172.21.15.83:0/2913424315 wait complete. 2024-04-24T08:57:54.355 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T08:57:54.355 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T08:57:54.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.557+0000 7f7b37e00640 1 -- 172.21.15.83:0/2906524899 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b381071a0 msgr2=0x7f7b3810ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:54.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.557+0000 7f7b37e00640 1 --2- 172.21.15.83:0/2906524899 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b381071a0 0x7f7b3810ed40 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f7b30009900 tx=0x7f7b3002a410 comp rx=0 tx=0).stop 2024-04-24T08:57:54.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.557+0000 7f7b37e00640 1 -- 172.21.15.83:0/2906524899 shutdown_connections 2024-04-24T08:57:54.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.557+0000 7f7b37e00640 1 --2- 172.21.15.83:0/2906524899 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b381071a0 0x7f7b3810ed40 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:54.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.557+0000 7f7b37e00640 1 --2- 172.21.15.83:0/2906524899 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7b38069db0 0x7f7b38106a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:54.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.557+0000 7f7b37e00640 1 --2- 172.21.15.83:0/2906524899 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7b38069400 0x7f7b380697e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:54.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.557+0000 7f7b37e00640 1 -- 172.21.15.83:0/2906524899 >> 172.21.15.83:0/2906524899 conn(0x7f7b38076840 msgr2=0x7f7b38076c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:54.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.557+0000 7f7b37e00640 1 -- 172.21.15.83:0/2906524899 shutdown_connections 2024-04-24T08:57:54.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.557+0000 7f7b37e00640 1 -- 172.21.15.83:0/2906524899 wait complete. 2024-04-24T08:57:54.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.558+0000 7f7b37e00640 1 Processor -- start 2024-04-24T08:57:54.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.558+0000 7f7b37e00640 1 -- start start 2024-04-24T08:57:54.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.558+0000 7f7b37e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7b38069400 0x7f7b381a0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:54.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.558+0000 7f7b37e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b381071a0 0x7f7b381a0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:54.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.558+0000 7f7b37e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7b3819a600 0x7f7b3819aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:54.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.558+0000 7f7b37e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7b38112130 con 0x7f7b381071a0 2024-04-24T08:57:54.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.558+0000 7f7b37e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7b38111fb0 con 0x7f7b38069400 2024-04-24T08:57:54.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.558+0000 7f7b37e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7b381122b0 con 0x7f7b3819a600 2024-04-24T08:57:54.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.558+0000 7f7b36000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b381071a0 0x7f7b381a0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:54.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.559+0000 7f7b36000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b381071a0 0x7f7b381a0940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41590/0 (socket says 172.21.15.83:41590) 2024-04-24T08:57:54.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.559+0000 7f7b36000640 1 -- 172.21.15.83:0/425860834 learned_addr learned my addr 172.21.15.83:0/425860834 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:57:54.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.559+0000 7f7b36a00640 1 --2- 172.21.15.83:0/425860834 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7b38069400 0x7f7b381a0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:54.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.559+0000 7f7b37400640 1 --2- 172.21.15.83:0/425860834 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7b3819a600 0x7f7b3819aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:54.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.559+0000 7f7b36000640 1 -- 172.21.15.83:0/425860834 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7b3819a600 msgr2=0x7f7b3819aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:54.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.559+0000 7f7b36000640 1 --2- 172.21.15.83:0/425860834 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7b3819a600 0x7f7b3819aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:54.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.559+0000 7f7b36000640 1 -- 172.21.15.83:0/425860834 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7b38069400 msgr2=0x7f7b381a0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:54.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.559+0000 7f7b36000640 1 --2- 172.21.15.83:0/425860834 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7b38069400 0x7f7b381a0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:54.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.559+0000 7f7b36000640 1 -- 172.21.15.83:0/425860834 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b3819b120 con 0x7f7b381071a0 2024-04-24T08:57:54.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.559+0000 7f7b36000640 1 --2- 172.21.15.83:0/425860834 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b381071a0 0x7f7b381a0940 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f7b2800cce0 tx=0x7f7b2800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:54.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.560+0000 7f7b3ca00640 1 -- 172.21.15.83:0/425860834 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b2800f450 con 0x7f7b381071a0 2024-04-24T08:57:54.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.560+0000 7f7b3ca00640 1 -- 172.21.15.83:0/425860834 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7b2800faa0 con 0x7f7b381071a0 2024-04-24T08:57:54.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.560+0000 7f7b37e00640 1 -- 172.21.15.83:0/425860834 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b3819b410 con 0x7f7b381071a0 2024-04-24T08:57:54.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.560+0000 7f7b3ca00640 1 -- 172.21.15.83:0/425860834 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b28018c50 con 0x7f7b381071a0 2024-04-24T08:57:54.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.560+0000 7f7b37e00640 1 -- 172.21.15.83:0/425860834 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b381a7610 con 0x7f7b381071a0 2024-04-24T08:57:54.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.561+0000 7f7b37e00640 1 -- 172.21.15.83:0/425860834 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b38068d40 con 0x7f7b381071a0 2024-04-24T08:57:54.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.563+0000 7f7b3ca00640 1 -- 172.21.15.83:0/425860834 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7b2800fc50 con 0x7f7b381071a0 2024-04-24T08:57:54.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.565+0000 7f7b3ca00640 1 --2- 172.21.15.83:0/425860834 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f7b1c0a3d20 0x7f7b1c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:54.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.565+0000 7f7b3ca00640 1 -- 172.21.15.83:0/425860834 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f7b280d1120 con 0x7f7b381071a0 2024-04-24T08:57:54.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.565+0000 7f7b36a00640 1 --2- 172.21.15.83:0/425860834 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f7b1c0a3d20 0x7f7b1c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:54.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.566+0000 7f7b36a00640 1 --2- 172.21.15.83:0/425860834 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f7b1c0a3d20 0x7f7b1c0a61e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7b20005fd0 tx=0x7f7b20005e20 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:54.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.571+0000 7f7b3ca00640 1 -- 172.21.15.83:0/425860834 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7b28014030 con 0x7f7b381071a0 2024-04-24T08:57:54.583 INFO:tasks.workunit.client.0.smithi083.stdout: 1 32360 0.65 MB/sec execute 148 sec latency 48.206 ms 2024-04-24T08:57:54.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.876+0000 7f7b37e00640 1 -- 172.21.15.83:0/425860834 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7b3819bae0 con 0x7f7b381071a0 2024-04-24T08:57:54.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.877+0000 7f7b3ca00640 1 -- 172.21.15.83:0/425860834 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11944 (secure 0 0 0) 0x7f7b2809d390 con 0x7f7b381071a0 2024-04-24T08:57:54.878 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:57:54.878 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/358872257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":358872257},{"type":"v1","addr":"172.21.15.83:6835","nonce":358872257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3157070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3157070851},{"type":"v1","addr":"172.21.15.83:6837","nonce":3157070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":33,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T08:48:29.422576+0000","modified":"2024-04-24T08:56:17.048967+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34520,"mds_1":34493,"mds_2":34451,"mds_3":34427,"mds_4":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14658":{"gid":14658,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6839/4085470844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":4085470844},{"type":"v1","addr":"172.21.15.83:6839","nonce":4085470844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14676":{"gid":14676,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6841/2634754210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":2634754210},{"type":"v1","addr":"172.21.15.83:6841","nonce":2634754210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24382":{"gid":24382,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/1859921310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1859921310},{"type":"v1","addr":"172.21.15.189:6833","nonce":1859921310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/2241923197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2241923197},{"type":"v1","addr":"172.21.15.189:6835","nonce":2241923197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6837/3172311220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":3172311220},{"type":"v1","addr":"172.21.15.189:6837","nonce":3172311220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.189:6839/2548965336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":2548965336},{"type":"v1","addr":"172.21.15.189:6839","nonce":2548965336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.138:6833/1157967658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1157967658},{"type":"v1","addr":"172.21.15.138:6833","nonce":1157967658}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.138:6835/1909219758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1909219758},{"type":"v1","addr":"172.21.15.138:6835","nonce":1909219758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34493":{"gid":34493,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.138:6837/900566754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":900566754},{"type":"v1","addr":"172.21.15.138:6837","nonce":900566754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34520":{"gid":34520,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.138:6839/2236932393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2236932393},{"type":"v1","addr":"172.21.15.138:6839","nonce":2236932393}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34520,"qdb_cluster":[24454,34520,34427,34451,34493]},"id":1}]} 2024-04-24T08:57:54.878 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 33 2024-04-24T08:57:54.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.880+0000 7f7b37e00640 1 -- 172.21.15.83:0/425860834 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f7b1c0a3d20 msgr2=0x7f7b1c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:54.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.880+0000 7f7b37e00640 1 --2- 172.21.15.83:0/425860834 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f7b1c0a3d20 0x7f7b1c0a61e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7b20005fd0 tx=0x7f7b20005e20 comp rx=0 tx=0).stop 2024-04-24T08:57:54.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.880+0000 7f7b37e00640 1 -- 172.21.15.83:0/425860834 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b381071a0 msgr2=0x7f7b381a0940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:54.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.881+0000 7f7b37e00640 1 --2- 172.21.15.83:0/425860834 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b381071a0 0x7f7b381a0940 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f7b2800cce0 tx=0x7f7b2800a620 comp rx=0 tx=0).stop 2024-04-24T08:57:54.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.881+0000 7f7b37e00640 1 -- 172.21.15.83:0/425860834 shutdown_connections 2024-04-24T08:57:54.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.881+0000 7f7b37e00640 1 --2- 172.21.15.83:0/425860834 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f7b1c0a3d20 0x7f7b1c0a61e0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:54.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.881+0000 7f7b37e00640 1 --2- 172.21.15.83:0/425860834 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7b3819a600 0x7f7b3819aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:54.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.881+0000 7f7b37e00640 1 --2- 172.21.15.83:0/425860834 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b381071a0 0x7f7b381a0940 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:54.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.881+0000 7f7b37e00640 1 --2- 172.21.15.83:0/425860834 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7b38069400 0x7f7b381a0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:54.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.881+0000 7f7b37e00640 1 -- 172.21.15.83:0/425860834 >> 172.21.15.83:0/425860834 conn(0x7f7b38076840 msgr2=0x7f7b3810d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:54.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.881+0000 7f7b37e00640 1 -- 172.21.15.83:0/425860834 shutdown_connections 2024-04-24T08:57:54.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:54.881+0000 7f7b37e00640 1 -- 172.21.15.83:0/425860834 wait complete. 2024-04-24T08:57:54.904 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-24T08:57:55.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.098+0000 7f9594a00640 1 -- 172.21.15.83:0/2674374068 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f959010e6e0 msgr2=0x7f9590110b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:55.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.098+0000 7f9594a00640 1 --2- 172.21.15.83:0/2674374068 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f959010e6e0 0x7f9590110b30 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f9588009ca0 tx=0x7f958802a950 comp rx=0 tx=0).stop 2024-04-24T08:57:55.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.098+0000 7f9594a00640 1 -- 172.21.15.83:0/2674374068 shutdown_connections 2024-04-24T08:57:55.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.099+0000 7f9594a00640 1 --2- 172.21.15.83:0/2674374068 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f959010e6e0 0x7f9590110b30 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.099+0000 7f9594a00640 1 --2- 172.21.15.83:0/2674374068 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9590074d50 0x7f959010e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.099+0000 7f9594a00640 1 --2- 172.21.15.83:0/2674374068 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9590074430 0x7f9590074810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.099+0000 7f9594a00640 1 -- 172.21.15.83:0/2674374068 >> 172.21.15.83:0/2674374068 conn(0x7f95900fd9d0 msgr2=0x7f95900ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:55.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.099+0000 7f9594a00640 1 -- 172.21.15.83:0/2674374068 shutdown_connections 2024-04-24T08:57:55.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.099+0000 7f9594a00640 1 -- 172.21.15.83:0/2674374068 wait complete. 2024-04-24T08:57:55.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.100+0000 7f9594a00640 1 Processor -- start 2024-04-24T08:57:55.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.100+0000 7f9594a00640 1 -- start start 2024-04-24T08:57:55.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.100+0000 7f9594a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9590074430 0x7f95901a0590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:55.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.100+0000 7f9594a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9590074d50 0x7f95901a0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:55.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.100+0000 7f9594a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f959010e6e0 0x7f959019a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:55.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.100+0000 7f9594a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9590114360 con 0x7f959010e6e0 2024-04-24T08:57:55.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.100+0000 7f9594a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f95901141e0 con 0x7f9590074430 2024-04-24T08:57:55.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.100+0000 7f9594a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f95901144e0 con 0x7f9590074d50 2024-04-24T08:57:55.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.100+0000 7f958f200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f959010e6e0 0x7f959019a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:55.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.101+0000 7f958e800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9590074430 0x7f95901a0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:55.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.101+0000 7f958f200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f959010e6e0 0x7f959019a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41612/0 (socket says 172.21.15.83:41612) 2024-04-24T08:57:55.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.101+0000 7f958f200640 1 -- 172.21.15.83:0/1691250262 learned_addr learned my addr 172.21.15.83:0/1691250262 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:57:55.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.101+0000 7f9587e00640 1 --2- 172.21.15.83:0/1691250262 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9590074d50 0x7f95901a0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:55.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.101+0000 7f958f200640 1 -- 172.21.15.83:0/1691250262 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9590074d50 msgr2=0x7f95901a0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:55.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.101+0000 7f958f200640 1 --2- 172.21.15.83:0/1691250262 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9590074d50 0x7f95901a0ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.101+0000 7f958f200640 1 -- 172.21.15.83:0/1691250262 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9590074430 msgr2=0x7f95901a0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:55.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.101+0000 7f958f200640 1 --2- 172.21.15.83:0/1691250262 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9590074430 0x7f95901a0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.101+0000 7f958f200640 1 -- 172.21.15.83:0/1691250262 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f959019af60 con 0x7f959010e6e0 2024-04-24T08:57:55.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.101+0000 7f958f200640 1 --2- 172.21.15.83:0/1691250262 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f959010e6e0 0x7f959019a6a0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f9588009dc0 tx=0x7f9588003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:55.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.102+0000 7f9587400640 1 -- 172.21.15.83:0/1691250262 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9588034040 con 0x7f959010e6e0 2024-04-24T08:57:55.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.102+0000 7f9587400640 1 -- 172.21.15.83:0/1691250262 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9588003e50 con 0x7f959010e6e0 2024-04-24T08:57:55.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.102+0000 7f9587400640 1 -- 172.21.15.83:0/1691250262 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95880045b0 con 0x7f959010e6e0 2024-04-24T08:57:55.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.102+0000 7f9594a00640 1 -- 172.21.15.83:0/1691250262 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f959019b1f0 con 0x7f959010e6e0 2024-04-24T08:57:55.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.102+0000 7f9594a00640 1 -- 172.21.15.83:0/1691250262 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95901a7330 con 0x7f959010e6e0 2024-04-24T08:57:55.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.106+0000 7f95983cf740 1 Processor -- start 2024-04-24T08:57:55.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.107+0000 7f9587400640 1 -- 172.21.15.83:0/1691250262 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9588037020 con 0x7f959010e6e0 2024-04-24T08:57:55.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.107+0000 7f95983cf740 1 -- start start 2024-04-24T08:57:55.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.107+0000 7f95983cf740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55832f2d0b40 0x55832f2d0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:55.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.107+0000 7f95983cf740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55832f2d1460 0x55832f2db5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:55.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.107+0000 7f95983cf740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55832f2dbb30 0x55832f2ddf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:55.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.107+0000 7f95983cf740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55832f2361d0 con 0x55832f2d1460 2024-04-24T08:57:55.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.107+0000 7f95983cf740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55832f1dc450 con 0x55832f2dbb30 2024-04-24T08:57:55.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.107+0000 7f95983cf740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55832f232190 con 0x55832f2d0b40 2024-04-24T08:57:55.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.108+0000 7f958e800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55832f2d1460 0x55832f2db5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:55.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.108+0000 7f958e800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55832f2d1460 0x55832f2db5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41624/0 (socket says 172.21.15.83:41624) 2024-04-24T08:57:55.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.108+0000 7f958e800640 1 -- 172.21.15.83:0/3400796154 learned_addr learned my addr 172.21.15.83:0/3400796154 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:57:55.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.108+0000 7f9587e00640 1 --2- 172.21.15.83:0/3400796154 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55832f2dbb30 0x55832f2ddf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:55.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.108+0000 7f9587e00640 1 --2- 172.21.15.83:0/3400796154 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55832f2d0b40 0x55832f2d0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:55.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.108+0000 7f958e800640 1 -- 172.21.15.83:0/3400796154 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55832f2d0b40 msgr2=0x55832f2d0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:55.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.108+0000 7f958e800640 1 --2- 172.21.15.83:0/3400796154 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55832f2d0b40 0x55832f2d0f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.108+0000 7f958e800640 1 -- 172.21.15.83:0/3400796154 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55832f2dbb30 msgr2=0x55832f2ddf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:55.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.108+0000 7f958e800640 1 --2- 172.21.15.83:0/3400796154 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55832f2dbb30 0x55832f2ddf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.108+0000 7f958e800640 1 -- 172.21.15.83:0/3400796154 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55832f299310 con 0x55832f2d1460 2024-04-24T08:57:55.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.108+0000 7f9587e00640 1 --2- 172.21.15.83:0/3400796154 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55832f2d0b40 0x55832f2d0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:57:55.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.108+0000 7f9587400640 1 --2- 172.21.15.83:0/1691250262 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f95740a3d20 0x7f95740a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:55.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.108+0000 7f958e800640 1 --2- 172.21.15.83:0/3400796154 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55832f2d1460 0x55832f2db5f0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f9580009d20 tx=0x7f9580011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79dca1c42090158c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:55.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.109+0000 7f9587400640 1 -- 172.21.15.83:0/1691250262 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f95880f1830 con 0x7f959010e6e0 2024-04-24T08:57:55.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.109+0000 7f9587e00640 1 --2- 172.21.15.83:0/1691250262 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f95740a3d20 0x7f95740a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:55.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.109+0000 7f957f400640 1 -- 172.21.15.83:0/3400796154 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f958001e070 con 0x55832f2d1460 2024-04-24T08:57:55.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.109+0000 7f957f400640 1 -- 172.21.15.83:0/3400796154 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9580019ae0 con 0x55832f2d1460 2024-04-24T08:57:55.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.109+0000 7f957f400640 1 -- 172.21.15.83:0/3400796154 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f958000eac0 con 0x55832f2d1460 2024-04-24T08:57:55.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.110+0000 7f95983cf740 1 -- 172.21.15.83:0/3400796154 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55832f2d1460 msgr2=0x55832f2db5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:55.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.110+0000 7f95983cf740 1 --2- 172.21.15.83:0/3400796154 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55832f2d1460 0x55832f2db5f0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f9580009d20 tx=0x7f9580011810 comp rx=0 tx=0).stop 2024-04-24T08:57:55.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.110+0000 7f9587e00640 1 --2- 172.21.15.83:0/1691250262 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f95740a3d20 0x7f95740a61e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x55832f2cdff0 tx=0x7f9578014040 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:55.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.110+0000 7f95983cf740 1 -- 172.21.15.83:0/3400796154 shutdown_connections 2024-04-24T08:57:55.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.110+0000 7f95983cf740 1 --2- 172.21.15.83:0/3400796154 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55832f2dbb30 0x55832f2ddf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.110+0000 7f95983cf740 1 --2- 172.21.15.83:0/3400796154 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55832f2d1460 0x55832f2db5f0 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.110+0000 7f95983cf740 1 --2- 172.21.15.83:0/3400796154 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55832f2d0b40 0x55832f2d0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.110+0000 7f95983cf740 1 -- 172.21.15.83:0/3400796154 >> 172.21.15.83:0/3400796154 conn(0x55832f2cce60 msgr2=0x55832f2cd240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:55.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.110+0000 7f95983cf740 1 -- 172.21.15.83:0/3400796154 shutdown_connections 2024-04-24T08:57:55.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.110+0000 7f95983cf740 1 -- 172.21.15.83:0/3400796154 wait complete. 2024-04-24T08:57:55.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.111+0000 7f95983cf740 1 -- start start 2024-04-24T08:57:55.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.111+0000 7f95983cf740 1 Processor -- start 2024-04-24T08:57:55.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.112+0000 7f95983cf740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55832f2d0b40 0x55832f2db5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:55.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.112+0000 7f95983cf740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55832f2d1460 0x55832f2d88a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:55.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.112+0000 7f95983cf740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55832f2dbb30 0x55832f2d8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:55.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.112+0000 7f95983cf740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55832f2de9c0 con 0x55832f2d1460 2024-04-24T08:57:55.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.112+0000 7f95983cf740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55832f1ef810 con 0x55832f2dbb30 2024-04-24T08:57:55.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.112+0000 7f95983cf740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55832f2deb40 con 0x55832f2d0b40 2024-04-24T08:57:55.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.112+0000 7f95983cf740 10 client.0 _ll_register_callbacks cb 0x55832f1bd140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:57:55.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.112+0000 7f9587e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55832f2d1460 0x55832f2d88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:55.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.112+0000 7f9587e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55832f2d1460 0x55832f2d88a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41636/0 (socket says 172.21.15.83:41636) 2024-04-24T08:57:55.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.112+0000 7f9587e00640 1 -- 172.21.15.83:0/2843472507 learned_addr learned my addr 172.21.15.83:0/2843472507 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:57:55.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.112+0000 7f958f200640 1 --2- 172.21.15.83:0/2843472507 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55832f2dbb30 0x55832f2d8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:55.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.112+0000 7f958e800640 1 --2- 172.21.15.83:0/2843472507 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55832f2d0b40 0x55832f2db5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:55.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.113+0000 7f9587e00640 1 -- 172.21.15.83:0/2843472507 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55832f2d0b40 msgr2=0x55832f2db5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:55.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.113+0000 7f9587e00640 1 --2- 172.21.15.83:0/2843472507 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55832f2d0b40 0x55832f2db5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.113+0000 7f9587e00640 1 -- 172.21.15.83:0/2843472507 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55832f2dbb30 msgr2=0x55832f2d8fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:55.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.113+0000 7f9587e00640 1 --2- 172.21.15.83:0/2843472507 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55832f2dbb30 0x55832f2d8fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.113+0000 7f9587e00640 1 -- 172.21.15.83:0/2843472507 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55832f2003f0 con 0x55832f2d1460 2024-04-24T08:57:55.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.113+0000 7f9573e00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:57:55.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.113+0000 7f9587e00640 1 --2- 172.21.15.83:0/2843472507 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55832f2d1460 0x55832f2d88a0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f9578003870 tx=0x7f95780095e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12b0b14bd1cd78cb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:55.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.113+0000 7f9573e00640 1 -- 172.21.15.83:0/2843472507 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9578003d30 con 0x55832f2d1460 2024-04-24T08:57:55.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.114+0000 7f9573e00640 1 -- 172.21.15.83:0/2843472507 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f9578010930 con 0x55832f2d1460 2024-04-24T08:57:55.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.114+0000 7f95983cf740 1 -- 172.21.15.83:0/2843472507 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9580019ae0 con 0x55832f2d1460 2024-04-24T08:57:55.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.114+0000 7f9573e00640 1 -- 172.21.15.83:0/2843472507 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f9578010b30 con 0x55832f2d1460 2024-04-24T08:57:55.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.114+0000 7f95983cf740 10 client.17865 fetch_fsmap learned FSMap version 33 2024-04-24T08:57:55.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.114+0000 7f95983cf740 1 -- 172.21.15.83:0/2843472507 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55832ef483f0 con 0x55832f2d1460 2024-04-24T08:57:55.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.115+0000 7f9573e00640 1 -- 172.21.15.83:0/2843472507 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9580019ae0 con 0x55832f2d1460 2024-04-24T08:57:55.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.115+0000 7f9573e00640 1 -- 172.21.15.83:0/2843472507 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f957801b030 con 0x55832f2d1460 2024-04-24T08:57:55.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.115+0000 7f95983cf740 10 client.17865 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T08:57:55.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.115+0000 7f95983cf740 10 client.17865 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-24T08:57:55.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.115+0000 7f95983cf740 20 client.17865 populate_metadata read hostname 'smithi083' 2024-04-24T08:57:55.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.115+0000 7f95983cf740 1 --2- 172.21.15.83:0/2843472507 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55832f2ebd40 0x55832f2ee1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:55.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.115+0000 7f95983cf740 4 client.17865 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:57:55.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.115+0000 7f95983cf740 1 -- 172.21.15.83:0/2843472507 --> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55832f243bf0 con 0x55832f2ebd40 2024-04-24T08:57:55.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.116+0000 7f958e800640 1 --2- 172.21.15.83:0/2843472507 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55832f2ebd40 0x55832f2ee1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:55.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.117+0000 7f958e800640 1 --2- 172.21.15.83:0/2843472507 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55832f2ebd40 0x55832f2ee1c0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ff7aa3759b1c7a4e server_cookie=1e19841c41fb2644 in_seq=0 out_seq=0 2024-04-24T08:57:55.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.117+0000 7f9573e00640 10 client.17865 ms_handle_connect on v2:172.21.15.189:6838/2548965336 2024-04-24T08:57:55.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.118+0000 7f9573e00640 1 -- 172.21.15.83:0/2843472507 <== mds.4 v2:172.21.15.189:6838/2548965336 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55832f243bf0 con 0x55832f2ebd40 2024-04-24T08:57:55.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.118+0000 7f9573e00640 10 client.17865 handle_command_reply: tid=0 2024-04-24T08:57:55.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.118+0000 7f95983cf740 1 client.17865 shutdown 2024-04-24T08:57:55.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.119+0000 7f95983cf740 1 -- 172.21.15.83:0/2843472507 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55832f2d1460 msgr2=0x55832f2d88a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:55.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.119+0000 7f95983cf740 1 --2- 172.21.15.83:0/2843472507 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55832f2d1460 0x55832f2d88a0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f9578003870 tx=0x7f95780095e0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.119+0000 7f95983cf740 1 -- 172.21.15.83:0/2843472507 shutdown_connections 2024-04-24T08:57:55.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.119+0000 7f95983cf740 1 --2- 172.21.15.83:0/2843472507 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55832f2ebd40 0x55832f2ee1c0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.119+0000 7f95983cf740 1 --2- 172.21.15.83:0/2843472507 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55832f2dbb30 0x55832f2d8fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.119+0000 7f95983cf740 1 --2- 172.21.15.83:0/2843472507 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55832f2d1460 0x55832f2d88a0 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.119+0000 7f95983cf740 1 --2- 172.21.15.83:0/2843472507 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55832f2d0b40 0x55832f2db5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.119+0000 7f9573e00640 0 client.17865 ms_handle_reset on v2:172.21.15.189:6838/2548965336 2024-04-24T08:57:55.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.119+0000 7f95983cf740 1 -- 172.21.15.83:0/2843472507 >> 172.21.15.83:0/2843472507 conn(0x55832f2cce60 msgr2=0x55832f2d2b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:55.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.119+0000 7f95983cf740 1 -- 172.21.15.83:0/2843472507 shutdown_connections 2024-04-24T08:57:55.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.119+0000 7f95983cf740 1 -- 172.21.15.83:0/2843472507 wait complete. 2024-04-24T08:57:55.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.119+0000 7f95983cf740 20 client.17865 trim_cache size 0 max 16384 2024-04-24T08:57:55.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.134+0000 7f95983cf740 1 Processor -- start 2024-04-24T08:57:55.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.134+0000 7f95983cf740 1 -- start start 2024-04-24T08:57:55.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.135+0000 7f95983cf740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55832f2d0b40 0x55832f2e78b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:55.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.135+0000 7f95983cf740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55832f2d1460 0x55832f2d7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:55.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.135+0000 7f95983cf740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55832f2dbb30 0x55832f2db0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:55.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.135+0000 7f95983cf740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f95901b1450 con 0x55832f2d0b40 2024-04-24T08:57:55.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.135+0000 7f95983cf740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f95901b2f90 con 0x55832f2dbb30 2024-04-24T08:57:55.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.135+0000 7f95983cf740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f95901b1b20 con 0x55832f2d1460 2024-04-24T08:57:55.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.135+0000 7f958e800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55832f2d0b40 0x55832f2e78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:55.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.135+0000 7f958e800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55832f2d0b40 0x55832f2e78b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41646/0 (socket says 172.21.15.83:41646) 2024-04-24T08:57:55.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.135+0000 7f958e800640 1 -- 172.21.15.83:0/1010120562 learned_addr learned my addr 172.21.15.83:0/1010120562 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:57:55.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.135+0000 7f958f200640 1 --2- 172.21.15.83:0/1010120562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55832f2dbb30 0x55832f2db0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:55.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.136+0000 7f9587e00640 1 --2- 172.21.15.83:0/1010120562 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55832f2d1460 0x55832f2d7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:55.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.136+0000 7f958e800640 1 -- 172.21.15.83:0/1010120562 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55832f2d1460 msgr2=0x55832f2d7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:55.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.136+0000 7f958e800640 1 --2- 172.21.15.83:0/1010120562 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55832f2d1460 0x55832f2d7880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.136+0000 7f958e800640 1 -- 172.21.15.83:0/1010120562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55832f2dbb30 msgr2=0x55832f2db0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:55.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.136+0000 7f958e800640 1 --2- 172.21.15.83:0/1010120562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55832f2dbb30 0x55832f2db0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.136+0000 7f958e800640 1 -- 172.21.15.83:0/1010120562 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55832f2d1290 con 0x55832f2d0b40 2024-04-24T08:57:55.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.136+0000 7f9587e00640 1 --2- 172.21.15.83:0/1010120562 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55832f2d1460 0x55832f2d7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:57:55.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.136+0000 7f958f200640 1 --2- 172.21.15.83:0/1010120562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55832f2dbb30 0x55832f2db0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:57:55.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.136+0000 7f958e800640 1 --2- 172.21.15.83:0/1010120562 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55832f2d0b40 0x55832f2e78b0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f95800058c0 tx=0x7f9580005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15dc65aa99bbfadc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:55.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.136+0000 7f957ea00640 1 -- 172.21.15.83:0/1010120562 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f958001e040 con 0x55832f2d0b40 2024-04-24T08:57:55.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.136+0000 7f957ea00640 1 -- 172.21.15.83:0/1010120562 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9580029030 con 0x55832f2d0b40 2024-04-24T08:57:55.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.136+0000 7f957ea00640 1 -- 172.21.15.83:0/1010120562 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f958000eb90 con 0x55832f2d0b40 2024-04-24T08:57:55.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.137+0000 7f95983cf740 1 -- 172.21.15.83:0/1010120562 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55832f2d0b40 msgr2=0x55832f2e78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:55.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.137+0000 7f95983cf740 1 --2- 172.21.15.83:0/1010120562 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55832f2d0b40 0x55832f2e78b0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f95800058c0 tx=0x7f9580005fd0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.137+0000 7f95983cf740 1 -- 172.21.15.83:0/1010120562 shutdown_connections 2024-04-24T08:57:55.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.137+0000 7f95983cf740 1 --2- 172.21.15.83:0/1010120562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55832f2dbb30 0x55832f2db0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.137+0000 7f95983cf740 1 --2- 172.21.15.83:0/1010120562 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55832f2d1460 0x55832f2d7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.137+0000 7f95983cf740 1 --2- 172.21.15.83:0/1010120562 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55832f2d0b40 0x55832f2e78b0 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.137+0000 7f95983cf740 1 -- 172.21.15.83:0/1010120562 >> 172.21.15.83:0/1010120562 conn(0x55832f2cce60 msgr2=0x55832f2d7170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:55.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.137+0000 7f95983cf740 1 -- 172.21.15.83:0/1010120562 shutdown_connections 2024-04-24T08:57:55.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.137+0000 7f95983cf740 1 -- 172.21.15.83:0/1010120562 wait complete. 2024-04-24T08:57:55.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.138+0000 7f95983cf740 1 -- start start 2024-04-24T08:57:55.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.138+0000 7f95983cf740 1 Processor -- start 2024-04-24T08:57:55.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.139+0000 7f95983cf740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55832f2d0b40 0x55832f2dc700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:55.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.139+0000 7f95983cf740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55832f2d1460 0x55832f2dcc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:55.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.139+0000 7f95983cf740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55832f2dbb30 0x55832f2e2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:55.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.139+0000 7f95983cf740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f95901b28c0 con 0x55832f2d1460 2024-04-24T08:57:55.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.139+0000 7f95983cf740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f95901b21f0 con 0x55832f2dbb30 2024-04-24T08:57:55.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.139+0000 7f95983cf740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f95901af430 con 0x55832f2d0b40 2024-04-24T08:57:55.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.139+0000 7f95983cf740 10 client.0 _ll_register_callbacks cb 0x7f95780099e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:57:55.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.139+0000 7f9587e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55832f2d1460 0x55832f2dcc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:55.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.139+0000 7f9587e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55832f2d1460 0x55832f2dcc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41654/0 (socket says 172.21.15.83:41654) 2024-04-24T08:57:55.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.139+0000 7f9587e00640 1 -- 172.21.15.83:0/439017229 learned_addr learned my addr 172.21.15.83:0/439017229 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:57:55.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.139+0000 7f958e800640 1 --2- 172.21.15.83:0/439017229 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55832f2d0b40 0x55832f2dc700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:55.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.140+0000 7f958f200640 1 --2- 172.21.15.83:0/439017229 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55832f2dbb30 0x55832f2e2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:55.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.140+0000 7f9587e00640 1 -- 172.21.15.83:0/439017229 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55832f2d0b40 msgr2=0x55832f2dc700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:55.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.140+0000 7f9587e00640 1 --2- 172.21.15.83:0/439017229 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55832f2d0b40 0x55832f2dc700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.140+0000 7f9587e00640 1 -- 172.21.15.83:0/439017229 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55832f2dbb30 msgr2=0x55832f2e2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:55.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.140+0000 7f9587e00640 1 --2- 172.21.15.83:0/439017229 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55832f2dbb30 0x55832f2e2dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.140+0000 7f9587e00640 1 -- 172.21.15.83:0/439017229 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55832f0127d0 con 0x55832f2d1460 2024-04-24T08:57:55.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.140+0000 7f9587e00640 1 --2- 172.21.15.83:0/439017229 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55832f2d1460 0x55832f2dcc40 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x55832f2d8100 tx=0x7f957801e470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ae84fc427fa0026 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:55.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.140+0000 7f9573e00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:57:55.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.140+0000 7f9573e00640 1 -- 172.21.15.83:0/439017229 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9578039060 con 0x55832f2d1460 2024-04-24T08:57:55.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.140+0000 7f9573e00640 1 -- 172.21.15.83:0/439017229 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f957801eb50 con 0x55832f2d1460 2024-04-24T08:57:55.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.140+0000 7f95983cf740 1 -- 172.21.15.83:0/439017229 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9580029030 con 0x55832f2d1460 2024-04-24T08:57:55.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.141+0000 7f9573e00640 1 -- 172.21.15.83:0/439017229 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f9578040050 con 0x55832f2d1460 2024-04-24T08:57:55.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.141+0000 7f95983cf740 10 client.17871 fetch_fsmap learned FSMap version 33 2024-04-24T08:57:55.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.141+0000 7f95983cf740 1 -- 172.21.15.83:0/439017229 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55832f2e44a0 con 0x55832f2d1460 2024-04-24T08:57:55.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.141+0000 7f9573e00640 1 -- 172.21.15.83:0/439017229 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9580029030 con 0x55832f2d1460 2024-04-24T08:57:55.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.141+0000 7f9573e00640 1 -- 172.21.15.83:0/439017229 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f957801b040 con 0x55832f2d1460 2024-04-24T08:57:55.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.141+0000 7f95983cf740 10 client.17871 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T08:57:55.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.141+0000 7f95983cf740 10 client.17871 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-24T08:57:55.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.141+0000 7f95983cf740 20 client.17871 populate_metadata read hostname 'smithi083' 2024-04-24T08:57:55.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.141+0000 7f95983cf740 1 --2- 172.21.15.83:0/439017229 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55832f35aed0 0x55832f35d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:55.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.141+0000 7f95983cf740 4 client.17871 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T08:57:55.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.141+0000 7f95983cf740 1 -- 172.21.15.83:0/439017229 --> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55832f23da60 con 0x55832f35aed0 2024-04-24T08:57:55.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.142+0000 7f958e800640 1 --2- 172.21.15.83:0/439017229 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55832f35aed0 0x55832f35d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:55.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.143+0000 7f958e800640 1 --2- 172.21.15.83:0/439017229 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55832f35aed0 0x55832f35d2f0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=53f9718c6b3714d9 server_cookie=8fddccc695e923fe in_seq=0 out_seq=0 2024-04-24T08:57:55.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.143+0000 7f9573e00640 10 client.17871 ms_handle_connect on v2:172.21.15.189:6838/2548965336 2024-04-24T08:57:55.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.143+0000 7f9573e00640 1 -- 172.21.15.83:0/439017229 <== mds.4 v2:172.21.15.189:6838/2548965336 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55832f23da60 con 0x55832f35aed0 2024-04-24T08:57:55.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.143+0000 7f9573e00640 10 client.17871 handle_command_reply: tid=0 2024-04-24T08:57:55.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.143+0000 7f95983cf740 1 client.17871 shutdown 2024-04-24T08:57:55.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.143+0000 7f95983cf740 1 -- 172.21.15.83:0/439017229 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55832f2d1460 msgr2=0x55832f2dcc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:55.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.143+0000 7f95983cf740 1 --2- 172.21.15.83:0/439017229 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55832f2d1460 0x55832f2dcc40 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x55832f2d8100 tx=0x7f957801e470 comp rx=0 tx=0).stop 2024-04-24T08:57:55.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.144+0000 7f95983cf740 1 -- 172.21.15.83:0/439017229 shutdown_connections 2024-04-24T08:57:55.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.144+0000 7f95983cf740 1 --2- 172.21.15.83:0/439017229 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55832f35aed0 0x55832f35d2f0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.144+0000 7f95983cf740 1 --2- 172.21.15.83:0/439017229 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55832f2dbb30 0x55832f2e2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.144+0000 7f95983cf740 1 --2- 172.21.15.83:0/439017229 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55832f2d1460 0x55832f2dcc40 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.144+0000 7f95983cf740 1 --2- 172.21.15.83:0/439017229 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55832f2d0b40 0x55832f2dc700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.144+0000 7f95983cf740 1 -- 172.21.15.83:0/439017229 >> 172.21.15.83:0/439017229 conn(0x55832f2cce60 msgr2=0x55832f2cb6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:55.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.144+0000 7f9573e00640 0 client.17871 ms_handle_reset on v2:172.21.15.189:6838/2548965336 2024-04-24T08:57:55.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.144+0000 7f95983cf740 1 -- 172.21.15.83:0/439017229 shutdown_connections 2024-04-24T08:57:55.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.144+0000 7f95983cf740 1 -- 172.21.15.83:0/439017229 wait complete. 2024-04-24T08:57:55.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.144+0000 7f95983cf740 20 client.17871 trim_cache size 0 max 16384 2024-04-24T08:57:55.150 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T08:57:55.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.144+0000 7f9594a00640 1 -- 172.21.15.83:0/1691250262 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f95740a3d20 msgr2=0x7f95740a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:55.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.144+0000 7f9594a00640 1 --2- 172.21.15.83:0/1691250262 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f95740a3d20 0x7f95740a61e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x55832f2cdff0 tx=0x7f9578014040 comp rx=0 tx=0).stop 2024-04-24T08:57:55.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.145+0000 7f9594a00640 1 -- 172.21.15.83:0/1691250262 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f959010e6e0 msgr2=0x7f959019a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:55.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.145+0000 7f9594a00640 1 --2- 172.21.15.83:0/1691250262 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f959010e6e0 0x7f959019a6a0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f9588009dc0 tx=0x7f9588003ce0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.145+0000 7f9594a00640 1 -- 172.21.15.83:0/1691250262 shutdown_connections 2024-04-24T08:57:55.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.145+0000 7f9594a00640 1 --2- 172.21.15.83:0/1691250262 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f95740a3d20 0x7f95740a61e0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.145+0000 7f9594a00640 1 --2- 172.21.15.83:0/1691250262 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f959010e6e0 0x7f959019a6a0 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.145+0000 7f9594a00640 1 --2- 172.21.15.83:0/1691250262 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9590074d50 0x7f95901a0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.145+0000 7f9594a00640 1 --2- 172.21.15.83:0/1691250262 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9590074430 0x7f95901a0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.145+0000 7f9594a00640 1 -- 172.21.15.83:0/1691250262 >> 172.21.15.83:0/1691250262 conn(0x7f95900fd9d0 msgr2=0x7f95900fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:55.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.145+0000 7f9594a00640 1 -- 172.21.15.83:0/1691250262 shutdown_connections 2024-04-24T08:57:55.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.145+0000 7f9594a00640 1 -- 172.21.15.83:0/1691250262 wait complete. 2024-04-24T08:57:55.164 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-24T08:57:55.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.370+0000 7f61f6c00640 1 -- 172.21.15.83:0/1122434572 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61f006acb0 msgr2=0x7f61f010ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:55.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.370+0000 7f61f6c00640 1 --2- 172.21.15.83:0/1122434572 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61f006acb0 0x7f61f010ee30 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f61e8009ca0 tx=0x7f61e802a8b0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.370+0000 7f61f6c00640 1 -- 172.21.15.83:0/1122434572 shutdown_connections 2024-04-24T08:57:55.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.370+0000 7f61f6c00640 1 --2- 172.21.15.83:0/1122434572 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61f006acb0 0x7f61f010ee30 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.370+0000 7f61f6c00640 1 --2- 172.21.15.83:0/1122434572 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61f006a310 0x7f61f006a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.370+0000 7f61f6c00640 1 --2- 172.21.15.83:0/1122434572 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f61f0106fc0 0x7f61f01073a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.370+0000 7f61f6c00640 1 -- 172.21.15.83:0/1122434572 >> 172.21.15.83:0/1122434572 conn(0x7f61f0076a80 msgr2=0x7f61f0076e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:55.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.370+0000 7f61f6c00640 1 -- 172.21.15.83:0/1122434572 shutdown_connections 2024-04-24T08:57:55.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.370+0000 7f61f6c00640 1 -- 172.21.15.83:0/1122434572 wait complete. 2024-04-24T08:57:55.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.371+0000 7f61f6c00640 1 Processor -- start 2024-04-24T08:57:55.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.371+0000 7f61f6c00640 1 -- start start 2024-04-24T08:57:55.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.372+0000 7f61f6c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61f006a310 0x7f61f01a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:55.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.372+0000 7f61f6c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61f006acb0 0x7f61f01a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:55.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.372+0000 7f61f6c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f61f0106fc0 0x7f61f019a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:55.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.372+0000 7f61f6c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f61f0112380 con 0x7f61f006acb0 2024-04-24T08:57:55.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.372+0000 7f61f6c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f61f0112200 con 0x7f61f006a310 2024-04-24T08:57:55.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.372+0000 7f61f6c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f61f0112500 con 0x7f61f0106fc0 2024-04-24T08:57:55.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.372+0000 7f61ef400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61f006acb0 0x7f61f01a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:55.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.372+0000 7f61ef400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61f006acb0 0x7f61f01a0bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41666/0 (socket says 172.21.15.83:41666) 2024-04-24T08:57:55.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.372+0000 7f61ef400640 1 -- 172.21.15.83:0/4088265825 learned_addr learned my addr 172.21.15.83:0/4088265825 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:57:55.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.372+0000 7f61efe00640 1 --2- 172.21.15.83:0/4088265825 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61f006a310 0x7f61f01a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:55.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.373+0000 7f61f4c00640 1 --2- 172.21.15.83:0/4088265825 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f61f0106fc0 0x7f61f019a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:55.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.373+0000 7f61ef400640 1 -- 172.21.15.83:0/4088265825 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f61f0106fc0 msgr2=0x7f61f019a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:55.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.373+0000 7f61ef400640 1 --2- 172.21.15.83:0/4088265825 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f61f0106fc0 0x7f61f019a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.373+0000 7f61ef400640 1 -- 172.21.15.83:0/4088265825 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61f006a310 msgr2=0x7f61f01a06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:55.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.373+0000 7f61ef400640 1 --2- 172.21.15.83:0/4088265825 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61f006a310 0x7f61f01a06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.373+0000 7f61ef400640 1 -- 172.21.15.83:0/4088265825 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61f019afe0 con 0x7f61f006acb0 2024-04-24T08:57:55.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.373+0000 7f61ef400640 1 --2- 172.21.15.83:0/4088265825 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61f006acb0 0x7f61f01a0bf0 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f61dc00d950 tx=0x7f61dc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:55.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.373+0000 7f61ecc00640 1 -- 172.21.15.83:0/4088265825 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61dc017070 con 0x7f61f006acb0 2024-04-24T08:57:55.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.374+0000 7f61f6c00640 1 -- 172.21.15.83:0/4088265825 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61f019b2d0 con 0x7f61f006acb0 2024-04-24T08:57:55.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.374+0000 7f61ecc00640 1 -- 172.21.15.83:0/4088265825 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f61dc009c30 con 0x7f61f006acb0 2024-04-24T08:57:55.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.374+0000 7f61ecc00640 1 -- 172.21.15.83:0/4088265825 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61dc00f620 con 0x7f61f006acb0 2024-04-24T08:57:55.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.374+0000 7f61f6c00640 1 -- 172.21.15.83:0/4088265825 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61f01a7570 con 0x7f61f006acb0 2024-04-24T08:57:55.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.377+0000 7f61f9a0b740 1 Processor -- start 2024-04-24T08:57:55.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.377+0000 7f61f9a0b740 1 -- start start 2024-04-24T08:57:55.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.378+0000 7f61f9a0b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55694ad90950 0x55694ad90d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:55.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.378+0000 7f61f9a0b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55694ad91270 0x55694ad9b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:55.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.378+0000 7f61f9a0b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55694ad9b940 0x55694ad9dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:55.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.378+0000 7f61f9a0b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55694acf61d0 con 0x55694ad9b940 2024-04-24T08:57:55.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.378+0000 7f61f9a0b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55694ac9c450 con 0x55694ad90950 2024-04-24T08:57:55.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.378+0000 7f61f9a0b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55694acf2190 con 0x55694ad91270 2024-04-24T08:57:55.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.378+0000 7f61ecc00640 1 -- 172.21.15.83:0/4088265825 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f61dc0109a0 con 0x7f61f006acb0 2024-04-24T08:57:55.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.378+0000 7f61efe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55694ad91270 0x55694ad9b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:55.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.378+0000 7f61f4c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55694ad90950 0x55694ad90d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:55.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.378+0000 7f61efe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55694ad91270 0x55694ad9b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:37120/0 (socket says 172.21.15.83:37120) 2024-04-24T08:57:55.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.378+0000 7f61efe00640 1 -- 172.21.15.83:0/166956226 learned_addr learned my addr 172.21.15.83:0/166956226 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:57:55.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.378+0000 7f61f4c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55694ad90950 0x55694ad90d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:39650/0 (socket says 172.21.15.83:39650) 2024-04-24T08:57:55.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.379+0000 7f61ef400640 1 --2- 172.21.15.83:0/166956226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55694ad9b940 0x55694ad9dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:55.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.379+0000 7f61f4c00640 1 -- 172.21.15.83:0/166956226 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55694ad91270 msgr2=0x55694ad9b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:55.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.379+0000 7f61f4c00640 1 --2- 172.21.15.83:0/166956226 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55694ad91270 0x55694ad9b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.379+0000 7f61f4c00640 1 -- 172.21.15.83:0/166956226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55694ad9b940 msgr2=0x55694ad9dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:55.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.379+0000 7f61f4c00640 1 --2- 172.21.15.83:0/166956226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55694ad9b940 0x55694ad9dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.379+0000 7f61f4c00640 1 -- 172.21.15.83:0/166956226 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55694ad59310 con 0x55694ad90950 2024-04-24T08:57:55.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.379+0000 7f61ef400640 1 --2- 172.21.15.83:0/166956226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55694ad9b940 0x55694ad9dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:57:55.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.380+0000 7f61f4c00640 1 --2- 172.21.15.83:0/166956226 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55694ad90950 0x55694ad90d30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f61e802add0 tx=0x7f61e8030fa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f057f58a7c16a234 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:55.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.380+0000 7f61ecc00640 1 --2- 172.21.15.83:0/4088265825 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f61d40a3d20 0x7f61d40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:55.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.380+0000 7f61e5200640 1 -- 172.21.15.83:0/166956226 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e80307f0 con 0x55694ad90950 2024-04-24T08:57:55.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.380+0000 7f61e5200640 1 -- 172.21.15.83:0/166956226 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f61e8036040 con 0x55694ad90950 2024-04-24T08:57:55.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.380+0000 7f61ecc00640 1 -- 172.21.15.83:0/4088265825 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f61dc0d07d0 con 0x7f61f006acb0 2024-04-24T08:57:55.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.380+0000 7f61e5200640 1 -- 172.21.15.83:0/166956226 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e8004460 con 0x55694ad90950 2024-04-24T08:57:55.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.380+0000 7f61efe00640 1 --2- 172.21.15.83:0/4088265825 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f61d40a3d20 0x7f61d40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:55.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.381+0000 7f61efe00640 1 --2- 172.21.15.83:0/4088265825 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f61d40a3d20 0x7f61d40a61e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x55694ad91f10 tx=0x7f61e000b390 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:55.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.381+0000 7f61f9a0b740 1 -- 172.21.15.83:0/166956226 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55694ad90950 msgr2=0x55694ad90d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:55.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.381+0000 7f61f9a0b740 1 --2- 172.21.15.83:0/166956226 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55694ad90950 0x55694ad90d30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f61e802add0 tx=0x7f61e8030fa0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.382+0000 7f61f9a0b740 1 -- 172.21.15.83:0/166956226 shutdown_connections 2024-04-24T08:57:55.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.382+0000 7f61f9a0b740 1 --2- 172.21.15.83:0/166956226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55694ad9b940 0x55694ad9dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.382+0000 7f61f9a0b740 1 --2- 172.21.15.83:0/166956226 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55694ad91270 0x55694ad9b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.382+0000 7f61f9a0b740 1 --2- 172.21.15.83:0/166956226 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55694ad90950 0x55694ad90d30 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.382+0000 7f61f9a0b740 1 -- 172.21.15.83:0/166956226 >> 172.21.15.83:0/166956226 conn(0x55694ad8cc70 msgr2=0x55694ad8d050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:55.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.382+0000 7f61f9a0b740 1 -- 172.21.15.83:0/166956226 shutdown_connections 2024-04-24T08:57:55.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.382+0000 7f61f9a0b740 1 -- 172.21.15.83:0/166956226 wait complete. 2024-04-24T08:57:55.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.382+0000 7f61f9a0b740 1 -- start start 2024-04-24T08:57:55.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.383+0000 7f61f9a0b740 1 Processor -- start 2024-04-24T08:57:55.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.383+0000 7f61f9a0b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55694ad90950 0x55694ad98700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:55.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.383+0000 7f61f9a0b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55694ad91270 0x55694ad98c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:55.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.384+0000 7f61f9a0b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55694ad9b940 0x55694ad99180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:55.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.384+0000 7f61f9a0b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55694ad9e770 con 0x55694ad90950 2024-04-24T08:57:55.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.384+0000 7f61f9a0b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55694acaf810 con 0x55694ad91270 2024-04-24T08:57:55.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.384+0000 7f61f9a0b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55694ad9e8f0 con 0x55694ad9b940 2024-04-24T08:57:55.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.384+0000 7f61f9a0b740 10 client.0 _ll_register_callbacks cb 0x55694ac7d140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:57:55.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.384+0000 7f61efe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55694ad90950 0x55694ad98700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:55.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.384+0000 7f61efe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55694ad90950 0x55694ad98700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41678/0 (socket says 172.21.15.83:41678) 2024-04-24T08:57:55.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.384+0000 7f61efe00640 1 -- 172.21.15.83:0/1347628122 learned_addr learned my addr 172.21.15.83:0/1347628122 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:57:55.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.384+0000 7f61ef400640 1 --2- 172.21.15.83:0/1347628122 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55694ad91270 0x55694ad98c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:55.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.384+0000 7f61f4c00640 1 --2- 172.21.15.83:0/1347628122 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55694ad9b940 0x55694ad99180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:55.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.384+0000 7f61efe00640 1 -- 172.21.15.83:0/1347628122 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55694ad9b940 msgr2=0x55694ad99180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:55.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.384+0000 7f61efe00640 1 --2- 172.21.15.83:0/1347628122 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55694ad9b940 0x55694ad99180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.384+0000 7f61efe00640 1 -- 172.21.15.83:0/1347628122 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55694ad91270 msgr2=0x55694ad98c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:55.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.384+0000 7f61efe00640 1 --2- 172.21.15.83:0/1347628122 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55694ad91270 0x55694ad98c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.384+0000 7f61efe00640 1 -- 172.21.15.83:0/1347628122 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55694acc03f0 con 0x55694ad90950 2024-04-24T08:57:55.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.385+0000 7f61efe00640 1 --2- 172.21.15.83:0/1347628122 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55694ad90950 0x55694ad98700 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f61e0003e00 tx=0x7f61e0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7728cfec543465b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:55.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.385+0000 7f61d9600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:57:55.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.385+0000 7f61d9600640 1 -- 172.21.15.83:0/1347628122 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e002c060 con 0x55694ad90950 2024-04-24T08:57:55.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.385+0000 7f61d9600640 1 -- 172.21.15.83:0/1347628122 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f61e0017030 con 0x55694ad90950 2024-04-24T08:57:55.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.385+0000 7f61f9a0b740 1 -- 172.21.15.83:0/1347628122 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61e8036040 con 0x55694ad90950 2024-04-24T08:57:55.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.386+0000 7f61d9600640 1 -- 172.21.15.83:0/1347628122 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f61e0033050 con 0x55694ad90950 2024-04-24T08:57:55.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.386+0000 7f61f9a0b740 10 client.17880 fetch_fsmap learned FSMap version 33 2024-04-24T08:57:55.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.386+0000 7f61f9a0b740 1 -- 172.21.15.83:0/1347628122 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55694aa083f0 con 0x55694ad90950 2024-04-24T08:57:55.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.386+0000 7f61d9600640 1 -- 172.21.15.83:0/1347628122 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f61e8036040 con 0x55694ad90950 2024-04-24T08:57:55.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.386+0000 7f61d9600640 1 -- 172.21.15.83:0/1347628122 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f61e001c3f0 con 0x55694ad90950 2024-04-24T08:57:55.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.386+0000 7f61f9a0b740 10 client.17880 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T08:57:55.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.386+0000 7f61f9a0b740 10 client.17880 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-24T08:57:55.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.386+0000 7f61f9a0b740 20 client.17880 populate_metadata read hostname 'smithi083' 2024-04-24T08:57:55.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.386+0000 7f61f9a0b740 1 --2- 172.21.15.83:0/1347628122 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55694adabb00 0x55694adadef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:55.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.386+0000 7f61f9a0b740 4 client.17880 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:57:55.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.386+0000 7f61f9a0b740 1 -- 172.21.15.83:0/1347628122 --> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55694ad03bf0 con 0x55694adabb00 2024-04-24T08:57:55.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.387+0000 7f61ef400640 1 --2- 172.21.15.83:0/1347628122 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55694adabb00 0x55694adadef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:55.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.388+0000 7f61d9600640 10 client.17880 ms_handle_connect on v2:172.21.15.138:6832/1157967658 2024-04-24T08:57:55.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.388+0000 7f61ef400640 1 --2- 172.21.15.83:0/1347628122 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55694adabb00 0x55694adadef0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c64c64ef69bf17a0 server_cookie=3f3373fd193b03e8 in_seq=0 out_seq=0 2024-04-24T08:57:55.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.390+0000 7f61d9600640 1 -- 172.21.15.83:0/1347628122 <== mds.3 v2:172.21.15.138:6832/1157967658 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55694ad03bf0 con 0x55694adabb00 2024-04-24T08:57:55.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.390+0000 7f61d9600640 10 client.17880 handle_command_reply: tid=0 2024-04-24T08:57:55.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.390+0000 7f61f9a0b740 1 client.17880 shutdown 2024-04-24T08:57:55.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.390+0000 7f61f9a0b740 1 -- 172.21.15.83:0/1347628122 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55694ad90950 msgr2=0x55694ad98700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:55.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.390+0000 7f61f9a0b740 1 --2- 172.21.15.83:0/1347628122 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55694ad90950 0x55694ad98700 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f61e0003e00 tx=0x7f61e0004730 comp rx=0 tx=0).stop 2024-04-24T08:57:55.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.390+0000 7f61f9a0b740 1 -- 172.21.15.83:0/1347628122 shutdown_connections 2024-04-24T08:57:55.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.390+0000 7f61f9a0b740 1 --2- 172.21.15.83:0/1347628122 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55694adabb00 0x55694adadef0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.390+0000 7f61f9a0b740 1 --2- 172.21.15.83:0/1347628122 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55694ad9b940 0x55694ad99180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.390+0000 7f61d9600640 0 client.17880 ms_handle_reset on v2:172.21.15.138:6832/1157967658 2024-04-24T08:57:55.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.390+0000 7f61f9a0b740 1 --2- 172.21.15.83:0/1347628122 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55694ad91270 0x55694ad98c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.390+0000 7f61f9a0b740 1 --2- 172.21.15.83:0/1347628122 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55694ad90950 0x55694ad98700 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.390+0000 7f61f9a0b740 1 -- 172.21.15.83:0/1347628122 >> 172.21.15.83:0/1347628122 conn(0x55694ad8cc70 msgr2=0x55694ad92a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:55.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.390+0000 7f61f9a0b740 1 -- 172.21.15.83:0/1347628122 shutdown_connections 2024-04-24T08:57:55.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.390+0000 7f61f9a0b740 1 -- 172.21.15.83:0/1347628122 wait complete. 2024-04-24T08:57:55.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.391+0000 7f61f9a0b740 20 client.17880 trim_cache size 0 max 16384 2024-04-24T08:57:55.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.406+0000 7f61f9a0b740 1 Processor -- start 2024-04-24T08:57:55.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.406+0000 7f61f9a0b740 1 -- start start 2024-04-24T08:57:55.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.407+0000 7f61f9a0b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55694ad90950 0x55694ada7630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:55.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.407+0000 7f61f9a0b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55694ad91270 0x55694ad8c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:55.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.407+0000 7f61f9a0b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55694ad9b940 0x55694ad98020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:55.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.407+0000 7f61f9a0b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f61f01b1700 con 0x55694ad91270 2024-04-24T08:57:55.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.407+0000 7f61f9a0b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f61f01b3240 con 0x55694ad90950 2024-04-24T08:57:55.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.407+0000 7f61f9a0b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f61f01b1dd0 con 0x55694ad9b940 2024-04-24T08:57:55.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.407+0000 7f61ef400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55694ad91270 0x55694ad8c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:55.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.407+0000 7f61ef400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55694ad91270 0x55694ad8c030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41692/0 (socket says 172.21.15.83:41692) 2024-04-24T08:57:55.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.407+0000 7f61ef400640 1 -- 172.21.15.83:0/4270953527 learned_addr learned my addr 172.21.15.83:0/4270953527 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:57:55.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.407+0000 7f61efe00640 1 --2- 172.21.15.83:0/4270953527 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55694ad90950 0x55694ada7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:55.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.408+0000 7f61f4c00640 1 --2- 172.21.15.83:0/4270953527 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55694ad9b940 0x55694ad98020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:55.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.408+0000 7f61ef400640 1 -- 172.21.15.83:0/4270953527 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55694ad9b940 msgr2=0x55694ad98020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:55.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.408+0000 7f61ef400640 1 --2- 172.21.15.83:0/4270953527 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55694ad9b940 0x55694ad98020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.408+0000 7f61ef400640 1 -- 172.21.15.83:0/4270953527 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55694ad90950 msgr2=0x55694ada7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:55.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.408+0000 7f61ef400640 1 --2- 172.21.15.83:0/4270953527 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55694ad90950 0x55694ada7630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.408+0000 7f61ef400640 1 -- 172.21.15.83:0/4270953527 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55694aad27d0 con 0x55694ad91270 2024-04-24T08:57:55.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.408+0000 7f61ef400640 1 --2- 172.21.15.83:0/4270953527 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55694ad91270 0x55694ad8c030 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f61dc0cdbb0 tx=0x7f61dc098040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7ad6b0cb5ec16e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:55.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.408+0000 7f61dbe00640 1 -- 172.21.15.83:0/4270953527 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61dc020040 con 0x55694ad91270 2024-04-24T08:57:55.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.408+0000 7f61dbe00640 1 -- 172.21.15.83:0/4270953527 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f61dc0a0750 con 0x55694ad91270 2024-04-24T08:57:55.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.408+0000 7f61dbe00640 1 -- 172.21.15.83:0/4270953527 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61dc0d03b0 con 0x55694ad91270 2024-04-24T08:57:55.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.409+0000 7f61f9a0b740 1 -- 172.21.15.83:0/4270953527 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55694ad91270 msgr2=0x55694ad8c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:55.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.409+0000 7f61f9a0b740 1 --2- 172.21.15.83:0/4270953527 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55694ad91270 0x55694ad8c030 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f61dc0cdbb0 tx=0x7f61dc098040 comp rx=0 tx=0).stop 2024-04-24T08:57:55.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.409+0000 7f61f9a0b740 1 -- 172.21.15.83:0/4270953527 shutdown_connections 2024-04-24T08:57:55.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.409+0000 7f61f9a0b740 1 --2- 172.21.15.83:0/4270953527 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55694ad9b940 0x55694ad98020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.409+0000 7f61f9a0b740 1 --2- 172.21.15.83:0/4270953527 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55694ad91270 0x55694ad8c030 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.409+0000 7f61f9a0b740 1 --2- 172.21.15.83:0/4270953527 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55694ad90950 0x55694ada7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.409+0000 7f61f9a0b740 1 -- 172.21.15.83:0/4270953527 >> 172.21.15.83:0/4270953527 conn(0x55694ad8cc70 msgr2=0x55694ad98f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:55.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.409+0000 7f61f9a0b740 1 -- 172.21.15.83:0/4270953527 shutdown_connections 2024-04-24T08:57:55.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.409+0000 7f61f9a0b740 1 -- 172.21.15.83:0/4270953527 wait complete. 2024-04-24T08:57:55.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.410+0000 7f61f9a0b740 1 -- start start 2024-04-24T08:57:55.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.410+0000 7f61f9a0b740 1 Processor -- start 2024-04-24T08:57:55.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.411+0000 7f61f9a0b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55694ad90950 0x55694ad94fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:55.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.411+0000 7f61f9a0b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55694ad91270 0x55694ad954f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:55.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.411+0000 7f61f9a0b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55694ad9b940 0x55694ada2b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:55.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.411+0000 7f61f9a0b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f61f01b2b70 con 0x55694ad9b940 2024-04-24T08:57:55.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.411+0000 7f61f9a0b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f61f01b24a0 con 0x55694ad91270 2024-04-24T08:57:55.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.411+0000 7f61f9a0b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f61f01af6e0 con 0x55694ad90950 2024-04-24T08:57:55.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.411+0000 7f61f9a0b740 10 client.0 _ll_register_callbacks cb 0x7f61e0003c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:57:55.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.411+0000 7f61f4c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55694ad9b940 0x55694ada2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:55.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.411+0000 7f61f4c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55694ad9b940 0x55694ada2b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41702/0 (socket says 172.21.15.83:41702) 2024-04-24T08:57:55.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.411+0000 7f61f4c00640 1 -- 172.21.15.83:0/1678440316 learned_addr learned my addr 172.21.15.83:0/1678440316 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:57:55.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.411+0000 7f61efe00640 1 --2- 172.21.15.83:0/1678440316 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55694ad90950 0x55694ad94fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:55.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.411+0000 7f61ef400640 1 --2- 172.21.15.83:0/1678440316 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55694ad91270 0x55694ad954f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:55.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.412+0000 7f61f4c00640 1 -- 172.21.15.83:0/1678440316 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55694ad90950 msgr2=0x55694ad94fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:55.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.412+0000 7f61f4c00640 1 --2- 172.21.15.83:0/1678440316 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55694ad90950 0x55694ad94fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.412+0000 7f61f4c00640 1 -- 172.21.15.83:0/1678440316 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55694ad91270 msgr2=0x55694ad954f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:55.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.412+0000 7f61f4c00640 1 --2- 172.21.15.83:0/1678440316 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55694ad91270 0x55694ad954f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.412+0000 7f61f4c00640 1 -- 172.21.15.83:0/1678440316 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55694ada3080 con 0x55694ad9b940 2024-04-24T08:57:55.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.412+0000 7f61f4c00640 1 --2- 172.21.15.83:0/1678440316 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55694ad9b940 0x55694ada2b40 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f61e8000f80 tx=0x7f61e80095d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1d4951d8fc4eccf server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:55.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.412+0000 7f61d9600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:57:55.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.413+0000 7f61d9600640 1 -- 172.21.15.83:0/1678440316 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e804a040 con 0x55694ad9b940 2024-04-24T08:57:55.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.413+0000 7f61d9600640 1 -- 172.21.15.83:0/1678440316 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f61e80309f0 con 0x55694ad9b940 2024-04-24T08:57:55.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.413+0000 7f61f9a0b740 1 -- 172.21.15.83:0/1678440316 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61dc0a0750 con 0x55694ad9b940 2024-04-24T08:57:55.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.413+0000 7f61d9600640 1 -- 172.21.15.83:0/1678440316 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f61e805b050 con 0x55694ad9b940 2024-04-24T08:57:55.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.413+0000 7f61f9a0b740 10 client.17886 fetch_fsmap learned FSMap version 33 2024-04-24T08:57:55.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.413+0000 7f61f9a0b740 1 -- 172.21.15.83:0/1678440316 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55694ada4380 con 0x55694ad9b940 2024-04-24T08:57:55.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.414+0000 7f61d9600640 1 -- 172.21.15.83:0/1678440316 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f61dc0a0750 con 0x55694ad9b940 2024-04-24T08:57:55.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.414+0000 7f61d9600640 1 -- 172.21.15.83:0/1678440316 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f61e8056030 con 0x55694ad9b940 2024-04-24T08:57:55.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.414+0000 7f61f9a0b740 10 client.17886 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T08:57:55.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.414+0000 7f61f9a0b740 10 client.17886 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-24T08:57:55.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.414+0000 7f61f9a0b740 20 client.17886 populate_metadata read hostname 'smithi083' 2024-04-24T08:57:55.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.414+0000 7f61f9a0b740 1 --2- 172.21.15.83:0/1678440316 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55694ae120b0 0x55694ae144a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:55.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.414+0000 7f61f9a0b740 4 client.17886 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T08:57:55.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.414+0000 7f61f9a0b740 1 -- 172.21.15.83:0/1678440316 --> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55694acfda60 con 0x55694ae120b0 2024-04-24T08:57:55.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.415+0000 7f61efe00640 1 --2- 172.21.15.83:0/1678440316 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55694ae120b0 0x55694ae144a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:55.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.417+0000 7f61efe00640 1 --2- 172.21.15.83:0/1678440316 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55694ae120b0 0x55694ae144a0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d12ac22257b72c84 server_cookie=6f3b189e64aff3a8 in_seq=0 out_seq=0 2024-04-24T08:57:55.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.417+0000 7f61d9600640 10 client.17886 ms_handle_connect on v2:172.21.15.138:6832/1157967658 2024-04-24T08:57:55.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.417+0000 7f61d9600640 1 -- 172.21.15.83:0/1678440316 <== mds.3 v2:172.21.15.138:6832/1157967658 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55694acfda60 con 0x55694ae120b0 2024-04-24T08:57:55.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.417+0000 7f61d9600640 10 client.17886 handle_command_reply: tid=0 2024-04-24T08:57:55.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.418+0000 7f61f9a0b740 1 client.17886 shutdown 2024-04-24T08:57:55.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.418+0000 7f61f9a0b740 1 -- 172.21.15.83:0/1678440316 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55694ad9b940 msgr2=0x55694ada2b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:55.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.418+0000 7f61f9a0b740 1 --2- 172.21.15.83:0/1678440316 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55694ad9b940 0x55694ada2b40 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f61e8000f80 tx=0x7f61e80095d0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.418+0000 7f61f9a0b740 1 -- 172.21.15.83:0/1678440316 shutdown_connections 2024-04-24T08:57:55.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.418+0000 7f61f9a0b740 1 --2- 172.21.15.83:0/1678440316 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55694ad9b940 0x55694ada2b40 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.418+0000 7f61f9a0b740 1 --2- 172.21.15.83:0/1678440316 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55694ad91270 0x55694ad954f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.418+0000 7f61f9a0b740 1 --2- 172.21.15.83:0/1678440316 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55694ae120b0 0x55694ae144a0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.418+0000 7f61f9a0b740 1 --2- 172.21.15.83:0/1678440316 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55694ad90950 0x55694ad94fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.418+0000 7f61f9a0b740 1 -- 172.21.15.83:0/1678440316 >> 172.21.15.83:0/1678440316 conn(0x55694ad8cc70 msgr2=0x55694ad92180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:55.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.418+0000 7f61d9600640 0 client.17886 ms_handle_reset on v2:172.21.15.138:6832/1157967658 2024-04-24T08:57:55.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.418+0000 7f61f9a0b740 1 -- 172.21.15.83:0/1678440316 shutdown_connections 2024-04-24T08:57:55.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.418+0000 7f61f9a0b740 1 -- 172.21.15.83:0/1678440316 wait complete. 2024-04-24T08:57:55.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.418+0000 7f61f9a0b740 20 client.17886 trim_cache size 0 max 16384 2024-04-24T08:57:55.420 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T08:57:55.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.419+0000 7f61f6c00640 1 -- 172.21.15.83:0/4088265825 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f61d40a3d20 msgr2=0x7f61d40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:55.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.419+0000 7f61f6c00640 1 --2- 172.21.15.83:0/4088265825 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f61d40a3d20 0x7f61d40a61e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x55694ad91f10 tx=0x7f61e000b390 comp rx=0 tx=0).stop 2024-04-24T08:57:55.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.419+0000 7f61f6c00640 1 -- 172.21.15.83:0/4088265825 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61f006acb0 msgr2=0x7f61f01a0bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:55.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.419+0000 7f61f6c00640 1 --2- 172.21.15.83:0/4088265825 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61f006acb0 0x7f61f01a0bf0 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f61dc00d950 tx=0x7f61dc00de10 comp rx=0 tx=0).stop 2024-04-24T08:57:55.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.419+0000 7f61f6c00640 1 -- 172.21.15.83:0/4088265825 shutdown_connections 2024-04-24T08:57:55.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.419+0000 7f61f6c00640 1 --2- 172.21.15.83:0/4088265825 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f61d40a3d20 0x7f61d40a61e0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.419+0000 7f61f6c00640 1 --2- 172.21.15.83:0/4088265825 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f61f0106fc0 0x7f61f019a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.419+0000 7f61f6c00640 1 --2- 172.21.15.83:0/4088265825 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61f006acb0 0x7f61f01a0bf0 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.419+0000 7f61f6c00640 1 --2- 172.21.15.83:0/4088265825 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61f006a310 0x7f61f01a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.419+0000 7f61f6c00640 1 -- 172.21.15.83:0/4088265825 >> 172.21.15.83:0/4088265825 conn(0x7f61f0076a80 msgr2=0x7f61f00ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:55.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.419+0000 7f61f6c00640 1 -- 172.21.15.83:0/4088265825 shutdown_connections 2024-04-24T08:57:55.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.419+0000 7f61f6c00640 1 -- 172.21.15.83:0/4088265825 wait complete. 2024-04-24T08:57:55.438 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T08:57:55.583 INFO:tasks.workunit.client.0.smithi083.stdout: 1 32440 0.64 MB/sec execute 149 sec latency 76.535 ms 2024-04-24T08:57:55.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.642+0000 7fb277200640 1 -- 172.21.15.83:0/2975606016 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb278102110 msgr2=0x7fb2781024f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:55.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.642+0000 7fb277200640 1 --2- 172.21.15.83:0/2975606016 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb278102110 0x7fb2781024f0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7fb268009900 tx=0x7fb26802a410 comp rx=0 tx=0).stop 2024-04-24T08:57:55.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.642+0000 7fb277200640 1 -- 172.21.15.83:0/2975606016 shutdown_connections 2024-04-24T08:57:55.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.642+0000 7fb277200640 1 --2- 172.21.15.83:0/2975606016 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2781071e0 0x7fb27810edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.642+0000 7fb277200640 1 --2- 172.21.15.83:0/2975606016 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb278102ac0 0x7fb278106b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.642+0000 7fb277200640 1 --2- 172.21.15.83:0/2975606016 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb278102110 0x7fb2781024f0 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.642+0000 7fb277200640 1 -- 172.21.15.83:0/2975606016 >> 172.21.15.83:0/2975606016 conn(0x7fb2780fd9d0 msgr2=0x7fb2780ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:55.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.642+0000 7fb277200640 1 -- 172.21.15.83:0/2975606016 shutdown_connections 2024-04-24T08:57:55.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.642+0000 7fb277200640 1 -- 172.21.15.83:0/2975606016 wait complete. 2024-04-24T08:57:55.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.643+0000 7fb277200640 1 Processor -- start 2024-04-24T08:57:55.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.643+0000 7fb277200640 1 -- start start 2024-04-24T08:57:55.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.643+0000 7fb277200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb278102110 0x7fb278074300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:55.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.643+0000 7fb277200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb2781071e0 0x7fb278074840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:55.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.644+0000 7fb277200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb278074d80 0x7fb2781a6900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:55.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.644+0000 7fb277200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb278112140 con 0x7fb278102110 2024-04-24T08:57:55.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.644+0000 7fb277200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb278111fc0 con 0x7fb278074d80 2024-04-24T08:57:55.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.644+0000 7fb277200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb2781122c0 con 0x7fb2781071e0 2024-04-24T08:57:55.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.644+0000 7fb275e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb278102110 0x7fb278074300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:55.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.644+0000 7fb275e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb278102110 0x7fb278074300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41728/0 (socket says 172.21.15.83:41728) 2024-04-24T08:57:55.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.644+0000 7fb275e00640 1 -- 172.21.15.83:0/286690481 learned_addr learned my addr 172.21.15.83:0/286690481 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:57:55.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.644+0000 7fb276800640 1 --2- 172.21.15.83:0/286690481 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb278074d80 0x7fb2781a6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:55.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.644+0000 7fb275400640 1 --2- 172.21.15.83:0/286690481 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb2781071e0 0x7fb278074840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:55.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.644+0000 7fb275e00640 1 -- 172.21.15.83:0/286690481 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb2781071e0 msgr2=0x7fb278074840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:55.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.644+0000 7fb275e00640 1 --2- 172.21.15.83:0/286690481 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb2781071e0 0x7fb278074840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.644+0000 7fb275e00640 1 -- 172.21.15.83:0/286690481 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb278074d80 msgr2=0x7fb2781a6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:55.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.644+0000 7fb275e00640 1 --2- 172.21.15.83:0/286690481 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb278074d80 0x7fb2781a6900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.644+0000 7fb275e00640 1 -- 172.21.15.83:0/286690481 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2781a7000 con 0x7fb278102110 2024-04-24T08:57:55.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.645+0000 7fb275400640 1 --2- 172.21.15.83:0/286690481 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb2781071e0 0x7fb278074840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:57:55.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.645+0000 7fb275e00640 1 --2- 172.21.15.83:0/286690481 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb278102110 0x7fb278074300 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7fb268005940 tx=0x7fb268004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:55.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.645+0000 7fb26ea00640 1 -- 172.21.15.83:0/286690481 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb26803f030 con 0x7fb278102110 2024-04-24T08:57:55.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.645+0000 7fb26ea00640 1 -- 172.21.15.83:0/286690481 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb268003aa0 con 0x7fb278102110 2024-04-24T08:57:55.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.645+0000 7fb277200640 1 -- 172.21.15.83:0/286690481 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2781a7290 con 0x7fb278102110 2024-04-24T08:57:55.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.645+0000 7fb26ea00640 1 -- 172.21.15.83:0/286690481 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb268035070 con 0x7fb278102110 2024-04-24T08:57:55.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.645+0000 7fb277200640 1 -- 172.21.15.83:0/286690481 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2781a77d0 con 0x7fb278102110 2024-04-24T08:57:55.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.650+0000 7fb280045740 1 Processor -- start 2024-04-24T08:57:55.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.650+0000 7fb280045740 1 -- start start 2024-04-24T08:57:55.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.650+0000 7fb280045740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a050b97b40 0x55a050b97f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:55.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.650+0000 7fb280045740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a050b98460 0x55a050ba25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:55.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.650+0000 7fb280045740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a050ba2b30 0x55a050ba4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:55.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.650+0000 7fb280045740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a050afd1d0 con 0x55a050ba2b30 2024-04-24T08:57:55.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.650+0000 7fb280045740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a050aa3450 con 0x55a050b97b40 2024-04-24T08:57:55.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.650+0000 7fb280045740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a050af9190 con 0x55a050b98460 2024-04-24T08:57:55.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.651+0000 7fb26ea00640 1 -- 172.21.15.83:0/286690481 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb268042030 con 0x7fb278102110 2024-04-24T08:57:55.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.651+0000 7fb276800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a050b97b40 0x55a050b97f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:55.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.651+0000 7fb276800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a050b97b40 0x55a050b97f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:39706/0 (socket says 172.21.15.83:39706) 2024-04-24T08:57:55.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.651+0000 7fb276800640 1 -- 172.21.15.83:0/2045680204 learned_addr learned my addr 172.21.15.83:0/2045680204 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:57:55.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.651+0000 7fb275400640 1 --2- 172.21.15.83:0/2045680204 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a050ba2b30 0x55a050ba4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:55.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.651+0000 7fb275e00640 1 --2- 172.21.15.83:0/2045680204 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a050b98460 0x55a050ba25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:55.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.651+0000 7fb276800640 1 -- 172.21.15.83:0/2045680204 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a050b98460 msgr2=0x55a050ba25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:55.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.651+0000 7fb276800640 1 --2- 172.21.15.83:0/2045680204 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a050b98460 0x55a050ba25f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.651+0000 7fb276800640 1 -- 172.21.15.83:0/2045680204 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a050ba2b30 msgr2=0x55a050ba4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:55.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.651+0000 7fb276800640 1 --2- 172.21.15.83:0/2045680204 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a050ba2b30 0x55a050ba4f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.651+0000 7fb276800640 1 -- 172.21.15.83:0/2045680204 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a050b60310 con 0x55a050b97b40 2024-04-24T08:57:55.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.651+0000 7fb275e00640 1 --2- 172.21.15.83:0/2045680204 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a050b98460 0x55a050ba25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:57:55.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.652+0000 7fb276800640 1 --2- 172.21.15.83:0/2045680204 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a050b97b40 0x55a050b97f20 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb270009f70 tx=0x7fb270010800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76138bd622366da server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:55.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.652+0000 7fb263400640 1 -- 172.21.15.83:0/2045680204 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb270016070 con 0x55a050b97b40 2024-04-24T08:57:55.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.652+0000 7fb263400640 1 -- 172.21.15.83:0/2045680204 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb270004d20 con 0x55a050b97b40 2024-04-24T08:57:55.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.652+0000 7fb263400640 1 -- 172.21.15.83:0/2045680204 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb270012790 con 0x55a050b97b40 2024-04-24T08:57:55.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.652+0000 7fb26ea00640 1 --2- 172.21.15.83:0/286690481 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb25c0a3dd0 0x7fb25c0a6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:55.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.653+0000 7fb26ea00640 1 -- 172.21.15.83:0/286690481 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7fb2680f12d0 con 0x7fb278102110 2024-04-24T08:57:55.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.653+0000 7fb275e00640 1 --2- 172.21.15.83:0/286690481 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb25c0a3dd0 0x7fb25c0a6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:55.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.653+0000 7fb280045740 1 -- 172.21.15.83:0/2045680204 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a050b97b40 msgr2=0x55a050b97f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:55.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.653+0000 7fb280045740 1 --2- 172.21.15.83:0/2045680204 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a050b97b40 0x55a050b97f20 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb270009f70 tx=0x7fb270010800 comp rx=0 tx=0).stop 2024-04-24T08:57:55.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.653+0000 7fb280045740 1 -- 172.21.15.83:0/2045680204 shutdown_connections 2024-04-24T08:57:55.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.653+0000 7fb280045740 1 --2- 172.21.15.83:0/2045680204 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a050ba2b30 0x55a050ba4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.653+0000 7fb280045740 1 --2- 172.21.15.83:0/2045680204 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a050b98460 0x55a050ba25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.653+0000 7fb280045740 1 --2- 172.21.15.83:0/2045680204 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a050b97b40 0x55a050b97f20 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.653+0000 7fb280045740 1 -- 172.21.15.83:0/2045680204 >> 172.21.15.83:0/2045680204 conn(0x55a050b93e60 msgr2=0x55a050b94240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:55.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.654+0000 7fb275e00640 1 --2- 172.21.15.83:0/286690481 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb25c0a3dd0 0x7fb25c0a6290 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb2680ee700 tx=0x7fb26802a410 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:55.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.654+0000 7fb280045740 1 -- 172.21.15.83:0/2045680204 shutdown_connections 2024-04-24T08:57:55.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.654+0000 7fb280045740 1 -- 172.21.15.83:0/2045680204 wait complete. 2024-04-24T08:57:55.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.654+0000 7fb280045740 1 -- start start 2024-04-24T08:57:55.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.654+0000 7fb280045740 1 Processor -- start 2024-04-24T08:57:55.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.655+0000 7fb280045740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a050b97b40 0x55a050ba25c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:55.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.655+0000 7fb280045740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a050b98460 0x55a050b9f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:55.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.655+0000 7fb280045740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a050ba2b30 0x55a050b9ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:55.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.655+0000 7fb280045740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a050ba59c0 con 0x55a050b98460 2024-04-24T08:57:55.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.655+0000 7fb280045740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a050ab6810 con 0x55a050ba2b30 2024-04-24T08:57:55.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.655+0000 7fb280045740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a050ba5b40 con 0x55a050b97b40 2024-04-24T08:57:55.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.655+0000 7fb280045740 10 client.0 _ll_register_callbacks cb 0x55a050a84140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:57:55.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.655+0000 7fb275400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a050b98460 0x55a050b9f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:55.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.655+0000 7fb275400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a050b98460 0x55a050b9f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41754/0 (socket says 172.21.15.83:41754) 2024-04-24T08:57:55.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.655+0000 7fb275400640 1 -- 172.21.15.83:0/2973417323 learned_addr learned my addr 172.21.15.83:0/2973417323 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:57:55.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.655+0000 7fb276800640 1 --2- 172.21.15.83:0/2973417323 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a050b97b40 0x55a050ba25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:55.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.655+0000 7fb276800640 1 --2- 172.21.15.83:0/2973417323 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a050ba2b30 0x55a050b9ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:55.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.656+0000 7fb275400640 1 -- 172.21.15.83:0/2973417323 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a050b97b40 msgr2=0x55a050ba25c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:55.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.656+0000 7fb275400640 1 --2- 172.21.15.83:0/2973417323 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a050b97b40 0x55a050ba25c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.656+0000 7fb275400640 1 -- 172.21.15.83:0/2973417323 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a050ba2b30 msgr2=0x55a050b9ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:55.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.656+0000 7fb275400640 1 --2- 172.21.15.83:0/2973417323 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a050ba2b30 0x55a050b9ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.656+0000 7fb275400640 1 -- 172.21.15.83:0/2973417323 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a050ac73f0 con 0x55a050b98460 2024-04-24T08:57:55.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.656+0000 7fb275400640 1 --2- 172.21.15.83:0/2973417323 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a050b98460 0x55a050b9f8b0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7fb26400bf40 tx=0x7fb2640128e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2a6d18ec17828e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:55.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.656+0000 7fb257e00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:57:55.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.656+0000 7fb257e00640 1 -- 172.21.15.83:0/2973417323 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb26401f060 con 0x55a050b98460 2024-04-24T08:57:55.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.656+0000 7fb257e00640 1 -- 172.21.15.83:0/2973417323 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7fb264010960 con 0x55a050b98460 2024-04-24T08:57:55.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.657+0000 7fb280045740 1 -- 172.21.15.83:0/2973417323 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb270004d20 con 0x55a050b98460 2024-04-24T08:57:55.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.657+0000 7fb257e00640 1 -- 172.21.15.83:0/2973417323 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fb264026050 con 0x55a050b98460 2024-04-24T08:57:55.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.657+0000 7fb280045740 10 client.17898 fetch_fsmap learned FSMap version 33 2024-04-24T08:57:55.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.657+0000 7fb280045740 1 -- 172.21.15.83:0/2973417323 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55a05080f3f0 con 0x55a050b98460 2024-04-24T08:57:55.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.657+0000 7fb257e00640 1 -- 172.21.15.83:0/2973417323 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb270004d20 con 0x55a050b98460 2024-04-24T08:57:55.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.657+0000 7fb257e00640 1 -- 172.21.15.83:0/2973417323 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fb264019030 con 0x55a050b98460 2024-04-24T08:57:55.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.658+0000 7fb280045740 10 client.17898 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T08:57:55.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.658+0000 7fb280045740 10 client.17898 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-24T08:57:55.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.658+0000 7fb280045740 20 client.17898 populate_metadata read hostname 'smithi083' 2024-04-24T08:57:55.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.658+0000 7fb280045740 1 --2- 172.21.15.83:0/2973417323 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55a050bb2d60 0x55a050bb5150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:55.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.658+0000 7fb280045740 4 client.17898 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:57:55.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.658+0000 7fb280045740 1 -- 172.21.15.83:0/2973417323 --> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a050b0abf0 con 0x55a050bb2d60 2024-04-24T08:57:55.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.658+0000 7fb276800640 1 --2- 172.21.15.83:0/2973417323 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55a050bb2d60 0x55a050bb5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:55.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.659+0000 7fb257e00640 10 client.17898 ms_handle_connect on v2:172.21.15.138:6834/1909219758 2024-04-24T08:57:55.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.659+0000 7fb276800640 1 --2- 172.21.15.83:0/2973417323 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55a050bb2d60 0x55a050bb5150 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=73ecde8c385bf945 server_cookie=91104432c19c3016 in_seq=0 out_seq=0 2024-04-24T08:57:55.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.660+0000 7fb257e00640 1 -- 172.21.15.83:0/2973417323 <== mds.2 v2:172.21.15.138:6834/1909219758 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a050b0abf0 con 0x55a050bb2d60 2024-04-24T08:57:55.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.660+0000 7fb257e00640 10 client.17898 handle_command_reply: tid=0 2024-04-24T08:57:55.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.661+0000 7fb280045740 1 client.17898 shutdown 2024-04-24T08:57:55.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.661+0000 7fb280045740 1 -- 172.21.15.83:0/2973417323 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a050b98460 msgr2=0x55a050b9f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:55.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.661+0000 7fb280045740 1 --2- 172.21.15.83:0/2973417323 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a050b98460 0x55a050b9f8b0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7fb26400bf40 tx=0x7fb2640128e0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.661+0000 7fb280045740 1 -- 172.21.15.83:0/2973417323 shutdown_connections 2024-04-24T08:57:55.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.661+0000 7fb280045740 1 --2- 172.21.15.83:0/2973417323 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55a050bb2d60 0x55a050bb5150 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.661+0000 7fb280045740 1 --2- 172.21.15.83:0/2973417323 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a050ba2b30 0x55a050b9ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.661+0000 7fb280045740 1 --2- 172.21.15.83:0/2973417323 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a050b98460 0x55a050b9f8b0 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.661+0000 7fb280045740 1 --2- 172.21.15.83:0/2973417323 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a050b97b40 0x55a050ba25c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.661+0000 7fb257e00640 0 client.17898 ms_handle_reset on v2:172.21.15.138:6834/1909219758 2024-04-24T08:57:55.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.661+0000 7fb280045740 1 -- 172.21.15.83:0/2973417323 >> 172.21.15.83:0/2973417323 conn(0x55a050b93e60 msgr2=0x55a050b99c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:55.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.661+0000 7fb280045740 1 -- 172.21.15.83:0/2973417323 shutdown_connections 2024-04-24T08:57:55.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.661+0000 7fb280045740 1 -- 172.21.15.83:0/2973417323 wait complete. 2024-04-24T08:57:55.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.661+0000 7fb280045740 20 client.17898 trim_cache size 0 max 16384 2024-04-24T08:57:55.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.678+0000 7fb280045740 1 Processor -- start 2024-04-24T08:57:55.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.678+0000 7fb280045740 1 -- start start 2024-04-24T08:57:55.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.678+0000 7fb280045740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a050b97b40 0x55a050b9f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:55.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.678+0000 7fb280045740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a050b98460 0x55a050bae880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:55.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.678+0000 7fb280045740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a050ba2b30 0x55a050ba20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:55.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.678+0000 7fb280045740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb2781b1520 con 0x55a050ba2b30 2024-04-24T08:57:55.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.678+0000 7fb280045740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb2781b3060 con 0x55a050b98460 2024-04-24T08:57:55.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.678+0000 7fb280045740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb2781b1bf0 con 0x55a050b97b40 2024-04-24T08:57:55.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.679+0000 7fb276800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a050ba2b30 0x55a050ba20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:55.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.679+0000 7fb275400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a050b98460 0x55a050bae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:55.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.679+0000 7fb276800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a050ba2b30 0x55a050ba20f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41758/0 (socket says 172.21.15.83:41758) 2024-04-24T08:57:55.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.679+0000 7fb276800640 1 -- 172.21.15.83:0/2788387945 learned_addr learned my addr 172.21.15.83:0/2788387945 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:57:55.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.679+0000 7fb275400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a050b98460 0x55a050bae880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:39724/0 (socket says 172.21.15.83:39724) 2024-04-24T08:57:55.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.679+0000 7fb276800640 1 --2- 172.21.15.83:0/2788387945 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a050b97b40 0x55a050b9f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:55.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.679+0000 7fb276800640 1 -- 172.21.15.83:0/2788387945 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a050b97b40 msgr2=0x55a050b9f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:55.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.679+0000 7fb276800640 1 --2- 172.21.15.83:0/2788387945 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a050b97b40 0x55a050b9f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.679+0000 7fb276800640 1 -- 172.21.15.83:0/2788387945 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a050b98460 msgr2=0x55a050bae880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:55.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.679+0000 7fb276800640 1 --2- 172.21.15.83:0/2788387945 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a050b98460 0x55a050bae880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.679+0000 7fb276800640 1 -- 172.21.15.83:0/2788387945 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0508d97d0 con 0x55a050ba2b30 2024-04-24T08:57:55.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.680+0000 7fb276800640 1 --2- 172.21.15.83:0/2788387945 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a050ba2b30 0x55a050ba20f0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7fb270010d80 tx=0x7fb270010fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e56e4705c044cda server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:55.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.680+0000 7fb262a00640 1 -- 172.21.15.83:0/2788387945 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb270016040 con 0x55a050ba2b30 2024-04-24T08:57:55.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.680+0000 7fb262a00640 1 -- 172.21.15.83:0/2788387945 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb270003e20 con 0x55a050ba2b30 2024-04-24T08:57:55.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.680+0000 7fb262a00640 1 -- 172.21.15.83:0/2788387945 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb270023040 con 0x55a050ba2b30 2024-04-24T08:57:55.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.681+0000 7fb280045740 1 -- 172.21.15.83:0/2788387945 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a050ba2b30 msgr2=0x55a050ba20f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:55.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.681+0000 7fb280045740 1 --2- 172.21.15.83:0/2788387945 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a050ba2b30 0x55a050ba20f0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7fb270010d80 tx=0x7fb270010fd0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.681+0000 7fb280045740 1 -- 172.21.15.83:0/2788387945 shutdown_connections 2024-04-24T08:57:55.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.681+0000 7fb280045740 1 --2- 172.21.15.83:0/2788387945 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a050ba2b30 0x55a050ba20f0 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.681+0000 7fb280045740 1 --2- 172.21.15.83:0/2788387945 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a050b98460 0x55a050bae880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.681+0000 7fb280045740 1 --2- 172.21.15.83:0/2788387945 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a050b97b40 0x55a050b9f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.681+0000 7fb280045740 1 -- 172.21.15.83:0/2788387945 >> 172.21.15.83:0/2788387945 conn(0x55a050b93e60 msgr2=0x55a050b9f140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:55.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.681+0000 7fb280045740 1 -- 172.21.15.83:0/2788387945 shutdown_connections 2024-04-24T08:57:55.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.681+0000 7fb280045740 1 -- 172.21.15.83:0/2788387945 wait complete. 2024-04-24T08:57:55.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.682+0000 7fb280045740 1 -- start start 2024-04-24T08:57:55.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.682+0000 7fb280045740 1 Processor -- start 2024-04-24T08:57:55.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.682+0000 7fb280045740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a050b97b40 0x55a050bab150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:55.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.682+0000 7fb280045740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a050b98460 0x55a050ba35b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:55.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.682+0000 7fb280045740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a050ba2b30 0x55a050ba3b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:55.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.682+0000 7fb280045740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb2781b2990 con 0x55a050b97b40 2024-04-24T08:57:55.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.682+0000 7fb280045740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb2781b22c0 con 0x55a050b98460 2024-04-24T08:57:55.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.682+0000 7fb280045740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb2781af500 con 0x55a050ba2b30 2024-04-24T08:57:55.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.682+0000 7fb280045740 10 client.0 _ll_register_callbacks cb 0x7fb26400bb50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:57:55.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.683+0000 7fb276800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a050b97b40 0x55a050bab150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:55.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.683+0000 7fb276800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a050b97b40 0x55a050bab150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41768/0 (socket says 172.21.15.83:41768) 2024-04-24T08:57:55.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.683+0000 7fb276800640 1 -- 172.21.15.83:0/2212806605 learned_addr learned my addr 172.21.15.83:0/2212806605 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:57:55.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.683+0000 7fb275400640 1 --2- 172.21.15.83:0/2212806605 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a050b98460 0x55a050ba35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:55.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.683+0000 7fb276800640 1 -- 172.21.15.83:0/2212806605 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a050ba2b30 msgr2=0x55a050ba3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:55.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.683+0000 7fb276800640 1 --2- 172.21.15.83:0/2212806605 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a050ba2b30 0x55a050ba3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.683+0000 7fb276800640 1 -- 172.21.15.83:0/2212806605 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a050b98460 msgr2=0x55a050ba35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:55.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.683+0000 7fb276800640 1 --2- 172.21.15.83:0/2212806605 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a050b98460 0x55a050ba35b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.683+0000 7fb276800640 1 -- 172.21.15.83:0/2212806605 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a050ba4340 con 0x55a050b97b40 2024-04-24T08:57:55.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.683+0000 7fb275400640 1 --2- 172.21.15.83:0/2212806605 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a050b98460 0x55a050ba35b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:57:55.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.683+0000 7fb276800640 1 --2- 172.21.15.83:0/2212806605 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a050b97b40 0x55a050bab150 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7fb270009640 tx=0x7fb270005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81399ce7d5f42183 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:55.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.683+0000 7fb257e00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:57:55.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.684+0000 7fb257e00640 1 -- 172.21.15.83:0/2212806605 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb270016070 con 0x55a050b97b40 2024-04-24T08:57:55.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.684+0000 7fb257e00640 1 -- 172.21.15.83:0/2212806605 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7fb27002b490 con 0x55a050b97b40 2024-04-24T08:57:55.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.684+0000 7fb280045740 1 -- 172.21.15.83:0/2212806605 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb270003e20 con 0x55a050b97b40 2024-04-24T08:57:55.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.684+0000 7fb257e00640 1 -- 172.21.15.83:0/2212806605 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fb27002b6b0 con 0x55a050b97b40 2024-04-24T08:57:55.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.684+0000 7fb280045740 10 client.17904 fetch_fsmap learned FSMap version 33 2024-04-24T08:57:55.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.684+0000 7fb280045740 1 -- 172.21.15.83:0/2212806605 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55a050c21b40 con 0x55a050b97b40 2024-04-24T08:57:55.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.684+0000 7fb257e00640 1 -- 172.21.15.83:0/2212806605 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb270003e20 con 0x55a050b97b40 2024-04-24T08:57:55.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.685+0000 7fb257e00640 1 -- 172.21.15.83:0/2212806605 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fb270034070 con 0x55a050b97b40 2024-04-24T08:57:55.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.685+0000 7fb280045740 10 client.17904 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T08:57:55.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.685+0000 7fb280045740 10 client.17904 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-24T08:57:55.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.685+0000 7fb280045740 20 client.17904 populate_metadata read hostname 'smithi083' 2024-04-24T08:57:55.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.685+0000 7fb280045740 1 --2- 172.21.15.83:0/2212806605 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55a050c21f60 0x55a050c24350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:55.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.685+0000 7fb280045740 4 client.17904 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T08:57:55.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.685+0000 7fb280045740 1 -- 172.21.15.83:0/2212806605 --> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a050b04a60 con 0x55a050c21f60 2024-04-24T08:57:55.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.686+0000 7fb275400640 1 --2- 172.21.15.83:0/2212806605 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55a050c21f60 0x55a050c24350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:55.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.687+0000 7fb257e00640 10 client.17904 ms_handle_connect on v2:172.21.15.138:6834/1909219758 2024-04-24T08:57:55.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.687+0000 7fb275400640 1 --2- 172.21.15.83:0/2212806605 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55a050c21f60 0x55a050c24350 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=686389549c0da33f server_cookie=5304c81371c6743 in_seq=0 out_seq=0 2024-04-24T08:57:55.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.687+0000 7fb257e00640 1 -- 172.21.15.83:0/2212806605 <== mds.2 v2:172.21.15.138:6834/1909219758 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a050b04a60 con 0x55a050c21f60 2024-04-24T08:57:55.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.687+0000 7fb257e00640 10 client.17904 handle_command_reply: tid=0 2024-04-24T08:57:55.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.687+0000 7fb280045740 1 client.17904 shutdown 2024-04-24T08:57:55.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.688+0000 7fb280045740 1 -- 172.21.15.83:0/2212806605 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a050b97b40 msgr2=0x55a050bab150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:55.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.688+0000 7fb280045740 1 --2- 172.21.15.83:0/2212806605 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a050b97b40 0x55a050bab150 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7fb270009640 tx=0x7fb270005b00 comp rx=0 tx=0).stop 2024-04-24T08:57:55.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.688+0000 7fb280045740 1 -- 172.21.15.83:0/2212806605 shutdown_connections 2024-04-24T08:57:55.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.688+0000 7fb280045740 1 --2- 172.21.15.83:0/2212806605 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55a050c21f60 0x55a050c24350 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.688+0000 7fb280045740 1 --2- 172.21.15.83:0/2212806605 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a050ba2b30 0x55a050ba3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.688+0000 7fb280045740 1 --2- 172.21.15.83:0/2212806605 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a050b98460 0x55a050ba35b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.688+0000 7fb280045740 1 --2- 172.21.15.83:0/2212806605 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a050b97b40 0x55a050bab150 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.688+0000 7fb280045740 1 -- 172.21.15.83:0/2212806605 >> 172.21.15.83:0/2212806605 conn(0x55a050b93e60 msgr2=0x55a050b926c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:55.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.688+0000 7fb257e00640 0 client.17904 ms_handle_reset on v2:172.21.15.138:6834/1909219758 2024-04-24T08:57:55.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.688+0000 7fb280045740 1 -- 172.21.15.83:0/2212806605 shutdown_connections 2024-04-24T08:57:55.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.688+0000 7fb280045740 1 -- 172.21.15.83:0/2212806605 wait complete. 2024-04-24T08:57:55.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.688+0000 7fb280045740 20 client.17904 trim_cache size 0 max 16384 2024-04-24T08:57:55.690 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T08:57:55.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.689+0000 7fb277200640 1 -- 172.21.15.83:0/286690481 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb25c0a3dd0 msgr2=0x7fb25c0a6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:55.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.689+0000 7fb277200640 1 --2- 172.21.15.83:0/286690481 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb25c0a3dd0 0x7fb25c0a6290 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb2680ee700 tx=0x7fb26802a410 comp rx=0 tx=0).stop 2024-04-24T08:57:55.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.689+0000 7fb277200640 1 -- 172.21.15.83:0/286690481 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb278102110 msgr2=0x7fb278074300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:55.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.689+0000 7fb277200640 1 --2- 172.21.15.83:0/286690481 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb278102110 0x7fb278074300 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7fb268005940 tx=0x7fb268004270 comp rx=0 tx=0).stop 2024-04-24T08:57:55.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.689+0000 7fb277200640 1 -- 172.21.15.83:0/286690481 shutdown_connections 2024-04-24T08:57:55.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.689+0000 7fb277200640 1 --2- 172.21.15.83:0/286690481 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb25c0a3dd0 0x7fb25c0a6290 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.689+0000 7fb277200640 1 --2- 172.21.15.83:0/286690481 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb278074d80 0x7fb2781a6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.689+0000 7fb277200640 1 --2- 172.21.15.83:0/286690481 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb2781071e0 0x7fb278074840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.689+0000 7fb277200640 1 --2- 172.21.15.83:0/286690481 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb278102110 0x7fb278074300 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.689+0000 7fb277200640 1 -- 172.21.15.83:0/286690481 >> 172.21.15.83:0/286690481 conn(0x7fb2780fd9d0 msgr2=0x7fb2780feec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:55.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.689+0000 7fb277200640 1 -- 172.21.15.83:0/286690481 shutdown_connections 2024-04-24T08:57:55.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.689+0000 7fb277200640 1 -- 172.21.15.83:0/286690481 wait complete. 2024-04-24T08:57:55.705 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:57:55 smithi083 ceph-mon[27907]: pgmap v192: 129 pgs: 129 active+clean; 40 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 434 KiB/s wr, 43 op/s 2024-04-24T08:57:55.705 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:57:55 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:57:55.705 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:57:55 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/425860834' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:57:55.707 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T08:57:55.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.913+0000 7f8674a00640 1 -- 172.21.15.83:0/3503510688 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86700743a0 msgr2=0x7f8670074780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:55.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.913+0000 7f8674a00640 1 --2- 172.21.15.83:0/3503510688 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86700743a0 0x7f8670074780 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f8660009900 tx=0x7f866002a480 comp rx=0 tx=0).stop 2024-04-24T08:57:55.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.913+0000 7f8674a00640 1 -- 172.21.15.83:0/3503510688 shutdown_connections 2024-04-24T08:57:55.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.913+0000 7f8674a00640 1 --2- 172.21.15.83:0/3503510688 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8670112eb0 0x7f86701152a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.913+0000 7f8674a00640 1 --2- 172.21.15.83:0/3503510688 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8670074cc0 0x7f8670112970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.913+0000 7f8674a00640 1 --2- 172.21.15.83:0/3503510688 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86700743a0 0x7f8670074780 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.913+0000 7f8674a00640 1 -- 172.21.15.83:0/3503510688 >> 172.21.15.83:0/3503510688 conn(0x7f86700fd9d0 msgr2=0x7f86700ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:55.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.913+0000 7f8674a00640 1 -- 172.21.15.83:0/3503510688 shutdown_connections 2024-04-24T08:57:55.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.913+0000 7f8674a00640 1 -- 172.21.15.83:0/3503510688 wait complete. 2024-04-24T08:57:55.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.914+0000 7f8674a00640 1 Processor -- start 2024-04-24T08:57:55.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.914+0000 7f8674a00640 1 -- start start 2024-04-24T08:57:55.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.915+0000 7f8674a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f86700743a0 0x7f86701a4a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:55.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.915+0000 7f8674a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8670074cc0 0x7f86701a4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:55.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.915+0000 7f8674a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8670112eb0 0x7f867019eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:55.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.915+0000 7f8674a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f86701187a0 con 0x7f8670112eb0 2024-04-24T08:57:55.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.915+0000 7f8674a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8670118620 con 0x7f86700743a0 2024-04-24T08:57:55.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.915+0000 7f8674a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8670118920 con 0x7f8670074cc0 2024-04-24T08:57:55.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.915+0000 7f866f200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8670112eb0 0x7f867019eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:55.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.915+0000 7f8667e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8670074cc0 0x7f86701a4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:55.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.916+0000 7f8667e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8670074cc0 0x7f86701a4fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:37220/0 (socket says 172.21.15.83:37220) 2024-04-24T08:57:55.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.916+0000 7f8667e00640 1 -- 172.21.15.83:0/1473926693 learned_addr learned my addr 172.21.15.83:0/1473926693 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:57:55.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.916+0000 7f866e800640 1 --2- 172.21.15.83:0/1473926693 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f86700743a0 0x7f86701a4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:55.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.916+0000 7f866f200640 1 -- 172.21.15.83:0/1473926693 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8670074cc0 msgr2=0x7f86701a4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:55.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.916+0000 7f866f200640 1 --2- 172.21.15.83:0/1473926693 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8670074cc0 0x7f86701a4fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.916+0000 7f866f200640 1 -- 172.21.15.83:0/1473926693 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f86700743a0 msgr2=0x7f86701a4a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:55.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.916+0000 7f866f200640 1 --2- 172.21.15.83:0/1473926693 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f86700743a0 0x7f86701a4a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.916+0000 7f866f200640 1 -- 172.21.15.83:0/1473926693 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f867019f440 con 0x7f8670112eb0 2024-04-24T08:57:55.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.916+0000 7f8667e00640 1 --2- 172.21.15.83:0/1473926693 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8670074cc0 0x7f86701a4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T08:57:55.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.916+0000 7f866f200640 1 --2- 172.21.15.83:0/1473926693 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8670112eb0 0x7f867019eb80 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f8668005fd0 tx=0x7f866800ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:55.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.917+0000 7f8667400640 1 -- 172.21.15.83:0/1473926693 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f866800cdf0 con 0x7f8670112eb0 2024-04-24T08:57:55.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.917+0000 7f8674a00640 1 -- 172.21.15.83:0/1473926693 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f867019f730 con 0x7f8670112eb0 2024-04-24T08:57:55.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.917+0000 7f8667400640 1 -- 172.21.15.83:0/1473926693 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8668005020 con 0x7f8670112eb0 2024-04-24T08:57:55.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.917+0000 7f8667400640 1 -- 172.21.15.83:0/1473926693 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8668018670 con 0x7f8670112eb0 2024-04-24T08:57:55.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.917+0000 7f8674a00640 1 -- 172.21.15.83:0/1473926693 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86701ab9f0 con 0x7f8670112eb0 2024-04-24T08:57:55.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.921+0000 7f8678224740 1 Processor -- start 2024-04-24T08:57:55.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.921+0000 7f8667400640 1 -- 172.21.15.83:0/1473926693 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8668004a40 con 0x7f8670112eb0 2024-04-24T08:57:55.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.921+0000 7f8678224740 1 -- start start 2024-04-24T08:57:55.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.921+0000 7f8678224740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567939bab40 0x5567939baf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:55.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.921+0000 7f8678224740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5567939bb460 0x5567939c55f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:55.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.921+0000 7f8678224740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5567939c5b30 0x5567939c7f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:55.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.921+0000 7f8678224740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5567939201d0 con 0x5567939bab40 2024-04-24T08:57:55.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.921+0000 7f8678224740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5567938c6450 con 0x5567939c5b30 2024-04-24T08:57:55.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.921+0000 7f8678224740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55679391c190 con 0x5567939bb460 2024-04-24T08:57:55.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.922+0000 7f8667e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567939bab40 0x5567939baf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:55.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.922+0000 7f8667e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567939bab40 0x5567939baf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41788/0 (socket says 172.21.15.83:41788) 2024-04-24T08:57:55.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.922+0000 7f8667e00640 1 -- 172.21.15.83:0/619481462 learned_addr learned my addr 172.21.15.83:0/619481462 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:57:55.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.922+0000 7f866e800640 1 --2- 172.21.15.83:0/619481462 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5567939bb460 0x5567939c55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:55.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.922+0000 7f8667e00640 1 --2- 172.21.15.83:0/619481462 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5567939c5b30 0x5567939c7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:55.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.922+0000 7f8667e00640 1 -- 172.21.15.83:0/619481462 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5567939bb460 msgr2=0x5567939c55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:55.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.922+0000 7f8667e00640 1 --2- 172.21.15.83:0/619481462 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5567939bb460 0x5567939c55f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.922+0000 7f8667e00640 1 -- 172.21.15.83:0/619481462 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5567939c5b30 msgr2=0x5567939c7f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:55.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.922+0000 7f8667e00640 1 --2- 172.21.15.83:0/619481462 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5567939c5b30 0x5567939c7f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.922+0000 7f8667e00640 1 -- 172.21.15.83:0/619481462 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556793983310 con 0x5567939bab40 2024-04-24T08:57:55.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.923+0000 7f8667e00640 1 --2- 172.21.15.83:0/619481462 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567939bab40 0x5567939baf20 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f8658012a70 tx=0x7f865800bd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a418225dc89d00a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:55.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.923+0000 7f8667400640 1 --2- 172.21.15.83:0/1473926693 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f86540a3d20 0x7f86540a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:55.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.923+0000 7f8667400640 1 -- 172.21.15.83:0/1473926693 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f86680d1160 con 0x7f8670112eb0 2024-04-24T08:57:55.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.923+0000 7f866e800640 1 --2- 172.21.15.83:0/1473926693 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f86540a3d20 0x7f86540a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:55.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.923+0000 7f865f400640 1 -- 172.21.15.83:0/619481462 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8658004030 con 0x5567939bab40 2024-04-24T08:57:55.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.923+0000 7f865f400640 1 -- 172.21.15.83:0/619481462 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f865800e910 con 0x5567939bab40 2024-04-24T08:57:55.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.923+0000 7f865f400640 1 -- 172.21.15.83:0/619481462 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8658003a10 con 0x5567939bab40 2024-04-24T08:57:55.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.924+0000 7f8678224740 1 -- 172.21.15.83:0/619481462 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567939bab40 msgr2=0x5567939baf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:55.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.924+0000 7f866e800640 1 --2- 172.21.15.83:0/1473926693 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f86540a3d20 0x7f86540a61e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8660005ab0 tx=0x7f8660005a20 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:55.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.924+0000 7f8678224740 1 --2- 172.21.15.83:0/619481462 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567939bab40 0x5567939baf20 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f8658012a70 tx=0x7f865800bd20 comp rx=0 tx=0).stop 2024-04-24T08:57:55.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.924+0000 7f8678224740 1 -- 172.21.15.83:0/619481462 shutdown_connections 2024-04-24T08:57:55.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.924+0000 7f8678224740 1 --2- 172.21.15.83:0/619481462 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5567939c5b30 0x5567939c7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.924+0000 7f8678224740 1 --2- 172.21.15.83:0/619481462 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5567939bb460 0x5567939c55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.924+0000 7f8678224740 1 --2- 172.21.15.83:0/619481462 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567939bab40 0x5567939baf20 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.924+0000 7f8678224740 1 -- 172.21.15.83:0/619481462 >> 172.21.15.83:0/619481462 conn(0x5567939b6e60 msgr2=0x5567939b7240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:55.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.924+0000 7f8678224740 1 -- 172.21.15.83:0/619481462 shutdown_connections 2024-04-24T08:57:55.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.924+0000 7f8678224740 1 -- 172.21.15.83:0/619481462 wait complete. 2024-04-24T08:57:55.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.925+0000 7f8678224740 1 -- start start 2024-04-24T08:57:55.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.925+0000 7f8678224740 1 Processor -- start 2024-04-24T08:57:55.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.926+0000 7f8678224740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567939bab40 0x5567939c5580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:55.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.926+0000 7f8678224740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5567939bb460 0x5567939c2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:55.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.926+0000 7f8678224740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5567939c5b30 0x5567939c2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:55.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.926+0000 7f8678224740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5567939c89c0 con 0x5567939bab40 2024-04-24T08:57:55.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.926+0000 7f8678224740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5567938d9810 con 0x5567939bb460 2024-04-24T08:57:55.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.926+0000 7f8678224740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5567939c8b40 con 0x5567939c5b30 2024-04-24T08:57:55.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.926+0000 7f8678224740 10 client.0 _ll_register_callbacks cb 0x5567938a7140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:57:55.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.926+0000 7f866e800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567939bab40 0x5567939c5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:55.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.926+0000 7f866e800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567939bab40 0x5567939c5580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41802/0 (socket says 172.21.15.83:41802) 2024-04-24T08:57:55.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.926+0000 7f866e800640 1 -- 172.21.15.83:0/4168670062 learned_addr learned my addr 172.21.15.83:0/4168670062 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:57:55.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.926+0000 7f866f200640 1 --2- 172.21.15.83:0/4168670062 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5567939c5b30 0x5567939c2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:55.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.926+0000 7f8667e00640 1 --2- 172.21.15.83:0/4168670062 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5567939bb460 0x5567939c2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:55.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.927+0000 7f866f200640 1 -- 172.21.15.83:0/4168670062 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5567939bb460 msgr2=0x5567939c2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:55.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.927+0000 7f866f200640 1 --2- 172.21.15.83:0/4168670062 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5567939bb460 0x5567939c2870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.927+0000 7f866f200640 1 -- 172.21.15.83:0/4168670062 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567939bab40 msgr2=0x5567939c5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:55.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.927+0000 7f866f200640 1 --2- 172.21.15.83:0/4168670062 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567939bab40 0x5567939c5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.927+0000 7f866f200640 1 -- 172.21.15.83:0/4168670062 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567938ea3f0 con 0x5567939c5b30 2024-04-24T08:57:55.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.927+0000 7f866e800640 1 --2- 172.21.15.83:0/4168670062 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567939bab40 0x5567939c5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T08:57:55.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.927+0000 7f866f200640 1 --2- 172.21.15.83:0/4168670062 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5567939c5b30 0x5567939c2fb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f86680ce4e0 tx=0x7f866809cd30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d3d8ad511d6d8de server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:55.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.927+0000 7f8653e00640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-04-24T08:57:55.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.927+0000 7f8653e00640 1 -- 172.21.15.83:0/4168670062 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86680051d0 con 0x5567939c5b30 2024-04-24T08:57:55.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.927+0000 7f8678224740 1 -- 172.21.15.83:0/4168670062 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f865800e910 con 0x5567939c5b30 2024-04-24T08:57:55.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.927+0000 7f8653e00640 1 -- 172.21.15.83:0/4168670062 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f86680993a0 con 0x5567939c5b30 2024-04-24T08:57:55.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.928+0000 7f8653e00640 1 -- 172.21.15.83:0/4168670062 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f86680b6200 con 0x5567939c5b30 2024-04-24T08:57:55.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.928+0000 7f8678224740 10 client.26374 fetch_fsmap learned FSMap version 33 2024-04-24T08:57:55.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.928+0000 7f8678224740 1 -- 172.21.15.83:0/4168670062 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5567936323f0 con 0x5567939c5b30 2024-04-24T08:57:55.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.928+0000 7f8653e00640 1 -- 172.21.15.83:0/4168670062 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f865800e910 con 0x5567939c5b30 2024-04-24T08:57:55.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.928+0000 7f8653e00640 1 -- 172.21.15.83:0/4168670062 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f86680bd0d0 con 0x5567939c5b30 2024-04-24T08:57:55.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.928+0000 7f8678224740 10 client.26374 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T08:57:55.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.928+0000 7f8678224740 10 client.26374 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-24T08:57:55.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.928+0000 7f8678224740 20 client.26374 populate_metadata read hostname 'smithi083' 2024-04-24T08:57:55.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.928+0000 7f8678224740 1 --2- 172.21.15.83:0/4168670062 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x5567939d5d00 0x5567939d80f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:55.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.928+0000 7f8678224740 4 client.26374 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:57:55.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.928+0000 7f8678224740 1 -- 172.21.15.83:0/4168670062 --> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55679392dbf0 con 0x5567939d5d00 2024-04-24T08:57:55.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.929+0000 7f866e800640 1 --2- 172.21.15.83:0/4168670062 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x5567939d5d00 0x5567939d80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:55.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.930+0000 7f866e800640 1 --2- 172.21.15.83:0/4168670062 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x5567939d5d00 0x5567939d80f0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=60c6c558b0730098 server_cookie=97d23faff95fe026 in_seq=0 out_seq=0 2024-04-24T08:57:55.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.930+0000 7f8653e00640 10 client.26374 ms_handle_connect on v2:172.21.15.138:6836/900566754 2024-04-24T08:57:55.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.932+0000 7f8653e00640 1 -- 172.21.15.83:0/4168670062 <== mds.1 v2:172.21.15.138:6836/900566754 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55679392dbf0 con 0x5567939d5d00 2024-04-24T08:57:55.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.932+0000 7f8653e00640 10 client.26374 handle_command_reply: tid=0 2024-04-24T08:57:55.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.932+0000 7f8678224740 1 client.26374 shutdown 2024-04-24T08:57:55.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.932+0000 7f8678224740 1 -- 172.21.15.83:0/4168670062 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5567939c5b30 msgr2=0x5567939c2fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:55.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.932+0000 7f8678224740 1 --2- 172.21.15.83:0/4168670062 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5567939c5b30 0x5567939c2fb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f86680ce4e0 tx=0x7f866809cd30 comp rx=0 tx=0).stop 2024-04-24T08:57:55.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.932+0000 7f8678224740 1 -- 172.21.15.83:0/4168670062 shutdown_connections 2024-04-24T08:57:55.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.932+0000 7f8678224740 1 --2- 172.21.15.83:0/4168670062 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x5567939d5d00 0x5567939d80f0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.932+0000 7f8678224740 1 --2- 172.21.15.83:0/4168670062 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5567939c5b30 0x5567939c2fb0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.932+0000 7f8678224740 1 --2- 172.21.15.83:0/4168670062 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5567939bb460 0x5567939c2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.932+0000 7f8653e00640 0 client.26374 ms_handle_reset on v2:172.21.15.138:6836/900566754 2024-04-24T08:57:55.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.932+0000 7f8678224740 1 --2- 172.21.15.83:0/4168670062 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567939bab40 0x5567939c5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.932+0000 7f8678224740 1 -- 172.21.15.83:0/4168670062 >> 172.21.15.83:0/4168670062 conn(0x5567939b6e60 msgr2=0x5567939bcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:55.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.932+0000 7f8678224740 1 -- 172.21.15.83:0/4168670062 shutdown_connections 2024-04-24T08:57:55.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.932+0000 7f8678224740 1 -- 172.21.15.83:0/4168670062 wait complete. 2024-04-24T08:57:55.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.932+0000 7f8678224740 20 client.26374 trim_cache size 0 max 16384 2024-04-24T08:57:55.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.948+0000 7f8678224740 1 Processor -- start 2024-04-24T08:57:55.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.948+0000 7f8678224740 1 -- start start 2024-04-24T08:57:55.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.948+0000 7f8678224740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5567939bab40 0x5567939d1880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:55.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.948+0000 7f8678224740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567939bb460 0x5567939c1850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:55.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.948+0000 7f8678224740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5567939c5b30 0x5567939c50b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:55.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.948+0000 7f8678224740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f86701b5960 con 0x5567939bb460 2024-04-24T08:57:55.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.948+0000 7f8678224740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f86701b74a0 con 0x5567939bab40 2024-04-24T08:57:55.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.948+0000 7f8678224740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f86701b6030 con 0x5567939c5b30 2024-04-24T08:57:55.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.949+0000 7f8667e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567939bb460 0x5567939c1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:55.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.949+0000 7f866f200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5567939c5b30 0x5567939c50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:55.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.949+0000 7f8667e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567939bb460 0x5567939c1850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41812/0 (socket says 172.21.15.83:41812) 2024-04-24T08:57:55.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.949+0000 7f8667e00640 1 -- 172.21.15.83:0/2647018380 learned_addr learned my addr 172.21.15.83:0/2647018380 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:57:55.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.949+0000 7f866e800640 1 --2- 172.21.15.83:0/2647018380 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5567939bab40 0x5567939d1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:55.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.949+0000 7f8667e00640 1 -- 172.21.15.83:0/2647018380 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5567939c5b30 msgr2=0x5567939c50b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:55.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.949+0000 7f8667e00640 1 --2- 172.21.15.83:0/2647018380 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5567939c5b30 0x5567939c50b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.949+0000 7f8667e00640 1 -- 172.21.15.83:0/2647018380 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5567939bab40 msgr2=0x5567939d1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:55.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.949+0000 7f8667e00640 1 --2- 172.21.15.83:0/2647018380 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5567939bab40 0x5567939d1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.949+0000 7f8667e00640 1 -- 172.21.15.83:0/2647018380 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567936fc7d0 con 0x5567939bb460 2024-04-24T08:57:55.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.949+0000 7f866e800640 1 --2- 172.21.15.83:0/2647018380 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5567939bab40 0x5567939d1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:57:55.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.950+0000 7f8667e00640 1 --2- 172.21.15.83:0/2647018380 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567939bb460 0x5567939c1850 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f865800c010 tx=0x7f865800bd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d58d871fa7119925 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:55.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.950+0000 7f865ea00640 1 -- 172.21.15.83:0/2647018380 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f865800e910 con 0x5567939bb460 2024-04-24T08:57:55.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.950+0000 7f865ea00640 1 -- 172.21.15.83:0/2647018380 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f86580050d0 con 0x5567939bb460 2024-04-24T08:57:55.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.950+0000 7f865ea00640 1 -- 172.21.15.83:0/2647018380 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8658003980 con 0x5567939bb460 2024-04-24T08:57:55.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.951+0000 7f8678224740 1 -- 172.21.15.83:0/2647018380 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567939bb460 msgr2=0x5567939c1850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:55.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.951+0000 7f8678224740 1 --2- 172.21.15.83:0/2647018380 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567939bb460 0x5567939c1850 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f865800c010 tx=0x7f865800bd90 comp rx=0 tx=0).stop 2024-04-24T08:57:55.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.951+0000 7f8678224740 1 -- 172.21.15.83:0/2647018380 shutdown_connections 2024-04-24T08:57:55.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.951+0000 7f8678224740 1 --2- 172.21.15.83:0/2647018380 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5567939c5b30 0x5567939c50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.951+0000 7f8678224740 1 --2- 172.21.15.83:0/2647018380 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567939bb460 0x5567939c1850 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.951+0000 7f8678224740 1 --2- 172.21.15.83:0/2647018380 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5567939bab40 0x5567939d1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.951+0000 7f8678224740 1 -- 172.21.15.83:0/2647018380 >> 172.21.15.83:0/2647018380 conn(0x5567939b6e60 msgr2=0x5567939c1140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:55.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.951+0000 7f8678224740 1 -- 172.21.15.83:0/2647018380 shutdown_connections 2024-04-24T08:57:55.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.951+0000 7f8678224740 1 -- 172.21.15.83:0/2647018380 wait complete. 2024-04-24T08:57:55.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.952+0000 7f8678224740 1 -- start start 2024-04-24T08:57:55.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.952+0000 7f8678224740 1 Processor -- start 2024-04-24T08:57:55.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.952+0000 7f8678224740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5567939bab40 0x5567939c66c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:55.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.952+0000 7f8678224740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567939bb460 0x5567939c6c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:55.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.952+0000 7f8678224740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5567939c5b30 0x5567939ccd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:55.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.952+0000 7f8678224740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f86701b6dd0 con 0x5567939bb460 2024-04-24T08:57:55.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.952+0000 7f8678224740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f86701b6700 con 0x5567939c5b30 2024-04-24T08:57:55.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.952+0000 7f8678224740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f86701b3940 con 0x5567939bab40 2024-04-24T08:57:55.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.952+0000 7f8678224740 10 client.0 _ll_register_callbacks cb 0x7f86680057a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:57:55.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.952+0000 7f8667e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567939bb460 0x5567939c6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:55.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.953+0000 7f8667e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567939bb460 0x5567939c6c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41828/0 (socket says 172.21.15.83:41828) 2024-04-24T08:57:55.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.953+0000 7f8667e00640 1 -- 172.21.15.83:0/2882874314 learned_addr learned my addr 172.21.15.83:0/2882874314 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:57:55.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.953+0000 7f866e800640 1 --2- 172.21.15.83:0/2882874314 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5567939bab40 0x5567939c66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:55.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.953+0000 7f866f200640 1 --2- 172.21.15.83:0/2882874314 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5567939c5b30 0x5567939ccd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:55.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.953+0000 7f8667e00640 1 -- 172.21.15.83:0/2882874314 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5567939bab40 msgr2=0x5567939c66c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:55.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.953+0000 7f8667e00640 1 --2- 172.21.15.83:0/2882874314 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5567939bab40 0x5567939c66c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.953+0000 7f8667e00640 1 -- 172.21.15.83:0/2882874314 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5567939c5b30 msgr2=0x5567939ccd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:55.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.953+0000 7f8667e00640 1 --2- 172.21.15.83:0/2882874314 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5567939c5b30 0x5567939ccd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.953+0000 7f8667e00640 1 -- 172.21.15.83:0/2882874314 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567939cd3d0 con 0x5567939bb460 2024-04-24T08:57:55.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:57:55 smithi138 ceph-mon[31441]: pgmap v192: 129 pgs: 129 active+clean; 40 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 434 KiB/s wr, 43 op/s 2024-04-24T08:57:55.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:57:55 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:57:55.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:57:55 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/425860834' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:57:55.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.953+0000 7f8667e00640 1 --2- 172.21.15.83:0/2882874314 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567939bb460 0x5567939c6c00 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f865800bfe0 tx=0x7f8658012300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b655922ffe8d9a18 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:55.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.953+0000 7f8653e00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:57:55.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.953+0000 7f8653e00640 1 -- 172.21.15.83:0/2882874314 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f865800eb10 con 0x5567939bb460 2024-04-24T08:57:55.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.954+0000 7f8678224740 1 -- 172.21.15.83:0/2882874314 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86580050d0 con 0x5567939bb460 2024-04-24T08:57:55.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.954+0000 7f8653e00640 1 -- 172.21.15.83:0/2882874314 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f865800ecb0 con 0x5567939bb460 2024-04-24T08:57:55.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.954+0000 7f8653e00640 1 -- 172.21.15.83:0/2882874314 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f865803c050 con 0x5567939bb460 2024-04-24T08:57:55.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.954+0000 7f8678224740 10 client.17922 fetch_fsmap learned FSMap version 33 2024-04-24T08:57:55.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.954+0000 7f8678224740 1 -- 172.21.15.83:0/2882874314 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5567939ce650 con 0x5567939bb460 2024-04-24T08:57:55.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.954+0000 7f8653e00640 1 -- 172.21.15.83:0/2882874314 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f86580050d0 con 0x5567939bb460 2024-04-24T08:57:55.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.954+0000 7f8653e00640 1 -- 172.21.15.83:0/2882874314 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f865801c020 con 0x5567939bb460 2024-04-24T08:57:55.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.955+0000 7f8678224740 10 client.17922 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T08:57:55.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.955+0000 7f8678224740 10 client.17922 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-24T08:57:55.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.955+0000 7f8678224740 20 client.17922 populate_metadata read hostname 'smithi083' 2024-04-24T08:57:55.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.955+0000 7f8678224740 1 --2- 172.21.15.83:0/2882874314 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x556793a3c5b0 0x556793a3e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:55.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.955+0000 7f8678224740 4 client.17922 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T08:57:55.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.955+0000 7f8678224740 1 -- 172.21.15.83:0/2882874314 --> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556793927a60 con 0x556793a3c5b0 2024-04-24T08:57:55.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.955+0000 7f866e800640 1 --2- 172.21.15.83:0/2882874314 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x556793a3c5b0 0x556793a3e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:55.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.956+0000 7f866e800640 1 --2- 172.21.15.83:0/2882874314 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x556793a3c5b0 0x556793a3e9d0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=88ac56fab255aba5 server_cookie=800ca485fbda2dfa in_seq=0 out_seq=0 2024-04-24T08:57:55.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.956+0000 7f8653e00640 10 client.17922 ms_handle_connect on v2:172.21.15.138:6836/900566754 2024-04-24T08:57:55.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.956+0000 7f8653e00640 1 -- 172.21.15.83:0/2882874314 <== mds.1 v2:172.21.15.138:6836/900566754 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556793927a60 con 0x556793a3c5b0 2024-04-24T08:57:55.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.956+0000 7f8653e00640 10 client.17922 handle_command_reply: tid=0 2024-04-24T08:57:55.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.956+0000 7f8678224740 1 client.17922 shutdown 2024-04-24T08:57:55.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.956+0000 7f8678224740 1 -- 172.21.15.83:0/2882874314 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567939bb460 msgr2=0x5567939c6c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:55.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.956+0000 7f8678224740 1 --2- 172.21.15.83:0/2882874314 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567939bb460 0x5567939c6c00 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f865800bfe0 tx=0x7f8658012300 comp rx=0 tx=0).stop 2024-04-24T08:57:55.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.957+0000 7f8678224740 1 -- 172.21.15.83:0/2882874314 shutdown_connections 2024-04-24T08:57:55.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.957+0000 7f8678224740 1 --2- 172.21.15.83:0/2882874314 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5567939c5b30 0x5567939ccd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.957+0000 7f8678224740 1 --2- 172.21.15.83:0/2882874314 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567939bb460 0x5567939c6c00 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.957+0000 7f8678224740 1 --2- 172.21.15.83:0/2882874314 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x556793a3c5b0 0x556793a3e9d0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.957+0000 7f8678224740 1 --2- 172.21.15.83:0/2882874314 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5567939bab40 0x5567939c66c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.957+0000 7f8653e00640 0 client.17922 ms_handle_reset on v2:172.21.15.138:6836/900566754 2024-04-24T08:57:55.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.957+0000 7f8678224740 1 -- 172.21.15.83:0/2882874314 >> 172.21.15.83:0/2882874314 conn(0x5567939b6e60 msgr2=0x5567939c19d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:55.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.957+0000 7f8678224740 1 -- 172.21.15.83:0/2882874314 shutdown_connections 2024-04-24T08:57:55.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.957+0000 7f8678224740 1 -- 172.21.15.83:0/2882874314 wait complete. 2024-04-24T08:57:55.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.957+0000 7f8678224740 20 client.17922 trim_cache size 0 max 16384 2024-04-24T08:57:55.960 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T08:57:55.961 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:57:55 smithi189 ceph-mon[31791]: pgmap v192: 129 pgs: 129 active+clean; 40 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 434 KiB/s wr, 43 op/s 2024-04-24T08:57:55.961 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:57:55 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:57:55.961 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:57:55 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/425860834' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:57:55.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.957+0000 7f8674a00640 1 -- 172.21.15.83:0/1473926693 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f86540a3d20 msgr2=0x7f86540a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:55.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.957+0000 7f8674a00640 1 --2- 172.21.15.83:0/1473926693 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f86540a3d20 0x7f86540a61e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8660005ab0 tx=0x7f8660005a20 comp rx=0 tx=0).stop 2024-04-24T08:57:55.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.957+0000 7f8674a00640 1 -- 172.21.15.83:0/1473926693 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8670112eb0 msgr2=0x7f867019eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:55.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.957+0000 7f8674a00640 1 --2- 172.21.15.83:0/1473926693 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8670112eb0 0x7f867019eb80 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f8668005fd0 tx=0x7f866800ef10 comp rx=0 tx=0).stop 2024-04-24T08:57:55.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.958+0000 7f8674a00640 1 -- 172.21.15.83:0/1473926693 shutdown_connections 2024-04-24T08:57:55.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.958+0000 7f8674a00640 1 --2- 172.21.15.83:0/1473926693 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f86540a3d20 0x7f86540a61e0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.958+0000 7f8674a00640 1 --2- 172.21.15.83:0/1473926693 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8670112eb0 0x7f867019eb80 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.958+0000 7f8674a00640 1 --2- 172.21.15.83:0/1473926693 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8670074cc0 0x7f86701a4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.958+0000 7f8674a00640 1 --2- 172.21.15.83:0/1473926693 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f86700743a0 0x7f86701a4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:55.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.958+0000 7f8674a00640 1 -- 172.21.15.83:0/1473926693 >> 172.21.15.83:0/1473926693 conn(0x7f86700fd9d0 msgr2=0x7f86700ff2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:55.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.958+0000 7f8674a00640 1 -- 172.21.15.83:0/1473926693 shutdown_connections 2024-04-24T08:57:55.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:55.958+0000 7f8674a00640 1 -- 172.21.15.83:0/1473926693 wait complete. 2024-04-24T08:57:55.975 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T08:57:56.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.172+0000 7fbe29c00640 1 -- 172.21.15.83:0/3108297954 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbe2410e760 msgr2=0x7fbe24110b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:56.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.172+0000 7fbe29c00640 1 --2- 172.21.15.83:0/3108297954 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbe2410e760 0x7fbe24110b50 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c009ca0 tx=0x7fbe1c02a950 comp rx=0 tx=0).stop 2024-04-24T08:57:56.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.172+0000 7fbe29c00640 1 -- 172.21.15.83:0/3108297954 shutdown_connections 2024-04-24T08:57:56.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.172+0000 7fbe29c00640 1 --2- 172.21.15.83:0/3108297954 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbe2410e760 0x7fbe24110b50 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:56.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.172+0000 7fbe29c00640 1 --2- 172.21.15.83:0/3108297954 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe24074cd0 0x7fbe2410e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:56.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.172+0000 7fbe29c00640 1 --2- 172.21.15.83:0/3108297954 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbe240743b0 0x7fbe24074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:56.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.172+0000 7fbe29c00640 1 -- 172.21.15.83:0/3108297954 >> 172.21.15.83:0/3108297954 conn(0x7fbe240fd9d0 msgr2=0x7fbe240ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:56.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.173+0000 7fbe29c00640 1 -- 172.21.15.83:0/3108297954 shutdown_connections 2024-04-24T08:57:56.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.173+0000 7fbe29c00640 1 -- 172.21.15.83:0/3108297954 wait complete. 2024-04-24T08:57:56.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.174+0000 7fbe29c00640 1 Processor -- start 2024-04-24T08:57:56.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.174+0000 7fbe29c00640 1 -- start start 2024-04-24T08:57:56.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.174+0000 7fbe29c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe240743b0 0x7fbe241a05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:56.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.174+0000 7fbe29c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbe24074cd0 0x7fbe241a0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:56.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.174+0000 7fbe29c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbe2410e760 0x7fbe2419a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:56.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.174+0000 7fbe29c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbe24114340 con 0x7fbe24074cd0 2024-04-24T08:57:56.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.174+0000 7fbe29c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbe241141c0 con 0x7fbe240743b0 2024-04-24T08:57:56.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.174+0000 7fbe29c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fbe241144c0 con 0x7fbe2410e760 2024-04-24T08:57:56.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.175+0000 7fbe23400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbe24074cd0 0x7fbe241a0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:56.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.175+0000 7fbe23400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbe24074cd0 0x7fbe241a0b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41836/0 (socket says 172.21.15.83:41836) 2024-04-24T08:57:56.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.175+0000 7fbe23400640 1 -- 172.21.15.83:0/2461098367 learned_addr learned my addr 172.21.15.83:0/2461098367 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:57:56.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.175+0000 7fbe23e00640 1 --2- 172.21.15.83:0/2461098367 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe240743b0 0x7fbe241a05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:56.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.175+0000 7fbe29200640 1 --2- 172.21.15.83:0/2461098367 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbe2410e760 0x7fbe2419a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:56.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.175+0000 7fbe23400640 1 -- 172.21.15.83:0/2461098367 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbe2410e760 msgr2=0x7fbe2419a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:56.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.175+0000 7fbe23400640 1 --2- 172.21.15.83:0/2461098367 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbe2410e760 0x7fbe2419a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:56.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.175+0000 7fbe23400640 1 -- 172.21.15.83:0/2461098367 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe240743b0 msgr2=0x7fbe241a05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:56.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.175+0000 7fbe23400640 1 --2- 172.21.15.83:0/2461098367 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe240743b0 0x7fbe241a05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:56.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.175+0000 7fbe23400640 1 -- 172.21.15.83:0/2461098367 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe2419afa0 con 0x7fbe24074cd0 2024-04-24T08:57:56.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.175+0000 7fbe29200640 1 --2- 172.21.15.83:0/2461098367 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbe2410e760 0x7fbe2419a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:57:56.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.176+0000 7fbe23400640 1 --2- 172.21.15.83:0/2461098367 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbe24074cd0 0x7fbe241a0b10 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7fbe1000c890 tx=0x7fbe1000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:56.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.176+0000 7fbe20c00640 1 -- 172.21.15.83:0/2461098367 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe10004220 con 0x7fbe24074cd0 2024-04-24T08:57:56.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.176+0000 7fbe20c00640 1 -- 172.21.15.83:0/2461098367 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbe100043c0 con 0x7fbe24074cd0 2024-04-24T08:57:56.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.176+0000 7fbe29c00640 1 -- 172.21.15.83:0/2461098367 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe2419b290 con 0x7fbe24074cd0 2024-04-24T08:57:56.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.176+0000 7fbe20c00640 1 -- 172.21.15.83:0/2461098367 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe10004a00 con 0x7fbe24074cd0 2024-04-24T08:57:56.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.176+0000 7fbe29c00640 1 -- 172.21.15.83:0/2461098367 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe241a7550 con 0x7fbe24074cd0 2024-04-24T08:57:56.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.180+0000 7fbe2e043740 1 Processor -- start 2024-04-24T08:57:56.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.181+0000 7fbe2e043740 1 -- start start 2024-04-24T08:57:56.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.181+0000 7fbe20c00640 1 -- 172.21.15.83:0/2461098367 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fbe10004ba0 con 0x7fbe24074cd0 2024-04-24T08:57:56.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.181+0000 7fbe2e043740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e50a13b40 0x559e50a13f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:56.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.181+0000 7fbe2e043740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e50a14460 0x559e50a1e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:56.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.181+0000 7fbe2e043740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e50a1eb30 0x559e50a20f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:56.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.181+0000 7fbe2e043740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559e509791d0 con 0x559e50a1eb30 2024-04-24T08:57:56.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.181+0000 7fbe2e043740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559e5091f450 con 0x559e50a13b40 2024-04-24T08:57:56.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.181+0000 7fbe2e043740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559e50975190 con 0x559e50a14460 2024-04-24T08:57:56.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.181+0000 7fbe23400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e50a1eb30 0x559e50a20f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:56.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.181+0000 7fbe23400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e50a1eb30 0x559e50a20f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41844/0 (socket says 172.21.15.83:41844) 2024-04-24T08:57:56.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.181+0000 7fbe23400640 1 -- 172.21.15.83:0/2556386981 learned_addr learned my addr 172.21.15.83:0/2556386981 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:57:56.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.182+0000 7fbe29200640 1 --2- 172.21.15.83:0/2556386981 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e50a13b40 0x559e50a13f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:56.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.182+0000 7fbe23e00640 1 --2- 172.21.15.83:0/2556386981 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e50a14460 0x559e50a1e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:56.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.182+0000 7fbe23400640 1 -- 172.21.15.83:0/2556386981 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e50a14460 msgr2=0x559e50a1e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:56.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.182+0000 7fbe23400640 1 --2- 172.21.15.83:0/2556386981 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e50a14460 0x559e50a1e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:56.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.182+0000 7fbe23400640 1 -- 172.21.15.83:0/2556386981 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e50a13b40 msgr2=0x559e50a13f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:56.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.182+0000 7fbe23400640 1 --2- 172.21.15.83:0/2556386981 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e50a13b40 0x559e50a13f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:56.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.182+0000 7fbe23400640 1 -- 172.21.15.83:0/2556386981 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e509dc310 con 0x559e50a1eb30 2024-04-24T08:57:56.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.183+0000 7fbe23400640 1 --2- 172.21.15.83:0/2556386981 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e50a1eb30 0x559e50a20f20 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7fbe100cdad0 tx=0x7fbe100d8690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1d870dcaa553f72 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:56.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.183+0000 7fbe20c00640 1 --2- 172.21.15.83:0/2461098367 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fbe080a3da0 0x7fbe080a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:56.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.183+0000 7fbe20c00640 1 -- 172.21.15.83:0/2461098367 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7fbe100d07e0 con 0x7fbe24074cd0 2024-04-24T08:57:56.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.183+0000 7fbe29200640 1 --2- 172.21.15.83:0/2461098367 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fbe080a3da0 0x7fbe080a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:56.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.183+0000 7fbe19600640 1 -- 172.21.15.83:0/2556386981 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe100df070 con 0x559e50a1eb30 2024-04-24T08:57:56.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.183+0000 7fbe19600640 1 -- 172.21.15.83:0/2556386981 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbe1009cab0 con 0x559e50a1eb30 2024-04-24T08:57:56.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.183+0000 7fbe19600640 1 -- 172.21.15.83:0/2556386981 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe10098650 con 0x559e50a1eb30 2024-04-24T08:57:56.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.184+0000 7fbe2e043740 1 -- 172.21.15.83:0/2556386981 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e50a1eb30 msgr2=0x559e50a20f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:56.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.184+0000 7fbe29200640 1 --2- 172.21.15.83:0/2461098367 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fbe080a3da0 0x7fbe080a6260 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c004af0 tx=0x7fbe1c004a40 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:56.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.184+0000 7fbe2e043740 1 --2- 172.21.15.83:0/2556386981 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e50a1eb30 0x559e50a20f20 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7fbe100cdad0 tx=0x7fbe100d8690 comp rx=0 tx=0).stop 2024-04-24T08:57:56.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.184+0000 7fbe2e043740 1 -- 172.21.15.83:0/2556386981 shutdown_connections 2024-04-24T08:57:56.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.184+0000 7fbe2e043740 1 --2- 172.21.15.83:0/2556386981 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e50a1eb30 0x559e50a20f20 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:56.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.184+0000 7fbe2e043740 1 --2- 172.21.15.83:0/2556386981 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e50a14460 0x559e50a1e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:56.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.184+0000 7fbe2e043740 1 --2- 172.21.15.83:0/2556386981 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e50a13b40 0x559e50a13f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:56.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.184+0000 7fbe2e043740 1 -- 172.21.15.83:0/2556386981 >> 172.21.15.83:0/2556386981 conn(0x559e50a0fe60 msgr2=0x559e50a10240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:56.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.184+0000 7fbe2e043740 1 -- 172.21.15.83:0/2556386981 shutdown_connections 2024-04-24T08:57:56.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.184+0000 7fbe2e043740 1 -- 172.21.15.83:0/2556386981 wait complete. 2024-04-24T08:57:56.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.185+0000 7fbe2e043740 1 -- start start 2024-04-24T08:57:56.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.185+0000 7fbe2e043740 1 Processor -- start 2024-04-24T08:57:56.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.185+0000 7fbe2e043740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e50a13b40 0x559e50a1e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:56.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.186+0000 7fbe2e043740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e50a14460 0x559e50a1b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:56.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.186+0000 7fbe2e043740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e50a1eb30 0x559e50a1bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:56.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.186+0000 7fbe2e043740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559e50a219c0 con 0x559e50a1eb30 2024-04-24T08:57:56.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.186+0000 7fbe2e043740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559e50932810 con 0x559e50a14460 2024-04-24T08:57:56.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.186+0000 7fbe2e043740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559e50a21b40 con 0x559e50a13b40 2024-04-24T08:57:56.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.186+0000 7fbe2e043740 10 client.0 _ll_register_callbacks cb 0x559e50900140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:57:56.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.186+0000 7fbe29200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e50a1eb30 0x559e50a1bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:56.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.186+0000 7fbe29200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e50a1eb30 0x559e50a1bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41846/0 (socket says 172.21.15.83:41846) 2024-04-24T08:57:56.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.186+0000 7fbe29200640 1 -- 172.21.15.83:0/4080350697 learned_addr learned my addr 172.21.15.83:0/4080350697 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:57:56.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.186+0000 7fbe23e00640 1 --2- 172.21.15.83:0/4080350697 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e50a13b40 0x559e50a1e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:56.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.186+0000 7fbe23400640 1 --2- 172.21.15.83:0/4080350697 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e50a14460 0x559e50a1b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:56.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.186+0000 7fbe29200640 1 -- 172.21.15.83:0/4080350697 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e50a13b40 msgr2=0x559e50a1e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:56.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.187+0000 7fbe29200640 1 --2- 172.21.15.83:0/4080350697 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e50a13b40 0x559e50a1e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:56.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.187+0000 7fbe29200640 1 -- 172.21.15.83:0/4080350697 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e50a14460 msgr2=0x559e50a1b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:56.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.187+0000 7fbe29200640 1 --2- 172.21.15.83:0/4080350697 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e50a14460 0x559e50a1b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:56.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.187+0000 7fbe29200640 1 -- 172.21.15.83:0/4080350697 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e509433f0 con 0x559e50a1eb30 2024-04-24T08:57:56.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.187+0000 7fbe23e00640 1 --2- 172.21.15.83:0/4080350697 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e50a13b40 0x559e50a1e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:57:56.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.187+0000 7fbe29200640 1 --2- 172.21.15.83:0/4080350697 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e50a1eb30 0x559e50a1bfb0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c003f00 tx=0x7fbe1c02a950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dc76684a51c1ed5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:56.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.187+0000 7fbe0e000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:57:56.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.187+0000 7fbe0e000640 1 -- 172.21.15.83:0/4080350697 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe1c034040 con 0x559e50a1eb30 2024-04-24T08:57:56.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.187+0000 7fbe0e000640 1 -- 172.21.15.83:0/4080350697 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7fbe1c004000 con 0x559e50a1eb30 2024-04-24T08:57:56.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.188+0000 7fbe2e043740 1 -- 172.21.15.83:0/4080350697 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe1009cab0 con 0x559e50a1eb30 2024-04-24T08:57:56.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.188+0000 7fbe0e000640 1 -- 172.21.15.83:0/4080350697 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fbe1c02f670 con 0x559e50a1eb30 2024-04-24T08:57:56.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.188+0000 7fbe2e043740 10 client.17934 fetch_fsmap learned FSMap version 33 2024-04-24T08:57:56.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.188+0000 7fbe2e043740 1 -- 172.21.15.83:0/4080350697 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x559e5068b3f0 con 0x559e50a1eb30 2024-04-24T08:57:56.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.188+0000 7fbe0e000640 1 -- 172.21.15.83:0/4080350697 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbe1009cab0 con 0x559e50a1eb30 2024-04-24T08:57:56.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.189+0000 7fbe0e000640 1 -- 172.21.15.83:0/4080350697 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fbe1c04f070 con 0x559e50a1eb30 2024-04-24T08:57:56.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.189+0000 7fbe2e043740 10 client.17934 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T08:57:56.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.189+0000 7fbe2e043740 10 client.17934 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:57:56.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.189+0000 7fbe2e043740 20 client.17934 populate_metadata read hostname 'smithi083' 2024-04-24T08:57:56.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.189+0000 7fbe2e043740 1 --2- 172.21.15.83:0/4080350697 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x559e50a2ecd0 0x559e50a31140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:56.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.189+0000 7fbe2e043740 4 client.17934 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:57:56.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.189+0000 7fbe2e043740 1 -- 172.21.15.83:0/4080350697 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559e50986bf0 con 0x559e50a2ecd0 2024-04-24T08:57:56.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.190+0000 7fbe23e00640 1 --2- 172.21.15.83:0/4080350697 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x559e50a2ecd0 0x559e50a31140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:56.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.191+0000 7fbe0e000640 10 client.17934 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:57:56.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.191+0000 7fbe23e00640 1 --2- 172.21.15.83:0/4080350697 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x559e50a2ecd0 0x559e50a31140 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9a6fddebe319d26 server_cookie=d92df2b1ca7cacb7 in_seq=0 out_seq=0 2024-04-24T08:57:56.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.192+0000 7fbe0e000640 1 -- 172.21.15.83:0/4080350697 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559e50986bf0 con 0x559e50a2ecd0 2024-04-24T08:57:56.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.192+0000 7fbe0e000640 10 client.17934 handle_command_reply: tid=0 2024-04-24T08:57:56.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.192+0000 7fbe2e043740 1 client.17934 shutdown 2024-04-24T08:57:56.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.193+0000 7fbe2e043740 1 -- 172.21.15.83:0/4080350697 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e50a1eb30 msgr2=0x559e50a1bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:56.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.193+0000 7fbe2e043740 1 --2- 172.21.15.83:0/4080350697 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e50a1eb30 0x559e50a1bfb0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c003f00 tx=0x7fbe1c02a950 comp rx=0 tx=0).stop 2024-04-24T08:57:56.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.193+0000 7fbe2e043740 1 -- 172.21.15.83:0/4080350697 shutdown_connections 2024-04-24T08:57:56.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.193+0000 7fbe2e043740 1 --2- 172.21.15.83:0/4080350697 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x559e50a2ecd0 0x559e50a31140 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:56.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.193+0000 7fbe2e043740 1 --2- 172.21.15.83:0/4080350697 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e50a1eb30 0x559e50a1bfb0 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:56.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.193+0000 7fbe2e043740 1 --2- 172.21.15.83:0/4080350697 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e50a14460 0x559e50a1b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:56.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.193+0000 7fbe2e043740 1 --2- 172.21.15.83:0/4080350697 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e50a13b40 0x559e50a1e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:56.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.193+0000 7fbe2e043740 1 -- 172.21.15.83:0/4080350697 >> 172.21.15.83:0/4080350697 conn(0x559e50a0fe60 msgr2=0x559e50a15c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:56.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.193+0000 7fbe0e000640 0 client.17934 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:57:56.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.193+0000 7fbe2e043740 1 -- 172.21.15.83:0/4080350697 shutdown_connections 2024-04-24T08:57:56.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.193+0000 7fbe2e043740 1 -- 172.21.15.83:0/4080350697 wait complete. 2024-04-24T08:57:56.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.193+0000 7fbe2e043740 20 client.17934 trim_cache size 0 max 16384 2024-04-24T08:57:56.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.210+0000 7fbe2e043740 1 Processor -- start 2024-04-24T08:57:56.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.210+0000 7fbe2e043740 1 -- start start 2024-04-24T08:57:56.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.211+0000 7fbe2e043740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e50a13b40 0x559e50a2a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:56.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.211+0000 7fbe2e043740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e50a14460 0x559e50a0ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:56.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.211+0000 7fbe2e043740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e50a1eb30 0x559e50a1a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:56.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.211+0000 7fbe2e043740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbe241b1500 con 0x559e50a14460 2024-04-24T08:57:56.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.211+0000 7fbe2e043740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbe241b3040 con 0x559e50a13b40 2024-04-24T08:57:56.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.211+0000 7fbe2e043740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fbe241b1bd0 con 0x559e50a1eb30 2024-04-24T08:57:56.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.211+0000 7fbe23400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e50a14460 0x559e50a0ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:56.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.211+0000 7fbe23400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e50a14460 0x559e50a0ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41856/0 (socket says 172.21.15.83:41856) 2024-04-24T08:57:56.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.211+0000 7fbe23400640 1 -- 172.21.15.83:0/4264318937 learned_addr learned my addr 172.21.15.83:0/4264318937 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:57:56.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.211+0000 7fbe29200640 1 --2- 172.21.15.83:0/4264318937 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e50a1eb30 0x559e50a1a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:56.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.211+0000 7fbe23e00640 1 --2- 172.21.15.83:0/4264318937 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e50a13b40 0x559e50a2a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:56.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.212+0000 7fbe23400640 1 -- 172.21.15.83:0/4264318937 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e50a1eb30 msgr2=0x559e50a1a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:56.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.212+0000 7fbe23400640 1 --2- 172.21.15.83:0/4264318937 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e50a1eb30 0x559e50a1a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:56.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.212+0000 7fbe23400640 1 -- 172.21.15.83:0/4264318937 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e50a13b40 msgr2=0x559e50a2a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:56.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.212+0000 7fbe23400640 1 --2- 172.21.15.83:0/4264318937 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e50a13b40 0x559e50a2a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:56.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.212+0000 7fbe23400640 1 -- 172.21.15.83:0/4264318937 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e507557d0 con 0x559e50a14460 2024-04-24T08:57:56.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.212+0000 7fbe23400640 1 --2- 172.21.15.83:0/4264318937 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e50a14460 0x559e50a0ade0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7fbe10098d80 tx=0x7fbe100d6b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d378f0bf042abd9e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:56.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.212+0000 7fbe18c00640 1 -- 172.21.15.83:0/4264318937 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe100df070 con 0x559e50a14460 2024-04-24T08:57:56.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.212+0000 7fbe18c00640 1 -- 172.21.15.83:0/4264318937 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbe100b0e50 con 0x559e50a14460 2024-04-24T08:57:56.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.212+0000 7fbe18c00640 1 -- 172.21.15.83:0/4264318937 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe1001d040 con 0x559e50a14460 2024-04-24T08:57:56.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.213+0000 7fbe2e043740 1 -- 172.21.15.83:0/4264318937 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e50a14460 msgr2=0x559e50a0ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:56.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.213+0000 7fbe2e043740 1 --2- 172.21.15.83:0/4264318937 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e50a14460 0x559e50a0ade0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7fbe10098d80 tx=0x7fbe100d6b80 comp rx=0 tx=0).stop 2024-04-24T08:57:56.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.213+0000 7fbe2e043740 1 -- 172.21.15.83:0/4264318937 shutdown_connections 2024-04-24T08:57:56.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.213+0000 7fbe2e043740 1 --2- 172.21.15.83:0/4264318937 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e50a1eb30 0x559e50a1a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:56.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.213+0000 7fbe2e043740 1 --2- 172.21.15.83:0/4264318937 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e50a14460 0x559e50a0ade0 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:56.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.213+0000 7fbe2e043740 1 --2- 172.21.15.83:0/4264318937 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e50a13b40 0x559e50a2a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:56.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.213+0000 7fbe2e043740 1 -- 172.21.15.83:0/4264318937 >> 172.21.15.83:0/4264318937 conn(0x559e50a0fe60 msgr2=0x559e50a0e6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:56.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.213+0000 7fbe2e043740 1 -- 172.21.15.83:0/4264318937 shutdown_connections 2024-04-24T08:57:56.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.213+0000 7fbe2e043740 1 -- 172.21.15.83:0/4264318937 wait complete. 2024-04-24T08:57:56.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.214+0000 7fbe2e043740 1 -- start start 2024-04-24T08:57:56.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.214+0000 7fbe2e043740 1 Processor -- start 2024-04-24T08:57:56.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.214+0000 7fbe2e043740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e50a13b40 0x559e50a27330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:56.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.214+0000 7fbe2e043740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e50a14460 0x559e50a18f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:56.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.214+0000 7fbe2e043740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e50a1eb30 0x559e50a1fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:56.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.214+0000 7fbe2e043740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbe241b2970 con 0x559e50a13b40 2024-04-24T08:57:56.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.214+0000 7fbe2e043740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbe241b22a0 con 0x559e50a1eb30 2024-04-24T08:57:56.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.214+0000 7fbe2e043740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fbe241af4e0 con 0x559e50a14460 2024-04-24T08:57:56.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.214+0000 7fbe2e043740 10 client.0 _ll_register_callbacks cb 0x7fbe1c009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:57:56.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.214+0000 7fbe23e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e50a13b40 0x559e50a27330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:56.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.215+0000 7fbe23e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e50a13b40 0x559e50a27330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41860/0 (socket says 172.21.15.83:41860) 2024-04-24T08:57:56.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.215+0000 7fbe23e00640 1 -- 172.21.15.83:0/1676148401 learned_addr learned my addr 172.21.15.83:0/1676148401 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:57:56.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.215+0000 7fbe23400640 1 --2- 172.21.15.83:0/1676148401 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e50a14460 0x559e50a18f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:56.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.215+0000 7fbe29200640 1 --2- 172.21.15.83:0/1676148401 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e50a1eb30 0x559e50a1fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:56.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.215+0000 7fbe23e00640 1 -- 172.21.15.83:0/1676148401 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e50a14460 msgr2=0x559e50a18f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:56.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.215+0000 7fbe23e00640 1 --2- 172.21.15.83:0/1676148401 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e50a14460 0x559e50a18f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:56.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.215+0000 7fbe23e00640 1 -- 172.21.15.83:0/1676148401 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e50a1eb30 msgr2=0x559e50a1fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:56.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.215+0000 7fbe23e00640 1 --2- 172.21.15.83:0/1676148401 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e50a1eb30 0x559e50a1fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:56.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.215+0000 7fbe23e00640 1 -- 172.21.15.83:0/1676148401 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e50a19910 con 0x559e50a13b40 2024-04-24T08:57:56.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.215+0000 7fbe23e00640 1 --2- 172.21.15.83:0/1676148401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e50a13b40 0x559e50a27330 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x559e50a319b0 tx=0x7fbe140045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85036e4ab4705504 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:56.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.215+0000 7fbe0e000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:57:56.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.215+0000 7fbe0e000640 1 -- 172.21.15.83:0/1676148401 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe14005020 con 0x559e50a13b40 2024-04-24T08:57:56.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.215+0000 7fbe0e000640 1 -- 172.21.15.83:0/1676148401 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7fbe140051c0 con 0x559e50a13b40 2024-04-24T08:57:56.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.216+0000 7fbe2e043740 1 -- 172.21.15.83:0/1676148401 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe100b0e50 con 0x559e50a13b40 2024-04-24T08:57:56.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.216+0000 7fbe0e000640 1 -- 172.21.15.83:0/1676148401 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fbe1402f050 con 0x559e50a13b40 2024-04-24T08:57:56.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.216+0000 7fbe2e043740 10 client.17940 fetch_fsmap learned FSMap version 33 2024-04-24T08:57:56.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.216+0000 7fbe2e043740 1 -- 172.21.15.83:0/1676148401 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x559e50a95880 con 0x559e50a13b40 2024-04-24T08:57:56.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.216+0000 7fbe0e000640 1 -- 172.21.15.83:0/1676148401 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbe100b0e50 con 0x559e50a13b40 2024-04-24T08:57:56.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.216+0000 7fbe0e000640 1 -- 172.21.15.83:0/1676148401 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fbe1401f200 con 0x559e50a13b40 2024-04-24T08:57:56.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.216+0000 7fbe2e043740 10 client.17940 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T08:57:56.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.216+0000 7fbe2e043740 10 client.17940 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:57:56.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.216+0000 7fbe2e043740 20 client.17940 populate_metadata read hostname 'smithi083' 2024-04-24T08:57:56.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.216+0000 7fbe2e043740 1 --2- 172.21.15.83:0/1676148401 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x559e50a95fb0 0x559e50a983d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:56.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.216+0000 7fbe2e043740 4 client.17940 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T08:57:56.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.216+0000 7fbe2e043740 1 -- 172.21.15.83:0/1676148401 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559e50980a60 con 0x559e50a95fb0 2024-04-24T08:57:56.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.217+0000 7fbe23400640 1 --2- 172.21.15.83:0/1676148401 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x559e50a95fb0 0x559e50a983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:56.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.218+0000 7fbe23400640 1 --2- 172.21.15.83:0/1676148401 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x559e50a95fb0 0x559e50a983d0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf2def2bc4ad7803 server_cookie=7bf0cc1d0c0f9ec2 in_seq=0 out_seq=0 2024-04-24T08:57:56.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.218+0000 7fbe0e000640 10 client.17940 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:57:56.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.218+0000 7fbe0e000640 1 -- 172.21.15.83:0/1676148401 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559e50980a60 con 0x559e50a95fb0 2024-04-24T08:57:56.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.218+0000 7fbe0e000640 10 client.17940 handle_command_reply: tid=0 2024-04-24T08:57:56.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.218+0000 7fbe2e043740 1 client.17940 shutdown 2024-04-24T08:57:56.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.218+0000 7fbe2e043740 1 -- 172.21.15.83:0/1676148401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e50a13b40 msgr2=0x559e50a27330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:56.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.218+0000 7fbe2e043740 1 --2- 172.21.15.83:0/1676148401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e50a13b40 0x559e50a27330 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x559e50a319b0 tx=0x7fbe140045e0 comp rx=0 tx=0).stop 2024-04-24T08:57:56.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.219+0000 7fbe2e043740 1 -- 172.21.15.83:0/1676148401 shutdown_connections 2024-04-24T08:57:56.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.219+0000 7fbe2e043740 1 --2- 172.21.15.83:0/1676148401 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x559e50a95fb0 0x559e50a983d0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:56.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.219+0000 7fbe2e043740 1 --2- 172.21.15.83:0/1676148401 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e50a1eb30 0x559e50a1fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:56.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.219+0000 7fbe2e043740 1 --2- 172.21.15.83:0/1676148401 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e50a14460 0x559e50a18f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:56.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.219+0000 7fbe2e043740 1 --2- 172.21.15.83:0/1676148401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e50a13b40 0x559e50a27330 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:56.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.219+0000 7fbe0e000640 0 client.17940 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:57:56.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.219+0000 7fbe2e043740 1 -- 172.21.15.83:0/1676148401 >> 172.21.15.83:0/1676148401 conn(0x559e50a0fe60 msgr2=0x559e50a0e6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:56.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.219+0000 7fbe2e043740 1 -- 172.21.15.83:0/1676148401 shutdown_connections 2024-04-24T08:57:56.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.219+0000 7fbe2e043740 1 -- 172.21.15.83:0/1676148401 wait complete. 2024-04-24T08:57:56.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.219+0000 7fbe2e043740 20 client.17940 trim_cache size 0 max 16384 2024-04-24T08:57:56.222 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T08:57:56.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.220+0000 7fbe29c00640 1 -- 172.21.15.83:0/2461098367 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fbe080a3da0 msgr2=0x7fbe080a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:56.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.220+0000 7fbe29c00640 1 --2- 172.21.15.83:0/2461098367 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fbe080a3da0 0x7fbe080a6260 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c004af0 tx=0x7fbe1c004a40 comp rx=0 tx=0).stop 2024-04-24T08:57:56.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.220+0000 7fbe29c00640 1 -- 172.21.15.83:0/2461098367 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbe24074cd0 msgr2=0x7fbe241a0b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:56.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.220+0000 7fbe29c00640 1 --2- 172.21.15.83:0/2461098367 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbe24074cd0 0x7fbe241a0b10 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7fbe1000c890 tx=0x7fbe1000cd50 comp rx=0 tx=0).stop 2024-04-24T08:57:56.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.220+0000 7fbe29c00640 1 -- 172.21.15.83:0/2461098367 shutdown_connections 2024-04-24T08:57:56.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.220+0000 7fbe29c00640 1 --2- 172.21.15.83:0/2461098367 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fbe080a3da0 0x7fbe080a6260 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:56.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.220+0000 7fbe29c00640 1 --2- 172.21.15.83:0/2461098367 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbe2410e760 0x7fbe2419a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:56.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.220+0000 7fbe29c00640 1 --2- 172.21.15.83:0/2461098367 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbe24074cd0 0x7fbe241a0b10 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:56.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.220+0000 7fbe29c00640 1 --2- 172.21.15.83:0/2461098367 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe240743b0 0x7fbe241a05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:56.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.220+0000 7fbe29c00640 1 -- 172.21.15.83:0/2461098367 >> 172.21.15.83:0/2461098367 conn(0x7fbe240fd9d0 msgr2=0x7fbe240ff200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:56.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.220+0000 7fbe29c00640 1 -- 172.21.15.83:0/2461098367 shutdown_connections 2024-04-24T08:57:56.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:56.220+0000 7fbe29c00640 1 -- 172.21.15.83:0/2461098367 wait complete. 2024-04-24T08:57:56.583 INFO:tasks.workunit.client.0.smithi083.stdout: 1 32554 0.64 MB/sec execute 150 sec latency 37.776 ms 2024-04-24T08:57:56.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:57:56 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:57:56.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:57:56 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:57:56.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:57:56 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T08:57:56.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:57:56 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:57:56.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:57:56 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:57:56.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:57:56 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T08:57:57.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:57:56 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:57:57.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:57:56 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:57:57.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:57:56 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T08:57:57.240 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T08:57:57.240 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T08:57:57.240 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T08:57:57.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.472+0000 7ff640c00640 1 -- 172.21.15.83:0/2434385073 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff63c102be0 msgr2=0x7ff63c106c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:57.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.472+0000 7ff640c00640 1 --2- 172.21.15.83:0/2434385073 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff63c102be0 0x7ff63c106c20 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7ff628009900 tx=0x7ff62802a410 comp rx=0 tx=0).stop 2024-04-24T08:57:57.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.472+0000 7ff640c00640 1 -- 172.21.15.83:0/2434385073 shutdown_connections 2024-04-24T08:57:57.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.472+0000 7ff640c00640 1 --2- 172.21.15.83:0/2434385073 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff63c107300 0x7ff63c10eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:57.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.472+0000 7ff640c00640 1 --2- 172.21.15.83:0/2434385073 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff63c102be0 0x7ff63c106c20 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:57.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.472+0000 7ff640c00640 1 --2- 172.21.15.83:0/2434385073 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff63c102230 0x7ff63c102610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:57.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.472+0000 7ff640c00640 1 -- 172.21.15.83:0/2434385073 >> 172.21.15.83:0/2434385073 conn(0x7ff63c0fda70 msgr2=0x7ff63c0ffe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:57.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.472+0000 7ff640c00640 1 -- 172.21.15.83:0/2434385073 shutdown_connections 2024-04-24T08:57:57.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.472+0000 7ff640c00640 1 -- 172.21.15.83:0/2434385073 wait complete. 2024-04-24T08:57:57.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.473+0000 7ff640c00640 1 Processor -- start 2024-04-24T08:57:57.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.473+0000 7ff640c00640 1 -- start start 2024-04-24T08:57:57.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.474+0000 7ff640c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff63c102230 0x7ff63c074300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:57.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.474+0000 7ff640c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff63c107300 0x7ff63c074840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:57.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.474+0000 7ff640c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff63c074d80 0x7ff63c1a69d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:57.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.474+0000 7ff640c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff63c112240 con 0x7ff63c074d80 2024-04-24T08:57:57.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.474+0000 7ff640c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff63c1120c0 con 0x7ff63c102230 2024-04-24T08:57:57.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.474+0000 7ff640c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff63c1123c0 con 0x7ff63c107300 2024-04-24T08:57:57.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.474+0000 7ff63a800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff63c074d80 0x7ff63c1a69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:57.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.474+0000 7ff63a800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff63c074d80 0x7ff63c1a69d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41866/0 (socket says 172.21.15.83:41866) 2024-04-24T08:57:57.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.474+0000 7ff63a800640 1 -- 172.21.15.83:0/2306606656 learned_addr learned my addr 172.21.15.83:0/2306606656 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:57:57.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.474+0000 7ff639e00640 1 --2- 172.21.15.83:0/2306606656 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff63c102230 0x7ff63c074300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:57.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.474+0000 7ff639400640 1 --2- 172.21.15.83:0/2306606656 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff63c107300 0x7ff63c074840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:57.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.475+0000 7ff63a800640 1 -- 172.21.15.83:0/2306606656 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff63c107300 msgr2=0x7ff63c074840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:57.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.475+0000 7ff63a800640 1 --2- 172.21.15.83:0/2306606656 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff63c107300 0x7ff63c074840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:57.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.475+0000 7ff63a800640 1 -- 172.21.15.83:0/2306606656 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff63c102230 msgr2=0x7ff63c074300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:57.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.475+0000 7ff63a800640 1 --2- 172.21.15.83:0/2306606656 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff63c102230 0x7ff63c074300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:57.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.475+0000 7ff63a800640 1 -- 172.21.15.83:0/2306606656 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff63c1a70d0 con 0x7ff63c074d80 2024-04-24T08:57:57.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.475+0000 7ff639e00640 1 --2- 172.21.15.83:0/2306606656 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff63c102230 0x7ff63c074300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:57:57.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.475+0000 7ff63a800640 1 --2- 172.21.15.83:0/2306606656 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff63c074d80 0x7ff63c1a69d0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7ff63400eb10 tx=0x7ff63400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:57.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.475+0000 7ff632a00640 1 -- 172.21.15.83:0/2306606656 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff634004b20 con 0x7ff63c074d80 2024-04-24T08:57:57.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.475+0000 7ff640c00640 1 -- 172.21.15.83:0/2306606656 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff63c1a73c0 con 0x7ff63c074d80 2024-04-24T08:57:57.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.475+0000 7ff632a00640 1 -- 172.21.15.83:0/2306606656 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff634004cc0 con 0x7ff63c074d80 2024-04-24T08:57:57.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.475+0000 7ff632a00640 1 -- 172.21.15.83:0/2306606656 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff634010690 con 0x7ff63c074d80 2024-04-24T08:57:57.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.475+0000 7ff640c00640 1 -- 172.21.15.83:0/2306606656 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff63c1a7930 con 0x7ff63c074d80 2024-04-24T08:57:57.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.478+0000 7ff643b63740 1 Processor -- start 2024-04-24T08:57:57.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.479+0000 7ff632a00640 1 -- 172.21.15.83:0/2306606656 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff634005020 con 0x7ff63c074d80 2024-04-24T08:57:57.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.479+0000 7ff643b63740 1 -- start start 2024-04-24T08:57:57.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.479+0000 7ff643b63740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c8f105090 0x555c8f105470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:57.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.479+0000 7ff643b63740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c8f1059b0 0x555c8f10fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:57.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.479+0000 7ff643b63740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555c8f110080 0x555c8f112470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:57.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.479+0000 7ff643b63740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x555c8f06ae70 con 0x555c8f105090 2024-04-24T08:57:57.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.479+0000 7ff643b63740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555c8f011380 con 0x555c8f1059b0 2024-04-24T08:57:57.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.479+0000 7ff643b63740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x555c8f066e30 con 0x555c8f110080 2024-04-24T08:57:57.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.479+0000 7ff639400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c8f105090 0x555c8f105470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:57.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.479+0000 7ff639400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c8f105090 0x555c8f105470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41868/0 (socket says 172.21.15.83:41868) 2024-04-24T08:57:57.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.479+0000 7ff639400640 1 -- 172.21.15.83:0/3203617354 learned_addr learned my addr 172.21.15.83:0/3203617354 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:57:57.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.480+0000 7ff639e00640 1 --2- 172.21.15.83:0/3203617354 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c8f1059b0 0x555c8f10fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:57.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.480+0000 7ff639400640 1 -- 172.21.15.83:0/3203617354 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555c8f110080 msgr2=0x555c8f112470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:57.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.480+0000 7ff639400640 1 --2- 172.21.15.83:0/3203617354 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555c8f110080 0x555c8f112470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:57.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.480+0000 7ff639400640 1 -- 172.21.15.83:0/3203617354 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c8f1059b0 msgr2=0x555c8f10fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:57.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.480+0000 7ff639400640 1 --2- 172.21.15.83:0/3203617354 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c8f1059b0 0x555c8f10fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:57.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.480+0000 7ff639400640 1 -- 172.21.15.83:0/3203617354 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c8ee457e0 con 0x555c8f105090 2024-04-24T08:57:57.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.480+0000 7ff639e00640 1 --2- 172.21.15.83:0/3203617354 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c8f1059b0 0x555c8f10fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:57:57.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.480+0000 7ff632a00640 1 --2- 172.21.15.83:0/2306606656 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff6200a3dd0 0x7ff6200a6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:57.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.480+0000 7ff639400640 1 --2- 172.21.15.83:0/3203617354 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c8f105090 0x555c8f105470 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7ff628031440 tx=0x7ff628032c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=833e1c513873f494 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:57.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.481+0000 7ff632a00640 1 -- 172.21.15.83:0/2306606656 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7ff6340d1190 con 0x7ff63c074d80 2024-04-24T08:57:57.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.481+0000 7ff627000640 1 -- 172.21.15.83:0/3203617354 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff628035050 con 0x555c8f105090 2024-04-24T08:57:57.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.481+0000 7ff639e00640 1 --2- 172.21.15.83:0/2306606656 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff6200a3dd0 0x7ff6200a6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:57.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.481+0000 7ff627000640 1 -- 172.21.15.83:0/3203617354 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff628004030 con 0x555c8f105090 2024-04-24T08:57:57.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.481+0000 7ff627000640 1 -- 172.21.15.83:0/3203617354 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6280054c0 con 0x555c8f105090 2024-04-24T08:57:57.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.482+0000 7ff643b63740 1 -- 172.21.15.83:0/3203617354 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c8f105090 msgr2=0x555c8f105470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:57.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.482+0000 7ff639e00640 1 --2- 172.21.15.83:0/2306606656 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff6200a3dd0 0x7ff6200a6290 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff62c005d00 tx=0x7ff62c00b2e0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:57.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.482+0000 7ff643b63740 1 --2- 172.21.15.83:0/3203617354 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c8f105090 0x555c8f105470 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7ff628031440 tx=0x7ff628032c00 comp rx=0 tx=0).stop 2024-04-24T08:57:57.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.482+0000 7ff643b63740 1 -- 172.21.15.83:0/3203617354 shutdown_connections 2024-04-24T08:57:57.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.482+0000 7ff643b63740 1 --2- 172.21.15.83:0/3203617354 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555c8f110080 0x555c8f112470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:57.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.482+0000 7ff643b63740 1 --2- 172.21.15.83:0/3203617354 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c8f1059b0 0x555c8f10fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:57.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.482+0000 7ff643b63740 1 --2- 172.21.15.83:0/3203617354 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c8f105090 0x555c8f105470 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:57.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.482+0000 7ff643b63740 1 -- 172.21.15.83:0/3203617354 >> 172.21.15.83:0/3203617354 conn(0x555c8f1013b0 msgr2=0x555c8f101790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:57.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.482+0000 7ff643b63740 1 -- 172.21.15.83:0/3203617354 shutdown_connections 2024-04-24T08:57:57.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.482+0000 7ff643b63740 1 -- 172.21.15.83:0/3203617354 wait complete. 2024-04-24T08:57:57.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.482+0000 7ff643b63740 1 -- start start 2024-04-24T08:57:57.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.483+0000 7ff643b63740 1 Processor -- start 2024-04-24T08:57:57.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.483+0000 7ff643b63740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555c8f105090 0x555c8f10d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:57.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.483+0000 7ff643b63740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c8f1059b0 0x555c8f10d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:57.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.483+0000 7ff643b63740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c8f110080 0x555c8f10dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:57.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.483+0000 7ff643b63740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x555c8f112e10 con 0x555c8f1059b0 2024-04-24T08:57:57.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.483+0000 7ff643b63740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555c8f024660 con 0x555c8f110080 2024-04-24T08:57:57.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.483+0000 7ff643b63740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x555c8f112f90 con 0x555c8f105090 2024-04-24T08:57:57.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.483+0000 7ff643b63740 10 client.0 _ll_register_callbacks cb 0x555c8f012aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:57:57.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.483+0000 7ff639400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c8f1059b0 0x555c8f10d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:57.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.484+0000 7ff639400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c8f1059b0 0x555c8f10d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41884/0 (socket says 172.21.15.83:41884) 2024-04-24T08:57:57.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.484+0000 7ff639400640 1 -- 172.21.15.83:0/3585293882 learned_addr learned my addr 172.21.15.83:0/3585293882 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:57:57.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.484+0000 7ff639e00640 1 --2- 172.21.15.83:0/3585293882 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555c8f105090 0x555c8f10d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:57.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.484+0000 7ff63a800640 1 --2- 172.21.15.83:0/3585293882 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c8f110080 0x555c8f10dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:57.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.484+0000 7ff639400640 1 -- 172.21.15.83:0/3585293882 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555c8f105090 msgr2=0x555c8f10d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:57.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.484+0000 7ff639400640 1 --2- 172.21.15.83:0/3585293882 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555c8f105090 0x555c8f10d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:57.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.484+0000 7ff639400640 1 -- 172.21.15.83:0/3585293882 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c8f110080 msgr2=0x555c8f10dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:57.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.484+0000 7ff639400640 1 --2- 172.21.15.83:0/3585293882 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c8f110080 0x555c8f10dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:57.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.484+0000 7ff639400640 1 -- 172.21.15.83:0/3585293882 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c8eea5f70 con 0x555c8f1059b0 2024-04-24T08:57:57.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.484+0000 7ff639400640 1 --2- 172.21.15.83:0/3585293882 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c8f1059b0 0x555c8f10d580 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7ff628005310 tx=0x7ff62802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f15db46bb86cf34a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:57.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.484+0000 7ff61b400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:57:57.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.485+0000 7ff61b400640 1 -- 172.21.15.83:0/3585293882 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff628035040 con 0x555c8f1059b0 2024-04-24T08:57:57.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.485+0000 7ff61b400640 1 -- 172.21.15.83:0/3585293882 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7ff6280316a0 con 0x555c8f1059b0 2024-04-24T08:57:57.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.485+0000 7ff643b63740 1 -- 172.21.15.83:0/3585293882 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff628004030 con 0x555c8f1059b0 2024-04-24T08:57:57.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.485+0000 7ff61b400640 1 -- 172.21.15.83:0/3585293882 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7ff628057050 con 0x555c8f1059b0 2024-04-24T08:57:57.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.485+0000 7ff643b63740 10 client.17952 fetch_fsmap learned FSMap version 33 2024-04-24T08:57:57.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.485+0000 7ff643b63740 1 -- 172.21.15.83:0/3585293882 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x555c8ed3d170 con 0x555c8f1059b0 2024-04-24T08:57:57.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.485+0000 7ff61b400640 1 -- 172.21.15.83:0/3585293882 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff628004030 con 0x555c8f1059b0 2024-04-24T08:57:57.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.486+0000 7ff61b400640 1 -- 172.21.15.83:0/3585293882 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7ff628003850 con 0x555c8f1059b0 2024-04-24T08:57:57.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.486+0000 7ff643b63740 10 client.17952 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T08:57:57.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.486+0000 7ff643b63740 10 client.17952 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:57:57.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.486+0000 7ff643b63740 20 client.17952 populate_metadata read hostname 'smithi083' 2024-04-24T08:57:57.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.486+0000 7ff643b63740 1 --2- 172.21.15.83:0/3585293882 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x555c8f1201d0 0x555c8f1225c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:57.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.486+0000 7ff643b63740 4 client.17952 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:57:57.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.486+0000 7ff643b63740 1 -- 172.21.15.83:0/3585293882 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555c8f078890 con 0x555c8f1201d0 2024-04-24T08:57:57.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.486+0000 7ff639e00640 1 --2- 172.21.15.83:0/3585293882 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x555c8f1201d0 0x555c8f1225c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:57.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.488+0000 7ff639e00640 1 --2- 172.21.15.83:0/3585293882 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x555c8f1201d0 0x555c8f1225c0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6c9ab4b68f6e531 server_cookie=f06d2365d8ad363d in_seq=0 out_seq=0 2024-04-24T08:57:57.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.488+0000 7ff61b400640 10 client.17952 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:57:57.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.490+0000 7ff61b400640 1 -- 172.21.15.83:0/3585293882 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555c8f078890 con 0x555c8f1201d0 2024-04-24T08:57:57.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.490+0000 7ff61b400640 10 client.17952 handle_command_reply: tid=0 2024-04-24T08:57:57.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.491+0000 7ff643b63740 1 client.17952 shutdown 2024-04-24T08:57:57.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.491+0000 7ff643b63740 1 -- 172.21.15.83:0/3585293882 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c8f1059b0 msgr2=0x555c8f10d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:57.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.491+0000 7ff643b63740 1 --2- 172.21.15.83:0/3585293882 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c8f1059b0 0x555c8f10d580 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7ff628005310 tx=0x7ff62802a410 comp rx=0 tx=0).stop 2024-04-24T08:57:57.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.491+0000 7ff643b63740 1 -- 172.21.15.83:0/3585293882 shutdown_connections 2024-04-24T08:57:57.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.491+0000 7ff643b63740 1 --2- 172.21.15.83:0/3585293882 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x555c8f1201d0 0x555c8f1225c0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:57.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.491+0000 7ff643b63740 1 --2- 172.21.15.83:0/3585293882 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c8f110080 0x555c8f10dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:57.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.491+0000 7ff643b63740 1 --2- 172.21.15.83:0/3585293882 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c8f1059b0 0x555c8f10d580 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:57.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.491+0000 7ff61b400640 0 client.17952 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:57:57.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.491+0000 7ff643b63740 1 --2- 172.21.15.83:0/3585293882 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555c8f105090 0x555c8f10d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:57.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.491+0000 7ff643b63740 1 -- 172.21.15.83:0/3585293882 >> 172.21.15.83:0/3585293882 conn(0x555c8f1013b0 msgr2=0x555c8f1070b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:57.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.491+0000 7ff643b63740 1 -- 172.21.15.83:0/3585293882 shutdown_connections 2024-04-24T08:57:57.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.491+0000 7ff643b63740 1 -- 172.21.15.83:0/3585293882 wait complete. 2024-04-24T08:57:57.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.491+0000 7ff643b63740 20 client.17952 trim_cache size 0 max 16384 2024-04-24T08:57:57.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.512+0000 7ff643b63740 1 Processor -- start 2024-04-24T08:57:57.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.513+0000 7ff643b63740 1 -- start start 2024-04-24T08:57:57.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.513+0000 7ff643b63740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c8f105090 0x555c8f1255e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:57.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.513+0000 7ff643b63740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c8f1059b0 0x555c8f125b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:57.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.513+0000 7ff643b63740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555c8f110080 0x555c8f10a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:57.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.513+0000 7ff643b63740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff63c1b16d0 con 0x555c8f105090 2024-04-24T08:57:57.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.513+0000 7ff643b63740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff63c1b3210 con 0x555c8f1059b0 2024-04-24T08:57:57.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.513+0000 7ff643b63740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff63c1b1da0 con 0x555c8f110080 2024-04-24T08:57:57.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.513+0000 7ff639e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c8f105090 0x555c8f1255e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:57.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.514+0000 7ff639e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c8f105090 0x555c8f1255e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41898/0 (socket says 172.21.15.83:41898) 2024-04-24T08:57:57.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.514+0000 7ff639e00640 1 -- 172.21.15.83:0/1300678181 learned_addr learned my addr 172.21.15.83:0/1300678181 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:57:57.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.514+0000 7ff63a800640 1 --2- 172.21.15.83:0/1300678181 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555c8f110080 0x555c8f10a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:57.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.514+0000 7ff639400640 1 --2- 172.21.15.83:0/1300678181 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c8f1059b0 0x555c8f125b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:57.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.514+0000 7ff639e00640 1 -- 172.21.15.83:0/1300678181 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555c8f110080 msgr2=0x555c8f10a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:57.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.514+0000 7ff639e00640 1 --2- 172.21.15.83:0/1300678181 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555c8f110080 0x555c8f10a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:57.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.514+0000 7ff639e00640 1 -- 172.21.15.83:0/1300678181 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c8f1059b0 msgr2=0x555c8f125b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:57.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.514+0000 7ff639e00640 1 --2- 172.21.15.83:0/1300678181 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c8f1059b0 0x555c8f125b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:57.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.514+0000 7ff639e00640 1 -- 172.21.15.83:0/1300678181 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c8f108990 con 0x555c8f105090 2024-04-24T08:57:57.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.514+0000 7ff639e00640 1 --2- 172.21.15.83:0/1300678181 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c8f105090 0x555c8f1255e0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x555c8f06af80 tx=0x7ff62c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bdf4aa951c2bced server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:57.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.515+0000 7ff626600640 1 -- 172.21.15.83:0/1300678181 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff62c0285a0 con 0x555c8f105090 2024-04-24T08:57:57.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.515+0000 7ff626600640 1 -- 172.21.15.83:0/1300678181 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff62c02e070 con 0x555c8f105090 2024-04-24T08:57:57.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.515+0000 7ff626600640 1 -- 172.21.15.83:0/1300678181 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff62c01e450 con 0x555c8f105090 2024-04-24T08:57:57.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.516+0000 7ff643b63740 1 -- 172.21.15.83:0/1300678181 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c8f105090 msgr2=0x555c8f1255e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:57.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.516+0000 7ff643b63740 1 --2- 172.21.15.83:0/1300678181 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c8f105090 0x555c8f1255e0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x555c8f06af80 tx=0x7ff62c013000 comp rx=0 tx=0).stop 2024-04-24T08:57:57.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.516+0000 7ff643b63740 1 -- 172.21.15.83:0/1300678181 shutdown_connections 2024-04-24T08:57:57.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.516+0000 7ff643b63740 1 --2- 172.21.15.83:0/1300678181 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555c8f110080 0x555c8f10a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:57.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.516+0000 7ff643b63740 1 --2- 172.21.15.83:0/1300678181 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c8f1059b0 0x555c8f125b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:57.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.516+0000 7ff643b63740 1 --2- 172.21.15.83:0/1300678181 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c8f105090 0x555c8f1255e0 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:57.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.516+0000 7ff643b63740 1 -- 172.21.15.83:0/1300678181 >> 172.21.15.83:0/1300678181 conn(0x555c8f1013b0 msgr2=0x555c8f10f100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:57.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.516+0000 7ff643b63740 1 -- 172.21.15.83:0/1300678181 shutdown_connections 2024-04-24T08:57:57.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.516+0000 7ff643b63740 1 -- 172.21.15.83:0/1300678181 wait complete. 2024-04-24T08:57:57.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.516+0000 7ff643b63740 1 -- start start 2024-04-24T08:57:57.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.517+0000 7ff643b63740 1 Processor -- start 2024-04-24T08:57:57.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.517+0000 7ff643b63740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555c8f105090 0x555c8f1230f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:57.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.517+0000 7ff643b63740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c8f1059b0 0x555c8f123630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:57.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.517+0000 7ff643b63740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c8f110080 0x555c8f123b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:57.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.517+0000 7ff643b63740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff63c1b2b40 con 0x555c8f1059b0 2024-04-24T08:57:57.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.517+0000 7ff643b63740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff63c1b2470 con 0x555c8f110080 2024-04-24T08:57:57.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.517+0000 7ff643b63740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff63c1af6b0 con 0x555c8f105090 2024-04-24T08:57:57.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.517+0000 7ff643b63740 10 client.0 _ll_register_callbacks cb 0x7ff628009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:57:57.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.517+0000 7ff639400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c8f1059b0 0x555c8f123630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:57.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.517+0000 7ff639400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c8f1059b0 0x555c8f123630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41902/0 (socket says 172.21.15.83:41902) 2024-04-24T08:57:57.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.517+0000 7ff639400640 1 -- 172.21.15.83:0/1058543692 learned_addr learned my addr 172.21.15.83:0/1058543692 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:57:57.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.518+0000 7ff63a800640 1 --2- 172.21.15.83:0/1058543692 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c8f110080 0x555c8f123b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:57.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.518+0000 7ff639400640 1 -- 172.21.15.83:0/1058543692 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555c8f105090 msgr2=0x555c8f1230f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:57.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.518+0000 7ff639400640 1 --2- 172.21.15.83:0/1058543692 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555c8f105090 0x555c8f1230f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:57.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.518+0000 7ff639400640 1 -- 172.21.15.83:0/1058543692 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c8f110080 msgr2=0x555c8f123b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:57.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.518+0000 7ff639400640 1 --2- 172.21.15.83:0/1058543692 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c8f110080 0x555c8f123b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:57.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.518+0000 7ff639400640 1 -- 172.21.15.83:0/1058543692 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c8f185480 con 0x555c8f1059b0 2024-04-24T08:57:57.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.518+0000 7ff63a800640 1 --2- 172.21.15.83:0/1058543692 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c8f110080 0x555c8f123b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:57:57.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.518+0000 7ff639400640 1 --2- 172.21.15.83:0/1058543692 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c8f1059b0 0x555c8f123630 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7ff6280057c0 tx=0x7ff6280040f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4940dad4dcbb924f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:57.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.518+0000 7ff61b400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:57:57.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.518+0000 7ff61b400640 1 -- 172.21.15.83:0/1058543692 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff628035040 con 0x555c8f1059b0 2024-04-24T08:57:57.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.518+0000 7ff61b400640 1 -- 172.21.15.83:0/1058543692 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7ff628003a00 con 0x555c8f1059b0 2024-04-24T08:57:57.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.518+0000 7ff643b63740 1 -- 172.21.15.83:0/1058543692 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff62c02e070 con 0x555c8f1059b0 2024-04-24T08:57:57.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.519+0000 7ff61b400640 1 -- 172.21.15.83:0/1058543692 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7ff628032070 con 0x555c8f1059b0 2024-04-24T08:57:57.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.519+0000 7ff643b63740 10 client.17958 fetch_fsmap learned FSMap version 33 2024-04-24T08:57:57.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.519+0000 7ff643b63740 1 -- 172.21.15.83:0/1058543692 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x555c8f1867a0 con 0x555c8f1059b0 2024-04-24T08:57:57.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.519+0000 7ff61b400640 1 -- 172.21.15.83:0/1058543692 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff62c02e070 con 0x555c8f1059b0 2024-04-24T08:57:57.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.519+0000 7ff61b400640 1 -- 172.21.15.83:0/1058543692 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7ff628031800 con 0x555c8f1059b0 2024-04-24T08:57:57.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.519+0000 7ff643b63740 10 client.17958 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T08:57:57.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.519+0000 7ff643b63740 10 client.17958 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:57:57.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.519+0000 7ff643b63740 20 client.17958 populate_metadata read hostname 'smithi083' 2024-04-24T08:57:57.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.519+0000 7ff643b63740 1 --2- 172.21.15.83:0/1058543692 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x555c8f186db0 0x555c8f1891d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:57.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.519+0000 7ff643b63740 4 client.17958 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T08:57:57.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.519+0000 7ff643b63740 1 -- 172.21.15.83:0/1058543692 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x555c8f072700 con 0x555c8f186db0 2024-04-24T08:57:57.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.520+0000 7ff639e00640 1 --2- 172.21.15.83:0/1058543692 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x555c8f186db0 0x555c8f1891d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:57.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.522+0000 7ff639e00640 1 --2- 172.21.15.83:0/1058543692 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x555c8f186db0 0x555c8f1891d0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9de4ac5ba84f5419 server_cookie=37d29b62e865502d in_seq=0 out_seq=0 2024-04-24T08:57:57.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.522+0000 7ff61b400640 10 client.17958 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:57:57.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.525+0000 7ff61b400640 1 -- 172.21.15.83:0/1058543692 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x555c8f072700 con 0x555c8f186db0 2024-04-24T08:57:57.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.525+0000 7ff61b400640 10 client.17958 handle_command_reply: tid=0 2024-04-24T08:57:57.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.525+0000 7ff643b63740 1 client.17958 shutdown 2024-04-24T08:57:57.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.525+0000 7ff643b63740 1 -- 172.21.15.83:0/1058543692 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c8f1059b0 msgr2=0x555c8f123630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:57.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.525+0000 7ff643b63740 1 --2- 172.21.15.83:0/1058543692 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c8f1059b0 0x555c8f123630 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7ff6280057c0 tx=0x7ff6280040f0 comp rx=0 tx=0).stop 2024-04-24T08:57:57.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.525+0000 7ff643b63740 1 -- 172.21.15.83:0/1058543692 shutdown_connections 2024-04-24T08:57:57.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.525+0000 7ff643b63740 1 --2- 172.21.15.83:0/1058543692 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x555c8f186db0 0x555c8f1891d0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:57.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.525+0000 7ff643b63740 1 --2- 172.21.15.83:0/1058543692 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c8f110080 0x555c8f123b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:57.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.525+0000 7ff643b63740 1 --2- 172.21.15.83:0/1058543692 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c8f1059b0 0x555c8f123630 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:57.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.525+0000 7ff643b63740 1 --2- 172.21.15.83:0/1058543692 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555c8f105090 0x555c8f1230f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:57.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.525+0000 7ff643b63740 1 -- 172.21.15.83:0/1058543692 >> 172.21.15.83:0/1058543692 conn(0x555c8f1013b0 msgr2=0x555c8f10cfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:57.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.525+0000 7ff61b400640 0 client.17958 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:57:57.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.526+0000 7ff643b63740 1 -- 172.21.15.83:0/1058543692 shutdown_connections 2024-04-24T08:57:57.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.526+0000 7ff643b63740 1 -- 172.21.15.83:0/1058543692 wait complete. 2024-04-24T08:57:57.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.526+0000 7ff643b63740 20 client.17958 trim_cache size 0 max 16384 2024-04-24T08:57:57.527 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T08:57:57.527 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-04-24T08:57:57.527 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "802626c3-afa3-4589-9bfa-0992fc8ff7e2", 2024-04-24T08:57:57.527 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-04-24T08:57:57.527 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T08:57:57.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.526+0000 7ff640c00640 1 -- 172.21.15.83:0/2306606656 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff6200a3dd0 msgr2=0x7ff6200a6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:57.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.526+0000 7ff640c00640 1 --2- 172.21.15.83:0/2306606656 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff6200a3dd0 0x7ff6200a6290 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff62c005d00 tx=0x7ff62c00b2e0 comp rx=0 tx=0).stop 2024-04-24T08:57:57.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.526+0000 7ff640c00640 1 -- 172.21.15.83:0/2306606656 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff63c074d80 msgr2=0x7ff63c1a69d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:57.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.526+0000 7ff640c00640 1 --2- 172.21.15.83:0/2306606656 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff63c074d80 0x7ff63c1a69d0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7ff63400eb10 tx=0x7ff63400c2d0 comp rx=0 tx=0).stop 2024-04-24T08:57:57.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.526+0000 7ff640c00640 1 -- 172.21.15.83:0/2306606656 shutdown_connections 2024-04-24T08:57:57.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.526+0000 7ff640c00640 1 --2- 172.21.15.83:0/2306606656 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff6200a3dd0 0x7ff6200a6290 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:57.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.526+0000 7ff640c00640 1 --2- 172.21.15.83:0/2306606656 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff63c074d80 0x7ff63c1a69d0 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:57.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.526+0000 7ff640c00640 1 --2- 172.21.15.83:0/2306606656 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff63c107300 0x7ff63c074840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:57.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.526+0000 7ff640c00640 1 --2- 172.21.15.83:0/2306606656 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff63c102230 0x7ff63c074300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:57.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.526+0000 7ff640c00640 1 -- 172.21.15.83:0/2306606656 >> 172.21.15.83:0/2306606656 conn(0x7ff63c0fda70 msgr2=0x7ff63c0feec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:57.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.527+0000 7ff640c00640 1 -- 172.21.15.83:0/2306606656 shutdown_connections 2024-04-24T08:57:57.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.527+0000 7ff640c00640 1 -- 172.21.15.83:0/2306606656 wait complete. 2024-04-24T08:57:57.545 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T08:57:57.546 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T08:57:57.583 INFO:tasks.workunit.client.0.smithi083.stdout: 1 32657 0.63 MB/sec execute 151 sec latency 59.551 ms 2024-04-24T08:57:57.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.747+0000 7ff789c00640 1 -- 172.21.15.83:0/2181000076 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff784104cd0 msgr2=0x7ff784108d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:57.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.747+0000 7ff789c00640 1 --2- 172.21.15.83:0/2181000076 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff784104cd0 0x7ff784108d10 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7ff770009900 tx=0x7ff77002a410 comp rx=0 tx=0).stop 2024-04-24T08:57:57.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.748+0000 7ff789c00640 1 -- 172.21.15.83:0/2181000076 shutdown_connections 2024-04-24T08:57:57.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.748+0000 7ff789c00640 1 --2- 172.21.15.83:0/2181000076 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff7841093f0 0x7ff784110fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:57.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.748+0000 7ff789c00640 1 --2- 172.21.15.83:0/2181000076 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff784104cd0 0x7ff784108d10 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:57.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.748+0000 7ff789c00640 1 --2- 172.21.15.83:0/2181000076 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff784104320 0x7ff784104700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:57.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.748+0000 7ff789c00640 1 -- 172.21.15.83:0/2181000076 >> 172.21.15.83:0/2181000076 conn(0x7ff7840ffb40 msgr2=0x7ff784101f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:57.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.748+0000 7ff789c00640 1 -- 172.21.15.83:0/2181000076 shutdown_connections 2024-04-24T08:57:57.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.748+0000 7ff789c00640 1 -- 172.21.15.83:0/2181000076 wait complete. 2024-04-24T08:57:57.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.749+0000 7ff789c00640 1 Processor -- start 2024-04-24T08:57:57.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.749+0000 7ff789c00640 1 -- start start 2024-04-24T08:57:57.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.749+0000 7ff789c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff784104320 0x7ff7841a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:57.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.749+0000 7ff789c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7841093f0 0x7ff7841a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:57.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.749+0000 7ff789c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff78419a7a0 0x7ff78419ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:57.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.749+0000 7ff789c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff7841143e0 con 0x7ff7841093f0 2024-04-24T08:57:57.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.749+0000 7ff789c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff784114260 con 0x7ff78419a7a0 2024-04-24T08:57:57.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.749+0000 7ff789c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff784114560 con 0x7ff784104320 2024-04-24T08:57:57.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.750+0000 7ff783400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7841093f0 0x7ff7841a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:57.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.750+0000 7ff783400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7841093f0 0x7ff7841a0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41924/0 (socket says 172.21.15.83:41924) 2024-04-24T08:57:57.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.750+0000 7ff783400640 1 -- 172.21.15.83:0/2780438461 learned_addr learned my addr 172.21.15.83:0/2780438461 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:57:57.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.750+0000 7ff783e00640 1 --2- 172.21.15.83:0/2780438461 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff784104320 0x7ff7841a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:57.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.750+0000 7ff789200640 1 --2- 172.21.15.83:0/2780438461 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff78419a7a0 0x7ff78419ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:57.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.750+0000 7ff783400640 1 -- 172.21.15.83:0/2780438461 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff784104320 msgr2=0x7ff7841a0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:57.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.750+0000 7ff783400640 1 --2- 172.21.15.83:0/2780438461 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff784104320 0x7ff7841a0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:57.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.750+0000 7ff783400640 1 -- 172.21.15.83:0/2780438461 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff78419a7a0 msgr2=0x7ff78419ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:57.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.750+0000 7ff783400640 1 --2- 172.21.15.83:0/2780438461 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff78419a7a0 0x7ff78419ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:57.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.750+0000 7ff783400640 1 -- 172.21.15.83:0/2780438461 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff78419b3a0 con 0x7ff7841093f0 2024-04-24T08:57:57.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.751+0000 7ff783400640 1 --2- 172.21.15.83:0/2780438461 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7841093f0 0x7ff7841a0bd0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7ff770031440 tx=0x7ff770003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:57.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.751+0000 7ff780c00640 1 -- 172.21.15.83:0/2780438461 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff77003f070 con 0x7ff7841093f0 2024-04-24T08:57:57.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.751+0000 7ff780c00640 1 -- 172.21.15.83:0/2780438461 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff770031d60 con 0x7ff7841093f0 2024-04-24T08:57:57.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.751+0000 7ff780c00640 1 -- 172.21.15.83:0/2780438461 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff770035030 con 0x7ff7841093f0 2024-04-24T08:57:57.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.751+0000 7ff789c00640 1 -- 172.21.15.83:0/2780438461 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff78419b630 con 0x7ff7841093f0 2024-04-24T08:57:57.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.751+0000 7ff789c00640 1 -- 172.21.15.83:0/2780438461 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7841a76d0 con 0x7ff7841093f0 2024-04-24T08:57:57.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.755+0000 7ff78e004740 1 Processor -- start 2024-04-24T08:57:57.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.755+0000 7ff780c00640 1 -- 172.21.15.83:0/2780438461 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff770042070 con 0x7ff7841093f0 2024-04-24T08:57:57.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.755+0000 7ff78e004740 1 -- start start 2024-04-24T08:57:57.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.755+0000 7ff78e004740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bd39bd2a30 0x55bd39bd2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:57.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.755+0000 7ff78e004740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bd39bd3350 0x55bd39bdd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:57.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.755+0000 7ff78e004740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bd39bdda20 0x55bd39bdfe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:57.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.755+0000 7ff78e004740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55bd39b381f0 con 0x55bd39bd3350 2024-04-24T08:57:57.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.755+0000 7ff78e004740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55bd39ade350 con 0x55bd39bdda20 2024-04-24T08:57:57.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.755+0000 7ff78e004740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55bd39b341b0 con 0x55bd39bd2a30 2024-04-24T08:57:57.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.756+0000 7ff783e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bd39bd3350 0x55bd39bdd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:57.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.756+0000 7ff783e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bd39bd3350 0x55bd39bdd4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41940/0 (socket says 172.21.15.83:41940) 2024-04-24T08:57:57.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.756+0000 7ff783e00640 1 -- 172.21.15.83:0/2349207812 learned_addr learned my addr 172.21.15.83:0/2349207812 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:57:57.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.756+0000 7ff789200640 1 --2- 172.21.15.83:0/2349207812 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bd39bd2a30 0x55bd39bd2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:57.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.756+0000 7ff783400640 1 --2- 172.21.15.83:0/2349207812 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bd39bdda20 0x55bd39bdfe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:57.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.756+0000 7ff783e00640 1 -- 172.21.15.83:0/2349207812 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bd39bd2a30 msgr2=0x55bd39bd2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:57.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.756+0000 7ff783e00640 1 --2- 172.21.15.83:0/2349207812 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bd39bd2a30 0x55bd39bd2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:57.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.756+0000 7ff783e00640 1 -- 172.21.15.83:0/2349207812 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bd39bdda20 msgr2=0x55bd39bdfe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:57.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.756+0000 7ff783e00640 1 --2- 172.21.15.83:0/2349207812 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bd39bdda20 0x55bd39bdfe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:57.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.756+0000 7ff783e00640 1 -- 172.21.15.83:0/2349207812 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd39b32f50 con 0x55bd39bd3350 2024-04-24T08:57:57.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.756+0000 7ff789200640 1 --2- 172.21.15.83:0/2349207812 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bd39bd2a30 0x55bd39bd2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:57:57.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.757+0000 7ff780c00640 1 --2- 172.21.15.83:0/2780438461 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff7680a3d20 0x7ff7680a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:57.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.757+0000 7ff783e00640 1 --2- 172.21.15.83:0/2349207812 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bd39bd3350 0x55bd39bdd4e0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7ff77400cbf0 tx=0x7ff77400e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bf633967193481d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:57.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.757+0000 7ff780c00640 1 -- 172.21.15.83:0/2780438461 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7ff7700f1550 con 0x7ff7841093f0 2024-04-24T08:57:57.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.757+0000 7ff789200640 1 --2- 172.21.15.83:0/2780438461 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff7680a3d20 0x7ff7680a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:57.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.757+0000 7ff779600640 1 -- 172.21.15.83:0/2349207812 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff774010630 con 0x55bd39bd3350 2024-04-24T08:57:57.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.757+0000 7ff779600640 1 -- 172.21.15.83:0/2349207812 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff774010c80 con 0x55bd39bd3350 2024-04-24T08:57:57.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.757+0000 7ff779600640 1 -- 172.21.15.83:0/2349207812 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff77400fba0 con 0x55bd39bd3350 2024-04-24T08:57:57.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.758+0000 7ff78e004740 1 -- 172.21.15.83:0/2349207812 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bd39bd3350 msgr2=0x55bd39bdd4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:57.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.758+0000 7ff789200640 1 --2- 172.21.15.83:0/2780438461 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff7680a3d20 0x7ff7680a61e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff77c009fd0 tx=0x7ff77c00e040 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:57.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.758+0000 7ff78e004740 1 --2- 172.21.15.83:0/2349207812 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bd39bd3350 0x55bd39bdd4e0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7ff77400cbf0 tx=0x7ff77400e720 comp rx=0 tx=0).stop 2024-04-24T08:57:57.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.758+0000 7ff78e004740 1 -- 172.21.15.83:0/2349207812 shutdown_connections 2024-04-24T08:57:57.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.758+0000 7ff78e004740 1 --2- 172.21.15.83:0/2349207812 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bd39bdda20 0x55bd39bdfe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:57.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.758+0000 7ff78e004740 1 --2- 172.21.15.83:0/2349207812 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bd39bd3350 0x55bd39bdd4e0 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:57.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.758+0000 7ff78e004740 1 --2- 172.21.15.83:0/2349207812 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bd39bd2a30 0x55bd39bd2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:57.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.758+0000 7ff78e004740 1 -- 172.21.15.83:0/2349207812 >> 172.21.15.83:0/2349207812 conn(0x55bd39bced50 msgr2=0x55bd39bcf130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:57.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.758+0000 7ff78e004740 1 -- 172.21.15.83:0/2349207812 shutdown_connections 2024-04-24T08:57:57.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.758+0000 7ff78e004740 1 -- 172.21.15.83:0/2349207812 wait complete. 2024-04-24T08:57:57.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.759+0000 7ff78e004740 1 -- start start 2024-04-24T08:57:57.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.759+0000 7ff78e004740 1 Processor -- start 2024-04-24T08:57:57.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.759+0000 7ff78e004740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bd39bd2a30 0x55bd39bda980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:57.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.759+0000 7ff78e004740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bd39bd3350 0x55bd39bdaec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:57.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.759+0000 7ff78e004740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bd39bdda20 0x55bd39bdb400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:57.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.759+0000 7ff78e004740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55bd39be0810 con 0x55bd39bd2a30 2024-04-24T08:57:57.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.759+0000 7ff78e004740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55bd39af1690 con 0x55bd39bdda20 2024-04-24T08:57:57.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.759+0000 7ff78e004740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55bd39be0990 con 0x55bd39bd3350 2024-04-24T08:57:57.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.759+0000 7ff78e004740 10 client.0 _ll_register_callbacks cb 0x55bd39b2d650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:57:57.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.760+0000 7ff783e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bd39bd2a30 0x55bd39bda980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:57.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.760+0000 7ff783e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bd39bd2a30 0x55bd39bda980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41950/0 (socket says 172.21.15.83:41950) 2024-04-24T08:57:57.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.760+0000 7ff783e00640 1 -- 172.21.15.83:0/238101259 learned_addr learned my addr 172.21.15.83:0/238101259 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:57:57.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.760+0000 7ff783400640 1 --2- 172.21.15.83:0/238101259 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bd39bd3350 0x55bd39bdaec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:57.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.760+0000 7ff789200640 1 --2- 172.21.15.83:0/238101259 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bd39bdda20 0x55bd39bdb400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:57.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.760+0000 7ff783e00640 1 -- 172.21.15.83:0/238101259 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bd39bd3350 msgr2=0x55bd39bdaec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:57.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.760+0000 7ff783e00640 1 --2- 172.21.15.83:0/238101259 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bd39bd3350 0x55bd39bdaec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:57.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.760+0000 7ff783e00640 1 -- 172.21.15.83:0/238101259 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bd39bdda20 msgr2=0x55bd39bdb400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:57.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.760+0000 7ff783e00640 1 --2- 172.21.15.83:0/238101259 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bd39bdda20 0x55bd39bdb400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:57.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.760+0000 7ff783e00640 1 -- 172.21.15.83:0/238101259 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd39af0920 con 0x55bd39bd2a30 2024-04-24T08:57:57.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.760+0000 7ff789200640 1 --2- 172.21.15.83:0/238101259 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bd39bdda20 0x55bd39bdb400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:57:57.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.761+0000 7ff783e00640 1 --2- 172.21.15.83:0/238101259 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bd39bd2a30 0x55bd39bda980 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7ff77400ec20 tx=0x7ff77400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac8ace58e7b9eee7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:57.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.761+0000 7ff76e000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:57:57.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.761+0000 7ff76e000640 1 -- 172.21.15.83:0/238101259 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff774029060 con 0x55bd39bd2a30 2024-04-24T08:57:57.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.761+0000 7ff76e000640 1 -- 172.21.15.83:0/238101259 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7ff774005250 con 0x55bd39bd2a30 2024-04-24T08:57:57.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.761+0000 7ff78e004740 1 -- 172.21.15.83:0/238101259 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff774010c80 con 0x55bd39bd2a30 2024-04-24T08:57:57.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.761+0000 7ff76e000640 1 -- 172.21.15.83:0/238101259 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7ff774030050 con 0x55bd39bd2a30 2024-04-24T08:57:57.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.761+0000 7ff78e004740 10 client.17970 fetch_fsmap learned FSMap version 33 2024-04-24T08:57:57.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.761+0000 7ff78e004740 1 -- 172.21.15.83:0/238101259 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55bd399147d0 con 0x55bd39bd2a30 2024-04-24T08:57:57.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.762+0000 7ff76e000640 1 -- 172.21.15.83:0/238101259 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff774010c80 con 0x55bd39bd2a30 2024-04-24T08:57:57.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.762+0000 7ff76e000640 1 -- 172.21.15.83:0/238101259 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7ff7740046c0 con 0x55bd39bd2a30 2024-04-24T08:57:57.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.762+0000 7ff78e004740 10 client.17970 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T08:57:57.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.762+0000 7ff78e004740 10 client.17970 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:57:57.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.762+0000 7ff78e004740 20 client.17970 populate_metadata read hostname 'smithi083' 2024-04-24T08:57:57.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.762+0000 7ff78e004740 1 --2- 172.21.15.83:0/238101259 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55bd39bedca0 0x55bd39bf0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:57.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.762+0000 7ff78e004740 4 client.17970 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:57:57.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.762+0000 7ff78e004740 1 -- 172.21.15.83:0/238101259 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bd39b45c10 con 0x55bd39bedca0 2024-04-24T08:57:57.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.763+0000 7ff783400640 1 --2- 172.21.15.83:0/238101259 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55bd39bedca0 0x55bd39bf0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:57.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.763+0000 7ff783400640 1 --2- 172.21.15.83:0/238101259 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55bd39bedca0 0x55bd39bf0090 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3449d3026451ff28 server_cookie=d096f6c48bed48c0 in_seq=0 out_seq=0 2024-04-24T08:57:57.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.763+0000 7ff76e000640 10 client.17970 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:57:57.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.765+0000 7ff76e000640 1 -- 172.21.15.83:0/238101259 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bd39b45c10 con 0x55bd39bedca0 2024-04-24T08:57:57.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.765+0000 7ff76e000640 10 client.17970 handle_command_reply: tid=0 2024-04-24T08:57:57.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.765+0000 7ff78e004740 1 client.17970 shutdown 2024-04-24T08:57:57.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.766+0000 7ff78e004740 1 -- 172.21.15.83:0/238101259 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bd39bd2a30 msgr2=0x55bd39bda980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:57.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.766+0000 7ff78e004740 1 --2- 172.21.15.83:0/238101259 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bd39bd2a30 0x55bd39bda980 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7ff77400ec20 tx=0x7ff77400c960 comp rx=0 tx=0).stop 2024-04-24T08:57:57.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.766+0000 7ff78e004740 1 -- 172.21.15.83:0/238101259 shutdown_connections 2024-04-24T08:57:57.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.766+0000 7ff78e004740 1 --2- 172.21.15.83:0/238101259 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55bd39bedca0 0x55bd39bf0090 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:57.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.766+0000 7ff78e004740 1 --2- 172.21.15.83:0/238101259 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bd39bdda20 0x55bd39bdb400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:57.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.766+0000 7ff78e004740 1 --2- 172.21.15.83:0/238101259 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bd39bd3350 0x55bd39bdaec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:57.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.766+0000 7ff78e004740 1 --2- 172.21.15.83:0/238101259 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bd39bd2a30 0x55bd39bda980 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:57.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.766+0000 7ff78e004740 1 -- 172.21.15.83:0/238101259 >> 172.21.15.83:0/238101259 conn(0x55bd39bced50 msgr2=0x55bd39bd4b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:57.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.766+0000 7ff76e000640 0 client.17970 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:57:57.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.766+0000 7ff78e004740 1 -- 172.21.15.83:0/238101259 shutdown_connections 2024-04-24T08:57:57.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.766+0000 7ff78e004740 1 -- 172.21.15.83:0/238101259 wait complete. 2024-04-24T08:57:57.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.766+0000 7ff78e004740 20 client.17970 trim_cache size 0 max 16384 2024-04-24T08:57:57.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.785+0000 7ff78e004740 1 Processor -- start 2024-04-24T08:57:57.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.786+0000 7ff78e004740 1 -- start start 2024-04-24T08:57:57.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.786+0000 7ff78e004740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bd39bd2a30 0x55bd39bda260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:57.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.786+0000 7ff78e004740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bd39bd3350 0x55bd39bd5190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:57.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.786+0000 7ff78e004740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bd39bdda20 0x55bd39bdb050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:57.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.786+0000 7ff78e004740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff7841b1560 con 0x55bd39bd3350 2024-04-24T08:57:57.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.786+0000 7ff78e004740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff7841b30a0 con 0x55bd39bd2a30 2024-04-24T08:57:57.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.786+0000 7ff78e004740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff7841b1c30 con 0x55bd39bdda20 2024-04-24T08:57:57.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.786+0000 7ff783400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bd39bd3350 0x55bd39bd5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:57.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.786+0000 7ff783400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bd39bd3350 0x55bd39bd5190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41956/0 (socket says 172.21.15.83:41956) 2024-04-24T08:57:57.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.786+0000 7ff783400640 1 -- 172.21.15.83:0/2073792401 learned_addr learned my addr 172.21.15.83:0/2073792401 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:57:57.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.786+0000 7ff789200640 1 --2- 172.21.15.83:0/2073792401 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bd39bdda20 0x55bd39bdb050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:57.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.787+0000 7ff783e00640 1 --2- 172.21.15.83:0/2073792401 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bd39bd2a30 0x55bd39bda260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:57.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.787+0000 7ff783400640 1 -- 172.21.15.83:0/2073792401 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bd39bdda20 msgr2=0x55bd39bdb050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:57.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.787+0000 7ff783400640 1 --2- 172.21.15.83:0/2073792401 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bd39bdda20 0x55bd39bdb050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:57.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.787+0000 7ff783400640 1 -- 172.21.15.83:0/2073792401 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bd39bd2a30 msgr2=0x55bd39bda260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:57.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.787+0000 7ff783400640 1 --2- 172.21.15.83:0/2073792401 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bd39bd2a30 0x55bd39bda260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:57.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.787+0000 7ff783400640 1 -- 172.21.15.83:0/2073792401 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd3984a3f0 con 0x55bd39bd3350 2024-04-24T08:57:57.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.787+0000 7ff789200640 1 --2- 172.21.15.83:0/2073792401 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bd39bdda20 0x55bd39bdb050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:57:57.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.787+0000 7ff783e00640 1 --2- 172.21.15.83:0/2073792401 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bd39bd2a30 0x55bd39bda260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:57:57.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.787+0000 7ff783400640 1 --2- 172.21.15.83:0/2073792401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bd39bd3350 0x55bd39bd5190 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7ff7700b9580 tx=0x7ff770009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16e5c8b934234b43 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:57.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.787+0000 7ff778c00640 1 -- 172.21.15.83:0/2073792401 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff770049040 con 0x55bd39bd3350 2024-04-24T08:57:57.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.787+0000 7ff778c00640 1 -- 172.21.15.83:0/2073792401 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff7700bd820 con 0x55bd39bd3350 2024-04-24T08:57:57.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.787+0000 7ff778c00640 1 -- 172.21.15.83:0/2073792401 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7700bb420 con 0x55bd39bd3350 2024-04-24T08:57:57.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.788+0000 7ff78e004740 1 -- 172.21.15.83:0/2073792401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bd39bd3350 msgr2=0x55bd39bd5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:57.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.788+0000 7ff78e004740 1 --2- 172.21.15.83:0/2073792401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bd39bd3350 0x55bd39bd5190 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7ff7700b9580 tx=0x7ff770009260 comp rx=0 tx=0).stop 2024-04-24T08:57:57.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.788+0000 7ff78e004740 1 -- 172.21.15.83:0/2073792401 shutdown_connections 2024-04-24T08:57:57.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.788+0000 7ff78e004740 1 --2- 172.21.15.83:0/2073792401 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bd39bdda20 0x55bd39bdb050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:57.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.788+0000 7ff78e004740 1 --2- 172.21.15.83:0/2073792401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bd39bd3350 0x55bd39bd5190 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:57.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.788+0000 7ff78e004740 1 --2- 172.21.15.83:0/2073792401 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bd39bd2a30 0x55bd39bda260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:57.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.788+0000 7ff78e004740 1 -- 172.21.15.83:0/2073792401 >> 172.21.15.83:0/2073792401 conn(0x55bd39bced50 msgr2=0x55bd39bda940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:57.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.788+0000 7ff78e004740 1 -- 172.21.15.83:0/2073792401 shutdown_connections 2024-04-24T08:57:57.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.788+0000 7ff78e004740 1 -- 172.21.15.83:0/2073792401 wait complete. 2024-04-24T08:57:57.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.789+0000 7ff78e004740 1 -- start start 2024-04-24T08:57:57.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.789+0000 7ff78e004740 1 Processor -- start 2024-04-24T08:57:57.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.789+0000 7ff78e004740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bd39bd2a30 0x55bd39be0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:57.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.789+0000 7ff78e004740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bd39bd3350 0x55bd39be10e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:57.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.789+0000 7ff78e004740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bd39bdda20 0x55bd39be1620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:57.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.789+0000 7ff78e004740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff7841b29d0 con 0x55bd39bd3350 2024-04-24T08:57:57.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.789+0000 7ff78e004740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff7841b2300 con 0x55bd39bd2a30 2024-04-24T08:57:57.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.789+0000 7ff78e004740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff7841af540 con 0x55bd39bdda20 2024-04-24T08:57:57.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.789+0000 7ff78e004740 10 client.0 _ll_register_callbacks cb 0x7ff774014070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:57:57.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.790+0000 7ff783400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bd39bd3350 0x55bd39be10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:57.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.790+0000 7ff783400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bd39bd3350 0x55bd39be10e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41966/0 (socket says 172.21.15.83:41966) 2024-04-24T08:57:57.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.790+0000 7ff783400640 1 -- 172.21.15.83:0/1599321358 learned_addr learned my addr 172.21.15.83:0/1599321358 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:57:57.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.790+0000 7ff783e00640 1 --2- 172.21.15.83:0/1599321358 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bd39bd2a30 0x55bd39be0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:57.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.790+0000 7ff789200640 1 --2- 172.21.15.83:0/1599321358 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bd39bdda20 0x55bd39be1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:57.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.790+0000 7ff783400640 1 -- 172.21.15.83:0/1599321358 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bd39bdda20 msgr2=0x55bd39be1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:57.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.790+0000 7ff783400640 1 --2- 172.21.15.83:0/1599321358 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bd39bdda20 0x55bd39be1620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:57.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.790+0000 7ff783400640 1 -- 172.21.15.83:0/1599321358 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bd39bd2a30 msgr2=0x55bd39be0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:57.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.790+0000 7ff783400640 1 --2- 172.21.15.83:0/1599321358 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bd39bd2a30 0x55bd39be0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:57.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.790+0000 7ff783400640 1 -- 172.21.15.83:0/1599321358 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd3980a0f0 con 0x55bd39bd3350 2024-04-24T08:57:57.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.790+0000 7ff783e00640 1 --2- 172.21.15.83:0/1599321358 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bd39bd2a30 0x55bd39be0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:57:57.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.790+0000 7ff789200640 1 --2- 172.21.15.83:0/1599321358 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bd39bdda20 0x55bd39be1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:57:57.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.790+0000 7ff783400640 1 --2- 172.21.15.83:0/1599321358 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bd39bd3350 0x55bd39be10e0 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7ff770006010 tx=0x7ff7700f18e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21d2b3201891cd3e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:57.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.790+0000 7ff76e000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:57:57.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.791+0000 7ff76e000640 1 -- 172.21.15.83:0/1599321358 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff770049070 con 0x55bd39bd3350 2024-04-24T08:57:57.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.791+0000 7ff76e000640 1 -- 172.21.15.83:0/1599321358 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7ff7700c1170 con 0x55bd39bd3350 2024-04-24T08:57:57.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.791+0000 7ff78e004740 1 -- 172.21.15.83:0/1599321358 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7700bd820 con 0x55bd39bd3350 2024-04-24T08:57:57.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.791+0000 7ff76e000640 1 -- 172.21.15.83:0/1599321358 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7ff7700bca80 con 0x55bd39bd3350 2024-04-24T08:57:57.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.791+0000 7ff78e004740 10 client.17976 fetch_fsmap learned FSMap version 33 2024-04-24T08:57:57.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.791+0000 7ff78e004740 1 -- 172.21.15.83:0/1599321358 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55bd39afd810 con 0x55bd39bd3350 2024-04-24T08:57:57.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.791+0000 7ff76e000640 1 -- 172.21.15.83:0/1599321358 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff7700bd820 con 0x55bd39bd3350 2024-04-24T08:57:57.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.791+0000 7ff76e000640 1 -- 172.21.15.83:0/1599321358 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7ff77003b680 con 0x55bd39bd3350 2024-04-24T08:57:57.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.791+0000 7ff78e004740 10 client.17976 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T08:57:57.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.791+0000 7ff78e004740 10 client.17976 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:57:57.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.791+0000 7ff78e004740 20 client.17976 populate_metadata read hostname 'smithi083' 2024-04-24T08:57:57.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.791+0000 7ff78e004740 1 --2- 172.21.15.83:0/1599321358 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55bd39c53580 0x55bd39c559a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:57.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.792+0000 7ff78e004740 4 client.17976 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T08:57:57.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.792+0000 7ff78e004740 1 -- 172.21.15.83:0/1599321358 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55bd39b3fa80 con 0x55bd39c53580 2024-04-24T08:57:57.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.792+0000 7ff783e00640 1 --2- 172.21.15.83:0/1599321358 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55bd39c53580 0x55bd39c559a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:57.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.793+0000 7ff783e00640 1 --2- 172.21.15.83:0/1599321358 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55bd39c53580 0x55bd39c559a0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b2a1b0c8603e6b0 server_cookie=556d9d64493b4ac7 in_seq=0 out_seq=0 2024-04-24T08:57:57.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.793+0000 7ff76e000640 10 client.17976 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:57:57.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.793+0000 7ff76e000640 1 -- 172.21.15.83:0/1599321358 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55bd39b3fa80 con 0x55bd39c53580 2024-04-24T08:57:57.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.793+0000 7ff76e000640 10 client.17976 handle_command_reply: tid=0 2024-04-24T08:57:57.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.793+0000 7ff78e004740 1 client.17976 shutdown 2024-04-24T08:57:57.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.793+0000 7ff78e004740 1 -- 172.21.15.83:0/1599321358 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bd39bd3350 msgr2=0x55bd39be10e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:57.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.793+0000 7ff78e004740 1 --2- 172.21.15.83:0/1599321358 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bd39bd3350 0x55bd39be10e0 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7ff770006010 tx=0x7ff7700f18e0 comp rx=0 tx=0).stop 2024-04-24T08:57:57.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.794+0000 7ff78e004740 1 -- 172.21.15.83:0/1599321358 shutdown_connections 2024-04-24T08:57:57.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.794+0000 7ff78e004740 1 --2- 172.21.15.83:0/1599321358 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55bd39c53580 0x55bd39c559a0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:57.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.794+0000 7ff78e004740 1 --2- 172.21.15.83:0/1599321358 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bd39bdda20 0x55bd39be1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:57.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.794+0000 7ff78e004740 1 --2- 172.21.15.83:0/1599321358 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bd39bd3350 0x55bd39be10e0 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:57.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.794+0000 7ff78e004740 1 --2- 172.21.15.83:0/1599321358 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bd39bd2a30 0x55bd39be0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:57.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.794+0000 7ff76e000640 0 client.17976 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:57:57.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.794+0000 7ff78e004740 1 -- 172.21.15.83:0/1599321358 >> 172.21.15.83:0/1599321358 conn(0x55bd39bced50 msgr2=0x55bd39bdb1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:57.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.794+0000 7ff78e004740 1 -- 172.21.15.83:0/1599321358 shutdown_connections 2024-04-24T08:57:57.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.794+0000 7ff78e004740 1 -- 172.21.15.83:0/1599321358 wait complete. 2024-04-24T08:57:57.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.794+0000 7ff78e004740 20 client.17976 trim_cache size 0 max 16384 2024-04-24T08:57:57.798 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T08:57:57.799 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T08:57:57.799 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-04-24T08:57:57.799 INFO:teuthology.orchestra.run.smithi083.stdout: "802626c3-afa3-4589-9bfa-0992fc8ff7e2": { 2024-04-24T08:57:57.799 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-04-24T08:57:57.799 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "802626c3-afa3-4589-9bfa-0992fc8ff7e2", 2024-04-24T08:57:57.799 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-04-24T08:57:57.799 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T08:57:57.799 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T08:57:57.799 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T08:57:57.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.794+0000 7ff789c00640 1 -- 172.21.15.83:0/2780438461 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff7680a3d20 msgr2=0x7ff7680a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:57.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.794+0000 7ff789c00640 1 --2- 172.21.15.83:0/2780438461 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff7680a3d20 0x7ff7680a61e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff77c009fd0 tx=0x7ff77c00e040 comp rx=0 tx=0).stop 2024-04-24T08:57:57.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.794+0000 7ff789c00640 1 -- 172.21.15.83:0/2780438461 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7841093f0 msgr2=0x7ff7841a0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:57.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.795+0000 7ff789c00640 1 --2- 172.21.15.83:0/2780438461 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7841093f0 0x7ff7841a0bd0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7ff770031440 tx=0x7ff770003d50 comp rx=0 tx=0).stop 2024-04-24T08:57:57.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.795+0000 7ff789c00640 1 -- 172.21.15.83:0/2780438461 shutdown_connections 2024-04-24T08:57:57.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.795+0000 7ff789c00640 1 --2- 172.21.15.83:0/2780438461 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff7680a3d20 0x7ff7680a61e0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:57.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.795+0000 7ff789c00640 1 --2- 172.21.15.83:0/2780438461 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff78419a7a0 0x7ff78419ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:57.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.795+0000 7ff789c00640 1 --2- 172.21.15.83:0/2780438461 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7841093f0 0x7ff7841a0bd0 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:57.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.795+0000 7ff789c00640 1 --2- 172.21.15.83:0/2780438461 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff784104320 0x7ff7841a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:57.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.795+0000 7ff789c00640 1 -- 172.21.15.83:0/2780438461 >> 172.21.15.83:0/2780438461 conn(0x7ff7840ffb40 msgr2=0x7ff784100f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:57.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.795+0000 7ff789c00640 1 -- 172.21.15.83:0/2780438461 shutdown_connections 2024-04-24T08:57:57.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:57:57.795+0000 7ff789c00640 1 -- 172.21.15.83:0/2780438461 wait complete. 2024-04-24T08:57:57.812 INFO:tasks.cephfs.filesystem:scrub status for tag:802626c3-afa3-4589-9bfa-0992fc8ff7e2 - {'path': '/', 'tag': '802626c3-afa3-4589-9bfa-0992fc8ff7e2', 'options': 'recursive,force'} 2024-04-24T08:57:58.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:57:58 smithi138 ceph-mon[31441]: pgmap v193: 129 pgs: 129 active+clean; 40 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 434 KiB/s wr, 43 op/s 2024-04-24T08:57:58.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:57:58 smithi138 ceph-mon[31441]: scrub queued for path: / 2024-04-24T08:57:58.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:57:58 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T08:57:58.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:57:58 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T08:57:58.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:57:58 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T08:57:58.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:57:58 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T08:57:58.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:57:58 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T08:57:58.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:57:58 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T08:57:58.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:57:58 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T08:57:58.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:57:58 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T08:57:58.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:57:58 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T08:57:58.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:57:58 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T08:57:58.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:57:58 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T08:57:58.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:57:58 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T08:57:58.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:57:58 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T08:57:58.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:57:58 smithi189 ceph-mon[31791]: pgmap v193: 129 pgs: 129 active+clean; 40 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 434 KiB/s wr, 43 op/s 2024-04-24T08:57:58.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:57:58 smithi189 ceph-mon[31791]: scrub queued for path: / 2024-04-24T08:57:58.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:57:58 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T08:57:58.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:57:58 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T08:57:58.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:57:58 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T08:57:58.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:57:58 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T08:57:58.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:57:58 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T08:57:58.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:57:58 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T08:57:58.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:57:58 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T08:57:58.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:57:58 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T08:57:58.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:57:58 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T08:57:58.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:57:58 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T08:57:58.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:57:58 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T08:57:58.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:57:58 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T08:57:58.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:57:58 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T08:57:58.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:57:58 smithi083 ceph-mon[27907]: pgmap v193: 129 pgs: 129 active+clean; 40 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 434 KiB/s wr, 43 op/s 2024-04-24T08:57:58.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:57:58 smithi083 ceph-mon[27907]: scrub queued for path: / 2024-04-24T08:57:58.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:57:58 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T08:57:58.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:57:58 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T08:57:58.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:57:58 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T08:57:58.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:57:58 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T08:57:58.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:57:58 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T08:57:58.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:57:58 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T08:57:58.555 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:57:58 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T08:57:58.555 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:57:58 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T08:57:58.555 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:57:58 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T08:57:58.555 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:57:58 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T08:57:58.555 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:57:58 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T08:57:58.555 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:57:58 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T08:57:58.555 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:57:58 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T08:57:58.583 INFO:tasks.workunit.client.0.smithi083.stdout: 1 32791 0.63 MB/sec execute 152 sec latency 60.225 ms 2024-04-24T08:57:59.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:57:59 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T08:57:59.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:57:59 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T08:57:59.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:57:59 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T08:57:59.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:57:59 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T08:57:59.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:57:59 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T08:57:59.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:57:59 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T08:57:59.583 INFO:tasks.workunit.client.0.smithi083.stdout: 1 32905 0.63 MB/sec execute 153 sec latency 47.393 ms 2024-04-24T08:57:59.651 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.651+0000 7f7d42e00640 1 -- 172.21.15.83:0/1808827118 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d3c104c50 msgr2=0x7f7d3c108c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:59.651 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.651+0000 7f7d42e00640 1 --2- 172.21.15.83:0/1808827118 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d3c104c50 0x7f7d3c108c90 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f7d28009900 tx=0x7f7d2802a410 comp rx=0 tx=0).stop 2024-04-24T08:57:59.651 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.651+0000 7f7d42e00640 1 -- 172.21.15.83:0/1808827118 shutdown_connections 2024-04-24T08:57:59.651 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.651+0000 7f7d42e00640 1 --2- 172.21.15.83:0/1808827118 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d3c109370 0x7f7d3c110f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:59.651 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.651+0000 7f7d42e00640 1 --2- 172.21.15.83:0/1808827118 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d3c104c50 0x7f7d3c108c90 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:59.651 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.651+0000 7f7d42e00640 1 --2- 172.21.15.83:0/1808827118 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7d3c1042a0 0x7f7d3c104680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:59.651 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.651+0000 7f7d42e00640 1 -- 172.21.15.83:0/1808827118 >> 172.21.15.83:0/1808827118 conn(0x7f7d3c0ffae0 msgr2=0x7f7d3c101f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:59.652 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.651+0000 7f7d42e00640 1 -- 172.21.15.83:0/1808827118 shutdown_connections 2024-04-24T08:57:59.652 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.651+0000 7f7d42e00640 1 -- 172.21.15.83:0/1808827118 wait complete. 2024-04-24T08:57:59.654 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.652+0000 7f7d42e00640 1 Processor -- start 2024-04-24T08:57:59.654 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.652+0000 7f7d42e00640 1 -- start start 2024-04-24T08:57:59.654 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.653+0000 7f7d42e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7d3c1042a0 0x7f7d3c1a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:59.654 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.653+0000 7f7d42e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d3c109370 0x7f7d3c1a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:59.654 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.653+0000 7f7d42e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d3c19a770 0x7f7d3c19abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:59.654 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.653+0000 7f7d42e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7d3c1142b0 con 0x7f7d3c109370 2024-04-24T08:57:59.654 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.653+0000 7f7d42e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7d3c114130 con 0x7f7d3c19a770 2024-04-24T08:57:59.655 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.653+0000 7f7d42e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7d3c114430 con 0x7f7d3c1042a0 2024-04-24T08:57:59.655 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.653+0000 7f7d41000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d3c109370 0x7f7d3c1a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:59.655 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.653+0000 7f7d41000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d3c109370 0x7f7d3c1a0ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46740/0 (socket says 172.21.15.83:46740) 2024-04-24T08:57:59.655 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.653+0000 7f7d41000640 1 -- 172.21.15.83:0/53015841 learned_addr learned my addr 172.21.15.83:0/53015841 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:57:59.655 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.654+0000 7f7d42400640 1 --2- 172.21.15.83:0/53015841 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d3c19a770 0x7f7d3c19abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:59.655 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.654+0000 7f7d41000640 1 -- 172.21.15.83:0/53015841 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7d3c1042a0 msgr2=0x7f7d3c1a0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:59.655 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.654+0000 7f7d41a00640 1 --2- 172.21.15.83:0/53015841 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7d3c1042a0 0x7f7d3c1a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:59.655 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.654+0000 7f7d41000640 1 --2- 172.21.15.83:0/53015841 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7d3c1042a0 0x7f7d3c1a0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:59.655 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.654+0000 7f7d41000640 1 -- 172.21.15.83:0/53015841 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d3c19a770 msgr2=0x7f7d3c19abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:59.655 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.654+0000 7f7d41000640 1 --2- 172.21.15.83:0/53015841 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d3c19a770 0x7f7d3c19abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:59.655 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.654+0000 7f7d41000640 1 -- 172.21.15.83:0/53015841 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d3c19b370 con 0x7f7d3c109370 2024-04-24T08:57:59.655 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.654+0000 7f7d41000640 1 --2- 172.21.15.83:0/53015841 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d3c109370 0x7f7d3c1a0ba0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f7d28004270 tx=0x7f7d280042a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:59.655 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.654+0000 7f7d3a000640 1 -- 172.21.15.83:0/53015841 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d2803f030 con 0x7f7d3c109370 2024-04-24T08:57:59.655 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.654+0000 7f7d3a000640 1 -- 172.21.15.83:0/53015841 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7d280044c0 con 0x7f7d3c109370 2024-04-24T08:57:59.655 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.654+0000 7f7d42e00640 1 -- 172.21.15.83:0/53015841 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d3c19b600 con 0x7f7d3c109370 2024-04-24T08:57:59.656 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.654+0000 7f7d3a000640 1 -- 172.21.15.83:0/53015841 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d28035070 con 0x7f7d3c109370 2024-04-24T08:57:59.656 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.654+0000 7f7d42e00640 1 -- 172.21.15.83:0/53015841 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d3c1a7740 con 0x7f7d3c109370 2024-04-24T08:57:59.657 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.658+0000 7f7d47195740 1 Processor -- start 2024-04-24T08:57:59.657 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.659+0000 7f7d47195740 1 -- start start 2024-04-24T08:57:59.659 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.659+0000 7f7d47195740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55daa91013c0 0x55daa91017a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:59.659 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.659+0000 7f7d47195740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55daa9101ce0 0x55daa910c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:59.659 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.659+0000 7f7d47195740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55daa910c5a0 0x55daa910e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:59.659 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.659+0000 7f7d47195740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55daa9067900 con 0x55daa910c5a0 2024-04-24T08:57:59.660 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.659+0000 7f7d47195740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55daa900df20 con 0x55daa91013c0 2024-04-24T08:57:59.660 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.659+0000 7f7d47195740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55daa90638c0 con 0x55daa9101ce0 2024-04-24T08:57:59.660 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.659+0000 7f7d3a000640 1 -- 172.21.15.83:0/53015841 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7d28042030 con 0x7f7d3c109370 2024-04-24T08:57:59.660 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.660+0000 7f7d41000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55daa910c5a0 0x55daa910e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:59.660 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.660+0000 7f7d41000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55daa910c5a0 0x55daa910e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46750/0 (socket says 172.21.15.83:46750) 2024-04-24T08:57:59.660 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.660+0000 7f7d41000640 1 -- 172.21.15.83:0/4259468292 learned_addr learned my addr 172.21.15.83:0/4259468292 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:57:59.660 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.660+0000 7f7d41a00640 1 --2- 172.21.15.83:0/4259468292 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55daa9101ce0 0x55daa910c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:59.660 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.660+0000 7f7d42400640 1 --2- 172.21.15.83:0/4259468292 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55daa91013c0 0x55daa91017a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:59.660 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.660+0000 7f7d41000640 1 -- 172.21.15.83:0/4259468292 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55daa9101ce0 msgr2=0x55daa910c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:59.660 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.660+0000 7f7d41000640 1 --2- 172.21.15.83:0/4259468292 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55daa9101ce0 0x55daa910c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:59.660 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.660+0000 7f7d41000640 1 -- 172.21.15.83:0/4259468292 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55daa91013c0 msgr2=0x55daa91017a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:59.660 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.660+0000 7f7d41000640 1 --2- 172.21.15.83:0/4259468292 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55daa91013c0 0x55daa91017a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:59.660 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.660+0000 7f7d41000640 1 -- 172.21.15.83:0/4259468292 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55daa9032e10 con 0x55daa910c5a0 2024-04-24T08:57:59.661 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.660+0000 7f7d41000640 1 --2- 172.21.15.83:0/4259468292 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55daa910c5a0 0x55daa910e990 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f7d280ee350 tx=0x7f7d280f93b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cd029e9f9a09380 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:59.664 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.661+0000 7f7d32a00640 1 -- 172.21.15.83:0/4259468292 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d280f9810 con 0x55daa910c5a0 2024-04-24T08:57:59.664 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.661+0000 7f7d32a00640 1 -- 172.21.15.83:0/4259468292 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7d280fe030 con 0x55daa910c5a0 2024-04-24T08:57:59.664 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.661+0000 7f7d32a00640 1 -- 172.21.15.83:0/4259468292 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d280f1db0 con 0x55daa910c5a0 2024-04-24T08:57:59.664 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.661+0000 7f7d3a000640 1 --2- 172.21.15.83:0/53015841 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f7d200a3d90 0x7f7d200a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:59.664 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.661+0000 7f7d3a000640 1 -- 172.21.15.83:0/53015841 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f7d280f10f0 con 0x7f7d3c109370 2024-04-24T08:57:59.664 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.661+0000 7f7d42400640 1 --2- 172.21.15.83:0/53015841 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f7d200a3d90 0x7f7d200a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:59.665 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.662+0000 7f7d47195740 1 -- 172.21.15.83:0/4259468292 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55daa910c5a0 msgr2=0x55daa910e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:59.665 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.662+0000 7f7d47195740 1 --2- 172.21.15.83:0/4259468292 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55daa910c5a0 0x55daa910e990 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f7d280ee350 tx=0x7f7d280f93b0 comp rx=0 tx=0).stop 2024-04-24T08:57:59.665 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.662+0000 7f7d47195740 1 -- 172.21.15.83:0/4259468292 shutdown_connections 2024-04-24T08:57:59.665 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.662+0000 7f7d47195740 1 --2- 172.21.15.83:0/4259468292 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55daa910c5a0 0x55daa910e990 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:59.665 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.662+0000 7f7d47195740 1 --2- 172.21.15.83:0/4259468292 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55daa9101ce0 0x55daa910c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:59.665 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.662+0000 7f7d47195740 1 --2- 172.21.15.83:0/4259468292 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55daa91013c0 0x55daa91017a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:59.665 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.662+0000 7f7d47195740 1 -- 172.21.15.83:0/4259468292 >> 172.21.15.83:0/4259468292 conn(0x55daa90fd8d0 msgr2=0x55daa90fdcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:59.665 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.662+0000 7f7d47195740 1 -- 172.21.15.83:0/4259468292 shutdown_connections 2024-04-24T08:57:59.665 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.662+0000 7f7d42400640 1 --2- 172.21.15.83:0/53015841 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f7d200a3d90 0x7f7d200a6250 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c005fd0 tx=0x7f7d2c0093c0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:59.665 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.662+0000 7f7d47195740 1 -- 172.21.15.83:0/4259468292 wait complete. 2024-04-24T08:57:59.665 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.662+0000 7f7d47195740 1 -- start start 2024-04-24T08:57:59.665 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.663+0000 7f7d47195740 1 Processor -- start 2024-04-24T08:57:59.665 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.663+0000 7f7d47195740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55daa91013c0 0x55daa9109500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:59.665 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.663+0000 7f7d47195740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55daa9101ce0 0x55daa9109a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:59.665 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.663+0000 7f7d47195740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55daa910c5a0 0x55daa9109f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:59.666 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.663+0000 7f7d47195740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55daa910ef90 con 0x55daa910c5a0 2024-04-24T08:57:59.666 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.663+0000 7f7d47195740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55daa90212c0 con 0x55daa91013c0 2024-04-24T08:57:59.666 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.663+0000 7f7d47195740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55daa910f110 con 0x55daa9101ce0 2024-04-24T08:57:59.666 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.663+0000 7f7d47195740 10 client.0 _ll_register_callbacks cb 0x55daa8d79a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:57:59.666 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.663+0000 7f7d42400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55daa910c5a0 0x55daa9109f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:59.666 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.664+0000 7f7d42400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55daa910c5a0 0x55daa9109f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46764/0 (socket says 172.21.15.83:46764) 2024-04-24T08:57:59.666 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.664+0000 7f7d42400640 1 -- 172.21.15.83:0/3912947219 learned_addr learned my addr 172.21.15.83:0/3912947219 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:57:59.666 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.664+0000 7f7d41a00640 1 --2- 172.21.15.83:0/3912947219 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55daa91013c0 0x55daa9109500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:59.666 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.664+0000 7f7d41000640 1 --2- 172.21.15.83:0/3912947219 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55daa9101ce0 0x55daa9109a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:59.666 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.664+0000 7f7d42400640 1 -- 172.21.15.83:0/3912947219 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55daa9101ce0 msgr2=0x55daa9109a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:59.666 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.664+0000 7f7d42400640 1 --2- 172.21.15.83:0/3912947219 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55daa9101ce0 0x55daa9109a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:59.667 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.664+0000 7f7d42400640 1 -- 172.21.15.83:0/3912947219 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55daa91013c0 msgr2=0x55daa9109500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:59.667 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.664+0000 7f7d42400640 1 --2- 172.21.15.83:0/3912947219 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55daa91013c0 0x55daa9109500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:59.667 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.664+0000 7f7d42400640 1 -- 172.21.15.83:0/3912947219 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55daa9031fd0 con 0x55daa910c5a0 2024-04-24T08:57:59.667 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.664+0000 7f7d42400640 1 --2- 172.21.15.83:0/3912947219 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55daa910c5a0 0x55daa9109f80 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c0127d0 tx=0x7f7d2c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58c5db49e869a30c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:59.667 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.664+0000 7f7d27400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:57:59.667 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.665+0000 7f7d27400640 1 -- 172.21.15.83:0/3912947219 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d2c00fd60 con 0x55daa910c5a0 2024-04-24T08:57:59.667 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.665+0000 7f7d27400640 1 -- 172.21.15.83:0/3912947219 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f7d2c01f420 con 0x55daa910c5a0 2024-04-24T08:57:59.667 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.665+0000 7f7d47195740 1 -- 172.21.15.83:0/3912947219 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d280fe030 con 0x55daa910c5a0 2024-04-24T08:57:59.667 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.665+0000 7f7d27400640 1 -- 172.21.15.83:0/3912947219 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f7d2c030050 con 0x55daa910c5a0 2024-04-24T08:57:59.667 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.665+0000 7f7d47195740 10 client.17988 fetch_fsmap learned FSMap version 33 2024-04-24T08:57:59.667 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.665+0000 7f7d47195740 1 -- 172.21.15.83:0/3912947219 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55daa8d7a210 con 0x55daa910c5a0 2024-04-24T08:57:59.667 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.665+0000 7f7d27400640 1 -- 172.21.15.83:0/3912947219 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7d280fe030 con 0x55daa910c5a0 2024-04-24T08:57:59.668 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.666+0000 7f7d27400640 1 -- 172.21.15.83:0/3912947219 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f7d2c01d440 con 0x55daa910c5a0 2024-04-24T08:57:59.668 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.666+0000 7f7d47195740 10 client.17988 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T08:57:59.669 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.666+0000 7f7d47195740 10 client.17988 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T08:57:59.669 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.666+0000 7f7d47195740 20 client.17988 populate_metadata read hostname 'smithi083' 2024-04-24T08:57:59.669 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.666+0000 7f7d47195740 1 --2- 172.21.15.83:0/3912947219 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55daa911c430 0x55daa911e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:59.669 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.666+0000 7f7d47195740 4 client.17988 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:57:59.669 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.666+0000 7f7d47195740 1 -- 172.21.15.83:0/3912947219 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55daa9075320 con 0x55daa911c430 2024-04-24T08:57:59.669 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.666+0000 7f7d41a00640 1 --2- 172.21.15.83:0/3912947219 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55daa911c430 0x55daa911e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:59.669 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.667+0000 7f7d41a00640 1 --2- 172.21.15.83:0/3912947219 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55daa911c430 0x55daa911e820 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66088f095faca481 server_cookie=47d96029010ea7b in_seq=0 out_seq=0 2024-04-24T08:57:59.669 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.667+0000 7f7d27400640 10 client.17988 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:57:59.670 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.670+0000 7f7d27400640 1 -- 172.21.15.83:0/3912947219 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55daa9075320 con 0x55daa911c430 2024-04-24T08:57:59.670 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.670+0000 7f7d27400640 10 client.17988 handle_command_reply: tid=0 2024-04-24T08:57:59.670 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.670+0000 7f7d47195740 1 client.17988 shutdown 2024-04-24T08:57:59.671 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.670+0000 7f7d47195740 1 -- 172.21.15.83:0/3912947219 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55daa910c5a0 msgr2=0x55daa9109f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:59.671 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.670+0000 7f7d47195740 1 --2- 172.21.15.83:0/3912947219 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55daa910c5a0 0x55daa9109f80 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c0127d0 tx=0x7f7d2c004a40 comp rx=0 tx=0).stop 2024-04-24T08:57:59.671 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.671+0000 7f7d47195740 1 -- 172.21.15.83:0/3912947219 shutdown_connections 2024-04-24T08:57:59.671 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.671+0000 7f7d47195740 1 --2- 172.21.15.83:0/3912947219 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55daa911c430 0x55daa911e820 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:59.671 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.671+0000 7f7d47195740 1 --2- 172.21.15.83:0/3912947219 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55daa910c5a0 0x55daa9109f80 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:59.671 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.671+0000 7f7d47195740 1 --2- 172.21.15.83:0/3912947219 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55daa9101ce0 0x55daa9109a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:59.671 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.671+0000 7f7d47195740 1 --2- 172.21.15.83:0/3912947219 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55daa91013c0 0x55daa9109500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:59.671 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.671+0000 7f7d47195740 1 -- 172.21.15.83:0/3912947219 >> 172.21.15.83:0/3912947219 conn(0x55daa90fd8d0 msgr2=0x55daa90fca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:59.671 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.671+0000 7f7d27400640 0 client.17988 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:57:59.671 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.671+0000 7f7d47195740 1 -- 172.21.15.83:0/3912947219 shutdown_connections 2024-04-24T08:57:59.671 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.671+0000 7f7d47195740 1 -- 172.21.15.83:0/3912947219 wait complete. 2024-04-24T08:57:59.671 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.671+0000 7f7d47195740 20 client.17988 trim_cache size 0 max 16384 2024-04-24T08:57:59.695 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.688+0000 7f7d47195740 1 Processor -- start 2024-04-24T08:57:59.695 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.689+0000 7f7d47195740 1 -- start start 2024-04-24T08:57:59.696 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.689+0000 7f7d47195740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55daa91013c0 0x55daa9117dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:59.696 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.689+0000 7f7d47195740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55daa9101ce0 0x55daa90fc910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:59.696 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.689+0000 7f7d47195740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55daa910c5a0 0x55daa9124110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:59.696 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.689+0000 7f7d47195740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7d3c1b1570 con 0x55daa9101ce0 2024-04-24T08:57:59.696 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.689+0000 7f7d47195740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7d3c1b30b0 con 0x55daa910c5a0 2024-04-24T08:57:59.696 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.689+0000 7f7d47195740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7d3c1b1c40 con 0x55daa91013c0 2024-04-24T08:57:59.696 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.689+0000 7f7d41000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55daa9101ce0 0x55daa90fc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:59.697 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.689+0000 7f7d41000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55daa9101ce0 0x55daa90fc910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46778/0 (socket says 172.21.15.83:46778) 2024-04-24T08:57:59.697 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.689+0000 7f7d41000640 1 -- 172.21.15.83:0/262493273 learned_addr learned my addr 172.21.15.83:0/262493273 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:57:59.697 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.690+0000 7f7d41000640 1 -- 172.21.15.83:0/262493273 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55daa91013c0 msgr2=0x55daa9117dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:59.697 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.690+0000 7f7d41000640 1 --2- 172.21.15.83:0/262493273 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55daa91013c0 0x55daa9117dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:59.697 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.690+0000 7f7d41000640 1 -- 172.21.15.83:0/262493273 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55daa910c5a0 msgr2=0x55daa9124110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:59.697 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.690+0000 7f7d41000640 1 --2- 172.21.15.83:0/262493273 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55daa910c5a0 0x55daa9124110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:59.697 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.690+0000 7f7d41000640 1 -- 172.21.15.83:0/262493273 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55daa8e46620 con 0x55daa9101ce0 2024-04-24T08:57:59.698 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.690+0000 7f7d41000640 1 --2- 172.21.15.83:0/262493273 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55daa9101ce0 0x55daa90fc910 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f7d28000c00 tx=0x7f7d2802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c56ffa35df6b0ef9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:59.698 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.690+0000 7f7d32000640 1 -- 172.21.15.83:0/262493273 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d28049030 con 0x55daa9101ce0 2024-04-24T08:57:59.698 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.690+0000 7f7d32000640 1 -- 172.21.15.83:0/262493273 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7d280fe070 con 0x55daa9101ce0 2024-04-24T08:57:59.698 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.690+0000 7f7d32000640 1 -- 172.21.15.83:0/262493273 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d280bcc60 con 0x55daa9101ce0 2024-04-24T08:57:59.698 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.691+0000 7f7d47195740 1 -- 172.21.15.83:0/262493273 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55daa9101ce0 msgr2=0x55daa90fc910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:59.698 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.691+0000 7f7d47195740 1 --2- 172.21.15.83:0/262493273 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55daa9101ce0 0x55daa90fc910 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f7d28000c00 tx=0x7f7d2802a410 comp rx=0 tx=0).stop 2024-04-24T08:57:59.698 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.691+0000 7f7d47195740 1 -- 172.21.15.83:0/262493273 shutdown_connections 2024-04-24T08:57:59.699 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.691+0000 7f7d47195740 1 --2- 172.21.15.83:0/262493273 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55daa910c5a0 0x55daa9124110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:59.699 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.691+0000 7f7d47195740 1 --2- 172.21.15.83:0/262493273 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55daa9101ce0 0x55daa90fc910 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:59.699 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.691+0000 7f7d47195740 1 --2- 172.21.15.83:0/262493273 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55daa91013c0 0x55daa9117dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:59.699 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.691+0000 7f7d47195740 1 -- 172.21.15.83:0/262493273 >> 172.21.15.83:0/262493273 conn(0x55daa90fd8d0 msgr2=0x55daa90fc200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:59.699 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.691+0000 7f7d47195740 1 -- 172.21.15.83:0/262493273 shutdown_connections 2024-04-24T08:57:59.699 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.691+0000 7f7d47195740 1 -- 172.21.15.83:0/262493273 wait complete. 2024-04-24T08:57:59.699 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.692+0000 7f7d47195740 1 -- start start 2024-04-24T08:57:59.700 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.692+0000 7f7d47195740 1 Processor -- start 2024-04-24T08:57:59.700 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.692+0000 7f7d47195740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55daa91013c0 0x55daa9114540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:59.700 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.692+0000 7f7d47195740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55daa9101ce0 0x55daa910f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:59.700 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.692+0000 7f7d47195740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55daa910c5a0 0x55daa910fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:59.700 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.692+0000 7f7d47195740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7d3c1b29e0 con 0x55daa9101ce0 2024-04-24T08:57:59.700 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.692+0000 7f7d47195740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7d3c1b2310 con 0x55daa910c5a0 2024-04-24T08:57:59.700 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.692+0000 7f7d47195740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7d3c1af550 con 0x55daa91013c0 2024-04-24T08:57:59.701 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.693+0000 7f7d47195740 10 client.0 _ll_register_callbacks cb 0x7f7d2c009530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:57:59.701 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.693+0000 7f7d41000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55daa9101ce0 0x55daa910f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:59.701 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.693+0000 7f7d41000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55daa9101ce0 0x55daa910f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46794/0 (socket says 172.21.15.83:46794) 2024-04-24T08:57:59.701 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.693+0000 7f7d41000640 1 -- 172.21.15.83:0/732651007 learned_addr learned my addr 172.21.15.83:0/732651007 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:57:59.701 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.693+0000 7f7d41a00640 1 --2- 172.21.15.83:0/732651007 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55daa91013c0 0x55daa9114540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:59.701 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.693+0000 7f7d41000640 1 -- 172.21.15.83:0/732651007 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55daa91013c0 msgr2=0x55daa9114540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:59.702 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.693+0000 7f7d41000640 1 --2- 172.21.15.83:0/732651007 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55daa91013c0 0x55daa9114540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:59.702 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.693+0000 7f7d41000640 1 -- 172.21.15.83:0/732651007 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55daa910c5a0 msgr2=0x55daa910fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:57:59.702 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.693+0000 7f7d41000640 1 --2- 172.21.15.83:0/732651007 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55daa910c5a0 0x55daa910fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:59.702 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.693+0000 7f7d41000640 1 -- 172.21.15.83:0/732651007 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55daa9110490 con 0x55daa9101ce0 2024-04-24T08:57:59.702 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.693+0000 7f7d41000640 1 --2- 172.21.15.83:0/732651007 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55daa9101ce0 0x55daa910f8b0 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f7d28031440 tx=0x7f7d280bc6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c92b6021d8be4d05 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:57:59.702 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.693+0000 7f7d27400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:57:59.702 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.694+0000 7f7d27400640 1 -- 172.21.15.83:0/732651007 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d28049070 con 0x55daa9101ce0 2024-04-24T08:57:59.703 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.694+0000 7f7d27400640 1 -- 172.21.15.83:0/732651007 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9495+0+0 (secure 0 0 0) 0x7f7d280baf60 con 0x55daa9101ce0 2024-04-24T08:57:59.703 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.694+0000 7f7d47195740 1 -- 172.21.15.83:0/732651007 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d280fe070 con 0x55daa9101ce0 2024-04-24T08:57:59.703 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.694+0000 7f7d27400640 1 -- 172.21.15.83:0/732651007 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f7d280b8f00 con 0x55daa9101ce0 2024-04-24T08:57:59.703 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.694+0000 7f7d47195740 10 client.17994 fetch_fsmap learned FSMap version 33 2024-04-24T08:57:59.703 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.694+0000 7f7d47195740 1 -- 172.21.15.83:0/732651007 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55daa9110490 con 0x55daa9101ce0 2024-04-24T08:57:59.703 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.694+0000 7f7d27400640 1 -- 172.21.15.83:0/732651007 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7d280fe070 con 0x55daa9101ce0 2024-04-24T08:57:59.703 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.694+0000 7f7d27400640 1 -- 172.21.15.83:0/732651007 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f7d280f4350 con 0x55daa9101ce0 2024-04-24T08:57:59.703 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.694+0000 7f7d47195740 10 client.17994 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T08:57:59.704 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.694+0000 7f7d47195740 10 client.17994 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T08:57:59.704 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.694+0000 7f7d47195740 20 client.17994 populate_metadata read hostname 'smithi083' 2024-04-24T08:57:59.704 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.694+0000 7f7d47195740 1 --2- 172.21.15.83:0/732651007 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55daa9182940 0x55daa9184d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:57:59.704 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.694+0000 7f7d47195740 4 client.17994 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T08:57:59.704 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.694+0000 7f7d47195740 1 -- 172.21.15.83:0/732651007 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55daa906f190 con 0x55daa9182940 2024-04-24T08:57:59.704 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.695+0000 7f7d41a00640 1 --2- 172.21.15.83:0/732651007 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55daa9182940 0x55daa9184d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:57:59.704 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.696+0000 7f7d41a00640 1 --2- 172.21.15.83:0/732651007 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55daa9182940 0x55daa9184d30 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5bae500ab35a28d2 server_cookie=ec7cea98482ca4f in_seq=0 out_seq=0 2024-04-24T08:57:59.704 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.696+0000 7f7d27400640 10 client.17994 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:57:59.755 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.755+0000 7f7d27400640 1 -- 172.21.15.83:0/732651007 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55daa906f190 con 0x55daa9182940 2024-04-24T08:57:59.755 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.755+0000 7f7d27400640 10 client.17994 handle_command_reply: tid=0 2024-04-24T08:57:59.755 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.755+0000 7f7d47195740 1 client.17994 shutdown 2024-04-24T08:57:59.755 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.756+0000 7f7d47195740 1 -- 172.21.15.83:0/732651007 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55daa9101ce0 msgr2=0x55daa910f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:59.755 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.756+0000 7f7d47195740 1 --2- 172.21.15.83:0/732651007 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55daa9101ce0 0x55daa910f8b0 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f7d28031440 tx=0x7f7d280bc6b0 comp rx=0 tx=0).stop 2024-04-24T08:57:59.755 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.756+0000 7f7d47195740 1 -- 172.21.15.83:0/732651007 shutdown_connections 2024-04-24T08:57:59.755 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.756+0000 7f7d47195740 1 --2- 172.21.15.83:0/732651007 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55daa9182940 0x55daa9184d30 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:59.756 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.756+0000 7f7d47195740 1 --2- 172.21.15.83:0/732651007 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55daa910c5a0 0x55daa910fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:59.756 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.756+0000 7f7d47195740 1 --2- 172.21.15.83:0/732651007 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55daa9101ce0 0x55daa910f8b0 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:59.756 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.756+0000 7f7d47195740 1 --2- 172.21.15.83:0/732651007 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55daa91013c0 0x55daa9114540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:59.756 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.756+0000 7f7d47195740 1 -- 172.21.15.83:0/732651007 >> 172.21.15.83:0/732651007 conn(0x55daa90fd8d0 msgr2=0x55daa90fca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:59.756 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.756+0000 7f7d27400640 0 client.17994 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:57:59.757 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.756+0000 7f7d47195740 1 -- 172.21.15.83:0/732651007 shutdown_connections 2024-04-24T08:57:59.757 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.756+0000 7f7d47195740 1 -- 172.21.15.83:0/732651007 wait complete. 2024-04-24T08:57:59.757 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.756+0000 7f7d47195740 20 client.17994 trim_cache size 0 max 16384 2024-04-24T08:57:59.757 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T08:57:59.757 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T08:57:59.757 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T08:57:59.757 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T08:57:59.758 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.757+0000 7f7d42e00640 1 -- 172.21.15.83:0/53015841 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f7d200a3d90 msgr2=0x7f7d200a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:59.758 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.757+0000 7f7d42e00640 1 --2- 172.21.15.83:0/53015841 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f7d200a3d90 0x7f7d200a6250 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c005fd0 tx=0x7f7d2c0093c0 comp rx=0 tx=0).stop 2024-04-24T08:57:59.758 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.757+0000 7f7d42e00640 1 -- 172.21.15.83:0/53015841 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d3c109370 msgr2=0x7f7d3c1a0ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:57:59.758 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.757+0000 7f7d42e00640 1 --2- 172.21.15.83:0/53015841 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d3c109370 0x7f7d3c1a0ba0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f7d28004270 tx=0x7f7d280042a0 comp rx=0 tx=0).stop 2024-04-24T08:57:59.759 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.758+0000 7f7d42e00640 1 -- 172.21.15.83:0/53015841 shutdown_connections 2024-04-24T08:57:59.759 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.758+0000 7f7d42e00640 1 --2- 172.21.15.83:0/53015841 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f7d200a3d90 0x7f7d200a6250 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:59.759 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.758+0000 7f7d42e00640 1 --2- 172.21.15.83:0/53015841 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d3c19a770 0x7f7d3c19abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:59.759 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.758+0000 7f7d42e00640 1 --2- 172.21.15.83:0/53015841 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d3c109370 0x7f7d3c1a0ba0 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:59.759 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.758+0000 7f7d42e00640 1 --2- 172.21.15.83:0/53015841 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7d3c1042a0 0x7f7d3c1a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:57:59.759 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.758+0000 7f7d42e00640 1 -- 172.21.15.83:0/53015841 >> 172.21.15.83:0/53015841 conn(0x7f7d3c0ffae0 msgr2=0x7f7d3c100ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:57:59.759 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.758+0000 7f7d42e00640 1 -- 172.21.15.83:0/53015841 shutdown_connections 2024-04-24T08:57:59.759 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:57:59.758+0000 7f7d42e00640 1 -- 172.21.15.83:0/53015841 wait complete. 2024-04-24T08:58:00.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:58:00 smithi138 ceph-mon[31441]: pgmap v194: 129 pgs: 129 active+clean; 42 MiB data, 609 MiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 544 KiB/s wr, 67 op/s 2024-04-24T08:58:00.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:58:00 smithi189 ceph-mon[31791]: pgmap v194: 129 pgs: 129 active+clean; 42 MiB data, 609 MiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 544 KiB/s wr, 67 op/s 2024-04-24T08:58:00.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:58:00 smithi083 ceph-mon[27907]: pgmap v194: 129 pgs: 129 active+clean; 42 MiB data, 609 MiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 544 KiB/s wr, 67 op/s 2024-04-24T08:58:00.584 INFO:tasks.workunit.client.0.smithi083.stdout: 1 33023 0.63 MB/sec execute 154 sec latency 64.229 ms 2024-04-24T08:58:01.584 INFO:tasks.workunit.client.0.smithi083.stdout: 1 33140 0.63 MB/sec execute 155 sec latency 52.717 ms 2024-04-24T08:58:02.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:58:02 smithi083 ceph-mon[27907]: pgmap v195: 129 pgs: 129 active+clean; 42 MiB data, 609 MiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 338 KiB/s wr, 44 op/s 2024-04-24T08:58:02.584 INFO:tasks.workunit.client.0.smithi083.stdout: 1 33249 0.63 MB/sec execute 156 sec latency 47.709 ms 2024-04-24T08:58:02.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:58:02 smithi138 ceph-mon[31441]: pgmap v195: 129 pgs: 129 active+clean; 42 MiB data, 609 MiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 338 KiB/s wr, 44 op/s 2024-04-24T08:58:02.706 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:58:02 smithi189 ceph-mon[31791]: pgmap v195: 129 pgs: 129 active+clean; 42 MiB data, 609 MiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 338 KiB/s wr, 44 op/s 2024-04-24T08:58:03.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:58:03 smithi083 ceph-mon[27907]: osdmap e95: 12 total, 12 up, 12 in 2024-04-24T08:58:03.584 INFO:tasks.workunit.client.0.smithi083.stdout: 1 33412 0.64 MB/sec execute 157 sec latency 63.477 ms 2024-04-24T08:58:03.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:58:03 smithi138 ceph-mon[31441]: osdmap e95: 12 total, 12 up, 12 in 2024-04-24T08:58:03.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:58:03 smithi189 ceph-mon[31791]: osdmap e95: 12 total, 12 up, 12 in 2024-04-24T08:58:04.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:58:04 smithi083 ceph-mon[27907]: pgmap v197: 129 pgs: 1 active+clean+snaptrim, 120 active+clean, 8 active+clean+snaptrim_wait; 40 MiB data, 611 MiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 443 KiB/s wr, 97 op/s 2024-04-24T08:58:04.584 INFO:tasks.workunit.client.0.smithi083.stdout: 1 33498 0.64 MB/sec execute 158 sec latency 60.479 ms 2024-04-24T08:58:04.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:58:04 smithi138 ceph-mon[31441]: pgmap v197: 129 pgs: 1 active+clean+snaptrim, 120 active+clean, 8 active+clean+snaptrim_wait; 40 MiB data, 611 MiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 443 KiB/s wr, 97 op/s 2024-04-24T08:58:04.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:58:04 smithi189 ceph-mon[31791]: pgmap v197: 129 pgs: 1 active+clean+snaptrim, 120 active+clean, 8 active+clean+snaptrim_wait; 40 MiB data, 611 MiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 443 KiB/s wr, 97 op/s 2024-04-24T08:58:05.584 INFO:tasks.workunit.client.0.smithi083.stdout: 1 33611 0.63 MB/sec execute 159 sec latency 66.686 ms 2024-04-24T08:58:05.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:58:05 smithi083 ceph-mon[27907]: osdmap e96: 12 total, 12 up, 12 in 2024-04-24T08:58:05.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:58:05 smithi083 ceph-mon[27907]: pgmap v199: 129 pgs: 1 active+clean+snaptrim, 120 active+clean, 8 active+clean+snaptrim_wait; 40 MiB data, 611 MiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 554 KiB/s wr, 122 op/s 2024-04-24T08:58:05.954 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:58:05 smithi138 ceph-mon[31441]: osdmap e96: 12 total, 12 up, 12 in 2024-04-24T08:58:05.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:58:05 smithi138 ceph-mon[31441]: pgmap v199: 129 pgs: 1 active+clean+snaptrim, 120 active+clean, 8 active+clean+snaptrim_wait; 40 MiB data, 611 MiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 554 KiB/s wr, 122 op/s 2024-04-24T08:58:05.956 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:58:05 smithi189 ceph-mon[31791]: osdmap e96: 12 total, 12 up, 12 in 2024-04-24T08:58:05.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:58:05 smithi189 ceph-mon[31791]: pgmap v199: 129 pgs: 1 active+clean+snaptrim, 120 active+clean, 8 active+clean+snaptrim_wait; 40 MiB data, 611 MiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 554 KiB/s wr, 122 op/s 2024-04-24T08:58:06.584 INFO:tasks.workunit.client.0.smithi083.stdout: 1 33740 0.63 MB/sec execute 160 sec latency 70.280 ms 2024-04-24T08:58:07.585 INFO:tasks.workunit.client.0.smithi083.stdout: 1 33839 0.63 MB/sec execute 161 sec latency 63.312 ms 2024-04-24T08:58:07.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:58:07 smithi138 ceph-mon[31441]: pgmap v200: 129 pgs: 1 active+clean+snaptrim, 120 active+clean, 8 active+clean+snaptrim_wait; 40 MiB data, 611 MiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 390 KiB/s wr, 86 op/s 2024-04-24T08:58:07.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:58:07 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T08:58:07.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:58:07 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T08:58:07.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:58:07 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T08:58:07.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:58:07 smithi189 ceph-mon[31791]: pgmap v200: 129 pgs: 1 active+clean+snaptrim, 120 active+clean, 8 active+clean+snaptrim_wait; 40 MiB data, 611 MiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 390 KiB/s wr, 86 op/s 2024-04-24T08:58:07.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:58:07 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T08:58:07.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:58:07 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T08:58:07.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:58:07 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T08:58:08.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:58:07 smithi083 ceph-mon[27907]: pgmap v200: 129 pgs: 1 active+clean+snaptrim, 120 active+clean, 8 active+clean+snaptrim_wait; 40 MiB data, 611 MiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 390 KiB/s wr, 86 op/s 2024-04-24T08:58:08.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:58:07 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T08:58:08.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:58:07 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T08:58:08.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:58:07 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T08:58:08.585 INFO:tasks.workunit.client.0.smithi083.stdout: 1 33992 0.62 MB/sec execute 162 sec latency 63.418 ms 2024-04-24T08:58:08.954 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:58:08 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T08:58:08.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:58:08 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T08:58:09.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:58:08 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T08:58:09.585 INFO:tasks.workunit.client.0.smithi083.stdout: 1 34107 0.62 MB/sec execute 163 sec latency 82.027 ms 2024-04-24T08:58:09.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:58:09 smithi138 ceph-mon[31441]: pgmap v201: 129 pgs: 129 active+clean; 40 MiB data, 601 MiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 485 KiB/s wr, 114 op/s 2024-04-24T08:58:09.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:58:09 smithi138 ceph-mon[31441]: osdmap e97: 12 total, 12 up, 12 in 2024-04-24T08:58:09.956 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:58:09 smithi189 ceph-mon[31791]: pgmap v201: 129 pgs: 129 active+clean; 40 MiB data, 601 MiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 485 KiB/s wr, 114 op/s 2024-04-24T08:58:09.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:58:09 smithi189 ceph-mon[31791]: osdmap e97: 12 total, 12 up, 12 in 2024-04-24T08:58:10.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:58:09 smithi083 ceph-mon[27907]: pgmap v201: 129 pgs: 129 active+clean; 40 MiB data, 601 MiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 485 KiB/s wr, 114 op/s 2024-04-24T08:58:10.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:58:09 smithi083 ceph-mon[27907]: osdmap e97: 12 total, 12 up, 12 in 2024-04-24T08:58:10.585 INFO:tasks.workunit.client.0.smithi083.stdout: 1 34265 0.62 MB/sec execute 164 sec latency 50.290 ms 2024-04-24T08:58:11.585 INFO:tasks.workunit.client.0.smithi083.stdout: 1 34348 0.62 MB/sec execute 165 sec latency 53.702 ms 2024-04-24T08:58:11.954 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:58:11 smithi138 ceph-mon[31441]: pgmap v203: 129 pgs: 129 active+clean; 40 MiB data, 601 MiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 96 KiB/s wr, 28 op/s 2024-04-24T08:58:11.956 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:58:11 smithi189 ceph-mon[31791]: pgmap v203: 129 pgs: 129 active+clean; 40 MiB data, 601 MiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 96 KiB/s wr, 28 op/s 2024-04-24T08:58:12.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:58:11 smithi083 ceph-mon[27907]: pgmap v203: 129 pgs: 129 active+clean; 40 MiB data, 601 MiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 96 KiB/s wr, 28 op/s 2024-04-24T08:58:12.585 INFO:tasks.workunit.client.0.smithi083.stdout: 1 34432 0.62 MB/sec execute 166 sec latency 48.088 ms 2024-04-24T08:58:12.915 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.915+0000 7f8539400640 1 -- 172.21.15.83:0/3667186789 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8534102a20 msgr2=0x7f8534106a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:12.915 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.915+0000 7f8539400640 1 --2- 172.21.15.83:0/3667186789 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8534102a20 0x7f8534106a60 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f8520009900 tx=0x7f852002a3f0 comp rx=0 tx=0).stop 2024-04-24T08:58:12.915 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.915+0000 7f8539400640 1 -- 172.21.15.83:0/3667186789 shutdown_connections 2024-04-24T08:58:12.915 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.915+0000 7f8539400640 1 --2- 172.21.15.83:0/3667186789 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8534107140 0x7f853410ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:12.915 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.915+0000 7f8539400640 1 --2- 172.21.15.83:0/3667186789 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8534102a20 0x7f8534106a60 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:12.915 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.915+0000 7f8539400640 1 --2- 172.21.15.83:0/3667186789 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8534102070 0x7f8534102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:12.915 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.915+0000 7f8539400640 1 -- 172.21.15.83:0/3667186789 >> 172.21.15.83:0/3667186789 conn(0x7f85340fd950 msgr2=0x7f85340ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:58:12.915 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.916+0000 7f8539400640 1 -- 172.21.15.83:0/3667186789 shutdown_connections 2024-04-24T08:58:12.915 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.916+0000 7f8539400640 1 -- 172.21.15.83:0/3667186789 wait complete. 2024-04-24T08:58:12.917 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.917+0000 7f8539400640 1 Processor -- start 2024-04-24T08:58:12.917 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.917+0000 7f8539400640 1 -- start start 2024-04-24T08:58:12.917 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.917+0000 7f8539400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8534102070 0x7f85341a05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:12.917 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.917+0000 7f8539400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8534107140 0x7f85341a0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:12.917 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.917+0000 7f8539400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f853419a690 0x7f853419ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:12.917 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.917+0000 7f8539400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f85341120d0 con 0x7f8534102070 2024-04-24T08:58:12.917 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.917+0000 7f8539400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8534111f50 con 0x7f8534107140 2024-04-24T08:58:12.917 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.917+0000 7f8539400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8534112250 con 0x7f853419a690 2024-04-24T08:58:12.917 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.918+0000 7f8533e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8534102070 0x7f85341a05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:12.917 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.918+0000 7f8533e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8534102070 0x7f85341a05d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51878/0 (socket says 172.21.15.83:51878) 2024-04-24T08:58:12.917 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.918+0000 7f8533e00640 1 -- 172.21.15.83:0/4214387445 learned_addr learned my addr 172.21.15.83:0/4214387445 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:58:12.917 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.918+0000 7f8538a00640 1 --2- 172.21.15.83:0/4214387445 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f853419a690 0x7f853419ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:12.917 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.918+0000 7f8533400640 1 --2- 172.21.15.83:0/4214387445 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8534107140 0x7f85341a0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:12.918 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.918+0000 7f8533e00640 1 -- 172.21.15.83:0/4214387445 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f853419a690 msgr2=0x7f853419ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:12.918 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.918+0000 7f8533e00640 1 --2- 172.21.15.83:0/4214387445 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f853419a690 0x7f853419ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:12.918 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.918+0000 7f8533e00640 1 -- 172.21.15.83:0/4214387445 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8534107140 msgr2=0x7f85341a0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:12.918 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.918+0000 7f8533e00640 1 --2- 172.21.15.83:0/4214387445 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8534107140 0x7f85341a0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:12.919 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.918+0000 7f8533e00640 1 -- 172.21.15.83:0/4214387445 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f853419b290 con 0x7f8534102070 2024-04-24T08:58:12.919 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.919+0000 7f8533e00640 1 --2- 172.21.15.83:0/4214387445 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8534102070 0x7f85341a05d0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f852400d770 tx=0x7f852400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:58:12.919 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.919+0000 7f8530c00640 1 -- 172.21.15.83:0/4214387445 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8524017070 con 0x7f8534102070 2024-04-24T08:58:12.919 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.919+0000 7f8539400640 1 -- 172.21.15.83:0/4214387445 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85341a7370 con 0x7f8534102070 2024-04-24T08:58:12.919 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.919+0000 7f8530c00640 1 -- 172.21.15.83:0/4214387445 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f85240054d0 con 0x7f8534102070 2024-04-24T08:58:12.919 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.919+0000 7f8530c00640 1 -- 172.21.15.83:0/4214387445 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8524004cf0 con 0x7f8534102070 2024-04-24T08:58:12.919 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.919+0000 7f8539400640 1 -- 172.21.15.83:0/4214387445 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85341a7720 con 0x7f8534102070 2024-04-24T08:58:12.922 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.923+0000 7f853d9b6740 1 Processor -- start 2024-04-24T08:58:12.922 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.923+0000 7f853d9b6740 1 -- start start 2024-04-24T08:58:12.922 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.923+0000 7f8530c00640 1 -- 172.21.15.83:0/4214387445 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8524004e90 con 0x7f8534102070 2024-04-24T08:58:12.922 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.923+0000 7f853d9b6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55855c7033c0 0x55855c7037a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:12.922 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.923+0000 7f853d9b6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55855c703ce0 0x55855c70e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:12.922 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.923+0000 7f853d9b6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55855c70e5a0 0x55855c710990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:12.922 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.923+0000 7f853d9b6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55855c669900 con 0x55855c70e5a0 2024-04-24T08:58:12.922 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.923+0000 7f853d9b6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55855c60ff20 con 0x55855c7033c0 2024-04-24T08:58:12.922 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.923+0000 7f853d9b6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55855c6658c0 con 0x55855c703ce0 2024-04-24T08:58:12.923 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.923+0000 7f8533400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55855c70e5a0 0x55855c710990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:12.923 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.924+0000 7f8533400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55855c70e5a0 0x55855c710990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51884/0 (socket says 172.21.15.83:51884) 2024-04-24T08:58:12.924 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.924+0000 7f8533400640 1 -- 172.21.15.83:0/754506715 learned_addr learned my addr 172.21.15.83:0/754506715 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:58:12.924 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.924+0000 7f8538a00640 1 --2- 172.21.15.83:0/754506715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55855c7033c0 0x55855c7037a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:12.924 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.924+0000 7f8533e00640 1 --2- 172.21.15.83:0/754506715 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55855c703ce0 0x55855c70e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:12.924 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.924+0000 7f8533400640 1 -- 172.21.15.83:0/754506715 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55855c703ce0 msgr2=0x55855c70e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:58:12.924 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.924+0000 7f8533400640 1 --2- 172.21.15.83:0/754506715 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55855c703ce0 0x55855c70e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:12.924 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.924+0000 7f8533400640 1 -- 172.21.15.83:0/754506715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55855c7033c0 msgr2=0x55855c7037a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:58:12.924 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.924+0000 7f8533400640 1 --2- 172.21.15.83:0/754506715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55855c7033c0 0x55855c7037a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:12.924 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.924+0000 7f8533400640 1 -- 172.21.15.83:0/754506715 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55855c634e10 con 0x55855c70e5a0 2024-04-24T08:58:12.924 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.925+0000 7f8533400640 1 --2- 172.21.15.83:0/754506715 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55855c70e5a0 0x55855c710990 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f853406b480 tx=0x7f8520004390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ea81c29c303be5e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:58:12.926 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.925+0000 7f8529200640 1 -- 172.21.15.83:0/754506715 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8520035030 con 0x55855c70e5a0 2024-04-24T08:58:12.926 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.925+0000 7f8530c00640 1 --2- 172.21.15.83:0/4214387445 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f85180a3d10 0x7f85180a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:12.926 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.925+0000 7f8529200640 1 -- 172.21.15.83:0/754506715 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f85200046c0 con 0x55855c70e5a0 2024-04-24T08:58:12.926 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.925+0000 7f8529200640 1 -- 172.21.15.83:0/754506715 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8520005d50 con 0x55855c70e5a0 2024-04-24T08:58:12.926 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.925+0000 7f8530c00640 1 -- 172.21.15.83:0/4214387445 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9567+0+0 (secure 0 0 0) 0x7f85240d07a0 con 0x7f8534102070 2024-04-24T08:58:12.926 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.925+0000 7f8538a00640 1 --2- 172.21.15.83:0/4214387445 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f85180a3d10 0x7f85180a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:12.926 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.926+0000 7f853d9b6740 1 -- 172.21.15.83:0/754506715 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55855c70e5a0 msgr2=0x55855c710990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:12.926 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.926+0000 7f853d9b6740 1 --2- 172.21.15.83:0/754506715 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55855c70e5a0 0x55855c710990 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f853406b480 tx=0x7f8520004390 comp rx=0 tx=0).stop 2024-04-24T08:58:12.926 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.926+0000 7f853d9b6740 1 -- 172.21.15.83:0/754506715 shutdown_connections 2024-04-24T08:58:12.926 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.926+0000 7f853d9b6740 1 --2- 172.21.15.83:0/754506715 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55855c70e5a0 0x55855c710990 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:12.927 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.926+0000 7f853d9b6740 1 --2- 172.21.15.83:0/754506715 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55855c703ce0 0x55855c70e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:12.927 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.926+0000 7f853d9b6740 1 --2- 172.21.15.83:0/754506715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55855c7033c0 0x55855c7037a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:12.927 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.926+0000 7f853d9b6740 1 -- 172.21.15.83:0/754506715 >> 172.21.15.83:0/754506715 conn(0x55855c6ff8d0 msgr2=0x55855c6ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:58:12.927 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.926+0000 7f8538a00640 1 --2- 172.21.15.83:0/4214387445 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f85180a3d10 0x7f85180a61d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f852c007e50 tx=0x7f852c007600 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:58:12.927 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.926+0000 7f853d9b6740 1 -- 172.21.15.83:0/754506715 shutdown_connections 2024-04-24T08:58:12.927 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.926+0000 7f853d9b6740 1 -- 172.21.15.83:0/754506715 wait complete. 2024-04-24T08:58:12.927 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.927+0000 7f853d9b6740 1 -- start start 2024-04-24T08:58:12.927 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.927+0000 7f853d9b6740 1 Processor -- start 2024-04-24T08:58:12.929 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.928+0000 7f853d9b6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55855c7033c0 0x55855c70b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:12.929 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.928+0000 7f853d9b6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55855c703ce0 0x55855c70bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:12.929 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.928+0000 7f853d9b6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55855c70e5a0 0x55855c70c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:12.930 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.928+0000 7f853d9b6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55855c711020 con 0x55855c70e5a0 2024-04-24T08:58:12.930 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.928+0000 7f853d9b6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55855c6232c0 con 0x55855c703ce0 2024-04-24T08:58:12.930 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.928+0000 7f853d9b6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55855c7111a0 con 0x55855c7033c0 2024-04-24T08:58:12.930 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.928+0000 7f853d9b6740 10 client.0 _ll_register_callbacks cb 0x55855c37ba00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:58:12.930 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.928+0000 7f8538a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55855c70e5a0 0x55855c70c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:12.930 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.928+0000 7f8538a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55855c70e5a0 0x55855c70c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51896/0 (socket says 172.21.15.83:51896) 2024-04-24T08:58:12.930 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.928+0000 7f8538a00640 1 -- 172.21.15.83:0/1520127445 learned_addr learned my addr 172.21.15.83:0/1520127445 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:58:12.930 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.928+0000 7f8533e00640 1 --2- 172.21.15.83:0/1520127445 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55855c7033c0 0x55855c70b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:12.930 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.928+0000 7f8533400640 1 --2- 172.21.15.83:0/1520127445 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55855c703ce0 0x55855c70bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:12.930 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.929+0000 7f8538a00640 1 -- 172.21.15.83:0/1520127445 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55855c7033c0 msgr2=0x55855c70b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:58:12.930 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.929+0000 7f8538a00640 1 --2- 172.21.15.83:0/1520127445 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55855c7033c0 0x55855c70b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:12.930 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.929+0000 7f8538a00640 1 -- 172.21.15.83:0/1520127445 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55855c703ce0 msgr2=0x55855c70bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:58:12.930 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.929+0000 7f8538a00640 1 --2- 172.21.15.83:0/1520127445 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55855c703ce0 0x55855c70bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:12.930 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.929+0000 7f8538a00640 1 -- 172.21.15.83:0/1520127445 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55855c633fd0 con 0x55855c70e5a0 2024-04-24T08:58:12.931 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.929+0000 7f8533e00640 1 --2- 172.21.15.83:0/1520127445 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55855c7033c0 0x55855c70b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:58:12.931 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.929+0000 7f8538a00640 1 --2- 172.21.15.83:0/1520127445 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55855c70e5a0 0x55855c70c050 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f852c00eda0 tx=0x7f852c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d66b1798c8221ac server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:58:12.931 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.929+0000 7f851d600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:58:12.931 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.929+0000 7f851d600640 1 -- 172.21.15.83:0/1520127445 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f852c01cd30 con 0x55855c70e5a0 2024-04-24T08:58:12.931 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.929+0000 7f851d600640 1 -- 172.21.15.83:0/1520127445 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9567+0+0 (secure 0 0 0) 0x7f852c028480 con 0x55855c70e5a0 2024-04-24T08:58:12.931 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.929+0000 7f853d9b6740 1 -- 172.21.15.83:0/1520127445 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85200046c0 con 0x55855c70e5a0 2024-04-24T08:58:12.931 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.930+0000 7f851d600640 1 -- 172.21.15.83:0/1520127445 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f852c031050 con 0x55855c70e5a0 2024-04-24T08:58:12.931 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.930+0000 7f853d9b6740 10 client.18006 fetch_fsmap learned FSMap version 33 2024-04-24T08:58:12.931 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.930+0000 7f853d9b6740 1 -- 172.21.15.83:0/1520127445 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55855c37c210 con 0x55855c70e5a0 2024-04-24T08:58:12.931 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.930+0000 7f851d600640 1 -- 172.21.15.83:0/1520127445 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f85200046c0 con 0x55855c70e5a0 2024-04-24T08:58:12.931 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.930+0000 7f851d600640 1 -- 172.21.15.83:0/1520127445 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f852c01a390 con 0x55855c70e5a0 2024-04-24T08:58:12.931 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.930+0000 7f853d9b6740 10 client.18006 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T08:58:12.931 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.930+0000 7f853d9b6740 10 client.18006 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T08:58:12.931 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.930+0000 7f853d9b6740 20 client.18006 populate_metadata read hostname 'smithi083' 2024-04-24T08:58:12.931 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.930+0000 7f853d9b6740 1 --2- 172.21.15.83:0/1520127445 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55855c71e7b0 0x55855c720bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:12.932 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.930+0000 7f853d9b6740 4 client.18006 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:58:12.932 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.930+0000 7f853d9b6740 1 -- 172.21.15.83:0/1520127445 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55855c677320 con 0x55855c71e7b0 2024-04-24T08:58:12.932 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.931+0000 7f8533e00640 1 --2- 172.21.15.83:0/1520127445 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55855c71e7b0 0x55855c720bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:12.932 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.932+0000 7f8533e00640 1 --2- 172.21.15.83:0/1520127445 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55855c71e7b0 0x55855c720bc0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d8bcf63b58c5ae server_cookie=4ce976b6333cc819 in_seq=0 out_seq=0 2024-04-24T08:58:12.932 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.932+0000 7f851d600640 10 client.18006 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:58:12.934 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.935+0000 7f851d600640 1 -- 172.21.15.83:0/1520127445 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55855c677320 con 0x55855c71e7b0 2024-04-24T08:58:12.934 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.935+0000 7f851d600640 10 client.18006 handle_command_reply: tid=0 2024-04-24T08:58:12.935 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.935+0000 7f853d9b6740 1 client.18006 shutdown 2024-04-24T08:58:12.935 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.935+0000 7f853d9b6740 1 -- 172.21.15.83:0/1520127445 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55855c70e5a0 msgr2=0x55855c70c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:12.935 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.935+0000 7f853d9b6740 1 --2- 172.21.15.83:0/1520127445 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55855c70e5a0 0x55855c70c050 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f852c00eda0 tx=0x7f852c004a40 comp rx=0 tx=0).stop 2024-04-24T08:58:12.935 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.935+0000 7f853d9b6740 1 -- 172.21.15.83:0/1520127445 shutdown_connections 2024-04-24T08:58:12.935 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.935+0000 7f853d9b6740 1 --2- 172.21.15.83:0/1520127445 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55855c71e7b0 0x55855c720bc0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:12.935 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.935+0000 7f853d9b6740 1 --2- 172.21.15.83:0/1520127445 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55855c70e5a0 0x55855c70c050 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:12.935 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.935+0000 7f853d9b6740 1 --2- 172.21.15.83:0/1520127445 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55855c703ce0 0x55855c70bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:12.935 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.935+0000 7f853d9b6740 1 --2- 172.21.15.83:0/1520127445 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55855c7033c0 0x55855c70b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:12.935 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.935+0000 7f851d600640 0 client.18006 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:58:12.935 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.935+0000 7f853d9b6740 1 -- 172.21.15.83:0/1520127445 >> 172.21.15.83:0/1520127445 conn(0x55855c6ff8d0 msgr2=0x55855c6fea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:58:12.935 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.935+0000 7f853d9b6740 1 -- 172.21.15.83:0/1520127445 shutdown_connections 2024-04-24T08:58:12.935 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.936+0000 7f853d9b6740 1 -- 172.21.15.83:0/1520127445 wait complete. 2024-04-24T08:58:12.935 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.936+0000 7f853d9b6740 20 client.18006 trim_cache size 0 max 16384 2024-04-24T08:58:12.951 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.952+0000 7f853d9b6740 1 Processor -- start 2024-04-24T08:58:12.951 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.952+0000 7f853d9b6740 1 -- start start 2024-04-24T08:58:12.952 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.952+0000 7f853d9b6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55855c7033c0 0x55855c71a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:12.952 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.952+0000 7f853d9b6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55855c703ce0 0x55855c6ff210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:12.952 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.952+0000 7f853d9b6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55855c70e5a0 0x55855c704420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:12.953 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.952+0000 7f853d9b6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f85341b1510 con 0x55855c7033c0 2024-04-24T08:58:12.953 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.952+0000 7f853d9b6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f85341b3050 con 0x55855c703ce0 2024-04-24T08:58:12.953 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.952+0000 7f853d9b6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f85341b1be0 con 0x55855c70e5a0 2024-04-24T08:58:12.953 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.953+0000 7f8533e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55855c7033c0 0x55855c71a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:12.953 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.953+0000 7f8533e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55855c7033c0 0x55855c71a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51908/0 (socket says 172.21.15.83:51908) 2024-04-24T08:58:12.953 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.953+0000 7f8533e00640 1 -- 172.21.15.83:0/2793198545 learned_addr learned my addr 172.21.15.83:0/2793198545 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:58:12.953 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.953+0000 7f8533400640 1 --2- 172.21.15.83:0/2793198545 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55855c703ce0 0x55855c6ff210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:12.953 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.953+0000 7f8533e00640 1 -- 172.21.15.83:0/2793198545 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55855c70e5a0 msgr2=0x55855c704420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:58:12.953 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.953+0000 7f8533e00640 1 --2- 172.21.15.83:0/2793198545 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55855c70e5a0 0x55855c704420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:12.953 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.953+0000 7f8533e00640 1 -- 172.21.15.83:0/2793198545 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55855c703ce0 msgr2=0x55855c6ff210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:58:12.953 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.953+0000 7f8533e00640 1 --2- 172.21.15.83:0/2793198545 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55855c703ce0 0x55855c6ff210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:12.953 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.953+0000 7f8533e00640 1 -- 172.21.15.83:0/2793198545 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55855c70a580 con 0x55855c7033c0 2024-04-24T08:58:12.953 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.953+0000 7f8533400640 1 --2- 172.21.15.83:0/2793198545 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55855c703ce0 0x55855c6ff210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:58:12.953 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.953+0000 7f8533e00640 1 --2- 172.21.15.83:0/2793198545 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55855c7033c0 0x55855c71a080 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f8524009610 tx=0x7f85240d6530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ea419eef4b57f3a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:58:12.954 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.954+0000 7f851fe00640 1 -- 172.21.15.83:0/2793198545 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8524020040 con 0x55855c7033c0 2024-04-24T08:58:12.955 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.954+0000 7f851fe00640 1 -- 172.21.15.83:0/2793198545 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8524009c30 con 0x55855c7033c0 2024-04-24T08:58:12.955 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.954+0000 7f851fe00640 1 -- 172.21.15.83:0/2793198545 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85240a4f40 con 0x55855c7033c0 2024-04-24T08:58:12.955 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.955+0000 7f853d9b6740 1 -- 172.21.15.83:0/2793198545 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55855c7033c0 msgr2=0x55855c71a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:12.955 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.955+0000 7f853d9b6740 1 --2- 172.21.15.83:0/2793198545 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55855c7033c0 0x55855c71a080 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f8524009610 tx=0x7f85240d6530 comp rx=0 tx=0).stop 2024-04-24T08:58:12.955 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.955+0000 7f853d9b6740 1 -- 172.21.15.83:0/2793198545 shutdown_connections 2024-04-24T08:58:12.955 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.955+0000 7f853d9b6740 1 --2- 172.21.15.83:0/2793198545 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55855c70e5a0 0x55855c704420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:12.955 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.955+0000 7f853d9b6740 1 --2- 172.21.15.83:0/2793198545 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55855c703ce0 0x55855c6ff210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:12.955 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.955+0000 7f853d9b6740 1 --2- 172.21.15.83:0/2793198545 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55855c7033c0 0x55855c71a080 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:12.955 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.955+0000 7f853d9b6740 1 -- 172.21.15.83:0/2793198545 >> 172.21.15.83:0/2793198545 conn(0x55855c6ff8d0 msgr2=0x55855c70be20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:58:12.955 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.955+0000 7f853d9b6740 1 -- 172.21.15.83:0/2793198545 shutdown_connections 2024-04-24T08:58:12.955 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.955+0000 7f853d9b6740 1 -- 172.21.15.83:0/2793198545 wait complete. 2024-04-24T08:58:12.955 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.955+0000 7f853d9b6740 1 -- start start 2024-04-24T08:58:12.955 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.956+0000 7f853d9b6740 1 Processor -- start 2024-04-24T08:58:12.957 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.956+0000 7f853d9b6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55855c7033c0 0x55855c7127b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:12.957 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.956+0000 7f853d9b6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55855c703ce0 0x55855c712cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:12.958 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.956+0000 7f853d9b6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55855c70e5a0 0x55855c783d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:12.958 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.956+0000 7f853d9b6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f85341b2980 con 0x55855c7033c0 2024-04-24T08:58:12.958 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.956+0000 7f853d9b6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f85341b22b0 con 0x55855c703ce0 2024-04-24T08:58:12.958 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.956+0000 7f853d9b6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f85341af4f0 con 0x55855c70e5a0 2024-04-24T08:58:12.958 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.956+0000 7f853d9b6740 10 client.0 _ll_register_callbacks cb 0x7f852c007840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:58:12.958 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.956+0000 7f8533e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55855c7033c0 0x55855c7127b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:12.958 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.956+0000 7f8533e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55855c7033c0 0x55855c7127b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51918/0 (socket says 172.21.15.83:51918) 2024-04-24T08:58:12.958 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.956+0000 7f8533e00640 1 -- 172.21.15.83:0/606894027 learned_addr learned my addr 172.21.15.83:0/606894027 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:58:12.958 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.957+0000 7f8533e00640 1 -- 172.21.15.83:0/606894027 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55855c70e5a0 msgr2=0x55855c783d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:58:12.958 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.957+0000 7f8533e00640 1 --2- 172.21.15.83:0/606894027 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55855c70e5a0 0x55855c783d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:12.958 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.957+0000 7f8533e00640 1 -- 172.21.15.83:0/606894027 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55855c703ce0 msgr2=0x55855c712cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:58:12.958 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.957+0000 7f8533e00640 1 --2- 172.21.15.83:0/606894027 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55855c703ce0 0x55855c712cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:12.958 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.957+0000 7f8533e00640 1 -- 172.21.15.83:0/606894027 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55855c448620 con 0x55855c7033c0 2024-04-24T08:58:12.958 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.957+0000 7f8533e00640 1 --2- 172.21.15.83:0/606894027 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55855c7033c0 0x55855c7127b0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f8524098730 tx=0x7f85240a07d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5421089338f15208 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:58:12.958 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.957+0000 7f851d600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:58:12.959 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.957+0000 7f851d600640 1 -- 172.21.15.83:0/606894027 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8524085020 con 0x55855c7033c0 2024-04-24T08:58:12.959 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.957+0000 7f851d600640 1 -- 172.21.15.83:0/606894027 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9567+0+0 (secure 0 0 0) 0x7f8524098940 con 0x55855c7033c0 2024-04-24T08:58:12.959 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.957+0000 7f853d9b6740 1 -- 172.21.15.83:0/606894027 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55855c671190 con 0x55855c7033c0 2024-04-24T08:58:12.959 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.957+0000 7f851d600640 1 -- 172.21.15.83:0/606894027 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f8524020070 con 0x55855c7033c0 2024-04-24T08:58:12.959 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.957+0000 7f853d9b6740 10 client.18012 fetch_fsmap learned FSMap version 33 2024-04-24T08:58:12.959 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.957+0000 7f853d9b6740 1 -- 172.21.15.83:0/606894027 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55855c785400 con 0x55855c7033c0 2024-04-24T08:58:12.959 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.958+0000 7f851d600640 1 -- 172.21.15.83:0/606894027 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55855c671190 con 0x55855c7033c0 2024-04-24T08:58:12.959 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.958+0000 7f851d600640 1 -- 172.21.15.83:0/606894027 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f85240cdd00 con 0x55855c7033c0 2024-04-24T08:58:12.959 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.958+0000 7f853d9b6740 10 client.18012 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T08:58:12.959 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.958+0000 7f853d9b6740 10 client.18012 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T08:58:12.959 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.958+0000 7f853d9b6740 20 client.18012 populate_metadata read hostname 'smithi083' 2024-04-24T08:58:12.959 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.958+0000 7f853d9b6740 1 --2- 172.21.15.83:0/606894027 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55855c785b40 0x55855c787f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:12.959 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.958+0000 7f853d9b6740 4 client.18012 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T08:58:12.959 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.958+0000 7f853d9b6740 1 -- 172.21.15.83:0/606894027 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55855c5ed4b0 con 0x55855c785b40 2024-04-24T08:58:12.960 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.959+0000 7f8533400640 1 --2- 172.21.15.83:0/606894027 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55855c785b40 0x55855c787f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:12.960 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.960+0000 7f8533400640 1 --2- 172.21.15.83:0/606894027 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55855c785b40 0x55855c787f80 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f63e84112bbbdd2 server_cookie=654eb8a3113e7847 in_seq=0 out_seq=0 2024-04-24T08:58:12.960 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:12.960+0000 7f851d600640 10 client.18012 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:58:13.020 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:13.021+0000 7f851d600640 1 -- 172.21.15.83:0/606894027 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55855c5ed4b0 con 0x55855c785b40 2024-04-24T08:58:13.021 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:13.021+0000 7f851d600640 10 client.18012 handle_command_reply: tid=0 2024-04-24T08:58:13.021 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:13.021+0000 7f853d9b6740 1 client.18012 shutdown 2024-04-24T08:58:13.021 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:13.021+0000 7f853d9b6740 1 -- 172.21.15.83:0/606894027 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55855c7033c0 msgr2=0x55855c7127b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:13.021 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:13.021+0000 7f853d9b6740 1 --2- 172.21.15.83:0/606894027 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55855c7033c0 0x55855c7127b0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f8524098730 tx=0x7f85240a07d0 comp rx=0 tx=0).stop 2024-04-24T08:58:13.021 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:13.021+0000 7f853d9b6740 1 -- 172.21.15.83:0/606894027 shutdown_connections 2024-04-24T08:58:13.021 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:13.021+0000 7f853d9b6740 1 --2- 172.21.15.83:0/606894027 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55855c785b40 0x55855c787f80 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:13.021 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:13.021+0000 7f853d9b6740 1 --2- 172.21.15.83:0/606894027 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55855c70e5a0 0x55855c783d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:13.021 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:13.021+0000 7f853d9b6740 1 --2- 172.21.15.83:0/606894027 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55855c703ce0 0x55855c712cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:13.021 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:13.021+0000 7f853d9b6740 1 --2- 172.21.15.83:0/606894027 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55855c7033c0 0x55855c7127b0 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:13.021 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:13.021+0000 7f851d600640 0 client.18012 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:58:13.021 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:13.021+0000 7f853d9b6740 1 -- 172.21.15.83:0/606894027 >> 172.21.15.83:0/606894027 conn(0x55855c6ff8d0 msgr2=0x55855c704d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:58:13.021 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:13.022+0000 7f853d9b6740 1 -- 172.21.15.83:0/606894027 shutdown_connections 2024-04-24T08:58:13.021 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:13.022+0000 7f853d9b6740 1 -- 172.21.15.83:0/606894027 wait complete. 2024-04-24T08:58:13.022 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:13.022+0000 7f853d9b6740 20 client.18012 trim_cache size 0 max 16384 2024-04-24T08:58:13.022 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T08:58:13.022 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T08:58:13.022 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T08:58:13.022 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T08:58:13.023 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:13.023+0000 7f8539400640 1 -- 172.21.15.83:0/4214387445 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f85180a3d10 msgr2=0x7f85180a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:13.023 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:13.023+0000 7f8539400640 1 --2- 172.21.15.83:0/4214387445 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f85180a3d10 0x7f85180a61d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f852c007e50 tx=0x7f852c007600 comp rx=0 tx=0).stop 2024-04-24T08:58:13.023 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:13.023+0000 7f8539400640 1 -- 172.21.15.83:0/4214387445 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8534102070 msgr2=0x7f85341a05d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:13.023 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:13.023+0000 7f8539400640 1 --2- 172.21.15.83:0/4214387445 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8534102070 0x7f85341a05d0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f852400d770 tx=0x7f852400dc30 comp rx=0 tx=0).stop 2024-04-24T08:58:13.023 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:13.023+0000 7f8539400640 1 -- 172.21.15.83:0/4214387445 shutdown_connections 2024-04-24T08:58:13.023 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:13.023+0000 7f8539400640 1 --2- 172.21.15.83:0/4214387445 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f85180a3d10 0x7f85180a61d0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:13.023 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:13.023+0000 7f8539400640 1 --2- 172.21.15.83:0/4214387445 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f853419a690 0x7f853419ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:13.023 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:13.023+0000 7f8539400640 1 --2- 172.21.15.83:0/4214387445 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8534107140 0x7f85341a0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:13.024 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:13.023+0000 7f8539400640 1 --2- 172.21.15.83:0/4214387445 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8534102070 0x7f85341a05d0 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:13.024 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:13.023+0000 7f8539400640 1 -- 172.21.15.83:0/4214387445 >> 172.21.15.83:0/4214387445 conn(0x7f85340fd950 msgr2=0x7f85340ffcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:58:13.024 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:13.024+0000 7f8539400640 1 -- 172.21.15.83:0/4214387445 shutdown_connections 2024-04-24T08:58:13.024 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:13.024+0000 7f8539400640 1 -- 172.21.15.83:0/4214387445 wait complete. 2024-04-24T08:58:13.585 INFO:tasks.workunit.client.0.smithi083.stdout: 1 34556 0.62 MB/sec execute 167 sec latency 61.641 ms 2024-04-24T08:58:13.954 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:58:13 smithi138 ceph-mon[31441]: pgmap v204: 129 pgs: 129 active+clean; 42 MiB data, 605 MiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 319 KiB/s wr, 54 op/s 2024-04-24T08:58:13.956 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:58:13 smithi189 ceph-mon[31791]: pgmap v204: 129 pgs: 129 active+clean; 42 MiB data, 605 MiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 319 KiB/s wr, 54 op/s 2024-04-24T08:58:14.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:58:13 smithi083 ceph-mon[27907]: pgmap v204: 129 pgs: 129 active+clean; 42 MiB data, 605 MiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 319 KiB/s wr, 54 op/s 2024-04-24T08:58:14.585 INFO:tasks.workunit.client.0.smithi083.stdout: 1 34743 0.63 MB/sec execute 168 sec latency 53.874 ms 2024-04-24T08:58:15.585 INFO:tasks.workunit.client.0.smithi083.stdout: 1 34937 0.64 MB/sec execute 169 sec latency 66.507 ms 2024-04-24T08:58:15.954 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:58:15 smithi138 ceph-mon[31441]: pgmap v205: 129 pgs: 129 active+clean; 42 MiB data, 605 MiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 259 KiB/s wr, 44 op/s 2024-04-24T08:58:15.956 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:58:15 smithi189 ceph-mon[31791]: pgmap v205: 129 pgs: 129 active+clean; 42 MiB data, 605 MiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 259 KiB/s wr, 44 op/s 2024-04-24T08:58:16.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:58:15 smithi083 ceph-mon[27907]: pgmap v205: 129 pgs: 129 active+clean; 42 MiB data, 605 MiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 259 KiB/s wr, 44 op/s 2024-04-24T08:58:16.586 INFO:tasks.workunit.client.0.smithi083.stdout: 1 35161 0.64 MB/sec execute 170 sec latency 54.946 ms 2024-04-24T08:58:17.586 INFO:tasks.workunit.client.0.smithi083.stdout: 1 35256 0.64 MB/sec execute 171 sec latency 53.575 ms 2024-04-24T08:58:17.954 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:58:17 smithi138 ceph-mon[31441]: pgmap v206: 129 pgs: 129 active+clean; 42 MiB data, 605 MiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 259 KiB/s wr, 44 op/s 2024-04-24T08:58:17.956 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:58:17 smithi189 ceph-mon[31791]: pgmap v206: 129 pgs: 129 active+clean; 42 MiB data, 605 MiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 259 KiB/s wr, 44 op/s 2024-04-24T08:58:18.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:58:17 smithi083 ceph-mon[27907]: pgmap v206: 129 pgs: 129 active+clean; 42 MiB data, 605 MiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 259 KiB/s wr, 44 op/s 2024-04-24T08:58:18.586 INFO:tasks.workunit.client.0.smithi083.stdout: 1 35290 0.64 MB/sec execute 172 sec latency 54.030 ms 2024-04-24T08:58:19.586 INFO:tasks.workunit.client.0.smithi083.stdout: 1 35338 0.64 MB/sec execute 173 sec latency 51.066 ms 2024-04-24T08:58:19.954 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:58:19 smithi138 ceph-mon[31441]: pgmap v207: 129 pgs: 129 active+clean; 45 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 499 KiB/s wr, 50 op/s 2024-04-24T08:58:19.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:58:19 smithi189 ceph-mon[31791]: pgmap v207: 129 pgs: 129 active+clean; 45 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 499 KiB/s wr, 50 op/s 2024-04-24T08:58:20.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:58:19 smithi083 ceph-mon[27907]: pgmap v207: 129 pgs: 129 active+clean; 45 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 499 KiB/s wr, 50 op/s 2024-04-24T08:58:20.586 INFO:tasks.workunit.client.0.smithi083.stdout: 1 35515 0.65 MB/sec execute 174 sec latency 133.999 ms 2024-04-24T08:58:21.586 INFO:tasks.workunit.client.0.smithi083.stdout: 1 35629 0.65 MB/sec execute 175 sec latency 55.343 ms 2024-04-24T08:58:21.954 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:58:21 smithi138 ceph-mon[31441]: pgmap v208: 129 pgs: 129 active+clean; 45 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 449 KiB/s wr, 45 op/s 2024-04-24T08:58:21.956 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:58:21 smithi189 ceph-mon[31791]: pgmap v208: 129 pgs: 129 active+clean; 45 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 449 KiB/s wr, 45 op/s 2024-04-24T08:58:22.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:58:21 smithi083 ceph-mon[27907]: pgmap v208: 129 pgs: 129 active+clean; 45 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 449 KiB/s wr, 45 op/s 2024-04-24T08:58:22.586 INFO:tasks.workunit.client.0.smithi083.stdout: 1 35738 0.65 MB/sec execute 176 sec latency 48.376 ms 2024-04-24T08:58:23.586 INFO:tasks.workunit.client.0.smithi083.stdout: 1 35794 0.65 MB/sec execute 177 sec latency 44.949 ms 2024-04-24T08:58:23.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:58:23 smithi138 ceph-mon[31441]: pgmap v209: 129 pgs: 129 active+clean; 46 MiB data, 653 MiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 647 KiB/s wr, 62 op/s 2024-04-24T08:58:23.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:58:23 smithi189 ceph-mon[31791]: pgmap v209: 129 pgs: 129 active+clean; 46 MiB data, 653 MiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 647 KiB/s wr, 62 op/s 2024-04-24T08:58:24.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:58:23 smithi083 ceph-mon[27907]: pgmap v209: 129 pgs: 129 active+clean; 46 MiB data, 653 MiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 647 KiB/s wr, 62 op/s 2024-04-24T08:58:24.586 INFO:tasks.workunit.client.0.smithi083.stdout: 1 35861 0.64 MB/sec execute 178 sec latency 51.168 ms 2024-04-24T08:58:25.586 INFO:tasks.workunit.client.0.smithi083.stdout: 1 35917 0.64 MB/sec execute 179 sec latency 46.414 ms 2024-04-24T08:58:25.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:58:25 smithi138 ceph-mon[31441]: pgmap v210: 129 pgs: 129 active+clean; 46 MiB data, 653 MiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 495 KiB/s wr, 43 op/s 2024-04-24T08:58:25.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:58:25 smithi189 ceph-mon[31791]: pgmap v210: 129 pgs: 129 active+clean; 46 MiB data, 653 MiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 495 KiB/s wr, 43 op/s 2024-04-24T08:58:26.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:58:25 smithi083 ceph-mon[27907]: pgmap v210: 129 pgs: 129 active+clean; 46 MiB data, 653 MiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 495 KiB/s wr, 43 op/s 2024-04-24T08:58:26.177 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.176+0000 7fe1ef200640 1 -- 172.21.15.83:0/2158240045 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1e8108ec0 msgr2=0x7fe1e8115300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:26.177 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.176+0000 7fe1ef200640 1 --2- 172.21.15.83:0/2158240045 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1e8108ec0 0x7fe1e8115300 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7fe1e4009cd0 tx=0x7fe1e402a8f0 comp rx=0 tx=0).stop 2024-04-24T08:58:26.177 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.177+0000 7fe1ef200640 1 -- 172.21.15.83:0/2158240045 shutdown_connections 2024-04-24T08:58:26.177 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.177+0000 7fe1ef200640 1 --2- 172.21.15.83:0/2158240045 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1e8108ec0 0x7fe1e8115300 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:26.177 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.177+0000 7fe1ef200640 1 --2- 172.21.15.83:0/2158240045 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe1e8108540 0x7fe1e8108980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:26.177 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.177+0000 7fe1ef200640 1 --2- 172.21.15.83:0/2158240045 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1e810e4d0 0x7fe1e810e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:26.177 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.177+0000 7fe1ef200640 1 -- 172.21.15.83:0/2158240045 >> 172.21.15.83:0/2158240045 conn(0x7fe1e8077de0 msgr2=0x7fe1e8106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:58:26.177 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.177+0000 7fe1ef200640 1 -- 172.21.15.83:0/2158240045 shutdown_connections 2024-04-24T08:58:26.178 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.177+0000 7fe1ef200640 1 -- 172.21.15.83:0/2158240045 wait complete. 2024-04-24T08:58:26.180 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.178+0000 7fe1ef200640 1 Processor -- start 2024-04-24T08:58:26.180 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.178+0000 7fe1ef200640 1 -- start start 2024-04-24T08:58:26.180 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.178+0000 7fe1ef200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1e8108540 0x7fe1e81a4870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:26.180 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.178+0000 7fe1ef200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe1e8108ec0 0x7fe1e81a4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:26.180 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.178+0000 7fe1ef200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1e810e4d0 0x7fe1e819ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:26.180 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.178+0000 7fe1ef200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe1e8118700 con 0x7fe1e8108540 2024-04-24T08:58:26.180 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.178+0000 7fe1ef200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe1e8118580 con 0x7fe1e810e4d0 2024-04-24T08:58:26.180 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.178+0000 7fe1ef200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe1e8118880 con 0x7fe1e8108ec0 2024-04-24T08:58:26.180 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.179+0000 7fe1ede00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1e8108540 0x7fe1e81a4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:26.180 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.179+0000 7fe1ede00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1e8108540 0x7fe1e81a4870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58530/0 (socket says 172.21.15.83:58530) 2024-04-24T08:58:26.181 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.179+0000 7fe1ede00640 1 -- 172.21.15.83:0/2716114712 learned_addr learned my addr 172.21.15.83:0/2716114712 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:58:26.181 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.179+0000 7fe1ee800640 1 --2- 172.21.15.83:0/2716114712 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1e810e4d0 0x7fe1e819ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:26.181 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.179+0000 7fe1ed400640 1 --2- 172.21.15.83:0/2716114712 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe1e8108ec0 0x7fe1e81a4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:26.181 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.179+0000 7fe1ede00640 1 -- 172.21.15.83:0/2716114712 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe1e8108ec0 msgr2=0x7fe1e81a4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:26.181 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.179+0000 7fe1ede00640 1 --2- 172.21.15.83:0/2716114712 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe1e8108ec0 0x7fe1e81a4db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:26.181 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.179+0000 7fe1ede00640 1 -- 172.21.15.83:0/2716114712 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1e810e4d0 msgr2=0x7fe1e819ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:26.181 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.179+0000 7fe1ede00640 1 --2- 172.21.15.83:0/2716114712 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1e810e4d0 0x7fe1e819ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:26.181 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.179+0000 7fe1ede00640 1 -- 172.21.15.83:0/2716114712 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1e819f1a0 con 0x7fe1e8108540 2024-04-24T08:58:26.181 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.179+0000 7fe1ed400640 1 --2- 172.21.15.83:0/2716114712 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe1e8108ec0 0x7fe1e81a4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:58:26.181 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.180+0000 7fe1ede00640 1 --2- 172.21.15.83:0/2716114712 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1e8108540 0x7fe1e81a4870 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc00d950 tx=0x7fe1dc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:58:26.181 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.180+0000 7fe1e2a00640 1 -- 172.21.15.83:0/2716114712 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1dc017070 con 0x7fe1e8108540 2024-04-24T08:58:26.181 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.180+0000 7fe1e2a00640 1 -- 172.21.15.83:0/2716114712 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe1dc009c30 con 0x7fe1e8108540 2024-04-24T08:58:26.181 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.180+0000 7fe1ef200640 1 -- 172.21.15.83:0/2716114712 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1e819f490 con 0x7fe1e8108540 2024-04-24T08:58:26.181 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.180+0000 7fe1e2a00640 1 -- 172.21.15.83:0/2716114712 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1dc00f620 con 0x7fe1e8108540 2024-04-24T08:58:26.181 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.180+0000 7fe1ef200640 1 -- 172.21.15.83:0/2716114712 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1e81ab730 con 0x7fe1e8108540 2024-04-24T08:58:26.184 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.183+0000 7fe1f3632740 1 Processor -- start 2024-04-24T08:58:26.184 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.183+0000 7fe1f3632740 1 -- start start 2024-04-24T08:58:26.184 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.183+0000 7fe1e2a00640 1 -- 172.21.15.83:0/2716114712 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe1dc0109a0 con 0x7fe1e8108540 2024-04-24T08:58:26.184 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.184+0000 7fe1f3632740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563962e43c0 0x5563962e47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:26.184 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.184+0000 7fe1f3632740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563962e4ce0 0x5563962ef060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:26.184 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.184+0000 7fe1f3632740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563962ef5a0 0x5563962f1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:26.185 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.184+0000 7fe1f3632740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55639624a900 con 0x5563962e4ce0 2024-04-24T08:58:26.185 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.184+0000 7fe1f3632740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5563961f0f20 con 0x5563962ef5a0 2024-04-24T08:58:26.185 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.184+0000 7fe1f3632740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5563962468c0 con 0x5563962e43c0 2024-04-24T08:58:26.185 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.184+0000 7fe1ede00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563962e4ce0 0x5563962ef060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:26.185 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.184+0000 7fe1ede00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563962e4ce0 0x5563962ef060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58544/0 (socket says 172.21.15.83:58544) 2024-04-24T08:58:26.185 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.184+0000 7fe1ede00640 1 -- 172.21.15.83:0/3497173002 learned_addr learned my addr 172.21.15.83:0/3497173002 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:58:26.185 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.184+0000 7fe1ee800640 1 --2- 172.21.15.83:0/3497173002 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563962e43c0 0x5563962e47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:26.185 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.184+0000 7fe1ed400640 1 --2- 172.21.15.83:0/3497173002 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563962ef5a0 0x5563962f1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.185+0000 7fe1ede00640 1 -- 172.21.15.83:0/3497173002 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563962e43c0 msgr2=0x5563962e47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:58:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.185+0000 7fe1ede00640 1 --2- 172.21.15.83:0/3497173002 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563962e43c0 0x5563962e47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.185+0000 7fe1ede00640 1 -- 172.21.15.83:0/3497173002 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563962ef5a0 msgr2=0x5563962f1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:58:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.185+0000 7fe1ede00640 1 --2- 172.21.15.83:0/3497173002 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563962ef5a0 0x5563962f1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.185+0000 7fe1ede00640 1 -- 172.21.15.83:0/3497173002 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556396215e10 con 0x5563962e4ce0 2024-04-24T08:58:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.185+0000 7fe1ede00640 1 --2- 172.21.15.83:0/3497173002 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563962e4ce0 0x5563962ef060 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc0cdac0 tx=0x7fe1dc0d8750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9963fa10ffdf64cc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:58:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.185+0000 7fe1e2a00640 1 --2- 172.21.15.83:0/2716114712 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe1d00a3d10 0x7fe1d00a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.185+0000 7fe1d7000640 1 -- 172.21.15.83:0/3497173002 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1dc098670 con 0x5563962e4ce0 2024-04-24T08:58:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.186+0000 7fe1d7000640 1 -- 172.21.15.83:0/3497173002 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe1dc0dd070 con 0x5563962e4ce0 2024-04-24T08:58:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.186+0000 7fe1d7000640 1 -- 172.21.15.83:0/3497173002 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1dc09bb50 con 0x5563962e4ce0 2024-04-24T08:58:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.186+0000 7fe1e2a00640 1 -- 172.21.15.83:0/2716114712 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9567+0+0 (secure 0 0 0) 0x7fe1dc0d08a0 con 0x7fe1e8108540 2024-04-24T08:58:26.188 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.186+0000 7fe1ee800640 1 --2- 172.21.15.83:0/2716114712 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe1d00a3d10 0x7fe1d00a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:26.188 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.187+0000 7fe1f3632740 1 -- 172.21.15.83:0/3497173002 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563962e4ce0 msgr2=0x5563962ef060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:26.188 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.187+0000 7fe1ee800640 1 --2- 172.21.15.83:0/2716114712 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe1d00a3d10 0x7fe1d00a61d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe1e4004af0 tx=0x7fe1e4005d80 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:58:26.188 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.187+0000 7fe1f3632740 1 --2- 172.21.15.83:0/3497173002 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563962e4ce0 0x5563962ef060 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc0cdac0 tx=0x7fe1dc0d8750 comp rx=0 tx=0).stop 2024-04-24T08:58:26.188 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.187+0000 7fe1f3632740 1 -- 172.21.15.83:0/3497173002 shutdown_connections 2024-04-24T08:58:26.188 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.187+0000 7fe1f3632740 1 --2- 172.21.15.83:0/3497173002 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563962ef5a0 0x5563962f1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:26.188 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.187+0000 7fe1f3632740 1 --2- 172.21.15.83:0/3497173002 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563962e4ce0 0x5563962ef060 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:26.188 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.187+0000 7fe1f3632740 1 --2- 172.21.15.83:0/3497173002 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563962e43c0 0x5563962e47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:26.188 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.187+0000 7fe1f3632740 1 -- 172.21.15.83:0/3497173002 >> 172.21.15.83:0/3497173002 conn(0x5563962e08d0 msgr2=0x5563962e0cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:58:26.188 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.187+0000 7fe1f3632740 1 -- 172.21.15.83:0/3497173002 shutdown_connections 2024-04-24T08:58:26.188 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.187+0000 7fe1f3632740 1 -- 172.21.15.83:0/3497173002 wait complete. 2024-04-24T08:58:26.192 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.187+0000 7fe1f3632740 1 -- start start 2024-04-24T08:58:26.192 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.188+0000 7fe1f3632740 1 Processor -- start 2024-04-24T08:58:26.192 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.188+0000 7fe1f3632740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563962e43c0 0x5563962ec5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:26.192 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.188+0000 7fe1f3632740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563962e4ce0 0x5563962ecae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:26.192 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.188+0000 7fe1f3632740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563962ef5a0 0x5563962ed020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:26.192 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.188+0000 7fe1f3632740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5563962f2140 con 0x5563962ef5a0 2024-04-24T08:58:26.192 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.188+0000 7fe1f3632740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5563962042c0 con 0x5563962e4ce0 2024-04-24T08:58:26.192 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.188+0000 7fe1f3632740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5563962f22c0 con 0x5563962e43c0 2024-04-24T08:58:26.192 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.188+0000 7fe1f3632740 10 client.0 _ll_register_callbacks cb 0x556395f5ca00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:58:26.192 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.188+0000 7fe1ee800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563962ef5a0 0x5563962ed020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:26.192 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.189+0000 7fe1ee800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563962ef5a0 0x5563962ed020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58552/0 (socket says 172.21.15.83:58552) 2024-04-24T08:58:26.192 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.189+0000 7fe1ee800640 1 -- 172.21.15.83:0/3651345344 learned_addr learned my addr 172.21.15.83:0/3651345344 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:58:26.192 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.189+0000 7fe1ede00640 1 --2- 172.21.15.83:0/3651345344 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563962e43c0 0x5563962ec5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:26.193 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.189+0000 7fe1ed400640 1 --2- 172.21.15.83:0/3651345344 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563962e4ce0 0x5563962ecae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:26.193 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.189+0000 7fe1ee800640 1 -- 172.21.15.83:0/3651345344 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563962e43c0 msgr2=0x5563962ec5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:58:26.193 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.189+0000 7fe1ee800640 1 --2- 172.21.15.83:0/3651345344 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563962e43c0 0x5563962ec5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:26.193 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.189+0000 7fe1ee800640 1 -- 172.21.15.83:0/3651345344 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563962e4ce0 msgr2=0x5563962ecae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:58:26.193 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.189+0000 7fe1ee800640 1 --2- 172.21.15.83:0/3651345344 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563962e4ce0 0x5563962ecae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:26.193 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.189+0000 7fe1ee800640 1 -- 172.21.15.83:0/3651345344 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556396214fd0 con 0x5563962ef5a0 2024-04-24T08:58:26.193 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.189+0000 7fe1ee800640 1 --2- 172.21.15.83:0/3651345344 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563962ef5a0 0x5563962ed020 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7fe1e4009cd0 tx=0x7fe1e402a8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2c69bfb8f3e31e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:58:26.193 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.189+0000 7fe1cb400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:58:26.193 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.190+0000 7fe1cb400640 1 -- 172.21.15.83:0/3651345344 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e4052440 con 0x5563962ef5a0 2024-04-24T08:58:26.193 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.190+0000 7fe1cb400640 1 -- 172.21.15.83:0/3651345344 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9567+0+0 (secure 0 0 0) 0x7fe1e4031440 con 0x5563962ef5a0 2024-04-24T08:58:26.193 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.190+0000 7fe1f3632740 1 -- 172.21.15.83:0/3651345344 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1dc0dd070 con 0x5563962ef5a0 2024-04-24T08:58:26.193 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.190+0000 7fe1cb400640 1 -- 172.21.15.83:0/3651345344 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fe1e4031640 con 0x5563962ef5a0 2024-04-24T08:58:26.193 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.190+0000 7fe1f3632740 10 client.18024 fetch_fsmap learned FSMap version 33 2024-04-24T08:58:26.193 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.190+0000 7fe1f3632740 1 -- 172.21.15.83:0/3651345344 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x556395f5d210 con 0x5563962ef5a0 2024-04-24T08:58:26.194 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.190+0000 7fe1cb400640 1 -- 172.21.15.83:0/3651345344 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe1dc0dd070 con 0x5563962ef5a0 2024-04-24T08:58:26.194 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.191+0000 7fe1cb400640 1 -- 172.21.15.83:0/3651345344 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fe1e4050070 con 0x5563962ef5a0 2024-04-24T08:58:26.194 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.191+0000 7fe1f3632740 10 client.18024 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T08:58:26.194 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.191+0000 7fe1f3632740 10 client.18024 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T08:58:26.194 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.191+0000 7fe1f3632740 20 client.18024 populate_metadata read hostname 'smithi083' 2024-04-24T08:58:26.194 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.191+0000 7fe1f3632740 1 --2- 172.21.15.83:0/3651345344 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5563962ff720 0x556396301b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:26.194 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.191+0000 7fe1f3632740 4 client.18024 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:58:26.194 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.191+0000 7fe1f3632740 1 -- 172.21.15.83:0/3651345344 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556396258320 con 0x5563962ff720 2024-04-24T08:58:26.194 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.191+0000 7fe1ede00640 1 --2- 172.21.15.83:0/3651345344 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5563962ff720 0x556396301b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:26.195 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.192+0000 7fe1ede00640 1 --2- 172.21.15.83:0/3651345344 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5563962ff720 0x556396301b50 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e2ed6ef71887968 server_cookie=3b9ebe01f71eb308 in_seq=0 out_seq=0 2024-04-24T08:58:26.195 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.192+0000 7fe1cb400640 10 client.18024 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:58:26.196 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.194+0000 7fe1cb400640 1 -- 172.21.15.83:0/3651345344 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556396258320 con 0x5563962ff720 2024-04-24T08:58:26.196 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.194+0000 7fe1cb400640 10 client.18024 handle_command_reply: tid=0 2024-04-24T08:58:26.196 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.194+0000 7fe1f3632740 1 client.18024 shutdown 2024-04-24T08:58:26.196 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.194+0000 7fe1f3632740 1 -- 172.21.15.83:0/3651345344 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563962ef5a0 msgr2=0x5563962ed020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:26.196 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.194+0000 7fe1f3632740 1 --2- 172.21.15.83:0/3651345344 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563962ef5a0 0x5563962ed020 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7fe1e4009cd0 tx=0x7fe1e402a8f0 comp rx=0 tx=0).stop 2024-04-24T08:58:26.196 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.195+0000 7fe1f3632740 1 -- 172.21.15.83:0/3651345344 shutdown_connections 2024-04-24T08:58:26.196 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.195+0000 7fe1f3632740 1 --2- 172.21.15.83:0/3651345344 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5563962ff720 0x556396301b50 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:26.196 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.195+0000 7fe1f3632740 1 --2- 172.21.15.83:0/3651345344 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563962ef5a0 0x5563962ed020 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:26.196 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.195+0000 7fe1f3632740 1 --2- 172.21.15.83:0/3651345344 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563962e4ce0 0x5563962ecae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:26.196 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.195+0000 7fe1f3632740 1 --2- 172.21.15.83:0/3651345344 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563962e43c0 0x5563962ec5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:26.196 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.195+0000 7fe1f3632740 1 -- 172.21.15.83:0/3651345344 >> 172.21.15.83:0/3651345344 conn(0x5563962e08d0 msgr2=0x5563962dfa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:58:26.196 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.195+0000 7fe1cb400640 0 client.18024 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:58:26.196 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.195+0000 7fe1f3632740 1 -- 172.21.15.83:0/3651345344 shutdown_connections 2024-04-24T08:58:26.196 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.195+0000 7fe1f3632740 1 -- 172.21.15.83:0/3651345344 wait complete. 2024-04-24T08:58:26.196 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.195+0000 7fe1f3632740 20 client.18024 trim_cache size 0 max 16384 2024-04-24T08:58:26.212 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.212+0000 7fe1f3632740 1 Processor -- start 2024-04-24T08:58:26.212 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.212+0000 7fe1f3632740 1 -- start start 2024-04-24T08:58:26.212 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.212+0000 7fe1f3632740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563962e43c0 0x5563962eecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:26.212 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.212+0000 7fe1f3632740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563962e4ce0 0x556396301b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:26.213 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.212+0000 7fe1f3632740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563962ef5a0 0x5563962fb110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:26.213 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.212+0000 7fe1f3632740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe1e81b5ad0 con 0x5563962e4ce0 2024-04-24T08:58:26.213 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.212+0000 7fe1f3632740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe1e81b7610 con 0x5563962ef5a0 2024-04-24T08:58:26.213 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.212+0000 7fe1f3632740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe1e81b61a0 con 0x5563962e43c0 2024-04-24T08:58:26.215 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.213+0000 7fe1ed400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563962e4ce0 0x556396301b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:26.215 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.213+0000 7fe1ed400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563962e4ce0 0x556396301b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58560/0 (socket says 172.21.15.83:58560) 2024-04-24T08:58:26.215 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.213+0000 7fe1ed400640 1 -- 172.21.15.83:0/115130586 learned_addr learned my addr 172.21.15.83:0/115130586 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:58:26.215 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.213+0000 7fe1ede00640 1 --2- 172.21.15.83:0/115130586 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563962e43c0 0x5563962eecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:26.215 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.213+0000 7fe1ee800640 1 --2- 172.21.15.83:0/115130586 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563962ef5a0 0x5563962fb110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:26.215 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.213+0000 7fe1ed400640 1 -- 172.21.15.83:0/115130586 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563962e43c0 msgr2=0x5563962eecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:58:26.215 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.213+0000 7fe1ed400640 1 --2- 172.21.15.83:0/115130586 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563962e43c0 0x5563962eecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:26.216 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.213+0000 7fe1ed400640 1 -- 172.21.15.83:0/115130586 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563962ef5a0 msgr2=0x5563962fb110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:58:26.216 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.213+0000 7fe1ed400640 1 --2- 172.21.15.83:0/115130586 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563962ef5a0 0x5563962fb110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:26.216 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.213+0000 7fe1ed400640 1 -- 172.21.15.83:0/115130586 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556396029620 con 0x5563962e4ce0 2024-04-24T08:58:26.216 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.214+0000 7fe1ed400640 1 --2- 172.21.15.83:0/115130586 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563962e4ce0 0x556396301b10 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7fe1e8108090 tx=0x7fe1d80123f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ee92e2f220abe20 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:58:26.216 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.214+0000 7fe1d6600640 1 -- 172.21.15.83:0/115130586 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1d801b640 con 0x5563962e4ce0 2024-04-24T08:58:26.216 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.214+0000 7fe1d6600640 1 -- 172.21.15.83:0/115130586 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe1d801a890 con 0x5563962e4ce0 2024-04-24T08:58:26.216 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.214+0000 7fe1d6600640 1 -- 172.21.15.83:0/115130586 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1d8010a30 con 0x5563962e4ce0 2024-04-24T08:58:26.216 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.215+0000 7fe1f3632740 1 -- 172.21.15.83:0/115130586 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563962e4ce0 msgr2=0x556396301b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:26.216 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.215+0000 7fe1f3632740 1 --2- 172.21.15.83:0/115130586 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563962e4ce0 0x556396301b10 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7fe1e8108090 tx=0x7fe1d80123f0 comp rx=0 tx=0).stop 2024-04-24T08:58:26.216 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.215+0000 7fe1f3632740 1 -- 172.21.15.83:0/115130586 shutdown_connections 2024-04-24T08:58:26.216 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.215+0000 7fe1f3632740 1 --2- 172.21.15.83:0/115130586 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563962ef5a0 0x5563962fb110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:26.216 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.215+0000 7fe1f3632740 1 --2- 172.21.15.83:0/115130586 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563962e4ce0 0x556396301b10 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:26.216 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.215+0000 7fe1f3632740 1 --2- 172.21.15.83:0/115130586 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563962e43c0 0x5563962eecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:26.216 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.215+0000 7fe1f3632740 1 -- 172.21.15.83:0/115130586 >> 172.21.15.83:0/115130586 conn(0x5563962e08d0 msgr2=0x5563962dba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:58:26.216 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.215+0000 7fe1f3632740 1 -- 172.21.15.83:0/115130586 shutdown_connections 2024-04-24T08:58:26.217 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.215+0000 7fe1f3632740 1 -- 172.21.15.83:0/115130586 wait complete. 2024-04-24T08:58:26.217 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.215+0000 7fe1f3632740 1 -- start start 2024-04-24T08:58:26.217 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.215+0000 7fe1f3632740 1 Processor -- start 2024-04-24T08:58:26.219 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.216+0000 7fe1f3632740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563962e43c0 0x5563962f2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:26.219 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.216+0000 7fe1f3632740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563962e4ce0 0x5563962f3500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:26.219 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.216+0000 7fe1f3632740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563962ef5a0 0x5563962f3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:26.219 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.216+0000 7fe1f3632740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe1e81b6f40 con 0x5563962ef5a0 2024-04-24T08:58:26.219 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.216+0000 7fe1f3632740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe1e81b6870 con 0x5563962e4ce0 2024-04-24T08:58:26.219 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.216+0000 7fe1f3632740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe1e81b3ab0 con 0x5563962e43c0 2024-04-24T08:58:26.219 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.216+0000 7fe1f3632740 10 client.0 _ll_register_callbacks cb 0x7fe1e4009800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:58:26.220 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.216+0000 7fe1ee800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563962ef5a0 0x5563962f3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:26.220 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.216+0000 7fe1ee800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563962ef5a0 0x5563962f3a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58568/0 (socket says 172.21.15.83:58568) 2024-04-24T08:58:26.220 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.216+0000 7fe1ee800640 1 -- 172.21.15.83:0/1660923261 learned_addr learned my addr 172.21.15.83:0/1660923261 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:58:26.220 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.216+0000 7fe1ede00640 1 --2- 172.21.15.83:0/1660923261 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563962e43c0 0x5563962f2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:26.220 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.216+0000 7fe1ed400640 1 --2- 172.21.15.83:0/1660923261 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563962e4ce0 0x5563962f3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:26.220 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.217+0000 7fe1ee800640 1 -- 172.21.15.83:0/1660923261 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563962e43c0 msgr2=0x5563962f2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:58:26.220 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.217+0000 7fe1ee800640 1 --2- 172.21.15.83:0/1660923261 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563962e43c0 0x5563962f2fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:26.220 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.217+0000 7fe1ee800640 1 -- 172.21.15.83:0/1660923261 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563962e4ce0 msgr2=0x5563962f3500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:58:26.220 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.217+0000 7fe1ee800640 1 --2- 172.21.15.83:0/1660923261 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563962e4ce0 0x5563962f3500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:26.220 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.217+0000 7fe1ee800640 1 -- 172.21.15.83:0/1660923261 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556396364cd0 con 0x5563962ef5a0 2024-04-24T08:58:26.220 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.217+0000 7fe1ee800640 1 --2- 172.21.15.83:0/1660923261 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563962ef5a0 0x5563962f3a40 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7fe1e4009bf0 tx=0x7fe1e4031a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a313e592d23d0850 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:58:26.220 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.217+0000 7fe1cb400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:58:26.220 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.217+0000 7fe1cb400640 1 -- 172.21.15.83:0/1660923261 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e4065060 con 0x5563962ef5a0 2024-04-24T08:58:26.220 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.217+0000 7fe1cb400640 1 -- 172.21.15.83:0/1660923261 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9567+0+0 (secure 0 0 0) 0x7fe1e4060440 con 0x5563962ef5a0 2024-04-24T08:58:26.220 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.217+0000 7fe1f3632740 1 -- 172.21.15.83:0/1660923261 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1d801a890 con 0x5563962ef5a0 2024-04-24T08:58:26.221 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.217+0000 7fe1cb400640 1 -- 172.21.15.83:0/1660923261 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fe1e406c050 con 0x5563962ef5a0 2024-04-24T08:58:26.221 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.217+0000 7fe1f3632740 10 client.18030 fetch_fsmap learned FSMap version 33 2024-04-24T08:58:26.221 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.217+0000 7fe1f3632740 1 -- 172.21.15.83:0/1660923261 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x556396365f40 con 0x5563962ef5a0 2024-04-24T08:58:26.221 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.218+0000 7fe1cb400640 1 -- 172.21.15.83:0/1660923261 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe1d801a890 con 0x5563962ef5a0 2024-04-24T08:58:26.221 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.218+0000 7fe1cb400640 1 -- 172.21.15.83:0/1660923261 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fe1e4050040 con 0x5563962ef5a0 2024-04-24T08:58:26.221 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.218+0000 7fe1f3632740 10 client.18030 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T08:58:26.221 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.218+0000 7fe1f3632740 10 client.18030 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T08:58:26.221 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.218+0000 7fe1f3632740 20 client.18030 populate_metadata read hostname 'smithi083' 2024-04-24T08:58:26.221 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.218+0000 7fe1f3632740 1 --2- 172.21.15.83:0/1660923261 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556396366660 0x556396368a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:26.221 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.218+0000 7fe1f3632740 4 client.18030 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T08:58:26.221 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.218+0000 7fe1f3632740 1 -- 172.21.15.83:0/1660923261 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556396252190 con 0x556396366660 2024-04-24T08:58:26.221 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.219+0000 7fe1ede00640 1 --2- 172.21.15.83:0/1660923261 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556396366660 0x556396368a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:26.222 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.220+0000 7fe1cb400640 10 client.18030 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:58:26.222 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.220+0000 7fe1ede00640 1 --2- 172.21.15.83:0/1660923261 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556396366660 0x556396368a50 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3474fdb307a4a7d9 server_cookie=926ab07eeb2a6371 in_seq=0 out_seq=0 2024-04-24T08:58:26.274 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.274+0000 7fe1cb400640 1 -- 172.21.15.83:0/1660923261 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556396252190 con 0x556396366660 2024-04-24T08:58:26.274 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.274+0000 7fe1cb400640 10 client.18030 handle_command_reply: tid=0 2024-04-24T08:58:26.274 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.274+0000 7fe1f3632740 1 client.18030 shutdown 2024-04-24T08:58:26.274 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.274+0000 7fe1f3632740 1 -- 172.21.15.83:0/1660923261 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563962ef5a0 msgr2=0x5563962f3a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:26.274 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.274+0000 7fe1f3632740 1 --2- 172.21.15.83:0/1660923261 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563962ef5a0 0x5563962f3a40 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7fe1e4009bf0 tx=0x7fe1e4031a90 comp rx=0 tx=0).stop 2024-04-24T08:58:26.275 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.274+0000 7fe1f3632740 1 -- 172.21.15.83:0/1660923261 shutdown_connections 2024-04-24T08:58:26.275 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.274+0000 7fe1f3632740 1 --2- 172.21.15.83:0/1660923261 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556396366660 0x556396368a50 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:26.275 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.274+0000 7fe1f3632740 1 --2- 172.21.15.83:0/1660923261 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563962ef5a0 0x5563962f3a40 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:26.275 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.274+0000 7fe1cb400640 0 client.18030 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:58:26.275 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.274+0000 7fe1f3632740 1 --2- 172.21.15.83:0/1660923261 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563962e4ce0 0x5563962f3500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:26.275 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.274+0000 7fe1f3632740 1 --2- 172.21.15.83:0/1660923261 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563962e43c0 0x5563962f2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:26.275 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.274+0000 7fe1f3632740 1 -- 172.21.15.83:0/1660923261 >> 172.21.15.83:0/1660923261 conn(0x5563962e08d0 msgr2=0x5563962df200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:58:26.276 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.275+0000 7fe1f3632740 1 -- 172.21.15.83:0/1660923261 shutdown_connections 2024-04-24T08:58:26.276 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.275+0000 7fe1f3632740 1 -- 172.21.15.83:0/1660923261 wait complete. 2024-04-24T08:58:26.276 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.275+0000 7fe1f3632740 20 client.18030 trim_cache size 0 max 16384 2024-04-24T08:58:26.276 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T08:58:26.276 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T08:58:26.276 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T08:58:26.276 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T08:58:26.277 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.276+0000 7fe1ef200640 1 -- 172.21.15.83:0/2716114712 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe1d00a3d10 msgr2=0x7fe1d00a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:26.277 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.276+0000 7fe1ef200640 1 --2- 172.21.15.83:0/2716114712 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe1d00a3d10 0x7fe1d00a61d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe1e4004af0 tx=0x7fe1e4005d80 comp rx=0 tx=0).stop 2024-04-24T08:58:26.277 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.276+0000 7fe1ef200640 1 -- 172.21.15.83:0/2716114712 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1e8108540 msgr2=0x7fe1e81a4870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:26.277 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.276+0000 7fe1ef200640 1 --2- 172.21.15.83:0/2716114712 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1e8108540 0x7fe1e81a4870 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc00d950 tx=0x7fe1dc00de10 comp rx=0 tx=0).stop 2024-04-24T08:58:26.277 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.276+0000 7fe1ef200640 1 -- 172.21.15.83:0/2716114712 shutdown_connections 2024-04-24T08:58:26.277 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.276+0000 7fe1ef200640 1 --2- 172.21.15.83:0/2716114712 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe1d00a3d10 0x7fe1d00a61d0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:26.277 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.276+0000 7fe1ef200640 1 --2- 172.21.15.83:0/2716114712 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1e810e4d0 0x7fe1e819ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:26.277 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.276+0000 7fe1ef200640 1 --2- 172.21.15.83:0/2716114712 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe1e8108ec0 0x7fe1e81a4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:26.277 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.276+0000 7fe1ef200640 1 --2- 172.21.15.83:0/2716114712 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1e8108540 0x7fe1e81a4870 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:26.277 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.277+0000 7fe1ef200640 1 -- 172.21.15.83:0/2716114712 >> 172.21.15.83:0/2716114712 conn(0x7fe1e8077de0 msgr2=0x7fe1e8078400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:58:26.278 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.277+0000 7fe1ef200640 1 -- 172.21.15.83:0/2716114712 shutdown_connections 2024-04-24T08:58:26.278 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:26.277+0000 7fe1ef200640 1 -- 172.21.15.83:0/2716114712 wait complete. 2024-04-24T08:58:26.587 INFO:tasks.workunit.client.0.smithi083.stdout: 1 35978 0.64 MB/sec execute 180 sec latency 52.652 ms 2024-04-24T08:58:27.587 INFO:tasks.workunit.client.0.smithi083.stdout: 1 36054 0.63 MB/sec execute 181 sec latency 51.616 ms 2024-04-24T08:58:27.813 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T08:58:27.813 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T08:58:27.957 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:58:27 smithi138 ceph-mon[31441]: pgmap v211: 129 pgs: 129 active+clean; 46 MiB data, 653 MiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 495 KiB/s wr, 43 op/s 2024-04-24T08:58:27.959 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:58:27 smithi189 ceph-mon[31791]: pgmap v211: 129 pgs: 129 active+clean; 46 MiB data, 653 MiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 495 KiB/s wr, 43 op/s 2024-04-24T08:58:28.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.037+0000 7f06f2200640 1 -- 172.21.15.83:0/3482473249 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06ec1093f0 msgr2=0x7f06ec110fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:28.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.037+0000 7f06f2200640 1 --2- 172.21.15.83:0/3482473249 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06ec1093f0 0x7f06ec110fc0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f06e4009ca0 tx=0x7f06e402a8c0 comp rx=0 tx=0).stop 2024-04-24T08:58:28.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.037+0000 7f06f2200640 1 -- 172.21.15.83:0/3482473249 shutdown_connections 2024-04-24T08:58:28.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.037+0000 7f06f2200640 1 --2- 172.21.15.83:0/3482473249 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06ec1093f0 0x7f06ec110fc0 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:28.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.037+0000 7f06f2200640 1 --2- 172.21.15.83:0/3482473249 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06ec104cd0 0x7f06ec108d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:28.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.037+0000 7f06f2200640 1 --2- 172.21.15.83:0/3482473249 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f06ec104320 0x7f06ec104700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:28.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.037+0000 7f06f2200640 1 -- 172.21.15.83:0/3482473249 >> 172.21.15.83:0/3482473249 conn(0x7f06ec0ffb40 msgr2=0x7f06ec101f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:58:28.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.037+0000 7f06f2200640 1 -- 172.21.15.83:0/3482473249 shutdown_connections 2024-04-24T08:58:28.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.037+0000 7f06f2200640 1 -- 172.21.15.83:0/3482473249 wait complete. 2024-04-24T08:58:28.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.038+0000 7f06f2200640 1 Processor -- start 2024-04-24T08:58:28.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.038+0000 7f06f2200640 1 -- start start 2024-04-24T08:58:28.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.038+0000 7f06f2200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f06ec104320 0x7f06ec1a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:28.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.038+0000 7f06f2200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06ec1093f0 0x7f06ec1a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:28.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.039+0000 7f06f2200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06ec19a6e0 0x7f06ec19ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:28.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.039+0000 7f06f2200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f06ec114360 con 0x7f06ec19a6e0 2024-04-24T08:58:28.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.039+0000 7f06f2200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f06ec1141e0 con 0x7f06ec1093f0 2024-04-24T08:58:28.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.039+0000 7f06f2200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f06ec1144e0 con 0x7f06ec104320 2024-04-24T08:58:28.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.039+0000 7f06f1800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06ec19a6e0 0x7f06ec19ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:28.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.039+0000 7f06f1800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06ec19a6e0 0x7f06ec19ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58584/0 (socket says 172.21.15.83:58584) 2024-04-24T08:58:28.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.039+0000 7f06f1800640 1 -- 172.21.15.83:0/2564421643 learned_addr learned my addr 172.21.15.83:0/2564421643 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:58:28.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.039+0000 7f06ebe00640 1 --2- 172.21.15.83:0/2564421643 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06ec1093f0 0x7f06ec1a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:28.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.039+0000 7f06f0e00640 1 --2- 172.21.15.83:0/2564421643 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f06ec104320 0x7f06ec1a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:28.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.039+0000 7f06f1800640 1 -- 172.21.15.83:0/2564421643 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f06ec104320 msgr2=0x7f06ec1a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:28.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.039+0000 7f06f1800640 1 --2- 172.21.15.83:0/2564421643 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f06ec104320 0x7f06ec1a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:28.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.039+0000 7f06f1800640 1 -- 172.21.15.83:0/2564421643 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06ec1093f0 msgr2=0x7f06ec1a0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:28.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.039+0000 7f06f1800640 1 --2- 172.21.15.83:0/2564421643 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06ec1093f0 0x7f06ec1a0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:28.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.039+0000 7f06f1800640 1 -- 172.21.15.83:0/2564421643 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06ec19b2e0 con 0x7f06ec19a6e0 2024-04-24T08:58:28.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.040+0000 7f06f1800640 1 --2- 172.21.15.83:0/2564421643 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06ec19a6e0 0x7f06ec19ab60 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f06e4009dc0 tx=0x7f06e4032ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:58:28.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.040+0000 7f06e9600640 1 -- 172.21.15.83:0/2564421643 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e4003cf0 con 0x7f06ec19a6e0 2024-04-24T08:58:28.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.040+0000 7f06e9600640 1 -- 172.21.15.83:0/2564421643 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f06e40044f0 con 0x7f06ec19a6e0 2024-04-24T08:58:28.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.040+0000 7f06f2200640 1 -- 172.21.15.83:0/2564421643 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06ec19b570 con 0x7f06ec19a6e0 2024-04-24T08:58:28.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.040+0000 7f06e9600640 1 -- 172.21.15.83:0/2564421643 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e4030bf0 con 0x7f06ec19a6e0 2024-04-24T08:58:28.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.040+0000 7f06f2200640 1 -- 172.21.15.83:0/2564421643 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06ec1a7710 con 0x7f06ec19a6e0 2024-04-24T08:58:28.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.044+0000 7f06f6597740 1 Processor -- start 2024-04-24T08:58:28.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.045+0000 7f06e9600640 1 -- 172.21.15.83:0/2564421643 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f06e4004040 con 0x7f06ec19a6e0 2024-04-24T08:58:28.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.045+0000 7f06f6597740 1 -- start start 2024-04-24T08:58:28.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.045+0000 7f06f6597740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560004956a30 0x560004956e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:28.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.045+0000 7f06f6597740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560004957350 0x5600049614e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:28.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.045+0000 7f06f6597740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560004961a20 0x560004963e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:28.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.045+0000 7f06f6597740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5600048bc1f0 con 0x560004956a30 2024-04-24T08:58:28.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.045+0000 7f06f6597740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560004862350 con 0x560004957350 2024-04-24T08:58:28.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.045+0000 7f06f6597740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5600048b81b0 con 0x560004961a20 2024-04-24T08:58:28.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.045+0000 7f06ebe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560004956a30 0x560004956e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:28.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.045+0000 7f06ebe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560004956a30 0x560004956e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58590/0 (socket says 172.21.15.83:58590) 2024-04-24T08:58:28.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.045+0000 7f06ebe00640 1 -- 172.21.15.83:0/2810102058 learned_addr learned my addr 172.21.15.83:0/2810102058 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:58:28.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.046+0000 7f06f0e00640 1 --2- 172.21.15.83:0/2810102058 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560004957350 0x5600049614e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:28.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.046+0000 7f06ebe00640 1 -- 172.21.15.83:0/2810102058 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560004961a20 msgr2=0x560004963e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:58:28.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.046+0000 7f06ebe00640 1 --2- 172.21.15.83:0/2810102058 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560004961a20 0x560004963e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:28.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.046+0000 7f06ebe00640 1 -- 172.21.15.83:0/2810102058 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560004957350 msgr2=0x5600049614e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:58:28.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.046+0000 7f06ebe00640 1 --2- 172.21.15.83:0/2810102058 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560004957350 0x5600049614e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:28.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.046+0000 7f06ebe00640 1 -- 172.21.15.83:0/2810102058 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600048b6f50 con 0x560004956a30 2024-04-24T08:58:28.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.046+0000 7f06e9600640 1 --2- 172.21.15.83:0/2564421643 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f06d40a3d20 0x7f06d40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:28.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.046+0000 7f06e9600640 1 -- 172.21.15.83:0/2564421643 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9567+0+0 (secure 0 0 0) 0x7f06e40f23f0 con 0x7f06ec19a6e0 2024-04-24T08:58:28.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.046+0000 7f06ebe00640 1 --2- 172.21.15.83:0/2810102058 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560004956a30 0x560004956e10 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f06d800bc80 tx=0x7f06d8011750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b6190bb3745e191 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:58:28.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.046+0000 7f06f0e00640 1 --2- 172.21.15.83:0/2564421643 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f06d40a3d20 0x7f06d40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:28.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.047+0000 7f06de000640 1 -- 172.21.15.83:0/2810102058 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d801a400 con 0x560004956a30 2024-04-24T08:58:28.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.047+0000 7f06de000640 1 -- 172.21.15.83:0/2810102058 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f06d801aa50 con 0x560004956a30 2024-04-24T08:58:28.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.047+0000 7f06de000640 1 -- 172.21.15.83:0/2810102058 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d80128e0 con 0x560004956a30 2024-04-24T08:58:28.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.048+0000 7f06f6597740 1 -- 172.21.15.83:0/2810102058 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560004956a30 msgr2=0x560004956e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:28.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.048+0000 7f06f6597740 1 --2- 172.21.15.83:0/2810102058 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560004956a30 0x560004956e10 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f06d800bc80 tx=0x7f06d8011750 comp rx=0 tx=0).stop 2024-04-24T08:58:28.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.048+0000 7f06f0e00640 1 --2- 172.21.15.83:0/2564421643 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f06d40a3d20 0x7f06d40a61e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f06e0005fd0 tx=0x7f06e000b650 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:58:28.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.048+0000 7f06f6597740 1 -- 172.21.15.83:0/2810102058 shutdown_connections 2024-04-24T08:58:28.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.048+0000 7f06f6597740 1 --2- 172.21.15.83:0/2810102058 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560004961a20 0x560004963e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:28.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.048+0000 7f06f6597740 1 --2- 172.21.15.83:0/2810102058 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560004957350 0x5600049614e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:28.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.048+0000 7f06f6597740 1 --2- 172.21.15.83:0/2810102058 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560004956a30 0x560004956e10 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:28.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.048+0000 7f06f6597740 1 -- 172.21.15.83:0/2810102058 >> 172.21.15.83:0/2810102058 conn(0x560004952d50 msgr2=0x560004953130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:58:28.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.048+0000 7f06f6597740 1 -- 172.21.15.83:0/2810102058 shutdown_connections 2024-04-24T08:58:28.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.048+0000 7f06f6597740 1 -- 172.21.15.83:0/2810102058 wait complete. 2024-04-24T08:58:28.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.048+0000 7f06f6597740 1 -- start start 2024-04-24T08:58:28.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.049+0000 7f06f6597740 1 Processor -- start 2024-04-24T08:58:28.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.049+0000 7f06f6597740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560004956a30 0x56000495e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:28.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.049+0000 7f06f6597740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560004957350 0x56000495eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:28.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.049+0000 7f06f6597740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560004961a20 0x56000495f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:28.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.049+0000 7f06f6597740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x560004964860 con 0x560004961a20 2024-04-24T08:58:28.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.049+0000 7f06f6597740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560004875690 con 0x560004957350 2024-04-24T08:58:28.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.049+0000 7f06f6597740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5600049649e0 con 0x560004956a30 2024-04-24T08:58:28.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.049+0000 7f06f6597740 10 client.0 _ll_register_callbacks cb 0x5600048b1650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:58:28.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.049+0000 7f06f1800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560004961a20 0x56000495f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:28.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.049+0000 7f06f1800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560004961a20 0x56000495f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58596/0 (socket says 172.21.15.83:58596) 2024-04-24T08:58:28.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.049+0000 7f06f1800640 1 -- 172.21.15.83:0/3626261536 learned_addr learned my addr 172.21.15.83:0/3626261536 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:58:28.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.050+0000 7f06ebe00640 1 --2- 172.21.15.83:0/3626261536 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560004957350 0x56000495eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:28.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.050+0000 7f06f0e00640 1 --2- 172.21.15.83:0/3626261536 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560004956a30 0x56000495e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:28.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.050+0000 7f06f1800640 1 -- 172.21.15.83:0/3626261536 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560004956a30 msgr2=0x56000495e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:58:28.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.050+0000 7f06f1800640 1 --2- 172.21.15.83:0/3626261536 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560004956a30 0x56000495e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:28.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.050+0000 7f06f1800640 1 -- 172.21.15.83:0/3626261536 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560004957350 msgr2=0x56000495eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:58:28.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.050+0000 7f06f1800640 1 --2- 172.21.15.83:0/3626261536 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560004957350 0x56000495eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:28.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.050+0000 7f06f1800640 1 -- 172.21.15.83:0/3626261536 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560004874920 con 0x560004961a20 2024-04-24T08:58:28.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.050+0000 7f06ebe00640 1 --2- 172.21.15.83:0/3626261536 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560004957350 0x56000495eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:58:28.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.050+0000 7f06f0e00640 1 --2- 172.21.15.83:0/3626261536 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560004956a30 0x56000495e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:58:28.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.050+0000 7f06f1800640 1 --2- 172.21.15.83:0/3626261536 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560004961a20 0x56000495f430 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f06e40baa00 tx=0x7f06e40bdec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4e04f87318c6c75 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:58:28.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.050+0000 7f06d2a00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:58:28.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.050+0000 7f06d2a00640 1 -- 172.21.15.83:0/3626261536 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e40d7060 con 0x560004961a20 2024-04-24T08:58:28.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.050+0000 7f06d2a00640 1 -- 172.21.15.83:0/3626261536 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9567+0+0 (secure 0 0 0) 0x7f06e40c20d0 con 0x560004961a20 2024-04-24T08:58:28.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.050+0000 7f06f6597740 1 -- 172.21.15.83:0/3626261536 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06d801aa50 con 0x560004961a20 2024-04-24T08:58:28.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.051+0000 7f06d2a00640 1 -- 172.21.15.83:0/3626261536 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f06e40be730 con 0x560004961a20 2024-04-24T08:58:28.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.051+0000 7f06f6597740 10 client.18042 fetch_fsmap learned FSMap version 33 2024-04-24T08:58:28.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.051+0000 7f06f6597740 1 -- 172.21.15.83:0/3626261536 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5600046987d0 con 0x560004961a20 2024-04-24T08:58:28.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.051+0000 7f06d2a00640 1 -- 172.21.15.83:0/3626261536 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f06d801aa50 con 0x560004961a20 2024-04-24T08:58:28.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.051+0000 7f06d2a00640 1 -- 172.21.15.83:0/3626261536 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f06e40ef950 con 0x560004961a20 2024-04-24T08:58:28.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.051+0000 7f06f6597740 10 client.18042 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T08:58:28.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.051+0000 7f06f6597740 10 client.18042 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:58:28.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.051+0000 7f06f6597740 20 client.18042 populate_metadata read hostname 'smithi083' 2024-04-24T08:58:28.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.051+0000 7f06f6597740 1 --2- 172.21.15.83:0/3626261536 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x560004971dc0 0x5600049741f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:28.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.051+0000 7f06f6597740 4 client.18042 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:58:28.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.051+0000 7f06f6597740 1 -- 172.21.15.83:0/3626261536 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5600048c9c10 con 0x560004971dc0 2024-04-24T08:58:28.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.052+0000 7f06f0e00640 1 --2- 172.21.15.83:0/3626261536 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x560004971dc0 0x5600049741f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:28.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:58:27 smithi083 ceph-mon[27907]: pgmap v211: 129 pgs: 129 active+clean; 46 MiB data, 653 MiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 495 KiB/s wr, 43 op/s 2024-04-24T08:58:28.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.053+0000 7f06f0e00640 1 --2- 172.21.15.83:0/3626261536 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x560004971dc0 0x5600049741f0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e22ef9476b49bc4 server_cookie=16f2eb40709a5b56 in_seq=0 out_seq=0 2024-04-24T08:58:28.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.053+0000 7f06d2a00640 10 client.18042 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:58:28.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.055+0000 7f06d2a00640 1 -- 172.21.15.83:0/3626261536 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5600048c9c10 con 0x560004971dc0 2024-04-24T08:58:28.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.055+0000 7f06d2a00640 10 client.18042 handle_command_reply: tid=0 2024-04-24T08:58:28.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.055+0000 7f06f6597740 1 client.18042 shutdown 2024-04-24T08:58:28.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.055+0000 7f06f6597740 1 -- 172.21.15.83:0/3626261536 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560004961a20 msgr2=0x56000495f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:28.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.055+0000 7f06f6597740 1 --2- 172.21.15.83:0/3626261536 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560004961a20 0x56000495f430 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f06e40baa00 tx=0x7f06e40bdec0 comp rx=0 tx=0).stop 2024-04-24T08:58:28.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.055+0000 7f06f6597740 1 -- 172.21.15.83:0/3626261536 shutdown_connections 2024-04-24T08:58:28.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.055+0000 7f06f6597740 1 --2- 172.21.15.83:0/3626261536 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x560004971dc0 0x5600049741f0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:28.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.055+0000 7f06f6597740 1 --2- 172.21.15.83:0/3626261536 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560004961a20 0x56000495f430 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:28.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.055+0000 7f06f6597740 1 --2- 172.21.15.83:0/3626261536 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560004957350 0x56000495eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:28.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.055+0000 7f06d2a00640 0 client.18042 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:58:28.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.055+0000 7f06f6597740 1 --2- 172.21.15.83:0/3626261536 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560004956a30 0x56000495e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:28.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.055+0000 7f06f6597740 1 -- 172.21.15.83:0/3626261536 >> 172.21.15.83:0/3626261536 conn(0x560004952d50 msgr2=0x560004951fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:58:28.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.055+0000 7f06f6597740 1 -- 172.21.15.83:0/3626261536 shutdown_connections 2024-04-24T08:58:28.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.056+0000 7f06f6597740 1 -- 172.21.15.83:0/3626261536 wait complete. 2024-04-24T08:58:28.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.056+0000 7f06f6597740 20 client.18042 trim_cache size 0 max 16384 2024-04-24T08:58:28.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.077+0000 7f06f6597740 1 Processor -- start 2024-04-24T08:58:28.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.077+0000 7f06f6597740 1 -- start start 2024-04-24T08:58:28.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.077+0000 7f06f6597740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560004956a30 0x56000496d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:28.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.077+0000 7f06f6597740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560004957350 0x560004979b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:28.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.077+0000 7f06f6597740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560004961a20 0x56000497a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:28.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.077+0000 7f06f6597740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f06ec1b1630 con 0x560004956a30 2024-04-24T08:58:28.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.077+0000 7f06f6597740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f06ec1b3170 con 0x560004957350 2024-04-24T08:58:28.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.077+0000 7f06f6597740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f06ec1b1d00 con 0x560004961a20 2024-04-24T08:58:28.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.078+0000 7f06f0e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560004956a30 0x56000496d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:28.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.078+0000 7f06f0e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560004956a30 0x56000496d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58612/0 (socket says 172.21.15.83:58612) 2024-04-24T08:58:28.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.078+0000 7f06f0e00640 1 -- 172.21.15.83:0/4163232596 learned_addr learned my addr 172.21.15.83:0/4163232596 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:58:28.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.078+0000 7f06f0e00640 1 -- 172.21.15.83:0/4163232596 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560004961a20 msgr2=0x56000497a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:58:28.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.078+0000 7f06f0e00640 1 --2- 172.21.15.83:0/4163232596 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560004961a20 0x56000497a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:28.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.078+0000 7f06f0e00640 1 -- 172.21.15.83:0/4163232596 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560004957350 msgr2=0x560004979b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:58:28.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.078+0000 7f06f0e00640 1 --2- 172.21.15.83:0/4163232596 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560004957350 0x560004979b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:28.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.078+0000 7f06f0e00640 1 -- 172.21.15.83:0/4163232596 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600045ce3f0 con 0x560004956a30 2024-04-24T08:58:28.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.078+0000 7f06f0e00640 1 --2- 172.21.15.83:0/4163232596 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560004956a30 0x56000496d720 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f06e000b450 tx=0x7f06e0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e801dfb706091004 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:58:28.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.079+0000 7f06dd600640 1 -- 172.21.15.83:0/4163232596 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e0012ce0 con 0x560004956a30 2024-04-24T08:58:28.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.079+0000 7f06dd600640 1 -- 172.21.15.83:0/4163232596 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f06e0028850 con 0x560004956a30 2024-04-24T08:58:28.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.079+0000 7f06dd600640 1 -- 172.21.15.83:0/4163232596 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e00047a0 con 0x560004956a30 2024-04-24T08:58:28.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.080+0000 7f06f6597740 1 -- 172.21.15.83:0/4163232596 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560004956a30 msgr2=0x56000496d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:28.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.080+0000 7f06f6597740 1 --2- 172.21.15.83:0/4163232596 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560004956a30 0x56000496d720 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f06e000b450 tx=0x7f06e0014000 comp rx=0 tx=0).stop 2024-04-24T08:58:28.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.080+0000 7f06f6597740 1 -- 172.21.15.83:0/4163232596 shutdown_connections 2024-04-24T08:58:28.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.080+0000 7f06f6597740 1 --2- 172.21.15.83:0/4163232596 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560004961a20 0x56000497a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:28.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.080+0000 7f06f6597740 1 --2- 172.21.15.83:0/4163232596 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560004957350 0x560004979b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:28.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.080+0000 7f06f6597740 1 --2- 172.21.15.83:0/4163232596 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560004956a30 0x56000496d720 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:28.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.080+0000 7f06f6597740 1 -- 172.21.15.83:0/4163232596 >> 172.21.15.83:0/4163232596 conn(0x560004952d50 msgr2=0x56000495f200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:58:28.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.080+0000 7f06f6597740 1 -- 172.21.15.83:0/4163232596 shutdown_connections 2024-04-24T08:58:28.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.080+0000 7f06f6597740 1 -- 172.21.15.83:0/4163232596 wait complete. 2024-04-24T08:58:28.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.080+0000 7f06f6597740 1 -- start start 2024-04-24T08:58:28.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.080+0000 7f06f6597740 1 Processor -- start 2024-04-24T08:58:28.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.081+0000 7f06f6597740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560004956a30 0x560004965f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:28.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.081+0000 7f06f6597740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560004957350 0x560004966470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:28.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.081+0000 7f06f6597740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560004961a20 0x5600049d13b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:28.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.081+0000 7f06f6597740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f06ec1b2aa0 con 0x560004961a20 2024-04-24T08:58:28.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.081+0000 7f06f6597740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f06ec1b23d0 con 0x560004957350 2024-04-24T08:58:28.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.081+0000 7f06f6597740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f06ec1af610 con 0x560004956a30 2024-04-24T08:58:28.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.081+0000 7f06f6597740 10 client.0 _ll_register_callbacks cb 0x7f06e4005f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:58:28.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.081+0000 7f06f1800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560004961a20 0x5600049d13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:28.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.082+0000 7f06f1800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560004961a20 0x5600049d13b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58622/0 (socket says 172.21.15.83:58622) 2024-04-24T08:58:28.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.082+0000 7f06f1800640 1 -- 172.21.15.83:0/109571730 learned_addr learned my addr 172.21.15.83:0/109571730 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:58:28.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.082+0000 7f06f0e00640 1 --2- 172.21.15.83:0/109571730 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560004956a30 0x560004965f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:28.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.082+0000 7f06ebe00640 1 --2- 172.21.15.83:0/109571730 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560004957350 0x560004966470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:28.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.082+0000 7f06f1800640 1 -- 172.21.15.83:0/109571730 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560004956a30 msgr2=0x560004965f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:58:28.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.082+0000 7f06f1800640 1 --2- 172.21.15.83:0/109571730 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560004956a30 0x560004965f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:28.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.082+0000 7f06f1800640 1 -- 172.21.15.83:0/109571730 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560004957350 msgr2=0x560004966470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:58:28.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.082+0000 7f06f1800640 1 --2- 172.21.15.83:0/109571730 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560004957350 0x560004966470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:28.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.082+0000 7f06f1800640 1 -- 172.21.15.83:0/109571730 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56000458e0f0 con 0x560004961a20 2024-04-24T08:58:28.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.082+0000 7f06ebe00640 1 --2- 172.21.15.83:0/109571730 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560004957350 0x560004966470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:58:28.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.082+0000 7f06f1800640 1 --2- 172.21.15.83:0/109571730 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560004961a20 0x5600049d13b0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f06e40c6e00 tx=0x7f06e40c6ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab5ef9c72347c57f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:58:28.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.082+0000 7f06d2a00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:58:28.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.083+0000 7f06d2a00640 1 -- 172.21.15.83:0/109571730 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e40bd990 con 0x560004961a20 2024-04-24T08:58:28.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.083+0000 7f06d2a00640 1 -- 172.21.15.83:0/109571730 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9567+0+0 (secure 0 0 0) 0x7f06e40be2e0 con 0x560004961a20 2024-04-24T08:58:28.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.083+0000 7f06f6597740 1 -- 172.21.15.83:0/109571730 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06e0028850 con 0x560004961a20 2024-04-24T08:58:28.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.083+0000 7f06d2a00640 1 -- 172.21.15.83:0/109571730 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f06e40be4e0 con 0x560004961a20 2024-04-24T08:58:28.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.083+0000 7f06f6597740 10 client.18048 fetch_fsmap learned FSMap version 33 2024-04-24T08:58:28.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.083+0000 7f06f6597740 1 -- 172.21.15.83:0/109571730 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x560004881810 con 0x560004961a20 2024-04-24T08:58:28.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.083+0000 7f06d2a00640 1 -- 172.21.15.83:0/109571730 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f06e0028850 con 0x560004961a20 2024-04-24T08:58:28.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.083+0000 7f06d2a00640 1 -- 172.21.15.83:0/109571730 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f06e40b8ba0 con 0x560004961a20 2024-04-24T08:58:28.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.083+0000 7f06f6597740 10 client.18048 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T08:58:28.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.083+0000 7f06f6597740 10 client.18048 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:58:28.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.083+0000 7f06f6597740 20 client.18048 populate_metadata read hostname 'smithi083' 2024-04-24T08:58:28.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.083+0000 7f06f6597740 1 --2- 172.21.15.83:0/109571730 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5600049d8f60 0x5600049db380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:28.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.083+0000 7f06f6597740 4 client.18048 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T08:58:28.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.083+0000 7f06f6597740 1 -- 172.21.15.83:0/109571730 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5600048c3a80 con 0x5600049d8f60 2024-04-24T08:58:28.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.084+0000 7f06f0e00640 1 --2- 172.21.15.83:0/109571730 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5600049d8f60 0x5600049db380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:28.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.086+0000 7f06f0e00640 1 --2- 172.21.15.83:0/109571730 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5600049d8f60 0x5600049db380 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=293525d89bbb67a6 server_cookie=86c734cee59d86a1 in_seq=0 out_seq=0 2024-04-24T08:58:28.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.086+0000 7f06d2a00640 10 client.18048 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:58:28.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.087+0000 7f06d2a00640 1 -- 172.21.15.83:0/109571730 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5600048c3a80 con 0x5600049d8f60 2024-04-24T08:58:28.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.087+0000 7f06d2a00640 10 client.18048 handle_command_reply: tid=0 2024-04-24T08:58:28.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.087+0000 7f06f6597740 1 client.18048 shutdown 2024-04-24T08:58:28.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.088+0000 7f06f6597740 1 -- 172.21.15.83:0/109571730 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560004961a20 msgr2=0x5600049d13b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:28.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.088+0000 7f06f6597740 1 --2- 172.21.15.83:0/109571730 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560004961a20 0x5600049d13b0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f06e40c6e00 tx=0x7f06e40c6ca0 comp rx=0 tx=0).stop 2024-04-24T08:58:28.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.088+0000 7f06f6597740 1 -- 172.21.15.83:0/109571730 shutdown_connections 2024-04-24T08:58:28.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.088+0000 7f06f6597740 1 --2- 172.21.15.83:0/109571730 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5600049d8f60 0x5600049db380 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:28.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.088+0000 7f06f6597740 1 --2- 172.21.15.83:0/109571730 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560004961a20 0x5600049d13b0 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:28.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.088+0000 7f06f6597740 1 --2- 172.21.15.83:0/109571730 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560004957350 0x560004966470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:28.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.088+0000 7f06f6597740 1 --2- 172.21.15.83:0/109571730 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560004956a30 0x560004965f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:28.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.088+0000 7f06f6597740 1 -- 172.21.15.83:0/109571730 >> 172.21.15.83:0/109571730 conn(0x560004952d50 msgr2=0x56000495e970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:58:28.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.088+0000 7f06d2a00640 0 client.18048 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:58:28.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.088+0000 7f06f6597740 1 -- 172.21.15.83:0/109571730 shutdown_connections 2024-04-24T08:58:28.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.088+0000 7f06f6597740 1 -- 172.21.15.83:0/109571730 wait complete. 2024-04-24T08:58:28.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.089+0000 7f06f6597740 20 client.18048 trim_cache size 0 max 16384 2024-04-24T08:58:28.089 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T08:58:28.089 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-04-24T08:58:28.090 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-04-24T08:58:28.090 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T08:58:28.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.089+0000 7f06f2200640 1 -- 172.21.15.83:0/2564421643 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f06d40a3d20 msgr2=0x7f06d40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:28.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.089+0000 7f06f2200640 1 --2- 172.21.15.83:0/2564421643 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f06d40a3d20 0x7f06d40a61e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f06e0005fd0 tx=0x7f06e000b650 comp rx=0 tx=0).stop 2024-04-24T08:58:28.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.089+0000 7f06f2200640 1 -- 172.21.15.83:0/2564421643 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06ec19a6e0 msgr2=0x7f06ec19ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:28.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.089+0000 7f06f2200640 1 --2- 172.21.15.83:0/2564421643 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06ec19a6e0 0x7f06ec19ab60 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f06e4009dc0 tx=0x7f06e4032ff0 comp rx=0 tx=0).stop 2024-04-24T08:58:28.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.089+0000 7f06f2200640 1 -- 172.21.15.83:0/2564421643 shutdown_connections 2024-04-24T08:58:28.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.089+0000 7f06f2200640 1 --2- 172.21.15.83:0/2564421643 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f06d40a3d20 0x7f06d40a61e0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:28.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.089+0000 7f06f2200640 1 --2- 172.21.15.83:0/2564421643 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06ec19a6e0 0x7f06ec19ab60 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:28.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.089+0000 7f06f2200640 1 --2- 172.21.15.83:0/2564421643 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06ec1093f0 0x7f06ec1a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:28.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.089+0000 7f06f2200640 1 --2- 172.21.15.83:0/2564421643 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f06ec104320 0x7f06ec1a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:28.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.089+0000 7f06f2200640 1 -- 172.21.15.83:0/2564421643 >> 172.21.15.83:0/2564421643 conn(0x7f06ec0ffb40 msgr2=0x7f06ec100e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:58:28.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.090+0000 7f06f2200640 1 -- 172.21.15.83:0/2564421643 shutdown_connections 2024-04-24T08:58:28.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.090+0000 7f06f2200640 1 -- 172.21.15.83:0/2564421643 wait complete. 2024-04-24T08:58:28.109 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T08:58:28.110 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T08:58:28.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.313+0000 7fd375c00640 1 -- 172.21.15.83:0/173375297 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd370107160 msgr2=0x7fd37010ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:28.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.313+0000 7fd375c00640 1 --2- 172.21.15.83:0/173375297 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd370107160 0x7fd37010ed30 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7fd368009ca0 tx=0x7fd36802a8c0 comp rx=0 tx=0).stop 2024-04-24T08:58:28.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.313+0000 7fd375c00640 1 -- 172.21.15.83:0/173375297 shutdown_connections 2024-04-24T08:58:28.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.313+0000 7fd375c00640 1 --2- 172.21.15.83:0/173375297 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd370107160 0x7fd37010ed30 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:28.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.313+0000 7fd375c00640 1 --2- 172.21.15.83:0/173375297 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd370102a40 0x7fd370106a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:28.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.313+0000 7fd375c00640 1 --2- 172.21.15.83:0/173375297 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd370102090 0x7fd370102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:28.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.313+0000 7fd375c00640 1 -- 172.21.15.83:0/173375297 >> 172.21.15.83:0/173375297 conn(0x7fd3700fd930 msgr2=0x7fd3700ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:58:28.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.313+0000 7fd375c00640 1 -- 172.21.15.83:0/173375297 shutdown_connections 2024-04-24T08:58:28.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.313+0000 7fd375c00640 1 -- 172.21.15.83:0/173375297 wait complete. 2024-04-24T08:58:28.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.314+0000 7fd375c00640 1 Processor -- start 2024-04-24T08:58:28.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.315+0000 7fd375c00640 1 -- start start 2024-04-24T08:58:28.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.315+0000 7fd375c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd370102090 0x7fd3701a05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:28.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.315+0000 7fd375c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd370107160 0x7fd3701a0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:28.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.315+0000 7fd375c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd37019a6c0 0x7fd37019ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:28.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.315+0000 7fd375c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd370112120 con 0x7fd37019a6c0 2024-04-24T08:58:28.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.315+0000 7fd375c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd370111fa0 con 0x7fd370102090 2024-04-24T08:58:28.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.315+0000 7fd375c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd3701122a0 con 0x7fd370107160 2024-04-24T08:58:28.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.315+0000 7fd375200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd37019a6c0 0x7fd37019ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:28.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.315+0000 7fd375200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd37019a6c0 0x7fd37019ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58632/0 (socket says 172.21.15.83:58632) 2024-04-24T08:58:28.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.315+0000 7fd375200640 1 -- 172.21.15.83:0/2767749272 learned_addr learned my addr 172.21.15.83:0/2767749272 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:58:28.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.316+0000 7fd36fe00640 1 --2- 172.21.15.83:0/2767749272 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd370102090 0x7fd3701a05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:28.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.316+0000 7fd36f400640 1 --2- 172.21.15.83:0/2767749272 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd370107160 0x7fd3701a0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:28.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.316+0000 7fd375200640 1 -- 172.21.15.83:0/2767749272 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd370107160 msgr2=0x7fd3701a0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:28.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.316+0000 7fd375200640 1 --2- 172.21.15.83:0/2767749272 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd370107160 0x7fd3701a0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:28.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.316+0000 7fd375200640 1 -- 172.21.15.83:0/2767749272 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd370102090 msgr2=0x7fd3701a05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:28.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.316+0000 7fd375200640 1 --2- 172.21.15.83:0/2767749272 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd370102090 0x7fd3701a05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:28.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.316+0000 7fd375200640 1 -- 172.21.15.83:0/2767749272 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd37019b2c0 con 0x7fd37019a6c0 2024-04-24T08:58:28.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.316+0000 7fd36f400640 1 --2- 172.21.15.83:0/2767749272 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd370107160 0x7fd3701a0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:58:28.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.316+0000 7fd375200640 1 --2- 172.21.15.83:0/2767749272 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd37019a6c0 0x7fd37019ab40 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7fd368031040 tx=0x7fd368003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:58:28.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.317+0000 7fd36cc00640 1 -- 172.21.15.83:0/2767749272 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd368003ce0 con 0x7fd37019a6c0 2024-04-24T08:58:28.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.317+0000 7fd36cc00640 1 -- 172.21.15.83:0/2767749272 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd3680044f0 con 0x7fd37019a6c0 2024-04-24T08:58:28.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.317+0000 7fd375c00640 1 -- 172.21.15.83:0/2767749272 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd37019b550 con 0x7fd37019a6c0 2024-04-24T08:58:28.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.317+0000 7fd36cc00640 1 -- 172.21.15.83:0/2767749272 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd368030c90 con 0x7fd37019a6c0 2024-04-24T08:58:28.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.317+0000 7fd375c00640 1 -- 172.21.15.83:0/2767749272 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3701a76f0 con 0x7fd37019a6c0 2024-04-24T08:58:28.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.318+0000 7fd375c00640 1 -- 172.21.15.83:0/2767749272 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd370103560 con 0x7fd37019a6c0 2024-04-24T08:58:28.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.321+0000 7fd36cc00640 1 -- 172.21.15.83:0/2767749272 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd3680046a0 con 0x7fd37019a6c0 2024-04-24T08:58:28.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.323+0000 7fd36cc00640 1 --2- 172.21.15.83:0/2767749272 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fd3540a3da0 0x7fd3540a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:28.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.323+0000 7fd36cc00640 1 -- 172.21.15.83:0/2767749272 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9567+0+0 (secure 0 0 0) 0x7fd3680f2490 con 0x7fd37019a6c0 2024-04-24T08:58:28.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.323+0000 7fd36fe00640 1 --2- 172.21.15.83:0/2767749272 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fd3540a3da0 0x7fd3540a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:28.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.325+0000 7fd36fe00640 1 --2- 172.21.15.83:0/2767749272 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fd3540a3da0 0x7fd3540a6260 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd360005fd0 tx=0x7fd360005850 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:58:28.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.326+0000 7fd36cc00640 1 -- 172.21.15.83:0/2767749272 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd3680be8f0 con 0x7fd37019a6c0 2024-04-24T08:58:28.587 INFO:tasks.workunit.client.0.smithi083.stdout: 1 36132 0.63 MB/sec execute 182 sec latency 41.250 ms 2024-04-24T08:58:28.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.596+0000 7fd375c00640 1 -- 172.21.15.83:0/2767749272 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd370102ae0 con 0x7fd37019a6c0 2024-04-24T08:58:28.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.598+0000 7fd36cc00640 1 -- 172.21.15.83:0/2767749272 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11944 (secure 0 0 0) 0x7fd3680be1f0 con 0x7fd37019a6c0 2024-04-24T08:58:28.598 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:58:28.598 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/358872257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":358872257},{"type":"v1","addr":"172.21.15.83:6835","nonce":358872257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3157070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3157070851},{"type":"v1","addr":"172.21.15.83:6837","nonce":3157070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":33,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T08:48:29.422576+0000","modified":"2024-04-24T08:56:17.048967+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34520,"mds_1":34493,"mds_2":34451,"mds_3":34427,"mds_4":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14658":{"gid":14658,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6839/4085470844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":4085470844},{"type":"v1","addr":"172.21.15.83:6839","nonce":4085470844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14676":{"gid":14676,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6841/2634754210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":2634754210},{"type":"v1","addr":"172.21.15.83:6841","nonce":2634754210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24382":{"gid":24382,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/1859921310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1859921310},{"type":"v1","addr":"172.21.15.189:6833","nonce":1859921310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/2241923197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2241923197},{"type":"v1","addr":"172.21.15.189:6835","nonce":2241923197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6837/3172311220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":3172311220},{"type":"v1","addr":"172.21.15.189:6837","nonce":3172311220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.189:6839/2548965336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":2548965336},{"type":"v1","addr":"172.21.15.189:6839","nonce":2548965336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.138:6833/1157967658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1157967658},{"type":"v1","addr":"172.21.15.138:6833","nonce":1157967658}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.138:6835/1909219758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1909219758},{"type":"v1","addr":"172.21.15.138:6835","nonce":1909219758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34493":{"gid":34493,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.138:6837/900566754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":900566754},{"type":"v1","addr":"172.21.15.138:6837","nonce":900566754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34520":{"gid":34520,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.138:6839/2236932393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2236932393},{"type":"v1","addr":"172.21.15.138:6839","nonce":2236932393}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34520,"qdb_cluster":[24454,34520,34427,34451,34493]},"id":1}]} 2024-04-24T08:58:28.598 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 33 2024-04-24T08:58:28.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.601+0000 7fd375c00640 1 -- 172.21.15.83:0/2767749272 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fd3540a3da0 msgr2=0x7fd3540a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:28.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.601+0000 7fd375c00640 1 --2- 172.21.15.83:0/2767749272 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fd3540a3da0 0x7fd3540a6260 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd360005fd0 tx=0x7fd360005850 comp rx=0 tx=0).stop 2024-04-24T08:58:28.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.601+0000 7fd375c00640 1 -- 172.21.15.83:0/2767749272 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd37019a6c0 msgr2=0x7fd37019ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:28.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.601+0000 7fd375c00640 1 --2- 172.21.15.83:0/2767749272 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd37019a6c0 0x7fd37019ab40 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7fd368031040 tx=0x7fd368003970 comp rx=0 tx=0).stop 2024-04-24T08:58:28.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.601+0000 7fd375c00640 1 -- 172.21.15.83:0/2767749272 shutdown_connections 2024-04-24T08:58:28.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.601+0000 7fd375c00640 1 --2- 172.21.15.83:0/2767749272 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fd3540a3da0 0x7fd3540a6260 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:28.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.601+0000 7fd375c00640 1 --2- 172.21.15.83:0/2767749272 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd37019a6c0 0x7fd37019ab40 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:28.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.601+0000 7fd375c00640 1 --2- 172.21.15.83:0/2767749272 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd370107160 0x7fd3701a0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:28.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.601+0000 7fd375c00640 1 --2- 172.21.15.83:0/2767749272 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd370102090 0x7fd3701a05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:28.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.601+0000 7fd375c00640 1 -- 172.21.15.83:0/2767749272 >> 172.21.15.83:0/2767749272 conn(0x7fd3700fd930 msgr2=0x7fd3700fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:58:28.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.601+0000 7fd375c00640 1 -- 172.21.15.83:0/2767749272 shutdown_connections 2024-04-24T08:58:28.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.601+0000 7fd375c00640 1 -- 172.21.15.83:0/2767749272 wait complete. 2024-04-24T08:58:28.620 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-24T08:58:28.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.821+0000 7f79eea00640 1 -- 172.21.15.83:0/1362360243 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79e8104cc0 msgr2=0x7f79e8108d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:28.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.821+0000 7f79eea00640 1 --2- 172.21.15.83:0/1362360243 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79e8104cc0 0x7f79e8108d00 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f79d4009900 tx=0x7f79d402a3f0 comp rx=0 tx=0).stop 2024-04-24T08:58:28.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.821+0000 7f79eea00640 1 -- 172.21.15.83:0/1362360243 shutdown_connections 2024-04-24T08:58:28.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.821+0000 7f79eea00640 1 --2- 172.21.15.83:0/1362360243 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79e81093e0 0x7f79e8110fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:28.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.821+0000 7f79eea00640 1 --2- 172.21.15.83:0/1362360243 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79e8104cc0 0x7f79e8108d00 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:28.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.821+0000 7f79eea00640 1 --2- 172.21.15.83:0/1362360243 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f79e8104310 0x7f79e81046f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:28.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.821+0000 7f79eea00640 1 -- 172.21.15.83:0/1362360243 >> 172.21.15.83:0/1362360243 conn(0x7f79e80ffb70 msgr2=0x7f79e8101f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:58:28.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.821+0000 7f79eea00640 1 -- 172.21.15.83:0/1362360243 shutdown_connections 2024-04-24T08:58:28.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.821+0000 7f79eea00640 1 -- 172.21.15.83:0/1362360243 wait complete. 2024-04-24T08:58:28.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.822+0000 7f79eea00640 1 Processor -- start 2024-04-24T08:58:28.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.822+0000 7f79eea00640 1 -- start start 2024-04-24T08:58:28.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.823+0000 7f79eea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79e8104310 0x7f79e81a0590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:28.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.823+0000 7f79eea00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79e81093e0 0x7f79e81a0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:28.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.823+0000 7f79eea00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f79e819a6a0 0x7f79e819ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:28.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.823+0000 7f79eea00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f79e8114300 con 0x7f79e8104310 2024-04-24T08:58:28.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.823+0000 7f79eea00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f79e8114180 con 0x7f79e81093e0 2024-04-24T08:58:28.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.823+0000 7f79eea00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f79e8114480 con 0x7f79e819a6a0 2024-04-24T08:58:28.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.823+0000 7f79ed600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79e8104310 0x7f79e81a0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:28.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.823+0000 7f79ed600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79e8104310 0x7f79e81a0590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58636/0 (socket says 172.21.15.83:58636) 2024-04-24T08:58:28.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.823+0000 7f79ed600640 1 -- 172.21.15.83:0/3440512417 learned_addr learned my addr 172.21.15.83:0/3440512417 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:58:28.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.823+0000 7f79ee000640 1 --2- 172.21.15.83:0/3440512417 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f79e819a6a0 0x7f79e819ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:28.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.823+0000 7f79ecc00640 1 --2- 172.21.15.83:0/3440512417 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79e81093e0 0x7f79e81a0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:28.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.823+0000 7f79ed600640 1 -- 172.21.15.83:0/3440512417 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f79e819a6a0 msgr2=0x7f79e819ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:28.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.823+0000 7f79ed600640 1 --2- 172.21.15.83:0/3440512417 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f79e819a6a0 0x7f79e819ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:28.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.824+0000 7f79ed600640 1 -- 172.21.15.83:0/3440512417 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79e81093e0 msgr2=0x7f79e81a0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:28.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.824+0000 7f79ed600640 1 --2- 172.21.15.83:0/3440512417 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79e81093e0 0x7f79e81a0ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:28.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.824+0000 7f79ed600640 1 -- 172.21.15.83:0/3440512417 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79e819b2a0 con 0x7f79e8104310 2024-04-24T08:58:28.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.824+0000 7f79ecc00640 1 --2- 172.21.15.83:0/3440512417 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79e81093e0 0x7f79e81a0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:58:28.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.824+0000 7f79ed600640 1 --2- 172.21.15.83:0/3440512417 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79e8104310 0x7f79e81a0590 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f79dc00c890 tx=0x7f79dc00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:58:28.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.824+0000 7f79e6000640 1 -- 172.21.15.83:0/3440512417 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79dc004220 con 0x7f79e8104310 2024-04-24T08:58:28.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.824+0000 7f79e6000640 1 -- 172.21.15.83:0/3440512417 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f79dc0043c0 con 0x7f79e8104310 2024-04-24T08:58:28.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.824+0000 7f79e6000640 1 -- 172.21.15.83:0/3440512417 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79dc004a00 con 0x7f79e8104310 2024-04-24T08:58:28.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.824+0000 7f79eea00640 1 -- 172.21.15.83:0/3440512417 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79e81a7330 con 0x7f79e8104310 2024-04-24T08:58:28.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.824+0000 7f79eea00640 1 -- 172.21.15.83:0/3440512417 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79e81a76e0 con 0x7f79e8104310 2024-04-24T08:58:28.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.828+0000 7f79f2dc8740 1 Processor -- start 2024-04-24T08:58:28.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.828+0000 7f79e6000640 1 -- 172.21.15.83:0/3440512417 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f79dc004ba0 con 0x7f79e8104310 2024-04-24T08:58:28.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.828+0000 7f79f2dc8740 1 -- start start 2024-04-24T08:58:28.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.828+0000 7f79f2dc8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5580b3406b40 0x5580b3406f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:28.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.828+0000 7f79f2dc8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5580b3407460 0x5580b34115f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:28.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.828+0000 7f79f2dc8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5580b3411b30 0x5580b3413f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:28.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.828+0000 7f79f2dc8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5580b336c1d0 con 0x5580b3407460 2024-04-24T08:58:28.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.828+0000 7f79f2dc8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5580b3312450 con 0x5580b3406b40 2024-04-24T08:58:28.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.828+0000 7f79f2dc8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5580b3368190 con 0x5580b3411b30 2024-04-24T08:58:28.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.829+0000 7f79ed600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5580b3407460 0x5580b34115f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:28.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.829+0000 7f79ed600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5580b3407460 0x5580b34115f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58640/0 (socket says 172.21.15.83:58640) 2024-04-24T08:58:28.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.829+0000 7f79ed600640 1 -- 172.21.15.83:0/2832931288 learned_addr learned my addr 172.21.15.83:0/2832931288 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:58:28.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.829+0000 7f79ee000640 1 --2- 172.21.15.83:0/2832931288 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5580b3406b40 0x5580b3406f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:28.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.829+0000 7f79ecc00640 1 --2- 172.21.15.83:0/2832931288 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5580b3411b30 0x5580b3413f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:28.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.829+0000 7f79ed600640 1 -- 172.21.15.83:0/2832931288 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5580b3411b30 msgr2=0x5580b3413f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:58:28.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.829+0000 7f79ed600640 1 --2- 172.21.15.83:0/2832931288 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5580b3411b30 0x5580b3413f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:28.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.829+0000 7f79ed600640 1 -- 172.21.15.83:0/2832931288 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5580b3406b40 msgr2=0x5580b3406f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:58:28.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.829+0000 7f79ed600640 1 --2- 172.21.15.83:0/2832931288 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5580b3406b40 0x5580b3406f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:28.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.829+0000 7f79ed600640 1 -- 172.21.15.83:0/2832931288 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580b33cf310 con 0x5580b3407460 2024-04-24T08:58:28.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.829+0000 7f79ee000640 1 --2- 172.21.15.83:0/2832931288 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5580b3406b40 0x5580b3406f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:58:28.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.829+0000 7f79ecc00640 1 --2- 172.21.15.83:0/2832931288 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5580b3411b30 0x5580b3413f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:58:28.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.830+0000 7f79ed600640 1 --2- 172.21.15.83:0/2832931288 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5580b3407460 0x5580b34115f0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f79dc0cdbe0 tx=0x7f79dc0d8610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f27f038c978bb6fd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:58:28.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.830+0000 7f79e6000640 1 --2- 172.21.15.83:0/3440512417 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f79d00a3d20 0x7f79d00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:28.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.830+0000 7f79da600640 1 -- 172.21.15.83:0/2832931288 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79dc0e0020 con 0x5580b3407460 2024-04-24T08:58:28.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.830+0000 7f79e6000640 1 -- 172.21.15.83:0/3440512417 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9567+0+0 (secure 0 0 0) 0x7f79dc0d09c0 con 0x7f79e8104310 2024-04-24T08:58:28.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.830+0000 7f79da600640 1 -- 172.21.15.83:0/2832931288 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f79dc0d8e40 con 0x5580b3407460 2024-04-24T08:58:28.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.830+0000 7f79da600640 1 -- 172.21.15.83:0/2832931288 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79dc098790 con 0x5580b3407460 2024-04-24T08:58:28.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.830+0000 7f79ee000640 1 --2- 172.21.15.83:0/3440512417 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f79d00a3d20 0x7f79d00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:28.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.831+0000 7f79ee000640 1 --2- 172.21.15.83:0/3440512417 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f79d00a3d20 0x7f79d00a61e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f79e00073f0 tx=0x7f79e0007380 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:58:28.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.831+0000 7f79f2dc8740 1 -- 172.21.15.83:0/2832931288 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5580b3407460 msgr2=0x5580b34115f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:28.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.831+0000 7f79f2dc8740 1 --2- 172.21.15.83:0/2832931288 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5580b3407460 0x5580b34115f0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f79dc0cdbe0 tx=0x7f79dc0d8610 comp rx=0 tx=0).stop 2024-04-24T08:58:28.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.831+0000 7f79f2dc8740 1 -- 172.21.15.83:0/2832931288 shutdown_connections 2024-04-24T08:58:28.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.831+0000 7f79f2dc8740 1 --2- 172.21.15.83:0/2832931288 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5580b3411b30 0x5580b3413f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:28.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.831+0000 7f79f2dc8740 1 --2- 172.21.15.83:0/2832931288 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5580b3407460 0x5580b34115f0 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:28.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.831+0000 7f79f2dc8740 1 --2- 172.21.15.83:0/2832931288 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5580b3406b40 0x5580b3406f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:28.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.831+0000 7f79f2dc8740 1 -- 172.21.15.83:0/2832931288 >> 172.21.15.83:0/2832931288 conn(0x5580b3402e60 msgr2=0x5580b3403240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:58:28.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.831+0000 7f79f2dc8740 1 -- 172.21.15.83:0/2832931288 shutdown_connections 2024-04-24T08:58:28.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.831+0000 7f79f2dc8740 1 -- 172.21.15.83:0/2832931288 wait complete. 2024-04-24T08:58:28.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.832+0000 7f79f2dc8740 1 -- start start 2024-04-24T08:58:28.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.832+0000 7f79f2dc8740 1 Processor -- start 2024-04-24T08:58:28.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.833+0000 7f79f2dc8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5580b3406b40 0x5580b3411590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:28.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.833+0000 7f79f2dc8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5580b3407460 0x5580b340e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:28.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.833+0000 7f79f2dc8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5580b3411b30 0x5580b340efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:28.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.833+0000 7f79f2dc8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5580b34149c0 con 0x5580b3411b30 2024-04-24T08:58:28.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.833+0000 7f79f2dc8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5580b3325810 con 0x5580b3407460 2024-04-24T08:58:28.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.833+0000 7f79f2dc8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5580b3414b40 con 0x5580b3406b40 2024-04-24T08:58:28.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.833+0000 7f79f2dc8740 10 client.0 _ll_register_callbacks cb 0x5580b32f3140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:58:28.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.833+0000 7f79ee000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5580b3411b30 0x5580b340efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:28.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.833+0000 7f79ee000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5580b3411b30 0x5580b340efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47400/0 (socket says 172.21.15.83:47400) 2024-04-24T08:58:28.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.833+0000 7f79ee000640 1 -- 172.21.15.83:0/2345049006 learned_addr learned my addr 172.21.15.83:0/2345049006 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:58:28.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.833+0000 7f79ed600640 1 --2- 172.21.15.83:0/2345049006 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5580b3406b40 0x5580b3411590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:28.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.833+0000 7f79ecc00640 1 --2- 172.21.15.83:0/2345049006 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5580b3407460 0x5580b340e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:28.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.833+0000 7f79ee000640 1 -- 172.21.15.83:0/2345049006 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5580b3406b40 msgr2=0x5580b3411590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:58:28.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.833+0000 7f79ee000640 1 --2- 172.21.15.83:0/2345049006 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5580b3406b40 0x5580b3411590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:28.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.833+0000 7f79ee000640 1 -- 172.21.15.83:0/2345049006 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5580b3407460 msgr2=0x5580b340e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:58:28.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.833+0000 7f79ee000640 1 --2- 172.21.15.83:0/2345049006 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5580b3407460 0x5580b340e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:28.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.834+0000 7f79ee000640 1 -- 172.21.15.83:0/2345049006 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580b33363f0 con 0x5580b3411b30 2024-04-24T08:58:28.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.834+0000 7f79ee000640 1 --2- 172.21.15.83:0/2345049006 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5580b3411b30 0x5580b340efa0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f79e0011e10 tx=0x7f79e0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b089cc728b7672df server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:58:28.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.834+0000 7f79cea00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:58:28.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.834+0000 7f79cea00640 1 -- 172.21.15.83:0/2345049006 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e001e620 con 0x5580b3411b30 2024-04-24T08:58:28.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.834+0000 7f79cea00640 1 -- 172.21.15.83:0/2345049006 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9567+0+0 (secure 0 0 0) 0x7f79e001e7c0 con 0x5580b3411b30 2024-04-24T08:58:28.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.834+0000 7f79f2dc8740 1 -- 172.21.15.83:0/2345049006 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5580b3379bf0 con 0x5580b3411b30 2024-04-24T08:58:28.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.834+0000 7f79cea00640 1 -- 172.21.15.83:0/2345049006 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f79e002f050 con 0x5580b3411b30 2024-04-24T08:58:28.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.835+0000 7f79f2dc8740 10 client.18066 fetch_fsmap learned FSMap version 33 2024-04-24T08:58:28.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.835+0000 7f79f2dc8740 1 -- 172.21.15.83:0/2345049006 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5580b307e3f0 con 0x5580b3411b30 2024-04-24T08:58:28.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.835+0000 7f79cea00640 1 -- 172.21.15.83:0/2345049006 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x5580b3379bf0 con 0x5580b3411b30 2024-04-24T08:58:28.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.835+0000 7f79cea00640 1 -- 172.21.15.83:0/2345049006 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f79e001c910 con 0x5580b3411b30 2024-04-24T08:58:28.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.835+0000 7f79f2dc8740 10 client.18066 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T08:58:28.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.835+0000 7f79f2dc8740 10 client.18066 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-24T08:58:28.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.835+0000 7f79f2dc8740 20 client.18066 populate_metadata read hostname 'smithi083' 2024-04-24T08:58:28.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.835+0000 7f79f2dc8740 1 --2- 172.21.15.83:0/2345049006 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x5580b3421d80 0x5580b34241b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:28.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.835+0000 7f79f2dc8740 4 client.18066 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:58:28.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.835+0000 7f79f2dc8740 1 -- 172.21.15.83:0/2345049006 --> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5580b3373a60 con 0x5580b3421d80 2024-04-24T08:58:28.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.836+0000 7f79ed600640 1 --2- 172.21.15.83:0/2345049006 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x5580b3421d80 0x5580b34241b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:28.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.837+0000 7f79ed600640 1 --2- 172.21.15.83:0/2345049006 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x5580b3421d80 0x5580b34241b0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=28a95a7c6a428a7f server_cookie=49b51a307ac1b5b9 in_seq=0 out_seq=0 2024-04-24T08:58:28.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.837+0000 7f79cea00640 10 client.18066 ms_handle_connect on v2:172.21.15.189:6838/2548965336 2024-04-24T08:58:28.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.839+0000 7f79cea00640 1 -- 172.21.15.83:0/2345049006 <== mds.4 v2:172.21.15.189:6838/2548965336 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5580b3373a60 con 0x5580b3421d80 2024-04-24T08:58:28.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.839+0000 7f79cea00640 10 client.18066 handle_command_reply: tid=0 2024-04-24T08:58:28.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.839+0000 7f79f2dc8740 1 client.18066 shutdown 2024-04-24T08:58:28.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.839+0000 7f79f2dc8740 1 -- 172.21.15.83:0/2345049006 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5580b3411b30 msgr2=0x5580b340efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:28.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.839+0000 7f79f2dc8740 1 --2- 172.21.15.83:0/2345049006 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5580b3411b30 0x5580b340efa0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f79e0011e10 tx=0x7f79e0004a40 comp rx=0 tx=0).stop 2024-04-24T08:58:28.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.839+0000 7f79f2dc8740 1 -- 172.21.15.83:0/2345049006 shutdown_connections 2024-04-24T08:58:28.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.839+0000 7f79f2dc8740 1 --2- 172.21.15.83:0/2345049006 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x5580b3421d80 0x5580b34241b0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:28.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.839+0000 7f79f2dc8740 1 --2- 172.21.15.83:0/2345049006 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5580b3411b30 0x5580b340efa0 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:28.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.839+0000 7f79f2dc8740 1 --2- 172.21.15.83:0/2345049006 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5580b3407460 0x5580b340e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:28.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.839+0000 7f79f2dc8740 1 --2- 172.21.15.83:0/2345049006 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5580b3406b40 0x5580b3411590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:28.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.839+0000 7f79f2dc8740 1 -- 172.21.15.83:0/2345049006 >> 172.21.15.83:0/2345049006 conn(0x5580b3402e60 msgr2=0x5580b3408c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:58:28.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.839+0000 7f79cea00640 0 client.18066 ms_handle_reset on v2:172.21.15.189:6838/2548965336 2024-04-24T08:58:28.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.839+0000 7f79f2dc8740 1 -- 172.21.15.83:0/2345049006 shutdown_connections 2024-04-24T08:58:28.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.839+0000 7f79f2dc8740 1 -- 172.21.15.83:0/2345049006 wait complete. 2024-04-24T08:58:28.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.840+0000 7f79f2dc8740 20 client.18066 trim_cache size 0 max 16384 2024-04-24T08:58:28.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.856+0000 7f79f2dc8740 1 Processor -- start 2024-04-24T08:58:28.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.856+0000 7f79f2dc8740 1 -- start start 2024-04-24T08:58:28.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.857+0000 7f79f2dc8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5580b3406b40 0x5580b34092a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:28.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.857+0000 7f79f2dc8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5580b3407460 0x5580b3401dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:28.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.857+0000 7f79f2dc8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5580b3411b30 0x5580b34110c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:28.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.857+0000 7f79f2dc8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f79e81b1680 con 0x5580b3407460 2024-04-24T08:58:28.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.857+0000 7f79f2dc8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f79e81b31c0 con 0x5580b3406b40 2024-04-24T08:58:28.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.857+0000 7f79f2dc8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f79e81b1d50 con 0x5580b3411b30 2024-04-24T08:58:28.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.857+0000 7f79ecc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5580b3407460 0x5580b3401dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:28.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.857+0000 7f79ecc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5580b3407460 0x5580b3401dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47410/0 (socket says 172.21.15.83:47410) 2024-04-24T08:58:28.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.857+0000 7f79ecc00640 1 -- 172.21.15.83:0/1422351073 learned_addr learned my addr 172.21.15.83:0/1422351073 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:58:28.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.858+0000 7f79ed600640 1 --2- 172.21.15.83:0/1422351073 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5580b3406b40 0x5580b34092a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:28.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.858+0000 7f79ee000640 1 --2- 172.21.15.83:0/1422351073 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5580b3411b30 0x5580b34110c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:28.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.858+0000 7f79ecc00640 1 -- 172.21.15.83:0/1422351073 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5580b3411b30 msgr2=0x5580b34110c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:58:28.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.858+0000 7f79ecc00640 1 --2- 172.21.15.83:0/1422351073 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5580b3411b30 0x5580b34110c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:28.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.858+0000 7f79ecc00640 1 -- 172.21.15.83:0/1422351073 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5580b3406b40 msgr2=0x5580b34092a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:58:28.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.858+0000 7f79ecc00640 1 --2- 172.21.15.83:0/1422351073 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5580b3406b40 0x5580b34092a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:28.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.858+0000 7f79ecc00640 1 -- 172.21.15.83:0/1422351073 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580b31487d0 con 0x5580b3407460 2024-04-24T08:58:28.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.858+0000 7f79ecc00640 1 --2- 172.21.15.83:0/1422351073 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5580b3407460 0x5580b3401dd0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f79d4005ab0 tx=0x7f79d4032f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8552f8b77f441fd8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:58:28.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.859+0000 7f79d9c00640 1 -- 172.21.15.83:0/1422351073 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79d4035030 con 0x5580b3407460 2024-04-24T08:58:28.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.859+0000 7f79d9c00640 1 -- 172.21.15.83:0/1422351073 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f79d403f070 con 0x5580b3407460 2024-04-24T08:58:28.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.859+0000 7f79d9c00640 1 -- 172.21.15.83:0/1422351073 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79d40053c0 con 0x5580b3407460 2024-04-24T08:58:28.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.859+0000 7f79f2dc8740 1 -- 172.21.15.83:0/1422351073 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5580b3407460 msgr2=0x5580b3401dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:28.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.859+0000 7f79f2dc8740 1 --2- 172.21.15.83:0/1422351073 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5580b3407460 0x5580b3401dd0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f79d4005ab0 tx=0x7f79d4032f30 comp rx=0 tx=0).stop 2024-04-24T08:58:28.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.860+0000 7f79f2dc8740 1 -- 172.21.15.83:0/1422351073 shutdown_connections 2024-04-24T08:58:28.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.860+0000 7f79f2dc8740 1 --2- 172.21.15.83:0/1422351073 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5580b3411b30 0x5580b34110c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:28.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.860+0000 7f79f2dc8740 1 --2- 172.21.15.83:0/1422351073 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5580b3407460 0x5580b3401dd0 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:28.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.860+0000 7f79f2dc8740 1 --2- 172.21.15.83:0/1422351073 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5580b3406b40 0x5580b34092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:28.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.860+0000 7f79f2dc8740 1 -- 172.21.15.83:0/1422351073 >> 172.21.15.83:0/1422351073 conn(0x5580b3402e60 msgr2=0x5580b34016c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:58:28.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.860+0000 7f79f2dc8740 1 -- 172.21.15.83:0/1422351073 shutdown_connections 2024-04-24T08:58:28.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.860+0000 7f79f2dc8740 1 -- 172.21.15.83:0/1422351073 wait complete. 2024-04-24T08:58:28.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.860+0000 7f79f2dc8740 1 -- start start 2024-04-24T08:58:28.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.860+0000 7f79f2dc8740 1 Processor -- start 2024-04-24T08:58:28.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.861+0000 7f79f2dc8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5580b3406b40 0x5580b341a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:28.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.861+0000 7f79f2dc8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5580b3407460 0x5580b3412510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:28.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.861+0000 7f79f2dc8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5580b3411b30 0x5580b3412a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:28.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.861+0000 7f79f2dc8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f79e81b2af0 con 0x5580b3407460 2024-04-24T08:58:28.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.861+0000 7f79f2dc8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f79e81b2420 con 0x5580b3411b30 2024-04-24T08:58:28.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.861+0000 7f79f2dc8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f79e81af660 con 0x5580b3406b40 2024-04-24T08:58:28.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.861+0000 7f79f2dc8740 10 client.0 _ll_register_callbacks cb 0x7f79e0007700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:58:28.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.861+0000 7f79ecc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5580b3407460 0x5580b3412510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:28.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.861+0000 7f79ecc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5580b3407460 0x5580b3412510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47422/0 (socket says 172.21.15.83:47422) 2024-04-24T08:58:28.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.861+0000 7f79ecc00640 1 -- 172.21.15.83:0/70890386 learned_addr learned my addr 172.21.15.83:0/70890386 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:58:28.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.861+0000 7f79ee000640 1 --2- 172.21.15.83:0/70890386 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5580b3411b30 0x5580b3412a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:28.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.861+0000 7f79ed600640 1 --2- 172.21.15.83:0/70890386 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5580b3406b40 0x5580b341a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:28.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.862+0000 7f79ecc00640 1 -- 172.21.15.83:0/70890386 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5580b3406b40 msgr2=0x5580b341a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:58:28.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.862+0000 7f79ecc00640 1 --2- 172.21.15.83:0/70890386 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5580b3406b40 0x5580b341a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:28.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.862+0000 7f79ecc00640 1 -- 172.21.15.83:0/70890386 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5580b3411b30 msgr2=0x5580b3412a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:58:28.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.862+0000 7f79ecc00640 1 --2- 172.21.15.83:0/70890386 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5580b3411b30 0x5580b3412a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:28.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.862+0000 7f79ecc00640 1 -- 172.21.15.83:0/70890386 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580b34130c0 con 0x5580b3407460 2024-04-24T08:58:28.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.862+0000 7f79ecc00640 1 --2- 172.21.15.83:0/70890386 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5580b3407460 0x5580b3412510 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f79d4031df0 tx=0x7f79d4031e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cac998f3c794175 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:58:28.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.862+0000 7f79cea00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:58:28.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.862+0000 7f79cea00640 1 -- 172.21.15.83:0/70890386 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79d4035070 con 0x5580b3407460 2024-04-24T08:58:28.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.862+0000 7f79cea00640 1 -- 172.21.15.83:0/70890386 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9567+0+0 (secure 0 0 0) 0x7f79d40055d0 con 0x5580b3407460 2024-04-24T08:58:28.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.862+0000 7f79f2dc8740 1 -- 172.21.15.83:0/70890386 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79d403f070 con 0x5580b3407460 2024-04-24T08:58:28.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.863+0000 7f79cea00640 1 -- 172.21.15.83:0/70890386 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f79d4055050 con 0x5580b3407460 2024-04-24T08:58:28.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.863+0000 7f79f2dc8740 10 client.18072 fetch_fsmap learned FSMap version 33 2024-04-24T08:58:28.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.863+0000 7f79f2dc8740 1 -- 172.21.15.83:0/70890386 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5580b34882f0 con 0x5580b3407460 2024-04-24T08:58:28.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.863+0000 7f79cea00640 1 -- 172.21.15.83:0/70890386 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f79d403f070 con 0x5580b3407460 2024-04-24T08:58:28.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.863+0000 7f79cea00640 1 -- 172.21.15.83:0/70890386 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f79d404c8e0 con 0x5580b3407460 2024-04-24T08:58:28.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.863+0000 7f79f2dc8740 10 client.18072 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T08:58:28.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.863+0000 7f79f2dc8740 10 client.18072 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-24T08:58:28.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.863+0000 7f79f2dc8740 20 client.18072 populate_metadata read hostname 'smithi083' 2024-04-24T08:58:28.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.863+0000 7f79f2dc8740 1 --2- 172.21.15.83:0/70890386 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x5580b34888b0 0x5580b348acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:28.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.863+0000 7f79f2dc8740 4 client.18072 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T08:58:28.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.863+0000 7f79f2dc8740 1 -- 172.21.15.83:0/70890386 --> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5580b32ef4d0 con 0x5580b34888b0 2024-04-24T08:58:28.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.864+0000 7f79ed600640 1 --2- 172.21.15.83:0/70890386 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x5580b34888b0 0x5580b348acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:28.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.865+0000 7f79ed600640 1 --2- 172.21.15.83:0/70890386 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x5580b34888b0 0x5580b348acd0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9541ba35474b989c server_cookie=9e83ae833460ae0 in_seq=0 out_seq=0 2024-04-24T08:58:28.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.865+0000 7f79cea00640 10 client.18072 ms_handle_connect on v2:172.21.15.189:6838/2548965336 2024-04-24T08:58:28.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.866+0000 7f79cea00640 1 -- 172.21.15.83:0/70890386 <== mds.4 v2:172.21.15.189:6838/2548965336 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5580b32ef4d0 con 0x5580b34888b0 2024-04-24T08:58:28.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.866+0000 7f79cea00640 10 client.18072 handle_command_reply: tid=0 2024-04-24T08:58:28.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.866+0000 7f79f2dc8740 1 client.18072 shutdown 2024-04-24T08:58:28.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.866+0000 7f79f2dc8740 1 -- 172.21.15.83:0/70890386 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5580b3407460 msgr2=0x5580b3412510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:28.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.866+0000 7f79f2dc8740 1 --2- 172.21.15.83:0/70890386 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5580b3407460 0x5580b3412510 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f79d4031df0 tx=0x7f79d4031e20 comp rx=0 tx=0).stop 2024-04-24T08:58:28.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.866+0000 7f79f2dc8740 1 -- 172.21.15.83:0/70890386 shutdown_connections 2024-04-24T08:58:28.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.866+0000 7f79f2dc8740 1 --2- 172.21.15.83:0/70890386 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x5580b34888b0 0x5580b348acd0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:28.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.866+0000 7f79f2dc8740 1 --2- 172.21.15.83:0/70890386 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5580b3411b30 0x5580b3412a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:28.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.866+0000 7f79f2dc8740 1 --2- 172.21.15.83:0/70890386 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5580b3407460 0x5580b3412510 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:28.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.866+0000 7f79cea00640 0 client.18072 ms_handle_reset on v2:172.21.15.189:6838/2548965336 2024-04-24T08:58:28.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.866+0000 7f79f2dc8740 1 --2- 172.21.15.83:0/70890386 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5580b3406b40 0x5580b341a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:28.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.866+0000 7f79f2dc8740 1 -- 172.21.15.83:0/70890386 >> 172.21.15.83:0/70890386 conn(0x5580b3402e60 msgr2=0x5580b340d8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:58:28.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.867+0000 7f79f2dc8740 1 -- 172.21.15.83:0/70890386 shutdown_connections 2024-04-24T08:58:28.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.867+0000 7f79f2dc8740 1 -- 172.21.15.83:0/70890386 wait complete. 2024-04-24T08:58:28.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.867+0000 7f79f2dc8740 20 client.18072 trim_cache size 0 max 16384 2024-04-24T08:58:28.871 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T08:58:28.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.868+0000 7f79eea00640 1 -- 172.21.15.83:0/3440512417 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f79d00a3d20 msgr2=0x7f79d00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:28.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.868+0000 7f79eea00640 1 --2- 172.21.15.83:0/3440512417 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f79d00a3d20 0x7f79d00a61e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f79e00073f0 tx=0x7f79e0007380 comp rx=0 tx=0).stop 2024-04-24T08:58:28.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.868+0000 7f79eea00640 1 -- 172.21.15.83:0/3440512417 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79e8104310 msgr2=0x7f79e81a0590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:28.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.868+0000 7f79eea00640 1 --2- 172.21.15.83:0/3440512417 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79e8104310 0x7f79e81a0590 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f79dc00c890 tx=0x7f79dc00cd50 comp rx=0 tx=0).stop 2024-04-24T08:58:28.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.868+0000 7f79eea00640 1 -- 172.21.15.83:0/3440512417 shutdown_connections 2024-04-24T08:58:28.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.868+0000 7f79eea00640 1 --2- 172.21.15.83:0/3440512417 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f79d00a3d20 0x7f79d00a61e0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:28.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.868+0000 7f79eea00640 1 --2- 172.21.15.83:0/3440512417 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f79e819a6a0 0x7f79e819ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:28.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.868+0000 7f79eea00640 1 --2- 172.21.15.83:0/3440512417 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79e81093e0 0x7f79e81a0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:28.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.868+0000 7f79eea00640 1 --2- 172.21.15.83:0/3440512417 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79e8104310 0x7f79e81a0590 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:28.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.868+0000 7f79eea00640 1 -- 172.21.15.83:0/3440512417 >> 172.21.15.83:0/3440512417 conn(0x7f79e80ffb70 msgr2=0x7f79e8100ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:58:28.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.868+0000 7f79eea00640 1 -- 172.21.15.83:0/3440512417 shutdown_connections 2024-04-24T08:58:28.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:28.868+0000 7f79eea00640 1 -- 172.21.15.83:0/3440512417 wait complete. 2024-04-24T08:58:28.886 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:58:28 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/2767749272' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:58:28.888 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-24T08:58:29.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.088+0000 7f261f200640 1 -- 172.21.15.83:0/3331394313 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2620104310 msgr2=0x7f26201046f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:29.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.088+0000 7f261f200640 1 --2- 172.21.15.83:0/3331394313 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2620104310 0x7f26201046f0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f2610009900 tx=0x7f261002a3f0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.088+0000 7f261f200640 1 -- 172.21.15.83:0/3331394313 shutdown_connections 2024-04-24T08:58:29.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.088+0000 7f261f200640 1 --2- 172.21.15.83:0/3331394313 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26201093e0 0x7f2620110fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.088+0000 7f261f200640 1 --2- 172.21.15.83:0/3331394313 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2620104cc0 0x7f2620108d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.088+0000 7f261f200640 1 --2- 172.21.15.83:0/3331394313 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2620104310 0x7f26201046f0 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.088+0000 7f261f200640 1 -- 172.21.15.83:0/3331394313 >> 172.21.15.83:0/3331394313 conn(0x7f26200ffb10 msgr2=0x7f2620101f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:58:29.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.088+0000 7f261f200640 1 -- 172.21.15.83:0/3331394313 shutdown_connections 2024-04-24T08:58:29.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.088+0000 7f261f200640 1 -- 172.21.15.83:0/3331394313 wait complete. 2024-04-24T08:58:29.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.089+0000 7f261f200640 1 Processor -- start 2024-04-24T08:58:29.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.089+0000 7f261f200640 1 -- start start 2024-04-24T08:58:29.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.090+0000 7f261f200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2620104310 0x7f26201a0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:29.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.090+0000 7f261f200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26201093e0 0x7f26201a0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:29.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.090+0000 7f261f200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f262019a690 0x7f262019ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:29.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.090+0000 7f261f200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2620114370 con 0x7f2620104310 2024-04-24T08:58:29.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.090+0000 7f261f200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f26201141f0 con 0x7f26201093e0 2024-04-24T08:58:29.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.090+0000 7f261f200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f26201144f0 con 0x7f262019a690 2024-04-24T08:58:29.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.090+0000 7f261de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2620104310 0x7f26201a0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:29.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.090+0000 7f261de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2620104310 0x7f26201a0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47450/0 (socket says 172.21.15.83:47450) 2024-04-24T08:58:29.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.090+0000 7f261de00640 1 -- 172.21.15.83:0/404207530 learned_addr learned my addr 172.21.15.83:0/404207530 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:58:29.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.090+0000 7f261d400640 1 --2- 172.21.15.83:0/404207530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26201093e0 0x7f26201a0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:29.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.090+0000 7f261e800640 1 --2- 172.21.15.83:0/404207530 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f262019a690 0x7f262019ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:29.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.090+0000 7f261de00640 1 -- 172.21.15.83:0/404207530 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f262019a690 msgr2=0x7f262019ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:29.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.090+0000 7f261de00640 1 --2- 172.21.15.83:0/404207530 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f262019a690 0x7f262019ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.090+0000 7f261de00640 1 -- 172.21.15.83:0/404207530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26201093e0 msgr2=0x7f26201a0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:29.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.090+0000 7f261de00640 1 --2- 172.21.15.83:0/404207530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26201093e0 0x7f26201a0ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.090+0000 7f261de00640 1 -- 172.21.15.83:0/404207530 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f262019b290 con 0x7f2620104310 2024-04-24T08:58:29.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.090+0000 7f261d400640 1 --2- 172.21.15.83:0/404207530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26201093e0 0x7f26201a0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:58:29.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.090+0000 7f261e800640 1 --2- 172.21.15.83:0/404207530 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f262019a690 0x7f262019ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:58:29.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.091+0000 7f261de00640 1 --2- 172.21.15.83:0/404207530 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2620104310 0x7f26201a0580 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f2610005ab0 tx=0x7f2610003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:58:29.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.091+0000 7f2616a00640 1 -- 172.21.15.83:0/404207530 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f261003f070 con 0x7f2620104310 2024-04-24T08:58:29.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.091+0000 7f2616a00640 1 -- 172.21.15.83:0/404207530 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2610031d60 con 0x7f2620104310 2024-04-24T08:58:29.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.091+0000 7f2616a00640 1 -- 172.21.15.83:0/404207530 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2610035030 con 0x7f2620104310 2024-04-24T08:58:29.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.091+0000 7f261f200640 1 -- 172.21.15.83:0/404207530 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f262019b520 con 0x7f2620104310 2024-04-24T08:58:29.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.091+0000 7f261f200640 1 -- 172.21.15.83:0/404207530 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26201a75c0 con 0x7f2620104310 2024-04-24T08:58:29.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.096+0000 7f2627804740 1 Processor -- start 2024-04-24T08:58:29.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.096+0000 7f2627804740 1 -- start start 2024-04-24T08:58:29.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.096+0000 7f2616a00640 1 -- 172.21.15.83:0/404207530 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2610042070 con 0x7f2620104310 2024-04-24T08:58:29.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.096+0000 7f2627804740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55571101f350 0x55571101f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:29.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.096+0000 7f2627804740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55571101fc70 0x555711029e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:29.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.097+0000 7f2627804740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55571102a340 0x55571102c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:29.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.097+0000 7f2627804740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x555710f7c1d0 con 0x55571101f350 2024-04-24T08:58:29.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.097+0000 7f2627804740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555710f22450 con 0x55571102a340 2024-04-24T08:58:29.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.097+0000 7f2627804740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x555710f78190 con 0x55571101fc70 2024-04-24T08:58:29.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.097+0000 7f261e800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55571101f350 0x55571101f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:29.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.097+0000 7f261e800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55571101f350 0x55571101f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47462/0 (socket says 172.21.15.83:47462) 2024-04-24T08:58:29.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.097+0000 7f261e800640 1 -- 172.21.15.83:0/1378523306 learned_addr learned my addr 172.21.15.83:0/1378523306 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:58:29.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.097+0000 7f261de00640 1 --2- 172.21.15.83:0/1378523306 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55571101fc70 0x555711029e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:29.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.097+0000 7f261d400640 1 --2- 172.21.15.83:0/1378523306 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55571102a340 0x55571102c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:29.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.098+0000 7f261e800640 1 -- 172.21.15.83:0/1378523306 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55571101fc70 msgr2=0x555711029e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:58:29.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.098+0000 7f261e800640 1 --2- 172.21.15.83:0/1378523306 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55571101fc70 0x555711029e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.098+0000 7f261e800640 1 -- 172.21.15.83:0/1378523306 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55571102a340 msgr2=0x55571102c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:58:29.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.098+0000 7f261e800640 1 --2- 172.21.15.83:0/1378523306 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55571102a340 0x55571102c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.098+0000 7f261e800640 1 -- 172.21.15.83:0/1378523306 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555710fdf310 con 0x55571101f350 2024-04-24T08:58:29.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.098+0000 7f261e800640 1 --2- 172.21.15.83:0/1378523306 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55571101f350 0x55571101f730 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f2618008d50 tx=0x7f261800d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8c1835a6e2b18f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:58:29.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.099+0000 7f260b000640 1 -- 172.21.15.83:0/1378523306 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f261800f830 con 0x55571101f350 2024-04-24T08:58:29.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.099+0000 7f260b000640 1 -- 172.21.15.83:0/1378523306 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f261800de20 con 0x55571101f350 2024-04-24T08:58:29.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.099+0000 7f260b000640 1 -- 172.21.15.83:0/1378523306 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2618018dd0 con 0x55571101f350 2024-04-24T08:58:29.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.099+0000 7f2616a00640 1 --2- 172.21.15.83:0/404207530 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f26040a3da0 0x7f26040a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:29.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.099+0000 7f2616a00640 1 -- 172.21.15.83:0/404207530 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9567+0+0 (secure 0 0 0) 0x7f26100f1620 con 0x7f2620104310 2024-04-24T08:58:29.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.099+0000 7f261de00640 1 --2- 172.21.15.83:0/404207530 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f26040a3da0 0x7f26040a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:29.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.100+0000 7f2627804740 1 -- 172.21.15.83:0/1378523306 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55571101f350 msgr2=0x55571101f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:29.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.100+0000 7f2627804740 1 --2- 172.21.15.83:0/1378523306 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55571101f350 0x55571101f730 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f2618008d50 tx=0x7f261800d710 comp rx=0 tx=0).stop 2024-04-24T08:58:29.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.100+0000 7f2627804740 1 -- 172.21.15.83:0/1378523306 shutdown_connections 2024-04-24T08:58:29.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.100+0000 7f261de00640 1 --2- 172.21.15.83:0/404207530 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f26040a3da0 0x7f26040a6260 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f26100eea10 tx=0x7f261002a3f0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:58:29.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.100+0000 7f2627804740 1 --2- 172.21.15.83:0/1378523306 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55571102a340 0x55571102c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.100+0000 7f2627804740 1 --2- 172.21.15.83:0/1378523306 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55571101fc70 0x555711029e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.100+0000 7f2627804740 1 --2- 172.21.15.83:0/1378523306 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55571101f350 0x55571101f730 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.100+0000 7f2627804740 1 -- 172.21.15.83:0/1378523306 >> 172.21.15.83:0/1378523306 conn(0x55571101b670 msgr2=0x55571101ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:58:29.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.100+0000 7f2627804740 1 -- 172.21.15.83:0/1378523306 shutdown_connections 2024-04-24T08:58:29.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.100+0000 7f2627804740 1 -- 172.21.15.83:0/1378523306 wait complete. 2024-04-24T08:58:29.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.100+0000 7f2627804740 1 -- start start 2024-04-24T08:58:29.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.101+0000 7f2627804740 1 Processor -- start 2024-04-24T08:58:29.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.101+0000 7f2627804740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55571101f350 0x555711029dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:29.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.101+0000 7f2627804740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55571101fc70 0x5557110270c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:29.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.101+0000 7f2627804740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55571102a340 0x5557110277b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:29.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.101+0000 7f2627804740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55571102d1d0 con 0x55571101f350 2024-04-24T08:58:29.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.101+0000 7f2627804740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555710f35810 con 0x55571101fc70 2024-04-24T08:58:29.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.101+0000 7f2627804740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55571102d350 con 0x55571102a340 2024-04-24T08:58:29.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.101+0000 7f2627804740 10 client.0 _ll_register_callbacks cb 0x555710f03140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:58:29.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.101+0000 7f261e800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55571101f350 0x555711029dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:29.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.102+0000 7f261e800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55571101f350 0x555711029dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47474/0 (socket says 172.21.15.83:47474) 2024-04-24T08:58:29.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.102+0000 7f261e800640 1 -- 172.21.15.83:0/2635905532 learned_addr learned my addr 172.21.15.83:0/2635905532 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:58:29.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.102+0000 7f261d400640 1 --2- 172.21.15.83:0/2635905532 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55571101fc70 0x5557110270c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:29.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.102+0000 7f261e800640 1 -- 172.21.15.83:0/2635905532 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55571102a340 msgr2=0x5557110277b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:58:29.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.102+0000 7f261e800640 1 --2- 172.21.15.83:0/2635905532 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55571102a340 0x5557110277b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.102+0000 7f261e800640 1 -- 172.21.15.83:0/2635905532 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55571101fc70 msgr2=0x5557110270c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:58:29.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.102+0000 7f261e800640 1 --2- 172.21.15.83:0/2635905532 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55571101fc70 0x5557110270c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.102+0000 7f261e800640 1 -- 172.21.15.83:0/2635905532 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555710f463f0 con 0x55571101f350 2024-04-24T08:58:29.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.102+0000 7f261e800640 1 --2- 172.21.15.83:0/2635905532 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55571101f350 0x555711029dd0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f261800e010 tx=0x7f2618018480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96c05751004a1697 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:58:29.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.102+0000 7f25ff400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:58:29.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.103+0000 7f25ff400640 1 -- 172.21.15.83:0/2635905532 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2618027030 con 0x55571101f350 2024-04-24T08:58:29.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.103+0000 7f25ff400640 1 -- 172.21.15.83:0/2635905532 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9567+0+0 (secure 0 0 0) 0x7f2618020040 con 0x55571101f350 2024-04-24T08:58:29.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.103+0000 7f2627804740 1 -- 172.21.15.83:0/2635905532 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f261800de20 con 0x55571101f350 2024-04-24T08:58:29.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.103+0000 7f25ff400640 1 -- 172.21.15.83:0/2635905532 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f2618031050 con 0x55571101f350 2024-04-24T08:58:29.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.103+0000 7f2627804740 10 client.18084 fetch_fsmap learned FSMap version 33 2024-04-24T08:58:29.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.103+0000 7f2627804740 1 -- 172.21.15.83:0/2635905532 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x555710c8e3f0 con 0x55571101f350 2024-04-24T08:58:29.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.103+0000 7f25ff400640 1 -- 172.21.15.83:0/2635905532 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f261800de20 con 0x55571101f350 2024-04-24T08:58:29.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.103+0000 7f25ff400640 1 -- 172.21.15.83:0/2635905532 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f261802c820 con 0x55571101f350 2024-04-24T08:58:29.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.104+0000 7f2627804740 10 client.18084 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T08:58:29.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.104+0000 7f2627804740 10 client.18084 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-24T08:58:29.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.104+0000 7f2627804740 20 client.18084 populate_metadata read hostname 'smithi083' 2024-04-24T08:58:29.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.104+0000 7f2627804740 1 --2- 172.21.15.83:0/2635905532 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55571103a440 0x55571103c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:29.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.104+0000 7f2627804740 4 client.18084 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:58:29.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.104+0000 7f2627804740 1 -- 172.21.15.83:0/2635905532 --> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555710f89bf0 con 0x55571103a440 2024-04-24T08:58:29.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.104+0000 7f261d400640 1 --2- 172.21.15.83:0/2635905532 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55571103a440 0x55571103c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:29.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.106+0000 7f261d400640 1 --2- 172.21.15.83:0/2635905532 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55571103a440 0x55571103c830 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7848d55d0a3e0782 server_cookie=4cc6fba1044974e4 in_seq=0 out_seq=0 2024-04-24T08:58:29.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.106+0000 7f25ff400640 10 client.18084 ms_handle_connect on v2:172.21.15.138:6832/1157967658 2024-04-24T08:58:29.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.109+0000 7f25ff400640 1 -- 172.21.15.83:0/2635905532 <== mds.3 v2:172.21.15.138:6832/1157967658 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555710f89bf0 con 0x55571103a440 2024-04-24T08:58:29.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.109+0000 7f25ff400640 10 client.18084 handle_command_reply: tid=0 2024-04-24T08:58:29.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.110+0000 7f2627804740 1 client.18084 shutdown 2024-04-24T08:58:29.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.110+0000 7f2627804740 1 -- 172.21.15.83:0/2635905532 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55571101f350 msgr2=0x555711029dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:29.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.110+0000 7f2627804740 1 --2- 172.21.15.83:0/2635905532 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55571101f350 0x555711029dd0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f261800e010 tx=0x7f2618018480 comp rx=0 tx=0).stop 2024-04-24T08:58:29.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.110+0000 7f2627804740 1 -- 172.21.15.83:0/2635905532 shutdown_connections 2024-04-24T08:58:29.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.110+0000 7f2627804740 1 --2- 172.21.15.83:0/2635905532 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55571103a440 0x55571103c830 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.110+0000 7f2627804740 1 --2- 172.21.15.83:0/2635905532 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55571102a340 0x5557110277b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.110+0000 7f2627804740 1 --2- 172.21.15.83:0/2635905532 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55571101fc70 0x5557110270c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.110+0000 7f2627804740 1 --2- 172.21.15.83:0/2635905532 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55571101f350 0x555711029dd0 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.110+0000 7f2627804740 1 -- 172.21.15.83:0/2635905532 >> 172.21.15.83:0/2635905532 conn(0x55571101b670 msgr2=0x555711021470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:58:29.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.110+0000 7f25ff400640 0 client.18084 ms_handle_reset on v2:172.21.15.138:6832/1157967658 2024-04-24T08:58:29.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.110+0000 7f2627804740 1 -- 172.21.15.83:0/2635905532 shutdown_connections 2024-04-24T08:58:29.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.110+0000 7f2627804740 1 -- 172.21.15.83:0/2635905532 wait complete. 2024-04-24T08:58:29.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.110+0000 7f2627804740 20 client.18084 trim_cache size 0 max 16384 2024-04-24T08:58:29.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.127+0000 7f2627804740 1 Processor -- start 2024-04-24T08:58:29.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.127+0000 7f2627804740 1 -- start start 2024-04-24T08:58:29.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.127+0000 7f2627804740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55571101f350 0x555711036090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:29.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.127+0000 7f2627804740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55571101fc70 0x555711029900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:29.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.127+0000 7f2627804740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55571102a340 0x555711041e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:29.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.127+0000 7f2627804740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f26201ad1b0 con 0x55571101fc70 2024-04-24T08:58:29.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.127+0000 7f2627804740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f26201aecf0 con 0x55571101f350 2024-04-24T08:58:29.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.127+0000 7f2627804740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f26201ad880 con 0x55571102a340 2024-04-24T08:58:29.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.128+0000 7f261d400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55571101fc70 0x555711029900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:29.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.128+0000 7f261d400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55571101fc70 0x555711029900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47480/0 (socket says 172.21.15.83:47480) 2024-04-24T08:58:29.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.128+0000 7f261d400640 1 -- 172.21.15.83:0/486356188 learned_addr learned my addr 172.21.15.83:0/486356188 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:58:29.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.128+0000 7f261e800640 1 --2- 172.21.15.83:0/486356188 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55571101f350 0x555711036090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:29.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.128+0000 7f261d400640 1 -- 172.21.15.83:0/486356188 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55571102a340 msgr2=0x555711041e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:58:29.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.128+0000 7f261d400640 1 --2- 172.21.15.83:0/486356188 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55571102a340 0x555711041e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.128+0000 7f261d400640 1 -- 172.21.15.83:0/486356188 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55571101f350 msgr2=0x555711036090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:58:29.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.128+0000 7f261d400640 1 --2- 172.21.15.83:0/486356188 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55571101f350 0x555711036090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.128+0000 7f261d400640 1 -- 172.21.15.83:0/486356188 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555711027b20 con 0x55571101fc70 2024-04-24T08:58:29.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.128+0000 7f261e800640 1 --2- 172.21.15.83:0/486356188 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55571101f350 0x555711036090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:58:29.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.129+0000 7f261d400640 1 --2- 172.21.15.83:0/486356188 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55571101fc70 0x555711029900 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f262019be90 tx=0x7f260c00d6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b2d654675f4b42f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:58:29.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.129+0000 7f260a600640 1 -- 172.21.15.83:0/486356188 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f260c012a40 con 0x55571101fc70 2024-04-24T08:58:29.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.129+0000 7f260a600640 1 -- 172.21.15.83:0/486356188 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f260c0197e0 con 0x55571101fc70 2024-04-24T08:58:29.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.129+0000 7f260a600640 1 -- 172.21.15.83:0/486356188 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f260c010720 con 0x55571101fc70 2024-04-24T08:58:29.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.130+0000 7f2627804740 1 -- 172.21.15.83:0/486356188 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55571101fc70 msgr2=0x555711029900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:29.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.130+0000 7f2627804740 1 --2- 172.21.15.83:0/486356188 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55571101fc70 0x555711029900 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f262019be90 tx=0x7f260c00d6b0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.130+0000 7f2627804740 1 -- 172.21.15.83:0/486356188 shutdown_connections 2024-04-24T08:58:29.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.130+0000 7f2627804740 1 --2- 172.21.15.83:0/486356188 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55571102a340 0x555711041e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.130+0000 7f2627804740 1 --2- 172.21.15.83:0/486356188 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55571101fc70 0x555711029900 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.130+0000 7f2627804740 1 --2- 172.21.15.83:0/486356188 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55571101f350 0x555711036090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.130+0000 7f2627804740 1 -- 172.21.15.83:0/486356188 >> 172.21.15.83:0/486356188 conn(0x55571101b670 msgr2=0x555711022380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:58:29.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.130+0000 7f2627804740 1 -- 172.21.15.83:0/486356188 shutdown_connections 2024-04-24T08:58:29.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.130+0000 7f2627804740 1 -- 172.21.15.83:0/486356188 wait complete. 2024-04-24T08:58:29.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.130+0000 7f2627804740 1 -- start start 2024-04-24T08:58:29.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.131+0000 7f2627804740 1 Processor -- start 2024-04-24T08:58:29.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.131+0000 7f2627804740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55571101f350 0x555711032790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:29.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.131+0000 7f2627804740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55571101fc70 0x555711024360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:29.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.131+0000 7f2627804740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55571102a340 0x5557110248a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:29.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.131+0000 7f2627804740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f26201ae620 con 0x55571101f350 2024-04-24T08:58:29.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.131+0000 7f2627804740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f26201adf50 con 0x55571102a340 2024-04-24T08:58:29.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.131+0000 7f2627804740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f26201ab190 con 0x55571101fc70 2024-04-24T08:58:29.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.131+0000 7f2627804740 10 client.0 _ll_register_callbacks cb 0x7f2618008bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:58:29.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.131+0000 7f261e800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55571101f350 0x555711032790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:29.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.132+0000 7f261e800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55571101f350 0x555711032790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47484/0 (socket says 172.21.15.83:47484) 2024-04-24T08:58:29.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.132+0000 7f261e800640 1 -- 172.21.15.83:0/1898737334 learned_addr learned my addr 172.21.15.83:0/1898737334 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:58:29.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.132+0000 7f261d400640 1 --2- 172.21.15.83:0/1898737334 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55571101fc70 0x555711024360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:29.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.132+0000 7f261e800640 1 -- 172.21.15.83:0/1898737334 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55571101fc70 msgr2=0x555711024360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:58:29.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.132+0000 7f261e800640 1 --2- 172.21.15.83:0/1898737334 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55571101fc70 0x555711024360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.132+0000 7f261e800640 1 -- 172.21.15.83:0/1898737334 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55571102a340 msgr2=0x5557110248a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:58:29.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.132+0000 7f261e800640 1 --2- 172.21.15.83:0/1898737334 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55571102a340 0x5557110248a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.132+0000 7f261e800640 1 -- 172.21.15.83:0/1898737334 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555710d587d0 con 0x55571101f350 2024-04-24T08:58:29.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.132+0000 7f261e800640 1 --2- 172.21.15.83:0/1898737334 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55571101f350 0x555711032790 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f26180086d0 tx=0x7f2618004a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b9f54cc4473e912 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:58:29.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.132+0000 7f25ff400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:58:29.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.132+0000 7f25ff400640 1 -- 172.21.15.83:0/1898737334 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f261803a060 con 0x55571101f350 2024-04-24T08:58:29.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.132+0000 7f25ff400640 1 -- 172.21.15.83:0/1898737334 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9567+0+0 (secure 0 0 0) 0x7f261802c6c0 con 0x55571101f350 2024-04-24T08:58:29.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.133+0000 7f2627804740 1 -- 172.21.15.83:0/1898737334 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f260c0197e0 con 0x55571101f350 2024-04-24T08:58:29.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.133+0000 7f25ff400640 1 -- 172.21.15.83:0/1898737334 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f2618041050 con 0x55571101f350 2024-04-24T08:58:29.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.133+0000 7f2627804740 10 client.18090 fetch_fsmap learned FSMap version 33 2024-04-24T08:58:29.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.133+0000 7f2627804740 1 -- 172.21.15.83:0/1898737334 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55571102b670 con 0x55571101f350 2024-04-24T08:58:29.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.133+0000 7f25ff400640 1 -- 172.21.15.83:0/1898737334 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f260c0197e0 con 0x55571101f350 2024-04-24T08:58:29.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.133+0000 7f25ff400640 1 -- 172.21.15.83:0/1898737334 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f26180236b0 con 0x55571101f350 2024-04-24T08:58:29.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.133+0000 7f2627804740 10 client.18090 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T08:58:29.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.133+0000 7f2627804740 10 client.18090 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-24T08:58:29.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.133+0000 7f2627804740 20 client.18090 populate_metadata read hostname 'smithi083' 2024-04-24T08:58:29.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.133+0000 7f2627804740 1 --2- 172.21.15.83:0/1898737334 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x5557110a0940 0x5557110a2d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:29.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.133+0000 7f2627804740 4 client.18090 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T08:58:29.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.133+0000 7f2627804740 1 -- 172.21.15.83:0/1898737334 --> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555710f83a60 con 0x5557110a0940 2024-04-24T08:58:29.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.134+0000 7f261d400640 1 --2- 172.21.15.83:0/1898737334 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x5557110a0940 0x5557110a2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:29.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.135+0000 7f261d400640 1 --2- 172.21.15.83:0/1898737334 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x5557110a0940 0x5557110a2d60 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ca9856838451acd0 server_cookie=c1453a08cab876b1 in_seq=0 out_seq=0 2024-04-24T08:58:29.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.135+0000 7f25ff400640 10 client.18090 ms_handle_connect on v2:172.21.15.138:6832/1157967658 2024-04-24T08:58:29.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.135+0000 7f25ff400640 1 -- 172.21.15.83:0/1898737334 <== mds.3 v2:172.21.15.138:6832/1157967658 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555710f83a60 con 0x5557110a0940 2024-04-24T08:58:29.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.135+0000 7f25ff400640 10 client.18090 handle_command_reply: tid=0 2024-04-24T08:58:29.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.136+0000 7f2627804740 1 client.18090 shutdown 2024-04-24T08:58:29.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.136+0000 7f2627804740 1 -- 172.21.15.83:0/1898737334 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55571101f350 msgr2=0x555711032790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:29.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.136+0000 7f2627804740 1 --2- 172.21.15.83:0/1898737334 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55571101f350 0x555711032790 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f26180086d0 tx=0x7f2618004a70 comp rx=0 tx=0).stop 2024-04-24T08:58:29.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.136+0000 7f2627804740 1 -- 172.21.15.83:0/1898737334 shutdown_connections 2024-04-24T08:58:29.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.136+0000 7f2627804740 1 --2- 172.21.15.83:0/1898737334 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55571102a340 0x5557110248a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.136+0000 7f2627804740 1 --2- 172.21.15.83:0/1898737334 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x5557110a0940 0x5557110a2d60 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.136+0000 7f2627804740 1 --2- 172.21.15.83:0/1898737334 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55571101fc70 0x555711024360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.136+0000 7f2627804740 1 --2- 172.21.15.83:0/1898737334 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55571101f350 0x555711032790 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.136+0000 7f25ff400640 0 client.18090 ms_handle_reset on v2:172.21.15.138:6832/1157967658 2024-04-24T08:58:29.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.136+0000 7f2627804740 1 -- 172.21.15.83:0/1898737334 >> 172.21.15.83:0/1898737334 conn(0x55571101b670 msgr2=0x555711025840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:58:29.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.136+0000 7f2627804740 1 -- 172.21.15.83:0/1898737334 shutdown_connections 2024-04-24T08:58:29.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.136+0000 7f2627804740 1 -- 172.21.15.83:0/1898737334 wait complete. 2024-04-24T08:58:29.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.136+0000 7f2627804740 20 client.18090 trim_cache size 0 max 16384 2024-04-24T08:58:29.139 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T08:58:29.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.137+0000 7f261f200640 1 -- 172.21.15.83:0/404207530 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f26040a3da0 msgr2=0x7f26040a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:29.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.137+0000 7f261f200640 1 --2- 172.21.15.83:0/404207530 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f26040a3da0 0x7f26040a6260 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f26100eea10 tx=0x7f261002a3f0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.137+0000 7f261f200640 1 -- 172.21.15.83:0/404207530 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2620104310 msgr2=0x7f26201a0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:29.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.137+0000 7f261f200640 1 --2- 172.21.15.83:0/404207530 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2620104310 0x7f26201a0580 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f2610005ab0 tx=0x7f2610003d50 comp rx=0 tx=0).stop 2024-04-24T08:58:29.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.137+0000 7f261f200640 1 -- 172.21.15.83:0/404207530 shutdown_connections 2024-04-24T08:58:29.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.137+0000 7f261f200640 1 --2- 172.21.15.83:0/404207530 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f26040a3da0 0x7f26040a6260 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.137+0000 7f261f200640 1 --2- 172.21.15.83:0/404207530 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f262019a690 0x7f262019ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.137+0000 7f261f200640 1 --2- 172.21.15.83:0/404207530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26201093e0 0x7f26201a0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.137+0000 7f261f200640 1 --2- 172.21.15.83:0/404207530 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2620104310 0x7f26201a0580 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.137+0000 7f261f200640 1 -- 172.21.15.83:0/404207530 >> 172.21.15.83:0/404207530 conn(0x7f26200ffb10 msgr2=0x7f2620101e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:58:29.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.137+0000 7f261f200640 1 -- 172.21.15.83:0/404207530 shutdown_connections 2024-04-24T08:58:29.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.137+0000 7f261f200640 1 -- 172.21.15.83:0/404207530 wait complete. 2024-04-24T08:58:29.155 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T08:58:29.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:58:28 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/2767749272' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:58:29.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:58:28 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/2767749272' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:58:29.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.361+0000 7f9537200640 1 -- 172.21.15.83:0/2920293204 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9538102110 msgr2=0x7f95381024f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:29.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.361+0000 7f9537200640 1 --2- 172.21.15.83:0/2920293204 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9538102110 0x7f95381024f0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f9524009900 tx=0x7f952402a410 comp rx=0 tx=0).stop 2024-04-24T08:58:29.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.361+0000 7f9537200640 1 -- 172.21.15.83:0/2920293204 shutdown_connections 2024-04-24T08:58:29.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.361+0000 7f9537200640 1 --2- 172.21.15.83:0/2920293204 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95381071e0 0x7f953810edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.361+0000 7f9537200640 1 --2- 172.21.15.83:0/2920293204 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9538102ac0 0x7f9538106b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.361+0000 7f9537200640 1 --2- 172.21.15.83:0/2920293204 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9538102110 0x7f95381024f0 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.361+0000 7f9537200640 1 -- 172.21.15.83:0/2920293204 >> 172.21.15.83:0/2920293204 conn(0x7f95380fd9d0 msgr2=0x7f95380ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:58:29.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.361+0000 7f9537200640 1 -- 172.21.15.83:0/2920293204 shutdown_connections 2024-04-24T08:58:29.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.361+0000 7f9537200640 1 -- 172.21.15.83:0/2920293204 wait complete. 2024-04-24T08:58:29.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.362+0000 7f9537200640 1 Processor -- start 2024-04-24T08:58:29.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.362+0000 7f9537200640 1 -- start start 2024-04-24T08:58:29.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.362+0000 7f9537200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9538102110 0x7f95381a05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:29.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.362+0000 7f9537200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f95381071e0 0x7f95381a0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:29.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.362+0000 7f9537200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f953819a690 0x7f953819ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:29.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.362+0000 7f9537200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9538112190 con 0x7f9538102110 2024-04-24T08:58:29.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.362+0000 7f9537200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9538112010 con 0x7f953819a690 2024-04-24T08:58:29.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.362+0000 7f9537200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9538112310 con 0x7f95381071e0 2024-04-24T08:58:29.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.363+0000 7f9535e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9538102110 0x7f95381a05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:29.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.363+0000 7f9535e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9538102110 0x7f95381a05d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47508/0 (socket says 172.21.15.83:47508) 2024-04-24T08:58:29.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.363+0000 7f9535e00640 1 -- 172.21.15.83:0/279839912 learned_addr learned my addr 172.21.15.83:0/279839912 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:58:29.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.363+0000 7f9535400640 1 --2- 172.21.15.83:0/279839912 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f95381071e0 0x7f95381a0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:29.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.363+0000 7f9535e00640 1 -- 172.21.15.83:0/279839912 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f95381071e0 msgr2=0x7f95381a0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:29.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.363+0000 7f9535e00640 1 --2- 172.21.15.83:0/279839912 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f95381071e0 0x7f95381a0b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.363+0000 7f9535e00640 1 -- 172.21.15.83:0/279839912 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f953819a690 msgr2=0x7f953819ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:29.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.363+0000 7f9535e00640 1 --2- 172.21.15.83:0/279839912 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f953819a690 0x7f953819ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.363+0000 7f9535e00640 1 -- 172.21.15.83:0/279839912 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f953819b290 con 0x7f9538102110 2024-04-24T08:58:29.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.363+0000 7f9535e00640 1 --2- 172.21.15.83:0/279839912 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9538102110 0x7f95381a05d0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f9524005ab0 tx=0x7f9524003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:58:29.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.364+0000 7f952ea00640 1 -- 172.21.15.83:0/279839912 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f952403f070 con 0x7f9538102110 2024-04-24T08:58:29.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.364+0000 7f952ea00640 1 -- 172.21.15.83:0/279839912 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9524031d60 con 0x7f9538102110 2024-04-24T08:58:29.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.364+0000 7f9537200640 1 -- 172.21.15.83:0/279839912 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f953819b520 con 0x7f9538102110 2024-04-24T08:58:29.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.364+0000 7f952ea00640 1 -- 172.21.15.83:0/279839912 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9524035030 con 0x7f9538102110 2024-04-24T08:58:29.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.364+0000 7f9537200640 1 -- 172.21.15.83:0/279839912 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95381a7660 con 0x7f9538102110 2024-04-24T08:58:29.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.367+0000 7f952ea00640 1 -- 172.21.15.83:0/279839912 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9524042070 con 0x7f9538102110 2024-04-24T08:58:29.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.367+0000 7f953f7ed740 1 Processor -- start 2024-04-24T08:58:29.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.368+0000 7f953f7ed740 1 -- start start 2024-04-24T08:58:29.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.368+0000 7f953f7ed740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffc8039b40 0x55ffc8039f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:29.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.368+0000 7f953f7ed740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ffc803a460 0x55ffc80445f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:29.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.368+0000 7f953f7ed740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ffc8044b30 0x55ffc8046f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:29.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.368+0000 7f953f7ed740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ffc7f9f1d0 con 0x55ffc8044b30 2024-04-24T08:58:29.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.368+0000 7f953f7ed740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ffc7f45450 con 0x55ffc8039b40 2024-04-24T08:58:29.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.368+0000 7f953f7ed740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55ffc7f9b190 con 0x55ffc803a460 2024-04-24T08:58:29.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.368+0000 7f9535400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ffc8044b30 0x55ffc8046f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:29.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.369+0000 7f9535400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ffc8044b30 0x55ffc8046f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47512/0 (socket says 172.21.15.83:47512) 2024-04-24T08:58:29.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.369+0000 7f9535400640 1 -- 172.21.15.83:0/211025821 learned_addr learned my addr 172.21.15.83:0/211025821 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:58:29.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.369+0000 7f9535e00640 1 --2- 172.21.15.83:0/211025821 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ffc803a460 0x55ffc80445f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:29.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.369+0000 7f9536800640 1 --2- 172.21.15.83:0/211025821 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffc8039b40 0x55ffc8039f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:29.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.369+0000 7f952ea00640 1 --2- 172.21.15.83:0/279839912 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f951c0a3d20 0x7f951c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:29.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.369+0000 7f9535400640 1 -- 172.21.15.83:0/211025821 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ffc803a460 msgr2=0x55ffc80445f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:58:29.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.369+0000 7f9535400640 1 --2- 172.21.15.83:0/211025821 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ffc803a460 0x55ffc80445f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.369+0000 7f9535400640 1 -- 172.21.15.83:0/211025821 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffc8039b40 msgr2=0x55ffc8039f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:58:29.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.369+0000 7f9535400640 1 --2- 172.21.15.83:0/211025821 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffc8039b40 0x55ffc8039f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.369+0000 7f9535400640 1 -- 172.21.15.83:0/211025821 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffc8002310 con 0x55ffc8044b30 2024-04-24T08:58:29.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.369+0000 7f9535e00640 1 --2- 172.21.15.83:0/211025821 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ffc803a460 0x55ffc80445f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:58:29.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.369+0000 7f952ea00640 1 -- 172.21.15.83:0/279839912 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9567+0+0 (secure 0 0 0) 0x7f95240f1580 con 0x7f9538102110 2024-04-24T08:58:29.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.369+0000 7f9536800640 1 --2- 172.21.15.83:0/279839912 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f951c0a3d20 0x7f951c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:29.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.369+0000 7f9535400640 1 --2- 172.21.15.83:0/211025821 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ffc8044b30 0x55ffc8046f20 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f9528009de0 tx=0x7f952800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f88c7e39bd5e318 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:58:29.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.370+0000 7f9523000640 1 -- 172.21.15.83:0/211025821 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9528003b70 con 0x55ffc8044b30 2024-04-24T08:58:29.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.370+0000 7f9523000640 1 -- 172.21.15.83:0/211025821 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9528003d10 con 0x55ffc8044b30 2024-04-24T08:58:29.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.370+0000 7f9523000640 1 -- 172.21.15.83:0/211025821 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9528011970 con 0x55ffc8044b30 2024-04-24T08:58:29.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.370+0000 7f953f7ed740 1 -- 172.21.15.83:0/211025821 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ffc8044b30 msgr2=0x55ffc8046f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:29.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.370+0000 7f9536800640 1 --2- 172.21.15.83:0/279839912 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f951c0a3d20 0x7f951c0a61e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9530005940 tx=0x7f95300046c0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:58:29.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.370+0000 7f953f7ed740 1 --2- 172.21.15.83:0/211025821 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ffc8044b30 0x55ffc8046f20 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f9528009de0 tx=0x7f952800f730 comp rx=0 tx=0).stop 2024-04-24T08:58:29.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.371+0000 7f953f7ed740 1 -- 172.21.15.83:0/211025821 shutdown_connections 2024-04-24T08:58:29.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.371+0000 7f953f7ed740 1 --2- 172.21.15.83:0/211025821 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ffc8044b30 0x55ffc8046f20 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.371+0000 7f953f7ed740 1 --2- 172.21.15.83:0/211025821 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ffc803a460 0x55ffc80445f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.371+0000 7f953f7ed740 1 --2- 172.21.15.83:0/211025821 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffc8039b40 0x55ffc8039f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.371+0000 7f953f7ed740 1 -- 172.21.15.83:0/211025821 >> 172.21.15.83:0/211025821 conn(0x55ffc8035e60 msgr2=0x55ffc8036240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:58:29.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.371+0000 7f953f7ed740 1 -- 172.21.15.83:0/211025821 shutdown_connections 2024-04-24T08:58:29.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.371+0000 7f953f7ed740 1 -- 172.21.15.83:0/211025821 wait complete. 2024-04-24T08:58:29.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.371+0000 7f953f7ed740 1 -- start start 2024-04-24T08:58:29.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.371+0000 7f953f7ed740 1 Processor -- start 2024-04-24T08:58:29.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.372+0000 7f953f7ed740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ffc8039b40 0x55ffc8044560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:29.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.372+0000 7f953f7ed740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffc803a460 0x55ffc8041850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:29.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.372+0000 7f953f7ed740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ffc8044b30 0x55ffc8041f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:29.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.372+0000 7f953f7ed740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ffc80479c0 con 0x55ffc8044b30 2024-04-24T08:58:29.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.372+0000 7f953f7ed740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ffc7f58810 con 0x55ffc803a460 2024-04-24T08:58:29.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.372+0000 7f953f7ed740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55ffc8047b40 con 0x55ffc8039b40 2024-04-24T08:58:29.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.372+0000 7f953f7ed740 10 client.0 _ll_register_callbacks cb 0x55ffc7f26140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:58:29.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.372+0000 7f9536800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ffc8044b30 0x55ffc8041f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:29.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.372+0000 7f9536800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ffc8044b30 0x55ffc8041f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47514/0 (socket says 172.21.15.83:47514) 2024-04-24T08:58:29.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.372+0000 7f9536800640 1 -- 172.21.15.83:0/3581419705 learned_addr learned my addr 172.21.15.83:0/3581419705 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:58:29.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.372+0000 7f9535e00640 1 --2- 172.21.15.83:0/3581419705 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ffc8039b40 0x55ffc8044560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:29.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.372+0000 7f9535400640 1 --2- 172.21.15.83:0/3581419705 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffc803a460 0x55ffc8041850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:29.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.372+0000 7f9536800640 1 -- 172.21.15.83:0/3581419705 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ffc8039b40 msgr2=0x55ffc8044560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:58:29.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.373+0000 7f9536800640 1 --2- 172.21.15.83:0/3581419705 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ffc8039b40 0x55ffc8044560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.373+0000 7f9536800640 1 -- 172.21.15.83:0/3581419705 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffc803a460 msgr2=0x55ffc8041850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:58:29.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.373+0000 7f9536800640 1 --2- 172.21.15.83:0/3581419705 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffc803a460 0x55ffc8041850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.373+0000 7f9536800640 1 -- 172.21.15.83:0/3581419705 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ffc7f693f0 con 0x55ffc8044b30 2024-04-24T08:58:29.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.373+0000 7f9535400640 1 --2- 172.21.15.83:0/3581419705 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffc803a460 0x55ffc8041850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:58:29.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.373+0000 7f9536800640 1 --2- 172.21.15.83:0/3581419705 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ffc8044b30 0x55ffc8041f90 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f95300119f0 tx=0x7f9530005e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb5febb0bf598826 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:58:29.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.373+0000 7f9517400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:58:29.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.373+0000 7f9517400640 1 -- 172.21.15.83:0/3581419705 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9530022020 con 0x55ffc8044b30 2024-04-24T08:58:29.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.373+0000 7f9517400640 1 -- 172.21.15.83:0/3581419705 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9567+0+0 (secure 0 0 0) 0x7f9530010d60 con 0x55ffc8044b30 2024-04-24T08:58:29.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.373+0000 7f953f7ed740 1 -- 172.21.15.83:0/3581419705 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9528003d10 con 0x55ffc8044b30 2024-04-24T08:58:29.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.374+0000 7f9517400640 1 -- 172.21.15.83:0/3581419705 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f953002d050 con 0x55ffc8044b30 2024-04-24T08:58:29.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.374+0000 7f953f7ed740 10 client.18102 fetch_fsmap learned FSMap version 33 2024-04-24T08:58:29.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.374+0000 7f953f7ed740 1 -- 172.21.15.83:0/3581419705 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55ffc7cb13f0 con 0x55ffc8044b30 2024-04-24T08:58:29.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.374+0000 7f9517400640 1 -- 172.21.15.83:0/3581419705 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9528003d10 con 0x55ffc8044b30 2024-04-24T08:58:29.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.374+0000 7f9517400640 1 -- 172.21.15.83:0/3581419705 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f953000f040 con 0x55ffc8044b30 2024-04-24T08:58:29.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.374+0000 7f953f7ed740 10 client.18102 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T08:58:29.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.374+0000 7f953f7ed740 10 client.18102 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-24T08:58:29.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.374+0000 7f953f7ed740 20 client.18102 populate_metadata read hostname 'smithi083' 2024-04-24T08:58:29.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.374+0000 7f953f7ed740 1 --2- 172.21.15.83:0/3581419705 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55ffc8054dd0 0x55ffc8057250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:29.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.374+0000 7f953f7ed740 4 client.18102 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:58:29.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.374+0000 7f953f7ed740 1 -- 172.21.15.83:0/3581419705 --> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ffc7facbf0 con 0x55ffc8054dd0 2024-04-24T08:58:29.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.375+0000 7f9535e00640 1 --2- 172.21.15.83:0/3581419705 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55ffc8054dd0 0x55ffc8057250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:29.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.376+0000 7f9517400640 10 client.18102 ms_handle_connect on v2:172.21.15.138:6834/1909219758 2024-04-24T08:58:29.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.376+0000 7f9535e00640 1 --2- 172.21.15.83:0/3581419705 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55ffc8054dd0 0x55ffc8057250 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=15cda3b28836011d server_cookie=9dd60730a58e949c in_seq=0 out_seq=0 2024-04-24T08:58:29.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.378+0000 7f9517400640 1 -- 172.21.15.83:0/3581419705 <== mds.2 v2:172.21.15.138:6834/1909219758 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ffc7facbf0 con 0x55ffc8054dd0 2024-04-24T08:58:29.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.378+0000 7f9517400640 10 client.18102 handle_command_reply: tid=0 2024-04-24T08:58:29.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.378+0000 7f953f7ed740 1 client.18102 shutdown 2024-04-24T08:58:29.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.379+0000 7f953f7ed740 1 -- 172.21.15.83:0/3581419705 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ffc8044b30 msgr2=0x55ffc8041f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:29.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.379+0000 7f953f7ed740 1 --2- 172.21.15.83:0/3581419705 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ffc8044b30 0x55ffc8041f90 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f95300119f0 tx=0x7f9530005e90 comp rx=0 tx=0).stop 2024-04-24T08:58:29.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.379+0000 7f953f7ed740 1 -- 172.21.15.83:0/3581419705 shutdown_connections 2024-04-24T08:58:29.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.379+0000 7f953f7ed740 1 --2- 172.21.15.83:0/3581419705 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55ffc8054dd0 0x55ffc8057250 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.379+0000 7f953f7ed740 1 --2- 172.21.15.83:0/3581419705 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ffc8044b30 0x55ffc8041f90 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.379+0000 7f953f7ed740 1 --2- 172.21.15.83:0/3581419705 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffc803a460 0x55ffc8041850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.379+0000 7f953f7ed740 1 --2- 172.21.15.83:0/3581419705 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ffc8039b40 0x55ffc8044560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.379+0000 7f9517400640 0 client.18102 ms_handle_reset on v2:172.21.15.138:6834/1909219758 2024-04-24T08:58:29.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.379+0000 7f953f7ed740 1 -- 172.21.15.83:0/3581419705 >> 172.21.15.83:0/3581419705 conn(0x55ffc8035e60 msgr2=0x55ffc803bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:58:29.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.379+0000 7f953f7ed740 1 -- 172.21.15.83:0/3581419705 shutdown_connections 2024-04-24T08:58:29.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.379+0000 7f953f7ed740 1 -- 172.21.15.83:0/3581419705 wait complete. 2024-04-24T08:58:29.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.379+0000 7f953f7ed740 20 client.18102 trim_cache size 0 max 16384 2024-04-24T08:58:29.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.395+0000 7f953f7ed740 1 Processor -- start 2024-04-24T08:58:29.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.395+0000 7f953f7ed740 1 -- start start 2024-04-24T08:58:29.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.396+0000 7f953f7ed740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffc8039b40 0x55ffc8050880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:29.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.396+0000 7f953f7ed740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ffc803a460 0x55ffc8044090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:29.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.396+0000 7f953f7ed740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ffc8044b30 0x55ffc80445d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:29.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.396+0000 7f953f7ed740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f95381b1490 con 0x55ffc803a460 2024-04-24T08:58:29.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.396+0000 7f953f7ed740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f95381b2fd0 con 0x55ffc8039b40 2024-04-24T08:58:29.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.396+0000 7f953f7ed740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f95381b1b60 con 0x55ffc8044b30 2024-04-24T08:58:29.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.396+0000 7f9535400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ffc803a460 0x55ffc8044090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:29.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.396+0000 7f9535400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ffc803a460 0x55ffc8044090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47518/0 (socket says 172.21.15.83:47518) 2024-04-24T08:58:29.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.396+0000 7f9535400640 1 -- 172.21.15.83:0/2340208642 learned_addr learned my addr 172.21.15.83:0/2340208642 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:58:29.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.396+0000 7f9535e00640 1 --2- 172.21.15.83:0/2340208642 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffc8039b40 0x55ffc8050880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:29.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.397+0000 7f9535400640 1 -- 172.21.15.83:0/2340208642 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ffc8044b30 msgr2=0x55ffc80445d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:58:29.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.397+0000 7f9536800640 1 --2- 172.21.15.83:0/2340208642 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ffc8044b30 0x55ffc80445d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:29.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.397+0000 7f9535400640 1 --2- 172.21.15.83:0/2340208642 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ffc8044b30 0x55ffc80445d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.397+0000 7f9535400640 1 -- 172.21.15.83:0/2340208642 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffc8039b40 msgr2=0x55ffc8050880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:58:29.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.397+0000 7f9535400640 1 --2- 172.21.15.83:0/2340208642 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffc8039b40 0x55ffc8050880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.397+0000 7f9535400640 1 -- 172.21.15.83:0/2340208642 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffc7d7b7d0 con 0x55ffc803a460 2024-04-24T08:58:29.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.397+0000 7f9535e00640 1 --2- 172.21.15.83:0/2340208642 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffc8039b40 0x55ffc8050880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:58:29.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.397+0000 7f9536800640 1 --2- 172.21.15.83:0/2340208642 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ffc8044b30 0x55ffc80445d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:58:29.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.397+0000 7f9535400640 1 --2- 172.21.15.83:0/2340208642 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ffc803a460 0x55ffc8044090 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f9528009de0 tx=0x7f95280099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8172c6aa26d8bb77 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:58:29.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.397+0000 7f9522600640 1 -- 172.21.15.83:0/2340208642 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9528003970 con 0x55ffc803a460 2024-04-24T08:58:29.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.397+0000 7f9522600640 1 -- 172.21.15.83:0/2340208642 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9528005040 con 0x55ffc803a460 2024-04-24T08:58:29.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.397+0000 7f9522600640 1 -- 172.21.15.83:0/2340208642 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9528011680 con 0x55ffc803a460 2024-04-24T08:58:29.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.398+0000 7f953f7ed740 1 -- 172.21.15.83:0/2340208642 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ffc803a460 msgr2=0x55ffc8044090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:29.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.398+0000 7f953f7ed740 1 --2- 172.21.15.83:0/2340208642 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ffc803a460 0x55ffc8044090 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f9528009de0 tx=0x7f95280099e0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.398+0000 7f953f7ed740 1 -- 172.21.15.83:0/2340208642 shutdown_connections 2024-04-24T08:58:29.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.398+0000 7f953f7ed740 1 --2- 172.21.15.83:0/2340208642 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ffc8044b30 0x55ffc80445d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.398+0000 7f953f7ed740 1 --2- 172.21.15.83:0/2340208642 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ffc803a460 0x55ffc8044090 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.398+0000 7f953f7ed740 1 --2- 172.21.15.83:0/2340208642 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffc8039b40 0x55ffc8050880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.398+0000 7f953f7ed740 1 -- 172.21.15.83:0/2340208642 >> 172.21.15.83:0/2340208642 conn(0x55ffc8035e60 msgr2=0x55ffc803cb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:58:29.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.398+0000 7f953f7ed740 1 -- 172.21.15.83:0/2340208642 shutdown_connections 2024-04-24T08:58:29.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.398+0000 7f953f7ed740 1 -- 172.21.15.83:0/2340208642 wait complete. 2024-04-24T08:58:29.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.399+0000 7f953f7ed740 1 -- start start 2024-04-24T08:58:29.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.399+0000 7f953f7ed740 1 Processor -- start 2024-04-24T08:58:29.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.399+0000 7f953f7ed740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ffc8039b40 0x55ffc8045920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:29.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.399+0000 7f953f7ed740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ffc803a460 0x55ffc8045e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:29.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.399+0000 7f953f7ed740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffc8044b30 0x55ffc80463a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:29.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.399+0000 7f953f7ed740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f95381b2900 con 0x55ffc8039b40 2024-04-24T08:58:29.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.399+0000 7f953f7ed740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f95381b2230 con 0x55ffc8044b30 2024-04-24T08:58:29.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.399+0000 7f953f7ed740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f95381af470 con 0x55ffc803a460 2024-04-24T08:58:29.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.400+0000 7f953f7ed740 10 client.0 _ll_register_callbacks cb 0x7f953000a780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:58:29.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.400+0000 7f9535e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ffc8039b40 0x55ffc8045920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:29.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.400+0000 7f9535e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ffc8039b40 0x55ffc8045920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47524/0 (socket says 172.21.15.83:47524) 2024-04-24T08:58:29.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.400+0000 7f9535e00640 1 -- 172.21.15.83:0/3501155606 learned_addr learned my addr 172.21.15.83:0/3501155606 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:58:29.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.400+0000 7f9535e00640 1 -- 172.21.15.83:0/3501155606 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ffc803a460 msgr2=0x55ffc8045e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:58:29.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.400+0000 7f9535e00640 1 --2- 172.21.15.83:0/3501155606 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ffc803a460 0x55ffc8045e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.400+0000 7f9535e00640 1 -- 172.21.15.83:0/3501155606 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffc8044b30 msgr2=0x55ffc80463a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:58:29.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.400+0000 7f9535e00640 1 --2- 172.21.15.83:0/3501155606 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffc8044b30 0x55ffc80463a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.400+0000 7f9535e00640 1 -- 172.21.15.83:0/3501155606 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ffc804b090 con 0x55ffc8039b40 2024-04-24T08:58:29.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.400+0000 7f9535e00640 1 --2- 172.21.15.83:0/3501155606 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ffc8039b40 0x55ffc8045920 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f9524005ec0 tx=0x7f952402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcb5e42b205905d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:58:29.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.400+0000 7f9517400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:58:29.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.401+0000 7f9517400640 1 -- 172.21.15.83:0/3501155606 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9524049040 con 0x55ffc8039b40 2024-04-24T08:58:29.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.401+0000 7f9517400640 1 -- 172.21.15.83:0/3501155606 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9567+0+0 (secure 0 0 0) 0x7f95240b9780 con 0x55ffc8039b40 2024-04-24T08:58:29.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.401+0000 7f953f7ed740 1 -- 172.21.15.83:0/3501155606 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9528005040 con 0x55ffc8039b40 2024-04-24T08:58:29.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.401+0000 7f9517400640 1 -- 172.21.15.83:0/3501155606 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f95240d6050 con 0x55ffc8039b40 2024-04-24T08:58:29.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.401+0000 7f953f7ed740 10 client.18108 fetch_fsmap learned FSMap version 33 2024-04-24T08:58:29.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.401+0000 7f953f7ed740 1 -- 172.21.15.83:0/3501155606 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55ffc80bb6b0 con 0x55ffc8039b40 2024-04-24T08:58:29.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.401+0000 7f9517400640 1 -- 172.21.15.83:0/3501155606 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9528005040 con 0x55ffc8039b40 2024-04-24T08:58:29.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.401+0000 7f9517400640 1 -- 172.21.15.83:0/3501155606 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f952403b5f0 con 0x55ffc8039b40 2024-04-24T08:58:29.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.401+0000 7f953f7ed740 10 client.18108 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T08:58:29.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.401+0000 7f953f7ed740 10 client.18108 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-24T08:58:29.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.401+0000 7f953f7ed740 20 client.18108 populate_metadata read hostname 'smithi083' 2024-04-24T08:58:29.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.401+0000 7f953f7ed740 1 --2- 172.21.15.83:0/3501155606 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55ffc80bbe10 0x55ffc80be2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:29.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.401+0000 7f953f7ed740 4 client.18108 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T08:58:29.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.401+0000 7f953f7ed740 1 -- 172.21.15.83:0/3501155606 --> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ffc7fa6a60 con 0x55ffc80bbe10 2024-04-24T08:58:29.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.402+0000 7f9535400640 1 --2- 172.21.15.83:0/3501155606 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55ffc80bbe10 0x55ffc80be2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:29.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.403+0000 7f9535400640 1 --2- 172.21.15.83:0/3501155606 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55ffc80bbe10 0x55ffc80be2d0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8f9d50f583e96652 server_cookie=ccf9b22e065e6e0a in_seq=0 out_seq=0 2024-04-24T08:58:29.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.403+0000 7f9517400640 10 client.18108 ms_handle_connect on v2:172.21.15.138:6834/1909219758 2024-04-24T08:58:29.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.404+0000 7f9517400640 1 -- 172.21.15.83:0/3501155606 <== mds.2 v2:172.21.15.138:6834/1909219758 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ffc7fa6a60 con 0x55ffc80bbe10 2024-04-24T08:58:29.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.404+0000 7f9517400640 10 client.18108 handle_command_reply: tid=0 2024-04-24T08:58:29.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.404+0000 7f953f7ed740 1 client.18108 shutdown 2024-04-24T08:58:29.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.404+0000 7f953f7ed740 1 -- 172.21.15.83:0/3501155606 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ffc8039b40 msgr2=0x55ffc8045920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:29.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.404+0000 7f953f7ed740 1 --2- 172.21.15.83:0/3501155606 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ffc8039b40 0x55ffc8045920 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f9524005ec0 tx=0x7f952402a410 comp rx=0 tx=0).stop 2024-04-24T08:58:29.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.404+0000 7f953f7ed740 1 -- 172.21.15.83:0/3501155606 shutdown_connections 2024-04-24T08:58:29.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.404+0000 7f953f7ed740 1 --2- 172.21.15.83:0/3501155606 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55ffc80bbe10 0x55ffc80be2d0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.404+0000 7f953f7ed740 1 --2- 172.21.15.83:0/3501155606 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffc8044b30 0x55ffc80463a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.404+0000 7f953f7ed740 1 --2- 172.21.15.83:0/3501155606 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ffc803a460 0x55ffc8045e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.404+0000 7f9517400640 0 client.18108 ms_handle_reset on v2:172.21.15.138:6834/1909219758 2024-04-24T08:58:29.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.404+0000 7f953f7ed740 1 --2- 172.21.15.83:0/3501155606 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ffc8039b40 0x55ffc8045920 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.404+0000 7f953f7ed740 1 -- 172.21.15.83:0/3501155606 >> 172.21.15.83:0/3501155606 conn(0x55ffc8035e60 msgr2=0x55ffc80346c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:58:29.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.404+0000 7f953f7ed740 1 -- 172.21.15.83:0/3501155606 shutdown_connections 2024-04-24T08:58:29.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.404+0000 7f953f7ed740 1 -- 172.21.15.83:0/3501155606 wait complete. 2024-04-24T08:58:29.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.404+0000 7f953f7ed740 20 client.18108 trim_cache size 0 max 16384 2024-04-24T08:58:29.407 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T08:58:29.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.405+0000 7f9537200640 1 -- 172.21.15.83:0/279839912 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f951c0a3d20 msgr2=0x7f951c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:29.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.405+0000 7f9537200640 1 --2- 172.21.15.83:0/279839912 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f951c0a3d20 0x7f951c0a61e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9530005940 tx=0x7f95300046c0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.405+0000 7f9537200640 1 -- 172.21.15.83:0/279839912 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9538102110 msgr2=0x7f95381a05d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:29.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.405+0000 7f9537200640 1 --2- 172.21.15.83:0/279839912 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9538102110 0x7f95381a05d0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f9524005ab0 tx=0x7f9524003cd0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.405+0000 7f9537200640 1 -- 172.21.15.83:0/279839912 shutdown_connections 2024-04-24T08:58:29.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.405+0000 7f9537200640 1 --2- 172.21.15.83:0/279839912 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f951c0a3d20 0x7f951c0a61e0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.405+0000 7f9537200640 1 --2- 172.21.15.83:0/279839912 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f953819a690 0x7f953819ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.405+0000 7f9537200640 1 --2- 172.21.15.83:0/279839912 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f95381071e0 0x7f95381a0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.405+0000 7f9537200640 1 --2- 172.21.15.83:0/279839912 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9538102110 0x7f95381a05d0 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.405+0000 7f9537200640 1 -- 172.21.15.83:0/279839912 >> 172.21.15.83:0/279839912 conn(0x7f95380fd9d0 msgr2=0x7f95380ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:58:29.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.405+0000 7f9537200640 1 -- 172.21.15.83:0/279839912 shutdown_connections 2024-04-24T08:58:29.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.405+0000 7f9537200640 1 -- 172.21.15.83:0/279839912 wait complete. 2024-04-24T08:58:29.423 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T08:58:29.587 INFO:tasks.workunit.client.0.smithi083.stdout: 1 36299 0.63 MB/sec execute 183 sec latency 51.517 ms 2024-04-24T08:58:29.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.634+0000 7fa8d3200640 1 -- 172.21.15.83:0/1355321706 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8d4076e50 msgr2=0x7fa8d4112810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:29.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.634+0000 7fa8d3200640 1 --2- 172.21.15.83:0/1355321706 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8d4076e50 0x7fa8d4112810 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7fa8c0009900 tx=0x7fa8c002a3f0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.634+0000 7fa8d3200640 1 -- 172.21.15.83:0/1355321706 shutdown_connections 2024-04-24T08:58:29.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.634+0000 7fa8d3200640 1 --2- 172.21.15.83:0/1355321706 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa8d4112d50 0x7fa8d4115140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.634+0000 7fa8d3200640 1 --2- 172.21.15.83:0/1355321706 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8d4076e50 0x7fa8d4112810 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.634+0000 7fa8d3200640 1 --2- 172.21.15.83:0/1355321706 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa8d4076530 0x7fa8d4076910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.634+0000 7fa8d3200640 1 -- 172.21.15.83:0/1355321706 >> 172.21.15.83:0/1355321706 conn(0x7fa8d40ffb40 msgr2=0x7fa8d4101f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:58:29.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.634+0000 7fa8d3200640 1 -- 172.21.15.83:0/1355321706 shutdown_connections 2024-04-24T08:58:29.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.634+0000 7fa8d3200640 1 -- 172.21.15.83:0/1355321706 wait complete. 2024-04-24T08:58:29.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.635+0000 7fa8d3200640 1 Processor -- start 2024-04-24T08:58:29.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.635+0000 7fa8d3200640 1 -- start start 2024-04-24T08:58:29.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.636+0000 7fa8d3200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa8d4076530 0x7fa8d41a4a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:29.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.636+0000 7fa8d3200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8d4076e50 0x7fa8d41a4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:29.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.636+0000 7fa8d3200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa8d4112d50 0x7fa8d419eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:29.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.636+0000 7fa8d3200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d41187c0 con 0x7fa8d4076e50 2024-04-24T08:58:29.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.636+0000 7fa8d3200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d4118640 con 0x7fa8d4076530 2024-04-24T08:58:29.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.636+0000 7fa8d3200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d4118940 con 0x7fa8d4112d50 2024-04-24T08:58:29.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.636+0000 7fa8d1400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8d4076e50 0x7fa8d41a4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:29.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.636+0000 7fa8d1400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8d4076e50 0x7fa8d41a4fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47546/0 (socket says 172.21.15.83:47546) 2024-04-24T08:58:29.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.636+0000 7fa8d1400640 1 -- 172.21.15.83:0/1731774128 learned_addr learned my addr 172.21.15.83:0/1731774128 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:58:29.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.636+0000 7fa8d1e00640 1 --2- 172.21.15.83:0/1731774128 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa8d4076530 0x7fa8d41a4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:29.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.636+0000 7fa8d2800640 1 --2- 172.21.15.83:0/1731774128 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa8d4112d50 0x7fa8d419eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:29.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.637+0000 7fa8d1400640 1 -- 172.21.15.83:0/1731774128 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa8d4112d50 msgr2=0x7fa8d419eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:29.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.637+0000 7fa8d1400640 1 --2- 172.21.15.83:0/1731774128 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa8d4112d50 0x7fa8d419eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.637+0000 7fa8d1400640 1 -- 172.21.15.83:0/1731774128 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa8d4076530 msgr2=0x7fa8d41a4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:29.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.637+0000 7fa8d1400640 1 --2- 172.21.15.83:0/1731774128 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa8d4076530 0x7fa8d41a4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.637+0000 7fa8d1400640 1 -- 172.21.15.83:0/1731774128 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8d419f3e0 con 0x7fa8d4076e50 2024-04-24T08:58:29.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.637+0000 7fa8d1e00640 1 --2- 172.21.15.83:0/1731774128 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa8d4076530 0x7fa8d41a4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:58:29.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.637+0000 7fa8d1400640 1 --2- 172.21.15.83:0/1731774128 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8d4076e50 0x7fa8d41a4fa0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7fa8c0004770 tx=0x7fa8c0003ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:58:29.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.638+0000 7fa8caa00640 1 -- 172.21.15.83:0/1731774128 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c003f070 con 0x7fa8d4076e50 2024-04-24T08:58:29.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.638+0000 7fa8caa00640 1 -- 172.21.15.83:0/1731774128 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa8c003bd20 con 0x7fa8d4076e50 2024-04-24T08:58:29.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.638+0000 7fa8caa00640 1 -- 172.21.15.83:0/1731774128 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c0035030 con 0x7fa8d4076e50 2024-04-24T08:58:29.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.638+0000 7fa8d3200640 1 -- 172.21.15.83:0/1731774128 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8d419f670 con 0x7fa8d4076e50 2024-04-24T08:58:29.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.638+0000 7fa8d3200640 1 -- 172.21.15.83:0/1731774128 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8d41ab9e0 con 0x7fa8d4076e50 2024-04-24T08:58:29.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.641+0000 7fa8caa00640 1 -- 172.21.15.83:0/1731774128 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa8c0042070 con 0x7fa8d4076e50 2024-04-24T08:58:29.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.642+0000 7fa8db722740 1 Processor -- start 2024-04-24T08:58:29.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.642+0000 7fa8db722740 1 -- start start 2024-04-24T08:58:29.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.642+0000 7fa8db722740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ce144fb40 0x555ce144ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:29.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.642+0000 7fa8db722740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555ce1450460 0x555ce145a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:29.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.642+0000 7fa8db722740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ce145ab30 0x555ce145cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:29.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.642+0000 7fa8db722740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x555ce13b51d0 con 0x555ce144fb40 2024-04-24T08:58:29.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.642+0000 7fa8db722740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555ce135b450 con 0x555ce145ab30 2024-04-24T08:58:29.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.642+0000 7fa8db722740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x555ce13b1190 con 0x555ce1450460 2024-04-24T08:58:29.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.643+0000 7fa8d2800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ce144fb40 0x555ce144ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:29.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.643+0000 7fa8d2800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ce144fb40 0x555ce144ff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47550/0 (socket says 172.21.15.83:47550) 2024-04-24T08:58:29.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.643+0000 7fa8d2800640 1 -- 172.21.15.83:0/495408145 learned_addr learned my addr 172.21.15.83:0/495408145 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:58:29.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.643+0000 7fa8d1e00640 1 --2- 172.21.15.83:0/495408145 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555ce1450460 0x555ce145a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:29.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.643+0000 7fa8d1400640 1 --2- 172.21.15.83:0/495408145 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ce145ab30 0x555ce145cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:29.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.643+0000 7fa8d2800640 1 -- 172.21.15.83:0/495408145 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555ce1450460 msgr2=0x555ce145a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:58:29.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.643+0000 7fa8d2800640 1 --2- 172.21.15.83:0/495408145 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555ce1450460 0x555ce145a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.643+0000 7fa8d2800640 1 -- 172.21.15.83:0/495408145 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ce145ab30 msgr2=0x555ce145cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:58:29.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.643+0000 7fa8d2800640 1 --2- 172.21.15.83:0/495408145 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ce145ab30 0x555ce145cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.643+0000 7fa8d2800640 1 -- 172.21.15.83:0/495408145 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ce1418310 con 0x555ce144fb40 2024-04-24T08:58:29.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.644+0000 7fa8caa00640 1 --2- 172.21.15.83:0/1731774128 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa8b80a3da0 0x7fa8b80a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:29.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.644+0000 7fa8caa00640 1 -- 172.21.15.83:0/1731774128 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9567+0+0 (secure 0 0 0) 0x7fa8c00f1460 con 0x7fa8d4076e50 2024-04-24T08:58:29.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.644+0000 7fa8d2800640 1 --2- 172.21.15.83:0/495408145 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ce144fb40 0x555ce144ff20 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc009e30 tx=0x7fa8cc010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1ff44f49416d68a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:58:29.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.644+0000 7fa8d1e00640 1 --2- 172.21.15.83:0/1731774128 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa8b80a3da0 0x7fa8b80a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:29.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.645+0000 7fa8bf000640 1 -- 172.21.15.83:0/495408145 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8cc016070 con 0x555ce144fb40 2024-04-24T08:58:29.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.645+0000 7fa8bf000640 1 -- 172.21.15.83:0/495408145 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa8cc004d20 con 0x555ce144fb40 2024-04-24T08:58:29.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.645+0000 7fa8bf000640 1 -- 172.21.15.83:0/495408145 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8cc012670 con 0x555ce144fb40 2024-04-24T08:58:29.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.646+0000 7fa8db722740 1 -- 172.21.15.83:0/495408145 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ce144fb40 msgr2=0x555ce144ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:29.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.646+0000 7fa8d1e00640 1 --2- 172.21.15.83:0/1731774128 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa8b80a3da0 0x7fa8b80a6260 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x555ce1451100 tx=0x7fa8c400b5a0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:58:29.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.646+0000 7fa8db722740 1 --2- 172.21.15.83:0/495408145 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ce144fb40 0x555ce144ff20 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc009e30 tx=0x7fa8cc010740 comp rx=0 tx=0).stop 2024-04-24T08:58:29.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.646+0000 7fa8db722740 1 -- 172.21.15.83:0/495408145 shutdown_connections 2024-04-24T08:58:29.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.646+0000 7fa8db722740 1 --2- 172.21.15.83:0/495408145 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ce145ab30 0x555ce145cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.646+0000 7fa8db722740 1 --2- 172.21.15.83:0/495408145 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555ce1450460 0x555ce145a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.646+0000 7fa8db722740 1 --2- 172.21.15.83:0/495408145 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ce144fb40 0x555ce144ff20 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.646+0000 7fa8db722740 1 -- 172.21.15.83:0/495408145 >> 172.21.15.83:0/495408145 conn(0x555ce144be60 msgr2=0x555ce144c240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:58:29.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.646+0000 7fa8db722740 1 -- 172.21.15.83:0/495408145 shutdown_connections 2024-04-24T08:58:29.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.646+0000 7fa8db722740 1 -- 172.21.15.83:0/495408145 wait complete. 2024-04-24T08:58:29.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.646+0000 7fa8db722740 1 -- start start 2024-04-24T08:58:29.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.647+0000 7fa8db722740 1 Processor -- start 2024-04-24T08:58:29.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.647+0000 7fa8db722740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ce144fb40 0x555ce1457930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:29.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.647+0000 7fa8db722740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ce1450460 0x555ce1457e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:29.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.647+0000 7fa8db722740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555ce145ab30 0x555ce14583b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:29.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.647+0000 7fa8db722740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x555ce145d9c0 con 0x555ce1450460 2024-04-24T08:58:29.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.647+0000 7fa8db722740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555ce136e810 con 0x555ce144fb40 2024-04-24T08:58:29.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.647+0000 7fa8db722740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x555ce145db40 con 0x555ce145ab30 2024-04-24T08:58:29.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.647+0000 7fa8db722740 10 client.0 _ll_register_callbacks cb 0x555ce133c140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:58:29.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.647+0000 7fa8d1400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ce1450460 0x555ce1457e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:29.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.647+0000 7fa8d1400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ce1450460 0x555ce1457e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47556/0 (socket says 172.21.15.83:47556) 2024-04-24T08:58:29.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.647+0000 7fa8d1400640 1 -- 172.21.15.83:0/3193601984 learned_addr learned my addr 172.21.15.83:0/3193601984 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:58:29.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.648+0000 7fa8d1e00640 1 --2- 172.21.15.83:0/3193601984 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ce144fb40 0x555ce1457930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:29.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.648+0000 7fa8d2800640 1 --2- 172.21.15.83:0/3193601984 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555ce145ab30 0x555ce14583b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:29.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.648+0000 7fa8d1400640 1 -- 172.21.15.83:0/3193601984 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555ce145ab30 msgr2=0x555ce14583b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:58:29.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.648+0000 7fa8d1400640 1 --2- 172.21.15.83:0/3193601984 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555ce145ab30 0x555ce14583b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.648+0000 7fa8d1400640 1 -- 172.21.15.83:0/3193601984 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ce144fb40 msgr2=0x555ce1457930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:58:29.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.648+0000 7fa8d1400640 1 --2- 172.21.15.83:0/3193601984 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ce144fb40 0x555ce1457930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.648+0000 7fa8d1400640 1 -- 172.21.15.83:0/3193601984 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ce137f3f0 con 0x555ce1450460 2024-04-24T08:58:29.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.648+0000 7fa8d1400640 1 --2- 172.21.15.83:0/3193601984 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ce1450460 0x555ce1457e70 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7fa8c00ee7a0 tx=0x7fa8c00bce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d17f80a64db3cb03 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:58:29.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.648+0000 7fa8b3400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:58:29.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.649+0000 7fa8b3400640 1 -- 172.21.15.83:0/3193601984 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c0049050 con 0x555ce1450460 2024-04-24T08:58:29.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.649+0000 7fa8b3400640 1 -- 172.21.15.83:0/3193601984 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9567+0+0 (secure 0 0 0) 0x7fa8c00b9600 con 0x555ce1450460 2024-04-24T08:58:29.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.649+0000 7fa8db722740 1 -- 172.21.15.83:0/3193601984 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8cc004d20 con 0x555ce1450460 2024-04-24T08:58:29.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.649+0000 7fa8b3400640 1 -- 172.21.15.83:0/3193601984 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fa8c00bcf90 con 0x555ce1450460 2024-04-24T08:58:29.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.649+0000 7fa8db722740 10 client.18120 fetch_fsmap learned FSMap version 33 2024-04-24T08:58:29.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.649+0000 7fa8db722740 1 -- 172.21.15.83:0/3193601984 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x555ce10c73f0 con 0x555ce1450460 2024-04-24T08:58:29.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.649+0000 7fa8b3400640 1 -- 172.21.15.83:0/3193601984 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa8cc004d20 con 0x555ce1450460 2024-04-24T08:58:29.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.650+0000 7fa8b3400640 1 -- 172.21.15.83:0/3193601984 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fa8c00f4c00 con 0x555ce1450460 2024-04-24T08:58:29.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.650+0000 7fa8db722740 10 client.18120 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T08:58:29.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.650+0000 7fa8db722740 10 client.18120 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-24T08:58:29.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.650+0000 7fa8db722740 20 client.18120 populate_metadata read hostname 'smithi083' 2024-04-24T08:58:29.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.650+0000 7fa8db722740 1 --2- 172.21.15.83:0/3193601984 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x555ce146ac80 0x555ce146d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:29.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.650+0000 7fa8db722740 4 client.18120 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:58:29.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.650+0000 7fa8db722740 1 -- 172.21.15.83:0/3193601984 --> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555ce13c2bf0 con 0x555ce146ac80 2024-04-24T08:58:29.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.650+0000 7fa8d1e00640 1 --2- 172.21.15.83:0/3193601984 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x555ce146ac80 0x555ce146d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:29.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.651+0000 7fa8d1e00640 1 --2- 172.21.15.83:0/3193601984 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x555ce146ac80 0x555ce146d090 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f4536cb150c00c90 server_cookie=67f3be82c6f38c18 in_seq=0 out_seq=0 2024-04-24T08:58:29.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.651+0000 7fa8b3400640 10 client.18120 ms_handle_connect on v2:172.21.15.138:6836/900566754 2024-04-24T08:58:29.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.654+0000 7fa8b3400640 1 -- 172.21.15.83:0/3193601984 <== mds.1 v2:172.21.15.138:6836/900566754 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555ce13c2bf0 con 0x555ce146ac80 2024-04-24T08:58:29.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.654+0000 7fa8b3400640 10 client.18120 handle_command_reply: tid=0 2024-04-24T08:58:29.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.654+0000 7fa8db722740 1 client.18120 shutdown 2024-04-24T08:58:29.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.654+0000 7fa8db722740 1 -- 172.21.15.83:0/3193601984 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ce1450460 msgr2=0x555ce1457e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:29.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.654+0000 7fa8db722740 1 --2- 172.21.15.83:0/3193601984 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ce1450460 0x555ce1457e70 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7fa8c00ee7a0 tx=0x7fa8c00bce30 comp rx=0 tx=0).stop 2024-04-24T08:58:29.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.655+0000 7fa8db722740 1 -- 172.21.15.83:0/3193601984 shutdown_connections 2024-04-24T08:58:29.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.655+0000 7fa8db722740 1 --2- 172.21.15.83:0/3193601984 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x555ce146ac80 0x555ce146d090 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.655+0000 7fa8db722740 1 --2- 172.21.15.83:0/3193601984 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555ce145ab30 0x555ce14583b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.655+0000 7fa8db722740 1 --2- 172.21.15.83:0/3193601984 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ce1450460 0x555ce1457e70 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.655+0000 7fa8db722740 1 --2- 172.21.15.83:0/3193601984 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ce144fb40 0x555ce1457930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.655+0000 7fa8db722740 1 -- 172.21.15.83:0/3193601984 >> 172.21.15.83:0/3193601984 conn(0x555ce144be60 msgr2=0x555ce1451b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:58:29.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.655+0000 7fa8b3400640 0 client.18120 ms_handle_reset on v2:172.21.15.138:6836/900566754 2024-04-24T08:58:29.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.655+0000 7fa8db722740 1 -- 172.21.15.83:0/3193601984 shutdown_connections 2024-04-24T08:58:29.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.655+0000 7fa8db722740 1 -- 172.21.15.83:0/3193601984 wait complete. 2024-04-24T08:58:29.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.655+0000 7fa8db722740 20 client.18120 trim_cache size 0 max 16384 2024-04-24T08:58:29.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.673+0000 7fa8db722740 1 Processor -- start 2024-04-24T08:58:29.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.673+0000 7fa8db722740 1 -- start start 2024-04-24T08:58:29.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.673+0000 7fa8db722740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ce144fb40 0x555ce14668b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:29.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.673+0000 7fa8db722740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ce1450460 0x555ce14522a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:29.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.673+0000 7fa8db722740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555ce145ab30 0x555ce14729a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:29.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.673+0000 7fa8db722740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d41b5990 con 0x555ce144fb40 2024-04-24T08:58:29.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.673+0000 7fa8db722740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d41b74d0 con 0x555ce1450460 2024-04-24T08:58:29.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.673+0000 7fa8db722740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d41b6060 con 0x555ce145ab30 2024-04-24T08:58:29.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.674+0000 7fa8d1e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ce144fb40 0x555ce14668b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:29.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.674+0000 7fa8d1e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ce144fb40 0x555ce14668b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47568/0 (socket says 172.21.15.83:47568) 2024-04-24T08:58:29.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.674+0000 7fa8d1e00640 1 -- 172.21.15.83:0/33830357 learned_addr learned my addr 172.21.15.83:0/33830357 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:58:29.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.674+0000 7fa8d2800640 1 --2- 172.21.15.83:0/33830357 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555ce145ab30 0x555ce14729a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:29.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.674+0000 7fa8d1400640 1 --2- 172.21.15.83:0/33830357 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ce1450460 0x555ce14522a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:29.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.674+0000 7fa8d1e00640 1 -- 172.21.15.83:0/33830357 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555ce145ab30 msgr2=0x555ce14729a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:58:29.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.674+0000 7fa8d1e00640 1 --2- 172.21.15.83:0/33830357 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555ce145ab30 0x555ce14729a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.674+0000 7fa8d1e00640 1 -- 172.21.15.83:0/33830357 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ce1450460 msgr2=0x555ce14522a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:58:29.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.674+0000 7fa8d1e00640 1 --2- 172.21.15.83:0/33830357 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ce1450460 0x555ce14522a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.674+0000 7fa8d1e00640 1 -- 172.21.15.83:0/33830357 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ce11917d0 con 0x555ce144fb40 2024-04-24T08:58:29.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.675+0000 7fa8d1e00640 1 --2- 172.21.15.83:0/33830357 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ce144fb40 0x555ce14668b0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7fa8c4012fd0 tx=0x7fa8c4015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b18a1a3ffd5d19cd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:58:29.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.675+0000 7fa8be600640 1 -- 172.21.15.83:0/33830357 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c4010be0 con 0x555ce144fb40 2024-04-24T08:58:29.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.675+0000 7fa8be600640 1 -- 172.21.15.83:0/33830357 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa8c4010d80 con 0x555ce144fb40 2024-04-24T08:58:29.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.675+0000 7fa8be600640 1 -- 172.21.15.83:0/33830357 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c4028930 con 0x555ce144fb40 2024-04-24T08:58:29.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.676+0000 7fa8db722740 1 -- 172.21.15.83:0/33830357 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ce144fb40 msgr2=0x555ce14668b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:29.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.676+0000 7fa8db722740 1 --2- 172.21.15.83:0/33830357 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ce144fb40 0x555ce14668b0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7fa8c4012fd0 tx=0x7fa8c4015000 comp rx=0 tx=0).stop 2024-04-24T08:58:29.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.676+0000 7fa8db722740 1 -- 172.21.15.83:0/33830357 shutdown_connections 2024-04-24T08:58:29.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.676+0000 7fa8db722740 1 --2- 172.21.15.83:0/33830357 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555ce145ab30 0x555ce14729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.676+0000 7fa8db722740 1 --2- 172.21.15.83:0/33830357 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ce1450460 0x555ce14522a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.676+0000 7fa8db722740 1 --2- 172.21.15.83:0/33830357 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ce144fb40 0x555ce14668b0 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.676+0000 7fa8db722740 1 -- 172.21.15.83:0/33830357 >> 172.21.15.83:0/33830357 conn(0x555ce144be60 msgr2=0x555ce1458180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:58:29.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.676+0000 7fa8db722740 1 -- 172.21.15.83:0/33830357 shutdown_connections 2024-04-24T08:58:29.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.676+0000 7fa8db722740 1 -- 172.21.15.83:0/33830357 wait complete. 2024-04-24T08:58:29.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.677+0000 7fa8db722740 1 -- start start 2024-04-24T08:58:29.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.677+0000 7fa8db722740 1 Processor -- start 2024-04-24T08:58:29.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.677+0000 7fa8db722740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ce144fb40 0x555ce1463290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:29.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.677+0000 7fa8db722740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ce1450460 0x555ce1454d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:29.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.678+0000 7fa8db722740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555ce145ab30 0x555ce1455370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:29.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.678+0000 7fa8db722740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d41b6e00 con 0x555ce144fb40 2024-04-24T08:58:29.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.678+0000 7fa8db722740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d41b6730 con 0x555ce1450460 2024-04-24T08:58:29.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.678+0000 7fa8db722740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d41b3970 con 0x555ce145ab30 2024-04-24T08:58:29.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.678+0000 7fa8db722740 10 client.0 _ll_register_callbacks cb 0x7fa8c0005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:58:29.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.678+0000 7fa8d1e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ce144fb40 0x555ce1463290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:29.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.678+0000 7fa8d1e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ce144fb40 0x555ce1463290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47584/0 (socket says 172.21.15.83:47584) 2024-04-24T08:58:29.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.678+0000 7fa8d1e00640 1 -- 172.21.15.83:0/2673412663 learned_addr learned my addr 172.21.15.83:0/2673412663 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:58:29.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.678+0000 7fa8d1400640 1 --2- 172.21.15.83:0/2673412663 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ce1450460 0x555ce1454d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:29.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.678+0000 7fa8d1e00640 1 -- 172.21.15.83:0/2673412663 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555ce145ab30 msgr2=0x555ce1455370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:58:29.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.678+0000 7fa8d2800640 1 --2- 172.21.15.83:0/2673412663 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555ce145ab30 0x555ce1455370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:29.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.678+0000 7fa8d1e00640 1 --2- 172.21.15.83:0/2673412663 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555ce145ab30 0x555ce1455370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.678+0000 7fa8d1e00640 1 -- 172.21.15.83:0/2673412663 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ce1450460 msgr2=0x555ce1454d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:58:29.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.678+0000 7fa8d1e00640 1 --2- 172.21.15.83:0/2673412663 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ce1450460 0x555ce1454d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.678+0000 7fa8d1e00640 1 -- 172.21.15.83:0/2673412663 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ce145b920 con 0x555ce144fb40 2024-04-24T08:58:29.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.678+0000 7fa8d2800640 1 --2- 172.21.15.83:0/2673412663 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555ce145ab30 0x555ce1455370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:58:29.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.679+0000 7fa8d1e00640 1 --2- 172.21.15.83:0/2673412663 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ce144fb40 0x555ce1463290 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7fa8c402b040 tx=0x7fa8c40051a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=114cec0cdd073628 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:58:29.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.679+0000 7fa8b3400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:58:29.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.679+0000 7fa8b3400640 1 -- 172.21.15.83:0/2673412663 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c40055d0 con 0x555ce144fb40 2024-04-24T08:58:29.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.679+0000 7fa8b3400640 1 -- 172.21.15.83:0/2673412663 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9567+0+0 (secure 0 0 0) 0x7fa8c4027040 con 0x555ce144fb40 2024-04-24T08:58:29.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.679+0000 7fa8db722740 1 -- 172.21.15.83:0/2673412663 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8c4010d80 con 0x555ce144fb40 2024-04-24T08:58:29.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.679+0000 7fa8b3400640 1 -- 172.21.15.83:0/2673412663 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fa8c4023020 con 0x555ce144fb40 2024-04-24T08:58:29.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.679+0000 7fa8db722740 10 client.18126 fetch_fsmap learned FSMap version 33 2024-04-24T08:58:29.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.679+0000 7fa8db722740 1 -- 172.21.15.83:0/2673412663 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x555ce14d1460 con 0x555ce144fb40 2024-04-24T08:58:29.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.680+0000 7fa8b3400640 1 -- 172.21.15.83:0/2673412663 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa8c4010d80 con 0x555ce144fb40 2024-04-24T08:58:29.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.680+0000 7fa8b3400640 1 -- 172.21.15.83:0/2673412663 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fa8c4004bb0 con 0x555ce144fb40 2024-04-24T08:58:29.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.680+0000 7fa8db722740 10 client.18126 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T08:58:29.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.680+0000 7fa8db722740 10 client.18126 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-24T08:58:29.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.680+0000 7fa8db722740 20 client.18126 populate_metadata read hostname 'smithi083' 2024-04-24T08:58:29.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.680+0000 7fa8db722740 1 --2- 172.21.15.83:0/2673412663 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x555ce14d1b20 0x555ce14d3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:29.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.680+0000 7fa8db722740 4 client.18126 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T08:58:29.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.680+0000 7fa8db722740 1 -- 172.21.15.83:0/2673412663 --> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555ce13bca60 con 0x555ce14d1b20 2024-04-24T08:58:29.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.681+0000 7fa8d1400640 1 --2- 172.21.15.83:0/2673412663 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x555ce14d1b20 0x555ce14d3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:29.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.682+0000 7fa8d1400640 1 --2- 172.21.15.83:0/2673412663 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x555ce14d1b20 0x555ce14d3f40 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=662a8b184d7b2dfa server_cookie=9ceb01799db3901 in_seq=0 out_seq=0 2024-04-24T08:58:29.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.682+0000 7fa8b3400640 10 client.18126 ms_handle_connect on v2:172.21.15.138:6836/900566754 2024-04-24T08:58:29.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.682+0000 7fa8b3400640 1 -- 172.21.15.83:0/2673412663 <== mds.1 v2:172.21.15.138:6836/900566754 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555ce13bca60 con 0x555ce14d1b20 2024-04-24T08:58:29.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.682+0000 7fa8b3400640 10 client.18126 handle_command_reply: tid=0 2024-04-24T08:58:29.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.682+0000 7fa8db722740 1 client.18126 shutdown 2024-04-24T08:58:29.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.683+0000 7fa8db722740 1 -- 172.21.15.83:0/2673412663 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ce144fb40 msgr2=0x555ce1463290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:29.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.683+0000 7fa8db722740 1 --2- 172.21.15.83:0/2673412663 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ce144fb40 0x555ce1463290 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7fa8c402b040 tx=0x7fa8c40051a0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.683+0000 7fa8db722740 1 -- 172.21.15.83:0/2673412663 shutdown_connections 2024-04-24T08:58:29.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.683+0000 7fa8db722740 1 --2- 172.21.15.83:0/2673412663 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x555ce14d1b20 0x555ce14d3f40 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.683+0000 7fa8db722740 1 --2- 172.21.15.83:0/2673412663 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555ce145ab30 0x555ce1455370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.683+0000 7fa8db722740 1 --2- 172.21.15.83:0/2673412663 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ce1450460 0x555ce1454d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.683+0000 7fa8b3400640 0 client.18126 ms_handle_reset on v2:172.21.15.138:6836/900566754 2024-04-24T08:58:29.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.683+0000 7fa8db722740 1 --2- 172.21.15.83:0/2673412663 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ce144fb40 0x555ce1463290 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.683+0000 7fa8db722740 1 -- 172.21.15.83:0/2673412663 >> 172.21.15.83:0/2673412663 conn(0x555ce144be60 msgr2=0x555ce144a6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:58:29.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.683+0000 7fa8db722740 1 -- 172.21.15.83:0/2673412663 shutdown_connections 2024-04-24T08:58:29.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.683+0000 7fa8db722740 1 -- 172.21.15.83:0/2673412663 wait complete. 2024-04-24T08:58:29.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.683+0000 7fa8db722740 20 client.18126 trim_cache size 0 max 16384 2024-04-24T08:58:29.685 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T08:58:29.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.684+0000 7fa8d3200640 1 -- 172.21.15.83:0/1731774128 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa8b80a3da0 msgr2=0x7fa8b80a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:29.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.684+0000 7fa8d3200640 1 --2- 172.21.15.83:0/1731774128 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa8b80a3da0 0x7fa8b80a6260 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x555ce1451100 tx=0x7fa8c400b5a0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.684+0000 7fa8d3200640 1 -- 172.21.15.83:0/1731774128 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8d4076e50 msgr2=0x7fa8d41a4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:29.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.684+0000 7fa8d3200640 1 --2- 172.21.15.83:0/1731774128 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8d4076e50 0x7fa8d41a4fa0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7fa8c0004770 tx=0x7fa8c0003ca0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.684+0000 7fa8d3200640 1 -- 172.21.15.83:0/1731774128 shutdown_connections 2024-04-24T08:58:29.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.684+0000 7fa8d3200640 1 --2- 172.21.15.83:0/1731774128 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa8b80a3da0 0x7fa8b80a6260 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.684+0000 7fa8d3200640 1 --2- 172.21.15.83:0/1731774128 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa8d4112d50 0x7fa8d419eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.684+0000 7fa8d3200640 1 --2- 172.21.15.83:0/1731774128 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8d4076e50 0x7fa8d41a4fa0 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.684+0000 7fa8d3200640 1 --2- 172.21.15.83:0/1731774128 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa8d4076530 0x7fa8d41a4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.684+0000 7fa8d3200640 1 -- 172.21.15.83:0/1731774128 >> 172.21.15.83:0/1731774128 conn(0x7fa8d40ffb40 msgr2=0x7fa8d4101f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:58:29.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.684+0000 7fa8d3200640 1 -- 172.21.15.83:0/1731774128 shutdown_connections 2024-04-24T08:58:29.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.684+0000 7fa8d3200640 1 -- 172.21.15.83:0/1731774128 wait complete. 2024-04-24T08:58:29.701 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T08:58:29.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.911+0000 7f38a8000640 1 -- 172.21.15.83:0/826269352 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38a0112fe0 msgr2=0x7f38a01153d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:29.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.911+0000 7f38a8000640 1 --2- 172.21.15.83:0/826269352 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38a0112fe0 0x7f38a01153d0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f3898009be0 tx=0x7f389802a750 comp rx=0 tx=0).stop 2024-04-24T08:58:29.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.911+0000 7f38a8000640 1 -- 172.21.15.83:0/826269352 shutdown_connections 2024-04-24T08:58:29.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.912+0000 7f38a8000640 1 --2- 172.21.15.83:0/826269352 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38a0112fe0 0x7f38a01153d0 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.912+0000 7f38a8000640 1 --2- 172.21.15.83:0/826269352 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f38a0074cc0 0x7f38a0112aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.912+0000 7f38a8000640 1 --2- 172.21.15.83:0/826269352 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f38a00743a0 0x7f38a0074780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.912+0000 7f38a8000640 1 -- 172.21.15.83:0/826269352 >> 172.21.15.83:0/826269352 conn(0x7f38a00fda40 msgr2=0x7f38a00ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:58:29.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.912+0000 7f38a8000640 1 -- 172.21.15.83:0/826269352 shutdown_connections 2024-04-24T08:58:29.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.912+0000 7f38a8000640 1 -- 172.21.15.83:0/826269352 wait complete. 2024-04-24T08:58:29.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.913+0000 7f38a8000640 1 Processor -- start 2024-04-24T08:58:29.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.913+0000 7f38a8000640 1 -- start start 2024-04-24T08:58:29.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.914+0000 7f38a8000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38a00743a0 0x7f38a019edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:29.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.914+0000 7f38a8000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f38a0074cc0 0x7f38a019f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:29.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.914+0000 7f38a8000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f38a0112fe0 0x7f38a01a36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:29.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.914+0000 7f38a8000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f38a0118890 con 0x7f38a00743a0 2024-04-24T08:58:29.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.914+0000 7f38a8000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f38a0118710 con 0x7f38a0074cc0 2024-04-24T08:58:29.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.914+0000 7f38a8000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f38a0118a10 con 0x7f38a0112fe0 2024-04-24T08:58:29.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.914+0000 7f38a5600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38a00743a0 0x7f38a019edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:29.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.914+0000 7f38a5600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38a00743a0 0x7f38a019edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47602/0 (socket says 172.21.15.83:47602) 2024-04-24T08:58:29.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.914+0000 7f38a5600640 1 -- 172.21.15.83:0/868033055 learned_addr learned my addr 172.21.15.83:0/868033055 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:58:29.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.915+0000 7f38a6000640 1 --2- 172.21.15.83:0/868033055 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f38a0112fe0 0x7f38a01a36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:29.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.915+0000 7f38a4c00640 1 --2- 172.21.15.83:0/868033055 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f38a0074cc0 0x7f38a019f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:29.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.915+0000 7f38a5600640 1 -- 172.21.15.83:0/868033055 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f38a0112fe0 msgr2=0x7f38a01a36b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:29.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.915+0000 7f38a5600640 1 --2- 172.21.15.83:0/868033055 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f38a0112fe0 0x7f38a01a36b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.915+0000 7f38a5600640 1 -- 172.21.15.83:0/868033055 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f38a0074cc0 msgr2=0x7f38a019f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:29.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.915+0000 7f38a5600640 1 --2- 172.21.15.83:0/868033055 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f38a0074cc0 0x7f38a019f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.915+0000 7f38a5600640 1 -- 172.21.15.83:0/868033055 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38a01a3de0 con 0x7f38a00743a0 2024-04-24T08:58:29.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.915+0000 7f38a6000640 1 --2- 172.21.15.83:0/868033055 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f38a0112fe0 0x7f38a01a36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:58:29.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.915+0000 7f38a5600640 1 --2- 172.21.15.83:0/868033055 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38a00743a0 0x7f38a019edf0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f389400e9c0 tx=0x7f389400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:58:29.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.915+0000 7f389e000640 1 -- 172.21.15.83:0/868033055 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f389400ccc0 con 0x7f38a00743a0 2024-04-24T08:58:29.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.915+0000 7f389e000640 1 -- 172.21.15.83:0/868033055 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f38940054d0 con 0x7f38a00743a0 2024-04-24T08:58:29.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.915+0000 7f389e000640 1 -- 172.21.15.83:0/868033055 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3894010430 con 0x7f38a00743a0 2024-04-24T08:58:29.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.915+0000 7f38a8000640 1 -- 172.21.15.83:0/868033055 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38a01a40d0 con 0x7f38a00743a0 2024-04-24T08:58:29.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.916+0000 7f38a8000640 1 -- 172.21.15.83:0/868033055 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38a01ab960 con 0x7f38a00743a0 2024-04-24T08:58:29.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.923+0000 7f38aafe2740 1 Processor -- start 2024-04-24T08:58:29.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.923+0000 7f38aafe2740 1 -- start start 2024-04-24T08:58:29.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.923+0000 7f38aafe2740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55df55243950 0x55df55243d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:29.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.923+0000 7f38aafe2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df55244270 0x55df5524e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:29.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.924+0000 7f38aafe2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df5524e940 0x55df55250d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:29.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.924+0000 7f38aafe2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55df551a91d0 con 0x55df5524e940 2024-04-24T08:58:29.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.924+0000 7f38aafe2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55df5514f450 con 0x55df55244270 2024-04-24T08:58:29.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.924+0000 7f38aafe2740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55df551a5190 con 0x55df55243950 2024-04-24T08:58:29.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.924+0000 7f38a6000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55df55243950 0x55df55243d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:29.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.924+0000 7f38a6000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55df55243950 0x55df55243d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:59308/0 (socket says 172.21.15.83:59308) 2024-04-24T08:58:29.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.924+0000 7f38a6000640 1 -- 172.21.15.83:0/2330368163 learned_addr learned my addr 172.21.15.83:0/2330368163 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:58:29.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.924+0000 7f38a4c00640 1 --2- 172.21.15.83:0/2330368163 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df5524e940 0x55df55250d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:29.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.925+0000 7f38a4c00640 1 -- 172.21.15.83:0/2330368163 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55df55243950 msgr2=0x55df55243d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:58:29.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.925+0000 7f38a4c00640 1 --2- 172.21.15.83:0/2330368163 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55df55243950 0x55df55243d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.925+0000 7f38a4c00640 1 -- 172.21.15.83:0/2330368163 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df55244270 msgr2=0x55df5524e400 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T08:58:29.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.925+0000 7f38a4c00640 1 --2- 172.21.15.83:0/2330368163 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df55244270 0x55df5524e400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.925+0000 7f38a4c00640 1 -- 172.21.15.83:0/2330368163 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df5520c310 con 0x55df5524e940 2024-04-24T08:58:29.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.926+0000 7f389e000640 1 -- 172.21.15.83:0/868033055 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f38940046c0 con 0x7f38a00743a0 2024-04-24T08:58:29.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.926+0000 7f38a4c00640 1 --2- 172.21.15.83:0/2330368163 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df5524e940 0x55df55250d30 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f388c00bb30 tx=0x7f388c00f360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8bf572e10ef2c1d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:58:29.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.927+0000 7f3892600640 1 -- 172.21.15.83:0/2330368163 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f388c015070 con 0x55df5524e940 2024-04-24T08:58:29.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.927+0000 7f3892600640 1 -- 172.21.15.83:0/2330368163 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f388c00fd90 con 0x55df5524e940 2024-04-24T08:58:29.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.927+0000 7f3892600640 1 -- 172.21.15.83:0/2330368163 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f388c011440 con 0x55df5524e940 2024-04-24T08:58:29.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.928+0000 7f389e000640 1 --2- 172.21.15.83:0/868033055 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f38880a3d20 0x7f38880a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:29.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.928+0000 7f389e000640 1 -- 172.21.15.83:0/868033055 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9567+0+0 (secure 0 0 0) 0x7f38940d0fc0 con 0x7f38a00743a0 2024-04-24T08:58:29.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.928+0000 7f38a6000640 1 --2- 172.21.15.83:0/868033055 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f38880a3d20 0x7f38880a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:29.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.928+0000 7f38aafe2740 1 -- 172.21.15.83:0/2330368163 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df5524e940 msgr2=0x55df55250d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:29.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.928+0000 7f38aafe2740 1 --2- 172.21.15.83:0/2330368163 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df5524e940 0x55df55250d30 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f388c00bb30 tx=0x7f388c00f360 comp rx=0 tx=0).stop 2024-04-24T08:58:29.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.929+0000 7f38aafe2740 1 -- 172.21.15.83:0/2330368163 shutdown_connections 2024-04-24T08:58:29.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.929+0000 7f38aafe2740 1 --2- 172.21.15.83:0/2330368163 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df5524e940 0x55df55250d30 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.929+0000 7f38aafe2740 1 --2- 172.21.15.83:0/2330368163 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df55244270 0x55df5524e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.929+0000 7f38aafe2740 1 --2- 172.21.15.83:0/2330368163 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55df55243950 0x55df55243d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.929+0000 7f38aafe2740 1 -- 172.21.15.83:0/2330368163 >> 172.21.15.83:0/2330368163 conn(0x55df5523fc70 msgr2=0x55df55240050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:58:29.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.929+0000 7f38aafe2740 1 -- 172.21.15.83:0/2330368163 shutdown_connections 2024-04-24T08:58:29.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.929+0000 7f38aafe2740 1 -- 172.21.15.83:0/2330368163 wait complete. 2024-04-24T08:58:29.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.929+0000 7f38a6000640 1 --2- 172.21.15.83:0/868033055 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f38880a3d20 0x7f38880a61e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f389802ac70 tx=0x7f3898009840 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:58:29.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.929+0000 7f38aafe2740 1 -- start start 2024-04-24T08:58:29.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.930+0000 7f38aafe2740 1 Processor -- start 2024-04-24T08:58:29.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.930+0000 7f38aafe2740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55df55243950 0x55df5524e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:29.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.930+0000 7f38aafe2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df55244270 0x55df5524b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:29.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.930+0000 7f38aafe2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df5524e940 0x55df5524bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:29.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.930+0000 7f38aafe2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55df552517d0 con 0x55df5524e940 2024-04-24T08:58:29.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.930+0000 7f38aafe2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55df55162810 con 0x55df55244270 2024-04-24T08:58:29.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.930+0000 7f38aafe2740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55df55251950 con 0x55df55243950 2024-04-24T08:58:29.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.930+0000 7f38aafe2740 10 client.0 _ll_register_callbacks cb 0x55df55130140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:58:29.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.930+0000 7f38a6000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df5524e940 0x55df5524bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:29.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.931+0000 7f38a6000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df5524e940 0x55df5524bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47612/0 (socket says 172.21.15.83:47612) 2024-04-24T08:58:29.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.931+0000 7f38a6000640 1 -- 172.21.15.83:0/2416889920 learned_addr learned my addr 172.21.15.83:0/2416889920 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:58:29.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.931+0000 7f38a5600640 1 --2- 172.21.15.83:0/2416889920 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55df55243950 0x55df5524e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:29.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.931+0000 7f38a4c00640 1 --2- 172.21.15.83:0/2416889920 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df55244270 0x55df5524b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:29.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.931+0000 7f38a6000640 1 -- 172.21.15.83:0/2416889920 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55df55243950 msgr2=0x55df5524e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:58:29.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.931+0000 7f38a6000640 1 --2- 172.21.15.83:0/2416889920 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55df55243950 0x55df5524e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.931+0000 7f38a6000640 1 -- 172.21.15.83:0/2416889920 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df55244270 msgr2=0x55df5524b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:58:29.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.931+0000 7f38a6000640 1 --2- 172.21.15.83:0/2416889920 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df55244270 0x55df5524b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.931+0000 7f38a6000640 1 -- 172.21.15.83:0/2416889920 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df551733f0 con 0x55df5524e940 2024-04-24T08:58:29.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.931+0000 7f38a4c00640 1 --2- 172.21.15.83:0/2416889920 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df55244270 0x55df5524b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:58:29.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.931+0000 7f38a6000640 1 --2- 172.21.15.83:0/2416889920 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df5524e940 0x55df5524bdb0 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f3898009d00 tx=0x7f3898009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0270cf986f0c236 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:58:29.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.931+0000 7f3886a00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:58:29.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.932+0000 7f3886a00640 1 -- 172.21.15.83:0/2416889920 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f389804f060 con 0x55df5524e940 2024-04-24T08:58:29.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.932+0000 7f3886a00640 1 -- 172.21.15.83:0/2416889920 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9567+0+0 (secure 0 0 0) 0x7f3898004240 con 0x55df5524e940 2024-04-24T08:58:29.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.932+0000 7f38aafe2740 1 -- 172.21.15.83:0/2416889920 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f388c00fd90 con 0x55df5524e940 2024-04-24T08:58:29.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.932+0000 7f3886a00640 1 -- 172.21.15.83:0/2416889920 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f3898056050 con 0x55df5524e940 2024-04-24T08:58:29.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.932+0000 7f38aafe2740 10 client.18138 fetch_fsmap learned FSMap version 33 2024-04-24T08:58:29.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.932+0000 7f38aafe2740 1 -- 172.21.15.83:0/2416889920 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55df54ebb3f0 con 0x55df5524e940 2024-04-24T08:58:29.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.932+0000 7f3886a00640 1 -- 172.21.15.83:0/2416889920 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f388c00fd90 con 0x55df5524e940 2024-04-24T08:58:29.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.933+0000 7f3886a00640 1 -- 172.21.15.83:0/2416889920 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f389803c040 con 0x55df5524e940 2024-04-24T08:58:29.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.933+0000 7f38aafe2740 10 client.18138 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T08:58:29.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.933+0000 7f38aafe2740 10 client.18138 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:58:29.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.933+0000 7f38aafe2740 20 client.18138 populate_metadata read hostname 'smithi083' 2024-04-24T08:58:29.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.933+0000 7f38aafe2740 1 --2- 172.21.15.83:0/2416889920 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55df5525eb80 0x55df55260fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:29.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.933+0000 7f38aafe2740 4 client.18138 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:58:29.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.933+0000 7f38aafe2740 1 -- 172.21.15.83:0/2416889920 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55df551b6bf0 con 0x55df5525eb80 2024-04-24T08:58:29.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.933+0000 7f38a5600640 1 --2- 172.21.15.83:0/2416889920 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55df5525eb80 0x55df55260fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:29.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.934+0000 7f38a5600640 1 --2- 172.21.15.83:0/2416889920 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55df5525eb80 0x55df55260fb0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5a0ec0c3951168d server_cookie=aea982d5baf5d290 in_seq=0 out_seq=0 2024-04-24T08:58:29.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.934+0000 7f3886a00640 10 client.18138 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:58:29.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.938+0000 7f3886a00640 1 -- 172.21.15.83:0/2416889920 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55df551b6bf0 con 0x55df5525eb80 2024-04-24T08:58:29.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.938+0000 7f3886a00640 10 client.18138 handle_command_reply: tid=0 2024-04-24T08:58:29.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.938+0000 7f38aafe2740 1 client.18138 shutdown 2024-04-24T08:58:29.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.938+0000 7f38aafe2740 1 -- 172.21.15.83:0/2416889920 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df5524e940 msgr2=0x55df5524bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:29.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.938+0000 7f38aafe2740 1 --2- 172.21.15.83:0/2416889920 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df5524e940 0x55df5524bdb0 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f3898009d00 tx=0x7f3898009500 comp rx=0 tx=0).stop 2024-04-24T08:58:29.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.938+0000 7f38aafe2740 1 -- 172.21.15.83:0/2416889920 shutdown_connections 2024-04-24T08:58:29.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.938+0000 7f38aafe2740 1 --2- 172.21.15.83:0/2416889920 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55df5525eb80 0x55df55260fb0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.938+0000 7f38aafe2740 1 --2- 172.21.15.83:0/2416889920 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df5524e940 0x55df5524bdb0 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.938+0000 7f38aafe2740 1 --2- 172.21.15.83:0/2416889920 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df55244270 0x55df5524b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.938+0000 7f38aafe2740 1 --2- 172.21.15.83:0/2416889920 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55df55243950 0x55df5524e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.938+0000 7f3886a00640 0 client.18138 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:58:29.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.938+0000 7f38aafe2740 1 -- 172.21.15.83:0/2416889920 >> 172.21.15.83:0/2416889920 conn(0x55df5523fc70 msgr2=0x55df55245a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:58:29.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.938+0000 7f38aafe2740 1 -- 172.21.15.83:0/2416889920 shutdown_connections 2024-04-24T08:58:29.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.938+0000 7f38aafe2740 1 -- 172.21.15.83:0/2416889920 wait complete. 2024-04-24T08:58:29.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.939+0000 7f38aafe2740 20 client.18138 trim_cache size 0 max 16384 2024-04-24T08:58:29.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.957+0000 7f38aafe2740 1 Processor -- start 2024-04-24T08:58:29.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.957+0000 7f38aafe2740 1 -- start start 2024-04-24T08:58:29.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.957+0000 7f38aafe2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df55243950 0x55df552460f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:29.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.957+0000 7f38aafe2740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55df55244270 0x55df55240050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:29.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.957+0000 7f38aafe2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df5524e940 0x55df5524df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:29.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.957+0000 7f38aafe2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f38a01b5d40 con 0x55df5524e940 2024-04-24T08:58:29.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.957+0000 7f38aafe2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f38a01b7880 con 0x55df55243950 2024-04-24T08:58:29.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.957+0000 7f38aafe2740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f38a01b6410 con 0x55df55244270 2024-04-24T08:58:29.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.958+0000 7f38a6000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df5524e940 0x55df5524df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:29.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.958+0000 7f38a6000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df5524e940 0x55df5524df00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47614/0 (socket says 172.21.15.83:47614) 2024-04-24T08:58:29.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.958+0000 7f38a6000640 1 -- 172.21.15.83:0/3900281530 learned_addr learned my addr 172.21.15.83:0/3900281530 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:58:29.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.958+0000 7f38a6000640 1 -- 172.21.15.83:0/3900281530 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55df55244270 msgr2=0x55df55240050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:58:29.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.958+0000 7f38a6000640 1 --2- 172.21.15.83:0/3900281530 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55df55244270 0x55df55240050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.958+0000 7f38a6000640 1 -- 172.21.15.83:0/3900281530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df55243950 msgr2=0x55df552460f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:58:29.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.958+0000 7f38a6000640 1 --2- 172.21.15.83:0/3900281530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df55243950 0x55df552460f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.958+0000 7f38a6000640 1 -- 172.21.15.83:0/3900281530 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df552440a0 con 0x55df5524e940 2024-04-24T08:58:29.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.959+0000 7f38a6000640 1 --2- 172.21.15.83:0/3900281530 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df5524e940 0x55df5524df00 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f3898031b60 tx=0x7f3898005fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=274e689afc32396c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:58:29.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.959+0000 7f3891c00640 1 -- 172.21.15.83:0/3900281530 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f389804f070 con 0x55df5524e940 2024-04-24T08:58:29.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.959+0000 7f3891c00640 1 -- 172.21.15.83:0/3900281530 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f389805f070 con 0x55df5524e940 2024-04-24T08:58:29.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.959+0000 7f3891c00640 1 -- 172.21.15.83:0/3900281530 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3898045070 con 0x55df5524e940 2024-04-24T08:58:29.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.960+0000 7f38aafe2740 1 -- 172.21.15.83:0/3900281530 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df5524e940 msgr2=0x55df5524df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:29.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.960+0000 7f38aafe2740 1 --2- 172.21.15.83:0/3900281530 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df5524e940 0x55df5524df00 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f3898031b60 tx=0x7f3898005fb0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.961+0000 7f38aafe2740 1 -- 172.21.15.83:0/3900281530 shutdown_connections 2024-04-24T08:58:29.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.961+0000 7f38aafe2740 1 --2- 172.21.15.83:0/3900281530 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df5524e940 0x55df5524df00 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.961+0000 7f38aafe2740 1 --2- 172.21.15.83:0/3900281530 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55df55244270 0x55df55240050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.961+0000 7f38aafe2740 1 --2- 172.21.15.83:0/3900281530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df55243950 0x55df552460f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.961+0000 7f38aafe2740 1 -- 172.21.15.83:0/3900281530 >> 172.21.15.83:0/3900281530 conn(0x55df5523fc70 msgr2=0x55df5524af50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:58:29.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.961+0000 7f38aafe2740 1 -- 172.21.15.83:0/3900281530 shutdown_connections 2024-04-24T08:58:29.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.961+0000 7f38aafe2740 1 -- 172.21.15.83:0/3900281530 wait complete. 2024-04-24T08:58:29.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.962+0000 7f38aafe2740 1 -- start start 2024-04-24T08:58:29.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.962+0000 7f38aafe2740 1 Processor -- start 2024-04-24T08:58:29.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.962+0000 7f38aafe2740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55df55243950 0x55df55248210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:29.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.962+0000 7f38aafe2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df55244270 0x55df55248750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:29.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.962+0000 7f38aafe2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df5524e940 0x55df55248c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:29.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.962+0000 7f38aafe2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f38a01b71b0 con 0x55df55244270 2024-04-24T08:58:29.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.962+0000 7f38aafe2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f38a01b6ae0 con 0x55df5524e940 2024-04-24T08:58:29.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.962+0000 7f38aafe2740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f38a01b3d20 con 0x55df55243950 2024-04-24T08:58:29.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.963+0000 7f38aafe2740 10 client.0 _ll_register_callbacks cb 0x7f3898034070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:58:29.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.963+0000 7f38a4c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df55244270 0x55df55248750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:29.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.963+0000 7f38a4c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df55244270 0x55df55248750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47618/0 (socket says 172.21.15.83:47618) 2024-04-24T08:58:29.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.963+0000 7f38a4c00640 1 -- 172.21.15.83:0/306581996 learned_addr learned my addr 172.21.15.83:0/306581996 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:58:29.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.963+0000 7f38a4c00640 1 -- 172.21.15.83:0/306581996 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55df55243950 msgr2=0x55df55248210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:58:29.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.963+0000 7f38a4c00640 1 --2- 172.21.15.83:0/306581996 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55df55243950 0x55df55248210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.963+0000 7f38a4c00640 1 -- 172.21.15.83:0/306581996 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df5524e940 msgr2=0x55df55248c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:58:29.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.963+0000 7f38a4c00640 1 --2- 172.21.15.83:0/306581996 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df5524e940 0x55df55248c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.963+0000 7f38a4c00640 1 -- 172.21.15.83:0/306581996 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df54f857d0 con 0x55df55244270 2024-04-24T08:58:29.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.964+0000 7f38a4c00640 1 --2- 172.21.15.83:0/306581996 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df55244270 0x55df55248750 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f388c00ba80 tx=0x7f388c00fdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1465045b966e6b85 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:58:29.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.964+0000 7f3886a00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:58:29.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.964+0000 7f3886a00640 1 -- 172.21.15.83:0/306581996 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f388c015040 con 0x55df55244270 2024-04-24T08:58:29.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.964+0000 7f38aafe2740 1 -- 172.21.15.83:0/306581996 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f389805f070 con 0x55df55244270 2024-04-24T08:58:29.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.964+0000 7f3886a00640 1 -- 172.21.15.83:0/306581996 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9567+0+0 (secure 0 0 0) 0x7f388c005230 con 0x55df55244270 2024-04-24T08:58:29.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.964+0000 7f3886a00640 1 -- 172.21.15.83:0/306581996 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f388c005430 con 0x55df55244270 2024-04-24T08:58:29.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.964+0000 7f38aafe2740 10 client.18144 fetch_fsmap learned FSMap version 33 2024-04-24T08:58:29.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.964+0000 7f38aafe2740 1 -- 172.21.15.83:0/306581996 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55df552c5120 con 0x55df55244270 2024-04-24T08:58:29.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.965+0000 7f3886a00640 1 -- 172.21.15.83:0/306581996 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f389805f070 con 0x55df55244270 2024-04-24T08:58:29.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.965+0000 7f3886a00640 1 -- 172.21.15.83:0/306581996 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f388c003db0 con 0x55df55244270 2024-04-24T08:58:29.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.965+0000 7f38aafe2740 10 client.18144 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T08:58:29.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.965+0000 7f38aafe2740 10 client.18144 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:58:29.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.965+0000 7f38aafe2740 20 client.18144 populate_metadata read hostname 'smithi083' 2024-04-24T08:58:29.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.965+0000 7f38aafe2740 1 --2- 172.21.15.83:0/306581996 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55df552c56e0 0x55df552c7b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:29.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.965+0000 7f38aafe2740 4 client.18144 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T08:58:29.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.965+0000 7f38aafe2740 1 -- 172.21.15.83:0/306581996 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55df551b0a60 con 0x55df552c56e0 2024-04-24T08:58:29.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.965+0000 7f38a5600640 1 --2- 172.21.15.83:0/306581996 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55df552c56e0 0x55df552c7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:29.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.967+0000 7f3886a00640 10 client.18144 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:58:29.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.967+0000 7f38a5600640 1 --2- 172.21.15.83:0/306581996 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55df552c56e0 0x55df552c7b00 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4099365513187c6e server_cookie=8f0e50b22d70c7bc in_seq=0 out_seq=0 2024-04-24T08:58:29.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.968+0000 7f3886a00640 1 -- 172.21.15.83:0/306581996 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55df551b0a60 con 0x55df552c56e0 2024-04-24T08:58:29.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.968+0000 7f3886a00640 10 client.18144 handle_command_reply: tid=0 2024-04-24T08:58:29.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.968+0000 7f38aafe2740 1 client.18144 shutdown 2024-04-24T08:58:29.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.969+0000 7f38aafe2740 1 -- 172.21.15.83:0/306581996 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df55244270 msgr2=0x55df55248750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:29.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.969+0000 7f38aafe2740 1 --2- 172.21.15.83:0/306581996 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df55244270 0x55df55248750 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f388c00ba80 tx=0x7f388c00fdc0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.969+0000 7f38aafe2740 1 -- 172.21.15.83:0/306581996 shutdown_connections 2024-04-24T08:58:29.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.969+0000 7f38aafe2740 1 --2- 172.21.15.83:0/306581996 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55df552c56e0 0x55df552c7b00 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.969+0000 7f38aafe2740 1 --2- 172.21.15.83:0/306581996 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df5524e940 0x55df55248c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.969+0000 7f38aafe2740 1 --2- 172.21.15.83:0/306581996 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df55244270 0x55df55248750 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.969+0000 7f38aafe2740 1 --2- 172.21.15.83:0/306581996 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55df55243950 0x55df55248210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.969+0000 7f3886a00640 0 client.18144 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:58:29.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.969+0000 7f38aafe2740 1 -- 172.21.15.83:0/306581996 >> 172.21.15.83:0/306581996 conn(0x55df5523fc70 msgr2=0x55df55240050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:58:29.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.969+0000 7f38aafe2740 1 -- 172.21.15.83:0/306581996 shutdown_connections 2024-04-24T08:58:29.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.969+0000 7f38aafe2740 1 -- 172.21.15.83:0/306581996 wait complete. 2024-04-24T08:58:29.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.970+0000 7f38aafe2740 20 client.18144 trim_cache size 0 max 16384 2024-04-24T08:58:29.971 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T08:58:29.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.972+0000 7f38a8000640 1 -- 172.21.15.83:0/868033055 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f38880a3d20 msgr2=0x7f38880a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:29.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.972+0000 7f38a8000640 1 --2- 172.21.15.83:0/868033055 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f38880a3d20 0x7f38880a61e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f389802ac70 tx=0x7f3898009840 comp rx=0 tx=0).stop 2024-04-24T08:58:29.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.972+0000 7f38a8000640 1 -- 172.21.15.83:0/868033055 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38a00743a0 msgr2=0x7f38a019edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:29.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.972+0000 7f38a8000640 1 --2- 172.21.15.83:0/868033055 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38a00743a0 0x7f38a019edf0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f389400e9c0 tx=0x7f389400ee80 comp rx=0 tx=0).stop 2024-04-24T08:58:29.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.972+0000 7f38a8000640 1 -- 172.21.15.83:0/868033055 shutdown_connections 2024-04-24T08:58:29.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.972+0000 7f38a8000640 1 --2- 172.21.15.83:0/868033055 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f38880a3d20 0x7f38880a61e0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.972+0000 7f38a8000640 1 --2- 172.21.15.83:0/868033055 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f38a0112fe0 0x7f38a01a36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.972+0000 7f38a8000640 1 --2- 172.21.15.83:0/868033055 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f38a0074cc0 0x7f38a019f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.972+0000 7f38a8000640 1 --2- 172.21.15.83:0/868033055 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38a00743a0 0x7f38a019edf0 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:29.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.972+0000 7f38a8000640 1 -- 172.21.15.83:0/868033055 >> 172.21.15.83:0/868033055 conn(0x7f38a00fda40 msgr2=0x7f38a00ff340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:58:29.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.972+0000 7f38a8000640 1 -- 172.21.15.83:0/868033055 shutdown_connections 2024-04-24T08:58:29.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:29.973+0000 7f38a8000640 1 -- 172.21.15.83:0/868033055 wait complete. 2024-04-24T08:58:29.989 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:58:29 smithi083 ceph-mon[27907]: pgmap v212: 129 pgs: 129 active+clean; 47 MiB data, 657 MiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 616 KiB/s wr, 69 op/s 2024-04-24T08:58:30.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:58:29 smithi138 ceph-mon[31441]: pgmap v212: 129 pgs: 129 active+clean; 47 MiB data, 657 MiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 616 KiB/s wr, 69 op/s 2024-04-24T08:58:30.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:58:29 smithi189 ceph-mon[31791]: pgmap v212: 129 pgs: 129 active+clean; 47 MiB data, 657 MiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 616 KiB/s wr, 69 op/s 2024-04-24T08:58:30.587 INFO:tasks.workunit.client.0.smithi083.stdout: 1 36446 0.63 MB/sec execute 184 sec latency 47.428 ms 2024-04-24T08:58:30.992 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T08:58:30.992 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T08:58:30.993 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T08:58:31.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.221+0000 7fcfcd400640 1 -- 172.21.15.83:0/1461012342 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcfc8104320 msgr2=0x7fcfc8104700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:31.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.221+0000 7fcfcd400640 1 --2- 172.21.15.83:0/1461012342 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcfc8104320 0x7fcfc8104700 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7fcfb8009900 tx=0x7fcfb802a400 comp rx=0 tx=0).stop 2024-04-24T08:58:31.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.221+0000 7fcfcd400640 1 -- 172.21.15.83:0/1461012342 shutdown_connections 2024-04-24T08:58:31.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.221+0000 7fcfcd400640 1 --2- 172.21.15.83:0/1461012342 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcfc81093f0 0x7fcfc8110fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:31.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.221+0000 7fcfcd400640 1 --2- 172.21.15.83:0/1461012342 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcfc8104cd0 0x7fcfc8108d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:31.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.221+0000 7fcfcd400640 1 --2- 172.21.15.83:0/1461012342 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcfc8104320 0x7fcfc8104700 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:31.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.221+0000 7fcfcd400640 1 -- 172.21.15.83:0/1461012342 >> 172.21.15.83:0/1461012342 conn(0x7fcfc80ffb40 msgr2=0x7fcfc8101f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:58:31.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.222+0000 7fcfcd400640 1 -- 172.21.15.83:0/1461012342 shutdown_connections 2024-04-24T08:58:31.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.222+0000 7fcfcd400640 1 -- 172.21.15.83:0/1461012342 wait complete. 2024-04-24T08:58:31.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.223+0000 7fcfcd400640 1 Processor -- start 2024-04-24T08:58:31.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.223+0000 7fcfcd400640 1 -- start start 2024-04-24T08:58:31.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.223+0000 7fcfcd400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcfc8104320 0x7fcfc81a05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:31.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.223+0000 7fcfcd400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcfc81093f0 0x7fcfc81a0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:31.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.223+0000 7fcfcd400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcfc819a680 0x7fcfc819ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:31.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.223+0000 7fcfcd400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc8114380 con 0x7fcfc819a680 2024-04-24T08:58:31.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.223+0000 7fcfcd400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc8114200 con 0x7fcfc8104320 2024-04-24T08:58:31.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.223+0000 7fcfcd400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc8114500 con 0x7fcfc81093f0 2024-04-24T08:58:31.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.223+0000 7fcfcca00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcfc819a680 0x7fcfc819ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:31.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.223+0000 7fcfcca00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcfc819a680 0x7fcfc819ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47636/0 (socket says 172.21.15.83:47636) 2024-04-24T08:58:31.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.223+0000 7fcfcca00640 1 -- 172.21.15.83:0/2450309155 learned_addr learned my addr 172.21.15.83:0/2450309155 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:58:31.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.224+0000 7fcfc7e00640 1 --2- 172.21.15.83:0/2450309155 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcfc8104320 0x7fcfc81a05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:31.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.224+0000 7fcfc7400640 1 --2- 172.21.15.83:0/2450309155 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcfc81093f0 0x7fcfc81a0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:31.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.224+0000 7fcfcca00640 1 -- 172.21.15.83:0/2450309155 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcfc81093f0 msgr2=0x7fcfc81a0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:31.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.224+0000 7fcfcca00640 1 --2- 172.21.15.83:0/2450309155 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcfc81093f0 0x7fcfc81a0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:31.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.224+0000 7fcfcca00640 1 -- 172.21.15.83:0/2450309155 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcfc8104320 msgr2=0x7fcfc81a05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:31.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.224+0000 7fcfcca00640 1 --2- 172.21.15.83:0/2450309155 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcfc8104320 0x7fcfc81a05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:31.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.224+0000 7fcfcca00640 1 -- 172.21.15.83:0/2450309155 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfc819b280 con 0x7fcfc819a680 2024-04-24T08:58:31.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.224+0000 7fcfcca00640 1 --2- 172.21.15.83:0/2450309155 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcfc819a680 0x7fcfc819ab00 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7fcfc000cce0 tx=0x7fcfc000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:58:31.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.225+0000 7fcfc4c00640 1 -- 172.21.15.83:0/2450309155 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc000f450 con 0x7fcfc819a680 2024-04-24T08:58:31.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.225+0000 7fcfc4c00640 1 -- 172.21.15.83:0/2450309155 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcfc000faa0 con 0x7fcfc819a680 2024-04-24T08:58:31.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.225+0000 7fcfcd400640 1 -- 172.21.15.83:0/2450309155 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfc81a7360 con 0x7fcfc819a680 2024-04-24T08:58:31.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.225+0000 7fcfc4c00640 1 -- 172.21.15.83:0/2450309155 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc0018b70 con 0x7fcfc819a680 2024-04-24T08:58:31.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.225+0000 7fcfcd400640 1 -- 172.21.15.83:0/2450309155 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfc81a7710 con 0x7fcfc819a680 2024-04-24T08:58:31.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.228+0000 7fcfc4c00640 1 -- 172.21.15.83:0/2450309155 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fcfc0018d10 con 0x7fcfc819a680 2024-04-24T08:58:31.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.228+0000 7fcfd19b4740 1 Processor -- start 2024-04-24T08:58:31.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.228+0000 7fcfd19b4740 1 -- start start 2024-04-24T08:58:31.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.229+0000 7fcfd19b4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c9a9649270 0x55c9a9649650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:31.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.229+0000 7fcfd19b4740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c9a9649b90 0x55c9a9653d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:31.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.229+0000 7fcfd19b4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9a9654260 0x55c9a9656650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:31.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.229+0000 7fcfd19b4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c9a95aee70 con 0x55c9a9654260 2024-04-24T08:58:31.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.229+0000 7fcfd19b4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c9a9555380 con 0x55c9a9649270 2024-04-24T08:58:31.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.229+0000 7fcfd19b4740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55c9a95aae30 con 0x55c9a9649b90 2024-04-24T08:58:31.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.229+0000 7fcfc7400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9a9654260 0x55c9a9656650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:31.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.229+0000 7fcfc7400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9a9654260 0x55c9a9656650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47650/0 (socket says 172.21.15.83:47650) 2024-04-24T08:58:31.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.229+0000 7fcfc7400640 1 -- 172.21.15.83:0/3457689980 learned_addr learned my addr 172.21.15.83:0/3457689980 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:58:31.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.229+0000 7fcfc7400640 1 --2- 172.21.15.83:0/3457689980 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c9a9649270 0x55c9a9649650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:31.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.229+0000 7fcfc7e00640 1 --2- 172.21.15.83:0/3457689980 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c9a9649b90 0x55c9a9653d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:31.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.230+0000 7fcfc7400640 1 -- 172.21.15.83:0/3457689980 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c9a9649b90 msgr2=0x55c9a9653d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:58:31.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.230+0000 7fcfc7400640 1 --2- 172.21.15.83:0/3457689980 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c9a9649b90 0x55c9a9653d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:31.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.230+0000 7fcfc7400640 1 -- 172.21.15.83:0/3457689980 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c9a9649270 msgr2=0x55c9a9649650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:58:31.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.230+0000 7fcfc7400640 1 --2- 172.21.15.83:0/3457689980 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c9a9649270 0x55c9a9649650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:31.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.230+0000 7fcfc7400640 1 -- 172.21.15.83:0/3457689980 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9a93897e0 con 0x55c9a9654260 2024-04-24T08:58:31.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.230+0000 7fcfc4c00640 1 --2- 172.21.15.83:0/2450309155 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fcfac0a3d20 0x7fcfac0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:31.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.230+0000 7fcfc7400640 1 --2- 172.21.15.83:0/3457689980 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9a9654260 0x55c9a9656650 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7fcfb400ef60 tx=0x7fcfb400e430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb67382efd991801 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:58:31.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.230+0000 7fcfc4c00640 1 -- 172.21.15.83:0/2450309155 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9567+0+0 (secure 0 0 0) 0x7fcfc00d1100 con 0x7fcfc819a680 2024-04-24T08:58:31.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.230+0000 7fcfc7e00640 1 --2- 172.21.15.83:0/2450309155 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fcfac0a3d20 0x7fcfac0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:31.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.230+0000 7fcfbd200640 1 -- 172.21.15.83:0/3457689980 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb4018070 con 0x55c9a9654260 2024-04-24T08:58:31.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.230+0000 7fcfbd200640 1 -- 172.21.15.83:0/3457689980 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcfb40047a0 con 0x55c9a9654260 2024-04-24T08:58:31.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.230+0000 7fcfbd200640 1 -- 172.21.15.83:0/3457689980 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb40109f0 con 0x55c9a9654260 2024-04-24T08:58:31.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.231+0000 7fcfd19b4740 1 -- 172.21.15.83:0/3457689980 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9a9654260 msgr2=0x55c9a9656650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:31.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.231+0000 7fcfd19b4740 1 --2- 172.21.15.83:0/3457689980 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9a9654260 0x55c9a9656650 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7fcfb400ef60 tx=0x7fcfb400e430 comp rx=0 tx=0).stop 2024-04-24T08:58:31.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.231+0000 7fcfc7e00640 1 --2- 172.21.15.83:0/2450309155 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fcfac0a3d20 0x7fcfac0a61e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fcfb8000c00 tx=0x7fcfb8031400 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:58:31.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.231+0000 7fcfd19b4740 1 -- 172.21.15.83:0/3457689980 shutdown_connections 2024-04-24T08:58:31.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.231+0000 7fcfd19b4740 1 --2- 172.21.15.83:0/3457689980 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9a9654260 0x55c9a9656650 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:31.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.231+0000 7fcfd19b4740 1 --2- 172.21.15.83:0/3457689980 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c9a9649b90 0x55c9a9653d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:31.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.231+0000 7fcfd19b4740 1 --2- 172.21.15.83:0/3457689980 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c9a9649270 0x55c9a9649650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:31.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.231+0000 7fcfd19b4740 1 -- 172.21.15.83:0/3457689980 >> 172.21.15.83:0/3457689980 conn(0x55c9a9645590 msgr2=0x55c9a9645970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:58:31.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.231+0000 7fcfd19b4740 1 -- 172.21.15.83:0/3457689980 shutdown_connections 2024-04-24T08:58:31.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.231+0000 7fcfd19b4740 1 -- 172.21.15.83:0/3457689980 wait complete. 2024-04-24T08:58:31.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.232+0000 7fcfd19b4740 1 -- start start 2024-04-24T08:58:31.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.232+0000 7fcfd19b4740 1 Processor -- start 2024-04-24T08:58:31.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.233+0000 7fcfd19b4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9a9649270 0x55c9a9651260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:31.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.233+0000 7fcfd19b4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c9a9649b90 0x55c9a96517a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:31.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.233+0000 7fcfd19b4740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c9a9654260 0x55c9a9651ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:31.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.233+0000 7fcfd19b4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c9a9657050 con 0x55c9a9649270 2024-04-24T08:58:31.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.233+0000 7fcfd19b4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c9a9568660 con 0x55c9a9649b90 2024-04-24T08:58:31.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.233+0000 7fcfd19b4740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55c9a96571d0 con 0x55c9a9654260 2024-04-24T08:58:31.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.233+0000 7fcfd19b4740 10 client.0 _ll_register_callbacks cb 0x55c9a9556aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:58:31.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.233+0000 7fcfc7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9a9649270 0x55c9a9651260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:31.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.233+0000 7fcfc7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9a9649270 0x55c9a9651260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47664/0 (socket says 172.21.15.83:47664) 2024-04-24T08:58:31.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.233+0000 7fcfc7e00640 1 -- 172.21.15.83:0/1314043171 learned_addr learned my addr 172.21.15.83:0/1314043171 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:58:31.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.233+0000 7fcfc7400640 1 --2- 172.21.15.83:0/1314043171 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c9a9649b90 0x55c9a96517a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:31.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.233+0000 7fcfcca00640 1 --2- 172.21.15.83:0/1314043171 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c9a9654260 0x55c9a9651ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:31.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.234+0000 7fcfc7e00640 1 -- 172.21.15.83:0/1314043171 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c9a9654260 msgr2=0x55c9a9651ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:58:31.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.234+0000 7fcfc7e00640 1 --2- 172.21.15.83:0/1314043171 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c9a9654260 0x55c9a9651ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:31.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.234+0000 7fcfc7e00640 1 -- 172.21.15.83:0/1314043171 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c9a9649b90 msgr2=0x55c9a96517a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:58:31.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.234+0000 7fcfc7e00640 1 --2- 172.21.15.83:0/1314043171 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c9a9649b90 0x55c9a96517a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:31.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.234+0000 7fcfc7e00640 1 -- 172.21.15.83:0/1314043171 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9a93e9f70 con 0x55c9a9649270 2024-04-24T08:58:31.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.234+0000 7fcfc7e00640 1 --2- 172.21.15.83:0/1314043171 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9a9649270 0x55c9a9651260 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7fcfb8033000 tx=0x7fcfb8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a63a7239f561921 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:58:31.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.234+0000 7fcfb1600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:58:31.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.234+0000 7fcfb1600640 1 -- 172.21.15.83:0/1314043171 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb804c070 con 0x55c9a9649270 2024-04-24T08:58:31.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.234+0000 7fcfb1600640 1 -- 172.21.15.83:0/1314043171 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9567+0+0 (secure 0 0 0) 0x7fcfb8031db0 con 0x55c9a9649270 2024-04-24T08:58:31.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.234+0000 7fcfd19b4740 1 -- 172.21.15.83:0/1314043171 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcfb40047a0 con 0x55c9a9649270 2024-04-24T08:58:31.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.235+0000 7fcfb1600640 1 -- 172.21.15.83:0/1314043171 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fcfb8058050 con 0x55c9a9649270 2024-04-24T08:58:31.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.235+0000 7fcfd19b4740 10 client.18156 fetch_fsmap learned FSMap version 33 2024-04-24T08:58:31.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.235+0000 7fcfd19b4740 1 -- 172.21.15.83:0/1314043171 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55c9a9281170 con 0x55c9a9649270 2024-04-24T08:58:31.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.235+0000 7fcfb1600640 1 -- 172.21.15.83:0/1314043171 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcfb40047a0 con 0x55c9a9649270 2024-04-24T08:58:31.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.235+0000 7fcfb1600640 1 -- 172.21.15.83:0/1314043171 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fcfb803b430 con 0x55c9a9649270 2024-04-24T08:58:31.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.235+0000 7fcfd19b4740 10 client.18156 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T08:58:31.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.235+0000 7fcfd19b4740 10 client.18156 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:58:31.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.235+0000 7fcfd19b4740 20 client.18156 populate_metadata read hostname 'smithi083' 2024-04-24T08:58:31.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.235+0000 7fcfd19b4740 1 --2- 172.21.15.83:0/1314043171 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55c9a96645e0 0x55c9a9666a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:31.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.235+0000 7fcfd19b4740 4 client.18156 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:58:31.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.235+0000 7fcfd19b4740 1 -- 172.21.15.83:0/1314043171 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c9a95bc890 con 0x55c9a96645e0 2024-04-24T08:58:31.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.236+0000 7fcfc7400640 1 --2- 172.21.15.83:0/1314043171 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55c9a96645e0 0x55c9a9666a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:31.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.237+0000 7fcfc7400640 1 --2- 172.21.15.83:0/1314043171 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55c9a96645e0 0x55c9a9666a30 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9cb9f345f53f908a server_cookie=4f32e004ed58a848 in_seq=0 out_seq=0 2024-04-24T08:58:31.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.237+0000 7fcfb1600640 10 client.18156 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:58:31.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.240+0000 7fcfb1600640 1 -- 172.21.15.83:0/1314043171 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c9a95bc890 con 0x55c9a96645e0 2024-04-24T08:58:31.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.240+0000 7fcfb1600640 10 client.18156 handle_command_reply: tid=0 2024-04-24T08:58:31.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.241+0000 7fcfd19b4740 1 client.18156 shutdown 2024-04-24T08:58:31.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.241+0000 7fcfd19b4740 1 -- 172.21.15.83:0/1314043171 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9a9649270 msgr2=0x55c9a9651260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:31.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.241+0000 7fcfd19b4740 1 --2- 172.21.15.83:0/1314043171 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9a9649270 0x55c9a9651260 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7fcfb8033000 tx=0x7fcfb8009260 comp rx=0 tx=0).stop 2024-04-24T08:58:31.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.241+0000 7fcfd19b4740 1 -- 172.21.15.83:0/1314043171 shutdown_connections 2024-04-24T08:58:31.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.241+0000 7fcfd19b4740 1 --2- 172.21.15.83:0/1314043171 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55c9a96645e0 0x55c9a9666a30 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:31.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.241+0000 7fcfd19b4740 1 --2- 172.21.15.83:0/1314043171 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c9a9654260 0x55c9a9651ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:31.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.241+0000 7fcfd19b4740 1 --2- 172.21.15.83:0/1314043171 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c9a9649b90 0x55c9a96517a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:31.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.241+0000 7fcfb1600640 0 client.18156 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:58:31.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.241+0000 7fcfd19b4740 1 --2- 172.21.15.83:0/1314043171 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9a9649270 0x55c9a9651260 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:31.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.241+0000 7fcfd19b4740 1 -- 172.21.15.83:0/1314043171 >> 172.21.15.83:0/1314043171 conn(0x55c9a9645590 msgr2=0x55c9a964b390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:58:31.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.241+0000 7fcfd19b4740 1 -- 172.21.15.83:0/1314043171 shutdown_connections 2024-04-24T08:58:31.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.241+0000 7fcfd19b4740 1 -- 172.21.15.83:0/1314043171 wait complete. 2024-04-24T08:58:31.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.241+0000 7fcfd19b4740 20 client.18156 trim_cache size 0 max 16384 2024-04-24T08:58:31.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.262+0000 7fcfd19b4740 1 Processor -- start 2024-04-24T08:58:31.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.262+0000 7fcfd19b4740 1 -- start start 2024-04-24T08:58:31.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.262+0000 7fcfd19b4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c9a9649270 0x55c9a965ff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:31.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.262+0000 7fcfd19b4740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c9a9649b90 0x55c9a964b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:31.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.262+0000 7fcfd19b4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9a9654260 0x55c9a9644570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:31.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.262+0000 7fcfd19b4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc81b14a0 con 0x55c9a9654260 2024-04-24T08:58:31.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.262+0000 7fcfd19b4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc81b2fe0 con 0x55c9a9649270 2024-04-24T08:58:31.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.262+0000 7fcfd19b4740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc81b1b70 con 0x55c9a9649b90 2024-04-24T08:58:31.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.263+0000 7fcfcca00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9a9654260 0x55c9a9644570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:31.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.263+0000 7fcfcca00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9a9654260 0x55c9a9644570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47680/0 (socket says 172.21.15.83:47680) 2024-04-24T08:58:31.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.263+0000 7fcfcca00640 1 -- 172.21.15.83:0/1768692305 learned_addr learned my addr 172.21.15.83:0/1768692305 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:58:31.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.263+0000 7fcfc7e00640 1 --2- 172.21.15.83:0/1768692305 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c9a9649270 0x55c9a965ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:31.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.263+0000 7fcfc7400640 1 --2- 172.21.15.83:0/1768692305 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c9a9649b90 0x55c9a964b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:31.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.263+0000 7fcfcca00640 1 -- 172.21.15.83:0/1768692305 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c9a9649b90 msgr2=0x55c9a964b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:58:31.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.263+0000 7fcfcca00640 1 --2- 172.21.15.83:0/1768692305 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c9a9649b90 0x55c9a964b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:31.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.263+0000 7fcfcca00640 1 -- 172.21.15.83:0/1768692305 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c9a9649270 msgr2=0x55c9a965ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:58:31.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.263+0000 7fcfcca00640 1 --2- 172.21.15.83:0/1768692305 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c9a9649270 0x55c9a965ff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:31.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.263+0000 7fcfcca00640 1 -- 172.21.15.83:0/1768692305 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9a964f770 con 0x55c9a9654260 2024-04-24T08:58:31.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.264+0000 7fcfcca00640 1 --2- 172.21.15.83:0/1768692305 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9a9654260 0x55c9a9644570 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fcfc009cbc0 tx=0x7fcfc000fd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c7b3d18abe487a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:58:31.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.264+0000 7fcfb3e00640 1 -- 172.21.15.83:0/1768692305 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc0087040 con 0x55c9a9654260 2024-04-24T08:58:31.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.264+0000 7fcfb3e00640 1 -- 172.21.15.83:0/1768692305 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcfc00a0fe0 con 0x55c9a9654260 2024-04-24T08:58:31.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.264+0000 7fcfb3e00640 1 -- 172.21.15.83:0/1768692305 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc00992a0 con 0x55c9a9654260 2024-04-24T08:58:31.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.265+0000 7fcfd19b4740 1 -- 172.21.15.83:0/1768692305 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9a9654260 msgr2=0x55c9a9644570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:31.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.265+0000 7fcfd19b4740 1 --2- 172.21.15.83:0/1768692305 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9a9654260 0x55c9a9644570 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fcfc009cbc0 tx=0x7fcfc000fd10 comp rx=0 tx=0).stop 2024-04-24T08:58:31.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.265+0000 7fcfd19b4740 1 -- 172.21.15.83:0/1768692305 shutdown_connections 2024-04-24T08:58:31.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.265+0000 7fcfd19b4740 1 --2- 172.21.15.83:0/1768692305 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9a9654260 0x55c9a9644570 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:31.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.265+0000 7fcfd19b4740 1 --2- 172.21.15.83:0/1768692305 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c9a9649b90 0x55c9a964b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:31.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.265+0000 7fcfd19b4740 1 --2- 172.21.15.83:0/1768692305 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c9a9649270 0x55c9a965ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:31.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.265+0000 7fcfd19b4740 1 -- 172.21.15.83:0/1768692305 >> 172.21.15.83:0/1768692305 conn(0x55c9a9645590 msgr2=0x55c9a9643e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:58:31.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.265+0000 7fcfd19b4740 1 -- 172.21.15.83:0/1768692305 shutdown_connections 2024-04-24T08:58:31.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.265+0000 7fcfd19b4740 1 -- 172.21.15.83:0/1768692305 wait complete. 2024-04-24T08:58:31.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.265+0000 7fcfd19b4740 1 -- start start 2024-04-24T08:58:31.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.266+0000 7fcfd19b4740 1 Processor -- start 2024-04-24T08:58:31.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.266+0000 7fcfd19b4740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c9a9649270 0x55c9a965c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:31.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.266+0000 7fcfd19b4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9a9649b90 0x55c9a9658900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:31.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.266+0000 7fcfd19b4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c9a9654260 0x55c9a9658e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:31.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.266+0000 7fcfd19b4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc81b2910 con 0x55c9a9649b90 2024-04-24T08:58:31.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.266+0000 7fcfd19b4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc81b2240 con 0x55c9a9654260 2024-04-24T08:58:31.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.266+0000 7fcfd19b4740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc81af480 con 0x55c9a9649270 2024-04-24T08:58:31.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.266+0000 7fcfd19b4740 10 client.0 _ll_register_callbacks cb 0x7fcfb8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:58:31.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.266+0000 7fcfc7400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9a9649b90 0x55c9a9658900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:31.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.267+0000 7fcfc7400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9a9649b90 0x55c9a9658900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47688/0 (socket says 172.21.15.83:47688) 2024-04-24T08:58:31.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.267+0000 7fcfc7400640 1 -- 172.21.15.83:0/1802467723 learned_addr learned my addr 172.21.15.83:0/1802467723 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:58:31.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.267+0000 7fcfcca00640 1 --2- 172.21.15.83:0/1802467723 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c9a9654260 0x55c9a9658e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:31.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.267+0000 7fcfc7e00640 1 --2- 172.21.15.83:0/1802467723 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c9a9649270 0x55c9a965c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:31.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.267+0000 7fcfc7400640 1 -- 172.21.15.83:0/1802467723 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c9a9649270 msgr2=0x55c9a965c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:58:31.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.267+0000 7fcfc7400640 1 --2- 172.21.15.83:0/1802467723 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c9a9649270 0x55c9a965c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:31.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.267+0000 7fcfc7400640 1 -- 172.21.15.83:0/1802467723 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c9a9654260 msgr2=0x55c9a9658e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:58:31.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.267+0000 7fcfc7400640 1 --2- 172.21.15.83:0/1802467723 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c9a9654260 0x55c9a9658e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:31.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.267+0000 7fcfc7400640 1 -- 172.21.15.83:0/1802467723 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9a9659400 con 0x55c9a9649b90 2024-04-24T08:58:31.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.267+0000 7fcfc7400640 1 --2- 172.21.15.83:0/1802467723 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9a9649b90 0x55c9a9658900 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7fcfb4012f10 tx=0x7fcfb4005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8631d8c84c595701 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:58:31.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.267+0000 7fcfb1600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:58:31.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.268+0000 7fcfb1600640 1 -- 172.21.15.83:0/1802467723 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb4030070 con 0x55c9a9649b90 2024-04-24T08:58:31.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.268+0000 7fcfd19b4740 1 -- 172.21.15.83:0/1802467723 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcfc00a0fe0 con 0x55c9a9649b90 2024-04-24T08:58:31.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.268+0000 7fcfb1600640 1 -- 172.21.15.83:0/1802467723 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9567+0+0 (secure 0 0 0) 0x7fcfb4004220 con 0x55c9a9649b90 2024-04-24T08:58:31.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.268+0000 7fcfb1600640 1 -- 172.21.15.83:0/1802467723 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fcfb403b050 con 0x55c9a9649b90 2024-04-24T08:58:31.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.268+0000 7fcfd19b4740 10 client.18162 fetch_fsmap learned FSMap version 33 2024-04-24T08:58:31.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.268+0000 7fcfd19b4740 1 -- 172.21.15.83:0/1802467723 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55c9a96d4160 con 0x55c9a9649b90 2024-04-24T08:58:31.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.268+0000 7fcfb1600640 1 -- 172.21.15.83:0/1802467723 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcfc00a0fe0 con 0x55c9a9649b90 2024-04-24T08:58:31.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.269+0000 7fcfb1600640 1 -- 172.21.15.83:0/1802467723 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fcfb4003990 con 0x55c9a9649b90 2024-04-24T08:58:31.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.269+0000 7fcfd19b4740 10 client.18162 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T08:58:31.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.269+0000 7fcfd19b4740 10 client.18162 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:58:31.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.269+0000 7fcfd19b4740 20 client.18162 populate_metadata read hostname 'smithi083' 2024-04-24T08:58:31.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.269+0000 7fcfd19b4740 1 --2- 172.21.15.83:0/1802467723 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55c9a96d4550 0x55c9a96d6940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:31.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.269+0000 7fcfd19b4740 4 client.18162 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T08:58:31.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.269+0000 7fcfd19b4740 1 -- 172.21.15.83:0/1802467723 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55c9a95b6700 con 0x55c9a96d4550 2024-04-24T08:58:31.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.269+0000 7fcfc7e00640 1 --2- 172.21.15.83:0/1802467723 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55c9a96d4550 0x55c9a96d6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:31.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.270+0000 7fcfc7e00640 1 --2- 172.21.15.83:0/1802467723 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55c9a96d4550 0x55c9a96d6940 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1cd4f6134fafcbb server_cookie=517d67a0ba48b1ba in_seq=0 out_seq=0 2024-04-24T08:58:31.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.270+0000 7fcfb1600640 10 client.18162 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:58:31.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.273+0000 7fcfb1600640 1 -- 172.21.15.83:0/1802467723 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55c9a95b6700 con 0x55c9a96d4550 2024-04-24T08:58:31.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.273+0000 7fcfb1600640 10 client.18162 handle_command_reply: tid=0 2024-04-24T08:58:31.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.273+0000 7fcfd19b4740 1 client.18162 shutdown 2024-04-24T08:58:31.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.273+0000 7fcfd19b4740 1 -- 172.21.15.83:0/1802467723 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9a9649b90 msgr2=0x55c9a9658900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:31.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.273+0000 7fcfd19b4740 1 --2- 172.21.15.83:0/1802467723 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9a9649b90 0x55c9a9658900 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7fcfb4012f10 tx=0x7fcfb4005c00 comp rx=0 tx=0).stop 2024-04-24T08:58:31.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.273+0000 7fcfd19b4740 1 -- 172.21.15.83:0/1802467723 shutdown_connections 2024-04-24T08:58:31.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.273+0000 7fcfd19b4740 1 --2- 172.21.15.83:0/1802467723 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55c9a96d4550 0x55c9a96d6940 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:31.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.273+0000 7fcfd19b4740 1 --2- 172.21.15.83:0/1802467723 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c9a9654260 0x55c9a9658e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:31.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.273+0000 7fcfb1600640 0 client.18162 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:58:31.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.273+0000 7fcfd19b4740 1 --2- 172.21.15.83:0/1802467723 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c9a9649b90 0x55c9a9658900 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:31.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.273+0000 7fcfd19b4740 1 --2- 172.21.15.83:0/1802467723 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c9a9649270 0x55c9a965c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:31.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.273+0000 7fcfd19b4740 1 -- 172.21.15.83:0/1802467723 >> 172.21.15.83:0/1802467723 conn(0x55c9a9645590 msgr2=0x55c9a96446f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:58:31.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.273+0000 7fcfd19b4740 1 -- 172.21.15.83:0/1802467723 shutdown_connections 2024-04-24T08:58:31.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.273+0000 7fcfd19b4740 1 -- 172.21.15.83:0/1802467723 wait complete. 2024-04-24T08:58:31.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.274+0000 7fcfd19b4740 20 client.18162 trim_cache size 0 max 16384 2024-04-24T08:58:31.275 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T08:58:31.275 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-04-24T08:58:31.275 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "c6a76872-6bea-4691-b0b6-1772e2aec54a", 2024-04-24T08:58:31.275 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-04-24T08:58:31.275 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T08:58:31.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.274+0000 7fcfcd400640 1 -- 172.21.15.83:0/2450309155 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fcfac0a3d20 msgr2=0x7fcfac0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:31.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.274+0000 7fcfcd400640 1 --2- 172.21.15.83:0/2450309155 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fcfac0a3d20 0x7fcfac0a61e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fcfb8000c00 tx=0x7fcfb8031400 comp rx=0 tx=0).stop 2024-04-24T08:58:31.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.275+0000 7fcfcd400640 1 -- 172.21.15.83:0/2450309155 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcfc819a680 msgr2=0x7fcfc819ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:31.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.275+0000 7fcfcd400640 1 --2- 172.21.15.83:0/2450309155 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcfc819a680 0x7fcfc819ab00 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7fcfc000cce0 tx=0x7fcfc000a620 comp rx=0 tx=0).stop 2024-04-24T08:58:31.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.275+0000 7fcfcd400640 1 -- 172.21.15.83:0/2450309155 shutdown_connections 2024-04-24T08:58:31.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.275+0000 7fcfcd400640 1 --2- 172.21.15.83:0/2450309155 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fcfac0a3d20 0x7fcfac0a61e0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:31.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.275+0000 7fcfcd400640 1 --2- 172.21.15.83:0/2450309155 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcfc819a680 0x7fcfc819ab00 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:31.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.275+0000 7fcfcd400640 1 --2- 172.21.15.83:0/2450309155 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcfc81093f0 0x7fcfc81a0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:31.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.275+0000 7fcfcd400640 1 --2- 172.21.15.83:0/2450309155 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcfc8104320 0x7fcfc81a05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:31.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.275+0000 7fcfcd400640 1 -- 172.21.15.83:0/2450309155 >> 172.21.15.83:0/2450309155 conn(0x7fcfc80ffb40 msgr2=0x7fcfc8101e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:58:31.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.275+0000 7fcfcd400640 1 -- 172.21.15.83:0/2450309155 shutdown_connections 2024-04-24T08:58:31.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.275+0000 7fcfcd400640 1 -- 172.21.15.83:0/2450309155 wait complete. 2024-04-24T08:58:31.292 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T08:58:31.292 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T08:58:31.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.494+0000 7f686ae00640 1 -- 172.21.15.83:0/169856515 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6864101610 msgr2=0x7f686410e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:31.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.494+0000 7f686ae00640 1 --2- 172.21.15.83:0/169856515 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6864101610 0x7f686410e2d0 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f6858009900 tx=0x7f685802a410 comp rx=0 tx=0).stop 2024-04-24T08:58:31.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.494+0000 7f686ae00640 1 -- 172.21.15.83:0/169856515 shutdown_connections 2024-04-24T08:58:31.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.494+0000 7f686ae00640 1 --2- 172.21.15.83:0/169856515 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f686410e810 0x7f6864110c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:31.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.494+0000 7f686ae00640 1 --2- 172.21.15.83:0/169856515 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6864101610 0x7f686410e2d0 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:31.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.494+0000 7f686ae00640 1 --2- 172.21.15.83:0/169856515 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6864100c60 0x7f6864101040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:31.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.494+0000 7f686ae00640 1 -- 172.21.15.83:0/169856515 >> 172.21.15.83:0/169856515 conn(0x7f68640fc880 msgr2=0x7f68640feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:58:31.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.494+0000 7f686ae00640 1 -- 172.21.15.83:0/169856515 shutdown_connections 2024-04-24T08:58:31.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.494+0000 7f686ae00640 1 -- 172.21.15.83:0/169856515 wait complete. 2024-04-24T08:58:31.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.495+0000 7f686ae00640 1 Processor -- start 2024-04-24T08:58:31.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.495+0000 7f686ae00640 1 -- start start 2024-04-24T08:58:31.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.495+0000 7f686ae00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6864100c60 0x7f68641a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:31.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.495+0000 7f686ae00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6864101610 0x7f68641a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:31.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.495+0000 7f686ae00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f686410e810 0x7f686419a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:31.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.495+0000 7f686ae00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6864114360 con 0x7f6864100c60 2024-04-24T08:58:31.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.495+0000 7f686ae00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f68641141e0 con 0x7f6864101610 2024-04-24T08:58:31.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.495+0000 7f686ae00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f68641144e0 con 0x7f686410e810 2024-04-24T08:58:31.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.496+0000 7f6869a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6864100c60 0x7f68641a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:31.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.496+0000 7f6869a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6864100c60 0x7f68641a0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47708/0 (socket says 172.21.15.83:47708) 2024-04-24T08:58:31.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.496+0000 7f6869a00640 1 -- 172.21.15.83:0/2646322311 learned_addr learned my addr 172.21.15.83:0/2646322311 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:58:31.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.496+0000 7f6869000640 1 --2- 172.21.15.83:0/2646322311 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6864101610 0x7f68641a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:31.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.496+0000 7f6869a00640 1 -- 172.21.15.83:0/2646322311 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f686410e810 msgr2=0x7f686419a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:31.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.496+0000 7f686a400640 1 --2- 172.21.15.83:0/2646322311 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f686410e810 0x7f686419a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:31.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.496+0000 7f6869a00640 1 --2- 172.21.15.83:0/2646322311 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f686410e810 0x7f686419a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:31.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.496+0000 7f6869a00640 1 -- 172.21.15.83:0/2646322311 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6864101610 msgr2=0x7f68641a0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:31.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.496+0000 7f6869a00640 1 --2- 172.21.15.83:0/2646322311 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6864101610 0x7f68641a0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:31.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.496+0000 7f6869a00640 1 -- 172.21.15.83:0/2646322311 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f686419afe0 con 0x7f6864100c60 2024-04-24T08:58:31.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.496+0000 7f6869000640 1 --2- 172.21.15.83:0/2646322311 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6864101610 0x7f68641a0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:58:31.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.496+0000 7f686a400640 1 --2- 172.21.15.83:0/2646322311 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f686410e810 0x7f686419a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:58:31.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.497+0000 7f6869a00640 1 --2- 172.21.15.83:0/2646322311 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6864100c60 0x7f68641a0660 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f685c00eab0 tx=0x7f685c00ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:58:31.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.497+0000 7f6862000640 1 -- 172.21.15.83:0/2646322311 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f685c00ce40 con 0x7f6864100c60 2024-04-24T08:58:31.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.497+0000 7f6862000640 1 -- 172.21.15.83:0/2646322311 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f685c011920 con 0x7f6864100c60 2024-04-24T08:58:31.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.497+0000 7f686ae00640 1 -- 172.21.15.83:0/2646322311 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f686419b2d0 con 0x7f6864100c60 2024-04-24T08:58:31.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.497+0000 7f6862000640 1 -- 172.21.15.83:0/2646322311 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f685c010a00 con 0x7f6864100c60 2024-04-24T08:58:31.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.497+0000 7f686ae00640 1 -- 172.21.15.83:0/2646322311 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68641a75e0 con 0x7f6864100c60 2024-04-24T08:58:31.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.501+0000 7f686f374740 1 Processor -- start 2024-04-24T08:58:31.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.501+0000 7f686f374740 1 -- start start 2024-04-24T08:58:31.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.501+0000 7f6862000640 1 -- 172.21.15.83:0/2646322311 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f685c011ad0 con 0x7f6864100c60 2024-04-24T08:58:31.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.501+0000 7f686f374740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56408f79ea30 0x56408f79ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:31.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.501+0000 7f686f374740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56408f79f350 0x56408f7a94e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:31.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.501+0000 7f686f374740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56408f7a9a20 0x56408f7abe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:31.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.501+0000 7f686f374740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56408f7041f0 con 0x56408f7a9a20 2024-04-24T08:58:31.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.501+0000 7f686f374740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56408f6aa350 con 0x56408f79f350 2024-04-24T08:58:31.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.501+0000 7f686f374740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56408f7001b0 con 0x56408f79ea30 2024-04-24T08:58:31.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.501+0000 7f6869000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56408f7a9a20 0x56408f7abe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:31.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.501+0000 7f6869000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56408f7a9a20 0x56408f7abe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47718/0 (socket says 172.21.15.83:47718) 2024-04-24T08:58:31.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.501+0000 7f6869000640 1 -- 172.21.15.83:0/2994128984 learned_addr learned my addr 172.21.15.83:0/2994128984 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:58:31.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.502+0000 7f6869a00640 1 --2- 172.21.15.83:0/2994128984 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56408f79f350 0x56408f7a94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:31.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.502+0000 7f686a400640 1 --2- 172.21.15.83:0/2994128984 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56408f79ea30 0x56408f79ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:31.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.502+0000 7f6869000640 1 -- 172.21.15.83:0/2994128984 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56408f79ea30 msgr2=0x56408f79ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:58:31.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.502+0000 7f6869000640 1 --2- 172.21.15.83:0/2994128984 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56408f79ea30 0x56408f79ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:31.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.502+0000 7f6869000640 1 -- 172.21.15.83:0/2994128984 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56408f79f350 msgr2=0x56408f7a94e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:58:31.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.502+0000 7f6869000640 1 --2- 172.21.15.83:0/2994128984 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56408f79f350 0x56408f7a94e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:31.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.502+0000 7f6869000640 1 -- 172.21.15.83:0/2994128984 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56408f6fef50 con 0x56408f7a9a20 2024-04-24T08:58:31.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.503+0000 7f6869000640 1 --2- 172.21.15.83:0/2994128984 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56408f7a9a20 0x56408f7abe10 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f6858006010 tx=0x7f6858032cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e8040841217ebbf server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:58:31.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.503+0000 7f6852a00640 1 -- 172.21.15.83:0/2994128984 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f685803f070 con 0x56408f7a9a20 2024-04-24T08:58:31.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.503+0000 7f6862000640 1 --2- 172.21.15.83:0/2646322311 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f684c0a3d20 0x7f684c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:31.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.503+0000 7f6852a00640 1 -- 172.21.15.83:0/2994128984 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f68580316a0 con 0x56408f7a9a20 2024-04-24T08:58:31.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.503+0000 7f6852a00640 1 -- 172.21.15.83:0/2994128984 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6858035070 con 0x56408f7a9a20 2024-04-24T08:58:31.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.503+0000 7f6862000640 1 -- 172.21.15.83:0/2646322311 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9567+0+0 (secure 0 0 0) 0x7f685c0d1320 con 0x7f6864100c60 2024-04-24T08:58:31.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.503+0000 7f686a400640 1 --2- 172.21.15.83:0/2646322311 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f684c0a3d20 0x7f684c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:31.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.504+0000 7f686f374740 1 -- 172.21.15.83:0/2994128984 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56408f7a9a20 msgr2=0x56408f7abe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:31.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.504+0000 7f686f374740 1 --2- 172.21.15.83:0/2994128984 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56408f7a9a20 0x56408f7abe10 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f6858006010 tx=0x7f6858032cc0 comp rx=0 tx=0).stop 2024-04-24T08:58:31.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.504+0000 7f686f374740 1 -- 172.21.15.83:0/2994128984 shutdown_connections 2024-04-24T08:58:31.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.504+0000 7f686f374740 1 --2- 172.21.15.83:0/2994128984 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56408f7a9a20 0x56408f7abe10 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:31.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.504+0000 7f686f374740 1 --2- 172.21.15.83:0/2994128984 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56408f79f350 0x56408f7a94e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:31.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.504+0000 7f686f374740 1 --2- 172.21.15.83:0/2994128984 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56408f79ea30 0x56408f79ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:31.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.504+0000 7f686f374740 1 -- 172.21.15.83:0/2994128984 >> 172.21.15.83:0/2994128984 conn(0x56408f79ad50 msgr2=0x56408f79b130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:58:31.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.504+0000 7f686f374740 1 -- 172.21.15.83:0/2994128984 shutdown_connections 2024-04-24T08:58:31.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.504+0000 7f686a400640 1 --2- 172.21.15.83:0/2646322311 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f684c0a3d20 0x7f684c0a61e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f686406a270 tx=0x7f685400c5e0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:58:31.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.504+0000 7f686f374740 1 -- 172.21.15.83:0/2994128984 wait complete. 2024-04-24T08:58:31.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.505+0000 7f686f374740 1 -- start start 2024-04-24T08:58:31.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.505+0000 7f686f374740 1 Processor -- start 2024-04-24T08:58:31.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.505+0000 7f686f374740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56408f79ea30 0x56408f7a69b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:31.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.505+0000 7f686f374740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56408f79f350 0x56408f7a6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:31.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.505+0000 7f686f374740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56408f7a9a20 0x56408f7a7430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:31.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.505+0000 7f686f374740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56408f7ac7b0 con 0x56408f7a9a20 2024-04-24T08:58:31.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.505+0000 7f686f374740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56408f6bd690 con 0x56408f79f350 2024-04-24T08:58:31.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.505+0000 7f686f374740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56408f7ac930 con 0x56408f79ea30 2024-04-24T08:58:31.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.506+0000 7f686f374740 10 client.0 _ll_register_callbacks cb 0x56408f6f9650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:58:31.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.506+0000 7f686a400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56408f7a9a20 0x56408f7a7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:31.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.506+0000 7f686a400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56408f7a9a20 0x56408f7a7430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47734/0 (socket says 172.21.15.83:47734) 2024-04-24T08:58:31.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.506+0000 7f686a400640 1 -- 172.21.15.83:0/724021850 learned_addr learned my addr 172.21.15.83:0/724021850 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:58:31.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.506+0000 7f6869a00640 1 --2- 172.21.15.83:0/724021850 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56408f79ea30 0x56408f7a69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:31.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.506+0000 7f6869000640 1 --2- 172.21.15.83:0/724021850 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56408f79f350 0x56408f7a6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:31.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.506+0000 7f686a400640 1 -- 172.21.15.83:0/724021850 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56408f79ea30 msgr2=0x56408f7a69b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:58:31.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.506+0000 7f686a400640 1 --2- 172.21.15.83:0/724021850 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56408f79ea30 0x56408f7a69b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:31.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.506+0000 7f686a400640 1 -- 172.21.15.83:0/724021850 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56408f79f350 msgr2=0x56408f7a6ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:58:31.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.506+0000 7f686a400640 1 --2- 172.21.15.83:0/724021850 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56408f79f350 0x56408f7a6ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:31.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.506+0000 7f686a400640 1 -- 172.21.15.83:0/724021850 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56408f6bc920 con 0x56408f7a9a20 2024-04-24T08:58:31.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.507+0000 7f686a400640 1 --2- 172.21.15.83:0/724021850 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56408f7a9a20 0x56408f7a7430 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f6854012450 tx=0x7f6854004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=489766cd756030f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:58:31.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.507+0000 7f6847400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:58:31.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.507+0000 7f6847400640 1 -- 172.21.15.83:0/724021850 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6854023070 con 0x56408f7a9a20 2024-04-24T08:58:31.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.507+0000 7f6847400640 1 -- 172.21.15.83:0/724021850 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9567+0+0 (secure 0 0 0) 0x7f685401f860 con 0x56408f7a9a20 2024-04-24T08:58:31.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.507+0000 7f686f374740 1 -- 172.21.15.83:0/724021850 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68580316a0 con 0x56408f7a9a20 2024-04-24T08:58:31.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.507+0000 7f6847400640 1 -- 172.21.15.83:0/724021850 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f685402f050 con 0x56408f7a9a20 2024-04-24T08:58:31.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.507+0000 7f686f374740 10 client.18174 fetch_fsmap learned FSMap version 33 2024-04-24T08:58:31.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.507+0000 7f686f374740 1 -- 172.21.15.83:0/724021850 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56408f4e07d0 con 0x56408f7a9a20 2024-04-24T08:58:31.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.508+0000 7f6847400640 1 -- 172.21.15.83:0/724021850 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f68580316a0 con 0x56408f7a9a20 2024-04-24T08:58:31.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.508+0000 7f6847400640 1 -- 172.21.15.83:0/724021850 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f685400f040 con 0x56408f7a9a20 2024-04-24T08:58:31.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.508+0000 7f686f374740 10 client.18174 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T08:58:31.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.508+0000 7f686f374740 10 client.18174 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:58:31.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.508+0000 7f686f374740 20 client.18174 populate_metadata read hostname 'smithi083' 2024-04-24T08:58:31.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.508+0000 7f686f374740 1 --2- 172.21.15.83:0/724021850 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56408f7b9d40 0x56408f7bc190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:31.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.508+0000 7f686f374740 4 client.18174 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:58:31.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.508+0000 7f686f374740 1 -- 172.21.15.83:0/724021850 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56408f711c10 con 0x56408f7b9d40 2024-04-24T08:58:31.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.509+0000 7f6869a00640 1 --2- 172.21.15.83:0/724021850 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56408f7b9d40 0x56408f7bc190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:31.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.510+0000 7f6869a00640 1 --2- 172.21.15.83:0/724021850 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56408f7b9d40 0x56408f7bc190 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de640442771ab754 server_cookie=54616ba7be4173b7 in_seq=0 out_seq=0 2024-04-24T08:58:31.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.510+0000 7f6847400640 10 client.18174 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:58:31.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.513+0000 7f6847400640 1 -- 172.21.15.83:0/724021850 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56408f711c10 con 0x56408f7b9d40 2024-04-24T08:58:31.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.513+0000 7f6847400640 10 client.18174 handle_command_reply: tid=0 2024-04-24T08:58:31.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.513+0000 7f686f374740 1 client.18174 shutdown 2024-04-24T08:58:31.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.513+0000 7f686f374740 1 -- 172.21.15.83:0/724021850 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56408f7a9a20 msgr2=0x56408f7a7430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:31.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.513+0000 7f686f374740 1 --2- 172.21.15.83:0/724021850 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56408f7a9a20 0x56408f7a7430 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f6854012450 tx=0x7f6854004730 comp rx=0 tx=0).stop 2024-04-24T08:58:31.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.513+0000 7f686f374740 1 -- 172.21.15.83:0/724021850 shutdown_connections 2024-04-24T08:58:31.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.513+0000 7f686f374740 1 --2- 172.21.15.83:0/724021850 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56408f7b9d40 0x56408f7bc190 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:31.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.513+0000 7f686f374740 1 --2- 172.21.15.83:0/724021850 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56408f7a9a20 0x56408f7a7430 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:31.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.513+0000 7f686f374740 1 --2- 172.21.15.83:0/724021850 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56408f79f350 0x56408f7a6ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:31.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.513+0000 7f686f374740 1 --2- 172.21.15.83:0/724021850 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56408f79ea30 0x56408f7a69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:31.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.513+0000 7f686f374740 1 -- 172.21.15.83:0/724021850 >> 172.21.15.83:0/724021850 conn(0x56408f79ad50 msgr2=0x56408f799fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:58:31.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.513+0000 7f6847400640 0 client.18174 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:58:31.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.513+0000 7f686f374740 1 -- 172.21.15.83:0/724021850 shutdown_connections 2024-04-24T08:58:31.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.513+0000 7f686f374740 1 -- 172.21.15.83:0/724021850 wait complete. 2024-04-24T08:58:31.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.513+0000 7f686f374740 20 client.18174 trim_cache size 0 max 16384 2024-04-24T08:58:31.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.534+0000 7f686f374740 1 Processor -- start 2024-04-24T08:58:31.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.534+0000 7f686f374740 1 -- start start 2024-04-24T08:58:31.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.534+0000 7f686f374740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56408f79ea30 0x56408f7a90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:31.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.534+0000 7f686f374740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56408f79f350 0x56408f7b5670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:31.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.534+0000 7f686f374740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56408f7a9a20 0x56408f7c1a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:31.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.534+0000 7f686f374740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f68641b16c0 con 0x56408f79f350 2024-04-24T08:58:31.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.534+0000 7f686f374740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f68641b3200 con 0x56408f79ea30 2024-04-24T08:58:31.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.534+0000 7f686f374740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f68641b1d90 con 0x56408f7a9a20 2024-04-24T08:58:31.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.534+0000 7f6869000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56408f79f350 0x56408f7b5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:31.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.534+0000 7f6869000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56408f79f350 0x56408f7b5670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47750/0 (socket says 172.21.15.83:47750) 2024-04-24T08:58:31.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.534+0000 7f6869000640 1 -- 172.21.15.83:0/1480562845 learned_addr learned my addr 172.21.15.83:0/1480562845 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:58:31.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.535+0000 7f6869a00640 1 --2- 172.21.15.83:0/1480562845 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56408f79ea30 0x56408f7a90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:31.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.535+0000 7f686a400640 1 --2- 172.21.15.83:0/1480562845 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56408f7a9a20 0x56408f7c1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:31.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.535+0000 7f6869000640 1 -- 172.21.15.83:0/1480562845 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56408f7a9a20 msgr2=0x56408f7c1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:58:31.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.535+0000 7f6869000640 1 --2- 172.21.15.83:0/1480562845 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56408f7a9a20 0x56408f7c1a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:31.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.535+0000 7f6869000640 1 -- 172.21.15.83:0/1480562845 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56408f79ea30 msgr2=0x56408f7a90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:58:31.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.535+0000 7f6869000640 1 --2- 172.21.15.83:0/1480562845 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56408f79ea30 0x56408f7a90e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:31.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.535+0000 7f6869000640 1 -- 172.21.15.83:0/1480562845 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56408f4163f0 con 0x56408f79f350 2024-04-24T08:58:31.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.535+0000 7f6869a00640 1 --2- 172.21.15.83:0/1480562845 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56408f79ea30 0x56408f7a90e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:58:31.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.535+0000 7f6869000640 1 --2- 172.21.15.83:0/1480562845 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56408f79f350 0x56408f7b5670 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f6858000c00 tx=0x7f6858005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57d9255f02e5cb2b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:58:31.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.536+0000 7f6852000640 1 -- 172.21.15.83:0/1480562845 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f685803f070 con 0x56408f79f350 2024-04-24T08:58:31.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.536+0000 7f6852000640 1 -- 172.21.15.83:0/1480562845 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f685804c070 con 0x56408f79f350 2024-04-24T08:58:31.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.536+0000 7f6852000640 1 -- 172.21.15.83:0/1480562845 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6858035040 con 0x56408f79f350 2024-04-24T08:58:31.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.536+0000 7f686f374740 1 -- 172.21.15.83:0/1480562845 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56408f79f350 msgr2=0x56408f7b5670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:31.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.536+0000 7f686f374740 1 --2- 172.21.15.83:0/1480562845 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56408f79f350 0x56408f7b5670 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f6858000c00 tx=0x7f6858005c00 comp rx=0 tx=0).stop 2024-04-24T08:58:31.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.536+0000 7f686f374740 1 -- 172.21.15.83:0/1480562845 shutdown_connections 2024-04-24T08:58:31.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.536+0000 7f686f374740 1 --2- 172.21.15.83:0/1480562845 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56408f7a9a20 0x56408f7c1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:31.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.536+0000 7f686f374740 1 --2- 172.21.15.83:0/1480562845 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56408f79f350 0x56408f7b5670 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:31.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.536+0000 7f686f374740 1 --2- 172.21.15.83:0/1480562845 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56408f79ea30 0x56408f7a90e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:31.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.537+0000 7f686f374740 1 -- 172.21.15.83:0/1480562845 >> 172.21.15.83:0/1480562845 conn(0x56408f79ad50 msgr2=0x56408f7a0b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:58:31.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.537+0000 7f686f374740 1 -- 172.21.15.83:0/1480562845 shutdown_connections 2024-04-24T08:58:31.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.537+0000 7f686f374740 1 -- 172.21.15.83:0/1480562845 wait complete. 2024-04-24T08:58:31.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.537+0000 7f686f374740 1 -- start start 2024-04-24T08:58:31.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.537+0000 7f686f374740 1 Processor -- start 2024-04-24T08:58:31.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.538+0000 7f686f374740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56408f79ea30 0x56408f7b1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:31.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.538+0000 7f686f374740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56408f79f350 0x56408f7ad120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:31.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.538+0000 7f686f374740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56408f7a9a20 0x56408f7ad660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:31.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.538+0000 7f686f374740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f68641b2b30 con 0x56408f79f350 2024-04-24T08:58:31.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.538+0000 7f686f374740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f68641b2460 con 0x56408f79ea30 2024-04-24T08:58:31.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.538+0000 7f686f374740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f68641af6a0 con 0x56408f7a9a20 2024-04-24T08:58:31.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.538+0000 7f686f374740 10 client.0 _ll_register_callbacks cb 0x7f6854016070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:58:31.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.538+0000 7f6869000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56408f79f350 0x56408f7ad120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:31.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.538+0000 7f6869000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56408f79f350 0x56408f7ad120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47758/0 (socket says 172.21.15.83:47758) 2024-04-24T08:58:31.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.538+0000 7f6869000640 1 -- 172.21.15.83:0/2083417386 learned_addr learned my addr 172.21.15.83:0/2083417386 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:58:31.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.538+0000 7f6869a00640 1 --2- 172.21.15.83:0/2083417386 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56408f79ea30 0x56408f7b1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:31.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.538+0000 7f6869000640 1 -- 172.21.15.83:0/2083417386 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56408f7a9a20 msgr2=0x56408f7ad660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:58:31.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.538+0000 7f6869000640 1 --2- 172.21.15.83:0/2083417386 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56408f7a9a20 0x56408f7ad660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:31.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.538+0000 7f6869000640 1 -- 172.21.15.83:0/2083417386 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56408f79ea30 msgr2=0x56408f7b1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:58:31.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.538+0000 7f6869000640 1 --2- 172.21.15.83:0/2083417386 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56408f79ea30 0x56408f7b1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:31.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.538+0000 7f6869000640 1 -- 172.21.15.83:0/2083417386 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56408f3d60f0 con 0x56408f79f350 2024-04-24T08:58:31.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.538+0000 7f6869a00640 1 --2- 172.21.15.83:0/2083417386 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56408f79ea30 0x56408f7b1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:58:31.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.539+0000 7f6869000640 1 --2- 172.21.15.83:0/2083417386 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56408f79f350 0x56408f7ad120 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f6858009900 tx=0x7f6858031810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dafc02434204a320 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:58:31.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.539+0000 7f6847400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:58:31.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.539+0000 7f6847400640 1 -- 172.21.15.83:0/2083417386 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6858035030 con 0x56408f79f350 2024-04-24T08:58:31.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.539+0000 7f6847400640 1 -- 172.21.15.83:0/2083417386 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9567+0+0 (secure 0 0 0) 0x7f6858004440 con 0x56408f79f350 2024-04-24T08:58:31.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.539+0000 7f686f374740 1 -- 172.21.15.83:0/2083417386 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f685804c070 con 0x56408f79f350 2024-04-24T08:58:31.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.539+0000 7f6847400640 1 -- 172.21.15.83:0/2083417386 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f685805e050 con 0x56408f79f350 2024-04-24T08:58:31.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.539+0000 7f686f374740 10 client.18180 fetch_fsmap learned FSMap version 33 2024-04-24T08:58:31.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.539+0000 7f686f374740 1 -- 172.21.15.83:0/2083417386 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56408f6c9810 con 0x56408f79f350 2024-04-24T08:58:31.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.540+0000 7f6847400640 1 -- 172.21.15.83:0/2083417386 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f685804c070 con 0x56408f79f350 2024-04-24T08:58:31.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.540+0000 7f6847400640 1 -- 172.21.15.83:0/2083417386 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f685803b960 con 0x56408f79f350 2024-04-24T08:58:31.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.540+0000 7f686f374740 10 client.18180 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T08:58:31.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.540+0000 7f686f374740 10 client.18180 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:58:31.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.540+0000 7f686f374740 20 client.18180 populate_metadata read hostname 'smithi083' 2024-04-24T08:58:31.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.540+0000 7f686f374740 1 --2- 172.21.15.83:0/2083417386 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56408f820120 0x56408f822510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:31.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.540+0000 7f686f374740 4 client.18180 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T08:58:31.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.540+0000 7f686f374740 1 -- 172.21.15.83:0/2083417386 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56408f70ba80 con 0x56408f820120 2024-04-24T08:58:31.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.540+0000 7f6869a00640 1 --2- 172.21.15.83:0/2083417386 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56408f820120 0x56408f822510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:31.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.541+0000 7f6869a00640 1 --2- 172.21.15.83:0/2083417386 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56408f820120 0x56408f822510 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fcce4d182519a809 server_cookie=f132a10ca10fbca8 in_seq=0 out_seq=0 2024-04-24T08:58:31.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.541+0000 7f6847400640 10 client.18180 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:58:31.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.543+0000 7f6847400640 1 -- 172.21.15.83:0/2083417386 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56408f70ba80 con 0x56408f820120 2024-04-24T08:58:31.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.543+0000 7f6847400640 10 client.18180 handle_command_reply: tid=0 2024-04-24T08:58:31.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.544+0000 7f686f374740 1 client.18180 shutdown 2024-04-24T08:58:31.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.544+0000 7f686f374740 1 -- 172.21.15.83:0/2083417386 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56408f79f350 msgr2=0x56408f7ad120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:31.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.544+0000 7f686f374740 1 --2- 172.21.15.83:0/2083417386 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56408f79f350 0x56408f7ad120 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f6858009900 tx=0x7f6858031810 comp rx=0 tx=0).stop 2024-04-24T08:58:31.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.544+0000 7f686f374740 1 -- 172.21.15.83:0/2083417386 shutdown_connections 2024-04-24T08:58:31.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.544+0000 7f686f374740 1 --2- 172.21.15.83:0/2083417386 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56408f820120 0x56408f822510 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:31.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.544+0000 7f686f374740 1 --2- 172.21.15.83:0/2083417386 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56408f7a9a20 0x56408f7ad660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:31.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.544+0000 7f686f374740 1 --2- 172.21.15.83:0/2083417386 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56408f79f350 0x56408f7ad120 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:31.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.544+0000 7f686f374740 1 --2- 172.21.15.83:0/2083417386 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56408f79ea30 0x56408f7b1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:31.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.544+0000 7f6847400640 0 client.18180 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:58:31.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.544+0000 7f686f374740 1 -- 172.21.15.83:0/2083417386 >> 172.21.15.83:0/2083417386 conn(0x56408f79ad50 msgr2=0x56408f7a0180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:58:31.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.544+0000 7f686f374740 1 -- 172.21.15.83:0/2083417386 shutdown_connections 2024-04-24T08:58:31.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.544+0000 7f686f374740 1 -- 172.21.15.83:0/2083417386 wait complete. 2024-04-24T08:58:31.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.544+0000 7f686f374740 20 client.18180 trim_cache size 0 max 16384 2024-04-24T08:58:31.546 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T08:58:31.546 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T08:58:31.546 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-04-24T08:58:31.546 INFO:teuthology.orchestra.run.smithi083.stdout: "c6a76872-6bea-4691-b0b6-1772e2aec54a": { 2024-04-24T08:58:31.546 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-04-24T08:58:31.546 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "c6a76872-6bea-4691-b0b6-1772e2aec54a", 2024-04-24T08:58:31.546 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-04-24T08:58:31.546 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T08:58:31.546 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T08:58:31.546 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T08:58:31.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.545+0000 7f686ae00640 1 -- 172.21.15.83:0/2646322311 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f684c0a3d20 msgr2=0x7f684c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:31.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.545+0000 7f686ae00640 1 --2- 172.21.15.83:0/2646322311 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f684c0a3d20 0x7f684c0a61e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f686406a270 tx=0x7f685400c5e0 comp rx=0 tx=0).stop 2024-04-24T08:58:31.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.545+0000 7f686ae00640 1 -- 172.21.15.83:0/2646322311 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6864100c60 msgr2=0x7f68641a0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:31.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.545+0000 7f686ae00640 1 --2- 172.21.15.83:0/2646322311 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6864100c60 0x7f68641a0660 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f685c00eab0 tx=0x7f685c00ef70 comp rx=0 tx=0).stop 2024-04-24T08:58:31.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.545+0000 7f686ae00640 1 -- 172.21.15.83:0/2646322311 shutdown_connections 2024-04-24T08:58:31.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.545+0000 7f686ae00640 1 --2- 172.21.15.83:0/2646322311 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f684c0a3d20 0x7f684c0a61e0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:31.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.545+0000 7f686ae00640 1 --2- 172.21.15.83:0/2646322311 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f686410e810 0x7f686419a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:31.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.545+0000 7f686ae00640 1 --2- 172.21.15.83:0/2646322311 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6864101610 0x7f68641a0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:31.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.545+0000 7f686ae00640 1 --2- 172.21.15.83:0/2646322311 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6864100c60 0x7f68641a0660 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:31.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.545+0000 7f686ae00640 1 -- 172.21.15.83:0/2646322311 >> 172.21.15.83:0/2646322311 conn(0x7f68640fc880 msgr2=0x7f68640fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:58:31.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.545+0000 7f686ae00640 1 -- 172.21.15.83:0/2646322311 shutdown_connections 2024-04-24T08:58:31.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:58:31.545+0000 7f686ae00640 1 -- 172.21.15.83:0/2646322311 wait complete. 2024-04-24T08:58:31.564 INFO:tasks.cephfs.filesystem:scrub status for tag:c6a76872-6bea-4691-b0b6-1772e2aec54a - {'path': '/', 'tag': 'c6a76872-6bea-4691-b0b6-1772e2aec54a', 'options': 'recursive,force'} 2024-04-24T08:58:31.587 INFO:tasks.workunit.client.0.smithi083.stdout: 1 36567 0.63 MB/sec execute 185 sec latency 102.337 ms 2024-04-24T08:58:32.009 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:58:31 smithi189 ceph-mon[31791]: pgmap v213: 129 pgs: 129 active+clean; 47 MiB data, 657 MiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 353 KiB/s wr, 46 op/s 2024-04-24T08:58:32.010 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:58:31 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T08:58:32.010 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:58:31 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T08:58:32.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:58:31 smithi083 ceph-mon[27907]: pgmap v213: 129 pgs: 129 active+clean; 47 MiB data, 657 MiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 353 KiB/s wr, 46 op/s 2024-04-24T08:58:32.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:58:31 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T08:58:32.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:58:31 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T08:58:32.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:58:31 smithi138 ceph-mon[31441]: pgmap v213: 129 pgs: 129 active+clean; 47 MiB data, 657 MiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 353 KiB/s wr, 46 op/s 2024-04-24T08:58:32.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:58:31 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T08:58:32.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:58:31 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T08:58:32.587 INFO:tasks.workunit.client.0.smithi083.stdout: 1 36684 0.63 MB/sec execute 186 sec latency 53.243 ms 2024-04-24T08:58:33.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:58:32 smithi083 ceph-mon[27907]: scrub queued for path: / 2024-04-24T08:58:33.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:58:32 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T08:58:33.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:58:32 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T08:58:33.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:58:32 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T08:58:33.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:58:32 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T08:58:33.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:58:32 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T08:58:33.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:58:32 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T08:58:33.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:58:32 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T08:58:33.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:58:32 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T08:58:33.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:58:32 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T08:58:33.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:58:32 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T08:58:33.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:58:32 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T08:58:33.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:58:32 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T08:58:33.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:58:32 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T08:58:33.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:58:32 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T08:58:33.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:58:32 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T08:58:33.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:58:32 smithi138 ceph-mon[31441]: scrub queued for path: / 2024-04-24T08:58:33.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:58:32 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T08:58:33.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:58:32 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T08:58:33.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:58:32 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T08:58:33.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:58:32 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T08:58:33.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:58:32 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T08:58:33.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:58:32 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T08:58:33.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:58:32 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T08:58:33.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:58:32 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T08:58:33.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:58:32 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T08:58:33.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:58:32 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T08:58:33.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:58:32 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T08:58:33.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:58:32 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T08:58:33.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:58:32 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T08:58:33.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:58:32 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T08:58:33.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:58:32 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T08:58:33.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:58:32 smithi189 ceph-mon[31791]: scrub queued for path: / 2024-04-24T08:58:33.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:58:32 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T08:58:33.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:58:32 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T08:58:33.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:58:32 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T08:58:33.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:58:32 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T08:58:33.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:58:32 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T08:58:33.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:58:32 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T08:58:33.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:58:32 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T08:58:33.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:58:32 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T08:58:33.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:58:32 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T08:58:33.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:58:32 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T08:58:33.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:58:32 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T08:58:33.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:58:32 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T08:58:33.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:58:32 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T08:58:33.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:58:32 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T08:58:33.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:58:32 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T08:58:33.587 INFO:tasks.workunit.client.0.smithi083.stdout: 1 36792 0.63 MB/sec execute 187 sec latency 47.991 ms 2024-04-24T08:58:34.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:58:33 smithi083 ceph-mon[27907]: pgmap v214: 129 pgs: 129 active+clean; 45 MiB data, 659 MiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 572 KiB/s wr, 84 op/s 2024-04-24T08:58:34.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:58:33 smithi138 ceph-mon[31441]: pgmap v214: 129 pgs: 129 active+clean; 45 MiB data, 659 MiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 572 KiB/s wr, 84 op/s 2024-04-24T08:58:34.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:58:33 smithi189 ceph-mon[31791]: pgmap v214: 129 pgs: 129 active+clean; 45 MiB data, 659 MiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 572 KiB/s wr, 84 op/s 2024-04-24T08:58:34.587 INFO:tasks.workunit.client.0.smithi083.stdout: 1 36956 0.64 MB/sec execute 188 sec latency 54.585 ms 2024-04-24T08:58:35.588 INFO:tasks.workunit.client.0.smithi083.stdout: 1 37042 0.63 MB/sec execute 189 sec latency 64.787 ms 2024-04-24T08:58:36.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:58:35 smithi083 ceph-mon[27907]: pgmap v215: 129 pgs: 129 active+clean; 45 MiB data, 659 MiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 340 KiB/s wr, 64 op/s 2024-04-24T08:58:36.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:58:35 smithi138 ceph-mon[31441]: pgmap v215: 129 pgs: 129 active+clean; 45 MiB data, 659 MiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 340 KiB/s wr, 64 op/s 2024-04-24T08:58:36.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:58:35 smithi189 ceph-mon[31791]: pgmap v215: 129 pgs: 129 active+clean; 45 MiB data, 659 MiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 340 KiB/s wr, 64 op/s 2024-04-24T08:58:36.588 INFO:tasks.workunit.client.0.smithi083.stdout: 1 37155 0.63 MB/sec execute 190 sec latency 68.642 ms 2024-04-24T08:58:37.588 INFO:tasks.workunit.client.0.smithi083.stdout: 1 37284 0.63 MB/sec execute 191 sec latency 68.544 ms 2024-04-24T08:58:38.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:58:37 smithi083 ceph-mon[27907]: pgmap v216: 129 pgs: 129 active+clean; 45 MiB data, 659 MiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 340 KiB/s wr, 64 op/s 2024-04-24T08:58:38.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:58:37 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T08:58:38.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:58:37 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T08:58:38.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:58:37 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T08:58:38.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:58:37 smithi138 ceph-mon[31441]: pgmap v216: 129 pgs: 129 active+clean; 45 MiB data, 659 MiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 340 KiB/s wr, 64 op/s 2024-04-24T08:58:38.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:58:37 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T08:58:38.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:58:37 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T08:58:38.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:58:37 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T08:58:38.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:58:37 smithi189 ceph-mon[31791]: pgmap v216: 129 pgs: 129 active+clean; 45 MiB data, 659 MiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 340 KiB/s wr, 64 op/s 2024-04-24T08:58:38.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:58:37 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T08:58:38.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:58:37 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T08:58:38.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:58:37 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T08:58:38.588 INFO:tasks.workunit.client.0.smithi083.stdout: 1 37382 0.63 MB/sec execute 192 sec latency 77.726 ms 2024-04-24T08:58:39.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:58:38 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T08:58:39.106 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:58:38 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T08:58:39.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:58:38 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T08:58:39.440 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.435+0000 7fa1a9400640 1 -- 172.21.15.83:0/3118686602 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1a4109540 msgr2=0x7fa1a4111110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:39.440 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.435+0000 7fa1a9400640 1 --2- 172.21.15.83:0/3118686602 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1a4109540 0x7fa1a4111110 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7fa19c009ca0 tx=0x7fa19c02a8c0 comp rx=0 tx=0).stop 2024-04-24T08:58:39.440 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.435+0000 7fa1a9400640 1 -- 172.21.15.83:0/3118686602 shutdown_connections 2024-04-24T08:58:39.440 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.435+0000 7fa1a9400640 1 --2- 172.21.15.83:0/3118686602 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1a4109540 0x7fa1a4111110 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:39.440 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.435+0000 7fa1a9400640 1 --2- 172.21.15.83:0/3118686602 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa1a4104e20 0x7fa1a4108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:39.440 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.435+0000 7fa1a9400640 1 --2- 172.21.15.83:0/3118686602 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa1a4104470 0x7fa1a4104850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:39.440 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.435+0000 7fa1a9400640 1 -- 172.21.15.83:0/3118686602 >> 172.21.15.83:0/3118686602 conn(0x7fa1a40ffcd0 msgr2=0x7fa1a41020f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:58:39.440 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.436+0000 7fa1a9400640 1 -- 172.21.15.83:0/3118686602 shutdown_connections 2024-04-24T08:58:39.440 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.436+0000 7fa1a9400640 1 -- 172.21.15.83:0/3118686602 wait complete. 2024-04-24T08:58:39.440 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.437+0000 7fa1a9400640 1 Processor -- start 2024-04-24T08:58:39.441 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.437+0000 7fa1a9400640 1 -- start start 2024-04-24T08:58:39.441 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.437+0000 7fa1a9400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa1a4104470 0x7fa1a41a05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:39.441 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.437+0000 7fa1a9400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1a4109540 0x7fa1a41a0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:39.441 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.437+0000 7fa1a9400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa1a419a6c0 0x7fa1a419ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:39.441 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.437+0000 7fa1a9400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a4114500 con 0x7fa1a4109540 2024-04-24T08:58:39.441 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.437+0000 7fa1a9400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a4114380 con 0x7fa1a4104470 2024-04-24T08:58:39.441 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.437+0000 7fa1a9400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a4114680 con 0x7fa1a419a6c0 2024-04-24T08:58:39.441 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.437+0000 7fa199e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1a4109540 0x7fa1a41a0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:39.441 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.438+0000 7fa199e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1a4109540 0x7fa1a41a0af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42474/0 (socket says 172.21.15.83:42474) 2024-04-24T08:58:39.441 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.438+0000 7fa199e00640 1 -- 172.21.15.83:0/3141183158 learned_addr learned my addr 172.21.15.83:0/3141183158 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:58:39.441 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.438+0000 7fa1a3200640 1 --2- 172.21.15.83:0/3141183158 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa1a419a6c0 0x7fa1a419ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:39.441 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.438+0000 7fa1a2800640 1 --2- 172.21.15.83:0/3141183158 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa1a4104470 0x7fa1a41a05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:39.441 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.438+0000 7fa199e00640 1 -- 172.21.15.83:0/3141183158 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa1a419a6c0 msgr2=0x7fa1a419ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:39.441 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.438+0000 7fa199e00640 1 --2- 172.21.15.83:0/3141183158 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa1a419a6c0 0x7fa1a419ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:39.442 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.438+0000 7fa199e00640 1 -- 172.21.15.83:0/3141183158 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa1a4104470 msgr2=0x7fa1a41a05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:39.442 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.438+0000 7fa199e00640 1 --2- 172.21.15.83:0/3141183158 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa1a4104470 0x7fa1a41a05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:39.442 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.438+0000 7fa199e00640 1 -- 172.21.15.83:0/3141183158 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1a419b2c0 con 0x7fa1a4109540 2024-04-24T08:58:39.442 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.438+0000 7fa199e00640 1 --2- 172.21.15.83:0/3141183158 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1a4109540 0x7fa1a41a0af0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fa18c00c970 tx=0x7fa18c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:58:39.442 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.439+0000 7fa19be00640 1 -- 172.21.15.83:0/3141183158 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa18c007bb0 con 0x7fa1a4109540 2024-04-24T08:58:39.442 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.439+0000 7fa19be00640 1 -- 172.21.15.83:0/3141183158 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa18c007d50 con 0x7fa1a4109540 2024-04-24T08:58:39.442 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.439+0000 7fa1a9400640 1 -- 172.21.15.83:0/3141183158 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1a41a7350 con 0x7fa1a4109540 2024-04-24T08:58:39.442 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.439+0000 7fa19be00640 1 -- 172.21.15.83:0/3141183158 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa18c018730 con 0x7fa1a4109540 2024-04-24T08:58:39.442 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.439+0000 7fa1a9400640 1 -- 172.21.15.83:0/3141183158 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1a41a7700 con 0x7fa1a4109540 2024-04-24T08:58:39.444 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.442+0000 7fa1ac24e740 1 Processor -- start 2024-04-24T08:58:39.444 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.442+0000 7fa1ac24e740 1 -- start start 2024-04-24T08:58:39.444 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.443+0000 7fa19be00640 1 -- 172.21.15.83:0/3141183158 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa18c00f450 con 0x7fa1a4109540 2024-04-24T08:58:39.445 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.443+0000 7fa1ac24e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe8c6f11b0 0x55fe8c6f1590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:39.445 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.443+0000 7fa1ac24e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fe8c6f1ad0 0x55fe8c6fbe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:39.445 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.443+0000 7fa1ac24e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe8c6fc390 0x55fe8c6fe780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:39.445 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.443+0000 7fa1ac24e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55fe8c657900 con 0x55fe8c6fc390 2024-04-24T08:58:39.445 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.443+0000 7fa1ac24e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55fe8c5fdf20 con 0x55fe8c6f11b0 2024-04-24T08:58:39.445 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.443+0000 7fa1ac24e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55fe8c6538c0 con 0x55fe8c6f1ad0 2024-04-24T08:58:39.445 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.443+0000 7fa199e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe8c6fc390 0x55fe8c6fe780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:39.445 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.443+0000 7fa199e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe8c6fc390 0x55fe8c6fe780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42488/0 (socket says 172.21.15.83:42488) 2024-04-24T08:58:39.445 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.443+0000 7fa199e00640 1 -- 172.21.15.83:0/2876564973 learned_addr learned my addr 172.21.15.83:0/2876564973 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:58:39.445 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.443+0000 7fa1a2800640 1 --2- 172.21.15.83:0/2876564973 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fe8c6f1ad0 0x55fe8c6fbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:39.445 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.443+0000 7fa1a3200640 1 --2- 172.21.15.83:0/2876564973 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe8c6f11b0 0x55fe8c6f1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:39.445 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.443+0000 7fa199e00640 1 -- 172.21.15.83:0/2876564973 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fe8c6f1ad0 msgr2=0x55fe8c6fbe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:58:39.446 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.443+0000 7fa199e00640 1 --2- 172.21.15.83:0/2876564973 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fe8c6f1ad0 0x55fe8c6fbe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:39.446 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.444+0000 7fa199e00640 1 -- 172.21.15.83:0/2876564973 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe8c6f11b0 msgr2=0x55fe8c6f1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:58:39.446 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.444+0000 7fa199e00640 1 --2- 172.21.15.83:0/2876564973 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe8c6f11b0 0x55fe8c6f1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:39.446 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.444+0000 7fa199e00640 1 -- 172.21.15.83:0/2876564973 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe8c622e10 con 0x55fe8c6fc390 2024-04-24T08:58:39.446 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.444+0000 7fa199e00640 1 --2- 172.21.15.83:0/2876564973 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe8c6fc390 0x55fe8c6fe780 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fa18c0cdea0 tx=0x7fa18c0d9740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6325bf7cfa0f3b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:58:39.446 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.444+0000 7fa193400640 1 -- 172.21.15.83:0/2876564973 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa18c0db860 con 0x55fe8c6fc390 2024-04-24T08:58:39.446 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.444+0000 7fa193400640 1 -- 172.21.15.83:0/2876564973 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa18c0d9e50 con 0x55fe8c6fc390 2024-04-24T08:58:39.446 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.444+0000 7fa19be00640 1 --2- 172.21.15.83:0/3141183158 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa1840a3dc0 0x7fa1840a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:39.446 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.444+0000 7fa193400640 1 -- 172.21.15.83:0/2876564973 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa18c0e3d50 con 0x55fe8c6fc390 2024-04-24T08:58:39.446 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.445+0000 7fa19be00640 1 -- 172.21.15.83:0/3141183158 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9567+0+0 (secure 0 0 0) 0x7fa18c0d0c80 con 0x7fa1a4109540 2024-04-24T08:58:39.450 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.445+0000 7fa1a3200640 1 --2- 172.21.15.83:0/3141183158 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa1840a3dc0 0x7fa1840a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:39.450 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.445+0000 7fa1ac24e740 1 -- 172.21.15.83:0/2876564973 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe8c6fc390 msgr2=0x55fe8c6fe780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:39.450 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.445+0000 7fa1ac24e740 1 --2- 172.21.15.83:0/2876564973 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe8c6fc390 0x55fe8c6fe780 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fa18c0cdea0 tx=0x7fa18c0d9740 comp rx=0 tx=0).stop 2024-04-24T08:58:39.450 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.446+0000 7fa1ac24e740 1 -- 172.21.15.83:0/2876564973 shutdown_connections 2024-04-24T08:58:39.450 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.446+0000 7fa1ac24e740 1 --2- 172.21.15.83:0/2876564973 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe8c6fc390 0x55fe8c6fe780 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:39.450 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.446+0000 7fa1ac24e740 1 --2- 172.21.15.83:0/2876564973 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fe8c6f1ad0 0x55fe8c6fbe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:39.451 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.446+0000 7fa1ac24e740 1 --2- 172.21.15.83:0/2876564973 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe8c6f11b0 0x55fe8c6f1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:39.451 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.446+0000 7fa1ac24e740 1 -- 172.21.15.83:0/2876564973 >> 172.21.15.83:0/2876564973 conn(0x55fe8c6ed6c0 msgr2=0x55fe8c6edaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:58:39.451 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.446+0000 7fa1ac24e740 1 -- 172.21.15.83:0/2876564973 shutdown_connections 2024-04-24T08:58:39.451 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.446+0000 7fa1a3200640 1 --2- 172.21.15.83:0/3141183158 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa1840a3dc0 0x7fa1840a6280 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa19c02f3a0 tx=0x7fa19c02f310 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:58:39.451 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.446+0000 7fa1ac24e740 1 -- 172.21.15.83:0/2876564973 wait complete. 2024-04-24T08:58:39.451 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.446+0000 7fa1ac24e740 1 -- start start 2024-04-24T08:58:39.451 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.446+0000 7fa1ac24e740 1 Processor -- start 2024-04-24T08:58:39.451 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.447+0000 7fa1ac24e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fe8c6f11b0 0x55fe8c6f9360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:39.451 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.447+0000 7fa1ac24e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe8c6f1ad0 0x55fe8c6f98a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:39.451 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.447+0000 7fa1ac24e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe8c6fc390 0x55fe8c6f9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:39.451 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.447+0000 7fa1ac24e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55fe8c6fefc0 con 0x55fe8c6fc390 2024-04-24T08:58:39.451 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.447+0000 7fa1ac24e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55fe8c6112c0 con 0x55fe8c6f1ad0 2024-04-24T08:58:39.451 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.447+0000 7fa1ac24e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55fe8c6ff140 con 0x55fe8c6f11b0 2024-04-24T08:58:39.452 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.447+0000 7fa1ac24e740 10 client.0 _ll_register_callbacks cb 0x55fe8c369a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:58:39.452 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.447+0000 7fa1a3200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe8c6fc390 0x55fe8c6f9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:39.452 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.447+0000 7fa1a3200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe8c6fc390 0x55fe8c6f9de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42492/0 (socket says 172.21.15.83:42492) 2024-04-24T08:58:39.452 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.447+0000 7fa1a3200640 1 -- 172.21.15.83:0/1417634597 learned_addr learned my addr 172.21.15.83:0/1417634597 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:58:39.452 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.447+0000 7fa1a2800640 1 --2- 172.21.15.83:0/1417634597 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fe8c6f11b0 0x55fe8c6f9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:39.452 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.447+0000 7fa199e00640 1 --2- 172.21.15.83:0/1417634597 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe8c6f1ad0 0x55fe8c6f98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:39.452 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.448+0000 7fa1a3200640 1 -- 172.21.15.83:0/1417634597 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fe8c6f11b0 msgr2=0x55fe8c6f9360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:58:39.452 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.448+0000 7fa1a3200640 1 --2- 172.21.15.83:0/1417634597 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fe8c6f11b0 0x55fe8c6f9360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:39.452 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.448+0000 7fa1a3200640 1 -- 172.21.15.83:0/1417634597 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe8c6f1ad0 msgr2=0x55fe8c6f98a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:58:39.452 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.448+0000 7fa1a3200640 1 --2- 172.21.15.83:0/1417634597 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe8c6f1ad0 0x55fe8c6f98a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:39.452 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.448+0000 7fa1a3200640 1 -- 172.21.15.83:0/1417634597 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe8c621fd0 con 0x55fe8c6fc390 2024-04-24T08:58:39.452 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.448+0000 7fa1a3200640 1 --2- 172.21.15.83:0/1417634597 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe8c6fc390 0x55fe8c6f9de0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7fa19c009430 tx=0x7fa19c0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=221b7ba16c35c5d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:58:39.453 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.448+0000 7fa18be00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:58:39.453 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.448+0000 7fa18be00640 1 -- 172.21.15.83:0/1417634597 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa19c04f070 con 0x55fe8c6fc390 2024-04-24T08:58:39.453 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.448+0000 7fa18be00640 1 -- 172.21.15.83:0/1417634597 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9567+0+0 (secure 0 0 0) 0x7fa19c004450 con 0x55fe8c6fc390 2024-04-24T08:58:39.453 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.448+0000 7fa1ac24e740 1 -- 172.21.15.83:0/1417634597 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa18c0d9e50 con 0x55fe8c6fc390 2024-04-24T08:58:39.453 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.449+0000 7fa18be00640 1 -- 172.21.15.83:0/1417634597 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fa19c004650 con 0x55fe8c6fc390 2024-04-24T08:58:39.453 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.449+0000 7fa1ac24e740 10 client.18192 fetch_fsmap learned FSMap version 33 2024-04-24T08:58:39.453 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.449+0000 7fa1ac24e740 1 -- 172.21.15.83:0/1417634597 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55fe8c36a210 con 0x55fe8c6fc390 2024-04-24T08:58:39.453 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.449+0000 7fa18be00640 1 -- 172.21.15.83:0/1417634597 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa18c0d9e50 con 0x55fe8c6fc390 2024-04-24T08:58:39.453 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.449+0000 7fa18be00640 1 -- 172.21.15.83:0/1417634597 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fa19c044040 con 0x55fe8c6fc390 2024-04-24T08:58:39.453 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.449+0000 7fa1ac24e740 10 client.18192 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T08:58:39.453 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.449+0000 7fa1ac24e740 10 client.18192 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T08:58:39.453 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.449+0000 7fa1ac24e740 20 client.18192 populate_metadata read hostname 'smithi083' 2024-04-24T08:58:39.453 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.449+0000 7fa1ac24e740 1 --2- 172.21.15.83:0/1417634597 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55fe8c70c560 0x55fe8c70e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:39.454 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.449+0000 7fa1ac24e740 4 client.18192 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:58:39.454 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.449+0000 7fa1ac24e740 1 -- 172.21.15.83:0/1417634597 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fe8c665320 con 0x55fe8c70c560 2024-04-24T08:58:39.454 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.450+0000 7fa1a2800640 1 --2- 172.21.15.83:0/1417634597 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55fe8c70c560 0x55fe8c70e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:39.455 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.451+0000 7fa1a2800640 1 --2- 172.21.15.83:0/1417634597 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55fe8c70c560 0x55fe8c70e990 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0ead48f0eaa051f server_cookie=d240b5216367e857 in_seq=0 out_seq=0 2024-04-24T08:58:39.455 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.451+0000 7fa18be00640 10 client.18192 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:58:39.455 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.453+0000 7fa18be00640 1 -- 172.21.15.83:0/1417634597 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fe8c665320 con 0x55fe8c70c560 2024-04-24T08:58:39.455 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.453+0000 7fa18be00640 10 client.18192 handle_command_reply: tid=0 2024-04-24T08:58:39.455 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.453+0000 7fa1ac24e740 1 client.18192 shutdown 2024-04-24T08:58:39.455 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.453+0000 7fa1ac24e740 1 -- 172.21.15.83:0/1417634597 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe8c6fc390 msgr2=0x55fe8c6f9de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:39.455 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.453+0000 7fa1ac24e740 1 --2- 172.21.15.83:0/1417634597 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe8c6fc390 0x55fe8c6f9de0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7fa19c009430 tx=0x7fa19c0095c0 comp rx=0 tx=0).stop 2024-04-24T08:58:39.455 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.454+0000 7fa1ac24e740 1 -- 172.21.15.83:0/1417634597 shutdown_connections 2024-04-24T08:58:39.456 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.454+0000 7fa1ac24e740 1 --2- 172.21.15.83:0/1417634597 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55fe8c70c560 0x55fe8c70e990 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:39.456 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.454+0000 7fa1ac24e740 1 --2- 172.21.15.83:0/1417634597 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe8c6fc390 0x55fe8c6f9de0 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:39.456 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.454+0000 7fa1ac24e740 1 --2- 172.21.15.83:0/1417634597 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe8c6f1ad0 0x55fe8c6f98a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:39.456 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.454+0000 7fa1ac24e740 1 --2- 172.21.15.83:0/1417634597 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fe8c6f11b0 0x55fe8c6f9360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:39.456 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.454+0000 7fa18be00640 0 client.18192 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:58:39.456 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.454+0000 7fa1ac24e740 1 -- 172.21.15.83:0/1417634597 >> 172.21.15.83:0/1417634597 conn(0x55fe8c6ed6c0 msgr2=0x55fe8c6f33f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:58:39.456 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.454+0000 7fa1ac24e740 1 -- 172.21.15.83:0/1417634597 shutdown_connections 2024-04-24T08:58:39.456 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.454+0000 7fa1ac24e740 1 -- 172.21.15.83:0/1417634597 wait complete. 2024-04-24T08:58:39.456 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.454+0000 7fa1ac24e740 20 client.18192 trim_cache size 0 max 16384 2024-04-24T08:58:39.472 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.472+0000 7fa1ac24e740 1 Processor -- start 2024-04-24T08:58:39.473 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.472+0000 7fa1ac24e740 1 -- start start 2024-04-24T08:58:39.473 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.472+0000 7fa1ac24e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe8c6f11b0 0x55fe8c707e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:39.473 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.472+0000 7fa1ac24e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe8c6f1ad0 0x55fe8c6f3270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:39.473 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.472+0000 7fa1ac24e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fe8c6fc390 0x55fe8c7142a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:39.473 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.472+0000 7fa1ac24e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a41b1660 con 0x55fe8c6f11b0 2024-04-24T08:58:39.473 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.472+0000 7fa1ac24e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a41b31a0 con 0x55fe8c6f1ad0 2024-04-24T08:58:39.473 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.472+0000 7fa1ac24e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a41b1d30 con 0x55fe8c6fc390 2024-04-24T08:58:39.473 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.473+0000 7fa1a2800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe8c6f11b0 0x55fe8c707e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:39.473 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.473+0000 7fa1a2800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe8c6f11b0 0x55fe8c707e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42500/0 (socket says 172.21.15.83:42500) 2024-04-24T08:58:39.473 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.473+0000 7fa1a2800640 1 -- 172.21.15.83:0/3956817168 learned_addr learned my addr 172.21.15.83:0/3956817168 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:58:39.475 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.473+0000 7fa1a3200640 1 --2- 172.21.15.83:0/3956817168 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fe8c6fc390 0x55fe8c7142a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:39.475 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.473+0000 7fa199e00640 1 --2- 172.21.15.83:0/3956817168 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe8c6f1ad0 0x55fe8c6f3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:39.475 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.473+0000 7fa1a2800640 1 -- 172.21.15.83:0/3956817168 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fe8c6fc390 msgr2=0x55fe8c7142a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:58:39.475 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.473+0000 7fa1a2800640 1 --2- 172.21.15.83:0/3956817168 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fe8c6fc390 0x55fe8c7142a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:39.475 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.473+0000 7fa1a2800640 1 -- 172.21.15.83:0/3956817168 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe8c6f1ad0 msgr2=0x55fe8c6f3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:58:39.476 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.473+0000 7fa1a2800640 1 --2- 172.21.15.83:0/3956817168 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe8c6f1ad0 0x55fe8c6f3270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:39.476 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.473+0000 7fa1a2800640 1 -- 172.21.15.83:0/3956817168 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe8c436620 con 0x55fe8c6f11b0 2024-04-24T08:58:39.476 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.473+0000 7fa1a3200640 1 --2- 172.21.15.83:0/3956817168 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fe8c6fc390 0x55fe8c7142a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:58:39.476 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.473+0000 7fa199e00640 1 --2- 172.21.15.83:0/3956817168 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe8c6f1ad0 0x55fe8c6f3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:58:39.476 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.474+0000 7fa1a2800640 1 --2- 172.21.15.83:0/3956817168 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe8c6f11b0 0x55fe8c707e80 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x55fe8c70f240 tx=0x7fa194005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89b2aa34bac06976 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:58:39.476 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.474+0000 7fa192a00640 1 -- 172.21.15.83:0/3956817168 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa194013a30 con 0x55fe8c6f11b0 2024-04-24T08:58:39.476 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.474+0000 7fa192a00640 1 -- 172.21.15.83:0/3956817168 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa19401a7e0 con 0x55fe8c6f11b0 2024-04-24T08:58:39.476 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.474+0000 7fa192a00640 1 -- 172.21.15.83:0/3956817168 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa194011900 con 0x55fe8c6f11b0 2024-04-24T08:58:39.476 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.475+0000 7fa1ac24e740 1 -- 172.21.15.83:0/3956817168 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe8c6f11b0 msgr2=0x55fe8c707e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:39.476 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.475+0000 7fa1ac24e740 1 --2- 172.21.15.83:0/3956817168 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe8c6f11b0 0x55fe8c707e80 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x55fe8c70f240 tx=0x7fa194005c00 comp rx=0 tx=0).stop 2024-04-24T08:58:39.476 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.475+0000 7fa1ac24e740 1 -- 172.21.15.83:0/3956817168 shutdown_connections 2024-04-24T08:58:39.476 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.475+0000 7fa1ac24e740 1 --2- 172.21.15.83:0/3956817168 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fe8c6fc390 0x55fe8c7142a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:39.476 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.475+0000 7fa1ac24e740 1 --2- 172.21.15.83:0/3956817168 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe8c6f1ad0 0x55fe8c6f3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:39.477 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.475+0000 7fa1ac24e740 1 --2- 172.21.15.83:0/3956817168 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe8c6f11b0 0x55fe8c707e80 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:39.477 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.475+0000 7fa1ac24e740 1 -- 172.21.15.83:0/3956817168 >> 172.21.15.83:0/3956817168 conn(0x55fe8c6ed6c0 msgr2=0x55fe8c6f2b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:58:39.477 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.475+0000 7fa1ac24e740 1 -- 172.21.15.83:0/3956817168 shutdown_connections 2024-04-24T08:58:39.477 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.475+0000 7fa1ac24e740 1 -- 172.21.15.83:0/3956817168 wait complete. 2024-04-24T08:58:39.477 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.475+0000 7fa1ac24e740 1 -- start start 2024-04-24T08:58:39.479 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.476+0000 7fa1ac24e740 1 Processor -- start 2024-04-24T08:58:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.476+0000 7fa1ac24e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fe8c6f11b0 0x55fe8c6ffe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.476+0000 7fa1ac24e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe8c6f1ad0 0x55fe8c7003c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.476+0000 7fa1ac24e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe8c6fc390 0x55fe8c700900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.476+0000 7fa1ac24e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a41b2ad0 con 0x55fe8c6fc390 2024-04-24T08:58:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.476+0000 7fa1ac24e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a41b2400 con 0x55fe8c6f1ad0 2024-04-24T08:58:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.476+0000 7fa1ac24e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a41af640 con 0x55fe8c6f11b0 2024-04-24T08:58:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.476+0000 7fa1ac24e740 10 client.0 _ll_register_callbacks cb 0x7fa19c009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:58:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.477+0000 7fa1a3200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe8c6fc390 0x55fe8c700900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.477+0000 7fa1a3200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe8c6fc390 0x55fe8c700900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42504/0 (socket says 172.21.15.83:42504) 2024-04-24T08:58:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.477+0000 7fa1a3200640 1 -- 172.21.15.83:0/3889162478 learned_addr learned my addr 172.21.15.83:0/3889162478 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:58:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.477+0000 7fa1a2800640 1 --2- 172.21.15.83:0/3889162478 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fe8c6f11b0 0x55fe8c6ffe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.477+0000 7fa199e00640 1 --2- 172.21.15.83:0/3889162478 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe8c6f1ad0 0x55fe8c7003c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.477+0000 7fa1a3200640 1 -- 172.21.15.83:0/3889162478 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fe8c6f11b0 msgr2=0x55fe8c6ffe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:58:39.481 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.477+0000 7fa1a3200640 1 --2- 172.21.15.83:0/3889162478 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fe8c6f11b0 0x55fe8c6ffe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:39.481 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.477+0000 7fa1a3200640 1 -- 172.21.15.83:0/3889162478 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe8c6f1ad0 msgr2=0x55fe8c7003c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:58:39.481 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.477+0000 7fa1a3200640 1 --2- 172.21.15.83:0/3889162478 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe8c6f1ad0 0x55fe8c7003c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:39.481 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.477+0000 7fa1a3200640 1 -- 172.21.15.83:0/3889162478 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe8c771be0 con 0x55fe8c6fc390 2024-04-24T08:58:39.481 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.477+0000 7fa1a3200640 1 --2- 172.21.15.83:0/3889162478 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe8c6fc390 0x55fe8c700900 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7fa19c046460 tx=0x7fa19c0048c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51dbf903ba0461b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:58:39.481 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.477+0000 7fa18be00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:58:39.481 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.478+0000 7fa18be00640 1 -- 172.21.15.83:0/3889162478 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa19c04f070 con 0x55fe8c6fc390 2024-04-24T08:58:39.481 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.478+0000 7fa18be00640 1 -- 172.21.15.83:0/3889162478 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9567+0+0 (secure 0 0 0) 0x7fa19c031d00 con 0x55fe8c6fc390 2024-04-24T08:58:39.481 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.478+0000 7fa1ac24e740 1 -- 172.21.15.83:0/3889162478 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa19401a7e0 con 0x55fe8c6fc390 2024-04-24T08:58:39.481 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.478+0000 7fa18be00640 1 -- 172.21.15.83:0/3889162478 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fa19c067050 con 0x55fe8c6fc390 2024-04-24T08:58:39.481 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.478+0000 7fa1ac24e740 10 client.18198 fetch_fsmap learned FSMap version 33 2024-04-24T08:58:39.481 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.478+0000 7fa1ac24e740 1 -- 172.21.15.83:0/3889162478 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55fe8c772e60 con 0x55fe8c6fc390 2024-04-24T08:58:39.481 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.478+0000 7fa18be00640 1 -- 172.21.15.83:0/3889162478 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa19401a7e0 con 0x55fe8c6fc390 2024-04-24T08:58:39.482 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.479+0000 7fa18be00640 1 -- 172.21.15.83:0/3889162478 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fa19c044070 con 0x55fe8c6fc390 2024-04-24T08:58:39.482 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.479+0000 7fa1ac24e740 10 client.18198 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T08:58:39.482 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.479+0000 7fa1ac24e740 10 client.18198 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T08:58:39.482 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.479+0000 7fa1ac24e740 20 client.18198 populate_metadata read hostname 'smithi083' 2024-04-24T08:58:39.482 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.479+0000 7fa1ac24e740 1 --2- 172.21.15.83:0/3889162478 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55fe8c7735b0 0x55fe8c7759d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:39.482 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.479+0000 7fa1ac24e740 4 client.18198 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T08:58:39.482 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.479+0000 7fa1ac24e740 1 -- 172.21.15.83:0/3889162478 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fe8c65f190 con 0x55fe8c7735b0 2024-04-24T08:58:39.482 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.479+0000 7fa1a2800640 1 --2- 172.21.15.83:0/3889162478 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55fe8c7735b0 0x55fe8c7759d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:39.482 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.480+0000 7fa1a2800640 1 --2- 172.21.15.83:0/3889162478 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55fe8c7735b0 0x55fe8c7759d0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=227cdeabde0ac6bf server_cookie=3ff327ed69b8d80c in_seq=0 out_seq=0 2024-04-24T08:58:39.483 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.480+0000 7fa18be00640 10 client.18198 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:58:39.525 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.524+0000 7fa18be00640 1 -- 172.21.15.83:0/3889162478 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fe8c65f190 con 0x55fe8c7735b0 2024-04-24T08:58:39.525 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.524+0000 7fa18be00640 10 client.18198 handle_command_reply: tid=0 2024-04-24T08:58:39.525 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.524+0000 7fa1ac24e740 1 client.18198 shutdown 2024-04-24T08:58:39.525 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.524+0000 7fa1ac24e740 1 -- 172.21.15.83:0/3889162478 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe8c6fc390 msgr2=0x55fe8c700900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:39.525 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.524+0000 7fa1ac24e740 1 --2- 172.21.15.83:0/3889162478 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe8c6fc390 0x55fe8c700900 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7fa19c046460 tx=0x7fa19c0048c0 comp rx=0 tx=0).stop 2024-04-24T08:58:39.525 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.524+0000 7fa1ac24e740 1 -- 172.21.15.83:0/3889162478 shutdown_connections 2024-04-24T08:58:39.525 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.524+0000 7fa1ac24e740 1 --2- 172.21.15.83:0/3889162478 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55fe8c7735b0 0x55fe8c7759d0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:39.525 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.524+0000 7fa1ac24e740 1 --2- 172.21.15.83:0/3889162478 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe8c6fc390 0x55fe8c700900 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:39.525 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.524+0000 7fa1ac24e740 1 --2- 172.21.15.83:0/3889162478 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fe8c6f1ad0 0x55fe8c7003c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:39.525 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.524+0000 7fa1ac24e740 1 --2- 172.21.15.83:0/3889162478 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fe8c6f11b0 0x55fe8c6ffe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:39.526 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.524+0000 7fa1ac24e740 1 -- 172.21.15.83:0/3889162478 >> 172.21.15.83:0/3889162478 conn(0x55fe8c6ed6c0 msgr2=0x55fe8c6f33f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:58:39.526 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.524+0000 7fa18be00640 0 client.18198 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:58:39.526 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.525+0000 7fa1ac24e740 1 -- 172.21.15.83:0/3889162478 shutdown_connections 2024-04-24T08:58:39.526 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.525+0000 7fa1ac24e740 1 -- 172.21.15.83:0/3889162478 wait complete. 2024-04-24T08:58:39.526 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.525+0000 7fa1ac24e740 20 client.18198 trim_cache size 0 max 16384 2024-04-24T08:58:39.526 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T08:58:39.526 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T08:58:39.526 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T08:58:39.526 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T08:58:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.526+0000 7fa1a9400640 1 -- 172.21.15.83:0/3141183158 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa1840a3dc0 msgr2=0x7fa1840a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.526+0000 7fa1a9400640 1 --2- 172.21.15.83:0/3141183158 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa1840a3dc0 0x7fa1840a6280 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa19c02f3a0 tx=0x7fa19c02f310 comp rx=0 tx=0).stop 2024-04-24T08:58:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.526+0000 7fa1a9400640 1 -- 172.21.15.83:0/3141183158 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1a4109540 msgr2=0x7fa1a41a0af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.526+0000 7fa1a9400640 1 --2- 172.21.15.83:0/3141183158 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1a4109540 0x7fa1a41a0af0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fa18c00c970 tx=0x7fa18c00ce30 comp rx=0 tx=0).stop 2024-04-24T08:58:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.526+0000 7fa1a9400640 1 -- 172.21.15.83:0/3141183158 shutdown_connections 2024-04-24T08:58:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.526+0000 7fa1a9400640 1 --2- 172.21.15.83:0/3141183158 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa1840a3dc0 0x7fa1840a6280 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.526+0000 7fa1a9400640 1 --2- 172.21.15.83:0/3141183158 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa1a419a6c0 0x7fa1a419ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.526+0000 7fa1a9400640 1 --2- 172.21.15.83:0/3141183158 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1a4109540 0x7fa1a41a0af0 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.526+0000 7fa1a9400640 1 --2- 172.21.15.83:0/3141183158 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa1a4104470 0x7fa1a41a05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.526+0000 7fa1a9400640 1 -- 172.21.15.83:0/3141183158 >> 172.21.15.83:0/3141183158 conn(0x7fa1a40ffcd0 msgr2=0x7fa1a4100670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:58:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.526+0000 7fa1a9400640 1 -- 172.21.15.83:0/3141183158 shutdown_connections 2024-04-24T08:58:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:39.526+0000 7fa1a9400640 1 -- 172.21.15.83:0/3141183158 wait complete. 2024-04-24T08:58:39.588 INFO:tasks.workunit.client.0.smithi083.stdout: 1 37553 0.62 MB/sec execute 193 sec latency 67.529 ms 2024-04-24T08:58:40.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:58:39 smithi083 ceph-mon[27907]: pgmap v217: 129 pgs: 129 active+clean; 46 MiB data, 668 MiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 401 KiB/s wr, 83 op/s 2024-04-24T08:58:40.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:58:39 smithi138 ceph-mon[31441]: pgmap v217: 129 pgs: 129 active+clean; 46 MiB data, 668 MiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 401 KiB/s wr, 83 op/s 2024-04-24T08:58:40.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:58:39 smithi189 ceph-mon[31791]: pgmap v217: 129 pgs: 129 active+clean; 46 MiB data, 668 MiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 401 KiB/s wr, 83 op/s 2024-04-24T08:58:40.588 INFO:tasks.workunit.client.0.smithi083.stdout: 1 37665 0.62 MB/sec execute 194 sec latency 72.826 ms 2024-04-24T08:58:41.588 INFO:tasks.workunit.client.0.smithi083.stdout: 1 37815 0.62 MB/sec execute 195 sec latency 54.175 ms 2024-04-24T08:58:42.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:58:41 smithi083 ceph-mon[27907]: pgmap v218: 129 pgs: 129 active+clean; 46 MiB data, 668 MiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 280 KiB/s wr, 58 op/s 2024-04-24T08:58:42.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:58:41 smithi138 ceph-mon[31441]: pgmap v218: 129 pgs: 129 active+clean; 46 MiB data, 668 MiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 280 KiB/s wr, 58 op/s 2024-04-24T08:58:42.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:58:41 smithi189 ceph-mon[31791]: pgmap v218: 129 pgs: 129 active+clean; 46 MiB data, 668 MiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 280 KiB/s wr, 58 op/s 2024-04-24T08:58:42.589 INFO:tasks.workunit.client.0.smithi083.stdout: 1 37898 0.62 MB/sec execute 196 sec latency 57.476 ms 2024-04-24T08:58:43.589 INFO:tasks.workunit.client.0.smithi083.stdout: 1 37982 0.62 MB/sec execute 197 sec latency 52.222 ms 2024-04-24T08:58:44.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:58:43 smithi083 ceph-mon[27907]: pgmap v219: 129 pgs: 129 active+clean; 43 MiB data, 662 MiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 434 KiB/s wr, 79 op/s 2024-04-24T08:58:44.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:58:43 smithi138 ceph-mon[31441]: pgmap v219: 129 pgs: 129 active+clean; 43 MiB data, 662 MiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 434 KiB/s wr, 79 op/s 2024-04-24T08:58:44.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:58:43 smithi189 ceph-mon[31791]: pgmap v219: 129 pgs: 129 active+clean; 43 MiB data, 662 MiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 434 KiB/s wr, 79 op/s 2024-04-24T08:58:44.589 INFO:tasks.workunit.client.0.smithi083.stdout: 1 38101 0.62 MB/sec execute 198 sec latency 56.878 ms 2024-04-24T08:58:45.589 INFO:tasks.workunit.client.0.smithi083.stdout: 1 38285 0.63 MB/sec execute 199 sec latency 54.482 ms 2024-04-24T08:58:46.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:58:45 smithi083 ceph-mon[27907]: pgmap v220: 129 pgs: 129 active+clean; 43 MiB data, 662 MiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 215 KiB/s wr, 41 op/s 2024-04-24T08:58:46.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:58:45 smithi138 ceph-mon[31441]: pgmap v220: 129 pgs: 129 active+clean; 43 MiB data, 662 MiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 215 KiB/s wr, 41 op/s 2024-04-24T08:58:46.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:58:45 smithi189 ceph-mon[31791]: pgmap v220: 129 pgs: 129 active+clean; 43 MiB data, 662 MiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 215 KiB/s wr, 41 op/s 2024-04-24T08:58:46.589 INFO:tasks.workunit.client.0.smithi083.stdout: 1 38479 0.64 MB/sec execute 200 sec latency 58.351 ms 2024-04-24T08:58:47.589 INFO:tasks.workunit.client.0.smithi083.stdout: 1 38703 0.64 MB/sec execute 201 sec latency 53.330 ms 2024-04-24T08:58:48.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:58:47 smithi083 ceph-mon[27907]: pgmap v221: 129 pgs: 129 active+clean; 43 MiB data, 662 MiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 215 KiB/s wr, 41 op/s 2024-04-24T08:58:48.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:58:47 smithi138 ceph-mon[31441]: pgmap v221: 129 pgs: 129 active+clean; 43 MiB data, 662 MiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 215 KiB/s wr, 41 op/s 2024-04-24T08:58:48.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:58:47 smithi189 ceph-mon[31791]: pgmap v221: 129 pgs: 129 active+clean; 43 MiB data, 662 MiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 215 KiB/s wr, 41 op/s 2024-04-24T08:58:48.589 INFO:tasks.workunit.client.0.smithi083.stdout: 1 38799 0.64 MB/sec execute 202 sec latency 58.509 ms 2024-04-24T08:58:49.589 INFO:tasks.workunit.client.0.smithi083.stdout: 1 38828 0.64 MB/sec execute 203 sec latency 50.853 ms 2024-04-24T08:58:50.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:58:49 smithi138 ceph-mon[31441]: pgmap v222: 129 pgs: 129 active+clean; 45 MiB data, 668 MiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 469 KiB/s wr, 62 op/s 2024-04-24T08:58:50.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:58:49 smithi189 ceph-mon[31791]: pgmap v222: 129 pgs: 129 active+clean; 45 MiB data, 668 MiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 469 KiB/s wr, 62 op/s 2024-04-24T08:58:50.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:58:49 smithi083 ceph-mon[27907]: pgmap v222: 129 pgs: 129 active+clean; 45 MiB data, 668 MiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 469 KiB/s wr, 62 op/s 2024-04-24T08:58:50.589 INFO:tasks.workunit.client.0.smithi083.stdout: 1 38882 0.63 MB/sec execute 204 sec latency 47.914 ms 2024-04-24T08:58:51.589 INFO:tasks.workunit.client.0.smithi083.stdout: 1 39063 0.64 MB/sec execute 205 sec latency 54.119 ms 2024-04-24T08:58:52.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:58:51 smithi138 ceph-mon[31441]: pgmap v223: 129 pgs: 129 active+clean; 45 MiB data, 668 MiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 408 KiB/s wr, 43 op/s 2024-04-24T08:58:52.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:58:51 smithi189 ceph-mon[31791]: pgmap v223: 129 pgs: 129 active+clean; 45 MiB data, 668 MiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 408 KiB/s wr, 43 op/s 2024-04-24T08:58:52.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:58:51 smithi083 ceph-mon[27907]: pgmap v223: 129 pgs: 129 active+clean; 45 MiB data, 668 MiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 408 KiB/s wr, 43 op/s 2024-04-24T08:58:52.590 INFO:tasks.workunit.client.0.smithi083.stdout: 1 39184 0.64 MB/sec execute 206 sec latency 51.258 ms 2024-04-24T08:58:52.680 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.680+0000 7f479f200640 1 -- 172.21.15.83:0/909216729 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4798107350 msgr2=0x7f479810ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:52.680 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.680+0000 7f479f200640 1 --2- 172.21.15.83:0/909216729 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4798107350 0x7f479810ef20 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f4790009ca0 tx=0x7f479002a8c0 comp rx=0 tx=0).stop 2024-04-24T08:58:52.680 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.681+0000 7f479f200640 1 -- 172.21.15.83:0/909216729 shutdown_connections 2024-04-24T08:58:52.680 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.681+0000 7f479f200640 1 --2- 172.21.15.83:0/909216729 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4798107350 0x7f479810ef20 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:52.680 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.681+0000 7f479f200640 1 --2- 172.21.15.83:0/909216729 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4798102c30 0x7f4798106c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:52.681 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.681+0000 7f479f200640 1 --2- 172.21.15.83:0/909216729 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4798102280 0x7f4798102660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:52.681 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.681+0000 7f479f200640 1 -- 172.21.15.83:0/909216729 >> 172.21.15.83:0/909216729 conn(0x7f4798079490 msgr2=0x7f47980798a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:58:52.681 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.681+0000 7f479f200640 1 -- 172.21.15.83:0/909216729 shutdown_connections 2024-04-24T08:58:52.681 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.681+0000 7f479f200640 1 -- 172.21.15.83:0/909216729 wait complete. 2024-04-24T08:58:52.683 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.682+0000 7f479f200640 1 Processor -- start 2024-04-24T08:58:52.683 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.682+0000 7f479f200640 1 -- start start 2024-04-24T08:58:52.683 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.683+0000 7f479f200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4798102280 0x7f4798074300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:52.683 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.683+0000 7f479f200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4798107350 0x7f4798074840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:52.683 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.683+0000 7f479f200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4798074d80 0x7f47981a6900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:52.683 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.683+0000 7f479f200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4798112310 con 0x7f4798107350 2024-04-24T08:58:52.683 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.683+0000 7f479f200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4798112190 con 0x7f4798074d80 2024-04-24T08:58:52.683 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.683+0000 7f479f200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4798112490 con 0x7f4798102280 2024-04-24T08:58:52.683 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.683+0000 7f478fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4798107350 0x7f4798074840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:52.683 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.683+0000 7f478fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4798107350 0x7f4798074840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51360/0 (socket says 172.21.15.83:51360) 2024-04-24T08:58:52.683 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.683+0000 7f478fe00640 1 -- 172.21.15.83:0/321050243 learned_addr learned my addr 172.21.15.83:0/321050243 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:58:52.683 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.683+0000 7f4797e00640 1 --2- 172.21.15.83:0/321050243 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4798102280 0x7f4798074300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:52.684 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.683+0000 7f479d200640 1 --2- 172.21.15.83:0/321050243 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4798074d80 0x7f47981a6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:52.684 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.684+0000 7f478fe00640 1 -- 172.21.15.83:0/321050243 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4798102280 msgr2=0x7f4798074300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:52.684 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.684+0000 7f478fe00640 1 --2- 172.21.15.83:0/321050243 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4798102280 0x7f4798074300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:52.684 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.684+0000 7f478fe00640 1 -- 172.21.15.83:0/321050243 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4798074d80 msgr2=0x7f47981a6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:52.684 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.684+0000 7f478fe00640 1 --2- 172.21.15.83:0/321050243 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4798074d80 0x7f47981a6900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:52.684 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.684+0000 7f478fe00640 1 -- 172.21.15.83:0/321050243 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47981a6fa0 con 0x7f4798107350 2024-04-24T08:58:52.685 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.684+0000 7f478fe00640 1 --2- 172.21.15.83:0/321050243 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4798107350 0x7f4798074840 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f478000d770 tx=0x7f478000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:58:52.685 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.684+0000 7f4795600640 1 -- 172.21.15.83:0/321050243 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4780017070 con 0x7f4798107350 2024-04-24T08:58:52.685 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.685+0000 7f4795600640 1 -- 172.21.15.83:0/321050243 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4780009c30 con 0x7f4798107350 2024-04-24T08:58:52.685 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.685+0000 7f4795600640 1 -- 172.21.15.83:0/321050243 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f478000f620 con 0x7f4798107350 2024-04-24T08:58:52.685 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.685+0000 7f479f200640 1 -- 172.21.15.83:0/321050243 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47981a7290 con 0x7f4798107350 2024-04-24T08:58:52.685 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.685+0000 7f479f200640 1 -- 172.21.15.83:0/321050243 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47981a7790 con 0x7f4798107350 2024-04-24T08:58:52.687 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.688+0000 7f47a2071740 1 Processor -- start 2024-04-24T08:58:52.687 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.689+0000 7f47a2071740 1 -- start start 2024-04-24T08:58:52.688 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.689+0000 7f4795600640 1 -- 172.21.15.83:0/321050243 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f478001a070 con 0x7f4798107350 2024-04-24T08:58:52.688 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.689+0000 7f47a2071740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fbb8aca0f0 0x55fbb8aca4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:52.688 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.689+0000 7f47a2071740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbb8acaa10 0x55fbb8add660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:52.688 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.689+0000 7f47a2071740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fbb8addba0 0x55fbb8adff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:52.689 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.689+0000 7f47a2071740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55fbb8a30900 con 0x55fbb8acaa10 2024-04-24T08:58:52.689 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.689+0000 7f47a2071740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55fbb89d6f20 con 0x55fbb8aca0f0 2024-04-24T08:58:52.689 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.689+0000 7f47a2071740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55fbb8a2c8c0 con 0x55fbb8addba0 2024-04-24T08:58:52.690 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.690+0000 7f4797e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbb8acaa10 0x55fbb8add660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:52.691 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.690+0000 7f4797e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbb8acaa10 0x55fbb8add660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51362/0 (socket says 172.21.15.83:51362) 2024-04-24T08:58:52.691 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.690+0000 7f4797e00640 1 -- 172.21.15.83:0/279503006 learned_addr learned my addr 172.21.15.83:0/279503006 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:58:52.691 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.690+0000 7f478fe00640 1 --2- 172.21.15.83:0/279503006 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fbb8addba0 0x55fbb8adff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:52.691 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.690+0000 7f4797e00640 1 -- 172.21.15.83:0/279503006 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fbb8addba0 msgr2=0x55fbb8adff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:58:52.691 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.690+0000 7f4797e00640 1 --2- 172.21.15.83:0/279503006 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fbb8addba0 0x55fbb8adff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:52.691 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.690+0000 7f4797e00640 1 -- 172.21.15.83:0/279503006 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fbb8aca0f0 msgr2=0x55fbb8aca4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:58:52.691 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.690+0000 7f4797e00640 1 --2- 172.21.15.83:0/279503006 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fbb8aca0f0 0x55fbb8aca4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:52.691 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.690+0000 7f4797e00640 1 -- 172.21.15.83:0/279503006 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fbb89fbe10 con 0x55fbb8acaa10 2024-04-24T08:58:52.691 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.691+0000 7f4797e00640 1 --2- 172.21.15.83:0/279503006 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbb8acaa10 0x55fbb8add660 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f478800bcf0 tx=0x7f478800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=141a335437d76a33 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:58:52.691 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.691+0000 7f478d600640 1 -- 172.21.15.83:0/279503006 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4788015070 con 0x55fbb8acaa10 2024-04-24T08:58:52.691 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.691+0000 7f4795600640 1 --2- 172.21.15.83:0/321050243 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f477c0a3e40 0x7f477c0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:52.691 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.691+0000 7f478d600640 1 -- 172.21.15.83:0/279503006 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f478801a910 con 0x55fbb8acaa10 2024-04-24T08:58:52.692 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.691+0000 7f478d600640 1 -- 172.21.15.83:0/279503006 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4788011990 con 0x55fbb8acaa10 2024-04-24T08:58:52.692 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.691+0000 7f4795600640 1 -- 172.21.15.83:0/321050243 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9567+0+0 (secure 0 0 0) 0x7f47800d06e0 con 0x7f4798107350 2024-04-24T08:58:52.692 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.691+0000 7f479d200640 1 --2- 172.21.15.83:0/321050243 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f477c0a3e40 0x7f477c0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:52.692 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.692+0000 7f47a2071740 1 -- 172.21.15.83:0/279503006 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbb8acaa10 msgr2=0x55fbb8add660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:52.692 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.692+0000 7f47a2071740 1 --2- 172.21.15.83:0/279503006 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbb8acaa10 0x55fbb8add660 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f478800bcf0 tx=0x7f478800f730 comp rx=0 tx=0).stop 2024-04-24T08:58:52.696 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.692+0000 7f47a2071740 1 -- 172.21.15.83:0/279503006 shutdown_connections 2024-04-24T08:58:52.696 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.692+0000 7f47a2071740 1 --2- 172.21.15.83:0/279503006 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fbb8addba0 0x55fbb8adff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:52.696 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.692+0000 7f47a2071740 1 --2- 172.21.15.83:0/279503006 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbb8acaa10 0x55fbb8add660 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:52.696 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.692+0000 7f47a2071740 1 --2- 172.21.15.83:0/279503006 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fbb8aca0f0 0x55fbb8aca4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:52.696 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.692+0000 7f47a2071740 1 -- 172.21.15.83:0/279503006 >> 172.21.15.83:0/279503006 conn(0x55fbb8ac66c0 msgr2=0x55fbb8ac6aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:58:52.696 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.692+0000 7f479d200640 1 --2- 172.21.15.83:0/321050243 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f477c0a3e40 0x7f477c0a6300 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4790005fe0 tx=0x7f4790005d80 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:58:52.696 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.692+0000 7f47a2071740 1 -- 172.21.15.83:0/279503006 shutdown_connections 2024-04-24T08:58:52.696 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.692+0000 7f47a2071740 1 -- 172.21.15.83:0/279503006 wait complete. 2024-04-24T08:58:52.696 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.693+0000 7f47a2071740 1 -- start start 2024-04-24T08:58:52.696 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.693+0000 7f47a2071740 1 Processor -- start 2024-04-24T08:58:52.696 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.693+0000 7f47a2071740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fbb8aca0f0 0x55fbb8adab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:52.697 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.693+0000 7f47a2071740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbb8acaa10 0x55fbb8adb0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:52.697 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.693+0000 7f47a2071740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fbb8addba0 0x55fbb8adb610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:52.697 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.693+0000 7f47a2071740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55fbb8ae0770 con 0x55fbb8acaa10 2024-04-24T08:58:52.697 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.693+0000 7f47a2071740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55fbb89ea2c0 con 0x55fbb8addba0 2024-04-24T08:58:52.697 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.693+0000 7f47a2071740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55fbb8ae08f0 con 0x55fbb8aca0f0 2024-04-24T08:58:52.697 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.693+0000 7f47a2071740 10 client.0 _ll_register_callbacks cb 0x55fbb8742a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:58:52.697 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.694+0000 7f478fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbb8acaa10 0x55fbb8adb0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:52.697 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.694+0000 7f478fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbb8acaa10 0x55fbb8adb0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51370/0 (socket says 172.21.15.83:51370) 2024-04-24T08:58:52.697 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.694+0000 7f478fe00640 1 -- 172.21.15.83:0/2416098841 learned_addr learned my addr 172.21.15.83:0/2416098841 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:58:52.697 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.694+0000 7f4797e00640 1 --2- 172.21.15.83:0/2416098841 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fbb8aca0f0 0x55fbb8adab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:52.697 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.694+0000 7f479d200640 1 --2- 172.21.15.83:0/2416098841 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fbb8addba0 0x55fbb8adb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:52.697 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.694+0000 7f478fe00640 1 -- 172.21.15.83:0/2416098841 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fbb8aca0f0 msgr2=0x55fbb8adab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:58:52.698 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.694+0000 7f478fe00640 1 --2- 172.21.15.83:0/2416098841 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fbb8aca0f0 0x55fbb8adab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:52.698 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.694+0000 7f478fe00640 1 -- 172.21.15.83:0/2416098841 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fbb8addba0 msgr2=0x55fbb8adb610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:58:52.698 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.694+0000 7f478fe00640 1 --2- 172.21.15.83:0/2416098841 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fbb8addba0 0x55fbb8adb610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:52.698 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.694+0000 7f478fe00640 1 -- 172.21.15.83:0/2416098841 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fbb89fafd0 con 0x55fbb8acaa10 2024-04-24T08:58:52.698 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.694+0000 7f479d200640 1 --2- 172.21.15.83:0/2416098841 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fbb8addba0 0x55fbb8adb610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:58:52.698 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.694+0000 7f478fe00640 1 --2- 172.21.15.83:0/2416098841 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbb8acaa10 0x55fbb8adb0d0 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f478009c1e0 tx=0x7f478009c0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4a287470237c9b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:58:52.698 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.694+0000 7f4786000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:58:52.698 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.695+0000 7f4786000640 1 -- 172.21.15.83:0/2416098841 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4780021070 con 0x55fbb8acaa10 2024-04-24T08:58:52.698 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.695+0000 7f4786000640 1 -- 172.21.15.83:0/2416098841 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9567+0+0 (secure 0 0 0) 0x7f47800d0250 con 0x55fbb8acaa10 2024-04-24T08:58:52.698 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.695+0000 7f47a2071740 1 -- 172.21.15.83:0/2416098841 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f478801a910 con 0x55fbb8acaa10 2024-04-24T08:58:52.698 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.695+0000 7f4786000640 1 -- 172.21.15.83:0/2416098841 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f47800cde80 con 0x55fbb8acaa10 2024-04-24T08:58:52.699 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.695+0000 7f47a2071740 10 client.18210 fetch_fsmap learned FSMap version 33 2024-04-24T08:58:52.699 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.695+0000 7f47a2071740 1 -- 172.21.15.83:0/2416098841 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55fbb8743210 con 0x55fbb8acaa10 2024-04-24T08:58:52.699 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.695+0000 7f4786000640 1 -- 172.21.15.83:0/2416098841 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f478801a910 con 0x55fbb8acaa10 2024-04-24T08:58:52.699 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.695+0000 7f4786000640 1 -- 172.21.15.83:0/2416098841 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f47800a0160 con 0x55fbb8acaa10 2024-04-24T08:58:52.699 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.695+0000 7f47a2071740 10 client.18210 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T08:58:52.699 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.696+0000 7f47a2071740 10 client.18210 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T08:58:52.699 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.696+0000 7f47a2071740 20 client.18210 populate_metadata read hostname 'smithi083' 2024-04-24T08:58:52.699 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.696+0000 7f47a2071740 1 --2- 172.21.15.83:0/2416098841 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55fbb8aedda0 0x55fbb8af01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:52.699 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.696+0000 7f47a2071740 4 client.18210 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:58:52.699 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.696+0000 7f47a2071740 1 -- 172.21.15.83:0/2416098841 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fbb8a3e320 con 0x55fbb8aedda0 2024-04-24T08:58:52.699 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.696+0000 7f4797e00640 1 --2- 172.21.15.83:0/2416098841 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55fbb8aedda0 0x55fbb8af01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:52.701 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.697+0000 7f4797e00640 1 --2- 172.21.15.83:0/2416098841 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55fbb8aedda0 0x55fbb8af01b0 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45a61e252372f3bc server_cookie=bfc0b9160c6ed5e3 in_seq=0 out_seq=0 2024-04-24T08:58:52.701 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.697+0000 7f4786000640 10 client.18210 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:58:52.701 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.699+0000 7f4786000640 1 -- 172.21.15.83:0/2416098841 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fbb8a3e320 con 0x55fbb8aedda0 2024-04-24T08:58:52.701 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.699+0000 7f4786000640 10 client.18210 handle_command_reply: tid=0 2024-04-24T08:58:52.701 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.699+0000 7f47a2071740 1 client.18210 shutdown 2024-04-24T08:58:52.701 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.700+0000 7f47a2071740 1 -- 172.21.15.83:0/2416098841 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbb8acaa10 msgr2=0x55fbb8adb0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:52.701 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.700+0000 7f47a2071740 1 --2- 172.21.15.83:0/2416098841 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbb8acaa10 0x55fbb8adb0d0 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f478009c1e0 tx=0x7f478009c0e0 comp rx=0 tx=0).stop 2024-04-24T08:58:52.701 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.700+0000 7f47a2071740 1 -- 172.21.15.83:0/2416098841 shutdown_connections 2024-04-24T08:58:52.701 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.700+0000 7f47a2071740 1 --2- 172.21.15.83:0/2416098841 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55fbb8aedda0 0x55fbb8af01b0 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:52.702 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.700+0000 7f47a2071740 1 --2- 172.21.15.83:0/2416098841 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fbb8addba0 0x55fbb8adb610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:52.702 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.700+0000 7f47a2071740 1 --2- 172.21.15.83:0/2416098841 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbb8acaa10 0x55fbb8adb0d0 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:52.702 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.700+0000 7f4786000640 0 client.18210 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:58:52.702 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.700+0000 7f47a2071740 1 --2- 172.21.15.83:0/2416098841 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fbb8aca0f0 0x55fbb8adab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:52.702 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.700+0000 7f47a2071740 1 -- 172.21.15.83:0/2416098841 >> 172.21.15.83:0/2416098841 conn(0x55fbb8ac66c0 msgr2=0x55fbb8acc440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:58:52.702 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.700+0000 7f47a2071740 1 -- 172.21.15.83:0/2416098841 shutdown_connections 2024-04-24T08:58:52.702 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.700+0000 7f47a2071740 1 -- 172.21.15.83:0/2416098841 wait complete. 2024-04-24T08:58:52.702 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.700+0000 7f47a2071740 20 client.18210 trim_cache size 0 max 16384 2024-04-24T08:58:52.717 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.718+0000 7f47a2071740 1 Processor -- start 2024-04-24T08:58:52.717 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.718+0000 7f47a2071740 1 -- start start 2024-04-24T08:58:52.717 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.718+0000 7f47a2071740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fbb8aca0f0 0x55fbb8ac5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:52.717 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.718+0000 7f47a2071740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fbb8acaa10 0x55fbb8ae9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:52.717 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.718+0000 7f47a2071740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbb8addba0 0x55fbb8af5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:52.717 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.718+0000 7f47a2071740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f47981ad350 con 0x55fbb8addba0 2024-04-24T08:58:52.717 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.718+0000 7f47a2071740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f47981aee90 con 0x55fbb8aca0f0 2024-04-24T08:58:52.717 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.718+0000 7f47a2071740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f47981ada20 con 0x55fbb8acaa10 2024-04-24T08:58:52.720 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.718+0000 7f479d200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbb8addba0 0x55fbb8af5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:52.720 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.718+0000 7f479d200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbb8addba0 0x55fbb8af5ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51382/0 (socket says 172.21.15.83:51382) 2024-04-24T08:58:52.720 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.718+0000 7f479d200640 1 -- 172.21.15.83:0/3597506328 learned_addr learned my addr 172.21.15.83:0/3597506328 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:58:52.720 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.719+0000 7f4797e00640 1 --2- 172.21.15.83:0/3597506328 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fbb8aca0f0 0x55fbb8ac5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:52.720 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.719+0000 7f478fe00640 1 --2- 172.21.15.83:0/3597506328 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fbb8acaa10 0x55fbb8ae9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:52.720 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.719+0000 7f479d200640 1 -- 172.21.15.83:0/3597506328 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fbb8acaa10 msgr2=0x55fbb8ae9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:58:52.720 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.719+0000 7f479d200640 1 --2- 172.21.15.83:0/3597506328 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fbb8acaa10 0x55fbb8ae9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:52.720 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.719+0000 7f479d200640 1 -- 172.21.15.83:0/3597506328 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fbb8aca0f0 msgr2=0x55fbb8ac5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:58:52.720 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.719+0000 7f479d200640 1 --2- 172.21.15.83:0/3597506328 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fbb8aca0f0 0x55fbb8ac5fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:52.720 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.719+0000 7f479d200640 1 -- 172.21.15.83:0/3597506328 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fbb880f620 con 0x55fbb8addba0 2024-04-24T08:58:52.720 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.719+0000 7f4797e00640 1 --2- 172.21.15.83:0/3597506328 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fbb8aca0f0 0x55fbb8ac5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:58:52.721 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.719+0000 7f478fe00640 1 --2- 172.21.15.83:0/3597506328 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fbb8acaa10 0x55fbb8ae9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:58:52.721 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.719+0000 7f479d200640 1 --2- 172.21.15.83:0/3597506328 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbb8addba0 0x55fbb8af5ac0 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f4790009ca0 tx=0x7f479003e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=941ed7be99533c0b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:58:52.721 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.720+0000 7f478cc00640 1 -- 172.21.15.83:0/3597506328 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4790003b70 con 0x55fbb8addba0 2024-04-24T08:58:52.721 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.720+0000 7f478cc00640 1 -- 172.21.15.83:0/3597506328 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f479004d070 con 0x55fbb8addba0 2024-04-24T08:58:52.721 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.720+0000 7f478cc00640 1 -- 172.21.15.83:0/3597506328 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4790030940 con 0x55fbb8addba0 2024-04-24T08:58:52.721 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.720+0000 7f47a2071740 1 -- 172.21.15.83:0/3597506328 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbb8addba0 msgr2=0x55fbb8af5ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:52.721 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.720+0000 7f47a2071740 1 --2- 172.21.15.83:0/3597506328 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbb8addba0 0x55fbb8af5ac0 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f4790009ca0 tx=0x7f479003e000 comp rx=0 tx=0).stop 2024-04-24T08:58:52.721 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.721+0000 7f47a2071740 1 -- 172.21.15.83:0/3597506328 shutdown_connections 2024-04-24T08:58:52.721 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.721+0000 7f47a2071740 1 --2- 172.21.15.83:0/3597506328 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbb8addba0 0x55fbb8af5ac0 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:52.721 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.721+0000 7f47a2071740 1 --2- 172.21.15.83:0/3597506328 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fbb8acaa10 0x55fbb8ae9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:52.721 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.721+0000 7f47a2071740 1 --2- 172.21.15.83:0/3597506328 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fbb8aca0f0 0x55fbb8ac5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:52.722 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.721+0000 7f47a2071740 1 -- 172.21.15.83:0/3597506328 >> 172.21.15.83:0/3597506328 conn(0x55fbb8ac66c0 msgr2=0x55fbb8adb3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:58:52.722 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.721+0000 7f47a2071740 1 -- 172.21.15.83:0/3597506328 shutdown_connections 2024-04-24T08:58:52.722 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.721+0000 7f47a2071740 1 -- 172.21.15.83:0/3597506328 wait complete. 2024-04-24T08:58:52.725 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.721+0000 7f47a2071740 1 -- start start 2024-04-24T08:58:52.725 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.721+0000 7f47a2071740 1 Processor -- start 2024-04-24T08:58:52.725 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.722+0000 7f47a2071740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fbb8aca0f0 0x55fbb8ad7a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:52.725 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.722+0000 7f47a2071740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbb8acaa10 0x55fbb8ad7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:52.725 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.722+0000 7f47a2071740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fbb8addba0 0x55fbb8ad8500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:52.725 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.722+0000 7f47a2071740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f47981ae7c0 con 0x55fbb8acaa10 2024-04-24T08:58:52.725 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.722+0000 7f47a2071740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f47981ae0f0 con 0x55fbb8addba0 2024-04-24T08:58:52.725 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.722+0000 7f47a2071740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f47981ab350 con 0x55fbb8aca0f0 2024-04-24T08:58:52.725 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.722+0000 7f47a2071740 10 client.0 _ll_register_callbacks cb 0x7f478000d6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:58:52.725 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.722+0000 7f478fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbb8acaa10 0x55fbb8ad7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:52.726 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.722+0000 7f478fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbb8acaa10 0x55fbb8ad7fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51386/0 (socket says 172.21.15.83:51386) 2024-04-24T08:58:52.726 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.722+0000 7f478fe00640 1 -- 172.21.15.83:0/3324627395 learned_addr learned my addr 172.21.15.83:0/3324627395 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:58:52.726 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.722+0000 7f4797e00640 1 --2- 172.21.15.83:0/3324627395 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fbb8aca0f0 0x55fbb8ad7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:52.726 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.723+0000 7f479d200640 1 --2- 172.21.15.83:0/3324627395 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fbb8addba0 0x55fbb8ad8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:52.726 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.723+0000 7f478fe00640 1 -- 172.21.15.83:0/3324627395 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fbb8aca0f0 msgr2=0x55fbb8ad7a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:58:52.726 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.723+0000 7f478fe00640 1 --2- 172.21.15.83:0/3324627395 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fbb8aca0f0 0x55fbb8ad7a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:52.726 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.723+0000 7f478fe00640 1 -- 172.21.15.83:0/3324627395 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fbb8addba0 msgr2=0x55fbb8ad8500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:58:52.726 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.723+0000 7f478fe00640 1 --2- 172.21.15.83:0/3324627395 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fbb8addba0 0x55fbb8ad8500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:52.726 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.723+0000 7f478fe00640 1 -- 172.21.15.83:0/3324627395 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fbb8b53580 con 0x55fbb8acaa10 2024-04-24T08:58:52.726 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.723+0000 7f479d200640 1 --2- 172.21.15.83:0/3324627395 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fbb8addba0 0x55fbb8ad8500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:58:52.726 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.723+0000 7f478fe00640 1 --2- 172.21.15.83:0/3324627395 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbb8acaa10 0x55fbb8ad7fc0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f4780010b40 tx=0x7f47800a0390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d0680add7baedc8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:58:52.727 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.723+0000 7f4786000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:58:52.727 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.723+0000 7f4786000640 1 -- 172.21.15.83:0/3324627395 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f478009bbb0 con 0x55fbb8acaa10 2024-04-24T08:58:52.727 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.723+0000 7f4786000640 1 -- 172.21.15.83:0/3324627395 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9567+0+0 (secure 0 0 0) 0x7f47800d0be0 con 0x55fbb8acaa10 2024-04-24T08:58:52.727 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.723+0000 7f47a2071740 1 -- 172.21.15.83:0/3324627395 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f479004d070 con 0x55fbb8acaa10 2024-04-24T08:58:52.727 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.723+0000 7f4786000640 1 -- 172.21.15.83:0/3324627395 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f47800cde90 con 0x55fbb8acaa10 2024-04-24T08:58:52.727 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.723+0000 7f47a2071740 10 client.18216 fetch_fsmap learned FSMap version 33 2024-04-24T08:58:52.727 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.723+0000 7f47a2071740 1 -- 172.21.15.83:0/3324627395 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55fbb8b547e0 con 0x55fbb8acaa10 2024-04-24T08:58:52.727 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.724+0000 7f4786000640 1 -- 172.21.15.83:0/3324627395 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f479004d070 con 0x55fbb8acaa10 2024-04-24T08:58:52.727 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.724+0000 7f4786000640 1 -- 172.21.15.83:0/3324627395 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f47800d02e0 con 0x55fbb8acaa10 2024-04-24T08:58:52.727 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.724+0000 7f47a2071740 10 client.18216 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T08:58:52.727 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.724+0000 7f47a2071740 10 client.18216 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T08:58:52.728 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.724+0000 7f47a2071740 20 client.18216 populate_metadata read hostname 'smithi083' 2024-04-24T08:58:52.728 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.724+0000 7f47a2071740 1 --2- 172.21.15.83:0/3324627395 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55fbb8b54f20 0x55fbb8b57340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:58:52.728 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.724+0000 7f47a2071740 4 client.18216 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T08:58:52.728 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.724+0000 7f47a2071740 1 -- 172.21.15.83:0/3324627395 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fbb8a38190 con 0x55fbb8b54f20 2024-04-24T08:58:52.728 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.725+0000 7f4797e00640 1 --2- 172.21.15.83:0/3324627395 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55fbb8b54f20 0x55fbb8b57340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:58:52.728 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.726+0000 7f4797e00640 1 --2- 172.21.15.83:0/3324627395 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55fbb8b54f20 0x55fbb8b57340 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88934b34da11f24f server_cookie=d98fe8bd87b5aeee in_seq=0 out_seq=0 2024-04-24T08:58:52.728 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.726+0000 7f4786000640 10 client.18216 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:58:52.774 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.774+0000 7f4786000640 1 -- 172.21.15.83:0/3324627395 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fbb8a38190 con 0x55fbb8b54f20 2024-04-24T08:58:52.774 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.774+0000 7f4786000640 10 client.18216 handle_command_reply: tid=0 2024-04-24T08:58:52.774 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.774+0000 7f47a2071740 1 client.18216 shutdown 2024-04-24T08:58:52.774 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.775+0000 7f47a2071740 1 -- 172.21.15.83:0/3324627395 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbb8acaa10 msgr2=0x55fbb8ad7fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:52.775 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.775+0000 7f47a2071740 1 --2- 172.21.15.83:0/3324627395 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbb8acaa10 0x55fbb8ad7fc0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f4780010b40 tx=0x7f47800a0390 comp rx=0 tx=0).stop 2024-04-24T08:58:52.775 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.775+0000 7f47a2071740 1 -- 172.21.15.83:0/3324627395 shutdown_connections 2024-04-24T08:58:52.775 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.775+0000 7f47a2071740 1 --2- 172.21.15.83:0/3324627395 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55fbb8b54f20 0x55fbb8b57340 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:52.775 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.775+0000 7f47a2071740 1 --2- 172.21.15.83:0/3324627395 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fbb8addba0 0x55fbb8ad8500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:52.775 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.775+0000 7f47a2071740 1 --2- 172.21.15.83:0/3324627395 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbb8acaa10 0x55fbb8ad7fc0 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:52.775 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.775+0000 7f47a2071740 1 --2- 172.21.15.83:0/3324627395 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fbb8aca0f0 0x55fbb8ad7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:52.775 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.775+0000 7f4786000640 0 client.18216 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:58:52.775 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.775+0000 7f47a2071740 1 -- 172.21.15.83:0/3324627395 >> 172.21.15.83:0/3324627395 conn(0x55fbb8ac66c0 msgr2=0x55fbb8adb3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:58:52.775 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.775+0000 7f47a2071740 1 -- 172.21.15.83:0/3324627395 shutdown_connections 2024-04-24T08:58:52.775 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.775+0000 7f47a2071740 1 -- 172.21.15.83:0/3324627395 wait complete. 2024-04-24T08:58:52.775 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.775+0000 7f47a2071740 20 client.18216 trim_cache size 0 max 16384 2024-04-24T08:58:52.776 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T08:58:52.776 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T08:58:52.776 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T08:58:52.776 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T08:58:52.776 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.776+0000 7f479f200640 1 -- 172.21.15.83:0/321050243 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f477c0a3e40 msgr2=0x7f477c0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:52.777 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.776+0000 7f479f200640 1 --2- 172.21.15.83:0/321050243 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f477c0a3e40 0x7f477c0a6300 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4790005fe0 tx=0x7f4790005d80 comp rx=0 tx=0).stop 2024-04-24T08:58:52.777 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.776+0000 7f479f200640 1 -- 172.21.15.83:0/321050243 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4798107350 msgr2=0x7f4798074840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:58:52.777 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.776+0000 7f479f200640 1 --2- 172.21.15.83:0/321050243 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4798107350 0x7f4798074840 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f478000d770 tx=0x7f478000dc30 comp rx=0 tx=0).stop 2024-04-24T08:58:52.777 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.776+0000 7f479f200640 1 -- 172.21.15.83:0/321050243 shutdown_connections 2024-04-24T08:58:52.777 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.776+0000 7f479f200640 1 --2- 172.21.15.83:0/321050243 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f477c0a3e40 0x7f477c0a6300 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:52.777 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.776+0000 7f479f200640 1 --2- 172.21.15.83:0/321050243 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4798074d80 0x7f47981a6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:52.777 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.776+0000 7f479f200640 1 --2- 172.21.15.83:0/321050243 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4798107350 0x7f4798074840 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:52.777 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.776+0000 7f479f200640 1 --2- 172.21.15.83:0/321050243 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4798102280 0x7f4798074300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:58:52.777 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.776+0000 7f479f200640 1 -- 172.21.15.83:0/321050243 >> 172.21.15.83:0/321050243 conn(0x7f4798079490 msgr2=0x7f47980ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:58:52.777 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.777+0000 7f479f200640 1 -- 172.21.15.83:0/321050243 shutdown_connections 2024-04-24T08:58:52.777 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:58:52.777+0000 7f479f200640 1 -- 172.21.15.83:0/321050243 wait complete. 2024-04-24T08:58:53.590 INFO:tasks.workunit.client.0.smithi083.stdout: 1 39291 0.65 MB/sec execute 207 sec latency 57.143 ms 2024-04-24T08:58:54.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:58:53 smithi138 ceph-mon[31441]: pgmap v224: 129 pgs: 129 active+clean; 46 MiB data, 654 MiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 558 KiB/s wr, 60 op/s 2024-04-24T08:58:54.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:58:53 smithi189 ceph-mon[31791]: pgmap v224: 129 pgs: 129 active+clean; 46 MiB data, 654 MiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 558 KiB/s wr, 60 op/s 2024-04-24T08:58:54.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:58:53 smithi083 ceph-mon[27907]: pgmap v224: 129 pgs: 129 active+clean; 46 MiB data, 654 MiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 558 KiB/s wr, 60 op/s 2024-04-24T08:58:54.590 INFO:tasks.workunit.client.0.smithi083.stdout: 1 39347 0.64 MB/sec execute 208 sec latency 43.236 ms 2024-04-24T08:58:55.590 INFO:tasks.workunit.client.0.smithi083.stdout: 1 39418 0.64 MB/sec execute 209 sec latency 55.744 ms 2024-04-24T08:58:56.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:58:55 smithi138 ceph-mon[31441]: pgmap v225: 129 pgs: 129 active+clean; 46 MiB data, 654 MiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 404 KiB/s wr, 38 op/s 2024-04-24T08:58:56.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:58:55 smithi189 ceph-mon[31791]: pgmap v225: 129 pgs: 129 active+clean; 46 MiB data, 654 MiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 404 KiB/s wr, 38 op/s 2024-04-24T08:58:56.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:58:55 smithi083 ceph-mon[27907]: pgmap v225: 129 pgs: 129 active+clean; 46 MiB data, 654 MiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 404 KiB/s wr, 38 op/s 2024-04-24T08:58:56.590 INFO:tasks.workunit.client.0.smithi083.stdout: 1 39470 0.64 MB/sec execute 210 sec latency 46.811 ms 2024-04-24T08:58:57.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:58:56 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:58:57.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:58:56 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:58:57.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:58:56 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T08:58:57.590 INFO:tasks.workunit.client.0.smithi083.stdout: 1 39554 0.63 MB/sec execute 211 sec latency 49.452 ms 2024-04-24T08:58:58.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:58:57 smithi138 ceph-mon[31441]: pgmap v226: 129 pgs: 129 active+clean; 46 MiB data, 654 MiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 404 KiB/s wr, 38 op/s 2024-04-24T08:58:58.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:58:57 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T08:58:58.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:58:57 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T08:58:58.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:58:57 smithi189 ceph-mon[31791]: pgmap v226: 129 pgs: 129 active+clean; 46 MiB data, 654 MiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 404 KiB/s wr, 38 op/s 2024-04-24T08:58:58.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:58:57 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T08:58:58.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:58:57 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T08:58:58.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:58:57 smithi083 ceph-mon[27907]: pgmap v226: 129 pgs: 129 active+clean; 46 MiB data, 654 MiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 404 KiB/s wr, 38 op/s 2024-04-24T08:58:58.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:58:57 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T08:58:58.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:58:57 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T08:58:58.590 INFO:tasks.workunit.client.0.smithi083.stdout: 1 39650 0.63 MB/sec execute 212 sec latency 47.247 ms 2024-04-24T08:58:59.591 INFO:tasks.workunit.client.0.smithi083.stdout: 1 39766 0.63 MB/sec execute 213 sec latency 55.356 ms 2024-04-24T08:59:00.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:58:59 smithi138 ceph-mon[31441]: pgmap v227: 129 pgs: 129 active+clean; 47 MiB data, 662 MiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 611 KiB/s wr, 66 op/s 2024-04-24T08:59:00.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:58:59 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:59:00.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:58:59 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:59:00.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:58:59 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T08:59:00.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:58:59 smithi189 ceph-mon[31791]: pgmap v227: 129 pgs: 129 active+clean; 47 MiB data, 662 MiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 611 KiB/s wr, 66 op/s 2024-04-24T08:59:00.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:58:59 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:59:00.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:58:59 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:59:00.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:58:59 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T08:59:00.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:58:59 smithi083 ceph-mon[27907]: pgmap v227: 129 pgs: 129 active+clean; 47 MiB data, 662 MiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 611 KiB/s wr, 66 op/s 2024-04-24T08:59:00.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:58:59 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T08:59:00.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:58:59 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T08:59:00.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:58:59 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T08:59:00.590 INFO:tasks.workunit.client.0.smithi083.stdout: 1 39884 0.63 MB/sec execute 214 sec latency 58.351 ms 2024-04-24T08:59:01.565 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T08:59:01.566 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T08:59:01.590 INFO:tasks.workunit.client.0.smithi083.stdout: 1 39997 0.63 MB/sec execute 215 sec latency 66.155 ms 2024-04-24T08:59:01.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.788+0000 7f9a24000640 1 -- 172.21.15.83:0/3752487560 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a1c1044b0 msgr2=0x7f9a1c104890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:01.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.788+0000 7f9a24000640 1 --2- 172.21.15.83:0/3752487560 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a1c1044b0 0x7f9a1c104890 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f9a10009900 tx=0x7f9a1002a410 comp rx=0 tx=0).stop 2024-04-24T08:59:01.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.788+0000 7f9a24000640 1 -- 172.21.15.83:0/3752487560 shutdown_connections 2024-04-24T08:59:01.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.788+0000 7f9a24000640 1 --2- 172.21.15.83:0/3752487560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a1c109580 0x7f9a1c111150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:01.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.788+0000 7f9a24000640 1 --2- 172.21.15.83:0/3752487560 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9a1c104e60 0x7f9a1c108ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:01.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.788+0000 7f9a24000640 1 --2- 172.21.15.83:0/3752487560 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a1c1044b0 0x7f9a1c104890 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:01.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.788+0000 7f9a24000640 1 -- 172.21.15.83:0/3752487560 >> 172.21.15.83:0/3752487560 conn(0x7f9a1c0ffd90 msgr2=0x7f9a1c1021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:01.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.788+0000 7f9a24000640 1 -- 172.21.15.83:0/3752487560 shutdown_connections 2024-04-24T08:59:01.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.788+0000 7f9a24000640 1 -- 172.21.15.83:0/3752487560 wait complete. 2024-04-24T08:59:01.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.789+0000 7f9a24000640 1 Processor -- start 2024-04-24T08:59:01.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.789+0000 7f9a24000640 1 -- start start 2024-04-24T08:59:01.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.789+0000 7f9a24000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a1c1044b0 0x7f9a1c1a07a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:01.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.789+0000 7f9a24000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9a1c109580 0x7f9a1c1a0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:01.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.789+0000 7f9a24000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a1c19a9a0 0x7f9a1c19ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:01.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.789+0000 7f9a24000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9a1c1144e0 con 0x7f9a1c19a9a0 2024-04-24T08:59:01.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.789+0000 7f9a24000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9a1c114360 con 0x7f9a1c1044b0 2024-04-24T08:59:01.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.789+0000 7f9a24000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9a1c114660 con 0x7f9a1c109580 2024-04-24T08:59:01.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.790+0000 7f9a22000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a1c19a9a0 0x7f9a1c19ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:01.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.790+0000 7f9a22000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a1c19a9a0 0x7f9a1c19ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41088/0 (socket says 172.21.15.83:41088) 2024-04-24T08:59:01.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.790+0000 7f9a22000640 1 -- 172.21.15.83:0/3831610749 learned_addr learned my addr 172.21.15.83:0/3831610749 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:59:01.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.790+0000 7f9a21600640 1 --2- 172.21.15.83:0/3831610749 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a1c1044b0 0x7f9a1c1a07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:01.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.790+0000 7f9a20c00640 1 --2- 172.21.15.83:0/3831610749 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9a1c109580 0x7f9a1c1a0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:01.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.790+0000 7f9a22000640 1 -- 172.21.15.83:0/3831610749 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9a1c109580 msgr2=0x7f9a1c1a0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:01.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.790+0000 7f9a22000640 1 --2- 172.21.15.83:0/3831610749 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9a1c109580 0x7f9a1c1a0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:01.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.790+0000 7f9a22000640 1 -- 172.21.15.83:0/3831610749 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a1c1044b0 msgr2=0x7f9a1c1a07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:01.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.790+0000 7f9a22000640 1 --2- 172.21.15.83:0/3831610749 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a1c1044b0 0x7f9a1c1a07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:01.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.790+0000 7f9a22000640 1 -- 172.21.15.83:0/3831610749 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a1c19b4c0 con 0x7f9a1c19a9a0 2024-04-24T08:59:01.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.790+0000 7f9a20c00640 1 --2- 172.21.15.83:0/3831610749 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9a1c109580 0x7f9a1c1a0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:59:01.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.790+0000 7f9a21600640 1 --2- 172.21.15.83:0/3831610749 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a1c1044b0 0x7f9a1c1a07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:59:01.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.791+0000 7f9a22000640 1 --2- 172.21.15.83:0/3831610749 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a1c19a9a0 0x7f9a1c19ae50 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f9a1400eaa0 tx=0x7f9a1400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:01.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.791+0000 7f9a1a000640 1 -- 172.21.15.83:0/3831610749 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a1400cdb0 con 0x7f9a1c19a9a0 2024-04-24T08:59:01.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.791+0000 7f9a1a000640 1 -- 172.21.15.83:0/3831610749 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9a14005020 con 0x7f9a1c19a9a0 2024-04-24T08:59:01.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.791+0000 7f9a24000640 1 -- 172.21.15.83:0/3831610749 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a1c19b7b0 con 0x7f9a1c19a9a0 2024-04-24T08:59:01.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.791+0000 7f9a1a000640 1 -- 172.21.15.83:0/3831610749 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a140116d0 con 0x7f9a1c19a9a0 2024-04-24T08:59:01.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.791+0000 7f9a24000640 1 -- 172.21.15.83:0/3831610749 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a1c1a7960 con 0x7f9a1c19a9a0 2024-04-24T08:59:01.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.796+0000 7f9a26e7d740 1 Processor -- start 2024-04-24T08:59:01.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.796+0000 7f9a1a000640 1 -- 172.21.15.83:0/3831610749 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9a140051d0 con 0x7f9a1c19a9a0 2024-04-24T08:59:01.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.796+0000 7f9a26e7d740 1 -- start start 2024-04-24T08:59:01.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.796+0000 7f9a26e7d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559581a6c860 0x559581a6cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:01.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.796+0000 7f9a26e7d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559581a6d180 0x559581a77310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:01.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.796+0000 7f9a26e7d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559581a77850 0x559581a79c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:01.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.796+0000 7f9a26e7d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5595819d21f0 con 0x559581a6d180 2024-04-24T08:59:01.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.796+0000 7f9a26e7d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559581978350 con 0x559581a6c860 2024-04-24T08:59:01.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.796+0000 7f9a26e7d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5595819ce1b0 con 0x559581a77850 2024-04-24T08:59:01.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.797+0000 7f9a21600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559581a6d180 0x559581a77310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:01.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.797+0000 7f9a21600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559581a6d180 0x559581a77310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41100/0 (socket says 172.21.15.83:41100) 2024-04-24T08:59:01.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.797+0000 7f9a21600640 1 -- 172.21.15.83:0/2865700810 learned_addr learned my addr 172.21.15.83:0/2865700810 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:59:01.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.797+0000 7f9a20c00640 1 --2- 172.21.15.83:0/2865700810 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559581a6c860 0x559581a6cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:01.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.797+0000 7f9a20c00640 1 --2- 172.21.15.83:0/2865700810 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559581a77850 0x559581a79c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:01.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.797+0000 7f9a21600640 1 -- 172.21.15.83:0/2865700810 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559581a77850 msgr2=0x559581a79c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:01.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.797+0000 7f9a21600640 1 --2- 172.21.15.83:0/2865700810 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559581a77850 0x559581a79c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:01.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.797+0000 7f9a21600640 1 -- 172.21.15.83:0/2865700810 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559581a6c860 msgr2=0x559581a6cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:01.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.797+0000 7f9a21600640 1 --2- 172.21.15.83:0/2865700810 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559581a6c860 0x559581a6cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:01.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.797+0000 7f9a21600640 1 -- 172.21.15.83:0/2865700810 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595819ccf50 con 0x559581a6d180 2024-04-24T08:59:01.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.797+0000 7f9a20c00640 1 --2- 172.21.15.83:0/2865700810 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559581a77850 0x559581a79c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:59:01.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.798+0000 7f9a21600640 1 --2- 172.21.15.83:0/2865700810 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559581a6d180 0x559581a77310 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f9a1002a930 tx=0x7f9a10032cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41d91a36a12cc165 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:01.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.798+0000 7f9a0e600640 1 -- 172.21.15.83:0/2865700810 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a1003f070 con 0x559581a6d180 2024-04-24T08:59:01.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.798+0000 7f9a0e600640 1 -- 172.21.15.83:0/2865700810 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9a100316a0 con 0x559581a6d180 2024-04-24T08:59:01.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.798+0000 7f9a0e600640 1 -- 172.21.15.83:0/2865700810 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a10035070 con 0x559581a6d180 2024-04-24T08:59:01.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.798+0000 7f9a1a000640 1 --2- 172.21.15.83:0/3831610749 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f9a040a3d20 0x7f9a040a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:01.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.799+0000 7f9a1a000640 1 -- 172.21.15.83:0/3831610749 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9567+0+0 (secure 0 0 0) 0x7f9a140d0ea0 con 0x7f9a1c19a9a0 2024-04-24T08:59:01.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.799+0000 7f9a20c00640 1 --2- 172.21.15.83:0/3831610749 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f9a040a3d20 0x7f9a040a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:01.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.799+0000 7f9a26e7d740 1 -- 172.21.15.83:0/2865700810 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559581a6d180 msgr2=0x559581a77310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:01.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.799+0000 7f9a26e7d740 1 --2- 172.21.15.83:0/2865700810 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559581a6d180 0x559581a77310 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f9a1002a930 tx=0x7f9a10032cb0 comp rx=0 tx=0).stop 2024-04-24T08:59:01.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.799+0000 7f9a26e7d740 1 -- 172.21.15.83:0/2865700810 shutdown_connections 2024-04-24T08:59:01.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.799+0000 7f9a26e7d740 1 --2- 172.21.15.83:0/2865700810 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559581a77850 0x559581a79c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:01.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.799+0000 7f9a26e7d740 1 --2- 172.21.15.83:0/2865700810 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559581a6d180 0x559581a77310 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:01.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.799+0000 7f9a26e7d740 1 --2- 172.21.15.83:0/2865700810 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559581a6c860 0x559581a6cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:01.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.799+0000 7f9a26e7d740 1 -- 172.21.15.83:0/2865700810 >> 172.21.15.83:0/2865700810 conn(0x559581a68b80 msgr2=0x559581a68f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:01.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.799+0000 7f9a26e7d740 1 -- 172.21.15.83:0/2865700810 shutdown_connections 2024-04-24T08:59:01.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.799+0000 7f9a26e7d740 1 -- 172.21.15.83:0/2865700810 wait complete. 2024-04-24T08:59:01.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.800+0000 7f9a26e7d740 1 -- start start 2024-04-24T08:59:01.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.800+0000 7f9a20c00640 1 --2- 172.21.15.83:0/3831610749 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f9a040a3d20 0x7f9a040a61e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x559581a69d10 tx=0x7f9a0800b3c0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:01.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.800+0000 7f9a26e7d740 1 Processor -- start 2024-04-24T08:59:01.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.800+0000 7f9a26e7d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559581a6c860 0x559581a74750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:01.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.800+0000 7f9a26e7d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559581a6d180 0x559581a74c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:01.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.800+0000 7f9a26e7d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559581a77850 0x559581a751d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:01.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.800+0000 7f9a26e7d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559581a7a5e0 con 0x559581a6c860 2024-04-24T08:59:01.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.800+0000 7f9a26e7d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55958198b690 con 0x559581a77850 2024-04-24T08:59:01.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.800+0000 7f9a26e7d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559581a7a760 con 0x559581a6d180 2024-04-24T08:59:01.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.800+0000 7f9a26e7d740 10 client.0 _ll_register_callbacks cb 0x5595819c7650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:59:01.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.801+0000 7f9a21600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559581a6c860 0x559581a74750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:01.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.801+0000 7f9a21600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559581a6c860 0x559581a74750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41102/0 (socket says 172.21.15.83:41102) 2024-04-24T08:59:01.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.801+0000 7f9a21600640 1 -- 172.21.15.83:0/2788645982 learned_addr learned my addr 172.21.15.83:0/2788645982 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:59:01.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.801+0000 7f9a20c00640 1 --2- 172.21.15.83:0/2788645982 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559581a6d180 0x559581a74c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:01.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.801+0000 7f9a22000640 1 --2- 172.21.15.83:0/2788645982 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559581a77850 0x559581a751d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:01.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.801+0000 7f9a21600640 1 -- 172.21.15.83:0/2788645982 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559581a6d180 msgr2=0x559581a74c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:01.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.801+0000 7f9a21600640 1 --2- 172.21.15.83:0/2788645982 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559581a6d180 0x559581a74c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:01.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.801+0000 7f9a21600640 1 -- 172.21.15.83:0/2788645982 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559581a77850 msgr2=0x559581a751d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:01.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.801+0000 7f9a21600640 1 --2- 172.21.15.83:0/2788645982 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559581a77850 0x559581a751d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:01.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.801+0000 7f9a21600640 1 -- 172.21.15.83:0/2788645982 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55958198a920 con 0x559581a6c860 2024-04-24T08:59:01.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.801+0000 7f9a21600640 1 --2- 172.21.15.83:0/2788645982 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559581a6c860 0x559581a74750 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f9a100053b0 tx=0x7f9a10031510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bae9d98fb1f8955 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:01.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.802+0000 7f9a02a00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:59:01.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.802+0000 7f9a02a00640 1 -- 172.21.15.83:0/2788645982 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a1004a020 con 0x559581a6c860 2024-04-24T08:59:01.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.802+0000 7f9a02a00640 1 -- 172.21.15.83:0/2788645982 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9567+0+0 (secure 0 0 0) 0x7f9a100035f0 con 0x559581a6c860 2024-04-24T08:59:01.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.802+0000 7f9a26e7d740 1 -- 172.21.15.83:0/2788645982 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a100316a0 con 0x559581a6c860 2024-04-24T08:59:01.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.802+0000 7f9a02a00640 1 -- 172.21.15.83:0/2788645982 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f9a10055050 con 0x559581a6c860 2024-04-24T08:59:01.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.802+0000 7f9a26e7d740 10 client.18228 fetch_fsmap learned FSMap version 33 2024-04-24T08:59:01.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.802+0000 7f9a26e7d740 1 -- 172.21.15.83:0/2788645982 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5595817ae7d0 con 0x559581a6c860 2024-04-24T08:59:01.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.803+0000 7f9a02a00640 1 -- 172.21.15.83:0/2788645982 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9a100316a0 con 0x559581a6c860 2024-04-24T08:59:01.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.803+0000 7f9a02a00640 1 -- 172.21.15.83:0/2788645982 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f9a10004b00 con 0x559581a6c860 2024-04-24T08:59:01.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.803+0000 7f9a26e7d740 10 client.18228 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T08:59:01.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.803+0000 7f9a26e7d740 10 client.18228 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:59:01.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.803+0000 7f9a26e7d740 20 client.18228 populate_metadata read hostname 'smithi083' 2024-04-24T08:59:01.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.803+0000 7f9a26e7d740 1 --2- 172.21.15.83:0/2788645982 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x559581a87900 0x559581a89cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:01.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.803+0000 7f9a26e7d740 4 client.18228 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:59:01.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.803+0000 7f9a26e7d740 1 -- 172.21.15.83:0/2788645982 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5595819dfc10 con 0x559581a87900 2024-04-24T08:59:01.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.804+0000 7f9a20c00640 1 --2- 172.21.15.83:0/2788645982 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x559581a87900 0x559581a89cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:01.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.805+0000 7f9a20c00640 1 --2- 172.21.15.83:0/2788645982 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x559581a87900 0x559581a89cf0 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9927968987a5d3f server_cookie=12cc9afa22730f33 in_seq=0 out_seq=0 2024-04-24T08:59:01.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.805+0000 7f9a02a00640 10 client.18228 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:59:01.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.808+0000 7f9a02a00640 1 -- 172.21.15.83:0/2788645982 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5595819dfc10 con 0x559581a87900 2024-04-24T08:59:01.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.808+0000 7f9a02a00640 10 client.18228 handle_command_reply: tid=0 2024-04-24T08:59:01.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.808+0000 7f9a26e7d740 1 client.18228 shutdown 2024-04-24T08:59:01.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.808+0000 7f9a26e7d740 1 -- 172.21.15.83:0/2788645982 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559581a6c860 msgr2=0x559581a74750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:01.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.809+0000 7f9a26e7d740 1 --2- 172.21.15.83:0/2788645982 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559581a6c860 0x559581a74750 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f9a100053b0 tx=0x7f9a10031510 comp rx=0 tx=0).stop 2024-04-24T08:59:01.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.809+0000 7f9a26e7d740 1 -- 172.21.15.83:0/2788645982 shutdown_connections 2024-04-24T08:59:01.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.809+0000 7f9a26e7d740 1 --2- 172.21.15.83:0/2788645982 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x559581a87900 0x559581a89cf0 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:01.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.809+0000 7f9a26e7d740 1 --2- 172.21.15.83:0/2788645982 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559581a77850 0x559581a751d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:01.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.809+0000 7f9a26e7d740 1 --2- 172.21.15.83:0/2788645982 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559581a6d180 0x559581a74c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:01.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.809+0000 7f9a26e7d740 1 --2- 172.21.15.83:0/2788645982 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559581a6c860 0x559581a74750 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:01.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.809+0000 7f9a26e7d740 1 -- 172.21.15.83:0/2788645982 >> 172.21.15.83:0/2788645982 conn(0x559581a68b80 msgr2=0x559581a6e880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:01.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.809+0000 7f9a02a00640 0 client.18228 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:59:01.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.809+0000 7f9a26e7d740 1 -- 172.21.15.83:0/2788645982 shutdown_connections 2024-04-24T08:59:01.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.809+0000 7f9a26e7d740 1 -- 172.21.15.83:0/2788645982 wait complete. 2024-04-24T08:59:01.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.809+0000 7f9a26e7d740 20 client.18228 trim_cache size 0 max 16384 2024-04-24T08:59:01.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.829+0000 7f9a26e7d740 1 Processor -- start 2024-04-24T08:59:01.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.829+0000 7f9a26e7d740 1 -- start start 2024-04-24T08:59:01.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.829+0000 7f9a26e7d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559581a6c860 0x559581a834d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:01.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.829+0000 7f9a26e7d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559581a6d180 0x559581a6eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:01.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.830+0000 7f9a26e7d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559581a77850 0x559581a8f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:01.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.830+0000 7f9a26e7d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9a1c1b1830 con 0x559581a77850 2024-04-24T08:59:01.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.830+0000 7f9a26e7d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9a1c1b3370 con 0x559581a6c860 2024-04-24T08:59:01.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.830+0000 7f9a26e7d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9a1c1b1f00 con 0x559581a6d180 2024-04-24T08:59:01.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.830+0000 7f9a22000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559581a77850 0x559581a8f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:01.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.830+0000 7f9a22000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559581a77850 0x559581a8f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41104/0 (socket says 172.21.15.83:41104) 2024-04-24T08:59:01.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.830+0000 7f9a22000640 1 -- 172.21.15.83:0/3066086046 learned_addr learned my addr 172.21.15.83:0/3066086046 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:59:01.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.830+0000 7f9a21600640 1 --2- 172.21.15.83:0/3066086046 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559581a6c860 0x559581a834d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:01.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.830+0000 7f9a20c00640 1 --2- 172.21.15.83:0/3066086046 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559581a6d180 0x559581a6eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:01.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.830+0000 7f9a22000640 1 -- 172.21.15.83:0/3066086046 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559581a6d180 msgr2=0x559581a6eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:01.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.830+0000 7f9a22000640 1 --2- 172.21.15.83:0/3066086046 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559581a6d180 0x559581a6eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:01.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.830+0000 7f9a22000640 1 -- 172.21.15.83:0/3066086046 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559581a6c860 msgr2=0x559581a834d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:01.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.830+0000 7f9a22000640 1 --2- 172.21.15.83:0/3066086046 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559581a6c860 0x559581a834d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:01.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.830+0000 7f9a22000640 1 -- 172.21.15.83:0/3066086046 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559581a6cfb0 con 0x559581a77850 2024-04-24T08:59:01.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.831+0000 7f9a22000640 1 --2- 172.21.15.83:0/3066086046 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559581a77850 0x559581a8f320 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f9a1400cfd0 tx=0x7f9a1409c620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f22eba41eae0f1d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:01.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.831+0000 7f9a0dc00640 1 -- 172.21.15.83:0/3066086046 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a1401f040 con 0x559581a77850 2024-04-24T08:59:01.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.831+0000 7f9a0dc00640 1 -- 172.21.15.83:0/3066086046 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9a14005560 con 0x559581a77850 2024-04-24T08:59:01.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.831+0000 7f9a0dc00640 1 -- 172.21.15.83:0/3066086046 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a140bbb90 con 0x559581a77850 2024-04-24T08:59:01.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.832+0000 7f9a26e7d740 1 -- 172.21.15.83:0/3066086046 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559581a77850 msgr2=0x559581a8f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:01.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.832+0000 7f9a26e7d740 1 --2- 172.21.15.83:0/3066086046 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559581a77850 0x559581a8f320 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f9a1400cfd0 tx=0x7f9a1409c620 comp rx=0 tx=0).stop 2024-04-24T08:59:01.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.832+0000 7f9a26e7d740 1 -- 172.21.15.83:0/3066086046 shutdown_connections 2024-04-24T08:59:01.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.832+0000 7f9a26e7d740 1 --2- 172.21.15.83:0/3066086046 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559581a77850 0x559581a8f320 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:01.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.832+0000 7f9a26e7d740 1 --2- 172.21.15.83:0/3066086046 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559581a6d180 0x559581a6eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:01.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.832+0000 7f9a26e7d740 1 --2- 172.21.15.83:0/3066086046 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559581a6c860 0x559581a834d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:01.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.832+0000 7f9a26e7d740 1 -- 172.21.15.83:0/3066086046 >> 172.21.15.83:0/3066086046 conn(0x559581a68b80 msgr2=0x559581a6f9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:01.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.832+0000 7f9a26e7d740 1 -- 172.21.15.83:0/3066086046 shutdown_connections 2024-04-24T08:59:01.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.832+0000 7f9a26e7d740 1 -- 172.21.15.83:0/3066086046 wait complete. 2024-04-24T08:59:01.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.832+0000 7f9a26e7d740 1 -- start start 2024-04-24T08:59:01.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.832+0000 7f9a26e7d740 1 Processor -- start 2024-04-24T08:59:01.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.833+0000 7f9a26e7d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559581a6c860 0x559581a7f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:01.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.833+0000 7f9a26e7d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559581a6d180 0x559581a7ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:01.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.833+0000 7f9a26e7d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559581a77850 0x559581a70b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:01.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.833+0000 7f9a26e7d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9a1c1b2ca0 con 0x559581a6d180 2024-04-24T08:59:01.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.833+0000 7f9a26e7d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9a1c1b25d0 con 0x559581a6c860 2024-04-24T08:59:01.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.833+0000 7f9a26e7d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9a1c1af810 con 0x559581a77850 2024-04-24T08:59:01.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.833+0000 7f9a26e7d740 10 client.0 _ll_register_callbacks cb 0x7f9a10032b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:59:01.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.833+0000 7f9a20c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559581a6d180 0x559581a7ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:01.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.833+0000 7f9a20c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559581a6d180 0x559581a7ff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41106/0 (socket says 172.21.15.83:41106) 2024-04-24T08:59:01.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.833+0000 7f9a20c00640 1 -- 172.21.15.83:0/1381055718 learned_addr learned my addr 172.21.15.83:0/1381055718 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:59:01.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.833+0000 7f9a21600640 1 --2- 172.21.15.83:0/1381055718 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559581a6c860 0x559581a7f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:01.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.833+0000 7f9a20c00640 1 -- 172.21.15.83:0/1381055718 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559581a77850 msgr2=0x559581a70b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:01.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.833+0000 7f9a20c00640 1 --2- 172.21.15.83:0/1381055718 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559581a77850 0x559581a70b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:01.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.833+0000 7f9a20c00640 1 -- 172.21.15.83:0/1381055718 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559581a6c860 msgr2=0x559581a7f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:01.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.833+0000 7f9a20c00640 1 --2- 172.21.15.83:0/1381055718 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559581a6c860 0x559581a7f9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:01.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.833+0000 7f9a20c00640 1 -- 172.21.15.83:0/1381055718 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595816e43f0 con 0x559581a6d180 2024-04-24T08:59:01.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.834+0000 7f9a20c00640 1 --2- 172.21.15.83:0/1381055718 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559581a6d180 0x559581a7ff20 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c1002f0 tx=0x7f9a08005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95023c5c378b2ca1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:01.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.834+0000 7f9a02a00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:59:01.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.834+0000 7f9a02a00640 1 -- 172.21.15.83:0/1381055718 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a08012c20 con 0x559581a6d180 2024-04-24T08:59:01.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.834+0000 7f9a26e7d740 1 -- 172.21.15.83:0/1381055718 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a14005560 con 0x559581a6d180 2024-04-24T08:59:01.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.834+0000 7f9a02a00640 1 -- 172.21.15.83:0/1381055718 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9567+0+0 (secure 0 0 0) 0x7f9a08012dc0 con 0x559581a6d180 2024-04-24T08:59:01.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.835+0000 7f9a02a00640 1 -- 172.21.15.83:0/1381055718 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f9a0803b050 con 0x559581a6d180 2024-04-24T08:59:01.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.835+0000 7f9a26e7d740 10 client.18234 fetch_fsmap learned FSMap version 33 2024-04-24T08:59:01.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.835+0000 7f9a26e7d740 1 -- 172.21.15.83:0/1381055718 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5595816a40f0 con 0x559581a6d180 2024-04-24T08:59:01.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.835+0000 7f9a02a00640 1 -- 172.21.15.83:0/1381055718 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9a14005560 con 0x559581a6d180 2024-04-24T08:59:01.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.835+0000 7f9a02a00640 1 -- 172.21.15.83:0/1381055718 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f9a0800f040 con 0x559581a6d180 2024-04-24T08:59:01.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.835+0000 7f9a26e7d740 10 client.18234 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T08:59:01.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.835+0000 7f9a26e7d740 10 client.18234 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:59:01.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.835+0000 7f9a26e7d740 20 client.18234 populate_metadata read hostname 'smithi083' 2024-04-24T08:59:01.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.835+0000 7f9a26e7d740 1 --2- 172.21.15.83:0/1381055718 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x559581aed9e0 0x559581aefe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:01.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.835+0000 7f9a26e7d740 4 client.18234 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T08:59:01.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.835+0000 7f9a26e7d740 1 -- 172.21.15.83:0/1381055718 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559581a6f500 con 0x559581aed9e0 2024-04-24T08:59:01.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.836+0000 7f9a21600640 1 --2- 172.21.15.83:0/1381055718 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x559581aed9e0 0x559581aefe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:01.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.837+0000 7f9a21600640 1 --2- 172.21.15.83:0/1381055718 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x559581aed9e0 0x559581aefe00 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62feb45cf3ed2606 server_cookie=2ed59a7957cb86f6 in_seq=0 out_seq=0 2024-04-24T08:59:01.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.837+0000 7f9a02a00640 10 client.18234 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:59:01.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.839+0000 7f9a02a00640 1 -- 172.21.15.83:0/1381055718 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x559581a6f500 con 0x559581aed9e0 2024-04-24T08:59:01.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.839+0000 7f9a02a00640 10 client.18234 handle_command_reply: tid=0 2024-04-24T08:59:01.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.839+0000 7f9a26e7d740 1 client.18234 shutdown 2024-04-24T08:59:01.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.839+0000 7f9a26e7d740 1 -- 172.21.15.83:0/1381055718 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559581a6d180 msgr2=0x559581a7ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:01.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.839+0000 7f9a26e7d740 1 --2- 172.21.15.83:0/1381055718 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559581a6d180 0x559581a7ff20 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c1002f0 tx=0x7f9a08005c90 comp rx=0 tx=0).stop 2024-04-24T08:59:01.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.839+0000 7f9a26e7d740 1 -- 172.21.15.83:0/1381055718 shutdown_connections 2024-04-24T08:59:01.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.839+0000 7f9a26e7d740 1 --2- 172.21.15.83:0/1381055718 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x559581aed9e0 0x559581aefe00 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:01.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.839+0000 7f9a26e7d740 1 --2- 172.21.15.83:0/1381055718 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559581a77850 0x559581a70b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:01.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.839+0000 7f9a26e7d740 1 --2- 172.21.15.83:0/1381055718 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559581a6d180 0x559581a7ff20 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:01.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.839+0000 7f9a26e7d740 1 --2- 172.21.15.83:0/1381055718 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559581a6c860 0x559581a7f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:01.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.839+0000 7f9a02a00640 0 client.18234 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:59:01.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.839+0000 7f9a26e7d740 1 -- 172.21.15.83:0/1381055718 >> 172.21.15.83:0/1381055718 conn(0x559581a68b80 msgr2=0x559581a72fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:01.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.840+0000 7f9a26e7d740 1 -- 172.21.15.83:0/1381055718 shutdown_connections 2024-04-24T08:59:01.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.840+0000 7f9a26e7d740 1 -- 172.21.15.83:0/1381055718 wait complete. 2024-04-24T08:59:01.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.840+0000 7f9a26e7d740 20 client.18234 trim_cache size 0 max 16384 2024-04-24T08:59:01.842 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T08:59:01.843 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-04-24T08:59:01.843 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-04-24T08:59:01.843 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T08:59:01.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.840+0000 7f9a24000640 1 -- 172.21.15.83:0/3831610749 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f9a040a3d20 msgr2=0x7f9a040a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:01.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.840+0000 7f9a24000640 1 --2- 172.21.15.83:0/3831610749 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f9a040a3d20 0x7f9a040a61e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x559581a69d10 tx=0x7f9a0800b3c0 comp rx=0 tx=0).stop 2024-04-24T08:59:01.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.840+0000 7f9a24000640 1 -- 172.21.15.83:0/3831610749 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a1c19a9a0 msgr2=0x7f9a1c19ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:01.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.840+0000 7f9a24000640 1 --2- 172.21.15.83:0/3831610749 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a1c19a9a0 0x7f9a1c19ae50 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f9a1400eaa0 tx=0x7f9a1400ef60 comp rx=0 tx=0).stop 2024-04-24T08:59:01.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.841+0000 7f9a24000640 1 -- 172.21.15.83:0/3831610749 shutdown_connections 2024-04-24T08:59:01.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.841+0000 7f9a24000640 1 --2- 172.21.15.83:0/3831610749 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f9a040a3d20 0x7f9a040a61e0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:01.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.841+0000 7f9a24000640 1 --2- 172.21.15.83:0/3831610749 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a1c19a9a0 0x7f9a1c19ae50 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:01.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.841+0000 7f9a24000640 1 --2- 172.21.15.83:0/3831610749 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9a1c109580 0x7f9a1c1a0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:01.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.841+0000 7f9a24000640 1 --2- 172.21.15.83:0/3831610749 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a1c1044b0 0x7f9a1c1a07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:01.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.841+0000 7f9a24000640 1 -- 172.21.15.83:0/3831610749 >> 172.21.15.83:0/3831610749 conn(0x7f9a1c0ffd90 msgr2=0x7f9a1c1011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:01.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.841+0000 7f9a24000640 1 -- 172.21.15.83:0/3831610749 shutdown_connections 2024-04-24T08:59:01.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:01.841+0000 7f9a24000640 1 -- 172.21.15.83:0/3831610749 wait complete. 2024-04-24T08:59:01.859 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T08:59:01.859 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T08:59:01.980 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:59:01 smithi083 ceph-mon[27907]: pgmap v228: 129 pgs: 129 active+clean; 47 MiB data, 662 MiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 358 KiB/s wr, 45 op/s 2024-04-24T08:59:02.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.123+0000 7f4524c00640 1 -- 172.21.15.83:0/2302245620 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45201084a0 msgr2=0x7f4520108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:02.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.123+0000 7f4524c00640 1 --2- 172.21.15.83:0/2302245620 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45201084a0 0x7f4520108900 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f4508009900 tx=0x7f450802a480 comp rx=0 tx=0).stop 2024-04-24T08:59:02.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.123+0000 7f4524c00640 1 -- 172.21.15.83:0/2302245620 shutdown_connections 2024-04-24T08:59:02.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.123+0000 7f4524c00640 1 --2- 172.21.15.83:0/2302245620 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4520108e40 0x7f45201152b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:02.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.123+0000 7f4524c00640 1 --2- 172.21.15.83:0/2302245620 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45201084a0 0x7f4520108900 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:02.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.123+0000 7f4524c00640 1 --2- 172.21.15.83:0/2302245620 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f452010d3c0 0x7f452010d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:02.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.123+0000 7f4524c00640 1 -- 172.21.15.83:0/2302245620 >> 172.21.15.83:0/2302245620 conn(0x7f4520077b80 msgr2=0x7f4520079fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:02.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.123+0000 7f4524c00640 1 -- 172.21.15.83:0/2302245620 shutdown_connections 2024-04-24T08:59:02.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.123+0000 7f4524c00640 1 -- 172.21.15.83:0/2302245620 wait complete. 2024-04-24T08:59:02.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.124+0000 7f4524c00640 1 Processor -- start 2024-04-24T08:59:02.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.124+0000 7f4524c00640 1 -- start start 2024-04-24T08:59:02.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.124+0000 7f4524c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45201084a0 0x7f45201a4800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:02.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.124+0000 7f4524c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4520108e40 0x7f45201a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:02.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.124+0000 7f4524c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f452010d3c0 0x7f452019e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:02.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.124+0000 7f4524c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4520118690 con 0x7f45201084a0 2024-04-24T08:59:02.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.124+0000 7f4524c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4520118510 con 0x7f4520108e40 2024-04-24T08:59:02.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.124+0000 7f4524c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4520118810 con 0x7f452010d3c0 2024-04-24T08:59:02.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.125+0000 7f451f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45201084a0 0x7f45201a4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:02.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.125+0000 7f451f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45201084a0 0x7f45201a4800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41122/0 (socket says 172.21.15.83:41122) 2024-04-24T08:59:02.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.125+0000 7f451f400640 1 -- 172.21.15.83:0/441067524 learned_addr learned my addr 172.21.15.83:0/441067524 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:59:02.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.125+0000 7f451fe00640 1 --2- 172.21.15.83:0/441067524 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f452010d3c0 0x7f452019e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:02.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.125+0000 7f451f400640 1 -- 172.21.15.83:0/441067524 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f452010d3c0 msgr2=0x7f452019e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:02.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.125+0000 7f451f400640 1 --2- 172.21.15.83:0/441067524 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f452010d3c0 0x7f452019e9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:02.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.125+0000 7f451f400640 1 -- 172.21.15.83:0/441067524 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4520108e40 msgr2=0x7f45201a4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:02.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.125+0000 7f451f400640 1 --2- 172.21.15.83:0/441067524 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4520108e40 0x7f45201a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:02.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.125+0000 7f451f400640 1 -- 172.21.15.83:0/441067524 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f452019f130 con 0x7f45201084a0 2024-04-24T08:59:02.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.125+0000 7f451f400640 1 --2- 172.21.15.83:0/441067524 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45201084a0 0x7f45201a4800 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f451000c920 tx=0x7f451000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:02.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.126+0000 7f4517e00640 1 -- 172.21.15.83:0/441067524 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4510007c70 con 0x7f45201084a0 2024-04-24T08:59:02.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.126+0000 7f4517e00640 1 -- 172.21.15.83:0/441067524 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4510007e10 con 0x7f45201084a0 2024-04-24T08:59:02.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.126+0000 7f4524c00640 1 -- 172.21.15.83:0/441067524 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f452019f420 con 0x7f45201084a0 2024-04-24T08:59:02.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.126+0000 7f4517e00640 1 -- 172.21.15.83:0/441067524 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4510018820 con 0x7f45201084a0 2024-04-24T08:59:02.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.126+0000 7f4524c00640 1 -- 172.21.15.83:0/441067524 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45201ab6c0 con 0x7f45201084a0 2024-04-24T08:59:02.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.127+0000 7f4524c00640 1 -- 172.21.15.83:0/441067524 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4520069190 con 0x7f45201084a0 2024-04-24T08:59:02.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.129+0000 7f4517e00640 1 -- 172.21.15.83:0/441067524 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f45100189c0 con 0x7f45201084a0 2024-04-24T08:59:02.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.130+0000 7f4517e00640 1 --2- 172.21.15.83:0/441067524 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f45040a3d20 0x7f45040a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:02.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.131+0000 7f4517e00640 1 -- 172.21.15.83:0/441067524 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9567+0+0 (secure 0 0 0) 0x7f45100d1070 con 0x7f45201084a0 2024-04-24T08:59:02.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.131+0000 7f451ea00640 1 --2- 172.21.15.83:0/441067524 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f45040a3d20 0x7f45040a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:02.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.132+0000 7f451ea00640 1 --2- 172.21.15.83:0/441067524 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f45040a3d20 0x7f45040a61e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f45080096c0 tx=0x7f4508031400 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:02.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.134+0000 7f4517e00640 1 -- 172.21.15.83:0/441067524 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4510014030 con 0x7f45201084a0 2024-04-24T08:59:02.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:59:01 smithi138 ceph-mon[31441]: pgmap v228: 129 pgs: 129 active+clean; 47 MiB data, 662 MiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 358 KiB/s wr, 45 op/s 2024-04-24T08:59:02.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:59:01 smithi189 ceph-mon[31791]: pgmap v228: 129 pgs: 129 active+clean; 47 MiB data, 662 MiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 358 KiB/s wr, 45 op/s 2024-04-24T08:59:02.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.405+0000 7f4524c00640 1 -- 172.21.15.83:0/441067524 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f452019fee0 con 0x7f45201084a0 2024-04-24T08:59:02.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.407+0000 7f4517e00640 1 -- 172.21.15.83:0/441067524 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11944 (secure 0 0 0) 0x7f451009d330 con 0x7f45201084a0 2024-04-24T08:59:02.407 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:59:02.407 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/358872257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":358872257},{"type":"v1","addr":"172.21.15.83:6835","nonce":358872257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3157070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3157070851},{"type":"v1","addr":"172.21.15.83:6837","nonce":3157070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":33,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T08:48:29.422576+0000","modified":"2024-04-24T08:56:17.048967+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34520,"mds_1":34493,"mds_2":34451,"mds_3":34427,"mds_4":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14658":{"gid":14658,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6839/4085470844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":4085470844},{"type":"v1","addr":"172.21.15.83:6839","nonce":4085470844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14676":{"gid":14676,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6841/2634754210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":2634754210},{"type":"v1","addr":"172.21.15.83:6841","nonce":2634754210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24382":{"gid":24382,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/1859921310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1859921310},{"type":"v1","addr":"172.21.15.189:6833","nonce":1859921310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/2241923197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2241923197},{"type":"v1","addr":"172.21.15.189:6835","nonce":2241923197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6837/3172311220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":3172311220},{"type":"v1","addr":"172.21.15.189:6837","nonce":3172311220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.189:6839/2548965336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":2548965336},{"type":"v1","addr":"172.21.15.189:6839","nonce":2548965336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.138:6833/1157967658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1157967658},{"type":"v1","addr":"172.21.15.138:6833","nonce":1157967658}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.138:6835/1909219758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1909219758},{"type":"v1","addr":"172.21.15.138:6835","nonce":1909219758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34493":{"gid":34493,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.138:6837/900566754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":900566754},{"type":"v1","addr":"172.21.15.138:6837","nonce":900566754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34520":{"gid":34520,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.138:6839/2236932393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2236932393},{"type":"v1","addr":"172.21.15.138:6839","nonce":2236932393}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34520,"qdb_cluster":[24454,34520,34427,34451,34493]},"id":1}]} 2024-04-24T08:59:02.407 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 33 2024-04-24T08:59:02.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.409+0000 7f4524c00640 1 -- 172.21.15.83:0/441067524 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f45040a3d20 msgr2=0x7f45040a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:02.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.409+0000 7f4524c00640 1 --2- 172.21.15.83:0/441067524 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f45040a3d20 0x7f45040a61e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f45080096c0 tx=0x7f4508031400 comp rx=0 tx=0).stop 2024-04-24T08:59:02.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.409+0000 7f4524c00640 1 -- 172.21.15.83:0/441067524 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45201084a0 msgr2=0x7f45201a4800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:02.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.409+0000 7f4524c00640 1 --2- 172.21.15.83:0/441067524 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45201084a0 0x7f45201a4800 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f451000c920 tx=0x7f451000cde0 comp rx=0 tx=0).stop 2024-04-24T08:59:02.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.409+0000 7f4524c00640 1 -- 172.21.15.83:0/441067524 shutdown_connections 2024-04-24T08:59:02.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.409+0000 7f4524c00640 1 --2- 172.21.15.83:0/441067524 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f45040a3d20 0x7f45040a61e0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:02.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.409+0000 7f4524c00640 1 --2- 172.21.15.83:0/441067524 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f452010d3c0 0x7f452019e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:02.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.409+0000 7f4524c00640 1 --2- 172.21.15.83:0/441067524 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4520108e40 0x7f45201a4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:02.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.409+0000 7f4524c00640 1 --2- 172.21.15.83:0/441067524 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45201084a0 0x7f45201a4800 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:02.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.410+0000 7f4524c00640 1 -- 172.21.15.83:0/441067524 >> 172.21.15.83:0/441067524 conn(0x7f4520077b80 msgr2=0x7f4520078310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:02.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.410+0000 7f4524c00640 1 -- 172.21.15.83:0/441067524 shutdown_connections 2024-04-24T08:59:02.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.410+0000 7f4524c00640 1 -- 172.21.15.83:0/441067524 wait complete. 2024-04-24T08:59:02.428 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-24T08:59:02.591 INFO:tasks.workunit.client.0.smithi083.stdout: 1 40138 0.63 MB/sec execute 216 sec latency 48.957 ms 2024-04-24T08:59:02.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.650+0000 7f4d70c00640 1 -- 172.21.15.83:0/3321416978 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d6c10e820 msgr2=0x7f4d6c110c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:02.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.650+0000 7f4d70c00640 1 --2- 172.21.15.83:0/3321416978 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d6c10e820 0x7f4d6c110c10 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f4d64009ca0 tx=0x7f4d6402a8c0 comp rx=0 tx=0).stop 2024-04-24T08:59:02.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.650+0000 7f4d70c00640 1 -- 172.21.15.83:0/3321416978 shutdown_connections 2024-04-24T08:59:02.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.650+0000 7f4d70c00640 1 --2- 172.21.15.83:0/3321416978 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d6c10e820 0x7f4d6c110c10 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:02.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.650+0000 7f4d70c00640 1 --2- 172.21.15.83:0/3321416978 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d6c101620 0x7f4d6c10e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:02.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.650+0000 7f4d70c00640 1 --2- 172.21.15.83:0/3321416978 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4d6c100c70 0x7f4d6c101050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:02.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.650+0000 7f4d70c00640 1 -- 172.21.15.83:0/3321416978 >> 172.21.15.83:0/3321416978 conn(0x7f4d6c0fc880 msgr2=0x7f4d6c0feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:02.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.650+0000 7f4d70c00640 1 -- 172.21.15.83:0/3321416978 shutdown_connections 2024-04-24T08:59:02.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.650+0000 7f4d70c00640 1 -- 172.21.15.83:0/3321416978 wait complete. 2024-04-24T08:59:02.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.651+0000 7f4d70c00640 1 Processor -- start 2024-04-24T08:59:02.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.652+0000 7f4d70c00640 1 -- start start 2024-04-24T08:59:02.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.652+0000 7f4d70c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4d6c100c70 0x7f4d6c1a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:02.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.652+0000 7f4d70c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d6c101620 0x7f4d6c1a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:02.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.652+0000 7f4d70c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d6c10e820 0x7f4d6c19a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:02.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.652+0000 7f4d70c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4d6c114370 con 0x7f4d6c101620 2024-04-24T08:59:02.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.652+0000 7f4d70c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4d6c1141f0 con 0x7f4d6c10e820 2024-04-24T08:59:02.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.652+0000 7f4d70c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4d6c1144f0 con 0x7f4d6c100c70 2024-04-24T08:59:02.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.652+0000 7f4d6aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d6c101620 0x7f4d6c1a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:02.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.652+0000 7f4d6aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d6c101620 0x7f4d6c1a0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41146/0 (socket says 172.21.15.83:41146) 2024-04-24T08:59:02.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.652+0000 7f4d6aa00640 1 -- 172.21.15.83:0/2683689628 learned_addr learned my addr 172.21.15.83:0/2683689628 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:59:02.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.653+0000 7f4d6b400640 1 --2- 172.21.15.83:0/2683689628 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4d6c100c70 0x7f4d6c1a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:02.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.653+0000 7f4d6aa00640 1 -- 172.21.15.83:0/2683689628 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4d6c100c70 msgr2=0x7f4d6c1a06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:02.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.653+0000 7f4d6aa00640 1 --2- 172.21.15.83:0/2683689628 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4d6c100c70 0x7f4d6c1a06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:02.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.653+0000 7f4d6aa00640 1 -- 172.21.15.83:0/2683689628 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d6c10e820 msgr2=0x7f4d6c19a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:02.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.653+0000 7f4d6aa00640 1 --2- 172.21.15.83:0/2683689628 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d6c10e820 0x7f4d6c19a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:02.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.653+0000 7f4d6aa00640 1 -- 172.21.15.83:0/2683689628 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d6c19b020 con 0x7f4d6c101620 2024-04-24T08:59:02.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.653+0000 7f4d6b400640 1 --2- 172.21.15.83:0/2683689628 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4d6c100c70 0x7f4d6c1a06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:59:02.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.653+0000 7f4d6aa00640 1 --2- 172.21.15.83:0/2683689628 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d6c101620 0x7f4d6c1a0be0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f4d5400d950 tx=0x7f4d5400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:02.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.654+0000 7f4d63e00640 1 -- 172.21.15.83:0/2683689628 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d54017070 con 0x7f4d6c101620 2024-04-24T08:59:02.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.654+0000 7f4d70c00640 1 -- 172.21.15.83:0/2683689628 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d6c19b310 con 0x7f4d6c101620 2024-04-24T08:59:02.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.654+0000 7f4d63e00640 1 -- 172.21.15.83:0/2683689628 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4d54009c30 con 0x7f4d6c101620 2024-04-24T08:59:02.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.654+0000 7f4d70c00640 1 -- 172.21.15.83:0/2683689628 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d6c1a7620 con 0x7f4d6c101620 2024-04-24T08:59:02.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.654+0000 7f4d63e00640 1 -- 172.21.15.83:0/2683689628 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d54020060 con 0x7f4d6c101620 2024-04-24T08:59:02.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.657+0000 7f4d75041740 1 Processor -- start 2024-04-24T08:59:02.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.658+0000 7f4d63e00640 1 -- 172.21.15.83:0/2683689628 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4d540109a0 con 0x7f4d6c101620 2024-04-24T08:59:02.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.658+0000 7f4d75041740 1 -- start start 2024-04-24T08:59:02.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.658+0000 7f4d75041740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b9da38ab40 0x55b9da38af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:02.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.658+0000 7f4d75041740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b9da38b460 0x55b9da3955f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:02.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.658+0000 7f4d75041740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9da395b30 0x55b9da397f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:02.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.658+0000 7f4d75041740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55b9da2f01d0 con 0x55b9da38b460 2024-04-24T08:59:02.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.658+0000 7f4d75041740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b9da296450 con 0x55b9da395b30 2024-04-24T08:59:02.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.658+0000 7f4d75041740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55b9da2ec190 con 0x55b9da38ab40 2024-04-24T08:59:02.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.658+0000 7f4d6b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b9da38b460 0x55b9da3955f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:02.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.658+0000 7f4d6b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b9da38b460 0x55b9da3955f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41152/0 (socket says 172.21.15.83:41152) 2024-04-24T08:59:02.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.658+0000 7f4d6b400640 1 -- 172.21.15.83:0/2262751365 learned_addr learned my addr 172.21.15.83:0/2262751365 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:59:02.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.659+0000 7f4d6be00640 1 --2- 172.21.15.83:0/2262751365 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b9da38ab40 0x55b9da38af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:02.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.659+0000 7f4d6b400640 1 -- 172.21.15.83:0/2262751365 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b9da38ab40 msgr2=0x55b9da38af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:02.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.659+0000 7f4d6b400640 1 --2- 172.21.15.83:0/2262751365 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b9da38ab40 0x55b9da38af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:02.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.659+0000 7f4d6b400640 1 -- 172.21.15.83:0/2262751365 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9da395b30 msgr2=0x55b9da397f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:02.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.659+0000 7f4d6b400640 1 --2- 172.21.15.83:0/2262751365 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9da395b30 0x55b9da397f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:02.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.659+0000 7f4d6b400640 1 -- 172.21.15.83:0/2262751365 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9da353310 con 0x55b9da38b460 2024-04-24T08:59:02.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.659+0000 7f4d63e00640 1 --2- 172.21.15.83:0/2683689628 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f4d500a3d20 0x7f4d500a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:02.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.659+0000 7f4d6b400640 1 --2- 172.21.15.83:0/2262751365 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b9da38b460 0x55b9da3955f0 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c00bd40 tx=0x7f4d5c00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e908af0ece33652 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:02.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.659+0000 7f4d63e00640 1 -- 172.21.15.83:0/2683689628 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9567+0+0 (secure 0 0 0) 0x7f4d540d08d0 con 0x7f4d6c101620 2024-04-24T08:59:02.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.659+0000 7f4d6be00640 1 --2- 172.21.15.83:0/2683689628 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f4d500a3d20 0x7f4d500a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:02.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.659+0000 7f4d5be00640 1 -- 172.21.15.83:0/2262751365 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d5c015070 con 0x55b9da38b460 2024-04-24T08:59:02.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.660+0000 7f4d5be00640 1 -- 172.21.15.83:0/2262751365 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4d5c0118f0 con 0x55b9da38b460 2024-04-24T08:59:02.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.660+0000 7f4d5be00640 1 -- 172.21.15.83:0/2262751365 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d5c010870 con 0x55b9da38b460 2024-04-24T08:59:02.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.660+0000 7f4d75041740 1 -- 172.21.15.83:0/2262751365 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b9da38b460 msgr2=0x55b9da3955f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:02.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.660+0000 7f4d6be00640 1 --2- 172.21.15.83:0/2683689628 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f4d500a3d20 0x7f4d500a61e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f4d6402f3a0 tx=0x7f4d64005d80 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:02.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.660+0000 7f4d75041740 1 --2- 172.21.15.83:0/2262751365 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b9da38b460 0x55b9da3955f0 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c00bd40 tx=0x7f4d5c00f730 comp rx=0 tx=0).stop 2024-04-24T08:59:02.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.661+0000 7f4d75041740 1 -- 172.21.15.83:0/2262751365 shutdown_connections 2024-04-24T08:59:02.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.661+0000 7f4d75041740 1 --2- 172.21.15.83:0/2262751365 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9da395b30 0x55b9da397f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:02.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.661+0000 7f4d75041740 1 --2- 172.21.15.83:0/2262751365 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b9da38b460 0x55b9da3955f0 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:02.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.661+0000 7f4d75041740 1 --2- 172.21.15.83:0/2262751365 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b9da38ab40 0x55b9da38af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:02.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.661+0000 7f4d75041740 1 -- 172.21.15.83:0/2262751365 >> 172.21.15.83:0/2262751365 conn(0x55b9da386e60 msgr2=0x55b9da387240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:02.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.661+0000 7f4d75041740 1 -- 172.21.15.83:0/2262751365 shutdown_connections 2024-04-24T08:59:02.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.661+0000 7f4d75041740 1 -- 172.21.15.83:0/2262751365 wait complete. 2024-04-24T08:59:02.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.661+0000 7f4d75041740 1 -- start start 2024-04-24T08:59:02.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.662+0000 7f4d75041740 1 Processor -- start 2024-04-24T08:59:02.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.662+0000 7f4d75041740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b9da38ab40 0x55b9da3955e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:02.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.662+0000 7f4d75041740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b9da38b460 0x55b9da3928d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:02.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.662+0000 7f4d75041740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9da395b30 0x55b9da393010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:02.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.662+0000 7f4d75041740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55b9da3989c0 con 0x55b9da38ab40 2024-04-24T08:59:02.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.662+0000 7f4d75041740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b9da2a9810 con 0x55b9da395b30 2024-04-24T08:59:02.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.662+0000 7f4d75041740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55b9da398b40 con 0x55b9da38b460 2024-04-24T08:59:02.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.662+0000 7f4d75041740 10 client.0 _ll_register_callbacks cb 0x55b9da277140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:59:02.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.663+0000 7f4d6b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b9da38ab40 0x55b9da3955e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:02.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.663+0000 7f4d6b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b9da38ab40 0x55b9da3955e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41160/0 (socket says 172.21.15.83:41160) 2024-04-24T08:59:02.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.663+0000 7f4d6b400640 1 -- 172.21.15.83:0/1042591491 learned_addr learned my addr 172.21.15.83:0/1042591491 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:59:02.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.663+0000 7f4d6aa00640 1 --2- 172.21.15.83:0/1042591491 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b9da38b460 0x55b9da3928d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:02.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.663+0000 7f4d6be00640 1 --2- 172.21.15.83:0/1042591491 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9da395b30 0x55b9da393010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:02.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.663+0000 7f4d6b400640 1 -- 172.21.15.83:0/1042591491 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b9da38b460 msgr2=0x55b9da3928d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:02.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.663+0000 7f4d6b400640 1 --2- 172.21.15.83:0/1042591491 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b9da38b460 0x55b9da3928d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:02.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.663+0000 7f4d6b400640 1 -- 172.21.15.83:0/1042591491 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9da395b30 msgr2=0x55b9da393010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:02.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.663+0000 7f4d6b400640 1 --2- 172.21.15.83:0/1042591491 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9da395b30 0x55b9da393010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:02.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.663+0000 7f4d6b400640 1 -- 172.21.15.83:0/1042591491 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9da2ba3f0 con 0x55b9da38ab40 2024-04-24T08:59:02.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.663+0000 7f4d6b400640 1 --2- 172.21.15.83:0/1042591491 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b9da38ab40 0x55b9da3955e0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c00bab0 tx=0x7f4d5c010500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec2ce1a83c75f6f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:02.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.663+0000 7f4d58c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:59:02.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.664+0000 7f4d58c00640 1 -- 172.21.15.83:0/1042591491 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d5c015040 con 0x55b9da38ab40 2024-04-24T08:59:02.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.664+0000 7f4d58c00640 1 -- 172.21.15.83:0/1042591491 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9567+0+0 (secure 0 0 0) 0x7f4d5c011ca0 con 0x55b9da38ab40 2024-04-24T08:59:02.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.664+0000 7f4d75041740 1 -- 172.21.15.83:0/1042591491 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d5c0118f0 con 0x55b9da38ab40 2024-04-24T08:59:02.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.664+0000 7f4d58c00640 1 -- 172.21.15.83:0/1042591491 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f4d5c030050 con 0x55b9da38ab40 2024-04-24T08:59:02.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.664+0000 7f4d75041740 10 client.18252 fetch_fsmap learned FSMap version 33 2024-04-24T08:59:02.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.664+0000 7f4d75041740 1 -- 172.21.15.83:0/1042591491 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55b9da0023f0 con 0x55b9da38ab40 2024-04-24T08:59:02.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.664+0000 7f4d58c00640 1 -- 172.21.15.83:0/1042591491 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4d5c0118f0 con 0x55b9da38ab40 2024-04-24T08:59:02.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.665+0000 7f4d58c00640 1 -- 172.21.15.83:0/1042591491 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f4d5c02b030 con 0x55b9da38ab40 2024-04-24T08:59:02.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.665+0000 7f4d75041740 10 client.18252 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T08:59:02.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.665+0000 7f4d75041740 10 client.18252 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-24T08:59:02.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.665+0000 7f4d75041740 20 client.18252 populate_metadata read hostname 'smithi083' 2024-04-24T08:59:02.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.665+0000 7f4d75041740 1 --2- 172.21.15.83:0/1042591491 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55b9da3a5c80 0x55b9da3a8070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:02.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.665+0000 7f4d75041740 4 client.18252 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:59:02.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.665+0000 7f4d75041740 1 -- 172.21.15.83:0/1042591491 --> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b9da2fdbf0 con 0x55b9da3a5c80 2024-04-24T08:59:02.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.665+0000 7f4d6aa00640 1 --2- 172.21.15.83:0/1042591491 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55b9da3a5c80 0x55b9da3a8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:02.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.666+0000 7f4d6aa00640 1 --2- 172.21.15.83:0/1042591491 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55b9da3a5c80 0x55b9da3a8070 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=94b72c0b387f01ea server_cookie=97d1d5f2432556df in_seq=0 out_seq=0 2024-04-24T08:59:02.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.666+0000 7f4d58c00640 10 client.18252 ms_handle_connect on v2:172.21.15.189:6838/2548965336 2024-04-24T08:59:02.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.668+0000 7f4d58c00640 1 -- 172.21.15.83:0/1042591491 <== mds.4 v2:172.21.15.189:6838/2548965336 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b9da2fdbf0 con 0x55b9da3a5c80 2024-04-24T08:59:02.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.668+0000 7f4d58c00640 10 client.18252 handle_command_reply: tid=0 2024-04-24T08:59:02.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.668+0000 7f4d75041740 1 client.18252 shutdown 2024-04-24T08:59:02.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.668+0000 7f4d75041740 1 -- 172.21.15.83:0/1042591491 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b9da38ab40 msgr2=0x55b9da3955e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:02.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.668+0000 7f4d75041740 1 --2- 172.21.15.83:0/1042591491 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b9da38ab40 0x55b9da3955e0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c00bab0 tx=0x7f4d5c010500 comp rx=0 tx=0).stop 2024-04-24T08:59:02.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.668+0000 7f4d75041740 1 -- 172.21.15.83:0/1042591491 shutdown_connections 2024-04-24T08:59:02.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.668+0000 7f4d75041740 1 --2- 172.21.15.83:0/1042591491 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55b9da3a5c80 0x55b9da3a8070 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:02.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.668+0000 7f4d75041740 1 --2- 172.21.15.83:0/1042591491 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9da395b30 0x55b9da393010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:02.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.668+0000 7f4d75041740 1 --2- 172.21.15.83:0/1042591491 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b9da38b460 0x55b9da3928d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:02.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.668+0000 7f4d75041740 1 --2- 172.21.15.83:0/1042591491 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b9da38ab40 0x55b9da3955e0 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:02.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.668+0000 7f4d75041740 1 -- 172.21.15.83:0/1042591491 >> 172.21.15.83:0/1042591491 conn(0x55b9da386e60 msgr2=0x55b9da38cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:02.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.668+0000 7f4d58c00640 0 client.18252 ms_handle_reset on v2:172.21.15.189:6838/2548965336 2024-04-24T08:59:02.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.668+0000 7f4d75041740 1 -- 172.21.15.83:0/1042591491 shutdown_connections 2024-04-24T08:59:02.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.669+0000 7f4d75041740 1 -- 172.21.15.83:0/1042591491 wait complete. 2024-04-24T08:59:02.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.669+0000 7f4d75041740 20 client.18252 trim_cache size 0 max 16384 2024-04-24T08:59:02.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.684+0000 7f4d75041740 1 Processor -- start 2024-04-24T08:59:02.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.684+0000 7f4d75041740 1 -- start start 2024-04-24T08:59:02.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.685+0000 7f4d75041740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b9da38ab40 0x55b9da3a18b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:02.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.685+0000 7f4d75041740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9da38b460 0x55b9da395110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:02.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.685+0000 7f4d75041740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b9da395b30 0x55b9da3ad6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:02.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.685+0000 7f4d75041740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4d6c1b1750 con 0x55b9da38ab40 2024-04-24T08:59:02.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.685+0000 7f4d75041740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4d6c1b3290 con 0x55b9da38b460 2024-04-24T08:59:02.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.685+0000 7f4d75041740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4d6c1b1e20 con 0x55b9da395b30 2024-04-24T08:59:02.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.685+0000 7f4d6b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b9da38ab40 0x55b9da3a18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:02.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.685+0000 7f4d6b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b9da38ab40 0x55b9da3a18b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41176/0 (socket says 172.21.15.83:41176) 2024-04-24T08:59:02.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.685+0000 7f4d6b400640 1 -- 172.21.15.83:0/457201561 learned_addr learned my addr 172.21.15.83:0/457201561 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:59:02.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.685+0000 7f4d6aa00640 1 --2- 172.21.15.83:0/457201561 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9da38b460 0x55b9da395110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:02.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.685+0000 7f4d6be00640 1 --2- 172.21.15.83:0/457201561 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b9da395b30 0x55b9da3ad6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:02.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.685+0000 7f4d6b400640 1 -- 172.21.15.83:0/457201561 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b9da395b30 msgr2=0x55b9da3ad6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:02.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.686+0000 7f4d6b400640 1 --2- 172.21.15.83:0/457201561 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b9da395b30 0x55b9da3ad6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:02.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.686+0000 7f4d6b400640 1 -- 172.21.15.83:0/457201561 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9da38b460 msgr2=0x55b9da395110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:02.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.686+0000 7f4d6b400640 1 --2- 172.21.15.83:0/457201561 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9da38b460 0x55b9da395110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:02.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.686+0000 7f4d6b400640 1 -- 172.21.15.83:0/457201561 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9da38b290 con 0x55b9da38ab40 2024-04-24T08:59:02.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.686+0000 7f4d6aa00640 1 --2- 172.21.15.83:0/457201561 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9da38b460 0x55b9da395110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:59:02.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.686+0000 7f4d6be00640 1 --2- 172.21.15.83:0/457201561 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b9da395b30 0x55b9da3ad6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:59:02.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.686+0000 7f4d6b400640 1 --2- 172.21.15.83:0/457201561 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b9da38ab40 0x55b9da3a18b0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c010450 tx=0x7f4d5c005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=330585b11302efcc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:02.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.686+0000 7f4d5b400640 1 -- 172.21.15.83:0/457201561 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d5c039070 con 0x55b9da38ab40 2024-04-24T08:59:02.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.686+0000 7f4d5b400640 1 -- 172.21.15.83:0/457201561 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4d5c01a9f0 con 0x55b9da38ab40 2024-04-24T08:59:02.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.686+0000 7f4d5b400640 1 -- 172.21.15.83:0/457201561 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d5c011cb0 con 0x55b9da38ab40 2024-04-24T08:59:02.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.687+0000 7f4d75041740 1 -- 172.21.15.83:0/457201561 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b9da38ab40 msgr2=0x55b9da3a18b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:02.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.687+0000 7f4d75041740 1 --2- 172.21.15.83:0/457201561 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b9da38ab40 0x55b9da3a18b0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c010450 tx=0x7f4d5c005850 comp rx=0 tx=0).stop 2024-04-24T08:59:02.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.687+0000 7f4d75041740 1 -- 172.21.15.83:0/457201561 shutdown_connections 2024-04-24T08:59:02.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.687+0000 7f4d75041740 1 --2- 172.21.15.83:0/457201561 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b9da395b30 0x55b9da3ad6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:02.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.687+0000 7f4d75041740 1 --2- 172.21.15.83:0/457201561 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9da38b460 0x55b9da395110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:02.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.687+0000 7f4d75041740 1 --2- 172.21.15.83:0/457201561 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b9da38ab40 0x55b9da3a18b0 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:02.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.687+0000 7f4d75041740 1 -- 172.21.15.83:0/457201561 >> 172.21.15.83:0/457201561 conn(0x55b9da386e60 msgr2=0x55b9da38c2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:02.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.687+0000 7f4d75041740 1 -- 172.21.15.83:0/457201561 shutdown_connections 2024-04-24T08:59:02.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.687+0000 7f4d75041740 1 -- 172.21.15.83:0/457201561 wait complete. 2024-04-24T08:59:02.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.687+0000 7f4d75041740 1 -- start start 2024-04-24T08:59:02.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.688+0000 7f4d75041740 1 Processor -- start 2024-04-24T08:59:02.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.688+0000 7f4d75041740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b9da38ab40 0x55b9da395fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:02.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.688+0000 7f4d75041740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b9da38b460 0x55b9da3964f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:02.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.688+0000 7f4d75041740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9da395b30 0x55b9da396a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:02.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.688+0000 7f4d75041740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4d6c1b2bc0 con 0x55b9da38b460 2024-04-24T08:59:02.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.688+0000 7f4d75041740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4d6c1b24f0 con 0x55b9da395b30 2024-04-24T08:59:02.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.688+0000 7f4d75041740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4d6c1af730 con 0x55b9da38ab40 2024-04-24T08:59:02.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.688+0000 7f4d75041740 10 client.0 _ll_register_callbacks cb 0x7f4d5c00bbc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:59:02.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.688+0000 7f4d6aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b9da38b460 0x55b9da3964f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:02.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.688+0000 7f4d6aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b9da38b460 0x55b9da3964f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41188/0 (socket says 172.21.15.83:41188) 2024-04-24T08:59:02.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.688+0000 7f4d6aa00640 1 -- 172.21.15.83:0/3001962980 learned_addr learned my addr 172.21.15.83:0/3001962980 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:59:02.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.689+0000 7f4d6b400640 1 --2- 172.21.15.83:0/3001962980 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b9da38ab40 0x55b9da395fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:02.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.689+0000 7f4d6be00640 1 --2- 172.21.15.83:0/3001962980 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9da395b30 0x55b9da396a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:02.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.689+0000 7f4d6aa00640 1 -- 172.21.15.83:0/3001962980 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b9da38ab40 msgr2=0x55b9da395fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:02.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.689+0000 7f4d6aa00640 1 --2- 172.21.15.83:0/3001962980 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b9da38ab40 0x55b9da395fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:02.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.689+0000 7f4d6aa00640 1 -- 172.21.15.83:0/3001962980 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9da395b30 msgr2=0x55b9da396a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:02.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.689+0000 7f4d6aa00640 1 --2- 172.21.15.83:0/3001962980 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9da395b30 0x55b9da396a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:02.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.689+0000 7f4d6aa00640 1 -- 172.21.15.83:0/3001962980 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9da0cc7d0 con 0x55b9da38b460 2024-04-24T08:59:02.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.689+0000 7f4d6be00640 1 --2- 172.21.15.83:0/3001962980 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9da395b30 0x55b9da396a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:59:02.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.689+0000 7f4d6aa00640 1 --2- 172.21.15.83:0/3001962980 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b9da38b460 0x55b9da3964f0 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f4d5409c160 tx=0x7f4d54009ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7893a1a466a2b35 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:02.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.689+0000 7f4d58c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:59:02.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.689+0000 7f4d58c00640 1 -- 172.21.15.83:0/3001962980 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d54085020 con 0x55b9da38b460 2024-04-24T08:59:02.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.690+0000 7f4d58c00640 1 -- 172.21.15.83:0/3001962980 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9567+0+0 (secure 0 0 0) 0x7f4d540cde90 con 0x55b9da38b460 2024-04-24T08:59:02.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.690+0000 7f4d75041740 1 -- 172.21.15.83:0/3001962980 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d5c01a9f0 con 0x55b9da38b460 2024-04-24T08:59:02.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.690+0000 7f4d58c00640 1 -- 172.21.15.83:0/3001962980 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f4d54014040 con 0x55b9da38b460 2024-04-24T08:59:02.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.690+0000 7f4d75041740 10 client.18258 fetch_fsmap learned FSMap version 33 2024-04-24T08:59:02.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.690+0000 7f4d75041740 1 -- 172.21.15.83:0/3001962980 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55b9da39e120 con 0x55b9da38b460 2024-04-24T08:59:02.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.690+0000 7f4d58c00640 1 -- 172.21.15.83:0/3001962980 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4d5c01a9f0 con 0x55b9da38b460 2024-04-24T08:59:02.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.690+0000 7f4d58c00640 1 -- 172.21.15.83:0/3001962980 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f4d54098240 con 0x55b9da38b460 2024-04-24T08:59:02.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.690+0000 7f4d75041740 10 client.18258 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T08:59:02.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.690+0000 7f4d75041740 10 client.18258 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-24T08:59:02.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.690+0000 7f4d75041740 20 client.18258 populate_metadata read hostname 'smithi083' 2024-04-24T08:59:02.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.690+0000 7f4d75041740 1 --2- 172.21.15.83:0/3001962980 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55b9da40bf10 0x55b9da40e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:02.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.690+0000 7f4d75041740 4 client.18258 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T08:59:02.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.690+0000 7f4d75041740 1 -- 172.21.15.83:0/3001962980 --> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b9da2f7a60 con 0x55b9da40bf10 2024-04-24T08:59:02.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.691+0000 7f4d6b400640 1 --2- 172.21.15.83:0/3001962980 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55b9da40bf10 0x55b9da40e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:02.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.692+0000 7f4d6b400640 1 --2- 172.21.15.83:0/3001962980 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55b9da40bf10 0x55b9da40e300 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ee4d46eb6da4c926 server_cookie=1230e101c54564e4 in_seq=0 out_seq=0 2024-04-24T08:59:02.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.692+0000 7f4d58c00640 10 client.18258 ms_handle_connect on v2:172.21.15.189:6838/2548965336 2024-04-24T08:59:02.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.693+0000 7f4d58c00640 1 -- 172.21.15.83:0/3001962980 <== mds.4 v2:172.21.15.189:6838/2548965336 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b9da2f7a60 con 0x55b9da40bf10 2024-04-24T08:59:02.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.693+0000 7f4d58c00640 10 client.18258 handle_command_reply: tid=0 2024-04-24T08:59:02.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.693+0000 7f4d75041740 1 client.18258 shutdown 2024-04-24T08:59:02.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.693+0000 7f4d75041740 1 -- 172.21.15.83:0/3001962980 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b9da38b460 msgr2=0x55b9da3964f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:02.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.693+0000 7f4d75041740 1 --2- 172.21.15.83:0/3001962980 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b9da38b460 0x55b9da3964f0 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f4d5409c160 tx=0x7f4d54009ea0 comp rx=0 tx=0).stop 2024-04-24T08:59:02.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.693+0000 7f4d75041740 1 -- 172.21.15.83:0/3001962980 shutdown_connections 2024-04-24T08:59:02.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.693+0000 7f4d75041740 1 --2- 172.21.15.83:0/3001962980 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55b9da40bf10 0x55b9da40e300 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:02.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.693+0000 7f4d75041740 1 --2- 172.21.15.83:0/3001962980 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9da395b30 0x55b9da396a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:02.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.693+0000 7f4d75041740 1 --2- 172.21.15.83:0/3001962980 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b9da38b460 0x55b9da3964f0 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:02.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.693+0000 7f4d75041740 1 --2- 172.21.15.83:0/3001962980 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b9da38ab40 0x55b9da395fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:02.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.693+0000 7f4d75041740 1 -- 172.21.15.83:0/3001962980 >> 172.21.15.83:0/3001962980 conn(0x55b9da386e60 msgr2=0x55b9da38cb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:02.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.693+0000 7f4d58c00640 0 client.18258 ms_handle_reset on v2:172.21.15.189:6838/2548965336 2024-04-24T08:59:02.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.693+0000 7f4d75041740 1 -- 172.21.15.83:0/3001962980 shutdown_connections 2024-04-24T08:59:02.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.693+0000 7f4d75041740 1 -- 172.21.15.83:0/3001962980 wait complete. 2024-04-24T08:59:02.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.693+0000 7f4d75041740 20 client.18258 trim_cache size 0 max 16384 2024-04-24T08:59:02.697 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T08:59:02.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.694+0000 7f4d70c00640 1 -- 172.21.15.83:0/2683689628 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f4d500a3d20 msgr2=0x7f4d500a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:02.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.694+0000 7f4d70c00640 1 --2- 172.21.15.83:0/2683689628 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f4d500a3d20 0x7f4d500a61e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f4d6402f3a0 tx=0x7f4d64005d80 comp rx=0 tx=0).stop 2024-04-24T08:59:02.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.694+0000 7f4d70c00640 1 -- 172.21.15.83:0/2683689628 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d6c101620 msgr2=0x7f4d6c1a0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:02.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.694+0000 7f4d70c00640 1 --2- 172.21.15.83:0/2683689628 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d6c101620 0x7f4d6c1a0be0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f4d5400d950 tx=0x7f4d5400de10 comp rx=0 tx=0).stop 2024-04-24T08:59:02.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.694+0000 7f4d70c00640 1 -- 172.21.15.83:0/2683689628 shutdown_connections 2024-04-24T08:59:02.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.694+0000 7f4d70c00640 1 --2- 172.21.15.83:0/2683689628 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f4d500a3d20 0x7f4d500a61e0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:02.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.694+0000 7f4d70c00640 1 --2- 172.21.15.83:0/2683689628 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d6c10e820 0x7f4d6c19a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:02.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.694+0000 7f4d70c00640 1 --2- 172.21.15.83:0/2683689628 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d6c101620 0x7f4d6c1a0be0 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:02.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.694+0000 7f4d70c00640 1 --2- 172.21.15.83:0/2683689628 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4d6c100c70 0x7f4d6c1a06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:02.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.694+0000 7f4d70c00640 1 -- 172.21.15.83:0/2683689628 >> 172.21.15.83:0/2683689628 conn(0x7f4d6c0fc880 msgr2=0x7f4d6c0fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:02.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.694+0000 7f4d70c00640 1 -- 172.21.15.83:0/2683689628 shutdown_connections 2024-04-24T08:59:02.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.694+0000 7f4d70c00640 1 -- 172.21.15.83:0/2683689628 wait complete. 2024-04-24T08:59:02.715 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-24T08:59:02.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.942+0000 7f0a1e600640 1 -- 172.21.15.83:0/3845685431 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a18104320 msgr2=0x7f0a18104700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:02.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.942+0000 7f0a1e600640 1 --2- 172.21.15.83:0/3845685431 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a18104320 0x7f0a18104700 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0a10009ca0 tx=0x7f0a1002a8e0 comp rx=0 tx=0).stop 2024-04-24T08:59:02.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.943+0000 7f0a1e600640 1 -- 172.21.15.83:0/3845685431 shutdown_connections 2024-04-24T08:59:02.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.943+0000 7f0a1e600640 1 --2- 172.21.15.83:0/3845685431 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a181093f0 0x7f0a18110fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:02.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.943+0000 7f0a1e600640 1 --2- 172.21.15.83:0/3845685431 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0a18104cd0 0x7f0a18108d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:02.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.943+0000 7f0a1e600640 1 --2- 172.21.15.83:0/3845685431 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a18104320 0x7f0a18104700 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:02.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.943+0000 7f0a1e600640 1 -- 172.21.15.83:0/3845685431 >> 172.21.15.83:0/3845685431 conn(0x7f0a180ffb40 msgr2=0x7f0a18101f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:02.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.943+0000 7f0a1e600640 1 -- 172.21.15.83:0/3845685431 shutdown_connections 2024-04-24T08:59:02.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.943+0000 7f0a1e600640 1 -- 172.21.15.83:0/3845685431 wait complete. 2024-04-24T08:59:02.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.944+0000 7f0a1e600640 1 Processor -- start 2024-04-24T08:59:02.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.944+0000 7f0a1e600640 1 -- start start 2024-04-24T08:59:02.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.945+0000 7f0a1e600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0a18104320 0x7f0a181a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:02.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.945+0000 7f0a1e600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a181093f0 0x7f0a181a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:02.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.945+0000 7f0a1e600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a1819a720 0x7f0a1819aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:02.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.945+0000 7f0a1e600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0a181143a0 con 0x7f0a1819a720 2024-04-24T08:59:02.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.945+0000 7f0a1e600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0a18114220 con 0x7f0a181093f0 2024-04-24T08:59:02.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.945+0000 7f0a1e600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0a18114520 con 0x7f0a18104320 2024-04-24T08:59:02.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.945+0000 7f0a1dc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a1819a720 0x7f0a1819aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:02.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.945+0000 7f0a1dc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a1819a720 0x7f0a1819aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41212/0 (socket says 172.21.15.83:41212) 2024-04-24T08:59:02.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.945+0000 7f0a1dc00640 1 -- 172.21.15.83:0/2511820791 learned_addr learned my addr 172.21.15.83:0/2511820791 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:59:02.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.945+0000 7f0a1d200640 1 --2- 172.21.15.83:0/2511820791 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0a18104320 0x7f0a181a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:02.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.946+0000 7f0a17e00640 1 --2- 172.21.15.83:0/2511820791 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a181093f0 0x7f0a181a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:02.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.946+0000 7f0a1dc00640 1 -- 172.21.15.83:0/2511820791 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0a18104320 msgr2=0x7f0a181a0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:02.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.946+0000 7f0a1dc00640 1 --2- 172.21.15.83:0/2511820791 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0a18104320 0x7f0a181a0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:02.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.946+0000 7f0a1dc00640 1 -- 172.21.15.83:0/2511820791 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a181093f0 msgr2=0x7f0a181a0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:02.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.946+0000 7f0a1dc00640 1 --2- 172.21.15.83:0/2511820791 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a181093f0 0x7f0a181a0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:02.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.946+0000 7f0a1dc00640 1 -- 172.21.15.83:0/2511820791 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a1819b320 con 0x7f0a1819a720 2024-04-24T08:59:02.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.946+0000 7f0a1dc00640 1 --2- 172.21.15.83:0/2511820791 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a1819a720 0x7f0a1819aba0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f0a0800e970 tx=0x7f0a0800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:02.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.947+0000 7f0a15600640 1 -- 172.21.15.83:0/2511820791 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a0800ccc0 con 0x7f0a1819a720 2024-04-24T08:59:02.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.947+0000 7f0a1e600640 1 -- 172.21.15.83:0/2511820791 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a181a7400 con 0x7f0a1819a720 2024-04-24T08:59:02.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.947+0000 7f0a1e600640 1 -- 172.21.15.83:0/2511820791 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a181a7800 con 0x7f0a1819a720 2024-04-24T08:59:02.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.947+0000 7f0a15600640 1 -- 172.21.15.83:0/2511820791 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0a0800ce60 con 0x7f0a1819a720 2024-04-24T08:59:02.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.947+0000 7f0a15600640 1 -- 172.21.15.83:0/2511820791 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a08010720 con 0x7f0a1819a720 2024-04-24T08:59:02.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.951+0000 7f0a22969740 1 Processor -- start 2024-04-24T08:59:02.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.952+0000 7f0a15600640 1 -- 172.21.15.83:0/2511820791 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0a080108c0 con 0x7f0a1819a720 2024-04-24T08:59:02.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.952+0000 7f0a22969740 1 -- start start 2024-04-24T08:59:02.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.952+0000 7f0a22969740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55adb3ec9b40 0x55adb3ec9f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:02.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.952+0000 7f0a22969740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55adb3eca460 0x55adb3ed45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:02.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.952+0000 7f0a22969740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55adb3ed4b30 0x55adb3ed6f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:02.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.952+0000 7f0a22969740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55adb3e2f1d0 con 0x55adb3ec9b40 2024-04-24T08:59:02.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.952+0000 7f0a22969740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55adb3dd5450 con 0x55adb3ed4b30 2024-04-24T08:59:02.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.952+0000 7f0a22969740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55adb3e2b190 con 0x55adb3eca460 2024-04-24T08:59:02.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.953+0000 7f0a17e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55adb3ec9b40 0x55adb3ec9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:02.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.953+0000 7f0a17e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55adb3ec9b40 0x55adb3ec9f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41220/0 (socket says 172.21.15.83:41220) 2024-04-24T08:59:02.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.953+0000 7f0a17e00640 1 -- 172.21.15.83:0/3137082381 learned_addr learned my addr 172.21.15.83:0/3137082381 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:59:02.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.953+0000 7f0a1d200640 1 --2- 172.21.15.83:0/3137082381 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55adb3eca460 0x55adb3ed45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:02.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.953+0000 7f0a17e00640 1 --2- 172.21.15.83:0/3137082381 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55adb3ed4b30 0x55adb3ed6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:02.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.953+0000 7f0a17e00640 1 -- 172.21.15.83:0/3137082381 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55adb3eca460 msgr2=0x55adb3ed45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:02.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.953+0000 7f0a17e00640 1 --2- 172.21.15.83:0/3137082381 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55adb3eca460 0x55adb3ed45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:02.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.953+0000 7f0a17e00640 1 -- 172.21.15.83:0/3137082381 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55adb3ed4b30 msgr2=0x55adb3ed6f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:02.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.953+0000 7f0a17e00640 1 --2- 172.21.15.83:0/3137082381 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55adb3ed4b30 0x55adb3ed6f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:02.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.953+0000 7f0a17e00640 1 -- 172.21.15.83:0/3137082381 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55adb3e92310 con 0x55adb3ec9b40 2024-04-24T08:59:02.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.953+0000 7f0a15600640 1 --2- 172.21.15.83:0/2511820791 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f09fc0a3d20 0x7f09fc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:02.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.954+0000 7f0a17e00640 1 --2- 172.21.15.83:0/3137082381 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55adb3ec9b40 0x55adb3ec9f20 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f0a04013520 tx=0x7f0a04009760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40e193b02fc90a06 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:02.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.954+0000 7f0a15600640 1 -- 172.21.15.83:0/2511820791 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9671+0+0 (secure 0 0 0) 0x7f0a080d0fe0 con 0x7f0a1819a720 2024-04-24T08:59:02.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.954+0000 7f0a1d200640 1 --2- 172.21.15.83:0/2511820791 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f09fc0a3d20 0x7f09fc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:02.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.954+0000 7f0a0e000640 1 -- 172.21.15.83:0/3137082381 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a04013270 con 0x55adb3ec9b40 2024-04-24T08:59:02.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.954+0000 7f0a0e000640 1 -- 172.21.15.83:0/3137082381 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0a04010940 con 0x55adb3ec9b40 2024-04-24T08:59:02.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.954+0000 7f0a0e000640 1 -- 172.21.15.83:0/3137082381 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a04011970 con 0x55adb3ec9b40 2024-04-24T08:59:02.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.955+0000 7f0a22969740 1 -- 172.21.15.83:0/3137082381 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55adb3ec9b40 msgr2=0x55adb3ec9f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:02.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.955+0000 7f0a22969740 1 --2- 172.21.15.83:0/3137082381 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55adb3ec9b40 0x55adb3ec9f20 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f0a04013520 tx=0x7f0a04009760 comp rx=0 tx=0).stop 2024-04-24T08:59:02.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.955+0000 7f0a1d200640 1 --2- 172.21.15.83:0/2511820791 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f09fc0a3d20 0x7f09fc0a61e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0a10005e30 tx=0x7f0a10031f90 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:02.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.955+0000 7f0a22969740 1 -- 172.21.15.83:0/3137082381 shutdown_connections 2024-04-24T08:59:02.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.955+0000 7f0a22969740 1 --2- 172.21.15.83:0/3137082381 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55adb3ed4b30 0x55adb3ed6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:02.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.955+0000 7f0a22969740 1 --2- 172.21.15.83:0/3137082381 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55adb3eca460 0x55adb3ed45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:02.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.955+0000 7f0a22969740 1 --2- 172.21.15.83:0/3137082381 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55adb3ec9b40 0x55adb3ec9f20 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:02.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.955+0000 7f0a22969740 1 -- 172.21.15.83:0/3137082381 >> 172.21.15.83:0/3137082381 conn(0x55adb3ec5e60 msgr2=0x55adb3ec6240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:02.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.955+0000 7f0a22969740 1 -- 172.21.15.83:0/3137082381 shutdown_connections 2024-04-24T08:59:02.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.955+0000 7f0a22969740 1 -- 172.21.15.83:0/3137082381 wait complete. 2024-04-24T08:59:02.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.956+0000 7f0a22969740 1 -- start start 2024-04-24T08:59:02.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.956+0000 7f0a22969740 1 Processor -- start 2024-04-24T08:59:02.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.957+0000 7f0a22969740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55adb3ec9b40 0x55adb3ed4560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:02.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.957+0000 7f0a22969740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55adb3eca460 0x55adb3ed1850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:02.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.957+0000 7f0a22969740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55adb3ed4b30 0x55adb3ed1f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:02.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.957+0000 7f0a22969740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55adb3ed7960 con 0x55adb3eca460 2024-04-24T08:59:02.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.957+0000 7f0a22969740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55adb3de8810 con 0x55adb3ed4b30 2024-04-24T08:59:02.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.957+0000 7f0a22969740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55adb3ed7ae0 con 0x55adb3ec9b40 2024-04-24T08:59:02.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.957+0000 7f0a22969740 10 client.0 _ll_register_callbacks cb 0x55adb3db6140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:59:02.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.957+0000 7f0a1d200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55adb3ec9b40 0x55adb3ed4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:02.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.957+0000 7f0a17e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55adb3eca460 0x55adb3ed1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:02.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.957+0000 7f0a1d200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55adb3ec9b40 0x55adb3ed4560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:49148/0 (socket says 172.21.15.83:49148) 2024-04-24T08:59:02.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.957+0000 7f0a1d200640 1 -- 172.21.15.83:0/2106791021 learned_addr learned my addr 172.21.15.83:0/2106791021 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:59:02.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.957+0000 7f0a1dc00640 1 --2- 172.21.15.83:0/2106791021 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55adb3ed4b30 0x55adb3ed1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:02.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.958+0000 7f0a1d200640 1 -- 172.21.15.83:0/2106791021 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55adb3ed4b30 msgr2=0x55adb3ed1f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:02.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.958+0000 7f0a1d200640 1 --2- 172.21.15.83:0/2106791021 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55adb3ed4b30 0x55adb3ed1f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:02.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.958+0000 7f0a1d200640 1 -- 172.21.15.83:0/2106791021 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55adb3eca460 msgr2=0x55adb3ed1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:02.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.958+0000 7f0a1d200640 1 --2- 172.21.15.83:0/2106791021 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55adb3eca460 0x55adb3ed1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:02.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.958+0000 7f0a1d200640 1 -- 172.21.15.83:0/2106791021 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55adb3df93f0 con 0x55adb3ec9b40 2024-04-24T08:59:02.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.958+0000 7f0a17e00640 1 --2- 172.21.15.83:0/2106791021 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55adb3eca460 0x55adb3ed1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T08:59:02.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.958+0000 7f0a1d200640 1 --2- 172.21.15.83:0/2106791021 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55adb3ec9b40 0x55adb3ed4560 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0a10035000 tx=0x7f0a100095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=52721b71b908a01f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:02.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.958+0000 7f0a02a00640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-04-24T08:59:02.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.959+0000 7f0a02a00640 1 -- 172.21.15.83:0/2106791021 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a10003b70 con 0x55adb3ec9b40 2024-04-24T08:59:02.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.959+0000 7f0a22969740 1 -- 172.21.15.83:0/2106791021 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a04010940 con 0x55adb3ec9b40 2024-04-24T08:59:02.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.959+0000 7f0a02a00640 1 -- 172.21.15.83:0/2106791021 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9671+0+0 (secure 0 0 0) 0x7f0a10003d10 con 0x55adb3ec9b40 2024-04-24T08:59:02.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.959+0000 7f0a02a00640 1 -- 172.21.15.83:0/2106791021 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f0a100307e0 con 0x55adb3ec9b40 2024-04-24T08:59:02.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.959+0000 7f0a22969740 10 client.26605 fetch_fsmap learned FSMap version 33 2024-04-24T08:59:02.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.959+0000 7f0a22969740 1 -- 172.21.15.83:0/2106791021 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55adb3b413f0 con 0x55adb3ec9b40 2024-04-24T08:59:02.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.960+0000 7f0a02a00640 1 -- 172.21.15.83:0/2106791021 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0a04010940 con 0x55adb3ec9b40 2024-04-24T08:59:02.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.960+0000 7f0a02a00640 1 -- 172.21.15.83:0/2106791021 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f0a100045b0 con 0x55adb3ec9b40 2024-04-24T08:59:02.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.960+0000 7f0a22969740 10 client.26605 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T08:59:02.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.960+0000 7f0a22969740 10 client.26605 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-24T08:59:02.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.960+0000 7f0a22969740 20 client.26605 populate_metadata read hostname 'smithi083' 2024-04-24T08:59:02.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.960+0000 7f0a22969740 1 --2- 172.21.15.83:0/2106791021 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55adb3ee4d70 0x55adb3ee71f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:02.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.960+0000 7f0a22969740 4 client.26605 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:59:02.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.960+0000 7f0a22969740 1 -- 172.21.15.83:0/2106791021 --> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55adb3e3cbf0 con 0x55adb3ee4d70 2024-04-24T08:59:02.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.961+0000 7f0a17e00640 1 --2- 172.21.15.83:0/2106791021 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55adb3ee4d70 0x55adb3ee71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:02.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.961+0000 7f0a17e00640 1 --2- 172.21.15.83:0/2106791021 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55adb3ee4d70 0x55adb3ee71f0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8461c8d34e0a5953 server_cookie=b85f08eced0ca5d1 in_seq=0 out_seq=0 2024-04-24T08:59:02.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.961+0000 7f0a02a00640 10 client.26605 ms_handle_connect on v2:172.21.15.138:6832/1157967658 2024-04-24T08:59:02.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.962+0000 7f0a02a00640 1 -- 172.21.15.83:0/2106791021 <== mds.3 v2:172.21.15.138:6832/1157967658 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55adb3e3cbf0 con 0x55adb3ee4d70 2024-04-24T08:59:02.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.962+0000 7f0a02a00640 10 client.26605 handle_command_reply: tid=0 2024-04-24T08:59:02.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.963+0000 7f0a22969740 1 client.26605 shutdown 2024-04-24T08:59:02.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.963+0000 7f0a22969740 1 -- 172.21.15.83:0/2106791021 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55adb3ec9b40 msgr2=0x55adb3ed4560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:02.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.963+0000 7f0a22969740 1 --2- 172.21.15.83:0/2106791021 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55adb3ec9b40 0x55adb3ed4560 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0a10035000 tx=0x7f0a100095c0 comp rx=0 tx=0).stop 2024-04-24T08:59:02.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.963+0000 7f0a22969740 1 -- 172.21.15.83:0/2106791021 shutdown_connections 2024-04-24T08:59:02.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.963+0000 7f0a22969740 1 --2- 172.21.15.83:0/2106791021 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55adb3ed4b30 0x55adb3ed1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:02.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.963+0000 7f0a22969740 1 --2- 172.21.15.83:0/2106791021 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55adb3eca460 0x55adb3ed1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:02.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.963+0000 7f0a22969740 1 --2- 172.21.15.83:0/2106791021 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55adb3ee4d70 0x55adb3ee71f0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:02.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.963+0000 7f0a22969740 1 --2- 172.21.15.83:0/2106791021 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55adb3ec9b40 0x55adb3ed4560 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:02.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.963+0000 7f0a22969740 1 -- 172.21.15.83:0/2106791021 >> 172.21.15.83:0/2106791021 conn(0x55adb3ec5e60 msgr2=0x55adb3ecbb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:02.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.963+0000 7f0a02a00640 0 client.26605 ms_handle_reset on v2:172.21.15.138:6832/1157967658 2024-04-24T08:59:02.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.963+0000 7f0a22969740 1 -- 172.21.15.83:0/2106791021 shutdown_connections 2024-04-24T08:59:02.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.963+0000 7f0a22969740 1 -- 172.21.15.83:0/2106791021 wait complete. 2024-04-24T08:59:02.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.963+0000 7f0a22969740 20 client.26605 trim_cache size 0 max 16384 2024-04-24T08:59:02.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.980+0000 7f0a22969740 1 Processor -- start 2024-04-24T08:59:02.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.980+0000 7f0a22969740 1 -- start start 2024-04-24T08:59:02.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.981+0000 7f0a22969740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55adb3ec9b40 0x55adb3ee0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:02.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.981+0000 7f0a22969740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55adb3eca460 0x55adb3ed4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:02.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.981+0000 7f0a22969740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55adb3ed4b30 0x55adb3ed45d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:02.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.981+0000 7f0a22969740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f09f8010220 con 0x55adb3ed4b30 2024-04-24T08:59:02.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.981+0000 7f0a22969740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f09f8011d60 con 0x55adb3eca460 2024-04-24T08:59:02.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.981+0000 7f0a22969740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f09f80108f0 con 0x55adb3ec9b40 2024-04-24T08:59:02.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.981+0000 7f0a1dc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55adb3ed4b30 0x55adb3ed45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:02.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.981+0000 7f0a1dc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55adb3ed4b30 0x55adb3ed45d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41240/0 (socket says 172.21.15.83:41240) 2024-04-24T08:59:02.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.981+0000 7f0a1dc00640 1 -- 172.21.15.83:0/1498424040 learned_addr learned my addr 172.21.15.83:0/1498424040 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:59:02.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.981+0000 7f0a1d200640 1 --2- 172.21.15.83:0/1498424040 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55adb3ec9b40 0x55adb3ee0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:02.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.981+0000 7f0a17e00640 1 --2- 172.21.15.83:0/1498424040 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55adb3eca460 0x55adb3ed4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:02.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.981+0000 7f0a1dc00640 1 -- 172.21.15.83:0/1498424040 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55adb3ec9b40 msgr2=0x55adb3ee0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:02.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.981+0000 7f0a1dc00640 1 --2- 172.21.15.83:0/1498424040 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55adb3ec9b40 0x55adb3ee0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:02.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.982+0000 7f0a1dc00640 1 -- 172.21.15.83:0/1498424040 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55adb3eca460 msgr2=0x55adb3ed4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:02.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.982+0000 7f0a1dc00640 1 --2- 172.21.15.83:0/1498424040 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55adb3eca460 0x55adb3ed4090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:02.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.982+0000 7f0a1dc00640 1 -- 172.21.15.83:0/1498424040 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55adb3c0b7d0 con 0x55adb3ed4b30 2024-04-24T08:59:02.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.982+0000 7f0a1dc00640 1 --2- 172.21.15.83:0/1498424040 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55adb3ed4b30 0x55adb3ed45d0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f0a080994b0 tx=0x7f0a08009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdddd44665c23fd3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:02.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.982+0000 7f0a0d600640 1 -- 172.21.15.83:0/1498424040 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a080a5750 con 0x55adb3ed4b30 2024-04-24T08:59:02.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.982+0000 7f0a0d600640 1 -- 172.21.15.83:0/1498424040 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0a0809ade0 con 0x55adb3ed4b30 2024-04-24T08:59:02.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.982+0000 7f0a0d600640 1 -- 172.21.15.83:0/1498424040 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a08010bb0 con 0x55adb3ed4b30 2024-04-24T08:59:02.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.983+0000 7f0a22969740 1 -- 172.21.15.83:0/1498424040 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55adb3ed4b30 msgr2=0x55adb3ed45d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:02.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.983+0000 7f0a22969740 1 --2- 172.21.15.83:0/1498424040 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55adb3ed4b30 0x55adb3ed45d0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f0a080994b0 tx=0x7f0a08009810 comp rx=0 tx=0).stop 2024-04-24T08:59:02.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.983+0000 7f0a22969740 1 -- 172.21.15.83:0/1498424040 shutdown_connections 2024-04-24T08:59:02.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.983+0000 7f0a22969740 1 --2- 172.21.15.83:0/1498424040 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55adb3ed4b30 0x55adb3ed45d0 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:02.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.983+0000 7f0a22969740 1 --2- 172.21.15.83:0/1498424040 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55adb3eca460 0x55adb3ed4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:02.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.983+0000 7f0a22969740 1 --2- 172.21.15.83:0/1498424040 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55adb3ec9b40 0x55adb3ee0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:02.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.983+0000 7f0a22969740 1 -- 172.21.15.83:0/1498424040 >> 172.21.15.83:0/1498424040 conn(0x55adb3ec5e60 msgr2=0x55adb3eccad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:02.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.983+0000 7f0a22969740 1 -- 172.21.15.83:0/1498424040 shutdown_connections 2024-04-24T08:59:02.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.983+0000 7f0a22969740 1 -- 172.21.15.83:0/1498424040 wait complete. 2024-04-24T08:59:02.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.984+0000 7f0a22969740 1 -- start start 2024-04-24T08:59:02.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.984+0000 7f0a22969740 1 Processor -- start 2024-04-24T08:59:02.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.984+0000 7f0a22969740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55adb3ec9b40 0x55adb3ed6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:02.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.984+0000 7f0a22969740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55adb3eca460 0x55adb3ed4f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:02.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.984+0000 7f0a22969740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55adb3ed4b30 0x55adb3ed54b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:02.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.984+0000 7f0a22969740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f09f8011690 con 0x55adb3ec9b40 2024-04-24T08:59:02.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.985+0000 7f0a22969740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f09f8010fc0 con 0x55adb3ed4b30 2024-04-24T08:59:02.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.985+0000 7f0a22969740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f09f800e220 con 0x55adb3eca460 2024-04-24T08:59:02.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.985+0000 7f0a22969740 10 client.0 _ll_register_callbacks cb 0x7f0a10009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:59:02.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.985+0000 7f0a1d200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55adb3ec9b40 0x55adb3ed6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:02.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.985+0000 7f0a1d200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55adb3ec9b40 0x55adb3ed6760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41252/0 (socket says 172.21.15.83:41252) 2024-04-24T08:59:02.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.985+0000 7f0a1d200640 1 -- 172.21.15.83:0/38166621 learned_addr learned my addr 172.21.15.83:0/38166621 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:59:02.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.985+0000 7f0a1dc00640 1 --2- 172.21.15.83:0/38166621 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55adb3ed4b30 0x55adb3ed54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:02.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.985+0000 7f0a1d200640 1 -- 172.21.15.83:0/38166621 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55adb3eca460 msgr2=0x55adb3ed4f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:02.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.985+0000 7f0a17e00640 1 --2- 172.21.15.83:0/38166621 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55adb3eca460 0x55adb3ed4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:02.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.985+0000 7f0a1d200640 1 --2- 172.21.15.83:0/38166621 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55adb3eca460 0x55adb3ed4f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:02.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.985+0000 7f0a1d200640 1 -- 172.21.15.83:0/38166621 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55adb3ed4b30 msgr2=0x55adb3ed54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:02.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.985+0000 7f0a1d200640 1 --2- 172.21.15.83:0/38166621 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55adb3ed4b30 0x55adb3ed54b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:02.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.985+0000 7f0a1d200640 1 -- 172.21.15.83:0/38166621 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55adb3ed5ce0 con 0x55adb3ec9b40 2024-04-24T08:59:02.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.985+0000 7f0a17e00640 1 --2- 172.21.15.83:0/38166621 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55adb3eca460 0x55adb3ed4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:59:02.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.986+0000 7f0a1d200640 1 --2- 172.21.15.83:0/38166621 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55adb3ec9b40 0x55adb3ed6760 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f0a1002aed0 tx=0x7f0a10005c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f59a00c8cd06393 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:02.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.986+0000 7f0a02a00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:59:02.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.986+0000 7f0a02a00640 1 -- 172.21.15.83:0/38166621 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a10063070 con 0x55adb3ec9b40 2024-04-24T08:59:02.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.986+0000 7f0a02a00640 1 -- 172.21.15.83:0/38166621 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9671+0+0 (secure 0 0 0) 0x7f0a10052ac0 con 0x55adb3ec9b40 2024-04-24T08:59:02.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.986+0000 7f0a22969740 1 -- 172.21.15.83:0/38166621 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a0809ade0 con 0x55adb3ec9b40 2024-04-24T08:59:02.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.986+0000 7f0a02a00640 1 -- 172.21.15.83:0/38166621 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f0a10031470 con 0x55adb3ec9b40 2024-04-24T08:59:02.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.986+0000 7f0a22969740 10 client.18276 fetch_fsmap learned FSMap version 33 2024-04-24T08:59:02.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.986+0000 7f0a22969740 1 -- 172.21.15.83:0/38166621 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55adb3edcec0 con 0x55adb3ec9b40 2024-04-24T08:59:02.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.987+0000 7f0a02a00640 1 -- 172.21.15.83:0/38166621 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0a0809ade0 con 0x55adb3ec9b40 2024-04-24T08:59:02.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.987+0000 7f0a02a00640 1 -- 172.21.15.83:0/38166621 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f0a1005f430 con 0x55adb3ec9b40 2024-04-24T08:59:02.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.987+0000 7f0a22969740 10 client.18276 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T08:59:02.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.987+0000 7f0a22969740 10 client.18276 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-24T08:59:02.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.987+0000 7f0a22969740 20 client.18276 populate_metadata read hostname 'smithi083' 2024-04-24T08:59:02.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.987+0000 7f0a22969740 1 --2- 172.21.15.83:0/38166621 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55adb3f4b340 0x55adb3f4d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:02.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.987+0000 7f0a22969740 4 client.18276 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T08:59:02.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.987+0000 7f0a22969740 1 -- 172.21.15.83:0/38166621 --> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55adb3e36a60 con 0x55adb3f4b340 2024-04-24T08:59:02.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.988+0000 7f0a17e00640 1 --2- 172.21.15.83:0/38166621 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55adb3f4b340 0x55adb3f4d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:02.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.988+0000 7f0a17e00640 1 --2- 172.21.15.83:0/38166621 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55adb3f4b340 0x55adb3f4d730 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8f30290088e97a10 server_cookie=e286af336699783c in_seq=0 out_seq=0 2024-04-24T08:59:02.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.988+0000 7f0a02a00640 10 client.18276 ms_handle_connect on v2:172.21.15.138:6832/1157967658 2024-04-24T08:59:02.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.989+0000 7f0a02a00640 1 -- 172.21.15.83:0/38166621 <== mds.3 v2:172.21.15.138:6832/1157967658 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55adb3e36a60 con 0x55adb3f4b340 2024-04-24T08:59:02.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.989+0000 7f0a02a00640 10 client.18276 handle_command_reply: tid=0 2024-04-24T08:59:02.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.989+0000 7f0a22969740 1 client.18276 shutdown 2024-04-24T08:59:02.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.989+0000 7f0a22969740 1 -- 172.21.15.83:0/38166621 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55adb3ec9b40 msgr2=0x55adb3ed6760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:02.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.989+0000 7f0a22969740 1 --2- 172.21.15.83:0/38166621 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55adb3ec9b40 0x55adb3ed6760 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f0a1002aed0 tx=0x7f0a10005c30 comp rx=0 tx=0).stop 2024-04-24T08:59:02.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.989+0000 7f0a22969740 1 -- 172.21.15.83:0/38166621 shutdown_connections 2024-04-24T08:59:02.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.989+0000 7f0a22969740 1 --2- 172.21.15.83:0/38166621 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55adb3ed4b30 0x55adb3ed54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:02.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.989+0000 7f0a22969740 1 --2- 172.21.15.83:0/38166621 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55adb3f4b340 0x55adb3f4d730 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:02.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.989+0000 7f0a22969740 1 --2- 172.21.15.83:0/38166621 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55adb3eca460 0x55adb3ed4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:02.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.989+0000 7f0a02a00640 0 client.18276 ms_handle_reset on v2:172.21.15.138:6832/1157967658 2024-04-24T08:59:02.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.989+0000 7f0a22969740 1 --2- 172.21.15.83:0/38166621 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55adb3ec9b40 0x55adb3ed6760 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:02.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.989+0000 7f0a22969740 1 -- 172.21.15.83:0/38166621 >> 172.21.15.83:0/38166621 conn(0x55adb3ec5e60 msgr2=0x55adb3ecfcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:02.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.989+0000 7f0a22969740 1 -- 172.21.15.83:0/38166621 shutdown_connections 2024-04-24T08:59:02.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.989+0000 7f0a22969740 1 -- 172.21.15.83:0/38166621 wait complete. 2024-04-24T08:59:02.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.990+0000 7f0a22969740 20 client.18276 trim_cache size 0 max 16384 2024-04-24T08:59:02.993 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T08:59:02.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.990+0000 7f0a1e600640 1 -- 172.21.15.83:0/2511820791 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f09fc0a3d20 msgr2=0x7f09fc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:02.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.990+0000 7f0a1e600640 1 --2- 172.21.15.83:0/2511820791 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f09fc0a3d20 0x7f09fc0a61e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0a10005e30 tx=0x7f0a10031f90 comp rx=0 tx=0).stop 2024-04-24T08:59:02.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.990+0000 7f0a1e600640 1 -- 172.21.15.83:0/2511820791 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a1819a720 msgr2=0x7f0a1819aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:02.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.990+0000 7f0a1e600640 1 --2- 172.21.15.83:0/2511820791 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a1819a720 0x7f0a1819aba0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f0a0800e970 tx=0x7f0a0800ed70 comp rx=0 tx=0).stop 2024-04-24T08:59:02.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.990+0000 7f0a1e600640 1 -- 172.21.15.83:0/2511820791 shutdown_connections 2024-04-24T08:59:02.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.990+0000 7f0a1e600640 1 --2- 172.21.15.83:0/2511820791 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f09fc0a3d20 0x7f09fc0a61e0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:02.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.990+0000 7f0a1e600640 1 --2- 172.21.15.83:0/2511820791 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a1819a720 0x7f0a1819aba0 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:02.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.990+0000 7f0a1e600640 1 --2- 172.21.15.83:0/2511820791 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a181093f0 0x7f0a181a0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:02.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.990+0000 7f0a1e600640 1 --2- 172.21.15.83:0/2511820791 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0a18104320 0x7f0a181a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:02.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.990+0000 7f0a1e600640 1 -- 172.21.15.83:0/2511820791 >> 172.21.15.83:0/2511820791 conn(0x7f0a180ffb40 msgr2=0x7f0a18100f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:02.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.991+0000 7f0a1e600640 1 -- 172.21.15.83:0/2511820791 shutdown_connections 2024-04-24T08:59:02.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:02.991+0000 7f0a1e600640 1 -- 172.21.15.83:0/2511820791 wait complete. 2024-04-24T08:59:03.006 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:59:02 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/441067524' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:59:03.010 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T08:59:03.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.201+0000 7fc070e00640 1 -- 172.21.15.83:0/1352843284 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc06c109520 msgr2=0x7fc06c1110f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:03.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.201+0000 7fc070e00640 1 --2- 172.21.15.83:0/1352843284 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc06c109520 0x7fc06c1110f0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7fc064009ca0 tx=0x7fc06402a8c0 comp rx=0 tx=0).stop 2024-04-24T08:59:03.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.201+0000 7fc070e00640 1 -- 172.21.15.83:0/1352843284 shutdown_connections 2024-04-24T08:59:03.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.201+0000 7fc070e00640 1 --2- 172.21.15.83:0/1352843284 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc06c109520 0x7fc06c1110f0 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:03.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.201+0000 7fc070e00640 1 --2- 172.21.15.83:0/1352843284 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc06c104e00 0x7fc06c108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:03.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.201+0000 7fc070e00640 1 --2- 172.21.15.83:0/1352843284 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc06c104450 0x7fc06c104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:03.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.201+0000 7fc070e00640 1 -- 172.21.15.83:0/1352843284 >> 172.21.15.83:0/1352843284 conn(0x7fc06c0ffc70 msgr2=0x7fc06c102090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:03.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.201+0000 7fc070e00640 1 -- 172.21.15.83:0/1352843284 shutdown_connections 2024-04-24T08:59:03.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.201+0000 7fc070e00640 1 -- 172.21.15.83:0/1352843284 wait complete. 2024-04-24T08:59:03.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.202+0000 7fc070e00640 1 Processor -- start 2024-04-24T08:59:03.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.202+0000 7fc070e00640 1 -- start start 2024-04-24T08:59:03.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.203+0000 7fc070e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc06c104450 0x7fc06c1a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:03.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.203+0000 7fc070e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc06c109520 0x7fc06c1a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:03.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.203+0000 7fc070e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc06c19a760 0x7fc06c19abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:03.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.203+0000 7fc070e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc06c1144c0 con 0x7fc06c104450 2024-04-24T08:59:03.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.203+0000 7fc070e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc06c114340 con 0x7fc06c19a760 2024-04-24T08:59:03.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.203+0000 7fc070e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc06c114640 con 0x7fc06c109520 2024-04-24T08:59:03.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.203+0000 7fc069e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc06c104450 0x7fc06c1a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:03.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.203+0000 7fc069e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc06c104450 0x7fc06c1a06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41282/0 (socket says 172.21.15.83:41282) 2024-04-24T08:59:03.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.203+0000 7fc069e00640 1 -- 172.21.15.83:0/2312363386 learned_addr learned my addr 172.21.15.83:0/2312363386 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:59:03.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.204+0000 7fc069400640 1 --2- 172.21.15.83:0/2312363386 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc06c109520 0x7fc06c1a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:03.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.204+0000 7fc06a800640 1 --2- 172.21.15.83:0/2312363386 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc06c19a760 0x7fc06c19abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:03.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.204+0000 7fc069e00640 1 -- 172.21.15.83:0/2312363386 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc06c109520 msgr2=0x7fc06c1a0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:03.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.204+0000 7fc069e00640 1 --2- 172.21.15.83:0/2312363386 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc06c109520 0x7fc06c1a0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:03.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.204+0000 7fc069e00640 1 -- 172.21.15.83:0/2312363386 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc06c19a760 msgr2=0x7fc06c19abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:03.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.204+0000 7fc069e00640 1 --2- 172.21.15.83:0/2312363386 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc06c19a760 0x7fc06c19abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:03.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.204+0000 7fc069e00640 1 -- 172.21.15.83:0/2312363386 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc06c19b360 con 0x7fc06c104450 2024-04-24T08:59:03.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.204+0000 7fc069400640 1 --2- 172.21.15.83:0/2312363386 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc06c109520 0x7fc06c1a0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:59:03.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.204+0000 7fc069e00640 1 --2- 172.21.15.83:0/2312363386 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc06c104450 0x7fc06c1a06a0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7fc05c00d950 tx=0x7fc05c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:03.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.204+0000 7fc062a00640 1 -- 172.21.15.83:0/2312363386 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc05c017070 con 0x7fc06c104450 2024-04-24T08:59:03.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.204+0000 7fc062a00640 1 -- 172.21.15.83:0/2312363386 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc05c009c30 con 0x7fc06c104450 2024-04-24T08:59:03.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.204+0000 7fc070e00640 1 -- 172.21.15.83:0/2312363386 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc06c1a7440 con 0x7fc06c104450 2024-04-24T08:59:03.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.205+0000 7fc062a00640 1 -- 172.21.15.83:0/2312363386 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc05c00f620 con 0x7fc06c104450 2024-04-24T08:59:03.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.205+0000 7fc070e00640 1 -- 172.21.15.83:0/2312363386 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc06c1a7840 con 0x7fc06c104450 2024-04-24T08:59:03.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:59:02 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/441067524' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:59:03.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:59:02 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/441067524' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:59:03.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.208+0000 7fc062a00640 1 -- 172.21.15.83:0/2312363386 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc05c0109a0 con 0x7fc06c104450 2024-04-24T08:59:03.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.208+0000 7fc073d66740 1 Processor -- start 2024-04-24T08:59:03.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.208+0000 7fc073d66740 1 -- start start 2024-04-24T08:59:03.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.209+0000 7fc073d66740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a53eb7a950 0x55a53eb7ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:03.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.209+0000 7fc073d66740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a53eb7b270 0x55a53eb85400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:03.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.209+0000 7fc073d66740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a53eb85940 0x55a53eb87d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:03.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.209+0000 7fc073d66740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a53eae01d0 con 0x55a53eb7a950 2024-04-24T08:59:03.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.209+0000 7fc073d66740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a53ea86450 con 0x55a53eb85940 2024-04-24T08:59:03.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.209+0000 7fc073d66740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a53eadc190 con 0x55a53eb7b270 2024-04-24T08:59:03.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.209+0000 7fc06a800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a53eb7a950 0x55a53eb7ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:03.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.209+0000 7fc06a800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a53eb7a950 0x55a53eb7ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41290/0 (socket says 172.21.15.83:41290) 2024-04-24T08:59:03.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.209+0000 7fc06a800640 1 -- 172.21.15.83:0/3214495417 learned_addr learned my addr 172.21.15.83:0/3214495417 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:59:03.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.209+0000 7fc069e00640 1 --2- 172.21.15.83:0/3214495417 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a53eb7b270 0x55a53eb85400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:03.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.209+0000 7fc069400640 1 --2- 172.21.15.83:0/3214495417 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a53eb85940 0x55a53eb87d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:03.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.210+0000 7fc06a800640 1 -- 172.21.15.83:0/3214495417 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a53eb7b270 msgr2=0x55a53eb85400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:03.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.210+0000 7fc06a800640 1 --2- 172.21.15.83:0/3214495417 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a53eb7b270 0x55a53eb85400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:03.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.210+0000 7fc06a800640 1 -- 172.21.15.83:0/3214495417 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a53eb85940 msgr2=0x55a53eb87d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:03.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.210+0000 7fc06a800640 1 --2- 172.21.15.83:0/3214495417 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a53eb85940 0x55a53eb87d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:03.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.210+0000 7fc06a800640 1 -- 172.21.15.83:0/3214495417 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a53eb43310 con 0x55a53eb7a950 2024-04-24T08:59:03.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.210+0000 7fc062a00640 1 --2- 172.21.15.83:0/2312363386 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fc0500a3d20 0x7fc0500a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:03.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.210+0000 7fc06a800640 1 --2- 172.21.15.83:0/3214495417 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a53eb7a950 0x55a53eb7ad30 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7fc064005fa0 tx=0x7fc064009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a93c15b4aa91a75 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:03.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.210+0000 7fc062a00640 1 -- 172.21.15.83:0/2312363386 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9671+0+0 (secure 0 0 0) 0x7fc05c0d0990 con 0x7fc06c104450 2024-04-24T08:59:03.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.210+0000 7fc069e00640 1 --2- 172.21.15.83:0/2312363386 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fc0500a3d20 0x7fc0500a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:03.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.210+0000 7fc057400640 1 -- 172.21.15.83:0/3214495417 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc064031d40 con 0x55a53eb7a950 2024-04-24T08:59:03.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.211+0000 7fc057400640 1 -- 172.21.15.83:0/3214495417 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc064036070 con 0x55a53eb7a950 2024-04-24T08:59:03.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.211+0000 7fc057400640 1 -- 172.21.15.83:0/3214495417 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc064004330 con 0x55a53eb7a950 2024-04-24T08:59:03.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.211+0000 7fc073d66740 1 -- 172.21.15.83:0/3214495417 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a53eb7a950 msgr2=0x55a53eb7ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:03.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.211+0000 7fc069e00640 1 --2- 172.21.15.83:0/2312363386 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fc0500a3d20 0x7fc0500a61e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x55a53eb7bf10 tx=0x7fc05c098040 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:03.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.211+0000 7fc073d66740 1 --2- 172.21.15.83:0/3214495417 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a53eb7a950 0x55a53eb7ad30 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7fc064005fa0 tx=0x7fc064009a20 comp rx=0 tx=0).stop 2024-04-24T08:59:03.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.212+0000 7fc073d66740 1 -- 172.21.15.83:0/3214495417 shutdown_connections 2024-04-24T08:59:03.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.212+0000 7fc073d66740 1 --2- 172.21.15.83:0/3214495417 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a53eb85940 0x55a53eb87d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:03.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.212+0000 7fc073d66740 1 --2- 172.21.15.83:0/3214495417 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a53eb7b270 0x55a53eb85400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:03.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.212+0000 7fc073d66740 1 --2- 172.21.15.83:0/3214495417 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a53eb7a950 0x55a53eb7ad30 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:03.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.212+0000 7fc073d66740 1 -- 172.21.15.83:0/3214495417 >> 172.21.15.83:0/3214495417 conn(0x55a53eb76c70 msgr2=0x55a53eb77050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:03.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.212+0000 7fc073d66740 1 -- 172.21.15.83:0/3214495417 shutdown_connections 2024-04-24T08:59:03.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.212+0000 7fc073d66740 1 -- 172.21.15.83:0/3214495417 wait complete. 2024-04-24T08:59:03.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.212+0000 7fc073d66740 1 -- start start 2024-04-24T08:59:03.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.212+0000 7fc073d66740 1 Processor -- start 2024-04-24T08:59:03.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.213+0000 7fc073d66740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a53eb7a950 0x55a53eb85340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:03.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.213+0000 7fc073d66740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a53eb7b270 0x55a53eb82630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:03.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.213+0000 7fc073d66740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a53eb85940 0x55a53eb82c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:03.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.213+0000 7fc073d66740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a53eb88770 con 0x55a53eb85940 2024-04-24T08:59:03.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.213+0000 7fc073d66740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a53ea99810 con 0x55a53eb7b270 2024-04-24T08:59:03.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.213+0000 7fc073d66740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a53eb888f0 con 0x55a53eb7a950 2024-04-24T08:59:03.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.213+0000 7fc073d66740 10 client.0 _ll_register_callbacks cb 0x55a53ea67140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:59:03.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.213+0000 7fc06a800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a53eb85940 0x55a53eb82c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:03.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.213+0000 7fc06a800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a53eb85940 0x55a53eb82c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41294/0 (socket says 172.21.15.83:41294) 2024-04-24T08:59:03.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.213+0000 7fc06a800640 1 -- 172.21.15.83:0/4273335098 learned_addr learned my addr 172.21.15.83:0/4273335098 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:59:03.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.213+0000 7fc06a800640 1 --2- 172.21.15.83:0/4273335098 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a53eb7a950 0x55a53eb85340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:03.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.214+0000 7fc069400640 1 --2- 172.21.15.83:0/4273335098 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a53eb7b270 0x55a53eb82630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:03.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.214+0000 7fc06a800640 1 -- 172.21.15.83:0/4273335098 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a53eb7a950 msgr2=0x55a53eb85340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:03.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.214+0000 7fc06a800640 1 --2- 172.21.15.83:0/4273335098 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a53eb7a950 0x55a53eb85340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:03.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.214+0000 7fc06a800640 1 -- 172.21.15.83:0/4273335098 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a53eb7b270 msgr2=0x55a53eb82630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:03.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.214+0000 7fc06a800640 1 --2- 172.21.15.83:0/4273335098 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a53eb7b270 0x55a53eb82630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:03.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.214+0000 7fc06a800640 1 -- 172.21.15.83:0/4273335098 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a53eaaa3f0 con 0x55a53eb85940 2024-04-24T08:59:03.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.214+0000 7fc06a800640 1 --2- 172.21.15.83:0/4273335098 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a53eb85940 0x55a53eb82c40 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7fc064031f90 tx=0x7fc06403d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c773a1d0533d9333 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:03.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.214+0000 7fc04be00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:59:03.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.215+0000 7fc04be00640 1 -- 172.21.15.83:0/4273335098 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc064058060 con 0x55a53eb85940 2024-04-24T08:59:03.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.215+0000 7fc04be00640 1 -- 172.21.15.83:0/4273335098 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9671+0+0 (secure 0 0 0) 0x7fc0640040b0 con 0x55a53eb85940 2024-04-24T08:59:03.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.215+0000 7fc073d66740 1 -- 172.21.15.83:0/4273335098 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc064036070 con 0x55a53eb85940 2024-04-24T08:59:03.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.215+0000 7fc04be00640 1 -- 172.21.15.83:0/4273335098 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fc064003d80 con 0x55a53eb85940 2024-04-24T08:59:03.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.215+0000 7fc073d66740 10 client.18288 fetch_fsmap learned FSMap version 33 2024-04-24T08:59:03.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.215+0000 7fc073d66740 1 -- 172.21.15.83:0/4273335098 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55a53e7f23f0 con 0x55a53eb85940 2024-04-24T08:59:03.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.215+0000 7fc04be00640 1 -- 172.21.15.83:0/4273335098 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc064036070 con 0x55a53eb85940 2024-04-24T08:59:03.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.216+0000 7fc04be00640 1 -- 172.21.15.83:0/4273335098 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fc064030880 con 0x55a53eb85940 2024-04-24T08:59:03.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.216+0000 7fc073d66740 10 client.18288 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T08:59:03.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.216+0000 7fc073d66740 10 client.18288 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-24T08:59:03.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.216+0000 7fc073d66740 20 client.18288 populate_metadata read hostname 'smithi083' 2024-04-24T08:59:03.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.216+0000 7fc073d66740 1 --2- 172.21.15.83:0/4273335098 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55a53eb95940 0x55a53eb97d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:03.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.216+0000 7fc073d66740 4 client.18288 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:59:03.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.216+0000 7fc073d66740 1 -- 172.21.15.83:0/4273335098 --> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a53eaedbf0 con 0x55a53eb95940 2024-04-24T08:59:03.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.216+0000 7fc069400640 1 --2- 172.21.15.83:0/4273335098 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55a53eb95940 0x55a53eb97d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:03.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.217+0000 7fc069400640 1 --2- 172.21.15.83:0/4273335098 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55a53eb95940 0x55a53eb97d30 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d6f7366fa454795e server_cookie=20b8b8a97be480b6 in_seq=0 out_seq=0 2024-04-24T08:59:03.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.217+0000 7fc04be00640 10 client.18288 ms_handle_connect on v2:172.21.15.138:6834/1909219758 2024-04-24T08:59:03.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.219+0000 7fc04be00640 1 -- 172.21.15.83:0/4273335098 <== mds.2 v2:172.21.15.138:6834/1909219758 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a53eaedbf0 con 0x55a53eb95940 2024-04-24T08:59:03.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.219+0000 7fc04be00640 10 client.18288 handle_command_reply: tid=0 2024-04-24T08:59:03.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.219+0000 7fc073d66740 1 client.18288 shutdown 2024-04-24T08:59:03.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.219+0000 7fc073d66740 1 -- 172.21.15.83:0/4273335098 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a53eb85940 msgr2=0x55a53eb82c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:03.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.219+0000 7fc073d66740 1 --2- 172.21.15.83:0/4273335098 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a53eb85940 0x55a53eb82c40 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7fc064031f90 tx=0x7fc06403d040 comp rx=0 tx=0).stop 2024-04-24T08:59:03.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.219+0000 7fc073d66740 1 -- 172.21.15.83:0/4273335098 shutdown_connections 2024-04-24T08:59:03.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.219+0000 7fc073d66740 1 --2- 172.21.15.83:0/4273335098 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55a53eb95940 0x55a53eb97d30 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:03.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.219+0000 7fc073d66740 1 --2- 172.21.15.83:0/4273335098 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a53eb85940 0x55a53eb82c40 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:03.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.219+0000 7fc073d66740 1 --2- 172.21.15.83:0/4273335098 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a53eb7b270 0x55a53eb82630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:03.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.219+0000 7fc073d66740 1 --2- 172.21.15.83:0/4273335098 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a53eb7a950 0x55a53eb85340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:03.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.219+0000 7fc04be00640 0 client.18288 ms_handle_reset on v2:172.21.15.138:6834/1909219758 2024-04-24T08:59:03.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.219+0000 7fc073d66740 1 -- 172.21.15.83:0/4273335098 >> 172.21.15.83:0/4273335098 conn(0x55a53eb76c70 msgr2=0x55a53eb7c950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:03.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.220+0000 7fc073d66740 1 -- 172.21.15.83:0/4273335098 shutdown_connections 2024-04-24T08:59:03.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.220+0000 7fc073d66740 1 -- 172.21.15.83:0/4273335098 wait complete. 2024-04-24T08:59:03.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.220+0000 7fc073d66740 20 client.18288 trim_cache size 0 max 16384 2024-04-24T08:59:03.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.234+0000 7fc073d66740 1 Processor -- start 2024-04-24T08:59:03.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.234+0000 7fc073d66740 1 -- start start 2024-04-24T08:59:03.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.235+0000 7fc073d66740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a53eb7a950 0x55a53eb814e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:03.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.235+0000 7fc073d66740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a53eb7b270 0x55a53eb84e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:03.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.235+0000 7fc073d66740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a53eb85940 0x55a53eb9ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:03.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.235+0000 7fc073d66740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc06c1b18a0 con 0x55a53eb85940 2024-04-24T08:59:03.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.235+0000 7fc073d66740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc06c1b33e0 con 0x55a53eb7b270 2024-04-24T08:59:03.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.235+0000 7fc073d66740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc06c1b1f70 con 0x55a53eb7a950 2024-04-24T08:59:03.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.235+0000 7fc06a800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a53eb85940 0x55a53eb9ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:03.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.235+0000 7fc06a800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a53eb85940 0x55a53eb9ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41310/0 (socket says 172.21.15.83:41310) 2024-04-24T08:59:03.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.235+0000 7fc06a800640 1 -- 172.21.15.83:0/654591515 learned_addr learned my addr 172.21.15.83:0/654591515 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:59:03.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.235+0000 7fc069400640 1 --2- 172.21.15.83:0/654591515 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a53eb7b270 0x55a53eb84e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:03.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.235+0000 7fc06a800640 1 --2- 172.21.15.83:0/654591515 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a53eb7a950 0x55a53eb814e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:03.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.236+0000 7fc06a800640 1 -- 172.21.15.83:0/654591515 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a53eb7a950 msgr2=0x55a53eb814e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:03.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.236+0000 7fc06a800640 1 --2- 172.21.15.83:0/654591515 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a53eb7a950 0x55a53eb814e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:03.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.236+0000 7fc06a800640 1 -- 172.21.15.83:0/654591515 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a53eb7b270 msgr2=0x55a53eb84e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:03.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.236+0000 7fc06a800640 1 --2- 172.21.15.83:0/654591515 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a53eb7b270 0x55a53eb84e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:03.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.236+0000 7fc06a800640 1 -- 172.21.15.83:0/654591515 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a53e8bc7d0 con 0x55a53eb85940 2024-04-24T08:59:03.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.236+0000 7fc06a800640 1 --2- 172.21.15.83:0/654591515 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a53eb85940 0x55a53eb9ad50 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7fc0640319e0 tx=0x7fc064009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cccfca1c88806450 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:03.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.236+0000 7fc056a00640 1 -- 172.21.15.83:0/654591515 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc064058070 con 0x55a53eb85940 2024-04-24T08:59:03.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.236+0000 7fc056a00640 1 -- 172.21.15.83:0/654591515 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc064036040 con 0x55a53eb85940 2024-04-24T08:59:03.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.236+0000 7fc056a00640 1 -- 172.21.15.83:0/654591515 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0640316a0 con 0x55a53eb85940 2024-04-24T08:59:03.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.237+0000 7fc073d66740 1 -- 172.21.15.83:0/654591515 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a53eb85940 msgr2=0x55a53eb9ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:03.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.237+0000 7fc073d66740 1 --2- 172.21.15.83:0/654591515 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a53eb85940 0x55a53eb9ad50 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7fc0640319e0 tx=0x7fc064009a20 comp rx=0 tx=0).stop 2024-04-24T08:59:03.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.237+0000 7fc073d66740 1 -- 172.21.15.83:0/654591515 shutdown_connections 2024-04-24T08:59:03.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.237+0000 7fc073d66740 1 --2- 172.21.15.83:0/654591515 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a53eb85940 0x55a53eb9ad50 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:03.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.237+0000 7fc073d66740 1 --2- 172.21.15.83:0/654591515 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a53eb7b270 0x55a53eb84e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:03.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.237+0000 7fc073d66740 1 --2- 172.21.15.83:0/654591515 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a53eb7a950 0x55a53eb814e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:03.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.237+0000 7fc073d66740 1 -- 172.21.15.83:0/654591515 >> 172.21.15.83:0/654591515 conn(0x55a53eb76c70 msgr2=0x55a53eb80dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:03.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.237+0000 7fc073d66740 1 -- 172.21.15.83:0/654591515 shutdown_connections 2024-04-24T08:59:03.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.237+0000 7fc073d66740 1 -- 172.21.15.83:0/654591515 wait complete. 2024-04-24T08:59:03.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.238+0000 7fc073d66740 1 -- start start 2024-04-24T08:59:03.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.238+0000 7fc073d66740 1 Processor -- start 2024-04-24T08:59:03.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.238+0000 7fc073d66740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a53eb7a950 0x55a53eb96950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:03.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.238+0000 7fc073d66740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a53eb7b270 0x55a53eb96e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:03.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.238+0000 7fc073d66740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a53eb85940 0x55a53eb973d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:03.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.238+0000 7fc073d66740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc06c1b2d10 con 0x55a53eb7b270 2024-04-24T08:59:03.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.238+0000 7fc073d66740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc06c1b2640 con 0x55a53eb7a950 2024-04-24T08:59:03.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.238+0000 7fc073d66740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc06c1af8a0 con 0x55a53eb85940 2024-04-24T08:59:03.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.238+0000 7fc073d66740 10 client.0 _ll_register_callbacks cb 0x7fc064005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:59:03.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.239+0000 7fc069400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a53eb7b270 0x55a53eb96e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:03.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.239+0000 7fc069400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a53eb7b270 0x55a53eb96e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41322/0 (socket says 172.21.15.83:41322) 2024-04-24T08:59:03.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.239+0000 7fc069400640 1 -- 172.21.15.83:0/2809923669 learned_addr learned my addr 172.21.15.83:0/2809923669 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:59:03.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.239+0000 7fc069400640 1 -- 172.21.15.83:0/2809923669 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a53eb85940 msgr2=0x55a53eb973d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:03.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.239+0000 7fc069400640 1 --2- 172.21.15.83:0/2809923669 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a53eb85940 0x55a53eb973d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:03.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.239+0000 7fc069400640 1 -- 172.21.15.83:0/2809923669 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a53eb7a950 msgr2=0x55a53eb96950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:03.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.239+0000 7fc069400640 1 --2- 172.21.15.83:0/2809923669 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a53eb7a950 0x55a53eb96950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:03.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.239+0000 7fc069400640 1 -- 172.21.15.83:0/2809923669 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a53ebfacc0 con 0x55a53eb7b270 2024-04-24T08:59:03.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.239+0000 7fc069400640 1 --2- 172.21.15.83:0/2809923669 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a53eb7b270 0x55a53eb96e90 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7fc058000c00 tx=0x7fc058009f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76c83ce3cd0c07fe server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:03.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.239+0000 7fc04be00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:59:03.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.240+0000 7fc04be00640 1 -- 172.21.15.83:0/2809923669 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc058012ca0 con 0x55a53eb7b270 2024-04-24T08:59:03.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.240+0000 7fc073d66740 1 -- 172.21.15.83:0/2809923669 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc064036040 con 0x55a53eb7b270 2024-04-24T08:59:03.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.240+0000 7fc04be00640 1 -- 172.21.15.83:0/2809923669 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9671+0+0 (secure 0 0 0) 0x7fc058012e40 con 0x55a53eb7b270 2024-04-24T08:59:03.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.240+0000 7fc04be00640 1 -- 172.21.15.83:0/2809923669 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fc058025440 con 0x55a53eb7b270 2024-04-24T08:59:03.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.240+0000 7fc073d66740 10 client.18294 fetch_fsmap learned FSMap version 33 2024-04-24T08:59:03.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.240+0000 7fc073d66740 1 -- 172.21.15.83:0/2809923669 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55a53ebfc010 con 0x55a53eb7b270 2024-04-24T08:59:03.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.240+0000 7fc04be00640 1 -- 172.21.15.83:0/2809923669 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc064036040 con 0x55a53eb7b270 2024-04-24T08:59:03.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.240+0000 7fc04be00640 1 -- 172.21.15.83:0/2809923669 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fc058023070 con 0x55a53eb7b270 2024-04-24T08:59:03.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.240+0000 7fc073d66740 10 client.18294 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T08:59:03.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.240+0000 7fc073d66740 10 client.18294 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-24T08:59:03.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.240+0000 7fc073d66740 20 client.18294 populate_metadata read hostname 'smithi083' 2024-04-24T08:59:03.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.240+0000 7fc073d66740 1 --2- 172.21.15.83:0/2809923669 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55a53ebfc7c0 0x55a53ebfebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:03.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.240+0000 7fc073d66740 4 client.18294 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T08:59:03.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.240+0000 7fc073d66740 1 -- 172.21.15.83:0/2809923669 --> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a53eae7a60 con 0x55a53ebfc7c0 2024-04-24T08:59:03.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.241+0000 7fc06a800640 1 --2- 172.21.15.83:0/2809923669 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55a53ebfc7c0 0x55a53ebfebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:03.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.242+0000 7fc06a800640 1 --2- 172.21.15.83:0/2809923669 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55a53ebfc7c0 0x55a53ebfebe0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=60a515a269cbd234 server_cookie=f2e63eda08bb844e in_seq=0 out_seq=0 2024-04-24T08:59:03.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.242+0000 7fc04be00640 10 client.18294 ms_handle_connect on v2:172.21.15.138:6834/1909219758 2024-04-24T08:59:03.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.243+0000 7fc04be00640 1 -- 172.21.15.83:0/2809923669 <== mds.2 v2:172.21.15.138:6834/1909219758 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a53eae7a60 con 0x55a53ebfc7c0 2024-04-24T08:59:03.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.243+0000 7fc04be00640 10 client.18294 handle_command_reply: tid=0 2024-04-24T08:59:03.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.243+0000 7fc073d66740 1 client.18294 shutdown 2024-04-24T08:59:03.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.243+0000 7fc073d66740 1 -- 172.21.15.83:0/2809923669 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a53eb7b270 msgr2=0x55a53eb96e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:03.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.243+0000 7fc073d66740 1 --2- 172.21.15.83:0/2809923669 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a53eb7b270 0x55a53eb96e90 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7fc058000c00 tx=0x7fc058009f10 comp rx=0 tx=0).stop 2024-04-24T08:59:03.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.243+0000 7fc073d66740 1 -- 172.21.15.83:0/2809923669 shutdown_connections 2024-04-24T08:59:03.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.243+0000 7fc073d66740 1 --2- 172.21.15.83:0/2809923669 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55a53ebfc7c0 0x55a53ebfebe0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:03.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.243+0000 7fc073d66740 1 --2- 172.21.15.83:0/2809923669 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a53eb85940 0x55a53eb973d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:03.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.243+0000 7fc073d66740 1 --2- 172.21.15.83:0/2809923669 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a53eb7b270 0x55a53eb96e90 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:03.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.243+0000 7fc073d66740 1 --2- 172.21.15.83:0/2809923669 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a53eb7a950 0x55a53eb96950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:03.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.243+0000 7fc04be00640 0 client.18294 ms_handle_reset on v2:172.21.15.138:6834/1909219758 2024-04-24T08:59:03.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.243+0000 7fc073d66740 1 -- 172.21.15.83:0/2809923669 >> 172.21.15.83:0/2809923669 conn(0x55a53eb76c70 msgr2=0x55a53eb81660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:03.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.243+0000 7fc073d66740 1 -- 172.21.15.83:0/2809923669 shutdown_connections 2024-04-24T08:59:03.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.243+0000 7fc073d66740 1 -- 172.21.15.83:0/2809923669 wait complete. 2024-04-24T08:59:03.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.243+0000 7fc073d66740 20 client.18294 trim_cache size 0 max 16384 2024-04-24T08:59:03.247 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T08:59:03.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.244+0000 7fc070e00640 1 -- 172.21.15.83:0/2312363386 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fc0500a3d20 msgr2=0x7fc0500a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:03.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.244+0000 7fc070e00640 1 --2- 172.21.15.83:0/2312363386 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fc0500a3d20 0x7fc0500a61e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x55a53eb7bf10 tx=0x7fc05c098040 comp rx=0 tx=0).stop 2024-04-24T08:59:03.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.244+0000 7fc070e00640 1 -- 172.21.15.83:0/2312363386 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc06c104450 msgr2=0x7fc06c1a06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:03.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.244+0000 7fc070e00640 1 --2- 172.21.15.83:0/2312363386 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc06c104450 0x7fc06c1a06a0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7fc05c00d950 tx=0x7fc05c00de10 comp rx=0 tx=0).stop 2024-04-24T08:59:03.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.244+0000 7fc070e00640 1 -- 172.21.15.83:0/2312363386 shutdown_connections 2024-04-24T08:59:03.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.244+0000 7fc070e00640 1 --2- 172.21.15.83:0/2312363386 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fc0500a3d20 0x7fc0500a61e0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:03.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.244+0000 7fc070e00640 1 --2- 172.21.15.83:0/2312363386 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc06c19a760 0x7fc06c19abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:03.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.244+0000 7fc070e00640 1 --2- 172.21.15.83:0/2312363386 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc06c109520 0x7fc06c1a0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:03.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.244+0000 7fc070e00640 1 --2- 172.21.15.83:0/2312363386 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc06c104450 0x7fc06c1a06a0 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:03.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.244+0000 7fc070e00640 1 -- 172.21.15.83:0/2312363386 >> 172.21.15.83:0/2312363386 conn(0x7fc06c0ffc70 msgr2=0x7fc06c100800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:03.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.244+0000 7fc070e00640 1 -- 172.21.15.83:0/2312363386 shutdown_connections 2024-04-24T08:59:03.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.244+0000 7fc070e00640 1 -- 172.21.15.83:0/2312363386 wait complete. 2024-04-24T08:59:03.263 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T08:59:03.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.465+0000 7f190b000640 1 -- 172.21.15.83:0/2067723372 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1904102110 msgr2=0x7f19041024f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:03.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.465+0000 7f190b000640 1 --2- 172.21.15.83:0/2067723372 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1904102110 0x7f19041024f0 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f18fc009900 tx=0x7f18fc02a410 comp rx=0 tx=0).stop 2024-04-24T08:59:03.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.465+0000 7f190b000640 1 -- 172.21.15.83:0/2067723372 shutdown_connections 2024-04-24T08:59:03.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.465+0000 7f190b000640 1 --2- 172.21.15.83:0/2067723372 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19041071e0 0x7f190410edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:03.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.465+0000 7f190b000640 1 --2- 172.21.15.83:0/2067723372 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1904102ac0 0x7f1904106b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:03.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.465+0000 7f190b000640 1 --2- 172.21.15.83:0/2067723372 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1904102110 0x7f19041024f0 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:03.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.465+0000 7f190b000640 1 -- 172.21.15.83:0/2067723372 >> 172.21.15.83:0/2067723372 conn(0x7f19040fd9d0 msgr2=0x7f19040ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:03.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.465+0000 7f190b000640 1 -- 172.21.15.83:0/2067723372 shutdown_connections 2024-04-24T08:59:03.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.465+0000 7f190b000640 1 -- 172.21.15.83:0/2067723372 wait complete. 2024-04-24T08:59:03.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.466+0000 7f190b000640 1 Processor -- start 2024-04-24T08:59:03.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.466+0000 7f190b000640 1 -- start start 2024-04-24T08:59:03.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.467+0000 7f190b000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1904102110 0x7f190419e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:03.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.467+0000 7f190b000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19041071e0 0x7f190419e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:03.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.467+0000 7f190b000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19041985d0 0x7f1904198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:03.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.467+0000 7f190b000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f19041121f0 con 0x7f19041071e0 2024-04-24T08:59:03.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.467+0000 7f190b000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1904112070 con 0x7f19041985d0 2024-04-24T08:59:03.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.467+0000 7f190b000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1904112370 con 0x7f1904102110 2024-04-24T08:59:03.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.467+0000 7f1909200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19041071e0 0x7f190419e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:03.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.467+0000 7f1909200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19041071e0 0x7f190419e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41328/0 (socket says 172.21.15.83:41328) 2024-04-24T08:59:03.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.467+0000 7f1909200640 1 -- 172.21.15.83:0/3366139006 learned_addr learned my addr 172.21.15.83:0/3366139006 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:59:03.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.467+0000 7f1909c00640 1 --2- 172.21.15.83:0/3366139006 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1904102110 0x7f190419e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:03.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.467+0000 7f190a600640 1 --2- 172.21.15.83:0/3366139006 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19041985d0 0x7f1904198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:03.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.467+0000 7f1909200640 1 -- 172.21.15.83:0/3366139006 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1904102110 msgr2=0x7f190419e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:03.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.467+0000 7f1909200640 1 --2- 172.21.15.83:0/3366139006 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1904102110 0x7f190419e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:03.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.467+0000 7f1909200640 1 -- 172.21.15.83:0/3366139006 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19041985d0 msgr2=0x7f1904198a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:03.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.468+0000 7f1909200640 1 --2- 172.21.15.83:0/3366139006 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19041985d0 0x7f1904198a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:03.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.468+0000 7f1909200640 1 -- 172.21.15.83:0/3366139006 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19041990f0 con 0x7f19041071e0 2024-04-24T08:59:03.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.468+0000 7f190a600640 1 --2- 172.21.15.83:0/3366139006 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19041985d0 0x7f1904198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:59:03.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.468+0000 7f1909200640 1 --2- 172.21.15.83:0/3366139006 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19041071e0 0x7f190419e960 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f18f000b450 tx=0x7f18f000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:03.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.468+0000 7f1902000640 1 -- 172.21.15.83:0/3366139006 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18f0015020 con 0x7f19041071e0 2024-04-24T08:59:03.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.468+0000 7f1902000640 1 -- 172.21.15.83:0/3366139006 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f18f0011d40 con 0x7f19041071e0 2024-04-24T08:59:03.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.468+0000 7f190b000640 1 -- 172.21.15.83:0/3366139006 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19041993e0 con 0x7f19041071e0 2024-04-24T08:59:03.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.468+0000 7f1902000640 1 -- 172.21.15.83:0/3366139006 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18f000f340 con 0x7f19041071e0 2024-04-24T08:59:03.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.468+0000 7f190b000640 1 -- 172.21.15.83:0/3366139006 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19041a55e0 con 0x7f19041071e0 2024-04-24T08:59:03.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.472+0000 7f190f3dc740 1 Processor -- start 2024-04-24T08:59:03.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.472+0000 7f190f3dc740 1 -- start start 2024-04-24T08:59:03.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.473+0000 7f1902000640 1 -- 172.21.15.83:0/3366139006 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f18f0018020 con 0x7f19041071e0 2024-04-24T08:59:03.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.473+0000 7f190f3dc740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610e8a19b40 0x5610e8a19f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:03.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.473+0000 7f190f3dc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610e8a1a460 0x5610e8a245f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:03.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.473+0000 7f190f3dc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610e8a24b30 0x5610e8a26f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:03.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.473+0000 7f190f3dc740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5610e897f1d0 con 0x5610e8a24b30 2024-04-24T08:59:03.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.473+0000 7f190f3dc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5610e8925450 con 0x5610e8a1a460 2024-04-24T08:59:03.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.473+0000 7f190f3dc740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5610e897b190 con 0x5610e8a19b40 2024-04-24T08:59:03.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.473+0000 7f1909200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610e8a24b30 0x5610e8a26f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:03.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.473+0000 7f1909200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610e8a24b30 0x5610e8a26f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41342/0 (socket says 172.21.15.83:41342) 2024-04-24T08:59:03.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.473+0000 7f1909200640 1 -- 172.21.15.83:0/2245626501 learned_addr learned my addr 172.21.15.83:0/2245626501 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:59:03.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.473+0000 7f1909c00640 1 --2- 172.21.15.83:0/2245626501 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610e8a1a460 0x5610e8a245f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:03.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.473+0000 7f190a600640 1 --2- 172.21.15.83:0/2245626501 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610e8a19b40 0x5610e8a19f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:03.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.474+0000 7f1909200640 1 -- 172.21.15.83:0/2245626501 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610e8a19b40 msgr2=0x5610e8a19f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:03.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.474+0000 7f1909200640 1 --2- 172.21.15.83:0/2245626501 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610e8a19b40 0x5610e8a19f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:03.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.474+0000 7f1909200640 1 -- 172.21.15.83:0/2245626501 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610e8a1a460 msgr2=0x5610e8a245f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:03.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.474+0000 7f1909200640 1 --2- 172.21.15.83:0/2245626501 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610e8a1a460 0x5610e8a245f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:03.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.474+0000 7f1909200640 1 -- 172.21.15.83:0/2245626501 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610e89e2310 con 0x5610e8a24b30 2024-04-24T08:59:03.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.474+0000 7f1909200640 1 --2- 172.21.15.83:0/2245626501 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610e8a24b30 0x5610e8a26f20 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f18f00cda40 tx=0x7f18f00d6db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79a6efc596f5249e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:03.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.474+0000 7f1902000640 1 --2- 172.21.15.83:0/3366139006 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f18e80a3d20 0x7f18e80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:03.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.474+0000 7f18faa00640 1 -- 172.21.15.83:0/2245626501 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18f00985c0 con 0x5610e8a24b30 2024-04-24T08:59:03.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.474+0000 7f1902000640 1 -- 172.21.15.83:0/3366139006 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9671+0+0 (secure 0 0 0) 0x7f18f00d06f0 con 0x7f19041071e0 2024-04-24T08:59:03.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.475+0000 7f18faa00640 1 -- 172.21.15.83:0/2245626501 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f18f00dd070 con 0x5610e8a24b30 2024-04-24T08:59:03.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.475+0000 7f190a600640 1 --2- 172.21.15.83:0/3366139006 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f18e80a3d20 0x7f18e80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:03.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.475+0000 7f18faa00640 1 -- 172.21.15.83:0/2245626501 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18f009b770 con 0x5610e8a24b30 2024-04-24T08:59:03.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.475+0000 7f190f3dc740 1 -- 172.21.15.83:0/2245626501 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610e8a24b30 msgr2=0x5610e8a26f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:03.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.475+0000 7f190f3dc740 1 --2- 172.21.15.83:0/2245626501 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610e8a24b30 0x5610e8a26f20 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f18f00cda40 tx=0x7f18f00d6db0 comp rx=0 tx=0).stop 2024-04-24T08:59:03.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.475+0000 7f190a600640 1 --2- 172.21.15.83:0/3366139006 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f18e80a3d20 0x7f18e80a61e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f18f4005fd0 tx=0x7f18f40083f0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:03.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.476+0000 7f190f3dc740 1 -- 172.21.15.83:0/2245626501 shutdown_connections 2024-04-24T08:59:03.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.476+0000 7f190f3dc740 1 --2- 172.21.15.83:0/2245626501 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610e8a24b30 0x5610e8a26f20 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:03.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.476+0000 7f190f3dc740 1 --2- 172.21.15.83:0/2245626501 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610e8a1a460 0x5610e8a245f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:03.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.476+0000 7f190f3dc740 1 --2- 172.21.15.83:0/2245626501 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610e8a19b40 0x5610e8a19f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:03.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.476+0000 7f190f3dc740 1 -- 172.21.15.83:0/2245626501 >> 172.21.15.83:0/2245626501 conn(0x5610e8a15e60 msgr2=0x5610e8a16240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:03.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.476+0000 7f190f3dc740 1 -- 172.21.15.83:0/2245626501 shutdown_connections 2024-04-24T08:59:03.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.476+0000 7f190f3dc740 1 -- 172.21.15.83:0/2245626501 wait complete. 2024-04-24T08:59:03.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.476+0000 7f190f3dc740 1 -- start start 2024-04-24T08:59:03.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.476+0000 7f190f3dc740 1 Processor -- start 2024-04-24T08:59:03.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.477+0000 7f190f3dc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610e8a19b40 0x5610e8a245b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:03.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.477+0000 7f190f3dc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610e8a1a460 0x5610e8a218a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:03.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.477+0000 7f190f3dc740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610e8a24b30 0x5610e8a21fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:03.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.477+0000 7f190f3dc740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5610e8a279c0 con 0x5610e8a1a460 2024-04-24T08:59:03.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.477+0000 7f190f3dc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5610e8938810 con 0x5610e8a19b40 2024-04-24T08:59:03.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.477+0000 7f190f3dc740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5610e8a27b40 con 0x5610e8a24b30 2024-04-24T08:59:03.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.477+0000 7f190f3dc740 10 client.0 _ll_register_callbacks cb 0x5610e8906140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:59:03.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.477+0000 7f1909200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610e8a1a460 0x5610e8a218a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:03.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.477+0000 7f1909200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610e8a1a460 0x5610e8a218a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41346/0 (socket says 172.21.15.83:41346) 2024-04-24T08:59:03.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.477+0000 7f1909200640 1 -- 172.21.15.83:0/1226883250 learned_addr learned my addr 172.21.15.83:0/1226883250 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:59:03.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.477+0000 7f1909200640 1 -- 172.21.15.83:0/1226883250 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610e8a24b30 msgr2=0x5610e8a21fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:03.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.477+0000 7f1909200640 1 --2- 172.21.15.83:0/1226883250 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610e8a24b30 0x5610e8a21fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:03.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.477+0000 7f1909200640 1 -- 172.21.15.83:0/1226883250 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610e8a19b40 msgr2=0x5610e8a245b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:03.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.477+0000 7f1909200640 1 --2- 172.21.15.83:0/1226883250 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610e8a19b40 0x5610e8a245b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:03.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.477+0000 7f1909200640 1 -- 172.21.15.83:0/1226883250 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610e89493f0 con 0x5610e8a1a460 2024-04-24T08:59:03.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.478+0000 7f1909200640 1 --2- 172.21.15.83:0/1226883250 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610e8a1a460 0x5610e8a218a0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f18f0004700 tx=0x7f18f00d6490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8beeac71153e06be server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:03.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.478+0000 7f18ef400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:59:03.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.478+0000 7f18ef400640 1 -- 172.21.15.83:0/1226883250 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18f009baa0 con 0x5610e8a1a460 2024-04-24T08:59:03.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.478+0000 7f18ef400640 1 -- 172.21.15.83:0/1226883250 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9671+0+0 (secure 0 0 0) 0x7f18f009a620 con 0x5610e8a1a460 2024-04-24T08:59:03.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.478+0000 7f190f3dc740 1 -- 172.21.15.83:0/1226883250 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18f00dd070 con 0x5610e8a1a460 2024-04-24T08:59:03.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.478+0000 7f18ef400640 1 -- 172.21.15.83:0/1226883250 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f18f009bc80 con 0x5610e8a1a460 2024-04-24T08:59:03.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.478+0000 7f190f3dc740 10 client.18306 fetch_fsmap learned FSMap version 33 2024-04-24T08:59:03.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.478+0000 7f190f3dc740 1 -- 172.21.15.83:0/1226883250 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5610e86913f0 con 0x5610e8a1a460 2024-04-24T08:59:03.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.478+0000 7f18ef400640 1 -- 172.21.15.83:0/1226883250 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f18f00dd070 con 0x5610e8a1a460 2024-04-24T08:59:03.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.479+0000 7f18ef400640 1 -- 172.21.15.83:0/1226883250 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f18f009b770 con 0x5610e8a1a460 2024-04-24T08:59:03.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.479+0000 7f190f3dc740 10 client.18306 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T08:59:03.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.479+0000 7f190f3dc740 10 client.18306 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-24T08:59:03.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.479+0000 7f190f3dc740 20 client.18306 populate_metadata read hostname 'smithi083' 2024-04-24T08:59:03.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.479+0000 7f190f3dc740 1 --2- 172.21.15.83:0/1226883250 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x5610e8a38fe0 0x5610e8a3b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:03.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.479+0000 7f190f3dc740 4 client.18306 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:59:03.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.479+0000 7f190f3dc740 1 -- 172.21.15.83:0/1226883250 --> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5610e898cbf0 con 0x5610e8a38fe0 2024-04-24T08:59:03.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.479+0000 7f1909c00640 1 --2- 172.21.15.83:0/1226883250 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x5610e8a38fe0 0x5610e8a3b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:03.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.481+0000 7f1909c00640 1 --2- 172.21.15.83:0/1226883250 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x5610e8a38fe0 0x5610e8a3b3d0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ce4ad673ed623a6 server_cookie=17147b2726f1293f in_seq=0 out_seq=0 2024-04-24T08:59:03.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.481+0000 7f18ef400640 10 client.18306 ms_handle_connect on v2:172.21.15.138:6836/900566754 2024-04-24T08:59:03.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.483+0000 7f18ef400640 1 -- 172.21.15.83:0/1226883250 <== mds.1 v2:172.21.15.138:6836/900566754 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5610e898cbf0 con 0x5610e8a38fe0 2024-04-24T08:59:03.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.483+0000 7f18ef400640 10 client.18306 handle_command_reply: tid=0 2024-04-24T08:59:03.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.484+0000 7f190f3dc740 1 client.18306 shutdown 2024-04-24T08:59:03.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.484+0000 7f190f3dc740 1 -- 172.21.15.83:0/1226883250 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610e8a1a460 msgr2=0x5610e8a218a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:03.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.484+0000 7f190f3dc740 1 --2- 172.21.15.83:0/1226883250 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610e8a1a460 0x5610e8a218a0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f18f0004700 tx=0x7f18f00d6490 comp rx=0 tx=0).stop 2024-04-24T08:59:03.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.484+0000 7f190f3dc740 1 -- 172.21.15.83:0/1226883250 shutdown_connections 2024-04-24T08:59:03.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.484+0000 7f190f3dc740 1 --2- 172.21.15.83:0/1226883250 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x5610e8a38fe0 0x5610e8a3b3d0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:03.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.484+0000 7f190f3dc740 1 --2- 172.21.15.83:0/1226883250 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610e8a24b30 0x5610e8a21fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:03.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.484+0000 7f190f3dc740 1 --2- 172.21.15.83:0/1226883250 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610e8a1a460 0x5610e8a218a0 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:03.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.484+0000 7f18ef400640 0 client.18306 ms_handle_reset on v2:172.21.15.138:6836/900566754 2024-04-24T08:59:03.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.484+0000 7f190f3dc740 1 --2- 172.21.15.83:0/1226883250 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610e8a19b40 0x5610e8a245b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:03.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.484+0000 7f190f3dc740 1 -- 172.21.15.83:0/1226883250 >> 172.21.15.83:0/1226883250 conn(0x5610e8a15e60 msgr2=0x5610e8a1bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:03.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.484+0000 7f190f3dc740 1 -- 172.21.15.83:0/1226883250 shutdown_connections 2024-04-24T08:59:03.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.484+0000 7f190f3dc740 1 -- 172.21.15.83:0/1226883250 wait complete. 2024-04-24T08:59:03.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.484+0000 7f190f3dc740 20 client.18306 trim_cache size 0 max 16384 2024-04-24T08:59:03.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.501+0000 7f190f3dc740 1 Processor -- start 2024-04-24T08:59:03.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.501+0000 7f190f3dc740 1 -- start start 2024-04-24T08:59:03.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.501+0000 7f190f3dc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610e8a19b40 0x5610e8a240e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:03.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.501+0000 7f190f3dc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610e8a1a460 0x5610e8a1eb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:03.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.501+0000 7f190f3dc740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610e8a24b30 0x5610e8a1f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:03.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.501+0000 7f190f3dc740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f19041ad470 con 0x5610e8a1a460 2024-04-24T08:59:03.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.501+0000 7f190f3dc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f19041aefb0 con 0x5610e8a19b40 2024-04-24T08:59:03.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.501+0000 7f190f3dc740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f19041adb40 con 0x5610e8a24b30 2024-04-24T08:59:03.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.501+0000 7f1909200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610e8a1a460 0x5610e8a1eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:03.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.502+0000 7f1909200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610e8a1a460 0x5610e8a1eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41350/0 (socket says 172.21.15.83:41350) 2024-04-24T08:59:03.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.502+0000 7f1909200640 1 -- 172.21.15.83:0/1886151151 learned_addr learned my addr 172.21.15.83:0/1886151151 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:59:03.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.502+0000 7f190a600640 1 --2- 172.21.15.83:0/1886151151 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610e8a24b30 0x5610e8a1f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:03.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.502+0000 7f1909c00640 1 --2- 172.21.15.83:0/1886151151 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610e8a19b40 0x5610e8a240e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:03.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.502+0000 7f1909200640 1 -- 172.21.15.83:0/1886151151 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610e8a24b30 msgr2=0x5610e8a1f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:03.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.502+0000 7f1909200640 1 --2- 172.21.15.83:0/1886151151 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610e8a24b30 0x5610e8a1f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:03.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.502+0000 7f1909200640 1 -- 172.21.15.83:0/1886151151 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610e8a19b40 msgr2=0x5610e8a240e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:03.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.502+0000 7f1909200640 1 --2- 172.21.15.83:0/1886151151 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610e8a19b40 0x5610e8a240e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:03.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.502+0000 7f1909200640 1 -- 172.21.15.83:0/1886151151 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610e875b7d0 con 0x5610e8a1a460 2024-04-24T08:59:03.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.502+0000 7f1909200640 1 --2- 172.21.15.83:0/1886151151 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610e8a1a460 0x5610e8a1eb20 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f18f0098cf0 tx=0x7f18f00d6db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8b0a4134709395e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:03.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.503+0000 7f18fa000640 1 -- 172.21.15.83:0/1886151151 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18f009be60 con 0x5610e8a1a460 2024-04-24T08:59:03.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.503+0000 7f18fa000640 1 -- 172.21.15.83:0/1886151151 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f18f00dd070 con 0x5610e8a1a460 2024-04-24T08:59:03.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.503+0000 7f18fa000640 1 -- 172.21.15.83:0/1886151151 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18f009c0f0 con 0x5610e8a1a460 2024-04-24T08:59:03.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.504+0000 7f190f3dc740 1 -- 172.21.15.83:0/1886151151 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610e8a1a460 msgr2=0x5610e8a1eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:03.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.504+0000 7f190f3dc740 1 --2- 172.21.15.83:0/1886151151 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610e8a1a460 0x5610e8a1eb20 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f18f0098cf0 tx=0x7f18f00d6db0 comp rx=0 tx=0).stop 2024-04-24T08:59:03.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.504+0000 7f190f3dc740 1 -- 172.21.15.83:0/1886151151 shutdown_connections 2024-04-24T08:59:03.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.504+0000 7f190f3dc740 1 --2- 172.21.15.83:0/1886151151 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610e8a24b30 0x5610e8a1f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:03.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.504+0000 7f190f3dc740 1 --2- 172.21.15.83:0/1886151151 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610e8a1a460 0x5610e8a1eb20 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:03.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.504+0000 7f190f3dc740 1 --2- 172.21.15.83:0/1886151151 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610e8a19b40 0x5610e8a240e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:03.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.504+0000 7f190f3dc740 1 -- 172.21.15.83:0/1886151151 >> 172.21.15.83:0/1886151151 conn(0x5610e8a15e60 msgr2=0x5610e8a34220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:03.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.504+0000 7f190f3dc740 1 -- 172.21.15.83:0/1886151151 shutdown_connections 2024-04-24T08:59:03.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.504+0000 7f190f3dc740 1 -- 172.21.15.83:0/1886151151 wait complete. 2024-04-24T08:59:03.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.504+0000 7f190f3dc740 1 -- start start 2024-04-24T08:59:03.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.504+0000 7f190f3dc740 1 Processor -- start 2024-04-24T08:59:03.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.505+0000 7f190f3dc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610e8a19b40 0x5610e8a3e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:03.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.505+0000 7f190f3dc740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610e8a1a460 0x5610e8a1e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:03.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.505+0000 7f190f3dc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610e8a24b30 0x5610e8a1e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:03.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.505+0000 7f190f3dc740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f19041ae8e0 con 0x5610e8a24b30 2024-04-24T08:59:03.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.505+0000 7f190f3dc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f19041ae210 con 0x5610e8a19b40 2024-04-24T08:59:03.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.505+0000 7f190f3dc740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f19041ab470 con 0x5610e8a1a460 2024-04-24T08:59:03.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.505+0000 7f190f3dc740 10 client.0 _ll_register_callbacks cb 0x7f18f00d6b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:59:03.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.505+0000 7f190a600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610e8a24b30 0x5610e8a1e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:03.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.505+0000 7f190a600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610e8a24b30 0x5610e8a1e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41366/0 (socket says 172.21.15.83:41366) 2024-04-24T08:59:03.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.505+0000 7f190a600640 1 -- 172.21.15.83:0/2294682227 learned_addr learned my addr 172.21.15.83:0/2294682227 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:59:03.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.506+0000 7f1909200640 1 --2- 172.21.15.83:0/2294682227 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610e8a1a460 0x5610e8a1e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:03.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.506+0000 7f1909c00640 1 --2- 172.21.15.83:0/2294682227 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610e8a19b40 0x5610e8a3e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:03.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.506+0000 7f190a600640 1 -- 172.21.15.83:0/2294682227 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610e8a1a460 msgr2=0x5610e8a1e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:03.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.506+0000 7f190a600640 1 --2- 172.21.15.83:0/2294682227 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610e8a1a460 0x5610e8a1e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:03.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.506+0000 7f190a600640 1 -- 172.21.15.83:0/2294682227 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610e8a19b40 msgr2=0x5610e8a3e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:03.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.506+0000 7f190a600640 1 --2- 172.21.15.83:0/2294682227 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610e8a19b40 0x5610e8a3e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:03.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.506+0000 7f190a600640 1 -- 172.21.15.83:0/2294682227 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610e8a1ee50 con 0x5610e8a24b30 2024-04-24T08:59:03.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.506+0000 7f1909c00640 1 --2- 172.21.15.83:0/2294682227 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610e8a19b40 0x5610e8a3e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:59:03.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.506+0000 7f190a600640 1 --2- 172.21.15.83:0/2294682227 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610e8a24b30 0x5610e8a1e710 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f18f4010e50 tx=0x7f18f4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69f4c3c560f05c85 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:03.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.506+0000 7f18ef400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:59:03.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.506+0000 7f18ef400640 1 -- 172.21.15.83:0/2294682227 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18f4027430 con 0x5610e8a24b30 2024-04-24T08:59:03.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.506+0000 7f18ef400640 1 -- 172.21.15.83:0/2294682227 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9671+0+0 (secure 0 0 0) 0x7f18f401e430 con 0x5610e8a24b30 2024-04-24T08:59:03.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.506+0000 7f190f3dc740 1 -- 172.21.15.83:0/2294682227 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18f00dd070 con 0x5610e8a24b30 2024-04-24T08:59:03.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.507+0000 7f18ef400640 1 -- 172.21.15.83:0/2294682227 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f18f401e630 con 0x5610e8a24b30 2024-04-24T08:59:03.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.507+0000 7f190f3dc740 10 client.18312 fetch_fsmap learned FSMap version 33 2024-04-24T08:59:03.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.507+0000 7f190f3dc740 1 -- 172.21.15.83:0/2294682227 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5610e8a9f160 con 0x5610e8a24b30 2024-04-24T08:59:03.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.507+0000 7f18ef400640 1 -- 172.21.15.83:0/2294682227 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f18f00dd070 con 0x5610e8a24b30 2024-04-24T08:59:03.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.507+0000 7f18ef400640 1 -- 172.21.15.83:0/2294682227 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f18f4027a20 con 0x5610e8a24b30 2024-04-24T08:59:03.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.507+0000 7f190f3dc740 10 client.18312 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T08:59:03.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.507+0000 7f190f3dc740 10 client.18312 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-24T08:59:03.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.507+0000 7f190f3dc740 20 client.18312 populate_metadata read hostname 'smithi083' 2024-04-24T08:59:03.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.507+0000 7f190f3dc740 1 --2- 172.21.15.83:0/2294682227 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x5610e8a9f960 0x5610e8aa1d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:03.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.507+0000 7f190f3dc740 4 client.18312 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T08:59:03.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.507+0000 7f190f3dc740 1 -- 172.21.15.83:0/2294682227 --> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5610e8986a60 con 0x5610e8a9f960 2024-04-24T08:59:03.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.508+0000 7f1909c00640 1 --2- 172.21.15.83:0/2294682227 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x5610e8a9f960 0x5610e8aa1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:03.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.509+0000 7f1909c00640 1 --2- 172.21.15.83:0/2294682227 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x5610e8a9f960 0x5610e8aa1d80 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a113091d4d2d2bcb server_cookie=6e23bfb5a171042f in_seq=0 out_seq=0 2024-04-24T08:59:03.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.509+0000 7f18ef400640 10 client.18312 ms_handle_connect on v2:172.21.15.138:6836/900566754 2024-04-24T08:59:03.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.509+0000 7f18ef400640 1 -- 172.21.15.83:0/2294682227 <== mds.1 v2:172.21.15.138:6836/900566754 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5610e8986a60 con 0x5610e8a9f960 2024-04-24T08:59:03.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.509+0000 7f18ef400640 10 client.18312 handle_command_reply: tid=0 2024-04-24T08:59:03.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.509+0000 7f190f3dc740 1 client.18312 shutdown 2024-04-24T08:59:03.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.509+0000 7f190f3dc740 1 -- 172.21.15.83:0/2294682227 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610e8a24b30 msgr2=0x5610e8a1e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:03.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.509+0000 7f190f3dc740 1 --2- 172.21.15.83:0/2294682227 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610e8a24b30 0x5610e8a1e710 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f18f4010e50 tx=0x7f18f4004ab0 comp rx=0 tx=0).stop 2024-04-24T08:59:03.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.509+0000 7f190f3dc740 1 -- 172.21.15.83:0/2294682227 shutdown_connections 2024-04-24T08:59:03.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.509+0000 7f190f3dc740 1 --2- 172.21.15.83:0/2294682227 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610e8a24b30 0x5610e8a1e710 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:03.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.509+0000 7f190f3dc740 1 --2- 172.21.15.83:0/2294682227 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x5610e8a9f960 0x5610e8aa1d80 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:03.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.510+0000 7f190f3dc740 1 --2- 172.21.15.83:0/2294682227 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610e8a1a460 0x5610e8a1e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:03.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.510+0000 7f190f3dc740 1 --2- 172.21.15.83:0/2294682227 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610e8a19b40 0x5610e8a3e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:03.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.510+0000 7f18ef400640 0 client.18312 ms_handle_reset on v2:172.21.15.138:6836/900566754 2024-04-24T08:59:03.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.510+0000 7f190f3dc740 1 -- 172.21.15.83:0/2294682227 >> 172.21.15.83:0/2294682227 conn(0x5610e8a15e60 msgr2=0x5610e8a20040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:03.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.510+0000 7f190f3dc740 1 -- 172.21.15.83:0/2294682227 shutdown_connections 2024-04-24T08:59:03.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.510+0000 7f190f3dc740 1 -- 172.21.15.83:0/2294682227 wait complete. 2024-04-24T08:59:03.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.510+0000 7f190f3dc740 20 client.18312 trim_cache size 0 max 16384 2024-04-24T08:59:03.514 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T08:59:03.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.510+0000 7f190b000640 1 -- 172.21.15.83:0/3366139006 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f18e80a3d20 msgr2=0x7f18e80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:03.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.510+0000 7f190b000640 1 --2- 172.21.15.83:0/3366139006 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f18e80a3d20 0x7f18e80a61e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f18f4005fd0 tx=0x7f18f40083f0 comp rx=0 tx=0).stop 2024-04-24T08:59:03.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.510+0000 7f190b000640 1 -- 172.21.15.83:0/3366139006 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19041071e0 msgr2=0x7f190419e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:03.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.510+0000 7f190b000640 1 --2- 172.21.15.83:0/3366139006 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19041071e0 0x7f190419e960 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f18f000b450 tx=0x7f18f000b910 comp rx=0 tx=0).stop 2024-04-24T08:59:03.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.511+0000 7f190b000640 1 -- 172.21.15.83:0/3366139006 shutdown_connections 2024-04-24T08:59:03.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.511+0000 7f190b000640 1 --2- 172.21.15.83:0/3366139006 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f18e80a3d20 0x7f18e80a61e0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:03.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.511+0000 7f190b000640 1 --2- 172.21.15.83:0/3366139006 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19041985d0 0x7f1904198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:03.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.511+0000 7f190b000640 1 --2- 172.21.15.83:0/3366139006 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19041071e0 0x7f190419e960 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:03.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.511+0000 7f190b000640 1 --2- 172.21.15.83:0/3366139006 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1904102110 0x7f190419e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:03.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.511+0000 7f190b000640 1 -- 172.21.15.83:0/3366139006 >> 172.21.15.83:0/3366139006 conn(0x7f19040fd9d0 msgr2=0x7f19040fed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:03.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.511+0000 7f190b000640 1 -- 172.21.15.83:0/3366139006 shutdown_connections 2024-04-24T08:59:03.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.511+0000 7f190b000640 1 -- 172.21.15.83:0/3366139006 wait complete. 2024-04-24T08:59:03.529 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T08:59:03.591 INFO:tasks.workunit.client.0.smithi083.stdout: 1 40276 0.63 MB/sec execute 217 sec latency 47.758 ms 2024-04-24T08:59:03.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.735+0000 7f2f53e00640 1 -- 172.21.15.83:0/1896050567 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f54108580 msgr2=0x7f2f54108a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:03.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.735+0000 7f2f53e00640 1 --2- 172.21.15.83:0/1896050567 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f54108580 0x7f2f54108a00 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f2f40009930 tx=0x7f2f4002a420 comp rx=0 tx=0).stop 2024-04-24T08:59:03.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.735+0000 7f2f53e00640 1 -- 172.21.15.83:0/1896050567 shutdown_connections 2024-04-24T08:59:03.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.735+0000 7f2f53e00640 1 --2- 172.21.15.83:0/1896050567 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f54108f40 0x7f2f541153b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:03.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.735+0000 7f2f53e00640 1 --2- 172.21.15.83:0/1896050567 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f54108580 0x7f2f54108a00 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:03.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.735+0000 7f2f53e00640 1 --2- 172.21.15.83:0/1896050567 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2f5410d500 0x7f2f5410d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:03.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.735+0000 7f2f53e00640 1 -- 172.21.15.83:0/1896050567 >> 172.21.15.83:0/1896050567 conn(0x7f2f54077e20 msgr2=0x7f2f54106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:03.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.735+0000 7f2f53e00640 1 -- 172.21.15.83:0/1896050567 shutdown_connections 2024-04-24T08:59:03.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.735+0000 7f2f53e00640 1 -- 172.21.15.83:0/1896050567 wait complete. 2024-04-24T08:59:03.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.736+0000 7f2f53e00640 1 Processor -- start 2024-04-24T08:59:03.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.736+0000 7f2f53e00640 1 -- start start 2024-04-24T08:59:03.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.737+0000 7f2f53e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f54108580 0x7f2f541a48d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:03.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.737+0000 7f2f53e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2f54108f40 0x7f2f541a4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:03.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.737+0000 7f2f53e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f5410d500 0x7f2f5419ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:03.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.737+0000 7f2f53e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2f541187b0 con 0x7f2f54108580 2024-04-24T08:59:03.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.737+0000 7f2f53e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2f54118630 con 0x7f2f5410d500 2024-04-24T08:59:03.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.737+0000 7f2f53e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2f54118930 con 0x7f2f54108f40 2024-04-24T08:59:03.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.737+0000 7f2f52a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f54108580 0x7f2f541a48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:03.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.737+0000 7f2f52a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f54108580 0x7f2f541a48d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41390/0 (socket says 172.21.15.83:41390) 2024-04-24T08:59:03.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.737+0000 7f2f52a00640 1 -- 172.21.15.83:0/519756766 learned_addr learned my addr 172.21.15.83:0/519756766 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:59:03.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.737+0000 7f2f52000640 1 --2- 172.21.15.83:0/519756766 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2f54108f40 0x7f2f541a4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:03.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.737+0000 7f2f53400640 1 --2- 172.21.15.83:0/519756766 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f5410d500 0x7f2f5419ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:03.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.737+0000 7f2f52a00640 1 -- 172.21.15.83:0/519756766 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2f54108f40 msgr2=0x7f2f541a4e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:03.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.737+0000 7f2f52a00640 1 --2- 172.21.15.83:0/519756766 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2f54108f40 0x7f2f541a4e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:03.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.738+0000 7f2f52a00640 1 -- 172.21.15.83:0/519756766 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f5410d500 msgr2=0x7f2f5419ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:03.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.738+0000 7f2f52a00640 1 --2- 172.21.15.83:0/519756766 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f5410d500 0x7f2f5419ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:03.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.738+0000 7f2f52a00640 1 -- 172.21.15.83:0/519756766 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f5419f250 con 0x7f2f54108580 2024-04-24T08:59:03.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.738+0000 7f2f53400640 1 --2- 172.21.15.83:0/519756766 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f5410d500 0x7f2f5419ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:59:03.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.738+0000 7f2f52a00640 1 --2- 172.21.15.83:0/519756766 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f54108580 0x7f2f541a48d0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f2f4400d770 tx=0x7f2f4400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:03.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.738+0000 7f2f4b400640 1 -- 172.21.15.83:0/519756766 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f44017070 con 0x7f2f54108580 2024-04-24T08:59:03.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.738+0000 7f2f4b400640 1 -- 172.21.15.83:0/519756766 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2f44009c30 con 0x7f2f54108580 2024-04-24T08:59:03.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.738+0000 7f2f53e00640 1 -- 172.21.15.83:0/519756766 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f5419f540 con 0x7f2f54108580 2024-04-24T08:59:03.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.738+0000 7f2f4b400640 1 -- 172.21.15.83:0/519756766 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f44003d00 con 0x7f2f54108580 2024-04-24T08:59:03.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.738+0000 7f2f53e00640 1 -- 172.21.15.83:0/519756766 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f541ab790 con 0x7f2f54108580 2024-04-24T08:59:03.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.742+0000 7f2f4b400640 1 -- 172.21.15.83:0/519756766 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2f44003ea0 con 0x7f2f54108580 2024-04-24T08:59:03.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.742+0000 7f2f5c9e8740 1 Processor -- start 2024-04-24T08:59:03.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.742+0000 7f2f5c9e8740 1 -- start start 2024-04-24T08:59:03.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.742+0000 7f2f5c9e8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558613e90b40 0x558613e90f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:03.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.743+0000 7f2f5c9e8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558613e91460 0x558613e9b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:03.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.743+0000 7f2f5c9e8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558613e9bb30 0x558613e9df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:03.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.743+0000 7f2f5c9e8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x558613df61d0 con 0x558613e91460 2024-04-24T08:59:03.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.743+0000 7f2f5c9e8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558613d9c450 con 0x558613e90b40 2024-04-24T08:59:03.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.743+0000 7f2f5c9e8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x558613df2190 con 0x558613e9bb30 2024-04-24T08:59:03.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.743+0000 7f2f52a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558613e91460 0x558613e9b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:03.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.743+0000 7f2f52a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558613e91460 0x558613e9b5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41396/0 (socket says 172.21.15.83:41396) 2024-04-24T08:59:03.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.743+0000 7f2f52a00640 1 -- 172.21.15.83:0/1209575348 learned_addr learned my addr 172.21.15.83:0/1209575348 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:59:03.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.743+0000 7f2f53400640 1 --2- 172.21.15.83:0/1209575348 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558613e90b40 0x558613e90f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:03.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.743+0000 7f2f52000640 1 --2- 172.21.15.83:0/1209575348 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558613e9bb30 0x558613e9df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:03.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.743+0000 7f2f52a00640 1 -- 172.21.15.83:0/1209575348 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558613e9bb30 msgr2=0x558613e9df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:03.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.743+0000 7f2f52a00640 1 --2- 172.21.15.83:0/1209575348 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558613e9bb30 0x558613e9df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:03.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.743+0000 7f2f52a00640 1 -- 172.21.15.83:0/1209575348 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558613e90b40 msgr2=0x558613e90f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:03.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.743+0000 7f2f52a00640 1 --2- 172.21.15.83:0/1209575348 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558613e90b40 0x558613e90f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:03.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.743+0000 7f2f52a00640 1 -- 172.21.15.83:0/1209575348 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558613e59310 con 0x558613e91460 2024-04-24T08:59:03.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.744+0000 7f2f52a00640 1 --2- 172.21.15.83:0/1209575348 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558613e91460 0x558613e9b5f0 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f2f440cdc90 tx=0x7f2f440d4d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65c57d4efa375679 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:03.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.744+0000 7f2f4b400640 1 --2- 172.21.15.83:0/519756766 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f2f380a3d20 0x7f2f380a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:03.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.744+0000 7f2f4b400640 1 -- 172.21.15.83:0/519756766 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9671+0+0 (secure 0 0 0) 0x7f2f440d0a50 con 0x7f2f54108580 2024-04-24T08:59:03.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.744+0000 7f2f53400640 1 --2- 172.21.15.83:0/519756766 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f2f380a3d20 0x7f2f380a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:03.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.744+0000 7f2f3fe00640 1 -- 172.21.15.83:0/1209575348 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f44098810 con 0x558613e91460 2024-04-24T08:59:03.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.744+0000 7f2f3fe00640 1 -- 172.21.15.83:0/1209575348 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2f4409cc70 con 0x558613e91460 2024-04-24T08:59:03.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.744+0000 7f2f3fe00640 1 -- 172.21.15.83:0/1209575348 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f4409bed0 con 0x558613e91460 2024-04-24T08:59:03.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.745+0000 7f2f5c9e8740 1 -- 172.21.15.83:0/1209575348 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558613e91460 msgr2=0x558613e9b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:03.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.745+0000 7f2f5c9e8740 1 --2- 172.21.15.83:0/1209575348 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558613e91460 0x558613e9b5f0 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f2f440cdc90 tx=0x7f2f440d4d10 comp rx=0 tx=0).stop 2024-04-24T08:59:03.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.745+0000 7f2f53400640 1 --2- 172.21.15.83:0/519756766 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f2f380a3d20 0x7f2f380a61e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2f4c005fd0 tx=0x7f2f4c009630 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:03.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.745+0000 7f2f5c9e8740 1 -- 172.21.15.83:0/1209575348 shutdown_connections 2024-04-24T08:59:03.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.745+0000 7f2f5c9e8740 1 --2- 172.21.15.83:0/1209575348 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558613e9bb30 0x558613e9df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:03.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.745+0000 7f2f5c9e8740 1 --2- 172.21.15.83:0/1209575348 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558613e91460 0x558613e9b5f0 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:03.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.745+0000 7f2f5c9e8740 1 --2- 172.21.15.83:0/1209575348 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558613e90b40 0x558613e90f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:03.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.745+0000 7f2f5c9e8740 1 -- 172.21.15.83:0/1209575348 >> 172.21.15.83:0/1209575348 conn(0x558613e8ce60 msgr2=0x558613e8d240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:03.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.745+0000 7f2f5c9e8740 1 -- 172.21.15.83:0/1209575348 shutdown_connections 2024-04-24T08:59:03.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.745+0000 7f2f5c9e8740 1 -- 172.21.15.83:0/1209575348 wait complete. 2024-04-24T08:59:03.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.746+0000 7f2f5c9e8740 1 -- start start 2024-04-24T08:59:03.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.746+0000 7f2f5c9e8740 1 Processor -- start 2024-04-24T08:59:03.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.746+0000 7f2f5c9e8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558613e90b40 0x558613e9b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:03.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.746+0000 7f2f5c9e8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558613e91460 0x558613e98880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:03.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.746+0000 7f2f5c9e8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558613e9bb30 0x558613e98fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:03.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.746+0000 7f2f5c9e8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x558613e9e9c0 con 0x558613e9bb30 2024-04-24T08:59:03.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.746+0000 7f2f5c9e8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558613daf810 con 0x558613e90b40 2024-04-24T08:59:03.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.746+0000 7f2f5c9e8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x558613e9eb40 con 0x558613e91460 2024-04-24T08:59:03.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.746+0000 7f2f5c9e8740 10 client.0 _ll_register_callbacks cb 0x558613d7d140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:59:03.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.747+0000 7f2f53400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558613e9bb30 0x558613e98fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:03.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.747+0000 7f2f53400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558613e9bb30 0x558613e98fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41412/0 (socket says 172.21.15.83:41412) 2024-04-24T08:59:03.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.747+0000 7f2f53400640 1 -- 172.21.15.83:0/2647956640 learned_addr learned my addr 172.21.15.83:0/2647956640 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:59:03.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.747+0000 7f2f52000640 1 --2- 172.21.15.83:0/2647956640 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558613e91460 0x558613e98880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:03.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.747+0000 7f2f52a00640 1 --2- 172.21.15.83:0/2647956640 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558613e90b40 0x558613e9b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:03.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.747+0000 7f2f53400640 1 -- 172.21.15.83:0/2647956640 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558613e91460 msgr2=0x558613e98880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:03.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.747+0000 7f2f53400640 1 --2- 172.21.15.83:0/2647956640 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558613e91460 0x558613e98880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:03.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.747+0000 7f2f53400640 1 -- 172.21.15.83:0/2647956640 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558613e90b40 msgr2=0x558613e9b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:03.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.747+0000 7f2f53400640 1 --2- 172.21.15.83:0/2647956640 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558613e90b40 0x558613e9b590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:03.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.747+0000 7f2f53400640 1 -- 172.21.15.83:0/2647956640 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558613dc03f0 con 0x558613e9bb30 2024-04-24T08:59:03.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.747+0000 7f2f53400640 1 --2- 172.21.15.83:0/2647956640 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558613e9bb30 0x558613e98fa0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f2f4c012900 tx=0x7f2f4c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33348bb0b0ac6a57 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:03.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.747+0000 7f2f3cc00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:59:03.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.748+0000 7f2f3cc00640 1 -- 172.21.15.83:0/2647956640 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f4c00fd60 con 0x558613e9bb30 2024-04-24T08:59:03.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.748+0000 7f2f3cc00640 1 -- 172.21.15.83:0/2647956640 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9671+0+0 (secure 0 0 0) 0x7f2f4c01f420 con 0x558613e9bb30 2024-04-24T08:59:03.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.748+0000 7f2f5c9e8740 1 -- 172.21.15.83:0/2647956640 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f4409cc70 con 0x558613e9bb30 2024-04-24T08:59:03.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.748+0000 7f2f3cc00640 1 -- 172.21.15.83:0/2647956640 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f2f4c030050 con 0x558613e9bb30 2024-04-24T08:59:03.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.748+0000 7f2f5c9e8740 10 client.18324 fetch_fsmap learned FSMap version 33 2024-04-24T08:59:03.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.748+0000 7f2f5c9e8740 1 -- 172.21.15.83:0/2647956640 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x558613b083f0 con 0x558613e9bb30 2024-04-24T08:59:03.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.749+0000 7f2f3cc00640 1 -- 172.21.15.83:0/2647956640 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2f4409cc70 con 0x558613e9bb30 2024-04-24T08:59:03.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.749+0000 7f2f3cc00640 1 -- 172.21.15.83:0/2647956640 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f2f4c01d570 con 0x558613e9bb30 2024-04-24T08:59:03.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.749+0000 7f2f5c9e8740 10 client.18324 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T08:59:03.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.749+0000 7f2f5c9e8740 10 client.18324 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:59:03.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.749+0000 7f2f5c9e8740 20 client.18324 populate_metadata read hostname 'smithi083' 2024-04-24T08:59:03.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.749+0000 7f2f5c9e8740 1 --2- 172.21.15.83:0/2647956640 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x558613eabdb0 0x558613eae1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:03.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.749+0000 7f2f5c9e8740 4 client.18324 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:59:03.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.749+0000 7f2f5c9e8740 1 -- 172.21.15.83:0/2647956640 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558613e03bf0 con 0x558613eabdb0 2024-04-24T08:59:03.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.750+0000 7f2f52a00640 1 --2- 172.21.15.83:0/2647956640 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x558613eabdb0 0x558613eae1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:03.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.751+0000 7f2f3cc00640 10 client.18324 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:59:03.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.751+0000 7f2f52a00640 1 --2- 172.21.15.83:0/2647956640 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x558613eabdb0 0x558613eae1e0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a861ee92c9883fc server_cookie=37cf285223849be5 in_seq=0 out_seq=0 2024-04-24T08:59:03.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.753+0000 7f2f3cc00640 1 -- 172.21.15.83:0/2647956640 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558613e03bf0 con 0x558613eabdb0 2024-04-24T08:59:03.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.753+0000 7f2f3cc00640 10 client.18324 handle_command_reply: tid=0 2024-04-24T08:59:03.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.753+0000 7f2f5c9e8740 1 client.18324 shutdown 2024-04-24T08:59:03.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.753+0000 7f2f5c9e8740 1 -- 172.21.15.83:0/2647956640 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558613e9bb30 msgr2=0x558613e98fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:03.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.753+0000 7f2f5c9e8740 1 --2- 172.21.15.83:0/2647956640 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558613e9bb30 0x558613e98fa0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f2f4c012900 tx=0x7f2f4c004a40 comp rx=0 tx=0).stop 2024-04-24T08:59:03.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.754+0000 7f2f5c9e8740 1 -- 172.21.15.83:0/2647956640 shutdown_connections 2024-04-24T08:59:03.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.754+0000 7f2f5c9e8740 1 --2- 172.21.15.83:0/2647956640 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x558613eabdb0 0x558613eae1e0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:03.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.754+0000 7f2f5c9e8740 1 --2- 172.21.15.83:0/2647956640 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558613e9bb30 0x558613e98fa0 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:03.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.754+0000 7f2f5c9e8740 1 --2- 172.21.15.83:0/2647956640 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558613e91460 0x558613e98880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:03.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.754+0000 7f2f5c9e8740 1 --2- 172.21.15.83:0/2647956640 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558613e90b40 0x558613e9b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:03.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.754+0000 7f2f5c9e8740 1 -- 172.21.15.83:0/2647956640 >> 172.21.15.83:0/2647956640 conn(0x558613e8ce60 msgr2=0x558613e92b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:03.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.754+0000 7f2f3cc00640 0 client.18324 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:59:03.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.754+0000 7f2f5c9e8740 1 -- 172.21.15.83:0/2647956640 shutdown_connections 2024-04-24T08:59:03.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.754+0000 7f2f5c9e8740 1 -- 172.21.15.83:0/2647956640 wait complete. 2024-04-24T08:59:03.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.754+0000 7f2f5c9e8740 20 client.18324 trim_cache size 0 max 16384 2024-04-24T08:59:03.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.770+0000 7f2f5c9e8740 1 Processor -- start 2024-04-24T08:59:03.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.770+0000 7f2f5c9e8740 1 -- start start 2024-04-24T08:59:03.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.771+0000 7f2f5c9e8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558613e90b40 0x558613ea78b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:03.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.771+0000 7f2f5c9e8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558613e91460 0x558613e9b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:03.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.771+0000 7f2f5c9e8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558613e9bb30 0x558613eb3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:03.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.771+0000 7f2f5c9e8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2f541b5bb0 con 0x558613e90b40 2024-04-24T08:59:03.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.771+0000 7f2f5c9e8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2f541b76f0 con 0x558613e91460 2024-04-24T08:59:03.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.771+0000 7f2f5c9e8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2f541b6280 con 0x558613e9bb30 2024-04-24T08:59:03.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.771+0000 7f2f52a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558613e90b40 0x558613ea78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:03.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.771+0000 7f2f52a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558613e90b40 0x558613ea78b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41414/0 (socket says 172.21.15.83:41414) 2024-04-24T08:59:03.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.771+0000 7f2f52a00640 1 -- 172.21.15.83:0/3418892068 learned_addr learned my addr 172.21.15.83:0/3418892068 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:59:03.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.771+0000 7f2f52a00640 1 -- 172.21.15.83:0/3418892068 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558613e9bb30 msgr2=0x558613eb3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:03.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.771+0000 7f2f52a00640 1 --2- 172.21.15.83:0/3418892068 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558613e9bb30 0x558613eb3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:03.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.771+0000 7f2f52a00640 1 -- 172.21.15.83:0/3418892068 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558613e91460 msgr2=0x558613e9b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:03.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.771+0000 7f2f52a00640 1 --2- 172.21.15.83:0/3418892068 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558613e91460 0x558613e9b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:03.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.771+0000 7f2f52a00640 1 -- 172.21.15.83:0/3418892068 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558613bd27d0 con 0x558613e90b40 2024-04-24T08:59:03.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.772+0000 7f2f52a00640 1 --2- 172.21.15.83:0/3418892068 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558613e90b40 0x558613ea78b0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f2f44098f40 tx=0x7f2f440d4590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34f3061faa66760 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:03.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.772+0000 7f2f3f400640 1 -- 172.21.15.83:0/3418892068 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f44020070 con 0x558613e90b40 2024-04-24T08:59:03.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.772+0000 7f2f3f400640 1 -- 172.21.15.83:0/3418892068 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2f44009de0 con 0x558613e90b40 2024-04-24T08:59:03.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.772+0000 7f2f3f400640 1 -- 172.21.15.83:0/3418892068 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f4409c0b0 con 0x558613e90b40 2024-04-24T08:59:03.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.773+0000 7f2f5c9e8740 1 -- 172.21.15.83:0/3418892068 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558613e90b40 msgr2=0x558613ea78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:03.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.773+0000 7f2f5c9e8740 1 --2- 172.21.15.83:0/3418892068 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558613e90b40 0x558613ea78b0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f2f44098f40 tx=0x7f2f440d4590 comp rx=0 tx=0).stop 2024-04-24T08:59:03.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.773+0000 7f2f5c9e8740 1 -- 172.21.15.83:0/3418892068 shutdown_connections 2024-04-24T08:59:03.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.773+0000 7f2f5c9e8740 1 --2- 172.21.15.83:0/3418892068 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558613e9bb30 0x558613eb3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:03.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.773+0000 7f2f5c9e8740 1 --2- 172.21.15.83:0/3418892068 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558613e91460 0x558613e9b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:03.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.773+0000 7f2f5c9e8740 1 --2- 172.21.15.83:0/3418892068 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558613e90b40 0x558613ea78b0 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:03.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.773+0000 7f2f5c9e8740 1 -- 172.21.15.83:0/3418892068 >> 172.21.15.83:0/3418892068 conn(0x558613e8ce60 msgr2=0x558613e92290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:03.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.773+0000 7f2f5c9e8740 1 -- 172.21.15.83:0/3418892068 shutdown_connections 2024-04-24T08:59:03.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.773+0000 7f2f5c9e8740 1 -- 172.21.15.83:0/3418892068 wait complete. 2024-04-24T08:59:03.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.773+0000 7f2f5c9e8740 1 -- start start 2024-04-24T08:59:03.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.774+0000 7f2f5c9e8740 1 Processor -- start 2024-04-24T08:59:03.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.774+0000 7f2f5c9e8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558613e90b40 0x558613e9c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:03.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.774+0000 7f2f5c9e8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558613e91460 0x558613e9ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:03.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.774+0000 7f2f5c9e8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558613e9bb30 0x558613e9d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:03.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.774+0000 7f2f5c9e8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2f541b7020 con 0x558613e91460 2024-04-24T08:59:03.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.774+0000 7f2f5c9e8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2f541b6950 con 0x558613e90b40 2024-04-24T08:59:03.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.774+0000 7f2f5c9e8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2f541b3bb0 con 0x558613e9bb30 2024-04-24T08:59:03.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.774+0000 7f2f5c9e8740 10 client.0 _ll_register_callbacks cb 0x7f2f5406b700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:59:03.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.774+0000 7f2f52000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558613e91460 0x558613e9ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:03.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.774+0000 7f2f52000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558613e91460 0x558613e9ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41422/0 (socket says 172.21.15.83:41422) 2024-04-24T08:59:03.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.774+0000 7f2f52000640 1 -- 172.21.15.83:0/27292240 learned_addr learned my addr 172.21.15.83:0/27292240 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:59:03.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.775+0000 7f2f53400640 1 --2- 172.21.15.83:0/27292240 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558613e9bb30 0x558613e9d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:03.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.775+0000 7f2f52a00640 1 --2- 172.21.15.83:0/27292240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558613e90b40 0x558613e9c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:03.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.775+0000 7f2f52000640 1 -- 172.21.15.83:0/27292240 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558613e9bb30 msgr2=0x558613e9d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:03.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.775+0000 7f2f52000640 1 --2- 172.21.15.83:0/27292240 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558613e9bb30 0x558613e9d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:03.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.775+0000 7f2f52000640 1 -- 172.21.15.83:0/27292240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558613e90b40 msgr2=0x558613e9c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:03.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.775+0000 7f2f52000640 1 --2- 172.21.15.83:0/27292240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558613e90b40 0x558613e9c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:03.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.775+0000 7f2f52000640 1 -- 172.21.15.83:0/27292240 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558613ea1e90 con 0x558613e91460 2024-04-24T08:59:03.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.775+0000 7f2f53400640 1 --2- 172.21.15.83:0/27292240 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558613e9bb30 0x558613e9d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:59:03.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.775+0000 7f2f52000640 1 --2- 172.21.15.83:0/27292240 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558613e91460 0x558613e9ccb0 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f2f40033000 tx=0x7f2f40005250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58b32537cfd5cbc5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:03.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.775+0000 7f2f3cc00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:59:03.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.775+0000 7f2f3cc00640 1 -- 172.21.15.83:0/27292240 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f40035070 con 0x558613e91460 2024-04-24T08:59:03.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.775+0000 7f2f3cc00640 1 -- 172.21.15.83:0/27292240 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9671+0+0 (secure 0 0 0) 0x7f2f400048c0 con 0x558613e91460 2024-04-24T08:59:03.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.775+0000 7f2f5c9e8740 1 -- 172.21.15.83:0/27292240 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f44009de0 con 0x558613e91460 2024-04-24T08:59:03.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.776+0000 7f2f3cc00640 1 -- 172.21.15.83:0/27292240 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f2f4004d050 con 0x558613e91460 2024-04-24T08:59:03.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.776+0000 7f2f5c9e8740 10 client.18330 fetch_fsmap learned FSMap version 33 2024-04-24T08:59:03.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.776+0000 7f2f5c9e8740 1 -- 172.21.15.83:0/27292240 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x558613f12530 con 0x558613e91460 2024-04-24T08:59:03.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.776+0000 7f2f3cc00640 1 -- 172.21.15.83:0/27292240 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2f44009de0 con 0x558613e91460 2024-04-24T08:59:03.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.776+0000 7f2f3cc00640 1 -- 172.21.15.83:0/27292240 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f2f40042020 con 0x558613e91460 2024-04-24T08:59:03.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.776+0000 7f2f5c9e8740 10 client.18330 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T08:59:03.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.776+0000 7f2f5c9e8740 10 client.18330 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:59:03.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.776+0000 7f2f5c9e8740 20 client.18330 populate_metadata read hostname 'smithi083' 2024-04-24T08:59:03.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.776+0000 7f2f5c9e8740 1 --2- 172.21.15.83:0/27292240 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x558613f12ce0 0x558613f15100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:03.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.776+0000 7f2f5c9e8740 4 client.18330 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T08:59:03.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.776+0000 7f2f5c9e8740 1 -- 172.21.15.83:0/27292240 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558613dfda60 con 0x558613f12ce0 2024-04-24T08:59:03.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.777+0000 7f2f52a00640 1 --2- 172.21.15.83:0/27292240 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x558613f12ce0 0x558613f15100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:03.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.778+0000 7f2f3cc00640 10 client.18330 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:59:03.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.778+0000 7f2f52a00640 1 --2- 172.21.15.83:0/27292240 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x558613f12ce0 0x558613f15100 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb8bd570e4a7e6f1 server_cookie=cb8d3c889cd0ba3d in_seq=0 out_seq=0 2024-04-24T08:59:03.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.778+0000 7f2f3cc00640 1 -- 172.21.15.83:0/27292240 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558613dfda60 con 0x558613f12ce0 2024-04-24T08:59:03.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.778+0000 7f2f3cc00640 10 client.18330 handle_command_reply: tid=0 2024-04-24T08:59:03.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.778+0000 7f2f5c9e8740 1 client.18330 shutdown 2024-04-24T08:59:03.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.779+0000 7f2f5c9e8740 1 -- 172.21.15.83:0/27292240 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558613e91460 msgr2=0x558613e9ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:03.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.779+0000 7f2f5c9e8740 1 --2- 172.21.15.83:0/27292240 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558613e91460 0x558613e9ccb0 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f2f40033000 tx=0x7f2f40005250 comp rx=0 tx=0).stop 2024-04-24T08:59:03.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.779+0000 7f2f5c9e8740 1 -- 172.21.15.83:0/27292240 shutdown_connections 2024-04-24T08:59:03.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.779+0000 7f2f5c9e8740 1 --2- 172.21.15.83:0/27292240 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x558613f12ce0 0x558613f15100 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:03.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.779+0000 7f2f5c9e8740 1 --2- 172.21.15.83:0/27292240 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558613e9bb30 0x558613e9d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:03.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.779+0000 7f2f3cc00640 0 client.18330 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:59:03.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.779+0000 7f2f5c9e8740 1 --2- 172.21.15.83:0/27292240 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558613e91460 0x558613e9ccb0 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:03.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.779+0000 7f2f5c9e8740 1 --2- 172.21.15.83:0/27292240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558613e90b40 0x558613e9c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:03.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.779+0000 7f2f5c9e8740 1 -- 172.21.15.83:0/27292240 >> 172.21.15.83:0/27292240 conn(0x558613e8ce60 msgr2=0x558613ea7130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:03.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.779+0000 7f2f5c9e8740 1 -- 172.21.15.83:0/27292240 shutdown_connections 2024-04-24T08:59:03.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.779+0000 7f2f5c9e8740 1 -- 172.21.15.83:0/27292240 wait complete. 2024-04-24T08:59:03.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.779+0000 7f2f5c9e8740 20 client.18330 trim_cache size 0 max 16384 2024-04-24T08:59:03.785 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T08:59:03.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.780+0000 7f2f53e00640 1 -- 172.21.15.83:0/519756766 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f2f380a3d20 msgr2=0x7f2f380a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:03.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.780+0000 7f2f53e00640 1 --2- 172.21.15.83:0/519756766 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f2f380a3d20 0x7f2f380a61e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2f4c005fd0 tx=0x7f2f4c009630 comp rx=0 tx=0).stop 2024-04-24T08:59:03.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.780+0000 7f2f53e00640 1 -- 172.21.15.83:0/519756766 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f54108580 msgr2=0x7f2f541a48d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:03.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.780+0000 7f2f53e00640 1 --2- 172.21.15.83:0/519756766 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f54108580 0x7f2f541a48d0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f2f4400d770 tx=0x7f2f4400dc30 comp rx=0 tx=0).stop 2024-04-24T08:59:03.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.780+0000 7f2f53e00640 1 -- 172.21.15.83:0/519756766 shutdown_connections 2024-04-24T08:59:03.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.780+0000 7f2f53e00640 1 --2- 172.21.15.83:0/519756766 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f2f380a3d20 0x7f2f380a61e0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:03.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.780+0000 7f2f53e00640 1 --2- 172.21.15.83:0/519756766 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f5410d500 0x7f2f5419ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:03.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.780+0000 7f2f53e00640 1 --2- 172.21.15.83:0/519756766 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2f54108f40 0x7f2f541a4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:03.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.780+0000 7f2f53e00640 1 --2- 172.21.15.83:0/519756766 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f54108580 0x7f2f541a48d0 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:03.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.780+0000 7f2f53e00640 1 -- 172.21.15.83:0/519756766 >> 172.21.15.83:0/519756766 conn(0x7f2f54077e20 msgr2=0x7f2f540783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:03.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.780+0000 7f2f53e00640 1 -- 172.21.15.83:0/519756766 shutdown_connections 2024-04-24T08:59:03.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:03.780+0000 7f2f53e00640 1 -- 172.21.15.83:0/519756766 wait complete. 2024-04-24T08:59:04.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:59:03 smithi083 ceph-mon[27907]: pgmap v229: 129 pgs: 129 active+clean; 46 MiB data, 668 MiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 535 KiB/s wr, 90 op/s 2024-04-24T08:59:04.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:59:03 smithi083 ceph-mon[27907]: osdmap e98: 12 total, 12 up, 12 in 2024-04-24T08:59:04.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:59:03 smithi138 ceph-mon[31441]: pgmap v229: 129 pgs: 129 active+clean; 46 MiB data, 668 MiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 535 KiB/s wr, 90 op/s 2024-04-24T08:59:04.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:59:03 smithi138 ceph-mon[31441]: osdmap e98: 12 total, 12 up, 12 in 2024-04-24T08:59:04.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:59:03 smithi189 ceph-mon[31791]: pgmap v229: 129 pgs: 129 active+clean; 46 MiB data, 668 MiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 535 KiB/s wr, 90 op/s 2024-04-24T08:59:04.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:59:03 smithi189 ceph-mon[31791]: osdmap e98: 12 total, 12 up, 12 in 2024-04-24T08:59:04.591 INFO:tasks.workunit.client.0.smithi083.stdout: 1 40454 0.64 MB/sec execute 218 sec latency 63.003 ms 2024-04-24T08:59:04.800 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T08:59:04.800 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T08:59:04.800 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T08:59:05.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.028+0000 7f24e3000640 1 -- 172.21.15.83:0/759820906 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24dc109520 msgr2=0x7f24dc1110f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:05.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.028+0000 7f24e3000640 1 --2- 172.21.15.83:0/759820906 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24dc109520 0x7f24dc1110f0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f24d4009ca0 tx=0x7f24d402a8c0 comp rx=0 tx=0).stop 2024-04-24T08:59:05.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.028+0000 7f24e3000640 1 -- 172.21.15.83:0/759820906 shutdown_connections 2024-04-24T08:59:05.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.028+0000 7f24e3000640 1 --2- 172.21.15.83:0/759820906 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24dc109520 0x7f24dc1110f0 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:05.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.028+0000 7f24e3000640 1 --2- 172.21.15.83:0/759820906 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f24dc104e00 0x7f24dc108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:05.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.028+0000 7f24e3000640 1 --2- 172.21.15.83:0/759820906 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f24dc104450 0x7f24dc104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:05.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.028+0000 7f24e3000640 1 -- 172.21.15.83:0/759820906 >> 172.21.15.83:0/759820906 conn(0x7f24dc0ffc70 msgr2=0x7f24dc102090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:05.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.028+0000 7f24e3000640 1 -- 172.21.15.83:0/759820906 shutdown_connections 2024-04-24T08:59:05.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.028+0000 7f24e3000640 1 -- 172.21.15.83:0/759820906 wait complete. 2024-04-24T08:59:05.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.029+0000 7f24e3000640 1 Processor -- start 2024-04-24T08:59:05.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.029+0000 7f24e3000640 1 -- start start 2024-04-24T08:59:05.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.030+0000 7f24e3000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f24dc104450 0x7f24dc1a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:05.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.030+0000 7f24e3000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f24dc109520 0x7f24dc1a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:05.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.030+0000 7f24e3000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24dc19a7c0 0x7f24dc19ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:05.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.030+0000 7f24e3000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f24dc1144e0 con 0x7f24dc19a7c0 2024-04-24T08:59:05.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.030+0000 7f24e3000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f24dc114360 con 0x7f24dc104450 2024-04-24T08:59:05.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.030+0000 7f24e3000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f24dc114660 con 0x7f24dc109520 2024-04-24T08:59:05.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.030+0000 7f24e1000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24dc19a7c0 0x7f24dc19ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:05.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.030+0000 7f24e1000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24dc19a7c0 0x7f24dc19ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41444/0 (socket says 172.21.15.83:41444) 2024-04-24T08:59:05.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.030+0000 7f24e1000640 1 -- 172.21.15.83:0/3721689527 learned_addr learned my addr 172.21.15.83:0/3721689527 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:59:05.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.030+0000 7f24dbe00640 1 --2- 172.21.15.83:0/3721689527 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f24dc104450 0x7f24dc1a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:05.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.031+0000 7f24d3400640 1 --2- 172.21.15.83:0/3721689527 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f24dc109520 0x7f24dc1a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:05.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.031+0000 7f24e1000640 1 -- 172.21.15.83:0/3721689527 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f24dc109520 msgr2=0x7f24dc1a0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:05.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.031+0000 7f24e1000640 1 --2- 172.21.15.83:0/3721689527 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f24dc109520 0x7f24dc1a0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:05.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.031+0000 7f24e1000640 1 -- 172.21.15.83:0/3721689527 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f24dc104450 msgr2=0x7f24dc1a0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:05.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.031+0000 7f24e1000640 1 --2- 172.21.15.83:0/3721689527 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f24dc104450 0x7f24dc1a0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:05.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.031+0000 7f24e1000640 1 -- 172.21.15.83:0/3721689527 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24dc19b350 con 0x7f24dc19a7c0 2024-04-24T08:59:05.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.031+0000 7f24dbe00640 1 --2- 172.21.15.83:0/3721689527 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f24dc104450 0x7f24dc1a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:59:05.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.031+0000 7f24e1000640 1 --2- 172.21.15.83:0/3721689527 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24dc19a7c0 0x7f24dc19ac70 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f24d4031040 tx=0x7f24d4003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:05.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.031+0000 7f24d9600640 1 -- 172.21.15.83:0/3721689527 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d4003ce0 con 0x7f24dc19a7c0 2024-04-24T08:59:05.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.031+0000 7f24d9600640 1 -- 172.21.15.83:0/3721689527 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f24d40044f0 con 0x7f24dc19a7c0 2024-04-24T08:59:05.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.031+0000 7f24e3000640 1 -- 172.21.15.83:0/3721689527 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24dc19b5e0 con 0x7f24dc19a7c0 2024-04-24T08:59:05.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.031+0000 7f24d9600640 1 -- 172.21.15.83:0/3721689527 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d4030c90 con 0x7f24dc19a7c0 2024-04-24T08:59:05.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.032+0000 7f24e3000640 1 -- 172.21.15.83:0/3721689527 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24dc1a7810 con 0x7f24dc19a7c0 2024-04-24T08:59:05.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.035+0000 7f24d9600640 1 -- 172.21.15.83:0/3721689527 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f24d40046a0 con 0x7f24dc19a7c0 2024-04-24T08:59:05.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.035+0000 7f24e5e2a740 1 Processor -- start 2024-04-24T08:59:05.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.035+0000 7f24e5e2a740 1 -- start start 2024-04-24T08:59:05.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.036+0000 7f24e5e2a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5621c7eb4090 0x5621c7eb4470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:05.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.036+0000 7f24e5e2a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5621c7eb49b0 0x5621c7ebeb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:05.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.036+0000 7f24e5e2a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5621c7ebf080 0x5621c7ec1470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:05.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.036+0000 7f24e5e2a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5621c7e19e70 con 0x5621c7eb4090 2024-04-24T08:59:05.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.036+0000 7f24e5e2a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5621c7dc0380 con 0x5621c7eb49b0 2024-04-24T08:59:05.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.036+0000 7f24e5e2a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5621c7e15e30 con 0x5621c7ebf080 2024-04-24T08:59:05.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.036+0000 7f24d3400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5621c7eb4090 0x5621c7eb4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:05.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.036+0000 7f24d3400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5621c7eb4090 0x5621c7eb4470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41448/0 (socket says 172.21.15.83:41448) 2024-04-24T08:59:05.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.036+0000 7f24d3400640 1 -- 172.21.15.83:0/1490699670 learned_addr learned my addr 172.21.15.83:0/1490699670 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:59:05.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.036+0000 7f24dbe00640 1 --2- 172.21.15.83:0/1490699670 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5621c7eb49b0 0x5621c7ebeb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:05.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.037+0000 7f24d3400640 1 --2- 172.21.15.83:0/1490699670 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5621c7ebf080 0x5621c7ec1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:05.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.037+0000 7f24d3400640 1 -- 172.21.15.83:0/1490699670 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5621c7ebf080 msgr2=0x5621c7ec1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:05.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.037+0000 7f24d3400640 1 --2- 172.21.15.83:0/1490699670 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5621c7ebf080 0x5621c7ec1470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:05.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.037+0000 7f24d3400640 1 -- 172.21.15.83:0/1490699670 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5621c7eb49b0 msgr2=0x5621c7ebeb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:05.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.037+0000 7f24d3400640 1 --2- 172.21.15.83:0/1490699670 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5621c7eb49b0 0x5621c7ebeb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:05.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.037+0000 7f24d3400640 1 -- 172.21.15.83:0/1490699670 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621c7bf47e0 con 0x5621c7eb4090 2024-04-24T08:59:05.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.037+0000 7f24d9600640 1 --2- 172.21.15.83:0/3721689527 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f24c00a3da0 0x7f24c00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:05.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.037+0000 7f24d3400640 1 --2- 172.21.15.83:0/1490699670 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5621c7eb4090 0x5621c7eb4470 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f24c4013450 tx=0x7f24c400b5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2652bf51ebd2ee50 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:05.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.037+0000 7f24d9600640 1 -- 172.21.15.83:0/3721689527 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9671+0+0 (secure 0 0 0) 0x7f24d40f2200 con 0x7f24dc19a7c0 2024-04-24T08:59:05.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.038+0000 7f24dbe00640 1 --2- 172.21.15.83:0/3721689527 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f24c00a3da0 0x7f24c00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:05.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.038+0000 7f24d1600640 1 -- 172.21.15.83:0/1490699670 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c4019030 con 0x5621c7eb4090 2024-04-24T08:59:05.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.038+0000 7f24d1600640 1 -- 172.21.15.83:0/1490699670 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f24c40054d0 con 0x5621c7eb4090 2024-04-24T08:59:05.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.038+0000 7f24d1600640 1 -- 172.21.15.83:0/1490699670 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c4011440 con 0x5621c7eb4090 2024-04-24T08:59:05.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.039+0000 7f24e5e2a740 1 -- 172.21.15.83:0/1490699670 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5621c7eb4090 msgr2=0x5621c7eb4470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:05.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.039+0000 7f24e5e2a740 1 --2- 172.21.15.83:0/1490699670 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5621c7eb4090 0x5621c7eb4470 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f24c4013450 tx=0x7f24c400b5e0 comp rx=0 tx=0).stop 2024-04-24T08:59:05.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.039+0000 7f24dbe00640 1 --2- 172.21.15.83:0/3721689527 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f24c00a3da0 0x7f24c00a6260 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x5621c7eb5650 tx=0x7f24cc0084c0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:05.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.039+0000 7f24e5e2a740 1 -- 172.21.15.83:0/1490699670 shutdown_connections 2024-04-24T08:59:05.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.039+0000 7f24e5e2a740 1 --2- 172.21.15.83:0/1490699670 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5621c7ebf080 0x5621c7ec1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:05.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.039+0000 7f24e5e2a740 1 --2- 172.21.15.83:0/1490699670 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5621c7eb49b0 0x5621c7ebeb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:05.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.039+0000 7f24e5e2a740 1 --2- 172.21.15.83:0/1490699670 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5621c7eb4090 0x5621c7eb4470 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:05.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.039+0000 7f24e5e2a740 1 -- 172.21.15.83:0/1490699670 >> 172.21.15.83:0/1490699670 conn(0x5621c7eb03b0 msgr2=0x5621c7eb0790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:05.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.039+0000 7f24e5e2a740 1 -- 172.21.15.83:0/1490699670 shutdown_connections 2024-04-24T08:59:05.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.039+0000 7f24e5e2a740 1 -- 172.21.15.83:0/1490699670 wait complete. 2024-04-24T08:59:05.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.039+0000 7f24e5e2a740 1 -- start start 2024-04-24T08:59:05.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.040+0000 7f24e5e2a740 1 Processor -- start 2024-04-24T08:59:05.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.040+0000 7f24e5e2a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5621c7eb4090 0x5621c7ebbf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:05.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.040+0000 7f24e5e2a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5621c7eb49b0 0x5621c7ebc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:05.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.040+0000 7f24e5e2a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5621c7ebf080 0x5621c7ebc9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:05.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.040+0000 7f24e5e2a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5621c7ec1e70 con 0x5621c7eb49b0 2024-04-24T08:59:05.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.040+0000 7f24e5e2a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5621c7dd3660 con 0x5621c7eb4090 2024-04-24T08:59:05.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.040+0000 7f24e5e2a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5621c7ec1ff0 con 0x5621c7ebf080 2024-04-24T08:59:05.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.040+0000 7f24e5e2a740 10 client.0 _ll_register_callbacks cb 0x5621c7dc1aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:59:05.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.041+0000 7f24d3400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5621c7eb49b0 0x5621c7ebc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:05.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.041+0000 7f24d3400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5621c7eb49b0 0x5621c7ebc4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41464/0 (socket says 172.21.15.83:41464) 2024-04-24T08:59:05.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.041+0000 7f24d3400640 1 -- 172.21.15.83:0/4107594573 learned_addr learned my addr 172.21.15.83:0/4107594573 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:59:05.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.041+0000 7f24dbe00640 1 --2- 172.21.15.83:0/4107594573 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5621c7eb4090 0x5621c7ebbf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:05.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.041+0000 7f24e1000640 1 --2- 172.21.15.83:0/4107594573 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5621c7ebf080 0x5621c7ebc9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:05.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.041+0000 7f24d3400640 1 -- 172.21.15.83:0/4107594573 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5621c7ebf080 msgr2=0x5621c7ebc9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:05.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.041+0000 7f24d3400640 1 --2- 172.21.15.83:0/4107594573 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5621c7ebf080 0x5621c7ebc9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:05.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.041+0000 7f24d3400640 1 -- 172.21.15.83:0/4107594573 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5621c7eb4090 msgr2=0x5621c7ebbf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:05.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.041+0000 7f24d3400640 1 --2- 172.21.15.83:0/4107594573 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5621c7eb4090 0x5621c7ebbf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:05.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.041+0000 7f24d3400640 1 -- 172.21.15.83:0/4107594573 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621c7c54f70 con 0x5621c7eb49b0 2024-04-24T08:59:05.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.041+0000 7f24d3400640 1 --2- 172.21.15.83:0/4107594573 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5621c7eb49b0 0x5621c7ebc4a0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f24c400b450 tx=0x7f24c400bc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d016eae760d7c35b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:05.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.042+0000 7f24ca000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:59:05.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.042+0000 7f24ca000640 1 -- 172.21.15.83:0/4107594573 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c402b060 con 0x5621c7eb49b0 2024-04-24T08:59:05.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.042+0000 7f24ca000640 1 -- 172.21.15.83:0/4107594573 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9671+0+0 (secure 0 0 0) 0x7f24c40048c0 con 0x5621c7eb49b0 2024-04-24T08:59:05.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.042+0000 7f24e5e2a740 1 -- 172.21.15.83:0/4107594573 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24c40054d0 con 0x5621c7eb49b0 2024-04-24T08:59:05.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.042+0000 7f24ca000640 1 -- 172.21.15.83:0/4107594573 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f24c4032050 con 0x5621c7eb49b0 2024-04-24T08:59:05.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.042+0000 7f24e5e2a740 10 client.18342 fetch_fsmap learned FSMap version 33 2024-04-24T08:59:05.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.042+0000 7f24e5e2a740 1 -- 172.21.15.83:0/4107594573 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5621c7aec170 con 0x5621c7eb49b0 2024-04-24T08:59:05.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.042+0000 7f24ca000640 1 -- 172.21.15.83:0/4107594573 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f24c40054d0 con 0x5621c7eb49b0 2024-04-24T08:59:05.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.043+0000 7f24ca000640 1 -- 172.21.15.83:0/4107594573 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f24c401ec30 con 0x5621c7eb49b0 2024-04-24T08:59:05.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.043+0000 7f24e5e2a740 10 client.18342 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T08:59:05.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.043+0000 7f24e5e2a740 10 client.18342 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:59:05.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.043+0000 7f24e5e2a740 20 client.18342 populate_metadata read hostname 'smithi083' 2024-04-24T08:59:05.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.043+0000 7f24e5e2a740 1 --2- 172.21.15.83:0/4107594573 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5621c7ecf2e0 0x5621c7ed16d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:05.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.043+0000 7f24e5e2a740 4 client.18342 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:59:05.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.043+0000 7f24e5e2a740 1 -- 172.21.15.83:0/4107594573 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5621c7e27890 con 0x5621c7ecf2e0 2024-04-24T08:59:05.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.043+0000 7f24dbe00640 1 --2- 172.21.15.83:0/4107594573 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5621c7ecf2e0 0x5621c7ed16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:05.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.044+0000 7f24dbe00640 1 --2- 172.21.15.83:0/4107594573 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5621c7ecf2e0 0x5621c7ed16d0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5406055ede7e873f server_cookie=e25bea20f7644add in_seq=0 out_seq=0 2024-04-24T08:59:05.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.044+0000 7f24ca000640 10 client.18342 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:59:05.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.047+0000 7f24ca000640 1 -- 172.21.15.83:0/4107594573 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5621c7e27890 con 0x5621c7ecf2e0 2024-04-24T08:59:05.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.047+0000 7f24ca000640 10 client.18342 handle_command_reply: tid=0 2024-04-24T08:59:05.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.047+0000 7f24e5e2a740 1 client.18342 shutdown 2024-04-24T08:59:05.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.047+0000 7f24e5e2a740 1 -- 172.21.15.83:0/4107594573 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5621c7eb49b0 msgr2=0x5621c7ebc4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:05.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.047+0000 7f24e5e2a740 1 --2- 172.21.15.83:0/4107594573 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5621c7eb49b0 0x5621c7ebc4a0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f24c400b450 tx=0x7f24c400bc70 comp rx=0 tx=0).stop 2024-04-24T08:59:05.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.047+0000 7f24e5e2a740 1 -- 172.21.15.83:0/4107594573 shutdown_connections 2024-04-24T08:59:05.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.047+0000 7f24e5e2a740 1 --2- 172.21.15.83:0/4107594573 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5621c7ecf2e0 0x5621c7ed16d0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:05.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.047+0000 7f24e5e2a740 1 --2- 172.21.15.83:0/4107594573 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5621c7ebf080 0x5621c7ebc9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:05.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.047+0000 7f24e5e2a740 1 --2- 172.21.15.83:0/4107594573 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5621c7eb49b0 0x5621c7ebc4a0 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:05.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.047+0000 7f24ca000640 0 client.18342 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:59:05.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.047+0000 7f24e5e2a740 1 --2- 172.21.15.83:0/4107594573 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5621c7eb4090 0x5621c7ebbf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:05.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.047+0000 7f24e5e2a740 1 -- 172.21.15.83:0/4107594573 >> 172.21.15.83:0/4107594573 conn(0x5621c7eb03b0 msgr2=0x5621c7eb60b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:05.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.047+0000 7f24e5e2a740 1 -- 172.21.15.83:0/4107594573 shutdown_connections 2024-04-24T08:59:05.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.047+0000 7f24e5e2a740 1 -- 172.21.15.83:0/4107594573 wait complete. 2024-04-24T08:59:05.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.047+0000 7f24e5e2a740 20 client.18342 trim_cache size 0 max 16384 2024-04-24T08:59:05.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.067+0000 7f24e5e2a740 1 Processor -- start 2024-04-24T08:59:05.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.067+0000 7f24e5e2a740 1 -- start start 2024-04-24T08:59:05.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.067+0000 7f24e5e2a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5621c7eb4090 0x5621c7ed46f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:05.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.067+0000 7f24e5e2a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5621c7eb49b0 0x5621c7ed4c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:05.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.067+0000 7f24e5e2a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5621c7ebf080 0x5621c7eb91c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:05.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.067+0000 7f24e5e2a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f24dc1b18b0 con 0x5621c7ebf080 2024-04-24T08:59:05.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.067+0000 7f24e5e2a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f24dc1b33f0 con 0x5621c7eb4090 2024-04-24T08:59:05.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.067+0000 7f24e5e2a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f24dc1b1f80 con 0x5621c7eb49b0 2024-04-24T08:59:05.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.068+0000 7f24e1000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5621c7ebf080 0x5621c7eb91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:05.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.068+0000 7f24e1000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5621c7ebf080 0x5621c7eb91c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41480/0 (socket says 172.21.15.83:41480) 2024-04-24T08:59:05.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.068+0000 7f24e1000640 1 -- 172.21.15.83:0/1050114519 learned_addr learned my addr 172.21.15.83:0/1050114519 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:59:05.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.068+0000 7f24dbe00640 1 --2- 172.21.15.83:0/1050114519 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5621c7eb4090 0x5621c7ed46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:05.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.068+0000 7f24d3400640 1 --2- 172.21.15.83:0/1050114519 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5621c7eb49b0 0x5621c7ed4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:05.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.068+0000 7f24e1000640 1 -- 172.21.15.83:0/1050114519 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5621c7eb49b0 msgr2=0x5621c7ed4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:05.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.068+0000 7f24e1000640 1 --2- 172.21.15.83:0/1050114519 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5621c7eb49b0 0x5621c7ed4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:05.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.068+0000 7f24e1000640 1 -- 172.21.15.83:0/1050114519 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5621c7eb4090 msgr2=0x5621c7ed46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:05.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.068+0000 7f24e1000640 1 --2- 172.21.15.83:0/1050114519 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5621c7eb4090 0x5621c7ed46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:05.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.068+0000 7f24e1000640 1 -- 172.21.15.83:0/1050114519 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621c7eb47e0 con 0x5621c7ebf080 2024-04-24T08:59:05.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.069+0000 7f24e1000640 1 --2- 172.21.15.83:0/1050114519 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5621c7ebf080 0x5621c7eb91c0 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f24d40bde80 tx=0x7f24d4030e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd789194775ce88f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:05.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.069+0000 7f24d0c00640 1 -- 172.21.15.83:0/1050114519 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d40d6070 con 0x5621c7ebf080 2024-04-24T08:59:05.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.069+0000 7f24d0c00640 1 -- 172.21.15.83:0/1050114519 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f24d4036040 con 0x5621c7ebf080 2024-04-24T08:59:05.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.069+0000 7f24d0c00640 1 -- 172.21.15.83:0/1050114519 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d40ef6f0 con 0x5621c7ebf080 2024-04-24T08:59:05.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.070+0000 7f24e5e2a740 1 -- 172.21.15.83:0/1050114519 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5621c7ebf080 msgr2=0x5621c7eb91c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:05.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.070+0000 7f24e5e2a740 1 --2- 172.21.15.83:0/1050114519 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5621c7ebf080 0x5621c7eb91c0 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f24d40bde80 tx=0x7f24d4030e30 comp rx=0 tx=0).stop 2024-04-24T08:59:05.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.070+0000 7f24e5e2a740 1 -- 172.21.15.83:0/1050114519 shutdown_connections 2024-04-24T08:59:05.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.070+0000 7f24e5e2a740 1 --2- 172.21.15.83:0/1050114519 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5621c7ebf080 0x5621c7eb91c0 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:05.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.070+0000 7f24e5e2a740 1 --2- 172.21.15.83:0/1050114519 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5621c7eb49b0 0x5621c7ed4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:05.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.070+0000 7f24e5e2a740 1 --2- 172.21.15.83:0/1050114519 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5621c7eb4090 0x5621c7ed46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:05.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.070+0000 7f24e5e2a740 1 -- 172.21.15.83:0/1050114519 >> 172.21.15.83:0/1050114519 conn(0x5621c7eb03b0 msgr2=0x5621c7ecaad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:05.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.070+0000 7f24e5e2a740 1 -- 172.21.15.83:0/1050114519 shutdown_connections 2024-04-24T08:59:05.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.070+0000 7f24e5e2a740 1 -- 172.21.15.83:0/1050114519 wait complete. 2024-04-24T08:59:05.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.070+0000 7f24e5e2a740 1 -- start start 2024-04-24T08:59:05.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.071+0000 7f24e5e2a740 1 Processor -- start 2024-04-24T08:59:05.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.071+0000 7f24e5e2a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5621c7eb4090 0x5621c7ed47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:05.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.071+0000 7f24e5e2a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5621c7eb49b0 0x5621c7ed2b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:05.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.071+0000 7f24e5e2a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5621c7ebf080 0x5621c7ed3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:05.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.071+0000 7f24e5e2a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f24dc1b2d20 con 0x5621c7eb49b0 2024-04-24T08:59:05.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.071+0000 7f24e5e2a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f24dc1b2650 con 0x5621c7ebf080 2024-04-24T08:59:05.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.071+0000 7f24e5e2a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f24dc1af8b0 con 0x5621c7eb4090 2024-04-24T08:59:05.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.071+0000 7f24e5e2a740 10 client.0 _ll_register_callbacks cb 0x7f24c4013f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:59:05.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.072+0000 7f24d3400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5621c7eb49b0 0x5621c7ed2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:05.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.072+0000 7f24d3400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5621c7eb49b0 0x5621c7ed2b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41486/0 (socket says 172.21.15.83:41486) 2024-04-24T08:59:05.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.072+0000 7f24d3400640 1 -- 172.21.15.83:0/2012289698 learned_addr learned my addr 172.21.15.83:0/2012289698 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:59:05.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.072+0000 7f24dbe00640 1 --2- 172.21.15.83:0/2012289698 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5621c7eb4090 0x5621c7ed47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:05.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.072+0000 7f24d3400640 1 -- 172.21.15.83:0/2012289698 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5621c7eb4090 msgr2=0x5621c7ed47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:05.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.072+0000 7f24d3400640 1 --2- 172.21.15.83:0/2012289698 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5621c7eb4090 0x5621c7ed47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:05.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.072+0000 7f24d3400640 1 -- 172.21.15.83:0/2012289698 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5621c7ebf080 msgr2=0x5621c7ed3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:05.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.072+0000 7f24d3400640 1 --2- 172.21.15.83:0/2012289698 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5621c7ebf080 0x5621c7ed3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:05.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.072+0000 7f24d3400640 1 -- 172.21.15.83:0/2012289698 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621c7ed3600 con 0x5621c7eb49b0 2024-04-24T08:59:05.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.072+0000 7f24d3400640 1 --2- 172.21.15.83:0/2012289698 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5621c7eb49b0 0x5621c7ed2b50 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f24c4021020 tx=0x7f24c400bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=231015e65ddbadbf server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:05.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.072+0000 7f24ca000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:59:05.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.073+0000 7f24ca000640 1 -- 172.21.15.83:0/2012289698 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c402b070 con 0x5621c7eb49b0 2024-04-24T08:59:05.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.073+0000 7f24ca000640 1 -- 172.21.15.83:0/2012289698 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9671+0+0 (secure 0 0 0) 0x7f24c4011e30 con 0x5621c7eb49b0 2024-04-24T08:59:05.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.073+0000 7f24e5e2a740 1 -- 172.21.15.83:0/2012289698 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24d4036040 con 0x5621c7eb49b0 2024-04-24T08:59:05.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.073+0000 7f24ca000640 1 -- 172.21.15.83:0/2012289698 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f24c4043050 con 0x5621c7eb49b0 2024-04-24T08:59:05.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.073+0000 7f24e5e2a740 10 client.18348 fetch_fsmap learned FSMap version 33 2024-04-24T08:59:05.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.073+0000 7f24e5e2a740 1 -- 172.21.15.83:0/2012289698 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5621c7f362b0 con 0x5621c7eb49b0 2024-04-24T08:59:05.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.073+0000 7f24ca000640 1 -- 172.21.15.83:0/2012289698 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f24d4036040 con 0x5621c7eb49b0 2024-04-24T08:59:05.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.073+0000 7f24ca000640 1 -- 172.21.15.83:0/2012289698 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f24c403e030 con 0x5621c7eb49b0 2024-04-24T08:59:05.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.074+0000 7f24e5e2a740 10 client.18348 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T08:59:05.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.074+0000 7f24e5e2a740 10 client.18348 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:59:05.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.074+0000 7f24e5e2a740 20 client.18348 populate_metadata read hostname 'smithi083' 2024-04-24T08:59:05.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.074+0000 7f24e5e2a740 1 --2- 172.21.15.83:0/2012289698 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5621c7f36aa0 0x5621c7f38ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:05.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.074+0000 7f24e5e2a740 4 client.18348 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T08:59:05.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.074+0000 7f24e5e2a740 1 -- 172.21.15.83:0/2012289698 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5621c7e21700 con 0x5621c7f36aa0 2024-04-24T08:59:05.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.074+0000 7f24dbe00640 1 --2- 172.21.15.83:0/2012289698 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5621c7f36aa0 0x5621c7f38ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:05.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.075+0000 7f24dbe00640 1 --2- 172.21.15.83:0/2012289698 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5621c7f36aa0 0x5621c7f38ec0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c8bd1510c88bc69 server_cookie=ddca994dc906ae8a in_seq=0 out_seq=0 2024-04-24T08:59:05.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.075+0000 7f24ca000640 10 client.18348 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:59:05.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.078+0000 7f24ca000640 1 -- 172.21.15.83:0/2012289698 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5621c7e21700 con 0x5621c7f36aa0 2024-04-24T08:59:05.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.078+0000 7f24ca000640 10 client.18348 handle_command_reply: tid=0 2024-04-24T08:59:05.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.078+0000 7f24e5e2a740 1 client.18348 shutdown 2024-04-24T08:59:05.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.078+0000 7f24e5e2a740 1 -- 172.21.15.83:0/2012289698 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5621c7eb49b0 msgr2=0x5621c7ed2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:05.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.078+0000 7f24e5e2a740 1 --2- 172.21.15.83:0/2012289698 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5621c7eb49b0 0x5621c7ed2b50 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f24c4021020 tx=0x7f24c400bf90 comp rx=0 tx=0).stop 2024-04-24T08:59:05.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.078+0000 7f24e5e2a740 1 -- 172.21.15.83:0/2012289698 shutdown_connections 2024-04-24T08:59:05.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.078+0000 7f24e5e2a740 1 --2- 172.21.15.83:0/2012289698 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5621c7f36aa0 0x5621c7f38ec0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:05.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.078+0000 7f24e5e2a740 1 --2- 172.21.15.83:0/2012289698 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5621c7ebf080 0x5621c7ed3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:05.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.078+0000 7f24e5e2a740 1 --2- 172.21.15.83:0/2012289698 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5621c7eb49b0 0x5621c7ed2b50 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:05.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.078+0000 7f24e5e2a740 1 --2- 172.21.15.83:0/2012289698 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5621c7eb4090 0x5621c7ed47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:05.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.078+0000 7f24e5e2a740 1 -- 172.21.15.83:0/2012289698 >> 172.21.15.83:0/2012289698 conn(0x5621c7eb03b0 msgr2=0x5621c7ebbf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:05.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.078+0000 7f24ca000640 0 client.18348 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:59:05.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.078+0000 7f24e5e2a740 1 -- 172.21.15.83:0/2012289698 shutdown_connections 2024-04-24T08:59:05.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.078+0000 7f24e5e2a740 1 -- 172.21.15.83:0/2012289698 wait complete. 2024-04-24T08:59:05.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.079+0000 7f24e5e2a740 20 client.18348 trim_cache size 0 max 16384 2024-04-24T08:59:05.080 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T08:59:05.080 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-04-24T08:59:05.080 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "7de44b22-0069-453b-8919-1c6eb966e694", 2024-04-24T08:59:05.080 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-04-24T08:59:05.080 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T08:59:05.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.079+0000 7f24e3000640 1 -- 172.21.15.83:0/3721689527 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f24c00a3da0 msgr2=0x7f24c00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:05.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.079+0000 7f24e3000640 1 --2- 172.21.15.83:0/3721689527 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f24c00a3da0 0x7f24c00a6260 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x5621c7eb5650 tx=0x7f24cc0084c0 comp rx=0 tx=0).stop 2024-04-24T08:59:05.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.079+0000 7f24e3000640 1 -- 172.21.15.83:0/3721689527 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24dc19a7c0 msgr2=0x7f24dc19ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:05.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.079+0000 7f24e3000640 1 --2- 172.21.15.83:0/3721689527 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24dc19a7c0 0x7f24dc19ac70 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f24d4031040 tx=0x7f24d4003970 comp rx=0 tx=0).stop 2024-04-24T08:59:05.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.079+0000 7f24e3000640 1 -- 172.21.15.83:0/3721689527 shutdown_connections 2024-04-24T08:59:05.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.079+0000 7f24e3000640 1 --2- 172.21.15.83:0/3721689527 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f24c00a3da0 0x7f24c00a6260 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:05.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.079+0000 7f24e3000640 1 --2- 172.21.15.83:0/3721689527 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24dc19a7c0 0x7f24dc19ac70 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:05.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.079+0000 7f24e3000640 1 --2- 172.21.15.83:0/3721689527 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f24dc109520 0x7f24dc1a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:05.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.079+0000 7f24e3000640 1 --2- 172.21.15.83:0/3721689527 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f24dc104450 0x7f24dc1a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:05.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.079+0000 7f24e3000640 1 -- 172.21.15.83:0/3721689527 >> 172.21.15.83:0/3721689527 conn(0x7f24dc0ffc70 msgr2=0x7f24dc101f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:05.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.080+0000 7f24e3000640 1 -- 172.21.15.83:0/3721689527 shutdown_connections 2024-04-24T08:59:05.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.080+0000 7f24e3000640 1 -- 172.21.15.83:0/3721689527 wait complete. 2024-04-24T08:59:05.097 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T08:59:05.097 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T08:59:05.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.300+0000 7f5241200640 1 -- 172.21.15.83:0/49809447 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f523c1093f0 msgr2=0x7f523c110fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:05.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.300+0000 7f5241200640 1 --2- 172.21.15.83:0/49809447 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f523c1093f0 0x7f523c110fc0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f5234009ca0 tx=0x7f523402a8c0 comp rx=0 tx=0).stop 2024-04-24T08:59:05.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.300+0000 7f5241200640 1 -- 172.21.15.83:0/49809447 shutdown_connections 2024-04-24T08:59:05.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.300+0000 7f5241200640 1 --2- 172.21.15.83:0/49809447 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f523c1093f0 0x7f523c110fc0 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:05.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.300+0000 7f5241200640 1 --2- 172.21.15.83:0/49809447 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f523c104cd0 0x7f523c108d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:05.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.300+0000 7f5241200640 1 --2- 172.21.15.83:0/49809447 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f523c104320 0x7f523c104700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:05.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.300+0000 7f5241200640 1 -- 172.21.15.83:0/49809447 >> 172.21.15.83:0/49809447 conn(0x7f523c0ffb40 msgr2=0x7f523c101f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:05.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.301+0000 7f5241200640 1 -- 172.21.15.83:0/49809447 shutdown_connections 2024-04-24T08:59:05.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.301+0000 7f5241200640 1 -- 172.21.15.83:0/49809447 wait complete. 2024-04-24T08:59:05.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.302+0000 7f5241200640 1 Processor -- start 2024-04-24T08:59:05.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.302+0000 7f5241200640 1 -- start start 2024-04-24T08:59:05.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.302+0000 7f5241200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f523c104320 0x7f523c1a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:05.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.302+0000 7f5241200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f523c1093f0 0x7f523c1a0b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:05.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.302+0000 7f5241200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f523c19a6b0 0x7f523c19ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:05.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.302+0000 7f5241200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f523c1143b0 con 0x7f523c19a6b0 2024-04-24T08:59:05.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.302+0000 7f5241200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f523c114230 con 0x7f523c1093f0 2024-04-24T08:59:05.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.302+0000 7f5241200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f523c114530 con 0x7f523c104320 2024-04-24T08:59:05.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.303+0000 7f523be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f523c19a6b0 0x7f523c19ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:05.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.303+0000 7f523be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f523c19a6b0 0x7f523c19ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41500/0 (socket says 172.21.15.83:41500) 2024-04-24T08:59:05.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.303+0000 7f523be00640 1 -- 172.21.15.83:0/3635127638 learned_addr learned my addr 172.21.15.83:0/3635127638 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:59:05.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.303+0000 7f523b400640 1 --2- 172.21.15.83:0/3635127638 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f523c104320 0x7f523c1a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:05.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.303+0000 7f523aa00640 1 --2- 172.21.15.83:0/3635127638 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f523c1093f0 0x7f523c1a0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:05.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.303+0000 7f523be00640 1 -- 172.21.15.83:0/3635127638 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f523c104320 msgr2=0x7f523c1a05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:05.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.303+0000 7f523be00640 1 --2- 172.21.15.83:0/3635127638 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f523c104320 0x7f523c1a05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:05.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.303+0000 7f523be00640 1 -- 172.21.15.83:0/3635127638 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f523c1093f0 msgr2=0x7f523c1a0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:05.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.303+0000 7f523be00640 1 --2- 172.21.15.83:0/3635127638 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f523c1093f0 0x7f523c1a0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:05.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.303+0000 7f523be00640 1 -- 172.21.15.83:0/3635127638 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f523c19b2b0 con 0x7f523c19a6b0 2024-04-24T08:59:05.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.303+0000 7f523be00640 1 --2- 172.21.15.83:0/3635127638 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f523c19a6b0 0x7f523c19ab30 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f5234031040 tx=0x7f5234003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:05.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.304+0000 7f5233e00640 1 -- 172.21.15.83:0/3635127638 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5234003ce0 con 0x7f523c19a6b0 2024-04-24T08:59:05.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.304+0000 7f5233e00640 1 -- 172.21.15.83:0/3635127638 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f52340044f0 con 0x7f523c19a6b0 2024-04-24T08:59:05.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.304+0000 7f5241200640 1 -- 172.21.15.83:0/3635127638 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f523c19b540 con 0x7f523c19a6b0 2024-04-24T08:59:05.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.304+0000 7f5241200640 1 -- 172.21.15.83:0/3635127638 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f523c1a76e0 con 0x7f523c19a6b0 2024-04-24T08:59:05.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.304+0000 7f5233e00640 1 -- 172.21.15.83:0/3635127638 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5234030c30 con 0x7f523c19a6b0 2024-04-24T08:59:05.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.307+0000 7f52457bd740 1 Processor -- start 2024-04-24T08:59:05.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.307+0000 7f5233e00640 1 -- 172.21.15.83:0/3635127638 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f52340046a0 con 0x7f523c19a6b0 2024-04-24T08:59:05.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.308+0000 7f52457bd740 1 -- start start 2024-04-24T08:59:05.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.308+0000 7f52457bd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f287c3a30 0x564f287c3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:05.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.308+0000 7f52457bd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564f287c4350 0x564f287ce4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:05.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.308+0000 7f52457bd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564f287cea20 0x564f287d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:05.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.308+0000 7f52457bd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x564f287291f0 con 0x564f287c4350 2024-04-24T08:59:05.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.308+0000 7f52457bd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x564f286cf350 con 0x564f287c3a30 2024-04-24T08:59:05.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.308+0000 7f52457bd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x564f287251b0 con 0x564f287cea20 2024-04-24T08:59:05.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.308+0000 7f523b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564f287c4350 0x564f287ce4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:05.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.308+0000 7f523b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564f287c4350 0x564f287ce4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41510/0 (socket says 172.21.15.83:41510) 2024-04-24T08:59:05.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.308+0000 7f523b400640 1 -- 172.21.15.83:0/2608229164 learned_addr learned my addr 172.21.15.83:0/2608229164 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:59:05.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.308+0000 7f523aa00640 1 --2- 172.21.15.83:0/2608229164 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f287c3a30 0x564f287c3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:05.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.309+0000 7f523aa00640 1 --2- 172.21.15.83:0/2608229164 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564f287cea20 0x564f287d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:05.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.309+0000 7f523b400640 1 -- 172.21.15.83:0/2608229164 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564f287cea20 msgr2=0x564f287d0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:05.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.309+0000 7f523b400640 1 --2- 172.21.15.83:0/2608229164 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564f287cea20 0x564f287d0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:05.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.309+0000 7f523b400640 1 -- 172.21.15.83:0/2608229164 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f287c3a30 msgr2=0x564f287c3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:05.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.309+0000 7f523b400640 1 --2- 172.21.15.83:0/2608229164 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f287c3a30 0x564f287c3e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:05.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.309+0000 7f523b400640 1 -- 172.21.15.83:0/2608229164 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f28723f50 con 0x564f287c4350 2024-04-24T08:59:05.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.309+0000 7f523aa00640 1 --2- 172.21.15.83:0/2608229164 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f287c3a30 0x564f287c3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:59:05.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.309+0000 7f523b400640 1 --2- 172.21.15.83:0/2608229164 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564f287c4350 0x564f287ce4e0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f522c00bd10 tx=0x7f522c0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faa2d3a04560211f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:05.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.309+0000 7f5230c00640 1 -- 172.21.15.83:0/2608229164 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f522c004030 con 0x564f287c4350 2024-04-24T08:59:05.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.309+0000 7f5230c00640 1 -- 172.21.15.83:0/2608229164 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f522c0035f0 con 0x564f287c4350 2024-04-24T08:59:05.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.309+0000 7f5230c00640 1 -- 172.21.15.83:0/2608229164 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f522c0117b0 con 0x564f287c4350 2024-04-24T08:59:05.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.310+0000 7f5233e00640 1 --2- 172.21.15.83:0/3635127638 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f52200a3d20 0x7f52200a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:05.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.310+0000 7f5233e00640 1 -- 172.21.15.83:0/3635127638 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9671+0+0 (secure 0 0 0) 0x7f52340f21a0 con 0x7f523c19a6b0 2024-04-24T08:59:05.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.310+0000 7f523aa00640 1 --2- 172.21.15.83:0/3635127638 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f52200a3d20 0x7f52200a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:05.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.310+0000 7f52457bd740 1 -- 172.21.15.83:0/2608229164 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564f287c4350 msgr2=0x564f287ce4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:05.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.310+0000 7f52457bd740 1 --2- 172.21.15.83:0/2608229164 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564f287c4350 0x564f287ce4e0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f522c00bd10 tx=0x7f522c0107a0 comp rx=0 tx=0).stop 2024-04-24T08:59:05.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.310+0000 7f52457bd740 1 -- 172.21.15.83:0/2608229164 shutdown_connections 2024-04-24T08:59:05.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.310+0000 7f52457bd740 1 --2- 172.21.15.83:0/2608229164 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564f287cea20 0x564f287d0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:05.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.310+0000 7f52457bd740 1 --2- 172.21.15.83:0/2608229164 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564f287c4350 0x564f287ce4e0 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:05.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.310+0000 7f52457bd740 1 --2- 172.21.15.83:0/2608229164 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f287c3a30 0x564f287c3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:05.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.310+0000 7f52457bd740 1 -- 172.21.15.83:0/2608229164 >> 172.21.15.83:0/2608229164 conn(0x564f287bfd50 msgr2=0x564f287c0130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:05.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.311+0000 7f52457bd740 1 -- 172.21.15.83:0/2608229164 shutdown_connections 2024-04-24T08:59:05.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.311+0000 7f52457bd740 1 -- 172.21.15.83:0/2608229164 wait complete. 2024-04-24T08:59:05.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.311+0000 7f523aa00640 1 --2- 172.21.15.83:0/3635127638 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f52200a3d20 0x7f52200a61e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x564f287c0ee0 tx=0x7f5228004730 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:05.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.311+0000 7f52457bd740 1 -- start start 2024-04-24T08:59:05.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.311+0000 7f52457bd740 1 Processor -- start 2024-04-24T08:59:05.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.312+0000 7f52457bd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564f287c3a30 0x564f287cb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:05.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.312+0000 7f52457bd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564f287c4350 0x564f287cbf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:05.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.312+0000 7f52457bd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f287cea20 0x564f287cc460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:05.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.312+0000 7f52457bd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x564f287d1810 con 0x564f287c3a30 2024-04-24T08:59:05.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.312+0000 7f52457bd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x564f286e2690 con 0x564f287cea20 2024-04-24T08:59:05.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.312+0000 7f52457bd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x564f287d1990 con 0x564f287c4350 2024-04-24T08:59:05.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.312+0000 7f52457bd740 10 client.0 _ll_register_callbacks cb 0x564f2871e650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:59:05.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.312+0000 7f523b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564f287c3a30 0x564f287cb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:05.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.312+0000 7f523b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564f287c3a30 0x564f287cb9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41526/0 (socket says 172.21.15.83:41526) 2024-04-24T08:59:05.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.312+0000 7f523b400640 1 -- 172.21.15.83:0/3244941460 learned_addr learned my addr 172.21.15.83:0/3244941460 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:59:05.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.312+0000 7f523aa00640 1 --2- 172.21.15.83:0/3244941460 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564f287c4350 0x564f287cbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:05.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.313+0000 7f523be00640 1 --2- 172.21.15.83:0/3244941460 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f287cea20 0x564f287cc460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:05.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.313+0000 7f523b400640 1 -- 172.21.15.83:0/3244941460 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564f287c4350 msgr2=0x564f287cbf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:05.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.313+0000 7f523b400640 1 --2- 172.21.15.83:0/3244941460 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564f287c4350 0x564f287cbf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:05.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.313+0000 7f523b400640 1 -- 172.21.15.83:0/3244941460 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f287cea20 msgr2=0x564f287cc460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:05.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.313+0000 7f523b400640 1 --2- 172.21.15.83:0/3244941460 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f287cea20 0x564f287cc460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:05.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.313+0000 7f523b400640 1 -- 172.21.15.83:0/3244941460 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f286e1920 con 0x564f287c3a30 2024-04-24T08:59:05.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.313+0000 7f523b400640 1 --2- 172.21.15.83:0/3244941460 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564f287c3a30 0x564f287cb9e0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f522c00b950 tx=0x7f522c00b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c35c00308cb580b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:05.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.313+0000 7f5225600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:59:05.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.313+0000 7f5225600640 1 -- 172.21.15.83:0/3244941460 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f522c029060 con 0x564f287c3a30 2024-04-24T08:59:05.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.313+0000 7f5225600640 1 -- 172.21.15.83:0/3244941460 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9671+0+0 (secure 0 0 0) 0x7f522c004000 con 0x564f287c3a30 2024-04-24T08:59:05.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.313+0000 7f52457bd740 1 -- 172.21.15.83:0/3244941460 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f522c0035f0 con 0x564f287c3a30 2024-04-24T08:59:05.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.314+0000 7f5225600640 1 -- 172.21.15.83:0/3244941460 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f522c030050 con 0x564f287c3a30 2024-04-24T08:59:05.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.314+0000 7f52457bd740 10 client.18360 fetch_fsmap learned FSMap version 33 2024-04-24T08:59:05.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.314+0000 7f52457bd740 1 -- 172.21.15.83:0/3244941460 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x564f285057d0 con 0x564f287c3a30 2024-04-24T08:59:05.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.314+0000 7f5225600640 1 -- 172.21.15.83:0/3244941460 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f522c0035f0 con 0x564f287c3a30 2024-04-24T08:59:05.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.314+0000 7f5225600640 1 -- 172.21.15.83:0/3244941460 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f522c023030 con 0x564f287c3a30 2024-04-24T08:59:05.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.314+0000 7f52457bd740 10 client.18360 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T08:59:05.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.314+0000 7f52457bd740 10 client.18360 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:59:05.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.314+0000 7f52457bd740 20 client.18360 populate_metadata read hostname 'smithi083' 2024-04-24T08:59:05.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.315+0000 7f52457bd740 1 --2- 172.21.15.83:0/3244941460 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x564f287dece0 0x564f287e10d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:05.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.315+0000 7f52457bd740 4 client.18360 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:59:05.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.315+0000 7f52457bd740 1 -- 172.21.15.83:0/3244941460 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564f28736c10 con 0x564f287dece0 2024-04-24T08:59:05.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.315+0000 7f523aa00640 1 --2- 172.21.15.83:0/3244941460 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x564f287dece0 0x564f287e10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:05.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.316+0000 7f523aa00640 1 --2- 172.21.15.83:0/3244941460 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x564f287dece0 0x564f287e10d0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ce848b09859b099 server_cookie=c1aaf17bf4ff5757 in_seq=0 out_seq=0 2024-04-24T08:59:05.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.316+0000 7f5225600640 10 client.18360 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:59:05.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.318+0000 7f5225600640 1 -- 172.21.15.83:0/3244941460 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564f28736c10 con 0x564f287dece0 2024-04-24T08:59:05.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.318+0000 7f5225600640 10 client.18360 handle_command_reply: tid=0 2024-04-24T08:59:05.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.318+0000 7f52457bd740 1 client.18360 shutdown 2024-04-24T08:59:05.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.318+0000 7f52457bd740 1 -- 172.21.15.83:0/3244941460 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564f287c3a30 msgr2=0x564f287cb9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:05.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.318+0000 7f52457bd740 1 --2- 172.21.15.83:0/3244941460 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564f287c3a30 0x564f287cb9e0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f522c00b950 tx=0x7f522c00b980 comp rx=0 tx=0).stop 2024-04-24T08:59:05.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.319+0000 7f52457bd740 1 -- 172.21.15.83:0/3244941460 shutdown_connections 2024-04-24T08:59:05.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.319+0000 7f52457bd740 1 --2- 172.21.15.83:0/3244941460 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x564f287dece0 0x564f287e10d0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:05.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.319+0000 7f52457bd740 1 --2- 172.21.15.83:0/3244941460 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f287cea20 0x564f287cc460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:05.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.319+0000 7f52457bd740 1 --2- 172.21.15.83:0/3244941460 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564f287c4350 0x564f287cbf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:05.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.319+0000 7f52457bd740 1 --2- 172.21.15.83:0/3244941460 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564f287c3a30 0x564f287cb9e0 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:05.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.319+0000 7f52457bd740 1 -- 172.21.15.83:0/3244941460 >> 172.21.15.83:0/3244941460 conn(0x564f287bfd50 msgr2=0x564f287befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:05.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.319+0000 7f5225600640 0 client.18360 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:59:05.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.319+0000 7f52457bd740 1 -- 172.21.15.83:0/3244941460 shutdown_connections 2024-04-24T08:59:05.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.319+0000 7f52457bd740 1 -- 172.21.15.83:0/3244941460 wait complete. 2024-04-24T08:59:05.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.319+0000 7f52457bd740 20 client.18360 trim_cache size 0 max 16384 2024-04-24T08:59:05.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.338+0000 7f52457bd740 1 Processor -- start 2024-04-24T08:59:05.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.339+0000 7f52457bd740 1 -- start start 2024-04-24T08:59:05.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.339+0000 7f52457bd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564f287c3a30 0x564f287c5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:05.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.339+0000 7f52457bd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564f287c4350 0x564f287da6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:05.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.339+0000 7f52457bd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f287cea20 0x564f287e66d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:05.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.339+0000 7f52457bd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f523c1b1730 con 0x564f287c3a30 2024-04-24T08:59:05.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.339+0000 7f52457bd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f523c1b3270 con 0x564f287cea20 2024-04-24T08:59:05.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.339+0000 7f52457bd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f523c1b1e00 con 0x564f287c4350 2024-04-24T08:59:05.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.339+0000 7f523b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564f287c3a30 0x564f287c5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:05.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.339+0000 7f523b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564f287c3a30 0x564f287c5cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41538/0 (socket says 172.21.15.83:41538) 2024-04-24T08:59:05.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.339+0000 7f523b400640 1 -- 172.21.15.83:0/2358118027 learned_addr learned my addr 172.21.15.83:0/2358118027 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:59:05.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.340+0000 7f523aa00640 1 --2- 172.21.15.83:0/2358118027 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564f287c4350 0x564f287da6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:05.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.340+0000 7f523b400640 1 -- 172.21.15.83:0/2358118027 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564f287c4350 msgr2=0x564f287da6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:05.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.340+0000 7f523b400640 1 --2- 172.21.15.83:0/2358118027 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564f287c4350 0x564f287da6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:05.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.340+0000 7f523b400640 1 -- 172.21.15.83:0/2358118027 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f287cea20 msgr2=0x564f287e66d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:05.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.340+0000 7f523b400640 1 --2- 172.21.15.83:0/2358118027 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f287cea20 0x564f287e66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:05.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.340+0000 7f523b400640 1 -- 172.21.15.83:0/2358118027 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f2843b3f0 con 0x564f287c3a30 2024-04-24T08:59:05.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.340+0000 7f523aa00640 1 --2- 172.21.15.83:0/2358118027 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564f287c4350 0x564f287da6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:59:05.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.340+0000 7f523b400640 1 --2- 172.21.15.83:0/2358118027 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564f287c3a30 0x564f287c5cd0 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f522c005940 tx=0x7f522c0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7442671d15f764e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:05.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.340+0000 7f5227e00640 1 -- 172.21.15.83:0/2358118027 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f522c029070 con 0x564f287c3a30 2024-04-24T08:59:05.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.340+0000 7f5227e00640 1 -- 172.21.15.83:0/2358118027 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f522c003a30 con 0x564f287c3a30 2024-04-24T08:59:05.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.340+0000 7f5227e00640 1 -- 172.21.15.83:0/2358118027 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f522c0117d0 con 0x564f287c3a30 2024-04-24T08:59:05.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.341+0000 7f52457bd740 1 -- 172.21.15.83:0/2358118027 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564f287c3a30 msgr2=0x564f287c5cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:05.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.341+0000 7f52457bd740 1 --2- 172.21.15.83:0/2358118027 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564f287c3a30 0x564f287c5cd0 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f522c005940 tx=0x7f522c0107a0 comp rx=0 tx=0).stop 2024-04-24T08:59:05.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.341+0000 7f52457bd740 1 -- 172.21.15.83:0/2358118027 shutdown_connections 2024-04-24T08:59:05.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.341+0000 7f52457bd740 1 --2- 172.21.15.83:0/2358118027 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f287cea20 0x564f287e66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:05.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.341+0000 7f52457bd740 1 --2- 172.21.15.83:0/2358118027 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564f287c4350 0x564f287da6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:05.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.341+0000 7f52457bd740 1 --2- 172.21.15.83:0/2358118027 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564f287c3a30 0x564f287c5cd0 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:05.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.341+0000 7f52457bd740 1 -- 172.21.15.83:0/2358118027 >> 172.21.15.83:0/2358118027 conn(0x564f287bfd50 msgr2=0x564f287befd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:05.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.341+0000 7f52457bd740 1 -- 172.21.15.83:0/2358118027 shutdown_connections 2024-04-24T08:59:05.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.342+0000 7f52457bd740 1 -- 172.21.15.83:0/2358118027 wait complete. 2024-04-24T08:59:05.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.342+0000 7f52457bd740 1 -- start start 2024-04-24T08:59:05.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.342+0000 7f52457bd740 1 Processor -- start 2024-04-24T08:59:05.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.343+0000 7f52457bd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564f287c3a30 0x564f287c8d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:05.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.343+0000 7f52457bd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564f287c4350 0x564f287c9290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:05.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.343+0000 7f52457bd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f287cea20 0x564f28843f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:05.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.343+0000 7f52457bd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f523c1b2ba0 con 0x564f287c4350 2024-04-24T08:59:05.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.343+0000 7f52457bd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f523c1b24d0 con 0x564f287cea20 2024-04-24T08:59:05.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.343+0000 7f52457bd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f523c1af730 con 0x564f287c3a30 2024-04-24T08:59:05.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.343+0000 7f52457bd740 10 client.0 _ll_register_callbacks cb 0x7f522c00bb60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:59:05.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.343+0000 7f523aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564f287c4350 0x564f287c9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:05.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.343+0000 7f523aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564f287c4350 0x564f287c9290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41550/0 (socket says 172.21.15.83:41550) 2024-04-24T08:59:05.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.343+0000 7f523aa00640 1 -- 172.21.15.83:0/2643920093 learned_addr learned my addr 172.21.15.83:0/2643920093 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:59:05.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.343+0000 7f523b400640 1 --2- 172.21.15.83:0/2643920093 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564f287c3a30 0x564f287c8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:05.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.343+0000 7f523be00640 1 --2- 172.21.15.83:0/2643920093 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f287cea20 0x564f28843f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:05.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.343+0000 7f523aa00640 1 -- 172.21.15.83:0/2643920093 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564f287c3a30 msgr2=0x564f287c8d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:05.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.343+0000 7f523aa00640 1 --2- 172.21.15.83:0/2643920093 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564f287c3a30 0x564f287c8d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:05.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.344+0000 7f523aa00640 1 -- 172.21.15.83:0/2643920093 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f287cea20 msgr2=0x564f28843f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:05.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.344+0000 7f523aa00640 1 --2- 172.21.15.83:0/2643920093 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f287cea20 0x564f28843f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:05.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.344+0000 7f523aa00640 1 -- 172.21.15.83:0/2643920093 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f283fb0f0 con 0x564f287c4350 2024-04-24T08:59:05.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.344+0000 7f523aa00640 1 --2- 172.21.15.83:0/2643920093 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564f287c4350 0x564f287c9290 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f52280058c0 tx=0x7f5228005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a674703db31aa235 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:05.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.344+0000 7f5225600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:59:05.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.344+0000 7f5225600640 1 -- 172.21.15.83:0/2643920093 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5228029330 con 0x564f287c4350 2024-04-24T08:59:05.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.344+0000 7f5225600640 1 -- 172.21.15.83:0/2643920093 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9671+0+0 (secure 0 0 0) 0x7f52280294d0 con 0x564f287c4350 2024-04-24T08:59:05.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.344+0000 7f52457bd740 1 -- 172.21.15.83:0/2643920093 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f522c003a30 con 0x564f287c4350 2024-04-24T08:59:05.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.345+0000 7f5225600640 1 -- 172.21.15.83:0/2643920093 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f522803a050 con 0x564f287c4350 2024-04-24T08:59:05.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.345+0000 7f52457bd740 10 client.18366 fetch_fsmap learned FSMap version 33 2024-04-24T08:59:05.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.345+0000 7f52457bd740 1 -- 172.21.15.83:0/2643920093 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x564f286ee810 con 0x564f287c4350 2024-04-24T08:59:05.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.345+0000 7f5225600640 1 -- 172.21.15.83:0/2643920093 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f522c003a30 con 0x564f287c4350 2024-04-24T08:59:05.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.345+0000 7f5225600640 1 -- 172.21.15.83:0/2643920093 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7f5228020400 con 0x564f287c4350 2024-04-24T08:59:05.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.345+0000 7f52457bd740 10 client.18366 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T08:59:05.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.345+0000 7f52457bd740 10 client.18366 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:59:05.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.345+0000 7f52457bd740 20 client.18366 populate_metadata read hostname 'smithi083' 2024-04-24T08:59:05.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.345+0000 7f52457bd740 1 --2- 172.21.15.83:0/2643920093 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x564f28845ab0 0x564f28847ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:05.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.345+0000 7f52457bd740 4 client.18366 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T08:59:05.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.345+0000 7f52457bd740 1 -- 172.21.15.83:0/2643920093 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564f28730a80 con 0x564f28845ab0 2024-04-24T08:59:05.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.346+0000 7f523b400640 1 --2- 172.21.15.83:0/2643920093 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x564f28845ab0 0x564f28847ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:05.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.347+0000 7f523b400640 1 --2- 172.21.15.83:0/2643920093 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x564f28845ab0 0x564f28847ed0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da67d159c2270289 server_cookie=695aa8f658ac6d6b in_seq=0 out_seq=0 2024-04-24T08:59:05.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.347+0000 7f5225600640 10 client.18366 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:59:05.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.348+0000 7f5225600640 1 -- 172.21.15.83:0/2643920093 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x564f28730a80 con 0x564f28845ab0 2024-04-24T08:59:05.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.348+0000 7f5225600640 10 client.18366 handle_command_reply: tid=0 2024-04-24T08:59:05.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.348+0000 7f52457bd740 1 client.18366 shutdown 2024-04-24T08:59:05.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.348+0000 7f52457bd740 1 -- 172.21.15.83:0/2643920093 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564f287c4350 msgr2=0x564f287c9290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:05.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.348+0000 7f52457bd740 1 --2- 172.21.15.83:0/2643920093 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564f287c4350 0x564f287c9290 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f52280058c0 tx=0x7f5228005850 comp rx=0 tx=0).stop 2024-04-24T08:59:05.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.348+0000 7f52457bd740 1 -- 172.21.15.83:0/2643920093 shutdown_connections 2024-04-24T08:59:05.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.348+0000 7f52457bd740 1 --2- 172.21.15.83:0/2643920093 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x564f28845ab0 0x564f28847ed0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:05.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.349+0000 7f52457bd740 1 --2- 172.21.15.83:0/2643920093 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f287cea20 0x564f28843f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:05.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.349+0000 7f52457bd740 1 --2- 172.21.15.83:0/2643920093 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564f287c4350 0x564f287c9290 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:05.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.349+0000 7f52457bd740 1 --2- 172.21.15.83:0/2643920093 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564f287c3a30 0x564f287c8d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:05.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.349+0000 7f5225600640 0 client.18366 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:59:05.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.349+0000 7f52457bd740 1 -- 172.21.15.83:0/2643920093 >> 172.21.15.83:0/2643920093 conn(0x564f287bfd50 msgr2=0x564f287cb980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:05.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.349+0000 7f52457bd740 1 -- 172.21.15.83:0/2643920093 shutdown_connections 2024-04-24T08:59:05.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.349+0000 7f52457bd740 1 -- 172.21.15.83:0/2643920093 wait complete. 2024-04-24T08:59:05.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.349+0000 7f52457bd740 20 client.18366 trim_cache size 0 max 16384 2024-04-24T08:59:05.352 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T08:59:05.352 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T08:59:05.353 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-04-24T08:59:05.353 INFO:teuthology.orchestra.run.smithi083.stdout: "7de44b22-0069-453b-8919-1c6eb966e694": { 2024-04-24T08:59:05.353 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-04-24T08:59:05.353 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "7de44b22-0069-453b-8919-1c6eb966e694", 2024-04-24T08:59:05.353 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-04-24T08:59:05.353 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T08:59:05.353 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T08:59:05.353 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T08:59:05.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.350+0000 7f5241200640 1 -- 172.21.15.83:0/3635127638 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f52200a3d20 msgr2=0x7f52200a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:05.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.350+0000 7f5241200640 1 --2- 172.21.15.83:0/3635127638 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f52200a3d20 0x7f52200a61e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x564f287c0ee0 tx=0x7f5228004730 comp rx=0 tx=0).stop 2024-04-24T08:59:05.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.350+0000 7f5241200640 1 -- 172.21.15.83:0/3635127638 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f523c19a6b0 msgr2=0x7f523c19ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:05.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.350+0000 7f5241200640 1 --2- 172.21.15.83:0/3635127638 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f523c19a6b0 0x7f523c19ab30 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f5234031040 tx=0x7f5234003970 comp rx=0 tx=0).stop 2024-04-24T08:59:05.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.350+0000 7f5241200640 1 -- 172.21.15.83:0/3635127638 shutdown_connections 2024-04-24T08:59:05.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.350+0000 7f5241200640 1 --2- 172.21.15.83:0/3635127638 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f52200a3d20 0x7f52200a61e0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:05.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.350+0000 7f5241200640 1 --2- 172.21.15.83:0/3635127638 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f523c19a6b0 0x7f523c19ab30 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:05.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.350+0000 7f5241200640 1 --2- 172.21.15.83:0/3635127638 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f523c1093f0 0x7f523c1a0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:05.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.350+0000 7f5241200640 1 --2- 172.21.15.83:0/3635127638 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f523c104320 0x7f523c1a05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:05.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.350+0000 7f5241200640 1 -- 172.21.15.83:0/3635127638 >> 172.21.15.83:0/3635127638 conn(0x7f523c0ffb40 msgr2=0x7f523c101e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:05.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.350+0000 7f5241200640 1 -- 172.21.15.83:0/3635127638 shutdown_connections 2024-04-24T08:59:05.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:05.350+0000 7f5241200640 1 -- 172.21.15.83:0/3635127638 wait complete. 2024-04-24T08:59:05.368 INFO:tasks.cephfs.filesystem:scrub status for tag:7de44b22-0069-453b-8919-1c6eb966e694 - {'path': '/', 'tag': '7de44b22-0069-453b-8919-1c6eb966e694', 'options': 'recursive,force'} 2024-04-24T08:59:05.591 INFO:tasks.workunit.client.0.smithi083.stdout: 1 40555 0.63 MB/sec execute 219 sec latency 69.142 ms 2024-04-24T08:59:05.928 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.928+0000 7fc295a00640 1 -- 172.21.15.83:0/586060890 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc290108540 msgr2=0x7fc290108980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:05.928 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.928+0000 7fc295a00640 1 --2- 172.21.15.83:0/586060890 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc290108540 0x7fc290108980 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7fc278009930 tx=0x7fc27802a420 comp rx=0 tx=0).stop 2024-04-24T08:59:05.928 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.928+0000 7fc295a00640 1 -- 172.21.15.83:0/586060890 shutdown_connections 2024-04-24T08:59:05.928 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.928+0000 7fc295a00640 1 --2- 172.21.15.83:0/586060890 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc290108ec0 0x7fc290115300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:05.928 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.928+0000 7fc295a00640 1 --2- 172.21.15.83:0/586060890 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc290108540 0x7fc290108980 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:05.929 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.928+0000 7fc295a00640 1 --2- 172.21.15.83:0/586060890 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc29010e4d0 0x7fc29010e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:05.929 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.928+0000 7fc295a00640 1 -- 172.21.15.83:0/586060890 >> 172.21.15.83:0/586060890 conn(0x7fc290077de0 msgr2=0x7fc290106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:05.929 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.928+0000 7fc295a00640 1 -- 172.21.15.83:0/586060890 shutdown_connections 2024-04-24T08:59:05.929 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.928+0000 7fc295a00640 1 -- 172.21.15.83:0/586060890 wait complete. 2024-04-24T08:59:05.931 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.929+0000 7fc295a00640 1 Processor -- start 2024-04-24T08:59:05.931 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.929+0000 7fc295a00640 1 -- start start 2024-04-24T08:59:05.931 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.929+0000 7fc295a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc290108540 0x7fc2901a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:05.931 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.930+0000 7fc295a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc290108ec0 0x7fc2901a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:05.931 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.930+0000 7fc295a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc29010e4d0 0x7fc29019e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:05.931 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.930+0000 7fc295a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc290118680 con 0x7fc290108ec0 2024-04-24T08:59:05.931 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.930+0000 7fc295a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc290118500 con 0x7fc290108540 2024-04-24T08:59:05.932 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.930+0000 7fc295a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc290118800 con 0x7fc29010e4d0 2024-04-24T08:59:05.932 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.930+0000 7fc28f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc290108ec0 0x7fc2901a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:05.932 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.930+0000 7fc28f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc290108ec0 0x7fc2901a4d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41568/0 (socket says 172.21.15.83:41568) 2024-04-24T08:59:05.932 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.930+0000 7fc28f400640 1 -- 172.21.15.83:0/601892629 learned_addr learned my addr 172.21.15.83:0/601892629 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:59:05.932 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.930+0000 7fc28fe00640 1 --2- 172.21.15.83:0/601892629 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc290108540 0x7fc2901a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:05.932 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.930+0000 7fc295000640 1 --2- 172.21.15.83:0/601892629 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc29010e4d0 0x7fc29019e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:05.932 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.930+0000 7fc28f400640 1 -- 172.21.15.83:0/601892629 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc29010e4d0 msgr2=0x7fc29019e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:05.932 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.930+0000 7fc28f400640 1 --2- 172.21.15.83:0/601892629 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc29010e4d0 0x7fc29019e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:05.932 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.930+0000 7fc28f400640 1 -- 172.21.15.83:0/601892629 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc290108540 msgr2=0x7fc2901a4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:05.932 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.930+0000 7fc28f400640 1 --2- 172.21.15.83:0/601892629 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc290108540 0x7fc2901a4810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:05.932 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.930+0000 7fc28f400640 1 -- 172.21.15.83:0/601892629 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc29019f140 con 0x7fc290108ec0 2024-04-24T08:59:05.932 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.930+0000 7fc295000640 1 --2- 172.21.15.83:0/601892629 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc29010e4d0 0x7fc29019e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:59:05.932 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.931+0000 7fc28f400640 1 --2- 172.21.15.83:0/601892629 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc290108ec0 0x7fc2901a4d50 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7fc278005940 tx=0x7fc2780042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:05.933 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.931+0000 7fc28cc00640 1 -- 172.21.15.83:0/601892629 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc27803f030 con 0x7fc290108ec0 2024-04-24T08:59:05.933 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.931+0000 7fc28cc00640 1 -- 172.21.15.83:0/601892629 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc278003cb0 con 0x7fc290108ec0 2024-04-24T08:59:05.933 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.931+0000 7fc295a00640 1 -- 172.21.15.83:0/601892629 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc29019f3d0 con 0x7fc290108ec0 2024-04-24T08:59:05.933 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.931+0000 7fc28cc00640 1 -- 172.21.15.83:0/601892629 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc278035070 con 0x7fc290108ec0 2024-04-24T08:59:05.933 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.931+0000 7fc295a00640 1 -- 172.21.15.83:0/601892629 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2901ab6d0 con 0x7fc290108ec0 2024-04-24T08:59:05.936 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.935+0000 7fc299fca740 1 Processor -- start 2024-04-24T08:59:05.936 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.935+0000 7fc28cc00640 1 -- 172.21.15.83:0/601892629 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc278042030 con 0x7fc290108ec0 2024-04-24T08:59:05.936 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.935+0000 7fc299fca740 1 -- start start 2024-04-24T08:59:05.936 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.935+0000 7fc299fca740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bdf5663c0 0x562bdf5667a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:05.936 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.935+0000 7fc299fca740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562bdf566ce0 0x562bdf571060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:05.936 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.935+0000 7fc299fca740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bdf5715a0 0x562bdf573990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:05.936 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.935+0000 7fc299fca740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562bdf4cc900 con 0x562bdf5663c0 2024-04-24T08:59:05.936 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.935+0000 7fc299fca740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562bdf472f20 con 0x562bdf566ce0 2024-04-24T08:59:05.936 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.935+0000 7fc299fca740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x562bdf4c88c0 con 0x562bdf5715a0 2024-04-24T08:59:05.936 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.936+0000 7fc295000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bdf5663c0 0x562bdf5667a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:05.936 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.936+0000 7fc295000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bdf5663c0 0x562bdf5667a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41580/0 (socket says 172.21.15.83:41580) 2024-04-24T08:59:05.936 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.936+0000 7fc295000640 1 -- 172.21.15.83:0/113100328 learned_addr learned my addr 172.21.15.83:0/113100328 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:59:05.936 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.936+0000 7fc28f400640 1 --2- 172.21.15.83:0/113100328 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bdf5715a0 0x562bdf573990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:05.937 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.936+0000 7fc28fe00640 1 --2- 172.21.15.83:0/113100328 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562bdf566ce0 0x562bdf571060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:05.938 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.936+0000 7fc295000640 1 -- 172.21.15.83:0/113100328 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bdf5715a0 msgr2=0x562bdf573990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:05.938 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.936+0000 7fc295000640 1 --2- 172.21.15.83:0/113100328 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bdf5715a0 0x562bdf573990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:05.938 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.936+0000 7fc295000640 1 -- 172.21.15.83:0/113100328 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562bdf566ce0 msgr2=0x562bdf571060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:05.938 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.936+0000 7fc295000640 1 --2- 172.21.15.83:0/113100328 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562bdf566ce0 0x562bdf571060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:05.938 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.936+0000 7fc295000640 1 -- 172.21.15.83:0/113100328 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562bdf497e10 con 0x562bdf5663c0 2024-04-24T08:59:05.938 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.937+0000 7fc295000640 1 --2- 172.21.15.83:0/113100328 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bdf5663c0 0x562bdf5667a0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7fc280008e60 tx=0x7fc28000d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57bcffbe13818784 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:05.938 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.937+0000 7fc28cc00640 1 --2- 172.21.15.83:0/601892629 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fc2740a3d10 0x7fc2740a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:05.939 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.937+0000 7fc28cc00640 1 -- 172.21.15.83:0/601892629 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9671+0+0 (secure 0 0 0) 0x7fc2780f1300 con 0x7fc290108ec0 2024-04-24T08:59:05.939 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.937+0000 7fc28fe00640 1 --2- 172.21.15.83:0/601892629 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fc2740a3d10 0x7fc2740a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:05.939 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.937+0000 7fc285600640 1 -- 172.21.15.83:0/113100328 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc28000f830 con 0x562bdf5663c0 2024-04-24T08:59:05.939 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.937+0000 7fc285600640 1 -- 172.21.15.83:0/113100328 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc280018480 con 0x562bdf5663c0 2024-04-24T08:59:05.939 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.937+0000 7fc285600640 1 -- 172.21.15.83:0/113100328 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc28000e5b0 con 0x562bdf5663c0 2024-04-24T08:59:05.939 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.938+0000 7fc299fca740 1 -- 172.21.15.83:0/113100328 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bdf5663c0 msgr2=0x562bdf5667a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:05.939 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.938+0000 7fc28fe00640 1 --2- 172.21.15.83:0/601892629 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fc2740a3d10 0x7fc2740a61d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fc288005d90 tx=0x7fc288005d20 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:05.939 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.938+0000 7fc299fca740 1 --2- 172.21.15.83:0/113100328 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bdf5663c0 0x562bdf5667a0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7fc280008e60 tx=0x7fc28000d710 comp rx=0 tx=0).stop 2024-04-24T08:59:05.939 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.938+0000 7fc299fca740 1 -- 172.21.15.83:0/113100328 shutdown_connections 2024-04-24T08:59:05.939 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.938+0000 7fc299fca740 1 --2- 172.21.15.83:0/113100328 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bdf5715a0 0x562bdf573990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:05.939 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.938+0000 7fc299fca740 1 --2- 172.21.15.83:0/113100328 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562bdf566ce0 0x562bdf571060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:05.939 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.938+0000 7fc299fca740 1 --2- 172.21.15.83:0/113100328 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bdf5663c0 0x562bdf5667a0 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:05.939 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.938+0000 7fc299fca740 1 -- 172.21.15.83:0/113100328 >> 172.21.15.83:0/113100328 conn(0x562bdf5628d0 msgr2=0x562bdf562cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:05.939 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.938+0000 7fc299fca740 1 -- 172.21.15.83:0/113100328 shutdown_connections 2024-04-24T08:59:05.939 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.938+0000 7fc299fca740 1 -- 172.21.15.83:0/113100328 wait complete. 2024-04-24T08:59:05.942 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.939+0000 7fc299fca740 1 -- start start 2024-04-24T08:59:05.942 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.939+0000 7fc299fca740 1 Processor -- start 2024-04-24T08:59:05.942 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.939+0000 7fc299fca740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bdf5663c0 0x562bdf56e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:05.943 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.939+0000 7fc299fca740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562bdf566ce0 0x562bdf56eb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:05.943 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.939+0000 7fc299fca740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bdf5715a0 0x562bdf56f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:05.943 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.939+0000 7fc299fca740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562bdf574170 con 0x562bdf5663c0 2024-04-24T08:59:05.943 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.939+0000 7fc299fca740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562bdf4862c0 con 0x562bdf566ce0 2024-04-24T08:59:05.943 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.939+0000 7fc299fca740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x562bdf5742f0 con 0x562bdf5715a0 2024-04-24T08:59:05.943 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.939+0000 7fc299fca740 10 client.0 _ll_register_callbacks cb 0x562bdf1dea00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:59:05.943 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.940+0000 7fc28fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bdf5663c0 0x562bdf56e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:05.943 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.940+0000 7fc28fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bdf5663c0 0x562bdf56e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41584/0 (socket says 172.21.15.83:41584) 2024-04-24T08:59:05.943 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.940+0000 7fc28fe00640 1 -- 172.21.15.83:0/2085846222 learned_addr learned my addr 172.21.15.83:0/2085846222 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:59:05.943 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.940+0000 7fc295000640 1 --2- 172.21.15.83:0/2085846222 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bdf5715a0 0x562bdf56f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:05.943 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.940+0000 7fc28f400640 1 --2- 172.21.15.83:0/2085846222 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562bdf566ce0 0x562bdf56eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:05.943 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.940+0000 7fc28fe00640 1 -- 172.21.15.83:0/2085846222 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bdf5715a0 msgr2=0x562bdf56f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:05.943 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.940+0000 7fc28fe00640 1 --2- 172.21.15.83:0/2085846222 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bdf5715a0 0x562bdf56f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:05.943 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.940+0000 7fc28fe00640 1 -- 172.21.15.83:0/2085846222 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562bdf566ce0 msgr2=0x562bdf56eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:05.944 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.940+0000 7fc28fe00640 1 --2- 172.21.15.83:0/2085846222 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562bdf566ce0 0x562bdf56eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:05.944 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.940+0000 7fc28fe00640 1 -- 172.21.15.83:0/2085846222 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562bdf496fd0 con 0x562bdf5663c0 2024-04-24T08:59:05.944 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.941+0000 7fc28fe00640 1 --2- 172.21.15.83:0/2085846222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bdf5663c0 0x562bdf56e610 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7fc288011a70 tx=0x7fc2880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12599bc4a89ce8c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:05.944 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.941+0000 7fc27e000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:59:05.944 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.941+0000 7fc27e000640 1 -- 172.21.15.83:0/2085846222 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc28800ed90 con 0x562bdf5663c0 2024-04-24T08:59:05.944 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.941+0000 7fc27e000640 1 -- 172.21.15.83:0/2085846222 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9671+0+0 (secure 0 0 0) 0x7fc288004020 con 0x562bdf5663c0 2024-04-24T08:59:05.944 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.941+0000 7fc299fca740 1 -- 172.21.15.83:0/2085846222 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc280018480 con 0x562bdf5663c0 2024-04-24T08:59:05.944 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.941+0000 7fc27e000640 1 -- 172.21.15.83:0/2085846222 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fc28802e050 con 0x562bdf5663c0 2024-04-24T08:59:05.944 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.941+0000 7fc299fca740 10 client.18378 fetch_fsmap learned FSMap version 33 2024-04-24T08:59:05.944 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.941+0000 7fc299fca740 1 -- 172.21.15.83:0/2085846222 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x562bdf1df210 con 0x562bdf5663c0 2024-04-24T08:59:05.944 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.942+0000 7fc27e000640 1 -- 172.21.15.83:0/2085846222 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc280018480 con 0x562bdf5663c0 2024-04-24T08:59:05.944 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.942+0000 7fc27e000640 1 -- 172.21.15.83:0/2085846222 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fc28801e420 con 0x562bdf5663c0 2024-04-24T08:59:05.944 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.942+0000 7fc299fca740 10 client.18378 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T08:59:05.944 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.942+0000 7fc299fca740 10 client.18378 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T08:59:05.945 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.942+0000 7fc299fca740 20 client.18378 populate_metadata read hostname 'smithi083' 2024-04-24T08:59:05.945 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.942+0000 7fc299fca740 1 --2- 172.21.15.83:0/2085846222 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x562bdf581860 0x562bdf583cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:05.945 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.942+0000 7fc299fca740 4 client.18378 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:59:05.945 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.942+0000 7fc299fca740 1 -- 172.21.15.83:0/2085846222 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562bdf4da320 con 0x562bdf581860 2024-04-24T08:59:05.945 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.943+0000 7fc28f400640 1 --2- 172.21.15.83:0/2085846222 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x562bdf581860 0x562bdf583cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:05.945 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.944+0000 7fc28f400640 1 --2- 172.21.15.83:0/2085846222 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x562bdf581860 0x562bdf583cb0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52bab24a7335047e server_cookie=409edd3f79247a17 in_seq=0 out_seq=0 2024-04-24T08:59:05.945 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.944+0000 7fc27e000640 10 client.18378 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:59:05.948 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.947+0000 7fc27e000640 1 -- 172.21.15.83:0/2085846222 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562bdf4da320 con 0x562bdf581860 2024-04-24T08:59:05.948 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.947+0000 7fc27e000640 10 client.18378 handle_command_reply: tid=0 2024-04-24T08:59:05.948 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.947+0000 7fc299fca740 1 client.18378 shutdown 2024-04-24T08:59:05.948 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.947+0000 7fc299fca740 1 -- 172.21.15.83:0/2085846222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bdf5663c0 msgr2=0x562bdf56e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:05.948 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.947+0000 7fc299fca740 1 --2- 172.21.15.83:0/2085846222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bdf5663c0 0x562bdf56e610 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7fc288011a70 tx=0x7fc2880046c0 comp rx=0 tx=0).stop 2024-04-24T08:59:05.948 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.948+0000 7fc299fca740 1 -- 172.21.15.83:0/2085846222 shutdown_connections 2024-04-24T08:59:05.948 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.948+0000 7fc299fca740 1 --2- 172.21.15.83:0/2085846222 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x562bdf581860 0x562bdf583cb0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:05.948 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.948+0000 7fc299fca740 1 --2- 172.21.15.83:0/2085846222 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bdf5715a0 0x562bdf56f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:05.948 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.948+0000 7fc299fca740 1 --2- 172.21.15.83:0/2085846222 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562bdf566ce0 0x562bdf56eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:05.948 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.948+0000 7fc299fca740 1 --2- 172.21.15.83:0/2085846222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bdf5663c0 0x562bdf56e610 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:05.949 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.948+0000 7fc299fca740 1 -- 172.21.15.83:0/2085846222 >> 172.21.15.83:0/2085846222 conn(0x562bdf5628d0 msgr2=0x562bdf568650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:05.949 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.948+0000 7fc27e000640 0 client.18378 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:59:05.949 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.948+0000 7fc299fca740 1 -- 172.21.15.83:0/2085846222 shutdown_connections 2024-04-24T08:59:05.949 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.948+0000 7fc299fca740 1 -- 172.21.15.83:0/2085846222 wait complete. 2024-04-24T08:59:05.949 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.948+0000 7fc299fca740 20 client.18378 trim_cache size 0 max 16384 2024-04-24T08:59:05.968 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.967+0000 7fc299fca740 1 Processor -- start 2024-04-24T08:59:05.968 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.968+0000 7fc299fca740 1 -- start start 2024-04-24T08:59:05.968 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.968+0000 7fc299fca740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bdf5663c0 0x562bdf562210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:05.968 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.968+0000 7fc299fca740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bdf566ce0 0x562bdf56ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:05.968 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.968+0000 7fc299fca740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562bdf5715a0 0x562bdf5895c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:05.968 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.968+0000 7fc299fca740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc2901b5ab0 con 0x562bdf566ce0 2024-04-24T08:59:05.969 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.968+0000 7fc299fca740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc2901b75f0 con 0x562bdf5715a0 2024-04-24T08:59:05.969 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.968+0000 7fc299fca740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc2901b6180 con 0x562bdf5663c0 2024-04-24T08:59:05.969 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.968+0000 7fc28f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bdf566ce0 0x562bdf56ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:05.969 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.968+0000 7fc28f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bdf566ce0 0x562bdf56ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41588/0 (socket says 172.21.15.83:41588) 2024-04-24T08:59:05.969 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.968+0000 7fc28f400640 1 -- 172.21.15.83:0/2913677549 learned_addr learned my addr 172.21.15.83:0/2913677549 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:59:05.970 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.969+0000 7fc28fe00640 1 --2- 172.21.15.83:0/2913677549 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bdf5663c0 0x562bdf562210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:05.971 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.969+0000 7fc28f400640 1 -- 172.21.15.83:0/2913677549 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bdf5663c0 msgr2=0x562bdf562210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:05.971 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.969+0000 7fc28f400640 1 --2- 172.21.15.83:0/2913677549 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bdf5663c0 0x562bdf562210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:05.971 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.969+0000 7fc28f400640 1 -- 172.21.15.83:0/2913677549 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562bdf5715a0 msgr2=0x562bdf5895c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:05.971 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.969+0000 7fc295000640 1 --2- 172.21.15.83:0/2913677549 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562bdf5715a0 0x562bdf5895c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:05.971 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.969+0000 7fc28f400640 1 --2- 172.21.15.83:0/2913677549 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562bdf5715a0 0x562bdf5895c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:05.971 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.969+0000 7fc28f400640 1 -- 172.21.15.83:0/2913677549 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562bdf56d600 con 0x562bdf566ce0 2024-04-24T08:59:05.971 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.969+0000 7fc28f400640 1 --2- 172.21.15.83:0/2913677549 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bdf566ce0 0x562bdf56ece0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7fc2780bcac0 tx=0x7fc27802a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b98b5e8df3c9760 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:05.971 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.969+0000 7fc284c00640 1 -- 172.21.15.83:0/2913677549 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc278049040 con 0x562bdf566ce0 2024-04-24T08:59:05.971 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.969+0000 7fc284c00640 1 -- 172.21.15.83:0/2913677549 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc2780bd4c0 con 0x562bdf566ce0 2024-04-24T08:59:05.971 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.969+0000 7fc284c00640 1 -- 172.21.15.83:0/2913677549 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2780c5a30 con 0x562bdf566ce0 2024-04-24T08:59:05.971 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.970+0000 7fc299fca740 1 -- 172.21.15.83:0/2913677549 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bdf566ce0 msgr2=0x562bdf56ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:05.971 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.970+0000 7fc299fca740 1 --2- 172.21.15.83:0/2913677549 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bdf566ce0 0x562bdf56ece0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7fc2780bcac0 tx=0x7fc27802a420 comp rx=0 tx=0).stop 2024-04-24T08:59:05.971 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.970+0000 7fc299fca740 1 -- 172.21.15.83:0/2913677549 shutdown_connections 2024-04-24T08:59:05.971 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.970+0000 7fc299fca740 1 --2- 172.21.15.83:0/2913677549 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562bdf5715a0 0x562bdf5895c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:05.972 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.970+0000 7fc299fca740 1 --2- 172.21.15.83:0/2913677549 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bdf566ce0 0x562bdf56ece0 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:05.972 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.970+0000 7fc299fca740 1 --2- 172.21.15.83:0/2913677549 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bdf5663c0 0x562bdf562210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:05.972 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.970+0000 7fc299fca740 1 -- 172.21.15.83:0/2913677549 >> 172.21.15.83:0/2913677549 conn(0x562bdf5628d0 msgr2=0x562bdf56e5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:05.972 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.970+0000 7fc299fca740 1 -- 172.21.15.83:0/2913677549 shutdown_connections 2024-04-24T08:59:05.972 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.970+0000 7fc299fca740 1 -- 172.21.15.83:0/2913677549 wait complete. 2024-04-24T08:59:05.972 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.971+0000 7fc299fca740 1 -- start start 2024-04-24T08:59:05.974 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.971+0000 7fc299fca740 1 Processor -- start 2024-04-24T08:59:05.974 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.971+0000 7fc299fca740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bdf5663c0 0x562bdf56b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:05.975 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.971+0000 7fc299fca740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562bdf566ce0 0x562bdf56b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:05.975 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.971+0000 7fc299fca740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bdf5715a0 0x562bdf56bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:05.975 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.971+0000 7fc299fca740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc2901b6f20 con 0x562bdf5663c0 2024-04-24T08:59:05.975 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.971+0000 7fc299fca740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc2901b6850 con 0x562bdf566ce0 2024-04-24T08:59:05.975 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.972+0000 7fc299fca740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc2901b3ab0 con 0x562bdf5715a0 2024-04-24T08:59:05.975 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.972+0000 7fc299fca740 10 client.0 _ll_register_callbacks cb 0x7fc28800c330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:59:05.975 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.972+0000 7fc28fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bdf5663c0 0x562bdf56b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:05.975 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.972+0000 7fc28fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bdf5663c0 0x562bdf56b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41598/0 (socket says 172.21.15.83:41598) 2024-04-24T08:59:05.975 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.972+0000 7fc28fe00640 1 -- 172.21.15.83:0/1970882641 learned_addr learned my addr 172.21.15.83:0/1970882641 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:59:05.975 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.972+0000 7fc28f400640 1 --2- 172.21.15.83:0/1970882641 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562bdf566ce0 0x562bdf56b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:05.975 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.972+0000 7fc295000640 1 --2- 172.21.15.83:0/1970882641 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bdf5715a0 0x562bdf56bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:05.975 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.972+0000 7fc28fe00640 1 -- 172.21.15.83:0/1970882641 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bdf5715a0 msgr2=0x562bdf56bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:05.975 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.972+0000 7fc28fe00640 1 --2- 172.21.15.83:0/1970882641 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bdf5715a0 0x562bdf56bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:05.976 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.972+0000 7fc28fe00640 1 -- 172.21.15.83:0/1970882641 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562bdf566ce0 msgr2=0x562bdf56b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:05.976 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.972+0000 7fc28fe00640 1 --2- 172.21.15.83:0/1970882641 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562bdf566ce0 0x562bdf56b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:05.976 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.972+0000 7fc28fe00640 1 -- 172.21.15.83:0/1970882641 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562bdf2ab620 con 0x562bdf5663c0 2024-04-24T08:59:05.976 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.972+0000 7fc28f400640 1 --2- 172.21.15.83:0/1970882641 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562bdf566ce0 0x562bdf56b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:59:05.976 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.972+0000 7fc295000640 1 --2- 172.21.15.83:0/1970882641 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bdf5715a0 0x562bdf56bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:59:05.976 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.973+0000 7fc28fe00640 1 --2- 172.21.15.83:0/1970882641 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bdf5663c0 0x562bdf56b200 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7fc28801e810 tx=0x7fc28801e6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b4a0988c64940d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:05.976 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.973+0000 7fc27e000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:59:05.976 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.973+0000 7fc27e000640 1 -- 172.21.15.83:0/1970882641 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc28800ed90 con 0x562bdf5663c0 2024-04-24T08:59:05.976 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.973+0000 7fc299fca740 1 -- 172.21.15.83:0/1970882641 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2780bd4c0 con 0x562bdf5663c0 2024-04-24T08:59:05.976 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.973+0000 7fc27e000640 1 -- 172.21.15.83:0/1970882641 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9671+0+0 (secure 0 0 0) 0x7fc288004020 con 0x562bdf5663c0 2024-04-24T08:59:05.976 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.973+0000 7fc27e000640 1 -- 172.21.15.83:0/1970882641 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fc28801ec90 con 0x562bdf5663c0 2024-04-24T08:59:05.976 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.973+0000 7fc299fca740 10 client.18384 fetch_fsmap learned FSMap version 33 2024-04-24T08:59:05.976 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.973+0000 7fc299fca740 1 -- 172.21.15.83:0/1970882641 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x562bdf5e7ee0 con 0x562bdf5663c0 2024-04-24T08:59:05.976 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.974+0000 7fc27e000640 1 -- 172.21.15.83:0/1970882641 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc2780bd4c0 con 0x562bdf5663c0 2024-04-24T08:59:05.977 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.974+0000 7fc27e000640 1 -- 172.21.15.83:0/1970882641 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 33) ==== 8362+0+0 (secure 0 0 0) 0x7fc288004730 con 0x562bdf5663c0 2024-04-24T08:59:05.977 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.974+0000 7fc299fca740 10 client.18384 fetch_fsmap finished waiting for FSMap version 33 2024-04-24T08:59:05.977 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.974+0000 7fc299fca740 10 client.18384 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T08:59:05.977 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.974+0000 7fc299fca740 20 client.18384 populate_metadata read hostname 'smithi083' 2024-04-24T08:59:05.977 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.974+0000 7fc299fca740 1 --2- 172.21.15.83:0/1970882641 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x562bdf5e86c0 0x562bdf5eaae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:05.977 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.974+0000 7fc299fca740 4 client.18384 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T08:59:05.977 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.974+0000 7fc299fca740 1 -- 172.21.15.83:0/1970882641 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562bdf4d4190 con 0x562bdf5e86c0 2024-04-24T08:59:05.977 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.974+0000 7fc28f400640 1 --2- 172.21.15.83:0/1970882641 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x562bdf5e86c0 0x562bdf5eaae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:05.977 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.975+0000 7fc28f400640 1 --2- 172.21.15.83:0/1970882641 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x562bdf5e86c0 0x562bdf5eaae0 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5768d248e7e9845c server_cookie=bd9c1867af9f15bf in_seq=0 out_seq=0 2024-04-24T08:59:05.977 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:05.975+0000 7fc27e000640 10 client.18384 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:59:06.026 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:06.026+0000 7fc27e000640 1 -- 172.21.15.83:0/1970882641 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562bdf4d4190 con 0x562bdf5e86c0 2024-04-24T08:59:06.027 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:06.026+0000 7fc27e000640 10 client.18384 handle_command_reply: tid=0 2024-04-24T08:59:06.027 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:06.026+0000 7fc299fca740 1 client.18384 shutdown 2024-04-24T08:59:06.027 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:06.027+0000 7fc299fca740 1 -- 172.21.15.83:0/1970882641 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bdf5663c0 msgr2=0x562bdf56b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:06.027 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:06.027+0000 7fc299fca740 1 --2- 172.21.15.83:0/1970882641 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bdf5663c0 0x562bdf56b200 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7fc28801e810 tx=0x7fc28801e6b0 comp rx=0 tx=0).stop 2024-04-24T08:59:06.027 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:06.027+0000 7fc299fca740 1 -- 172.21.15.83:0/1970882641 shutdown_connections 2024-04-24T08:59:06.028 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:06.027+0000 7fc299fca740 1 --2- 172.21.15.83:0/1970882641 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x562bdf5e86c0 0x562bdf5eaae0 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:06.028 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:06.027+0000 7fc299fca740 1 --2- 172.21.15.83:0/1970882641 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bdf5715a0 0x562bdf56bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:06.028 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:06.027+0000 7fc299fca740 1 --2- 172.21.15.83:0/1970882641 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562bdf566ce0 0x562bdf56b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:06.028 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:06.027+0000 7fc299fca740 1 --2- 172.21.15.83:0/1970882641 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bdf5663c0 0x562bdf56b200 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:06.028 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:06.027+0000 7fc299fca740 1 -- 172.21.15.83:0/1970882641 >> 172.21.15.83:0/1970882641 conn(0x562bdf5628d0 msgr2=0x562bdf56ee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:06.028 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:06.027+0000 7fc27e000640 0 client.18384 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:59:06.028 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:06.027+0000 7fc299fca740 1 -- 172.21.15.83:0/1970882641 shutdown_connections 2024-04-24T08:59:06.028 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:06.027+0000 7fc299fca740 1 -- 172.21.15.83:0/1970882641 wait complete. 2024-04-24T08:59:06.028 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:06.027+0000 7fc299fca740 20 client.18384 trim_cache size 0 max 16384 2024-04-24T08:59:06.028 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T08:59:06.029 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T08:59:06.029 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T08:59:06.029 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T08:59:06.029 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:06.028+0000 7fc295a00640 1 -- 172.21.15.83:0/601892629 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fc2740a3d10 msgr2=0x7fc2740a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:06.029 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:06.028+0000 7fc295a00640 1 --2- 172.21.15.83:0/601892629 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fc2740a3d10 0x7fc2740a61d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fc288005d90 tx=0x7fc288005d20 comp rx=0 tx=0).stop 2024-04-24T08:59:06.030 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:06.028+0000 7fc295a00640 1 -- 172.21.15.83:0/601892629 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc290108ec0 msgr2=0x7fc2901a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:06.030 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:06.028+0000 7fc295a00640 1 --2- 172.21.15.83:0/601892629 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc290108ec0 0x7fc2901a4d50 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7fc278005940 tx=0x7fc2780042f0 comp rx=0 tx=0).stop 2024-04-24T08:59:06.030 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:06.029+0000 7fc295a00640 1 -- 172.21.15.83:0/601892629 shutdown_connections 2024-04-24T08:59:06.030 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:06.029+0000 7fc295a00640 1 --2- 172.21.15.83:0/601892629 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fc2740a3d10 0x7fc2740a61d0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:06.030 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:06.029+0000 7fc295a00640 1 --2- 172.21.15.83:0/601892629 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc29010e4d0 0x7fc29019e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:06.030 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:06.029+0000 7fc295a00640 1 --2- 172.21.15.83:0/601892629 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc290108ec0 0x7fc2901a4d50 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:06.030 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:06.029+0000 7fc295a00640 1 --2- 172.21.15.83:0/601892629 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc290108540 0x7fc2901a4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:06.030 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:06.029+0000 7fc295a00640 1 -- 172.21.15.83:0/601892629 >> 172.21.15.83:0/601892629 conn(0x7fc290077de0 msgr2=0x7fc2900784a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:06.030 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:06.029+0000 7fc295a00640 1 -- 172.21.15.83:0/601892629 shutdown_connections 2024-04-24T08:59:06.030 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:06.029+0000 7fc295a00640 1 -- 172.21.15.83:0/601892629 wait complete. 2024-04-24T08:59:06.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:59:05 smithi138 ceph-mon[31441]: pgmap v231: 129 pgs: 129 active+clean; 46 MiB data, 668 MiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 461 KiB/s wr, 88 op/s 2024-04-24T08:59:06.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:59:05 smithi138 ceph-mon[31441]: scrub queued for path: / 2024-04-24T08:59:06.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:59:05 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T08:59:06.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:59:05 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T08:59:06.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:59:05 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T08:59:06.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:59:05 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T08:59:06.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:59:05 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T08:59:06.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:59:05 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T08:59:06.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:59:05 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T08:59:06.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:59:05 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T08:59:06.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:59:05 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T08:59:06.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:59:05 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T08:59:06.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:59:05 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T08:59:06.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:59:05 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T08:59:06.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:59:05 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T08:59:06.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:59:05 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T08:59:06.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:59:05 smithi189 ceph-mon[31791]: pgmap v231: 129 pgs: 129 active+clean; 46 MiB data, 668 MiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 461 KiB/s wr, 88 op/s 2024-04-24T08:59:06.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:59:05 smithi189 ceph-mon[31791]: scrub queued for path: / 2024-04-24T08:59:06.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:59:05 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T08:59:06.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:59:05 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T08:59:06.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:59:05 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T08:59:06.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:59:05 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T08:59:06.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:59:05 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T08:59:06.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:59:05 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T08:59:06.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:59:05 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T08:59:06.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:59:05 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T08:59:06.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:59:05 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T08:59:06.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:59:05 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T08:59:06.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:59:05 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T08:59:06.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:59:05 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T08:59:06.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:59:05 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T08:59:06.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:59:05 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T08:59:06.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:59:05 smithi083 ceph-mon[27907]: pgmap v231: 129 pgs: 129 active+clean; 46 MiB data, 668 MiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 461 KiB/s wr, 88 op/s 2024-04-24T08:59:06.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:59:05 smithi083 ceph-mon[27907]: scrub queued for path: / 2024-04-24T08:59:06.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:59:05 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T08:59:06.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:59:05 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T08:59:06.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:59:05 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T08:59:06.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:59:05 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T08:59:06.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:59:05 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T08:59:06.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:59:05 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T08:59:06.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:59:05 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T08:59:06.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:59:05 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T08:59:06.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:59:05 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T08:59:06.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:59:05 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T08:59:06.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:59:05 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T08:59:06.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:59:05 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T08:59:06.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:59:05 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T08:59:06.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:59:05 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T08:59:06.591 INFO:tasks.workunit.client.0.smithi083.stdout: 1 40674 0.63 MB/sec execute 220 sec latency 50.427 ms 2024-04-24T08:59:06.894 INFO:journalctl@ceph.mds.c.smithi189.stdout:Apr 24 08:59:06 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-c[61724]: ignoring --setuser ceph since I am not root 2024-04-24T08:59:06.895 INFO:journalctl@ceph.mds.c.smithi189.stdout:Apr 24 08:59:06 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-c[61724]: ignoring --setgroup ceph since I am not root 2024-04-24T08:59:06.895 INFO:journalctl@ceph.mds.c.smithi189.stdout:Apr 24 08:59:06 smithi189 ceph-mds[61728]: -- 172.21.15.189:0/1265757385 <== mon.1 v2:172.21.15.189:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5607e07c5a40 con 0x5607e0799000 2024-04-24T08:59:06.895 INFO:journalctl@ceph.mds.c.smithi189.stdout:Apr 24 08:59:06 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-c[61724]: starting mds.c at 2024-04-24T08:59:07.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:59:06 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T08:59:07.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:59:06 smithi138 ceph-mon[31441]: daemon mds.c restarted 2024-04-24T08:59:07.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:59:06 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T08:59:07.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:59:06 smithi189 ceph-mon[31791]: daemon mds.c restarted 2024-04-24T08:59:07.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:59:06 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T08:59:07.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:59:06 smithi083 ceph-mon[27907]: daemon mds.c restarted 2024-04-24T08:59:07.591 INFO:tasks.workunit.client.0.smithi083.stdout: 1 40783 0.63 MB/sec execute 221 sec latency 60.052 ms 2024-04-24T08:59:08.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:59:07 smithi138 ceph-mon[31441]: pgmap v232: 129 pgs: 129 active+clean; 46 MiB data, 668 MiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 461 KiB/s wr, 88 op/s 2024-04-24T08:59:08.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:59:07 smithi138 ceph-mon[31441]: osdmap e99: 12 total, 12 up, 12 in 2024-04-24T08:59:08.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:59:07 smithi138 ceph-mon[31441]: mds.? [v2:172.21.15.189:6836/1678817161,v1:172.21.15.189:6837/1678817161] up:boot 2024-04-24T08:59:08.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:59:07 smithi138 ceph-mon[31441]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-04-24T08:59:08.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:59:07 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-24T08:59:08.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:59:07 smithi138 ceph-mon[31441]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T08:59:08.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:59:07 smithi189 ceph-mon[31791]: pgmap v232: 129 pgs: 129 active+clean; 46 MiB data, 668 MiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 461 KiB/s wr, 88 op/s 2024-04-24T08:59:08.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:59:07 smithi189 ceph-mon[31791]: osdmap e99: 12 total, 12 up, 12 in 2024-04-24T08:59:08.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:59:07 smithi189 ceph-mon[31791]: mds.? [v2:172.21.15.189:6836/1678817161,v1:172.21.15.189:6837/1678817161] up:boot 2024-04-24T08:59:08.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:59:07 smithi189 ceph-mon[31791]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-04-24T08:59:08.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:59:07 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-24T08:59:08.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:59:07 smithi189 ceph-mon[31791]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T08:59:08.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:59:07 smithi083 ceph-mon[27907]: pgmap v232: 129 pgs: 129 active+clean; 46 MiB data, 668 MiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 461 KiB/s wr, 88 op/s 2024-04-24T08:59:08.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:59:07 smithi083 ceph-mon[27907]: osdmap e99: 12 total, 12 up, 12 in 2024-04-24T08:59:08.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:59:07 smithi083 ceph-mon[27907]: mds.? [v2:172.21.15.189:6836/1678817161,v1:172.21.15.189:6837/1678817161] up:boot 2024-04-24T08:59:08.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:59:07 smithi083 ceph-mon[27907]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-04-24T08:59:08.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:59:07 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-24T08:59:08.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:59:07 smithi083 ceph-mon[27907]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T08:59:08.591 INFO:tasks.workunit.client.0.smithi083.stdout: 1 40921 0.63 MB/sec execute 222 sec latency 66.495 ms 2024-04-24T08:59:09.591 INFO:tasks.workunit.client.0.smithi083.stdout: 1 41045 0.62 MB/sec execute 223 sec latency 77.306 ms 2024-04-24T08:59:10.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:59:09 smithi138 ceph-mon[31441]: pgmap v234: 129 pgs: 129 active+clean; 43 MiB data, 665 MiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 547 KiB/s wr, 122 op/s 2024-04-24T08:59:10.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:59:09 smithi138 ceph-mon[31441]: osdmap e100: 12 total, 12 up, 12 in 2024-04-24T08:59:10.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:59:09 smithi189 ceph-mon[31791]: pgmap v234: 129 pgs: 129 active+clean; 43 MiB data, 665 MiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 547 KiB/s wr, 122 op/s 2024-04-24T08:59:10.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:59:09 smithi189 ceph-mon[31791]: osdmap e100: 12 total, 12 up, 12 in 2024-04-24T08:59:10.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:59:09 smithi083 ceph-mon[27907]: pgmap v234: 129 pgs: 129 active+clean; 43 MiB data, 665 MiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 547 KiB/s wr, 122 op/s 2024-04-24T08:59:10.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:59:09 smithi083 ceph-mon[27907]: osdmap e100: 12 total, 12 up, 12 in 2024-04-24T08:59:10.592 INFO:tasks.workunit.client.0.smithi083.stdout: 1 41157 0.62 MB/sec execute 224 sec latency 55.803 ms 2024-04-24T08:59:11.592 INFO:tasks.workunit.client.0.smithi083.stdout: 1 41318 0.62 MB/sec execute 225 sec latency 60.620 ms 2024-04-24T08:59:12.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:59:11 smithi138 ceph-mon[31441]: pgmap v236: 129 pgs: 129 active+clean; 43 MiB data, 665 MiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 291 KiB/s wr, 55 op/s 2024-04-24T08:59:12.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:59:11 smithi189 ceph-mon[31791]: pgmap v236: 129 pgs: 129 active+clean; 43 MiB data, 665 MiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 291 KiB/s wr, 55 op/s 2024-04-24T08:59:12.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:59:11 smithi083 ceph-mon[27907]: pgmap v236: 129 pgs: 129 active+clean; 43 MiB data, 665 MiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 291 KiB/s wr, 55 op/s 2024-04-24T08:59:12.592 INFO:tasks.workunit.client.0.smithi083.stdout: 1 41423 0.62 MB/sec execute 226 sec latency 50.272 ms 2024-04-24T08:59:13.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:59:12 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T08:59:13.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:59:12 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T08:59:13.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:59:12 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T08:59:13.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:59:12 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T08:59:13.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:59:12 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T08:59:13.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:59:12 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T08:59:13.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:59:12 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T08:59:13.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:59:12 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T08:59:13.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:59:12 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T08:59:13.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:59:12 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T08:59:13.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:59:12 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T08:59:13.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:59:12 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T08:59:13.592 INFO:tasks.workunit.client.0.smithi083.stdout: 1 41500 0.62 MB/sec execute 227 sec latency 54.335 ms 2024-04-24T08:59:14.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:59:13 smithi138 ceph-mon[31441]: pgmap v237: 129 pgs: 129 active+clean; 44 MiB data, 658 MiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 486 KiB/s wr, 78 op/s 2024-04-24T08:59:14.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:59:13 smithi189 ceph-mon[31791]: pgmap v237: 129 pgs: 129 active+clean; 44 MiB data, 658 MiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 486 KiB/s wr, 78 op/s 2024-04-24T08:59:14.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:59:13 smithi083 ceph-mon[27907]: pgmap v237: 129 pgs: 129 active+clean; 44 MiB data, 658 MiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 486 KiB/s wr, 78 op/s 2024-04-24T08:59:14.592 INFO:tasks.workunit.client.0.smithi083.stdout: 1 41623 0.62 MB/sec execute 228 sec latency 55.369 ms 2024-04-24T08:59:15.592 INFO:tasks.workunit.client.0.smithi083.stdout: 1 41809 0.63 MB/sec execute 229 sec latency 48.957 ms 2024-04-24T08:59:16.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:59:15 smithi138 ceph-mon[31441]: pgmap v238: 129 pgs: 129 active+clean; 44 MiB data, 658 MiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 486 KiB/s wr, 78 op/s 2024-04-24T08:59:16.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:59:15 smithi189 ceph-mon[31791]: pgmap v238: 129 pgs: 129 active+clean; 44 MiB data, 658 MiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 486 KiB/s wr, 78 op/s 2024-04-24T08:59:16.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:59:15 smithi083 ceph-mon[27907]: pgmap v238: 129 pgs: 129 active+clean; 44 MiB data, 658 MiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 486 KiB/s wr, 78 op/s 2024-04-24T08:59:16.592 INFO:tasks.workunit.client.0.smithi083.stdout: 1 41992 0.64 MB/sec execute 230 sec latency 49.608 ms 2024-04-24T08:59:17.592 INFO:tasks.workunit.client.0.smithi083.stdout: 1 42223 0.64 MB/sec execute 231 sec latency 57.762 ms 2024-04-24T08:59:18.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:59:17 smithi138 ceph-mon[31441]: pgmap v239: 129 pgs: 129 active+clean; 44 MiB data, 658 MiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 400 KiB/s wr, 65 op/s 2024-04-24T08:59:18.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:59:17 smithi189 ceph-mon[31791]: pgmap v239: 129 pgs: 129 active+clean; 44 MiB data, 658 MiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 400 KiB/s wr, 65 op/s 2024-04-24T08:59:18.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:59:17 smithi083 ceph-mon[27907]: pgmap v239: 129 pgs: 129 active+clean; 44 MiB data, 658 MiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 400 KiB/s wr, 65 op/s 2024-04-24T08:59:18.592 INFO:tasks.workunit.client.0.smithi083.stdout: 1 42340 0.64 MB/sec execute 232 sec latency 52.006 ms 2024-04-24T08:59:19.175 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.175+0000 7f3107800640 1 -- 172.21.15.83:0/6313422 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3100107190 msgr2=0x7f310010ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:19.175 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.175+0000 7f3107800640 1 --2- 172.21.15.83:0/6313422 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3100107190 0x7f310010ed60 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f30f8009900 tx=0x7f30f802a3f0 comp rx=0 tx=0).stop 2024-04-24T08:59:19.176 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.175+0000 7f3107800640 1 -- 172.21.15.83:0/6313422 shutdown_connections 2024-04-24T08:59:19.176 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.175+0000 7f3107800640 1 --2- 172.21.15.83:0/6313422 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3100107190 0x7f310010ed60 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:19.176 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.175+0000 7f3107800640 1 --2- 172.21.15.83:0/6313422 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3100102a70 0x7f3100106ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:19.176 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.175+0000 7f3107800640 1 --2- 172.21.15.83:0/6313422 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f31001020c0 0x7f31001024a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:19.176 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.175+0000 7f3107800640 1 -- 172.21.15.83:0/6313422 >> 172.21.15.83:0/6313422 conn(0x7f31000fd940 msgr2=0x7f31000ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:19.176 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.175+0000 7f3107800640 1 -- 172.21.15.83:0/6313422 shutdown_connections 2024-04-24T08:59:19.177 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.175+0000 7f3107800640 1 -- 172.21.15.83:0/6313422 wait complete. 2024-04-24T08:59:19.180 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.176+0000 7f3107800640 1 Processor -- start 2024-04-24T08:59:19.180 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.176+0000 7f3107800640 1 -- start start 2024-04-24T08:59:19.180 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.177+0000 7f3107800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f31001020c0 0x7f31001985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:19.180 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.177+0000 7f3107800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3100107190 0x7f3100198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:19.180 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.177+0000 7f3107800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f310019ce70 0x7f310019d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:19.180 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.177+0000 7f3107800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3100112150 con 0x7f31001020c0 2024-04-24T08:59:19.180 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.177+0000 7f3107800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3100111fd0 con 0x7f310019ce70 2024-04-24T08:59:19.181 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.177+0000 7f3107800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f31001122d0 con 0x7f3100107190 2024-04-24T08:59:19.181 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.177+0000 7f3104e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f31001020c0 0x7f31001985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:19.181 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.177+0000 7f3104e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f31001020c0 0x7f31001985b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48906/0 (socket says 172.21.15.83:48906) 2024-04-24T08:59:19.181 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.177+0000 7f3104e00640 1 -- 172.21.15.83:0/1521298882 learned_addr learned my addr 172.21.15.83:0/1521298882 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:59:19.181 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.177+0000 7f30f7e00640 1 --2- 172.21.15.83:0/1521298882 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3100107190 0x7f3100198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:19.181 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.177+0000 7f3105800640 1 --2- 172.21.15.83:0/1521298882 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f310019ce70 0x7f310019d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:19.181 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.177+0000 7f3104e00640 1 -- 172.21.15.83:0/1521298882 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3100107190 msgr2=0x7f3100198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:19.181 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.177+0000 7f3104e00640 1 --2- 172.21.15.83:0/1521298882 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3100107190 0x7f3100198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:19.181 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.177+0000 7f3104e00640 1 -- 172.21.15.83:0/1521298882 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f310019ce70 msgr2=0x7f310019d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:19.181 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.177+0000 7f3104e00640 1 --2- 172.21.15.83:0/1521298882 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f310019ce70 0x7f310019d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:19.181 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.177+0000 7f3104e00640 1 -- 172.21.15.83:0/1521298882 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f310019d960 con 0x7f31001020c0 2024-04-24T08:59:19.181 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.178+0000 7f3105800640 1 --2- 172.21.15.83:0/1521298882 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f310019ce70 0x7f310019d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:59:19.181 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.178+0000 7f3104e00640 1 --2- 172.21.15.83:0/1521298882 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f31001020c0 0x7f31001985b0 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f30f000ea80 tx=0x7f30f000ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:19.181 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.178+0000 7f30fe000640 1 -- 172.21.15.83:0/1521298882 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f000ce40 con 0x7f31001020c0 2024-04-24T08:59:19.182 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.178+0000 7f30fe000640 1 -- 172.21.15.83:0/1521298882 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f30f0005020 con 0x7f31001020c0 2024-04-24T08:59:19.182 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.178+0000 7f3107800640 1 -- 172.21.15.83:0/1521298882 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f310019dc50 con 0x7f31001020c0 2024-04-24T08:59:19.182 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.178+0000 7f30fe000640 1 -- 172.21.15.83:0/1521298882 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f00116d0 con 0x7f31001020c0 2024-04-24T08:59:19.182 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.178+0000 7f3107800640 1 -- 172.21.15.83:0/1521298882 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31001a5440 con 0x7f31001020c0 2024-04-24T08:59:19.184 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.182+0000 7f310a5db740 1 Processor -- start 2024-04-24T08:59:19.184 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.182+0000 7f30fe000640 1 -- 172.21.15.83:0/1521298882 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f30f00051d0 con 0x7f31001020c0 2024-04-24T08:59:19.184 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.182+0000 7f310a5db740 1 -- start start 2024-04-24T08:59:19.184 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.182+0000 7f310a5db740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5634e55ae2e0 0x5634e55ae6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:19.184 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.182+0000 7f310a5db740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5634e55aec00 0x5634e55bd470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:19.184 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.182+0000 7f310a5db740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634e55bd9b0 0x5634e55bfda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:19.185 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.182+0000 7f310a5db740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5634e5514900 con 0x5634e55bd9b0 2024-04-24T08:59:19.185 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.182+0000 7f310a5db740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5634e54baf20 con 0x5634e55aec00 2024-04-24T08:59:19.185 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.182+0000 7f310a5db740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5634e55108c0 con 0x5634e55ae2e0 2024-04-24T08:59:19.185 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.183+0000 7f30f7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634e55bd9b0 0x5634e55bfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:19.185 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.183+0000 7f30f7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634e55bd9b0 0x5634e55bfda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48922/0 (socket says 172.21.15.83:48922) 2024-04-24T08:59:19.185 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.183+0000 7f30f7e00640 1 -- 172.21.15.83:0/3549580276 learned_addr learned my addr 172.21.15.83:0/3549580276 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:59:19.185 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.183+0000 7f3104e00640 1 --2- 172.21.15.83:0/3549580276 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5634e55aec00 0x5634e55bd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:19.185 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.183+0000 7f3105800640 1 --2- 172.21.15.83:0/3549580276 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5634e55ae2e0 0x5634e55ae6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:19.185 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.183+0000 7f30f7e00640 1 -- 172.21.15.83:0/3549580276 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5634e55ae2e0 msgr2=0x5634e55ae6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:19.185 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.183+0000 7f30f7e00640 1 --2- 172.21.15.83:0/3549580276 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5634e55ae2e0 0x5634e55ae6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:19.185 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.183+0000 7f30f7e00640 1 -- 172.21.15.83:0/3549580276 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5634e55aec00 msgr2=0x5634e55bd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:19.185 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.183+0000 7f30f7e00640 1 --2- 172.21.15.83:0/3549580276 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5634e55aec00 0x5634e55bd470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:19.185 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.183+0000 7f30f7e00640 1 -- 172.21.15.83:0/3549580276 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634e54dfe10 con 0x5634e55bd9b0 2024-04-24T08:59:19.185 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.183+0000 7f3105800640 1 --2- 172.21.15.83:0/3549580276 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5634e55ae2e0 0x5634e55ae6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:59:19.186 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.183+0000 7f3104e00640 1 --2- 172.21.15.83:0/3549580276 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5634e55aec00 0x5634e55bd470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:59:19.186 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.184+0000 7f30f7e00640 1 --2- 172.21.15.83:0/3549580276 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634e55bd9b0 0x5634e55bfda0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f30e800cb80 tx=0x7f30e800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7df4464c6897410e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:19.186 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.184+0000 7f30fe000640 1 --2- 172.21.15.83:0/1521298882 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f30e40a3d90 0x7f30e40a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:19.186 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.184+0000 7f30fe000640 1 -- 172.21.15.83:0/1521298882 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7f30f00d0f30 con 0x7f31001020c0 2024-04-24T08:59:19.186 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.184+0000 7f30f6000640 1 -- 172.21.15.83:0/3549580276 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e8005220 con 0x5634e55bd9b0 2024-04-24T08:59:19.186 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.184+0000 7f3105800640 1 --2- 172.21.15.83:0/1521298882 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f30e40a3d90 0x7f30e40a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:19.186 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.184+0000 7f30f6000640 1 -- 172.21.15.83:0/3549580276 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f30e8004c50 con 0x5634e55bd9b0 2024-04-24T08:59:19.186 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.184+0000 7f30f6000640 1 -- 172.21.15.83:0/3549580276 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e8010720 con 0x5634e55bd9b0 2024-04-24T08:59:19.190 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.185+0000 7f310a5db740 1 -- 172.21.15.83:0/3549580276 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634e55bd9b0 msgr2=0x5634e55bfda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:19.190 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.185+0000 7f3105800640 1 --2- 172.21.15.83:0/1521298882 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f30e40a3d90 0x7f30e40a6250 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f30f8031440 tx=0x7f30f8005b70 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:19.190 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.185+0000 7f310a5db740 1 --2- 172.21.15.83:0/3549580276 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634e55bd9b0 0x5634e55bfda0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f30e800cb80 tx=0x7f30e800e720 comp rx=0 tx=0).stop 2024-04-24T08:59:19.190 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.185+0000 7f310a5db740 1 -- 172.21.15.83:0/3549580276 shutdown_connections 2024-04-24T08:59:19.190 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.185+0000 7f310a5db740 1 --2- 172.21.15.83:0/3549580276 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634e55bd9b0 0x5634e55bfda0 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:19.190 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.185+0000 7f310a5db740 1 --2- 172.21.15.83:0/3549580276 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5634e55aec00 0x5634e55bd470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:19.190 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.185+0000 7f310a5db740 1 --2- 172.21.15.83:0/3549580276 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5634e55ae2e0 0x5634e55ae6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:19.190 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.185+0000 7f310a5db740 1 -- 172.21.15.83:0/3549580276 >> 172.21.15.83:0/3549580276 conn(0x5634e55aa8d0 msgr2=0x5634e55aacb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:19.191 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.185+0000 7f310a5db740 1 -- 172.21.15.83:0/3549580276 shutdown_connections 2024-04-24T08:59:19.191 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.185+0000 7f310a5db740 1 -- 172.21.15.83:0/3549580276 wait complete. 2024-04-24T08:59:19.191 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.186+0000 7f310a5db740 1 -- start start 2024-04-24T08:59:19.191 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.186+0000 7f310a5db740 1 Processor -- start 2024-04-24T08:59:19.191 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.187+0000 7f310a5db740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634e55ae2e0 0x5634e55ba900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:19.191 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.187+0000 7f310a5db740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5634e55aec00 0x5634e55bae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:19.191 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.187+0000 7f310a5db740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5634e55bd9b0 0x5634e55bb380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:19.191 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.187+0000 7f310a5db740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5634e55c05e0 con 0x5634e55ae2e0 2024-04-24T08:59:19.191 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.187+0000 7f310a5db740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5634e54ce2c0 con 0x5634e55aec00 2024-04-24T08:59:19.192 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.187+0000 7f310a5db740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5634e55c0760 con 0x5634e55bd9b0 2024-04-24T08:59:19.192 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.187+0000 7f310a5db740 10 client.0 _ll_register_callbacks cb 0x5634e5226a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:59:19.192 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.187+0000 7f3104e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634e55ae2e0 0x5634e55ba900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:19.192 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.187+0000 7f3104e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634e55ae2e0 0x5634e55ba900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48936/0 (socket says 172.21.15.83:48936) 2024-04-24T08:59:19.192 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.187+0000 7f3104e00640 1 -- 172.21.15.83:0/2226620797 learned_addr learned my addr 172.21.15.83:0/2226620797 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:59:19.192 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.187+0000 7f30f7e00640 1 --2- 172.21.15.83:0/2226620797 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5634e55aec00 0x5634e55bae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:19.192 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.187+0000 7f3105800640 1 --2- 172.21.15.83:0/2226620797 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5634e55bd9b0 0x5634e55bb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:19.192 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.187+0000 7f3104e00640 1 -- 172.21.15.83:0/2226620797 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5634e55bd9b0 msgr2=0x5634e55bb380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:19.192 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.187+0000 7f3104e00640 1 --2- 172.21.15.83:0/2226620797 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5634e55bd9b0 0x5634e55bb380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:19.192 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.187+0000 7f3104e00640 1 -- 172.21.15.83:0/2226620797 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5634e55aec00 msgr2=0x5634e55bae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:19.192 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.187+0000 7f3104e00640 1 --2- 172.21.15.83:0/2226620797 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5634e55aec00 0x5634e55bae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:19.192 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.187+0000 7f3104e00640 1 -- 172.21.15.83:0/2226620797 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634e54defd0 con 0x5634e55ae2e0 2024-04-24T08:59:19.192 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.187+0000 7f3105800640 1 --2- 172.21.15.83:0/2226620797 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5634e55bd9b0 0x5634e55bb380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:59:19.192 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.188+0000 7f3104e00640 1 --2- 172.21.15.83:0/2226620797 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634e55ae2e0 0x5634e55ba900 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f30f000efd0 tx=0x7f30f009c6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1ded444b10bd110 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:19.193 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.188+0000 7f30eea00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:59:19.193 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.188+0000 7f30eea00640 1 -- 172.21.15.83:0/2226620797 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f001f040 con 0x5634e55ae2e0 2024-04-24T08:59:19.193 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.188+0000 7f30eea00640 1 -- 172.21.15.83:0/2226620797 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7f30f0099120 con 0x5634e55ae2e0 2024-04-24T08:59:19.193 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.188+0000 7f310a5db740 1 -- 172.21.15.83:0/2226620797 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30e8004c50 con 0x5634e55ae2e0 2024-04-24T08:59:19.193 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.188+0000 7f30eea00640 1 -- 172.21.15.83:0/2226620797 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f30f00a0e60 con 0x5634e55ae2e0 2024-04-24T08:59:19.193 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.188+0000 7f310a5db740 10 client.18402 fetch_fsmap learned FSMap version 35 2024-04-24T08:59:19.193 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.188+0000 7f310a5db740 1 -- 172.21.15.83:0/2226620797 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5634e5227210 con 0x5634e55ae2e0 2024-04-24T08:59:19.193 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.189+0000 7f30eea00640 1 -- 172.21.15.83:0/2226620797 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f30e8004c50 con 0x5634e55ae2e0 2024-04-24T08:59:19.193 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.189+0000 7f30eea00640 1 -- 172.21.15.83:0/2226620797 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7f30f000f040 con 0x5634e55ae2e0 2024-04-24T08:59:19.193 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.189+0000 7f310a5db740 10 client.18402 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T08:59:19.193 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.189+0000 7f310a5db740 10 client.18402 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T08:59:19.193 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.189+0000 7f310a5db740 20 client.18402 populate_metadata read hostname 'smithi083' 2024-04-24T08:59:19.193 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.189+0000 7f310a5db740 1 --2- 172.21.15.83:0/2226620797 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5634e55cdaf0 0x5634e55cfee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:19.194 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.189+0000 7f310a5db740 4 client.18402 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:59:19.194 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.189+0000 7f310a5db740 1 -- 172.21.15.83:0/2226620797 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5634e5522320 con 0x5634e55cdaf0 2024-04-24T08:59:19.194 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.190+0000 7f30f7e00640 1 --2- 172.21.15.83:0/2226620797 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5634e55cdaf0 0x5634e55cfee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:19.195 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.191+0000 7f30eea00640 10 client.18402 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:59:19.195 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.190+0000 7f30f7e00640 1 --2- 172.21.15.83:0/2226620797 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5634e55cdaf0 0x5634e55cfee0 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4f19efe5237ac65 server_cookie=9e326b0f075e4ecc in_seq=0 out_seq=0 2024-04-24T08:59:19.195 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.193+0000 7f30eea00640 1 -- 172.21.15.83:0/2226620797 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5634e5522320 con 0x5634e55cdaf0 2024-04-24T08:59:19.195 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.193+0000 7f30eea00640 10 client.18402 handle_command_reply: tid=0 2024-04-24T08:59:19.195 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.193+0000 7f310a5db740 1 client.18402 shutdown 2024-04-24T08:59:19.195 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.193+0000 7f310a5db740 1 -- 172.21.15.83:0/2226620797 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634e55ae2e0 msgr2=0x5634e55ba900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:19.195 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.193+0000 7f310a5db740 1 --2- 172.21.15.83:0/2226620797 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634e55ae2e0 0x5634e55ba900 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f30f000efd0 tx=0x7f30f009c6b0 comp rx=0 tx=0).stop 2024-04-24T08:59:19.196 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.193+0000 7f310a5db740 1 -- 172.21.15.83:0/2226620797 shutdown_connections 2024-04-24T08:59:19.196 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.193+0000 7f310a5db740 1 --2- 172.21.15.83:0/2226620797 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5634e55cdaf0 0x5634e55cfee0 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:19.196 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.193+0000 7f310a5db740 1 --2- 172.21.15.83:0/2226620797 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5634e55bd9b0 0x5634e55bb380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:19.196 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.193+0000 7f310a5db740 1 --2- 172.21.15.83:0/2226620797 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5634e55aec00 0x5634e55bae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:19.196 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.193+0000 7f30eea00640 0 client.18402 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:59:19.196 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.193+0000 7f310a5db740 1 --2- 172.21.15.83:0/2226620797 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634e55ae2e0 0x5634e55ba900 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:19.197 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.193+0000 7f310a5db740 1 -- 172.21.15.83:0/2226620797 >> 172.21.15.83:0/2226620797 conn(0x5634e55aa8d0 msgr2=0x5634e55b0740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:19.197 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.193+0000 7f310a5db740 1 -- 172.21.15.83:0/2226620797 shutdown_connections 2024-04-24T08:59:19.197 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.193+0000 7f310a5db740 1 -- 172.21.15.83:0/2226620797 wait complete. 2024-04-24T08:59:19.197 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.194+0000 7f310a5db740 20 client.18402 trim_cache size 0 max 16384 2024-04-24T08:59:19.214 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.210+0000 7f310a5db740 1 Processor -- start 2024-04-24T08:59:19.214 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.210+0000 7f310a5db740 1 -- start start 2024-04-24T08:59:19.214 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.210+0000 7f310a5db740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5634e55ae2e0 0x5634e55c94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:19.214 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.210+0000 7f310a5db740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5634e55aec00 0x5634e55d57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:19.214 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.210+0000 7f310a5db740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634e55bd9b0 0x5634e55d5d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:19.214 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.210+0000 7f310a5db740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f31001ad050 con 0x5634e55bd9b0 2024-04-24T08:59:19.215 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.210+0000 7f310a5db740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f31001aeb90 con 0x5634e55ae2e0 2024-04-24T08:59:19.215 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.210+0000 7f310a5db740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f31001ad720 con 0x5634e55aec00 2024-04-24T08:59:19.215 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.211+0000 7f3105800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634e55bd9b0 0x5634e55d5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:19.215 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.211+0000 7f3105800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634e55bd9b0 0x5634e55d5d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48946/0 (socket says 172.21.15.83:48946) 2024-04-24T08:59:19.215 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.211+0000 7f3105800640 1 -- 172.21.15.83:0/3031577180 learned_addr learned my addr 172.21.15.83:0/3031577180 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:59:19.215 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.211+0000 7f3104e00640 1 --2- 172.21.15.83:0/3031577180 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5634e55ae2e0 0x5634e55c94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:19.215 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.211+0000 7f30f7e00640 1 --2- 172.21.15.83:0/3031577180 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5634e55aec00 0x5634e55d57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:19.215 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.211+0000 7f3105800640 1 -- 172.21.15.83:0/3031577180 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5634e55aec00 msgr2=0x5634e55d57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:19.216 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.211+0000 7f3105800640 1 --2- 172.21.15.83:0/3031577180 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5634e55aec00 0x5634e55d57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:19.216 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.211+0000 7f3105800640 1 -- 172.21.15.83:0/3031577180 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5634e55ae2e0 msgr2=0x5634e55c94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:19.216 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.211+0000 7f3105800640 1 --2- 172.21.15.83:0/3031577180 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5634e55ae2e0 0x5634e55c94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:19.216 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.211+0000 7f3105800640 1 -- 172.21.15.83:0/3031577180 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634e55aea30 con 0x5634e55bd9b0 2024-04-24T08:59:19.216 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.211+0000 7f3104e00640 1 --2- 172.21.15.83:0/3031577180 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5634e55ae2e0 0x5634e55c94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:59:19.216 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.211+0000 7f30f7e00640 1 --2- 172.21.15.83:0/3031577180 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5634e55aec00 0x5634e55d57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:59:19.216 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.212+0000 7f3105800640 1 --2- 172.21.15.83:0/3031577180 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634e55bd9b0 0x5634e55d5d10 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f30f8009900 tx=0x7f30f803c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5eebfc02b22dc9d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:19.216 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.212+0000 7f30f5600640 1 -- 172.21.15.83:0/3031577180 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f804c070 con 0x5634e55bd9b0 2024-04-24T08:59:19.216 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.212+0000 7f30f5600640 1 -- 172.21.15.83:0/3031577180 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f30f8004d70 con 0x5634e55bd9b0 2024-04-24T08:59:19.216 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.212+0000 7f30f5600640 1 -- 172.21.15.83:0/3031577180 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f8035050 con 0x5634e55bd9b0 2024-04-24T08:59:19.216 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.212+0000 7f310a5db740 1 -- 172.21.15.83:0/3031577180 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634e55bd9b0 msgr2=0x5634e55d5d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:19.216 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.212+0000 7f310a5db740 1 --2- 172.21.15.83:0/3031577180 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634e55bd9b0 0x5634e55d5d10 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f30f8009900 tx=0x7f30f803c000 comp rx=0 tx=0).stop 2024-04-24T08:59:19.217 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.213+0000 7f310a5db740 1 -- 172.21.15.83:0/3031577180 shutdown_connections 2024-04-24T08:59:19.217 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.213+0000 7f310a5db740 1 --2- 172.21.15.83:0/3031577180 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634e55bd9b0 0x5634e55d5d10 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:19.217 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.213+0000 7f310a5db740 1 --2- 172.21.15.83:0/3031577180 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5634e55aec00 0x5634e55d57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:19.217 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.213+0000 7f310a5db740 1 --2- 172.21.15.83:0/3031577180 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5634e55ae2e0 0x5634e55c94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:19.217 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.213+0000 7f310a5db740 1 -- 172.21.15.83:0/3031577180 >> 172.21.15.83:0/3031577180 conn(0x5634e55aa8d0 msgr2=0x5634e55b5bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:19.217 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.213+0000 7f310a5db740 1 -- 172.21.15.83:0/3031577180 shutdown_connections 2024-04-24T08:59:19.217 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.213+0000 7f310a5db740 1 -- 172.21.15.83:0/3031577180 wait complete. 2024-04-24T08:59:19.217 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.213+0000 7f310a5db740 1 -- start start 2024-04-24T08:59:19.217 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.213+0000 7f310a5db740 1 Processor -- start 2024-04-24T08:59:19.217 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.214+0000 7f310a5db740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634e55ae2e0 0x5634e55c5de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:19.217 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.214+0000 7f310a5db740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5634e55aec00 0x5634e55b72c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:19.217 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.214+0000 7f310a5db740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5634e55bd9b0 0x5634e55b7820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:19.217 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.214+0000 7f310a5db740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f31001ae4c0 con 0x5634e55ae2e0 2024-04-24T08:59:19.218 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.214+0000 7f310a5db740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f31001addf0 con 0x5634e55aec00 2024-04-24T08:59:19.218 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.214+0000 7f310a5db740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f31001ab050 con 0x5634e55bd9b0 2024-04-24T08:59:19.218 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.214+0000 7f310a5db740 10 client.0 _ll_register_callbacks cb 0x7f30f0098de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:59:19.218 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.214+0000 7f3104e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634e55ae2e0 0x5634e55c5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:19.218 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.214+0000 7f3104e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634e55ae2e0 0x5634e55c5de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48962/0 (socket says 172.21.15.83:48962) 2024-04-24T08:59:19.218 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.214+0000 7f3104e00640 1 -- 172.21.15.83:0/1210693281 learned_addr learned my addr 172.21.15.83:0/1210693281 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:59:19.220 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.215+0000 7f30f7e00640 1 --2- 172.21.15.83:0/1210693281 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5634e55aec00 0x5634e55b72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:19.221 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.215+0000 7f3105800640 1 --2- 172.21.15.83:0/1210693281 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5634e55bd9b0 0x5634e55b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:19.221 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.215+0000 7f3104e00640 1 -- 172.21.15.83:0/1210693281 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5634e55bd9b0 msgr2=0x5634e55b7820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:19.221 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.215+0000 7f3104e00640 1 --2- 172.21.15.83:0/1210693281 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5634e55bd9b0 0x5634e55b7820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:19.221 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.215+0000 7f3104e00640 1 -- 172.21.15.83:0/1210693281 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5634e55aec00 msgr2=0x5634e55b72c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:19.221 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.215+0000 7f3104e00640 1 --2- 172.21.15.83:0/1210693281 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5634e55aec00 0x5634e55b72c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:19.221 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.215+0000 7f3104e00640 1 -- 172.21.15.83:0/1210693281 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634e52f3620 con 0x5634e55ae2e0 2024-04-24T08:59:19.221 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.215+0000 7f30f7e00640 1 --2- 172.21.15.83:0/1210693281 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5634e55aec00 0x5634e55b72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:59:19.221 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.215+0000 7f3104e00640 1 --2- 172.21.15.83:0/1210693281 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634e55ae2e0 0x5634e55c5de0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f30f009ca40 tx=0x7f30f009c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49171603f0f1baba server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:19.221 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.215+0000 7f30eea00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:59:19.221 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.215+0000 7f30eea00640 1 -- 172.21.15.83:0/1210693281 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f001f070 con 0x5634e55ae2e0 2024-04-24T08:59:19.221 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.215+0000 7f30eea00640 1 -- 172.21.15.83:0/1210693281 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7f30f009bf00 con 0x5634e55ae2e0 2024-04-24T08:59:19.221 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.215+0000 7f310a5db740 1 -- 172.21.15.83:0/1210693281 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30f8004d70 con 0x5634e55ae2e0 2024-04-24T08:59:19.221 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.215+0000 7f30eea00640 1 -- 172.21.15.83:0/1210693281 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f30f009c880 con 0x5634e55ae2e0 2024-04-24T08:59:19.221 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.215+0000 7f310a5db740 10 client.18408 fetch_fsmap learned FSMap version 35 2024-04-24T08:59:19.221 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.215+0000 7f310a5db740 1 -- 172.21.15.83:0/1210693281 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5634e5633e70 con 0x5634e55ae2e0 2024-04-24T08:59:19.222 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.216+0000 7f30eea00640 1 -- 172.21.15.83:0/1210693281 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f30f8004d70 con 0x5634e55ae2e0 2024-04-24T08:59:19.222 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.216+0000 7f30eea00640 1 -- 172.21.15.83:0/1210693281 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7f30f0004c90 con 0x5634e55ae2e0 2024-04-24T08:59:19.222 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.216+0000 7f310a5db740 10 client.18408 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T08:59:19.222 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.216+0000 7f310a5db740 10 client.18408 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T08:59:19.222 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.216+0000 7f310a5db740 20 client.18408 populate_metadata read hostname 'smithi083' 2024-04-24T08:59:19.222 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.216+0000 7f310a5db740 1 --2- 172.21.15.83:0/1210693281 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5634e56343b0 0x5634e56367a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:19.222 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.216+0000 7f310a5db740 4 client.18408 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T08:59:19.222 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.216+0000 7f310a5db740 1 -- 172.21.15.83:0/1210693281 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5634e551c190 con 0x5634e56343b0 2024-04-24T08:59:19.223 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.217+0000 7f30f7e00640 1 --2- 172.21.15.83:0/1210693281 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5634e56343b0 0x5634e56367a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:19.223 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.218+0000 7f30f7e00640 1 --2- 172.21.15.83:0/1210693281 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5634e56343b0 0x5634e56367a0 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=102461d4c89498c8 server_cookie=9acb591a4d62bb7c in_seq=0 out_seq=0 2024-04-24T08:59:19.223 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.218+0000 7f30eea00640 10 client.18408 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:59:19.276 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.276+0000 7f30eea00640 1 -- 172.21.15.83:0/1210693281 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5634e551c190 con 0x5634e56343b0 2024-04-24T08:59:19.277 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.276+0000 7f30eea00640 10 client.18408 handle_command_reply: tid=0 2024-04-24T08:59:19.277 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.276+0000 7f310a5db740 1 client.18408 shutdown 2024-04-24T08:59:19.277 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.276+0000 7f310a5db740 1 -- 172.21.15.83:0/1210693281 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634e55ae2e0 msgr2=0x5634e55c5de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:19.277 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.276+0000 7f310a5db740 1 --2- 172.21.15.83:0/1210693281 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634e55ae2e0 0x5634e55c5de0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f30f009ca40 tx=0x7f30f009c540 comp rx=0 tx=0).stop 2024-04-24T08:59:19.277 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.276+0000 7f310a5db740 1 -- 172.21.15.83:0/1210693281 shutdown_connections 2024-04-24T08:59:19.277 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.276+0000 7f310a5db740 1 --2- 172.21.15.83:0/1210693281 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5634e56343b0 0x5634e56367a0 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:19.277 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.276+0000 7f310a5db740 1 --2- 172.21.15.83:0/1210693281 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5634e55bd9b0 0x5634e55b7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:19.277 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.276+0000 7f310a5db740 1 --2- 172.21.15.83:0/1210693281 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5634e55aec00 0x5634e55b72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:19.277 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.276+0000 7f310a5db740 1 --2- 172.21.15.83:0/1210693281 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634e55ae2e0 0x5634e55c5de0 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:19.277 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.276+0000 7f30eea00640 0 client.18408 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:59:19.277 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.276+0000 7f310a5db740 1 -- 172.21.15.83:0/1210693281 >> 172.21.15.83:0/1210693281 conn(0x5634e55aa8d0 msgr2=0x5634e55ba8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:19.277 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.277+0000 7f310a5db740 1 -- 172.21.15.83:0/1210693281 shutdown_connections 2024-04-24T08:59:19.278 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.277+0000 7f310a5db740 1 -- 172.21.15.83:0/1210693281 wait complete. 2024-04-24T08:59:19.278 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.277+0000 7f310a5db740 20 client.18408 trim_cache size 0 max 16384 2024-04-24T08:59:19.278 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T08:59:19.278 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T08:59:19.279 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T08:59:19.279 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T08:59:19.279 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.278+0000 7f3107800640 1 -- 172.21.15.83:0/1521298882 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f30e40a3d90 msgr2=0x7f30e40a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:19.280 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.278+0000 7f3107800640 1 --2- 172.21.15.83:0/1521298882 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f30e40a3d90 0x7f30e40a6250 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f30f8031440 tx=0x7f30f8005b70 comp rx=0 tx=0).stop 2024-04-24T08:59:19.280 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.278+0000 7f3107800640 1 -- 172.21.15.83:0/1521298882 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f31001020c0 msgr2=0x7f31001985b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:19.280 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.278+0000 7f3107800640 1 --2- 172.21.15.83:0/1521298882 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f31001020c0 0x7f31001985b0 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f30f000ea80 tx=0x7f30f000ef40 comp rx=0 tx=0).stop 2024-04-24T08:59:19.280 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.278+0000 7f3107800640 1 -- 172.21.15.83:0/1521298882 shutdown_connections 2024-04-24T08:59:19.280 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.278+0000 7f3107800640 1 --2- 172.21.15.83:0/1521298882 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f30e40a3d90 0x7f30e40a6250 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:19.280 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.278+0000 7f3107800640 1 --2- 172.21.15.83:0/1521298882 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f310019ce70 0x7f310019d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:19.280 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.278+0000 7f3107800640 1 --2- 172.21.15.83:0/1521298882 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3100107190 0x7f3100198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:19.280 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.278+0000 7f3107800640 1 --2- 172.21.15.83:0/1521298882 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f31001020c0 0x7f31001985b0 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:19.280 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.278+0000 7f3107800640 1 -- 172.21.15.83:0/1521298882 >> 172.21.15.83:0/1521298882 conn(0x7f31000fd940 msgr2=0x7f31000fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:19.280 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.278+0000 7f3107800640 1 -- 172.21.15.83:0/1521298882 shutdown_connections 2024-04-24T08:59:19.280 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:19.278+0000 7f3107800640 1 -- 172.21.15.83:0/1521298882 wait complete. 2024-04-24T08:59:19.593 INFO:tasks.workunit.client.0.smithi083.stdout: 1 42371 0.64 MB/sec execute 233 sec latency 58.188 ms 2024-04-24T08:59:20.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:59:19 smithi138 ceph-mon[31441]: pgmap v240: 129 pgs: 129 active+clean; 47 MiB data, 665 MiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 467 KiB/s wr, 45 op/s 2024-04-24T08:59:20.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:59:19 smithi189 ceph-mon[31791]: pgmap v240: 129 pgs: 129 active+clean; 47 MiB data, 665 MiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 467 KiB/s wr, 45 op/s 2024-04-24T08:59:20.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:59:19 smithi083 ceph-mon[27907]: pgmap v240: 129 pgs: 129 active+clean; 47 MiB data, 665 MiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 467 KiB/s wr, 45 op/s 2024-04-24T08:59:20.593 INFO:tasks.workunit.client.0.smithi083.stdout: 1 42409 0.63 MB/sec execute 234 sec latency 58.089 ms 2024-04-24T08:59:21.593 INFO:tasks.workunit.client.0.smithi083.stdout: 1 42589 0.64 MB/sec execute 235 sec latency 49.453 ms 2024-04-24T08:59:22.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:59:21 smithi138 ceph-mon[31441]: pgmap v241: 129 pgs: 129 active+clean; 47 MiB data, 665 MiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 420 KiB/s wr, 41 op/s 2024-04-24T08:59:22.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:59:21 smithi189 ceph-mon[31791]: pgmap v241: 129 pgs: 129 active+clean; 47 MiB data, 665 MiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 420 KiB/s wr, 41 op/s 2024-04-24T08:59:22.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:59:21 smithi083 ceph-mon[27907]: pgmap v241: 129 pgs: 129 active+clean; 47 MiB data, 665 MiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 420 KiB/s wr, 41 op/s 2024-04-24T08:59:22.593 INFO:tasks.workunit.client.0.smithi083.stdout: 1 42712 0.64 MB/sec execute 236 sec latency 52.376 ms 2024-04-24T08:59:23.593 INFO:tasks.workunit.client.0.smithi083.stdout: 1 42822 0.64 MB/sec execute 237 sec latency 53.958 ms 2024-04-24T08:59:24.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:59:23 smithi083 ceph-mon[27907]: pgmap v242: 129 pgs: 129 active+clean; 48 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 569 KiB/s wr, 58 op/s 2024-04-24T08:59:24.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:59:23 smithi138 ceph-mon[31441]: pgmap v242: 129 pgs: 129 active+clean; 48 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 569 KiB/s wr, 58 op/s 2024-04-24T08:59:24.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:59:23 smithi189 ceph-mon[31791]: pgmap v242: 129 pgs: 129 active+clean; 48 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 569 KiB/s wr, 58 op/s 2024-04-24T08:59:24.593 INFO:tasks.workunit.client.0.smithi083.stdout: 1 42877 0.64 MB/sec execute 238 sec latency 45.344 ms 2024-04-24T08:59:25.593 INFO:tasks.workunit.client.0.smithi083.stdout: 1 42939 0.64 MB/sec execute 239 sec latency 54.974 ms 2024-04-24T08:59:26.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:59:25 smithi083 ceph-mon[27907]: pgmap v243: 129 pgs: 129 active+clean; 48 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 433 KiB/s wr, 41 op/s 2024-04-24T08:59:26.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:59:25 smithi138 ceph-mon[31441]: pgmap v243: 129 pgs: 129 active+clean; 48 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 433 KiB/s wr, 41 op/s 2024-04-24T08:59:26.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:59:25 smithi189 ceph-mon[31791]: pgmap v243: 129 pgs: 129 active+clean; 48 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 433 KiB/s wr, 41 op/s 2024-04-24T08:59:26.594 INFO:tasks.workunit.client.0.smithi083.stdout: 1 42999 0.64 MB/sec execute 240 sec latency 49.737 ms 2024-04-24T08:59:27.594 INFO:tasks.workunit.client.0.smithi083.stdout: 1 43056 0.63 MB/sec execute 241 sec latency 47.231 ms 2024-04-24T08:59:28.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:59:27 smithi083 ceph-mon[27907]: pgmap v244: 129 pgs: 129 active+clean; 48 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 433 KiB/s wr, 41 op/s 2024-04-24T08:59:28.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:59:27 smithi138 ceph-mon[31441]: pgmap v244: 129 pgs: 129 active+clean; 48 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 433 KiB/s wr, 41 op/s 2024-04-24T08:59:28.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:59:27 smithi189 ceph-mon[31791]: pgmap v244: 129 pgs: 129 active+clean; 48 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 433 KiB/s wr, 41 op/s 2024-04-24T08:59:28.594 INFO:tasks.workunit.client.0.smithi083.stdout: 1 43125 0.63 MB/sec execute 242 sec latency 53.507 ms 2024-04-24T08:59:29.594 INFO:tasks.workunit.client.0.smithi083.stdout: 1 43196 0.63 MB/sec execute 243 sec latency 55.805 ms 2024-04-24T08:59:30.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:59:29 smithi083 ceph-mon[27907]: pgmap v245: 129 pgs: 129 active+clean; 49 MiB data, 689 MiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 631 KiB/s wr, 65 op/s 2024-04-24T08:59:30.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:59:29 smithi138 ceph-mon[31441]: pgmap v245: 129 pgs: 129 active+clean; 49 MiB data, 689 MiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 631 KiB/s wr, 65 op/s 2024-04-24T08:59:30.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:59:29 smithi189 ceph-mon[31791]: pgmap v245: 129 pgs: 129 active+clean; 49 MiB data, 689 MiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 631 KiB/s wr, 65 op/s 2024-04-24T08:59:30.594 INFO:tasks.workunit.client.0.smithi083.stdout: 1 43295 0.63 MB/sec execute 244 sec latency 52.018 ms 2024-04-24T08:59:31.594 INFO:tasks.workunit.client.0.smithi083.stdout: 1 43424 0.63 MB/sec execute 245 sec latency 59.247 ms 2024-04-24T08:59:32.259 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:59:31 smithi189 ceph-mon[31791]: pgmap v246: 129 pgs: 129 active+clean; 49 MiB data, 689 MiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 378 KiB/s wr, 44 op/s 2024-04-24T08:59:32.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:59:31 smithi083 ceph-mon[27907]: pgmap v246: 129 pgs: 129 active+clean; 49 MiB data, 689 MiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 378 KiB/s wr, 44 op/s 2024-04-24T08:59:32.432 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.432+0000 7fa377000640 1 -- 172.21.15.83:0/2649040727 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa370101910 msgr2=0x7fa370105950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:32.432 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.432+0000 7fa377000640 1 --2- 172.21.15.83:0/2649040727 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa370101910 0x7fa370105950 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7fa35c009900 tx=0x7fa35c02a3f0 comp rx=0 tx=0).stop 2024-04-24T08:59:32.432 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.432+0000 7fa377000640 1 -- 172.21.15.83:0/2649040727 shutdown_connections 2024-04-24T08:59:32.432 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.432+0000 7fa377000640 1 --2- 172.21.15.83:0/2649040727 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa370106030 0x7fa37010dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:32.432 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.432+0000 7fa377000640 1 --2- 172.21.15.83:0/2649040727 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa370101910 0x7fa370105950 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:32.432 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.432+0000 7fa377000640 1 --2- 172.21.15.83:0/2649040727 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa370100f60 0x7fa370101340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:32.432 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.432+0000 7fa377000640 1 -- 172.21.15.83:0/2649040727 >> 172.21.15.83:0/2649040727 conn(0x7fa3700fc820 msgr2=0x7fa3700fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:32.432 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.433+0000 7fa377000640 1 -- 172.21.15.83:0/2649040727 shutdown_connections 2024-04-24T08:59:32.433 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.433+0000 7fa377000640 1 -- 172.21.15.83:0/2649040727 wait complete. 2024-04-24T08:59:32.433 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.434+0000 7fa377000640 1 Processor -- start 2024-04-24T08:59:32.433 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.434+0000 7fa377000640 1 -- start start 2024-04-24T08:59:32.434 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.435+0000 7fa377000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa370100f60 0x7fa370198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:32.434 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.435+0000 7fa377000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa370106030 0x7fa370198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:32.434 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.435+0000 7fa377000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa37019cf00 0x7fa37019d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:32.434 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.435+0000 7fa377000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa370110f70 con 0x7fa370106030 2024-04-24T08:59:32.434 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.435+0000 7fa377000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa370110df0 con 0x7fa370100f60 2024-04-24T08:59:32.435 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.435+0000 7fa377000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa3701110f0 con 0x7fa37019cf00 2024-04-24T08:59:32.435 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.435+0000 7fa375200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa370106030 0x7fa370198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:32.435 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.435+0000 7fa375200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa370106030 0x7fa370198b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44044/0 (socket says 172.21.15.83:44044) 2024-04-24T08:59:32.435 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.435+0000 7fa375200640 1 -- 172.21.15.83:0/1119675240 learned_addr learned my addr 172.21.15.83:0/1119675240 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:59:32.435 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.435+0000 7fa375c00640 1 --2- 172.21.15.83:0/1119675240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa370100f60 0x7fa370198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:32.435 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.435+0000 7fa376600640 1 --2- 172.21.15.83:0/1119675240 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa37019cf00 0x7fa37019d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:32.436 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.436+0000 7fa375200640 1 -- 172.21.15.83:0/1119675240 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa37019cf00 msgr2=0x7fa37019d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:32.436 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.436+0000 7fa375200640 1 --2- 172.21.15.83:0/1119675240 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa37019cf00 0x7fa37019d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:32.436 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.436+0000 7fa375200640 1 -- 172.21.15.83:0/1119675240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa370100f60 msgr2=0x7fa370198640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:32.436 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.436+0000 7fa375200640 1 --2- 172.21.15.83:0/1119675240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa370100f60 0x7fa370198640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:32.436 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.436+0000 7fa375200640 1 -- 172.21.15.83:0/1119675240 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa37019d9f0 con 0x7fa370106030 2024-04-24T08:59:32.436 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.436+0000 7fa375200640 1 --2- 172.21.15.83:0/1119675240 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa370106030 0x7fa370198b80 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7fa35c004270 tx=0x7fa35c0042a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:32.436 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.437+0000 7fa36e000640 1 -- 172.21.15.83:0/1119675240 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa35c03f030 con 0x7fa370106030 2024-04-24T08:59:32.436 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.437+0000 7fa377000640 1 -- 172.21.15.83:0/1119675240 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa37019dc80 con 0x7fa370106030 2024-04-24T08:59:32.436 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.437+0000 7fa36e000640 1 -- 172.21.15.83:0/1119675240 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa35c0044c0 con 0x7fa370106030 2024-04-24T08:59:32.437 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.437+0000 7fa36e000640 1 -- 172.21.15.83:0/1119675240 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa35c035070 con 0x7fa370106030 2024-04-24T08:59:32.437 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.437+0000 7fa377000640 1 -- 172.21.15.83:0/1119675240 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3701a54a0 con 0x7fa370106030 2024-04-24T08:59:32.439 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.441+0000 7fa37b473740 1 Processor -- start 2024-04-24T08:59:32.440 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.441+0000 7fa37b473740 1 -- start start 2024-04-24T08:59:32.440 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.441+0000 7fa37b473740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55586d5a5300 0x55586d5a56e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:32.441 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.441+0000 7fa37b473740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55586d5a5c20 0x55586d5b4470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:32.441 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.441+0000 7fa37b473740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55586d5b49b0 0x55586d5b6da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:32.441 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.441+0000 7fa37b473740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55586d50b900 con 0x55586d5b49b0 2024-04-24T08:59:32.441 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.441+0000 7fa37b473740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55586d4b1f20 con 0x55586d5a5300 2024-04-24T08:59:32.441 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.441+0000 7fa37b473740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55586d5078c0 con 0x55586d5a5c20 2024-04-24T08:59:32.441 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.442+0000 7fa36e000640 1 -- 172.21.15.83:0/1119675240 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa35c042030 con 0x7fa370106030 2024-04-24T08:59:32.442 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.442+0000 7fa376600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55586d5a5300 0x55586d5a56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:32.442 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.442+0000 7fa376600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55586d5a5300 0x55586d5a56e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:58690/0 (socket says 172.21.15.83:58690) 2024-04-24T08:59:32.442 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.442+0000 7fa376600640 1 -- 172.21.15.83:0/1663389458 learned_addr learned my addr 172.21.15.83:0/1663389458 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:59:32.442 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.442+0000 7fa375c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55586d5a5c20 0x55586d5b4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:32.442 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.442+0000 7fa375200640 1 --2- 172.21.15.83:0/1663389458 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55586d5b49b0 0x55586d5b6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:32.442 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.443+0000 7fa376600640 1 -- 172.21.15.83:0/1663389458 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55586d5a5c20 msgr2=0x55586d5b4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:32.442 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.443+0000 7fa376600640 1 --2- 172.21.15.83:0/1663389458 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55586d5a5c20 0x55586d5b4470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:32.443 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.443+0000 7fa376600640 1 -- 172.21.15.83:0/1663389458 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55586d5b49b0 msgr2=0x55586d5b6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:32.443 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.443+0000 7fa376600640 1 --2- 172.21.15.83:0/1663389458 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55586d5b49b0 0x55586d5b6da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:32.443 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.443+0000 7fa376600640 1 -- 172.21.15.83:0/1663389458 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55586d4d6e10 con 0x55586d5a5300 2024-04-24T08:59:32.443 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.443+0000 7fa375200640 1 --2- 172.21.15.83:0/1663389458 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55586d5b49b0 0x55586d5b6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:59:32.443 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.443+0000 7fa376600640 1 --2- 172.21.15.83:0/1663389458 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55586d5a5300 0x55586d5a56e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa36800ae10 tx=0x7fa36800f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3d9c73a4de8fa06 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:32.444 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.444+0000 7fa36e000640 1 --2- 172.21.15.83:0/1119675240 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa3580a3d10 0x7fa3580a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:32.444 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.444+0000 7fa362a00640 1 -- 172.21.15.83:0/1663389458 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa368003b70 con 0x55586d5a5300 2024-04-24T08:59:32.445 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.444+0000 7fa36e000640 1 -- 172.21.15.83:0/1119675240 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7fa35c0f0df0 con 0x7fa370106030 2024-04-24T08:59:32.445 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.444+0000 7fa362a00640 1 -- 172.21.15.83:0/1663389458 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa368003d10 con 0x55586d5a5300 2024-04-24T08:59:32.445 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.444+0000 7fa362a00640 1 -- 172.21.15.83:0/1663389458 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3680109c0 con 0x55586d5a5300 2024-04-24T08:59:32.445 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.444+0000 7fa375c00640 1 --2- 172.21.15.83:0/1119675240 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa3580a3d10 0x7fa3580a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:32.445 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.445+0000 7fa37b473740 1 -- 172.21.15.83:0/1663389458 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55586d5a5300 msgr2=0x55586d5a56e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:32.445 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.445+0000 7fa375c00640 1 --2- 172.21.15.83:0/1119675240 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa3580a3d10 0x7fa3580a61d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa36400ad30 tx=0x7fa36400a2d0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:32.445 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.445+0000 7fa37b473740 1 --2- 172.21.15.83:0/1663389458 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55586d5a5300 0x55586d5a56e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa36800ae10 tx=0x7fa36800f790 comp rx=0 tx=0).stop 2024-04-24T08:59:32.445 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.445+0000 7fa37b473740 1 -- 172.21.15.83:0/1663389458 shutdown_connections 2024-04-24T08:59:32.445 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.445+0000 7fa37b473740 1 --2- 172.21.15.83:0/1663389458 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55586d5b49b0 0x55586d5b6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:32.445 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.445+0000 7fa37b473740 1 --2- 172.21.15.83:0/1663389458 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55586d5a5c20 0x55586d5b4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:32.445 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.445+0000 7fa37b473740 1 --2- 172.21.15.83:0/1663389458 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55586d5a5300 0x55586d5a56e0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:32.445 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.445+0000 7fa37b473740 1 -- 172.21.15.83:0/1663389458 >> 172.21.15.83:0/1663389458 conn(0x55586d5a18d0 msgr2=0x55586d5a1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:32.445 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.445+0000 7fa37b473740 1 -- 172.21.15.83:0/1663389458 shutdown_connections 2024-04-24T08:59:32.445 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.445+0000 7fa37b473740 1 -- 172.21.15.83:0/1663389458 wait complete. 2024-04-24T08:59:32.446 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.446+0000 7fa37b473740 1 -- start start 2024-04-24T08:59:32.446 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.446+0000 7fa37b473740 1 Processor -- start 2024-04-24T08:59:32.448 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.446+0000 7fa37b473740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55586d5a5300 0x55586d5b1930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:32.449 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.446+0000 7fa37b473740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55586d5a5c20 0x55586d5b1e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:32.449 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.446+0000 7fa37b473740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55586d5b49b0 0x55586d5b23b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:32.449 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.446+0000 7fa37b473740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55586d5b7580 con 0x55586d5a5c20 2024-04-24T08:59:32.449 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.446+0000 7fa37b473740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55586d4c52c0 con 0x55586d5a5300 2024-04-24T08:59:32.449 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.446+0000 7fa37b473740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55586d5b7700 con 0x55586d5b49b0 2024-04-24T08:59:32.449 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.446+0000 7fa37b473740 10 client.0 _ll_register_callbacks cb 0x55586d21da00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:59:32.449 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.447+0000 7fa375200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55586d5a5c20 0x55586d5b1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:32.449 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.447+0000 7fa375200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55586d5a5c20 0x55586d5b1e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44054/0 (socket says 172.21.15.83:44054) 2024-04-24T08:59:32.449 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.447+0000 7fa375200640 1 -- 172.21.15.83:0/1748652465 learned_addr learned my addr 172.21.15.83:0/1748652465 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:59:32.449 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.447+0000 7fa375c00640 1 --2- 172.21.15.83:0/1748652465 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55586d5a5300 0x55586d5b1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:32.449 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.447+0000 7fa376600640 1 --2- 172.21.15.83:0/1748652465 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55586d5b49b0 0x55586d5b23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:32.449 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.447+0000 7fa375200640 1 -- 172.21.15.83:0/1748652465 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55586d5b49b0 msgr2=0x55586d5b23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:32.450 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.447+0000 7fa375200640 1 --2- 172.21.15.83:0/1748652465 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55586d5b49b0 0x55586d5b23b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:32.450 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.447+0000 7fa375200640 1 -- 172.21.15.83:0/1748652465 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55586d5a5300 msgr2=0x55586d5b1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:32.450 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.447+0000 7fa375200640 1 --2- 172.21.15.83:0/1748652465 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55586d5a5300 0x55586d5b1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:32.450 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.447+0000 7fa375200640 1 -- 172.21.15.83:0/1748652465 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55586d4d5fd0 con 0x55586d5a5c20 2024-04-24T08:59:32.450 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.447+0000 7fa375c00640 1 --2- 172.21.15.83:0/1748652465 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55586d5a5300 0x55586d5b1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:59:32.450 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.447+0000 7fa375200640 1 --2- 172.21.15.83:0/1748652465 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55586d5a5c20 0x55586d5b1e70 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7fa35c0bc900 tx=0x7fa35c02a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=955d8df70cb2d327 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:32.450 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.447+0000 7fa357400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:59:32.450 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.448+0000 7fa357400640 1 -- 172.21.15.83:0/1748652465 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa35c049070 con 0x55586d5a5c20 2024-04-24T08:59:32.450 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.448+0000 7fa357400640 1 -- 172.21.15.83:0/1748652465 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7fa35c0b8fe0 con 0x55586d5a5c20 2024-04-24T08:59:32.450 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.448+0000 7fa37b473740 1 -- 172.21.15.83:0/1748652465 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa368003d10 con 0x55586d5a5c20 2024-04-24T08:59:32.450 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.448+0000 7fa357400640 1 -- 172.21.15.83:0/1748652465 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fa35c0bd070 con 0x55586d5a5c20 2024-04-24T08:59:32.450 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.448+0000 7fa37b473740 10 client.18420 fetch_fsmap learned FSMap version 35 2024-04-24T08:59:32.450 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.448+0000 7fa37b473740 1 -- 172.21.15.83:0/1748652465 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55586d21e210 con 0x55586d5a5c20 2024-04-24T08:59:32.451 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.448+0000 7fa357400640 1 -- 172.21.15.83:0/1748652465 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa368003d10 con 0x55586d5a5c20 2024-04-24T08:59:32.451 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.448+0000 7fa357400640 1 -- 172.21.15.83:0/1748652465 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7fa35c04c060 con 0x55586d5a5c20 2024-04-24T08:59:32.451 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.449+0000 7fa37b473740 10 client.18420 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T08:59:32.451 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.449+0000 7fa37b473740 10 client.18420 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T08:59:32.451 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.449+0000 7fa37b473740 20 client.18420 populate_metadata read hostname 'smithi083' 2024-04-24T08:59:32.451 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.449+0000 7fa37b473740 1 --2- 172.21.15.83:0/1748652465 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55586d5c4a30 0x55586d5c6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:32.451 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.449+0000 7fa37b473740 4 client.18420 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:59:32.451 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.449+0000 7fa37b473740 1 -- 172.21.15.83:0/1748652465 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55586d519320 con 0x55586d5c4a30 2024-04-24T08:59:32.451 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.449+0000 7fa375c00640 1 --2- 172.21.15.83:0/1748652465 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55586d5c4a30 0x55586d5c6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:32.452 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.450+0000 7fa375c00640 1 --2- 172.21.15.83:0/1748652465 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55586d5c4a30 0x55586d5c6ef0 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1262f63ff4d45ee server_cookie=112dda57edfc78b6 in_seq=0 out_seq=0 2024-04-24T08:59:32.452 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.450+0000 7fa357400640 10 client.18420 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:59:32.452 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.452+0000 7fa357400640 1 -- 172.21.15.83:0/1748652465 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55586d519320 con 0x55586d5c4a30 2024-04-24T08:59:32.452 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.452+0000 7fa357400640 10 client.18420 handle_command_reply: tid=0 2024-04-24T08:59:32.453 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.452+0000 7fa37b473740 1 client.18420 shutdown 2024-04-24T08:59:32.453 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.452+0000 7fa37b473740 1 -- 172.21.15.83:0/1748652465 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55586d5a5c20 msgr2=0x55586d5b1e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:32.453 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.452+0000 7fa37b473740 1 --2- 172.21.15.83:0/1748652465 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55586d5a5c20 0x55586d5b1e70 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7fa35c0bc900 tx=0x7fa35c02a3f0 comp rx=0 tx=0).stop 2024-04-24T08:59:32.453 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.452+0000 7fa37b473740 1 -- 172.21.15.83:0/1748652465 shutdown_connections 2024-04-24T08:59:32.453 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.452+0000 7fa37b473740 1 --2- 172.21.15.83:0/1748652465 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55586d5c4a30 0x55586d5c6ef0 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:32.453 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.452+0000 7fa37b473740 1 --2- 172.21.15.83:0/1748652465 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55586d5b49b0 0x55586d5b23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:32.453 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.453+0000 7fa37b473740 1 --2- 172.21.15.83:0/1748652465 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55586d5a5c20 0x55586d5b1e70 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:32.453 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.453+0000 7fa37b473740 1 --2- 172.21.15.83:0/1748652465 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55586d5a5300 0x55586d5b1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:32.453 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.453+0000 7fa357400640 0 client.18420 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:59:32.453 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.453+0000 7fa37b473740 1 -- 172.21.15.83:0/1748652465 >> 172.21.15.83:0/1748652465 conn(0x55586d5a18d0 msgr2=0x55586d5a7650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:32.453 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.453+0000 7fa37b473740 1 -- 172.21.15.83:0/1748652465 shutdown_connections 2024-04-24T08:59:32.453 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.453+0000 7fa37b473740 1 -- 172.21.15.83:0/1748652465 wait complete. 2024-04-24T08:59:32.453 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.453+0000 7fa37b473740 20 client.18420 trim_cache size 0 max 16384 2024-04-24T08:59:32.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:59:31 smithi138 ceph-mon[31441]: pgmap v246: 129 pgs: 129 active+clean; 49 MiB data, 689 MiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 378 KiB/s wr, 44 op/s 2024-04-24T08:59:32.469 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.470+0000 7fa37b473740 1 Processor -- start 2024-04-24T08:59:32.469 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.470+0000 7fa37b473740 1 -- start start 2024-04-24T08:59:32.469 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.470+0000 7fa37b473740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55586d5a5300 0x55586d5c0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:32.470 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.470+0000 7fa37b473740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55586d5a5c20 0x55586d5cc800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:32.470 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.470+0000 7fa37b473740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55586d5b49b0 0x55586d5ccd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:32.470 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.470+0000 7fa37b473740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa3701ad250 con 0x55586d5b49b0 2024-04-24T08:59:32.470 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.470+0000 7fa37b473740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa3701aed90 con 0x55586d5a5c20 2024-04-24T08:59:32.470 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.470+0000 7fa37b473740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa3701ad920 con 0x55586d5a5300 2024-04-24T08:59:32.471 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.471+0000 7fa376600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55586d5b49b0 0x55586d5ccd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:32.471 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.471+0000 7fa376600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55586d5b49b0 0x55586d5ccd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44070/0 (socket says 172.21.15.83:44070) 2024-04-24T08:59:32.471 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.471+0000 7fa376600640 1 -- 172.21.15.83:0/2054386158 learned_addr learned my addr 172.21.15.83:0/2054386158 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:59:32.471 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.471+0000 7fa375c00640 1 --2- 172.21.15.83:0/2054386158 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55586d5a5300 0x55586d5c0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:32.471 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.471+0000 7fa375200640 1 --2- 172.21.15.83:0/2054386158 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55586d5a5c20 0x55586d5cc800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:32.471 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.471+0000 7fa376600640 1 -- 172.21.15.83:0/2054386158 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55586d5a5300 msgr2=0x55586d5c0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:32.471 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.471+0000 7fa376600640 1 --2- 172.21.15.83:0/2054386158 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55586d5a5300 0x55586d5c0480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:32.471 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.471+0000 7fa376600640 1 -- 172.21.15.83:0/2054386158 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55586d5a5c20 msgr2=0x55586d5cc800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:32.471 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.471+0000 7fa376600640 1 --2- 172.21.15.83:0/2054386158 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55586d5a5c20 0x55586d5cc800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:32.472 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.471+0000 7fa376600640 1 -- 172.21.15.83:0/2054386158 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55586d5a5a50 con 0x55586d5b49b0 2024-04-24T08:59:32.472 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.471+0000 7fa375c00640 1 --2- 172.21.15.83:0/2054386158 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55586d5a5300 0x55586d5c0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:59:32.472 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.471+0000 7fa376600640 1 --2- 172.21.15.83:0/2054386158 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55586d5b49b0 0x55586d5ccd40 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7fa36800aba0 tx=0x7fa36800a7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d22de47a3d42ce3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:32.472 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.472+0000 7fa362000640 1 -- 172.21.15.83:0/2054386158 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa368005ea0 con 0x55586d5b49b0 2024-04-24T08:59:32.472 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.472+0000 7fa362000640 1 -- 172.21.15.83:0/2054386158 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa368010640 con 0x55586d5b49b0 2024-04-24T08:59:32.472 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.472+0000 7fa362000640 1 -- 172.21.15.83:0/2054386158 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3680236d0 con 0x55586d5b49b0 2024-04-24T08:59:32.474 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.473+0000 7fa37b473740 1 -- 172.21.15.83:0/2054386158 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55586d5b49b0 msgr2=0x55586d5ccd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:32.474 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.473+0000 7fa37b473740 1 --2- 172.21.15.83:0/2054386158 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55586d5b49b0 0x55586d5ccd40 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7fa36800aba0 tx=0x7fa36800a7f0 comp rx=0 tx=0).stop 2024-04-24T08:59:32.474 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.473+0000 7fa37b473740 1 -- 172.21.15.83:0/2054386158 shutdown_connections 2024-04-24T08:59:32.474 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.473+0000 7fa37b473740 1 --2- 172.21.15.83:0/2054386158 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55586d5b49b0 0x55586d5ccd40 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:32.474 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.473+0000 7fa37b473740 1 --2- 172.21.15.83:0/2054386158 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55586d5a5c20 0x55586d5cc800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:32.474 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.473+0000 7fa37b473740 1 --2- 172.21.15.83:0/2054386158 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55586d5a5300 0x55586d5c0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:32.474 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.473+0000 7fa37b473740 1 -- 172.21.15.83:0/2054386158 >> 172.21.15.83:0/2054386158 conn(0x55586d5a18d0 msgr2=0x55586d5a6dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:32.474 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.473+0000 7fa37b473740 1 -- 172.21.15.83:0/2054386158 shutdown_connections 2024-04-24T08:59:32.474 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.473+0000 7fa37b473740 1 -- 172.21.15.83:0/2054386158 wait complete. 2024-04-24T08:59:32.474 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.473+0000 7fa37b473740 1 -- start start 2024-04-24T08:59:32.475 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.474+0000 7fa37b473740 1 Processor -- start 2024-04-24T08:59:32.475 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.474+0000 7fa37b473740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55586d5a5300 0x55586d5bc990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:32.475 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.474+0000 7fa37b473740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55586d5a5c20 0x55586d5bced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:32.475 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.474+0000 7fa37b473740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55586d5b49b0 0x55586d5b9960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:32.475 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.474+0000 7fa37b473740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa3701ae6c0 con 0x55586d5a5300 2024-04-24T08:59:32.475 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.474+0000 7fa37b473740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa3701adff0 con 0x55586d5a5c20 2024-04-24T08:59:32.475 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.474+0000 7fa37b473740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa3701ab250 con 0x55586d5b49b0 2024-04-24T08:59:32.475 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.474+0000 7fa37b473740 10 client.0 _ll_register_callbacks cb 0x7fa35c009260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:59:32.475 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.474+0000 7fa375c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55586d5a5300 0x55586d5bc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:32.475 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.474+0000 7fa375c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55586d5a5300 0x55586d5bc990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44074/0 (socket says 172.21.15.83:44074) 2024-04-24T08:59:32.475 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.474+0000 7fa375c00640 1 -- 172.21.15.83:0/2684637874 learned_addr learned my addr 172.21.15.83:0/2684637874 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:59:32.475 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.475+0000 7fa375200640 1 --2- 172.21.15.83:0/2684637874 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55586d5a5c20 0x55586d5bced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:32.475 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.475+0000 7fa376600640 1 --2- 172.21.15.83:0/2684637874 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55586d5b49b0 0x55586d5b9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:32.475 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.475+0000 7fa375c00640 1 -- 172.21.15.83:0/2684637874 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55586d5b49b0 msgr2=0x55586d5b9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:32.475 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.475+0000 7fa375c00640 1 --2- 172.21.15.83:0/2684637874 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55586d5b49b0 0x55586d5b9960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:32.476 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.475+0000 7fa375c00640 1 -- 172.21.15.83:0/2684637874 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55586d5a5c20 msgr2=0x55586d5bced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:32.476 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.475+0000 7fa375c00640 1 --2- 172.21.15.83:0/2684637874 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55586d5a5c20 0x55586d5bced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:32.476 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.475+0000 7fa375c00640 1 -- 172.21.15.83:0/2684637874 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55586d2ea620 con 0x55586d5a5300 2024-04-24T08:59:32.477 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.475+0000 7fa375c00640 1 --2- 172.21.15.83:0/2684637874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55586d5a5300 0x55586d5bc990 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x55586d5c7740 tx=0x7fa3640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f07601a5e4f834c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:32.478 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.475+0000 7fa357400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:59:32.478 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.475+0000 7fa357400640 1 -- 172.21.15.83:0/2684637874 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3640290f0 con 0x55586d5a5300 2024-04-24T08:59:32.478 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.476+0000 7fa37b473740 1 -- 172.21.15.83:0/2684637874 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa368010640 con 0x55586d5a5300 2024-04-24T08:59:32.478 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.476+0000 7fa357400640 1 -- 172.21.15.83:0/2684637874 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7fa364029290 con 0x55586d5a5300 2024-04-24T08:59:32.478 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.476+0000 7fa357400640 1 -- 172.21.15.83:0/2684637874 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fa364039050 con 0x55586d5a5300 2024-04-24T08:59:32.478 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.476+0000 7fa37b473740 10 client.18426 fetch_fsmap learned FSMap version 35 2024-04-24T08:59:32.478 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.476+0000 7fa37b473740 1 -- 172.21.15.83:0/2684637874 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55586d62fe40 con 0x55586d5a5300 2024-04-24T08:59:32.478 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.476+0000 7fa357400640 1 -- 172.21.15.83:0/2684637874 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa368010640 con 0x55586d5a5300 2024-04-24T08:59:32.478 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.476+0000 7fa357400640 1 -- 172.21.15.83:0/2684637874 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7fa364027b90 con 0x55586d5a5300 2024-04-24T08:59:32.478 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.476+0000 7fa37b473740 10 client.18426 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T08:59:32.479 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.476+0000 7fa37b473740 10 client.18426 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T08:59:32.479 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.476+0000 7fa37b473740 20 client.18426 populate_metadata read hostname 'smithi083' 2024-04-24T08:59:32.479 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.476+0000 7fa37b473740 1 --2- 172.21.15.83:0/2684637874 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55586d630510 0x55586d632930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:32.479 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.476+0000 7fa37b473740 4 client.18426 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T08:59:32.479 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.476+0000 7fa37b473740 1 -- 172.21.15.83:0/2684637874 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55586d513190 con 0x55586d630510 2024-04-24T08:59:32.479 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.477+0000 7fa375200640 1 --2- 172.21.15.83:0/2684637874 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55586d630510 0x55586d632930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:32.479 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.478+0000 7fa375200640 1 --2- 172.21.15.83:0/2684637874 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55586d630510 0x55586d632930 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8662166b09de8c server_cookie=b0d01582972ac1c7 in_seq=0 out_seq=0 2024-04-24T08:59:32.479 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.478+0000 7fa357400640 10 client.18426 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:59:32.541 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.542+0000 7fa357400640 1 -- 172.21.15.83:0/2684637874 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55586d513190 con 0x55586d630510 2024-04-24T08:59:32.541 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.542+0000 7fa357400640 10 client.18426 handle_command_reply: tid=0 2024-04-24T08:59:32.541 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.542+0000 7fa37b473740 1 client.18426 shutdown 2024-04-24T08:59:32.542 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.542+0000 7fa37b473740 1 -- 172.21.15.83:0/2684637874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55586d5a5300 msgr2=0x55586d5bc990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:32.542 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.542+0000 7fa37b473740 1 --2- 172.21.15.83:0/2684637874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55586d5a5300 0x55586d5bc990 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x55586d5c7740 tx=0x7fa3640046c0 comp rx=0 tx=0).stop 2024-04-24T08:59:32.543 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.543+0000 7fa37b473740 1 -- 172.21.15.83:0/2684637874 shutdown_connections 2024-04-24T08:59:32.543 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.543+0000 7fa37b473740 1 --2- 172.21.15.83:0/2684637874 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55586d630510 0x55586d632930 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:32.543 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.543+0000 7fa37b473740 1 --2- 172.21.15.83:0/2684637874 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55586d5b49b0 0x55586d5b9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:32.543 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.543+0000 7fa37b473740 1 --2- 172.21.15.83:0/2684637874 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55586d5a5c20 0x55586d5bced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:32.543 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.543+0000 7fa37b473740 1 --2- 172.21.15.83:0/2684637874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55586d5a5300 0x55586d5bc990 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:32.543 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.543+0000 7fa37b473740 1 -- 172.21.15.83:0/2684637874 >> 172.21.15.83:0/2684637874 conn(0x55586d5a18d0 msgr2=0x55586d5a6d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:32.543 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.543+0000 7fa357400640 0 client.18426 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:59:32.543 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.543+0000 7fa37b473740 1 -- 172.21.15.83:0/2684637874 shutdown_connections 2024-04-24T08:59:32.543 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.543+0000 7fa37b473740 1 -- 172.21.15.83:0/2684637874 wait complete. 2024-04-24T08:59:32.543 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.543+0000 7fa37b473740 20 client.18426 trim_cache size 0 max 16384 2024-04-24T08:59:32.544 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T08:59:32.544 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T08:59:32.544 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T08:59:32.544 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T08:59:32.544 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.545+0000 7fa377000640 1 -- 172.21.15.83:0/1119675240 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa3580a3d10 msgr2=0x7fa3580a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:32.545 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.545+0000 7fa377000640 1 --2- 172.21.15.83:0/1119675240 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa3580a3d10 0x7fa3580a61d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa36400ad30 tx=0x7fa36400a2d0 comp rx=0 tx=0).stop 2024-04-24T08:59:32.545 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.545+0000 7fa377000640 1 -- 172.21.15.83:0/1119675240 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa370106030 msgr2=0x7fa370198b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:32.545 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.545+0000 7fa377000640 1 --2- 172.21.15.83:0/1119675240 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa370106030 0x7fa370198b80 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7fa35c004270 tx=0x7fa35c0042a0 comp rx=0 tx=0).stop 2024-04-24T08:59:32.545 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.545+0000 7fa377000640 1 -- 172.21.15.83:0/1119675240 shutdown_connections 2024-04-24T08:59:32.545 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.545+0000 7fa377000640 1 --2- 172.21.15.83:0/1119675240 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa3580a3d10 0x7fa3580a61d0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:32.545 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.545+0000 7fa377000640 1 --2- 172.21.15.83:0/1119675240 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa37019cf00 0x7fa37019d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:32.545 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.545+0000 7fa377000640 1 --2- 172.21.15.83:0/1119675240 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa370106030 0x7fa370198b80 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:32.545 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.545+0000 7fa377000640 1 --2- 172.21.15.83:0/1119675240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa370100f60 0x7fa370198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:32.545 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.545+0000 7fa377000640 1 -- 172.21.15.83:0/1119675240 >> 172.21.15.83:0/1119675240 conn(0x7fa3700fc820 msgr2=0x7fa3700699b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:32.545 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.545+0000 7fa377000640 1 -- 172.21.15.83:0/1119675240 shutdown_connections 2024-04-24T08:59:32.545 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:32.545+0000 7fa377000640 1 -- 172.21.15.83:0/1119675240 wait complete. 2024-04-24T08:59:32.594 INFO:tasks.workunit.client.0.smithi083.stdout: 1 43538 0.63 MB/sec execute 246 sec latency 61.621 ms 2024-04-24T08:59:33.594 INFO:tasks.workunit.client.0.smithi083.stdout: 1 43656 0.63 MB/sec execute 247 sec latency 49.582 ms 2024-04-24T08:59:34.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:59:33 smithi083 ceph-mon[27907]: pgmap v247: 129 pgs: 129 active+clean; 48 MiB data, 695 MiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 502 KiB/s wr, 71 op/s 2024-04-24T08:59:34.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:59:33 smithi138 ceph-mon[31441]: pgmap v247: 129 pgs: 129 active+clean; 48 MiB data, 695 MiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 502 KiB/s wr, 71 op/s 2024-04-24T08:59:34.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:59:33 smithi189 ceph-mon[31791]: pgmap v247: 129 pgs: 129 active+clean; 48 MiB data, 695 MiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 502 KiB/s wr, 71 op/s 2024-04-24T08:59:34.594 INFO:tasks.workunit.client.0.smithi083.stdout: 1 43772 0.63 MB/sec execute 248 sec latency 52.602 ms 2024-04-24T08:59:35.369 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T08:59:35.369 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T08:59:35.594 INFO:tasks.workunit.client.0.smithi083.stdout: 1 43878 0.62 MB/sec execute 249 sec latency 57.726 ms 2024-04-24T08:59:35.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.599+0000 7f2ca6200640 1 -- 172.21.15.83:0/227453267 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ca0104450 msgr2=0x7f2ca0104830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:35.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.599+0000 7f2ca6200640 1 --2- 172.21.15.83:0/227453267 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ca0104450 0x7f2ca0104830 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f2c90009900 tx=0x7f2c9002a410 comp rx=0 tx=0).stop 2024-04-24T08:59:35.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.599+0000 7f2ca6200640 1 -- 172.21.15.83:0/227453267 shutdown_connections 2024-04-24T08:59:35.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.599+0000 7f2ca6200640 1 --2- 172.21.15.83:0/227453267 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ca0109520 0x7f2ca01110f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:35.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.599+0000 7f2ca6200640 1 --2- 172.21.15.83:0/227453267 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2ca0104e00 0x7f2ca0108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:35.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.599+0000 7f2ca6200640 1 --2- 172.21.15.83:0/227453267 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ca0104450 0x7f2ca0104830 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:35.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.599+0000 7f2ca6200640 1 -- 172.21.15.83:0/227453267 >> 172.21.15.83:0/227453267 conn(0x7f2ca00ffc70 msgr2=0x7f2ca0102090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:35.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.599+0000 7f2ca6200640 1 -- 172.21.15.83:0/227453267 shutdown_connections 2024-04-24T08:59:35.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.599+0000 7f2ca6200640 1 -- 172.21.15.83:0/227453267 wait complete. 2024-04-24T08:59:35.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.600+0000 7f2ca6200640 1 Processor -- start 2024-04-24T08:59:35.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.600+0000 7f2ca6200640 1 -- start start 2024-04-24T08:59:35.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.601+0000 7f2ca6200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ca0104450 0x7f2ca01a0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:35.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.601+0000 7f2ca6200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2ca0109520 0x7f2ca01a0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:35.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.601+0000 7f2ca6200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ca019a960 0x7f2ca019ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:35.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.601+0000 7f2ca6200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca0114480 con 0x7f2ca019a960 2024-04-24T08:59:35.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.601+0000 7f2ca6200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca0114300 con 0x7f2ca0104450 2024-04-24T08:59:35.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.601+0000 7f2ca6200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca0114600 con 0x7f2ca0109520 2024-04-24T08:59:35.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.601+0000 7f2c9fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ca019a960 0x7f2ca019ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:35.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.601+0000 7f2c9fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ca019a960 0x7f2ca019ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44096/0 (socket says 172.21.15.83:44096) 2024-04-24T08:59:35.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.601+0000 7f2c9fe00640 1 -- 172.21.15.83:0/4091079570 learned_addr learned my addr 172.21.15.83:0/4091079570 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:59:35.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.601+0000 7f2c9fe00640 1 -- 172.21.15.83:0/4091079570 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2ca0109520 msgr2=0x7f2ca01a0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:35.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.601+0000 7f2c96a00640 1 --2- 172.21.15.83:0/4091079570 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2ca0109520 0x7f2ca01a0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:35.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.601+0000 7f2c9f400640 1 --2- 172.21.15.83:0/4091079570 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ca0104450 0x7f2ca01a0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:35.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.601+0000 7f2c9fe00640 1 --2- 172.21.15.83:0/4091079570 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2ca0109520 0x7f2ca01a0ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:35.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.601+0000 7f2c9fe00640 1 -- 172.21.15.83:0/4091079570 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ca0104450 msgr2=0x7f2ca01a0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:35.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.601+0000 7f2c9fe00640 1 --2- 172.21.15.83:0/4091079570 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ca0104450 0x7f2ca01a0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:35.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.601+0000 7f2c9fe00640 1 -- 172.21.15.83:0/4091079570 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ca019b480 con 0x7f2ca019a960 2024-04-24T08:59:35.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.601+0000 7f2c9f400640 1 --2- 172.21.15.83:0/4091079570 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ca0104450 0x7f2ca01a0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:59:35.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.602+0000 7f2c9fe00640 1 --2- 172.21.15.83:0/4091079570 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ca019a960 0x7f2ca019ae10 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f2c9800eaa0 tx=0x7f2c9800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:35.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.602+0000 7f2c9cc00640 1 -- 172.21.15.83:0/4091079570 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c9800cdb0 con 0x7f2ca019a960 2024-04-24T08:59:35.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.602+0000 7f2ca6200640 1 -- 172.21.15.83:0/4091079570 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ca019b770 con 0x7f2ca019a960 2024-04-24T08:59:35.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.602+0000 7f2c9cc00640 1 -- 172.21.15.83:0/4091079570 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2c98005020 con 0x7f2ca019a960 2024-04-24T08:59:35.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.602+0000 7f2c9cc00640 1 -- 172.21.15.83:0/4091079570 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c980116d0 con 0x7f2ca019a960 2024-04-24T08:59:35.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.602+0000 7f2ca6200640 1 -- 172.21.15.83:0/4091079570 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ca01a7920 con 0x7f2ca019a960 2024-04-24T08:59:35.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.606+0000 7f2ca904f740 1 Processor -- start 2024-04-24T08:59:35.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.606+0000 7f2c9cc00640 1 -- 172.21.15.83:0/4091079570 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2c980051d0 con 0x7f2ca019a960 2024-04-24T08:59:35.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.606+0000 7f2ca904f740 1 -- start start 2024-04-24T08:59:35.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.607+0000 7f2ca904f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e0a399860 0x560e0a399c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:35.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.607+0000 7f2ca904f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e0a39a180 0x560e0a3a4310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:35.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.607+0000 7f2ca904f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e0a3a4850 0x560e0a3a6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:35.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.607+0000 7f2ca904f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x560e0a2ff1f0 con 0x560e0a39a180 2024-04-24T08:59:35.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.607+0000 7f2ca904f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560e0a2a5350 con 0x560e0a3a4850 2024-04-24T08:59:35.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.607+0000 7f2ca904f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x560e0a2fb1b0 con 0x560e0a399860 2024-04-24T08:59:35.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.607+0000 7f2c9f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e0a39a180 0x560e0a3a4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:35.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.607+0000 7f2c9f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e0a39a180 0x560e0a3a4310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44100/0 (socket says 172.21.15.83:44100) 2024-04-24T08:59:35.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.607+0000 7f2c9f400640 1 -- 172.21.15.83:0/4165082501 learned_addr learned my addr 172.21.15.83:0/4165082501 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:59:35.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.607+0000 7f2c96a00640 1 --2- 172.21.15.83:0/4165082501 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e0a399860 0x560e0a399c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:35.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.608+0000 7f2c9f400640 1 -- 172.21.15.83:0/4165082501 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e0a399860 msgr2=0x560e0a399c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:35.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.608+0000 7f2c9f400640 1 --2- 172.21.15.83:0/4165082501 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e0a399860 0x560e0a399c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:35.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.608+0000 7f2c96a00640 1 --2- 172.21.15.83:0/4165082501 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e0a3a4850 0x560e0a3a6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:35.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.608+0000 7f2c9f400640 1 -- 172.21.15.83:0/4165082501 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e0a3a4850 msgr2=0x560e0a3a6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:35.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.608+0000 7f2c9f400640 1 --2- 172.21.15.83:0/4165082501 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e0a3a4850 0x560e0a3a6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:35.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.608+0000 7f2c9f400640 1 -- 172.21.15.83:0/4165082501 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e0a2f9f50 con 0x560e0a39a180 2024-04-24T08:59:35.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.608+0000 7f2c96a00640 1 --2- 172.21.15.83:0/4165082501 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e0a3a4850 0x560e0a3a6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:59:35.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.608+0000 7f2c9cc00640 1 --2- 172.21.15.83:0/4091079570 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f2c840a3d20 0x7f2c840a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:35.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.608+0000 7f2c9f400640 1 --2- 172.21.15.83:0/4165082501 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e0a39a180 0x560e0a3a4310 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f2c90006010 tx=0x7f2c90032cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bd8d1c6c0073fce server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:35.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.608+0000 7f2c9cc00640 1 -- 172.21.15.83:0/4091079570 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7f2c980d0eb0 con 0x7f2ca019a960 2024-04-24T08:59:35.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.608+0000 7f2c96a00640 1 --2- 172.21.15.83:0/4091079570 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f2c840a3d20 0x7f2c840a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:35.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.608+0000 7f2c8fe00640 1 -- 172.21.15.83:0/4165082501 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c9003f070 con 0x560e0a39a180 2024-04-24T08:59:35.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.609+0000 7f2c8fe00640 1 -- 172.21.15.83:0/4165082501 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2c900316a0 con 0x560e0a39a180 2024-04-24T08:59:35.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.609+0000 7f2c8fe00640 1 -- 172.21.15.83:0/4165082501 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c90035070 con 0x560e0a39a180 2024-04-24T08:59:35.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.609+0000 7f2ca904f740 1 -- 172.21.15.83:0/4165082501 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e0a39a180 msgr2=0x560e0a3a4310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:35.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.609+0000 7f2c96a00640 1 --2- 172.21.15.83:0/4091079570 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f2c840a3d20 0x7f2c840a61e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x560e0a396d10 tx=0x7f2c880046c0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:35.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.609+0000 7f2ca904f740 1 --2- 172.21.15.83:0/4165082501 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e0a39a180 0x560e0a3a4310 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f2c90006010 tx=0x7f2c90032cc0 comp rx=0 tx=0).stop 2024-04-24T08:59:35.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.609+0000 7f2ca904f740 1 -- 172.21.15.83:0/4165082501 shutdown_connections 2024-04-24T08:59:35.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.609+0000 7f2ca904f740 1 --2- 172.21.15.83:0/4165082501 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e0a3a4850 0x560e0a3a6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:35.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.610+0000 7f2ca904f740 1 --2- 172.21.15.83:0/4165082501 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e0a39a180 0x560e0a3a4310 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:35.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.610+0000 7f2ca904f740 1 --2- 172.21.15.83:0/4165082501 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e0a399860 0x560e0a399c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:35.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.610+0000 7f2ca904f740 1 -- 172.21.15.83:0/4165082501 >> 172.21.15.83:0/4165082501 conn(0x560e0a395b80 msgr2=0x560e0a395f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:35.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.610+0000 7f2ca904f740 1 -- 172.21.15.83:0/4165082501 shutdown_connections 2024-04-24T08:59:35.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.610+0000 7f2ca904f740 1 -- 172.21.15.83:0/4165082501 wait complete. 2024-04-24T08:59:35.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.610+0000 7f2ca904f740 1 -- start start 2024-04-24T08:59:35.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.610+0000 7f2ca904f740 1 Processor -- start 2024-04-24T08:59:35.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.611+0000 7f2ca904f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e0a399860 0x560e0a3a1800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:35.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.611+0000 7f2ca904f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e0a39a180 0x560e0a3a1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:35.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.611+0000 7f2ca904f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e0a3a4850 0x560e0a3a2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:35.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.611+0000 7f2ca904f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x560e0a3a75e0 con 0x560e0a3a4850 2024-04-24T08:59:35.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.611+0000 7f2ca904f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560e0a2b8690 con 0x560e0a399860 2024-04-24T08:59:35.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.611+0000 7f2ca904f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x560e0a3a7760 con 0x560e0a39a180 2024-04-24T08:59:35.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.611+0000 7f2ca904f740 10 client.0 _ll_register_callbacks cb 0x560e0a2f4650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:59:35.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.611+0000 7f2c9fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e0a3a4850 0x560e0a3a2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:35.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.611+0000 7f2c9fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e0a3a4850 0x560e0a3a2280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44112/0 (socket says 172.21.15.83:44112) 2024-04-24T08:59:35.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.611+0000 7f2c9fe00640 1 -- 172.21.15.83:0/1343871995 learned_addr learned my addr 172.21.15.83:0/1343871995 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:59:35.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.611+0000 7f2c9f400640 1 --2- 172.21.15.83:0/1343871995 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e0a399860 0x560e0a3a1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:35.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.611+0000 7f2c9fe00640 1 -- 172.21.15.83:0/1343871995 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e0a39a180 msgr2=0x560e0a3a1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:35.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.611+0000 7f2c9fe00640 1 --2- 172.21.15.83:0/1343871995 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e0a39a180 0x560e0a3a1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:35.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.611+0000 7f2c9fe00640 1 -- 172.21.15.83:0/1343871995 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e0a399860 msgr2=0x560e0a3a1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:35.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.611+0000 7f2c9fe00640 1 --2- 172.21.15.83:0/1343871995 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e0a399860 0x560e0a3a1800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:35.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.611+0000 7f2c9fe00640 1 -- 172.21.15.83:0/1343871995 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e0a2b7920 con 0x560e0a3a4850 2024-04-24T08:59:35.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.612+0000 7f2c9fe00640 1 --2- 172.21.15.83:0/1343871995 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e0a3a4850 0x560e0a3a2280 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f2c9809c6d0 tx=0x7f2c9809cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9436533e244f2d7e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:35.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.612+0000 7f2c8cc00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:59:35.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.612+0000 7f2c8cc00640 1 -- 172.21.15.83:0/1343871995 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c9801f040 con 0x560e0a3a4850 2024-04-24T08:59:35.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.612+0000 7f2ca904f740 1 -- 172.21.15.83:0/1343871995 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c900316a0 con 0x560e0a3a4850 2024-04-24T08:59:35.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.612+0000 7f2c8cc00640 1 -- 172.21.15.83:0/1343871995 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7f2c980bbb90 con 0x560e0a3a4850 2024-04-24T08:59:35.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.612+0000 7f2c8cc00640 1 -- 172.21.15.83:0/1343871995 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f2c980bbd90 con 0x560e0a3a4850 2024-04-24T08:59:35.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.612+0000 7f2ca904f740 10 client.18438 fetch_fsmap learned FSMap version 35 2024-04-24T08:59:35.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.612+0000 7f2ca904f740 1 -- 172.21.15.83:0/1343871995 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x560e0a0db7d0 con 0x560e0a3a4850 2024-04-24T08:59:35.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.612+0000 7f2c8cc00640 1 -- 172.21.15.83:0/1343871995 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2c900316a0 con 0x560e0a3a4850 2024-04-24T08:59:35.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.613+0000 7f2c8cc00640 1 -- 172.21.15.83:0/1343871995 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7f2c9800f040 con 0x560e0a3a4850 2024-04-24T08:59:35.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.613+0000 7f2ca904f740 10 client.18438 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T08:59:35.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.613+0000 7f2ca904f740 10 client.18438 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:59:35.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.613+0000 7f2ca904f740 20 client.18438 populate_metadata read hostname 'smithi083' 2024-04-24T08:59:35.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.613+0000 7f2ca904f740 1 --2- 172.21.15.83:0/1343871995 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x560e0a3b4b00 0x560e0a3b6f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:35.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.613+0000 7f2ca904f740 4 client.18438 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:59:35.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.613+0000 7f2ca904f740 1 -- 172.21.15.83:0/1343871995 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560e0a30cc10 con 0x560e0a3b4b00 2024-04-24T08:59:35.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.614+0000 7f2c9f400640 1 --2- 172.21.15.83:0/1343871995 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x560e0a3b4b00 0x560e0a3b6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:35.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.614+0000 7f2c9f400640 1 --2- 172.21.15.83:0/1343871995 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x560e0a3b4b00 0x560e0a3b6f90 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d19e5cce2147a50d server_cookie=4797e567ec4dd819 in_seq=0 out_seq=0 2024-04-24T08:59:35.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.614+0000 7f2c8cc00640 10 client.18438 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:59:35.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.617+0000 7f2c8cc00640 1 -- 172.21.15.83:0/1343871995 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560e0a30cc10 con 0x560e0a3b4b00 2024-04-24T08:59:35.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.617+0000 7f2c8cc00640 10 client.18438 handle_command_reply: tid=0 2024-04-24T08:59:35.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.617+0000 7f2ca904f740 1 client.18438 shutdown 2024-04-24T08:59:35.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.617+0000 7f2ca904f740 1 -- 172.21.15.83:0/1343871995 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e0a3a4850 msgr2=0x560e0a3a2280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:35.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.617+0000 7f2ca904f740 1 --2- 172.21.15.83:0/1343871995 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e0a3a4850 0x560e0a3a2280 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f2c9809c6d0 tx=0x7f2c9809cb10 comp rx=0 tx=0).stop 2024-04-24T08:59:35.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.618+0000 7f2ca904f740 1 -- 172.21.15.83:0/1343871995 shutdown_connections 2024-04-24T08:59:35.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.618+0000 7f2ca904f740 1 --2- 172.21.15.83:0/1343871995 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x560e0a3b4b00 0x560e0a3b6f90 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:35.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.618+0000 7f2ca904f740 1 --2- 172.21.15.83:0/1343871995 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e0a3a4850 0x560e0a3a2280 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:35.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.618+0000 7f2ca904f740 1 --2- 172.21.15.83:0/1343871995 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e0a39a180 0x560e0a3a1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:35.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.618+0000 7f2c8cc00640 0 client.18438 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:59:35.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.618+0000 7f2ca904f740 1 --2- 172.21.15.83:0/1343871995 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e0a399860 0x560e0a3a1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:35.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.618+0000 7f2ca904f740 1 -- 172.21.15.83:0/1343871995 >> 172.21.15.83:0/1343871995 conn(0x560e0a395b80 msgr2=0x560e0a39b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:35.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.618+0000 7f2ca904f740 1 -- 172.21.15.83:0/1343871995 shutdown_connections 2024-04-24T08:59:35.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.618+0000 7f2ca904f740 1 -- 172.21.15.83:0/1343871995 wait complete. 2024-04-24T08:59:35.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.618+0000 7f2ca904f740 20 client.18438 trim_cache size 0 max 16384 2024-04-24T08:59:35.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.640+0000 7f2ca904f740 1 Processor -- start 2024-04-24T08:59:35.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.640+0000 7f2ca904f740 1 -- start start 2024-04-24T08:59:35.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.640+0000 7f2ca904f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e0a395b80 0x560e0a39c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:35.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.640+0000 7f2ca904f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e0a399860 0x560e0a3a1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:35.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.640+0000 7f2ca904f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e0a39a180 0x560e0a3bc860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:35.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.640+0000 7f2ca904f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca01b1890 con 0x560e0a399860 2024-04-24T08:59:35.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.640+0000 7f2ca904f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca01b33d0 con 0x560e0a39a180 2024-04-24T08:59:35.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.640+0000 7f2ca904f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca01b1f60 con 0x560e0a395b80 2024-04-24T08:59:35.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.641+0000 7f2c96a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e0a399860 0x560e0a3a1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:35.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.641+0000 7f2c96a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e0a399860 0x560e0a3a1ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44114/0 (socket says 172.21.15.83:44114) 2024-04-24T08:59:35.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.641+0000 7f2c96a00640 1 -- 172.21.15.83:0/2083937861 learned_addr learned my addr 172.21.15.83:0/2083937861 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:59:35.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.641+0000 7f2c9f400640 1 --2- 172.21.15.83:0/2083937861 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e0a395b80 0x560e0a39c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:35.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.641+0000 7f2c9fe00640 1 --2- 172.21.15.83:0/2083937861 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e0a39a180 0x560e0a3bc860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:35.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.641+0000 7f2c96a00640 1 -- 172.21.15.83:0/2083937861 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e0a395b80 msgr2=0x560e0a39c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:35.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.641+0000 7f2c96a00640 1 --2- 172.21.15.83:0/2083937861 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e0a395b80 0x560e0a39c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:35.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.641+0000 7f2c96a00640 1 -- 172.21.15.83:0/2083937861 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e0a39a180 msgr2=0x560e0a3bc860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:35.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.641+0000 7f2c96a00640 1 --2- 172.21.15.83:0/2083937861 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e0a39a180 0x560e0a3bc860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:35.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.641+0000 7f2c96a00640 1 -- 172.21.15.83:0/2083937861 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e0a0113f0 con 0x560e0a399860 2024-04-24T08:59:35.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.641+0000 7f2c9f400640 1 --2- 172.21.15.83:0/2083937861 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e0a395b80 0x560e0a39c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:59:35.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.642+0000 7f2c96a00640 1 --2- 172.21.15.83:0/2083937861 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e0a399860 0x560e0a3a1ed0 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f2ca019c100 tx=0x7f2c88014070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1806a59b58aacbbf server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:35.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.642+0000 7f2c8f400640 1 -- 172.21.15.83:0/2083937861 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c8801f620 con 0x560e0a399860 2024-04-24T08:59:35.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.642+0000 7f2c8f400640 1 -- 172.21.15.83:0/2083937861 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2c8801fc70 con 0x560e0a399860 2024-04-24T08:59:35.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.642+0000 7f2c8f400640 1 -- 172.21.15.83:0/2083937861 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c88027bb0 con 0x560e0a399860 2024-04-24T08:59:35.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.643+0000 7f2ca904f740 1 -- 172.21.15.83:0/2083937861 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e0a399860 msgr2=0x560e0a3a1ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:35.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.643+0000 7f2ca904f740 1 --2- 172.21.15.83:0/2083937861 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e0a399860 0x560e0a3a1ed0 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f2ca019c100 tx=0x7f2c88014070 comp rx=0 tx=0).stop 2024-04-24T08:59:35.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.643+0000 7f2ca904f740 1 -- 172.21.15.83:0/2083937861 shutdown_connections 2024-04-24T08:59:35.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.643+0000 7f2ca904f740 1 --2- 172.21.15.83:0/2083937861 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e0a39a180 0x560e0a3bc860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:35.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.643+0000 7f2ca904f740 1 --2- 172.21.15.83:0/2083937861 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e0a399860 0x560e0a3a1ed0 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:35.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.643+0000 7f2ca904f740 1 --2- 172.21.15.83:0/2083937861 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e0a395b80 0x560e0a39c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:35.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.643+0000 7f2ca904f740 1 -- 172.21.15.83:0/2083937861 >> 172.21.15.83:0/2083937861 conn(0x560e0a3a0440 msgr2=0x560e0a3a17c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:35.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.643+0000 7f2ca904f740 1 -- 172.21.15.83:0/2083937861 shutdown_connections 2024-04-24T08:59:35.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.643+0000 7f2ca904f740 1 -- 172.21.15.83:0/2083937861 wait complete. 2024-04-24T08:59:35.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.644+0000 7f2ca904f740 1 -- start start 2024-04-24T08:59:35.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.644+0000 7f2ca904f740 1 Processor -- start 2024-04-24T08:59:35.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.644+0000 7f2ca904f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e0a395b80 0x560e0a39ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:35.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.644+0000 7f2ca904f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e0a399860 0x560e0a39f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:35.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.644+0000 7f2ca904f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e0a39a180 0x560e0a41a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:35.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.644+0000 7f2ca904f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca01b2d00 con 0x560e0a399860 2024-04-24T08:59:35.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.644+0000 7f2ca904f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca01b2630 con 0x560e0a395b80 2024-04-24T08:59:35.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.644+0000 7f2ca904f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca01af890 con 0x560e0a39a180 2024-04-24T08:59:35.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.644+0000 7f2ca904f740 10 client.0 _ll_register_callbacks cb 0x7f2c98098d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:59:35.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.645+0000 7f2c96a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e0a399860 0x560e0a39f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:35.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.645+0000 7f2c96a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e0a399860 0x560e0a39f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44120/0 (socket says 172.21.15.83:44120) 2024-04-24T08:59:35.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.645+0000 7f2c96a00640 1 -- 172.21.15.83:0/3400915790 learned_addr learned my addr 172.21.15.83:0/3400915790 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:59:35.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.645+0000 7f2c9fe00640 1 --2- 172.21.15.83:0/3400915790 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e0a39a180 0x560e0a41a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:35.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.645+0000 7f2c9f400640 1 --2- 172.21.15.83:0/3400915790 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e0a395b80 0x560e0a39ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:35.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.645+0000 7f2c96a00640 1 -- 172.21.15.83:0/3400915790 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e0a39a180 msgr2=0x560e0a41a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:35.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.645+0000 7f2c96a00640 1 --2- 172.21.15.83:0/3400915790 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e0a39a180 0x560e0a41a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:35.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.645+0000 7f2c96a00640 1 -- 172.21.15.83:0/3400915790 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e0a395b80 msgr2=0x560e0a39ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:35.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.645+0000 7f2c96a00640 1 --2- 172.21.15.83:0/3400915790 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e0a395b80 0x560e0a39ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:35.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.645+0000 7f2c96a00640 1 -- 172.21.15.83:0/3400915790 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e09fd10f0 con 0x560e0a399860 2024-04-24T08:59:35.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.645+0000 7f2c96a00640 1 --2- 172.21.15.83:0/3400915790 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e0a399860 0x560e0a39f120 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f2c88005d00 tx=0x7f2c88027fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a522f775a0bf5f0b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:35.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.645+0000 7f2c8cc00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:59:35.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.646+0000 7f2c8cc00640 1 -- 172.21.15.83:0/3400915790 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c88005440 con 0x560e0a399860 2024-04-24T08:59:35.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.646+0000 7f2ca904f740 1 -- 172.21.15.83:0/3400915790 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c8801fc70 con 0x560e0a399860 2024-04-24T08:59:35.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.646+0000 7f2c8cc00640 1 -- 172.21.15.83:0/3400915790 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7f2c8801f420 con 0x560e0a399860 2024-04-24T08:59:35.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.646+0000 7f2c8cc00640 1 -- 172.21.15.83:0/3400915790 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f2c880055e0 con 0x560e0a399860 2024-04-24T08:59:35.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.646+0000 7f2ca904f740 10 client.18444 fetch_fsmap learned FSMap version 35 2024-04-24T08:59:35.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.646+0000 7f2ca904f740 1 -- 172.21.15.83:0/3400915790 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x560e0a2c4810 con 0x560e0a399860 2024-04-24T08:59:35.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.646+0000 7f2c8cc00640 1 -- 172.21.15.83:0/3400915790 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2c8801fc70 con 0x560e0a399860 2024-04-24T08:59:35.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.646+0000 7f2c8cc00640 1 -- 172.21.15.83:0/3400915790 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7f2c8800f040 con 0x560e0a399860 2024-04-24T08:59:35.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.647+0000 7f2ca904f740 10 client.18444 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T08:59:35.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.647+0000 7f2ca904f740 10 client.18444 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:59:35.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.647+0000 7f2ca904f740 20 client.18444 populate_metadata read hostname 'smithi083' 2024-04-24T08:59:35.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.647+0000 7f2ca904f740 1 --2- 172.21.15.83:0/3400915790 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x560e0a3a4850 0x560e0a41dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:35.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.647+0000 7f2ca904f740 4 client.18444 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T08:59:35.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.647+0000 7f2ca904f740 1 -- 172.21.15.83:0/3400915790 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560e0a306a80 con 0x560e0a3a4850 2024-04-24T08:59:35.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.647+0000 7f2c9f400640 1 --2- 172.21.15.83:0/3400915790 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x560e0a3a4850 0x560e0a41dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:35.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.648+0000 7f2c9f400640 1 --2- 172.21.15.83:0/3400915790 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x560e0a3a4850 0x560e0a41dd10 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aed303ee0404ad50 server_cookie=63da980238faa53f in_seq=0 out_seq=0 2024-04-24T08:59:35.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.648+0000 7f2c8cc00640 10 client.18444 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:59:35.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.650+0000 7f2c8cc00640 1 -- 172.21.15.83:0/3400915790 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x560e0a306a80 con 0x560e0a3a4850 2024-04-24T08:59:35.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.650+0000 7f2c8cc00640 10 client.18444 handle_command_reply: tid=0 2024-04-24T08:59:35.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.650+0000 7f2ca904f740 1 client.18444 shutdown 2024-04-24T08:59:35.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.650+0000 7f2ca904f740 1 -- 172.21.15.83:0/3400915790 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e0a399860 msgr2=0x560e0a39f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:35.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.650+0000 7f2ca904f740 1 --2- 172.21.15.83:0/3400915790 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e0a399860 0x560e0a39f120 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f2c88005d00 tx=0x7f2c88027fb0 comp rx=0 tx=0).stop 2024-04-24T08:59:35.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.650+0000 7f2ca904f740 1 -- 172.21.15.83:0/3400915790 shutdown_connections 2024-04-24T08:59:35.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.650+0000 7f2ca904f740 1 --2- 172.21.15.83:0/3400915790 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x560e0a3a4850 0x560e0a41dd10 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:35.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.650+0000 7f2ca904f740 1 --2- 172.21.15.83:0/3400915790 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e0a39a180 0x560e0a41a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:35.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.650+0000 7f2ca904f740 1 --2- 172.21.15.83:0/3400915790 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e0a399860 0x560e0a39f120 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:35.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.650+0000 7f2ca904f740 1 --2- 172.21.15.83:0/3400915790 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e0a395b80 0x560e0a39ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:35.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.650+0000 7f2c8cc00640 0 client.18444 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:59:35.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.650+0000 7f2ca904f740 1 -- 172.21.15.83:0/3400915790 >> 172.21.15.83:0/3400915790 conn(0x560e0a3a0440 msgr2=0x560e0a3a2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:35.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.650+0000 7f2ca904f740 1 -- 172.21.15.83:0/3400915790 shutdown_connections 2024-04-24T08:59:35.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.650+0000 7f2ca904f740 1 -- 172.21.15.83:0/3400915790 wait complete. 2024-04-24T08:59:35.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.650+0000 7f2ca904f740 20 client.18444 trim_cache size 0 max 16384 2024-04-24T08:59:35.651 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T08:59:35.652 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-04-24T08:59:35.652 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-04-24T08:59:35.652 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T08:59:35.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.651+0000 7f2ca6200640 1 -- 172.21.15.83:0/4091079570 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f2c840a3d20 msgr2=0x7f2c840a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:35.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.651+0000 7f2ca6200640 1 --2- 172.21.15.83:0/4091079570 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f2c840a3d20 0x7f2c840a61e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x560e0a396d10 tx=0x7f2c880046c0 comp rx=0 tx=0).stop 2024-04-24T08:59:35.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.651+0000 7f2ca6200640 1 -- 172.21.15.83:0/4091079570 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ca019a960 msgr2=0x7f2ca019ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:35.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.651+0000 7f2ca6200640 1 --2- 172.21.15.83:0/4091079570 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ca019a960 0x7f2ca019ae10 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f2c9800eaa0 tx=0x7f2c9800ef60 comp rx=0 tx=0).stop 2024-04-24T08:59:35.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.651+0000 7f2ca6200640 1 -- 172.21.15.83:0/4091079570 shutdown_connections 2024-04-24T08:59:35.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.651+0000 7f2ca6200640 1 --2- 172.21.15.83:0/4091079570 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f2c840a3d20 0x7f2c840a61e0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:35.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.651+0000 7f2ca6200640 1 --2- 172.21.15.83:0/4091079570 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ca019a960 0x7f2ca019ae10 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:35.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.651+0000 7f2ca6200640 1 --2- 172.21.15.83:0/4091079570 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2ca0109520 0x7f2ca01a0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:35.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.651+0000 7f2ca6200640 1 --2- 172.21.15.83:0/4091079570 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ca0104450 0x7f2ca01a0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:35.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.651+0000 7f2ca6200640 1 -- 172.21.15.83:0/4091079570 >> 172.21.15.83:0/4091079570 conn(0x7f2ca00ffc70 msgr2=0x7f2ca0101060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:35.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.651+0000 7f2ca6200640 1 -- 172.21.15.83:0/4091079570 shutdown_connections 2024-04-24T08:59:35.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.651+0000 7f2ca6200640 1 -- 172.21.15.83:0/4091079570 wait complete. 2024-04-24T08:59:35.668 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T08:59:35.668 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T08:59:35.729 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:59:35 smithi083 ceph-mon[27907]: pgmap v248: 129 pgs: 129 active+clean; 48 MiB data, 695 MiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 322 KiB/s wr, 51 op/s 2024-04-24T08:59:35.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.864+0000 7ff344e00640 1 -- 172.21.15.83:0/842229738 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3401084a0 msgr2=0x7ff340108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:35.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.864+0000 7ff344e00640 1 --2- 172.21.15.83:0/842229738 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3401084a0 0x7ff340108900 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7ff32c009900 tx=0x7ff32c02a480 comp rx=0 tx=0).stop 2024-04-24T08:59:35.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.865+0000 7ff344e00640 1 -- 172.21.15.83:0/842229738 shutdown_connections 2024-04-24T08:59:35.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.865+0000 7ff344e00640 1 --2- 172.21.15.83:0/842229738 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff340108e40 0x7ff3401152b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:35.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.865+0000 7ff344e00640 1 --2- 172.21.15.83:0/842229738 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3401084a0 0x7ff340108900 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:35.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.865+0000 7ff344e00640 1 --2- 172.21.15.83:0/842229738 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff34010d3c0 0x7ff34010d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:35.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.865+0000 7ff344e00640 1 -- 172.21.15.83:0/842229738 >> 172.21.15.83:0/842229738 conn(0x7ff340077b80 msgr2=0x7ff340079fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:35.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.865+0000 7ff344e00640 1 -- 172.21.15.83:0/842229738 shutdown_connections 2024-04-24T08:59:35.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.865+0000 7ff344e00640 1 -- 172.21.15.83:0/842229738 wait complete. 2024-04-24T08:59:35.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.866+0000 7ff344e00640 1 Processor -- start 2024-04-24T08:59:35.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.866+0000 7ff344e00640 1 -- start start 2024-04-24T08:59:35.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.866+0000 7ff344e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3401084a0 0x7ff3401a4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:35.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.866+0000 7ff344e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff340108e40 0x7ff3401a4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:35.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.866+0000 7ff344e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff34010d3c0 0x7ff34019ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:35.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.866+0000 7ff344e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff340118690 con 0x7ff3401084a0 2024-04-24T08:59:35.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.866+0000 7ff344e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff340118510 con 0x7ff340108e40 2024-04-24T08:59:35.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.866+0000 7ff344e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff340118810 con 0x7ff34010d3c0 2024-04-24T08:59:35.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.866+0000 7ff33f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3401084a0 0x7ff3401a4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:35.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.867+0000 7ff33f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3401084a0 0x7ff3401a4880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44148/0 (socket says 172.21.15.83:44148) 2024-04-24T08:59:35.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.867+0000 7ff33f400640 1 -- 172.21.15.83:0/1692481467 learned_addr learned my addr 172.21.15.83:0/1692481467 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:59:35.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.867+0000 7ff33ea00640 1 --2- 172.21.15.83:0/1692481467 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff340108e40 0x7ff3401a4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:35.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.867+0000 7ff33fe00640 1 --2- 172.21.15.83:0/1692481467 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff34010d3c0 0x7ff34019ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:35.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.867+0000 7ff33f400640 1 -- 172.21.15.83:0/1692481467 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff34010d3c0 msgr2=0x7ff34019ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:35.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.867+0000 7ff33f400640 1 --2- 172.21.15.83:0/1692481467 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff34010d3c0 0x7ff34019ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:35.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.867+0000 7ff33f400640 1 -- 172.21.15.83:0/1692481467 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff340108e40 msgr2=0x7ff3401a4dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:35.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.867+0000 7ff33f400640 1 --2- 172.21.15.83:0/1692481467 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff340108e40 0x7ff3401a4dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:35.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.867+0000 7ff33f400640 1 -- 172.21.15.83:0/1692481467 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff34019f1b0 con 0x7ff3401084a0 2024-04-24T08:59:35.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.867+0000 7ff33ea00640 1 --2- 172.21.15.83:0/1692481467 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff340108e40 0x7ff3401a4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:59:35.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.867+0000 7ff33f400640 1 --2- 172.21.15.83:0/1692481467 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3401084a0 0x7ff3401a4880 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7ff33000d950 tx=0x7ff33000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:35.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.868+0000 7ff337e00640 1 -- 172.21.15.83:0/1692481467 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff330017070 con 0x7ff3401084a0 2024-04-24T08:59:35.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.868+0000 7ff337e00640 1 -- 172.21.15.83:0/1692481467 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff330009c30 con 0x7ff3401084a0 2024-04-24T08:59:35.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.868+0000 7ff337e00640 1 -- 172.21.15.83:0/1692481467 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff33000f620 con 0x7ff3401084a0 2024-04-24T08:59:35.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.868+0000 7ff344e00640 1 -- 172.21.15.83:0/1692481467 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff34019f4a0 con 0x7ff3401084a0 2024-04-24T08:59:35.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.868+0000 7ff344e00640 1 -- 172.21.15.83:0/1692481467 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3401ab740 con 0x7ff3401084a0 2024-04-24T08:59:35.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.869+0000 7ff344e00640 1 -- 172.21.15.83:0/1692481467 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff340069190 con 0x7ff3401084a0 2024-04-24T08:59:35.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.871+0000 7ff337e00640 1 -- 172.21.15.83:0/1692481467 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff3300109a0 con 0x7ff3401084a0 2024-04-24T08:59:35.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.873+0000 7ff337e00640 1 --2- 172.21.15.83:0/1692481467 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff3240a3da0 0x7ff3240a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:35.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.873+0000 7ff337e00640 1 -- 172.21.15.83:0/1692481467 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7ff3300d07d0 con 0x7ff3401084a0 2024-04-24T08:59:35.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.873+0000 7ff33ea00640 1 --2- 172.21.15.83:0/1692481467 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff3240a3da0 0x7ff3240a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:35.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.874+0000 7ff33ea00640 1 --2- 172.21.15.83:0/1692481467 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff3240a3da0 0x7ff3240a6260 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff32c000c00 tx=0x7ff32c031400 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:35.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:35.878+0000 7ff337e00640 1 -- 172.21.15.83:0/1692481467 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff330009de0 con 0x7ff3401084a0 2024-04-24T08:59:35.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:59:35 smithi138 ceph-mon[31441]: pgmap v248: 129 pgs: 129 active+clean; 48 MiB data, 695 MiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 322 KiB/s wr, 51 op/s 2024-04-24T08:59:35.956 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:59:35 smithi189 ceph-mon[31791]: pgmap v248: 129 pgs: 129 active+clean; 48 MiB data, 695 MiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 322 KiB/s wr, 51 op/s 2024-04-24T08:59:36.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.149+0000 7ff344e00640 1 -- 172.21.15.83:0/1692481467 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff34002d620 con 0x7ff3401084a0 2024-04-24T08:59:36.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.151+0000 7ff337e00640 1 -- 172.21.15.83:0/1692481467 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11944 (secure 0 0 0) 0x7ff33009ca50 con 0x7ff3401084a0 2024-04-24T08:59:36.151 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T08:59:36.151 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/358872257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":358872257},{"type":"v1","addr":"172.21.15.83:6835","nonce":358872257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3157070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3157070851},{"type":"v1","addr":"172.21.15.83:6837","nonce":3157070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":35,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T08:48:29.422576+0000","modified":"2024-04-24T08:59:06.888262+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34520,"mds_1":34493,"mds_2":34451,"mds_3":34427,"mds_4":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14658":{"gid":14658,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6839/4085470844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":4085470844},{"type":"v1","addr":"172.21.15.83:6839","nonce":4085470844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14676":{"gid":14676,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6841/2634754210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":2634754210},{"type":"v1","addr":"172.21.15.83:6841","nonce":2634754210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24382":{"gid":24382,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/1859921310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1859921310},{"type":"v1","addr":"172.21.15.189:6833","nonce":1859921310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/2241923197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2241923197},{"type":"v1","addr":"172.21.15.189:6835","nonce":2241923197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.189:6839/2548965336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":2548965336},{"type":"v1","addr":"172.21.15.189:6839","nonce":2548965336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26686":{"gid":26686,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6837/1678817161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":1678817161},{"type":"v1","addr":"172.21.15.189:6837","nonce":1678817161}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.138:6833/1157967658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1157967658},{"type":"v1","addr":"172.21.15.138:6833","nonce":1157967658}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.138:6835/1909219758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1909219758},{"type":"v1","addr":"172.21.15.138:6835","nonce":1909219758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34493":{"gid":34493,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.138:6837/900566754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":900566754},{"type":"v1","addr":"172.21.15.138:6837","nonce":900566754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34520":{"gid":34520,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.138:6839/2236932393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2236932393},{"type":"v1","addr":"172.21.15.138:6839","nonce":2236932393}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34520,"qdb_cluster":[34520,24454,34427,34493,34451]},"id":1}]} 2024-04-24T08:59:36.151 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 35 2024-04-24T08:59:36.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.154+0000 7ff344e00640 1 -- 172.21.15.83:0/1692481467 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff3240a3da0 msgr2=0x7ff3240a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:36.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.154+0000 7ff344e00640 1 --2- 172.21.15.83:0/1692481467 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff3240a3da0 0x7ff3240a6260 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff32c000c00 tx=0x7ff32c031400 comp rx=0 tx=0).stop 2024-04-24T08:59:36.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.154+0000 7ff344e00640 1 -- 172.21.15.83:0/1692481467 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3401084a0 msgr2=0x7ff3401a4880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:36.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.154+0000 7ff344e00640 1 --2- 172.21.15.83:0/1692481467 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3401084a0 0x7ff3401a4880 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7ff33000d950 tx=0x7ff33000de10 comp rx=0 tx=0).stop 2024-04-24T08:59:36.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.154+0000 7ff344e00640 1 -- 172.21.15.83:0/1692481467 shutdown_connections 2024-04-24T08:59:36.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.154+0000 7ff344e00640 1 --2- 172.21.15.83:0/1692481467 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff3240a3da0 0x7ff3240a6260 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:36.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.154+0000 7ff344e00640 1 --2- 172.21.15.83:0/1692481467 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff34010d3c0 0x7ff34019ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:36.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.154+0000 7ff344e00640 1 --2- 172.21.15.83:0/1692481467 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff340108e40 0x7ff3401a4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:36.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.154+0000 7ff344e00640 1 --2- 172.21.15.83:0/1692481467 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3401084a0 0x7ff3401a4880 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:36.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.154+0000 7ff344e00640 1 -- 172.21.15.83:0/1692481467 >> 172.21.15.83:0/1692481467 conn(0x7ff340077b80 msgr2=0x7ff340078400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:36.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.155+0000 7ff344e00640 1 -- 172.21.15.83:0/1692481467 shutdown_connections 2024-04-24T08:59:36.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.155+0000 7ff344e00640 1 -- 172.21.15.83:0/1692481467 wait complete. 2024-04-24T08:59:36.173 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-24T08:59:36.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.381+0000 7fcb28c00640 1 -- 172.21.15.83:0/1102255484 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb24108580 msgr2=0x7fcb24108a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:36.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.381+0000 7fcb28c00640 1 --2- 172.21.15.83:0/1102255484 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb24108580 0x7fcb24108a00 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7fcb10009930 tx=0x7fcb1002a420 comp rx=0 tx=0).stop 2024-04-24T08:59:36.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.381+0000 7fcb28c00640 1 -- 172.21.15.83:0/1102255484 shutdown_connections 2024-04-24T08:59:36.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.381+0000 7fcb28c00640 1 --2- 172.21.15.83:0/1102255484 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcb24108f40 0x7fcb241153b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:36.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.381+0000 7fcb28c00640 1 --2- 172.21.15.83:0/1102255484 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb24108580 0x7fcb24108a00 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:36.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.381+0000 7fcb28c00640 1 --2- 172.21.15.83:0/1102255484 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcb2410d500 0x7fcb2410d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:36.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.381+0000 7fcb28c00640 1 -- 172.21.15.83:0/1102255484 >> 172.21.15.83:0/1102255484 conn(0x7fcb24077e20 msgr2=0x7fcb24106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:36.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.381+0000 7fcb28c00640 1 -- 172.21.15.83:0/1102255484 shutdown_connections 2024-04-24T08:59:36.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.381+0000 7fcb28c00640 1 -- 172.21.15.83:0/1102255484 wait complete. 2024-04-24T08:59:36.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.382+0000 7fcb28c00640 1 Processor -- start 2024-04-24T08:59:36.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.382+0000 7fcb28c00640 1 -- start start 2024-04-24T08:59:36.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.383+0000 7fcb28c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcb24108580 0x7fcb241a4930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:36.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.383+0000 7fcb28c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcb24108f40 0x7fcb241a4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:36.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.383+0000 7fcb28c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb2410d500 0x7fcb2419eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:36.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.383+0000 7fcb28c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcb241187d0 con 0x7fcb2410d500 2024-04-24T08:59:36.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.383+0000 7fcb28c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcb24118650 con 0x7fcb24108580 2024-04-24T08:59:36.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.383+0000 7fcb28c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fcb24118950 con 0x7fcb24108f40 2024-04-24T08:59:36.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.383+0000 7fcb23e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb2410d500 0x7fcb2419eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:36.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.383+0000 7fcb23e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb2410d500 0x7fcb2419eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44166/0 (socket says 172.21.15.83:44166) 2024-04-24T08:59:36.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.383+0000 7fcb23e00640 1 -- 172.21.15.83:0/2901327043 learned_addr learned my addr 172.21.15.83:0/2901327043 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:59:36.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.383+0000 7fcb23400640 1 --2- 172.21.15.83:0/2901327043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcb24108580 0x7fcb241a4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:36.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.383+0000 7fcb22a00640 1 --2- 172.21.15.83:0/2901327043 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcb24108f40 0x7fcb241a4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:36.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.384+0000 7fcb23e00640 1 -- 172.21.15.83:0/2901327043 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcb24108f40 msgr2=0x7fcb241a4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:36.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.384+0000 7fcb23e00640 1 --2- 172.21.15.83:0/2901327043 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcb24108f40 0x7fcb241a4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:36.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.384+0000 7fcb23e00640 1 -- 172.21.15.83:0/2901327043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcb24108580 msgr2=0x7fcb241a4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:36.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.384+0000 7fcb23e00640 1 --2- 172.21.15.83:0/2901327043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcb24108580 0x7fcb241a4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:36.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.384+0000 7fcb23e00640 1 -- 172.21.15.83:0/2901327043 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb2419f2b0 con 0x7fcb2410d500 2024-04-24T08:59:36.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.384+0000 7fcb23400640 1 --2- 172.21.15.83:0/2901327043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcb24108580 0x7fcb241a4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:59:36.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.384+0000 7fcb23e00640 1 --2- 172.21.15.83:0/2901327043 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb2410d500 0x7fcb2419eb30 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c00cab0 tx=0x7fcb1c00cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:36.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.384+0000 7fcb1be00640 1 -- 172.21.15.83:0/2901327043 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb1c00ae20 con 0x7fcb2410d500 2024-04-24T08:59:36.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.385+0000 7fcb28c00640 1 -- 172.21.15.83:0/2901327043 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb2419f5a0 con 0x7fcb2410d500 2024-04-24T08:59:36.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.385+0000 7fcb1be00640 1 -- 172.21.15.83:0/2901327043 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcb1c00f900 con 0x7fcb2410d500 2024-04-24T08:59:36.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.385+0000 7fcb1be00640 1 -- 172.21.15.83:0/2901327043 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb1c018a00 con 0x7fcb2410d500 2024-04-24T08:59:36.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.385+0000 7fcb28c00640 1 -- 172.21.15.83:0/2901327043 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb241ab7f0 con 0x7fcb2410d500 2024-04-24T08:59:36.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.388+0000 7fcb1be00640 1 -- 172.21.15.83:0/2901327043 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fcb1c00fab0 con 0x7fcb2410d500 2024-04-24T08:59:36.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.389+0000 7fcb2cf99740 1 Processor -- start 2024-04-24T08:59:36.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.389+0000 7fcb2cf99740 1 -- start start 2024-04-24T08:59:36.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.389+0000 7fcb2cf99740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f66e900b40 0x55f66e900f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:36.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.389+0000 7fcb2cf99740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f66e901460 0x55f66e90b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:36.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.389+0000 7fcb2cf99740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f66e90bb30 0x55f66e90df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:36.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.389+0000 7fcb2cf99740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55f66e8661d0 con 0x55f66e901460 2024-04-24T08:59:36.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.389+0000 7fcb2cf99740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f66e80c450 con 0x55f66e90bb30 2024-04-24T08:59:36.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.389+0000 7fcb2cf99740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55f66e862190 con 0x55f66e900b40 2024-04-24T08:59:36.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.390+0000 7fcb23400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f66e901460 0x55f66e90b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:36.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.390+0000 7fcb23400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f66e901460 0x55f66e90b5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44172/0 (socket says 172.21.15.83:44172) 2024-04-24T08:59:36.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.390+0000 7fcb23400640 1 -- 172.21.15.83:0/1818676051 learned_addr learned my addr 172.21.15.83:0/1818676051 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:59:36.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.390+0000 7fcb22a00640 1 --2- 172.21.15.83:0/1818676051 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f66e900b40 0x55f66e900f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:36.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.390+0000 7fcb23400640 1 -- 172.21.15.83:0/1818676051 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f66e900b40 msgr2=0x55f66e900f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:36.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.390+0000 7fcb23400640 1 --2- 172.21.15.83:0/1818676051 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f66e900b40 0x55f66e900f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:36.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.390+0000 7fcb22a00640 1 --2- 172.21.15.83:0/1818676051 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f66e90bb30 0x55f66e90df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:36.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.390+0000 7fcb23400640 1 -- 172.21.15.83:0/1818676051 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f66e90bb30 msgr2=0x55f66e90df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:36.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.390+0000 7fcb23400640 1 --2- 172.21.15.83:0/1818676051 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f66e90bb30 0x55f66e90df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:36.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.390+0000 7fcb23400640 1 -- 172.21.15.83:0/1818676051 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f66e8c9310 con 0x55f66e901460 2024-04-24T08:59:36.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.390+0000 7fcb1be00640 1 --2- 172.21.15.83:0/2901327043 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fcb080a3d20 0x7fcb080a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:36.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.390+0000 7fcb1be00640 1 -- 172.21.15.83:0/2901327043 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7fcb1c0d14e0 con 0x7fcb2410d500 2024-04-24T08:59:36.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.391+0000 7fcb23400640 1 --2- 172.21.15.83:0/1818676051 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f66e901460 0x55f66e90b5f0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7fcb1400bcb0 tx=0x7fcb1400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af4b946658cc1a21 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:36.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.391+0000 7fcb22a00640 1 --2- 172.21.15.83:0/2901327043 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fcb080a3d20 0x7fcb080a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:36.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.391+0000 7fcb0fe00640 1 -- 172.21.15.83:0/1818676051 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb14015070 con 0x55f66e901460 2024-04-24T08:59:36.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.391+0000 7fcb0fe00640 1 -- 172.21.15.83:0/1818676051 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcb140118f0 con 0x55f66e901460 2024-04-24T08:59:36.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.391+0000 7fcb0fe00640 1 -- 172.21.15.83:0/1818676051 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb14010920 con 0x55f66e901460 2024-04-24T08:59:36.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.392+0000 7fcb2cf99740 1 -- 172.21.15.83:0/1818676051 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f66e901460 msgr2=0x55f66e90b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:36.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.392+0000 7fcb22a00640 1 --2- 172.21.15.83:0/2901327043 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fcb080a3d20 0x7fcb080a61e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fcb10006010 tx=0x7fcb10031400 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:36.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.392+0000 7fcb2cf99740 1 --2- 172.21.15.83:0/1818676051 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f66e901460 0x55f66e90b5f0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7fcb1400bcb0 tx=0x7fcb1400f730 comp rx=0 tx=0).stop 2024-04-24T08:59:36.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.392+0000 7fcb2cf99740 1 -- 172.21.15.83:0/1818676051 shutdown_connections 2024-04-24T08:59:36.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.392+0000 7fcb2cf99740 1 --2- 172.21.15.83:0/1818676051 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f66e90bb30 0x55f66e90df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:36.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.392+0000 7fcb2cf99740 1 --2- 172.21.15.83:0/1818676051 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f66e901460 0x55f66e90b5f0 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:36.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.392+0000 7fcb2cf99740 1 --2- 172.21.15.83:0/1818676051 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f66e900b40 0x55f66e900f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:36.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.392+0000 7fcb2cf99740 1 -- 172.21.15.83:0/1818676051 >> 172.21.15.83:0/1818676051 conn(0x55f66e8fce60 msgr2=0x55f66e8fd240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:36.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.392+0000 7fcb2cf99740 1 -- 172.21.15.83:0/1818676051 shutdown_connections 2024-04-24T08:59:36.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.392+0000 7fcb2cf99740 1 -- 172.21.15.83:0/1818676051 wait complete. 2024-04-24T08:59:36.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.392+0000 7fcb2cf99740 1 -- start start 2024-04-24T08:59:36.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.393+0000 7fcb2cf99740 1 Processor -- start 2024-04-24T08:59:36.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.393+0000 7fcb2cf99740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f66e900b40 0x55f66e90b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:36.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.393+0000 7fcb2cf99740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f66e901460 0x55f66e908880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:36.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.393+0000 7fcb2cf99740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f66e90bb30 0x55f66e908fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:36.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.393+0000 7fcb2cf99740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55f66e90e9c0 con 0x55f66e900b40 2024-04-24T08:59:36.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.393+0000 7fcb2cf99740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f66e81f810 con 0x55f66e901460 2024-04-24T08:59:36.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.393+0000 7fcb2cf99740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55f66e90eb40 con 0x55f66e90bb30 2024-04-24T08:59:36.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.393+0000 7fcb2cf99740 10 client.0 _ll_register_callbacks cb 0x55f66e7ed140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:59:36.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.393+0000 7fcb23400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f66e900b40 0x55f66e90b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:36.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.393+0000 7fcb23400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f66e900b40 0x55f66e90b590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44176/0 (socket says 172.21.15.83:44176) 2024-04-24T08:59:36.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.393+0000 7fcb23400640 1 -- 172.21.15.83:0/4092612432 learned_addr learned my addr 172.21.15.83:0/4092612432 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:59:36.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.394+0000 7fcb23e00640 1 --2- 172.21.15.83:0/4092612432 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f66e90bb30 0x55f66e908fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:36.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.394+0000 7fcb23400640 1 -- 172.21.15.83:0/4092612432 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f66e90bb30 msgr2=0x55f66e908fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:36.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.394+0000 7fcb23400640 1 --2- 172.21.15.83:0/4092612432 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f66e90bb30 0x55f66e908fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:36.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.394+0000 7fcb23400640 1 -- 172.21.15.83:0/4092612432 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f66e901460 msgr2=0x55f66e908880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:36.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.394+0000 7fcb23400640 1 --2- 172.21.15.83:0/4092612432 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f66e901460 0x55f66e908880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:36.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.394+0000 7fcb23400640 1 -- 172.21.15.83:0/4092612432 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f66e8303f0 con 0x55f66e900b40 2024-04-24T08:59:36.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.394+0000 7fcb23e00640 1 --2- 172.21.15.83:0/4092612432 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f66e90bb30 0x55f66e908fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:59:36.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.394+0000 7fcb23400640 1 --2- 172.21.15.83:0/4092612432 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f66e900b40 0x55f66e90b590 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7fcb1400ba00 tx=0x7fcb14011440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1679f2c9667da374 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:36.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.394+0000 7fcb0cc00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:59:36.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.394+0000 7fcb0cc00640 1 -- 172.21.15.83:0/4092612432 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb14015040 con 0x55f66e900b40 2024-04-24T08:59:36.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.394+0000 7fcb0cc00640 1 -- 172.21.15.83:0/4092612432 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7fcb14011ca0 con 0x55f66e900b40 2024-04-24T08:59:36.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.394+0000 7fcb2cf99740 1 -- 172.21.15.83:0/4092612432 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb140118f0 con 0x55f66e900b40 2024-04-24T08:59:36.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.395+0000 7fcb0cc00640 1 -- 172.21.15.83:0/4092612432 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fcb14004020 con 0x55f66e900b40 2024-04-24T08:59:36.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.395+0000 7fcb2cf99740 10 client.18462 fetch_fsmap learned FSMap version 35 2024-04-24T08:59:36.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.395+0000 7fcb2cf99740 1 -- 172.21.15.83:0/4092612432 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55f66e5783f0 con 0x55f66e900b40 2024-04-24T08:59:36.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.395+0000 7fcb0cc00640 1 -- 172.21.15.83:0/4092612432 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcb140118f0 con 0x55f66e900b40 2024-04-24T08:59:36.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.395+0000 7fcb0cc00640 1 -- 172.21.15.83:0/4092612432 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7fcb14022030 con 0x55f66e900b40 2024-04-24T08:59:36.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.395+0000 7fcb2cf99740 10 client.18462 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T08:59:36.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.395+0000 7fcb2cf99740 10 client.18462 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-24T08:59:36.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.395+0000 7fcb2cf99740 20 client.18462 populate_metadata read hostname 'smithi083' 2024-04-24T08:59:36.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.395+0000 7fcb2cf99740 1 --2- 172.21.15.83:0/4092612432 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55f66e91bcf0 0x55f66e91e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:36.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.395+0000 7fcb2cf99740 4 client.18462 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:59:36.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.395+0000 7fcb2cf99740 1 -- 172.21.15.83:0/4092612432 --> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f66e873bf0 con 0x55f66e91bcf0 2024-04-24T08:59:36.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.396+0000 7fcb22a00640 1 --2- 172.21.15.83:0/4092612432 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55f66e91bcf0 0x55f66e91e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:36.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.397+0000 7fcb22a00640 1 --2- 172.21.15.83:0/4092612432 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55f66e91bcf0 0x55f66e91e0e0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f714a5747ca640c1 server_cookie=aaf4d1d8b2292ea1 in_seq=0 out_seq=0 2024-04-24T08:59:36.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.397+0000 7fcb0cc00640 10 client.18462 ms_handle_connect on v2:172.21.15.189:6838/2548965336 2024-04-24T08:59:36.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.399+0000 7fcb0cc00640 1 -- 172.21.15.83:0/4092612432 <== mds.4 v2:172.21.15.189:6838/2548965336 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f66e873bf0 con 0x55f66e91bcf0 2024-04-24T08:59:36.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.399+0000 7fcb0cc00640 10 client.18462 handle_command_reply: tid=0 2024-04-24T08:59:36.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.399+0000 7fcb2cf99740 1 client.18462 shutdown 2024-04-24T08:59:36.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.399+0000 7fcb2cf99740 1 -- 172.21.15.83:0/4092612432 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f66e900b40 msgr2=0x55f66e90b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:36.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.399+0000 7fcb2cf99740 1 --2- 172.21.15.83:0/4092612432 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f66e900b40 0x55f66e90b590 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7fcb1400ba00 tx=0x7fcb14011440 comp rx=0 tx=0).stop 2024-04-24T08:59:36.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.399+0000 7fcb2cf99740 1 -- 172.21.15.83:0/4092612432 shutdown_connections 2024-04-24T08:59:36.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.399+0000 7fcb2cf99740 1 --2- 172.21.15.83:0/4092612432 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55f66e91bcf0 0x55f66e91e0e0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:36.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.399+0000 7fcb2cf99740 1 --2- 172.21.15.83:0/4092612432 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f66e90bb30 0x55f66e908fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:36.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.399+0000 7fcb2cf99740 1 --2- 172.21.15.83:0/4092612432 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f66e901460 0x55f66e908880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:36.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.399+0000 7fcb0cc00640 0 client.18462 ms_handle_reset on v2:172.21.15.189:6838/2548965336 2024-04-24T08:59:36.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.399+0000 7fcb2cf99740 1 --2- 172.21.15.83:0/4092612432 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f66e900b40 0x55f66e90b590 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:36.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.399+0000 7fcb2cf99740 1 -- 172.21.15.83:0/4092612432 >> 172.21.15.83:0/4092612432 conn(0x55f66e8fce60 msgr2=0x55f66e902b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:36.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.400+0000 7fcb2cf99740 1 -- 172.21.15.83:0/4092612432 shutdown_connections 2024-04-24T08:59:36.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.400+0000 7fcb2cf99740 1 -- 172.21.15.83:0/4092612432 wait complete. 2024-04-24T08:59:36.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.400+0000 7fcb2cf99740 20 client.18462 trim_cache size 0 max 16384 2024-04-24T08:59:36.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.415+0000 7fcb2cf99740 1 Processor -- start 2024-04-24T08:59:36.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.416+0000 7fcb2cf99740 1 -- start start 2024-04-24T08:59:36.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.416+0000 7fcb2cf99740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f66e900b40 0x55f66e9032a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:36.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.416+0000 7fcb2cf99740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f66e901460 0x55f66e908160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:36.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.416+0000 7fcb2cf99740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f66e90bb30 0x55f66e90b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:36.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.416+0000 7fcb2cf99740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcb241b59d0 con 0x55f66e901460 2024-04-24T08:59:36.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.416+0000 7fcb2cf99740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcb241b7510 con 0x55f66e90bb30 2024-04-24T08:59:36.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.416+0000 7fcb2cf99740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fcb241b60a0 con 0x55f66e900b40 2024-04-24T08:59:36.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.416+0000 7fcb22a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f66e901460 0x55f66e908160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:36.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.416+0000 7fcb22a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f66e901460 0x55f66e908160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44178/0 (socket says 172.21.15.83:44178) 2024-04-24T08:59:36.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.416+0000 7fcb22a00640 1 -- 172.21.15.83:0/2004034677 learned_addr learned my addr 172.21.15.83:0/2004034677 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:59:36.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.417+0000 7fcb23400640 1 --2- 172.21.15.83:0/2004034677 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f66e900b40 0x55f66e9032a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:36.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.417+0000 7fcb23e00640 1 --2- 172.21.15.83:0/2004034677 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f66e90bb30 0x55f66e90b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:36.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.417+0000 7fcb22a00640 1 -- 172.21.15.83:0/2004034677 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f66e900b40 msgr2=0x55f66e9032a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:36.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.417+0000 7fcb22a00640 1 --2- 172.21.15.83:0/2004034677 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f66e900b40 0x55f66e9032a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:36.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.417+0000 7fcb22a00640 1 -- 172.21.15.83:0/2004034677 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f66e90bb30 msgr2=0x55f66e90b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:36.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.417+0000 7fcb22a00640 1 --2- 172.21.15.83:0/2004034677 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f66e90bb30 0x55f66e90b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:36.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.417+0000 7fcb22a00640 1 -- 172.21.15.83:0/2004034677 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f66e909310 con 0x55f66e901460 2024-04-24T08:59:36.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.417+0000 7fcb22a00640 1 --2- 172.21.15.83:0/2004034677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f66e901460 0x55f66e908160 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7fcb10009930 tx=0x7fcb1003f070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c00eac9706a4f4e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:36.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.417+0000 7fcb0f400640 1 -- 172.21.15.83:0/2004034677 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb10057070 con 0x55f66e901460 2024-04-24T08:59:36.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.417+0000 7fcb0f400640 1 -- 172.21.15.83:0/2004034677 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcb10051e20 con 0x55f66e901460 2024-04-24T08:59:36.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.417+0000 7fcb0f400640 1 -- 172.21.15.83:0/2004034677 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb10035040 con 0x55f66e901460 2024-04-24T08:59:36.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.418+0000 7fcb2cf99740 1 -- 172.21.15.83:0/2004034677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f66e901460 msgr2=0x55f66e908160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:36.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.418+0000 7fcb2cf99740 1 --2- 172.21.15.83:0/2004034677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f66e901460 0x55f66e908160 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7fcb10009930 tx=0x7fcb1003f070 comp rx=0 tx=0).stop 2024-04-24T08:59:36.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.418+0000 7fcb2cf99740 1 -- 172.21.15.83:0/2004034677 shutdown_connections 2024-04-24T08:59:36.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.418+0000 7fcb2cf99740 1 --2- 172.21.15.83:0/2004034677 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f66e90bb30 0x55f66e90b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:36.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.418+0000 7fcb2cf99740 1 --2- 172.21.15.83:0/2004034677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f66e901460 0x55f66e908160 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:36.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.418+0000 7fcb2cf99740 1 --2- 172.21.15.83:0/2004034677 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f66e900b40 0x55f66e9032a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:36.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.418+0000 7fcb2cf99740 1 -- 172.21.15.83:0/2004034677 >> 172.21.15.83:0/2004034677 conn(0x55f66e8fce60 msgr2=0x55f66e8fbf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:36.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.418+0000 7fcb2cf99740 1 -- 172.21.15.83:0/2004034677 shutdown_connections 2024-04-24T08:59:36.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.418+0000 7fcb2cf99740 1 -- 172.21.15.83:0/2004034677 wait complete. 2024-04-24T08:59:36.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.419+0000 7fcb2cf99740 1 -- start start 2024-04-24T08:59:36.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.419+0000 7fcb2cf99740 1 Processor -- start 2024-04-24T08:59:36.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.419+0000 7fcb2cf99740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f66e900b40 0x55f66e9142d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:36.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.419+0000 7fcb2cf99740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f66e901460 0x55f66e90c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:36.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.419+0000 7fcb2cf99740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f66e90bb30 0x55f66e90cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:36.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.420+0000 7fcb2cf99740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcb241b6e40 con 0x55f66e900b40 2024-04-24T08:59:36.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.420+0000 7fcb2cf99740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcb241b6770 con 0x55f66e90bb30 2024-04-24T08:59:36.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.420+0000 7fcb2cf99740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fcb241b39d0 con 0x55f66e901460 2024-04-24T08:59:36.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.420+0000 7fcb2cf99740 10 client.0 _ll_register_callbacks cb 0x7fcb1400bb00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:59:36.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.420+0000 7fcb23400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f66e900b40 0x55f66e9142d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:36.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.420+0000 7fcb23400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f66e900b40 0x55f66e9142d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44182/0 (socket says 172.21.15.83:44182) 2024-04-24T08:59:36.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.420+0000 7fcb23400640 1 -- 172.21.15.83:0/1010443695 learned_addr learned my addr 172.21.15.83:0/1010443695 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:59:36.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.420+0000 7fcb23400640 1 -- 172.21.15.83:0/1010443695 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f66e901460 msgr2=0x55f66e90c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:36.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.420+0000 7fcb23400640 1 --2- 172.21.15.83:0/1010443695 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f66e901460 0x55f66e90c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:36.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.420+0000 7fcb23400640 1 -- 172.21.15.83:0/1010443695 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f66e90bb30 msgr2=0x55f66e90cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:36.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.420+0000 7fcb23400640 1 --2- 172.21.15.83:0/1010443695 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f66e90bb30 0x55f66e90cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:36.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.420+0000 7fcb23400640 1 -- 172.21.15.83:0/1010443695 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f66e6427d0 con 0x55f66e900b40 2024-04-24T08:59:36.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.420+0000 7fcb23400640 1 --2- 172.21.15.83:0/1010443695 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f66e900b40 0x55f66e9142d0 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7fcb14004570 tx=0x7fcb140052d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35ad930cd66ce31e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:36.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.420+0000 7fcb0cc00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:59:36.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.421+0000 7fcb0cc00640 1 -- 172.21.15.83:0/1010443695 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb14015040 con 0x55f66e900b40 2024-04-24T08:59:36.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.421+0000 7fcb0cc00640 1 -- 172.21.15.83:0/1010443695 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7fcb1400fc30 con 0x55f66e900b40 2024-04-24T08:59:36.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.421+0000 7fcb2cf99740 1 -- 172.21.15.83:0/1010443695 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb10051e20 con 0x55f66e900b40 2024-04-24T08:59:36.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.421+0000 7fcb0cc00640 1 -- 172.21.15.83:0/1010443695 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fcb14010a60 con 0x55f66e900b40 2024-04-24T08:59:36.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.421+0000 7fcb2cf99740 10 client.18468 fetch_fsmap learned FSMap version 35 2024-04-24T08:59:36.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.421+0000 7fcb2cf99740 1 -- 172.21.15.83:0/1010443695 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55f66e982130 con 0x55f66e900b40 2024-04-24T08:59:36.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.421+0000 7fcb0cc00640 1 -- 172.21.15.83:0/1010443695 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcb10051e20 con 0x55f66e900b40 2024-04-24T08:59:36.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.421+0000 7fcb0cc00640 1 -- 172.21.15.83:0/1010443695 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7fcb14030070 con 0x55f66e900b40 2024-04-24T08:59:36.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.422+0000 7fcb2cf99740 10 client.18468 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T08:59:36.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.422+0000 7fcb2cf99740 10 client.18468 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-24T08:59:36.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.422+0000 7fcb2cf99740 20 client.18468 populate_metadata read hostname 'smithi083' 2024-04-24T08:59:36.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.422+0000 7fcb2cf99740 1 --2- 172.21.15.83:0/1010443695 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55f66e9826e0 0x55f66e984b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:36.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.422+0000 7fcb2cf99740 4 client.18468 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T08:59:36.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.422+0000 7fcb2cf99740 1 -- 172.21.15.83:0/1010443695 --> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f66e90e4d0 con 0x55f66e9826e0 2024-04-24T08:59:36.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.422+0000 7fcb22a00640 1 --2- 172.21.15.83:0/1010443695 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55f66e9826e0 0x55f66e984b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:36.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.423+0000 7fcb22a00640 1 --2- 172.21.15.83:0/1010443695 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55f66e9826e0 0x55f66e984b20 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2b6e0088d812f273 server_cookie=eaa027bfc6a63ea8 in_seq=0 out_seq=0 2024-04-24T08:59:36.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.423+0000 7fcb0cc00640 10 client.18468 ms_handle_connect on v2:172.21.15.189:6838/2548965336 2024-04-24T08:59:36.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.424+0000 7fcb0cc00640 1 -- 172.21.15.83:0/1010443695 <== mds.4 v2:172.21.15.189:6838/2548965336 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f66e90e4d0 con 0x55f66e9826e0 2024-04-24T08:59:36.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.424+0000 7fcb0cc00640 10 client.18468 handle_command_reply: tid=0 2024-04-24T08:59:36.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.424+0000 7fcb2cf99740 1 client.18468 shutdown 2024-04-24T08:59:36.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.424+0000 7fcb2cf99740 1 -- 172.21.15.83:0/1010443695 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f66e900b40 msgr2=0x55f66e9142d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:36.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.424+0000 7fcb2cf99740 1 --2- 172.21.15.83:0/1010443695 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f66e900b40 0x55f66e9142d0 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7fcb14004570 tx=0x7fcb140052d0 comp rx=0 tx=0).stop 2024-04-24T08:59:36.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.424+0000 7fcb2cf99740 1 -- 172.21.15.83:0/1010443695 shutdown_connections 2024-04-24T08:59:36.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.424+0000 7fcb2cf99740 1 --2- 172.21.15.83:0/1010443695 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55f66e9826e0 0x55f66e984b20 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:36.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.424+0000 7fcb2cf99740 1 --2- 172.21.15.83:0/1010443695 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f66e90bb30 0x55f66e90cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:36.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.424+0000 7fcb2cf99740 1 --2- 172.21.15.83:0/1010443695 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f66e901460 0x55f66e90c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:36.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.424+0000 7fcb2cf99740 1 --2- 172.21.15.83:0/1010443695 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f66e900b40 0x55f66e9142d0 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:36.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.424+0000 7fcb0cc00640 0 client.18468 ms_handle_reset on v2:172.21.15.189:6838/2548965336 2024-04-24T08:59:36.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.424+0000 7fcb2cf99740 1 -- 172.21.15.83:0/1010443695 >> 172.21.15.83:0/1010443695 conn(0x55f66e8fce60 msgr2=0x55f66e902290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:36.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.424+0000 7fcb2cf99740 1 -- 172.21.15.83:0/1010443695 shutdown_connections 2024-04-24T08:59:36.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.424+0000 7fcb2cf99740 1 -- 172.21.15.83:0/1010443695 wait complete. 2024-04-24T08:59:36.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.424+0000 7fcb2cf99740 20 client.18468 trim_cache size 0 max 16384 2024-04-24T08:59:36.429 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T08:59:36.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.425+0000 7fcb28c00640 1 -- 172.21.15.83:0/2901327043 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fcb080a3d20 msgr2=0x7fcb080a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:36.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.425+0000 7fcb28c00640 1 --2- 172.21.15.83:0/2901327043 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fcb080a3d20 0x7fcb080a61e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fcb10006010 tx=0x7fcb10031400 comp rx=0 tx=0).stop 2024-04-24T08:59:36.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.425+0000 7fcb28c00640 1 -- 172.21.15.83:0/2901327043 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb2410d500 msgr2=0x7fcb2419eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:36.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.425+0000 7fcb28c00640 1 --2- 172.21.15.83:0/2901327043 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb2410d500 0x7fcb2419eb30 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c00cab0 tx=0x7fcb1c00cf70 comp rx=0 tx=0).stop 2024-04-24T08:59:36.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.425+0000 7fcb28c00640 1 -- 172.21.15.83:0/2901327043 shutdown_connections 2024-04-24T08:59:36.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.425+0000 7fcb28c00640 1 --2- 172.21.15.83:0/2901327043 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fcb080a3d20 0x7fcb080a61e0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:36.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.425+0000 7fcb28c00640 1 --2- 172.21.15.83:0/2901327043 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb2410d500 0x7fcb2419eb30 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:36.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.425+0000 7fcb28c00640 1 --2- 172.21.15.83:0/2901327043 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcb24108f40 0x7fcb241a4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:36.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.425+0000 7fcb28c00640 1 --2- 172.21.15.83:0/2901327043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcb24108580 0x7fcb241a4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:36.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.425+0000 7fcb28c00640 1 -- 172.21.15.83:0/2901327043 >> 172.21.15.83:0/2901327043 conn(0x7fcb24077e20 msgr2=0x7fcb24078480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:36.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.425+0000 7fcb28c00640 1 -- 172.21.15.83:0/2901327043 shutdown_connections 2024-04-24T08:59:36.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.425+0000 7fcb28c00640 1 -- 172.21.15.83:0/2901327043 wait complete. 2024-04-24T08:59:36.444 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-24T08:59:36.594 INFO:tasks.workunit.client.0.smithi083.stdout: 1 44065 0.63 MB/sec execute 250 sec latency 60.518 ms 2024-04-24T08:59:36.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.662+0000 7fa834200640 1 -- 172.21.15.83:0/3087318208 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa82c104e60 msgr2=0x7fa82c108ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:36.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.662+0000 7fa834200640 1 --2- 172.21.15.83:0/3087318208 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa82c104e60 0x7fa82c108ea0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7fa818009900 tx=0x7fa81802a3f0 comp rx=0 tx=0).stop 2024-04-24T08:59:36.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.662+0000 7fa834200640 1 -- 172.21.15.83:0/3087318208 shutdown_connections 2024-04-24T08:59:36.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.662+0000 7fa834200640 1 --2- 172.21.15.83:0/3087318208 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa82c109580 0x7fa82c111150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:36.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.662+0000 7fa834200640 1 --2- 172.21.15.83:0/3087318208 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa82c104e60 0x7fa82c108ea0 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:36.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.662+0000 7fa834200640 1 --2- 172.21.15.83:0/3087318208 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa82c1044b0 0x7fa82c104890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:36.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.662+0000 7fa834200640 1 -- 172.21.15.83:0/3087318208 >> 172.21.15.83:0/3087318208 conn(0x7fa82c0ffd90 msgr2=0x7fa82c1021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:36.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.662+0000 7fa834200640 1 -- 172.21.15.83:0/3087318208 shutdown_connections 2024-04-24T08:59:36.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.662+0000 7fa834200640 1 -- 172.21.15.83:0/3087318208 wait complete. 2024-04-24T08:59:36.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.663+0000 7fa834200640 1 Processor -- start 2024-04-24T08:59:36.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.663+0000 7fa834200640 1 -- start start 2024-04-24T08:59:36.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.664+0000 7fa834200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa82c1044b0 0x7fa82c1a06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:36.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.664+0000 7fa834200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa82c109580 0x7fa82c1a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:36.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.664+0000 7fa834200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa82c19a850 0x7fa82c19ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:36.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.664+0000 7fa834200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa82c114510 con 0x7fa82c109580 2024-04-24T08:59:36.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.664+0000 7fa834200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa82c114390 con 0x7fa82c19a850 2024-04-24T08:59:36.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.664+0000 7fa834200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa82c114690 con 0x7fa82c1044b0 2024-04-24T08:59:36.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.664+0000 7fa830e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa82c109580 0x7fa82c1a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:36.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.664+0000 7fa830e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa82c109580 0x7fa82c1a0c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44194/0 (socket says 172.21.15.83:44194) 2024-04-24T08:59:36.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.664+0000 7fa830e00640 1 -- 172.21.15.83:0/4207088352 learned_addr learned my addr 172.21.15.83:0/4207088352 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:59:36.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.664+0000 7fa831800640 1 --2- 172.21.15.83:0/4207088352 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa82c1044b0 0x7fa82c1a06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:36.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.664+0000 7fa832200640 1 --2- 172.21.15.83:0/4207088352 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa82c19a850 0x7fa82c19ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:36.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.665+0000 7fa830e00640 1 -- 172.21.15.83:0/4207088352 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa82c1044b0 msgr2=0x7fa82c1a06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:36.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.665+0000 7fa830e00640 1 --2- 172.21.15.83:0/4207088352 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa82c1044b0 0x7fa82c1a06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:36.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.665+0000 7fa830e00640 1 -- 172.21.15.83:0/4207088352 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa82c19a850 msgr2=0x7fa82c19ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:36.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.665+0000 7fa830e00640 1 --2- 172.21.15.83:0/4207088352 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa82c19a850 0x7fa82c19ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:36.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.665+0000 7fa830e00640 1 -- 172.21.15.83:0/4207088352 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa82c19b3e0 con 0x7fa82c109580 2024-04-24T08:59:36.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.665+0000 7fa830e00640 1 --2- 172.21.15.83:0/4207088352 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa82c109580 0x7fa82c1a0c20 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7fa818005ab0 tx=0x7fa818003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:36.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.665+0000 7fa82a000640 1 -- 172.21.15.83:0/4207088352 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa81803f070 con 0x7fa82c109580 2024-04-24T08:59:36.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.665+0000 7fa82a000640 1 -- 172.21.15.83:0/4207088352 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa818031d60 con 0x7fa82c109580 2024-04-24T08:59:36.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.665+0000 7fa82a000640 1 -- 172.21.15.83:0/4207088352 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa818035030 con 0x7fa82c109580 2024-04-24T08:59:36.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.665+0000 7fa834200640 1 -- 172.21.15.83:0/4207088352 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa82c19b670 con 0x7fa82c109580 2024-04-24T08:59:36.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.665+0000 7fa834200640 1 -- 172.21.15.83:0/4207088352 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa82c1a7820 con 0x7fa82c109580 2024-04-24T08:59:36.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.669+0000 7fa8371d1740 1 Processor -- start 2024-04-24T08:59:36.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.669+0000 7fa8371d1740 1 -- start start 2024-04-24T08:59:36.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.669+0000 7fa8371d1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570f4fdc950 0x5570f4fdcd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:36.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.669+0000 7fa8371d1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5570f4fdd270 0x5570f4fe7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:36.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.669+0000 7fa8371d1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570f4fe7940 0x5570f4fe9d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:36.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.669+0000 7fa8371d1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5570f4f421d0 con 0x5570f4fe7940 2024-04-24T08:59:36.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.669+0000 7fa8371d1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5570f4ee8450 con 0x5570f4fdc950 2024-04-24T08:59:36.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.669+0000 7fa8371d1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5570f4f3e190 con 0x5570f4fdd270 2024-04-24T08:59:36.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.669+0000 7fa82a000640 1 -- 172.21.15.83:0/4207088352 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa818042070 con 0x7fa82c109580 2024-04-24T08:59:36.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.670+0000 7fa830e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570f4fe7940 0x5570f4fe9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:36.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.670+0000 7fa830e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570f4fe7940 0x5570f4fe9d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44204/0 (socket says 172.21.15.83:44204) 2024-04-24T08:59:36.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.670+0000 7fa830e00640 1 -- 172.21.15.83:0/3040088222 learned_addr learned my addr 172.21.15.83:0/3040088222 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:59:36.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.670+0000 7fa831800640 1 --2- 172.21.15.83:0/3040088222 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5570f4fdd270 0x5570f4fe7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:36.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.670+0000 7fa832200640 1 --2- 172.21.15.83:0/3040088222 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570f4fdc950 0x5570f4fdcd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:36.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.670+0000 7fa830e00640 1 -- 172.21.15.83:0/3040088222 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5570f4fdd270 msgr2=0x5570f4fe7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:36.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.670+0000 7fa830e00640 1 --2- 172.21.15.83:0/3040088222 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5570f4fdd270 0x5570f4fe7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:36.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.670+0000 7fa830e00640 1 -- 172.21.15.83:0/3040088222 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570f4fdc950 msgr2=0x5570f4fdcd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:36.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.670+0000 7fa830e00640 1 --2- 172.21.15.83:0/3040088222 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570f4fdc950 0x5570f4fdcd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:36.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.670+0000 7fa830e00640 1 -- 172.21.15.83:0/3040088222 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570f4fa5310 con 0x5570f4fe7940 2024-04-24T08:59:36.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.670+0000 7fa832200640 1 --2- 172.21.15.83:0/3040088222 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570f4fdc950 0x5570f4fdcd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:59:36.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.671+0000 7fa830e00640 1 --2- 172.21.15.83:0/3040088222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570f4fe7940 0x5570f4fe9d30 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7fa8180ee3d0 tx=0x7fa8180f9540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3c57671e2b3067c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:36.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.671+0000 7fa82a000640 1 --2- 172.21.15.83:0/4207088352 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa8140a3d20 0x7fa8140a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:36.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.671+0000 7fa81ea00640 1 -- 172.21.15.83:0/3040088222 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8180f9e60 con 0x5570f4fe7940 2024-04-24T08:59:36.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.671+0000 7fa81ea00640 1 -- 172.21.15.83:0/3040088222 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa8180fe030 con 0x5570f4fe7940 2024-04-24T08:59:36.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.671+0000 7fa81ea00640 1 -- 172.21.15.83:0/3040088222 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8180bc640 con 0x5570f4fe7940 2024-04-24T08:59:36.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.671+0000 7fa832200640 1 --2- 172.21.15.83:0/4207088352 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa8140a3d20 0x7fa8140a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:36.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.671+0000 7fa82a000640 1 -- 172.21.15.83:0/4207088352 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7fa8180f11c0 con 0x7fa82c109580 2024-04-24T08:59:36.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.672+0000 7fa8371d1740 1 -- 172.21.15.83:0/3040088222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570f4fe7940 msgr2=0x5570f4fe9d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:36.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.672+0000 7fa832200640 1 --2- 172.21.15.83:0/4207088352 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa8140a3d20 0x7fa8140a61e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa8240092f0 tx=0x7fa824009280 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:36.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.672+0000 7fa8371d1740 1 --2- 172.21.15.83:0/3040088222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570f4fe7940 0x5570f4fe9d30 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7fa8180ee3d0 tx=0x7fa8180f9540 comp rx=0 tx=0).stop 2024-04-24T08:59:36.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.672+0000 7fa8371d1740 1 -- 172.21.15.83:0/3040088222 shutdown_connections 2024-04-24T08:59:36.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.672+0000 7fa8371d1740 1 --2- 172.21.15.83:0/3040088222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570f4fe7940 0x5570f4fe9d30 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:36.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.672+0000 7fa8371d1740 1 --2- 172.21.15.83:0/3040088222 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5570f4fdd270 0x5570f4fe7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:36.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.672+0000 7fa8371d1740 1 --2- 172.21.15.83:0/3040088222 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570f4fdc950 0x5570f4fdcd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:36.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.672+0000 7fa8371d1740 1 -- 172.21.15.83:0/3040088222 >> 172.21.15.83:0/3040088222 conn(0x5570f4fd8c70 msgr2=0x5570f4fd9050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:36.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.672+0000 7fa8371d1740 1 -- 172.21.15.83:0/3040088222 shutdown_connections 2024-04-24T08:59:36.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.673+0000 7fa8371d1740 1 -- 172.21.15.83:0/3040088222 wait complete. 2024-04-24T08:59:36.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.673+0000 7fa8371d1740 1 -- start start 2024-04-24T08:59:36.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.673+0000 7fa8371d1740 1 Processor -- start 2024-04-24T08:59:36.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.674+0000 7fa8371d1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570f4fdc950 0x5570f4fe7330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:36.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.674+0000 7fa8371d1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570f4fdd270 0x5570f4fe4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:36.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.674+0000 7fa8371d1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5570f4fe7940 0x5570f4fe4c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:36.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.674+0000 7fa8371d1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5570f4fea4f0 con 0x5570f4fdd270 2024-04-24T08:59:36.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.674+0000 7fa8371d1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5570f4efb810 con 0x5570f4fdc950 2024-04-24T08:59:36.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.674+0000 7fa8371d1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5570f4fea670 con 0x5570f4fe7940 2024-04-24T08:59:36.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.674+0000 7fa8371d1740 10 client.0 _ll_register_callbacks cb 0x5570f4ec9140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:59:36.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.674+0000 7fa830e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570f4fdd270 0x5570f4fe4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:36.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.674+0000 7fa830e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570f4fdd270 0x5570f4fe4620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44208/0 (socket says 172.21.15.83:44208) 2024-04-24T08:59:36.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.674+0000 7fa830e00640 1 -- 172.21.15.83:0/2472947398 learned_addr learned my addr 172.21.15.83:0/2472947398 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:59:36.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.674+0000 7fa831800640 1 --2- 172.21.15.83:0/2472947398 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570f4fdc950 0x5570f4fe7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:36.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.674+0000 7fa832200640 1 --2- 172.21.15.83:0/2472947398 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5570f4fe7940 0x5570f4fe4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:36.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.674+0000 7fa830e00640 1 -- 172.21.15.83:0/2472947398 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5570f4fe7940 msgr2=0x5570f4fe4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:36.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.674+0000 7fa830e00640 1 --2- 172.21.15.83:0/2472947398 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5570f4fe7940 0x5570f4fe4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:36.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.674+0000 7fa830e00640 1 -- 172.21.15.83:0/2472947398 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570f4fdc950 msgr2=0x5570f4fe7330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:36.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.674+0000 7fa830e00640 1 --2- 172.21.15.83:0/2472947398 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570f4fdc950 0x5570f4fe7330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:36.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.674+0000 7fa830e00640 1 -- 172.21.15.83:0/2472947398 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570f4f0c3f0 con 0x5570f4fdd270 2024-04-24T08:59:36.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.674+0000 7fa832200640 1 --2- 172.21.15.83:0/2472947398 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5570f4fe7940 0x5570f4fe4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:59:36.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.675+0000 7fa830e00640 1 --2- 172.21.15.83:0/2472947398 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570f4fdd270 0x5570f4fe4620 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7fa8180b9140 tx=0x7fa818031f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dc191e2a79bb4cb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:36.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.675+0000 7fa813400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:59:36.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.675+0000 7fa813400640 1 -- 172.21.15.83:0/2472947398 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa818049070 con 0x5570f4fdd270 2024-04-24T08:59:36.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.675+0000 7fa813400640 1 -- 172.21.15.83:0/2472947398 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7fa8180b9350 con 0x5570f4fdd270 2024-04-24T08:59:36.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.675+0000 7fa8371d1740 1 -- 172.21.15.83:0/2472947398 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8180fe030 con 0x5570f4fdd270 2024-04-24T08:59:36.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.675+0000 7fa813400640 1 -- 172.21.15.83:0/2472947398 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fa8180bc820 con 0x5570f4fdd270 2024-04-24T08:59:36.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.675+0000 7fa8371d1740 10 client.18480 fetch_fsmap learned FSMap version 35 2024-04-24T08:59:36.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.675+0000 7fa8371d1740 1 -- 172.21.15.83:0/2472947398 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5570f4c543f0 con 0x5570f4fdd270 2024-04-24T08:59:36.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.676+0000 7fa813400640 1 -- 172.21.15.83:0/2472947398 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa8180fe030 con 0x5570f4fdd270 2024-04-24T08:59:36.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.676+0000 7fa813400640 1 -- 172.21.15.83:0/2472947398 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7fa81804c060 con 0x5570f4fdd270 2024-04-24T08:59:36.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.676+0000 7fa8371d1740 10 client.18480 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T08:59:36.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.676+0000 7fa8371d1740 10 client.18480 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-24T08:59:36.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.676+0000 7fa8371d1740 20 client.18480 populate_metadata read hostname 'smithi083' 2024-04-24T08:59:36.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.676+0000 7fa8371d1740 1 --2- 172.21.15.83:0/2472947398 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x5570f4ff7660 0x5570f4ff9a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:36.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.676+0000 7fa8371d1740 4 client.18480 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:59:36.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.676+0000 7fa8371d1740 1 -- 172.21.15.83:0/2472947398 --> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5570f4f4fbf0 con 0x5570f4ff7660 2024-04-24T08:59:36.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.677+0000 7fa831800640 1 --2- 172.21.15.83:0/2472947398 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x5570f4ff7660 0x5570f4ff9a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:36.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.678+0000 7fa831800640 1 --2- 172.21.15.83:0/2472947398 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x5570f4ff7660 0x5570f4ff9a50 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=80900f7b6192b9f8 server_cookie=9e720ebeaa866e48 in_seq=0 out_seq=0 2024-04-24T08:59:36.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.678+0000 7fa813400640 10 client.18480 ms_handle_connect on v2:172.21.15.138:6832/1157967658 2024-04-24T08:59:36.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.680+0000 7fa813400640 1 -- 172.21.15.83:0/2472947398 <== mds.3 v2:172.21.15.138:6832/1157967658 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5570f4f4fbf0 con 0x5570f4ff7660 2024-04-24T08:59:36.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.680+0000 7fa813400640 10 client.18480 handle_command_reply: tid=0 2024-04-24T08:59:36.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.681+0000 7fa8371d1740 1 client.18480 shutdown 2024-04-24T08:59:36.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.681+0000 7fa8371d1740 1 -- 172.21.15.83:0/2472947398 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570f4fdd270 msgr2=0x5570f4fe4620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:36.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.681+0000 7fa8371d1740 1 --2- 172.21.15.83:0/2472947398 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570f4fdd270 0x5570f4fe4620 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7fa8180b9140 tx=0x7fa818031f10 comp rx=0 tx=0).stop 2024-04-24T08:59:36.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.681+0000 7fa8371d1740 1 -- 172.21.15.83:0/2472947398 shutdown_connections 2024-04-24T08:59:36.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.681+0000 7fa8371d1740 1 --2- 172.21.15.83:0/2472947398 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x5570f4ff7660 0x5570f4ff9a50 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:36.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.681+0000 7fa8371d1740 1 --2- 172.21.15.83:0/2472947398 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5570f4fe7940 0x5570f4fe4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:36.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.681+0000 7fa8371d1740 1 --2- 172.21.15.83:0/2472947398 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570f4fdd270 0x5570f4fe4620 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:36.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.681+0000 7fa8371d1740 1 --2- 172.21.15.83:0/2472947398 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570f4fdc950 0x5570f4fe7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:36.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.681+0000 7fa8371d1740 1 -- 172.21.15.83:0/2472947398 >> 172.21.15.83:0/2472947398 conn(0x5570f4fd8c70 msgr2=0x5570f4fdea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:36.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.681+0000 7fa813400640 0 client.18480 ms_handle_reset on v2:172.21.15.138:6832/1157967658 2024-04-24T08:59:36.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.681+0000 7fa8371d1740 1 -- 172.21.15.83:0/2472947398 shutdown_connections 2024-04-24T08:59:36.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.681+0000 7fa8371d1740 1 -- 172.21.15.83:0/2472947398 wait complete. 2024-04-24T08:59:36.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.681+0000 7fa8371d1740 20 client.18480 trim_cache size 0 max 16384 2024-04-24T08:59:36.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.697+0000 7fa8371d1740 1 Processor -- start 2024-04-24T08:59:36.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.697+0000 7fa8371d1740 1 -- start start 2024-04-24T08:59:36.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.697+0000 7fa8371d1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570f4fdc950 0x5570f4fd7c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:36.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.697+0000 7fa8371d1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570f4fdd270 0x5570f4fe6e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:36.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.697+0000 7fa8371d1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5570f4fe7940 0x5570f4fff260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:36.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.697+0000 7fa8371d1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa82c1b1820 con 0x5570f4fdc950 2024-04-24T08:59:36.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.697+0000 7fa8371d1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa82c1b3360 con 0x5570f4fdd270 2024-04-24T08:59:36.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.697+0000 7fa8371d1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa82c1b1ef0 con 0x5570f4fe7940 2024-04-24T08:59:36.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.697+0000 7fa831800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570f4fdc950 0x5570f4fd7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:36.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.697+0000 7fa831800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570f4fdc950 0x5570f4fd7c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44222/0 (socket says 172.21.15.83:44222) 2024-04-24T08:59:36.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.697+0000 7fa831800640 1 -- 172.21.15.83:0/365809038 learned_addr learned my addr 172.21.15.83:0/365809038 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:59:36.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.698+0000 7fa831800640 1 -- 172.21.15.83:0/365809038 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5570f4fe7940 msgr2=0x5570f4fff260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:36.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.698+0000 7fa831800640 1 --2- 172.21.15.83:0/365809038 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5570f4fe7940 0x5570f4fff260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:36.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.698+0000 7fa830e00640 1 --2- 172.21.15.83:0/365809038 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570f4fdd270 0x5570f4fe6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:36.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.698+0000 7fa831800640 1 -- 172.21.15.83:0/365809038 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570f4fdd270 msgr2=0x5570f4fe6e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:36.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.698+0000 7fa831800640 1 --2- 172.21.15.83:0/365809038 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570f4fdd270 0x5570f4fe6e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:36.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.698+0000 7fa831800640 1 -- 172.21.15.83:0/365809038 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570f4ff9dc0 con 0x5570f4fdc950 2024-04-24T08:59:36.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.698+0000 7fa831800640 1 --2- 172.21.15.83:0/365809038 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570f4fdc950 0x5570f4fd7c70 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x5570f4ffa200 tx=0x7fa82000cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acdb39b1a5e7404e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:36.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.698+0000 7fa81e000640 1 -- 172.21.15.83:0/365809038 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa820003ed0 con 0x5570f4fdc950 2024-04-24T08:59:36.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.698+0000 7fa81e000640 1 -- 172.21.15.83:0/365809038 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa82001c8e0 con 0x5570f4fdc950 2024-04-24T08:59:36.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.698+0000 7fa81e000640 1 -- 172.21.15.83:0/365809038 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa820004a90 con 0x5570f4fdc950 2024-04-24T08:59:36.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.699+0000 7fa8371d1740 1 -- 172.21.15.83:0/365809038 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570f4fdc950 msgr2=0x5570f4fd7c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:36.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.699+0000 7fa8371d1740 1 --2- 172.21.15.83:0/365809038 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570f4fdc950 0x5570f4fd7c70 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x5570f4ffa200 tx=0x7fa82000cf30 comp rx=0 tx=0).stop 2024-04-24T08:59:36.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.699+0000 7fa8371d1740 1 -- 172.21.15.83:0/365809038 shutdown_connections 2024-04-24T08:59:36.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.699+0000 7fa8371d1740 1 --2- 172.21.15.83:0/365809038 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5570f4fe7940 0x5570f4fff260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:36.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.699+0000 7fa8371d1740 1 --2- 172.21.15.83:0/365809038 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570f4fdd270 0x5570f4fe6e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:36.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.699+0000 7fa8371d1740 1 --2- 172.21.15.83:0/365809038 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570f4fdc950 0x5570f4fd7c70 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:36.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.699+0000 7fa8371d1740 1 -- 172.21.15.83:0/365809038 >> 172.21.15.83:0/365809038 conn(0x5570f4fd8c70 msgr2=0x5570f4fd7560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:36.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.699+0000 7fa8371d1740 1 -- 172.21.15.83:0/365809038 shutdown_connections 2024-04-24T08:59:36.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.699+0000 7fa8371d1740 1 -- 172.21.15.83:0/365809038 wait complete. 2024-04-24T08:59:36.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.700+0000 7fa8371d1740 1 -- start start 2024-04-24T08:59:36.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.700+0000 7fa8371d1740 1 Processor -- start 2024-04-24T08:59:36.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.700+0000 7fa8371d1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570f4fdc950 0x5570f4fef8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:36.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.700+0000 7fa8371d1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570f4fdd270 0x5570f4fefe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:36.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.700+0000 7fa8371d1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5570f4fe7940 0x5570f4fea880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:36.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.700+0000 7fa8371d1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa82c1b2c90 con 0x5570f4fdd270 2024-04-24T08:59:36.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.700+0000 7fa8371d1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa82c1b25c0 con 0x5570f4fdc950 2024-04-24T08:59:36.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.700+0000 7fa8371d1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa82c1af820 con 0x5570f4fe7940 2024-04-24T08:59:36.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.700+0000 7fa8371d1740 10 client.0 _ll_register_callbacks cb 0x7fa82000c350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:59:36.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.701+0000 7fa830e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570f4fdd270 0x5570f4fefe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:36.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.701+0000 7fa830e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570f4fdd270 0x5570f4fefe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44232/0 (socket says 172.21.15.83:44232) 2024-04-24T08:59:36.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.701+0000 7fa830e00640 1 -- 172.21.15.83:0/2632383320 learned_addr learned my addr 172.21.15.83:0/2632383320 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:59:36.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.701+0000 7fa832200640 1 --2- 172.21.15.83:0/2632383320 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5570f4fe7940 0x5570f4fea880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:36.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.701+0000 7fa830e00640 1 -- 172.21.15.83:0/2632383320 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5570f4fe7940 msgr2=0x5570f4fea880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:36.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.701+0000 7fa830e00640 1 --2- 172.21.15.83:0/2632383320 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5570f4fe7940 0x5570f4fea880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:36.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.701+0000 7fa830e00640 1 -- 172.21.15.83:0/2632383320 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570f4fdc950 msgr2=0x5570f4fef8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:36.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.701+0000 7fa830e00640 1 --2- 172.21.15.83:0/2632383320 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570f4fdc950 0x5570f4fef8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:36.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.701+0000 7fa830e00640 1 -- 172.21.15.83:0/2632383320 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570f4d1e7d0 con 0x5570f4fdd270 2024-04-24T08:59:36.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.701+0000 7fa832200640 1 --2- 172.21.15.83:0/2632383320 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5570f4fe7940 0x5570f4fea880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:59:36.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.701+0000 7fa830e00640 1 --2- 172.21.15.83:0/2632383320 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570f4fdd270 0x5570f4fefe10 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7fa8180b96b0 tx=0x7fa8180045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a94ea48faef3bad6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:36.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.701+0000 7fa813400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:59:36.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.701+0000 7fa813400640 1 -- 172.21.15.83:0/2632383320 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa818049050 con 0x5570f4fdd270 2024-04-24T08:59:36.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.701+0000 7fa813400640 1 -- 172.21.15.83:0/2632383320 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7fa8180ee710 con 0x5570f4fdd270 2024-04-24T08:59:36.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.702+0000 7fa8371d1740 1 -- 172.21.15.83:0/2632383320 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa82001c8e0 con 0x5570f4fdd270 2024-04-24T08:59:36.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.702+0000 7fa813400640 1 -- 172.21.15.83:0/2632383320 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fa8180d6080 con 0x5570f4fdd270 2024-04-24T08:59:36.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.702+0000 7fa8371d1740 10 client.18486 fetch_fsmap learned FSMap version 35 2024-04-24T08:59:36.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.702+0000 7fa8371d1740 1 -- 172.21.15.83:0/2632383320 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5570f4d1e7d0 con 0x5570f4fdd270 2024-04-24T08:59:36.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.702+0000 7fa813400640 1 -- 172.21.15.83:0/2632383320 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa82001c8e0 con 0x5570f4fdd270 2024-04-24T08:59:36.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.702+0000 7fa813400640 1 -- 172.21.15.83:0/2632383320 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7fa8180eebe0 con 0x5570f4fdd270 2024-04-24T08:59:36.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.702+0000 7fa8371d1740 10 client.18486 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T08:59:36.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.702+0000 7fa8371d1740 10 client.18486 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-24T08:59:36.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.702+0000 7fa8371d1740 20 client.18486 populate_metadata read hostname 'smithi083' 2024-04-24T08:59:36.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.702+0000 7fa8371d1740 1 --2- 172.21.15.83:0/2632383320 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x5570f505e670 0x5570f5060a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:36.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.702+0000 7fa8371d1740 4 client.18486 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T08:59:36.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.702+0000 7fa8371d1740 1 -- 172.21.15.83:0/2632383320 --> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5570f4f49a60 con 0x5570f505e670 2024-04-24T08:59:36.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.703+0000 7fa831800640 1 --2- 172.21.15.83:0/2632383320 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x5570f505e670 0x5570f5060a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:36.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.704+0000 7fa831800640 1 --2- 172.21.15.83:0/2632383320 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x5570f505e670 0x5570f5060a90 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=63f4d2f7acbb50d1 server_cookie=655b98f52bc14608 in_seq=0 out_seq=0 2024-04-24T08:59:36.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.704+0000 7fa813400640 10 client.18486 ms_handle_connect on v2:172.21.15.138:6832/1157967658 2024-04-24T08:59:36.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.705+0000 7fa813400640 1 -- 172.21.15.83:0/2632383320 <== mds.3 v2:172.21.15.138:6832/1157967658 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5570f4f49a60 con 0x5570f505e670 2024-04-24T08:59:36.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.705+0000 7fa813400640 10 client.18486 handle_command_reply: tid=0 2024-04-24T08:59:36.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.705+0000 7fa8371d1740 1 client.18486 shutdown 2024-04-24T08:59:36.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.705+0000 7fa8371d1740 1 -- 172.21.15.83:0/2632383320 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570f4fdd270 msgr2=0x5570f4fefe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:36.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.705+0000 7fa8371d1740 1 --2- 172.21.15.83:0/2632383320 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570f4fdd270 0x5570f4fefe10 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7fa8180b96b0 tx=0x7fa8180045e0 comp rx=0 tx=0).stop 2024-04-24T08:59:36.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.705+0000 7fa8371d1740 1 -- 172.21.15.83:0/2632383320 shutdown_connections 2024-04-24T08:59:36.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.705+0000 7fa8371d1740 1 --2- 172.21.15.83:0/2632383320 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x5570f505e670 0x5570f5060a90 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:36.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.705+0000 7fa8371d1740 1 --2- 172.21.15.83:0/2632383320 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5570f4fe7940 0x5570f4fea880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:36.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.705+0000 7fa8371d1740 1 --2- 172.21.15.83:0/2632383320 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570f4fdd270 0x5570f4fefe10 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:36.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.705+0000 7fa8371d1740 1 --2- 172.21.15.83:0/2632383320 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570f4fdc950 0x5570f4fef8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:36.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.705+0000 7fa8371d1740 1 -- 172.21.15.83:0/2632383320 >> 172.21.15.83:0/2632383320 conn(0x5570f4fd8c70 msgr2=0x5570f4fd7df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:36.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.705+0000 7fa813400640 0 client.18486 ms_handle_reset on v2:172.21.15.138:6832/1157967658 2024-04-24T08:59:36.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.706+0000 7fa8371d1740 1 -- 172.21.15.83:0/2632383320 shutdown_connections 2024-04-24T08:59:36.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.706+0000 7fa8371d1740 1 -- 172.21.15.83:0/2632383320 wait complete. 2024-04-24T08:59:36.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.706+0000 7fa8371d1740 20 client.18486 trim_cache size 0 max 16384 2024-04-24T08:59:36.707 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T08:59:36.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.706+0000 7fa834200640 1 -- 172.21.15.83:0/4207088352 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa8140a3d20 msgr2=0x7fa8140a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:36.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.706+0000 7fa834200640 1 --2- 172.21.15.83:0/4207088352 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa8140a3d20 0x7fa8140a61e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa8240092f0 tx=0x7fa824009280 comp rx=0 tx=0).stop 2024-04-24T08:59:36.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.706+0000 7fa834200640 1 -- 172.21.15.83:0/4207088352 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa82c109580 msgr2=0x7fa82c1a0c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:36.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.706+0000 7fa834200640 1 --2- 172.21.15.83:0/4207088352 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa82c109580 0x7fa82c1a0c20 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7fa818005ab0 tx=0x7fa818003cd0 comp rx=0 tx=0).stop 2024-04-24T08:59:36.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.707+0000 7fa834200640 1 -- 172.21.15.83:0/4207088352 shutdown_connections 2024-04-24T08:59:36.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.707+0000 7fa834200640 1 --2- 172.21.15.83:0/4207088352 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa8140a3d20 0x7fa8140a61e0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:36.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.707+0000 7fa834200640 1 --2- 172.21.15.83:0/4207088352 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa82c19a850 0x7fa82c19ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:36.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.707+0000 7fa834200640 1 --2- 172.21.15.83:0/4207088352 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa82c109580 0x7fa82c1a0c20 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:36.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.707+0000 7fa834200640 1 --2- 172.21.15.83:0/4207088352 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa82c1044b0 0x7fa82c1a06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:36.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.707+0000 7fa834200640 1 -- 172.21.15.83:0/4207088352 >> 172.21.15.83:0/4207088352 conn(0x7fa82c0ffd90 msgr2=0x7fa82c1020c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:36.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.707+0000 7fa834200640 1 -- 172.21.15.83:0/4207088352 shutdown_connections 2024-04-24T08:59:36.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.707+0000 7fa834200640 1 -- 172.21.15.83:0/4207088352 wait complete. 2024-04-24T08:59:36.722 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:59:36 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/1692481467' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:59:36.725 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T08:59:36.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.928+0000 7fae49000640 1 -- 172.21.15.83:0/2728743693 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae44104e60 msgr2=0x7fae44108ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:36.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.928+0000 7fae49000640 1 --2- 172.21.15.83:0/2728743693 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae44104e60 0x7fae44108ea0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7fae30009900 tx=0x7fae3002a400 comp rx=0 tx=0).stop 2024-04-24T08:59:36.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.928+0000 7fae49000640 1 -- 172.21.15.83:0/2728743693 shutdown_connections 2024-04-24T08:59:36.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.928+0000 7fae49000640 1 --2- 172.21.15.83:0/2728743693 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae44109580 0x7fae44111150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:36.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.928+0000 7fae49000640 1 --2- 172.21.15.83:0/2728743693 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae44104e60 0x7fae44108ea0 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:36.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.928+0000 7fae49000640 1 --2- 172.21.15.83:0/2728743693 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fae441044b0 0x7fae44104890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:36.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.928+0000 7fae49000640 1 -- 172.21.15.83:0/2728743693 >> 172.21.15.83:0/2728743693 conn(0x7fae440ffd90 msgr2=0x7fae441021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:36.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.928+0000 7fae49000640 1 -- 172.21.15.83:0/2728743693 shutdown_connections 2024-04-24T08:59:36.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.928+0000 7fae49000640 1 -- 172.21.15.83:0/2728743693 wait complete. 2024-04-24T08:59:36.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.929+0000 7fae49000640 1 Processor -- start 2024-04-24T08:59:36.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.929+0000 7fae49000640 1 -- start start 2024-04-24T08:59:36.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.930+0000 7fae49000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae441044b0 0x7fae441a06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:36.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.930+0000 7fae49000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae44109580 0x7fae441a0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:36.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.930+0000 7fae49000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fae4419a8a0 0x7fae4419ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:36.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.930+0000 7fae49000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fae44114520 con 0x7fae441044b0 2024-04-24T08:59:36.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.930+0000 7fae49000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fae441143a0 con 0x7fae44109580 2024-04-24T08:59:36.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.930+0000 7fae49000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fae441146a0 con 0x7fae4419a8a0 2024-04-24T08:59:36.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.930+0000 7fae41e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae441044b0 0x7fae441a06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:36.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.930+0000 7fae41e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae441044b0 0x7fae441a06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44256/0 (socket says 172.21.15.83:44256) 2024-04-24T08:59:36.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.930+0000 7fae41e00640 1 -- 172.21.15.83:0/102419036 learned_addr learned my addr 172.21.15.83:0/102419036 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:59:36.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.930+0000 7fae41400640 1 --2- 172.21.15.83:0/102419036 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae44109580 0x7fae441a0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:36.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.930+0000 7fae41e00640 1 -- 172.21.15.83:0/102419036 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fae4419a8a0 msgr2=0x7fae4419ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:36.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.930+0000 7fae41e00640 1 --2- 172.21.15.83:0/102419036 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fae4419a8a0 0x7fae4419ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:36.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.930+0000 7fae41e00640 1 -- 172.21.15.83:0/102419036 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae44109580 msgr2=0x7fae441a0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:36.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.930+0000 7fae41e00640 1 --2- 172.21.15.83:0/102419036 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae44109580 0x7fae441a0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:36.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.930+0000 7fae41e00640 1 -- 172.21.15.83:0/102419036 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae4419b3c0 con 0x7fae441044b0 2024-04-24T08:59:36.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.930+0000 7fae41400640 1 --2- 172.21.15.83:0/102419036 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae44109580 0x7fae441a0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:59:36.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.931+0000 7fae41e00640 1 --2- 172.21.15.83:0/102419036 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae441044b0 0x7fae441a06f0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7fae3400c920 tx=0x7fae3400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:36.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.931+0000 7fae3aa00640 1 -- 172.21.15.83:0/102419036 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae34007c70 con 0x7fae441044b0 2024-04-24T08:59:36.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.931+0000 7fae3aa00640 1 -- 172.21.15.83:0/102419036 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fae34007e10 con 0x7fae441044b0 2024-04-24T08:59:36.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.931+0000 7fae3aa00640 1 -- 172.21.15.83:0/102419036 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae34018820 con 0x7fae441044b0 2024-04-24T08:59:36.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.931+0000 7fae49000640 1 -- 172.21.15.83:0/102419036 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae4419b6b0 con 0x7fae441044b0 2024-04-24T08:59:36.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.931+0000 7fae49000640 1 -- 172.21.15.83:0/102419036 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae441a78b0 con 0x7fae441044b0 2024-04-24T08:59:36.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.935+0000 7fae4bdb4740 1 Processor -- start 2024-04-24T08:59:36.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.936+0000 7fae4bdb4740 1 -- start start 2024-04-24T08:59:36.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.936+0000 7fae3aa00640 1 -- 172.21.15.83:0/102419036 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fae340189c0 con 0x7fae441044b0 2024-04-24T08:59:36.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.936+0000 7fae4bdb4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5630ee61d950 0x5630ee61dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:36.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.936+0000 7fae4bdb4740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5630ee61e270 0x5630ee628400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:36.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.936+0000 7fae4bdb4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5630ee628940 0x5630ee62ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:36.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.936+0000 7fae4bdb4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5630ee5831d0 con 0x5630ee628940 2024-04-24T08:59:36.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.936+0000 7fae4bdb4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5630ee529450 con 0x5630ee61d950 2024-04-24T08:59:36.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.936+0000 7fae4bdb4740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5630ee57f190 con 0x5630ee61e270 2024-04-24T08:59:36.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.937+0000 7fae41400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5630ee628940 0x5630ee62ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:36.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.937+0000 7fae41400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5630ee628940 0x5630ee62ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44268/0 (socket says 172.21.15.83:44268) 2024-04-24T08:59:36.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.937+0000 7fae41400640 1 -- 172.21.15.83:0/488438364 learned_addr learned my addr 172.21.15.83:0/488438364 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:59:36.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.937+0000 7fae41e00640 1 --2- 172.21.15.83:0/488438364 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5630ee61e270 0x5630ee628400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:36.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.937+0000 7fae42800640 1 --2- 172.21.15.83:0/488438364 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5630ee61d950 0x5630ee61dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:36.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.937+0000 7fae41400640 1 -- 172.21.15.83:0/488438364 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5630ee61e270 msgr2=0x5630ee628400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:36.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.937+0000 7fae41400640 1 --2- 172.21.15.83:0/488438364 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5630ee61e270 0x5630ee628400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:36.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.937+0000 7fae41400640 1 -- 172.21.15.83:0/488438364 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5630ee61d950 msgr2=0x5630ee61dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:36.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.937+0000 7fae41400640 1 --2- 172.21.15.83:0/488438364 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5630ee61d950 0x5630ee61dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:36.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.937+0000 7fae41400640 1 -- 172.21.15.83:0/488438364 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630ee5e6310 con 0x5630ee628940 2024-04-24T08:59:36.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.938+0000 7fae41400640 1 --2- 172.21.15.83:0/488438364 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5630ee628940 0x5630ee62ad30 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7fae30000c00 tx=0x7fae30032e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f31e94546cc1e971 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:36.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.938+0000 7fae3aa00640 1 --2- 172.21.15.83:0/102419036 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fae280a3da0 0x7fae280a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:36.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.938+0000 7fae3aa00640 1 -- 172.21.15.83:0/102419036 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7fae340d0e30 con 0x7fae441044b0 2024-04-24T08:59:36.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.938+0000 7fae2f400640 1 -- 172.21.15.83:0/488438364 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae3003f070 con 0x5630ee628940 2024-04-24T08:59:36.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.938+0000 7fae2f400640 1 -- 172.21.15.83:0/488438364 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fae30031760 con 0x5630ee628940 2024-04-24T08:59:36.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.938+0000 7fae42800640 1 --2- 172.21.15.83:0/102419036 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fae280a3da0 0x7fae280a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:36.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.938+0000 7fae2f400640 1 -- 172.21.15.83:0/488438364 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae30035070 con 0x5630ee628940 2024-04-24T08:59:36.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.939+0000 7fae4bdb4740 1 -- 172.21.15.83:0/488438364 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5630ee628940 msgr2=0x5630ee62ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:36.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.939+0000 7fae42800640 1 --2- 172.21.15.83:0/102419036 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fae280a3da0 0x7fae280a6260 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fae3c009e40 tx=0x7fae3c009430 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:36.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.939+0000 7fae4bdb4740 1 --2- 172.21.15.83:0/488438364 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5630ee628940 0x5630ee62ad30 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7fae30000c00 tx=0x7fae30032e50 comp rx=0 tx=0).stop 2024-04-24T08:59:36.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.939+0000 7fae4bdb4740 1 -- 172.21.15.83:0/488438364 shutdown_connections 2024-04-24T08:59:36.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.939+0000 7fae4bdb4740 1 --2- 172.21.15.83:0/488438364 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5630ee628940 0x5630ee62ad30 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:36.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.939+0000 7fae4bdb4740 1 --2- 172.21.15.83:0/488438364 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5630ee61e270 0x5630ee628400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:36.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.939+0000 7fae4bdb4740 1 --2- 172.21.15.83:0/488438364 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5630ee61d950 0x5630ee61dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:36.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.939+0000 7fae4bdb4740 1 -- 172.21.15.83:0/488438364 >> 172.21.15.83:0/488438364 conn(0x5630ee619c70 msgr2=0x5630ee61a050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:36.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.939+0000 7fae4bdb4740 1 -- 172.21.15.83:0/488438364 shutdown_connections 2024-04-24T08:59:36.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.939+0000 7fae4bdb4740 1 -- 172.21.15.83:0/488438364 wait complete. 2024-04-24T08:59:36.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.940+0000 7fae4bdb4740 1 -- start start 2024-04-24T08:59:36.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.940+0000 7fae4bdb4740 1 Processor -- start 2024-04-24T08:59:36.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.940+0000 7fae4bdb4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5630ee61d950 0x5630ee6283e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:36.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.940+0000 7fae4bdb4740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5630ee61e270 0x5630ee6256d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:36.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.940+0000 7fae4bdb4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5630ee628940 0x5630ee625dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:36.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.940+0000 7fae4bdb4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5630ee62b770 con 0x5630ee628940 2024-04-24T08:59:36.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.940+0000 7fae4bdb4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5630ee53c810 con 0x5630ee61d950 2024-04-24T08:59:36.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.940+0000 7fae4bdb4740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5630ee62b8f0 con 0x5630ee61e270 2024-04-24T08:59:36.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.941+0000 7fae4bdb4740 10 client.0 _ll_register_callbacks cb 0x5630ee50a140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:59:36.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.941+0000 7fae42800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5630ee628940 0x5630ee625dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:36.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.941+0000 7fae42800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5630ee628940 0x5630ee625dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44270/0 (socket says 172.21.15.83:44270) 2024-04-24T08:59:36.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.941+0000 7fae42800640 1 -- 172.21.15.83:0/2373031433 learned_addr learned my addr 172.21.15.83:0/2373031433 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:59:36.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.941+0000 7fae41e00640 1 --2- 172.21.15.83:0/2373031433 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5630ee61d950 0x5630ee6283e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:36.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.941+0000 7fae41400640 1 --2- 172.21.15.83:0/2373031433 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5630ee61e270 0x5630ee6256d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:36.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.941+0000 7fae42800640 1 -- 172.21.15.83:0/2373031433 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5630ee61e270 msgr2=0x5630ee6256d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:36.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.941+0000 7fae42800640 1 --2- 172.21.15.83:0/2373031433 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5630ee61e270 0x5630ee6256d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:36.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.941+0000 7fae42800640 1 -- 172.21.15.83:0/2373031433 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5630ee61d950 msgr2=0x5630ee6283e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:36.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.941+0000 7fae42800640 1 --2- 172.21.15.83:0/2373031433 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5630ee61d950 0x5630ee6283e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:36.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.941+0000 7fae42800640 1 -- 172.21.15.83:0/2373031433 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630ee54d3f0 con 0x5630ee628940 2024-04-24T08:59:36.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.941+0000 7fae41400640 1 --2- 172.21.15.83:0/2373031433 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5630ee61e270 0x5630ee6256d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:59:36.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.942+0000 7fae42800640 1 --2- 172.21.15.83:0/2373031433 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5630ee628940 0x5630ee625dc0 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7fae3c00cfd0 tx=0x7fae3c004bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=588eba5a229f8f19 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:36.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.942+0000 7fae23e00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:59:36.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.942+0000 7fae23e00640 1 -- 172.21.15.83:0/2373031433 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae3c027060 con 0x5630ee628940 2024-04-24T08:59:36.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.942+0000 7fae23e00640 1 -- 172.21.15.83:0/2373031433 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7fae3c012030 con 0x5630ee628940 2024-04-24T08:59:36.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.942+0000 7fae4bdb4740 1 -- 172.21.15.83:0/2373031433 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae30031760 con 0x5630ee628940 2024-04-24T08:59:36.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.942+0000 7fae23e00640 1 -- 172.21.15.83:0/2373031433 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fae3c02e050 con 0x5630ee628940 2024-04-24T08:59:36.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.942+0000 7fae4bdb4740 10 client.18498 fetch_fsmap learned FSMap version 35 2024-04-24T08:59:36.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.942+0000 7fae4bdb4740 1 -- 172.21.15.83:0/2373031433 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5630ee2953f0 con 0x5630ee628940 2024-04-24T08:59:36.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.943+0000 7fae23e00640 1 -- 172.21.15.83:0/2373031433 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fae30031760 con 0x5630ee628940 2024-04-24T08:59:36.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.943+0000 7fae23e00640 1 -- 172.21.15.83:0/2373031433 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7fae3c0227f0 con 0x5630ee628940 2024-04-24T08:59:36.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.943+0000 7fae4bdb4740 10 client.18498 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T08:59:36.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.943+0000 7fae4bdb4740 10 client.18498 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-24T08:59:36.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.943+0000 7fae4bdb4740 20 client.18498 populate_metadata read hostname 'smithi083' 2024-04-24T08:59:36.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.943+0000 7fae4bdb4740 1 --2- 172.21.15.83:0/2373031433 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x5630ee638ba0 0x5630ee63b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:36.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.943+0000 7fae4bdb4740 4 client.18498 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:59:36.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.943+0000 7fae4bdb4740 1 -- 172.21.15.83:0/2373031433 --> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5630ee590bf0 con 0x5630ee638ba0 2024-04-24T08:59:36.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.944+0000 7fae41e00640 1 --2- 172.21.15.83:0/2373031433 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x5630ee638ba0 0x5630ee63b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:36.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.945+0000 7fae41e00640 1 --2- 172.21.15.83:0/2373031433 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x5630ee638ba0 0x5630ee63b000 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=22dd627fdd665e6a server_cookie=d80ddbc00b78339b in_seq=0 out_seq=0 2024-04-24T08:59:36.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.945+0000 7fae23e00640 10 client.18498 ms_handle_connect on v2:172.21.15.138:6834/1909219758 2024-04-24T08:59:36.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.947+0000 7fae23e00640 1 -- 172.21.15.83:0/2373031433 <== mds.2 v2:172.21.15.138:6834/1909219758 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5630ee590bf0 con 0x5630ee638ba0 2024-04-24T08:59:36.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.947+0000 7fae23e00640 10 client.18498 handle_command_reply: tid=0 2024-04-24T08:59:36.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.947+0000 7fae4bdb4740 1 client.18498 shutdown 2024-04-24T08:59:36.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.947+0000 7fae4bdb4740 1 -- 172.21.15.83:0/2373031433 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5630ee628940 msgr2=0x5630ee625dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:36.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.947+0000 7fae4bdb4740 1 --2- 172.21.15.83:0/2373031433 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5630ee628940 0x5630ee625dc0 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7fae3c00cfd0 tx=0x7fae3c004bc0 comp rx=0 tx=0).stop 2024-04-24T08:59:36.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.947+0000 7fae4bdb4740 1 -- 172.21.15.83:0/2373031433 shutdown_connections 2024-04-24T08:59:36.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.947+0000 7fae4bdb4740 1 --2- 172.21.15.83:0/2373031433 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x5630ee638ba0 0x5630ee63b000 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:36.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.947+0000 7fae4bdb4740 1 --2- 172.21.15.83:0/2373031433 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5630ee628940 0x5630ee625dc0 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:36.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.947+0000 7fae4bdb4740 1 --2- 172.21.15.83:0/2373031433 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5630ee61e270 0x5630ee6256d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:36.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.947+0000 7fae4bdb4740 1 --2- 172.21.15.83:0/2373031433 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5630ee61d950 0x5630ee6283e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:36.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.947+0000 7fae4bdb4740 1 -- 172.21.15.83:0/2373031433 >> 172.21.15.83:0/2373031433 conn(0x5630ee619c70 msgr2=0x5630ee61fa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:36.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.947+0000 7fae23e00640 0 client.18498 ms_handle_reset on v2:172.21.15.138:6834/1909219758 2024-04-24T08:59:36.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.948+0000 7fae4bdb4740 1 -- 172.21.15.83:0/2373031433 shutdown_connections 2024-04-24T08:59:36.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.948+0000 7fae4bdb4740 1 -- 172.21.15.83:0/2373031433 wait complete. 2024-04-24T08:59:36.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.948+0000 7fae4bdb4740 20 client.18498 trim_cache size 0 max 16384 2024-04-24T08:59:36.954 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:59:36 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/1692481467' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:59:36.956 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:59:36 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/1692481467' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T08:59:36.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.965+0000 7fae4bdb4740 1 Processor -- start 2024-04-24T08:59:36.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.965+0000 7fae4bdb4740 1 -- start start 2024-04-24T08:59:36.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.966+0000 7fae4bdb4740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5630ee61d950 0x5630ee6200f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:36.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.966+0000 7fae4bdb4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5630ee61e270 0x5630ee627f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:36.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.966+0000 7fae4bdb4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5630ee628940 0x5630ee6408d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:36.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.966+0000 7fae4bdb4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fae441b1820 con 0x5630ee61e270 2024-04-24T08:59:36.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.966+0000 7fae4bdb4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fae441b3360 con 0x5630ee628940 2024-04-24T08:59:36.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.966+0000 7fae4bdb4740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fae441b1ef0 con 0x5630ee61d950 2024-04-24T08:59:36.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.966+0000 7fae41400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5630ee61e270 0x5630ee627f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:36.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.966+0000 7fae41400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5630ee61e270 0x5630ee627f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44274/0 (socket says 172.21.15.83:44274) 2024-04-24T08:59:36.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.966+0000 7fae41400640 1 -- 172.21.15.83:0/2991251056 learned_addr learned my addr 172.21.15.83:0/2991251056 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:59:36.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.967+0000 7fae41e00640 1 --2- 172.21.15.83:0/2991251056 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5630ee61d950 0x5630ee6200f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:36.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.967+0000 7fae41400640 1 -- 172.21.15.83:0/2991251056 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5630ee61d950 msgr2=0x5630ee6200f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:36.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.967+0000 7fae41400640 1 --2- 172.21.15.83:0/2991251056 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5630ee61d950 0x5630ee6200f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:36.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.967+0000 7fae41400640 1 -- 172.21.15.83:0/2991251056 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5630ee628940 msgr2=0x5630ee6408d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:36.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.967+0000 7fae41400640 1 --2- 172.21.15.83:0/2991251056 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5630ee628940 0x5630ee6408d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:36.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.967+0000 7fae41400640 1 -- 172.21.15.83:0/2991251056 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630ee35f7d0 con 0x5630ee61e270 2024-04-24T08:59:36.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.967+0000 7fae41400640 1 --2- 172.21.15.83:0/2991251056 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5630ee61e270 0x5630ee627f10 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7fae30006010 tx=0x7fae30032fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81903fbac4512bda server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:36.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.968+0000 7fae2ea00640 1 -- 172.21.15.83:0/2991251056 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae3003f050 con 0x5630ee61e270 2024-04-24T08:59:36.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.968+0000 7fae2ea00640 1 -- 172.21.15.83:0/2991251056 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fae30048070 con 0x5630ee61e270 2024-04-24T08:59:36.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.968+0000 7fae2ea00640 1 -- 172.21.15.83:0/2991251056 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae30035040 con 0x5630ee61e270 2024-04-24T08:59:36.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.969+0000 7fae4bdb4740 1 -- 172.21.15.83:0/2991251056 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5630ee61e270 msgr2=0x5630ee627f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:36.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.969+0000 7fae4bdb4740 1 --2- 172.21.15.83:0/2991251056 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5630ee61e270 0x5630ee627f10 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7fae30006010 tx=0x7fae30032fc0 comp rx=0 tx=0).stop 2024-04-24T08:59:36.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.969+0000 7fae4bdb4740 1 -- 172.21.15.83:0/2991251056 shutdown_connections 2024-04-24T08:59:36.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.969+0000 7fae4bdb4740 1 --2- 172.21.15.83:0/2991251056 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5630ee628940 0x5630ee6408d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:36.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.969+0000 7fae4bdb4740 1 --2- 172.21.15.83:0/2991251056 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5630ee61e270 0x5630ee627f10 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:36.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.969+0000 7fae4bdb4740 1 --2- 172.21.15.83:0/2991251056 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5630ee61d950 0x5630ee6200f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:36.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.969+0000 7fae4bdb4740 1 -- 172.21.15.83:0/2991251056 >> 172.21.15.83:0/2991251056 conn(0x5630ee619c70 msgr2=0x5630ee620a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:36.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.969+0000 7fae4bdb4740 1 -- 172.21.15.83:0/2991251056 shutdown_connections 2024-04-24T08:59:36.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.969+0000 7fae4bdb4740 1 -- 172.21.15.83:0/2991251056 wait complete. 2024-04-24T08:59:36.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.970+0000 7fae4bdb4740 1 -- start start 2024-04-24T08:59:36.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.970+0000 7fae4bdb4740 1 Processor -- start 2024-04-24T08:59:36.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.970+0000 7fae4bdb4740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5630ee61d950 0x5630ee629770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:36.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.970+0000 7fae4bdb4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5630ee61e270 0x5630ee629cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:36.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.970+0000 7fae4bdb4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5630ee628940 0x5630ee62a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:36.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.970+0000 7fae4bdb4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fae441b2c90 con 0x5630ee61e270 2024-04-24T08:59:36.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.970+0000 7fae4bdb4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fae441b25c0 con 0x5630ee628940 2024-04-24T08:59:36.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.970+0000 7fae4bdb4740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fae441af820 con 0x5630ee61d950 2024-04-24T08:59:36.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.970+0000 7fae4bdb4740 10 client.0 _ll_register_callbacks cb 0x7fae3c009e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:59:36.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.970+0000 7fae41400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5630ee61e270 0x5630ee629cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:36.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.971+0000 7fae41400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5630ee61e270 0x5630ee629cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44286/0 (socket says 172.21.15.83:44286) 2024-04-24T08:59:36.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.971+0000 7fae41400640 1 -- 172.21.15.83:0/2961509143 learned_addr learned my addr 172.21.15.83:0/2961509143 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:59:36.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.971+0000 7fae41e00640 1 --2- 172.21.15.83:0/2961509143 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5630ee61d950 0x5630ee629770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:36.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.971+0000 7fae42800640 1 --2- 172.21.15.83:0/2961509143 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5630ee628940 0x5630ee62a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:36.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.971+0000 7fae41400640 1 -- 172.21.15.83:0/2961509143 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5630ee61d950 msgr2=0x5630ee629770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:36.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.971+0000 7fae41400640 1 --2- 172.21.15.83:0/2961509143 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5630ee61d950 0x5630ee629770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:36.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.971+0000 7fae41400640 1 -- 172.21.15.83:0/2961509143 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5630ee628940 msgr2=0x5630ee62a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:36.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.971+0000 7fae41400640 1 --2- 172.21.15.83:0/2961509143 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5630ee628940 0x5630ee62a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:36.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.971+0000 7fae41400640 1 -- 172.21.15.83:0/2961509143 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630ee62ecf0 con 0x5630ee61e270 2024-04-24T08:59:36.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.971+0000 7fae42800640 1 --2- 172.21.15.83:0/2961509143 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5630ee628940 0x5630ee62a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:59:36.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.971+0000 7fae41400640 1 --2- 172.21.15.83:0/2961509143 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5630ee61e270 0x5630ee629cb0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7fae30000c00 tx=0x7fae30031da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71cc1695548ec4b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:36.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.971+0000 7fae23e00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:59:36.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.972+0000 7fae23e00640 1 -- 172.21.15.83:0/2961509143 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae30035030 con 0x5630ee61e270 2024-04-24T08:59:36.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.972+0000 7fae23e00640 1 -- 172.21.15.83:0/2961509143 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7fae3004d470 con 0x5630ee61e270 2024-04-24T08:59:36.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.972+0000 7fae4bdb4740 1 -- 172.21.15.83:0/2961509143 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae30048070 con 0x5630ee61e270 2024-04-24T08:59:36.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.972+0000 7fae23e00640 1 -- 172.21.15.83:0/2961509143 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fae3005d050 con 0x5630ee61e270 2024-04-24T08:59:36.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.972+0000 7fae4bdb4740 10 client.18504 fetch_fsmap learned FSMap version 35 2024-04-24T08:59:36.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.972+0000 7fae4bdb4740 1 -- 172.21.15.83:0/2961509143 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5630ee69f3f0 con 0x5630ee61e270 2024-04-24T08:59:36.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.972+0000 7fae23e00640 1 -- 172.21.15.83:0/2961509143 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fae30048070 con 0x5630ee61e270 2024-04-24T08:59:36.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.972+0000 7fae23e00640 1 -- 172.21.15.83:0/2961509143 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7fae3003b7c0 con 0x5630ee61e270 2024-04-24T08:59:36.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.973+0000 7fae4bdb4740 10 client.18504 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T08:59:36.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.973+0000 7fae4bdb4740 10 client.18504 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-24T08:59:36.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.973+0000 7fae4bdb4740 20 client.18504 populate_metadata read hostname 'smithi083' 2024-04-24T08:59:36.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.973+0000 7fae4bdb4740 1 --2- 172.21.15.83:0/2961509143 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x5630ee69f670 0x5630ee6a1a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:36.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.973+0000 7fae4bdb4740 4 client.18504 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T08:59:36.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.973+0000 7fae4bdb4740 1 -- 172.21.15.83:0/2961509143 --> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5630ee58aa60 con 0x5630ee69f670 2024-04-24T08:59:36.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.973+0000 7fae41e00640 1 --2- 172.21.15.83:0/2961509143 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x5630ee69f670 0x5630ee6a1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:36.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.974+0000 7fae41e00640 1 --2- 172.21.15.83:0/2961509143 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x5630ee69f670 0x5630ee6a1a60 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=800a6660556706a1 server_cookie=d07b125cbdfc0a93 in_seq=0 out_seq=0 2024-04-24T08:59:36.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.974+0000 7fae23e00640 10 client.18504 ms_handle_connect on v2:172.21.15.138:6834/1909219758 2024-04-24T08:59:36.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.975+0000 7fae23e00640 1 -- 172.21.15.83:0/2961509143 <== mds.2 v2:172.21.15.138:6834/1909219758 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5630ee58aa60 con 0x5630ee69f670 2024-04-24T08:59:36.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.975+0000 7fae23e00640 10 client.18504 handle_command_reply: tid=0 2024-04-24T08:59:36.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.975+0000 7fae4bdb4740 1 client.18504 shutdown 2024-04-24T08:59:36.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.975+0000 7fae4bdb4740 1 -- 172.21.15.83:0/2961509143 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5630ee61e270 msgr2=0x5630ee629cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:36.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.975+0000 7fae4bdb4740 1 --2- 172.21.15.83:0/2961509143 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5630ee61e270 0x5630ee629cb0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7fae30000c00 tx=0x7fae30031da0 comp rx=0 tx=0).stop 2024-04-24T08:59:36.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.975+0000 7fae4bdb4740 1 -- 172.21.15.83:0/2961509143 shutdown_connections 2024-04-24T08:59:36.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.975+0000 7fae4bdb4740 1 --2- 172.21.15.83:0/2961509143 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x5630ee69f670 0x5630ee6a1a60 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:36.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.975+0000 7fae4bdb4740 1 --2- 172.21.15.83:0/2961509143 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5630ee628940 0x5630ee62a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:36.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.975+0000 7fae4bdb4740 1 --2- 172.21.15.83:0/2961509143 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5630ee61e270 0x5630ee629cb0 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:36.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.975+0000 7fae4bdb4740 1 --2- 172.21.15.83:0/2961509143 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5630ee61d950 0x5630ee629770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:36.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.975+0000 7fae23e00640 0 client.18504 ms_handle_reset on v2:172.21.15.138:6834/1909219758 2024-04-24T08:59:36.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.975+0000 7fae4bdb4740 1 -- 172.21.15.83:0/2961509143 >> 172.21.15.83:0/2961509143 conn(0x5630ee619c70 msgr2=0x5630ee618560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:36.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.975+0000 7fae4bdb4740 1 -- 172.21.15.83:0/2961509143 shutdown_connections 2024-04-24T08:59:36.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.975+0000 7fae4bdb4740 1 -- 172.21.15.83:0/2961509143 wait complete. 2024-04-24T08:59:36.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.975+0000 7fae4bdb4740 20 client.18504 trim_cache size 0 max 16384 2024-04-24T08:59:36.976 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T08:59:36.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.976+0000 7fae49000640 1 -- 172.21.15.83:0/102419036 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fae280a3da0 msgr2=0x7fae280a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:36.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.976+0000 7fae49000640 1 --2- 172.21.15.83:0/102419036 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fae280a3da0 0x7fae280a6260 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fae3c009e40 tx=0x7fae3c009430 comp rx=0 tx=0).stop 2024-04-24T08:59:36.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.976+0000 7fae49000640 1 -- 172.21.15.83:0/102419036 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae441044b0 msgr2=0x7fae441a06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:36.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.976+0000 7fae49000640 1 --2- 172.21.15.83:0/102419036 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae441044b0 0x7fae441a06f0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7fae3400c920 tx=0x7fae3400cde0 comp rx=0 tx=0).stop 2024-04-24T08:59:36.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.976+0000 7fae49000640 1 -- 172.21.15.83:0/102419036 shutdown_connections 2024-04-24T08:59:36.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.976+0000 7fae49000640 1 --2- 172.21.15.83:0/102419036 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fae280a3da0 0x7fae280a6260 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:36.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.976+0000 7fae49000640 1 --2- 172.21.15.83:0/102419036 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fae4419a8a0 0x7fae4419ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:36.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.976+0000 7fae49000640 1 --2- 172.21.15.83:0/102419036 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae44109580 0x7fae441a0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:36.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.976+0000 7fae49000640 1 --2- 172.21.15.83:0/102419036 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae441044b0 0x7fae441a06f0 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:36.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.976+0000 7fae49000640 1 -- 172.21.15.83:0/102419036 >> 172.21.15.83:0/102419036 conn(0x7fae440ffd90 msgr2=0x7fae44102090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:36.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.976+0000 7fae49000640 1 -- 172.21.15.83:0/102419036 shutdown_connections 2024-04-24T08:59:36.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:36.976+0000 7fae49000640 1 -- 172.21.15.83:0/102419036 wait complete. 2024-04-24T08:59:36.993 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T08:59:37.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.187+0000 7eff2be00640 1 -- 172.21.15.83:0/3789293189 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff2c10c480 msgr2=0x7eff2c10e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:37.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.187+0000 7eff2be00640 1 --2- 172.21.15.83:0/3789293189 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff2c10c480 0x7eff2c10e920 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7eff24009900 tx=0x7eff2402a480 comp rx=0 tx=0).stop 2024-04-24T08:59:37.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.187+0000 7eff2be00640 1 -- 172.21.15.83:0/3789293189 shutdown_connections 2024-04-24T08:59:37.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.187+0000 7eff2be00640 1 --2- 172.21.15.83:0/3789293189 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff2c10c480 0x7eff2c10e920 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:37.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.188+0000 7eff2be00640 1 --2- 172.21.15.83:0/3789293189 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff2c103980 0x7eff2c10be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:37.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.188+0000 7eff2be00640 1 --2- 172.21.15.83:0/3789293189 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7eff2c103060 0x7eff2c103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:37.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.188+0000 7eff2be00640 1 -- 172.21.15.83:0/3789293189 >> 172.21.15.83:0/3789293189 conn(0x7eff2c0fc880 msgr2=0x7eff2c0feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:37.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.188+0000 7eff2be00640 1 -- 172.21.15.83:0/3789293189 shutdown_connections 2024-04-24T08:59:37.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.188+0000 7eff2be00640 1 -- 172.21.15.83:0/3789293189 wait complete. 2024-04-24T08:59:37.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.189+0000 7eff2be00640 1 Processor -- start 2024-04-24T08:59:37.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.189+0000 7eff2be00640 1 -- start start 2024-04-24T08:59:37.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.189+0000 7eff2be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff2c103060 0x7eff2c100cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:37.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.189+0000 7eff2be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff2c103980 0x7eff2c1011f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:37.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.189+0000 7eff2be00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7eff2c10c480 0x7eff2c101730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:37.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.189+0000 7eff2be00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7eff2c112110 con 0x7eff2c103060 2024-04-24T08:59:37.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.189+0000 7eff2be00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7eff2c111f90 con 0x7eff2c103980 2024-04-24T08:59:37.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.189+0000 7eff2be00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7eff2c112290 con 0x7eff2c10c480 2024-04-24T08:59:37.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.190+0000 7eff2aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff2c103060 0x7eff2c100cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:37.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.190+0000 7eff2aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff2c103060 0x7eff2c100cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44298/0 (socket says 172.21.15.83:44298) 2024-04-24T08:59:37.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.190+0000 7eff2aa00640 1 -- 172.21.15.83:0/1356587120 learned_addr learned my addr 172.21.15.83:0/1356587120 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:59:37.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.190+0000 7eff22000640 1 --2- 172.21.15.83:0/1356587120 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff2c103980 0x7eff2c1011f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:37.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.190+0000 7eff2aa00640 1 -- 172.21.15.83:0/1356587120 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7eff2c10c480 msgr2=0x7eff2c101730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:37.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.190+0000 7eff2b400640 1 --2- 172.21.15.83:0/1356587120 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7eff2c10c480 0x7eff2c101730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:37.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.190+0000 7eff2aa00640 1 --2- 172.21.15.83:0/1356587120 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7eff2c10c480 0x7eff2c101730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:37.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.190+0000 7eff2aa00640 1 -- 172.21.15.83:0/1356587120 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff2c103980 msgr2=0x7eff2c1011f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:37.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.190+0000 7eff2aa00640 1 --2- 172.21.15.83:0/1356587120 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff2c103980 0x7eff2c1011f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:37.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.190+0000 7eff2aa00640 1 -- 172.21.15.83:0/1356587120 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff2c1a6b30 con 0x7eff2c103060 2024-04-24T08:59:37.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.190+0000 7eff22000640 1 --2- 172.21.15.83:0/1356587120 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff2c103980 0x7eff2c1011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:59:37.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.190+0000 7eff2aa00640 1 --2- 172.21.15.83:0/1356587120 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff2c103060 0x7eff2c100cb0 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7eff1c00ea80 tx=0x7eff1c00ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:37.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.191+0000 7eff23e00640 1 -- 172.21.15.83:0/1356587120 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff1c00ce40 con 0x7eff2c103060 2024-04-24T08:59:37.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.191+0000 7eff23e00640 1 -- 172.21.15.83:0/1356587120 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7eff1c005020 con 0x7eff2c103060 2024-04-24T08:59:37.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.191+0000 7eff2be00640 1 -- 172.21.15.83:0/1356587120 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff2c1a6e20 con 0x7eff2c103060 2024-04-24T08:59:37.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.191+0000 7eff23e00640 1 -- 172.21.15.83:0/1356587120 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff1c0116d0 con 0x7eff2c103060 2024-04-24T08:59:37.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.191+0000 7eff2be00640 1 -- 172.21.15.83:0/1356587120 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff2c1a7290 con 0x7eff2c103060 2024-04-24T08:59:37.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.195+0000 7eff345b1740 1 Processor -- start 2024-04-24T08:59:37.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.195+0000 7eff345b1740 1 -- start start 2024-04-24T08:59:37.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.195+0000 7eff23e00640 1 -- 172.21.15.83:0/1356587120 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7eff1c0051d0 con 0x7eff2c103060 2024-04-24T08:59:37.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.195+0000 7eff345b1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b960afb40 0x563b960aff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:37.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.195+0000 7eff345b1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b960b0460 0x563b960ba5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:37.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.195+0000 7eff345b1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563b960bab30 0x563b960bcf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:37.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.195+0000 7eff345b1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x563b960151d0 con 0x563b960b0460 2024-04-24T08:59:37.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.195+0000 7eff345b1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563b95fbb450 con 0x563b960bab30 2024-04-24T08:59:37.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.195+0000 7eff345b1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x563b96011190 con 0x563b960afb40 2024-04-24T08:59:37.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.196+0000 7eff2aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b960b0460 0x563b960ba5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:37.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.196+0000 7eff2aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b960b0460 0x563b960ba5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44312/0 (socket says 172.21.15.83:44312) 2024-04-24T08:59:37.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.196+0000 7eff2aa00640 1 -- 172.21.15.83:0/544220414 learned_addr learned my addr 172.21.15.83:0/544220414 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:59:37.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.196+0000 7eff22000640 1 --2- 172.21.15.83:0/544220414 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563b960bab30 0x563b960bcf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:37.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.196+0000 7eff2b400640 1 --2- 172.21.15.83:0/544220414 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b960afb40 0x563b960aff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:37.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.196+0000 7eff2aa00640 1 -- 172.21.15.83:0/544220414 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b960afb40 msgr2=0x563b960aff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:37.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.196+0000 7eff2aa00640 1 --2- 172.21.15.83:0/544220414 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b960afb40 0x563b960aff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:37.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.196+0000 7eff2aa00640 1 -- 172.21.15.83:0/544220414 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563b960bab30 msgr2=0x563b960bcf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:37.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.196+0000 7eff2aa00640 1 --2- 172.21.15.83:0/544220414 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563b960bab30 0x563b960bcf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:37.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.196+0000 7eff2aa00640 1 -- 172.21.15.83:0/544220414 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b96078310 con 0x563b960b0460 2024-04-24T08:59:37.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.196+0000 7eff2b400640 1 --2- 172.21.15.83:0/544220414 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b960afb40 0x563b960aff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:59:37.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.197+0000 7eff2aa00640 1 --2- 172.21.15.83:0/544220414 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b960b0460 0x563b960ba5f0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7eff1c0ce210 tx=0x7eff1c0d9780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a748f3c52fd6feba server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:37.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.197+0000 7eff23e00640 1 --2- 172.21.15.83:0/1356587120 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7eff100a3d50 0x7eff100a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:37.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.197+0000 7eff1b400640 1 -- 172.21.15.83:0/544220414 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff1c0e4070 con 0x563b960b0460 2024-04-24T08:59:37.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.197+0000 7eff1b400640 1 -- 172.21.15.83:0/544220414 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7eff1c01d120 con 0x563b960b0460 2024-04-24T08:59:37.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.197+0000 7eff1b400640 1 -- 172.21.15.83:0/544220414 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff1c09c630 con 0x563b960b0460 2024-04-24T08:59:37.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.197+0000 7eff23e00640 1 -- 172.21.15.83:0/1356587120 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7eff1c0d0f70 con 0x7eff2c103060 2024-04-24T08:59:37.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.197+0000 7eff2b400640 1 --2- 172.21.15.83:0/1356587120 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7eff100a3d50 0x7eff100a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:37.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.198+0000 7eff345b1740 1 -- 172.21.15.83:0/544220414 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b960b0460 msgr2=0x563b960ba5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:37.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.198+0000 7eff345b1740 1 --2- 172.21.15.83:0/544220414 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b960b0460 0x563b960ba5f0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7eff1c0ce210 tx=0x7eff1c0d9780 comp rx=0 tx=0).stop 2024-04-24T08:59:37.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.198+0000 7eff2b400640 1 --2- 172.21.15.83:0/1356587120 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7eff100a3d50 0x7eff100a6210 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7eff24005ab0 tx=0x7eff24005a20 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:37.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.198+0000 7eff345b1740 1 -- 172.21.15.83:0/544220414 shutdown_connections 2024-04-24T08:59:37.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.198+0000 7eff345b1740 1 --2- 172.21.15.83:0/544220414 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563b960bab30 0x563b960bcf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:37.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.198+0000 7eff345b1740 1 --2- 172.21.15.83:0/544220414 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b960b0460 0x563b960ba5f0 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:37.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.198+0000 7eff345b1740 1 --2- 172.21.15.83:0/544220414 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b960afb40 0x563b960aff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:37.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.198+0000 7eff345b1740 1 -- 172.21.15.83:0/544220414 >> 172.21.15.83:0/544220414 conn(0x563b960abe60 msgr2=0x563b960ac240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:37.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.198+0000 7eff345b1740 1 -- 172.21.15.83:0/544220414 shutdown_connections 2024-04-24T08:59:37.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.198+0000 7eff345b1740 1 -- 172.21.15.83:0/544220414 wait complete. 2024-04-24T08:59:37.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.199+0000 7eff345b1740 1 -- start start 2024-04-24T08:59:37.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.199+0000 7eff345b1740 1 Processor -- start 2024-04-24T08:59:37.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.199+0000 7eff345b1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b960afb40 0x563b960ba590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:37.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.199+0000 7eff345b1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b960b0460 0x563b960b7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:37.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.199+0000 7eff345b1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563b960bab30 0x563b960b7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:37.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.199+0000 7eff345b1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x563b960bd9c0 con 0x563b960afb40 2024-04-24T08:59:37.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.199+0000 7eff345b1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563b95fce810 con 0x563b960bab30 2024-04-24T08:59:37.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.199+0000 7eff345b1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x563b960bdb40 con 0x563b960b0460 2024-04-24T08:59:37.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.199+0000 7eff345b1740 10 client.0 _ll_register_callbacks cb 0x563b95f9c140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:59:37.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.200+0000 7eff2aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b960afb40 0x563b960ba590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:37.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.200+0000 7eff2aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b960afb40 0x563b960ba590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44328/0 (socket says 172.21.15.83:44328) 2024-04-24T08:59:37.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.200+0000 7eff2aa00640 1 -- 172.21.15.83:0/3102152974 learned_addr learned my addr 172.21.15.83:0/3102152974 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:59:37.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.200+0000 7eff22000640 1 --2- 172.21.15.83:0/3102152974 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b960b0460 0x563b960b7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:37.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.200+0000 7eff2aa00640 1 -- 172.21.15.83:0/3102152974 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b960b0460 msgr2=0x563b960b7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:37.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.200+0000 7eff2aa00640 1 --2- 172.21.15.83:0/3102152974 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b960b0460 0x563b960b7880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:37.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.200+0000 7eff2aa00640 1 -- 172.21.15.83:0/3102152974 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563b960bab30 msgr2=0x563b960b7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:37.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.200+0000 7eff2b400640 1 --2- 172.21.15.83:0/3102152974 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563b960bab30 0x563b960b7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:37.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.200+0000 7eff2aa00640 1 --2- 172.21.15.83:0/3102152974 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563b960bab30 0x563b960b7fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:37.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.200+0000 7eff2aa00640 1 -- 172.21.15.83:0/3102152974 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b95fdf3f0 con 0x563b960afb40 2024-04-24T08:59:37.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.200+0000 7eff2b400640 1 --2- 172.21.15.83:0/3102152974 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563b960bab30 0x563b960b7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:59:37.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.200+0000 7eff2aa00640 1 --2- 172.21.15.83:0/3102152974 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b960afb40 0x563b960ba590 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7eff1c098f50 tx=0x7eff1c01fc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95514bc84349e62a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:37.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.200+0000 7eff0fe00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:59:37.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.201+0000 7eff0fe00640 1 -- 172.21.15.83:0/3102152974 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff1c01f680 con 0x563b960afb40 2024-04-24T08:59:37.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.201+0000 7eff0fe00640 1 -- 172.21.15.83:0/3102152974 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7eff1c0d4cd0 con 0x563b960afb40 2024-04-24T08:59:37.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.201+0000 7eff345b1740 1 -- 172.21.15.83:0/3102152974 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff1c01d120 con 0x563b960afb40 2024-04-24T08:59:37.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.201+0000 7eff0fe00640 1 -- 172.21.15.83:0/3102152974 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7eff1c01f8a0 con 0x563b960afb40 2024-04-24T08:59:37.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.201+0000 7eff345b1740 10 client.18516 fetch_fsmap learned FSMap version 35 2024-04-24T08:59:37.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.201+0000 7eff345b1740 1 -- 172.21.15.83:0/3102152974 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x563b95d273f0 con 0x563b960afb40 2024-04-24T08:59:37.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.201+0000 7eff0fe00640 1 -- 172.21.15.83:0/3102152974 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7eff1c01d120 con 0x563b960afb40 2024-04-24T08:59:37.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.201+0000 7eff0fe00640 1 -- 172.21.15.83:0/3102152974 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7eff1c00f040 con 0x563b960afb40 2024-04-24T08:59:37.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.201+0000 7eff345b1740 10 client.18516 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T08:59:37.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.201+0000 7eff345b1740 10 client.18516 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-24T08:59:37.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.202+0000 7eff345b1740 20 client.18516 populate_metadata read hostname 'smithi083' 2024-04-24T08:59:37.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.202+0000 7eff345b1740 1 --2- 172.21.15.83:0/3102152974 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x563b960cabe0 0x563b960ccfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:37.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.202+0000 7eff345b1740 4 client.18516 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:59:37.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.202+0000 7eff345b1740 1 -- 172.21.15.83:0/3102152974 --> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563b96022bf0 con 0x563b960cabe0 2024-04-24T08:59:37.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.202+0000 7eff22000640 1 --2- 172.21.15.83:0/3102152974 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x563b960cabe0 0x563b960ccfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:37.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.203+0000 7eff22000640 1 --2- 172.21.15.83:0/3102152974 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x563b960cabe0 0x563b960ccfd0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=48840ed189c2ce0a server_cookie=80ddfe95597425cc in_seq=0 out_seq=0 2024-04-24T08:59:37.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.203+0000 7eff0fe00640 10 client.18516 ms_handle_connect on v2:172.21.15.138:6836/900566754 2024-04-24T08:59:37.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.205+0000 7eff0fe00640 1 -- 172.21.15.83:0/3102152974 <== mds.1 v2:172.21.15.138:6836/900566754 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563b96022bf0 con 0x563b960cabe0 2024-04-24T08:59:37.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.205+0000 7eff0fe00640 10 client.18516 handle_command_reply: tid=0 2024-04-24T08:59:37.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.205+0000 7eff345b1740 1 client.18516 shutdown 2024-04-24T08:59:37.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.205+0000 7eff345b1740 1 -- 172.21.15.83:0/3102152974 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b960afb40 msgr2=0x563b960ba590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:37.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.205+0000 7eff345b1740 1 --2- 172.21.15.83:0/3102152974 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b960afb40 0x563b960ba590 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7eff1c098f50 tx=0x7eff1c01fc60 comp rx=0 tx=0).stop 2024-04-24T08:59:37.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.205+0000 7eff345b1740 1 -- 172.21.15.83:0/3102152974 shutdown_connections 2024-04-24T08:59:37.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.205+0000 7eff345b1740 1 --2- 172.21.15.83:0/3102152974 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563b960bab30 0x563b960b7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:37.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.205+0000 7eff345b1740 1 --2- 172.21.15.83:0/3102152974 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x563b960cabe0 0x563b960ccfd0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:37.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.206+0000 7eff345b1740 1 --2- 172.21.15.83:0/3102152974 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b960b0460 0x563b960b7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:37.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.206+0000 7eff345b1740 1 --2- 172.21.15.83:0/3102152974 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b960afb40 0x563b960ba590 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:37.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.206+0000 7eff0fe00640 0 client.18516 ms_handle_reset on v2:172.21.15.138:6836/900566754 2024-04-24T08:59:37.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.206+0000 7eff345b1740 1 -- 172.21.15.83:0/3102152974 >> 172.21.15.83:0/3102152974 conn(0x563b960abe60 msgr2=0x563b960b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:37.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.206+0000 7eff345b1740 1 -- 172.21.15.83:0/3102152974 shutdown_connections 2024-04-24T08:59:37.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.206+0000 7eff345b1740 1 -- 172.21.15.83:0/3102152974 wait complete. 2024-04-24T08:59:37.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.206+0000 7eff345b1740 20 client.18516 trim_cache size 0 max 16384 2024-04-24T08:59:37.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.222+0000 7eff345b1740 1 Processor -- start 2024-04-24T08:59:37.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.222+0000 7eff345b1740 1 -- start start 2024-04-24T08:59:37.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.222+0000 7eff345b1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b960afb40 0x563b960b22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:37.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.222+0000 7eff345b1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563b960b0460 0x563b960ba0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:37.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.222+0000 7eff345b1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b960bab30 0x563b960d25a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:37.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.222+0000 7eff345b1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7eff2c1b1610 con 0x563b960bab30 2024-04-24T08:59:37.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.222+0000 7eff345b1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7eff2c1b3150 con 0x563b960b0460 2024-04-24T08:59:37.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.222+0000 7eff345b1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7eff2c1b1ce0 con 0x563b960afb40 2024-04-24T08:59:37.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.222+0000 7eff2b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b960bab30 0x563b960d25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:37.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.222+0000 7eff2b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b960bab30 0x563b960d25a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44336/0 (socket says 172.21.15.83:44336) 2024-04-24T08:59:37.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.222+0000 7eff2b400640 1 -- 172.21.15.83:0/1980818166 learned_addr learned my addr 172.21.15.83:0/1980818166 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:59:37.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.223+0000 7eff2aa00640 1 --2- 172.21.15.83:0/1980818166 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b960afb40 0x563b960b22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:37.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.223+0000 7eff22000640 1 --2- 172.21.15.83:0/1980818166 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563b960b0460 0x563b960ba0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:37.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.223+0000 7eff2b400640 1 -- 172.21.15.83:0/1980818166 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b960afb40 msgr2=0x563b960b22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:37.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.223+0000 7eff2b400640 1 --2- 172.21.15.83:0/1980818166 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b960afb40 0x563b960b22a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:37.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.223+0000 7eff2b400640 1 -- 172.21.15.83:0/1980818166 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563b960b0460 msgr2=0x563b960ba0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:37.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.223+0000 7eff2b400640 1 --2- 172.21.15.83:0/1980818166 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563b960b0460 0x563b960ba0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:37.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.223+0000 7eff2b400640 1 -- 172.21.15.83:0/1980818166 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b960b8310 con 0x563b960bab30 2024-04-24T08:59:37.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.223+0000 7eff2aa00640 1 --2- 172.21.15.83:0/1980818166 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b960afb40 0x563b960b22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:59:37.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.223+0000 7eff2b400640 1 --2- 172.21.15.83:0/1980818166 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b960bab30 0x563b960d25a0 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7eff24033000 tx=0x7eff2403c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab98192deedc8e80 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:37.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.224+0000 7eff1aa00640 1 -- 172.21.15.83:0/1980818166 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff24035050 con 0x563b960bab30 2024-04-24T08:59:37.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.224+0000 7eff1aa00640 1 -- 172.21.15.83:0/1980818166 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7eff2404d070 con 0x563b960bab30 2024-04-24T08:59:37.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.224+0000 7eff1aa00640 1 -- 172.21.15.83:0/1980818166 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff24004c40 con 0x563b960bab30 2024-04-24T08:59:37.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.224+0000 7eff345b1740 1 -- 172.21.15.83:0/1980818166 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b960bab30 msgr2=0x563b960d25a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:37.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.224+0000 7eff345b1740 1 --2- 172.21.15.83:0/1980818166 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b960bab30 0x563b960d25a0 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7eff24033000 tx=0x7eff2403c000 comp rx=0 tx=0).stop 2024-04-24T08:59:37.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.224+0000 7eff345b1740 1 -- 172.21.15.83:0/1980818166 shutdown_connections 2024-04-24T08:59:37.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.224+0000 7eff345b1740 1 --2- 172.21.15.83:0/1980818166 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b960bab30 0x563b960d25a0 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:37.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.224+0000 7eff345b1740 1 --2- 172.21.15.83:0/1980818166 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563b960b0460 0x563b960ba0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:37.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.224+0000 7eff345b1740 1 --2- 172.21.15.83:0/1980818166 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b960afb40 0x563b960b22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:37.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.224+0000 7eff345b1740 1 -- 172.21.15.83:0/1980818166 >> 172.21.15.83:0/1980818166 conn(0x563b960abe60 msgr2=0x563b960aa6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:37.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.224+0000 7eff345b1740 1 -- 172.21.15.83:0/1980818166 shutdown_connections 2024-04-24T08:59:37.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.225+0000 7eff345b1740 1 -- 172.21.15.83:0/1980818166 wait complete. 2024-04-24T08:59:37.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.225+0000 7eff345b1740 1 -- start start 2024-04-24T08:59:37.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.225+0000 7eff345b1740 1 Processor -- start 2024-04-24T08:59:37.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.226+0000 7eff345b1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b960afb40 0x563b960bbed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:37.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.226+0000 7eff345b1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563b960b0460 0x563b960bc410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:37.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.226+0000 7eff345b1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b960bab30 0x563b960c1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:37.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.226+0000 7eff345b1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7eff2c1b2a80 con 0x563b960afb40 2024-04-24T08:59:37.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.226+0000 7eff345b1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7eff2c1b23b0 con 0x563b960b0460 2024-04-24T08:59:37.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.226+0000 7eff345b1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7eff2c1af610 con 0x563b960bab30 2024-04-24T08:59:37.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.226+0000 7eff345b1740 10 client.0 _ll_register_callbacks cb 0x7eff1c0e2070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:59:37.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.226+0000 7eff2aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b960afb40 0x563b960bbed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:37.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.226+0000 7eff2aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b960afb40 0x563b960bbed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44348/0 (socket says 172.21.15.83:44348) 2024-04-24T08:59:37.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.226+0000 7eff2aa00640 1 -- 172.21.15.83:0/2188273316 learned_addr learned my addr 172.21.15.83:0/2188273316 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:59:37.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.226+0000 7eff22000640 1 --2- 172.21.15.83:0/2188273316 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563b960b0460 0x563b960bc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:37.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.226+0000 7eff2b400640 1 --2- 172.21.15.83:0/2188273316 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b960bab30 0x563b960c1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:37.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.226+0000 7eff2aa00640 1 -- 172.21.15.83:0/2188273316 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b960bab30 msgr2=0x563b960c1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:37.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.226+0000 7eff2aa00640 1 --2- 172.21.15.83:0/2188273316 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b960bab30 0x563b960c1450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:37.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.226+0000 7eff2aa00640 1 -- 172.21.15.83:0/2188273316 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563b960b0460 msgr2=0x563b960bc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:37.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.226+0000 7eff2aa00640 1 --2- 172.21.15.83:0/2188273316 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563b960b0460 0x563b960bc410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:37.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.227+0000 7eff2aa00640 1 -- 172.21.15.83:0/2188273316 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b95df17d0 con 0x563b960afb40 2024-04-24T08:59:37.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.227+0000 7eff2aa00640 1 --2- 172.21.15.83:0/2188273316 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b960afb40 0x563b960bbed0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7eff1c09c630 tx=0x7eff1c005b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d988b6f55727f317 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:37.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.227+0000 7eff0fe00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:59:37.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.227+0000 7eff0fe00640 1 -- 172.21.15.83:0/2188273316 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff1c01f040 con 0x563b960afb40 2024-04-24T08:59:37.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.227+0000 7eff0fe00640 1 -- 172.21.15.83:0/2188273316 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7eff1c099160 con 0x563b960afb40 2024-04-24T08:59:37.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.227+0000 7eff345b1740 1 -- 172.21.15.83:0/2188273316 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff2404d070 con 0x563b960afb40 2024-04-24T08:59:37.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.227+0000 7eff0fe00640 1 -- 172.21.15.83:0/2188273316 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7eff1c01fac0 con 0x563b960afb40 2024-04-24T08:59:37.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.227+0000 7eff345b1740 10 client.18522 fetch_fsmap learned FSMap version 35 2024-04-24T08:59:37.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.227+0000 7eff345b1740 1 -- 172.21.15.83:0/2188273316 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x563b95df17d0 con 0x563b960afb40 2024-04-24T08:59:37.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.228+0000 7eff0fe00640 1 -- 172.21.15.83:0/2188273316 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7eff2404d070 con 0x563b960afb40 2024-04-24T08:59:37.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.228+0000 7eff0fe00640 1 -- 172.21.15.83:0/2188273316 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7eff1c004be0 con 0x563b960afb40 2024-04-24T08:59:37.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.228+0000 7eff345b1740 10 client.18522 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T08:59:37.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.228+0000 7eff345b1740 10 client.18522 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-24T08:59:37.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.228+0000 7eff345b1740 20 client.18522 populate_metadata read hostname 'smithi083' 2024-04-24T08:59:37.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.228+0000 7eff345b1740 1 --2- 172.21.15.83:0/2188273316 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x563b961317a0 0x563b96133be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:37.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.228+0000 7eff345b1740 4 client.18522 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T08:59:37.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.228+0000 7eff345b1740 1 -- 172.21.15.83:0/2188273316 --> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563b9601ca60 con 0x563b961317a0 2024-04-24T08:59:37.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.229+0000 7eff22000640 1 --2- 172.21.15.83:0/2188273316 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x563b961317a0 0x563b96133be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:37.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.229+0000 7eff22000640 1 --2- 172.21.15.83:0/2188273316 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x563b961317a0 0x563b96133be0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=51e2451d74c76d9e server_cookie=339e2f9cde17d2a1 in_seq=0 out_seq=0 2024-04-24T08:59:37.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.229+0000 7eff0fe00640 10 client.18522 ms_handle_connect on v2:172.21.15.138:6836/900566754 2024-04-24T08:59:37.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.230+0000 7eff0fe00640 1 -- 172.21.15.83:0/2188273316 <== mds.1 v2:172.21.15.138:6836/900566754 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563b9601ca60 con 0x563b961317a0 2024-04-24T08:59:37.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.230+0000 7eff0fe00640 10 client.18522 handle_command_reply: tid=0 2024-04-24T08:59:37.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.230+0000 7eff345b1740 1 client.18522 shutdown 2024-04-24T08:59:37.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.230+0000 7eff345b1740 1 -- 172.21.15.83:0/2188273316 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b960afb40 msgr2=0x563b960bbed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:37.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.230+0000 7eff345b1740 1 --2- 172.21.15.83:0/2188273316 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b960afb40 0x563b960bbed0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7eff1c09c630 tx=0x7eff1c005b60 comp rx=0 tx=0).stop 2024-04-24T08:59:37.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.230+0000 7eff345b1740 1 -- 172.21.15.83:0/2188273316 shutdown_connections 2024-04-24T08:59:37.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.230+0000 7eff345b1740 1 --2- 172.21.15.83:0/2188273316 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x563b961317a0 0x563b96133be0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:37.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.230+0000 7eff345b1740 1 --2- 172.21.15.83:0/2188273316 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b960bab30 0x563b960c1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:37.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.230+0000 7eff345b1740 1 --2- 172.21.15.83:0/2188273316 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563b960b0460 0x563b960bc410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:37.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.230+0000 7eff0fe00640 0 client.18522 ms_handle_reset on v2:172.21.15.138:6836/900566754 2024-04-24T08:59:37.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.230+0000 7eff345b1740 1 --2- 172.21.15.83:0/2188273316 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b960afb40 0x563b960bbed0 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:37.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.230+0000 7eff345b1740 1 -- 172.21.15.83:0/2188273316 >> 172.21.15.83:0/2188273316 conn(0x563b960abe60 msgr2=0x563b960aaf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:37.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.230+0000 7eff345b1740 1 -- 172.21.15.83:0/2188273316 shutdown_connections 2024-04-24T08:59:37.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.230+0000 7eff345b1740 1 -- 172.21.15.83:0/2188273316 wait complete. 2024-04-24T08:59:37.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.231+0000 7eff345b1740 20 client.18522 trim_cache size 0 max 16384 2024-04-24T08:59:37.233 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T08:59:37.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.231+0000 7eff2be00640 1 -- 172.21.15.83:0/1356587120 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7eff100a3d50 msgr2=0x7eff100a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:37.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.231+0000 7eff2be00640 1 --2- 172.21.15.83:0/1356587120 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7eff100a3d50 0x7eff100a6210 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7eff24005ab0 tx=0x7eff24005a20 comp rx=0 tx=0).stop 2024-04-24T08:59:37.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.231+0000 7eff2be00640 1 -- 172.21.15.83:0/1356587120 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff2c103060 msgr2=0x7eff2c100cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:37.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.231+0000 7eff2be00640 1 --2- 172.21.15.83:0/1356587120 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff2c103060 0x7eff2c100cb0 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7eff1c00ea80 tx=0x7eff1c00ef40 comp rx=0 tx=0).stop 2024-04-24T08:59:37.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.231+0000 7eff2be00640 1 -- 172.21.15.83:0/1356587120 shutdown_connections 2024-04-24T08:59:37.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.231+0000 7eff2be00640 1 --2- 172.21.15.83:0/1356587120 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7eff100a3d50 0x7eff100a6210 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:37.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.231+0000 7eff2be00640 1 --2- 172.21.15.83:0/1356587120 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7eff2c10c480 0x7eff2c101730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:37.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.231+0000 7eff2be00640 1 --2- 172.21.15.83:0/1356587120 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff2c103980 0x7eff2c1011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:37.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.231+0000 7eff2be00640 1 --2- 172.21.15.83:0/1356587120 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff2c103060 0x7eff2c100cb0 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:37.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.231+0000 7eff2be00640 1 -- 172.21.15.83:0/1356587120 >> 172.21.15.83:0/1356587120 conn(0x7eff2c0fc880 msgr2=0x7eff2c0fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:37.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.232+0000 7eff2be00640 1 -- 172.21.15.83:0/1356587120 shutdown_connections 2024-04-24T08:59:37.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.232+0000 7eff2be00640 1 -- 172.21.15.83:0/1356587120 wait complete. 2024-04-24T08:59:37.251 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T08:59:37.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.449+0000 7fdb23200640 1 -- 172.21.15.83:0/3128174241 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb241093e0 msgr2=0x7fdb24110fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:37.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.449+0000 7fdb23200640 1 --2- 172.21.15.83:0/3128174241 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb241093e0 0x7fdb24110fb0 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7fdb1c009ca0 tx=0x7fdb1c02a8c0 comp rx=0 tx=0).stop 2024-04-24T08:59:37.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.449+0000 7fdb23200640 1 -- 172.21.15.83:0/3128174241 shutdown_connections 2024-04-24T08:59:37.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.449+0000 7fdb23200640 1 --2- 172.21.15.83:0/3128174241 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb241093e0 0x7fdb24110fb0 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:37.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.449+0000 7fdb23200640 1 --2- 172.21.15.83:0/3128174241 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb24104cc0 0x7fdb24108d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:37.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.449+0000 7fdb23200640 1 --2- 172.21.15.83:0/3128174241 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdb24104310 0x7fdb241046f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:37.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.449+0000 7fdb23200640 1 -- 172.21.15.83:0/3128174241 >> 172.21.15.83:0/3128174241 conn(0x7fdb240ffb70 msgr2=0x7fdb24101f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:37.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.450+0000 7fdb23200640 1 -- 172.21.15.83:0/3128174241 shutdown_connections 2024-04-24T08:59:37.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.450+0000 7fdb23200640 1 -- 172.21.15.83:0/3128174241 wait complete. 2024-04-24T08:59:37.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.451+0000 7fdb23200640 1 Processor -- start 2024-04-24T08:59:37.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.451+0000 7fdb23200640 1 -- start start 2024-04-24T08:59:37.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.451+0000 7fdb23200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdb24104310 0x7fdb241a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:37.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.451+0000 7fdb23200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb241093e0 0x7fdb241a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:37.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.451+0000 7fdb23200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb2419a7a0 0x7fdb2419ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:37.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.451+0000 7fdb23200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdb24114350 con 0x7fdb2419a7a0 2024-04-24T08:59:37.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.451+0000 7fdb23200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdb241141d0 con 0x7fdb241093e0 2024-04-24T08:59:37.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.451+0000 7fdb23200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdb241144d0 con 0x7fdb24104310 2024-04-24T08:59:37.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.451+0000 7fdb22800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb2419a7a0 0x7fdb2419ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:37.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.452+0000 7fdb22800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb2419a7a0 0x7fdb2419ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44356/0 (socket says 172.21.15.83:44356) 2024-04-24T08:59:37.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.452+0000 7fdb22800640 1 -- 172.21.15.83:0/495182308 learned_addr learned my addr 172.21.15.83:0/495182308 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:59:37.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.452+0000 7fdb21400640 1 --2- 172.21.15.83:0/495182308 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb241093e0 0x7fdb241a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:37.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.452+0000 7fdb21e00640 1 --2- 172.21.15.83:0/495182308 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdb24104310 0x7fdb241a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:37.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.452+0000 7fdb22800640 1 -- 172.21.15.83:0/495182308 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdb24104310 msgr2=0x7fdb241a0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:37.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.452+0000 7fdb22800640 1 --2- 172.21.15.83:0/495182308 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdb24104310 0x7fdb241a0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:37.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.452+0000 7fdb22800640 1 -- 172.21.15.83:0/495182308 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb241093e0 msgr2=0x7fdb241a0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:37.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.452+0000 7fdb22800640 1 --2- 172.21.15.83:0/495182308 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb241093e0 0x7fdb241a0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:37.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.452+0000 7fdb22800640 1 -- 172.21.15.83:0/495182308 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb2419b3a0 con 0x7fdb2419a7a0 2024-04-24T08:59:37.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.452+0000 7fdb22800640 1 --2- 172.21.15.83:0/495182308 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb2419a7a0 0x7fdb2419ac20 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7fdb1c009dc0 tx=0x7fdb1c032ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:37.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.453+0000 7fdb1aa00640 1 -- 172.21.15.83:0/495182308 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb1c003bf0 con 0x7fdb2419a7a0 2024-04-24T08:59:37.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.453+0000 7fdb1aa00640 1 -- 172.21.15.83:0/495182308 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdb1c003d90 con 0x7fdb2419a7a0 2024-04-24T08:59:37.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.453+0000 7fdb23200640 1 -- 172.21.15.83:0/495182308 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb2419b630 con 0x7fdb2419a7a0 2024-04-24T08:59:37.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.453+0000 7fdb1aa00640 1 -- 172.21.15.83:0/495182308 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb1c030b80 con 0x7fdb2419a7a0 2024-04-24T08:59:37.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.453+0000 7fdb23200640 1 -- 172.21.15.83:0/495182308 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb241a77d0 con 0x7fdb2419a7a0 2024-04-24T08:59:37.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.456+0000 7fdb2bdab740 1 Processor -- start 2024-04-24T08:59:37.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.457+0000 7fdb1aa00640 1 -- 172.21.15.83:0/495182308 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fdb1c004040 con 0x7fdb2419a7a0 2024-04-24T08:59:37.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.457+0000 7fdb2bdab740 1 -- start start 2024-04-24T08:59:37.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.458+0000 7fdb2bdab740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d1ed991b40 0x55d1ed991f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:37.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.458+0000 7fdb2bdab740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d1ed992460 0x55d1ed99c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:37.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.458+0000 7fdb2bdab740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d1ed99cb30 0x55d1ed99ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:37.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.458+0000 7fdb2bdab740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d1ed8f71d0 con 0x55d1ed992460 2024-04-24T08:59:37.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.458+0000 7fdb2bdab740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d1ed89d450 con 0x55d1ed99cb30 2024-04-24T08:59:37.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.458+0000 7fdb2bdab740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d1ed8f3190 con 0x55d1ed991b40 2024-04-24T08:59:37.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.458+0000 7fdb21e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d1ed992460 0x55d1ed99c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:37.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.458+0000 7fdb21e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d1ed992460 0x55d1ed99c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44364/0 (socket says 172.21.15.83:44364) 2024-04-24T08:59:37.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.458+0000 7fdb21e00640 1 -- 172.21.15.83:0/3073376846 learned_addr learned my addr 172.21.15.83:0/3073376846 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:59:37.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.459+0000 7fdb21400640 1 --2- 172.21.15.83:0/3073376846 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d1ed991b40 0x55d1ed991f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:37.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.459+0000 7fdb21400640 1 --2- 172.21.15.83:0/3073376846 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d1ed99cb30 0x55d1ed99ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:37.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.459+0000 7fdb21e00640 1 -- 172.21.15.83:0/3073376846 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d1ed991b40 msgr2=0x55d1ed991f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:37.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.459+0000 7fdb21e00640 1 --2- 172.21.15.83:0/3073376846 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d1ed991b40 0x55d1ed991f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:37.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.459+0000 7fdb21e00640 1 -- 172.21.15.83:0/3073376846 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d1ed99cb30 msgr2=0x55d1ed99ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:37.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.459+0000 7fdb21e00640 1 --2- 172.21.15.83:0/3073376846 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d1ed99cb30 0x55d1ed99ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:37.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.459+0000 7fdb21e00640 1 -- 172.21.15.83:0/3073376846 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1ed95a310 con 0x55d1ed992460 2024-04-24T08:59:37.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.459+0000 7fdb1aa00640 1 --2- 172.21.15.83:0/495182308 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fdb080a3da0 0x7fdb080a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:37.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.459+0000 7fdb21e00640 1 --2- 172.21.15.83:0/3073376846 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d1ed992460 0x55d1ed99c5f0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7fdb14009c90 tx=0x7fdb140117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42b81aa77ee00b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:37.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.459+0000 7fdb1aa00640 1 -- 172.21.15.83:0/495182308 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7fdb1c0f21f0 con 0x7fdb2419a7a0 2024-04-24T08:59:37.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.459+0000 7fdb21400640 1 --2- 172.21.15.83:0/495182308 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fdb080a3da0 0x7fdb080a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:37.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.460+0000 7fdb0f400640 1 -- 172.21.15.83:0/3073376846 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb14012450 con 0x55d1ed992460 2024-04-24T08:59:37.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.460+0000 7fdb0f400640 1 -- 172.21.15.83:0/3073376846 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdb14012aa0 con 0x55d1ed992460 2024-04-24T08:59:37.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.460+0000 7fdb0f400640 1 -- 172.21.15.83:0/3073376846 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb1400fb50 con 0x55d1ed992460 2024-04-24T08:59:37.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.461+0000 7fdb2bdab740 1 -- 172.21.15.83:0/3073376846 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d1ed992460 msgr2=0x55d1ed99c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:37.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.461+0000 7fdb2bdab740 1 --2- 172.21.15.83:0/3073376846 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d1ed992460 0x55d1ed99c5f0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7fdb14009c90 tx=0x7fdb140117b0 comp rx=0 tx=0).stop 2024-04-24T08:59:37.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.461+0000 7fdb21400640 1 --2- 172.21.15.83:0/495182308 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fdb080a3da0 0x7fdb080a6260 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x55d1ed98eff0 tx=0x7fdb100046c0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:37.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.461+0000 7fdb2bdab740 1 -- 172.21.15.83:0/3073376846 shutdown_connections 2024-04-24T08:59:37.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.461+0000 7fdb2bdab740 1 --2- 172.21.15.83:0/3073376846 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d1ed99cb30 0x55d1ed99ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:37.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.461+0000 7fdb2bdab740 1 --2- 172.21.15.83:0/3073376846 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d1ed992460 0x55d1ed99c5f0 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:37.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.461+0000 7fdb2bdab740 1 --2- 172.21.15.83:0/3073376846 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d1ed991b40 0x55d1ed991f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:37.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.461+0000 7fdb2bdab740 1 -- 172.21.15.83:0/3073376846 >> 172.21.15.83:0/3073376846 conn(0x55d1ed98de60 msgr2=0x55d1ed98e240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:37.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.461+0000 7fdb2bdab740 1 -- 172.21.15.83:0/3073376846 shutdown_connections 2024-04-24T08:59:37.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.461+0000 7fdb2bdab740 1 -- 172.21.15.83:0/3073376846 wait complete. 2024-04-24T08:59:37.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.462+0000 7fdb2bdab740 1 -- start start 2024-04-24T08:59:37.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.462+0000 7fdb2bdab740 1 Processor -- start 2024-04-24T08:59:37.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.462+0000 7fdb2bdab740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d1ed991b40 0x55d1ed99c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:37.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.462+0000 7fdb2bdab740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d1ed992460 0x55d1ed9998d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:37.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.462+0000 7fdb2bdab740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d1ed99cb30 0x55d1ed99a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:37.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.462+0000 7fdb2bdab740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d1ed99f9c0 con 0x55d1ed992460 2024-04-24T08:59:37.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.462+0000 7fdb2bdab740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d1ed8b0810 con 0x55d1ed991b40 2024-04-24T08:59:37.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.462+0000 7fdb2bdab740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d1ed99fb40 con 0x55d1ed99cb30 2024-04-24T08:59:37.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.462+0000 7fdb2bdab740 10 client.0 _ll_register_callbacks cb 0x55d1ed87e140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:59:37.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.463+0000 7fdb21400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d1ed992460 0x55d1ed9998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:37.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.463+0000 7fdb21400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d1ed992460 0x55d1ed9998d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44370/0 (socket says 172.21.15.83:44370) 2024-04-24T08:59:37.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.463+0000 7fdb21400640 1 -- 172.21.15.83:0/1415938537 learned_addr learned my addr 172.21.15.83:0/1415938537 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:59:37.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.463+0000 7fdb22800640 1 --2- 172.21.15.83:0/1415938537 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d1ed99cb30 0x55d1ed99a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:37.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.463+0000 7fdb21400640 1 -- 172.21.15.83:0/1415938537 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d1ed99cb30 msgr2=0x55d1ed99a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:37.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.463+0000 7fdb21e00640 1 --2- 172.21.15.83:0/1415938537 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d1ed991b40 0x55d1ed99c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:37.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.463+0000 7fdb21400640 1 --2- 172.21.15.83:0/1415938537 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d1ed99cb30 0x55d1ed99a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:37.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.463+0000 7fdb21400640 1 -- 172.21.15.83:0/1415938537 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d1ed991b40 msgr2=0x55d1ed99c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:37.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.463+0000 7fdb21400640 1 --2- 172.21.15.83:0/1415938537 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d1ed991b40 0x55d1ed99c5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:37.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.463+0000 7fdb21400640 1 -- 172.21.15.83:0/1415938537 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1ed8c13f0 con 0x55d1ed992460 2024-04-24T08:59:37.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.463+0000 7fdb22800640 1 --2- 172.21.15.83:0/1415938537 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d1ed99cb30 0x55d1ed99a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:59:37.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.464+0000 7fdb21400640 1 --2- 172.21.15.83:0/1415938537 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d1ed992460 0x55d1ed9998d0 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7fdb100126d0 tx=0x7fdb10004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e20d2ca815bf52f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:37.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.464+0000 7fdb03e00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:59:37.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.464+0000 7fdb03e00640 1 -- 172.21.15.83:0/1415938537 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb100119d0 con 0x55d1ed992460 2024-04-24T08:59:37.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.464+0000 7fdb03e00640 1 -- 172.21.15.83:0/1415938537 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7fdb10011b70 con 0x55d1ed992460 2024-04-24T08:59:37.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.464+0000 7fdb2bdab740 1 -- 172.21.15.83:0/1415938537 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb14012aa0 con 0x55d1ed992460 2024-04-24T08:59:37.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.464+0000 7fdb03e00640 1 -- 172.21.15.83:0/1415938537 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fdb10011d70 con 0x55d1ed992460 2024-04-24T08:59:37.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.464+0000 7fdb2bdab740 10 client.18534 fetch_fsmap learned FSMap version 35 2024-04-24T08:59:37.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.464+0000 7fdb2bdab740 1 -- 172.21.15.83:0/1415938537 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55d1ed6093f0 con 0x55d1ed992460 2024-04-24T08:59:37.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.464+0000 7fdb03e00640 1 -- 172.21.15.83:0/1415938537 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdb14012aa0 con 0x55d1ed992460 2024-04-24T08:59:37.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.465+0000 7fdb03e00640 1 -- 172.21.15.83:0/1415938537 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7fdb10024070 con 0x55d1ed992460 2024-04-24T08:59:37.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.465+0000 7fdb2bdab740 10 client.18534 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T08:59:37.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.465+0000 7fdb2bdab740 10 client.18534 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:59:37.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.465+0000 7fdb2bdab740 20 client.18534 populate_metadata read hostname 'smithi083' 2024-04-24T08:59:37.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.465+0000 7fdb2bdab740 1 --2- 172.21.15.83:0/1415938537 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d1ed9ace40 0x55d1ed9af2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:37.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.465+0000 7fdb2bdab740 4 client.18534 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:59:37.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.465+0000 7fdb2bdab740 1 -- 172.21.15.83:0/1415938537 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d1ed904bf0 con 0x55d1ed9ace40 2024-04-24T08:59:37.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.466+0000 7fdb21e00640 1 --2- 172.21.15.83:0/1415938537 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d1ed9ace40 0x55d1ed9af2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:37.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.467+0000 7fdb21e00640 1 --2- 172.21.15.83:0/1415938537 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d1ed9ace40 0x55d1ed9af2c0 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98636b036c82215e server_cookie=e9189d29798d6bbb in_seq=0 out_seq=0 2024-04-24T08:59:37.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.467+0000 7fdb03e00640 10 client.18534 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:59:37.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.469+0000 7fdb03e00640 1 -- 172.21.15.83:0/1415938537 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d1ed904bf0 con 0x55d1ed9ace40 2024-04-24T08:59:37.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.469+0000 7fdb03e00640 10 client.18534 handle_command_reply: tid=0 2024-04-24T08:59:37.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.469+0000 7fdb2bdab740 1 client.18534 shutdown 2024-04-24T08:59:37.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.469+0000 7fdb2bdab740 1 -- 172.21.15.83:0/1415938537 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d1ed992460 msgr2=0x55d1ed9998d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:37.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.469+0000 7fdb2bdab740 1 --2- 172.21.15.83:0/1415938537 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d1ed992460 0x55d1ed9998d0 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7fdb100126d0 tx=0x7fdb10004730 comp rx=0 tx=0).stop 2024-04-24T08:59:37.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.469+0000 7fdb2bdab740 1 -- 172.21.15.83:0/1415938537 shutdown_connections 2024-04-24T08:59:37.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.469+0000 7fdb2bdab740 1 --2- 172.21.15.83:0/1415938537 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d1ed9ace40 0x55d1ed9af2c0 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:37.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.469+0000 7fdb2bdab740 1 --2- 172.21.15.83:0/1415938537 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d1ed99cb30 0x55d1ed99a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:37.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.469+0000 7fdb2bdab740 1 --2- 172.21.15.83:0/1415938537 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d1ed992460 0x55d1ed9998d0 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:37.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.469+0000 7fdb2bdab740 1 --2- 172.21.15.83:0/1415938537 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d1ed991b40 0x55d1ed99c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:37.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.469+0000 7fdb03e00640 0 client.18534 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:59:37.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.469+0000 7fdb2bdab740 1 -- 172.21.15.83:0/1415938537 >> 172.21.15.83:0/1415938537 conn(0x55d1ed98de60 msgr2=0x55d1ed993b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:37.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.469+0000 7fdb2bdab740 1 -- 172.21.15.83:0/1415938537 shutdown_connections 2024-04-24T08:59:37.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.469+0000 7fdb2bdab740 1 -- 172.21.15.83:0/1415938537 wait complete. 2024-04-24T08:59:37.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.469+0000 7fdb2bdab740 20 client.18534 trim_cache size 0 max 16384 2024-04-24T08:59:37.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.486+0000 7fdb2bdab740 1 Processor -- start 2024-04-24T08:59:37.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.487+0000 7fdb2bdab740 1 -- start start 2024-04-24T08:59:37.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.487+0000 7fdb2bdab740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d1ed991b40 0x55d1ed9942a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:37.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.487+0000 7fdb2bdab740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d1ed992460 0x55d1ed99c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:37.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.487+0000 7fdb2bdab740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d1ed99cb30 0x55d1ed9b4b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:37.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.487+0000 7fdb2bdab740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdb241b15e0 con 0x55d1ed991b40 2024-04-24T08:59:37.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.487+0000 7fdb2bdab740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdb241b3120 con 0x55d1ed99cb30 2024-04-24T08:59:37.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.487+0000 7fdb2bdab740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdb241b1cb0 con 0x55d1ed992460 2024-04-24T08:59:37.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.487+0000 7fdb21e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d1ed991b40 0x55d1ed9942a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:37.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.487+0000 7fdb21e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d1ed991b40 0x55d1ed9942a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44376/0 (socket says 172.21.15.83:44376) 2024-04-24T08:59:37.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.487+0000 7fdb21e00640 1 -- 172.21.15.83:0/4181232993 learned_addr learned my addr 172.21.15.83:0/4181232993 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:59:37.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.488+0000 7fdb21400640 1 --2- 172.21.15.83:0/4181232993 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d1ed992460 0x55d1ed99c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:37.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.488+0000 7fdb21e00640 1 -- 172.21.15.83:0/4181232993 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d1ed992460 msgr2=0x55d1ed99c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:37.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.488+0000 7fdb21e00640 1 --2- 172.21.15.83:0/4181232993 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d1ed992460 0x55d1ed99c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:37.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.488+0000 7fdb21e00640 1 -- 172.21.15.83:0/4181232993 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d1ed99cb30 msgr2=0x55d1ed9b4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:37.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.488+0000 7fdb21e00640 1 --2- 172.21.15.83:0/4181232993 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d1ed99cb30 0x55d1ed9b4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:37.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.488+0000 7fdb21e00640 1 -- 172.21.15.83:0/4181232993 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1ed992290 con 0x55d1ed991b40 2024-04-24T08:59:37.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.488+0000 7fdb21e00640 1 --2- 172.21.15.83:0/4181232993 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d1ed991b40 0x55d1ed9942a0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7fdb14009c90 tx=0x7fdb1400f550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=301de99d37319309 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:37.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.488+0000 7fdb0ea00640 1 -- 172.21.15.83:0/4181232993 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb14030070 con 0x55d1ed991b40 2024-04-24T08:59:37.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.488+0000 7fdb0ea00640 1 -- 172.21.15.83:0/4181232993 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdb14005570 con 0x55d1ed991b40 2024-04-24T08:59:37.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.489+0000 7fdb0ea00640 1 -- 172.21.15.83:0/4181232993 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb1400f5c0 con 0x55d1ed991b40 2024-04-24T08:59:37.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.489+0000 7fdb2bdab740 1 -- 172.21.15.83:0/4181232993 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d1ed991b40 msgr2=0x55d1ed9942a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:37.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.489+0000 7fdb2bdab740 1 --2- 172.21.15.83:0/4181232993 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d1ed991b40 0x55d1ed9942a0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7fdb14009c90 tx=0x7fdb1400f550 comp rx=0 tx=0).stop 2024-04-24T08:59:37.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.489+0000 7fdb2bdab740 1 -- 172.21.15.83:0/4181232993 shutdown_connections 2024-04-24T08:59:37.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.489+0000 7fdb2bdab740 1 --2- 172.21.15.83:0/4181232993 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d1ed99cb30 0x55d1ed9b4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:37.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.490+0000 7fdb2bdab740 1 --2- 172.21.15.83:0/4181232993 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d1ed992460 0x55d1ed99c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:37.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.490+0000 7fdb2bdab740 1 --2- 172.21.15.83:0/4181232993 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d1ed991b40 0x55d1ed9942a0 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:37.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.490+0000 7fdb2bdab740 1 -- 172.21.15.83:0/4181232993 >> 172.21.15.83:0/4181232993 conn(0x55d1ed98de60 msgr2=0x55d1ed994bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:37.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.490+0000 7fdb2bdab740 1 -- 172.21.15.83:0/4181232993 shutdown_connections 2024-04-24T08:59:37.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.490+0000 7fdb2bdab740 1 -- 172.21.15.83:0/4181232993 wait complete. 2024-04-24T08:59:37.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.490+0000 7fdb2bdab740 1 -- start start 2024-04-24T08:59:37.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.490+0000 7fdb2bdab740 1 Processor -- start 2024-04-24T08:59:37.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.491+0000 7fdb2bdab740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d1ed991b40 0x55d1ed9a5240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:37.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.491+0000 7fdb2bdab740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d1ed992460 0x55d1ed996e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:37.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.491+0000 7fdb2bdab740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d1ed99cb30 0x55d1ed997410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:37.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.491+0000 7fdb2bdab740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdb241b2a50 con 0x55d1ed991b40 2024-04-24T08:59:37.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.491+0000 7fdb2bdab740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdb241b2380 con 0x55d1ed992460 2024-04-24T08:59:37.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.491+0000 7fdb2bdab740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdb241af5e0 con 0x55d1ed99cb30 2024-04-24T08:59:37.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.491+0000 7fdb2bdab740 10 client.0 _ll_register_callbacks cb 0x7fdb1000e4f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:59:37.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.491+0000 7fdb21e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d1ed991b40 0x55d1ed9a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:37.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.491+0000 7fdb21e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d1ed991b40 0x55d1ed9a5240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44384/0 (socket says 172.21.15.83:44384) 2024-04-24T08:59:37.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.491+0000 7fdb21e00640 1 -- 172.21.15.83:0/3577100723 learned_addr learned my addr 172.21.15.83:0/3577100723 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:59:37.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.491+0000 7fdb21400640 1 --2- 172.21.15.83:0/3577100723 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d1ed992460 0x55d1ed996e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:37.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.491+0000 7fdb21e00640 1 -- 172.21.15.83:0/3577100723 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d1ed99cb30 msgr2=0x55d1ed997410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:37.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.491+0000 7fdb21e00640 1 --2- 172.21.15.83:0/3577100723 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d1ed99cb30 0x55d1ed997410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:37.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.491+0000 7fdb21e00640 1 -- 172.21.15.83:0/3577100723 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d1ed992460 msgr2=0x55d1ed996e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:37.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.491+0000 7fdb21e00640 1 --2- 172.21.15.83:0/3577100723 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d1ed992460 0x55d1ed996e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:37.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.492+0000 7fdb21e00640 1 -- 172.21.15.83:0/3577100723 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1ed6d37d0 con 0x55d1ed991b40 2024-04-24T08:59:37.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.492+0000 7fdb21e00640 1 --2- 172.21.15.83:0/3577100723 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d1ed991b40 0x55d1ed9a5240 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7fdb14009f60 tx=0x7fdb14009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c8c2e2afc27c58d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:37.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.492+0000 7fdb03e00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:59:37.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.492+0000 7fdb03e00640 1 -- 172.21.15.83:0/3577100723 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb14030030 con 0x55d1ed991b40 2024-04-24T08:59:37.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.492+0000 7fdb03e00640 1 -- 172.21.15.83:0/3577100723 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7fdb14003800 con 0x55d1ed991b40 2024-04-24T08:59:37.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.492+0000 7fdb2bdab740 1 -- 172.21.15.83:0/3577100723 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb14005570 con 0x55d1ed991b40 2024-04-24T08:59:37.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.492+0000 7fdb03e00640 1 -- 172.21.15.83:0/3577100723 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fdb140057d0 con 0x55d1ed991b40 2024-04-24T08:59:37.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.492+0000 7fdb2bdab740 10 client.18540 fetch_fsmap learned FSMap version 35 2024-04-24T08:59:37.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.492+0000 7fdb2bdab740 1 -- 172.21.15.83:0/3577100723 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55d1eda13630 con 0x55d1ed991b40 2024-04-24T08:59:37.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.493+0000 7fdb03e00640 1 -- 172.21.15.83:0/3577100723 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdb14005570 con 0x55d1ed991b40 2024-04-24T08:59:37.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.493+0000 7fdb03e00640 1 -- 172.21.15.83:0/3577100723 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7fdb1402b040 con 0x55d1ed991b40 2024-04-24T08:59:37.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.493+0000 7fdb2bdab740 10 client.18540 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T08:59:37.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.493+0000 7fdb2bdab740 10 client.18540 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:59:37.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.493+0000 7fdb2bdab740 20 client.18540 populate_metadata read hostname 'smithi083' 2024-04-24T08:59:37.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.493+0000 7fdb2bdab740 1 --2- 172.21.15.83:0/3577100723 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d1eda13b80 0x55d1eda15fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:37.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.493+0000 7fdb2bdab740 4 client.18540 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T08:59:37.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.493+0000 7fdb2bdab740 1 -- 172.21.15.83:0/3577100723 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d1ed8fea60 con 0x55d1eda13b80 2024-04-24T08:59:37.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.494+0000 7fdb21400640 1 --2- 172.21.15.83:0/3577100723 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d1eda13b80 0x55d1eda15fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:37.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.495+0000 7fdb21400640 1 --2- 172.21.15.83:0/3577100723 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d1eda13b80 0x55d1eda15fa0 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ec34577910046f6 server_cookie=450b07f79a454176 in_seq=0 out_seq=0 2024-04-24T08:59:37.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.495+0000 7fdb03e00640 10 client.18540 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:59:37.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.496+0000 7fdb03e00640 1 -- 172.21.15.83:0/3577100723 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d1ed8fea60 con 0x55d1eda13b80 2024-04-24T08:59:37.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.496+0000 7fdb03e00640 10 client.18540 handle_command_reply: tid=0 2024-04-24T08:59:37.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.496+0000 7fdb2bdab740 1 client.18540 shutdown 2024-04-24T08:59:37.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.497+0000 7fdb2bdab740 1 -- 172.21.15.83:0/3577100723 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d1ed991b40 msgr2=0x55d1ed9a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:37.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.497+0000 7fdb2bdab740 1 --2- 172.21.15.83:0/3577100723 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d1ed991b40 0x55d1ed9a5240 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7fdb14009f60 tx=0x7fdb14009f90 comp rx=0 tx=0).stop 2024-04-24T08:59:37.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.497+0000 7fdb2bdab740 1 -- 172.21.15.83:0/3577100723 shutdown_connections 2024-04-24T08:59:37.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.497+0000 7fdb2bdab740 1 --2- 172.21.15.83:0/3577100723 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d1eda13b80 0x55d1eda15fa0 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:37.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.497+0000 7fdb2bdab740 1 --2- 172.21.15.83:0/3577100723 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d1ed99cb30 0x55d1ed997410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:37.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.497+0000 7fdb2bdab740 1 --2- 172.21.15.83:0/3577100723 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d1ed992460 0x55d1ed996e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:37.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.497+0000 7fdb2bdab740 1 --2- 172.21.15.83:0/3577100723 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d1ed991b40 0x55d1ed9a5240 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:37.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.497+0000 7fdb03e00640 0 client.18540 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:59:37.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.497+0000 7fdb2bdab740 1 -- 172.21.15.83:0/3577100723 >> 172.21.15.83:0/3577100723 conn(0x55d1ed98de60 msgr2=0x55d1ed98c6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:37.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.497+0000 7fdb2bdab740 1 -- 172.21.15.83:0/3577100723 shutdown_connections 2024-04-24T08:59:37.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.497+0000 7fdb2bdab740 1 -- 172.21.15.83:0/3577100723 wait complete. 2024-04-24T08:59:37.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.497+0000 7fdb2bdab740 20 client.18540 trim_cache size 0 max 16384 2024-04-24T08:59:37.498 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T08:59:37.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.498+0000 7fdb23200640 1 -- 172.21.15.83:0/495182308 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fdb080a3da0 msgr2=0x7fdb080a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:37.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.498+0000 7fdb23200640 1 --2- 172.21.15.83:0/495182308 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fdb080a3da0 0x7fdb080a6260 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x55d1ed98eff0 tx=0x7fdb100046c0 comp rx=0 tx=0).stop 2024-04-24T08:59:37.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.498+0000 7fdb23200640 1 -- 172.21.15.83:0/495182308 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb2419a7a0 msgr2=0x7fdb2419ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:37.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.498+0000 7fdb23200640 1 --2- 172.21.15.83:0/495182308 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb2419a7a0 0x7fdb2419ac20 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7fdb1c009dc0 tx=0x7fdb1c032ff0 comp rx=0 tx=0).stop 2024-04-24T08:59:37.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.498+0000 7fdb23200640 1 -- 172.21.15.83:0/495182308 shutdown_connections 2024-04-24T08:59:37.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.498+0000 7fdb23200640 1 --2- 172.21.15.83:0/495182308 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fdb080a3da0 0x7fdb080a6260 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:37.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.498+0000 7fdb23200640 1 --2- 172.21.15.83:0/495182308 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb2419a7a0 0x7fdb2419ac20 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:37.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.498+0000 7fdb23200640 1 --2- 172.21.15.83:0/495182308 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb241093e0 0x7fdb241a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:37.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.498+0000 7fdb23200640 1 --2- 172.21.15.83:0/495182308 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdb24104310 0x7fdb241a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:37.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.498+0000 7fdb23200640 1 -- 172.21.15.83:0/495182308 >> 172.21.15.83:0/495182308 conn(0x7fdb240ffb70 msgr2=0x7fdb24101ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:37.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.498+0000 7fdb23200640 1 -- 172.21.15.83:0/495182308 shutdown_connections 2024-04-24T08:59:37.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:37.498+0000 7fdb23200640 1 -- 172.21.15.83:0/495182308 wait complete. 2024-04-24T08:59:37.514 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:59:37 smithi083 ceph-mon[27907]: pgmap v249: 129 pgs: 129 active+clean; 48 MiB data, 695 MiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 322 KiB/s wr, 51 op/s 2024-04-24T08:59:37.594 INFO:tasks.workunit.client.0.smithi083.stdout: 1 44206 0.63 MB/sec execute 251 sec latency 55.926 ms 2024-04-24T08:59:37.954 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:59:37 smithi138 ceph-mon[31441]: pgmap v249: 129 pgs: 129 active+clean; 48 MiB data, 695 MiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 322 KiB/s wr, 51 op/s 2024-04-24T08:59:37.956 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:59:37 smithi189 ceph-mon[31791]: pgmap v249: 129 pgs: 129 active+clean; 48 MiB data, 695 MiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 322 KiB/s wr, 51 op/s 2024-04-24T08:59:38.516 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T08:59:38.517 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T08:59:38.517 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T08:59:38.594 INFO:tasks.workunit.client.0.smithi083.stdout: 1 44301 0.63 MB/sec execute 252 sec latency 74.944 ms 2024-04-24T08:59:38.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.749+0000 7fdb5fe00640 1 -- 172.21.15.83:0/4048303791 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb60102110 msgr2=0x7fdb601024f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:38.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.749+0000 7fdb5fe00640 1 --2- 172.21.15.83:0/4048303791 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb60102110 0x7fdb601024f0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7fdb50009900 tx=0x7fdb5002a410 comp rx=0 tx=0).stop 2024-04-24T08:59:38.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.749+0000 7fdb5fe00640 1 -- 172.21.15.83:0/4048303791 shutdown_connections 2024-04-24T08:59:38.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.749+0000 7fdb5fe00640 1 --2- 172.21.15.83:0/4048303791 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb601071e0 0x7fdb6010edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:38.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.749+0000 7fdb5fe00640 1 --2- 172.21.15.83:0/4048303791 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdb60102ac0 0x7fdb60106b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:38.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.749+0000 7fdb5fe00640 1 --2- 172.21.15.83:0/4048303791 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb60102110 0x7fdb601024f0 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:38.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.750+0000 7fdb5fe00640 1 -- 172.21.15.83:0/4048303791 >> 172.21.15.83:0/4048303791 conn(0x7fdb600fd9d0 msgr2=0x7fdb600ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:38.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.750+0000 7fdb5fe00640 1 -- 172.21.15.83:0/4048303791 shutdown_connections 2024-04-24T08:59:38.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.750+0000 7fdb5fe00640 1 -- 172.21.15.83:0/4048303791 wait complete. 2024-04-24T08:59:38.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.751+0000 7fdb5fe00640 1 Processor -- start 2024-04-24T08:59:38.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.751+0000 7fdb5fe00640 1 -- start start 2024-04-24T08:59:38.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.751+0000 7fdb5fe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdb60102110 0x7fdb60198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:38.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.751+0000 7fdb5fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb601071e0 0x7fdb60198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:38.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.751+0000 7fdb5fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb6019cf80 0x7fdb6019d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:38.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.751+0000 7fdb5fe00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdb60112170 con 0x7fdb601071e0 2024-04-24T08:59:38.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.751+0000 7fdb5fe00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdb60111ff0 con 0x7fdb6019cf80 2024-04-24T08:59:38.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.751+0000 7fdb5fe00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdb601122f0 con 0x7fdb60102110 2024-04-24T08:59:38.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.751+0000 7fdb5e000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb601071e0 0x7fdb60198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:38.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.752+0000 7fdb5e000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb601071e0 0x7fdb60198c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44406/0 (socket says 172.21.15.83:44406) 2024-04-24T08:59:38.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.752+0000 7fdb5e000640 1 -- 172.21.15.83:0/2833622216 learned_addr learned my addr 172.21.15.83:0/2833622216 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:59:38.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.752+0000 7fdb5ea00640 1 --2- 172.21.15.83:0/2833622216 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdb60102110 0x7fdb60198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:38.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.752+0000 7fdb5f400640 1 --2- 172.21.15.83:0/2833622216 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb6019cf80 0x7fdb6019d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:38.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.752+0000 7fdb5e000640 1 -- 172.21.15.83:0/2833622216 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdb60102110 msgr2=0x7fdb60198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:38.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.752+0000 7fdb5e000640 1 --2- 172.21.15.83:0/2833622216 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdb60102110 0x7fdb60198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:38.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.752+0000 7fdb5e000640 1 -- 172.21.15.83:0/2833622216 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb6019cf80 msgr2=0x7fdb6019d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:38.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.752+0000 7fdb5e000640 1 --2- 172.21.15.83:0/2833622216 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb6019cf80 0x7fdb6019d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:38.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.752+0000 7fdb5e000640 1 -- 172.21.15.83:0/2833622216 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb6019da70 con 0x7fdb601071e0 2024-04-24T08:59:38.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.752+0000 7fdb5f400640 1 --2- 172.21.15.83:0/2833622216 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb6019cf80 0x7fdb6019d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:59:38.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.752+0000 7fdb5e000640 1 --2- 172.21.15.83:0/2833622216 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb601071e0 0x7fdb60198c50 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c00d770 tx=0x7fdb4c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:38.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.753+0000 7fdb57400640 1 -- 172.21.15.83:0/2833622216 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb4c017070 con 0x7fdb601071e0 2024-04-24T08:59:38.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.753+0000 7fdb57400640 1 -- 172.21.15.83:0/2833622216 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdb4c009c30 con 0x7fdb601071e0 2024-04-24T08:59:38.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.753+0000 7fdb57400640 1 -- 172.21.15.83:0/2833622216 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb4c00f620 con 0x7fdb601071e0 2024-04-24T08:59:38.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.753+0000 7fdb5fe00640 1 -- 172.21.15.83:0/2833622216 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb6019dd60 con 0x7fdb601071e0 2024-04-24T08:59:38.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.753+0000 7fdb5fe00640 1 -- 172.21.15.83:0/2833622216 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb601a55a0 con 0x7fdb601071e0 2024-04-24T08:59:38.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.757+0000 7fdb687bc740 1 Processor -- start 2024-04-24T08:59:38.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.757+0000 7fdb687bc740 1 -- start start 2024-04-24T08:59:38.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.757+0000 7fdb687bc740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55daab079270 0x55daab079650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:38.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.757+0000 7fdb687bc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55daab079b90 0x55daab083d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:38.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.757+0000 7fdb687bc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55daab084260 0x55daab086650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:38.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.757+0000 7fdb687bc740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55daaafdee70 con 0x55daab084260 2024-04-24T08:59:38.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.757+0000 7fdb687bc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55daaaf85380 con 0x55daab079b90 2024-04-24T08:59:38.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.757+0000 7fdb687bc740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55daaafdae30 con 0x55daab079270 2024-04-24T08:59:38.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.758+0000 7fdb57400640 1 -- 172.21.15.83:0/2833622216 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fdb4c01a070 con 0x7fdb601071e0 2024-04-24T08:59:38.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.758+0000 7fdb5f400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55daab079270 0x55daab079650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:38.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.758+0000 7fdb5ea00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55daab079b90 0x55daab083d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:38.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.758+0000 7fdb5e000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55daab084260 0x55daab086650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:38.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.758+0000 7fdb5f400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55daab079270 0x55daab079650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:50560/0 (socket says 172.21.15.83:50560) 2024-04-24T08:59:38.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.758+0000 7fdb5f400640 1 -- 172.21.15.83:0/421577039 learned_addr learned my addr 172.21.15.83:0/421577039 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:59:38.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.759+0000 7fdb5e000640 1 -- 172.21.15.83:0/421577039 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55daab079270 msgr2=0x55daab079650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:38.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.759+0000 7fdb5e000640 1 --2- 172.21.15.83:0/421577039 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55daab079270 0x55daab079650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:38.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.759+0000 7fdb5e000640 1 -- 172.21.15.83:0/421577039 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55daab079b90 msgr2=0x55daab083d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:38.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.759+0000 7fdb5e000640 1 --2- 172.21.15.83:0/421577039 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55daab079b90 0x55daab083d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:38.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.759+0000 7fdb5e000640 1 -- 172.21.15.83:0/421577039 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55daaadb97e0 con 0x55daab084260 2024-04-24T08:59:38.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.759+0000 7fdb5e000640 1 --2- 172.21.15.83:0/421577039 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55daab084260 0x55daab086650 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c0cda80 tx=0x7fdb4c0d8670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=376c3964e6a80258 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:38.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.759+0000 7fdb4be00640 1 -- 172.21.15.83:0/421577039 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb4c0df070 con 0x55daab084260 2024-04-24T08:59:38.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.759+0000 7fdb57400640 1 --2- 172.21.15.83:0/2833622216 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fdb440a3da0 0x7fdb440a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:38.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.759+0000 7fdb4be00640 1 -- 172.21.15.83:0/421577039 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdb4c09c9d0 con 0x55daab084260 2024-04-24T08:59:38.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.759+0000 7fdb4be00640 1 -- 172.21.15.83:0/421577039 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb4c063990 con 0x55daab084260 2024-04-24T08:59:38.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.760+0000 7fdb57400640 1 -- 172.21.15.83:0/2833622216 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7fdb4c0d07b0 con 0x7fdb601071e0 2024-04-24T08:59:38.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.760+0000 7fdb5f400640 1 --2- 172.21.15.83:0/2833622216 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fdb440a3da0 0x7fdb440a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:38.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.760+0000 7fdb687bc740 1 -- 172.21.15.83:0/421577039 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55daab084260 msgr2=0x55daab086650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:38.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.760+0000 7fdb687bc740 1 --2- 172.21.15.83:0/421577039 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55daab084260 0x55daab086650 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c0cda80 tx=0x7fdb4c0d8670 comp rx=0 tx=0).stop 2024-04-24T08:59:38.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.761+0000 7fdb687bc740 1 -- 172.21.15.83:0/421577039 shutdown_connections 2024-04-24T08:59:38.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.761+0000 7fdb687bc740 1 --2- 172.21.15.83:0/421577039 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55daab084260 0x55daab086650 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:38.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.761+0000 7fdb687bc740 1 --2- 172.21.15.83:0/421577039 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55daab079b90 0x55daab083d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:38.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.761+0000 7fdb687bc740 1 --2- 172.21.15.83:0/421577039 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55daab079270 0x55daab079650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:38.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.761+0000 7fdb687bc740 1 -- 172.21.15.83:0/421577039 >> 172.21.15.83:0/421577039 conn(0x55daab075590 msgr2=0x55daab075970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:38.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.761+0000 7fdb5f400640 1 --2- 172.21.15.83:0/2833622216 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fdb440a3da0 0x7fdb440a6260 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fdb58005fd0 tx=0x7fdb58008690 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:38.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.761+0000 7fdb687bc740 1 -- 172.21.15.83:0/421577039 shutdown_connections 2024-04-24T08:59:38.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.761+0000 7fdb687bc740 1 -- 172.21.15.83:0/421577039 wait complete. 2024-04-24T08:59:38.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.761+0000 7fdb687bc740 1 -- start start 2024-04-24T08:59:38.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.762+0000 7fdb687bc740 1 Processor -- start 2024-04-24T08:59:38.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.762+0000 7fdb687bc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55daab079270 0x55daab08e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:38.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.762+0000 7fdb687bc740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55daab079b90 0x55daab08e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:38.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.762+0000 7fdb687bc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55daab084260 0x55daab088780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:38.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.762+0000 7fdb687bc740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55daab087050 con 0x55daab079270 2024-04-24T08:59:38.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.762+0000 7fdb687bc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55daaaf98660 con 0x55daab084260 2024-04-24T08:59:38.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.762+0000 7fdb687bc740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55daab0871d0 con 0x55daab079b90 2024-04-24T08:59:38.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.762+0000 7fdb687bc740 10 client.0 _ll_register_callbacks cb 0x55daaaf86aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:59:38.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.762+0000 7fdb5ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55daab079270 0x55daab08e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:38.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.763+0000 7fdb5ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55daab079270 0x55daab08e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44428/0 (socket says 172.21.15.83:44428) 2024-04-24T08:59:38.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.763+0000 7fdb5ea00640 1 -- 172.21.15.83:0/4020975119 learned_addr learned my addr 172.21.15.83:0/4020975119 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:59:38.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.763+0000 7fdb5e000640 1 --2- 172.21.15.83:0/4020975119 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55daab079b90 0x55daab08e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:38.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.763+0000 7fdb5f400640 1 --2- 172.21.15.83:0/4020975119 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55daab084260 0x55daab088780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:38.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.763+0000 7fdb5ea00640 1 -- 172.21.15.83:0/4020975119 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55daab079b90 msgr2=0x55daab08e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:38.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.763+0000 7fdb5ea00640 1 --2- 172.21.15.83:0/4020975119 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55daab079b90 0x55daab08e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:38.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.763+0000 7fdb5ea00640 1 -- 172.21.15.83:0/4020975119 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55daab084260 msgr2=0x55daab088780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:38.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.763+0000 7fdb5ea00640 1 --2- 172.21.15.83:0/4020975119 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55daab084260 0x55daab088780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:38.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.763+0000 7fdb5ea00640 1 -- 172.21.15.83:0/4020975119 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55daaae19f70 con 0x55daab079270 2024-04-24T08:59:38.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.763+0000 7fdb5f400640 1 --2- 172.21.15.83:0/4020975119 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55daab084260 0x55daab088780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:59:38.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.763+0000 7fdb5ea00640 1 --2- 172.21.15.83:0/4020975119 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55daab079270 0x55daab08e410 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7fdb50009230 tx=0x7fdb500052e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb5932a26cb52b8c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:38.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.763+0000 7fdb48c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:59:38.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.764+0000 7fdb48c00640 1 -- 172.21.15.83:0/4020975119 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb50035040 con 0x55daab079270 2024-04-24T08:59:38.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.764+0000 7fdb687bc740 1 -- 172.21.15.83:0/4020975119 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb4c09c9d0 con 0x55daab079270 2024-04-24T08:59:38.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.764+0000 7fdb48c00640 1 -- 172.21.15.83:0/4020975119 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7fdb500048c0 con 0x55daab079270 2024-04-24T08:59:38.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.764+0000 7fdb48c00640 1 -- 172.21.15.83:0/4020975119 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fdb5004e050 con 0x55daab079270 2024-04-24T08:59:38.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.764+0000 7fdb687bc740 10 client.18552 fetch_fsmap learned FSMap version 35 2024-04-24T08:59:38.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.764+0000 7fdb687bc740 1 -- 172.21.15.83:0/4020975119 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55daaacb1170 con 0x55daab079270 2024-04-24T08:59:38.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.764+0000 7fdb48c00640 1 -- 172.21.15.83:0/4020975119 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdb4c09c9d0 con 0x55daab079270 2024-04-24T08:59:38.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.765+0000 7fdb48c00640 1 -- 172.21.15.83:0/4020975119 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7fdb500314a0 con 0x55daab079270 2024-04-24T08:59:38.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.765+0000 7fdb687bc740 10 client.18552 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T08:59:38.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.765+0000 7fdb687bc740 10 client.18552 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:59:38.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.765+0000 7fdb687bc740 20 client.18552 populate_metadata read hostname 'smithi083' 2024-04-24T08:59:38.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.765+0000 7fdb687bc740 1 --2- 172.21.15.83:0/4020975119 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55daab098920 0x55daab09ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:38.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.765+0000 7fdb687bc740 4 client.18552 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:59:38.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.765+0000 7fdb687bc740 1 -- 172.21.15.83:0/4020975119 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55daaafec890 con 0x55daab098920 2024-04-24T08:59:38.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.765+0000 7fdb5e000640 1 --2- 172.21.15.83:0/4020975119 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55daab098920 0x55daab09ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:38.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.766+0000 7fdb48c00640 10 client.18552 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:59:38.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.766+0000 7fdb5e000640 1 --2- 172.21.15.83:0/4020975119 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55daab098920 0x55daab09ad70 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6317e2115d97b9cd server_cookie=273b3d7927955b84 in_seq=0 out_seq=0 2024-04-24T08:59:38.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.768+0000 7fdb48c00640 1 -- 172.21.15.83:0/4020975119 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55daaafec890 con 0x55daab098920 2024-04-24T08:59:38.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.768+0000 7fdb48c00640 10 client.18552 handle_command_reply: tid=0 2024-04-24T08:59:38.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.768+0000 7fdb687bc740 1 client.18552 shutdown 2024-04-24T08:59:38.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.768+0000 7fdb687bc740 1 -- 172.21.15.83:0/4020975119 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55daab079270 msgr2=0x55daab08e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:38.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.768+0000 7fdb687bc740 1 --2- 172.21.15.83:0/4020975119 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55daab079270 0x55daab08e410 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7fdb50009230 tx=0x7fdb500052e0 comp rx=0 tx=0).stop 2024-04-24T08:59:38.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.769+0000 7fdb687bc740 1 -- 172.21.15.83:0/4020975119 shutdown_connections 2024-04-24T08:59:38.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.769+0000 7fdb687bc740 1 --2- 172.21.15.83:0/4020975119 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55daab098920 0x55daab09ad70 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:38.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.769+0000 7fdb687bc740 1 --2- 172.21.15.83:0/4020975119 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55daab084260 0x55daab088780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:38.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.769+0000 7fdb48c00640 0 client.18552 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:59:38.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.769+0000 7fdb687bc740 1 --2- 172.21.15.83:0/4020975119 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55daab079b90 0x55daab08e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:38.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.769+0000 7fdb687bc740 1 --2- 172.21.15.83:0/4020975119 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55daab079270 0x55daab08e410 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:38.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.769+0000 7fdb687bc740 1 -- 172.21.15.83:0/4020975119 >> 172.21.15.83:0/4020975119 conn(0x55daab075590 msgr2=0x55daab07b390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:38.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.769+0000 7fdb687bc740 1 -- 172.21.15.83:0/4020975119 shutdown_connections 2024-04-24T08:59:38.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.769+0000 7fdb687bc740 1 -- 172.21.15.83:0/4020975119 wait complete. 2024-04-24T08:59:38.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.769+0000 7fdb687bc740 20 client.18552 trim_cache size 0 max 16384 2024-04-24T08:59:38.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.793+0000 7fdb687bc740 1 Processor -- start 2024-04-24T08:59:38.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.793+0000 7fdb687bc740 1 -- start start 2024-04-24T08:59:38.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.793+0000 7fdb687bc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55daab079270 0x55daab087a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:38.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.793+0000 7fdb687bc740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55daab079b90 0x55daab08e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:38.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.793+0000 7fdb687bc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55daab084260 0x55daab0a0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:38.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.793+0000 7fdb687bc740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdb601ad250 con 0x55daab084260 2024-04-24T08:59:38.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.793+0000 7fdb687bc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdb601aed90 con 0x55daab079270 2024-04-24T08:59:38.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.793+0000 7fdb687bc740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdb601ad920 con 0x55daab079b90 2024-04-24T08:59:38.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.794+0000 7fdb5f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55daab084260 0x55daab0a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:38.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.794+0000 7fdb5f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55daab084260 0x55daab0a0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44432/0 (socket says 172.21.15.83:44432) 2024-04-24T08:59:38.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.794+0000 7fdb5f400640 1 -- 172.21.15.83:0/2139166963 learned_addr learned my addr 172.21.15.83:0/2139166963 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:59:38.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.794+0000 7fdb5e000640 1 --2- 172.21.15.83:0/2139166963 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55daab079b90 0x55daab08e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:38.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.794+0000 7fdb5ea00640 1 --2- 172.21.15.83:0/2139166963 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55daab079270 0x55daab087a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:38.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.794+0000 7fdb5f400640 1 -- 172.21.15.83:0/2139166963 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55daab079b90 msgr2=0x55daab08e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:38.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.794+0000 7fdb5f400640 1 --2- 172.21.15.83:0/2139166963 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55daab079b90 0x55daab08e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:38.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.794+0000 7fdb5f400640 1 -- 172.21.15.83:0/2139166963 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55daab079270 msgr2=0x55daab087a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:38.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.794+0000 7fdb5f400640 1 --2- 172.21.15.83:0/2139166963 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55daab079270 0x55daab087a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:38.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.794+0000 7fdb5f400640 1 -- 172.21.15.83:0/2139166963 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55daab084080 con 0x55daab084260 2024-04-24T08:59:38.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.794+0000 7fdb5ea00640 1 --2- 172.21.15.83:0/2139166963 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55daab079270 0x55daab087a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:59:38.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.795+0000 7fdb5f400640 1 --2- 172.21.15.83:0/2139166963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55daab084260 0x55daab0a0640 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7fdb5801c570 tx=0x7fdb58013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f73701a69bc27d58 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:38.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.795+0000 7fdb4b400640 1 -- 172.21.15.83:0/2139166963 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb5801d8c0 con 0x55daab084260 2024-04-24T08:59:38.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.795+0000 7fdb4b400640 1 -- 172.21.15.83:0/2139166963 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdb58026450 con 0x55daab084260 2024-04-24T08:59:38.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.795+0000 7fdb4b400640 1 -- 172.21.15.83:0/2139166963 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb5801a680 con 0x55daab084260 2024-04-24T08:59:38.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.796+0000 7fdb687bc740 1 -- 172.21.15.83:0/2139166963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55daab084260 msgr2=0x55daab0a0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:38.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.796+0000 7fdb687bc740 1 --2- 172.21.15.83:0/2139166963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55daab084260 0x55daab0a0640 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7fdb5801c570 tx=0x7fdb58013000 comp rx=0 tx=0).stop 2024-04-24T08:59:38.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.796+0000 7fdb687bc740 1 -- 172.21.15.83:0/2139166963 shutdown_connections 2024-04-24T08:59:38.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.796+0000 7fdb687bc740 1 --2- 172.21.15.83:0/2139166963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55daab084260 0x55daab0a0640 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:38.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.796+0000 7fdb687bc740 1 --2- 172.21.15.83:0/2139166963 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55daab079b90 0x55daab08e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:38.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.796+0000 7fdb687bc740 1 --2- 172.21.15.83:0/2139166963 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55daab079270 0x55daab087a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:38.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.796+0000 7fdb687bc740 1 -- 172.21.15.83:0/2139166963 >> 172.21.15.83:0/2139166963 conn(0x55daab075590 msgr2=0x55daab08dfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:38.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.796+0000 7fdb687bc740 1 -- 172.21.15.83:0/2139166963 shutdown_connections 2024-04-24T08:59:38.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.796+0000 7fdb687bc740 1 -- 172.21.15.83:0/2139166963 wait complete. 2024-04-24T08:59:38.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.796+0000 7fdb687bc740 1 -- start start 2024-04-24T08:59:38.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.797+0000 7fdb687bc740 1 Processor -- start 2024-04-24T08:59:38.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.797+0000 7fdb687bc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55daab079270 0x55daab0921c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:38.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.797+0000 7fdb687bc740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55daab079b90 0x55daab092700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:38.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.797+0000 7fdb687bc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55daab084260 0x55daab092c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:38.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.797+0000 7fdb687bc740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdb601ae6c0 con 0x55daab084260 2024-04-24T08:59:38.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.797+0000 7fdb687bc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdb601adff0 con 0x55daab079270 2024-04-24T08:59:38.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.797+0000 7fdb687bc740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdb601ab250 con 0x55daab079b90 2024-04-24T08:59:38.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.797+0000 7fdb687bc740 10 client.0 _ll_register_callbacks cb 0x7fdb50004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:59:38.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.797+0000 7fdb5f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55daab084260 0x55daab092c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:38.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.797+0000 7fdb5f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55daab084260 0x55daab092c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44444/0 (socket says 172.21.15.83:44444) 2024-04-24T08:59:38.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.797+0000 7fdb5f400640 1 -- 172.21.15.83:0/3245963095 learned_addr learned my addr 172.21.15.83:0/3245963095 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:59:38.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.798+0000 7fdb5e000640 1 --2- 172.21.15.83:0/3245963095 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55daab079b90 0x55daab092700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:38.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.798+0000 7fdb5ea00640 1 --2- 172.21.15.83:0/3245963095 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55daab079270 0x55daab0921c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:38.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.798+0000 7fdb5f400640 1 -- 172.21.15.83:0/3245963095 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55daab079b90 msgr2=0x55daab092700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:38.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.798+0000 7fdb5f400640 1 --2- 172.21.15.83:0/3245963095 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55daab079b90 0x55daab092700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:38.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.798+0000 7fdb5f400640 1 -- 172.21.15.83:0/3245963095 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55daab079270 msgr2=0x55daab0921c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:38.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.798+0000 7fdb5f400640 1 --2- 172.21.15.83:0/3245963095 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55daab079270 0x55daab0921c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:38.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.798+0000 7fdb5f400640 1 -- 172.21.15.83:0/3245963095 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55daab0fdf70 con 0x55daab084260 2024-04-24T08:59:38.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.798+0000 7fdb5e000640 1 --2- 172.21.15.83:0/3245963095 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55daab079b90 0x55daab092700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:59:38.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.798+0000 7fdb5ea00640 1 --2- 172.21.15.83:0/3245963095 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55daab079270 0x55daab0921c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:59:38.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.798+0000 7fdb5f400640 1 --2- 172.21.15.83:0/3245963095 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55daab084260 0x55daab092c40 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7fdb58026e30 tx=0x7fdb58026890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64d41b112f2b3595 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:38.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.798+0000 7fdb48c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:59:38.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.798+0000 7fdb48c00640 1 -- 172.21.15.83:0/3245963095 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb58035060 con 0x55daab084260 2024-04-24T08:59:38.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.798+0000 7fdb48c00640 1 -- 172.21.15.83:0/3245963095 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7fdb5801d8d0 con 0x55daab084260 2024-04-24T08:59:38.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.798+0000 7fdb687bc740 1 -- 172.21.15.83:0/3245963095 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb58026450 con 0x55daab084260 2024-04-24T08:59:38.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.799+0000 7fdb48c00640 1 -- 172.21.15.83:0/3245963095 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fdb5803c050 con 0x55daab084260 2024-04-24T08:59:38.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.799+0000 7fdb687bc740 10 client.18558 fetch_fsmap learned FSMap version 35 2024-04-24T08:59:38.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.799+0000 7fdb687bc740 1 -- 172.21.15.83:0/3245963095 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55daab0ff200 con 0x55daab084260 2024-04-24T08:59:38.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.799+0000 7fdb48c00640 1 -- 172.21.15.83:0/3245963095 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdb58026450 con 0x55daab084260 2024-04-24T08:59:38.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.799+0000 7fdb48c00640 1 -- 172.21.15.83:0/3245963095 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7fdb580309c0 con 0x55daab084260 2024-04-24T08:59:38.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.799+0000 7fdb687bc740 10 client.18558 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T08:59:38.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.799+0000 7fdb687bc740 10 client.18558 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:59:38.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.799+0000 7fdb687bc740 20 client.18558 populate_metadata read hostname 'smithi083' 2024-04-24T08:59:38.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.799+0000 7fdb687bc740 1 --2- 172.21.15.83:0/3245963095 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55daab0ff900 0x55daab101d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:38.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.799+0000 7fdb687bc740 4 client.18558 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T08:59:38.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.799+0000 7fdb687bc740 1 -- 172.21.15.83:0/3245963095 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55daaafe6700 con 0x55daab0ff900 2024-04-24T08:59:38.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.800+0000 7fdb5ea00640 1 --2- 172.21.15.83:0/3245963095 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55daab0ff900 0x55daab101d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:38.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.801+0000 7fdb48c00640 10 client.18558 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:59:38.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.801+0000 7fdb5ea00640 1 --2- 172.21.15.83:0/3245963095 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55daab0ff900 0x55daab101d20 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c07ac162748b76a server_cookie=9e5fce2cce350e01 in_seq=0 out_seq=0 2024-04-24T08:59:38.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.804+0000 7fdb48c00640 1 -- 172.21.15.83:0/3245963095 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55daaafe6700 con 0x55daab0ff900 2024-04-24T08:59:38.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.804+0000 7fdb48c00640 10 client.18558 handle_command_reply: tid=0 2024-04-24T08:59:38.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.804+0000 7fdb687bc740 1 client.18558 shutdown 2024-04-24T08:59:38.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.804+0000 7fdb687bc740 1 -- 172.21.15.83:0/3245963095 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55daab084260 msgr2=0x55daab092c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:38.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.804+0000 7fdb687bc740 1 --2- 172.21.15.83:0/3245963095 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55daab084260 0x55daab092c40 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7fdb58026e30 tx=0x7fdb58026890 comp rx=0 tx=0).stop 2024-04-24T08:59:38.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.804+0000 7fdb687bc740 1 -- 172.21.15.83:0/3245963095 shutdown_connections 2024-04-24T08:59:38.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.804+0000 7fdb687bc740 1 --2- 172.21.15.83:0/3245963095 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55daab0ff900 0x55daab101d20 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:38.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.804+0000 7fdb687bc740 1 --2- 172.21.15.83:0/3245963095 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55daab084260 0x55daab092c40 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:38.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.804+0000 7fdb48c00640 0 client.18558 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:59:38.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.804+0000 7fdb687bc740 1 --2- 172.21.15.83:0/3245963095 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55daab079b90 0x55daab092700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:38.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.804+0000 7fdb687bc740 1 --2- 172.21.15.83:0/3245963095 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55daab079270 0x55daab0921c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:38.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.804+0000 7fdb687bc740 1 -- 172.21.15.83:0/3245963095 >> 172.21.15.83:0/3245963095 conn(0x55daab075590 msgr2=0x55daab0886c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:38.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.804+0000 7fdb687bc740 1 -- 172.21.15.83:0/3245963095 shutdown_connections 2024-04-24T08:59:38.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.804+0000 7fdb687bc740 1 -- 172.21.15.83:0/3245963095 wait complete. 2024-04-24T08:59:38.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.804+0000 7fdb687bc740 20 client.18558 trim_cache size 0 max 16384 2024-04-24T08:59:38.805 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T08:59:38.805 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-04-24T08:59:38.805 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "2df825ed-c6a8-4515-b005-50f66ea826c6", 2024-04-24T08:59:38.805 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-04-24T08:59:38.805 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T08:59:38.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.805+0000 7fdb5fe00640 1 -- 172.21.15.83:0/2833622216 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fdb440a3da0 msgr2=0x7fdb440a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:38.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.805+0000 7fdb5fe00640 1 --2- 172.21.15.83:0/2833622216 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fdb440a3da0 0x7fdb440a6260 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fdb58005fd0 tx=0x7fdb58008690 comp rx=0 tx=0).stop 2024-04-24T08:59:38.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.805+0000 7fdb5fe00640 1 -- 172.21.15.83:0/2833622216 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb601071e0 msgr2=0x7fdb60198c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:38.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.805+0000 7fdb5fe00640 1 --2- 172.21.15.83:0/2833622216 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb601071e0 0x7fdb60198c50 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c00d770 tx=0x7fdb4c00dc30 comp rx=0 tx=0).stop 2024-04-24T08:59:38.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.805+0000 7fdb5fe00640 1 -- 172.21.15.83:0/2833622216 shutdown_connections 2024-04-24T08:59:38.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.805+0000 7fdb5fe00640 1 --2- 172.21.15.83:0/2833622216 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fdb440a3da0 0x7fdb440a6260 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:38.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.805+0000 7fdb5fe00640 1 --2- 172.21.15.83:0/2833622216 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb6019cf80 0x7fdb6019d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:38.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.805+0000 7fdb5fe00640 1 --2- 172.21.15.83:0/2833622216 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb601071e0 0x7fdb60198c50 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:38.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.805+0000 7fdb5fe00640 1 --2- 172.21.15.83:0/2833622216 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdb60102110 0x7fdb60198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:38.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.805+0000 7fdb5fe00640 1 -- 172.21.15.83:0/2833622216 >> 172.21.15.83:0/2833622216 conn(0x7fdb600fd9d0 msgr2=0x7fdb600fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:38.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.805+0000 7fdb5fe00640 1 -- 172.21.15.83:0/2833622216 shutdown_connections 2024-04-24T08:59:38.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:38.805+0000 7fdb5fe00640 1 -- 172.21.15.83:0/2833622216 wait complete. 2024-04-24T08:59:38.823 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T08:59:38.823 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T08:59:39.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.027+0000 7f8a4be00640 1 -- 172.21.15.83:0/1346233572 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a4c102110 msgr2=0x7f8a4c1024f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:39.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.027+0000 7f8a4be00640 1 --2- 172.21.15.83:0/1346233572 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a4c102110 0x7f8a4c1024f0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c009900 tx=0x7f8a3c02a410 comp rx=0 tx=0).stop 2024-04-24T08:59:39.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.027+0000 7f8a4be00640 1 -- 172.21.15.83:0/1346233572 shutdown_connections 2024-04-24T08:59:39.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.027+0000 7f8a4be00640 1 --2- 172.21.15.83:0/1346233572 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a4c1071e0 0x7f8a4c10edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:39.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.027+0000 7f8a4be00640 1 --2- 172.21.15.83:0/1346233572 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8a4c102ac0 0x7f8a4c106b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:39.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.027+0000 7f8a4be00640 1 --2- 172.21.15.83:0/1346233572 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a4c102110 0x7f8a4c1024f0 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:39.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.027+0000 7f8a4be00640 1 -- 172.21.15.83:0/1346233572 >> 172.21.15.83:0/1346233572 conn(0x7f8a4c0fd9d0 msgr2=0x7f8a4c0ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:39.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.027+0000 7f8a4be00640 1 -- 172.21.15.83:0/1346233572 shutdown_connections 2024-04-24T08:59:39.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.028+0000 7f8a4be00640 1 -- 172.21.15.83:0/1346233572 wait complete. 2024-04-24T08:59:39.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.029+0000 7f8a4be00640 1 Processor -- start 2024-04-24T08:59:39.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.029+0000 7f8a4be00640 1 -- start start 2024-04-24T08:59:39.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.029+0000 7f8a4be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a4c102110 0x7f8a4c1a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:39.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.029+0000 7f8a4be00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8a4c1071e0 0x7f8a4c1a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:39.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.029+0000 7f8a4be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a4c19a6e0 0x7f8a4c19ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:39.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.029+0000 7f8a4be00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8a4c112120 con 0x7f8a4c102110 2024-04-24T08:59:39.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.029+0000 7f8a4be00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8a4c111fa0 con 0x7f8a4c19a6e0 2024-04-24T08:59:39.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.029+0000 7f8a4be00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8a4c1122a0 con 0x7f8a4c1071e0 2024-04-24T08:59:39.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.030+0000 7f8a4aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a4c102110 0x7f8a4c1a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:39.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.030+0000 7f8a4aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a4c102110 0x7f8a4c1a0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38194/0 (socket says 172.21.15.83:38194) 2024-04-24T08:59:39.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.030+0000 7f8a4aa00640 1 -- 172.21.15.83:0/2454152596 learned_addr learned my addr 172.21.15.83:0/2454152596 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:59:39.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.030+0000 7f8a4a000640 1 --2- 172.21.15.83:0/2454152596 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8a4c1071e0 0x7f8a4c1a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:39.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.030+0000 7f8a4b400640 1 --2- 172.21.15.83:0/2454152596 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a4c19a6e0 0x7f8a4c19ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:39.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.030+0000 7f8a4aa00640 1 -- 172.21.15.83:0/2454152596 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8a4c1071e0 msgr2=0x7f8a4c1a0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:39.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.030+0000 7f8a4aa00640 1 --2- 172.21.15.83:0/2454152596 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8a4c1071e0 0x7f8a4c1a0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:39.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.030+0000 7f8a4aa00640 1 -- 172.21.15.83:0/2454152596 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a4c19a6e0 msgr2=0x7f8a4c19ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:39.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.030+0000 7f8a4aa00640 1 --2- 172.21.15.83:0/2454152596 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a4c19a6e0 0x7f8a4c19ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:39.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.030+0000 7f8a4aa00640 1 -- 172.21.15.83:0/2454152596 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a4c19b2e0 con 0x7f8a4c102110 2024-04-24T08:59:39.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.031+0000 7f8a4aa00640 1 --2- 172.21.15.83:0/2454152596 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a4c102110 0x7f8a4c1a0620 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c005940 tx=0x7f8a3c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:39.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.032+0000 7f8a43400640 1 -- 172.21.15.83:0/2454152596 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a3c03f030 con 0x7f8a4c102110 2024-04-24T08:59:39.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.032+0000 7f8a43400640 1 -- 172.21.15.83:0/2454152596 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8a3c003cb0 con 0x7f8a4c102110 2024-04-24T08:59:39.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.032+0000 7f8a4be00640 1 -- 172.21.15.83:0/2454152596 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a4c19b570 con 0x7f8a4c102110 2024-04-24T08:59:39.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.032+0000 7f8a43400640 1 -- 172.21.15.83:0/2454152596 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a3c035070 con 0x7f8a4c102110 2024-04-24T08:59:39.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.032+0000 7f8a4be00640 1 -- 172.21.15.83:0/2454152596 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a4c1a76b0 con 0x7f8a4c102110 2024-04-24T08:59:39.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.037+0000 7f8a5439c740 1 Processor -- start 2024-04-24T08:59:39.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.037+0000 7f8a43400640 1 -- 172.21.15.83:0/2454152596 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8a3c042030 con 0x7f8a4c102110 2024-04-24T08:59:39.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.037+0000 7f8a5439c740 1 -- start start 2024-04-24T08:59:39.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.038+0000 7f8a5439c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56101ec9ea30 0x56101ec9ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:39.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.038+0000 7f8a5439c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56101ec9f350 0x56101eca94e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:39.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.038+0000 7f8a5439c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56101eca9a20 0x56101ecabe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:39.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.038+0000 7f8a5439c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56101ec041f0 con 0x56101ec9f350 2024-04-24T08:59:39.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.038+0000 7f8a5439c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56101ebaa350 con 0x56101ec9ea30 2024-04-24T08:59:39.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.038+0000 7f8a5439c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56101ec001b0 con 0x56101eca9a20 2024-04-24T08:59:39.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.038+0000 7f8a4aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56101ec9f350 0x56101eca94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:39.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.038+0000 7f8a4a000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56101eca9a20 0x56101ecabe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:39.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.039+0000 7f8a4a000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56101eca9a20 0x56101ecabe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:40244/0 (socket says 172.21.15.83:40244) 2024-04-24T08:59:39.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.039+0000 7f8a4a000640 1 -- 172.21.15.83:0/1425825977 learned_addr learned my addr 172.21.15.83:0/1425825977 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:59:39.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.039+0000 7f8a4b400640 1 --2- 172.21.15.83:0/1425825977 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56101ec9ea30 0x56101ec9ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:39.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.039+0000 7f8a4aa00640 1 -- 172.21.15.83:0/1425825977 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56101eca9a20 msgr2=0x56101ecabe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:39.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.039+0000 7f8a4aa00640 1 --2- 172.21.15.83:0/1425825977 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56101eca9a20 0x56101ecabe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:39.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.039+0000 7f8a4aa00640 1 -- 172.21.15.83:0/1425825977 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56101ec9ea30 msgr2=0x56101ec9ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:39.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.039+0000 7f8a4aa00640 1 --2- 172.21.15.83:0/1425825977 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56101ec9ea30 0x56101ec9ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:39.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.039+0000 7f8a4aa00640 1 -- 172.21.15.83:0/1425825977 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56101ebfef50 con 0x56101ec9f350 2024-04-24T08:59:39.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.039+0000 7f8a43400640 1 --2- 172.21.15.83:0/2454152596 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f8a300a3ef0 0x7f8a300a63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:39.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.040+0000 7f8a4b400640 1 --2- 172.21.15.83:0/2454152596 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f8a300a3ef0 0x7f8a300a63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:39.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.040+0000 7f8a43400640 1 -- 172.21.15.83:0/2454152596 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7f8a3c0f13c0 con 0x7f8a4c102110 2024-04-24T08:59:39.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.040+0000 7f8a4aa00640 1 --2- 172.21.15.83:0/1425825977 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56101ec9f350 0x56101eca94e0 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c0ee690 tx=0x7f8a3c0f95c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5681c08b6737cdf server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:39.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.040+0000 7f8a4b400640 1 --2- 172.21.15.83:0/2454152596 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f8a300a3ef0 0x7f8a300a63b0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8a44005fd0 tx=0x7f8a440096e0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:39.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.040+0000 7f8a37400640 1 -- 172.21.15.83:0/1425825977 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a3c049070 con 0x56101ec9f350 2024-04-24T08:59:39.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.040+0000 7f8a37400640 1 -- 172.21.15.83:0/1425825977 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8a3c03b460 con 0x56101ec9f350 2024-04-24T08:59:39.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.040+0000 7f8a37400640 1 -- 172.21.15.83:0/1425825977 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a3c0bca20 con 0x56101ec9f350 2024-04-24T08:59:39.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.041+0000 7f8a5439c740 1 -- 172.21.15.83:0/1425825977 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56101ec9f350 msgr2=0x56101eca94e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:39.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.041+0000 7f8a5439c740 1 --2- 172.21.15.83:0/1425825977 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56101ec9f350 0x56101eca94e0 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c0ee690 tx=0x7f8a3c0f95c0 comp rx=0 tx=0).stop 2024-04-24T08:59:39.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.041+0000 7f8a5439c740 1 -- 172.21.15.83:0/1425825977 shutdown_connections 2024-04-24T08:59:39.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.041+0000 7f8a5439c740 1 --2- 172.21.15.83:0/1425825977 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56101eca9a20 0x56101ecabe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:39.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.041+0000 7f8a5439c740 1 --2- 172.21.15.83:0/1425825977 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56101ec9f350 0x56101eca94e0 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:39.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.041+0000 7f8a5439c740 1 --2- 172.21.15.83:0/1425825977 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56101ec9ea30 0x56101ec9ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:39.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.041+0000 7f8a5439c740 1 -- 172.21.15.83:0/1425825977 >> 172.21.15.83:0/1425825977 conn(0x56101ec9ad50 msgr2=0x56101ec9b130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:39.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.041+0000 7f8a5439c740 1 -- 172.21.15.83:0/1425825977 shutdown_connections 2024-04-24T08:59:39.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.042+0000 7f8a5439c740 1 -- 172.21.15.83:0/1425825977 wait complete. 2024-04-24T08:59:39.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.042+0000 7f8a5439c740 1 -- start start 2024-04-24T08:59:39.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.042+0000 7f8a5439c740 1 Processor -- start 2024-04-24T08:59:39.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.043+0000 7f8a5439c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56101ec9ea30 0x56101eca6a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:39.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.043+0000 7f8a5439c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56101ec9f350 0x56101eca6f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:39.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.043+0000 7f8a5439c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56101eca9a20 0x56101eca7480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:39.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.043+0000 7f8a5439c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56101ecac5d0 con 0x56101eca9a20 2024-04-24T08:59:39.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.043+0000 7f8a5439c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56101ebbd690 con 0x56101ec9f350 2024-04-24T08:59:39.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.043+0000 7f8a5439c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56101ecac750 con 0x56101ec9ea30 2024-04-24T08:59:39.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.043+0000 7f8a5439c740 10 client.0 _ll_register_callbacks cb 0x56101ebf9650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:59:39.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.043+0000 7f8a4b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56101eca9a20 0x56101eca7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:39.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.043+0000 7f8a4b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56101eca9a20 0x56101eca7480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38218/0 (socket says 172.21.15.83:38218) 2024-04-24T08:59:39.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.043+0000 7f8a4b400640 1 -- 172.21.15.83:0/502344519 learned_addr learned my addr 172.21.15.83:0/502344519 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:59:39.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.043+0000 7f8a4a000640 1 --2- 172.21.15.83:0/502344519 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56101ec9f350 0x56101eca6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:39.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.043+0000 7f8a4aa00640 1 --2- 172.21.15.83:0/502344519 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56101ec9ea30 0x56101eca6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:39.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.043+0000 7f8a4b400640 1 -- 172.21.15.83:0/502344519 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56101ec9ea30 msgr2=0x56101eca6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:39.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.043+0000 7f8a4b400640 1 --2- 172.21.15.83:0/502344519 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56101ec9ea30 0x56101eca6a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:39.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.043+0000 7f8a4b400640 1 -- 172.21.15.83:0/502344519 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56101ec9f350 msgr2=0x56101eca6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:39.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.043+0000 7f8a4b400640 1 --2- 172.21.15.83:0/502344519 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56101ec9f350 0x56101eca6f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:39.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.043+0000 7f8a4b400640 1 -- 172.21.15.83:0/502344519 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56101ebbc920 con 0x56101eca9a20 2024-04-24T08:59:39.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.043+0000 7f8a4a000640 1 --2- 172.21.15.83:0/502344519 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56101ec9f350 0x56101eca6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:59:39.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.044+0000 7f8a4b400640 1 --2- 172.21.15.83:0/502344519 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56101eca9a20 0x56101eca7480 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f8a4401d850 tx=0x7f8a44004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d9394ced1d40f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:39.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.044+0000 7f8a2be00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:59:39.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.044+0000 7f8a2be00640 1 -- 172.21.15.83:0/502344519 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a4401e820 con 0x56101eca9a20 2024-04-24T08:59:39.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.044+0000 7f8a2be00640 1 -- 172.21.15.83:0/502344519 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7f8a4401e9c0 con 0x56101eca9a20 2024-04-24T08:59:39.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.044+0000 7f8a5439c740 1 -- 172.21.15.83:0/502344519 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56101ec11c10 con 0x56101eca9a20 2024-04-24T08:59:39.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.044+0000 7f8a2be00640 1 -- 172.21.15.83:0/502344519 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f8a4401ebc0 con 0x56101eca9a20 2024-04-24T08:59:39.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.045+0000 7f8a5439c740 10 client.18570 fetch_fsmap learned FSMap version 35 2024-04-24T08:59:39.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.045+0000 7f8a5439c740 1 -- 172.21.15.83:0/502344519 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56101e9e07d0 con 0x56101eca9a20 2024-04-24T08:59:39.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.045+0000 7f8a2be00640 1 -- 172.21.15.83:0/502344519 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x56101ec11c10 con 0x56101eca9a20 2024-04-24T08:59:39.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.045+0000 7f8a2be00640 1 -- 172.21.15.83:0/502344519 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7f8a4400f040 con 0x56101eca9a20 2024-04-24T08:59:39.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.045+0000 7f8a5439c740 10 client.18570 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T08:59:39.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.045+0000 7f8a5439c740 10 client.18570 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:59:39.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.045+0000 7f8a5439c740 20 client.18570 populate_metadata read hostname 'smithi083' 2024-04-24T08:59:39.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.045+0000 7f8a5439c740 1 --2- 172.21.15.83:0/502344519 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56101ecb9bd0 0x56101ecbc000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:39.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.045+0000 7f8a5439c740 4 client.18570 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:59:39.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.045+0000 7f8a5439c740 1 -- 172.21.15.83:0/502344519 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56101ec0ba80 con 0x56101ecb9bd0 2024-04-24T08:59:39.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.046+0000 7f8a4aa00640 1 --2- 172.21.15.83:0/502344519 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56101ecb9bd0 0x56101ecbc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:39.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.047+0000 7f8a4aa00640 1 --2- 172.21.15.83:0/502344519 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56101ecb9bd0 0x56101ecbc000 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b230a2c5e55d2334 server_cookie=2a14fa96664c2178 in_seq=0 out_seq=0 2024-04-24T08:59:39.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.047+0000 7f8a2be00640 10 client.18570 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:59:39.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.049+0000 7f8a2be00640 1 -- 172.21.15.83:0/502344519 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56101ec0ba80 con 0x56101ecb9bd0 2024-04-24T08:59:39.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.049+0000 7f8a2be00640 10 client.18570 handle_command_reply: tid=0 2024-04-24T08:59:39.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.050+0000 7f8a5439c740 1 client.18570 shutdown 2024-04-24T08:59:39.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.050+0000 7f8a5439c740 1 -- 172.21.15.83:0/502344519 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56101eca9a20 msgr2=0x56101eca7480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:39.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.050+0000 7f8a5439c740 1 --2- 172.21.15.83:0/502344519 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56101eca9a20 0x56101eca7480 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f8a4401d850 tx=0x7f8a44004a40 comp rx=0 tx=0).stop 2024-04-24T08:59:39.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.050+0000 7f8a5439c740 1 -- 172.21.15.83:0/502344519 shutdown_connections 2024-04-24T08:59:39.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.050+0000 7f8a5439c740 1 --2- 172.21.15.83:0/502344519 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56101ecb9bd0 0x56101ecbc000 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:39.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.050+0000 7f8a5439c740 1 --2- 172.21.15.83:0/502344519 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56101eca9a20 0x56101eca7480 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:39.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.050+0000 7f8a5439c740 1 --2- 172.21.15.83:0/502344519 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56101ec9f350 0x56101eca6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:39.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.050+0000 7f8a5439c740 1 --2- 172.21.15.83:0/502344519 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56101ec9ea30 0x56101eca6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:39.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.050+0000 7f8a5439c740 1 -- 172.21.15.83:0/502344519 >> 172.21.15.83:0/502344519 conn(0x56101ec9ad50 msgr2=0x56101ec99fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:39.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.050+0000 7f8a2be00640 0 client.18570 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:59:39.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.050+0000 7f8a5439c740 1 -- 172.21.15.83:0/502344519 shutdown_connections 2024-04-24T08:59:39.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.050+0000 7f8a5439c740 1 -- 172.21.15.83:0/502344519 wait complete. 2024-04-24T08:59:39.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.050+0000 7f8a5439c740 20 client.18570 trim_cache size 0 max 16384 2024-04-24T08:59:39.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.071+0000 7f8a5439c740 1 Processor -- start 2024-04-24T08:59:39.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.071+0000 7f8a5439c740 1 -- start start 2024-04-24T08:59:39.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.072+0000 7f8a5439c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56101ec9ea30 0x56101ec9a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:39.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.072+0000 7f8a5439c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56101ec9f350 0x56101ecb5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:39.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.072+0000 7f8a5439c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56101eca9a20 0x56101eca0890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:39.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.072+0000 7f8a5439c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8a4c1b15c0 con 0x56101eca9a20 2024-04-24T08:59:39.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.072+0000 7f8a5439c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8a4c1b3100 con 0x56101ec9ea30 2024-04-24T08:59:39.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.072+0000 7f8a5439c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8a4c1b1c90 con 0x56101ec9f350 2024-04-24T08:59:39.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.072+0000 7f8a4b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56101eca9a20 0x56101eca0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:39.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.072+0000 7f8a4b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56101eca9a20 0x56101eca0890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38232/0 (socket says 172.21.15.83:38232) 2024-04-24T08:59:39.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.072+0000 7f8a4b400640 1 -- 172.21.15.83:0/711796545 learned_addr learned my addr 172.21.15.83:0/711796545 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:59:39.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.073+0000 7f8a4aa00640 1 --2- 172.21.15.83:0/711796545 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56101ec9ea30 0x56101ec9a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:39.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.073+0000 7f8a4a000640 1 --2- 172.21.15.83:0/711796545 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56101ec9f350 0x56101ecb5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:39.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.073+0000 7f8a4b400640 1 -- 172.21.15.83:0/711796545 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56101ec9f350 msgr2=0x56101ecb5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:39.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.073+0000 7f8a4b400640 1 --2- 172.21.15.83:0/711796545 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56101ec9f350 0x56101ecb5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:39.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.073+0000 7f8a4b400640 1 -- 172.21.15.83:0/711796545 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56101ec9ea30 msgr2=0x56101ec9a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:39.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.073+0000 7f8a4b400640 1 --2- 172.21.15.83:0/711796545 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56101ec9ea30 0x56101ec9a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:39.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.073+0000 7f8a4b400640 1 -- 172.21.15.83:0/711796545 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56101e9163f0 con 0x56101eca9a20 2024-04-24T08:59:39.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.073+0000 7f8a4b400640 1 --2- 172.21.15.83:0/711796545 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56101eca9a20 0x56101eca0890 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f8a44000f80 tx=0x7f8a44012e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7009b3aa59bfe9f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:39.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.073+0000 7f8a36a00640 1 -- 172.21.15.83:0/711796545 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a4403c020 con 0x56101eca9a20 2024-04-24T08:59:39.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.073+0000 7f8a36a00640 1 -- 172.21.15.83:0/711796545 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8a4401b7d0 con 0x56101eca9a20 2024-04-24T08:59:39.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.073+0000 7f8a36a00640 1 -- 172.21.15.83:0/711796545 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a440296a0 con 0x56101eca9a20 2024-04-24T08:59:39.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.074+0000 7f8a5439c740 1 -- 172.21.15.83:0/711796545 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56101eca9a20 msgr2=0x56101eca0890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:39.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.074+0000 7f8a5439c740 1 --2- 172.21.15.83:0/711796545 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56101eca9a20 0x56101eca0890 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f8a44000f80 tx=0x7f8a44012e00 comp rx=0 tx=0).stop 2024-04-24T08:59:39.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.074+0000 7f8a5439c740 1 -- 172.21.15.83:0/711796545 shutdown_connections 2024-04-24T08:59:39.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.074+0000 7f8a5439c740 1 --2- 172.21.15.83:0/711796545 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56101eca9a20 0x56101eca0890 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:39.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.074+0000 7f8a5439c740 1 --2- 172.21.15.83:0/711796545 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56101ec9f350 0x56101ecb5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:39.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.074+0000 7f8a5439c740 1 --2- 172.21.15.83:0/711796545 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56101ec9ea30 0x56101ec9a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:39.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.074+0000 7f8a5439c740 1 -- 172.21.15.83:0/711796545 >> 172.21.15.83:0/711796545 conn(0x56101ec9ad50 msgr2=0x56101eca0180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:39.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.074+0000 7f8a5439c740 1 -- 172.21.15.83:0/711796545 shutdown_connections 2024-04-24T08:59:39.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.074+0000 7f8a5439c740 1 -- 172.21.15.83:0/711796545 wait complete. 2024-04-24T08:59:39.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.075+0000 7f8a5439c740 1 -- start start 2024-04-24T08:59:39.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.075+0000 7f8a5439c740 1 Processor -- start 2024-04-24T08:59:39.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.075+0000 7f8a5439c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56101ec9ea30 0x56101ecb1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:39.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.075+0000 7f8a5439c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56101ec9f350 0x56101eca3250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:39.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.075+0000 7f8a5439c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56101eca9a20 0x56101eca3790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:39.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.075+0000 7f8a5439c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8a4c1b2a30 con 0x56101ec9f350 2024-04-24T08:59:39.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.075+0000 7f8a5439c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8a4c1b2360 con 0x56101ec9ea30 2024-04-24T08:59:39.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.075+0000 7f8a5439c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8a4c1af5c0 con 0x56101eca9a20 2024-04-24T08:59:39.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.076+0000 7f8a5439c740 10 client.0 _ll_register_callbacks cb 0x7f8a4401d500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:59:39.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.076+0000 7f8a4a000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56101ec9f350 0x56101eca3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:39.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.076+0000 7f8a4a000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56101ec9f350 0x56101eca3250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38238/0 (socket says 172.21.15.83:38238) 2024-04-24T08:59:39.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.076+0000 7f8a4a000640 1 -- 172.21.15.83:0/3331830373 learned_addr learned my addr 172.21.15.83:0/3331830373 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:59:39.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.076+0000 7f8a4aa00640 1 --2- 172.21.15.83:0/3331830373 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56101ec9ea30 0x56101ecb1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:39.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.076+0000 7f8a4a000640 1 -- 172.21.15.83:0/3331830373 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56101eca9a20 msgr2=0x56101eca3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:39.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.076+0000 7f8a4b400640 1 --2- 172.21.15.83:0/3331830373 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56101eca9a20 0x56101eca3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:39.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.076+0000 7f8a4a000640 1 --2- 172.21.15.83:0/3331830373 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56101eca9a20 0x56101eca3790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:39.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.076+0000 7f8a4a000640 1 -- 172.21.15.83:0/3331830373 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56101ec9ea30 msgr2=0x56101ecb1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:39.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.076+0000 7f8a4a000640 1 --2- 172.21.15.83:0/3331830373 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56101ec9ea30 0x56101ecb1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:39.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.076+0000 7f8a4a000640 1 -- 172.21.15.83:0/3331830373 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56101e8d60f0 con 0x56101ec9f350 2024-04-24T08:59:39.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.076+0000 7f8a4aa00640 1 --2- 172.21.15.83:0/3331830373 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56101ec9ea30 0x56101ecb1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:59:39.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.077+0000 7f8a4a000640 1 --2- 172.21.15.83:0/3331830373 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56101ec9f350 0x56101eca3250 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x56101ec9bee0 tx=0x7f8a38012b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ff03c31ca40b785 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:39.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.077+0000 7f8a2be00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:59:39.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.077+0000 7f8a2be00640 1 -- 172.21.15.83:0/3331830373 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a38022060 con 0x56101ec9f350 2024-04-24T08:59:39.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.077+0000 7f8a2be00640 1 -- 172.21.15.83:0/3331830373 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7f8a38004250 con 0x56101ec9f350 2024-04-24T08:59:39.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.077+0000 7f8a5439c740 1 -- 172.21.15.83:0/3331830373 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a4401b7d0 con 0x56101ec9f350 2024-04-24T08:59:39.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.077+0000 7f8a2be00640 1 -- 172.21.15.83:0/3331830373 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f8a38029050 con 0x56101ec9f350 2024-04-24T08:59:39.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.077+0000 7f8a5439c740 10 client.18576 fetch_fsmap learned FSMap version 35 2024-04-24T08:59:39.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.077+0000 7f8a5439c740 1 -- 172.21.15.83:0/3331830373 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56101ebc9810 con 0x56101ec9f350 2024-04-24T08:59:39.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.077+0000 7f8a2be00640 1 -- 172.21.15.83:0/3331830373 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8a4401b7d0 con 0x56101ec9f350 2024-04-24T08:59:39.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.078+0000 7f8a2be00640 1 -- 172.21.15.83:0/3331830373 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7f8a3801c030 con 0x56101ec9f350 2024-04-24T08:59:39.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.078+0000 7f8a5439c740 10 client.18576 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T08:59:39.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.078+0000 7f8a5439c740 10 client.18576 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T08:59:39.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.078+0000 7f8a5439c740 20 client.18576 populate_metadata read hostname 'smithi083' 2024-04-24T08:59:39.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.078+0000 7f8a5439c740 1 --2- 172.21.15.83:0/3331830373 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56101ed20370 0x56101ed22790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:39.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.078+0000 7f8a5439c740 4 client.18576 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T08:59:39.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.078+0000 7f8a5439c740 1 -- 172.21.15.83:0/3331830373 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56101eb87360 con 0x56101ed20370 2024-04-24T08:59:39.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.078+0000 7f8a4aa00640 1 --2- 172.21.15.83:0/3331830373 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56101ed20370 0x56101ed22790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:39.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.079+0000 7f8a2be00640 10 client.18576 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:59:39.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.079+0000 7f8a4aa00640 1 --2- 172.21.15.83:0/3331830373 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56101ed20370 0x56101ed22790 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5da3e177edc0fb56 server_cookie=563ed32c943a2f6d in_seq=0 out_seq=0 2024-04-24T08:59:39.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.081+0000 7f8a2be00640 1 -- 172.21.15.83:0/3331830373 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56101eb87360 con 0x56101ed20370 2024-04-24T08:59:39.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.081+0000 7f8a2be00640 10 client.18576 handle_command_reply: tid=0 2024-04-24T08:59:39.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.081+0000 7f8a5439c740 1 client.18576 shutdown 2024-04-24T08:59:39.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.081+0000 7f8a5439c740 1 -- 172.21.15.83:0/3331830373 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56101ec9f350 msgr2=0x56101eca3250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:39.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.081+0000 7f8a5439c740 1 --2- 172.21.15.83:0/3331830373 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56101ec9f350 0x56101eca3250 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x56101ec9bee0 tx=0x7f8a38012b60 comp rx=0 tx=0).stop 2024-04-24T08:59:39.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.081+0000 7f8a5439c740 1 -- 172.21.15.83:0/3331830373 shutdown_connections 2024-04-24T08:59:39.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.081+0000 7f8a5439c740 1 --2- 172.21.15.83:0/3331830373 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56101ed20370 0x56101ed22790 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:39.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.081+0000 7f8a5439c740 1 --2- 172.21.15.83:0/3331830373 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56101eca9a20 0x56101eca3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:39.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.081+0000 7f8a5439c740 1 --2- 172.21.15.83:0/3331830373 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56101ec9f350 0x56101eca3250 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:39.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.081+0000 7f8a5439c740 1 --2- 172.21.15.83:0/3331830373 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56101ec9ea30 0x56101ecb1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:39.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.081+0000 7f8a5439c740 1 -- 172.21.15.83:0/3331830373 >> 172.21.15.83:0/3331830373 conn(0x56101ec9ad50 msgr2=0x56101eca7250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:39.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.081+0000 7f8a2be00640 0 client.18576 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:59:39.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.081+0000 7f8a5439c740 1 -- 172.21.15.83:0/3331830373 shutdown_connections 2024-04-24T08:59:39.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.081+0000 7f8a5439c740 1 -- 172.21.15.83:0/3331830373 wait complete. 2024-04-24T08:59:39.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.081+0000 7f8a5439c740 20 client.18576 trim_cache size 0 max 16384 2024-04-24T08:59:39.086 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T08:59:39.086 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T08:59:39.086 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-04-24T08:59:39.087 INFO:teuthology.orchestra.run.smithi083.stdout: "2df825ed-c6a8-4515-b005-50f66ea826c6": { 2024-04-24T08:59:39.087 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-04-24T08:59:39.087 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "2df825ed-c6a8-4515-b005-50f66ea826c6", 2024-04-24T08:59:39.087 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-04-24T08:59:39.087 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T08:59:39.087 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T08:59:39.087 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T08:59:39.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.082+0000 7f8a4be00640 1 -- 172.21.15.83:0/2454152596 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f8a300a3ef0 msgr2=0x7f8a300a63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:39.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.082+0000 7f8a4be00640 1 --2- 172.21.15.83:0/2454152596 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f8a300a3ef0 0x7f8a300a63b0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8a44005fd0 tx=0x7f8a440096e0 comp rx=0 tx=0).stop 2024-04-24T08:59:39.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.082+0000 7f8a4be00640 1 -- 172.21.15.83:0/2454152596 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a4c102110 msgr2=0x7f8a4c1a0620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:39.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.082+0000 7f8a4be00640 1 --2- 172.21.15.83:0/2454152596 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a4c102110 0x7f8a4c1a0620 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c005940 tx=0x7f8a3c0042f0 comp rx=0 tx=0).stop 2024-04-24T08:59:39.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.082+0000 7f8a4be00640 1 -- 172.21.15.83:0/2454152596 shutdown_connections 2024-04-24T08:59:39.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.082+0000 7f8a4be00640 1 --2- 172.21.15.83:0/2454152596 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f8a300a3ef0 0x7f8a300a63b0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:39.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.082+0000 7f8a4be00640 1 --2- 172.21.15.83:0/2454152596 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a4c19a6e0 0x7f8a4c19ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:39.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.082+0000 7f8a4be00640 1 --2- 172.21.15.83:0/2454152596 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8a4c1071e0 0x7f8a4c1a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:39.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.082+0000 7f8a4be00640 1 --2- 172.21.15.83:0/2454152596 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a4c102110 0x7f8a4c1a0620 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:39.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.082+0000 7f8a4be00640 1 -- 172.21.15.83:0/2454152596 >> 172.21.15.83:0/2454152596 conn(0x7f8a4c0fd9d0 msgr2=0x7f8a4c0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:39.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.082+0000 7f8a4be00640 1 -- 172.21.15.83:0/2454152596 shutdown_connections 2024-04-24T08:59:39.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T08:59:39.082+0000 7f8a4be00640 1 -- 172.21.15.83:0/2454152596 wait complete. 2024-04-24T08:59:39.101 INFO:tasks.cephfs.filesystem:scrub status for tag:2df825ed-c6a8-4515-b005-50f66ea826c6 - {'path': '/', 'tag': '2df825ed-c6a8-4515-b005-50f66ea826c6', 'options': 'recursive,force'} 2024-04-24T08:59:39.595 INFO:tasks.workunit.client.0.smithi083.stdout: 1 44461 0.63 MB/sec execute 253 sec latency 50.836 ms 2024-04-24T08:59:39.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:59:39 smithi138 ceph-mon[31441]: pgmap v250: 129 pgs: 129 active+clean; 48 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 506 KiB/s wr, 78 op/s 2024-04-24T08:59:39.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:59:39 smithi138 ceph-mon[31441]: scrub queued for path: / 2024-04-24T08:59:39.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:59:39 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T08:59:39.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:59:39 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T08:59:39.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:59:39 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T08:59:39.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:59:39 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T08:59:39.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:59:39 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T08:59:39.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:59:39 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T08:59:39.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:59:39 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T08:59:39.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:59:39 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T08:59:39.957 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:59:39 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T08:59:39.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:59:39 smithi189 ceph-mon[31791]: pgmap v250: 129 pgs: 129 active+clean; 48 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 506 KiB/s wr, 78 op/s 2024-04-24T08:59:39.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:59:39 smithi189 ceph-mon[31791]: scrub queued for path: / 2024-04-24T08:59:39.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:59:39 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T08:59:39.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:59:39 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T08:59:39.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:59:39 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T08:59:39.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:59:39 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T08:59:39.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:59:39 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T08:59:39.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:59:39 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T08:59:39.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:59:39 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T08:59:39.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:59:39 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T08:59:39.959 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:59:39 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T08:59:40.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:59:39 smithi083 ceph-mon[27907]: pgmap v250: 129 pgs: 129 active+clean; 48 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 506 KiB/s wr, 78 op/s 2024-04-24T08:59:40.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:59:39 smithi083 ceph-mon[27907]: scrub queued for path: / 2024-04-24T08:59:40.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:59:39 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T08:59:40.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:59:39 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T08:59:40.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:59:39 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T08:59:40.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:59:39 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T08:59:40.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:59:39 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T08:59:40.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:59:39 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T08:59:40.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:59:39 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T08:59:40.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:59:39 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T08:59:40.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:59:39 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T08:59:40.595 INFO:tasks.workunit.client.0.smithi083.stdout: 1 44591 0.62 MB/sec execute 254 sec latency 62.653 ms 2024-04-24T08:59:40.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:59:40 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T08:59:40.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:59:40 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T08:59:40.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:59:40 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T08:59:40.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:59:40 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T08:59:40.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:59:40 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T08:59:40.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:59:40 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T08:59:40.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:59:40 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T08:59:40.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:59:40 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T08:59:40.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:59:40 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T08:59:40.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:59:40 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T08:59:40.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:59:40 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T08:59:40.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:59:40 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T08:59:40.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:59:40 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T08:59:40.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:59:40 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T08:59:40.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:59:40 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T08:59:40.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:59:40 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T08:59:41.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:59:40 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T08:59:41.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:59:40 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T08:59:41.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:59:40 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T08:59:41.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:59:40 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T08:59:41.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:59:40 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T08:59:41.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:59:40 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T08:59:41.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:59:40 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T08:59:41.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:59:40 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T08:59:41.595 INFO:tasks.workunit.client.0.smithi083.stdout: 1 44705 0.62 MB/sec execute 255 sec latency 54.666 ms 2024-04-24T08:59:41.954 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:59:41 smithi138 ceph-mon[31441]: pgmap v251: 129 pgs: 129 active+clean; 48 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 307 KiB/s wr, 54 op/s 2024-04-24T08:59:41.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:59:41 smithi189 ceph-mon[31791]: pgmap v251: 129 pgs: 129 active+clean; 48 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 307 KiB/s wr, 54 op/s 2024-04-24T08:59:42.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:59:41 smithi083 ceph-mon[27907]: pgmap v251: 129 pgs: 129 active+clean; 48 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 307 KiB/s wr, 54 op/s 2024-04-24T08:59:42.595 INFO:tasks.workunit.client.0.smithi083.stdout: 1 44857 0.62 MB/sec execute 256 sec latency 76.472 ms 2024-04-24T08:59:43.595 INFO:tasks.workunit.client.0.smithi083.stdout: 1 44965 0.62 MB/sec execute 257 sec latency 50.704 ms 2024-04-24T08:59:43.954 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:59:43 smithi138 ceph-mon[31441]: pgmap v252: 129 pgs: 129 active+clean; 49 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 371 KiB/s wr, 81 op/s 2024-04-24T08:59:43.956 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:59:43 smithi189 ceph-mon[31791]: pgmap v252: 129 pgs: 129 active+clean; 49 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 371 KiB/s wr, 81 op/s 2024-04-24T08:59:44.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:59:43 smithi083 ceph-mon[27907]: pgmap v252: 129 pgs: 129 active+clean; 49 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 371 KiB/s wr, 81 op/s 2024-04-24T08:59:44.595 INFO:tasks.workunit.client.0.smithi083.stdout: 1 45046 0.62 MB/sec execute 258 sec latency 56.368 ms 2024-04-24T08:59:45.595 INFO:tasks.workunit.client.0.smithi083.stdout: 1 45173 0.62 MB/sec execute 259 sec latency 51.160 ms 2024-04-24T08:59:45.702 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.701+0000 7f394ea00640 1 -- 172.21.15.83:0/3473127672 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3948108190 msgr2=0x7f3948108570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:45.702 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.701+0000 7f394ea00640 1 --2- 172.21.15.83:0/3473127672 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3948108190 0x7f3948108570 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f3938009860 tx=0x7f393802a3d0 comp rx=0 tx=0).stop 2024-04-24T08:59:45.702 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.701+0000 7f394ea00640 1 -- 172.21.15.83:0/3473127672 shutdown_connections 2024-04-24T08:59:45.702 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.701+0000 7f394ea00640 1 --2- 172.21.15.83:0/3473127672 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f394806b1d0 0x7f394810eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:45.702 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.701+0000 7f394ea00640 1 --2- 172.21.15.83:0/3473127672 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f394806a830 0x7f394806ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:45.702 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.701+0000 7f394ea00640 1 --2- 172.21.15.83:0/3473127672 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3948108190 0x7f3948108570 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:45.702 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.701+0000 7f394ea00640 1 -- 172.21.15.83:0/3473127672 >> 172.21.15.83:0/3473127672 conn(0x7f3948076d00 msgr2=0x7f3948077110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:45.702 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.702+0000 7f394ea00640 1 -- 172.21.15.83:0/3473127672 shutdown_connections 2024-04-24T08:59:45.703 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.702+0000 7f394ea00640 1 -- 172.21.15.83:0/3473127672 wait complete. 2024-04-24T08:59:45.704 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.703+0000 7f394ea00640 1 Processor -- start 2024-04-24T08:59:45.704 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.703+0000 7f394ea00640 1 -- start start 2024-04-24T08:59:45.704 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.703+0000 7f394ea00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f394806a830 0x7f39481a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:45.704 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.703+0000 7f394ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f394806b1d0 0x7f39481a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:45.704 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.703+0000 7f394ea00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3948108190 0x7f394819a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:45.704 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.703+0000 7f394ea00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f39481122b0 con 0x7f394806b1d0 2024-04-24T08:59:45.704 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.703+0000 7f394ea00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3948112130 con 0x7f3948108190 2024-04-24T08:59:45.705 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.703+0000 7f394ea00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3948112430 con 0x7f394806a830 2024-04-24T08:59:45.705 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.704+0000 7f3947400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f394806b1d0 0x7f39481a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:45.705 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.704+0000 7f3947400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f394806b1d0 0x7f39481a0b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38252/0 (socket says 172.21.15.83:38252) 2024-04-24T08:59:45.705 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.704+0000 7f3947400640 1 -- 172.21.15.83:0/3395607817 learned_addr learned my addr 172.21.15.83:0/3395607817 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:59:45.705 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.704+0000 7f3947e00640 1 --2- 172.21.15.83:0/3395607817 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f394806a830 0x7f39481a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:45.705 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.704+0000 7f394ca00640 1 --2- 172.21.15.83:0/3395607817 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3948108190 0x7f394819a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:45.705 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.704+0000 7f3947400640 1 -- 172.21.15.83:0/3395607817 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f394806a830 msgr2=0x7f39481a0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:45.705 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.704+0000 7f3947400640 1 --2- 172.21.15.83:0/3395607817 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f394806a830 0x7f39481a0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:45.705 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.704+0000 7f3947400640 1 -- 172.21.15.83:0/3395607817 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3948108190 msgr2=0x7f394819a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:45.705 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.704+0000 7f3947400640 1 --2- 172.21.15.83:0/3395607817 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3948108190 0x7f394819a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:45.705 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.704+0000 7f3947400640 1 -- 172.21.15.83:0/3395607817 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f394819afb0 con 0x7f394806b1d0 2024-04-24T08:59:45.706 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.705+0000 7f3947400640 1 --2- 172.21.15.83:0/3395607817 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f394806b1d0 0x7f39481a0b70 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f393400e9f0 tx=0x7f393400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:45.706 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.705+0000 7f3944c00640 1 -- 172.21.15.83:0/3395607817 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f393400cd80 con 0x7f394806b1d0 2024-04-24T08:59:45.706 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.705+0000 7f3944c00640 1 -- 172.21.15.83:0/3395607817 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f39340054d0 con 0x7f394806b1d0 2024-04-24T08:59:45.706 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.705+0000 7f3944c00640 1 -- 172.21.15.83:0/3395607817 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3934010490 con 0x7f394806b1d0 2024-04-24T08:59:45.707 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.705+0000 7f394ea00640 1 -- 172.21.15.83:0/3395607817 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f394819b2a0 con 0x7f394806b1d0 2024-04-24T08:59:45.707 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.705+0000 7f394ea00640 1 -- 172.21.15.83:0/3395607817 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39481a75b0 con 0x7f394806b1d0 2024-04-24T08:59:45.709 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.709+0000 7f39517c2740 1 Processor -- start 2024-04-24T08:59:45.709 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.709+0000 7f39517c2740 1 -- start start 2024-04-24T08:59:45.709 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.709+0000 7f3944c00640 1 -- 172.21.15.83:0/3395607817 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3934005020 con 0x7f394806b1d0 2024-04-24T08:59:45.713 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.709+0000 7f39517c2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654c9a431b0 0x5654c9a43590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:45.713 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.709+0000 7f39517c2740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5654c9a43ad0 0x5654c9a4de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:45.713 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.709+0000 7f39517c2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654c9a4e390 0x5654c9a50780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:45.713 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.710+0000 7f39517c2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5654c99a9900 con 0x5654c9a4e390 2024-04-24T08:59:45.713 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.710+0000 7f39517c2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5654c994ff20 con 0x5654c9a431b0 2024-04-24T08:59:45.713 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.710+0000 7f39517c2740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5654c99a58c0 con 0x5654c9a43ad0 2024-04-24T08:59:45.713 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.710+0000 7f3947400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654c9a4e390 0x5654c9a50780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:45.713 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.710+0000 7f3947400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654c9a4e390 0x5654c9a50780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38268/0 (socket says 172.21.15.83:38268) 2024-04-24T08:59:45.714 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.710+0000 7f3947400640 1 -- 172.21.15.83:0/39581457 learned_addr learned my addr 172.21.15.83:0/39581457 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:59:45.714 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.710+0000 7f394ca00640 1 --2- 172.21.15.83:0/39581457 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654c9a431b0 0x5654c9a43590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:45.714 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.710+0000 7f3947e00640 1 --2- 172.21.15.83:0/39581457 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5654c9a43ad0 0x5654c9a4de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:45.714 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.711+0000 7f3944c00640 1 --2- 172.21.15.83:0/3395607817 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f392c0a3dc0 0x7f392c0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:45.714 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.711+0000 7f3947400640 1 -- 172.21.15.83:0/39581457 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5654c9a43ad0 msgr2=0x5654c9a4de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:45.714 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.711+0000 7f3947400640 1 --2- 172.21.15.83:0/39581457 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5654c9a43ad0 0x5654c9a4de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:45.714 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.711+0000 7f3947400640 1 -- 172.21.15.83:0/39581457 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654c9a431b0 msgr2=0x5654c9a43590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:45.714 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.711+0000 7f3947400640 1 --2- 172.21.15.83:0/39581457 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654c9a431b0 0x5654c9a43590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:45.714 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.711+0000 7f3947400640 1 -- 172.21.15.83:0/39581457 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654c9974e10 con 0x5654c9a4e390 2024-04-24T08:59:45.714 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.711+0000 7f394ca00640 1 --2- 172.21.15.83:0/3395607817 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f392c0a3dc0 0x7f392c0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:45.714 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.711+0000 7f3944c00640 1 -- 172.21.15.83:0/3395607817 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7f39340d10b0 con 0x7f394806b1d0 2024-04-24T08:59:45.715 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.711+0000 7f3947400640 1 --2- 172.21.15.83:0/39581457 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654c9a4e390 0x5654c9a50780 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f39340ce2c0 tx=0x7f3934067290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e7cbeb6e5a7894b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:45.715 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.711+0000 7f393d200640 1 -- 172.21.15.83:0/39581457 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f393409c710 con 0x5654c9a4e390 2024-04-24T08:59:45.715 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.711+0000 7f393d200640 1 -- 172.21.15.83:0/39581457 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f393407c2f0 con 0x5654c9a4e390 2024-04-24T08:59:45.715 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.711+0000 7f393d200640 1 -- 172.21.15.83:0/39581457 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39340a2110 con 0x5654c9a4e390 2024-04-24T08:59:45.715 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.712+0000 7f39517c2740 1 -- 172.21.15.83:0/39581457 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654c9a4e390 msgr2=0x5654c9a50780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:45.715 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.712+0000 7f394ca00640 1 --2- 172.21.15.83:0/3395607817 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f392c0a3dc0 0x7f392c0a6280 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3940005fd0 tx=0x7f3940011040 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:45.715 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.712+0000 7f39517c2740 1 --2- 172.21.15.83:0/39581457 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654c9a4e390 0x5654c9a50780 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f39340ce2c0 tx=0x7f3934067290 comp rx=0 tx=0).stop 2024-04-24T08:59:45.715 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.712+0000 7f39517c2740 1 -- 172.21.15.83:0/39581457 shutdown_connections 2024-04-24T08:59:45.715 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.712+0000 7f39517c2740 1 --2- 172.21.15.83:0/39581457 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654c9a4e390 0x5654c9a50780 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:45.715 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.712+0000 7f39517c2740 1 --2- 172.21.15.83:0/39581457 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5654c9a43ad0 0x5654c9a4de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:45.715 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.712+0000 7f39517c2740 1 --2- 172.21.15.83:0/39581457 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654c9a431b0 0x5654c9a43590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:45.715 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.712+0000 7f39517c2740 1 -- 172.21.15.83:0/39581457 >> 172.21.15.83:0/39581457 conn(0x5654c9a3f6c0 msgr2=0x5654c9a3faa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:45.715 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.712+0000 7f39517c2740 1 -- 172.21.15.83:0/39581457 shutdown_connections 2024-04-24T08:59:45.715 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.712+0000 7f39517c2740 1 -- 172.21.15.83:0/39581457 wait complete. 2024-04-24T08:59:45.716 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.713+0000 7f39517c2740 1 -- start start 2024-04-24T08:59:45.716 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.713+0000 7f39517c2740 1 Processor -- start 2024-04-24T08:59:45.719 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.714+0000 7f39517c2740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5654c9a431b0 0x5654c9a4b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:45.719 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.714+0000 7f39517c2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654c9a43ad0 0x5654c9a4b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:45.719 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.714+0000 7f39517c2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654c9a4e390 0x5654c9a4bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:45.719 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.714+0000 7f39517c2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5654c9a50f60 con 0x5654c9a4e390 2024-04-24T08:59:45.719 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.714+0000 7f39517c2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5654c99632c0 con 0x5654c9a43ad0 2024-04-24T08:59:45.719 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.714+0000 7f39517c2740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5654c9a510e0 con 0x5654c9a431b0 2024-04-24T08:59:45.719 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.714+0000 7f39517c2740 10 client.0 _ll_register_callbacks cb 0x5654c96bba00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:59:45.719 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.714+0000 7f394ca00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654c9a4e390 0x5654c9a4bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:45.719 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.714+0000 7f394ca00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654c9a4e390 0x5654c9a4bd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38278/0 (socket says 172.21.15.83:38278) 2024-04-24T08:59:45.719 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.714+0000 7f394ca00640 1 -- 172.21.15.83:0/1651384766 learned_addr learned my addr 172.21.15.83:0/1651384766 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:59:45.720 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.714+0000 7f3947400640 1 --2- 172.21.15.83:0/1651384766 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654c9a43ad0 0x5654c9a4b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:45.720 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.714+0000 7f3947e00640 1 --2- 172.21.15.83:0/1651384766 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5654c9a431b0 0x5654c9a4b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:45.720 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.715+0000 7f394ca00640 1 -- 172.21.15.83:0/1651384766 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5654c9a431b0 msgr2=0x5654c9a4b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:45.720 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.715+0000 7f394ca00640 1 --2- 172.21.15.83:0/1651384766 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5654c9a431b0 0x5654c9a4b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:45.720 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.715+0000 7f394ca00640 1 -- 172.21.15.83:0/1651384766 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654c9a43ad0 msgr2=0x5654c9a4b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:45.720 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.715+0000 7f394ca00640 1 --2- 172.21.15.83:0/1651384766 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654c9a43ad0 0x5654c9a4b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:45.720 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.715+0000 7f394ca00640 1 -- 172.21.15.83:0/1651384766 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654c9973fd0 con 0x5654c9a4e390 2024-04-24T08:59:45.720 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.715+0000 7f3947400640 1 --2- 172.21.15.83:0/1651384766 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654c9a43ad0 0x5654c9a4b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:59:45.720 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.715+0000 7f3947e00640 1 --2- 172.21.15.83:0/1651384766 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5654c9a431b0 0x5654c9a4b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T08:59:45.720 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.715+0000 7f394ca00640 1 --2- 172.21.15.83:0/1651384766 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654c9a4e390 0x5654c9a4bd70 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f394000dad0 tx=0x7f39400095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c16a003aa6c5574b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:45.720 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.715+0000 7f3931600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:59:45.720 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.715+0000 7f3931600640 1 -- 172.21.15.83:0/1651384766 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f394002b060 con 0x5654c9a4e390 2024-04-24T08:59:45.720 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.715+0000 7f3931600640 1 -- 172.21.15.83:0/1651384766 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7f3940026400 con 0x5654c9a4e390 2024-04-24T08:59:45.720 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.715+0000 7f39517c2740 1 -- 172.21.15.83:0/1651384766 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5654c99b7320 con 0x5654c9a4e390 2024-04-24T08:59:45.720 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.716+0000 7f3931600640 1 -- 172.21.15.83:0/1651384766 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f3940032050 con 0x5654c9a4e390 2024-04-24T08:59:45.721 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.716+0000 7f39517c2740 10 client.18588 fetch_fsmap learned FSMap version 35 2024-04-24T08:59:45.721 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.716+0000 7f39517c2740 1 -- 172.21.15.83:0/1651384766 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5654c96bc210 con 0x5654c9a4e390 2024-04-24T08:59:45.721 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.716+0000 7f3931600640 1 -- 172.21.15.83:0/1651384766 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x5654c99b7320 con 0x5654c9a4e390 2024-04-24T08:59:45.721 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.716+0000 7f3931600640 1 -- 172.21.15.83:0/1651384766 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7f394000b040 con 0x5654c9a4e390 2024-04-24T08:59:45.721 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.716+0000 7f39517c2740 10 client.18588 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T08:59:45.721 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.716+0000 7f39517c2740 10 client.18588 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T08:59:45.721 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.716+0000 7f39517c2740 20 client.18588 populate_metadata read hostname 'smithi083' 2024-04-24T08:59:45.721 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.716+0000 7f39517c2740 1 --2- 172.21.15.83:0/1651384766 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5654c9a5e490 0x5654c9a608a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:45.721 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.716+0000 7f39517c2740 4 client.18588 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:59:45.721 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.716+0000 7f39517c2740 1 -- 172.21.15.83:0/1651384766 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5654c99b1190 con 0x5654c9a5e490 2024-04-24T08:59:45.721 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.717+0000 7f3947e00640 1 --2- 172.21.15.83:0/1651384766 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5654c9a5e490 0x5654c9a608a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:45.721 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.718+0000 7f3931600640 10 client.18588 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:59:45.721 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.718+0000 7f3947e00640 1 --2- 172.21.15.83:0/1651384766 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5654c9a5e490 0x5654c9a608a0 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89f3f7b6b84a574d server_cookie=f88ca10ecf150c6c in_seq=0 out_seq=0 2024-04-24T08:59:45.723 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.721+0000 7f3931600640 1 -- 172.21.15.83:0/1651384766 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5654c99b1190 con 0x5654c9a5e490 2024-04-24T08:59:45.723 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.721+0000 7f3931600640 10 client.18588 handle_command_reply: tid=0 2024-04-24T08:59:45.723 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.721+0000 7f39517c2740 1 client.18588 shutdown 2024-04-24T08:59:45.723 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.721+0000 7f39517c2740 1 -- 172.21.15.83:0/1651384766 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654c9a4e390 msgr2=0x5654c9a4bd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:45.723 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.721+0000 7f39517c2740 1 --2- 172.21.15.83:0/1651384766 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654c9a4e390 0x5654c9a4bd70 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f394000dad0 tx=0x7f39400095c0 comp rx=0 tx=0).stop 2024-04-24T08:59:45.723 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.721+0000 7f39517c2740 1 -- 172.21.15.83:0/1651384766 shutdown_connections 2024-04-24T08:59:45.723 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.721+0000 7f39517c2740 1 --2- 172.21.15.83:0/1651384766 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5654c9a5e490 0x5654c9a608a0 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:45.723 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.721+0000 7f3931600640 0 client.18588 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:59:45.723 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.721+0000 7f39517c2740 1 --2- 172.21.15.83:0/1651384766 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654c9a4e390 0x5654c9a4bd70 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:45.723 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.721+0000 7f39517c2740 1 --2- 172.21.15.83:0/1651384766 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654c9a43ad0 0x5654c9a4b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:45.723 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.721+0000 7f39517c2740 1 --2- 172.21.15.83:0/1651384766 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5654c9a431b0 0x5654c9a4b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:45.723 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.721+0000 7f39517c2740 1 -- 172.21.15.83:0/1651384766 >> 172.21.15.83:0/1651384766 conn(0x5654c9a3f6c0 msgr2=0x5654c9a45440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:45.723 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.721+0000 7f39517c2740 1 -- 172.21.15.83:0/1651384766 shutdown_connections 2024-04-24T08:59:45.723 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.721+0000 7f39517c2740 1 -- 172.21.15.83:0/1651384766 wait complete. 2024-04-24T08:59:45.724 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.722+0000 7f39517c2740 20 client.18588 trim_cache size 0 max 16384 2024-04-24T08:59:45.740 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.739+0000 7f39517c2740 1 Processor -- start 2024-04-24T08:59:45.740 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.739+0000 7f39517c2740 1 -- start start 2024-04-24T08:59:45.740 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.739+0000 7f39517c2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654c9a431b0 0x5654c9a59e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:45.740 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.739+0000 7f39517c2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654c9a43ad0 0x5654c9a45b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:45.741 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.739+0000 7f39517c2740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5654c9a4e390 0x5654c9a4d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:45.741 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.739+0000 7f39517c2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f39481b16b0 con 0x5654c9a43ad0 2024-04-24T08:59:45.741 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.739+0000 7f39517c2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f39481b31f0 con 0x5654c9a431b0 2024-04-24T08:59:45.741 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.739+0000 7f39517c2740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f39481b1d80 con 0x5654c9a4e390 2024-04-24T08:59:45.741 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.740+0000 7f3947400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654c9a43ad0 0x5654c9a45b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:45.741 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.740+0000 7f3947400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654c9a43ad0 0x5654c9a45b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38286/0 (socket says 172.21.15.83:38286) 2024-04-24T08:59:45.741 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.740+0000 7f3947400640 1 -- 172.21.15.83:0/3538554020 learned_addr learned my addr 172.21.15.83:0/3538554020 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:59:45.741 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.740+0000 7f3947e00640 1 --2- 172.21.15.83:0/3538554020 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654c9a431b0 0x5654c9a59e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:45.741 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.740+0000 7f394ca00640 1 --2- 172.21.15.83:0/3538554020 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5654c9a4e390 0x5654c9a4d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:45.741 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.740+0000 7f3947400640 1 -- 172.21.15.83:0/3538554020 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5654c9a4e390 msgr2=0x5654c9a4d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:45.741 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.740+0000 7f3947400640 1 --2- 172.21.15.83:0/3538554020 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5654c9a4e390 0x5654c9a4d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:45.741 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.740+0000 7f3947400640 1 -- 172.21.15.83:0/3538554020 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654c9a431b0 msgr2=0x5654c9a59e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:45.741 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.740+0000 7f3947400640 1 --2- 172.21.15.83:0/3538554020 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654c9a431b0 0x5654c9a59e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:45.741 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.740+0000 7f3947400640 1 -- 172.21.15.83:0/3538554020 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654c9788620 con 0x5654c9a43ad0 2024-04-24T08:59:45.741 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.741+0000 7f3947400640 1 --2- 172.21.15.83:0/3538554020 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654c9a43ad0 0x5654c9a45b70 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f39340a5310 tx=0x7f393400c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1dc521827b5cd29 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:45.746 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.741+0000 7f3933e00640 1 -- 172.21.15.83:0/3538554020 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f393409c530 con 0x5654c9a43ad0 2024-04-24T08:59:45.746 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.741+0000 7f3933e00640 1 -- 172.21.15.83:0/3538554020 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3934005310 con 0x5654c9a43ad0 2024-04-24T08:59:45.746 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.741+0000 7f3933e00640 1 -- 172.21.15.83:0/3538554020 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39340a0f80 con 0x5654c9a43ad0 2024-04-24T08:59:45.746 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.741+0000 7f39517c2740 1 -- 172.21.15.83:0/3538554020 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654c9a43ad0 msgr2=0x5654c9a45b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:45.747 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.741+0000 7f39517c2740 1 --2- 172.21.15.83:0/3538554020 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654c9a43ad0 0x5654c9a45b70 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f39340a5310 tx=0x7f393400c7d0 comp rx=0 tx=0).stop 2024-04-24T08:59:45.747 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.742+0000 7f39517c2740 1 -- 172.21.15.83:0/3538554020 shutdown_connections 2024-04-24T08:59:45.747 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.742+0000 7f39517c2740 1 --2- 172.21.15.83:0/3538554020 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5654c9a4e390 0x5654c9a4d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:45.747 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.742+0000 7f39517c2740 1 --2- 172.21.15.83:0/3538554020 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654c9a43ad0 0x5654c9a45b70 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:45.747 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.742+0000 7f39517c2740 1 --2- 172.21.15.83:0/3538554020 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654c9a431b0 0x5654c9a59e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:45.747 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.742+0000 7f39517c2740 1 -- 172.21.15.83:0/3538554020 >> 172.21.15.83:0/3538554020 conn(0x5654c9a3f6c0 msgr2=0x5654c9a4cb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:45.747 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.742+0000 7f39517c2740 1 -- 172.21.15.83:0/3538554020 shutdown_connections 2024-04-24T08:59:45.747 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.742+0000 7f39517c2740 1 -- 172.21.15.83:0/3538554020 wait complete. 2024-04-24T08:59:45.747 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.742+0000 7f39517c2740 1 -- start start 2024-04-24T08:59:45.747 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.742+0000 7f39517c2740 1 Processor -- start 2024-04-24T08:59:45.747 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.743+0000 7f39517c2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654c9a431b0 0x5654c9a483e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:45.747 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.743+0000 7f39517c2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654c9a43ad0 0x5654c9a48920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:45.747 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.743+0000 7f39517c2740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5654c9a4e390 0x5654c9ac3a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:45.747 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.743+0000 7f39517c2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f39481b2b20 con 0x5654c9a43ad0 2024-04-24T08:59:45.748 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.743+0000 7f39517c2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f39481b2450 con 0x5654c9a431b0 2024-04-24T08:59:45.748 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.743+0000 7f39517c2740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f39481af6b0 con 0x5654c9a4e390 2024-04-24T08:59:45.748 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.743+0000 7f39517c2740 10 client.0 _ll_register_callbacks cb 0x7f3940009820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:59:45.748 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.743+0000 7f3947400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654c9a43ad0 0x5654c9a48920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:45.748 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.743+0000 7f3947400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654c9a43ad0 0x5654c9a48920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38296/0 (socket says 172.21.15.83:38296) 2024-04-24T08:59:45.748 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.743+0000 7f3947400640 1 -- 172.21.15.83:0/2929616021 learned_addr learned my addr 172.21.15.83:0/2929616021 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:59:45.748 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.743+0000 7f3947e00640 1 --2- 172.21.15.83:0/2929616021 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654c9a431b0 0x5654c9a483e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:45.748 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.743+0000 7f394ca00640 1 --2- 172.21.15.83:0/2929616021 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5654c9a4e390 0x5654c9ac3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:45.748 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.743+0000 7f3947400640 1 -- 172.21.15.83:0/2929616021 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5654c9a4e390 msgr2=0x5654c9ac3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:45.748 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.743+0000 7f3947400640 1 --2- 172.21.15.83:0/2929616021 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5654c9a4e390 0x5654c9ac3a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:45.748 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.743+0000 7f3947400640 1 -- 172.21.15.83:0/2929616021 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654c9a431b0 msgr2=0x5654c9a483e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:45.748 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.743+0000 7f3947400640 1 --2- 172.21.15.83:0/2929616021 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654c9a431b0 0x5654c9a483e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:45.749 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.743+0000 7f3947400640 1 -- 172.21.15.83:0/2929616021 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654c9ac3f60 con 0x5654c9a43ad0 2024-04-24T08:59:45.749 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.743+0000 7f394ca00640 1 --2- 172.21.15.83:0/2929616021 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5654c9a4e390 0x5654c9ac3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:59:45.749 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.743+0000 7f3947e00640 1 --2- 172.21.15.83:0/2929616021 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654c9a431b0 0x5654c9a483e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:59:45.749 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.744+0000 7f3947400640 1 --2- 172.21.15.83:0/2929616021 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654c9a43ad0 0x5654c9a48920 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f393400c6a0 tx=0x7f39340ce3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b910d0f919ce9dff server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:45.749 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.744+0000 7f3931600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:59:45.749 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.744+0000 7f3931600640 1 -- 172.21.15.83:0/2929616021 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39340b6060 con 0x5654c9a43ad0 2024-04-24T08:59:45.749 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.744+0000 7f3931600640 1 -- 172.21.15.83:0/2929616021 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7f3934099240 con 0x5654c9a43ad0 2024-04-24T08:59:45.749 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.744+0000 7f39517c2740 1 -- 172.21.15.83:0/2929616021 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5654c992d4b0 con 0x5654c9a43ad0 2024-04-24T08:59:45.749 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.744+0000 7f3931600640 1 -- 172.21.15.83:0/2929616021 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f393409c530 con 0x5654c9a43ad0 2024-04-24T08:59:45.749 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.744+0000 7f39517c2740 10 client.18594 fetch_fsmap learned FSMap version 35 2024-04-24T08:59:45.750 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.744+0000 7f39517c2740 1 -- 172.21.15.83:0/2929616021 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5654c9ac50f0 con 0x5654c9a43ad0 2024-04-24T08:59:45.750 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.744+0000 7f3931600640 1 -- 172.21.15.83:0/2929616021 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x5654c992d4b0 con 0x5654c9a43ad0 2024-04-24T08:59:45.750 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.745+0000 7f3931600640 1 -- 172.21.15.83:0/2929616021 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7f393406b0b0 con 0x5654c9a43ad0 2024-04-24T08:59:45.750 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.745+0000 7f39517c2740 10 client.18594 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T08:59:45.750 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.745+0000 7f39517c2740 10 client.18594 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T08:59:45.750 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.745+0000 7f39517c2740 20 client.18594 populate_metadata read hostname 'smithi083' 2024-04-24T08:59:45.750 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.745+0000 7f39517c2740 1 --2- 172.21.15.83:0/2929616021 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5654c9a5e490 0x5654c9ac7370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:45.750 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.745+0000 7f39517c2740 4 client.18594 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T08:59:45.750 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.745+0000 7f39517c2740 1 -- 172.21.15.83:0/2929616021 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5654c997a310 con 0x5654c9a5e490 2024-04-24T08:59:45.750 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.745+0000 7f3947e00640 1 --2- 172.21.15.83:0/2929616021 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5654c9a5e490 0x5654c9ac7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:45.751 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.746+0000 7f3947e00640 1 --2- 172.21.15.83:0/2929616021 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5654c9a5e490 0x5654c9ac7370 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d870e38555128398 server_cookie=fce3f1da3eaa8e33 in_seq=0 out_seq=0 2024-04-24T08:59:45.751 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.746+0000 7f3931600640 10 client.18594 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:59:45.803 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.801+0000 7f3931600640 1 -- 172.21.15.83:0/2929616021 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5654c997a310 con 0x5654c9a5e490 2024-04-24T08:59:45.803 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.801+0000 7f3931600640 10 client.18594 handle_command_reply: tid=0 2024-04-24T08:59:45.803 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.801+0000 7f39517c2740 1 client.18594 shutdown 2024-04-24T08:59:45.803 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.802+0000 7f39517c2740 1 -- 172.21.15.83:0/2929616021 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654c9a43ad0 msgr2=0x5654c9a48920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:45.803 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.802+0000 7f39517c2740 1 --2- 172.21.15.83:0/2929616021 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654c9a43ad0 0x5654c9a48920 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f393400c6a0 tx=0x7f39340ce3a0 comp rx=0 tx=0).stop 2024-04-24T08:59:45.803 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.802+0000 7f39517c2740 1 -- 172.21.15.83:0/2929616021 shutdown_connections 2024-04-24T08:59:45.803 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.802+0000 7f39517c2740 1 --2- 172.21.15.83:0/2929616021 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5654c9a5e490 0x5654c9ac7370 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:45.803 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.802+0000 7f39517c2740 1 --2- 172.21.15.83:0/2929616021 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5654c9a4e390 0x5654c9ac3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:45.803 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.802+0000 7f39517c2740 1 --2- 172.21.15.83:0/2929616021 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654c9a43ad0 0x5654c9a48920 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:45.803 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.802+0000 7f39517c2740 1 --2- 172.21.15.83:0/2929616021 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654c9a431b0 0x5654c9a483e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:45.803 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.802+0000 7f39517c2740 1 -- 172.21.15.83:0/2929616021 >> 172.21.15.83:0/2929616021 conn(0x5654c9a3f6c0 msgr2=0x5654c9a3e840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:45.803 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.802+0000 7f3931600640 0 client.18594 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:59:45.804 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.802+0000 7f39517c2740 1 -- 172.21.15.83:0/2929616021 shutdown_connections 2024-04-24T08:59:45.804 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.802+0000 7f39517c2740 1 -- 172.21.15.83:0/2929616021 wait complete. 2024-04-24T08:59:45.804 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.803+0000 7f39517c2740 20 client.18594 trim_cache size 0 max 16384 2024-04-24T08:59:45.804 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T08:59:45.804 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T08:59:45.804 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T08:59:45.804 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T08:59:45.805 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.804+0000 7f394ea00640 1 -- 172.21.15.83:0/3395607817 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f392c0a3dc0 msgr2=0x7f392c0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:45.805 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.804+0000 7f394ea00640 1 --2- 172.21.15.83:0/3395607817 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f392c0a3dc0 0x7f392c0a6280 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3940005fd0 tx=0x7f3940011040 comp rx=0 tx=0).stop 2024-04-24T08:59:45.805 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.804+0000 7f394ea00640 1 -- 172.21.15.83:0/3395607817 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f394806b1d0 msgr2=0x7f39481a0b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:45.805 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.804+0000 7f394ea00640 1 --2- 172.21.15.83:0/3395607817 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f394806b1d0 0x7f39481a0b70 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f393400e9f0 tx=0x7f393400edf0 comp rx=0 tx=0).stop 2024-04-24T08:59:45.805 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.804+0000 7f394ea00640 1 -- 172.21.15.83:0/3395607817 shutdown_connections 2024-04-24T08:59:45.805 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.804+0000 7f394ea00640 1 --2- 172.21.15.83:0/3395607817 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f392c0a3dc0 0x7f392c0a6280 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:45.806 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.804+0000 7f394ea00640 1 --2- 172.21.15.83:0/3395607817 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3948108190 0x7f394819a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:45.806 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.804+0000 7f394ea00640 1 --2- 172.21.15.83:0/3395607817 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f394806b1d0 0x7f39481a0b70 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:45.806 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.804+0000 7f394ea00640 1 --2- 172.21.15.83:0/3395607817 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f394806a830 0x7f39481a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:45.806 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.804+0000 7f394ea00640 1 -- 172.21.15.83:0/3395607817 >> 172.21.15.83:0/3395607817 conn(0x7f3948076d00 msgr2=0x7f394810d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:45.806 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.804+0000 7f394ea00640 1 -- 172.21.15.83:0/3395607817 shutdown_connections 2024-04-24T08:59:45.806 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:45.804+0000 7f394ea00640 1 -- 172.21.15.83:0/3395607817 wait complete. 2024-04-24T08:59:45.954 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:59:45 smithi138 ceph-mon[31441]: pgmap v253: 129 pgs: 129 active+clean; 49 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 248 KiB/s wr, 54 op/s 2024-04-24T08:59:45.956 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:59:45 smithi189 ceph-mon[31791]: pgmap v253: 129 pgs: 129 active+clean; 49 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 248 KiB/s wr, 54 op/s 2024-04-24T08:59:46.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:59:45 smithi083 ceph-mon[27907]: pgmap v253: 129 pgs: 129 active+clean; 49 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 248 KiB/s wr, 54 op/s 2024-04-24T08:59:46.596 INFO:tasks.workunit.client.0.smithi083.stdout: 1 45356 0.63 MB/sec execute 260 sec latency 50.975 ms 2024-04-24T08:59:47.596 INFO:tasks.workunit.client.0.smithi083.stdout: 1 45533 0.63 MB/sec execute 261 sec latency 54.561 ms 2024-04-24T08:59:48.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:59:47 smithi083 ceph-mon[27907]: pgmap v254: 129 pgs: 129 active+clean; 49 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 248 KiB/s wr, 54 op/s 2024-04-24T08:59:48.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:59:47 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T08:59:48.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:59:47 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T08:59:48.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:59:47 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T08:59:48.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:59:47 smithi138 ceph-mon[31441]: pgmap v254: 129 pgs: 129 active+clean; 49 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 248 KiB/s wr, 54 op/s 2024-04-24T08:59:48.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:59:47 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T08:59:48.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:59:47 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T08:59:48.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:59:47 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T08:59:48.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:59:47 smithi189 ceph-mon[31791]: pgmap v254: 129 pgs: 129 active+clean; 49 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 248 KiB/s wr, 54 op/s 2024-04-24T08:59:48.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:59:47 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T08:59:48.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:59:47 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T08:59:48.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:59:47 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T08:59:48.596 INFO:tasks.workunit.client.0.smithi083.stdout: 1 45770 0.63 MB/sec execute 262 sec latency 56.177 ms 2024-04-24T08:59:49.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:59:48 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T08:59:49.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:59:48 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T08:59:49.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:59:48 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T08:59:49.596 INFO:tasks.workunit.client.0.smithi083.stdout: 1 45887 0.64 MB/sec execute 263 sec latency 55.201 ms 2024-04-24T08:59:50.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:59:49 smithi083 ceph-mon[27907]: pgmap v255: 129 pgs: 129 active+clean; 48 MiB data, 703 MiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 594 KiB/s wr, 78 op/s 2024-04-24T08:59:50.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:59:49 smithi138 ceph-mon[31441]: pgmap v255: 129 pgs: 129 active+clean; 48 MiB data, 703 MiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 594 KiB/s wr, 78 op/s 2024-04-24T08:59:50.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:59:49 smithi189 ceph-mon[31791]: pgmap v255: 129 pgs: 129 active+clean; 48 MiB data, 703 MiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 594 KiB/s wr, 78 op/s 2024-04-24T08:59:50.596 INFO:tasks.workunit.client.0.smithi083.stdout: 1 45915 0.63 MB/sec execute 264 sec latency 58.928 ms 2024-04-24T08:59:51.596 INFO:tasks.workunit.client.0.smithi083.stdout: 1 45956 0.63 MB/sec execute 265 sec latency 55.736 ms 2024-04-24T08:59:52.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:59:51 smithi083 ceph-mon[27907]: pgmap v256: 129 pgs: 129 active+clean; 48 MiB data, 703 MiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 410 KiB/s wr, 51 op/s 2024-04-24T08:59:52.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:59:51 smithi138 ceph-mon[31441]: pgmap v256: 129 pgs: 129 active+clean; 48 MiB data, 703 MiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 410 KiB/s wr, 51 op/s 2024-04-24T08:59:52.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:59:51 smithi189 ceph-mon[31791]: pgmap v256: 129 pgs: 129 active+clean; 48 MiB data, 703 MiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 410 KiB/s wr, 51 op/s 2024-04-24T08:59:52.596 INFO:tasks.workunit.client.0.smithi083.stdout: 1 46135 0.64 MB/sec execute 266 sec latency 57.245 ms 2024-04-24T08:59:53.596 INFO:tasks.workunit.client.0.smithi083.stdout: 1 46261 0.64 MB/sec execute 267 sec latency 55.421 ms 2024-04-24T08:59:54.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:59:53 smithi083 ceph-mon[27907]: pgmap v257: 129 pgs: 129 active+clean; 48 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 573 KiB/s wr, 68 op/s 2024-04-24T08:59:54.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:59:53 smithi138 ceph-mon[31441]: pgmap v257: 129 pgs: 129 active+clean; 48 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 573 KiB/s wr, 68 op/s 2024-04-24T08:59:54.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:59:53 smithi189 ceph-mon[31791]: pgmap v257: 129 pgs: 129 active+clean; 48 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 573 KiB/s wr, 68 op/s 2024-04-24T08:59:54.596 INFO:tasks.workunit.client.0.smithi083.stdout: 1 46368 0.64 MB/sec execute 268 sec latency 55.958 ms 2024-04-24T08:59:55.596 INFO:tasks.workunit.client.0.smithi083.stdout: 1 46424 0.64 MB/sec execute 269 sec latency 46.879 ms 2024-04-24T08:59:56.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:59:55 smithi083 ceph-mon[27907]: pgmap v258: 129 pgs: 129 active+clean; 48 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 509 KiB/s wr, 42 op/s 2024-04-24T08:59:56.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:59:55 smithi138 ceph-mon[31441]: pgmap v258: 129 pgs: 129 active+clean; 48 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 509 KiB/s wr, 42 op/s 2024-04-24T08:59:56.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:59:55 smithi189 ceph-mon[31791]: pgmap v258: 129 pgs: 129 active+clean; 48 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 509 KiB/s wr, 42 op/s 2024-04-24T08:59:56.596 INFO:tasks.workunit.client.0.smithi083.stdout: 1 46486 0.64 MB/sec execute 270 sec latency 48.365 ms 2024-04-24T08:59:57.598 INFO:tasks.workunit.client.0.smithi083.stdout: 1 46547 0.64 MB/sec execute 271 sec latency 48.688 ms 2024-04-24T08:59:58.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:59:57 smithi083 ceph-mon[27907]: pgmap v259: 129 pgs: 129 active+clean; 48 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 509 KiB/s wr, 42 op/s 2024-04-24T08:59:58.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:59:57 smithi138 ceph-mon[31441]: pgmap v259: 129 pgs: 129 active+clean; 48 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 509 KiB/s wr, 42 op/s 2024-04-24T08:59:58.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:59:57 smithi189 ceph-mon[31791]: pgmap v259: 129 pgs: 129 active+clean; 48 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 509 KiB/s wr, 42 op/s 2024-04-24T08:59:58.597 INFO:tasks.workunit.client.0.smithi083.stdout: 1 46608 0.63 MB/sec execute 272 sec latency 39.746 ms 2024-04-24T08:59:58.957 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.957+0000 7f6828000640 1 -- 172.21.15.83:0/4228862141 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6820104e00 msgr2=0x7f6820108e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:58.957 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.957+0000 7f6828000640 1 --2- 172.21.15.83:0/4228862141 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6820104e00 0x7f6820108e40 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f680c009900 tx=0x7f680c02a410 comp rx=0 tx=0).stop 2024-04-24T08:59:58.957 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.957+0000 7f6828000640 1 -- 172.21.15.83:0/4228862141 shutdown_connections 2024-04-24T08:59:58.957 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.957+0000 7f6828000640 1 --2- 172.21.15.83:0/4228862141 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6820109520 0x7f68201110f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:58.957 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.957+0000 7f6828000640 1 --2- 172.21.15.83:0/4228862141 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6820104e00 0x7f6820108e40 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:58.958 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.957+0000 7f6828000640 1 --2- 172.21.15.83:0/4228862141 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6820104450 0x7f6820104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:58.958 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.957+0000 7f6828000640 1 -- 172.21.15.83:0/4228862141 >> 172.21.15.83:0/4228862141 conn(0x7f68200ffcf0 msgr2=0x7f6820102110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:58.958 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.957+0000 7f6828000640 1 -- 172.21.15.83:0/4228862141 shutdown_connections 2024-04-24T08:59:58.958 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.957+0000 7f6828000640 1 -- 172.21.15.83:0/4228862141 wait complete. 2024-04-24T08:59:58.958 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.958+0000 7f6828000640 1 Processor -- start 2024-04-24T08:59:58.958 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.958+0000 7f6828000640 1 -- start start 2024-04-24T08:59:58.959 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.959+0000 7f6828000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6820104450 0x7f68201a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:58.960 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.959+0000 7f6828000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6820109520 0x7f68201a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:58.960 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.959+0000 7f6828000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f682019a7e0 0x7f682019ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:58.960 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.959+0000 7f6828000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6820114460 con 0x7f682019a7e0 2024-04-24T08:59:58.960 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.959+0000 7f6828000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f68201142e0 con 0x7f6820109520 2024-04-24T08:59:58.960 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.959+0000 7f6828000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f68201145e0 con 0x7f6820104450 2024-04-24T08:59:58.960 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.959+0000 7f6826000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f682019a7e0 0x7f682019ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:58.960 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.959+0000 7f6826000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f682019a7e0 0x7f682019ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48476/0 (socket says 172.21.15.83:48476) 2024-04-24T08:59:58.960 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.959+0000 7f6826000640 1 -- 172.21.15.83:0/1415577237 learned_addr learned my addr 172.21.15.83:0/1415577237 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:59:58.960 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.959+0000 7f6825600640 1 --2- 172.21.15.83:0/1415577237 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6820104450 0x7f68201a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:58.960 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.960+0000 7f6824c00640 1 --2- 172.21.15.83:0/1415577237 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6820109520 0x7f68201a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:58.960 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.960+0000 7f6826000640 1 -- 172.21.15.83:0/1415577237 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6820104450 msgr2=0x7f68201a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:58.960 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.960+0000 7f6826000640 1 --2- 172.21.15.83:0/1415577237 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6820104450 0x7f68201a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:58.961 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.960+0000 7f6826000640 1 -- 172.21.15.83:0/1415577237 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6820109520 msgr2=0x7f68201a0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:58.961 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.960+0000 7f6826000640 1 --2- 172.21.15.83:0/1415577237 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6820109520 0x7f68201a0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:58.961 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.960+0000 7f6826000640 1 -- 172.21.15.83:0/1415577237 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f682019b370 con 0x7f682019a7e0 2024-04-24T08:59:58.962 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.960+0000 7f6826000640 1 --2- 172.21.15.83:0/1415577237 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f682019a7e0 0x7f682019ac90 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f681800eaa0 tx=0x7f681800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:58.962 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.960+0000 7f681e000640 1 -- 172.21.15.83:0/1415577237 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f681800cdb0 con 0x7f682019a7e0 2024-04-24T08:59:58.962 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.961+0000 7f681e000640 1 -- 172.21.15.83:0/1415577237 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6818005020 con 0x7f682019a7e0 2024-04-24T08:59:58.962 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.961+0000 7f6828000640 1 -- 172.21.15.83:0/1415577237 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f682019b660 con 0x7f682019a7e0 2024-04-24T08:59:58.962 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.961+0000 7f681e000640 1 -- 172.21.15.83:0/1415577237 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68180116d0 con 0x7f682019a7e0 2024-04-24T08:59:58.962 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.961+0000 7f6828000640 1 -- 172.21.15.83:0/1415577237 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68201a7770 con 0x7f682019a7e0 2024-04-24T08:59:58.964 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.964+0000 7f682ad91740 1 Processor -- start 2024-04-24T08:59:58.964 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.964+0000 7f681e000640 1 -- 172.21.15.83:0/1415577237 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f68180051d0 con 0x7f682019a7e0 2024-04-24T08:59:58.967 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.965+0000 7f682ad91740 1 -- start start 2024-04-24T08:59:58.967 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.965+0000 7f682ad91740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55db860fe1b0 0x55db860fe590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:58.967 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.965+0000 7f682ad91740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db860fead0 0x55db86108e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:58.967 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.965+0000 7f682ad91740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db86109390 0x55db8610b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:58.967 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.965+0000 7f682ad91740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55db86064900 con 0x55db860fead0 2024-04-24T08:59:58.967 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.965+0000 7f682ad91740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55db8600af20 con 0x55db86109390 2024-04-24T08:59:58.967 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.965+0000 7f682ad91740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55db860608c0 con 0x55db860fe1b0 2024-04-24T08:59:58.967 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.965+0000 7f6825600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db860fead0 0x55db86108e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:58.967 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.965+0000 7f6825600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db860fead0 0x55db86108e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48488/0 (socket says 172.21.15.83:48488) 2024-04-24T08:59:58.967 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.965+0000 7f6825600640 1 -- 172.21.15.83:0/3232114884 learned_addr learned my addr 172.21.15.83:0/3232114884 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:59:58.967 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.966+0000 7f6824c00640 1 --2- 172.21.15.83:0/3232114884 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55db860fe1b0 0x55db860fe590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:58.968 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.966+0000 7f6824c00640 1 --2- 172.21.15.83:0/3232114884 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db86109390 0x55db8610b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:58.968 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.966+0000 7f6825600640 1 -- 172.21.15.83:0/3232114884 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55db860fe1b0 msgr2=0x55db860fe590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:58.968 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.966+0000 7f6825600640 1 --2- 172.21.15.83:0/3232114884 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55db860fe1b0 0x55db860fe590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:58.968 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.966+0000 7f6825600640 1 -- 172.21.15.83:0/3232114884 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db86109390 msgr2=0x55db8610b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:58.968 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.966+0000 7f6825600640 1 --2- 172.21.15.83:0/3232114884 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db86109390 0x55db8610b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:58.968 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.966+0000 7f6825600640 1 -- 172.21.15.83:0/3232114884 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db8602fe10 con 0x55db860fead0 2024-04-24T08:59:58.968 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.966+0000 7f6824c00640 1 --2- 172.21.15.83:0/3232114884 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db86109390 0x55db8610b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:59:58.968 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.966+0000 7f6825600640 1 --2- 172.21.15.83:0/3232114884 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db860fead0 0x55db86108e50 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f681400bcf0 tx=0x7f681400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48cbc2ed4fa7742 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:58.968 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.967+0000 7f681e000640 1 --2- 172.21.15.83:0/1415577237 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f68080a3dc0 0x7f68080a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:58.968 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.967+0000 7f6812600640 1 -- 172.21.15.83:0/3232114884 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6814015070 con 0x55db860fead0 2024-04-24T08:59:58.969 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.967+0000 7f6812600640 1 -- 172.21.15.83:0/3232114884 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f681401a910 con 0x55db860fead0 2024-04-24T08:59:58.969 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.967+0000 7f6812600640 1 -- 172.21.15.83:0/3232114884 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6814011990 con 0x55db860fead0 2024-04-24T08:59:58.969 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.967+0000 7f681e000640 1 -- 172.21.15.83:0/1415577237 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7f68180d0eb0 con 0x7f682019a7e0 2024-04-24T08:59:58.969 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.967+0000 7f6824c00640 1 --2- 172.21.15.83:0/1415577237 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f68080a3dc0 0x7f68080a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:58.973 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.968+0000 7f682ad91740 1 -- 172.21.15.83:0/3232114884 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db860fead0 msgr2=0x55db86108e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:58.973 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.968+0000 7f682ad91740 1 --2- 172.21.15.83:0/3232114884 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db860fead0 0x55db86108e50 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f681400bcf0 tx=0x7f681400f730 comp rx=0 tx=0).stop 2024-04-24T08:59:58.973 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.968+0000 7f682ad91740 1 -- 172.21.15.83:0/3232114884 shutdown_connections 2024-04-24T08:59:58.973 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.968+0000 7f6824c00640 1 --2- 172.21.15.83:0/1415577237 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f68080a3dc0 0x7f68080a6280 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f680c000c00 tx=0x7f680c009260 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:58.973 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.968+0000 7f682ad91740 1 --2- 172.21.15.83:0/3232114884 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db86109390 0x55db8610b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:58.973 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.968+0000 7f682ad91740 1 --2- 172.21.15.83:0/3232114884 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db860fead0 0x55db86108e50 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:58.973 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.968+0000 7f682ad91740 1 --2- 172.21.15.83:0/3232114884 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55db860fe1b0 0x55db860fe590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:58.974 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.968+0000 7f682ad91740 1 -- 172.21.15.83:0/3232114884 >> 172.21.15.83:0/3232114884 conn(0x55db860fa6c0 msgr2=0x55db860faaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:58.974 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.968+0000 7f682ad91740 1 -- 172.21.15.83:0/3232114884 shutdown_connections 2024-04-24T08:59:58.974 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.968+0000 7f682ad91740 1 -- 172.21.15.83:0/3232114884 wait complete. 2024-04-24T08:59:58.974 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.968+0000 7f682ad91740 1 -- start start 2024-04-24T08:59:58.974 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.969+0000 7f682ad91740 1 Processor -- start 2024-04-24T08:59:58.974 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.969+0000 7f682ad91740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db860fe1b0 0x55db861062d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:58.974 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.969+0000 7f682ad91740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db860fead0 0x55db86106810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:58.974 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.969+0000 7f682ad91740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55db86109390 0x55db86106d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:58.974 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.969+0000 7f682ad91740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55db8610bfc0 con 0x55db860fead0 2024-04-24T08:59:58.974 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.969+0000 7f682ad91740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55db8601e2c0 con 0x55db860fe1b0 2024-04-24T08:59:58.974 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.969+0000 7f682ad91740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55db8610c140 con 0x55db86109390 2024-04-24T08:59:58.974 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.969+0000 7f682ad91740 10 client.0 _ll_register_callbacks cb 0x55db85d76a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:59:58.974 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.969+0000 7f6824c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db860fead0 0x55db86106810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:58.975 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.969+0000 7f6824c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db860fead0 0x55db86106810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48498/0 (socket says 172.21.15.83:48498) 2024-04-24T08:59:58.975 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.969+0000 7f6824c00640 1 -- 172.21.15.83:0/443449563 learned_addr learned my addr 172.21.15.83:0/443449563 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:59:58.975 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.970+0000 7f6826000640 1 --2- 172.21.15.83:0/443449563 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55db86109390 0x55db86106d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:58.975 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.970+0000 7f6825600640 1 --2- 172.21.15.83:0/443449563 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db860fe1b0 0x55db861062d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:58.975 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.970+0000 7f6824c00640 1 -- 172.21.15.83:0/443449563 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55db86109390 msgr2=0x55db86106d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:58.975 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.970+0000 7f6824c00640 1 --2- 172.21.15.83:0/443449563 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55db86109390 0x55db86106d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:58.975 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.970+0000 7f6824c00640 1 -- 172.21.15.83:0/443449563 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db860fe1b0 msgr2=0x55db861062d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:58.975 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.970+0000 7f6824c00640 1 --2- 172.21.15.83:0/443449563 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db860fe1b0 0x55db861062d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:58.975 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.970+0000 7f6824c00640 1 -- 172.21.15.83:0/443449563 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db8602efd0 con 0x55db860fead0 2024-04-24T08:59:58.975 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.970+0000 7f6825600640 1 --2- 172.21.15.83:0/443449563 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db860fe1b0 0x55db861062d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:59:58.975 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.970+0000 7f6824c00640 1 --2- 172.21.15.83:0/443449563 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db860fead0 0x55db86106810 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f680c033000 tx=0x7f680c031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=982c22549255918f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:58.975 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.970+0000 7f6806a00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:59:58.975 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.970+0000 7f6806a00640 1 -- 172.21.15.83:0/443449563 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f680c04e070 con 0x55db860fead0 2024-04-24T08:59:58.975 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.971+0000 7f6806a00640 1 -- 172.21.15.83:0/443449563 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7f680c005040 con 0x55db860fead0 2024-04-24T08:59:58.976 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.971+0000 7f682ad91740 1 -- 172.21.15.83:0/443449563 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f681401a910 con 0x55db860fead0 2024-04-24T08:59:58.976 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.971+0000 7f6806a00640 1 -- 172.21.15.83:0/443449563 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f680c05a050 con 0x55db860fead0 2024-04-24T08:59:58.976 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.971+0000 7f682ad91740 10 client.18606 fetch_fsmap learned FSMap version 35 2024-04-24T08:59:58.976 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.971+0000 7f682ad91740 1 -- 172.21.15.83:0/443449563 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55db85d77210 con 0x55db860fead0 2024-04-24T08:59:58.976 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.971+0000 7f6806a00640 1 -- 172.21.15.83:0/443449563 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f681401a910 con 0x55db860fead0 2024-04-24T08:59:58.976 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.971+0000 7f6806a00640 1 -- 172.21.15.83:0/443449563 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7f680c004cc0 con 0x55db860fead0 2024-04-24T08:59:58.976 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.972+0000 7f682ad91740 10 client.18606 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T08:59:58.976 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.972+0000 7f682ad91740 10 client.18606 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T08:59:58.976 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.972+0000 7f682ad91740 20 client.18606 populate_metadata read hostname 'smithi083' 2024-04-24T08:59:58.976 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.972+0000 7f682ad91740 1 --2- 172.21.15.83:0/443449563 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55db86119410 0x55db8611b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:58.976 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.972+0000 7f682ad91740 4 client.18606 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T08:59:58.977 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.972+0000 7f682ad91740 1 -- 172.21.15.83:0/443449563 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55db86072320 con 0x55db86119410 2024-04-24T08:59:58.977 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.972+0000 7f6825600640 1 --2- 172.21.15.83:0/443449563 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55db86119410 0x55db8611b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:58.978 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.973+0000 7f6806a00640 10 client.18606 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:59:58.978 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.973+0000 7f6825600640 1 --2- 172.21.15.83:0/443449563 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55db86119410 0x55db8611b870 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2c42c4d4f63df96 server_cookie=29f05fab2070d6bf in_seq=0 out_seq=0 2024-04-24T08:59:58.978 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.976+0000 7f6806a00640 1 -- 172.21.15.83:0/443449563 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55db86072320 con 0x55db86119410 2024-04-24T08:59:58.978 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.976+0000 7f6806a00640 10 client.18606 handle_command_reply: tid=0 2024-04-24T08:59:58.978 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.976+0000 7f682ad91740 1 client.18606 shutdown 2024-04-24T08:59:58.978 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.976+0000 7f682ad91740 1 -- 172.21.15.83:0/443449563 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db860fead0 msgr2=0x55db86106810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:58.978 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.976+0000 7f682ad91740 1 --2- 172.21.15.83:0/443449563 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db860fead0 0x55db86106810 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f680c033000 tx=0x7f680c031400 comp rx=0 tx=0).stop 2024-04-24T08:59:58.978 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.976+0000 7f682ad91740 1 -- 172.21.15.83:0/443449563 shutdown_connections 2024-04-24T08:59:58.978 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.976+0000 7f682ad91740 1 --2- 172.21.15.83:0/443449563 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55db86119410 0x55db8611b870 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:58.978 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.976+0000 7f682ad91740 1 --2- 172.21.15.83:0/443449563 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55db86109390 0x55db86106d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:58.979 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.976+0000 7f682ad91740 1 --2- 172.21.15.83:0/443449563 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db860fead0 0x55db86106810 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:58.979 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.976+0000 7f682ad91740 1 --2- 172.21.15.83:0/443449563 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db860fe1b0 0x55db861062d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:58.979 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.976+0000 7f6806a00640 0 client.18606 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:59:58.979 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.976+0000 7f682ad91740 1 -- 172.21.15.83:0/443449563 >> 172.21.15.83:0/443449563 conn(0x55db860fa6c0 msgr2=0x55db860f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:58.979 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.977+0000 7f682ad91740 1 -- 172.21.15.83:0/443449563 shutdown_connections 2024-04-24T08:59:58.979 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.977+0000 7f682ad91740 1 -- 172.21.15.83:0/443449563 wait complete. 2024-04-24T08:59:58.979 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.977+0000 7f682ad91740 20 client.18606 trim_cache size 0 max 16384 2024-04-24T08:59:58.995 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.995+0000 7f682ad91740 1 Processor -- start 2024-04-24T08:59:58.995 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.995+0000 7f682ad91740 1 -- start start 2024-04-24T08:59:58.995 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.995+0000 7f682ad91740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55db860fe1b0 0x55db86108a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:58.995 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.995+0000 7f682ad91740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db860fead0 0x55db86105430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:58.996 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.995+0000 7f682ad91740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db86109390 0x55db86100b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:58.996 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.995+0000 7f682ad91740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f68201b16b0 con 0x55db86109390 2024-04-24T08:59:58.996 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.995+0000 7f682ad91740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f68201b31f0 con 0x55db860fead0 2024-04-24T08:59:58.996 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.995+0000 7f682ad91740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f68201b1d80 con 0x55db860fe1b0 2024-04-24T08:59:58.997 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.996+0000 7f6826000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db86109390 0x55db86100b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:58.997 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.996+0000 7f6826000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db86109390 0x55db86100b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48510/0 (socket says 172.21.15.83:48510) 2024-04-24T08:59:58.997 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.996+0000 7f6826000640 1 -- 172.21.15.83:0/1612124746 learned_addr learned my addr 172.21.15.83:0/1612124746 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:59:58.997 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.996+0000 7f6824c00640 1 --2- 172.21.15.83:0/1612124746 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db860fead0 0x55db86105430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:58.997 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.996+0000 7f6825600640 1 --2- 172.21.15.83:0/1612124746 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55db860fe1b0 0x55db86108a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:58.997 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.996+0000 7f6826000640 1 -- 172.21.15.83:0/1612124746 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55db860fe1b0 msgr2=0x55db86108a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:58.997 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.996+0000 7f6826000640 1 --2- 172.21.15.83:0/1612124746 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55db860fe1b0 0x55db86108a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:58.997 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.996+0000 7f6826000640 1 -- 172.21.15.83:0/1612124746 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db860fead0 msgr2=0x55db86105430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:58.997 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.996+0000 7f6826000640 1 --2- 172.21.15.83:0/1612124746 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db860fead0 0x55db86105430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:58.997 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.996+0000 7f6826000640 1 -- 172.21.15.83:0/1612124746 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db85e43620 con 0x55db86109390 2024-04-24T08:59:58.998 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.997+0000 7f6826000640 1 --2- 172.21.15.83:0/1612124746 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db86109390 0x55db86100b70 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f68180092f0 tx=0x7f681809c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88b0a3bfe6dced41 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:58.998 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.997+0000 7f6811c00640 1 -- 172.21.15.83:0/1612124746 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f681801f040 con 0x55db86109390 2024-04-24T08:59:58.998 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.997+0000 7f6811c00640 1 -- 172.21.15.83:0/1612124746 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f68180a5640 con 0x55db86109390 2024-04-24T08:59:58.998 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.997+0000 7f6811c00640 1 -- 172.21.15.83:0/1612124746 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68180bbb90 con 0x55db86109390 2024-04-24T08:59:58.999 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.998+0000 7f682ad91740 1 -- 172.21.15.83:0/1612124746 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db86109390 msgr2=0x55db86100b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:58.999 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.998+0000 7f682ad91740 1 --2- 172.21.15.83:0/1612124746 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db86109390 0x55db86100b70 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f68180092f0 tx=0x7f681809c9d0 comp rx=0 tx=0).stop 2024-04-24T08:59:58.999 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.998+0000 7f682ad91740 1 -- 172.21.15.83:0/1612124746 shutdown_connections 2024-04-24T08:59:58.999 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.998+0000 7f682ad91740 1 --2- 172.21.15.83:0/1612124746 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db86109390 0x55db86100b70 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:58.999 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.998+0000 7f682ad91740 1 --2- 172.21.15.83:0/1612124746 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db860fead0 0x55db86105430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:58.999 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.998+0000 7f682ad91740 1 --2- 172.21.15.83:0/1612124746 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55db860fe1b0 0x55db86108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:58.999 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.998+0000 7f682ad91740 1 -- 172.21.15.83:0/1612124746 >> 172.21.15.83:0/1612124746 conn(0x55db860fa6c0 msgr2=0x55db860f8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:58.999 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.998+0000 7f682ad91740 1 -- 172.21.15.83:0/1612124746 shutdown_connections 2024-04-24T08:59:58.999 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.998+0000 7f682ad91740 1 -- 172.21.15.83:0/1612124746 wait complete. 2024-04-24T08:59:58.999 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.999+0000 7f682ad91740 1 -- start start 2024-04-24T08:59:58.999 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.999+0000 7f682ad91740 1 Processor -- start 2024-04-24T08:59:59.002 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:58.999+0000 7f682ad91740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db860fe1b0 0x55db861032a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:59.002 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:59.000+0000 7f682ad91740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55db860fead0 0x55db861037e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:59.002 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:59.000+0000 7f682ad91740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db86109390 0x55db86187200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:59.002 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:59.000+0000 7f682ad91740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f68201b2b20 con 0x55db86109390 2024-04-24T08:59:59.002 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:59.000+0000 7f682ad91740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f68201b2450 con 0x55db860fe1b0 2024-04-24T08:59:59.003 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:59.000+0000 7f682ad91740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f68201af6b0 con 0x55db860fead0 2024-04-24T08:59:59.003 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:59.000+0000 7f682ad91740 10 client.0 _ll_register_callbacks cb 0x7f680c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T08:59:59.003 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:59.000+0000 7f6826000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db86109390 0x55db86187200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:59.003 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:59.000+0000 7f6826000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db86109390 0x55db86187200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48520/0 (socket says 172.21.15.83:48520) 2024-04-24T08:59:59.003 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:59.000+0000 7f6826000640 1 -- 172.21.15.83:0/1210886327 learned_addr learned my addr 172.21.15.83:0/1210886327 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T08:59:59.003 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:59.000+0000 7f6825600640 1 --2- 172.21.15.83:0/1210886327 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db860fe1b0 0x55db861032a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:59.003 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:59.000+0000 7f6824c00640 1 --2- 172.21.15.83:0/1210886327 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55db860fead0 0x55db861037e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:59.003 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:59.000+0000 7f6826000640 1 -- 172.21.15.83:0/1210886327 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55db860fead0 msgr2=0x55db861037e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:59.003 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:59.000+0000 7f6826000640 1 --2- 172.21.15.83:0/1210886327 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55db860fead0 0x55db861037e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:59.003 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:59.000+0000 7f6826000640 1 -- 172.21.15.83:0/1210886327 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db860fe1b0 msgr2=0x55db861032a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T08:59:59.003 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:59.000+0000 7f6826000640 1 --2- 172.21.15.83:0/1210886327 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db860fe1b0 0x55db861032a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:59.003 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:59.000+0000 7f6826000640 1 -- 172.21.15.83:0/1210886327 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db86104070 con 0x55db86109390 2024-04-24T08:59:59.003 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:59.000+0000 7f6825600640 1 --2- 172.21.15.83:0/1210886327 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db860fe1b0 0x55db861032a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:59:59.003 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:59.000+0000 7f6824c00640 1 --2- 172.21.15.83:0/1210886327 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55db860fead0 0x55db861037e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T08:59:59.004 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:59.001+0000 7f6826000640 1 --2- 172.21.15.83:0/1210886327 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db86109390 0x55db86187200 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f6818098e30 tx=0x7f6818005560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cb1202ceeacc047 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T08:59:59.004 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:59.001+0000 7f6806a00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T08:59:59.004 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:59.001+0000 7f6806a00640 1 -- 172.21.15.83:0/1210886327 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68180bbd90 con 0x55db86109390 2024-04-24T08:59:59.004 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:59.001+0000 7f682ad91740 1 -- 172.21.15.83:0/1210886327 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68180a5640 con 0x55db86109390 2024-04-24T08:59:59.004 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:59.001+0000 7f6806a00640 1 -- 172.21.15.83:0/1210886327 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9583+0+0 (secure 0 0 0) 0x7f68180870b0 con 0x55db86109390 2024-04-24T08:59:59.004 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:59.001+0000 7f6806a00640 1 -- 172.21.15.83:0/1210886327 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f68180872b0 con 0x55db86109390 2024-04-24T08:59:59.004 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:59.001+0000 7f682ad91740 10 client.18612 fetch_fsmap learned FSMap version 35 2024-04-24T08:59:59.004 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:59.001+0000 7f682ad91740 1 -- 172.21.15.83:0/1210886327 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55db86188820 con 0x55db86109390 2024-04-24T08:59:59.004 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:59.001+0000 7f6806a00640 1 -- 172.21.15.83:0/1210886327 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f68180a5640 con 0x55db86109390 2024-04-24T08:59:59.004 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:59.002+0000 7f6806a00640 1 -- 172.21.15.83:0/1210886327 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7f68180ba020 con 0x55db86109390 2024-04-24T08:59:59.004 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:59.002+0000 7f682ad91740 10 client.18612 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T08:59:59.004 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:59.002+0000 7f682ad91740 10 client.18612 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T08:59:59.004 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:59.002+0000 7f682ad91740 20 client.18612 populate_metadata read hostname 'smithi083' 2024-04-24T08:59:59.004 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:59.002+0000 7f682ad91740 1 --2- 172.21.15.83:0/1210886327 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55db86188e40 0x55db8618b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T08:59:59.005 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:59.002+0000 7f682ad91740 4 client.18612 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T08:59:59.005 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:59.002+0000 7f682ad91740 1 -- 172.21.15.83:0/1210886327 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55db8606c190 con 0x55db86188e40 2024-04-24T08:59:59.005 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:59.002+0000 7f6825600640 1 --2- 172.21.15.83:0/1210886327 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55db86188e40 0x55db8618b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T08:59:59.005 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:59.003+0000 7f6825600640 1 --2- 172.21.15.83:0/1210886327 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55db86188e40 0x55db8618b230 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f44c123a8b550122 server_cookie=d0ad1ed168d38399 in_seq=0 out_seq=0 2024-04-24T08:59:59.005 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:59.003+0000 7f6806a00640 10 client.18612 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T08:59:59.060 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:59.060+0000 7f6806a00640 1 -- 172.21.15.83:0/1210886327 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55db8606c190 con 0x55db86188e40 2024-04-24T08:59:59.061 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:59.060+0000 7f6806a00640 10 client.18612 handle_command_reply: tid=0 2024-04-24T08:59:59.061 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:59.060+0000 7f682ad91740 1 client.18612 shutdown 2024-04-24T08:59:59.061 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:59.061+0000 7f682ad91740 1 -- 172.21.15.83:0/1210886327 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db86109390 msgr2=0x55db86187200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:59.061 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:59.061+0000 7f682ad91740 1 --2- 172.21.15.83:0/1210886327 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db86109390 0x55db86187200 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f6818098e30 tx=0x7f6818005560 comp rx=0 tx=0).stop 2024-04-24T08:59:59.062 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:59.061+0000 7f682ad91740 1 -- 172.21.15.83:0/1210886327 shutdown_connections 2024-04-24T08:59:59.062 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:59.061+0000 7f682ad91740 1 --2- 172.21.15.83:0/1210886327 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55db86188e40 0x55db8618b230 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:59.062 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:59.061+0000 7f682ad91740 1 --2- 172.21.15.83:0/1210886327 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db86109390 0x55db86187200 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:59.062 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:59.061+0000 7f682ad91740 1 --2- 172.21.15.83:0/1210886327 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55db860fead0 0x55db861037e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:59.062 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:59.061+0000 7f682ad91740 1 --2- 172.21.15.83:0/1210886327 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db860fe1b0 0x55db861032a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:59.062 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:59.061+0000 7f682ad91740 1 -- 172.21.15.83:0/1210886327 >> 172.21.15.83:0/1210886327 conn(0x55db860fa6c0 msgr2=0x55db860f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:59.062 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:59.061+0000 7f6806a00640 0 client.18612 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T08:59:59.062 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:59.061+0000 7f682ad91740 1 -- 172.21.15.83:0/1210886327 shutdown_connections 2024-04-24T08:59:59.062 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:59.061+0000 7f682ad91740 1 -- 172.21.15.83:0/1210886327 wait complete. 2024-04-24T08:59:59.063 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:59.062+0000 7f682ad91740 20 client.18612 trim_cache size 0 max 16384 2024-04-24T08:59:59.063 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T08:59:59.063 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T08:59:59.063 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T08:59:59.063 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T08:59:59.064 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:59.063+0000 7f6828000640 1 -- 172.21.15.83:0/1415577237 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f68080a3dc0 msgr2=0x7f68080a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:59.064 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:59.063+0000 7f6828000640 1 --2- 172.21.15.83:0/1415577237 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f68080a3dc0 0x7f68080a6280 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f680c000c00 tx=0x7f680c009260 comp rx=0 tx=0).stop 2024-04-24T08:59:59.064 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:59.063+0000 7f6828000640 1 -- 172.21.15.83:0/1415577237 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f682019a7e0 msgr2=0x7f682019ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T08:59:59.064 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:59.063+0000 7f6828000640 1 --2- 172.21.15.83:0/1415577237 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f682019a7e0 0x7f682019ac90 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f681800eaa0 tx=0x7f681800ef60 comp rx=0 tx=0).stop 2024-04-24T08:59:59.064 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:59.063+0000 7f6828000640 1 -- 172.21.15.83:0/1415577237 shutdown_connections 2024-04-24T08:59:59.064 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:59.063+0000 7f6828000640 1 --2- 172.21.15.83:0/1415577237 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f68080a3dc0 0x7f68080a6280 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:59.064 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:59.063+0000 7f6828000640 1 --2- 172.21.15.83:0/1415577237 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f682019a7e0 0x7f682019ac90 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:59.064 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:59.063+0000 7f6828000640 1 --2- 172.21.15.83:0/1415577237 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6820109520 0x7f68201a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:59.065 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:59.063+0000 7f6828000640 1 --2- 172.21.15.83:0/1415577237 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6820104450 0x7f68201a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T08:59:59.065 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:59.063+0000 7f6828000640 1 -- 172.21.15.83:0/1415577237 >> 172.21.15.83:0/1415577237 conn(0x7f68200ffcf0 msgr2=0x7f6820102040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T08:59:59.065 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:59.063+0000 7f6828000640 1 -- 172.21.15.83:0/1415577237 shutdown_connections 2024-04-24T08:59:59.065 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T08:59:59.064+0000 7f6828000640 1 -- 172.21.15.83:0/1415577237 wait complete. 2024-04-24T08:59:59.597 INFO:tasks.workunit.client.0.smithi083.stdout: 1 46700 0.63 MB/sec execute 273 sec latency 56.007 ms 2024-04-24T09:00:00.052 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:59:59 smithi083 ceph-mon[27907]: pgmap v260: 129 pgs: 129 active+clean; 50 MiB data, 711 MiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 706 KiB/s wr, 65 op/s 2024-04-24T09:00:00.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 08:59:59 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:00:00.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:59:59 smithi189 ceph-mon[31791]: pgmap v260: 129 pgs: 129 active+clean; 50 MiB data, 711 MiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 706 KiB/s wr, 65 op/s 2024-04-24T09:00:00.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 08:59:59 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:00:00.119 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:59:59 smithi138 ceph-mon[31441]: pgmap v260: 129 pgs: 129 active+clean; 50 MiB data, 711 MiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 706 KiB/s wr, 65 op/s 2024-04-24T09:00:00.119 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 08:59:59 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:00:00.597 INFO:tasks.workunit.client.0.smithi083.stdout: 1 46813 0.63 MB/sec execute 274 sec latency 42.408 ms 2024-04-24T09:00:01.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:00:00 smithi083 ceph-mon[27907]: overall HEALTH_OK 2024-04-24T09:00:01.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:00:00 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:00:01.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:00:00 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:00:01.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:00:00 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:00:01.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:00:00 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:00:01.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:00:00 smithi138 ceph-mon[31441]: overall HEALTH_OK 2024-04-24T09:00:01.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:00:00 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:00:01.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:00:00 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:00:01.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:00:00 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:00:01.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:00:00 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:00:01.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:00:00 smithi189 ceph-mon[31791]: overall HEALTH_OK 2024-04-24T09:00:01.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:00:00 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:00:01.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:00:00 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:00:01.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:00:00 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:00:01.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:00:00 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:00:01.597 INFO:tasks.workunit.client.0.smithi083.stdout: 1 46948 0.63 MB/sec execute 275 sec latency 50.391 ms 2024-04-24T09:00:02.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:00:02 smithi083 ceph-mon[27907]: pgmap v261: 129 pgs: 129 active+clean; 50 MiB data, 711 MiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 361 KiB/s wr, 40 op/s 2024-04-24T09:00:02.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:00:02 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:00:02.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:00:02 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:00:02.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:00:02 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:00:02.597 INFO:tasks.workunit.client.0.smithi083.stdout: 1 47071 0.63 MB/sec execute 276 sec latency 53.142 ms 2024-04-24T09:00:02.704 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:00:02 smithi138 ceph-mon[31441]: pgmap v261: 129 pgs: 129 active+clean; 50 MiB data, 711 MiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 361 KiB/s wr, 40 op/s 2024-04-24T09:00:02.704 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:00:02 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:00:02.704 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:00:02 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:00:02.704 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:00:02 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:00:02.706 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:00:02 smithi189 ceph-mon[31791]: pgmap v261: 129 pgs: 129 active+clean; 50 MiB data, 711 MiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 361 KiB/s wr, 40 op/s 2024-04-24T09:00:02.706 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:00:02 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:00:02.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:00:02 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:00:02.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:00:02 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:00:03.598 INFO:tasks.workunit.client.0.smithi083.stdout: 1 47202 0.63 MB/sec execute 277 sec latency 56.811 ms 2024-04-24T09:00:03.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:00:03 smithi189 ceph-mon[31791]: osdmap e101: 12 total, 12 up, 12 in 2024-04-24T09:00:03.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:00:03 smithi189 ceph-mon[31791]: pgmap v263: 129 pgs: 129 active+clean; 49 MiB data, 720 MiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 471 KiB/s wr, 88 op/s 2024-04-24T09:00:03.954 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:00:03 smithi138 ceph-mon[31441]: osdmap e101: 12 total, 12 up, 12 in 2024-04-24T09:00:03.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:00:03 smithi138 ceph-mon[31441]: pgmap v263: 129 pgs: 129 active+clean; 49 MiB data, 720 MiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 471 KiB/s wr, 88 op/s 2024-04-24T09:00:04.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:00:03 smithi083 ceph-mon[27907]: osdmap e101: 12 total, 12 up, 12 in 2024-04-24T09:00:04.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:00:03 smithi083 ceph-mon[27907]: pgmap v263: 129 pgs: 129 active+clean; 49 MiB data, 720 MiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 471 KiB/s wr, 88 op/s 2024-04-24T09:00:04.597 INFO:tasks.workunit.client.0.smithi083.stdout: 1 47312 0.63 MB/sec execute 278 sec latency 50.367 ms 2024-04-24T09:00:05.597 INFO:tasks.workunit.client.0.smithi083.stdout: 1 47408 0.62 MB/sec execute 279 sec latency 50.194 ms 2024-04-24T09:00:05.954 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:00:05 smithi138 ceph-mon[31441]: pgmap v264: 129 pgs: 129 active+clean; 49 MiB data, 720 MiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 471 KiB/s wr, 88 op/s 2024-04-24T09:00:05.956 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:00:05 smithi189 ceph-mon[31791]: pgmap v264: 129 pgs: 129 active+clean; 49 MiB data, 720 MiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 471 KiB/s wr, 88 op/s 2024-04-24T09:00:06.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:00:05 smithi083 ceph-mon[27907]: pgmap v264: 129 pgs: 129 active+clean; 49 MiB data, 720 MiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 471 KiB/s wr, 88 op/s 2024-04-24T09:00:06.598 INFO:tasks.workunit.client.0.smithi083.stdout: 1 47572 0.63 MB/sec execute 280 sec latency 54.810 ms 2024-04-24T09:00:07.598 INFO:tasks.workunit.client.0.smithi083.stdout: 1 47666 0.63 MB/sec execute 281 sec latency 67.269 ms 2024-04-24T09:00:07.954 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:00:07 smithi138 ceph-mon[31441]: pgmap v265: 129 pgs: 129 active+clean; 49 MiB data, 720 MiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 471 KiB/s wr, 88 op/s 2024-04-24T09:00:07.956 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:00:07 smithi189 ceph-mon[31791]: pgmap v265: 129 pgs: 129 active+clean; 49 MiB data, 720 MiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 471 KiB/s wr, 88 op/s 2024-04-24T09:00:08.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:00:07 smithi083 ceph-mon[27907]: pgmap v265: 129 pgs: 129 active+clean; 49 MiB data, 720 MiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 471 KiB/s wr, 88 op/s 2024-04-24T09:00:08.598 INFO:tasks.workunit.client.0.smithi083.stdout: 1 47767 0.63 MB/sec execute 282 sec latency 64.676 ms 2024-04-24T09:00:09.102 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T09:00:09.103 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T09:00:09.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.316+0000 7eff2e200640 1 -- 172.21.15.83:0/1539451519 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff28108580 msgr2=0x7eff28108a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:09.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.316+0000 7eff2e200640 1 --2- 172.21.15.83:0/1539451519 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff28108580 0x7eff28108a00 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7eff14009930 tx=0x7eff1402a420 comp rx=0 tx=0).stop 2024-04-24T09:00:09.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.317+0000 7eff2e200640 1 -- 172.21.15.83:0/1539451519 shutdown_connections 2024-04-24T09:00:09.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.317+0000 7eff2e200640 1 --2- 172.21.15.83:0/1539451519 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff28108f40 0x7eff281153b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:09.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.317+0000 7eff2e200640 1 --2- 172.21.15.83:0/1539451519 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff28108580 0x7eff28108a00 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:09.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.317+0000 7eff2e200640 1 --2- 172.21.15.83:0/1539451519 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7eff2810d500 0x7eff2810d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:09.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.317+0000 7eff2e200640 1 -- 172.21.15.83:0/1539451519 >> 172.21.15.83:0/1539451519 conn(0x7eff28077e20 msgr2=0x7eff28106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:09.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.317+0000 7eff2e200640 1 -- 172.21.15.83:0/1539451519 shutdown_connections 2024-04-24T09:00:09.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.317+0000 7eff2e200640 1 -- 172.21.15.83:0/1539451519 wait complete. 2024-04-24T09:00:09.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.318+0000 7eff2e200640 1 Processor -- start 2024-04-24T09:00:09.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.318+0000 7eff2e200640 1 -- start start 2024-04-24T09:00:09.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.318+0000 7eff2e200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7eff28108580 0x7eff281a4910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:09.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.318+0000 7eff2e200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff28108f40 0x7eff281a4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:09.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.318+0000 7eff2e200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff2810d500 0x7eff2819eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:09.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.318+0000 7eff2e200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7eff281187d0 con 0x7eff2810d500 2024-04-24T09:00:09.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.318+0000 7eff2e200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7eff28118650 con 0x7eff28108f40 2024-04-24T09:00:09.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.318+0000 7eff2e200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7eff28118950 con 0x7eff28108580 2024-04-24T09:00:09.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.318+0000 7eff2d800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff2810d500 0x7eff2819eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:09.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.319+0000 7eff2d800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff2810d500 0x7eff2819eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48542/0 (socket says 172.21.15.83:48542) 2024-04-24T09:00:09.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.319+0000 7eff2d800640 1 -- 172.21.15.83:0/1467909195 learned_addr learned my addr 172.21.15.83:0/1467909195 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:00:09.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.319+0000 7eff2ce00640 1 --2- 172.21.15.83:0/1467909195 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7eff28108580 0x7eff281a4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:09.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.319+0000 7eff27e00640 1 --2- 172.21.15.83:0/1467909195 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff28108f40 0x7eff281a4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:09.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.319+0000 7eff2d800640 1 -- 172.21.15.83:0/1467909195 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7eff28108580 msgr2=0x7eff281a4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:09.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.319+0000 7eff2d800640 1 --2- 172.21.15.83:0/1467909195 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7eff28108580 0x7eff281a4910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:09.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.319+0000 7eff2d800640 1 -- 172.21.15.83:0/1467909195 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff28108f40 msgr2=0x7eff281a4e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:09.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.319+0000 7eff2d800640 1 --2- 172.21.15.83:0/1467909195 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff28108f40 0x7eff281a4e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:09.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.319+0000 7eff2d800640 1 -- 172.21.15.83:0/1467909195 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff2819f290 con 0x7eff2810d500 2024-04-24T09:00:09.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.319+0000 7eff2d800640 1 --2- 172.21.15.83:0/1467909195 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff2810d500 0x7eff2819eb10 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7eff2000cab0 tx=0x7eff2000cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:09.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.320+0000 7eff25600640 1 -- 172.21.15.83:0/1467909195 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff2000adc0 con 0x7eff2810d500 2024-04-24T09:00:09.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.320+0000 7eff25600640 1 -- 172.21.15.83:0/1467909195 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7eff2000f900 con 0x7eff2810d500 2024-04-24T09:00:09.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.320+0000 7eff25600640 1 -- 172.21.15.83:0/1467909195 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff20018950 con 0x7eff2810d500 2024-04-24T09:00:09.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.320+0000 7eff2e200640 1 -- 172.21.15.83:0/1467909195 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff2819f580 con 0x7eff2810d500 2024-04-24T09:00:09.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.320+0000 7eff2e200640 1 -- 172.21.15.83:0/1467909195 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff281ab7d0 con 0x7eff2810d500 2024-04-24T09:00:09.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.323+0000 7eff32637740 1 Processor -- start 2024-04-24T09:00:09.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.323+0000 7eff25600640 1 -- 172.21.15.83:0/1467909195 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7eff2000fab0 con 0x7eff2810d500 2024-04-24T09:00:09.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.323+0000 7eff32637740 1 -- start start 2024-04-24T09:00:09.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.324+0000 7eff32637740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d859bda30 0x558d859bde10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:09.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.324+0000 7eff32637740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d859be350 0x558d859c84e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:09.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.324+0000 7eff32637740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558d859c8a20 0x558d859cae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:09.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.324+0000 7eff32637740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x558d859231f0 con 0x558d859be350 2024-04-24T09:00:09.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.324+0000 7eff32637740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558d858c9350 con 0x558d859bda30 2024-04-24T09:00:09.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.324+0000 7eff32637740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x558d8591f1b0 con 0x558d859c8a20 2024-04-24T09:00:09.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.324+0000 7eff2ce00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d859be350 0x558d859c84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:09.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.324+0000 7eff2ce00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d859be350 0x558d859c84e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48550/0 (socket says 172.21.15.83:48550) 2024-04-24T09:00:09.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.324+0000 7eff2ce00640 1 -- 172.21.15.83:0/1254374672 learned_addr learned my addr 172.21.15.83:0/1254374672 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:00:09.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.324+0000 7eff27e00640 1 --2- 172.21.15.83:0/1254374672 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d859bda30 0x558d859bde10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:09.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.325+0000 7eff27e00640 1 --2- 172.21.15.83:0/1254374672 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558d859c8a20 0x558d859cae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:09.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.325+0000 7eff2ce00640 1 -- 172.21.15.83:0/1254374672 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558d859c8a20 msgr2=0x558d859cae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:09.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.325+0000 7eff2ce00640 1 --2- 172.21.15.83:0/1254374672 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558d859c8a20 0x558d859cae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:09.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.325+0000 7eff2ce00640 1 -- 172.21.15.83:0/1254374672 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d859bda30 msgr2=0x558d859bde10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:09.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.325+0000 7eff2ce00640 1 --2- 172.21.15.83:0/1254374672 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d859bda30 0x558d859bde10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:09.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.325+0000 7eff2ce00640 1 -- 172.21.15.83:0/1254374672 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d8591df50 con 0x558d859be350 2024-04-24T09:00:09.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.325+0000 7eff27e00640 1 --2- 172.21.15.83:0/1254374672 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d859bda30 0x558d859bde10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:00:09.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.325+0000 7eff25600640 1 --2- 172.21.15.83:0/1467909195 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7eff100a3d20 0x7eff100a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:09.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.325+0000 7eff2ce00640 1 --2- 172.21.15.83:0/1254374672 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d859be350 0x558d859c84e0 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7eff1c00bc60 tx=0x7eff1c00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33bbddda16d56761 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:09.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.325+0000 7eff25600640 1 -- 172.21.15.83:0/1467909195 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9671+0+0 (secure 0 0 0) 0x7eff200d1090 con 0x7eff2810d500 2024-04-24T09:00:09.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.325+0000 7eff27e00640 1 --2- 172.21.15.83:0/1467909195 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7eff100a3d20 0x7eff100a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:09.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.325+0000 7eff1a000640 1 -- 172.21.15.83:0/1254374672 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff1c015070 con 0x558d859be350 2024-04-24T09:00:09.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.326+0000 7eff1a000640 1 -- 172.21.15.83:0/1254374672 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7eff1c0118f0 con 0x558d859be350 2024-04-24T09:00:09.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.326+0000 7eff1a000640 1 -- 172.21.15.83:0/1254374672 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff1c010760 con 0x558d859be350 2024-04-24T09:00:09.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.327+0000 7eff32637740 1 -- 172.21.15.83:0/1254374672 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d859be350 msgr2=0x558d859c84e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:09.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.327+0000 7eff27e00640 1 --2- 172.21.15.83:0/1467909195 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7eff100a3d20 0x7eff100a61e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7eff1402a940 tx=0x7eff14031400 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:09.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.327+0000 7eff32637740 1 --2- 172.21.15.83:0/1254374672 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d859be350 0x558d859c84e0 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7eff1c00bc60 tx=0x7eff1c00f730 comp rx=0 tx=0).stop 2024-04-24T09:00:09.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.327+0000 7eff32637740 1 -- 172.21.15.83:0/1254374672 shutdown_connections 2024-04-24T09:00:09.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.327+0000 7eff32637740 1 --2- 172.21.15.83:0/1254374672 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558d859c8a20 0x558d859cae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:09.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.327+0000 7eff32637740 1 --2- 172.21.15.83:0/1254374672 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d859be350 0x558d859c84e0 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:09.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.327+0000 7eff32637740 1 --2- 172.21.15.83:0/1254374672 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d859bda30 0x558d859bde10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:09.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.327+0000 7eff32637740 1 -- 172.21.15.83:0/1254374672 >> 172.21.15.83:0/1254374672 conn(0x558d859b9d50 msgr2=0x558d859ba130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:09.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.327+0000 7eff32637740 1 -- 172.21.15.83:0/1254374672 shutdown_connections 2024-04-24T09:00:09.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.327+0000 7eff32637740 1 -- 172.21.15.83:0/1254374672 wait complete. 2024-04-24T09:00:09.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.328+0000 7eff32637740 1 -- start start 2024-04-24T09:00:09.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.328+0000 7eff32637740 1 Processor -- start 2024-04-24T09:00:09.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.328+0000 7eff32637740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d859bda30 0x558d859c5960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:09.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.328+0000 7eff32637740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558d859be350 0x558d859c5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:09.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.328+0000 7eff32637740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d859c8a20 0x558d859c63e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:09.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.328+0000 7eff32637740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x558d859cb810 con 0x558d859c8a20 2024-04-24T09:00:09.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.328+0000 7eff32637740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558d858dc690 con 0x558d859bda30 2024-04-24T09:00:09.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.328+0000 7eff32637740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x558d859cb990 con 0x558d859be350 2024-04-24T09:00:09.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.329+0000 7eff32637740 10 client.0 _ll_register_callbacks cb 0x558d85918650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:00:09.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.329+0000 7eff2d800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d859c8a20 0x558d859c63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:09.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.329+0000 7eff2d800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d859c8a20 0x558d859c63e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48564/0 (socket says 172.21.15.83:48564) 2024-04-24T09:00:09.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.329+0000 7eff2d800640 1 -- 172.21.15.83:0/2906755541 learned_addr learned my addr 172.21.15.83:0/2906755541 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:00:09.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.329+0000 7eff2ce00640 1 --2- 172.21.15.83:0/2906755541 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d859bda30 0x558d859c5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:09.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.329+0000 7eff27e00640 1 --2- 172.21.15.83:0/2906755541 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558d859be350 0x558d859c5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:09.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.329+0000 7eff2d800640 1 -- 172.21.15.83:0/2906755541 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558d859be350 msgr2=0x558d859c5ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:09.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.329+0000 7eff2d800640 1 --2- 172.21.15.83:0/2906755541 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558d859be350 0x558d859c5ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:09.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.329+0000 7eff2d800640 1 -- 172.21.15.83:0/2906755541 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d859bda30 msgr2=0x558d859c5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:09.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.329+0000 7eff2d800640 1 --2- 172.21.15.83:0/2906755541 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d859bda30 0x558d859c5960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:09.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.329+0000 7eff2d800640 1 -- 172.21.15.83:0/2906755541 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d858db920 con 0x558d859c8a20 2024-04-24T09:00:09.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.330+0000 7eff2d800640 1 --2- 172.21.15.83:0/2906755541 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d859c8a20 0x558d859c63e0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7eff2009cae0 tx=0x7eff2000fee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9f11c36c1256ae2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:09.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.330+0000 7eff0ea00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:00:09.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.330+0000 7eff0ea00640 1 -- 172.21.15.83:0/2906755541 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff200b6060 con 0x558d859c8a20 2024-04-24T09:00:09.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.330+0000 7eff0ea00640 1 -- 172.21.15.83:0/2906755541 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9671+0+0 (secure 0 0 0) 0x7eff200a5330 con 0x558d859c8a20 2024-04-24T09:00:09.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.330+0000 7eff32637740 1 -- 172.21.15.83:0/2906755541 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff1c0118f0 con 0x558d859c8a20 2024-04-24T09:00:09.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.330+0000 7eff0ea00640 1 -- 172.21.15.83:0/2906755541 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7eff200a0f00 con 0x558d859c8a20 2024-04-24T09:00:09.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.330+0000 7eff32637740 10 client.18624 fetch_fsmap learned FSMap version 35 2024-04-24T09:00:09.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.330+0000 7eff32637740 1 -- 172.21.15.83:0/2906755541 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x558d856ff7d0 con 0x558d859c8a20 2024-04-24T09:00:09.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.331+0000 7eff0ea00640 1 -- 172.21.15.83:0/2906755541 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7eff1c0118f0 con 0x558d859c8a20 2024-04-24T09:00:09.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.331+0000 7eff0ea00640 1 -- 172.21.15.83:0/2906755541 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7eff20004b40 con 0x558d859c8a20 2024-04-24T09:00:09.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.331+0000 7eff32637740 10 client.18624 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T09:00:09.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.331+0000 7eff32637740 10 client.18624 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:00:09.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.331+0000 7eff32637740 20 client.18624 populate_metadata read hostname 'smithi083' 2024-04-24T09:00:09.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.331+0000 7eff32637740 1 --2- 172.21.15.83:0/2906755541 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x558d859d8de0 0x558d859db1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:09.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.331+0000 7eff32637740 4 client.18624 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:00:09.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.331+0000 7eff32637740 1 -- 172.21.15.83:0/2906755541 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558d85930c10 con 0x558d859d8de0 2024-04-24T09:00:09.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.332+0000 7eff2ce00640 1 --2- 172.21.15.83:0/2906755541 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x558d859d8de0 0x558d859db1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:09.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.333+0000 7eff2ce00640 1 --2- 172.21.15.83:0/2906755541 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x558d859d8de0 0x558d859db1f0 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1f2557371ca9833 server_cookie=8930843c7bb38741 in_seq=0 out_seq=0 2024-04-24T09:00:09.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.333+0000 7eff0ea00640 10 client.18624 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:00:09.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.336+0000 7eff0ea00640 1 -- 172.21.15.83:0/2906755541 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558d85930c10 con 0x558d859d8de0 2024-04-24T09:00:09.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.336+0000 7eff0ea00640 10 client.18624 handle_command_reply: tid=0 2024-04-24T09:00:09.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.336+0000 7eff32637740 1 client.18624 shutdown 2024-04-24T09:00:09.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.336+0000 7eff32637740 1 -- 172.21.15.83:0/2906755541 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d859c8a20 msgr2=0x558d859c63e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:09.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.336+0000 7eff32637740 1 --2- 172.21.15.83:0/2906755541 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d859c8a20 0x558d859c63e0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7eff2009cae0 tx=0x7eff2000fee0 comp rx=0 tx=0).stop 2024-04-24T09:00:09.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.336+0000 7eff32637740 1 -- 172.21.15.83:0/2906755541 shutdown_connections 2024-04-24T09:00:09.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.336+0000 7eff32637740 1 --2- 172.21.15.83:0/2906755541 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x558d859d8de0 0x558d859db1f0 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:09.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.336+0000 7eff32637740 1 --2- 172.21.15.83:0/2906755541 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d859c8a20 0x558d859c63e0 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:09.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.336+0000 7eff32637740 1 --2- 172.21.15.83:0/2906755541 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558d859be350 0x558d859c5ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:09.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.336+0000 7eff32637740 1 --2- 172.21.15.83:0/2906755541 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d859bda30 0x558d859c5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:09.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.336+0000 7eff0ea00640 0 client.18624 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:00:09.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.336+0000 7eff32637740 1 -- 172.21.15.83:0/2906755541 >> 172.21.15.83:0/2906755541 conn(0x558d859b9d50 msgr2=0x558d859b8fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:09.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.336+0000 7eff32637740 1 -- 172.21.15.83:0/2906755541 shutdown_connections 2024-04-24T09:00:09.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.336+0000 7eff32637740 1 -- 172.21.15.83:0/2906755541 wait complete. 2024-04-24T09:00:09.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.336+0000 7eff32637740 20 client.18624 trim_cache size 0 max 16384 2024-04-24T09:00:09.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.355+0000 7eff32637740 1 Processor -- start 2024-04-24T09:00:09.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.355+0000 7eff32637740 1 -- start start 2024-04-24T09:00:09.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.356+0000 7eff32637740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d859bda30 0x558d859d4700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:09.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.356+0000 7eff32637740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d859be350 0x558d859ba130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:09.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.356+0000 7eff32637740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558d859c8a20 0x558d859c6030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:09.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.356+0000 7eff32637740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7eff281b5aa0 con 0x558d859bda30 2024-04-24T09:00:09.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.356+0000 7eff32637740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7eff281b75e0 con 0x558d859be350 2024-04-24T09:00:09.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.356+0000 7eff32637740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7eff281b6170 con 0x558d859c8a20 2024-04-24T09:00:09.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.356+0000 7eff2ce00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d859bda30 0x558d859d4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:09.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.356+0000 7eff2ce00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d859bda30 0x558d859d4700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48576/0 (socket says 172.21.15.83:48576) 2024-04-24T09:00:09.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.356+0000 7eff2ce00640 1 -- 172.21.15.83:0/3791217644 learned_addr learned my addr 172.21.15.83:0/3791217644 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:00:09.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.356+0000 7eff2d800640 1 --2- 172.21.15.83:0/3791217644 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558d859c8a20 0x558d859c6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:09.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.356+0000 7eff2ce00640 1 -- 172.21.15.83:0/3791217644 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558d859c8a20 msgr2=0x558d859c6030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:09.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.356+0000 7eff2ce00640 1 --2- 172.21.15.83:0/3791217644 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558d859c8a20 0x558d859c6030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:09.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.356+0000 7eff2ce00640 1 -- 172.21.15.83:0/3791217644 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d859be350 msgr2=0x558d859ba130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:09.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.356+0000 7eff27e00640 1 --2- 172.21.15.83:0/3791217644 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d859be350 0x558d859ba130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:09.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.356+0000 7eff2ce00640 1 --2- 172.21.15.83:0/3791217644 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d859be350 0x558d859ba130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:09.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.356+0000 7eff2ce00640 1 -- 172.21.15.83:0/3791217644 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d856353f0 con 0x558d859bda30 2024-04-24T09:00:09.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.356+0000 7eff2d800640 1 --2- 172.21.15.83:0/3791217644 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558d859c8a20 0x558d859c6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:00:09.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.357+0000 7eff2ce00640 1 --2- 172.21.15.83:0/3791217644 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d859bda30 0x558d859d4700 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7eff1c00fc30 tx=0x7eff1c00b450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53ebc81ee99112a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:09.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.357+0000 7eff19600640 1 -- 172.21.15.83:0/3791217644 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff1c02f070 con 0x558d859bda30 2024-04-24T09:00:09.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.357+0000 7eff19600640 1 -- 172.21.15.83:0/3791217644 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7eff1c00be70 con 0x558d859bda30 2024-04-24T09:00:09.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.357+0000 7eff19600640 1 -- 172.21.15.83:0/3791217644 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff1c029070 con 0x558d859bda30 2024-04-24T09:00:09.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.358+0000 7eff32637740 1 -- 172.21.15.83:0/3791217644 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d859bda30 msgr2=0x558d859d4700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:09.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.358+0000 7eff32637740 1 --2- 172.21.15.83:0/3791217644 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d859bda30 0x558d859d4700 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7eff1c00fc30 tx=0x7eff1c00b450 comp rx=0 tx=0).stop 2024-04-24T09:00:09.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.358+0000 7eff32637740 1 -- 172.21.15.83:0/3791217644 shutdown_connections 2024-04-24T09:00:09.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.358+0000 7eff32637740 1 --2- 172.21.15.83:0/3791217644 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558d859c8a20 0x558d859c6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:09.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.358+0000 7eff32637740 1 --2- 172.21.15.83:0/3791217644 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d859be350 0x558d859ba130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:09.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.358+0000 7eff32637740 1 --2- 172.21.15.83:0/3791217644 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d859bda30 0x558d859d4700 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:09.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.358+0000 7eff32637740 1 -- 172.21.15.83:0/3791217644 >> 172.21.15.83:0/3791217644 conn(0x558d859b9d50 msgr2=0x558d859c5920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:09.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.358+0000 7eff32637740 1 -- 172.21.15.83:0/3791217644 shutdown_connections 2024-04-24T09:00:09.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.358+0000 7eff32637740 1 -- 172.21.15.83:0/3791217644 wait complete. 2024-04-24T09:00:09.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.358+0000 7eff32637740 1 -- start start 2024-04-24T09:00:09.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.359+0000 7eff32637740 1 Processor -- start 2024-04-24T09:00:09.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.359+0000 7eff32637740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d859bda30 0x558d859d11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:09.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.359+0000 7eff32637740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558d859be350 0x558d859c24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:09.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.359+0000 7eff32637740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d859c8a20 0x558d859c2be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:09.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.359+0000 7eff32637740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7eff281b6f10 con 0x558d859bda30 2024-04-24T09:00:09.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.359+0000 7eff32637740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7eff281b6840 con 0x558d859c8a20 2024-04-24T09:00:09.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.359+0000 7eff32637740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7eff281b3aa0 con 0x558d859be350 2024-04-24T09:00:09.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.359+0000 7eff32637740 10 client.0 _ll_register_callbacks cb 0x7eff200986d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:00:09.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.359+0000 7eff2ce00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d859bda30 0x558d859d11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:09.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.359+0000 7eff2ce00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d859bda30 0x558d859d11a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48584/0 (socket says 172.21.15.83:48584) 2024-04-24T09:00:09.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.359+0000 7eff2ce00640 1 -- 172.21.15.83:0/1777343950 learned_addr learned my addr 172.21.15.83:0/1777343950 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:00:09.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.360+0000 7eff27e00640 1 --2- 172.21.15.83:0/1777343950 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558d859be350 0x558d859c24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:09.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.360+0000 7eff2ce00640 1 -- 172.21.15.83:0/1777343950 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558d859be350 msgr2=0x558d859c24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:09.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.360+0000 7eff2ce00640 1 --2- 172.21.15.83:0/1777343950 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558d859be350 0x558d859c24a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:09.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.360+0000 7eff2d800640 1 --2- 172.21.15.83:0/1777343950 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d859c8a20 0x558d859c2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:09.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.360+0000 7eff2ce00640 1 -- 172.21.15.83:0/1777343950 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d859c8a20 msgr2=0x558d859c2be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:09.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.360+0000 7eff2ce00640 1 --2- 172.21.15.83:0/1777343950 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d859c8a20 0x558d859c2be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:09.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.360+0000 7eff2ce00640 1 -- 172.21.15.83:0/1777343950 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d855f50f0 con 0x558d859bda30 2024-04-24T09:00:09.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.360+0000 7eff2ce00640 1 --2- 172.21.15.83:0/1777343950 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d859bda30 0x558d859d11a0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7eff1c02d040 tx=0x7eff1c00b950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7c68605b5fba007 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:09.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.360+0000 7eff0ea00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:00:09.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.361+0000 7eff0ea00640 1 -- 172.21.15.83:0/1777343950 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff1c02f040 con 0x558d859bda30 2024-04-24T09:00:09.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.361+0000 7eff0ea00640 1 -- 172.21.15.83:0/1777343950 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9671+0+0 (secure 0 0 0) 0x7eff1c005400 con 0x558d859bda30 2024-04-24T09:00:09.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.361+0000 7eff32637740 1 -- 172.21.15.83:0/1777343950 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff1c00be70 con 0x558d859bda30 2024-04-24T09:00:09.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.361+0000 7eff0ea00640 1 -- 172.21.15.83:0/1777343950 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7eff1c015040 con 0x558d859bda30 2024-04-24T09:00:09.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.361+0000 7eff32637740 10 client.18630 fetch_fsmap learned FSMap version 35 2024-04-24T09:00:09.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.361+0000 7eff32637740 1 -- 172.21.15.83:0/1777343950 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x558d858e8810 con 0x558d859bda30 2024-04-24T09:00:09.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.361+0000 7eff0ea00640 1 -- 172.21.15.83:0/1777343950 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7eff1c00be70 con 0x558d859bda30 2024-04-24T09:00:09.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.361+0000 7eff0ea00640 1 -- 172.21.15.83:0/1777343950 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7eff1c004980 con 0x558d859bda30 2024-04-24T09:00:09.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.361+0000 7eff32637740 10 client.18630 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T09:00:09.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.361+0000 7eff32637740 10 client.18630 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:00:09.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.362+0000 7eff32637740 20 client.18630 populate_metadata read hostname 'smithi083' 2024-04-24T09:00:09.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.362+0000 7eff32637740 1 --2- 172.21.15.83:0/1777343950 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x558d85a3f520 0x558d85a41940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:09.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.362+0000 7eff32637740 4 client.18630 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T09:00:09.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.362+0000 7eff32637740 1 -- 172.21.15.83:0/1777343950 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558d8592aa80 con 0x558d85a3f520 2024-04-24T09:00:09.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.362+0000 7eff27e00640 1 --2- 172.21.15.83:0/1777343950 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x558d85a3f520 0x558d85a41940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:09.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.363+0000 7eff27e00640 1 --2- 172.21.15.83:0/1777343950 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x558d85a3f520 0x558d85a41940 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7683e1b9bed73e4 server_cookie=14645a0ebf180be8 in_seq=0 out_seq=0 2024-04-24T09:00:09.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.363+0000 7eff0ea00640 10 client.18630 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:00:09.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.365+0000 7eff0ea00640 1 -- 172.21.15.83:0/1777343950 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x558d8592aa80 con 0x558d85a3f520 2024-04-24T09:00:09.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.365+0000 7eff0ea00640 10 client.18630 handle_command_reply: tid=0 2024-04-24T09:00:09.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.365+0000 7eff32637740 1 client.18630 shutdown 2024-04-24T09:00:09.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.365+0000 7eff32637740 1 -- 172.21.15.83:0/1777343950 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d859bda30 msgr2=0x558d859d11a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:09.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.365+0000 7eff32637740 1 --2- 172.21.15.83:0/1777343950 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d859bda30 0x558d859d11a0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7eff1c02d040 tx=0x7eff1c00b950 comp rx=0 tx=0).stop 2024-04-24T09:00:09.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.365+0000 7eff32637740 1 -- 172.21.15.83:0/1777343950 shutdown_connections 2024-04-24T09:00:09.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.365+0000 7eff32637740 1 --2- 172.21.15.83:0/1777343950 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x558d85a3f520 0x558d85a41940 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:09.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.365+0000 7eff32637740 1 --2- 172.21.15.83:0/1777343950 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d859c8a20 0x558d859c2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:09.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.365+0000 7eff32637740 1 --2- 172.21.15.83:0/1777343950 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558d859be350 0x558d859c24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:09.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.365+0000 7eff32637740 1 --2- 172.21.15.83:0/1777343950 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d859bda30 0x558d859d11a0 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:09.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.365+0000 7eff0ea00640 0 client.18630 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:00:09.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.365+0000 7eff32637740 1 -- 172.21.15.83:0/1777343950 >> 172.21.15.83:0/1777343950 conn(0x558d859b9d50 msgr2=0x558d859bfa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:09.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.365+0000 7eff32637740 1 -- 172.21.15.83:0/1777343950 shutdown_connections 2024-04-24T09:00:09.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.365+0000 7eff32637740 1 -- 172.21.15.83:0/1777343950 wait complete. 2024-04-24T09:00:09.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.365+0000 7eff32637740 20 client.18630 trim_cache size 0 max 16384 2024-04-24T09:00:09.366 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T09:00:09.366 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-04-24T09:00:09.366 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-04-24T09:00:09.366 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T09:00:09.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.366+0000 7eff2e200640 1 -- 172.21.15.83:0/1467909195 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7eff100a3d20 msgr2=0x7eff100a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:09.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.366+0000 7eff2e200640 1 --2- 172.21.15.83:0/1467909195 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7eff100a3d20 0x7eff100a61e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7eff1402a940 tx=0x7eff14031400 comp rx=0 tx=0).stop 2024-04-24T09:00:09.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.366+0000 7eff2e200640 1 -- 172.21.15.83:0/1467909195 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff2810d500 msgr2=0x7eff2819eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:09.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.366+0000 7eff2e200640 1 --2- 172.21.15.83:0/1467909195 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff2810d500 0x7eff2819eb10 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7eff2000cab0 tx=0x7eff2000cf70 comp rx=0 tx=0).stop 2024-04-24T09:00:09.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.366+0000 7eff2e200640 1 -- 172.21.15.83:0/1467909195 shutdown_connections 2024-04-24T09:00:09.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.366+0000 7eff2e200640 1 --2- 172.21.15.83:0/1467909195 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7eff100a3d20 0x7eff100a61e0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:09.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.366+0000 7eff2e200640 1 --2- 172.21.15.83:0/1467909195 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff2810d500 0x7eff2819eb10 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:09.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.366+0000 7eff2e200640 1 --2- 172.21.15.83:0/1467909195 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff28108f40 0x7eff281a4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:09.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.366+0000 7eff2e200640 1 --2- 172.21.15.83:0/1467909195 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7eff28108580 0x7eff281a4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:09.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.366+0000 7eff2e200640 1 -- 172.21.15.83:0/1467909195 >> 172.21.15.83:0/1467909195 conn(0x7eff28077e20 msgr2=0x7eff280783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:09.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.366+0000 7eff2e200640 1 -- 172.21.15.83:0/1467909195 shutdown_connections 2024-04-24T09:00:09.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.366+0000 7eff2e200640 1 -- 172.21.15.83:0/1467909195 wait complete. 2024-04-24T09:00:09.384 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T09:00:09.384 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T09:00:09.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.599+0000 7fa52ae00640 1 -- 172.21.15.83:0/3552712859 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5240743a0 msgr2=0x7fa524074780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:09.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.599+0000 7fa52ae00640 1 --2- 172.21.15.83:0/3552712859 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5240743a0 0x7fa524074780 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7fa518009900 tx=0x7fa51802a480 comp rx=0 tx=0).stop 2024-04-24T09:00:09.598 INFO:tasks.workunit.client.0.smithi083.stdout: 1 47923 0.63 MB/sec execute 283 sec latency 63.345 ms 2024-04-24T09:00:09.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.599+0000 7fa52ae00640 1 -- 172.21.15.83:0/3552712859 shutdown_connections 2024-04-24T09:00:09.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.599+0000 7fa52ae00640 1 --2- 172.21.15.83:0/3552712859 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa524112df0 0x7fa5241151e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:09.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.599+0000 7fa52ae00640 1 --2- 172.21.15.83:0/3552712859 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa524074cc0 0x7fa5241128b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:09.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.599+0000 7fa52ae00640 1 --2- 172.21.15.83:0/3552712859 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5240743a0 0x7fa524074780 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:09.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.599+0000 7fa52ae00640 1 -- 172.21.15.83:0/3552712859 >> 172.21.15.83:0/3552712859 conn(0x7fa5240fd930 msgr2=0x7fa5240ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:09.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.599+0000 7fa52ae00640 1 -- 172.21.15.83:0/3552712859 shutdown_connections 2024-04-24T09:00:09.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.599+0000 7fa52ae00640 1 -- 172.21.15.83:0/3552712859 wait complete. 2024-04-24T09:00:09.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.600+0000 7fa52ae00640 1 Processor -- start 2024-04-24T09:00:09.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.600+0000 7fa52ae00640 1 -- start start 2024-04-24T09:00:09.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.600+0000 7fa52ae00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa5240743a0 0x7fa5241a48d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:09.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.600+0000 7fa52ae00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa524074cc0 0x7fa5241a4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:09.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.600+0000 7fa52ae00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa524112df0 0x7fa52419ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:09.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.600+0000 7fa52ae00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa5241186e0 con 0x7fa524112df0 2024-04-24T09:00:09.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.600+0000 7fa52ae00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa524118560 con 0x7fa524074cc0 2024-04-24T09:00:09.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.600+0000 7fa52ae00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa524118860 con 0x7fa5240743a0 2024-04-24T09:00:09.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.601+0000 7fa52a400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa524112df0 0x7fa52419ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:09.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.601+0000 7fa52a400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa524112df0 0x7fa52419ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48600/0 (socket says 172.21.15.83:48600) 2024-04-24T09:00:09.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.601+0000 7fa52a400640 1 -- 172.21.15.83:0/337268371 learned_addr learned my addr 172.21.15.83:0/337268371 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:00:09.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.601+0000 7fa529000640 1 --2- 172.21.15.83:0/337268371 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa524074cc0 0x7fa5241a4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:09.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.601+0000 7fa529a00640 1 --2- 172.21.15.83:0/337268371 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa5240743a0 0x7fa5241a48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:09.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.601+0000 7fa52a400640 1 -- 172.21.15.83:0/337268371 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa5240743a0 msgr2=0x7fa5241a48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:09.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.601+0000 7fa52a400640 1 --2- 172.21.15.83:0/337268371 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa5240743a0 0x7fa5241a48d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:09.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.601+0000 7fa52a400640 1 -- 172.21.15.83:0/337268371 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa524074cc0 msgr2=0x7fa5241a4e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:09.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.601+0000 7fa52a400640 1 --2- 172.21.15.83:0/337268371 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa524074cc0 0x7fa5241a4e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:09.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.601+0000 7fa52a400640 1 -- 172.21.15.83:0/337268371 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa52419f200 con 0x7fa524112df0 2024-04-24T09:00:09.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.601+0000 7fa529000640 1 --2- 172.21.15.83:0/337268371 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa524074cc0 0x7fa5241a4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:00:09.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.602+0000 7fa52a400640 1 --2- 172.21.15.83:0/337268371 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa524112df0 0x7fa52419ea80 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7fa51c00ccb0 tx=0x7fa51c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:09.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.602+0000 7fa522000640 1 -- 172.21.15.83:0/337268371 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa51c00f450 con 0x7fa524112df0 2024-04-24T09:00:09.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.602+0000 7fa522000640 1 -- 172.21.15.83:0/337268371 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa51c00faa0 con 0x7fa524112df0 2024-04-24T09:00:09.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.602+0000 7fa522000640 1 -- 172.21.15.83:0/337268371 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa51c018b10 con 0x7fa524112df0 2024-04-24T09:00:09.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.602+0000 7fa52ae00640 1 -- 172.21.15.83:0/337268371 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa52419f4f0 con 0x7fa524112df0 2024-04-24T09:00:09.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.602+0000 7fa52ae00640 1 -- 172.21.15.83:0/337268371 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5241ab790 con 0x7fa524112df0 2024-04-24T09:00:09.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.603+0000 7fa52ae00640 1 -- 172.21.15.83:0/337268371 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa524069190 con 0x7fa524112df0 2024-04-24T09:00:09.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.605+0000 7fa522000640 1 -- 172.21.15.83:0/337268371 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa51c00f5f0 con 0x7fa524112df0 2024-04-24T09:00:09.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.607+0000 7fa522000640 1 --2- 172.21.15.83:0/337268371 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa50c0a3da0 0x7fa50c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:09.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.607+0000 7fa522000640 1 -- 172.21.15.83:0/337268371 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7fa51c0d1410 con 0x7fa524112df0 2024-04-24T09:00:09.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.607+0000 7fa529a00640 1 --2- 172.21.15.83:0/337268371 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa50c0a3da0 0x7fa50c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:09.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.608+0000 7fa529a00640 1 --2- 172.21.15.83:0/337268371 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa50c0a3da0 0x7fa50c0a6260 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fa51802a9a0 tx=0x7fa518000bc0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:09.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.609+0000 7fa522000640 1 -- 172.21.15.83:0/337268371 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa51c014030 con 0x7fa524112df0 2024-04-24T09:00:09.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:00:09 smithi083 ceph-mon[27907]: pgmap v266: 129 pgs: 129 active+clean; 49 MiB data, 729 MiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 443 KiB/s wr, 86 op/s 2024-04-24T09:00:09.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:00:09 smithi083 ceph-mon[27907]: osdmap e102: 12 total, 12 up, 12 in 2024-04-24T09:00:09.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.878+0000 7fa52ae00640 1 -- 172.21.15.83:0/337268371 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa52419ffb0 con 0x7fa524112df0 2024-04-24T09:00:09.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.881+0000 7fa522000640 1 -- 172.21.15.83:0/337268371 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11944 (secure 0 0 0) 0x7fa51c09d740 con 0x7fa524112df0 2024-04-24T09:00:09.880 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T09:00:09.880 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/358872257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":358872257},{"type":"v1","addr":"172.21.15.83:6835","nonce":358872257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3157070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3157070851},{"type":"v1","addr":"172.21.15.83:6837","nonce":3157070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":35,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T08:48:29.422576+0000","modified":"2024-04-24T08:59:06.888262+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34520,"mds_1":34493,"mds_2":34451,"mds_3":34427,"mds_4":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14658":{"gid":14658,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6839/4085470844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":4085470844},{"type":"v1","addr":"172.21.15.83:6839","nonce":4085470844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14676":{"gid":14676,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6841/2634754210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":2634754210},{"type":"v1","addr":"172.21.15.83:6841","nonce":2634754210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24382":{"gid":24382,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/1859921310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1859921310},{"type":"v1","addr":"172.21.15.189:6833","nonce":1859921310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/2241923197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2241923197},{"type":"v1","addr":"172.21.15.189:6835","nonce":2241923197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.189:6839/2548965336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":2548965336},{"type":"v1","addr":"172.21.15.189:6839","nonce":2548965336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26686":{"gid":26686,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6837/1678817161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":1678817161},{"type":"v1","addr":"172.21.15.189:6837","nonce":1678817161}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.138:6833/1157967658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1157967658},{"type":"v1","addr":"172.21.15.138:6833","nonce":1157967658}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.138:6835/1909219758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1909219758},{"type":"v1","addr":"172.21.15.138:6835","nonce":1909219758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34493":{"gid":34493,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.138:6837/900566754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":900566754},{"type":"v1","addr":"172.21.15.138:6837","nonce":900566754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34520":{"gid":34520,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.138:6839/2236932393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2236932393},{"type":"v1","addr":"172.21.15.138:6839","nonce":2236932393}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34520,"qdb_cluster":[34520,24454,34427,34493,34451]},"id":1}]} 2024-04-24T09:00:09.880 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 35 2024-04-24T09:00:09.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.883+0000 7fa52ae00640 1 -- 172.21.15.83:0/337268371 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa50c0a3da0 msgr2=0x7fa50c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:09.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.883+0000 7fa52ae00640 1 --2- 172.21.15.83:0/337268371 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa50c0a3da0 0x7fa50c0a6260 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fa51802a9a0 tx=0x7fa518000bc0 comp rx=0 tx=0).stop 2024-04-24T09:00:09.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.883+0000 7fa52ae00640 1 -- 172.21.15.83:0/337268371 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa524112df0 msgr2=0x7fa52419ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:09.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.883+0000 7fa52ae00640 1 --2- 172.21.15.83:0/337268371 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa524112df0 0x7fa52419ea80 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7fa51c00ccb0 tx=0x7fa51c00a620 comp rx=0 tx=0).stop 2024-04-24T09:00:09.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.884+0000 7fa52ae00640 1 -- 172.21.15.83:0/337268371 shutdown_connections 2024-04-24T09:00:09.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.884+0000 7fa52ae00640 1 --2- 172.21.15.83:0/337268371 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa50c0a3da0 0x7fa50c0a6260 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:09.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.884+0000 7fa52ae00640 1 --2- 172.21.15.83:0/337268371 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa524112df0 0x7fa52419ea80 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:09.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.884+0000 7fa52ae00640 1 --2- 172.21.15.83:0/337268371 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa524074cc0 0x7fa5241a4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:09.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.884+0000 7fa52ae00640 1 --2- 172.21.15.83:0/337268371 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa5240743a0 0x7fa5241a48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:09.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.884+0000 7fa52ae00640 1 -- 172.21.15.83:0/337268371 >> 172.21.15.83:0/337268371 conn(0x7fa5240fd930 msgr2=0x7fa5240ff190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:09.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.884+0000 7fa52ae00640 1 -- 172.21.15.83:0/337268371 shutdown_connections 2024-04-24T09:00:09.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:09.884+0000 7fa52ae00640 1 -- 172.21.15.83:0/337268371 wait complete. 2024-04-24T09:00:09.901 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-24T09:00:09.954 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:00:09 smithi138 ceph-mon[31441]: pgmap v266: 129 pgs: 129 active+clean; 49 MiB data, 729 MiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 443 KiB/s wr, 86 op/s 2024-04-24T09:00:09.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:00:09 smithi138 ceph-mon[31441]: osdmap e102: 12 total, 12 up, 12 in 2024-04-24T09:00:09.956 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:00:09 smithi189 ceph-mon[31791]: pgmap v266: 129 pgs: 129 active+clean; 49 MiB data, 729 MiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 443 KiB/s wr, 86 op/s 2024-04-24T09:00:09.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:00:09 smithi189 ceph-mon[31791]: osdmap e102: 12 total, 12 up, 12 in 2024-04-24T09:00:10.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.112+0000 7fe6ea600640 1 -- 172.21.15.83:0/873217539 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6e4103060 msgr2=0x7fe6e4103440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:10.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.112+0000 7fe6ea600640 1 --2- 172.21.15.83:0/873217539 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6e4103060 0x7fe6e4103440 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7fe6d8009900 tx=0x7fe6d802a480 comp rx=0 tx=0).stop 2024-04-24T09:00:10.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.112+0000 7fe6ea600640 1 -- 172.21.15.83:0/873217539 shutdown_connections 2024-04-24T09:00:10.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.112+0000 7fe6ea600640 1 --2- 172.21.15.83:0/873217539 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe6e410c480 0x7fe6e410e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.112+0000 7fe6ea600640 1 --2- 172.21.15.83:0/873217539 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe6e4103980 0x7fe6e410be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.112+0000 7fe6ea600640 1 --2- 172.21.15.83:0/873217539 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6e4103060 0x7fe6e4103440 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.112+0000 7fe6ea600640 1 -- 172.21.15.83:0/873217539 >> 172.21.15.83:0/873217539 conn(0x7fe6e40fc880 msgr2=0x7fe6e40feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:10.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.112+0000 7fe6ea600640 1 -- 172.21.15.83:0/873217539 shutdown_connections 2024-04-24T09:00:10.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.112+0000 7fe6ea600640 1 -- 172.21.15.83:0/873217539 wait complete. 2024-04-24T09:00:10.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.114+0000 7fe6ea600640 1 Processor -- start 2024-04-24T09:00:10.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.114+0000 7fe6ea600640 1 -- start start 2024-04-24T09:00:10.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.114+0000 7fe6ea600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe6e4103060 0x7fe6e41a4a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:10.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.114+0000 7fe6ea600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe6e4103980 0x7fe6e41a4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:10.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.114+0000 7fe6ea600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6e410c480 0x7fe6e410b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:10.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.114+0000 7fe6ea600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe6e4112160 con 0x7fe6e410c480 2024-04-24T09:00:10.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.114+0000 7fe6ea600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe6e4111fe0 con 0x7fe6e4103060 2024-04-24T09:00:10.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.114+0000 7fe6ea600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe6e41122e0 con 0x7fe6e4103980 2024-04-24T09:00:10.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.115+0000 7fe6e9c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6e410c480 0x7fe6e410b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:10.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.115+0000 7fe6e9c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6e410c480 0x7fe6e410b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48612/0 (socket says 172.21.15.83:48612) 2024-04-24T09:00:10.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.115+0000 7fe6e9c00640 1 -- 172.21.15.83:0/2318379990 learned_addr learned my addr 172.21.15.83:0/2318379990 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:00:10.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.115+0000 7fe6e9200640 1 --2- 172.21.15.83:0/2318379990 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe6e4103060 0x7fe6e41a4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:10.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.115+0000 7fe6e3e00640 1 --2- 172.21.15.83:0/2318379990 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe6e4103980 0x7fe6e41a4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:10.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.115+0000 7fe6e9c00640 1 -- 172.21.15.83:0/2318379990 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe6e4103980 msgr2=0x7fe6e41a4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:10.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.115+0000 7fe6e9c00640 1 --2- 172.21.15.83:0/2318379990 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe6e4103980 0x7fe6e41a4f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.115+0000 7fe6e9c00640 1 -- 172.21.15.83:0/2318379990 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe6e4103060 msgr2=0x7fe6e41a4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:10.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.115+0000 7fe6e9c00640 1 --2- 172.21.15.83:0/2318379990 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe6e4103060 0x7fe6e41a4a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.115+0000 7fe6e9c00640 1 -- 172.21.15.83:0/2318379990 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6e410b9e0 con 0x7fe6e410c480 2024-04-24T09:00:10.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.115+0000 7fe6e3e00640 1 --2- 172.21.15.83:0/2318379990 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe6e4103980 0x7fe6e41a4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:00:10.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.116+0000 7fe6e9c00640 1 --2- 172.21.15.83:0/2318379990 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6e410c480 0x7fe6e410b120 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc00eaa0 tx=0x7fe6dc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:10.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.116+0000 7fe6e1600640 1 -- 172.21.15.83:0/2318379990 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6dc00cdb0 con 0x7fe6e410c480 2024-04-24T09:00:10.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.116+0000 7fe6e1600640 1 -- 172.21.15.83:0/2318379990 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe6dc005020 con 0x7fe6e410c480 2024-04-24T09:00:10.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.116+0000 7fe6e1600640 1 -- 172.21.15.83:0/2318379990 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6dc0116d0 con 0x7fe6e410c480 2024-04-24T09:00:10.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.116+0000 7fe6ea600640 1 -- 172.21.15.83:0/2318379990 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6e410bcd0 con 0x7fe6e410c480 2024-04-24T09:00:10.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.116+0000 7fe6ea600640 1 -- 172.21.15.83:0/2318379990 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6e41ab990 con 0x7fe6e410c480 2024-04-24T09:00:10.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.120+0000 7fe6ee9f6740 1 Processor -- start 2024-04-24T09:00:10.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.121+0000 7fe6e1600640 1 -- 172.21.15.83:0/2318379990 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe6dc0051d0 con 0x7fe6e410c480 2024-04-24T09:00:10.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.121+0000 7fe6ee9f6740 1 -- start start 2024-04-24T09:00:10.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.121+0000 7fe6ee9f6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e9a00e1b40 0x55e9a00e1f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:10.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.121+0000 7fe6ee9f6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9a00e2460 0x55e9a00ec5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:10.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.121+0000 7fe6ee9f6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9a00ecb30 0x55e9a00eef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:10.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.121+0000 7fe6ee9f6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e9a00471d0 con 0x55e9a00e2460 2024-04-24T09:00:10.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.121+0000 7fe6ee9f6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e99ffed450 con 0x55e9a00ecb30 2024-04-24T09:00:10.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.121+0000 7fe6ee9f6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55e9a0043190 con 0x55e9a00e1b40 2024-04-24T09:00:10.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.122+0000 7fe6e9200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9a00e2460 0x55e9a00ec5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:10.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.122+0000 7fe6e9200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9a00e2460 0x55e9a00ec5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48622/0 (socket says 172.21.15.83:48622) 2024-04-24T09:00:10.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.122+0000 7fe6e9200640 1 -- 172.21.15.83:0/659476727 learned_addr learned my addr 172.21.15.83:0/659476727 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:00:10.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.122+0000 7fe6e3e00640 1 --2- 172.21.15.83:0/659476727 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e9a00e1b40 0x55e9a00e1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:10.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.122+0000 7fe6e3e00640 1 --2- 172.21.15.83:0/659476727 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9a00ecb30 0x55e9a00eef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:10.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.122+0000 7fe6e9200640 1 -- 172.21.15.83:0/659476727 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e9a00e1b40 msgr2=0x55e9a00e1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:10.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.122+0000 7fe6e9200640 1 --2- 172.21.15.83:0/659476727 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e9a00e1b40 0x55e9a00e1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.122+0000 7fe6e9200640 1 -- 172.21.15.83:0/659476727 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9a00ecb30 msgr2=0x55e9a00eef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:10.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.122+0000 7fe6e9200640 1 --2- 172.21.15.83:0/659476727 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9a00ecb30 0x55e9a00eef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.122+0000 7fe6e9200640 1 -- 172.21.15.83:0/659476727 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9a00aa310 con 0x55e9a00e2460 2024-04-24T09:00:10.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.122+0000 7fe6e1600640 1 --2- 172.21.15.83:0/2318379990 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe6cc0a3d20 0x7fe6cc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:10.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.123+0000 7fe6e1600640 1 -- 172.21.15.83:0/2318379990 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7fe6dc0d0e00 con 0x7fe6e410c480 2024-04-24T09:00:10.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.123+0000 7fe6e9200640 1 --2- 172.21.15.83:0/659476727 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9a00e2460 0x55e9a00ec5f0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7fe6d8000c00 tx=0x7fe6d8004180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad6f2c2385bbabda server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:10.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.123+0000 7fe6e3e00640 1 --2- 172.21.15.83:0/2318379990 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe6cc0a3d20 0x7fe6cc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:10.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.123+0000 7fe6d2000640 1 -- 172.21.15.83:0/659476727 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d8035040 con 0x55e9a00e2460 2024-04-24T09:00:10.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.123+0000 7fe6d2000640 1 -- 172.21.15.83:0/659476727 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe6d80054f0 con 0x55e9a00e2460 2024-04-24T09:00:10.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.123+0000 7fe6d2000640 1 -- 172.21.15.83:0/659476727 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d80035f0 con 0x55e9a00e2460 2024-04-24T09:00:10.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.124+0000 7fe6ee9f6740 1 -- 172.21.15.83:0/659476727 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9a00e2460 msgr2=0x55e9a00ec5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:10.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.124+0000 7fe6e3e00640 1 --2- 172.21.15.83:0/2318379990 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe6cc0a3d20 0x7fe6cc0a61e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fe6d400c010 tx=0x7fe6d4005850 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:10.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.124+0000 7fe6ee9f6740 1 --2- 172.21.15.83:0/659476727 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9a00e2460 0x55e9a00ec5f0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7fe6d8000c00 tx=0x7fe6d8004180 comp rx=0 tx=0).stop 2024-04-24T09:00:10.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.124+0000 7fe6ee9f6740 1 -- 172.21.15.83:0/659476727 shutdown_connections 2024-04-24T09:00:10.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.124+0000 7fe6ee9f6740 1 --2- 172.21.15.83:0/659476727 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9a00ecb30 0x55e9a00eef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.124+0000 7fe6ee9f6740 1 --2- 172.21.15.83:0/659476727 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9a00e2460 0x55e9a00ec5f0 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.124+0000 7fe6ee9f6740 1 --2- 172.21.15.83:0/659476727 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e9a00e1b40 0x55e9a00e1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.124+0000 7fe6ee9f6740 1 -- 172.21.15.83:0/659476727 >> 172.21.15.83:0/659476727 conn(0x55e9a00dde60 msgr2=0x55e9a00de240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:10.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.124+0000 7fe6ee9f6740 1 -- 172.21.15.83:0/659476727 shutdown_connections 2024-04-24T09:00:10.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.124+0000 7fe6ee9f6740 1 -- 172.21.15.83:0/659476727 wait complete. 2024-04-24T09:00:10.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.125+0000 7fe6ee9f6740 1 -- start start 2024-04-24T09:00:10.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.125+0000 7fe6ee9f6740 1 Processor -- start 2024-04-24T09:00:10.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.125+0000 7fe6ee9f6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9a00e1b40 0x55e9a00ec580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:10.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.125+0000 7fe6ee9f6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e9a00e2460 0x55e9a00e9870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:10.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.126+0000 7fe6ee9f6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9a00ecb30 0x55e9a00e9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:10.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.126+0000 7fe6ee9f6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e9a00ef960 con 0x55e9a00e1b40 2024-04-24T09:00:10.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.126+0000 7fe6ee9f6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e9a0000810 con 0x55e9a00ecb30 2024-04-24T09:00:10.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.126+0000 7fe6ee9f6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55e9a00efae0 con 0x55e9a00e2460 2024-04-24T09:00:10.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.126+0000 7fe6ee9f6740 10 client.0 _ll_register_callbacks cb 0x55e99ffce140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:00:10.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.126+0000 7fe6e9200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9a00e1b40 0x55e9a00ec580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:10.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.126+0000 7fe6e9200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9a00e1b40 0x55e9a00ec580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48638/0 (socket says 172.21.15.83:48638) 2024-04-24T09:00:10.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.126+0000 7fe6e9200640 1 -- 172.21.15.83:0/2565787966 learned_addr learned my addr 172.21.15.83:0/2565787966 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:00:10.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.126+0000 7fe6e3e00640 1 --2- 172.21.15.83:0/2565787966 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e9a00e2460 0x55e9a00e9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:10.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.126+0000 7fe6e9c00640 1 --2- 172.21.15.83:0/2565787966 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9a00ecb30 0x55e9a00e9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:10.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.126+0000 7fe6e9200640 1 -- 172.21.15.83:0/2565787966 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e9a00e2460 msgr2=0x55e9a00e9870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:10.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.126+0000 7fe6e9200640 1 --2- 172.21.15.83:0/2565787966 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e9a00e2460 0x55e9a00e9870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.126+0000 7fe6e9200640 1 -- 172.21.15.83:0/2565787966 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9a00ecb30 msgr2=0x55e9a00e9fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:10.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.126+0000 7fe6e9200640 1 --2- 172.21.15.83:0/2565787966 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9a00ecb30 0x55e9a00e9fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.126+0000 7fe6e9200640 1 -- 172.21.15.83:0/2565787966 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9a00113f0 con 0x55e9a00e1b40 2024-04-24T09:00:10.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.127+0000 7fe6e9200640 1 --2- 172.21.15.83:0/2565787966 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9a00e1b40 0x55e9a00ec580 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7fe6d802a9a0 tx=0x7fe6d80056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b5abd27f707dbed server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:10.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.127+0000 7fe6c6a00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:00:10.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.127+0000 7fe6c6a00640 1 -- 172.21.15.83:0/2565787966 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d8052060 con 0x55e9a00e1b40 2024-04-24T09:00:10.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.127+0000 7fe6c6a00640 1 -- 172.21.15.83:0/2565787966 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7fe6d8004980 con 0x55e9a00e1b40 2024-04-24T09:00:10.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.127+0000 7fe6ee9f6740 1 -- 172.21.15.83:0/2565787966 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6d80054f0 con 0x55e9a00e1b40 2024-04-24T09:00:10.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.127+0000 7fe6c6a00640 1 -- 172.21.15.83:0/2565787966 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fe6d8059050 con 0x55e9a00e1b40 2024-04-24T09:00:10.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.127+0000 7fe6ee9f6740 10 client.18648 fetch_fsmap learned FSMap version 35 2024-04-24T09:00:10.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.127+0000 7fe6ee9f6740 1 -- 172.21.15.83:0/2565787966 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55e99fd593f0 con 0x55e9a00e1b40 2024-04-24T09:00:10.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.128+0000 7fe6c6a00640 1 -- 172.21.15.83:0/2565787966 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe6d80054f0 con 0x55e9a00e1b40 2024-04-24T09:00:10.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.128+0000 7fe6c6a00640 1 -- 172.21.15.83:0/2565787966 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7fe6d8003c40 con 0x55e9a00e1b40 2024-04-24T09:00:10.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.128+0000 7fe6ee9f6740 10 client.18648 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T09:00:10.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.128+0000 7fe6ee9f6740 10 client.18648 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-24T09:00:10.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.128+0000 7fe6ee9f6740 20 client.18648 populate_metadata read hostname 'smithi083' 2024-04-24T09:00:10.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.128+0000 7fe6ee9f6740 1 --2- 172.21.15.83:0/2565787966 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55e9a00fcb60 0x55e9a00fef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:10.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.128+0000 7fe6ee9f6740 4 client.18648 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:00:10.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.128+0000 7fe6ee9f6740 1 -- 172.21.15.83:0/2565787966 --> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e9a0054bf0 con 0x55e9a00fcb60 2024-04-24T09:00:10.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.129+0000 7fe6e3e00640 1 --2- 172.21.15.83:0/2565787966 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55e9a00fcb60 0x55e9a00fef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:10.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.130+0000 7fe6c6a00640 10 client.18648 ms_handle_connect on v2:172.21.15.189:6838/2548965336 2024-04-24T09:00:10.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.130+0000 7fe6e3e00640 1 --2- 172.21.15.83:0/2565787966 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55e9a00fcb60 0x55e9a00fef50 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d9eecee3b2e9e9f1 server_cookie=7b95dd9c69baebc0 in_seq=0 out_seq=0 2024-04-24T09:00:10.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.132+0000 7fe6c6a00640 1 -- 172.21.15.83:0/2565787966 <== mds.4 v2:172.21.15.189:6838/2548965336 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e9a0054bf0 con 0x55e9a00fcb60 2024-04-24T09:00:10.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.132+0000 7fe6c6a00640 10 client.18648 handle_command_reply: tid=0 2024-04-24T09:00:10.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.132+0000 7fe6ee9f6740 1 client.18648 shutdown 2024-04-24T09:00:10.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.132+0000 7fe6ee9f6740 1 -- 172.21.15.83:0/2565787966 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9a00e1b40 msgr2=0x55e9a00ec580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:10.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.132+0000 7fe6ee9f6740 1 --2- 172.21.15.83:0/2565787966 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9a00e1b40 0x55e9a00ec580 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7fe6d802a9a0 tx=0x7fe6d80056a0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.132+0000 7fe6ee9f6740 1 -- 172.21.15.83:0/2565787966 shutdown_connections 2024-04-24T09:00:10.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.132+0000 7fe6ee9f6740 1 --2- 172.21.15.83:0/2565787966 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55e9a00fcb60 0x55e9a00fef50 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.132+0000 7fe6ee9f6740 1 --2- 172.21.15.83:0/2565787966 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9a00ecb30 0x55e9a00e9fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.132+0000 7fe6ee9f6740 1 --2- 172.21.15.83:0/2565787966 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e9a00e2460 0x55e9a00e9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.132+0000 7fe6ee9f6740 1 --2- 172.21.15.83:0/2565787966 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9a00e1b40 0x55e9a00ec580 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.132+0000 7fe6c6a00640 0 client.18648 ms_handle_reset on v2:172.21.15.189:6838/2548965336 2024-04-24T09:00:10.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.132+0000 7fe6ee9f6740 1 -- 172.21.15.83:0/2565787966 >> 172.21.15.83:0/2565787966 conn(0x55e9a00dde60 msgr2=0x55e9a00e3b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:10.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.132+0000 7fe6ee9f6740 1 -- 172.21.15.83:0/2565787966 shutdown_connections 2024-04-24T09:00:10.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.132+0000 7fe6ee9f6740 1 -- 172.21.15.83:0/2565787966 wait complete. 2024-04-24T09:00:10.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.132+0000 7fe6ee9f6740 20 client.18648 trim_cache size 0 max 16384 2024-04-24T09:00:10.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.149+0000 7fe6ee9f6740 1 Processor -- start 2024-04-24T09:00:10.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.149+0000 7fe6ee9f6740 1 -- start start 2024-04-24T09:00:10.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.149+0000 7fe6ee9f6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9a00e1b40 0x55e9a00f8850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:10.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.149+0000 7fe6ee9f6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e9a00e2460 0x55e9a00ec0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:10.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.149+0000 7fe6ee9f6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9a00ecb30 0x55e9a00ec5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:10.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.149+0000 7fe6ee9f6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe6e41b59f0 con 0x55e9a00ecb30 2024-04-24T09:00:10.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.149+0000 7fe6ee9f6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe6e41b7530 con 0x55e9a00e1b40 2024-04-24T09:00:10.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.149+0000 7fe6ee9f6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe6e41b60c0 con 0x55e9a00e2460 2024-04-24T09:00:10.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.149+0000 7fe6e9c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9a00ecb30 0x55e9a00ec5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:10.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.149+0000 7fe6e9c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9a00ecb30 0x55e9a00ec5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48648/0 (socket says 172.21.15.83:48648) 2024-04-24T09:00:10.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.149+0000 7fe6e9c00640 1 -- 172.21.15.83:0/477510858 learned_addr learned my addr 172.21.15.83:0/477510858 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:00:10.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.150+0000 7fe6e3e00640 1 --2- 172.21.15.83:0/477510858 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e9a00e2460 0x55e9a00ec0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:10.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.150+0000 7fe6e9c00640 1 -- 172.21.15.83:0/477510858 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e9a00e2460 msgr2=0x55e9a00ec0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:10.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.150+0000 7fe6e9c00640 1 --2- 172.21.15.83:0/477510858 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e9a00e2460 0x55e9a00ec0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.150+0000 7fe6e9200640 1 --2- 172.21.15.83:0/477510858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9a00e1b40 0x55e9a00f8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:10.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.150+0000 7fe6e9c00640 1 -- 172.21.15.83:0/477510858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9a00e1b40 msgr2=0x55e9a00f8850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:10.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.150+0000 7fe6e9c00640 1 --2- 172.21.15.83:0/477510858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9a00e1b40 0x55e9a00f8850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.150+0000 7fe6e9c00640 1 -- 172.21.15.83:0/477510858 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9a00ea320 con 0x55e9a00ecb30 2024-04-24T09:00:10.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.150+0000 7fe6e3e00640 1 --2- 172.21.15.83:0/477510858 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e9a00e2460 0x55e9a00ec0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:00:10.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.150+0000 7fe6e9c00640 1 --2- 172.21.15.83:0/477510858 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9a00ecb30 0x55e9a00ec5f0 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc098d80 tx=0x7fe6dc09d0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c3eae577d55ece3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:10.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.150+0000 7fe6d1600640 1 -- 172.21.15.83:0/477510858 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6dc01f040 con 0x55e9a00ecb30 2024-04-24T09:00:10.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.150+0000 7fe6d1600640 1 -- 172.21.15.83:0/477510858 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe6dc0a5590 con 0x55e9a00ecb30 2024-04-24T09:00:10.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.150+0000 7fe6d1600640 1 -- 172.21.15.83:0/477510858 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6dc085e60 con 0x55e9a00ecb30 2024-04-24T09:00:10.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.151+0000 7fe6ee9f6740 1 -- 172.21.15.83:0/477510858 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9a00ecb30 msgr2=0x55e9a00ec5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:10.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.151+0000 7fe6ee9f6740 1 --2- 172.21.15.83:0/477510858 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9a00ecb30 0x55e9a00ec5f0 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc098d80 tx=0x7fe6dc09d0e0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.151+0000 7fe6ee9f6740 1 -- 172.21.15.83:0/477510858 shutdown_connections 2024-04-24T09:00:10.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.151+0000 7fe6ee9f6740 1 --2- 172.21.15.83:0/477510858 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9a00ecb30 0x55e9a00ec5f0 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.151+0000 7fe6ee9f6740 1 --2- 172.21.15.83:0/477510858 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e9a00e2460 0x55e9a00ec0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.151+0000 7fe6ee9f6740 1 --2- 172.21.15.83:0/477510858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9a00e1b40 0x55e9a00f8850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.151+0000 7fe6ee9f6740 1 -- 172.21.15.83:0/477510858 >> 172.21.15.83:0/477510858 conn(0x55e9a00dde60 msgr2=0x55e9a00eedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:10.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.151+0000 7fe6ee9f6740 1 -- 172.21.15.83:0/477510858 shutdown_connections 2024-04-24T09:00:10.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.151+0000 7fe6ee9f6740 1 -- 172.21.15.83:0/477510858 wait complete. 2024-04-24T09:00:10.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.152+0000 7fe6ee9f6740 1 -- start start 2024-04-24T09:00:10.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.152+0000 7fe6ee9f6740 1 Processor -- start 2024-04-24T09:00:10.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.152+0000 7fe6ee9f6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9a00e1b40 0x55e9a00ed9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:10.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.152+0000 7fe6ee9f6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9a00e2460 0x55e9a00edef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:10.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.152+0000 7fe6ee9f6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e9a00ecb30 0x55e9a00f3d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:10.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.152+0000 7fe6ee9f6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe6e41b6e60 con 0x55e9a00e2460 2024-04-24T09:00:10.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.152+0000 7fe6ee9f6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe6e41b6790 con 0x55e9a00e1b40 2024-04-24T09:00:10.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.152+0000 7fe6ee9f6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe6e41b39f0 con 0x55e9a00ecb30 2024-04-24T09:00:10.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.152+0000 7fe6ee9f6740 10 client.0 _ll_register_callbacks cb 0x7fe6e406a1f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:00:10.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.153+0000 7fe6e3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9a00e2460 0x55e9a00edef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:10.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.153+0000 7fe6e3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9a00e2460 0x55e9a00edef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48654/0 (socket says 172.21.15.83:48654) 2024-04-24T09:00:10.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.153+0000 7fe6e3e00640 1 -- 172.21.15.83:0/1556926808 learned_addr learned my addr 172.21.15.83:0/1556926808 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:00:10.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.153+0000 7fe6e9c00640 1 --2- 172.21.15.83:0/1556926808 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e9a00ecb30 0x55e9a00f3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:10.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.153+0000 7fe6e3e00640 1 -- 172.21.15.83:0/1556926808 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e9a00ecb30 msgr2=0x55e9a00f3d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:10.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.153+0000 7fe6e9200640 1 --2- 172.21.15.83:0/1556926808 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9a00e1b40 0x55e9a00ed9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:10.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.153+0000 7fe6e3e00640 1 --2- 172.21.15.83:0/1556926808 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e9a00ecb30 0x55e9a00f3d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.153+0000 7fe6e3e00640 1 -- 172.21.15.83:0/1556926808 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9a00e1b40 msgr2=0x55e9a00ed9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:10.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.153+0000 7fe6e3e00640 1 --2- 172.21.15.83:0/1556926808 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9a00e1b40 0x55e9a00ed9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.153+0000 7fe6e3e00640 1 -- 172.21.15.83:0/1556926808 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e99fe237d0 con 0x55e9a00e2460 2024-04-24T09:00:10.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.153+0000 7fe6e9c00640 1 --2- 172.21.15.83:0/1556926808 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e9a00ecb30 0x55e9a00f3d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:00:10.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.153+0000 7fe6e3e00640 1 --2- 172.21.15.83:0/1556926808 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9a00e2460 0x55e9a00edef0 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7fe6d4011810 tx=0x7fe6d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63f1f8f9365b11bd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:10.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.153+0000 7fe6c6a00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:00:10.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.154+0000 7fe6c6a00640 1 -- 172.21.15.83:0/1556926808 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d40273d0 con 0x55e9a00e2460 2024-04-24T09:00:10.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.154+0000 7fe6c6a00640 1 -- 172.21.15.83:0/1556926808 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7fe6d4027570 con 0x55e9a00e2460 2024-04-24T09:00:10.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.154+0000 7fe6ee9f6740 1 -- 172.21.15.83:0/1556926808 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6dc0a5590 con 0x55e9a00e2460 2024-04-24T09:00:10.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.154+0000 7fe6c6a00640 1 -- 172.21.15.83:0/1556926808 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fe6d4027770 con 0x55e9a00e2460 2024-04-24T09:00:10.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.154+0000 7fe6ee9f6740 10 client.18654 fetch_fsmap learned FSMap version 35 2024-04-24T09:00:10.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.154+0000 7fe6ee9f6740 1 -- 172.21.15.83:0/1556926808 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55e9a00f5230 con 0x55e9a00e2460 2024-04-24T09:00:10.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.154+0000 7fe6c6a00640 1 -- 172.21.15.83:0/1556926808 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe6dc0a5590 con 0x55e9a00e2460 2024-04-24T09:00:10.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.155+0000 7fe6c6a00640 1 -- 172.21.15.83:0/1556926808 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7fe6d40203f0 con 0x55e9a00e2460 2024-04-24T09:00:10.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.155+0000 7fe6ee9f6740 10 client.18654 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T09:00:10.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.155+0000 7fe6ee9f6740 10 client.18654 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-24T09:00:10.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.155+0000 7fe6ee9f6740 20 client.18654 populate_metadata read hostname 'smithi083' 2024-04-24T09:00:10.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.155+0000 7fe6ee9f6740 1 --2- 172.21.15.83:0/1556926808 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55e9a0162f40 0x55e9a0165360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:10.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.155+0000 7fe6ee9f6740 4 client.18654 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:00:10.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.155+0000 7fe6ee9f6740 1 -- 172.21.15.83:0/1556926808 --> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e9a004ea60 con 0x55e9a0162f40 2024-04-24T09:00:10.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.155+0000 7fe6e9200640 1 --2- 172.21.15.83:0/1556926808 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55e9a0162f40 0x55e9a0165360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:10.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.156+0000 7fe6e9200640 1 --2- 172.21.15.83:0/1556926808 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55e9a0162f40 0x55e9a0165360 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d9a1d8323a6dc5d3 server_cookie=5cfa6d22063d57cc in_seq=0 out_seq=0 2024-04-24T09:00:10.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.156+0000 7fe6c6a00640 10 client.18654 ms_handle_connect on v2:172.21.15.189:6838/2548965336 2024-04-24T09:00:10.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.157+0000 7fe6c6a00640 1 -- 172.21.15.83:0/1556926808 <== mds.4 v2:172.21.15.189:6838/2548965336 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e9a004ea60 con 0x55e9a0162f40 2024-04-24T09:00:10.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.157+0000 7fe6c6a00640 10 client.18654 handle_command_reply: tid=0 2024-04-24T09:00:10.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.157+0000 7fe6ee9f6740 1 client.18654 shutdown 2024-04-24T09:00:10.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.157+0000 7fe6ee9f6740 1 -- 172.21.15.83:0/1556926808 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9a00e2460 msgr2=0x55e9a00edef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:10.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.157+0000 7fe6ee9f6740 1 --2- 172.21.15.83:0/1556926808 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9a00e2460 0x55e9a00edef0 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7fe6d4011810 tx=0x7fe6d40046c0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.157+0000 7fe6ee9f6740 1 -- 172.21.15.83:0/1556926808 shutdown_connections 2024-04-24T09:00:10.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.157+0000 7fe6ee9f6740 1 --2- 172.21.15.83:0/1556926808 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55e9a0162f40 0x55e9a0165360 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.157+0000 7fe6ee9f6740 1 --2- 172.21.15.83:0/1556926808 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e9a00ecb30 0x55e9a00f3d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.157+0000 7fe6ee9f6740 1 --2- 172.21.15.83:0/1556926808 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9a00e2460 0x55e9a00edef0 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.157+0000 7fe6ee9f6740 1 --2- 172.21.15.83:0/1556926808 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9a00e1b40 0x55e9a00ed9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.157+0000 7fe6c6a00640 0 client.18654 ms_handle_reset on v2:172.21.15.189:6838/2548965336 2024-04-24T09:00:10.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.157+0000 7fe6ee9f6740 1 -- 172.21.15.83:0/1556926808 >> 172.21.15.83:0/1556926808 conn(0x55e9a00dde60 msgr2=0x55e9a00dc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:10.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.157+0000 7fe6ee9f6740 1 -- 172.21.15.83:0/1556926808 shutdown_connections 2024-04-24T09:00:10.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.157+0000 7fe6ee9f6740 1 -- 172.21.15.83:0/1556926808 wait complete. 2024-04-24T09:00:10.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.157+0000 7fe6ee9f6740 20 client.18654 trim_cache size 0 max 16384 2024-04-24T09:00:10.160 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:00:10.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.158+0000 7fe6ea600640 1 -- 172.21.15.83:0/2318379990 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe6cc0a3d20 msgr2=0x7fe6cc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:10.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.158+0000 7fe6ea600640 1 --2- 172.21.15.83:0/2318379990 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe6cc0a3d20 0x7fe6cc0a61e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fe6d400c010 tx=0x7fe6d4005850 comp rx=0 tx=0).stop 2024-04-24T09:00:10.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.158+0000 7fe6ea600640 1 -- 172.21.15.83:0/2318379990 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6e410c480 msgr2=0x7fe6e410b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:10.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.158+0000 7fe6ea600640 1 --2- 172.21.15.83:0/2318379990 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6e410c480 0x7fe6e410b120 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc00eaa0 tx=0x7fe6dc00ef60 comp rx=0 tx=0).stop 2024-04-24T09:00:10.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.158+0000 7fe6ea600640 1 -- 172.21.15.83:0/2318379990 shutdown_connections 2024-04-24T09:00:10.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.158+0000 7fe6ea600640 1 --2- 172.21.15.83:0/2318379990 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe6cc0a3d20 0x7fe6cc0a61e0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.158+0000 7fe6ea600640 1 --2- 172.21.15.83:0/2318379990 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe6e410c480 0x7fe6e410b120 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.158+0000 7fe6ea600640 1 --2- 172.21.15.83:0/2318379990 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe6e4103980 0x7fe6e41a4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.158+0000 7fe6ea600640 1 --2- 172.21.15.83:0/2318379990 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe6e4103060 0x7fe6e41a4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.158+0000 7fe6ea600640 1 -- 172.21.15.83:0/2318379990 >> 172.21.15.83:0/2318379990 conn(0x7fe6e40fc880 msgr2=0x7fe6e4101ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:10.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.158+0000 7fe6ea600640 1 -- 172.21.15.83:0/2318379990 shutdown_connections 2024-04-24T09:00:10.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.158+0000 7fe6ea600640 1 -- 172.21.15.83:0/2318379990 wait complete. 2024-04-24T09:00:10.175 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-24T09:00:10.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.384+0000 7f3259e00640 1 -- 172.21.15.83:0/1345363121 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3254100e00 msgr2=0x7f32541011e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:10.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.384+0000 7f3259e00640 1 --2- 172.21.15.83:0/1345363121 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3254100e00 0x7f32541011e0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f3244009900 tx=0x7f324402a410 comp rx=0 tx=0).stop 2024-04-24T09:00:10.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.384+0000 7f3259e00640 1 -- 172.21.15.83:0/1345363121 shutdown_connections 2024-04-24T09:00:10.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.384+0000 7f3259e00640 1 --2- 172.21.15.83:0/1345363121 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f325410e9b0 0x7f3254110da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.384+0000 7f3259e00640 1 --2- 172.21.15.83:0/1345363121 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3254101720 0x7f325410e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.384+0000 7f3259e00640 1 --2- 172.21.15.83:0/1345363121 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3254100e00 0x7f32541011e0 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.384+0000 7f3259e00640 1 -- 172.21.15.83:0/1345363121 >> 172.21.15.83:0/1345363121 conn(0x7f3254078000 msgr2=0x7f3254078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:10.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.385+0000 7f3259e00640 1 -- 172.21.15.83:0/1345363121 shutdown_connections 2024-04-24T09:00:10.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.385+0000 7f3259e00640 1 -- 172.21.15.83:0/1345363121 wait complete. 2024-04-24T09:00:10.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.386+0000 7f3259e00640 1 Processor -- start 2024-04-24T09:00:10.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.386+0000 7f3259e00640 1 -- start start 2024-04-24T09:00:10.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.386+0000 7f3259e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3254100e00 0x7f32541a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:10.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.386+0000 7f3259e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3254101720 0x7f32541a0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:10.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.386+0000 7f3259e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f325410e9b0 0x7f325419a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:10.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.386+0000 7f3259e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f32541144c0 con 0x7f3254101720 2024-04-24T09:00:10.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.386+0000 7f3259e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3254114340 con 0x7f3254100e00 2024-04-24T09:00:10.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.386+0000 7f3259e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3254114640 con 0x7f325410e9b0 2024-04-24T09:00:10.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.386+0000 7f324a000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3254101720 0x7f32541a0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:10.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.387+0000 7f324a000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3254101720 0x7f32541a0c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48664/0 (socket says 172.21.15.83:48664) 2024-04-24T09:00:10.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.387+0000 7f324a000640 1 -- 172.21.15.83:0/383198371 learned_addr learned my addr 172.21.15.83:0/383198371 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:00:10.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.387+0000 7f3252a00640 1 --2- 172.21.15.83:0/383198371 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3254100e00 0x7f32541a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:10.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.387+0000 7f3253400640 1 --2- 172.21.15.83:0/383198371 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f325410e9b0 0x7f325419a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:10.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.387+0000 7f324a000640 1 -- 172.21.15.83:0/383198371 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f325410e9b0 msgr2=0x7f325419a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:10.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.387+0000 7f324a000640 1 --2- 172.21.15.83:0/383198371 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f325410e9b0 0x7f325419a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.387+0000 7f324a000640 1 -- 172.21.15.83:0/383198371 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3254100e00 msgr2=0x7f32541a06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:10.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.387+0000 7f324a000640 1 --2- 172.21.15.83:0/383198371 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3254100e00 0x7f32541a06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.387+0000 7f324a000640 1 -- 172.21.15.83:0/383198371 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f325419b050 con 0x7f3254101720 2024-04-24T09:00:10.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.387+0000 7f324a000640 1 --2- 172.21.15.83:0/383198371 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3254101720 0x7f32541a0c00 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f323c00d950 tx=0x7f323c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:10.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.388+0000 7f3258a00640 1 -- 172.21.15.83:0/383198371 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f323c017070 con 0x7f3254101720 2024-04-24T09:00:10.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.388+0000 7f3258a00640 1 -- 172.21.15.83:0/383198371 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f323c009c30 con 0x7f3254101720 2024-04-24T09:00:10.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.388+0000 7f3259e00640 1 -- 172.21.15.83:0/383198371 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f325419b340 con 0x7f3254101720 2024-04-24T09:00:10.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.388+0000 7f3258a00640 1 -- 172.21.15.83:0/383198371 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f323c00f620 con 0x7f3254101720 2024-04-24T09:00:10.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.388+0000 7f3259e00640 1 -- 172.21.15.83:0/383198371 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32541a7510 con 0x7f3254101720 2024-04-24T09:00:10.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.391+0000 7f3258a00640 1 -- 172.21.15.83:0/383198371 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f323c0109a0 con 0x7f3254101720 2024-04-24T09:00:10.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.392+0000 7f325cbcd740 1 Processor -- start 2024-04-24T09:00:10.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.392+0000 7f325cbcd740 1 -- start start 2024-04-24T09:00:10.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.392+0000 7f325cbcd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a07fc0950 0x563a07fc0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:10.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.392+0000 7f325cbcd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a07fc1270 0x563a07fcb400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:10.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.392+0000 7f325cbcd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563a07fcb940 0x563a07fcdd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:10.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.392+0000 7f325cbcd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x563a07f261d0 con 0x563a07fc0950 2024-04-24T09:00:10.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.392+0000 7f325cbcd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563a07ecc450 con 0x563a07fc1270 2024-04-24T09:00:10.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.392+0000 7f325cbcd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x563a07f22190 con 0x563a07fcb940 2024-04-24T09:00:10.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.393+0000 7f3253400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a07fc0950 0x563a07fc0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:10.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.393+0000 7f3253400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a07fc0950 0x563a07fc0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48670/0 (socket says 172.21.15.83:48670) 2024-04-24T09:00:10.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.393+0000 7f3253400640 1 -- 172.21.15.83:0/2263896127 learned_addr learned my addr 172.21.15.83:0/2263896127 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:00:10.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.393+0000 7f324a000640 1 --2- 172.21.15.83:0/2263896127 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563a07fcb940 0x563a07fcdd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:10.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.393+0000 7f3252a00640 1 --2- 172.21.15.83:0/2263896127 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a07fc1270 0x563a07fcb400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:10.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.393+0000 7f3253400640 1 -- 172.21.15.83:0/2263896127 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563a07fcb940 msgr2=0x563a07fcdd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:10.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.393+0000 7f3253400640 1 --2- 172.21.15.83:0/2263896127 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563a07fcb940 0x563a07fcdd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.393+0000 7f3253400640 1 -- 172.21.15.83:0/2263896127 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a07fc1270 msgr2=0x563a07fcb400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:10.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.393+0000 7f3253400640 1 --2- 172.21.15.83:0/2263896127 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a07fc1270 0x563a07fcb400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.393+0000 7f3253400640 1 -- 172.21.15.83:0/2263896127 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a07f89310 con 0x563a07fc0950 2024-04-24T09:00:10.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.393+0000 7f3258a00640 1 --2- 172.21.15.83:0/383198371 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f32380a3d20 0x7f32380a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:10.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.394+0000 7f3258a00640 1 -- 172.21.15.83:0/383198371 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f323c0d0970 con 0x7f3254101720 2024-04-24T09:00:10.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.394+0000 7f3253400640 1 --2- 172.21.15.83:0/2263896127 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a07fc0950 0x563a07fc0d30 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f324c009ef0 tx=0x7f324c00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5afc6a17777dd55 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:10.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.394+0000 7f3252a00640 1 --2- 172.21.15.83:0/383198371 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f32380a3d20 0x7f32380a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:10.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.394+0000 7f3243e00640 1 -- 172.21.15.83:0/2263896127 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f324c015070 con 0x563a07fc0950 2024-04-24T09:00:10.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.394+0000 7f3243e00640 1 -- 172.21.15.83:0/2263896127 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f324c01ab10 con 0x563a07fc0950 2024-04-24T09:00:10.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.394+0000 7f3243e00640 1 -- 172.21.15.83:0/2263896127 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f324c011bb0 con 0x563a07fc0950 2024-04-24T09:00:10.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.395+0000 7f325cbcd740 1 -- 172.21.15.83:0/2263896127 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a07fc0950 msgr2=0x563a07fc0d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:10.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.395+0000 7f3252a00640 1 --2- 172.21.15.83:0/383198371 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f32380a3d20 0x7f32380a61e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3244000c00 tx=0x7f3244005a20 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:10.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.395+0000 7f325cbcd740 1 --2- 172.21.15.83:0/2263896127 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a07fc0950 0x563a07fc0d30 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f324c009ef0 tx=0x7f324c00f730 comp rx=0 tx=0).stop 2024-04-24T09:00:10.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.395+0000 7f325cbcd740 1 -- 172.21.15.83:0/2263896127 shutdown_connections 2024-04-24T09:00:10.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.395+0000 7f325cbcd740 1 --2- 172.21.15.83:0/2263896127 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563a07fcb940 0x563a07fcdd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.395+0000 7f325cbcd740 1 --2- 172.21.15.83:0/2263896127 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a07fc1270 0x563a07fcb400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.395+0000 7f325cbcd740 1 --2- 172.21.15.83:0/2263896127 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a07fc0950 0x563a07fc0d30 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.395+0000 7f325cbcd740 1 -- 172.21.15.83:0/2263896127 >> 172.21.15.83:0/2263896127 conn(0x563a07fbcc70 msgr2=0x563a07fbd050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:10.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.395+0000 7f325cbcd740 1 -- 172.21.15.83:0/2263896127 shutdown_connections 2024-04-24T09:00:10.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.395+0000 7f325cbcd740 1 -- 172.21.15.83:0/2263896127 wait complete. 2024-04-24T09:00:10.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.396+0000 7f325cbcd740 1 -- start start 2024-04-24T09:00:10.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.396+0000 7f325cbcd740 1 Processor -- start 2024-04-24T09:00:10.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.396+0000 7f325cbcd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563a07fc0950 0x563a07fcb3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:10.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.396+0000 7f325cbcd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a07fc1270 0x563a07fc86c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:10.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.396+0000 7f325cbcd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a07fcb940 0x563a07fc8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:10.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.396+0000 7f325cbcd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x563a07fce7d0 con 0x563a07fcb940 2024-04-24T09:00:10.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.396+0000 7f325cbcd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563a07edf810 con 0x563a07fc1270 2024-04-24T09:00:10.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.396+0000 7f325cbcd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x563a07fce950 con 0x563a07fc0950 2024-04-24T09:00:10.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.397+0000 7f325cbcd740 10 client.0 _ll_register_callbacks cb 0x563a07ead140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:00:10.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.397+0000 7f3253400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a07fcb940 0x563a07fc8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:10.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.397+0000 7f3253400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a07fcb940 0x563a07fc8db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48676/0 (socket says 172.21.15.83:48676) 2024-04-24T09:00:10.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.397+0000 7f3253400640 1 -- 172.21.15.83:0/265950893 learned_addr learned my addr 172.21.15.83:0/265950893 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:00:10.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.397+0000 7f324a000640 1 --2- 172.21.15.83:0/265950893 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a07fc1270 0x563a07fc86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:10.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.397+0000 7f3252a00640 1 --2- 172.21.15.83:0/265950893 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563a07fc0950 0x563a07fcb3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:10.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.397+0000 7f3253400640 1 -- 172.21.15.83:0/265950893 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563a07fc0950 msgr2=0x563a07fcb3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:10.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.397+0000 7f3253400640 1 --2- 172.21.15.83:0/265950893 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563a07fc0950 0x563a07fcb3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.397+0000 7f3253400640 1 -- 172.21.15.83:0/265950893 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a07fc1270 msgr2=0x563a07fc86c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:10.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.397+0000 7f3253400640 1 --2- 172.21.15.83:0/265950893 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a07fc1270 0x563a07fc86c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.397+0000 7f3253400640 1 -- 172.21.15.83:0/265950893 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a07ef03f0 con 0x563a07fcb940 2024-04-24T09:00:10.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.397+0000 7f324a000640 1 --2- 172.21.15.83:0/265950893 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a07fc1270 0x563a07fc86c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:00:10.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.397+0000 7f3252a00640 1 --2- 172.21.15.83:0/265950893 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563a07fc0950 0x563a07fcb3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:00:10.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.397+0000 7f3253400640 1 --2- 172.21.15.83:0/265950893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a07fcb940 0x563a07fc8db0 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f324c000f80 tx=0x7f324c009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a68fb1cdad8845f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:10.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.397+0000 7f3240c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:00:10.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.398+0000 7f3240c00640 1 -- 172.21.15.83:0/265950893 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f324c015040 con 0x563a07fcb940 2024-04-24T09:00:10.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.398+0000 7f3240c00640 1 -- 172.21.15.83:0/265950893 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f324c004c40 con 0x563a07fcb940 2024-04-24T09:00:10.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.398+0000 7f325cbcd740 1 -- 172.21.15.83:0/265950893 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f324c01ab10 con 0x563a07fcb940 2024-04-24T09:00:10.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.398+0000 7f3240c00640 1 -- 172.21.15.83:0/265950893 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f324c030050 con 0x563a07fcb940 2024-04-24T09:00:10.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.398+0000 7f325cbcd740 10 client.18666 fetch_fsmap learned FSMap version 35 2024-04-24T09:00:10.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.398+0000 7f325cbcd740 1 -- 172.21.15.83:0/265950893 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x563a07c383f0 con 0x563a07fcb940 2024-04-24T09:00:10.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.398+0000 7f3240c00640 1 -- 172.21.15.83:0/265950893 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f324c01ab10 con 0x563a07fcb940 2024-04-24T09:00:10.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.399+0000 7f3240c00640 1 -- 172.21.15.83:0/265950893 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7f324c02b030 con 0x563a07fcb940 2024-04-24T09:00:10.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.399+0000 7f325cbcd740 10 client.18666 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T09:00:10.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.399+0000 7f325cbcd740 10 client.18666 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-24T09:00:10.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.399+0000 7f325cbcd740 20 client.18666 populate_metadata read hostname 'smithi083' 2024-04-24T09:00:10.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.399+0000 7f325cbcd740 1 --2- 172.21.15.83:0/265950893 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x563a07fdbb20 0x563a07fddf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:10.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.399+0000 7f325cbcd740 4 client.18666 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:00:10.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.399+0000 7f325cbcd740 1 -- 172.21.15.83:0/265950893 --> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563a07f33bf0 con 0x563a07fdbb20 2024-04-24T09:00:10.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.400+0000 7f3252a00640 1 --2- 172.21.15.83:0/265950893 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x563a07fdbb20 0x563a07fddf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:10.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.401+0000 7f3252a00640 1 --2- 172.21.15.83:0/265950893 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x563a07fdbb20 0x563a07fddf10 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=423900264f8ec79 server_cookie=6aed9855fda3c0c9 in_seq=0 out_seq=0 2024-04-24T09:00:10.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.401+0000 7f3240c00640 10 client.18666 ms_handle_connect on v2:172.21.15.138:6832/1157967658 2024-04-24T09:00:10.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.405+0000 7f3240c00640 1 -- 172.21.15.83:0/265950893 <== mds.3 v2:172.21.15.138:6832/1157967658 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563a07f33bf0 con 0x563a07fdbb20 2024-04-24T09:00:10.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.405+0000 7f3240c00640 10 client.18666 handle_command_reply: tid=0 2024-04-24T09:00:10.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.405+0000 7f325cbcd740 1 client.18666 shutdown 2024-04-24T09:00:10.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.405+0000 7f325cbcd740 1 -- 172.21.15.83:0/265950893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a07fcb940 msgr2=0x563a07fc8db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:10.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.405+0000 7f325cbcd740 1 --2- 172.21.15.83:0/265950893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a07fcb940 0x563a07fc8db0 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f324c000f80 tx=0x7f324c009c30 comp rx=0 tx=0).stop 2024-04-24T09:00:10.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.405+0000 7f325cbcd740 1 -- 172.21.15.83:0/265950893 shutdown_connections 2024-04-24T09:00:10.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.405+0000 7f325cbcd740 1 --2- 172.21.15.83:0/265950893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a07fcb940 0x563a07fc8db0 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.405+0000 7f325cbcd740 1 --2- 172.21.15.83:0/265950893 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a07fc1270 0x563a07fc86c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.405+0000 7f325cbcd740 1 --2- 172.21.15.83:0/265950893 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x563a07fdbb20 0x563a07fddf10 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.405+0000 7f325cbcd740 1 --2- 172.21.15.83:0/265950893 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563a07fc0950 0x563a07fcb3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.405+0000 7f325cbcd740 1 -- 172.21.15.83:0/265950893 >> 172.21.15.83:0/265950893 conn(0x563a07fbcc70 msgr2=0x563a07fc2970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:10.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.405+0000 7f3240c00640 0 client.18666 ms_handle_reset on v2:172.21.15.138:6832/1157967658 2024-04-24T09:00:10.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.405+0000 7f325cbcd740 1 -- 172.21.15.83:0/265950893 shutdown_connections 2024-04-24T09:00:10.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.405+0000 7f325cbcd740 1 -- 172.21.15.83:0/265950893 wait complete. 2024-04-24T09:00:10.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.406+0000 7f325cbcd740 20 client.18666 trim_cache size 0 max 16384 2024-04-24T09:00:10.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.421+0000 7f325cbcd740 1 Processor -- start 2024-04-24T09:00:10.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.421+0000 7f325cbcd740 1 -- start start 2024-04-24T09:00:10.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.421+0000 7f325cbcd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563a07fc0950 0x563a07fbc570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:10.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.421+0000 7f325cbcd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a07fc1270 0x563a07fcaf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:10.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.421+0000 7f325cbcd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a07fcb940 0x563a07fe3510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:10.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.421+0000 7f325cbcd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f32541b1830 con 0x563a07fcb940 2024-04-24T09:00:10.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.421+0000 7f325cbcd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f32541b3370 con 0x563a07fc1270 2024-04-24T09:00:10.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.421+0000 7f325cbcd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f32541b1f00 con 0x563a07fc0950 2024-04-24T09:00:10.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.421+0000 7f3253400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a07fcb940 0x563a07fe3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:10.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.422+0000 7f3253400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a07fcb940 0x563a07fe3510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48680/0 (socket says 172.21.15.83:48680) 2024-04-24T09:00:10.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.422+0000 7f3253400640 1 -- 172.21.15.83:0/2345413118 learned_addr learned my addr 172.21.15.83:0/2345413118 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:00:10.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.422+0000 7f3252a00640 1 --2- 172.21.15.83:0/2345413118 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563a07fc0950 0x563a07fbc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:10.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.422+0000 7f324a000640 1 --2- 172.21.15.83:0/2345413118 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a07fc1270 0x563a07fcaf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:10.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.422+0000 7f3253400640 1 -- 172.21.15.83:0/2345413118 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563a07fc0950 msgr2=0x563a07fbc570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:10.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.422+0000 7f3253400640 1 --2- 172.21.15.83:0/2345413118 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563a07fc0950 0x563a07fbc570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.422+0000 7f3253400640 1 -- 172.21.15.83:0/2345413118 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a07fc1270 msgr2=0x563a07fcaf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:10.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.422+0000 7f3253400640 1 --2- 172.21.15.83:0/2345413118 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a07fc1270 0x563a07fcaf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.422+0000 7f3253400640 1 -- 172.21.15.83:0/2345413118 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a07d027d0 con 0x563a07fcb940 2024-04-24T09:00:10.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.422+0000 7f3252a00640 1 --2- 172.21.15.83:0/2345413118 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563a07fc0950 0x563a07fbc570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:00:10.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.422+0000 7f324a000640 1 --2- 172.21.15.83:0/2345413118 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a07fc1270 0x563a07fcaf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:00:10.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.422+0000 7f3253400640 1 --2- 172.21.15.83:0/2345413118 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a07fcb940 0x563a07fe3510 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f324c005cc0 tx=0x7f324c009df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7710acfbf358b738 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:10.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.422+0000 7f3243400640 1 -- 172.21.15.83:0/2345413118 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f324c039070 con 0x563a07fcb940 2024-04-24T09:00:10.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.422+0000 7f3243400640 1 -- 172.21.15.83:0/2345413118 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f324c011d70 con 0x563a07fcb940 2024-04-24T09:00:10.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.423+0000 7f3243400640 1 -- 172.21.15.83:0/2345413118 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f324c015070 con 0x563a07fcb940 2024-04-24T09:00:10.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.423+0000 7f325cbcd740 1 -- 172.21.15.83:0/2345413118 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a07fcb940 msgr2=0x563a07fe3510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:10.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.423+0000 7f325cbcd740 1 --2- 172.21.15.83:0/2345413118 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a07fcb940 0x563a07fe3510 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f324c005cc0 tx=0x7f324c009df0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.423+0000 7f325cbcd740 1 -- 172.21.15.83:0/2345413118 shutdown_connections 2024-04-24T09:00:10.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.423+0000 7f325cbcd740 1 --2- 172.21.15.83:0/2345413118 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a07fcb940 0x563a07fe3510 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.423+0000 7f325cbcd740 1 --2- 172.21.15.83:0/2345413118 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a07fc1270 0x563a07fcaf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.423+0000 7f325cbcd740 1 --2- 172.21.15.83:0/2345413118 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563a07fc0950 0x563a07fbc570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.423+0000 7f325cbcd740 1 -- 172.21.15.83:0/2345413118 >> 172.21.15.83:0/2345413118 conn(0x563a07fbcc70 msgr2=0x563a07fc6fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:10.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.424+0000 7f325cbcd740 1 -- 172.21.15.83:0/2345413118 shutdown_connections 2024-04-24T09:00:10.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.424+0000 7f325cbcd740 1 -- 172.21.15.83:0/2345413118 wait complete. 2024-04-24T09:00:10.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.424+0000 7f325cbcd740 1 -- start start 2024-04-24T09:00:10.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.424+0000 7f325cbcd740 1 Processor -- start 2024-04-24T09:00:10.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.425+0000 7f325cbcd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a07fc0950 0x563a07fd3970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:10.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.425+0000 7f325cbcd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563a07fc1270 0x563a07fd3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:10.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.425+0000 7f325cbcd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a07fcb940 0x563a07fc5960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:10.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.425+0000 7f325cbcd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f32541b2ca0 con 0x563a07fc0950 2024-04-24T09:00:10.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.425+0000 7f325cbcd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f32541b25d0 con 0x563a07fcb940 2024-04-24T09:00:10.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.425+0000 7f325cbcd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f32541af830 con 0x563a07fc1270 2024-04-24T09:00:10.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.425+0000 7f325cbcd740 10 client.0 _ll_register_callbacks cb 0x7f324c009d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:00:10.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.425+0000 7f3252a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a07fc0950 0x563a07fd3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:10.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.425+0000 7f3252a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a07fc0950 0x563a07fd3970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48694/0 (socket says 172.21.15.83:48694) 2024-04-24T09:00:10.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.425+0000 7f3252a00640 1 -- 172.21.15.83:0/1542888766 learned_addr learned my addr 172.21.15.83:0/1542888766 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:00:10.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.425+0000 7f3253400640 1 --2- 172.21.15.83:0/1542888766 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a07fcb940 0x563a07fc5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:10.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.425+0000 7f3252a00640 1 -- 172.21.15.83:0/1542888766 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563a07fc1270 msgr2=0x563a07fd3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:10.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.425+0000 7f3252a00640 1 --2- 172.21.15.83:0/1542888766 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563a07fc1270 0x563a07fd3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.425+0000 7f3252a00640 1 -- 172.21.15.83:0/1542888766 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a07fcb940 msgr2=0x563a07fc5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:10.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.425+0000 7f3252a00640 1 --2- 172.21.15.83:0/1542888766 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a07fcb940 0x563a07fc5960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.425+0000 7f3252a00640 1 -- 172.21.15.83:0/1542888766 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a07fc5fa0 con 0x563a07fc0950 2024-04-24T09:00:10.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.426+0000 7f3252a00640 1 --2- 172.21.15.83:0/1542888766 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a07fc0950 0x563a07fd3970 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f3244031440 tx=0x7f3244009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbeab84f6e2dca05 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:10.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.426+0000 7f3240c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:00:10.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.426+0000 7f3240c00640 1 -- 172.21.15.83:0/1542888766 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3244035050 con 0x563a07fc0950 2024-04-24T09:00:10.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.426+0000 7f325cbcd740 1 -- 172.21.15.83:0/1542888766 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f324c011d70 con 0x563a07fc0950 2024-04-24T09:00:10.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.426+0000 7f3240c00640 1 -- 172.21.15.83:0/1542888766 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f324404f1d0 con 0x563a07fc0950 2024-04-24T09:00:10.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.426+0000 7f3240c00640 1 -- 172.21.15.83:0/1542888766 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f324405f050 con 0x563a07fc0950 2024-04-24T09:00:10.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.426+0000 7f325cbcd740 10 client.18672 fetch_fsmap learned FSMap version 35 2024-04-24T09:00:10.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.426+0000 7f325cbcd740 1 -- 172.21.15.83:0/1542888766 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x563a07fcc970 con 0x563a07fc0950 2024-04-24T09:00:10.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.427+0000 7f3240c00640 1 -- 172.21.15.83:0/1542888766 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f324c011d70 con 0x563a07fc0950 2024-04-24T09:00:10.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.427+0000 7f3240c00640 1 -- 172.21.15.83:0/1542888766 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7f3244003a70 con 0x563a07fc0950 2024-04-24T09:00:10.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.427+0000 7f325cbcd740 10 client.18672 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T09:00:10.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.427+0000 7f325cbcd740 10 client.18672 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-24T09:00:10.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.427+0000 7f325cbcd740 20 client.18672 populate_metadata read hostname 'smithi083' 2024-04-24T09:00:10.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.427+0000 7f325cbcd740 1 --2- 172.21.15.83:0/1542888766 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x563a08041d80 0x563a080441b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:10.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.427+0000 7f325cbcd740 4 client.18672 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:00:10.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.427+0000 7f325cbcd740 1 -- 172.21.15.83:0/1542888766 --> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563a07f2da60 con 0x563a08041d80 2024-04-24T09:00:10.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.427+0000 7f324a000640 1 --2- 172.21.15.83:0/1542888766 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x563a08041d80 0x563a080441b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:10.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.428+0000 7f324a000640 1 --2- 172.21.15.83:0/1542888766 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x563a08041d80 0x563a080441b0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e52cf91b427c9004 server_cookie=edd0bb548e362a00 in_seq=0 out_seq=0 2024-04-24T09:00:10.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.428+0000 7f3240c00640 10 client.18672 ms_handle_connect on v2:172.21.15.138:6832/1157967658 2024-04-24T09:00:10.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.429+0000 7f3240c00640 1 -- 172.21.15.83:0/1542888766 <== mds.3 v2:172.21.15.138:6832/1157967658 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563a07f2da60 con 0x563a08041d80 2024-04-24T09:00:10.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.429+0000 7f3240c00640 10 client.18672 handle_command_reply: tid=0 2024-04-24T09:00:10.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.429+0000 7f325cbcd740 1 client.18672 shutdown 2024-04-24T09:00:10.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.429+0000 7f325cbcd740 1 -- 172.21.15.83:0/1542888766 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a07fc0950 msgr2=0x563a07fd3970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:10.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.429+0000 7f325cbcd740 1 --2- 172.21.15.83:0/1542888766 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a07fc0950 0x563a07fd3970 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f3244031440 tx=0x7f3244009260 comp rx=0 tx=0).stop 2024-04-24T09:00:10.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.429+0000 7f325cbcd740 1 -- 172.21.15.83:0/1542888766 shutdown_connections 2024-04-24T09:00:10.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.429+0000 7f325cbcd740 1 --2- 172.21.15.83:0/1542888766 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a07fcb940 0x563a07fc5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.429+0000 7f325cbcd740 1 --2- 172.21.15.83:0/1542888766 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x563a08041d80 0x563a080441b0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.429+0000 7f325cbcd740 1 --2- 172.21.15.83:0/1542888766 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563a07fc1270 0x563a07fd3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.429+0000 7f3240c00640 0 client.18672 ms_handle_reset on v2:172.21.15.138:6832/1157967658 2024-04-24T09:00:10.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.429+0000 7f325cbcd740 1 --2- 172.21.15.83:0/1542888766 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a07fc0950 0x563a07fd3970 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.429+0000 7f325cbcd740 1 -- 172.21.15.83:0/1542888766 >> 172.21.15.83:0/1542888766 conn(0x563a07fbcc70 msgr2=0x563a07fc20e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:10.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.429+0000 7f325cbcd740 1 -- 172.21.15.83:0/1542888766 shutdown_connections 2024-04-24T09:00:10.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.429+0000 7f325cbcd740 1 -- 172.21.15.83:0/1542888766 wait complete. 2024-04-24T09:00:10.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.429+0000 7f325cbcd740 20 client.18672 trim_cache size 0 max 16384 2024-04-24T09:00:10.431 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:00:10.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.430+0000 7f3259e00640 1 -- 172.21.15.83:0/383198371 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f32380a3d20 msgr2=0x7f32380a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:10.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.430+0000 7f3259e00640 1 --2- 172.21.15.83:0/383198371 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f32380a3d20 0x7f32380a61e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3244000c00 tx=0x7f3244005a20 comp rx=0 tx=0).stop 2024-04-24T09:00:10.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.430+0000 7f3259e00640 1 -- 172.21.15.83:0/383198371 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3254101720 msgr2=0x7f32541a0c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:10.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.430+0000 7f3259e00640 1 --2- 172.21.15.83:0/383198371 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3254101720 0x7f32541a0c00 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f323c00d950 tx=0x7f323c00de10 comp rx=0 tx=0).stop 2024-04-24T09:00:10.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.430+0000 7f3259e00640 1 -- 172.21.15.83:0/383198371 shutdown_connections 2024-04-24T09:00:10.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.430+0000 7f3259e00640 1 --2- 172.21.15.83:0/383198371 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f32380a3d20 0x7f32380a61e0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.430+0000 7f3259e00640 1 --2- 172.21.15.83:0/383198371 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f325410e9b0 0x7f325419a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.430+0000 7f3259e00640 1 --2- 172.21.15.83:0/383198371 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3254101720 0x7f32541a0c00 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.430+0000 7f3259e00640 1 --2- 172.21.15.83:0/383198371 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3254100e00 0x7f32541a06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.430+0000 7f3259e00640 1 -- 172.21.15.83:0/383198371 >> 172.21.15.83:0/383198371 conn(0x7f3254078000 msgr2=0x7f3254110090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:10.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.430+0000 7f3259e00640 1 -- 172.21.15.83:0/383198371 shutdown_connections 2024-04-24T09:00:10.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.430+0000 7f3259e00640 1 -- 172.21.15.83:0/383198371 wait complete. 2024-04-24T09:00:10.447 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T09:00:10.598 INFO:tasks.workunit.client.0.smithi083.stdout: 1 48056 0.63 MB/sec execute 284 sec latency 56.509 ms 2024-04-24T09:00:10.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.661+0000 7f4ec3200640 1 -- 172.21.15.83:0/2280812884 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ec4104cd0 msgr2=0x7f4ec4108d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:10.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.661+0000 7f4ec3200640 1 --2- 172.21.15.83:0/2280812884 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ec4104cd0 0x7f4ec4108d10 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f4eac009900 tx=0x7f4eac02a3f0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.661+0000 7f4ec3200640 1 -- 172.21.15.83:0/2280812884 shutdown_connections 2024-04-24T09:00:10.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.661+0000 7f4ec3200640 1 --2- 172.21.15.83:0/2280812884 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ec41093f0 0x7f4ec4110fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.661+0000 7f4ec3200640 1 --2- 172.21.15.83:0/2280812884 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ec4104cd0 0x7f4ec4108d10 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.661+0000 7f4ec3200640 1 --2- 172.21.15.83:0/2280812884 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4ec4104320 0x7f4ec4104700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.661+0000 7f4ec3200640 1 -- 172.21.15.83:0/2280812884 >> 172.21.15.83:0/2280812884 conn(0x7f4ec40ffb40 msgr2=0x7f4ec4101f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:10.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.661+0000 7f4ec3200640 1 -- 172.21.15.83:0/2280812884 shutdown_connections 2024-04-24T09:00:10.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.661+0000 7f4ec3200640 1 -- 172.21.15.83:0/2280812884 wait complete. 2024-04-24T09:00:10.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.662+0000 7f4ec3200640 1 Processor -- start 2024-04-24T09:00:10.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.662+0000 7f4ec3200640 1 -- start start 2024-04-24T09:00:10.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.663+0000 7f4ec3200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ec4104320 0x7f4ec41a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:10.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.663+0000 7f4ec3200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4ec41093f0 0x7f4ec41a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:10.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.663+0000 7f4ec3200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ec419a6e0 0x7f4ec419ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:10.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.663+0000 7f4ec3200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec4114360 con 0x7f4ec419a6e0 2024-04-24T09:00:10.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.663+0000 7f4ec3200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec41141e0 con 0x7f4ec4104320 2024-04-24T09:00:10.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.663+0000 7f4ec3200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec41144e0 con 0x7f4ec41093f0 2024-04-24T09:00:10.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.663+0000 7f4ec2800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ec419a6e0 0x7f4ec419ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:10.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.663+0000 7f4ec2800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ec419a6e0 0x7f4ec419ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48716/0 (socket says 172.21.15.83:48716) 2024-04-24T09:00:10.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.663+0000 7f4ec2800640 1 -- 172.21.15.83:0/4277181396 learned_addr learned my addr 172.21.15.83:0/4277181396 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:00:10.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.663+0000 7f4ec1e00640 1 --2- 172.21.15.83:0/4277181396 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ec4104320 0x7f4ec41a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:10.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.663+0000 7f4eb9400640 1 --2- 172.21.15.83:0/4277181396 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4ec41093f0 0x7f4ec41a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:10.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.664+0000 7f4ec2800640 1 -- 172.21.15.83:0/4277181396 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4ec41093f0 msgr2=0x7f4ec41a0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:10.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.664+0000 7f4ec2800640 1 --2- 172.21.15.83:0/4277181396 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4ec41093f0 0x7f4ec41a0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.664+0000 7f4ec2800640 1 -- 172.21.15.83:0/4277181396 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ec4104320 msgr2=0x7f4ec41a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:10.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.664+0000 7f4ec2800640 1 --2- 172.21.15.83:0/4277181396 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ec4104320 0x7f4ec41a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.664+0000 7f4ec2800640 1 -- 172.21.15.83:0/4277181396 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ec419b2e0 con 0x7f4ec419a6e0 2024-04-24T09:00:10.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.664+0000 7f4eb9400640 1 --2- 172.21.15.83:0/4277181396 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4ec41093f0 0x7f4ec41a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:00:10.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.664+0000 7f4ec2800640 1 --2- 172.21.15.83:0/4277181396 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ec419a6e0 0x7f4ec419ab60 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc00cce0 tx=0x7f4ebc00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:10.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.664+0000 7f4ebb400640 1 -- 172.21.15.83:0/4277181396 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ebc00f450 con 0x7f4ec419a6e0 2024-04-24T09:00:10.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.664+0000 7f4ebb400640 1 -- 172.21.15.83:0/4277181396 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4ebc00faa0 con 0x7f4ec419a6e0 2024-04-24T09:00:10.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.664+0000 7f4ec3200640 1 -- 172.21.15.83:0/4277181396 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ec41a73c0 con 0x7f4ec419a6e0 2024-04-24T09:00:10.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.664+0000 7f4ebb400640 1 -- 172.21.15.83:0/4277181396 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ebc018b70 con 0x7f4ec419a6e0 2024-04-24T09:00:10.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.664+0000 7f4ec3200640 1 -- 172.21.15.83:0/4277181396 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ec41a7770 con 0x7f4ec419a6e0 2024-04-24T09:00:10.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.668+0000 7f4ecc1dc740 1 Processor -- start 2024-04-24T09:00:10.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.669+0000 7f4ebb400640 1 -- 172.21.15.83:0/4277181396 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4ebc018d10 con 0x7f4ec419a6e0 2024-04-24T09:00:10.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.669+0000 7f4ecc1dc740 1 -- start start 2024-04-24T09:00:10.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.669+0000 7f4ecc1dc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622a74ddb40 0x5622a74ddf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:10.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.669+0000 7f4ecc1dc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622a74de460 0x5622a74e85f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:10.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.669+0000 7f4ecc1dc740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622a74e8b30 0x5622a74eaf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:10.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.669+0000 7f4ecc1dc740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5622a74431d0 con 0x5622a74de460 2024-04-24T09:00:10.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.669+0000 7f4ecc1dc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5622a73e9450 con 0x5622a74ddb40 2024-04-24T09:00:10.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.669+0000 7f4ecc1dc740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5622a743f190 con 0x5622a74e8b30 2024-04-24T09:00:10.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.669+0000 7f4ec1e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622a74de460 0x5622a74e85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:10.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.670+0000 7f4ec1e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622a74de460 0x5622a74e85f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48728/0 (socket says 172.21.15.83:48728) 2024-04-24T09:00:10.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.670+0000 7f4ec1e00640 1 -- 172.21.15.83:0/847845062 learned_addr learned my addr 172.21.15.83:0/847845062 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:00:10.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.670+0000 7f4eb9400640 1 --2- 172.21.15.83:0/847845062 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622a74ddb40 0x5622a74ddf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:10.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.670+0000 7f4ec1e00640 1 -- 172.21.15.83:0/847845062 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622a74e8b30 msgr2=0x5622a74eaf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:10.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.670+0000 7f4ec1e00640 1 --2- 172.21.15.83:0/847845062 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622a74e8b30 0x5622a74eaf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.670+0000 7f4ec1e00640 1 -- 172.21.15.83:0/847845062 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622a74ddb40 msgr2=0x5622a74ddf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:10.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.670+0000 7f4ec1e00640 1 --2- 172.21.15.83:0/847845062 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622a74ddb40 0x5622a74ddf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.670+0000 7f4ec1e00640 1 -- 172.21.15.83:0/847845062 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622a74a6310 con 0x5622a74de460 2024-04-24T09:00:10.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.670+0000 7f4ec1e00640 1 --2- 172.21.15.83:0/847845062 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622a74de460 0x5622a74e85f0 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f4eb400bd10 tx=0x7f4eb40107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29db2b57c088fc06 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:10.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.671+0000 7f4ebb400640 1 --2- 172.21.15.83:0/4277181396 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f4ea80a3da0 0x7f4ea80a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:10.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.671+0000 7f4eb3400640 1 -- 172.21.15.83:0/847845062 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb4004030 con 0x5622a74de460 2024-04-24T09:00:10.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.671+0000 7f4ebb400640 1 -- 172.21.15.83:0/4277181396 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f4ebc0d0f90 con 0x7f4ec419a6e0 2024-04-24T09:00:10.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.671+0000 7f4eb3400640 1 -- 172.21.15.83:0/847845062 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4eb40035f0 con 0x5622a74de460 2024-04-24T09:00:10.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.671+0000 7f4eb3400640 1 -- 172.21.15.83:0/847845062 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb4011770 con 0x5622a74de460 2024-04-24T09:00:10.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.671+0000 7f4eb9400640 1 --2- 172.21.15.83:0/4277181396 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f4ea80a3da0 0x7f4ea80a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:10.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.672+0000 7f4ecc1dc740 1 -- 172.21.15.83:0/847845062 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622a74de460 msgr2=0x5622a74e85f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:10.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.672+0000 7f4eb9400640 1 --2- 172.21.15.83:0/4277181396 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f4ea80a3da0 0x7f4ea80a6260 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f4eac006010 tx=0x7f4eac000bc0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:10.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.672+0000 7f4ecc1dc740 1 --2- 172.21.15.83:0/847845062 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622a74de460 0x5622a74e85f0 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f4eb400bd10 tx=0x7f4eb40107a0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.672+0000 7f4ecc1dc740 1 -- 172.21.15.83:0/847845062 shutdown_connections 2024-04-24T09:00:10.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.672+0000 7f4ecc1dc740 1 --2- 172.21.15.83:0/847845062 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622a74e8b30 0x5622a74eaf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.672+0000 7f4ecc1dc740 1 --2- 172.21.15.83:0/847845062 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622a74de460 0x5622a74e85f0 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.672+0000 7f4ecc1dc740 1 --2- 172.21.15.83:0/847845062 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622a74ddb40 0x5622a74ddf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.672+0000 7f4ecc1dc740 1 -- 172.21.15.83:0/847845062 >> 172.21.15.83:0/847845062 conn(0x5622a74d9e60 msgr2=0x5622a74da240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:10.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.672+0000 7f4ecc1dc740 1 -- 172.21.15.83:0/847845062 shutdown_connections 2024-04-24T09:00:10.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.672+0000 7f4ecc1dc740 1 -- 172.21.15.83:0/847845062 wait complete. 2024-04-24T09:00:10.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.673+0000 7f4ecc1dc740 1 -- start start 2024-04-24T09:00:10.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.673+0000 7f4ecc1dc740 1 Processor -- start 2024-04-24T09:00:10.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.673+0000 7f4ecc1dc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622a74ddb40 0x5622a74e85c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:10.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.673+0000 7f4ecc1dc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622a74de460 0x5622a74e58b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:10.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.673+0000 7f4ecc1dc740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622a74e8b30 0x5622a74e5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:10.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.673+0000 7f4ecc1dc740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5622a74eb9c0 con 0x5622a74ddb40 2024-04-24T09:00:10.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.673+0000 7f4ecc1dc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5622a73fc810 con 0x5622a74de460 2024-04-24T09:00:10.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.673+0000 7f4ecc1dc740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5622a74ebb40 con 0x5622a74e8b30 2024-04-24T09:00:10.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.673+0000 7f4ecc1dc740 10 client.0 _ll_register_callbacks cb 0x5622a73ca140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:00:10.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.674+0000 7f4ec1e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622a74ddb40 0x5622a74e85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:10.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.674+0000 7f4ec1e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622a74ddb40 0x5622a74e85c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48744/0 (socket says 172.21.15.83:48744) 2024-04-24T09:00:10.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.674+0000 7f4ec1e00640 1 -- 172.21.15.83:0/4101474918 learned_addr learned my addr 172.21.15.83:0/4101474918 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:00:10.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.674+0000 7f4eb9400640 1 --2- 172.21.15.83:0/4101474918 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622a74de460 0x5622a74e58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:10.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.674+0000 7f4ec2800640 1 --2- 172.21.15.83:0/4101474918 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622a74e8b30 0x5622a74e5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:10.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.674+0000 7f4ec1e00640 1 -- 172.21.15.83:0/4101474918 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622a74e8b30 msgr2=0x5622a74e5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:10.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.674+0000 7f4ec1e00640 1 --2- 172.21.15.83:0/4101474918 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622a74e8b30 0x5622a74e5fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.674+0000 7f4ec1e00640 1 -- 172.21.15.83:0/4101474918 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622a74de460 msgr2=0x5622a74e58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:10.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.674+0000 7f4ec1e00640 1 --2- 172.21.15.83:0/4101474918 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622a74de460 0x5622a74e58b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.674+0000 7f4ec1e00640 1 -- 172.21.15.83:0/4101474918 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622a740d3f0 con 0x5622a74ddb40 2024-04-24T09:00:10.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.674+0000 7f4ec1e00640 1 --2- 172.21.15.83:0/4101474918 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622a74ddb40 0x5622a74e85c0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f4eb4005cd0 tx=0x7f4eb400b950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5160696d5bd2dc39 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:10.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.675+0000 7f4ea7e00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:00:10.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.675+0000 7f4ea7e00640 1 -- 172.21.15.83:0/4101474918 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb4029060 con 0x5622a74ddb40 2024-04-24T09:00:10.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.675+0000 7f4ea7e00640 1 -- 172.21.15.83:0/4101474918 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f4eb4003a30 con 0x5622a74ddb40 2024-04-24T09:00:10.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.675+0000 7f4ecc1dc740 1 -- 172.21.15.83:0/4101474918 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4eb40035f0 con 0x5622a74ddb40 2024-04-24T09:00:10.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.675+0000 7f4ea7e00640 1 -- 172.21.15.83:0/4101474918 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f4eb4030050 con 0x5622a74ddb40 2024-04-24T09:00:10.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.675+0000 7f4ecc1dc740 10 client.18684 fetch_fsmap learned FSMap version 35 2024-04-24T09:00:10.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.675+0000 7f4ecc1dc740 1 -- 172.21.15.83:0/4101474918 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5622a71553f0 con 0x5622a74ddb40 2024-04-24T09:00:10.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.675+0000 7f4ea7e00640 1 -- 172.21.15.83:0/4101474918 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4eb40035f0 con 0x5622a74ddb40 2024-04-24T09:00:10.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.676+0000 7f4ea7e00640 1 -- 172.21.15.83:0/4101474918 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7f4eb40046e0 con 0x5622a74ddb40 2024-04-24T09:00:10.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.676+0000 7f4ecc1dc740 10 client.18684 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T09:00:10.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.676+0000 7f4ecc1dc740 10 client.18684 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-24T09:00:10.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.676+0000 7f4ecc1dc740 20 client.18684 populate_metadata read hostname 'smithi083' 2024-04-24T09:00:10.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.676+0000 7f4ecc1dc740 1 --2- 172.21.15.83:0/4101474918 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x5622a74f8c80 0x5622a74fb070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:10.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.676+0000 7f4ecc1dc740 4 client.18684 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:00:10.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.676+0000 7f4ecc1dc740 1 -- 172.21.15.83:0/4101474918 --> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5622a7450bf0 con 0x5622a74f8c80 2024-04-24T09:00:10.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.677+0000 7f4eb9400640 1 --2- 172.21.15.83:0/4101474918 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x5622a74f8c80 0x5622a74fb070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:10.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.678+0000 7f4eb9400640 1 --2- 172.21.15.83:0/4101474918 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x5622a74f8c80 0x5622a74fb070 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b4ff5380798bab7e server_cookie=f25ee3c11dfd6389 in_seq=0 out_seq=0 2024-04-24T09:00:10.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.678+0000 7f4ea7e00640 10 client.18684 ms_handle_connect on v2:172.21.15.138:6834/1909219758 2024-04-24T09:00:10.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.680+0000 7f4ea7e00640 1 -- 172.21.15.83:0/4101474918 <== mds.2 v2:172.21.15.138:6834/1909219758 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5622a7450bf0 con 0x5622a74f8c80 2024-04-24T09:00:10.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.680+0000 7f4ea7e00640 10 client.18684 handle_command_reply: tid=0 2024-04-24T09:00:10.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.681+0000 7f4ecc1dc740 1 client.18684 shutdown 2024-04-24T09:00:10.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.681+0000 7f4ecc1dc740 1 -- 172.21.15.83:0/4101474918 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622a74ddb40 msgr2=0x5622a74e85c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:10.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.681+0000 7f4ecc1dc740 1 --2- 172.21.15.83:0/4101474918 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622a74ddb40 0x5622a74e85c0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f4eb4005cd0 tx=0x7f4eb400b950 comp rx=0 tx=0).stop 2024-04-24T09:00:10.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.681+0000 7f4ecc1dc740 1 -- 172.21.15.83:0/4101474918 shutdown_connections 2024-04-24T09:00:10.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.681+0000 7f4ecc1dc740 1 --2- 172.21.15.83:0/4101474918 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x5622a74f8c80 0x5622a74fb070 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.681+0000 7f4ecc1dc740 1 --2- 172.21.15.83:0/4101474918 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622a74e8b30 0x5622a74e5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.681+0000 7f4ecc1dc740 1 --2- 172.21.15.83:0/4101474918 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622a74de460 0x5622a74e58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.681+0000 7f4ecc1dc740 1 --2- 172.21.15.83:0/4101474918 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622a74ddb40 0x5622a74e85c0 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.681+0000 7f4ea7e00640 0 client.18684 ms_handle_reset on v2:172.21.15.138:6834/1909219758 2024-04-24T09:00:10.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.681+0000 7f4ecc1dc740 1 -- 172.21.15.83:0/4101474918 >> 172.21.15.83:0/4101474918 conn(0x5622a74d9e60 msgr2=0x5622a74dfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:10.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.681+0000 7f4ecc1dc740 1 -- 172.21.15.83:0/4101474918 shutdown_connections 2024-04-24T09:00:10.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.681+0000 7f4ecc1dc740 1 -- 172.21.15.83:0/4101474918 wait complete. 2024-04-24T09:00:10.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.681+0000 7f4ecc1dc740 20 client.18684 trim_cache size 0 max 16384 2024-04-24T09:00:10.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.697+0000 7f4ecc1dc740 1 Processor -- start 2024-04-24T09:00:10.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.697+0000 7f4ecc1dc740 1 -- start start 2024-04-24T09:00:10.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.698+0000 7f4ecc1dc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622a74ddb40 0x5622a74f4880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:10.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.698+0000 7f4ecc1dc740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622a74de460 0x5622a74e80f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:10.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.698+0000 7f4ecc1dc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622a74e8b30 0x5622a7500610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:10.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.698+0000 7f4ecc1dc740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec41b1660 con 0x5622a74e8b30 2024-04-24T09:00:10.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.698+0000 7f4ecc1dc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec41b31a0 con 0x5622a74ddb40 2024-04-24T09:00:10.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.698+0000 7f4ecc1dc740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec41b1d30 con 0x5622a74de460 2024-04-24T09:00:10.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.698+0000 7f4ec2800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622a74e8b30 0x5622a7500610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:10.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.698+0000 7f4ec2800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622a74e8b30 0x5622a7500610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48752/0 (socket says 172.21.15.83:48752) 2024-04-24T09:00:10.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.698+0000 7f4ec2800640 1 -- 172.21.15.83:0/2769216911 learned_addr learned my addr 172.21.15.83:0/2769216911 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:00:10.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.698+0000 7f4eb9400640 1 --2- 172.21.15.83:0/2769216911 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622a74de460 0x5622a74e80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:10.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.698+0000 7f4ec1e00640 1 --2- 172.21.15.83:0/2769216911 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622a74ddb40 0x5622a74f4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:10.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.699+0000 7f4ec2800640 1 -- 172.21.15.83:0/2769216911 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622a74de460 msgr2=0x5622a74e80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:10.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.699+0000 7f4ec2800640 1 --2- 172.21.15.83:0/2769216911 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622a74de460 0x5622a74e80f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.699+0000 7f4ec2800640 1 -- 172.21.15.83:0/2769216911 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622a74ddb40 msgr2=0x5622a74f4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:10.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.699+0000 7f4ec2800640 1 --2- 172.21.15.83:0/2769216911 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622a74ddb40 0x5622a74f4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.699+0000 7f4ec2800640 1 -- 172.21.15.83:0/2769216911 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622a74e6310 con 0x5622a74e8b30 2024-04-24T09:00:10.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.699+0000 7f4ec1e00640 1 --2- 172.21.15.83:0/2769216911 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622a74ddb40 0x5622a74f4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:00:10.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.699+0000 7f4eb9400640 1 --2- 172.21.15.83:0/2769216911 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622a74de460 0x5622a74e80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:00:10.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.699+0000 7f4ec2800640 1 --2- 172.21.15.83:0/2769216911 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622a74e8b30 0x5622a7500610 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc099480 tx=0x7f4ebc09d270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c6c48c4208c024c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:10.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.699+0000 7f4eb2a00640 1 -- 172.21.15.83:0/2769216911 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ebc0a5720 con 0x5622a74e8b30 2024-04-24T09:00:10.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.699+0000 7f4eb2a00640 1 -- 172.21.15.83:0/2769216911 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4ebc09adb0 con 0x5622a74e8b30 2024-04-24T09:00:10.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.699+0000 7f4eb2a00640 1 -- 172.21.15.83:0/2769216911 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ebc0a5290 con 0x5622a74e8b30 2024-04-24T09:00:10.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.700+0000 7f4ecc1dc740 1 -- 172.21.15.83:0/2769216911 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622a74e8b30 msgr2=0x5622a7500610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:10.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.700+0000 7f4ecc1dc740 1 --2- 172.21.15.83:0/2769216911 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622a74e8b30 0x5622a7500610 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc099480 tx=0x7f4ebc09d270 comp rx=0 tx=0).stop 2024-04-24T09:00:10.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.700+0000 7f4ecc1dc740 1 -- 172.21.15.83:0/2769216911 shutdown_connections 2024-04-24T09:00:10.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.700+0000 7f4ecc1dc740 1 --2- 172.21.15.83:0/2769216911 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622a74e8b30 0x5622a7500610 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.700+0000 7f4ecc1dc740 1 --2- 172.21.15.83:0/2769216911 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622a74de460 0x5622a74e80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.700+0000 7f4ecc1dc740 1 --2- 172.21.15.83:0/2769216911 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622a74ddb40 0x5622a74f4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.700+0000 7f4ecc1dc740 1 -- 172.21.15.83:0/2769216911 >> 172.21.15.83:0/2769216911 conn(0x5622a74d9e60 msgr2=0x5622a74e41a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:10.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.700+0000 7f4ecc1dc740 1 -- 172.21.15.83:0/2769216911 shutdown_connections 2024-04-24T09:00:10.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.700+0000 7f4ecc1dc740 1 -- 172.21.15.83:0/2769216911 wait complete. 2024-04-24T09:00:10.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.701+0000 7f4ecc1dc740 1 -- start start 2024-04-24T09:00:10.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.701+0000 7f4ecc1dc740 1 Processor -- start 2024-04-24T09:00:10.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.702+0000 7f4ecc1dc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622a74ddb40 0x5622a74f0f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:10.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.702+0000 7f4ecc1dc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622a74de460 0x5622a74e2b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:10.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.702+0000 7f4ecc1dc740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622a74e8b30 0x5622a74e3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:10.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.702+0000 7f4ecc1dc740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec41b2ad0 con 0x5622a74ddb40 2024-04-24T09:00:10.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.702+0000 7f4ecc1dc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec41b2400 con 0x5622a74de460 2024-04-24T09:00:10.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.702+0000 7f4ecc1dc740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec41af660 con 0x5622a74e8b30 2024-04-24T09:00:10.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.702+0000 7f4ecc1dc740 10 client.0 _ll_register_callbacks cb 0x7f4eb400bb60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:00:10.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.702+0000 7f4ec1e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622a74ddb40 0x5622a74f0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:10.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.702+0000 7f4ec1e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622a74ddb40 0x5622a74f0f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48760/0 (socket says 172.21.15.83:48760) 2024-04-24T09:00:10.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.702+0000 7f4ec1e00640 1 -- 172.21.15.83:0/4144664543 learned_addr learned my addr 172.21.15.83:0/4144664543 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:00:10.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.702+0000 7f4eb9400640 1 --2- 172.21.15.83:0/4144664543 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622a74de460 0x5622a74e2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:10.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.702+0000 7f4ec1e00640 1 -- 172.21.15.83:0/4144664543 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622a74e8b30 msgr2=0x5622a74e3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:10.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.702+0000 7f4ec1e00640 1 --2- 172.21.15.83:0/4144664543 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622a74e8b30 0x5622a74e3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.702+0000 7f4ec1e00640 1 -- 172.21.15.83:0/4144664543 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622a74de460 msgr2=0x5622a74e2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:10.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.702+0000 7f4ec1e00640 1 --2- 172.21.15.83:0/4144664543 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622a74de460 0x5622a74e2b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.702+0000 7f4ec1e00640 1 -- 172.21.15.83:0/4144664543 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622a721f7d0 con 0x5622a74ddb40 2024-04-24T09:00:10.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.702+0000 7f4eb9400640 1 --2- 172.21.15.83:0/4144664543 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622a74de460 0x5622a74e2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:00:10.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.703+0000 7f4ec1e00640 1 --2- 172.21.15.83:0/4144664543 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622a74ddb40 0x5622a74f0f80 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f4eb40107e0 tx=0x7f4eb4005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e596bcc080435fe server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:10.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.703+0000 7f4ea7e00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:00:10.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.704+0000 7f4ea7e00640 1 -- 172.21.15.83:0/4144664543 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb4029040 con 0x5622a74ddb40 2024-04-24T09:00:10.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.704+0000 7f4ea7e00640 1 -- 172.21.15.83:0/4144664543 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f4eb4003a30 con 0x5622a74ddb40 2024-04-24T09:00:10.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.704+0000 7f4ecc1dc740 1 -- 172.21.15.83:0/4144664543 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ebc09adb0 con 0x5622a74ddb40 2024-04-24T09:00:10.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.705+0000 7f4ea7e00640 1 -- 172.21.15.83:0/4144664543 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f4eb40115f0 con 0x5622a74ddb40 2024-04-24T09:00:10.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.705+0000 7f4ecc1dc740 10 client.18690 fetch_fsmap learned FSMap version 35 2024-04-24T09:00:10.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.705+0000 7f4ecc1dc740 1 -- 172.21.15.83:0/4144664543 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5622a74ea080 con 0x5622a74ddb40 2024-04-24T09:00:10.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.705+0000 7f4ea7e00640 1 -- 172.21.15.83:0/4144664543 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4ebc09adb0 con 0x5622a74ddb40 2024-04-24T09:00:10.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.705+0000 7f4ea7e00640 1 -- 172.21.15.83:0/4144664543 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7f4eb4005220 con 0x5622a74ddb40 2024-04-24T09:00:10.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.705+0000 7f4ecc1dc740 10 client.18690 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T09:00:10.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.705+0000 7f4ecc1dc740 10 client.18690 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-24T09:00:10.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.705+0000 7f4ecc1dc740 20 client.18690 populate_metadata read hostname 'smithi083' 2024-04-24T09:00:10.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.705+0000 7f4ecc1dc740 1 --2- 172.21.15.83:0/4144664543 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x5622a755f0e0 0x5622a7561500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:10.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.705+0000 7f4ecc1dc740 4 client.18690 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:00:10.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.705+0000 7f4ecc1dc740 1 -- 172.21.15.83:0/4144664543 --> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5622a744aa60 con 0x5622a755f0e0 2024-04-24T09:00:10.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.706+0000 7f4eb9400640 1 --2- 172.21.15.83:0/4144664543 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x5622a755f0e0 0x5622a7561500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:10.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.707+0000 7f4eb9400640 1 --2- 172.21.15.83:0/4144664543 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x5622a755f0e0 0x5622a7561500 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e505896108ea1618 server_cookie=ad80e7079d7e5dd4 in_seq=0 out_seq=0 2024-04-24T09:00:10.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.707+0000 7f4ea7e00640 10 client.18690 ms_handle_connect on v2:172.21.15.138:6834/1909219758 2024-04-24T09:00:10.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.708+0000 7f4ea7e00640 1 -- 172.21.15.83:0/4144664543 <== mds.2 v2:172.21.15.138:6834/1909219758 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5622a744aa60 con 0x5622a755f0e0 2024-04-24T09:00:10.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.708+0000 7f4ea7e00640 10 client.18690 handle_command_reply: tid=0 2024-04-24T09:00:10.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.708+0000 7f4ecc1dc740 1 client.18690 shutdown 2024-04-24T09:00:10.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.708+0000 7f4ecc1dc740 1 -- 172.21.15.83:0/4144664543 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622a74ddb40 msgr2=0x5622a74f0f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:10.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.708+0000 7f4ecc1dc740 1 --2- 172.21.15.83:0/4144664543 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622a74ddb40 0x5622a74f0f80 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f4eb40107e0 tx=0x7f4eb4005e50 comp rx=0 tx=0).stop 2024-04-24T09:00:10.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.708+0000 7f4ecc1dc740 1 -- 172.21.15.83:0/4144664543 shutdown_connections 2024-04-24T09:00:10.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.708+0000 7f4ecc1dc740 1 --2- 172.21.15.83:0/4144664543 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x5622a755f0e0 0x5622a7561500 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.708+0000 7f4ecc1dc740 1 --2- 172.21.15.83:0/4144664543 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622a74e8b30 0x5622a74e3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.708+0000 7f4ecc1dc740 1 --2- 172.21.15.83:0/4144664543 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622a74de460 0x5622a74e2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.708+0000 7f4ea7e00640 0 client.18690 ms_handle_reset on v2:172.21.15.138:6834/1909219758 2024-04-24T09:00:10.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.708+0000 7f4ecc1dc740 1 --2- 172.21.15.83:0/4144664543 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622a74ddb40 0x5622a74f0f80 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.708+0000 7f4ecc1dc740 1 -- 172.21.15.83:0/4144664543 >> 172.21.15.83:0/4144664543 conn(0x5622a74d9e60 msgr2=0x5622a74d86c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:10.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.708+0000 7f4ecc1dc740 1 -- 172.21.15.83:0/4144664543 shutdown_connections 2024-04-24T09:00:10.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.708+0000 7f4ecc1dc740 1 -- 172.21.15.83:0/4144664543 wait complete. 2024-04-24T09:00:10.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.709+0000 7f4ecc1dc740 20 client.18690 trim_cache size 0 max 16384 2024-04-24T09:00:10.709 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:00:10.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.709+0000 7f4ec3200640 1 -- 172.21.15.83:0/4277181396 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f4ea80a3da0 msgr2=0x7f4ea80a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:10.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.709+0000 7f4ec3200640 1 --2- 172.21.15.83:0/4277181396 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f4ea80a3da0 0x7f4ea80a6260 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f4eac006010 tx=0x7f4eac000bc0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.709+0000 7f4ec3200640 1 -- 172.21.15.83:0/4277181396 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ec419a6e0 msgr2=0x7f4ec419ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:10.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.709+0000 7f4ec3200640 1 --2- 172.21.15.83:0/4277181396 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ec419a6e0 0x7f4ec419ab60 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc00cce0 tx=0x7f4ebc00a620 comp rx=0 tx=0).stop 2024-04-24T09:00:10.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.710+0000 7f4ec3200640 1 -- 172.21.15.83:0/4277181396 shutdown_connections 2024-04-24T09:00:10.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.710+0000 7f4ec3200640 1 --2- 172.21.15.83:0/4277181396 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f4ea80a3da0 0x7f4ea80a6260 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.710+0000 7f4ec3200640 1 --2- 172.21.15.83:0/4277181396 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ec419a6e0 0x7f4ec419ab60 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.710+0000 7f4ec3200640 1 --2- 172.21.15.83:0/4277181396 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4ec41093f0 0x7f4ec41a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.710+0000 7f4ec3200640 1 --2- 172.21.15.83:0/4277181396 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ec4104320 0x7f4ec41a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.710+0000 7f4ec3200640 1 -- 172.21.15.83:0/4277181396 >> 172.21.15.83:0/4277181396 conn(0x7f4ec40ffb40 msgr2=0x7f4ec4100e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:10.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.710+0000 7f4ec3200640 1 -- 172.21.15.83:0/4277181396 shutdown_connections 2024-04-24T09:00:10.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.710+0000 7f4ec3200640 1 -- 172.21.15.83:0/4277181396 wait complete. 2024-04-24T09:00:10.728 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T09:00:10.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.930+0000 7f9699800640 1 -- 172.21.15.83:0/2055819654 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9694100c70 msgr2=0x7f9694101050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:10.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.930+0000 7f9699800640 1 --2- 172.21.15.83:0/2055819654 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9694100c70 0x7f9694101050 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f968c009ca0 tx=0x7f968c02a8c0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.931+0000 7f9699800640 1 -- 172.21.15.83:0/2055819654 shutdown_connections 2024-04-24T09:00:10.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.931+0000 7f9699800640 1 --2- 172.21.15.83:0/2055819654 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f969410e820 0x7f9694110c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.931+0000 7f9699800640 1 --2- 172.21.15.83:0/2055819654 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9694101620 0x7f969410e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.931+0000 7f9699800640 1 --2- 172.21.15.83:0/2055819654 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9694100c70 0x7f9694101050 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.931+0000 7f9699800640 1 -- 172.21.15.83:0/2055819654 >> 172.21.15.83:0/2055819654 conn(0x7f96940fc880 msgr2=0x7f96940feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:10.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.931+0000 7f9699800640 1 -- 172.21.15.83:0/2055819654 shutdown_connections 2024-04-24T09:00:10.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.931+0000 7f9699800640 1 -- 172.21.15.83:0/2055819654 wait complete. 2024-04-24T09:00:10.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.932+0000 7f9699800640 1 Processor -- start 2024-04-24T09:00:10.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.932+0000 7f9699800640 1 -- start start 2024-04-24T09:00:10.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.932+0000 7f9699800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9694100c70 0x7f96941a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:10.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.932+0000 7f9699800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9694101620 0x7f96941a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:10.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.932+0000 7f9699800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f969410e820 0x7f969419a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:10.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.932+0000 7f9699800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f96941143c0 con 0x7f969410e820 2024-04-24T09:00:10.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.932+0000 7f9699800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9694114240 con 0x7f9694101620 2024-04-24T09:00:10.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.932+0000 7f9699800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9694114540 con 0x7f9694100c70 2024-04-24T09:00:10.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.933+0000 7f9698e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f969410e820 0x7f969419a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:10.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.933+0000 7f9698e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f969410e820 0x7f969419a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48774/0 (socket says 172.21.15.83:48774) 2024-04-24T09:00:10.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.933+0000 7f9698e00640 1 -- 172.21.15.83:0/3702032437 learned_addr learned my addr 172.21.15.83:0/3702032437 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:00:10.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.933+0000 7f9693400640 1 --2- 172.21.15.83:0/3702032437 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9694101620 0x7f96941a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:10.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.933+0000 7f9693e00640 1 --2- 172.21.15.83:0/3702032437 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9694100c70 0x7f96941a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:10.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.933+0000 7f9698e00640 1 -- 172.21.15.83:0/3702032437 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9694100c70 msgr2=0x7f96941a06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:10.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.933+0000 7f9698e00640 1 --2- 172.21.15.83:0/3702032437 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9694100c70 0x7f96941a06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.933+0000 7f9698e00640 1 -- 172.21.15.83:0/3702032437 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9694101620 msgr2=0x7f96941a0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:10.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.933+0000 7f9698e00640 1 --2- 172.21.15.83:0/3702032437 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9694101620 0x7f96941a0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.933+0000 7f9698e00640 1 -- 172.21.15.83:0/3702032437 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f969419b020 con 0x7f969410e820 2024-04-24T09:00:10.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.933+0000 7f9693400640 1 --2- 172.21.15.83:0/3702032437 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9694101620 0x7f96941a0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:00:10.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.933+0000 7f9698e00640 1 --2- 172.21.15.83:0/3702032437 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f969410e820 0x7f969419a760 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f968400d6a0 tx=0x7f968400db60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:10.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.934+0000 7f9690c00640 1 -- 172.21.15.83:0/3702032437 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9684017070 con 0x7f969410e820 2024-04-24T09:00:10.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.934+0000 7f9690c00640 1 -- 172.21.15.83:0/3702032437 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9684009820 con 0x7f969410e820 2024-04-24T09:00:10.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.934+0000 7f9699800640 1 -- 172.21.15.83:0/3702032437 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f969419b310 con 0x7f969410e820 2024-04-24T09:00:10.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.934+0000 7f9690c00640 1 -- 172.21.15.83:0/3702032437 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9684004c40 con 0x7f969410e820 2024-04-24T09:00:10.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.934+0000 7f9699800640 1 -- 172.21.15.83:0/3702032437 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96941a7620 con 0x7f969410e820 2024-04-24T09:00:10.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.937+0000 7f969dba1740 1 Processor -- start 2024-04-24T09:00:10.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.937+0000 7f9690c00640 1 -- 172.21.15.83:0/3702032437 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f96840099d0 con 0x7f969410e820 2024-04-24T09:00:10.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.938+0000 7f969dba1740 1 -- start start 2024-04-24T09:00:10.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.938+0000 7f969dba1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56426e37cb40 0x56426e37cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:10.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.938+0000 7f969dba1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56426e37d460 0x56426e3875f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:10.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.938+0000 7f969dba1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56426e387b30 0x56426e389f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:10.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.938+0000 7f969dba1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56426e2e21d0 con 0x56426e387b30 2024-04-24T09:00:10.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.938+0000 7f969dba1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56426e288450 con 0x56426e37cb40 2024-04-24T09:00:10.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.938+0000 7f969dba1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56426e2de190 con 0x56426e37d460 2024-04-24T09:00:10.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.938+0000 7f9693400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56426e387b30 0x56426e389f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:10.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.938+0000 7f9693400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56426e387b30 0x56426e389f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48782/0 (socket says 172.21.15.83:48782) 2024-04-24T09:00:10.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.938+0000 7f9693400640 1 -- 172.21.15.83:0/2459826985 learned_addr learned my addr 172.21.15.83:0/2459826985 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:00:10.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.939+0000 7f9693e00640 1 --2- 172.21.15.83:0/2459826985 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56426e37d460 0x56426e3875f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:10.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.939+0000 7f9693400640 1 --2- 172.21.15.83:0/2459826985 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56426e37cb40 0x56426e37cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:10.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.939+0000 7f9693400640 1 -- 172.21.15.83:0/2459826985 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56426e37d460 msgr2=0x56426e3875f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:10.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.939+0000 7f9693400640 1 --2- 172.21.15.83:0/2459826985 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56426e37d460 0x56426e3875f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.939+0000 7f9693400640 1 -- 172.21.15.83:0/2459826985 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56426e37cb40 msgr2=0x56426e37cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:10.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.939+0000 7f9693400640 1 --2- 172.21.15.83:0/2459826985 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56426e37cb40 0x56426e37cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.939+0000 7f9693400640 1 -- 172.21.15.83:0/2459826985 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56426e345310 con 0x56426e387b30 2024-04-24T09:00:10.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.939+0000 7f9690c00640 1 --2- 172.21.15.83:0/3702032437 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f96780a3d20 0x7f96780a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:10.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.939+0000 7f9693400640 1 --2- 172.21.15.83:0/2459826985 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56426e387b30 0x56426e389f20 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f967c014a30 tx=0x7f967c00c570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=777ff557058a32f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:10.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.940+0000 7f9690c00640 1 -- 172.21.15.83:0/3702032437 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f96840d05c0 con 0x7f969410e820 2024-04-24T09:00:10.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.940+0000 7f9689600640 1 -- 172.21.15.83:0/2459826985 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f967c0142d0 con 0x56426e387b30 2024-04-24T09:00:10.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.940+0000 7f9693e00640 1 --2- 172.21.15.83:0/3702032437 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f96780a3d20 0x7f96780a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:10.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.940+0000 7f9689600640 1 -- 172.21.15.83:0/2459826985 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f967c0054d0 con 0x56426e387b30 2024-04-24T09:00:10.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.940+0000 7f9689600640 1 -- 172.21.15.83:0/2459826985 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f967c0047a0 con 0x56426e387b30 2024-04-24T09:00:10.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.941+0000 7f969dba1740 1 -- 172.21.15.83:0/2459826985 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56426e387b30 msgr2=0x56426e389f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:10.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.941+0000 7f969dba1740 1 --2- 172.21.15.83:0/2459826985 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56426e387b30 0x56426e389f20 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f967c014a30 tx=0x7f967c00c570 comp rx=0 tx=0).stop 2024-04-24T09:00:10.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.941+0000 7f9693e00640 1 --2- 172.21.15.83:0/3702032437 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f96780a3d20 0x7f96780a61e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f968c02f3a0 tx=0x7f968c02f330 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:10.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.941+0000 7f969dba1740 1 -- 172.21.15.83:0/2459826985 shutdown_connections 2024-04-24T09:00:10.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.941+0000 7f969dba1740 1 --2- 172.21.15.83:0/2459826985 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56426e387b30 0x56426e389f20 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.941+0000 7f969dba1740 1 --2- 172.21.15.83:0/2459826985 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56426e37d460 0x56426e3875f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.941+0000 7f969dba1740 1 --2- 172.21.15.83:0/2459826985 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56426e37cb40 0x56426e37cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.941+0000 7f969dba1740 1 -- 172.21.15.83:0/2459826985 >> 172.21.15.83:0/2459826985 conn(0x56426e378e60 msgr2=0x56426e379240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:10.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.941+0000 7f969dba1740 1 -- 172.21.15.83:0/2459826985 shutdown_connections 2024-04-24T09:00:10.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.941+0000 7f969dba1740 1 -- 172.21.15.83:0/2459826985 wait complete. 2024-04-24T09:00:10.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.941+0000 7f969dba1740 1 -- start start 2024-04-24T09:00:10.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.942+0000 7f969dba1740 1 Processor -- start 2024-04-24T09:00:10.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.942+0000 7f969dba1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56426e37cb40 0x56426e387580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:10.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.942+0000 7f969dba1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56426e37d460 0x56426e384870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:10.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.942+0000 7f969dba1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56426e387b30 0x56426e384fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:10.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.943+0000 7f969dba1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56426e38a960 con 0x56426e37cb40 2024-04-24T09:00:10.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.943+0000 7f969dba1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56426e29b810 con 0x56426e37d460 2024-04-24T09:00:10.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.943+0000 7f969dba1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56426e38aae0 con 0x56426e387b30 2024-04-24T09:00:10.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.943+0000 7f969dba1740 10 client.0 _ll_register_callbacks cb 0x56426e269140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:00:10.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.943+0000 7f9693e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56426e37cb40 0x56426e387580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:10.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.943+0000 7f9693e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56426e37cb40 0x56426e387580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48794/0 (socket says 172.21.15.83:48794) 2024-04-24T09:00:10.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.943+0000 7f9693e00640 1 -- 172.21.15.83:0/2600350021 learned_addr learned my addr 172.21.15.83:0/2600350021 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:00:10.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.943+0000 7f9693400640 1 --2- 172.21.15.83:0/2600350021 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56426e37d460 0x56426e384870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:10.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.943+0000 7f9698e00640 1 --2- 172.21.15.83:0/2600350021 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56426e387b30 0x56426e384fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:10.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.943+0000 7f9693e00640 1 -- 172.21.15.83:0/2600350021 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56426e387b30 msgr2=0x56426e384fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:10.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.943+0000 7f9693e00640 1 --2- 172.21.15.83:0/2600350021 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56426e387b30 0x56426e384fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.943+0000 7f9693e00640 1 -- 172.21.15.83:0/2600350021 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56426e37d460 msgr2=0x56426e384870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:10.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.943+0000 7f9693e00640 1 --2- 172.21.15.83:0/2600350021 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56426e37d460 0x56426e384870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.943+0000 7f9693e00640 1 -- 172.21.15.83:0/2600350021 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56426e2ac3f0 con 0x56426e37cb40 2024-04-24T09:00:10.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.944+0000 7f9693e00640 1 --2- 172.21.15.83:0/2600350021 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56426e37cb40 0x56426e387580 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f968c009ca0 tx=0x7f968c02a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d725da848fc664fa server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:10.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.944+0000 7f9682000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:00:10.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.944+0000 7f9682000640 1 -- 172.21.15.83:0/2600350021 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f968c0305c0 con 0x56426e37cb40 2024-04-24T09:00:10.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.944+0000 7f9682000640 1 -- 172.21.15.83:0/2600350021 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f968c030760 con 0x56426e37cb40 2024-04-24T09:00:10.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.944+0000 7f969dba1740 1 -- 172.21.15.83:0/2600350021 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f967c0054d0 con 0x56426e37cb40 2024-04-24T09:00:10.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.945+0000 7f9682000640 1 -- 172.21.15.83:0/2600350021 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f968c030960 con 0x56426e37cb40 2024-04-24T09:00:10.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.945+0000 7f969dba1740 10 client.18702 fetch_fsmap learned FSMap version 35 2024-04-24T09:00:10.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.945+0000 7f969dba1740 1 -- 172.21.15.83:0/2600350021 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56426dff43f0 con 0x56426e37cb40 2024-04-24T09:00:10.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.945+0000 7f9682000640 1 -- 172.21.15.83:0/2600350021 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f967c0054d0 con 0x56426e37cb40 2024-04-24T09:00:10.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.945+0000 7f9682000640 1 -- 172.21.15.83:0/2600350021 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7f968c051070 con 0x56426e37cb40 2024-04-24T09:00:10.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.945+0000 7f969dba1740 10 client.18702 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T09:00:10.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.945+0000 7f969dba1740 10 client.18702 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-24T09:00:10.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.945+0000 7f969dba1740 20 client.18702 populate_metadata read hostname 'smithi083' 2024-04-24T09:00:10.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.945+0000 7f969dba1740 1 --2- 172.21.15.83:0/2600350021 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x56426e397cc0 0x56426e39a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:10.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.945+0000 7f969dba1740 4 client.18702 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:00:10.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.945+0000 7f969dba1740 1 -- 172.21.15.83:0/2600350021 --> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56426e2efbf0 con 0x56426e397cc0 2024-04-24T09:00:10.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.946+0000 7f9693400640 1 --2- 172.21.15.83:0/2600350021 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x56426e397cc0 0x56426e39a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:10.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.947+0000 7f9693400640 1 --2- 172.21.15.83:0/2600350021 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x56426e397cc0 0x56426e39a160 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=28da47fa81eeab1d server_cookie=ba073a7b7906e635 in_seq=0 out_seq=0 2024-04-24T09:00:10.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.947+0000 7f9682000640 10 client.18702 ms_handle_connect on v2:172.21.15.138:6836/900566754 2024-04-24T09:00:10.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.948+0000 7f9682000640 1 -- 172.21.15.83:0/2600350021 <== mds.1 v2:172.21.15.138:6836/900566754 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56426e2efbf0 con 0x56426e397cc0 2024-04-24T09:00:10.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.948+0000 7f9682000640 10 client.18702 handle_command_reply: tid=0 2024-04-24T09:00:10.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.949+0000 7f969dba1740 1 client.18702 shutdown 2024-04-24T09:00:10.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.949+0000 7f969dba1740 1 -- 172.21.15.83:0/2600350021 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56426e37cb40 msgr2=0x56426e387580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:10.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.949+0000 7f969dba1740 1 --2- 172.21.15.83:0/2600350021 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56426e37cb40 0x56426e387580 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f968c009ca0 tx=0x7f968c02a8c0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.949+0000 7f969dba1740 1 -- 172.21.15.83:0/2600350021 shutdown_connections 2024-04-24T09:00:10.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.949+0000 7f969dba1740 1 --2- 172.21.15.83:0/2600350021 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x56426e397cc0 0x56426e39a160 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.949+0000 7f969dba1740 1 --2- 172.21.15.83:0/2600350021 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56426e387b30 0x56426e384fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.949+0000 7f969dba1740 1 --2- 172.21.15.83:0/2600350021 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56426e37d460 0x56426e384870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.949+0000 7f9682000640 0 client.18702 ms_handle_reset on v2:172.21.15.138:6836/900566754 2024-04-24T09:00:10.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.949+0000 7f969dba1740 1 --2- 172.21.15.83:0/2600350021 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56426e37cb40 0x56426e387580 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.949+0000 7f969dba1740 1 -- 172.21.15.83:0/2600350021 >> 172.21.15.83:0/2600350021 conn(0x56426e378e60 msgr2=0x56426e37ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:10.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.949+0000 7f969dba1740 1 -- 172.21.15.83:0/2600350021 shutdown_connections 2024-04-24T09:00:10.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.949+0000 7f969dba1740 1 -- 172.21.15.83:0/2600350021 wait complete. 2024-04-24T09:00:10.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.949+0000 7f969dba1740 20 client.18702 trim_cache size 0 max 16384 2024-04-24T09:00:10.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.964+0000 7f969dba1740 1 Processor -- start 2024-04-24T09:00:10.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.964+0000 7f969dba1740 1 -- start start 2024-04-24T09:00:10.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.964+0000 7f969dba1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56426e37cb40 0x56426e37f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:10.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.964+0000 7f969dba1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56426e37d460 0x56426e377dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:10.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.964+0000 7f969dba1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56426e387b30 0x56426e3870b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:10.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.964+0000 7f969dba1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f96941b1600 con 0x56426e37d460 2024-04-24T09:00:10.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.964+0000 7f969dba1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f96941b3140 con 0x56426e37cb40 2024-04-24T09:00:10.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.964+0000 7f969dba1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f96941b1cd0 con 0x56426e387b30 2024-04-24T09:00:10.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.965+0000 7f9693400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56426e37d460 0x56426e377dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:10.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.965+0000 7f9693e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56426e37cb40 0x56426e37f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:10.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.965+0000 7f9693e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56426e37cb40 0x56426e37f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:36188/0 (socket says 172.21.15.83:36188) 2024-04-24T09:00:10.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.965+0000 7f9693e00640 1 -- 172.21.15.83:0/3992369486 learned_addr learned my addr 172.21.15.83:0/3992369486 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:00:10.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.965+0000 7f9698e00640 1 --2- 172.21.15.83:0/3992369486 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56426e387b30 0x56426e3870b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:10.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.965+0000 7f9693400640 1 -- 172.21.15.83:0/3992369486 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56426e387b30 msgr2=0x56426e3870b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:10.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.965+0000 7f9693400640 1 --2- 172.21.15.83:0/3992369486 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56426e387b30 0x56426e3870b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.965+0000 7f9693400640 1 -- 172.21.15.83:0/3992369486 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56426e37cb40 msgr2=0x56426e37f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:10.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.965+0000 7f9693400640 1 --2- 172.21.15.83:0/3992369486 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56426e37cb40 0x56426e37f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.966+0000 7f9693400640 1 -- 172.21.15.83:0/3992369486 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56426e0be7d0 con 0x56426e37d460 2024-04-24T09:00:10.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.966+0000 7f9693e00640 1 --2- 172.21.15.83:0/3992369486 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56426e37cb40 0x56426e37f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:00:10.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.966+0000 7f9693400640 1 --2- 172.21.15.83:0/3992369486 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56426e37d460 0x56426e377dd0 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f967c00fc60 tx=0x7f967c016040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1d085161a637df1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:10.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.966+0000 7f9688c00640 1 -- 172.21.15.83:0/3992369486 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f967c0140d0 con 0x56426e37d460 2024-04-24T09:00:10.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.966+0000 7f9688c00640 1 -- 172.21.15.83:0/3992369486 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f967c014270 con 0x56426e37d460 2024-04-24T09:00:10.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.966+0000 7f9688c00640 1 -- 172.21.15.83:0/3992369486 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f967c012450 con 0x56426e37d460 2024-04-24T09:00:10.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.967+0000 7f969dba1740 1 -- 172.21.15.83:0/3992369486 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56426e37d460 msgr2=0x56426e377dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:10.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.967+0000 7f969dba1740 1 --2- 172.21.15.83:0/3992369486 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56426e37d460 0x56426e377dd0 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f967c00fc60 tx=0x7f967c016040 comp rx=0 tx=0).stop 2024-04-24T09:00:10.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.967+0000 7f969dba1740 1 -- 172.21.15.83:0/3992369486 shutdown_connections 2024-04-24T09:00:10.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.967+0000 7f969dba1740 1 --2- 172.21.15.83:0/3992369486 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56426e387b30 0x56426e3870b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.967+0000 7f969dba1740 1 --2- 172.21.15.83:0/3992369486 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56426e37d460 0x56426e377dd0 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.967+0000 7f969dba1740 1 --2- 172.21.15.83:0/3992369486 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56426e37cb40 0x56426e37f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.967+0000 7f969dba1740 1 -- 172.21.15.83:0/3992369486 >> 172.21.15.83:0/3992369486 conn(0x56426e378e60 msgr2=0x56426e3776c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:10.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.967+0000 7f969dba1740 1 -- 172.21.15.83:0/3992369486 shutdown_connections 2024-04-24T09:00:10.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.967+0000 7f969dba1740 1 -- 172.21.15.83:0/3992369486 wait complete. 2024-04-24T09:00:10.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.968+0000 7f969dba1740 1 -- start start 2024-04-24T09:00:10.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.968+0000 7f969dba1740 1 Processor -- start 2024-04-24T09:00:10.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.968+0000 7f969dba1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56426e37cb40 0x56426e38fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:10.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.968+0000 7f969dba1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56426e37d460 0x56426e3902a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:10.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.968+0000 7f969dba1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56426e387b30 0x56426e388d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:10.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.968+0000 7f969dba1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f96941b2a70 con 0x56426e387b30 2024-04-24T09:00:10.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.968+0000 7f969dba1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f96941b23a0 con 0x56426e37cb40 2024-04-24T09:00:10.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.968+0000 7f969dba1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f96941af600 con 0x56426e37d460 2024-04-24T09:00:10.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.968+0000 7f969dba1740 10 client.0 _ll_register_callbacks cb 0x7f968c009360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:00:10.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.969+0000 7f9698e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56426e387b30 0x56426e388d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:10.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.969+0000 7f9698e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56426e387b30 0x56426e388d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48806/0 (socket says 172.21.15.83:48806) 2024-04-24T09:00:10.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.969+0000 7f9698e00640 1 -- 172.21.15.83:0/559604618 learned_addr learned my addr 172.21.15.83:0/559604618 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:00:10.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.969+0000 7f9693e00640 1 --2- 172.21.15.83:0/559604618 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56426e37cb40 0x56426e38fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:10.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.969+0000 7f9693400640 1 --2- 172.21.15.83:0/559604618 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56426e37d460 0x56426e3902a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:10.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.969+0000 7f9698e00640 1 -- 172.21.15.83:0/559604618 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56426e37d460 msgr2=0x56426e3902a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:10.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.969+0000 7f9698e00640 1 --2- 172.21.15.83:0/559604618 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56426e37d460 0x56426e3902a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.969+0000 7f9698e00640 1 -- 172.21.15.83:0/559604618 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56426e37cb40 msgr2=0x56426e38fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:10.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.969+0000 7f9698e00640 1 --2- 172.21.15.83:0/559604618 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56426e37cb40 0x56426e38fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.969+0000 7f9698e00640 1 -- 172.21.15.83:0/559604618 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56426e389260 con 0x56426e387b30 2024-04-24T09:00:10.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.969+0000 7f9693400640 1 --2- 172.21.15.83:0/559604618 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56426e37d460 0x56426e3902a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:00:10.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.969+0000 7f9693e00640 1 --2- 172.21.15.83:0/559604618 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56426e37cb40 0x56426e38fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:00:10.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.969+0000 7f9698e00640 1 --2- 172.21.15.83:0/559604618 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56426e387b30 0x56426e388d20 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f968409c070 tx=0x7f968409c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf3b0846fc4f8ade server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:10.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.969+0000 7f9682000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:00:10.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.970+0000 7f9682000640 1 -- 172.21.15.83:0/559604618 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96840b4020 con 0x56426e387b30 2024-04-24T09:00:10.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.970+0000 7f9682000640 1 -- 172.21.15.83:0/559604618 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f96840cdb10 con 0x56426e387b30 2024-04-24T09:00:10.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.970+0000 7f969dba1740 1 -- 172.21.15.83:0/559604618 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f967c014270 con 0x56426e387b30 2024-04-24T09:00:10.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.970+0000 7f9682000640 1 -- 172.21.15.83:0/559604618 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f9684020040 con 0x56426e387b30 2024-04-24T09:00:10.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.970+0000 7f969dba1740 10 client.18708 fetch_fsmap learned FSMap version 35 2024-04-24T09:00:10.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.970+0000 7f969dba1740 1 -- 172.21.15.83:0/559604618 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56426e3fea00 con 0x56426e387b30 2024-04-24T09:00:10.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.970+0000 7f9682000640 1 -- 172.21.15.83:0/559604618 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f967c014270 con 0x56426e387b30 2024-04-24T09:00:10.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.970+0000 7f9682000640 1 -- 172.21.15.83:0/559604618 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7f96840d0120 con 0x56426e387b30 2024-04-24T09:00:10.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.970+0000 7f969dba1740 10 client.18708 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T09:00:10.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.970+0000 7f969dba1740 10 client.18708 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-24T09:00:10.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.970+0000 7f969dba1740 20 client.18708 populate_metadata read hostname 'smithi083' 2024-04-24T09:00:10.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.970+0000 7f969dba1740 1 --2- 172.21.15.83:0/559604618 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x56426e3ff1a0 0x56426e4015c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:10.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.970+0000 7f969dba1740 4 client.18708 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:00:10.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.970+0000 7f969dba1740 1 -- 172.21.15.83:0/559604618 --> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56426e2e9a60 con 0x56426e3ff1a0 2024-04-24T09:00:10.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.971+0000 7f9693e00640 1 --2- 172.21.15.83:0/559604618 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x56426e3ff1a0 0x56426e4015c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:10.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.972+0000 7f9693e00640 1 --2- 172.21.15.83:0/559604618 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x56426e3ff1a0 0x56426e4015c0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=aa009fb7f5ec0ec2 server_cookie=d60071c4965bdc4d in_seq=0 out_seq=0 2024-04-24T09:00:10.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.972+0000 7f9682000640 10 client.18708 ms_handle_connect on v2:172.21.15.138:6836/900566754 2024-04-24T09:00:10.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.973+0000 7f9682000640 1 -- 172.21.15.83:0/559604618 <== mds.1 v2:172.21.15.138:6836/900566754 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56426e2e9a60 con 0x56426e3ff1a0 2024-04-24T09:00:10.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.973+0000 7f9682000640 10 client.18708 handle_command_reply: tid=0 2024-04-24T09:00:10.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.973+0000 7f969dba1740 1 client.18708 shutdown 2024-04-24T09:00:10.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.973+0000 7f969dba1740 1 -- 172.21.15.83:0/559604618 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56426e387b30 msgr2=0x56426e388d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:10.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.973+0000 7f969dba1740 1 --2- 172.21.15.83:0/559604618 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56426e387b30 0x56426e388d20 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f968409c070 tx=0x7f968409c8a0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.973+0000 7f969dba1740 1 -- 172.21.15.83:0/559604618 shutdown_connections 2024-04-24T09:00:10.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.973+0000 7f969dba1740 1 --2- 172.21.15.83:0/559604618 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56426e387b30 0x56426e388d20 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.973+0000 7f969dba1740 1 --2- 172.21.15.83:0/559604618 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x56426e3ff1a0 0x56426e4015c0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.973+0000 7f969dba1740 1 --2- 172.21.15.83:0/559604618 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56426e37d460 0x56426e3902a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.973+0000 7f969dba1740 1 --2- 172.21.15.83:0/559604618 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56426e37cb40 0x56426e38fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.973+0000 7f969dba1740 1 -- 172.21.15.83:0/559604618 >> 172.21.15.83:0/559604618 conn(0x56426e378e60 msgr2=0x56426e377f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:10.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.973+0000 7f9682000640 0 client.18708 ms_handle_reset on v2:172.21.15.138:6836/900566754 2024-04-24T09:00:10.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.973+0000 7f969dba1740 1 -- 172.21.15.83:0/559604618 shutdown_connections 2024-04-24T09:00:10.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.973+0000 7f969dba1740 1 -- 172.21.15.83:0/559604618 wait complete. 2024-04-24T09:00:10.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.973+0000 7f969dba1740 20 client.18708 trim_cache size 0 max 16384 2024-04-24T09:00:10.975 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:00:10.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.974+0000 7f9699800640 1 -- 172.21.15.83:0/3702032437 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f96780a3d20 msgr2=0x7f96780a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:10.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.974+0000 7f9699800640 1 --2- 172.21.15.83:0/3702032437 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f96780a3d20 0x7f96780a61e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f968c02f3a0 tx=0x7f968c02f330 comp rx=0 tx=0).stop 2024-04-24T09:00:10.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.974+0000 7f9699800640 1 -- 172.21.15.83:0/3702032437 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f969410e820 msgr2=0x7f969419a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:10.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.974+0000 7f9699800640 1 --2- 172.21.15.83:0/3702032437 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f969410e820 0x7f969419a760 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f968400d6a0 tx=0x7f968400db60 comp rx=0 tx=0).stop 2024-04-24T09:00:10.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.974+0000 7f9699800640 1 -- 172.21.15.83:0/3702032437 shutdown_connections 2024-04-24T09:00:10.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.974+0000 7f9699800640 1 --2- 172.21.15.83:0/3702032437 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f96780a3d20 0x7f96780a61e0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.974+0000 7f9699800640 1 --2- 172.21.15.83:0/3702032437 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f969410e820 0x7f969419a760 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.974+0000 7f9699800640 1 --2- 172.21.15.83:0/3702032437 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9694101620 0x7f96941a0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.974+0000 7f9699800640 1 --2- 172.21.15.83:0/3702032437 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9694100c70 0x7f96941a06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:10.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.974+0000 7f9699800640 1 -- 172.21.15.83:0/3702032437 >> 172.21.15.83:0/3702032437 conn(0x7f96940fc880 msgr2=0x7f96940fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:10.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.974+0000 7f9699800640 1 -- 172.21.15.83:0/3702032437 shutdown_connections 2024-04-24T09:00:10.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:10.974+0000 7f9699800640 1 -- 172.21.15.83:0/3702032437 wait complete. 2024-04-24T09:00:10.990 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:00:10 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/337268371' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:00:10.993 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T09:00:11.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:00:10 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/337268371' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:00:11.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:00:10 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/337268371' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:00:11.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.209+0000 7fa4b0e00640 1 -- 172.21.15.83:0/1380967433 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4ac104e60 msgr2=0x7fa4ac108ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:11.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.209+0000 7fa4b0e00640 1 --2- 172.21.15.83:0/1380967433 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4ac104e60 0x7fa4ac108ea0 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7fa498009900 tx=0x7fa49802a3f0 comp rx=0 tx=0).stop 2024-04-24T09:00:11.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.209+0000 7fa4b0e00640 1 -- 172.21.15.83:0/1380967433 shutdown_connections 2024-04-24T09:00:11.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.209+0000 7fa4b0e00640 1 --2- 172.21.15.83:0/1380967433 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4ac109580 0x7fa4ac111150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:11.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.209+0000 7fa4b0e00640 1 --2- 172.21.15.83:0/1380967433 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4ac104e60 0x7fa4ac108ea0 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:11.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.209+0000 7fa4b0e00640 1 --2- 172.21.15.83:0/1380967433 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa4ac1044b0 0x7fa4ac104890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:11.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.209+0000 7fa4b0e00640 1 -- 172.21.15.83:0/1380967433 >> 172.21.15.83:0/1380967433 conn(0x7fa4ac0ffd90 msgr2=0x7fa4ac1021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:11.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.209+0000 7fa4b0e00640 1 -- 172.21.15.83:0/1380967433 shutdown_connections 2024-04-24T09:00:11.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.210+0000 7fa4b0e00640 1 -- 172.21.15.83:0/1380967433 wait complete. 2024-04-24T09:00:11.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.211+0000 7fa4b0e00640 1 Processor -- start 2024-04-24T09:00:11.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.211+0000 7fa4b0e00640 1 -- start start 2024-04-24T09:00:11.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.211+0000 7fa4b0e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4ac1044b0 0x7fa4ac1a06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:11.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.211+0000 7fa4b0e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa4ac109580 0x7fa4ac1a0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:11.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.211+0000 7fa4b0e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4ac19a840 0x7fa4ac19acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:11.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.211+0000 7fa4b0e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ac1144a0 con 0x7fa4ac19a840 2024-04-24T09:00:11.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.211+0000 7fa4b0e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ac114320 con 0x7fa4ac1044b0 2024-04-24T09:00:11.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.211+0000 7fa4b0e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ac114620 con 0x7fa4ac109580 2024-04-24T09:00:11.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.212+0000 7fa4aa800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4ac19a840 0x7fa4ac19acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:11.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.212+0000 7fa4aa800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4ac19a840 0x7fa4ac19acf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48828/0 (socket says 172.21.15.83:48828) 2024-04-24T09:00:11.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.212+0000 7fa4aa800640 1 -- 172.21.15.83:0/3525314141 learned_addr learned my addr 172.21.15.83:0/3525314141 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:00:11.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.212+0000 7fa4a9e00640 1 --2- 172.21.15.83:0/3525314141 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4ac1044b0 0x7fa4ac1a06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:11.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.212+0000 7fa4a9400640 1 --2- 172.21.15.83:0/3525314141 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa4ac109580 0x7fa4ac1a0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:11.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.212+0000 7fa4aa800640 1 -- 172.21.15.83:0/3525314141 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa4ac109580 msgr2=0x7fa4ac1a0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:11.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.212+0000 7fa4aa800640 1 --2- 172.21.15.83:0/3525314141 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa4ac109580 0x7fa4ac1a0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:11.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.212+0000 7fa4aa800640 1 -- 172.21.15.83:0/3525314141 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4ac1044b0 msgr2=0x7fa4ac1a06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:11.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.212+0000 7fa4aa800640 1 --2- 172.21.15.83:0/3525314141 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4ac1044b0 0x7fa4ac1a06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:11.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.212+0000 7fa4aa800640 1 -- 172.21.15.83:0/3525314141 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4ac19b3d0 con 0x7fa4ac19a840 2024-04-24T09:00:11.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.212+0000 7fa4a9400640 1 --2- 172.21.15.83:0/3525314141 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa4ac109580 0x7fa4ac1a0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:00:11.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.213+0000 7fa4aa800640 1 --2- 172.21.15.83:0/3525314141 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4ac19a840 0x7fa4ac19acf0 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7fa4a400cad0 tx=0x7fa4a400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:11.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.213+0000 7fa4a2a00640 1 -- 172.21.15.83:0/3525314141 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a400ae20 con 0x7fa4ac19a840 2024-04-24T09:00:11.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.213+0000 7fa4a2a00640 1 -- 172.21.15.83:0/3525314141 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa4a4005020 con 0x7fa4ac19a840 2024-04-24T09:00:11.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.213+0000 7fa4a2a00640 1 -- 172.21.15.83:0/3525314141 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a400f6b0 con 0x7fa4ac19a840 2024-04-24T09:00:11.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.213+0000 7fa4b0e00640 1 -- 172.21.15.83:0/3525314141 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4ac19b6c0 con 0x7fa4ac19a840 2024-04-24T09:00:11.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.213+0000 7fa4b0e00640 1 -- 172.21.15.83:0/3525314141 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4ac1a7850 con 0x7fa4ac19a840 2024-04-24T09:00:11.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.216+0000 7fa4b3bff740 1 Processor -- start 2024-04-24T09:00:11.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.217+0000 7fa4a2a00640 1 -- 172.21.15.83:0/3525314141 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa4a40051d0 con 0x7fa4ac19a840 2024-04-24T09:00:11.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.217+0000 7fa4b3bff740 1 -- start start 2024-04-24T09:00:11.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.218+0000 7fa4b3bff740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55930718b950 0x55930718bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:11.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.218+0000 7fa4b3bff740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55930718c270 0x559307196400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:11.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.218+0000 7fa4b3bff740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559307196940 0x559307198d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:11.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.218+0000 7fa4b3bff740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5593070f11d0 con 0x55930718b950 2024-04-24T09:00:11.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.218+0000 7fa4b3bff740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559307097450 con 0x559307196940 2024-04-24T09:00:11.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.218+0000 7fa4b3bff740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5593070ed190 con 0x55930718c270 2024-04-24T09:00:11.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.218+0000 7fa4a9400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55930718b950 0x55930718bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:11.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.218+0000 7fa4a9400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55930718b950 0x55930718bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48838/0 (socket says 172.21.15.83:48838) 2024-04-24T09:00:11.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.218+0000 7fa4a9400640 1 -- 172.21.15.83:0/3979910195 learned_addr learned my addr 172.21.15.83:0/3979910195 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:00:11.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.218+0000 7fa4a9400640 1 --2- 172.21.15.83:0/3979910195 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559307196940 0x559307198d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:11.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.218+0000 7fa4a9e00640 1 --2- 172.21.15.83:0/3979910195 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55930718c270 0x559307196400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:11.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.219+0000 7fa4a9400640 1 -- 172.21.15.83:0/3979910195 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55930718c270 msgr2=0x559307196400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:11.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.219+0000 7fa4a9400640 1 --2- 172.21.15.83:0/3979910195 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55930718c270 0x559307196400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:11.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.219+0000 7fa4a9400640 1 -- 172.21.15.83:0/3979910195 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559307196940 msgr2=0x559307198d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:11.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.219+0000 7fa4a9400640 1 --2- 172.21.15.83:0/3979910195 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559307196940 0x559307198d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:11.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.219+0000 7fa4a9400640 1 -- 172.21.15.83:0/3979910195 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559307154310 con 0x55930718b950 2024-04-24T09:00:11.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.219+0000 7fa4a2a00640 1 --2- 172.21.15.83:0/3525314141 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa4900a3da0 0x7fa4900a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:11.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.219+0000 7fa4a9400640 1 --2- 172.21.15.83:0/3979910195 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55930718b950 0x55930718bd30 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7fa498000c00 tx=0x7fa498009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f60839f75410c71 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:11.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.219+0000 7fa4a2a00640 1 -- 172.21.15.83:0/3525314141 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7fa4a40d0c80 con 0x7fa4ac19a840 2024-04-24T09:00:11.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.219+0000 7fa4a9e00640 1 --2- 172.21.15.83:0/3525314141 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa4900a3da0 0x7fa4900a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:11.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.219+0000 7fa497400640 1 -- 172.21.15.83:0/3979910195 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa498035050 con 0x55930718b950 2024-04-24T09:00:11.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.219+0000 7fa497400640 1 -- 172.21.15.83:0/3979910195 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa4980318e0 con 0x55930718b950 2024-04-24T09:00:11.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.219+0000 7fa497400640 1 -- 172.21.15.83:0/3979910195 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa498004920 con 0x55930718b950 2024-04-24T09:00:11.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.220+0000 7fa4b3bff740 1 -- 172.21.15.83:0/3979910195 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55930718b950 msgr2=0x55930718bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:11.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.220+0000 7fa4a9e00640 1 --2- 172.21.15.83:0/3525314141 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa4900a3da0 0x7fa4900a6260 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x55930718cf10 tx=0x7fa49c005ec0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:11.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.220+0000 7fa4b3bff740 1 --2- 172.21.15.83:0/3979910195 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55930718b950 0x55930718bd30 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7fa498000c00 tx=0x7fa498009560 comp rx=0 tx=0).stop 2024-04-24T09:00:11.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.221+0000 7fa4b3bff740 1 -- 172.21.15.83:0/3979910195 shutdown_connections 2024-04-24T09:00:11.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.221+0000 7fa4b3bff740 1 --2- 172.21.15.83:0/3979910195 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559307196940 0x559307198d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:11.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.221+0000 7fa4b3bff740 1 --2- 172.21.15.83:0/3979910195 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55930718c270 0x559307196400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:11.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.221+0000 7fa4b3bff740 1 --2- 172.21.15.83:0/3979910195 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55930718b950 0x55930718bd30 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:11.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.221+0000 7fa4b3bff740 1 -- 172.21.15.83:0/3979910195 >> 172.21.15.83:0/3979910195 conn(0x559307187c70 msgr2=0x559307188050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:11.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.221+0000 7fa4b3bff740 1 -- 172.21.15.83:0/3979910195 shutdown_connections 2024-04-24T09:00:11.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.221+0000 7fa4b3bff740 1 -- 172.21.15.83:0/3979910195 wait complete. 2024-04-24T09:00:11.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.221+0000 7fa4b3bff740 1 -- start start 2024-04-24T09:00:11.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.221+0000 7fa4b3bff740 1 Processor -- start 2024-04-24T09:00:11.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.222+0000 7fa4b3bff740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55930718b950 0x5593071963f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:11.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.222+0000 7fa4b3bff740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55930718c270 0x5593071936e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:11.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.222+0000 7fa4b3bff740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559307196940 0x559307193e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:11.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.222+0000 7fa4b3bff740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559307199770 con 0x55930718b950 2024-04-24T09:00:11.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.222+0000 7fa4b3bff740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5593070aa810 con 0x559307196940 2024-04-24T09:00:11.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.222+0000 7fa4b3bff740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5593071998f0 con 0x55930718c270 2024-04-24T09:00:11.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.222+0000 7fa4b3bff740 10 client.0 _ll_register_callbacks cb 0x559307078140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:00:11.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.222+0000 7fa4a9e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55930718b950 0x5593071963f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:11.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.222+0000 7fa4a9e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55930718b950 0x5593071963f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48842/0 (socket says 172.21.15.83:48842) 2024-04-24T09:00:11.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.222+0000 7fa4a9e00640 1 -- 172.21.15.83:0/3752017160 learned_addr learned my addr 172.21.15.83:0/3752017160 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:00:11.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.223+0000 7fa4aa800640 1 --2- 172.21.15.83:0/3752017160 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559307196940 0x559307193e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:11.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.223+0000 7fa4a9400640 1 --2- 172.21.15.83:0/3752017160 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55930718c270 0x5593071936e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:11.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.223+0000 7fa4a9e00640 1 -- 172.21.15.83:0/3752017160 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55930718c270 msgr2=0x5593071936e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:11.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.223+0000 7fa4a9e00640 1 --2- 172.21.15.83:0/3752017160 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55930718c270 0x5593071936e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:11.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.223+0000 7fa4a9e00640 1 -- 172.21.15.83:0/3752017160 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559307196940 msgr2=0x559307193e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:11.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.223+0000 7fa4a9e00640 1 --2- 172.21.15.83:0/3752017160 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559307196940 0x559307193e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:11.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.223+0000 7fa4a9e00640 1 -- 172.21.15.83:0/3752017160 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593070bb3f0 con 0x55930718b950 2024-04-24T09:00:11.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.223+0000 7fa4a9400640 1 --2- 172.21.15.83:0/3752017160 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55930718c270 0x5593071936e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:00:11.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.223+0000 7fa4a9e00640 1 --2- 172.21.15.83:0/3752017160 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55930718b950 0x5593071963f0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7fa49c0116c0 tx=0x7fa49c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f6c742654cb1194 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:11.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.223+0000 7fa48be00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:00:11.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.223+0000 7fa48be00640 1 -- 172.21.15.83:0/3752017160 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa49c022070 con 0x55930718b950 2024-04-24T09:00:11.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.223+0000 7fa48be00640 1 -- 172.21.15.83:0/3752017160 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7fa49c010920 con 0x55930718b950 2024-04-24T09:00:11.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.223+0000 7fa4b3bff740 1 -- 172.21.15.83:0/3752017160 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5593070febf0 con 0x55930718b950 2024-04-24T09:00:11.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.224+0000 7fa48be00640 1 -- 172.21.15.83:0/3752017160 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fa49c02e050 con 0x55930718b950 2024-04-24T09:00:11.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.224+0000 7fa4b3bff740 10 client.18720 fetch_fsmap learned FSMap version 35 2024-04-24T09:00:11.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.224+0000 7fa4b3bff740 1 -- 172.21.15.83:0/3752017160 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x559306e033f0 con 0x55930718b950 2024-04-24T09:00:11.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.224+0000 7fa48be00640 1 -- 172.21.15.83:0/3752017160 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x5593070febf0 con 0x55930718b950 2024-04-24T09:00:11.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.224+0000 7fa48be00640 1 -- 172.21.15.83:0/3752017160 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7fa49c01ecc0 con 0x55930718b950 2024-04-24T09:00:11.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.224+0000 7fa4b3bff740 10 client.18720 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T09:00:11.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.224+0000 7fa4b3bff740 10 client.18720 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:00:11.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.224+0000 7fa4b3bff740 20 client.18720 populate_metadata read hostname 'smithi083' 2024-04-24T09:00:11.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.224+0000 7fa4b3bff740 1 --2- 172.21.15.83:0/3752017160 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5593071a6b60 0x5593071a9000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:11.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.225+0000 7fa4b3bff740 4 client.18720 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:00:11.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.225+0000 7fa4b3bff740 1 -- 172.21.15.83:0/3752017160 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5593070f8a60 con 0x5593071a6b60 2024-04-24T09:00:11.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.225+0000 7fa4a9400640 1 --2- 172.21.15.83:0/3752017160 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5593071a6b60 0x5593071a9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:11.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.226+0000 7fa4a9400640 1 --2- 172.21.15.83:0/3752017160 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5593071a6b60 0x5593071a9000 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f71ad19e2710b7b server_cookie=f60fc7a9b3bf83ad in_seq=0 out_seq=0 2024-04-24T09:00:11.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.226+0000 7fa48be00640 10 client.18720 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:00:11.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.229+0000 7fa48be00640 1 -- 172.21.15.83:0/3752017160 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5593070f8a60 con 0x5593071a6b60 2024-04-24T09:00:11.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.229+0000 7fa48be00640 10 client.18720 handle_command_reply: tid=0 2024-04-24T09:00:11.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.229+0000 7fa4b3bff740 1 client.18720 shutdown 2024-04-24T09:00:11.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.230+0000 7fa4b3bff740 1 -- 172.21.15.83:0/3752017160 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55930718b950 msgr2=0x5593071963f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:11.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.230+0000 7fa4b3bff740 1 --2- 172.21.15.83:0/3752017160 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55930718b950 0x5593071963f0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7fa49c0116c0 tx=0x7fa49c004730 comp rx=0 tx=0).stop 2024-04-24T09:00:11.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.230+0000 7fa4b3bff740 1 -- 172.21.15.83:0/3752017160 shutdown_connections 2024-04-24T09:00:11.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.230+0000 7fa4b3bff740 1 --2- 172.21.15.83:0/3752017160 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5593071a6b60 0x5593071a9000 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:11.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.230+0000 7fa4b3bff740 1 --2- 172.21.15.83:0/3752017160 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559307196940 0x559307193e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:11.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.230+0000 7fa4b3bff740 1 --2- 172.21.15.83:0/3752017160 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55930718c270 0x5593071936e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:11.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.230+0000 7fa4b3bff740 1 --2- 172.21.15.83:0/3752017160 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55930718b950 0x5593071963f0 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:11.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.230+0000 7fa4b3bff740 1 -- 172.21.15.83:0/3752017160 >> 172.21.15.83:0/3752017160 conn(0x559307187c70 msgr2=0x55930718da70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:11.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.230+0000 7fa48be00640 0 client.18720 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:00:11.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.230+0000 7fa4b3bff740 1 -- 172.21.15.83:0/3752017160 shutdown_connections 2024-04-24T09:00:11.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.230+0000 7fa4b3bff740 1 -- 172.21.15.83:0/3752017160 wait complete. 2024-04-24T09:00:11.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.230+0000 7fa4b3bff740 20 client.18720 trim_cache size 0 max 16384 2024-04-24T09:00:11.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.246+0000 7fa4b3bff740 1 Processor -- start 2024-04-24T09:00:11.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.246+0000 7fa4b3bff740 1 -- start start 2024-04-24T09:00:11.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.246+0000 7fa4b3bff740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55930718b950 0x5593071a2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:11.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.246+0000 7fa4b3bff740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55930718c270 0x559307195f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:11.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.246+0000 7fa4b3bff740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559307196940 0x5593071ae8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:11.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.246+0000 7fa4b3bff740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ac1b1770 con 0x55930718c270 2024-04-24T09:00:11.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.246+0000 7fa4b3bff740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ac1b32b0 con 0x55930718b950 2024-04-24T09:00:11.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.246+0000 7fa4b3bff740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ac1b1e40 con 0x559307196940 2024-04-24T09:00:11.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.246+0000 7fa4a9400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55930718c270 0x559307195f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:11.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.246+0000 7fa4a9400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55930718c270 0x559307195f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48854/0 (socket says 172.21.15.83:48854) 2024-04-24T09:00:11.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.246+0000 7fa4a9400640 1 -- 172.21.15.83:0/1088860792 learned_addr learned my addr 172.21.15.83:0/1088860792 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:00:11.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.247+0000 7fa4a9e00640 1 --2- 172.21.15.83:0/1088860792 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55930718b950 0x5593071a2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:11.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.247+0000 7fa4aa800640 1 --2- 172.21.15.83:0/1088860792 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559307196940 0x5593071ae8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:11.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.247+0000 7fa4a9400640 1 -- 172.21.15.83:0/1088860792 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559307196940 msgr2=0x5593071ae8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:11.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.247+0000 7fa4a9400640 1 --2- 172.21.15.83:0/1088860792 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559307196940 0x5593071ae8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:11.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.247+0000 7fa4a9400640 1 -- 172.21.15.83:0/1088860792 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55930718b950 msgr2=0x5593071a2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:11.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.247+0000 7fa4a9400640 1 --2- 172.21.15.83:0/1088860792 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55930718b950 0x5593071a2630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:11.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.247+0000 7fa4a9400640 1 -- 172.21.15.83:0/1088860792 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559306ecd7d0 con 0x55930718c270 2024-04-24T09:00:11.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.247+0000 7fa4a9400640 1 --2- 172.21.15.83:0/1088860792 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55930718c270 0x559307195f20 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7fa498005ec0 tx=0x7fa498005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b3da3608a6c4b0c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:11.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.247+0000 7fa496a00640 1 -- 172.21.15.83:0/1088860792 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa498035040 con 0x55930718c270 2024-04-24T09:00:11.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.248+0000 7fa496a00640 1 -- 172.21.15.83:0/1088860792 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa498056070 con 0x55930718c270 2024-04-24T09:00:11.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.248+0000 7fa496a00640 1 -- 172.21.15.83:0/1088860792 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa498050940 con 0x55930718c270 2024-04-24T09:00:11.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.248+0000 7fa4b3bff740 1 -- 172.21.15.83:0/1088860792 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55930718c270 msgr2=0x559307195f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:11.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.248+0000 7fa4b3bff740 1 --2- 172.21.15.83:0/1088860792 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55930718c270 0x559307195f20 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7fa498005ec0 tx=0x7fa498005a20 comp rx=0 tx=0).stop 2024-04-24T09:00:11.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.248+0000 7fa4b3bff740 1 -- 172.21.15.83:0/1088860792 shutdown_connections 2024-04-24T09:00:11.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.248+0000 7fa4b3bff740 1 --2- 172.21.15.83:0/1088860792 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559307196940 0x5593071ae8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:11.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.248+0000 7fa4b3bff740 1 --2- 172.21.15.83:0/1088860792 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55930718c270 0x559307195f20 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:11.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.248+0000 7fa4b3bff740 1 --2- 172.21.15.83:0/1088860792 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55930718b950 0x5593071a2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:11.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.248+0000 7fa4b3bff740 1 -- 172.21.15.83:0/1088860792 >> 172.21.15.83:0/1088860792 conn(0x559307187c70 msgr2=0x55930718e9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:11.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.249+0000 7fa4b3bff740 1 -- 172.21.15.83:0/1088860792 shutdown_connections 2024-04-24T09:00:11.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.249+0000 7fa4b3bff740 1 -- 172.21.15.83:0/1088860792 wait complete. 2024-04-24T09:00:11.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.249+0000 7fa4b3bff740 1 -- start start 2024-04-24T09:00:11.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.249+0000 7fa4b3bff740 1 Processor -- start 2024-04-24T09:00:11.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.250+0000 7fa4b3bff740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55930718b950 0x559307191410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:11.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.250+0000 7fa4b3bff740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55930718c270 0x55930719eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:11.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.250+0000 7fa4b3bff740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559307196940 0x559307197b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:11.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.250+0000 7fa4b3bff740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ac1b2be0 con 0x559307196940 2024-04-24T09:00:11.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.250+0000 7fa4b3bff740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ac1b2510 con 0x55930718b950 2024-04-24T09:00:11.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.250+0000 7fa4b3bff740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ac1af770 con 0x55930718c270 2024-04-24T09:00:11.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.250+0000 7fa4b3bff740 10 client.0 _ll_register_callbacks cb 0x7fa49c015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:00:11.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.250+0000 7fa4aa800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559307196940 0x559307197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:11.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.250+0000 7fa4aa800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559307196940 0x559307197b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48858/0 (socket says 172.21.15.83:48858) 2024-04-24T09:00:11.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.250+0000 7fa4aa800640 1 -- 172.21.15.83:0/2690884019 learned_addr learned my addr 172.21.15.83:0/2690884019 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:00:11.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.250+0000 7fa4a9e00640 1 --2- 172.21.15.83:0/2690884019 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55930718b950 0x559307191410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:11.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.250+0000 7fa4a9400640 1 --2- 172.21.15.83:0/2690884019 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55930718c270 0x55930719eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:11.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.250+0000 7fa4aa800640 1 -- 172.21.15.83:0/2690884019 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55930718c270 msgr2=0x55930719eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:11.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.250+0000 7fa4aa800640 1 --2- 172.21.15.83:0/2690884019 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55930718c270 0x55930719eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:11.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.250+0000 7fa4aa800640 1 -- 172.21.15.83:0/2690884019 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55930718b950 msgr2=0x559307191410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:11.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.250+0000 7fa4aa800640 1 --2- 172.21.15.83:0/2690884019 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55930718b950 0x559307191410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:11.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.250+0000 7fa4aa800640 1 -- 172.21.15.83:0/2690884019 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559307198090 con 0x559307196940 2024-04-24T09:00:11.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.251+0000 7fa4aa800640 1 --2- 172.21.15.83:0/2690884019 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559307196940 0x559307197b50 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7fa4a409c790 tx=0x7fa4a409cfc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=590188388327abba server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:11.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.251+0000 7fa48be00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:00:11.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.251+0000 7fa48be00640 1 -- 172.21.15.83:0/2690884019 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a401d040 con 0x559307196940 2024-04-24T09:00:11.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.251+0000 7fa48be00640 1 -- 172.21.15.83:0/2690884019 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7fa4a4098e70 con 0x559307196940 2024-04-24T09:00:11.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.251+0000 7fa4b3bff740 1 -- 172.21.15.83:0/2690884019 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa498056070 con 0x559307196940 2024-04-24T09:00:11.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.251+0000 7fa48be00640 1 -- 172.21.15.83:0/2690884019 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fa4a40a5470 con 0x559307196940 2024-04-24T09:00:11.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.251+0000 7fa4b3bff740 10 client.18726 fetch_fsmap learned FSMap version 35 2024-04-24T09:00:11.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.251+0000 7fa4b3bff740 1 -- 172.21.15.83:0/2690884019 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55930720d850 con 0x559307196940 2024-04-24T09:00:11.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.252+0000 7fa48be00640 1 -- 172.21.15.83:0/2690884019 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa498056070 con 0x559307196940 2024-04-24T09:00:11.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.252+0000 7fa48be00640 1 -- 172.21.15.83:0/2690884019 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7fa4a4098670 con 0x559307196940 2024-04-24T09:00:11.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.252+0000 7fa4b3bff740 10 client.18726 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T09:00:11.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.252+0000 7fa4b3bff740 10 client.18726 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:00:11.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.252+0000 7fa4b3bff740 20 client.18726 populate_metadata read hostname 'smithi083' 2024-04-24T09:00:11.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.252+0000 7fa4b3bff740 1 --2- 172.21.15.83:0/2690884019 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55930720dfc0 0x5593072103b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:11.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.252+0000 7fa4b3bff740 4 client.18726 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:00:11.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.252+0000 7fa4b3bff740 1 -- 172.21.15.83:0/2690884019 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5593070744d0 con 0x55930720dfc0 2024-04-24T09:00:11.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.252+0000 7fa4a9e00640 1 --2- 172.21.15.83:0/2690884019 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55930720dfc0 0x5593072103b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:11.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.253+0000 7fa4a9e00640 1 --2- 172.21.15.83:0/2690884019 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55930720dfc0 0x5593072103b0 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a9132ff1b11e9ac server_cookie=2d14c1a5a7185be1 in_seq=0 out_seq=0 2024-04-24T09:00:11.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.253+0000 7fa48be00640 10 client.18726 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:00:11.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.256+0000 7fa48be00640 1 -- 172.21.15.83:0/2690884019 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5593070744d0 con 0x55930720dfc0 2024-04-24T09:00:11.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.256+0000 7fa48be00640 10 client.18726 handle_command_reply: tid=0 2024-04-24T09:00:11.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.256+0000 7fa4b3bff740 1 client.18726 shutdown 2024-04-24T09:00:11.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.256+0000 7fa4b3bff740 1 -- 172.21.15.83:0/2690884019 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559307196940 msgr2=0x559307197b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:11.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.256+0000 7fa4b3bff740 1 --2- 172.21.15.83:0/2690884019 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559307196940 0x559307197b50 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7fa4a409c790 tx=0x7fa4a409cfc0 comp rx=0 tx=0).stop 2024-04-24T09:00:11.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.256+0000 7fa4b3bff740 1 -- 172.21.15.83:0/2690884019 shutdown_connections 2024-04-24T09:00:11.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.256+0000 7fa4b3bff740 1 --2- 172.21.15.83:0/2690884019 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55930720dfc0 0x5593072103b0 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:11.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.256+0000 7fa4b3bff740 1 --2- 172.21.15.83:0/2690884019 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559307196940 0x559307197b50 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:11.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.256+0000 7fa4b3bff740 1 --2- 172.21.15.83:0/2690884019 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55930718c270 0x55930719eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:11.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.256+0000 7fa4b3bff740 1 --2- 172.21.15.83:0/2690884019 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55930718b950 0x559307191410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:11.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.256+0000 7fa48be00640 0 client.18726 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:00:11.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.256+0000 7fa4b3bff740 1 -- 172.21.15.83:0/2690884019 >> 172.21.15.83:0/2690884019 conn(0x559307187c70 msgr2=0x559307186560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:11.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.257+0000 7fa4b3bff740 1 -- 172.21.15.83:0/2690884019 shutdown_connections 2024-04-24T09:00:11.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.257+0000 7fa4b3bff740 1 -- 172.21.15.83:0/2690884019 wait complete. 2024-04-24T09:00:11.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.257+0000 7fa4b3bff740 20 client.18726 trim_cache size 0 max 16384 2024-04-24T09:00:11.258 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:00:11.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.257+0000 7fa4b0e00640 1 -- 172.21.15.83:0/3525314141 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa4900a3da0 msgr2=0x7fa4900a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:11.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.257+0000 7fa4b0e00640 1 --2- 172.21.15.83:0/3525314141 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa4900a3da0 0x7fa4900a6260 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x55930718cf10 tx=0x7fa49c005ec0 comp rx=0 tx=0).stop 2024-04-24T09:00:11.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.257+0000 7fa4b0e00640 1 -- 172.21.15.83:0/3525314141 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4ac19a840 msgr2=0x7fa4ac19acf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:11.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.257+0000 7fa4b0e00640 1 --2- 172.21.15.83:0/3525314141 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4ac19a840 0x7fa4ac19acf0 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7fa4a400cad0 tx=0x7fa4a400cf90 comp rx=0 tx=0).stop 2024-04-24T09:00:11.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.258+0000 7fa4b0e00640 1 -- 172.21.15.83:0/3525314141 shutdown_connections 2024-04-24T09:00:11.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.258+0000 7fa4b0e00640 1 --2- 172.21.15.83:0/3525314141 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa4900a3da0 0x7fa4900a6260 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:11.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.258+0000 7fa4b0e00640 1 --2- 172.21.15.83:0/3525314141 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa4ac19a840 0x7fa4ac19acf0 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:11.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.258+0000 7fa4b0e00640 1 --2- 172.21.15.83:0/3525314141 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa4ac109580 0x7fa4ac1a0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:11.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.258+0000 7fa4b0e00640 1 --2- 172.21.15.83:0/3525314141 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4ac1044b0 0x7fa4ac1a06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:11.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.258+0000 7fa4b0e00640 1 -- 172.21.15.83:0/3525314141 >> 172.21.15.83:0/3525314141 conn(0x7fa4ac0ffd90 msgr2=0x7fa4ac1010f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:11.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.258+0000 7fa4b0e00640 1 -- 172.21.15.83:0/3525314141 shutdown_connections 2024-04-24T09:00:11.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:11.258+0000 7fa4b0e00640 1 -- 172.21.15.83:0/3525314141 wait complete. 2024-04-24T09:00:11.598 INFO:tasks.workunit.client.0.smithi083.stdout: 1 48233 0.62 MB/sec execute 285 sec latency 57.929 ms 2024-04-24T09:00:11.954 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:00:11 smithi138 ceph-mon[31441]: pgmap v268: 129 pgs: 129 active+clean; 49 MiB data, 729 MiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 260 KiB/s wr, 32 op/s 2024-04-24T09:00:12.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:00:11 smithi083 ceph-mon[27907]: pgmap v268: 129 pgs: 129 active+clean; 49 MiB data, 729 MiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 260 KiB/s wr, 32 op/s 2024-04-24T09:00:12.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:00:11 smithi189 ceph-mon[31791]: pgmap v268: 129 pgs: 129 active+clean; 49 MiB data, 729 MiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 260 KiB/s wr, 32 op/s 2024-04-24T09:00:12.214 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.215+0000 7f3e7f200640 1 -- 172.21.15.83:0/112188616 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e78100e10 msgr2=0x7f3e781011f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:12.214 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.215+0000 7f3e7f200640 1 --2- 172.21.15.83:0/112188616 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e78100e10 0x7f3e781011f0 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f3e68009ca0 tx=0x7f3e6802a8c0 comp rx=0 tx=0).stop 2024-04-24T09:00:12.215 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.215+0000 7f3e7f200640 1 -- 172.21.15.83:0/112188616 shutdown_connections 2024-04-24T09:00:12.215 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.215+0000 7f3e7f200640 1 --2- 172.21.15.83:0/112188616 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3e7810e9c0 0x7f3e78110db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:12.215 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.215+0000 7f3e7f200640 1 --2- 172.21.15.83:0/112188616 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3e78101730 0x7f3e7810e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:12.215 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.215+0000 7f3e7f200640 1 --2- 172.21.15.83:0/112188616 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e78100e10 0x7f3e781011f0 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:12.215 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.215+0000 7f3e7f200640 1 -- 172.21.15.83:0/112188616 >> 172.21.15.83:0/112188616 conn(0x7f3e78078080 msgr2=0x7f3e78078490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:12.215 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.215+0000 7f3e7f200640 1 -- 172.21.15.83:0/112188616 shutdown_connections 2024-04-24T09:00:12.215 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.215+0000 7f3e7f200640 1 -- 172.21.15.83:0/112188616 wait complete. 2024-04-24T09:00:12.216 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.216+0000 7f3e7f200640 1 Processor -- start 2024-04-24T09:00:12.217 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.217+0000 7f3e7f200640 1 -- start start 2024-04-24T09:00:12.217 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.217+0000 7f3e7f200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e78100e10 0x7f3e781a0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:12.217 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.217+0000 7f3e7f200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3e78101730 0x7f3e781a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:12.217 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.217+0000 7f3e7f200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3e7810e9c0 0x7f3e7819a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:12.217 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.217+0000 7f3e7f200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3e78114480 con 0x7f3e78100e10 2024-04-24T09:00:12.217 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.217+0000 7f3e7f200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3e78114300 con 0x7f3e7810e9c0 2024-04-24T09:00:12.217 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.217+0000 7f3e7f200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3e78114600 con 0x7f3e78101730 2024-04-24T09:00:12.217 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.217+0000 7f3e77e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e78100e10 0x7f3e781a0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:12.217 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.217+0000 7f3e77e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e78100e10 0x7f3e781a0730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48882/0 (socket says 172.21.15.83:48882) 2024-04-24T09:00:12.217 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.217+0000 7f3e77e00640 1 -- 172.21.15.83:0/3256499343 learned_addr learned my addr 172.21.15.83:0/3256499343 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:00:12.217 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.218+0000 7f3e77400640 1 --2- 172.21.15.83:0/3256499343 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3e78101730 0x7f3e781a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:12.217 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.218+0000 7f3e7d200640 1 --2- 172.21.15.83:0/3256499343 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3e7810e9c0 0x7f3e7819a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:12.217 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.218+0000 7f3e77e00640 1 -- 172.21.15.83:0/3256499343 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3e78101730 msgr2=0x7f3e781a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:12.218 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.218+0000 7f3e77e00640 1 --2- 172.21.15.83:0/3256499343 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3e78101730 0x7f3e781a0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:12.218 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.218+0000 7f3e77e00640 1 -- 172.21.15.83:0/3256499343 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3e7810e9c0 msgr2=0x7f3e7819a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:12.218 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.218+0000 7f3e77e00640 1 --2- 172.21.15.83:0/3256499343 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3e7810e9c0 0x7f3e7819a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:12.218 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.218+0000 7f3e77e00640 1 -- 172.21.15.83:0/3256499343 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e7819b060 con 0x7f3e78100e10 2024-04-24T09:00:12.218 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.218+0000 7f3e77400640 1 --2- 172.21.15.83:0/3256499343 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3e78101730 0x7f3e781a0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:00:12.218 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.218+0000 7f3e7d200640 1 --2- 172.21.15.83:0/3256499343 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3e7810e9c0 0x7f3e7819a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:00:12.219 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.218+0000 7f3e77e00640 1 --2- 172.21.15.83:0/3256499343 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e78100e10 0x7f3e781a0730 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f3e68006010 tx=0x7f3e68003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:12.219 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.218+0000 7f3e74c00640 1 -- 172.21.15.83:0/3256499343 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e68004040 con 0x7f3e78100e10 2024-04-24T09:00:12.219 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.218+0000 7f3e74c00640 1 -- 172.21.15.83:0/3256499343 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e68004690 con 0x7f3e78100e10 2024-04-24T09:00:12.219 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.218+0000 7f3e7f200640 1 -- 172.21.15.83:0/3256499343 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e7819b2f0 con 0x7f3e78100e10 2024-04-24T09:00:12.219 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.218+0000 7f3e74c00640 1 -- 172.21.15.83:0/3256499343 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e6803c410 con 0x7f3e78100e10 2024-04-24T09:00:12.219 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.219+0000 7f3e7f200640 1 -- 172.21.15.83:0/3256499343 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e781a75f0 con 0x7f3e78100e10 2024-04-24T09:00:12.221 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.222+0000 7f3e82169740 1 Processor -- start 2024-04-24T09:00:12.221 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.222+0000 7f3e74c00640 1 -- 172.21.15.83:0/3256499343 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3e680305c0 con 0x7f3e78100e10 2024-04-24T09:00:12.221 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.222+0000 7f3e82169740 1 -- start start 2024-04-24T09:00:12.223 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.223+0000 7f3e82169740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55afdc8c01b0 0x55afdc8c0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:12.223 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.223+0000 7f3e82169740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55afdc8c0ad0 0x55afdc8cae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:12.223 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.223+0000 7f3e82169740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55afdc8cb390 0x55afdc8cd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:12.223 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.223+0000 7f3e82169740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55afdc826900 con 0x55afdc8c0ad0 2024-04-24T09:00:12.223 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.223+0000 7f3e82169740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55afdc7ccf20 con 0x55afdc8c01b0 2024-04-24T09:00:12.223 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.223+0000 7f3e82169740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55afdc8228c0 con 0x55afdc8cb390 2024-04-24T09:00:12.223 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.223+0000 7f3e77e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55afdc8c0ad0 0x55afdc8cae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:12.223 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.223+0000 7f3e77e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55afdc8c0ad0 0x55afdc8cae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48888/0 (socket says 172.21.15.83:48888) 2024-04-24T09:00:12.223 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.223+0000 7f3e77e00640 1 -- 172.21.15.83:0/2316539861 learned_addr learned my addr 172.21.15.83:0/2316539861 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:00:12.224 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.223+0000 7f3e7d200640 1 --2- 172.21.15.83:0/2316539861 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55afdc8c01b0 0x55afdc8c0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:12.224 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.223+0000 7f3e77400640 1 --2- 172.21.15.83:0/2316539861 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55afdc8cb390 0x55afdc8cd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:12.224 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.223+0000 7f3e77e00640 1 -- 172.21.15.83:0/2316539861 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55afdc8cb390 msgr2=0x55afdc8cd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:12.224 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.223+0000 7f3e77e00640 1 --2- 172.21.15.83:0/2316539861 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55afdc8cb390 0x55afdc8cd780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:12.224 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.223+0000 7f3e77e00640 1 -- 172.21.15.83:0/2316539861 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55afdc8c01b0 msgr2=0x55afdc8c0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:12.224 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.223+0000 7f3e77e00640 1 --2- 172.21.15.83:0/2316539861 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55afdc8c01b0 0x55afdc8c0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:12.224 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.224+0000 7f3e77e00640 1 -- 172.21.15.83:0/2316539861 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55afdc7f1e10 con 0x55afdc8c0ad0 2024-04-24T09:00:12.224 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.224+0000 7f3e74c00640 1 --2- 172.21.15.83:0/3256499343 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f3e5c0a3dc0 0x7f3e5c0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:12.224 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.224+0000 7f3e77e00640 1 --2- 172.21.15.83:0/2316539861 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55afdc8c0ad0 0x55afdc8cae50 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f3e680ef520 tx=0x7f3e680fa320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=587baf6c5e869c39 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:12.224 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.224+0000 7f3e74c00640 1 -- 172.21.15.83:0/3256499343 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f3e680f2200 con 0x7f3e78100e10 2024-04-24T09:00:12.224 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.224+0000 7f3e7d200640 1 --2- 172.21.15.83:0/3256499343 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f3e5c0a3dc0 0x7f3e5c0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:12.227 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.224+0000 7f3e6d600640 1 -- 172.21.15.83:0/2316539861 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e680ba0d0 con 0x55afdc8c0ad0 2024-04-24T09:00:12.227 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.224+0000 7f3e6d600640 1 -- 172.21.15.83:0/2316539861 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e68036020 con 0x55afdc8c0ad0 2024-04-24T09:00:12.227 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.224+0000 7f3e6d600640 1 -- 172.21.15.83:0/2316539861 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e680bd790 con 0x55afdc8c0ad0 2024-04-24T09:00:12.228 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.225+0000 7f3e82169740 1 -- 172.21.15.83:0/2316539861 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55afdc8c0ad0 msgr2=0x55afdc8cae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:12.228 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.225+0000 7f3e7d200640 1 --2- 172.21.15.83:0/3256499343 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f3e5c0a3dc0 0x7f3e5c0a6280 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3e70005fd0 tx=0x7f3e7000b340 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:12.228 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.225+0000 7f3e82169740 1 --2- 172.21.15.83:0/2316539861 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55afdc8c0ad0 0x55afdc8cae50 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f3e680ef520 tx=0x7f3e680fa320 comp rx=0 tx=0).stop 2024-04-24T09:00:12.228 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.225+0000 7f3e82169740 1 -- 172.21.15.83:0/2316539861 shutdown_connections 2024-04-24T09:00:12.228 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.225+0000 7f3e82169740 1 --2- 172.21.15.83:0/2316539861 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55afdc8cb390 0x55afdc8cd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:12.228 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.226+0000 7f3e82169740 1 --2- 172.21.15.83:0/2316539861 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55afdc8c0ad0 0x55afdc8cae50 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:12.228 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.226+0000 7f3e82169740 1 --2- 172.21.15.83:0/2316539861 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55afdc8c01b0 0x55afdc8c0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:12.228 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.226+0000 7f3e82169740 1 -- 172.21.15.83:0/2316539861 >> 172.21.15.83:0/2316539861 conn(0x55afdc8bc6c0 msgr2=0x55afdc8bcaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:12.228 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.226+0000 7f3e82169740 1 -- 172.21.15.83:0/2316539861 shutdown_connections 2024-04-24T09:00:12.228 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.226+0000 7f3e82169740 1 -- 172.21.15.83:0/2316539861 wait complete. 2024-04-24T09:00:12.228 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.226+0000 7f3e82169740 1 -- start start 2024-04-24T09:00:12.228 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.226+0000 7f3e82169740 1 Processor -- start 2024-04-24T09:00:12.228 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.227+0000 7f3e82169740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55afdc8c01b0 0x55afdc8c8250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:12.228 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.227+0000 7f3e82169740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55afdc8c0ad0 0x55afdc8c8790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:12.229 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.227+0000 7f3e82169740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55afdc8cb390 0x55afdc8c8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:12.229 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.227+0000 7f3e82169740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55afdc826900 con 0x55afdc8c0ad0 2024-04-24T09:00:12.229 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.227+0000 7f3e82169740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55afdc7e02c0 con 0x55afdc8c01b0 2024-04-24T09:00:12.229 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.227+0000 7f3e82169740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55afdc8cdd80 con 0x55afdc8cb390 2024-04-24T09:00:12.229 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.227+0000 7f3e82169740 10 client.0 _ll_register_callbacks cb 0x55afdc538a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:00:12.229 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.227+0000 7f3e77400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55afdc8c0ad0 0x55afdc8c8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:12.229 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.227+0000 7f3e77400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55afdc8c0ad0 0x55afdc8c8790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48890/0 (socket says 172.21.15.83:48890) 2024-04-24T09:00:12.229 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.227+0000 7f3e77400640 1 -- 172.21.15.83:0/514753714 learned_addr learned my addr 172.21.15.83:0/514753714 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:00:12.229 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.227+0000 7f3e77e00640 1 --2- 172.21.15.83:0/514753714 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55afdc8c01b0 0x55afdc8c8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:12.229 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.227+0000 7f3e7d200640 1 --2- 172.21.15.83:0/514753714 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55afdc8cb390 0x55afdc8c8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:12.229 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.228+0000 7f3e77400640 1 -- 172.21.15.83:0/514753714 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55afdc8cb390 msgr2=0x55afdc8c8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:12.229 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.228+0000 7f3e77400640 1 --2- 172.21.15.83:0/514753714 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55afdc8cb390 0x55afdc8c8cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:12.229 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.228+0000 7f3e77400640 1 -- 172.21.15.83:0/514753714 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55afdc8c01b0 msgr2=0x55afdc8c8250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:12.229 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.228+0000 7f3e77400640 1 --2- 172.21.15.83:0/514753714 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55afdc8c01b0 0x55afdc8c8250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:12.229 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.228+0000 7f3e77400640 1 -- 172.21.15.83:0/514753714 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55afdc7f0fd0 con 0x55afdc8c0ad0 2024-04-24T09:00:12.230 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.228+0000 7f3e77400640 1 --2- 172.21.15.83:0/514753714 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55afdc8c0ad0 0x55afdc8c8790 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f3e6000f740 tx=0x7f3e6000fc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76fca75f2eb00267 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:12.230 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.228+0000 7f3e66000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:00:12.230 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.228+0000 7f3e66000640 1 -- 172.21.15.83:0/514753714 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e60016070 con 0x55afdc8c0ad0 2024-04-24T09:00:12.230 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.228+0000 7f3e66000640 1 -- 172.21.15.83:0/514753714 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f3e6000bd10 con 0x55afdc8c0ad0 2024-04-24T09:00:12.230 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.228+0000 7f3e82169740 1 -- 172.21.15.83:0/514753714 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e68036020 con 0x55afdc8c0ad0 2024-04-24T09:00:12.231 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.229+0000 7f3e66000640 1 -- 172.21.15.83:0/514753714 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f3e6000fe50 con 0x55afdc8c0ad0 2024-04-24T09:00:12.231 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.229+0000 7f3e82169740 10 client.18738 fetch_fsmap learned FSMap version 35 2024-04-24T09:00:12.231 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.229+0000 7f3e82169740 1 -- 172.21.15.83:0/514753714 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55afdc539210 con 0x55afdc8c0ad0 2024-04-24T09:00:12.231 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.229+0000 7f3e66000640 1 -- 172.21.15.83:0/514753714 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e68036020 con 0x55afdc8c0ad0 2024-04-24T09:00:12.232 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.229+0000 7f3e66000640 1 -- 172.21.15.83:0/514753714 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7f3e6001a550 con 0x55afdc8c0ad0 2024-04-24T09:00:12.232 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.229+0000 7f3e82169740 10 client.18738 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T09:00:12.232 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.229+0000 7f3e82169740 10 client.18738 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:00:12.232 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.230+0000 7f3e82169740 20 client.18738 populate_metadata read hostname 'smithi083' 2024-04-24T09:00:12.232 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.230+0000 7f3e82169740 1 --2- 172.21.15.83:0/514753714 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55afdc8db0e0 0x55afdc8dd580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:12.232 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.230+0000 7f3e82169740 4 client.18738 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:00:12.232 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.230+0000 7f3e82169740 1 -- 172.21.15.83:0/514753714 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55afdc834320 con 0x55afdc8db0e0 2024-04-24T09:00:12.232 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.230+0000 7f3e77e00640 1 --2- 172.21.15.83:0/514753714 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55afdc8db0e0 0x55afdc8dd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:12.232 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.231+0000 7f3e77e00640 1 --2- 172.21.15.83:0/514753714 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55afdc8db0e0 0x55afdc8dd580 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ccce030c9f250c5 server_cookie=5ddb657e665c78f0 in_seq=0 out_seq=0 2024-04-24T09:00:12.232 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.231+0000 7f3e66000640 10 client.18738 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:00:12.233 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.234+0000 7f3e66000640 1 -- 172.21.15.83:0/514753714 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55afdc834320 con 0x55afdc8db0e0 2024-04-24T09:00:12.233 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.234+0000 7f3e66000640 10 client.18738 handle_command_reply: tid=0 2024-04-24T09:00:12.233 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.234+0000 7f3e82169740 1 client.18738 shutdown 2024-04-24T09:00:12.234 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.235+0000 7f3e82169740 1 -- 172.21.15.83:0/514753714 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55afdc8c0ad0 msgr2=0x55afdc8c8790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:12.234 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.235+0000 7f3e82169740 1 --2- 172.21.15.83:0/514753714 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55afdc8c0ad0 0x55afdc8c8790 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f3e6000f740 tx=0x7f3e6000fc00 comp rx=0 tx=0).stop 2024-04-24T09:00:12.234 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.235+0000 7f3e82169740 1 -- 172.21.15.83:0/514753714 shutdown_connections 2024-04-24T09:00:12.234 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.235+0000 7f3e82169740 1 --2- 172.21.15.83:0/514753714 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55afdc8db0e0 0x55afdc8dd580 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:12.235 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.235+0000 7f3e82169740 1 --2- 172.21.15.83:0/514753714 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55afdc8cb390 0x55afdc8c8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:12.235 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.235+0000 7f3e82169740 1 --2- 172.21.15.83:0/514753714 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55afdc8c0ad0 0x55afdc8c8790 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:12.235 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.235+0000 7f3e82169740 1 --2- 172.21.15.83:0/514753714 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55afdc8c01b0 0x55afdc8c8250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:12.235 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.235+0000 7f3e66000640 0 client.18738 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:00:12.235 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.235+0000 7f3e82169740 1 -- 172.21.15.83:0/514753714 >> 172.21.15.83:0/514753714 conn(0x55afdc8bc6c0 msgr2=0x55afdc8c2440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:12.235 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.235+0000 7f3e82169740 1 -- 172.21.15.83:0/514753714 shutdown_connections 2024-04-24T09:00:12.235 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.235+0000 7f3e82169740 1 -- 172.21.15.83:0/514753714 wait complete. 2024-04-24T09:00:12.235 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.235+0000 7f3e82169740 20 client.18738 trim_cache size 0 max 16384 2024-04-24T09:00:12.251 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.253+0000 7f3e82169740 1 Processor -- start 2024-04-24T09:00:12.251 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.253+0000 7f3e82169740 1 -- start start 2024-04-24T09:00:12.252 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.253+0000 7f3e82169740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55afdc8c01b0 0x55afdc8c2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:12.252 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.253+0000 7f3e82169740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55afdc8c0ad0 0x55afdc8d69a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:12.252 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.253+0000 7f3e82169740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55afdc8cb390 0x55afdc8c8920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:12.253 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.253+0000 7f3e82169740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3e781b1790 con 0x55afdc8cb390 2024-04-24T09:00:12.253 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.253+0000 7f3e82169740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3e781b32d0 con 0x55afdc8c0ad0 2024-04-24T09:00:12.253 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.253+0000 7f3e82169740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3e781b1e60 con 0x55afdc8c01b0 2024-04-24T09:00:12.253 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.253+0000 7f3e7d200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55afdc8cb390 0x55afdc8c8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:12.253 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.254+0000 7f3e7d200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55afdc8cb390 0x55afdc8c8920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48898/0 (socket says 172.21.15.83:48898) 2024-04-24T09:00:12.253 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.254+0000 7f3e7d200640 1 -- 172.21.15.83:0/4206556325 learned_addr learned my addr 172.21.15.83:0/4206556325 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:00:12.255 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.254+0000 7f3e77400640 1 --2- 172.21.15.83:0/4206556325 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55afdc8c0ad0 0x55afdc8d69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:12.255 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.254+0000 7f3e77e00640 1 --2- 172.21.15.83:0/4206556325 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55afdc8c01b0 0x55afdc8c2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:12.255 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.254+0000 7f3e7d200640 1 -- 172.21.15.83:0/4206556325 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55afdc8c01b0 msgr2=0x55afdc8c2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:12.255 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.254+0000 7f3e7d200640 1 --2- 172.21.15.83:0/4206556325 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55afdc8c01b0 0x55afdc8c2bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:12.255 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.254+0000 7f3e7d200640 1 -- 172.21.15.83:0/4206556325 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55afdc8c0ad0 msgr2=0x55afdc8d69a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:12.255 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.254+0000 7f3e7d200640 1 --2- 172.21.15.83:0/4206556325 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55afdc8c0ad0 0x55afdc8d69a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:12.255 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.254+0000 7f3e7d200640 1 -- 172.21.15.83:0/4206556325 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55afdc605620 con 0x55afdc8cb390 2024-04-24T09:00:12.255 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.254+0000 7f3e77400640 1 --2- 172.21.15.83:0/4206556325 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55afdc8c0ad0 0x55afdc8d69a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:00:12.255 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.254+0000 7f3e77e00640 1 --2- 172.21.15.83:0/4206556325 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55afdc8c01b0 0x55afdc8c2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:00:12.255 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.254+0000 7f3e7d200640 1 --2- 172.21.15.83:0/4206556325 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55afdc8cb390 0x55afdc8c8920 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f3e70010e50 tx=0x7f3e70013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caf4a73f5c513787 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:12.255 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.255+0000 7f3e6cc00640 1 -- 172.21.15.83:0/4206556325 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e70027430 con 0x55afdc8cb390 2024-04-24T09:00:12.255 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.255+0000 7f3e6cc00640 1 -- 172.21.15.83:0/4206556325 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e7001e8e0 con 0x55afdc8cb390 2024-04-24T09:00:12.255 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.255+0000 7f3e6cc00640 1 -- 172.21.15.83:0/4206556325 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e700268b0 con 0x55afdc8cb390 2024-04-24T09:00:12.256 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.255+0000 7f3e82169740 1 -- 172.21.15.83:0/4206556325 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55afdc8cb390 msgr2=0x55afdc8c8920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:12.256 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.255+0000 7f3e82169740 1 --2- 172.21.15.83:0/4206556325 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55afdc8cb390 0x55afdc8c8920 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f3e70010e50 tx=0x7f3e70013000 comp rx=0 tx=0).stop 2024-04-24T09:00:12.256 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.256+0000 7f3e82169740 1 -- 172.21.15.83:0/4206556325 shutdown_connections 2024-04-24T09:00:12.256 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.256+0000 7f3e82169740 1 --2- 172.21.15.83:0/4206556325 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55afdc8cb390 0x55afdc8c8920 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:12.256 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.256+0000 7f3e82169740 1 --2- 172.21.15.83:0/4206556325 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55afdc8c0ad0 0x55afdc8d69a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:12.256 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.256+0000 7f3e82169740 1 --2- 172.21.15.83:0/4206556325 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55afdc8c01b0 0x55afdc8c2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:12.256 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.256+0000 7f3e82169740 1 -- 172.21.15.83:0/4206556325 >> 172.21.15.83:0/4206556325 conn(0x55afdc8bc6c0 msgr2=0x55afdc8c8210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:12.256 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.256+0000 7f3e82169740 1 -- 172.21.15.83:0/4206556325 shutdown_connections 2024-04-24T09:00:12.256 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.256+0000 7f3e82169740 1 -- 172.21.15.83:0/4206556325 wait complete. 2024-04-24T09:00:12.259 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.256+0000 7f3e82169740 1 -- start start 2024-04-24T09:00:12.259 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.256+0000 7f3e82169740 1 Processor -- start 2024-04-24T09:00:12.259 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.257+0000 7f3e82169740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55afdc8c01b0 0x55afdc8d2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:12.259 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.257+0000 7f3e82169740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55afdc8c0ad0 0x55afdc8c4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:12.259 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.257+0000 7f3e82169740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55afdc8cb390 0x55afdc8c4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:12.259 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.257+0000 7f3e82169740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3e781b2c00 con 0x55afdc8c01b0 2024-04-24T09:00:12.259 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.257+0000 7f3e82169740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3e781b2530 con 0x55afdc8cb390 2024-04-24T09:00:12.259 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.257+0000 7f3e82169740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3e781af790 con 0x55afdc8c0ad0 2024-04-24T09:00:12.259 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.257+0000 7f3e82169740 10 client.0 _ll_register_callbacks cb 0x7f3e6000d360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:00:12.260 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.257+0000 7f3e77e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55afdc8c01b0 0x55afdc8d2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:12.260 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.257+0000 7f3e77e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55afdc8c01b0 0x55afdc8d2fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48900/0 (socket says 172.21.15.83:48900) 2024-04-24T09:00:12.260 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.257+0000 7f3e77e00640 1 -- 172.21.15.83:0/511825515 learned_addr learned my addr 172.21.15.83:0/511825515 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:00:12.260 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.257+0000 7f3e7d200640 1 --2- 172.21.15.83:0/511825515 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55afdc8cb390 0x55afdc8c4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:12.260 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.257+0000 7f3e77400640 1 --2- 172.21.15.83:0/511825515 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55afdc8c0ad0 0x55afdc8c4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:12.260 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.257+0000 7f3e77e00640 1 -- 172.21.15.83:0/511825515 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55afdc8c0ad0 msgr2=0x55afdc8c4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:12.260 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.257+0000 7f3e77e00640 1 --2- 172.21.15.83:0/511825515 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55afdc8c0ad0 0x55afdc8c4aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:12.260 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.257+0000 7f3e77e00640 1 -- 172.21.15.83:0/511825515 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55afdc8cb390 msgr2=0x55afdc8c4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:12.260 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.257+0000 7f3e77e00640 1 --2- 172.21.15.83:0/511825515 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55afdc8cb390 0x55afdc8c4fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:12.260 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.257+0000 7f3e77e00640 1 -- 172.21.15.83:0/511825515 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55afdc8c5520 con 0x55afdc8c01b0 2024-04-24T09:00:12.260 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.258+0000 7f3e7d200640 1 --2- 172.21.15.83:0/511825515 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55afdc8cb390 0x55afdc8c4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:00:12.260 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.258+0000 7f3e77e00640 1 --2- 172.21.15.83:0/511825515 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55afdc8c01b0 0x55afdc8d2fc0 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f3e6802ade0 tx=0x7f3e6803e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abf184df4ad14305 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:12.260 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.258+0000 7f3e66000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:00:12.260 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.258+0000 7f3e66000640 1 -- 172.21.15.83:0/511825515 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e680bd970 con 0x55afdc8c01b0 2024-04-24T09:00:12.260 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.258+0000 7f3e66000640 1 -- 172.21.15.83:0/511825515 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f3e680bbed0 con 0x55afdc8c01b0 2024-04-24T09:00:12.261 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.258+0000 7f3e82169740 1 -- 172.21.15.83:0/511825515 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e7001e8e0 con 0x55afdc8c01b0 2024-04-24T09:00:12.261 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.258+0000 7f3e66000640 1 -- 172.21.15.83:0/511825515 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f3e680bdc00 con 0x55afdc8c01b0 2024-04-24T09:00:12.261 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.258+0000 7f3e82169740 10 client.18744 fetch_fsmap learned FSMap version 35 2024-04-24T09:00:12.261 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.258+0000 7f3e82169740 1 -- 172.21.15.83:0/511825515 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55afdc941370 con 0x55afdc8c01b0 2024-04-24T09:00:12.261 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.258+0000 7f3e66000640 1 -- 172.21.15.83:0/511825515 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e7001e8e0 con 0x55afdc8c01b0 2024-04-24T09:00:12.261 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.259+0000 7f3e66000640 1 -- 172.21.15.83:0/511825515 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7f3e680d7200 con 0x55afdc8c01b0 2024-04-24T09:00:12.261 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.259+0000 7f3e82169740 10 client.18744 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T09:00:12.261 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.259+0000 7f3e82169740 10 client.18744 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:00:12.261 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.259+0000 7f3e82169740 20 client.18744 populate_metadata read hostname 'smithi083' 2024-04-24T09:00:12.261 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.259+0000 7f3e82169740 1 --2- 172.21.15.83:0/511825515 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55afdc941a30 0x55afdc943e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:12.261 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.259+0000 7f3e82169740 4 client.18744 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T09:00:12.261 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.259+0000 7f3e82169740 1 -- 172.21.15.83:0/511825515 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55afdc82e190 con 0x55afdc941a30 2024-04-24T09:00:12.261 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.259+0000 7f3e77400640 1 --2- 172.21.15.83:0/511825515 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55afdc941a30 0x55afdc943e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:12.261 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.260+0000 7f3e77400640 1 --2- 172.21.15.83:0/511825515 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55afdc941a30 0x55afdc943e20 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d95f57bc9f9d708 server_cookie=e657c77680750a2a in_seq=0 out_seq=0 2024-04-24T09:00:12.262 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.260+0000 7f3e66000640 10 client.18744 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:00:12.276 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T09:00:12.276 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T09:00:12.276 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T09:00:12.312 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.312+0000 7f3e66000640 1 -- 172.21.15.83:0/511825515 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55afdc82e190 con 0x55afdc941a30 2024-04-24T09:00:12.312 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.312+0000 7f3e66000640 10 client.18744 handle_command_reply: tid=0 2024-04-24T09:00:12.312 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.312+0000 7f3e82169740 1 client.18744 shutdown 2024-04-24T09:00:12.312 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.313+0000 7f3e82169740 1 -- 172.21.15.83:0/511825515 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55afdc8c01b0 msgr2=0x55afdc8d2fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:12.312 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.313+0000 7f3e82169740 1 --2- 172.21.15.83:0/511825515 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55afdc8c01b0 0x55afdc8d2fc0 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f3e6802ade0 tx=0x7f3e6803e040 comp rx=0 tx=0).stop 2024-04-24T09:00:12.312 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.313+0000 7f3e82169740 1 -- 172.21.15.83:0/511825515 shutdown_connections 2024-04-24T09:00:12.312 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.313+0000 7f3e82169740 1 --2- 172.21.15.83:0/511825515 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55afdc941a30 0x55afdc943e20 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:12.312 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.313+0000 7f3e82169740 1 --2- 172.21.15.83:0/511825515 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55afdc8cb390 0x55afdc8c4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:12.312 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.313+0000 7f3e82169740 1 --2- 172.21.15.83:0/511825515 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55afdc8c0ad0 0x55afdc8c4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:12.312 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.313+0000 7f3e82169740 1 --2- 172.21.15.83:0/511825515 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55afdc8c01b0 0x55afdc8d2fc0 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:12.312 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.313+0000 7f3e66000640 0 client.18744 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:00:12.312 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.313+0000 7f3e82169740 1 -- 172.21.15.83:0/511825515 >> 172.21.15.83:0/511825515 conn(0x55afdc8bc6c0 msgr2=0x55afdc8c81a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:12.313 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.313+0000 7f3e82169740 1 -- 172.21.15.83:0/511825515 shutdown_connections 2024-04-24T09:00:12.313 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.313+0000 7f3e82169740 1 -- 172.21.15.83:0/511825515 wait complete. 2024-04-24T09:00:12.313 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.313+0000 7f3e82169740 20 client.18744 trim_cache size 0 max 16384 2024-04-24T09:00:12.314 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T09:00:12.314 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T09:00:12.314 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T09:00:12.314 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T09:00:12.314 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.314+0000 7f3e7f200640 1 -- 172.21.15.83:0/3256499343 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f3e5c0a3dc0 msgr2=0x7f3e5c0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:12.315 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.314+0000 7f3e7f200640 1 --2- 172.21.15.83:0/3256499343 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f3e5c0a3dc0 0x7f3e5c0a6280 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3e70005fd0 tx=0x7f3e7000b340 comp rx=0 tx=0).stop 2024-04-24T09:00:12.315 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.314+0000 7f3e7f200640 1 -- 172.21.15.83:0/3256499343 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e78100e10 msgr2=0x7f3e781a0730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:12.315 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.314+0000 7f3e7f200640 1 --2- 172.21.15.83:0/3256499343 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e78100e10 0x7f3e781a0730 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f3e68006010 tx=0x7f3e68003ba0 comp rx=0 tx=0).stop 2024-04-24T09:00:12.315 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.314+0000 7f3e7f200640 1 -- 172.21.15.83:0/3256499343 shutdown_connections 2024-04-24T09:00:12.315 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.314+0000 7f3e7f200640 1 --2- 172.21.15.83:0/3256499343 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f3e5c0a3dc0 0x7f3e5c0a6280 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:12.315 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.314+0000 7f3e7f200640 1 --2- 172.21.15.83:0/3256499343 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3e7810e9c0 0x7f3e7819a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:12.315 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.314+0000 7f3e7f200640 1 --2- 172.21.15.83:0/3256499343 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3e78101730 0x7f3e781a0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:12.315 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.314+0000 7f3e7f200640 1 --2- 172.21.15.83:0/3256499343 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e78100e10 0x7f3e781a0730 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:12.315 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.314+0000 7f3e7f200640 1 -- 172.21.15.83:0/3256499343 >> 172.21.15.83:0/3256499343 conn(0x7f3e78078080 msgr2=0x7f3e7810ff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:12.315 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.315+0000 7f3e7f200640 1 -- 172.21.15.83:0/3256499343 shutdown_connections 2024-04-24T09:00:12.315 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:12.315+0000 7f3e7f200640 1 -- 172.21.15.83:0/3256499343 wait complete. 2024-04-24T09:00:12.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.480+0000 7fd558a00640 1 -- 172.21.15.83:0/2538116298 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd55410c490 msgr2=0x7fd55410e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:12.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.480+0000 7fd558a00640 1 --2- 172.21.15.83:0/2538116298 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd55410c490 0x7fd55410e910 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7fd54c009ca0 tx=0x7fd54c02a950 comp rx=0 tx=0).stop 2024-04-24T09:00:12.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.480+0000 7fd558a00640 1 -- 172.21.15.83:0/2538116298 shutdown_connections 2024-04-24T09:00:12.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.480+0000 7fd558a00640 1 --2- 172.21.15.83:0/2538116298 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd55410c490 0x7fd55410e910 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:12.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.480+0000 7fd558a00640 1 --2- 172.21.15.83:0/2538116298 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd5541039e0 0x7fd55410bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:12.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.480+0000 7fd558a00640 1 --2- 172.21.15.83:0/2538116298 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd5541030c0 0x7fd5541034a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:12.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.480+0000 7fd558a00640 1 -- 172.21.15.83:0/2538116298 >> 172.21.15.83:0/2538116298 conn(0x7fd5540fc880 msgr2=0x7fd5540feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:12.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.480+0000 7fd558a00640 1 -- 172.21.15.83:0/2538116298 shutdown_connections 2024-04-24T09:00:12.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.480+0000 7fd558a00640 1 -- 172.21.15.83:0/2538116298 wait complete. 2024-04-24T09:00:12.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.481+0000 7fd558a00640 1 Processor -- start 2024-04-24T09:00:12.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.481+0000 7fd558a00640 1 -- start start 2024-04-24T09:00:12.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.481+0000 7fd558a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd5541030c0 0x7fd554198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:12.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.481+0000 7fd558a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd5541039e0 0x7fd554198ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:12.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.482+0000 7fd558a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd55410c490 0x7fd55419cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:12.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.482+0000 7fd558a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd5541120e0 con 0x7fd5541030c0 2024-04-24T09:00:12.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.482+0000 7fd558a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd554111f60 con 0x7fd5541039e0 2024-04-24T09:00:12.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.482+0000 7fd558a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd554112260 con 0x7fd55410c490 2024-04-24T09:00:12.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.482+0000 7fd552800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd5541030c0 0x7fd554198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:12.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.482+0000 7fd552800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd5541030c0 0x7fd554198660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48924/0 (socket says 172.21.15.83:48924) 2024-04-24T09:00:12.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.482+0000 7fd552800640 1 -- 172.21.15.83:0/3095539819 learned_addr learned my addr 172.21.15.83:0/3095539819 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:00:12.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.482+0000 7fd551e00640 1 --2- 172.21.15.83:0/3095539819 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd5541039e0 0x7fd554198ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:12.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.482+0000 7fd553200640 1 --2- 172.21.15.83:0/3095539819 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd55410c490 0x7fd55419cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:12.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.482+0000 7fd552800640 1 -- 172.21.15.83:0/3095539819 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd55410c490 msgr2=0x7fd55419cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:12.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.482+0000 7fd552800640 1 --2- 172.21.15.83:0/3095539819 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd55410c490 0x7fd55419cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:12.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.482+0000 7fd552800640 1 -- 172.21.15.83:0/3095539819 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd5541039e0 msgr2=0x7fd554198ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:12.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.482+0000 7fd552800640 1 --2- 172.21.15.83:0/3095539819 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd5541039e0 0x7fd554198ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:12.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.483+0000 7fd552800640 1 -- 172.21.15.83:0/3095539819 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd55419d650 con 0x7fd5541030c0 2024-04-24T09:00:12.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.483+0000 7fd552800640 1 --2- 172.21.15.83:0/3095539819 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd5541030c0 0x7fd554198660 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7fd54400d950 tx=0x7fd54400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:12.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.483+0000 7fd54b400640 1 -- 172.21.15.83:0/3095539819 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd544017030 con 0x7fd5541030c0 2024-04-24T09:00:12.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.483+0000 7fd54b400640 1 -- 172.21.15.83:0/3095539819 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd544004020 con 0x7fd5541030c0 2024-04-24T09:00:12.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.483+0000 7fd558a00640 1 -- 172.21.15.83:0/3095539819 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd55419d940 con 0x7fd5541030c0 2024-04-24T09:00:12.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.483+0000 7fd54b400640 1 -- 172.21.15.83:0/3095539819 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5440047a0 con 0x7fd5541030c0 2024-04-24T09:00:12.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.483+0000 7fd558a00640 1 -- 172.21.15.83:0/3095539819 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5541a51d0 con 0x7fd5541030c0 2024-04-24T09:00:12.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.487+0000 7fd55c1cf740 1 Processor -- start 2024-04-24T09:00:12.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.487+0000 7fd54b400640 1 -- 172.21.15.83:0/3095539819 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd5440041d0 con 0x7fd5541030c0 2024-04-24T09:00:12.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.487+0000 7fd55c1cf740 1 -- start start 2024-04-24T09:00:12.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.487+0000 7fd55c1cf740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557fb3e72fa0 0x557fb3e73380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:12.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.487+0000 7fd55c1cf740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557fb3e73950 0x557fb3e82130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:12.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.488+0000 7fd55c1cf740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557fb3e82670 0x557fb3e84a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:12.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.488+0000 7fd55c1cf740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x557fb3dd8e70 con 0x557fb3e72fa0 2024-04-24T09:00:12.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.488+0000 7fd55c1cf740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x557fb3d7f380 con 0x557fb3e73950 2024-04-24T09:00:12.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.488+0000 7fd55c1cf740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x557fb3dd4e30 con 0x557fb3e82670 2024-04-24T09:00:12.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.488+0000 7fd553200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557fb3e72fa0 0x557fb3e73380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:12.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.488+0000 7fd553200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557fb3e72fa0 0x557fb3e73380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48938/0 (socket says 172.21.15.83:48938) 2024-04-24T09:00:12.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.488+0000 7fd553200640 1 -- 172.21.15.83:0/2883065393 learned_addr learned my addr 172.21.15.83:0/2883065393 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:00:12.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.488+0000 7fd552800640 1 --2- 172.21.15.83:0/2883065393 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557fb3e73950 0x557fb3e82130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:12.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.488+0000 7fd551e00640 1 --2- 172.21.15.83:0/2883065393 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557fb3e82670 0x557fb3e84a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:12.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.489+0000 7fd553200640 1 -- 172.21.15.83:0/2883065393 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557fb3e82670 msgr2=0x557fb3e84a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:12.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.489+0000 7fd553200640 1 --2- 172.21.15.83:0/2883065393 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557fb3e82670 0x557fb3e84a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:12.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.489+0000 7fd553200640 1 -- 172.21.15.83:0/2883065393 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557fb3e73950 msgr2=0x557fb3e82130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:12.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.489+0000 7fd553200640 1 --2- 172.21.15.83:0/2883065393 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557fb3e73950 0x557fb3e82130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:12.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.489+0000 7fd553200640 1 -- 172.21.15.83:0/2883065393 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557fb3bb37e0 con 0x557fb3e72fa0 2024-04-24T09:00:12.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.489+0000 7fd54b400640 1 --2- 172.21.15.83:0/3095539819 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fd5380a3d20 0x7fd5380a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:12.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.489+0000 7fd54b400640 1 -- 172.21.15.83:0/3095539819 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7fd5440d06a0 con 0x7fd5541030c0 2024-04-24T09:00:12.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.489+0000 7fd552800640 1 --2- 172.21.15.83:0/3095539819 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fd5380a3d20 0x7fd5380a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:12.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.489+0000 7fd553200640 1 --2- 172.21.15.83:0/2883065393 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557fb3e72fa0 0x557fb3e73380 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7fd54c009a20 tx=0x7fd54c009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab304e38340000f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:12.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.490+0000 7fd53f400640 1 -- 172.21.15.83:0/2883065393 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd54c034040 con 0x557fb3e72fa0 2024-04-24T09:00:12.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.490+0000 7fd53f400640 1 -- 172.21.15.83:0/2883065393 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd54c02fec0 con 0x557fb3e72fa0 2024-04-24T09:00:12.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.490+0000 7fd53f400640 1 -- 172.21.15.83:0/2883065393 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd54c0043c0 con 0x557fb3e72fa0 2024-04-24T09:00:12.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.491+0000 7fd55c1cf740 1 -- 172.21.15.83:0/2883065393 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557fb3e72fa0 msgr2=0x557fb3e73380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:12.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.491+0000 7fd552800640 1 --2- 172.21.15.83:0/3095539819 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fd5380a3d20 0x7fd5380a61e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd5440cda80 tx=0x7fd5440d6550 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:12.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.491+0000 7fd55c1cf740 1 --2- 172.21.15.83:0/2883065393 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557fb3e72fa0 0x557fb3e73380 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7fd54c009a20 tx=0x7fd54c009360 comp rx=0 tx=0).stop 2024-04-24T09:00:12.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.491+0000 7fd55c1cf740 1 -- 172.21.15.83:0/2883065393 shutdown_connections 2024-04-24T09:00:12.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.491+0000 7fd55c1cf740 1 --2- 172.21.15.83:0/2883065393 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557fb3e82670 0x557fb3e84a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:12.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.491+0000 7fd55c1cf740 1 --2- 172.21.15.83:0/2883065393 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557fb3e73950 0x557fb3e82130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:12.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.491+0000 7fd55c1cf740 1 --2- 172.21.15.83:0/2883065393 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557fb3e72fa0 0x557fb3e73380 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:12.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.491+0000 7fd55c1cf740 1 -- 172.21.15.83:0/2883065393 >> 172.21.15.83:0/2883065393 conn(0x557fb3e6f590 msgr2=0x557fb3e6f970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:12.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.491+0000 7fd55c1cf740 1 -- 172.21.15.83:0/2883065393 shutdown_connections 2024-04-24T09:00:12.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.491+0000 7fd55c1cf740 1 -- 172.21.15.83:0/2883065393 wait complete. 2024-04-24T09:00:12.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.491+0000 7fd55c1cf740 1 -- start start 2024-04-24T09:00:12.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.492+0000 7fd55c1cf740 1 Processor -- start 2024-04-24T09:00:12.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.492+0000 7fd55c1cf740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557fb3e72fa0 0x557fb3e7f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:12.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.492+0000 7fd55c1cf740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557fb3e73950 0x557fb3e7fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:12.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.492+0000 7fd55c1cf740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557fb3e82670 0x557fb3e800d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:12.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.492+0000 7fd55c1cf740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x557fb3e853d0 con 0x557fb3e73950 2024-04-24T09:00:12.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.492+0000 7fd55c1cf740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x557fb3d92660 con 0x557fb3e82670 2024-04-24T09:00:12.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.492+0000 7fd55c1cf740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x557fb3e85550 con 0x557fb3e72fa0 2024-04-24T09:00:12.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.492+0000 7fd55c1cf740 10 client.0 _ll_register_callbacks cb 0x557fb3d80aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:00:12.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.492+0000 7fd551e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557fb3e73950 0x557fb3e7fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:12.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.493+0000 7fd551e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557fb3e73950 0x557fb3e7fb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48942/0 (socket says 172.21.15.83:48942) 2024-04-24T09:00:12.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.493+0000 7fd551e00640 1 -- 172.21.15.83:0/4027512287 learned_addr learned my addr 172.21.15.83:0/4027512287 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:00:12.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.493+0000 7fd553200640 1 --2- 172.21.15.83:0/4027512287 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557fb3e72fa0 0x557fb3e7f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:12.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.493+0000 7fd553200640 1 --2- 172.21.15.83:0/4027512287 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557fb3e82670 0x557fb3e800d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:12.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.493+0000 7fd551e00640 1 -- 172.21.15.83:0/4027512287 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557fb3e72fa0 msgr2=0x557fb3e7f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:12.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.493+0000 7fd551e00640 1 --2- 172.21.15.83:0/4027512287 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557fb3e72fa0 0x557fb3e7f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:12.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.493+0000 7fd551e00640 1 -- 172.21.15.83:0/4027512287 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557fb3e82670 msgr2=0x557fb3e800d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:12.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.493+0000 7fd551e00640 1 --2- 172.21.15.83:0/4027512287 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557fb3e82670 0x557fb3e800d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:12.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.493+0000 7fd551e00640 1 -- 172.21.15.83:0/4027512287 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557fb3c13f70 con 0x557fb3e73950 2024-04-24T09:00:12.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.493+0000 7fd553200640 1 --2- 172.21.15.83:0/4027512287 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557fb3e82670 0x557fb3e800d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:00:12.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.493+0000 7fd551e00640 1 --2- 172.21.15.83:0/4027512287 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557fb3e73950 0x557fb3e7fb90 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7fd540012700 tx=0x7fd540012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db6f483fe0271413 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:12.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.493+0000 7fd533e00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:00:12.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.494+0000 7fd533e00640 1 -- 172.21.15.83:0/4027512287 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd540004050 con 0x557fb3e73950 2024-04-24T09:00:12.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.494+0000 7fd533e00640 1 -- 172.21.15.83:0/4027512287 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7fd5400041f0 con 0x557fb3e73950 2024-04-24T09:00:12.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.494+0000 7fd55c1cf740 1 -- 172.21.15.83:0/4027512287 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd54c02fec0 con 0x557fb3e73950 2024-04-24T09:00:12.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.494+0000 7fd533e00640 1 -- 172.21.15.83:0/4027512287 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fd5400043f0 con 0x557fb3e73950 2024-04-24T09:00:12.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.494+0000 7fd55c1cf740 10 client.18756 fetch_fsmap learned FSMap version 35 2024-04-24T09:00:12.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.494+0000 7fd55c1cf740 1 -- 172.21.15.83:0/4027512287 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x557fb3aab170 con 0x557fb3e73950 2024-04-24T09:00:12.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.494+0000 7fd533e00640 1 -- 172.21.15.83:0/4027512287 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd54c02fec0 con 0x557fb3e73950 2024-04-24T09:00:12.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.495+0000 7fd533e00640 1 -- 172.21.15.83:0/4027512287 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7fd540003ae0 con 0x557fb3e73950 2024-04-24T09:00:12.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.495+0000 7fd55c1cf740 10 client.18756 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T09:00:12.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.495+0000 7fd55c1cf740 10 client.18756 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:00:12.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.495+0000 7fd55c1cf740 20 client.18756 populate_metadata read hostname 'smithi083' 2024-04-24T09:00:12.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.495+0000 7fd55c1cf740 1 --2- 172.21.15.83:0/4027512287 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x557fb3e92aa0 0x557fb3e94f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:12.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.495+0000 7fd55c1cf740 4 client.18756 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:00:12.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.495+0000 7fd55c1cf740 1 -- 172.21.15.83:0/4027512287 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557fb3de6890 con 0x557fb3e92aa0 2024-04-24T09:00:12.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.495+0000 7fd553200640 1 --2- 172.21.15.83:0/4027512287 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x557fb3e92aa0 0x557fb3e94f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:12.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.496+0000 7fd553200640 1 --2- 172.21.15.83:0/4027512287 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x557fb3e92aa0 0x557fb3e94f20 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ec081c1260d50b2 server_cookie=821fc641dfa52473 in_seq=0 out_seq=0 2024-04-24T09:00:12.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.496+0000 7fd533e00640 10 client.18756 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:00:12.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.499+0000 7fd533e00640 1 -- 172.21.15.83:0/4027512287 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557fb3de6890 con 0x557fb3e92aa0 2024-04-24T09:00:12.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.499+0000 7fd533e00640 10 client.18756 handle_command_reply: tid=0 2024-04-24T09:00:12.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.499+0000 7fd55c1cf740 1 client.18756 shutdown 2024-04-24T09:00:12.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.499+0000 7fd55c1cf740 1 -- 172.21.15.83:0/4027512287 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557fb3e73950 msgr2=0x557fb3e7fb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:12.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.499+0000 7fd55c1cf740 1 --2- 172.21.15.83:0/4027512287 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557fb3e73950 0x557fb3e7fb90 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7fd540012700 tx=0x7fd540012bc0 comp rx=0 tx=0).stop 2024-04-24T09:00:12.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.499+0000 7fd55c1cf740 1 -- 172.21.15.83:0/4027512287 shutdown_connections 2024-04-24T09:00:12.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.499+0000 7fd55c1cf740 1 --2- 172.21.15.83:0/4027512287 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x557fb3e92aa0 0x557fb3e94f20 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:12.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.499+0000 7fd55c1cf740 1 --2- 172.21.15.83:0/4027512287 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557fb3e82670 0x557fb3e800d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:12.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.499+0000 7fd55c1cf740 1 --2- 172.21.15.83:0/4027512287 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557fb3e73950 0x557fb3e7fb90 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:12.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.499+0000 7fd55c1cf740 1 --2- 172.21.15.83:0/4027512287 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557fb3e72fa0 0x557fb3e7f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:12.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.499+0000 7fd55c1cf740 1 -- 172.21.15.83:0/4027512287 >> 172.21.15.83:0/4027512287 conn(0x557fb3e6f590 msgr2=0x557fb3e75400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:12.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.499+0000 7fd533e00640 0 client.18756 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:00:12.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.499+0000 7fd55c1cf740 1 -- 172.21.15.83:0/4027512287 shutdown_connections 2024-04-24T09:00:12.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.499+0000 7fd55c1cf740 1 -- 172.21.15.83:0/4027512287 wait complete. 2024-04-24T09:00:12.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.499+0000 7fd55c1cf740 20 client.18756 trim_cache size 0 max 16384 2024-04-24T09:00:12.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.522+0000 7fd55c1cf740 1 Processor -- start 2024-04-24T09:00:12.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.522+0000 7fd55c1cf740 1 -- start start 2024-04-24T09:00:12.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.522+0000 7fd55c1cf740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557fb3e72fa0 0x557fb3e8e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:12.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.522+0000 7fd55c1cf740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557fb3e82670 0x557fb3e6e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:12.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.522+0000 7fd55c1cf740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557fb3e81e90 0x557fb3e9a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:12.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.522+0000 7fd55c1cf740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd5541ad300 con 0x557fb3e81e90 2024-04-24T09:00:12.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.522+0000 7fd55c1cf740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd5541aee40 con 0x557fb3e72fa0 2024-04-24T09:00:12.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.522+0000 7fd55c1cf740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd5541ad9d0 con 0x557fb3e82670 2024-04-24T09:00:12.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.522+0000 7fd553200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557fb3e81e90 0x557fb3e9a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:12.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.523+0000 7fd553200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557fb3e81e90 0x557fb3e9a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48948/0 (socket says 172.21.15.83:48948) 2024-04-24T09:00:12.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.523+0000 7fd553200640 1 -- 172.21.15.83:0/2519868015 learned_addr learned my addr 172.21.15.83:0/2519868015 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:00:12.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.523+0000 7fd553200640 1 --2- 172.21.15.83:0/2519868015 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557fb3e72fa0 0x557fb3e8e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:12.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.523+0000 7fd551e00640 1 --2- 172.21.15.83:0/2519868015 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557fb3e82670 0x557fb3e6e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:12.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.523+0000 7fd553200640 1 -- 172.21.15.83:0/2519868015 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557fb3e82670 msgr2=0x557fb3e6e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:12.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.523+0000 7fd553200640 1 --2- 172.21.15.83:0/2519868015 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557fb3e82670 0x557fb3e6e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:12.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.523+0000 7fd553200640 1 -- 172.21.15.83:0/2519868015 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557fb3e72fa0 msgr2=0x557fb3e8e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:12.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.523+0000 7fd553200640 1 --2- 172.21.15.83:0/2519868015 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557fb3e72fa0 0x557fb3e8e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:12.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.523+0000 7fd553200640 1 -- 172.21.15.83:0/2519868015 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557fb3e84a60 con 0x557fb3e81e90 2024-04-24T09:00:12.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.523+0000 7fd553200640 1 --2- 172.21.15.83:0/2519868015 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557fb3e81e90 0x557fb3e9a830 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7fd54c009a20 tx=0x7fd54c046040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95f77b7cfdbd0e81 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:12.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.523+0000 7fd53ea00640 1 -- 172.21.15.83:0/2519868015 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd54c02fec0 con 0x557fb3e81e90 2024-04-24T09:00:12.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.523+0000 7fd53ea00640 1 -- 172.21.15.83:0/2519868015 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd54c056070 con 0x557fb3e81e90 2024-04-24T09:00:12.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.524+0000 7fd53ea00640 1 -- 172.21.15.83:0/2519868015 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd54c050650 con 0x557fb3e81e90 2024-04-24T09:00:12.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.524+0000 7fd55c1cf740 1 -- 172.21.15.83:0/2519868015 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557fb3e81e90 msgr2=0x557fb3e9a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:12.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.524+0000 7fd55c1cf740 1 --2- 172.21.15.83:0/2519868015 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557fb3e81e90 0x557fb3e9a830 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7fd54c009a20 tx=0x7fd54c046040 comp rx=0 tx=0).stop 2024-04-24T09:00:12.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.524+0000 7fd55c1cf740 1 -- 172.21.15.83:0/2519868015 shutdown_connections 2024-04-24T09:00:12.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.524+0000 7fd55c1cf740 1 --2- 172.21.15.83:0/2519868015 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557fb3e81e90 0x557fb3e9a830 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:12.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.524+0000 7fd55c1cf740 1 --2- 172.21.15.83:0/2519868015 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557fb3e82670 0x557fb3e6e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:12.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.524+0000 7fd55c1cf740 1 --2- 172.21.15.83:0/2519868015 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557fb3e72fa0 0x557fb3e8e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:12.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.524+0000 7fd55c1cf740 1 -- 172.21.15.83:0/2519868015 >> 172.21.15.83:0/2519868015 conn(0x557fb3e6f590 msgr2=0x557fb3e6de60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:12.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.524+0000 7fd55c1cf740 1 -- 172.21.15.83:0/2519868015 shutdown_connections 2024-04-24T09:00:12.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.525+0000 7fd55c1cf740 1 -- 172.21.15.83:0/2519868015 wait complete. 2024-04-24T09:00:12.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.525+0000 7fd55c1cf740 1 -- start start 2024-04-24T09:00:12.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.525+0000 7fd55c1cf740 1 Processor -- start 2024-04-24T09:00:12.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.525+0000 7fd55c1cf740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557fb3e72fa0 0x557fb3e8a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:12.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.525+0000 7fd55c1cf740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557fb3e81e90 0x557fb3e8ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:12.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.525+0000 7fd55c1cf740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557fb3e82670 0x557fb3e877b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:12.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.526+0000 7fd55c1cf740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd5541ae770 con 0x557fb3e81e90 2024-04-24T09:00:12.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.526+0000 7fd55c1cf740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd5541ae0a0 con 0x557fb3e72fa0 2024-04-24T09:00:12.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.526+0000 7fd55c1cf740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd5541ab300 con 0x557fb3e82670 2024-04-24T09:00:12.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.526+0000 7fd55c1cf740 10 client.0 _ll_register_callbacks cb 0x7fd5400097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:00:12.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.526+0000 7fd551e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557fb3e81e90 0x557fb3e8ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:12.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.526+0000 7fd553200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557fb3e72fa0 0x557fb3e8a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:12.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.526+0000 7fd551e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557fb3e81e90 0x557fb3e8ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48960/0 (socket says 172.21.15.83:48960) 2024-04-24T09:00:12.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.526+0000 7fd551e00640 1 -- 172.21.15.83:0/1684828291 learned_addr learned my addr 172.21.15.83:0/1684828291 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:00:12.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.526+0000 7fd553200640 1 --2- 172.21.15.83:0/1684828291 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557fb3e82670 0x557fb3e877b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:12.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.526+0000 7fd551e00640 1 -- 172.21.15.83:0/1684828291 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557fb3e82670 msgr2=0x557fb3e877b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:12.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.526+0000 7fd551e00640 1 --2- 172.21.15.83:0/1684828291 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557fb3e82670 0x557fb3e877b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:12.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.526+0000 7fd551e00640 1 -- 172.21.15.83:0/1684828291 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557fb3e72fa0 msgr2=0x557fb3e8a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:12.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.526+0000 7fd551e00640 1 --2- 172.21.15.83:0/1684828291 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557fb3e72fa0 0x557fb3e8a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:12.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.526+0000 7fd551e00640 1 -- 172.21.15.83:0/1684828291 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557fb3e87e40 con 0x557fb3e81e90 2024-04-24T09:00:12.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.526+0000 7fd553200640 1 --2- 172.21.15.83:0/1684828291 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557fb3e82670 0x557fb3e877b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:00:12.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.527+0000 7fd551e00640 1 --2- 172.21.15.83:0/1684828291 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557fb3e81e90 0x557fb3e8ad20 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x557fb3e80c20 tx=0x7fd540005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60254ffc62b8bdb5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:12.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.527+0000 7fd533e00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:00:12.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.527+0000 7fd533e00640 1 -- 172.21.15.83:0/1684828291 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd54002d020 con 0x557fb3e81e90 2024-04-24T09:00:12.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.527+0000 7fd533e00640 1 -- 172.21.15.83:0/1684828291 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7fd5400041e0 con 0x557fb3e81e90 2024-04-24T09:00:12.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.527+0000 7fd55c1cf740 1 -- 172.21.15.83:0/1684828291 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd54c056070 con 0x557fb3e81e90 2024-04-24T09:00:12.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.527+0000 7fd533e00640 1 -- 172.21.15.83:0/1684828291 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fd54000f920 con 0x557fb3e81e90 2024-04-24T09:00:12.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.527+0000 7fd55c1cf740 10 client.18762 fetch_fsmap learned FSMap version 35 2024-04-24T09:00:12.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.527+0000 7fd55c1cf740 1 -- 172.21.15.83:0/1684828291 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x557fb3ef9aa0 con 0x557fb3e81e90 2024-04-24T09:00:12.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.528+0000 7fd533e00640 1 -- 172.21.15.83:0/1684828291 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd54c056070 con 0x557fb3e81e90 2024-04-24T09:00:12.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.528+0000 7fd533e00640 1 -- 172.21.15.83:0/1684828291 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7fd5400035f0 con 0x557fb3e81e90 2024-04-24T09:00:12.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.528+0000 7fd55c1cf740 10 client.18762 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T09:00:12.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.528+0000 7fd55c1cf740 10 client.18762 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:00:12.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.528+0000 7fd55c1cf740 20 client.18762 populate_metadata read hostname 'smithi083' 2024-04-24T09:00:12.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.528+0000 7fd55c1cf740 1 --2- 172.21.15.83:0/1684828291 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x557fb3efa270 0x557fb3efc690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:12.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.528+0000 7fd55c1cf740 4 client.18762 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T09:00:12.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.528+0000 7fd55c1cf740 1 -- 172.21.15.83:0/1684828291 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x557fb3de0700 con 0x557fb3efa270 2024-04-24T09:00:12.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.529+0000 7fd553200640 1 --2- 172.21.15.83:0/1684828291 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x557fb3efa270 0x557fb3efc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:12.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.529+0000 7fd553200640 1 --2- 172.21.15.83:0/1684828291 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x557fb3efa270 0x557fb3efc690 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc5d144e144fdf31 server_cookie=63cafa5336c68399 in_seq=0 out_seq=0 2024-04-24T09:00:12.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.529+0000 7fd533e00640 10 client.18762 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:00:12.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.532+0000 7fd533e00640 1 -- 172.21.15.83:0/1684828291 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x557fb3de0700 con 0x557fb3efa270 2024-04-24T09:00:12.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.532+0000 7fd533e00640 10 client.18762 handle_command_reply: tid=0 2024-04-24T09:00:12.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.532+0000 7fd55c1cf740 1 client.18762 shutdown 2024-04-24T09:00:12.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.533+0000 7fd55c1cf740 1 -- 172.21.15.83:0/1684828291 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557fb3e81e90 msgr2=0x557fb3e8ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:12.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.533+0000 7fd55c1cf740 1 --2- 172.21.15.83:0/1684828291 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557fb3e81e90 0x557fb3e8ad20 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x557fb3e80c20 tx=0x7fd540005710 comp rx=0 tx=0).stop 2024-04-24T09:00:12.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.533+0000 7fd55c1cf740 1 -- 172.21.15.83:0/1684828291 shutdown_connections 2024-04-24T09:00:12.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.533+0000 7fd55c1cf740 1 --2- 172.21.15.83:0/1684828291 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x557fb3efa270 0x557fb3efc690 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:12.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.533+0000 7fd55c1cf740 1 --2- 172.21.15.83:0/1684828291 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557fb3e82670 0x557fb3e877b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:12.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.533+0000 7fd55c1cf740 1 --2- 172.21.15.83:0/1684828291 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557fb3e81e90 0x557fb3e8ad20 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:12.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.533+0000 7fd55c1cf740 1 --2- 172.21.15.83:0/1684828291 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557fb3e72fa0 0x557fb3e8a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:12.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.533+0000 7fd533e00640 0 client.18762 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:00:12.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.533+0000 7fd55c1cf740 1 -- 172.21.15.83:0/1684828291 >> 172.21.15.83:0/1684828291 conn(0x557fb3e6f590 msgr2=0x557fb3e6e6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:12.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.533+0000 7fd55c1cf740 1 -- 172.21.15.83:0/1684828291 shutdown_connections 2024-04-24T09:00:12.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.533+0000 7fd55c1cf740 1 -- 172.21.15.83:0/1684828291 wait complete. 2024-04-24T09:00:12.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.533+0000 7fd55c1cf740 20 client.18762 trim_cache size 0 max 16384 2024-04-24T09:00:12.533 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T09:00:12.533 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-04-24T09:00:12.533 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "85540fc2-14ea-4744-bd0a-88a98038b8cf", 2024-04-24T09:00:12.534 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-04-24T09:00:12.534 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T09:00:12.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.534+0000 7fd558a00640 1 -- 172.21.15.83:0/3095539819 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fd5380a3d20 msgr2=0x7fd5380a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:12.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.534+0000 7fd558a00640 1 --2- 172.21.15.83:0/3095539819 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fd5380a3d20 0x7fd5380a61e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd5440cda80 tx=0x7fd5440d6550 comp rx=0 tx=0).stop 2024-04-24T09:00:12.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.534+0000 7fd558a00640 1 -- 172.21.15.83:0/3095539819 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd5541030c0 msgr2=0x7fd554198660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:12.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.534+0000 7fd558a00640 1 --2- 172.21.15.83:0/3095539819 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd5541030c0 0x7fd554198660 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7fd54400d950 tx=0x7fd54400de10 comp rx=0 tx=0).stop 2024-04-24T09:00:12.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.534+0000 7fd558a00640 1 -- 172.21.15.83:0/3095539819 shutdown_connections 2024-04-24T09:00:12.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.534+0000 7fd558a00640 1 --2- 172.21.15.83:0/3095539819 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fd5380a3d20 0x7fd5380a61e0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:12.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.534+0000 7fd558a00640 1 --2- 172.21.15.83:0/3095539819 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd55410c490 0x7fd55419cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:12.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.534+0000 7fd558a00640 1 --2- 172.21.15.83:0/3095539819 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd5541039e0 0x7fd554198ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:12.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.534+0000 7fd558a00640 1 --2- 172.21.15.83:0/3095539819 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd5541030c0 0x7fd554198660 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:12.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.534+0000 7fd558a00640 1 -- 172.21.15.83:0/3095539819 >> 172.21.15.83:0/3095539819 conn(0x7fd5540fc880 msgr2=0x7fd5540fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:12.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.534+0000 7fd558a00640 1 -- 172.21.15.83:0/3095539819 shutdown_connections 2024-04-24T09:00:12.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.534+0000 7fd558a00640 1 -- 172.21.15.83:0/3095539819 wait complete. 2024-04-24T09:00:12.551 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T09:00:12.551 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T09:00:12.600 INFO:tasks.workunit.client.0.smithi083.stdout: 1 48407 0.62 MB/sec execute 286 sec latency 64.005 ms 2024-04-24T09:00:12.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.810+0000 7fd3d5800640 1 -- 172.21.15.83:0/761132207 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3d0102110 msgr2=0x7fd3d01024f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:12.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.810+0000 7fd3d5800640 1 --2- 172.21.15.83:0/761132207 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3d0102110 0x7fd3d01024f0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7fd3c8009ca0 tx=0x7fd3c802a8c0 comp rx=0 tx=0).stop 2024-04-24T09:00:12.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.810+0000 7fd3d5800640 1 -- 172.21.15.83:0/761132207 shutdown_connections 2024-04-24T09:00:12.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.810+0000 7fd3d5800640 1 --2- 172.21.15.83:0/761132207 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd3d01071e0 0x7fd3d010edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:12.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.810+0000 7fd3d5800640 1 --2- 172.21.15.83:0/761132207 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd3d0102ac0 0x7fd3d0106b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:12.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.810+0000 7fd3d5800640 1 --2- 172.21.15.83:0/761132207 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3d0102110 0x7fd3d01024f0 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:12.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.810+0000 7fd3d5800640 1 -- 172.21.15.83:0/761132207 >> 172.21.15.83:0/761132207 conn(0x7fd3d00fd9d0 msgr2=0x7fd3d00ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:12.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.810+0000 7fd3d5800640 1 -- 172.21.15.83:0/761132207 shutdown_connections 2024-04-24T09:00:12.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.811+0000 7fd3d5800640 1 -- 172.21.15.83:0/761132207 wait complete. 2024-04-24T09:00:12.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.812+0000 7fd3d5800640 1 Processor -- start 2024-04-24T09:00:12.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.812+0000 7fd3d5800640 1 -- start start 2024-04-24T09:00:12.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.812+0000 7fd3d5800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd3d0102110 0x7fd3d0074300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:12.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.812+0000 7fd3d5800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd3d01071e0 0x7fd3d0074840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:12.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.812+0000 7fd3d5800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3d0074d80 0x7fd3d01a6860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:12.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.812+0000 7fd3d5800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd3d0112150 con 0x7fd3d0074d80 2024-04-24T09:00:12.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.812+0000 7fd3d5800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd3d0111fd0 con 0x7fd3d0102110 2024-04-24T09:00:12.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.812+0000 7fd3d5800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd3d01122d0 con 0x7fd3d01071e0 2024-04-24T09:00:12.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.813+0000 7fd3d4e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3d0074d80 0x7fd3d01a6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:12.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.813+0000 7fd3d4e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3d0074d80 0x7fd3d01a6860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48984/0 (socket says 172.21.15.83:48984) 2024-04-24T09:00:12.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.813+0000 7fd3d4e00640 1 -- 172.21.15.83:0/1461110377 learned_addr learned my addr 172.21.15.83:0/1461110377 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:00:12.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.813+0000 7fd3cfe00640 1 --2- 172.21.15.83:0/1461110377 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd3d0102110 0x7fd3d0074300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:12.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.813+0000 7fd3cf400640 1 --2- 172.21.15.83:0/1461110377 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd3d01071e0 0x7fd3d0074840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:12.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.813+0000 7fd3d4e00640 1 -- 172.21.15.83:0/1461110377 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd3d01071e0 msgr2=0x7fd3d0074840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:12.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.813+0000 7fd3d4e00640 1 --2- 172.21.15.83:0/1461110377 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd3d01071e0 0x7fd3d0074840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:12.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.813+0000 7fd3d4e00640 1 -- 172.21.15.83:0/1461110377 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd3d0102110 msgr2=0x7fd3d0074300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:12.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.813+0000 7fd3d4e00640 1 --2- 172.21.15.83:0/1461110377 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd3d0102110 0x7fd3d0074300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:12.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.813+0000 7fd3d4e00640 1 -- 172.21.15.83:0/1461110377 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3d01a6f60 con 0x7fd3d0074d80 2024-04-24T09:00:12.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.813+0000 7fd3cfe00640 1 --2- 172.21.15.83:0/1461110377 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd3d0102110 0x7fd3d0074300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:00:12.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.814+0000 7fd3d4e00640 1 --2- 172.21.15.83:0/1461110377 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3d0074d80 0x7fd3d01a6860 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7fd3c000c970 tx=0x7fd3c000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:12.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.814+0000 7fd3ccc00640 1 -- 172.21.15.83:0/1461110377 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c0007bb0 con 0x7fd3d0074d80 2024-04-24T09:00:12.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.814+0000 7fd3ccc00640 1 -- 172.21.15.83:0/1461110377 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd3c0007d50 con 0x7fd3d0074d80 2024-04-24T09:00:12.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.814+0000 7fd3d5800640 1 -- 172.21.15.83:0/1461110377 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3d01a7250 con 0x7fd3d0074d80 2024-04-24T09:00:12.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.814+0000 7fd3ccc00640 1 -- 172.21.15.83:0/1461110377 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c0004e10 con 0x7fd3d0074d80 2024-04-24T09:00:12.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.814+0000 7fd3d5800640 1 -- 172.21.15.83:0/1461110377 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3d01a77a0 con 0x7fd3d0074d80 2024-04-24T09:00:12.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.818+0000 7fd3d9b8b740 1 Processor -- start 2024-04-24T09:00:12.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.818+0000 7fd3ccc00640 1 -- 172.21.15.83:0/1461110377 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd3c0005020 con 0x7fd3d0074d80 2024-04-24T09:00:12.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.818+0000 7fd3d9b8b740 1 -- start start 2024-04-24T09:00:12.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.818+0000 7fd3d9b8b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e05c7f5a30 0x55e05c7f5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:12.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.818+0000 7fd3d9b8b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e05c7f6350 0x55e05c8004e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:12.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.818+0000 7fd3d9b8b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e05c800a20 0x55e05c802e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:12.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.818+0000 7fd3d9b8b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e05c75b1f0 con 0x55e05c7f6350 2024-04-24T09:00:12.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.819+0000 7fd3d9b8b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e05c701350 con 0x55e05c800a20 2024-04-24T09:00:12.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.819+0000 7fd3d9b8b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55e05c7571b0 con 0x55e05c7f5a30 2024-04-24T09:00:12.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.819+0000 7fd3cfe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e05c7f6350 0x55e05c8004e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:12.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.819+0000 7fd3cfe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e05c7f6350 0x55e05c8004e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48994/0 (socket says 172.21.15.83:48994) 2024-04-24T09:00:12.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.819+0000 7fd3cfe00640 1 -- 172.21.15.83:0/3284932284 learned_addr learned my addr 172.21.15.83:0/3284932284 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:00:12.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.819+0000 7fd3cf400640 1 --2- 172.21.15.83:0/3284932284 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e05c7f5a30 0x55e05c7f5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:12.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.819+0000 7fd3cf400640 1 --2- 172.21.15.83:0/3284932284 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e05c800a20 0x55e05c802e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:12.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.819+0000 7fd3cfe00640 1 -- 172.21.15.83:0/3284932284 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e05c7f5a30 msgr2=0x55e05c7f5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:12.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.819+0000 7fd3ccc00640 1 --2- 172.21.15.83:0/1461110377 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fd3b40a3d50 0x7fd3b40a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:12.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.819+0000 7fd3cfe00640 1 --2- 172.21.15.83:0/3284932284 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e05c7f5a30 0x55e05c7f5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:12.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.819+0000 7fd3cfe00640 1 -- 172.21.15.83:0/3284932284 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e05c800a20 msgr2=0x55e05c802e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:12.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.819+0000 7fd3cfe00640 1 --2- 172.21.15.83:0/3284932284 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e05c800a20 0x55e05c802e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:12.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.819+0000 7fd3cfe00640 1 -- 172.21.15.83:0/3284932284 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e05c755f50 con 0x55e05c7f6350 2024-04-24T09:00:12.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.820+0000 7fd3ccc00640 1 -- 172.21.15.83:0/1461110377 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7fd3c00d08b0 con 0x7fd3d0074d80 2024-04-24T09:00:12.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.820+0000 7fd3d4e00640 1 --2- 172.21.15.83:0/1461110377 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fd3b40a3d50 0x7fd3b40a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:12.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.820+0000 7fd3cfe00640 1 --2- 172.21.15.83:0/3284932284 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e05c7f6350 0x55e05c8004e0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7fd3c8005cc0 tx=0x7fd3c8031b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=537a9bd0df2e59a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:12.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.820+0000 7fd3c5600640 1 -- 172.21.15.83:0/3284932284 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c8031650 con 0x55e05c7f6350 2024-04-24T09:00:12.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.820+0000 7fd3c5600640 1 -- 172.21.15.83:0/3284932284 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd3c8036040 con 0x55e05c7f6350 2024-04-24T09:00:12.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.820+0000 7fd3c5600640 1 -- 172.21.15.83:0/3284932284 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c8004670 con 0x55e05c7f6350 2024-04-24T09:00:12.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.821+0000 7fd3d9b8b740 1 -- 172.21.15.83:0/3284932284 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e05c7f6350 msgr2=0x55e05c8004e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:12.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.821+0000 7fd3d4e00640 1 --2- 172.21.15.83:0/1461110377 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fd3b40a3d50 0x7fd3b40a6210 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd3c00cdc90 tx=0x7fd3c0098040 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:12.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.821+0000 7fd3d9b8b740 1 --2- 172.21.15.83:0/3284932284 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e05c7f6350 0x55e05c8004e0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7fd3c8005cc0 tx=0x7fd3c8031b90 comp rx=0 tx=0).stop 2024-04-24T09:00:12.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.821+0000 7fd3d9b8b740 1 -- 172.21.15.83:0/3284932284 shutdown_connections 2024-04-24T09:00:12.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.821+0000 7fd3d9b8b740 1 --2- 172.21.15.83:0/3284932284 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e05c800a20 0x55e05c802e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:12.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.821+0000 7fd3d9b8b740 1 --2- 172.21.15.83:0/3284932284 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e05c7f6350 0x55e05c8004e0 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:12.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.821+0000 7fd3d9b8b740 1 --2- 172.21.15.83:0/3284932284 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e05c7f5a30 0x55e05c7f5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:12.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.821+0000 7fd3d9b8b740 1 -- 172.21.15.83:0/3284932284 >> 172.21.15.83:0/3284932284 conn(0x55e05c7f1d50 msgr2=0x55e05c7f2130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:12.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.821+0000 7fd3d9b8b740 1 -- 172.21.15.83:0/3284932284 shutdown_connections 2024-04-24T09:00:12.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.821+0000 7fd3d9b8b740 1 -- 172.21.15.83:0/3284932284 wait complete. 2024-04-24T09:00:12.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.822+0000 7fd3d9b8b740 1 -- start start 2024-04-24T09:00:12.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.822+0000 7fd3d9b8b740 1 Processor -- start 2024-04-24T09:00:12.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.823+0000 7fd3d9b8b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e05c7f5a30 0x55e05c7fd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:12.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.823+0000 7fd3d9b8b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e05c7f6350 0x55e05c7fdf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:12.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.823+0000 7fd3d9b8b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e05c800a20 0x55e05c7fe460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:12.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.823+0000 7fd3d9b8b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e05c8037b0 con 0x55e05c800a20 2024-04-24T09:00:12.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.823+0000 7fd3d9b8b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e05c714690 con 0x55e05c7f6350 2024-04-24T09:00:12.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.823+0000 7fd3d9b8b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55e05c803930 con 0x55e05c7f5a30 2024-04-24T09:00:12.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.823+0000 7fd3d9b8b740 10 client.0 _ll_register_callbacks cb 0x55e05c750650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:00:12.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.823+0000 7fd3cf400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e05c800a20 0x55e05c7fe460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:12.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.823+0000 7fd3cf400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e05c800a20 0x55e05c7fe460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48996/0 (socket says 172.21.15.83:48996) 2024-04-24T09:00:12.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.823+0000 7fd3cf400640 1 -- 172.21.15.83:0/3760246537 learned_addr learned my addr 172.21.15.83:0/3760246537 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:00:12.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.823+0000 7fd3cf400640 1 --2- 172.21.15.83:0/3760246537 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e05c7f5a30 0x55e05c7fd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:12.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.823+0000 7fd3cfe00640 1 --2- 172.21.15.83:0/3760246537 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e05c7f6350 0x55e05c7fdf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:12.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.823+0000 7fd3cf400640 1 -- 172.21.15.83:0/3760246537 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e05c7f5a30 msgr2=0x55e05c7fd9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:12.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.824+0000 7fd3cf400640 1 --2- 172.21.15.83:0/3760246537 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e05c7f5a30 0x55e05c7fd9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:12.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.824+0000 7fd3cf400640 1 -- 172.21.15.83:0/3760246537 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e05c7f6350 msgr2=0x55e05c7fdf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:12.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.824+0000 7fd3cf400640 1 --2- 172.21.15.83:0/3760246537 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e05c7f6350 0x55e05c7fdf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:12.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.824+0000 7fd3cf400640 1 -- 172.21.15.83:0/3760246537 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e05c713920 con 0x55e05c800a20 2024-04-24T09:00:12.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.824+0000 7fd3cfe00640 1 --2- 172.21.15.83:0/3760246537 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e05c7f6350 0x55e05c7fdf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:00:12.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.824+0000 7fd3cf400640 1 --2- 172.21.15.83:0/3760246537 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e05c800a20 0x55e05c7fe460 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7fd3b8011fc0 tx=0x7fd3b800e9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1893abc0abcf83c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:12.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.824+0000 7fd3be000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:00:12.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.824+0000 7fd3be000640 1 -- 172.21.15.83:0/3760246537 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b800de10 con 0x55e05c800a20 2024-04-24T09:00:12.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.824+0000 7fd3be000640 1 -- 172.21.15.83:0/3760246537 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7fd3b80047a0 con 0x55e05c800a20 2024-04-24T09:00:12.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.824+0000 7fd3d9b8b740 1 -- 172.21.15.83:0/3760246537 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3c8036040 con 0x55e05c800a20 2024-04-24T09:00:12.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.824+0000 7fd3be000640 1 -- 172.21.15.83:0/3760246537 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fd3b8025050 con 0x55e05c800a20 2024-04-24T09:00:12.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.824+0000 7fd3d9b8b740 10 client.18774 fetch_fsmap learned FSMap version 35 2024-04-24T09:00:12.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.824+0000 7fd3d9b8b740 1 -- 172.21.15.83:0/3760246537 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55e05c5377d0 con 0x55e05c800a20 2024-04-24T09:00:12.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.825+0000 7fd3be000640 1 -- 172.21.15.83:0/3760246537 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd3c8036040 con 0x55e05c800a20 2024-04-24T09:00:12.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.825+0000 7fd3be000640 1 -- 172.21.15.83:0/3760246537 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7fd3b801d580 con 0x55e05c800a20 2024-04-24T09:00:12.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.825+0000 7fd3d9b8b740 10 client.18774 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T09:00:12.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.825+0000 7fd3d9b8b740 10 client.18774 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:00:12.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.825+0000 7fd3d9b8b740 20 client.18774 populate_metadata read hostname 'smithi083' 2024-04-24T09:00:12.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.825+0000 7fd3d9b8b740 1 --2- 172.21.15.83:0/3760246537 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55e05c810dd0 0x55e05c813230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:12.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.825+0000 7fd3d9b8b740 4 client.18774 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:00:12.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.825+0000 7fd3d9b8b740 1 -- 172.21.15.83:0/3760246537 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e05c768c10 con 0x55e05c810dd0 2024-04-24T09:00:12.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.826+0000 7fd3cfe00640 1 --2- 172.21.15.83:0/3760246537 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55e05c810dd0 0x55e05c813230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:12.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.827+0000 7fd3cfe00640 1 --2- 172.21.15.83:0/3760246537 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55e05c810dd0 0x55e05c813230 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a68898b989a5ca22 server_cookie=45c5932bc56d1b3d in_seq=0 out_seq=0 2024-04-24T09:00:12.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.827+0000 7fd3be000640 10 client.18774 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:00:12.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.829+0000 7fd3be000640 1 -- 172.21.15.83:0/3760246537 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e05c768c10 con 0x55e05c810dd0 2024-04-24T09:00:12.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.829+0000 7fd3be000640 10 client.18774 handle_command_reply: tid=0 2024-04-24T09:00:12.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.829+0000 7fd3d9b8b740 1 client.18774 shutdown 2024-04-24T09:00:12.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.829+0000 7fd3d9b8b740 1 -- 172.21.15.83:0/3760246537 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e05c800a20 msgr2=0x55e05c7fe460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:12.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.829+0000 7fd3d9b8b740 1 --2- 172.21.15.83:0/3760246537 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e05c800a20 0x55e05c7fe460 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7fd3b8011fc0 tx=0x7fd3b800e9d0 comp rx=0 tx=0).stop 2024-04-24T09:00:12.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.829+0000 7fd3d9b8b740 1 -- 172.21.15.83:0/3760246537 shutdown_connections 2024-04-24T09:00:12.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.829+0000 7fd3d9b8b740 1 --2- 172.21.15.83:0/3760246537 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55e05c810dd0 0x55e05c813230 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:12.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.829+0000 7fd3d9b8b740 1 --2- 172.21.15.83:0/3760246537 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e05c800a20 0x55e05c7fe460 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:12.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.829+0000 7fd3d9b8b740 1 --2- 172.21.15.83:0/3760246537 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e05c7f6350 0x55e05c7fdf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:12.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.829+0000 7fd3d9b8b740 1 --2- 172.21.15.83:0/3760246537 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e05c7f5a30 0x55e05c7fd9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:12.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.829+0000 7fd3be000640 0 client.18774 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:00:12.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.829+0000 7fd3d9b8b740 1 -- 172.21.15.83:0/3760246537 >> 172.21.15.83:0/3760246537 conn(0x55e05c7f1d50 msgr2=0x55e05c7f0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:12.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.829+0000 7fd3d9b8b740 1 -- 172.21.15.83:0/3760246537 shutdown_connections 2024-04-24T09:00:12.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.829+0000 7fd3d9b8b740 1 -- 172.21.15.83:0/3760246537 wait complete. 2024-04-24T09:00:12.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.830+0000 7fd3d9b8b740 20 client.18774 trim_cache size 0 max 16384 2024-04-24T09:00:12.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.850+0000 7fd3d9b8b740 1 Processor -- start 2024-04-24T09:00:12.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.850+0000 7fd3d9b8b740 1 -- start start 2024-04-24T09:00:12.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.850+0000 7fd3d9b8b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e05c7f5a30 0x55e05c7fe230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:12.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.850+0000 7fd3d9b8b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e05c7f6350 0x55e05c7f81d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:12.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.850+0000 7fd3d9b8b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e05c800a20 0x55e05c818b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:12.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.850+0000 7fd3d9b8b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd3d01b1670 con 0x55e05c7f5a30 2024-04-24T09:00:12.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.850+0000 7fd3d9b8b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd3d01b31b0 con 0x55e05c7f6350 2024-04-24T09:00:12.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.850+0000 7fd3d9b8b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd3d01b1d40 con 0x55e05c800a20 2024-04-24T09:00:12.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.850+0000 7fd3cf400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e05c7f5a30 0x55e05c7fe230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:12.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.850+0000 7fd3cf400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e05c7f5a30 0x55e05c7fe230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49012/0 (socket says 172.21.15.83:49012) 2024-04-24T09:00:12.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.850+0000 7fd3cf400640 1 -- 172.21.15.83:0/1168592797 learned_addr learned my addr 172.21.15.83:0/1168592797 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:00:12.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.851+0000 7fd3cf400640 1 -- 172.21.15.83:0/1168592797 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e05c800a20 msgr2=0x55e05c818b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:12.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.851+0000 7fd3cf400640 1 --2- 172.21.15.83:0/1168592797 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e05c800a20 0x55e05c818b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:12.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.851+0000 7fd3cf400640 1 -- 172.21.15.83:0/1168592797 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e05c7f6350 msgr2=0x55e05c7f81d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:12.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.851+0000 7fd3cf400640 1 --2- 172.21.15.83:0/1168592797 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e05c7f6350 0x55e05c7f81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:12.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.851+0000 7fd3cf400640 1 -- 172.21.15.83:0/1168592797 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e05c7f6180 con 0x55e05c7f5a30 2024-04-24T09:00:12.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.851+0000 7fd3cf400640 1 --2- 172.21.15.83:0/1168592797 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e05c7f5a30 0x55e05c7fe230 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7fd3b8005ee0 tx=0x7fd3b8011c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceaaa2460e601eb6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:12.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.851+0000 7fd3c4c00640 1 -- 172.21.15.83:0/1168592797 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b800dba0 con 0x55e05c7f5a30 2024-04-24T09:00:12.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.851+0000 7fd3c4c00640 1 -- 172.21.15.83:0/1168592797 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd3b802e070 con 0x55e05c7f5a30 2024-04-24T09:00:12.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.851+0000 7fd3c4c00640 1 -- 172.21.15.83:0/1168592797 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b8005430 con 0x55e05c7f5a30 2024-04-24T09:00:12.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.852+0000 7fd3d9b8b740 1 -- 172.21.15.83:0/1168592797 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e05c7f5a30 msgr2=0x55e05c7fe230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:12.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.852+0000 7fd3d9b8b740 1 --2- 172.21.15.83:0/1168592797 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e05c7f5a30 0x55e05c7fe230 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7fd3b8005ee0 tx=0x7fd3b8011c90 comp rx=0 tx=0).stop 2024-04-24T09:00:12.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.852+0000 7fd3d9b8b740 1 -- 172.21.15.83:0/1168592797 shutdown_connections 2024-04-24T09:00:12.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.852+0000 7fd3d9b8b740 1 --2- 172.21.15.83:0/1168592797 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e05c800a20 0x55e05c818b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:12.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.852+0000 7fd3d9b8b740 1 --2- 172.21.15.83:0/1168592797 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e05c7f6350 0x55e05c7f81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:12.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.852+0000 7fd3d9b8b740 1 --2- 172.21.15.83:0/1168592797 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e05c7f5a30 0x55e05c7fe230 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:12.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.852+0000 7fd3d9b8b740 1 -- 172.21.15.83:0/1168592797 >> 172.21.15.83:0/1168592797 conn(0x55e05c7f1d50 msgr2=0x55e05c7f0740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:12.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.852+0000 7fd3d9b8b740 1 -- 172.21.15.83:0/1168592797 shutdown_connections 2024-04-24T09:00:12.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.852+0000 7fd3d9b8b740 1 -- 172.21.15.83:0/1168592797 wait complete. 2024-04-24T09:00:12.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.853+0000 7fd3d9b8b740 1 -- start start 2024-04-24T09:00:12.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.853+0000 7fd3d9b8b740 1 Processor -- start 2024-04-24T09:00:12.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.853+0000 7fd3d9b8b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e05c7f5a30 0x55e05c808eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:12.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.853+0000 7fd3d9b8b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e05c7f6350 0x55e05c7fa330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:12.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.853+0000 7fd3d9b8b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e05c800a20 0x55e05c7fa870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:12.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.853+0000 7fd3d9b8b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd3d01b2ae0 con 0x55e05c7f6350 2024-04-24T09:00:12.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.853+0000 7fd3d9b8b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd3d01b2410 con 0x55e05c800a20 2024-04-24T09:00:12.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.853+0000 7fd3d9b8b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd3d01af670 con 0x55e05c7f5a30 2024-04-24T09:00:12.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.853+0000 7fd3d9b8b740 10 client.0 _ll_register_callbacks cb 0x7fd3b800e8f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:00:12.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.854+0000 7fd3cfe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e05c7f6350 0x55e05c7fa330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:12.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.854+0000 7fd3cfe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e05c7f6350 0x55e05c7fa330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49024/0 (socket says 172.21.15.83:49024) 2024-04-24T09:00:12.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.854+0000 7fd3cfe00640 1 -- 172.21.15.83:0/148414469 learned_addr learned my addr 172.21.15.83:0/148414469 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:00:12.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.854+0000 7fd3cf400640 1 --2- 172.21.15.83:0/148414469 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e05c7f5a30 0x55e05c808eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:12.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.854+0000 7fd3cf400640 1 --2- 172.21.15.83:0/148414469 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e05c800a20 0x55e05c7fa870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:12.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.854+0000 7fd3cfe00640 1 -- 172.21.15.83:0/148414469 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e05c7f5a30 msgr2=0x55e05c808eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:12.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.854+0000 7fd3cfe00640 1 --2- 172.21.15.83:0/148414469 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e05c7f5a30 0x55e05c808eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:12.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.854+0000 7fd3cfe00640 1 -- 172.21.15.83:0/148414469 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e05c800a20 msgr2=0x55e05c7fa870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:12.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.854+0000 7fd3cfe00640 1 --2- 172.21.15.83:0/148414469 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e05c800a20 0x55e05c7fa870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:12.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.854+0000 7fd3cfe00640 1 -- 172.21.15.83:0/148414469 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e05c46d3f0 con 0x55e05c7f6350 2024-04-24T09:00:12.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.854+0000 7fd3cfe00640 1 --2- 172.21.15.83:0/148414469 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e05c7f6350 0x55e05c7fa330 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7fd3c8009a60 tx=0x7fd3c802a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fb07f7cfa44407a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:12.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.854+0000 7fd3be000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:00:12.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.855+0000 7fd3be000640 1 -- 172.21.15.83:0/148414469 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c8058060 con 0x55e05c7f6350 2024-04-24T09:00:12.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.855+0000 7fd3be000640 1 -- 172.21.15.83:0/148414469 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7fd3c80315f0 con 0x55e05c7f6350 2024-04-24T09:00:12.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.855+0000 7fd3d9b8b740 1 -- 172.21.15.83:0/148414469 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3b802e070 con 0x55e05c7f6350 2024-04-24T09:00:12.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.855+0000 7fd3be000640 1 -- 172.21.15.83:0/148414469 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fd3c80520d0 con 0x55e05c7f6350 2024-04-24T09:00:12.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.855+0000 7fd3d9b8b740 10 client.18780 fetch_fsmap learned FSMap version 35 2024-04-24T09:00:12.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.855+0000 7fd3d9b8b740 1 -- 172.21.15.83:0/148414469 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55e05c42d0f0 con 0x55e05c7f6350 2024-04-24T09:00:12.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.855+0000 7fd3be000640 1 -- 172.21.15.83:0/148414469 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd3b802e070 con 0x55e05c7f6350 2024-04-24T09:00:12.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.856+0000 7fd3be000640 1 -- 172.21.15.83:0/148414469 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7fd3c80308c0 con 0x55e05c7f6350 2024-04-24T09:00:12.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.856+0000 7fd3d9b8b740 10 client.18780 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T09:00:12.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.856+0000 7fd3d9b8b740 10 client.18780 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:00:12.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.856+0000 7fd3d9b8b740 20 client.18780 populate_metadata read hostname 'smithi083' 2024-04-24T09:00:12.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.856+0000 7fd3d9b8b740 1 --2- 172.21.15.83:0/148414469 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55e05c877250 0x55e05c879640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:12.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.856+0000 7fd3d9b8b740 4 client.18780 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T09:00:12.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.856+0000 7fd3d9b8b740 1 -- 172.21.15.83:0/148414469 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e05c762a80 con 0x55e05c877250 2024-04-24T09:00:12.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.857+0000 7fd3cf400640 1 --2- 172.21.15.83:0/148414469 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55e05c877250 0x55e05c879640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:12.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.858+0000 7fd3cf400640 1 --2- 172.21.15.83:0/148414469 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55e05c877250 0x55e05c879640 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3737e4d3907944a server_cookie=9d9b209f708b79d0 in_seq=0 out_seq=0 2024-04-24T09:00:12.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.858+0000 7fd3be000640 10 client.18780 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:00:12.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.860+0000 7fd3be000640 1 -- 172.21.15.83:0/148414469 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55e05c762a80 con 0x55e05c877250 2024-04-24T09:00:12.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.860+0000 7fd3be000640 10 client.18780 handle_command_reply: tid=0 2024-04-24T09:00:12.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.861+0000 7fd3d9b8b740 1 client.18780 shutdown 2024-04-24T09:00:12.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.861+0000 7fd3d9b8b740 1 -- 172.21.15.83:0/148414469 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e05c7f6350 msgr2=0x55e05c7fa330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:12.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.861+0000 7fd3d9b8b740 1 --2- 172.21.15.83:0/148414469 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e05c7f6350 0x55e05c7fa330 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7fd3c8009a60 tx=0x7fd3c802a8c0 comp rx=0 tx=0).stop 2024-04-24T09:00:12.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.861+0000 7fd3d9b8b740 1 -- 172.21.15.83:0/148414469 shutdown_connections 2024-04-24T09:00:12.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.861+0000 7fd3d9b8b740 1 --2- 172.21.15.83:0/148414469 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55e05c877250 0x55e05c879640 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:12.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.861+0000 7fd3d9b8b740 1 --2- 172.21.15.83:0/148414469 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e05c800a20 0x55e05c7fa870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:12.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.861+0000 7fd3d9b8b740 1 --2- 172.21.15.83:0/148414469 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e05c7f6350 0x55e05c7fa330 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:12.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.861+0000 7fd3be000640 0 client.18780 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:00:12.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.861+0000 7fd3d9b8b740 1 --2- 172.21.15.83:0/148414469 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e05c7f5a30 0x55e05c808eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:12.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.861+0000 7fd3d9b8b740 1 -- 172.21.15.83:0/148414469 >> 172.21.15.83:0/148414469 conn(0x55e05c7f1d50 msgr2=0x55e05c7fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:12.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.861+0000 7fd3d9b8b740 1 -- 172.21.15.83:0/148414469 shutdown_connections 2024-04-24T09:00:12.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.861+0000 7fd3d9b8b740 1 -- 172.21.15.83:0/148414469 wait complete. 2024-04-24T09:00:12.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.861+0000 7fd3d9b8b740 20 client.18780 trim_cache size 0 max 16384 2024-04-24T09:00:12.862 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T09:00:12.862 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T09:00:12.862 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-04-24T09:00:12.862 INFO:teuthology.orchestra.run.smithi083.stdout: "85540fc2-14ea-4744-bd0a-88a98038b8cf": { 2024-04-24T09:00:12.862 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-04-24T09:00:12.862 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "85540fc2-14ea-4744-bd0a-88a98038b8cf", 2024-04-24T09:00:12.862 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-04-24T09:00:12.862 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:00:12.862 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:00:12.862 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T09:00:12.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.862+0000 7fd3d5800640 1 -- 172.21.15.83:0/1461110377 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fd3b40a3d50 msgr2=0x7fd3b40a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:12.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.862+0000 7fd3d5800640 1 --2- 172.21.15.83:0/1461110377 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fd3b40a3d50 0x7fd3b40a6210 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd3c00cdc90 tx=0x7fd3c0098040 comp rx=0 tx=0).stop 2024-04-24T09:00:12.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.862+0000 7fd3d5800640 1 -- 172.21.15.83:0/1461110377 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3d0074d80 msgr2=0x7fd3d01a6860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:12.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.862+0000 7fd3d5800640 1 --2- 172.21.15.83:0/1461110377 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3d0074d80 0x7fd3d01a6860 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7fd3c000c970 tx=0x7fd3c000ce30 comp rx=0 tx=0).stop 2024-04-24T09:00:12.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.862+0000 7fd3d5800640 1 -- 172.21.15.83:0/1461110377 shutdown_connections 2024-04-24T09:00:12.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.862+0000 7fd3d5800640 1 --2- 172.21.15.83:0/1461110377 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fd3b40a3d50 0x7fd3b40a6210 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:12.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.862+0000 7fd3d5800640 1 --2- 172.21.15.83:0/1461110377 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3d0074d80 0x7fd3d01a6860 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:12.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.862+0000 7fd3d5800640 1 --2- 172.21.15.83:0/1461110377 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd3d01071e0 0x7fd3d0074840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:12.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.862+0000 7fd3d5800640 1 --2- 172.21.15.83:0/1461110377 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd3d0102110 0x7fd3d0074300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:12.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.862+0000 7fd3d5800640 1 -- 172.21.15.83:0/1461110377 >> 172.21.15.83:0/1461110377 conn(0x7fd3d00fd9d0 msgr2=0x7fd3d00ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:12.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.862+0000 7fd3d5800640 1 -- 172.21.15.83:0/1461110377 shutdown_connections 2024-04-24T09:00:12.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:12.863+0000 7fd3d5800640 1 -- 172.21.15.83:0/1461110377 wait complete. 2024-04-24T09:00:12.881 INFO:tasks.cephfs.filesystem:scrub status for tag:85540fc2-14ea-4744-bd0a-88a98038b8cf - {'path': '/', 'tag': '85540fc2-14ea-4744-bd0a-88a98038b8cf', 'options': 'recursive,force'} 2024-04-24T09:00:12.955 INFO:journalctl@ceph.mds.b.smithi138.stdout:Apr 24 09:00:12 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-b[61515]: 2024-04-24T09:00:12.559+0000 7f7fbbc00700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000020c [...e,head] /volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/EXCEL/ auth{3=1} v8328 ap=2 f(v470 m2024-04-24T08:59:59.580626+0000 2=2+0) n(v35 rc2024-04-24T08:59:59.580626+0000 b520192 3=2+1) old_inodes=6 (inest sync->mix g=3) caps={15603=pAsLsXsFs/-@7473} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x55cf8760a580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000020c:[<0x10000000009/EXCEL v347>,<0x10000000008/~dmtmp v57>,<0x10000000007/client0 v33>,<0x10000000006/clients v23>,<0x10000000005/tmp v23>,<0x10000000003/client.0 v22>,<0x10000000002/e395bed6-5bba-41b8-a256-3505e4afcca2 v36>,<0x10000000001/sv_0 v55>,<0x10000000000/qa v43>,<0x1/volumes v39>]//[]","memoryvalue":"(2)0x1000000020c:[<0x10000000009/EXCEL v8328>,<0x10000000008/~dmtmp v7755>,<0x10000000007/client0 v915>,<0x10000000006/clients v907>,<0x10000000005/tmp v904>,<0x10000000003/client.0 v901>,<0x10000000002/e395bed6-5bba-41b8-a256-3505e4afcca2 v896>,<0x10000000001/sv_0 v891>,<0x10000000000/qa v881>,<0x1/volumes v855>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-04-24T08:59:59.562795+0000 3=3+0)","ondisk_value.rstat":"n(v0 rc2024-04-24T08:59:59.580626+0000 b520192 3=2+1)","memory_value.dirstat":"f(v470 m2024-04-24T08:59:59.580626+0000 2=2+0)","memory_value.rstat":"n(v35 rc2024-04-24T08:59:59.580626+0000 b520192 3=2+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-04-24T09:00:12.955 INFO:journalctl@ceph.mds.b.smithi138.stdout:Apr 24 09:00:12 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-b[61515]: 2024-04-24T09:00:12.564+0000 7f7fbbc00700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000215 [...e,head] /volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/COREL/ auth{2=1} v8360 ap=2 f(v338 m2024-04-24T08:59:59.813863+0000 11=11+0) n(v19 rc2024-04-24T08:59:59.813863+0000 b2861056 12=11+1) old_inodes=6 (inest sync->mix g=2) caps={15603=pAsLsXsFs/-@6982} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x55cf8760d700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000000215:[<0x10000000009/COREL v369>,<0x10000000008/~dmtmp v57>,<0x10000000007/client0 v33>,<0x10000000006/clients v23>,<0x10000000005/tmp v23>,<0x10000000003/client.0 v22>,<0x10000000002/e395bed6-5bba-41b8-a256-3505e4afcca2 v36>,<0x10000000001/sv_0 v55>,<0x10000000000/qa v43>,<0x1/volumes v39>]//[]","memoryvalue":"(2)0x10000000215:[<0x10000000009/COREL v8360>,<0x10000000008/~dmtmp v7755>,<0x10000000007/client0 v915>,<0x10000000006/clients v907>,<0x10000000005/tmp v904>,<0x10000000003/client.0 v901>,<0x10000000002/e395bed6-5bba-41b8-a256-3505e4afcca2 v896>,<0x10000000001/sv_0 v891>,<0x10000000000/qa v881>,<0x1/volumes v855>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-04-24T08:59:59.773366+0000 12=12+0)","ondisk_value.rstat":"n(v0 rc2024-04-24T08:59:59.813863+0000 b2861056 12=11+1)","memory_value.dirstat":"f(v338 m2024-04-24T08:59:59.813863+0000 11=11+0)","memory_value.rstat":"n(v19 rc2024-04-24T08:59:59.813863+0000 b2861056 12=11+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-04-24T09:00:12.955 INFO:journalctl@ceph.mds.b.smithi138.stdout:Apr 24 09:00:12 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-b[61515]: 2024-04-24T09:00:12.568+0000 7f7fbbc00700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000221 [...e,head] /volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/WORDPRO/ auth{3=1} v8330 ap=2 f(v455 m2024-04-24T09:00:00.024203+0000 3=3+0) n(v29 rc2024-04-24T09:00:00.024203+0000 b780288 4=3+1) old_inodes=6 (inest sync->mix g=3) caps={15603=pAsLsXsFs/-@7358} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x55cf87627700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000000221:[<0x10000000009/WORDPRO v375>,<0x10000000008/~dmtmp v57>,<0x10000000007/client0 v33>,<0x10000000006/clients v23>,<0x10000000005/tmp v23>,<0x10000000003/client.0 v22>,<0x10000000002/e395bed6-5bba-41b8-a256-3505e4afcca2 v36>,<0x10000000001/sv_0 v55>,<0x10000000000/qa v43>,<0x1/volumes v39>]//[]","memoryvalue":"(2)0x10000000221:[<0x10000000009/WORDPRO v8330>,<0x10000000008/~dmtmp v7757>,<0x10000000007/client0 v917>,<0x10000000006/clients v909>,<0x10000000005/tmp v906>,<0x10000000003/client.0 v903>,<0x10000000002/e395bed6-5bba-41b8-a256-3505e4afcca2 v898>,<0x10000000001/sv_0 v893>,<0x10000000000/qa v881>,<0x1/volumes v855>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-04-24T08:59:59.991043+0000 4=4+0)","ondisk_value.rstat":"n(v0 rc2024-04-24T09:00:00.024203+0000 b780288 4=3+1)","memory_value.dirstat":"f(v455 m2024-04-24T09:00:00.024203+0000 3=3+0)","memory_value.rstat":"n(v29 rc2024-04-24T09:00:00.024203+0000 b780288 4=3+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-04-24T09:00:12.955 INFO:journalctl@ceph.mds.b.smithi138.stdout:Apr 24 09:00:12 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-b[61515]: 2024-04-24T09:00:12.570+0000 7f7fbbc00700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000225 [...e,head] /volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/PM/ auth{3=1} v8390 ap=2 f(v962 m2024-04-24T09:00:00.048241+0000 3=3+0) n(v26 rc2024-04-24T09:00:00.048241+0000 b780288 4=3+1) old_inodes=6 (inest sync->mix g=3) caps={15603=pAsLsXsFs/-@9436} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x55cf8762f180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000000225:[<0x10000000009/PM v381>,<0x10000000008/~dmtmp v57>,<0x10000000007/client0 v33>,<0x10000000006/clients v23>,<0x10000000005/tmp v23>,<0x10000000003/client.0 v22>,<0x10000000002/e395bed6-5bba-41b8-a256-3505e4afcca2 v36>,<0x10000000001/sv_0 v55>,<0x10000000000/qa v43>,<0x1/volumes v39>]//[]","memoryvalue":"(2)0x10000000225:[<0x10000000009/PM v8390>,<0x10000000008/~dmtmp v7757>,<0x10000000007/client0 v917>,<0x10000000006/clients v909>,<0x10000000005/tmp v906>,<0x10000000003/client.0 v903>,<0x10000000002/e395bed6-5bba-41b8-a256-3505e4afcca2 v898>,<0x10000000001/sv_0 v893>,<0x10000000000/qa v881>,<0x1/volumes v855>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-04-24T08:59:58.861529+0000 4=4+0)","ondisk_value.rstat":"n(v0 rc2024-04-24T09:00:00.048241+0000 b780288 4=3+1)","memory_value.dirstat":"f(v962 m2024-04-24T09:00:00.048241+0000 3=3+0)","memory_value.rstat":"n(v26 rc2024-04-24T09:00:00.048241+0000 b780288 4=3+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-04-24T09:00:12.955 INFO:journalctl@ceph.mds.b.smithi138.stdout:Apr 24 09:00:12 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-b[61515]: 2024-04-24T09:00:12.580+0000 7f7fbbc00700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000023e [...e,head] /volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/ACCESS/ auth{3=1} v8334 ap=2 f(v182 m2024-04-24T09:00:00.275353+0000 1=1+0) n(v28 rc2024-04-24T09:00:00.275353+0000 b260096 2=1+1) old_inodes=6 (inest sync->mix g=3) caps={15603=pAsLsXsFs/-@6284} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x55cf87666b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000023e:[<0x10000000009/ACCESS v463>,<0x10000000008/~dmtmp v57>,<0x10000000007/client0 v33>,<0x10000000006/clients v23>,<0x10000000005/tmp v23>,<0x10000000003/client.0 v22>,<0x10000000002/e395bed6-5bba-41b8-a256-3505e4afcca2 v36>,<0x10000000001/sv_0 v55>,<0x10000000000/qa v43>,<0x1/volumes v39>]//[]","memoryvalue":"(2)0x1000000023e:[<0x10000000009/ACCESS v8334>,<0x10000000008/~dmtmp v7759>,<0x10000000007/client0 v917>,<0x10000000006/clients v909>,<0x10000000005/tmp v906>,<0x10000000003/client.0 v903>,<0x10000000002/e395bed6-5bba-41b8-a256-3505e4afcca2 v898>,<0x10000000001/sv_0 v893>,<0x10000000000/qa v881>,<0x1/volumes v855>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-04-24T09:00:00.244871+0000 2=2+0)","ondisk_value.rstat":"n(v0 rc2024-04-24T09:00:00.275353+0000 b260096 2=1+1)","memory_value.dirstat":"f(v182 m2024-04-24T09:00:00.275353+0000 1=1+0)","memory_value.rstat":"n(v28 rc2024-04-24T09:00:00.275353+0000 b260096 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-04-24T09:00:13.598 INFO:tasks.workunit.client.0.smithi083.stdout: 1 48521 0.62 MB/sec execute 287 sec latency 55.271 ms 2024-04-24T09:00:14.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:00:13 smithi083 ceph-mon[27907]: scrub queued for path: / 2024-04-24T09:00:14.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:00:13 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:00:14.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:00:13 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:00:14.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:00:13 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:00:14.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:00:13 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:00:14.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:00:13 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:00:14.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:00:13 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:00:14.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:00:13 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:00:14.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:00:13 smithi083 ceph-mon[27907]: Scrub error on inode 0x1000000020c (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/EXCEL) see mds.b log and `damage ls` output for details 2024-04-24T09:00:14.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:00:13 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:00:14.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:00:13 smithi083 ceph-mon[27907]: Scrub error on inode 0x10000000215 (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/COREL) see mds.b log and `damage ls` output for details 2024-04-24T09:00:14.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:00:13 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:00:14.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:00:13 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:00:14.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:00:13 smithi083 ceph-mon[27907]: Scrub error on inode 0x10000000221 (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/WORDPRO) see mds.b log and `damage ls` output for details 2024-04-24T09:00:14.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:00:13 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:00:14.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:00:13 smithi083 ceph-mon[27907]: Scrub error on inode 0x10000000225 (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/PM) see mds.b log and `damage ls` output for details 2024-04-24T09:00:14.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:00:13 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:00:14.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:00:13 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:00:14.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:00:13 smithi083 ceph-mon[27907]: Scrub error on inode 0x1000000023e (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/ACCESS) see mds.b log and `damage ls` output for details 2024-04-24T09:00:14.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:00:13 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:00:14.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:00:13 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:00:14.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:00:13 smithi083 ceph-mon[27907]: pgmap v269: 129 pgs: 129 active+clean; 50 MiB data, 732 MiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 317 KiB/s wr, 54 op/s 2024-04-24T09:00:14.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:00:13 smithi138 ceph-mon[31441]: scrub queued for path: / 2024-04-24T09:00:14.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:00:13 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:00:14.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:00:13 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:00:14.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:00:13 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:00:14.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:00:13 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:00:14.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:00:13 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:00:14.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:00:13 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:00:14.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:00:13 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:00:14.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:00:13 smithi138 ceph-mon[31441]: Scrub error on inode 0x1000000020c (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/EXCEL) see mds.b log and `damage ls` output for details 2024-04-24T09:00:14.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:00:13 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:00:14.207 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:00:13 smithi138 ceph-mon[31441]: Scrub error on inode 0x10000000215 (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/COREL) see mds.b log and `damage ls` output for details 2024-04-24T09:00:14.207 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:00:13 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:00:14.207 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:00:13 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:00:14.207 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:00:13 smithi138 ceph-mon[31441]: Scrub error on inode 0x10000000221 (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/WORDPRO) see mds.b log and `damage ls` output for details 2024-04-24T09:00:14.207 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:00:13 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:00:14.207 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:00:13 smithi138 ceph-mon[31441]: Scrub error on inode 0x10000000225 (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/PM) see mds.b log and `damage ls` output for details 2024-04-24T09:00:14.207 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:00:13 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:00:14.207 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:00:13 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:00:14.207 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:00:13 smithi138 ceph-mon[31441]: Scrub error on inode 0x1000000023e (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/ACCESS) see mds.b log and `damage ls` output for details 2024-04-24T09:00:14.207 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:00:13 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:00:14.207 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:00:13 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:00:14.207 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:00:13 smithi138 ceph-mon[31441]: pgmap v269: 129 pgs: 129 active+clean; 50 MiB data, 732 MiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 317 KiB/s wr, 54 op/s 2024-04-24T09:00:14.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:00:13 smithi189 ceph-mon[31791]: scrub queued for path: / 2024-04-24T09:00:14.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:00:13 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:00:14.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:00:13 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:00:14.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:00:13 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:00:14.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:00:13 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:00:14.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:00:13 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:00:14.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:00:13 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:00:14.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:00:13 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:00:14.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:00:13 smithi189 ceph-mon[31791]: Scrub error on inode 0x1000000020c (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/EXCEL) see mds.b log and `damage ls` output for details 2024-04-24T09:00:14.209 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:00:13 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:00:14.209 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:00:13 smithi189 ceph-mon[31791]: Scrub error on inode 0x10000000215 (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/COREL) see mds.b log and `damage ls` output for details 2024-04-24T09:00:14.209 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:00:13 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:00:14.209 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:00:13 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:00:14.209 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:00:13 smithi189 ceph-mon[31791]: Scrub error on inode 0x10000000221 (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/WORDPRO) see mds.b log and `damage ls` output for details 2024-04-24T09:00:14.209 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:00:13 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:00:14.209 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:00:13 smithi189 ceph-mon[31791]: Scrub error on inode 0x10000000225 (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/PM) see mds.b log and `damage ls` output for details 2024-04-24T09:00:14.209 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:00:13 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:00:14.209 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:00:13 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:00:14.209 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:00:13 smithi189 ceph-mon[31791]: Scrub error on inode 0x1000000023e (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/ACCESS) see mds.b log and `damage ls` output for details 2024-04-24T09:00:14.209 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:00:13 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:00:14.209 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:00:13 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:00:14.209 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:00:13 smithi189 ceph-mon[31791]: pgmap v269: 129 pgs: 129 active+clean; 50 MiB data, 732 MiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 317 KiB/s wr, 54 op/s 2024-04-24T09:00:14.598 INFO:tasks.workunit.client.0.smithi083.stdout: 1 48605 0.62 MB/sec execute 288 sec latency 54.062 ms 2024-04-24T09:00:15.598 INFO:tasks.workunit.client.0.smithi083.stdout: 1 48727 0.62 MB/sec execute 289 sec latency 57.095 ms 2024-04-24T09:00:16.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:00:15 smithi083 ceph-mon[27907]: pgmap v270: 129 pgs: 129 active+clean; 50 MiB data, 732 MiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 317 KiB/s wr, 54 op/s 2024-04-24T09:00:16.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:00:15 smithi138 ceph-mon[31441]: pgmap v270: 129 pgs: 129 active+clean; 50 MiB data, 732 MiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 317 KiB/s wr, 54 op/s 2024-04-24T09:00:16.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:00:15 smithi189 ceph-mon[31791]: pgmap v270: 129 pgs: 129 active+clean; 50 MiB data, 732 MiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 317 KiB/s wr, 54 op/s 2024-04-24T09:00:16.598 INFO:tasks.workunit.client.0.smithi083.stdout: 1 48908 0.63 MB/sec execute 290 sec latency 52.691 ms 2024-04-24T09:00:17.599 INFO:tasks.workunit.client.0.smithi083.stdout: 1 49109 0.63 MB/sec execute 291 sec latency 55.979 ms 2024-04-24T09:00:18.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:00:17 smithi083 ceph-mon[27907]: pgmap v271: 129 pgs: 129 active+clean; 50 MiB data, 732 MiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 317 KiB/s wr, 54 op/s 2024-04-24T09:00:18.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:00:17 smithi138 ceph-mon[31441]: pgmap v271: 129 pgs: 129 active+clean; 50 MiB data, 732 MiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 317 KiB/s wr, 54 op/s 2024-04-24T09:00:18.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:00:17 smithi189 ceph-mon[31791]: pgmap v271: 129 pgs: 129 active+clean; 50 MiB data, 732 MiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 317 KiB/s wr, 54 op/s 2024-04-24T09:00:18.599 INFO:tasks.workunit.client.0.smithi083.stdout: 1 49318 0.63 MB/sec execute 292 sec latency 55.109 ms 2024-04-24T09:00:19.599 INFO:tasks.workunit.client.0.smithi083.stdout: 1 49432 0.64 MB/sec execute 293 sec latency 70.003 ms 2024-04-24T09:00:20.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:00:19 smithi083 ceph-mon[27907]: pgmap v272: 129 pgs: 129 active+clean; 53 MiB data, 741 MiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 511 KiB/s wr, 62 op/s 2024-04-24T09:00:20.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:00:19 smithi138 ceph-mon[31441]: pgmap v272: 129 pgs: 129 active+clean; 53 MiB data, 741 MiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 511 KiB/s wr, 62 op/s 2024-04-24T09:00:20.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:00:19 smithi189 ceph-mon[31791]: pgmap v272: 129 pgs: 129 active+clean; 53 MiB data, 741 MiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 511 KiB/s wr, 62 op/s 2024-04-24T09:00:20.599 INFO:tasks.workunit.client.0.smithi083.stdout: 1 49461 0.63 MB/sec execute 294 sec latency 61.824 ms 2024-04-24T09:00:21.599 INFO:tasks.workunit.client.0.smithi083.stdout: 1 49502 0.63 MB/sec execute 295 sec latency 59.351 ms 2024-04-24T09:00:22.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:00:21 smithi083 ceph-mon[27907]: pgmap v273: 129 pgs: 129 active+clean; 53 MiB data, 741 MiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 459 KiB/s wr, 56 op/s 2024-04-24T09:00:22.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:00:21 smithi138 ceph-mon[31441]: pgmap v273: 129 pgs: 129 active+clean; 53 MiB data, 741 MiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 459 KiB/s wr, 56 op/s 2024-04-24T09:00:22.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:00:21 smithi189 ceph-mon[31791]: pgmap v273: 129 pgs: 129 active+clean; 53 MiB data, 741 MiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 459 KiB/s wr, 56 op/s 2024-04-24T09:00:22.599 INFO:tasks.workunit.client.0.smithi083.stdout: 1 49679 0.64 MB/sec execute 296 sec latency 62.436 ms 2024-04-24T09:00:23.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:00:22 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:00:23.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:00:22 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:00:23.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:00:22 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:00:23.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:00:22 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:00:23.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:00:22 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:00:23.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:00:22 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:00:23.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:00:22 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:00:23.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:00:22 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:00:23.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:00:22 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:00:23.599 INFO:tasks.workunit.client.0.smithi083.stdout: 1 49801 0.64 MB/sec execute 297 sec latency 51.526 ms 2024-04-24T09:00:24.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:00:23 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:00:24.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:00:23 smithi083 ceph-mon[27907]: pgmap v274: 129 pgs: 129 active+clean; 53 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 588 KiB/s wr, 69 op/s 2024-04-24T09:00:24.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:00:23 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:00:24.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:00:23 smithi138 ceph-mon[31441]: pgmap v274: 129 pgs: 129 active+clean; 53 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 588 KiB/s wr, 69 op/s 2024-04-24T09:00:24.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:00:23 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:00:24.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:00:23 smithi189 ceph-mon[31791]: pgmap v274: 129 pgs: 129 active+clean; 53 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 588 KiB/s wr, 69 op/s 2024-04-24T09:00:24.599 INFO:tasks.workunit.client.0.smithi083.stdout: 1 49906 0.64 MB/sec execute 298 sec latency 90.232 ms 2024-04-24T09:00:25.461 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.461+0000 7fa059e00640 1 -- 172.21.15.83:0/2436136099 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa0541002f0 msgr2=0x7fa054100770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:25.461 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.461+0000 7fa059e00640 1 --2- 172.21.15.83:0/2436136099 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa0541002f0 0x7fa054100770 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7fa040009930 tx=0x7fa04002a420 comp rx=0 tx=0).stop 2024-04-24T09:00:25.461 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.461+0000 7fa059e00640 1 -- 172.21.15.83:0/2436136099 shutdown_connections 2024-04-24T09:00:25.462 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.461+0000 7fa059e00640 1 --2- 172.21.15.83:0/2436136099 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa054100cb0 0x7fa05410ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:25.462 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.461+0000 7fa059e00640 1 --2- 172.21.15.83:0/2436136099 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa0541002f0 0x7fa054100770 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:25.462 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.461+0000 7fa059e00640 1 --2- 172.21.15.83:0/2436136099 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa054106e30 0x7fa054107210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:25.462 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.461+0000 7fa059e00640 1 -- 172.21.15.83:0/2436136099 >> 172.21.15.83:0/2436136099 conn(0x7fa054077240 msgr2=0x7fa0540fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:25.462 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.461+0000 7fa059e00640 1 -- 172.21.15.83:0/2436136099 shutdown_connections 2024-04-24T09:00:25.462 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.461+0000 7fa059e00640 1 -- 172.21.15.83:0/2436136099 wait complete. 2024-04-24T09:00:25.462 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.462+0000 7fa059e00640 1 Processor -- start 2024-04-24T09:00:25.463 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.463+0000 7fa059e00640 1 -- start start 2024-04-24T09:00:25.464 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.463+0000 7fa059e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa0541002f0 0x7fa0541a05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:25.464 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.463+0000 7fa059e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa054100cb0 0x7fa0541a0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:25.464 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.463+0000 7fa059e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa054106e30 0x7fa05419a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:25.464 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.463+0000 7fa059e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa0541121e0 con 0x7fa054100cb0 2024-04-24T09:00:25.465 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.463+0000 7fa059e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa054112060 con 0x7fa054106e30 2024-04-24T09:00:25.465 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.463+0000 7fa059e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa054112360 con 0x7fa0541002f0 2024-04-24T09:00:25.465 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.463+0000 7fa053e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa054100cb0 0x7fa0541a0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:25.465 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.463+0000 7fa053e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa054100cb0 0x7fa0541a0b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35918/0 (socket says 172.21.15.83:35918) 2024-04-24T09:00:25.465 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.463+0000 7fa053e00640 1 -- 172.21.15.83:0/2636434236 learned_addr learned my addr 172.21.15.83:0/2636434236 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:00:25.465 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.464+0000 7fa058a00640 1 --2- 172.21.15.83:0/2636434236 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa0541002f0 0x7fa0541a05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:25.465 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.464+0000 7fa059400640 1 --2- 172.21.15.83:0/2636434236 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa054106e30 0x7fa05419a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:25.465 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.464+0000 7fa053e00640 1 -- 172.21.15.83:0/2636434236 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa0541002f0 msgr2=0x7fa0541a05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:25.465 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.464+0000 7fa053e00640 1 --2- 172.21.15.83:0/2636434236 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa0541002f0 0x7fa0541a05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:25.465 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.464+0000 7fa053e00640 1 -- 172.21.15.83:0/2636434236 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa054106e30 msgr2=0x7fa05419a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:25.465 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.464+0000 7fa053e00640 1 --2- 172.21.15.83:0/2636434236 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa054106e30 0x7fa05419a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:25.465 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.464+0000 7fa053e00640 1 -- 172.21.15.83:0/2636434236 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa05419af50 con 0x7fa054100cb0 2024-04-24T09:00:25.465 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.464+0000 7fa059400640 1 --2- 172.21.15.83:0/2636434236 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa054106e30 0x7fa05419a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:00:25.465 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.464+0000 7fa053e00640 1 --2- 172.21.15.83:0/2636434236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa054100cb0 0x7fa0541a0b10 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7fa040005ab0 tx=0x7fa040003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:25.466 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.465+0000 7fa051600640 1 -- 172.21.15.83:0/2636434236 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa04003f070 con 0x7fa054100cb0 2024-04-24T09:00:25.466 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.465+0000 7fa059e00640 1 -- 172.21.15.83:0/2636434236 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa05419b1e0 con 0x7fa054100cb0 2024-04-24T09:00:25.466 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.465+0000 7fa051600640 1 -- 172.21.15.83:0/2636434236 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa040031d60 con 0x7fa054100cb0 2024-04-24T09:00:25.466 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.465+0000 7fa051600640 1 -- 172.21.15.83:0/2636434236 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa040035030 con 0x7fa054100cb0 2024-04-24T09:00:25.466 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.465+0000 7fa059e00640 1 -- 172.21.15.83:0/2636434236 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0541a7370 con 0x7fa054100cb0 2024-04-24T09:00:25.469 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.468+0000 7fa05e17e740 1 Processor -- start 2024-04-24T09:00:25.469 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.468+0000 7fa051600640 1 -- 172.21.15.83:0/2636434236 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa040042070 con 0x7fa054100cb0 2024-04-24T09:00:25.469 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.468+0000 7fa05e17e740 1 -- start start 2024-04-24T09:00:25.469 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.469+0000 7fa05e17e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f3ef883c0 0x556f3ef887a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:25.469 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.469+0000 7fa05e17e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f3ef88ce0 0x556f3ef93060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:25.469 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.469+0000 7fa05e17e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f3ef935a0 0x556f3ef95990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:25.469 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.469+0000 7fa05e17e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556f3eeee900 con 0x556f3ef88ce0 2024-04-24T09:00:25.470 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.469+0000 7fa05e17e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556f3ee94f20 con 0x556f3ef883c0 2024-04-24T09:00:25.470 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.469+0000 7fa05e17e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x556f3eeea8c0 con 0x556f3ef935a0 2024-04-24T09:00:25.470 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.469+0000 7fa058a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f3ef88ce0 0x556f3ef93060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:25.470 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.469+0000 7fa058a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f3ef88ce0 0x556f3ef93060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35924/0 (socket says 172.21.15.83:35924) 2024-04-24T09:00:25.470 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.469+0000 7fa058a00640 1 -- 172.21.15.83:0/4078140187 learned_addr learned my addr 172.21.15.83:0/4078140187 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:00:25.470 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.469+0000 7fa053e00640 1 --2- 172.21.15.83:0/4078140187 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f3ef935a0 0x556f3ef95990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:25.470 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.469+0000 7fa059400640 1 --2- 172.21.15.83:0/4078140187 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f3ef883c0 0x556f3ef887a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:25.472 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.469+0000 7fa058a00640 1 -- 172.21.15.83:0/4078140187 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f3ef935a0 msgr2=0x556f3ef95990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:25.472 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.470+0000 7fa058a00640 1 --2- 172.21.15.83:0/4078140187 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f3ef935a0 0x556f3ef95990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:25.472 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.470+0000 7fa058a00640 1 -- 172.21.15.83:0/4078140187 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f3ef883c0 msgr2=0x556f3ef887a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:25.472 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.470+0000 7fa058a00640 1 --2- 172.21.15.83:0/4078140187 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f3ef883c0 0x556f3ef887a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:25.472 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.470+0000 7fa058a00640 1 -- 172.21.15.83:0/4078140187 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f3eeb9e10 con 0x556f3ef88ce0 2024-04-24T09:00:25.472 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.470+0000 7fa058a00640 1 --2- 172.21.15.83:0/4078140187 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f3ef88ce0 0x556f3ef93060 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7fa04800bcf0 tx=0x7fa04800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49bb5f59b1f5ae99 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:25.472 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.470+0000 7fa045c00640 1 -- 172.21.15.83:0/4078140187 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa048015070 con 0x556f3ef88ce0 2024-04-24T09:00:25.472 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.470+0000 7fa045c00640 1 -- 172.21.15.83:0/4078140187 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa04801a910 con 0x556f3ef88ce0 2024-04-24T09:00:25.472 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.470+0000 7fa045c00640 1 -- 172.21.15.83:0/4078140187 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa048011930 con 0x556f3ef88ce0 2024-04-24T09:00:25.472 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.471+0000 7fa051600640 1 --2- 172.21.15.83:0/2636434236 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa03c0a3d10 0x7fa03c0a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:25.472 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.471+0000 7fa05e17e740 1 -- 172.21.15.83:0/4078140187 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f3ef88ce0 msgr2=0x556f3ef93060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:25.472 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.471+0000 7fa05e17e740 1 --2- 172.21.15.83:0/4078140187 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f3ef88ce0 0x556f3ef93060 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7fa04800bcf0 tx=0x7fa04800f730 comp rx=0 tx=0).stop 2024-04-24T09:00:25.473 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.471+0000 7fa051600640 1 -- 172.21.15.83:0/2636434236 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7fa0400f14f0 con 0x7fa054100cb0 2024-04-24T09:00:25.473 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.471+0000 7fa059400640 1 --2- 172.21.15.83:0/2636434236 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa03c0a3d10 0x7fa03c0a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:25.473 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.471+0000 7fa05e17e740 1 -- 172.21.15.83:0/4078140187 shutdown_connections 2024-04-24T09:00:25.473 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.471+0000 7fa05e17e740 1 --2- 172.21.15.83:0/4078140187 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f3ef935a0 0x556f3ef95990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:25.473 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.471+0000 7fa05e17e740 1 --2- 172.21.15.83:0/4078140187 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f3ef88ce0 0x556f3ef93060 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:25.473 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.471+0000 7fa05e17e740 1 --2- 172.21.15.83:0/4078140187 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f3ef883c0 0x556f3ef887a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:25.473 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.471+0000 7fa05e17e740 1 -- 172.21.15.83:0/4078140187 >> 172.21.15.83:0/4078140187 conn(0x556f3ef848d0 msgr2=0x556f3ef84cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:25.473 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.472+0000 7fa05e17e740 1 -- 172.21.15.83:0/4078140187 shutdown_connections 2024-04-24T09:00:25.473 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.472+0000 7fa05e17e740 1 -- 172.21.15.83:0/4078140187 wait complete. 2024-04-24T09:00:25.473 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.472+0000 7fa059400640 1 --2- 172.21.15.83:0/2636434236 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa03c0a3d10 0x7fa03c0a61d0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa04c005fd0 tx=0x7fa04c008470 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:25.476 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.472+0000 7fa05e17e740 1 -- start start 2024-04-24T09:00:25.476 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.473+0000 7fa05e17e740 1 Processor -- start 2024-04-24T09:00:25.476 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.473+0000 7fa05e17e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f3ef883c0 0x556f3ef905a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:25.476 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.473+0000 7fa05e17e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f3ef88ce0 0x556f3ef90ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:25.476 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.473+0000 7fa05e17e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f3ef935a0 0x556f3ef91020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:25.476 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.473+0000 7fa05e17e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556f3ef96140 con 0x556f3ef883c0 2024-04-24T09:00:25.476 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.473+0000 7fa05e17e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556f3eea82c0 con 0x556f3ef935a0 2024-04-24T09:00:25.476 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.473+0000 7fa05e17e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x556f3ef962c0 con 0x556f3ef88ce0 2024-04-24T09:00:25.476 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.473+0000 7fa05e17e740 10 client.0 _ll_register_callbacks cb 0x556f3ec00a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:00:25.476 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.473+0000 7fa058a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f3ef883c0 0x556f3ef905a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:25.476 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.473+0000 7fa058a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f3ef883c0 0x556f3ef905a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35934/0 (socket says 172.21.15.83:35934) 2024-04-24T09:00:25.477 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.473+0000 7fa058a00640 1 -- 172.21.15.83:0/2311697931 learned_addr learned my addr 172.21.15.83:0/2311697931 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:00:25.477 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.474+0000 7fa053e00640 1 --2- 172.21.15.83:0/2311697931 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f3ef88ce0 0x556f3ef90ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:25.477 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.474+0000 7fa059400640 1 --2- 172.21.15.83:0/2311697931 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f3ef935a0 0x556f3ef91020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:25.477 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.474+0000 7fa058a00640 1 -- 172.21.15.83:0/2311697931 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f3ef88ce0 msgr2=0x556f3ef90ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:25.477 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.474+0000 7fa058a00640 1 --2- 172.21.15.83:0/2311697931 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f3ef88ce0 0x556f3ef90ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:25.477 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.474+0000 7fa058a00640 1 -- 172.21.15.83:0/2311697931 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f3ef935a0 msgr2=0x556f3ef91020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:25.477 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.474+0000 7fa058a00640 1 --2- 172.21.15.83:0/2311697931 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f3ef935a0 0x556f3ef91020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:25.477 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.474+0000 7fa058a00640 1 -- 172.21.15.83:0/2311697931 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f3eeb8fd0 con 0x556f3ef883c0 2024-04-24T09:00:25.477 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.474+0000 7fa058a00640 1 --2- 172.21.15.83:0/2311697931 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f3ef883c0 0x556f3ef905a0 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7fa04800bcc0 tx=0x7fa04800ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77e05a2374fbfbf8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:25.477 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.474+0000 7fa03a000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:00:25.477 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.475+0000 7fa03a000640 1 -- 172.21.15.83:0/2311697931 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa048015050 con 0x556f3ef883c0 2024-04-24T09:00:25.477 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.475+0000 7fa05e17e740 1 -- 172.21.15.83:0/2311697931 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa04801a910 con 0x556f3ef883c0 2024-04-24T09:00:25.477 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.475+0000 7fa03a000640 1 -- 172.21.15.83:0/2311697931 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7fa04801aac0 con 0x556f3ef883c0 2024-04-24T09:00:25.477 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.475+0000 7fa03a000640 1 -- 172.21.15.83:0/2311697931 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fa048031050 con 0x556f3ef883c0 2024-04-24T09:00:25.477 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.475+0000 7fa05e17e740 10 client.18792 fetch_fsmap learned FSMap version 35 2024-04-24T09:00:25.478 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.475+0000 7fa05e17e740 1 -- 172.21.15.83:0/2311697931 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x556f3ec01210 con 0x556f3ef883c0 2024-04-24T09:00:25.478 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.475+0000 7fa03a000640 1 -- 172.21.15.83:0/2311697931 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa04801a910 con 0x556f3ef883c0 2024-04-24T09:00:25.478 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.476+0000 7fa03a000640 1 -- 172.21.15.83:0/2311697931 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7fa04802c030 con 0x556f3ef883c0 2024-04-24T09:00:25.478 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.476+0000 7fa05e17e740 10 client.18792 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T09:00:25.478 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.476+0000 7fa05e17e740 10 client.18792 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:00:25.478 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.476+0000 7fa05e17e740 20 client.18792 populate_metadata read hostname 'smithi083' 2024-04-24T09:00:25.478 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.476+0000 7fa05e17e740 1 --2- 172.21.15.83:0/2311697931 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556f3efa3680 0x556f3efa5a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:25.478 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.476+0000 7fa05e17e740 4 client.18792 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:00:25.478 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.476+0000 7fa05e17e740 1 -- 172.21.15.83:0/2311697931 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556f3eefc320 con 0x556f3efa3680 2024-04-24T09:00:25.479 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.476+0000 7fa053e00640 1 --2- 172.21.15.83:0/2311697931 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556f3efa3680 0x556f3efa5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:25.479 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.477+0000 7fa053e00640 1 --2- 172.21.15.83:0/2311697931 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556f3efa3680 0x556f3efa5a70 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc7238b06c0a899a server_cookie=7e07af3edce7aebb in_seq=0 out_seq=0 2024-04-24T09:00:25.479 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.477+0000 7fa03a000640 10 client.18792 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:00:25.480 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.480+0000 7fa03a000640 1 -- 172.21.15.83:0/2311697931 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556f3eefc320 con 0x556f3efa3680 2024-04-24T09:00:25.480 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.480+0000 7fa03a000640 10 client.18792 handle_command_reply: tid=0 2024-04-24T09:00:25.481 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.480+0000 7fa05e17e740 1 client.18792 shutdown 2024-04-24T09:00:25.481 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.480+0000 7fa05e17e740 1 -- 172.21.15.83:0/2311697931 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f3ef883c0 msgr2=0x556f3ef905a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:25.481 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.480+0000 7fa05e17e740 1 --2- 172.21.15.83:0/2311697931 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f3ef883c0 0x556f3ef905a0 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7fa04800bcc0 tx=0x7fa04800ba30 comp rx=0 tx=0).stop 2024-04-24T09:00:25.481 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.480+0000 7fa05e17e740 1 -- 172.21.15.83:0/2311697931 shutdown_connections 2024-04-24T09:00:25.481 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.480+0000 7fa05e17e740 1 --2- 172.21.15.83:0/2311697931 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556f3efa3680 0x556f3efa5a70 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:25.481 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.480+0000 7fa05e17e740 1 --2- 172.21.15.83:0/2311697931 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f3ef935a0 0x556f3ef91020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:25.481 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.480+0000 7fa05e17e740 1 --2- 172.21.15.83:0/2311697931 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f3ef88ce0 0x556f3ef90ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:25.481 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.480+0000 7fa05e17e740 1 --2- 172.21.15.83:0/2311697931 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f3ef883c0 0x556f3ef905a0 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:25.481 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.480+0000 7fa05e17e740 1 -- 172.21.15.83:0/2311697931 >> 172.21.15.83:0/2311697931 conn(0x556f3ef848d0 msgr2=0x556f3ef8a650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:25.481 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.480+0000 7fa03a000640 0 client.18792 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:00:25.481 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.480+0000 7fa05e17e740 1 -- 172.21.15.83:0/2311697931 shutdown_connections 2024-04-24T09:00:25.481 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.480+0000 7fa05e17e740 1 -- 172.21.15.83:0/2311697931 wait complete. 2024-04-24T09:00:25.481 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.480+0000 7fa05e17e740 20 client.18792 trim_cache size 0 max 16384 2024-04-24T09:00:25.499 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.499+0000 7fa05e17e740 1 Processor -- start 2024-04-24T09:00:25.500 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.499+0000 7fa05e17e740 1 -- start start 2024-04-24T09:00:25.500 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.499+0000 7fa05e17e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f3ef883c0 0x556f3ef92cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:25.500 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.499+0000 7fa05e17e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f3ef88ce0 0x556f3ef90560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:25.500 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.499+0000 7fa05e17e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f3ef935a0 0x556f3ef8fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:25.500 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.499+0000 7fa05e17e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa0541b15a0 con 0x556f3ef88ce0 2024-04-24T09:00:25.500 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.499+0000 7fa05e17e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa0541b30e0 con 0x556f3ef935a0 2024-04-24T09:00:25.500 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.499+0000 7fa05e17e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa0541b1c70 con 0x556f3ef883c0 2024-04-24T09:00:25.500 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.500+0000 7fa053e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f3ef88ce0 0x556f3ef90560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:25.500 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.500+0000 7fa053e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f3ef88ce0 0x556f3ef90560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35940/0 (socket says 172.21.15.83:35940) 2024-04-24T09:00:25.500 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.500+0000 7fa053e00640 1 -- 172.21.15.83:0/1463745120 learned_addr learned my addr 172.21.15.83:0/1463745120 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:00:25.500 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.500+0000 7fa058a00640 1 --2- 172.21.15.83:0/1463745120 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f3ef883c0 0x556f3ef92cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:25.500 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.500+0000 7fa059400640 1 --2- 172.21.15.83:0/1463745120 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f3ef935a0 0x556f3ef8fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:25.502 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.500+0000 7fa053e00640 1 -- 172.21.15.83:0/1463745120 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f3ef883c0 msgr2=0x556f3ef92cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:25.502 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.500+0000 7fa053e00640 1 --2- 172.21.15.83:0/1463745120 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f3ef883c0 0x556f3ef92cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:25.502 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.500+0000 7fa053e00640 1 -- 172.21.15.83:0/1463745120 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f3ef935a0 msgr2=0x556f3ef8fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:25.502 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.500+0000 7fa053e00640 1 --2- 172.21.15.83:0/1463745120 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f3ef935a0 0x556f3ef8fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:25.502 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.500+0000 7fa053e00640 1 -- 172.21.15.83:0/1463745120 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f3eccd620 con 0x556f3ef88ce0 2024-04-24T09:00:25.502 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.500+0000 7fa053e00640 1 --2- 172.21.15.83:0/1463745120 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f3ef88ce0 0x556f3ef90560 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7fa04002a9e0 tx=0x7fa04002a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7111cb6c4a3f0f17 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:25.502 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.501+0000 7fa045200640 1 -- 172.21.15.83:0/1463745120 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa040049040 con 0x556f3ef88ce0 2024-04-24T09:00:25.502 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.501+0000 7fa045200640 1 -- 172.21.15.83:0/1463745120 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa0400bd710 con 0x556f3ef88ce0 2024-04-24T09:00:25.502 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.501+0000 7fa045200640 1 -- 172.21.15.83:0/1463745120 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0400bb310 con 0x556f3ef88ce0 2024-04-24T09:00:25.502 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.501+0000 7fa05e17e740 1 -- 172.21.15.83:0/1463745120 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f3ef88ce0 msgr2=0x556f3ef90560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:25.503 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.501+0000 7fa05e17e740 1 --2- 172.21.15.83:0/1463745120 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f3ef88ce0 0x556f3ef90560 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7fa04002a9e0 tx=0x7fa04002a420 comp rx=0 tx=0).stop 2024-04-24T09:00:25.503 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.502+0000 7fa05e17e740 1 -- 172.21.15.83:0/1463745120 shutdown_connections 2024-04-24T09:00:25.503 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.502+0000 7fa05e17e740 1 --2- 172.21.15.83:0/1463745120 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f3ef935a0 0x556f3ef8fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:25.503 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.502+0000 7fa05e17e740 1 --2- 172.21.15.83:0/1463745120 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f3ef88ce0 0x556f3ef90560 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:25.503 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.502+0000 7fa05e17e740 1 --2- 172.21.15.83:0/1463745120 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f3ef883c0 0x556f3ef92cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:25.503 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.502+0000 7fa05e17e740 1 -- 172.21.15.83:0/1463745120 >> 172.21.15.83:0/1463745120 conn(0x556f3ef848d0 msgr2=0x556f3ef83a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:25.503 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.502+0000 7fa05e17e740 1 -- 172.21.15.83:0/1463745120 shutdown_connections 2024-04-24T09:00:25.503 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.502+0000 7fa05e17e740 1 -- 172.21.15.83:0/1463745120 wait complete. 2024-04-24T09:00:25.503 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.502+0000 7fa05e17e740 1 -- start start 2024-04-24T09:00:25.503 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.502+0000 7fa05e17e740 1 Processor -- start 2024-04-24T09:00:25.505 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.503+0000 7fa05e17e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f3ef883c0 0x556f3ef9b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:25.506 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.503+0000 7fa05e17e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f3ef88ce0 0x556f3ef8ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:25.506 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.503+0000 7fa05e17e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f3ef935a0 0x556f3ef8d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:25.506 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.503+0000 7fa05e17e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa0541b2a10 con 0x556f3ef935a0 2024-04-24T09:00:25.506 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.503+0000 7fa05e17e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa0541b2340 con 0x556f3ef883c0 2024-04-24T09:00:25.506 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.503+0000 7fa05e17e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa0541af5a0 con 0x556f3ef88ce0 2024-04-24T09:00:25.506 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.503+0000 7fa05e17e740 10 client.0 _ll_register_callbacks cb 0x7fa04800bb40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:00:25.506 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.503+0000 7fa059400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f3ef935a0 0x556f3ef8d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:25.506 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.503+0000 7fa059400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f3ef935a0 0x556f3ef8d390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35956/0 (socket says 172.21.15.83:35956) 2024-04-24T09:00:25.506 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.503+0000 7fa059400640 1 -- 172.21.15.83:0/3168345077 learned_addr learned my addr 172.21.15.83:0/3168345077 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:00:25.506 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.503+0000 7fa058a00640 1 --2- 172.21.15.83:0/3168345077 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f3ef883c0 0x556f3ef9b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:25.506 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.503+0000 7fa053e00640 1 --2- 172.21.15.83:0/3168345077 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f3ef88ce0 0x556f3ef8ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:25.506 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.503+0000 7fa059400640 1 -- 172.21.15.83:0/3168345077 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f3ef88ce0 msgr2=0x556f3ef8ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:25.506 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.503+0000 7fa059400640 1 --2- 172.21.15.83:0/3168345077 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f3ef88ce0 0x556f3ef8ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:25.507 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.503+0000 7fa059400640 1 -- 172.21.15.83:0/3168345077 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f3ef883c0 msgr2=0x556f3ef9b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:25.507 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.503+0000 7fa059400640 1 --2- 172.21.15.83:0/3168345077 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f3ef883c0 0x556f3ef9b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:25.507 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.503+0000 7fa059400640 1 -- 172.21.15.83:0/3168345077 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f3ef8dad0 con 0x556f3ef935a0 2024-04-24T09:00:25.507 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.504+0000 7fa059400640 1 --2- 172.21.15.83:0/3168345077 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f3ef935a0 0x556f3ef8d390 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7fa04c01fe80 tx=0x7fa04c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a500cdd79b937d3c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:25.507 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.504+0000 7fa03a000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:00:25.507 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.504+0000 7fa03a000640 1 -- 172.21.15.83:0/3168345077 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa04c01f220 con 0x556f3ef935a0 2024-04-24T09:00:25.507 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.504+0000 7fa05e17e740 1 -- 172.21.15.83:0/3168345077 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0400bd710 con 0x556f3ef935a0 2024-04-24T09:00:25.507 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.504+0000 7fa03a000640 1 -- 172.21.15.83:0/3168345077 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7fa04c004b20 con 0x556f3ef935a0 2024-04-24T09:00:25.507 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.504+0000 7fa03a000640 1 -- 172.21.15.83:0/3168345077 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fa04c010e40 con 0x556f3ef935a0 2024-04-24T09:00:25.507 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.504+0000 7fa05e17e740 10 client.18798 fetch_fsmap learned FSMap version 35 2024-04-24T09:00:25.507 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.504+0000 7fa05e17e740 1 -- 172.21.15.83:0/3168345077 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x556f3f009900 con 0x556f3ef935a0 2024-04-24T09:00:25.507 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.505+0000 7fa03a000640 1 -- 172.21.15.83:0/3168345077 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa0400bd710 con 0x556f3ef935a0 2024-04-24T09:00:25.507 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.505+0000 7fa03a000640 1 -- 172.21.15.83:0/3168345077 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7fa04c026070 con 0x556f3ef935a0 2024-04-24T09:00:25.507 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.505+0000 7fa05e17e740 10 client.18798 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T09:00:25.507 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.505+0000 7fa05e17e740 10 client.18798 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:00:25.508 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.505+0000 7fa05e17e740 20 client.18798 populate_metadata read hostname 'smithi083' 2024-04-24T09:00:25.508 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.505+0000 7fa05e17e740 1 --2- 172.21.15.83:0/3168345077 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556f3f00a080 0x556f3f00c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:25.508 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.505+0000 7fa05e17e740 4 client.18798 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T09:00:25.508 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.505+0000 7fa05e17e740 1 -- 172.21.15.83:0/3168345077 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556f3eef6190 con 0x556f3f00a080 2024-04-24T09:00:25.508 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.506+0000 7fa058a00640 1 --2- 172.21.15.83:0/3168345077 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556f3f00a080 0x556f3f00c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:25.508 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.507+0000 7fa058a00640 1 --2- 172.21.15.83:0/3168345077 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556f3f00a080 0x556f3f00c4a0 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=574adeef8bba5fde server_cookie=3550ad2231eb6d76 in_seq=0 out_seq=0 2024-04-24T09:00:25.508 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.507+0000 7fa03a000640 10 client.18798 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:00:25.567 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.567+0000 7fa03a000640 1 -- 172.21.15.83:0/3168345077 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556f3eef6190 con 0x556f3f00a080 2024-04-24T09:00:25.568 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.567+0000 7fa03a000640 10 client.18798 handle_command_reply: tid=0 2024-04-24T09:00:25.568 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.567+0000 7fa05e17e740 1 client.18798 shutdown 2024-04-24T09:00:25.568 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.568+0000 7fa05e17e740 1 -- 172.21.15.83:0/3168345077 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f3ef935a0 msgr2=0x556f3ef8d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:25.568 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.568+0000 7fa05e17e740 1 --2- 172.21.15.83:0/3168345077 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f3ef935a0 0x556f3ef8d390 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7fa04c01fe80 tx=0x7fa04c004a40 comp rx=0 tx=0).stop 2024-04-24T09:00:25.568 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.568+0000 7fa05e17e740 1 -- 172.21.15.83:0/3168345077 shutdown_connections 2024-04-24T09:00:25.568 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.568+0000 7fa05e17e740 1 --2- 172.21.15.83:0/3168345077 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556f3f00a080 0x556f3f00c4a0 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:25.568 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.568+0000 7fa05e17e740 1 --2- 172.21.15.83:0/3168345077 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f3ef935a0 0x556f3ef8d390 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:25.568 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.568+0000 7fa05e17e740 1 --2- 172.21.15.83:0/3168345077 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f3ef88ce0 0x556f3ef8ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:25.568 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.568+0000 7fa05e17e740 1 --2- 172.21.15.83:0/3168345077 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f3ef883c0 0x556f3ef9b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:25.568 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.568+0000 7fa03a000640 0 client.18798 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:00:25.568 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.568+0000 7fa05e17e740 1 -- 172.21.15.83:0/3168345077 >> 172.21.15.83:0/3168345077 conn(0x556f3ef848d0 msgr2=0x556f3ef9e0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:25.569 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.568+0000 7fa05e17e740 1 -- 172.21.15.83:0/3168345077 shutdown_connections 2024-04-24T09:00:25.569 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.568+0000 7fa05e17e740 1 -- 172.21.15.83:0/3168345077 wait complete. 2024-04-24T09:00:25.569 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.568+0000 7fa05e17e740 20 client.18798 trim_cache size 0 max 16384 2024-04-24T09:00:25.569 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T09:00:25.570 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T09:00:25.570 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T09:00:25.570 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T09:00:25.570 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.569+0000 7fa059e00640 1 -- 172.21.15.83:0/2636434236 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa03c0a3d10 msgr2=0x7fa03c0a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:25.571 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.569+0000 7fa059e00640 1 --2- 172.21.15.83:0/2636434236 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa03c0a3d10 0x7fa03c0a61d0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa04c005fd0 tx=0x7fa04c008470 comp rx=0 tx=0).stop 2024-04-24T09:00:25.571 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.569+0000 7fa059e00640 1 -- 172.21.15.83:0/2636434236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa054100cb0 msgr2=0x7fa0541a0b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:25.571 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.569+0000 7fa059e00640 1 --2- 172.21.15.83:0/2636434236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa054100cb0 0x7fa0541a0b10 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7fa040005ab0 tx=0x7fa040003cd0 comp rx=0 tx=0).stop 2024-04-24T09:00:25.571 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.570+0000 7fa059e00640 1 -- 172.21.15.83:0/2636434236 shutdown_connections 2024-04-24T09:00:25.571 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.570+0000 7fa059e00640 1 --2- 172.21.15.83:0/2636434236 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa03c0a3d10 0x7fa03c0a61d0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:25.571 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.570+0000 7fa059e00640 1 --2- 172.21.15.83:0/2636434236 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa054106e30 0x7fa05419a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:25.571 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.570+0000 7fa059e00640 1 --2- 172.21.15.83:0/2636434236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa054100cb0 0x7fa0541a0b10 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:25.571 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.570+0000 7fa059e00640 1 --2- 172.21.15.83:0/2636434236 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa0541002f0 0x7fa0541a05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:25.571 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.570+0000 7fa059e00640 1 -- 172.21.15.83:0/2636434236 >> 172.21.15.83:0/2636434236 conn(0x7fa054077240 msgr2=0x7fa0541050c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:25.571 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.570+0000 7fa059e00640 1 -- 172.21.15.83:0/2636434236 shutdown_connections 2024-04-24T09:00:25.571 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:25.570+0000 7fa059e00640 1 -- 172.21.15.83:0/2636434236 wait complete. 2024-04-24T09:00:25.599 INFO:tasks.workunit.client.0.smithi083.stdout: 1 49950 0.64 MB/sec execute 299 sec latency 279.463 ms 2024-04-24T09:00:26.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:00:25 smithi083 ceph-mon[27907]: pgmap v275: 129 pgs: 129 active+clean; 53 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 498 KiB/s wr, 46 op/s 2024-04-24T09:00:26.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:00:25 smithi138 ceph-mon[31441]: pgmap v275: 129 pgs: 129 active+clean; 53 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 498 KiB/s wr, 46 op/s 2024-04-24T09:00:26.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:00:25 smithi189 ceph-mon[31791]: pgmap v275: 129 pgs: 129 active+clean; 53 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 498 KiB/s wr, 46 op/s 2024-04-24T09:00:26.600 INFO:tasks.workunit.client.0.smithi083.stdout: 1 50007 0.64 MB/sec execute 300 sec latency 47.378 ms 2024-04-24T09:00:27.600 INFO:tasks.workunit.client.0.smithi083.stdout: 1 50075 0.64 MB/sec execute 301 sec latency 54.119 ms 2024-04-24T09:00:28.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:00:27 smithi083 ceph-mon[27907]: pgmap v276: 129 pgs: 129 active+clean; 53 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 498 KiB/s wr, 46 op/s 2024-04-24T09:00:28.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:00:27 smithi138 ceph-mon[31441]: pgmap v276: 129 pgs: 129 active+clean; 53 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 498 KiB/s wr, 46 op/s 2024-04-24T09:00:28.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:00:27 smithi189 ceph-mon[31791]: pgmap v276: 129 pgs: 129 active+clean; 53 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 498 KiB/s wr, 46 op/s 2024-04-24T09:00:28.600 INFO:tasks.workunit.client.0.smithi083.stdout: 1 50133 0.63 MB/sec execute 302 sec latency 43.863 ms 2024-04-24T09:00:29.600 INFO:tasks.workunit.client.0.smithi083.stdout: 1 50195 0.63 MB/sec execute 303 sec latency 56.366 ms 2024-04-24T09:00:30.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:00:29 smithi083 ceph-mon[27907]: pgmap v277: 129 pgs: 129 active+clean; 51 MiB data, 720 MiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 707 KiB/s wr, 72 op/s 2024-04-24T09:00:30.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:00:29 smithi138 ceph-mon[31441]: pgmap v277: 129 pgs: 129 active+clean; 51 MiB data, 720 MiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 707 KiB/s wr, 72 op/s 2024-04-24T09:00:30.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:00:29 smithi189 ceph-mon[31791]: pgmap v277: 129 pgs: 129 active+clean; 51 MiB data, 720 MiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 707 KiB/s wr, 72 op/s 2024-04-24T09:00:30.600 INFO:tasks.workunit.client.0.smithi083.stdout: 1 50272 0.63 MB/sec execute 304 sec latency 48.156 ms 2024-04-24T09:00:31.600 INFO:tasks.workunit.client.0.smithi083.stdout: 1 50355 0.63 MB/sec execute 305 sec latency 55.719 ms 2024-04-24T09:00:32.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:00:31 smithi083 ceph-mon[27907]: pgmap v278: 129 pgs: 129 active+clean; 51 MiB data, 720 MiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 371 KiB/s wr, 43 op/s 2024-04-24T09:00:32.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:00:31 smithi138 ceph-mon[31441]: pgmap v278: 129 pgs: 129 active+clean; 51 MiB data, 720 MiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 371 KiB/s wr, 43 op/s 2024-04-24T09:00:32.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:00:31 smithi189 ceph-mon[31791]: pgmap v278: 129 pgs: 129 active+clean; 51 MiB data, 720 MiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 371 KiB/s wr, 43 op/s 2024-04-24T09:00:32.600 INFO:tasks.workunit.client.0.smithi083.stdout: 1 50486 0.63 MB/sec execute 306 sec latency 51.183 ms 2024-04-24T09:00:33.601 INFO:tasks.workunit.client.0.smithi083.stdout: 1 50602 0.63 MB/sec execute 307 sec latency 56.112 ms 2024-04-24T09:00:34.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:00:33 smithi083 ceph-mon[27907]: pgmap v279: 129 pgs: 129 active+clean; 50 MiB data, 726 MiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 483 KiB/s wr, 69 op/s 2024-04-24T09:00:34.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:00:33 smithi138 ceph-mon[31441]: pgmap v279: 129 pgs: 129 active+clean; 50 MiB data, 726 MiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 483 KiB/s wr, 69 op/s 2024-04-24T09:00:34.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:00:33 smithi189 ceph-mon[31791]: pgmap v279: 129 pgs: 129 active+clean; 50 MiB data, 726 MiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 483 KiB/s wr, 69 op/s 2024-04-24T09:00:34.601 INFO:tasks.workunit.client.0.smithi083.stdout: 1 50725 0.63 MB/sec execute 308 sec latency 55.266 ms 2024-04-24T09:00:35.601 INFO:tasks.workunit.client.0.smithi083.stdout: 1 50829 0.63 MB/sec execute 309 sec latency 55.124 ms 2024-04-24T09:00:36.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:00:35 smithi138 ceph-mon[31441]: pgmap v280: 129 pgs: 129 active+clean; 50 MiB data, 726 MiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 320 KiB/s wr, 52 op/s 2024-04-24T09:00:36.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:00:35 smithi189 ceph-mon[31791]: pgmap v280: 129 pgs: 129 active+clean; 50 MiB data, 726 MiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 320 KiB/s wr, 52 op/s 2024-04-24T09:00:36.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:00:35 smithi083 ceph-mon[27907]: pgmap v280: 129 pgs: 129 active+clean; 50 MiB data, 726 MiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 320 KiB/s wr, 52 op/s 2024-04-24T09:00:36.601 INFO:tasks.workunit.client.0.smithi083.stdout: 1 50928 0.62 MB/sec execute 310 sec latency 51.157 ms 2024-04-24T09:00:37.601 INFO:tasks.workunit.client.0.smithi083.stdout: 1 51093 0.63 MB/sec execute 311 sec latency 58.375 ms 2024-04-24T09:00:38.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:00:37 smithi138 ceph-mon[31441]: pgmap v281: 129 pgs: 129 active+clean; 50 MiB data, 726 MiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 320 KiB/s wr, 52 op/s 2024-04-24T09:00:38.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:00:37 smithi189 ceph-mon[31791]: pgmap v281: 129 pgs: 129 active+clean; 50 MiB data, 726 MiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 320 KiB/s wr, 52 op/s 2024-04-24T09:00:38.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:00:37 smithi083 ceph-mon[27907]: pgmap v281: 129 pgs: 129 active+clean; 50 MiB data, 726 MiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 320 KiB/s wr, 52 op/s 2024-04-24T09:00:38.601 INFO:tasks.workunit.client.0.smithi083.stdout: 1 51185 0.63 MB/sec execute 312 sec latency 66.348 ms 2024-04-24T09:00:38.723 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.723+0000 7f4a80200640 1 -- 172.21.15.83:0/1551369263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a78103290 msgr2=0x7f4a78103670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:38.723 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.723+0000 7f4a80200640 1 --2- 172.21.15.83:0/1551369263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a78103290 0x7f4a78103670 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c009900 tx=0x7f4a6c02a510 comp rx=0 tx=0).stop 2024-04-24T09:00:38.723 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.723+0000 7f4a80200640 1 -- 172.21.15.83:0/1551369263 shutdown_connections 2024-04-24T09:00:38.723 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.723+0000 7f4a80200640 1 --2- 172.21.15.83:0/1551369263 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a7810c5d0 0x7f4a7810ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:38.723 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.723+0000 7f4a80200640 1 --2- 172.21.15.83:0/1551369263 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4a78103bb0 0x7f4a7810c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:38.723 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.723+0000 7f4a80200640 1 --2- 172.21.15.83:0/1551369263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a78103290 0x7f4a78103670 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:38.723 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.723+0000 7f4a80200640 1 -- 172.21.15.83:0/1551369263 >> 172.21.15.83:0/1551369263 conn(0x7f4a78078080 msgr2=0x7f4a78078490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:38.723 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.723+0000 7f4a80200640 1 -- 172.21.15.83:0/1551369263 shutdown_connections 2024-04-24T09:00:38.724 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.723+0000 7f4a80200640 1 -- 172.21.15.83:0/1551369263 wait complete. 2024-04-24T09:00:38.725 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.724+0000 7f4a80200640 1 Processor -- start 2024-04-24T09:00:38.725 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.724+0000 7f4a80200640 1 -- start start 2024-04-24T09:00:38.725 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.724+0000 7f4a80200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a78103290 0x7f4a78100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:38.725 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.724+0000 7f4a80200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a78103bb0 0x7f4a78101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:38.725 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.724+0000 7f4a80200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4a7810c5d0 0x7f4a78101960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:38.725 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.724+0000 7f4a80200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4a781122b0 con 0x7f4a78103bb0 2024-04-24T09:00:38.725 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.724+0000 7f4a80200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4a78112130 con 0x7f4a78103290 2024-04-24T09:00:38.725 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.724+0000 7f4a80200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4a78112430 con 0x7f4a7810c5d0 2024-04-24T09:00:38.725 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.725+0000 7f4a7ce00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a78103bb0 0x7f4a78101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:38.725 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.725+0000 7f4a7ce00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a78103bb0 0x7f4a78101420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56930/0 (socket says 172.21.15.83:56930) 2024-04-24T09:00:38.725 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.725+0000 7f4a7ce00640 1 -- 172.21.15.83:0/1839021848 learned_addr learned my addr 172.21.15.83:0/1839021848 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:00:38.727 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.725+0000 7f4a7d800640 1 --2- 172.21.15.83:0/1839021848 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a78103290 0x7f4a78100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:38.727 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.725+0000 7f4a7ce00640 1 -- 172.21.15.83:0/1839021848 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4a7810c5d0 msgr2=0x7f4a78101960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:38.727 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.725+0000 7f4a7e200640 1 --2- 172.21.15.83:0/1839021848 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4a7810c5d0 0x7f4a78101960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:38.727 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.725+0000 7f4a7ce00640 1 --2- 172.21.15.83:0/1839021848 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4a7810c5d0 0x7f4a78101960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:38.727 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.725+0000 7f4a7ce00640 1 -- 172.21.15.83:0/1839021848 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a78103290 msgr2=0x7f4a78100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:38.727 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.725+0000 7f4a7ce00640 1 --2- 172.21.15.83:0/1839021848 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a78103290 0x7f4a78100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:38.727 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.725+0000 7f4a7ce00640 1 -- 172.21.15.83:0/1839021848 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a781a6c50 con 0x7f4a78103bb0 2024-04-24T09:00:38.727 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.725+0000 7f4a7d800640 1 --2- 172.21.15.83:0/1839021848 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a78103290 0x7f4a78100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:00:38.727 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.725+0000 7f4a7e200640 1 --2- 172.21.15.83:0/1839021848 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4a7810c5d0 0x7f4a78101960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:00:38.727 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.726+0000 7f4a7ce00640 1 --2- 172.21.15.83:0/1839021848 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a78103bb0 0x7f4a78101420 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f4a6400c970 tx=0x7f4a6400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:38.727 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.726+0000 7f4a76000640 1 -- 172.21.15.83:0/1839021848 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a64007bb0 con 0x7f4a78103bb0 2024-04-24T09:00:38.727 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.726+0000 7f4a76000640 1 -- 172.21.15.83:0/1839021848 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4a64007d50 con 0x7f4a78103bb0 2024-04-24T09:00:38.727 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.726+0000 7f4a80200640 1 -- 172.21.15.83:0/1839021848 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a781a6f40 con 0x7f4a78103bb0 2024-04-24T09:00:38.728 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.726+0000 7f4a76000640 1 -- 172.21.15.83:0/1839021848 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a6400f450 con 0x7f4a78103bb0 2024-04-24T09:00:38.728 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.726+0000 7f4a80200640 1 -- 172.21.15.83:0/1839021848 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a781a7400 con 0x7f4a78103bb0 2024-04-24T09:00:38.729 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.729+0000 7f4a831d8740 1 Processor -- start 2024-04-24T09:00:38.729 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.730+0000 7f4a831d8740 1 -- start start 2024-04-24T09:00:38.732 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.730+0000 7f4a76000640 1 -- 172.21.15.83:0/1839021848 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4a64004ee0 con 0x7f4a78103bb0 2024-04-24T09:00:38.732 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.730+0000 7f4a831d8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556eaeb171b0 0x556eaeb17590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:38.732 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.730+0000 7f4a831d8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556eaeb17ad0 0x556eaeb21e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:38.732 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.730+0000 7f4a831d8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556eaeb22390 0x556eaeb24780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:38.733 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.730+0000 7f4a831d8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556eaea7d900 con 0x556eaeb22390 2024-04-24T09:00:38.733 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.730+0000 7f4a831d8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556eaea23f20 con 0x556eaeb17ad0 2024-04-24T09:00:38.733 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.730+0000 7f4a831d8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x556eaea798c0 con 0x556eaeb171b0 2024-04-24T09:00:38.733 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.730+0000 7f4a7ce00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556eaeb22390 0x556eaeb24780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:38.733 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.730+0000 7f4a7ce00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556eaeb22390 0x556eaeb24780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56940/0 (socket says 172.21.15.83:56940) 2024-04-24T09:00:38.733 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.730+0000 7f4a7ce00640 1 -- 172.21.15.83:0/733730502 learned_addr learned my addr 172.21.15.83:0/733730502 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:00:38.733 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.731+0000 7f4a7e200640 1 --2- 172.21.15.83:0/733730502 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556eaeb171b0 0x556eaeb17590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:38.733 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.731+0000 7f4a7d800640 1 --2- 172.21.15.83:0/733730502 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556eaeb17ad0 0x556eaeb21e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:38.733 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.731+0000 7f4a7ce00640 1 -- 172.21.15.83:0/733730502 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556eaeb171b0 msgr2=0x556eaeb17590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:38.733 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.731+0000 7f4a7ce00640 1 --2- 172.21.15.83:0/733730502 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556eaeb171b0 0x556eaeb17590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:38.733 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.731+0000 7f4a7ce00640 1 -- 172.21.15.83:0/733730502 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556eaeb17ad0 msgr2=0x556eaeb21e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:38.733 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.731+0000 7f4a7ce00640 1 --2- 172.21.15.83:0/733730502 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556eaeb17ad0 0x556eaeb21e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:38.733 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.731+0000 7f4a7ce00640 1 -- 172.21.15.83:0/733730502 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556eaea48e10 con 0x556eaeb22390 2024-04-24T09:00:38.733 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.731+0000 7f4a7ce00640 1 --2- 172.21.15.83:0/733730502 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556eaeb22390 0x556eaeb24780 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f4a640cdc40 tx=0x7f4a640d94f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ab11a1a978f22e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:38.734 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.732+0000 7f4a6aa00640 1 -- 172.21.15.83:0/733730502 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a640e4070 con 0x556eaeb22390 2024-04-24T09:00:38.734 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.732+0000 7f4a76000640 1 --2- 172.21.15.83:0/1839021848 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f4a600a3e70 0x7f4a600a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:38.734 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.732+0000 7f4a6aa00640 1 -- 172.21.15.83:0/733730502 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4a640d9e10 con 0x556eaeb22390 2024-04-24T09:00:38.734 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.732+0000 7f4a6aa00640 1 -- 172.21.15.83:0/733730502 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a640de760 con 0x556eaeb22390 2024-04-24T09:00:38.734 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.732+0000 7f4a76000640 1 -- 172.21.15.83:0/1839021848 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f4a640d0a30 con 0x7f4a78103bb0 2024-04-24T09:00:38.734 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.732+0000 7f4a7e200640 1 --2- 172.21.15.83:0/1839021848 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f4a600a3e70 0x7f4a600a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:38.738 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.733+0000 7f4a831d8740 1 -- 172.21.15.83:0/733730502 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556eaeb22390 msgr2=0x556eaeb24780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:38.738 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.733+0000 7f4a831d8740 1 --2- 172.21.15.83:0/733730502 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556eaeb22390 0x556eaeb24780 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f4a640cdc40 tx=0x7f4a640d94f0 comp rx=0 tx=0).stop 2024-04-24T09:00:38.738 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.733+0000 7f4a831d8740 1 -- 172.21.15.83:0/733730502 shutdown_connections 2024-04-24T09:00:38.738 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.733+0000 7f4a831d8740 1 --2- 172.21.15.83:0/733730502 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556eaeb22390 0x556eaeb24780 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:38.738 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.733+0000 7f4a831d8740 1 --2- 172.21.15.83:0/733730502 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556eaeb17ad0 0x556eaeb21e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:38.738 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.733+0000 7f4a831d8740 1 --2- 172.21.15.83:0/733730502 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556eaeb171b0 0x556eaeb17590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:38.738 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.733+0000 7f4a831d8740 1 -- 172.21.15.83:0/733730502 >> 172.21.15.83:0/733730502 conn(0x556eaeb136c0 msgr2=0x556eaeb13aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:38.738 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.733+0000 7f4a7e200640 1 --2- 172.21.15.83:0/1839021848 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f4a600a3e70 0x7f4a600a6330 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f4a70005cc0 tx=0x7f4a700083f0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:38.738 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.733+0000 7f4a831d8740 1 -- 172.21.15.83:0/733730502 shutdown_connections 2024-04-24T09:00:38.739 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.733+0000 7f4a831d8740 1 -- 172.21.15.83:0/733730502 wait complete. 2024-04-24T09:00:38.739 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.733+0000 7f4a831d8740 1 -- start start 2024-04-24T09:00:38.739 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.734+0000 7f4a831d8740 1 Processor -- start 2024-04-24T09:00:38.739 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.734+0000 7f4a831d8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556eaeb171b0 0x556eaeb1f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:38.739 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.734+0000 7f4a831d8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556eaeb17ad0 0x556eaeb1f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:38.739 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.734+0000 7f4a831d8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556eaeb22390 0x556eaeb1fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:38.739 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.734+0000 7f4a831d8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556eaeb24fc0 con 0x556eaeb22390 2024-04-24T09:00:38.739 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.734+0000 7f4a831d8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556eaea372c0 con 0x556eaeb17ad0 2024-04-24T09:00:38.739 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.734+0000 7f4a831d8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x556eaeb25140 con 0x556eaeb171b0 2024-04-24T09:00:38.739 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.734+0000 7f4a831d8740 10 client.0 _ll_register_callbacks cb 0x556eae78fa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:00:38.739 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.734+0000 7f4a7e200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556eaeb22390 0x556eaeb1fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:38.739 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.734+0000 7f4a7e200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556eaeb22390 0x556eaeb1fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56946/0 (socket says 172.21.15.83:56946) 2024-04-24T09:00:38.739 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.734+0000 7f4a7e200640 1 -- 172.21.15.83:0/3318215044 learned_addr learned my addr 172.21.15.83:0/3318215044 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:00:38.739 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.735+0000 7f4a7ce00640 1 --2- 172.21.15.83:0/3318215044 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556eaeb17ad0 0x556eaeb1f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:38.740 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.735+0000 7f4a7d800640 1 --2- 172.21.15.83:0/3318215044 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556eaeb171b0 0x556eaeb1f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:38.740 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.735+0000 7f4a7e200640 1 -- 172.21.15.83:0/3318215044 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556eaeb171b0 msgr2=0x556eaeb1f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:38.740 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.735+0000 7f4a7e200640 1 --2- 172.21.15.83:0/3318215044 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556eaeb171b0 0x556eaeb1f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:38.740 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.735+0000 7f4a7e200640 1 -- 172.21.15.83:0/3318215044 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556eaeb17ad0 msgr2=0x556eaeb1f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:38.740 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.735+0000 7f4a7e200640 1 --2- 172.21.15.83:0/3318215044 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556eaeb17ad0 0x556eaeb1f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:38.740 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.735+0000 7f4a7e200640 1 -- 172.21.15.83:0/3318215044 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556eaea47fd0 con 0x556eaeb22390 2024-04-24T09:00:38.740 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.735+0000 7f4a7e200640 1 --2- 172.21.15.83:0/3318215044 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556eaeb22390 0x556eaeb1fe50 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f4a70010d90 tx=0x7f4a70004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69798add2f32ef51 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:38.740 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.735+0000 7f4a5f400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:00:38.740 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.736+0000 7f4a5f400640 1 -- 172.21.15.83:0/3318215044 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a7001d620 con 0x556eaeb22390 2024-04-24T09:00:38.740 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.736+0000 7f4a5f400640 1 -- 172.21.15.83:0/3318215044 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f4a7001d7c0 con 0x556eaeb22390 2024-04-24T09:00:38.740 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.736+0000 7f4a831d8740 1 -- 172.21.15.83:0/3318215044 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a640d9e10 con 0x556eaeb22390 2024-04-24T09:00:38.740 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.736+0000 7f4a5f400640 1 -- 172.21.15.83:0/3318215044 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f4a7002e050 con 0x556eaeb22390 2024-04-24T09:00:38.740 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.736+0000 7f4a831d8740 10 client.18810 fetch_fsmap learned FSMap version 35 2024-04-24T09:00:38.741 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.736+0000 7f4a831d8740 1 -- 172.21.15.83:0/3318215044 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x556eae790210 con 0x556eaeb22390 2024-04-24T09:00:38.741 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.736+0000 7f4a5f400640 1 -- 172.21.15.83:0/3318215044 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4a640d9e10 con 0x556eaeb22390 2024-04-24T09:00:38.741 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.736+0000 7f4a5f400640 1 -- 172.21.15.83:0/3318215044 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7f4a7001b910 con 0x556eaeb22390 2024-04-24T09:00:38.741 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.737+0000 7f4a831d8740 10 client.18810 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T09:00:38.741 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.737+0000 7f4a831d8740 10 client.18810 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:00:38.741 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.737+0000 7f4a831d8740 20 client.18810 populate_metadata read hostname 'smithi083' 2024-04-24T09:00:38.741 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.737+0000 7f4a831d8740 1 --2- 172.21.15.83:0/3318215044 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556eaeb325f0 0x556eaeb349e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:38.741 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.737+0000 7f4a831d8740 4 client.18810 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:00:38.741 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.737+0000 7f4a831d8740 1 -- 172.21.15.83:0/3318215044 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556eaea8b320 con 0x556eaeb325f0 2024-04-24T09:00:38.741 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.737+0000 7f4a7d800640 1 --2- 172.21.15.83:0/3318215044 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556eaeb325f0 0x556eaeb349e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:38.742 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.738+0000 7f4a7d800640 1 --2- 172.21.15.83:0/3318215044 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556eaeb325f0 0x556eaeb349e0 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c50c3d90b90e17e1 server_cookie=e2602041a04e91c6 in_seq=0 out_seq=0 2024-04-24T09:00:38.742 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.738+0000 7f4a5f400640 10 client.18810 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:00:38.743 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.742+0000 7f4a5f400640 1 -- 172.21.15.83:0/3318215044 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556eaea8b320 con 0x556eaeb325f0 2024-04-24T09:00:38.743 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.742+0000 7f4a5f400640 10 client.18810 handle_command_reply: tid=0 2024-04-24T09:00:38.743 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.742+0000 7f4a831d8740 1 client.18810 shutdown 2024-04-24T09:00:38.743 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.743+0000 7f4a831d8740 1 -- 172.21.15.83:0/3318215044 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556eaeb22390 msgr2=0x556eaeb1fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:38.743 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.743+0000 7f4a831d8740 1 --2- 172.21.15.83:0/3318215044 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556eaeb22390 0x556eaeb1fe50 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f4a70010d90 tx=0x7f4a70004a40 comp rx=0 tx=0).stop 2024-04-24T09:00:38.743 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.743+0000 7f4a831d8740 1 -- 172.21.15.83:0/3318215044 shutdown_connections 2024-04-24T09:00:38.743 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.743+0000 7f4a831d8740 1 --2- 172.21.15.83:0/3318215044 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556eaeb325f0 0x556eaeb349e0 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:38.743 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.743+0000 7f4a831d8740 1 --2- 172.21.15.83:0/3318215044 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556eaeb22390 0x556eaeb1fe50 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:38.743 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.743+0000 7f4a831d8740 1 --2- 172.21.15.83:0/3318215044 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556eaeb17ad0 0x556eaeb1f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:38.743 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.743+0000 7f4a831d8740 1 --2- 172.21.15.83:0/3318215044 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556eaeb171b0 0x556eaeb1f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:38.743 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.743+0000 7f4a5f400640 0 client.18810 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:00:38.744 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.743+0000 7f4a831d8740 1 -- 172.21.15.83:0/3318215044 >> 172.21.15.83:0/3318215044 conn(0x556eaeb136c0 msgr2=0x556eaeb12840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:38.744 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.743+0000 7f4a831d8740 1 -- 172.21.15.83:0/3318215044 shutdown_connections 2024-04-24T09:00:38.744 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.743+0000 7f4a831d8740 1 -- 172.21.15.83:0/3318215044 wait complete. 2024-04-24T09:00:38.744 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.743+0000 7f4a831d8740 20 client.18810 trim_cache size 0 max 16384 2024-04-24T09:00:38.762 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.762+0000 7f4a831d8740 1 Processor -- start 2024-04-24T09:00:38.762 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.762+0000 7f4a831d8740 1 -- start start 2024-04-24T09:00:38.763 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.762+0000 7f4a831d8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556eaeb171b0 0x556eaeb2de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:38.763 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.762+0000 7f4a831d8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556eaeb17ad0 0x556eaeb19b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:38.763 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.762+0000 7f4a831d8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556eaeb22390 0x556eaeb126c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:38.763 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.762+0000 7f4a831d8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4a781b1830 con 0x556eaeb22390 2024-04-24T09:00:38.763 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.762+0000 7f4a831d8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4a781b3370 con 0x556eaeb17ad0 2024-04-24T09:00:38.763 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.762+0000 7f4a831d8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4a781b1f00 con 0x556eaeb171b0 2024-04-24T09:00:38.763 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.763+0000 7f4a7e200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556eaeb22390 0x556eaeb126c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:38.764 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.763+0000 7f4a7e200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556eaeb22390 0x556eaeb126c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56958/0 (socket says 172.21.15.83:56958) 2024-04-24T09:00:38.764 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.763+0000 7f4a7e200640 1 -- 172.21.15.83:0/1837283604 learned_addr learned my addr 172.21.15.83:0/1837283604 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:00:38.764 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.763+0000 7f4a7ce00640 1 --2- 172.21.15.83:0/1837283604 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556eaeb17ad0 0x556eaeb19b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:38.764 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.763+0000 7f4a7d800640 1 --2- 172.21.15.83:0/1837283604 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556eaeb171b0 0x556eaeb2de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:38.764 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.763+0000 7f4a7e200640 1 -- 172.21.15.83:0/1837283604 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556eaeb171b0 msgr2=0x556eaeb2de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:38.764 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.763+0000 7f4a7e200640 1 --2- 172.21.15.83:0/1837283604 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556eaeb171b0 0x556eaeb2de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:38.764 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.763+0000 7f4a7e200640 1 -- 172.21.15.83:0/1837283604 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556eaeb17ad0 msgr2=0x556eaeb19b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:38.764 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.763+0000 7f4a7e200640 1 --2- 172.21.15.83:0/1837283604 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556eaeb17ad0 0x556eaeb19b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:38.764 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.763+0000 7f4a7e200640 1 -- 172.21.15.83:0/1837283604 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556eaeb34d50 con 0x556eaeb22390 2024-04-24T09:00:38.764 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.763+0000 7f4a7ce00640 1 --2- 172.21.15.83:0/1837283604 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556eaeb17ad0 0x556eaeb19b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:00:38.767 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.764+0000 7f4a7e200640 1 --2- 172.21.15.83:0/1837283604 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556eaeb22390 0x556eaeb126c0 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f4a7000e010 tx=0x7f4a70008b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b18c0df1fda1a143 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:38.767 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.764+0000 7f4a6a000640 1 -- 172.21.15.83:0/1837283604 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a70004ab0 con 0x556eaeb22390 2024-04-24T09:00:38.767 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.764+0000 7f4a6a000640 1 -- 172.21.15.83:0/1837283604 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4a70004c50 con 0x556eaeb22390 2024-04-24T09:00:38.767 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.764+0000 7f4a6a000640 1 -- 172.21.15.83:0/1837283604 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a7001c930 con 0x556eaeb22390 2024-04-24T09:00:38.767 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.765+0000 7f4a831d8740 1 -- 172.21.15.83:0/1837283604 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556eaeb22390 msgr2=0x556eaeb126c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:38.767 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.765+0000 7f4a831d8740 1 --2- 172.21.15.83:0/1837283604 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556eaeb22390 0x556eaeb126c0 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f4a7000e010 tx=0x7f4a70008b00 comp rx=0 tx=0).stop 2024-04-24T09:00:38.767 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.765+0000 7f4a831d8740 1 -- 172.21.15.83:0/1837283604 shutdown_connections 2024-04-24T09:00:38.767 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.765+0000 7f4a831d8740 1 --2- 172.21.15.83:0/1837283604 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556eaeb22390 0x556eaeb126c0 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:38.768 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.765+0000 7f4a831d8740 1 --2- 172.21.15.83:0/1837283604 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556eaeb17ad0 0x556eaeb19b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:38.768 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.765+0000 7f4a831d8740 1 --2- 172.21.15.83:0/1837283604 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556eaeb171b0 0x556eaeb2de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:38.768 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.765+0000 7f4a831d8740 1 -- 172.21.15.83:0/1837283604 >> 172.21.15.83:0/1837283604 conn(0x556eaeb136c0 msgr2=0x556eaeb11fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:38.768 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.765+0000 7f4a831d8740 1 -- 172.21.15.83:0/1837283604 shutdown_connections 2024-04-24T09:00:38.768 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.765+0000 7f4a831d8740 1 -- 172.21.15.83:0/1837283604 wait complete. 2024-04-24T09:00:38.768 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.765+0000 7f4a831d8740 1 -- start start 2024-04-24T09:00:38.768 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.765+0000 7f4a831d8740 1 Processor -- start 2024-04-24T09:00:38.768 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.766+0000 7f4a831d8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556eaeb171b0 0x556eaeb269b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:38.768 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.766+0000 7f4a831d8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556eaeb17ad0 0x556eaeb26ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:38.768 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.766+0000 7f4a831d8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556eaeb22390 0x556eaeb90b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:38.768 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.766+0000 7f4a831d8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4a781b2ca0 con 0x556eaeb171b0 2024-04-24T09:00:38.768 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.766+0000 7f4a831d8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4a781b25d0 con 0x556eaeb22390 2024-04-24T09:00:38.769 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.766+0000 7f4a831d8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4a781af830 con 0x556eaeb17ad0 2024-04-24T09:00:38.769 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.766+0000 7f4a831d8740 10 client.0 _ll_register_callbacks cb 0x7f4a70008a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:00:38.769 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.766+0000 7f4a7d800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556eaeb171b0 0x556eaeb269b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:38.769 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.766+0000 7f4a7d800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556eaeb171b0 0x556eaeb269b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56968/0 (socket says 172.21.15.83:56968) 2024-04-24T09:00:38.769 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.766+0000 7f4a7d800640 1 -- 172.21.15.83:0/65369204 learned_addr learned my addr 172.21.15.83:0/65369204 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:00:38.769 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.766+0000 7f4a7e200640 1 --2- 172.21.15.83:0/65369204 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556eaeb22390 0x556eaeb90b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:38.769 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.767+0000 7f4a7ce00640 1 --2- 172.21.15.83:0/65369204 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556eaeb17ad0 0x556eaeb26ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:38.769 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.767+0000 7f4a7d800640 1 -- 172.21.15.83:0/65369204 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556eaeb17ad0 msgr2=0x556eaeb26ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:38.769 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.767+0000 7f4a7d800640 1 --2- 172.21.15.83:0/65369204 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556eaeb17ad0 0x556eaeb26ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:38.769 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.767+0000 7f4a7d800640 1 -- 172.21.15.83:0/65369204 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556eaeb22390 msgr2=0x556eaeb90b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:38.769 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.767+0000 7f4a7d800640 1 --2- 172.21.15.83:0/65369204 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556eaeb22390 0x556eaeb90b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:38.769 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.767+0000 7f4a7d800640 1 -- 172.21.15.83:0/65369204 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556eae85c620 con 0x556eaeb171b0 2024-04-24T09:00:38.769 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.767+0000 7f4a7e200640 1 --2- 172.21.15.83:0/65369204 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556eaeb22390 0x556eaeb90b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:00:38.769 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.767+0000 7f4a7d800640 1 --2- 172.21.15.83:0/65369204 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556eaeb171b0 0x556eaeb269b0 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c005ab0 tx=0x7f4a6c005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cfac7df34c9e09 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:38.770 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.767+0000 7f4a5f400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:00:38.770 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.767+0000 7f4a5f400640 1 -- 172.21.15.83:0/65369204 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a6c035040 con 0x556eaeb171b0 2024-04-24T09:00:38.770 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.767+0000 7f4a5f400640 1 -- 172.21.15.83:0/65369204 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f4a6c042d60 con 0x556eaeb171b0 2024-04-24T09:00:38.770 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.767+0000 7f4a831d8740 1 -- 172.21.15.83:0/65369204 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a70004c50 con 0x556eaeb171b0 2024-04-24T09:00:38.771 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.767+0000 7f4a5f400640 1 -- 172.21.15.83:0/65369204 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f4a6c051050 con 0x556eaeb171b0 2024-04-24T09:00:38.771 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.768+0000 7f4a831d8740 10 client.18816 fetch_fsmap learned FSMap version 35 2024-04-24T09:00:38.771 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.768+0000 7f4a831d8740 1 -- 172.21.15.83:0/65369204 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x556eaeb99480 con 0x556eaeb171b0 2024-04-24T09:00:38.771 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.768+0000 7f4a5f400640 1 -- 172.21.15.83:0/65369204 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4a70004c50 con 0x556eaeb171b0 2024-04-24T09:00:38.771 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.768+0000 7f4a5f400640 1 -- 172.21.15.83:0/65369204 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7f4a6c042740 con 0x556eaeb171b0 2024-04-24T09:00:38.771 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.768+0000 7f4a831d8740 10 client.18816 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T09:00:38.771 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.768+0000 7f4a831d8740 10 client.18816 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:00:38.771 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.768+0000 7f4a831d8740 20 client.18816 populate_metadata read hostname 'smithi083' 2024-04-24T09:00:38.772 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.768+0000 7f4a831d8740 1 --2- 172.21.15.83:0/65369204 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556eaeb99ba0 0x556eaeb9bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:38.772 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.768+0000 7f4a831d8740 4 client.18816 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T09:00:38.772 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.768+0000 7f4a831d8740 1 -- 172.21.15.83:0/65369204 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556eaeb13510 con 0x556eaeb99ba0 2024-04-24T09:00:38.772 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.769+0000 7f4a7ce00640 1 --2- 172.21.15.83:0/65369204 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556eaeb99ba0 0x556eaeb9bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:38.772 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.770+0000 7f4a7ce00640 1 --2- 172.21.15.83:0/65369204 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556eaeb99ba0 0x556eaeb9bfc0 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95765774c2f88bad server_cookie=d6f8ff7e0ebac16b in_seq=0 out_seq=0 2024-04-24T09:00:38.772 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.770+0000 7f4a5f400640 10 client.18816 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:00:38.830 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.830+0000 7f4a5f400640 1 -- 172.21.15.83:0/65369204 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556eaeb13510 con 0x556eaeb99ba0 2024-04-24T09:00:38.830 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.830+0000 7f4a5f400640 10 client.18816 handle_command_reply: tid=0 2024-04-24T09:00:38.830 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.830+0000 7f4a831d8740 1 client.18816 shutdown 2024-04-24T09:00:38.831 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.830+0000 7f4a831d8740 1 -- 172.21.15.83:0/65369204 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556eaeb171b0 msgr2=0x556eaeb269b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:38.831 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.830+0000 7f4a831d8740 1 --2- 172.21.15.83:0/65369204 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556eaeb171b0 0x556eaeb269b0 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c005ab0 tx=0x7f4a6c005d50 comp rx=0 tx=0).stop 2024-04-24T09:00:38.831 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.831+0000 7f4a831d8740 1 -- 172.21.15.83:0/65369204 shutdown_connections 2024-04-24T09:00:38.831 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.831+0000 7f4a831d8740 1 --2- 172.21.15.83:0/65369204 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556eaeb99ba0 0x556eaeb9bfc0 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:38.831 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.831+0000 7f4a831d8740 1 --2- 172.21.15.83:0/65369204 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556eaeb22390 0x556eaeb90b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:38.831 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.831+0000 7f4a831d8740 1 --2- 172.21.15.83:0/65369204 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556eaeb17ad0 0x556eaeb26ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:38.831 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.831+0000 7f4a831d8740 1 --2- 172.21.15.83:0/65369204 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556eaeb171b0 0x556eaeb269b0 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:38.831 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.831+0000 7f4a831d8740 1 -- 172.21.15.83:0/65369204 >> 172.21.15.83:0/65369204 conn(0x556eaeb136c0 msgr2=0x556eaeb12840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:38.831 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.831+0000 7f4a5f400640 0 client.18816 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:00:38.832 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.831+0000 7f4a831d8740 1 -- 172.21.15.83:0/65369204 shutdown_connections 2024-04-24T09:00:38.832 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.832+0000 7f4a831d8740 1 -- 172.21.15.83:0/65369204 wait complete. 2024-04-24T09:00:38.832 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.832+0000 7f4a831d8740 20 client.18816 trim_cache size 0 max 16384 2024-04-24T09:00:38.833 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T09:00:38.833 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T09:00:38.833 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T09:00:38.833 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T09:00:38.834 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.833+0000 7f4a80200640 1 -- 172.21.15.83:0/1839021848 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f4a600a3e70 msgr2=0x7f4a600a6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:38.834 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.833+0000 7f4a80200640 1 --2- 172.21.15.83:0/1839021848 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f4a600a3e70 0x7f4a600a6330 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f4a70005cc0 tx=0x7f4a700083f0 comp rx=0 tx=0).stop 2024-04-24T09:00:38.834 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.833+0000 7f4a80200640 1 -- 172.21.15.83:0/1839021848 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a78103bb0 msgr2=0x7f4a78101420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:38.834 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.833+0000 7f4a80200640 1 --2- 172.21.15.83:0/1839021848 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a78103bb0 0x7f4a78101420 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f4a6400c970 tx=0x7f4a6400ce30 comp rx=0 tx=0).stop 2024-04-24T09:00:38.834 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.834+0000 7f4a80200640 1 -- 172.21.15.83:0/1839021848 shutdown_connections 2024-04-24T09:00:38.834 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.834+0000 7f4a80200640 1 --2- 172.21.15.83:0/1839021848 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f4a600a3e70 0x7f4a600a6330 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:38.834 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.834+0000 7f4a80200640 1 --2- 172.21.15.83:0/1839021848 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4a7810c5d0 0x7f4a78101960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:38.834 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.834+0000 7f4a80200640 1 --2- 172.21.15.83:0/1839021848 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a78103bb0 0x7f4a78101420 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:38.835 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.834+0000 7f4a80200640 1 --2- 172.21.15.83:0/1839021848 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a78103290 0x7f4a78100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:38.835 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.834+0000 7f4a80200640 1 -- 172.21.15.83:0/1839021848 >> 172.21.15.83:0/1839021848 conn(0x7f4a78078080 msgr2=0x7f4a780fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:38.835 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.834+0000 7f4a80200640 1 -- 172.21.15.83:0/1839021848 shutdown_connections 2024-04-24T09:00:38.835 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:38.834+0000 7f4a80200640 1 -- 172.21.15.83:0/1839021848 wait complete. 2024-04-24T09:00:39.601 INFO:tasks.workunit.client.0.smithi083.stdout: 1 51298 0.63 MB/sec execute 313 sec latency 66.378 ms 2024-04-24T09:00:40.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:00:39 smithi138 ceph-mon[31441]: pgmap v282: 129 pgs: 129 active+clean; 50 MiB data, 732 MiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 501 KiB/s wr, 77 op/s 2024-04-24T09:00:40.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:00:39 smithi189 ceph-mon[31791]: pgmap v282: 129 pgs: 129 active+clean; 50 MiB data, 732 MiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 501 KiB/s wr, 77 op/s 2024-04-24T09:00:40.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:00:39 smithi083 ceph-mon[27907]: pgmap v282: 129 pgs: 129 active+clean; 50 MiB data, 732 MiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 501 KiB/s wr, 77 op/s 2024-04-24T09:00:40.602 INFO:tasks.workunit.client.0.smithi083.stdout: 1 51394 0.63 MB/sec execute 314 sec latency 66.323 ms 2024-04-24T09:00:41.602 INFO:tasks.workunit.client.0.smithi083.stdout: 1 51533 0.62 MB/sec execute 315 sec latency 64.214 ms 2024-04-24T09:00:42.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:00:41 smithi138 ceph-mon[31441]: pgmap v283: 129 pgs: 129 active+clean; 50 MiB data, 732 MiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 292 KiB/s wr, 50 op/s 2024-04-24T09:00:42.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:00:41 smithi189 ceph-mon[31791]: pgmap v283: 129 pgs: 129 active+clean; 50 MiB data, 732 MiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 292 KiB/s wr, 50 op/s 2024-04-24T09:00:42.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:00:41 smithi083 ceph-mon[27907]: pgmap v283: 129 pgs: 129 active+clean; 50 MiB data, 732 MiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 292 KiB/s wr, 50 op/s 2024-04-24T09:00:42.602 INFO:tasks.workunit.client.0.smithi083.stdout: 1 51658 0.62 MB/sec execute 316 sec latency 52.029 ms 2024-04-24T09:00:42.882 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T09:00:42.883 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T09:00:43.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.120+0000 7f3e03200640 1 -- 172.21.15.83:0/2065829972 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e041003b0 msgr2=0x7f3e04100790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:43.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.120+0000 7f3e03200640 1 --2- 172.21.15.83:0/2065829972 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e041003b0 0x7f3e04100790 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f3df4009900 tx=0x7f3df402a410 comp rx=0 tx=0).stop 2024-04-24T09:00:43.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.120+0000 7f3e03200640 1 -- 172.21.15.83:0/2065829972 shutdown_connections 2024-04-24T09:00:43.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.120+0000 7f3e03200640 1 --2- 172.21.15.83:0/2065829972 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3e0410e8f0 0x7f3e04110ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:43.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.120+0000 7f3e03200640 1 --2- 172.21.15.83:0/2065829972 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3e04100cd0 0x7f3e0410e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:43.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.120+0000 7f3e03200640 1 --2- 172.21.15.83:0/2065829972 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e041003b0 0x7f3e04100790 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:43.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.120+0000 7f3e03200640 1 -- 172.21.15.83:0/2065829972 >> 172.21.15.83:0/2065829972 conn(0x7f3e04075d00 msgr2=0x7f3e04076110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:43.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.120+0000 7f3e03200640 1 -- 172.21.15.83:0/2065829972 shutdown_connections 2024-04-24T09:00:43.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.120+0000 7f3e03200640 1 -- 172.21.15.83:0/2065829972 wait complete. 2024-04-24T09:00:43.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.121+0000 7f3e03200640 1 Processor -- start 2024-04-24T09:00:43.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.121+0000 7f3e03200640 1 -- start start 2024-04-24T09:00:43.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.122+0000 7f3e03200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e041003b0 0x7f3e041a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:43.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.122+0000 7f3e03200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3e04100cd0 0x7f3e041a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:43.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.122+0000 7f3e03200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3e0410e8f0 0x7f3e0419a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:43.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.122+0000 7f3e03200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3e041143a0 con 0x7f3e041003b0 2024-04-24T09:00:43.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.122+0000 7f3e03200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3e04114220 con 0x7f3e04100cd0 2024-04-24T09:00:43.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.122+0000 7f3e03200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3e04114520 con 0x7f3e0410e8f0 2024-04-24T09:00:43.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.122+0000 7f3e01e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e041003b0 0x7f3e041a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:43.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.122+0000 7f3e01e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e041003b0 0x7f3e041a0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48490/0 (socket says 172.21.15.83:48490) 2024-04-24T09:00:43.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.122+0000 7f3e01e00640 1 -- 172.21.15.83:0/604641044 learned_addr learned my addr 172.21.15.83:0/604641044 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:00:43.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.122+0000 7f3e01400640 1 --2- 172.21.15.83:0/604641044 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3e04100cd0 0x7f3e041a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:43.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.122+0000 7f3e01e00640 1 -- 172.21.15.83:0/604641044 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3e0410e8f0 msgr2=0x7f3e0419a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:43.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.122+0000 7f3e01e00640 1 --2- 172.21.15.83:0/604641044 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3e0410e8f0 0x7f3e0419a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:43.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.122+0000 7f3e01e00640 1 -- 172.21.15.83:0/604641044 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3e04100cd0 msgr2=0x7f3e041a0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:43.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.122+0000 7f3e01e00640 1 --2- 172.21.15.83:0/604641044 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3e04100cd0 0x7f3e041a0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:43.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.122+0000 7f3e01e00640 1 -- 172.21.15.83:0/604641044 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e0419afe0 con 0x7f3e041003b0 2024-04-24T09:00:43.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.123+0000 7f3e01e00640 1 --2- 172.21.15.83:0/604641044 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e041003b0 0x7f3e041a0610 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f3df4005ab0 tx=0x7f3df4003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:43.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.123+0000 7f3dfaa00640 1 -- 172.21.15.83:0/604641044 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df403f070 con 0x7f3e041003b0 2024-04-24T09:00:43.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.123+0000 7f3dfaa00640 1 -- 172.21.15.83:0/604641044 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3df4031d60 con 0x7f3e041003b0 2024-04-24T09:00:43.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.123+0000 7f3e03200640 1 -- 172.21.15.83:0/604641044 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e0419b270 con 0x7f3e041003b0 2024-04-24T09:00:43.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.123+0000 7f3dfaa00640 1 -- 172.21.15.83:0/604641044 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df4035030 con 0x7f3e041003b0 2024-04-24T09:00:43.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.123+0000 7f3e03200640 1 -- 172.21.15.83:0/604641044 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e041a73b0 con 0x7f3e041003b0 2024-04-24T09:00:43.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.127+0000 7f3e0bfc4740 1 Processor -- start 2024-04-24T09:00:43.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.127+0000 7f3dfaa00640 1 -- 172.21.15.83:0/604641044 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3df4042070 con 0x7f3e041003b0 2024-04-24T09:00:43.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.127+0000 7f3e0bfc4740 1 -- start start 2024-04-24T09:00:43.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.128+0000 7f3e0bfc4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e317f8a30 0x556e317f8e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:43.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.128+0000 7f3e0bfc4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556e317f9350 0x556e318034e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:43.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.128+0000 7f3e0bfc4740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556e31803a20 0x556e31805e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:43.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.128+0000 7f3e0bfc4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556e3175e1f0 con 0x556e317f8a30 2024-04-24T09:00:43.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.128+0000 7f3e0bfc4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556e31704350 con 0x556e317f9350 2024-04-24T09:00:43.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.128+0000 7f3e0bfc4740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x556e3175a1b0 con 0x556e31803a20 2024-04-24T09:00:43.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.128+0000 7f3e02800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e317f8a30 0x556e317f8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:43.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.128+0000 7f3e02800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e317f8a30 0x556e317f8e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48502/0 (socket says 172.21.15.83:48502) 2024-04-24T09:00:43.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.128+0000 7f3e02800640 1 -- 172.21.15.83:0/1238720772 learned_addr learned my addr 172.21.15.83:0/1238720772 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:00:43.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.128+0000 7f3e01e00640 1 --2- 172.21.15.83:0/1238720772 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556e317f9350 0x556e318034e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:43.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.128+0000 7f3e01400640 1 --2- 172.21.15.83:0/1238720772 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556e31803a20 0x556e31805e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:43.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.129+0000 7f3e02800640 1 -- 172.21.15.83:0/1238720772 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556e31803a20 msgr2=0x556e31805e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:43.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.129+0000 7f3e02800640 1 --2- 172.21.15.83:0/1238720772 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556e31803a20 0x556e31805e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:43.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.129+0000 7f3e02800640 1 -- 172.21.15.83:0/1238720772 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556e317f9350 msgr2=0x556e318034e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:43.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.129+0000 7f3e02800640 1 --2- 172.21.15.83:0/1238720772 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556e317f9350 0x556e318034e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:43.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.129+0000 7f3e02800640 1 -- 172.21.15.83:0/1238720772 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e31758f50 con 0x556e317f8a30 2024-04-24T09:00:43.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.129+0000 7f3e02800640 1 --2- 172.21.15.83:0/1238720772 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e317f8a30 0x556e317f8e10 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc009cd0 tx=0x7f3dfc00e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ca419bba09cbe81 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:43.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.130+0000 7f3dfaa00640 1 --2- 172.21.15.83:0/604641044 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f3de80a3d20 0x7f3de80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:43.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.130+0000 7f3def400640 1 -- 172.21.15.83:0/1238720772 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dfc016070 con 0x556e317f8a30 2024-04-24T09:00:43.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.130+0000 7f3def400640 1 -- 172.21.15.83:0/1238720772 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3dfc005020 con 0x556e317f8a30 2024-04-24T09:00:43.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.130+0000 7f3def400640 1 -- 172.21.15.83:0/1238720772 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dfc0116d0 con 0x556e317f8a30 2024-04-24T09:00:43.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.130+0000 7f3e01e00640 1 --2- 172.21.15.83:0/604641044 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f3de80a3d20 0x7f3de80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:43.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.130+0000 7f3dfaa00640 1 -- 172.21.15.83:0/604641044 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f3df40f15f0 con 0x7f3e041003b0 2024-04-24T09:00:43.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.130+0000 7f3e0bfc4740 1 -- 172.21.15.83:0/1238720772 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e317f8a30 msgr2=0x556e317f8e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:43.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.130+0000 7f3e0bfc4740 1 --2- 172.21.15.83:0/1238720772 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e317f8a30 0x556e317f8e10 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc009cd0 tx=0x7f3dfc00e6c0 comp rx=0 tx=0).stop 2024-04-24T09:00:43.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.131+0000 7f3e0bfc4740 1 -- 172.21.15.83:0/1238720772 shutdown_connections 2024-04-24T09:00:43.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.131+0000 7f3e0bfc4740 1 --2- 172.21.15.83:0/1238720772 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556e31803a20 0x556e31805e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:43.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.131+0000 7f3e0bfc4740 1 --2- 172.21.15.83:0/1238720772 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556e317f9350 0x556e318034e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:43.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.131+0000 7f3e01e00640 1 --2- 172.21.15.83:0/604641044 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f3de80a3d20 0x7f3de80a61e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x556e317f9ff0 tx=0x7f3df4031400 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:43.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.131+0000 7f3e0bfc4740 1 --2- 172.21.15.83:0/1238720772 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e317f8a30 0x556e317f8e10 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:43.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.131+0000 7f3e0bfc4740 1 -- 172.21.15.83:0/1238720772 >> 172.21.15.83:0/1238720772 conn(0x556e317f4d50 msgr2=0x556e317f5130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:43.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.131+0000 7f3e0bfc4740 1 -- 172.21.15.83:0/1238720772 shutdown_connections 2024-04-24T09:00:43.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.131+0000 7f3e0bfc4740 1 -- 172.21.15.83:0/1238720772 wait complete. 2024-04-24T09:00:43.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.131+0000 7f3e0bfc4740 1 -- start start 2024-04-24T09:00:43.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.132+0000 7f3e0bfc4740 1 Processor -- start 2024-04-24T09:00:43.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.132+0000 7f3e0bfc4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556e317f8a30 0x556e31800980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:43.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.132+0000 7f3e0bfc4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e317f9350 0x556e31800ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:43.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.132+0000 7f3e0bfc4740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556e31803a20 0x556e31801400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:43.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.132+0000 7f3e0bfc4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556e31806810 con 0x556e317f9350 2024-04-24T09:00:43.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.132+0000 7f3e0bfc4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556e31717690 con 0x556e317f8a30 2024-04-24T09:00:43.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.132+0000 7f3e0bfc4740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x556e31806990 con 0x556e31803a20 2024-04-24T09:00:43.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.132+0000 7f3e0bfc4740 10 client.0 _ll_register_callbacks cb 0x556e31753650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:00:43.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.133+0000 7f3e01400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e317f9350 0x556e31800ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:43.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.133+0000 7f3e01400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e317f9350 0x556e31800ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48514/0 (socket says 172.21.15.83:48514) 2024-04-24T09:00:43.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.133+0000 7f3e01400640 1 -- 172.21.15.83:0/3800883565 learned_addr learned my addr 172.21.15.83:0/3800883565 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:00:43.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.133+0000 7f3e02800640 1 --2- 172.21.15.83:0/3800883565 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556e317f8a30 0x556e31800980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:43.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.133+0000 7f3e02800640 1 --2- 172.21.15.83:0/3800883565 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556e31803a20 0x556e31801400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:43.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.133+0000 7f3e01400640 1 -- 172.21.15.83:0/3800883565 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556e31803a20 msgr2=0x556e31801400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:43.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.133+0000 7f3e01400640 1 --2- 172.21.15.83:0/3800883565 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556e31803a20 0x556e31801400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:43.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.133+0000 7f3e01400640 1 -- 172.21.15.83:0/3800883565 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556e317f8a30 msgr2=0x556e31800980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:43.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.133+0000 7f3e01400640 1 --2- 172.21.15.83:0/3800883565 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556e317f8a30 0x556e31800980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:43.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.133+0000 7f3e01400640 1 -- 172.21.15.83:0/3800883565 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e31716920 con 0x556e317f9350 2024-04-24T09:00:43.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.133+0000 7f3e02800640 1 --2- 172.21.15.83:0/3800883565 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556e317f8a30 0x556e31800980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:00:43.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.134+0000 7f3e01400640 1 --2- 172.21.15.83:0/3800883565 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e317f9350 0x556e31800ec0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f3df000be10 tx=0x7f3df00117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55cadb6d9aedd4c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:43.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.134+0000 7f3de3e00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:00:43.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.134+0000 7f3de3e00640 1 -- 172.21.15.83:0/3800883565 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df0004230 con 0x556e317f9350 2024-04-24T09:00:43.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.134+0000 7f3de3e00640 1 -- 172.21.15.83:0/3800883565 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f3df00043d0 con 0x556e317f9350 2024-04-24T09:00:43.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.134+0000 7f3e0bfc4740 1 -- 172.21.15.83:0/3800883565 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3dfc005020 con 0x556e317f9350 2024-04-24T09:00:43.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.134+0000 7f3de3e00640 1 -- 172.21.15.83:0/3800883565 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f3df0024050 con 0x556e317f9350 2024-04-24T09:00:43.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.134+0000 7f3e0bfc4740 10 client.18828 fetch_fsmap learned FSMap version 35 2024-04-24T09:00:43.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.134+0000 7f3e0bfc4740 1 -- 172.21.15.83:0/3800883565 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x556e3153a7d0 con 0x556e317f9350 2024-04-24T09:00:43.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.135+0000 7f3de3e00640 1 -- 172.21.15.83:0/3800883565 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3dfc005020 con 0x556e317f9350 2024-04-24T09:00:43.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.135+0000 7f3de3e00640 1 -- 172.21.15.83:0/3800883565 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7f3df001b730 con 0x556e317f9350 2024-04-24T09:00:43.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.135+0000 7f3e0bfc4740 10 client.18828 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T09:00:43.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.135+0000 7f3e0bfc4740 10 client.18828 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:00:43.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.135+0000 7f3e0bfc4740 20 client.18828 populate_metadata read hostname 'smithi083' 2024-04-24T09:00:43.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.135+0000 7f3e0bfc4740 1 --2- 172.21.15.83:0/3800883565 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556e3181c630 0x556e3181ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:43.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.135+0000 7f3e0bfc4740 4 client.18828 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:00:43.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.135+0000 7f3e0bfc4740 1 -- 172.21.15.83:0/3800883565 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556e3176bc10 con 0x556e3181c630 2024-04-24T09:00:43.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.136+0000 7f3e02800640 1 --2- 172.21.15.83:0/3800883565 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556e3181c630 0x556e3181ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:43.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.138+0000 7f3e02800640 1 --2- 172.21.15.83:0/3800883565 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556e3181c630 0x556e3181ea80 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa340ab9d48ac1e5 server_cookie=edd0d3ef9ad497ce in_seq=0 out_seq=0 2024-04-24T09:00:43.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.138+0000 7f3de3e00640 10 client.18828 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:00:43.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.143+0000 7f3de3e00640 1 -- 172.21.15.83:0/3800883565 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556e3176bc10 con 0x556e3181c630 2024-04-24T09:00:43.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.143+0000 7f3de3e00640 10 client.18828 handle_command_reply: tid=0 2024-04-24T09:00:43.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.143+0000 7f3e0bfc4740 1 client.18828 shutdown 2024-04-24T09:00:43.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.143+0000 7f3e0bfc4740 1 -- 172.21.15.83:0/3800883565 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e317f9350 msgr2=0x556e31800ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:43.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.143+0000 7f3e0bfc4740 1 --2- 172.21.15.83:0/3800883565 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e317f9350 0x556e31800ec0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f3df000be10 tx=0x7f3df00117b0 comp rx=0 tx=0).stop 2024-04-24T09:00:43.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.143+0000 7f3e0bfc4740 1 -- 172.21.15.83:0/3800883565 shutdown_connections 2024-04-24T09:00:43.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.143+0000 7f3e0bfc4740 1 --2- 172.21.15.83:0/3800883565 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556e3181c630 0x556e3181ea80 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:43.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.143+0000 7f3e0bfc4740 1 --2- 172.21.15.83:0/3800883565 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556e31803a20 0x556e31801400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:43.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.143+0000 7f3e0bfc4740 1 --2- 172.21.15.83:0/3800883565 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e317f9350 0x556e31800ec0 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:43.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.143+0000 7f3e0bfc4740 1 --2- 172.21.15.83:0/3800883565 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556e317f8a30 0x556e31800980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:43.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.143+0000 7f3de3e00640 0 client.18828 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:00:43.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.143+0000 7f3e0bfc4740 1 -- 172.21.15.83:0/3800883565 >> 172.21.15.83:0/3800883565 conn(0x556e317f4d50 msgr2=0x556e317f3fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:43.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.143+0000 7f3e0bfc4740 1 -- 172.21.15.83:0/3800883565 shutdown_connections 2024-04-24T09:00:43.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.143+0000 7f3e0bfc4740 1 -- 172.21.15.83:0/3800883565 wait complete. 2024-04-24T09:00:43.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.143+0000 7f3e0bfc4740 20 client.18828 trim_cache size 0 max 16384 2024-04-24T09:00:43.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.163+0000 7f3e0bfc4740 1 Processor -- start 2024-04-24T09:00:43.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.163+0000 7f3e0bfc4740 1 -- start start 2024-04-24T09:00:43.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.163+0000 7f3e0bfc4740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556e317f8a30 0x556e31817fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:43.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.163+0000 7f3e0bfc4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556e317f9350 0x556e31824390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:43.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.163+0000 7f3e0bfc4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e31803a20 0x556e318248d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:43.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.163+0000 7f3e0bfc4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3e041ad190 con 0x556e31803a20 2024-04-24T09:00:43.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.163+0000 7f3e0bfc4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3e041aecd0 con 0x556e317f9350 2024-04-24T09:00:43.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.163+0000 7f3e0bfc4740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3e041ad860 con 0x556e317f8a30 2024-04-24T09:00:43.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.164+0000 7f3e02800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e31803a20 0x556e318248d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:43.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.164+0000 7f3e02800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e31803a20 0x556e318248d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48526/0 (socket says 172.21.15.83:48526) 2024-04-24T09:00:43.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.164+0000 7f3e02800640 1 -- 172.21.15.83:0/1434403897 learned_addr learned my addr 172.21.15.83:0/1434403897 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:00:43.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.164+0000 7f3e02800640 1 --2- 172.21.15.83:0/1434403897 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556e317f8a30 0x556e31817fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:43.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.164+0000 7f3e01400640 1 --2- 172.21.15.83:0/1434403897 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556e317f9350 0x556e31824390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:43.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.164+0000 7f3e02800640 1 -- 172.21.15.83:0/1434403897 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556e317f8a30 msgr2=0x556e31817fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:43.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.164+0000 7f3e02800640 1 --2- 172.21.15.83:0/1434403897 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556e317f8a30 0x556e31817fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:43.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.164+0000 7f3e02800640 1 -- 172.21.15.83:0/1434403897 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556e317f9350 msgr2=0x556e31824390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:43.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.164+0000 7f3e02800640 1 --2- 172.21.15.83:0/1434403897 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556e317f9350 0x556e31824390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:43.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.164+0000 7f3e02800640 1 -- 172.21.15.83:0/1434403897 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e314703f0 con 0x556e31803a20 2024-04-24T09:00:43.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.164+0000 7f3e02800640 1 --2- 172.21.15.83:0/1434403897 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e31803a20 0x556e318248d0 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc009a60 tx=0x7f3dfc0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a98342bbab19f7c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:43.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.165+0000 7f3deea00640 1 -- 172.21.15.83:0/1434403897 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dfc016040 con 0x556e31803a20 2024-04-24T09:00:43.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.165+0000 7f3deea00640 1 -- 172.21.15.83:0/1434403897 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3dfc0297f0 con 0x556e31803a20 2024-04-24T09:00:43.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.165+0000 7f3deea00640 1 -- 172.21.15.83:0/1434403897 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dfc01a660 con 0x556e31803a20 2024-04-24T09:00:43.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.166+0000 7f3e0bfc4740 1 -- 172.21.15.83:0/1434403897 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e31803a20 msgr2=0x556e318248d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:43.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.166+0000 7f3e0bfc4740 1 --2- 172.21.15.83:0/1434403897 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e31803a20 0x556e318248d0 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc009a60 tx=0x7f3dfc0095c0 comp rx=0 tx=0).stop 2024-04-24T09:00:43.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.166+0000 7f3e0bfc4740 1 -- 172.21.15.83:0/1434403897 shutdown_connections 2024-04-24T09:00:43.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.166+0000 7f3e0bfc4740 1 --2- 172.21.15.83:0/1434403897 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e31803a20 0x556e318248d0 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:43.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.166+0000 7f3e0bfc4740 1 --2- 172.21.15.83:0/1434403897 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556e317f9350 0x556e31824390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:43.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.166+0000 7f3e0bfc4740 1 --2- 172.21.15.83:0/1434403897 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556e317f8a30 0x556e31817fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:43.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.166+0000 7f3e0bfc4740 1 -- 172.21.15.83:0/1434403897 >> 172.21.15.83:0/1434403897 conn(0x556e317f4d50 msgr2=0x556e317fa180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:43.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.166+0000 7f3e0bfc4740 1 -- 172.21.15.83:0/1434403897 shutdown_connections 2024-04-24T09:00:43.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.166+0000 7f3e0bfc4740 1 -- 172.21.15.83:0/1434403897 wait complete. 2024-04-24T09:00:43.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.166+0000 7f3e0bfc4740 1 -- start start 2024-04-24T09:00:43.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.166+0000 7f3e0bfc4740 1 Processor -- start 2024-04-24T09:00:43.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.167+0000 7f3e0bfc4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e317f8a30 0x556e318144d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:43.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.167+0000 7f3e0bfc4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556e317f9350 0x556e31814a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:43.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.167+0000 7f3e0bfc4740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556e31803a20 0x556e31811650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:43.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.167+0000 7f3e0bfc4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3e041ae600 con 0x556e317f8a30 2024-04-24T09:00:43.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.167+0000 7f3e0bfc4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3e041adf30 con 0x556e317f9350 2024-04-24T09:00:43.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.167+0000 7f3e0bfc4740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3e041ab190 con 0x556e31803a20 2024-04-24T09:00:43.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.167+0000 7f3e0bfc4740 10 client.0 _ll_register_callbacks cb 0x7f3df000b3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:00:43.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.167+0000 7f3e02800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e317f8a30 0x556e318144d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:43.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.167+0000 7f3e02800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e317f8a30 0x556e318144d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48528/0 (socket says 172.21.15.83:48528) 2024-04-24T09:00:43.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.167+0000 7f3e02800640 1 -- 172.21.15.83:0/2880497773 learned_addr learned my addr 172.21.15.83:0/2880497773 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:00:43.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.168+0000 7f3e01400640 1 --2- 172.21.15.83:0/2880497773 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556e317f9350 0x556e31814a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:43.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.168+0000 7f3e02800640 1 -- 172.21.15.83:0/2880497773 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556e31803a20 msgr2=0x556e31811650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:43.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.168+0000 7f3e02800640 1 --2- 172.21.15.83:0/2880497773 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556e31803a20 0x556e31811650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:43.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.168+0000 7f3e02800640 1 -- 172.21.15.83:0/2880497773 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556e317f9350 msgr2=0x556e31814a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:43.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.168+0000 7f3e02800640 1 --2- 172.21.15.83:0/2880497773 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556e317f9350 0x556e31814a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:43.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.168+0000 7f3e02800640 1 -- 172.21.15.83:0/2880497773 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e314300f0 con 0x556e317f8a30 2024-04-24T09:00:43.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.168+0000 7f3e02800640 1 --2- 172.21.15.83:0/2880497773 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e317f8a30 0x556e318144d0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x556e3176bcb0 tx=0x7f3dfc005cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcfe0312dd030faa server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:43.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.168+0000 7f3de3e00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:00:43.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.168+0000 7f3de3e00640 1 -- 172.21.15.83:0/2880497773 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dfc037030 con 0x556e317f8a30 2024-04-24T09:00:43.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.168+0000 7f3de3e00640 1 -- 172.21.15.83:0/2880497773 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f3dfc0295d0 con 0x556e317f8a30 2024-04-24T09:00:43.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.168+0000 7f3e0bfc4740 1 -- 172.21.15.83:0/2880497773 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3dfc0297f0 con 0x556e317f8a30 2024-04-24T09:00:43.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.169+0000 7f3de3e00640 1 -- 172.21.15.83:0/2880497773 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f3dfc00ebc0 con 0x556e317f8a30 2024-04-24T09:00:43.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.169+0000 7f3e0bfc4740 10 client.18834 fetch_fsmap learned FSMap version 35 2024-04-24T09:00:43.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.169+0000 7f3e0bfc4740 1 -- 172.21.15.83:0/2880497773 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x556e31723810 con 0x556e317f8a30 2024-04-24T09:00:43.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.169+0000 7f3de3e00640 1 -- 172.21.15.83:0/2880497773 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3dfc0297f0 con 0x556e317f8a30 2024-04-24T09:00:43.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.169+0000 7f3de3e00640 1 -- 172.21.15.83:0/2880497773 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7f3dfc00f040 con 0x556e317f8a30 2024-04-24T09:00:43.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.169+0000 7f3e0bfc4740 10 client.18834 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T09:00:43.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.169+0000 7f3e0bfc4740 10 client.18834 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:00:43.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.169+0000 7f3e0bfc4740 20 client.18834 populate_metadata read hostname 'smithi083' 2024-04-24T09:00:43.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.169+0000 7f3e0bfc4740 1 --2- 172.21.15.83:0/2880497773 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556e318838e0 0x556e31885cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:43.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.169+0000 7f3e0bfc4740 4 client.18834 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T09:00:43.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.169+0000 7f3e0bfc4740 1 -- 172.21.15.83:0/2880497773 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556e31765a80 con 0x556e318838e0 2024-04-24T09:00:43.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.170+0000 7f3e01400640 1 --2- 172.21.15.83:0/2880497773 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556e318838e0 0x556e31885cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:43.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.171+0000 7f3e01400640 1 --2- 172.21.15.83:0/2880497773 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556e318838e0 0x556e31885cd0 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1eaabf003d54da8 server_cookie=7c41275541f7e6ea in_seq=0 out_seq=0 2024-04-24T09:00:43.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.171+0000 7f3de3e00640 10 client.18834 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:00:43.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.175+0000 7f3de3e00640 1 -- 172.21.15.83:0/2880497773 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x556e31765a80 con 0x556e318838e0 2024-04-24T09:00:43.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.175+0000 7f3de3e00640 10 client.18834 handle_command_reply: tid=0 2024-04-24T09:00:43.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.175+0000 7f3e0bfc4740 1 client.18834 shutdown 2024-04-24T09:00:43.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.175+0000 7f3e0bfc4740 1 -- 172.21.15.83:0/2880497773 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e317f8a30 msgr2=0x556e318144d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:43.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.175+0000 7f3e0bfc4740 1 --2- 172.21.15.83:0/2880497773 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e317f8a30 0x556e318144d0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x556e3176bcb0 tx=0x7f3dfc005cf0 comp rx=0 tx=0).stop 2024-04-24T09:00:43.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.176+0000 7f3e0bfc4740 1 -- 172.21.15.83:0/2880497773 shutdown_connections 2024-04-24T09:00:43.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.176+0000 7f3e0bfc4740 1 --2- 172.21.15.83:0/2880497773 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556e318838e0 0x556e31885cd0 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:43.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.176+0000 7f3e0bfc4740 1 --2- 172.21.15.83:0/2880497773 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556e31803a20 0x556e31811650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:43.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.176+0000 7f3e0bfc4740 1 --2- 172.21.15.83:0/2880497773 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556e317f9350 0x556e31814a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:43.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.176+0000 7f3e0bfc4740 1 --2- 172.21.15.83:0/2880497773 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e317f8a30 0x556e318144d0 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:43.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.176+0000 7f3de3e00640 0 client.18834 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:00:43.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.176+0000 7f3e0bfc4740 1 -- 172.21.15.83:0/2880497773 >> 172.21.15.83:0/2880497773 conn(0x556e317f4d50 msgr2=0x556e317faa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:43.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.176+0000 7f3e0bfc4740 1 -- 172.21.15.83:0/2880497773 shutdown_connections 2024-04-24T09:00:43.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.176+0000 7f3e0bfc4740 1 -- 172.21.15.83:0/2880497773 wait complete. 2024-04-24T09:00:43.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.176+0000 7f3e0bfc4740 20 client.18834 trim_cache size 0 max 16384 2024-04-24T09:00:43.177 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T09:00:43.177 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-04-24T09:00:43.177 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-04-24T09:00:43.177 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T09:00:43.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.177+0000 7f3e03200640 1 -- 172.21.15.83:0/604641044 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f3de80a3d20 msgr2=0x7f3de80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:43.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.177+0000 7f3e03200640 1 --2- 172.21.15.83:0/604641044 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f3de80a3d20 0x7f3de80a61e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x556e317f9ff0 tx=0x7f3df4031400 comp rx=0 tx=0).stop 2024-04-24T09:00:43.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.177+0000 7f3e03200640 1 -- 172.21.15.83:0/604641044 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e041003b0 msgr2=0x7f3e041a0610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:43.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.177+0000 7f3e03200640 1 --2- 172.21.15.83:0/604641044 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e041003b0 0x7f3e041a0610 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f3df4005ab0 tx=0x7f3df4003cd0 comp rx=0 tx=0).stop 2024-04-24T09:00:43.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.177+0000 7f3e03200640 1 -- 172.21.15.83:0/604641044 shutdown_connections 2024-04-24T09:00:43.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.177+0000 7f3e03200640 1 --2- 172.21.15.83:0/604641044 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f3de80a3d20 0x7f3de80a61e0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:43.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.177+0000 7f3e03200640 1 --2- 172.21.15.83:0/604641044 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3e0410e8f0 0x7f3e0419a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:43.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.177+0000 7f3e03200640 1 --2- 172.21.15.83:0/604641044 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3e04100cd0 0x7f3e041a0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:43.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.177+0000 7f3e03200640 1 --2- 172.21.15.83:0/604641044 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e041003b0 0x7f3e041a0610 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:43.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.177+0000 7f3e03200640 1 -- 172.21.15.83:0/604641044 >> 172.21.15.83:0/604641044 conn(0x7f3e04075d00 msgr2=0x7f3e0410eeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:43.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.177+0000 7f3e03200640 1 -- 172.21.15.83:0/604641044 shutdown_connections 2024-04-24T09:00:43.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.177+0000 7f3e03200640 1 -- 172.21.15.83:0/604641044 wait complete. 2024-04-24T09:00:43.195 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T09:00:43.195 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T09:00:43.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.400+0000 7fbafee00640 1 -- 172.21.15.83:0/292758792 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbaf8109510 msgr2=0x7fbaf81110e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:43.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.400+0000 7fbafee00640 1 --2- 172.21.15.83:0/292758792 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbaf8109510 0x7fbaf81110e0 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7fbaf0009ca0 tx=0x7fbaf002a8c0 comp rx=0 tx=0).stop 2024-04-24T09:00:43.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.400+0000 7fbafee00640 1 -- 172.21.15.83:0/292758792 shutdown_connections 2024-04-24T09:00:43.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.400+0000 7fbafee00640 1 --2- 172.21.15.83:0/292758792 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbaf8109510 0x7fbaf81110e0 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:43.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.400+0000 7fbafee00640 1 --2- 172.21.15.83:0/292758792 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbaf8104df0 0x7fbaf8108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:43.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.400+0000 7fbafee00640 1 --2- 172.21.15.83:0/292758792 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbaf8104440 0x7fbaf8104820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:43.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.400+0000 7fbafee00640 1 -- 172.21.15.83:0/292758792 >> 172.21.15.83:0/292758792 conn(0x7fbaf80ffcc0 msgr2=0x7fbaf81020e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:43.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.400+0000 7fbafee00640 1 -- 172.21.15.83:0/292758792 shutdown_connections 2024-04-24T09:00:43.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.400+0000 7fbafee00640 1 -- 172.21.15.83:0/292758792 wait complete. 2024-04-24T09:00:43.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.401+0000 7fbafee00640 1 Processor -- start 2024-04-24T09:00:43.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.401+0000 7fbafee00640 1 -- start start 2024-04-24T09:00:43.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.402+0000 7fbafee00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbaf8104440 0x7fbaf81a06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:43.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.402+0000 7fbafee00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbaf8109510 0x7fbaf81a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:43.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.402+0000 7fbafee00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbaf819a8a0 0x7fbaf819ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:43.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.402+0000 7fbafee00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf8114480 con 0x7fbaf8104440 2024-04-24T09:00:43.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.402+0000 7fbafee00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf8114300 con 0x7fbaf819a8a0 2024-04-24T09:00:43.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.402+0000 7fbafee00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf8114600 con 0x7fbaf8109510 2024-04-24T09:00:43.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.402+0000 7fbaf7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbaf8104440 0x7fbaf81a06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:43.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.402+0000 7fbaf7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbaf8104440 0x7fbaf81a06e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48546/0 (socket says 172.21.15.83:48546) 2024-04-24T09:00:43.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.402+0000 7fbaf7e00640 1 -- 172.21.15.83:0/3814176226 learned_addr learned my addr 172.21.15.83:0/3814176226 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:00:43.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.402+0000 7fbaf7400640 1 --2- 172.21.15.83:0/3814176226 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbaf8109510 0x7fbaf81a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:43.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.402+0000 7fbaf7e00640 1 -- 172.21.15.83:0/3814176226 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbaf8109510 msgr2=0x7fbaf81a0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:43.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.403+0000 7fbaf7e00640 1 --2- 172.21.15.83:0/3814176226 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbaf8109510 0x7fbaf81a0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:43.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.402+0000 7fbafce00640 1 --2- 172.21.15.83:0/3814176226 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbaf819a8a0 0x7fbaf819ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:43.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.403+0000 7fbaf7e00640 1 -- 172.21.15.83:0/3814176226 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbaf819a8a0 msgr2=0x7fbaf819ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:43.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.403+0000 7fbaf7e00640 1 --2- 172.21.15.83:0/3814176226 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbaf819a8a0 0x7fbaf819ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:43.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.403+0000 7fbaf7e00640 1 -- 172.21.15.83:0/3814176226 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaf819b430 con 0x7fbaf8104440 2024-04-24T09:00:43.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.403+0000 7fbaf7e00640 1 --2- 172.21.15.83:0/3814176226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbaf8104440 0x7fbaf81a06e0 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7fbae800d950 tx=0x7fbae800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:43.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.403+0000 7fbaf4c00640 1 -- 172.21.15.83:0/3814176226 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbae8017070 con 0x7fbaf8104440 2024-04-24T09:00:43.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.403+0000 7fbaf4c00640 1 -- 172.21.15.83:0/3814176226 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbae8009c30 con 0x7fbaf8104440 2024-04-24T09:00:43.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.403+0000 7fbafee00640 1 -- 172.21.15.83:0/3814176226 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbaf819b720 con 0x7fbaf8104440 2024-04-24T09:00:43.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.403+0000 7fbaf4c00640 1 -- 172.21.15.83:0/3814176226 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbae800f620 con 0x7fbaf8104440 2024-04-24T09:00:43.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.403+0000 7fbafee00640 1 -- 172.21.15.83:0/3814176226 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbaf81a78b0 con 0x7fbaf8104440 2024-04-24T09:00:43.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.405+0000 7fbafee00640 1 -- 172.21.15.83:0/3814176226 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbaf8105910 con 0x7fbaf8104440 2024-04-24T09:00:43.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.407+0000 7fbaf4c00640 1 -- 172.21.15.83:0/3814176226 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fbae80109a0 con 0x7fbaf8104440 2024-04-24T09:00:43.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.408+0000 7fbaf4c00640 1 --2- 172.21.15.83:0/3814176226 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fbadc0a3d20 0x7fbadc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:43.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.408+0000 7fbaf4c00640 1 -- 172.21.15.83:0/3814176226 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7fbae80d0970 con 0x7fbaf8104440 2024-04-24T09:00:43.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.409+0000 7fbaf7400640 1 --2- 172.21.15.83:0/3814176226 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fbadc0a3d20 0x7fbadc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:43.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.409+0000 7fbaf7400640 1 --2- 172.21.15.83:0/3814176226 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fbadc0a3d20 0x7fbadc0a61e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fbae4005fd0 tx=0x7fbae400b510 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:43.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.414+0000 7fbaf4c00640 1 -- 172.21.15.83:0/3814176226 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbaf819b720 con 0x7fbaf8104440 2024-04-24T09:00:43.602 INFO:tasks.workunit.client.0.smithi083.stdout: 1 51798 0.62 MB/sec execute 317 sec latency 64.448 ms 2024-04-24T09:00:43.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.691+0000 7fbafee00640 1 -- 172.21.15.83:0/3814176226 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbaf8069270 con 0x7fbaf8104440 2024-04-24T09:00:43.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.693+0000 7fbaf4c00640 1 -- 172.21.15.83:0/3814176226 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11944 (secure 0 0 0) 0x7fbae8009de0 con 0x7fbaf8104440 2024-04-24T09:00:43.693 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T09:00:43.693 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/358872257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":358872257},{"type":"v1","addr":"172.21.15.83:6835","nonce":358872257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3157070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3157070851},{"type":"v1","addr":"172.21.15.83:6837","nonce":3157070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":35,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T08:48:29.422576+0000","modified":"2024-04-24T08:59:06.888262+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34520,"mds_1":34493,"mds_2":34451,"mds_3":34427,"mds_4":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14658":{"gid":14658,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6839/4085470844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":4085470844},{"type":"v1","addr":"172.21.15.83:6839","nonce":4085470844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14676":{"gid":14676,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6841/2634754210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":2634754210},{"type":"v1","addr":"172.21.15.83:6841","nonce":2634754210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24382":{"gid":24382,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/1859921310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1859921310},{"type":"v1","addr":"172.21.15.189:6833","nonce":1859921310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/2241923197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2241923197},{"type":"v1","addr":"172.21.15.189:6835","nonce":2241923197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.189:6839/2548965336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":2548965336},{"type":"v1","addr":"172.21.15.189:6839","nonce":2548965336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26686":{"gid":26686,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6837/1678817161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":1678817161},{"type":"v1","addr":"172.21.15.189:6837","nonce":1678817161}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.138:6833/1157967658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1157967658},{"type":"v1","addr":"172.21.15.138:6833","nonce":1157967658}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.138:6835/1909219758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1909219758},{"type":"v1","addr":"172.21.15.138:6835","nonce":1909219758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34493":{"gid":34493,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.138:6837/900566754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":900566754},{"type":"v1","addr":"172.21.15.138:6837","nonce":900566754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34520":{"gid":34520,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.138:6839/2236932393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2236932393},{"type":"v1","addr":"172.21.15.138:6839","nonce":2236932393}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34520,"qdb_cluster":[34520,24454,34427,34493,34451]},"id":1}]} 2024-04-24T09:00:43.694 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 35 2024-04-24T09:00:43.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.696+0000 7fbafee00640 1 -- 172.21.15.83:0/3814176226 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fbadc0a3d20 msgr2=0x7fbadc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:43.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.696+0000 7fbafee00640 1 --2- 172.21.15.83:0/3814176226 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fbadc0a3d20 0x7fbadc0a61e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fbae4005fd0 tx=0x7fbae400b510 comp rx=0 tx=0).stop 2024-04-24T09:00:43.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.696+0000 7fbafee00640 1 -- 172.21.15.83:0/3814176226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbaf8104440 msgr2=0x7fbaf81a06e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:43.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.696+0000 7fbafee00640 1 --2- 172.21.15.83:0/3814176226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbaf8104440 0x7fbaf81a06e0 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7fbae800d950 tx=0x7fbae800de10 comp rx=0 tx=0).stop 2024-04-24T09:00:43.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.696+0000 7fbafee00640 1 -- 172.21.15.83:0/3814176226 shutdown_connections 2024-04-24T09:00:43.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.696+0000 7fbafee00640 1 --2- 172.21.15.83:0/3814176226 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fbadc0a3d20 0x7fbadc0a61e0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:43.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.696+0000 7fbafee00640 1 --2- 172.21.15.83:0/3814176226 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbaf819a8a0 0x7fbaf819ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:43.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.696+0000 7fbafee00640 1 --2- 172.21.15.83:0/3814176226 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbaf8109510 0x7fbaf81a0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:43.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.696+0000 7fbafee00640 1 --2- 172.21.15.83:0/3814176226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbaf8104440 0x7fbaf81a06e0 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:43.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.696+0000 7fbafee00640 1 -- 172.21.15.83:0/3814176226 >> 172.21.15.83:0/3814176226 conn(0x7fbaf80ffcc0 msgr2=0x7fbaf8102000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:43.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.696+0000 7fbafee00640 1 -- 172.21.15.83:0/3814176226 shutdown_connections 2024-04-24T09:00:43.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.696+0000 7fbafee00640 1 -- 172.21.15.83:0/3814176226 wait complete. 2024-04-24T09:00:43.716 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-24T09:00:43.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.923+0000 7f4639c00640 1 -- 172.21.15.83:0/4176405492 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f463410e660 msgr2=0x7f463410ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:43.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.923+0000 7f4639c00640 1 --2- 172.21.15.83:0/4176405492 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f463410e660 0x7f463410ea40 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f4624009cd0 tx=0x7f462402a8f0 comp rx=0 tx=0).stop 2024-04-24T09:00:43.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.923+0000 7f4639c00640 1 -- 172.21.15.83:0/4176405492 shutdown_connections 2024-04-24T09:00:43.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.923+0000 7f4639c00640 1 --2- 172.21.15.83:0/4176405492 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4634108fd0 0x7f4634115490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:43.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.923+0000 7f4639c00640 1 --2- 172.21.15.83:0/4176405492 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4634108610 0x7f4634108a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:43.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.923+0000 7f4639c00640 1 --2- 172.21.15.83:0/4176405492 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f463410e660 0x7f463410ea40 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:43.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.923+0000 7f4639c00640 1 -- 172.21.15.83:0/4176405492 >> 172.21.15.83:0/4176405492 conn(0x7f4634076540 msgr2=0x7f4634078960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:43.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.924+0000 7f4639c00640 1 -- 172.21.15.83:0/4176405492 shutdown_connections 2024-04-24T09:00:43.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.924+0000 7f4639c00640 1 -- 172.21.15.83:0/4176405492 wait complete. 2024-04-24T09:00:43.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.925+0000 7f4639c00640 1 Processor -- start 2024-04-24T09:00:43.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.925+0000 7f4639c00640 1 -- start start 2024-04-24T09:00:43.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.925+0000 7f4639c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4634108610 0x7f463419ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:43.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.925+0000 7f4639c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4634108fd0 0x7f463419f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:43.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.925+0000 7f4639c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f463410e660 0x7f46341a3640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:43.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.925+0000 7f4639c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f46341188e0 con 0x7f463410e660 2024-04-24T09:00:43.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.925+0000 7f4639c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4634118760 con 0x7f4634108610 2024-04-24T09:00:43.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.925+0000 7f4639c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4634118a60 con 0x7f4634108fd0 2024-04-24T09:00:43.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.926+0000 7f4633400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f463410e660 0x7f46341a3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:43.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.926+0000 7f4633400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f463410e660 0x7f46341a3640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48566/0 (socket says 172.21.15.83:48566) 2024-04-24T09:00:43.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.926+0000 7f4633400640 1 -- 172.21.15.83:0/2198408005 learned_addr learned my addr 172.21.15.83:0/2198408005 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:00:43.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.926+0000 7f4632a00640 1 --2- 172.21.15.83:0/2198408005 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4634108610 0x7f463419ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:43.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.926+0000 7f4632000640 1 --2- 172.21.15.83:0/2198408005 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4634108fd0 0x7f463419f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:43.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.926+0000 7f4633400640 1 -- 172.21.15.83:0/2198408005 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4634108fd0 msgr2=0x7f463419f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:43.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.926+0000 7f4633400640 1 --2- 172.21.15.83:0/2198408005 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4634108fd0 0x7f463419f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:43.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.926+0000 7f4633400640 1 -- 172.21.15.83:0/2198408005 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4634108610 msgr2=0x7f463419ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:43.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.926+0000 7f4633400640 1 --2- 172.21.15.83:0/2198408005 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4634108610 0x7f463419ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:43.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.926+0000 7f4633400640 1 -- 172.21.15.83:0/2198408005 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46341a3d70 con 0x7f463410e660 2024-04-24T09:00:43.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.927+0000 7f4633400640 1 --2- 172.21.15.83:0/2198408005 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f463410e660 0x7f46341a3640 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f462c00c970 tx=0x7f462c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:43.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.927+0000 7f462b400640 1 -- 172.21.15.83:0/2198408005 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f462c007bb0 con 0x7f463410e660 2024-04-24T09:00:43.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.927+0000 7f462b400640 1 -- 172.21.15.83:0/2198408005 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f462c007d50 con 0x7f463410e660 2024-04-24T09:00:43.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.927+0000 7f4639c00640 1 -- 172.21.15.83:0/2198408005 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46341a4060 con 0x7f463410e660 2024-04-24T09:00:43.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.927+0000 7f462b400640 1 -- 172.21.15.83:0/2198408005 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f462c018700 con 0x7f463410e660 2024-04-24T09:00:43.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.927+0000 7f4639c00640 1 -- 172.21.15.83:0/2198408005 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46341ab8f0 con 0x7f463410e660 2024-04-24T09:00:43.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.931+0000 7f462b400640 1 -- 172.21.15.83:0/2198408005 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f462c00fa20 con 0x7f463410e660 2024-04-24T09:00:43.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.931+0000 7f463cb64740 1 Processor -- start 2024-04-24T09:00:43.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.931+0000 7f463cb64740 1 -- start start 2024-04-24T09:00:43.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.931+0000 7f463cb64740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba2c519950 0x55ba2c519d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:43.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.931+0000 7f463cb64740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba2c51a270 0x55ba2c524400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:43.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.931+0000 7f463cb64740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ba2c524940 0x55ba2c526d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:43.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.931+0000 7f463cb64740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ba2c47f1d0 con 0x55ba2c519950 2024-04-24T09:00:43.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.931+0000 7f463cb64740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ba2c425450 con 0x55ba2c51a270 2024-04-24T09:00:43.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.931+0000 7f463cb64740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55ba2c47b190 con 0x55ba2c524940 2024-04-24T09:00:43.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.932+0000 7f4632000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba2c519950 0x55ba2c519d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:43.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.932+0000 7f4632000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba2c519950 0x55ba2c519d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48574/0 (socket says 172.21.15.83:48574) 2024-04-24T09:00:43.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.932+0000 7f4632000640 1 -- 172.21.15.83:0/3867196447 learned_addr learned my addr 172.21.15.83:0/3867196447 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:00:43.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.932+0000 7f4632a00640 1 --2- 172.21.15.83:0/3867196447 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba2c51a270 0x55ba2c524400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:43.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.932+0000 7f4632000640 1 -- 172.21.15.83:0/3867196447 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ba2c524940 msgr2=0x55ba2c526d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:43.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.932+0000 7f4632000640 1 --2- 172.21.15.83:0/3867196447 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ba2c524940 0x55ba2c526d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:43.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.932+0000 7f4632000640 1 -- 172.21.15.83:0/3867196447 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba2c51a270 msgr2=0x55ba2c524400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:43.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.932+0000 7f4632000640 1 --2- 172.21.15.83:0/3867196447 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba2c51a270 0x55ba2c524400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:43.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.932+0000 7f4632000640 1 -- 172.21.15.83:0/3867196447 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba2c4e2310 con 0x55ba2c519950 2024-04-24T09:00:43.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.932+0000 7f4632000640 1 --2- 172.21.15.83:0/3867196447 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba2c519950 0x55ba2c519d30 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f461c00bd40 tx=0x7f461c011750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4141dcf8111e1139 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:43.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.932+0000 7f462b400640 1 --2- 172.21.15.83:0/2198408005 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f46140a3da0 0x7f46140a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:43.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.933+0000 7f4623e00640 1 -- 172.21.15.83:0/3867196447 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f461c01a400 con 0x55ba2c519950 2024-04-24T09:00:43.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.933+0000 7f462b400640 1 -- 172.21.15.83:0/2198408005 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f462c0d0d70 con 0x7f463410e660 2024-04-24T09:00:43.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.933+0000 7f4632a00640 1 --2- 172.21.15.83:0/2198408005 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f46140a3da0 0x7f46140a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:43.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.933+0000 7f4623e00640 1 -- 172.21.15.83:0/3867196447 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f461c01aa50 con 0x55ba2c519950 2024-04-24T09:00:43.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.933+0000 7f4623e00640 1 -- 172.21.15.83:0/3867196447 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f461c012a30 con 0x55ba2c519950 2024-04-24T09:00:43.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.934+0000 7f463cb64740 1 -- 172.21.15.83:0/3867196447 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba2c519950 msgr2=0x55ba2c519d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:43.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.934+0000 7f463cb64740 1 --2- 172.21.15.83:0/3867196447 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba2c519950 0x55ba2c519d30 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f461c00bd40 tx=0x7f461c011750 comp rx=0 tx=0).stop 2024-04-24T09:00:43.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.934+0000 7f4632a00640 1 --2- 172.21.15.83:0/2198408005 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f46140a3da0 0x7f46140a6260 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4624006010 tx=0x7f46240303a0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:43.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.934+0000 7f463cb64740 1 -- 172.21.15.83:0/3867196447 shutdown_connections 2024-04-24T09:00:43.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.934+0000 7f463cb64740 1 --2- 172.21.15.83:0/3867196447 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ba2c524940 0x55ba2c526d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:43.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.934+0000 7f463cb64740 1 --2- 172.21.15.83:0/3867196447 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba2c51a270 0x55ba2c524400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:43.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.934+0000 7f463cb64740 1 --2- 172.21.15.83:0/3867196447 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba2c519950 0x55ba2c519d30 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:43.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.934+0000 7f463cb64740 1 -- 172.21.15.83:0/3867196447 >> 172.21.15.83:0/3867196447 conn(0x55ba2c515c70 msgr2=0x55ba2c516050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:43.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.934+0000 7f463cb64740 1 -- 172.21.15.83:0/3867196447 shutdown_connections 2024-04-24T09:00:43.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.934+0000 7f463cb64740 1 -- 172.21.15.83:0/3867196447 wait complete. 2024-04-24T09:00:43.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.934+0000 7f463cb64740 1 -- start start 2024-04-24T09:00:43.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.935+0000 7f463cb64740 1 Processor -- start 2024-04-24T09:00:43.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.935+0000 7f463cb64740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba2c519950 0x55ba2c5243a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:43.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.935+0000 7f463cb64740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ba2c51a270 0x55ba2c521690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:43.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.935+0000 7f463cb64740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba2c524940 0x55ba2c521db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:43.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.935+0000 7f463cb64740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ba2c5277d0 con 0x55ba2c519950 2024-04-24T09:00:43.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.935+0000 7f463cb64740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ba2c438810 con 0x55ba2c524940 2024-04-24T09:00:43.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.935+0000 7f463cb64740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55ba2c527950 con 0x55ba2c51a270 2024-04-24T09:00:43.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.935+0000 7f463cb64740 10 client.0 _ll_register_callbacks cb 0x55ba2c406140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:00:43.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.935+0000 7f4632a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba2c519950 0x55ba2c5243a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:43.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.935+0000 7f4632a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba2c519950 0x55ba2c5243a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48586/0 (socket says 172.21.15.83:48586) 2024-04-24T09:00:43.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.935+0000 7f4632a00640 1 -- 172.21.15.83:0/605762531 learned_addr learned my addr 172.21.15.83:0/605762531 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:00:43.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.936+0000 7f4632000640 1 --2- 172.21.15.83:0/605762531 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ba2c51a270 0x55ba2c521690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:43.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.936+0000 7f4633400640 1 --2- 172.21.15.83:0/605762531 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba2c524940 0x55ba2c521db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:43.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.936+0000 7f4632a00640 1 -- 172.21.15.83:0/605762531 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ba2c51a270 msgr2=0x55ba2c521690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:43.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.936+0000 7f4632a00640 1 --2- 172.21.15.83:0/605762531 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ba2c51a270 0x55ba2c521690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:43.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.936+0000 7f4632a00640 1 -- 172.21.15.83:0/605762531 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba2c524940 msgr2=0x55ba2c521db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:43.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.936+0000 7f4632a00640 1 --2- 172.21.15.83:0/605762531 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba2c524940 0x55ba2c521db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:43.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.936+0000 7f4632a00640 1 -- 172.21.15.83:0/605762531 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba2c4493f0 con 0x55ba2c519950 2024-04-24T09:00:43.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.936+0000 7f4632a00640 1 --2- 172.21.15.83:0/605762531 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba2c519950 0x55ba2c5243a0 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f4624009a60 tx=0x7f46240095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad3d78b7088d48ca server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:43.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.936+0000 7f4620c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:00:43.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.937+0000 7f4620c00640 1 -- 172.21.15.83:0/605762531 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4624004250 con 0x55ba2c519950 2024-04-24T09:00:43.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.937+0000 7f4620c00640 1 -- 172.21.15.83:0/605762531 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f46240043f0 con 0x55ba2c519950 2024-04-24T09:00:43.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.937+0000 7f463cb64740 1 -- 172.21.15.83:0/605762531 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f461c01aa50 con 0x55ba2c519950 2024-04-24T09:00:43.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.937+0000 7f4620c00640 1 -- 172.21.15.83:0/605762531 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f46240045f0 con 0x55ba2c519950 2024-04-24T09:00:43.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.937+0000 7f463cb64740 10 client.18852 fetch_fsmap learned FSMap version 35 2024-04-24T09:00:43.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.937+0000 7f463cb64740 1 -- 172.21.15.83:0/605762531 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55ba2c1913f0 con 0x55ba2c519950 2024-04-24T09:00:43.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.937+0000 7f4620c00640 1 -- 172.21.15.83:0/605762531 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f461c01aa50 con 0x55ba2c519950 2024-04-24T09:00:43.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.937+0000 7f4620c00640 1 -- 172.21.15.83:0/605762531 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7f462404e070 con 0x55ba2c519950 2024-04-24T09:00:43.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.938+0000 7f463cb64740 10 client.18852 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T09:00:43.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.938+0000 7f463cb64740 10 client.18852 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-24T09:00:43.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.938+0000 7f463cb64740 20 client.18852 populate_metadata read hostname 'smithi083' 2024-04-24T09:00:43.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.938+0000 7f463cb64740 1 --2- 172.21.15.83:0/605762531 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55ba2c534a60 0x55ba2c536f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:43.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.938+0000 7f463cb64740 4 client.18852 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:00:43.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.938+0000 7f463cb64740 1 -- 172.21.15.83:0/605762531 --> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ba2c48cbf0 con 0x55ba2c534a60 2024-04-24T09:00:43.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.938+0000 7f4632000640 1 --2- 172.21.15.83:0/605762531 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55ba2c534a60 0x55ba2c536f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:43.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.939+0000 7f4632000640 1 --2- 172.21.15.83:0/605762531 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55ba2c534a60 0x55ba2c536f00 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f07431d8f71d018 server_cookie=5c97f0f3043964d1 in_seq=0 out_seq=0 2024-04-24T09:00:43.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.939+0000 7f4620c00640 10 client.18852 ms_handle_connect on v2:172.21.15.189:6838/2548965336 2024-04-24T09:00:43.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.941+0000 7f4620c00640 1 -- 172.21.15.83:0/605762531 <== mds.4 v2:172.21.15.189:6838/2548965336 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ba2c48cbf0 con 0x55ba2c534a60 2024-04-24T09:00:43.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.941+0000 7f4620c00640 10 client.18852 handle_command_reply: tid=0 2024-04-24T09:00:43.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.941+0000 7f463cb64740 1 client.18852 shutdown 2024-04-24T09:00:43.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.941+0000 7f463cb64740 1 -- 172.21.15.83:0/605762531 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba2c519950 msgr2=0x55ba2c5243a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:43.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.942+0000 7f463cb64740 1 --2- 172.21.15.83:0/605762531 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba2c519950 0x55ba2c5243a0 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f4624009a60 tx=0x7f46240095c0 comp rx=0 tx=0).stop 2024-04-24T09:00:43.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.942+0000 7f463cb64740 1 -- 172.21.15.83:0/605762531 shutdown_connections 2024-04-24T09:00:43.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.942+0000 7f463cb64740 1 --2- 172.21.15.83:0/605762531 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55ba2c534a60 0x55ba2c536f00 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:43.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.942+0000 7f463cb64740 1 --2- 172.21.15.83:0/605762531 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba2c524940 0x55ba2c521db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:43.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.942+0000 7f463cb64740 1 --2- 172.21.15.83:0/605762531 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ba2c51a270 0x55ba2c521690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:43.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.942+0000 7f4620c00640 0 client.18852 ms_handle_reset on v2:172.21.15.189:6838/2548965336 2024-04-24T09:00:43.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.942+0000 7f463cb64740 1 --2- 172.21.15.83:0/605762531 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba2c519950 0x55ba2c5243a0 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:43.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.942+0000 7f463cb64740 1 -- 172.21.15.83:0/605762531 >> 172.21.15.83:0/605762531 conn(0x55ba2c515c70 msgr2=0x55ba2c51b970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:43.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.942+0000 7f463cb64740 1 -- 172.21.15.83:0/605762531 shutdown_connections 2024-04-24T09:00:43.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.942+0000 7f463cb64740 1 -- 172.21.15.83:0/605762531 wait complete. 2024-04-24T09:00:43.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.942+0000 7f463cb64740 20 client.18852 trim_cache size 0 max 16384 2024-04-24T09:00:43.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.959+0000 7f463cb64740 1 Processor -- start 2024-04-24T09:00:43.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.959+0000 7f463cb64740 1 -- start start 2024-04-24T09:00:43.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.959+0000 7f463cb64740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba2c519950 0x55ba2c51c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:43.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.959+0000 7f463cb64740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba2c51a270 0x55ba2c514c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:43.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.959+0000 7f463cb64740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ba2c524940 0x55ba2c523ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:43.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.959+0000 7f463cb64740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f46341b5b40 con 0x55ba2c519950 2024-04-24T09:00:43.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.959+0000 7f463cb64740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f46341b7680 con 0x55ba2c51a270 2024-04-24T09:00:43.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.959+0000 7f463cb64740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f46341b6210 con 0x55ba2c524940 2024-04-24T09:00:43.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.959+0000 7f4632a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba2c519950 0x55ba2c51c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:43.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.960+0000 7f4632a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba2c519950 0x55ba2c51c0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48598/0 (socket says 172.21.15.83:48598) 2024-04-24T09:00:43.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.960+0000 7f4632a00640 1 -- 172.21.15.83:0/3552643363 learned_addr learned my addr 172.21.15.83:0/3552643363 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:00:43.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.960+0000 7f4633400640 1 --2- 172.21.15.83:0/3552643363 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ba2c524940 0x55ba2c523ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:43.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.960+0000 7f4632000640 1 --2- 172.21.15.83:0/3552643363 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba2c51a270 0x55ba2c514c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:43.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.960+0000 7f4632a00640 1 -- 172.21.15.83:0/3552643363 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ba2c524940 msgr2=0x55ba2c523ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:43.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.960+0000 7f4632a00640 1 --2- 172.21.15.83:0/3552643363 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ba2c524940 0x55ba2c523ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:43.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.960+0000 7f4632a00640 1 -- 172.21.15.83:0/3552643363 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba2c51a270 msgr2=0x55ba2c514c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:43.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.960+0000 7f4632a00640 1 --2- 172.21.15.83:0/3552643363 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba2c51a270 0x55ba2c514c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:43.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.960+0000 7f4632a00640 1 -- 172.21.15.83:0/3552643363 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba2c51a0a0 con 0x55ba2c519950 2024-04-24T09:00:43.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.960+0000 7f4632000640 1 --2- 172.21.15.83:0/3552643363 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba2c51a270 0x55ba2c514c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:00:43.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.960+0000 7f4632a00640 1 --2- 172.21.15.83:0/3552643363 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba2c519950 0x55ba2c51c0b0 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f4624050610 tx=0x7f46240504c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=131ea727b6661ed4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:43.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.961+0000 7f4623400640 1 -- 172.21.15.83:0/3552643363 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4624004250 con 0x55ba2c519950 2024-04-24T09:00:43.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.961+0000 7f4623400640 1 -- 172.21.15.83:0/3552643363 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4624036070 con 0x55ba2c519950 2024-04-24T09:00:43.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.961+0000 7f4623400640 1 -- 172.21.15.83:0/3552643363 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4624005040 con 0x55ba2c519950 2024-04-24T09:00:43.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.961+0000 7f463cb64740 1 -- 172.21.15.83:0/3552643363 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba2c519950 msgr2=0x55ba2c51c0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:43.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.961+0000 7f463cb64740 1 --2- 172.21.15.83:0/3552643363 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba2c519950 0x55ba2c51c0b0 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f4624050610 tx=0x7f46240504c0 comp rx=0 tx=0).stop 2024-04-24T09:00:43.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.961+0000 7f463cb64740 1 -- 172.21.15.83:0/3552643363 shutdown_connections 2024-04-24T09:00:43.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.961+0000 7f463cb64740 1 --2- 172.21.15.83:0/3552643363 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ba2c524940 0x55ba2c523ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:43.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.961+0000 7f463cb64740 1 --2- 172.21.15.83:0/3552643363 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba2c51a270 0x55ba2c514c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:43.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.961+0000 7f463cb64740 1 --2- 172.21.15.83:0/3552643363 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba2c519950 0x55ba2c51c0b0 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:43.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.961+0000 7f463cb64740 1 -- 172.21.15.83:0/3552643363 >> 172.21.15.83:0/3552643363 conn(0x55ba2c515c70 msgr2=0x55ba2c514560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:43.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.962+0000 7f463cb64740 1 -- 172.21.15.83:0/3552643363 shutdown_connections 2024-04-24T09:00:43.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.962+0000 7f463cb64740 1 -- 172.21.15.83:0/3552643363 wait complete. 2024-04-24T09:00:43.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.962+0000 7f463cb64740 1 -- start start 2024-04-24T09:00:43.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.962+0000 7f463cb64740 1 Processor -- start 2024-04-24T09:00:43.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.963+0000 7f463cb64740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba2c519950 0x55ba2c525850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:43.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.963+0000 7f463cb64740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba2c51a270 0x55ba2c525d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:43.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.963+0000 7f463cb64740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ba2c524940 0x55ba2c5262d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:43.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.963+0000 7f463cb64740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f46341b6fb0 con 0x55ba2c51a270 2024-04-24T09:00:43.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.963+0000 7f463cb64740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f46341b68e0 con 0x55ba2c519950 2024-04-24T09:00:43.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.963+0000 7f463cb64740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f46341b3b40 con 0x55ba2c524940 2024-04-24T09:00:43.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.963+0000 7f463cb64740 10 client.0 _ll_register_callbacks cb 0x7f4624009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:00:43.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.963+0000 7f4632000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba2c51a270 0x55ba2c525d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:43.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.963+0000 7f4632000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba2c51a270 0x55ba2c525d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48606/0 (socket says 172.21.15.83:48606) 2024-04-24T09:00:43.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.963+0000 7f4632000640 1 -- 172.21.15.83:0/897404574 learned_addr learned my addr 172.21.15.83:0/897404574 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:00:43.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.963+0000 7f4632a00640 1 --2- 172.21.15.83:0/897404574 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba2c519950 0x55ba2c525850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:43.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.963+0000 7f4633400640 1 --2- 172.21.15.83:0/897404574 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ba2c524940 0x55ba2c5262d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:43.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.963+0000 7f4632000640 1 -- 172.21.15.83:0/897404574 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ba2c524940 msgr2=0x55ba2c5262d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:43.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.963+0000 7f4632000640 1 --2- 172.21.15.83:0/897404574 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ba2c524940 0x55ba2c5262d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:43.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.963+0000 7f4632000640 1 -- 172.21.15.83:0/897404574 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba2c519950 msgr2=0x55ba2c525850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:43.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.963+0000 7f4632000640 1 --2- 172.21.15.83:0/897404574 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba2c519950 0x55ba2c525850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:43.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.963+0000 7f4632000640 1 -- 172.21.15.83:0/897404574 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba2c25b7d0 con 0x55ba2c51a270 2024-04-24T09:00:43.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.963+0000 7f4633400640 1 --2- 172.21.15.83:0/897404574 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ba2c524940 0x55ba2c5262d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:00:43.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.964+0000 7f4632000640 1 --2- 172.21.15.83:0/897404574 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba2c51a270 0x55ba2c525d90 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f461c00bce0 tx=0x7f461c00b7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cad2601f4803067e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:43.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.964+0000 7f4620c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:00:43.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.964+0000 7f4620c00640 1 -- 172.21.15.83:0/897404574 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f461c029020 con 0x55ba2c51a270 2024-04-24T09:00:43.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.964+0000 7f4620c00640 1 -- 172.21.15.83:0/897404574 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f461c005440 con 0x55ba2c51a270 2024-04-24T09:00:43.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.964+0000 7f463cb64740 1 -- 172.21.15.83:0/897404574 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4624036070 con 0x55ba2c51a270 2024-04-24T09:00:43.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.964+0000 7f4620c00640 1 -- 172.21.15.83:0/897404574 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f461c035050 con 0x55ba2c51a270 2024-04-24T09:00:43.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.964+0000 7f463cb64740 10 client.18858 fetch_fsmap learned FSMap version 35 2024-04-24T09:00:43.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.964+0000 7f463cb64740 1 -- 172.21.15.83:0/897404574 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55ba2c59b2f0 con 0x55ba2c51a270 2024-04-24T09:00:43.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.965+0000 7f4620c00640 1 -- 172.21.15.83:0/897404574 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4624036070 con 0x55ba2c51a270 2024-04-24T09:00:43.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.965+0000 7f4620c00640 1 -- 172.21.15.83:0/897404574 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7f461c029350 con 0x55ba2c51a270 2024-04-24T09:00:43.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.965+0000 7f463cb64740 10 client.18858 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T09:00:43.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.965+0000 7f463cb64740 10 client.18858 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-24T09:00:43.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.965+0000 7f463cb64740 20 client.18858 populate_metadata read hostname 'smithi083' 2024-04-24T09:00:43.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.965+0000 7f463cb64740 1 --2- 172.21.15.83:0/897404574 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55ba2c59b840 0x55ba2c59dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:43.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.965+0000 7f463cb64740 4 client.18858 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:00:43.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.965+0000 7f463cb64740 1 -- 172.21.15.83:0/897404574 --> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ba2c486a60 con 0x55ba2c59b840 2024-04-24T09:00:43.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.966+0000 7f4632a00640 1 --2- 172.21.15.83:0/897404574 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55ba2c59b840 0x55ba2c59dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:43.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.967+0000 7f4632a00640 1 --2- 172.21.15.83:0/897404574 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55ba2c59b840 0x55ba2c59dc60 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=da4645840de6148a server_cookie=b7c9aea354acef79 in_seq=0 out_seq=0 2024-04-24T09:00:43.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.967+0000 7f4620c00640 10 client.18858 ms_handle_connect on v2:172.21.15.189:6838/2548965336 2024-04-24T09:00:43.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.967+0000 7f4620c00640 1 -- 172.21.15.83:0/897404574 <== mds.4 v2:172.21.15.189:6838/2548965336 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ba2c486a60 con 0x55ba2c59b840 2024-04-24T09:00:43.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.967+0000 7f4620c00640 10 client.18858 handle_command_reply: tid=0 2024-04-24T09:00:43.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.967+0000 7f463cb64740 1 client.18858 shutdown 2024-04-24T09:00:43.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.968+0000 7f463cb64740 1 -- 172.21.15.83:0/897404574 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba2c51a270 msgr2=0x55ba2c525d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:43.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.968+0000 7f463cb64740 1 --2- 172.21.15.83:0/897404574 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba2c51a270 0x55ba2c525d90 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f461c00bce0 tx=0x7f461c00b7d0 comp rx=0 tx=0).stop 2024-04-24T09:00:43.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.968+0000 7f463cb64740 1 -- 172.21.15.83:0/897404574 shutdown_connections 2024-04-24T09:00:43.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.968+0000 7f463cb64740 1 --2- 172.21.15.83:0/897404574 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55ba2c59b840 0x55ba2c59dc60 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:43.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.968+0000 7f463cb64740 1 --2- 172.21.15.83:0/897404574 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ba2c524940 0x55ba2c5262d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:43.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.968+0000 7f4620c00640 0 client.18858 ms_handle_reset on v2:172.21.15.189:6838/2548965336 2024-04-24T09:00:43.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.968+0000 7f463cb64740 1 --2- 172.21.15.83:0/897404574 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba2c51a270 0x55ba2c525d90 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:43.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.968+0000 7f463cb64740 1 --2- 172.21.15.83:0/897404574 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba2c519950 0x55ba2c525850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:43.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.968+0000 7f463cb64740 1 -- 172.21.15.83:0/897404574 >> 172.21.15.83:0/897404574 conn(0x55ba2c515c70 msgr2=0x55ba2c514df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:43.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.968+0000 7f463cb64740 1 -- 172.21.15.83:0/897404574 shutdown_connections 2024-04-24T09:00:43.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.968+0000 7f463cb64740 1 -- 172.21.15.83:0/897404574 wait complete. 2024-04-24T09:00:43.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.968+0000 7f463cb64740 20 client.18858 trim_cache size 0 max 16384 2024-04-24T09:00:43.971 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:00:43.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.969+0000 7f4639c00640 1 -- 172.21.15.83:0/2198408005 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f46140a3da0 msgr2=0x7f46140a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:43.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.969+0000 7f4639c00640 1 --2- 172.21.15.83:0/2198408005 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f46140a3da0 0x7f46140a6260 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4624006010 tx=0x7f46240303a0 comp rx=0 tx=0).stop 2024-04-24T09:00:43.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.969+0000 7f4639c00640 1 -- 172.21.15.83:0/2198408005 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f463410e660 msgr2=0x7f46341a3640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:43.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.969+0000 7f4639c00640 1 --2- 172.21.15.83:0/2198408005 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f463410e660 0x7f46341a3640 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f462c00c970 tx=0x7f462c00ce30 comp rx=0 tx=0).stop 2024-04-24T09:00:43.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.969+0000 7f4639c00640 1 -- 172.21.15.83:0/2198408005 shutdown_connections 2024-04-24T09:00:43.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.969+0000 7f4639c00640 1 --2- 172.21.15.83:0/2198408005 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f46140a3da0 0x7f46140a6260 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:43.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.969+0000 7f4639c00640 1 --2- 172.21.15.83:0/2198408005 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f463410e660 0x7f46341a3640 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:43.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.969+0000 7f4639c00640 1 --2- 172.21.15.83:0/2198408005 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4634108fd0 0x7f463419f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:43.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.969+0000 7f4639c00640 1 --2- 172.21.15.83:0/2198408005 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4634108610 0x7f463419ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:43.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.969+0000 7f4639c00640 1 -- 172.21.15.83:0/2198408005 >> 172.21.15.83:0/2198408005 conn(0x7f4634076540 msgr2=0x7f4634077cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:43.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.969+0000 7f4639c00640 1 -- 172.21.15.83:0/2198408005 shutdown_connections 2024-04-24T09:00:43.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:43.969+0000 7f4639c00640 1 -- 172.21.15.83:0/2198408005 wait complete. 2024-04-24T09:00:43.984 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:00:43 smithi083 ceph-mon[27907]: pgmap v284: 129 pgs: 129 active+clean; 51 MiB data, 718 MiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 373 KiB/s wr, 73 op/s 2024-04-24T09:00:43.984 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:00:43 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/3814176226' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:00:43.986 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-24T09:00:44.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.186+0000 7fc03a000640 1 -- 172.21.15.83:0/1032992298 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc034103280 msgr2=0x7fc034103660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:44.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.186+0000 7fc03a000640 1 --2- 172.21.15.83:0/1032992298 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc034103280 0x7fc034103660 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7fc024009900 tx=0x7fc02402a510 comp rx=0 tx=0).stop 2024-04-24T09:00:44.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.186+0000 7fc03a000640 1 -- 172.21.15.83:0/1032992298 shutdown_connections 2024-04-24T09:00:44.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.186+0000 7fc03a000640 1 --2- 172.21.15.83:0/1032992298 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc03410c5c0 0x7fc03410ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:44.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.186+0000 7fc03a000640 1 --2- 172.21.15.83:0/1032992298 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc034103ba0 0x7fc03410c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:44.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.186+0000 7fc03a000640 1 --2- 172.21.15.83:0/1032992298 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc034103280 0x7fc034103660 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:44.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.186+0000 7fc03a000640 1 -- 172.21.15.83:0/1032992298 >> 172.21.15.83:0/1032992298 conn(0x7fc034078000 msgr2=0x7fc034078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:44.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.186+0000 7fc03a000640 1 -- 172.21.15.83:0/1032992298 shutdown_connections 2024-04-24T09:00:44.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.186+0000 7fc03a000640 1 -- 172.21.15.83:0/1032992298 wait complete. 2024-04-24T09:00:44.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.187+0000 7fc03a000640 1 Processor -- start 2024-04-24T09:00:44.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.188+0000 7fc03a000640 1 -- start start 2024-04-24T09:00:44.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.188+0000 7fc03a000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc034103280 0x7fc0341a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:44.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.188+0000 7fc03a000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc034103ba0 0x7fc0341a0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:44.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.188+0000 7fc03a000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc03410c5c0 0x7fc03419a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:44.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.188+0000 7fc03a000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc0341122a0 con 0x7fc034103280 2024-04-24T09:00:44.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.188+0000 7fc03a000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc034112120 con 0x7fc034103ba0 2024-04-24T09:00:44.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.188+0000 7fc03a000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc034112420 con 0x7fc03410c5c0 2024-04-24T09:00:44.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.188+0000 7fc033400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc034103280 0x7fc0341a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:44.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.188+0000 7fc033400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc034103280 0x7fc0341a0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48614/0 (socket says 172.21.15.83:48614) 2024-04-24T09:00:44.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.188+0000 7fc033400640 1 -- 172.21.15.83:0/3434308585 learned_addr learned my addr 172.21.15.83:0/3434308585 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:00:44.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.189+0000 7fc02aa00640 1 --2- 172.21.15.83:0/3434308585 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc034103ba0 0x7fc0341a0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:44.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.189+0000 7fc033400640 1 -- 172.21.15.83:0/3434308585 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc03410c5c0 msgr2=0x7fc03419a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:44.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.189+0000 7fc033400640 1 --2- 172.21.15.83:0/3434308585 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc03410c5c0 0x7fc03419a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:44.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.189+0000 7fc033400640 1 -- 172.21.15.83:0/3434308585 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc034103ba0 msgr2=0x7fc0341a0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:44.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.189+0000 7fc033400640 1 --2- 172.21.15.83:0/3434308585 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc034103ba0 0x7fc0341a0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:44.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.189+0000 7fc033400640 1 -- 172.21.15.83:0/3434308585 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc03419af80 con 0x7fc034103280 2024-04-24T09:00:44.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.189+0000 7fc02aa00640 1 --2- 172.21.15.83:0/3434308585 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc034103ba0 0x7fc0341a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:00:44.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.189+0000 7fc033400640 1 --2- 172.21.15.83:0/3434308585 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc034103280 0x7fc0341a0640 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7fc024005ab0 tx=0x7fc024003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:44.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.189+0000 7fc030c00640 1 -- 172.21.15.83:0/3434308585 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc02403f070 con 0x7fc034103280 2024-04-24T09:00:44.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.189+0000 7fc030c00640 1 -- 172.21.15.83:0/3434308585 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc024031d60 con 0x7fc034103280 2024-04-24T09:00:44.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.189+0000 7fc03a000640 1 -- 172.21.15.83:0/3434308585 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc03419b210 con 0x7fc034103280 2024-04-24T09:00:44.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.189+0000 7fc030c00640 1 -- 172.21.15.83:0/3434308585 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc024035030 con 0x7fc034103280 2024-04-24T09:00:44.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.189+0000 7fc03a000640 1 -- 172.21.15.83:0/3434308585 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0341a7490 con 0x7fc034103280 2024-04-24T09:00:44.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.193+0000 7fc030c00640 1 -- 172.21.15.83:0/3434308585 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc024042070 con 0x7fc034103280 2024-04-24T09:00:44.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.193+0000 7fc03ce5b740 1 Processor -- start 2024-04-24T09:00:44.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.193+0000 7fc03ce5b740 1 -- start start 2024-04-24T09:00:44.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.193+0000 7fc03ce5b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f958862950 0x55f958862d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:44.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.193+0000 7fc03ce5b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f958863270 0x55f95886d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:44.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.193+0000 7fc03ce5b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f95886d940 0x55f95886fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:44.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.193+0000 7fc03ce5b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55f9587c81d0 con 0x55f95886d940 2024-04-24T09:00:44.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.193+0000 7fc03ce5b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f95876e450 con 0x55f958862950 2024-04-24T09:00:44.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.193+0000 7fc03ce5b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55f9587c4190 con 0x55f958863270 2024-04-24T09:00:44.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.194+0000 7fc02aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f95886d940 0x55f95886fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:44.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.194+0000 7fc02aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f95886d940 0x55f95886fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48616/0 (socket says 172.21.15.83:48616) 2024-04-24T09:00:44.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.194+0000 7fc02aa00640 1 -- 172.21.15.83:0/1910647989 learned_addr learned my addr 172.21.15.83:0/1910647989 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:00:44.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.194+0000 7fc033400640 1 --2- 172.21.15.83:0/1910647989 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f958863270 0x55f95886d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:44.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.194+0000 7fc033e00640 1 --2- 172.21.15.83:0/1910647989 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f958862950 0x55f958862d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:44.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.194+0000 7fc02aa00640 1 -- 172.21.15.83:0/1910647989 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f958863270 msgr2=0x55f95886d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:44.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.194+0000 7fc02aa00640 1 --2- 172.21.15.83:0/1910647989 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f958863270 0x55f95886d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:44.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.194+0000 7fc02aa00640 1 -- 172.21.15.83:0/1910647989 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f958862950 msgr2=0x55f958862d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:44.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.194+0000 7fc02aa00640 1 --2- 172.21.15.83:0/1910647989 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f958862950 0x55f958862d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:44.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.194+0000 7fc02aa00640 1 -- 172.21.15.83:0/1910647989 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f95882b310 con 0x55f95886d940 2024-04-24T09:00:44.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.195+0000 7fc02aa00640 1 --2- 172.21.15.83:0/1910647989 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f95886d940 0x55f95886fd30 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7fc01c008eb0 tx=0x7fc01c00d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e28766fcc4dab4fc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:44.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.195+0000 7fc030c00640 1 --2- 172.21.15.83:0/3434308585 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fc0180a3da0 0x7fc0180a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:44.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.195+0000 7fc030c00640 1 -- 172.21.15.83:0/3434308585 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7fc0240f15b0 con 0x7fc034103280 2024-04-24T09:00:44.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.195+0000 7fc023e00640 1 -- 172.21.15.83:0/1910647989 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc01c00f830 con 0x55f95886d940 2024-04-24T09:00:44.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.195+0000 7fc033e00640 1 --2- 172.21.15.83:0/3434308585 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fc0180a3da0 0x7fc0180a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:44.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.195+0000 7fc023e00640 1 -- 172.21.15.83:0/1910647989 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc01c018480 con 0x55f95886d940 2024-04-24T09:00:44.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.195+0000 7fc023e00640 1 -- 172.21.15.83:0/1910647989 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc01c00e610 con 0x55f95886d940 2024-04-24T09:00:44.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.196+0000 7fc03ce5b740 1 -- 172.21.15.83:0/1910647989 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f95886d940 msgr2=0x55f95886fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:44.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.196+0000 7fc033e00640 1 --2- 172.21.15.83:0/3434308585 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fc0180a3da0 0x7fc0180a6260 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc02c0058c0 tx=0x7fc02c005eb0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:44.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.196+0000 7fc03ce5b740 1 --2- 172.21.15.83:0/1910647989 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f95886d940 0x55f95886fd30 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7fc01c008eb0 tx=0x7fc01c00d710 comp rx=0 tx=0).stop 2024-04-24T09:00:44.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.196+0000 7fc03ce5b740 1 -- 172.21.15.83:0/1910647989 shutdown_connections 2024-04-24T09:00:44.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.196+0000 7fc03ce5b740 1 --2- 172.21.15.83:0/1910647989 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f95886d940 0x55f95886fd30 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:44.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.196+0000 7fc03ce5b740 1 --2- 172.21.15.83:0/1910647989 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f958863270 0x55f95886d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:44.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.196+0000 7fc03ce5b740 1 --2- 172.21.15.83:0/1910647989 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f958862950 0x55f958862d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:44.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.196+0000 7fc03ce5b740 1 -- 172.21.15.83:0/1910647989 >> 172.21.15.83:0/1910647989 conn(0x55f95885ec70 msgr2=0x55f95885f050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:44.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.196+0000 7fc03ce5b740 1 -- 172.21.15.83:0/1910647989 shutdown_connections 2024-04-24T09:00:44.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.196+0000 7fc03ce5b740 1 -- 172.21.15.83:0/1910647989 wait complete. 2024-04-24T09:00:44.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.197+0000 7fc03ce5b740 1 -- start start 2024-04-24T09:00:44.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.197+0000 7fc03ce5b740 1 Processor -- start 2024-04-24T09:00:44.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.197+0000 7fc03ce5b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f958862950 0x55f95886d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:44.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.197+0000 7fc03ce5b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f958863270 0x55f95886a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:44.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.197+0000 7fc03ce5b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f95886d940 0x55f95886ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:44.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.197+0000 7fc03ce5b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55f9588707d0 con 0x55f958862950 2024-04-24T09:00:44.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.197+0000 7fc03ce5b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f958781810 con 0x55f958863270 2024-04-24T09:00:44.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.197+0000 7fc03ce5b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55f958870950 con 0x55f95886d940 2024-04-24T09:00:44.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.198+0000 7fc033400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f958862950 0x55f95886d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:44.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.197+0000 7fc03ce5b740 10 client.0 _ll_register_callbacks cb 0x55f95874f140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:00:44.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.198+0000 7fc033400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f958862950 0x55f95886d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48624/0 (socket says 172.21.15.83:48624) 2024-04-24T09:00:44.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.198+0000 7fc033400640 1 -- 172.21.15.83:0/4285427041 learned_addr learned my addr 172.21.15.83:0/4285427041 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:00:44.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.198+0000 7fc033400640 1 -- 172.21.15.83:0/4285427041 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f95886d940 msgr2=0x55f95886ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:44.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.198+0000 7fc033400640 1 --2- 172.21.15.83:0/4285427041 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f95886d940 0x55f95886ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:44.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.198+0000 7fc02aa00640 1 --2- 172.21.15.83:0/4285427041 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f958863270 0x55f95886a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:44.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.198+0000 7fc033400640 1 -- 172.21.15.83:0/4285427041 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f958863270 msgr2=0x55f95886a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:44.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.198+0000 7fc033400640 1 --2- 172.21.15.83:0/4285427041 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f958863270 0x55f95886a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:44.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.198+0000 7fc033400640 1 -- 172.21.15.83:0/4285427041 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9587923f0 con 0x55f958862950 2024-04-24T09:00:44.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.198+0000 7fc033400640 1 --2- 172.21.15.83:0/4285427041 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f958862950 0x55f95886d340 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7fc024005ec0 tx=0x7fc024009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21e354982000e1ff server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:44.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.198+0000 7fc020c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:00:44.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.198+0000 7fc020c00640 1 -- 172.21.15.83:0/4285427041 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc024049070 con 0x55f958862950 2024-04-24T09:00:44.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.198+0000 7fc020c00640 1 -- 172.21.15.83:0/4285427041 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7fc0240eeb00 con 0x55f958862950 2024-04-24T09:00:44.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.198+0000 7fc03ce5b740 1 -- 172.21.15.83:0/4285427041 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc01c018480 con 0x55f958862950 2024-04-24T09:00:44.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.199+0000 7fc020c00640 1 -- 172.21.15.83:0/4285427041 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fc0240bca30 con 0x55f958862950 2024-04-24T09:00:44.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.199+0000 7fc03ce5b740 10 client.18870 fetch_fsmap learned FSMap version 35 2024-04-24T09:00:44.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.199+0000 7fc03ce5b740 1 -- 172.21.15.83:0/4285427041 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55f9584da3f0 con 0x55f958862950 2024-04-24T09:00:44.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.199+0000 7fc020c00640 1 -- 172.21.15.83:0/4285427041 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc01c018480 con 0x55f958862950 2024-04-24T09:00:44.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.199+0000 7fc020c00640 1 -- 172.21.15.83:0/4285427041 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7fc0240c58b0 con 0x55f958862950 2024-04-24T09:00:44.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.199+0000 7fc03ce5b740 10 client.18870 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T09:00:44.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.199+0000 7fc03ce5b740 10 client.18870 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-24T09:00:44.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.199+0000 7fc03ce5b740 20 client.18870 populate_metadata read hostname 'smithi083' 2024-04-24T09:00:44.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.199+0000 7fc03ce5b740 1 --2- 172.21.15.83:0/4285427041 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55f95887da80 0x55f95887fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:44.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.199+0000 7fc03ce5b740 4 client.18870 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:00:44.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.199+0000 7fc03ce5b740 1 -- 172.21.15.83:0/4285427041 --> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f9587d5bf0 con 0x55f95887da80 2024-04-24T09:00:44.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.200+0000 7fc02aa00640 1 --2- 172.21.15.83:0/4285427041 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55f95887da80 0x55f95887fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:44.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.201+0000 7fc02aa00640 1 --2- 172.21.15.83:0/4285427041 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55f95887da80 0x55f95887fed0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b94456fb8646e42 server_cookie=34d7155cd61b7d62 in_seq=0 out_seq=0 2024-04-24T09:00:44.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.201+0000 7fc020c00640 10 client.18870 ms_handle_connect on v2:172.21.15.138:6832/1157967658 2024-04-24T09:00:44.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.203+0000 7fc020c00640 1 -- 172.21.15.83:0/4285427041 <== mds.3 v2:172.21.15.138:6832/1157967658 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f9587d5bf0 con 0x55f95887da80 2024-04-24T09:00:44.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.203+0000 7fc020c00640 10 client.18870 handle_command_reply: tid=0 2024-04-24T09:00:44.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.203+0000 7fc03ce5b740 1 client.18870 shutdown 2024-04-24T09:00:44.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.203+0000 7fc03ce5b740 1 -- 172.21.15.83:0/4285427041 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f958862950 msgr2=0x55f95886d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:44.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.203+0000 7fc03ce5b740 1 --2- 172.21.15.83:0/4285427041 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f958862950 0x55f95886d340 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7fc024005ec0 tx=0x7fc024009260 comp rx=0 tx=0).stop 2024-04-24T09:00:44.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.203+0000 7fc03ce5b740 1 -- 172.21.15.83:0/4285427041 shutdown_connections 2024-04-24T09:00:44.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.203+0000 7fc03ce5b740 1 --2- 172.21.15.83:0/4285427041 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55f95887da80 0x55f95887fed0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:44.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.203+0000 7fc03ce5b740 1 --2- 172.21.15.83:0/4285427041 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f95886d940 0x55f95886ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:44.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.203+0000 7fc03ce5b740 1 --2- 172.21.15.83:0/4285427041 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f958863270 0x55f95886a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:44.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.203+0000 7fc020c00640 0 client.18870 ms_handle_reset on v2:172.21.15.138:6832/1157967658 2024-04-24T09:00:44.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.203+0000 7fc03ce5b740 1 --2- 172.21.15.83:0/4285427041 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f958862950 0x55f95886d340 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:44.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.203+0000 7fc03ce5b740 1 -- 172.21.15.83:0/4285427041 >> 172.21.15.83:0/4285427041 conn(0x55f95885ec70 msgr2=0x55f958864a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:44.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.203+0000 7fc03ce5b740 1 -- 172.21.15.83:0/4285427041 shutdown_connections 2024-04-24T09:00:44.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.203+0000 7fc03ce5b740 1 -- 172.21.15.83:0/4285427041 wait complete. 2024-04-24T09:00:44.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.203+0000 7fc03ce5b740 20 client.18870 trim_cache size 0 max 16384 2024-04-24T09:00:44.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:00:43 smithi138 ceph-mon[31441]: pgmap v284: 129 pgs: 129 active+clean; 51 MiB data, 718 MiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 373 KiB/s wr, 73 op/s 2024-04-24T09:00:44.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:00:43 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/3814176226' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:00:44.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:00:43 smithi189 ceph-mon[31791]: pgmap v284: 129 pgs: 129 active+clean; 51 MiB data, 718 MiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 373 KiB/s wr, 73 op/s 2024-04-24T09:00:44.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:00:43 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/3814176226' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:00:44.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.222+0000 7fc03ce5b740 1 Processor -- start 2024-04-24T09:00:44.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.222+0000 7fc03ce5b740 1 -- start start 2024-04-24T09:00:44.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.222+0000 7fc03ce5b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f958862950 0x55f9588796c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:44.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.222+0000 7fc03ce5b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f958863270 0x55f95886cee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:44.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.222+0000 7fc03ce5b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f95886d940 0x55f9588857e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:44.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.222+0000 7fc03ce5b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc0341b1770 con 0x55f958862950 2024-04-24T09:00:44.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.222+0000 7fc03ce5b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc0341b32b0 con 0x55f958863270 2024-04-24T09:00:44.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.222+0000 7fc03ce5b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc0341b1e40 con 0x55f95886d940 2024-04-24T09:00:44.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.223+0000 7fc033400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f958862950 0x55f9588796c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:44.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.223+0000 7fc033400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f958862950 0x55f9588796c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48628/0 (socket says 172.21.15.83:48628) 2024-04-24T09:00:44.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.223+0000 7fc033400640 1 -- 172.21.15.83:0/1602998467 learned_addr learned my addr 172.21.15.83:0/1602998467 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:00:44.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.223+0000 7fc02aa00640 1 --2- 172.21.15.83:0/1602998467 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f958863270 0x55f95886cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:44.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.223+0000 7fc033e00640 1 --2- 172.21.15.83:0/1602998467 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f95886d940 0x55f9588857e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:44.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.223+0000 7fc033400640 1 -- 172.21.15.83:0/1602998467 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f95886d940 msgr2=0x55f9588857e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:44.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.223+0000 7fc033400640 1 --2- 172.21.15.83:0/1602998467 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f95886d940 0x55f9588857e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:44.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.223+0000 7fc033400640 1 -- 172.21.15.83:0/1602998467 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f958863270 msgr2=0x55f95886cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:44.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.223+0000 7fc033400640 1 --2- 172.21.15.83:0/1602998467 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f958863270 0x55f95886cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:44.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.223+0000 7fc033400640 1 -- 172.21.15.83:0/1602998467 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9588630a0 con 0x55f958862950 2024-04-24T09:00:44.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.224+0000 7fc033400640 1 --2- 172.21.15.83:0/1602998467 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f958862950 0x55f9588796c0 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7fc0240b9aa0 tx=0x7fc024031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c765b0f92cf7efff server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:44.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.224+0000 7fc023400640 1 -- 172.21.15.83:0/1602998467 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc024049070 con 0x55f958862950 2024-04-24T09:00:44.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.224+0000 7fc023400640 1 -- 172.21.15.83:0/1602998467 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc0240bd7d0 con 0x55f958862950 2024-04-24T09:00:44.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.224+0000 7fc023400640 1 -- 172.21.15.83:0/1602998467 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0240d6020 con 0x55f958862950 2024-04-24T09:00:44.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.225+0000 7fc03ce5b740 1 -- 172.21.15.83:0/1602998467 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f958862950 msgr2=0x55f9588796c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:44.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.225+0000 7fc03ce5b740 1 --2- 172.21.15.83:0/1602998467 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f958862950 0x55f9588796c0 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7fc0240b9aa0 tx=0x7fc024031400 comp rx=0 tx=0).stop 2024-04-24T09:00:44.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.225+0000 7fc03ce5b740 1 -- 172.21.15.83:0/1602998467 shutdown_connections 2024-04-24T09:00:44.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.225+0000 7fc03ce5b740 1 --2- 172.21.15.83:0/1602998467 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f95886d940 0x55f9588857e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:44.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.225+0000 7fc03ce5b740 1 --2- 172.21.15.83:0/1602998467 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f958863270 0x55f95886cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:44.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.225+0000 7fc03ce5b740 1 --2- 172.21.15.83:0/1602998467 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f958862950 0x55f9588796c0 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:44.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.225+0000 7fc03ce5b740 1 -- 172.21.15.83:0/1602998467 >> 172.21.15.83:0/1602998467 conn(0x55f95885ec70 msgr2=0x55f95885f050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:44.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.225+0000 7fc03ce5b740 1 -- 172.21.15.83:0/1602998467 shutdown_connections 2024-04-24T09:00:44.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.225+0000 7fc03ce5b740 1 -- 172.21.15.83:0/1602998467 wait complete. 2024-04-24T09:00:44.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.226+0000 7fc03ce5b740 1 -- start start 2024-04-24T09:00:44.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.226+0000 7fc03ce5b740 1 Processor -- start 2024-04-24T09:00:44.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.226+0000 7fc03ce5b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f958862950 0x55f958875e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:44.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.226+0000 7fc03ce5b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f958863270 0x55f95886e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:44.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.226+0000 7fc03ce5b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f95886d940 0x55f95886e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:44.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.226+0000 7fc03ce5b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc0341b2be0 con 0x55f958863270 2024-04-24T09:00:44.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.226+0000 7fc03ce5b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc0341b2510 con 0x55f95886d940 2024-04-24T09:00:44.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.226+0000 7fc03ce5b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc0341af770 con 0x55f958862950 2024-04-24T09:00:44.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.226+0000 7fc03ce5b740 10 client.0 _ll_register_callbacks cb 0x7fc024003ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:00:44.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.227+0000 7fc02aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f958863270 0x55f95886e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:44.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.227+0000 7fc02aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f958863270 0x55f95886e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48630/0 (socket says 172.21.15.83:48630) 2024-04-24T09:00:44.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.227+0000 7fc02aa00640 1 -- 172.21.15.83:0/1196588261 learned_addr learned my addr 172.21.15.83:0/1196588261 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:00:44.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.227+0000 7fc033400640 1 --2- 172.21.15.83:0/1196588261 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f958862950 0x55f958875e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:44.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.227+0000 7fc033e00640 1 --2- 172.21.15.83:0/1196588261 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f95886d940 0x55f95886e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:44.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.227+0000 7fc02aa00640 1 -- 172.21.15.83:0/1196588261 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f958862950 msgr2=0x55f958875e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:44.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.227+0000 7fc02aa00640 1 --2- 172.21.15.83:0/1196588261 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f958862950 0x55f958875e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:44.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.227+0000 7fc02aa00640 1 -- 172.21.15.83:0/1196588261 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f95886d940 msgr2=0x55f95886e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:44.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.227+0000 7fc02aa00640 1 --2- 172.21.15.83:0/1196588261 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f95886d940 0x55f95886e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:44.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.227+0000 7fc02aa00640 1 -- 172.21.15.83:0/1196588261 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9585a47d0 con 0x55f958863270 2024-04-24T09:00:44.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.227+0000 7fc033400640 1 --2- 172.21.15.83:0/1196588261 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f958862950 0x55f958875e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:00:44.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.227+0000 7fc02aa00640 1 --2- 172.21.15.83:0/1196588261 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f958863270 0x55f95886e320 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7fc01c018e50 tx=0x7fc01c008bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f220ad6736319f45 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:44.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.227+0000 7fc020c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:00:44.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.228+0000 7fc020c00640 1 -- 172.21.15.83:0/1196588261 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc01c004be0 con 0x55f958863270 2024-04-24T09:00:44.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.228+0000 7fc020c00640 1 -- 172.21.15.83:0/1196588261 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7fc01c004d80 con 0x55f958863270 2024-04-24T09:00:44.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.228+0000 7fc03ce5b740 1 -- 172.21.15.83:0/1196588261 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0240bd7d0 con 0x55f958863270 2024-04-24T09:00:44.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.228+0000 7fc020c00640 1 -- 172.21.15.83:0/1196588261 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fc01c018480 con 0x55f958863270 2024-04-24T09:00:44.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.228+0000 7fc03ce5b740 10 client.18876 fetch_fsmap learned FSMap version 35 2024-04-24T09:00:44.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.228+0000 7fc03ce5b740 1 -- 172.21.15.83:0/1196588261 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55f958875890 con 0x55f958863270 2024-04-24T09:00:44.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.228+0000 7fc020c00640 1 -- 172.21.15.83:0/1196588261 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc0240bd7d0 con 0x55f958863270 2024-04-24T09:00:44.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.228+0000 7fc020c00640 1 -- 172.21.15.83:0/1196588261 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7fc01c0250f0 con 0x55f958863270 2024-04-24T09:00:44.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.229+0000 7fc03ce5b740 10 client.18876 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T09:00:44.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.229+0000 7fc03ce5b740 10 client.18876 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-24T09:00:44.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.229+0000 7fc03ce5b740 20 client.18876 populate_metadata read hostname 'smithi083' 2024-04-24T09:00:44.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.229+0000 7fc03ce5b740 1 --2- 172.21.15.83:0/1196588261 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55f9588e43d0 0x55f9588e67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:44.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.229+0000 7fc03ce5b740 4 client.18876 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:00:44.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.229+0000 7fc03ce5b740 1 -- 172.21.15.83:0/1196588261 --> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f9587cfa60 con 0x55f9588e43d0 2024-04-24T09:00:44.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.229+0000 7fc033400640 1 --2- 172.21.15.83:0/1196588261 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55f9588e43d0 0x55f9588e67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:44.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.230+0000 7fc033400640 1 --2- 172.21.15.83:0/1196588261 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55f9588e43d0 0x55f9588e67f0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f45a14abbaa1ca80 server_cookie=4a68bae919ef25d in_seq=0 out_seq=0 2024-04-24T09:00:44.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.230+0000 7fc020c00640 10 client.18876 ms_handle_connect on v2:172.21.15.138:6832/1157967658 2024-04-24T09:00:44.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.231+0000 7fc020c00640 1 -- 172.21.15.83:0/1196588261 <== mds.3 v2:172.21.15.138:6832/1157967658 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f9587cfa60 con 0x55f9588e43d0 2024-04-24T09:00:44.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.231+0000 7fc020c00640 10 client.18876 handle_command_reply: tid=0 2024-04-24T09:00:44.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.231+0000 7fc03ce5b740 1 client.18876 shutdown 2024-04-24T09:00:44.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.231+0000 7fc03ce5b740 1 -- 172.21.15.83:0/1196588261 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f958863270 msgr2=0x55f95886e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:44.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.231+0000 7fc03ce5b740 1 --2- 172.21.15.83:0/1196588261 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f958863270 0x55f95886e320 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7fc01c018e50 tx=0x7fc01c008bd0 comp rx=0 tx=0).stop 2024-04-24T09:00:44.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.231+0000 7fc03ce5b740 1 -- 172.21.15.83:0/1196588261 shutdown_connections 2024-04-24T09:00:44.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.231+0000 7fc03ce5b740 1 --2- 172.21.15.83:0/1196588261 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f95886d940 0x55f95886e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:44.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.231+0000 7fc03ce5b740 1 --2- 172.21.15.83:0/1196588261 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f958863270 0x55f95886e320 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:44.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.231+0000 7fc03ce5b740 1 --2- 172.21.15.83:0/1196588261 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55f9588e43d0 0x55f9588e67f0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:44.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.231+0000 7fc03ce5b740 1 --2- 172.21.15.83:0/1196588261 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f958862950 0x55f958875e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:44.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.231+0000 7fc020c00640 0 client.18876 ms_handle_reset on v2:172.21.15.138:6832/1157967658 2024-04-24T09:00:44.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.231+0000 7fc03ce5b740 1 -- 172.21.15.83:0/1196588261 >> 172.21.15.83:0/1196588261 conn(0x55f95885ec70 msgr2=0x55f958868e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:44.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.231+0000 7fc03ce5b740 1 -- 172.21.15.83:0/1196588261 shutdown_connections 2024-04-24T09:00:44.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.231+0000 7fc03ce5b740 1 -- 172.21.15.83:0/1196588261 wait complete. 2024-04-24T09:00:44.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.231+0000 7fc03ce5b740 20 client.18876 trim_cache size 0 max 16384 2024-04-24T09:00:44.235 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:00:44.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.232+0000 7fc03a000640 1 -- 172.21.15.83:0/3434308585 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fc0180a3da0 msgr2=0x7fc0180a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:44.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.232+0000 7fc03a000640 1 --2- 172.21.15.83:0/3434308585 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fc0180a3da0 0x7fc0180a6260 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc02c0058c0 tx=0x7fc02c005eb0 comp rx=0 tx=0).stop 2024-04-24T09:00:44.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.232+0000 7fc03a000640 1 -- 172.21.15.83:0/3434308585 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc034103280 msgr2=0x7fc0341a0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:44.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.232+0000 7fc03a000640 1 --2- 172.21.15.83:0/3434308585 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc034103280 0x7fc0341a0640 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7fc024005ab0 tx=0x7fc024003cd0 comp rx=0 tx=0).stop 2024-04-24T09:00:44.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.232+0000 7fc03a000640 1 -- 172.21.15.83:0/3434308585 shutdown_connections 2024-04-24T09:00:44.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.232+0000 7fc03a000640 1 --2- 172.21.15.83:0/3434308585 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fc0180a3da0 0x7fc0180a6260 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:44.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.232+0000 7fc03a000640 1 --2- 172.21.15.83:0/3434308585 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc03410c5c0 0x7fc03419a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:44.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.232+0000 7fc03a000640 1 --2- 172.21.15.83:0/3434308585 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc034103ba0 0x7fc0341a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:44.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.232+0000 7fc03a000640 1 --2- 172.21.15.83:0/3434308585 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc034103280 0x7fc0341a0640 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:44.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.232+0000 7fc03a000640 1 -- 172.21.15.83:0/3434308585 >> 172.21.15.83:0/3434308585 conn(0x7fc034078000 msgr2=0x7fc034108490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:44.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.232+0000 7fc03a000640 1 -- 172.21.15.83:0/3434308585 shutdown_connections 2024-04-24T09:00:44.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.232+0000 7fc03a000640 1 -- 172.21.15.83:0/3434308585 wait complete. 2024-04-24T09:00:44.250 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T09:00:44.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.454+0000 7f9970600640 1 -- 172.21.15.83:0/4241363935 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9968106210 msgr2=0x7f996810dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:44.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.454+0000 7f9970600640 1 --2- 172.21.15.83:0/4241363935 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9968106210 0x7f996810dde0 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f9960009ca0 tx=0x7f996002a8c0 comp rx=0 tx=0).stop 2024-04-24T09:00:44.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.455+0000 7f9970600640 1 -- 172.21.15.83:0/4241363935 shutdown_connections 2024-04-24T09:00:44.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.455+0000 7f9970600640 1 --2- 172.21.15.83:0/4241363935 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9968106210 0x7f996810dde0 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:44.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.455+0000 7f9970600640 1 --2- 172.21.15.83:0/4241363935 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9968101af0 0x7f9968105b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:44.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.455+0000 7f9970600640 1 --2- 172.21.15.83:0/4241363935 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9968101140 0x7f9968101520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:44.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.455+0000 7f9970600640 1 -- 172.21.15.83:0/4241363935 >> 172.21.15.83:0/4241363935 conn(0x7f9968078000 msgr2=0x7f9968078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:44.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.455+0000 7f9970600640 1 -- 172.21.15.83:0/4241363935 shutdown_connections 2024-04-24T09:00:44.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.455+0000 7f9970600640 1 -- 172.21.15.83:0/4241363935 wait complete. 2024-04-24T09:00:44.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.456+0000 7f9970600640 1 Processor -- start 2024-04-24T09:00:44.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.456+0000 7f9970600640 1 -- start start 2024-04-24T09:00:44.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.456+0000 7f9970600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9968101140 0x7f99681a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:44.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.456+0000 7f9970600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9968106210 0x7f99681a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:44.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.456+0000 7f9970600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f996819a7f0 0x7f996819aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:44.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.456+0000 7f9970600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9968069660 con 0x7f9968101140 2024-04-24T09:00:44.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.456+0000 7f9970600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f99680694e0 con 0x7f996819a7f0 2024-04-24T09:00:44.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.456+0000 7f9970600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f99680697e0 con 0x7f9968106210 2024-04-24T09:00:44.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.457+0000 7f996dc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9968101140 0x7f99681a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:44.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.457+0000 7f996dc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9968101140 0x7f99681a0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48636/0 (socket says 172.21.15.83:48636) 2024-04-24T09:00:44.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.457+0000 7f996dc00640 1 -- 172.21.15.83:0/3451492390 learned_addr learned my addr 172.21.15.83:0/3451492390 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:00:44.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.457+0000 7f996d200640 1 --2- 172.21.15.83:0/3451492390 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9968106210 0x7f99681a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:44.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.457+0000 7f996dc00640 1 -- 172.21.15.83:0/3451492390 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9968106210 msgr2=0x7f99681a0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:44.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.457+0000 7f996dc00640 1 --2- 172.21.15.83:0/3451492390 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9968106210 0x7f99681a0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:44.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.457+0000 7f996e600640 1 --2- 172.21.15.83:0/3451492390 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f996819a7f0 0x7f996819aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:44.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.457+0000 7f996dc00640 1 -- 172.21.15.83:0/3451492390 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f996819a7f0 msgr2=0x7f996819aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:44.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.457+0000 7f996dc00640 1 --2- 172.21.15.83:0/3451492390 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f996819a7f0 0x7f996819aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:44.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.457+0000 7f996dc00640 1 -- 172.21.15.83:0/3451492390 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f996819b380 con 0x7f9968101140 2024-04-24T09:00:44.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.458+0000 7f996dc00640 1 --2- 172.21.15.83:0/3451492390 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9968101140 0x7f99681a0680 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f995c00c970 tx=0x7f995c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:44.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.458+0000 7f9966000640 1 -- 172.21.15.83:0/3451492390 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f995c007bb0 con 0x7f9968101140 2024-04-24T09:00:44.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.458+0000 7f9966000640 1 -- 172.21.15.83:0/3451492390 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f995c007d50 con 0x7f9968101140 2024-04-24T09:00:44.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.458+0000 7f9966000640 1 -- 172.21.15.83:0/3451492390 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f995c00f450 con 0x7f9968101140 2024-04-24T09:00:44.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.458+0000 7f9970600640 1 -- 172.21.15.83:0/3451492390 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f996819b670 con 0x7f9968101140 2024-04-24T09:00:44.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.458+0000 7f9970600640 1 -- 172.21.15.83:0/3451492390 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99681a7800 con 0x7f9968101140 2024-04-24T09:00:44.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.461+0000 7f9973570740 1 Processor -- start 2024-04-24T09:00:44.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.462+0000 7f9966000640 1 -- 172.21.15.83:0/3451492390 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f995c004f20 con 0x7f9968101140 2024-04-24T09:00:44.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.462+0000 7f9973570740 1 -- start start 2024-04-24T09:00:44.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.462+0000 7f9973570740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f6a8e7950 0x557f6a8e7d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:44.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.462+0000 7f9973570740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557f6a8e8270 0x557f6a8f2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:44.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.462+0000 7f9973570740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557f6a8f2940 0x557f6a8f4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:44.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.462+0000 7f9973570740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x557f6a84d1d0 con 0x557f6a8e7950 2024-04-24T09:00:44.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.462+0000 7f9973570740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x557f6a7f3450 con 0x557f6a8e8270 2024-04-24T09:00:44.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.462+0000 7f9973570740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x557f6a849190 con 0x557f6a8f2940 2024-04-24T09:00:44.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.462+0000 7f996e600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f6a8e7950 0x557f6a8e7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:44.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.462+0000 7f996e600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f6a8e7950 0x557f6a8e7d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48644/0 (socket says 172.21.15.83:48644) 2024-04-24T09:00:44.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.462+0000 7f996e600640 1 -- 172.21.15.83:0/3450663315 learned_addr learned my addr 172.21.15.83:0/3450663315 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:00:44.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.463+0000 7f996d200640 1 --2- 172.21.15.83:0/3450663315 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557f6a8f2940 0x557f6a8f4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:44.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.463+0000 7f996dc00640 1 --2- 172.21.15.83:0/3450663315 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557f6a8e8270 0x557f6a8f2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:44.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.463+0000 7f996e600640 1 -- 172.21.15.83:0/3450663315 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557f6a8f2940 msgr2=0x557f6a8f4d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:44.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.463+0000 7f996e600640 1 --2- 172.21.15.83:0/3450663315 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557f6a8f2940 0x557f6a8f4d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:44.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.463+0000 7f996e600640 1 -- 172.21.15.83:0/3450663315 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557f6a8e8270 msgr2=0x557f6a8f2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:44.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.463+0000 7f996e600640 1 --2- 172.21.15.83:0/3450663315 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557f6a8e8270 0x557f6a8f2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:44.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.463+0000 7f996e600640 1 -- 172.21.15.83:0/3450663315 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f6a8b0310 con 0x557f6a8e7950 2024-04-24T09:00:44.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.463+0000 7f996dc00640 1 --2- 172.21.15.83:0/3450663315 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557f6a8e8270 0x557f6a8f2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:00:44.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.463+0000 7f996e600640 1 --2- 172.21.15.83:0/3450663315 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f6a8e7950 0x557f6a8e7d30 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f9960031af0 tx=0x7f9960009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d5769e396755c31 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:44.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.464+0000 7f995aa00640 1 -- 172.21.15.83:0/3450663315 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9960031450 con 0x557f6a8e7950 2024-04-24T09:00:44.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.464+0000 7f995aa00640 1 -- 172.21.15.83:0/3450663315 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9960036070 con 0x557f6a8e7950 2024-04-24T09:00:44.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.464+0000 7f995aa00640 1 -- 172.21.15.83:0/3450663315 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9960004510 con 0x557f6a8e7950 2024-04-24T09:00:44.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.464+0000 7f9966000640 1 --2- 172.21.15.83:0/3451492390 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f99500a3d20 0x7f99500a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:44.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.464+0000 7f9966000640 1 -- 172.21.15.83:0/3451492390 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f995c0d0a60 con 0x7f9968101140 2024-04-24T09:00:44.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.464+0000 7f996dc00640 1 --2- 172.21.15.83:0/3451492390 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f99500a3d20 0x7f99500a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:44.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.465+0000 7f9973570740 1 -- 172.21.15.83:0/3450663315 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f6a8e7950 msgr2=0x557f6a8e7d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:44.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.465+0000 7f9973570740 1 --2- 172.21.15.83:0/3450663315 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f6a8e7950 0x557f6a8e7d30 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f9960031af0 tx=0x7f9960009a20 comp rx=0 tx=0).stop 2024-04-24T09:00:44.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.465+0000 7f9973570740 1 -- 172.21.15.83:0/3450663315 shutdown_connections 2024-04-24T09:00:44.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.465+0000 7f9973570740 1 --2- 172.21.15.83:0/3450663315 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557f6a8f2940 0x557f6a8f4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:44.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.465+0000 7f9973570740 1 --2- 172.21.15.83:0/3450663315 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557f6a8e8270 0x557f6a8f2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:44.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.465+0000 7f9973570740 1 --2- 172.21.15.83:0/3450663315 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f6a8e7950 0x557f6a8e7d30 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:44.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.465+0000 7f9973570740 1 -- 172.21.15.83:0/3450663315 >> 172.21.15.83:0/3450663315 conn(0x557f6a8e3c70 msgr2=0x557f6a8e4050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:44.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.465+0000 7f9973570740 1 -- 172.21.15.83:0/3450663315 shutdown_connections 2024-04-24T09:00:44.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.465+0000 7f9973570740 1 -- 172.21.15.83:0/3450663315 wait complete. 2024-04-24T09:00:44.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.465+0000 7f996dc00640 1 --2- 172.21.15.83:0/3451492390 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f99500a3d20 0x7f99500a61e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f995c0cde40 tx=0x7f995c004920 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:44.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.466+0000 7f9973570740 1 -- start start 2024-04-24T09:00:44.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.466+0000 7f9973570740 1 Processor -- start 2024-04-24T09:00:44.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.466+0000 7f9973570740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557f6a8e7950 0x557f6a8f23a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:44.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.467+0000 7f9973570740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f6a8e8270 0x557f6a8ef690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:44.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.467+0000 7f9973570740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557f6a8f2940 0x557f6a8efdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:44.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.467+0000 7f9973570740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x557f6a8f5770 con 0x557f6a8e8270 2024-04-24T09:00:44.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.467+0000 7f9973570740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x557f6a806810 con 0x557f6a8f2940 2024-04-24T09:00:44.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.467+0000 7f9973570740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x557f6a8f58f0 con 0x557f6a8e7950 2024-04-24T09:00:44.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.467+0000 7f9973570740 10 client.0 _ll_register_callbacks cb 0x557f6a7d4140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:00:44.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.467+0000 7f996d200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f6a8e8270 0x557f6a8ef690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:44.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.467+0000 7f996d200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f6a8e8270 0x557f6a8ef690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48650/0 (socket says 172.21.15.83:48650) 2024-04-24T09:00:44.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.467+0000 7f996d200640 1 -- 172.21.15.83:0/1825130393 learned_addr learned my addr 172.21.15.83:0/1825130393 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:00:44.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.467+0000 7f996e600640 1 --2- 172.21.15.83:0/1825130393 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557f6a8e7950 0x557f6a8f23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:44.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.467+0000 7f996e600640 1 --2- 172.21.15.83:0/1825130393 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557f6a8f2940 0x557f6a8efdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:44.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.467+0000 7f996d200640 1 -- 172.21.15.83:0/1825130393 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557f6a8e7950 msgr2=0x557f6a8f23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:44.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.467+0000 7f996d200640 1 --2- 172.21.15.83:0/1825130393 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557f6a8e7950 0x557f6a8f23a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:44.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.467+0000 7f996d200640 1 -- 172.21.15.83:0/1825130393 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557f6a8f2940 msgr2=0x557f6a8efdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:44.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.467+0000 7f996d200640 1 --2- 172.21.15.83:0/1825130393 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557f6a8f2940 0x557f6a8efdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:44.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.467+0000 7f996d200640 1 -- 172.21.15.83:0/1825130393 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f6a8173f0 con 0x557f6a8e8270 2024-04-24T09:00:44.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.467+0000 7f996e600640 1 --2- 172.21.15.83:0/1825130393 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557f6a8f2940 0x557f6a8efdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:00:44.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.468+0000 7f996d200640 1 --2- 172.21.15.83:0/1825130393 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f6a8e8270 0x557f6a8ef690 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f9954010760 tx=0x7f9954010c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5b9ff0735485cdc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:44.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.468+0000 7f994f400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:00:44.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.468+0000 7f994f400640 1 -- 172.21.15.83:0/1825130393 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f995400cd20 con 0x557f6a8e8270 2024-04-24T09:00:44.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.468+0000 7f994f400640 1 -- 172.21.15.83:0/1825130393 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f9954011440 con 0x557f6a8e8270 2024-04-24T09:00:44.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.468+0000 7f9973570740 1 -- 172.21.15.83:0/1825130393 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9960036070 con 0x557f6a8e8270 2024-04-24T09:00:44.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.468+0000 7f994f400640 1 -- 172.21.15.83:0/1825130393 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f9954024050 con 0x557f6a8e8270 2024-04-24T09:00:44.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.468+0000 7f9973570740 10 client.18888 fetch_fsmap learned FSMap version 35 2024-04-24T09:00:44.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.468+0000 7f9973570740 1 -- 172.21.15.83:0/1825130393 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x557f6a55f3f0 con 0x557f6a8e8270 2024-04-24T09:00:44.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.468+0000 7f994f400640 1 -- 172.21.15.83:0/1825130393 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9960036070 con 0x557f6a8e8270 2024-04-24T09:00:44.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.469+0000 7f994f400640 1 -- 172.21.15.83:0/1825130393 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7f995401c5c0 con 0x557f6a8e8270 2024-04-24T09:00:44.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.469+0000 7f9973570740 10 client.18888 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T09:00:44.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.469+0000 7f9973570740 10 client.18888 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-24T09:00:44.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.469+0000 7f9973570740 20 client.18888 populate_metadata read hostname 'smithi083' 2024-04-24T09:00:44.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.469+0000 7f9973570740 1 --2- 172.21.15.83:0/1825130393 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x557f6a902b40 0x557f6a904fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:44.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.469+0000 7f9973570740 4 client.18888 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:00:44.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.469+0000 7f9973570740 1 -- 172.21.15.83:0/1825130393 --> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557f6a85abf0 con 0x557f6a902b40 2024-04-24T09:00:44.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.470+0000 7f996e600640 1 --2- 172.21.15.83:0/1825130393 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x557f6a902b40 0x557f6a904fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:44.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.471+0000 7f996e600640 1 --2- 172.21.15.83:0/1825130393 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x557f6a902b40 0x557f6a904fa0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7be005f771d43b3 server_cookie=2441eb2bb665e590 in_seq=0 out_seq=0 2024-04-24T09:00:44.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.471+0000 7f994f400640 10 client.18888 ms_handle_connect on v2:172.21.15.138:6834/1909219758 2024-04-24T09:00:44.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.472+0000 7f994f400640 1 -- 172.21.15.83:0/1825130393 <== mds.2 v2:172.21.15.138:6834/1909219758 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557f6a85abf0 con 0x557f6a902b40 2024-04-24T09:00:44.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.472+0000 7f994f400640 10 client.18888 handle_command_reply: tid=0 2024-04-24T09:00:44.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.472+0000 7f9973570740 1 client.18888 shutdown 2024-04-24T09:00:44.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.473+0000 7f9973570740 1 -- 172.21.15.83:0/1825130393 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f6a8e8270 msgr2=0x557f6a8ef690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:44.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.473+0000 7f9973570740 1 --2- 172.21.15.83:0/1825130393 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f6a8e8270 0x557f6a8ef690 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f9954010760 tx=0x7f9954010c20 comp rx=0 tx=0).stop 2024-04-24T09:00:44.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.473+0000 7f9973570740 1 -- 172.21.15.83:0/1825130393 shutdown_connections 2024-04-24T09:00:44.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.473+0000 7f9973570740 1 --2- 172.21.15.83:0/1825130393 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x557f6a902b40 0x557f6a904fa0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:44.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.473+0000 7f9973570740 1 --2- 172.21.15.83:0/1825130393 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557f6a8f2940 0x557f6a8efdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:44.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.473+0000 7f9973570740 1 --2- 172.21.15.83:0/1825130393 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f6a8e8270 0x557f6a8ef690 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:44.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.473+0000 7f994f400640 0 client.18888 ms_handle_reset on v2:172.21.15.138:6834/1909219758 2024-04-24T09:00:44.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.473+0000 7f9973570740 1 --2- 172.21.15.83:0/1825130393 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557f6a8e7950 0x557f6a8f23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:44.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.473+0000 7f9973570740 1 -- 172.21.15.83:0/1825130393 >> 172.21.15.83:0/1825130393 conn(0x557f6a8e3c70 msgr2=0x557f6a8e9a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:44.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.473+0000 7f9973570740 1 -- 172.21.15.83:0/1825130393 shutdown_connections 2024-04-24T09:00:44.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.473+0000 7f9973570740 1 -- 172.21.15.83:0/1825130393 wait complete. 2024-04-24T09:00:44.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.473+0000 7f9973570740 20 client.18888 trim_cache size 0 max 16384 2024-04-24T09:00:44.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.489+0000 7f9973570740 1 Processor -- start 2024-04-24T09:00:44.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.489+0000 7f9973570740 1 -- start start 2024-04-24T09:00:44.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.489+0000 7f9973570740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f6a8e7950 0x557f6a8ea0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:44.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.489+0000 7f9973570740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557f6a8e8270 0x557f6a8ee7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:44.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.489+0000 7f9973570740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557f6a8f2940 0x557f6a8f1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:44.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.489+0000 7f9973570740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f99681b17f0 con 0x557f6a8e7950 2024-04-24T09:00:44.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.489+0000 7f9973570740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f99681b3330 con 0x557f6a8f2940 2024-04-24T09:00:44.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.489+0000 7f9973570740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f99681b1ec0 con 0x557f6a8e8270 2024-04-24T09:00:44.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.490+0000 7f996e600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f6a8e7950 0x557f6a8ea0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:44.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.490+0000 7f996e600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f6a8e7950 0x557f6a8ea0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48666/0 (socket says 172.21.15.83:48666) 2024-04-24T09:00:44.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.490+0000 7f996e600640 1 -- 172.21.15.83:0/2574818493 learned_addr learned my addr 172.21.15.83:0/2574818493 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:00:44.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.490+0000 7f996d200640 1 --2- 172.21.15.83:0/2574818493 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557f6a8e8270 0x557f6a8ee7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:44.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.490+0000 7f996e600640 1 -- 172.21.15.83:0/2574818493 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557f6a8e8270 msgr2=0x557f6a8ee7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:44.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.490+0000 7f996e600640 1 --2- 172.21.15.83:0/2574818493 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557f6a8e8270 0x557f6a8ee7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:44.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.490+0000 7f996e600640 1 -- 172.21.15.83:0/2574818493 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557f6a8f2940 msgr2=0x557f6a8f1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:44.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.490+0000 7f996e600640 1 --2- 172.21.15.83:0/2574818493 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557f6a8f2940 0x557f6a8f1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:44.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.490+0000 7f996e600640 1 -- 172.21.15.83:0/2574818493 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f6a8e80a0 con 0x557f6a8e7950 2024-04-24T09:00:44.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.491+0000 7f996e600640 1 --2- 172.21.15.83:0/2574818493 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f6a8e7950 0x557f6a8ea0b0 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f996002c010 tx=0x7f9960005c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cb40535eab5894c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:44.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.491+0000 7f995a000640 1 -- 172.21.15.83:0/2574818493 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9960049040 con 0x557f6a8e7950 2024-04-24T09:00:44.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.491+0000 7f995a000640 1 -- 172.21.15.83:0/2574818493 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9960053e50 con 0x557f6a8e7950 2024-04-24T09:00:44.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.491+0000 7f995a000640 1 -- 172.21.15.83:0/2574818493 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9960004030 con 0x557f6a8e7950 2024-04-24T09:00:44.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.492+0000 7f9973570740 1 -- 172.21.15.83:0/2574818493 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f6a8e7950 msgr2=0x557f6a8ea0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:44.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.492+0000 7f9973570740 1 --2- 172.21.15.83:0/2574818493 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f6a8e7950 0x557f6a8ea0b0 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f996002c010 tx=0x7f9960005c30 comp rx=0 tx=0).stop 2024-04-24T09:00:44.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.492+0000 7f9973570740 1 -- 172.21.15.83:0/2574818493 shutdown_connections 2024-04-24T09:00:44.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.492+0000 7f9973570740 1 --2- 172.21.15.83:0/2574818493 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557f6a8f2940 0x557f6a8f1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:44.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.492+0000 7f9973570740 1 --2- 172.21.15.83:0/2574818493 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557f6a8e8270 0x557f6a8ee7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:44.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.492+0000 7f9973570740 1 --2- 172.21.15.83:0/2574818493 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f6a8e7950 0x557f6a8ea0b0 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:44.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.492+0000 7f9973570740 1 -- 172.21.15.83:0/2574818493 >> 172.21.15.83:0/2574818493 conn(0x557f6a8e3c70 msgr2=0x557f6a8e2560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:44.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.492+0000 7f9973570740 1 -- 172.21.15.83:0/2574818493 shutdown_connections 2024-04-24T09:00:44.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.492+0000 7f9973570740 1 -- 172.21.15.83:0/2574818493 wait complete. 2024-04-24T09:00:44.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.492+0000 7f9973570740 1 -- start start 2024-04-24T09:00:44.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.493+0000 7f9973570740 1 Processor -- start 2024-04-24T09:00:44.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.493+0000 7f9973570740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f6a8e7950 0x557f6a8f3770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:44.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.493+0000 7f9973570740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557f6a8e8270 0x557f6a8f3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:44.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.493+0000 7f9973570740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557f6a8f2940 0x557f6a8f41f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:44.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.493+0000 7f9973570740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f99681b2c60 con 0x557f6a8e7950 2024-04-24T09:00:44.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.493+0000 7f9973570740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f99681b2590 con 0x557f6a8e8270 2024-04-24T09:00:44.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.493+0000 7f9973570740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f99681af7f0 con 0x557f6a8f2940 2024-04-24T09:00:44.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.493+0000 7f9973570740 10 client.0 _ll_register_callbacks cb 0x7f995400e3b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:00:44.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.493+0000 7f996e600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f6a8e7950 0x557f6a8f3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:44.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.493+0000 7f996e600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f6a8e7950 0x557f6a8f3770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48678/0 (socket says 172.21.15.83:48678) 2024-04-24T09:00:44.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.493+0000 7f996e600640 1 -- 172.21.15.83:0/898765720 learned_addr learned my addr 172.21.15.83:0/898765720 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:00:44.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.494+0000 7f996d200640 1 --2- 172.21.15.83:0/898765720 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557f6a8e8270 0x557f6a8f3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:44.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.494+0000 7f996e600640 1 -- 172.21.15.83:0/898765720 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557f6a8f2940 msgr2=0x557f6a8f41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:44.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.494+0000 7f996e600640 1 --2- 172.21.15.83:0/898765720 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557f6a8f2940 0x557f6a8f41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:44.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.494+0000 7f996e600640 1 -- 172.21.15.83:0/898765720 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557f6a8e8270 msgr2=0x557f6a8f3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:44.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.494+0000 7f996e600640 1 --2- 172.21.15.83:0/898765720 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557f6a8e8270 0x557f6a8f3cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:44.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.494+0000 7f996e600640 1 -- 172.21.15.83:0/898765720 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f6a6297d0 con 0x557f6a8e7950 2024-04-24T09:00:44.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.494+0000 7f996d200640 1 --2- 172.21.15.83:0/898765720 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557f6a8e8270 0x557f6a8f3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:00:44.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.494+0000 7f996e600640 1 --2- 172.21.15.83:0/898765720 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f6a8e7950 0x557f6a8f3770 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f9960009bc0 tx=0x7f9960053780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d2fa5f831b88cf server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:44.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.494+0000 7f994f400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:00:44.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.494+0000 7f994f400640 1 -- 172.21.15.83:0/898765720 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9960053370 con 0x557f6a8e7950 2024-04-24T09:00:44.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.494+0000 7f994f400640 1 -- 172.21.15.83:0/898765720 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f9960003970 con 0x557f6a8e7950 2024-04-24T09:00:44.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.494+0000 7f9973570740 1 -- 172.21.15.83:0/898765720 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9960053e50 con 0x557f6a8e7950 2024-04-24T09:00:44.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.494+0000 7f994f400640 1 -- 172.21.15.83:0/898765720 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f9960031830 con 0x557f6a8e7950 2024-04-24T09:00:44.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.494+0000 7f9973570740 10 client.18894 fetch_fsmap learned FSMap version 35 2024-04-24T09:00:44.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.494+0000 7f9973570740 1 -- 172.21.15.83:0/898765720 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x557f6a9692a0 con 0x557f6a8e7950 2024-04-24T09:00:44.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.495+0000 7f994f400640 1 -- 172.21.15.83:0/898765720 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9960053e50 con 0x557f6a8e7950 2024-04-24T09:00:44.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.495+0000 7f994f400640 1 -- 172.21.15.83:0/898765720 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7f99600305c0 con 0x557f6a8e7950 2024-04-24T09:00:44.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.495+0000 7f9973570740 10 client.18894 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T09:00:44.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.495+0000 7f9973570740 10 client.18894 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-24T09:00:44.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.495+0000 7f9973570740 20 client.18894 populate_metadata read hostname 'smithi083' 2024-04-24T09:00:44.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.495+0000 7f9973570740 1 --2- 172.21.15.83:0/898765720 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x557f6a969580 0x557f6a96b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:44.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.495+0000 7f9973570740 4 client.18894 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:00:44.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.495+0000 7f9973570740 1 -- 172.21.15.83:0/898765720 --> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557f6a854a60 con 0x557f6a969580 2024-04-24T09:00:44.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.496+0000 7f996d200640 1 --2- 172.21.15.83:0/898765720 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x557f6a969580 0x557f6a96b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:44.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.497+0000 7f996d200640 1 --2- 172.21.15.83:0/898765720 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x557f6a969580 0x557f6a96b970 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=adfd090846c673f8 server_cookie=8ca44697379e0874 in_seq=0 out_seq=0 2024-04-24T09:00:44.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.497+0000 7f994f400640 10 client.18894 ms_handle_connect on v2:172.21.15.138:6834/1909219758 2024-04-24T09:00:44.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.497+0000 7f994f400640 1 -- 172.21.15.83:0/898765720 <== mds.2 v2:172.21.15.138:6834/1909219758 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557f6a854a60 con 0x557f6a969580 2024-04-24T09:00:44.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.497+0000 7f994f400640 10 client.18894 handle_command_reply: tid=0 2024-04-24T09:00:44.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.497+0000 7f9973570740 1 client.18894 shutdown 2024-04-24T09:00:44.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.498+0000 7f9973570740 1 -- 172.21.15.83:0/898765720 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f6a8e7950 msgr2=0x557f6a8f3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:44.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.498+0000 7f9973570740 1 --2- 172.21.15.83:0/898765720 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f6a8e7950 0x557f6a8f3770 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f9960009bc0 tx=0x7f9960053780 comp rx=0 tx=0).stop 2024-04-24T09:00:44.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.498+0000 7f9973570740 1 -- 172.21.15.83:0/898765720 shutdown_connections 2024-04-24T09:00:44.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.498+0000 7f9973570740 1 --2- 172.21.15.83:0/898765720 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x557f6a969580 0x557f6a96b970 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:44.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.498+0000 7f9973570740 1 --2- 172.21.15.83:0/898765720 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557f6a8f2940 0x557f6a8f41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:44.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.498+0000 7f9973570740 1 --2- 172.21.15.83:0/898765720 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557f6a8e8270 0x557f6a8f3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:44.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.498+0000 7f994f400640 0 client.18894 ms_handle_reset on v2:172.21.15.138:6834/1909219758 2024-04-24T09:00:44.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.498+0000 7f9973570740 1 --2- 172.21.15.83:0/898765720 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557f6a8e7950 0x557f6a8f3770 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:44.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.498+0000 7f9973570740 1 -- 172.21.15.83:0/898765720 >> 172.21.15.83:0/898765720 conn(0x557f6a8e3c70 msgr2=0x557f6a8ee670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:44.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.498+0000 7f9973570740 1 -- 172.21.15.83:0/898765720 shutdown_connections 2024-04-24T09:00:44.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.498+0000 7f9973570740 1 -- 172.21.15.83:0/898765720 wait complete. 2024-04-24T09:00:44.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.498+0000 7f9973570740 20 client.18894 trim_cache size 0 max 16384 2024-04-24T09:00:44.500 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:00:44.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.499+0000 7f9970600640 1 -- 172.21.15.83:0/3451492390 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f99500a3d20 msgr2=0x7f99500a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:44.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.499+0000 7f9970600640 1 --2- 172.21.15.83:0/3451492390 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f99500a3d20 0x7f99500a61e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f995c0cde40 tx=0x7f995c004920 comp rx=0 tx=0).stop 2024-04-24T09:00:44.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.499+0000 7f9970600640 1 -- 172.21.15.83:0/3451492390 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9968101140 msgr2=0x7f99681a0680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:44.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.499+0000 7f9970600640 1 --2- 172.21.15.83:0/3451492390 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9968101140 0x7f99681a0680 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f995c00c970 tx=0x7f995c00ce30 comp rx=0 tx=0).stop 2024-04-24T09:00:44.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.499+0000 7f9970600640 1 -- 172.21.15.83:0/3451492390 shutdown_connections 2024-04-24T09:00:44.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.499+0000 7f9970600640 1 --2- 172.21.15.83:0/3451492390 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f99500a3d20 0x7f99500a61e0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:44.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.499+0000 7f9970600640 1 --2- 172.21.15.83:0/3451492390 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f996819a7f0 0x7f996819aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:44.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.499+0000 7f9970600640 1 --2- 172.21.15.83:0/3451492390 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9968106210 0x7f99681a0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:44.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.499+0000 7f9970600640 1 --2- 172.21.15.83:0/3451492390 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9968101140 0x7f99681a0680 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:44.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.499+0000 7f9970600640 1 -- 172.21.15.83:0/3451492390 >> 172.21.15.83:0/3451492390 conn(0x7f9968078000 msgr2=0x7f996810c6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:44.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.499+0000 7f9970600640 1 -- 172.21.15.83:0/3451492390 shutdown_connections 2024-04-24T09:00:44.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.499+0000 7f9970600640 1 -- 172.21.15.83:0/3451492390 wait complete. 2024-04-24T09:00:44.517 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T09:00:44.601 INFO:tasks.workunit.client.0.smithi083.stdout: 1 52013 0.62 MB/sec execute 318 sec latency 60.622 ms 2024-04-24T09:00:44.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.727+0000 7f5bf2c00640 1 -- 172.21.15.83:0/2956311 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5bec1044b0 msgr2=0x7f5bec104890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:44.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.727+0000 7f5bf2c00640 1 --2- 172.21.15.83:0/2956311 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5bec1044b0 0x7f5bec104890 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f5be4009ca0 tx=0x7f5be402a8c0 comp rx=0 tx=0).stop 2024-04-24T09:00:44.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.727+0000 7f5bf2c00640 1 -- 172.21.15.83:0/2956311 shutdown_connections 2024-04-24T09:00:44.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.727+0000 7f5bf2c00640 1 --2- 172.21.15.83:0/2956311 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5bec109580 0x7f5bec111150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:44.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.727+0000 7f5bf2c00640 1 --2- 172.21.15.83:0/2956311 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5bec104e60 0x7f5bec108ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:44.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.727+0000 7f5bf2c00640 1 --2- 172.21.15.83:0/2956311 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5bec1044b0 0x7f5bec104890 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:44.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.727+0000 7f5bf2c00640 1 -- 172.21.15.83:0/2956311 >> 172.21.15.83:0/2956311 conn(0x7f5bec0ffd90 msgr2=0x7f5bec1021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:44.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.728+0000 7f5bf2c00640 1 -- 172.21.15.83:0/2956311 shutdown_connections 2024-04-24T09:00:44.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.728+0000 7f5bf2c00640 1 -- 172.21.15.83:0/2956311 wait complete. 2024-04-24T09:00:44.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.729+0000 7f5bf2c00640 1 Processor -- start 2024-04-24T09:00:44.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.729+0000 7f5bf2c00640 1 -- start start 2024-04-24T09:00:44.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.729+0000 7f5bf2c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5bec1044b0 0x7f5bec1a07c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:44.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.729+0000 7f5bf2c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5bec109580 0x7f5bec1a0d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:44.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.730+0000 7f5bf2c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5bec19a9c0 0x7f5bec19ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:44.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.730+0000 7f5bf2c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5bec114540 con 0x7f5bec1044b0 2024-04-24T09:00:44.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.730+0000 7f5bf2c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5bec1143c0 con 0x7f5bec19a9c0 2024-04-24T09:00:44.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.730+0000 7f5bf2c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5bec1146c0 con 0x7f5bec109580 2024-04-24T09:00:44.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.730+0000 7f5bebe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5bec1044b0 0x7f5bec1a07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:44.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.730+0000 7f5bebe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5bec1044b0 0x7f5bec1a07c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48696/0 (socket says 172.21.15.83:48696) 2024-04-24T09:00:44.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.730+0000 7f5bebe00640 1 -- 172.21.15.83:0/961638429 learned_addr learned my addr 172.21.15.83:0/961638429 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:00:44.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.730+0000 7f5bf0c00640 1 --2- 172.21.15.83:0/961638429 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5bec19a9c0 0x7f5bec19ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:44.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.730+0000 7f5beb400640 1 --2- 172.21.15.83:0/961638429 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5bec109580 0x7f5bec1a0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:44.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.730+0000 7f5bebe00640 1 -- 172.21.15.83:0/961638429 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5bec109580 msgr2=0x7f5bec1a0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:44.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.730+0000 7f5bebe00640 1 --2- 172.21.15.83:0/961638429 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5bec109580 0x7f5bec1a0d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:44.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.730+0000 7f5bebe00640 1 -- 172.21.15.83:0/961638429 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5bec19a9c0 msgr2=0x7f5bec19ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:44.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.730+0000 7f5bebe00640 1 --2- 172.21.15.83:0/961638429 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5bec19a9c0 0x7f5bec19ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:44.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.730+0000 7f5bebe00640 1 -- 172.21.15.83:0/961638429 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bec19b4e0 con 0x7f5bec1044b0 2024-04-24T09:00:44.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.731+0000 7f5bebe00640 1 --2- 172.21.15.83:0/961638429 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5bec1044b0 0x7f5bec1a07c0 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f5be4031040 tx=0x7f5be4032ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:44.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.731+0000 7f5be8c00640 1 -- 172.21.15.83:0/961638429 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be4003cf0 con 0x7f5bec1044b0 2024-04-24T09:00:44.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.731+0000 7f5be8c00640 1 -- 172.21.15.83:0/961638429 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5be40044f0 con 0x7f5bec1044b0 2024-04-24T09:00:44.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.731+0000 7f5bf2c00640 1 -- 172.21.15.83:0/961638429 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bec19b770 con 0x7f5bec1044b0 2024-04-24T09:00:44.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.731+0000 7f5be8c00640 1 -- 172.21.15.83:0/961638429 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be4030bd0 con 0x7f5bec1044b0 2024-04-24T09:00:44.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.731+0000 7f5bf2c00640 1 -- 172.21.15.83:0/961638429 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bec1a7970 con 0x7f5bec1044b0 2024-04-24T09:00:44.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.734+0000 7f5be8c00640 1 -- 172.21.15.83:0/961638429 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5be4004040 con 0x7f5bec1044b0 2024-04-24T09:00:44.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.735+0000 7f5bf5a1b740 1 Processor -- start 2024-04-24T09:00:44.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.735+0000 7f5bf5a1b740 1 -- start start 2024-04-24T09:00:44.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.735+0000 7f5bf5a1b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560c5f074950 0x560c5f074d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:44.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.735+0000 7f5bf5a1b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560c5f075270 0x560c5f07f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:44.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.735+0000 7f5bf5a1b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c5f07f940 0x560c5f081d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:44.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.735+0000 7f5bf5a1b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x560c5efda1d0 con 0x560c5f075270 2024-04-24T09:00:44.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.735+0000 7f5bf5a1b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560c5ef80450 con 0x560c5f07f940 2024-04-24T09:00:44.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.735+0000 7f5bf5a1b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x560c5efd6190 con 0x560c5f074950 2024-04-24T09:00:44.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.735+0000 7f5bebe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560c5f075270 0x560c5f07f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:44.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.736+0000 7f5bebe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560c5f075270 0x560c5f07f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48698/0 (socket says 172.21.15.83:48698) 2024-04-24T09:00:44.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.736+0000 7f5bebe00640 1 -- 172.21.15.83:0/4132503612 learned_addr learned my addr 172.21.15.83:0/4132503612 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:00:44.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.736+0000 7f5bf0c00640 1 --2- 172.21.15.83:0/4132503612 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560c5f074950 0x560c5f074d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:44.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.736+0000 7f5beb400640 1 --2- 172.21.15.83:0/4132503612 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c5f07f940 0x560c5f081d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:44.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.736+0000 7f5bebe00640 1 -- 172.21.15.83:0/4132503612 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560c5f074950 msgr2=0x560c5f074d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:44.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.736+0000 7f5bebe00640 1 --2- 172.21.15.83:0/4132503612 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560c5f074950 0x560c5f074d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:44.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.736+0000 7f5bebe00640 1 -- 172.21.15.83:0/4132503612 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c5f07f940 msgr2=0x560c5f081d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:44.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.736+0000 7f5bebe00640 1 --2- 172.21.15.83:0/4132503612 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c5f07f940 0x560c5f081d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:44.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.736+0000 7f5bebe00640 1 -- 172.21.15.83:0/4132503612 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c5f03d310 con 0x560c5f075270 2024-04-24T09:00:44.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.736+0000 7f5be8c00640 1 --2- 172.21.15.83:0/961638429 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f5bd00a3d20 0x7f5bd00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:44.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.737+0000 7f5bebe00640 1 --2- 172.21.15.83:0/4132503612 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560c5f075270 0x560c5f07f400 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f5be40ef570 tx=0x7f5be40f5f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ba8ff5db988b243 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:44.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.737+0000 7f5bf0c00640 1 --2- 172.21.15.83:0/961638429 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f5bd00a3d20 0x7f5bd00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:44.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.737+0000 7f5be8c00640 1 -- 172.21.15.83:0/961638429 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f5be40f2250 con 0x7f5bec1044b0 2024-04-24T09:00:44.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.737+0000 7f5be1200640 1 -- 172.21.15.83:0/4132503612 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be40ba120 con 0x560c5f075270 2024-04-24T09:00:44.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.737+0000 7f5be1200640 1 -- 172.21.15.83:0/4132503612 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5be4036020 con 0x560c5f075270 2024-04-24T09:00:44.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.737+0000 7f5be1200640 1 -- 172.21.15.83:0/4132503612 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be40bd7e0 con 0x560c5f075270 2024-04-24T09:00:44.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.738+0000 7f5bf5a1b740 1 -- 172.21.15.83:0/4132503612 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560c5f075270 msgr2=0x560c5f07f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:44.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.738+0000 7f5bf5a1b740 1 --2- 172.21.15.83:0/4132503612 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560c5f075270 0x560c5f07f400 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f5be40ef570 tx=0x7f5be40f5f90 comp rx=0 tx=0).stop 2024-04-24T09:00:44.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.738+0000 7f5bf0c00640 1 --2- 172.21.15.83:0/961638429 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f5bd00a3d20 0x7f5bd00a61e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc005fd0 tx=0x7f5bdc00b530 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:44.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.738+0000 7f5bf5a1b740 1 -- 172.21.15.83:0/4132503612 shutdown_connections 2024-04-24T09:00:44.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.738+0000 7f5bf5a1b740 1 --2- 172.21.15.83:0/4132503612 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c5f07f940 0x560c5f081d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:44.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.738+0000 7f5bf5a1b740 1 --2- 172.21.15.83:0/4132503612 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560c5f075270 0x560c5f07f400 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:44.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.738+0000 7f5bf5a1b740 1 --2- 172.21.15.83:0/4132503612 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560c5f074950 0x560c5f074d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:44.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.738+0000 7f5bf5a1b740 1 -- 172.21.15.83:0/4132503612 >> 172.21.15.83:0/4132503612 conn(0x560c5f070c70 msgr2=0x560c5f071050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:44.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.738+0000 7f5bf5a1b740 1 -- 172.21.15.83:0/4132503612 shutdown_connections 2024-04-24T09:00:44.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.738+0000 7f5bf5a1b740 1 -- 172.21.15.83:0/4132503612 wait complete. 2024-04-24T09:00:44.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.738+0000 7f5bf5a1b740 1 -- start start 2024-04-24T09:00:44.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.739+0000 7f5bf5a1b740 1 Processor -- start 2024-04-24T09:00:44.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.739+0000 7f5bf5a1b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560c5f074950 0x560c5f07f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:44.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.739+0000 7f5bf5a1b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560c5f075270 0x560c5f07c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:44.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.739+0000 7f5bf5a1b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c5f07f940 0x560c5f07cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:44.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.739+0000 7f5bf5a1b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x560c5efda1d0 con 0x560c5f074950 2024-04-24T09:00:44.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.739+0000 7f5bf5a1b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560c5ef93810 con 0x560c5f07f940 2024-04-24T09:00:44.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.739+0000 7f5bf5a1b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x560c5f0824f0 con 0x560c5f075270 2024-04-24T09:00:44.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.739+0000 7f5bf5a1b740 10 client.0 _ll_register_callbacks cb 0x560c5ef61140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:00:44.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.739+0000 7f5bebe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560c5f074950 0x560c5f07f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:44.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.739+0000 7f5bebe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560c5f074950 0x560c5f07f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48706/0 (socket says 172.21.15.83:48706) 2024-04-24T09:00:44.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.739+0000 7f5bebe00640 1 -- 172.21.15.83:0/370561203 learned_addr learned my addr 172.21.15.83:0/370561203 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:00:44.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.740+0000 7f5bf0c00640 1 --2- 172.21.15.83:0/370561203 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c5f07f940 0x560c5f07cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:44.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.740+0000 7f5beb400640 1 --2- 172.21.15.83:0/370561203 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560c5f075270 0x560c5f07c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:44.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.740+0000 7f5bebe00640 1 -- 172.21.15.83:0/370561203 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560c5f075270 msgr2=0x560c5f07c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:44.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.740+0000 7f5bebe00640 1 --2- 172.21.15.83:0/370561203 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560c5f075270 0x560c5f07c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:44.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.740+0000 7f5bebe00640 1 -- 172.21.15.83:0/370561203 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c5f07f940 msgr2=0x560c5f07cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:44.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.740+0000 7f5bebe00640 1 --2- 172.21.15.83:0/370561203 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c5f07f940 0x560c5f07cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:44.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.740+0000 7f5bebe00640 1 -- 172.21.15.83:0/370561203 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c5efa43f0 con 0x560c5f074950 2024-04-24T09:00:44.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.740+0000 7f5bf0c00640 1 --2- 172.21.15.83:0/370561203 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c5f07f940 0x560c5f07cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:00:44.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.740+0000 7f5bebe00640 1 --2- 172.21.15.83:0/370561203 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560c5f074950 0x560c5f07f360 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f5be402ae80 tx=0x7f5be40f5ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aad64785088c77df server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:44.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.740+0000 7f5bd5600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:00:44.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.740+0000 7f5bd5600640 1 -- 172.21.15.83:0/370561203 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be40d7060 con 0x560c5f074950 2024-04-24T09:00:44.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.740+0000 7f5bd5600640 1 -- 172.21.15.83:0/370561203 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f5be40046a0 con 0x560c5f074950 2024-04-24T09:00:44.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.740+0000 7f5bf5a1b740 1 -- 172.21.15.83:0/370561203 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5be4036020 con 0x560c5f074950 2024-04-24T09:00:44.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.741+0000 7f5bd5600640 1 -- 172.21.15.83:0/370561203 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f5be40bd9c0 con 0x560c5f074950 2024-04-24T09:00:44.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.741+0000 7f5bf5a1b740 10 client.18906 fetch_fsmap learned FSMap version 35 2024-04-24T09:00:44.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.741+0000 7f5bf5a1b740 1 -- 172.21.15.83:0/370561203 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x560c5ecec3f0 con 0x560c5f074950 2024-04-24T09:00:44.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.741+0000 7f5bd5600640 1 -- 172.21.15.83:0/370561203 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5be4036020 con 0x560c5f074950 2024-04-24T09:00:44.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.741+0000 7f5bd5600640 1 -- 172.21.15.83:0/370561203 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7f5be40ba4f0 con 0x560c5f074950 2024-04-24T09:00:44.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.741+0000 7f5bf5a1b740 10 client.18906 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T09:00:44.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.741+0000 7f5bf5a1b740 10 client.18906 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-24T09:00:44.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.741+0000 7f5bf5a1b740 20 client.18906 populate_metadata read hostname 'smithi083' 2024-04-24T09:00:44.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.741+0000 7f5bf5a1b740 1 --2- 172.21.15.83:0/370561203 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x560c5f08f5f0 0x560c5f091a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:44.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.741+0000 7f5bf5a1b740 4 client.18906 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:00:44.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.741+0000 7f5bf5a1b740 1 -- 172.21.15.83:0/370561203 --> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560c5efe7bf0 con 0x560c5f08f5f0 2024-04-24T09:00:44.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.742+0000 7f5beb400640 1 --2- 172.21.15.83:0/370561203 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x560c5f08f5f0 0x560c5f091a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:44.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.743+0000 7f5beb400640 1 --2- 172.21.15.83:0/370561203 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x560c5f08f5f0 0x560c5f091a00 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=80c92c2dabc45d4c server_cookie=bb91c0be4dc60028 in_seq=0 out_seq=0 2024-04-24T09:00:44.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.743+0000 7f5bd5600640 10 client.18906 ms_handle_connect on v2:172.21.15.138:6836/900566754 2024-04-24T09:00:44.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.745+0000 7f5bd5600640 1 -- 172.21.15.83:0/370561203 <== mds.1 v2:172.21.15.138:6836/900566754 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560c5efe7bf0 con 0x560c5f08f5f0 2024-04-24T09:00:44.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.745+0000 7f5bd5600640 10 client.18906 handle_command_reply: tid=0 2024-04-24T09:00:44.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.745+0000 7f5bf5a1b740 1 client.18906 shutdown 2024-04-24T09:00:44.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.745+0000 7f5bf5a1b740 1 -- 172.21.15.83:0/370561203 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560c5f074950 msgr2=0x560c5f07f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:44.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.745+0000 7f5bf5a1b740 1 --2- 172.21.15.83:0/370561203 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560c5f074950 0x560c5f07f360 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f5be402ae80 tx=0x7f5be40f5ea0 comp rx=0 tx=0).stop 2024-04-24T09:00:44.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.745+0000 7f5bf5a1b740 1 -- 172.21.15.83:0/370561203 shutdown_connections 2024-04-24T09:00:44.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.745+0000 7f5bf5a1b740 1 --2- 172.21.15.83:0/370561203 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c5f07f940 0x560c5f07cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:44.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.745+0000 7f5bf5a1b740 1 --2- 172.21.15.83:0/370561203 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x560c5f08f5f0 0x560c5f091a00 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:44.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.745+0000 7f5bf5a1b740 1 --2- 172.21.15.83:0/370561203 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560c5f075270 0x560c5f07c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:44.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.745+0000 7f5bf5a1b740 1 --2- 172.21.15.83:0/370561203 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560c5f074950 0x560c5f07f360 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:44.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.745+0000 7f5bf5a1b740 1 -- 172.21.15.83:0/370561203 >> 172.21.15.83:0/370561203 conn(0x560c5f070c70 msgr2=0x560c5f076950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:44.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.745+0000 7f5bd5600640 0 client.18906 ms_handle_reset on v2:172.21.15.138:6836/900566754 2024-04-24T09:00:44.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.745+0000 7f5bf5a1b740 1 -- 172.21.15.83:0/370561203 shutdown_connections 2024-04-24T09:00:44.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.745+0000 7f5bf5a1b740 1 -- 172.21.15.83:0/370561203 wait complete. 2024-04-24T09:00:44.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.745+0000 7f5bf5a1b740 20 client.18906 trim_cache size 0 max 16384 2024-04-24T09:00:44.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.763+0000 7f5bf5a1b740 1 Processor -- start 2024-04-24T09:00:44.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.763+0000 7f5bf5a1b740 1 -- start start 2024-04-24T09:00:44.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.763+0000 7f5bf5a1b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560c5f074950 0x560c5f0770b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:44.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.763+0000 7f5bf5a1b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c5f075270 0x560c5f07bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:44.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.763+0000 7f5bf5a1b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560c5f07f940 0x560c5f07ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:44.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.763+0000 7f5bf5a1b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5bec1b1850 con 0x560c5f07f940 2024-04-24T09:00:44.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.763+0000 7f5bf5a1b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5bec1b3390 con 0x560c5f075270 2024-04-24T09:00:44.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.763+0000 7f5bf5a1b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5bec1b1f20 con 0x560c5f074950 2024-04-24T09:00:44.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.763+0000 7f5bf0c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560c5f07f940 0x560c5f07ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:44.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.764+0000 7f5beb400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c5f075270 0x560c5f07bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:44.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.764+0000 7f5bf0c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560c5f07f940 0x560c5f07ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48718/0 (socket says 172.21.15.83:48718) 2024-04-24T09:00:44.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.764+0000 7f5bf0c00640 1 -- 172.21.15.83:0/3499841917 learned_addr learned my addr 172.21.15.83:0/3499841917 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:00:44.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.764+0000 7f5bebe00640 1 --2- 172.21.15.83:0/3499841917 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560c5f074950 0x560c5f0770b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:44.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.764+0000 7f5bf0c00640 1 -- 172.21.15.83:0/3499841917 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560c5f074950 msgr2=0x560c5f0770b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:44.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.764+0000 7f5bf0c00640 1 --2- 172.21.15.83:0/3499841917 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560c5f074950 0x560c5f0770b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:44.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.764+0000 7f5bf0c00640 1 -- 172.21.15.83:0/3499841917 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c5f075270 msgr2=0x560c5f07bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:44.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.764+0000 7f5bf0c00640 1 --2- 172.21.15.83:0/3499841917 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c5f075270 0x560c5f07bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:44.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.764+0000 7f5bf0c00640 1 -- 172.21.15.83:0/3499841917 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c5edb67d0 con 0x560c5f07f940 2024-04-24T09:00:44.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.764+0000 7f5bf0c00640 1 --2- 172.21.15.83:0/3499841917 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560c5f07f940 0x560c5f07ee90 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc012530 tx=0x7f5bdc014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83d4733ad262912d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:44.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.765+0000 7f5bd7e00640 1 -- 172.21.15.83:0/3499841917 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bdc01f6e0 con 0x560c5f07f940 2024-04-24T09:00:44.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.765+0000 7f5bd7e00640 1 -- 172.21.15.83:0/3499841917 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5bdc023070 con 0x560c5f07f940 2024-04-24T09:00:44.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.765+0000 7f5bd7e00640 1 -- 172.21.15.83:0/3499841917 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bdc01cc20 con 0x560c5f07f940 2024-04-24T09:00:44.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.765+0000 7f5bf5a1b740 1 -- 172.21.15.83:0/3499841917 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560c5f07f940 msgr2=0x560c5f07ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:44.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.765+0000 7f5bf5a1b740 1 --2- 172.21.15.83:0/3499841917 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560c5f07f940 0x560c5f07ee90 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc012530 tx=0x7f5bdc014000 comp rx=0 tx=0).stop 2024-04-24T09:00:44.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.766+0000 7f5bf5a1b740 1 -- 172.21.15.83:0/3499841917 shutdown_connections 2024-04-24T09:00:44.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.766+0000 7f5bf5a1b740 1 --2- 172.21.15.83:0/3499841917 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560c5f07f940 0x560c5f07ee90 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:44.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.766+0000 7f5bf5a1b740 1 --2- 172.21.15.83:0/3499841917 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c5f075270 0x560c5f07bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:44.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.766+0000 7f5bf5a1b740 1 --2- 172.21.15.83:0/3499841917 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560c5f074950 0x560c5f0770b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:44.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.766+0000 7f5bf5a1b740 1 -- 172.21.15.83:0/3499841917 >> 172.21.15.83:0/3499841917 conn(0x560c5f070c70 msgr2=0x560c5f06fdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:44.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.766+0000 7f5bf5a1b740 1 -- 172.21.15.83:0/3499841917 shutdown_connections 2024-04-24T09:00:44.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.766+0000 7f5bf5a1b740 1 -- 172.21.15.83:0/3499841917 wait complete. 2024-04-24T09:00:44.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.766+0000 7f5bf5a1b740 1 -- start start 2024-04-24T09:00:44.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.766+0000 7f5bf5a1b740 1 Processor -- start 2024-04-24T09:00:44.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.767+0000 7f5bf5a1b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560c5f074950 0x560c5f07fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:44.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.767+0000 7f5bf5a1b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c5f075270 0x560c5f0802a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:44.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.767+0000 7f5bf5a1b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560c5f07f940 0x560c5f0807e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:44.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.767+0000 7f5bf5a1b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5bec1b2cc0 con 0x560c5f074950 2024-04-24T09:00:44.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.767+0000 7f5bf5a1b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5bec1b25f0 con 0x560c5f075270 2024-04-24T09:00:44.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.767+0000 7f5bf5a1b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5bec1af850 con 0x560c5f07f940 2024-04-24T09:00:44.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.767+0000 7f5bf5a1b740 10 client.0 _ll_register_callbacks cb 0x7f5be40fa170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:00:44.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.767+0000 7f5bebe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560c5f074950 0x560c5f07fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:44.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.767+0000 7f5bebe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560c5f074950 0x560c5f07fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48720/0 (socket says 172.21.15.83:48720) 2024-04-24T09:00:44.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.767+0000 7f5bebe00640 1 -- 172.21.15.83:0/2800418427 learned_addr learned my addr 172.21.15.83:0/2800418427 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:00:44.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.767+0000 7f5beb400640 1 --2- 172.21.15.83:0/2800418427 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c5f075270 0x560c5f0802a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:44.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.767+0000 7f5bf0c00640 1 --2- 172.21.15.83:0/2800418427 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560c5f07f940 0x560c5f0807e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:44.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.767+0000 7f5bebe00640 1 -- 172.21.15.83:0/2800418427 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560c5f07f940 msgr2=0x560c5f0807e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:44.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.767+0000 7f5bebe00640 1 --2- 172.21.15.83:0/2800418427 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560c5f07f940 0x560c5f0807e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:44.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.767+0000 7f5bebe00640 1 -- 172.21.15.83:0/2800418427 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c5f075270 msgr2=0x560c5f0802a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:44.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.767+0000 7f5bebe00640 1 --2- 172.21.15.83:0/2800418427 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c5f075270 0x560c5f0802a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:44.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.767+0000 7f5bebe00640 1 -- 172.21.15.83:0/2800418427 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c5f0868f0 con 0x560c5f074950 2024-04-24T09:00:44.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.768+0000 7f5bebe00640 1 --2- 172.21.15.83:0/2800418427 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560c5f074950 0x560c5f07fd60 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f5be4006010 tx=0x7f5be40fa220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bef092b3cc9f5514 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:44.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.768+0000 7f5bd5600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:00:44.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.768+0000 7f5bd5600640 1 -- 172.21.15.83:0/2800418427 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be40bdc50 con 0x560c5f074950 2024-04-24T09:00:44.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.768+0000 7f5bd5600640 1 -- 172.21.15.83:0/2800418427 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f5be40ba4f0 con 0x560c5f074950 2024-04-24T09:00:44.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.768+0000 7f5bf5a1b740 1 -- 172.21.15.83:0/2800418427 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5bdc023070 con 0x560c5f074950 2024-04-24T09:00:44.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.768+0000 7f5bd5600640 1 -- 172.21.15.83:0/2800418427 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f5be40ba120 con 0x560c5f074950 2024-04-24T09:00:44.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.768+0000 7f5bf5a1b740 10 client.18912 fetch_fsmap learned FSMap version 35 2024-04-24T09:00:44.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.768+0000 7f5bf5a1b740 1 -- 172.21.15.83:0/2800418427 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x560c5f0868f0 con 0x560c5f074950 2024-04-24T09:00:44.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.769+0000 7f5bd5600640 1 -- 172.21.15.83:0/2800418427 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5bdc023070 con 0x560c5f074950 2024-04-24T09:00:44.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.769+0000 7f5bd5600640 1 -- 172.21.15.83:0/2800418427 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7f5be40be0e0 con 0x560c5f074950 2024-04-24T09:00:44.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.769+0000 7f5bf5a1b740 10 client.18912 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T09:00:44.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.769+0000 7f5bf5a1b740 10 client.18912 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-24T09:00:44.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.769+0000 7f5bf5a1b740 20 client.18912 populate_metadata read hostname 'smithi083' 2024-04-24T09:00:44.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.769+0000 7f5bf5a1b740 1 --2- 172.21.15.83:0/2800418427 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x560c5f0f5b40 0x560c5f0f7f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:44.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.769+0000 7f5bf5a1b740 4 client.18912 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:00:44.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.769+0000 7f5bf5a1b740 1 -- 172.21.15.83:0/2800418427 --> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560c5efe1a60 con 0x560c5f0f5b40 2024-04-24T09:00:44.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.769+0000 7f5beb400640 1 --2- 172.21.15.83:0/2800418427 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x560c5f0f5b40 0x560c5f0f7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:44.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.771+0000 7f5beb400640 1 --2- 172.21.15.83:0/2800418427 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x560c5f0f5b40 0x560c5f0f7f30 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ee25277959f45c09 server_cookie=23f7b560a76b7796 in_seq=0 out_seq=0 2024-04-24T09:00:44.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.771+0000 7f5bd5600640 10 client.18912 ms_handle_connect on v2:172.21.15.138:6836/900566754 2024-04-24T09:00:44.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.771+0000 7f5bd5600640 1 -- 172.21.15.83:0/2800418427 <== mds.1 v2:172.21.15.138:6836/900566754 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560c5efe1a60 con 0x560c5f0f5b40 2024-04-24T09:00:44.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.771+0000 7f5bd5600640 10 client.18912 handle_command_reply: tid=0 2024-04-24T09:00:44.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.771+0000 7f5bf5a1b740 1 client.18912 shutdown 2024-04-24T09:00:44.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.771+0000 7f5bf5a1b740 1 -- 172.21.15.83:0/2800418427 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560c5f074950 msgr2=0x560c5f07fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:44.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.771+0000 7f5bf5a1b740 1 --2- 172.21.15.83:0/2800418427 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560c5f074950 0x560c5f07fd60 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f5be4006010 tx=0x7f5be40fa220 comp rx=0 tx=0).stop 2024-04-24T09:00:44.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.771+0000 7f5bf5a1b740 1 -- 172.21.15.83:0/2800418427 shutdown_connections 2024-04-24T09:00:44.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.771+0000 7f5bf5a1b740 1 --2- 172.21.15.83:0/2800418427 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x560c5f0f5b40 0x560c5f0f7f30 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:44.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.771+0000 7f5bf5a1b740 1 --2- 172.21.15.83:0/2800418427 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560c5f07f940 0x560c5f0807e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:44.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.771+0000 7f5bf5a1b740 1 --2- 172.21.15.83:0/2800418427 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c5f075270 0x560c5f0802a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:44.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.771+0000 7f5bf5a1b740 1 --2- 172.21.15.83:0/2800418427 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560c5f074950 0x560c5f07fd60 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:44.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.771+0000 7f5bd5600640 0 client.18912 ms_handle_reset on v2:172.21.15.138:6836/900566754 2024-04-24T09:00:44.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.771+0000 7f5bf5a1b740 1 -- 172.21.15.83:0/2800418427 >> 172.21.15.83:0/2800418427 conn(0x560c5f070c70 msgr2=0x560c5f07af20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:44.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.772+0000 7f5bf5a1b740 1 -- 172.21.15.83:0/2800418427 shutdown_connections 2024-04-24T09:00:44.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.772+0000 7f5bf5a1b740 1 -- 172.21.15.83:0/2800418427 wait complete. 2024-04-24T09:00:44.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.772+0000 7f5bf5a1b740 20 client.18912 trim_cache size 0 max 16384 2024-04-24T09:00:44.774 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:00:44.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.772+0000 7f5bf2c00640 1 -- 172.21.15.83:0/961638429 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f5bd00a3d20 msgr2=0x7f5bd00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:44.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.772+0000 7f5bf2c00640 1 --2- 172.21.15.83:0/961638429 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f5bd00a3d20 0x7f5bd00a61e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc005fd0 tx=0x7f5bdc00b530 comp rx=0 tx=0).stop 2024-04-24T09:00:44.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.772+0000 7f5bf2c00640 1 -- 172.21.15.83:0/961638429 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5bec1044b0 msgr2=0x7f5bec1a07c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:44.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.772+0000 7f5bf2c00640 1 --2- 172.21.15.83:0/961638429 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5bec1044b0 0x7f5bec1a07c0 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f5be4031040 tx=0x7f5be4032ff0 comp rx=0 tx=0).stop 2024-04-24T09:00:44.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.773+0000 7f5bf2c00640 1 -- 172.21.15.83:0/961638429 shutdown_connections 2024-04-24T09:00:44.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.773+0000 7f5bf2c00640 1 --2- 172.21.15.83:0/961638429 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f5bd00a3d20 0x7f5bd00a61e0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:44.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.773+0000 7f5bf2c00640 1 --2- 172.21.15.83:0/961638429 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5bec19a9c0 0x7f5bec19ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:44.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.773+0000 7f5bf2c00640 1 --2- 172.21.15.83:0/961638429 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5bec109580 0x7f5bec1a0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:44.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.773+0000 7f5bf2c00640 1 --2- 172.21.15.83:0/961638429 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5bec1044b0 0x7f5bec1a07c0 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:44.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.773+0000 7f5bf2c00640 1 -- 172.21.15.83:0/961638429 >> 172.21.15.83:0/961638429 conn(0x7f5bec0ffd90 msgr2=0x7f5bec101120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:44.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.773+0000 7f5bf2c00640 1 -- 172.21.15.83:0/961638429 shutdown_connections 2024-04-24T09:00:44.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.773+0000 7f5bf2c00640 1 -- 172.21.15.83:0/961638429 wait complete. 2024-04-24T09:00:44.792 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T09:00:44.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.990+0000 7f16a1a00640 1 -- 172.21.15.83:0/1884758698 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f169c103060 msgr2=0x7f169c103440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:44.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.990+0000 7f16a1a00640 1 --2- 172.21.15.83:0/1884758698 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f169c103060 0x7f169c103440 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f168c009900 tx=0x7f168c02a480 comp rx=0 tx=0).stop 2024-04-24T09:00:44.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.990+0000 7f16a1a00640 1 -- 172.21.15.83:0/1884758698 shutdown_connections 2024-04-24T09:00:44.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.990+0000 7f16a1a00640 1 --2- 172.21.15.83:0/1884758698 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f169c10c480 0x7f169c10e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:44.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.990+0000 7f16a1a00640 1 --2- 172.21.15.83:0/1884758698 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f169c103980 0x7f169c10be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:44.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.990+0000 7f16a1a00640 1 --2- 172.21.15.83:0/1884758698 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f169c103060 0x7f169c103440 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:44.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.990+0000 7f16a1a00640 1 -- 172.21.15.83:0/1884758698 >> 172.21.15.83:0/1884758698 conn(0x7f169c0fc880 msgr2=0x7f169c0feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:44.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.991+0000 7f16a1a00640 1 -- 172.21.15.83:0/1884758698 shutdown_connections 2024-04-24T09:00:44.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.991+0000 7f16a1a00640 1 -- 172.21.15.83:0/1884758698 wait complete. 2024-04-24T09:00:44.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.992+0000 7f16a1a00640 1 Processor -- start 2024-04-24T09:00:44.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.992+0000 7f16a1a00640 1 -- start start 2024-04-24T09:00:44.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.992+0000 7f16a1a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f169c103060 0x7f169c1a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:44.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.992+0000 7f16a1a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f169c103980 0x7f169c1a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:44.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.992+0000 7f16a1a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f169c10c480 0x7f169c19a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:44.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.992+0000 7f16a1a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f169c1121c0 con 0x7f169c103980 2024-04-24T09:00:44.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.992+0000 7f16a1a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f169c112040 con 0x7f169c103060 2024-04-24T09:00:44.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.992+0000 7f16a1a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f169c112340 con 0x7f169c10c480 2024-04-24T09:00:44.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.992+0000 7f169b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f169c103980 0x7f169c1a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:44.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.993+0000 7f169b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f169c103980 0x7f169c1a0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48750/0 (socket says 172.21.15.83:48750) 2024-04-24T09:00:44.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.993+0000 7f169b400640 1 -- 172.21.15.83:0/4140566432 learned_addr learned my addr 172.21.15.83:0/4140566432 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:00:44.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.993+0000 7f169be00640 1 --2- 172.21.15.83:0/4140566432 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f169c103060 0x7f169c1a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:44.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.993+0000 7f16a1000640 1 --2- 172.21.15.83:0/4140566432 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f169c10c480 0x7f169c19a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:44.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.993+0000 7f169b400640 1 -- 172.21.15.83:0/4140566432 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f169c10c480 msgr2=0x7f169c19a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:44.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.993+0000 7f169b400640 1 --2- 172.21.15.83:0/4140566432 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f169c10c480 0x7f169c19a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:44.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.993+0000 7f169b400640 1 -- 172.21.15.83:0/4140566432 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f169c103060 msgr2=0x7f169c1a0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:44.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.993+0000 7f169b400640 1 --2- 172.21.15.83:0/4140566432 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f169c103060 0x7f169c1a0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:44.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.993+0000 7f169b400640 1 -- 172.21.15.83:0/4140566432 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f169c19b000 con 0x7f169c103980 2024-04-24T09:00:44.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.993+0000 7f169b400640 1 --2- 172.21.15.83:0/4140566432 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f169c103980 0x7f169c1a0bc0 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f168800c940 tx=0x7f168800ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:44.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.994+0000 7f1698c00640 1 -- 172.21.15.83:0/4140566432 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1688004220 con 0x7f169c103980 2024-04-24T09:00:44.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.994+0000 7f16a1a00640 1 -- 172.21.15.83:0/4140566432 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f169c19b2f0 con 0x7f169c103980 2024-04-24T09:00:44.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.994+0000 7f1698c00640 1 -- 172.21.15.83:0/4140566432 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f16880043c0 con 0x7f169c103980 2024-04-24T09:00:44.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.994+0000 7f1698c00640 1 -- 172.21.15.83:0/4140566432 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f168801d060 con 0x7f169c103980 2024-04-24T09:00:44.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.994+0000 7f16a1a00640 1 -- 172.21.15.83:0/4140566432 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f169c1a7600 con 0x7f169c103980 2024-04-24T09:00:44.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.997+0000 7f16a5fe8740 1 Processor -- start 2024-04-24T09:00:44.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.998+0000 7f16a5fe8740 1 -- start start 2024-04-24T09:00:44.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.998+0000 7f16a5fe8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ee990bb40 0x559ee990bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:44.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.998+0000 7f1698c00640 1 -- 172.21.15.83:0/4140566432 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f16880079b0 con 0x7f169c103980 2024-04-24T09:00:44.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.998+0000 7f16a5fe8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559ee990c460 0x559ee99165f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:44.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.998+0000 7f16a5fe8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ee9916b30 0x559ee9918f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:44.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.998+0000 7f16a5fe8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559ee98711d0 con 0x559ee990bb40 2024-04-24T09:00:44.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.998+0000 7f16a5fe8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559ee9817450 con 0x559ee9916b30 2024-04-24T09:00:44.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.998+0000 7f16a5fe8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559ee986d190 con 0x559ee990c460 2024-04-24T09:00:44.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.998+0000 7f16a1000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ee990bb40 0x559ee990bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:44.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.998+0000 7f16a1000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ee990bb40 0x559ee990bf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48754/0 (socket says 172.21.15.83:48754) 2024-04-24T09:00:44.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.998+0000 7f16a1000640 1 -- 172.21.15.83:0/2873891641 learned_addr learned my addr 172.21.15.83:0/2873891641 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:00:44.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.999+0000 7f169be00640 1 --2- 172.21.15.83:0/2873891641 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559ee990c460 0x559ee99165f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:44.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.999+0000 7f16a1000640 1 -- 172.21.15.83:0/2873891641 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559ee990c460 msgr2=0x559ee99165f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:44.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.999+0000 7f169b400640 1 --2- 172.21.15.83:0/2873891641 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ee9916b30 0x559ee9918f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:44.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.999+0000 7f16a1000640 1 --2- 172.21.15.83:0/2873891641 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559ee990c460 0x559ee99165f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:44.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.999+0000 7f16a1000640 1 -- 172.21.15.83:0/2873891641 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ee9916b30 msgr2=0x559ee9918f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:45.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.999+0000 7f16a1000640 1 --2- 172.21.15.83:0/2873891641 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ee9916b30 0x559ee9918f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:45.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.999+0000 7f16a1000640 1 -- 172.21.15.83:0/2873891641 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ee98d4310 con 0x559ee990bb40 2024-04-24T09:00:45.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.999+0000 7f16a1000640 1 --2- 172.21.15.83:0/2873891641 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ee990bb40 0x559ee990bf20 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f1694009f20 tx=0x7f169400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a78c08fe97178e1e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:45.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.999+0000 7f1691600640 1 -- 172.21.15.83:0/2873891641 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f169401a070 con 0x559ee990bb40 2024-04-24T09:00:45.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:44.999+0000 7f1691600640 1 -- 172.21.15.83:0/2873891641 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f16940158c0 con 0x559ee990bb40 2024-04-24T09:00:45.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.000+0000 7f1691600640 1 -- 172.21.15.83:0/2873891641 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f169400a820 con 0x559ee990bb40 2024-04-24T09:00:45.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.000+0000 7f1698c00640 1 --2- 172.21.15.83:0/4140566432 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f16800a3d20 0x7f16800a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:45.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.000+0000 7f1698c00640 1 -- 172.21.15.83:0/4140566432 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f16880d05e0 con 0x7f169c103980 2024-04-24T09:00:45.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.000+0000 7f169be00640 1 --2- 172.21.15.83:0/4140566432 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f16800a3d20 0x7f16800a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:45.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.000+0000 7f16a5fe8740 1 -- 172.21.15.83:0/2873891641 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ee990bb40 msgr2=0x559ee990bf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:45.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.000+0000 7f16a5fe8740 1 --2- 172.21.15.83:0/2873891641 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ee990bb40 0x559ee990bf20 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f1694009f20 tx=0x7f169400c8b0 comp rx=0 tx=0).stop 2024-04-24T09:00:45.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.001+0000 7f16a5fe8740 1 -- 172.21.15.83:0/2873891641 shutdown_connections 2024-04-24T09:00:45.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.001+0000 7f16a5fe8740 1 --2- 172.21.15.83:0/2873891641 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ee9916b30 0x559ee9918f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:45.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.001+0000 7f16a5fe8740 1 --2- 172.21.15.83:0/2873891641 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559ee990c460 0x559ee99165f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:45.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.001+0000 7f16a5fe8740 1 --2- 172.21.15.83:0/2873891641 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ee990bb40 0x559ee990bf20 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:45.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.001+0000 7f16a5fe8740 1 -- 172.21.15.83:0/2873891641 >> 172.21.15.83:0/2873891641 conn(0x559ee9907e60 msgr2=0x559ee9908240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:45.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.001+0000 7f16a5fe8740 1 -- 172.21.15.83:0/2873891641 shutdown_connections 2024-04-24T09:00:45.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.001+0000 7f16a5fe8740 1 -- 172.21.15.83:0/2873891641 wait complete. 2024-04-24T09:00:45.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.001+0000 7f169be00640 1 --2- 172.21.15.83:0/4140566432 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f16800a3d20 0x7f16800a61e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f168c000c00 tx=0x7f168c031400 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:45.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.001+0000 7f16a5fe8740 1 -- start start 2024-04-24T09:00:45.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.002+0000 7f16a5fe8740 1 Processor -- start 2024-04-24T09:00:45.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.002+0000 7f16a5fe8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ee990bb40 0x559ee99165f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:45.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.002+0000 7f16a5fe8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ee990c460 0x559ee99138e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:45.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.002+0000 7f16a5fe8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559ee9916b30 0x559ee9914020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:45.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.002+0000 7f16a5fe8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559ee99199c0 con 0x559ee990bb40 2024-04-24T09:00:45.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.002+0000 7f16a5fe8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559ee982a810 con 0x559ee990c460 2024-04-24T09:00:45.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.002+0000 7f16a5fe8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559ee9919b40 con 0x559ee9916b30 2024-04-24T09:00:45.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.002+0000 7f16a5fe8740 10 client.0 _ll_register_callbacks cb 0x559ee97f8140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:00:45.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.003+0000 7f169be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ee990bb40 0x559ee99165f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:45.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.003+0000 7f169be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ee990bb40 0x559ee99165f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48756/0 (socket says 172.21.15.83:48756) 2024-04-24T09:00:45.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.003+0000 7f169be00640 1 -- 172.21.15.83:0/2329164632 learned_addr learned my addr 172.21.15.83:0/2329164632 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:00:45.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.003+0000 7f169b400640 1 --2- 172.21.15.83:0/2329164632 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ee990c460 0x559ee99138e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:45.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.003+0000 7f16a1000640 1 --2- 172.21.15.83:0/2329164632 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559ee9916b30 0x559ee9914020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:45.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.003+0000 7f169be00640 1 -- 172.21.15.83:0/2329164632 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559ee9916b30 msgr2=0x559ee9914020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:45.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.003+0000 7f169be00640 1 --2- 172.21.15.83:0/2329164632 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559ee9916b30 0x559ee9914020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:45.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.003+0000 7f169be00640 1 -- 172.21.15.83:0/2329164632 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ee990c460 msgr2=0x559ee99138e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:45.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.003+0000 7f169be00640 1 --2- 172.21.15.83:0/2329164632 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ee990c460 0x559ee99138e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:45.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.003+0000 7f169be00640 1 -- 172.21.15.83:0/2329164632 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ee983b3f0 con 0x559ee990bb40 2024-04-24T09:00:45.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.004+0000 7f169be00640 1 --2- 172.21.15.83:0/2329164632 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ee990bb40 0x559ee99165f0 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f168c009900 tx=0x7f168c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90ac398af33d231f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:45.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.004+0000 7f1686000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:00:45.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.004+0000 7f1686000640 1 -- 172.21.15.83:0/2329164632 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f168c04c070 con 0x559ee990bb40 2024-04-24T09:00:45.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.004+0000 7f1686000640 1 -- 172.21.15.83:0/2329164632 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f168c031e00 con 0x559ee990bb40 2024-04-24T09:00:45.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.004+0000 7f16a5fe8740 1 -- 172.21.15.83:0/2329164632 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16940158c0 con 0x559ee990bb40 2024-04-24T09:00:45.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.005+0000 7f1686000640 1 -- 172.21.15.83:0/2329164632 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f168c058050 con 0x559ee990bb40 2024-04-24T09:00:45.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.005+0000 7f16a5fe8740 10 client.18924 fetch_fsmap learned FSMap version 35 2024-04-24T09:00:45.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.005+0000 7f16a5fe8740 1 -- 172.21.15.83:0/2329164632 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x559ee95833f0 con 0x559ee990bb40 2024-04-24T09:00:45.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.005+0000 7f1686000640 1 -- 172.21.15.83:0/2329164632 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f16940158c0 con 0x559ee990bb40 2024-04-24T09:00:45.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.005+0000 7f1686000640 1 -- 172.21.15.83:0/2329164632 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7f168c03b430 con 0x559ee990bb40 2024-04-24T09:00:45.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.005+0000 7f16a5fe8740 10 client.18924 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T09:00:45.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.005+0000 7f16a5fe8740 10 client.18924 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:00:45.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.005+0000 7f16a5fe8740 20 client.18924 populate_metadata read hostname 'smithi083' 2024-04-24T09:00:45.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.005+0000 7f16a5fe8740 1 --2- 172.21.15.83:0/2329164632 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x559ee992f510 0x559ee9931960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:45.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.005+0000 7f16a5fe8740 4 client.18924 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:00:45.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.005+0000 7f16a5fe8740 1 -- 172.21.15.83:0/2329164632 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559ee987ebf0 con 0x559ee992f510 2024-04-24T09:00:45.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.006+0000 7f169b400640 1 --2- 172.21.15.83:0/2329164632 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x559ee992f510 0x559ee9931960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:45.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.007+0000 7f169b400640 1 --2- 172.21.15.83:0/2329164632 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x559ee992f510 0x559ee9931960 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bcae65344835ed68 server_cookie=d774a40477a9244b in_seq=0 out_seq=0 2024-04-24T09:00:45.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.007+0000 7f1686000640 10 client.18924 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:00:45.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.012+0000 7f1686000640 1 -- 172.21.15.83:0/2329164632 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559ee987ebf0 con 0x559ee992f510 2024-04-24T09:00:45.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.012+0000 7f1686000640 10 client.18924 handle_command_reply: tid=0 2024-04-24T09:00:45.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.012+0000 7f16a5fe8740 1 client.18924 shutdown 2024-04-24T09:00:45.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.012+0000 7f16a5fe8740 1 -- 172.21.15.83:0/2329164632 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ee990bb40 msgr2=0x559ee99165f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:45.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.012+0000 7f16a5fe8740 1 --2- 172.21.15.83:0/2329164632 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ee990bb40 0x559ee99165f0 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f168c009900 tx=0x7f168c009260 comp rx=0 tx=0).stop 2024-04-24T09:00:45.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.012+0000 7f16a5fe8740 1 -- 172.21.15.83:0/2329164632 shutdown_connections 2024-04-24T09:00:45.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.012+0000 7f16a5fe8740 1 --2- 172.21.15.83:0/2329164632 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x559ee992f510 0x559ee9931960 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:45.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.012+0000 7f16a5fe8740 1 --2- 172.21.15.83:0/2329164632 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559ee9916b30 0x559ee9914020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:45.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.012+0000 7f16a5fe8740 1 --2- 172.21.15.83:0/2329164632 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ee990c460 0x559ee99138e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:45.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.012+0000 7f16a5fe8740 1 --2- 172.21.15.83:0/2329164632 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ee990bb40 0x559ee99165f0 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:45.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.012+0000 7f1686000640 0 client.18924 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:00:45.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.012+0000 7f16a5fe8740 1 -- 172.21.15.83:0/2329164632 >> 172.21.15.83:0/2329164632 conn(0x559ee9907e60 msgr2=0x559ee990dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:45.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.012+0000 7f16a5fe8740 1 -- 172.21.15.83:0/2329164632 shutdown_connections 2024-04-24T09:00:45.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.013+0000 7f16a5fe8740 1 -- 172.21.15.83:0/2329164632 wait complete. 2024-04-24T09:00:45.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.013+0000 7f16a5fe8740 20 client.18924 trim_cache size 0 max 16384 2024-04-24T09:00:45.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.029+0000 7f16a5fe8740 1 Processor -- start 2024-04-24T09:00:45.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.029+0000 7f16a5fe8740 1 -- start start 2024-04-24T09:00:45.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.029+0000 7f16a5fe8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ee990bb40 0x559ee9916120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:45.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.029+0000 7f16a5fe8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559ee990c460 0x559ee9937230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:45.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.029+0000 7f16a5fe8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ee9916b30 0x559ee9937770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:45.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.029+0000 7f16a5fe8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f169c1ad1f0 con 0x559ee9916b30 2024-04-24T09:00:45.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.029+0000 7f16a5fe8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f169c1aed30 con 0x559ee990bb40 2024-04-24T09:00:45.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.029+0000 7f16a5fe8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f169c1ad8c0 con 0x559ee990c460 2024-04-24T09:00:45.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.030+0000 7f16a1000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ee9916b30 0x559ee9937770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:45.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.030+0000 7f16a1000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ee9916b30 0x559ee9937770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48764/0 (socket says 172.21.15.83:48764) 2024-04-24T09:00:45.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.030+0000 7f16a1000640 1 -- 172.21.15.83:0/3546747009 learned_addr learned my addr 172.21.15.83:0/3546747009 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:00:45.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.030+0000 7f169b400640 1 --2- 172.21.15.83:0/3546747009 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559ee990c460 0x559ee9937230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:45.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.030+0000 7f169be00640 1 --2- 172.21.15.83:0/3546747009 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ee990bb40 0x559ee9916120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:45.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.030+0000 7f16a1000640 1 -- 172.21.15.83:0/3546747009 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559ee990c460 msgr2=0x559ee9937230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:45.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.030+0000 7f16a1000640 1 --2- 172.21.15.83:0/3546747009 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559ee990c460 0x559ee9937230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:45.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.030+0000 7f16a1000640 1 -- 172.21.15.83:0/3546747009 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ee990bb40 msgr2=0x559ee9916120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:45.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.030+0000 7f16a1000640 1 --2- 172.21.15.83:0/3546747009 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ee990bb40 0x559ee9916120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:45.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.030+0000 7f16a1000640 1 -- 172.21.15.83:0/3546747009 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ee990c290 con 0x559ee9916b30 2024-04-24T09:00:45.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.030+0000 7f16a1000640 1 --2- 172.21.15.83:0/3546747009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ee9916b30 0x559ee9937770 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f1694009ec0 tx=0x7f1694009a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=413c6aa3c609c746 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:45.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.031+0000 7f1690c00640 1 -- 172.21.15.83:0/3546747009 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1694024020 con 0x559ee9916b30 2024-04-24T09:00:45.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.031+0000 7f1690c00640 1 -- 172.21.15.83:0/3546747009 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f169400a920 con 0x559ee9916b30 2024-04-24T09:00:45.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.031+0000 7f1690c00640 1 -- 172.21.15.83:0/3546747009 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f169401a040 con 0x559ee9916b30 2024-04-24T09:00:45.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.032+0000 7f16a5fe8740 1 -- 172.21.15.83:0/3546747009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ee9916b30 msgr2=0x559ee9937770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:45.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.032+0000 7f16a5fe8740 1 --2- 172.21.15.83:0/3546747009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ee9916b30 0x559ee9937770 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f1694009ec0 tx=0x7f1694009a50 comp rx=0 tx=0).stop 2024-04-24T09:00:45.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.032+0000 7f16a5fe8740 1 -- 172.21.15.83:0/3546747009 shutdown_connections 2024-04-24T09:00:45.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.032+0000 7f16a5fe8740 1 --2- 172.21.15.83:0/3546747009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ee9916b30 0x559ee9937770 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:45.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.032+0000 7f16a5fe8740 1 --2- 172.21.15.83:0/3546747009 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559ee990c460 0x559ee9937230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:45.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.032+0000 7f16a5fe8740 1 --2- 172.21.15.83:0/3546747009 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ee990bb40 0x559ee9916120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:45.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.032+0000 7f16a5fe8740 1 -- 172.21.15.83:0/3546747009 >> 172.21.15.83:0/3546747009 conn(0x559ee9907e60 msgr2=0x559ee9906f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:45.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.032+0000 7f16a5fe8740 1 -- 172.21.15.83:0/3546747009 shutdown_connections 2024-04-24T09:00:45.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.032+0000 7f16a5fe8740 1 -- 172.21.15.83:0/3546747009 wait complete. 2024-04-24T09:00:45.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.032+0000 7f16a5fe8740 1 -- start start 2024-04-24T09:00:45.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.032+0000 7f16a5fe8740 1 Processor -- start 2024-04-24T09:00:45.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.033+0000 7f16a5fe8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ee990bb40 0x559ee9927af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:45.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.033+0000 7f16a5fe8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ee990c460 0x559ee9910e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:45.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.033+0000 7f16a5fe8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559ee9916b30 0x559ee9911500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:45.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.033+0000 7f16a5fe8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f169c1ae660 con 0x559ee990bb40 2024-04-24T09:00:45.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.033+0000 7f16a5fe8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f169c1adf90 con 0x559ee990c460 2024-04-24T09:00:45.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.033+0000 7f16a5fe8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f169c1ab210 con 0x559ee9916b30 2024-04-24T09:00:45.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.033+0000 7f16a5fe8740 10 client.0 _ll_register_callbacks cb 0x7f168c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:00:45.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.033+0000 7f169be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ee990bb40 0x559ee9927af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:45.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.033+0000 7f169be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ee990bb40 0x559ee9927af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48776/0 (socket says 172.21.15.83:48776) 2024-04-24T09:00:45.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.033+0000 7f169be00640 1 -- 172.21.15.83:0/3129515308 learned_addr learned my addr 172.21.15.83:0/3129515308 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:00:45.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.034+0000 7f169b400640 1 --2- 172.21.15.83:0/3129515308 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ee990c460 0x559ee9910e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:45.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.034+0000 7f16a1000640 1 --2- 172.21.15.83:0/3129515308 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559ee9916b30 0x559ee9911500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:45.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.034+0000 7f169be00640 1 -- 172.21.15.83:0/3129515308 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559ee9916b30 msgr2=0x559ee9911500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:45.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.034+0000 7f169be00640 1 --2- 172.21.15.83:0/3129515308 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559ee9916b30 0x559ee9911500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:45.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.034+0000 7f169be00640 1 -- 172.21.15.83:0/3129515308 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ee990c460 msgr2=0x559ee9910e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:45.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.034+0000 7f169be00640 1 --2- 172.21.15.83:0/3129515308 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ee990c460 0x559ee9910e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:45.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.034+0000 7f169be00640 1 -- 172.21.15.83:0/3129515308 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ee964d7d0 con 0x559ee990bb40 2024-04-24T09:00:45.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.034+0000 7f16a1000640 1 --2- 172.21.15.83:0/3129515308 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559ee9916b30 0x559ee9911500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:00:45.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.034+0000 7f169be00640 1 --2- 172.21.15.83:0/3129515308 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ee990bb40 0x559ee9927af0 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f168c03b9a0 tx=0x7f168c03b660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=569633f6e403c9d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:45.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.034+0000 7f1686000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:00:45.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.034+0000 7f1686000640 1 -- 172.21.15.83:0/3129515308 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f168c060070 con 0x559ee990bb40 2024-04-24T09:00:45.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.034+0000 7f1686000640 1 -- 172.21.15.83:0/3129515308 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f168c048450 con 0x559ee990bb40 2024-04-24T09:00:45.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.034+0000 7f16a5fe8740 1 -- 172.21.15.83:0/3129515308 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f169400a920 con 0x559ee990bb40 2024-04-24T09:00:45.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.035+0000 7f1686000640 1 -- 172.21.15.83:0/3129515308 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f168c06c050 con 0x559ee990bb40 2024-04-24T09:00:45.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.035+0000 7f16a5fe8740 10 client.18930 fetch_fsmap learned FSMap version 35 2024-04-24T09:00:45.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.035+0000 7f16a5fe8740 1 -- 172.21.15.83:0/3129515308 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x559ee9995d70 con 0x559ee990bb40 2024-04-24T09:00:45.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.035+0000 7f1686000640 1 -- 172.21.15.83:0/3129515308 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f169400a920 con 0x559ee990bb40 2024-04-24T09:00:45.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.035+0000 7f1686000640 1 -- 172.21.15.83:0/3129515308 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7f168c048c90 con 0x559ee990bb40 2024-04-24T09:00:45.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.035+0000 7f16a5fe8740 10 client.18930 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T09:00:45.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.035+0000 7f16a5fe8740 10 client.18930 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:00:45.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.035+0000 7f16a5fe8740 20 client.18930 populate_metadata read hostname 'smithi083' 2024-04-24T09:00:45.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.035+0000 7f16a5fe8740 1 --2- 172.21.15.83:0/3129515308 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x559ee99964f0 0x559ee9998910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:45.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.035+0000 7f16a5fe8740 4 client.18930 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:00:45.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.035+0000 7f16a5fe8740 1 -- 172.21.15.83:0/3129515308 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559ee9907cb0 con 0x559ee99964f0 2024-04-24T09:00:45.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.036+0000 7f169b400640 1 --2- 172.21.15.83:0/3129515308 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x559ee99964f0 0x559ee9998910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:45.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.037+0000 7f169b400640 1 --2- 172.21.15.83:0/3129515308 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x559ee99964f0 0x559ee9998910 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5cacee9548413a9 server_cookie=75ac94c6d4069779 in_seq=0 out_seq=0 2024-04-24T09:00:45.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.037+0000 7f1686000640 10 client.18930 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:00:45.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.037+0000 7f1686000640 1 -- 172.21.15.83:0/3129515308 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559ee9907cb0 con 0x559ee99964f0 2024-04-24T09:00:45.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.037+0000 7f1686000640 10 client.18930 handle_command_reply: tid=0 2024-04-24T09:00:45.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.037+0000 7f16a5fe8740 1 client.18930 shutdown 2024-04-24T09:00:45.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.037+0000 7f16a5fe8740 1 -- 172.21.15.83:0/3129515308 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ee990bb40 msgr2=0x559ee9927af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:45.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.037+0000 7f16a5fe8740 1 --2- 172.21.15.83:0/3129515308 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ee990bb40 0x559ee9927af0 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f168c03b9a0 tx=0x7f168c03b660 comp rx=0 tx=0).stop 2024-04-24T09:00:45.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.038+0000 7f16a5fe8740 1 -- 172.21.15.83:0/3129515308 shutdown_connections 2024-04-24T09:00:45.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.038+0000 7f16a5fe8740 1 --2- 172.21.15.83:0/3129515308 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x559ee99964f0 0x559ee9998910 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:45.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.038+0000 7f16a5fe8740 1 --2- 172.21.15.83:0/3129515308 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559ee9916b30 0x559ee9911500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:45.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.038+0000 7f16a5fe8740 1 --2- 172.21.15.83:0/3129515308 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ee990c460 0x559ee9910e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:45.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.038+0000 7f16a5fe8740 1 --2- 172.21.15.83:0/3129515308 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ee990bb40 0x559ee9927af0 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:45.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.038+0000 7f1686000640 0 client.18930 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:00:45.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.038+0000 7f16a5fe8740 1 -- 172.21.15.83:0/3129515308 >> 172.21.15.83:0/3129515308 conn(0x559ee9907e60 msgr2=0x559ee992a100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:45.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.038+0000 7f16a5fe8740 1 -- 172.21.15.83:0/3129515308 shutdown_connections 2024-04-24T09:00:45.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.038+0000 7f16a5fe8740 1 -- 172.21.15.83:0/3129515308 wait complete. 2024-04-24T09:00:45.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.038+0000 7f16a5fe8740 20 client.18930 trim_cache size 0 max 16384 2024-04-24T09:00:45.040 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:00:45.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.039+0000 7f16a1a00640 1 -- 172.21.15.83:0/4140566432 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f16800a3d20 msgr2=0x7f16800a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:45.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.039+0000 7f16a1a00640 1 --2- 172.21.15.83:0/4140566432 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f16800a3d20 0x7f16800a61e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f168c000c00 tx=0x7f168c031400 comp rx=0 tx=0).stop 2024-04-24T09:00:45.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.039+0000 7f16a1a00640 1 -- 172.21.15.83:0/4140566432 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f169c103980 msgr2=0x7f169c1a0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:45.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.039+0000 7f16a1a00640 1 --2- 172.21.15.83:0/4140566432 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f169c103980 0x7f169c1a0bc0 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f168800c940 tx=0x7f168800ce00 comp rx=0 tx=0).stop 2024-04-24T09:00:45.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.039+0000 7f16a1a00640 1 -- 172.21.15.83:0/4140566432 shutdown_connections 2024-04-24T09:00:45.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.039+0000 7f16a1a00640 1 --2- 172.21.15.83:0/4140566432 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f16800a3d20 0x7f16800a61e0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:45.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.039+0000 7f16a1a00640 1 --2- 172.21.15.83:0/4140566432 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f169c10c480 0x7f169c19a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:45.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.039+0000 7f16a1a00640 1 --2- 172.21.15.83:0/4140566432 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f169c103980 0x7f169c1a0bc0 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:45.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.039+0000 7f16a1a00640 1 --2- 172.21.15.83:0/4140566432 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f169c103060 0x7f169c1a0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:45.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.039+0000 7f16a1a00640 1 -- 172.21.15.83:0/4140566432 >> 172.21.15.83:0/4140566432 conn(0x7f169c0fc880 msgr2=0x7f169c101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:45.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.039+0000 7f16a1a00640 1 -- 172.21.15.83:0/4140566432 shutdown_connections 2024-04-24T09:00:45.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:45.039+0000 7f16a1a00640 1 -- 172.21.15.83:0/4140566432 wait complete. 2024-04-24T09:00:45.602 INFO:tasks.workunit.client.0.smithi083.stdout: 1 52099 0.62 MB/sec execute 319 sec latency 49.103 ms 2024-04-24T09:00:46.058 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T09:00:46.058 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T09:00:46.058 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T09:00:46.131 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:00:45 smithi083 ceph-mon[27907]: pgmap v285: 129 pgs: 129 active+clean; 51 MiB data, 718 MiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 262 KiB/s wr, 47 op/s 2024-04-24T09:00:46.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:00:45 smithi138 ceph-mon[31441]: pgmap v285: 129 pgs: 129 active+clean; 51 MiB data, 718 MiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 262 KiB/s wr, 47 op/s 2024-04-24T09:00:46.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:00:45 smithi189 ceph-mon[31791]: pgmap v285: 129 pgs: 129 active+clean; 51 MiB data, 718 MiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 262 KiB/s wr, 47 op/s 2024-04-24T09:00:46.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.277+0000 7f4797e00640 1 -- 172.21.15.83:0/262831174 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4798102ac0 msgr2=0x7f4798106b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:46.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.277+0000 7f4797e00640 1 --2- 172.21.15.83:0/262831174 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4798102ac0 0x7f4798106b00 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f4784009900 tx=0x7f478402a410 comp rx=0 tx=0).stop 2024-04-24T09:00:46.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.277+0000 7f4797e00640 1 -- 172.21.15.83:0/262831174 shutdown_connections 2024-04-24T09:00:46.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.277+0000 7f4797e00640 1 --2- 172.21.15.83:0/262831174 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47981071e0 0x7f479810edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:46.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.277+0000 7f4797e00640 1 --2- 172.21.15.83:0/262831174 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4798102ac0 0x7f4798106b00 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:46.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.277+0000 7f4797e00640 1 --2- 172.21.15.83:0/262831174 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4798102110 0x7f47981024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:46.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.277+0000 7f4797e00640 1 -- 172.21.15.83:0/262831174 >> 172.21.15.83:0/262831174 conn(0x7f47980fd9d0 msgr2=0x7f47980ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:46.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.277+0000 7f4797e00640 1 -- 172.21.15.83:0/262831174 shutdown_connections 2024-04-24T09:00:46.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.277+0000 7f4797e00640 1 -- 172.21.15.83:0/262831174 wait complete. 2024-04-24T09:00:46.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.278+0000 7f4797e00640 1 Processor -- start 2024-04-24T09:00:46.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.279+0000 7f4797e00640 1 -- start start 2024-04-24T09:00:46.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.279+0000 7f4797e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4798102110 0x7f47981a0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:46.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.279+0000 7f4797e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47981071e0 0x7f47981a0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:46.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.279+0000 7f4797e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f479819a640 0x7f479819aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:46.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.279+0000 7f4797e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4798112100 con 0x7f479819a640 2024-04-24T09:00:46.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.279+0000 7f4797e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4798111f80 con 0x7f47981071e0 2024-04-24T09:00:46.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.279+0000 7f4797e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4798112280 con 0x7f4798102110 2024-04-24T09:00:46.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.279+0000 7f4797400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f479819a640 0x7f479819aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:46.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.279+0000 7f4797400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f479819a640 0x7f479819aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48792/0 (socket says 172.21.15.83:48792) 2024-04-24T09:00:46.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.279+0000 7f4797400640 1 -- 172.21.15.83:0/2389550033 learned_addr learned my addr 172.21.15.83:0/2389550033 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:00:46.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.280+0000 7f4796a00640 1 --2- 172.21.15.83:0/2389550033 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4798102110 0x7f47981a0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:46.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.280+0000 7f4796000640 1 --2- 172.21.15.83:0/2389550033 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47981071e0 0x7f47981a0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:46.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.280+0000 7f4797400640 1 -- 172.21.15.83:0/2389550033 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4798102110 msgr2=0x7f47981a0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:46.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.280+0000 7f4797400640 1 --2- 172.21.15.83:0/2389550033 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4798102110 0x7f47981a0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:46.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.280+0000 7f4797400640 1 -- 172.21.15.83:0/2389550033 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47981071e0 msgr2=0x7f47981a0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:46.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.280+0000 7f4797400640 1 --2- 172.21.15.83:0/2389550033 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47981071e0 0x7f47981a0ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:46.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.280+0000 7f4797400640 1 -- 172.21.15.83:0/2389550033 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f479819b240 con 0x7f479819a640 2024-04-24T09:00:46.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.280+0000 7f4797400640 1 --2- 172.21.15.83:0/2389550033 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f479819a640 0x7f479819aac0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f479000eaa0 tx=0x7f479000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:46.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.280+0000 7f478f400640 1 -- 172.21.15.83:0/2389550033 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f479000cdb0 con 0x7f479819a640 2024-04-24T09:00:46.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.280+0000 7f478f400640 1 -- 172.21.15.83:0/2389550033 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4790005020 con 0x7f479819a640 2024-04-24T09:00:46.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.280+0000 7f4797e00640 1 -- 172.21.15.83:0/2389550033 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47981a7320 con 0x7f479819a640 2024-04-24T09:00:46.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.281+0000 7f478f400640 1 -- 172.21.15.83:0/2389550033 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47900116d0 con 0x7f479819a640 2024-04-24T09:00:46.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.281+0000 7f4797e00640 1 -- 172.21.15.83:0/2389550033 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47981a76d0 con 0x7f479819a640 2024-04-24T09:00:46.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.284+0000 7f47a0b84740 1 Processor -- start 2024-04-24T09:00:46.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.284+0000 7f478f400640 1 -- 172.21.15.83:0/2389550033 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4790011870 con 0x7f479819a640 2024-04-24T09:00:46.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.284+0000 7f47a0b84740 1 -- start start 2024-04-24T09:00:46.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.285+0000 7f47a0b84740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5576bee2a270 0x5576bee2a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:46.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.285+0000 7f47a0b84740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5576bee2ab90 0x5576bee34d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:46.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.285+0000 7f47a0b84740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5576bee35260 0x5576bee37650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:46.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.285+0000 7f47a0b84740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5576bed8fe70 con 0x5576bee35260 2024-04-24T09:00:46.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.285+0000 7f47a0b84740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5576bed36380 con 0x5576bee2ab90 2024-04-24T09:00:46.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.285+0000 7f47a0b84740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5576bed8be30 con 0x5576bee2a270 2024-04-24T09:00:46.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.285+0000 7f4796000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5576bee35260 0x5576bee37650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:46.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.285+0000 7f4796000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5576bee35260 0x5576bee37650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48802/0 (socket says 172.21.15.83:48802) 2024-04-24T09:00:46.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.285+0000 7f4796000640 1 -- 172.21.15.83:0/3982220981 learned_addr learned my addr 172.21.15.83:0/3982220981 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:00:46.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.285+0000 7f4796a00640 1 --2- 172.21.15.83:0/3982220981 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5576bee2ab90 0x5576bee34d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:46.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.285+0000 7f4796000640 1 --2- 172.21.15.83:0/3982220981 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5576bee2a270 0x5576bee2a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:46.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.286+0000 7f4796a00640 1 -- 172.21.15.83:0/3982220981 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5576bee2a270 msgr2=0x5576bee2a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:46.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.286+0000 7f4796a00640 1 --2- 172.21.15.83:0/3982220981 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5576bee2a270 0x5576bee2a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:46.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.286+0000 7f4796a00640 1 -- 172.21.15.83:0/3982220981 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5576bee35260 msgr2=0x5576bee37650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:46.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.286+0000 7f4796a00640 1 --2- 172.21.15.83:0/3982220981 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5576bee35260 0x5576bee37650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:46.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.286+0000 7f4796a00640 1 -- 172.21.15.83:0/3982220981 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576beb6a7e0 con 0x5576bee2ab90 2024-04-24T09:00:46.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.286+0000 7f4796000640 1 --2- 172.21.15.83:0/3982220981 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5576bee35260 0x5576bee37650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:00:46.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.286+0000 7f478f400640 1 --2- 172.21.15.83:0/2389550033 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f477c0a3d20 0x7f477c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:46.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.286+0000 7f478f400640 1 -- 172.21.15.83:0/2389550033 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f47900d0e10 con 0x7f479819a640 2024-04-24T09:00:46.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.286+0000 7f4796a00640 1 --2- 172.21.15.83:0/3982220981 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5576bee2ab90 0x5576bee34d20 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f478800cba0 tx=0x7f478800e720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7b93531e1c4c019 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:46.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.287+0000 7f4796000640 1 --2- 172.21.15.83:0/2389550033 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f477c0a3d20 0x7f477c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:46.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.287+0000 7f4783e00640 1 -- 172.21.15.83:0/3982220981 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4788004220 con 0x5576bee2ab90 2024-04-24T09:00:46.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.287+0000 7f4783e00640 1 -- 172.21.15.83:0/3982220981 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f47880043c0 con 0x5576bee2ab90 2024-04-24T09:00:46.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.287+0000 7f4783e00640 1 -- 172.21.15.83:0/3982220981 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47880107f0 con 0x5576bee2ab90 2024-04-24T09:00:46.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.288+0000 7f47a0b84740 1 -- 172.21.15.83:0/3982220981 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5576bee2ab90 msgr2=0x5576bee34d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:46.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.288+0000 7f47a0b84740 1 --2- 172.21.15.83:0/3982220981 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5576bee2ab90 0x5576bee34d20 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f478800cba0 tx=0x7f478800e720 comp rx=0 tx=0).stop 2024-04-24T09:00:46.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.288+0000 7f4796000640 1 --2- 172.21.15.83:0/2389550033 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f477c0a3d20 0x7f477c0a61e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x5576bee27720 tx=0x7f4784040040 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:46.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.288+0000 7f47a0b84740 1 -- 172.21.15.83:0/3982220981 shutdown_connections 2024-04-24T09:00:46.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.288+0000 7f47a0b84740 1 --2- 172.21.15.83:0/3982220981 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5576bee35260 0x5576bee37650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:46.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.288+0000 7f47a0b84740 1 --2- 172.21.15.83:0/3982220981 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5576bee2ab90 0x5576bee34d20 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:46.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.288+0000 7f47a0b84740 1 --2- 172.21.15.83:0/3982220981 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5576bee2a270 0x5576bee2a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:46.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.288+0000 7f47a0b84740 1 -- 172.21.15.83:0/3982220981 >> 172.21.15.83:0/3982220981 conn(0x5576bee26590 msgr2=0x5576bee26970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:46.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.288+0000 7f47a0b84740 1 -- 172.21.15.83:0/3982220981 shutdown_connections 2024-04-24T09:00:46.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.288+0000 7f47a0b84740 1 -- 172.21.15.83:0/3982220981 wait complete. 2024-04-24T09:00:46.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.288+0000 7f47a0b84740 1 -- start start 2024-04-24T09:00:46.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.289+0000 7f47a0b84740 1 Processor -- start 2024-04-24T09:00:46.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.289+0000 7f47a0b84740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5576bee2a270 0x5576bee321f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:46.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.289+0000 7f47a0b84740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5576bee2ab90 0x5576bee32730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:46.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.289+0000 7f47a0b84740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5576bee35260 0x5576bee32c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:46.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.289+0000 7f47a0b84740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5576bee38050 con 0x5576bee35260 2024-04-24T09:00:46.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.289+0000 7f47a0b84740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5576bed49660 con 0x5576bee2ab90 2024-04-24T09:00:46.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.289+0000 7f47a0b84740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5576bee381d0 con 0x5576bee2a270 2024-04-24T09:00:46.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.289+0000 7f47a0b84740 10 client.0 _ll_register_callbacks cb 0x5576bed37aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:00:46.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.290+0000 7f4797400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5576bee35260 0x5576bee32c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:46.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.290+0000 7f4797400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5576bee35260 0x5576bee32c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48816/0 (socket says 172.21.15.83:48816) 2024-04-24T09:00:46.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.290+0000 7f4797400640 1 -- 172.21.15.83:0/1762310676 learned_addr learned my addr 172.21.15.83:0/1762310676 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:00:46.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.290+0000 7f4796000640 1 --2- 172.21.15.83:0/1762310676 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5576bee2ab90 0x5576bee32730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:46.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.290+0000 7f4796a00640 1 --2- 172.21.15.83:0/1762310676 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5576bee2a270 0x5576bee321f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:46.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.290+0000 7f4797400640 1 -- 172.21.15.83:0/1762310676 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5576bee2a270 msgr2=0x5576bee321f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:46.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.290+0000 7f4797400640 1 --2- 172.21.15.83:0/1762310676 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5576bee2a270 0x5576bee321f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:46.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.290+0000 7f4797400640 1 -- 172.21.15.83:0/1762310676 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5576bee2ab90 msgr2=0x5576bee32730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:46.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.290+0000 7f4797400640 1 --2- 172.21.15.83:0/1762310676 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5576bee2ab90 0x5576bee32730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:46.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.290+0000 7f4797400640 1 -- 172.21.15.83:0/1762310676 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576bebcaf70 con 0x5576bee35260 2024-04-24T09:00:46.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.290+0000 7f4797400640 1 --2- 172.21.15.83:0/1762310676 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5576bee35260 0x5576bee32c70 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f47900ce1a0 tx=0x7f479009ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dc560151b3a2d80 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:46.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.290+0000 7f4780c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:00:46.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.291+0000 7f4780c00640 1 -- 172.21.15.83:0/1762310676 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f479001f040 con 0x5576bee35260 2024-04-24T09:00:46.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.291+0000 7f4780c00640 1 -- 172.21.15.83:0/1762310676 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f47900051d0 con 0x5576bee35260 2024-04-24T09:00:46.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.291+0000 7f47a0b84740 1 -- 172.21.15.83:0/1762310676 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47880043c0 con 0x5576bee35260 2024-04-24T09:00:46.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.291+0000 7f4780c00640 1 -- 172.21.15.83:0/1762310676 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f47900870c0 con 0x5576bee35260 2024-04-24T09:00:46.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.291+0000 7f47a0b84740 10 client.18942 fetch_fsmap learned FSMap version 35 2024-04-24T09:00:46.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.291+0000 7f47a0b84740 1 -- 172.21.15.83:0/1762310676 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5576bea62170 con 0x5576bee35260 2024-04-24T09:00:46.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.291+0000 7f4780c00640 1 -- 172.21.15.83:0/1762310676 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f47880043c0 con 0x5576bee35260 2024-04-24T09:00:46.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.291+0000 7f4780c00640 1 -- 172.21.15.83:0/1762310676 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7f479000f040 con 0x5576bee35260 2024-04-24T09:00:46.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.292+0000 7f47a0b84740 10 client.18942 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T09:00:46.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.292+0000 7f47a0b84740 10 client.18942 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:00:46.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.292+0000 7f47a0b84740 20 client.18942 populate_metadata read hostname 'smithi083' 2024-04-24T09:00:46.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.292+0000 7f47a0b84740 1 --2- 172.21.15.83:0/1762310676 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5576bee456c0 0x5576bee47ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:46.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.292+0000 7f47a0b84740 4 client.18942 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:00:46.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.292+0000 7f47a0b84740 1 -- 172.21.15.83:0/1762310676 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5576bed9d890 con 0x5576bee456c0 2024-04-24T09:00:46.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.292+0000 7f4796a00640 1 --2- 172.21.15.83:0/1762310676 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5576bee456c0 0x5576bee47ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:46.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.293+0000 7f4796a00640 1 --2- 172.21.15.83:0/1762310676 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5576bee456c0 0x5576bee47ab0 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=802aed8d83fa626a server_cookie=f8364c02089acf51 in_seq=0 out_seq=0 2024-04-24T09:00:46.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.293+0000 7f4780c00640 10 client.18942 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:00:46.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.295+0000 7f4780c00640 1 -- 172.21.15.83:0/1762310676 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5576bed9d890 con 0x5576bee456c0 2024-04-24T09:00:46.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.295+0000 7f4780c00640 10 client.18942 handle_command_reply: tid=0 2024-04-24T09:00:46.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.295+0000 7f47a0b84740 1 client.18942 shutdown 2024-04-24T09:00:46.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.295+0000 7f47a0b84740 1 -- 172.21.15.83:0/1762310676 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5576bee35260 msgr2=0x5576bee32c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:46.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.295+0000 7f47a0b84740 1 --2- 172.21.15.83:0/1762310676 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5576bee35260 0x5576bee32c70 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f47900ce1a0 tx=0x7f479009ca50 comp rx=0 tx=0).stop 2024-04-24T09:00:46.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.296+0000 7f47a0b84740 1 -- 172.21.15.83:0/1762310676 shutdown_connections 2024-04-24T09:00:46.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.296+0000 7f47a0b84740 1 --2- 172.21.15.83:0/1762310676 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5576bee456c0 0x5576bee47ab0 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:46.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.296+0000 7f47a0b84740 1 --2- 172.21.15.83:0/1762310676 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5576bee35260 0x5576bee32c70 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:46.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.296+0000 7f47a0b84740 1 --2- 172.21.15.83:0/1762310676 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5576bee2ab90 0x5576bee32730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:46.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.296+0000 7f47a0b84740 1 --2- 172.21.15.83:0/1762310676 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5576bee2a270 0x5576bee321f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:46.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.296+0000 7f47a0b84740 1 -- 172.21.15.83:0/1762310676 >> 172.21.15.83:0/1762310676 conn(0x5576bee26590 msgr2=0x5576bee2c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:46.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.296+0000 7f4780c00640 0 client.18942 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:00:46.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.296+0000 7f47a0b84740 1 -- 172.21.15.83:0/1762310676 shutdown_connections 2024-04-24T09:00:46.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.296+0000 7f47a0b84740 1 -- 172.21.15.83:0/1762310676 wait complete. 2024-04-24T09:00:46.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.296+0000 7f47a0b84740 20 client.18942 trim_cache size 0 max 16384 2024-04-24T09:00:46.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.317+0000 7f47a0b84740 1 Processor -- start 2024-04-24T09:00:46.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.317+0000 7f47a0b84740 1 -- start start 2024-04-24T09:00:46.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.317+0000 7f47a0b84740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5576bee2a270 0x5576bee40f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:46.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.317+0000 7f47a0b84740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5576bee2ab90 0x5576bee31150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:46.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.317+0000 7f47a0b84740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5576bee35260 0x5576bee2ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:46.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.317+0000 7f47a0b84740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f47981b15f0 con 0x5576bee2a270 2024-04-24T09:00:46.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.317+0000 7f47a0b84740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f47981b3130 con 0x5576bee2ab90 2024-04-24T09:00:46.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.317+0000 7f47a0b84740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f47981b1cc0 con 0x5576bee35260 2024-04-24T09:00:46.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.317+0000 7f4796a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5576bee2a270 0x5576bee40f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:46.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.317+0000 7f4796a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5576bee2a270 0x5576bee40f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48830/0 (socket says 172.21.15.83:48830) 2024-04-24T09:00:46.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.317+0000 7f4796a00640 1 -- 172.21.15.83:0/3415945529 learned_addr learned my addr 172.21.15.83:0/3415945529 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:00:46.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.318+0000 7f4797400640 1 --2- 172.21.15.83:0/3415945529 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5576bee35260 0x5576bee2ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:46.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.318+0000 7f4796000640 1 --2- 172.21.15.83:0/3415945529 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5576bee2ab90 0x5576bee31150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:46.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.318+0000 7f4796a00640 1 -- 172.21.15.83:0/3415945529 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5576bee35260 msgr2=0x5576bee2ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:46.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.318+0000 7f4796a00640 1 --2- 172.21.15.83:0/3415945529 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5576bee35260 0x5576bee2ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:46.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.318+0000 7f4796a00640 1 -- 172.21.15.83:0/3415945529 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5576bee2ab90 msgr2=0x5576bee31150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:46.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.318+0000 7f4796a00640 1 --2- 172.21.15.83:0/3415945529 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5576bee2ab90 0x5576bee31150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:46.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.318+0000 7f4796a00640 1 -- 172.21.15.83:0/3415945529 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576bee44260 con 0x5576bee2a270 2024-04-24T09:00:46.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.318+0000 7f4796a00640 1 --2- 172.21.15.83:0/3415945529 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5576bee2a270 0x5576bee40f40 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f478800cac0 tx=0x7f4788003d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=394d5452a2819712 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:46.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.318+0000 7f4783400640 1 -- 172.21.15.83:0/3415945529 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47880037f0 con 0x5576bee2a270 2024-04-24T09:00:46.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.319+0000 7f4783400640 1 -- 172.21.15.83:0/3415945529 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f47880054f0 con 0x5576bee2a270 2024-04-24T09:00:46.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.319+0000 7f4783400640 1 -- 172.21.15.83:0/3415945529 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4788004380 con 0x5576bee2a270 2024-04-24T09:00:46.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.319+0000 7f47a0b84740 1 -- 172.21.15.83:0/3415945529 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5576bee2a270 msgr2=0x5576bee40f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:46.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.319+0000 7f47a0b84740 1 --2- 172.21.15.83:0/3415945529 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5576bee2a270 0x5576bee40f40 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f478800cac0 tx=0x7f4788003d70 comp rx=0 tx=0).stop 2024-04-24T09:00:46.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.319+0000 7f47a0b84740 1 -- 172.21.15.83:0/3415945529 shutdown_connections 2024-04-24T09:00:46.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.319+0000 7f47a0b84740 1 --2- 172.21.15.83:0/3415945529 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5576bee35260 0x5576bee2ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:46.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.319+0000 7f47a0b84740 1 --2- 172.21.15.83:0/3415945529 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5576bee2ab90 0x5576bee31150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:46.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.319+0000 7f47a0b84740 1 --2- 172.21.15.83:0/3415945529 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5576bee2a270 0x5576bee40f40 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:46.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.319+0000 7f47a0b84740 1 -- 172.21.15.83:0/3415945529 >> 172.21.15.83:0/3415945529 conn(0x5576bee26590 msgr2=0x5576bee256f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:46.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.320+0000 7f47a0b84740 1 -- 172.21.15.83:0/3415945529 shutdown_connections 2024-04-24T09:00:46.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.320+0000 7f47a0b84740 1 -- 172.21.15.83:0/3415945529 wait complete. 2024-04-24T09:00:46.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.320+0000 7f47a0b84740 1 -- start start 2024-04-24T09:00:46.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.320+0000 7f47a0b84740 1 Processor -- start 2024-04-24T09:00:46.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.321+0000 7f47a0b84740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5576bee2a270 0x5576bee2f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:46.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.321+0000 7f47a0b84740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5576bee2ab90 0x5576bee2fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:46.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.321+0000 7f47a0b84740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5576bee35260 0x5576beeaaba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:46.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.321+0000 7f47a0b84740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f47981b2a60 con 0x5576bee2a270 2024-04-24T09:00:46.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.321+0000 7f47a0b84740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f47981b2390 con 0x5576bee35260 2024-04-24T09:00:46.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.321+0000 7f47a0b84740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f47981af5f0 con 0x5576bee2ab90 2024-04-24T09:00:46.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.321+0000 7f47a0b84740 10 client.0 _ll_register_callbacks cb 0x7f4790098c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:00:46.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.321+0000 7f4796a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5576bee2a270 0x5576bee2f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:46.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.321+0000 7f4796a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5576bee2a270 0x5576bee2f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48846/0 (socket says 172.21.15.83:48846) 2024-04-24T09:00:46.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.321+0000 7f4796a00640 1 -- 172.21.15.83:0/4189630284 learned_addr learned my addr 172.21.15.83:0/4189630284 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:00:46.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.322+0000 7f4796000640 1 --2- 172.21.15.83:0/4189630284 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5576bee2ab90 0x5576bee2fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:46.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.322+0000 7f4796a00640 1 -- 172.21.15.83:0/4189630284 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5576bee2ab90 msgr2=0x5576bee2fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:46.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.322+0000 7f4797400640 1 --2- 172.21.15.83:0/4189630284 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5576bee35260 0x5576beeaaba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:46.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.322+0000 7f4796a00640 1 --2- 172.21.15.83:0/4189630284 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5576bee2ab90 0x5576bee2fb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:46.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.322+0000 7f4796a00640 1 -- 172.21.15.83:0/4189630284 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5576bee35260 msgr2=0x5576beeaaba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:46.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.322+0000 7f4796a00640 1 --2- 172.21.15.83:0/4189630284 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5576bee35260 0x5576beeaaba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:46.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.322+0000 7f4796a00640 1 -- 172.21.15.83:0/4189630284 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576beeab0e0 con 0x5576bee2a270 2024-04-24T09:00:46.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.322+0000 7f4796a00640 1 --2- 172.21.15.83:0/4189630284 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5576bee2a270 0x5576bee2f5e0 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f4788005890 tx=0x7f478800cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0bb1ccdb5d62df6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:46.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.322+0000 7f4780c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:00:46.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.322+0000 7f4780c00640 1 -- 172.21.15.83:0/4189630284 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47880035f0 con 0x5576bee2a270 2024-04-24T09:00:46.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.322+0000 7f4780c00640 1 -- 172.21.15.83:0/4189630284 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7f4788003790 con 0x5576bee2a270 2024-04-24T09:00:46.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.322+0000 7f47a0b84740 1 -- 172.21.15.83:0/4189630284 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47880054f0 con 0x5576bee2a270 2024-04-24T09:00:46.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.323+0000 7f4780c00640 1 -- 172.21.15.83:0/4189630284 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f4788003990 con 0x5576bee2a270 2024-04-24T09:00:46.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.323+0000 7f47a0b84740 10 client.18948 fetch_fsmap learned FSMap version 35 2024-04-24T09:00:46.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.323+0000 7f47a0b84740 1 -- 172.21.15.83:0/4189630284 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5576beeac3d0 con 0x5576bee2a270 2024-04-24T09:00:46.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.323+0000 7f4780c00640 1 -- 172.21.15.83:0/4189630284 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f47880054f0 con 0x5576bee2a270 2024-04-24T09:00:46.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.323+0000 7f4780c00640 1 -- 172.21.15.83:0/4189630284 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7f4788026480 con 0x5576bee2a270 2024-04-24T09:00:46.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.323+0000 7f47a0b84740 10 client.18948 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T09:00:46.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.323+0000 7f47a0b84740 10 client.18948 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:00:46.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.323+0000 7f47a0b84740 20 client.18948 populate_metadata read hostname 'smithi083' 2024-04-24T09:00:46.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.323+0000 7f47a0b84740 1 --2- 172.21.15.83:0/4189630284 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5576beeac970 0x5576beeaed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:46.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.323+0000 7f47a0b84740 4 client.18948 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T09:00:46.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.323+0000 7f47a0b84740 1 -- 172.21.15.83:0/4189630284 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5576bed97700 con 0x5576beeac970 2024-04-24T09:00:46.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.324+0000 7f4796000640 1 --2- 172.21.15.83:0/4189630284 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5576beeac970 0x5576beeaed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:46.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.325+0000 7f4796000640 1 --2- 172.21.15.83:0/4189630284 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5576beeac970 0x5576beeaed90 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=604db7ed1b971c89 server_cookie=8604e22cbd287b60 in_seq=0 out_seq=0 2024-04-24T09:00:46.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.325+0000 7f4780c00640 10 client.18948 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:00:46.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.328+0000 7f4780c00640 1 -- 172.21.15.83:0/4189630284 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5576bed97700 con 0x5576beeac970 2024-04-24T09:00:46.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.328+0000 7f4780c00640 10 client.18948 handle_command_reply: tid=0 2024-04-24T09:00:46.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.328+0000 7f47a0b84740 1 client.18948 shutdown 2024-04-24T09:00:46.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.328+0000 7f47a0b84740 1 -- 172.21.15.83:0/4189630284 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5576bee2a270 msgr2=0x5576bee2f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:46.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.328+0000 7f47a0b84740 1 --2- 172.21.15.83:0/4189630284 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5576bee2a270 0x5576bee2f5e0 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f4788005890 tx=0x7f478800cb60 comp rx=0 tx=0).stop 2024-04-24T09:00:46.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.329+0000 7f47a0b84740 1 -- 172.21.15.83:0/4189630284 shutdown_connections 2024-04-24T09:00:46.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.329+0000 7f47a0b84740 1 --2- 172.21.15.83:0/4189630284 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5576beeac970 0x5576beeaed90 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:46.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.329+0000 7f47a0b84740 1 --2- 172.21.15.83:0/4189630284 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5576bee35260 0x5576beeaaba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:46.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.329+0000 7f47a0b84740 1 --2- 172.21.15.83:0/4189630284 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5576bee2ab90 0x5576bee2fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:46.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.329+0000 7f4780c00640 0 client.18948 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:00:46.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.329+0000 7f47a0b84740 1 --2- 172.21.15.83:0/4189630284 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5576bee2a270 0x5576bee2f5e0 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:46.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.329+0000 7f47a0b84740 1 -- 172.21.15.83:0/4189630284 >> 172.21.15.83:0/4189630284 conn(0x5576bee26590 msgr2=0x5576bee321b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:46.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.329+0000 7f47a0b84740 1 -- 172.21.15.83:0/4189630284 shutdown_connections 2024-04-24T09:00:46.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.329+0000 7f47a0b84740 1 -- 172.21.15.83:0/4189630284 wait complete. 2024-04-24T09:00:46.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.329+0000 7f47a0b84740 20 client.18948 trim_cache size 0 max 16384 2024-04-24T09:00:46.330 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T09:00:46.330 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-04-24T09:00:46.330 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "1dc86011-0659-4e57-9057-47314c0dc613", 2024-04-24T09:00:46.330 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-04-24T09:00:46.330 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T09:00:46.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.330+0000 7f4797e00640 1 -- 172.21.15.83:0/2389550033 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f477c0a3d20 msgr2=0x7f477c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:46.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.330+0000 7f4797e00640 1 --2- 172.21.15.83:0/2389550033 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f477c0a3d20 0x7f477c0a61e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x5576bee27720 tx=0x7f4784040040 comp rx=0 tx=0).stop 2024-04-24T09:00:46.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.330+0000 7f4797e00640 1 -- 172.21.15.83:0/2389550033 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f479819a640 msgr2=0x7f479819aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:46.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.330+0000 7f4797e00640 1 --2- 172.21.15.83:0/2389550033 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f479819a640 0x7f479819aac0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f479000eaa0 tx=0x7f479000ef60 comp rx=0 tx=0).stop 2024-04-24T09:00:46.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.330+0000 7f4797e00640 1 -- 172.21.15.83:0/2389550033 shutdown_connections 2024-04-24T09:00:46.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.330+0000 7f4797e00640 1 --2- 172.21.15.83:0/2389550033 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f477c0a3d20 0x7f477c0a61e0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:46.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.330+0000 7f4797e00640 1 --2- 172.21.15.83:0/2389550033 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f479819a640 0x7f479819aac0 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:46.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.330+0000 7f4797e00640 1 --2- 172.21.15.83:0/2389550033 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47981071e0 0x7f47981a0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:46.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.330+0000 7f4797e00640 1 --2- 172.21.15.83:0/2389550033 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4798102110 0x7f47981a0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:46.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.330+0000 7f4797e00640 1 -- 172.21.15.83:0/2389550033 >> 172.21.15.83:0/2389550033 conn(0x7f47980fd9d0 msgr2=0x7f47980ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:46.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.330+0000 7f4797e00640 1 -- 172.21.15.83:0/2389550033 shutdown_connections 2024-04-24T09:00:46.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.330+0000 7f4797e00640 1 -- 172.21.15.83:0/2389550033 wait complete. 2024-04-24T09:00:46.348 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T09:00:46.349 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T09:00:46.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.557+0000 7fa27b200640 1 -- 172.21.15.83:0/4173466502 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa27c100e50 msgr2=0x7fa27c101230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:46.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.557+0000 7fa27b200640 1 --2- 172.21.15.83:0/4173466502 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa27c100e50 0x7fa27c101230 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7fa26c009900 tx=0x7fa26c02a470 comp rx=0 tx=0).stop 2024-04-24T09:00:46.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.557+0000 7fa27b200640 1 -- 172.21.15.83:0/4173466502 shutdown_connections 2024-04-24T09:00:46.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.557+0000 7fa27b200640 1 --2- 172.21.15.83:0/4173466502 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa27c106250 0x7fa27c1066e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:46.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.557+0000 7fa27b200640 1 --2- 172.21.15.83:0/4173466502 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa27c101770 0x7fa27c105c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:46.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.557+0000 7fa27b200640 1 --2- 172.21.15.83:0/4173466502 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa27c100e50 0x7fa27c101230 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:46.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.557+0000 7fa27b200640 1 -- 172.21.15.83:0/4173466502 >> 172.21.15.83:0/4173466502 conn(0x7fa27c0fc880 msgr2=0x7fa27c0feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:46.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.558+0000 7fa27b200640 1 -- 172.21.15.83:0/4173466502 shutdown_connections 2024-04-24T09:00:46.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.558+0000 7fa27b200640 1 -- 172.21.15.83:0/4173466502 wait complete. 2024-04-24T09:00:46.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.558+0000 7fa27b200640 1 Processor -- start 2024-04-24T09:00:46.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.559+0000 7fa27b200640 1 -- start start 2024-04-24T09:00:46.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.559+0000 7fa27b200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa27c100e50 0x7fa27c1a4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:46.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.559+0000 7fa27b200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa27c101770 0x7fa27c1a4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:46.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.559+0000 7fa27b200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa27c106250 0x7fa27c19e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:46.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.559+0000 7fa27b200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa27c118750 con 0x7fa27c101770 2024-04-24T09:00:46.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.559+0000 7fa27b200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa27c1185d0 con 0x7fa27c100e50 2024-04-24T09:00:46.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.559+0000 7fa27b200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa27c1188d0 con 0x7fa27c106250 2024-04-24T09:00:46.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.559+0000 7fa273e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa27c101770 0x7fa27c1a4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:46.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.559+0000 7fa273e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa27c101770 0x7fa27c1a4d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48870/0 (socket says 172.21.15.83:48870) 2024-04-24T09:00:46.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.559+0000 7fa273e00640 1 -- 172.21.15.83:0/720295596 learned_addr learned my addr 172.21.15.83:0/720295596 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:00:46.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.560+0000 7fa279e00640 1 --2- 172.21.15.83:0/720295596 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa27c100e50 0x7fa27c1a4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:46.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.560+0000 7fa27a800640 1 --2- 172.21.15.83:0/720295596 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa27c106250 0x7fa27c19e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:46.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.560+0000 7fa273e00640 1 -- 172.21.15.83:0/720295596 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa27c106250 msgr2=0x7fa27c19e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:46.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.560+0000 7fa273e00640 1 --2- 172.21.15.83:0/720295596 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa27c106250 0x7fa27c19e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:46.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.560+0000 7fa273e00640 1 -- 172.21.15.83:0/720295596 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa27c100e50 msgr2=0x7fa27c1a4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:46.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.560+0000 7fa273e00640 1 --2- 172.21.15.83:0/720295596 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa27c100e50 0x7fa27c1a4840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:46.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.560+0000 7fa273e00640 1 -- 172.21.15.83:0/720295596 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa27c19f170 con 0x7fa27c101770 2024-04-24T09:00:46.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.560+0000 7fa273e00640 1 --2- 172.21.15.83:0/720295596 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa27c101770 0x7fa27c1a4d80 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7fa26400c970 tx=0x7fa26400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:46.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.560+0000 7fa272a00640 1 -- 172.21.15.83:0/720295596 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa264007bb0 con 0x7fa27c101770 2024-04-24T09:00:46.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.561+0000 7fa272a00640 1 -- 172.21.15.83:0/720295596 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa264007d50 con 0x7fa27c101770 2024-04-24T09:00:46.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.561+0000 7fa27b200640 1 -- 172.21.15.83:0/720295596 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa27c19f460 con 0x7fa27c101770 2024-04-24T09:00:46.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.561+0000 7fa272a00640 1 -- 172.21.15.83:0/720295596 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa264018780 con 0x7fa27c101770 2024-04-24T09:00:46.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.561+0000 7fa27b200640 1 -- 172.21.15.83:0/720295596 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa27c1ab700 con 0x7fa27c101770 2024-04-24T09:00:46.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.564+0000 7fa2839a8740 1 Processor -- start 2024-04-24T09:00:46.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.564+0000 7fa2839a8740 1 -- start start 2024-04-24T09:00:46.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.564+0000 7fa272a00640 1 -- 172.21.15.83:0/720295596 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa26400f450 con 0x7fa27c101770 2024-04-24T09:00:46.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.565+0000 7fa2839a8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562afb5caa30 0x562afb5cae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:46.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.565+0000 7fa2839a8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562afb5cb350 0x562afb5d54e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:46.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.565+0000 7fa2839a8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562afb5d5a20 0x562afb5d7e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:46.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.565+0000 7fa2839a8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562afb5301f0 con 0x562afb5cb350 2024-04-24T09:00:46.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.565+0000 7fa2839a8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562afb4d6350 con 0x562afb5caa30 2024-04-24T09:00:46.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.565+0000 7fa2839a8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x562afb52c1b0 con 0x562afb5d5a20 2024-04-24T09:00:46.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.565+0000 7fa279e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562afb5cb350 0x562afb5d54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:46.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.565+0000 7fa279e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562afb5cb350 0x562afb5d54e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48880/0 (socket says 172.21.15.83:48880) 2024-04-24T09:00:46.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.565+0000 7fa279e00640 1 -- 172.21.15.83:0/2027141368 learned_addr learned my addr 172.21.15.83:0/2027141368 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:00:46.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.565+0000 7fa27a800640 1 --2- 172.21.15.83:0/2027141368 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562afb5caa30 0x562afb5cae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:46.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.565+0000 7fa273e00640 1 --2- 172.21.15.83:0/2027141368 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562afb5d5a20 0x562afb5d7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:46.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.566+0000 7fa279e00640 1 -- 172.21.15.83:0/2027141368 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562afb5d5a20 msgr2=0x562afb5d7e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:46.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.566+0000 7fa279e00640 1 --2- 172.21.15.83:0/2027141368 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562afb5d5a20 0x562afb5d7e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:46.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.566+0000 7fa279e00640 1 -- 172.21.15.83:0/2027141368 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562afb5caa30 msgr2=0x562afb5cae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:46.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.566+0000 7fa279e00640 1 --2- 172.21.15.83:0/2027141368 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562afb5caa30 0x562afb5cae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:46.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.566+0000 7fa279e00640 1 -- 172.21.15.83:0/2027141368 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562afb52af50 con 0x562afb5cb350 2024-04-24T09:00:46.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.566+0000 7fa279e00640 1 --2- 172.21.15.83:0/2027141368 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562afb5cb350 0x562afb5d54e0 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7fa26c000c00 tx=0x7fa26c004390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3f943aa33b0e6a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:46.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.566+0000 7fa272a00640 1 --2- 172.21.15.83:0/720295596 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa2600a3d20 0x7fa2600a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:46.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.566+0000 7fa272a00640 1 -- 172.21.15.83:0/720295596 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7fa2640d0d70 con 0x7fa27c101770 2024-04-24T09:00:46.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.566+0000 7fa26b000640 1 -- 172.21.15.83:0/2027141368 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa26c03f070 con 0x562afb5cb350 2024-04-24T09:00:46.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.566+0000 7fa26b000640 1 -- 172.21.15.83:0/2027141368 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa26c003ca0 con 0x562afb5cb350 2024-04-24T09:00:46.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.566+0000 7fa26b000640 1 -- 172.21.15.83:0/2027141368 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa26c035070 con 0x562afb5cb350 2024-04-24T09:00:46.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.566+0000 7fa27a800640 1 --2- 172.21.15.83:0/720295596 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa2600a3d20 0x7fa2600a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:46.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.567+0000 7fa2839a8740 1 -- 172.21.15.83:0/2027141368 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562afb5cb350 msgr2=0x562afb5d54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:46.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.567+0000 7fa27a800640 1 --2- 172.21.15.83:0/720295596 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa2600a3d20 0x7fa2600a61e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa274007db0 tx=0x7fa274007380 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:46.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.567+0000 7fa2839a8740 1 --2- 172.21.15.83:0/2027141368 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562afb5cb350 0x562afb5d54e0 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7fa26c000c00 tx=0x7fa26c004390 comp rx=0 tx=0).stop 2024-04-24T09:00:46.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.567+0000 7fa2839a8740 1 -- 172.21.15.83:0/2027141368 shutdown_connections 2024-04-24T09:00:46.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.567+0000 7fa2839a8740 1 --2- 172.21.15.83:0/2027141368 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562afb5d5a20 0x562afb5d7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:46.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.567+0000 7fa2839a8740 1 --2- 172.21.15.83:0/2027141368 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562afb5cb350 0x562afb5d54e0 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:46.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.567+0000 7fa2839a8740 1 --2- 172.21.15.83:0/2027141368 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562afb5caa30 0x562afb5cae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:46.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.568+0000 7fa2839a8740 1 -- 172.21.15.83:0/2027141368 >> 172.21.15.83:0/2027141368 conn(0x562afb5c6d50 msgr2=0x562afb5c7130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:46.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.568+0000 7fa2839a8740 1 -- 172.21.15.83:0/2027141368 shutdown_connections 2024-04-24T09:00:46.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.568+0000 7fa2839a8740 1 -- 172.21.15.83:0/2027141368 wait complete. 2024-04-24T09:00:46.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.568+0000 7fa2839a8740 1 -- start start 2024-04-24T09:00:46.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.569+0000 7fa2839a8740 1 Processor -- start 2024-04-24T09:00:46.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.569+0000 7fa2839a8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562afb5caa30 0x562afb5d2980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:46.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.569+0000 7fa2839a8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562afb5cb350 0x562afb5d2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:46.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.569+0000 7fa2839a8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562afb5d5a20 0x562afb5d3400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:46.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.569+0000 7fa2839a8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562afb5d87b0 con 0x562afb5caa30 2024-04-24T09:00:46.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.569+0000 7fa2839a8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562afb4e9690 con 0x562afb5d5a20 2024-04-24T09:00:46.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.569+0000 7fa2839a8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x562afb5d8930 con 0x562afb5cb350 2024-04-24T09:00:46.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.569+0000 7fa2839a8740 10 client.0 _ll_register_callbacks cb 0x562afb525650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:00:46.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.569+0000 7fa279e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562afb5caa30 0x562afb5d2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:46.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.570+0000 7fa279e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562afb5caa30 0x562afb5d2980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48884/0 (socket says 172.21.15.83:48884) 2024-04-24T09:00:46.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.570+0000 7fa279e00640 1 -- 172.21.15.83:0/4180006482 learned_addr learned my addr 172.21.15.83:0/4180006482 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:00:46.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.570+0000 7fa273e00640 1 --2- 172.21.15.83:0/4180006482 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562afb5cb350 0x562afb5d2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:46.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.570+0000 7fa27a800640 1 --2- 172.21.15.83:0/4180006482 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562afb5d5a20 0x562afb5d3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:46.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.570+0000 7fa279e00640 1 -- 172.21.15.83:0/4180006482 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562afb5cb350 msgr2=0x562afb5d2ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:46.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.570+0000 7fa279e00640 1 --2- 172.21.15.83:0/4180006482 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562afb5cb350 0x562afb5d2ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:46.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.570+0000 7fa279e00640 1 -- 172.21.15.83:0/4180006482 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562afb5d5a20 msgr2=0x562afb5d3400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:46.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.570+0000 7fa279e00640 1 --2- 172.21.15.83:0/4180006482 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562afb5d5a20 0x562afb5d3400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:46.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.570+0000 7fa279e00640 1 -- 172.21.15.83:0/4180006482 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562afb4e8920 con 0x562afb5caa30 2024-04-24T09:00:46.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.570+0000 7fa279e00640 1 --2- 172.21.15.83:0/4180006482 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562afb5caa30 0x562afb5d2980 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7fa26c02aa60 tx=0x7fa26c031bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4362ec2a19730ddc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:46.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.570+0000 7fa25f400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:00:46.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.571+0000 7fa25f400640 1 -- 172.21.15.83:0/4180006482 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa26c035040 con 0x562afb5caa30 2024-04-24T09:00:46.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.571+0000 7fa25f400640 1 -- 172.21.15.83:0/4180006482 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7fa26c003a00 con 0x562afb5caa30 2024-04-24T09:00:46.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.571+0000 7fa2839a8740 1 -- 172.21.15.83:0/4180006482 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa26c003ca0 con 0x562afb5caa30 2024-04-24T09:00:46.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.571+0000 7fa25f400640 1 -- 172.21.15.83:0/4180006482 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fa26c03f040 con 0x562afb5caa30 2024-04-24T09:00:46.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.571+0000 7fa2839a8740 10 client.18960 fetch_fsmap learned FSMap version 35 2024-04-24T09:00:46.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.571+0000 7fa2839a8740 1 -- 172.21.15.83:0/4180006482 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x562afb30c7d0 con 0x562afb5caa30 2024-04-24T09:00:46.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.571+0000 7fa25f400640 1 -- 172.21.15.83:0/4180006482 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa26c003ca0 con 0x562afb5caa30 2024-04-24T09:00:46.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.571+0000 7fa25f400640 1 -- 172.21.15.83:0/4180006482 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7fa26c004b80 con 0x562afb5caa30 2024-04-24T09:00:46.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.572+0000 7fa2839a8740 10 client.18960 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T09:00:46.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.572+0000 7fa2839a8740 10 client.18960 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:00:46.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.572+0000 7fa2839a8740 20 client.18960 populate_metadata read hostname 'smithi083' 2024-04-24T09:00:46.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.572+0000 7fa2839a8740 1 --2- 172.21.15.83:0/4180006482 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x562afb5e5b90 0x562afb5e7f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:46.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.572+0000 7fa2839a8740 4 client.18960 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:00:46.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.572+0000 7fa2839a8740 1 -- 172.21.15.83:0/4180006482 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562afb53dc10 con 0x562afb5e5b90 2024-04-24T09:00:46.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.572+0000 7fa273e00640 1 --2- 172.21.15.83:0/4180006482 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x562afb5e5b90 0x562afb5e7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:46.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.573+0000 7fa273e00640 1 --2- 172.21.15.83:0/4180006482 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x562afb5e5b90 0x562afb5e7f80 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e66888621826658 server_cookie=f7aff44222de72ed in_seq=0 out_seq=0 2024-04-24T09:00:46.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.573+0000 7fa25f400640 10 client.18960 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:00:46.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.575+0000 7fa25f400640 1 -- 172.21.15.83:0/4180006482 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562afb53dc10 con 0x562afb5e5b90 2024-04-24T09:00:46.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.575+0000 7fa25f400640 10 client.18960 handle_command_reply: tid=0 2024-04-24T09:00:46.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.575+0000 7fa2839a8740 1 client.18960 shutdown 2024-04-24T09:00:46.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.576+0000 7fa2839a8740 1 -- 172.21.15.83:0/4180006482 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562afb5caa30 msgr2=0x562afb5d2980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:46.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.576+0000 7fa2839a8740 1 --2- 172.21.15.83:0/4180006482 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562afb5caa30 0x562afb5d2980 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7fa26c02aa60 tx=0x7fa26c031bd0 comp rx=0 tx=0).stop 2024-04-24T09:00:46.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.576+0000 7fa2839a8740 1 -- 172.21.15.83:0/4180006482 shutdown_connections 2024-04-24T09:00:46.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.576+0000 7fa2839a8740 1 --2- 172.21.15.83:0/4180006482 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x562afb5e5b90 0x562afb5e7f80 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:46.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.576+0000 7fa2839a8740 1 --2- 172.21.15.83:0/4180006482 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562afb5d5a20 0x562afb5d3400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:46.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.576+0000 7fa2839a8740 1 --2- 172.21.15.83:0/4180006482 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562afb5cb350 0x562afb5d2ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:46.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.576+0000 7fa2839a8740 1 --2- 172.21.15.83:0/4180006482 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562afb5caa30 0x562afb5d2980 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:46.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.576+0000 7fa25f400640 0 client.18960 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:00:46.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.576+0000 7fa2839a8740 1 -- 172.21.15.83:0/4180006482 >> 172.21.15.83:0/4180006482 conn(0x562afb5c6d50 msgr2=0x562afb5c5fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:46.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.576+0000 7fa2839a8740 1 -- 172.21.15.83:0/4180006482 shutdown_connections 2024-04-24T09:00:46.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.576+0000 7fa2839a8740 1 -- 172.21.15.83:0/4180006482 wait complete. 2024-04-24T09:00:46.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.576+0000 7fa2839a8740 20 client.18960 trim_cache size 0 max 16384 2024-04-24T09:00:46.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.595+0000 7fa2839a8740 1 Processor -- start 2024-04-24T09:00:46.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.596+0000 7fa2839a8740 1 -- start start 2024-04-24T09:00:46.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.596+0000 7fa2839a8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562afb5caa30 0x562afb5e1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:46.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.596+0000 7fa2839a8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562afb5cb350 0x562afb5cccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:46.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.596+0000 7fa2839a8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562afb5d5a20 0x562afb5d4a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:46.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.596+0000 7fa2839a8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa27c1b57f0 con 0x562afb5caa30 2024-04-24T09:00:46.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.596+0000 7fa2839a8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa27c1b7330 con 0x562afb5cb350 2024-04-24T09:00:46.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.596+0000 7fa2839a8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa27c1b5ec0 con 0x562afb5d5a20 2024-04-24T09:00:46.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.596+0000 7fa279e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562afb5caa30 0x562afb5e1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:46.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.596+0000 7fa279e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562afb5caa30 0x562afb5e1670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48892/0 (socket says 172.21.15.83:48892) 2024-04-24T09:00:46.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.596+0000 7fa279e00640 1 -- 172.21.15.83:0/2834034396 learned_addr learned my addr 172.21.15.83:0/2834034396 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:00:46.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.596+0000 7fa273e00640 1 --2- 172.21.15.83:0/2834034396 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562afb5cb350 0x562afb5cccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:46.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.596+0000 7fa27a800640 1 --2- 172.21.15.83:0/2834034396 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562afb5d5a20 0x562afb5d4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:46.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.597+0000 7fa279e00640 1 -- 172.21.15.83:0/2834034396 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562afb5d5a20 msgr2=0x562afb5d4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:46.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.597+0000 7fa279e00640 1 --2- 172.21.15.83:0/2834034396 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562afb5d5a20 0x562afb5d4a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:46.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.597+0000 7fa279e00640 1 -- 172.21.15.83:0/2834034396 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562afb5cb350 msgr2=0x562afb5cccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:46.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.597+0000 7fa279e00640 1 --2- 172.21.15.83:0/2834034396 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562afb5cb350 0x562afb5cccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:46.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.597+0000 7fa279e00640 1 -- 172.21.15.83:0/2834034396 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562afb5d1930 con 0x562afb5caa30 2024-04-24T09:00:46.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.597+0000 7fa273e00640 1 --2- 172.21.15.83:0/2834034396 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562afb5cb350 0x562afb5cccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:00:46.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.597+0000 7fa27a800640 1 --2- 172.21.15.83:0/2834034396 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562afb5d5a20 0x562afb5d4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:00:46.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.597+0000 7fa279e00640 1 --2- 172.21.15.83:0/2834034396 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562afb5caa30 0x562afb5e1670 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7fa26c004ef0 tx=0x7fa26c004390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f8eea8ea159177d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:46.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.597+0000 7fa26a600640 1 -- 172.21.15.83:0/2834034396 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa26c035070 con 0x562afb5caa30 2024-04-24T09:00:46.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.597+0000 7fa26a600640 1 -- 172.21.15.83:0/2834034396 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa26c0047a0 con 0x562afb5caa30 2024-04-24T09:00:46.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.597+0000 7fa26a600640 1 -- 172.21.15.83:0/2834034396 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa26c005d50 con 0x562afb5caa30 2024-04-24T09:00:46.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.598+0000 7fa2839a8740 1 -- 172.21.15.83:0/2834034396 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562afb5caa30 msgr2=0x562afb5e1670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:46.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.598+0000 7fa2839a8740 1 --2- 172.21.15.83:0/2834034396 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562afb5caa30 0x562afb5e1670 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7fa26c004ef0 tx=0x7fa26c004390 comp rx=0 tx=0).stop 2024-04-24T09:00:46.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.598+0000 7fa2839a8740 1 -- 172.21.15.83:0/2834034396 shutdown_connections 2024-04-24T09:00:46.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.598+0000 7fa2839a8740 1 --2- 172.21.15.83:0/2834034396 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562afb5d5a20 0x562afb5d4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:46.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.598+0000 7fa2839a8740 1 --2- 172.21.15.83:0/2834034396 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562afb5cb350 0x562afb5cccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:46.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.598+0000 7fa2839a8740 1 --2- 172.21.15.83:0/2834034396 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562afb5caa30 0x562afb5e1670 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:46.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.598+0000 7fa2839a8740 1 -- 172.21.15.83:0/2834034396 >> 172.21.15.83:0/2834034396 conn(0x562afb5c6d50 msgr2=0x562afb5c7130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:46.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.598+0000 7fa2839a8740 1 -- 172.21.15.83:0/2834034396 shutdown_connections 2024-04-24T09:00:46.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.598+0000 7fa2839a8740 1 -- 172.21.15.83:0/2834034396 wait complete. 2024-04-24T09:00:46.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.599+0000 7fa2839a8740 1 -- start start 2024-04-24T09:00:46.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.599+0000 7fa2839a8740 1 Processor -- start 2024-04-24T09:00:46.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.599+0000 7fa2839a8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562afb5caa30 0x562afb645150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:46.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.599+0000 7fa2839a8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562afb5cb350 0x562afb645690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:46.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.599+0000 7fa2839a8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562afb5d5a20 0x562afb5d9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:46.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.599+0000 7fa2839a8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa27c1b6c60 con 0x562afb5caa30 2024-04-24T09:00:46.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.599+0000 7fa2839a8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa27c1b6590 con 0x562afb5d5a20 2024-04-24T09:00:46.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.599+0000 7fa2839a8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa27c1b37f0 con 0x562afb5cb350 2024-04-24T09:00:46.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.599+0000 7fa2839a8740 10 client.0 _ll_register_callbacks cb 0x7fa26c004260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:00:46.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.599+0000 7fa279e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562afb5caa30 0x562afb645150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:46.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.600+0000 7fa279e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562afb5caa30 0x562afb645150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48902/0 (socket says 172.21.15.83:48902) 2024-04-24T09:00:46.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.600+0000 7fa279e00640 1 -- 172.21.15.83:0/1314580254 learned_addr learned my addr 172.21.15.83:0/1314580254 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:00:46.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.600+0000 7fa27a800640 1 --2- 172.21.15.83:0/1314580254 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562afb5d5a20 0x562afb5d9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:46.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.600+0000 7fa273e00640 1 --2- 172.21.15.83:0/1314580254 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562afb5cb350 0x562afb645690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:46.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.600+0000 7fa279e00640 1 -- 172.21.15.83:0/1314580254 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562afb5cb350 msgr2=0x562afb645690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:46.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.600+0000 7fa279e00640 1 --2- 172.21.15.83:0/1314580254 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562afb5cb350 0x562afb645690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:46.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.600+0000 7fa279e00640 1 -- 172.21.15.83:0/1314580254 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562afb5d5a20 msgr2=0x562afb5d9d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:46.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.600+0000 7fa279e00640 1 --2- 172.21.15.83:0/1314580254 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562afb5d5a20 0x562afb5d9d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:46.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.600+0000 7fa279e00640 1 -- 172.21.15.83:0/1314580254 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562afb2423f0 con 0x562afb5caa30 2024-04-24T09:00:46.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.600+0000 7fa273e00640 1 --2- 172.21.15.83:0/1314580254 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562afb5cb350 0x562afb645690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:00:46.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.600+0000 7fa27a800640 1 --2- 172.21.15.83:0/1314580254 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562afb5d5a20 0x562afb5d9d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:00:46.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.600+0000 7fa279e00640 1 --2- 172.21.15.83:0/1314580254 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562afb5caa30 0x562afb645150 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7fa26c0059c0 tx=0x7fa26c005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3242897193918629 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:46.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.600+0000 7fa25f400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:00:46.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.601+0000 7fa25f400640 1 -- 172.21.15.83:0/1314580254 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa26c035040 con 0x562afb5caa30 2024-04-24T09:00:46.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.601+0000 7fa25f400640 1 -- 172.21.15.83:0/1314580254 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7fa26c0449d0 con 0x562afb5caa30 2024-04-24T09:00:46.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.601+0000 7fa2839a8740 1 -- 172.21.15.83:0/1314580254 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa26c0047a0 con 0x562afb5caa30 2024-04-24T09:00:46.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.601+0000 7fa25f400640 1 -- 172.21.15.83:0/1314580254 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fa26c06a050 con 0x562afb5caa30 2024-04-24T09:00:46.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.601+0000 7fa2839a8740 10 client.18966 fetch_fsmap learned FSMap version 35 2024-04-24T09:00:46.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.601+0000 7fa2839a8740 1 -- 172.21.15.83:0/1314580254 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x562afb2020f0 con 0x562afb5caa30 2024-04-24T09:00:46.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.601+0000 7fa25f400640 1 -- 172.21.15.83:0/1314580254 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa26c0047a0 con 0x562afb5caa30 2024-04-24T09:00:46.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.601+0000 7fa25f400640 1 -- 172.21.15.83:0/1314580254 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7fa26c003980 con 0x562afb5caa30 2024-04-24T09:00:46.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.601+0000 7fa2839a8740 10 client.18966 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T09:00:46.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.601+0000 7fa2839a8740 10 client.18966 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:00:46.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.601+0000 7fa2839a8740 20 client.18966 populate_metadata read hostname 'smithi083' 2024-04-24T09:00:46.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.601+0000 7fa2839a8740 1 --2- 172.21.15.83:0/1314580254 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x562afb64c8f0 0x562afb64ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:46.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.601+0000 7fa2839a8740 4 client.18966 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T09:00:46.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.601+0000 7fa2839a8740 1 -- 172.21.15.83:0/1314580254 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562afb537a80 con 0x562afb64c8f0 2024-04-24T09:00:46.605 INFO:tasks.workunit.client.0.smithi083.stdout: 1 52239 0.62 MB/sec execute 320 sec latency 53.779 ms 2024-04-24T09:00:46.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.602+0000 7fa273e00640 1 --2- 172.21.15.83:0/1314580254 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x562afb64c8f0 0x562afb64ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:46.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.603+0000 7fa273e00640 1 --2- 172.21.15.83:0/1314580254 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x562afb64c8f0 0x562afb64ece0 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e89dbd47f59028e server_cookie=86e5d4429a0fb8a3 in_seq=0 out_seq=0 2024-04-24T09:00:46.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.603+0000 7fa25f400640 10 client.18966 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:00:46.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.605+0000 7fa25f400640 1 -- 172.21.15.83:0/1314580254 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x562afb537a80 con 0x562afb64c8f0 2024-04-24T09:00:46.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.605+0000 7fa25f400640 10 client.18966 handle_command_reply: tid=0 2024-04-24T09:00:46.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.605+0000 7fa2839a8740 1 client.18966 shutdown 2024-04-24T09:00:46.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.605+0000 7fa2839a8740 1 -- 172.21.15.83:0/1314580254 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562afb5caa30 msgr2=0x562afb645150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:46.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.605+0000 7fa2839a8740 1 --2- 172.21.15.83:0/1314580254 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562afb5caa30 0x562afb645150 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7fa26c0059c0 tx=0x7fa26c005d50 comp rx=0 tx=0).stop 2024-04-24T09:00:46.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.605+0000 7fa2839a8740 1 -- 172.21.15.83:0/1314580254 shutdown_connections 2024-04-24T09:00:46.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.605+0000 7fa2839a8740 1 --2- 172.21.15.83:0/1314580254 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x562afb64c8f0 0x562afb64ece0 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:46.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.605+0000 7fa2839a8740 1 --2- 172.21.15.83:0/1314580254 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562afb5d5a20 0x562afb5d9d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:46.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.605+0000 7fa2839a8740 1 --2- 172.21.15.83:0/1314580254 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562afb5cb350 0x562afb645690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:46.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.605+0000 7fa2839a8740 1 --2- 172.21.15.83:0/1314580254 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562afb5caa30 0x562afb645150 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:46.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.605+0000 7fa25f400640 0 client.18966 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:00:46.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.605+0000 7fa2839a8740 1 -- 172.21.15.83:0/1314580254 >> 172.21.15.83:0/1314580254 conn(0x562afb5c6d50 msgr2=0x562afb5d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:46.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.605+0000 7fa2839a8740 1 -- 172.21.15.83:0/1314580254 shutdown_connections 2024-04-24T09:00:46.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.605+0000 7fa2839a8740 1 -- 172.21.15.83:0/1314580254 wait complete. 2024-04-24T09:00:46.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.605+0000 7fa2839a8740 20 client.18966 trim_cache size 0 max 16384 2024-04-24T09:00:46.608 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T09:00:46.608 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T09:00:46.608 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-04-24T09:00:46.608 INFO:teuthology.orchestra.run.smithi083.stdout: "1dc86011-0659-4e57-9057-47314c0dc613": { 2024-04-24T09:00:46.608 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-04-24T09:00:46.608 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "1dc86011-0659-4e57-9057-47314c0dc613", 2024-04-24T09:00:46.608 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-04-24T09:00:46.608 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:00:46.609 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:00:46.609 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T09:00:46.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.606+0000 7fa27b200640 1 -- 172.21.15.83:0/720295596 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa2600a3d20 msgr2=0x7fa2600a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:46.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.606+0000 7fa27b200640 1 --2- 172.21.15.83:0/720295596 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa2600a3d20 0x7fa2600a61e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa274007db0 tx=0x7fa274007380 comp rx=0 tx=0).stop 2024-04-24T09:00:46.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.606+0000 7fa27b200640 1 -- 172.21.15.83:0/720295596 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa27c101770 msgr2=0x7fa27c1a4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:46.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.606+0000 7fa27b200640 1 --2- 172.21.15.83:0/720295596 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa27c101770 0x7fa27c1a4d80 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7fa26400c970 tx=0x7fa26400ce30 comp rx=0 tx=0).stop 2024-04-24T09:00:46.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.606+0000 7fa27b200640 1 -- 172.21.15.83:0/720295596 shutdown_connections 2024-04-24T09:00:46.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.606+0000 7fa27b200640 1 --2- 172.21.15.83:0/720295596 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa2600a3d20 0x7fa2600a61e0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:46.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.606+0000 7fa27b200640 1 --2- 172.21.15.83:0/720295596 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa27c106250 0x7fa27c19e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:46.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.606+0000 7fa27b200640 1 --2- 172.21.15.83:0/720295596 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa27c101770 0x7fa27c1a4d80 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:46.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.606+0000 7fa27b200640 1 --2- 172.21.15.83:0/720295596 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa27c100e50 0x7fa27c1a4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:46.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.606+0000 7fa27b200640 1 -- 172.21.15.83:0/720295596 >> 172.21.15.83:0/720295596 conn(0x7fa27c0fc880 msgr2=0x7fa27c0fd850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:46.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.606+0000 7fa27b200640 1 -- 172.21.15.83:0/720295596 shutdown_connections 2024-04-24T09:00:46.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:00:46.606+0000 7fa27b200640 1 -- 172.21.15.83:0/720295596 wait complete. 2024-04-24T09:00:46.623 INFO:tasks.cephfs.filesystem:scrub status for tag:1dc86011-0659-4e57-9057-47314c0dc613 - {'path': '/', 'tag': '1dc86011-0659-4e57-9057-47314c0dc613', 'options': 'recursive,force'} 2024-04-24T09:00:46.705 INFO:journalctl@ceph.mds.b.smithi138.stdout:Apr 24 09:00:46 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-b[61515]: 2024-04-24T09:00:46.358+0000 7f7fbbc00700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000215 [...e,head] /volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/COREL/ auth{2=1} v8934 ap=2 f(v364 m2024-04-24T09:00:30.407990+0000 11=11+0) n(v21 rc2024-04-24T09:00:30.407990+0000 b2861056 12=11+1) old_inodes=6 (inest sync->mix g=2) caps={15603=pAsLsXsFs/-@7536} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x55cf8760d700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000000215:[<0x10000000009/COREL v369>,<0x10000000008/~dmtmp v57>,<0x10000000007/client0 v33>,<0x10000000006/clients v23>,<0x10000000005/tmp v23>,<0x10000000003/client.0 v22>,<0x10000000002/e395bed6-5bba-41b8-a256-3505e4afcca2 v36>,<0x10000000001/sv_0 v55>,<0x10000000000/qa v43>,<0x1/volumes v39>]//[]","memoryvalue":"(2)0x10000000215:[<0x10000000009/COREL v8934>,<0x10000000008/~dmtmp v8395>,<0x10000000007/client0 v990>,<0x10000000006/clients v982>,<0x10000000005/tmp v976>,<0x10000000003/client.0 v973>,<0x10000000002/e395bed6-5bba-41b8-a256-3505e4afcca2 v964>,<0x10000000001/sv_0 v960>,<0x10000000000/qa v945>,<0x1/volumes v918>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-04-24T09:00:30.364268+0000 12=12+0)","ondisk_value.rstat":"n(v0 rc2024-04-24T09:00:30.407990+0000 b2861056 12=11+1)","memory_value.dirstat":"f(v364 m2024-04-24T09:00:30.407990+0000 11=11+0)","memory_value.rstat":"n(v21 rc2024-04-24T09:00:30.407990+0000 b2861056 12=11+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-04-24T09:00:46.705 INFO:journalctl@ceph.mds.b.smithi138.stdout:Apr 24 09:00:46 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-b[61515]: 2024-04-24T09:00:46.363+0000 7f7fbbc00700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000221 [...e,head] /volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/WORDPRO/ auth{3=1} v8892 ap=2 f(v490 m2024-04-24T09:00:30.659840+0000 3=3+0) n(v31 rc2024-04-24T09:00:30.659840+0000 b780288 4=3+1) old_inodes=6 (inest sync->mix g=3) caps={15603=pAsLsXsFs/-@7944} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x55cf87627700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000000221:[<0x10000000009/WORDPRO v375>,<0x10000000008/~dmtmp v57>,<0x10000000007/client0 v33>,<0x10000000006/clients v23>,<0x10000000005/tmp v23>,<0x10000000003/client.0 v22>,<0x10000000002/e395bed6-5bba-41b8-a256-3505e4afcca2 v36>,<0x10000000001/sv_0 v55>,<0x10000000000/qa v43>,<0x1/volumes v39>]//[]","memoryvalue":"(2)0x10000000221:[<0x10000000009/WORDPRO v8892>,<0x10000000008/~dmtmp v8395>,<0x10000000007/client0 v990>,<0x10000000006/clients v982>,<0x10000000005/tmp v976>,<0x10000000003/client.0 v973>,<0x10000000002/e395bed6-5bba-41b8-a256-3505e4afcca2 v964>,<0x10000000001/sv_0 v960>,<0x10000000000/qa v945>,<0x1/volumes v918>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-04-24T09:00:30.619412+0000 4=4+0)","ondisk_value.rstat":"n(v0 rc2024-04-24T09:00:30.659840+0000 b780288 4=3+1)","memory_value.dirstat":"f(v490 m2024-04-24T09:00:30.659840+0000 3=3+0)","memory_value.rstat":"n(v31 rc2024-04-24T09:00:30.659840+0000 b780288 4=3+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-04-24T09:00:47.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:00:46 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:00:47.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:00:46 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:00:47.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:00:46 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:00:47.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:00:46 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:00:47.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:00:46 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:00:47.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:00:46 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:00:47.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:00:46 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:00:47.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:00:46 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:00:47.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:00:46 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:00:47.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:00:46 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:00:47.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:00:46 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:00:47.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:00:46 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:00:47.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:00:46 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:00:47.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:00:46 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:00:47.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:00:46 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:00:47.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:00:46 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:00:47.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:00:46 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:00:47.209 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:00:46 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:00:47.209 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:00:46 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:00:47.209 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:00:46 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:00:47.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:00:46 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:00:47.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:00:46 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:00:47.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:00:46 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:00:47.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:00:46 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:00:47.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:00:46 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:00:47.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:00:46 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:00:47.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:00:46 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:00:47.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:00:46 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:00:47.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:00:46 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:00:47.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:00:46 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:00:47.602 INFO:tasks.workunit.client.0.smithi083.stdout: 1 52308 0.62 MB/sec execute 321 sec latency 64.433 ms 2024-04-24T09:00:48.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:00:47 smithi138 ceph-mon[31441]: scrub queued for path: / 2024-04-24T09:00:48.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:00:47 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:00:48.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:00:47 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:00:48.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:00:47 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:00:48.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:00:47 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:00:48.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:00:47 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:00:48.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:00:47 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:00:48.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:00:47 smithi138 ceph-mon[31441]: Scrub error on inode 0x10000000215 (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/COREL) see mds.b log and `damage ls` output for details 2024-04-24T09:00:48.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:00:47 smithi138 ceph-mon[31441]: Scrub error on inode 0x10000000221 (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/WORDPRO) see mds.b log and `damage ls` output for details 2024-04-24T09:00:48.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:00:47 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:00:48.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:00:47 smithi138 ceph-mon[31441]: pgmap v286: 129 pgs: 129 active+clean; 51 MiB data, 718 MiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 262 KiB/s wr, 47 op/s 2024-04-24T09:00:48.209 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:00:47 smithi189 ceph-mon[31791]: scrub queued for path: / 2024-04-24T09:00:48.210 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:00:47 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:00:48.210 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:00:47 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:00:48.210 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:00:47 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:00:48.210 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:00:47 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:00:48.210 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:00:47 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:00:48.210 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:00:47 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:00:48.210 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:00:47 smithi189 ceph-mon[31791]: Scrub error on inode 0x10000000215 (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/COREL) see mds.b log and `damage ls` output for details 2024-04-24T09:00:48.210 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:00:47 smithi189 ceph-mon[31791]: Scrub error on inode 0x10000000221 (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/WORDPRO) see mds.b log and `damage ls` output for details 2024-04-24T09:00:48.210 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:00:47 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:00:48.210 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:00:47 smithi189 ceph-mon[31791]: pgmap v286: 129 pgs: 129 active+clean; 51 MiB data, 718 MiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 262 KiB/s wr, 47 op/s 2024-04-24T09:00:48.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:00:47 smithi083 ceph-mon[27907]: scrub queued for path: / 2024-04-24T09:00:48.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:00:47 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:00:48.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:00:47 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:00:48.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:00:47 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:00:48.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:00:47 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:00:48.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:00:47 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:00:48.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:00:47 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:00:48.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:00:47 smithi083 ceph-mon[27907]: Scrub error on inode 0x10000000215 (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/COREL) see mds.b log and `damage ls` output for details 2024-04-24T09:00:48.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:00:47 smithi083 ceph-mon[27907]: Scrub error on inode 0x10000000221 (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/WORDPRO) see mds.b log and `damage ls` output for details 2024-04-24T09:00:48.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:00:47 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:00:48.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:00:47 smithi083 ceph-mon[27907]: pgmap v286: 129 pgs: 129 active+clean; 51 MiB data, 718 MiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 262 KiB/s wr, 47 op/s 2024-04-24T09:00:48.602 INFO:tasks.workunit.client.0.smithi083.stdout: 1 52588 0.63 MB/sec execute 322 sec latency 51.795 ms 2024-04-24T09:00:49.602 INFO:tasks.workunit.client.0.smithi083.stdout: 1 52700 0.63 MB/sec execute 323 sec latency 47.572 ms 2024-04-24T09:00:50.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:00:49 smithi138 ceph-mon[31441]: pgmap v287: 129 pgs: 129 active+clean; 52 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 499 KiB/s wr, 76 op/s 2024-04-24T09:00:50.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:00:49 smithi189 ceph-mon[31791]: pgmap v287: 129 pgs: 129 active+clean; 52 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 499 KiB/s wr, 76 op/s 2024-04-24T09:00:50.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:00:49 smithi083 ceph-mon[27907]: pgmap v287: 129 pgs: 129 active+clean; 52 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 499 KiB/s wr, 76 op/s 2024-04-24T09:00:50.602 INFO:tasks.workunit.client.0.smithi083.stdout: 1 52967 0.63 MB/sec execute 324 sec latency 58.371 ms 2024-04-24T09:00:51.603 INFO:tasks.workunit.client.0.smithi083.stdout: 1 52997 0.63 MB/sec execute 325 sec latency 53.757 ms 2024-04-24T09:00:51.981 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:51.982+0000 7fb3ca600640 1 -- 172.21.15.83:0/4118817475 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb3c4108540 msgr2=0x7fb3c4108980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:51.981 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:51.982+0000 7fb3ca600640 1 --2- 172.21.15.83:0/4118817475 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb3c4108540 0x7fb3c4108980 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac009930 tx=0x7fb3ac02a420 comp rx=0 tx=0).stop 2024-04-24T09:00:51.981 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:51.982+0000 7fb3ca600640 1 -- 172.21.15.83:0/4118817475 shutdown_connections 2024-04-24T09:00:51.981 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:51.982+0000 7fb3ca600640 1 --2- 172.21.15.83:0/4118817475 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb3c4108ec0 0x7fb3c4115300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:51.981 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:51.982+0000 7fb3ca600640 1 --2- 172.21.15.83:0/4118817475 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb3c4108540 0x7fb3c4108980 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:51.981 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:51.982+0000 7fb3ca600640 1 --2- 172.21.15.83:0/4118817475 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3c410e4d0 0x7fb3c410e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:51.981 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:51.982+0000 7fb3ca600640 1 -- 172.21.15.83:0/4118817475 >> 172.21.15.83:0/4118817475 conn(0x7fb3c4077de0 msgr2=0x7fb3c4106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:51.982 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:51.982+0000 7fb3ca600640 1 -- 172.21.15.83:0/4118817475 shutdown_connections 2024-04-24T09:00:51.982 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:51.982+0000 7fb3ca600640 1 -- 172.21.15.83:0/4118817475 wait complete. 2024-04-24T09:00:51.983 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:51.983+0000 7fb3ca600640 1 Processor -- start 2024-04-24T09:00:51.983 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:51.983+0000 7fb3ca600640 1 -- start start 2024-04-24T09:00:51.983 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:51.984+0000 7fb3ca600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb3c4108540 0x7fb3c41a48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:51.983 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:51.984+0000 7fb3ca600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3c4108ec0 0x7fb3c41a4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:51.983 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:51.984+0000 7fb3ca600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb3c410e4d0 0x7fb3c419ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:51.983 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:51.984+0000 7fb3ca600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb3c41186f0 con 0x7fb3c4108540 2024-04-24T09:00:51.983 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:51.984+0000 7fb3ca600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb3c4118570 con 0x7fb3c4108ec0 2024-04-24T09:00:51.983 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:51.984+0000 7fb3ca600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb3c4118870 con 0x7fb3c410e4d0 2024-04-24T09:00:51.983 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:51.984+0000 7fb3c9200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb3c4108540 0x7fb3c41a48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:51.983 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:51.984+0000 7fb3c9200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb3c4108540 0x7fb3c41a48e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49970/0 (socket says 172.21.15.83:49970) 2024-04-24T09:00:51.983 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:51.984+0000 7fb3c9200640 1 -- 172.21.15.83:0/1607815795 learned_addr learned my addr 172.21.15.83:0/1607815795 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:00:51.984 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:51.984+0000 7fb3bbe00640 1 --2- 172.21.15.83:0/1607815795 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3c4108ec0 0x7fb3c41a4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:51.984 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:51.984+0000 7fb3c9200640 1 -- 172.21.15.83:0/1607815795 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb3c410e4d0 msgr2=0x7fb3c419ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:51.985 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:51.984+0000 7fb3c9200640 1 --2- 172.21.15.83:0/1607815795 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb3c410e4d0 0x7fb3c419ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:51.985 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:51.984+0000 7fb3c9200640 1 -- 172.21.15.83:0/1607815795 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3c4108ec0 msgr2=0x7fb3c41a4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:51.985 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:51.984+0000 7fb3c9200640 1 --2- 172.21.15.83:0/1607815795 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3c4108ec0 0x7fb3c41a4e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:51.985 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:51.984+0000 7fb3c9200640 1 -- 172.21.15.83:0/1607815795 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3c419f210 con 0x7fb3c4108540 2024-04-24T09:00:51.985 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:51.985+0000 7fb3c9200640 1 --2- 172.21.15.83:0/1607815795 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb3c4108540 0x7fb3c41a48e0 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7fb3b400cce0 tx=0x7fb3b400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:51.985 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:51.985+0000 7fb3c2000640 1 -- 172.21.15.83:0/1607815795 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b4004b20 con 0x7fb3c4108540 2024-04-24T09:00:51.985 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:51.985+0000 7fb3c2000640 1 -- 172.21.15.83:0/1607815795 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb3b4004cc0 con 0x7fb3c4108540 2024-04-24T09:00:51.985 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:51.985+0000 7fb3c2000640 1 -- 172.21.15.83:0/1607815795 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b400f6b0 con 0x7fb3c4108540 2024-04-24T09:00:51.985 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:51.985+0000 7fb3ca600640 1 -- 172.21.15.83:0/1607815795 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3c419f500 con 0x7fb3c4108540 2024-04-24T09:00:51.985 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:51.985+0000 7fb3ca600640 1 -- 172.21.15.83:0/1607815795 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3c41ab7a0 con 0x7fb3c4108540 2024-04-24T09:00:51.989 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:51.989+0000 7fb3ceb92740 1 Processor -- start 2024-04-24T09:00:51.989 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:51.989+0000 7fb3c2000640 1 -- 172.21.15.83:0/1607815795 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb3b400f850 con 0x7fb3c4108540 2024-04-24T09:00:51.989 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:51.989+0000 7fb3ceb92740 1 -- start start 2024-04-24T09:00:51.989 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:51.989+0000 7fb3ceb92740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55813556f3c0 0x55813556f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:51.989 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:51.989+0000 7fb3ceb92740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55813556fce0 0x55813557a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:51.990 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:51.989+0000 7fb3ceb92740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55813557a5a0 0x55813557c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:51.990 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:51.989+0000 7fb3ceb92740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5581354d5900 con 0x55813556fce0 2024-04-24T09:00:51.990 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:51.989+0000 7fb3ceb92740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55813547bf20 con 0x55813556f3c0 2024-04-24T09:00:51.990 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:51.989+0000 7fb3ceb92740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5581354d18c0 con 0x55813557a5a0 2024-04-24T09:00:51.990 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:51.989+0000 7fb3c9200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55813556fce0 0x55813557a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:51.990 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:51.989+0000 7fb3c9200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55813556fce0 0x55813557a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49974/0 (socket says 172.21.15.83:49974) 2024-04-24T09:00:51.990 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:51.989+0000 7fb3c9200640 1 -- 172.21.15.83:0/3271075017 learned_addr learned my addr 172.21.15.83:0/3271075017 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:00:51.990 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:51.990+0000 7fb3c9c00640 1 --2- 172.21.15.83:0/3271075017 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55813556f3c0 0x55813556f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:51.990 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:51.990+0000 7fb3bbe00640 1 --2- 172.21.15.83:0/3271075017 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55813557a5a0 0x55813557c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:51.990 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:51.990+0000 7fb3c9200640 1 -- 172.21.15.83:0/3271075017 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55813557a5a0 msgr2=0x55813557c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:51.990 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:51.990+0000 7fb3c9200640 1 --2- 172.21.15.83:0/3271075017 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55813557a5a0 0x55813557c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:51.990 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:51.990+0000 7fb3c9200640 1 -- 172.21.15.83:0/3271075017 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55813556f3c0 msgr2=0x55813556f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:51.990 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:51.990+0000 7fb3c9200640 1 --2- 172.21.15.83:0/3271075017 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55813556f3c0 0x55813556f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:51.991 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:51.990+0000 7fb3c9200640 1 -- 172.21.15.83:0/3271075017 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581354a0e10 con 0x55813556fce0 2024-04-24T09:00:51.991 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:51.990+0000 7fb3bbe00640 1 --2- 172.21.15.83:0/3271075017 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55813557a5a0 0x55813557c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:00:51.991 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:51.990+0000 7fb3c2000640 1 --2- 172.21.15.83:0/1607815795 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb3a80a3d10 0x7fb3a80a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:51.991 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:51.990+0000 7fb3c9200640 1 --2- 172.21.15.83:0/3271075017 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55813556fce0 0x55813557a060 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7fb3b40cdf50 tx=0x7fb3b40d4f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77602fc5ba41e3ee server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:51.995 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:51.991+0000 7fb3c9c00640 1 --2- 172.21.15.83:0/1607815795 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb3a80a3d10 0x7fb3a80a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:51.995 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:51.991+0000 7fb3c2000640 1 -- 172.21.15.83:0/1607815795 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7fb3b40d0d40 con 0x7fb3c4108540 2024-04-24T09:00:51.995 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:51.991+0000 7fb3ba000640 1 -- 172.21.15.83:0/3271075017 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b40e6070 con 0x55813556fce0 2024-04-24T09:00:51.995 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:51.991+0000 7fb3ba000640 1 -- 172.21.15.83:0/3271075017 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb3b409cf60 con 0x55813556fce0 2024-04-24T09:00:51.996 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:51.991+0000 7fb3ba000640 1 -- 172.21.15.83:0/3271075017 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b409c3a0 con 0x55813556fce0 2024-04-24T09:00:51.996 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:51.992+0000 7fb3ceb92740 1 -- 172.21.15.83:0/3271075017 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55813556fce0 msgr2=0x55813557a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:51.996 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:51.992+0000 7fb3c9c00640 1 --2- 172.21.15.83:0/1607815795 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb3a80a3d10 0x7fb3a80a61d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc005fd0 tx=0x7fb3bc005e00 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:51.996 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:51.992+0000 7fb3ceb92740 1 --2- 172.21.15.83:0/3271075017 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55813556fce0 0x55813557a060 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7fb3b40cdf50 tx=0x7fb3b40d4f90 comp rx=0 tx=0).stop 2024-04-24T09:00:51.996 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:51.992+0000 7fb3ceb92740 1 -- 172.21.15.83:0/3271075017 shutdown_connections 2024-04-24T09:00:51.996 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:51.992+0000 7fb3ceb92740 1 --2- 172.21.15.83:0/3271075017 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55813557a5a0 0x55813557c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:51.996 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:51.992+0000 7fb3ceb92740 1 --2- 172.21.15.83:0/3271075017 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55813556fce0 0x55813557a060 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:51.996 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:51.992+0000 7fb3ceb92740 1 --2- 172.21.15.83:0/3271075017 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55813556f3c0 0x55813556f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:51.996 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:51.992+0000 7fb3ceb92740 1 -- 172.21.15.83:0/3271075017 >> 172.21.15.83:0/3271075017 conn(0x55813556b8d0 msgr2=0x55813556bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:51.996 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:51.992+0000 7fb3ceb92740 1 -- 172.21.15.83:0/3271075017 shutdown_connections 2024-04-24T09:00:51.996 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:51.992+0000 7fb3ceb92740 1 -- 172.21.15.83:0/3271075017 wait complete. 2024-04-24T09:00:51.996 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:51.993+0000 7fb3ceb92740 1 -- start start 2024-04-24T09:00:51.996 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:51.993+0000 7fb3ceb92740 1 Processor -- start 2024-04-24T09:00:51.996 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:51.993+0000 7fb3ceb92740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55813556f3c0 0x5581355775f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:51.996 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:51.993+0000 7fb3ceb92740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55813556fce0 0x558135577b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:51.997 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:51.993+0000 7fb3ceb92740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55813557a5a0 0x558135578070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:51.997 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:51.993+0000 7fb3ceb92740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55813557d140 con 0x55813557a5a0 2024-04-24T09:00:51.997 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:51.993+0000 7fb3ceb92740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55813548f2c0 con 0x55813556fce0 2024-04-24T09:00:51.997 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:51.993+0000 7fb3ceb92740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55813557d2c0 con 0x55813556f3c0 2024-04-24T09:00:51.997 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:51.994+0000 7fb3ceb92740 10 client.0 _ll_register_callbacks cb 0x5581351e7a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:00:51.997 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:51.994+0000 7fb3c9c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55813557a5a0 0x558135578070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:51.997 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:51.994+0000 7fb3c9c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55813557a5a0 0x558135578070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49990/0 (socket says 172.21.15.83:49990) 2024-04-24T09:00:51.997 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:51.994+0000 7fb3c9c00640 1 -- 172.21.15.83:0/2541645490 learned_addr learned my addr 172.21.15.83:0/2541645490 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:00:51.997 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:51.994+0000 7fb3c9200640 1 --2- 172.21.15.83:0/2541645490 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55813556f3c0 0x5581355775f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:51.997 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:51.994+0000 7fb3bbe00640 1 --2- 172.21.15.83:0/2541645490 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55813556fce0 0x558135577b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:51.997 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:51.994+0000 7fb3c9c00640 1 -- 172.21.15.83:0/2541645490 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55813556f3c0 msgr2=0x5581355775f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:51.997 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:51.994+0000 7fb3c9c00640 1 --2- 172.21.15.83:0/2541645490 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55813556f3c0 0x5581355775f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:51.997 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:51.994+0000 7fb3c9c00640 1 -- 172.21.15.83:0/2541645490 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55813556fce0 msgr2=0x558135577b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:51.997 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:51.994+0000 7fb3c9c00640 1 --2- 172.21.15.83:0/2541645490 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55813556fce0 0x558135577b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:51.997 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:51.994+0000 7fb3c9c00640 1 -- 172.21.15.83:0/2541645490 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55813549ffd0 con 0x55813557a5a0 2024-04-24T09:00:51.998 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:51.994+0000 7fb3c9200640 1 --2- 172.21.15.83:0/2541645490 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55813556f3c0 0x5581355775f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:00:51.998 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:51.995+0000 7fb3c9c00640 1 --2- 172.21.15.83:0/2541645490 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55813557a5a0 0x558135578070 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc0107b0 tx=0x7fb3bc010bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=173a12bfee6204fa server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:51.998 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:51.995+0000 7fb3b2a00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:00:51.998 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:51.995+0000 7fb3b2a00640 1 -- 172.21.15.83:0/2541645490 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3bc020070 con 0x55813557a5a0 2024-04-24T09:00:51.998 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:51.995+0000 7fb3b2a00640 1 -- 172.21.15.83:0/2541645490 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7fb3bc00fcd0 con 0x55813557a5a0 2024-04-24T09:00:51.998 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:51.995+0000 7fb3ceb92740 1 -- 172.21.15.83:0/2541645490 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3b409cf60 con 0x55813557a5a0 2024-04-24T09:00:51.998 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:51.995+0000 7fb3b2a00640 1 -- 172.21.15.83:0/2541645490 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fb3bc02e050 con 0x55813557a5a0 2024-04-24T09:00:51.998 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:51.995+0000 7fb3ceb92740 10 client.18978 fetch_fsmap learned FSMap version 35 2024-04-24T09:00:51.998 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:51.995+0000 7fb3ceb92740 1 -- 172.21.15.83:0/2541645490 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5581351e8210 con 0x55813557a5a0 2024-04-24T09:00:51.999 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:51.996+0000 7fb3b2a00640 1 -- 172.21.15.83:0/2541645490 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb3b409cf60 con 0x55813557a5a0 2024-04-24T09:00:51.999 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:51.996+0000 7fb3b2a00640 1 -- 172.21.15.83:0/2541645490 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7fb3bc01c420 con 0x55813557a5a0 2024-04-24T09:00:51.999 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:51.996+0000 7fb3ceb92740 10 client.18978 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T09:00:51.999 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:51.996+0000 7fb3ceb92740 10 client.18978 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:00:51.999 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:51.996+0000 7fb3ceb92740 20 client.18978 populate_metadata read hostname 'smithi083' 2024-04-24T09:00:51.999 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:51.996+0000 7fb3ceb92740 1 --2- 172.21.15.83:0/2541645490 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55813558a820 0x55813558cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:51.999 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:51.996+0000 7fb3ceb92740 4 client.18978 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:00:51.999 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:51.996+0000 7fb3ceb92740 1 -- 172.21.15.83:0/2541645490 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5581354e3320 con 0x55813558a820 2024-04-24T09:00:51.999 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:51.997+0000 7fb3c9200640 1 --2- 172.21.15.83:0/2541645490 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55813558a820 0x55813558cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:52.003 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:51.997+0000 7fb3c9200640 1 --2- 172.21.15.83:0/2541645490 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55813558a820 0x55813558cc30 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7fa8eaa856ec69f server_cookie=4cb853b0001c962b in_seq=0 out_seq=0 2024-04-24T09:00:52.003 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:51.998+0000 7fb3b2a00640 10 client.18978 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:00:52.003 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:52.001+0000 7fb3b2a00640 1 -- 172.21.15.83:0/2541645490 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5581354e3320 con 0x55813558a820 2024-04-24T09:00:52.003 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:52.001+0000 7fb3b2a00640 10 client.18978 handle_command_reply: tid=0 2024-04-24T09:00:52.003 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:52.001+0000 7fb3ceb92740 1 client.18978 shutdown 2024-04-24T09:00:52.003 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:52.001+0000 7fb3ceb92740 1 -- 172.21.15.83:0/2541645490 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55813557a5a0 msgr2=0x558135578070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:52.003 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:52.001+0000 7fb3ceb92740 1 --2- 172.21.15.83:0/2541645490 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55813557a5a0 0x558135578070 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc0107b0 tx=0x7fb3bc010bb0 comp rx=0 tx=0).stop 2024-04-24T09:00:52.003 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:52.001+0000 7fb3ceb92740 1 -- 172.21.15.83:0/2541645490 shutdown_connections 2024-04-24T09:00:52.003 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:52.001+0000 7fb3ceb92740 1 --2- 172.21.15.83:0/2541645490 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55813558a820 0x55813558cc30 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:52.003 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:52.001+0000 7fb3ceb92740 1 --2- 172.21.15.83:0/2541645490 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55813557a5a0 0x558135578070 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:52.003 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:52.001+0000 7fb3ceb92740 1 --2- 172.21.15.83:0/2541645490 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55813556fce0 0x558135577b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:52.003 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:52.001+0000 7fb3ceb92740 1 --2- 172.21.15.83:0/2541645490 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55813556f3c0 0x5581355775f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:52.004 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:52.001+0000 7fb3b2a00640 0 client.18978 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:00:52.004 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:52.001+0000 7fb3ceb92740 1 -- 172.21.15.83:0/2541645490 >> 172.21.15.83:0/2541645490 conn(0x55813556b8d0 msgr2=0x558135571650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:52.004 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:52.001+0000 7fb3ceb92740 1 -- 172.21.15.83:0/2541645490 shutdown_connections 2024-04-24T09:00:52.004 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:52.002+0000 7fb3ceb92740 1 -- 172.21.15.83:0/2541645490 wait complete. 2024-04-24T09:00:52.004 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:52.002+0000 7fb3ceb92740 20 client.18978 trim_cache size 0 max 16384 2024-04-24T09:00:52.020 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:52.021+0000 7fb3ceb92740 1 Processor -- start 2024-04-24T09:00:52.020 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:52.022+0000 7fb3ceb92740 1 -- start start 2024-04-24T09:00:52.022 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:52.022+0000 7fb3ceb92740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55813556f3c0 0x558135579d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:52.022 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:52.022+0000 7fb3ceb92740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55813556fce0 0x55813556b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:52.022 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:52.022+0000 7fb3ceb92740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55813557a5a0 0x558135592540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:52.022 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:52.022+0000 7fb3ceb92740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb3c41b5c00 con 0x55813557a5a0 2024-04-24T09:00:52.022 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:52.022+0000 7fb3ceb92740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb3c41b7740 con 0x55813556fce0 2024-04-24T09:00:52.022 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:52.022+0000 7fb3ceb92740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb3c41b62d0 con 0x55813556f3c0 2024-04-24T09:00:52.022 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:52.022+0000 7fb3c9c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55813557a5a0 0x558135592540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:52.022 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:52.022+0000 7fb3c9c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55813557a5a0 0x558135592540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50002/0 (socket says 172.21.15.83:50002) 2024-04-24T09:00:52.022 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:52.022+0000 7fb3c9c00640 1 -- 172.21.15.83:0/2335959370 learned_addr learned my addr 172.21.15.83:0/2335959370 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:00:52.022 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:52.023+0000 7fb3c9200640 1 --2- 172.21.15.83:0/2335959370 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55813556f3c0 0x558135579d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:52.023 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:52.023+0000 7fb3bbe00640 1 --2- 172.21.15.83:0/2335959370 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55813556fce0 0x55813556b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:52.023 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:52.023+0000 7fb3c9c00640 1 -- 172.21.15.83:0/2335959370 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55813556f3c0 msgr2=0x558135579d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:52.023 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:52.023+0000 7fb3c9c00640 1 --2- 172.21.15.83:0/2335959370 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55813556f3c0 0x558135579d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:52.023 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:52.023+0000 7fb3c9c00640 1 -- 172.21.15.83:0/2335959370 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55813556fce0 msgr2=0x55813556b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:52.023 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:52.023+0000 7fb3c9c00640 1 --2- 172.21.15.83:0/2335959370 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55813556fce0 0x55813556b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:52.023 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:52.023+0000 7fb3c9c00640 1 -- 172.21.15.83:0/2335959370 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581352b4620 con 0x55813557a5a0 2024-04-24T09:00:52.023 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:52.023+0000 7fb3bbe00640 1 --2- 172.21.15.83:0/2335959370 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55813556fce0 0x55813556b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:00:52.023 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:52.023+0000 7fb3c9c00640 1 --2- 172.21.15.83:0/2335959370 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55813557a5a0 0x558135592540 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc00edf0 tx=0x7fb3bc005a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d73e73c72d4ffb72 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:52.024 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:52.023+0000 7fb3b9600640 1 -- 172.21.15.83:0/2335959370 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3bc02b040 con 0x55813557a5a0 2024-04-24T09:00:52.025 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:52.023+0000 7fb3b9600640 1 -- 172.21.15.83:0/2335959370 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb3bc01c420 con 0x55813557a5a0 2024-04-24T09:00:52.025 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:52.023+0000 7fb3b9600640 1 -- 172.21.15.83:0/2335959370 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3bc020040 con 0x55813557a5a0 2024-04-24T09:00:52.025 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:52.024+0000 7fb3ceb92740 1 -- 172.21.15.83:0/2335959370 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55813557a5a0 msgr2=0x558135592540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:52.025 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:52.024+0000 7fb3ceb92740 1 --2- 172.21.15.83:0/2335959370 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55813557a5a0 0x558135592540 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc00edf0 tx=0x7fb3bc005a10 comp rx=0 tx=0).stop 2024-04-24T09:00:52.025 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:52.024+0000 7fb3ceb92740 1 -- 172.21.15.83:0/2335959370 shutdown_connections 2024-04-24T09:00:52.025 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:52.024+0000 7fb3ceb92740 1 --2- 172.21.15.83:0/2335959370 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55813557a5a0 0x558135592540 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:52.025 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:52.024+0000 7fb3ceb92740 1 --2- 172.21.15.83:0/2335959370 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55813556fce0 0x55813556b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:52.025 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:52.024+0000 7fb3ceb92740 1 --2- 172.21.15.83:0/2335959370 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55813556f3c0 0x558135579d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:52.025 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:52.024+0000 7fb3ceb92740 1 -- 172.21.15.83:0/2335959370 >> 172.21.15.83:0/2335959370 conn(0x55813556b8d0 msgr2=0x558135570d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:52.025 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:52.024+0000 7fb3ceb92740 1 -- 172.21.15.83:0/2335959370 shutdown_connections 2024-04-24T09:00:52.025 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:52.024+0000 7fb3ceb92740 1 -- 172.21.15.83:0/2335959370 wait complete. 2024-04-24T09:00:52.025 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:52.025+0000 7fb3ceb92740 1 -- start start 2024-04-24T09:00:52.025 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:52.025+0000 7fb3ceb92740 1 Processor -- start 2024-04-24T09:00:52.025 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:52.025+0000 7fb3ceb92740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55813556f3c0 0x558135582810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:52.025 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:52.025+0000 7fb3ceb92740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55813556fce0 0x55813557f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:52.026 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:52.025+0000 7fb3ceb92740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55813557a5a0 0x55813557fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:52.026 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:52.025+0000 7fb3ceb92740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb3c41b7070 con 0x55813557a5a0 2024-04-24T09:00:52.026 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:52.025+0000 7fb3ceb92740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb3c41b69a0 con 0x55813556f3c0 2024-04-24T09:00:52.026 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:52.025+0000 7fb3ceb92740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb3c41b3c00 con 0x55813556fce0 2024-04-24T09:00:52.026 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:52.026+0000 7fb3ceb92740 10 client.0 _ll_register_callbacks cb 0x7fb3bc014050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:00:52.026 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:52.026+0000 7fb3c9c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55813557a5a0 0x55813557fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:52.026 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:52.026+0000 7fb3c9c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55813557a5a0 0x55813557fb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50006/0 (socket says 172.21.15.83:50006) 2024-04-24T09:00:52.026 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:52.026+0000 7fb3c9c00640 1 -- 172.21.15.83:0/4063353015 learned_addr learned my addr 172.21.15.83:0/4063353015 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:00:52.028 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:52.026+0000 7fb3c9200640 1 --2- 172.21.15.83:0/4063353015 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55813556f3c0 0x558135582810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:52.028 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:52.026+0000 7fb3bbe00640 1 --2- 172.21.15.83:0/4063353015 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55813556fce0 0x55813557f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:52.028 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:52.026+0000 7fb3c9c00640 1 -- 172.21.15.83:0/4063353015 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55813556fce0 msgr2=0x55813557f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:52.028 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:52.026+0000 7fb3c9c00640 1 --2- 172.21.15.83:0/4063353015 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55813556fce0 0x55813557f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:52.029 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:52.026+0000 7fb3c9c00640 1 -- 172.21.15.83:0/4063353015 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55813556f3c0 msgr2=0x558135582810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:00:52.029 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:52.026+0000 7fb3c9c00640 1 --2- 172.21.15.83:0/4063353015 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55813556f3c0 0x558135582810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:52.029 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:52.026+0000 7fb3c9c00640 1 -- 172.21.15.83:0/4063353015 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558135580070 con 0x55813557a5a0 2024-04-24T09:00:52.029 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:52.026+0000 7fb3c9200640 1 --2- 172.21.15.83:0/4063353015 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55813556f3c0 0x558135582810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:00:52.029 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:52.026+0000 7fb3bbe00640 1 --2- 172.21.15.83:0/4063353015 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55813556fce0 0x55813557f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:00:52.029 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:52.026+0000 7fb3c9c00640 1 --2- 172.21.15.83:0/4063353015 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55813557a5a0 0x55813557fb30 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc01cf30 tx=0x7fb3bc01cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d16b1bb8583b8c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:00:52.029 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:52.027+0000 7fb3b2a00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:00:52.029 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:52.027+0000 7fb3b2a00640 1 -- 172.21.15.83:0/4063353015 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3bc03e020 con 0x55813557a5a0 2024-04-24T09:00:52.029 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:52.027+0000 7fb3b2a00640 1 -- 172.21.15.83:0/4063353015 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9583+0+0 (secure 0 0 0) 0x7fb3bc025460 con 0x55813557a5a0 2024-04-24T09:00:52.029 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:52.027+0000 7fb3ceb92740 1 -- 172.21.15.83:0/4063353015 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3bc01c420 con 0x55813557a5a0 2024-04-24T09:00:52.030 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:52.027+0000 7fb3b2a00640 1 -- 172.21.15.83:0/4063353015 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fb3bc049050 con 0x55813557a5a0 2024-04-24T09:00:52.030 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:52.027+0000 7fb3ceb92740 10 client.18984 fetch_fsmap learned FSMap version 35 2024-04-24T09:00:52.030 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:52.027+0000 7fb3ceb92740 1 -- 172.21.15.83:0/4063353015 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5581355f19c0 con 0x55813557a5a0 2024-04-24T09:00:52.030 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:52.027+0000 7fb3b2a00640 1 -- 172.21.15.83:0/4063353015 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb3bc01c420 con 0x55813557a5a0 2024-04-24T09:00:52.030 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:52.028+0000 7fb3b2a00640 1 -- 172.21.15.83:0/4063353015 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7fb3bc03a670 con 0x55813557a5a0 2024-04-24T09:00:52.030 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:52.028+0000 7fb3ceb92740 10 client.18984 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T09:00:52.030 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:52.028+0000 7fb3ceb92740 10 client.18984 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:00:52.030 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:52.028+0000 7fb3ceb92740 20 client.18984 populate_metadata read hostname 'smithi083' 2024-04-24T09:00:52.030 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:52.028+0000 7fb3ceb92740 1 --2- 172.21.15.83:0/4063353015 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5581355f1fc0 0x5581355f43e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:00:52.030 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:52.028+0000 7fb3ceb92740 4 client.18984 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T09:00:52.030 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:52.028+0000 7fb3ceb92740 1 -- 172.21.15.83:0/4063353015 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5581354dd190 con 0x5581355f1fc0 2024-04-24T09:00:52.031 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:52.028+0000 7fb3c9200640 1 --2- 172.21.15.83:0/4063353015 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5581355f1fc0 0x5581355f43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:00:52.031 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:52.029+0000 7fb3c9200640 1 --2- 172.21.15.83:0/4063353015 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5581355f1fc0 0x5581355f43e0 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e833e4f4ad2d0c7e server_cookie=aeb0dfc401a4bb09 in_seq=0 out_seq=0 2024-04-24T09:00:52.031 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:52.029+0000 7fb3b2a00640 10 client.18984 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:00:52.096 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:52.097+0000 7fb3b2a00640 1 -- 172.21.15.83:0/4063353015 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5581354dd190 con 0x5581355f1fc0 2024-04-24T09:00:52.096 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:52.097+0000 7fb3b2a00640 10 client.18984 handle_command_reply: tid=0 2024-04-24T09:00:52.096 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:52.097+0000 7fb3ceb92740 1 client.18984 shutdown 2024-04-24T09:00:52.097 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:52.098+0000 7fb3ceb92740 1 -- 172.21.15.83:0/4063353015 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55813557a5a0 msgr2=0x55813557fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:52.097 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:52.098+0000 7fb3ceb92740 1 --2- 172.21.15.83:0/4063353015 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55813557a5a0 0x55813557fb30 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc01cf30 tx=0x7fb3bc01cc10 comp rx=0 tx=0).stop 2024-04-24T09:00:52.097 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:52.098+0000 7fb3ceb92740 1 -- 172.21.15.83:0/4063353015 shutdown_connections 2024-04-24T09:00:52.097 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:52.098+0000 7fb3ceb92740 1 --2- 172.21.15.83:0/4063353015 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5581355f1fc0 0x5581355f43e0 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:52.097 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:52.098+0000 7fb3ceb92740 1 --2- 172.21.15.83:0/4063353015 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55813557a5a0 0x55813557fb30 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:52.097 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:52.098+0000 7fb3ceb92740 1 --2- 172.21.15.83:0/4063353015 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55813556fce0 0x55813557f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:52.097 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:52.098+0000 7fb3b2a00640 0 client.18984 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:00:52.097 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:52.098+0000 7fb3ceb92740 1 --2- 172.21.15.83:0/4063353015 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55813556f3c0 0x558135582810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:52.097 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:52.098+0000 7fb3ceb92740 1 -- 172.21.15.83:0/4063353015 >> 172.21.15.83:0/4063353015 conn(0x55813556b8d0 msgr2=0x558135571600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:52.098 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:52.098+0000 7fb3ceb92740 1 -- 172.21.15.83:0/4063353015 shutdown_connections 2024-04-24T09:00:52.098 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:52.098+0000 7fb3ceb92740 1 -- 172.21.15.83:0/4063353015 wait complete. 2024-04-24T09:00:52.098 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:52.099+0000 7fb3ceb92740 20 client.18984 trim_cache size 0 max 16384 2024-04-24T09:00:52.098 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T09:00:52.099 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T09:00:52.099 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T09:00:52.099 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T09:00:52.099 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:52.100+0000 7fb3ca600640 1 -- 172.21.15.83:0/1607815795 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb3a80a3d10 msgr2=0x7fb3a80a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:52.099 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:52.100+0000 7fb3ca600640 1 --2- 172.21.15.83:0/1607815795 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb3a80a3d10 0x7fb3a80a61d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc005fd0 tx=0x7fb3bc005e00 comp rx=0 tx=0).stop 2024-04-24T09:00:52.100 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:52.100+0000 7fb3ca600640 1 -- 172.21.15.83:0/1607815795 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb3c4108540 msgr2=0x7fb3c41a48e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:00:52.100 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:52.100+0000 7fb3ca600640 1 --2- 172.21.15.83:0/1607815795 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb3c4108540 0x7fb3c41a48e0 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7fb3b400cce0 tx=0x7fb3b400a620 comp rx=0 tx=0).stop 2024-04-24T09:00:52.100 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:52.100+0000 7fb3ca600640 1 -- 172.21.15.83:0/1607815795 shutdown_connections 2024-04-24T09:00:52.100 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:52.100+0000 7fb3ca600640 1 --2- 172.21.15.83:0/1607815795 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb3a80a3d10 0x7fb3a80a61d0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:52.100 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:52.100+0000 7fb3ca600640 1 --2- 172.21.15.83:0/1607815795 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb3c410e4d0 0x7fb3c419ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:52.100 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:52.100+0000 7fb3ca600640 1 --2- 172.21.15.83:0/1607815795 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3c4108ec0 0x7fb3c41a4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:52.100 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:52.100+0000 7fb3ca600640 1 --2- 172.21.15.83:0/1607815795 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb3c4108540 0x7fb3c41a48e0 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:00:52.100 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:52.100+0000 7fb3ca600640 1 -- 172.21.15.83:0/1607815795 >> 172.21.15.83:0/1607815795 conn(0x7fb3c4077de0 msgr2=0x7fb3c4078460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:00:52.100 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:52.100+0000 7fb3ca600640 1 -- 172.21.15.83:0/1607815795 shutdown_connections 2024-04-24T09:00:52.100 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:00:52.101+0000 7fb3ca600640 1 -- 172.21.15.83:0/1607815795 wait complete. 2024-04-24T09:00:52.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:00:51 smithi138 ceph-mon[31441]: pgmap v288: 129 pgs: 129 active+clean; 52 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 318 KiB/s wr, 51 op/s 2024-04-24T09:00:52.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:00:51 smithi189 ceph-mon[31791]: pgmap v288: 129 pgs: 129 active+clean; 52 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 318 KiB/s wr, 51 op/s 2024-04-24T09:00:52.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:00:51 smithi083 ceph-mon[27907]: pgmap v288: 129 pgs: 129 active+clean; 52 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 318 KiB/s wr, 51 op/s 2024-04-24T09:00:52.603 INFO:tasks.workunit.client.0.smithi083.stdout: 1 53022 0.63 MB/sec execute 326 sec latency 54.946 ms 2024-04-24T09:00:53.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:00:52 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:00:53.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:00:52 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:00:53.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:00:52 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:00:53.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:00:52 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:00:53.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:00:52 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:00:53.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:00:52 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:00:53.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:00:52 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:00:53.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:00:52 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:00:53.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:00:52 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:00:53.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:00:52 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:00:53.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:00:52 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:00:53.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:00:52 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:00:53.603 INFO:tasks.workunit.client.0.smithi083.stdout: 1 53195 0.64 MB/sec execute 327 sec latency 49.533 ms 2024-04-24T09:00:54.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:00:53 smithi138 ceph-mon[31441]: pgmap v289: 129 pgs: 129 active+clean; 54 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 510 KiB/s wr, 71 op/s 2024-04-24T09:00:54.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:00:53 smithi189 ceph-mon[31791]: pgmap v289: 129 pgs: 129 active+clean; 54 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 510 KiB/s wr, 71 op/s 2024-04-24T09:00:54.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:00:53 smithi083 ceph-mon[27907]: pgmap v289: 129 pgs: 129 active+clean; 54 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 510 KiB/s wr, 71 op/s 2024-04-24T09:00:54.604 INFO:tasks.workunit.client.0.smithi083.stdout: 1 53318 0.64 MB/sec execute 328 sec latency 57.570 ms 2024-04-24T09:00:55.603 INFO:tasks.workunit.client.0.smithi083.stdout: 1 53438 0.64 MB/sec execute 329 sec latency 48.874 ms 2024-04-24T09:00:56.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:00:55 smithi138 ceph-mon[31441]: pgmap v290: 129 pgs: 129 active+clean; 54 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 429 KiB/s wr, 48 op/s 2024-04-24T09:00:56.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:00:55 smithi189 ceph-mon[31791]: pgmap v290: 129 pgs: 129 active+clean; 54 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 429 KiB/s wr, 48 op/s 2024-04-24T09:00:56.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:00:55 smithi083 ceph-mon[27907]: pgmap v290: 129 pgs: 129 active+clean; 54 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 429 KiB/s wr, 48 op/s 2024-04-24T09:00:56.603 INFO:tasks.workunit.client.0.smithi083.stdout: 1 53493 0.64 MB/sec execute 330 sec latency 51.971 ms 2024-04-24T09:00:57.603 INFO:tasks.workunit.client.0.smithi083.stdout: 1 53553 0.63 MB/sec execute 331 sec latency 48.466 ms 2024-04-24T09:00:58.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:00:57 smithi138 ceph-mon[31441]: pgmap v291: 129 pgs: 129 active+clean; 54 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 429 KiB/s wr, 48 op/s 2024-04-24T09:00:58.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:00:57 smithi189 ceph-mon[31791]: pgmap v291: 129 pgs: 129 active+clean; 54 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 429 KiB/s wr, 48 op/s 2024-04-24T09:00:58.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:00:57 smithi083 ceph-mon[27907]: pgmap v291: 129 pgs: 129 active+clean; 54 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 429 KiB/s wr, 48 op/s 2024-04-24T09:00:58.603 INFO:tasks.workunit.client.0.smithi083.stdout: 1 53619 0.63 MB/sec execute 332 sec latency 52.658 ms 2024-04-24T09:00:59.604 INFO:tasks.workunit.client.0.smithi083.stdout: 1 53677 0.63 MB/sec execute 333 sec latency 42.587 ms 2024-04-24T09:01:00.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:00:59 smithi138 ceph-mon[31441]: pgmap v292: 129 pgs: 129 active+clean; 55 MiB data, 816 MiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 682 KiB/s wr, 71 op/s 2024-04-24T09:01:00.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:00:59 smithi189 ceph-mon[31791]: pgmap v292: 129 pgs: 129 active+clean; 55 MiB data, 816 MiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 682 KiB/s wr, 71 op/s 2024-04-24T09:01:00.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:00:59 smithi083 ceph-mon[27907]: pgmap v292: 129 pgs: 129 active+clean; 55 MiB data, 816 MiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 682 KiB/s wr, 71 op/s 2024-04-24T09:01:00.603 INFO:tasks.workunit.client.0.smithi083.stdout: 1 53738 0.63 MB/sec execute 334 sec latency 66.939 ms 2024-04-24T09:01:01.604 INFO:tasks.workunit.client.0.smithi083.stdout: 1 53818 0.63 MB/sec execute 335 sec latency 49.260 ms 2024-04-24T09:01:02.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:01:01 smithi138 ceph-mon[31441]: pgmap v293: 129 pgs: 129 active+clean; 55 MiB data, 816 MiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 444 KiB/s wr, 42 op/s 2024-04-24T09:01:02.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:01:01 smithi189 ceph-mon[31791]: pgmap v293: 129 pgs: 129 active+clean; 55 MiB data, 816 MiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 444 KiB/s wr, 42 op/s 2024-04-24T09:01:02.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:01:01 smithi083 ceph-mon[27907]: pgmap v293: 129 pgs: 129 active+clean; 55 MiB data, 816 MiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 444 KiB/s wr, 42 op/s 2024-04-24T09:01:02.604 INFO:tasks.workunit.client.0.smithi083.stdout: 1 53940 0.63 MB/sec execute 336 sec latency 47.819 ms 2024-04-24T09:01:03.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:01:02 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:01:03.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:01:02 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:01:03.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:01:02 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:01:03.604 INFO:tasks.workunit.client.0.smithi083.stdout: 1 54110 0.63 MB/sec execute 337 sec latency 45.436 ms 2024-04-24T09:01:04.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:01:03 smithi138 ceph-mon[31441]: pgmap v294: 129 pgs: 129 active+clean; 57 MiB data, 838 MiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 621 KiB/s wr, 82 op/s 2024-04-24T09:01:04.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:01:03 smithi138 ceph-mon[31441]: osdmap e103: 12 total, 12 up, 12 in 2024-04-24T09:01:04.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:01:03 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:01:04.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:01:03 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:01:04.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:01:03 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:01:04.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:01:03 smithi189 ceph-mon[31791]: pgmap v294: 129 pgs: 129 active+clean; 57 MiB data, 838 MiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 621 KiB/s wr, 82 op/s 2024-04-24T09:01:04.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:01:03 smithi189 ceph-mon[31791]: osdmap e103: 12 total, 12 up, 12 in 2024-04-24T09:01:04.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:01:03 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:01:04.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:01:03 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:01:04.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:01:03 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:01:04.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:01:03 smithi083 ceph-mon[27907]: pgmap v294: 129 pgs: 129 active+clean; 57 MiB data, 838 MiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 621 KiB/s wr, 82 op/s 2024-04-24T09:01:04.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:01:03 smithi083 ceph-mon[27907]: osdmap e103: 12 total, 12 up, 12 in 2024-04-24T09:01:04.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:01:03 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:01:04.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:01:03 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:01:04.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:01:03 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:01:04.604 INFO:tasks.workunit.client.0.smithi083.stdout: 1 54216 0.62 MB/sec execute 338 sec latency 56.517 ms 2024-04-24T09:01:05.262 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.261+0000 7fbb8e600640 1 -- 172.21.15.83:0/2347317204 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb88104c60 msgr2=0x7fbb88108ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:05.263 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.261+0000 7fbb8e600640 1 --2- 172.21.15.83:0/2347317204 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb88104c60 0x7fbb88108ca0 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7fbb70009900 tx=0x7fbb7002a3f0 comp rx=0 tx=0).stop 2024-04-24T09:01:05.263 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.261+0000 7fbb8e600640 1 -- 172.21.15.83:0/2347317204 shutdown_connections 2024-04-24T09:01:05.263 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.261+0000 7fbb8e600640 1 --2- 172.21.15.83:0/2347317204 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb88109380 0x7fbb88110f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:05.263 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.261+0000 7fbb8e600640 1 --2- 172.21.15.83:0/2347317204 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb88104c60 0x7fbb88108ca0 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:05.263 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.261+0000 7fbb8e600640 1 --2- 172.21.15.83:0/2347317204 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbb881042b0 0x7fbb88104690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:05.263 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.261+0000 7fbb8e600640 1 -- 172.21.15.83:0/2347317204 >> 172.21.15.83:0/2347317204 conn(0x7fbb880ffb10 msgr2=0x7fbb88101f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:05.263 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.262+0000 7fbb8e600640 1 -- 172.21.15.83:0/2347317204 shutdown_connections 2024-04-24T09:01:05.263 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.262+0000 7fbb8e600640 1 -- 172.21.15.83:0/2347317204 wait complete. 2024-04-24T09:01:05.263 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.262+0000 7fbb8e600640 1 Processor -- start 2024-04-24T09:01:05.263 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.263+0000 7fbb8e600640 1 -- start start 2024-04-24T09:01:05.265 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.263+0000 7fbb8e600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb881042b0 0x7fbb881a05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:05.265 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.263+0000 7fbb8e600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb88109380 0x7fbb881a0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:05.265 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.263+0000 7fbb8e600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbb8819a690 0x7fbb8819ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:05.265 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.263+0000 7fbb8e600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbb881142f0 con 0x7fbb88109380 2024-04-24T09:01:05.265 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.263+0000 7fbb8e600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbb88114170 con 0x7fbb881042b0 2024-04-24T09:01:05.265 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.263+0000 7fbb8e600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fbb88114470 con 0x7fbb8819a690 2024-04-24T09:01:05.265 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.263+0000 7fbb7fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb88109380 0x7fbb881a0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:05.265 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.263+0000 7fbb7fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb88109380 0x7fbb881a0b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35028/0 (socket says 172.21.15.83:35028) 2024-04-24T09:01:05.266 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.263+0000 7fbb7fe00640 1 -- 172.21.15.83:0/2052024936 learned_addr learned my addr 172.21.15.83:0/2052024936 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:01:05.266 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.264+0000 7fbb8dc00640 1 --2- 172.21.15.83:0/2052024936 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbb8819a690 0x7fbb8819ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:05.267 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.264+0000 7fbb7fe00640 1 -- 172.21.15.83:0/2052024936 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbb8819a690 msgr2=0x7fbb8819ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:05.267 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.264+0000 7fbb7fe00640 1 --2- 172.21.15.83:0/2052024936 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbb8819a690 0x7fbb8819ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:05.267 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.264+0000 7fbb7fe00640 1 -- 172.21.15.83:0/2052024936 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb881042b0 msgr2=0x7fbb881a05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:05.267 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.264+0000 7fbb7fe00640 1 --2- 172.21.15.83:0/2052024936 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb881042b0 0x7fbb881a05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:05.267 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.264+0000 7fbb7fe00640 1 -- 172.21.15.83:0/2052024936 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb8819b290 con 0x7fbb88109380 2024-04-24T09:01:05.267 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.264+0000 7fbb7fe00640 1 --2- 172.21.15.83:0/2052024936 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb88109380 0x7fbb881a0b10 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7fbb70005ab0 tx=0x7fbb70003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:05.267 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.264+0000 7fbb86000640 1 -- 172.21.15.83:0/2052024936 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb7003f070 con 0x7fbb88109380 2024-04-24T09:01:05.267 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.264+0000 7fbb86000640 1 -- 172.21.15.83:0/2052024936 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbb70031d60 con 0x7fbb88109380 2024-04-24T09:01:05.268 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.264+0000 7fbb8e600640 1 -- 172.21.15.83:0/2052024936 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb8819b520 con 0x7fbb88109380 2024-04-24T09:01:05.268 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.264+0000 7fbb86000640 1 -- 172.21.15.83:0/2052024936 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb70035030 con 0x7fbb88109380 2024-04-24T09:01:05.268 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.265+0000 7fbb8e600640 1 -- 172.21.15.83:0/2052024936 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb881a7660 con 0x7fbb88109380 2024-04-24T09:01:05.268 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.269+0000 7fbb9299d740 1 Processor -- start 2024-04-24T09:01:05.277 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.269+0000 7fbb9299d740 1 -- start start 2024-04-24T09:01:05.277 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.269+0000 7fbb86000640 1 -- 172.21.15.83:0/2052024936 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fbb70042070 con 0x7fbb88109380 2024-04-24T09:01:05.277 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.269+0000 7fbb9299d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cc40d013c0 0x55cc40d017a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:05.278 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.269+0000 7fbb9299d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cc40d01ce0 0x55cc40d0c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:05.278 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.269+0000 7fbb9299d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cc40d0c5a0 0x55cc40d0e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:05.278 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.269+0000 7fbb9299d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55cc40c67900 con 0x55cc40d0c5a0 2024-04-24T09:01:05.278 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.269+0000 7fbb9299d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55cc40c0df20 con 0x55cc40d013c0 2024-04-24T09:01:05.278 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.269+0000 7fbb9299d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55cc40c638c0 con 0x55cc40d01ce0 2024-04-24T09:01:05.278 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.270+0000 7fbb7fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cc40d0c5a0 0x55cc40d0e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:05.278 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.270+0000 7fbb7fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cc40d0c5a0 0x55cc40d0e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35038/0 (socket says 172.21.15.83:35038) 2024-04-24T09:01:05.279 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.270+0000 7fbb7fe00640 1 -- 172.21.15.83:0/131560715 learned_addr learned my addr 172.21.15.83:0/131560715 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:01:05.279 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.270+0000 7fbb8d200640 1 --2- 172.21.15.83:0/131560715 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cc40d01ce0 0x55cc40d0c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:05.279 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.270+0000 7fbb8dc00640 1 --2- 172.21.15.83:0/131560715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cc40d013c0 0x55cc40d017a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:05.279 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.270+0000 7fbb7fe00640 1 -- 172.21.15.83:0/131560715 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cc40d01ce0 msgr2=0x55cc40d0c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:05.279 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.270+0000 7fbb7fe00640 1 --2- 172.21.15.83:0/131560715 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cc40d01ce0 0x55cc40d0c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:05.279 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.270+0000 7fbb7fe00640 1 -- 172.21.15.83:0/131560715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cc40d013c0 msgr2=0x55cc40d017a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:05.279 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.270+0000 7fbb7fe00640 1 --2- 172.21.15.83:0/131560715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cc40d013c0 0x55cc40d017a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:05.280 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.270+0000 7fbb7fe00640 1 -- 172.21.15.83:0/131560715 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc40c32e10 con 0x55cc40d0c5a0 2024-04-24T09:01:05.280 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.271+0000 7fbb7fe00640 1 --2- 172.21.15.83:0/131560715 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cc40d0c5a0 0x55cc40d0e990 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7fbb700ee720 tx=0x7fbb700f7770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11381630ad918ed5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:05.280 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.271+0000 7fbb86000640 1 --2- 172.21.15.83:0/2052024936 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fbb6c0a3d10 0x7fbb6c0a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:05.280 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.271+0000 7fbb7e000640 1 -- 172.21.15.83:0/131560715 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb700b92d0 con 0x55cc40d0c5a0 2024-04-24T09:01:05.280 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.271+0000 7fbb7e000640 1 -- 172.21.15.83:0/131560715 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbb700fe030 con 0x55cc40d0c5a0 2024-04-24T09:01:05.280 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.271+0000 7fbb7e000640 1 -- 172.21.15.83:0/131560715 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb700bc7b0 con 0x55cc40d0c5a0 2024-04-24T09:01:05.280 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.271+0000 7fbb8dc00640 1 --2- 172.21.15.83:0/2052024936 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fbb6c0a3d10 0x7fbb6c0a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:05.280 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.272+0000 7fbb86000640 1 -- 172.21.15.83:0/2052024936 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9671+0+0 (secure 0 0 0) 0x7fbb700f1570 con 0x7fbb88109380 2024-04-24T09:01:05.280 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.272+0000 7fbb8dc00640 1 --2- 172.21.15.83:0/2052024936 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fbb6c0a3d10 0x7fbb6c0a61d0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fbb80005fd0 tx=0x7fbb800096e0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:05.280 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.272+0000 7fbb9299d740 1 -- 172.21.15.83:0/131560715 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cc40d0c5a0 msgr2=0x55cc40d0e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:05.280 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.272+0000 7fbb9299d740 1 --2- 172.21.15.83:0/131560715 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cc40d0c5a0 0x55cc40d0e990 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7fbb700ee720 tx=0x7fbb700f7770 comp rx=0 tx=0).stop 2024-04-24T09:01:05.280 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.272+0000 7fbb9299d740 1 -- 172.21.15.83:0/131560715 shutdown_connections 2024-04-24T09:01:05.280 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.272+0000 7fbb9299d740 1 --2- 172.21.15.83:0/131560715 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cc40d0c5a0 0x55cc40d0e990 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:05.281 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.272+0000 7fbb9299d740 1 --2- 172.21.15.83:0/131560715 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cc40d01ce0 0x55cc40d0c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:05.281 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.272+0000 7fbb9299d740 1 --2- 172.21.15.83:0/131560715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cc40d013c0 0x55cc40d017a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:05.281 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.272+0000 7fbb9299d740 1 -- 172.21.15.83:0/131560715 >> 172.21.15.83:0/131560715 conn(0x55cc40cfd8d0 msgr2=0x55cc40cfdcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:05.281 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.273+0000 7fbb9299d740 1 -- 172.21.15.83:0/131560715 shutdown_connections 2024-04-24T09:01:05.281 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.273+0000 7fbb9299d740 1 -- 172.21.15.83:0/131560715 wait complete. 2024-04-24T09:01:05.281 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.273+0000 7fbb9299d740 1 -- start start 2024-04-24T09:01:05.281 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.273+0000 7fbb9299d740 1 Processor -- start 2024-04-24T09:01:05.281 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.274+0000 7fbb9299d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cc40d013c0 0x55cc40d094d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:05.281 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.274+0000 7fbb9299d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cc40d01ce0 0x55cc40d09a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:05.281 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.274+0000 7fbb9299d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cc40d0c5a0 0x55cc40d09f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:05.281 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.274+0000 7fbb9299d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55cc40d0ef90 con 0x55cc40d013c0 2024-04-24T09:01:05.282 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.274+0000 7fbb9299d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55cc40c212c0 con 0x55cc40d01ce0 2024-04-24T09:01:05.282 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.274+0000 7fbb9299d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55cc40d0f110 con 0x55cc40d0c5a0 2024-04-24T09:01:05.282 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.274+0000 7fbb9299d740 10 client.0 _ll_register_callbacks cb 0x55cc40979a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:01:05.282 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.274+0000 7fbb8d200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cc40d013c0 0x55cc40d094d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:05.282 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.274+0000 7fbb8d200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cc40d013c0 0x55cc40d094d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35048/0 (socket says 172.21.15.83:35048) 2024-04-24T09:01:05.282 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.274+0000 7fbb8d200640 1 -- 172.21.15.83:0/4010016476 learned_addr learned my addr 172.21.15.83:0/4010016476 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:01:05.282 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.274+0000 7fbb8dc00640 1 --2- 172.21.15.83:0/4010016476 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cc40d0c5a0 0x55cc40d09f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:05.283 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.274+0000 7fbb7fe00640 1 --2- 172.21.15.83:0/4010016476 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cc40d01ce0 0x55cc40d09a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:05.283 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.274+0000 7fbb8d200640 1 -- 172.21.15.83:0/4010016476 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cc40d0c5a0 msgr2=0x55cc40d09f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:05.283 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.274+0000 7fbb8d200640 1 --2- 172.21.15.83:0/4010016476 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cc40d0c5a0 0x55cc40d09f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:05.284 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.275+0000 7fbb8d200640 1 -- 172.21.15.83:0/4010016476 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cc40d01ce0 msgr2=0x55cc40d09a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:05.284 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.275+0000 7fbb8d200640 1 --2- 172.21.15.83:0/4010016476 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cc40d01ce0 0x55cc40d09a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:05.284 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.275+0000 7fbb8d200640 1 -- 172.21.15.83:0/4010016476 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc40c31fd0 con 0x55cc40d013c0 2024-04-24T09:01:05.284 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.275+0000 7fbb7fe00640 1 --2- 172.21.15.83:0/4010016476 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cc40d01ce0 0x55cc40d09a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:01:05.284 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.275+0000 7fbb8d200640 1 --2- 172.21.15.83:0/4010016476 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cc40d013c0 0x55cc40d094d0 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7fbb7800f6e0 tx=0x7fbb7800fba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad5e8f5e80b58bd7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:05.284 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.275+0000 7fbb76a00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:01:05.284 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.275+0000 7fbb76a00640 1 -- 172.21.15.83:0/4010016476 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb78004220 con 0x55cc40d013c0 2024-04-24T09:01:05.284 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.275+0000 7fbb76a00640 1 -- 172.21.15.83:0/4010016476 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9671+0+0 (secure 0 0 0) 0x7fbb780043c0 con 0x55cc40d013c0 2024-04-24T09:01:05.285 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.275+0000 7fbb9299d740 1 -- 172.21.15.83:0/4010016476 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb700fe030 con 0x55cc40d013c0 2024-04-24T09:01:05.285 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.276+0000 7fbb76a00640 1 -- 172.21.15.83:0/4010016476 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fbb7800d8c0 con 0x55cc40d013c0 2024-04-24T09:01:05.285 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.276+0000 7fbb9299d740 10 client.18996 fetch_fsmap learned FSMap version 35 2024-04-24T09:01:05.285 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.276+0000 7fbb9299d740 1 -- 172.21.15.83:0/4010016476 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55cc4097a210 con 0x55cc40d013c0 2024-04-24T09:01:05.285 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.276+0000 7fbb76a00640 1 -- 172.21.15.83:0/4010016476 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbb700fe030 con 0x55cc40d013c0 2024-04-24T09:01:05.285 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.276+0000 7fbb76a00640 1 -- 172.21.15.83:0/4010016476 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7fbb78005020 con 0x55cc40d013c0 2024-04-24T09:01:05.285 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.276+0000 7fbb9299d740 10 client.18996 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T09:01:05.285 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.276+0000 7fbb9299d740 10 client.18996 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:01:05.285 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.277+0000 7fbb9299d740 20 client.18996 populate_metadata read hostname 'smithi083' 2024-04-24T09:01:05.285 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.277+0000 7fbb9299d740 1 --2- 172.21.15.83:0/4010016476 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55cc40d1c460 0x55cc40d1e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:05.286 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.277+0000 7fbb9299d740 4 client.18996 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:01:05.286 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.277+0000 7fbb9299d740 1 -- 172.21.15.83:0/4010016476 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cc40c75320 con 0x55cc40d1c460 2024-04-24T09:01:05.286 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.277+0000 7fbb7fe00640 1 --2- 172.21.15.83:0/4010016476 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55cc40d1c460 0x55cc40d1e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:05.287 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.279+0000 7fbb76a00640 10 client.18996 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:01:05.287 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.279+0000 7fbb7fe00640 1 --2- 172.21.15.83:0/4010016476 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55cc40d1c460 0x55cc40d1e8b0 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=117d819d479b639f server_cookie=e63f651d7f151157 in_seq=0 out_seq=0 2024-04-24T09:01:05.288 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.283+0000 7fbb76a00640 1 -- 172.21.15.83:0/4010016476 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7fbb7009be50 con 0x55cc40d1c460 2024-04-24T09:01:05.288 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.283+0000 7fbb76a00640 10 client.18996 handle_command_reply: tid=0 2024-04-24T09:01:05.288 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.283+0000 7fbb9299d740 1 client.18996 shutdown 2024-04-24T09:01:05.288 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.284+0000 7fbb9299d740 1 -- 172.21.15.83:0/4010016476 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cc40d013c0 msgr2=0x55cc40d094d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:05.288 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.284+0000 7fbb9299d740 1 --2- 172.21.15.83:0/4010016476 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cc40d013c0 0x55cc40d094d0 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7fbb7800f6e0 tx=0x7fbb7800fba0 comp rx=0 tx=0).stop 2024-04-24T09:01:05.288 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.284+0000 7fbb9299d740 1 -- 172.21.15.83:0/4010016476 shutdown_connections 2024-04-24T09:01:05.288 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.284+0000 7fbb9299d740 1 --2- 172.21.15.83:0/4010016476 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55cc40d1c460 0x55cc40d1e8b0 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:05.288 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.284+0000 7fbb9299d740 1 --2- 172.21.15.83:0/4010016476 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cc40d0c5a0 0x55cc40d09f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:05.288 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.284+0000 7fbb9299d740 1 --2- 172.21.15.83:0/4010016476 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cc40d01ce0 0x55cc40d09a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:05.288 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.284+0000 7fbb9299d740 1 --2- 172.21.15.83:0/4010016476 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cc40d013c0 0x55cc40d094d0 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:05.289 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.284+0000 7fbb76a00640 0 client.18996 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:01:05.289 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.284+0000 7fbb9299d740 1 -- 172.21.15.83:0/4010016476 >> 172.21.15.83:0/4010016476 conn(0x55cc40cfd8d0 msgr2=0x55cc40d03600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:05.289 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.284+0000 7fbb9299d740 1 -- 172.21.15.83:0/4010016476 shutdown_connections 2024-04-24T09:01:05.289 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.284+0000 7fbb9299d740 1 -- 172.21.15.83:0/4010016476 wait complete. 2024-04-24T09:01:05.289 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.284+0000 7fbb9299d740 20 client.18996 trim_cache size 0 max 16384 2024-04-24T09:01:05.302 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.302+0000 7fbb9299d740 1 Processor -- start 2024-04-24T09:01:05.302 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.302+0000 7fbb9299d740 1 -- start start 2024-04-24T09:01:05.304 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.302+0000 7fbb9299d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cc40d013c0 0x55cc40d17cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:05.304 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.302+0000 7fbb9299d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cc40d01ce0 0x55cc40cfc910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:05.304 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.302+0000 7fbb9299d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cc40d0c5a0 0x55cc40d241c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:05.304 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.302+0000 7fbb9299d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbb881b17e0 con 0x55cc40d0c5a0 2024-04-24T09:01:05.305 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.302+0000 7fbb9299d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbb881b3320 con 0x55cc40d01ce0 2024-04-24T09:01:05.305 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.302+0000 7fbb9299d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fbb881b1eb0 con 0x55cc40d013c0 2024-04-24T09:01:05.305 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.303+0000 7fbb8dc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cc40d0c5a0 0x55cc40d241c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:05.305 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.303+0000 7fbb8dc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cc40d0c5a0 0x55cc40d241c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35064/0 (socket says 172.21.15.83:35064) 2024-04-24T09:01:05.305 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.303+0000 7fbb8dc00640 1 -- 172.21.15.83:0/2241357005 learned_addr learned my addr 172.21.15.83:0/2241357005 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:01:05.305 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.303+0000 7fbb8d200640 1 --2- 172.21.15.83:0/2241357005 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cc40d013c0 0x55cc40d17cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:05.305 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.303+0000 7fbb8dc00640 1 -- 172.21.15.83:0/2241357005 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cc40d013c0 msgr2=0x55cc40d17cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:05.305 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.303+0000 7fbb7fe00640 1 --2- 172.21.15.83:0/2241357005 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cc40d01ce0 0x55cc40cfc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:05.306 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.303+0000 7fbb8dc00640 1 --2- 172.21.15.83:0/2241357005 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cc40d013c0 0x55cc40d17cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:05.306 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.303+0000 7fbb8dc00640 1 -- 172.21.15.83:0/2241357005 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cc40d01ce0 msgr2=0x55cc40cfc910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:05.306 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.303+0000 7fbb8dc00640 1 --2- 172.21.15.83:0/2241357005 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cc40d01ce0 0x55cc40cfc910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:05.306 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.303+0000 7fbb8dc00640 1 -- 172.21.15.83:0/2241357005 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc40a46620 con 0x55cc40d0c5a0 2024-04-24T09:01:05.306 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.304+0000 7fbb8dc00640 1 --2- 172.21.15.83:0/2241357005 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cc40d0c5a0 0x55cc40d241c0 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7fbb8001f520 tx=0x7fbb80015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ebb52c73d836bf0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:05.306 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.304+0000 7fbb7d600640 1 -- 172.21.15.83:0/2241357005 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb80028640 con 0x55cc40d0c5a0 2024-04-24T09:01:05.306 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.304+0000 7fbb7d600640 1 -- 172.21.15.83:0/2241357005 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbb8001d8e0 con 0x55cc40d0c5a0 2024-04-24T09:01:05.307 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.304+0000 7fbb7d600640 1 -- 172.21.15.83:0/2241357005 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb800279e0 con 0x55cc40d0c5a0 2024-04-24T09:01:05.312 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.305+0000 7fbb9299d740 1 -- 172.21.15.83:0/2241357005 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cc40d0c5a0 msgr2=0x55cc40d241c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:05.313 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.305+0000 7fbb9299d740 1 --2- 172.21.15.83:0/2241357005 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cc40d0c5a0 0x55cc40d241c0 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7fbb8001f520 tx=0x7fbb80015000 comp rx=0 tx=0).stop 2024-04-24T09:01:05.313 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.305+0000 7fbb9299d740 1 -- 172.21.15.83:0/2241357005 shutdown_connections 2024-04-24T09:01:05.313 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.305+0000 7fbb9299d740 1 --2- 172.21.15.83:0/2241357005 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cc40d0c5a0 0x55cc40d241c0 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:05.313 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.305+0000 7fbb9299d740 1 --2- 172.21.15.83:0/2241357005 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cc40d01ce0 0x55cc40cfc910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:05.313 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.305+0000 7fbb9299d740 1 --2- 172.21.15.83:0/2241357005 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cc40d013c0 0x55cc40d17cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:05.313 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.305+0000 7fbb9299d740 1 -- 172.21.15.83:0/2241357005 >> 172.21.15.83:0/2241357005 conn(0x55cc40cfd8d0 msgr2=0x55cc40cfc200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:05.313 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.305+0000 7fbb9299d740 1 -- 172.21.15.83:0/2241357005 shutdown_connections 2024-04-24T09:01:05.313 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.305+0000 7fbb9299d740 1 -- 172.21.15.83:0/2241357005 wait complete. 2024-04-24T09:01:05.314 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.305+0000 7fbb9299d740 1 -- start start 2024-04-24T09:01:05.314 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.306+0000 7fbb9299d740 1 Processor -- start 2024-04-24T09:01:05.314 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.306+0000 7fbb9299d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cc40d013c0 0x55cc40d143d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:05.314 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.306+0000 7fbb9299d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cc40d01ce0 0x55cc40d111b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:05.314 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.306+0000 7fbb9299d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cc40d0c5a0 0x55cc40d116f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:05.315 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.306+0000 7fbb9299d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbb881b2c50 con 0x55cc40d01ce0 2024-04-24T09:01:05.315 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.306+0000 7fbb9299d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbb881b2580 con 0x55cc40d0c5a0 2024-04-24T09:01:05.315 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.306+0000 7fbb9299d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fbb881af7e0 con 0x55cc40d013c0 2024-04-24T09:01:05.315 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.306+0000 7fbb9299d740 10 client.0 _ll_register_callbacks cb 0x7fbb7800a5d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:01:05.316 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.306+0000 7fbb7fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cc40d01ce0 0x55cc40d111b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:05.316 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.306+0000 7fbb7fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cc40d01ce0 0x55cc40d111b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35072/0 (socket says 172.21.15.83:35072) 2024-04-24T09:01:05.316 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.306+0000 7fbb7fe00640 1 -- 172.21.15.83:0/727235015 learned_addr learned my addr 172.21.15.83:0/727235015 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:01:05.316 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.307+0000 7fbb8d200640 1 --2- 172.21.15.83:0/727235015 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cc40d013c0 0x55cc40d143d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:05.316 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.307+0000 7fbb8dc00640 1 --2- 172.21.15.83:0/727235015 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cc40d0c5a0 0x55cc40d116f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:05.316 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.307+0000 7fbb7fe00640 1 -- 172.21.15.83:0/727235015 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cc40d013c0 msgr2=0x55cc40d143d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:05.316 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.307+0000 7fbb7fe00640 1 --2- 172.21.15.83:0/727235015 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cc40d013c0 0x55cc40d143d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:05.316 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.307+0000 7fbb7fe00640 1 -- 172.21.15.83:0/727235015 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cc40d0c5a0 msgr2=0x55cc40d116f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:05.316 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.307+0000 7fbb7fe00640 1 --2- 172.21.15.83:0/727235015 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cc40d0c5a0 0x55cc40d116f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:05.316 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.307+0000 7fbb7fe00640 1 -- 172.21.15.83:0/727235015 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc40d11c30 con 0x55cc40d01ce0 2024-04-24T09:01:05.316 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.307+0000 7fbb8dc00640 1 --2- 172.21.15.83:0/727235015 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cc40d0c5a0 0x55cc40d116f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:01:05.317 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.307+0000 7fbb7fe00640 1 --2- 172.21.15.83:0/727235015 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cc40d01ce0 0x55cc40d111b0 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7fbb700b9490 tx=0x7fbb70031f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dff6338ab98e7d66 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:05.317 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.307+0000 7fbb76a00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:01:05.317 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.307+0000 7fbb76a00640 1 -- 172.21.15.83:0/727235015 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb70049040 con 0x55cc40d01ce0 2024-04-24T09:01:05.317 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.307+0000 7fbb76a00640 1 -- 172.21.15.83:0/727235015 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9671+0+0 (secure 0 0 0) 0x7fbb700bb330 con 0x55cc40d01ce0 2024-04-24T09:01:05.317 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.307+0000 7fbb9299d740 1 -- 172.21.15.83:0/727235015 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb8001d8e0 con 0x55cc40d01ce0 2024-04-24T09:01:05.317 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.307+0000 7fbb76a00640 1 -- 172.21.15.83:0/727235015 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fbb700bc990 con 0x55cc40d01ce0 2024-04-24T09:01:05.317 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.307+0000 7fbb9299d740 10 client.19002 fetch_fsmap learned FSMap version 35 2024-04-24T09:01:05.318 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.307+0000 7fbb9299d740 1 -- 172.21.15.83:0/727235015 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55cc40d8bf20 con 0x55cc40d01ce0 2024-04-24T09:01:05.318 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.308+0000 7fbb76a00640 1 -- 172.21.15.83:0/727235015 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbb8001d8e0 con 0x55cc40d01ce0 2024-04-24T09:01:05.318 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.308+0000 7fbb76a00640 1 -- 172.21.15.83:0/727235015 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7fbb700b96a0 con 0x55cc40d01ce0 2024-04-24T09:01:05.318 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.308+0000 7fbb9299d740 10 client.19002 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T09:01:05.318 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.308+0000 7fbb9299d740 10 client.19002 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:01:05.318 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.308+0000 7fbb9299d740 20 client.19002 populate_metadata read hostname 'smithi083' 2024-04-24T09:01:05.318 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.308+0000 7fbb9299d740 1 --2- 172.21.15.83:0/727235015 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55cc40d8c500 0x55cc40d8e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:05.318 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.308+0000 7fbb9299d740 4 client.19002 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T09:01:05.318 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.308+0000 7fbb9299d740 1 -- 172.21.15.83:0/727235015 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cc40c6f190 con 0x55cc40d8c500 2024-04-24T09:01:05.318 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.309+0000 7fbb8d200640 1 --2- 172.21.15.83:0/727235015 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55cc40d8c500 0x55cc40d8e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:05.319 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.310+0000 7fbb76a00640 10 client.19002 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:01:05.319 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.310+0000 7fbb8d200640 1 --2- 172.21.15.83:0/727235015 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55cc40d8c500 0x55cc40d8e920 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9abc4c6d158c8a6b server_cookie=5fdb7cdb86581485 in_seq=0 out_seq=0 2024-04-24T09:01:05.367 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.366+0000 7fbb76a00640 1 -- 172.21.15.83:0/727235015 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55cc40c6f190 con 0x55cc40d8c500 2024-04-24T09:01:05.367 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.366+0000 7fbb76a00640 10 client.19002 handle_command_reply: tid=0 2024-04-24T09:01:05.367 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.366+0000 7fbb9299d740 1 client.19002 shutdown 2024-04-24T09:01:05.367 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.367+0000 7fbb9299d740 1 -- 172.21.15.83:0/727235015 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cc40d01ce0 msgr2=0x55cc40d111b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:05.367 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.367+0000 7fbb9299d740 1 --2- 172.21.15.83:0/727235015 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cc40d01ce0 0x55cc40d111b0 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7fbb700b9490 tx=0x7fbb70031f10 comp rx=0 tx=0).stop 2024-04-24T09:01:05.367 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.367+0000 7fbb9299d740 1 -- 172.21.15.83:0/727235015 shutdown_connections 2024-04-24T09:01:05.367 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.367+0000 7fbb9299d740 1 --2- 172.21.15.83:0/727235015 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55cc40d8c500 0x55cc40d8e920 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:05.368 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.367+0000 7fbb9299d740 1 --2- 172.21.15.83:0/727235015 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cc40d0c5a0 0x55cc40d116f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:05.368 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.367+0000 7fbb9299d740 1 --2- 172.21.15.83:0/727235015 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cc40d01ce0 0x55cc40d111b0 unknown :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:05.368 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.367+0000 7fbb9299d740 1 --2- 172.21.15.83:0/727235015 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cc40d013c0 0x55cc40d143d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:05.368 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.367+0000 7fbb9299d740 1 -- 172.21.15.83:0/727235015 >> 172.21.15.83:0/727235015 conn(0x55cc40cfd8d0 msgr2=0x55cc40cfca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:05.368 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.367+0000 7fbb76a00640 0 client.19002 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:01:05.368 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.367+0000 7fbb9299d740 1 -- 172.21.15.83:0/727235015 shutdown_connections 2024-04-24T09:01:05.368 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.367+0000 7fbb9299d740 1 -- 172.21.15.83:0/727235015 wait complete. 2024-04-24T09:01:05.369 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.368+0000 7fbb9299d740 20 client.19002 trim_cache size 0 max 16384 2024-04-24T09:01:05.369 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T09:01:05.369 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T09:01:05.369 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T09:01:05.369 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T09:01:05.370 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.369+0000 7fbb8e600640 1 -- 172.21.15.83:0/2052024936 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fbb6c0a3d10 msgr2=0x7fbb6c0a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:05.370 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.369+0000 7fbb8e600640 1 --2- 172.21.15.83:0/2052024936 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fbb6c0a3d10 0x7fbb6c0a61d0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fbb80005fd0 tx=0x7fbb800096e0 comp rx=0 tx=0).stop 2024-04-24T09:01:05.370 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.369+0000 7fbb8e600640 1 -- 172.21.15.83:0/2052024936 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb88109380 msgr2=0x7fbb881a0b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:05.370 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.369+0000 7fbb8e600640 1 --2- 172.21.15.83:0/2052024936 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb88109380 0x7fbb881a0b10 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7fbb70005ab0 tx=0x7fbb70003cd0 comp rx=0 tx=0).stop 2024-04-24T09:01:05.370 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.369+0000 7fbb8e600640 1 -- 172.21.15.83:0/2052024936 shutdown_connections 2024-04-24T09:01:05.370 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.369+0000 7fbb8e600640 1 --2- 172.21.15.83:0/2052024936 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fbb6c0a3d10 0x7fbb6c0a61d0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:05.371 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.369+0000 7fbb8e600640 1 --2- 172.21.15.83:0/2052024936 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbb8819a690 0x7fbb8819ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:05.371 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.369+0000 7fbb8e600640 1 --2- 172.21.15.83:0/2052024936 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb88109380 0x7fbb881a0b10 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:05.371 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.369+0000 7fbb8e600640 1 --2- 172.21.15.83:0/2052024936 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb881042b0 0x7fbb881a05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:05.371 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.369+0000 7fbb8e600640 1 -- 172.21.15.83:0/2052024936 >> 172.21.15.83:0/2052024936 conn(0x7fbb880ffb10 msgr2=0x7fbb88100f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:05.371 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.369+0000 7fbb8e600640 1 -- 172.21.15.83:0/2052024936 shutdown_connections 2024-04-24T09:01:05.371 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:05.369+0000 7fbb8e600640 1 -- 172.21.15.83:0/2052024936 wait complete. 2024-04-24T09:01:05.604 INFO:tasks.workunit.client.0.smithi083.stdout: 1 54344 0.63 MB/sec execute 339 sec latency 53.480 ms 2024-04-24T09:01:06.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:01:05 smithi138 ceph-mon[31441]: pgmap v296: 129 pgs: 129 active+clean; 57 MiB data, 838 MiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 515 KiB/s wr, 75 op/s 2024-04-24T09:01:06.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:01:05 smithi189 ceph-mon[31791]: pgmap v296: 129 pgs: 129 active+clean; 57 MiB data, 838 MiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 515 KiB/s wr, 75 op/s 2024-04-24T09:01:06.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:01:05 smithi083 ceph-mon[27907]: pgmap v296: 129 pgs: 129 active+clean; 57 MiB data, 838 MiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 515 KiB/s wr, 75 op/s 2024-04-24T09:01:06.604 INFO:tasks.workunit.client.0.smithi083.stdout: 1 54445 0.62 MB/sec execute 340 sec latency 54.238 ms 2024-04-24T09:01:07.604 INFO:tasks.workunit.client.0.smithi083.stdout: 1 54621 0.63 MB/sec execute 341 sec latency 49.057 ms 2024-04-24T09:01:08.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:01:07 smithi138 ceph-mon[31441]: pgmap v297: 129 pgs: 129 active+clean; 57 MiB data, 838 MiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 515 KiB/s wr, 75 op/s 2024-04-24T09:01:08.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:01:07 smithi189 ceph-mon[31791]: pgmap v297: 129 pgs: 129 active+clean; 57 MiB data, 838 MiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 515 KiB/s wr, 75 op/s 2024-04-24T09:01:08.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:01:07 smithi083 ceph-mon[27907]: pgmap v297: 129 pgs: 129 active+clean; 57 MiB data, 838 MiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 515 KiB/s wr, 75 op/s 2024-04-24T09:01:08.604 INFO:tasks.workunit.client.0.smithi083.stdout: 1 54708 0.63 MB/sec execute 342 sec latency 64.718 ms 2024-04-24T09:01:09.604 INFO:tasks.workunit.client.0.smithi083.stdout: 1 54800 0.63 MB/sec execute 343 sec latency 77.009 ms 2024-04-24T09:01:10.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:01:09 smithi138 ceph-mon[31441]: pgmap v298: 129 pgs: 129 active+clean; 53 MiB data, 820 MiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 473 KiB/s wr, 87 op/s 2024-04-24T09:01:10.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:01:09 smithi138 ceph-mon[31441]: osdmap e104: 12 total, 12 up, 12 in 2024-04-24T09:01:10.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:01:09 smithi189 ceph-mon[31791]: pgmap v298: 129 pgs: 129 active+clean; 53 MiB data, 820 MiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 473 KiB/s wr, 87 op/s 2024-04-24T09:01:10.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:01:09 smithi189 ceph-mon[31791]: osdmap e104: 12 total, 12 up, 12 in 2024-04-24T09:01:10.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:01:09 smithi083 ceph-mon[27907]: pgmap v298: 129 pgs: 129 active+clean; 53 MiB data, 820 MiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 473 KiB/s wr, 87 op/s 2024-04-24T09:01:10.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:01:09 smithi083 ceph-mon[27907]: osdmap e104: 12 total, 12 up, 12 in 2024-04-24T09:01:10.605 INFO:tasks.workunit.client.0.smithi083.stdout: 1 54902 0.63 MB/sec execute 344 sec latency 61.179 ms 2024-04-24T09:01:11.605 INFO:tasks.workunit.client.0.smithi083.stdout: 1 55047 0.63 MB/sec execute 345 sec latency 70.847 ms 2024-04-24T09:01:12.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:01:11 smithi138 ceph-mon[31441]: pgmap v300: 129 pgs: 129 active+clean; 53 MiB data, 820 MiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 327 KiB/s wr, 49 op/s 2024-04-24T09:01:12.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:01:11 smithi189 ceph-mon[31791]: pgmap v300: 129 pgs: 129 active+clean; 53 MiB data, 820 MiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 327 KiB/s wr, 49 op/s 2024-04-24T09:01:12.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:01:11 smithi083 ceph-mon[27907]: pgmap v300: 129 pgs: 129 active+clean; 53 MiB data, 820 MiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 327 KiB/s wr, 49 op/s 2024-04-24T09:01:12.605 INFO:tasks.workunit.client.0.smithi083.stdout: 1 55145 0.62 MB/sec execute 346 sec latency 56.708 ms 2024-04-24T09:01:13.605 INFO:tasks.workunit.client.0.smithi083.stdout: 1 55290 0.62 MB/sec execute 347 sec latency 69.412 ms 2024-04-24T09:01:14.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:01:13 smithi138 ceph-mon[31441]: pgmap v301: 129 pgs: 129 active+clean; 54 MiB data, 823 MiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 357 KiB/s wr, 64 op/s 2024-04-24T09:01:14.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:01:13 smithi189 ceph-mon[31791]: pgmap v301: 129 pgs: 129 active+clean; 54 MiB data, 823 MiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 357 KiB/s wr, 64 op/s 2024-04-24T09:01:14.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:01:13 smithi083 ceph-mon[27907]: pgmap v301: 129 pgs: 129 active+clean; 54 MiB data, 823 MiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 357 KiB/s wr, 64 op/s 2024-04-24T09:01:14.606 INFO:tasks.workunit.client.0.smithi083.stdout: 1 55413 0.62 MB/sec execute 348 sec latency 72.900 ms 2024-04-24T09:01:15.605 INFO:tasks.workunit.client.0.smithi083.stdout: 1 55562 0.62 MB/sec execute 349 sec latency 55.165 ms 2024-04-24T09:01:16.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:01:15 smithi083 ceph-mon[27907]: pgmap v302: 129 pgs: 129 active+clean; 54 MiB data, 823 MiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 348 KiB/s wr, 63 op/s 2024-04-24T09:01:16.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:01:15 smithi138 ceph-mon[31441]: pgmap v302: 129 pgs: 129 active+clean; 54 MiB data, 823 MiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 348 KiB/s wr, 63 op/s 2024-04-24T09:01:16.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:01:15 smithi189 ceph-mon[31791]: pgmap v302: 129 pgs: 129 active+clean; 54 MiB data, 823 MiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 348 KiB/s wr, 63 op/s 2024-04-24T09:01:16.605 INFO:tasks.workunit.client.0.smithi083.stdout: 1 55634 0.62 MB/sec execute 350 sec latency 54.810 ms 2024-04-24T09:01:16.624 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T09:01:16.624 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T09:01:16.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.848+0000 7fb03fa00640 1 -- 172.21.15.83:0/4003186621 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb038104e60 msgr2=0x7fb038108ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:16.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.848+0000 7fb03fa00640 1 --2- 172.21.15.83:0/4003186621 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb038104e60 0x7fb038108ea0 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7fb024009900 tx=0x7fb02402a3f0 comp rx=0 tx=0).stop 2024-04-24T09:01:16.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.849+0000 7fb03fa00640 1 -- 172.21.15.83:0/4003186621 shutdown_connections 2024-04-24T09:01:16.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.849+0000 7fb03fa00640 1 --2- 172.21.15.83:0/4003186621 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb038109580 0x7fb038111150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:16.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.849+0000 7fb03fa00640 1 --2- 172.21.15.83:0/4003186621 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb038104e60 0x7fb038108ea0 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:16.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.849+0000 7fb03fa00640 1 --2- 172.21.15.83:0/4003186621 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb0381044b0 0x7fb038104890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:16.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.849+0000 7fb03fa00640 1 -- 172.21.15.83:0/4003186621 >> 172.21.15.83:0/4003186621 conn(0x7fb0380ffd90 msgr2=0x7fb0381021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:16.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.849+0000 7fb03fa00640 1 -- 172.21.15.83:0/4003186621 shutdown_connections 2024-04-24T09:01:16.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.849+0000 7fb03fa00640 1 -- 172.21.15.83:0/4003186621 wait complete. 2024-04-24T09:01:16.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.850+0000 7fb03fa00640 1 Processor -- start 2024-04-24T09:01:16.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.850+0000 7fb03fa00640 1 -- start start 2024-04-24T09:01:16.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.850+0000 7fb03fa00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb0381044b0 0x7fb0381a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:16.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.850+0000 7fb03fa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb038109580 0x7fb0381a0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:16.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.850+0000 7fb03fa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb03819a870 0x7fb03819ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:16.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.850+0000 7fb03fa00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb0381144f0 con 0x7fb038109580 2024-04-24T09:01:16.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.850+0000 7fb03fa00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb038114370 con 0x7fb03819a870 2024-04-24T09:01:16.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.850+0000 7fb03fa00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb038114670 con 0x7fb0381044b0 2024-04-24T09:01:16.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.851+0000 7fb037e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb038109580 0x7fb0381a0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:16.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.851+0000 7fb037e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb038109580 0x7fb0381a0c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48790/0 (socket says 172.21.15.83:48790) 2024-04-24T09:01:16.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.851+0000 7fb037e00640 1 -- 172.21.15.83:0/3318141592 learned_addr learned my addr 172.21.15.83:0/3318141592 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:01:16.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.851+0000 7fb03d000640 1 --2- 172.21.15.83:0/3318141592 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb0381044b0 0x7fb0381a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:16.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.851+0000 7fb03da00640 1 --2- 172.21.15.83:0/3318141592 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb03819a870 0x7fb03819ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:16.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.851+0000 7fb037e00640 1 -- 172.21.15.83:0/3318141592 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb0381044b0 msgr2=0x7fb0381a06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:16.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.851+0000 7fb037e00640 1 --2- 172.21.15.83:0/3318141592 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb0381044b0 0x7fb0381a06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:16.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.851+0000 7fb037e00640 1 -- 172.21.15.83:0/3318141592 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb03819a870 msgr2=0x7fb03819ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:16.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.851+0000 7fb037e00640 1 --2- 172.21.15.83:0/3318141592 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb03819a870 0x7fb03819ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:16.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.851+0000 7fb037e00640 1 -- 172.21.15.83:0/3318141592 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb03819b390 con 0x7fb038109580 2024-04-24T09:01:16.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.851+0000 7fb037e00640 1 --2- 172.21.15.83:0/3318141592 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb038109580 0x7fb0381a0c00 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7fb024031440 tx=0x7fb024003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:16.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.852+0000 7fb035600640 1 -- 172.21.15.83:0/3318141592 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb02403f070 con 0x7fb038109580 2024-04-24T09:01:16.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.852+0000 7fb035600640 1 -- 172.21.15.83:0/3318141592 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb024031d60 con 0x7fb038109580 2024-04-24T09:01:16.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.852+0000 7fb03fa00640 1 -- 172.21.15.83:0/3318141592 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb03819b620 con 0x7fb038109580 2024-04-24T09:01:16.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.852+0000 7fb035600640 1 -- 172.21.15.83:0/3318141592 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb024035030 con 0x7fb038109580 2024-04-24T09:01:16.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.852+0000 7fb03fa00640 1 -- 172.21.15.83:0/3318141592 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0381a7810 con 0x7fb038109580 2024-04-24T09:01:16.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.856+0000 7fb0427e5740 1 Processor -- start 2024-04-24T09:01:16.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.856+0000 7fb035600640 1 -- 172.21.15.83:0/3318141592 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb024042070 con 0x7fb038109580 2024-04-24T09:01:16.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.856+0000 7fb0427e5740 1 -- start start 2024-04-24T09:01:16.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.856+0000 7fb0427e5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556642cde860 0x556642cdec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:16.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.856+0000 7fb0427e5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556642cdf180 0x556642ce9310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:16.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.856+0000 7fb0427e5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556642ce9850 0x556642cebc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:16.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.856+0000 7fb0427e5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556642c441f0 con 0x556642ce9850 2024-04-24T09:01:16.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.856+0000 7fb0427e5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556642bea350 con 0x556642cdf180 2024-04-24T09:01:16.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.856+0000 7fb0427e5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x556642c401b0 con 0x556642cde860 2024-04-24T09:01:16.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.857+0000 7fb037e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556642ce9850 0x556642cebc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:16.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.857+0000 7fb037e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556642ce9850 0x556642cebc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48802/0 (socket says 172.21.15.83:48802) 2024-04-24T09:01:16.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.857+0000 7fb037e00640 1 -- 172.21.15.83:0/4159760523 learned_addr learned my addr 172.21.15.83:0/4159760523 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:01:16.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.857+0000 7fb03da00640 1 --2- 172.21.15.83:0/4159760523 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556642cde860 0x556642cdec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:16.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.857+0000 7fb03d000640 1 --2- 172.21.15.83:0/4159760523 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556642cdf180 0x556642ce9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:16.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.857+0000 7fb037e00640 1 -- 172.21.15.83:0/4159760523 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556642cde860 msgr2=0x556642cdec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:16.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.857+0000 7fb037e00640 1 --2- 172.21.15.83:0/4159760523 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556642cde860 0x556642cdec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:16.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.857+0000 7fb037e00640 1 -- 172.21.15.83:0/4159760523 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556642cdf180 msgr2=0x556642ce9310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:16.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.857+0000 7fb037e00640 1 --2- 172.21.15.83:0/4159760523 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556642cdf180 0x556642ce9310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:16.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.857+0000 7fb037e00640 1 -- 172.21.15.83:0/4159760523 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556642c3ef50 con 0x556642ce9850 2024-04-24T09:01:16.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.858+0000 7fb037e00640 1 --2- 172.21.15.83:0/4159760523 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556642ce9850 0x556642cebc40 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7fb0240ee4a0 tx=0x7fb0240f9540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2aad7d571d7169d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:16.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.858+0000 7fb035600640 1 --2- 172.21.15.83:0/3318141592 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb0200a3d20 0x7fb0200a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:16.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.858+0000 7fb02a000640 1 -- 172.21.15.83:0/4159760523 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0240b9050 con 0x556642ce9850 2024-04-24T09:01:16.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.858+0000 7fb02a000640 1 -- 172.21.15.83:0/4159760523 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb0240fe030 con 0x556642ce9850 2024-04-24T09:01:16.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.858+0000 7fb02a000640 1 -- 172.21.15.83:0/4159760523 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0240bc200 con 0x556642ce9850 2024-04-24T09:01:16.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.858+0000 7fb035600640 1 -- 172.21.15.83:0/3318141592 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7fb0240f1290 con 0x7fb038109580 2024-04-24T09:01:16.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.858+0000 7fb03da00640 1 --2- 172.21.15.83:0/3318141592 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb0200a3d20 0x7fb0200a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:16.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.859+0000 7fb0427e5740 1 -- 172.21.15.83:0/4159760523 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556642ce9850 msgr2=0x556642cebc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:16.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.859+0000 7fb03da00640 1 --2- 172.21.15.83:0/3318141592 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb0200a3d20 0x7fb0200a61e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb030007db0 tx=0x7fb030007380 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:16.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.859+0000 7fb0427e5740 1 --2- 172.21.15.83:0/4159760523 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556642ce9850 0x556642cebc40 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7fb0240ee4a0 tx=0x7fb0240f9540 comp rx=0 tx=0).stop 2024-04-24T09:01:16.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.859+0000 7fb0427e5740 1 -- 172.21.15.83:0/4159760523 shutdown_connections 2024-04-24T09:01:16.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.859+0000 7fb0427e5740 1 --2- 172.21.15.83:0/4159760523 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556642ce9850 0x556642cebc40 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:16.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.859+0000 7fb0427e5740 1 --2- 172.21.15.83:0/4159760523 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556642cdf180 0x556642ce9310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:16.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.859+0000 7fb0427e5740 1 --2- 172.21.15.83:0/4159760523 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556642cde860 0x556642cdec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:16.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.859+0000 7fb0427e5740 1 -- 172.21.15.83:0/4159760523 >> 172.21.15.83:0/4159760523 conn(0x556642cdab80 msgr2=0x556642cdaf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:16.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.859+0000 7fb0427e5740 1 -- 172.21.15.83:0/4159760523 shutdown_connections 2024-04-24T09:01:16.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.859+0000 7fb0427e5740 1 -- 172.21.15.83:0/4159760523 wait complete. 2024-04-24T09:01:16.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.860+0000 7fb0427e5740 1 -- start start 2024-04-24T09:01:16.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.860+0000 7fb0427e5740 1 Processor -- start 2024-04-24T09:01:16.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.861+0000 7fb0427e5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556642cde860 0x556642ce6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:16.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.861+0000 7fb0427e5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556642cdf180 0x556642ce6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:16.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.861+0000 7fb0427e5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556642ce9850 0x556642ce71e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:16.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.861+0000 7fb0427e5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556642cec400 con 0x556642ce9850 2024-04-24T09:01:16.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.861+0000 7fb0427e5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556642bfd690 con 0x556642cde860 2024-04-24T09:01:16.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.861+0000 7fb0427e5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x556642cec580 con 0x556642cdf180 2024-04-24T09:01:16.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.861+0000 7fb0427e5740 10 client.0 _ll_register_callbacks cb 0x556642c39650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:01:16.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.861+0000 7fb03da00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556642ce9850 0x556642ce71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:16.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.861+0000 7fb03da00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556642ce9850 0x556642ce71e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48814/0 (socket says 172.21.15.83:48814) 2024-04-24T09:01:16.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.861+0000 7fb03da00640 1 -- 172.21.15.83:0/2032287626 learned_addr learned my addr 172.21.15.83:0/2032287626 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:01:16.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.861+0000 7fb03d000640 1 --2- 172.21.15.83:0/2032287626 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556642cde860 0x556642ce6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:16.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.861+0000 7fb037e00640 1 --2- 172.21.15.83:0/2032287626 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556642cdf180 0x556642ce6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:16.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.861+0000 7fb03da00640 1 -- 172.21.15.83:0/2032287626 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556642cdf180 msgr2=0x556642ce6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:16.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.861+0000 7fb03da00640 1 --2- 172.21.15.83:0/2032287626 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556642cdf180 0x556642ce6ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:16.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.861+0000 7fb03da00640 1 -- 172.21.15.83:0/2032287626 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556642cde860 msgr2=0x556642ce6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:16.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.861+0000 7fb03da00640 1 --2- 172.21.15.83:0/2032287626 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556642cde860 0x556642ce6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:16.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.861+0000 7fb03da00640 1 -- 172.21.15.83:0/2032287626 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556642bfc920 con 0x556642ce9850 2024-04-24T09:01:16.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.862+0000 7fb03da00640 1 --2- 172.21.15.83:0/2032287626 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556642ce9850 0x556642ce71e0 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7fb030011e70 tx=0x7fb030004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b04c448ffd2443a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:16.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.862+0000 7fb01ea00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:01:16.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.862+0000 7fb01ea00640 1 -- 172.21.15.83:0/2032287626 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb03001e420 con 0x556642ce9850 2024-04-24T09:01:16.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.862+0000 7fb01ea00640 1 -- 172.21.15.83:0/2032287626 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7fb03001e5c0 con 0x556642ce9850 2024-04-24T09:01:16.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.862+0000 7fb0427e5740 1 -- 172.21.15.83:0/2032287626 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0240fe030 con 0x556642ce9850 2024-04-24T09:01:16.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.862+0000 7fb01ea00640 1 -- 172.21.15.83:0/2032287626 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fb03001e7c0 con 0x556642ce9850 2024-04-24T09:01:16.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.862+0000 7fb0427e5740 10 client.19014 fetch_fsmap learned FSMap version 35 2024-04-24T09:01:16.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.862+0000 7fb0427e5740 1 -- 172.21.15.83:0/2032287626 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x556642a207d0 con 0x556642ce9850 2024-04-24T09:01:16.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.863+0000 7fb01ea00640 1 -- 172.21.15.83:0/2032287626 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb0240fe030 con 0x556642ce9850 2024-04-24T09:01:16.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.863+0000 7fb01ea00640 1 -- 172.21.15.83:0/2032287626 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7fb03001d3b0 con 0x556642ce9850 2024-04-24T09:01:16.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.863+0000 7fb0427e5740 10 client.19014 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T09:01:16.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.863+0000 7fb0427e5740 10 client.19014 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:01:16.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.863+0000 7fb0427e5740 20 client.19014 populate_metadata read hostname 'smithi083' 2024-04-24T09:01:16.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.863+0000 7fb0427e5740 1 --2- 172.21.15.83:0/2032287626 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556642cf98e0 0x556642cfbd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:16.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.863+0000 7fb0427e5740 4 client.19014 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:01:16.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.863+0000 7fb0427e5740 1 -- 172.21.15.83:0/2032287626 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556642c51c10 con 0x556642cf98e0 2024-04-24T09:01:16.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.864+0000 7fb03d000640 1 --2- 172.21.15.83:0/2032287626 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556642cf98e0 0x556642cfbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:16.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.865+0000 7fb03d000640 1 --2- 172.21.15.83:0/2032287626 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556642cf98e0 0x556642cfbd50 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=258d140a8c09d45a server_cookie=4122b12794b30ad in_seq=0 out_seq=0 2024-04-24T09:01:16.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.865+0000 7fb01ea00640 10 client.19014 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:01:16.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.869+0000 7fb01ea00640 1 -- 172.21.15.83:0/2032287626 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556642c51c10 con 0x556642cf98e0 2024-04-24T09:01:16.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.869+0000 7fb01ea00640 10 client.19014 handle_command_reply: tid=0 2024-04-24T09:01:16.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.869+0000 7fb0427e5740 1 client.19014 shutdown 2024-04-24T09:01:16.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.869+0000 7fb0427e5740 1 -- 172.21.15.83:0/2032287626 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556642ce9850 msgr2=0x556642ce71e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:16.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.869+0000 7fb0427e5740 1 --2- 172.21.15.83:0/2032287626 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556642ce9850 0x556642ce71e0 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7fb030011e70 tx=0x7fb030004a40 comp rx=0 tx=0).stop 2024-04-24T09:01:16.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.869+0000 7fb0427e5740 1 -- 172.21.15.83:0/2032287626 shutdown_connections 2024-04-24T09:01:16.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.869+0000 7fb0427e5740 1 --2- 172.21.15.83:0/2032287626 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556642cf98e0 0x556642cfbd50 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:16.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.869+0000 7fb0427e5740 1 --2- 172.21.15.83:0/2032287626 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556642ce9850 0x556642ce71e0 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:16.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.869+0000 7fb0427e5740 1 --2- 172.21.15.83:0/2032287626 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556642cdf180 0x556642ce6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:16.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.869+0000 7fb0427e5740 1 --2- 172.21.15.83:0/2032287626 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556642cde860 0x556642ce6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:16.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.869+0000 7fb0427e5740 1 -- 172.21.15.83:0/2032287626 >> 172.21.15.83:0/2032287626 conn(0x556642cdab80 msgr2=0x556642ce0980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:16.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.869+0000 7fb01ea00640 0 client.19014 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:01:16.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.869+0000 7fb0427e5740 1 -- 172.21.15.83:0/2032287626 shutdown_connections 2024-04-24T09:01:16.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.869+0000 7fb0427e5740 1 -- 172.21.15.83:0/2032287626 wait complete. 2024-04-24T09:01:16.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.869+0000 7fb0427e5740 20 client.19014 trim_cache size 0 max 16384 2024-04-24T09:01:16.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.890+0000 7fb0427e5740 1 Processor -- start 2024-04-24T09:01:16.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.890+0000 7fb0427e5740 1 -- start start 2024-04-24T09:01:16.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.890+0000 7fb0427e5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556642cde860 0x556642cf5230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:16.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.890+0000 7fb0427e5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556642cdf180 0x556642ce1000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:16.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.890+0000 7fb0427e5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556642ce9850 0x556642cd9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:16.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.890+0000 7fb0427e5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb0381b1810 con 0x556642ce9850 2024-04-24T09:01:16.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.890+0000 7fb0427e5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb0381b3350 con 0x556642cde860 2024-04-24T09:01:16.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.890+0000 7fb0427e5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb0381b1ee0 con 0x556642cdf180 2024-04-24T09:01:16.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.891+0000 7fb03da00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556642ce9850 0x556642cd9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:16.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.891+0000 7fb03da00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556642ce9850 0x556642cd9aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48820/0 (socket says 172.21.15.83:48820) 2024-04-24T09:01:16.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.891+0000 7fb03da00640 1 -- 172.21.15.83:0/4086827291 learned_addr learned my addr 172.21.15.83:0/4086827291 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:01:16.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.891+0000 7fb03d000640 1 --2- 172.21.15.83:0/4086827291 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556642cde860 0x556642cf5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:16.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.891+0000 7fb037e00640 1 --2- 172.21.15.83:0/4086827291 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556642cdf180 0x556642ce1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:16.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.891+0000 7fb03da00640 1 -- 172.21.15.83:0/4086827291 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556642cdf180 msgr2=0x556642ce1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:16.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.891+0000 7fb03da00640 1 --2- 172.21.15.83:0/4086827291 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556642cdf180 0x556642ce1000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:16.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.891+0000 7fb03da00640 1 -- 172.21.15.83:0/4086827291 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556642cde860 msgr2=0x556642cf5230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:16.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.891+0000 7fb03da00640 1 --2- 172.21.15.83:0/4086827291 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556642cde860 0x556642cf5230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:16.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.891+0000 7fb03da00640 1 -- 172.21.15.83:0/4086827291 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566429563f0 con 0x556642ce9850 2024-04-24T09:01:16.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.891+0000 7fb037e00640 1 --2- 172.21.15.83:0/4086827291 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556642cdf180 0x556642ce1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:01:16.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.892+0000 7fb03da00640 1 --2- 172.21.15.83:0/4086827291 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556642ce9850 0x556642cd9aa0 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7fb030000f80 tx=0x7fb030010cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a00dbb844aef7185 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:16.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.892+0000 7fb029600640 1 -- 172.21.15.83:0/4086827291 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb03001e620 con 0x556642ce9850 2024-04-24T09:01:16.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.892+0000 7fb029600640 1 -- 172.21.15.83:0/4086827291 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb03001d3b0 con 0x556642ce9850 2024-04-24T09:01:16.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.892+0000 7fb029600640 1 -- 172.21.15.83:0/4086827291 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb030003db0 con 0x556642ce9850 2024-04-24T09:01:16.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.893+0000 7fb0427e5740 1 -- 172.21.15.83:0/4086827291 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556642ce9850 msgr2=0x556642cd9aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:16.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.893+0000 7fb0427e5740 1 --2- 172.21.15.83:0/4086827291 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556642ce9850 0x556642cd9aa0 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7fb030000f80 tx=0x7fb030010cb0 comp rx=0 tx=0).stop 2024-04-24T09:01:16.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.893+0000 7fb0427e5740 1 -- 172.21.15.83:0/4086827291 shutdown_connections 2024-04-24T09:01:16.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.893+0000 7fb0427e5740 1 --2- 172.21.15.83:0/4086827291 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556642ce9850 0x556642cd9aa0 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:16.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.893+0000 7fb0427e5740 1 --2- 172.21.15.83:0/4086827291 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556642cdf180 0x556642ce1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:16.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.893+0000 7fb0427e5740 1 --2- 172.21.15.83:0/4086827291 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556642cde860 0x556642cf5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:16.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.893+0000 7fb0427e5740 1 -- 172.21.15.83:0/4086827291 >> 172.21.15.83:0/4086827291 conn(0x556642cdab80 msgr2=0x556642cd9390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:16.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.893+0000 7fb0427e5740 1 -- 172.21.15.83:0/4086827291 shutdown_connections 2024-04-24T09:01:16.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.893+0000 7fb0427e5740 1 -- 172.21.15.83:0/4086827291 wait complete. 2024-04-24T09:01:16.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.893+0000 7fb0427e5740 1 -- start start 2024-04-24T09:01:16.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.894+0000 7fb0427e5740 1 Processor -- start 2024-04-24T09:01:16.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.894+0000 7fb0427e5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556642cde860 0x556642cf1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:16.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.894+0000 7fb0427e5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556642cdf180 0x556642ce3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:16.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.894+0000 7fb0427e5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556642ce9850 0x556642ce3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:16.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.894+0000 7fb0427e5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb0381b2c80 con 0x556642cdf180 2024-04-24T09:01:16.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.894+0000 7fb0427e5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb0381b25b0 con 0x556642cde860 2024-04-24T09:01:16.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.894+0000 7fb0427e5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb0381af810 con 0x556642ce9850 2024-04-24T09:01:16.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.894+0000 7fb0427e5740 10 client.0 _ll_register_callbacks cb 0x7fb030007e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:01:16.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.894+0000 7fb037e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556642cdf180 0x556642ce3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:16.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.895+0000 7fb03da00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556642ce9850 0x556642ce3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:16.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.895+0000 7fb03d000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556642cde860 0x556642cf1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:16.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.895+0000 7fb037e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556642cdf180 0x556642ce3260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48824/0 (socket says 172.21.15.83:48824) 2024-04-24T09:01:16.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.895+0000 7fb037e00640 1 -- 172.21.15.83:0/3564503476 learned_addr learned my addr 172.21.15.83:0/3564503476 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:01:16.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.895+0000 7fb037e00640 1 -- 172.21.15.83:0/3564503476 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556642ce9850 msgr2=0x556642ce3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:16.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.895+0000 7fb037e00640 1 --2- 172.21.15.83:0/3564503476 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556642ce9850 0x556642ce3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:16.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.895+0000 7fb037e00640 1 -- 172.21.15.83:0/3564503476 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556642cde860 msgr2=0x556642cf1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:16.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.895+0000 7fb037e00640 1 --2- 172.21.15.83:0/3564503476 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556642cde860 0x556642cf1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:16.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.895+0000 7fb037e00640 1 -- 172.21.15.83:0/3564503476 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566429160f0 con 0x556642cdf180 2024-04-24T09:01:16.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.895+0000 7fb03d000640 1 --2- 172.21.15.83:0/3564503476 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556642cde860 0x556642cf1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:01:16.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.896+0000 7fb037e00640 1 --2- 172.21.15.83:0/3564503476 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556642cdf180 0x556642ce3260 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7fb024047040 tx=0x7fb0240f9fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ffb80c9319b1bad server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:16.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.896+0000 7fb01ea00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:01:16.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.896+0000 7fb01ea00640 1 -- 172.21.15.83:0/3564503476 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb024049070 con 0x556642cdf180 2024-04-24T09:01:16.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.896+0000 7fb01ea00640 1 -- 172.21.15.83:0/3564503476 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7fb0240bae50 con 0x556642cdf180 2024-04-24T09:01:16.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.896+0000 7fb0427e5740 1 -- 172.21.15.83:0/3564503476 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb03001d3b0 con 0x556642cdf180 2024-04-24T09:01:16.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.896+0000 7fb01ea00640 1 -- 172.21.15.83:0/3564503476 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fb0240bc530 con 0x556642cdf180 2024-04-24T09:01:16.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.896+0000 7fb0427e5740 10 client.19020 fetch_fsmap learned FSMap version 35 2024-04-24T09:01:16.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.896+0000 7fb0427e5740 1 -- 172.21.15.83:0/3564503476 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x556642c09810 con 0x556642cdf180 2024-04-24T09:01:16.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.897+0000 7fb01ea00640 1 -- 172.21.15.83:0/3564503476 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb03001d3b0 con 0x556642cdf180 2024-04-24T09:01:16.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.897+0000 7fb01ea00640 1 -- 172.21.15.83:0/3564503476 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7fb0240a7070 con 0x556642cdf180 2024-04-24T09:01:16.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.897+0000 7fb0427e5740 10 client.19020 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T09:01:16.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.897+0000 7fb0427e5740 10 client.19020 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:01:16.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.897+0000 7fb0427e5740 20 client.19020 populate_metadata read hostname 'smithi083' 2024-04-24T09:01:16.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.897+0000 7fb0427e5740 1 --2- 172.21.15.83:0/3564503476 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556642d60170 0x556642d62590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:16.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.897+0000 7fb0427e5740 4 client.19020 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T09:01:16.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.897+0000 7fb0427e5740 1 -- 172.21.15.83:0/3564503476 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556642c4ba80 con 0x556642d60170 2024-04-24T09:01:16.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.898+0000 7fb03d000640 1 --2- 172.21.15.83:0/3564503476 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556642d60170 0x556642d62590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:16.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.899+0000 7fb03d000640 1 --2- 172.21.15.83:0/3564503476 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556642d60170 0x556642d62590 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbc75324c72cc580 server_cookie=95ffbedd3a9ae9ef in_seq=0 out_seq=0 2024-04-24T09:01:16.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.899+0000 7fb01ea00640 10 client.19020 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:01:16.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.899+0000 7fb01ea00640 1 -- 172.21.15.83:0/3564503476 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x556642c4ba80 con 0x556642d60170 2024-04-24T09:01:16.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.899+0000 7fb01ea00640 10 client.19020 handle_command_reply: tid=0 2024-04-24T09:01:16.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.899+0000 7fb0427e5740 1 client.19020 shutdown 2024-04-24T09:01:16.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.900+0000 7fb0427e5740 1 -- 172.21.15.83:0/3564503476 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556642cdf180 msgr2=0x556642ce3260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:16.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.900+0000 7fb0427e5740 1 --2- 172.21.15.83:0/3564503476 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556642cdf180 0x556642ce3260 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7fb024047040 tx=0x7fb0240f9fd0 comp rx=0 tx=0).stop 2024-04-24T09:01:16.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.900+0000 7fb0427e5740 1 -- 172.21.15.83:0/3564503476 shutdown_connections 2024-04-24T09:01:16.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.900+0000 7fb0427e5740 1 --2- 172.21.15.83:0/3564503476 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556642d60170 0x556642d62590 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:16.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.900+0000 7fb0427e5740 1 --2- 172.21.15.83:0/3564503476 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556642ce9850 0x556642ce3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:16.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.900+0000 7fb0427e5740 1 --2- 172.21.15.83:0/3564503476 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556642cdf180 0x556642ce3260 unknown :-1 s=CLOSED pgs=1622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:16.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.900+0000 7fb0427e5740 1 --2- 172.21.15.83:0/3564503476 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556642cde860 0x556642cf1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:16.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.900+0000 7fb01ea00640 0 client.19020 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:01:16.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.900+0000 7fb0427e5740 1 -- 172.21.15.83:0/3564503476 >> 172.21.15.83:0/3564503476 conn(0x556642cdab80 msgr2=0x556642cd9c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:16.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.900+0000 7fb0427e5740 1 -- 172.21.15.83:0/3564503476 shutdown_connections 2024-04-24T09:01:16.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.900+0000 7fb0427e5740 1 -- 172.21.15.83:0/3564503476 wait complete. 2024-04-24T09:01:16.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.900+0000 7fb0427e5740 20 client.19020 trim_cache size 0 max 16384 2024-04-24T09:01:16.902 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T09:01:16.902 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-04-24T09:01:16.902 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-04-24T09:01:16.902 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T09:01:16.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.901+0000 7fb03fa00640 1 -- 172.21.15.83:0/3318141592 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb0200a3d20 msgr2=0x7fb0200a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:16.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.901+0000 7fb03fa00640 1 --2- 172.21.15.83:0/3318141592 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb0200a3d20 0x7fb0200a61e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb030007db0 tx=0x7fb030007380 comp rx=0 tx=0).stop 2024-04-24T09:01:16.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.901+0000 7fb03fa00640 1 -- 172.21.15.83:0/3318141592 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb038109580 msgr2=0x7fb0381a0c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:16.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.901+0000 7fb03fa00640 1 --2- 172.21.15.83:0/3318141592 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb038109580 0x7fb0381a0c00 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7fb024031440 tx=0x7fb024003d50 comp rx=0 tx=0).stop 2024-04-24T09:01:16.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.901+0000 7fb03fa00640 1 -- 172.21.15.83:0/3318141592 shutdown_connections 2024-04-24T09:01:16.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.901+0000 7fb03fa00640 1 --2- 172.21.15.83:0/3318141592 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb0200a3d20 0x7fb0200a61e0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:16.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.901+0000 7fb03fa00640 1 --2- 172.21.15.83:0/3318141592 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb03819a870 0x7fb03819ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:16.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.901+0000 7fb03fa00640 1 --2- 172.21.15.83:0/3318141592 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb038109580 0x7fb0381a0c00 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:16.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.901+0000 7fb03fa00640 1 --2- 172.21.15.83:0/3318141592 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb0381044b0 0x7fb0381a06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:16.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.901+0000 7fb03fa00640 1 -- 172.21.15.83:0/3318141592 >> 172.21.15.83:0/3318141592 conn(0x7fb0380ffd90 msgr2=0x7fb0381010a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:16.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.901+0000 7fb03fa00640 1 -- 172.21.15.83:0/3318141592 shutdown_connections 2024-04-24T09:01:16.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:16.901+0000 7fb03fa00640 1 -- 172.21.15.83:0/3318141592 wait complete. 2024-04-24T09:01:16.920 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T09:01:16.920 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T09:01:17.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.127+0000 7f9477a00640 1 -- 172.21.15.83:0/2285421845 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9470103290 msgr2=0x7f9470103670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:17.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.127+0000 7f9477a00640 1 --2- 172.21.15.83:0/2285421845 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9470103290 0x7f9470103670 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f9464009900 tx=0x7f946402a510 comp rx=0 tx=0).stop 2024-04-24T09:01:17.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.127+0000 7f9477a00640 1 -- 172.21.15.83:0/2285421845 shutdown_connections 2024-04-24T09:01:17.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.127+0000 7f9477a00640 1 --2- 172.21.15.83:0/2285421845 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f947010c5d0 0x7f947010ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:17.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.127+0000 7f9477a00640 1 --2- 172.21.15.83:0/2285421845 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9470103bb0 0x7f947010c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:17.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.127+0000 7f9477a00640 1 --2- 172.21.15.83:0/2285421845 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9470103290 0x7f9470103670 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:17.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.127+0000 7f9477a00640 1 -- 172.21.15.83:0/2285421845 >> 172.21.15.83:0/2285421845 conn(0x7f9470078000 msgr2=0x7f9470078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:17.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.127+0000 7f9477a00640 1 -- 172.21.15.83:0/2285421845 shutdown_connections 2024-04-24T09:01:17.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.127+0000 7f9477a00640 1 -- 172.21.15.83:0/2285421845 wait complete. 2024-04-24T09:01:17.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.128+0000 7f9477a00640 1 Processor -- start 2024-04-24T09:01:17.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.128+0000 7f9477a00640 1 -- start start 2024-04-24T09:01:17.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.128+0000 7f9477a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9470103290 0x7f947010bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:17.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.128+0000 7f9477a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9470103bb0 0x7f947010c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:17.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.128+0000 7f9477a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f947010c5d0 0x7f9470107e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:17.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.129+0000 7f9477a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f947006c030 con 0x7f947010c5d0 2024-04-24T09:01:17.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.129+0000 7f9477a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f947006beb0 con 0x7f9470103290 2024-04-24T09:01:17.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.129+0000 7f9477a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f947006c1b0 con 0x7f9470103bb0 2024-04-24T09:01:17.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.129+0000 7f9475a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f947010c5d0 0x7f9470107e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:17.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.129+0000 7f9475a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f947010c5d0 0x7f9470107e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48840/0 (socket says 172.21.15.83:48840) 2024-04-24T09:01:17.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.129+0000 7f9475a00640 1 -- 172.21.15.83:0/67821077 learned_addr learned my addr 172.21.15.83:0/67821077 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:01:17.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.129+0000 7f946fe00640 1 --2- 172.21.15.83:0/67821077 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9470103bb0 0x7f947010c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:17.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.129+0000 7f9475000640 1 --2- 172.21.15.83:0/67821077 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9470103290 0x7f947010bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:17.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.129+0000 7f9475a00640 1 -- 172.21.15.83:0/67821077 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9470103bb0 msgr2=0x7f947010c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:17.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.129+0000 7f9475a00640 1 --2- 172.21.15.83:0/67821077 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9470103bb0 0x7f947010c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:17.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.129+0000 7f9475a00640 1 -- 172.21.15.83:0/67821077 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9470103290 msgr2=0x7f947010bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:17.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.129+0000 7f9475a00640 1 --2- 172.21.15.83:0/67821077 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9470103290 0x7f947010bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:17.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.129+0000 7f9475a00640 1 -- 172.21.15.83:0/67821077 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94701086d0 con 0x7f947010c5d0 2024-04-24T09:01:17.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.129+0000 7f946fe00640 1 --2- 172.21.15.83:0/67821077 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9470103bb0 0x7f947010c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:01:17.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.130+0000 7f9475a00640 1 --2- 172.21.15.83:0/67821077 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f947010c5d0 0x7f9470107e10 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f946800eaa0 tx=0x7f946800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:17.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.130+0000 7f946d600640 1 -- 172.21.15.83:0/67821077 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f946800cdb0 con 0x7f947010c5d0 2024-04-24T09:01:17.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.130+0000 7f946d600640 1 -- 172.21.15.83:0/67821077 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9468005020 con 0x7f947010c5d0 2024-04-24T09:01:17.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.130+0000 7f946d600640 1 -- 172.21.15.83:0/67821077 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94680116d0 con 0x7f947010c5d0 2024-04-24T09:01:17.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.130+0000 7f9477a00640 1 -- 172.21.15.83:0/67821077 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94701089c0 con 0x7f947010c5d0 2024-04-24T09:01:17.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.130+0000 7f9477a00640 1 -- 172.21.15.83:0/67821077 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94701afc50 con 0x7f947010c5d0 2024-04-24T09:01:17.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.132+0000 7f9477a00640 1 -- 172.21.15.83:0/67821077 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9470069990 con 0x7f947010c5d0 2024-04-24T09:01:17.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.134+0000 7f946d600640 1 -- 172.21.15.83:0/67821077 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f94680051d0 con 0x7f947010c5d0 2024-04-24T09:01:17.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.135+0000 7f946d600640 1 --2- 172.21.15.83:0/67821077 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f94580a3da0 0x7f94580a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:17.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.135+0000 7f946d600640 1 -- 172.21.15.83:0/67821077 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7f94680d0e10 con 0x7f947010c5d0 2024-04-24T09:01:17.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.136+0000 7f9475000640 1 --2- 172.21.15.83:0/67821077 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f94580a3da0 0x7f94580a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:17.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.136+0000 7f9475000640 1 --2- 172.21.15.83:0/67821077 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f94580a3da0 0x7f94580a6260 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9464000c00 tx=0x7f9464031400 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:17.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.140+0000 7f946d600640 1 -- 172.21.15.83:0/67821077 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f94680d4030 con 0x7f947010c5d0 2024-04-24T09:01:17.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.423+0000 7f9477a00640 1 -- 172.21.15.83:0/67821077 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f94701b0060 con 0x7f947010c5d0 2024-04-24T09:01:17.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.424+0000 7f946d600640 1 -- 172.21.15.83:0/67821077 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11944 (secure 0 0 0) 0x7f9468016030 con 0x7f947010c5d0 2024-04-24T09:01:17.425 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T09:01:17.425 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/358872257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":358872257},{"type":"v1","addr":"172.21.15.83:6835","nonce":358872257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3157070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3157070851},{"type":"v1","addr":"172.21.15.83:6837","nonce":3157070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":35,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T08:48:29.422576+0000","modified":"2024-04-24T08:59:06.888262+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34520,"mds_1":34493,"mds_2":34451,"mds_3":34427,"mds_4":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14658":{"gid":14658,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6839/4085470844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":4085470844},{"type":"v1","addr":"172.21.15.83:6839","nonce":4085470844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14676":{"gid":14676,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6841/2634754210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":2634754210},{"type":"v1","addr":"172.21.15.83:6841","nonce":2634754210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24382":{"gid":24382,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/1859921310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1859921310},{"type":"v1","addr":"172.21.15.189:6833","nonce":1859921310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/2241923197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2241923197},{"type":"v1","addr":"172.21.15.189:6835","nonce":2241923197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.189:6839/2548965336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":2548965336},{"type":"v1","addr":"172.21.15.189:6839","nonce":2548965336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26686":{"gid":26686,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6837/1678817161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":1678817161},{"type":"v1","addr":"172.21.15.189:6837","nonce":1678817161}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.138:6833/1157967658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1157967658},{"type":"v1","addr":"172.21.15.138:6833","nonce":1157967658}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.138:6835/1909219758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1909219758},{"type":"v1","addr":"172.21.15.138:6835","nonce":1909219758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34493":{"gid":34493,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.138:6837/900566754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":900566754},{"type":"v1","addr":"172.21.15.138:6837","nonce":900566754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34520":{"gid":34520,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.138:6839/2236932393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2236932393},{"type":"v1","addr":"172.21.15.138:6839","nonce":2236932393}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34520,"qdb_cluster":[34520,24454,34427,34493,34451]},"id":1}]} 2024-04-24T09:01:17.425 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 35 2024-04-24T09:01:17.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.427+0000 7f9477a00640 1 -- 172.21.15.83:0/67821077 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f94580a3da0 msgr2=0x7f94580a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:17.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.427+0000 7f9477a00640 1 --2- 172.21.15.83:0/67821077 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f94580a3da0 0x7f94580a6260 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9464000c00 tx=0x7f9464031400 comp rx=0 tx=0).stop 2024-04-24T09:01:17.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.427+0000 7f9477a00640 1 -- 172.21.15.83:0/67821077 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f947010c5d0 msgr2=0x7f9470107e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:17.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.427+0000 7f9477a00640 1 --2- 172.21.15.83:0/67821077 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f947010c5d0 0x7f9470107e10 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f946800eaa0 tx=0x7f946800ef60 comp rx=0 tx=0).stop 2024-04-24T09:01:17.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.428+0000 7f9477a00640 1 -- 172.21.15.83:0/67821077 shutdown_connections 2024-04-24T09:01:17.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.428+0000 7f9477a00640 1 --2- 172.21.15.83:0/67821077 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f94580a3da0 0x7f94580a6260 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:17.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.428+0000 7f9477a00640 1 --2- 172.21.15.83:0/67821077 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f947010c5d0 0x7f9470107e10 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:17.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.428+0000 7f9477a00640 1 --2- 172.21.15.83:0/67821077 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9470103bb0 0x7f947010c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:17.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.428+0000 7f9477a00640 1 --2- 172.21.15.83:0/67821077 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9470103290 0x7f947010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:17.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.428+0000 7f9477a00640 1 -- 172.21.15.83:0/67821077 >> 172.21.15.83:0/67821077 conn(0x7f9470078000 msgr2=0x7f9470070c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:17.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.428+0000 7f9477a00640 1 -- 172.21.15.83:0/67821077 shutdown_connections 2024-04-24T09:01:17.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.428+0000 7f9477a00640 1 -- 172.21.15.83:0/67821077 wait complete. 2024-04-24T09:01:17.457 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-24T09:01:17.605 INFO:tasks.workunit.client.0.smithi083.stdout: 1 55740 0.62 MB/sec execute 351 sec latency 45.528 ms 2024-04-24T09:01:17.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.662+0000 7f6d43200640 1 -- 172.21.15.83:0/478134912 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d4410c480 msgr2=0x7f6d4410e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:17.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.662+0000 7f6d43200640 1 --2- 172.21.15.83:0/478134912 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d4410c480 0x7f6d4410e920 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c009ca0 tx=0x7f6d3c02a950 comp rx=0 tx=0).stop 2024-04-24T09:01:17.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.663+0000 7f6d43200640 1 -- 172.21.15.83:0/478134912 shutdown_connections 2024-04-24T09:01:17.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.663+0000 7f6d43200640 1 --2- 172.21.15.83:0/478134912 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d4410c480 0x7f6d4410e920 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:17.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.663+0000 7f6d43200640 1 --2- 172.21.15.83:0/478134912 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d44103980 0x7f6d4410be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:17.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.663+0000 7f6d43200640 1 --2- 172.21.15.83:0/478134912 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6d44103060 0x7f6d44103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:17.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.663+0000 7f6d43200640 1 -- 172.21.15.83:0/478134912 >> 172.21.15.83:0/478134912 conn(0x7f6d440fc880 msgr2=0x7f6d440feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:17.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.663+0000 7f6d43200640 1 -- 172.21.15.83:0/478134912 shutdown_connections 2024-04-24T09:01:17.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.663+0000 7f6d43200640 1 -- 172.21.15.83:0/478134912 wait complete. 2024-04-24T09:01:17.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.664+0000 7f6d43200640 1 Processor -- start 2024-04-24T09:01:17.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.664+0000 7f6d43200640 1 -- start start 2024-04-24T09:01:17.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.664+0000 7f6d43200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d44103060 0x7f6d441986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:17.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.664+0000 7f6d43200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d44103980 0x7f6d44198c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:17.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.664+0000 7f6d43200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6d4410c480 0x7f6d4419cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:17.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.664+0000 7f6d43200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6d44112160 con 0x7f6d44103980 2024-04-24T09:01:17.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.664+0000 7f6d43200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6d44111fe0 con 0x7f6d44103060 2024-04-24T09:01:17.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.664+0000 7f6d43200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6d441122e0 con 0x7f6d4410c480 2024-04-24T09:01:17.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.664+0000 7f6d41400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d44103980 0x7f6d44198c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:17.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.664+0000 7f6d41400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d44103980 0x7f6d44198c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48858/0 (socket says 172.21.15.83:48858) 2024-04-24T09:01:17.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.664+0000 7f6d41400640 1 -- 172.21.15.83:0/3079504351 learned_addr learned my addr 172.21.15.83:0/3079504351 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:01:17.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.665+0000 7f6d41e00640 1 --2- 172.21.15.83:0/3079504351 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d44103060 0x7f6d441986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:17.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.665+0000 7f6d42800640 1 --2- 172.21.15.83:0/3079504351 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6d4410c480 0x7f6d4419cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:17.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.665+0000 7f6d41400640 1 -- 172.21.15.83:0/3079504351 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6d4410c480 msgr2=0x7f6d4419cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:17.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.665+0000 7f6d41400640 1 --2- 172.21.15.83:0/3079504351 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6d4410c480 0x7f6d4419cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:17.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.665+0000 7f6d41400640 1 -- 172.21.15.83:0/3079504351 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d44103060 msgr2=0x7f6d441986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:17.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.665+0000 7f6d41400640 1 --2- 172.21.15.83:0/3079504351 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d44103060 0x7f6d441986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:17.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.665+0000 7f6d41400640 1 -- 172.21.15.83:0/3079504351 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d4419d660 con 0x7f6d44103980 2024-04-24T09:01:17.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.665+0000 7f6d41400640 1 --2- 172.21.15.83:0/3079504351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d44103980 0x7f6d44198c00 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f6d3000c940 tx=0x7f6d3000ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:17.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.665+0000 7f6d3aa00640 1 -- 172.21.15.83:0/3079504351 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d30004220 con 0x7f6d44103980 2024-04-24T09:01:17.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.666+0000 7f6d3aa00640 1 -- 172.21.15.83:0/3079504351 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6d300043c0 con 0x7f6d44103980 2024-04-24T09:01:17.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.666+0000 7f6d43200640 1 -- 172.21.15.83:0/3079504351 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d4419d950 con 0x7f6d44103980 2024-04-24T09:01:17.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.666+0000 7f6d3aa00640 1 -- 172.21.15.83:0/3079504351 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d30004a00 con 0x7f6d44103980 2024-04-24T09:01:17.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.666+0000 7f6d43200640 1 -- 172.21.15.83:0/3079504351 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d44100be0 con 0x7f6d44103980 2024-04-24T09:01:17.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.669+0000 7f6d4c1ee740 1 Processor -- start 2024-04-24T09:01:17.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.670+0000 7f6d4c1ee740 1 -- start start 2024-04-24T09:01:17.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.670+0000 7f6d4c1ee740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561815856b40 0x561815856f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:17.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.670+0000 7f6d4c1ee740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561815857460 0x5618158615f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:17.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.670+0000 7f6d4c1ee740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561815861b30 0x561815863f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:17.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.670+0000 7f6d4c1ee740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5618157bc1d0 con 0x561815857460 2024-04-24T09:01:17.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.670+0000 7f6d4c1ee740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x561815762450 con 0x561815861b30 2024-04-24T09:01:17.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.670+0000 7f6d4c1ee740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5618157b8190 con 0x561815856b40 2024-04-24T09:01:17.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.670+0000 7f6d41e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561815857460 0x5618158615f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:17.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.670+0000 7f6d41e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561815857460 0x5618158615f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48864/0 (socket says 172.21.15.83:48864) 2024-04-24T09:01:17.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.670+0000 7f6d41e00640 1 -- 172.21.15.83:0/281066822 learned_addr learned my addr 172.21.15.83:0/281066822 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:01:17.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.670+0000 7f6d3aa00640 1 -- 172.21.15.83:0/3079504351 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6d300079b0 con 0x7f6d44103980 2024-04-24T09:01:17.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.671+0000 7f6d42800640 1 --2- 172.21.15.83:0/281066822 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561815856b40 0x561815856f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:17.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.671+0000 7f6d41e00640 1 -- 172.21.15.83:0/281066822 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561815856b40 msgr2=0x561815856f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:17.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.671+0000 7f6d41e00640 1 --2- 172.21.15.83:0/281066822 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561815856b40 0x561815856f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:17.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.671+0000 7f6d41e00640 1 -- 172.21.15.83:0/281066822 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561815861b30 msgr2=0x561815863f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:17.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.671+0000 7f6d41e00640 1 --2- 172.21.15.83:0/281066822 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561815861b30 0x561815863f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:17.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.671+0000 7f6d41e00640 1 -- 172.21.15.83:0/281066822 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56181581f310 con 0x561815857460 2024-04-24T09:01:17.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.671+0000 7f6d41e00640 1 --2- 172.21.15.83:0/281066822 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561815857460 0x5618158615f0 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f6d34009c90 tx=0x7f6d340117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db4658c4e1c5e95b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:17.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.671+0000 7f6d2f400640 1 -- 172.21.15.83:0/281066822 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d34012450 con 0x561815857460 2024-04-24T09:01:17.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.672+0000 7f6d2f400640 1 -- 172.21.15.83:0/281066822 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6d34012aa0 con 0x561815857460 2024-04-24T09:01:17.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.672+0000 7f6d2f400640 1 -- 172.21.15.83:0/281066822 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d3400faf0 con 0x561815857460 2024-04-24T09:01:17.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.672+0000 7f6d4c1ee740 1 -- 172.21.15.83:0/281066822 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561815857460 msgr2=0x5618158615f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:17.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.672+0000 7f6d4c1ee740 1 --2- 172.21.15.83:0/281066822 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561815857460 0x5618158615f0 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f6d34009c90 tx=0x7f6d340117b0 comp rx=0 tx=0).stop 2024-04-24T09:01:17.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.672+0000 7f6d4c1ee740 1 -- 172.21.15.83:0/281066822 shutdown_connections 2024-04-24T09:01:17.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.672+0000 7f6d4c1ee740 1 --2- 172.21.15.83:0/281066822 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561815861b30 0x561815863f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:17.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.672+0000 7f6d4c1ee740 1 --2- 172.21.15.83:0/281066822 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561815857460 0x5618158615f0 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:17.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.672+0000 7f6d4c1ee740 1 --2- 172.21.15.83:0/281066822 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561815856b40 0x561815856f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:17.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.672+0000 7f6d4c1ee740 1 -- 172.21.15.83:0/281066822 >> 172.21.15.83:0/281066822 conn(0x561815852e60 msgr2=0x561815853240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:17.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.673+0000 7f6d3aa00640 1 --2- 172.21.15.83:0/3079504351 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f6d280a3d20 0x7f6d280a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:17.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.673+0000 7f6d4c1ee740 1 -- 172.21.15.83:0/281066822 shutdown_connections 2024-04-24T09:01:17.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.673+0000 7f6d3aa00640 1 -- 172.21.15.83:0/3079504351 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7f6d300d0990 con 0x7f6d44103980 2024-04-24T09:01:17.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.673+0000 7f6d4c1ee740 1 -- 172.21.15.83:0/281066822 wait complete. 2024-04-24T09:01:17.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.673+0000 7f6d42800640 1 --2- 172.21.15.83:0/3079504351 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f6d280a3d20 0x7f6d280a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:17.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.673+0000 7f6d4c1ee740 1 -- start start 2024-04-24T09:01:17.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.674+0000 7f6d4c1ee740 1 Processor -- start 2024-04-24T09:01:17.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.674+0000 7f6d42800640 1 --2- 172.21.15.83:0/3079504351 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f6d280a3d20 0x7f6d280a61e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c0096c0 tx=0x7f6d3c009630 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:17.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.674+0000 7f6d4c1ee740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561815856b40 0x561815861580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:17.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.674+0000 7f6d4c1ee740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561815857460 0x56181585e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:17.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.674+0000 7f6d4c1ee740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561815861b30 0x56181585efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:17.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.674+0000 7f6d4c1ee740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5618158649c0 con 0x561815861b30 2024-04-24T09:01:17.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.674+0000 7f6d4c1ee740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x561815775810 con 0x561815857460 2024-04-24T09:01:17.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.674+0000 7f6d4c1ee740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x561815864b40 con 0x561815856b40 2024-04-24T09:01:17.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.674+0000 7f6d4c1ee740 10 client.0 _ll_register_callbacks cb 0x561815743140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:01:17.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.675+0000 7f6d42800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561815861b30 0x56181585efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:17.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.675+0000 7f6d42800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561815861b30 0x56181585efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48874/0 (socket says 172.21.15.83:48874) 2024-04-24T09:01:17.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.675+0000 7f6d42800640 1 -- 172.21.15.83:0/3855154395 learned_addr learned my addr 172.21.15.83:0/3855154395 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:01:17.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.675+0000 7f6d41e00640 1 --2- 172.21.15.83:0/3855154395 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561815856b40 0x561815861580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:17.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.675+0000 7f6d41400640 1 --2- 172.21.15.83:0/3855154395 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561815857460 0x56181585e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:17.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.675+0000 7f6d42800640 1 -- 172.21.15.83:0/3855154395 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561815856b40 msgr2=0x561815861580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:17.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.675+0000 7f6d42800640 1 --2- 172.21.15.83:0/3855154395 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561815856b40 0x561815861580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:17.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.675+0000 7f6d42800640 1 -- 172.21.15.83:0/3855154395 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561815857460 msgr2=0x56181585e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:17.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.675+0000 7f6d42800640 1 --2- 172.21.15.83:0/3855154395 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561815857460 0x56181585e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:17.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.675+0000 7f6d42800640 1 -- 172.21.15.83:0/3855154395 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618157863f0 con 0x561815861b30 2024-04-24T09:01:17.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.676+0000 7f6d42800640 1 --2- 172.21.15.83:0/3855154395 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561815861b30 0x56181585efb0 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c02faf0 tx=0x7f6d3c0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a78656ee2b5cdf3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:17.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.676+0000 7f6d23e00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:01:17.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.676+0000 7f6d23e00640 1 -- 172.21.15.83:0/3855154395 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d3c034040 con 0x561815861b30 2024-04-24T09:01:17.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.676+0000 7f6d23e00640 1 -- 172.21.15.83:0/3855154395 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7f6d3c03cbd0 con 0x561815861b30 2024-04-24T09:01:17.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.676+0000 7f6d4c1ee740 1 -- 172.21.15.83:0/3855154395 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d34012aa0 con 0x561815861b30 2024-04-24T09:01:17.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.676+0000 7f6d23e00640 1 -- 172.21.15.83:0/3855154395 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f6d3c03cdd0 con 0x561815861b30 2024-04-24T09:01:17.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.676+0000 7f6d4c1ee740 10 client.19038 fetch_fsmap learned FSMap version 35 2024-04-24T09:01:17.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.676+0000 7f6d4c1ee740 1 -- 172.21.15.83:0/3855154395 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5618154ce3f0 con 0x561815861b30 2024-04-24T09:01:17.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.676+0000 7f6d23e00640 1 -- 172.21.15.83:0/3855154395 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6d34012aa0 con 0x561815861b30 2024-04-24T09:01:17.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.677+0000 7f6d23e00640 1 -- 172.21.15.83:0/3855154395 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7f6d3c052070 con 0x561815861b30 2024-04-24T09:01:17.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.677+0000 7f6d4c1ee740 10 client.19038 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T09:01:17.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.677+0000 7f6d4c1ee740 10 client.19038 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-24T09:01:17.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.677+0000 7f6d4c1ee740 20 client.19038 populate_metadata read hostname 'smithi083' 2024-04-24T09:01:17.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.677+0000 7f6d4c1ee740 1 --2- 172.21.15.83:0/3855154395 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x561815871c10 0x561815874060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:17.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.677+0000 7f6d4c1ee740 4 client.19038 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:01:17.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.677+0000 7f6d4c1ee740 1 -- 172.21.15.83:0/3855154395 --> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5618157c9bf0 con 0x561815871c10 2024-04-24T09:01:17.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.678+0000 7f6d41e00640 1 --2- 172.21.15.83:0/3855154395 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x561815871c10 0x561815874060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:17.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.679+0000 7f6d41e00640 1 --2- 172.21.15.83:0/3855154395 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x561815871c10 0x561815874060 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=405e647de8b68d66 server_cookie=3fe9137ebe5a6491 in_seq=0 out_seq=0 2024-04-24T09:01:17.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.679+0000 7f6d23e00640 10 client.19038 ms_handle_connect on v2:172.21.15.189:6838/2548965336 2024-04-24T09:01:17.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.680+0000 7f6d23e00640 1 -- 172.21.15.83:0/3855154395 <== mds.4 v2:172.21.15.189:6838/2548965336 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5618157c9bf0 con 0x561815871c10 2024-04-24T09:01:17.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.680+0000 7f6d23e00640 10 client.19038 handle_command_reply: tid=0 2024-04-24T09:01:17.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.681+0000 7f6d4c1ee740 1 client.19038 shutdown 2024-04-24T09:01:17.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.681+0000 7f6d4c1ee740 1 -- 172.21.15.83:0/3855154395 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561815861b30 msgr2=0x56181585efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:17.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.681+0000 7f6d4c1ee740 1 --2- 172.21.15.83:0/3855154395 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561815861b30 0x56181585efb0 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c02faf0 tx=0x7f6d3c0095c0 comp rx=0 tx=0).stop 2024-04-24T09:01:17.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.681+0000 7f6d4c1ee740 1 -- 172.21.15.83:0/3855154395 shutdown_connections 2024-04-24T09:01:17.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.681+0000 7f6d4c1ee740 1 --2- 172.21.15.83:0/3855154395 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x561815871c10 0x561815874060 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:17.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.681+0000 7f6d4c1ee740 1 --2- 172.21.15.83:0/3855154395 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561815861b30 0x56181585efb0 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:17.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.681+0000 7f6d4c1ee740 1 --2- 172.21.15.83:0/3855154395 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561815857460 0x56181585e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:17.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.681+0000 7f6d4c1ee740 1 --2- 172.21.15.83:0/3855154395 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561815856b40 0x561815861580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:17.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.681+0000 7f6d4c1ee740 1 -- 172.21.15.83:0/3855154395 >> 172.21.15.83:0/3855154395 conn(0x561815852e60 msgr2=0x561815858b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:17.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.681+0000 7f6d23e00640 0 client.19038 ms_handle_reset on v2:172.21.15.189:6838/2548965336 2024-04-24T09:01:17.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.681+0000 7f6d4c1ee740 1 -- 172.21.15.83:0/3855154395 shutdown_connections 2024-04-24T09:01:17.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.681+0000 7f6d4c1ee740 1 -- 172.21.15.83:0/3855154395 wait complete. 2024-04-24T09:01:17.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.681+0000 7f6d4c1ee740 20 client.19038 trim_cache size 0 max 16384 2024-04-24T09:01:17.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.699+0000 7f6d4c1ee740 1 Processor -- start 2024-04-24T09:01:17.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.700+0000 7f6d4c1ee740 1 -- start start 2024-04-24T09:01:17.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.700+0000 7f6d4c1ee740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561815856b40 0x5618158592e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:17.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.700+0000 7f6d4c1ee740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561815857460 0x56181585d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:17.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.700+0000 7f6d4c1ee740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561815861b30 0x561815851dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:17.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.700+0000 7f6d4c1ee740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6d441ad240 con 0x561815861b30 2024-04-24T09:01:17.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.700+0000 7f6d4c1ee740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6d441aed80 con 0x561815856b40 2024-04-24T09:01:17.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.700+0000 7f6d4c1ee740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6d441ad910 con 0x561815857460 2024-04-24T09:01:17.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.700+0000 7f6d42800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561815861b30 0x561815851dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:17.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.700+0000 7f6d42800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561815861b30 0x561815851dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48886/0 (socket says 172.21.15.83:48886) 2024-04-24T09:01:17.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.700+0000 7f6d42800640 1 -- 172.21.15.83:0/3604142269 learned_addr learned my addr 172.21.15.83:0/3604142269 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:01:17.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.701+0000 7f6d41400640 1 --2- 172.21.15.83:0/3604142269 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561815857460 0x56181585d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:17.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.701+0000 7f6d41e00640 1 --2- 172.21.15.83:0/3604142269 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561815856b40 0x5618158592e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:17.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.701+0000 7f6d42800640 1 -- 172.21.15.83:0/3604142269 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561815857460 msgr2=0x56181585d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:17.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.701+0000 7f6d42800640 1 --2- 172.21.15.83:0/3604142269 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561815857460 0x56181585d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:17.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.701+0000 7f6d42800640 1 -- 172.21.15.83:0/3604142269 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561815856b40 msgr2=0x5618158592e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:17.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.701+0000 7f6d42800640 1 --2- 172.21.15.83:0/3604142269 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561815856b40 0x5618158592e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:17.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.701+0000 7f6d42800640 1 -- 172.21.15.83:0/3604142269 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618155987d0 con 0x561815861b30 2024-04-24T09:01:17.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.701+0000 7f6d42800640 1 --2- 172.21.15.83:0/3604142269 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561815861b30 0x561815851dd0 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c02ffb0 tx=0x7f6d3c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e5d3be7fc9d770c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:17.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.701+0000 7f6d2ea00640 1 -- 172.21.15.83:0/3604142269 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d3c03c9d0 con 0x561815861b30 2024-04-24T09:01:17.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.701+0000 7f6d2ea00640 1 -- 172.21.15.83:0/3604142269 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6d3c03cb70 con 0x561815861b30 2024-04-24T09:01:17.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.702+0000 7f6d2ea00640 1 -- 172.21.15.83:0/3604142269 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d3c04a710 con 0x561815861b30 2024-04-24T09:01:17.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.702+0000 7f6d4c1ee740 1 -- 172.21.15.83:0/3604142269 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561815861b30 msgr2=0x561815851dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:17.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.702+0000 7f6d4c1ee740 1 --2- 172.21.15.83:0/3604142269 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561815861b30 0x561815851dd0 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c02ffb0 tx=0x7f6d3c005fd0 comp rx=0 tx=0).stop 2024-04-24T09:01:17.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.702+0000 7f6d4c1ee740 1 -- 172.21.15.83:0/3604142269 shutdown_connections 2024-04-24T09:01:17.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.702+0000 7f6d4c1ee740 1 --2- 172.21.15.83:0/3604142269 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561815861b30 0x561815851dd0 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:17.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.702+0000 7f6d4c1ee740 1 --2- 172.21.15.83:0/3604142269 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561815857460 0x56181585d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:17.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.702+0000 7f6d4c1ee740 1 --2- 172.21.15.83:0/3604142269 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561815856b40 0x5618158592e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:17.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.702+0000 7f6d4c1ee740 1 -- 172.21.15.83:0/3604142269 >> 172.21.15.83:0/3604142269 conn(0x561815852e60 msgr2=0x5618158516c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:17.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.702+0000 7f6d4c1ee740 1 -- 172.21.15.83:0/3604142269 shutdown_connections 2024-04-24T09:01:17.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.703+0000 7f6d4c1ee740 1 -- 172.21.15.83:0/3604142269 wait complete. 2024-04-24T09:01:17.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.703+0000 7f6d4c1ee740 1 -- start start 2024-04-24T09:01:17.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.703+0000 7f6d4c1ee740 1 Processor -- start 2024-04-24T09:01:17.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.704+0000 7f6d4c1ee740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561815856b40 0x56181585c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:17.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.704+0000 7f6d4c1ee740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561815857460 0x561815869d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:17.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.704+0000 7f6d4c1ee740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561815861b30 0x561815862c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:17.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.704+0000 7f6d4c1ee740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6d441ae6b0 con 0x561815857460 2024-04-24T09:01:17.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.704+0000 7f6d4c1ee740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6d441adfe0 con 0x561815861b30 2024-04-24T09:01:17.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.704+0000 7f6d4c1ee740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6d441ab240 con 0x561815856b40 2024-04-24T09:01:17.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.704+0000 7f6d4c1ee740 10 client.0 _ll_register_callbacks cb 0x7f6d34005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:01:17.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.704+0000 7f6d41400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561815857460 0x561815869d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:17.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.704+0000 7f6d41400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561815857460 0x561815869d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48890/0 (socket says 172.21.15.83:48890) 2024-04-24T09:01:17.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.704+0000 7f6d41400640 1 -- 172.21.15.83:0/4017622329 learned_addr learned my addr 172.21.15.83:0/4017622329 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:01:17.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.704+0000 7f6d41e00640 1 --2- 172.21.15.83:0/4017622329 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561815856b40 0x56181585c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:17.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.704+0000 7f6d42800640 1 --2- 172.21.15.83:0/4017622329 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561815861b30 0x561815862c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:17.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.704+0000 7f6d41400640 1 -- 172.21.15.83:0/4017622329 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561815856b40 msgr2=0x56181585c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:17.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.704+0000 7f6d41400640 1 --2- 172.21.15.83:0/4017622329 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561815856b40 0x56181585c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:17.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.704+0000 7f6d41400640 1 -- 172.21.15.83:0/4017622329 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561815861b30 msgr2=0x561815862c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:17.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.704+0000 7f6d41400640 1 --2- 172.21.15.83:0/4017622329 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561815861b30 0x561815862c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:17.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.704+0000 7f6d41400640 1 -- 172.21.15.83:0/4017622329 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56181586a630 con 0x561815857460 2024-04-24T09:01:17.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.704+0000 7f6d42800640 1 --2- 172.21.15.83:0/4017622329 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561815861b30 0x561815862c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:01:17.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.705+0000 7f6d41400640 1 --2- 172.21.15.83:0/4017622329 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561815857460 0x561815869d90 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f6d30004770 tx=0x7f6d3009be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d279bf7ba430f495 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:17.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.705+0000 7f6d23e00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:01:17.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.705+0000 7f6d23e00640 1 -- 172.21.15.83:0/4017622329 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d3001d040 con 0x561815857460 2024-04-24T09:01:17.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.705+0000 7f6d4c1ee740 1 -- 172.21.15.83:0/4017622329 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d3c03cb70 con 0x561815857460 2024-04-24T09:01:17.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.705+0000 7f6d23e00640 1 -- 172.21.15.83:0/4017622329 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7f6d300cdee0 con 0x561815857460 2024-04-24T09:01:17.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.705+0000 7f6d23e00640 1 -- 172.21.15.83:0/4017622329 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f6d300ce0e0 con 0x561815857460 2024-04-24T09:01:17.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.705+0000 7f6d4c1ee740 10 client.19044 fetch_fsmap learned FSMap version 35 2024-04-24T09:01:17.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.705+0000 7f6d4c1ee740 1 -- 172.21.15.83:0/4017622329 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5618158d87e0 con 0x561815857460 2024-04-24T09:01:17.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.706+0000 7f6d23e00640 1 -- 172.21.15.83:0/4017622329 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6d3c03cb70 con 0x561815857460 2024-04-24T09:01:17.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.706+0000 7f6d23e00640 1 -- 172.21.15.83:0/4017622329 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7f6d30098200 con 0x561815857460 2024-04-24T09:01:17.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.706+0000 7f6d4c1ee740 10 client.19044 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T09:01:17.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.706+0000 7f6d4c1ee740 10 client.19044 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-24T09:01:17.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.706+0000 7f6d4c1ee740 20 client.19044 populate_metadata read hostname 'smithi083' 2024-04-24T09:01:17.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.706+0000 7f6d4c1ee740 1 --2- 172.21.15.83:0/4017622329 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x5618158d8f90 0x5618158db3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:17.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.706+0000 7f6d4c1ee740 4 client.19044 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:01:17.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.706+0000 7f6d4c1ee740 1 -- 172.21.15.83:0/4017622329 --> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5618157c3a60 con 0x5618158d8f90 2024-04-24T09:01:17.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.707+0000 7f6d41e00640 1 --2- 172.21.15.83:0/4017622329 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x5618158d8f90 0x5618158db3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:17.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.707+0000 7f6d41e00640 1 --2- 172.21.15.83:0/4017622329 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x5618158d8f90 0x5618158db3b0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2678d30c72ed11e0 server_cookie=66760ba486163c65 in_seq=0 out_seq=0 2024-04-24T09:01:17.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.707+0000 7f6d23e00640 10 client.19044 ms_handle_connect on v2:172.21.15.189:6838/2548965336 2024-04-24T09:01:17.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.708+0000 7f6d23e00640 1 -- 172.21.15.83:0/4017622329 <== mds.4 v2:172.21.15.189:6838/2548965336 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5618157c3a60 con 0x5618158d8f90 2024-04-24T09:01:17.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.708+0000 7f6d23e00640 10 client.19044 handle_command_reply: tid=0 2024-04-24T09:01:17.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.708+0000 7f6d4c1ee740 1 client.19044 shutdown 2024-04-24T09:01:17.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.708+0000 7f6d4c1ee740 1 -- 172.21.15.83:0/4017622329 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561815857460 msgr2=0x561815869d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:17.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.708+0000 7f6d4c1ee740 1 --2- 172.21.15.83:0/4017622329 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561815857460 0x561815869d90 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f6d30004770 tx=0x7f6d3009be10 comp rx=0 tx=0).stop 2024-04-24T09:01:17.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.708+0000 7f6d4c1ee740 1 -- 172.21.15.83:0/4017622329 shutdown_connections 2024-04-24T09:01:17.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.708+0000 7f6d4c1ee740 1 --2- 172.21.15.83:0/4017622329 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x5618158d8f90 0x5618158db3b0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:17.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.708+0000 7f6d4c1ee740 1 --2- 172.21.15.83:0/4017622329 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561815861b30 0x561815862c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:17.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.708+0000 7f6d4c1ee740 1 --2- 172.21.15.83:0/4017622329 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561815857460 0x561815869d90 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:17.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.708+0000 7f6d4c1ee740 1 --2- 172.21.15.83:0/4017622329 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561815856b40 0x56181585c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:17.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.708+0000 7f6d4c1ee740 1 -- 172.21.15.83:0/4017622329 >> 172.21.15.83:0/4017622329 conn(0x561815852e60 msgr2=0x561815851f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:17.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.708+0000 7f6d23e00640 0 client.19044 ms_handle_reset on v2:172.21.15.189:6838/2548965336 2024-04-24T09:01:17.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.708+0000 7f6d4c1ee740 1 -- 172.21.15.83:0/4017622329 shutdown_connections 2024-04-24T09:01:17.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.708+0000 7f6d4c1ee740 1 -- 172.21.15.83:0/4017622329 wait complete. 2024-04-24T09:01:17.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.709+0000 7f6d4c1ee740 20 client.19044 trim_cache size 0 max 16384 2024-04-24T09:01:17.712 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:01:17.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.709+0000 7f6d43200640 1 -- 172.21.15.83:0/3079504351 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f6d280a3d20 msgr2=0x7f6d280a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:17.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.709+0000 7f6d43200640 1 --2- 172.21.15.83:0/3079504351 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f6d280a3d20 0x7f6d280a61e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c0096c0 tx=0x7f6d3c009630 comp rx=0 tx=0).stop 2024-04-24T09:01:17.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.709+0000 7f6d43200640 1 -- 172.21.15.83:0/3079504351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d44103980 msgr2=0x7f6d44198c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:17.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.709+0000 7f6d43200640 1 --2- 172.21.15.83:0/3079504351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d44103980 0x7f6d44198c00 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f6d3000c940 tx=0x7f6d3000ce00 comp rx=0 tx=0).stop 2024-04-24T09:01:17.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.709+0000 7f6d43200640 1 -- 172.21.15.83:0/3079504351 shutdown_connections 2024-04-24T09:01:17.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.709+0000 7f6d43200640 1 --2- 172.21.15.83:0/3079504351 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f6d280a3d20 0x7f6d280a61e0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:17.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.709+0000 7f6d43200640 1 --2- 172.21.15.83:0/3079504351 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6d4410c480 0x7f6d4419cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:17.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.709+0000 7f6d43200640 1 --2- 172.21.15.83:0/3079504351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d44103980 0x7f6d44198c00 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:17.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.709+0000 7f6d43200640 1 --2- 172.21.15.83:0/3079504351 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d44103060 0x7f6d441986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:17.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.709+0000 7f6d43200640 1 -- 172.21.15.83:0/3079504351 >> 172.21.15.83:0/3079504351 conn(0x7f6d440fc880 msgr2=0x7f6d440fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:17.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.710+0000 7f6d43200640 1 -- 172.21.15.83:0/3079504351 shutdown_connections 2024-04-24T09:01:17.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.710+0000 7f6d43200640 1 -- 172.21.15.83:0/3079504351 wait complete. 2024-04-24T09:01:17.735 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-24T09:01:17.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.936+0000 7f8679a00640 1 -- 172.21.15.83:0/703328446 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86741071e0 msgr2=0x7f867410edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:17.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.936+0000 7f8679a00640 1 --2- 172.21.15.83:0/703328446 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86741071e0 0x7f867410edb0 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f866c009900 tx=0x7f866c02a410 comp rx=0 tx=0).stop 2024-04-24T09:01:17.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.937+0000 7f8679a00640 1 -- 172.21.15.83:0/703328446 shutdown_connections 2024-04-24T09:01:17.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.937+0000 7f8679a00640 1 --2- 172.21.15.83:0/703328446 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86741071e0 0x7f867410edb0 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:17.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.937+0000 7f8679a00640 1 --2- 172.21.15.83:0/703328446 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8674102ac0 0x7f8674106b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:17.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.937+0000 7f8679a00640 1 --2- 172.21.15.83:0/703328446 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8674102110 0x7f86741024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:17.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.937+0000 7f8679a00640 1 -- 172.21.15.83:0/703328446 >> 172.21.15.83:0/703328446 conn(0x7f86740fd9d0 msgr2=0x7f86740ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:17.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.937+0000 7f8679a00640 1 -- 172.21.15.83:0/703328446 shutdown_connections 2024-04-24T09:01:17.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.937+0000 7f8679a00640 1 -- 172.21.15.83:0/703328446 wait complete. 2024-04-24T09:01:17.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.938+0000 7f8679a00640 1 Processor -- start 2024-04-24T09:01:17.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.938+0000 7f8679a00640 1 -- start start 2024-04-24T09:01:17.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.938+0000 7f8679a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8674102110 0x7f86741a05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:17.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.938+0000 7f8679a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86741071e0 0x7f86741a0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:17.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.938+0000 7f8679a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f867419a670 0x7f867419aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:17.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.938+0000 7f8679a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8674112130 con 0x7f86741071e0 2024-04-24T09:01:17.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.938+0000 7f8679a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8674111fb0 con 0x7f8674102110 2024-04-24T09:01:17.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.938+0000 7f8679a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f86741122b0 con 0x7f867419a670 2024-04-24T09:01:17.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.939+0000 7f866b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86741071e0 0x7f86741a0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:17.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.939+0000 7f866b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86741071e0 0x7f86741a0af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48906/0 (socket says 172.21.15.83:48906) 2024-04-24T09:01:17.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.939+0000 7f866b400640 1 -- 172.21.15.83:0/2369274839 learned_addr learned my addr 172.21.15.83:0/2369274839 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:01:17.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.939+0000 7f8673e00640 1 --2- 172.21.15.83:0/2369274839 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8674102110 0x7f86741a05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:17.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.939+0000 7f8679000640 1 --2- 172.21.15.83:0/2369274839 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f867419a670 0x7f867419aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:17.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.939+0000 7f866b400640 1 -- 172.21.15.83:0/2369274839 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f867419a670 msgr2=0x7f867419aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:17.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.939+0000 7f866b400640 1 --2- 172.21.15.83:0/2369274839 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f867419a670 0x7f867419aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:17.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.939+0000 7f866b400640 1 -- 172.21.15.83:0/2369274839 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8674102110 msgr2=0x7f86741a05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:17.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.939+0000 7f866b400640 1 --2- 172.21.15.83:0/2369274839 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8674102110 0x7f86741a05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:17.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.939+0000 7f866b400640 1 -- 172.21.15.83:0/2369274839 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f867419b270 con 0x7f86741071e0 2024-04-24T09:01:17.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.939+0000 7f8673e00640 1 --2- 172.21.15.83:0/2369274839 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8674102110 0x7f86741a05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:01:17.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.940+0000 7f866b400640 1 --2- 172.21.15.83:0/2369274839 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86741071e0 0x7f86741a0af0 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f865c00d770 tx=0x7f865c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:17.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.940+0000 7f8671600640 1 -- 172.21.15.83:0/2369274839 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f865c017070 con 0x7f86741071e0 2024-04-24T09:01:17.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.940+0000 7f8679a00640 1 -- 172.21.15.83:0/2369274839 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86741a7350 con 0x7f86741071e0 2024-04-24T09:01:17.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.940+0000 7f8671600640 1 -- 172.21.15.83:0/2369274839 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f865c009c30 con 0x7f86741071e0 2024-04-24T09:01:17.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.940+0000 7f8671600640 1 -- 172.21.15.83:0/2369274839 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f865c00f620 con 0x7f86741071e0 2024-04-24T09:01:17.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.940+0000 7f8679a00640 1 -- 172.21.15.83:0/2369274839 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86741a77b0 con 0x7f86741071e0 2024-04-24T09:01:17.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.944+0000 7f867dfa3740 1 Processor -- start 2024-04-24T09:01:17.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.944+0000 7f867dfa3740 1 -- start start 2024-04-24T09:01:17.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.944+0000 7f8671600640 1 -- 172.21.15.83:0/2369274839 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f865c01a070 con 0x7f86741071e0 2024-04-24T09:01:17.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.944+0000 7f867dfa3740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561af4b27b40 0x561af4b27f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:17.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.944+0000 7f867dfa3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561af4b28460 0x561af4b325f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:17.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.944+0000 7f867dfa3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561af4b32b30 0x561af4b34f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:17.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.944+0000 7f867dfa3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561af4a8d1d0 con 0x561af4b32b30 2024-04-24T09:01:17.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.944+0000 7f867dfa3740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x561af4a33450 con 0x561af4b27b40 2024-04-24T09:01:17.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.944+0000 7f867dfa3740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x561af4a89190 con 0x561af4b28460 2024-04-24T09:01:17.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.945+0000 7f866b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561af4b32b30 0x561af4b34f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:17.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.945+0000 7f866b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561af4b32b30 0x561af4b34f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48908/0 (socket says 172.21.15.83:48908) 2024-04-24T09:01:17.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.945+0000 7f866b400640 1 -- 172.21.15.83:0/688117928 learned_addr learned my addr 172.21.15.83:0/688117928 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:01:17.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.945+0000 7f8673e00640 1 --2- 172.21.15.83:0/688117928 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561af4b28460 0x561af4b325f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:17.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.945+0000 7f8679000640 1 --2- 172.21.15.83:0/688117928 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561af4b27b40 0x561af4b27f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:17.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.945+0000 7f866b400640 1 -- 172.21.15.83:0/688117928 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561af4b28460 msgr2=0x561af4b325f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:17.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.945+0000 7f866b400640 1 --2- 172.21.15.83:0/688117928 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561af4b28460 0x561af4b325f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:17.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.945+0000 7f866b400640 1 -- 172.21.15.83:0/688117928 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561af4b27b40 msgr2=0x561af4b27f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:17.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.945+0000 7f866b400640 1 --2- 172.21.15.83:0/688117928 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561af4b27b40 0x561af4b27f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:17.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.945+0000 7f866b400640 1 -- 172.21.15.83:0/688117928 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561af4af0310 con 0x561af4b32b30 2024-04-24T09:01:17.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.946+0000 7f866b400640 1 --2- 172.21.15.83:0/688117928 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561af4b32b30 0x561af4b34f20 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f865c0cda00 tx=0x7f865c0d8670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1bdf0d2932617db server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:17.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.946+0000 7f8669600640 1 -- 172.21.15.83:0/688117928 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f865c0df070 con 0x561af4b32b30 2024-04-24T09:01:17.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.946+0000 7f8669600640 1 -- 172.21.15.83:0/688117928 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f865c0d0de0 con 0x561af4b32b30 2024-04-24T09:01:17.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.946+0000 7f8669600640 1 -- 172.21.15.83:0/688117928 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f865c0db640 con 0x561af4b32b30 2024-04-24T09:01:17.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.946+0000 7f8671600640 1 --2- 172.21.15.83:0/2369274839 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f86580a3d20 0x7f86580a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:17.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.947+0000 7f8671600640 1 -- 172.21.15.83:0/2369274839 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7f865c0d07f0 con 0x7f86741071e0 2024-04-24T09:01:17.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.947+0000 7f8679000640 1 --2- 172.21.15.83:0/2369274839 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f86580a3d20 0x7f86580a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:17.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.947+0000 7f867dfa3740 1 -- 172.21.15.83:0/688117928 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561af4b32b30 msgr2=0x561af4b34f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:17.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.947+0000 7f867dfa3740 1 --2- 172.21.15.83:0/688117928 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561af4b32b30 0x561af4b34f20 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f865c0cda00 tx=0x7f865c0d8670 comp rx=0 tx=0).stop 2024-04-24T09:01:17.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.947+0000 7f867dfa3740 1 -- 172.21.15.83:0/688117928 shutdown_connections 2024-04-24T09:01:17.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.947+0000 7f867dfa3740 1 --2- 172.21.15.83:0/688117928 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561af4b32b30 0x561af4b34f20 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:17.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.947+0000 7f867dfa3740 1 --2- 172.21.15.83:0/688117928 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561af4b28460 0x561af4b325f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:17.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.947+0000 7f867dfa3740 1 --2- 172.21.15.83:0/688117928 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561af4b27b40 0x561af4b27f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:17.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.947+0000 7f867dfa3740 1 -- 172.21.15.83:0/688117928 >> 172.21.15.83:0/688117928 conn(0x561af4b23e60 msgr2=0x561af4b24240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:17.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.947+0000 7f867dfa3740 1 -- 172.21.15.83:0/688117928 shutdown_connections 2024-04-24T09:01:17.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.947+0000 7f8679000640 1 --2- 172.21.15.83:0/2369274839 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f86580a3d20 0x7f86580a61e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f866c005940 tx=0x7f866c0058b0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:17.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.947+0000 7f867dfa3740 1 -- 172.21.15.83:0/688117928 wait complete. 2024-04-24T09:01:17.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.948+0000 7f867dfa3740 1 -- start start 2024-04-24T09:01:17.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.948+0000 7f867dfa3740 1 Processor -- start 2024-04-24T09:01:17.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.948+0000 7f867dfa3740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561af4b27b40 0x561af4b32580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:17.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.949+0000 7f867dfa3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561af4b28460 0x561af4b2f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:17.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.949+0000 7f867dfa3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561af4b32b30 0x561af4b2ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:17.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.949+0000 7f867dfa3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561af4b359c0 con 0x561af4b28460 2024-04-24T09:01:17.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.949+0000 7f867dfa3740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x561af4a46810 con 0x561af4b27b40 2024-04-24T09:01:17.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.949+0000 7f867dfa3740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x561af4b35b40 con 0x561af4b32b30 2024-04-24T09:01:17.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.949+0000 7f867dfa3740 10 client.0 _ll_register_callbacks cb 0x561af4a14140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:01:17.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.949+0000 7f866b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561af4b28460 0x561af4b2f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:17.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.949+0000 7f866b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561af4b28460 0x561af4b2f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48912/0 (socket says 172.21.15.83:48912) 2024-04-24T09:01:17.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.949+0000 7f866b400640 1 -- 172.21.15.83:0/119410066 learned_addr learned my addr 172.21.15.83:0/119410066 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:01:17.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.949+0000 7f8673e00640 1 --2- 172.21.15.83:0/119410066 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561af4b27b40 0x561af4b32580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:17.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.949+0000 7f8679000640 1 --2- 172.21.15.83:0/119410066 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561af4b32b30 0x561af4b2ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:17.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.949+0000 7f866b400640 1 -- 172.21.15.83:0/119410066 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561af4b32b30 msgr2=0x561af4b2ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:17.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.949+0000 7f866b400640 1 --2- 172.21.15.83:0/119410066 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561af4b32b30 0x561af4b2ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:17.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.949+0000 7f866b400640 1 -- 172.21.15.83:0/119410066 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561af4b27b40 msgr2=0x561af4b32580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:17.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.949+0000 7f866b400640 1 --2- 172.21.15.83:0/119410066 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561af4b27b40 0x561af4b32580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:17.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.950+0000 7f866b400640 1 -- 172.21.15.83:0/119410066 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561af4a573f0 con 0x561af4b28460 2024-04-24T09:01:17.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.950+0000 7f866b400640 1 --2- 172.21.15.83:0/119410066 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561af4b28460 0x561af4b2f870 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f865c00d740 tx=0x7f865c00de80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f76a8d110b1cc0d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:17.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.950+0000 7f8662000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:01:17.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.950+0000 7f8662000640 1 -- 172.21.15.83:0/119410066 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f865c021070 con 0x561af4b28460 2024-04-24T09:01:17.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.950+0000 7f8662000640 1 -- 172.21.15.83:0/119410066 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7f865c0d66d0 con 0x561af4b28460 2024-04-24T09:01:17.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.950+0000 7f867dfa3740 1 -- 172.21.15.83:0/119410066 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f865c0d0de0 con 0x561af4b28460 2024-04-24T09:01:17.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.950+0000 7f8662000640 1 -- 172.21.15.83:0/119410066 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f865c0df030 con 0x561af4b28460 2024-04-24T09:01:17.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.950+0000 7f867dfa3740 10 client.19056 fetch_fsmap learned FSMap version 35 2024-04-24T09:01:17.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.951+0000 7f867dfa3740 1 -- 172.21.15.83:0/119410066 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x561af479f3f0 con 0x561af4b28460 2024-04-24T09:01:17.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.951+0000 7f8662000640 1 -- 172.21.15.83:0/119410066 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f865c0d0de0 con 0x561af4b28460 2024-04-24T09:01:17.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.951+0000 7f8662000640 1 -- 172.21.15.83:0/119410066 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7f865c098980 con 0x561af4b28460 2024-04-24T09:01:17.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.951+0000 7f867dfa3740 10 client.19056 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T09:01:17.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.951+0000 7f867dfa3740 10 client.19056 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-24T09:01:17.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.951+0000 7f867dfa3740 20 client.19056 populate_metadata read hostname 'smithi083' 2024-04-24T09:01:17.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.951+0000 7f867dfa3740 1 --2- 172.21.15.83:0/119410066 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x561af4b42bc0 0x561af4b44fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:17.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.951+0000 7f867dfa3740 4 client.19056 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:01:17.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.951+0000 7f867dfa3740 1 -- 172.21.15.83:0/119410066 --> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561af4a9abf0 con 0x561af4b42bc0 2024-04-24T09:01:17.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.952+0000 7f8673e00640 1 --2- 172.21.15.83:0/119410066 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x561af4b42bc0 0x561af4b44fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:17.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.953+0000 7f8662000640 10 client.19056 ms_handle_connect on v2:172.21.15.138:6832/1157967658 2024-04-24T09:01:17.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.953+0000 7f8673e00640 1 --2- 172.21.15.83:0/119410066 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x561af4b42bc0 0x561af4b44fb0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b459e277a0e1633a server_cookie=54a2907b26ee11f7 in_seq=0 out_seq=0 2024-04-24T09:01:17.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.956+0000 7f8662000640 1 -- 172.21.15.83:0/119410066 <== mds.3 v2:172.21.15.138:6832/1157967658 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561af4a9abf0 con 0x561af4b42bc0 2024-04-24T09:01:17.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.956+0000 7f8662000640 10 client.19056 handle_command_reply: tid=0 2024-04-24T09:01:17.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.957+0000 7f867dfa3740 1 client.19056 shutdown 2024-04-24T09:01:17.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.957+0000 7f867dfa3740 1 -- 172.21.15.83:0/119410066 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561af4b28460 msgr2=0x561af4b2f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:17.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.957+0000 7f867dfa3740 1 --2- 172.21.15.83:0/119410066 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561af4b28460 0x561af4b2f870 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f865c00d740 tx=0x7f865c00de80 comp rx=0 tx=0).stop 2024-04-24T09:01:17.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.957+0000 7f867dfa3740 1 -- 172.21.15.83:0/119410066 shutdown_connections 2024-04-24T09:01:17.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.957+0000 7f867dfa3740 1 --2- 172.21.15.83:0/119410066 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x561af4b42bc0 0x561af4b44fb0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:17.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.957+0000 7f867dfa3740 1 --2- 172.21.15.83:0/119410066 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561af4b32b30 0x561af4b2ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:17.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.957+0000 7f867dfa3740 1 --2- 172.21.15.83:0/119410066 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561af4b28460 0x561af4b2f870 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:17.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.957+0000 7f867dfa3740 1 --2- 172.21.15.83:0/119410066 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561af4b27b40 0x561af4b32580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:17.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.957+0000 7f867dfa3740 1 -- 172.21.15.83:0/119410066 >> 172.21.15.83:0/119410066 conn(0x561af4b23e60 msgr2=0x561af4b29c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:17.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.957+0000 7f8662000640 0 client.19056 ms_handle_reset on v2:172.21.15.138:6832/1157967658 2024-04-24T09:01:17.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.957+0000 7f867dfa3740 1 -- 172.21.15.83:0/119410066 shutdown_connections 2024-04-24T09:01:17.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.957+0000 7f867dfa3740 1 -- 172.21.15.83:0/119410066 wait complete. 2024-04-24T09:01:17.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.957+0000 7f867dfa3740 20 client.19056 trim_cache size 0 max 16384 2024-04-24T09:01:17.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.987+0000 7f867dfa3740 1 Processor -- start 2024-04-24T09:01:17.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.987+0000 7f867dfa3740 1 -- start start 2024-04-24T09:01:17.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.988+0000 7f867dfa3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561af4b27b40 0x561af4b3e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:17.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.988+0000 7f867dfa3740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561af4b28460 0x561af4b320b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:17.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.988+0000 7f867dfa3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561af4b32b30 0x561af4b325f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:17.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.988+0000 7f867dfa3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f86741b1730 con 0x561af4b32b30 2024-04-24T09:01:17.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.988+0000 7f867dfa3740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f86741b3270 con 0x561af4b28460 2024-04-24T09:01:17.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.988+0000 7f867dfa3740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f86741b1e00 con 0x561af4b27b40 2024-04-24T09:01:17.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.988+0000 7f8679000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561af4b32b30 0x561af4b325f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:17.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.988+0000 7f8679000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561af4b32b30 0x561af4b325f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48928/0 (socket says 172.21.15.83:48928) 2024-04-24T09:01:17.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.988+0000 7f8679000640 1 -- 172.21.15.83:0/1846037622 learned_addr learned my addr 172.21.15.83:0/1846037622 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:01:17.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.989+0000 7f8673e00640 1 --2- 172.21.15.83:0/1846037622 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561af4b27b40 0x561af4b3e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:17.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.989+0000 7f866b400640 1 --2- 172.21.15.83:0/1846037622 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561af4b28460 0x561af4b320b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:17.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.989+0000 7f8679000640 1 -- 172.21.15.83:0/1846037622 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561af4b27b40 msgr2=0x561af4b3e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:17.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.989+0000 7f8679000640 1 --2- 172.21.15.83:0/1846037622 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561af4b27b40 0x561af4b3e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:17.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.989+0000 7f8679000640 1 -- 172.21.15.83:0/1846037622 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561af4b28460 msgr2=0x561af4b320b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:17.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.989+0000 7f8679000640 1 --2- 172.21.15.83:0/1846037622 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561af4b28460 0x561af4b320b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:17.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.989+0000 7f8679000640 1 -- 172.21.15.83:0/1846037622 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561af48697d0 con 0x561af4b32b30 2024-04-24T09:01:17.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.989+0000 7f8679000640 1 --2- 172.21.15.83:0/1846037622 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561af4b32b30 0x561af4b325f0 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f866c031840 tx=0x7f866c03d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60e315d8b5d3c5b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:17.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.990+0000 7f8668c00640 1 -- 172.21.15.83:0/1846037622 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f866c04c070 con 0x561af4b32b30 2024-04-24T09:01:17.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.990+0000 7f8668c00640 1 -- 172.21.15.83:0/1846037622 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f866c004440 con 0x561af4b32b30 2024-04-24T09:01:17.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.990+0000 7f8668c00640 1 -- 172.21.15.83:0/1846037622 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f866c035050 con 0x561af4b32b30 2024-04-24T09:01:17.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.991+0000 7f867dfa3740 1 -- 172.21.15.83:0/1846037622 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561af4b32b30 msgr2=0x561af4b325f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:17.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.991+0000 7f867dfa3740 1 --2- 172.21.15.83:0/1846037622 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561af4b32b30 0x561af4b325f0 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f866c031840 tx=0x7f866c03d000 comp rx=0 tx=0).stop 2024-04-24T09:01:17.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.991+0000 7f867dfa3740 1 -- 172.21.15.83:0/1846037622 shutdown_connections 2024-04-24T09:01:17.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.991+0000 7f867dfa3740 1 --2- 172.21.15.83:0/1846037622 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561af4b32b30 0x561af4b325f0 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:17.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.991+0000 7f867dfa3740 1 --2- 172.21.15.83:0/1846037622 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561af4b28460 0x561af4b320b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:17.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.991+0000 7f867dfa3740 1 --2- 172.21.15.83:0/1846037622 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561af4b27b40 0x561af4b3e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:17.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.991+0000 7f867dfa3740 1 -- 172.21.15.83:0/1846037622 >> 172.21.15.83:0/1846037622 conn(0x561af4b23e60 msgr2=0x561af4b3ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:17.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.991+0000 7f867dfa3740 1 -- 172.21.15.83:0/1846037622 shutdown_connections 2024-04-24T09:01:17.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.991+0000 7f867dfa3740 1 -- 172.21.15.83:0/1846037622 wait complete. 2024-04-24T09:01:17.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.992+0000 7f867dfa3740 1 -- start start 2024-04-24T09:01:17.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.992+0000 7f867dfa3740 1 Processor -- start 2024-04-24T09:01:17.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.993+0000 7f867dfa3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561af4b27b40 0x561af4b47db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:17.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.993+0000 7f867dfa3740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561af4b28460 0x561af4b431e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:17.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.993+0000 7f867dfa3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561af4b32b30 0x561af4b43720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:17.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.993+0000 7f867dfa3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f86741b2ba0 con 0x561af4b27b40 2024-04-24T09:01:17.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.993+0000 7f867dfa3740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f86741b24d0 con 0x561af4b28460 2024-04-24T09:01:17.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.993+0000 7f867dfa3740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f86741af730 con 0x561af4b32b30 2024-04-24T09:01:17.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.993+0000 7f867dfa3740 10 client.0 _ll_register_callbacks cb 0x7f865c0d6a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:01:17.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.993+0000 7f8673e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561af4b27b40 0x561af4b47db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:17.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.993+0000 7f8673e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561af4b27b40 0x561af4b47db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48932/0 (socket says 172.21.15.83:48932) 2024-04-24T09:01:17.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.993+0000 7f8673e00640 1 -- 172.21.15.83:0/1239073270 learned_addr learned my addr 172.21.15.83:0/1239073270 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:01:17.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.993+0000 7f866b400640 1 --2- 172.21.15.83:0/1239073270 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561af4b28460 0x561af4b431e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:17.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.993+0000 7f8679000640 1 --2- 172.21.15.83:0/1239073270 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561af4b32b30 0x561af4b43720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:17.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.993+0000 7f8673e00640 1 -- 172.21.15.83:0/1239073270 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561af4b32b30 msgr2=0x561af4b43720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:17.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.993+0000 7f8673e00640 1 --2- 172.21.15.83:0/1239073270 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561af4b32b30 0x561af4b43720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:17.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.993+0000 7f8673e00640 1 -- 172.21.15.83:0/1239073270 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561af4b28460 msgr2=0x561af4b431e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:17.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.993+0000 7f8673e00640 1 --2- 172.21.15.83:0/1239073270 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561af4b28460 0x561af4b431e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:17.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.993+0000 7f8673e00640 1 -- 172.21.15.83:0/1239073270 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561af4b43d60 con 0x561af4b27b40 2024-04-24T09:01:17.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.994+0000 7f8673e00640 1 --2- 172.21.15.83:0/1239073270 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561af4b27b40 0x561af4b47db0 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f8664005fd0 tx=0x7f866401bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9786d7b9cd89339 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:17.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.994+0000 7f8662000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:01:17.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.994+0000 7f8662000640 1 -- 172.21.15.83:0/1239073270 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f866401b880 con 0x561af4b27b40 2024-04-24T09:01:17.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.994+0000 7f8662000640 1 -- 172.21.15.83:0/1239073270 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7f86640135a0 con 0x561af4b27b40 2024-04-24T09:01:17.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.994+0000 7f867dfa3740 1 -- 172.21.15.83:0/1239073270 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f866c004440 con 0x561af4b27b40 2024-04-24T09:01:17.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.995+0000 7f8662000640 1 -- 172.21.15.83:0/1239073270 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f866402d050 con 0x561af4b27b40 2024-04-24T09:01:17.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.995+0000 7f867dfa3740 10 client.19062 fetch_fsmap learned FSMap version 35 2024-04-24T09:01:17.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.995+0000 7f867dfa3740 1 -- 172.21.15.83:0/1239073270 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x561af4ba8c00 con 0x561af4b27b40 2024-04-24T09:01:17.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.995+0000 7f8662000640 1 -- 172.21.15.83:0/1239073270 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f866c004440 con 0x561af4b27b40 2024-04-24T09:01:17.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.995+0000 7f8662000640 1 -- 172.21.15.83:0/1239073270 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7f8664005550 con 0x561af4b27b40 2024-04-24T09:01:17.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.995+0000 7f867dfa3740 10 client.19062 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T09:01:17.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.995+0000 7f867dfa3740 10 client.19062 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-24T09:01:17.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.995+0000 7f867dfa3740 20 client.19062 populate_metadata read hostname 'smithi083' 2024-04-24T09:01:17.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.995+0000 7f867dfa3740 1 --2- 172.21.15.83:0/1239073270 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x561af4ba9360 0x561af4bab780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:17.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.995+0000 7f867dfa3740 4 client.19062 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:01:17.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.995+0000 7f867dfa3740 1 -- 172.21.15.83:0/1239073270 --> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561af4a94a60 con 0x561af4ba9360 2024-04-24T09:01:17.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.996+0000 7f866b400640 1 --2- 172.21.15.83:0/1239073270 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x561af4ba9360 0x561af4bab780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:17.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.997+0000 7f866b400640 1 --2- 172.21.15.83:0/1239073270 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x561af4ba9360 0x561af4bab780 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=54df7215dff99a04 server_cookie=a02ed502c48c8bb8 in_seq=0 out_seq=0 2024-04-24T09:01:17.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.997+0000 7f8662000640 10 client.19062 ms_handle_connect on v2:172.21.15.138:6832/1157967658 2024-04-24T09:01:17.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.998+0000 7f8662000640 1 -- 172.21.15.83:0/1239073270 <== mds.3 v2:172.21.15.138:6832/1157967658 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561af4a94a60 con 0x561af4ba9360 2024-04-24T09:01:17.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.998+0000 7f8662000640 10 client.19062 handle_command_reply: tid=0 2024-04-24T09:01:17.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.998+0000 7f867dfa3740 1 client.19062 shutdown 2024-04-24T09:01:17.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.998+0000 7f867dfa3740 1 -- 172.21.15.83:0/1239073270 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561af4b27b40 msgr2=0x561af4b47db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:17.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.998+0000 7f867dfa3740 1 --2- 172.21.15.83:0/1239073270 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561af4b27b40 0x561af4b47db0 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f8664005fd0 tx=0x7f866401bf90 comp rx=0 tx=0).stop 2024-04-24T09:01:17.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.998+0000 7f867dfa3740 1 -- 172.21.15.83:0/1239073270 shutdown_connections 2024-04-24T09:01:17.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.998+0000 7f867dfa3740 1 --2- 172.21.15.83:0/1239073270 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x561af4ba9360 0x561af4bab780 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:17.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.998+0000 7f867dfa3740 1 --2- 172.21.15.83:0/1239073270 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561af4b32b30 0x561af4b43720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:17.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.998+0000 7f867dfa3740 1 --2- 172.21.15.83:0/1239073270 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561af4b28460 0x561af4b431e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.998+0000 7f8662000640 0 client.19062 ms_handle_reset on v2:172.21.15.138:6832/1157967658 2024-04-24T09:01:18.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.998+0000 7f867dfa3740 1 --2- 172.21.15.83:0/1239073270 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561af4b27b40 0x561af4b47db0 unknown :-1 s=CLOSED pgs=1636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.999+0000 7f867dfa3740 1 -- 172.21.15.83:0/1239073270 >> 172.21.15.83:0/1239073270 conn(0x561af4b23e60 msgr2=0x561af4b2dd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:18.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.999+0000 7f867dfa3740 1 -- 172.21.15.83:0/1239073270 shutdown_connections 2024-04-24T09:01:18.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.999+0000 7f867dfa3740 1 -- 172.21.15.83:0/1239073270 wait complete. 2024-04-24T09:01:18.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:17.999+0000 7f867dfa3740 20 client.19062 trim_cache size 0 max 16384 2024-04-24T09:01:18.000 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:01:18.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.000+0000 7f8679a00640 1 -- 172.21.15.83:0/2369274839 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f86580a3d20 msgr2=0x7f86580a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:18.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.000+0000 7f8679a00640 1 --2- 172.21.15.83:0/2369274839 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f86580a3d20 0x7f86580a61e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f866c005940 tx=0x7f866c0058b0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.000+0000 7f8679a00640 1 -- 172.21.15.83:0/2369274839 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86741071e0 msgr2=0x7f86741a0af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:18.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.000+0000 7f8679a00640 1 --2- 172.21.15.83:0/2369274839 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86741071e0 0x7f86741a0af0 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f865c00d770 tx=0x7f865c00dc30 comp rx=0 tx=0).stop 2024-04-24T09:01:18.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.000+0000 7f8679a00640 1 -- 172.21.15.83:0/2369274839 shutdown_connections 2024-04-24T09:01:18.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.000+0000 7f8679a00640 1 --2- 172.21.15.83:0/2369274839 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f86580a3d20 0x7f86580a61e0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.000+0000 7f8679a00640 1 --2- 172.21.15.83:0/2369274839 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f867419a670 0x7f867419aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.000+0000 7f8679a00640 1 --2- 172.21.15.83:0/2369274839 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86741071e0 0x7f86741a0af0 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.000+0000 7f8679a00640 1 --2- 172.21.15.83:0/2369274839 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8674102110 0x7f86741a05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.000+0000 7f8679a00640 1 -- 172.21.15.83:0/2369274839 >> 172.21.15.83:0/2369274839 conn(0x7f86740fd9d0 msgr2=0x7f86740fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:18.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.000+0000 7f8679a00640 1 -- 172.21.15.83:0/2369274839 shutdown_connections 2024-04-24T09:01:18.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.000+0000 7f8679a00640 1 -- 172.21.15.83:0/2369274839 wait complete. 2024-04-24T09:01:18.017 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:01:17 smithi083 ceph-mon[27907]: pgmap v303: 129 pgs: 129 active+clean; 54 MiB data, 823 MiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 348 KiB/s wr, 63 op/s 2024-04-24T09:01:18.017 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:01:17 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/67821077' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:01:18.020 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T09:01:18.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.220+0000 7f1fbbe00640 1 -- 172.21.15.83:0/499565959 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1fbc102110 msgr2=0x7f1fbc1024f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:18.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.220+0000 7f1fbbe00640 1 --2- 172.21.15.83:0/499565959 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1fbc102110 0x7f1fbc1024f0 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f1fac009900 tx=0x7f1fac02a410 comp rx=0 tx=0).stop 2024-04-24T09:01:18.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.220+0000 7f1fbbe00640 1 -- 172.21.15.83:0/499565959 shutdown_connections 2024-04-24T09:01:18.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.220+0000 7f1fbbe00640 1 --2- 172.21.15.83:0/499565959 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1fbc1071e0 0x7f1fbc10edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.220+0000 7f1fbbe00640 1 --2- 172.21.15.83:0/499565959 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1fbc102ac0 0x7f1fbc106b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.220+0000 7f1fbbe00640 1 --2- 172.21.15.83:0/499565959 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1fbc102110 0x7f1fbc1024f0 unknown :-1 s=CLOSED pgs=1637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.220+0000 7f1fbbe00640 1 -- 172.21.15.83:0/499565959 >> 172.21.15.83:0/499565959 conn(0x7f1fbc0fd9d0 msgr2=0x7f1fbc0ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:18.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.221+0000 7f1fbbe00640 1 -- 172.21.15.83:0/499565959 shutdown_connections 2024-04-24T09:01:18.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.221+0000 7f1fbbe00640 1 -- 172.21.15.83:0/499565959 wait complete. 2024-04-24T09:01:18.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.222+0000 7f1fbbe00640 1 Processor -- start 2024-04-24T09:01:18.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.222+0000 7f1fbbe00640 1 -- start start 2024-04-24T09:01:18.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.222+0000 7f1fbbe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1fbc102110 0x7f1fbc1a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:18.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.222+0000 7f1fbbe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1fbc1071e0 0x7f1fbc1a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:18.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.222+0000 7f1fbbe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1fbc19a770 0x7f1fbc19abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:18.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.222+0000 7f1fbbe00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1fbc112190 con 0x7f1fbc19a770 2024-04-24T09:01:18.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.222+0000 7f1fbbe00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1fbc112010 con 0x7f1fbc102110 2024-04-24T09:01:18.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.222+0000 7f1fbbe00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1fbc112310 con 0x7f1fbc1071e0 2024-04-24T09:01:18.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.223+0000 7f1fbb400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1fbc19a770 0x7f1fbc19abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:18.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.223+0000 7f1fbb400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1fbc19a770 0x7f1fbc19abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48954/0 (socket says 172.21.15.83:48954) 2024-04-24T09:01:18.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.223+0000 7f1fbb400640 1 -- 172.21.15.83:0/3618483264 learned_addr learned my addr 172.21.15.83:0/3618483264 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:01:18.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.223+0000 7f1fb3e00640 1 --2- 172.21.15.83:0/3618483264 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1fbc1071e0 0x7f1fbc1a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:18.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.223+0000 7f1fbaa00640 1 --2- 172.21.15.83:0/3618483264 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1fbc102110 0x7f1fbc1a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:18.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.224+0000 7f1fbb400640 1 -- 172.21.15.83:0/3618483264 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1fbc1071e0 msgr2=0x7f1fbc1a0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:18.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.224+0000 7f1fbb400640 1 --2- 172.21.15.83:0/3618483264 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1fbc1071e0 0x7f1fbc1a0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.224+0000 7f1fbb400640 1 -- 172.21.15.83:0/3618483264 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1fbc102110 msgr2=0x7f1fbc1a06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:18.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.224+0000 7f1fbb400640 1 --2- 172.21.15.83:0/3618483264 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1fbc102110 0x7f1fbc1a06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.224+0000 7f1fbb400640 1 -- 172.21.15.83:0/3618483264 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fbc19b370 con 0x7f1fbc19a770 2024-04-24T09:01:18.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.224+0000 7f1fbaa00640 1 --2- 172.21.15.83:0/3618483264 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1fbc102110 0x7f1fbc1a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:01:18.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.224+0000 7f1fbb400640 1 --2- 172.21.15.83:0/3618483264 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1fbc19a770 0x7f1fbc19abf0 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f1fb400eaa0 tx=0x7f1fb400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:18.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.224+0000 7f1fb3400640 1 -- 172.21.15.83:0/3618483264 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb400cdb0 con 0x7f1fbc19a770 2024-04-24T09:01:18.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.224+0000 7f1fb3400640 1 -- 172.21.15.83:0/3618483264 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1fb4005020 con 0x7f1fbc19a770 2024-04-24T09:01:18.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.224+0000 7f1fbbe00640 1 -- 172.21.15.83:0/3618483264 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fbc108d00 con 0x7f1fbc19a770 2024-04-24T09:01:18.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.224+0000 7f1fb3400640 1 -- 172.21.15.83:0/3618483264 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb40116d0 con 0x7f1fbc19a770 2024-04-24T09:01:18.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.224+0000 7f1fbbe00640 1 -- 172.21.15.83:0/3618483264 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fbc109200 con 0x7f1fbc19a770 2024-04-24T09:01:18.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.228+0000 7f1fb3400640 1 -- 172.21.15.83:0/3618483264 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1fb40051d0 con 0x7f1fbc19a770 2024-04-24T09:01:18.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.228+0000 7f1fc43c9740 1 Processor -- start 2024-04-24T09:01:18.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.228+0000 7f1fc43c9740 1 -- start start 2024-04-24T09:01:18.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.229+0000 7f1fc43c9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b756f73b40 0x55b756f73f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:18.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.229+0000 7f1fc43c9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b756f74460 0x55b756f7e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:18.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.229+0000 7f1fc43c9740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b756f7eb30 0x55b756f80f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:18.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.229+0000 7f1fc43c9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55b756ed91d0 con 0x55b756f73b40 2024-04-24T09:01:18.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.229+0000 7f1fc43c9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b756e7f450 con 0x55b756f74460 2024-04-24T09:01:18.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.229+0000 7f1fc43c9740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55b756ed5190 con 0x55b756f7eb30 2024-04-24T09:01:18.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.229+0000 7f1fb3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b756f73b40 0x55b756f73f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:18.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.229+0000 7f1fb3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b756f73b40 0x55b756f73f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48962/0 (socket says 172.21.15.83:48962) 2024-04-24T09:01:18.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.229+0000 7f1fb3e00640 1 -- 172.21.15.83:0/1023436983 learned_addr learned my addr 172.21.15.83:0/1023436983 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:01:18.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.230+0000 7f1fbaa00640 1 --2- 172.21.15.83:0/1023436983 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b756f74460 0x55b756f7e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:18.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.230+0000 7f1fb3e00640 1 --2- 172.21.15.83:0/1023436983 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b756f7eb30 0x55b756f80f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:18.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.230+0000 7f1fb3e00640 1 -- 172.21.15.83:0/1023436983 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b756f7eb30 msgr2=0x55b756f80f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:18.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.230+0000 7f1fb3e00640 1 --2- 172.21.15.83:0/1023436983 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b756f7eb30 0x55b756f80f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.230+0000 7f1fb3e00640 1 -- 172.21.15.83:0/1023436983 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b756f74460 msgr2=0x55b756f7e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:18.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.230+0000 7f1fb3e00640 1 --2- 172.21.15.83:0/1023436983 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b756f74460 0x55b756f7e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.230+0000 7f1fb3e00640 1 -- 172.21.15.83:0/1023436983 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b756f3c310 con 0x55b756f73b40 2024-04-24T09:01:18.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.230+0000 7f1fb3400640 1 --2- 172.21.15.83:0/3618483264 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f1fa00a3d20 0x7f1fa00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:18.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.230+0000 7f1fb3400640 1 -- 172.21.15.83:0/3618483264 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7f1fb40d0e00 con 0x7f1fbc19a770 2024-04-24T09:01:18.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.230+0000 7f1fb3e00640 1 --2- 172.21.15.83:0/1023436983 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b756f73b40 0x55b756f73f20 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f1fa400be60 tx=0x7f1fa400b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d741e184ef99939 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:18.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.230+0000 7f1fbaa00640 1 --2- 172.21.15.83:0/3618483264 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f1fa00a3d20 0x7f1fa00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:18.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.230+0000 7f1fab400640 1 -- 172.21.15.83:0/1023436983 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa4005220 con 0x55b756f73b40 2024-04-24T09:01:18.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.231+0000 7f1fab400640 1 -- 172.21.15.83:0/1023436983 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1fa4004c50 con 0x55b756f73b40 2024-04-24T09:01:18.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.231+0000 7f1fab400640 1 -- 172.21.15.83:0/1023436983 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa4010790 con 0x55b756f73b40 2024-04-24T09:01:18.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.231+0000 7f1fc43c9740 1 -- 172.21.15.83:0/1023436983 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b756f73b40 msgr2=0x55b756f73f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:18.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.231+0000 7f1fbaa00640 1 --2- 172.21.15.83:0/3618483264 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f1fa00a3d20 0x7f1fa00a61e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1fac000c00 tx=0x7f1fac005a40 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:18.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.231+0000 7f1fc43c9740 1 --2- 172.21.15.83:0/1023436983 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b756f73b40 0x55b756f73f20 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f1fa400be60 tx=0x7f1fa400b820 comp rx=0 tx=0).stop 2024-04-24T09:01:18.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.232+0000 7f1fc43c9740 1 -- 172.21.15.83:0/1023436983 shutdown_connections 2024-04-24T09:01:18.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.232+0000 7f1fc43c9740 1 --2- 172.21.15.83:0/1023436983 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b756f7eb30 0x55b756f80f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.232+0000 7f1fc43c9740 1 --2- 172.21.15.83:0/1023436983 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b756f74460 0x55b756f7e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.232+0000 7f1fc43c9740 1 --2- 172.21.15.83:0/1023436983 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b756f73b40 0x55b756f73f20 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.232+0000 7f1fc43c9740 1 -- 172.21.15.83:0/1023436983 >> 172.21.15.83:0/1023436983 conn(0x55b756f6fe60 msgr2=0x55b756f70240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:18.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.232+0000 7f1fc43c9740 1 -- 172.21.15.83:0/1023436983 shutdown_connections 2024-04-24T09:01:18.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.232+0000 7f1fc43c9740 1 -- 172.21.15.83:0/1023436983 wait complete. 2024-04-24T09:01:18.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.232+0000 7f1fc43c9740 1 -- start start 2024-04-24T09:01:18.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.232+0000 7f1fc43c9740 1 Processor -- start 2024-04-24T09:01:18.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.233+0000 7f1fc43c9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b756f73b40 0x55b756f7b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:18.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.233+0000 7f1fc43c9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b756f74460 0x55b756f7be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:18.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.233+0000 7f1fc43c9740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b756f7eb30 0x55b756f7c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:18.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.233+0000 7f1fc43c9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55b756f819c0 con 0x55b756f73b40 2024-04-24T09:01:18.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.233+0000 7f1fc43c9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b756e92810 con 0x55b756f74460 2024-04-24T09:01:18.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.233+0000 7f1fc43c9740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55b756f81b40 con 0x55b756f7eb30 2024-04-24T09:01:18.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.233+0000 7f1fc43c9740 10 client.0 _ll_register_callbacks cb 0x55b756e60140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:01:18.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.233+0000 7f1fbaa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b756f73b40 0x55b756f7b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:18.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.233+0000 7f1fbaa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b756f73b40 0x55b756f7b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48970/0 (socket says 172.21.15.83:48970) 2024-04-24T09:01:18.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.233+0000 7f1fbaa00640 1 -- 172.21.15.83:0/3666942652 learned_addr learned my addr 172.21.15.83:0/3666942652 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:01:18.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.233+0000 7f1fbb400640 1 --2- 172.21.15.83:0/3666942652 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b756f7eb30 0x55b756f7c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:18.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.234+0000 7f1fb3e00640 1 --2- 172.21.15.83:0/3666942652 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b756f74460 0x55b756f7be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:18.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.234+0000 7f1fbaa00640 1 -- 172.21.15.83:0/3666942652 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b756f7eb30 msgr2=0x55b756f7c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:18.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.234+0000 7f1fbaa00640 1 --2- 172.21.15.83:0/3666942652 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b756f7eb30 0x55b756f7c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.234+0000 7f1fbaa00640 1 -- 172.21.15.83:0/3666942652 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b756f74460 msgr2=0x55b756f7be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:18.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.234+0000 7f1fbaa00640 1 --2- 172.21.15.83:0/3666942652 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b756f74460 0x55b756f7be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.234+0000 7f1fbaa00640 1 -- 172.21.15.83:0/3666942652 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b756ea33f0 con 0x55b756f73b40 2024-04-24T09:01:18.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.234+0000 7f1fb3e00640 1 --2- 172.21.15.83:0/3666942652 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b756f74460 0x55b756f7be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:01:18.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.234+0000 7f1fbaa00640 1 --2- 172.21.15.83:0/3666942652 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b756f73b40 0x55b756f7b8f0 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f1fac005ab0 tx=0x7f1fac009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2810ba0ba40a5638 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:18.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.234+0000 7f1f9fe00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:01:18.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.235+0000 7f1f9fe00640 1 -- 172.21.15.83:0/3666942652 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fac04c070 con 0x55b756f73b40 2024-04-24T09:01:18.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.235+0000 7f1f9fe00640 1 -- 172.21.15.83:0/3666942652 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7f1fac004c10 con 0x55b756f73b40 2024-04-24T09:01:18.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.235+0000 7f1fc43c9740 1 -- 172.21.15.83:0/3666942652 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1fa4004c50 con 0x55b756f73b40 2024-04-24T09:01:18.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.235+0000 7f1f9fe00640 1 -- 172.21.15.83:0/3666942652 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f1fac058050 con 0x55b756f73b40 2024-04-24T09:01:18.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.235+0000 7f1fc43c9740 10 client.19074 fetch_fsmap learned FSMap version 35 2024-04-24T09:01:18.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.235+0000 7f1fc43c9740 1 -- 172.21.15.83:0/3666942652 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55b756beb3f0 con 0x55b756f73b40 2024-04-24T09:01:18.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.235+0000 7f1f9fe00640 1 -- 172.21.15.83:0/3666942652 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1fa4004c50 con 0x55b756f73b40 2024-04-24T09:01:18.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.236+0000 7f1f9fe00640 1 -- 172.21.15.83:0/3666942652 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7f1fac032600 con 0x55b756f73b40 2024-04-24T09:01:18.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.236+0000 7f1fc43c9740 10 client.19074 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T09:01:18.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.236+0000 7f1fc43c9740 10 client.19074 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-24T09:01:18.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.236+0000 7f1fc43c9740 20 client.19074 populate_metadata read hostname 'smithi083' 2024-04-24T09:01:18.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.236+0000 7f1fc43c9740 1 --2- 172.21.15.83:0/3666942652 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55b756f8ed10 0x55b756f911b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:18.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.236+0000 7f1fc43c9740 4 client.19074 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:01:18.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.236+0000 7f1fc43c9740 1 -- 172.21.15.83:0/3666942652 --> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b756ee6bf0 con 0x55b756f8ed10 2024-04-24T09:01:18.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.236+0000 7f1fb3e00640 1 --2- 172.21.15.83:0/3666942652 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55b756f8ed10 0x55b756f911b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:18.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.238+0000 7f1f9fe00640 10 client.19074 ms_handle_connect on v2:172.21.15.138:6834/1909219758 2024-04-24T09:01:18.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.238+0000 7f1fb3e00640 1 --2- 172.21.15.83:0/3666942652 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55b756f8ed10 0x55b756f911b0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1535ef75bb20f185 server_cookie=a9662cae4068af3f in_seq=0 out_seq=0 2024-04-24T09:01:18.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.240+0000 7f1f9fe00640 1 -- 172.21.15.83:0/3666942652 <== mds.2 v2:172.21.15.138:6834/1909219758 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b756ee6bf0 con 0x55b756f8ed10 2024-04-24T09:01:18.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.240+0000 7f1f9fe00640 10 client.19074 handle_command_reply: tid=0 2024-04-24T09:01:18.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.240+0000 7f1fc43c9740 1 client.19074 shutdown 2024-04-24T09:01:18.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.240+0000 7f1fc43c9740 1 -- 172.21.15.83:0/3666942652 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b756f73b40 msgr2=0x55b756f7b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:18.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.240+0000 7f1fc43c9740 1 --2- 172.21.15.83:0/3666942652 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b756f73b40 0x55b756f7b8f0 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f1fac005ab0 tx=0x7f1fac009260 comp rx=0 tx=0).stop 2024-04-24T09:01:18.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.240+0000 7f1fc43c9740 1 -- 172.21.15.83:0/3666942652 shutdown_connections 2024-04-24T09:01:18.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.240+0000 7f1fc43c9740 1 --2- 172.21.15.83:0/3666942652 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55b756f8ed10 0x55b756f911b0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.240+0000 7f1fc43c9740 1 --2- 172.21.15.83:0/3666942652 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b756f7eb30 0x55b756f7c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.240+0000 7f1fc43c9740 1 --2- 172.21.15.83:0/3666942652 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b756f74460 0x55b756f7be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.240+0000 7f1fc43c9740 1 --2- 172.21.15.83:0/3666942652 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b756f73b40 0x55b756f7b8f0 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.240+0000 7f1f9fe00640 0 client.19074 ms_handle_reset on v2:172.21.15.138:6834/1909219758 2024-04-24T09:01:18.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.240+0000 7f1fc43c9740 1 -- 172.21.15.83:0/3666942652 >> 172.21.15.83:0/3666942652 conn(0x55b756f6fe60 msgr2=0x55b756f75c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:18.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.240+0000 7f1fc43c9740 1 -- 172.21.15.83:0/3666942652 shutdown_connections 2024-04-24T09:01:18.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.240+0000 7f1fc43c9740 1 -- 172.21.15.83:0/3666942652 wait complete. 2024-04-24T09:01:18.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.240+0000 7f1fc43c9740 20 client.19074 trim_cache size 0 max 16384 2024-04-24T09:01:18.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.258+0000 7f1fc43c9740 1 Processor -- start 2024-04-24T09:01:18.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.258+0000 7f1fc43c9740 1 -- start start 2024-04-24T09:01:18.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.259+0000 7f1fc43c9740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b756f73b40 0x55b756f7e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:18.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.259+0000 7f1fc43c9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b756f74460 0x55b756f8a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:18.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.259+0000 7f1fc43c9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b756f7eb30 0x55b756f96a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:18.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.259+0000 7f1fc43c9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1fbc1b59b0 con 0x55b756f7eb30 2024-04-24T09:01:18.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.259+0000 7f1fc43c9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1fbc1b74f0 con 0x55b756f74460 2024-04-24T09:01:18.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.259+0000 7f1fc43c9740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1fbc1b6080 con 0x55b756f73b40 2024-04-24T09:01:18.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.259+0000 7f1fbaa00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b756f73b40 0x55b756f7e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:18.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.259+0000 7f1fbaa00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b756f73b40 0x55b756f7e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:48472/0 (socket says 172.21.15.83:48472) 2024-04-24T09:01:18.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.259+0000 7f1fbaa00640 1 -- 172.21.15.83:0/4274194747 learned_addr learned my addr 172.21.15.83:0/4274194747 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:01:18.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.260+0000 7f1fb3e00640 1 --2- 172.21.15.83:0/4274194747 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b756f74460 0x55b756f8a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:18.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.260+0000 7f1fbb400640 1 --2- 172.21.15.83:0/4274194747 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b756f7eb30 0x55b756f96a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:18.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.260+0000 7f1fbaa00640 1 -- 172.21.15.83:0/4274194747 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b756f74460 msgr2=0x55b756f8a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:18.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.260+0000 7f1fbaa00640 1 --2- 172.21.15.83:0/4274194747 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b756f74460 0x55b756f8a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.260+0000 7f1fbaa00640 1 -- 172.21.15.83:0/4274194747 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b756f7eb30 msgr2=0x55b756f96a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:18.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.260+0000 7f1fbaa00640 1 --2- 172.21.15.83:0/4274194747 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b756f7eb30 0x55b756f96a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.260+0000 7f1fbaa00640 1 -- 172.21.15.83:0/4274194747 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b756cb57d0 con 0x55b756f73b40 2024-04-24T09:01:18.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.260+0000 7f1fbb400640 1 --2- 172.21.15.83:0/4274194747 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b756f7eb30 0x55b756f96a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:01:18.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.261+0000 7f1fbaa00640 1 --2- 172.21.15.83:0/4274194747 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b756f73b40 0x55b756f7e130 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1fac032a50 tx=0x7f1fac004220 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e90ccb7ce9eac60 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:18.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.262+0000 7f1faaa00640 1 -- 172.21.15.83:0/4274194747 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fac060070 con 0x55b756f73b40 2024-04-24T09:01:18.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.262+0000 7f1faaa00640 1 -- 172.21.15.83:0/4274194747 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1fac004a10 con 0x55b756f73b40 2024-04-24T09:01:18.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.262+0000 7f1faaa00640 1 -- 172.21.15.83:0/4274194747 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fac04c050 con 0x55b756f73b40 2024-04-24T09:01:18.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.263+0000 7f1fc43c9740 1 -- 172.21.15.83:0/4274194747 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b756f73b40 msgr2=0x55b756f7e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:18.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.263+0000 7f1fc43c9740 1 --2- 172.21.15.83:0/4274194747 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b756f73b40 0x55b756f7e130 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1fac032a50 tx=0x7f1fac004220 comp rx=0 tx=0).stop 2024-04-24T09:01:18.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.263+0000 7f1fc43c9740 1 -- 172.21.15.83:0/4274194747 shutdown_connections 2024-04-24T09:01:18.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.263+0000 7f1fc43c9740 1 --2- 172.21.15.83:0/4274194747 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b756f7eb30 0x55b756f96a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.263+0000 7f1fc43c9740 1 --2- 172.21.15.83:0/4274194747 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b756f74460 0x55b756f8a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.263+0000 7f1fc43c9740 1 --2- 172.21.15.83:0/4274194747 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b756f73b40 0x55b756f7e130 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.263+0000 7f1fc43c9740 1 -- 172.21.15.83:0/4274194747 >> 172.21.15.83:0/4274194747 conn(0x55b756f6fe60 msgr2=0x55b756f6ef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:18.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.263+0000 7f1fc43c9740 1 -- 172.21.15.83:0/4274194747 shutdown_connections 2024-04-24T09:01:18.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.263+0000 7f1fc43c9740 1 -- 172.21.15.83:0/4274194747 wait complete. 2024-04-24T09:01:18.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.263+0000 7f1fc43c9740 1 -- start start 2024-04-24T09:01:18.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.264+0000 7f1fc43c9740 1 Processor -- start 2024-04-24T09:01:18.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.264+0000 7f1fc43c9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b756f73b40 0x55b756f7f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:18.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.264+0000 7f1fc43c9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b756f74460 0x55b756f7fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:18.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.264+0000 7f1fc43c9740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b756f7eb30 0x55b756f802c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:18.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.264+0000 7f1fc43c9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1fbc1b6e20 con 0x55b756f73b40 2024-04-24T09:01:18.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.264+0000 7f1fc43c9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1fbc1b6750 con 0x55b756f74460 2024-04-24T09:01:18.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.264+0000 7f1fc43c9740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1fbc1b39d0 con 0x55b756f7eb30 2024-04-24T09:01:18.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.264+0000 7f1fc43c9740 10 client.0 _ll_register_callbacks cb 0x7f1fac009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:01:18.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.265+0000 7f1fbaa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b756f73b40 0x55b756f7f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:18.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.265+0000 7f1fbaa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b756f73b40 0x55b756f7f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48986/0 (socket says 172.21.15.83:48986) 2024-04-24T09:01:18.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.265+0000 7f1fbaa00640 1 -- 172.21.15.83:0/3526203107 learned_addr learned my addr 172.21.15.83:0/3526203107 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:01:18.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.265+0000 7f1fbb400640 1 --2- 172.21.15.83:0/3526203107 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b756f7eb30 0x55b756f802c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:18.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.266+0000 7f1fb3e00640 1 --2- 172.21.15.83:0/3526203107 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b756f74460 0x55b756f7fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:18.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.266+0000 7f1fbaa00640 1 -- 172.21.15.83:0/3526203107 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b756f7eb30 msgr2=0x55b756f802c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:18.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.266+0000 7f1fbaa00640 1 --2- 172.21.15.83:0/3526203107 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b756f7eb30 0x55b756f802c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.266+0000 7f1fbaa00640 1 -- 172.21.15.83:0/3526203107 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b756f74460 msgr2=0x55b756f7fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:18.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.266+0000 7f1fbaa00640 1 --2- 172.21.15.83:0/3526203107 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b756f74460 0x55b756f7fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.266+0000 7f1fbaa00640 1 -- 172.21.15.83:0/3526203107 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b756f84f40 con 0x55b756f73b40 2024-04-24T09:01:18.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.266+0000 7f1fbaa00640 1 --2- 172.21.15.83:0/3526203107 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b756f73b40 0x55b756f7f840 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f1fac006010 tx=0x7f1fac004ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5063724e2075b1b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:18.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.266+0000 7f1f9fe00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:01:18.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.266+0000 7f1f9fe00640 1 -- 172.21.15.83:0/3526203107 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fac069020 con 0x55b756f73b40 2024-04-24T09:01:18.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.266+0000 7f1f9fe00640 1 -- 172.21.15.83:0/3526203107 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7f1fac0035f0 con 0x55b756f73b40 2024-04-24T09:01:18.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.267+0000 7f1fc43c9740 1 -- 172.21.15.83:0/3526203107 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1fac004a10 con 0x55b756f73b40 2024-04-24T09:01:18.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.267+0000 7f1f9fe00640 1 -- 172.21.15.83:0/3526203107 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f1fac074050 con 0x55b756f73b40 2024-04-24T09:01:18.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.267+0000 7f1fc43c9740 10 client.19080 fetch_fsmap learned FSMap version 35 2024-04-24T09:01:18.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.267+0000 7f1fc43c9740 1 -- 172.21.15.83:0/3526203107 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55b756ff5550 con 0x55b756f73b40 2024-04-24T09:01:18.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.267+0000 7f1f9fe00640 1 -- 172.21.15.83:0/3526203107 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1fac004a10 con 0x55b756f73b40 2024-04-24T09:01:18.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.267+0000 7f1f9fe00640 1 -- 172.21.15.83:0/3526203107 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7f1fac0487f0 con 0x55b756f73b40 2024-04-24T09:01:18.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.268+0000 7f1fc43c9740 10 client.19080 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T09:01:18.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.268+0000 7f1fc43c9740 10 client.19080 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-24T09:01:18.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.268+0000 7f1fc43c9740 20 client.19080 populate_metadata read hostname 'smithi083' 2024-04-24T09:01:18.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.268+0000 7f1fc43c9740 1 --2- 172.21.15.83:0/3526203107 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55b756ff5bb0 0x55b756ff7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:18.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.268+0000 7f1fc43c9740 4 client.19080 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:01:18.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.268+0000 7f1fc43c9740 1 -- 172.21.15.83:0/3526203107 --> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b756ee0a60 con 0x55b756ff5bb0 2024-04-24T09:01:18.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.268+0000 7f1fb3e00640 1 --2- 172.21.15.83:0/3526203107 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55b756ff5bb0 0x55b756ff7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:18.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.269+0000 7f1fb3e00640 1 --2- 172.21.15.83:0/3526203107 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55b756ff5bb0 0x55b756ff7fd0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2557fccb51cb9d48 server_cookie=f0e131355d4a632f in_seq=0 out_seq=0 2024-04-24T09:01:18.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.269+0000 7f1f9fe00640 10 client.19080 ms_handle_connect on v2:172.21.15.138:6834/1909219758 2024-04-24T09:01:18.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.270+0000 7f1f9fe00640 1 -- 172.21.15.83:0/3526203107 <== mds.2 v2:172.21.15.138:6834/1909219758 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b756ee0a60 con 0x55b756ff5bb0 2024-04-24T09:01:18.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.270+0000 7f1f9fe00640 10 client.19080 handle_command_reply: tid=0 2024-04-24T09:01:18.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.270+0000 7f1fc43c9740 1 client.19080 shutdown 2024-04-24T09:01:18.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.270+0000 7f1fc43c9740 1 -- 172.21.15.83:0/3526203107 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b756f73b40 msgr2=0x55b756f7f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:18.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.270+0000 7f1fc43c9740 1 --2- 172.21.15.83:0/3526203107 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b756f73b40 0x55b756f7f840 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f1fac006010 tx=0x7f1fac004ef0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.270+0000 7f1fc43c9740 1 -- 172.21.15.83:0/3526203107 shutdown_connections 2024-04-24T09:01:18.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.270+0000 7f1fc43c9740 1 --2- 172.21.15.83:0/3526203107 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55b756ff5bb0 0x55b756ff7fd0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.270+0000 7f1fc43c9740 1 --2- 172.21.15.83:0/3526203107 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b756f7eb30 0x55b756f802c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.270+0000 7f1fc43c9740 1 --2- 172.21.15.83:0/3526203107 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b756f74460 0x55b756f7fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.270+0000 7f1fc43c9740 1 --2- 172.21.15.83:0/3526203107 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b756f73b40 0x55b756f7f840 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.270+0000 7f1fc43c9740 1 -- 172.21.15.83:0/3526203107 >> 172.21.15.83:0/3526203107 conn(0x55b756f6fe60 msgr2=0x55b756f6ef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:18.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.270+0000 7f1f9fe00640 0 client.19080 ms_handle_reset on v2:172.21.15.138:6834/1909219758 2024-04-24T09:01:18.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.271+0000 7f1fc43c9740 1 -- 172.21.15.83:0/3526203107 shutdown_connections 2024-04-24T09:01:18.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.271+0000 7f1fc43c9740 1 -- 172.21.15.83:0/3526203107 wait complete. 2024-04-24T09:01:18.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.271+0000 7f1fc43c9740 20 client.19080 trim_cache size 0 max 16384 2024-04-24T09:01:18.272 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:01:18.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.271+0000 7f1fbbe00640 1 -- 172.21.15.83:0/3618483264 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f1fa00a3d20 msgr2=0x7f1fa00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:18.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.271+0000 7f1fbbe00640 1 --2- 172.21.15.83:0/3618483264 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f1fa00a3d20 0x7f1fa00a61e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1fac000c00 tx=0x7f1fac005a40 comp rx=0 tx=0).stop 2024-04-24T09:01:18.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.271+0000 7f1fbbe00640 1 -- 172.21.15.83:0/3618483264 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1fbc19a770 msgr2=0x7f1fbc19abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:18.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.271+0000 7f1fbbe00640 1 --2- 172.21.15.83:0/3618483264 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1fbc19a770 0x7f1fbc19abf0 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f1fb400eaa0 tx=0x7f1fb400ef60 comp rx=0 tx=0).stop 2024-04-24T09:01:18.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.272+0000 7f1fbbe00640 1 -- 172.21.15.83:0/3618483264 shutdown_connections 2024-04-24T09:01:18.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.272+0000 7f1fbbe00640 1 --2- 172.21.15.83:0/3618483264 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f1fa00a3d20 0x7f1fa00a61e0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.272+0000 7f1fbbe00640 1 --2- 172.21.15.83:0/3618483264 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1fbc19a770 0x7f1fbc19abf0 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.272+0000 7f1fbbe00640 1 --2- 172.21.15.83:0/3618483264 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1fbc1071e0 0x7f1fbc1a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.272+0000 7f1fbbe00640 1 --2- 172.21.15.83:0/3618483264 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1fbc102110 0x7f1fbc1a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.272+0000 7f1fbbe00640 1 -- 172.21.15.83:0/3618483264 >> 172.21.15.83:0/3618483264 conn(0x7f1fbc0fd9d0 msgr2=0x7f1fbc0ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:18.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.272+0000 7f1fbbe00640 1 -- 172.21.15.83:0/3618483264 shutdown_connections 2024-04-24T09:01:18.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.272+0000 7f1fbbe00640 1 -- 172.21.15.83:0/3618483264 wait complete. 2024-04-24T09:01:18.289 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T09:01:18.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:01:17 smithi138 ceph-mon[31441]: pgmap v303: 129 pgs: 129 active+clean; 54 MiB data, 823 MiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 348 KiB/s wr, 63 op/s 2024-04-24T09:01:18.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:01:17 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/67821077' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:01:18.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:01:17 smithi189 ceph-mon[31791]: pgmap v303: 129 pgs: 129 active+clean; 54 MiB data, 823 MiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 348 KiB/s wr, 63 op/s 2024-04-24T09:01:18.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:01:17 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/67821077' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:01:18.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.493+0000 7fc049000640 1 -- 172.21.15.83:0/1223843625 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc04410e7f0 msgr2=0x7fc044110be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:18.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.493+0000 7fc049000640 1 --2- 172.21.15.83:0/1223843625 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc04410e7f0 0x7fc044110be0 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7fc03c009ca0 tx=0x7fc03c02a8c0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.493+0000 7fc049000640 1 -- 172.21.15.83:0/1223843625 shutdown_connections 2024-04-24T09:01:18.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.493+0000 7fc049000640 1 --2- 172.21.15.83:0/1223843625 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc04410e7f0 0x7fc044110be0 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.493+0000 7fc049000640 1 --2- 172.21.15.83:0/1223843625 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0441015f0 0x7fc04410e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.493+0000 7fc049000640 1 --2- 172.21.15.83:0/1223843625 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc044100c40 0x7fc044101020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.493+0000 7fc049000640 1 -- 172.21.15.83:0/1223843625 >> 172.21.15.83:0/1223843625 conn(0x7fc0440fc880 msgr2=0x7fc0440feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:18.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.493+0000 7fc049000640 1 -- 172.21.15.83:0/1223843625 shutdown_connections 2024-04-24T09:01:18.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.493+0000 7fc049000640 1 -- 172.21.15.83:0/1223843625 wait complete. 2024-04-24T09:01:18.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.495+0000 7fc049000640 1 Processor -- start 2024-04-24T09:01:18.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.495+0000 7fc049000640 1 -- start start 2024-04-24T09:01:18.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.496+0000 7fc049000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc044100c40 0x7fc0441a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:18.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.496+0000 7fc049000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc0441015f0 0x7fc0441a0b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:18.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.496+0000 7fc049000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc04410e7f0 0x7fc04419a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:18.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.496+0000 7fc049000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc044114310 con 0x7fc04410e7f0 2024-04-24T09:01:18.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.496+0000 7fc049000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc044114190 con 0x7fc044100c40 2024-04-24T09:01:18.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.496+0000 7fc049000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc044114490 con 0x7fc0441015f0 2024-04-24T09:01:18.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.496+0000 7fc043e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc04410e7f0 0x7fc04419a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:18.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.496+0000 7fc043e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc04410e7f0 0x7fc04419a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49002/0 (socket says 172.21.15.83:49002) 2024-04-24T09:01:18.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.496+0000 7fc043e00640 1 -- 172.21.15.83:0/2256641293 learned_addr learned my addr 172.21.15.83:0/2256641293 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:01:18.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.496+0000 7fc043400640 1 --2- 172.21.15.83:0/2256641293 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc044100c40 0x7fc0441a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:18.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.496+0000 7fc042a00640 1 --2- 172.21.15.83:0/2256641293 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc0441015f0 0x7fc0441a0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:18.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.496+0000 7fc043e00640 1 -- 172.21.15.83:0/2256641293 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc0441015f0 msgr2=0x7fc0441a0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:18.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.497+0000 7fc043e00640 1 --2- 172.21.15.83:0/2256641293 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc0441015f0 0x7fc0441a0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.497+0000 7fc043e00640 1 -- 172.21.15.83:0/2256641293 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc044100c40 msgr2=0x7fc0441a05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:18.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.497+0000 7fc043e00640 1 --2- 172.21.15.83:0/2256641293 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc044100c40 0x7fc0441a05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.497+0000 7fc043e00640 1 -- 172.21.15.83:0/2256641293 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc04419afc0 con 0x7fc04410e7f0 2024-04-24T09:01:18.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.497+0000 7fc043400640 1 --2- 172.21.15.83:0/2256641293 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc044100c40 0x7fc0441a05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:01:18.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.497+0000 7fc042a00640 1 --2- 172.21.15.83:0/2256641293 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc0441015f0 0x7fc0441a0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:01:18.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.497+0000 7fc043e00640 1 --2- 172.21.15.83:0/2256641293 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc04410e7f0 0x7fc04419a700 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7fc03c0098d0 tx=0x7fc03c032ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:18.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.497+0000 7fc03be00640 1 -- 172.21.15.83:0/2256641293 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc03c003d30 con 0x7fc04410e7f0 2024-04-24T09:01:18.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.497+0000 7fc03be00640 1 -- 172.21.15.83:0/2256641293 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc03c0044f0 con 0x7fc04410e7f0 2024-04-24T09:01:18.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.497+0000 7fc049000640 1 -- 172.21.15.83:0/2256641293 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc04419b250 con 0x7fc04410e7f0 2024-04-24T09:01:18.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.497+0000 7fc03be00640 1 -- 172.21.15.83:0/2256641293 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc03c030c30 con 0x7fc04410e7f0 2024-04-24T09:01:18.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.498+0000 7fc049000640 1 -- 172.21.15.83:0/2256641293 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0441a7570 con 0x7fc04410e7f0 2024-04-24T09:01:18.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.501+0000 7fc03be00640 1 -- 172.21.15.83:0/2256641293 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc03c0046a0 con 0x7fc04410e7f0 2024-04-24T09:01:18.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.502+0000 7fc04d44c740 1 Processor -- start 2024-04-24T09:01:18.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.502+0000 7fc04d44c740 1 -- start start 2024-04-24T09:01:18.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.503+0000 7fc04d44c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56527663bb40 0x56527663bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:18.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.503+0000 7fc04d44c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56527663c460 0x5652766465f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:18.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.503+0000 7fc04d44c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565276646b30 0x565276648f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:18.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.503+0000 7fc04d44c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5652765a11d0 con 0x565276646b30 2024-04-24T09:01:18.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.503+0000 7fc04d44c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x565276547450 con 0x56527663c460 2024-04-24T09:01:18.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.503+0000 7fc04d44c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56527659d190 con 0x56527663bb40 2024-04-24T09:01:18.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.503+0000 7fc042a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565276646b30 0x565276648f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:18.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.504+0000 7fc042a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565276646b30 0x565276648f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49012/0 (socket says 172.21.15.83:49012) 2024-04-24T09:01:18.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.504+0000 7fc042a00640 1 -- 172.21.15.83:0/768628747 learned_addr learned my addr 172.21.15.83:0/768628747 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:01:18.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.504+0000 7fc03be00640 1 --2- 172.21.15.83:0/2256641293 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fc0280a3d20 0x7fc0280a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:18.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.504+0000 7fc043400640 1 --2- 172.21.15.83:0/768628747 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56527663c460 0x5652766465f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:18.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.504+0000 7fc042a00640 1 --2- 172.21.15.83:0/768628747 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56527663bb40 0x56527663bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:18.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.504+0000 7fc03be00640 1 -- 172.21.15.83:0/2256641293 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7fc03c0f2310 con 0x7fc04410e7f0 2024-04-24T09:01:18.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.504+0000 7fc043e00640 1 --2- 172.21.15.83:0/2256641293 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fc0280a3d20 0x7fc0280a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:18.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.504+0000 7fc042a00640 1 -- 172.21.15.83:0/768628747 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56527663bb40 msgr2=0x56527663bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:18.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.504+0000 7fc042a00640 1 --2- 172.21.15.83:0/768628747 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56527663bb40 0x56527663bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.504+0000 7fc042a00640 1 -- 172.21.15.83:0/768628747 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56527663c460 msgr2=0x5652766465f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:18.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.504+0000 7fc042a00640 1 --2- 172.21.15.83:0/768628747 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56527663c460 0x5652766465f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.504+0000 7fc042a00640 1 -- 172.21.15.83:0/768628747 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565276604310 con 0x565276646b30 2024-04-24T09:01:18.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.505+0000 7fc042a00640 1 --2- 172.21.15.83:0/768628747 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565276646b30 0x565276648f20 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7fc030010790 tx=0x7fc03000e3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52f8f5d861c635ca server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:18.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.505+0000 7fc038c00640 1 -- 172.21.15.83:0/768628747 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc03000b6c0 con 0x565276646b30 2024-04-24T09:01:18.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.505+0000 7fc038c00640 1 -- 172.21.15.83:0/768628747 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc03000bd10 con 0x565276646b30 2024-04-24T09:01:18.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.505+0000 7fc038c00640 1 -- 172.21.15.83:0/768628747 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc03001ba60 con 0x565276646b30 2024-04-24T09:01:18.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.505+0000 7fc043e00640 1 --2- 172.21.15.83:0/2256641293 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fc0280a3d20 0x7fc0280a61e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc03c0c6680 tx=0x7fc03c0095c0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:18.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.506+0000 7fc04d44c740 1 -- 172.21.15.83:0/768628747 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565276646b30 msgr2=0x565276648f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:18.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.506+0000 7fc04d44c740 1 --2- 172.21.15.83:0/768628747 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565276646b30 0x565276648f20 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7fc030010790 tx=0x7fc03000e3d0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.506+0000 7fc04d44c740 1 -- 172.21.15.83:0/768628747 shutdown_connections 2024-04-24T09:01:18.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.506+0000 7fc04d44c740 1 --2- 172.21.15.83:0/768628747 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565276646b30 0x565276648f20 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.506+0000 7fc04d44c740 1 --2- 172.21.15.83:0/768628747 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56527663c460 0x5652766465f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.506+0000 7fc04d44c740 1 --2- 172.21.15.83:0/768628747 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56527663bb40 0x56527663bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.506+0000 7fc04d44c740 1 -- 172.21.15.83:0/768628747 >> 172.21.15.83:0/768628747 conn(0x565276637e60 msgr2=0x565276638240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:18.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.507+0000 7fc04d44c740 1 -- 172.21.15.83:0/768628747 shutdown_connections 2024-04-24T09:01:18.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.507+0000 7fc04d44c740 1 -- 172.21.15.83:0/768628747 wait complete. 2024-04-24T09:01:18.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.507+0000 7fc04d44c740 1 -- start start 2024-04-24T09:01:18.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.507+0000 7fc04d44c740 1 Processor -- start 2024-04-24T09:01:18.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.508+0000 7fc04d44c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56527663bb40 0x5652766438f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:18.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.508+0000 7fc04d44c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56527663c460 0x565276643e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:18.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.508+0000 7fc04d44c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565276646b30 0x565276644370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:18.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.508+0000 7fc04d44c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5652766499c0 con 0x565276646b30 2024-04-24T09:01:18.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.508+0000 7fc04d44c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56527655a810 con 0x56527663bb40 2024-04-24T09:01:18.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.508+0000 7fc04d44c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x565276649b40 con 0x56527663c460 2024-04-24T09:01:18.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.508+0000 7fc04d44c740 10 client.0 _ll_register_callbacks cb 0x565276528140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:01:18.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.508+0000 7fc042a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565276646b30 0x565276644370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:18.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.508+0000 7fc042a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565276646b30 0x565276644370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49028/0 (socket says 172.21.15.83:49028) 2024-04-24T09:01:18.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.508+0000 7fc042a00640 1 -- 172.21.15.83:0/1628831383 learned_addr learned my addr 172.21.15.83:0/1628831383 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:01:18.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.509+0000 7fc042a00640 1 --2- 172.21.15.83:0/1628831383 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56527663bb40 0x5652766438f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:18.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.509+0000 7fc043400640 1 --2- 172.21.15.83:0/1628831383 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56527663c460 0x565276643e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:18.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.509+0000 7fc042a00640 1 -- 172.21.15.83:0/1628831383 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56527663c460 msgr2=0x565276643e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:18.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.509+0000 7fc042a00640 1 --2- 172.21.15.83:0/1628831383 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56527663c460 0x565276643e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.509+0000 7fc042a00640 1 -- 172.21.15.83:0/1628831383 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56527663bb40 msgr2=0x5652766438f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:18.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.509+0000 7fc042a00640 1 --2- 172.21.15.83:0/1628831383 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56527663bb40 0x5652766438f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.509+0000 7fc042a00640 1 -- 172.21.15.83:0/1628831383 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56527656b3f0 con 0x565276646b30 2024-04-24T09:01:18.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.509+0000 7fc042a00640 1 --2- 172.21.15.83:0/1628831383 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565276646b30 0x565276644370 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7fc030005eb0 tx=0x7fc03000ead0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f70da5b565c52da server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:18.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.509+0000 7fc02d600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:01:18.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.510+0000 7fc02d600640 1 -- 172.21.15.83:0/1628831383 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc03000b6c0 con 0x565276646b30 2024-04-24T09:01:18.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.510+0000 7fc02d600640 1 -- 172.21.15.83:0/1628831383 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7fc03000b860 con 0x565276646b30 2024-04-24T09:01:18.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.510+0000 7fc04d44c740 1 -- 172.21.15.83:0/1628831383 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc03000bd10 con 0x565276646b30 2024-04-24T09:01:18.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.510+0000 7fc02d600640 1 -- 172.21.15.83:0/1628831383 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fc03000ba60 con 0x565276646b30 2024-04-24T09:01:18.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.510+0000 7fc04d44c740 10 client.19092 fetch_fsmap learned FSMap version 35 2024-04-24T09:01:18.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.510+0000 7fc04d44c740 1 -- 172.21.15.83:0/1628831383 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5652762b33f0 con 0x565276646b30 2024-04-24T09:01:18.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.510+0000 7fc02d600640 1 -- 172.21.15.83:0/1628831383 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc03000bd10 con 0x565276646b30 2024-04-24T09:01:18.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.511+0000 7fc02d600640 1 -- 172.21.15.83:0/1628831383 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7fc030025070 con 0x565276646b30 2024-04-24T09:01:18.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.511+0000 7fc04d44c740 10 client.19092 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T09:01:18.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.511+0000 7fc04d44c740 10 client.19092 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-24T09:01:18.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.511+0000 7fc04d44c740 20 client.19092 populate_metadata read hostname 'smithi083' 2024-04-24T09:01:18.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.511+0000 7fc04d44c740 1 --2- 172.21.15.83:0/1628831383 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x565276656ca0 0x565276659090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:18.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.511+0000 7fc04d44c740 4 client.19092 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:01:18.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.511+0000 7fc04d44c740 1 -- 172.21.15.83:0/1628831383 --> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5652765aebf0 con 0x565276656ca0 2024-04-24T09:01:18.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.511+0000 7fc043400640 1 --2- 172.21.15.83:0/1628831383 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x565276656ca0 0x565276659090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:18.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.513+0000 7fc02d600640 10 client.19092 ms_handle_connect on v2:172.21.15.138:6836/900566754 2024-04-24T09:01:18.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.513+0000 7fc043400640 1 --2- 172.21.15.83:0/1628831383 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x565276656ca0 0x565276659090 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=815f655ab672ebe8 server_cookie=c745debcdb3034e1 in_seq=0 out_seq=0 2024-04-24T09:01:18.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.515+0000 7fc02d600640 1 -- 172.21.15.83:0/1628831383 <== mds.1 v2:172.21.15.138:6836/900566754 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5652765aebf0 con 0x565276656ca0 2024-04-24T09:01:18.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.515+0000 7fc02d600640 10 client.19092 handle_command_reply: tid=0 2024-04-24T09:01:18.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.515+0000 7fc04d44c740 1 client.19092 shutdown 2024-04-24T09:01:18.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.515+0000 7fc04d44c740 1 -- 172.21.15.83:0/1628831383 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565276646b30 msgr2=0x565276644370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:18.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.515+0000 7fc04d44c740 1 --2- 172.21.15.83:0/1628831383 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565276646b30 0x565276644370 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7fc030005eb0 tx=0x7fc03000ead0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.515+0000 7fc04d44c740 1 -- 172.21.15.83:0/1628831383 shutdown_connections 2024-04-24T09:01:18.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.515+0000 7fc04d44c740 1 --2- 172.21.15.83:0/1628831383 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565276646b30 0x565276644370 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.515+0000 7fc04d44c740 1 --2- 172.21.15.83:0/1628831383 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x565276656ca0 0x565276659090 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.515+0000 7fc04d44c740 1 --2- 172.21.15.83:0/1628831383 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56527663c460 0x565276643e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.515+0000 7fc04d44c740 1 --2- 172.21.15.83:0/1628831383 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56527663bb40 0x5652766438f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.515+0000 7fc02d600640 0 client.19092 ms_handle_reset on v2:172.21.15.138:6836/900566754 2024-04-24T09:01:18.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.515+0000 7fc04d44c740 1 -- 172.21.15.83:0/1628831383 >> 172.21.15.83:0/1628831383 conn(0x565276637e60 msgr2=0x56527663dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:18.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.516+0000 7fc04d44c740 1 -- 172.21.15.83:0/1628831383 shutdown_connections 2024-04-24T09:01:18.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.516+0000 7fc04d44c740 1 -- 172.21.15.83:0/1628831383 wait complete. 2024-04-24T09:01:18.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.516+0000 7fc04d44c740 20 client.19092 trim_cache size 0 max 16384 2024-04-24T09:01:18.518 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.517+0000 7f1f94400640 1 -- 172.21.15.83:0/3229113382 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f8c074cf0 msgr2=0x7f1f8c10e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:18.518 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.517+0000 7f1f94400640 1 --2- 172.21.15.83:0/3229113382 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f8c074cf0 0x7f1f8c10e380 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f1f78009900 tx=0x7f1f7802a480 comp rx=0 tx=0).stop 2024-04-24T09:01:18.518 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.517+0000 7f1f94400640 1 -- 172.21.15.83:0/3229113382 shutdown_connections 2024-04-24T09:01:18.519 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.517+0000 7f1f94400640 1 --2- 172.21.15.83:0/3229113382 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f8c10e8c0 0x7f1f8c110cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.519 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.517+0000 7f1f94400640 1 --2- 172.21.15.83:0/3229113382 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f8c074cf0 0x7f1f8c10e380 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.519 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.517+0000 7f1f94400640 1 --2- 172.21.15.83:0/3229113382 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1f8c0743d0 0x7f1f8c0747b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.519 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.517+0000 7f1f94400640 1 -- 172.21.15.83:0/3229113382 >> 172.21.15.83:0/3229113382 conn(0x7f1f8c0fdab0 msgr2=0x7f1f8c0ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:18.519 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.517+0000 7f1f94400640 1 -- 172.21.15.83:0/3229113382 shutdown_connections 2024-04-24T09:01:18.519 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.517+0000 7f1f94400640 1 -- 172.21.15.83:0/3229113382 wait complete. 2024-04-24T09:01:18.519 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.518+0000 7f1f94400640 1 Processor -- start 2024-04-24T09:01:18.519 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.518+0000 7f1f94400640 1 -- start start 2024-04-24T09:01:18.519 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.519+0000 7f1f94400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1f8c0743d0 0x7f1f8c10ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:18.519 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.519+0000 7f1f94400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f8c074cf0 0x7f1f8c10d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:18.519 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.519+0000 7f1f94400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f8c10e8c0 0x7f1f8c1a48d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:18.519 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.519+0000 7f1f94400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1f8c1030c0 con 0x7f1f8c10e8c0 2024-04-24T09:01:18.519 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.519+0000 7f1f94400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1f8c102f40 con 0x7f1f8c074cf0 2024-04-24T09:01:18.519 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.519+0000 7f1f94400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1f8c103240 con 0x7f1f8c0743d0 2024-04-24T09:01:18.521 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.519+0000 7f1f92400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f8c10e8c0 0x7f1f8c1a48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:18.521 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.519+0000 7f1f92400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f8c10e8c0 0x7f1f8c1a48d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49050/0 (socket says 172.21.15.83:49050) 2024-04-24T09:01:18.521 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.519+0000 7f1f92400640 1 -- 172.21.15.83:0/2407685980 learned_addr learned my addr 172.21.15.83:0/2407685980 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:01:18.521 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.519+0000 7f1f91a00640 1 --2- 172.21.15.83:0/2407685980 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1f8c0743d0 0x7f1f8c10ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:18.521 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.519+0000 7f1f91000640 1 --2- 172.21.15.83:0/2407685980 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f8c074cf0 0x7f1f8c10d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:18.522 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.520+0000 7f1f92400640 1 -- 172.21.15.83:0/2407685980 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1f8c0743d0 msgr2=0x7f1f8c10ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:18.522 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.520+0000 7f1f92400640 1 --2- 172.21.15.83:0/2407685980 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1f8c0743d0 0x7f1f8c10ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.522 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.520+0000 7f1f92400640 1 -- 172.21.15.83:0/2407685980 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f8c074cf0 msgr2=0x7f1f8c10d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:18.522 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.520+0000 7f1f92400640 1 --2- 172.21.15.83:0/2407685980 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f8c074cf0 0x7f1f8c10d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.522 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.520+0000 7f1f92400640 1 -- 172.21.15.83:0/2407685980 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f8c1a5000 con 0x7f1f8c10e8c0 2024-04-24T09:01:18.522 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.520+0000 7f1f92400640 1 --2- 172.21.15.83:0/2407685980 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f8c10e8c0 0x7f1f8c1a48d0 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f1f84009fd0 tx=0x7f1f8400ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:18.522 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.520+0000 7f1f8a000640 1 -- 172.21.15.83:0/2407685980 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f84004d20 con 0x7f1f8c10e8c0 2024-04-24T09:01:18.522 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.520+0000 7f1f8a000640 1 -- 172.21.15.83:0/2407685980 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1f84005020 con 0x7f1f8c10e8c0 2024-04-24T09:01:18.522 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.520+0000 7f1f8a000640 1 -- 172.21.15.83:0/2407685980 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f840116b0 con 0x7f1f8c10e8c0 2024-04-24T09:01:18.522 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.520+0000 7f1f94400640 1 -- 172.21.15.83:0/2407685980 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f8c1a52f0 con 0x7f1f8c10e8c0 2024-04-24T09:01:18.522 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.521+0000 7f1f94400640 1 -- 172.21.15.83:0/2407685980 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f8c1ab980 con 0x7f1f8c10e8c0 2024-04-24T09:01:18.525 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.525+0000 7f1f971b7740 1 Processor -- start 2024-04-24T09:01:18.525 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.526+0000 7f1f8a000640 1 -- 172.21.15.83:0/2407685980 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1f840051d0 con 0x7f1f8c10e8c0 2024-04-24T09:01:18.526 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.526+0000 7f1f971b7740 1 -- start start 2024-04-24T09:01:18.527 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.526+0000 7f1f971b7740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55afc1b201b0 0x55afc1b20590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:18.527 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.526+0000 7f1f971b7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55afc1b20ad0 0x55afc1b2ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:18.527 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.526+0000 7f1f971b7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55afc1b2b390 0x55afc1b2d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:18.527 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.526+0000 7f1f971b7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55afc1a86900 con 0x55afc1b2b390 2024-04-24T09:01:18.527 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.526+0000 7f1f971b7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55afc1a2cf20 con 0x55afc1b20ad0 2024-04-24T09:01:18.527 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.526+0000 7f1f971b7740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55afc1a828c0 con 0x55afc1b201b0 2024-04-24T09:01:18.527 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.527+0000 7f1f91000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55afc1b2b390 0x55afc1b2d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:18.527 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.527+0000 7f1f91000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55afc1b2b390 0x55afc1b2d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49052/0 (socket says 172.21.15.83:49052) 2024-04-24T09:01:18.527 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.527+0000 7f1f91000640 1 -- 172.21.15.83:0/3844532719 learned_addr learned my addr 172.21.15.83:0/3844532719 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:01:18.527 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.527+0000 7f1f91000640 1 --2- 172.21.15.83:0/3844532719 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55afc1b201b0 0x55afc1b20590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:18.527 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.527+0000 7f1f91a00640 1 --2- 172.21.15.83:0/3844532719 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55afc1b20ad0 0x55afc1b2ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:18.528 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.527+0000 7f1f8a000640 1 --2- 172.21.15.83:0/2407685980 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f1f740a3e40 0x7f1f740a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:18.529 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.527+0000 7f1f91000640 1 -- 172.21.15.83:0/3844532719 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55afc1b201b0 msgr2=0x55afc1b20590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:18.529 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.527+0000 7f1f91000640 1 --2- 172.21.15.83:0/3844532719 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55afc1b201b0 0x55afc1b20590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.529 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.527+0000 7f1f91000640 1 -- 172.21.15.83:0/3844532719 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55afc1b20ad0 msgr2=0x55afc1b2ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:18.529 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.527+0000 7f1f91000640 1 --2- 172.21.15.83:0/3844532719 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55afc1b20ad0 0x55afc1b2ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.529 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.527+0000 7f1f91000640 1 -- 172.21.15.83:0/3844532719 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55afc1a51e10 con 0x55afc1b2b390 2024-04-24T09:01:18.529 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.528+0000 7f1f8a000640 1 -- 172.21.15.83:0/2407685980 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7f1f840d0d20 con 0x7f1f8c10e8c0 2024-04-24T09:01:18.529 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.528+0000 7f1f91000640 1 --2- 172.21.15.83:0/3844532719 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55afc1b2b390 0x55afc1b2d780 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f1f780096c0 tx=0x7f1f78005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=242362c10dc3f728 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:18.529 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.528+0000 7f1f92400640 1 --2- 172.21.15.83:0/2407685980 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f1f740a3e40 0x7f1f740a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:18.529 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.528+0000 7f1f7ea00640 1 -- 172.21.15.83:0/3844532719 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f78035030 con 0x55afc1b2b390 2024-04-24T09:01:18.529 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.528+0000 7f1f7ea00640 1 -- 172.21.15.83:0/3844532719 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1f78005550 con 0x55afc1b2b390 2024-04-24T09:01:18.529 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.528+0000 7f1f7ea00640 1 -- 172.21.15.83:0/3844532719 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f78003af0 con 0x55afc1b2b390 2024-04-24T09:01:18.531 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.529+0000 7f1f971b7740 1 -- 172.21.15.83:0/3844532719 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55afc1b2b390 msgr2=0x55afc1b2d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:18.532 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.529+0000 7f1f92400640 1 --2- 172.21.15.83:0/2407685980 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f1f740a3e40 0x7f1f740a6300 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f1f840a4f80 tx=0x7f1f84005b40 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:18.532 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.529+0000 7f1f971b7740 1 --2- 172.21.15.83:0/3844532719 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55afc1b2b390 0x55afc1b2d780 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f1f780096c0 tx=0x7f1f78005fd0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.532 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.529+0000 7f1f971b7740 1 -- 172.21.15.83:0/3844532719 shutdown_connections 2024-04-24T09:01:18.532 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.529+0000 7f1f971b7740 1 --2- 172.21.15.83:0/3844532719 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55afc1b2b390 0x55afc1b2d780 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.532 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.529+0000 7f1f971b7740 1 --2- 172.21.15.83:0/3844532719 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55afc1b20ad0 0x55afc1b2ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.532 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.529+0000 7f1f971b7740 1 --2- 172.21.15.83:0/3844532719 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55afc1b201b0 0x55afc1b20590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.532 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.529+0000 7f1f971b7740 1 -- 172.21.15.83:0/3844532719 >> 172.21.15.83:0/3844532719 conn(0x55afc1b1c6c0 msgr2=0x55afc1b1caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:18.532 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.530+0000 7f1f971b7740 1 -- 172.21.15.83:0/3844532719 shutdown_connections 2024-04-24T09:01:18.532 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.530+0000 7f1f971b7740 1 -- 172.21.15.83:0/3844532719 wait complete. 2024-04-24T09:01:18.532 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.530+0000 7f1f971b7740 1 -- start start 2024-04-24T09:01:18.532 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.530+0000 7f1f971b7740 1 Processor -- start 2024-04-24T09:01:18.532 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.531+0000 7f1f971b7740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55afc1b201b0 0x55afc1b28400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:18.533 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.531+0000 7f1f971b7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55afc1b20ad0 0x55afc1b28940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:18.533 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.531+0000 7f1f971b7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55afc1b2b390 0x55afc1b28e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:18.533 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.531+0000 7f1f971b7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55afc1b2df60 con 0x55afc1b2b390 2024-04-24T09:01:18.533 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.531+0000 7f1f971b7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55afc1a402c0 con 0x55afc1b20ad0 2024-04-24T09:01:18.533 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.531+0000 7f1f971b7740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55afc1b2e0e0 con 0x55afc1b201b0 2024-04-24T09:01:18.533 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.531+0000 7f1f971b7740 10 client.0 _ll_register_callbacks cb 0x55afc1798a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:01:18.533 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.531+0000 7f1f91000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55afc1b2b390 0x55afc1b28e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:18.533 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.532+0000 7f1f91000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55afc1b2b390 0x55afc1b28e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49064/0 (socket says 172.21.15.83:49064) 2024-04-24T09:01:18.533 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.532+0000 7f1f91000640 1 -- 172.21.15.83:0/3111958524 learned_addr learned my addr 172.21.15.83:0/3111958524 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:01:18.533 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.532+0000 7f1f91a00640 1 --2- 172.21.15.83:0/3111958524 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55afc1b20ad0 0x55afc1b28940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:18.533 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.532+0000 7f1f91000640 1 --2- 172.21.15.83:0/3111958524 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55afc1b201b0 0x55afc1b28400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:18.535 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.532+0000 7f1f91000640 1 -- 172.21.15.83:0/3111958524 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55afc1b201b0 msgr2=0x55afc1b28400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:18.535 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.532+0000 7f1f91000640 1 --2- 172.21.15.83:0/3111958524 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55afc1b201b0 0x55afc1b28400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.535 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.532+0000 7f1f91000640 1 -- 172.21.15.83:0/3111958524 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55afc1b20ad0 msgr2=0x55afc1b28940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:18.535 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.532+0000 7f1f91000640 1 --2- 172.21.15.83:0/3111958524 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55afc1b20ad0 0x55afc1b28940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.535 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.532+0000 7f1f91000640 1 -- 172.21.15.83:0/3111958524 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55afc1a50fd0 con 0x55afc1b2b390 2024-04-24T09:01:18.535 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.532+0000 7f1f91000640 1 --2- 172.21.15.83:0/3111958524 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55afc1b2b390 0x55afc1b28e80 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f1f78001eb0 tx=0x7f1f78041040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5a73ef21790cf79 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:18.535 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.532+0000 7f1f73400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:01:18.535 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.533+0000 7f1f73400640 1 -- 172.21.15.83:0/3111958524 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f78035030 con 0x55afc1b2b390 2024-04-24T09:01:18.536 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.533+0000 7f1f73400640 1 -- 172.21.15.83:0/3111958524 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7f1f78003cb0 con 0x55afc1b2b390 2024-04-24T09:01:18.536 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.533+0000 7f1f971b7740 1 -- 172.21.15.83:0/3111958524 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f78005550 con 0x55afc1b2b390 2024-04-24T09:01:18.536 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.533+0000 7f1f73400640 1 -- 172.21.15.83:0/3111958524 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f1f78005040 con 0x55afc1b2b390 2024-04-24T09:01:18.536 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.533+0000 7f1f971b7740 10 client.19104 fetch_fsmap learned FSMap version 35 2024-04-24T09:01:18.536 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.533+0000 7f1f971b7740 1 -- 172.21.15.83:0/3111958524 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55afc1799210 con 0x55afc1b2b390 2024-04-24T09:01:18.536 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.534+0000 7f1f73400640 1 -- 172.21.15.83:0/3111958524 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1f78005550 con 0x55afc1b2b390 2024-04-24T09:01:18.536 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.534+0000 7f1f73400640 1 -- 172.21.15.83:0/3111958524 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7f1f78031810 con 0x55afc1b2b390 2024-04-24T09:01:18.536 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.534+0000 7f1f971b7740 10 client.19104 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T09:01:18.536 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.534+0000 7f1f971b7740 10 client.19104 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:01:18.536 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.534+0000 7f1f971b7740 20 client.19104 populate_metadata read hostname 'smithi083' 2024-04-24T09:01:18.536 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.534+0000 7f1f971b7740 1 --2- 172.21.15.83:0/3111958524 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55afc1b3b350 0x55afc1b3d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:18.536 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.534+0000 7f1f971b7740 4 client.19104 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:01:18.536 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.534+0000 7f1f971b7740 1 -- 172.21.15.83:0/3111958524 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55afc1a94320 con 0x55afc1b3b350 2024-04-24T09:01:18.537 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.535+0000 7f1f91a00640 1 --2- 172.21.15.83:0/3111958524 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55afc1b3b350 0x55afc1b3d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:18.538 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.536+0000 7f1f91a00640 1 --2- 172.21.15.83:0/3111958524 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55afc1b3b350 0x55afc1b3d740 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a861f7374dd554ce server_cookie=be36060da0b9ba59 in_seq=0 out_seq=0 2024-04-24T09:01:18.538 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.536+0000 7f1f73400640 10 client.19104 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:01:18.538 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.537+0000 7f1f73400640 1 -- 172.21.15.83:0/3111958524 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55afc1a94320 con 0x55afc1b3b350 2024-04-24T09:01:18.538 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.537+0000 7f1f73400640 10 client.19104 handle_command_reply: tid=0 2024-04-24T09:01:18.538 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.537+0000 7f1f971b7740 1 client.19104 shutdown 2024-04-24T09:01:18.538 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.538+0000 7f1f971b7740 1 -- 172.21.15.83:0/3111958524 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55afc1b2b390 msgr2=0x55afc1b28e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:18.538 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.538+0000 7f1f971b7740 1 --2- 172.21.15.83:0/3111958524 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55afc1b2b390 0x55afc1b28e80 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f1f78001eb0 tx=0x7f1f78041040 comp rx=0 tx=0).stop 2024-04-24T09:01:18.538 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.538+0000 7f1f971b7740 1 -- 172.21.15.83:0/3111958524 shutdown_connections 2024-04-24T09:01:18.538 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.538+0000 7f1f971b7740 1 --2- 172.21.15.83:0/3111958524 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55afc1b3b350 0x55afc1b3d740 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.539 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.538+0000 7f1f971b7740 1 --2- 172.21.15.83:0/3111958524 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55afc1b2b390 0x55afc1b28e80 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.539 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.538+0000 7f1f971b7740 1 --2- 172.21.15.83:0/3111958524 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55afc1b20ad0 0x55afc1b28940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.539 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.538+0000 7f1f971b7740 1 --2- 172.21.15.83:0/3111958524 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55afc1b201b0 0x55afc1b28400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.539 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.538+0000 7f1f971b7740 1 -- 172.21.15.83:0/3111958524 >> 172.21.15.83:0/3111958524 conn(0x55afc1b1c6c0 msgr2=0x55afc1b223f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:18.539 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.538+0000 7f1f73400640 0 client.19104 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:01:18.539 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.538+0000 7f1f971b7740 1 -- 172.21.15.83:0/3111958524 shutdown_connections 2024-04-24T09:01:18.539 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.538+0000 7f1f971b7740 1 -- 172.21.15.83:0/3111958524 wait complete. 2024-04-24T09:01:18.539 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.538+0000 7f1f971b7740 20 client.19104 trim_cache size 0 max 16384 2024-04-24T09:01:18.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.546+0000 7fc04d44c740 1 Processor -- start 2024-04-24T09:01:18.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.546+0000 7fc04d44c740 1 -- start start 2024-04-24T09:01:18.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.546+0000 7fc04d44c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56527663bb40 0x5652766528b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:18.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.546+0000 7fc04d44c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56527663c460 0x56527663e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:18.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.546+0000 7fc04d44c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565276646b30 0x56527665e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:18.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.546+0000 7fc04d44c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc0441ad1d0 con 0x56527663c460 2024-04-24T09:01:18.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.546+0000 7fc04d44c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc0441aed10 con 0x56527663bb40 2024-04-24T09:01:18.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.546+0000 7fc04d44c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc0441ad8a0 con 0x565276646b30 2024-04-24T09:01:18.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.546+0000 7fc043400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56527663c460 0x56527663e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:18.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.547+0000 7fc043400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56527663c460 0x56527663e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49070/0 (socket says 172.21.15.83:49070) 2024-04-24T09:01:18.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.547+0000 7fc043400640 1 -- 172.21.15.83:0/1443992936 learned_addr learned my addr 172.21.15.83:0/1443992936 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:01:18.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.547+0000 7fc042a00640 1 --2- 172.21.15.83:0/1443992936 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56527663bb40 0x5652766528b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:18.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.547+0000 7fc042a00640 1 --2- 172.21.15.83:0/1443992936 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565276646b30 0x56527665e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:18.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.547+0000 7fc043400640 1 -- 172.21.15.83:0/1443992936 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565276646b30 msgr2=0x56527665e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:18.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.547+0000 7fc043400640 1 --2- 172.21.15.83:0/1443992936 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565276646b30 0x56527665e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.547+0000 7fc043400640 1 -- 172.21.15.83:0/1443992936 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56527663bb40 msgr2=0x5652766528b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:18.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.547+0000 7fc043400640 1 --2- 172.21.15.83:0/1443992936 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56527663bb40 0x5652766528b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.547+0000 7fc043400640 1 -- 172.21.15.83:0/1443992936 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56527637d7d0 con 0x56527663c460 2024-04-24T09:01:18.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.547+0000 7fc042a00640 1 --2- 172.21.15.83:0/1443992936 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56527663bb40 0x5652766528b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:01:18.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.547+0000 7fc043400640 1 --2- 172.21.15.83:0/1443992936 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56527663c460 0x56527663e2a0 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7fc03400bfd0 tx=0x7fc0340139a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1361093541c35f7f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:18.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.548+0000 7fc02fe00640 1 -- 172.21.15.83:0/1443992936 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc03401d430 con 0x56527663c460 2024-04-24T09:01:18.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.549+0000 7fc02fe00640 1 -- 172.21.15.83:0/1443992936 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc034005020 con 0x56527663c460 2024-04-24T09:01:18.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.549+0000 7fc02fe00640 1 -- 172.21.15.83:0/1443992936 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc034004b70 con 0x56527663c460 2024-04-24T09:01:18.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.550+0000 7fc04d44c740 1 -- 172.21.15.83:0/1443992936 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56527663c460 msgr2=0x56527663e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:18.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.550+0000 7fc04d44c740 1 --2- 172.21.15.83:0/1443992936 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56527663c460 0x56527663e2a0 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7fc03400bfd0 tx=0x7fc0340139a0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.550+0000 7fc04d44c740 1 -- 172.21.15.83:0/1443992936 shutdown_connections 2024-04-24T09:01:18.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.550+0000 7fc04d44c740 1 --2- 172.21.15.83:0/1443992936 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565276646b30 0x56527665e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.550+0000 7fc04d44c740 1 --2- 172.21.15.83:0/1443992936 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56527663c460 0x56527663e2a0 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.550+0000 7fc04d44c740 1 --2- 172.21.15.83:0/1443992936 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56527663bb40 0x5652766528b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.550+0000 7fc04d44c740 1 -- 172.21.15.83:0/1443992936 >> 172.21.15.83:0/1443992936 conn(0x565276637e60 msgr2=0x565276642920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:18.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.550+0000 7fc04d44c740 1 -- 172.21.15.83:0/1443992936 shutdown_connections 2024-04-24T09:01:18.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.550+0000 7fc04d44c740 1 -- 172.21.15.83:0/1443992936 wait complete. 2024-04-24T09:01:18.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.551+0000 7fc04d44c740 1 -- start start 2024-04-24T09:01:18.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.551+0000 7fc04d44c740 1 Processor -- start 2024-04-24T09:01:18.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.552+0000 7fc04d44c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56527663bb40 0x565276647340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:18.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.552+0000 7fc04d44c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56527663c460 0x565276647880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:18.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.552+0000 7fc04d44c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565276646b30 0x56527664ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:18.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.552+0000 7fc04d44c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc0441ae640 con 0x56527663c460 2024-04-24T09:01:18.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.552+0000 7fc04d44c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc0441adf70 con 0x56527663bb40 2024-04-24T09:01:18.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.552+0000 7fc04d44c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc0441ab1d0 con 0x565276646b30 2024-04-24T09:01:18.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.552+0000 7fc04d44c740 10 client.0 _ll_register_callbacks cb 0x7fc03400b440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:01:18.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.552+0000 7fc043400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56527663c460 0x565276647880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:18.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.552+0000 7fc043400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56527663c460 0x565276647880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49074/0 (socket says 172.21.15.83:49074) 2024-04-24T09:01:18.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.552+0000 7fc043400640 1 -- 172.21.15.83:0/2453331182 learned_addr learned my addr 172.21.15.83:0/2453331182 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:01:18.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.552+0000 7fc042a00640 1 --2- 172.21.15.83:0/2453331182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56527663bb40 0x565276647340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:18.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.552+0000 7fc042a00640 1 --2- 172.21.15.83:0/2453331182 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565276646b30 0x56527664ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:18.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.553+0000 7fc043400640 1 -- 172.21.15.83:0/2453331182 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565276646b30 msgr2=0x56527664ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:18.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.553+0000 7fc043400640 1 --2- 172.21.15.83:0/2453331182 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565276646b30 0x56527664ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.553+0000 7fc043400640 1 -- 172.21.15.83:0/2453331182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56527663bb40 msgr2=0x565276647340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:18.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.553+0000 7fc043400640 1 --2- 172.21.15.83:0/2453331182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56527663bb40 0x565276647340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.553+0000 7fc043400640 1 -- 172.21.15.83:0/2453331182 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565276648130 con 0x56527663c460 2024-04-24T09:01:18.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.553+0000 7fc042a00640 1 --2- 172.21.15.83:0/2453331182 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565276646b30 0x56527664ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:01:18.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.553+0000 7fc043400640 1 --2- 172.21.15.83:0/2453331182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56527663c460 0x565276647880 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7fc03400baa0 tx=0x7fc0340132e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a92971da9b40735e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:18.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.553+0000 7fc02d600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:01:18.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.553+0000 7fc02d600640 1 -- 172.21.15.83:0/2453331182 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc03401d430 con 0x56527663c460 2024-04-24T09:01:18.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.554+0000 7fc04d44c740 1 -- 172.21.15.83:0/2453331182 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc034005020 con 0x56527663c460 2024-04-24T09:01:18.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.554+0000 7fc02d600640 1 -- 172.21.15.83:0/2453331182 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7fc03401d5d0 con 0x56527663c460 2024-04-24T09:01:18.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.554+0000 7fc02d600640 1 -- 172.21.15.83:0/2453331182 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fc034034050 con 0x56527663c460 2024-04-24T09:01:18.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.554+0000 7fc04d44c740 10 client.19110 fetch_fsmap learned FSMap version 35 2024-04-24T09:01:18.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.554+0000 7fc04d44c740 1 -- 172.21.15.83:0/2453331182 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56527664f430 con 0x56527663c460 2024-04-24T09:01:18.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.554+0000 7fc02d600640 1 -- 172.21.15.83:0/2453331182 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc034005020 con 0x56527663c460 2024-04-24T09:01:18.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.555+0000 7fc02d600640 1 -- 172.21.15.83:0/2453331182 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7fc034010650 con 0x56527663c460 2024-04-24T09:01:18.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.555+0000 7fc04d44c740 10 client.19110 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T09:01:18.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.555+0000 7fc04d44c740 10 client.19110 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-24T09:01:18.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.555+0000 7fc04d44c740 20 client.19110 populate_metadata read hostname 'smithi083' 2024-04-24T09:01:18.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.555+0000 7fc04d44c740 1 --2- 172.21.15.83:0/2453331182 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x5652766bd290 0x5652766bf6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:18.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.555+0000 7fc04d44c740 4 client.19110 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:01:18.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.555+0000 7fc04d44c740 1 -- 172.21.15.83:0/2453331182 --> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5652765a8a60 con 0x5652766bd290 2024-04-24T09:01:18.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.555+0000 7fc042a00640 1 --2- 172.21.15.83:0/2453331182 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x5652766bd290 0x5652766bf6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:18.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.556+0000 7fc042a00640 1 --2- 172.21.15.83:0/2453331182 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x5652766bd290 0x5652766bf6b0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8d9ded56c01c3b4f server_cookie=7ec8ae1057168c90 in_seq=0 out_seq=0 2024-04-24T09:01:18.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.556+0000 7fc02d600640 10 client.19110 ms_handle_connect on v2:172.21.15.138:6836/900566754 2024-04-24T09:01:18.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.557+0000 7fc02d600640 1 -- 172.21.15.83:0/2453331182 <== mds.1 v2:172.21.15.138:6836/900566754 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5652765a8a60 con 0x5652766bd290 2024-04-24T09:01:18.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.557+0000 7fc02d600640 10 client.19110 handle_command_reply: tid=0 2024-04-24T09:01:18.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.557+0000 7fc04d44c740 1 client.19110 shutdown 2024-04-24T09:01:18.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.557+0000 7fc04d44c740 1 -- 172.21.15.83:0/2453331182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56527663c460 msgr2=0x565276647880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:18.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.557+0000 7fc04d44c740 1 --2- 172.21.15.83:0/2453331182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56527663c460 0x565276647880 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7fc03400baa0 tx=0x7fc0340132e0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.557+0000 7fc04d44c740 1 -- 172.21.15.83:0/2453331182 shutdown_connections 2024-04-24T09:01:18.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.557+0000 7fc04d44c740 1 --2- 172.21.15.83:0/2453331182 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x5652766bd290 0x5652766bf6b0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.557+0000 7fc04d44c740 1 --2- 172.21.15.83:0/2453331182 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565276646b30 0x56527664ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.557+0000 7fc04d44c740 1 --2- 172.21.15.83:0/2453331182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56527663c460 0x565276647880 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.557+0000 7fc04d44c740 1 --2- 172.21.15.83:0/2453331182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56527663bb40 0x565276647340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.557+0000 7fc04d44c740 1 -- 172.21.15.83:0/2453331182 >> 172.21.15.83:0/2453331182 conn(0x565276637e60 msgr2=0x5652766438b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:18.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.557+0000 7fc02d600640 0 client.19110 ms_handle_reset on v2:172.21.15.138:6836/900566754 2024-04-24T09:01:18.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.557+0000 7fc04d44c740 1 -- 172.21.15.83:0/2453331182 shutdown_connections 2024-04-24T09:01:18.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.557+0000 7fc04d44c740 1 -- 172.21.15.83:0/2453331182 wait complete. 2024-04-24T09:01:18.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.558+0000 7fc04d44c740 20 client.19110 trim_cache size 0 max 16384 2024-04-24T09:01:18.560 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:01:18.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.558+0000 7fc049000640 1 -- 172.21.15.83:0/2256641293 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fc0280a3d20 msgr2=0x7fc0280a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:18.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.558+0000 7fc049000640 1 --2- 172.21.15.83:0/2256641293 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fc0280a3d20 0x7fc0280a61e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc03c0c6680 tx=0x7fc03c0095c0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.558+0000 7fc049000640 1 -- 172.21.15.83:0/2256641293 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc04410e7f0 msgr2=0x7fc04419a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:18.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.558+0000 7fc049000640 1 --2- 172.21.15.83:0/2256641293 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc04410e7f0 0x7fc04419a700 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7fc03c0098d0 tx=0x7fc03c032ff0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.559+0000 7fc049000640 1 -- 172.21.15.83:0/2256641293 shutdown_connections 2024-04-24T09:01:18.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.559+0000 7fc049000640 1 --2- 172.21.15.83:0/2256641293 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fc0280a3d20 0x7fc0280a61e0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.559+0000 7fc049000640 1 --2- 172.21.15.83:0/2256641293 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc04410e7f0 0x7fc04419a700 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.559+0000 7fc049000640 1 --2- 172.21.15.83:0/2256641293 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc0441015f0 0x7fc0441a0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.559+0000 7fc049000640 1 --2- 172.21.15.83:0/2256641293 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc044100c40 0x7fc0441a05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.559+0000 7fc049000640 1 -- 172.21.15.83:0/2256641293 >> 172.21.15.83:0/2256641293 conn(0x7fc0440fc880 msgr2=0x7fc0440fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:18.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.559+0000 7fc049000640 1 -- 172.21.15.83:0/2256641293 shutdown_connections 2024-04-24T09:01:18.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.559+0000 7fc049000640 1 -- 172.21.15.83:0/2256641293 wait complete. 2024-04-24T09:01:18.564 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.560+0000 7f1f971b7740 1 Processor -- start 2024-04-24T09:01:18.564 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.560+0000 7f1f971b7740 1 -- start start 2024-04-24T09:01:18.565 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.560+0000 7f1f971b7740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55afc1b201b0 0x55afc1b40380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:18.565 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.560+0000 7f1f971b7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55afc1b20ad0 0x55afc1b408c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:18.565 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.560+0000 7f1f971b7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55afc1b2b390 0x55afc1b26cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:18.565 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.560+0000 7f1f971b7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1f8c1b5b70 con 0x55afc1b20ad0 2024-04-24T09:01:18.565 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.560+0000 7f1f971b7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1f8c1b76b0 con 0x55afc1b2b390 2024-04-24T09:01:18.565 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.560+0000 7f1f971b7740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1f8c1b6240 con 0x55afc1b201b0 2024-04-24T09:01:18.565 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.560+0000 7f1f91a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55afc1b20ad0 0x55afc1b408c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:18.565 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.561+0000 7f1f91a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55afc1b20ad0 0x55afc1b408c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49086/0 (socket says 172.21.15.83:49086) 2024-04-24T09:01:18.565 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.561+0000 7f1f91a00640 1 -- 172.21.15.83:0/3103241294 learned_addr learned my addr 172.21.15.83:0/3103241294 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:01:18.565 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.561+0000 7f1f91000640 1 --2- 172.21.15.83:0/3103241294 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55afc1b201b0 0x55afc1b40380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:18.565 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.561+0000 7f1f91a00640 1 -- 172.21.15.83:0/3103241294 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55afc1b201b0 msgr2=0x55afc1b40380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:18.566 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.561+0000 7f1f91000640 1 --2- 172.21.15.83:0/3103241294 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55afc1b2b390 0x55afc1b26cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:18.566 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.561+0000 7f1f91a00640 1 --2- 172.21.15.83:0/3103241294 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55afc1b201b0 0x55afc1b40380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.566 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.561+0000 7f1f91a00640 1 -- 172.21.15.83:0/3103241294 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55afc1b2b390 msgr2=0x55afc1b26cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:18.566 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.561+0000 7f1f91a00640 1 --2- 172.21.15.83:0/3103241294 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55afc1b2b390 0x55afc1b26cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.566 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.561+0000 7f1f91a00640 1 -- 172.21.15.83:0/3103241294 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55afc1b20900 con 0x55afc1b20ad0 2024-04-24T09:01:18.566 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.561+0000 7f1f91000640 1 --2- 172.21.15.83:0/3103241294 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55afc1b2b390 0x55afc1b26cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:01:18.566 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.561+0000 7f1f91a00640 1 --2- 172.21.15.83:0/3103241294 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55afc1b20ad0 0x55afc1b408c0 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x55afc1b21750 tx=0x7f1f8000a2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d51d68dda751bf6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:18.566 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.562+0000 7f1f7e000640 1 -- 172.21.15.83:0/3103241294 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f800200f0 con 0x55afc1b20ad0 2024-04-24T09:01:18.566 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.562+0000 7f1f7e000640 1 -- 172.21.15.83:0/3103241294 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1f80024070 con 0x55afc1b20ad0 2024-04-24T09:01:18.566 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.562+0000 7f1f7e000640 1 -- 172.21.15.83:0/3103241294 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f8001ebf0 con 0x55afc1b20ad0 2024-04-24T09:01:18.566 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.563+0000 7f1f971b7740 1 -- 172.21.15.83:0/3103241294 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55afc1b20ad0 msgr2=0x55afc1b408c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:18.566 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.563+0000 7f1f971b7740 1 --2- 172.21.15.83:0/3103241294 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55afc1b20ad0 0x55afc1b408c0 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x55afc1b21750 tx=0x7f1f8000a2d0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.567 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.563+0000 7f1f971b7740 1 -- 172.21.15.83:0/3103241294 shutdown_connections 2024-04-24T09:01:18.567 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.563+0000 7f1f971b7740 1 --2- 172.21.15.83:0/3103241294 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55afc1b2b390 0x55afc1b26cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.567 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.563+0000 7f1f971b7740 1 --2- 172.21.15.83:0/3103241294 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55afc1b20ad0 0x55afc1b408c0 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.567 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.563+0000 7f1f971b7740 1 --2- 172.21.15.83:0/3103241294 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55afc1b201b0 0x55afc1b40380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.567 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.563+0000 7f1f971b7740 1 -- 172.21.15.83:0/3103241294 >> 172.21.15.83:0/3103241294 conn(0x55afc1b1c6c0 msgr2=0x55afc1b1caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:18.567 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.563+0000 7f1f971b7740 1 -- 172.21.15.83:0/3103241294 shutdown_connections 2024-04-24T09:01:18.567 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.563+0000 7f1f971b7740 1 -- 172.21.15.83:0/3103241294 wait complete. 2024-04-24T09:01:18.567 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.563+0000 7f1f971b7740 1 -- start start 2024-04-24T09:01:18.567 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.563+0000 7f1f971b7740 1 Processor -- start 2024-04-24T09:01:18.567 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.564+0000 7f1f971b7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55afc1b201b0 0x55afc1b400a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:18.567 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.564+0000 7f1f971b7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55afc1b20ad0 0x55afc1b405e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:18.567 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.564+0000 7f1f971b7740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55afc1b2b390 0x55afc1b3d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:18.567 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.564+0000 7f1f971b7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1f8c1b6fe0 con 0x55afc1b201b0 2024-04-24T09:01:18.567 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.564+0000 7f1f971b7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1f8c1b6910 con 0x55afc1b20ad0 2024-04-24T09:01:18.568 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.564+0000 7f1f971b7740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1f8c1b3b70 con 0x55afc1b2b390 2024-04-24T09:01:18.568 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.564+0000 7f1f971b7740 10 client.0 _ll_register_callbacks cb 0x7f1f8000a590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:01:18.568 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.564+0000 7f1f91000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55afc1b201b0 0x55afc1b400a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:18.568 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.564+0000 7f1f91000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55afc1b201b0 0x55afc1b400a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49092/0 (socket says 172.21.15.83:49092) 2024-04-24T09:01:18.568 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.564+0000 7f1f91000640 1 -- 172.21.15.83:0/3248207706 learned_addr learned my addr 172.21.15.83:0/3248207706 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:01:18.568 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.565+0000 7f1f91a00640 1 --2- 172.21.15.83:0/3248207706 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55afc1b20ad0 0x55afc1b405e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:18.568 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.565+0000 7f1f91000640 1 --2- 172.21.15.83:0/3248207706 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55afc1b2b390 0x55afc1b3d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:18.570 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.565+0000 7f1f91000640 1 -- 172.21.15.83:0/3248207706 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55afc1b2b390 msgr2=0x55afc1b3d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:18.570 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.565+0000 7f1f91000640 1 --2- 172.21.15.83:0/3248207706 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55afc1b2b390 0x55afc1b3d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.570 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.565+0000 7f1f91000640 1 -- 172.21.15.83:0/3248207706 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55afc1b20ad0 msgr2=0x55afc1b405e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:18.570 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.565+0000 7f1f91000640 1 --2- 172.21.15.83:0/3248207706 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55afc1b20ad0 0x55afc1b405e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.570 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.565+0000 7f1f91000640 1 -- 172.21.15.83:0/3248207706 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55afc1865620 con 0x55afc1b201b0 2024-04-24T09:01:18.571 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.565+0000 7f1f91000640 1 --2- 172.21.15.83:0/3248207706 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55afc1b201b0 0x55afc1b400a0 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f1f78005e00 tx=0x7f1f78041040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fd0fc46a382aa68 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:18.571 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.565+0000 7f1f73400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:01:18.571 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.566+0000 7f1f73400640 1 -- 172.21.15.83:0/3248207706 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f78035040 con 0x55afc1b201b0 2024-04-24T09:01:18.571 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.566+0000 7f1f73400640 1 -- 172.21.15.83:0/3248207706 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7f1f78003eb0 con 0x55afc1b201b0 2024-04-24T09:01:18.571 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.566+0000 7f1f971b7740 1 -- 172.21.15.83:0/3248207706 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f80024070 con 0x55afc1b201b0 2024-04-24T09:01:18.571 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.566+0000 7f1f73400640 1 -- 172.21.15.83:0/3248207706 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f1f780036b0 con 0x55afc1b201b0 2024-04-24T09:01:18.571 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.566+0000 7f1f971b7740 10 client.19116 fetch_fsmap learned FSMap version 35 2024-04-24T09:01:18.571 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.566+0000 7f1f971b7740 1 -- 172.21.15.83:0/3248207706 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55afc1ba0dd0 con 0x55afc1b201b0 2024-04-24T09:01:18.571 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.567+0000 7f1f73400640 1 -- 172.21.15.83:0/3248207706 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1f80024070 con 0x55afc1b201b0 2024-04-24T09:01:18.571 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.567+0000 7f1f73400640 1 -- 172.21.15.83:0/3248207706 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7f1f78004750 con 0x55afc1b201b0 2024-04-24T09:01:18.571 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.567+0000 7f1f971b7740 10 client.19116 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T09:01:18.571 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.567+0000 7f1f971b7740 10 client.19116 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:01:18.571 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.567+0000 7f1f971b7740 20 client.19116 populate_metadata read hostname 'smithi083' 2024-04-24T09:01:18.572 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.567+0000 7f1f971b7740 1 --2- 172.21.15.83:0/3248207706 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55afc1ba1490 0x55afc1ba3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:18.572 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.567+0000 7f1f971b7740 4 client.19116 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T09:01:18.572 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.567+0000 7f1f971b7740 1 -- 172.21.15.83:0/3248207706 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55afc1a8e190 con 0x55afc1ba1490 2024-04-24T09:01:18.572 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.568+0000 7f1f91a00640 1 --2- 172.21.15.83:0/3248207706 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55afc1ba1490 0x55afc1ba3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:18.572 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.569+0000 7f1f91a00640 1 --2- 172.21.15.83:0/3248207706 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55afc1ba1490 0x55afc1ba3880 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec82dc4fd49463dd server_cookie=ae76b3e32c5fe8c3 in_seq=0 out_seq=0 2024-04-24T09:01:18.572 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.569+0000 7f1f73400640 10 client.19116 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:01:18.577 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T09:01:18.605 INFO:tasks.workunit.client.0.smithi083.stdout: 1 55996 0.63 MB/sec execute 352 sec latency 45.829 ms 2024-04-24T09:01:18.639 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.638+0000 7f1f73400640 1 -- 172.21.15.83:0/3248207706 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55afc1a8e190 con 0x55afc1ba1490 2024-04-24T09:01:18.639 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.638+0000 7f1f73400640 10 client.19116 handle_command_reply: tid=0 2024-04-24T09:01:18.639 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.638+0000 7f1f971b7740 1 client.19116 shutdown 2024-04-24T09:01:18.639 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.638+0000 7f1f971b7740 1 -- 172.21.15.83:0/3248207706 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55afc1b201b0 msgr2=0x55afc1b400a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:18.639 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.638+0000 7f1f971b7740 1 --2- 172.21.15.83:0/3248207706 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55afc1b201b0 0x55afc1b400a0 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f1f78005e00 tx=0x7f1f78041040 comp rx=0 tx=0).stop 2024-04-24T09:01:18.639 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.639+0000 7f1f971b7740 1 -- 172.21.15.83:0/3248207706 shutdown_connections 2024-04-24T09:01:18.639 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.639+0000 7f1f971b7740 1 --2- 172.21.15.83:0/3248207706 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55afc1ba1490 0x55afc1ba3880 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.639 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.639+0000 7f1f971b7740 1 --2- 172.21.15.83:0/3248207706 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55afc1b2b390 0x55afc1b3d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.639 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.639+0000 7f1f971b7740 1 --2- 172.21.15.83:0/3248207706 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55afc1b20ad0 0x55afc1b405e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.639 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.639+0000 7f1f971b7740 1 --2- 172.21.15.83:0/3248207706 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55afc1b201b0 0x55afc1b400a0 unknown :-1 s=CLOSED pgs=1653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.640 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.639+0000 7f1f73400640 0 client.19116 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:01:18.640 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.639+0000 7f1f971b7740 1 -- 172.21.15.83:0/3248207706 >> 172.21.15.83:0/3248207706 conn(0x55afc1b1c6c0 msgr2=0x55afc1b26be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:18.640 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.639+0000 7f1f971b7740 1 -- 172.21.15.83:0/3248207706 shutdown_connections 2024-04-24T09:01:18.640 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.639+0000 7f1f971b7740 1 -- 172.21.15.83:0/3248207706 wait complete. 2024-04-24T09:01:18.640 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.639+0000 7f1f971b7740 20 client.19116 trim_cache size 0 max 16384 2024-04-24T09:01:18.640 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T09:01:18.640 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T09:01:18.640 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T09:01:18.640 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T09:01:18.641 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.640+0000 7f1f94400640 1 -- 172.21.15.83:0/2407685980 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f1f740a3e40 msgr2=0x7f1f740a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:18.641 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.640+0000 7f1f94400640 1 --2- 172.21.15.83:0/2407685980 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f1f740a3e40 0x7f1f740a6300 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f1f840a4f80 tx=0x7f1f84005b40 comp rx=0 tx=0).stop 2024-04-24T09:01:18.641 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.640+0000 7f1f94400640 1 -- 172.21.15.83:0/2407685980 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f8c10e8c0 msgr2=0x7f1f8c1a48d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:18.641 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.640+0000 7f1f94400640 1 --2- 172.21.15.83:0/2407685980 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f8c10e8c0 0x7f1f8c1a48d0 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f1f84009fd0 tx=0x7f1f8400ee60 comp rx=0 tx=0).stop 2024-04-24T09:01:18.641 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.640+0000 7f1f94400640 1 -- 172.21.15.83:0/2407685980 shutdown_connections 2024-04-24T09:01:18.641 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.640+0000 7f1f94400640 1 --2- 172.21.15.83:0/2407685980 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f1f740a3e40 0x7f1f740a6300 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.642 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.640+0000 7f1f94400640 1 --2- 172.21.15.83:0/2407685980 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f8c10e8c0 0x7f1f8c1a48d0 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.642 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.640+0000 7f1f94400640 1 --2- 172.21.15.83:0/2407685980 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f8c074cf0 0x7f1f8c10d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.642 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.640+0000 7f1f94400640 1 --2- 172.21.15.83:0/2407685980 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1f8c0743d0 0x7f1f8c10ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.642 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.640+0000 7f1f94400640 1 -- 172.21.15.83:0/2407685980 >> 172.21.15.83:0/2407685980 conn(0x7f1f8c0fdab0 msgr2=0x7f1f8c0fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:18.642 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.640+0000 7f1f94400640 1 -- 172.21.15.83:0/2407685980 shutdown_connections 2024-04-24T09:01:18.642 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:18.640+0000 7f1f94400640 1 -- 172.21.15.83:0/2407685980 wait complete. 2024-04-24T09:01:18.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.793+0000 7ff3d5c00640 1 -- 172.21.15.83:0/3116973085 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3d0074cc0 msgr2=0x7ff3d0112970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:18.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.793+0000 7ff3d5c00640 1 --2- 172.21.15.83:0/3116973085 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3d0074cc0 0x7ff3d0112970 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7ff3b8009900 tx=0x7ff3b802a480 comp rx=0 tx=0).stop 2024-04-24T09:01:18.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.793+0000 7ff3d5c00640 1 -- 172.21.15.83:0/3116973085 shutdown_connections 2024-04-24T09:01:18.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.793+0000 7ff3d5c00640 1 --2- 172.21.15.83:0/3116973085 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff3d0112eb0 0x7ff3d01152a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.793+0000 7ff3d5c00640 1 --2- 172.21.15.83:0/3116973085 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3d0074cc0 0x7ff3d0112970 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.793+0000 7ff3d5c00640 1 --2- 172.21.15.83:0/3116973085 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff3d00743a0 0x7ff3d0074780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.793+0000 7ff3d5c00640 1 -- 172.21.15.83:0/3116973085 >> 172.21.15.83:0/3116973085 conn(0x7ff3d00fd9d0 msgr2=0x7ff3d00ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:18.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.793+0000 7ff3d5c00640 1 -- 172.21.15.83:0/3116973085 shutdown_connections 2024-04-24T09:01:18.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.793+0000 7ff3d5c00640 1 -- 172.21.15.83:0/3116973085 wait complete. 2024-04-24T09:01:18.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.794+0000 7ff3d5c00640 1 Processor -- start 2024-04-24T09:01:18.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.794+0000 7ff3d5c00640 1 -- start start 2024-04-24T09:01:18.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.795+0000 7ff3d5c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff3d00743a0 0x7ff3d01a48d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:18.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.795+0000 7ff3d5c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff3d0074cc0 0x7ff3d01a4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:18.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.795+0000 7ff3d5c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3d0112eb0 0x7ff3d019ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:18.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.795+0000 7ff3d5c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d0118780 con 0x7ff3d0112eb0 2024-04-24T09:01:18.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.795+0000 7ff3d5c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d0118600 con 0x7ff3d00743a0 2024-04-24T09:01:18.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.795+0000 7ff3d5c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d0118900 con 0x7ff3d0074cc0 2024-04-24T09:01:18.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.795+0000 7ff3d5200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3d0112eb0 0x7ff3d019ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:18.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.795+0000 7ff3d5200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3d0112eb0 0x7ff3d019ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49102/0 (socket says 172.21.15.83:49102) 2024-04-24T09:01:18.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.795+0000 7ff3d5200640 1 -- 172.21.15.83:0/161032363 learned_addr learned my addr 172.21.15.83:0/161032363 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:01:18.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.795+0000 7ff3cfe00640 1 --2- 172.21.15.83:0/161032363 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff3d00743a0 0x7ff3d01a48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:18.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.795+0000 7ff3c7e00640 1 --2- 172.21.15.83:0/161032363 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff3d0074cc0 0x7ff3d01a4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:18.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.795+0000 7ff3d5200640 1 -- 172.21.15.83:0/161032363 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff3d0074cc0 msgr2=0x7ff3d01a4e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:18.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.795+0000 7ff3d5200640 1 --2- 172.21.15.83:0/161032363 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff3d0074cc0 0x7ff3d01a4e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.795+0000 7ff3d5200640 1 -- 172.21.15.83:0/161032363 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff3d00743a0 msgr2=0x7ff3d01a48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:18.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.795+0000 7ff3d5200640 1 --2- 172.21.15.83:0/161032363 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff3d00743a0 0x7ff3d01a48d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.796+0000 7ff3d5200640 1 -- 172.21.15.83:0/161032363 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3d019f200 con 0x7ff3d0112eb0 2024-04-24T09:01:18.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.796+0000 7ff3c7e00640 1 --2- 172.21.15.83:0/161032363 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff3d0074cc0 0x7ff3d01a4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:01:18.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.796+0000 7ff3d5200640 1 --2- 172.21.15.83:0/161032363 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3d0112eb0 0x7ff3d019ea80 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7ff3c800e9d0 tx=0x7ff3c800ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:18.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.796+0000 7ff3cd600640 1 -- 172.21.15.83:0/161032363 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3c800cd80 con 0x7ff3d0112eb0 2024-04-24T09:01:18.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.796+0000 7ff3cd600640 1 -- 172.21.15.83:0/161032363 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff3c8005020 con 0x7ff3d0112eb0 2024-04-24T09:01:18.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.796+0000 7ff3d5c00640 1 -- 172.21.15.83:0/161032363 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3d019f4f0 con 0x7ff3d0112eb0 2024-04-24T09:01:18.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.796+0000 7ff3cd600640 1 -- 172.21.15.83:0/161032363 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3c80116d0 con 0x7ff3d0112eb0 2024-04-24T09:01:18.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.796+0000 7ff3d5c00640 1 -- 172.21.15.83:0/161032363 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3d01ab790 con 0x7ff3d0112eb0 2024-04-24T09:01:18.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.800+0000 7ff3d9fc1740 1 Processor -- start 2024-04-24T09:01:18.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.801+0000 7ff3cd600640 1 -- 172.21.15.83:0/161032363 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff3c80051d0 con 0x7ff3d0112eb0 2024-04-24T09:01:18.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.801+0000 7ff3d9fc1740 1 -- start start 2024-04-24T09:01:18.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.801+0000 7ff3d9fc1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563157e6eb40 0x563157e6ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:18.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.801+0000 7ff3d9fc1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563157e6f460 0x563157e795f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:18.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.801+0000 7ff3d9fc1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563157e79b30 0x563157e7bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:18.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.801+0000 7ff3d9fc1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x563157dd41d0 con 0x563157e6eb40 2024-04-24T09:01:18.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.801+0000 7ff3d9fc1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563157d7a450 con 0x563157e6f460 2024-04-24T09:01:18.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.801+0000 7ff3d9fc1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x563157dd0190 con 0x563157e79b30 2024-04-24T09:01:18.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.802+0000 7ff3cfe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563157e6f460 0x563157e795f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:18.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.802+0000 7ff3cfe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563157e6f460 0x563157e795f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:51970/0 (socket says 172.21.15.83:51970) 2024-04-24T09:01:18.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.802+0000 7ff3cfe00640 1 -- 172.21.15.83:0/3460604848 learned_addr learned my addr 172.21.15.83:0/3460604848 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:01:18.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.802+0000 7ff3c7e00640 1 --2- 172.21.15.83:0/3460604848 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563157e79b30 0x563157e7bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:18.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.802+0000 7ff3c7e00640 1 --2- 172.21.15.83:0/3460604848 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563157e6eb40 0x563157e6ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:18.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.803+0000 7ff3cd600640 1 --2- 172.21.15.83:0/161032363 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff3b40a3d20 0x7ff3b40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:18.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.803+0000 7ff3cfe00640 1 -- 172.21.15.83:0/3460604848 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563157e79b30 msgr2=0x563157e7bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:18.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.803+0000 7ff3cfe00640 1 --2- 172.21.15.83:0/3460604848 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563157e79b30 0x563157e7bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.803+0000 7ff3cfe00640 1 -- 172.21.15.83:0/3460604848 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563157e6eb40 msgr2=0x563157e6ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:18.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.803+0000 7ff3cfe00640 1 --2- 172.21.15.83:0/3460604848 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563157e6eb40 0x563157e6ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.803+0000 7ff3cfe00640 1 -- 172.21.15.83:0/3460604848 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563157e37310 con 0x563157e6f460 2024-04-24T09:01:18.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.803+0000 7ff3c7e00640 1 --2- 172.21.15.83:0/3460604848 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563157e79b30 0x563157e7bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:01:18.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.803+0000 7ff3cd600640 1 -- 172.21.15.83:0/161032363 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7ff3c80d0ee0 con 0x7ff3d0112eb0 2024-04-24T09:01:18.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.803+0000 7ff3d5200640 1 --2- 172.21.15.83:0/161032363 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff3b40a3d20 0x7ff3b40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:18.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.804+0000 7ff3cfe00640 1 --2- 172.21.15.83:0/3460604848 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563157e6f460 0x563157e795f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff3c000cc20 tx=0x7ff3c000e720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18d3e2c99a4d05dd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:18.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.804+0000 7ff3d5200640 1 --2- 172.21.15.83:0/161032363 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff3b40a3d20 0x7ff3b40a61e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ff3c80a5140 tx=0x7ff3c800c7f0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:18.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.804+0000 7ff3c5600640 1 -- 172.21.15.83:0/3460604848 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3c0014070 con 0x563157e6f460 2024-04-24T09:01:18.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.804+0000 7ff3c5600640 1 -- 172.21.15.83:0/3460604848 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff3c00047a0 con 0x563157e6f460 2024-04-24T09:01:18.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.804+0000 7ff3c5600640 1 -- 172.21.15.83:0/3460604848 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3c0010670 con 0x563157e6f460 2024-04-24T09:01:18.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.805+0000 7ff3d9fc1740 1 -- 172.21.15.83:0/3460604848 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563157e6f460 msgr2=0x563157e795f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:18.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.805+0000 7ff3d9fc1740 1 --2- 172.21.15.83:0/3460604848 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563157e6f460 0x563157e795f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff3c000cc20 tx=0x7ff3c000e720 comp rx=0 tx=0).stop 2024-04-24T09:01:18.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.805+0000 7ff3d9fc1740 1 -- 172.21.15.83:0/3460604848 shutdown_connections 2024-04-24T09:01:18.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.805+0000 7ff3d9fc1740 1 --2- 172.21.15.83:0/3460604848 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563157e79b30 0x563157e7bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.805+0000 7ff3d9fc1740 1 --2- 172.21.15.83:0/3460604848 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563157e6f460 0x563157e795f0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.805+0000 7ff3d9fc1740 1 --2- 172.21.15.83:0/3460604848 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563157e6eb40 0x563157e6ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.805+0000 7ff3d9fc1740 1 -- 172.21.15.83:0/3460604848 >> 172.21.15.83:0/3460604848 conn(0x563157e6ae60 msgr2=0x563157e6b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:18.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.805+0000 7ff3d9fc1740 1 -- 172.21.15.83:0/3460604848 shutdown_connections 2024-04-24T09:01:18.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.806+0000 7ff3d9fc1740 1 -- 172.21.15.83:0/3460604848 wait complete. 2024-04-24T09:01:18.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.806+0000 7ff3d9fc1740 1 -- start start 2024-04-24T09:01:18.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.806+0000 7ff3d9fc1740 1 Processor -- start 2024-04-24T09:01:18.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.807+0000 7ff3d9fc1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563157e6eb40 0x563157e795f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:18.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.807+0000 7ff3d9fc1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563157e6f460 0x563157e768e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:18.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.807+0000 7ff3d9fc1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563157e79b30 0x563157e77020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:18.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.807+0000 7ff3d9fc1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x563157e7c9c0 con 0x563157e6eb40 2024-04-24T09:01:18.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.807+0000 7ff3d9fc1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563157d8d810 con 0x563157e6f460 2024-04-24T09:01:18.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.807+0000 7ff3d9fc1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x563157e7cb40 con 0x563157e79b30 2024-04-24T09:01:18.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.807+0000 7ff3d9fc1740 10 client.0 _ll_register_callbacks cb 0x563157d5b140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:01:18.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.807+0000 7ff3c7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563157e6eb40 0x563157e795f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:18.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.807+0000 7ff3c7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563157e6eb40 0x563157e795f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49126/0 (socket says 172.21.15.83:49126) 2024-04-24T09:01:18.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.807+0000 7ff3c7e00640 1 -- 172.21.15.83:0/3830423994 learned_addr learned my addr 172.21.15.83:0/3830423994 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:01:18.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.807+0000 7ff3cfe00640 1 --2- 172.21.15.83:0/3830423994 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563157e6f460 0x563157e768e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:18.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.807+0000 7ff3c7e00640 1 -- 172.21.15.83:0/3830423994 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563157e79b30 msgr2=0x563157e77020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:18.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.807+0000 7ff3c7e00640 1 --2- 172.21.15.83:0/3830423994 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563157e79b30 0x563157e77020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.807+0000 7ff3c7e00640 1 -- 172.21.15.83:0/3830423994 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563157e6f460 msgr2=0x563157e768e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:18.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.807+0000 7ff3c7e00640 1 --2- 172.21.15.83:0/3830423994 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563157e6f460 0x563157e768e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.807+0000 7ff3c7e00640 1 -- 172.21.15.83:0/3830423994 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563157d9e3f0 con 0x563157e6eb40 2024-04-24T09:01:18.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.808+0000 7ff3c7e00640 1 --2- 172.21.15.83:0/3830423994 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563157e6eb40 0x563157e795f0 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7ff3b8009900 tx=0x7ff3b8003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2afca19a33b933e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:18.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.808+0000 7ff3be000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:01:18.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.808+0000 7ff3be000640 1 -- 172.21.15.83:0/3830423994 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b8035040 con 0x563157e6eb40 2024-04-24T09:01:18.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.808+0000 7ff3be000640 1 -- 172.21.15.83:0/3830423994 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7ff3b80314a0 con 0x563157e6eb40 2024-04-24T09:01:18.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.808+0000 7ff3d9fc1740 1 -- 172.21.15.83:0/3830423994 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3c00047a0 con 0x563157e6eb40 2024-04-24T09:01:18.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.808+0000 7ff3be000640 1 -- 172.21.15.83:0/3830423994 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7ff3b8050050 con 0x563157e6eb40 2024-04-24T09:01:18.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.808+0000 7ff3d9fc1740 10 client.19128 fetch_fsmap learned FSMap version 35 2024-04-24T09:01:18.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.808+0000 7ff3d9fc1740 1 -- 172.21.15.83:0/3830423994 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x563157ae63f0 con 0x563157e6eb40 2024-04-24T09:01:18.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.809+0000 7ff3be000640 1 -- 172.21.15.83:0/3830423994 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff3c00047a0 con 0x563157e6eb40 2024-04-24T09:01:18.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.809+0000 7ff3be000640 1 -- 172.21.15.83:0/3830423994 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7ff3b8047590 con 0x563157e6eb40 2024-04-24T09:01:18.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.809+0000 7ff3d9fc1740 10 client.19128 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T09:01:18.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.809+0000 7ff3d9fc1740 10 client.19128 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:01:18.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.809+0000 7ff3d9fc1740 20 client.19128 populate_metadata read hostname 'smithi083' 2024-04-24T09:01:18.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.809+0000 7ff3d9fc1740 1 --2- 172.21.15.83:0/3830423994 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x563157e89d90 0x563157e8c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:18.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.809+0000 7ff3d9fc1740 4 client.19128 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:01:18.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.809+0000 7ff3d9fc1740 1 -- 172.21.15.83:0/3830423994 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563157de1bf0 con 0x563157e89d90 2024-04-24T09:01:18.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.810+0000 7ff3cfe00640 1 --2- 172.21.15.83:0/3830423994 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x563157e89d90 0x563157e8c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:18.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.811+0000 7ff3be000640 10 client.19128 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:01:18.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.811+0000 7ff3cfe00640 1 --2- 172.21.15.83:0/3830423994 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x563157e89d90 0x563157e8c180 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59237fc3eea59cb0 server_cookie=217213088ab77f90 in_seq=0 out_seq=0 2024-04-24T09:01:18.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.812+0000 7ff3be000640 1 -- 172.21.15.83:0/3830423994 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563157de1bf0 con 0x563157e89d90 2024-04-24T09:01:18.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.812+0000 7ff3be000640 10 client.19128 handle_command_reply: tid=0 2024-04-24T09:01:18.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.812+0000 7ff3d9fc1740 1 client.19128 shutdown 2024-04-24T09:01:18.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.813+0000 7ff3d9fc1740 1 -- 172.21.15.83:0/3830423994 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563157e6eb40 msgr2=0x563157e795f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:18.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.813+0000 7ff3d9fc1740 1 --2- 172.21.15.83:0/3830423994 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563157e6eb40 0x563157e795f0 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7ff3b8009900 tx=0x7ff3b8003820 comp rx=0 tx=0).stop 2024-04-24T09:01:18.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.813+0000 7ff3d9fc1740 1 -- 172.21.15.83:0/3830423994 shutdown_connections 2024-04-24T09:01:18.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.813+0000 7ff3d9fc1740 1 --2- 172.21.15.83:0/3830423994 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x563157e89d90 0x563157e8c180 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.813+0000 7ff3d9fc1740 1 --2- 172.21.15.83:0/3830423994 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563157e79b30 0x563157e77020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.813+0000 7ff3d9fc1740 1 --2- 172.21.15.83:0/3830423994 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563157e6f460 0x563157e768e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.813+0000 7ff3d9fc1740 1 --2- 172.21.15.83:0/3830423994 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563157e6eb40 0x563157e795f0 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.813+0000 7ff3d9fc1740 1 -- 172.21.15.83:0/3830423994 >> 172.21.15.83:0/3830423994 conn(0x563157e6ae60 msgr2=0x563157e70b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:18.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.813+0000 7ff3be000640 0 client.19128 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:01:18.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.813+0000 7ff3d9fc1740 1 -- 172.21.15.83:0/3830423994 shutdown_connections 2024-04-24T09:01:18.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.813+0000 7ff3d9fc1740 1 -- 172.21.15.83:0/3830423994 wait complete. 2024-04-24T09:01:18.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.813+0000 7ff3d9fc1740 20 client.19128 trim_cache size 0 max 16384 2024-04-24T09:01:18.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.830+0000 7ff3d9fc1740 1 Processor -- start 2024-04-24T09:01:18.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.830+0000 7ff3d9fc1740 1 -- start start 2024-04-24T09:01:18.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.831+0000 7ff3d9fc1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563157e6f460 0x563157e858b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:18.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.831+0000 7ff3d9fc1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563157e79b30 0x563157e761a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:18.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.831+0000 7ff3d9fc1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563157e79120 0x563157e79580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:18.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.831+0000 7ff3d9fc1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d01b5a20 con 0x563157e6f460 2024-04-24T09:01:18.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.831+0000 7ff3d9fc1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d01b7560 con 0x563157e79b30 2024-04-24T09:01:18.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.831+0000 7ff3d9fc1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d01b60f0 con 0x563157e79120 2024-04-24T09:01:18.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.831+0000 7ff3c7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563157e6f460 0x563157e858b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:18.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.831+0000 7ff3c7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563157e6f460 0x563157e858b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52452/0 (socket says 172.21.15.83:52452) 2024-04-24T09:01:18.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.831+0000 7ff3c7e00640 1 -- 172.21.15.83:0/4001062806 learned_addr learned my addr 172.21.15.83:0/4001062806 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:01:18.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.831+0000 7ff3c7e00640 1 -- 172.21.15.83:0/4001062806 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563157e79120 msgr2=0x563157e79580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:18.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.831+0000 7ff3c7e00640 1 --2- 172.21.15.83:0/4001062806 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563157e79120 0x563157e79580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.831+0000 7ff3c7e00640 1 -- 172.21.15.83:0/4001062806 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563157e79b30 msgr2=0x563157e761a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:18.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.831+0000 7ff3c7e00640 1 --2- 172.21.15.83:0/4001062806 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563157e79b30 0x563157e761a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.831+0000 7ff3c7e00640 1 -- 172.21.15.83:0/4001062806 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563157bb07d0 con 0x563157e6f460 2024-04-24T09:01:18.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.832+0000 7ff3c7e00640 1 --2- 172.21.15.83:0/4001062806 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563157e6f460 0x563157e858b0 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7ff3b8004520 tx=0x7ff3b8005b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be07cfbd6d92d564 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:18.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.832+0000 7ff3c4c00640 1 -- 172.21.15.83:0/4001062806 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b8059070 con 0x563157e6f460 2024-04-24T09:01:18.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.832+0000 7ff3c4c00640 1 -- 172.21.15.83:0/4001062806 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff3b8042070 con 0x563157e6f460 2024-04-24T09:01:18.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.832+0000 7ff3c4c00640 1 -- 172.21.15.83:0/4001062806 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b8035040 con 0x563157e6f460 2024-04-24T09:01:18.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.833+0000 7ff3d9fc1740 1 -- 172.21.15.83:0/4001062806 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563157e6f460 msgr2=0x563157e858b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:18.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.833+0000 7ff3d9fc1740 1 --2- 172.21.15.83:0/4001062806 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563157e6f460 0x563157e858b0 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7ff3b8004520 tx=0x7ff3b8005b70 comp rx=0 tx=0).stop 2024-04-24T09:01:18.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.833+0000 7ff3d9fc1740 1 -- 172.21.15.83:0/4001062806 shutdown_connections 2024-04-24T09:01:18.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.833+0000 7ff3d9fc1740 1 --2- 172.21.15.83:0/4001062806 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563157e79120 0x563157e79580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.833+0000 7ff3d9fc1740 1 --2- 172.21.15.83:0/4001062806 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563157e79b30 0x563157e761a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.833+0000 7ff3d9fc1740 1 --2- 172.21.15.83:0/4001062806 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563157e6f460 0x563157e858b0 unknown :-1 s=CLOSED pgs=1657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.833+0000 7ff3d9fc1740 1 -- 172.21.15.83:0/4001062806 >> 172.21.15.83:0/4001062806 conn(0x563157e6ae60 msgr2=0x563157e6b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:18.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.833+0000 7ff3d9fc1740 1 -- 172.21.15.83:0/4001062806 shutdown_connections 2024-04-24T09:01:18.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.833+0000 7ff3d9fc1740 1 -- 172.21.15.83:0/4001062806 wait complete. 2024-04-24T09:01:18.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.833+0000 7ff3d9fc1740 1 -- start start 2024-04-24T09:01:18.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.833+0000 7ff3d9fc1740 1 Processor -- start 2024-04-24T09:01:18.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.834+0000 7ff3d9fc1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563157e6f460 0x563157e81dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:18.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.834+0000 7ff3d9fc1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563157e79120 0x563157e82300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:18.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.834+0000 7ff3d9fc1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563157e79b30 0x563157e73b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:18.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.834+0000 7ff3d9fc1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d01b6e90 con 0x563157e79b30 2024-04-24T09:01:18.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.834+0000 7ff3d9fc1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d01b67c0 con 0x563157e79120 2024-04-24T09:01:18.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.834+0000 7ff3d9fc1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d01b3a20 con 0x563157e6f460 2024-04-24T09:01:18.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.834+0000 7ff3d9fc1740 10 client.0 _ll_register_callbacks cb 0x7ff3b8004550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:01:18.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.834+0000 7ff3c7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563157e79b30 0x563157e73b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:18.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.834+0000 7ff3c7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563157e79b30 0x563157e73b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52464/0 (socket says 172.21.15.83:52464) 2024-04-24T09:01:18.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.834+0000 7ff3c7e00640 1 -- 172.21.15.83:0/775148118 learned_addr learned my addr 172.21.15.83:0/775148118 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:01:18.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.834+0000 7ff3cfe00640 1 --2- 172.21.15.83:0/775148118 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563157e79120 0x563157e82300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:18.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.835+0000 7ff3c7e00640 1 --2- 172.21.15.83:0/775148118 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563157e6f460 0x563157e81dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:18.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.835+0000 7ff3c7e00640 1 -- 172.21.15.83:0/775148118 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563157e6f460 msgr2=0x563157e81dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:18.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.835+0000 7ff3c7e00640 1 --2- 172.21.15.83:0/775148118 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563157e6f460 0x563157e81dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.835+0000 7ff3c7e00640 1 -- 172.21.15.83:0/775148118 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563157e79120 msgr2=0x563157e82300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:18.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.835+0000 7ff3c7e00640 1 --2- 172.21.15.83:0/775148118 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563157e79120 0x563157e82300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.835+0000 7ff3c7e00640 1 -- 172.21.15.83:0/775148118 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563157e74390 con 0x563157e79b30 2024-04-24T09:01:18.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.835+0000 7ff3cfe00640 1 --2- 172.21.15.83:0/775148118 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563157e79120 0x563157e82300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:01:18.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.835+0000 7ff3c7e00640 1 --2- 172.21.15.83:0/775148118 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563157e79b30 0x563157e73b80 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7ff3b8057040 tx=0x7ff3b8049040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f29f8c5dc7199a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:18.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.835+0000 7ff3be000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:01:18.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.835+0000 7ff3be000640 1 -- 172.21.15.83:0/775148118 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b8035070 con 0x563157e79b30 2024-04-24T09:01:18.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.835+0000 7ff3be000640 1 -- 172.21.15.83:0/775148118 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7ff3b8003e60 con 0x563157e79b30 2024-04-24T09:01:18.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.835+0000 7ff3d9fc1740 1 -- 172.21.15.83:0/775148118 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3b8042070 con 0x563157e79b30 2024-04-24T09:01:18.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.836+0000 7ff3be000640 1 -- 172.21.15.83:0/775148118 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7ff3b8059040 con 0x563157e79b30 2024-04-24T09:01:18.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.836+0000 7ff3d9fc1740 10 client.19134 fetch_fsmap learned FSMap version 35 2024-04-24T09:01:18.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.836+0000 7ff3d9fc1740 1 -- 172.21.15.83:0/775148118 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x563157e7ac10 con 0x563157e79b30 2024-04-24T09:01:18.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.836+0000 7ff3be000640 1 -- 172.21.15.83:0/775148118 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff3b8042070 con 0x563157e79b30 2024-04-24T09:01:18.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.836+0000 7ff3be000640 1 -- 172.21.15.83:0/775148118 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7ff3b80545f0 con 0x563157e79b30 2024-04-24T09:01:18.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.836+0000 7ff3d9fc1740 10 client.19134 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T09:01:18.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.836+0000 7ff3d9fc1740 10 client.19134 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:01:18.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.836+0000 7ff3d9fc1740 20 client.19134 populate_metadata read hostname 'smithi083' 2024-04-24T09:01:18.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.836+0000 7ff3d9fc1740 1 --2- 172.21.15.83:0/775148118 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x563157ef04d0 0x563157ef28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:18.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.836+0000 7ff3d9fc1740 4 client.19134 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:01:18.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.836+0000 7ff3d9fc1740 1 -- 172.21.15.83:0/775148118 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563157ddba60 con 0x563157ef04d0 2024-04-24T09:01:18.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.837+0000 7ff3cfe00640 1 --2- 172.21.15.83:0/775148118 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x563157ef04d0 0x563157ef28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:18.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.838+0000 7ff3cfe00640 1 --2- 172.21.15.83:0/775148118 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x563157ef04d0 0x563157ef28f0 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2d4dfc4400d52fe server_cookie=838c91917521db0a in_seq=0 out_seq=0 2024-04-24T09:01:18.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.838+0000 7ff3be000640 10 client.19134 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:01:18.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.841+0000 7ff3be000640 1 -- 172.21.15.83:0/775148118 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563157ddba60 con 0x563157ef04d0 2024-04-24T09:01:18.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.841+0000 7ff3be000640 10 client.19134 handle_command_reply: tid=0 2024-04-24T09:01:18.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.841+0000 7ff3d9fc1740 1 client.19134 shutdown 2024-04-24T09:01:18.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.841+0000 7ff3d9fc1740 1 -- 172.21.15.83:0/775148118 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563157e79b30 msgr2=0x563157e73b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:18.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.841+0000 7ff3d9fc1740 1 --2- 172.21.15.83:0/775148118 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563157e79b30 0x563157e73b80 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7ff3b8057040 tx=0x7ff3b8049040 comp rx=0 tx=0).stop 2024-04-24T09:01:18.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.841+0000 7ff3d9fc1740 1 -- 172.21.15.83:0/775148118 shutdown_connections 2024-04-24T09:01:18.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.841+0000 7ff3d9fc1740 1 --2- 172.21.15.83:0/775148118 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x563157ef04d0 0x563157ef28f0 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.841+0000 7ff3d9fc1740 1 --2- 172.21.15.83:0/775148118 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563157e79b30 0x563157e73b80 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.841+0000 7ff3d9fc1740 1 --2- 172.21.15.83:0/775148118 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563157e79120 0x563157e82300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.841+0000 7ff3be000640 0 client.19134 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:01:18.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.841+0000 7ff3d9fc1740 1 --2- 172.21.15.83:0/775148118 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563157e6f460 0x563157e81dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.841+0000 7ff3d9fc1740 1 -- 172.21.15.83:0/775148118 >> 172.21.15.83:0/775148118 conn(0x563157e6ae60 msgr2=0x563157e696c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:18.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.841+0000 7ff3d9fc1740 1 -- 172.21.15.83:0/775148118 shutdown_connections 2024-04-24T09:01:18.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.841+0000 7ff3d9fc1740 1 -- 172.21.15.83:0/775148118 wait complete. 2024-04-24T09:01:18.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.841+0000 7ff3d9fc1740 20 client.19134 trim_cache size 0 max 16384 2024-04-24T09:01:18.842 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:01:18.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.842+0000 7ff3d5c00640 1 -- 172.21.15.83:0/161032363 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff3b40a3d20 msgr2=0x7ff3b40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:18.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.842+0000 7ff3d5c00640 1 --2- 172.21.15.83:0/161032363 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff3b40a3d20 0x7ff3b40a61e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ff3c80a5140 tx=0x7ff3c800c7f0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.842+0000 7ff3d5c00640 1 -- 172.21.15.83:0/161032363 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3d0112eb0 msgr2=0x7ff3d019ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:18.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.842+0000 7ff3d5c00640 1 --2- 172.21.15.83:0/161032363 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3d0112eb0 0x7ff3d019ea80 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7ff3c800e9d0 tx=0x7ff3c800ee90 comp rx=0 tx=0).stop 2024-04-24T09:01:18.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.842+0000 7ff3d5c00640 1 -- 172.21.15.83:0/161032363 shutdown_connections 2024-04-24T09:01:18.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.842+0000 7ff3d5c00640 1 --2- 172.21.15.83:0/161032363 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff3b40a3d20 0x7ff3b40a61e0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.842+0000 7ff3d5c00640 1 --2- 172.21.15.83:0/161032363 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3d0112eb0 0x7ff3d019ea80 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.842+0000 7ff3d5c00640 1 --2- 172.21.15.83:0/161032363 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff3d0074cc0 0x7ff3d01a4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.842+0000 7ff3d5c00640 1 --2- 172.21.15.83:0/161032363 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff3d00743a0 0x7ff3d01a48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:18.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.842+0000 7ff3d5c00640 1 -- 172.21.15.83:0/161032363 >> 172.21.15.83:0/161032363 conn(0x7ff3d00fd9d0 msgr2=0x7ff3d00fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:18.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.842+0000 7ff3d5c00640 1 -- 172.21.15.83:0/161032363 shutdown_connections 2024-04-24T09:01:18.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:18.842+0000 7ff3d5c00640 1 -- 172.21.15.83:0/161032363 wait complete. 2024-04-24T09:01:19.606 INFO:tasks.workunit.client.0.smithi083.stdout: 1 56209 0.63 MB/sec execute 353 sec latency 62.384 ms 2024-04-24T09:01:19.861 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T09:01:19.862 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T09:01:19.862 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T09:01:20.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.094+0000 7f7232200640 1 -- 172.21.15.83:0/459823251 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f722c103060 msgr2=0x7f722c103440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:20.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.094+0000 7f7232200640 1 --2- 172.21.15.83:0/459823251 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f722c103060 0x7f722c103440 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f7220009900 tx=0x7f722002a480 comp rx=0 tx=0).stop 2024-04-24T09:01:20.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.094+0000 7f7232200640 1 -- 172.21.15.83:0/459823251 shutdown_connections 2024-04-24T09:01:20.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.094+0000 7f7232200640 1 --2- 172.21.15.83:0/459823251 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f722c10c480 0x7f722c10e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:20.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.094+0000 7f7232200640 1 --2- 172.21.15.83:0/459823251 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f722c103980 0x7f722c10be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:20.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.094+0000 7f7232200640 1 --2- 172.21.15.83:0/459823251 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f722c103060 0x7f722c103440 unknown :-1 s=CLOSED pgs=1659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:20.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.094+0000 7f7232200640 1 -- 172.21.15.83:0/459823251 >> 172.21.15.83:0/459823251 conn(0x7f722c0fc880 msgr2=0x7f722c0feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:20.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.094+0000 7f7232200640 1 -- 172.21.15.83:0/459823251 shutdown_connections 2024-04-24T09:01:20.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.094+0000 7f7232200640 1 -- 172.21.15.83:0/459823251 wait complete. 2024-04-24T09:01:20.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.095+0000 7f7232200640 1 Processor -- start 2024-04-24T09:01:20.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.096+0000 7f7232200640 1 -- start start 2024-04-24T09:01:20.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.096+0000 7f7232200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f722c103060 0x7f722c1a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:20.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.096+0000 7f7232200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f722c103980 0x7f722c1a0b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:20.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.096+0000 7f7232200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f722c10c480 0x7f722c19a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:20.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.096+0000 7f7232200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f722c1120f0 con 0x7f722c103980 2024-04-24T09:01:20.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.096+0000 7f7232200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f722c111f70 con 0x7f722c103060 2024-04-24T09:01:20.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.096+0000 7f7232200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f722c112270 con 0x7f722c10c480 2024-04-24T09:01:20.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.096+0000 7f722be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f722c103980 0x7f722c1a0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:20.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.096+0000 7f722be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f722c103980 0x7f722c1a0b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52474/0 (socket says 172.21.15.83:52474) 2024-04-24T09:01:20.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.096+0000 7f722be00640 1 -- 172.21.15.83:0/458298761 learned_addr learned my addr 172.21.15.83:0/458298761 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:01:20.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.097+0000 7f7231800640 1 --2- 172.21.15.83:0/458298761 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f722c10c480 0x7f722c19a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:20.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.097+0000 7f7230e00640 1 --2- 172.21.15.83:0/458298761 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f722c103060 0x7f722c1a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:20.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.097+0000 7f722be00640 1 -- 172.21.15.83:0/458298761 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f722c10c480 msgr2=0x7f722c19a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:20.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.097+0000 7f722be00640 1 --2- 172.21.15.83:0/458298761 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f722c10c480 0x7f722c19a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:20.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.097+0000 7f722be00640 1 -- 172.21.15.83:0/458298761 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f722c103060 msgr2=0x7f722c1a05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:20.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.097+0000 7f722be00640 1 --2- 172.21.15.83:0/458298761 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f722c103060 0x7f722c1a05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:20.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.097+0000 7f722be00640 1 -- 172.21.15.83:0/458298761 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f722c19af70 con 0x7f722c103980 2024-04-24T09:01:20.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.097+0000 7f7230e00640 1 --2- 172.21.15.83:0/458298761 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f722c103060 0x7f722c1a05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:01:20.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.097+0000 7f722be00640 1 --2- 172.21.15.83:0/458298761 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f722c103980 0x7f722c1a0b30 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f721800c970 tx=0x7f721800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:20.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.097+0000 7f7229600640 1 -- 172.21.15.83:0/458298761 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7218007bb0 con 0x7f722c103980 2024-04-24T09:01:20.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.098+0000 7f7229600640 1 -- 172.21.15.83:0/458298761 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7218007d50 con 0x7f722c103980 2024-04-24T09:01:20.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.098+0000 7f7232200640 1 -- 172.21.15.83:0/458298761 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f722c19b260 con 0x7f722c103980 2024-04-24T09:01:20.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.098+0000 7f7229600640 1 -- 172.21.15.83:0/458298761 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7218018780 con 0x7f722c103980 2024-04-24T09:01:20.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.098+0000 7f7232200640 1 -- 172.21.15.83:0/458298761 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f722c1a7570 con 0x7f722c103980 2024-04-24T09:01:20.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.101+0000 7f7229600640 1 -- 172.21.15.83:0/458298761 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f721800f450 con 0x7f722c103980 2024-04-24T09:01:20.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.101+0000 7f7236794740 1 Processor -- start 2024-04-24T09:01:20.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.101+0000 7f7236794740 1 -- start start 2024-04-24T09:01:20.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.102+0000 7f7236794740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a143343270 0x55a143343650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:20.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.102+0000 7f7236794740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a143343b90 0x55a14334dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:20.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.102+0000 7f7236794740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a14334e260 0x55a143350650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:20.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.102+0000 7f7236794740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a1432a8e70 con 0x55a14334e260 2024-04-24T09:01:20.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.102+0000 7f7236794740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a14324f380 con 0x55a143343b90 2024-04-24T09:01:20.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.102+0000 7f7236794740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a1432a4e30 con 0x55a143343270 2024-04-24T09:01:20.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.102+0000 7f722be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a14334e260 0x55a143350650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:20.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.102+0000 7f722be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a14334e260 0x55a143350650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52478/0 (socket says 172.21.15.83:52478) 2024-04-24T09:01:20.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.102+0000 7f722be00640 1 -- 172.21.15.83:0/3210969487 learned_addr learned my addr 172.21.15.83:0/3210969487 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:01:20.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.102+0000 7f7230e00640 1 --2- 172.21.15.83:0/3210969487 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a143343b90 0x55a14334dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:20.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.102+0000 7f7231800640 1 --2- 172.21.15.83:0/3210969487 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a143343270 0x55a143343650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:20.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.103+0000 7f722be00640 1 -- 172.21.15.83:0/3210969487 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a143343270 msgr2=0x55a143343650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:20.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.103+0000 7f722be00640 1 --2- 172.21.15.83:0/3210969487 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a143343270 0x55a143343650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:20.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.103+0000 7f722be00640 1 -- 172.21.15.83:0/3210969487 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a143343b90 msgr2=0x55a14334dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:20.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.103+0000 7f722be00640 1 --2- 172.21.15.83:0/3210969487 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a143343b90 0x55a14334dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:20.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.103+0000 7f722be00640 1 -- 172.21.15.83:0/3210969487 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1430837e0 con 0x55a14334e260 2024-04-24T09:01:20.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.103+0000 7f7231800640 1 --2- 172.21.15.83:0/3210969487 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a143343270 0x55a143343650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:01:20.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.103+0000 7f7229600640 1 --2- 172.21.15.83:0/458298761 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f72140a3da0 0x7f72140a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:20.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.103+0000 7f722be00640 1 --2- 172.21.15.83:0/3210969487 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a14334e260 0x55a143350650 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f72180cdf90 tx=0x7f72180d9460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76d1508ef4bc3875 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:20.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.104+0000 7f7231800640 1 --2- 172.21.15.83:0/458298761 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f72140a3da0 0x7f72140a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:20.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.104+0000 7f7229600640 1 -- 172.21.15.83:0/458298761 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7f72180d0cf0 con 0x7f722c103980 2024-04-24T09:01:20.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.104+0000 7f721e000640 1 -- 172.21.15.83:0/3210969487 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7218098b10 con 0x55a14334e260 2024-04-24T09:01:20.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.104+0000 7f721e000640 1 -- 172.21.15.83:0/3210969487 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f721800fa90 con 0x55a14334e260 2024-04-24T09:01:20.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.104+0000 7f721e000640 1 -- 172.21.15.83:0/3210969487 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f721809c1d0 con 0x55a14334e260 2024-04-24T09:01:20.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.105+0000 7f7236794740 1 -- 172.21.15.83:0/3210969487 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a14334e260 msgr2=0x55a143350650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:20.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.105+0000 7f7231800640 1 --2- 172.21.15.83:0/458298761 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f72140a3da0 0x7f72140a6260 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f72240092f0 tx=0x7f7224009280 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:20.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.105+0000 7f7236794740 1 --2- 172.21.15.83:0/3210969487 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a14334e260 0x55a143350650 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f72180cdf90 tx=0x7f72180d9460 comp rx=0 tx=0).stop 2024-04-24T09:01:20.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.105+0000 7f7236794740 1 -- 172.21.15.83:0/3210969487 shutdown_connections 2024-04-24T09:01:20.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.105+0000 7f7236794740 1 --2- 172.21.15.83:0/3210969487 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a14334e260 0x55a143350650 unknown :-1 s=CLOSED pgs=1661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:20.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.105+0000 7f7236794740 1 --2- 172.21.15.83:0/3210969487 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a143343b90 0x55a14334dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:20.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.105+0000 7f7236794740 1 --2- 172.21.15.83:0/3210969487 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a143343270 0x55a143343650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:20.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.105+0000 7f7236794740 1 -- 172.21.15.83:0/3210969487 >> 172.21.15.83:0/3210969487 conn(0x55a14333f590 msgr2=0x55a14333f970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:20.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.105+0000 7f7236794740 1 -- 172.21.15.83:0/3210969487 shutdown_connections 2024-04-24T09:01:20.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.105+0000 7f7236794740 1 -- 172.21.15.83:0/3210969487 wait complete. 2024-04-24T09:01:20.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.106+0000 7f7236794740 1 -- start start 2024-04-24T09:01:20.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.106+0000 7f7236794740 1 Processor -- start 2024-04-24T09:01:20.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.106+0000 7f7236794740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a143343270 0x55a14334b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:20.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.106+0000 7f7236794740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a143343b90 0x55a14334b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:20.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.106+0000 7f7236794740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a14334e260 0x55a14334bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:20.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.106+0000 7f7236794740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a143351050 con 0x55a14334e260 2024-04-24T09:01:20.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.106+0000 7f7236794740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a143262660 con 0x55a143343270 2024-04-24T09:01:20.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.106+0000 7f7236794740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a1433511d0 con 0x55a143343b90 2024-04-24T09:01:20.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.106+0000 7f7236794740 10 client.0 _ll_register_callbacks cb 0x55a143250aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:01:20.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.107+0000 7f7231800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a14334e260 0x55a14334bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:20.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.107+0000 7f7231800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a14334e260 0x55a14334bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52480/0 (socket says 172.21.15.83:52480) 2024-04-24T09:01:20.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.107+0000 7f7231800640 1 -- 172.21.15.83:0/1585300304 learned_addr learned my addr 172.21.15.83:0/1585300304 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:01:20.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.107+0000 7f7231800640 1 -- 172.21.15.83:0/1585300304 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a143343b90 msgr2=0x55a14334b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:20.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.107+0000 7f7231800640 1 --2- 172.21.15.83:0/1585300304 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a143343b90 0x55a14334b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:20.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.107+0000 7f7231800640 1 -- 172.21.15.83:0/1585300304 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a143343270 msgr2=0x55a14334b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:20.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.107+0000 7f7231800640 1 --2- 172.21.15.83:0/1585300304 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a143343270 0x55a14334b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:20.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.107+0000 7f7231800640 1 -- 172.21.15.83:0/1585300304 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1430e3f70 con 0x55a14334e260 2024-04-24T09:01:20.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.107+0000 7f7231800640 1 --2- 172.21.15.83:0/1585300304 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a14334e260 0x55a14334bc70 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f7224005cc0 tx=0x7f7224004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b7817c024df1f72 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:20.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.107+0000 7f7212a00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:01:20.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.108+0000 7f7212a00640 1 -- 172.21.15.83:0/1585300304 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f722400ead0 con 0x55a14334e260 2024-04-24T09:01:20.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.108+0000 7f7236794740 1 -- 172.21.15.83:0/1585300304 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a1432b6890 con 0x55a14334e260 2024-04-24T09:01:20.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.108+0000 7f7212a00640 1 -- 172.21.15.83:0/1585300304 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7f722400ec70 con 0x55a14334e260 2024-04-24T09:01:20.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.108+0000 7f7212a00640 1 -- 172.21.15.83:0/1585300304 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f7224030050 con 0x55a14334e260 2024-04-24T09:01:20.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.108+0000 7f7236794740 10 client.19146 fetch_fsmap learned FSMap version 35 2024-04-24T09:01:20.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.108+0000 7f7236794740 1 -- 172.21.15.83:0/1585300304 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55a142f7b170 con 0x55a14334e260 2024-04-24T09:01:20.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.108+0000 7f7212a00640 1 -- 172.21.15.83:0/1585300304 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55a1432b6890 con 0x55a14334e260 2024-04-24T09:01:20.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.108+0000 7f7212a00640 1 -- 172.21.15.83:0/1585300304 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7f722400f040 con 0x55a14334e260 2024-04-24T09:01:20.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.109+0000 7f7236794740 10 client.19146 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T09:01:20.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.109+0000 7f7236794740 10 client.19146 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:01:20.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.109+0000 7f7236794740 20 client.19146 populate_metadata read hostname 'smithi083' 2024-04-24T09:01:20.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.109+0000 7f7236794740 1 --2- 172.21.15.83:0/1585300304 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55a14335e6c0 0x55a143360b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:20.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.109+0000 7f7236794740 4 client.19146 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:01:20.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.109+0000 7f7236794740 1 -- 172.21.15.83:0/1585300304 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a1432b0700 con 0x55a14335e6c0 2024-04-24T09:01:20.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.109+0000 7f7230e00640 1 --2- 172.21.15.83:0/1585300304 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55a14335e6c0 0x55a143360b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:20.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.110+0000 7f7230e00640 1 --2- 172.21.15.83:0/1585300304 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55a14335e6c0 0x55a143360b50 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e656ef371197030 server_cookie=8e0b5190da06efc in_seq=0 out_seq=0 2024-04-24T09:01:20.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.110+0000 7f7212a00640 10 client.19146 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:01:20.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.113+0000 7f7212a00640 1 -- 172.21.15.83:0/1585300304 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a1432b0700 con 0x55a14335e6c0 2024-04-24T09:01:20.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.113+0000 7f7212a00640 10 client.19146 handle_command_reply: tid=0 2024-04-24T09:01:20.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.113+0000 7f7236794740 1 client.19146 shutdown 2024-04-24T09:01:20.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.114+0000 7f7236794740 1 -- 172.21.15.83:0/1585300304 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a14334e260 msgr2=0x55a14334bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:20.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.114+0000 7f7236794740 1 --2- 172.21.15.83:0/1585300304 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a14334e260 0x55a14334bc70 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f7224005cc0 tx=0x7f7224004a40 comp rx=0 tx=0).stop 2024-04-24T09:01:20.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.114+0000 7f7236794740 1 -- 172.21.15.83:0/1585300304 shutdown_connections 2024-04-24T09:01:20.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.114+0000 7f7236794740 1 --2- 172.21.15.83:0/1585300304 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55a14335e6c0 0x55a143360b50 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:20.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.114+0000 7f7236794740 1 --2- 172.21.15.83:0/1585300304 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a14334e260 0x55a14334bc70 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:20.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.114+0000 7f7236794740 1 --2- 172.21.15.83:0/1585300304 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a143343b90 0x55a14334b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:20.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.114+0000 7f7236794740 1 --2- 172.21.15.83:0/1585300304 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a143343270 0x55a14334b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:20.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.114+0000 7f7212a00640 0 client.19146 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:01:20.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.114+0000 7f7236794740 1 -- 172.21.15.83:0/1585300304 >> 172.21.15.83:0/1585300304 conn(0x55a14333f590 msgr2=0x55a143345390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:20.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.114+0000 7f7236794740 1 -- 172.21.15.83:0/1585300304 shutdown_connections 2024-04-24T09:01:20.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.114+0000 7f7236794740 1 -- 172.21.15.83:0/1585300304 wait complete. 2024-04-24T09:01:20.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.114+0000 7f7236794740 20 client.19146 trim_cache size 0 max 16384 2024-04-24T09:01:20.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.135+0000 7f7236794740 1 Processor -- start 2024-04-24T09:01:20.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.135+0000 7f7236794740 1 -- start start 2024-04-24T09:01:20.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.135+0000 7f7236794740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a143343270 0x55a143359f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:20.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.135+0000 7f7236794740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a143343b90 0x55a1433459d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:20.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.135+0000 7f7236794740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a14334e260 0x55a143366420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:20.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.135+0000 7f7236794740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f722c1b1680 con 0x55a143343270 2024-04-24T09:01:20.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.135+0000 7f7236794740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f722c1b31c0 con 0x55a14334e260 2024-04-24T09:01:20.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.135+0000 7f7236794740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f722c1b1d50 con 0x55a143343b90 2024-04-24T09:01:20.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.136+0000 7f7230e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a143343270 0x55a143359f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:20.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.136+0000 7f7230e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a143343270 0x55a143359f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52486/0 (socket says 172.21.15.83:52486) 2024-04-24T09:01:20.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.136+0000 7f7230e00640 1 -- 172.21.15.83:0/958147491 learned_addr learned my addr 172.21.15.83:0/958147491 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:01:20.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.136+0000 7f722be00640 1 --2- 172.21.15.83:0/958147491 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a143343b90 0x55a1433459d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:20.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.136+0000 7f7231800640 1 --2- 172.21.15.83:0/958147491 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a14334e260 0x55a143366420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:20.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.136+0000 7f7230e00640 1 -- 172.21.15.83:0/958147491 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a143343b90 msgr2=0x55a1433459d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:20.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.136+0000 7f7230e00640 1 --2- 172.21.15.83:0/958147491 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a143343b90 0x55a1433459d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:20.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.136+0000 7f7230e00640 1 -- 172.21.15.83:0/958147491 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a14334e260 msgr2=0x55a143366420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:20.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.136+0000 7f7230e00640 1 --2- 172.21.15.83:0/958147491 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a14334e260 0x55a143366420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:20.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.136+0000 7f7230e00640 1 -- 172.21.15.83:0/958147491 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a143339ac0 con 0x55a143343270 2024-04-24T09:01:20.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.137+0000 7f7230e00640 1 --2- 172.21.15.83:0/958147491 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a143343270 0x55a143359f40 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f7220009900 tx=0x7f7220004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51f6bb08f30b96d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:20.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.137+0000 7f721d600640 1 -- 172.21.15.83:0/958147491 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7220035040 con 0x55a143343270 2024-04-24T09:01:20.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.137+0000 7f721d600640 1 -- 172.21.15.83:0/958147491 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7220005040 con 0x55a143343270 2024-04-24T09:01:20.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.137+0000 7f721d600640 1 -- 172.21.15.83:0/958147491 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7220004b80 con 0x55a143343270 2024-04-24T09:01:20.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.138+0000 7f7236794740 1 -- 172.21.15.83:0/958147491 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a143343270 msgr2=0x55a143359f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:20.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.138+0000 7f7236794740 1 --2- 172.21.15.83:0/958147491 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a143343270 0x55a143359f40 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f7220009900 tx=0x7f7220004040 comp rx=0 tx=0).stop 2024-04-24T09:01:20.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.138+0000 7f7236794740 1 -- 172.21.15.83:0/958147491 shutdown_connections 2024-04-24T09:01:20.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.138+0000 7f7236794740 1 --2- 172.21.15.83:0/958147491 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a14334e260 0x55a143366420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:20.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.138+0000 7f7236794740 1 --2- 172.21.15.83:0/958147491 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a143343b90 0x55a1433459d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:20.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.138+0000 7f7236794740 1 --2- 172.21.15.83:0/958147491 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a143343270 0x55a143359f40 unknown :-1 s=CLOSED pgs=1663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:20.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.138+0000 7f7236794740 1 -- 172.21.15.83:0/958147491 >> 172.21.15.83:0/958147491 conn(0x55a14333f590 msgr2=0x55a14333e6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:20.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.138+0000 7f7236794740 1 -- 172.21.15.83:0/958147491 shutdown_connections 2024-04-24T09:01:20.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.138+0000 7f7236794740 1 -- 172.21.15.83:0/958147491 wait complete. 2024-04-24T09:01:20.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.138+0000 7f7236794740 1 -- start start 2024-04-24T09:01:20.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.139+0000 7f7236794740 1 Processor -- start 2024-04-24T09:01:20.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.139+0000 7f7236794740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a143343270 0x55a143352670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:20.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.139+0000 7f7236794740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a143343b90 0x55a143352bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:20.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.139+0000 7f7236794740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a14334e260 0x55a1433c3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:20.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.139+0000 7f7236794740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f722c1b2af0 con 0x55a143343b90 2024-04-24T09:01:20.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.139+0000 7f7236794740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f722c1b2420 con 0x55a143343270 2024-04-24T09:01:20.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.139+0000 7f7236794740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f722c1af680 con 0x55a14334e260 2024-04-24T09:01:20.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.139+0000 7f7236794740 10 client.0 _ll_register_callbacks cb 0x7f72240096e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:01:20.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.140+0000 7f722be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a143343b90 0x55a143352bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:20.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.140+0000 7f722be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a143343b90 0x55a143352bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52496/0 (socket says 172.21.15.83:52496) 2024-04-24T09:01:20.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.140+0000 7f722be00640 1 -- 172.21.15.83:0/2868520944 learned_addr learned my addr 172.21.15.83:0/2868520944 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:01:20.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.140+0000 7f7230e00640 1 --2- 172.21.15.83:0/2868520944 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a143343270 0x55a143352670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:20.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.140+0000 7f7231800640 1 --2- 172.21.15.83:0/2868520944 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a14334e260 0x55a1433c3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:20.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.140+0000 7f722be00640 1 -- 172.21.15.83:0/2868520944 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a14334e260 msgr2=0x55a1433c3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:20.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.140+0000 7f722be00640 1 --2- 172.21.15.83:0/2868520944 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a14334e260 0x55a1433c3c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:20.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.140+0000 7f722be00640 1 -- 172.21.15.83:0/2868520944 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a143343270 msgr2=0x55a143352670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:20.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.140+0000 7f722be00640 1 --2- 172.21.15.83:0/2868520944 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a143343270 0x55a143352670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:20.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.140+0000 7f722be00640 1 -- 172.21.15.83:0/2868520944 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1433c4200 con 0x55a143343b90 2024-04-24T09:01:20.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.141+0000 7f722be00640 1 --2- 172.21.15.83:0/2868520944 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a143343b90 0x55a143352bb0 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f72180ce620 tx=0x7f72180d4e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=398fff30533653dd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:20.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.141+0000 7f7212a00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:01:20.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.141+0000 7f7212a00640 1 -- 172.21.15.83:0/2868520944 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f721809c3b0 con 0x55a143343b90 2024-04-24T09:01:20.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.141+0000 7f7212a00640 1 -- 172.21.15.83:0/2868520944 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7f72180d4a20 con 0x55a143343b90 2024-04-24T09:01:20.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.141+0000 7f7236794740 1 -- 172.21.15.83:0/2868520944 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7220005040 con 0x55a143343b90 2024-04-24T09:01:20.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.141+0000 7f7212a00640 1 -- 172.21.15.83:0/2868520944 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f721809c640 con 0x55a143343b90 2024-04-24T09:01:20.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.141+0000 7f7236794740 10 client.19152 fetch_fsmap learned FSMap version 35 2024-04-24T09:01:20.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.141+0000 7f7236794740 1 -- 172.21.15.83:0/2868520944 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55a1433c5540 con 0x55a143343b90 2024-04-24T09:01:20.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.141+0000 7f7212a00640 1 -- 172.21.15.83:0/2868520944 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7220005040 con 0x55a143343b90 2024-04-24T09:01:20.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.142+0000 7f7212a00640 1 -- 172.21.15.83:0/2868520944 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7f721800f740 con 0x55a143343b90 2024-04-24T09:01:20.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.142+0000 7f7236794740 10 client.19152 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T09:01:20.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.142+0000 7f7236794740 10 client.19152 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:01:20.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.142+0000 7f7236794740 20 client.19152 populate_metadata read hostname 'smithi083' 2024-04-24T09:01:20.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.142+0000 7f7236794740 1 --2- 172.21.15.83:0/2868520944 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55a1433c5870 0x55a1433c7c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:20.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.142+0000 7f7236794740 4 client.19152 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T09:01:20.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.142+0000 7f7236794740 1 -- 172.21.15.83:0/2868520944 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a14322c980 con 0x55a1433c5870 2024-04-24T09:01:20.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.142+0000 7f7230e00640 1 --2- 172.21.15.83:0/2868520944 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55a1433c5870 0x55a1433c7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:20.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.143+0000 7f7230e00640 1 --2- 172.21.15.83:0/2868520944 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55a1433c5870 0x55a1433c7c60 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13631fa3d3d866f2 server_cookie=c1669aff68feaed in_seq=0 out_seq=0 2024-04-24T09:01:20.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.143+0000 7f7212a00640 10 client.19152 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:01:20.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.147+0000 7f7212a00640 1 -- 172.21.15.83:0/2868520944 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a14322c980 con 0x55a1433c5870 2024-04-24T09:01:20.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.147+0000 7f7212a00640 10 client.19152 handle_command_reply: tid=0 2024-04-24T09:01:20.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.147+0000 7f7236794740 1 client.19152 shutdown 2024-04-24T09:01:20.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.147+0000 7f7236794740 1 -- 172.21.15.83:0/2868520944 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a143343b90 msgr2=0x55a143352bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:20.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.147+0000 7f7236794740 1 --2- 172.21.15.83:0/2868520944 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a143343b90 0x55a143352bb0 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f72180ce620 tx=0x7f72180d4e30 comp rx=0 tx=0).stop 2024-04-24T09:01:20.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.147+0000 7f7236794740 1 -- 172.21.15.83:0/2868520944 shutdown_connections 2024-04-24T09:01:20.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.147+0000 7f7236794740 1 --2- 172.21.15.83:0/2868520944 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55a1433c5870 0x55a1433c7c60 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:20.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.147+0000 7f7236794740 1 --2- 172.21.15.83:0/2868520944 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a14334e260 0x55a1433c3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:20.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.147+0000 7f7236794740 1 --2- 172.21.15.83:0/2868520944 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a143343b90 0x55a143352bb0 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:20.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.147+0000 7f7236794740 1 --2- 172.21.15.83:0/2868520944 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a143343270 0x55a143352670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:20.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.147+0000 7f7236794740 1 -- 172.21.15.83:0/2868520944 >> 172.21.15.83:0/2868520944 conn(0x55a14333f590 msgr2=0x55a14334b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:20.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.147+0000 7f7212a00640 0 client.19152 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:01:20.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.148+0000 7f7236794740 1 -- 172.21.15.83:0/2868520944 shutdown_connections 2024-04-24T09:01:20.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.148+0000 7f7236794740 1 -- 172.21.15.83:0/2868520944 wait complete. 2024-04-24T09:01:20.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.148+0000 7f7236794740 20 client.19152 trim_cache size 0 max 16384 2024-04-24T09:01:20.149 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T09:01:20.149 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-04-24T09:01:20.149 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "c8c021c2-94f4-40e6-984e-11bdd767f715", 2024-04-24T09:01:20.149 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-04-24T09:01:20.149 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T09:01:20.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.148+0000 7f7232200640 1 -- 172.21.15.83:0/458298761 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f72140a3da0 msgr2=0x7f72140a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:20.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.148+0000 7f7232200640 1 --2- 172.21.15.83:0/458298761 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f72140a3da0 0x7f72140a6260 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f72240092f0 tx=0x7f7224009280 comp rx=0 tx=0).stop 2024-04-24T09:01:20.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.148+0000 7f7232200640 1 -- 172.21.15.83:0/458298761 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f722c103980 msgr2=0x7f722c1a0b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:20.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.148+0000 7f7232200640 1 --2- 172.21.15.83:0/458298761 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f722c103980 0x7f722c1a0b30 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f721800c970 tx=0x7f721800ce30 comp rx=0 tx=0).stop 2024-04-24T09:01:20.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.149+0000 7f7232200640 1 -- 172.21.15.83:0/458298761 shutdown_connections 2024-04-24T09:01:20.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.149+0000 7f7232200640 1 --2- 172.21.15.83:0/458298761 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f72140a3da0 0x7f72140a6260 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:20.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.149+0000 7f7232200640 1 --2- 172.21.15.83:0/458298761 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f722c10c480 0x7f722c19a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:20.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.149+0000 7f7232200640 1 --2- 172.21.15.83:0/458298761 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f722c103980 0x7f722c1a0b30 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:20.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.149+0000 7f7232200640 1 --2- 172.21.15.83:0/458298761 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f722c103060 0x7f722c1a05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:20.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.149+0000 7f7232200640 1 -- 172.21.15.83:0/458298761 >> 172.21.15.83:0/458298761 conn(0x7f722c0fc880 msgr2=0x7f722c101e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:20.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.149+0000 7f7232200640 1 -- 172.21.15.83:0/458298761 shutdown_connections 2024-04-24T09:01:20.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.149+0000 7f7232200640 1 -- 172.21.15.83:0/458298761 wait complete. 2024-04-24T09:01:20.166 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T09:01:20.166 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T09:01:20.233 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:01:19 smithi083 ceph-mon[27907]: pgmap v304: 129 pgs: 129 active+clean; 55 MiB data, 826 MiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 334 KiB/s wr, 46 op/s 2024-04-24T09:01:20.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.375+0000 7f7842000640 1 -- 172.21.15.83:0/3775769635 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f783c104cd0 msgr2=0x7f783c108d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:20.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.375+0000 7f7842000640 1 --2- 172.21.15.83:0/3775769635 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f783c104cd0 0x7f783c108d10 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f7828009900 tx=0x7f782802a410 comp rx=0 tx=0).stop 2024-04-24T09:01:20.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.375+0000 7f7842000640 1 -- 172.21.15.83:0/3775769635 shutdown_connections 2024-04-24T09:01:20.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.375+0000 7f7842000640 1 --2- 172.21.15.83:0/3775769635 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f783c1093f0 0x7f783c110fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:20.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.375+0000 7f7842000640 1 --2- 172.21.15.83:0/3775769635 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f783c104cd0 0x7f783c108d10 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:20.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.375+0000 7f7842000640 1 --2- 172.21.15.83:0/3775769635 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f783c104320 0x7f783c104700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:20.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.375+0000 7f7842000640 1 -- 172.21.15.83:0/3775769635 >> 172.21.15.83:0/3775769635 conn(0x7f783c0ffb40 msgr2=0x7f783c101f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:20.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.375+0000 7f7842000640 1 -- 172.21.15.83:0/3775769635 shutdown_connections 2024-04-24T09:01:20.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.375+0000 7f7842000640 1 -- 172.21.15.83:0/3775769635 wait complete. 2024-04-24T09:01:20.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.377+0000 7f7842000640 1 Processor -- start 2024-04-24T09:01:20.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.377+0000 7f7842000640 1 -- start start 2024-04-24T09:01:20.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.377+0000 7f7842000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f783c104320 0x7f783c1a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:20.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.377+0000 7f7842000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f783c1093f0 0x7f783c1a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:20.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.377+0000 7f7842000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f783c19a780 0x7f783c19ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:20.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.377+0000 7f7842000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f783c1143e0 con 0x7f783c19a780 2024-04-24T09:01:20.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.377+0000 7f7842000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f783c114260 con 0x7f783c104320 2024-04-24T09:01:20.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.377+0000 7f7842000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f783c114560 con 0x7f783c1093f0 2024-04-24T09:01:20.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.377+0000 7f7841600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f783c19a780 0x7f783c19ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:20.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.377+0000 7f7841600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f783c19a780 0x7f783c19ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52518/0 (socket says 172.21.15.83:52518) 2024-04-24T09:01:20.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.377+0000 7f7841600640 1 -- 172.21.15.83:0/3362909401 learned_addr learned my addr 172.21.15.83:0/3362909401 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:01:20.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.378+0000 7f7840c00640 1 --2- 172.21.15.83:0/3362909401 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f783c104320 0x7f783c1a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:20.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.378+0000 7f783be00640 1 --2- 172.21.15.83:0/3362909401 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f783c1093f0 0x7f783c1a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:20.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.378+0000 7f7841600640 1 -- 172.21.15.83:0/3362909401 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f783c1093f0 msgr2=0x7f783c1a0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:20.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.378+0000 7f7841600640 1 --2- 172.21.15.83:0/3362909401 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f783c1093f0 0x7f783c1a0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:20.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.378+0000 7f7841600640 1 -- 172.21.15.83:0/3362909401 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f783c104320 msgr2=0x7f783c1a0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:20.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.378+0000 7f7841600640 1 --2- 172.21.15.83:0/3362909401 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f783c104320 0x7f783c1a0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:20.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.378+0000 7f7841600640 1 -- 172.21.15.83:0/3362909401 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f783c19b310 con 0x7f783c19a780 2024-04-24T09:01:20.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.378+0000 7f7841600640 1 --2- 172.21.15.83:0/3362909401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f783c19a780 0x7f783c19ac30 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f783400b970 tx=0x7f783400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:20.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.379+0000 7f7839600640 1 -- 172.21.15.83:0/3362909401 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7834019070 con 0x7f783c19a780 2024-04-24T09:01:20.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.379+0000 7f7839600640 1 -- 172.21.15.83:0/3362909401 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7834014440 con 0x7f783c19a780 2024-04-24T09:01:20.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.379+0000 7f7839600640 1 -- 172.21.15.83:0/3362909401 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78340075d0 con 0x7f783c19a780 2024-04-24T09:01:20.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.379+0000 7f7842000640 1 -- 172.21.15.83:0/3362909401 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f783c19b600 con 0x7f783c19a780 2024-04-24T09:01:20.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.379+0000 7f7842000640 1 -- 172.21.15.83:0/3362909401 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f783c1a7760 con 0x7f783c19a780 2024-04-24T09:01:20.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.382+0000 7f7846385740 1 Processor -- start 2024-04-24T09:01:20.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.382+0000 7f7839600640 1 -- 172.21.15.83:0/3362909401 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f783401c070 con 0x7f783c19a780 2024-04-24T09:01:20.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.382+0000 7f7846385740 1 -- start start 2024-04-24T09:01:20.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.383+0000 7f7846385740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d958b73a30 0x55d958b73e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:20.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.383+0000 7f7846385740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d958b74350 0x55d958b7e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:20.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.383+0000 7f7846385740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d958b7ea20 0x55d958b80e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:20.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.383+0000 7f7846385740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d958ad91f0 con 0x55d958b7ea20 2024-04-24T09:01:20.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.383+0000 7f7846385740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d958a7f350 con 0x55d958b74350 2024-04-24T09:01:20.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.383+0000 7f7846385740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d958ad51b0 con 0x55d958b73a30 2024-04-24T09:01:20.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.383+0000 7f783be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d958b7ea20 0x55d958b80e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:20.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.383+0000 7f783be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d958b7ea20 0x55d958b80e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52526/0 (socket says 172.21.15.83:52526) 2024-04-24T09:01:20.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.383+0000 7f783be00640 1 -- 172.21.15.83:0/3451074221 learned_addr learned my addr 172.21.15.83:0/3451074221 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:01:20.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.383+0000 7f7840c00640 1 --2- 172.21.15.83:0/3451074221 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d958b74350 0x55d958b7e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:20.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.383+0000 7f783be00640 1 --2- 172.21.15.83:0/3451074221 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d958b73a30 0x55d958b73e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:20.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.384+0000 7f783be00640 1 -- 172.21.15.83:0/3451074221 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d958b73a30 msgr2=0x55d958b73e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:20.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.384+0000 7f783be00640 1 --2- 172.21.15.83:0/3451074221 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d958b73a30 0x55d958b73e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:20.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.384+0000 7f783be00640 1 -- 172.21.15.83:0/3451074221 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d958b74350 msgr2=0x55d958b7e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:20.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.384+0000 7f783be00640 1 --2- 172.21.15.83:0/3451074221 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d958b74350 0x55d958b7e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:20.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.384+0000 7f783be00640 1 -- 172.21.15.83:0/3451074221 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d958ad3f50 con 0x55d958b7ea20 2024-04-24T09:01:20.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.384+0000 7f7839600640 1 --2- 172.21.15.83:0/3362909401 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f78240a3da0 0x7f78240a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:20.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.384+0000 7f783be00640 1 --2- 172.21.15.83:0/3451074221 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d958b7ea20 0x55d958b80e10 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f7828009900 tx=0x7f7828000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=301e117c4ba47825 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:20.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.384+0000 7f7839600640 1 -- 172.21.15.83:0/3362909401 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7f78340d00e0 con 0x7f783c19a780 2024-04-24T09:01:20.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.384+0000 7f7840c00640 1 --2- 172.21.15.83:0/3362909401 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f78240a3da0 0x7f78240a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:20.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.384+0000 7f782dc00640 1 -- 172.21.15.83:0/3451074221 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7828043020 con 0x55d958b7ea20 2024-04-24T09:01:20.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.384+0000 7f782dc00640 1 -- 172.21.15.83:0/3451074221 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f78280046c0 con 0x55d958b7ea20 2024-04-24T09:01:20.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.384+0000 7f782dc00640 1 -- 172.21.15.83:0/3451074221 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7828035040 con 0x55d958b7ea20 2024-04-24T09:01:20.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.385+0000 7f7846385740 1 -- 172.21.15.83:0/3451074221 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d958b7ea20 msgr2=0x55d958b80e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:20.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.385+0000 7f7840c00640 1 --2- 172.21.15.83:0/3362909401 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f78240a3da0 0x7f78240a6260 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x55d958b74ff0 tx=0x7f783000b4d0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:20.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.385+0000 7f7846385740 1 --2- 172.21.15.83:0/3451074221 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d958b7ea20 0x55d958b80e10 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f7828009900 tx=0x7f7828000bc0 comp rx=0 tx=0).stop 2024-04-24T09:01:20.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.385+0000 7f7846385740 1 -- 172.21.15.83:0/3451074221 shutdown_connections 2024-04-24T09:01:20.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.385+0000 7f7846385740 1 --2- 172.21.15.83:0/3451074221 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d958b7ea20 0x55d958b80e10 unknown :-1 s=CLOSED pgs=1667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:20.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.385+0000 7f7846385740 1 --2- 172.21.15.83:0/3451074221 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d958b74350 0x55d958b7e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:20.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.385+0000 7f7846385740 1 --2- 172.21.15.83:0/3451074221 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d958b73a30 0x55d958b73e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:20.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.386+0000 7f7846385740 1 -- 172.21.15.83:0/3451074221 >> 172.21.15.83:0/3451074221 conn(0x55d958b6fd50 msgr2=0x55d958b70130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:20.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.386+0000 7f7846385740 1 -- 172.21.15.83:0/3451074221 shutdown_connections 2024-04-24T09:01:20.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.386+0000 7f7846385740 1 -- 172.21.15.83:0/3451074221 wait complete. 2024-04-24T09:01:20.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.386+0000 7f7846385740 1 -- start start 2024-04-24T09:01:20.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.386+0000 7f7846385740 1 Processor -- start 2024-04-24T09:01:20.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.387+0000 7f7846385740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d958b73a30 0x55d958b7ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:20.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.387+0000 7f7846385740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d958b74350 0x55d958b7bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:20.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.387+0000 7f7846385740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d958b7ea20 0x55d958b7c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:20.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.387+0000 7f7846385740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d958b817b0 con 0x55d958b74350 2024-04-24T09:01:20.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.387+0000 7f7846385740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d958a92690 con 0x55d958b7ea20 2024-04-24T09:01:20.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.387+0000 7f7846385740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d958b81930 con 0x55d958b73a30 2024-04-24T09:01:20.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.387+0000 7f7846385740 10 client.0 _ll_register_callbacks cb 0x55d958ace650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:01:20.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.387+0000 7f783be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d958b74350 0x55d958b7bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:20.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.387+0000 7f783be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d958b74350 0x55d958b7bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52540/0 (socket says 172.21.15.83:52540) 2024-04-24T09:01:20.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.387+0000 7f783be00640 1 -- 172.21.15.83:0/1407153735 learned_addr learned my addr 172.21.15.83:0/1407153735 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:01:20.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.387+0000 7f7840c00640 1 --2- 172.21.15.83:0/1407153735 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d958b73a30 0x55d958b7ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:20.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.387+0000 7f7841600640 1 --2- 172.21.15.83:0/1407153735 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d958b7ea20 0x55d958b7c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:20.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.387+0000 7f783be00640 1 -- 172.21.15.83:0/1407153735 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d958b73a30 msgr2=0x55d958b7ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:20.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.387+0000 7f783be00640 1 --2- 172.21.15.83:0/1407153735 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d958b73a30 0x55d958b7ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:20.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.387+0000 7f783be00640 1 -- 172.21.15.83:0/1407153735 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d958b7ea20 msgr2=0x55d958b7c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:20.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.387+0000 7f783be00640 1 --2- 172.21.15.83:0/1407153735 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d958b7ea20 0x55d958b7c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:20.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.388+0000 7f783be00640 1 -- 172.21.15.83:0/1407153735 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d958a91920 con 0x55d958b74350 2024-04-24T09:01:20.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.388+0000 7f7841600640 1 --2- 172.21.15.83:0/1407153735 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d958b7ea20 0x55d958b7c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:01:20.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.388+0000 7f783be00640 1 --2- 172.21.15.83:0/1407153735 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d958b74350 0x55d958b7bf80 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f782802a930 tx=0x7f782802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b60914e944734a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:20.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.388+0000 7f7822000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:01:20.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.388+0000 7f7822000640 1 -- 172.21.15.83:0/1407153735 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7828043030 con 0x55d958b74350 2024-04-24T09:01:20.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.388+0000 7f7822000640 1 -- 172.21.15.83:0/1407153735 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7f78280035f0 con 0x55d958b74350 2024-04-24T09:01:20.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.388+0000 7f7846385740 1 -- 172.21.15.83:0/1407153735 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78280046c0 con 0x55d958b74350 2024-04-24T09:01:20.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.389+0000 7f7822000640 1 -- 172.21.15.83:0/1407153735 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f7828058050 con 0x55d958b74350 2024-04-24T09:01:20.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.389+0000 7f7846385740 10 client.19164 fetch_fsmap learned FSMap version 35 2024-04-24T09:01:20.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.389+0000 7f7846385740 1 -- 172.21.15.83:0/1407153735 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55d9588b57d0 con 0x55d958b74350 2024-04-24T09:01:20.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.389+0000 7f7822000640 1 -- 172.21.15.83:0/1407153735 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f78280046c0 con 0x55d958b74350 2024-04-24T09:01:20.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.389+0000 7f7822000640 1 -- 172.21.15.83:0/1407153735 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7f782803a040 con 0x55d958b74350 2024-04-24T09:01:20.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.389+0000 7f7846385740 10 client.19164 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T09:01:20.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.389+0000 7f7846385740 10 client.19164 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:01:20.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.389+0000 7f7846385740 20 client.19164 populate_metadata read hostname 'smithi083' 2024-04-24T09:01:20.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.389+0000 7f7846385740 1 --2- 172.21.15.83:0/1407153735 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d958b8ec60 0x55d958b91050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:20.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.389+0000 7f7846385740 4 client.19164 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:01:20.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.389+0000 7f7846385740 1 -- 172.21.15.83:0/1407153735 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d958ae6c10 con 0x55d958b8ec60 2024-04-24T09:01:20.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.390+0000 7f7840c00640 1 --2- 172.21.15.83:0/1407153735 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d958b8ec60 0x55d958b91050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:20.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.391+0000 7f7840c00640 1 --2- 172.21.15.83:0/1407153735 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d958b8ec60 0x55d958b91050 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3a9b82f2853063f server_cookie=544fd43bb8af6a6d in_seq=0 out_seq=0 2024-04-24T09:01:20.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.391+0000 7f7822000640 10 client.19164 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:01:20.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.394+0000 7f7822000640 1 -- 172.21.15.83:0/1407153735 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d958ae6c10 con 0x55d958b8ec60 2024-04-24T09:01:20.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.394+0000 7f7822000640 10 client.19164 handle_command_reply: tid=0 2024-04-24T09:01:20.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.394+0000 7f7846385740 1 client.19164 shutdown 2024-04-24T09:01:20.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.394+0000 7f7846385740 1 -- 172.21.15.83:0/1407153735 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d958b74350 msgr2=0x55d958b7bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:20.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.394+0000 7f7846385740 1 --2- 172.21.15.83:0/1407153735 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d958b74350 0x55d958b7bf80 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f782802a930 tx=0x7f782802a410 comp rx=0 tx=0).stop 2024-04-24T09:01:20.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.394+0000 7f7846385740 1 -- 172.21.15.83:0/1407153735 shutdown_connections 2024-04-24T09:01:20.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.394+0000 7f7846385740 1 --2- 172.21.15.83:0/1407153735 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d958b8ec60 0x55d958b91050 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:20.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.394+0000 7f7846385740 1 --2- 172.21.15.83:0/1407153735 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d958b7ea20 0x55d958b7c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:20.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.394+0000 7f7846385740 1 --2- 172.21.15.83:0/1407153735 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d958b74350 0x55d958b7bf80 unknown :-1 s=CLOSED pgs=1668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:20.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.394+0000 7f7846385740 1 --2- 172.21.15.83:0/1407153735 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d958b73a30 0x55d958b7ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:20.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.394+0000 7f7846385740 1 -- 172.21.15.83:0/1407153735 >> 172.21.15.83:0/1407153735 conn(0x55d958b6fd50 msgr2=0x55d958b75b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:20.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.394+0000 7f7822000640 0 client.19164 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:01:20.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.395+0000 7f7846385740 1 -- 172.21.15.83:0/1407153735 shutdown_connections 2024-04-24T09:01:20.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.395+0000 7f7846385740 1 -- 172.21.15.83:0/1407153735 wait complete. 2024-04-24T09:01:20.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.395+0000 7f7846385740 20 client.19164 trim_cache size 0 max 16384 2024-04-24T09:01:20.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.417+0000 7f7846385740 1 Processor -- start 2024-04-24T09:01:20.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.417+0000 7f7846385740 1 -- start start 2024-04-24T09:01:20.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.417+0000 7f7846385740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d958b73a30 0x55d958b761d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:20.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.417+0000 7f7846385740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d958b74350 0x55d958b8a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:20.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.417+0000 7f7846385740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d958b7ea20 0x55d958b93c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:20.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.417+0000 7f7846385740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f783c1b15f0 con 0x55d958b7ea20 2024-04-24T09:01:20.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.417+0000 7f7846385740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f783c1b3130 con 0x55d958b73a30 2024-04-24T09:01:20.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.417+0000 7f7846385740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f783c1b1cc0 con 0x55d958b74350 2024-04-24T09:01:20.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.418+0000 7f7841600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d958b7ea20 0x55d958b93c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:20.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.418+0000 7f7841600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d958b7ea20 0x55d958b93c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52544/0 (socket says 172.21.15.83:52544) 2024-04-24T09:01:20.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.418+0000 7f7841600640 1 -- 172.21.15.83:0/3247363588 learned_addr learned my addr 172.21.15.83:0/3247363588 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:01:20.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.418+0000 7f783be00640 1 --2- 172.21.15.83:0/3247363588 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d958b74350 0x55d958b8a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:20.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.418+0000 7f7840c00640 1 --2- 172.21.15.83:0/3247363588 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d958b73a30 0x55d958b761d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:20.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.418+0000 7f7841600640 1 -- 172.21.15.83:0/3247363588 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d958b74350 msgr2=0x55d958b8a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:20.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.418+0000 7f7841600640 1 --2- 172.21.15.83:0/3247363588 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d958b74350 0x55d958b8a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:20.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.418+0000 7f7841600640 1 -- 172.21.15.83:0/3247363588 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d958b73a30 msgr2=0x55d958b761d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:20.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.418+0000 7f7841600640 1 --2- 172.21.15.83:0/3247363588 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d958b73a30 0x55d958b761d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:20.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.418+0000 7f7841600640 1 -- 172.21.15.83:0/3247363588 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9587eb3f0 con 0x55d958b7ea20 2024-04-24T09:01:20.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.419+0000 7f7841600640 1 --2- 172.21.15.83:0/3247363588 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d958b7ea20 0x55d958b93c90 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f783400b940 tx=0x7f7834004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d9e5313d4480249 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:20.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.419+0000 7f782d200640 1 -- 172.21.15.83:0/3247363588 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78340b5070 con 0x55d958b7ea20 2024-04-24T09:01:20.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.419+0000 7f782d200640 1 -- 172.21.15.83:0/3247363588 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f78340a00c0 con 0x55d958b7ea20 2024-04-24T09:01:20.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.419+0000 7f782d200640 1 -- 172.21.15.83:0/3247363588 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78340145f0 con 0x55d958b7ea20 2024-04-24T09:01:20.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.420+0000 7f7846385740 1 -- 172.21.15.83:0/3247363588 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d958b7ea20 msgr2=0x55d958b93c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:20.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.420+0000 7f7846385740 1 --2- 172.21.15.83:0/3247363588 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d958b7ea20 0x55d958b93c90 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f783400b940 tx=0x7f7834004a40 comp rx=0 tx=0).stop 2024-04-24T09:01:20.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.420+0000 7f7846385740 1 -- 172.21.15.83:0/3247363588 shutdown_connections 2024-04-24T09:01:20.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.420+0000 7f7846385740 1 --2- 172.21.15.83:0/3247363588 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d958b7ea20 0x55d958b93c90 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:20.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.420+0000 7f7846385740 1 --2- 172.21.15.83:0/3247363588 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d958b74350 0x55d958b8a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:20.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.420+0000 7f7846385740 1 --2- 172.21.15.83:0/3247363588 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d958b73a30 0x55d958b761d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:20.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.420+0000 7f7846385740 1 -- 172.21.15.83:0/3247363588 >> 172.21.15.83:0/3247363588 conn(0x55d958b6fd50 msgr2=0x55d958b89b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:20.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.420+0000 7f7846385740 1 -- 172.21.15.83:0/3247363588 shutdown_connections 2024-04-24T09:01:20.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.420+0000 7f7846385740 1 -- 172.21.15.83:0/3247363588 wait complete. 2024-04-24T09:01:20.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.420+0000 7f7846385740 1 -- start start 2024-04-24T09:01:20.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.421+0000 7f7846385740 1 Processor -- start 2024-04-24T09:01:20.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.421+0000 7f7846385740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d958b73a30 0x55d958b90f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:20.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.421+0000 7f7846385740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d958b74350 0x55d958b91470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:20.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.421+0000 7f7846385740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d958b7ea20 0x55d958bece70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:20.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.421+0000 7f7846385740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f783c1b2a60 con 0x55d958b7ea20 2024-04-24T09:01:20.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.421+0000 7f7846385740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f783c1b2390 con 0x55d958b74350 2024-04-24T09:01:20.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.421+0000 7f7846385740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f783c1af5f0 con 0x55d958b73a30 2024-04-24T09:01:20.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.421+0000 7f7846385740 10 client.0 _ll_register_callbacks cb 0x7f783000b3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:01:20.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.421+0000 7f7841600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d958b7ea20 0x55d958bece70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:20.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.422+0000 7f7841600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d958b7ea20 0x55d958bece70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52554/0 (socket says 172.21.15.83:52554) 2024-04-24T09:01:20.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.422+0000 7f7841600640 1 -- 172.21.15.83:0/4044786724 learned_addr learned my addr 172.21.15.83:0/4044786724 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:01:20.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.422+0000 7f7840c00640 1 --2- 172.21.15.83:0/4044786724 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d958b73a30 0x55d958b90f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:20.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.422+0000 7f783be00640 1 --2- 172.21.15.83:0/4044786724 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d958b74350 0x55d958b91470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:20.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.422+0000 7f7841600640 1 -- 172.21.15.83:0/4044786724 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d958b73a30 msgr2=0x55d958b90f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:20.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.422+0000 7f7841600640 1 --2- 172.21.15.83:0/4044786724 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d958b73a30 0x55d958b90f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:20.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.422+0000 7f7841600640 1 -- 172.21.15.83:0/4044786724 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d958b74350 msgr2=0x55d958b91470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:20.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.422+0000 7f7841600640 1 --2- 172.21.15.83:0/4044786724 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d958b74350 0x55d958b91470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:20.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.422+0000 7f7841600640 1 -- 172.21.15.83:0/4044786724 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9587ab0f0 con 0x55d958b7ea20 2024-04-24T09:01:20.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.422+0000 7f7841600640 1 --2- 172.21.15.83:0/4044786724 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d958b7ea20 0x55d958bece70 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f78340a4450 tx=0x7f7834014c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=426cb9505642cbdb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:20.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.422+0000 7f7822000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:01:20.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.422+0000 7f7822000640 1 -- 172.21.15.83:0/4044786724 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78340ba060 con 0x55d958b7ea20 2024-04-24T09:01:20.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.422+0000 7f7822000640 1 -- 172.21.15.83:0/4044786724 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7f78340145f0 con 0x55d958b7ea20 2024-04-24T09:01:20.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.422+0000 7f7846385740 1 -- 172.21.15.83:0/4044786724 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78340a00c0 con 0x55d958b7ea20 2024-04-24T09:01:20.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.423+0000 7f7822000640 1 -- 172.21.15.83:0/4044786724 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f78340b5040 con 0x55d958b7ea20 2024-04-24T09:01:20.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.423+0000 7f7846385740 10 client.19170 fetch_fsmap learned FSMap version 35 2024-04-24T09:01:20.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.423+0000 7f7846385740 1 -- 172.21.15.83:0/4044786724 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55d958a9e810 con 0x55d958b7ea20 2024-04-24T09:01:20.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.423+0000 7f7822000640 1 -- 172.21.15.83:0/4044786724 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f78340a00c0 con 0x55d958b7ea20 2024-04-24T09:01:20.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.423+0000 7f7822000640 1 -- 172.21.15.83:0/4044786724 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7f783409b160 con 0x55d958b7ea20 2024-04-24T09:01:20.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.423+0000 7f7846385740 10 client.19170 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T09:01:20.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.423+0000 7f7846385740 10 client.19170 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:01:20.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.423+0000 7f7846385740 20 client.19170 populate_metadata read hostname 'smithi083' 2024-04-24T09:01:20.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.423+0000 7f7846385740 1 --2- 172.21.15.83:0/4044786724 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d958bf4b00 0x55d958bf6f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:20.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.423+0000 7f7846385740 4 client.19170 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T09:01:20.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.423+0000 7f7846385740 1 -- 172.21.15.83:0/4044786724 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d958ae0a80 con 0x55d958bf4b00 2024-04-24T09:01:20.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.424+0000 7f7840c00640 1 --2- 172.21.15.83:0/4044786724 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d958bf4b00 0x55d958bf6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:20.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.425+0000 7f7840c00640 1 --2- 172.21.15.83:0/4044786724 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d958bf4b00 0x55d958bf6f20 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc2de4b60150a089 server_cookie=124b6bb7e1ec9f9b in_seq=0 out_seq=0 2024-04-24T09:01:20.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.425+0000 7f7822000640 10 client.19170 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:01:20.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.427+0000 7f7822000640 1 -- 172.21.15.83:0/4044786724 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55d958ae0a80 con 0x55d958bf4b00 2024-04-24T09:01:20.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.427+0000 7f7822000640 10 client.19170 handle_command_reply: tid=0 2024-04-24T09:01:20.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.427+0000 7f7846385740 1 client.19170 shutdown 2024-04-24T09:01:20.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.427+0000 7f7846385740 1 -- 172.21.15.83:0/4044786724 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d958b7ea20 msgr2=0x55d958bece70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:20.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.427+0000 7f7846385740 1 --2- 172.21.15.83:0/4044786724 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d958b7ea20 0x55d958bece70 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f78340a4450 tx=0x7f7834014c00 comp rx=0 tx=0).stop 2024-04-24T09:01:20.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.428+0000 7f7846385740 1 -- 172.21.15.83:0/4044786724 shutdown_connections 2024-04-24T09:01:20.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.428+0000 7f7846385740 1 --2- 172.21.15.83:0/4044786724 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d958bf4b00 0x55d958bf6f20 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:20.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.428+0000 7f7846385740 1 --2- 172.21.15.83:0/4044786724 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d958b7ea20 0x55d958bece70 unknown :-1 s=CLOSED pgs=1670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:20.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.428+0000 7f7846385740 1 --2- 172.21.15.83:0/4044786724 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d958b74350 0x55d958b91470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:20.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.428+0000 7f7846385740 1 --2- 172.21.15.83:0/4044786724 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d958b73a30 0x55d958b90f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:20.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.428+0000 7f7822000640 0 client.19170 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:01:20.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.428+0000 7f7846385740 1 -- 172.21.15.83:0/4044786724 >> 172.21.15.83:0/4044786724 conn(0x55d958b6fd50 msgr2=0x55d958b8a410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:20.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.428+0000 7f7846385740 1 -- 172.21.15.83:0/4044786724 shutdown_connections 2024-04-24T09:01:20.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.428+0000 7f7846385740 1 -- 172.21.15.83:0/4044786724 wait complete. 2024-04-24T09:01:20.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.428+0000 7f7846385740 20 client.19170 trim_cache size 0 max 16384 2024-04-24T09:01:20.429 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T09:01:20.430 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T09:01:20.430 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-04-24T09:01:20.430 INFO:teuthology.orchestra.run.smithi083.stdout: "c8c021c2-94f4-40e6-984e-11bdd767f715": { 2024-04-24T09:01:20.430 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-04-24T09:01:20.430 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "c8c021c2-94f4-40e6-984e-11bdd767f715", 2024-04-24T09:01:20.430 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-04-24T09:01:20.430 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:01:20.430 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:01:20.430 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T09:01:20.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.429+0000 7f7842000640 1 -- 172.21.15.83:0/3362909401 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f78240a3da0 msgr2=0x7f78240a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:20.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.429+0000 7f7842000640 1 --2- 172.21.15.83:0/3362909401 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f78240a3da0 0x7f78240a6260 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x55d958b74ff0 tx=0x7f783000b4d0 comp rx=0 tx=0).stop 2024-04-24T09:01:20.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.429+0000 7f7842000640 1 -- 172.21.15.83:0/3362909401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f783c19a780 msgr2=0x7f783c19ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:20.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.429+0000 7f7842000640 1 --2- 172.21.15.83:0/3362909401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f783c19a780 0x7f783c19ac30 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f783400b970 tx=0x7f783400be30 comp rx=0 tx=0).stop 2024-04-24T09:01:20.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.429+0000 7f7842000640 1 -- 172.21.15.83:0/3362909401 shutdown_connections 2024-04-24T09:01:20.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.429+0000 7f7842000640 1 --2- 172.21.15.83:0/3362909401 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f78240a3da0 0x7f78240a6260 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:20.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.429+0000 7f7842000640 1 --2- 172.21.15.83:0/3362909401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f783c19a780 0x7f783c19ac30 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:20.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.429+0000 7f7842000640 1 --2- 172.21.15.83:0/3362909401 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f783c1093f0 0x7f783c1a0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:20.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.429+0000 7f7842000640 1 --2- 172.21.15.83:0/3362909401 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f783c104320 0x7f783c1a0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:20.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.429+0000 7f7842000640 1 -- 172.21.15.83:0/3362909401 >> 172.21.15.83:0/3362909401 conn(0x7f783c0ffb40 msgr2=0x7f783c101e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:20.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.429+0000 7f7842000640 1 -- 172.21.15.83:0/3362909401 shutdown_connections 2024-04-24T09:01:20.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:20.429+0000 7f7842000640 1 -- 172.21.15.83:0/3362909401 wait complete. 2024-04-24T09:01:20.447 INFO:tasks.cephfs.filesystem:scrub status for tag:c8c021c2-94f4-40e6-984e-11bdd767f715 - {'path': '/', 'tag': 'c8c021c2-94f4-40e6-984e-11bdd767f715', 'options': 'recursive,force'} 2024-04-24T09:01:20.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:01:19 smithi138 ceph-mon[31441]: pgmap v304: 129 pgs: 129 active+clean; 55 MiB data, 826 MiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 334 KiB/s wr, 46 op/s 2024-04-24T09:01:20.455 INFO:journalctl@ceph.mds.b.smithi138.stdout:Apr 24 09:01:20 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-b[61515]: 2024-04-24T09:01:20.184+0000 7f7fbbc00700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000215 [...10,head] /volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/COREL/ auth{2=1} v9578 ap=2 f(v390 m2024-04-24T09:01:01.451273+0000 11=11+0) n(v23 rc2024-04-24T09:01:01.451273+0000 b2861056 12=11+1) old_inodes=6 (inest sync->mix g=2) caps={15603=pAsLsXsFs/-@8098} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x55cf8760d700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000000215:[<0x10000000009/COREL v369>,<0x10000000008/~dmtmp v57>,<0x10000000007/client0 v33>,<0x10000000006/clients v23>,<0x10000000005/tmp v23>,<0x10000000003/client.0 v22>,<0x10000000002/e395bed6-5bba-41b8-a256-3505e4afcca2 v36>,<0x10000000001/sv_0 v55>,<0x10000000000/qa v43>,<0x1/volumes v39>]//[]","memoryvalue":"(2)0x10000000215:[<0x10000000009/COREL v9578>,<0x10000000008/~dmtmp v9059>,<0x10000000007/client0 v1068>,<0x10000000006/clients v1060>,<0x10000000005/tmp v1048>,<0x10000000003/client.0 v1045>,<0x10000000002/e395bed6-5bba-41b8-a256-3505e4afcca2 v1038>,<0x10000000001/sv_0 v1033>,<0x10000000000/qa v1013>,<0x1/volumes v983>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-04-24T09:01:01.419357+0000 12=12+0)","ondisk_value.rstat":"n(v0 rc2024-04-24T09:01:01.451273+0000 b2861056 12=11+1)","memory_value.dirstat":"f(v390 m2024-04-24T09:01:01.451273+0000 11=11+0)","memory_value.rstat":"n(v23 rc2024-04-24T09:01:01.451273+0000 b2861056 12=11+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-04-24T09:01:20.455 INFO:journalctl@ceph.mds.b.smithi138.stdout:Apr 24 09:01:20 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-b[61515]: 2024-04-24T09:01:20.187+0000 7f7fbbc00700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000208 [...10,head] /volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/WORD/ auth{2=1} v9650 ap=2 f(v360 m2024-04-24T09:01:07.258521+0000 5=5+0) n(v34 rc2024-04-24T09:01:07.258521+0000 b814326 6=5+1) old_inodes=6 (inest sync->mix g=2) caps={15603=pAsLsXsFs/-@8059} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x55cf87274c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000000208:[<0x10000000009/WORD v427>,<0x10000000008/~dmtmp v57>,<0x10000000007/client0 v33>,<0x10000000006/clients v23>,<0x10000000005/tmp v23>,<0x10000000003/client.0 v22>,<0x10000000002/e395bed6-5bba-41b8-a256-3505e4afcca2 v36>,<0x10000000001/sv_0 v55>,<0x10000000000/qa v43>,<0x1/volumes v39>]//[]","memoryvalue":"(2)0x10000000208:[<0x10000000009/WORD v9650>,<0x10000000008/~dmtmp v9059>,<0x10000000007/client0 v1068>,<0x10000000006/clients v1060>,<0x10000000005/tmp v1048>,<0x10000000003/client.0 v1045>,<0x10000000002/e395bed6-5bba-41b8-a256-3505e4afcca2 v1038>,<0x10000000001/sv_0 v1033>,<0x10000000000/qa v1013>,<0x1/volumes v983>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-04-24T09:01:07.211567+0000 6=6+0)","ondisk_value.rstat":"n(v0 rc2024-04-24T09:01:07.258521+0000 b814326 6=5+1)","memory_value.dirstat":"f(v360 m2024-04-24T09:01:07.258521+0000 5=5+0)","memory_value.rstat":"n(v34 rc2024-04-24T09:01:07.258521+0000 b814326 6=5+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-04-24T09:01:20.455 INFO:journalctl@ceph.mds.b.smithi138.stdout:Apr 24 09:01:20 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-b[61515]: 2024-04-24T09:01:20.196+0000 7f7fbbc00700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000225 [...10,head] /volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/PM/ auth{3=1} v9556 ap=2 f(v1110 m2024-04-24T09:01:01.740661+0000 3=3+0) n(v30 rc2024-04-24T09:01:01.740661+0000 b780288 4=3+1) old_inodes=6 (inest sync->mix g=3) caps={15603=pAsLsXsFs/-@10936} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x55cf8762f180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000000225:[<0x10000000009/PM v381>,<0x10000000008/~dmtmp v57>,<0x10000000007/client0 v33>,<0x10000000006/clients v23>,<0x10000000005/tmp v23>,<0x10000000003/client.0 v22>,<0x10000000002/e395bed6-5bba-41b8-a256-3505e4afcca2 v36>,<0x10000000001/sv_0 v55>,<0x10000000000/qa v43>,<0x1/volumes v39>]//[]","memoryvalue":"(2)0x10000000225:[<0x10000000009/PM v9556>,<0x10000000008/~dmtmp v9061>,<0x10000000007/client0 v1068>,<0x10000000006/clients v1060>,<0x10000000005/tmp v1048>,<0x10000000003/client.0 v1045>,<0x10000000002/e395bed6-5bba-41b8-a256-3505e4afcca2 v1038>,<0x10000000001/sv_0 v1033>,<0x10000000000/qa v1013>,<0x1/volumes v983>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-04-24T09:01:00.159151+0000 4=4+0)","ondisk_value.rstat":"n(v0 rc2024-04-24T09:01:01.740661+0000 b780288 4=3+1)","memory_value.dirstat":"f(v1110 m2024-04-24T09:01:01.740661+0000 3=3+0)","memory_value.rstat":"n(v30 rc2024-04-24T09:01:01.740661+0000 b780288 4=3+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-04-24T09:01:20.456 INFO:journalctl@ceph.mds.b.smithi138.stdout:Apr 24 09:01:20 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-b[61515]: 2024-04-24T09:01:20.203+0000 7f7fbbc00700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000023e [...10,head] /volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/ACCESS/ auth{3=1} v9554 ap=2 f(v210 m2024-04-24T09:01:02.056748+0000 1=1+0) n(v32 rc2024-04-24T09:01:02.056748+0000 b260096 2=1+1) old_inodes=6 (inest sync->mix g=3) caps={15603=pAsLsXsFs/-@7301} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x55cf87666b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000023e:[<0x10000000009/ACCESS v463>,<0x10000000008/~dmtmp v57>,<0x10000000007/client0 v33>,<0x10000000006/clients v23>,<0x10000000005/tmp v23>,<0x10000000003/client.0 v22>,<0x10000000002/e395bed6-5bba-41b8-a256-3505e4afcca2 v36>,<0x10000000001/sv_0 v55>,<0x10000000000/qa v43>,<0x1/volumes v39>]//[]","memoryvalue":"(2)0x1000000023e:[<0x10000000009/ACCESS v9554>,<0x10000000008/~dmtmp v9063>,<0x10000000007/client0 v1068>,<0x10000000006/clients v1060>,<0x10000000005/tmp v1048>,<0x10000000003/client.0 v1045>,<0x10000000002/e395bed6-5bba-41b8-a256-3505e4afcca2 v1038>,<0x10000000001/sv_0 v1033>,<0x10000000000/qa v1013>,<0x1/volumes v983>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-04-24T09:01:02.008038+0000 2=2+0)","ondisk_value.rstat":"n(v0 rc2024-04-24T09:01:02.056748+0000 b260096 2=1+1)","memory_value.dirstat":"f(v210 m2024-04-24T09:01:02.056748+0000 1=1+0)","memory_value.rstat":"n(v32 rc2024-04-24T09:01:02.056748+0000 b260096 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-04-24T09:01:20.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:01:19 smithi189 ceph-mon[31791]: pgmap v304: 129 pgs: 129 active+clean; 55 MiB data, 826 MiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 334 KiB/s wr, 46 op/s 2024-04-24T09:01:20.606 INFO:tasks.workunit.client.0.smithi083.stdout: 1 56444 0.63 MB/sec execute 354 sec latency 48.246 ms 2024-04-24T09:01:21.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:01:20 smithi083 ceph-mon[27907]: scrub queued for path: / 2024-04-24T09:01:21.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:01:20 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:01:21.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:01:20 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:01:21.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:01:20 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:01:21.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:01:20 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:01:21.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:01:20 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:01:21.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:01:20 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:01:21.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:01:20 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:01:21.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:01:20 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:01:21.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:01:20 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:01:21.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:01:20 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:01:21.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:01:20 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:01:21.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:01:20 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:01:21.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:01:20 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:01:21.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:01:20 smithi138 ceph-mon[31441]: scrub queued for path: / 2024-04-24T09:01:21.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:01:20 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:01:21.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:01:20 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:01:21.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:01:20 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:01:21.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:01:20 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:01:21.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:01:20 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:01:21.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:01:20 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:01:21.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:01:20 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:01:21.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:01:20 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:01:21.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:01:20 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:01:21.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:01:20 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:01:21.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:01:20 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:01:21.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:01:20 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:01:21.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:01:20 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:01:21.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:01:20 smithi189 ceph-mon[31791]: scrub queued for path: / 2024-04-24T09:01:21.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:01:20 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:01:21.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:01:20 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:01:21.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:01:20 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:01:21.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:01:20 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:01:21.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:01:20 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:01:21.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:01:20 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:01:21.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:01:20 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:01:21.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:01:20 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:01:21.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:01:20 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:01:21.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:01:20 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:01:21.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:01:20 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:01:21.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:01:20 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:01:21.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:01:20 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:01:21.606 INFO:tasks.workunit.client.0.smithi083.stdout: 1 56532 0.63 MB/sec execute 355 sec latency 58.514 ms 2024-04-24T09:01:22.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:01:21 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:01:22.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:01:21 smithi083 ceph-mon[27907]: Scrub error on inode 0x10000000215 (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/COREL) see mds.b log and `damage ls` output for details 2024-04-24T09:01:22.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:01:21 smithi083 ceph-mon[27907]: Scrub error on inode 0x10000000208 (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/WORD) see mds.b log and `damage ls` output for details 2024-04-24T09:01:22.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:01:21 smithi083 ceph-mon[27907]: Scrub error on inode 0x10000000225 (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/PM) see mds.b log and `damage ls` output for details 2024-04-24T09:01:22.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:01:21 smithi083 ceph-mon[27907]: Scrub error on inode 0x1000000023e (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/ACCESS) see mds.b log and `damage ls` output for details 2024-04-24T09:01:22.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:01:21 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:01:22.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:01:21 smithi083 ceph-mon[27907]: pgmap v305: 129 pgs: 129 active+clean; 55 MiB data, 826 MiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 300 KiB/s wr, 41 op/s 2024-04-24T09:01:22.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:01:21 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:01:22.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:01:21 smithi138 ceph-mon[31441]: Scrub error on inode 0x10000000215 (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/COREL) see mds.b log and `damage ls` output for details 2024-04-24T09:01:22.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:01:21 smithi138 ceph-mon[31441]: Scrub error on inode 0x10000000208 (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/WORD) see mds.b log and `damage ls` output for details 2024-04-24T09:01:22.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:01:21 smithi138 ceph-mon[31441]: Scrub error on inode 0x10000000225 (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/PM) see mds.b log and `damage ls` output for details 2024-04-24T09:01:22.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:01:21 smithi138 ceph-mon[31441]: Scrub error on inode 0x1000000023e (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/ACCESS) see mds.b log and `damage ls` output for details 2024-04-24T09:01:22.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:01:21 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:01:22.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:01:21 smithi138 ceph-mon[31441]: pgmap v305: 129 pgs: 129 active+clean; 55 MiB data, 826 MiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 300 KiB/s wr, 41 op/s 2024-04-24T09:01:22.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:01:21 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:01:22.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:01:21 smithi189 ceph-mon[31791]: Scrub error on inode 0x10000000215 (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/COREL) see mds.b log and `damage ls` output for details 2024-04-24T09:01:22.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:01:21 smithi189 ceph-mon[31791]: Scrub error on inode 0x10000000208 (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/WORD) see mds.b log and `damage ls` output for details 2024-04-24T09:01:22.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:01:21 smithi189 ceph-mon[31791]: Scrub error on inode 0x10000000225 (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/PM) see mds.b log and `damage ls` output for details 2024-04-24T09:01:22.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:01:21 smithi189 ceph-mon[31791]: Scrub error on inode 0x1000000023e (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/ACCESS) see mds.b log and `damage ls` output for details 2024-04-24T09:01:22.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:01:21 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:01:22.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:01:21 smithi189 ceph-mon[31791]: pgmap v305: 129 pgs: 129 active+clean; 55 MiB data, 826 MiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 300 KiB/s wr, 41 op/s 2024-04-24T09:01:22.606 INFO:tasks.workunit.client.0.smithi083.stdout: 1 56559 0.63 MB/sec execute 356 sec latency 53.736 ms 2024-04-24T09:01:23.606 INFO:tasks.workunit.client.0.smithi083.stdout: 1 56613 0.63 MB/sec execute 357 sec latency 54.402 ms 2024-04-24T09:01:24.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:01:23 smithi083 ceph-mon[27907]: pgmap v306: 129 pgs: 129 active+clean; 58 MiB data, 840 MiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 497 KiB/s wr, 68 op/s 2024-04-24T09:01:24.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:01:23 smithi138 ceph-mon[31441]: pgmap v306: 129 pgs: 129 active+clean; 58 MiB data, 840 MiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 497 KiB/s wr, 68 op/s 2024-04-24T09:01:24.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:01:23 smithi189 ceph-mon[31791]: pgmap v306: 129 pgs: 129 active+clean; 58 MiB data, 840 MiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 497 KiB/s wr, 68 op/s 2024-04-24T09:01:24.606 INFO:tasks.workunit.client.0.smithi083.stdout: 1 56800 0.64 MB/sec execute 358 sec latency 51.241 ms 2024-04-24T09:01:25.606 INFO:tasks.workunit.client.0.smithi083.stdout: 1 56920 0.64 MB/sec execute 359 sec latency 49.860 ms 2024-04-24T09:01:26.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:01:26 smithi083 ceph-mon[27907]: pgmap v307: 129 pgs: 129 active+clean; 58 MiB data, 840 MiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 425 KiB/s wr, 48 op/s 2024-04-24T09:01:26.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:01:26 smithi138 ceph-mon[31441]: pgmap v307: 129 pgs: 129 active+clean; 58 MiB data, 840 MiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 425 KiB/s wr, 48 op/s 2024-04-24T09:01:26.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:01:26 smithi189 ceph-mon[31791]: pgmap v307: 129 pgs: 129 active+clean; 58 MiB data, 840 MiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 425 KiB/s wr, 48 op/s 2024-04-24T09:01:26.606 INFO:tasks.workunit.client.0.smithi083.stdout: 1 57018 0.64 MB/sec execute 360 sec latency 47.512 ms 2024-04-24T09:01:27.606 INFO:tasks.workunit.client.0.smithi083.stdout: 1 57051 0.63 MB/sec execute 361 sec latency 421.963 ms 2024-04-24T09:01:28.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:01:28 smithi083 ceph-mon[27907]: pgmap v308: 129 pgs: 129 active+clean; 58 MiB data, 840 MiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 425 KiB/s wr, 48 op/s 2024-04-24T09:01:28.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:01:28 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:01:28.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:01:28 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:01:28.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:01:28 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:01:28.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:01:28 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:01:28.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:01:28 smithi138 ceph-mon[31441]: pgmap v308: 129 pgs: 129 active+clean; 58 MiB data, 840 MiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 425 KiB/s wr, 48 op/s 2024-04-24T09:01:28.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:01:28 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:01:28.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:01:28 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:01:28.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:01:28 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:01:28.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:01:28 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:01:28.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:01:28 smithi189 ceph-mon[31791]: pgmap v308: 129 pgs: 129 active+clean; 58 MiB data, 840 MiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 425 KiB/s wr, 48 op/s 2024-04-24T09:01:28.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:01:28 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:01:28.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:01:28 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:01:28.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:01:28 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:01:28.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:01:28 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:01:28.606 INFO:tasks.workunit.client.0.smithi083.stdout: 1 57108 0.63 MB/sec execute 362 sec latency 51.113 ms 2024-04-24T09:01:29.607 INFO:tasks.workunit.client.0.smithi083.stdout: 1 57172 0.63 MB/sec execute 363 sec latency 54.169 ms 2024-04-24T09:01:30.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:01:30 smithi083 ceph-mon[27907]: pgmap v309: 129 pgs: 129 active+clean; 59 MiB data, 860 MiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 667 KiB/s wr, 69 op/s 2024-04-24T09:01:30.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:01:30 smithi138 ceph-mon[31441]: pgmap v309: 129 pgs: 129 active+clean; 59 MiB data, 860 MiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 667 KiB/s wr, 69 op/s 2024-04-24T09:01:30.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:01:30 smithi189 ceph-mon[31791]: pgmap v309: 129 pgs: 129 active+clean; 59 MiB data, 860 MiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 667 KiB/s wr, 69 op/s 2024-04-24T09:01:30.607 INFO:tasks.workunit.client.0.smithi083.stdout: 1 57230 0.63 MB/sec execute 364 sec latency 47.099 ms 2024-04-24T09:01:31.607 INFO:tasks.workunit.client.0.smithi083.stdout: 1 57294 0.63 MB/sec execute 365 sec latency 52.821 ms 2024-04-24T09:01:31.783 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.783+0000 7f1814800640 1 -- 172.21.15.83:0/3174926672 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f180c1085f0 msgr2=0x7f180c108a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:31.783 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.783+0000 7f1814800640 1 --2- 172.21.15.83:0/3174926672 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f180c1085f0 0x7f180c108a70 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f17fc009930 tx=0x7f17fc02a420 comp rx=0 tx=0).stop 2024-04-24T09:01:31.783 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.783+0000 7f1814800640 1 -- 172.21.15.83:0/3174926672 shutdown_connections 2024-04-24T09:01:31.783 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.783+0000 7f1814800640 1 --2- 172.21.15.83:0/3174926672 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f180c108fb0 0x7f180c1154c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:31.783 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.783+0000 7f1814800640 1 --2- 172.21.15.83:0/3174926672 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f180c1085f0 0x7f180c108a70 unknown :-1 s=CLOSED pgs=1671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:31.783 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.783+0000 7f1814800640 1 --2- 172.21.15.83:0/3174926672 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f180c10e690 0x7f180c10ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:31.783 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.783+0000 7f1814800640 1 -- 172.21.15.83:0/3174926672 >> 172.21.15.83:0/3174926672 conn(0x7f180c076590 msgr2=0x7f180c0789b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:31.783 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.784+0000 7f1814800640 1 -- 172.21.15.83:0/3174926672 shutdown_connections 2024-04-24T09:01:31.783 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.784+0000 7f1814800640 1 -- 172.21.15.83:0/3174926672 wait complete. 2024-04-24T09:01:31.785 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.784+0000 7f1814800640 1 Processor -- start 2024-04-24T09:01:31.786 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.785+0000 7f1814800640 1 -- start start 2024-04-24T09:01:31.786 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.785+0000 7f1814800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f180c1085f0 0x7f180c19edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:31.786 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.785+0000 7f1814800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f180c108fb0 0x7f180c19f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:31.786 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.785+0000 7f1814800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f180c10e690 0x7f180c1a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:31.786 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.785+0000 7f1814800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f180c1188e0 con 0x7f180c10e690 2024-04-24T09:01:31.786 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.785+0000 7f1814800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f180c118760 con 0x7f180c108fb0 2024-04-24T09:01:31.786 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.785+0000 7f1814800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f180c118a60 con 0x7f180c1085f0 2024-04-24T09:01:31.786 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.785+0000 7f1812800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f180c10e690 0x7f180c1a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:31.786 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.785+0000 7f1812800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f180c10e690 0x7f180c1a3680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:59376/0 (socket says 172.21.15.83:59376) 2024-04-24T09:01:31.786 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.785+0000 7f1812800640 1 -- 172.21.15.83:0/1314552074 learned_addr learned my addr 172.21.15.83:0/1314552074 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:01:31.786 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.786+0000 7f1811e00640 1 --2- 172.21.15.83:0/1314552074 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f180c1085f0 0x7f180c19edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:31.786 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.786+0000 7f1811400640 1 --2- 172.21.15.83:0/1314552074 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f180c108fb0 0x7f180c19f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:31.787 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.786+0000 7f1812800640 1 -- 172.21.15.83:0/1314552074 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f180c1085f0 msgr2=0x7f180c19edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:31.787 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.786+0000 7f1812800640 1 --2- 172.21.15.83:0/1314552074 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f180c1085f0 0x7f180c19edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:31.787 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.786+0000 7f1812800640 1 -- 172.21.15.83:0/1314552074 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f180c108fb0 msgr2=0x7f180c19f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:31.787 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.786+0000 7f1812800640 1 --2- 172.21.15.83:0/1314552074 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f180c108fb0 0x7f180c19f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:31.787 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.786+0000 7f1812800640 1 -- 172.21.15.83:0/1314552074 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f180c1a3db0 con 0x7f180c10e690 2024-04-24T09:01:31.787 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.786+0000 7f1812800640 1 --2- 172.21.15.83:0/1314552074 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f180c10e690 0x7f180c1a3680 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f180800c970 tx=0x7f180800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:31.787 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.786+0000 7f1802a00640 1 -- 172.21.15.83:0/1314552074 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1808007bb0 con 0x7f180c10e690 2024-04-24T09:01:31.787 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.786+0000 7f1802a00640 1 -- 172.21.15.83:0/1314552074 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1808007d50 con 0x7f180c10e690 2024-04-24T09:01:31.787 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.786+0000 7f1814800640 1 -- 172.21.15.83:0/1314552074 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f180c1a40a0 con 0x7f180c10e690 2024-04-24T09:01:31.787 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.787+0000 7f1802a00640 1 -- 172.21.15.83:0/1314552074 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1808004e10 con 0x7f180c10e690 2024-04-24T09:01:31.787 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.787+0000 7f1814800640 1 -- 172.21.15.83:0/1314552074 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f180c1ab930 con 0x7f180c10e690 2024-04-24T09:01:31.789 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.790+0000 7f1802a00640 1 -- 172.21.15.83:0/1314552074 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1808005020 con 0x7f180c10e690 2024-04-24T09:01:31.789 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.790+0000 7f1817621740 1 Processor -- start 2024-04-24T09:01:31.789 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.790+0000 7f1817621740 1 -- start start 2024-04-24T09:01:31.790 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.791+0000 7f1817621740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c8b845f1b0 0x55c8b845f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:31.790 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.791+0000 7f1817621740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8b845fad0 0x55c8b8469e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:31.790 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.791+0000 7f1817621740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c8b846a390 0x55c8b846c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:31.790 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.791+0000 7f1817621740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c8b83c5900 con 0x55c8b846a390 2024-04-24T09:01:31.790 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.791+0000 7f1817621740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c8b836bf20 con 0x55c8b845fad0 2024-04-24T09:01:31.790 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.791+0000 7f1817621740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55c8b83c18c0 con 0x55c8b845f1b0 2024-04-24T09:01:31.790 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.791+0000 7f1811400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c8b846a390 0x55c8b846c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:31.791 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.791+0000 7f1811400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c8b846a390 0x55c8b846c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:59388/0 (socket says 172.21.15.83:59388) 2024-04-24T09:01:31.791 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.791+0000 7f1811400640 1 -- 172.21.15.83:0/2270199329 learned_addr learned my addr 172.21.15.83:0/2270199329 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:01:31.791 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.791+0000 7f1811400640 1 --2- 172.21.15.83:0/2270199329 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c8b845f1b0 0x55c8b845f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:31.791 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.791+0000 7f1811e00640 1 --2- 172.21.15.83:0/2270199329 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8b845fad0 0x55c8b8469e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:31.792 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.792+0000 7f1802a00640 1 --2- 172.21.15.83:0/1314552074 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f17f80a3e40 0x7f17f80a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:31.792 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.792+0000 7f1811400640 1 -- 172.21.15.83:0/2270199329 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c8b845f1b0 msgr2=0x55c8b845f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:31.793 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.792+0000 7f1811400640 1 --2- 172.21.15.83:0/2270199329 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c8b845f1b0 0x55c8b845f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:31.793 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.792+0000 7f1811400640 1 -- 172.21.15.83:0/2270199329 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8b845fad0 msgr2=0x55c8b8469e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:31.793 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.792+0000 7f1811400640 1 --2- 172.21.15.83:0/2270199329 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8b845fad0 0x55c8b8469e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:31.793 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.792+0000 7f1811400640 1 -- 172.21.15.83:0/2270199329 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8b8390e10 con 0x55c8b846a390 2024-04-24T09:01:31.793 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.792+0000 7f1802a00640 1 -- 172.21.15.83:0/1314552074 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7f18080d08c0 con 0x7f180c10e690 2024-04-24T09:01:31.793 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.792+0000 7f1812800640 1 --2- 172.21.15.83:0/1314552074 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f17f80a3e40 0x7f17f80a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:31.793 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.792+0000 7f1811400640 1 --2- 172.21.15.83:0/2270199329 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c8b846a390 0x55c8b846c780 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f17fc02a940 tx=0x7f17fc0058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c00686c6d8f6be4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:31.793 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.792+0000 7f17f7000640 1 -- 172.21.15.83:0/2270199329 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17fc035040 con 0x55c8b846a390 2024-04-24T09:01:31.793 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.793+0000 7f17f7000640 1 -- 172.21.15.83:0/2270199329 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f17fc005020 con 0x55c8b846a390 2024-04-24T09:01:31.793 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.793+0000 7f17f7000640 1 -- 172.21.15.83:0/2270199329 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17fc0049a0 con 0x55c8b846a390 2024-04-24T09:01:31.793 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.793+0000 7f1817621740 1 -- 172.21.15.83:0/2270199329 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c8b846a390 msgr2=0x55c8b846c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:31.793 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.793+0000 7f1817621740 1 --2- 172.21.15.83:0/2270199329 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c8b846a390 0x55c8b846c780 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f17fc02a940 tx=0x7f17fc0058b0 comp rx=0 tx=0).stop 2024-04-24T09:01:31.793 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.793+0000 7f1812800640 1 --2- 172.21.15.83:0/1314552074 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f17f80a3e40 0x7f17f80a6300 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f1808004fd0 tx=0x7f180800af20 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:31.794 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.794+0000 7f1817621740 1 -- 172.21.15.83:0/2270199329 shutdown_connections 2024-04-24T09:01:31.794 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.794+0000 7f1817621740 1 --2- 172.21.15.83:0/2270199329 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c8b846a390 0x55c8b846c780 unknown :-1 s=CLOSED pgs=1673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:31.794 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.794+0000 7f1817621740 1 --2- 172.21.15.83:0/2270199329 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8b845fad0 0x55c8b8469e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:31.794 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.794+0000 7f1817621740 1 --2- 172.21.15.83:0/2270199329 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c8b845f1b0 0x55c8b845f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:31.794 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.794+0000 7f1817621740 1 -- 172.21.15.83:0/2270199329 >> 172.21.15.83:0/2270199329 conn(0x55c8b845b6c0 msgr2=0x55c8b845baa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:31.794 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.794+0000 7f1817621740 1 -- 172.21.15.83:0/2270199329 shutdown_connections 2024-04-24T09:01:31.794 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.794+0000 7f1817621740 1 -- 172.21.15.83:0/2270199329 wait complete. 2024-04-24T09:01:31.797 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.794+0000 7f1817621740 1 -- start start 2024-04-24T09:01:31.797 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.794+0000 7f1817621740 1 Processor -- start 2024-04-24T09:01:31.797 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.795+0000 7f1817621740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c8b845f1b0 0x55c8b8467320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:31.797 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.795+0000 7f1817621740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8b845fad0 0x55c8b8467860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:31.797 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.795+0000 7f1817621740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c8b846a390 0x55c8b8467da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:31.797 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.795+0000 7f1817621740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c8b846cf60 con 0x55c8b846a390 2024-04-24T09:01:31.797 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.795+0000 7f1817621740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c8b837f2c0 con 0x55c8b845fad0 2024-04-24T09:01:31.797 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.795+0000 7f1817621740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55c8b846d0e0 con 0x55c8b845f1b0 2024-04-24T09:01:31.798 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.795+0000 7f1817621740 10 client.0 _ll_register_callbacks cb 0x55c8b80d7a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:01:31.798 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.795+0000 7f1811400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c8b846a390 0x55c8b8467da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:31.798 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.795+0000 7f1811400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c8b846a390 0x55c8b8467da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:59390/0 (socket says 172.21.15.83:59390) 2024-04-24T09:01:31.798 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.795+0000 7f1811400640 1 -- 172.21.15.83:0/1481091214 learned_addr learned my addr 172.21.15.83:0/1481091214 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:01:31.798 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.795+0000 7f1811400640 1 --2- 172.21.15.83:0/1481091214 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c8b845f1b0 0x55c8b8467320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:31.798 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.795+0000 7f1811e00640 1 --2- 172.21.15.83:0/1481091214 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8b845fad0 0x55c8b8467860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:31.798 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.796+0000 7f1811400640 1 -- 172.21.15.83:0/1481091214 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c8b845f1b0 msgr2=0x55c8b8467320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:31.798 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.796+0000 7f1811400640 1 --2- 172.21.15.83:0/1481091214 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c8b845f1b0 0x55c8b8467320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:31.798 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.796+0000 7f1811400640 1 -- 172.21.15.83:0/1481091214 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8b845fad0 msgr2=0x55c8b8467860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:31.798 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.796+0000 7f1811400640 1 --2- 172.21.15.83:0/1481091214 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8b845fad0 0x55c8b8467860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:31.798 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.796+0000 7f1811400640 1 -- 172.21.15.83:0/1481091214 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8b838ffd0 con 0x55c8b846a390 2024-04-24T09:01:31.798 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.796+0000 7f1811400640 1 --2- 172.21.15.83:0/1481091214 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c8b846a390 0x55c8b8467da0 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7f17fc009650 tx=0x7f17fc02a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=677ffc2b690ad97c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:31.798 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.796+0000 7f17eb400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:01:31.799 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.796+0000 7f17eb400640 1 -- 172.21.15.83:0/1481091214 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17fc054060 con 0x55c8b846a390 2024-04-24T09:01:31.799 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.796+0000 7f1817621740 1 -- 172.21.15.83:0/1481091214 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17fc005020 con 0x55c8b846a390 2024-04-24T09:01:31.799 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.796+0000 7f17eb400640 1 -- 172.21.15.83:0/1481091214 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7f17fc0053d0 con 0x55c8b846a390 2024-04-24T09:01:31.799 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.796+0000 7f17eb400640 1 -- 172.21.15.83:0/1481091214 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f17fc05b050 con 0x55c8b846a390 2024-04-24T09:01:31.799 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.797+0000 7f1817621740 10 client.19182 fetch_fsmap learned FSMap version 35 2024-04-24T09:01:31.799 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.797+0000 7f1817621740 1 -- 172.21.15.83:0/1481091214 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55c8b80d8210 con 0x55c8b846a390 2024-04-24T09:01:31.799 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.797+0000 7f17eb400640 1 -- 172.21.15.83:0/1481091214 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f17fc005020 con 0x55c8b846a390 2024-04-24T09:01:31.799 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.797+0000 7f17eb400640 1 -- 172.21.15.83:0/1481091214 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7f17fc03a040 con 0x55c8b846a390 2024-04-24T09:01:31.799 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.797+0000 7f1817621740 10 client.19182 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T09:01:31.799 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.797+0000 7f1817621740 10 client.19182 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:01:31.799 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.797+0000 7f1817621740 20 client.19182 populate_metadata read hostname 'smithi083' 2024-04-24T09:01:31.799 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.797+0000 7f1817621740 1 --2- 172.21.15.83:0/1481091214 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55c8b847a300 0x55c8b847c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:31.799 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.797+0000 7f1817621740 4 client.19182 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:01:31.799 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.797+0000 7f1817621740 1 -- 172.21.15.83:0/1481091214 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c8b83d3320 con 0x55c8b847a300 2024-04-24T09:01:31.800 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.798+0000 7f1811e00640 1 --2- 172.21.15.83:0/1481091214 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55c8b847a300 0x55c8b847c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:31.800 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.799+0000 7f1811e00640 1 --2- 172.21.15.83:0/1481091214 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55c8b847a300 0x55c8b847c6f0 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae67f6341d0c871e server_cookie=ad0496b1dc73641e in_seq=0 out_seq=0 2024-04-24T09:01:31.800 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.799+0000 7f17eb400640 10 client.19182 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:01:31.801 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.802+0000 7f17eb400640 1 -- 172.21.15.83:0/1481091214 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c8b83d3320 con 0x55c8b847a300 2024-04-24T09:01:31.801 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.802+0000 7f17eb400640 10 client.19182 handle_command_reply: tid=0 2024-04-24T09:01:31.801 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.802+0000 7f1817621740 1 client.19182 shutdown 2024-04-24T09:01:31.801 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.802+0000 7f1817621740 1 -- 172.21.15.83:0/1481091214 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c8b846a390 msgr2=0x55c8b8467da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:31.801 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.802+0000 7f1817621740 1 --2- 172.21.15.83:0/1481091214 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c8b846a390 0x55c8b8467da0 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7f17fc009650 tx=0x7f17fc02a420 comp rx=0 tx=0).stop 2024-04-24T09:01:31.802 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.802+0000 7f1817621740 1 -- 172.21.15.83:0/1481091214 shutdown_connections 2024-04-24T09:01:31.802 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.802+0000 7f1817621740 1 --2- 172.21.15.83:0/1481091214 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55c8b847a300 0x55c8b847c6f0 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:31.802 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.802+0000 7f1817621740 1 --2- 172.21.15.83:0/1481091214 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c8b846a390 0x55c8b8467da0 unknown :-1 s=CLOSED pgs=1674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:31.802 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.802+0000 7f17eb400640 0 client.19182 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:01:31.802 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.802+0000 7f1817621740 1 --2- 172.21.15.83:0/1481091214 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8b845fad0 0x55c8b8467860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:31.802 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.802+0000 7f1817621740 1 --2- 172.21.15.83:0/1481091214 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c8b845f1b0 0x55c8b8467320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:31.802 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.802+0000 7f1817621740 1 -- 172.21.15.83:0/1481091214 >> 172.21.15.83:0/1481091214 conn(0x55c8b845b6c0 msgr2=0x55c8b845a840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:31.802 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.803+0000 7f1817621740 1 -- 172.21.15.83:0/1481091214 shutdown_connections 2024-04-24T09:01:31.802 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.803+0000 7f1817621740 1 -- 172.21.15.83:0/1481091214 wait complete. 2024-04-24T09:01:31.803 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.803+0000 7f1817621740 20 client.19182 trim_cache size 0 max 16384 2024-04-24T09:01:31.819 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.820+0000 7f1817621740 1 Processor -- start 2024-04-24T09:01:31.819 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.820+0000 7f1817621740 1 -- start start 2024-04-24T09:01:31.819 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.820+0000 7f1817621740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c8b845f1b0 0x55c8b8461b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:31.819 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.820+0000 7f1817621740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8b845fad0 0x55c8b845a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:31.819 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.820+0000 7f1817621740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c8b846a390 0x55c8b847f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:31.819 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.820+0000 7f1817621740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f180c1b5bd0 con 0x55c8b845f1b0 2024-04-24T09:01:31.820 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.820+0000 7f1817621740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f180c1b7710 con 0x55c8b845fad0 2024-04-24T09:01:31.820 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.820+0000 7f1817621740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f180c1b62a0 con 0x55c8b846a390 2024-04-24T09:01:31.820 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.820+0000 7f1811400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c8b845f1b0 0x55c8b8461b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:31.820 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.820+0000 7f1811400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c8b845f1b0 0x55c8b8461b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:59392/0 (socket says 172.21.15.83:59392) 2024-04-24T09:01:31.820 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.820+0000 7f1811400640 1 -- 172.21.15.83:0/1380927260 learned_addr learned my addr 172.21.15.83:0/1380927260 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:01:31.821 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.821+0000 7f1811400640 1 -- 172.21.15.83:0/1380927260 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c8b846a390 msgr2=0x55c8b847f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:31.821 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.821+0000 7f1811400640 1 --2- 172.21.15.83:0/1380927260 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c8b846a390 0x55c8b847f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:31.821 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.821+0000 7f1811400640 1 -- 172.21.15.83:0/1380927260 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8b845fad0 msgr2=0x55c8b845a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:31.821 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.821+0000 7f1811e00640 1 --2- 172.21.15.83:0/1380927260 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8b845fad0 0x55c8b845a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:31.821 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.821+0000 7f1811400640 1 --2- 172.21.15.83:0/1380927260 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8b845fad0 0x55c8b845a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:31.822 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.821+0000 7f1811400640 1 -- 172.21.15.83:0/1380927260 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8b81a4620 con 0x55c8b845f1b0 2024-04-24T09:01:31.822 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.821+0000 7f1811400640 1 --2- 172.21.15.83:0/1380927260 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c8b845f1b0 0x55c8b8461b70 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f17fc03d7c0 tx=0x7f17fc005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2079ba00ad5eaa41 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:31.822 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.821+0000 7f17f6600640 1 -- 172.21.15.83:0/1380927260 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17fc054070 con 0x55c8b845f1b0 2024-04-24T09:01:31.822 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.821+0000 7f17f6600640 1 -- 172.21.15.83:0/1380927260 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f17fc009780 con 0x55c8b845f1b0 2024-04-24T09:01:31.822 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.821+0000 7f17f6600640 1 -- 172.21.15.83:0/1380927260 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17fc035040 con 0x55c8b845f1b0 2024-04-24T09:01:31.822 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.822+0000 7f1817621740 1 -- 172.21.15.83:0/1380927260 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c8b845f1b0 msgr2=0x55c8b8461b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:31.822 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.822+0000 7f1817621740 1 --2- 172.21.15.83:0/1380927260 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c8b845f1b0 0x55c8b8461b70 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f17fc03d7c0 tx=0x7f17fc005fd0 comp rx=0 tx=0).stop 2024-04-24T09:01:31.822 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.822+0000 7f1817621740 1 -- 172.21.15.83:0/1380927260 shutdown_connections 2024-04-24T09:01:31.822 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.822+0000 7f1817621740 1 --2- 172.21.15.83:0/1380927260 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c8b846a390 0x55c8b847f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:31.822 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.822+0000 7f1817621740 1 --2- 172.21.15.83:0/1380927260 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8b845fad0 0x55c8b845a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:31.822 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.822+0000 7f1817621740 1 --2- 172.21.15.83:0/1380927260 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c8b845f1b0 0x55c8b8461b70 unknown :-1 s=CLOSED pgs=1675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:31.822 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.822+0000 7f1817621740 1 -- 172.21.15.83:0/1380927260 >> 172.21.15.83:0/1380927260 conn(0x55c8b845b6c0 msgr2=0x55c8b8459fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:31.822 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.822+0000 7f1817621740 1 -- 172.21.15.83:0/1380927260 shutdown_connections 2024-04-24T09:01:31.823 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.822+0000 7f1817621740 1 -- 172.21.15.83:0/1380927260 wait complete. 2024-04-24T09:01:31.825 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.823+0000 7f1817621740 1 -- start start 2024-04-24T09:01:31.825 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.823+0000 7f1817621740 1 Processor -- start 2024-04-24T09:01:31.825 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.824+0000 7f1817621740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c8b845f1b0 0x55c8b847d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:31.825 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.824+0000 7f1817621740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8b845fad0 0x55c8b847d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:31.825 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.824+0000 7f1817621740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c8b846a390 0x55c8b847dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:31.825 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.824+0000 7f1817621740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f180c1b7040 con 0x55c8b845f1b0 2024-04-24T09:01:31.825 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.824+0000 7f1817621740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f180c1b6970 con 0x55c8b845fad0 2024-04-24T09:01:31.825 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.824+0000 7f1817621740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f180c1b3bd0 con 0x55c8b846a390 2024-04-24T09:01:31.825 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.824+0000 7f1817621740 10 client.0 _ll_register_callbacks cb 0x7f1804009a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:01:31.825 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.824+0000 7f1811400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c8b845f1b0 0x55c8b847d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:31.826 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.824+0000 7f1811400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c8b845f1b0 0x55c8b847d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:59400/0 (socket says 172.21.15.83:59400) 2024-04-24T09:01:31.826 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.824+0000 7f1811400640 1 -- 172.21.15.83:0/388163915 learned_addr learned my addr 172.21.15.83:0/388163915 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:01:31.826 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.824+0000 7f1811e00640 1 --2- 172.21.15.83:0/388163915 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8b845fad0 0x55c8b847d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:31.826 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.824+0000 7f1811400640 1 -- 172.21.15.83:0/388163915 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c8b846a390 msgr2=0x55c8b847dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:31.826 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.824+0000 7f1811400640 1 --2- 172.21.15.83:0/388163915 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c8b846a390 0x55c8b847dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:31.826 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.824+0000 7f1811400640 1 -- 172.21.15.83:0/388163915 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8b845fad0 msgr2=0x55c8b847d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:31.826 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.824+0000 7f1811400640 1 --2- 172.21.15.83:0/388163915 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8b845fad0 0x55c8b847d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:31.826 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.824+0000 7f1811400640 1 -- 172.21.15.83:0/388163915 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8b84df690 con 0x55c8b845f1b0 2024-04-24T09:01:31.826 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.825+0000 7f1811400640 1 --2- 172.21.15.83:0/388163915 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c8b845f1b0 0x55c8b847d2d0 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f17fc04b040 tx=0x7f17fc031570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54a3f6105e50efb0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:31.826 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.825+0000 7f17eb400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:01:31.826 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.825+0000 7f17eb400640 1 -- 172.21.15.83:0/388163915 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17fc067070 con 0x55c8b845f1b0 2024-04-24T09:01:31.826 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.825+0000 7f17eb400640 1 -- 172.21.15.83:0/388163915 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7f17fc031c40 con 0x55c8b845f1b0 2024-04-24T09:01:31.826 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.825+0000 7f1817621740 1 -- 172.21.15.83:0/388163915 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17fc009780 con 0x55c8b845f1b0 2024-04-24T09:01:31.826 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.825+0000 7f17eb400640 1 -- 172.21.15.83:0/388163915 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f17fc035070 con 0x55c8b845f1b0 2024-04-24T09:01:31.827 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.825+0000 7f1817621740 10 client.19188 fetch_fsmap learned FSMap version 35 2024-04-24T09:01:31.827 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.825+0000 7f1817621740 1 -- 172.21.15.83:0/388163915 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55c8b84e0960 con 0x55c8b845f1b0 2024-04-24T09:01:31.827 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.826+0000 7f17eb400640 1 -- 172.21.15.83:0/388163915 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f17fc009780 con 0x55c8b845f1b0 2024-04-24T09:01:31.827 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.826+0000 7f17eb400640 1 -- 172.21.15.83:0/388163915 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7f17fc03dce0 con 0x55c8b845f1b0 2024-04-24T09:01:31.827 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.826+0000 7f1817621740 10 client.19188 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T09:01:31.827 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.826+0000 7f1817621740 10 client.19188 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:01:31.827 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.826+0000 7f1817621740 20 client.19188 populate_metadata read hostname 'smithi083' 2024-04-24T09:01:31.827 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.826+0000 7f1817621740 1 --2- 172.21.15.83:0/388163915 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55c8b84e0f70 0x55c8b84e3360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:31.827 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.826+0000 7f1817621740 4 client.19188 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T09:01:31.827 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.826+0000 7f1817621740 1 -- 172.21.15.83:0/388163915 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c8b83cd190 con 0x55c8b84e0f70 2024-04-24T09:01:31.828 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.827+0000 7f1811e00640 1 --2- 172.21.15.83:0/388163915 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55c8b84e0f70 0x55c8b84e3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:31.828 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.828+0000 7f1811e00640 1 --2- 172.21.15.83:0/388163915 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55c8b84e0f70 0x55c8b84e3360 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f55d9240cee7579c server_cookie=bcfbbf7cf3c5202c in_seq=0 out_seq=0 2024-04-24T09:01:31.828 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.828+0000 7f17eb400640 10 client.19188 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:01:31.882 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.883+0000 7f17eb400640 1 -- 172.21.15.83:0/388163915 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c8b83cd190 con 0x55c8b84e0f70 2024-04-24T09:01:31.882 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.883+0000 7f17eb400640 10 client.19188 handle_command_reply: tid=0 2024-04-24T09:01:31.882 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.883+0000 7f1817621740 1 client.19188 shutdown 2024-04-24T09:01:31.883 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.883+0000 7f1817621740 1 -- 172.21.15.83:0/388163915 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c8b845f1b0 msgr2=0x55c8b847d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:31.883 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.883+0000 7f1817621740 1 --2- 172.21.15.83:0/388163915 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c8b845f1b0 0x55c8b847d2d0 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f17fc04b040 tx=0x7f17fc031570 comp rx=0 tx=0).stop 2024-04-24T09:01:31.883 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.883+0000 7f1817621740 1 -- 172.21.15.83:0/388163915 shutdown_connections 2024-04-24T09:01:31.883 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.883+0000 7f1817621740 1 --2- 172.21.15.83:0/388163915 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55c8b84e0f70 0x55c8b84e3360 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:31.883 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.883+0000 7f1817621740 1 --2- 172.21.15.83:0/388163915 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c8b846a390 0x55c8b847dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:31.883 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.883+0000 7f1817621740 1 --2- 172.21.15.83:0/388163915 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8b845fad0 0x55c8b847d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:31.883 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.883+0000 7f17eb400640 0 client.19188 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:01:31.883 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.883+0000 7f1817621740 1 --2- 172.21.15.83:0/388163915 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c8b845f1b0 0x55c8b847d2d0 unknown :-1 s=CLOSED pgs=1676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:31.883 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.884+0000 7f1817621740 1 -- 172.21.15.83:0/388163915 >> 172.21.15.83:0/388163915 conn(0x55c8b845b6c0 msgr2=0x55c8b845a840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:31.883 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.884+0000 7f1817621740 1 -- 172.21.15.83:0/388163915 shutdown_connections 2024-04-24T09:01:31.884 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.884+0000 7f1817621740 1 -- 172.21.15.83:0/388163915 wait complete. 2024-04-24T09:01:31.884 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.884+0000 7f1817621740 20 client.19188 trim_cache size 0 max 16384 2024-04-24T09:01:31.884 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T09:01:31.884 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T09:01:31.884 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T09:01:31.884 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T09:01:31.885 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.885+0000 7f1814800640 1 -- 172.21.15.83:0/1314552074 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f17f80a3e40 msgr2=0x7f17f80a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:31.885 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.885+0000 7f1814800640 1 --2- 172.21.15.83:0/1314552074 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f17f80a3e40 0x7f17f80a6300 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f1808004fd0 tx=0x7f180800af20 comp rx=0 tx=0).stop 2024-04-24T09:01:31.885 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.885+0000 7f1814800640 1 -- 172.21.15.83:0/1314552074 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f180c10e690 msgr2=0x7f180c1a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:31.885 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.885+0000 7f1814800640 1 --2- 172.21.15.83:0/1314552074 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f180c10e690 0x7f180c1a3680 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f180800c970 tx=0x7f180800ce30 comp rx=0 tx=0).stop 2024-04-24T09:01:31.885 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.885+0000 7f1814800640 1 -- 172.21.15.83:0/1314552074 shutdown_connections 2024-04-24T09:01:31.885 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.885+0000 7f1814800640 1 --2- 172.21.15.83:0/1314552074 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f17f80a3e40 0x7f17f80a6300 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:31.885 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.885+0000 7f1814800640 1 --2- 172.21.15.83:0/1314552074 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f180c10e690 0x7f180c1a3680 unknown :-1 s=CLOSED pgs=1672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:31.885 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.885+0000 7f1814800640 1 --2- 172.21.15.83:0/1314552074 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f180c108fb0 0x7f180c19f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:31.886 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.885+0000 7f1814800640 1 --2- 172.21.15.83:0/1314552074 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f180c1085f0 0x7f180c19edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:31.886 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.885+0000 7f1814800640 1 -- 172.21.15.83:0/1314552074 >> 172.21.15.83:0/1314552074 conn(0x7f180c076590 msgr2=0x7f180c077c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:31.886 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.886+0000 7f1814800640 1 -- 172.21.15.83:0/1314552074 shutdown_connections 2024-04-24T09:01:31.886 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:31.886+0000 7f1814800640 1 -- 172.21.15.83:0/1314552074 wait complete. 2024-04-24T09:01:32.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:01:32 smithi083 ceph-mon[27907]: pgmap v310: 129 pgs: 129 active+clean; 59 MiB data, 860 MiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 461 KiB/s wr, 50 op/s 2024-04-24T09:01:32.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:01:32 smithi138 ceph-mon[31441]: pgmap v310: 129 pgs: 129 active+clean; 59 MiB data, 860 MiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 461 KiB/s wr, 50 op/s 2024-04-24T09:01:32.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:01:32 smithi189 ceph-mon[31791]: pgmap v310: 129 pgs: 129 active+clean; 59 MiB data, 860 MiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 461 KiB/s wr, 50 op/s 2024-04-24T09:01:32.607 INFO:tasks.workunit.client.0.smithi083.stdout: 1 57369 0.63 MB/sec execute 366 sec latency 76.316 ms 2024-04-24T09:01:33.607 INFO:tasks.workunit.client.0.smithi083.stdout: 1 57459 0.63 MB/sec execute 367 sec latency 62.098 ms 2024-04-24T09:01:34.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:01:34 smithi083 ceph-mon[27907]: pgmap v311: 129 pgs: 129 active+clean; 60 MiB data, 843 MiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 606 KiB/s wr, 78 op/s 2024-04-24T09:01:34.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:01:34 smithi138 ceph-mon[31441]: pgmap v311: 129 pgs: 129 active+clean; 60 MiB data, 843 MiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 606 KiB/s wr, 78 op/s 2024-04-24T09:01:34.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:01:34 smithi189 ceph-mon[31791]: pgmap v311: 129 pgs: 129 active+clean; 60 MiB data, 843 MiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 606 KiB/s wr, 78 op/s 2024-04-24T09:01:34.607 INFO:tasks.workunit.client.0.smithi083.stdout: 1 57583 0.62 MB/sec execute 368 sec latency 60.761 ms 2024-04-24T09:01:35.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:01:35 smithi083 ceph-mon[27907]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:01:35.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:01:35 smithi138 ceph-mon[31441]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:01:35.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:01:35 smithi189 ceph-mon[31791]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:01:35.607 INFO:tasks.workunit.client.0.smithi083.stdout: 1 57694 0.62 MB/sec execute 369 sec latency 51.743 ms 2024-04-24T09:01:36.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:01:36 smithi083 ceph-mon[27907]: pgmap v312: 129 pgs: 129 active+clean; 60 MiB data, 843 MiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 387 KiB/s wr, 48 op/s 2024-04-24T09:01:36.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:01:36 smithi138 ceph-mon[31441]: pgmap v312: 129 pgs: 129 active+clean; 60 MiB data, 843 MiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 387 KiB/s wr, 48 op/s 2024-04-24T09:01:36.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:01:36 smithi189 ceph-mon[31791]: pgmap v312: 129 pgs: 129 active+clean; 60 MiB data, 843 MiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 387 KiB/s wr, 48 op/s 2024-04-24T09:01:36.607 INFO:tasks.workunit.client.0.smithi083.stdout: 1 57818 0.63 MB/sec execute 370 sec latency 51.121 ms 2024-04-24T09:01:37.607 INFO:tasks.workunit.client.0.smithi083.stdout: 1 57926 0.62 MB/sec execute 371 sec latency 54.565 ms 2024-04-24T09:01:38.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:01:38 smithi138 ceph-mon[31441]: pgmap v313: 129 pgs: 129 active+clean; 60 MiB data, 843 MiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 387 KiB/s wr, 48 op/s 2024-04-24T09:01:38.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:01:38 smithi189 ceph-mon[31791]: pgmap v313: 129 pgs: 129 active+clean; 60 MiB data, 843 MiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 387 KiB/s wr, 48 op/s 2024-04-24T09:01:38.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:01:38 smithi083 ceph-mon[27907]: pgmap v313: 129 pgs: 129 active+clean; 60 MiB data, 843 MiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 387 KiB/s wr, 48 op/s 2024-04-24T09:01:38.608 INFO:tasks.workunit.client.0.smithi083.stdout: 1 58031 0.62 MB/sec execute 372 sec latency 50.838 ms 2024-04-24T09:01:39.608 INFO:tasks.workunit.client.0.smithi083.stdout: 1 58186 0.63 MB/sec execute 373 sec latency 59.977 ms 2024-04-24T09:01:40.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:01:40 smithi138 ceph-mon[31441]: pgmap v314: 129 pgs: 129 active+clean; 58 MiB data, 845 MiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 588 KiB/s wr, 76 op/s 2024-04-24T09:01:40.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:01:40 smithi189 ceph-mon[31791]: pgmap v314: 129 pgs: 129 active+clean; 58 MiB data, 845 MiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 588 KiB/s wr, 76 op/s 2024-04-24T09:01:40.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:01:40 smithi083 ceph-mon[27907]: pgmap v314: 129 pgs: 129 active+clean; 58 MiB data, 845 MiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 588 KiB/s wr, 76 op/s 2024-04-24T09:01:40.610 INFO:tasks.workunit.client.0.smithi083.stdout: 1 58277 0.63 MB/sec execute 374 sec latency 68.365 ms 2024-04-24T09:01:41.608 INFO:tasks.workunit.client.0.smithi083.stdout: 1 58389 0.63 MB/sec execute 375 sec latency 63.663 ms 2024-04-24T09:01:42.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:01:42 smithi138 ceph-mon[31441]: pgmap v315: 129 pgs: 129 active+clean; 58 MiB data, 845 MiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 346 KiB/s wr, 55 op/s 2024-04-24T09:01:42.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:01:42 smithi189 ceph-mon[31791]: pgmap v315: 129 pgs: 129 active+clean; 58 MiB data, 845 MiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 346 KiB/s wr, 55 op/s 2024-04-24T09:01:42.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:01:42 smithi083 ceph-mon[27907]: pgmap v315: 129 pgs: 129 active+clean; 58 MiB data, 845 MiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 346 KiB/s wr, 55 op/s 2024-04-24T09:01:42.608 INFO:tasks.workunit.client.0.smithi083.stdout: 1 58484 0.62 MB/sec execute 376 sec latency 62.630 ms 2024-04-24T09:01:43.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:01:43 smithi138 ceph-mon[31441]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:01:43.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:01:43 smithi189 ceph-mon[31791]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:01:43.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:01:43 smithi083 ceph-mon[27907]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:01:43.608 INFO:tasks.workunit.client.0.smithi083.stdout: 1 58623 0.62 MB/sec execute 377 sec latency 65.626 ms 2024-04-24T09:01:44.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:01:44 smithi138 ceph-mon[31441]: pgmap v316: 129 pgs: 129 active+clean; 59 MiB data, 830 MiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 428 KiB/s wr, 76 op/s 2024-04-24T09:01:44.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:01:44 smithi189 ceph-mon[31791]: pgmap v316: 129 pgs: 129 active+clean; 59 MiB data, 830 MiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 428 KiB/s wr, 76 op/s 2024-04-24T09:01:44.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:01:44 smithi083 ceph-mon[27907]: pgmap v316: 129 pgs: 129 active+clean; 59 MiB data, 830 MiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 428 KiB/s wr, 76 op/s 2024-04-24T09:01:44.608 INFO:tasks.workunit.client.0.smithi083.stdout: 1 58747 0.62 MB/sec execute 378 sec latency 50.958 ms 2024-04-24T09:01:45.039 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.039+0000 7f8487200640 1 -- 172.21.15.83:0/4070311097 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f848810e750 msgr2=0x7f8488110b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:45.039 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.039+0000 7f8487200640 1 --2- 172.21.15.83:0/4070311097 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f848810e750 0x7f8488110b40 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f8480009ca0 tx=0x7f848002a8c0 comp rx=0 tx=0).stop 2024-04-24T09:01:45.039 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.039+0000 7f8487200640 1 -- 172.21.15.83:0/4070311097 shutdown_connections 2024-04-24T09:01:45.039 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.039+0000 7f8487200640 1 --2- 172.21.15.83:0/4070311097 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f848810e750 0x7f8488110b40 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:45.039 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.039+0000 7f8487200640 1 --2- 172.21.15.83:0/4070311097 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8488101550 0x7f848810e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:45.039 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.039+0000 7f8487200640 1 --2- 172.21.15.83:0/4070311097 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8488100ba0 0x7f8488100f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:45.039 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.039+0000 7f8487200640 1 -- 172.21.15.83:0/4070311097 >> 172.21.15.83:0/4070311097 conn(0x7f84880fc820 msgr2=0x7f84880fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:45.039 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.040+0000 7f8487200640 1 -- 172.21.15.83:0/4070311097 shutdown_connections 2024-04-24T09:01:45.040 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.040+0000 7f8487200640 1 -- 172.21.15.83:0/4070311097 wait complete. 2024-04-24T09:01:45.040 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.041+0000 7f8487200640 1 Processor -- start 2024-04-24T09:01:45.040 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.041+0000 7f8487200640 1 -- start start 2024-04-24T09:01:45.042 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.042+0000 7f8487200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8488100ba0 0x7f84881a0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:45.042 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.042+0000 7f8487200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8488101550 0x7f84881a0960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:45.042 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.042+0000 7f8487200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f848810e750 0x7f848819a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:45.042 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.042+0000 7f8487200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8488114250 con 0x7f8488101550 2024-04-24T09:01:45.042 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.042+0000 7f8487200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f84881140d0 con 0x7f848810e750 2024-04-24T09:01:45.042 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.042+0000 7f8487200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f84881143d0 con 0x7f8488100ba0 2024-04-24T09:01:45.043 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.042+0000 7f8485400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8488101550 0x7f84881a0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:45.043 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.042+0000 7f8485400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8488101550 0x7f84881a0960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48472/0 (socket says 172.21.15.83:48472) 2024-04-24T09:01:45.043 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.042+0000 7f8485400640 1 -- 172.21.15.83:0/2897110519 learned_addr learned my addr 172.21.15.83:0/2897110519 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:01:45.043 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.043+0000 7f8485e00640 1 --2- 172.21.15.83:0/2897110519 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8488100ba0 0x7f84881a0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:45.043 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.043+0000 7f8486800640 1 --2- 172.21.15.83:0/2897110519 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f848810e750 0x7f848819a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:45.043 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.043+0000 7f8485400640 1 -- 172.21.15.83:0/2897110519 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8488100ba0 msgr2=0x7f84881a0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:45.043 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.043+0000 7f8485400640 1 --2- 172.21.15.83:0/2897110519 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8488100ba0 0x7f84881a0420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:45.043 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.043+0000 7f8485400640 1 -- 172.21.15.83:0/2897110519 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f848810e750 msgr2=0x7f848819a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:45.043 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.043+0000 7f8485400640 1 --2- 172.21.15.83:0/2897110519 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f848810e750 0x7f848819a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:45.043 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.043+0000 7f8485400640 1 -- 172.21.15.83:0/2897110519 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f848819ad50 con 0x7f8488101550 2024-04-24T09:01:45.043 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.043+0000 7f8486800640 1 --2- 172.21.15.83:0/2897110519 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f848810e750 0x7f848819a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:01:45.043 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.043+0000 7f8485400640 1 --2- 172.21.15.83:0/2897110519 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8488101550 0x7f84881a0960 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f847400d950 tx=0x7f847400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:45.044 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.044+0000 7f847ea00640 1 -- 172.21.15.83:0/2897110519 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8474017070 con 0x7f8488101550 2024-04-24T09:01:45.044 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.044+0000 7f847ea00640 1 -- 172.21.15.83:0/2897110519 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8474009c30 con 0x7f8488101550 2024-04-24T09:01:45.044 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.044+0000 7f8487200640 1 -- 172.21.15.83:0/2897110519 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f848819b040 con 0x7f8488101550 2024-04-24T09:01:45.044 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.044+0000 7f847ea00640 1 -- 172.21.15.83:0/2897110519 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f847400f620 con 0x7f8488101550 2024-04-24T09:01:45.044 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.044+0000 7f8487200640 1 -- 172.21.15.83:0/2897110519 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84881a72e0 con 0x7f8488101550 2024-04-24T09:01:45.047 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.048+0000 7f848ffc1740 1 Processor -- start 2024-04-24T09:01:45.047 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.048+0000 7f848ffc1740 1 -- start start 2024-04-24T09:01:45.049 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.049+0000 7f848ffc1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5571490c73c0 0x5571490c77a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:45.049 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.049+0000 7f848ffc1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5571490c7ce0 0x5571490d2060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:45.049 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.049+0000 7f848ffc1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5571490d25a0 0x5571490d4990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:45.049 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.049+0000 7f848ffc1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55714902d900 con 0x5571490c73c0 2024-04-24T09:01:45.049 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.049+0000 7f848ffc1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x557148fd3f20 con 0x5571490d25a0 2024-04-24T09:01:45.049 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.049+0000 7f848ffc1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5571490298c0 con 0x5571490c7ce0 2024-04-24T09:01:45.049 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.049+0000 7f8486800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5571490c73c0 0x5571490c77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:45.049 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.049+0000 7f8486800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5571490c73c0 0x5571490c77a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48480/0 (socket says 172.21.15.83:48480) 2024-04-24T09:01:45.049 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.049+0000 7f8486800640 1 -- 172.21.15.83:0/1919055287 learned_addr learned my addr 172.21.15.83:0/1919055287 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:01:45.050 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.049+0000 7f847ea00640 1 -- 172.21.15.83:0/2897110519 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f84740109a0 con 0x7f8488101550 2024-04-24T09:01:45.050 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.050+0000 7f8485e00640 1 --2- 172.21.15.83:0/1919055287 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5571490c7ce0 0x5571490d2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:45.050 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.050+0000 7f8486800640 1 -- 172.21.15.83:0/1919055287 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5571490c7ce0 msgr2=0x5571490d2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:45.050 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.050+0000 7f8486800640 1 --2- 172.21.15.83:0/1919055287 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5571490c7ce0 0x5571490d2060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:45.050 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.050+0000 7f8486800640 1 -- 172.21.15.83:0/1919055287 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5571490d25a0 msgr2=0x5571490d4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:45.050 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.050+0000 7f8486800640 1 --2- 172.21.15.83:0/1919055287 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5571490d25a0 0x5571490d4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:45.050 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.050+0000 7f8486800640 1 -- 172.21.15.83:0/1919055287 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557148ff8e10 con 0x5571490c73c0 2024-04-24T09:01:45.050 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.050+0000 7f8486800640 1 --2- 172.21.15.83:0/1919055287 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5571490c73c0 0x5571490c77a0 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f84800097b0 tx=0x7f8480009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3536a141564fdb51 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:45.052 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.051+0000 7f8473400640 1 -- 172.21.15.83:0/1919055287 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8480031450 con 0x5571490c73c0 2024-04-24T09:01:45.052 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.051+0000 7f8473400640 1 -- 172.21.15.83:0/1919055287 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8480036070 con 0x5571490c73c0 2024-04-24T09:01:45.052 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.051+0000 7f8473400640 1 -- 172.21.15.83:0/1919055287 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84800043a0 con 0x5571490c73c0 2024-04-24T09:01:45.052 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.051+0000 7f848ffc1740 1 -- 172.21.15.83:0/1919055287 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5571490c73c0 msgr2=0x5571490c77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:45.052 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.051+0000 7f848ffc1740 1 --2- 172.21.15.83:0/1919055287 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5571490c73c0 0x5571490c77a0 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f84800097b0 tx=0x7f8480009a20 comp rx=0 tx=0).stop 2024-04-24T09:01:45.052 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.052+0000 7f848ffc1740 1 -- 172.21.15.83:0/1919055287 shutdown_connections 2024-04-24T09:01:45.052 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.052+0000 7f847ea00640 1 --2- 172.21.15.83:0/2897110519 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f846c0a3d90 0x7f846c0a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:45.052 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.052+0000 7f848ffc1740 1 --2- 172.21.15.83:0/1919055287 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5571490d25a0 0x5571490d4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:45.052 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.052+0000 7f848ffc1740 1 --2- 172.21.15.83:0/1919055287 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5571490c7ce0 0x5571490d2060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:45.052 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.052+0000 7f848ffc1740 1 --2- 172.21.15.83:0/1919055287 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5571490c73c0 0x5571490c77a0 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:45.052 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.052+0000 7f848ffc1740 1 -- 172.21.15.83:0/1919055287 >> 172.21.15.83:0/1919055287 conn(0x5571490c38d0 msgr2=0x5571490c3cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:45.052 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.052+0000 7f847ea00640 1 -- 172.21.15.83:0/2897110519 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7f84740d0780 con 0x7f8488101550 2024-04-24T09:01:45.052 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.052+0000 7f848ffc1740 1 -- 172.21.15.83:0/1919055287 shutdown_connections 2024-04-24T09:01:45.053 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.052+0000 7f8486800640 1 --2- 172.21.15.83:0/2897110519 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f846c0a3d90 0x7f846c0a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:45.053 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.052+0000 7f848ffc1740 1 -- 172.21.15.83:0/1919055287 wait complete. 2024-04-24T09:01:45.053 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.052+0000 7f848ffc1740 1 -- start start 2024-04-24T09:01:45.053 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.053+0000 7f848ffc1740 1 Processor -- start 2024-04-24T09:01:45.053 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.053+0000 7f8486800640 1 --2- 172.21.15.83:0/2897110519 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f846c0a3d90 0x7f846c0a6250 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f8480009960 tx=0x7f848003d040 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:45.056 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.053+0000 7f848ffc1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5571490c73c0 0x5571490cf650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:45.056 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.053+0000 7f848ffc1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5571490c7ce0 0x5571490cfb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:45.056 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.053+0000 7f848ffc1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5571490d25a0 0x5571490d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:45.056 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.053+0000 7f848ffc1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5571490d5020 con 0x5571490c73c0 2024-04-24T09:01:45.056 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.053+0000 7f848ffc1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x557148fe72c0 con 0x5571490c7ce0 2024-04-24T09:01:45.057 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.053+0000 7f848ffc1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5571490d51a0 con 0x5571490d25a0 2024-04-24T09:01:45.057 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.054+0000 7f848ffc1740 10 client.0 _ll_register_callbacks cb 0x557148d3fa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:01:45.057 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.054+0000 7f8485e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5571490c73c0 0x5571490cf650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:45.057 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.054+0000 7f8485e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5571490c73c0 0x5571490cf650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48492/0 (socket says 172.21.15.83:48492) 2024-04-24T09:01:45.057 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.054+0000 7f8485e00640 1 -- 172.21.15.83:0/331742644 learned_addr learned my addr 172.21.15.83:0/331742644 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:01:45.057 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.054+0000 7f8485400640 1 --2- 172.21.15.83:0/331742644 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5571490c7ce0 0x5571490cfb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:45.057 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.054+0000 7f8485e00640 1 -- 172.21.15.83:0/331742644 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5571490d25a0 msgr2=0x5571490d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:45.057 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.054+0000 7f8486800640 1 --2- 172.21.15.83:0/331742644 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5571490d25a0 0x5571490d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:45.057 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.054+0000 7f8485e00640 1 --2- 172.21.15.83:0/331742644 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5571490d25a0 0x5571490d00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:45.057 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.054+0000 7f8485e00640 1 -- 172.21.15.83:0/331742644 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5571490c7ce0 msgr2=0x5571490cfb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:45.057 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.054+0000 7f8485e00640 1 --2- 172.21.15.83:0/331742644 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5571490c7ce0 0x5571490cfb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:45.057 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.054+0000 7f8485e00640 1 -- 172.21.15.83:0/331742644 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557148ff7fd0 con 0x5571490c73c0 2024-04-24T09:01:45.057 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.054+0000 7f8485400640 1 --2- 172.21.15.83:0/331742644 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5571490c7ce0 0x5571490cfb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:01:45.058 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.054+0000 7f8486800640 1 --2- 172.21.15.83:0/331742644 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5571490d25a0 0x5571490d00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:01:45.058 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.055+0000 7f8485e00640 1 --2- 172.21.15.83:0/331742644 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5571490c73c0 0x5571490cf650 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f847800b9f0 tx=0x7f847800beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ded61dc1fb3862f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:45.058 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.055+0000 7f8467e00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:01:45.058 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.055+0000 7f8467e00640 1 -- 172.21.15.83:0/331742644 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f847801b440 con 0x5571490c73c0 2024-04-24T09:01:45.058 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.055+0000 7f8467e00640 1 -- 172.21.15.83:0/331742644 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7f847801b5e0 con 0x5571490c73c0 2024-04-24T09:01:45.058 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.055+0000 7f848ffc1740 1 -- 172.21.15.83:0/331742644 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8480036070 con 0x5571490c73c0 2024-04-24T09:01:45.058 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.055+0000 7f8467e00640 1 -- 172.21.15.83:0/331742644 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f8478024050 con 0x5571490c73c0 2024-04-24T09:01:45.058 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.055+0000 7f848ffc1740 10 client.19200 fetch_fsmap learned FSMap version 37 2024-04-24T09:01:45.058 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.055+0000 7f848ffc1740 1 -- 172.21.15.83:0/331742644 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x557148d40210 con 0x5571490c73c0 2024-04-24T09:01:45.059 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.056+0000 7f8467e00640 1 -- 172.21.15.83:0/331742644 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8480036070 con 0x5571490c73c0 2024-04-24T09:01:45.059 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.056+0000 7f8467e00640 1 -- 172.21.15.83:0/331742644 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f8478019020 con 0x5571490c73c0 2024-04-24T09:01:45.059 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.056+0000 7f848ffc1740 10 client.19200 fetch_fsmap finished waiting for FSMap version 37 2024-04-24T09:01:45.059 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.056+0000 7f848ffc1740 10 client.19200 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:01:45.059 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.056+0000 7f848ffc1740 20 client.19200 populate_metadata read hostname 'smithi083' 2024-04-24T09:01:45.059 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.056+0000 7f848ffc1740 1 --2- 172.21.15.83:0/331742644 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5571490e27e0 0x5571490e4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:45.059 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.056+0000 7f848ffc1740 4 client.19200 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:01:45.059 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.056+0000 7f848ffc1740 1 -- 172.21.15.83:0/331742644 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55714903b320 con 0x5571490e27e0 2024-04-24T09:01:45.059 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.057+0000 7f8485400640 1 --2- 172.21.15.83:0/331742644 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5571490e27e0 0x5571490e4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:45.060 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.058+0000 7f8485400640 1 --2- 172.21.15.83:0/331742644 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5571490e27e0 0x5571490e4bd0 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e354bb4512562c44 server_cookie=ea12ff11e207b371 in_seq=0 out_seq=0 2024-04-24T09:01:45.060 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.058+0000 7f8467e00640 10 client.19200 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:01:45.061 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.060+0000 7f8467e00640 1 -- 172.21.15.83:0/331742644 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55714903b320 con 0x5571490e27e0 2024-04-24T09:01:45.061 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.060+0000 7f8467e00640 10 client.19200 handle_command_reply: tid=0 2024-04-24T09:01:45.061 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.060+0000 7f848ffc1740 1 client.19200 shutdown 2024-04-24T09:01:45.061 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.060+0000 7f848ffc1740 1 -- 172.21.15.83:0/331742644 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5571490c73c0 msgr2=0x5571490cf650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:45.061 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.060+0000 7f848ffc1740 1 --2- 172.21.15.83:0/331742644 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5571490c73c0 0x5571490cf650 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f847800b9f0 tx=0x7f847800beb0 comp rx=0 tx=0).stop 2024-04-24T09:01:45.061 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.060+0000 7f848ffc1740 1 -- 172.21.15.83:0/331742644 shutdown_connections 2024-04-24T09:01:45.061 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.060+0000 7f848ffc1740 1 --2- 172.21.15.83:0/331742644 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5571490e27e0 0x5571490e4bd0 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:45.061 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.061+0000 7f848ffc1740 1 --2- 172.21.15.83:0/331742644 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5571490d25a0 0x5571490d00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:45.061 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.061+0000 7f848ffc1740 1 --2- 172.21.15.83:0/331742644 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5571490c7ce0 0x5571490cfb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:45.061 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.061+0000 7f848ffc1740 1 --2- 172.21.15.83:0/331742644 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5571490c73c0 0x5571490cf650 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:45.061 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.061+0000 7f8467e00640 0 client.19200 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:01:45.061 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.061+0000 7f848ffc1740 1 -- 172.21.15.83:0/331742644 >> 172.21.15.83:0/331742644 conn(0x5571490c38d0 msgr2=0x5571490c9600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:45.061 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.061+0000 7f848ffc1740 1 -- 172.21.15.83:0/331742644 shutdown_connections 2024-04-24T09:01:45.061 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.061+0000 7f848ffc1740 1 -- 172.21.15.83:0/331742644 wait complete. 2024-04-24T09:01:45.062 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.061+0000 7f848ffc1740 20 client.19200 trim_cache size 0 max 16384 2024-04-24T09:01:45.078 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.078+0000 7f848ffc1740 1 Processor -- start 2024-04-24T09:01:45.078 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.078+0000 7f848ffc1740 1 -- start start 2024-04-24T09:01:45.078 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.078+0000 7f848ffc1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5571490c73c0 0x5571490d1d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:45.078 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.078+0000 7f848ffc1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5571490c7ce0 0x5571490c3210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:45.078 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.078+0000 7f848ffc1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5571490d25a0 0x5571490c9d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:45.078 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.078+0000 7f848ffc1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f84881b1670 con 0x5571490c73c0 2024-04-24T09:01:45.078 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.078+0000 7f848ffc1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f84881b3210 con 0x5571490c7ce0 2024-04-24T09:01:45.078 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.078+0000 7f848ffc1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f84881b1d40 con 0x5571490d25a0 2024-04-24T09:01:45.078 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.078+0000 7f8485e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5571490c73c0 0x5571490d1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:45.079 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.078+0000 7f8485e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5571490c73c0 0x5571490d1d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48494/0 (socket says 172.21.15.83:48494) 2024-04-24T09:01:45.079 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.078+0000 7f8485e00640 1 -- 172.21.15.83:0/3494869855 learned_addr learned my addr 172.21.15.83:0/3494869855 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:01:45.079 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.079+0000 7f8485e00640 1 -- 172.21.15.83:0/3494869855 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5571490d25a0 msgr2=0x5571490c9d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:45.079 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.079+0000 7f8485e00640 1 --2- 172.21.15.83:0/3494869855 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5571490d25a0 0x5571490c9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:45.079 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.079+0000 7f8485e00640 1 -- 172.21.15.83:0/3494869855 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5571490c7ce0 msgr2=0x5571490c3210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:45.079 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.079+0000 7f8485e00640 1 --2- 172.21.15.83:0/3494869855 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5571490c7ce0 0x5571490c3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:45.079 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.079+0000 7f8485e00640 1 -- 172.21.15.83:0/3494869855 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557148e0c620 con 0x5571490c73c0 2024-04-24T09:01:45.079 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.079+0000 7f8485e00640 1 --2- 172.21.15.83:0/3494869855 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5571490c73c0 0x5571490d1d80 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f84780056f0 tx=0x7f8478005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54c3fd084b23db81 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:45.083 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.079+0000 7f8472a00640 1 -- 172.21.15.83:0/3494869855 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f847802e020 con 0x5571490c73c0 2024-04-24T09:01:45.083 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.079+0000 7f8472a00640 1 -- 172.21.15.83:0/3494869855 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8478005850 con 0x5571490c73c0 2024-04-24T09:01:45.083 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.079+0000 7f8472a00640 1 -- 172.21.15.83:0/3494869855 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8478010a60 con 0x5571490c73c0 2024-04-24T09:01:45.083 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.080+0000 7f848ffc1740 1 -- 172.21.15.83:0/3494869855 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5571490c73c0 msgr2=0x5571490d1d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:45.083 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.080+0000 7f848ffc1740 1 --2- 172.21.15.83:0/3494869855 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5571490c73c0 0x5571490d1d80 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f84780056f0 tx=0x7f8478005720 comp rx=0 tx=0).stop 2024-04-24T09:01:45.083 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.080+0000 7f848ffc1740 1 -- 172.21.15.83:0/3494869855 shutdown_connections 2024-04-24T09:01:45.083 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.080+0000 7f848ffc1740 1 --2- 172.21.15.83:0/3494869855 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5571490d25a0 0x5571490c9d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:45.083 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.080+0000 7f848ffc1740 1 --2- 172.21.15.83:0/3494869855 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5571490c7ce0 0x5571490c3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:45.083 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.080+0000 7f848ffc1740 1 --2- 172.21.15.83:0/3494869855 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5571490c73c0 0x5571490d1d80 unknown :-1 s=CLOSED pgs=1681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:45.083 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.080+0000 7f848ffc1740 1 -- 172.21.15.83:0/3494869855 >> 172.21.15.83:0/3494869855 conn(0x5571490c38d0 msgr2=0x5571490cf610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:45.083 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.080+0000 7f848ffc1740 1 -- 172.21.15.83:0/3494869855 shutdown_connections 2024-04-24T09:01:45.083 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.080+0000 7f848ffc1740 1 -- 172.21.15.83:0/3494869855 wait complete. 2024-04-24T09:01:45.084 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.081+0000 7f848ffc1740 1 -- start start 2024-04-24T09:01:45.084 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.081+0000 7f848ffc1740 1 Processor -- start 2024-04-24T09:01:45.084 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.081+0000 7f848ffc1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5571490c73c0 0x5571490cc3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:45.084 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.081+0000 7f848ffc1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5571490c7ce0 0x5571490cc930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:45.084 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.081+0000 7f848ffc1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5571490d25a0 0x5571490cce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:45.084 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.081+0000 7f848ffc1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f84881b2b10 con 0x5571490c73c0 2024-04-24T09:01:45.084 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.081+0000 7f848ffc1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f84881b2440 con 0x5571490c7ce0 2024-04-24T09:01:45.084 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.081+0000 7f848ffc1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f84881af670 con 0x5571490d25a0 2024-04-24T09:01:45.084 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.081+0000 7f848ffc1740 10 client.0 _ll_register_callbacks cb 0x7f8478004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:01:45.084 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.082+0000 7f8485e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5571490c73c0 0x5571490cc3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:45.084 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.082+0000 7f8485e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5571490c73c0 0x5571490cc3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48510/0 (socket says 172.21.15.83:48510) 2024-04-24T09:01:45.084 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.082+0000 7f8485e00640 1 -- 172.21.15.83:0/656643416 learned_addr learned my addr 172.21.15.83:0/656643416 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:01:45.084 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.082+0000 7f8486800640 1 --2- 172.21.15.83:0/656643416 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5571490d25a0 0x5571490cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:45.084 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.082+0000 7f8485400640 1 --2- 172.21.15.83:0/656643416 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5571490c7ce0 0x5571490cc930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:45.085 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.082+0000 7f8485e00640 1 -- 172.21.15.83:0/656643416 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5571490d25a0 msgr2=0x5571490cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:45.085 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.082+0000 7f8485e00640 1 --2- 172.21.15.83:0/656643416 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5571490d25a0 0x5571490cce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:45.085 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.082+0000 7f8485e00640 1 -- 172.21.15.83:0/656643416 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5571490c7ce0 msgr2=0x5571490cc930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:45.085 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.082+0000 7f8485e00640 1 --2- 172.21.15.83:0/656643416 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5571490c7ce0 0x5571490cc930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:45.085 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.082+0000 7f8485e00640 1 -- 172.21.15.83:0/656643416 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557149147e20 con 0x5571490c73c0 2024-04-24T09:01:45.085 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.082+0000 7f8485e00640 1 --2- 172.21.15.83:0/656643416 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5571490c73c0 0x5571490cc3f0 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f8478011cc0 tx=0x7f8478004370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5670aa04a393971 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:45.085 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.082+0000 7f8467e00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:01:45.085 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.083+0000 7f8467e00640 1 -- 172.21.15.83:0/656643416 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8478011440 con 0x5571490c73c0 2024-04-24T09:01:45.085 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.083+0000 7f8467e00640 1 -- 172.21.15.83:0/656643416 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7f84780115e0 con 0x5571490c73c0 2024-04-24T09:01:45.085 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.083+0000 7f848ffc1740 1 -- 172.21.15.83:0/656643416 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8478005850 con 0x5571490c73c0 2024-04-24T09:01:45.085 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.083+0000 7f8467e00640 1 -- 172.21.15.83:0/656643416 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f8478043050 con 0x5571490c73c0 2024-04-24T09:01:45.085 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.083+0000 7f848ffc1740 10 client.19206 fetch_fsmap learned FSMap version 37 2024-04-24T09:01:45.085 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.083+0000 7f848ffc1740 1 -- 172.21.15.83:0/656643416 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x557149149130 con 0x5571490c73c0 2024-04-24T09:01:45.085 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.083+0000 7f8467e00640 1 -- 172.21.15.83:0/656643416 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8478005850 con 0x5571490c73c0 2024-04-24T09:01:45.086 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.083+0000 7f8467e00640 1 -- 172.21.15.83:0/656643416 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f8478019050 con 0x5571490c73c0 2024-04-24T09:01:45.086 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.084+0000 7f848ffc1740 10 client.19206 fetch_fsmap finished waiting for FSMap version 37 2024-04-24T09:01:45.086 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.084+0000 7f848ffc1740 10 client.19206 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:01:45.086 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.084+0000 7f848ffc1740 20 client.19206 populate_metadata read hostname 'smithi083' 2024-04-24T09:01:45.086 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.084+0000 7f848ffc1740 1 --2- 172.21.15.83:0/656643416 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5571491497b0 0x55714914bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:45.086 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.084+0000 7f848ffc1740 4 client.19206 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T09:01:45.086 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.084+0000 7f848ffc1740 1 -- 172.21.15.83:0/656643416 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557149035190 con 0x5571491497b0 2024-04-24T09:01:45.087 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.084+0000 7f8485400640 1 --2- 172.21.15.83:0/656643416 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5571491497b0 0x55714914bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:45.087 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.085+0000 7f8485400640 1 --2- 172.21.15.83:0/656643416 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5571491497b0 0x55714914bbd0 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1eb012bdf87d060d server_cookie=8033d8ca96ee9915 in_seq=0 out_seq=0 2024-04-24T09:01:45.087 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.085+0000 7f8467e00640 10 client.19206 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:01:45.143 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.142+0000 7f8467e00640 1 -- 172.21.15.83:0/656643416 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557149035190 con 0x5571491497b0 2024-04-24T09:01:45.143 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.142+0000 7f8467e00640 10 client.19206 handle_command_reply: tid=0 2024-04-24T09:01:45.143 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.142+0000 7f848ffc1740 1 client.19206 shutdown 2024-04-24T09:01:45.143 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.143+0000 7f848ffc1740 1 -- 172.21.15.83:0/656643416 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5571490c73c0 msgr2=0x5571490cc3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:45.143 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.143+0000 7f848ffc1740 1 --2- 172.21.15.83:0/656643416 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5571490c73c0 0x5571490cc3f0 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f8478011cc0 tx=0x7f8478004370 comp rx=0 tx=0).stop 2024-04-24T09:01:45.143 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.143+0000 7f848ffc1740 1 -- 172.21.15.83:0/656643416 shutdown_connections 2024-04-24T09:01:45.143 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.143+0000 7f848ffc1740 1 --2- 172.21.15.83:0/656643416 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5571491497b0 0x55714914bbd0 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:45.143 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.143+0000 7f848ffc1740 1 --2- 172.21.15.83:0/656643416 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5571490d25a0 0x5571490cce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:45.143 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.143+0000 7f8467e00640 0 client.19206 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:01:45.143 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.143+0000 7f848ffc1740 1 --2- 172.21.15.83:0/656643416 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5571490c7ce0 0x5571490cc930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:45.144 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.143+0000 7f848ffc1740 1 --2- 172.21.15.83:0/656643416 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5571490c73c0 0x5571490cc3f0 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:45.144 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.143+0000 7f848ffc1740 1 -- 172.21.15.83:0/656643416 >> 172.21.15.83:0/656643416 conn(0x5571490c38d0 msgr2=0x5571490cfe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:45.144 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.143+0000 7f848ffc1740 1 -- 172.21.15.83:0/656643416 shutdown_connections 2024-04-24T09:01:45.144 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.144+0000 7f848ffc1740 1 -- 172.21.15.83:0/656643416 wait complete. 2024-04-24T09:01:45.144 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.144+0000 7f848ffc1740 20 client.19206 trim_cache size 0 max 16384 2024-04-24T09:01:45.144 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T09:01:45.144 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T09:01:45.144 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T09:01:45.145 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T09:01:45.145 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.145+0000 7f8487200640 1 -- 172.21.15.83:0/2897110519 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f846c0a3d90 msgr2=0x7f846c0a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:45.145 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.145+0000 7f8487200640 1 --2- 172.21.15.83:0/2897110519 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f846c0a3d90 0x7f846c0a6250 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f8480009960 tx=0x7f848003d040 comp rx=0 tx=0).stop 2024-04-24T09:01:45.145 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.145+0000 7f8487200640 1 -- 172.21.15.83:0/2897110519 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8488101550 msgr2=0x7f84881a0960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:45.145 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.145+0000 7f8487200640 1 --2- 172.21.15.83:0/2897110519 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8488101550 0x7f84881a0960 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f847400d950 tx=0x7f847400de10 comp rx=0 tx=0).stop 2024-04-24T09:01:45.146 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.146+0000 7f8487200640 1 -- 172.21.15.83:0/2897110519 shutdown_connections 2024-04-24T09:01:45.146 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.146+0000 7f8487200640 1 --2- 172.21.15.83:0/2897110519 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f846c0a3d90 0x7f846c0a6250 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:45.146 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.146+0000 7f8487200640 1 --2- 172.21.15.83:0/2897110519 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f848810e750 0x7f848819a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:45.146 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.146+0000 7f8487200640 1 --2- 172.21.15.83:0/2897110519 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8488101550 0x7f84881a0960 unknown :-1 s=CLOSED pgs=1678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:45.146 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.146+0000 7f8487200640 1 --2- 172.21.15.83:0/2897110519 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8488100ba0 0x7f84881a0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:45.146 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.146+0000 7f8487200640 1 -- 172.21.15.83:0/2897110519 >> 172.21.15.83:0/2897110519 conn(0x7f84880fc820 msgr2=0x7f84880fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:45.146 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.146+0000 7f8487200640 1 -- 172.21.15.83:0/2897110519 shutdown_connections 2024-04-24T09:01:45.146 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:45.146+0000 7f8487200640 1 -- 172.21.15.83:0/2897110519 wait complete. 2024-04-24T09:01:45.608 INFO:tasks.workunit.client.0.smithi083.stdout: 1 58873 0.62 MB/sec execute 379 sec latency 64.167 ms 2024-04-24T09:01:46.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:01:46 smithi138 ceph-mon[31441]: pgmap v317: 129 pgs: 129 active+clean; 59 MiB data, 830 MiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 283 KiB/s wr, 49 op/s 2024-04-24T09:01:46.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:01:46 smithi189 ceph-mon[31791]: pgmap v317: 129 pgs: 129 active+clean; 59 MiB data, 830 MiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 283 KiB/s wr, 49 op/s 2024-04-24T09:01:46.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:01:46 smithi083 ceph-mon[27907]: pgmap v317: 129 pgs: 129 active+clean; 59 MiB data, 830 MiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 283 KiB/s wr, 49 op/s 2024-04-24T09:01:46.608 INFO:tasks.workunit.client.0.smithi083.stdout: 1 59003 0.62 MB/sec execute 380 sec latency 69.042 ms 2024-04-24T09:01:47.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:01:47 smithi138 ceph-mon[31441]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:01:47.455 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:01:47 smithi083 ceph-mon[27907]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:01:47.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:01:47 smithi189 ceph-mon[31791]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:01:47.609 INFO:tasks.workunit.client.0.smithi083.stdout: 1 59142 0.62 MB/sec execute 381 sec latency 52.114 ms 2024-04-24T09:01:48.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:01:48 smithi138 ceph-mon[31441]: pgmap v318: 129 pgs: 129 active+clean; 59 MiB data, 830 MiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 283 KiB/s wr, 49 op/s 2024-04-24T09:01:48.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:01:48 smithi138 ceph-mon[31441]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:01:48.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:01:48 smithi189 ceph-mon[31791]: pgmap v318: 129 pgs: 129 active+clean; 59 MiB data, 830 MiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 283 KiB/s wr, 49 op/s 2024-04-24T09:01:48.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:01:48 smithi189 ceph-mon[31791]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:01:48.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:01:48 smithi083 ceph-mon[27907]: pgmap v318: 129 pgs: 129 active+clean; 59 MiB data, 830 MiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 283 KiB/s wr, 49 op/s 2024-04-24T09:01:48.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:01:48 smithi083 ceph-mon[27907]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:01:48.609 INFO:tasks.workunit.client.0.smithi083.stdout: 1 59216 0.62 MB/sec execute 382 sec latency 49.991 ms 2024-04-24T09:01:49.609 INFO:tasks.workunit.client.0.smithi083.stdout: 1 59338 0.62 MB/sec execute 383 sec latency 48.755 ms 2024-04-24T09:01:50.449 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T09:01:50.449 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T09:01:50.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:01:50 smithi138 ceph-mon[31441]: pgmap v319: 129 pgs: 129 active+clean; 56 MiB data, 820 MiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 431 KiB/s wr, 70 op/s 2024-04-24T09:01:50.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:01:50 smithi189 ceph-mon[31791]: pgmap v319: 129 pgs: 129 active+clean; 56 MiB data, 820 MiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 431 KiB/s wr, 70 op/s 2024-04-24T09:01:50.535 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:01:50 smithi083 ceph-mon[27907]: pgmap v319: 129 pgs: 129 active+clean; 56 MiB data, 820 MiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 431 KiB/s wr, 70 op/s 2024-04-24T09:01:50.609 INFO:tasks.workunit.client.0.smithi083.stdout: 1 59525 0.63 MB/sec execute 384 sec latency 53.132 ms 2024-04-24T09:01:50.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.677+0000 7f7e3e000640 1 -- 172.21.15.83:0/4075764517 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e381071e0 msgr2=0x7f7e3810edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:50.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.677+0000 7f7e3e000640 1 --2- 172.21.15.83:0/4075764517 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e381071e0 0x7f7e3810edb0 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f7e30009ca0 tx=0x7f7e3002a8c0 comp rx=0 tx=0).stop 2024-04-24T09:01:50.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.677+0000 7f7e3e000640 1 -- 172.21.15.83:0/4075764517 shutdown_connections 2024-04-24T09:01:50.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.677+0000 7f7e3e000640 1 --2- 172.21.15.83:0/4075764517 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e381071e0 0x7f7e3810edb0 unknown :-1 s=CLOSED pgs=1683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:50.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.677+0000 7f7e3e000640 1 --2- 172.21.15.83:0/4075764517 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e38102ac0 0x7f7e38106b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:50.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.677+0000 7f7e3e000640 1 --2- 172.21.15.83:0/4075764517 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7e38102110 0x7f7e381024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:50.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.677+0000 7f7e3e000640 1 -- 172.21.15.83:0/4075764517 >> 172.21.15.83:0/4075764517 conn(0x7f7e380fd9d0 msgr2=0x7f7e380ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:50.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.677+0000 7f7e3e000640 1 -- 172.21.15.83:0/4075764517 shutdown_connections 2024-04-24T09:01:50.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.677+0000 7f7e3e000640 1 -- 172.21.15.83:0/4075764517 wait complete. 2024-04-24T09:01:50.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.678+0000 7f7e3e000640 1 Processor -- start 2024-04-24T09:01:50.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.678+0000 7f7e3e000640 1 -- start start 2024-04-24T09:01:50.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.679+0000 7f7e3e000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e38102110 0x7f7e381a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:50.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.679+0000 7f7e3e000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7e381071e0 0x7f7e381a0b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:50.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.679+0000 7f7e3e000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e3819a700 0x7f7e3819ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:50.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.679+0000 7f7e3e000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7e381121a0 con 0x7f7e3819a700 2024-04-24T09:01:50.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.679+0000 7f7e3e000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7e38112020 con 0x7f7e38102110 2024-04-24T09:01:50.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.679+0000 7f7e3e000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7e38112320 con 0x7f7e381071e0 2024-04-24T09:01:50.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.679+0000 7f7e3d600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e3819a700 0x7f7e3819ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:50.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.679+0000 7f7e3d600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e3819a700 0x7f7e3819ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45422/0 (socket says 172.21.15.83:45422) 2024-04-24T09:01:50.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.679+0000 7f7e3d600640 1 -- 172.21.15.83:0/1214945557 learned_addr learned my addr 172.21.15.83:0/1214945557 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:01:50.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.679+0000 7f7e3cc00640 1 --2- 172.21.15.83:0/1214945557 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e38102110 0x7f7e381a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:50.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.679+0000 7f7e37e00640 1 --2- 172.21.15.83:0/1214945557 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7e381071e0 0x7f7e381a0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:50.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.680+0000 7f7e3d600640 1 -- 172.21.15.83:0/1214945557 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7e381071e0 msgr2=0x7f7e381a0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:50.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.680+0000 7f7e3d600640 1 --2- 172.21.15.83:0/1214945557 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7e381071e0 0x7f7e381a0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:50.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.680+0000 7f7e3d600640 1 -- 172.21.15.83:0/1214945557 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e38102110 msgr2=0x7f7e381a05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:50.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.680+0000 7f7e3d600640 1 --2- 172.21.15.83:0/1214945557 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e38102110 0x7f7e381a05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:50.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.680+0000 7f7e3d600640 1 -- 172.21.15.83:0/1214945557 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e3819b300 con 0x7f7e3819a700 2024-04-24T09:01:50.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.680+0000 7f7e3cc00640 1 --2- 172.21.15.83:0/1214945557 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e38102110 0x7f7e381a05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:01:50.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.680+0000 7f7e3d600640 1 --2- 172.21.15.83:0/1214945557 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e3819a700 0x7f7e3819ab80 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f7e30031040 tx=0x7f7e30003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:50.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.681+0000 7f7e35600640 1 -- 172.21.15.83:0/1214945557 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e30003ce0 con 0x7f7e3819a700 2024-04-24T09:01:50.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.681+0000 7f7e35600640 1 -- 172.21.15.83:0/1214945557 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7e300044f0 con 0x7f7e3819a700 2024-04-24T09:01:50.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.681+0000 7f7e3e000640 1 -- 172.21.15.83:0/1214945557 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e3819b590 con 0x7f7e3819a700 2024-04-24T09:01:50.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.681+0000 7f7e35600640 1 -- 172.21.15.83:0/1214945557 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e30030c90 con 0x7f7e3819a700 2024-04-24T09:01:50.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.681+0000 7f7e3e000640 1 -- 172.21.15.83:0/1214945557 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e381a7730 con 0x7f7e3819a700 2024-04-24T09:01:50.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.685+0000 7f7e42367740 1 Processor -- start 2024-04-24T09:01:50.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.686+0000 7f7e35600640 1 -- 172.21.15.83:0/1214945557 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7e300046a0 con 0x7f7e3819a700 2024-04-24T09:01:50.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.686+0000 7f7e42367740 1 -- start start 2024-04-24T09:01:50.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.686+0000 7f7e42367740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d0b1d40a30 0x55d0b1d40e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:50.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.686+0000 7f7e42367740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d0b1d41350 0x55d0b1d4b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:50.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.686+0000 7f7e42367740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0b1d4ba20 0x55d0b1d4de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:50.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.686+0000 7f7e42367740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d0b1ca61f0 con 0x55d0b1d4ba20 2024-04-24T09:01:50.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.686+0000 7f7e42367740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d0b1c4c350 con 0x55d0b1d40a30 2024-04-24T09:01:50.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.686+0000 7f7e42367740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d0b1ca21b0 con 0x55d0b1d41350 2024-04-24T09:01:50.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.687+0000 7f7e37e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0b1d4ba20 0x55d0b1d4de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:50.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.687+0000 7f7e37e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0b1d4ba20 0x55d0b1d4de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45428/0 (socket says 172.21.15.83:45428) 2024-04-24T09:01:50.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.687+0000 7f7e37e00640 1 -- 172.21.15.83:0/3232350770 learned_addr learned my addr 172.21.15.83:0/3232350770 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:01:50.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.687+0000 7f7e3cc00640 1 --2- 172.21.15.83:0/3232350770 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d0b1d41350 0x55d0b1d4b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:50.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.687+0000 7f7e37e00640 1 --2- 172.21.15.83:0/3232350770 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d0b1d40a30 0x55d0b1d40e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:50.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.687+0000 7f7e37e00640 1 -- 172.21.15.83:0/3232350770 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d0b1d41350 msgr2=0x55d0b1d4b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:50.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.687+0000 7f7e37e00640 1 --2- 172.21.15.83:0/3232350770 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d0b1d41350 0x55d0b1d4b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:50.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.687+0000 7f7e37e00640 1 -- 172.21.15.83:0/3232350770 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d0b1d40a30 msgr2=0x55d0b1d40e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:50.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.687+0000 7f7e37e00640 1 --2- 172.21.15.83:0/3232350770 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d0b1d40a30 0x55d0b1d40e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:50.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.687+0000 7f7e37e00640 1 -- 172.21.15.83:0/3232350770 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0b1ca0f50 con 0x55d0b1d4ba20 2024-04-24T09:01:50.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.687+0000 7f7e37e00640 1 --2- 172.21.15.83:0/3232350770 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0b1d4ba20 0x55d0b1d4de10 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c00cf60 tx=0x7f7e2c00c610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78a157e9eea1eb9f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:50.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.688+0000 7f7e29c00640 1 -- 172.21.15.83:0/3232350770 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e2c019030 con 0x55d0b1d4ba20 2024-04-24T09:01:50.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.688+0000 7f7e29c00640 1 -- 172.21.15.83:0/3232350770 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7e2c0119d0 con 0x55d0b1d4ba20 2024-04-24T09:01:50.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.688+0000 7f7e29c00640 1 -- 172.21.15.83:0/3232350770 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e2c00f8a0 con 0x55d0b1d4ba20 2024-04-24T09:01:50.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.688+0000 7f7e35600640 1 --2- 172.21.15.83:0/1214945557 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f7e200a3da0 0x7f7e200a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:50.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.688+0000 7f7e42367740 1 -- 172.21.15.83:0/3232350770 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0b1d4ba20 msgr2=0x55d0b1d4de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:50.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.688+0000 7f7e42367740 1 --2- 172.21.15.83:0/3232350770 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0b1d4ba20 0x55d0b1d4de10 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c00cf60 tx=0x7f7e2c00c610 comp rx=0 tx=0).stop 2024-04-24T09:01:50.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.688+0000 7f7e35600640 1 -- 172.21.15.83:0/1214945557 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7f7e300f21e0 con 0x7f7e3819a700 2024-04-24T09:01:50.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.688+0000 7f7e3cc00640 1 --2- 172.21.15.83:0/1214945557 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f7e200a3da0 0x7f7e200a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:50.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.689+0000 7f7e42367740 1 -- 172.21.15.83:0/3232350770 shutdown_connections 2024-04-24T09:01:50.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.689+0000 7f7e42367740 1 --2- 172.21.15.83:0/3232350770 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0b1d4ba20 0x55d0b1d4de10 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:50.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.689+0000 7f7e42367740 1 --2- 172.21.15.83:0/3232350770 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d0b1d41350 0x55d0b1d4b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:50.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.689+0000 7f7e42367740 1 --2- 172.21.15.83:0/3232350770 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d0b1d40a30 0x55d0b1d40e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:50.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.689+0000 7f7e42367740 1 -- 172.21.15.83:0/3232350770 >> 172.21.15.83:0/3232350770 conn(0x55d0b1d3cd50 msgr2=0x55d0b1d3d130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:50.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.689+0000 7f7e42367740 1 -- 172.21.15.83:0/3232350770 shutdown_connections 2024-04-24T09:01:50.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.689+0000 7f7e42367740 1 -- 172.21.15.83:0/3232350770 wait complete. 2024-04-24T09:01:50.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.689+0000 7f7e42367740 1 -- start start 2024-04-24T09:01:50.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.689+0000 7f7e3cc00640 1 --2- 172.21.15.83:0/1214945557 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f7e200a3da0 0x7f7e200a6260 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x55d0b1d41ff0 tx=0x7f7e2400b7b0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:50.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.690+0000 7f7e42367740 1 Processor -- start 2024-04-24T09:01:50.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.690+0000 7f7e42367740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0b1d40a30 0x55d0b1d48990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:50.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.690+0000 7f7e42367740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d0b1d41350 0x55d0b1d48ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:50.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.690+0000 7f7e42367740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d0b1d4ba20 0x55d0b1d49410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:50.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.690+0000 7f7e42367740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d0b1d4e810 con 0x55d0b1d40a30 2024-04-24T09:01:50.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.690+0000 7f7e42367740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d0b1c5f690 con 0x55d0b1d41350 2024-04-24T09:01:50.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.690+0000 7f7e42367740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d0b1d4e990 con 0x55d0b1d4ba20 2024-04-24T09:01:50.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.690+0000 7f7e42367740 10 client.0 _ll_register_callbacks cb 0x55d0b1c9b650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:01:50.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.690+0000 7f7e3cc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0b1d40a30 0x55d0b1d48990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:50.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.690+0000 7f7e3cc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0b1d40a30 0x55d0b1d48990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45444/0 (socket says 172.21.15.83:45444) 2024-04-24T09:01:50.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.690+0000 7f7e3cc00640 1 -- 172.21.15.83:0/3177377741 learned_addr learned my addr 172.21.15.83:0/3177377741 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:01:50.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.691+0000 7f7e37e00640 1 --2- 172.21.15.83:0/3177377741 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d0b1d41350 0x55d0b1d48ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:50.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.691+0000 7f7e3d600640 1 --2- 172.21.15.83:0/3177377741 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d0b1d4ba20 0x55d0b1d49410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:50.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.691+0000 7f7e3cc00640 1 -- 172.21.15.83:0/3177377741 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d0b1d4ba20 msgr2=0x55d0b1d49410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:50.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.691+0000 7f7e3cc00640 1 --2- 172.21.15.83:0/3177377741 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d0b1d4ba20 0x55d0b1d49410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:50.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.691+0000 7f7e3cc00640 1 -- 172.21.15.83:0/3177377741 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d0b1d41350 msgr2=0x55d0b1d48ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:50.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.691+0000 7f7e3cc00640 1 --2- 172.21.15.83:0/3177377741 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d0b1d41350 0x55d0b1d48ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:50.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.691+0000 7f7e3cc00640 1 -- 172.21.15.83:0/3177377741 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0b1c5e920 con 0x55d0b1d40a30 2024-04-24T09:01:50.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.691+0000 7f7e3d600640 1 --2- 172.21.15.83:0/3177377741 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d0b1d4ba20 0x55d0b1d49410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:01:50.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.691+0000 7f7e3cc00640 1 --2- 172.21.15.83:0/3177377741 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0b1d40a30 0x55d0b1d48990 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f7e240126b0 tx=0x7f7e240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85b8968d1c680077 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:50.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.691+0000 7f7e1e000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:01:50.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.692+0000 7f7e1e000640 1 -- 172.21.15.83:0/3177377741 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e2400f9a0 con 0x55d0b1d40a30 2024-04-24T09:01:50.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.692+0000 7f7e42367740 1 -- 172.21.15.83:0/3177377741 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e2c0119d0 con 0x55d0b1d40a30 2024-04-24T09:01:50.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.692+0000 7f7e1e000640 1 -- 172.21.15.83:0/3177377741 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7f7e2400fb40 con 0x55d0b1d40a30 2024-04-24T09:01:50.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.692+0000 7f7e1e000640 1 -- 172.21.15.83:0/3177377741 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f7e2400fd40 con 0x55d0b1d40a30 2024-04-24T09:01:50.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.692+0000 7f7e42367740 10 client.19218 fetch_fsmap learned FSMap version 39 2024-04-24T09:01:50.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.692+0000 7f7e42367740 1 -- 172.21.15.83:0/3177377741 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55d0b1a827d0 con 0x55d0b1d40a30 2024-04-24T09:01:50.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.692+0000 7f7e1e000640 1 -- 172.21.15.83:0/3177377741 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7e2c0119d0 con 0x55d0b1d40a30 2024-04-24T09:01:50.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.692+0000 7f7e1e000640 1 -- 172.21.15.83:0/3177377741 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f7e24027490 con 0x55d0b1d40a30 2024-04-24T09:01:50.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.692+0000 7f7e42367740 10 client.19218 fetch_fsmap finished waiting for FSMap version 39 2024-04-24T09:01:50.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.692+0000 7f7e42367740 10 client.19218 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:01:50.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.692+0000 7f7e42367740 20 client.19218 populate_metadata read hostname 'smithi083' 2024-04-24T09:01:50.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.692+0000 7f7e42367740 1 --2- 172.21.15.83:0/3177377741 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d0b1d5be50 0x55d0b1d5e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:50.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.692+0000 7f7e42367740 4 client.19218 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:01:50.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.692+0000 7f7e42367740 1 -- 172.21.15.83:0/3177377741 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d0b1cb3c10 con 0x55d0b1d5be50 2024-04-24T09:01:50.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.693+0000 7f7e37e00640 1 --2- 172.21.15.83:0/3177377741 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d0b1d5be50 0x55d0b1d5e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:50.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.694+0000 7f7e37e00640 1 --2- 172.21.15.83:0/3177377741 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d0b1d5be50 0x55d0b1d5e2e0 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e51c7005c7a9176 server_cookie=9dbe08735743f93c in_seq=0 out_seq=0 2024-04-24T09:01:50.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.694+0000 7f7e1e000640 10 client.19218 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:01:50.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.697+0000 7f7e1e000640 1 -- 172.21.15.83:0/3177377741 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d0b1cb3c10 con 0x55d0b1d5be50 2024-04-24T09:01:50.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.697+0000 7f7e1e000640 10 client.19218 handle_command_reply: tid=0 2024-04-24T09:01:50.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.697+0000 7f7e42367740 1 client.19218 shutdown 2024-04-24T09:01:50.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.697+0000 7f7e42367740 1 -- 172.21.15.83:0/3177377741 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0b1d40a30 msgr2=0x55d0b1d48990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:50.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.697+0000 7f7e42367740 1 --2- 172.21.15.83:0/3177377741 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0b1d40a30 0x55d0b1d48990 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f7e240126b0 tx=0x7f7e240046c0 comp rx=0 tx=0).stop 2024-04-24T09:01:50.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.697+0000 7f7e42367740 1 -- 172.21.15.83:0/3177377741 shutdown_connections 2024-04-24T09:01:50.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.697+0000 7f7e42367740 1 --2- 172.21.15.83:0/3177377741 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d0b1d5be50 0x55d0b1d5e2e0 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:50.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.697+0000 7f7e42367740 1 --2- 172.21.15.83:0/3177377741 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d0b1d4ba20 0x55d0b1d49410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:50.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.697+0000 7f7e42367740 1 --2- 172.21.15.83:0/3177377741 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d0b1d41350 0x55d0b1d48ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:50.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.697+0000 7f7e1e000640 0 client.19218 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:01:50.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.697+0000 7f7e42367740 1 --2- 172.21.15.83:0/3177377741 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0b1d40a30 0x55d0b1d48990 unknown :-1 s=CLOSED pgs=1686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:50.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.697+0000 7f7e42367740 1 -- 172.21.15.83:0/3177377741 >> 172.21.15.83:0/3177377741 conn(0x55d0b1d3cd50 msgr2=0x55d0b1d42b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:50.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.698+0000 7f7e42367740 1 -- 172.21.15.83:0/3177377741 shutdown_connections 2024-04-24T09:01:50.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.698+0000 7f7e42367740 1 -- 172.21.15.83:0/3177377741 wait complete. 2024-04-24T09:01:50.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.698+0000 7f7e42367740 20 client.19218 trim_cache size 0 max 16384 2024-04-24T09:01:50.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.719+0000 7f7e42367740 1 Processor -- start 2024-04-24T09:01:50.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.719+0000 7f7e42367740 1 -- start start 2024-04-24T09:01:50.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.720+0000 7f7e42367740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d0b1d40a30 0x55d0b1d491e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:50.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.720+0000 7f7e42367740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d0b1d41350 0x55d0b1d43190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:50.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.720+0000 7f7e42367740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0b1d4ba20 0x55d0b1d63bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:50.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.720+0000 7f7e42367740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7e381b15d0 con 0x55d0b1d4ba20 2024-04-24T09:01:50.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.720+0000 7f7e42367740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7e381b31d0 con 0x55d0b1d41350 2024-04-24T09:01:50.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.720+0000 7f7e42367740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7e381b1ca0 con 0x55d0b1d40a30 2024-04-24T09:01:50.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.720+0000 7f7e3d600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0b1d4ba20 0x55d0b1d63bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:50.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.720+0000 7f7e3d600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0b1d4ba20 0x55d0b1d63bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45458/0 (socket says 172.21.15.83:45458) 2024-04-24T09:01:50.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.720+0000 7f7e3d600640 1 -- 172.21.15.83:0/456171038 learned_addr learned my addr 172.21.15.83:0/456171038 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:01:50.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.720+0000 7f7e3cc00640 1 --2- 172.21.15.83:0/456171038 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d0b1d40a30 0x55d0b1d491e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:50.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.720+0000 7f7e3d600640 1 -- 172.21.15.83:0/456171038 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d0b1d40a30 msgr2=0x55d0b1d491e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:50.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.720+0000 7f7e3d600640 1 --2- 172.21.15.83:0/456171038 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d0b1d40a30 0x55d0b1d491e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:50.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.721+0000 7f7e3d600640 1 -- 172.21.15.83:0/456171038 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d0b1d41350 msgr2=0x55d0b1d43190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:50.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.721+0000 7f7e37e00640 1 --2- 172.21.15.83:0/456171038 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d0b1d41350 0x55d0b1d43190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:50.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.721+0000 7f7e3d600640 1 --2- 172.21.15.83:0/456171038 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d0b1d41350 0x55d0b1d43190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:50.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.721+0000 7f7e3d600640 1 -- 172.21.15.83:0/456171038 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0b19b83f0 con 0x55d0b1d4ba20 2024-04-24T09:01:50.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.721+0000 7f7e3d600640 1 --2- 172.21.15.83:0/456171038 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0b1d4ba20 0x55d0b1d63bb0 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f7e300c6550 tx=0x7f7e30030e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20dab9646c1867e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:50.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.721+0000 7f7e29200640 1 -- 172.21.15.83:0/456171038 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e30039e10 con 0x55d0b1d4ba20 2024-04-24T09:01:50.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.721+0000 7f7e29200640 1 -- 172.21.15.83:0/456171038 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7e30036040 con 0x55d0b1d4ba20 2024-04-24T09:01:50.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.721+0000 7f7e29200640 1 -- 172.21.15.83:0/456171038 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e300ba480 con 0x55d0b1d4ba20 2024-04-24T09:01:50.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.722+0000 7f7e42367740 1 -- 172.21.15.83:0/456171038 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0b1d4ba20 msgr2=0x55d0b1d63bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:50.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.722+0000 7f7e42367740 1 --2- 172.21.15.83:0/456171038 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0b1d4ba20 0x55d0b1d63bb0 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f7e300c6550 tx=0x7f7e30030e30 comp rx=0 tx=0).stop 2024-04-24T09:01:50.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.722+0000 7f7e42367740 1 -- 172.21.15.83:0/456171038 shutdown_connections 2024-04-24T09:01:50.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.722+0000 7f7e42367740 1 --2- 172.21.15.83:0/456171038 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0b1d4ba20 0x55d0b1d63bb0 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:50.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.722+0000 7f7e42367740 1 --2- 172.21.15.83:0/456171038 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d0b1d41350 0x55d0b1d43190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:50.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.722+0000 7f7e42367740 1 --2- 172.21.15.83:0/456171038 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d0b1d40a30 0x55d0b1d491e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:50.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.722+0000 7f7e42367740 1 -- 172.21.15.83:0/456171038 >> 172.21.15.83:0/456171038 conn(0x55d0b1d3cd50 msgr2=0x55d0b1d3b740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:50.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.722+0000 7f7e42367740 1 -- 172.21.15.83:0/456171038 shutdown_connections 2024-04-24T09:01:50.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.722+0000 7f7e42367740 1 -- 172.21.15.83:0/456171038 wait complete. 2024-04-24T09:01:50.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.723+0000 7f7e42367740 1 -- start start 2024-04-24T09:01:50.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.723+0000 7f7e42367740 1 Processor -- start 2024-04-24T09:01:50.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.723+0000 7f7e42367740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d0b1d40a30 0x55d0b1d53e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:50.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.723+0000 7f7e42367740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d0b1d41350 0x55d0b1d4f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:50.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.723+0000 7f7e42367740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0b1d4ba20 0x55d0b1d4f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:50.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.723+0000 7f7e42367740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7e381b2aa0 con 0x55d0b1d4ba20 2024-04-24T09:01:50.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.723+0000 7f7e42367740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7e381b23a0 con 0x55d0b1d40a30 2024-04-24T09:01:50.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.723+0000 7f7e42367740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7e381af5d0 con 0x55d0b1d41350 2024-04-24T09:01:50.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.723+0000 7f7e42367740 10 client.0 _ll_register_callbacks cb 0x7f7e2400b980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:01:50.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.724+0000 7f7e3d600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0b1d4ba20 0x55d0b1d4f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:50.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.724+0000 7f7e3d600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0b1d4ba20 0x55d0b1d4f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45472/0 (socket says 172.21.15.83:45472) 2024-04-24T09:01:50.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.724+0000 7f7e3d600640 1 -- 172.21.15.83:0/364931411 learned_addr learned my addr 172.21.15.83:0/364931411 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:01:50.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.724+0000 7f7e3cc00640 1 --2- 172.21.15.83:0/364931411 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d0b1d40a30 0x55d0b1d53e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:50.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.724+0000 7f7e37e00640 1 --2- 172.21.15.83:0/364931411 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d0b1d41350 0x55d0b1d4f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:50.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.724+0000 7f7e3d600640 1 -- 172.21.15.83:0/364931411 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d0b1d41350 msgr2=0x55d0b1d4f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:50.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.724+0000 7f7e3d600640 1 --2- 172.21.15.83:0/364931411 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d0b1d41350 0x55d0b1d4f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:50.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.724+0000 7f7e3d600640 1 -- 172.21.15.83:0/364931411 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d0b1d40a30 msgr2=0x55d0b1d53e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:50.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.724+0000 7f7e3d600640 1 --2- 172.21.15.83:0/364931411 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d0b1d40a30 0x55d0b1d53e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:50.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.724+0000 7f7e3d600640 1 -- 172.21.15.83:0/364931411 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0b19780f0 con 0x55d0b1d4ba20 2024-04-24T09:01:50.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.724+0000 7f7e3cc00640 1 --2- 172.21.15.83:0/364931411 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d0b1d40a30 0x55d0b1d53e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:01:50.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.724+0000 7f7e3d600640 1 --2- 172.21.15.83:0/364931411 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0b1d4ba20 0x55d0b1d4f790 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f7e300ba7e0 tx=0x7f7e300ba1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d85073d7b28c72d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:50.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.724+0000 7f7e1e000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:01:50.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.724+0000 7f7e1e000640 1 -- 172.21.15.83:0/364931411 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e300c21c0 con 0x55d0b1d4ba20 2024-04-24T09:01:50.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.725+0000 7f7e1e000640 1 -- 172.21.15.83:0/364931411 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7f7e300ef730 con 0x55d0b1d4ba20 2024-04-24T09:01:50.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.725+0000 7f7e42367740 1 -- 172.21.15.83:0/364931411 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e30036040 con 0x55d0b1d4ba20 2024-04-24T09:01:50.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.725+0000 7f7e1e000640 1 -- 172.21.15.83:0/364931411 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f7e300a5050 con 0x55d0b1d4ba20 2024-04-24T09:01:50.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.725+0000 7f7e42367740 10 client.19224 fetch_fsmap learned FSMap version 39 2024-04-24T09:01:50.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.725+0000 7f7e42367740 1 -- 172.21.15.83:0/364931411 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55d0b1c6b810 con 0x55d0b1d4ba20 2024-04-24T09:01:50.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.725+0000 7f7e1e000640 1 -- 172.21.15.83:0/364931411 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7e30036040 con 0x55d0b1d4ba20 2024-04-24T09:01:50.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.725+0000 7f7e1e000640 1 -- 172.21.15.83:0/364931411 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f7e300c65f0 con 0x55d0b1d4ba20 2024-04-24T09:01:50.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.725+0000 7f7e42367740 10 client.19224 fetch_fsmap finished waiting for FSMap version 39 2024-04-24T09:01:50.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.725+0000 7f7e42367740 10 client.19224 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:01:50.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.725+0000 7f7e42367740 20 client.19224 populate_metadata read hostname 'smithi083' 2024-04-24T09:01:50.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.725+0000 7f7e42367740 1 --2- 172.21.15.83:0/364931411 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d0b1dcae40 0x55d0b1dcd260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:50.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.725+0000 7f7e42367740 4 client.19224 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T09:01:50.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.725+0000 7f7e42367740 1 -- 172.21.15.83:0/364931411 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d0b1d44590 con 0x55d0b1dcae40 2024-04-24T09:01:50.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.726+0000 7f7e3cc00640 1 --2- 172.21.15.83:0/364931411 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d0b1dcae40 0x55d0b1dcd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:50.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.728+0000 7f7e1e000640 10 client.19224 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:01:50.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.728+0000 7f7e3cc00640 1 --2- 172.21.15.83:0/364931411 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d0b1dcae40 0x55d0b1dcd260 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4f689dcb7d35bd2 server_cookie=880701e768025a7e in_seq=0 out_seq=0 2024-04-24T09:01:50.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.734+0000 7f7e1e000640 1 -- 172.21.15.83:0/364931411 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55d0b1d44590 con 0x55d0b1dcae40 2024-04-24T09:01:50.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.734+0000 7f7e1e000640 10 client.19224 handle_command_reply: tid=0 2024-04-24T09:01:50.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.734+0000 7f7e42367740 1 client.19224 shutdown 2024-04-24T09:01:50.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.734+0000 7f7e42367740 1 -- 172.21.15.83:0/364931411 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0b1d4ba20 msgr2=0x55d0b1d4f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:50.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.734+0000 7f7e42367740 1 --2- 172.21.15.83:0/364931411 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0b1d4ba20 0x55d0b1d4f790 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f7e300ba7e0 tx=0x7f7e300ba1e0 comp rx=0 tx=0).stop 2024-04-24T09:01:50.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.734+0000 7f7e42367740 1 -- 172.21.15.83:0/364931411 shutdown_connections 2024-04-24T09:01:50.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.734+0000 7f7e42367740 1 --2- 172.21.15.83:0/364931411 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d0b1dcae40 0x55d0b1dcd260 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:50.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.734+0000 7f7e42367740 1 --2- 172.21.15.83:0/364931411 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d0b1d4ba20 0x55d0b1d4f790 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:50.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.734+0000 7f7e42367740 1 --2- 172.21.15.83:0/364931411 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d0b1d41350 0x55d0b1d4f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:50.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.734+0000 7f7e1e000640 0 client.19224 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:01:50.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.734+0000 7f7e42367740 1 --2- 172.21.15.83:0/364931411 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d0b1d40a30 0x55d0b1d53e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:50.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.734+0000 7f7e42367740 1 -- 172.21.15.83:0/364931411 >> 172.21.15.83:0/364931411 conn(0x55d0b1d3cd50 msgr2=0x55d0b1d3b740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:50.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.734+0000 7f7e42367740 1 -- 172.21.15.83:0/364931411 shutdown_connections 2024-04-24T09:01:50.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.734+0000 7f7e42367740 1 -- 172.21.15.83:0/364931411 wait complete. 2024-04-24T09:01:50.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.734+0000 7f7e42367740 20 client.19224 trim_cache size 0 max 16384 2024-04-24T09:01:50.735 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T09:01:50.736 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-04-24T09:01:50.736 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-04-24T09:01:50.736 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T09:01:50.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.735+0000 7f7e3e000640 1 -- 172.21.15.83:0/1214945557 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f7e200a3da0 msgr2=0x7f7e200a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:50.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.735+0000 7f7e3e000640 1 --2- 172.21.15.83:0/1214945557 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f7e200a3da0 0x7f7e200a6260 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x55d0b1d41ff0 tx=0x7f7e2400b7b0 comp rx=0 tx=0).stop 2024-04-24T09:01:50.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.735+0000 7f7e3e000640 1 -- 172.21.15.83:0/1214945557 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e3819a700 msgr2=0x7f7e3819ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:50.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.735+0000 7f7e3e000640 1 --2- 172.21.15.83:0/1214945557 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e3819a700 0x7f7e3819ab80 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f7e30031040 tx=0x7f7e30003970 comp rx=0 tx=0).stop 2024-04-24T09:01:50.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.735+0000 7f7e3e000640 1 -- 172.21.15.83:0/1214945557 shutdown_connections 2024-04-24T09:01:50.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.735+0000 7f7e3e000640 1 --2- 172.21.15.83:0/1214945557 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f7e200a3da0 0x7f7e200a6260 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:50.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.735+0000 7f7e3e000640 1 --2- 172.21.15.83:0/1214945557 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e3819a700 0x7f7e3819ab80 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:50.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.735+0000 7f7e3e000640 1 --2- 172.21.15.83:0/1214945557 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7e381071e0 0x7f7e381a0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:50.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.735+0000 7f7e3e000640 1 --2- 172.21.15.83:0/1214945557 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e38102110 0x7f7e381a05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:50.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.735+0000 7f7e3e000640 1 -- 172.21.15.83:0/1214945557 >> 172.21.15.83:0/1214945557 conn(0x7f7e380fd9d0 msgr2=0x7f7e380fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:50.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.735+0000 7f7e3e000640 1 -- 172.21.15.83:0/1214945557 shutdown_connections 2024-04-24T09:01:50.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.735+0000 7f7e3e000640 1 -- 172.21.15.83:0/1214945557 wait complete. 2024-04-24T09:01:50.752 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T09:01:50.753 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T09:01:50.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.960+0000 7f883de00640 1 -- 172.21.15.83:0/1402988302 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8838108e40 msgr2=0x7f88381152b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:50.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.960+0000 7f883de00640 1 --2- 172.21.15.83:0/1402988302 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8838108e40 0x7f88381152b0 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7f8830009900 tx=0x7f883002a480 comp rx=0 tx=0).stop 2024-04-24T09:01:50.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.960+0000 7f883de00640 1 -- 172.21.15.83:0/1402988302 shutdown_connections 2024-04-24T09:01:50.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.960+0000 7f883de00640 1 --2- 172.21.15.83:0/1402988302 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8838108e40 0x7f88381152b0 unknown :-1 s=CLOSED pgs=1689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:50.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.960+0000 7f883de00640 1 --2- 172.21.15.83:0/1402988302 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f88381084a0 0x7f8838108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:50.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.960+0000 7f883de00640 1 --2- 172.21.15.83:0/1402988302 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f883810d3c0 0x7f883810d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:50.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.960+0000 7f883de00640 1 -- 172.21.15.83:0/1402988302 >> 172.21.15.83:0/1402988302 conn(0x7f8838077b80 msgr2=0x7f8838079fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:50.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.960+0000 7f883de00640 1 -- 172.21.15.83:0/1402988302 shutdown_connections 2024-04-24T09:01:50.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.961+0000 7f883de00640 1 -- 172.21.15.83:0/1402988302 wait complete. 2024-04-24T09:01:50.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.962+0000 7f883de00640 1 Processor -- start 2024-04-24T09:01:50.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.962+0000 7f883de00640 1 -- start start 2024-04-24T09:01:50.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.962+0000 7f883de00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f88381084a0 0x7f88381a4830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:50.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.962+0000 7f883de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8838108e40 0x7f88381a4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:50.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.962+0000 7f883de00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f883810d3c0 0x7f883819e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:50.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.962+0000 7f883de00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8838118700 con 0x7f8838108e40 2024-04-24T09:01:50.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.962+0000 7f883de00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8838118580 con 0x7f883810d3c0 2024-04-24T09:01:50.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.962+0000 7f883de00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8838118880 con 0x7f88381084a0 2024-04-24T09:01:50.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.962+0000 7f8837e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8838108e40 0x7f88381a4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:50.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.962+0000 7f8837e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8838108e40 0x7f88381a4d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45490/0 (socket says 172.21.15.83:45490) 2024-04-24T09:01:50.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.962+0000 7f8837e00640 1 -- 172.21.15.83:0/967350755 learned_addr learned my addr 172.21.15.83:0/967350755 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:01:50.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.963+0000 7f883ca00640 1 --2- 172.21.15.83:0/967350755 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f88381084a0 0x7f88381a4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:50.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.963+0000 7f883d400640 1 --2- 172.21.15.83:0/967350755 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f883810d3c0 0x7f883819e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:50.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.963+0000 7f8837e00640 1 -- 172.21.15.83:0/967350755 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f88381084a0 msgr2=0x7f88381a4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:50.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.963+0000 7f8837e00640 1 --2- 172.21.15.83:0/967350755 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f88381084a0 0x7f88381a4830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:50.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.963+0000 7f8837e00640 1 -- 172.21.15.83:0/967350755 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f883810d3c0 msgr2=0x7f883819e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:50.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.963+0000 7f8837e00640 1 --2- 172.21.15.83:0/967350755 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f883810d3c0 0x7f883819e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:50.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.963+0000 7f8837e00640 1 -- 172.21.15.83:0/967350755 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f883819f160 con 0x7f8838108e40 2024-04-24T09:01:50.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.963+0000 7f883d400640 1 --2- 172.21.15.83:0/967350755 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f883810d3c0 0x7f883819e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:01:50.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.963+0000 7f8837e00640 1 --2- 172.21.15.83:0/967350755 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8838108e40 0x7f88381a4d70 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f882400d770 tx=0x7f882400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:50.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.964+0000 7f8835600640 1 -- 172.21.15.83:0/967350755 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8824017070 con 0x7f8838108e40 2024-04-24T09:01:50.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.964+0000 7f883de00640 1 -- 172.21.15.83:0/967350755 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f883819f450 con 0x7f8838108e40 2024-04-24T09:01:50.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.964+0000 7f8835600640 1 -- 172.21.15.83:0/967350755 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f88240054d0 con 0x7f8838108e40 2024-04-24T09:01:50.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.964+0000 7f8835600640 1 -- 172.21.15.83:0/967350755 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8824020060 con 0x7f8838108e40 2024-04-24T09:01:50.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.964+0000 7f883de00640 1 -- 172.21.15.83:0/967350755 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88381ab6f0 con 0x7f8838108e40 2024-04-24T09:01:50.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.965+0000 7f883de00640 1 -- 172.21.15.83:0/967350755 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8838069190 con 0x7f8838108e40 2024-04-24T09:01:50.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.967+0000 7f8835600640 1 -- 172.21.15.83:0/967350755 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8824004c90 con 0x7f8838108e40 2024-04-24T09:01:50.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.968+0000 7f8835600640 1 --2- 172.21.15.83:0/967350755 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f88200a3d20 0x7f88200a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:50.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.969+0000 7f8835600640 1 -- 172.21.15.83:0/967350755 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7f88240d08b0 con 0x7f8838108e40 2024-04-24T09:01:50.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.969+0000 7f883ca00640 1 --2- 172.21.15.83:0/967350755 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f88200a3d20 0x7f88200a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:50.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.970+0000 7f883ca00640 1 --2- 172.21.15.83:0/967350755 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f88200a3d20 0x7f88200a61e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f882c005c40 tx=0x7f882c005bd0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:50.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:50.974+0000 7f8835600640 1 -- 172.21.15.83:0/967350755 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f882409cad0 con 0x7f8838108e40 2024-04-24T09:01:51.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.251+0000 7f883de00640 1 -- 172.21.15.83:0/967350755 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f883802d620 con 0x7f8838108e40 2024-04-24T09:01:51.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.252+0000 7f8835600640 1 -- 172.21.15.83:0/967350755 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+11951 (secure 0 0 0) 0x7f882409c3d0 con 0x7f8838108e40 2024-04-24T09:01:51.252 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T09:01:51.253 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/358872257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":358872257},{"type":"v1","addr":"172.21.15.83:6835","nonce":358872257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3157070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3157070851},{"type":"v1","addr":"172.21.15.83:6837","nonce":3157070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":40,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T08:48:29.422576+0000","modified":"2024-04-24T09:01:50.405140+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34520,"mds_1":34493,"mds_2":34451,"mds_3":34427,"mds_4":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14658":{"gid":14658,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6839/4085470844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":4085470844},{"type":"v1","addr":"172.21.15.83:6839","nonce":4085470844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14676":{"gid":14676,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6841/2634754210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":2634754210},{"type":"v1","addr":"172.21.15.83:6841","nonce":2634754210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24382":{"gid":24382,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/1859921310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1859921310},{"type":"v1","addr":"172.21.15.189:6833","nonce":1859921310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/2241923197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2241923197},{"type":"v1","addr":"172.21.15.189:6835","nonce":2241923197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.189:6839/2548965336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":2548965336},{"type":"v1","addr":"172.21.15.189:6839","nonce":2548965336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26686":{"gid":26686,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6837/1678817161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":1678817161},{"type":"v1","addr":"172.21.15.189:6837","nonce":1678817161}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.138:6833/1157967658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1157967658},{"type":"v1","addr":"172.21.15.138:6833","nonce":1157967658}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.138:6835/1909219758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1909219758},{"type":"v1","addr":"172.21.15.138:6835","nonce":1909219758}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34493":{"gid":34493,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.138:6837/900566754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":900566754},{"type":"v1","addr":"172.21.15.138:6837","nonce":900566754}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34520":{"gid":34520,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.138:6839/2236932393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2236932393},{"type":"v1","addr":"172.21.15.138:6839","nonce":2236932393}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34520,"qdb_cluster":[34451,34493,34427,34520,24454]},"id":1}]} 2024-04-24T09:01:51.253 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 40 2024-04-24T09:01:51.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.255+0000 7f883de00640 1 -- 172.21.15.83:0/967350755 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f88200a3d20 msgr2=0x7f88200a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:51.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.255+0000 7f883de00640 1 --2- 172.21.15.83:0/967350755 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f88200a3d20 0x7f88200a61e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f882c005c40 tx=0x7f882c005bd0 comp rx=0 tx=0).stop 2024-04-24T09:01:51.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.255+0000 7f883de00640 1 -- 172.21.15.83:0/967350755 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8838108e40 msgr2=0x7f88381a4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:51.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.255+0000 7f883de00640 1 --2- 172.21.15.83:0/967350755 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8838108e40 0x7f88381a4d70 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f882400d770 tx=0x7f882400dc30 comp rx=0 tx=0).stop 2024-04-24T09:01:51.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.255+0000 7f883de00640 1 -- 172.21.15.83:0/967350755 shutdown_connections 2024-04-24T09:01:51.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.255+0000 7f883de00640 1 --2- 172.21.15.83:0/967350755 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f88200a3d20 0x7f88200a61e0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:51.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.255+0000 7f883de00640 1 --2- 172.21.15.83:0/967350755 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f883810d3c0 0x7f883819e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:51.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.255+0000 7f883de00640 1 --2- 172.21.15.83:0/967350755 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8838108e40 0x7f88381a4d70 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:51.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.255+0000 7f883de00640 1 --2- 172.21.15.83:0/967350755 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f88381084a0 0x7f88381a4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:51.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.256+0000 7f883de00640 1 -- 172.21.15.83:0/967350755 >> 172.21.15.83:0/967350755 conn(0x7f8838077b80 msgr2=0x7f8838078330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:51.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.256+0000 7f883de00640 1 -- 172.21.15.83:0/967350755 shutdown_connections 2024-04-24T09:01:51.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.256+0000 7f883de00640 1 -- 172.21.15.83:0/967350755 wait complete. 2024-04-24T09:01:51.276 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-24T09:01:51.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.485+0000 7f4204e00640 1 -- 172.21.15.83:0/1130234250 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4200101720 msgr2=0x7f420010e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:51.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.485+0000 7f4204e00640 1 --2- 172.21.15.83:0/1130234250 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4200101720 0x7f420010e470 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7f41ec009900 tx=0x7f41ec02a3f0 comp rx=0 tx=0).stop 2024-04-24T09:01:51.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.485+0000 7f4204e00640 1 -- 172.21.15.83:0/1130234250 shutdown_connections 2024-04-24T09:01:51.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.485+0000 7f4204e00640 1 --2- 172.21.15.83:0/1130234250 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f420010e9b0 0x7f4200110da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:51.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.485+0000 7f4204e00640 1 --2- 172.21.15.83:0/1130234250 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4200101720 0x7f420010e470 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:51.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.485+0000 7f4204e00640 1 --2- 172.21.15.83:0/1130234250 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4200100e00 0x7f42001011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:51.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.485+0000 7f4204e00640 1 -- 172.21.15.83:0/1130234250 >> 172.21.15.83:0/1130234250 conn(0x7f4200078000 msgr2=0x7f4200078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:51.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.485+0000 7f4204e00640 1 -- 172.21.15.83:0/1130234250 shutdown_connections 2024-04-24T09:01:51.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.485+0000 7f4204e00640 1 -- 172.21.15.83:0/1130234250 wait complete. 2024-04-24T09:01:51.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.486+0000 7f4204e00640 1 Processor -- start 2024-04-24T09:01:51.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.486+0000 7f4204e00640 1 -- start start 2024-04-24T09:01:51.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.487+0000 7f4204e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4200100e00 0x7f42001a06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:51.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.487+0000 7f4204e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4200101720 0x7f42001a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:51.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.487+0000 7f4204e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f420010e9b0 0x7f420019a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:51.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.487+0000 7f4204e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4200114470 con 0x7f4200101720 2024-04-24T09:01:51.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.487+0000 7f4204e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f42001142f0 con 0x7f420010e9b0 2024-04-24T09:01:51.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.487+0000 7f4204e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f42001145f0 con 0x7f4200100e00 2024-04-24T09:01:51.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.487+0000 7f41fd400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4200101720 0x7f42001a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:51.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.487+0000 7f41fd400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4200101720 0x7f42001a0c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45506/0 (socket says 172.21.15.83:45506) 2024-04-24T09:01:51.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.487+0000 7f41fd400640 1 -- 172.21.15.83:0/2251931290 learned_addr learned my addr 172.21.15.83:0/2251931290 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:01:51.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.487+0000 7f41fde00640 1 --2- 172.21.15.83:0/2251931290 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4200100e00 0x7f42001a06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:51.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.487+0000 7f41fe800640 1 --2- 172.21.15.83:0/2251931290 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f420010e9b0 0x7f420019a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:51.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.487+0000 7f41fd400640 1 -- 172.21.15.83:0/2251931290 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4200100e00 msgr2=0x7f42001a06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:51.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.487+0000 7f41fd400640 1 --2- 172.21.15.83:0/2251931290 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4200100e00 0x7f42001a06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:51.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.487+0000 7f41fd400640 1 -- 172.21.15.83:0/2251931290 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f420010e9b0 msgr2=0x7f420019a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:51.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.487+0000 7f41fd400640 1 --2- 172.21.15.83:0/2251931290 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f420010e9b0 0x7f420019a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:51.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.487+0000 7f41fd400640 1 -- 172.21.15.83:0/2251931290 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f420019b010 con 0x7f4200101720 2024-04-24T09:01:51.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.487+0000 7f41fe800640 1 --2- 172.21.15.83:0/2251931290 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f420010e9b0 0x7f420019a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:01:51.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.488+0000 7f41fd400640 1 --2- 172.21.15.83:0/2251931290 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4200101720 0x7f42001a0c20 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7f41ec005940 tx=0x7f41ec0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:51.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.488+0000 7f41f6a00640 1 -- 172.21.15.83:0/2251931290 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41ec03f030 con 0x7f4200101720 2024-04-24T09:01:51.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.488+0000 7f41f6a00640 1 -- 172.21.15.83:0/2251931290 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f41ec003cb0 con 0x7f4200101720 2024-04-24T09:01:51.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.488+0000 7f4204e00640 1 -- 172.21.15.83:0/2251931290 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f420019b2a0 con 0x7f4200101720 2024-04-24T09:01:51.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.488+0000 7f41f6a00640 1 -- 172.21.15.83:0/2251931290 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41ec035070 con 0x7f4200101720 2024-04-24T09:01:51.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.488+0000 7f4204e00640 1 -- 172.21.15.83:0/2251931290 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42001a75a0 con 0x7f4200101720 2024-04-24T09:01:51.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.493+0000 7f4207c10740 1 Processor -- start 2024-04-24T09:01:51.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.493+0000 7f41f6a00640 1 -- 172.21.15.83:0/2251931290 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f41ec042030 con 0x7f4200101720 2024-04-24T09:01:51.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.493+0000 7f4207c10740 1 -- start start 2024-04-24T09:01:51.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.493+0000 7f4207c10740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557ce942c950 0x557ce942cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:51.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.493+0000 7f4207c10740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557ce942d270 0x557ce9437400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:51.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.493+0000 7f4207c10740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557ce9437940 0x557ce9439d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:51.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.494+0000 7f4207c10740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x557ce93921d0 con 0x557ce942d270 2024-04-24T09:01:51.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.494+0000 7f4207c10740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x557ce9338450 con 0x557ce9437940 2024-04-24T09:01:51.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.494+0000 7f4207c10740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x557ce938e190 con 0x557ce942c950 2024-04-24T09:01:51.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.494+0000 7f41fde00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557ce942d270 0x557ce9437400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:51.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.494+0000 7f41fde00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557ce942d270 0x557ce9437400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45516/0 (socket says 172.21.15.83:45516) 2024-04-24T09:01:51.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.494+0000 7f41fde00640 1 -- 172.21.15.83:0/3599157454 learned_addr learned my addr 172.21.15.83:0/3599157454 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:01:51.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.494+0000 7f41fe800640 1 --2- 172.21.15.83:0/3599157454 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557ce942c950 0x557ce942cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:51.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.494+0000 7f41fd400640 1 --2- 172.21.15.83:0/3599157454 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557ce9437940 0x557ce9439d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:51.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.494+0000 7f41fde00640 1 -- 172.21.15.83:0/3599157454 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557ce942c950 msgr2=0x557ce942cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:51.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.494+0000 7f41fde00640 1 --2- 172.21.15.83:0/3599157454 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557ce942c950 0x557ce942cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:51.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.494+0000 7f41fde00640 1 -- 172.21.15.83:0/3599157454 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557ce9437940 msgr2=0x557ce9439d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:51.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.494+0000 7f41fde00640 1 --2- 172.21.15.83:0/3599157454 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557ce9437940 0x557ce9439d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:51.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.494+0000 7f41fde00640 1 -- 172.21.15.83:0/3599157454 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ce93f5310 con 0x557ce942d270 2024-04-24T09:01:51.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.495+0000 7f41f6a00640 1 --2- 172.21.15.83:0/2251931290 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f41e40a3d20 0x7f41e40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:51.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.495+0000 7f41fde00640 1 --2- 172.21.15.83:0/3599157454 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557ce942d270 0x557ce9437400 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f41f000cbb0 tx=0x7f41f000e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffd402d4f0bfa91f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:51.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.495+0000 7f41f6a00640 1 -- 172.21.15.83:0/2251931290 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7f41ec0f1500 con 0x7f4200101720 2024-04-24T09:01:51.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.495+0000 7f41eb400640 1 -- 172.21.15.83:0/3599157454 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f0005220 con 0x557ce942d270 2024-04-24T09:01:51.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.495+0000 7f41eb400640 1 -- 172.21.15.83:0/3599157454 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f41f0004c50 con 0x557ce942d270 2024-04-24T09:01:51.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.495+0000 7f41eb400640 1 -- 172.21.15.83:0/3599157454 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f0010720 con 0x557ce942d270 2024-04-24T09:01:51.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.495+0000 7f41fe800640 1 --2- 172.21.15.83:0/2251931290 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f41e40a3d20 0x7f41e40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:51.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.496+0000 7f4207c10740 1 -- 172.21.15.83:0/3599157454 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557ce942d270 msgr2=0x557ce9437400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:51.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.496+0000 7f4207c10740 1 --2- 172.21.15.83:0/3599157454 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557ce942d270 0x557ce9437400 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f41f000cbb0 tx=0x7f41f000e720 comp rx=0 tx=0).stop 2024-04-24T09:01:51.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.496+0000 7f41fe800640 1 --2- 172.21.15.83:0/2251931290 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f41e40a3d20 0x7f41e40a61e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f420006b790 tx=0x7f41f8009530 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:51.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.496+0000 7f4207c10740 1 -- 172.21.15.83:0/3599157454 shutdown_connections 2024-04-24T09:01:51.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.496+0000 7f4207c10740 1 --2- 172.21.15.83:0/3599157454 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557ce9437940 0x557ce9439d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:51.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.496+0000 7f4207c10740 1 --2- 172.21.15.83:0/3599157454 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557ce942d270 0x557ce9437400 unknown :-1 s=CLOSED pgs=1693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:51.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.496+0000 7f4207c10740 1 --2- 172.21.15.83:0/3599157454 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557ce942c950 0x557ce942cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:51.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.496+0000 7f4207c10740 1 -- 172.21.15.83:0/3599157454 >> 172.21.15.83:0/3599157454 conn(0x557ce9428c70 msgr2=0x557ce9429050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:51.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.496+0000 7f4207c10740 1 -- 172.21.15.83:0/3599157454 shutdown_connections 2024-04-24T09:01:51.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.496+0000 7f4207c10740 1 -- 172.21.15.83:0/3599157454 wait complete. 2024-04-24T09:01:51.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.497+0000 7f4207c10740 1 -- start start 2024-04-24T09:01:51.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.497+0000 7f4207c10740 1 Processor -- start 2024-04-24T09:01:51.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.497+0000 7f4207c10740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557ce942c950 0x557ce9437390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:51.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.497+0000 7f4207c10740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557ce942d270 0x557ce9434680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:51.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.498+0000 7f4207c10740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557ce9437940 0x557ce9434dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:51.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.498+0000 7f4207c10740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x557ce943a7d0 con 0x557ce942d270 2024-04-24T09:01:51.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.498+0000 7f4207c10740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x557ce934b810 con 0x557ce9437940 2024-04-24T09:01:51.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.498+0000 7f4207c10740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x557ce943a950 con 0x557ce942c950 2024-04-24T09:01:51.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.498+0000 7f4207c10740 10 client.0 _ll_register_callbacks cb 0x557ce9319140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:01:51.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.498+0000 7f41fd400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557ce942d270 0x557ce9434680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:51.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.498+0000 7f41fd400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557ce942d270 0x557ce9434680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45522/0 (socket says 172.21.15.83:45522) 2024-04-24T09:01:51.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.498+0000 7f41fd400640 1 -- 172.21.15.83:0/1691117530 learned_addr learned my addr 172.21.15.83:0/1691117530 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:01:51.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.498+0000 7f41fe800640 1 --2- 172.21.15.83:0/1691117530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557ce9437940 0x557ce9434dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:51.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.498+0000 7f41fde00640 1 --2- 172.21.15.83:0/1691117530 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557ce942c950 0x557ce9437390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:51.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.498+0000 7f41fd400640 1 -- 172.21.15.83:0/1691117530 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557ce942c950 msgr2=0x557ce9437390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:51.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.498+0000 7f41fd400640 1 --2- 172.21.15.83:0/1691117530 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557ce942c950 0x557ce9437390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:51.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.498+0000 7f41fd400640 1 -- 172.21.15.83:0/1691117530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557ce9437940 msgr2=0x557ce9434dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:51.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.498+0000 7f41fd400640 1 --2- 172.21.15.83:0/1691117530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557ce9437940 0x557ce9434dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:51.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.498+0000 7f41fd400640 1 -- 172.21.15.83:0/1691117530 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ce935c3f0 con 0x557ce942d270 2024-04-24T09:01:51.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.498+0000 7f41fde00640 1 --2- 172.21.15.83:0/1691117530 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557ce942c950 0x557ce9437390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:01:51.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.499+0000 7f41fd400640 1 --2- 172.21.15.83:0/1691117530 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557ce942d270 0x557ce9434680 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f41ec0bcd80 tx=0x7f41ec02a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25a159e771a61d51 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:51.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.499+0000 7f41dfe00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:01:51.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.499+0000 7f41dfe00640 1 -- 172.21.15.83:0/1691117530 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41ec049070 con 0x557ce942d270 2024-04-24T09:01:51.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.499+0000 7f41dfe00640 1 -- 172.21.15.83:0/1691117530 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7f41ec0b96f0 con 0x557ce942d270 2024-04-24T09:01:51.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.499+0000 7f4207c10740 1 -- 172.21.15.83:0/1691117530 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41f0004c50 con 0x557ce942d270 2024-04-24T09:01:51.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.499+0000 7f41dfe00640 1 -- 172.21.15.83:0/1691117530 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f41ec0bcbc0 con 0x557ce942d270 2024-04-24T09:01:51.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.499+0000 7f4207c10740 10 client.19242 fetch_fsmap learned FSMap version 40 2024-04-24T09:01:51.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.499+0000 7f4207c10740 1 -- 172.21.15.83:0/1691117530 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x557ce90a43f0 con 0x557ce942d270 2024-04-24T09:01:51.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.499+0000 7f41dfe00640 1 -- 172.21.15.83:0/1691117530 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f41f0004c50 con 0x557ce942d270 2024-04-24T09:01:51.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.500+0000 7f41dfe00640 1 -- 172.21.15.83:0/1691117530 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 40) ==== 8382+0+0 (secure 0 0 0) 0x7f41ec0c1080 con 0x557ce942d270 2024-04-24T09:01:51.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.500+0000 7f4207c10740 10 client.19242 fetch_fsmap finished waiting for FSMap version 40 2024-04-24T09:01:51.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.500+0000 7f4207c10740 10 client.19242 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-24T09:01:51.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.500+0000 7f4207c10740 20 client.19242 populate_metadata read hostname 'smithi083' 2024-04-24T09:01:51.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.500+0000 7f4207c10740 1 --2- 172.21.15.83:0/1691117530 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x557ce94479a0 0x557ce9449db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:51.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.500+0000 7f4207c10740 4 client.19242 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:01:51.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.500+0000 7f4207c10740 1 -- 172.21.15.83:0/1691117530 --> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557ce939fbf0 con 0x557ce94479a0 2024-04-24T09:01:51.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.500+0000 7f41fde00640 1 --2- 172.21.15.83:0/1691117530 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x557ce94479a0 0x557ce9449db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:51.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.502+0000 7f41fde00640 1 --2- 172.21.15.83:0/1691117530 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x557ce94479a0 0x557ce9449db0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=820d5ee26f6eac9 server_cookie=ebb0a4668533bd06 in_seq=0 out_seq=0 2024-04-24T09:01:51.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.502+0000 7f41dfe00640 10 client.19242 ms_handle_connect on v2:172.21.15.189:6838/2548965336 2024-04-24T09:01:51.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.503+0000 7f41dfe00640 1 -- 172.21.15.83:0/1691117530 <== mds.4 v2:172.21.15.189:6838/2548965336 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557ce939fbf0 con 0x557ce94479a0 2024-04-24T09:01:51.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.503+0000 7f41dfe00640 10 client.19242 handle_command_reply: tid=0 2024-04-24T09:01:51.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.503+0000 7f4207c10740 1 client.19242 shutdown 2024-04-24T09:01:51.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.504+0000 7f4207c10740 1 -- 172.21.15.83:0/1691117530 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557ce942d270 msgr2=0x557ce9434680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:51.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.504+0000 7f4207c10740 1 --2- 172.21.15.83:0/1691117530 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557ce942d270 0x557ce9434680 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f41ec0bcd80 tx=0x7f41ec02a3f0 comp rx=0 tx=0).stop 2024-04-24T09:01:51.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.504+0000 7f4207c10740 1 -- 172.21.15.83:0/1691117530 shutdown_connections 2024-04-24T09:01:51.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.504+0000 7f4207c10740 1 --2- 172.21.15.83:0/1691117530 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x557ce94479a0 0x557ce9449db0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:51.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.504+0000 7f4207c10740 1 --2- 172.21.15.83:0/1691117530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557ce9437940 0x557ce9434dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:51.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.504+0000 7f4207c10740 1 --2- 172.21.15.83:0/1691117530 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557ce942d270 0x557ce9434680 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:51.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.504+0000 7f41dfe00640 0 client.19242 ms_handle_reset on v2:172.21.15.189:6838/2548965336 2024-04-24T09:01:51.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.504+0000 7f4207c10740 1 --2- 172.21.15.83:0/1691117530 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557ce942c950 0x557ce9437390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:51.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.504+0000 7f4207c10740 1 -- 172.21.15.83:0/1691117530 >> 172.21.15.83:0/1691117530 conn(0x557ce9428c70 msgr2=0x557ce942ea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:51.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.504+0000 7f4207c10740 1 -- 172.21.15.83:0/1691117530 shutdown_connections 2024-04-24T09:01:51.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.504+0000 7f4207c10740 1 -- 172.21.15.83:0/1691117530 wait complete. 2024-04-24T09:01:51.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.504+0000 7f4207c10740 20 client.19242 trim_cache size 0 max 16384 2024-04-24T09:01:51.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.521+0000 7f4207c10740 1 Processor -- start 2024-04-24T09:01:51.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.521+0000 7f4207c10740 1 -- start start 2024-04-24T09:01:51.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.521+0000 7f4207c10740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557ce942c950 0x557ce9449d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:51.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.521+0000 7f4207c10740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557ce942d270 0x557ce9443690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:51.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.521+0000 7f4207c10740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557ce9437940 0x557ce9436ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:51.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.521+0000 7f4207c10740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f42001b17d0 con 0x557ce942c950 2024-04-24T09:01:51.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.521+0000 7f4207c10740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f42001b3400 con 0x557ce942d270 2024-04-24T09:01:51.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.521+0000 7f4207c10740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f42001b1ea0 con 0x557ce9437940 2024-04-24T09:01:51.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.522+0000 7f41fde00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557ce942c950 0x557ce9449d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:51.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.522+0000 7f41fde00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557ce942c950 0x557ce9449d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45528/0 (socket says 172.21.15.83:45528) 2024-04-24T09:01:51.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.522+0000 7f41fde00640 1 -- 172.21.15.83:0/3846656335 learned_addr learned my addr 172.21.15.83:0/3846656335 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:01:51.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.522+0000 7f41fe800640 1 --2- 172.21.15.83:0/3846656335 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557ce9437940 0x557ce9436ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:51.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.522+0000 7f41fd400640 1 --2- 172.21.15.83:0/3846656335 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557ce942d270 0x557ce9443690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:51.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.522+0000 7f41fde00640 1 -- 172.21.15.83:0/3846656335 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557ce9437940 msgr2=0x557ce9436ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:51.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.522+0000 7f41fde00640 1 --2- 172.21.15.83:0/3846656335 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557ce9437940 0x557ce9436ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:51.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.522+0000 7f41fde00640 1 -- 172.21.15.83:0/3846656335 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557ce942d270 msgr2=0x557ce9443690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:51.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.522+0000 7f41fde00640 1 --2- 172.21.15.83:0/3846656335 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557ce942d270 0x557ce9443690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:51.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.522+0000 7f41fde00640 1 -- 172.21.15.83:0/3846656335 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ce916e7d0 con 0x557ce942c950 2024-04-24T09:01:51.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.522+0000 7f41fde00640 1 --2- 172.21.15.83:0/3846656335 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557ce942c950 0x557ce9449d90 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f41f0005fd0 tx=0x7f41f0005ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb32d9878206fb0f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:51.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.523+0000 7f41eaa00640 1 -- 172.21.15.83:0/3846656335 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f0004220 con 0x557ce942c950 2024-04-24T09:01:51.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.523+0000 7f41eaa00640 1 -- 172.21.15.83:0/3846656335 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f41f00043c0 con 0x557ce942c950 2024-04-24T09:01:51.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.523+0000 7f41eaa00640 1 -- 172.21.15.83:0/3846656335 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f0010720 con 0x557ce942c950 2024-04-24T09:01:51.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.524+0000 7f4207c10740 1 -- 172.21.15.83:0/3846656335 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557ce942c950 msgr2=0x557ce9449d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:51.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.524+0000 7f4207c10740 1 --2- 172.21.15.83:0/3846656335 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557ce942c950 0x557ce9449d90 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f41f0005fd0 tx=0x7f41f0005ce0 comp rx=0 tx=0).stop 2024-04-24T09:01:51.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.524+0000 7f4207c10740 1 -- 172.21.15.83:0/3846656335 shutdown_connections 2024-04-24T09:01:51.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.524+0000 7f4207c10740 1 --2- 172.21.15.83:0/3846656335 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557ce9437940 0x557ce9436ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:51.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.524+0000 7f4207c10740 1 --2- 172.21.15.83:0/3846656335 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557ce942d270 0x557ce9443690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:51.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.524+0000 7f4207c10740 1 --2- 172.21.15.83:0/3846656335 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557ce942c950 0x557ce9449d90 unknown :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:51.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.524+0000 7f4207c10740 1 -- 172.21.15.83:0/3846656335 >> 172.21.15.83:0/3846656335 conn(0x557ce9428c70 msgr2=0x557ce942f940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:51.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.524+0000 7f4207c10740 1 -- 172.21.15.83:0/3846656335 shutdown_connections 2024-04-24T09:01:51.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.524+0000 7f4207c10740 1 -- 172.21.15.83:0/3846656335 wait complete. 2024-04-24T09:01:51.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.524+0000 7f4207c10740 1 -- start start 2024-04-24T09:01:51.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.524+0000 7f4207c10740 1 Processor -- start 2024-04-24T09:01:51.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.525+0000 7f4207c10740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557ce942c950 0x557ce9438670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:51.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.525+0000 7f4207c10740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557ce942d270 0x557ce9438bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:51.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.525+0000 7f4207c10740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557ce9437940 0x557ce94390f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:51.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.525+0000 7f4207c10740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f42001b2cd0 con 0x557ce9437940 2024-04-24T09:01:51.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.525+0000 7f4207c10740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f42001b25a0 con 0x557ce942c950 2024-04-24T09:01:51.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.525+0000 7f4207c10740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f42001af7d0 con 0x557ce942d270 2024-04-24T09:01:51.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.525+0000 7f4207c10740 10 client.0 _ll_register_callbacks cb 0x7f41ec005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:01:51.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.525+0000 7f41fe800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557ce9437940 0x557ce94390f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:51.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.525+0000 7f41fe800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557ce9437940 0x557ce94390f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45544/0 (socket says 172.21.15.83:45544) 2024-04-24T09:01:51.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.525+0000 7f41fe800640 1 -- 172.21.15.83:0/149426477 learned_addr learned my addr 172.21.15.83:0/149426477 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:01:51.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.525+0000 7f41fde00640 1 --2- 172.21.15.83:0/149426477 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557ce942c950 0x557ce9438670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:51.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.525+0000 7f41fd400640 1 --2- 172.21.15.83:0/149426477 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557ce942d270 0x557ce9438bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:51.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.526+0000 7f41fe800640 1 -- 172.21.15.83:0/149426477 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557ce942d270 msgr2=0x557ce9438bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:51.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.526+0000 7f41fe800640 1 --2- 172.21.15.83:0/149426477 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557ce942d270 0x557ce9438bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:51.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.526+0000 7f41fe800640 1 -- 172.21.15.83:0/149426477 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557ce942c950 msgr2=0x557ce9438670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:51.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.526+0000 7f41fe800640 1 --2- 172.21.15.83:0/149426477 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557ce942c950 0x557ce9438670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:51.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.526+0000 7f41fe800640 1 -- 172.21.15.83:0/149426477 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ce943dd50 con 0x557ce9437940 2024-04-24T09:01:51.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.526+0000 7f41fe800640 1 --2- 172.21.15.83:0/149426477 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557ce9437940 0x557ce94390f0 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f41f8012a20 tx=0x7f41f8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e0b9f6701f02605 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:51.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.526+0000 7f41dfe00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:01:51.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.526+0000 7f41dfe00640 1 -- 172.21.15.83:0/149426477 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f8020dc0 con 0x557ce9437940 2024-04-24T09:01:51.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.526+0000 7f4207c10740 1 -- 172.21.15.83:0/149426477 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41f00043c0 con 0x557ce9437940 2024-04-24T09:01:51.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.526+0000 7f41dfe00640 1 -- 172.21.15.83:0/149426477 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7f41f801e440 con 0x557ce9437940 2024-04-24T09:01:51.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.527+0000 7f41dfe00640 1 -- 172.21.15.83:0/149426477 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f41f801e640 con 0x557ce9437940 2024-04-24T09:01:51.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.527+0000 7f4207c10740 10 client.19248 fetch_fsmap learned FSMap version 40 2024-04-24T09:01:51.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.527+0000 7f4207c10740 1 -- 172.21.15.83:0/149426477 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x557ce94ae220 con 0x557ce9437940 2024-04-24T09:01:51.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.527+0000 7f41dfe00640 1 -- 172.21.15.83:0/149426477 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f41f00043c0 con 0x557ce9437940 2024-04-24T09:01:51.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.527+0000 7f41dfe00640 1 -- 172.21.15.83:0/149426477 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 40) ==== 8382+0+0 (secure 0 0 0) 0x7f41f8027070 con 0x557ce9437940 2024-04-24T09:01:51.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.527+0000 7f4207c10740 10 client.19248 fetch_fsmap finished waiting for FSMap version 40 2024-04-24T09:01:51.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.527+0000 7f4207c10740 10 client.19248 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-24T09:01:51.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.527+0000 7f4207c10740 20 client.19248 populate_metadata read hostname 'smithi083' 2024-04-24T09:01:51.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.527+0000 7f4207c10740 1 --2- 172.21.15.83:0/149426477 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x557ce94ae970 0x557ce94b0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:51.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.527+0000 7f4207c10740 4 client.19248 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:01:51.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.527+0000 7f4207c10740 1 -- 172.21.15.83:0/149426477 --> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557ce9399a60 con 0x557ce94ae970 2024-04-24T09:01:51.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.528+0000 7f41fde00640 1 --2- 172.21.15.83:0/149426477 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x557ce94ae970 0x557ce94b0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:51.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.529+0000 7f41fde00640 1 --2- 172.21.15.83:0/149426477 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x557ce94ae970 0x557ce94b0d90 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=dea1a2cf910cf4fb server_cookie=5b7a15b46a67081b in_seq=0 out_seq=0 2024-04-24T09:01:51.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.529+0000 7f41dfe00640 10 client.19248 ms_handle_connect on v2:172.21.15.189:6838/2548965336 2024-04-24T09:01:51.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.529+0000 7f41dfe00640 1 -- 172.21.15.83:0/149426477 <== mds.4 v2:172.21.15.189:6838/2548965336 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557ce9399a60 con 0x557ce94ae970 2024-04-24T09:01:51.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.529+0000 7f41dfe00640 10 client.19248 handle_command_reply: tid=0 2024-04-24T09:01:51.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.529+0000 7f4207c10740 1 client.19248 shutdown 2024-04-24T09:01:51.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.529+0000 7f4207c10740 1 -- 172.21.15.83:0/149426477 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557ce9437940 msgr2=0x557ce94390f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:51.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.529+0000 7f4207c10740 1 --2- 172.21.15.83:0/149426477 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557ce9437940 0x557ce94390f0 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f41f8012a20 tx=0x7f41f8004a40 comp rx=0 tx=0).stop 2024-04-24T09:01:51.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.529+0000 7f4207c10740 1 -- 172.21.15.83:0/149426477 shutdown_connections 2024-04-24T09:01:51.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.529+0000 7f4207c10740 1 --2- 172.21.15.83:0/149426477 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x557ce94ae970 0x557ce94b0d90 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:51.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.529+0000 7f4207c10740 1 --2- 172.21.15.83:0/149426477 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557ce9437940 0x557ce94390f0 unknown :-1 s=CLOSED pgs=1696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:51.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.529+0000 7f4207c10740 1 --2- 172.21.15.83:0/149426477 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557ce942d270 0x557ce9438bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:51.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.529+0000 7f4207c10740 1 --2- 172.21.15.83:0/149426477 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557ce942c950 0x557ce9438670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:51.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.529+0000 7f41dfe00640 0 client.19248 ms_handle_reset on v2:172.21.15.189:6838/2548965336 2024-04-24T09:01:51.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.529+0000 7f4207c10740 1 -- 172.21.15.83:0/149426477 >> 172.21.15.83:0/149426477 conn(0x557ce9428c70 msgr2=0x557ce9427560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:51.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.530+0000 7f4207c10740 1 -- 172.21.15.83:0/149426477 shutdown_connections 2024-04-24T09:01:51.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.530+0000 7f4207c10740 1 -- 172.21.15.83:0/149426477 wait complete. 2024-04-24T09:01:51.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.530+0000 7f4207c10740 20 client.19248 trim_cache size 0 max 16384 2024-04-24T09:01:51.532 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:01:51.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.530+0000 7f4204e00640 1 -- 172.21.15.83:0/2251931290 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f41e40a3d20 msgr2=0x7f41e40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:51.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.530+0000 7f4204e00640 1 --2- 172.21.15.83:0/2251931290 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f41e40a3d20 0x7f41e40a61e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f420006b790 tx=0x7f41f8009530 comp rx=0 tx=0).stop 2024-04-24T09:01:51.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.530+0000 7f4204e00640 1 -- 172.21.15.83:0/2251931290 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4200101720 msgr2=0x7f42001a0c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:51.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.530+0000 7f4204e00640 1 --2- 172.21.15.83:0/2251931290 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4200101720 0x7f42001a0c20 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7f41ec005940 tx=0x7f41ec0042f0 comp rx=0 tx=0).stop 2024-04-24T09:01:51.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.531+0000 7f4204e00640 1 -- 172.21.15.83:0/2251931290 shutdown_connections 2024-04-24T09:01:51.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.531+0000 7f4204e00640 1 --2- 172.21.15.83:0/2251931290 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f41e40a3d20 0x7f41e40a61e0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:51.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.531+0000 7f4204e00640 1 --2- 172.21.15.83:0/2251931290 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f420010e9b0 0x7f420019a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:51.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.531+0000 7f4204e00640 1 --2- 172.21.15.83:0/2251931290 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4200101720 0x7f42001a0c20 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:51.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.531+0000 7f4204e00640 1 --2- 172.21.15.83:0/2251931290 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4200100e00 0x7f42001a06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:51.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.531+0000 7f4204e00640 1 -- 172.21.15.83:0/2251931290 >> 172.21.15.83:0/2251931290 conn(0x7f4200078000 msgr2=0x7f42001101a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:51.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.531+0000 7f4204e00640 1 -- 172.21.15.83:0/2251931290 shutdown_connections 2024-04-24T09:01:51.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.531+0000 7f4204e00640 1 -- 172.21.15.83:0/2251931290 wait complete. 2024-04-24T09:01:51.549 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-24T09:01:51.609 INFO:tasks.workunit.client.0.smithi083.stdout: 1 59750 0.63 MB/sec execute 385 sec latency 52.582 ms 2024-04-24T09:01:51.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.766+0000 7ff1ef800640 1 -- 172.21.15.83:0/3661296275 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff1e8108610 msgr2=0x7ff1e8108a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:51.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.766+0000 7ff1ef800640 1 --2- 172.21.15.83:0/3661296275 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff1e8108610 0x7ff1e8108a90 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7ff1d4009930 tx=0x7ff1d402a420 comp rx=0 tx=0).stop 2024-04-24T09:01:51.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.766+0000 7ff1ef800640 1 -- 172.21.15.83:0/3661296275 shutdown_connections 2024-04-24T09:01:51.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.766+0000 7ff1ef800640 1 --2- 172.21.15.83:0/3661296275 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1e8108fd0 0x7ff1e8115490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:51.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.766+0000 7ff1ef800640 1 --2- 172.21.15.83:0/3661296275 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff1e8108610 0x7ff1e8108a90 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:51.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.766+0000 7ff1ef800640 1 --2- 172.21.15.83:0/3661296275 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff1e810e660 0x7ff1e810ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:51.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.766+0000 7ff1ef800640 1 -- 172.21.15.83:0/3661296275 >> 172.21.15.83:0/3661296275 conn(0x7ff1e8076540 msgr2=0x7ff1e8078960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:51.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.766+0000 7ff1ef800640 1 -- 172.21.15.83:0/3661296275 shutdown_connections 2024-04-24T09:01:51.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.766+0000 7ff1ef800640 1 -- 172.21.15.83:0/3661296275 wait complete. 2024-04-24T09:01:51.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.767+0000 7ff1ef800640 1 Processor -- start 2024-04-24T09:01:51.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.767+0000 7ff1ef800640 1 -- start start 2024-04-24T09:01:51.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.767+0000 7ff1ef800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1e8108610 0x7ff1e819efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:51.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.767+0000 7ff1ef800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff1e8108fd0 0x7ff1e819f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:51.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.767+0000 7ff1ef800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff1e810e660 0x7ff1e81a3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:51.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.767+0000 7ff1ef800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff1e8118980 con 0x7ff1e8108fd0 2024-04-24T09:01:51.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.767+0000 7ff1ef800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff1e8118800 con 0x7ff1e8108610 2024-04-24T09:01:51.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.767+0000 7ff1ef800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff1e8118b00 con 0x7ff1e810e660 2024-04-24T09:01:51.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.768+0000 7ff1dfe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff1e8108fd0 0x7ff1e819f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:51.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.768+0000 7ff1dfe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff1e8108fd0 0x7ff1e819f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45556/0 (socket says 172.21.15.83:45556) 2024-04-24T09:01:51.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.768+0000 7ff1dfe00640 1 -- 172.21.15.83:0/1026513079 learned_addr learned my addr 172.21.15.83:0/1026513079 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:01:51.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.768+0000 7ff1ece00640 1 --2- 172.21.15.83:0/1026513079 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1e8108610 0x7ff1e819efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:51.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.768+0000 7ff1dfe00640 1 -- 172.21.15.83:0/1026513079 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff1e810e660 msgr2=0x7ff1e81a3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:51.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.768+0000 7ff1dfe00640 1 --2- 172.21.15.83:0/1026513079 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff1e810e660 0x7ff1e81a3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:51.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.768+0000 7ff1dfe00640 1 -- 172.21.15.83:0/1026513079 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1e8108610 msgr2=0x7ff1e819efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:51.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.768+0000 7ff1dfe00640 1 --2- 172.21.15.83:0/1026513079 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1e8108610 0x7ff1e819efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:51.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.768+0000 7ff1dfe00640 1 -- 172.21.15.83:0/1026513079 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1e81a3f90 con 0x7ff1e8108fd0 2024-04-24T09:01:51.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.768+0000 7ff1ece00640 1 --2- 172.21.15.83:0/1026513079 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1e8108610 0x7ff1e819efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:01:51.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.768+0000 7ff1dfe00640 1 --2- 172.21.15.83:0/1026513079 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff1e8108fd0 0x7ff1e819f4e0 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7ff1d4031440 tx=0x7ff1d4003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:51.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.769+0000 7ff1e6000640 1 -- 172.21.15.83:0/1026513079 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1d403f070 con 0x7ff1e8108fd0 2024-04-24T09:01:51.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.769+0000 7ff1e6000640 1 -- 172.21.15.83:0/1026513079 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff1d4031d60 con 0x7ff1e8108fd0 2024-04-24T09:01:51.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.769+0000 7ff1e6000640 1 -- 172.21.15.83:0/1026513079 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1d4035030 con 0x7ff1e8108fd0 2024-04-24T09:01:51.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.769+0000 7ff1ef800640 1 -- 172.21.15.83:0/1026513079 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1e81a4220 con 0x7ff1e8108fd0 2024-04-24T09:01:51.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.769+0000 7ff1ef800640 1 -- 172.21.15.83:0/1026513079 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1e81abb10 con 0x7ff1e8108fd0 2024-04-24T09:01:51.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.772+0000 7ff1e6000640 1 -- 172.21.15.83:0/1026513079 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff1d4042070 con 0x7ff1e8108fd0 2024-04-24T09:01:51.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.772+0000 7ff1f2684740 1 Processor -- start 2024-04-24T09:01:51.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.772+0000 7ff1f2684740 1 -- start start 2024-04-24T09:01:51.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.773+0000 7ff1f2684740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55badd6c5950 0x55badd6c5d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:51.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.773+0000 7ff1f2684740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55badd6c6270 0x55badd6d0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:51.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.773+0000 7ff1f2684740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55badd6d0940 0x55badd6d2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:51.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.773+0000 7ff1f2684740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55badd62b1d0 con 0x55badd6c5950 2024-04-24T09:01:51.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.773+0000 7ff1f2684740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55badd5d1450 con 0x55badd6d0940 2024-04-24T09:01:51.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.773+0000 7ff1f2684740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55badd627190 con 0x55badd6c6270 2024-04-24T09:01:51.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.773+0000 7ff1ed800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55badd6c5950 0x55badd6c5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:51.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.773+0000 7ff1ed800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55badd6c5950 0x55badd6c5d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45566/0 (socket says 172.21.15.83:45566) 2024-04-24T09:01:51.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.773+0000 7ff1ed800640 1 -- 172.21.15.83:0/960778130 learned_addr learned my addr 172.21.15.83:0/960778130 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:01:51.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.773+0000 7ff1ece00640 1 --2- 172.21.15.83:0/960778130 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55badd6c6270 0x55badd6d0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:51.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.773+0000 7ff1dfe00640 1 --2- 172.21.15.83:0/960778130 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55badd6d0940 0x55badd6d2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:51.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.774+0000 7ff1ed800640 1 -- 172.21.15.83:0/960778130 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55badd6c6270 msgr2=0x55badd6d0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:51.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.774+0000 7ff1ed800640 1 --2- 172.21.15.83:0/960778130 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55badd6c6270 0x55badd6d0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:51.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.774+0000 7ff1ed800640 1 -- 172.21.15.83:0/960778130 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55badd6d0940 msgr2=0x55badd6d2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:51.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.774+0000 7ff1ed800640 1 --2- 172.21.15.83:0/960778130 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55badd6d0940 0x55badd6d2d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:51.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.774+0000 7ff1ed800640 1 -- 172.21.15.83:0/960778130 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55badd68e310 con 0x55badd6c5950 2024-04-24T09:01:51.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.774+0000 7ff1dfe00640 1 --2- 172.21.15.83:0/960778130 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55badd6d0940 0x55badd6d2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:01:51.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.774+0000 7ff1ed800640 1 --2- 172.21.15.83:0/960778130 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55badd6c5950 0x55badd6c5d30 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7ff1e0009d00 tx=0x7ff1e000b420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41196d8e546a9e6b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:51.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.774+0000 7ff1e6000640 1 --2- 172.21.15.83:0/1026513079 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff1c40a3da0 0x7ff1c40a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:51.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.774+0000 7ff1de000640 1 -- 172.21.15.83:0/960778130 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e000bd40 con 0x55badd6c5950 2024-04-24T09:01:51.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.774+0000 7ff1de000640 1 -- 172.21.15.83:0/960778130 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff1e0017070 con 0x55badd6c5950 2024-04-24T09:01:51.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.774+0000 7ff1de000640 1 -- 172.21.15.83:0/960778130 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e00115f0 con 0x55badd6c5950 2024-04-24T09:01:51.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.774+0000 7ff1e6000640 1 -- 172.21.15.83:0/1026513079 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7ff1d40f1550 con 0x7ff1e8108fd0 2024-04-24T09:01:51.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.775+0000 7ff1ece00640 1 --2- 172.21.15.83:0/1026513079 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff1c40a3da0 0x7ff1c40a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:51.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.775+0000 7ff1f2684740 1 -- 172.21.15.83:0/960778130 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55badd6c5950 msgr2=0x55badd6c5d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:51.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.775+0000 7ff1f2684740 1 --2- 172.21.15.83:0/960778130 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55badd6c5950 0x55badd6c5d30 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7ff1e0009d00 tx=0x7ff1e000b420 comp rx=0 tx=0).stop 2024-04-24T09:01:51.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.775+0000 7ff1ece00640 1 --2- 172.21.15.83:0/1026513079 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff1c40a3da0 0x7ff1c40a6260 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x55badd6c6f10 tx=0x7ff1cc008040 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:51.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.776+0000 7ff1f2684740 1 -- 172.21.15.83:0/960778130 shutdown_connections 2024-04-24T09:01:51.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.776+0000 7ff1f2684740 1 --2- 172.21.15.83:0/960778130 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55badd6d0940 0x55badd6d2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:51.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.776+0000 7ff1f2684740 1 --2- 172.21.15.83:0/960778130 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55badd6c6270 0x55badd6d0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:51.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.776+0000 7ff1f2684740 1 --2- 172.21.15.83:0/960778130 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55badd6c5950 0x55badd6c5d30 unknown :-1 s=CLOSED pgs=1699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:51.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.776+0000 7ff1f2684740 1 -- 172.21.15.83:0/960778130 >> 172.21.15.83:0/960778130 conn(0x55badd6c1c70 msgr2=0x55badd6c2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:51.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.776+0000 7ff1f2684740 1 -- 172.21.15.83:0/960778130 shutdown_connections 2024-04-24T09:01:51.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.776+0000 7ff1f2684740 1 -- 172.21.15.83:0/960778130 wait complete. 2024-04-24T09:01:51.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.776+0000 7ff1f2684740 1 -- start start 2024-04-24T09:01:51.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.776+0000 7ff1f2684740 1 Processor -- start 2024-04-24T09:01:51.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.777+0000 7ff1f2684740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55badd6c5950 0x55badd6d03d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:51.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.777+0000 7ff1f2684740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55badd6c6270 0x55badd6cd6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:51.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.777+0000 7ff1f2684740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55badd6d0940 0x55badd6cddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:51.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.777+0000 7ff1f2684740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55badd6d37d0 con 0x55badd6d0940 2024-04-24T09:01:51.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.777+0000 7ff1f2684740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55badd5e4810 con 0x55badd6c6270 2024-04-24T09:01:51.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.777+0000 7ff1f2684740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55badd6d3950 con 0x55badd6c5950 2024-04-24T09:01:51.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.777+0000 7ff1f2684740 10 client.0 _ll_register_callbacks cb 0x55badd5b2140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:01:51.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.777+0000 7ff1ed800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55badd6d0940 0x55badd6cddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:51.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.777+0000 7ff1ed800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55badd6d0940 0x55badd6cddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45570/0 (socket says 172.21.15.83:45570) 2024-04-24T09:01:51.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.777+0000 7ff1ed800640 1 -- 172.21.15.83:0/1910243745 learned_addr learned my addr 172.21.15.83:0/1910243745 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:01:51.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.777+0000 7ff1dfe00640 1 --2- 172.21.15.83:0/1910243745 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55badd6c6270 0x55badd6cd6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:51.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.777+0000 7ff1ece00640 1 --2- 172.21.15.83:0/1910243745 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55badd6c5950 0x55badd6d03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:51.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.777+0000 7ff1ed800640 1 -- 172.21.15.83:0/1910243745 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55badd6c5950 msgr2=0x55badd6d03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:51.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.778+0000 7ff1ed800640 1 --2- 172.21.15.83:0/1910243745 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55badd6c5950 0x55badd6d03d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:51.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.778+0000 7ff1ed800640 1 -- 172.21.15.83:0/1910243745 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55badd6c6270 msgr2=0x55badd6cd6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:51.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.778+0000 7ff1ed800640 1 --2- 172.21.15.83:0/1910243745 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55badd6c6270 0x55badd6cd6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:51.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.778+0000 7ff1ed800640 1 -- 172.21.15.83:0/1910243745 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55badd5f53f0 con 0x55badd6d0940 2024-04-24T09:01:51.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.778+0000 7ff1ed800640 1 --2- 172.21.15.83:0/1910243745 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55badd6d0940 0x55badd6cddb0 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7ff1e000d010 tx=0x7ff1e00124b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62cb44ea930798de server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:51.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.778+0000 7ff1dae00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:01:51.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.778+0000 7ff1dae00640 1 -- 172.21.15.83:0/1910243745 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e0021070 con 0x55badd6d0940 2024-04-24T09:01:51.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.778+0000 7ff1dae00640 1 -- 172.21.15.83:0/1910243745 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7ff1e001b3f0 con 0x55badd6d0940 2024-04-24T09:01:51.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.778+0000 7ff1f2684740 1 -- 172.21.15.83:0/1910243745 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1e0017070 con 0x55badd6d0940 2024-04-24T09:01:51.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.779+0000 7ff1dae00640 1 -- 172.21.15.83:0/1910243745 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7ff1e00128b0 con 0x55badd6d0940 2024-04-24T09:01:51.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.779+0000 7ff1f2684740 10 client.19260 fetch_fsmap learned FSMap version 40 2024-04-24T09:01:51.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.779+0000 7ff1f2684740 1 -- 172.21.15.83:0/1910243745 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55badd33d3f0 con 0x55badd6d0940 2024-04-24T09:01:51.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.779+0000 7ff1dae00640 1 -- 172.21.15.83:0/1910243745 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff1e0017070 con 0x55badd6d0940 2024-04-24T09:01:51.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.779+0000 7ff1dae00640 1 -- 172.21.15.83:0/1910243745 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 40) ==== 8382+0+0 (secure 0 0 0) 0x7ff1e0011070 con 0x55badd6d0940 2024-04-24T09:01:51.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.779+0000 7ff1f2684740 10 client.19260 fetch_fsmap finished waiting for FSMap version 40 2024-04-24T09:01:51.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.779+0000 7ff1f2684740 10 client.19260 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-24T09:01:51.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.779+0000 7ff1f2684740 20 client.19260 populate_metadata read hostname 'smithi083' 2024-04-24T09:01:51.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.780+0000 7ff1f2684740 1 --2- 172.21.15.83:0/1910243745 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55badd6e0ac0 0x55badd6e2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:51.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.780+0000 7ff1f2684740 4 client.19260 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:01:51.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.780+0000 7ff1f2684740 1 -- 172.21.15.83:0/1910243745 --> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55badd638bf0 con 0x55badd6e0ac0 2024-04-24T09:01:51.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.780+0000 7ff1ece00640 1 --2- 172.21.15.83:0/1910243745 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55badd6e0ac0 0x55badd6e2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:51.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.781+0000 7ff1ece00640 1 --2- 172.21.15.83:0/1910243745 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55badd6e0ac0 0x55badd6e2eb0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b76ceb69c1f71aa server_cookie=a7a56b871bd9e32d in_seq=0 out_seq=0 2024-04-24T09:01:51.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.781+0000 7ff1dae00640 10 client.19260 ms_handle_connect on v2:172.21.15.138:6832/1157967658 2024-04-24T09:01:51.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.785+0000 7ff1dae00640 1 -- 172.21.15.83:0/1910243745 <== mds.3 v2:172.21.15.138:6832/1157967658 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55badd638bf0 con 0x55badd6e0ac0 2024-04-24T09:01:51.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.785+0000 7ff1dae00640 10 client.19260 handle_command_reply: tid=0 2024-04-24T09:01:51.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.785+0000 7ff1f2684740 1 client.19260 shutdown 2024-04-24T09:01:51.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.786+0000 7ff1f2684740 1 -- 172.21.15.83:0/1910243745 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55badd6d0940 msgr2=0x55badd6cddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:51.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.786+0000 7ff1f2684740 1 --2- 172.21.15.83:0/1910243745 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55badd6d0940 0x55badd6cddb0 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7ff1e000d010 tx=0x7ff1e00124b0 comp rx=0 tx=0).stop 2024-04-24T09:01:51.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.786+0000 7ff1f2684740 1 -- 172.21.15.83:0/1910243745 shutdown_connections 2024-04-24T09:01:51.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.786+0000 7ff1f2684740 1 --2- 172.21.15.83:0/1910243745 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55badd6d0940 0x55badd6cddb0 unknown :-1 s=CLOSED pgs=1700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:51.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.786+0000 7ff1f2684740 1 --2- 172.21.15.83:0/1910243745 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55badd6c6270 0x55badd6cd6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:51.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.786+0000 7ff1f2684740 1 --2- 172.21.15.83:0/1910243745 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55badd6e0ac0 0x55badd6e2eb0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:51.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.786+0000 7ff1f2684740 1 --2- 172.21.15.83:0/1910243745 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55badd6c5950 0x55badd6d03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:51.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.786+0000 7ff1dae00640 0 client.19260 ms_handle_reset on v2:172.21.15.138:6832/1157967658 2024-04-24T09:01:51.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.786+0000 7ff1f2684740 1 -- 172.21.15.83:0/1910243745 >> 172.21.15.83:0/1910243745 conn(0x55badd6c1c70 msgr2=0x55badd6c7a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:51.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.786+0000 7ff1f2684740 1 -- 172.21.15.83:0/1910243745 shutdown_connections 2024-04-24T09:01:51.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.786+0000 7ff1f2684740 1 -- 172.21.15.83:0/1910243745 wait complete. 2024-04-24T09:01:51.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.786+0000 7ff1f2684740 20 client.19260 trim_cache size 0 max 16384 2024-04-24T09:01:51.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.802+0000 7ff1f2684740 1 Processor -- start 2024-04-24T09:01:51.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.803+0000 7ff1f2684740 1 -- start start 2024-04-24T09:01:51.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.803+0000 7ff1f2684740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55badd6c5950 0x55badd6dc690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:51.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.803+0000 7ff1f2684740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55badd6c6270 0x55badd6cc820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:51.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.803+0000 7ff1f2684740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55badd6d0940 0x55badd6cff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:51.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.803+0000 7ff1f2684740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff1e81b5cf0 con 0x55badd6c6270 2024-04-24T09:01:51.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.803+0000 7ff1f2684740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff1e81b7920 con 0x55badd6d0940 2024-04-24T09:01:51.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.803+0000 7ff1f2684740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff1e81b63c0 con 0x55badd6c5950 2024-04-24T09:01:51.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.803+0000 7ff1dfe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55badd6c6270 0x55badd6cc820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:51.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.803+0000 7ff1dfe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55badd6c6270 0x55badd6cc820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45578/0 (socket says 172.21.15.83:45578) 2024-04-24T09:01:51.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.803+0000 7ff1dfe00640 1 -- 172.21.15.83:0/167837829 learned_addr learned my addr 172.21.15.83:0/167837829 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:01:51.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.804+0000 7ff1ece00640 1 --2- 172.21.15.83:0/167837829 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55badd6c5950 0x55badd6dc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:51.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.804+0000 7ff1ed800640 1 --2- 172.21.15.83:0/167837829 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55badd6d0940 0x55badd6cff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:51.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.804+0000 7ff1dfe00640 1 -- 172.21.15.83:0/167837829 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55badd6c5950 msgr2=0x55badd6dc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:51.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.804+0000 7ff1dfe00640 1 --2- 172.21.15.83:0/167837829 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55badd6c5950 0x55badd6dc690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:51.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.804+0000 7ff1dfe00640 1 -- 172.21.15.83:0/167837829 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55badd6d0940 msgr2=0x55badd6cff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:51.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.804+0000 7ff1dfe00640 1 --2- 172.21.15.83:0/167837829 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55badd6d0940 0x55badd6cff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:51.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.804+0000 7ff1dfe00640 1 -- 172.21.15.83:0/167837829 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55badd6c60a0 con 0x55badd6c6270 2024-04-24T09:01:51.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.804+0000 7ff1ed800640 1 --2- 172.21.15.83:0/167837829 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55badd6d0940 0x55badd6cff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:01:51.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.804+0000 7ff1dfe00640 1 --2- 172.21.15.83:0/167837829 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55badd6c6270 0x55badd6cc820 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7ff1d40b9530 tx=0x7ff1d40091c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f9344bbfcbf7afa server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:51.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.804+0000 7ff1dd600640 1 -- 172.21.15.83:0/167837829 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1d4049070 con 0x55badd6c6270 2024-04-24T09:01:51.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.804+0000 7ff1dd600640 1 -- 172.21.15.83:0/167837829 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff1d40c1480 con 0x55badd6c6270 2024-04-24T09:01:51.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.804+0000 7ff1dd600640 1 -- 172.21.15.83:0/167837829 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1d40b98d0 con 0x55badd6c6270 2024-04-24T09:01:51.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.805+0000 7ff1f2684740 1 -- 172.21.15.83:0/167837829 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55badd6c6270 msgr2=0x55badd6cc820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:51.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.805+0000 7ff1f2684740 1 --2- 172.21.15.83:0/167837829 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55badd6c6270 0x55badd6cc820 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7ff1d40b9530 tx=0x7ff1d40091c0 comp rx=0 tx=0).stop 2024-04-24T09:01:51.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.805+0000 7ff1f2684740 1 -- 172.21.15.83:0/167837829 shutdown_connections 2024-04-24T09:01:51.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.805+0000 7ff1f2684740 1 --2- 172.21.15.83:0/167837829 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55badd6d0940 0x55badd6cff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:51.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.805+0000 7ff1f2684740 1 --2- 172.21.15.83:0/167837829 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55badd6c6270 0x55badd6cc820 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:51.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.805+0000 7ff1f2684740 1 --2- 172.21.15.83:0/167837829 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55badd6c5950 0x55badd6dc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:51.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.805+0000 7ff1f2684740 1 -- 172.21.15.83:0/167837829 >> 172.21.15.83:0/167837829 conn(0x55badd6c1c70 msgr2=0x55badd6c70e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:51.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.805+0000 7ff1f2684740 1 -- 172.21.15.83:0/167837829 shutdown_connections 2024-04-24T09:01:51.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.805+0000 7ff1f2684740 1 -- 172.21.15.83:0/167837829 wait complete. 2024-04-24T09:01:51.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.806+0000 7ff1f2684740 1 -- start start 2024-04-24T09:01:51.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.806+0000 7ff1f2684740 1 Processor -- start 2024-04-24T09:01:51.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.806+0000 7ff1f2684740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55badd6c5950 0x55badd6d8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:51.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.806+0000 7ff1f2684740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55badd6c6270 0x55badd6d90f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:51.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.806+0000 7ff1f2684740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55badd6d0940 0x55badd6d3b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:51.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.806+0000 7ff1f2684740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff1e81b71f0 con 0x55badd6d0940 2024-04-24T09:01:51.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.806+0000 7ff1f2684740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff1e81b6ac0 con 0x55badd6c6270 2024-04-24T09:01:51.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.806+0000 7ff1f2684740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff1e81b3cf0 con 0x55badd6c5950 2024-04-24T09:01:51.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.806+0000 7ff1f2684740 10 client.0 _ll_register_callbacks cb 0x7ff1e0009bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:01:51.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.807+0000 7ff1ed800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55badd6d0940 0x55badd6d3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:51.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.807+0000 7ff1ed800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55badd6d0940 0x55badd6d3b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45594/0 (socket says 172.21.15.83:45594) 2024-04-24T09:01:51.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.807+0000 7ff1ed800640 1 -- 172.21.15.83:0/2892336736 learned_addr learned my addr 172.21.15.83:0/2892336736 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:01:51.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.807+0000 7ff1ece00640 1 --2- 172.21.15.83:0/2892336736 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55badd6c5950 0x55badd6d8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:51.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.807+0000 7ff1dfe00640 1 --2- 172.21.15.83:0/2892336736 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55badd6c6270 0x55badd6d90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:51.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.807+0000 7ff1ed800640 1 -- 172.21.15.83:0/2892336736 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55badd6c5950 msgr2=0x55badd6d8bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:51.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.807+0000 7ff1ed800640 1 --2- 172.21.15.83:0/2892336736 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55badd6c5950 0x55badd6d8bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:51.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.807+0000 7ff1ed800640 1 -- 172.21.15.83:0/2892336736 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55badd6c6270 msgr2=0x55badd6d90f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:51.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.807+0000 7ff1ed800640 1 --2- 172.21.15.83:0/2892336736 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55badd6c6270 0x55badd6d90f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:51.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.807+0000 7ff1ed800640 1 -- 172.21.15.83:0/2892336736 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55badd4077d0 con 0x55badd6d0940 2024-04-24T09:01:51.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.807+0000 7ff1ed800640 1 --2- 172.21.15.83:0/2892336736 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55badd6d0940 0x55badd6d3b60 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7ff1e0002230 tx=0x7ff1e00247f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66af6c9c734b2c50 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:51.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.807+0000 7ff1dae00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:01:51.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.808+0000 7ff1dae00640 1 -- 172.21.15.83:0/2892336736 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e0021040 con 0x55badd6d0940 2024-04-24T09:01:51.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.808+0000 7ff1f2684740 1 -- 172.21.15.83:0/2892336736 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1d40c1480 con 0x55badd6d0940 2024-04-24T09:01:51.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.808+0000 7ff1dae00640 1 -- 172.21.15.83:0/2892336736 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7ff1e00248e0 con 0x55badd6d0940 2024-04-24T09:01:51.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.808+0000 7ff1dae00640 1 -- 172.21.15.83:0/2892336736 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7ff1e000b920 con 0x55badd6d0940 2024-04-24T09:01:51.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.808+0000 7ff1f2684740 10 client.19266 fetch_fsmap learned FSMap version 40 2024-04-24T09:01:51.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.808+0000 7ff1f2684740 1 -- 172.21.15.83:0/2892336736 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55badd747620 con 0x55badd6d0940 2024-04-24T09:01:51.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.808+0000 7ff1dae00640 1 -- 172.21.15.83:0/2892336736 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff1d40c1480 con 0x55badd6d0940 2024-04-24T09:01:51.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.809+0000 7ff1dae00640 1 -- 172.21.15.83:0/2892336736 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 40) ==== 8382+0+0 (secure 0 0 0) 0x7ff1e0011040 con 0x55badd6d0940 2024-04-24T09:01:51.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.809+0000 7ff1f2684740 10 client.19266 fetch_fsmap finished waiting for FSMap version 40 2024-04-24T09:01:51.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.809+0000 7ff1f2684740 10 client.19266 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-24T09:01:51.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.809+0000 7ff1f2684740 20 client.19266 populate_metadata read hostname 'smithi083' 2024-04-24T09:01:51.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.809+0000 7ff1f2684740 1 --2- 172.21.15.83:0/2892336736 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55badd747da0 0x55badd74a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:51.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.809+0000 7ff1f2684740 4 client.19266 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:01:51.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.809+0000 7ff1f2684740 1 -- 172.21.15.83:0/2892336736 --> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55badd632a60 con 0x55badd747da0 2024-04-24T09:01:51.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.809+0000 7ff1ece00640 1 --2- 172.21.15.83:0/2892336736 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55badd747da0 0x55badd74a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:51.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.810+0000 7ff1dae00640 10 client.19266 ms_handle_connect on v2:172.21.15.138:6832/1157967658 2024-04-24T09:01:51.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.810+0000 7ff1ece00640 1 --2- 172.21.15.83:0/2892336736 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55badd747da0 0x55badd74a1c0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=733c5ba6b25a7013 server_cookie=7cccf860f65b374c in_seq=0 out_seq=0 2024-04-24T09:01:51.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.811+0000 7ff1dae00640 1 -- 172.21.15.83:0/2892336736 <== mds.3 v2:172.21.15.138:6832/1157967658 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55badd632a60 con 0x55badd747da0 2024-04-24T09:01:51.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.811+0000 7ff1dae00640 10 client.19266 handle_command_reply: tid=0 2024-04-24T09:01:51.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.811+0000 7ff1f2684740 1 client.19266 shutdown 2024-04-24T09:01:51.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.811+0000 7ff1f2684740 1 -- 172.21.15.83:0/2892336736 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55badd6d0940 msgr2=0x55badd6d3b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:51.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.811+0000 7ff1f2684740 1 --2- 172.21.15.83:0/2892336736 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55badd6d0940 0x55badd6d3b60 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7ff1e0002230 tx=0x7ff1e00247f0 comp rx=0 tx=0).stop 2024-04-24T09:01:51.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.811+0000 7ff1f2684740 1 -- 172.21.15.83:0/2892336736 shutdown_connections 2024-04-24T09:01:51.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.811+0000 7ff1f2684740 1 --2- 172.21.15.83:0/2892336736 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55badd6d0940 0x55badd6d3b60 unknown :-1 s=CLOSED pgs=1702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:51.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.811+0000 7ff1f2684740 1 --2- 172.21.15.83:0/2892336736 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55badd6c6270 0x55badd6d90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:51.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.811+0000 7ff1f2684740 1 --2- 172.21.15.83:0/2892336736 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55badd747da0 0x55badd74a1c0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:51.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.811+0000 7ff1f2684740 1 --2- 172.21.15.83:0/2892336736 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55badd6c5950 0x55badd6d8bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:51.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.811+0000 7ff1dae00640 0 client.19266 ms_handle_reset on v2:172.21.15.138:6832/1157967658 2024-04-24T09:01:51.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.811+0000 7ff1f2684740 1 -- 172.21.15.83:0/2892336736 >> 172.21.15.83:0/2892336736 conn(0x55badd6c1c70 msgr2=0x55badd6cc6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:51.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.811+0000 7ff1f2684740 1 -- 172.21.15.83:0/2892336736 shutdown_connections 2024-04-24T09:01:51.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.811+0000 7ff1f2684740 1 -- 172.21.15.83:0/2892336736 wait complete. 2024-04-24T09:01:51.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.811+0000 7ff1f2684740 20 client.19266 trim_cache size 0 max 16384 2024-04-24T09:01:51.815 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:01:51.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.812+0000 7ff1ef800640 1 -- 172.21.15.83:0/1026513079 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff1c40a3da0 msgr2=0x7ff1c40a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:51.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.812+0000 7ff1ef800640 1 --2- 172.21.15.83:0/1026513079 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff1c40a3da0 0x7ff1c40a6260 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x55badd6c6f10 tx=0x7ff1cc008040 comp rx=0 tx=0).stop 2024-04-24T09:01:51.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.812+0000 7ff1ef800640 1 -- 172.21.15.83:0/1026513079 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff1e8108fd0 msgr2=0x7ff1e819f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:51.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.812+0000 7ff1ef800640 1 --2- 172.21.15.83:0/1026513079 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff1e8108fd0 0x7ff1e819f4e0 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7ff1d4031440 tx=0x7ff1d4003d50 comp rx=0 tx=0).stop 2024-04-24T09:01:51.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.812+0000 7ff1ef800640 1 -- 172.21.15.83:0/1026513079 shutdown_connections 2024-04-24T09:01:51.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.812+0000 7ff1ef800640 1 --2- 172.21.15.83:0/1026513079 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff1c40a3da0 0x7ff1c40a6260 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:51.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.812+0000 7ff1ef800640 1 --2- 172.21.15.83:0/1026513079 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff1e810e660 0x7ff1e81a3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:51.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.812+0000 7ff1ef800640 1 --2- 172.21.15.83:0/1026513079 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff1e8108fd0 0x7ff1e819f4e0 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:51.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.812+0000 7ff1ef800640 1 --2- 172.21.15.83:0/1026513079 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1e8108610 0x7ff1e819efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:51.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.812+0000 7ff1ef800640 1 -- 172.21.15.83:0/1026513079 >> 172.21.15.83:0/1026513079 conn(0x7ff1e8076540 msgr2=0x7ff1e8077d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:51.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.812+0000 7ff1ef800640 1 -- 172.21.15.83:0/1026513079 shutdown_connections 2024-04-24T09:01:51.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:51.812+0000 7ff1ef800640 1 -- 172.21.15.83:0/1026513079 wait complete. 2024-04-24T09:01:51.830 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T09:01:52.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.037+0000 7ffbd3000640 1 -- 172.21.15.83:0/4081481261 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffbcc1093f0 msgr2=0x7ffbcc110fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:52.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.037+0000 7ffbd3000640 1 --2- 172.21.15.83:0/4081481261 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffbcc1093f0 0x7ffbcc110fc0 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7ffbc4009900 tx=0x7ffbc402a410 comp rx=0 tx=0).stop 2024-04-24T09:01:52.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.037+0000 7ffbd3000640 1 -- 172.21.15.83:0/4081481261 shutdown_connections 2024-04-24T09:01:52.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.037+0000 7ffbd3000640 1 --2- 172.21.15.83:0/4081481261 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffbcc1093f0 0x7ffbcc110fc0 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:52.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.037+0000 7ffbd3000640 1 --2- 172.21.15.83:0/4081481261 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffbcc104cd0 0x7ffbcc108d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:52.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.037+0000 7ffbd3000640 1 --2- 172.21.15.83:0/4081481261 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffbcc104320 0x7ffbcc104700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:52.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.037+0000 7ffbd3000640 1 -- 172.21.15.83:0/4081481261 >> 172.21.15.83:0/4081481261 conn(0x7ffbcc0ffb40 msgr2=0x7ffbcc101f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:52.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.038+0000 7ffbd3000640 1 -- 172.21.15.83:0/4081481261 shutdown_connections 2024-04-24T09:01:52.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.038+0000 7ffbd3000640 1 -- 172.21.15.83:0/4081481261 wait complete. 2024-04-24T09:01:52.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.039+0000 7ffbd3000640 1 Processor -- start 2024-04-24T09:01:52.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.039+0000 7ffbd3000640 1 -- start start 2024-04-24T09:01:52.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.039+0000 7ffbd3000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffbcc104320 0x7ffbcc1a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:52.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.039+0000 7ffbd3000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffbcc1093f0 0x7ffbcc1a0b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:52.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.039+0000 7ffbd3000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffbcc19a6b0 0x7ffbcc19ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:52.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.039+0000 7ffbd3000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ffbcc1143b0 con 0x7ffbcc1093f0 2024-04-24T09:01:52.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.039+0000 7ffbd3000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ffbcc114230 con 0x7ffbcc19a6b0 2024-04-24T09:01:52.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.039+0000 7ffbd3000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ffbcc114530 con 0x7ffbcc104320 2024-04-24T09:01:52.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.039+0000 7ffbd1200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffbcc1093f0 0x7ffbcc1a0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:52.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.039+0000 7ffbd1200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffbcc1093f0 0x7ffbcc1a0b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45612/0 (socket says 172.21.15.83:45612) 2024-04-24T09:01:52.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.039+0000 7ffbd1200640 1 -- 172.21.15.83:0/2016996018 learned_addr learned my addr 172.21.15.83:0/2016996018 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:01:52.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.040+0000 7ffbd1c00640 1 --2- 172.21.15.83:0/2016996018 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffbcc104320 0x7ffbcc1a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:52.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.040+0000 7ffbd2600640 1 --2- 172.21.15.83:0/2016996018 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffbcc19a6b0 0x7ffbcc19ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:52.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.040+0000 7ffbd1200640 1 -- 172.21.15.83:0/2016996018 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffbcc104320 msgr2=0x7ffbcc1a05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:52.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.040+0000 7ffbd1200640 1 --2- 172.21.15.83:0/2016996018 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffbcc104320 0x7ffbcc1a05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:52.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.040+0000 7ffbd1200640 1 -- 172.21.15.83:0/2016996018 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffbcc19a6b0 msgr2=0x7ffbcc19ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:52.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.040+0000 7ffbd1200640 1 --2- 172.21.15.83:0/2016996018 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffbcc19a6b0 0x7ffbcc19ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:52.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.040+0000 7ffbd1200640 1 -- 172.21.15.83:0/2016996018 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbcc19b2b0 con 0x7ffbcc1093f0 2024-04-24T09:01:52.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.040+0000 7ffbd1200640 1 --2- 172.21.15.83:0/2016996018 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffbcc1093f0 0x7ffbcc1a0b30 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7ffbb800d740 tx=0x7ffbb800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:52.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.040+0000 7ffbca000640 1 -- 172.21.15.83:0/2016996018 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbb8017070 con 0x7ffbcc1093f0 2024-04-24T09:01:52.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.041+0000 7ffbd3000640 1 -- 172.21.15.83:0/2016996018 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbcc1a7390 con 0x7ffbcc1093f0 2024-04-24T09:01:52.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.041+0000 7ffbca000640 1 -- 172.21.15.83:0/2016996018 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffbb800de50 con 0x7ffbcc1093f0 2024-04-24T09:01:52.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.041+0000 7ffbca000640 1 -- 172.21.15.83:0/2016996018 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbb8020060 con 0x7ffbcc1093f0 2024-04-24T09:01:52.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.041+0000 7ffbd3000640 1 -- 172.21.15.83:0/2016996018 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbcc1a7790 con 0x7ffbcc1093f0 2024-04-24T09:01:52.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.044+0000 7ffbd7422740 1 Processor -- start 2024-04-24T09:01:52.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.044+0000 7ffbd7422740 1 -- start start 2024-04-24T09:01:52.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.045+0000 7ffbca000640 1 -- 172.21.15.83:0/2016996018 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ffbb8004d00 con 0x7ffbcc1093f0 2024-04-24T09:01:52.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.045+0000 7ffbd7422740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aa29c2cb40 0x55aa29c2cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:52.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.045+0000 7ffbd7422740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aa29c2d460 0x55aa29c375f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:52.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.045+0000 7ffbd7422740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aa29c37b30 0x55aa29c39f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:52.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.045+0000 7ffbd7422740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55aa29b921d0 con 0x55aa29c2cb40 2024-04-24T09:01:52.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.045+0000 7ffbd7422740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55aa29b38450 con 0x55aa29c2d460 2024-04-24T09:01:52.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.045+0000 7ffbd7422740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55aa29b8e190 con 0x55aa29c37b30 2024-04-24T09:01:52.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.045+0000 7ffbd2600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aa29c2cb40 0x55aa29c2cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:52.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.045+0000 7ffbd2600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aa29c2cb40 0x55aa29c2cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45622/0 (socket says 172.21.15.83:45622) 2024-04-24T09:01:52.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.045+0000 7ffbd2600640 1 -- 172.21.15.83:0/4254422231 learned_addr learned my addr 172.21.15.83:0/4254422231 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:01:52.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.046+0000 7ffbd1c00640 1 --2- 172.21.15.83:0/4254422231 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aa29c2d460 0x55aa29c375f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:52.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.046+0000 7ffbd1200640 1 --2- 172.21.15.83:0/4254422231 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aa29c37b30 0x55aa29c39f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:52.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.046+0000 7ffbd2600640 1 -- 172.21.15.83:0/4254422231 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aa29c37b30 msgr2=0x55aa29c39f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:52.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.046+0000 7ffbd2600640 1 --2- 172.21.15.83:0/4254422231 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aa29c37b30 0x55aa29c39f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:52.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.046+0000 7ffbd2600640 1 -- 172.21.15.83:0/4254422231 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aa29c2d460 msgr2=0x55aa29c375f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:52.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.046+0000 7ffbd2600640 1 --2- 172.21.15.83:0/4254422231 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aa29c2d460 0x55aa29c375f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:52.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.046+0000 7ffbd2600640 1 -- 172.21.15.83:0/4254422231 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa29bf5310 con 0x55aa29c2cb40 2024-04-24T09:01:52.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.046+0000 7ffbd1200640 1 --2- 172.21.15.83:0/4254422231 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aa29c37b30 0x55aa29c39f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:01:52.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.047+0000 7ffbd2600640 1 --2- 172.21.15.83:0/4254422231 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aa29c2cb40 0x55aa29c2cf20 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7ffbc4009900 tx=0x7ffbc4009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3f8e335423881b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:52.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.047+0000 7ffbca000640 1 --2- 172.21.15.83:0/2016996018 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ffbb40a3d20 0x7ffbb40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:52.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.047+0000 7ffbbea00640 1 -- 172.21.15.83:0/4254422231 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc4035030 con 0x55aa29c2cb40 2024-04-24T09:01:52.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.047+0000 7ffbbea00640 1 -- 172.21.15.83:0/4254422231 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffbc40054f0 con 0x55aa29c2cb40 2024-04-24T09:01:52.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.047+0000 7ffbbea00640 1 -- 172.21.15.83:0/4254422231 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc40035f0 con 0x55aa29c2cb40 2024-04-24T09:01:52.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.047+0000 7ffbca000640 1 -- 172.21.15.83:0/2016996018 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7ffbb80d0720 con 0x7ffbcc1093f0 2024-04-24T09:01:52.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.047+0000 7ffbd1c00640 1 --2- 172.21.15.83:0/2016996018 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ffbb40a3d20 0x7ffbb40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:52.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.048+0000 7ffbd7422740 1 -- 172.21.15.83:0/4254422231 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aa29c2cb40 msgr2=0x55aa29c2cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:52.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.048+0000 7ffbd1c00640 1 --2- 172.21.15.83:0/2016996018 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ffbb40a3d20 0x7ffbb40a61e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x55aa29c2e100 tx=0x7ffbc00097c0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:52.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.048+0000 7ffbd7422740 1 --2- 172.21.15.83:0/4254422231 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aa29c2cb40 0x55aa29c2cf20 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7ffbc4009900 tx=0x7ffbc4009680 comp rx=0 tx=0).stop 2024-04-24T09:01:52.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.048+0000 7ffbd7422740 1 -- 172.21.15.83:0/4254422231 shutdown_connections 2024-04-24T09:01:52.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.048+0000 7ffbd7422740 1 --2- 172.21.15.83:0/4254422231 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aa29c37b30 0x55aa29c39f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:52.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.048+0000 7ffbd7422740 1 --2- 172.21.15.83:0/4254422231 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aa29c2d460 0x55aa29c375f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:52.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.048+0000 7ffbd7422740 1 --2- 172.21.15.83:0/4254422231 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aa29c2cb40 0x55aa29c2cf20 unknown :-1 s=CLOSED pgs=1705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:52.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.048+0000 7ffbd7422740 1 -- 172.21.15.83:0/4254422231 >> 172.21.15.83:0/4254422231 conn(0x55aa29c28e60 msgr2=0x55aa29c29240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:52.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.048+0000 7ffbd7422740 1 -- 172.21.15.83:0/4254422231 shutdown_connections 2024-04-24T09:01:52.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.048+0000 7ffbd7422740 1 -- 172.21.15.83:0/4254422231 wait complete. 2024-04-24T09:01:52.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.049+0000 7ffbd7422740 1 -- start start 2024-04-24T09:01:52.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.049+0000 7ffbd7422740 1 Processor -- start 2024-04-24T09:01:52.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.049+0000 7ffbd7422740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aa29c2cb40 0x55aa29c375d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:52.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.049+0000 7ffbd7422740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aa29c2d460 0x55aa29c348c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:52.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.049+0000 7ffbd7422740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aa29c37b30 0x55aa29c34fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:52.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.049+0000 7ffbd7422740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55aa29c3a960 con 0x55aa29c2d460 2024-04-24T09:01:52.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.050+0000 7ffbd7422740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55aa29b4b810 con 0x55aa29c2cb40 2024-04-24T09:01:52.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.050+0000 7ffbd7422740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55aa29c3aae0 con 0x55aa29c37b30 2024-04-24T09:01:52.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.050+0000 7ffbd7422740 10 client.0 _ll_register_callbacks cb 0x55aa29b19140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:01:52.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.050+0000 7ffbd1200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aa29c2d460 0x55aa29c348c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:52.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.050+0000 7ffbd1200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aa29c2d460 0x55aa29c348c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45634/0 (socket says 172.21.15.83:45634) 2024-04-24T09:01:52.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.050+0000 7ffbd1200640 1 -- 172.21.15.83:0/3357858050 learned_addr learned my addr 172.21.15.83:0/3357858050 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:01:52.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.050+0000 7ffbd1c00640 1 --2- 172.21.15.83:0/3357858050 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aa29c2cb40 0x55aa29c375d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:52.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.050+0000 7ffbd1200640 1 -- 172.21.15.83:0/3357858050 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aa29c37b30 msgr2=0x55aa29c34fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:52.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.050+0000 7ffbd1200640 1 --2- 172.21.15.83:0/3357858050 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aa29c37b30 0x55aa29c34fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:52.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.050+0000 7ffbd1200640 1 -- 172.21.15.83:0/3357858050 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aa29c2cb40 msgr2=0x55aa29c375d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:52.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.050+0000 7ffbd1200640 1 --2- 172.21.15.83:0/3357858050 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aa29c2cb40 0x55aa29c375d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:52.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.050+0000 7ffbd1200640 1 -- 172.21.15.83:0/3357858050 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa29b5c3f0 con 0x55aa29c2d460 2024-04-24T09:01:52.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.050+0000 7ffbd1200640 1 --2- 172.21.15.83:0/3357858050 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aa29c2d460 0x55aa29c348c0 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7ffbb809c230 tx=0x7ffbb809c160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5d2df11371ddb61 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:52.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.050+0000 7ffbb3400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:01:52.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.051+0000 7ffbb3400640 1 -- 172.21.15.83:0/3357858050 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbb80046c0 con 0x55aa29c2d460 2024-04-24T09:01:52.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.051+0000 7ffbb3400640 1 -- 172.21.15.83:0/3357858050 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7ffbb80b56d0 con 0x55aa29c2d460 2024-04-24T09:01:52.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.051+0000 7ffbd7422740 1 -- 172.21.15.83:0/3357858050 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffbc40054f0 con 0x55aa29c2d460 2024-04-24T09:01:52.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.051+0000 7ffbb3400640 1 -- 172.21.15.83:0/3357858050 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7ffbb80ba050 con 0x55aa29c2d460 2024-04-24T09:01:52.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.051+0000 7ffbd7422740 10 client.19278 fetch_fsmap learned FSMap version 40 2024-04-24T09:01:52.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.051+0000 7ffbd7422740 1 -- 172.21.15.83:0/3357858050 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55aa298a43f0 con 0x55aa29c2d460 2024-04-24T09:01:52.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.051+0000 7ffbb3400640 1 -- 172.21.15.83:0/3357858050 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffbc40054f0 con 0x55aa29c2d460 2024-04-24T09:01:52.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.051+0000 7ffbb3400640 1 -- 172.21.15.83:0/3357858050 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 40) ==== 8382+0+0 (secure 0 0 0) 0x7ffbb80d0bd0 con 0x55aa29c2d460 2024-04-24T09:01:52.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.052+0000 7ffbd7422740 10 client.19278 fetch_fsmap finished waiting for FSMap version 40 2024-04-24T09:01:52.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.052+0000 7ffbd7422740 10 client.19278 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-24T09:01:52.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.052+0000 7ffbd7422740 20 client.19278 populate_metadata read hostname 'smithi083' 2024-04-24T09:01:52.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.052+0000 7ffbd7422740 1 --2- 172.21.15.83:0/3357858050 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55aa29c47d30 0x55aa29c4a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:52.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.052+0000 7ffbd7422740 4 client.19278 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:01:52.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.052+0000 7ffbd7422740 1 -- 172.21.15.83:0/3357858050 --> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aa29b9fbf0 con 0x55aa29c47d30 2024-04-24T09:01:52.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.052+0000 7ffbd1c00640 1 --2- 172.21.15.83:0/3357858050 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55aa29c47d30 0x55aa29c4a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:52.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.053+0000 7ffbd1c00640 1 --2- 172.21.15.83:0/3357858050 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55aa29c47d30 0x55aa29c4a1f0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=98b04d7d5aa006f1 server_cookie=325eb2bd04d81f4 in_seq=0 out_seq=0 2024-04-24T09:01:52.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.053+0000 7ffbb3400640 10 client.19278 ms_handle_connect on v2:172.21.15.138:6834/1909219758 2024-04-24T09:01:52.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.055+0000 7ffbb3400640 1 -- 172.21.15.83:0/3357858050 <== mds.2 v2:172.21.15.138:6834/1909219758 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55aa29b9fbf0 con 0x55aa29c47d30 2024-04-24T09:01:52.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.055+0000 7ffbb3400640 10 client.19278 handle_command_reply: tid=0 2024-04-24T09:01:52.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.055+0000 7ffbd7422740 1 client.19278 shutdown 2024-04-24T09:01:52.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.056+0000 7ffbd7422740 1 -- 172.21.15.83:0/3357858050 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aa29c2d460 msgr2=0x55aa29c348c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:52.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.056+0000 7ffbd7422740 1 --2- 172.21.15.83:0/3357858050 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aa29c2d460 0x55aa29c348c0 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7ffbb809c230 tx=0x7ffbb809c160 comp rx=0 tx=0).stop 2024-04-24T09:01:52.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.056+0000 7ffbd7422740 1 -- 172.21.15.83:0/3357858050 shutdown_connections 2024-04-24T09:01:52.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.056+0000 7ffbd7422740 1 --2- 172.21.15.83:0/3357858050 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55aa29c47d30 0x55aa29c4a1f0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:52.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.056+0000 7ffbd7422740 1 --2- 172.21.15.83:0/3357858050 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aa29c37b30 0x55aa29c34fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:52.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.056+0000 7ffbd7422740 1 --2- 172.21.15.83:0/3357858050 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aa29c2d460 0x55aa29c348c0 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:52.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.056+0000 7ffbd7422740 1 --2- 172.21.15.83:0/3357858050 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aa29c2cb40 0x55aa29c375d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:52.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.056+0000 7ffbb3400640 0 client.19278 ms_handle_reset on v2:172.21.15.138:6834/1909219758 2024-04-24T09:01:52.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.056+0000 7ffbd7422740 1 -- 172.21.15.83:0/3357858050 >> 172.21.15.83:0/3357858050 conn(0x55aa29c28e60 msgr2=0x55aa29c2eb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:52.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.056+0000 7ffbd7422740 1 -- 172.21.15.83:0/3357858050 shutdown_connections 2024-04-24T09:01:52.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.056+0000 7ffbd7422740 1 -- 172.21.15.83:0/3357858050 wait complete. 2024-04-24T09:01:52.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.056+0000 7ffbd7422740 20 client.19278 trim_cache size 0 max 16384 2024-04-24T09:01:52.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.071+0000 7ffbd7422740 1 Processor -- start 2024-04-24T09:01:52.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.072+0000 7ffbd7422740 1 -- start start 2024-04-24T09:01:52.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.072+0000 7ffbd7422740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aa29c2cb40 0x55aa29c43820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:52.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.072+0000 7ffbd7422740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aa29c2d460 0x55aa29c37100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:52.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.072+0000 7ffbd7422740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aa29c37b30 0x55aa29c4fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:52.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.072+0000 7ffbd7422740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ffbcc1b1550 con 0x55aa29c2d460 2024-04-24T09:01:52.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.072+0000 7ffbd7422740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ffbcc1b3180 con 0x55aa29c2cb40 2024-04-24T09:01:52.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.072+0000 7ffbd7422740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ffbcc1b1c20 con 0x55aa29c37b30 2024-04-24T09:01:52.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.072+0000 7ffbd1200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aa29c2d460 0x55aa29c37100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:52.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.072+0000 7ffbd1200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aa29c2d460 0x55aa29c37100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45650/0 (socket says 172.21.15.83:45650) 2024-04-24T09:01:52.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.072+0000 7ffbd1200640 1 -- 172.21.15.83:0/524469942 learned_addr learned my addr 172.21.15.83:0/524469942 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:01:52.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.073+0000 7ffbd2600640 1 --2- 172.21.15.83:0/524469942 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aa29c37b30 0x55aa29c4fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:52.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.073+0000 7ffbd1200640 1 -- 172.21.15.83:0/524469942 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aa29c37b30 msgr2=0x55aa29c4fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:52.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.073+0000 7ffbd1200640 1 --2- 172.21.15.83:0/524469942 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aa29c37b30 0x55aa29c4fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:52.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.073+0000 7ffbd1200640 1 -- 172.21.15.83:0/524469942 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aa29c2cb40 msgr2=0x55aa29c43820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:52.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.073+0000 7ffbd1200640 1 --2- 172.21.15.83:0/524469942 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aa29c2cb40 0x55aa29c43820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:52.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.073+0000 7ffbd1200640 1 -- 172.21.15.83:0/524469942 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa29c2d290 con 0x55aa29c2d460 2024-04-24T09:01:52.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.073+0000 7ffbd2600640 1 --2- 172.21.15.83:0/524469942 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aa29c37b30 0x55aa29c4fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:01:52.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.073+0000 7ffbd1200640 1 --2- 172.21.15.83:0/524469942 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aa29c2d460 0x55aa29c37100 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7ffbb8098c70 tx=0x7ffbb8005d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe1006b52414a7b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:52.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.073+0000 7ffbbe000640 1 -- 172.21.15.83:0/524469942 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbb80bb240 con 0x55aa29c2d460 2024-04-24T09:01:52.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.073+0000 7ffbbe000640 1 -- 172.21.15.83:0/524469942 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffbb80046c0 con 0x55aa29c2d460 2024-04-24T09:01:52.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.073+0000 7ffbbe000640 1 -- 172.21.15.83:0/524469942 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbb809bc00 con 0x55aa29c2d460 2024-04-24T09:01:52.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.074+0000 7ffbd7422740 1 -- 172.21.15.83:0/524469942 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aa29c2d460 msgr2=0x55aa29c37100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:52.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.074+0000 7ffbd7422740 1 --2- 172.21.15.83:0/524469942 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aa29c2d460 0x55aa29c37100 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7ffbb8098c70 tx=0x7ffbb8005d40 comp rx=0 tx=0).stop 2024-04-24T09:01:52.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.074+0000 7ffbd7422740 1 -- 172.21.15.83:0/524469942 shutdown_connections 2024-04-24T09:01:52.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.074+0000 7ffbd7422740 1 --2- 172.21.15.83:0/524469942 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aa29c37b30 0x55aa29c4fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:52.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.074+0000 7ffbd7422740 1 --2- 172.21.15.83:0/524469942 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aa29c2d460 0x55aa29c37100 unknown :-1 s=CLOSED pgs=1707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:52.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.074+0000 7ffbd7422740 1 --2- 172.21.15.83:0/524469942 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aa29c2cb40 0x55aa29c43820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:52.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.074+0000 7ffbd7422740 1 -- 172.21.15.83:0/524469942 >> 172.21.15.83:0/524469942 conn(0x55aa29c28e60 msgr2=0x55aa29c2fb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:52.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.074+0000 7ffbd7422740 1 -- 172.21.15.83:0/524469942 shutdown_connections 2024-04-24T09:01:52.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.074+0000 7ffbd7422740 1 -- 172.21.15.83:0/524469942 wait complete. 2024-04-24T09:01:52.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.075+0000 7ffbd7422740 1 -- start start 2024-04-24T09:01:52.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.075+0000 7ffbd7422740 1 Processor -- start 2024-04-24T09:01:52.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.075+0000 7ffbd7422740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aa29c2cb40 0x55aa29c389c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:52.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.075+0000 7ffbd7422740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aa29c2d460 0x55aa29c38f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:52.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.075+0000 7ffbd7422740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aa29c37b30 0x55aa29c39440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:52.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.075+0000 7ffbd7422740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ffbcc1b2a50 con 0x55aa29c2cb40 2024-04-24T09:01:52.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.075+0000 7ffbd7422740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ffbcc1b2320 con 0x55aa29c37b30 2024-04-24T09:01:52.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.075+0000 7ffbd7422740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ffbcc1af550 con 0x55aa29c2d460 2024-04-24T09:01:52.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.075+0000 7ffbd7422740 10 client.0 _ll_register_callbacks cb 0x7ffbb80985b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:01:52.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.076+0000 7ffbd1c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aa29c2cb40 0x55aa29c389c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:52.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.076+0000 7ffbd1c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aa29c2cb40 0x55aa29c389c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45664/0 (socket says 172.21.15.83:45664) 2024-04-24T09:01:52.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.076+0000 7ffbd1c00640 1 -- 172.21.15.83:0/4284895056 learned_addr learned my addr 172.21.15.83:0/4284895056 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:01:52.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.076+0000 7ffbd1200640 1 --2- 172.21.15.83:0/4284895056 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aa29c2d460 0x55aa29c38f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:52.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.076+0000 7ffbd1c00640 1 -- 172.21.15.83:0/4284895056 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aa29c2d460 msgr2=0x55aa29c38f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:52.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.076+0000 7ffbd1c00640 1 --2- 172.21.15.83:0/4284895056 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aa29c2d460 0x55aa29c38f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:52.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.076+0000 7ffbd1c00640 1 -- 172.21.15.83:0/4284895056 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aa29c37b30 msgr2=0x55aa29c39440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:52.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.076+0000 7ffbd1c00640 1 --2- 172.21.15.83:0/4284895056 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aa29c37b30 0x55aa29c39440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:52.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.076+0000 7ffbd1c00640 1 -- 172.21.15.83:0/4284895056 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa2996e7d0 con 0x55aa29c2cb40 2024-04-24T09:01:52.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.076+0000 7ffbd1c00640 1 --2- 172.21.15.83:0/4284895056 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aa29c2cb40 0x55aa29c389c0 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x55aa29c4aa60 tx=0x7ffbc0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3b31a6138489c44 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:52.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.076+0000 7ffbb3400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:01:52.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.077+0000 7ffbb3400640 1 -- 172.21.15.83:0/4284895056 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc0028280 con 0x55aa29c2cb40 2024-04-24T09:01:52.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.077+0000 7ffbb3400640 1 -- 172.21.15.83:0/4284895056 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7ffbc0028420 con 0x55aa29c2cb40 2024-04-24T09:01:52.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.077+0000 7ffbd7422740 1 -- 172.21.15.83:0/4284895056 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55aa29b99a60 con 0x55aa29c2cb40 2024-04-24T09:01:52.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.077+0000 7ffbb3400640 1 -- 172.21.15.83:0/4284895056 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7ffbc0037050 con 0x55aa29c2cb40 2024-04-24T09:01:52.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.077+0000 7ffbd7422740 10 client.19284 fetch_fsmap learned FSMap version 40 2024-04-24T09:01:52.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.077+0000 7ffbd7422740 1 -- 172.21.15.83:0/4284895056 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55aa29cae550 con 0x55aa29c2cb40 2024-04-24T09:01:52.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.077+0000 7ffbb3400640 1 -- 172.21.15.83:0/4284895056 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55aa29b99a60 con 0x55aa29c2cb40 2024-04-24T09:01:52.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.077+0000 7ffbb3400640 1 -- 172.21.15.83:0/4284895056 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 40) ==== 8382+0+0 (secure 0 0 0) 0x7ffbc0019040 con 0x55aa29c2cb40 2024-04-24T09:01:52.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.078+0000 7ffbd7422740 10 client.19284 fetch_fsmap finished waiting for FSMap version 40 2024-04-24T09:01:52.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.078+0000 7ffbd7422740 10 client.19284 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-24T09:01:52.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.078+0000 7ffbd7422740 20 client.19284 populate_metadata read hostname 'smithi083' 2024-04-24T09:01:52.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.078+0000 7ffbd7422740 1 --2- 172.21.15.83:0/4284895056 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55aa29caed60 0x55aa29cb1180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:52.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.078+0000 7ffbd7422740 4 client.19284 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:01:52.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.078+0000 7ffbd7422740 1 -- 172.21.15.83:0/4284895056 --> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55aa29b154d0 con 0x55aa29caed60 2024-04-24T09:01:52.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.079+0000 7ffbd1200640 1 --2- 172.21.15.83:0/4284895056 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55aa29caed60 0x55aa29cb1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:52.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.082+0000 7ffbd1200640 1 --2- 172.21.15.83:0/4284895056 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55aa29caed60 0x55aa29cb1180 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fc0ef8dc85b3adc server_cookie=83fadd1f05ad1ae2 in_seq=0 out_seq=0 2024-04-24T09:01:52.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.082+0000 7ffbb3400640 10 client.19284 ms_handle_connect on v2:172.21.15.138:6834/1909219758 2024-04-24T09:01:52.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.083+0000 7ffbb3400640 1 -- 172.21.15.83:0/4284895056 <== mds.2 v2:172.21.15.138:6834/1909219758 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55aa29b154d0 con 0x55aa29caed60 2024-04-24T09:01:52.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.083+0000 7ffbb3400640 10 client.19284 handle_command_reply: tid=0 2024-04-24T09:01:52.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.083+0000 7ffbd7422740 1 client.19284 shutdown 2024-04-24T09:01:52.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.083+0000 7ffbd7422740 1 -- 172.21.15.83:0/4284895056 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aa29c2cb40 msgr2=0x55aa29c389c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:52.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.083+0000 7ffbd7422740 1 --2- 172.21.15.83:0/4284895056 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aa29c2cb40 0x55aa29c389c0 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x55aa29c4aa60 tx=0x7ffbc0004ab0 comp rx=0 tx=0).stop 2024-04-24T09:01:52.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.083+0000 7ffbd7422740 1 -- 172.21.15.83:0/4284895056 shutdown_connections 2024-04-24T09:01:52.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.083+0000 7ffbd7422740 1 --2- 172.21.15.83:0/4284895056 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55aa29caed60 0x55aa29cb1180 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:52.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.083+0000 7ffbd7422740 1 --2- 172.21.15.83:0/4284895056 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aa29c37b30 0x55aa29c39440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:52.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.083+0000 7ffbd7422740 1 --2- 172.21.15.83:0/4284895056 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aa29c2d460 0x55aa29c38f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:52.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.083+0000 7ffbd7422740 1 --2- 172.21.15.83:0/4284895056 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aa29c2cb40 0x55aa29c389c0 unknown :-1 s=CLOSED pgs=1708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:52.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.083+0000 7ffbb3400640 0 client.19284 ms_handle_reset on v2:172.21.15.138:6834/1909219758 2024-04-24T09:01:52.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.083+0000 7ffbd7422740 1 -- 172.21.15.83:0/4284895056 >> 172.21.15.83:0/4284895056 conn(0x55aa29c28e60 msgr2=0x55aa29c276c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:52.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.083+0000 7ffbd7422740 1 -- 172.21.15.83:0/4284895056 shutdown_connections 2024-04-24T09:01:52.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.083+0000 7ffbd7422740 1 -- 172.21.15.83:0/4284895056 wait complete. 2024-04-24T09:01:52.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.084+0000 7ffbd7422740 20 client.19284 trim_cache size 0 max 16384 2024-04-24T09:01:52.085 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:01:52.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.084+0000 7ffbd3000640 1 -- 172.21.15.83:0/2016996018 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ffbb40a3d20 msgr2=0x7ffbb40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:52.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.084+0000 7ffbd3000640 1 --2- 172.21.15.83:0/2016996018 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ffbb40a3d20 0x7ffbb40a61e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x55aa29c2e100 tx=0x7ffbc00097c0 comp rx=0 tx=0).stop 2024-04-24T09:01:52.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.084+0000 7ffbd3000640 1 -- 172.21.15.83:0/2016996018 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffbcc1093f0 msgr2=0x7ffbcc1a0b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:52.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.084+0000 7ffbd3000640 1 --2- 172.21.15.83:0/2016996018 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffbcc1093f0 0x7ffbcc1a0b30 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7ffbb800d740 tx=0x7ffbb800dc00 comp rx=0 tx=0).stop 2024-04-24T09:01:52.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.085+0000 7ffbd3000640 1 -- 172.21.15.83:0/2016996018 shutdown_connections 2024-04-24T09:01:52.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.085+0000 7ffbd3000640 1 --2- 172.21.15.83:0/2016996018 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ffbb40a3d20 0x7ffbb40a61e0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:52.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.085+0000 7ffbd3000640 1 --2- 172.21.15.83:0/2016996018 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffbcc19a6b0 0x7ffbcc19ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:52.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.085+0000 7ffbd3000640 1 --2- 172.21.15.83:0/2016996018 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffbcc1093f0 0x7ffbcc1a0b30 unknown :-1 s=CLOSED pgs=1704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:52.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.085+0000 7ffbd3000640 1 --2- 172.21.15.83:0/2016996018 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffbcc104320 0x7ffbcc1a05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:52.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.085+0000 7ffbd3000640 1 -- 172.21.15.83:0/2016996018 >> 172.21.15.83:0/2016996018 conn(0x7ffbcc0ffb40 msgr2=0x7ffbcc101e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:52.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.085+0000 7ffbd3000640 1 -- 172.21.15.83:0/2016996018 shutdown_connections 2024-04-24T09:01:52.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.085+0000 7ffbd3000640 1 -- 172.21.15.83:0/2016996018 wait complete. 2024-04-24T09:01:52.104 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T09:01:52.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.319+0000 7fe697a00640 1 -- 172.21.15.83:0/2391363511 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe690104e60 msgr2=0x7fe690108ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:52.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.319+0000 7fe697a00640 1 --2- 172.21.15.83:0/2391363511 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe690104e60 0x7fe690108ea0 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7fe67c009900 tx=0x7fe67c02a410 comp rx=0 tx=0).stop 2024-04-24T09:01:52.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.319+0000 7fe697a00640 1 -- 172.21.15.83:0/2391363511 shutdown_connections 2024-04-24T09:01:52.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.319+0000 7fe697a00640 1 --2- 172.21.15.83:0/2391363511 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe690109580 0x7fe690111150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:52.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.319+0000 7fe697a00640 1 --2- 172.21.15.83:0/2391363511 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe690104e60 0x7fe690108ea0 unknown :-1 s=CLOSED pgs=1709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:52.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.319+0000 7fe697a00640 1 --2- 172.21.15.83:0/2391363511 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe6901044b0 0x7fe690104890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:52.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.319+0000 7fe697a00640 1 -- 172.21.15.83:0/2391363511 >> 172.21.15.83:0/2391363511 conn(0x7fe6900ffd90 msgr2=0x7fe6901021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:52.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.319+0000 7fe697a00640 1 -- 172.21.15.83:0/2391363511 shutdown_connections 2024-04-24T09:01:52.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.319+0000 7fe697a00640 1 -- 172.21.15.83:0/2391363511 wait complete. 2024-04-24T09:01:52.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.320+0000 7fe697a00640 1 Processor -- start 2024-04-24T09:01:52.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.320+0000 7fe697a00640 1 -- start start 2024-04-24T09:01:52.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.320+0000 7fe697a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe6901044b0 0x7fe6901a0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:52.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.320+0000 7fe697a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe690109580 0x7fe6901a0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:52.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.320+0000 7fe697a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe69019a950 0x7fe69019ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:52.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.320+0000 7fe697a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe6901144f0 con 0x7fe690109580 2024-04-24T09:01:52.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.320+0000 7fe697a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe690114370 con 0x7fe69019a950 2024-04-24T09:01:52.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.320+0000 7fe697a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe690114670 con 0x7fe6901044b0 2024-04-24T09:01:52.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.321+0000 7fe68fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe690109580 0x7fe6901a0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:52.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.321+0000 7fe68fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe690109580 0x7fe6901a0c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45676/0 (socket says 172.21.15.83:45676) 2024-04-24T09:01:52.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.321+0000 7fe68fe00640 1 -- 172.21.15.83:0/1939053914 learned_addr learned my addr 172.21.15.83:0/1939053914 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:01:52.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.321+0000 7fe695000640 1 --2- 172.21.15.83:0/1939053914 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe6901044b0 0x7fe6901a0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:52.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.321+0000 7fe68fe00640 1 -- 172.21.15.83:0/1939053914 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe6901044b0 msgr2=0x7fe6901a0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:52.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.321+0000 7fe68fe00640 1 --2- 172.21.15.83:0/1939053914 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe6901044b0 0x7fe6901a0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:52.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.321+0000 7fe68fe00640 1 -- 172.21.15.83:0/1939053914 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe69019a950 msgr2=0x7fe69019ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:52.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.321+0000 7fe68fe00640 1 --2- 172.21.15.83:0/1939053914 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe69019a950 0x7fe69019ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:52.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.321+0000 7fe68fe00640 1 -- 172.21.15.83:0/1939053914 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe69019b470 con 0x7fe690109580 2024-04-24T09:01:52.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.322+0000 7fe68fe00640 1 --2- 172.21.15.83:0/1939053914 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe690109580 0x7fe6901a0c90 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7fe67c005ab0 tx=0x7fe67c003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:52.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.322+0000 7fe68d600640 1 -- 172.21.15.83:0/1939053914 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe67c03f070 con 0x7fe690109580 2024-04-24T09:01:52.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.322+0000 7fe68d600640 1 -- 172.21.15.83:0/1939053914 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe67c031b50 con 0x7fe690109580 2024-04-24T09:01:52.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.322+0000 7fe697a00640 1 -- 172.21.15.83:0/1939053914 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe69019b700 con 0x7fe690109580 2024-04-24T09:01:52.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.322+0000 7fe697a00640 1 -- 172.21.15.83:0/1939053914 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6901a78a0 con 0x7fe690109580 2024-04-24T09:01:52.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.323+0000 7fe68d600640 1 -- 172.21.15.83:0/1939053914 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe67c035030 con 0x7fe690109580 2024-04-24T09:01:52.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.326+0000 7fe68d600640 1 -- 172.21.15.83:0/1939053914 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe67c042070 con 0x7fe690109580 2024-04-24T09:01:52.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.326+0000 7fe69a9ad740 1 Processor -- start 2024-04-24T09:01:52.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.327+0000 7fe69a9ad740 1 -- start start 2024-04-24T09:01:52.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.327+0000 7fe69a9ad740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0221d5950 0x55f0221d5d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:52.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.327+0000 7fe69a9ad740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0221d6270 0x55f0221e0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:52.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.327+0000 7fe69a9ad740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0221e0940 0x55f0221e2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:52.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.327+0000 7fe69a9ad740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55f02213b1d0 con 0x55f0221d6270 2024-04-24T09:01:52.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.327+0000 7fe69a9ad740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f0220e1450 con 0x55f0221e0940 2024-04-24T09:01:52.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.327+0000 7fe69a9ad740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55f022137190 con 0x55f0221d5950 2024-04-24T09:01:52.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.327+0000 7fe695000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0221d6270 0x55f0221e0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:52.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.327+0000 7fe695000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0221d6270 0x55f0221e0400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45688/0 (socket says 172.21.15.83:45688) 2024-04-24T09:01:52.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.327+0000 7fe695000640 1 -- 172.21.15.83:0/3101505756 learned_addr learned my addr 172.21.15.83:0/3101505756 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:01:52.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.328+0000 7fe695a00640 1 --2- 172.21.15.83:0/3101505756 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0221d5950 0x55f0221d5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:52.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.328+0000 7fe68fe00640 1 --2- 172.21.15.83:0/3101505756 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0221e0940 0x55f0221e2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:52.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.328+0000 7fe695000640 1 -- 172.21.15.83:0/3101505756 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0221d5950 msgr2=0x55f0221d5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:52.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.328+0000 7fe695000640 1 --2- 172.21.15.83:0/3101505756 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0221d5950 0x55f0221d5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:52.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.328+0000 7fe695000640 1 -- 172.21.15.83:0/3101505756 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0221e0940 msgr2=0x55f0221e2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:52.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.328+0000 7fe695000640 1 --2- 172.21.15.83:0/3101505756 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0221e0940 0x55f0221e2d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:52.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.328+0000 7fe695000640 1 -- 172.21.15.83:0/3101505756 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f02219e310 con 0x55f0221d6270 2024-04-24T09:01:52.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.328+0000 7fe68d600640 1 --2- 172.21.15.83:0/1939053914 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe6780a3e70 0x7fe6780a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:52.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.328+0000 7fe695000640 1 --2- 172.21.15.83:0/3101505756 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0221d6270 0x55f0221e0400 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7fe68400bc90 tx=0x7fe68400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e2595b3f279c933 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:52.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.328+0000 7fe68d600640 1 -- 172.21.15.83:0/1939053914 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7fe67c0f10d0 con 0x7fe690109580 2024-04-24T09:01:52.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.328+0000 7fe695a00640 1 --2- 172.21.15.83:0/1939053914 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe6780a3e70 0x7fe6780a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:52.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.328+0000 7fe682000640 1 -- 172.21.15.83:0/3101505756 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe684015070 con 0x55f0221d6270 2024-04-24T09:01:52.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.329+0000 7fe682000640 1 -- 172.21.15.83:0/3101505756 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe68401a910 con 0x55f0221d6270 2024-04-24T09:01:52.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.329+0000 7fe682000640 1 -- 172.21.15.83:0/3101505756 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe684011900 con 0x55f0221d6270 2024-04-24T09:01:52.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.329+0000 7fe695a00640 1 --2- 172.21.15.83:0/1939053914 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe6780a3e70 0x7fe6780a6330 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fe688005c40 tx=0x7fe688005bd0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:52.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.329+0000 7fe69a9ad740 1 -- 172.21.15.83:0/3101505756 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0221d6270 msgr2=0x55f0221e0400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:52.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.329+0000 7fe69a9ad740 1 --2- 172.21.15.83:0/3101505756 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0221d6270 0x55f0221e0400 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7fe68400bc90 tx=0x7fe68400f730 comp rx=0 tx=0).stop 2024-04-24T09:01:52.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.330+0000 7fe69a9ad740 1 -- 172.21.15.83:0/3101505756 shutdown_connections 2024-04-24T09:01:52.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.330+0000 7fe69a9ad740 1 --2- 172.21.15.83:0/3101505756 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0221e0940 0x55f0221e2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:52.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.330+0000 7fe69a9ad740 1 --2- 172.21.15.83:0/3101505756 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0221d6270 0x55f0221e0400 unknown :-1 s=CLOSED pgs=1711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:52.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.330+0000 7fe69a9ad740 1 --2- 172.21.15.83:0/3101505756 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0221d5950 0x55f0221d5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:52.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.330+0000 7fe69a9ad740 1 -- 172.21.15.83:0/3101505756 >> 172.21.15.83:0/3101505756 conn(0x55f0221d1c70 msgr2=0x55f0221d2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:52.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.330+0000 7fe69a9ad740 1 -- 172.21.15.83:0/3101505756 shutdown_connections 2024-04-24T09:01:52.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.330+0000 7fe69a9ad740 1 -- 172.21.15.83:0/3101505756 wait complete. 2024-04-24T09:01:52.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.330+0000 7fe69a9ad740 1 -- start start 2024-04-24T09:01:52.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.331+0000 7fe69a9ad740 1 Processor -- start 2024-04-24T09:01:52.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.331+0000 7fe69a9ad740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0221d5950 0x55f0221e0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:52.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.331+0000 7fe69a9ad740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0221d6270 0x55f0221dd660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:52.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.331+0000 7fe69a9ad740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0221e0940 0x55f0221ddda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:52.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.331+0000 7fe69a9ad740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55f0221e37d0 con 0x55f0221e0940 2024-04-24T09:01:52.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.331+0000 7fe69a9ad740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f0220f4810 con 0x55f0221d6270 2024-04-24T09:01:52.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.331+0000 7fe69a9ad740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55f0221e3950 con 0x55f0221d5950 2024-04-24T09:01:52.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.331+0000 7fe69a9ad740 10 client.0 _ll_register_callbacks cb 0x55f0220c2140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:01:52.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.331+0000 7fe695a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0221e0940 0x55f0221ddda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:52.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.331+0000 7fe695a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0221e0940 0x55f0221ddda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45694/0 (socket says 172.21.15.83:45694) 2024-04-24T09:01:52.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.331+0000 7fe695a00640 1 -- 172.21.15.83:0/63799210 learned_addr learned my addr 172.21.15.83:0/63799210 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:01:52.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.332+0000 7fe695000640 1 --2- 172.21.15.83:0/63799210 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0221d5950 0x55f0221e0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:52.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.332+0000 7fe68fe00640 1 --2- 172.21.15.83:0/63799210 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0221d6270 0x55f0221dd660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:52.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.332+0000 7fe695a00640 1 -- 172.21.15.83:0/63799210 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0221d5950 msgr2=0x55f0221e0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:52.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.332+0000 7fe695a00640 1 --2- 172.21.15.83:0/63799210 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0221d5950 0x55f0221e0370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:52.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.332+0000 7fe695a00640 1 -- 172.21.15.83:0/63799210 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0221d6270 msgr2=0x55f0221dd660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:52.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.332+0000 7fe695a00640 1 --2- 172.21.15.83:0/63799210 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0221d6270 0x55f0221dd660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:52.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.332+0000 7fe695a00640 1 -- 172.21.15.83:0/63799210 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0221053f0 con 0x55f0221e0940 2024-04-24T09:01:52.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.332+0000 7fe695a00640 1 --2- 172.21.15.83:0/63799210 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0221e0940 0x55f0221ddda0 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7fe688011c30 tx=0x7fe688004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c91a51249949afe server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:52.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.332+0000 7fe676a00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:01:52.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.333+0000 7fe676a00640 1 -- 172.21.15.83:0/63799210 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe68801e620 con 0x55f0221e0940 2024-04-24T09:01:52.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.333+0000 7fe676a00640 1 -- 172.21.15.83:0/63799210 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7fe68801e7c0 con 0x55f0221e0940 2024-04-24T09:01:52.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.333+0000 7fe69a9ad740 1 -- 172.21.15.83:0/63799210 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe68401a910 con 0x55f0221e0940 2024-04-24T09:01:52.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.333+0000 7fe676a00640 1 -- 172.21.15.83:0/63799210 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fe68802f050 con 0x55f0221e0940 2024-04-24T09:01:52.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.333+0000 7fe69a9ad740 10 client.19296 fetch_fsmap learned FSMap version 40 2024-04-24T09:01:52.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.333+0000 7fe69a9ad740 1 -- 172.21.15.83:0/63799210 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55f021e4d3f0 con 0x55f0221e0940 2024-04-24T09:01:52.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.333+0000 7fe676a00640 1 -- 172.21.15.83:0/63799210 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe68401a910 con 0x55f0221e0940 2024-04-24T09:01:52.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.334+0000 7fe676a00640 1 -- 172.21.15.83:0/63799210 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 40) ==== 8382+0+0 (secure 0 0 0) 0x7fe68801c710 con 0x55f0221e0940 2024-04-24T09:01:52.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.334+0000 7fe69a9ad740 10 client.19296 fetch_fsmap finished waiting for FSMap version 40 2024-04-24T09:01:52.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.334+0000 7fe69a9ad740 10 client.19296 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-24T09:01:52.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.334+0000 7fe69a9ad740 20 client.19296 populate_metadata read hostname 'smithi083' 2024-04-24T09:01:52.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.334+0000 7fe69a9ad740 1 --2- 172.21.15.83:0/63799210 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x55f0221f0c70 0x55f0221f30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:52.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.334+0000 7fe69a9ad740 4 client.19296 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:01:52.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.334+0000 7fe69a9ad740 1 -- 172.21.15.83:0/63799210 --> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f022148bf0 con 0x55f0221f0c70 2024-04-24T09:01:52.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.334+0000 7fe695000640 1 --2- 172.21.15.83:0/63799210 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x55f0221f0c70 0x55f0221f30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:52.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.335+0000 7fe695000640 1 --2- 172.21.15.83:0/63799210 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x55f0221f0c70 0x55f0221f30d0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=23c743f14f61b28 server_cookie=7c71c2c3af8e608d in_seq=0 out_seq=0 2024-04-24T09:01:52.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.335+0000 7fe676a00640 10 client.19296 ms_handle_connect on v2:172.21.15.138:6836/900566754 2024-04-24T09:01:52.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.337+0000 7fe676a00640 1 -- 172.21.15.83:0/63799210 <== mds.1 v2:172.21.15.138:6836/900566754 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f022148bf0 con 0x55f0221f0c70 2024-04-24T09:01:52.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.337+0000 7fe676a00640 10 client.19296 handle_command_reply: tid=0 2024-04-24T09:01:52.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.337+0000 7fe69a9ad740 1 client.19296 shutdown 2024-04-24T09:01:52.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.338+0000 7fe69a9ad740 1 -- 172.21.15.83:0/63799210 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0221e0940 msgr2=0x55f0221ddda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:52.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.338+0000 7fe69a9ad740 1 --2- 172.21.15.83:0/63799210 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0221e0940 0x55f0221ddda0 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7fe688011c30 tx=0x7fe688004a40 comp rx=0 tx=0).stop 2024-04-24T09:01:52.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.338+0000 7fe69a9ad740 1 -- 172.21.15.83:0/63799210 shutdown_connections 2024-04-24T09:01:52.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.338+0000 7fe69a9ad740 1 --2- 172.21.15.83:0/63799210 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0221e0940 0x55f0221ddda0 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:52.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.338+0000 7fe69a9ad740 1 --2- 172.21.15.83:0/63799210 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0221d6270 0x55f0221dd660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:52.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.338+0000 7fe69a9ad740 1 --2- 172.21.15.83:0/63799210 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x55f0221f0c70 0x55f0221f30d0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:52.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.338+0000 7fe69a9ad740 1 --2- 172.21.15.83:0/63799210 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0221d5950 0x55f0221e0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:52.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.338+0000 7fe69a9ad740 1 -- 172.21.15.83:0/63799210 >> 172.21.15.83:0/63799210 conn(0x55f0221d1c70 msgr2=0x55f0221d7970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:52.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.338+0000 7fe676a00640 0 client.19296 ms_handle_reset on v2:172.21.15.138:6836/900566754 2024-04-24T09:01:52.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.338+0000 7fe69a9ad740 1 -- 172.21.15.83:0/63799210 shutdown_connections 2024-04-24T09:01:52.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.338+0000 7fe69a9ad740 1 -- 172.21.15.83:0/63799210 wait complete. 2024-04-24T09:01:52.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.338+0000 7fe69a9ad740 20 client.19296 trim_cache size 0 max 16384 2024-04-24T09:01:52.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.356+0000 7fe69a9ad740 1 Processor -- start 2024-04-24T09:01:52.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.356+0000 7fe69a9ad740 1 -- start start 2024-04-24T09:01:52.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.356+0000 7fe69a9ad740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0221d5950 0x55f0221d80b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:52.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.356+0000 7fe69a9ad740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0221d6270 0x55f0221d0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:52.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.356+0000 7fe69a9ad740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0221e0940 0x55f0221dfea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:52.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.356+0000 7fe69a9ad740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe670010100 con 0x55f0221d5950 2024-04-24T09:01:52.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.356+0000 7fe69a9ad740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe670011d30 con 0x55f0221d6270 2024-04-24T09:01:52.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.356+0000 7fe69a9ad740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe6700107d0 con 0x55f0221e0940 2024-04-24T09:01:52.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.357+0000 7fe695000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0221d5950 0x55f0221d80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:52.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.357+0000 7fe695000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0221d5950 0x55f0221d80b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45698/0 (socket says 172.21.15.83:45698) 2024-04-24T09:01:52.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.357+0000 7fe695000640 1 -- 172.21.15.83:0/3972440969 learned_addr learned my addr 172.21.15.83:0/3972440969 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:01:52.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.357+0000 7fe68fe00640 1 --2- 172.21.15.83:0/3972440969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0221d6270 0x55f0221d0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:52.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.357+0000 7fe695a00640 1 --2- 172.21.15.83:0/3972440969 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0221e0940 0x55f0221dfea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:52.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.357+0000 7fe695000640 1 -- 172.21.15.83:0/3972440969 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0221e0940 msgr2=0x55f0221dfea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:52.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.357+0000 7fe695000640 1 --2- 172.21.15.83:0/3972440969 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0221e0940 0x55f0221dfea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:52.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.357+0000 7fe695000640 1 -- 172.21.15.83:0/3972440969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0221d6270 msgr2=0x55f0221d0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:52.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.357+0000 7fe695000640 1 --2- 172.21.15.83:0/3972440969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0221d6270 0x55f0221d0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:52.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.357+0000 7fe695000640 1 -- 172.21.15.83:0/3972440969 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0221cd450 con 0x55f0221d5950 2024-04-24T09:01:52.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.358+0000 7fe695000640 1 --2- 172.21.15.83:0/3972440969 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0221d5950 0x55f0221d80b0 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7fe68400bc90 tx=0x7fe684005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb63b0e90e5c21f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:52.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.358+0000 7fe681600640 1 -- 172.21.15.83:0/3972440969 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe68402c070 con 0x55f0221d5950 2024-04-24T09:01:52.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.358+0000 7fe681600640 1 -- 172.21.15.83:0/3972440969 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe684005040 con 0x55f0221d5950 2024-04-24T09:01:52.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.358+0000 7fe681600640 1 -- 172.21.15.83:0/3972440969 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe684015040 con 0x55f0221d5950 2024-04-24T09:01:52.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.359+0000 7fe69a9ad740 1 -- 172.21.15.83:0/3972440969 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0221d5950 msgr2=0x55f0221d80b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:52.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.359+0000 7fe69a9ad740 1 --2- 172.21.15.83:0/3972440969 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0221d5950 0x55f0221d80b0 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7fe68400bc90 tx=0x7fe684005c90 comp rx=0 tx=0).stop 2024-04-24T09:01:52.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.359+0000 7fe69a9ad740 1 -- 172.21.15.83:0/3972440969 shutdown_connections 2024-04-24T09:01:52.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.359+0000 7fe69a9ad740 1 --2- 172.21.15.83:0/3972440969 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0221e0940 0x55f0221dfea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:52.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.359+0000 7fe69a9ad740 1 --2- 172.21.15.83:0/3972440969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0221d6270 0x55f0221d0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:52.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.359+0000 7fe69a9ad740 1 --2- 172.21.15.83:0/3972440969 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0221d5950 0x55f0221d80b0 unknown :-1 s=CLOSED pgs=1713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:52.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.359+0000 7fe69a9ad740 1 -- 172.21.15.83:0/3972440969 >> 172.21.15.83:0/3972440969 conn(0x55f0221d1c70 msgr2=0x55f0221d0560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:52.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.359+0000 7fe69a9ad740 1 -- 172.21.15.83:0/3972440969 shutdown_connections 2024-04-24T09:01:52.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.359+0000 7fe69a9ad740 1 -- 172.21.15.83:0/3972440969 wait complete. 2024-04-24T09:01:52.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.359+0000 7fe69a9ad740 1 -- start start 2024-04-24T09:01:52.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.360+0000 7fe69a9ad740 1 Processor -- start 2024-04-24T09:01:52.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.360+0000 7fe69a9ad740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0221d5950 0x55f0221e8c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:52.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.360+0000 7fe69a9ad740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0221d6270 0x55f0221e9150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:52.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.360+0000 7fe69a9ad740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0221e0940 0x55f0221e3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:52.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.360+0000 7fe69a9ad740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe670011600 con 0x55f0221d5950 2024-04-24T09:01:52.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.360+0000 7fe69a9ad740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe670010ed0 con 0x55f0221d6270 2024-04-24T09:01:52.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.360+0000 7fe69a9ad740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe67000e100 con 0x55f0221e0940 2024-04-24T09:01:52.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.360+0000 7fe69a9ad740 10 client.0 _ll_register_callbacks cb 0x7fe688009c40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:01:52.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.360+0000 7fe695000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0221d5950 0x55f0221e8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:52.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.361+0000 7fe695000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0221d5950 0x55f0221e8c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45704/0 (socket says 172.21.15.83:45704) 2024-04-24T09:01:52.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.361+0000 7fe695000640 1 -- 172.21.15.83:0/2180080388 learned_addr learned my addr 172.21.15.83:0/2180080388 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:01:52.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.361+0000 7fe695a00640 1 --2- 172.21.15.83:0/2180080388 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0221e0940 0x55f0221e3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:52.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.361+0000 7fe68fe00640 1 --2- 172.21.15.83:0/2180080388 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0221d6270 0x55f0221e9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:52.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.361+0000 7fe695000640 1 -- 172.21.15.83:0/2180080388 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0221e0940 msgr2=0x55f0221e3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:52.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.361+0000 7fe695000640 1 --2- 172.21.15.83:0/2180080388 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0221e0940 0x55f0221e3bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:52.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.361+0000 7fe695000640 1 -- 172.21.15.83:0/2180080388 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0221d6270 msgr2=0x55f0221e9150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:52.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.361+0000 7fe695000640 1 --2- 172.21.15.83:0/2180080388 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0221d6270 0x55f0221e9150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:52.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.361+0000 7fe695000640 1 -- 172.21.15.83:0/2180080388 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f021f177d0 con 0x55f0221d5950 2024-04-24T09:01:52.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.361+0000 7fe695000640 1 --2- 172.21.15.83:0/2180080388 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0221d5950 0x55f0221e8c10 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7fe684005d20 tx=0x7fe68400b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39aebd09173efe11 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:52.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.361+0000 7fe676a00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:01:52.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.362+0000 7fe676a00640 1 -- 172.21.15.83:0/2180080388 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe68402c070 con 0x55f0221d5950 2024-04-24T09:01:52.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.362+0000 7fe676a00640 1 -- 172.21.15.83:0/2180080388 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7fe684004030 con 0x55f0221d5950 2024-04-24T09:01:52.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.362+0000 7fe69a9ad740 1 -- 172.21.15.83:0/2180080388 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe684005040 con 0x55f0221d5950 2024-04-24T09:01:52.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.362+0000 7fe676a00640 1 -- 172.21.15.83:0/2180080388 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fe684015050 con 0x55f0221d5950 2024-04-24T09:01:52.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.362+0000 7fe69a9ad740 10 client.19302 fetch_fsmap learned FSMap version 40 2024-04-24T09:01:52.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.362+0000 7fe69a9ad740 1 -- 172.21.15.83:0/2180080388 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55f0222579f0 con 0x55f0221d5950 2024-04-24T09:01:52.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.362+0000 7fe676a00640 1 -- 172.21.15.83:0/2180080388 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe684005040 con 0x55f0221d5950 2024-04-24T09:01:52.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.362+0000 7fe676a00640 1 -- 172.21.15.83:0/2180080388 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 40) ==== 8382+0+0 (secure 0 0 0) 0x7fe684004680 con 0x55f0221d5950 2024-04-24T09:01:52.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.363+0000 7fe69a9ad740 10 client.19302 fetch_fsmap finished waiting for FSMap version 40 2024-04-24T09:01:52.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.363+0000 7fe69a9ad740 10 client.19302 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-24T09:01:52.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.363+0000 7fe69a9ad740 20 client.19302 populate_metadata read hostname 'smithi083' 2024-04-24T09:01:52.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.363+0000 7fe69a9ad740 1 --2- 172.21.15.83:0/2180080388 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x55f022257fb0 0x55f02225a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:52.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.363+0000 7fe69a9ad740 4 client.19302 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:01:52.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.363+0000 7fe69a9ad740 1 -- 172.21.15.83:0/2180080388 --> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f022142a60 con 0x55f022257fb0 2024-04-24T09:01:52.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.363+0000 7fe68fe00640 1 --2- 172.21.15.83:0/2180080388 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x55f022257fb0 0x55f02225a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:52.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.364+0000 7fe68fe00640 1 --2- 172.21.15.83:0/2180080388 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x55f022257fb0 0x55f02225a3d0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ec6e66b1d67a8a1c server_cookie=c84c13b65417328d in_seq=0 out_seq=0 2024-04-24T09:01:52.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.364+0000 7fe676a00640 10 client.19302 ms_handle_connect on v2:172.21.15.138:6836/900566754 2024-04-24T09:01:52.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.365+0000 7fe676a00640 1 -- 172.21.15.83:0/2180080388 <== mds.1 v2:172.21.15.138:6836/900566754 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f022142a60 con 0x55f022257fb0 2024-04-24T09:01:52.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.365+0000 7fe676a00640 10 client.19302 handle_command_reply: tid=0 2024-04-24T09:01:52.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.365+0000 7fe69a9ad740 1 client.19302 shutdown 2024-04-24T09:01:52.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.365+0000 7fe69a9ad740 1 -- 172.21.15.83:0/2180080388 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0221d5950 msgr2=0x55f0221e8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:52.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.365+0000 7fe69a9ad740 1 --2- 172.21.15.83:0/2180080388 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0221d5950 0x55f0221e8c10 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7fe684005d20 tx=0x7fe68400b8a0 comp rx=0 tx=0).stop 2024-04-24T09:01:52.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.365+0000 7fe69a9ad740 1 -- 172.21.15.83:0/2180080388 shutdown_connections 2024-04-24T09:01:52.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.365+0000 7fe69a9ad740 1 --2- 172.21.15.83:0/2180080388 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x55f022257fb0 0x55f02225a3d0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:52.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.365+0000 7fe69a9ad740 1 --2- 172.21.15.83:0/2180080388 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0221e0940 0x55f0221e3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:52.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.365+0000 7fe69a9ad740 1 --2- 172.21.15.83:0/2180080388 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0221d6270 0x55f0221e9150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:52.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.365+0000 7fe676a00640 0 client.19302 ms_handle_reset on v2:172.21.15.138:6836/900566754 2024-04-24T09:01:52.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.365+0000 7fe69a9ad740 1 --2- 172.21.15.83:0/2180080388 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0221d5950 0x55f0221e8c10 unknown :-1 s=CLOSED pgs=1714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:52.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.365+0000 7fe69a9ad740 1 -- 172.21.15.83:0/2180080388 >> 172.21.15.83:0/2180080388 conn(0x55f0221d1c70 msgr2=0x55f0221d0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:52.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.365+0000 7fe69a9ad740 1 -- 172.21.15.83:0/2180080388 shutdown_connections 2024-04-24T09:01:52.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.365+0000 7fe69a9ad740 1 -- 172.21.15.83:0/2180080388 wait complete. 2024-04-24T09:01:52.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.365+0000 7fe69a9ad740 20 client.19302 trim_cache size 0 max 16384 2024-04-24T09:01:52.368 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:01:52.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.366+0000 7fe697a00640 1 -- 172.21.15.83:0/1939053914 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe6780a3e70 msgr2=0x7fe6780a6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:52.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.366+0000 7fe697a00640 1 --2- 172.21.15.83:0/1939053914 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe6780a3e70 0x7fe6780a6330 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fe688005c40 tx=0x7fe688005bd0 comp rx=0 tx=0).stop 2024-04-24T09:01:52.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.366+0000 7fe697a00640 1 -- 172.21.15.83:0/1939053914 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe690109580 msgr2=0x7fe6901a0c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:52.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.366+0000 7fe697a00640 1 --2- 172.21.15.83:0/1939053914 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe690109580 0x7fe6901a0c90 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7fe67c005ab0 tx=0x7fe67c003b90 comp rx=0 tx=0).stop 2024-04-24T09:01:52.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.366+0000 7fe697a00640 1 -- 172.21.15.83:0/1939053914 shutdown_connections 2024-04-24T09:01:52.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.366+0000 7fe697a00640 1 --2- 172.21.15.83:0/1939053914 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe6780a3e70 0x7fe6780a6330 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:52.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.366+0000 7fe697a00640 1 --2- 172.21.15.83:0/1939053914 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe69019a950 0x7fe69019ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:52.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.366+0000 7fe697a00640 1 --2- 172.21.15.83:0/1939053914 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe690109580 0x7fe6901a0c90 unknown :-1 s=CLOSED pgs=1710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:52.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.366+0000 7fe697a00640 1 --2- 172.21.15.83:0/1939053914 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe6901044b0 0x7fe6901a0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:52.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.366+0000 7fe697a00640 1 -- 172.21.15.83:0/1939053914 >> 172.21.15.83:0/1939053914 conn(0x7fe6900ffd90 msgr2=0x7fe6901011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:52.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.366+0000 7fe697a00640 1 -- 172.21.15.83:0/1939053914 shutdown_connections 2024-04-24T09:01:52.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.366+0000 7fe697a00640 1 -- 172.21.15.83:0/1939053914 wait complete. 2024-04-24T09:01:52.381 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:01:52 smithi083 ceph-mon[27907]: pgmap v320: 129 pgs: 129 active+clean; 56 MiB data, 820 MiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 230 KiB/s wr, 42 op/s 2024-04-24T09:01:52.381 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:01:52 smithi083 ceph-mon[27907]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:01:52.382 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:01:52 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/967350755' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:01:52.385 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T09:01:52.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:01:52 smithi138 ceph-mon[31441]: pgmap v320: 129 pgs: 129 active+clean; 56 MiB data, 820 MiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 230 KiB/s wr, 42 op/s 2024-04-24T09:01:52.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:01:52 smithi138 ceph-mon[31441]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:01:52.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:01:52 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/967350755' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:01:52.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:01:52 smithi189 ceph-mon[31791]: pgmap v320: 129 pgs: 129 active+clean; 56 MiB data, 820 MiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 230 KiB/s wr, 42 op/s 2024-04-24T09:01:52.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:01:52 smithi189 ceph-mon[31791]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:01:52.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:01:52 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/967350755' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:01:52.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.596+0000 7f4a73e00640 1 -- 172.21.15.83:0/2771818335 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a74102aa0 msgr2=0x7f4a74106ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:52.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.596+0000 7f4a73e00640 1 --2- 172.21.15.83:0/2771818335 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a74102aa0 0x7f4a74106ae0 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f4a5c009900 tx=0x7f4a5c02a3f0 comp rx=0 tx=0).stop 2024-04-24T09:01:52.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.596+0000 7f4a73e00640 1 -- 172.21.15.83:0/2771818335 shutdown_connections 2024-04-24T09:01:52.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.596+0000 7f4a73e00640 1 --2- 172.21.15.83:0/2771818335 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4a741071c0 0x7f4a7410ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:52.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.596+0000 7f4a73e00640 1 --2- 172.21.15.83:0/2771818335 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a74102aa0 0x7f4a74106ae0 unknown :-1 s=CLOSED pgs=1715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:52.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.596+0000 7f4a73e00640 1 --2- 172.21.15.83:0/2771818335 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a741020f0 0x7f4a741024d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:52.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.596+0000 7f4a73e00640 1 -- 172.21.15.83:0/2771818335 >> 172.21.15.83:0/2771818335 conn(0x7f4a740fd9d0 msgr2=0x7f4a740ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:52.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.596+0000 7f4a73e00640 1 -- 172.21.15.83:0/2771818335 shutdown_connections 2024-04-24T09:01:52.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.596+0000 7f4a73e00640 1 -- 172.21.15.83:0/2771818335 wait complete. 2024-04-24T09:01:52.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.597+0000 7f4a73e00640 1 Processor -- start 2024-04-24T09:01:52.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.598+0000 7f4a73e00640 1 -- start start 2024-04-24T09:01:52.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.598+0000 7f4a73e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a741020f0 0x7f4a741a05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:52.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.598+0000 7f4a73e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a741071c0 0x7f4a741a0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:52.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.598+0000 7f4a73e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4a7419a690 0x7f4a7419ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:52.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.598+0000 7f4a73e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4a74112130 con 0x7f4a741071c0 2024-04-24T09:01:52.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.598+0000 7f4a73e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4a74111fb0 con 0x7f4a741020f0 2024-04-24T09:01:52.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.598+0000 7f4a73e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4a741122b0 con 0x7f4a7419a690 2024-04-24T09:01:52.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.598+0000 7f4a6be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a741071c0 0x7f4a741a0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:52.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.598+0000 7f4a6be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a741071c0 0x7f4a741a0b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45724/0 (socket says 172.21.15.83:45724) 2024-04-24T09:01:52.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.599+0000 7f4a6be00640 1 -- 172.21.15.83:0/131757272 learned_addr learned my addr 172.21.15.83:0/131757272 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:01:52.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.599+0000 7f4a73400640 1 --2- 172.21.15.83:0/131757272 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4a7419a690 0x7f4a7419ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:52.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.599+0000 7f4a72a00640 1 --2- 172.21.15.83:0/131757272 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a741020f0 0x7f4a741a05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:52.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.599+0000 7f4a6be00640 1 -- 172.21.15.83:0/131757272 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4a7419a690 msgr2=0x7f4a7419ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:52.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.599+0000 7f4a6be00640 1 --2- 172.21.15.83:0/131757272 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4a7419a690 0x7f4a7419ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:52.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.599+0000 7f4a6be00640 1 -- 172.21.15.83:0/131757272 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a741020f0 msgr2=0x7f4a741a05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:52.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.599+0000 7f4a6be00640 1 --2- 172.21.15.83:0/131757272 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a741020f0 0x7f4a741a05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:52.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.599+0000 7f4a6be00640 1 -- 172.21.15.83:0/131757272 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a7419b290 con 0x7f4a741071c0 2024-04-24T09:01:52.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.599+0000 7f4a6be00640 1 --2- 172.21.15.83:0/131757272 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a741071c0 0x7f4a741a0b10 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f4a5c005ab0 tx=0x7f4a5c003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:52.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.600+0000 7f4a6b400640 1 -- 172.21.15.83:0/131757272 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a5c03f070 con 0x7f4a741071c0 2024-04-24T09:01:52.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.600+0000 7f4a6b400640 1 -- 172.21.15.83:0/131757272 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4a5c031d60 con 0x7f4a741071c0 2024-04-24T09:01:52.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.600+0000 7f4a73e00640 1 -- 172.21.15.83:0/131757272 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a7419b520 con 0x7f4a741071c0 2024-04-24T09:01:52.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.600+0000 7f4a6b400640 1 -- 172.21.15.83:0/131757272 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a5c035030 con 0x7f4a741071c0 2024-04-24T09:01:52.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.600+0000 7f4a73e00640 1 -- 172.21.15.83:0/131757272 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a741a76b0 con 0x7f4a741071c0 2024-04-24T09:01:52.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.603+0000 7f4a6b400640 1 -- 172.21.15.83:0/131757272 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4a5c042070 con 0x7f4a741071c0 2024-04-24T09:01:52.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.603+0000 7f4a7c36c740 1 Processor -- start 2024-04-24T09:01:52.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.603+0000 7f4a7c36c740 1 -- start start 2024-04-24T09:01:52.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.604+0000 7f4a7c36c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5643cf14cb40 0x5643cf14cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:52.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.604+0000 7f4a7c36c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5643cf14d460 0x5643cf1575f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:52.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.604+0000 7f4a7c36c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5643cf157b30 0x5643cf159f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:52.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.604+0000 7f4a7c36c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5643cf0b21d0 con 0x5643cf14cb40 2024-04-24T09:01:52.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.604+0000 7f4a7c36c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5643cf058450 con 0x5643cf14d460 2024-04-24T09:01:52.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.604+0000 7f4a7c36c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5643cf0ae190 con 0x5643cf157b30 2024-04-24T09:01:52.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.604+0000 7f4a73400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5643cf14cb40 0x5643cf14cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:52.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.604+0000 7f4a73400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5643cf14cb40 0x5643cf14cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45734/0 (socket says 172.21.15.83:45734) 2024-04-24T09:01:52.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.604+0000 7f4a73400640 1 -- 172.21.15.83:0/3632217550 learned_addr learned my addr 172.21.15.83:0/3632217550 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:01:52.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.604+0000 7f4a72a00640 1 --2- 172.21.15.83:0/3632217550 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5643cf14d460 0x5643cf1575f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:52.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.604+0000 7f4a6be00640 1 --2- 172.21.15.83:0/3632217550 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5643cf157b30 0x5643cf159f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:52.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.605+0000 7f4a73400640 1 -- 172.21.15.83:0/3632217550 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5643cf157b30 msgr2=0x5643cf159f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:52.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.605+0000 7f4a73400640 1 --2- 172.21.15.83:0/3632217550 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5643cf157b30 0x5643cf159f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:52.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.605+0000 7f4a73400640 1 -- 172.21.15.83:0/3632217550 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5643cf14d460 msgr2=0x5643cf1575f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:52.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.605+0000 7f4a73400640 1 --2- 172.21.15.83:0/3632217550 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5643cf14d460 0x5643cf1575f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:52.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.605+0000 7f4a73400640 1 -- 172.21.15.83:0/3632217550 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643cf115310 con 0x5643cf14cb40 2024-04-24T09:01:52.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.605+0000 7f4a6b400640 1 --2- 172.21.15.83:0/131757272 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f4a580a3da0 0x7f4a580a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:52.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.605+0000 7f4a73400640 1 --2- 172.21.15.83:0/3632217550 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5643cf14cb40 0x5643cf14cf20 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c007dd0 tx=0x7f4a6c00e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20f0a0c6a7d4b1f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:52.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.605+0000 7f4a6b400640 1 -- 172.21.15.83:0/131757272 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7f4a5c0f14f0 con 0x7f4a741071c0 2024-04-24T09:01:52.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.605+0000 7f4a63400640 1 -- 172.21.15.83:0/3632217550 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a6c014070 con 0x5643cf14cb40 2024-04-24T09:01:52.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.605+0000 7f4a63400640 1 -- 172.21.15.83:0/3632217550 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4a6c004d20 con 0x5643cf14cb40 2024-04-24T09:01:52.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.605+0000 7f4a63400640 1 -- 172.21.15.83:0/3632217550 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a6c0107b0 con 0x5643cf14cb40 2024-04-24T09:01:52.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.605+0000 7f4a72a00640 1 --2- 172.21.15.83:0/131757272 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f4a580a3da0 0x7f4a580a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:52.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.606+0000 7f4a7c36c740 1 -- 172.21.15.83:0/3632217550 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5643cf14cb40 msgr2=0x5643cf14cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:52.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.606+0000 7f4a72a00640 1 --2- 172.21.15.83:0/131757272 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f4a580a3da0 0x7f4a580a6260 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f4a64005fd0 tx=0x7f4a6400b650 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:52.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.606+0000 7f4a7c36c740 1 --2- 172.21.15.83:0/3632217550 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5643cf14cb40 0x5643cf14cf20 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c007dd0 tx=0x7f4a6c00e720 comp rx=0 tx=0).stop 2024-04-24T09:01:52.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.606+0000 7f4a7c36c740 1 -- 172.21.15.83:0/3632217550 shutdown_connections 2024-04-24T09:01:52.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.606+0000 7f4a7c36c740 1 --2- 172.21.15.83:0/3632217550 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5643cf157b30 0x5643cf159f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:52.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.606+0000 7f4a7c36c740 1 --2- 172.21.15.83:0/3632217550 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5643cf14d460 0x5643cf1575f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:52.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.606+0000 7f4a7c36c740 1 --2- 172.21.15.83:0/3632217550 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5643cf14cb40 0x5643cf14cf20 unknown :-1 s=CLOSED pgs=1717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:52.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.606+0000 7f4a7c36c740 1 -- 172.21.15.83:0/3632217550 >> 172.21.15.83:0/3632217550 conn(0x5643cf148e60 msgr2=0x5643cf149240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:52.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.606+0000 7f4a7c36c740 1 -- 172.21.15.83:0/3632217550 shutdown_connections 2024-04-24T09:01:52.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.606+0000 7f4a7c36c740 1 -- 172.21.15.83:0/3632217550 wait complete. 2024-04-24T09:01:52.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.607+0000 7f4a7c36c740 1 -- start start 2024-04-24T09:01:52.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.607+0000 7f4a7c36c740 1 Processor -- start 2024-04-24T09:01:52.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.607+0000 7f4a7c36c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5643cf14cb40 0x5643cf1575f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:52.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.608+0000 7f4a7c36c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5643cf14d460 0x5643cf1548e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:52.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.608+0000 7f4a7c36c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5643cf157b30 0x5643cf155020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:52.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.608+0000 7f4a7c36c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5643cf15a9c0 con 0x5643cf157b30 2024-04-24T09:01:52.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.608+0000 7f4a7c36c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5643cf06b810 con 0x5643cf14d460 2024-04-24T09:01:52.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.608+0000 7f4a7c36c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5643cf15ab40 con 0x5643cf14cb40 2024-04-24T09:01:52.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.608+0000 7f4a7c36c740 10 client.0 _ll_register_callbacks cb 0x5643cf039140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:01:52.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.608+0000 7f4a73400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5643cf157b30 0x5643cf155020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:52.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.608+0000 7f4a73400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5643cf157b30 0x5643cf155020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45748/0 (socket says 172.21.15.83:45748) 2024-04-24T09:01:52.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.608+0000 7f4a73400640 1 -- 172.21.15.83:0/3996396111 learned_addr learned my addr 172.21.15.83:0/3996396111 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:01:52.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.608+0000 7f4a72a00640 1 --2- 172.21.15.83:0/3996396111 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5643cf14cb40 0x5643cf1575f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:52.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.608+0000 7f4a6be00640 1 --2- 172.21.15.83:0/3996396111 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5643cf14d460 0x5643cf1548e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:52.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.608+0000 7f4a73400640 1 -- 172.21.15.83:0/3996396111 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5643cf14cb40 msgr2=0x5643cf1575f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:52.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.608+0000 7f4a73400640 1 --2- 172.21.15.83:0/3996396111 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5643cf14cb40 0x5643cf1575f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:52.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.608+0000 7f4a73400640 1 -- 172.21.15.83:0/3996396111 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5643cf14d460 msgr2=0x5643cf1548e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:52.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.608+0000 7f4a73400640 1 --2- 172.21.15.83:0/3996396111 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5643cf14d460 0x5643cf1548e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:52.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.608+0000 7f4a73400640 1 -- 172.21.15.83:0/3996396111 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643cf07c3f0 con 0x5643cf157b30 2024-04-24T09:01:52.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.609+0000 7f4a73400640 1 --2- 172.21.15.83:0/3996396111 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5643cf157b30 0x5643cf155020 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c005700 tx=0x7f4a6c005730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42cae59a222577a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:52.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.609+0000 7f4a57e00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:01:52.611 INFO:tasks.workunit.client.0.smithi083.stdout: 1 60042 0.63 MB/sec execute 386 sec latency 40.766 ms 2024-04-24T09:01:52.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.609+0000 7f4a57e00640 1 -- 172.21.15.83:0/3996396111 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a6c014030 con 0x5643cf157b30 2024-04-24T09:01:52.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.609+0000 7f4a7c36c740 1 -- 172.21.15.83:0/3996396111 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a6c004d20 con 0x5643cf157b30 2024-04-24T09:01:52.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.609+0000 7f4a57e00640 1 -- 172.21.15.83:0/3996396111 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7f4a6c004a40 con 0x5643cf157b30 2024-04-24T09:01:52.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.609+0000 7f4a57e00640 1 -- 172.21.15.83:0/3996396111 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f4a6c003970 con 0x5643cf157b30 2024-04-24T09:01:52.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.609+0000 7f4a7c36c740 10 client.19314 fetch_fsmap learned FSMap version 40 2024-04-24T09:01:52.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.609+0000 7f4a7c36c740 1 -- 172.21.15.83:0/3996396111 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5643cedc43f0 con 0x5643cf157b30 2024-04-24T09:01:52.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.610+0000 7f4a57e00640 1 -- 172.21.15.83:0/3996396111 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4a6c004d20 con 0x5643cf157b30 2024-04-24T09:01:52.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.610+0000 7f4a57e00640 1 -- 172.21.15.83:0/3996396111 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 40) ==== 8382+0+0 (secure 0 0 0) 0x7f4a6c010530 con 0x5643cf157b30 2024-04-24T09:01:52.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.610+0000 7f4a7c36c740 10 client.19314 fetch_fsmap finished waiting for FSMap version 40 2024-04-24T09:01:52.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.610+0000 7f4a7c36c740 10 client.19314 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:01:52.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.610+0000 7f4a7c36c740 20 client.19314 populate_metadata read hostname 'smithi083' 2024-04-24T09:01:52.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.610+0000 7f4a7c36c740 1 --2- 172.21.15.83:0/3996396111 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5643cf167ca0 0x5643cf16a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:52.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.610+0000 7f4a7c36c740 4 client.19314 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:01:52.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.610+0000 7f4a7c36c740 1 -- 172.21.15.83:0/3996396111 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5643cf0bfbf0 con 0x5643cf167ca0 2024-04-24T09:01:52.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.611+0000 7f4a72a00640 1 --2- 172.21.15.83:0/3996396111 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5643cf167ca0 0x5643cf16a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:52.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.612+0000 7f4a72a00640 1 --2- 172.21.15.83:0/3996396111 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5643cf167ca0 0x5643cf16a090 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2adb6382a7573a7b server_cookie=28fff757d58ff325 in_seq=0 out_seq=0 2024-04-24T09:01:52.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.612+0000 7f4a57e00640 10 client.19314 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:01:52.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.614+0000 7f4a57e00640 1 -- 172.21.15.83:0/3996396111 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5643cf0bfbf0 con 0x5643cf167ca0 2024-04-24T09:01:52.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.614+0000 7f4a57e00640 10 client.19314 handle_command_reply: tid=0 2024-04-24T09:01:52.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.614+0000 7f4a7c36c740 1 client.19314 shutdown 2024-04-24T09:01:52.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.614+0000 7f4a7c36c740 1 -- 172.21.15.83:0/3996396111 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5643cf157b30 msgr2=0x5643cf155020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:52.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.614+0000 7f4a7c36c740 1 --2- 172.21.15.83:0/3996396111 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5643cf157b30 0x5643cf155020 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c005700 tx=0x7f4a6c005730 comp rx=0 tx=0).stop 2024-04-24T09:01:52.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.614+0000 7f4a7c36c740 1 -- 172.21.15.83:0/3996396111 shutdown_connections 2024-04-24T09:01:52.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.614+0000 7f4a7c36c740 1 --2- 172.21.15.83:0/3996396111 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5643cf167ca0 0x5643cf16a090 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:52.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.614+0000 7f4a7c36c740 1 --2- 172.21.15.83:0/3996396111 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5643cf157b30 0x5643cf155020 unknown :-1 s=CLOSED pgs=1718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:52.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.614+0000 7f4a7c36c740 1 --2- 172.21.15.83:0/3996396111 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5643cf14d460 0x5643cf1548e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:52.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.614+0000 7f4a7c36c740 1 --2- 172.21.15.83:0/3996396111 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5643cf14cb40 0x5643cf1575f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:52.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.614+0000 7f4a57e00640 0 client.19314 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:01:52.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.614+0000 7f4a7c36c740 1 -- 172.21.15.83:0/3996396111 >> 172.21.15.83:0/3996396111 conn(0x5643cf148e60 msgr2=0x5643cf14eb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:52.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.614+0000 7f4a7c36c740 1 -- 172.21.15.83:0/3996396111 shutdown_connections 2024-04-24T09:01:52.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.614+0000 7f4a7c36c740 1 -- 172.21.15.83:0/3996396111 wait complete. 2024-04-24T09:01:52.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.614+0000 7f4a7c36c740 20 client.19314 trim_cache size 0 max 16384 2024-04-24T09:01:52.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.630+0000 7f4a7c36c740 1 Processor -- start 2024-04-24T09:01:52.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.630+0000 7f4a7c36c740 1 -- start start 2024-04-24T09:01:52.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.630+0000 7f4a7c36c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5643cf14cb40 0x5643cf157120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:52.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.630+0000 7f4a7c36c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5643cf14d460 0x5643cf16f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:52.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.630+0000 7f4a7c36c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5643cf157b30 0x5643cf16fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:52.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.630+0000 7f4a7c36c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4a741b1650 con 0x5643cf14d460 2024-04-24T09:01:52.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.630+0000 7f4a7c36c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4a741b3280 con 0x5643cf157b30 2024-04-24T09:01:52.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.630+0000 7f4a7c36c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4a741b1d20 con 0x5643cf14cb40 2024-04-24T09:01:52.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.630+0000 7f4a6be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5643cf14d460 0x5643cf16f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:52.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.630+0000 7f4a6be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5643cf14d460 0x5643cf16f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45760/0 (socket says 172.21.15.83:45760) 2024-04-24T09:01:52.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.630+0000 7f4a6be00640 1 -- 172.21.15.83:0/3826591492 learned_addr learned my addr 172.21.15.83:0/3826591492 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:01:52.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.631+0000 7f4a73400640 1 --2- 172.21.15.83:0/3826591492 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5643cf157b30 0x5643cf16fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:52.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.631+0000 7f4a72a00640 1 --2- 172.21.15.83:0/3826591492 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5643cf14cb40 0x5643cf157120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:52.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.631+0000 7f4a6be00640 1 -- 172.21.15.83:0/3826591492 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5643cf14cb40 msgr2=0x5643cf157120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:52.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.631+0000 7f4a6be00640 1 --2- 172.21.15.83:0/3826591492 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5643cf14cb40 0x5643cf157120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:52.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.631+0000 7f4a6be00640 1 -- 172.21.15.83:0/3826591492 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5643cf157b30 msgr2=0x5643cf16fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:52.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.631+0000 7f4a6be00640 1 --2- 172.21.15.83:0/3826591492 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5643cf157b30 0x5643cf16fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:52.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.631+0000 7f4a6be00640 1 -- 172.21.15.83:0/3826591492 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643cee8e7d0 con 0x5643cf14d460 2024-04-24T09:01:52.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.631+0000 7f4a73400640 1 --2- 172.21.15.83:0/3826591492 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5643cf157b30 0x5643cf16fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:01:52.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.631+0000 7f4a72a00640 1 --2- 172.21.15.83:0/3826591492 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5643cf14cb40 0x5643cf157120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:01:52.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.631+0000 7f4a6be00640 1 --2- 172.21.15.83:0/3826591492 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5643cf14d460 0x5643cf16f820 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f4a5c000c00 tx=0x7f4a5c031f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c4d43ca38079d50 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:52.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.631+0000 7f4a62a00640 1 -- 172.21.15.83:0/3826591492 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a5c049020 con 0x5643cf14d460 2024-04-24T09:01:52.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.631+0000 7f4a62a00640 1 -- 172.21.15.83:0/3826591492 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4a5c0bd710 con 0x5643cf14d460 2024-04-24T09:01:52.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.631+0000 7f4a62a00640 1 -- 172.21.15.83:0/3826591492 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a5c0b9680 con 0x5643cf14d460 2024-04-24T09:01:52.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.632+0000 7f4a7c36c740 1 -- 172.21.15.83:0/3826591492 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5643cf14d460 msgr2=0x5643cf16f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:52.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.632+0000 7f4a7c36c740 1 --2- 172.21.15.83:0/3826591492 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5643cf14d460 0x5643cf16f820 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f4a5c000c00 tx=0x7f4a5c031f10 comp rx=0 tx=0).stop 2024-04-24T09:01:52.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.632+0000 7f4a7c36c740 1 -- 172.21.15.83:0/3826591492 shutdown_connections 2024-04-24T09:01:52.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.632+0000 7f4a7c36c740 1 --2- 172.21.15.83:0/3826591492 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5643cf157b30 0x5643cf16fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:52.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.632+0000 7f4a7c36c740 1 --2- 172.21.15.83:0/3826591492 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5643cf14d460 0x5643cf16f820 unknown :-1 s=CLOSED pgs=1719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:52.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.632+0000 7f4a7c36c740 1 --2- 172.21.15.83:0/3826591492 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5643cf14cb40 0x5643cf157120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:52.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.632+0000 7f4a7c36c740 1 -- 172.21.15.83:0/3826591492 >> 172.21.15.83:0/3826591492 conn(0x5643cf148e60 msgr2=0x5643cf1476c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:52.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.632+0000 7f4a7c36c740 1 -- 172.21.15.83:0/3826591492 shutdown_connections 2024-04-24T09:01:52.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.632+0000 7f4a7c36c740 1 -- 172.21.15.83:0/3826591492 wait complete. 2024-04-24T09:01:52.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.633+0000 7f4a7c36c740 1 -- start start 2024-04-24T09:01:52.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.633+0000 7f4a7c36c740 1 Processor -- start 2024-04-24T09:01:52.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.633+0000 7f4a7c36c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5643cf14cb40 0x5643cf1518b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:52.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.633+0000 7f4a7c36c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5643cf14d460 0x5643cf151df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:52.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.633+0000 7f4a7c36c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5643cf157b30 0x5643cf152350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:52.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.633+0000 7f4a7c36c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4a741b2b50 con 0x5643cf14cb40 2024-04-24T09:01:52.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.633+0000 7f4a7c36c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4a741b2420 con 0x5643cf14d460 2024-04-24T09:01:52.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.633+0000 7f4a7c36c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4a741af650 con 0x5643cf157b30 2024-04-24T09:01:52.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.633+0000 7f4a7c36c740 10 client.0 _ll_register_callbacks cb 0x7f4a6c007c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:01:52.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.634+0000 7f4a72a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5643cf14cb40 0x5643cf1518b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:52.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.634+0000 7f4a72a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5643cf14cb40 0x5643cf1518b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45764/0 (socket says 172.21.15.83:45764) 2024-04-24T09:01:52.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.634+0000 7f4a72a00640 1 -- 172.21.15.83:0/379224864 learned_addr learned my addr 172.21.15.83:0/379224864 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:01:52.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.634+0000 7f4a72a00640 1 -- 172.21.15.83:0/379224864 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5643cf157b30 msgr2=0x5643cf152350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:52.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.634+0000 7f4a72a00640 1 --2- 172.21.15.83:0/379224864 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5643cf157b30 0x5643cf152350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:52.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.634+0000 7f4a72a00640 1 -- 172.21.15.83:0/379224864 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5643cf14d460 msgr2=0x5643cf151df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:52.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.634+0000 7f4a6be00640 1 --2- 172.21.15.83:0/379224864 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5643cf14d460 0x5643cf151df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:52.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.634+0000 7f4a72a00640 1 --2- 172.21.15.83:0/379224864 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5643cf14d460 0x5643cf151df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:52.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.634+0000 7f4a72a00640 1 -- 172.21.15.83:0/379224864 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643cf15e470 con 0x5643cf14cb40 2024-04-24T09:01:52.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.634+0000 7f4a72a00640 1 --2- 172.21.15.83:0/379224864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5643cf14cb40 0x5643cf1518b0 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x5643cf16a7c0 tx=0x7f4a64014070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ae5f2a9d1a73b83 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:52.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.634+0000 7f4a57e00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:01:52.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.635+0000 7f4a57e00640 1 -- 172.21.15.83:0/379224864 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a6402d060 con 0x5643cf14cb40 2024-04-24T09:01:52.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.635+0000 7f4a57e00640 1 -- 172.21.15.83:0/379224864 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7f4a64028d50 con 0x5643cf14cb40 2024-04-24T09:01:52.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.635+0000 7f4a7c36c740 1 -- 172.21.15.83:0/379224864 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a5c0bd710 con 0x5643cf14cb40 2024-04-24T09:01:52.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.635+0000 7f4a57e00640 1 -- 172.21.15.83:0/379224864 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f4a64034050 con 0x5643cf14cb40 2024-04-24T09:01:52.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.635+0000 7f4a7c36c740 10 client.19320 fetch_fsmap learned FSMap version 40 2024-04-24T09:01:52.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.635+0000 7f4a7c36c740 1 -- 172.21.15.83:0/379224864 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5643cf1ce620 con 0x5643cf14cb40 2024-04-24T09:01:52.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.635+0000 7f4a57e00640 1 -- 172.21.15.83:0/379224864 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4a5c0bd710 con 0x5643cf14cb40 2024-04-24T09:01:52.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.635+0000 7f4a57e00640 1 -- 172.21.15.83:0/379224864 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 40) ==== 8382+0+0 (secure 0 0 0) 0x7f4a64012910 con 0x5643cf14cb40 2024-04-24T09:01:52.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.636+0000 7f4a7c36c740 10 client.19320 fetch_fsmap finished waiting for FSMap version 40 2024-04-24T09:01:52.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.636+0000 7f4a7c36c740 10 client.19320 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:01:52.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.636+0000 7f4a7c36c740 20 client.19320 populate_metadata read hostname 'smithi083' 2024-04-24T09:01:52.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.636+0000 7f4a7c36c740 1 --2- 172.21.15.83:0/379224864 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5643cf1ced80 0x5643cf1d1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:52.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.636+0000 7f4a7c36c740 4 client.19320 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:01:52.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.636+0000 7f4a7c36c740 1 -- 172.21.15.83:0/379224864 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5643cf148cb0 con 0x5643cf1ced80 2024-04-24T09:01:52.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.636+0000 7f4a6be00640 1 --2- 172.21.15.83:0/379224864 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5643cf1ced80 0x5643cf1d1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:52.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.637+0000 7f4a57e00640 10 client.19320 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:01:52.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.637+0000 7f4a6be00640 1 --2- 172.21.15.83:0/379224864 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5643cf1ced80 0x5643cf1d1200 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3f9c44715a67b72 server_cookie=da666ed18c54ad92 in_seq=0 out_seq=0 2024-04-24T09:01:52.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.639+0000 7f4a57e00640 1 -- 172.21.15.83:0/379224864 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5643cf148cb0 con 0x5643cf1ced80 2024-04-24T09:01:52.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.639+0000 7f4a57e00640 10 client.19320 handle_command_reply: tid=0 2024-04-24T09:01:52.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.639+0000 7f4a7c36c740 1 client.19320 shutdown 2024-04-24T09:01:52.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.639+0000 7f4a7c36c740 1 -- 172.21.15.83:0/379224864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5643cf14cb40 msgr2=0x5643cf1518b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:52.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.639+0000 7f4a7c36c740 1 --2- 172.21.15.83:0/379224864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5643cf14cb40 0x5643cf1518b0 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x5643cf16a7c0 tx=0x7f4a64014070 comp rx=0 tx=0).stop 2024-04-24T09:01:52.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.639+0000 7f4a7c36c740 1 -- 172.21.15.83:0/379224864 shutdown_connections 2024-04-24T09:01:52.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.639+0000 7f4a7c36c740 1 --2- 172.21.15.83:0/379224864 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5643cf1ced80 0x5643cf1d1200 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:52.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.639+0000 7f4a7c36c740 1 --2- 172.21.15.83:0/379224864 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5643cf157b30 0x5643cf152350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:52.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.639+0000 7f4a7c36c740 1 --2- 172.21.15.83:0/379224864 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5643cf14d460 0x5643cf151df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:52.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.639+0000 7f4a57e00640 0 client.19320 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:01:52.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.639+0000 7f4a7c36c740 1 --2- 172.21.15.83:0/379224864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5643cf14cb40 0x5643cf1518b0 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:52.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.639+0000 7f4a7c36c740 1 -- 172.21.15.83:0/379224864 >> 172.21.15.83:0/379224864 conn(0x5643cf148e60 msgr2=0x5643cf147f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:52.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.639+0000 7f4a7c36c740 1 -- 172.21.15.83:0/379224864 shutdown_connections 2024-04-24T09:01:52.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.639+0000 7f4a7c36c740 1 -- 172.21.15.83:0/379224864 wait complete. 2024-04-24T09:01:52.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.640+0000 7f4a7c36c740 20 client.19320 trim_cache size 0 max 16384 2024-04-24T09:01:52.642 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:01:52.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.640+0000 7f4a73e00640 1 -- 172.21.15.83:0/131757272 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f4a580a3da0 msgr2=0x7f4a580a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:52.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.640+0000 7f4a73e00640 1 --2- 172.21.15.83:0/131757272 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f4a580a3da0 0x7f4a580a6260 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f4a64005fd0 tx=0x7f4a6400b650 comp rx=0 tx=0).stop 2024-04-24T09:01:52.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.640+0000 7f4a73e00640 1 -- 172.21.15.83:0/131757272 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a741071c0 msgr2=0x7f4a741a0b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:52.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.640+0000 7f4a73e00640 1 --2- 172.21.15.83:0/131757272 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a741071c0 0x7f4a741a0b10 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f4a5c005ab0 tx=0x7f4a5c003cd0 comp rx=0 tx=0).stop 2024-04-24T09:01:52.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.640+0000 7f4a73e00640 1 -- 172.21.15.83:0/131757272 shutdown_connections 2024-04-24T09:01:52.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.640+0000 7f4a73e00640 1 --2- 172.21.15.83:0/131757272 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f4a580a3da0 0x7f4a580a6260 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:52.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.640+0000 7f4a73e00640 1 --2- 172.21.15.83:0/131757272 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4a7419a690 0x7f4a7419ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:52.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.640+0000 7f4a73e00640 1 --2- 172.21.15.83:0/131757272 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a741071c0 0x7f4a741a0b10 unknown :-1 s=CLOSED pgs=1716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:52.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.640+0000 7f4a73e00640 1 --2- 172.21.15.83:0/131757272 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a741020f0 0x7f4a741a05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:52.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.640+0000 7f4a73e00640 1 -- 172.21.15.83:0/131757272 >> 172.21.15.83:0/131757272 conn(0x7f4a740fd9d0 msgr2=0x7f4a740ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:52.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.641+0000 7f4a73e00640 1 -- 172.21.15.83:0/131757272 shutdown_connections 2024-04-24T09:01:52.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:52.641+0000 7f4a73e00640 1 -- 172.21.15.83:0/131757272 wait complete. 2024-04-24T09:01:53.609 INFO:tasks.workunit.client.0.smithi083.stdout: 1 60082 0.63 MB/sec execute 387 sec latency 60.196 ms 2024-04-24T09:01:53.658 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T09:01:53.659 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T09:01:53.659 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T09:01:53.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.891+0000 7f67aee00640 1 -- 172.21.15.83:0/3369298688 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67a810d500 msgr2=0x7f67a810d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:53.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.891+0000 7f67aee00640 1 --2- 172.21.15.83:0/3369298688 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67a810d500 0x7f67a810d8e0 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f679c009930 tx=0x7f679c02a440 comp rx=0 tx=0).stop 2024-04-24T09:01:53.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.891+0000 7f67aee00640 1 -- 172.21.15.83:0/3369298688 shutdown_connections 2024-04-24T09:01:53.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.891+0000 7f67aee00640 1 --2- 172.21.15.83:0/3369298688 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f67a8108f40 0x7f67a81153b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:53.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.891+0000 7f67aee00640 1 --2- 172.21.15.83:0/3369298688 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f67a8108580 0x7f67a8108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:53.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.891+0000 7f67aee00640 1 --2- 172.21.15.83:0/3369298688 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67a810d500 0x7f67a810d8e0 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:53.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.891+0000 7f67aee00640 1 -- 172.21.15.83:0/3369298688 >> 172.21.15.83:0/3369298688 conn(0x7f67a8077e20 msgr2=0x7f67a8106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:53.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.891+0000 7f67aee00640 1 -- 172.21.15.83:0/3369298688 shutdown_connections 2024-04-24T09:01:53.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.891+0000 7f67aee00640 1 -- 172.21.15.83:0/3369298688 wait complete. 2024-04-24T09:01:53.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.892+0000 7f67aee00640 1 Processor -- start 2024-04-24T09:01:53.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.893+0000 7f67aee00640 1 -- start start 2024-04-24T09:01:53.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.893+0000 7f67aee00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67a8108580 0x7f67a819edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:53.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.893+0000 7f67aee00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f67a8108f40 0x7f67a819f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:53.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.893+0000 7f67aee00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f67a810d500 0x7f67a81a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:53.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.893+0000 7f67aee00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f67a81188a0 con 0x7f67a8108580 2024-04-24T09:01:53.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.893+0000 7f67aee00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f67a8118720 con 0x7f67a8108f40 2024-04-24T09:01:53.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.893+0000 7f67aee00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f67a8118a20 con 0x7f67a810d500 2024-04-24T09:01:53.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.893+0000 7f67ada00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67a8108580 0x7f67a819edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:53.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.893+0000 7f67ada00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67a8108580 0x7f67a819edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45788/0 (socket says 172.21.15.83:45788) 2024-04-24T09:01:53.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.893+0000 7f67ada00640 1 -- 172.21.15.83:0/3115412022 learned_addr learned my addr 172.21.15.83:0/3115412022 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:01:53.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.894+0000 7f67ad000640 1 --2- 172.21.15.83:0/3115412022 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f67a8108f40 0x7f67a819f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:53.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.894+0000 7f67ae400640 1 --2- 172.21.15.83:0/3115412022 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f67a810d500 0x7f67a81a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:53.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.894+0000 7f67ada00640 1 -- 172.21.15.83:0/3115412022 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f67a810d500 msgr2=0x7f67a81a3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:53.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.894+0000 7f67ada00640 1 --2- 172.21.15.83:0/3115412022 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f67a810d500 0x7f67a81a3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:53.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.894+0000 7f67ada00640 1 -- 172.21.15.83:0/3115412022 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f67a8108f40 msgr2=0x7f67a819f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:53.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.894+0000 7f67ada00640 1 --2- 172.21.15.83:0/3115412022 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f67a8108f40 0x7f67a819f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:53.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.894+0000 7f67ada00640 1 -- 172.21.15.83:0/3115412022 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67a81a3db0 con 0x7f67a8108580 2024-04-24T09:01:53.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.894+0000 7f67ada00640 1 --2- 172.21.15.83:0/3115412022 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67a8108580 0x7f67a819edc0 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f679c009900 tx=0x7f679c003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:53.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.895+0000 7f67a6000640 1 -- 172.21.15.83:0/3115412022 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f679c03f070 con 0x7f67a8108580 2024-04-24T09:01:53.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.895+0000 7f67a6000640 1 -- 172.21.15.83:0/3115412022 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f679c031d60 con 0x7f67a8108580 2024-04-24T09:01:53.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.895+0000 7f67a6000640 1 -- 172.21.15.83:0/3115412022 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f679c035030 con 0x7f67a8108580 2024-04-24T09:01:53.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.895+0000 7f67aee00640 1 -- 172.21.15.83:0/3115412022 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67a81a4040 con 0x7f67a8108580 2024-04-24T09:01:53.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.895+0000 7f67aee00640 1 -- 172.21.15.83:0/3115412022 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67a81ab930 con 0x7f67a8108580 2024-04-24T09:01:53.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.898+0000 7f67b31e5740 1 Processor -- start 2024-04-24T09:01:53.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.898+0000 7f67b31e5740 1 -- start start 2024-04-24T09:01:53.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.899+0000 7f67a6000640 1 -- 172.21.15.83:0/3115412022 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f679c042070 con 0x7f67a8108580 2024-04-24T09:01:53.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.899+0000 7f67b31e5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7ed42b270 0x55a7ed42b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:53.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.899+0000 7f67b31e5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7ed42bb90 0x55a7ed435d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:53.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.899+0000 7f67b31e5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a7ed436260 0x55a7ed438650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:53.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.899+0000 7f67b31e5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a7ed390e70 con 0x55a7ed42b270 2024-04-24T09:01:53.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.899+0000 7f67b31e5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a7ed337380 con 0x55a7ed42bb90 2024-04-24T09:01:53.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.899+0000 7f67b31e5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a7ed38ce30 con 0x55a7ed436260 2024-04-24T09:01:53.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.899+0000 7f67ae400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7ed42b270 0x55a7ed42b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:53.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.899+0000 7f67ae400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7ed42b270 0x55a7ed42b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45794/0 (socket says 172.21.15.83:45794) 2024-04-24T09:01:53.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.899+0000 7f67ae400640 1 -- 172.21.15.83:0/1856253995 learned_addr learned my addr 172.21.15.83:0/1856253995 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:01:53.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.899+0000 7f67ada00640 1 --2- 172.21.15.83:0/1856253995 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7ed42bb90 0x55a7ed435d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:53.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.900+0000 7f67ad000640 1 --2- 172.21.15.83:0/1856253995 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a7ed436260 0x55a7ed438650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:53.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.900+0000 7f67ae400640 1 -- 172.21.15.83:0/1856253995 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a7ed436260 msgr2=0x55a7ed438650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:53.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.900+0000 7f67ae400640 1 --2- 172.21.15.83:0/1856253995 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a7ed436260 0x55a7ed438650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:53.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.900+0000 7f67ae400640 1 -- 172.21.15.83:0/1856253995 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7ed42bb90 msgr2=0x55a7ed435d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:53.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.900+0000 7f67ae400640 1 --2- 172.21.15.83:0/1856253995 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7ed42bb90 0x55a7ed435d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:53.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.900+0000 7f67ae400640 1 -- 172.21.15.83:0/1856253995 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7ed16b7e0 con 0x55a7ed42b270 2024-04-24T09:01:53.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.900+0000 7f67ae400640 1 --2- 172.21.15.83:0/1856253995 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7ed42b270 0x55a7ed42b650 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f67a0009f10 tx=0x7f67a000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14d94d44395aba57 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:53.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.900+0000 7f679aa00640 1 -- 172.21.15.83:0/1856253995 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a001a070 con 0x55a7ed42b270 2024-04-24T09:01:53.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.900+0000 7f679aa00640 1 -- 172.21.15.83:0/1856253995 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f67a00158c0 con 0x55a7ed42b270 2024-04-24T09:01:53.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.900+0000 7f679aa00640 1 -- 172.21.15.83:0/1856253995 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a000a800 con 0x55a7ed42b270 2024-04-24T09:01:53.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.900+0000 7f67a6000640 1 --2- 172.21.15.83:0/3115412022 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f67900a3d20 0x7f67900a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:53.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.901+0000 7f67a6000640 1 -- 172.21.15.83:0/3115412022 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7f679c0f12c0 con 0x7f67a8108580 2024-04-24T09:01:53.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.901+0000 7f67ada00640 1 --2- 172.21.15.83:0/3115412022 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f67900a3d20 0x7f67900a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:53.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.901+0000 7f67b31e5740 1 -- 172.21.15.83:0/1856253995 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7ed42b270 msgr2=0x55a7ed42b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:53.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.901+0000 7f67b31e5740 1 --2- 172.21.15.83:0/1856253995 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7ed42b270 0x55a7ed42b650 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f67a0009f10 tx=0x7f67a000c7f0 comp rx=0 tx=0).stop 2024-04-24T09:01:53.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.902+0000 7f67b31e5740 1 -- 172.21.15.83:0/1856253995 shutdown_connections 2024-04-24T09:01:53.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.902+0000 7f67b31e5740 1 --2- 172.21.15.83:0/1856253995 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a7ed436260 0x55a7ed438650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:53.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.902+0000 7f67ada00640 1 --2- 172.21.15.83:0/3115412022 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f67900a3d20 0x7f67900a61e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x55a7ed42c830 tx=0x7f679c005e00 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:53.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.902+0000 7f67b31e5740 1 --2- 172.21.15.83:0/1856253995 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7ed42bb90 0x55a7ed435d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:53.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.902+0000 7f67b31e5740 1 --2- 172.21.15.83:0/1856253995 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7ed42b270 0x55a7ed42b650 unknown :-1 s=CLOSED pgs=1723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:53.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.902+0000 7f67b31e5740 1 -- 172.21.15.83:0/1856253995 >> 172.21.15.83:0/1856253995 conn(0x55a7ed427590 msgr2=0x55a7ed427970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:53.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.902+0000 7f67b31e5740 1 -- 172.21.15.83:0/1856253995 shutdown_connections 2024-04-24T09:01:53.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.902+0000 7f67b31e5740 1 -- 172.21.15.83:0/1856253995 wait complete. 2024-04-24T09:01:53.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.902+0000 7f67b31e5740 1 -- start start 2024-04-24T09:01:53.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.902+0000 7f67b31e5740 1 Processor -- start 2024-04-24T09:01:53.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.903+0000 7f67b31e5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7ed42b270 0x55a7ed4331f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:53.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.903+0000 7f67b31e5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7ed42bb90 0x55a7ed433730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:53.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.903+0000 7f67b31e5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a7ed436260 0x55a7ed433c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:53.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.903+0000 7f67b31e5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a7ed439050 con 0x55a7ed42b270 2024-04-24T09:01:53.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.903+0000 7f67b31e5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a7ed34a660 con 0x55a7ed42bb90 2024-04-24T09:01:53.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.903+0000 7f67b31e5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a7ed4391d0 con 0x55a7ed436260 2024-04-24T09:01:53.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.903+0000 7f67b31e5740 10 client.0 _ll_register_callbacks cb 0x55a7ed338aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:01:53.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.903+0000 7f67ae400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7ed42b270 0x55a7ed4331f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:53.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.903+0000 7f67ae400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7ed42b270 0x55a7ed4331f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45802/0 (socket says 172.21.15.83:45802) 2024-04-24T09:01:53.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.903+0000 7f67ae400640 1 -- 172.21.15.83:0/884203580 learned_addr learned my addr 172.21.15.83:0/884203580 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:01:53.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.904+0000 7f67ad000640 1 --2- 172.21.15.83:0/884203580 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7ed42bb90 0x55a7ed433730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:53.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.904+0000 7f67ae400640 1 -- 172.21.15.83:0/884203580 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a7ed436260 msgr2=0x55a7ed433c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:53.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.904+0000 7f67ae400640 1 --2- 172.21.15.83:0/884203580 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a7ed436260 0x55a7ed433c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:53.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.904+0000 7f67ae400640 1 -- 172.21.15.83:0/884203580 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7ed42bb90 msgr2=0x55a7ed433730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:53.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.904+0000 7f67ae400640 1 --2- 172.21.15.83:0/884203580 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7ed42bb90 0x55a7ed433730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:53.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.904+0000 7f67ae400640 1 -- 172.21.15.83:0/884203580 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7ed1cbf70 con 0x55a7ed42b270 2024-04-24T09:01:53.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.904+0000 7f67ae400640 1 --2- 172.21.15.83:0/884203580 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7ed42b270 0x55a7ed4331f0 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f67a000cfa0 tx=0x7f67a0009ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75bfc69dac1404b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:53.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.904+0000 7f678f400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:01:53.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.904+0000 7f678f400640 1 -- 172.21.15.83:0/884203580 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a001a030 con 0x55a7ed42b270 2024-04-24T09:01:53.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.904+0000 7f678f400640 1 -- 172.21.15.83:0/884203580 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7f67a0012040 con 0x55a7ed42b270 2024-04-24T09:01:53.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.904+0000 7f67b31e5740 1 -- 172.21.15.83:0/884203580 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67a00158c0 con 0x55a7ed42b270 2024-04-24T09:01:53.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.905+0000 7f678f400640 1 -- 172.21.15.83:0/884203580 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f67a002e050 con 0x55a7ed42b270 2024-04-24T09:01:53.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.905+0000 7f67b31e5740 10 client.19332 fetch_fsmap learned FSMap version 40 2024-04-24T09:01:53.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.905+0000 7f67b31e5740 1 -- 172.21.15.83:0/884203580 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55a7ed063170 con 0x55a7ed42b270 2024-04-24T09:01:53.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.905+0000 7f678f400640 1 -- 172.21.15.83:0/884203580 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f67a00158c0 con 0x55a7ed42b270 2024-04-24T09:01:53.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.905+0000 7f678f400640 1 -- 172.21.15.83:0/884203580 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 40) ==== 8382+0+0 (secure 0 0 0) 0x7f67a0020580 con 0x55a7ed42b270 2024-04-24T09:01:53.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.905+0000 7f67b31e5740 10 client.19332 fetch_fsmap finished waiting for FSMap version 40 2024-04-24T09:01:53.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.905+0000 7f67b31e5740 10 client.19332 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:01:53.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.905+0000 7f67b31e5740 20 client.19332 populate_metadata read hostname 'smithi083' 2024-04-24T09:01:53.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.905+0000 7f67b31e5740 1 --2- 172.21.15.83:0/884203580 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55a7ed4464a0 0x55a7ed448890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:53.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.905+0000 7f67b31e5740 4 client.19332 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:01:53.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.905+0000 7f67b31e5740 1 -- 172.21.15.83:0/884203580 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a7ed39e890 con 0x55a7ed4464a0 2024-04-24T09:01:53.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.906+0000 7f67ad000640 1 --2- 172.21.15.83:0/884203580 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55a7ed4464a0 0x55a7ed448890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:53.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.907+0000 7f678f400640 10 client.19332 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:01:53.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.907+0000 7f67ad000640 1 --2- 172.21.15.83:0/884203580 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55a7ed4464a0 0x55a7ed448890 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3cad0c09172f18c9 server_cookie=535f48e5dbda209a in_seq=0 out_seq=0 2024-04-24T09:01:53.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.910+0000 7f678f400640 1 -- 172.21.15.83:0/884203580 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a7ed39e890 con 0x55a7ed4464a0 2024-04-24T09:01:53.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.910+0000 7f678f400640 10 client.19332 handle_command_reply: tid=0 2024-04-24T09:01:53.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.910+0000 7f67b31e5740 1 client.19332 shutdown 2024-04-24T09:01:53.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.910+0000 7f67b31e5740 1 -- 172.21.15.83:0/884203580 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7ed42b270 msgr2=0x55a7ed4331f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:53.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.910+0000 7f67b31e5740 1 --2- 172.21.15.83:0/884203580 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7ed42b270 0x55a7ed4331f0 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f67a000cfa0 tx=0x7f67a0009ab0 comp rx=0 tx=0).stop 2024-04-24T09:01:53.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.910+0000 7f67b31e5740 1 -- 172.21.15.83:0/884203580 shutdown_connections 2024-04-24T09:01:53.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.910+0000 7f67b31e5740 1 --2- 172.21.15.83:0/884203580 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55a7ed4464a0 0x55a7ed448890 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:53.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.910+0000 7f67b31e5740 1 --2- 172.21.15.83:0/884203580 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a7ed436260 0x55a7ed433c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:53.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.910+0000 7f67b31e5740 1 --2- 172.21.15.83:0/884203580 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7ed42bb90 0x55a7ed433730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:53.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.910+0000 7f67b31e5740 1 --2- 172.21.15.83:0/884203580 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7ed42b270 0x55a7ed4331f0 unknown :-1 s=CLOSED pgs=1724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:53.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.910+0000 7f678f400640 0 client.19332 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:01:53.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.910+0000 7f67b31e5740 1 -- 172.21.15.83:0/884203580 >> 172.21.15.83:0/884203580 conn(0x55a7ed427590 msgr2=0x55a7ed42d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:53.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.910+0000 7f67b31e5740 1 -- 172.21.15.83:0/884203580 shutdown_connections 2024-04-24T09:01:53.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.910+0000 7f67b31e5740 1 -- 172.21.15.83:0/884203580 wait complete. 2024-04-24T09:01:53.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.911+0000 7f67b31e5740 20 client.19332 trim_cache size 0 max 16384 2024-04-24T09:01:53.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.931+0000 7f67b31e5740 1 Processor -- start 2024-04-24T09:01:53.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.931+0000 7f67b31e5740 1 -- start start 2024-04-24T09:01:53.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.931+0000 7f67b31e5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a7ed42b270 0x55a7ed441f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:53.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.931+0000 7f67b31e5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7ed42bb90 0x55a7ed42d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:53.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.931+0000 7f67b31e5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7ed436260 0x55a7ed426570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:53.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.931+0000 7f67b31e5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f67a81b5b50 con 0x55a7ed436260 2024-04-24T09:01:53.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.931+0000 7f67b31e5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f67a81b7780 con 0x55a7ed42bb90 2024-04-24T09:01:53.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.931+0000 7f67b31e5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f67a81b6220 con 0x55a7ed42b270 2024-04-24T09:01:53.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.932+0000 7f67ae400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7ed436260 0x55a7ed426570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:53.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.932+0000 7f67ae400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7ed436260 0x55a7ed426570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45810/0 (socket says 172.21.15.83:45810) 2024-04-24T09:01:53.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.932+0000 7f67ae400640 1 -- 172.21.15.83:0/2520877869 learned_addr learned my addr 172.21.15.83:0/2520877869 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:01:53.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.932+0000 7f67ae400640 1 --2- 172.21.15.83:0/2520877869 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a7ed42b270 0x55a7ed441f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:53.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.932+0000 7f67ad000640 1 --2- 172.21.15.83:0/2520877869 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7ed42bb90 0x55a7ed42d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:53.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.933+0000 7f67ae400640 1 -- 172.21.15.83:0/2520877869 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a7ed42b270 msgr2=0x55a7ed441f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:53.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.933+0000 7f67ae400640 1 --2- 172.21.15.83:0/2520877869 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a7ed42b270 0x55a7ed441f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:53.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.933+0000 7f67ae400640 1 -- 172.21.15.83:0/2520877869 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7ed42bb90 msgr2=0x55a7ed42d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:53.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.933+0000 7f67ae400640 1 --2- 172.21.15.83:0/2520877869 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7ed42bb90 0x55a7ed42d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:53.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.933+0000 7f67ae400640 1 -- 172.21.15.83:0/2520877869 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7ed4389c0 con 0x55a7ed436260 2024-04-24T09:01:53.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.933+0000 7f67ae400640 1 --2- 172.21.15.83:0/2520877869 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7ed436260 0x55a7ed426570 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f67a0020920 tx=0x7f67a000e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff406ea3c141bf5e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:53.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.934+0000 7f679a000640 1 -- 172.21.15.83:0/2520877869 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a001a050 con 0x55a7ed436260 2024-04-24T09:01:53.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.934+0000 7f679a000640 1 -- 172.21.15.83:0/2520877869 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f67a001b040 con 0x55a7ed436260 2024-04-24T09:01:53.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.934+0000 7f679a000640 1 -- 172.21.15.83:0/2520877869 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a000a6f0 con 0x55a7ed436260 2024-04-24T09:01:53.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.934+0000 7f67b31e5740 1 -- 172.21.15.83:0/2520877869 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7ed436260 msgr2=0x55a7ed426570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:53.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.934+0000 7f67b31e5740 1 --2- 172.21.15.83:0/2520877869 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7ed436260 0x55a7ed426570 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f67a0020920 tx=0x7f67a000e040 comp rx=0 tx=0).stop 2024-04-24T09:01:53.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.935+0000 7f67b31e5740 1 -- 172.21.15.83:0/2520877869 shutdown_connections 2024-04-24T09:01:53.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.935+0000 7f67b31e5740 1 --2- 172.21.15.83:0/2520877869 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7ed436260 0x55a7ed426570 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:53.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.935+0000 7f67b31e5740 1 --2- 172.21.15.83:0/2520877869 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7ed42bb90 0x55a7ed42d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:53.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.935+0000 7f67b31e5740 1 --2- 172.21.15.83:0/2520877869 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a7ed42b270 0x55a7ed441f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:53.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.935+0000 7f67b31e5740 1 -- 172.21.15.83:0/2520877869 >> 172.21.15.83:0/2520877869 conn(0x55a7ed427590 msgr2=0x55a7ed425e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:53.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.935+0000 7f67b31e5740 1 -- 172.21.15.83:0/2520877869 shutdown_connections 2024-04-24T09:01:53.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.935+0000 7f67b31e5740 1 -- 172.21.15.83:0/2520877869 wait complete. 2024-04-24T09:01:53.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.935+0000 7f67b31e5740 1 -- start start 2024-04-24T09:01:53.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.935+0000 7f67b31e5740 1 Processor -- start 2024-04-24T09:01:53.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.936+0000 7f67b31e5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7ed42b270 0x55a7ed430040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:53.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.936+0000 7f67b31e5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a7ed42bb90 0x55a7ed430580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:53.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.936+0000 7f67b31e5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7ed436260 0x55a7ed430ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:53.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.936+0000 7f67b31e5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f67a81b7050 con 0x55a7ed42b270 2024-04-24T09:01:53.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.936+0000 7f67b31e5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f67a81b6920 con 0x55a7ed436260 2024-04-24T09:01:53.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.936+0000 7f67b31e5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f67a81b3b50 con 0x55a7ed42bb90 2024-04-24T09:01:53.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.936+0000 7f67b31e5740 10 client.0 _ll_register_callbacks cb 0x7f67a0018070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:01:53.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.936+0000 7f67ae400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7ed42b270 0x55a7ed430040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:53.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.936+0000 7f67ae400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7ed42b270 0x55a7ed430040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45822/0 (socket says 172.21.15.83:45822) 2024-04-24T09:01:53.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.936+0000 7f67ae400640 1 -- 172.21.15.83:0/709504141 learned_addr learned my addr 172.21.15.83:0/709504141 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:01:53.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.936+0000 7f67ad000640 1 --2- 172.21.15.83:0/709504141 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a7ed42bb90 0x55a7ed430580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:53.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.936+0000 7f67ae400640 1 -- 172.21.15.83:0/709504141 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a7ed42bb90 msgr2=0x55a7ed430580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:53.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.936+0000 7f67ae400640 1 --2- 172.21.15.83:0/709504141 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a7ed42bb90 0x55a7ed430580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:53.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.936+0000 7f67ae400640 1 -- 172.21.15.83:0/709504141 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7ed436260 msgr2=0x55a7ed430ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:53.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.936+0000 7f67ae400640 1 --2- 172.21.15.83:0/709504141 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7ed436260 0x55a7ed430ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:53.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.936+0000 7f67ae400640 1 -- 172.21.15.83:0/709504141 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7ed4ab910 con 0x55a7ed42b270 2024-04-24T09:01:53.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.937+0000 7f67ae400640 1 --2- 172.21.15.83:0/709504141 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7ed42b270 0x55a7ed430040 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f67a0020b10 tx=0x7f67a00093c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e26f8ee80a63c177 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:53.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.937+0000 7f678f400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:01:53.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.937+0000 7f678f400640 1 -- 172.21.15.83:0/709504141 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a0042030 con 0x55a7ed42b270 2024-04-24T09:01:53.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.937+0000 7f678f400640 1 -- 172.21.15.83:0/709504141 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7f67a0012050 con 0x55a7ed42b270 2024-04-24T09:01:53.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.937+0000 7f67b31e5740 1 -- 172.21.15.83:0/709504141 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67a001b040 con 0x55a7ed42b270 2024-04-24T09:01:53.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.937+0000 7f678f400640 1 -- 172.21.15.83:0/709504141 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f67a004c050 con 0x55a7ed42b270 2024-04-24T09:01:53.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.937+0000 7f67b31e5740 10 client.19338 fetch_fsmap learned FSMap version 40 2024-04-24T09:01:53.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.937+0000 7f67b31e5740 1 -- 172.21.15.83:0/709504141 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55a7ed4acc30 con 0x55a7ed42b270 2024-04-24T09:01:53.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.938+0000 7f678f400640 1 -- 172.21.15.83:0/709504141 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f67a001b040 con 0x55a7ed42b270 2024-04-24T09:01:53.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.938+0000 7f678f400640 1 -- 172.21.15.83:0/709504141 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 40) ==== 8382+0+0 (secure 0 0 0) 0x7f67a000a880 con 0x55a7ed42b270 2024-04-24T09:01:53.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.938+0000 7f67b31e5740 10 client.19338 fetch_fsmap finished waiting for FSMap version 40 2024-04-24T09:01:53.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.938+0000 7f67b31e5740 10 client.19338 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:01:53.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.938+0000 7f67b31e5740 20 client.19338 populate_metadata read hostname 'smithi083' 2024-04-24T09:01:53.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.938+0000 7f67b31e5740 1 --2- 172.21.15.83:0/709504141 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55a7ed4ad310 0x55a7ed4af730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:53.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.938+0000 7f67b31e5740 4 client.19338 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T09:01:53.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.938+0000 7f67b31e5740 1 -- 172.21.15.83:0/709504141 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a7ed398700 con 0x55a7ed4ad310 2024-04-24T09:01:53.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.939+0000 7f67ad000640 1 --2- 172.21.15.83:0/709504141 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55a7ed4ad310 0x55a7ed4af730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:53.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.940+0000 7f67ad000640 1 --2- 172.21.15.83:0/709504141 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55a7ed4ad310 0x55a7ed4af730 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=feb3a74e8e8fa13c server_cookie=4d97c6239844c6f3 in_seq=0 out_seq=0 2024-04-24T09:01:53.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.940+0000 7f678f400640 10 client.19338 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:01:53.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.943+0000 7f678f400640 1 -- 172.21.15.83:0/709504141 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a7ed398700 con 0x55a7ed4ad310 2024-04-24T09:01:53.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.943+0000 7f678f400640 10 client.19338 handle_command_reply: tid=0 2024-04-24T09:01:53.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.943+0000 7f67b31e5740 1 client.19338 shutdown 2024-04-24T09:01:53.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.943+0000 7f67b31e5740 1 -- 172.21.15.83:0/709504141 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7ed42b270 msgr2=0x55a7ed430040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:53.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.943+0000 7f67b31e5740 1 --2- 172.21.15.83:0/709504141 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7ed42b270 0x55a7ed430040 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f67a0020b10 tx=0x7f67a00093c0 comp rx=0 tx=0).stop 2024-04-24T09:01:53.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.943+0000 7f67b31e5740 1 -- 172.21.15.83:0/709504141 shutdown_connections 2024-04-24T09:01:53.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.943+0000 7f67b31e5740 1 --2- 172.21.15.83:0/709504141 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55a7ed4ad310 0x55a7ed4af730 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:53.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.943+0000 7f67b31e5740 1 --2- 172.21.15.83:0/709504141 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7ed436260 0x55a7ed430ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:53.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.943+0000 7f67b31e5740 1 --2- 172.21.15.83:0/709504141 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a7ed42bb90 0x55a7ed430580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:53.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.943+0000 7f678f400640 0 client.19338 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:01:53.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.943+0000 7f67b31e5740 1 --2- 172.21.15.83:0/709504141 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7ed42b270 0x55a7ed430040 unknown :-1 s=CLOSED pgs=1726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:53.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.943+0000 7f67b31e5740 1 -- 172.21.15.83:0/709504141 >> 172.21.15.83:0/709504141 conn(0x55a7ed427590 msgr2=0x55a7ed433a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:53.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.943+0000 7f67b31e5740 1 -- 172.21.15.83:0/709504141 shutdown_connections 2024-04-24T09:01:53.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.943+0000 7f67b31e5740 1 -- 172.21.15.83:0/709504141 wait complete. 2024-04-24T09:01:53.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.944+0000 7f67b31e5740 20 client.19338 trim_cache size 0 max 16384 2024-04-24T09:01:53.945 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T09:01:53.945 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-04-24T09:01:53.945 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "720d5389-8ab6-4d54-8750-f06c623b56c6", 2024-04-24T09:01:53.945 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-04-24T09:01:53.945 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T09:01:53.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.944+0000 7f67aee00640 1 -- 172.21.15.83:0/3115412022 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f67900a3d20 msgr2=0x7f67900a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:53.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.944+0000 7f67aee00640 1 --2- 172.21.15.83:0/3115412022 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f67900a3d20 0x7f67900a61e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x55a7ed42c830 tx=0x7f679c005e00 comp rx=0 tx=0).stop 2024-04-24T09:01:53.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.944+0000 7f67aee00640 1 -- 172.21.15.83:0/3115412022 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67a8108580 msgr2=0x7f67a819edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:53.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.944+0000 7f67aee00640 1 --2- 172.21.15.83:0/3115412022 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67a8108580 0x7f67a819edc0 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f679c009900 tx=0x7f679c003d50 comp rx=0 tx=0).stop 2024-04-24T09:01:53.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.944+0000 7f67aee00640 1 -- 172.21.15.83:0/3115412022 shutdown_connections 2024-04-24T09:01:53.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.944+0000 7f67aee00640 1 --2- 172.21.15.83:0/3115412022 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f67900a3d20 0x7f67900a61e0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:53.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.944+0000 7f67aee00640 1 --2- 172.21.15.83:0/3115412022 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f67a810d500 0x7f67a81a3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:53.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.944+0000 7f67aee00640 1 --2- 172.21.15.83:0/3115412022 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f67a8108f40 0x7f67a819f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:53.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.944+0000 7f67aee00640 1 --2- 172.21.15.83:0/3115412022 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67a8108580 0x7f67a819edc0 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:53.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.944+0000 7f67aee00640 1 -- 172.21.15.83:0/3115412022 >> 172.21.15.83:0/3115412022 conn(0x7f67a8077e20 msgr2=0x7f67a8078440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:53.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.945+0000 7f67aee00640 1 -- 172.21.15.83:0/3115412022 shutdown_connections 2024-04-24T09:01:53.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:53.945+0000 7f67aee00640 1 -- 172.21.15.83:0/3115412022 wait complete. 2024-04-24T09:01:53.962 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T09:01:53.963 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T09:01:54.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.166+0000 7fe48f200640 1 -- 172.21.15.83:0/3096711785 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe490108f40 msgr2=0x7fe4901153b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:54.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.166+0000 7fe48f200640 1 --2- 172.21.15.83:0/3096711785 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe490108f40 0x7fe4901153b0 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7fe488009cd0 tx=0x7fe48802a8f0 comp rx=0 tx=0).stop 2024-04-24T09:01:54.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.167+0000 7fe48f200640 1 -- 172.21.15.83:0/3096711785 shutdown_connections 2024-04-24T09:01:54.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.167+0000 7fe48f200640 1 --2- 172.21.15.83:0/3096711785 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe490108f40 0x7fe4901153b0 unknown :-1 s=CLOSED pgs=1727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:54.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.167+0000 7fe48f200640 1 --2- 172.21.15.83:0/3096711785 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe490108580 0x7fe490108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:54.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.167+0000 7fe48f200640 1 --2- 172.21.15.83:0/3096711785 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe49010d500 0x7fe49010d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:54.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.167+0000 7fe48f200640 1 -- 172.21.15.83:0/3096711785 >> 172.21.15.83:0/3096711785 conn(0x7fe490077e20 msgr2=0x7fe490106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:54.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.167+0000 7fe48f200640 1 -- 172.21.15.83:0/3096711785 shutdown_connections 2024-04-24T09:01:54.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.167+0000 7fe48f200640 1 -- 172.21.15.83:0/3096711785 wait complete. 2024-04-24T09:01:54.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.169+0000 7fe48f200640 1 Processor -- start 2024-04-24T09:01:54.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.169+0000 7fe48f200640 1 -- start start 2024-04-24T09:01:54.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.169+0000 7fe48f200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe490108580 0x7fe4901a4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:54.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.169+0000 7fe48f200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe490108f40 0x7fe4901a4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:54.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.169+0000 7fe48f200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe49010d500 0x7fe49019ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:54.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.169+0000 7fe48f200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe490118780 con 0x7fe490108f40 2024-04-24T09:01:54.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.169+0000 7fe48f200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe490118600 con 0x7fe49010d500 2024-04-24T09:01:54.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.169+0000 7fe48f200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe490118900 con 0x7fe490108580 2024-04-24T09:01:54.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.169+0000 7fe487e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe490108f40 0x7fe4901a4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:54.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.169+0000 7fe487e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe490108f40 0x7fe4901a4dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45834/0 (socket says 172.21.15.83:45834) 2024-04-24T09:01:54.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.169+0000 7fe487e00640 1 -- 172.21.15.83:0/1537601803 learned_addr learned my addr 172.21.15.83:0/1537601803 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:01:54.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.170+0000 7fe48de00640 1 --2- 172.21.15.83:0/1537601803 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe490108580 0x7fe4901a4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:54.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.170+0000 7fe487e00640 1 -- 172.21.15.83:0/1537601803 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe490108580 msgr2=0x7fe4901a4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:54.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.170+0000 7fe487e00640 1 --2- 172.21.15.83:0/1537601803 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe490108580 0x7fe4901a4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:54.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.170+0000 7fe48e800640 1 --2- 172.21.15.83:0/1537601803 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe49010d500 0x7fe49019ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:54.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.170+0000 7fe487e00640 1 -- 172.21.15.83:0/1537601803 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe49010d500 msgr2=0x7fe49019ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:54.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.170+0000 7fe487e00640 1 --2- 172.21.15.83:0/1537601803 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe49010d500 0x7fe49019ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:54.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.170+0000 7fe487e00640 1 -- 172.21.15.83:0/1537601803 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe49019f200 con 0x7fe490108f40 2024-04-24T09:01:54.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.170+0000 7fe487e00640 1 --2- 172.21.15.83:0/1537601803 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe490108f40 0x7fe4901a4dc0 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7fe47800d950 tx=0x7fe47800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:54.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.173+0000 7fe486a00640 1 -- 172.21.15.83:0/1537601803 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe478017070 con 0x7fe490108f40 2024-04-24T09:01:54.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.173+0000 7fe486a00640 1 -- 172.21.15.83:0/1537601803 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe478009c30 con 0x7fe490108f40 2024-04-24T09:01:54.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.173+0000 7fe48f200640 1 -- 172.21.15.83:0/1537601803 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe49019f4f0 con 0x7fe490108f40 2024-04-24T09:01:54.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.173+0000 7fe486a00640 1 -- 172.21.15.83:0/1537601803 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe47800f620 con 0x7fe490108f40 2024-04-24T09:01:54.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.173+0000 7fe48f200640 1 -- 172.21.15.83:0/1537601803 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4901ab740 con 0x7fe490108f40 2024-04-24T09:01:54.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.177+0000 7fe486a00640 1 -- 172.21.15.83:0/1537601803 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe4780109a0 con 0x7fe490108f40 2024-04-24T09:01:54.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.177+0000 7fe497da8740 1 Processor -- start 2024-04-24T09:01:54.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.177+0000 7fe497da8740 1 -- start start 2024-04-24T09:01:54.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.178+0000 7fe497da8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d4a7b0a30 0x555d4a7b0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:54.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.178+0000 7fe497da8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555d4a7b1350 0x555d4a7bb4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:54.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.178+0000 7fe497da8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555d4a7bba20 0x555d4a7bde10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:54.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.178+0000 7fe497da8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x555d4a7161f0 con 0x555d4a7b1350 2024-04-24T09:01:54.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.178+0000 7fe497da8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555d4a6bc350 con 0x555d4a7b0a30 2024-04-24T09:01:54.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.178+0000 7fe497da8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x555d4a7121b0 con 0x555d4a7bba20 2024-04-24T09:01:54.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.178+0000 7fe48de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555d4a7b1350 0x555d4a7bb4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:54.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.178+0000 7fe48de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555d4a7b1350 0x555d4a7bb4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45842/0 (socket says 172.21.15.83:45842) 2024-04-24T09:01:54.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.178+0000 7fe48de00640 1 -- 172.21.15.83:0/1089419854 learned_addr learned my addr 172.21.15.83:0/1089419854 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:01:54.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.178+0000 7fe48e800640 1 --2- 172.21.15.83:0/1089419854 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d4a7b0a30 0x555d4a7b0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:54.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.178+0000 7fe48de00640 1 -- 172.21.15.83:0/1089419854 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555d4a7bba20 msgr2=0x555d4a7bde10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:54.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.178+0000 7fe48de00640 1 --2- 172.21.15.83:0/1089419854 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555d4a7bba20 0x555d4a7bde10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:54.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.178+0000 7fe48de00640 1 -- 172.21.15.83:0/1089419854 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d4a7b0a30 msgr2=0x555d4a7b0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:54.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.178+0000 7fe48de00640 1 --2- 172.21.15.83:0/1089419854 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d4a7b0a30 0x555d4a7b0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:54.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.178+0000 7fe48de00640 1 -- 172.21.15.83:0/1089419854 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d4a710f50 con 0x555d4a7b1350 2024-04-24T09:01:54.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.178+0000 7fe48e800640 1 --2- 172.21.15.83:0/1089419854 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d4a7b0a30 0x555d4a7b0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:01:54.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.179+0000 7fe48de00640 1 --2- 172.21.15.83:0/1089419854 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555d4a7b1350 0x555d4a7bb4e0 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7fe48000bc60 tx=0x7fe48000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2388588b8ab9a292 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:54.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.179+0000 7fe47f400640 1 -- 172.21.15.83:0/1089419854 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe480015070 con 0x555d4a7b1350 2024-04-24T09:01:54.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.179+0000 7fe47f400640 1 -- 172.21.15.83:0/1089419854 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe48001a910 con 0x555d4a7b1350 2024-04-24T09:01:54.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.179+0000 7fe47f400640 1 -- 172.21.15.83:0/1089419854 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe480011880 con 0x555d4a7b1350 2024-04-24T09:01:54.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.179+0000 7fe486a00640 1 --2- 172.21.15.83:0/1537601803 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe4740a3d20 0x7fe4740a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:54.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.179+0000 7fe486a00640 1 -- 172.21.15.83:0/1537601803 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7fe4780d0930 con 0x7fe490108f40 2024-04-24T09:01:54.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.179+0000 7fe48e800640 1 --2- 172.21.15.83:0/1537601803 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe4740a3d20 0x7fe4740a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:54.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.180+0000 7fe497da8740 1 -- 172.21.15.83:0/1089419854 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555d4a7b1350 msgr2=0x555d4a7bb4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:54.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.180+0000 7fe497da8740 1 --2- 172.21.15.83:0/1089419854 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555d4a7b1350 0x555d4a7bb4e0 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7fe48000bc60 tx=0x7fe48000f730 comp rx=0 tx=0).stop 2024-04-24T09:01:54.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.180+0000 7fe497da8740 1 -- 172.21.15.83:0/1089419854 shutdown_connections 2024-04-24T09:01:54.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.180+0000 7fe497da8740 1 --2- 172.21.15.83:0/1089419854 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555d4a7bba20 0x555d4a7bde10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:54.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.180+0000 7fe497da8740 1 --2- 172.21.15.83:0/1089419854 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555d4a7b1350 0x555d4a7bb4e0 unknown :-1 s=CLOSED pgs=1729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:54.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.180+0000 7fe497da8740 1 --2- 172.21.15.83:0/1089419854 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d4a7b0a30 0x555d4a7b0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:54.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.180+0000 7fe497da8740 1 -- 172.21.15.83:0/1089419854 >> 172.21.15.83:0/1089419854 conn(0x555d4a7acd50 msgr2=0x555d4a7ad130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:54.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.180+0000 7fe497da8740 1 -- 172.21.15.83:0/1089419854 shutdown_connections 2024-04-24T09:01:54.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.180+0000 7fe48e800640 1 --2- 172.21.15.83:0/1537601803 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe4740a3d20 0x7fe4740a61e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe488031040 tx=0x7fe488005ef0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:54.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.180+0000 7fe497da8740 1 -- 172.21.15.83:0/1089419854 wait complete. 2024-04-24T09:01:54.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.181+0000 7fe497da8740 1 -- start start 2024-04-24T09:01:54.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.181+0000 7fe497da8740 1 Processor -- start 2024-04-24T09:01:54.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.182+0000 7fe497da8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555d4a7b0a30 0x555d4a7b8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:54.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.182+0000 7fe497da8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555d4a7b1350 0x555d4a7b8f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:54.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.182+0000 7fe497da8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d4a7bba20 0x555d4a7b9490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:54.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.182+0000 7fe497da8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x555d4a7be810 con 0x555d4a7b0a30 2024-04-24T09:01:54.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.182+0000 7fe497da8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555d4a6cf690 con 0x555d4a7bba20 2024-04-24T09:01:54.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.182+0000 7fe497da8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x555d4a7be990 con 0x555d4a7b1350 2024-04-24T09:01:54.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.182+0000 7fe497da8740 10 client.0 _ll_register_callbacks cb 0x555d4a70b650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:01:54.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.182+0000 7fe48de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555d4a7b0a30 0x555d4a7b8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:54.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.182+0000 7fe48de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555d4a7b0a30 0x555d4a7b8a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45850/0 (socket says 172.21.15.83:45850) 2024-04-24T09:01:54.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.182+0000 7fe48de00640 1 -- 172.21.15.83:0/592974820 learned_addr learned my addr 172.21.15.83:0/592974820 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:01:54.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.182+0000 7fe487e00640 1 --2- 172.21.15.83:0/592974820 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555d4a7b1350 0x555d4a7b8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:54.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.182+0000 7fe48de00640 1 -- 172.21.15.83:0/592974820 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555d4a7b1350 msgr2=0x555d4a7b8f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:54.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.182+0000 7fe48e800640 1 --2- 172.21.15.83:0/592974820 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d4a7bba20 0x555d4a7b9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:54.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.182+0000 7fe48de00640 1 --2- 172.21.15.83:0/592974820 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555d4a7b1350 0x555d4a7b8f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:54.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.182+0000 7fe48de00640 1 -- 172.21.15.83:0/592974820 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d4a7bba20 msgr2=0x555d4a7b9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:54.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.182+0000 7fe48de00640 1 --2- 172.21.15.83:0/592974820 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d4a7bba20 0x555d4a7b9490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:54.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.182+0000 7fe48de00640 1 -- 172.21.15.83:0/592974820 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d4a6ce920 con 0x555d4a7b0a30 2024-04-24T09:01:54.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.182+0000 7fe487e00640 1 --2- 172.21.15.83:0/592974820 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555d4a7b1350 0x555d4a7b8f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:01:54.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.182+0000 7fe48e800640 1 --2- 172.21.15.83:0/592974820 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d4a7bba20 0x555d4a7b9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:01:54.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.183+0000 7fe48de00640 1 --2- 172.21.15.83:0/592974820 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555d4a7b0a30 0x555d4a7b8a10 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7fe480011520 tx=0x7fe48001a690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96ff8d3deb27f45a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:54.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.183+0000 7fe473e00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:01:54.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.183+0000 7fe473e00640 1 -- 172.21.15.83:0/592974820 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe480028060 con 0x555d4a7b0a30 2024-04-24T09:01:54.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.183+0000 7fe473e00640 1 -- 172.21.15.83:0/592974820 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7fe48001acd0 con 0x555d4a7b0a30 2024-04-24T09:01:54.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.183+0000 7fe497da8740 1 -- 172.21.15.83:0/592974820 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe48001a910 con 0x555d4a7b0a30 2024-04-24T09:01:54.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.183+0000 7fe473e00640 1 -- 172.21.15.83:0/592974820 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fe48002f050 con 0x555d4a7b0a30 2024-04-24T09:01:54.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.183+0000 7fe497da8740 10 client.19350 fetch_fsmap learned FSMap version 40 2024-04-24T09:01:54.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.183+0000 7fe497da8740 1 -- 172.21.15.83:0/592974820 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x555d4a4f27d0 con 0x555d4a7b0a30 2024-04-24T09:01:54.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.184+0000 7fe473e00640 1 -- 172.21.15.83:0/592974820 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe48001a910 con 0x555d4a7b0a30 2024-04-24T09:01:54.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.184+0000 7fe473e00640 1 -- 172.21.15.83:0/592974820 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 40) ==== 8382+0+0 (secure 0 0 0) 0x7fe4800049f0 con 0x555d4a7b0a30 2024-04-24T09:01:54.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.184+0000 7fe497da8740 10 client.19350 fetch_fsmap finished waiting for FSMap version 40 2024-04-24T09:01:54.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.184+0000 7fe497da8740 10 client.19350 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:01:54.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.184+0000 7fe497da8740 20 client.19350 populate_metadata read hostname 'smithi083' 2024-04-24T09:01:54.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.184+0000 7fe497da8740 1 --2- 172.21.15.83:0/592974820 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x555d4a7cbd70 0x555d4a7ce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:54.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.184+0000 7fe497da8740 4 client.19350 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:01:54.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.184+0000 7fe497da8740 1 -- 172.21.15.83:0/592974820 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555d4a723c10 con 0x555d4a7cbd70 2024-04-24T09:01:54.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.185+0000 7fe487e00640 1 --2- 172.21.15.83:0/592974820 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x555d4a7cbd70 0x555d4a7ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:54.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.186+0000 7fe487e00640 1 --2- 172.21.15.83:0/592974820 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x555d4a7cbd70 0x555d4a7ce160 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d68cdd0e99d9be41 server_cookie=723bb223916b662e in_seq=0 out_seq=0 2024-04-24T09:01:54.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.186+0000 7fe473e00640 10 client.19350 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:01:54.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.188+0000 7fe473e00640 1 -- 172.21.15.83:0/592974820 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555d4a723c10 con 0x555d4a7cbd70 2024-04-24T09:01:54.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.188+0000 7fe473e00640 10 client.19350 handle_command_reply: tid=0 2024-04-24T09:01:54.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.189+0000 7fe497da8740 1 client.19350 shutdown 2024-04-24T09:01:54.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.189+0000 7fe497da8740 1 -- 172.21.15.83:0/592974820 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555d4a7b0a30 msgr2=0x555d4a7b8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:54.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.189+0000 7fe497da8740 1 --2- 172.21.15.83:0/592974820 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555d4a7b0a30 0x555d4a7b8a10 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7fe480011520 tx=0x7fe48001a690 comp rx=0 tx=0).stop 2024-04-24T09:01:54.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.189+0000 7fe497da8740 1 -- 172.21.15.83:0/592974820 shutdown_connections 2024-04-24T09:01:54.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.189+0000 7fe497da8740 1 --2- 172.21.15.83:0/592974820 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x555d4a7cbd70 0x555d4a7ce160 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:54.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.189+0000 7fe497da8740 1 --2- 172.21.15.83:0/592974820 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d4a7bba20 0x555d4a7b9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:54.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.189+0000 7fe497da8740 1 --2- 172.21.15.83:0/592974820 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555d4a7b1350 0x555d4a7b8f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:54.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.189+0000 7fe473e00640 0 client.19350 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:01:54.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.189+0000 7fe497da8740 1 --2- 172.21.15.83:0/592974820 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555d4a7b0a30 0x555d4a7b8a10 unknown :-1 s=CLOSED pgs=1730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:54.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.189+0000 7fe497da8740 1 -- 172.21.15.83:0/592974820 >> 172.21.15.83:0/592974820 conn(0x555d4a7acd50 msgr2=0x555d4a7abfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:54.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.189+0000 7fe497da8740 1 -- 172.21.15.83:0/592974820 shutdown_connections 2024-04-24T09:01:54.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.189+0000 7fe497da8740 1 -- 172.21.15.83:0/592974820 wait complete. 2024-04-24T09:01:54.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.189+0000 7fe497da8740 20 client.19350 trim_cache size 0 max 16384 2024-04-24T09:01:54.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.209+0000 7fe497da8740 1 Processor -- start 2024-04-24T09:01:54.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.210+0000 7fe497da8740 1 -- start start 2024-04-24T09:01:54.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.210+0000 7fe497da8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d4a7b0a30 0x555d4a7c7730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:54.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.210+0000 7fe497da8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555d4a7b1350 0x555d4a7abe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:54.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.210+0000 7fe497da8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555d4a7bba20 0x555d4a7d37c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:54.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.210+0000 7fe497da8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe4901b5a10 con 0x555d4a7b1350 2024-04-24T09:01:54.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.210+0000 7fe497da8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe4901b7640 con 0x555d4a7b0a30 2024-04-24T09:01:54.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.210+0000 7fe497da8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe4901b60e0 con 0x555d4a7bba20 2024-04-24T09:01:54.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.210+0000 7fe487e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555d4a7b1350 0x555d4a7abe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:54.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.210+0000 7fe487e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555d4a7b1350 0x555d4a7abe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45860/0 (socket says 172.21.15.83:45860) 2024-04-24T09:01:54.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.210+0000 7fe487e00640 1 -- 172.21.15.83:0/813264814 learned_addr learned my addr 172.21.15.83:0/813264814 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:01:54.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.211+0000 7fe48de00640 1 --2- 172.21.15.83:0/813264814 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d4a7b0a30 0x555d4a7c7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:54.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.211+0000 7fe48e800640 1 --2- 172.21.15.83:0/813264814 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555d4a7bba20 0x555d4a7d37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:54.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.211+0000 7fe487e00640 1 -- 172.21.15.83:0/813264814 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555d4a7bba20 msgr2=0x555d4a7d37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:54.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.211+0000 7fe487e00640 1 --2- 172.21.15.83:0/813264814 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555d4a7bba20 0x555d4a7d37c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:54.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.211+0000 7fe487e00640 1 -- 172.21.15.83:0/813264814 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d4a7b0a30 msgr2=0x555d4a7c7730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:54.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.211+0000 7fe487e00640 1 --2- 172.21.15.83:0/813264814 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d4a7b0a30 0x555d4a7c7730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:54.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.211+0000 7fe487e00640 1 -- 172.21.15.83:0/813264814 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d4a4283f0 con 0x555d4a7b1350 2024-04-24T09:01:54.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.211+0000 7fe48de00640 1 --2- 172.21.15.83:0/813264814 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d4a7b0a30 0x555d4a7c7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:01:54.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.211+0000 7fe48e800640 1 --2- 172.21.15.83:0/813264814 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555d4a7bba20 0x555d4a7d37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:01:54.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.211+0000 7fe487e00640 1 --2- 172.21.15.83:0/813264814 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555d4a7b1350 0x555d4a7abe50 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7fe4780988b0 tx=0x7fe47800b530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51be9fe7acef09dd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:54.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.211+0000 7fe47ea00640 1 -- 172.21.15.83:0/813264814 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe478020040 con 0x555d4a7b1350 2024-04-24T09:01:54.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.211+0000 7fe47ea00640 1 -- 172.21.15.83:0/813264814 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe4780a0800 con 0x555d4a7b1350 2024-04-24T09:01:54.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.211+0000 7fe47ea00640 1 -- 172.21.15.83:0/813264814 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe478098c50 con 0x555d4a7b1350 2024-04-24T09:01:54.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.212+0000 7fe497da8740 1 -- 172.21.15.83:0/813264814 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555d4a7b1350 msgr2=0x555d4a7abe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:54.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.212+0000 7fe497da8740 1 --2- 172.21.15.83:0/813264814 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555d4a7b1350 0x555d4a7abe50 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7fe4780988b0 tx=0x7fe47800b530 comp rx=0 tx=0).stop 2024-04-24T09:01:54.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.212+0000 7fe497da8740 1 -- 172.21.15.83:0/813264814 shutdown_connections 2024-04-24T09:01:54.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.212+0000 7fe497da8740 1 --2- 172.21.15.83:0/813264814 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555d4a7bba20 0x555d4a7d37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:54.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.212+0000 7fe497da8740 1 --2- 172.21.15.83:0/813264814 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555d4a7b1350 0x555d4a7abe50 unknown :-1 s=CLOSED pgs=1731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:54.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.212+0000 7fe497da8740 1 --2- 172.21.15.83:0/813264814 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d4a7b0a30 0x555d4a7c7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:54.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.212+0000 7fe497da8740 1 -- 172.21.15.83:0/813264814 >> 172.21.15.83:0/813264814 conn(0x555d4a7acd50 msgr2=0x555d4a7ab740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:54.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.212+0000 7fe497da8740 1 -- 172.21.15.83:0/813264814 shutdown_connections 2024-04-24T09:01:54.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.212+0000 7fe497da8740 1 -- 172.21.15.83:0/813264814 wait complete. 2024-04-24T09:01:54.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.213+0000 7fe497da8740 1 -- start start 2024-04-24T09:01:54.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.213+0000 7fe497da8740 1 Processor -- start 2024-04-24T09:01:54.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.213+0000 7fe497da8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555d4a7b0a30 0x555d4a7b58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:54.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.213+0000 7fe497da8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555d4a7b1350 0x555d4a7b5e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:54.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.213+0000 7fe497da8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d4a7bba20 0x555d4a7b6360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:54.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.213+0000 7fe497da8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe4901b6f10 con 0x555d4a7b1350 2024-04-24T09:01:54.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.213+0000 7fe497da8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe4901b67e0 con 0x555d4a7bba20 2024-04-24T09:01:54.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.213+0000 7fe497da8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe4901b3a10 con 0x555d4a7b0a30 2024-04-24T09:01:54.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.213+0000 7fe497da8740 10 client.0 _ll_register_callbacks cb 0x7fe48000bae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:01:54.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.214+0000 7fe487e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555d4a7b1350 0x555d4a7b5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:54.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.214+0000 7fe487e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555d4a7b1350 0x555d4a7b5e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45862/0 (socket says 172.21.15.83:45862) 2024-04-24T09:01:54.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.214+0000 7fe487e00640 1 -- 172.21.15.83:0/3222718466 learned_addr learned my addr 172.21.15.83:0/3222718466 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:01:54.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.214+0000 7fe48de00640 1 --2- 172.21.15.83:0/3222718466 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555d4a7b0a30 0x555d4a7b58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:54.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.214+0000 7fe48e800640 1 --2- 172.21.15.83:0/3222718466 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d4a7bba20 0x555d4a7b6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:54.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.214+0000 7fe487e00640 1 -- 172.21.15.83:0/3222718466 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555d4a7b0a30 msgr2=0x555d4a7b58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:54.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.214+0000 7fe487e00640 1 --2- 172.21.15.83:0/3222718466 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555d4a7b0a30 0x555d4a7b58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:54.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.214+0000 7fe487e00640 1 -- 172.21.15.83:0/3222718466 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d4a7bba20 msgr2=0x555d4a7b6360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:54.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.214+0000 7fe487e00640 1 --2- 172.21.15.83:0/3222718466 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d4a7bba20 0x555d4a7b6360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:54.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.214+0000 7fe487e00640 1 -- 172.21.15.83:0/3222718466 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d4a3e80f0 con 0x555d4a7b1350 2024-04-24T09:01:54.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.214+0000 7fe48e800640 1 --2- 172.21.15.83:0/3222718466 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d4a7bba20 0x555d4a7b6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:01:54.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.214+0000 7fe487e00640 1 --2- 172.21.15.83:0/3222718466 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555d4a7b1350 0x555d4a7b5e20 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7fe47809c150 tx=0x7fe4780d0c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5953b6e028c9209f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:54.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.214+0000 7fe473e00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:01:54.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.215+0000 7fe473e00640 1 -- 172.21.15.83:0/3222718466 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4780b5060 con 0x555d4a7b1350 2024-04-24T09:01:54.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.215+0000 7fe473e00640 1 -- 172.21.15.83:0/3222718466 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7fe4780d0450 con 0x555d4a7b1350 2024-04-24T09:01:54.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.215+0000 7fe497da8740 1 -- 172.21.15.83:0/3222718466 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4780a0800 con 0x555d4a7b1350 2024-04-24T09:01:54.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.215+0000 7fe473e00640 1 -- 172.21.15.83:0/3222718466 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fe478020050 con 0x555d4a7b1350 2024-04-24T09:01:54.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.215+0000 7fe497da8740 10 client.19356 fetch_fsmap learned FSMap version 40 2024-04-24T09:01:54.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.215+0000 7fe497da8740 1 -- 172.21.15.83:0/3222718466 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x555d4a6db810 con 0x555d4a7b1350 2024-04-24T09:01:54.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.215+0000 7fe473e00640 1 -- 172.21.15.83:0/3222718466 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe4780a0800 con 0x555d4a7b1350 2024-04-24T09:01:54.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.215+0000 7fe473e00640 1 -- 172.21.15.83:0/3222718466 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 40) ==== 8382+0+0 (secure 0 0 0) 0x7fe4780bb120 con 0x555d4a7b1350 2024-04-24T09:01:54.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.215+0000 7fe497da8740 10 client.19356 fetch_fsmap finished waiting for FSMap version 40 2024-04-24T09:01:54.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.215+0000 7fe497da8740 10 client.19356 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:01:54.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.215+0000 7fe497da8740 20 client.19356 populate_metadata read hostname 'smithi083' 2024-04-24T09:01:54.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.215+0000 7fe497da8740 1 --2- 172.21.15.83:0/3222718466 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x555d4a832860 0x555d4a834c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:54.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.215+0000 7fe497da8740 4 client.19356 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T09:01:54.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.216+0000 7fe497da8740 1 -- 172.21.15.83:0/3222718466 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555d4a71da80 con 0x555d4a832860 2024-04-24T09:01:54.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.216+0000 7fe48de00640 1 --2- 172.21.15.83:0/3222718466 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x555d4a832860 0x555d4a834c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:54.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.217+0000 7fe48de00640 1 --2- 172.21.15.83:0/3222718466 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x555d4a832860 0x555d4a834c80 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d7195b7c273e60c server_cookie=dfc63b669fb656cd in_seq=0 out_seq=0 2024-04-24T09:01:54.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.217+0000 7fe473e00640 10 client.19356 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:01:54.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.220+0000 7fe473e00640 1 -- 172.21.15.83:0/3222718466 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x555d4a71da80 con 0x555d4a832860 2024-04-24T09:01:54.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.220+0000 7fe473e00640 10 client.19356 handle_command_reply: tid=0 2024-04-24T09:01:54.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.220+0000 7fe497da8740 1 client.19356 shutdown 2024-04-24T09:01:54.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.220+0000 7fe497da8740 1 -- 172.21.15.83:0/3222718466 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555d4a7b1350 msgr2=0x555d4a7b5e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:54.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.220+0000 7fe497da8740 1 --2- 172.21.15.83:0/3222718466 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555d4a7b1350 0x555d4a7b5e20 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7fe47809c150 tx=0x7fe4780d0c30 comp rx=0 tx=0).stop 2024-04-24T09:01:54.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.220+0000 7fe497da8740 1 -- 172.21.15.83:0/3222718466 shutdown_connections 2024-04-24T09:01:54.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.220+0000 7fe497da8740 1 --2- 172.21.15.83:0/3222718466 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x555d4a832860 0x555d4a834c80 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:54.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.220+0000 7fe497da8740 1 --2- 172.21.15.83:0/3222718466 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d4a7bba20 0x555d4a7b6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:54.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.220+0000 7fe497da8740 1 --2- 172.21.15.83:0/3222718466 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555d4a7b1350 0x555d4a7b5e20 unknown :-1 s=CLOSED pgs=1732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:54.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.220+0000 7fe497da8740 1 --2- 172.21.15.83:0/3222718466 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555d4a7b0a30 0x555d4a7b58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:54.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.220+0000 7fe497da8740 1 -- 172.21.15.83:0/3222718466 >> 172.21.15.83:0/3222718466 conn(0x555d4a7acd50 msgr2=0x555d4a7baad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:54.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.220+0000 7fe473e00640 0 client.19356 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:01:54.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.220+0000 7fe497da8740 1 -- 172.21.15.83:0/3222718466 shutdown_connections 2024-04-24T09:01:54.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.220+0000 7fe497da8740 1 -- 172.21.15.83:0/3222718466 wait complete. 2024-04-24T09:01:54.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.220+0000 7fe497da8740 20 client.19356 trim_cache size 0 max 16384 2024-04-24T09:01:54.221 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T09:01:54.222 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T09:01:54.222 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-04-24T09:01:54.222 INFO:teuthology.orchestra.run.smithi083.stdout: "720d5389-8ab6-4d54-8750-f06c623b56c6": { 2024-04-24T09:01:54.222 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-04-24T09:01:54.222 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "720d5389-8ab6-4d54-8750-f06c623b56c6", 2024-04-24T09:01:54.222 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-04-24T09:01:54.222 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:01:54.222 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:01:54.222 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T09:01:54.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.221+0000 7fe48f200640 1 -- 172.21.15.83:0/1537601803 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe4740a3d20 msgr2=0x7fe4740a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:54.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.221+0000 7fe48f200640 1 --2- 172.21.15.83:0/1537601803 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe4740a3d20 0x7fe4740a61e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe488031040 tx=0x7fe488005ef0 comp rx=0 tx=0).stop 2024-04-24T09:01:54.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.221+0000 7fe48f200640 1 -- 172.21.15.83:0/1537601803 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe490108f40 msgr2=0x7fe4901a4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:54.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.221+0000 7fe48f200640 1 --2- 172.21.15.83:0/1537601803 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe490108f40 0x7fe4901a4dc0 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7fe47800d950 tx=0x7fe47800de10 comp rx=0 tx=0).stop 2024-04-24T09:01:54.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.221+0000 7fe48f200640 1 -- 172.21.15.83:0/1537601803 shutdown_connections 2024-04-24T09:01:54.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.221+0000 7fe48f200640 1 --2- 172.21.15.83:0/1537601803 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe4740a3d20 0x7fe4740a61e0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:54.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.221+0000 7fe48f200640 1 --2- 172.21.15.83:0/1537601803 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe49010d500 0x7fe49019ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:54.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.221+0000 7fe48f200640 1 --2- 172.21.15.83:0/1537601803 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe490108f40 0x7fe4901a4dc0 unknown :-1 s=CLOSED pgs=1728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:54.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.221+0000 7fe48f200640 1 --2- 172.21.15.83:0/1537601803 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe490108580 0x7fe4901a4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:54.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.221+0000 7fe48f200640 1 -- 172.21.15.83:0/1537601803 >> 172.21.15.83:0/1537601803 conn(0x7fe490077e20 msgr2=0x7fe4900783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:54.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.222+0000 7fe48f200640 1 -- 172.21.15.83:0/1537601803 shutdown_connections 2024-04-24T09:01:54.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:01:54.222+0000 7fe48f200640 1 -- 172.21.15.83:0/1537601803 wait complete. 2024-04-24T09:01:54.242 INFO:tasks.cephfs.filesystem:scrub status for tag:720d5389-8ab6-4d54-8750-f06c623b56c6 - {'path': '/', 'tag': '720d5389-8ab6-4d54-8750-f06c623b56c6', 'options': 'recursive,force'} 2024-04-24T09:01:54.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:01:54 smithi083 ceph-mon[27907]: pgmap v321: 129 pgs: 129 active+clean; 59 MiB data, 827 MiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 525 KiB/s wr, 65 op/s 2024-04-24T09:01:54.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:01:54 smithi138 ceph-mon[31441]: pgmap v321: 129 pgs: 129 active+clean; 59 MiB data, 827 MiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 525 KiB/s wr, 65 op/s 2024-04-24T09:01:54.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:01:54 smithi189 ceph-mon[31791]: pgmap v321: 129 pgs: 129 active+clean; 59 MiB data, 827 MiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 525 KiB/s wr, 65 op/s 2024-04-24T09:01:54.609 INFO:tasks.workunit.client.0.smithi083.stdout: 1 60111 0.63 MB/sec execute 388 sec latency 57.624 ms 2024-04-24T09:01:55.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:01:55 smithi138 ceph-mon[31441]: scrub queued for path: / 2024-04-24T09:01:55.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:01:55 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:01:55.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:01:55 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:01:55.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:01:55 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:01:55.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:01:55 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:01:55.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:01:55 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:01:55.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:01:55 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:01:55.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:01:55 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:01:55.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:01:55 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:01:55.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:01:55 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:01:55.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:01:55 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:01:55.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:01:55 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:01:55.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:01:55 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:01:55.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:01:55 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:01:55.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:01:55 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:01:55.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:01:55 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:01:55.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:01:55 smithi189 ceph-mon[31791]: scrub queued for path: / 2024-04-24T09:01:55.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:01:55 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:01:55.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:01:55 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:01:55.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:01:55 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:01:55.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:01:55 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:01:55.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:01:55 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:01:55.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:01:55 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:01:55.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:01:55 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:01:55.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:01:55 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:01:55.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:01:55 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:01:55.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:01:55 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:01:55.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:01:55 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:01:55.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:01:55 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:01:55.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:01:55 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:01:55.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:01:55 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:01:55.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:01:55 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:01:55.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:01:55 smithi083 ceph-mon[27907]: scrub queued for path: / 2024-04-24T09:01:55.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:01:55 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:01:55.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:01:55 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:01:55.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:01:55 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:01:55.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:01:55 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:01:55.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:01:55 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:01:55.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:01:55 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:01:55.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:01:55 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:01:55.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:01:55 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:01:55.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:01:55 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:01:55.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:01:55 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:01:55.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:01:55 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:01:55.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:01:55 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:01:55.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:01:55 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:01:55.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:01:55 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:01:55.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:01:55 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:01:55.609 INFO:tasks.workunit.client.0.smithi083.stdout: 1 60272 0.63 MB/sec execute 389 sec latency 54.325 ms 2024-04-24T09:01:56.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:01:56 smithi138 ceph-mon[31441]: pgmap v322: 129 pgs: 129 active+clean; 59 MiB data, 827 MiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 444 KiB/s wr, 44 op/s 2024-04-24T09:01:56.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:01:56 smithi189 ceph-mon[31791]: pgmap v322: 129 pgs: 129 active+clean; 59 MiB data, 827 MiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 444 KiB/s wr, 44 op/s 2024-04-24T09:01:56.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:01:56 smithi083 ceph-mon[27907]: pgmap v322: 129 pgs: 129 active+clean; 59 MiB data, 827 MiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 444 KiB/s wr, 44 op/s 2024-04-24T09:01:56.609 INFO:tasks.workunit.client.0.smithi083.stdout: 1 60386 0.63 MB/sec execute 390 sec latency 59.594 ms 2024-04-24T09:01:57.610 INFO:tasks.workunit.client.0.smithi083.stdout: 1 60395 0.63 MB/sec execute 391 sec latency 986.667 ms 2024-04-24T09:01:58.305 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.304+0000 7f2a23200640 1 -- 172.21.15.83:0/3245979467 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a24104c50 msgr2=0x7f2a24108c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:58.305 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.304+0000 7f2a23200640 1 --2- 172.21.15.83:0/3245979467 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a24104c50 0x7f2a24108c90 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c009900 tx=0x7f2a0c02a410 comp rx=0 tx=0).stop 2024-04-24T09:01:58.305 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.304+0000 7f2a23200640 1 -- 172.21.15.83:0/3245979467 shutdown_connections 2024-04-24T09:01:58.305 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.304+0000 7f2a23200640 1 --2- 172.21.15.83:0/3245979467 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a24109370 0x7f2a24110f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:58.305 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.304+0000 7f2a23200640 1 --2- 172.21.15.83:0/3245979467 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a24104c50 0x7f2a24108c90 unknown :-1 s=CLOSED pgs=1733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:58.305 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.304+0000 7f2a23200640 1 --2- 172.21.15.83:0/3245979467 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2a241042a0 0x7f2a24104680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:58.305 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.304+0000 7f2a23200640 1 -- 172.21.15.83:0/3245979467 >> 172.21.15.83:0/3245979467 conn(0x7f2a240ffae0 msgr2=0x7f2a24101f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:58.305 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.304+0000 7f2a23200640 1 -- 172.21.15.83:0/3245979467 shutdown_connections 2024-04-24T09:01:58.305 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.304+0000 7f2a23200640 1 -- 172.21.15.83:0/3245979467 wait complete. 2024-04-24T09:01:58.307 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.305+0000 7f2a23200640 1 Processor -- start 2024-04-24T09:01:58.307 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.305+0000 7f2a23200640 1 -- start start 2024-04-24T09:01:58.307 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.306+0000 7f2a23200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a241042a0 0x7f2a241a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:58.307 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.306+0000 7f2a23200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a24109370 0x7f2a241a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:58.307 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.306+0000 7f2a23200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2a2419a740 0x7f2a2419abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:58.308 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.306+0000 7f2a23200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2a241142e0 con 0x7f2a24109370 2024-04-24T09:01:58.308 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.306+0000 7f2a23200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2a24114160 con 0x7f2a241042a0 2024-04-24T09:01:58.308 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.306+0000 7f2a23200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2a24114460 con 0x7f2a2419a740 2024-04-24T09:01:58.308 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.306+0000 7f2a1be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a24109370 0x7f2a241a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:58.308 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.306+0000 7f2a1be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a24109370 0x7f2a241a0b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45882/0 (socket says 172.21.15.83:45882) 2024-04-24T09:01:58.308 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.306+0000 7f2a1be00640 1 -- 172.21.15.83:0/907360624 learned_addr learned my addr 172.21.15.83:0/907360624 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:01:58.308 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.306+0000 7f2a22800640 1 --2- 172.21.15.83:0/907360624 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2a2419a740 0x7f2a2419abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:58.308 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.306+0000 7f2a21e00640 1 --2- 172.21.15.83:0/907360624 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a241042a0 0x7f2a241a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:58.308 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.306+0000 7f2a1be00640 1 -- 172.21.15.83:0/907360624 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2a2419a740 msgr2=0x7f2a2419abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:58.308 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.307+0000 7f2a1be00640 1 --2- 172.21.15.83:0/907360624 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2a2419a740 0x7f2a2419abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:58.308 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.307+0000 7f2a1be00640 1 -- 172.21.15.83:0/907360624 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a241042a0 msgr2=0x7f2a241a0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:58.308 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.307+0000 7f2a1be00640 1 --2- 172.21.15.83:0/907360624 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a241042a0 0x7f2a241a0630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:58.308 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.307+0000 7f2a1be00640 1 -- 172.21.15.83:0/907360624 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a2419b340 con 0x7f2a24109370 2024-04-24T09:01:58.308 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.307+0000 7f2a22800640 1 --2- 172.21.15.83:0/907360624 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2a2419a740 0x7f2a2419abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:01:58.309 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.307+0000 7f2a1be00640 1 --2- 172.21.15.83:0/907360624 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a24109370 0x7f2a241a0b70 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c031440 tx=0x7f2a0c003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:58.309 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.307+0000 7f2a1aa00640 1 -- 172.21.15.83:0/907360624 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a0c03f070 con 0x7f2a24109370 2024-04-24T09:01:58.309 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.307+0000 7f2a23200640 1 -- 172.21.15.83:0/907360624 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a2419b5d0 con 0x7f2a24109370 2024-04-24T09:01:58.309 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.307+0000 7f2a1aa00640 1 -- 172.21.15.83:0/907360624 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2a0c031d60 con 0x7f2a24109370 2024-04-24T09:01:58.309 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.307+0000 7f2a1aa00640 1 -- 172.21.15.83:0/907360624 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a0c035030 con 0x7f2a24109370 2024-04-24T09:01:58.309 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.307+0000 7f2a23200640 1 -- 172.21.15.83:0/907360624 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a241a76c0 con 0x7f2a24109370 2024-04-24T09:01:58.311 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.312+0000 7f2a2b76c740 1 Processor -- start 2024-04-24T09:01:58.312 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.312+0000 7f2a2b76c740 1 -- start start 2024-04-24T09:01:58.312 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.312+0000 7f2a1aa00640 1 -- 172.21.15.83:0/907360624 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2a0c042070 con 0x7f2a24109370 2024-04-24T09:01:58.313 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.312+0000 7f2a2b76c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a8516b43c0 0x55a8516b47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:58.313 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.312+0000 7f2a2b76c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8516b4ce0 0x55a8516bf060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:58.313 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.312+0000 7f2a2b76c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a8516bf5a0 0x55a8516c1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:58.313 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.312+0000 7f2a2b76c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a85161a900 con 0x55a8516b4ce0 2024-04-24T09:01:58.313 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.312+0000 7f2a2b76c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a8515c0f20 con 0x55a8516bf5a0 2024-04-24T09:01:58.313 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.312+0000 7f2a2b76c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a8516168c0 con 0x55a8516b43c0 2024-04-24T09:01:58.313 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.313+0000 7f2a21e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8516b4ce0 0x55a8516bf060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:58.313 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.313+0000 7f2a21e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8516b4ce0 0x55a8516bf060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45898/0 (socket says 172.21.15.83:45898) 2024-04-24T09:01:58.313 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.313+0000 7f2a21e00640 1 -- 172.21.15.83:0/2808564620 learned_addr learned my addr 172.21.15.83:0/2808564620 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:01:58.313 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.313+0000 7f2a1be00640 1 --2- 172.21.15.83:0/2808564620 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a8516bf5a0 0x55a8516c1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:58.313 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.313+0000 7f2a22800640 1 --2- 172.21.15.83:0/2808564620 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a8516b43c0 0x55a8516b47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:58.315 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.313+0000 7f2a21e00640 1 -- 172.21.15.83:0/2808564620 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a8516b43c0 msgr2=0x55a8516b47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:58.316 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.313+0000 7f2a21e00640 1 --2- 172.21.15.83:0/2808564620 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a8516b43c0 0x55a8516b47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:58.316 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.313+0000 7f2a21e00640 1 -- 172.21.15.83:0/2808564620 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a8516bf5a0 msgr2=0x55a8516c1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:58.316 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.313+0000 7f2a21e00640 1 --2- 172.21.15.83:0/2808564620 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a8516bf5a0 0x55a8516c1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:58.316 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.313+0000 7f2a21e00640 1 -- 172.21.15.83:0/2808564620 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8515e5e10 con 0x55a8516b4ce0 2024-04-24T09:01:58.316 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.314+0000 7f2a21e00640 1 --2- 172.21.15.83:0/2808564620 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8516b4ce0 0x55a8516bf060 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f2a1400baa0 tx=0x7f2a1400bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcf0bb89cd90a2e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:58.316 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.314+0000 7f2a13000640 1 -- 172.21.15.83:0/2808564620 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a14017420 con 0x55a8516b4ce0 2024-04-24T09:01:58.316 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.314+0000 7f2a1aa00640 1 --2- 172.21.15.83:0/907360624 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f2a040a3d10 0x7f2a040a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:58.316 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.314+0000 7f2a13000640 1 -- 172.21.15.83:0/2808564620 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2a14017a70 con 0x55a8516b4ce0 2024-04-24T09:01:58.316 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.314+0000 7f2a13000640 1 -- 172.21.15.83:0/2808564620 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a1400fac0 con 0x55a8516b4ce0 2024-04-24T09:01:58.316 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.314+0000 7f2a1aa00640 1 -- 172.21.15.83:0/907360624 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7f2a0c0f1500 con 0x7f2a24109370 2024-04-24T09:01:58.316 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.314+0000 7f2a22800640 1 --2- 172.21.15.83:0/907360624 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f2a040a3d10 0x7f2a040a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:58.316 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.315+0000 7f2a2b76c740 1 -- 172.21.15.83:0/2808564620 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8516b4ce0 msgr2=0x55a8516bf060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:58.316 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.315+0000 7f2a2b76c740 1 --2- 172.21.15.83:0/2808564620 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8516b4ce0 0x55a8516bf060 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f2a1400baa0 tx=0x7f2a1400bf60 comp rx=0 tx=0).stop 2024-04-24T09:01:58.316 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.315+0000 7f2a2b76c740 1 -- 172.21.15.83:0/2808564620 shutdown_connections 2024-04-24T09:01:58.317 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.315+0000 7f2a2b76c740 1 --2- 172.21.15.83:0/2808564620 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a8516bf5a0 0x55a8516c1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:58.317 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.315+0000 7f2a2b76c740 1 --2- 172.21.15.83:0/2808564620 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8516b4ce0 0x55a8516bf060 unknown :-1 s=CLOSED pgs=1735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:58.317 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.315+0000 7f2a2b76c740 1 --2- 172.21.15.83:0/2808564620 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a8516b43c0 0x55a8516b47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:58.317 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.315+0000 7f2a2b76c740 1 -- 172.21.15.83:0/2808564620 >> 172.21.15.83:0/2808564620 conn(0x55a8516b08d0 msgr2=0x55a8516b0cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:58.317 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.315+0000 7f2a22800640 1 --2- 172.21.15.83:0/907360624 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f2a040a3d10 0x7f2a040a61d0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c005fd0 tx=0x7f2a1c008370 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:58.317 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.315+0000 7f2a2b76c740 1 -- 172.21.15.83:0/2808564620 shutdown_connections 2024-04-24T09:01:58.317 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.315+0000 7f2a2b76c740 1 -- 172.21.15.83:0/2808564620 wait complete. 2024-04-24T09:01:58.317 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.315+0000 7f2a2b76c740 1 -- start start 2024-04-24T09:01:58.317 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.316+0000 7f2a2b76c740 1 Processor -- start 2024-04-24T09:01:58.320 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.316+0000 7f2a2b76c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8516b43c0 0x55a8516bc500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:58.320 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.316+0000 7f2a2b76c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a8516b4ce0 0x55a8516bca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:58.320 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.316+0000 7f2a2b76c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a8516bf5a0 0x55a8516bcf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:58.320 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.316+0000 7f2a2b76c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a8516c21d0 con 0x55a8516b43c0 2024-04-24T09:01:58.320 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.316+0000 7f2a2b76c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a8515d42c0 con 0x55a8516bf5a0 2024-04-24T09:01:58.320 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.316+0000 7f2a2b76c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a8516c2350 con 0x55a8516b4ce0 2024-04-24T09:01:58.320 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.317+0000 7f2a2b76c740 10 client.0 _ll_register_callbacks cb 0x55a85132ca00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:01:58.320 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.317+0000 7f2a21e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8516b43c0 0x55a8516bc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:58.320 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.317+0000 7f2a21e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8516b43c0 0x55a8516bc500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45902/0 (socket says 172.21.15.83:45902) 2024-04-24T09:01:58.320 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.317+0000 7f2a21e00640 1 -- 172.21.15.83:0/47666743 learned_addr learned my addr 172.21.15.83:0/47666743 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:01:58.321 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.317+0000 7f2a22800640 1 --2- 172.21.15.83:0/47666743 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a8516bf5a0 0x55a8516bcf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:58.321 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.317+0000 7f2a1be00640 1 --2- 172.21.15.83:0/47666743 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a8516b4ce0 0x55a8516bca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:58.321 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.317+0000 7f2a21e00640 1 -- 172.21.15.83:0/47666743 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a8516b4ce0 msgr2=0x55a8516bca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:58.321 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.317+0000 7f2a21e00640 1 --2- 172.21.15.83:0/47666743 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a8516b4ce0 0x55a8516bca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:58.321 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.317+0000 7f2a21e00640 1 -- 172.21.15.83:0/47666743 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a8516bf5a0 msgr2=0x55a8516bcf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:58.321 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.317+0000 7f2a21e00640 1 --2- 172.21.15.83:0/47666743 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a8516bf5a0 0x55a8516bcf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:58.321 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.317+0000 7f2a21e00640 1 -- 172.21.15.83:0/47666743 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8515e4fd0 con 0x55a8516b43c0 2024-04-24T09:01:58.321 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.317+0000 7f2a22800640 1 --2- 172.21.15.83:0/47666743 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a8516bf5a0 0x55a8516bcf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:01:58.321 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.318+0000 7f2a21e00640 1 --2- 172.21.15.83:0/47666743 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8516b43c0 0x55a8516bc500 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f2a1400e010 tx=0x7f2a14005c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=327c3a3029b018f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:58.321 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.318+0000 7f2a0b400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:01:58.321 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.318+0000 7f2a0b400640 1 -- 172.21.15.83:0/47666743 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a14027020 con 0x55a8516b43c0 2024-04-24T09:01:58.321 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.318+0000 7f2a0b400640 1 -- 172.21.15.83:0/47666743 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7f2a14017620 con 0x55a8516b43c0 2024-04-24T09:01:58.321 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.318+0000 7f2a2b76c740 1 -- 172.21.15.83:0/47666743 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a14017a70 con 0x55a8516b43c0 2024-04-24T09:01:58.321 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.318+0000 7f2a0b400640 1 -- 172.21.15.83:0/47666743 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f2a14032050 con 0x55a8516b43c0 2024-04-24T09:01:58.321 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.318+0000 7f2a2b76c740 10 client.19368 fetch_fsmap learned FSMap version 40 2024-04-24T09:01:58.322 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.318+0000 7f2a2b76c740 1 -- 172.21.15.83:0/47666743 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55a85132d210 con 0x55a8516b43c0 2024-04-24T09:01:58.322 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.319+0000 7f2a0b400640 1 -- 172.21.15.83:0/47666743 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2a14017a70 con 0x55a8516b43c0 2024-04-24T09:01:58.322 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.319+0000 7f2a0b400640 1 -- 172.21.15.83:0/47666743 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 40) ==== 8382+0+0 (secure 0 0 0) 0x7f2a140236a0 con 0x55a8516b43c0 2024-04-24T09:01:58.322 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.319+0000 7f2a2b76c740 10 client.19368 fetch_fsmap finished waiting for FSMap version 40 2024-04-24T09:01:58.322 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.319+0000 7f2a2b76c740 10 client.19368 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:01:58.322 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.319+0000 7f2a2b76c740 20 client.19368 populate_metadata read hostname 'smithi083' 2024-04-24T09:01:58.322 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.319+0000 7f2a2b76c740 1 --2- 172.21.15.83:0/47666743 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55a8516cf680 0x55a8516d1a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:58.322 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.319+0000 7f2a2b76c740 4 client.19368 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:01:58.322 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.319+0000 7f2a2b76c740 1 -- 172.21.15.83:0/47666743 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a851628320 con 0x55a8516cf680 2024-04-24T09:01:58.322 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.320+0000 7f2a1be00640 1 --2- 172.21.15.83:0/47666743 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55a8516cf680 0x55a8516d1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:58.323 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.321+0000 7f2a1be00640 1 --2- 172.21.15.83:0/47666743 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55a8516cf680 0x55a8516d1a70 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ae594ddc18d0c9a server_cookie=823eb6b0bd9f71a9 in_seq=0 out_seq=0 2024-04-24T09:01:58.323 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.321+0000 7f2a0b400640 10 client.19368 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:01:58.325 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.324+0000 7f2a0b400640 1 -- 172.21.15.83:0/47666743 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a851628320 con 0x55a8516cf680 2024-04-24T09:01:58.325 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.324+0000 7f2a0b400640 10 client.19368 handle_command_reply: tid=0 2024-04-24T09:01:58.325 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.324+0000 7f2a2b76c740 1 client.19368 shutdown 2024-04-24T09:01:58.325 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.325+0000 7f2a2b76c740 1 -- 172.21.15.83:0/47666743 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8516b43c0 msgr2=0x55a8516bc500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:58.325 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.325+0000 7f2a2b76c740 1 --2- 172.21.15.83:0/47666743 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8516b43c0 0x55a8516bc500 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f2a1400e010 tx=0x7f2a14005c10 comp rx=0 tx=0).stop 2024-04-24T09:01:58.325 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.325+0000 7f2a2b76c740 1 -- 172.21.15.83:0/47666743 shutdown_connections 2024-04-24T09:01:58.325 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.325+0000 7f2a2b76c740 1 --2- 172.21.15.83:0/47666743 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55a8516cf680 0x55a8516d1a70 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:58.325 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.325+0000 7f2a2b76c740 1 --2- 172.21.15.83:0/47666743 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a8516bf5a0 0x55a8516bcf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:58.325 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.325+0000 7f2a2b76c740 1 --2- 172.21.15.83:0/47666743 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a8516b4ce0 0x55a8516bca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:58.325 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.325+0000 7f2a0b400640 0 client.19368 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:01:58.325 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.325+0000 7f2a2b76c740 1 --2- 172.21.15.83:0/47666743 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8516b43c0 0x55a8516bc500 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:58.326 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.325+0000 7f2a2b76c740 1 -- 172.21.15.83:0/47666743 >> 172.21.15.83:0/47666743 conn(0x55a8516b08d0 msgr2=0x55a8516afa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:58.326 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.325+0000 7f2a2b76c740 1 -- 172.21.15.83:0/47666743 shutdown_connections 2024-04-24T09:01:58.326 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.325+0000 7f2a2b76c740 1 -- 172.21.15.83:0/47666743 wait complete. 2024-04-24T09:01:58.326 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.325+0000 7f2a2b76c740 20 client.19368 trim_cache size 0 max 16384 2024-04-24T09:01:58.344 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.344+0000 7f2a2b76c740 1 Processor -- start 2024-04-24T09:01:58.344 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.344+0000 7f2a2b76c740 1 -- start start 2024-04-24T09:01:58.344 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.344+0000 7f2a2b76c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a8516b43c0 0x55a8516cb090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:58.344 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.344+0000 7f2a2b76c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a8516b4ce0 0x55a8516d70a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:58.344 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.344+0000 7f2a2b76c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8516bf5a0 0x55a8516d75e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:58.344 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.344+0000 7f2a2b76c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2a241b16b0 con 0x55a8516bf5a0 2024-04-24T09:01:58.345 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.344+0000 7f2a2b76c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2a241b32e0 con 0x55a8516b43c0 2024-04-24T09:01:58.345 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.344+0000 7f2a2b76c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2a241b1d80 con 0x55a8516b4ce0 2024-04-24T09:01:58.345 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.345+0000 7f2a21e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a8516b43c0 0x55a8516cb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:58.345 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.345+0000 7f2a1be00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a8516b4ce0 0x55a8516d70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:58.345 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.345+0000 7f2a22800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8516bf5a0 0x55a8516d75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:58.345 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.345+0000 7f2a21e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a8516b43c0 0x55a8516cb090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:39532/0 (socket says 172.21.15.83:39532) 2024-04-24T09:01:58.345 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.345+0000 7f2a21e00640 1 -- 172.21.15.83:0/125781922 learned_addr learned my addr 172.21.15.83:0/125781922 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:01:58.346 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.346+0000 7f2a22800640 1 -- 172.21.15.83:0/125781922 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a8516b4ce0 msgr2=0x55a8516d70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:58.346 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.346+0000 7f2a22800640 1 --2- 172.21.15.83:0/125781922 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a8516b4ce0 0x55a8516d70a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:58.346 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.346+0000 7f2a22800640 1 -- 172.21.15.83:0/125781922 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a8516b43c0 msgr2=0x55a8516cb090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:58.346 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.346+0000 7f2a22800640 1 --2- 172.21.15.83:0/125781922 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a8516b43c0 0x55a8516cb090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:58.346 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.346+0000 7f2a22800640 1 -- 172.21.15.83:0/125781922 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8513f9620 con 0x55a8516bf5a0 2024-04-24T09:01:58.346 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.346+0000 7f2a22800640 1 --2- 172.21.15.83:0/125781922 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8516bf5a0 0x55a8516d75e0 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c00fc20 tx=0x7f2a1c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a582d462ca8282ef server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:58.347 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.347+0000 7f2a12600640 1 -- 172.21.15.83:0/125781922 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a1c01d940 con 0x55a8516bf5a0 2024-04-24T09:01:58.347 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.347+0000 7f2a12600640 1 -- 172.21.15.83:0/125781922 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2a1c01e430 con 0x55a8516bf5a0 2024-04-24T09:01:58.347 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.347+0000 7f2a12600640 1 -- 172.21.15.83:0/125781922 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a1c026630 con 0x55a8516bf5a0 2024-04-24T09:01:58.348 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.348+0000 7f2a2b76c740 1 -- 172.21.15.83:0/125781922 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8516bf5a0 msgr2=0x55a8516d75e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:58.348 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.348+0000 7f2a2b76c740 1 --2- 172.21.15.83:0/125781922 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8516bf5a0 0x55a8516d75e0 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c00fc20 tx=0x7f2a1c012000 comp rx=0 tx=0).stop 2024-04-24T09:01:58.348 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.348+0000 7f2a2b76c740 1 -- 172.21.15.83:0/125781922 shutdown_connections 2024-04-24T09:01:58.348 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.348+0000 7f2a2b76c740 1 --2- 172.21.15.83:0/125781922 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8516bf5a0 0x55a8516d75e0 unknown :-1 s=CLOSED pgs=1737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:58.348 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.348+0000 7f2a2b76c740 1 --2- 172.21.15.83:0/125781922 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a8516b4ce0 0x55a8516d70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:58.348 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.348+0000 7f2a2b76c740 1 --2- 172.21.15.83:0/125781922 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a8516b43c0 0x55a8516cb090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:58.348 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.348+0000 7f2a2b76c740 1 -- 172.21.15.83:0/125781922 >> 172.21.15.83:0/125781922 conn(0x55a8516b08d0 msgr2=0x55a8516badd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:58.348 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.348+0000 7f2a2b76c740 1 -- 172.21.15.83:0/125781922 shutdown_connections 2024-04-24T09:01:58.348 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.348+0000 7f2a2b76c740 1 -- 172.21.15.83:0/125781922 wait complete. 2024-04-24T09:01:58.349 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.348+0000 7f2a2b76c740 1 -- start start 2024-04-24T09:01:58.351 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.348+0000 7f2a2b76c740 1 Processor -- start 2024-04-24T09:01:58.351 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.349+0000 7f2a2b76c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8516b43c0 0x55a8516c7830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:58.351 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.349+0000 7f2a2b76c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a8516b4ce0 0x55a8516c2c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:58.351 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.349+0000 7f2a2b76c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a8516bf5a0 0x55a8516c3180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:58.351 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.349+0000 7f2a2b76c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2a241b2bb0 con 0x55a8516b43c0 2024-04-24T09:01:58.351 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.349+0000 7f2a2b76c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2a241b2480 con 0x55a8516bf5a0 2024-04-24T09:01:58.351 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.349+0000 7f2a2b76c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2a241af6b0 con 0x55a8516b4ce0 2024-04-24T09:01:58.351 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.349+0000 7f2a2b76c740 10 client.0 _ll_register_callbacks cb 0x7f2a1400b8f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:01:58.352 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.349+0000 7f2a21e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8516b43c0 0x55a8516c7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:58.352 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.349+0000 7f2a21e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8516b43c0 0x55a8516c7830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45922/0 (socket says 172.21.15.83:45922) 2024-04-24T09:01:58.352 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.349+0000 7f2a21e00640 1 -- 172.21.15.83:0/4088228037 learned_addr learned my addr 172.21.15.83:0/4088228037 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:01:58.352 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.349+0000 7f2a1be00640 1 --2- 172.21.15.83:0/4088228037 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a8516b4ce0 0x55a8516c2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:58.352 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.350+0000 7f2a22800640 1 --2- 172.21.15.83:0/4088228037 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a8516bf5a0 0x55a8516c3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:58.352 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.350+0000 7f2a21e00640 1 -- 172.21.15.83:0/4088228037 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a8516b4ce0 msgr2=0x55a8516c2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:58.352 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.350+0000 7f2a21e00640 1 --2- 172.21.15.83:0/4088228037 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a8516b4ce0 0x55a8516c2c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:58.352 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.350+0000 7f2a21e00640 1 -- 172.21.15.83:0/4088228037 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a8516bf5a0 msgr2=0x55a8516c3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:01:58.352 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.350+0000 7f2a21e00640 1 --2- 172.21.15.83:0/4088228037 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a8516bf5a0 0x55a8516c3180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:58.352 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.350+0000 7f2a21e00640 1 -- 172.21.15.83:0/4088228037 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8516c3870 con 0x55a8516b43c0 2024-04-24T09:01:58.352 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.350+0000 7f2a22800640 1 --2- 172.21.15.83:0/4088228037 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a8516bf5a0 0x55a8516c3180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:01:58.352 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.350+0000 7f2a21e00640 1 --2- 172.21.15.83:0/4088228037 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8516b43c0 0x55a8516c7830 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f2a14023b40 tx=0x7f2a1400b310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a009b61d3b279d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:01:58.352 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.350+0000 7f2a0b400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:01:58.352 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.350+0000 7f2a0b400640 1 -- 172.21.15.83:0/4088228037 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a14027040 con 0x55a8516b43c0 2024-04-24T09:01:58.352 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.350+0000 7f2a0b400640 1 -- 172.21.15.83:0/4088228037 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9583+0+0 (secure 0 0 0) 0x7f2a14005040 con 0x55a8516b43c0 2024-04-24T09:01:58.353 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.350+0000 7f2a2b76c740 1 -- 172.21.15.83:0/4088228037 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a1c01e430 con 0x55a8516b43c0 2024-04-24T09:01:58.353 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.351+0000 7f2a0b400640 1 -- 172.21.15.83:0/4088228037 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f2a14044050 con 0x55a8516b43c0 2024-04-24T09:01:58.353 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.351+0000 7f2a2b76c740 10 client.19374 fetch_fsmap learned FSMap version 40 2024-04-24T09:01:58.353 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.351+0000 7f2a2b76c740 1 -- 172.21.15.83:0/4088228037 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55a8517357e0 con 0x55a8516b43c0 2024-04-24T09:01:58.353 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.351+0000 7f2a0b400640 1 -- 172.21.15.83:0/4088228037 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2a1c01e430 con 0x55a8516b43c0 2024-04-24T09:01:58.353 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.351+0000 7f2a0b400640 1 -- 172.21.15.83:0/4088228037 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 40) ==== 8382+0+0 (secure 0 0 0) 0x7f2a14017420 con 0x55a8516b43c0 2024-04-24T09:01:58.353 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.351+0000 7f2a2b76c740 10 client.19374 fetch_fsmap finished waiting for FSMap version 40 2024-04-24T09:01:58.353 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.351+0000 7f2a2b76c740 10 client.19374 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:01:58.353 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.351+0000 7f2a2b76c740 20 client.19374 populate_metadata read hostname 'smithi083' 2024-04-24T09:01:58.353 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.351+0000 7f2a2b76c740 1 --2- 172.21.15.83:0/4088228037 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55a851735f50 0x55a851738370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:01:58.353 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.351+0000 7f2a2b76c740 4 client.19374 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T09:01:58.353 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.351+0000 7f2a2b76c740 1 -- 172.21.15.83:0/4088228037 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a851622190 con 0x55a851735f50 2024-04-24T09:01:58.354 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.352+0000 7f2a1be00640 1 --2- 172.21.15.83:0/4088228037 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55a851735f50 0x55a851738370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:01:58.354 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.353+0000 7f2a1be00640 1 --2- 172.21.15.83:0/4088228037 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55a851735f50 0x55a851738370 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19e4901871ce8ccb server_cookie=86ffbd21e3b72633 in_seq=0 out_seq=0 2024-04-24T09:01:58.354 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.353+0000 7f2a0b400640 10 client.19374 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:01:58.412 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.411+0000 7f2a0b400640 1 -- 172.21.15.83:0/4088228037 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a851622190 con 0x55a851735f50 2024-04-24T09:01:58.412 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.412+0000 7f2a0b400640 10 client.19374 handle_command_reply: tid=0 2024-04-24T09:01:58.412 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.412+0000 7f2a2b76c740 1 client.19374 shutdown 2024-04-24T09:01:58.412 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.412+0000 7f2a2b76c740 1 -- 172.21.15.83:0/4088228037 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8516b43c0 msgr2=0x55a8516c7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:58.412 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.412+0000 7f2a2b76c740 1 --2- 172.21.15.83:0/4088228037 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8516b43c0 0x55a8516c7830 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f2a14023b40 tx=0x7f2a1400b310 comp rx=0 tx=0).stop 2024-04-24T09:01:58.413 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.412+0000 7f2a2b76c740 1 -- 172.21.15.83:0/4088228037 shutdown_connections 2024-04-24T09:01:58.413 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.412+0000 7f2a2b76c740 1 --2- 172.21.15.83:0/4088228037 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55a851735f50 0x55a851738370 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:58.413 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.412+0000 7f2a2b76c740 1 --2- 172.21.15.83:0/4088228037 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a8516bf5a0 0x55a8516c3180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:58.413 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.412+0000 7f2a2b76c740 1 --2- 172.21.15.83:0/4088228037 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a8516b4ce0 0x55a8516c2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:58.413 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.412+0000 7f2a2b76c740 1 --2- 172.21.15.83:0/4088228037 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8516b43c0 0x55a8516c7830 unknown :-1 s=CLOSED pgs=1738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:58.414 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.412+0000 7f2a0b400640 0 client.19374 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:01:58.414 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.412+0000 7f2a2b76c740 1 -- 172.21.15.83:0/4088228037 >> 172.21.15.83:0/4088228037 conn(0x55a8516b08d0 msgr2=0x55a8516af200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:58.414 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.412+0000 7f2a2b76c740 1 -- 172.21.15.83:0/4088228037 shutdown_connections 2024-04-24T09:01:58.414 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.413+0000 7f2a2b76c740 1 -- 172.21.15.83:0/4088228037 wait complete. 2024-04-24T09:01:58.414 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.413+0000 7f2a2b76c740 20 client.19374 trim_cache size 0 max 16384 2024-04-24T09:01:58.414 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T09:01:58.414 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T09:01:58.414 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T09:01:58.415 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T09:01:58.415 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.414+0000 7f2a23200640 1 -- 172.21.15.83:0/907360624 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f2a040a3d10 msgr2=0x7f2a040a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:58.416 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.414+0000 7f2a23200640 1 --2- 172.21.15.83:0/907360624 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f2a040a3d10 0x7f2a040a61d0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c005fd0 tx=0x7f2a1c008370 comp rx=0 tx=0).stop 2024-04-24T09:01:58.416 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.414+0000 7f2a23200640 1 -- 172.21.15.83:0/907360624 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a24109370 msgr2=0x7f2a241a0b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:01:58.416 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.414+0000 7f2a23200640 1 --2- 172.21.15.83:0/907360624 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a24109370 0x7f2a241a0b70 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c031440 tx=0x7f2a0c003d50 comp rx=0 tx=0).stop 2024-04-24T09:01:58.416 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.414+0000 7f2a23200640 1 -- 172.21.15.83:0/907360624 shutdown_connections 2024-04-24T09:01:58.416 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.414+0000 7f2a23200640 1 --2- 172.21.15.83:0/907360624 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f2a040a3d10 0x7f2a040a61d0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:58.416 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.414+0000 7f2a23200640 1 --2- 172.21.15.83:0/907360624 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2a2419a740 0x7f2a2419abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:58.416 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.414+0000 7f2a23200640 1 --2- 172.21.15.83:0/907360624 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a24109370 0x7f2a241a0b70 unknown :-1 s=CLOSED pgs=1734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:58.416 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.414+0000 7f2a23200640 1 --2- 172.21.15.83:0/907360624 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a241042a0 0x7f2a241a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:01:58.416 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.414+0000 7f2a23200640 1 -- 172.21.15.83:0/907360624 >> 172.21.15.83:0/907360624 conn(0x7f2a240ffae0 msgr2=0x7f2a24100f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:01:58.416 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.414+0000 7f2a23200640 1 -- 172.21.15.83:0/907360624 shutdown_connections 2024-04-24T09:01:58.416 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:01:58.414+0000 7f2a23200640 1 -- 172.21.15.83:0/907360624 wait complete. 2024-04-24T09:01:58.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:01:58 smithi138 ceph-mon[31441]: pgmap v323: 129 pgs: 129 active+clean; 59 MiB data, 827 MiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 444 KiB/s wr, 44 op/s 2024-04-24T09:01:58.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:01:58 smithi189 ceph-mon[31791]: pgmap v323: 129 pgs: 129 active+clean; 59 MiB data, 827 MiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 444 KiB/s wr, 44 op/s 2024-04-24T09:01:58.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:01:58 smithi083 ceph-mon[27907]: pgmap v323: 129 pgs: 129 active+clean; 59 MiB data, 827 MiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 444 KiB/s wr, 44 op/s 2024-04-24T09:01:58.610 INFO:tasks.workunit.client.0.smithi083.stdout: 1 60439 0.63 MB/sec execute 392 sec latency 1479.707 ms 2024-04-24T09:01:59.610 INFO:tasks.workunit.client.0.smithi083.stdout: 1 60462 0.63 MB/sec execute 393 sec latency 674.287 ms 2024-04-24T09:02:00.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:02:00 smithi138 ceph-mon[31441]: pgmap v324: 129 pgs: 129 active+clean; 59 MiB data, 847 MiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 572 KiB/s wr, 67 op/s 2024-04-24T09:02:00.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:02:00 smithi189 ceph-mon[31791]: pgmap v324: 129 pgs: 129 active+clean; 59 MiB data, 847 MiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 572 KiB/s wr, 67 op/s 2024-04-24T09:02:00.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:02:00 smithi083 ceph-mon[27907]: pgmap v324: 129 pgs: 129 active+clean; 59 MiB data, 847 MiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 572 KiB/s wr, 67 op/s 2024-04-24T09:02:00.610 INFO:tasks.workunit.client.0.smithi083.stdout: 1 60462 0.63 MB/sec execute 394 sec latency 1674.422 ms 2024-04-24T09:02:01.610 INFO:tasks.workunit.client.0.smithi083.stdout: 1 60462 0.63 MB/sec execute 395 sec latency 2674.522 ms 2024-04-24T09:02:02.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:02:02 smithi083 ceph-mon[27907]: pgmap v325: 129 pgs: 129 active+clean; 59 MiB data, 847 MiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 423 KiB/s wr, 46 op/s 2024-04-24T09:02:02.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:02:02 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:02:02.610 INFO:tasks.workunit.client.0.smithi083.stdout: 1 60462 0.62 MB/sec execute 396 sec latency 3674.616 ms 2024-04-24T09:02:02.704 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:02:02 smithi138 ceph-mon[31441]: pgmap v325: 129 pgs: 129 active+clean; 59 MiB data, 847 MiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 423 KiB/s wr, 46 op/s 2024-04-24T09:02:02.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:02:02 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:02:02.706 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:02:02 smithi189 ceph-mon[31791]: pgmap v325: 129 pgs: 129 active+clean; 59 MiB data, 847 MiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 423 KiB/s wr, 46 op/s 2024-04-24T09:02:02.706 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:02:02 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:02:03.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:02:03 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:02:03.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:02:03 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:02:03.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:02:03 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:02:03.610 INFO:tasks.workunit.client.0.smithi083.stdout: 1 60525 0.63 MB/sec execute 397 sec latency 4172.599 ms 2024-04-24T09:02:03.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:02:03 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:02:03.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:02:03 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:02:03.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:02:03 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:02:03.706 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:02:03 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:02:03.706 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:02:03 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:02:03.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:02:03 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:02:04.552 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:02:04 smithi083 ceph-mon[27907]: pgmap v326: 129 pgs: 129 active+clean; 60 MiB data, 852 MiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 472 KiB/s wr, 64 op/s 2024-04-24T09:02:04.552 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:02:04 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:02:04.610 INFO:tasks.workunit.client.0.smithi083.stdout: 1 60604 0.62 MB/sec execute 398 sec latency 49.920 ms 2024-04-24T09:02:04.704 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:02:04 smithi138 ceph-mon[31441]: pgmap v326: 129 pgs: 129 active+clean; 60 MiB data, 852 MiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 472 KiB/s wr, 64 op/s 2024-04-24T09:02:04.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:02:04 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:02:04.706 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:02:04 smithi189 ceph-mon[31791]: pgmap v326: 129 pgs: 129 active+clean; 60 MiB data, 852 MiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 472 KiB/s wr, 64 op/s 2024-04-24T09:02:04.706 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:02:04 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:02:05.611 INFO:tasks.workunit.client.0.smithi083.stdout: 1 60705 0.62 MB/sec execute 399 sec latency 44.149 ms 2024-04-24T09:02:06.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:02:06 smithi083 ceph-mon[27907]: pgmap v327: 129 pgs: 129 active+clean; 60 MiB data, 852 MiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 177 KiB/s wr, 40 op/s 2024-04-24T09:02:06.611 INFO:tasks.workunit.client.0.smithi083.stdout: 1 60815 0.62 MB/sec execute 400 sec latency 30.793 ms 2024-04-24T09:02:06.704 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:02:06 smithi138 ceph-mon[31441]: pgmap v327: 129 pgs: 129 active+clean; 60 MiB data, 852 MiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 177 KiB/s wr, 40 op/s 2024-04-24T09:02:06.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:02:06 smithi189 ceph-mon[31791]: pgmap v327: 129 pgs: 129 active+clean; 60 MiB data, 852 MiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 177 KiB/s wr, 40 op/s 2024-04-24T09:02:07.611 INFO:tasks.workunit.client.0.smithi083.stdout: 1 60922 0.62 MB/sec execute 401 sec latency 51.802 ms 2024-04-24T09:02:07.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:02:07 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:02:07.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:02:07 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:02:07.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:02:07 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:02:07.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:02:07 smithi138 ceph-mon[31441]: pgmap v328: 129 pgs: 129 active+clean; 60 MiB data, 852 MiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 177 KiB/s wr, 40 op/s 2024-04-24T09:02:07.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:02:07 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:02:07.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:02:07 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:02:07.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:02:07 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:02:07.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:02:07 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:02:07.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:02:07 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:02:07.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:02:07 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:02:07.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:02:07 smithi189 ceph-mon[31791]: pgmap v328: 129 pgs: 129 active+clean; 60 MiB data, 852 MiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 177 KiB/s wr, 40 op/s 2024-04-24T09:02:07.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:02:07 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:02:07.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:02:07 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:02:07.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:02:07 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:02:08.052 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:02:07 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:02:08.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:02:07 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:02:08.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:02:07 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:02:08.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:02:07 smithi083 ceph-mon[27907]: pgmap v328: 129 pgs: 129 active+clean; 60 MiB data, 852 MiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 177 KiB/s wr, 40 op/s 2024-04-24T09:02:08.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:02:07 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:02:08.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:02:07 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:02:08.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:02:07 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:02:08.611 INFO:tasks.workunit.client.0.smithi083.stdout: 1 61051 0.62 MB/sec execute 402 sec latency 52.358 ms 2024-04-24T09:02:09.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:02:08 smithi083 ceph-mon[27907]: osdmap e105: 12 total, 12 up, 12 in 2024-04-24T09:02:09.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:02:08 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:02:09.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:02:08 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:02:09.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:02:08 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:02:09.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:02:08 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:02:09.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:02:08 smithi083 ceph-mon[27907]: Adjusting osd_memory_target on smithi138 to 176.4M 2024-04-24T09:02:09.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:02:08 smithi083 ceph-mon[27907]: Unable to set osd_memory_target on smithi138 to 185047654: error parsing value: Value '185047654' is below minimum 939524096 2024-04-24T09:02:09.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:02:08 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:02:09.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:02:08 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:02:09.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:02:08 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:02:09.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:02:08 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:02:09.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:02:08 smithi083 ceph-mon[27907]: Adjusting osd_memory_target on smithi189 to 176.4M 2024-04-24T09:02:09.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:02:08 smithi083 ceph-mon[27907]: Unable to set osd_memory_target on smithi189 to 185064857: error parsing value: Value '185064857' is below minimum 939524096 2024-04-24T09:02:09.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:02:08 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:02:09.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:02:08 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:02:09.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:02:08 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:02:09.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:02:08 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:02:09.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:02:08 smithi083 ceph-mon[27907]: Adjusting osd_memory_target on smithi083 to 176.4M 2024-04-24T09:02:09.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:02:08 smithi083 ceph-mon[27907]: Unable to set osd_memory_target on smithi083 to 185061990: error parsing value: Value '185061990' is below minimum 939524096 2024-04-24T09:02:09.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:02:08 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:02:09.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:02:08 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:02:09.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:02:08 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:02:09.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:02:08 smithi138 ceph-mon[31441]: osdmap e105: 12 total, 12 up, 12 in 2024-04-24T09:02:09.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:02:08 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:02:09.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:02:08 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:02:09.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:02:08 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:02:09.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:02:08 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:02:09.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:02:08 smithi138 ceph-mon[31441]: Adjusting osd_memory_target on smithi138 to 176.4M 2024-04-24T09:02:09.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:02:08 smithi138 ceph-mon[31441]: Unable to set osd_memory_target on smithi138 to 185047654: error parsing value: Value '185047654' is below minimum 939524096 2024-04-24T09:02:09.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:02:08 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:02:09.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:02:08 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:02:09.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:02:08 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:02:09.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:02:08 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:02:09.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:02:08 smithi138 ceph-mon[31441]: Adjusting osd_memory_target on smithi189 to 176.4M 2024-04-24T09:02:09.207 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:02:08 smithi138 ceph-mon[31441]: Unable to set osd_memory_target on smithi189 to 185064857: error parsing value: Value '185064857' is below minimum 939524096 2024-04-24T09:02:09.207 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:02:08 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:02:09.207 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:02:08 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:02:09.207 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:02:08 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:02:09.207 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:02:08 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:02:09.207 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:02:08 smithi138 ceph-mon[31441]: Adjusting osd_memory_target on smithi083 to 176.4M 2024-04-24T09:02:09.207 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:02:08 smithi138 ceph-mon[31441]: Unable to set osd_memory_target on smithi083 to 185061990: error parsing value: Value '185061990' is below minimum 939524096 2024-04-24T09:02:09.207 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:02:08 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:02:09.207 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:02:08 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:02:09.207 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:02:08 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:02:09.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:02:08 smithi189 ceph-mon[31791]: osdmap e105: 12 total, 12 up, 12 in 2024-04-24T09:02:09.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:02:08 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:02:09.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:02:08 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:02:09.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:02:08 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:02:09.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:02:08 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:02:09.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:02:08 smithi189 ceph-mon[31791]: Adjusting osd_memory_target on smithi138 to 176.4M 2024-04-24T09:02:09.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:02:08 smithi189 ceph-mon[31791]: Unable to set osd_memory_target on smithi138 to 185047654: error parsing value: Value '185047654' is below minimum 939524096 2024-04-24T09:02:09.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:02:08 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:02:09.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:02:08 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:02:09.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:02:08 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:02:09.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:02:08 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:02:09.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:02:08 smithi189 ceph-mon[31791]: Adjusting osd_memory_target on smithi189 to 176.4M 2024-04-24T09:02:09.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:02:08 smithi189 ceph-mon[31791]: Unable to set osd_memory_target on smithi189 to 185064857: error parsing value: Value '185064857' is below minimum 939524096 2024-04-24T09:02:09.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:02:08 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:02:09.209 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:02:08 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:02:09.209 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:02:08 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:02:09.209 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:02:08 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:02:09.209 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:02:08 smithi189 ceph-mon[31791]: Adjusting osd_memory_target on smithi083 to 176.4M 2024-04-24T09:02:09.209 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:02:08 smithi189 ceph-mon[31791]: Unable to set osd_memory_target on smithi083 to 185061990: error parsing value: Value '185061990' is below minimum 939524096 2024-04-24T09:02:09.209 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:02:08 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:02:09.209 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:02:08 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:02:09.209 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:02:08 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:02:09.611 INFO:tasks.workunit.client.0.smithi083.stdout: 1 61192 0.62 MB/sec execute 403 sec latency 58.946 ms 2024-04-24T09:02:10.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:02:10 smithi083 ceph-mon[27907]: pgmap v330: 129 pgs: 129 active+clean; 62 MiB data, 877 MiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 405 KiB/s wr, 72 op/s 2024-04-24T09:02:10.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:02:10 smithi138 ceph-mon[31441]: pgmap v330: 129 pgs: 129 active+clean; 62 MiB data, 877 MiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 405 KiB/s wr, 72 op/s 2024-04-24T09:02:10.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:02:10 smithi189 ceph-mon[31791]: pgmap v330: 129 pgs: 129 active+clean; 62 MiB data, 877 MiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 405 KiB/s wr, 72 op/s 2024-04-24T09:02:10.611 INFO:tasks.workunit.client.0.smithi083.stdout: 1 61275 0.62 MB/sec execute 404 sec latency 55.821 ms 2024-04-24T09:02:11.563 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.563+0000 7fdb57c00640 1 -- 172.21.15.83:0/3631410034 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb501000d0 msgr2=0x7fdb50100550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:11.563 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.563+0000 7fdb57c00640 1 --2- 172.21.15.83:0/3631410034 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb501000d0 0x7fdb50100550 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7fdb38009cd0 tx=0x7fdb3802a8f0 comp rx=0 tx=0).stop 2024-04-24T09:02:11.563 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.563+0000 7fdb57c00640 1 -- 172.21.15.83:0/3631410034 shutdown_connections 2024-04-24T09:02:11.564 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.563+0000 7fdb57c00640 1 --2- 172.21.15.83:0/3631410034 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb50100a90 0x7fdb5010ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:11.564 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.563+0000 7fdb57c00640 1 --2- 172.21.15.83:0/3631410034 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb501000d0 0x7fdb50100550 unknown :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:11.564 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.563+0000 7fdb57c00640 1 --2- 172.21.15.83:0/3631410034 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdb50107120 0x7fdb50107500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:11.564 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.563+0000 7fdb57c00640 1 -- 172.21.15.83:0/3631410034 >> 172.21.15.83:0/3631410034 conn(0x7fdb50077000 msgr2=0x7fdb500fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:02:11.564 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.563+0000 7fdb57c00640 1 -- 172.21.15.83:0/3631410034 shutdown_connections 2024-04-24T09:02:11.564 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.563+0000 7fdb57c00640 1 -- 172.21.15.83:0/3631410034 wait complete. 2024-04-24T09:02:11.566 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.564+0000 7fdb57c00640 1 Processor -- start 2024-04-24T09:02:11.566 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.564+0000 7fdb57c00640 1 -- start start 2024-04-24T09:02:11.566 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.565+0000 7fdb57c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdb501000d0 0x7fdb501a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:11.566 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.565+0000 7fdb57c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb50100a90 0x7fdb501a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:11.567 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.565+0000 7fdb57c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb50107120 0x7fdb5019a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:11.567 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.565+0000 7fdb57c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdb501122e0 con 0x7fdb50107120 2024-04-24T09:02:11.567 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.565+0000 7fdb57c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdb50112160 con 0x7fdb50100a90 2024-04-24T09:02:11.567 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.565+0000 7fdb57c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdb50112460 con 0x7fdb501000d0 2024-04-24T09:02:11.567 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.565+0000 7fdb55c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb50107120 0x7fdb5019a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:11.567 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.565+0000 7fdb55c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb50107120 0x7fdb5019a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41224/0 (socket says 172.21.15.83:41224) 2024-04-24T09:02:11.567 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.565+0000 7fdb55c00640 1 -- 172.21.15.83:0/395935987 learned_addr learned my addr 172.21.15.83:0/395935987 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:02:11.567 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.565+0000 7fdb55200640 1 --2- 172.21.15.83:0/395935987 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdb501000d0 0x7fdb501a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:11.567 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.565+0000 7fdb55c00640 1 -- 172.21.15.83:0/395935987 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdb501000d0 msgr2=0x7fdb501a0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:11.567 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.565+0000 7fdb55c00640 1 --2- 172.21.15.83:0/395935987 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdb501000d0 0x7fdb501a0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:11.567 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.565+0000 7fdb55c00640 1 -- 172.21.15.83:0/395935987 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb50100a90 msgr2=0x7fdb501a0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:11.567 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.565+0000 7fdb55c00640 1 --2- 172.21.15.83:0/395935987 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb50100a90 0x7fdb501a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:11.567 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.565+0000 7fdb55c00640 1 -- 172.21.15.83:0/395935987 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb5019af90 con 0x7fdb50107120 2024-04-24T09:02:11.567 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.565+0000 7fdb55200640 1 --2- 172.21.15.83:0/395935987 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdb501000d0 0x7fdb501a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:02:11.568 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.566+0000 7fdb55c00640 1 --2- 172.21.15.83:0/395935987 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb50107120 0x7fdb5019a7c0 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7fdb40009fd0 tx=0x7fdb4000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:02:11.568 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.566+0000 7fdb4d600640 1 -- 172.21.15.83:0/395935987 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb40011830 con 0x7fdb50107120 2024-04-24T09:02:11.568 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.566+0000 7fdb57c00640 1 -- 172.21.15.83:0/395935987 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb5019b280 con 0x7fdb50107120 2024-04-24T09:02:11.568 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.566+0000 7fdb4d600640 1 -- 172.21.15.83:0/395935987 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdb40016070 con 0x7fdb50107120 2024-04-24T09:02:11.568 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.566+0000 7fdb4d600640 1 -- 172.21.15.83:0/395935987 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb40010de0 con 0x7fdb50107120 2024-04-24T09:02:11.568 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.566+0000 7fdb57c00640 1 -- 172.21.15.83:0/395935987 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb501a7520 con 0x7fdb50107120 2024-04-24T09:02:11.569 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.570+0000 7fdb4d600640 1 -- 172.21.15.83:0/395935987 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fdb400119d0 con 0x7fdb50107120 2024-04-24T09:02:11.570 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.570+0000 7fdb5aa27740 1 Processor -- start 2024-04-24T09:02:11.570 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.570+0000 7fdb5aa27740 1 -- start start 2024-04-24T09:02:11.572 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.570+0000 7fdb5aa27740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7f93571b0 0x55d7f9357590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:11.572 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.570+0000 7fdb5aa27740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d7f9357ad0 0x55d7f9361e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:11.572 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.570+0000 7fdb5aa27740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d7f9362390 0x55d7f9364780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:11.572 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.570+0000 7fdb5aa27740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d7f92bd900 con 0x55d7f93571b0 2024-04-24T09:02:11.572 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.570+0000 7fdb5aa27740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d7f9263f20 con 0x55d7f9362390 2024-04-24T09:02:11.572 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.570+0000 7fdb5aa27740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d7f92b98c0 con 0x55d7f9357ad0 2024-04-24T09:02:11.572 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.571+0000 7fdb4fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7f93571b0 0x55d7f9357590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:11.572 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.571+0000 7fdb4fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7f93571b0 0x55d7f9357590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41236/0 (socket says 172.21.15.83:41236) 2024-04-24T09:02:11.572 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.571+0000 7fdb4fe00640 1 -- 172.21.15.83:0/3564542466 learned_addr learned my addr 172.21.15.83:0/3564542466 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:02:11.572 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.571+0000 7fdb55200640 1 --2- 172.21.15.83:0/3564542466 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d7f9357ad0 0x55d7f9361e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:11.572 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.571+0000 7fdb4d600640 1 --2- 172.21.15.83:0/395935987 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fdb340a3dc0 0x7fdb340a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:11.573 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.571+0000 7fdb4fe00640 1 -- 172.21.15.83:0/3564542466 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d7f9357ad0 msgr2=0x55d7f9361e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:11.573 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.571+0000 7fdb4fe00640 1 --2- 172.21.15.83:0/3564542466 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d7f9357ad0 0x55d7f9361e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:11.573 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.571+0000 7fdb4fe00640 1 -- 172.21.15.83:0/3564542466 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d7f9362390 msgr2=0x55d7f9364780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:11.573 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.571+0000 7fdb4fe00640 1 --2- 172.21.15.83:0/3564542466 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d7f9362390 0x55d7f9364780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:11.573 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.571+0000 7fdb4fe00640 1 -- 172.21.15.83:0/3564542466 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7f9288e10 con 0x55d7f93571b0 2024-04-24T09:02:11.573 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.572+0000 7fdb4d600640 1 -- 172.21.15.83:0/395935987 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9671+0+0 (secure 0 0 0) 0x7fdb400d03c0 con 0x7fdb50107120 2024-04-24T09:02:11.573 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.572+0000 7fdb55c00640 1 --2- 172.21.15.83:0/395935987 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fdb340a3dc0 0x7fdb340a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:11.573 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.572+0000 7fdb4fe00640 1 --2- 172.21.15.83:0/3564542466 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7f93571b0 0x55d7f9357590 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7fdb38005cc0 tx=0x7fdb38009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=706332e4bfb9a0b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:02:11.573 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.572+0000 7fdb46000640 1 -- 172.21.15.83:0/3564542466 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb38041070 con 0x55d7f93571b0 2024-04-24T09:02:11.573 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.572+0000 7fdb46000640 1 -- 172.21.15.83:0/3564542466 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdb38036070 con 0x55d7f93571b0 2024-04-24T09:02:11.573 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.572+0000 7fdb46000640 1 -- 172.21.15.83:0/3564542466 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb38004330 con 0x55d7f93571b0 2024-04-24T09:02:11.576 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.573+0000 7fdb5aa27740 1 -- 172.21.15.83:0/3564542466 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7f93571b0 msgr2=0x55d7f9357590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:11.576 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.573+0000 7fdb5aa27740 1 --2- 172.21.15.83:0/3564542466 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7f93571b0 0x55d7f9357590 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7fdb38005cc0 tx=0x7fdb38009a20 comp rx=0 tx=0).stop 2024-04-24T09:02:11.576 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.573+0000 7fdb55c00640 1 --2- 172.21.15.83:0/395935987 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fdb340a3dc0 0x7fdb340a6280 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fdb400a46d0 tx=0x7fdb400047d0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:02:11.576 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.573+0000 7fdb5aa27740 1 -- 172.21.15.83:0/3564542466 shutdown_connections 2024-04-24T09:02:11.577 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.573+0000 7fdb5aa27740 1 --2- 172.21.15.83:0/3564542466 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d7f9362390 0x55d7f9364780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:11.577 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.573+0000 7fdb5aa27740 1 --2- 172.21.15.83:0/3564542466 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d7f9357ad0 0x55d7f9361e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:11.577 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.573+0000 7fdb5aa27740 1 --2- 172.21.15.83:0/3564542466 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7f93571b0 0x55d7f9357590 unknown :-1 s=CLOSED pgs=1741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:11.577 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.573+0000 7fdb5aa27740 1 -- 172.21.15.83:0/3564542466 >> 172.21.15.83:0/3564542466 conn(0x55d7f93536c0 msgr2=0x55d7f9353aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:02:11.577 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.573+0000 7fdb5aa27740 1 -- 172.21.15.83:0/3564542466 shutdown_connections 2024-04-24T09:02:11.577 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.573+0000 7fdb5aa27740 1 -- 172.21.15.83:0/3564542466 wait complete. 2024-04-24T09:02:11.577 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.574+0000 7fdb5aa27740 1 -- start start 2024-04-24T09:02:11.577 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.574+0000 7fdb5aa27740 1 Processor -- start 2024-04-24T09:02:11.577 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.574+0000 7fdb5aa27740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d7f93571b0 0x55d7f935f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:11.577 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.574+0000 7fdb5aa27740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d7f9357ad0 0x55d7f935f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:11.577 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.574+0000 7fdb5aa27740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7f9362390 0x55d7f935fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:11.577 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.574+0000 7fdb5aa27740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d7f9364f60 con 0x55d7f9362390 2024-04-24T09:02:11.577 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.574+0000 7fdb5aa27740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d7f92772c0 con 0x55d7f93571b0 2024-04-24T09:02:11.577 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.574+0000 7fdb5aa27740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d7f93650e0 con 0x55d7f9357ad0 2024-04-24T09:02:11.578 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.575+0000 7fdb5aa27740 10 client.0 _ll_register_callbacks cb 0x55d7f8fcfa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:02:11.578 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.575+0000 7fdb4fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7f9362390 0x55d7f935fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:11.578 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.575+0000 7fdb4fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7f9362390 0x55d7f935fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41244/0 (socket says 172.21.15.83:41244) 2024-04-24T09:02:11.578 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.575+0000 7fdb4fe00640 1 -- 172.21.15.83:0/4126408647 learned_addr learned my addr 172.21.15.83:0/4126408647 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:02:11.578 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.575+0000 7fdb4fe00640 1 --2- 172.21.15.83:0/4126408647 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d7f93571b0 0x55d7f935f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:11.578 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.575+0000 7fdb55200640 1 --2- 172.21.15.83:0/4126408647 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d7f9357ad0 0x55d7f935f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:11.578 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.575+0000 7fdb4fe00640 1 -- 172.21.15.83:0/4126408647 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d7f9357ad0 msgr2=0x55d7f935f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:11.578 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.575+0000 7fdb4fe00640 1 --2- 172.21.15.83:0/4126408647 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d7f9357ad0 0x55d7f935f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:11.578 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.575+0000 7fdb4fe00640 1 -- 172.21.15.83:0/4126408647 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d7f93571b0 msgr2=0x55d7f935f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:11.578 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.575+0000 7fdb4fe00640 1 --2- 172.21.15.83:0/4126408647 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d7f93571b0 0x55d7f935f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:11.578 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.575+0000 7fdb4fe00640 1 -- 172.21.15.83:0/4126408647 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7f9287fd0 con 0x55d7f9362390 2024-04-24T09:02:11.578 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.576+0000 7fdb4fe00640 1 --2- 172.21.15.83:0/4126408647 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7f9362390 0x55d7f935fdc0 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7fdb38004440 tx=0x7fdb38003f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1997e5a3206b67ab server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:02:11.578 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.576+0000 7fdb3ea00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:02:11.579 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.576+0000 7fdb3ea00640 1 -- 172.21.15.83:0/4126408647 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb38041040 con 0x55d7f9362390 2024-04-24T09:02:11.579 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.576+0000 7fdb5aa27740 1 -- 172.21.15.83:0/4126408647 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb38036070 con 0x55d7f9362390 2024-04-24T09:02:11.579 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.576+0000 7fdb3ea00640 1 -- 172.21.15.83:0/4126408647 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9671+0+0 (secure 0 0 0) 0x7fdb38004720 con 0x55d7f9362390 2024-04-24T09:02:11.579 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.576+0000 7fdb3ea00640 1 -- 172.21.15.83:0/4126408647 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fdb38030860 con 0x55d7f9362390 2024-04-24T09:02:11.579 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.576+0000 7fdb5aa27740 10 client.19386 fetch_fsmap learned FSMap version 40 2024-04-24T09:02:11.579 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.576+0000 7fdb5aa27740 1 -- 172.21.15.83:0/4126408647 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55d7f8fd0210 con 0x55d7f9362390 2024-04-24T09:02:11.580 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.577+0000 7fdb3ea00640 1 -- 172.21.15.83:0/4126408647 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdb38036070 con 0x55d7f9362390 2024-04-24T09:02:11.580 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.577+0000 7fdb3ea00640 1 -- 172.21.15.83:0/4126408647 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 40) ==== 8382+0+0 (secure 0 0 0) 0x7fdb38046b40 con 0x55d7f9362390 2024-04-24T09:02:11.580 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.577+0000 7fdb5aa27740 10 client.19386 fetch_fsmap finished waiting for FSMap version 40 2024-04-24T09:02:11.580 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.577+0000 7fdb5aa27740 10 client.19386 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:02:11.580 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.577+0000 7fdb5aa27740 20 client.19386 populate_metadata read hostname 'smithi083' 2024-04-24T09:02:11.580 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.577+0000 7fdb5aa27740 1 --2- 172.21.15.83:0/4126408647 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d7f9372320 0x55d7f9374710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:11.580 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.577+0000 7fdb5aa27740 4 client.19386 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:02:11.581 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.577+0000 7fdb5aa27740 1 -- 172.21.15.83:0/4126408647 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d7f92cb320 con 0x55d7f9372320 2024-04-24T09:02:11.581 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.578+0000 7fdb55200640 1 --2- 172.21.15.83:0/4126408647 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d7f9372320 0x55d7f9374710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:11.581 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.579+0000 7fdb55200640 1 --2- 172.21.15.83:0/4126408647 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d7f9372320 0x55d7f9374710 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd4e61ffd4d1235f server_cookie=3d50b625068a57e3 in_seq=0 out_seq=0 2024-04-24T09:02:11.581 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.579+0000 7fdb3ea00640 10 client.19386 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:02:11.582 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.581+0000 7fdb3ea00640 1 -- 172.21.15.83:0/4126408647 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d7f92cb320 con 0x55d7f9372320 2024-04-24T09:02:11.582 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.581+0000 7fdb3ea00640 10 client.19386 handle_command_reply: tid=0 2024-04-24T09:02:11.582 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.581+0000 7fdb5aa27740 1 client.19386 shutdown 2024-04-24T09:02:11.582 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.581+0000 7fdb5aa27740 1 -- 172.21.15.83:0/4126408647 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7f9362390 msgr2=0x55d7f935fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:11.582 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.581+0000 7fdb5aa27740 1 --2- 172.21.15.83:0/4126408647 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7f9362390 0x55d7f935fdc0 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7fdb38004440 tx=0x7fdb38003f30 comp rx=0 tx=0).stop 2024-04-24T09:02:11.582 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.581+0000 7fdb5aa27740 1 -- 172.21.15.83:0/4126408647 shutdown_connections 2024-04-24T09:02:11.582 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.581+0000 7fdb5aa27740 1 --2- 172.21.15.83:0/4126408647 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d7f9372320 0x55d7f9374710 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:11.582 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.581+0000 7fdb5aa27740 1 --2- 172.21.15.83:0/4126408647 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7f9362390 0x55d7f935fdc0 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:11.582 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.581+0000 7fdb5aa27740 1 --2- 172.21.15.83:0/4126408647 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d7f9357ad0 0x55d7f935f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:11.583 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.581+0000 7fdb3ea00640 0 client.19386 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:02:11.583 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.581+0000 7fdb5aa27740 1 --2- 172.21.15.83:0/4126408647 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d7f93571b0 0x55d7f935f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:11.583 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.581+0000 7fdb5aa27740 1 -- 172.21.15.83:0/4126408647 >> 172.21.15.83:0/4126408647 conn(0x55d7f93536c0 msgr2=0x55d7f9359440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:02:11.583 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.581+0000 7fdb5aa27740 1 -- 172.21.15.83:0/4126408647 shutdown_connections 2024-04-24T09:02:11.583 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.581+0000 7fdb5aa27740 1 -- 172.21.15.83:0/4126408647 wait complete. 2024-04-24T09:02:11.583 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.581+0000 7fdb5aa27740 20 client.19386 trim_cache size 0 max 16384 2024-04-24T09:02:11.598 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.598+0000 7fdb5aa27740 1 Processor -- start 2024-04-24T09:02:11.598 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.598+0000 7fdb5aa27740 1 -- start start 2024-04-24T09:02:11.599 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.598+0000 7fdb5aa27740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7f93571b0 0x55d7f936de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:11.599 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.598+0000 7fdb5aa27740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d7f9357ad0 0x55d7f93526c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:11.599 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.598+0000 7fdb5aa27740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d7f9362390 0x55d7f9379d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:11.599 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.598+0000 7fdb5aa27740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdb501b18c0 con 0x55d7f93571b0 2024-04-24T09:02:11.599 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.598+0000 7fdb5aa27740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdb501b34f0 con 0x55d7f9357ad0 2024-04-24T09:02:11.599 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.598+0000 7fdb5aa27740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdb501b1f90 con 0x55d7f9362390 2024-04-24T09:02:11.599 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.599+0000 7fdb4fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7f93571b0 0x55d7f936de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:11.599 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.599+0000 7fdb4fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7f93571b0 0x55d7f936de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41248/0 (socket says 172.21.15.83:41248) 2024-04-24T09:02:11.599 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.599+0000 7fdb4fe00640 1 -- 172.21.15.83:0/2485803082 learned_addr learned my addr 172.21.15.83:0/2485803082 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:02:11.600 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.599+0000 7fdb55200640 1 --2- 172.21.15.83:0/2485803082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d7f9357ad0 0x55d7f93526c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:11.600 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.599+0000 7fdb4fe00640 1 -- 172.21.15.83:0/2485803082 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d7f9362390 msgr2=0x55d7f9379d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:11.600 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.599+0000 7fdb4fe00640 1 --2- 172.21.15.83:0/2485803082 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d7f9362390 0x55d7f9379d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:11.600 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.599+0000 7fdb4fe00640 1 -- 172.21.15.83:0/2485803082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d7f9357ad0 msgr2=0x55d7f93526c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:11.600 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.599+0000 7fdb4fe00640 1 --2- 172.21.15.83:0/2485803082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d7f9357ad0 0x55d7f93526c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:11.601 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.599+0000 7fdb4fe00640 1 -- 172.21.15.83:0/2485803082 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7f909c620 con 0x55d7f93571b0 2024-04-24T09:02:11.601 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.599+0000 7fdb55200640 1 --2- 172.21.15.83:0/2485803082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d7f9357ad0 0x55d7f93526c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:02:11.601 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.600+0000 7fdb4fe00640 1 --2- 172.21.15.83:0/2485803082 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7f93571b0 0x55d7f936de20 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7fdb38009960 tx=0x7fdb380093f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0dc76bc307fed12 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:02:11.601 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.600+0000 7fdb45600640 1 -- 172.21.15.83:0/2485803082 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb38041070 con 0x55d7f93571b0 2024-04-24T09:02:11.601 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.600+0000 7fdb45600640 1 -- 172.21.15.83:0/2485803082 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdb38036040 con 0x55d7f93571b0 2024-04-24T09:02:11.601 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.600+0000 7fdb45600640 1 -- 172.21.15.83:0/2485803082 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb38004720 con 0x55d7f93571b0 2024-04-24T09:02:11.603 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.601+0000 7fdb5aa27740 1 -- 172.21.15.83:0/2485803082 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7f93571b0 msgr2=0x55d7f936de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:11.603 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.601+0000 7fdb5aa27740 1 --2- 172.21.15.83:0/2485803082 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7f93571b0 0x55d7f936de20 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7fdb38009960 tx=0x7fdb380093f0 comp rx=0 tx=0).stop 2024-04-24T09:02:11.603 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.601+0000 7fdb5aa27740 1 -- 172.21.15.83:0/2485803082 shutdown_connections 2024-04-24T09:02:11.603 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.601+0000 7fdb5aa27740 1 --2- 172.21.15.83:0/2485803082 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d7f9362390 0x55d7f9379d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:11.603 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.601+0000 7fdb5aa27740 1 --2- 172.21.15.83:0/2485803082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d7f9357ad0 0x55d7f93526c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:11.603 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.601+0000 7fdb5aa27740 1 --2- 172.21.15.83:0/2485803082 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7f93571b0 0x55d7f936de20 unknown :-1 s=CLOSED pgs=1743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:11.603 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.601+0000 7fdb5aa27740 1 -- 172.21.15.83:0/2485803082 >> 172.21.15.83:0/2485803082 conn(0x55d7f93536c0 msgr2=0x55d7f9351fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:02:11.604 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.601+0000 7fdb5aa27740 1 -- 172.21.15.83:0/2485803082 shutdown_connections 2024-04-24T09:02:11.604 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.601+0000 7fdb5aa27740 1 -- 172.21.15.83:0/2485803082 wait complete. 2024-04-24T09:02:11.604 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.601+0000 7fdb5aa27740 1 -- start start 2024-04-24T09:02:11.604 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.602+0000 7fdb5aa27740 1 Processor -- start 2024-04-24T09:02:11.604 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.602+0000 7fdb5aa27740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d7f93571b0 0x55d7f936a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:11.604 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.602+0000 7fdb5aa27740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7f9357ad0 0x55d7f935be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:11.604 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.602+0000 7fdb5aa27740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d7f9362390 0x55d7f935c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:11.604 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.602+0000 7fdb5aa27740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdb501b2dc0 con 0x55d7f9357ad0 2024-04-24T09:02:11.604 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.602+0000 7fdb5aa27740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdb501b2690 con 0x55d7f93571b0 2024-04-24T09:02:11.604 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.602+0000 7fdb5aa27740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdb501af8c0 con 0x55d7f9362390 2024-04-24T09:02:11.604 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.602+0000 7fdb5aa27740 10 client.0 _ll_register_callbacks cb 0x7fdb38005f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:02:11.604 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.602+0000 7fdb55200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7f9357ad0 0x55d7f935be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:11.604 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.603+0000 7fdb55200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7f9357ad0 0x55d7f935be80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41252/0 (socket says 172.21.15.83:41252) 2024-04-24T09:02:11.604 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.603+0000 7fdb55200640 1 -- 172.21.15.83:0/379687902 learned_addr learned my addr 172.21.15.83:0/379687902 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:02:11.604 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.603+0000 7fdb4fe00640 1 --2- 172.21.15.83:0/379687902 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d7f93571b0 0x55d7f936a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:11.604 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.603+0000 7fdb55200640 1 -- 172.21.15.83:0/379687902 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d7f9362390 msgr2=0x55d7f935c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:11.605 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.603+0000 7fdb55200640 1 --2- 172.21.15.83:0/379687902 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d7f9362390 0x55d7f935c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:11.605 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.603+0000 7fdb55200640 1 -- 172.21.15.83:0/379687902 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d7f93571b0 msgr2=0x55d7f936a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:11.605 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.603+0000 7fdb55200640 1 --2- 172.21.15.83:0/379687902 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d7f93571b0 0x55d7f936a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:11.605 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.603+0000 7fdb55200640 1 -- 172.21.15.83:0/379687902 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7f93d7580 con 0x55d7f9357ad0 2024-04-24T09:02:11.605 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.603+0000 7fdb4fe00640 1 --2- 172.21.15.83:0/379687902 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d7f93571b0 0x55d7f936a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:02:11.605 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.603+0000 7fdb55200640 1 --2- 172.21.15.83:0/379687902 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7f9357ad0 0x55d7f935be80 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x55d7f9358750 tx=0x7fdb48018df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a3f43cad5fc9162 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:02:11.605 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.603+0000 7fdb3ea00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:02:11.607 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.604+0000 7fdb3ea00640 1 -- 172.21.15.83:0/379687902 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb4800fe20 con 0x55d7f9357ad0 2024-04-24T09:02:11.607 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.604+0000 7fdb3ea00640 1 -- 172.21.15.83:0/379687902 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9671+0+0 (secure 0 0 0) 0x7fdb480182c0 con 0x55d7f9357ad0 2024-04-24T09:02:11.607 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.604+0000 7fdb5aa27740 1 -- 172.21.15.83:0/379687902 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb38036040 con 0x55d7f9357ad0 2024-04-24T09:02:11.607 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.604+0000 7fdb3ea00640 1 -- 172.21.15.83:0/379687902 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fdb48029050 con 0x55d7f9357ad0 2024-04-24T09:02:11.607 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.604+0000 7fdb5aa27740 10 client.19392 fetch_fsmap learned FSMap version 40 2024-04-24T09:02:11.607 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.604+0000 7fdb5aa27740 1 -- 172.21.15.83:0/379687902 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55d7f93d86a0 con 0x55d7f9357ad0 2024-04-24T09:02:11.607 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.604+0000 7fdb3ea00640 1 -- 172.21.15.83:0/379687902 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdb38036040 con 0x55d7f9357ad0 2024-04-24T09:02:11.607 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.605+0000 7fdb3ea00640 1 -- 172.21.15.83:0/379687902 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 40) ==== 8382+0+0 (secure 0 0 0) 0x7fdb48021b10 con 0x55d7f9357ad0 2024-04-24T09:02:11.607 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.605+0000 7fdb5aa27740 10 client.19392 fetch_fsmap finished waiting for FSMap version 40 2024-04-24T09:02:11.607 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.605+0000 7fdb5aa27740 10 client.19392 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:02:11.607 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.605+0000 7fdb5aa27740 20 client.19392 populate_metadata read hostname 'smithi083' 2024-04-24T09:02:11.607 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.605+0000 7fdb5aa27740 1 --2- 172.21.15.83:0/379687902 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d7f93d8ea0 0x55d7f93db2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:11.608 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.605+0000 7fdb5aa27740 4 client.19392 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T09:02:11.608 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.605+0000 7fdb5aa27740 1 -- 172.21.15.83:0/379687902 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d7f92c5190 con 0x55d7f93d8ea0 2024-04-24T09:02:11.608 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.605+0000 7fdb4fe00640 1 --2- 172.21.15.83:0/379687902 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d7f93d8ea0 0x55d7f93db2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:11.608 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.606+0000 7fdb4fe00640 1 --2- 172.21.15.83:0/379687902 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d7f93d8ea0 0x55d7f93db2c0 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7263e6a230228c server_cookie=d74a72fcd4602175 in_seq=0 out_seq=0 2024-04-24T09:02:11.608 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.606+0000 7fdb3ea00640 10 client.19392 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:02:11.611 INFO:tasks.workunit.client.0.smithi083.stdout: 1 61400 0.62 MB/sec execute 405 sec latency 53.204 ms 2024-04-24T09:02:11.670 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.670+0000 7fdb3ea00640 1 -- 172.21.15.83:0/379687902 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d7f92c5190 con 0x55d7f93d8ea0 2024-04-24T09:02:11.670 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.670+0000 7fdb3ea00640 10 client.19392 handle_command_reply: tid=0 2024-04-24T09:02:11.670 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.670+0000 7fdb5aa27740 1 client.19392 shutdown 2024-04-24T09:02:11.671 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.671+0000 7fdb5aa27740 1 -- 172.21.15.83:0/379687902 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7f9357ad0 msgr2=0x55d7f935be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:11.671 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.671+0000 7fdb5aa27740 1 --2- 172.21.15.83:0/379687902 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7f9357ad0 0x55d7f935be80 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x55d7f9358750 tx=0x7fdb48018df0 comp rx=0 tx=0).stop 2024-04-24T09:02:11.672 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.671+0000 7fdb5aa27740 1 -- 172.21.15.83:0/379687902 shutdown_connections 2024-04-24T09:02:11.672 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.671+0000 7fdb5aa27740 1 --2- 172.21.15.83:0/379687902 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d7f93d8ea0 0x55d7f93db2c0 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:11.672 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.671+0000 7fdb5aa27740 1 --2- 172.21.15.83:0/379687902 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d7f9362390 0x55d7f935c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:11.672 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.671+0000 7fdb5aa27740 1 --2- 172.21.15.83:0/379687902 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7f9357ad0 0x55d7f935be80 unknown :-1 s=CLOSED pgs=1744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:11.672 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.671+0000 7fdb5aa27740 1 --2- 172.21.15.83:0/379687902 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d7f93571b0 0x55d7f936a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:11.672 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.671+0000 7fdb3ea00640 0 client.19392 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:02:11.672 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.671+0000 7fdb5aa27740 1 -- 172.21.15.83:0/379687902 >> 172.21.15.83:0/379687902 conn(0x55d7f93536c0 msgr2=0x55d7f935fb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:02:11.672 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.671+0000 7fdb5aa27740 1 -- 172.21.15.83:0/379687902 shutdown_connections 2024-04-24T09:02:11.672 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.672+0000 7fdb5aa27740 1 -- 172.21.15.83:0/379687902 wait complete. 2024-04-24T09:02:11.672 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.672+0000 7fdb5aa27740 20 client.19392 trim_cache size 0 max 16384 2024-04-24T09:02:11.673 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T09:02:11.673 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T09:02:11.673 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T09:02:11.673 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T09:02:11.674 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.673+0000 7fdb57c00640 1 -- 172.21.15.83:0/395935987 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fdb340a3dc0 msgr2=0x7fdb340a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:11.674 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.673+0000 7fdb57c00640 1 --2- 172.21.15.83:0/395935987 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fdb340a3dc0 0x7fdb340a6280 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fdb400a46d0 tx=0x7fdb400047d0 comp rx=0 tx=0).stop 2024-04-24T09:02:11.674 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.673+0000 7fdb57c00640 1 -- 172.21.15.83:0/395935987 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb50107120 msgr2=0x7fdb5019a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:11.674 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.673+0000 7fdb57c00640 1 --2- 172.21.15.83:0/395935987 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb50107120 0x7fdb5019a7c0 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7fdb40009fd0 tx=0x7fdb4000b8e0 comp rx=0 tx=0).stop 2024-04-24T09:02:11.674 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.673+0000 7fdb57c00640 1 -- 172.21.15.83:0/395935987 shutdown_connections 2024-04-24T09:02:11.674 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.673+0000 7fdb57c00640 1 --2- 172.21.15.83:0/395935987 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fdb340a3dc0 0x7fdb340a6280 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:11.674 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.673+0000 7fdb57c00640 1 --2- 172.21.15.83:0/395935987 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb50107120 0x7fdb5019a7c0 unknown :-1 s=CLOSED pgs=1740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:11.674 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.673+0000 7fdb57c00640 1 --2- 172.21.15.83:0/395935987 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb50100a90 0x7fdb501a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:11.674 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.673+0000 7fdb57c00640 1 --2- 172.21.15.83:0/395935987 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdb501000d0 0x7fdb501a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:11.674 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.673+0000 7fdb57c00640 1 -- 172.21.15.83:0/395935987 >> 172.21.15.83:0/395935987 conn(0x7fdb50077000 msgr2=0x7fdb5010e170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:02:11.675 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.673+0000 7fdb57c00640 1 -- 172.21.15.83:0/395935987 shutdown_connections 2024-04-24T09:02:11.675 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:11.673+0000 7fdb57c00640 1 -- 172.21.15.83:0/395935987 wait complete. 2024-04-24T09:02:12.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:02:12 smithi083 ceph-mon[27907]: pgmap v331: 129 pgs: 129 active+clean; 62 MiB data, 877 MiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 405 KiB/s wr, 72 op/s 2024-04-24T09:02:12.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:02:12 smithi138 ceph-mon[31441]: pgmap v331: 129 pgs: 129 active+clean; 62 MiB data, 877 MiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 405 KiB/s wr, 72 op/s 2024-04-24T09:02:12.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:02:12 smithi189 ceph-mon[31791]: pgmap v331: 129 pgs: 129 active+clean; 62 MiB data, 877 MiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 405 KiB/s wr, 72 op/s 2024-04-24T09:02:12.611 INFO:tasks.workunit.client.0.smithi083.stdout: 1 61508 0.62 MB/sec execute 406 sec latency 51.467 ms 2024-04-24T09:02:13.611 INFO:tasks.workunit.client.0.smithi083.stdout: 1 61617 0.62 MB/sec execute 407 sec latency 55.276 ms 2024-04-24T09:02:14.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:02:14 smithi083 ceph-mon[27907]: pgmap v332: 129 pgs: 129 active+clean; 58 MiB data, 878 MiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 618 KiB/s wr, 92 op/s 2024-04-24T09:02:14.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:02:14 smithi138 ceph-mon[31441]: pgmap v332: 129 pgs: 129 active+clean; 58 MiB data, 878 MiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 618 KiB/s wr, 92 op/s 2024-04-24T09:02:14.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:02:14 smithi189 ceph-mon[31791]: pgmap v332: 129 pgs: 129 active+clean; 58 MiB data, 878 MiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 618 KiB/s wr, 92 op/s 2024-04-24T09:02:14.611 INFO:tasks.workunit.client.0.smithi083.stdout: 1 61781 0.62 MB/sec execute 408 sec latency 63.522 ms 2024-04-24T09:02:15.612 INFO:tasks.workunit.client.0.smithi083.stdout: 1 61866 0.62 MB/sec execute 409 sec latency 69.665 ms 2024-04-24T09:02:15.954 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:02:15 smithi138 ceph-mon[31441]: osdmap e106: 12 total, 12 up, 12 in 2024-04-24T09:02:15.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:02:15 smithi138 ceph-mon[31441]: pgmap v334: 129 pgs: 129 active+clean; 58 MiB data, 878 MiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 772 KiB/s wr, 115 op/s 2024-04-24T09:02:15.956 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:02:15 smithi189 ceph-mon[31791]: osdmap e106: 12 total, 12 up, 12 in 2024-04-24T09:02:15.956 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:02:15 smithi189 ceph-mon[31791]: pgmap v334: 129 pgs: 129 active+clean; 58 MiB data, 878 MiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 772 KiB/s wr, 115 op/s 2024-04-24T09:02:16.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:02:15 smithi083 ceph-mon[27907]: osdmap e106: 12 total, 12 up, 12 in 2024-04-24T09:02:16.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:02:15 smithi083 ceph-mon[27907]: pgmap v334: 129 pgs: 129 active+clean; 58 MiB data, 878 MiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 772 KiB/s wr, 115 op/s 2024-04-24T09:02:16.612 INFO:tasks.workunit.client.0.smithi083.stdout: 1 61978 0.62 MB/sec execute 410 sec latency 66.505 ms 2024-04-24T09:02:17.612 INFO:tasks.workunit.client.0.smithi083.stdout: 1 62105 0.62 MB/sec execute 411 sec latency 71.072 ms 2024-04-24T09:02:18.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:02:17 smithi083 ceph-mon[27907]: pgmap v335: 129 pgs: 129 active+clean; 58 MiB data, 878 MiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 303 KiB/s wr, 46 op/s 2024-04-24T09:02:18.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:02:17 smithi138 ceph-mon[31441]: pgmap v335: 129 pgs: 129 active+clean; 58 MiB data, 878 MiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 303 KiB/s wr, 46 op/s 2024-04-24T09:02:18.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:02:17 smithi189 ceph-mon[31791]: pgmap v335: 129 pgs: 129 active+clean; 58 MiB data, 878 MiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 303 KiB/s wr, 46 op/s 2024-04-24T09:02:18.612 INFO:tasks.workunit.client.0.smithi083.stdout: 1 62204 0.62 MB/sec execute 412 sec latency 70.591 ms 2024-04-24T09:02:19.612 INFO:tasks.workunit.client.0.smithi083.stdout: 1 62368 0.62 MB/sec execute 413 sec latency 53.452 ms 2024-04-24T09:02:20.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:02:19 smithi083 ceph-mon[27907]: pgmap v336: 129 pgs: 129 active+clean; 59 MiB data, 864 MiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 379 KiB/s wr, 67 op/s 2024-04-24T09:02:20.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:02:19 smithi083 ceph-mon[27907]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:02:20.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:02:19 smithi138 ceph-mon[31441]: pgmap v336: 129 pgs: 129 active+clean; 59 MiB data, 864 MiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 379 KiB/s wr, 67 op/s 2024-04-24T09:02:20.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:02:19 smithi138 ceph-mon[31441]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:02:20.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:02:19 smithi189 ceph-mon[31791]: pgmap v336: 129 pgs: 129 active+clean; 59 MiB data, 864 MiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 379 KiB/s wr, 67 op/s 2024-04-24T09:02:20.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:02:19 smithi189 ceph-mon[31791]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:02:20.612 INFO:tasks.workunit.client.0.smithi083.stdout: 1 62508 0.61 MB/sec execute 414 sec latency 59.957 ms 2024-04-24T09:02:21.612 INFO:tasks.workunit.client.0.smithi083.stdout: 1 62658 0.62 MB/sec execute 415 sec latency 54.462 ms 2024-04-24T09:02:22.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:02:21 smithi083 ceph-mon[27907]: pgmap v337: 129 pgs: 129 active+clean; 59 MiB data, 864 MiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 379 KiB/s wr, 67 op/s 2024-04-24T09:02:22.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:02:21 smithi138 ceph-mon[31441]: pgmap v337: 129 pgs: 129 active+clean; 59 MiB data, 864 MiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 379 KiB/s wr, 67 op/s 2024-04-24T09:02:22.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:02:21 smithi189 ceph-mon[31791]: pgmap v337: 129 pgs: 129 active+clean; 59 MiB data, 864 MiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 379 KiB/s wr, 67 op/s 2024-04-24T09:02:22.612 INFO:tasks.workunit.client.0.smithi083.stdout: 1 62732 0.61 MB/sec execute 416 sec latency 53.879 ms 2024-04-24T09:02:23.612 INFO:tasks.workunit.client.0.smithi083.stdout: 1 62823 0.61 MB/sec execute 417 sec latency 49.195 ms 2024-04-24T09:02:24.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:02:23 smithi083 ceph-mon[27907]: pgmap v338: 129 pgs: 129 active+clean; 60 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 277 KiB/s wr, 47 op/s 2024-04-24T09:02:24.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:02:23 smithi138 ceph-mon[31441]: pgmap v338: 129 pgs: 129 active+clean; 60 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 277 KiB/s wr, 47 op/s 2024-04-24T09:02:24.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:02:23 smithi189 ceph-mon[31791]: pgmap v338: 129 pgs: 129 active+clean; 60 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 277 KiB/s wr, 47 op/s 2024-04-24T09:02:24.243 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T09:02:24.243 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T09:02:24.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.472+0000 7fd44dc00640 1 -- 172.21.15.83:0/2783871116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4481044b0 msgr2=0x7fd448104890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:24.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.472+0000 7fd44dc00640 1 --2- 172.21.15.83:0/2783871116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4481044b0 0x7fd448104890 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7fd438009900 tx=0x7fd43802a3f0 comp rx=0 tx=0).stop 2024-04-24T09:02:24.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.472+0000 7fd44dc00640 1 -- 172.21.15.83:0/2783871116 shutdown_connections 2024-04-24T09:02:24.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.472+0000 7fd44dc00640 1 --2- 172.21.15.83:0/2783871116 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd448109580 0x7fd448111150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:24.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.472+0000 7fd44dc00640 1 --2- 172.21.15.83:0/2783871116 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd448104e60 0x7fd448108ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:24.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.472+0000 7fd44dc00640 1 --2- 172.21.15.83:0/2783871116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4481044b0 0x7fd448104890 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:24.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.472+0000 7fd44dc00640 1 -- 172.21.15.83:0/2783871116 >> 172.21.15.83:0/2783871116 conn(0x7fd4480ffd90 msgr2=0x7fd4481021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:02:24.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.472+0000 7fd44dc00640 1 -- 172.21.15.83:0/2783871116 shutdown_connections 2024-04-24T09:02:24.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.472+0000 7fd44dc00640 1 -- 172.21.15.83:0/2783871116 wait complete. 2024-04-24T09:02:24.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.473+0000 7fd44dc00640 1 Processor -- start 2024-04-24T09:02:24.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.473+0000 7fd44dc00640 1 -- start start 2024-04-24T09:02:24.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.474+0000 7fd44dc00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd4481044b0 0x7fd4481a0850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:24.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.474+0000 7fd44dc00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd448109580 0x7fd4481a0d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:24.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.474+0000 7fd44dc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd44819aa00 0x7fd44819aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:24.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.474+0000 7fd44dc00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd4481145c0 con 0x7fd44819aa00 2024-04-24T09:02:24.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.474+0000 7fd44dc00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd448114440 con 0x7fd448109580 2024-04-24T09:02:24.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.474+0000 7fd44dc00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd448114740 con 0x7fd4481044b0 2024-04-24T09:02:24.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.474+0000 7fd447400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd44819aa00 0x7fd44819aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:24.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.474+0000 7fd447400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd44819aa00 0x7fd44819aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51118/0 (socket says 172.21.15.83:51118) 2024-04-24T09:02:24.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.474+0000 7fd447400640 1 -- 172.21.15.83:0/45085967 learned_addr learned my addr 172.21.15.83:0/45085967 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:02:24.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.474+0000 7fd446a00640 1 --2- 172.21.15.83:0/45085967 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd4481044b0 0x7fd4481a0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:24.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.474+0000 7fd446000640 1 --2- 172.21.15.83:0/45085967 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd448109580 0x7fd4481a0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:24.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.475+0000 7fd447400640 1 -- 172.21.15.83:0/45085967 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd4481044b0 msgr2=0x7fd4481a0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:24.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.475+0000 7fd447400640 1 --2- 172.21.15.83:0/45085967 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd4481044b0 0x7fd4481a0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:24.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.475+0000 7fd447400640 1 -- 172.21.15.83:0/45085967 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd448109580 msgr2=0x7fd4481a0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:24.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.475+0000 7fd447400640 1 --2- 172.21.15.83:0/45085967 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd448109580 0x7fd4481a0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:24.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.475+0000 7fd447400640 1 -- 172.21.15.83:0/45085967 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd44819b520 con 0x7fd44819aa00 2024-04-24T09:02:24.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.475+0000 7fd447400640 1 --2- 172.21.15.83:0/45085967 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd44819aa00 0x7fd44819aeb0 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7fd44000b970 tx=0x7fd44000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:02:24.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.475+0000 7fd43f400640 1 -- 172.21.15.83:0/45085967 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd440019070 con 0x7fd44819aa00 2024-04-24T09:02:24.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.475+0000 7fd43f400640 1 -- 172.21.15.83:0/45085967 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd440014440 con 0x7fd44819aa00 2024-04-24T09:02:24.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.475+0000 7fd43f400640 1 -- 172.21.15.83:0/45085967 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4400075d0 con 0x7fd44819aa00 2024-04-24T09:02:24.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.475+0000 7fd44dc00640 1 -- 172.21.15.83:0/45085967 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd44819b810 con 0x7fd44819aa00 2024-04-24T09:02:24.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.475+0000 7fd44dc00640 1 -- 172.21.15.83:0/45085967 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4481a7a60 con 0x7fd44819aa00 2024-04-24T09:02:24.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.479+0000 7fd43f400640 1 -- 172.21.15.83:0/45085967 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd44001c070 con 0x7fd44819aa00 2024-04-24T09:02:24.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.479+0000 7fd450ba1740 1 Processor -- start 2024-04-24T09:02:24.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.479+0000 7fd450ba1740 1 -- start start 2024-04-24T09:02:24.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.479+0000 7fd450ba1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a7929a860 0x563a7929ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:24.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.479+0000 7fd450ba1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563a7929b180 0x563a792a5310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:24.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.479+0000 7fd450ba1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a792a5850 0x563a792a7c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:24.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.479+0000 7fd450ba1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x563a792001f0 con 0x563a7929a860 2024-04-24T09:02:24.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.479+0000 7fd450ba1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563a791a6350 con 0x563a792a5850 2024-04-24T09:02:24.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.479+0000 7fd450ba1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x563a791fc1b0 con 0x563a7929b180 2024-04-24T09:02:24.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.480+0000 7fd446000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a7929a860 0x563a7929ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:24.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.480+0000 7fd446000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a7929a860 0x563a7929ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51132/0 (socket says 172.21.15.83:51132) 2024-04-24T09:02:24.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.480+0000 7fd446000640 1 -- 172.21.15.83:0/4178784607 learned_addr learned my addr 172.21.15.83:0/4178784607 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:02:24.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.480+0000 7fd446a00640 1 --2- 172.21.15.83:0/4178784607 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563a7929b180 0x563a792a5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:24.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.480+0000 7fd446000640 1 -- 172.21.15.83:0/4178784607 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563a7929b180 msgr2=0x563a792a5310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:24.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.480+0000 7fd446000640 1 --2- 172.21.15.83:0/4178784607 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563a7929b180 0x563a792a5310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:24.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.480+0000 7fd446000640 1 -- 172.21.15.83:0/4178784607 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a792a5850 msgr2=0x563a792a7c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:24.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.480+0000 7fd446000640 1 --2- 172.21.15.83:0/4178784607 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a792a5850 0x563a792a7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:24.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.480+0000 7fd446000640 1 -- 172.21.15.83:0/4178784607 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a791faf50 con 0x563a7929a860 2024-04-24T09:02:24.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.480+0000 7fd446000640 1 --2- 172.21.15.83:0/4178784607 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a7929a860 0x563a7929ac40 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7fd434006aa0 tx=0x7fd43400be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ac0d0486653b8bd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:02:24.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.480+0000 7fd43f400640 1 --2- 172.21.15.83:0/45085967 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fd42c0a3d20 0x7fd42c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:24.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.481+0000 7fd43f400640 1 -- 172.21.15.83:0/45085967 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9583+0+0 (secure 0 0 0) 0x7fd4400d0100 con 0x7fd44819aa00 2024-04-24T09:02:24.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.481+0000 7fd433e00640 1 -- 172.21.15.83:0/4178784607 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd43400fc90 con 0x563a7929a860 2024-04-24T09:02:24.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.481+0000 7fd433e00640 1 -- 172.21.15.83:0/4178784607 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd434010040 con 0x563a7929a860 2024-04-24T09:02:24.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.481+0000 7fd433e00640 1 -- 172.21.15.83:0/4178784607 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd434013700 con 0x563a7929a860 2024-04-24T09:02:24.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.481+0000 7fd446a00640 1 --2- 172.21.15.83:0/45085967 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fd42c0a3d20 0x7fd42c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:24.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.482+0000 7fd446a00640 1 --2- 172.21.15.83:0/45085967 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fd42c0a3d20 0x7fd42c0a61e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fd438000c00 tx=0x7fd438031c90 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:02:24.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.482+0000 7fd450ba1740 1 -- 172.21.15.83:0/4178784607 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a7929a860 msgr2=0x563a7929ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:24.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.482+0000 7fd450ba1740 1 --2- 172.21.15.83:0/4178784607 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a7929a860 0x563a7929ac40 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7fd434006aa0 tx=0x7fd43400be80 comp rx=0 tx=0).stop 2024-04-24T09:02:24.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.482+0000 7fd450ba1740 1 -- 172.21.15.83:0/4178784607 shutdown_connections 2024-04-24T09:02:24.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.482+0000 7fd450ba1740 1 --2- 172.21.15.83:0/4178784607 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a792a5850 0x563a792a7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:24.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.482+0000 7fd450ba1740 1 --2- 172.21.15.83:0/4178784607 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563a7929b180 0x563a792a5310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:24.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.482+0000 7fd450ba1740 1 --2- 172.21.15.83:0/4178784607 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a7929a860 0x563a7929ac40 unknown :-1 s=CLOSED pgs=1747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:24.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.482+0000 7fd450ba1740 1 -- 172.21.15.83:0/4178784607 >> 172.21.15.83:0/4178784607 conn(0x563a79296b80 msgr2=0x563a79296f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:02:24.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.482+0000 7fd450ba1740 1 -- 172.21.15.83:0/4178784607 shutdown_connections 2024-04-24T09:02:24.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.482+0000 7fd450ba1740 1 -- 172.21.15.83:0/4178784607 wait complete. 2024-04-24T09:02:24.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.482+0000 7fd450ba1740 1 -- start start 2024-04-24T09:02:24.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.483+0000 7fd450ba1740 1 Processor -- start 2024-04-24T09:02:24.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.483+0000 7fd450ba1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a7929a860 0x563a792a2810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:24.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.483+0000 7fd450ba1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a7929b180 0x563a792a2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:24.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.483+0000 7fd450ba1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563a792a5850 0x563a792a3290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:24.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.483+0000 7fd450ba1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x563a792a8640 con 0x563a7929b180 2024-04-24T09:02:24.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.483+0000 7fd450ba1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563a791b9690 con 0x563a7929a860 2024-04-24T09:02:24.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.483+0000 7fd450ba1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x563a792a87c0 con 0x563a792a5850 2024-04-24T09:02:24.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.483+0000 7fd450ba1740 10 client.0 _ll_register_callbacks cb 0x563a791f5650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:02:24.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.483+0000 7fd446000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a7929b180 0x563a792a2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:24.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.483+0000 7fd446000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a7929b180 0x563a792a2d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51136/0 (socket says 172.21.15.83:51136) 2024-04-24T09:02:24.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.484+0000 7fd446000640 1 -- 172.21.15.83:0/94965502 learned_addr learned my addr 172.21.15.83:0/94965502 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:02:24.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.484+0000 7fd447400640 1 --2- 172.21.15.83:0/94965502 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563a792a5850 0x563a792a3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:24.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.484+0000 7fd446a00640 1 --2- 172.21.15.83:0/94965502 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a7929a860 0x563a792a2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:24.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.484+0000 7fd446000640 1 -- 172.21.15.83:0/94965502 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563a792a5850 msgr2=0x563a792a3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:24.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.484+0000 7fd446000640 1 --2- 172.21.15.83:0/94965502 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563a792a5850 0x563a792a3290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:24.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.484+0000 7fd446000640 1 -- 172.21.15.83:0/94965502 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a7929a860 msgr2=0x563a792a2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:24.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.484+0000 7fd446000640 1 --2- 172.21.15.83:0/94965502 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a7929a860 0x563a792a2810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:24.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.484+0000 7fd446000640 1 -- 172.21.15.83:0/94965502 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a791b8920 con 0x563a7929b180 2024-04-24T09:02:24.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.484+0000 7fd446a00640 1 --2- 172.21.15.83:0/94965502 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a7929a860 0x563a792a2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:02:24.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.484+0000 7fd446000640 1 --2- 172.21.15.83:0/94965502 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a7929b180 0x563a792a2d50 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7fd434004700 tx=0x7fd4340146e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2badea7b9f43aa4d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:02:24.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.484+0000 7fd430c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:02:24.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.485+0000 7fd430c00640 1 -- 172.21.15.83:0/94965502 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd43401e610 con 0x563a7929b180 2024-04-24T09:02:24.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.485+0000 7fd430c00640 1 -- 172.21.15.83:0/94965502 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9583+0+0 (secure 0 0 0) 0x7fd43401e7b0 con 0x563a7929b180 2024-04-24T09:02:24.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.485+0000 7fd450ba1740 1 -- 172.21.15.83:0/94965502 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd434010040 con 0x563a7929b180 2024-04-24T09:02:24.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.485+0000 7fd430c00640 1 -- 172.21.15.83:0/94965502 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fd434014b10 con 0x563a7929b180 2024-04-24T09:02:24.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.485+0000 7fd450ba1740 10 client.19404 fetch_fsmap learned FSMap version 41 2024-04-24T09:02:24.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.485+0000 7fd450ba1740 1 -- 172.21.15.83:0/94965502 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x563a78fdc7d0 con 0x563a7929b180 2024-04-24T09:02:24.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.485+0000 7fd430c00640 1 -- 172.21.15.83:0/94965502 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd434010040 con 0x563a7929b180 2024-04-24T09:02:24.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.485+0000 7fd430c00640 1 -- 172.21.15.83:0/94965502 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7fd4340138f0 con 0x563a7929b180 2024-04-24T09:02:24.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.486+0000 7fd450ba1740 10 client.19404 fetch_fsmap finished waiting for FSMap version 41 2024-04-24T09:02:24.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.486+0000 7fd450ba1740 10 client.19404 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:02:24.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.486+0000 7fd450ba1740 20 client.19404 populate_metadata read hostname 'smithi083' 2024-04-24T09:02:24.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.486+0000 7fd450ba1740 1 --2- 172.21.15.83:0/94965502 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x563a792b5af0 0x563a792b7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:24.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.486+0000 7fd450ba1740 4 client.19404 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:02:24.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.486+0000 7fd450ba1740 1 -- 172.21.15.83:0/94965502 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563a7920dc10 con 0x563a792b5af0 2024-04-24T09:02:24.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.486+0000 7fd446a00640 1 --2- 172.21.15.83:0/94965502 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x563a792b5af0 0x563a792b7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:24.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.487+0000 7fd446a00640 1 --2- 172.21.15.83:0/94965502 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x563a792b5af0 0x563a792b7ee0 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c533ce36839bb9f server_cookie=dc7b9a177b9803f1 in_seq=0 out_seq=0 2024-04-24T09:02:24.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.487+0000 7fd430c00640 10 client.19404 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:02:24.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.489+0000 7fd430c00640 1 -- 172.21.15.83:0/94965502 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563a7920dc10 con 0x563a792b5af0 2024-04-24T09:02:24.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.489+0000 7fd430c00640 10 client.19404 handle_command_reply: tid=0 2024-04-24T09:02:24.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.489+0000 7fd450ba1740 1 client.19404 shutdown 2024-04-24T09:02:24.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.489+0000 7fd450ba1740 1 -- 172.21.15.83:0/94965502 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a7929b180 msgr2=0x563a792a2d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:24.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.489+0000 7fd450ba1740 1 --2- 172.21.15.83:0/94965502 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a7929b180 0x563a792a2d50 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7fd434004700 tx=0x7fd4340146e0 comp rx=0 tx=0).stop 2024-04-24T09:02:24.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.489+0000 7fd450ba1740 1 -- 172.21.15.83:0/94965502 shutdown_connections 2024-04-24T09:02:24.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.489+0000 7fd450ba1740 1 --2- 172.21.15.83:0/94965502 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x563a792b5af0 0x563a792b7ee0 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:24.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.489+0000 7fd450ba1740 1 --2- 172.21.15.83:0/94965502 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563a792a5850 0x563a792a3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:24.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.489+0000 7fd450ba1740 1 --2- 172.21.15.83:0/94965502 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a7929b180 0x563a792a2d50 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:24.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.490+0000 7fd450ba1740 1 --2- 172.21.15.83:0/94965502 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a7929a860 0x563a792a2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:24.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.490+0000 7fd430c00640 0 client.19404 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:02:24.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.490+0000 7fd450ba1740 1 -- 172.21.15.83:0/94965502 >> 172.21.15.83:0/94965502 conn(0x563a79296b80 msgr2=0x563a7929c980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:02:24.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.490+0000 7fd450ba1740 1 -- 172.21.15.83:0/94965502 shutdown_connections 2024-04-24T09:02:24.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.490+0000 7fd450ba1740 1 -- 172.21.15.83:0/94965502 wait complete. 2024-04-24T09:02:24.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.490+0000 7fd450ba1740 20 client.19404 trim_cache size 0 max 16384 2024-04-24T09:02:24.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.510+0000 7fd450ba1740 1 Processor -- start 2024-04-24T09:02:24.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.510+0000 7fd450ba1740 1 -- start start 2024-04-24T09:02:24.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.510+0000 7fd450ba1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a7929a860 0x563a792b1500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:24.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.510+0000 7fd450ba1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a7929b180 0x563a792bd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:24.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.510+0000 7fd450ba1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563a792a5850 0x563a792bdc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:24.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.510+0000 7fd450ba1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd4481b1a00 con 0x563a7929a860 2024-04-24T09:02:24.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.510+0000 7fd450ba1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd4481b3660 con 0x563a7929b180 2024-04-24T09:02:24.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.510+0000 7fd450ba1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd4481b20d0 con 0x563a792a5850 2024-04-24T09:02:24.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.511+0000 7fd446a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a7929a860 0x563a792b1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:24.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.511+0000 7fd446a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a7929a860 0x563a792b1500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51140/0 (socket says 172.21.15.83:51140) 2024-04-24T09:02:24.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.511+0000 7fd446a00640 1 -- 172.21.15.83:0/4253129484 learned_addr learned my addr 172.21.15.83:0/4253129484 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:02:24.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.511+0000 7fd446000640 1 --2- 172.21.15.83:0/4253129484 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a7929b180 0x563a792bd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:24.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.511+0000 7fd447400640 1 --2- 172.21.15.83:0/4253129484 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563a792a5850 0x563a792bdc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:24.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.511+0000 7fd446a00640 1 -- 172.21.15.83:0/4253129484 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563a792a5850 msgr2=0x563a792bdc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:24.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.511+0000 7fd446a00640 1 --2- 172.21.15.83:0/4253129484 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563a792a5850 0x563a792bdc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:24.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.511+0000 7fd446a00640 1 -- 172.21.15.83:0/4253129484 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a7929b180 msgr2=0x563a792bd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:24.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.511+0000 7fd446a00640 1 --2- 172.21.15.83:0/4253129484 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a7929b180 0x563a792bd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:24.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.511+0000 7fd446a00640 1 -- 172.21.15.83:0/4253129484 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a792a17e0 con 0x563a7929a860 2024-04-24T09:02:24.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.511+0000 7fd447400640 1 --2- 172.21.15.83:0/4253129484 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563a792a5850 0x563a792bdc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:02:24.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.511+0000 7fd446a00640 1 --2- 172.21.15.83:0/4253129484 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a7929a860 0x563a792b1500 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7fd43802a910 tx=0x7fd43803c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef024477bfb4b857 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:02:24.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.512+0000 7fd433400640 1 -- 172.21.15.83:0/4253129484 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd438035050 con 0x563a7929a860 2024-04-24T09:02:24.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.512+0000 7fd433400640 1 -- 172.21.15.83:0/4253129484 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd438054070 con 0x563a7929a860 2024-04-24T09:02:24.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.512+0000 7fd433400640 1 -- 172.21.15.83:0/4253129484 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd43803b520 con 0x563a7929a860 2024-04-24T09:02:24.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.513+0000 7fd450ba1740 1 -- 172.21.15.83:0/4253129484 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a7929a860 msgr2=0x563a792b1500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:24.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.513+0000 7fd450ba1740 1 --2- 172.21.15.83:0/4253129484 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a7929a860 0x563a792b1500 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7fd43802a910 tx=0x7fd43803c000 comp rx=0 tx=0).stop 2024-04-24T09:02:24.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.513+0000 7fd450ba1740 1 -- 172.21.15.83:0/4253129484 shutdown_connections 2024-04-24T09:02:24.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.513+0000 7fd450ba1740 1 --2- 172.21.15.83:0/4253129484 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563a792a5850 0x563a792bdc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:24.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.513+0000 7fd450ba1740 1 --2- 172.21.15.83:0/4253129484 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a7929b180 0x563a792bd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:24.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.513+0000 7fd450ba1740 1 --2- 172.21.15.83:0/4253129484 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a7929a860 0x563a792b1500 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:24.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.513+0000 7fd450ba1740 1 -- 172.21.15.83:0/4253129484 >> 172.21.15.83:0/4253129484 conn(0x563a79296b80 msgr2=0x563a79296f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:02:24.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.513+0000 7fd450ba1740 1 -- 172.21.15.83:0/4253129484 shutdown_connections 2024-04-24T09:02:24.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.513+0000 7fd450ba1740 1 -- 172.21.15.83:0/4253129484 wait complete. 2024-04-24T09:02:24.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.513+0000 7fd450ba1740 1 -- start start 2024-04-24T09:02:24.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.513+0000 7fd450ba1740 1 Processor -- start 2024-04-24T09:02:24.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.514+0000 7fd450ba1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a7929a860 0x563a7929f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:24.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.514+0000 7fd450ba1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a7929b180 0x563a7929fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:24.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.514+0000 7fd450ba1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563a792a5850 0x563a792a0100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:24.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.514+0000 7fd450ba1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd4481b2f30 con 0x563a7929b180 2024-04-24T09:02:24.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.514+0000 7fd450ba1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd4481b2800 con 0x563a7929a860 2024-04-24T09:02:24.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.514+0000 7fd450ba1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd4481afa00 con 0x563a792a5850 2024-04-24T09:02:24.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.514+0000 7fd450ba1740 10 client.0 _ll_register_callbacks cb 0x7fd43400b8f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:02:24.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.514+0000 7fd446000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a7929b180 0x563a7929fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:24.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.514+0000 7fd446000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a7929b180 0x563a7929fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51150/0 (socket says 172.21.15.83:51150) 2024-04-24T09:02:24.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.514+0000 7fd446000640 1 -- 172.21.15.83:0/2037152512 learned_addr learned my addr 172.21.15.83:0/2037152512 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:02:24.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.514+0000 7fd447400640 1 --2- 172.21.15.83:0/2037152512 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563a792a5850 0x563a792a0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:24.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.515+0000 7fd446a00640 1 --2- 172.21.15.83:0/2037152512 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a7929a860 0x563a7929f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:24.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.515+0000 7fd446000640 1 -- 172.21.15.83:0/2037152512 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563a792a5850 msgr2=0x563a792a0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:24.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.515+0000 7fd446000640 1 --2- 172.21.15.83:0/2037152512 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563a792a5850 0x563a792a0100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:24.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.515+0000 7fd446000640 1 -- 172.21.15.83:0/2037152512 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a7929a860 msgr2=0x563a7929f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:24.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.515+0000 7fd446000640 1 --2- 172.21.15.83:0/2037152512 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a7929a860 0x563a7929f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:24.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.515+0000 7fd446000640 1 -- 172.21.15.83:0/2037152512 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a7929afb0 con 0x563a7929b180 2024-04-24T09:02:24.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.515+0000 7fd446a00640 1 --2- 172.21.15.83:0/2037152512 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a7929a860 0x563a7929f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:02:24.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.515+0000 7fd446000640 1 --2- 172.21.15.83:0/2037152512 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a7929b180 0x563a7929fbc0 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7fd434006e10 tx=0x7fd434005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2008985da87487af server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:02:24.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.515+0000 7fd430c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:02:24.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.515+0000 7fd430c00640 1 -- 172.21.15.83:0/2037152512 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4340135d0 con 0x563a7929b180 2024-04-24T09:02:24.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.515+0000 7fd430c00640 1 -- 172.21.15.83:0/2037152512 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9583+0+0 (secure 0 0 0) 0x7fd434013770 con 0x563a7929b180 2024-04-24T09:02:24.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.515+0000 7fd450ba1740 1 -- 172.21.15.83:0/2037152512 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd438054070 con 0x563a7929b180 2024-04-24T09:02:24.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.516+0000 7fd430c00640 1 -- 172.21.15.83:0/2037152512 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fd43403c050 con 0x563a7929b180 2024-04-24T09:02:24.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.516+0000 7fd450ba1740 10 client.19410 fetch_fsmap learned FSMap version 41 2024-04-24T09:02:24.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.516+0000 7fd450ba1740 1 -- 172.21.15.83:0/2037152512 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x563a78f123f0 con 0x563a7929b180 2024-04-24T09:02:24.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.516+0000 7fd430c00640 1 -- 172.21.15.83:0/2037152512 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd438054070 con 0x563a7929b180 2024-04-24T09:02:24.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.516+0000 7fd430c00640 1 -- 172.21.15.83:0/2037152512 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7fd434026bf0 con 0x563a7929b180 2024-04-24T09:02:24.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.516+0000 7fd450ba1740 10 client.19410 fetch_fsmap finished waiting for FSMap version 41 2024-04-24T09:02:24.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.516+0000 7fd450ba1740 10 client.19410 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:02:24.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.516+0000 7fd450ba1740 20 client.19410 populate_metadata read hostname 'smithi083' 2024-04-24T09:02:24.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.516+0000 7fd450ba1740 1 --2- 172.21.15.83:0/2037152512 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x563a7931c730 0x563a7931eb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:24.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.516+0000 7fd450ba1740 4 client.19410 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T09:02:24.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.516+0000 7fd450ba1740 1 -- 172.21.15.83:0/2037152512 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563a79207a80 con 0x563a7931c730 2024-04-24T09:02:24.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.517+0000 7fd446a00640 1 --2- 172.21.15.83:0/2037152512 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x563a7931c730 0x563a7931eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:24.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.517+0000 7fd446a00640 1 --2- 172.21.15.83:0/2037152512 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x563a7931c730 0x563a7931eb50 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb969b78cb1aa996 server_cookie=31495e2fca687fbb in_seq=0 out_seq=0 2024-04-24T09:02:24.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.517+0000 7fd430c00640 10 client.19410 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:02:24.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.520+0000 7fd430c00640 1 -- 172.21.15.83:0/2037152512 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x563a79207a80 con 0x563a7931c730 2024-04-24T09:02:24.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.520+0000 7fd430c00640 10 client.19410 handle_command_reply: tid=0 2024-04-24T09:02:24.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.520+0000 7fd450ba1740 1 client.19410 shutdown 2024-04-24T09:02:24.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.521+0000 7fd450ba1740 1 -- 172.21.15.83:0/2037152512 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a7929b180 msgr2=0x563a7929fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:24.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.521+0000 7fd450ba1740 1 --2- 172.21.15.83:0/2037152512 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a7929b180 0x563a7929fbc0 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7fd434006e10 tx=0x7fd434005f90 comp rx=0 tx=0).stop 2024-04-24T09:02:24.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.521+0000 7fd450ba1740 1 -- 172.21.15.83:0/2037152512 shutdown_connections 2024-04-24T09:02:24.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.521+0000 7fd450ba1740 1 --2- 172.21.15.83:0/2037152512 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x563a7931c730 0x563a7931eb50 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:24.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.521+0000 7fd450ba1740 1 --2- 172.21.15.83:0/2037152512 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563a792a5850 0x563a792a0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:24.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.521+0000 7fd450ba1740 1 --2- 172.21.15.83:0/2037152512 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a7929b180 0x563a7929fbc0 unknown :-1 s=CLOSED pgs=1750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:24.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.521+0000 7fd450ba1740 1 --2- 172.21.15.83:0/2037152512 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a7929a860 0x563a7929f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:24.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.521+0000 7fd430c00640 0 client.19410 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:02:24.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.521+0000 7fd450ba1740 1 -- 172.21.15.83:0/2037152512 >> 172.21.15.83:0/2037152512 conn(0x563a79296b80 msgr2=0x563a79295390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:02:24.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.521+0000 7fd450ba1740 1 -- 172.21.15.83:0/2037152512 shutdown_connections 2024-04-24T09:02:24.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.521+0000 7fd450ba1740 1 -- 172.21.15.83:0/2037152512 wait complete. 2024-04-24T09:02:24.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.521+0000 7fd450ba1740 20 client.19410 trim_cache size 0 max 16384 2024-04-24T09:02:24.522 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T09:02:24.522 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-04-24T09:02:24.522 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-04-24T09:02:24.522 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T09:02:24.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.522+0000 7fd44dc00640 1 -- 172.21.15.83:0/45085967 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fd42c0a3d20 msgr2=0x7fd42c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:24.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.522+0000 7fd44dc00640 1 --2- 172.21.15.83:0/45085967 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fd42c0a3d20 0x7fd42c0a61e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fd438000c00 tx=0x7fd438031c90 comp rx=0 tx=0).stop 2024-04-24T09:02:24.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.522+0000 7fd44dc00640 1 -- 172.21.15.83:0/45085967 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd44819aa00 msgr2=0x7fd44819aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:24.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.522+0000 7fd44dc00640 1 --2- 172.21.15.83:0/45085967 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd44819aa00 0x7fd44819aeb0 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7fd44000b970 tx=0x7fd44000be30 comp rx=0 tx=0).stop 2024-04-24T09:02:24.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.522+0000 7fd44dc00640 1 -- 172.21.15.83:0/45085967 shutdown_connections 2024-04-24T09:02:24.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.522+0000 7fd44dc00640 1 --2- 172.21.15.83:0/45085967 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fd42c0a3d20 0x7fd42c0a61e0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:24.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.522+0000 7fd44dc00640 1 --2- 172.21.15.83:0/45085967 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd44819aa00 0x7fd44819aeb0 unknown :-1 s=CLOSED pgs=1746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:24.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.522+0000 7fd44dc00640 1 --2- 172.21.15.83:0/45085967 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd448109580 0x7fd4481a0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:24.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.522+0000 7fd44dc00640 1 --2- 172.21.15.83:0/45085967 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd4481044b0 0x7fd4481a0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:24.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.522+0000 7fd44dc00640 1 -- 172.21.15.83:0/45085967 >> 172.21.15.83:0/45085967 conn(0x7fd4480ffd90 msgr2=0x7fd448101180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:02:24.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.522+0000 7fd44dc00640 1 -- 172.21.15.83:0/45085967 shutdown_connections 2024-04-24T09:02:24.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.522+0000 7fd44dc00640 1 -- 172.21.15.83:0/45085967 wait complete. 2024-04-24T09:02:24.540 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T09:02:24.540 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T09:02:24.612 INFO:tasks.workunit.client.0.smithi083.stdout: 1 62937 0.61 MB/sec execute 418 sec latency 56.146 ms 2024-04-24T09:02:24.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.745+0000 7f83b1800640 1 -- 172.21.15.83:0/3620968436 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83ac106d40 msgr2=0x7f83ac069190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:24.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.745+0000 7f83b1800640 1 --2- 172.21.15.83:0/3620968436 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83ac106d40 0x7f83ac069190 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f839c009900 tx=0x7f839c02a3f0 comp rx=0 tx=0).stop 2024-04-24T09:02:24.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.745+0000 7f83b1800640 1 -- 172.21.15.83:0/3620968436 shutdown_connections 2024-04-24T09:02:24.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.745+0000 7f83b1800640 1 --2- 172.21.15.83:0/3620968436 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83ac104890 0x7f83ac104d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:24.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.745+0000 7f83b1800640 1 --2- 172.21.15.83:0/3620968436 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f83ac1071b0 0x7f83ac0696d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:24.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.745+0000 7f83b1800640 1 --2- 172.21.15.83:0/3620968436 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83ac106d40 0x7f83ac069190 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:24.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.745+0000 7f83b1800640 1 -- 172.21.15.83:0/3620968436 >> 172.21.15.83:0/3620968436 conn(0x7f83ac076840 msgr2=0x7f83ac076c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:02:24.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.746+0000 7f83b1800640 1 -- 172.21.15.83:0/3620968436 shutdown_connections 2024-04-24T09:02:24.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.746+0000 7f83b1800640 1 -- 172.21.15.83:0/3620968436 wait complete. 2024-04-24T09:02:24.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.747+0000 7f83b1800640 1 Processor -- start 2024-04-24T09:02:24.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.747+0000 7f83b1800640 1 -- start start 2024-04-24T09:02:24.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.747+0000 7f83b1800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83ac104890 0x7f83ac1a47e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:24.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.747+0000 7f83b1800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f83ac106d40 0x7f83ac1a4d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:24.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.747+0000 7f83b1800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83ac1071b0 0x7f83ac19e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:24.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.747+0000 7f83b1800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f83ac118630 con 0x7f83ac104890 2024-04-24T09:02:24.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.747+0000 7f83b1800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f83ac1184b0 con 0x7f83ac1071b0 2024-04-24T09:02:24.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.747+0000 7f83b1800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f83ac1187b0 con 0x7f83ac106d40 2024-04-24T09:02:24.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.748+0000 7f83abe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83ac104890 0x7f83ac1a47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:24.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.748+0000 7f83abe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83ac104890 0x7f83ac1a47e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51162/0 (socket says 172.21.15.83:51162) 2024-04-24T09:02:24.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.748+0000 7f83abe00640 1 -- 172.21.15.83:0/3071260272 learned_addr learned my addr 172.21.15.83:0/3071260272 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:02:24.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.748+0000 7f83a3400640 1 --2- 172.21.15.83:0/3071260272 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f83ac106d40 0x7f83ac1a4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:24.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.748+0000 7f83b0e00640 1 --2- 172.21.15.83:0/3071260272 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83ac1071b0 0x7f83ac19e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:24.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.748+0000 7f83abe00640 1 -- 172.21.15.83:0/3071260272 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f83ac106d40 msgr2=0x7f83ac1a4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:24.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.748+0000 7f83abe00640 1 --2- 172.21.15.83:0/3071260272 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f83ac106d40 0x7f83ac1a4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:24.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.748+0000 7f83abe00640 1 -- 172.21.15.83:0/3071260272 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83ac1071b0 msgr2=0x7f83ac19e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:24.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.748+0000 7f83abe00640 1 --2- 172.21.15.83:0/3071260272 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83ac1071b0 0x7f83ac19e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:24.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.748+0000 7f83abe00640 1 -- 172.21.15.83:0/3071260272 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83ac19f110 con 0x7f83ac104890 2024-04-24T09:02:24.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.748+0000 7f83abe00640 1 --2- 172.21.15.83:0/3071260272 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83ac104890 0x7f83ac1a47e0 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7f839c005ab0 tx=0x7f839c003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:02:24.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.749+0000 7f83a9600640 1 -- 172.21.15.83:0/3071260272 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f839c03f070 con 0x7f83ac104890 2024-04-24T09:02:24.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.749+0000 7f83a9600640 1 -- 172.21.15.83:0/3071260272 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f839c031d60 con 0x7f83ac104890 2024-04-24T09:02:24.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.749+0000 7f83b1800640 1 -- 172.21.15.83:0/3071260272 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83ac19f400 con 0x7f83ac104890 2024-04-24T09:02:24.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.749+0000 7f83a9600640 1 -- 172.21.15.83:0/3071260272 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f839c035030 con 0x7f83ac104890 2024-04-24T09:02:24.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.749+0000 7f83b1800640 1 -- 172.21.15.83:0/3071260272 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83ac1ab6a0 con 0x7f83ac104890 2024-04-24T09:02:24.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.750+0000 7f83b1800640 1 -- 172.21.15.83:0/3071260272 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83ac19fec0 con 0x7f83ac104890 2024-04-24T09:02:24.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.753+0000 7f83a9600640 1 -- 172.21.15.83:0/3071260272 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f839c042070 con 0x7f83ac104890 2024-04-24T09:02:24.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.754+0000 7f83a9600640 1 --2- 172.21.15.83:0/3071260272 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f83900a3d20 0x7f83900a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:24.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.754+0000 7f83a9600640 1 -- 172.21.15.83:0/3071260272 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9583+0+0 (secure 0 0 0) 0x7f839c0f1c10 con 0x7f83ac104890 2024-04-24T09:02:24.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.755+0000 7f83a3400640 1 --2- 172.21.15.83:0/3071260272 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f83900a3d20 0x7f83900a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:24.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.756+0000 7f83a3400640 1 --2- 172.21.15.83:0/3071260272 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f83900a3d20 0x7f83900a61e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f8394005fd0 tx=0x7f839400b510 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:02:24.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:24.757+0000 7f83a9600640 1 -- 172.21.15.83:0/3071260272 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f839c0bde30 con 0x7f83ac104890 2024-04-24T09:02:24.820 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.821+0000 7fc770200640 1 -- 172.21.15.83:0/1698678299 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc768074430 msgr2=0x7fc768074810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:24.820 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.821+0000 7fc770200640 1 --2- 172.21.15.83:0/1698678299 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc768074430 0x7fc768074810 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7fc75c009900 tx=0x7fc75c02a480 comp rx=0 tx=0).stop 2024-04-24T09:02:24.820 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.821+0000 7fc770200640 1 -- 172.21.15.83:0/1698678299 shutdown_connections 2024-04-24T09:02:24.820 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.821+0000 7fc770200640 1 --2- 172.21.15.83:0/1698678299 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc76810e820 0x7fc768110c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:24.820 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.821+0000 7fc770200640 1 --2- 172.21.15.83:0/1698678299 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc768074d50 0x7fc76810e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:24.821 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.821+0000 7fc770200640 1 --2- 172.21.15.83:0/1698678299 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc768074430 0x7fc768074810 unknown :-1 s=CLOSED pgs=1753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:24.821 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.821+0000 7fc770200640 1 -- 172.21.15.83:0/1698678299 >> 172.21.15.83:0/1698678299 conn(0x7fc7680fdab0 msgr2=0x7fc7680ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:02:24.821 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.821+0000 7fc770200640 1 -- 172.21.15.83:0/1698678299 shutdown_connections 2024-04-24T09:02:24.821 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.821+0000 7fc770200640 1 -- 172.21.15.83:0/1698678299 wait complete. 2024-04-24T09:02:24.821 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.822+0000 7fc770200640 1 Processor -- start 2024-04-24T09:02:24.821 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.823+0000 7fc770200640 1 -- start start 2024-04-24T09:02:24.824 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.823+0000 7fc770200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc768074430 0x7fc7681a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:24.824 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.823+0000 7fc770200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc768074d50 0x7fc7681a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:24.824 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.823+0000 7fc770200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc76810e820 0x7fc76819a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:24.824 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.823+0000 7fc770200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc7681144b0 con 0x7fc76810e820 2024-04-24T09:02:24.824 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.823+0000 7fc770200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc768114330 con 0x7fc768074d50 2024-04-24T09:02:24.824 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.823+0000 7fc770200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc768114630 con 0x7fc768074430 2024-04-24T09:02:24.824 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.824+0000 7fc76e200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc76810e820 0x7fc76819a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:24.824 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.824+0000 7fc76e200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc76810e820 0x7fc76819a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51186/0 (socket says 172.21.15.83:51186) 2024-04-24T09:02:24.824 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.824+0000 7fc76e200640 1 -- 172.21.15.83:0/1955059128 learned_addr learned my addr 172.21.15.83:0/1955059128 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:02:24.824 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.824+0000 7fc76d800640 1 --2- 172.21.15.83:0/1955059128 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc768074430 0x7fc7681a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:24.825 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.824+0000 7fc76ce00640 1 --2- 172.21.15.83:0/1955059128 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc768074d50 0x7fc7681a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:24.825 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.824+0000 7fc76e200640 1 -- 172.21.15.83:0/1955059128 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc768074430 msgr2=0x7fc7681a0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:24.825 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.824+0000 7fc76e200640 1 --2- 172.21.15.83:0/1955059128 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc768074430 0x7fc7681a0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:24.825 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.824+0000 7fc76e200640 1 -- 172.21.15.83:0/1955059128 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc768074d50 msgr2=0x7fc7681a0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:24.825 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.824+0000 7fc76e200640 1 --2- 172.21.15.83:0/1955059128 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc768074d50 0x7fc7681a0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:24.825 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.824+0000 7fc76e200640 1 -- 172.21.15.83:0/1955059128 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc76819afc0 con 0x7fc76810e820 2024-04-24T09:02:24.825 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.825+0000 7fc76e200640 1 --2- 172.21.15.83:0/1955059128 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc76810e820 0x7fc76819a7f0 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7fc76000eaa0 tx=0x7fc76000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:02:24.825 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.825+0000 7fc766000640 1 -- 172.21.15.83:0/1955059128 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc76000cdb0 con 0x7fc76810e820 2024-04-24T09:02:24.825 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.825+0000 7fc766000640 1 -- 172.21.15.83:0/1955059128 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc760005020 con 0x7fc76810e820 2024-04-24T09:02:24.825 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.825+0000 7fc770200640 1 -- 172.21.15.83:0/1955059128 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc76819b2b0 con 0x7fc76810e820 2024-04-24T09:02:24.825 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.825+0000 7fc766000640 1 -- 172.21.15.83:0/1955059128 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7600116d0 con 0x7fc76810e820 2024-04-24T09:02:24.825 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.825+0000 7fc770200640 1 -- 172.21.15.83:0/1955059128 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7681a7480 con 0x7fc76810e820 2024-04-24T09:02:24.827 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.828+0000 7fc7731c6740 1 Processor -- start 2024-04-24T09:02:24.828 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.829+0000 7fc766000640 1 -- 172.21.15.83:0/1955059128 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc7600051d0 con 0x7fc76810e820 2024-04-24T09:02:24.828 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.829+0000 7fc7731c6740 1 -- start start 2024-04-24T09:02:24.828 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.830+0000 7fc7731c6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55edf5b189c0 0x55edf5b18da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:24.829 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.830+0000 7fc7731c6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55edf5b192e0 0x55edf5b23660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:24.829 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.830+0000 7fc7731c6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55edf5b23ba0 0x55edf5b25f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:24.829 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.830+0000 7fc7731c6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55edf5a76900 con 0x55edf5b23ba0 2024-04-24T09:02:24.829 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.830+0000 7fc7731c6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55edf5a1cf20 con 0x55edf5b192e0 2024-04-24T09:02:24.829 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.830+0000 7fc7731c6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55edf5a728c0 con 0x55edf5b189c0 2024-04-24T09:02:24.830 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.830+0000 7fc76ce00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55edf5b23ba0 0x55edf5b25f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:24.830 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.830+0000 7fc76ce00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55edf5b23ba0 0x55edf5b25f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51202/0 (socket says 172.21.15.83:51202) 2024-04-24T09:02:24.830 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.830+0000 7fc76ce00640 1 -- 172.21.15.83:0/4184030573 learned_addr learned my addr 172.21.15.83:0/4184030573 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:02:24.830 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.830+0000 7fc76ce00640 1 --2- 172.21.15.83:0/4184030573 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55edf5b189c0 0x55edf5b18da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:24.830 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.830+0000 7fc76d800640 1 --2- 172.21.15.83:0/4184030573 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55edf5b192e0 0x55edf5b23660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:24.830 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.831+0000 7fc76ce00640 1 -- 172.21.15.83:0/4184030573 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55edf5b189c0 msgr2=0x55edf5b18da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:24.830 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.831+0000 7fc76ce00640 1 --2- 172.21.15.83:0/4184030573 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55edf5b189c0 0x55edf5b18da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:24.830 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.831+0000 7fc76ce00640 1 -- 172.21.15.83:0/4184030573 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55edf5b192e0 msgr2=0x55edf5b23660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:24.830 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.831+0000 7fc76ce00640 1 --2- 172.21.15.83:0/4184030573 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55edf5b192e0 0x55edf5b23660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:24.831 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.831+0000 7fc76ce00640 1 -- 172.21.15.83:0/4184030573 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55edf5a41e10 con 0x55edf5b23ba0 2024-04-24T09:02:24.831 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.831+0000 7fc76d800640 1 --2- 172.21.15.83:0/4184030573 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55edf5b192e0 0x55edf5b23660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:02:24.831 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.831+0000 7fc76ce00640 1 --2- 172.21.15.83:0/4184030573 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55edf5b23ba0 0x55edf5b25f90 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7fc75400bfc0 tx=0x7fc75400b430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1331ddee112826a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:02:24.832 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.832+0000 7fc766000640 1 --2- 172.21.15.83:0/1955059128 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fc7500a3dc0 0x7fc7500a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:24.832 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.832+0000 7fc75aa00640 1 -- 172.21.15.83:0/4184030573 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc754019030 con 0x55edf5b23ba0 2024-04-24T09:02:24.832 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.832+0000 7fc75aa00640 1 -- 172.21.15.83:0/4184030573 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc754010900 con 0x55edf5b23ba0 2024-04-24T09:02:24.832 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.832+0000 7fc75aa00640 1 -- 172.21.15.83:0/4184030573 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc754011760 con 0x55edf5b23ba0 2024-04-24T09:02:24.832 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.832+0000 7fc766000640 1 -- 172.21.15.83:0/1955059128 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9583+0+0 (secure 0 0 0) 0x7fc7600d0e00 con 0x7fc76810e820 2024-04-24T09:02:24.832 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.832+0000 7fc76d800640 1 --2- 172.21.15.83:0/1955059128 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fc7500a3dc0 0x7fc7500a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:24.832 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.833+0000 7fc7731c6740 1 -- 172.21.15.83:0/4184030573 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55edf5b23ba0 msgr2=0x55edf5b25f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:24.833 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.833+0000 7fc7731c6740 1 --2- 172.21.15.83:0/4184030573 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55edf5b23ba0 0x55edf5b25f90 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7fc75400bfc0 tx=0x7fc75400b430 comp rx=0 tx=0).stop 2024-04-24T09:02:24.833 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.833+0000 7fc76d800640 1 --2- 172.21.15.83:0/1955059128 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fc7500a3dc0 0x7fc7500a6280 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fc75c005ab0 tx=0x7fc75c005a20 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:02:24.833 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.833+0000 7fc7731c6740 1 -- 172.21.15.83:0/4184030573 shutdown_connections 2024-04-24T09:02:24.833 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.833+0000 7fc7731c6740 1 --2- 172.21.15.83:0/4184030573 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55edf5b23ba0 0x55edf5b25f90 unknown :-1 s=CLOSED pgs=1755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:24.833 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.833+0000 7fc7731c6740 1 --2- 172.21.15.83:0/4184030573 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55edf5b192e0 0x55edf5b23660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:24.833 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.833+0000 7fc7731c6740 1 --2- 172.21.15.83:0/4184030573 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55edf5b189c0 0x55edf5b18da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:24.833 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.833+0000 7fc7731c6740 1 -- 172.21.15.83:0/4184030573 >> 172.21.15.83:0/4184030573 conn(0x55edf5b14ed0 msgr2=0x55edf5b152b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:02:24.833 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.833+0000 7fc7731c6740 1 -- 172.21.15.83:0/4184030573 shutdown_connections 2024-04-24T09:02:24.833 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.833+0000 7fc7731c6740 1 -- 172.21.15.83:0/4184030573 wait complete. 2024-04-24T09:02:24.835 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.834+0000 7fc7731c6740 1 -- start start 2024-04-24T09:02:24.836 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.834+0000 7fc7731c6740 1 Processor -- start 2024-04-24T09:02:24.836 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.835+0000 7fc7731c6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55edf5b189c0 0x55edf5b20b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:24.836 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.835+0000 7fc7731c6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55edf5b192e0 0x55edf5b210b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:24.836 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.835+0000 7fc7731c6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55edf5b23ba0 0x55edf5b215f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:24.836 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.835+0000 7fc7731c6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55edf5b267d0 con 0x55edf5b189c0 2024-04-24T09:02:24.836 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.835+0000 7fc7731c6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55edf5a302c0 con 0x55edf5b192e0 2024-04-24T09:02:24.836 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.835+0000 7fc7731c6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55edf5b26950 con 0x55edf5b23ba0 2024-04-24T09:02:24.836 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.835+0000 7fc7731c6740 10 client.0 _ll_register_callbacks cb 0x55edf5788a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:02:24.836 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.835+0000 7fc76d800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55edf5b189c0 0x55edf5b20b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:24.836 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.835+0000 7fc76d800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55edf5b189c0 0x55edf5b20b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51212/0 (socket says 172.21.15.83:51212) 2024-04-24T09:02:24.836 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.835+0000 7fc76d800640 1 -- 172.21.15.83:0/2449617238 learned_addr learned my addr 172.21.15.83:0/2449617238 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:02:24.836 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.835+0000 7fc76ce00640 1 --2- 172.21.15.83:0/2449617238 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55edf5b192e0 0x55edf5b210b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:24.836 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.835+0000 7fc76e200640 1 --2- 172.21.15.83:0/2449617238 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55edf5b23ba0 0x55edf5b215f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:24.836 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.836+0000 7fc76d800640 1 -- 172.21.15.83:0/2449617238 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55edf5b23ba0 msgr2=0x55edf5b215f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:24.837 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.836+0000 7fc76d800640 1 --2- 172.21.15.83:0/2449617238 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55edf5b23ba0 0x55edf5b215f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:24.837 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.836+0000 7fc76d800640 1 -- 172.21.15.83:0/2449617238 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55edf5b192e0 msgr2=0x55edf5b210b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:24.837 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.836+0000 7fc76d800640 1 --2- 172.21.15.83:0/2449617238 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55edf5b192e0 0x55edf5b210b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:24.837 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.836+0000 7fc76d800640 1 -- 172.21.15.83:0/2449617238 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55edf5a40fd0 con 0x55edf5b189c0 2024-04-24T09:02:24.837 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.836+0000 7fc76ce00640 1 --2- 172.21.15.83:0/2449617238 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55edf5b192e0 0x55edf5b210b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:02:24.837 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.836+0000 7fc76d800640 1 --2- 172.21.15.83:0/2449617238 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55edf5b189c0 0x55edf5b20b70 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7fc75c006010 tx=0x7fc75c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae26772506191f85 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:02:24.837 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.836+0000 7fc74f400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:02:24.837 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.836+0000 7fc74f400640 1 -- 172.21.15.83:0/2449617238 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc75c04d070 con 0x55edf5b189c0 2024-04-24T09:02:24.837 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.836+0000 7fc74f400640 1 -- 172.21.15.83:0/2449617238 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9583+0+0 (secure 0 0 0) 0x7fc75c004d00 con 0x55edf5b189c0 2024-04-24T09:02:24.837 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.836+0000 7fc7731c6740 1 -- 172.21.15.83:0/2449617238 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc754010900 con 0x55edf5b189c0 2024-04-24T09:02:24.837 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.837+0000 7fc74f400640 1 -- 172.21.15.83:0/2449617238 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fc75c059050 con 0x55edf5b189c0 2024-04-24T09:02:24.837 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.837+0000 7fc7731c6740 10 client.19428 fetch_fsmap learned FSMap version 41 2024-04-24T09:02:24.837 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.837+0000 7fc7731c6740 1 -- 172.21.15.83:0/2449617238 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55edf5789210 con 0x55edf5b189c0 2024-04-24T09:02:24.839 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.837+0000 7fc74f400640 1 -- 172.21.15.83:0/2449617238 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc754010900 con 0x55edf5b189c0 2024-04-24T09:02:24.839 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.838+0000 7fc74f400640 1 -- 172.21.15.83:0/2449617238 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7fc75c0325b0 con 0x55edf5b189c0 2024-04-24T09:02:24.839 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.838+0000 7fc7731c6740 10 client.19428 fetch_fsmap finished waiting for FSMap version 41 2024-04-24T09:02:24.839 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.838+0000 7fc7731c6740 10 client.19428 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:02:24.839 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.838+0000 7fc7731c6740 20 client.19428 populate_metadata read hostname 'smithi083' 2024-04-24T09:02:24.839 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.838+0000 7fc7731c6740 1 --2- 172.21.15.83:0/2449617238 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55edf5b33dc0 0x55edf5b361b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:24.839 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.838+0000 7fc7731c6740 4 client.19428 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:02:24.839 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.838+0000 7fc7731c6740 1 -- 172.21.15.83:0/2449617238 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55edf5a84320 con 0x55edf5b33dc0 2024-04-24T09:02:24.839 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.839+0000 7fc76ce00640 1 --2- 172.21.15.83:0/2449617238 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55edf5b33dc0 0x55edf5b361b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:24.839 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.840+0000 7fc76ce00640 1 --2- 172.21.15.83:0/2449617238 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55edf5b33dc0 0x55edf5b361b0 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf92762d72d2d70f server_cookie=6cbe839d6c6d5bd1 in_seq=0 out_seq=0 2024-04-24T09:02:24.839 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.840+0000 7fc74f400640 10 client.19428 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:02:24.842 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.843+0000 7fc74f400640 1 -- 172.21.15.83:0/2449617238 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55edf5a84320 con 0x55edf5b33dc0 2024-04-24T09:02:24.842 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.843+0000 7fc74f400640 10 client.19428 handle_command_reply: tid=0 2024-04-24T09:02:24.842 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.843+0000 7fc7731c6740 1 client.19428 shutdown 2024-04-24T09:02:24.842 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.844+0000 7fc7731c6740 1 -- 172.21.15.83:0/2449617238 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55edf5b189c0 msgr2=0x55edf5b20b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:24.843 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.844+0000 7fc7731c6740 1 --2- 172.21.15.83:0/2449617238 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55edf5b189c0 0x55edf5b20b70 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7fc75c006010 tx=0x7fc75c009260 comp rx=0 tx=0).stop 2024-04-24T09:02:24.843 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.844+0000 7fc7731c6740 1 -- 172.21.15.83:0/2449617238 shutdown_connections 2024-04-24T09:02:24.843 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.844+0000 7fc7731c6740 1 --2- 172.21.15.83:0/2449617238 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55edf5b33dc0 0x55edf5b361b0 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:24.843 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.844+0000 7fc7731c6740 1 --2- 172.21.15.83:0/2449617238 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55edf5b23ba0 0x55edf5b215f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:24.843 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.844+0000 7fc7731c6740 1 --2- 172.21.15.83:0/2449617238 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55edf5b192e0 0x55edf5b210b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:24.844 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.844+0000 7fc7731c6740 1 --2- 172.21.15.83:0/2449617238 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55edf5b189c0 0x55edf5b20b70 unknown :-1 s=CLOSED pgs=1756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:24.844 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.844+0000 7fc7731c6740 1 -- 172.21.15.83:0/2449617238 >> 172.21.15.83:0/2449617238 conn(0x55edf5b14ed0 msgr2=0x55edf5b0b840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:02:24.844 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.844+0000 7fc74f400640 0 client.19428 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:02:24.844 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.844+0000 7fc7731c6740 1 -- 172.21.15.83:0/2449617238 shutdown_connections 2024-04-24T09:02:24.844 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.844+0000 7fc7731c6740 1 -- 172.21.15.83:0/2449617238 wait complete. 2024-04-24T09:02:24.844 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.844+0000 7fc7731c6740 20 client.19428 trim_cache size 0 max 16384 2024-04-24T09:02:24.862 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.861+0000 7fc7731c6740 1 Processor -- start 2024-04-24T09:02:24.862 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.861+0000 7fc7731c6740 1 -- start start 2024-04-24T09:02:24.862 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.862+0000 7fc7731c6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55edf5b189c0 0x55edf5b2f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:24.862 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.862+0000 7fc7731c6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55edf5b192e0 0x55edf5b1b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:24.862 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.862+0000 7fc7731c6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55edf5b23ba0 0x55edf5b3ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:24.862 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.862+0000 7fc7731c6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc7681ad3e0 con 0x55edf5b23ba0 2024-04-24T09:02:24.863 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.862+0000 7fc7731c6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc7681af040 con 0x55edf5b189c0 2024-04-24T09:02:24.863 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.862+0000 7fc7731c6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc7681adab0 con 0x55edf5b192e0 2024-04-24T09:02:24.863 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.862+0000 7fc76e200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55edf5b23ba0 0x55edf5b3ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:24.863 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.862+0000 7fc76e200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55edf5b23ba0 0x55edf5b3ba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51222/0 (socket says 172.21.15.83:51222) 2024-04-24T09:02:24.863 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.862+0000 7fc76e200640 1 -- 172.21.15.83:0/180947278 learned_addr learned my addr 172.21.15.83:0/180947278 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:02:24.863 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.862+0000 7fc76d800640 1 --2- 172.21.15.83:0/180947278 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55edf5b189c0 0x55edf5b2f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:24.863 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.863+0000 7fc76ce00640 1 --2- 172.21.15.83:0/180947278 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55edf5b192e0 0x55edf5b1b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:24.863 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.863+0000 7fc76e200640 1 -- 172.21.15.83:0/180947278 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55edf5b192e0 msgr2=0x55edf5b1b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:24.863 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.863+0000 7fc76e200640 1 --2- 172.21.15.83:0/180947278 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55edf5b192e0 0x55edf5b1b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:24.863 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.863+0000 7fc76e200640 1 -- 172.21.15.83:0/180947278 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55edf5b189c0 msgr2=0x55edf5b2f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:24.864 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.863+0000 7fc76e200640 1 --2- 172.21.15.83:0/180947278 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55edf5b189c0 0x55edf5b2f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:24.864 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.863+0000 7fc76e200640 1 -- 172.21.15.83:0/180947278 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55edf5b19110 con 0x55edf5b23ba0 2024-04-24T09:02:24.864 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.863+0000 7fc76e200640 1 --2- 172.21.15.83:0/180947278 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55edf5b23ba0 0x55edf5b3ba90 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7fc7600092f0 tx=0x7fc76009c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f796cbdc3bec9357 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:02:24.868 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.863+0000 7fc75a000640 1 -- 172.21.15.83:0/180947278 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc76001f040 con 0x55edf5b23ba0 2024-04-24T09:02:24.868 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.864+0000 7fc75a000640 1 -- 172.21.15.83:0/180947278 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc7600a5590 con 0x55edf5b23ba0 2024-04-24T09:02:24.868 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.864+0000 7fc75a000640 1 -- 172.21.15.83:0/180947278 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc760085e60 con 0x55edf5b23ba0 2024-04-24T09:02:24.868 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.864+0000 7fc7731c6740 1 -- 172.21.15.83:0/180947278 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55edf5b23ba0 msgr2=0x55edf5b3ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:24.868 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.864+0000 7fc7731c6740 1 --2- 172.21.15.83:0/180947278 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55edf5b23ba0 0x55edf5b3ba90 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7fc7600092f0 tx=0x7fc76009c920 comp rx=0 tx=0).stop 2024-04-24T09:02:24.868 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.864+0000 7fc7731c6740 1 -- 172.21.15.83:0/180947278 shutdown_connections 2024-04-24T09:02:24.868 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.864+0000 7fc7731c6740 1 --2- 172.21.15.83:0/180947278 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55edf5b23ba0 0x55edf5b3ba90 unknown :-1 s=CLOSED pgs=1757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:24.869 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.864+0000 7fc7731c6740 1 --2- 172.21.15.83:0/180947278 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55edf5b192e0 0x55edf5b1b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:24.869 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.864+0000 7fc7731c6740 1 --2- 172.21.15.83:0/180947278 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55edf5b189c0 0x55edf5b2f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:24.869 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.864+0000 7fc7731c6740 1 -- 172.21.15.83:0/180947278 >> 172.21.15.83:0/180947278 conn(0x55edf5b14ed0 msgr2=0x55edf5b0b840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:02:24.869 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.865+0000 7fc7731c6740 1 -- 172.21.15.83:0/180947278 shutdown_connections 2024-04-24T09:02:24.869 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.865+0000 7fc7731c6740 1 -- 172.21.15.83:0/180947278 wait complete. 2024-04-24T09:02:24.869 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.865+0000 7fc7731c6740 1 -- start start 2024-04-24T09:02:24.869 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.865+0000 7fc7731c6740 1 Processor -- start 2024-04-24T09:02:24.869 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.866+0000 7fc7731c6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55edf5b189c0 0x55edf5b27920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:24.869 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.866+0000 7fc7731c6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55edf5b192e0 0x55edf5b27e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:24.869 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.866+0000 7fc7731c6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55edf5b23ba0 0x55edf5b99300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:24.870 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.866+0000 7fc7731c6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc7681ae910 con 0x55edf5b23ba0 2024-04-24T09:02:24.870 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.866+0000 7fc7731c6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc7681ae1e0 con 0x55edf5b189c0 2024-04-24T09:02:24.870 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.866+0000 7fc7731c6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc7681ab3e0 con 0x55edf5b192e0 2024-04-24T09:02:24.870 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.866+0000 7fc7731c6740 10 client.0 _ll_register_callbacks cb 0x7fc75c005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:02:24.870 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.866+0000 7fc76e200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55edf5b23ba0 0x55edf5b99300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:24.870 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.866+0000 7fc76e200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55edf5b23ba0 0x55edf5b99300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51228/0 (socket says 172.21.15.83:51228) 2024-04-24T09:02:24.870 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.866+0000 7fc76e200640 1 -- 172.21.15.83:0/1535514678 learned_addr learned my addr 172.21.15.83:0/1535514678 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:02:24.870 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.866+0000 7fc76ce00640 1 --2- 172.21.15.83:0/1535514678 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55edf5b192e0 0x55edf5b27e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:24.870 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.866+0000 7fc76d800640 1 --2- 172.21.15.83:0/1535514678 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55edf5b189c0 0x55edf5b27920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:24.871 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.867+0000 7fc76e200640 1 -- 172.21.15.83:0/1535514678 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55edf5b192e0 msgr2=0x55edf5b27e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:24.871 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.867+0000 7fc76e200640 1 --2- 172.21.15.83:0/1535514678 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55edf5b192e0 0x55edf5b27e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:24.871 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.867+0000 7fc76e200640 1 -- 172.21.15.83:0/1535514678 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55edf5b189c0 msgr2=0x55edf5b27920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:24.871 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.867+0000 7fc76e200640 1 --2- 172.21.15.83:0/1535514678 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55edf5b189c0 0x55edf5b27920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:24.871 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.867+0000 7fc76e200640 1 -- 172.21.15.83:0/1535514678 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55edf5b36520 con 0x55edf5b23ba0 2024-04-24T09:02:24.871 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.867+0000 7fc76d800640 1 --2- 172.21.15.83:0/1535514678 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55edf5b189c0 0x55edf5b27920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:02:24.871 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.867+0000 7fc76e200640 1 --2- 172.21.15.83:0/1535514678 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55edf5b23ba0 0x55edf5b99300 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7fc760098d80 tx=0x7fc76009ae30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ceffb688b067b39 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:02:24.871 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.867+0000 7fc74f400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:02:24.871 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.867+0000 7fc74f400640 1 -- 172.21.15.83:0/1535514678 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc760085e60 con 0x55edf5b23ba0 2024-04-24T09:02:24.871 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.867+0000 7fc7731c6740 1 -- 172.21.15.83:0/1535514678 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7600a5590 con 0x55edf5b23ba0 2024-04-24T09:02:24.872 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.867+0000 7fc74f400640 1 -- 172.21.15.83:0/1535514678 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9583+0+0 (secure 0 0 0) 0x7fc7600a5300 con 0x55edf5b23ba0 2024-04-24T09:02:24.872 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.867+0000 7fc74f400640 1 -- 172.21.15.83:0/1535514678 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fc7600ce560 con 0x55edf5b23ba0 2024-04-24T09:02:24.872 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.868+0000 7fc7731c6740 10 client.19434 fetch_fsmap learned FSMap version 41 2024-04-24T09:02:24.872 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.868+0000 7fc7731c6740 1 -- 172.21.15.83:0/1535514678 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55edf5855620 con 0x55edf5b23ba0 2024-04-24T09:02:24.872 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.868+0000 7fc74f400640 1 -- 172.21.15.83:0/1535514678 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc7600a5590 con 0x55edf5b23ba0 2024-04-24T09:02:24.872 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.868+0000 7fc74f400640 1 -- 172.21.15.83:0/1535514678 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7fc76000f040 con 0x55edf5b23ba0 2024-04-24T09:02:24.872 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.868+0000 7fc7731c6740 10 client.19434 fetch_fsmap finished waiting for FSMap version 41 2024-04-24T09:02:24.872 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.868+0000 7fc7731c6740 10 client.19434 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:02:24.872 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.868+0000 7fc7731c6740 20 client.19434 populate_metadata read hostname 'smithi083' 2024-04-24T09:02:24.872 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.868+0000 7fc7731c6740 1 --2- 172.21.15.83:0/1535514678 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55edf5b9ac70 0x55edf5b9d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:24.872 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.868+0000 7fc7731c6740 4 client.19434 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T09:02:24.872 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.868+0000 7fc7731c6740 1 -- 172.21.15.83:0/1535514678 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55edf5a7e190 con 0x55edf5b9ac70 2024-04-24T09:02:24.872 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.869+0000 7fc76d800640 1 --2- 172.21.15.83:0/1535514678 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55edf5b9ac70 0x55edf5b9d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:24.873 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.870+0000 7fc76d800640 1 --2- 172.21.15.83:0/1535514678 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55edf5b9ac70 0x55edf5b9d090 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78412224974e4fa5 server_cookie=9778008c30fa7507 in_seq=0 out_seq=0 2024-04-24T09:02:24.873 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.870+0000 7fc74f400640 10 client.19434 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:02:24.938 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.939+0000 7fc74f400640 1 -- 172.21.15.83:0/1535514678 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55edf5a7e190 con 0x55edf5b9ac70 2024-04-24T09:02:24.938 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.939+0000 7fc74f400640 10 client.19434 handle_command_reply: tid=0 2024-04-24T09:02:24.938 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.939+0000 7fc7731c6740 1 client.19434 shutdown 2024-04-24T09:02:24.938 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.939+0000 7fc7731c6740 1 -- 172.21.15.83:0/1535514678 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55edf5b23ba0 msgr2=0x55edf5b99300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:24.938 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.939+0000 7fc7731c6740 1 --2- 172.21.15.83:0/1535514678 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55edf5b23ba0 0x55edf5b99300 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7fc760098d80 tx=0x7fc76009ae30 comp rx=0 tx=0).stop 2024-04-24T09:02:24.939 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.940+0000 7fc7731c6740 1 -- 172.21.15.83:0/1535514678 shutdown_connections 2024-04-24T09:02:24.939 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.940+0000 7fc7731c6740 1 --2- 172.21.15.83:0/1535514678 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55edf5b9ac70 0x55edf5b9d090 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:24.940 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.940+0000 7fc7731c6740 1 --2- 172.21.15.83:0/1535514678 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55edf5b23ba0 0x55edf5b99300 unknown :-1 s=CLOSED pgs=1758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:24.940 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.940+0000 7fc7731c6740 1 --2- 172.21.15.83:0/1535514678 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55edf5b192e0 0x55edf5b27e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:24.940 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.940+0000 7fc7731c6740 1 --2- 172.21.15.83:0/1535514678 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55edf5b189c0 0x55edf5b27920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:24.940 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.940+0000 7fc74f400640 0 client.19434 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:02:24.940 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.940+0000 7fc7731c6740 1 -- 172.21.15.83:0/1535514678 >> 172.21.15.83:0/1535514678 conn(0x55edf5b14ed0 msgr2=0x55edf5b20b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:02:24.940 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.940+0000 7fc7731c6740 1 -- 172.21.15.83:0/1535514678 shutdown_connections 2024-04-24T09:02:24.940 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.940+0000 7fc7731c6740 1 -- 172.21.15.83:0/1535514678 wait complete. 2024-04-24T09:02:24.940 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.940+0000 7fc7731c6740 20 client.19434 trim_cache size 0 max 16384 2024-04-24T09:02:24.940 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T09:02:24.940 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T09:02:24.941 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T09:02:24.941 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T09:02:24.941 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.941+0000 7fc770200640 1 -- 172.21.15.83:0/1955059128 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fc7500a3dc0 msgr2=0x7fc7500a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:24.941 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.941+0000 7fc770200640 1 --2- 172.21.15.83:0/1955059128 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fc7500a3dc0 0x7fc7500a6280 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fc75c005ab0 tx=0x7fc75c005a20 comp rx=0 tx=0).stop 2024-04-24T09:02:24.941 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.941+0000 7fc770200640 1 -- 172.21.15.83:0/1955059128 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc76810e820 msgr2=0x7fc76819a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:24.942 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.941+0000 7fc770200640 1 --2- 172.21.15.83:0/1955059128 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc76810e820 0x7fc76819a7f0 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7fc76000eaa0 tx=0x7fc76000ef60 comp rx=0 tx=0).stop 2024-04-24T09:02:24.942 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.942+0000 7fc770200640 1 -- 172.21.15.83:0/1955059128 shutdown_connections 2024-04-24T09:02:24.942 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.942+0000 7fc770200640 1 --2- 172.21.15.83:0/1955059128 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fc7500a3dc0 0x7fc7500a6280 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:24.942 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.942+0000 7fc770200640 1 --2- 172.21.15.83:0/1955059128 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc76810e820 0x7fc76819a7f0 unknown :-1 s=CLOSED pgs=1754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:24.942 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.942+0000 7fc770200640 1 --2- 172.21.15.83:0/1955059128 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc768074d50 0x7fc7681a0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:24.942 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.942+0000 7fc770200640 1 --2- 172.21.15.83:0/1955059128 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc768074430 0x7fc7681a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:24.942 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.942+0000 7fc770200640 1 -- 172.21.15.83:0/1955059128 >> 172.21.15.83:0/1955059128 conn(0x7fc7680fdab0 msgr2=0x7fc7680fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:02:24.942 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.942+0000 7fc770200640 1 -- 172.21.15.83:0/1955059128 shutdown_connections 2024-04-24T09:02:24.942 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:24.942+0000 7fc770200640 1 -- 172.21.15.83:0/1955059128 wait complete. 2024-04-24T09:02:25.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.042+0000 7f83b1800640 1 -- 172.21.15.83:0/3071260272 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f83ac068d20 con 0x7f83ac104890 2024-04-24T09:02:25.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.044+0000 7f83a9600640 1 -- 172.21.15.83:0/3071260272 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+11953 (secure 0 0 0) 0x7f839c0bd730 con 0x7f83ac104890 2024-04-24T09:02:25.043 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T09:02:25.043 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":41,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/358872257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":358872257},{"type":"v1","addr":"172.21.15.83:6835","nonce":358872257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3157070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3157070851},{"type":"v1","addr":"172.21.15.83:6837","nonce":3157070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":41,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T08:48:29.422576+0000","modified":"2024-04-24T09:02:18.079759+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34520,"mds_1":34493,"mds_2":34451,"mds_3":34427,"mds_4":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14658":{"gid":14658,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6839/4085470844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":4085470844},{"type":"v1","addr":"172.21.15.83:6839","nonce":4085470844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14676":{"gid":14676,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6841/2634754210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":2634754210},{"type":"v1","addr":"172.21.15.83:6841","nonce":2634754210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24382":{"gid":24382,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/1859921310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1859921310},{"type":"v1","addr":"172.21.15.189:6833","nonce":1859921310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/2241923197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2241923197},{"type":"v1","addr":"172.21.15.189:6835","nonce":2241923197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.189:6839/2548965336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":2548965336},{"type":"v1","addr":"172.21.15.189:6839","nonce":2548965336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26686":{"gid":26686,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6837/1678817161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":1678817161},{"type":"v1","addr":"172.21.15.189:6837","nonce":1678817161}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.138:6833/1157967658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1157967658},{"type":"v1","addr":"172.21.15.138:6833","nonce":1157967658}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.138:6835/1909219758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1909219758},{"type":"v1","addr":"172.21.15.138:6835","nonce":1909219758}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34493":{"gid":34493,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.138:6837/900566754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":900566754},{"type":"v1","addr":"172.21.15.138:6837","nonce":900566754}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34520":{"gid":34520,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.138:6839/2236932393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2236932393},{"type":"v1","addr":"172.21.15.138:6839","nonce":2236932393}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34520,"qdb_cluster":[24454,34520,34427,34451,34493]},"id":1}]} 2024-04-24T09:02:25.044 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 41 2024-04-24T09:02:25.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.046+0000 7f83b1800640 1 -- 172.21.15.83:0/3071260272 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f83900a3d20 msgr2=0x7f83900a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:25.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.046+0000 7f83b1800640 1 --2- 172.21.15.83:0/3071260272 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f83900a3d20 0x7f83900a61e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f8394005fd0 tx=0x7f839400b510 comp rx=0 tx=0).stop 2024-04-24T09:02:25.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.046+0000 7f83b1800640 1 -- 172.21.15.83:0/3071260272 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83ac104890 msgr2=0x7f83ac1a47e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:25.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.046+0000 7f83b1800640 1 --2- 172.21.15.83:0/3071260272 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83ac104890 0x7f83ac1a47e0 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7f839c005ab0 tx=0x7f839c003cd0 comp rx=0 tx=0).stop 2024-04-24T09:02:25.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.047+0000 7f83b1800640 1 -- 172.21.15.83:0/3071260272 shutdown_connections 2024-04-24T09:02:25.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.047+0000 7f83b1800640 1 --2- 172.21.15.83:0/3071260272 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f83900a3d20 0x7f83900a61e0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:25.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.047+0000 7f83b1800640 1 --2- 172.21.15.83:0/3071260272 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83ac1071b0 0x7f83ac19e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:25.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.047+0000 7f83b1800640 1 --2- 172.21.15.83:0/3071260272 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f83ac106d40 0x7f83ac1a4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:25.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.047+0000 7f83b1800640 1 --2- 172.21.15.83:0/3071260272 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83ac104890 0x7f83ac1a47e0 unknown :-1 s=CLOSED pgs=1752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:25.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.047+0000 7f83b1800640 1 -- 172.21.15.83:0/3071260272 >> 172.21.15.83:0/3071260272 conn(0x7f83ac076840 msgr2=0x7f83ac101d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:02:25.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.047+0000 7f83b1800640 1 -- 172.21.15.83:0/3071260272 shutdown_connections 2024-04-24T09:02:25.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.047+0000 7f83b1800640 1 -- 172.21.15.83:0/3071260272 wait complete. 2024-04-24T09:02:25.064 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-24T09:02:25.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.323+0000 7f2ede400640 1 -- 172.21.15.83:0/1773214570 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ed8103280 msgr2=0x7f2ed8103660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:25.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.323+0000 7f2ede400640 1 --2- 172.21.15.83:0/1773214570 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ed8103280 0x7f2ed8103660 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f2ec8009ca0 tx=0x7f2ec802a9e0 comp rx=0 tx=0).stop 2024-04-24T09:02:25.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.323+0000 7f2ede400640 1 -- 172.21.15.83:0/1773214570 shutdown_connections 2024-04-24T09:02:25.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.323+0000 7f2ede400640 1 --2- 172.21.15.83:0/1773214570 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ed810c5c0 0x7f2ed810ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:25.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.323+0000 7f2ede400640 1 --2- 172.21.15.83:0/1773214570 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2ed8103ba0 0x7f2ed810c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:25.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.323+0000 7f2ede400640 1 --2- 172.21.15.83:0/1773214570 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ed8103280 0x7f2ed8103660 unknown :-1 s=CLOSED pgs=1759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:25.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.323+0000 7f2ede400640 1 -- 172.21.15.83:0/1773214570 >> 172.21.15.83:0/1773214570 conn(0x7f2ed8078000 msgr2=0x7f2ed8078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:02:25.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.323+0000 7f2ede400640 1 -- 172.21.15.83:0/1773214570 shutdown_connections 2024-04-24T09:02:25.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.323+0000 7f2ede400640 1 -- 172.21.15.83:0/1773214570 wait complete. 2024-04-24T09:02:25.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.324+0000 7f2ede400640 1 Processor -- start 2024-04-24T09:02:25.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.324+0000 7f2ede400640 1 -- start start 2024-04-24T09:02:25.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.324+0000 7f2ede400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ed8103280 0x7f2ed819e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:25.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.324+0000 7f2ede400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ed8103ba0 0x7f2ed819eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:25.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.325+0000 7f2ede400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2ed810c5c0 0x7f2ed8198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:25.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.325+0000 7f2ede400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2ed8112280 con 0x7f2ed8103ba0 2024-04-24T09:02:25.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.325+0000 7f2ede400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2ed8112100 con 0x7f2ed8103280 2024-04-24T09:02:25.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.325+0000 7f2ede400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2ed8112400 con 0x7f2ed810c5c0 2024-04-24T09:02:25.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.325+0000 7f2ed6a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ed8103ba0 0x7f2ed819eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:25.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.325+0000 7f2ed6a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ed8103ba0 0x7f2ed819eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51238/0 (socket says 172.21.15.83:51238) 2024-04-24T09:02:25.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.325+0000 7f2ed6a00640 1 -- 172.21.15.83:0/4192200267 learned_addr learned my addr 172.21.15.83:0/4192200267 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:02:25.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.325+0000 7f2ed7400640 1 --2- 172.21.15.83:0/4192200267 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ed8103280 0x7f2ed819e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:25.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.325+0000 7f2ed6a00640 1 -- 172.21.15.83:0/4192200267 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2ed810c5c0 msgr2=0x7f2ed8198660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:25.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.325+0000 7f2ed7e00640 1 --2- 172.21.15.83:0/4192200267 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2ed810c5c0 0x7f2ed8198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:25.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.325+0000 7f2ed6a00640 1 --2- 172.21.15.83:0/4192200267 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2ed810c5c0 0x7f2ed8198660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:25.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.325+0000 7f2ed6a00640 1 -- 172.21.15.83:0/4192200267 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ed8103280 msgr2=0x7f2ed819e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:25.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.325+0000 7f2ed6a00640 1 --2- 172.21.15.83:0/4192200267 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ed8103280 0x7f2ed819e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:25.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.325+0000 7f2ed6a00640 1 -- 172.21.15.83:0/4192200267 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ed8198f20 con 0x7f2ed8103ba0 2024-04-24T09:02:25.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.325+0000 7f2ed7e00640 1 --2- 172.21.15.83:0/4192200267 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2ed810c5c0 0x7f2ed8198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:02:25.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.326+0000 7f2ed6a00640 1 --2- 172.21.15.83:0/4192200267 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ed8103ba0 0x7f2ed819eae0 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f2ec400c890 tx=0x7f2ec400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:02:25.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.326+0000 7f2ecfe00640 1 -- 172.21.15.83:0/4192200267 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec4004220 con 0x7f2ed8103ba0 2024-04-24T09:02:25.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.326+0000 7f2ecfe00640 1 -- 172.21.15.83:0/4192200267 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2ec40043c0 con 0x7f2ed8103ba0 2024-04-24T09:02:25.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.326+0000 7f2ecfe00640 1 -- 172.21.15.83:0/4192200267 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec4004a00 con 0x7f2ed8103ba0 2024-04-24T09:02:25.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.326+0000 7f2ede400640 1 -- 172.21.15.83:0/4192200267 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ed8199210 con 0x7f2ed8103ba0 2024-04-24T09:02:25.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.326+0000 7f2ede400640 1 -- 172.21.15.83:0/4192200267 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ed81a5520 con 0x7f2ed8103ba0 2024-04-24T09:02:25.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.330+0000 7f2ee1394740 1 Processor -- start 2024-04-24T09:02:25.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.330+0000 7f2ee1394740 1 -- start start 2024-04-24T09:02:25.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.330+0000 7f2ee1394740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5609fc770d60 0x5609fc771140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:25.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.330+0000 7f2ee1394740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5609fc771680 0x5609fc77b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:25.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.330+0000 7f2ee1394740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5609fc77bd50 0x5609fc77e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:25.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.330+0000 7f2ee1394740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5609fc6d21d0 con 0x5609fc771680 2024-04-24T09:02:25.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.330+0000 7f2ee1394740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5609fc678450 con 0x5609fc770d60 2024-04-24T09:02:25.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.330+0000 7f2ee1394740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5609fc6ce190 con 0x5609fc77bd50 2024-04-24T09:02:25.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.330+0000 7f2ed7400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5609fc771680 0x5609fc77b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:25.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.330+0000 7f2ed7400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5609fc771680 0x5609fc77b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51250/0 (socket says 172.21.15.83:51250) 2024-04-24T09:02:25.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.330+0000 7f2ed7400640 1 -- 172.21.15.83:0/712030759 learned_addr learned my addr 172.21.15.83:0/712030759 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:02:25.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.331+0000 7f2ecfe00640 1 -- 172.21.15.83:0/4192200267 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2ec4004ba0 con 0x7f2ed8103ba0 2024-04-24T09:02:25.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.331+0000 7f2ed7e00640 1 --2- 172.21.15.83:0/712030759 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5609fc770d60 0x5609fc771140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:25.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.331+0000 7f2ed7400640 1 -- 172.21.15.83:0/712030759 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5609fc77bd50 msgr2=0x5609fc77e140 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-24T09:02:25.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.331+0000 7f2ed7400640 1 --2- 172.21.15.83:0/712030759 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5609fc77bd50 0x5609fc77e140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:25.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.331+0000 7f2ed7400640 1 -- 172.21.15.83:0/712030759 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5609fc770d60 msgr2=0x5609fc771140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:25.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.331+0000 7f2ed7400640 1 --2- 172.21.15.83:0/712030759 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5609fc770d60 0x5609fc771140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:25.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.331+0000 7f2ed7400640 1 -- 172.21.15.83:0/712030759 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609fc735310 con 0x5609fc771680 2024-04-24T09:02:25.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.331+0000 7f2ed7400640 1 --2- 172.21.15.83:0/712030759 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5609fc771680 0x5609fc77b810 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f2ec8009ca0 tx=0x7f2ec802ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecf9e98662c11998 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:02:25.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.332+0000 7f2eccc00640 1 -- 172.21.15.83:0/712030759 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec8047070 con 0x5609fc771680 2024-04-24T09:02:25.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.332+0000 7f2eccc00640 1 -- 172.21.15.83:0/712030759 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2ec8003e20 con 0x5609fc771680 2024-04-24T09:02:25.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.332+0000 7f2eccc00640 1 -- 172.21.15.83:0/712030759 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec8033040 con 0x5609fc771680 2024-04-24T09:02:25.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.333+0000 7f2ecfe00640 1 --2- 172.21.15.83:0/4192200267 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f2ebc0a3d20 0x7f2ebc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:25.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.333+0000 7f2ee1394740 1 -- 172.21.15.83:0/712030759 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5609fc771680 msgr2=0x5609fc77b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:25.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.333+0000 7f2ee1394740 1 --2- 172.21.15.83:0/712030759 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5609fc771680 0x5609fc77b810 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f2ec8009ca0 tx=0x7f2ec802ec00 comp rx=0 tx=0).stop 2024-04-24T09:02:25.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.333+0000 7f2ecfe00640 1 -- 172.21.15.83:0/4192200267 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9583+0+0 (secure 0 0 0) 0x7f2ec40d0970 con 0x7f2ed8103ba0 2024-04-24T09:02:25.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.333+0000 7f2ed7e00640 1 --2- 172.21.15.83:0/4192200267 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f2ebc0a3d20 0x7f2ebc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:25.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.333+0000 7f2ee1394740 1 -- 172.21.15.83:0/712030759 shutdown_connections 2024-04-24T09:02:25.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.333+0000 7f2ee1394740 1 --2- 172.21.15.83:0/712030759 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5609fc77bd50 0x5609fc77e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:25.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.333+0000 7f2ee1394740 1 --2- 172.21.15.83:0/712030759 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5609fc771680 0x5609fc77b810 unknown :-1 s=CLOSED pgs=1761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:25.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.333+0000 7f2ee1394740 1 --2- 172.21.15.83:0/712030759 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5609fc770d60 0x5609fc771140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:25.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.333+0000 7f2ee1394740 1 -- 172.21.15.83:0/712030759 >> 172.21.15.83:0/712030759 conn(0x5609fc76d080 msgr2=0x5609fc76d460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:02:25.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.333+0000 7f2ee1394740 1 -- 172.21.15.83:0/712030759 shutdown_connections 2024-04-24T09:02:25.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.333+0000 7f2ee1394740 1 -- 172.21.15.83:0/712030759 wait complete. 2024-04-24T09:02:25.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.334+0000 7f2ed7e00640 1 --2- 172.21.15.83:0/4192200267 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f2ebc0a3d20 0x7f2ebc0a61e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f2ed000db00 tx=0x7f2ed0005d90 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:02:25.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.334+0000 7f2ee1394740 1 -- start start 2024-04-24T09:02:25.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.334+0000 7f2ee1394740 1 Processor -- start 2024-04-24T09:02:25.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.334+0000 7f2ee1394740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5609fc770d60 0x5609fc77b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:25.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.334+0000 7f2ee1394740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5609fc771680 0x5609fc778a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:25.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.334+0000 7f2ee1394740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5609fc77bd50 0x5609fc7791c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:25.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.334+0000 7f2ee1394740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5609fc77eb80 con 0x5609fc770d60 2024-04-24T09:02:25.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.334+0000 7f2ee1394740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5609fc68b810 con 0x5609fc771680 2024-04-24T09:02:25.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.334+0000 7f2ee1394740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5609fc77ed00 con 0x5609fc77bd50 2024-04-24T09:02:25.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.335+0000 7f2ee1394740 10 client.0 _ll_register_callbacks cb 0x5609fc659140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:02:25.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.335+0000 7f2ed7400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5609fc770d60 0x5609fc77b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:25.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.335+0000 7f2ed7400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5609fc770d60 0x5609fc77b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51252/0 (socket says 172.21.15.83:51252) 2024-04-24T09:02:25.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.335+0000 7f2ed7400640 1 -- 172.21.15.83:0/2816132182 learned_addr learned my addr 172.21.15.83:0/2816132182 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:02:25.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.335+0000 7f2ed6a00640 1 --2- 172.21.15.83:0/2816132182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5609fc771680 0x5609fc778a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:25.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.335+0000 7f2ed7400640 1 -- 172.21.15.83:0/2816132182 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5609fc77bd50 msgr2=0x5609fc7791c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:25.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.335+0000 7f2ed7400640 1 --2- 172.21.15.83:0/2816132182 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5609fc77bd50 0x5609fc7791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:25.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.335+0000 7f2ed7400640 1 -- 172.21.15.83:0/2816132182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5609fc771680 msgr2=0x5609fc778a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:25.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.335+0000 7f2ed7400640 1 --2- 172.21.15.83:0/2816132182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5609fc771680 0x5609fc778a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:25.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.335+0000 7f2ed7400640 1 -- 172.21.15.83:0/2816132182 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609fc69c3f0 con 0x5609fc770d60 2024-04-24T09:02:25.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.335+0000 7f2ed6a00640 1 --2- 172.21.15.83:0/2816132182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5609fc771680 0x5609fc778a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:02:25.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.335+0000 7f2ed7400640 1 --2- 172.21.15.83:0/2816132182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5609fc770d60 0x5609fc77b790 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f2ec802e8a0 tx=0x7f2ec8009540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b14b0934a56949b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:02:25.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.336+0000 7f2ec1600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:02:25.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.336+0000 7f2ec1600640 1 -- 172.21.15.83:0/2816132182 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec8050060 con 0x5609fc770d60 2024-04-24T09:02:25.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.336+0000 7f2ee1394740 1 -- 172.21.15.83:0/2816132182 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ec8003e20 con 0x5609fc770d60 2024-04-24T09:02:25.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.336+0000 7f2ec1600640 1 -- 172.21.15.83:0/2816132182 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9583+0+0 (secure 0 0 0) 0x7f2ec8003b80 con 0x5609fc770d60 2024-04-24T09:02:25.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.336+0000 7f2ec1600640 1 -- 172.21.15.83:0/2816132182 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f2ec8047070 con 0x5609fc770d60 2024-04-24T09:02:25.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.336+0000 7f2ee1394740 10 client.19446 fetch_fsmap learned FSMap version 41 2024-04-24T09:02:25.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.336+0000 7f2ee1394740 1 -- 172.21.15.83:0/2816132182 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5609fc3e43f0 con 0x5609fc770d60 2024-04-24T09:02:25.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.336+0000 7f2ec1600640 1 -- 172.21.15.83:0/2816132182 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2ec8003e20 con 0x5609fc770d60 2024-04-24T09:02:25.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.337+0000 7f2ec1600640 1 -- 172.21.15.83:0/2816132182 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7f2ec8004450 con 0x5609fc770d60 2024-04-24T09:02:25.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.337+0000 7f2ee1394740 10 client.19446 fetch_fsmap finished waiting for FSMap version 41 2024-04-24T09:02:25.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.337+0000 7f2ee1394740 10 client.19446 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-24T09:02:25.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.337+0000 7f2ee1394740 20 client.19446 populate_metadata read hostname 'smithi083' 2024-04-24T09:02:25.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.337+0000 7f2ee1394740 1 --2- 172.21.15.83:0/2816132182 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x5609fc78bd90 0x5609fc78e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:25.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.337+0000 7f2ee1394740 4 client.19446 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:02:25.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.337+0000 7f2ee1394740 1 -- 172.21.15.83:0/2816132182 --> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5609fc6dfbf0 con 0x5609fc78bd90 2024-04-24T09:02:25.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.338+0000 7f2ed6a00640 1 --2- 172.21.15.83:0/2816132182 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x5609fc78bd90 0x5609fc78e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:25.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.338+0000 7f2ec1600640 10 client.19446 ms_handle_connect on v2:172.21.15.189:6838/2548965336 2024-04-24T09:02:25.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.338+0000 7f2ed6a00640 1 --2- 172.21.15.83:0/2816132182 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x5609fc78bd90 0x5609fc78e180 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=968e420ad0ed1e28 server_cookie=8ba75a0981753e0a in_seq=0 out_seq=0 2024-04-24T09:02:25.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.340+0000 7f2ec1600640 1 -- 172.21.15.83:0/2816132182 <== mds.4 v2:172.21.15.189:6838/2548965336 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5609fc6dfbf0 con 0x5609fc78bd90 2024-04-24T09:02:25.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.340+0000 7f2ec1600640 10 client.19446 handle_command_reply: tid=0 2024-04-24T09:02:25.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.340+0000 7f2ee1394740 1 client.19446 shutdown 2024-04-24T09:02:25.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.341+0000 7f2ee1394740 1 -- 172.21.15.83:0/2816132182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5609fc770d60 msgr2=0x5609fc77b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:25.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.341+0000 7f2ee1394740 1 --2- 172.21.15.83:0/2816132182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5609fc770d60 0x5609fc77b790 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f2ec802e8a0 tx=0x7f2ec8009540 comp rx=0 tx=0).stop 2024-04-24T09:02:25.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.341+0000 7f2ee1394740 1 -- 172.21.15.83:0/2816132182 shutdown_connections 2024-04-24T09:02:25.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.341+0000 7f2ee1394740 1 --2- 172.21.15.83:0/2816132182 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x5609fc78bd90 0x5609fc78e180 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:25.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.341+0000 7f2ee1394740 1 --2- 172.21.15.83:0/2816132182 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5609fc77bd50 0x5609fc7791c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:25.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.341+0000 7f2ee1394740 1 --2- 172.21.15.83:0/2816132182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5609fc771680 0x5609fc778a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:25.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.341+0000 7f2ee1394740 1 --2- 172.21.15.83:0/2816132182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5609fc770d60 0x5609fc77b790 unknown :-1 s=CLOSED pgs=1762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:25.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.341+0000 7f2ee1394740 1 -- 172.21.15.83:0/2816132182 >> 172.21.15.83:0/2816132182 conn(0x5609fc76d080 msgr2=0x5609fc767df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:02:25.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.341+0000 7f2ec1600640 0 client.19446 ms_handle_reset on v2:172.21.15.189:6838/2548965336 2024-04-24T09:02:25.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.341+0000 7f2ee1394740 1 -- 172.21.15.83:0/2816132182 shutdown_connections 2024-04-24T09:02:25.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.341+0000 7f2ee1394740 1 -- 172.21.15.83:0/2816132182 wait complete. 2024-04-24T09:02:25.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.341+0000 7f2ee1394740 20 client.19446 trim_cache size 0 max 16384 2024-04-24T09:02:25.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.360+0000 7f2ee1394740 1 Processor -- start 2024-04-24T09:02:25.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.360+0000 7f2ee1394740 1 -- start start 2024-04-24T09:02:25.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.360+0000 7f2ee1394740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5609fc770d60 0x5609fc77e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:25.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.360+0000 7f2ee1394740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5609fc771680 0x5609fc773000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:25.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.360+0000 7f2ee1394740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5609fc77bd50 0x5609fc777a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:25.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.360+0000 7f2ee1394740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2ed81ad4a0 con 0x5609fc77bd50 2024-04-24T09:02:25.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.360+0000 7f2ee1394740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2ed81af100 con 0x5609fc771680 2024-04-24T09:02:25.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.360+0000 7f2ee1394740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2ed81adb70 con 0x5609fc770d60 2024-04-24T09:02:25.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.360+0000 7f2ed7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5609fc77bd50 0x5609fc777a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:25.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.361+0000 7f2ed7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5609fc77bd50 0x5609fc777a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51268/0 (socket says 172.21.15.83:51268) 2024-04-24T09:02:25.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.361+0000 7f2ed7e00640 1 -- 172.21.15.83:0/1999746500 learned_addr learned my addr 172.21.15.83:0/1999746500 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:02:25.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.361+0000 7f2ed7400640 1 --2- 172.21.15.83:0/1999746500 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5609fc770d60 0x5609fc77e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:25.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.361+0000 7f2ed6a00640 1 --2- 172.21.15.83:0/1999746500 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5609fc771680 0x5609fc773000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:25.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.361+0000 7f2ed7e00640 1 -- 172.21.15.83:0/1999746500 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5609fc770d60 msgr2=0x5609fc77e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:25.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.361+0000 7f2ed7e00640 1 --2- 172.21.15.83:0/1999746500 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5609fc770d60 0x5609fc77e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:25.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.361+0000 7f2ed7e00640 1 -- 172.21.15.83:0/1999746500 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5609fc771680 msgr2=0x5609fc773000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:25.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.361+0000 7f2ed7e00640 1 --2- 172.21.15.83:0/1999746500 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5609fc771680 0x5609fc773000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:25.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.361+0000 7f2ed7e00640 1 -- 172.21.15.83:0/1999746500 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609fc779530 con 0x5609fc77bd50 2024-04-24T09:02:25.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.361+0000 7f2ed7e00640 1 --2- 172.21.15.83:0/1999746500 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5609fc77bd50 0x5609fc777a60 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f2ed0005c80 tx=0x7f2ed0012070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8be2e7a18c59e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:02:25.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.362+0000 7f2ec3e00640 1 -- 172.21.15.83:0/1999746500 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed001d630 con 0x5609fc77bd50 2024-04-24T09:02:25.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.362+0000 7f2ec3e00640 1 -- 172.21.15.83:0/1999746500 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2ed001dc80 con 0x5609fc77bd50 2024-04-24T09:02:25.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.362+0000 7f2ec3e00640 1 -- 172.21.15.83:0/1999746500 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed001bdf0 con 0x5609fc77bd50 2024-04-24T09:02:25.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.362+0000 7f2ee1394740 1 -- 172.21.15.83:0/1999746500 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5609fc77bd50 msgr2=0x5609fc777a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:25.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.362+0000 7f2ee1394740 1 --2- 172.21.15.83:0/1999746500 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5609fc77bd50 0x5609fc777a60 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f2ed0005c80 tx=0x7f2ed0012070 comp rx=0 tx=0).stop 2024-04-24T09:02:25.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.362+0000 7f2ee1394740 1 -- 172.21.15.83:0/1999746500 shutdown_connections 2024-04-24T09:02:25.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.362+0000 7f2ee1394740 1 --2- 172.21.15.83:0/1999746500 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5609fc77bd50 0x5609fc777a60 unknown :-1 s=CLOSED pgs=1763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:25.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.362+0000 7f2ee1394740 1 --2- 172.21.15.83:0/1999746500 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5609fc771680 0x5609fc773000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:25.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.362+0000 7f2ee1394740 1 --2- 172.21.15.83:0/1999746500 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5609fc770d60 0x5609fc77e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:25.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.362+0000 7f2ee1394740 1 -- 172.21.15.83:0/1999746500 >> 172.21.15.83:0/1999746500 conn(0x5609fc76d080 msgr2=0x5609fc777350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:02:25.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.363+0000 7f2ee1394740 1 -- 172.21.15.83:0/1999746500 shutdown_connections 2024-04-24T09:02:25.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.363+0000 7f2ee1394740 1 -- 172.21.15.83:0/1999746500 wait complete. 2024-04-24T09:02:25.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.363+0000 7f2ee1394740 1 -- start start 2024-04-24T09:02:25.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.363+0000 7f2ee1394740 1 Processor -- start 2024-04-24T09:02:25.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.364+0000 7f2ee1394740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5609fc770d60 0x5609fc7844c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:25.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.364+0000 7f2ee1394740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5609fc771680 0x5609fc775560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:25.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.364+0000 7f2ee1394740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5609fc77bd50 0x5609fc775c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:25.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.364+0000 7f2ee1394740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2ed81ae9d0 con 0x5609fc77bd50 2024-04-24T09:02:25.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.364+0000 7f2ee1394740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2ed81ae2a0 con 0x5609fc771680 2024-04-24T09:02:25.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.364+0000 7f2ee1394740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2ed81ab4a0 con 0x5609fc770d60 2024-04-24T09:02:25.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.364+0000 7f2ee1394740 10 client.0 _ll_register_callbacks cb 0x7f2ec8045070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:02:25.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.365+0000 7f2ed7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5609fc77bd50 0x5609fc775c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:25.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.365+0000 7f2ed7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5609fc77bd50 0x5609fc775c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51282/0 (socket says 172.21.15.83:51282) 2024-04-24T09:02:25.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.365+0000 7f2ed7e00640 1 -- 172.21.15.83:0/4143954892 learned_addr learned my addr 172.21.15.83:0/4143954892 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:02:25.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.365+0000 7f2ed6a00640 1 --2- 172.21.15.83:0/4143954892 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5609fc771680 0x5609fc775560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:25.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.365+0000 7f2ed7400640 1 --2- 172.21.15.83:0/4143954892 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5609fc770d60 0x5609fc7844c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:25.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.365+0000 7f2ed7e00640 1 -- 172.21.15.83:0/4143954892 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5609fc770d60 msgr2=0x5609fc7844c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:25.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.365+0000 7f2ed7e00640 1 --2- 172.21.15.83:0/4143954892 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5609fc770d60 0x5609fc7844c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:25.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.365+0000 7f2ed7e00640 1 -- 172.21.15.83:0/4143954892 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5609fc771680 msgr2=0x5609fc775560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:25.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.365+0000 7f2ed7e00640 1 --2- 172.21.15.83:0/4143954892 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5609fc771680 0x5609fc775560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:25.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.365+0000 7f2ed7e00640 1 -- 172.21.15.83:0/4143954892 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609fc4ae7d0 con 0x5609fc77bd50 2024-04-24T09:02:25.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.365+0000 7f2ed7400640 1 --2- 172.21.15.83:0/4143954892 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5609fc770d60 0x5609fc7844c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:02:25.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.365+0000 7f2ed7e00640 1 --2- 172.21.15.83:0/4143954892 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5609fc77bd50 0x5609fc775c80 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f2ed000f790 tx=0x7f2ed001b5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d041ff7178ce516b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:02:25.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.366+0000 7f2ec1600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:02:25.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.366+0000 7f2ec1600640 1 -- 172.21.15.83:0/4143954892 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed000fbc0 con 0x5609fc77bd50 2024-04-24T09:02:25.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.366+0000 7f2ec1600640 1 -- 172.21.15.83:0/4143954892 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9583+0+0 (secure 0 0 0) 0x7f2ed0010e00 con 0x5609fc77bd50 2024-04-24T09:02:25.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.366+0000 7f2ee1394740 1 -- 172.21.15.83:0/4143954892 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ed001dc80 con 0x5609fc77bd50 2024-04-24T09:02:25.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.366+0000 7f2ec1600640 1 -- 172.21.15.83:0/4143954892 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f2ed0039050 con 0x5609fc77bd50 2024-04-24T09:02:25.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.366+0000 7f2ee1394740 10 client.19452 fetch_fsmap learned FSMap version 41 2024-04-24T09:02:25.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.366+0000 7f2ee1394740 1 -- 172.21.15.83:0/4143954892 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5609fc768640 con 0x5609fc77bd50 2024-04-24T09:02:25.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.366+0000 7f2ec1600640 1 -- 172.21.15.83:0/4143954892 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2ed001dc80 con 0x5609fc77bd50 2024-04-24T09:02:25.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.367+0000 7f2ec1600640 1 -- 172.21.15.83:0/4143954892 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7f2ed0030b50 con 0x5609fc77bd50 2024-04-24T09:02:25.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.367+0000 7f2ee1394740 10 client.19452 fetch_fsmap finished waiting for FSMap version 41 2024-04-24T09:02:25.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.367+0000 7f2ee1394740 10 client.19452 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-24T09:02:25.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.367+0000 7f2ee1394740 20 client.19452 populate_metadata read hostname 'smithi083' 2024-04-24T09:02:25.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.367+0000 7f2ee1394740 1 --2- 172.21.15.83:0/4143954892 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x5609fc768d10 0x5609fc76b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:25.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.367+0000 7f2ee1394740 4 client.19452 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:02:25.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.367+0000 7f2ee1394740 1 -- 172.21.15.83:0/4143954892 --> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5609fc6d9a60 con 0x5609fc768d10 2024-04-24T09:02:25.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.367+0000 7f2ed7400640 1 --2- 172.21.15.83:0/4143954892 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x5609fc768d10 0x5609fc76b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:25.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.368+0000 7f2ed7400640 1 --2- 172.21.15.83:0/4143954892 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x5609fc768d10 0x5609fc76b130 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=71600b51eb8ff83e server_cookie=3a2cfa665523941a in_seq=0 out_seq=0 2024-04-24T09:02:25.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.368+0000 7f2ec1600640 10 client.19452 ms_handle_connect on v2:172.21.15.189:6838/2548965336 2024-04-24T09:02:25.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.369+0000 7f2ec1600640 1 -- 172.21.15.83:0/4143954892 <== mds.4 v2:172.21.15.189:6838/2548965336 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5609fc6d9a60 con 0x5609fc768d10 2024-04-24T09:02:25.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.369+0000 7f2ec1600640 10 client.19452 handle_command_reply: tid=0 2024-04-24T09:02:25.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.369+0000 7f2ee1394740 1 client.19452 shutdown 2024-04-24T09:02:25.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.369+0000 7f2ee1394740 1 -- 172.21.15.83:0/4143954892 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5609fc77bd50 msgr2=0x5609fc775c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:25.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.369+0000 7f2ee1394740 1 --2- 172.21.15.83:0/4143954892 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5609fc77bd50 0x5609fc775c80 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f2ed000f790 tx=0x7f2ed001b5e0 comp rx=0 tx=0).stop 2024-04-24T09:02:25.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.369+0000 7f2ee1394740 1 -- 172.21.15.83:0/4143954892 shutdown_connections 2024-04-24T09:02:25.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.369+0000 7f2ee1394740 1 --2- 172.21.15.83:0/4143954892 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x5609fc768d10 0x5609fc76b130 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:25.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.369+0000 7f2ee1394740 1 --2- 172.21.15.83:0/4143954892 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5609fc77bd50 0x5609fc775c80 unknown :-1 s=CLOSED pgs=1764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:25.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.369+0000 7f2ee1394740 1 --2- 172.21.15.83:0/4143954892 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5609fc771680 0x5609fc775560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:25.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.369+0000 7f2ec1600640 0 client.19452 ms_handle_reset on v2:172.21.15.189:6838/2548965336 2024-04-24T09:02:25.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.369+0000 7f2ee1394740 1 --2- 172.21.15.83:0/4143954892 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5609fc770d60 0x5609fc7844c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:25.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.369+0000 7f2ee1394740 1 -- 172.21.15.83:0/4143954892 >> 172.21.15.83:0/4143954892 conn(0x5609fc76d080 msgr2=0x5609fc767df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:02:25.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.369+0000 7f2ee1394740 1 -- 172.21.15.83:0/4143954892 shutdown_connections 2024-04-24T09:02:25.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.370+0000 7f2ee1394740 1 -- 172.21.15.83:0/4143954892 wait complete. 2024-04-24T09:02:25.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.370+0000 7f2ee1394740 20 client.19452 trim_cache size 0 max 16384 2024-04-24T09:02:25.372 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:02:25.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.370+0000 7f2ede400640 1 -- 172.21.15.83:0/4192200267 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f2ebc0a3d20 msgr2=0x7f2ebc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:25.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.370+0000 7f2ede400640 1 --2- 172.21.15.83:0/4192200267 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f2ebc0a3d20 0x7f2ebc0a61e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f2ed000db00 tx=0x7f2ed0005d90 comp rx=0 tx=0).stop 2024-04-24T09:02:25.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.370+0000 7f2ede400640 1 -- 172.21.15.83:0/4192200267 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ed8103ba0 msgr2=0x7f2ed819eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:25.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.370+0000 7f2ede400640 1 --2- 172.21.15.83:0/4192200267 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ed8103ba0 0x7f2ed819eae0 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f2ec400c890 tx=0x7f2ec400cd50 comp rx=0 tx=0).stop 2024-04-24T09:02:25.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.370+0000 7f2ede400640 1 -- 172.21.15.83:0/4192200267 shutdown_connections 2024-04-24T09:02:25.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.370+0000 7f2ede400640 1 --2- 172.21.15.83:0/4192200267 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f2ebc0a3d20 0x7f2ebc0a61e0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:25.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.370+0000 7f2ede400640 1 --2- 172.21.15.83:0/4192200267 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2ed810c5c0 0x7f2ed8198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:25.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.370+0000 7f2ede400640 1 --2- 172.21.15.83:0/4192200267 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ed8103ba0 0x7f2ed819eae0 unknown :-1 s=CLOSED pgs=1760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:25.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.370+0000 7f2ede400640 1 --2- 172.21.15.83:0/4192200267 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ed8103280 0x7f2ed819e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:25.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.370+0000 7f2ede400640 1 -- 172.21.15.83:0/4192200267 >> 172.21.15.83:0/4192200267 conn(0x7f2ed8078000 msgr2=0x7f2ed80fe060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:02:25.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.371+0000 7f2ede400640 1 -- 172.21.15.83:0/4192200267 shutdown_connections 2024-04-24T09:02:25.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.371+0000 7f2ede400640 1 -- 172.21.15.83:0/4192200267 wait complete. 2024-04-24T09:02:25.389 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-24T09:02:25.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.589+0000 7f88c9200640 1 -- 172.21.15.83:0/1783561411 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88c4112eb0 msgr2=0x7f88c41152a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:25.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.589+0000 7f88c9200640 1 --2- 172.21.15.83:0/1783561411 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88c4112eb0 0x7f88c41152a0 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f88bc009ca0 tx=0x7f88bc02a950 comp rx=0 tx=0).stop 2024-04-24T09:02:25.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.589+0000 7f88c9200640 1 -- 172.21.15.83:0/1783561411 shutdown_connections 2024-04-24T09:02:25.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.589+0000 7f88c9200640 1 --2- 172.21.15.83:0/1783561411 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88c4112eb0 0x7f88c41152a0 unknown :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:25.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.589+0000 7f88c9200640 1 --2- 172.21.15.83:0/1783561411 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f88c4074cc0 0x7f88c4112970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:25.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.589+0000 7f88c9200640 1 --2- 172.21.15.83:0/1783561411 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88c40743a0 0x7f88c4074780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:25.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.589+0000 7f88c9200640 1 -- 172.21.15.83:0/1783561411 >> 172.21.15.83:0/1783561411 conn(0x7f88c40fd9d0 msgr2=0x7f88c40ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:02:25.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.589+0000 7f88c9200640 1 -- 172.21.15.83:0/1783561411 shutdown_connections 2024-04-24T09:02:25.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.589+0000 7f88c9200640 1 -- 172.21.15.83:0/1783561411 wait complete. 2024-04-24T09:02:25.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.590+0000 7f88c9200640 1 Processor -- start 2024-04-24T09:02:25.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.590+0000 7f88c9200640 1 -- start start 2024-04-24T09:02:25.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.590+0000 7f88c9200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f88c40743a0 0x7f88c41a48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:25.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.590+0000 7f88c9200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88c4074cc0 0x7f88c41a4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:25.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.591+0000 7f88c9200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88c4112eb0 0x7f88c419eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:25.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.591+0000 7f88c9200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f88c4118700 con 0x7f88c4112eb0 2024-04-24T09:02:25.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.591+0000 7f88c9200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f88c4118580 con 0x7f88c4074cc0 2024-04-24T09:02:25.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.591+0000 7f88c9200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f88c4118880 con 0x7f88c40743a0 2024-04-24T09:02:25.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.591+0000 7f88c3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88c4112eb0 0x7f88c419eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:25.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.591+0000 7f88c3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88c4112eb0 0x7f88c419eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51306/0 (socket says 172.21.15.83:51306) 2024-04-24T09:02:25.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.591+0000 7f88c3e00640 1 -- 172.21.15.83:0/1348375028 learned_addr learned my addr 172.21.15.83:0/1348375028 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:02:25.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.591+0000 7f88c2a00640 1 --2- 172.21.15.83:0/1348375028 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88c4074cc0 0x7f88c41a4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:25.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.591+0000 7f88c3400640 1 --2- 172.21.15.83:0/1348375028 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f88c40743a0 0x7f88c41a48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:25.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.591+0000 7f88c3e00640 1 -- 172.21.15.83:0/1348375028 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f88c40743a0 msgr2=0x7f88c41a48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:25.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.591+0000 7f88c3e00640 1 --2- 172.21.15.83:0/1348375028 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f88c40743a0 0x7f88c41a48e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:25.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.591+0000 7f88c3e00640 1 -- 172.21.15.83:0/1348375028 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88c4074cc0 msgr2=0x7f88c41a4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:25.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.591+0000 7f88c3e00640 1 --2- 172.21.15.83:0/1348375028 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88c4074cc0 0x7f88c41a4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:25.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.591+0000 7f88c3e00640 1 -- 172.21.15.83:0/1348375028 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88c419f260 con 0x7f88c4112eb0 2024-04-24T09:02:25.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.591+0000 7f88c3400640 1 --2- 172.21.15.83:0/1348375028 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f88c40743a0 0x7f88c41a48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:02:25.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.592+0000 7f88c3e00640 1 --2- 172.21.15.83:0/1348375028 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88c4112eb0 0x7f88c419eae0 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f88bc02ae70 tx=0x7f88bc02f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:02:25.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.592+0000 7f88bbe00640 1 -- 172.21.15.83:0/1348375028 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88bc034040 con 0x7f88c4112eb0 2024-04-24T09:02:25.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.592+0000 7f88bbe00640 1 -- 172.21.15.83:0/1348375028 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f88bc0308f0 con 0x7f88c4112eb0 2024-04-24T09:02:25.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.592+0000 7f88bbe00640 1 -- 172.21.15.83:0/1348375028 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88bc0047c0 con 0x7f88c4112eb0 2024-04-24T09:02:25.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.592+0000 7f88c9200640 1 -- 172.21.15.83:0/1348375028 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88c419f4f0 con 0x7f88c4112eb0 2024-04-24T09:02:25.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.592+0000 7f88c9200640 1 -- 172.21.15.83:0/1348375028 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88c41ab7a0 con 0x7f88c4112eb0 2024-04-24T09:02:25.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.595+0000 7f88cd322740 1 Processor -- start 2024-04-24T09:02:25.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.595+0000 7f88bbe00640 1 -- 172.21.15.83:0/1348375028 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f88bc037020 con 0x7f88c4112eb0 2024-04-24T09:02:25.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.596+0000 7f88cd322740 1 -- start start 2024-04-24T09:02:25.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.596+0000 7f88cd322740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bdca213b40 0x55bdca213f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:25.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.596+0000 7f88cd322740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bdca214460 0x55bdca21e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:25.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.596+0000 7f88cd322740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bdca21eb30 0x55bdca220f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:25.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.596+0000 7f88cd322740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55bdca1791d0 con 0x55bdca213b40 2024-04-24T09:02:25.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.596+0000 7f88cd322740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55bdca11f450 con 0x55bdca21eb30 2024-04-24T09:02:25.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.596+0000 7f88cd322740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55bdca175190 con 0x55bdca214460 2024-04-24T09:02:25.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.596+0000 7f88c2a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bdca213b40 0x55bdca213f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:25.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.596+0000 7f88c2a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bdca213b40 0x55bdca213f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51314/0 (socket says 172.21.15.83:51314) 2024-04-24T09:02:25.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.596+0000 7f88c2a00640 1 -- 172.21.15.83:0/1977643525 learned_addr learned my addr 172.21.15.83:0/1977643525 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:02:25.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.597+0000 7f88c3400640 1 --2- 172.21.15.83:0/1977643525 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bdca214460 0x55bdca21e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:25.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.597+0000 7f88c2a00640 1 --2- 172.21.15.83:0/1977643525 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bdca21eb30 0x55bdca220f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:25.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.597+0000 7f88c2a00640 1 -- 172.21.15.83:0/1977643525 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bdca214460 msgr2=0x55bdca21e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:25.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.597+0000 7f88c2a00640 1 --2- 172.21.15.83:0/1977643525 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bdca214460 0x55bdca21e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:25.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.597+0000 7f88c2a00640 1 -- 172.21.15.83:0/1977643525 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bdca21eb30 msgr2=0x55bdca220f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:25.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.597+0000 7f88c2a00640 1 --2- 172.21.15.83:0/1977643525 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bdca21eb30 0x55bdca220f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:25.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.597+0000 7f88c2a00640 1 -- 172.21.15.83:0/1977643525 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bdca1dc310 con 0x55bdca213b40 2024-04-24T09:02:25.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.597+0000 7f88bbe00640 1 --2- 172.21.15.83:0/1348375028 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f88a80a3da0 0x7f88a80a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:25.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.597+0000 7f88c2a00640 1 --2- 172.21.15.83:0/1977643525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bdca213b40 0x55bdca213f20 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f88ac00cd70 tx=0x7f88ac00c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24a48f604cf03aa2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:02:25.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.597+0000 7f88bbe00640 1 -- 172.21.15.83:0/1348375028 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9583+0+0 (secure 0 0 0) 0x7f88bc0f1b60 con 0x7f88c4112eb0 2024-04-24T09:02:25.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.597+0000 7f88c3400640 1 --2- 172.21.15.83:0/1348375028 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f88a80a3da0 0x7f88a80a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:25.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.598+0000 7f88b8c00640 1 -- 172.21.15.83:0/1977643525 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88ac004220 con 0x55bdca213b40 2024-04-24T09:02:25.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.598+0000 7f88b8c00640 1 -- 172.21.15.83:0/1977643525 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f88ac0043c0 con 0x55bdca213b40 2024-04-24T09:02:25.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.598+0000 7f88b8c00640 1 -- 172.21.15.83:0/1977643525 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88ac004e40 con 0x55bdca213b40 2024-04-24T09:02:25.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.599+0000 7f88cd322740 1 -- 172.21.15.83:0/1977643525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bdca213b40 msgr2=0x55bdca213f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:25.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.599+0000 7f88cd322740 1 --2- 172.21.15.83:0/1977643525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bdca213b40 0x55bdca213f20 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f88ac00cd70 tx=0x7f88ac00c4f0 comp rx=0 tx=0).stop 2024-04-24T09:02:25.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.599+0000 7f88c3400640 1 --2- 172.21.15.83:0/1348375028 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f88a80a3da0 0x7f88a80a6260 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f88b4005fd0 tx=0x7f88b4009350 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:02:25.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.599+0000 7f88cd322740 1 -- 172.21.15.83:0/1977643525 shutdown_connections 2024-04-24T09:02:25.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.599+0000 7f88cd322740 1 --2- 172.21.15.83:0/1977643525 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bdca21eb30 0x55bdca220f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:25.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.599+0000 7f88cd322740 1 --2- 172.21.15.83:0/1977643525 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bdca214460 0x55bdca21e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:25.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.599+0000 7f88cd322740 1 --2- 172.21.15.83:0/1977643525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bdca213b40 0x55bdca213f20 unknown :-1 s=CLOSED pgs=1767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:25.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.599+0000 7f88cd322740 1 -- 172.21.15.83:0/1977643525 >> 172.21.15.83:0/1977643525 conn(0x55bdca20fe60 msgr2=0x55bdca210240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:02:25.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.599+0000 7f88cd322740 1 -- 172.21.15.83:0/1977643525 shutdown_connections 2024-04-24T09:02:25.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.599+0000 7f88cd322740 1 -- 172.21.15.83:0/1977643525 wait complete. 2024-04-24T09:02:25.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.599+0000 7f88cd322740 1 -- start start 2024-04-24T09:02:25.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.600+0000 7f88cd322740 1 Processor -- start 2024-04-24T09:02:25.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.600+0000 7f88cd322740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bdca213b40 0x55bdca21b920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:25.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.600+0000 7f88cd322740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bdca214460 0x55bdca21be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:25.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.600+0000 7f88cd322740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bdca21eb30 0x55bdca21c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:25.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.600+0000 7f88cd322740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55bdca2219c0 con 0x55bdca213b40 2024-04-24T09:02:25.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.600+0000 7f88cd322740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55bdca132810 con 0x55bdca214460 2024-04-24T09:02:25.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.600+0000 7f88cd322740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55bdca221b40 con 0x55bdca21eb30 2024-04-24T09:02:25.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.600+0000 7f88cd322740 10 client.0 _ll_register_callbacks cb 0x55bdca100140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:02:25.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.601+0000 7f88c3400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bdca213b40 0x55bdca21b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:25.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.601+0000 7f88c3400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bdca213b40 0x55bdca21b920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51320/0 (socket says 172.21.15.83:51320) 2024-04-24T09:02:25.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.601+0000 7f88c3400640 1 -- 172.21.15.83:0/726908241 learned_addr learned my addr 172.21.15.83:0/726908241 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:02:25.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.601+0000 7f88c3e00640 1 --2- 172.21.15.83:0/726908241 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bdca21eb30 0x55bdca21c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:25.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.601+0000 7f88c2a00640 1 --2- 172.21.15.83:0/726908241 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bdca214460 0x55bdca21be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:25.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.601+0000 7f88c3400640 1 -- 172.21.15.83:0/726908241 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bdca21eb30 msgr2=0x55bdca21c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:25.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.601+0000 7f88c3400640 1 --2- 172.21.15.83:0/726908241 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bdca21eb30 0x55bdca21c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:25.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.601+0000 7f88c3400640 1 -- 172.21.15.83:0/726908241 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bdca214460 msgr2=0x55bdca21be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:25.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.601+0000 7f88c3400640 1 --2- 172.21.15.83:0/726908241 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bdca214460 0x55bdca21be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:25.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.601+0000 7f88c3400640 1 -- 172.21.15.83:0/726908241 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bdca1433f0 con 0x55bdca213b40 2024-04-24T09:02:25.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.602+0000 7f88c3400640 1 --2- 172.21.15.83:0/726908241 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bdca213b40 0x55bdca21b920 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f88b4011010 tx=0x7f88b4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcbd4ef1ee2176be server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:02:25.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.602+0000 7f88b1600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:02:25.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.602+0000 7f88b1600640 1 -- 172.21.15.83:0/726908241 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b4023070 con 0x55bdca213b40 2024-04-24T09:02:25.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.602+0000 7f88b1600640 1 -- 172.21.15.83:0/726908241 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9583+0+0 (secure 0 0 0) 0x7f88b4005220 con 0x55bdca213b40 2024-04-24T09:02:25.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.602+0000 7f88cd322740 1 -- 172.21.15.83:0/726908241 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88ac0043c0 con 0x55bdca213b40 2024-04-24T09:02:25.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.602+0000 7f88b1600640 1 -- 172.21.15.83:0/726908241 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f88b402e050 con 0x55bdca213b40 2024-04-24T09:02:25.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.602+0000 7f88cd322740 10 client.19464 fetch_fsmap learned FSMap version 41 2024-04-24T09:02:25.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.602+0000 7f88cd322740 1 -- 172.21.15.83:0/726908241 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55bdc9e8b3f0 con 0x55bdca213b40 2024-04-24T09:02:25.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.603+0000 7f88b1600640 1 -- 172.21.15.83:0/726908241 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f88ac0043c0 con 0x55bdca213b40 2024-04-24T09:02:25.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.603+0000 7f88b1600640 1 -- 172.21.15.83:0/726908241 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7f88b400f040 con 0x55bdca213b40 2024-04-24T09:02:25.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.603+0000 7f88cd322740 10 client.19464 fetch_fsmap finished waiting for FSMap version 41 2024-04-24T09:02:25.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.603+0000 7f88cd322740 10 client.19464 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-24T09:02:25.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.603+0000 7f88cd322740 20 client.19464 populate_metadata read hostname 'smithi083' 2024-04-24T09:02:25.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.603+0000 7f88cd322740 1 --2- 172.21.15.83:0/726908241 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55bdca22ee00 0x55bdca231280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:25.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.603+0000 7f88cd322740 4 client.19464 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:02:25.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.603+0000 7f88cd322740 1 -- 172.21.15.83:0/726908241 --> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bdca186bf0 con 0x55bdca22ee00 2024-04-24T09:02:25.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.604+0000 7f88c2a00640 1 --2- 172.21.15.83:0/726908241 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55bdca22ee00 0x55bdca231280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:25.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.605+0000 7f88c2a00640 1 --2- 172.21.15.83:0/726908241 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55bdca22ee00 0x55bdca231280 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8725cdf06951ef31 server_cookie=7e0eb631777dc85b in_seq=0 out_seq=0 2024-04-24T09:02:25.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.605+0000 7f88b1600640 10 client.19464 ms_handle_connect on v2:172.21.15.138:6832/1157967658 2024-04-24T09:02:25.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.607+0000 7f88b1600640 1 -- 172.21.15.83:0/726908241 <== mds.3 v2:172.21.15.138:6832/1157967658 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bdca186bf0 con 0x55bdca22ee00 2024-04-24T09:02:25.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.607+0000 7f88b1600640 10 client.19464 handle_command_reply: tid=0 2024-04-24T09:02:25.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.607+0000 7f88cd322740 1 client.19464 shutdown 2024-04-24T09:02:25.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.608+0000 7f88cd322740 1 -- 172.21.15.83:0/726908241 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bdca213b40 msgr2=0x55bdca21b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:25.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.608+0000 7f88cd322740 1 --2- 172.21.15.83:0/726908241 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bdca213b40 0x55bdca21b920 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f88b4011010 tx=0x7f88b4004730 comp rx=0 tx=0).stop 2024-04-24T09:02:25.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.608+0000 7f88cd322740 1 -- 172.21.15.83:0/726908241 shutdown_connections 2024-04-24T09:02:25.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.608+0000 7f88cd322740 1 --2- 172.21.15.83:0/726908241 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55bdca22ee00 0x55bdca231280 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:25.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.608+0000 7f88cd322740 1 --2- 172.21.15.83:0/726908241 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bdca21eb30 0x55bdca21c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:25.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.608+0000 7f88cd322740 1 --2- 172.21.15.83:0/726908241 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bdca214460 0x55bdca21be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:25.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.608+0000 7f88cd322740 1 --2- 172.21.15.83:0/726908241 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bdca213b40 0x55bdca21b920 unknown :-1 s=CLOSED pgs=1768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:25.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.608+0000 7f88b1600640 0 client.19464 ms_handle_reset on v2:172.21.15.138:6832/1157967658 2024-04-24T09:02:25.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.608+0000 7f88cd322740 1 -- 172.21.15.83:0/726908241 >> 172.21.15.83:0/726908241 conn(0x55bdca20fe60 msgr2=0x55bdca215c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:02:25.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.608+0000 7f88cd322740 1 -- 172.21.15.83:0/726908241 shutdown_connections 2024-04-24T09:02:25.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.608+0000 7f88cd322740 1 -- 172.21.15.83:0/726908241 wait complete. 2024-04-24T09:02:25.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.608+0000 7f88cd322740 20 client.19464 trim_cache size 0 max 16384 2024-04-24T09:02:25.612 INFO:tasks.workunit.client.0.smithi083.stdout: 1 63243 0.62 MB/sec execute 419 sec latency 60.430 ms 2024-04-24T09:02:25.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.623+0000 7f88cd322740 1 Processor -- start 2024-04-24T09:02:25.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.623+0000 7f88cd322740 1 -- start start 2024-04-24T09:02:25.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.623+0000 7f88cd322740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bdca213b40 0x55bdca22a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:25.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.623+0000 7f88cd322740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bdca214460 0x55bdca216300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:25.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.623+0000 7f88cd322740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bdca21eb30 0x55bdca20edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:25.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.623+0000 7f88cd322740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f88c41b5a20 con 0x55bdca21eb30 2024-04-24T09:02:25.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.623+0000 7f88cd322740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f88c41b7680 con 0x55bdca213b40 2024-04-24T09:02:25.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.623+0000 7f88cd322740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f88c41b60f0 con 0x55bdca214460 2024-04-24T09:02:25.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.624+0000 7f88c3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bdca21eb30 0x55bdca20edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:25.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.624+0000 7f88c3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bdca21eb30 0x55bdca20edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51324/0 (socket says 172.21.15.83:51324) 2024-04-24T09:02:25.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.624+0000 7f88c3e00640 1 -- 172.21.15.83:0/1914319624 learned_addr learned my addr 172.21.15.83:0/1914319624 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:02:25.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.624+0000 7f88c2a00640 1 --2- 172.21.15.83:0/1914319624 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bdca214460 0x55bdca216300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:25.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.624+0000 7f88c3400640 1 --2- 172.21.15.83:0/1914319624 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bdca213b40 0x55bdca22a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:25.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.624+0000 7f88c3e00640 1 -- 172.21.15.83:0/1914319624 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bdca214460 msgr2=0x55bdca216300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:25.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.624+0000 7f88c3e00640 1 --2- 172.21.15.83:0/1914319624 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bdca214460 0x55bdca216300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:25.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.624+0000 7f88c3e00640 1 -- 172.21.15.83:0/1914319624 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bdca213b40 msgr2=0x55bdca22a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:25.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.624+0000 7f88c3e00640 1 --2- 172.21.15.83:0/1914319624 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bdca213b40 0x55bdca22a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:25.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.624+0000 7f88c3e00640 1 -- 172.21.15.83:0/1914319624 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bdc9f557d0 con 0x55bdca21eb30 2024-04-24T09:02:25.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.624+0000 7f88c2a00640 1 --2- 172.21.15.83:0/1914319624 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bdca214460 0x55bdca216300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:02:25.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.624+0000 7f88c3400640 1 --2- 172.21.15.83:0/1914319624 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bdca213b40 0x55bdca22a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:02:25.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.625+0000 7f88c3e00640 1 --2- 172.21.15.83:0/1914319624 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bdca21eb30 0x55bdca20edd0 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f88bc0b9bf0 tx=0x7f88bc02ff90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf11769f4ac34d7f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:02:25.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.625+0000 7f88b3e00640 1 -- 172.21.15.83:0/1914319624 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88bc02c040 con 0x55bdca21eb30 2024-04-24T09:02:25.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.625+0000 7f88b3e00640 1 -- 172.21.15.83:0/1914319624 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f88bc04de40 con 0x55bdca21eb30 2024-04-24T09:02:25.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.625+0000 7f88b3e00640 1 -- 172.21.15.83:0/1914319624 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88bc0ef0b0 con 0x55bdca21eb30 2024-04-24T09:02:25.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.626+0000 7f88cd322740 1 -- 172.21.15.83:0/1914319624 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bdca21eb30 msgr2=0x55bdca20edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:25.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.626+0000 7f88cd322740 1 --2- 172.21.15.83:0/1914319624 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bdca21eb30 0x55bdca20edd0 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f88bc0b9bf0 tx=0x7f88bc02ff90 comp rx=0 tx=0).stop 2024-04-24T09:02:25.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.626+0000 7f88cd322740 1 -- 172.21.15.83:0/1914319624 shutdown_connections 2024-04-24T09:02:25.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.626+0000 7f88cd322740 1 --2- 172.21.15.83:0/1914319624 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bdca21eb30 0x55bdca20edd0 unknown :-1 s=CLOSED pgs=1769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:25.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.626+0000 7f88cd322740 1 --2- 172.21.15.83:0/1914319624 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bdca214460 0x55bdca216300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:25.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.626+0000 7f88cd322740 1 --2- 172.21.15.83:0/1914319624 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bdca213b40 0x55bdca22a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:25.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.626+0000 7f88cd322740 1 -- 172.21.15.83:0/1914319624 >> 172.21.15.83:0/1914319624 conn(0x55bdca20fe60 msgr2=0x55bdca20e6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:02:25.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.626+0000 7f88cd322740 1 -- 172.21.15.83:0/1914319624 shutdown_connections 2024-04-24T09:02:25.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.626+0000 7f88cd322740 1 -- 172.21.15.83:0/1914319624 wait complete. 2024-04-24T09:02:25.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.626+0000 7f88cd322740 1 -- start start 2024-04-24T09:02:25.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.627+0000 7f88cd322740 1 Processor -- start 2024-04-24T09:02:25.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.627+0000 7f88cd322740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bdca213b40 0x55bdca227200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:25.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.627+0000 7f88cd322740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bdca214460 0x55bdca218bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:25.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.627+0000 7f88cd322740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bdca21eb30 0x55bdca219120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:25.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.627+0000 7f88cd322740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f88c41b6f50 con 0x55bdca21eb30 2024-04-24T09:02:25.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.627+0000 7f88cd322740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f88c41b6820 con 0x55bdca213b40 2024-04-24T09:02:25.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.627+0000 7f88cd322740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f88c41b3a20 con 0x55bdca214460 2024-04-24T09:02:25.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.627+0000 7f88cd322740 10 client.0 _ll_register_callbacks cb 0x7f88b4016070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:02:25.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.627+0000 7f88c3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bdca21eb30 0x55bdca219120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:25.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.627+0000 7f88c3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bdca21eb30 0x55bdca219120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51328/0 (socket says 172.21.15.83:51328) 2024-04-24T09:02:25.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.627+0000 7f88c3e00640 1 -- 172.21.15.83:0/3791563245 learned_addr learned my addr 172.21.15.83:0/3791563245 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:02:25.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.627+0000 7f88c3400640 1 --2- 172.21.15.83:0/3791563245 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bdca213b40 0x55bdca227200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:25.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.628+0000 7f88c2a00640 1 --2- 172.21.15.83:0/3791563245 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bdca214460 0x55bdca218bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:25.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.628+0000 7f88c3e00640 1 -- 172.21.15.83:0/3791563245 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bdca214460 msgr2=0x55bdca218bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:25.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.628+0000 7f88c3e00640 1 --2- 172.21.15.83:0/3791563245 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bdca214460 0x55bdca218bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:25.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.628+0000 7f88c3e00640 1 -- 172.21.15.83:0/3791563245 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bdca213b40 msgr2=0x55bdca227200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:25.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.628+0000 7f88c3e00640 1 --2- 172.21.15.83:0/3791563245 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bdca213b40 0x55bdca227200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:25.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.628+0000 7f88c3e00640 1 -- 172.21.15.83:0/3791563245 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bdca219950 con 0x55bdca21eb30 2024-04-24T09:02:25.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.628+0000 7f88c2a00640 1 --2- 172.21.15.83:0/3791563245 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bdca214460 0x55bdca218bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:02:25.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.628+0000 7f88c3e00640 1 --2- 172.21.15.83:0/3791563245 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bdca21eb30 0x55bdca219120 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f88bc0bd490 tx=0x7f88bc009980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=813840d2a370c470 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:02:25.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.628+0000 7f88b1600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:02:25.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.628+0000 7f88b1600640 1 -- 172.21.15.83:0/3791563245 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88bc0da030 con 0x55bdca21eb30 2024-04-24T09:02:25.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.628+0000 7f88b1600640 1 -- 172.21.15.83:0/3791563245 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9583+0+0 (secure 0 0 0) 0x7f88bc004040 con 0x55bdca21eb30 2024-04-24T09:02:25.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.628+0000 7f88cd322740 1 -- 172.21.15.83:0/3791563245 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88bc04de40 con 0x55bdca21eb30 2024-04-24T09:02:25.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.628+0000 7f88b1600640 1 -- 172.21.15.83:0/3791563245 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f88bc02c040 con 0x55bdca21eb30 2024-04-24T09:02:25.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.628+0000 7f88cd322740 10 client.19470 fetch_fsmap learned FSMap version 41 2024-04-24T09:02:25.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.628+0000 7f88cd322740 1 -- 172.21.15.83:0/3791563245 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55bdca2953c0 con 0x55bdca21eb30 2024-04-24T09:02:25.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.629+0000 7f88b1600640 1 -- 172.21.15.83:0/3791563245 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f88bc04de40 con 0x55bdca21eb30 2024-04-24T09:02:25.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.629+0000 7f88b1600640 1 -- 172.21.15.83:0/3791563245 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7f88bc0c5f70 con 0x55bdca21eb30 2024-04-24T09:02:25.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.629+0000 7f88cd322740 10 client.19470 fetch_fsmap finished waiting for FSMap version 41 2024-04-24T09:02:25.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.629+0000 7f88cd322740 10 client.19470 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-24T09:02:25.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.629+0000 7f88cd322740 20 client.19470 populate_metadata read hostname 'smithi083' 2024-04-24T09:02:25.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.629+0000 7f88cd322740 1 --2- 172.21.15.83:0/3791563245 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55bdca2958d0 0x55bdca297cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:25.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.629+0000 7f88cd322740 4 client.19470 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:02:25.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.629+0000 7f88cd322740 1 -- 172.21.15.83:0/3791563245 --> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bdca180a60 con 0x55bdca2958d0 2024-04-24T09:02:25.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.630+0000 7f88c3400640 1 --2- 172.21.15.83:0/3791563245 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55bdca2958d0 0x55bdca297cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:25.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.631+0000 7f88b1600640 10 client.19470 ms_handle_connect on v2:172.21.15.138:6832/1157967658 2024-04-24T09:02:25.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.631+0000 7f88c3400640 1 --2- 172.21.15.83:0/3791563245 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55bdca2958d0 0x55bdca297cf0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2dfc0743c292567a server_cookie=9b4f4df8f9d55fd4 in_seq=0 out_seq=0 2024-04-24T09:02:25.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.631+0000 7f88b1600640 1 -- 172.21.15.83:0/3791563245 <== mds.3 v2:172.21.15.138:6832/1157967658 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bdca180a60 con 0x55bdca2958d0 2024-04-24T09:02:25.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.631+0000 7f88b1600640 10 client.19470 handle_command_reply: tid=0 2024-04-24T09:02:25.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.631+0000 7f88cd322740 1 client.19470 shutdown 2024-04-24T09:02:25.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.631+0000 7f88cd322740 1 -- 172.21.15.83:0/3791563245 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bdca21eb30 msgr2=0x55bdca219120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:25.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.631+0000 7f88cd322740 1 --2- 172.21.15.83:0/3791563245 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bdca21eb30 0x55bdca219120 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f88bc0bd490 tx=0x7f88bc009980 comp rx=0 tx=0).stop 2024-04-24T09:02:25.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.631+0000 7f88cd322740 1 -- 172.21.15.83:0/3791563245 shutdown_connections 2024-04-24T09:02:25.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.631+0000 7f88cd322740 1 --2- 172.21.15.83:0/3791563245 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bdca21eb30 0x55bdca219120 unknown :-1 s=CLOSED pgs=1770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:25.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.631+0000 7f88cd322740 1 --2- 172.21.15.83:0/3791563245 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55bdca2958d0 0x55bdca297cf0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:25.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.631+0000 7f88cd322740 1 --2- 172.21.15.83:0/3791563245 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bdca214460 0x55bdca218bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:25.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.631+0000 7f88cd322740 1 --2- 172.21.15.83:0/3791563245 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bdca213b40 0x55bdca227200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:25.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.631+0000 7f88cd322740 1 -- 172.21.15.83:0/3791563245 >> 172.21.15.83:0/3791563245 conn(0x55bdca20fe60 msgr2=0x55bdca21c170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:02:25.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.631+0000 7f88b1600640 0 client.19470 ms_handle_reset on v2:172.21.15.138:6832/1157967658 2024-04-24T09:02:25.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.632+0000 7f88cd322740 1 -- 172.21.15.83:0/3791563245 shutdown_connections 2024-04-24T09:02:25.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.632+0000 7f88cd322740 1 -- 172.21.15.83:0/3791563245 wait complete. 2024-04-24T09:02:25.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.632+0000 7f88cd322740 20 client.19470 trim_cache size 0 max 16384 2024-04-24T09:02:25.636 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:02:25.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.632+0000 7f88c9200640 1 -- 172.21.15.83:0/1348375028 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f88a80a3da0 msgr2=0x7f88a80a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:25.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.632+0000 7f88c9200640 1 --2- 172.21.15.83:0/1348375028 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f88a80a3da0 0x7f88a80a6260 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f88b4005fd0 tx=0x7f88b4009350 comp rx=0 tx=0).stop 2024-04-24T09:02:25.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.632+0000 7f88c9200640 1 -- 172.21.15.83:0/1348375028 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88c4112eb0 msgr2=0x7f88c419eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:25.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.632+0000 7f88c9200640 1 --2- 172.21.15.83:0/1348375028 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88c4112eb0 0x7f88c419eae0 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f88bc02ae70 tx=0x7f88bc02f5d0 comp rx=0 tx=0).stop 2024-04-24T09:02:25.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.633+0000 7f88c9200640 1 -- 172.21.15.83:0/1348375028 shutdown_connections 2024-04-24T09:02:25.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.633+0000 7f88c9200640 1 --2- 172.21.15.83:0/1348375028 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f88a80a3da0 0x7f88a80a6260 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:25.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.633+0000 7f88c9200640 1 --2- 172.21.15.83:0/1348375028 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88c4112eb0 0x7f88c419eae0 unknown :-1 s=CLOSED pgs=1766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:25.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.633+0000 7f88c9200640 1 --2- 172.21.15.83:0/1348375028 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88c4074cc0 0x7f88c41a4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:25.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.633+0000 7f88c9200640 1 --2- 172.21.15.83:0/1348375028 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f88c40743a0 0x7f88c41a48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:25.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.633+0000 7f88c9200640 1 -- 172.21.15.83:0/1348375028 >> 172.21.15.83:0/1348375028 conn(0x7f88c40fd9d0 msgr2=0x7f88c40ff2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:02:25.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.633+0000 7f88c9200640 1 -- 172.21.15.83:0/1348375028 shutdown_connections 2024-04-24T09:02:25.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.633+0000 7f88c9200640 1 -- 172.21.15.83:0/1348375028 wait complete. 2024-04-24T09:02:25.649 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T09:02:25.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.856+0000 7ff311200640 1 -- 172.21.15.83:0/1490429870 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff30c10d500 msgr2=0x7ff30c10d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:25.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.856+0000 7ff311200640 1 --2- 172.21.15.83:0/1490429870 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff30c10d500 0x7ff30c10d8e0 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc009930 tx=0x7ff2fc02a420 comp rx=0 tx=0).stop 2024-04-24T09:02:25.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.856+0000 7ff311200640 1 -- 172.21.15.83:0/1490429870 shutdown_connections 2024-04-24T09:02:25.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.856+0000 7ff311200640 1 --2- 172.21.15.83:0/1490429870 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff30c108f40 0x7ff30c1153b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:25.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.856+0000 7ff311200640 1 --2- 172.21.15.83:0/1490429870 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff30c108580 0x7ff30c108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:25.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.856+0000 7ff311200640 1 --2- 172.21.15.83:0/1490429870 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff30c10d500 0x7ff30c10d8e0 unknown :-1 s=CLOSED pgs=1771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:25.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.856+0000 7ff311200640 1 -- 172.21.15.83:0/1490429870 >> 172.21.15.83:0/1490429870 conn(0x7ff30c077e20 msgr2=0x7ff30c106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:02:25.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.856+0000 7ff311200640 1 -- 172.21.15.83:0/1490429870 shutdown_connections 2024-04-24T09:02:25.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.856+0000 7ff311200640 1 -- 172.21.15.83:0/1490429870 wait complete. 2024-04-24T09:02:25.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.857+0000 7ff311200640 1 Processor -- start 2024-04-24T09:02:25.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.858+0000 7ff311200640 1 -- start start 2024-04-24T09:02:25.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.858+0000 7ff311200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff30c108580 0x7ff30c1a4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:25.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.858+0000 7ff311200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff30c108f40 0x7ff30c1a4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:25.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.858+0000 7ff311200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff30c10d500 0x7ff30c19eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:25.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.858+0000 7ff311200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff30c118760 con 0x7ff30c10d500 2024-04-24T09:02:25.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.858+0000 7ff311200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff30c1185e0 con 0x7ff30c108f40 2024-04-24T09:02:25.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.858+0000 7ff311200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff30c1188e0 con 0x7ff30c108580 2024-04-24T09:02:25.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.858+0000 7ff30be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff30c10d500 0x7ff30c19eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:25.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.858+0000 7ff30be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff30c10d500 0x7ff30c19eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51356/0 (socket says 172.21.15.83:51356) 2024-04-24T09:02:25.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.858+0000 7ff30be00640 1 -- 172.21.15.83:0/147760057 learned_addr learned my addr 172.21.15.83:0/147760057 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:02:25.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.858+0000 7ff30b400640 1 --2- 172.21.15.83:0/147760057 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff30c108580 0x7ff30c1a4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:25.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.859+0000 7ff30aa00640 1 --2- 172.21.15.83:0/147760057 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff30c108f40 0x7ff30c1a4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:25.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.859+0000 7ff30be00640 1 -- 172.21.15.83:0/147760057 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff30c108580 msgr2=0x7ff30c1a4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:25.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.859+0000 7ff30be00640 1 --2- 172.21.15.83:0/147760057 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff30c108580 0x7ff30c1a4940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:25.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.859+0000 7ff30be00640 1 -- 172.21.15.83:0/147760057 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff30c108f40 msgr2=0x7ff30c1a4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:25.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.859+0000 7ff30be00640 1 --2- 172.21.15.83:0/147760057 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff30c108f40 0x7ff30c1a4e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:25.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.859+0000 7ff30be00640 1 -- 172.21.15.83:0/147760057 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff30c19f2c0 con 0x7ff30c10d500 2024-04-24T09:02:25.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.859+0000 7ff30aa00640 1 --2- 172.21.15.83:0/147760057 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff30c108f40 0x7ff30c1a4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:02:25.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.859+0000 7ff30be00640 1 --2- 172.21.15.83:0/147760057 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff30c10d500 0x7ff30c19eb40 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7ff30400cab0 tx=0x7ff30400cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:02:25.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.859+0000 7ff303e00640 1 -- 172.21.15.83:0/147760057 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff30400adc0 con 0x7ff30c10d500 2024-04-24T09:02:25.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.859+0000 7ff303e00640 1 -- 172.21.15.83:0/147760057 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff30400f900 con 0x7ff30c10d500 2024-04-24T09:02:25.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.859+0000 7ff311200640 1 -- 172.21.15.83:0/147760057 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff30c19f5b0 con 0x7ff30c10d500 2024-04-24T09:02:25.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.860+0000 7ff303e00640 1 -- 172.21.15.83:0/147760057 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff304018950 con 0x7ff30c10d500 2024-04-24T09:02:25.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.860+0000 7ff311200640 1 -- 172.21.15.83:0/147760057 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff30c1ab800 con 0x7ff30c10d500 2024-04-24T09:02:25.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.863+0000 7ff315793740 1 Processor -- start 2024-04-24T09:02:25.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.863+0000 7ff303e00640 1 -- 172.21.15.83:0/147760057 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff30400fab0 con 0x7ff30c10d500 2024-04-24T09:02:25.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.863+0000 7ff315793740 1 -- start start 2024-04-24T09:02:25.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.864+0000 7ff315793740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c5f5aab40 0x563c5f5aaf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:25.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.864+0000 7ff315793740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c5f5ab460 0x563c5f5b55f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:25.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.864+0000 7ff315793740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c5f5b5b30 0x563c5f5b7f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:25.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.864+0000 7ff315793740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x563c5f5101d0 con 0x563c5f5ab460 2024-04-24T09:02:25.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.864+0000 7ff315793740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563c5f4b6450 con 0x563c5f5b5b30 2024-04-24T09:02:25.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.864+0000 7ff315793740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x563c5f50c190 con 0x563c5f5aab40 2024-04-24T09:02:25.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.864+0000 7ff30b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c5f5ab460 0x563c5f5b55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:25.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.864+0000 7ff30b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c5f5ab460 0x563c5f5b55f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51370/0 (socket says 172.21.15.83:51370) 2024-04-24T09:02:25.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.864+0000 7ff30b400640 1 -- 172.21.15.83:0/2527775277 learned_addr learned my addr 172.21.15.83:0/2527775277 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:02:25.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.865+0000 7ff30aa00640 1 --2- 172.21.15.83:0/2527775277 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c5f5aab40 0x563c5f5aaf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:25.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.865+0000 7ff30aa00640 1 --2- 172.21.15.83:0/2527775277 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c5f5b5b30 0x563c5f5b7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:25.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.865+0000 7ff30b400640 1 -- 172.21.15.83:0/2527775277 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c5f5aab40 msgr2=0x563c5f5aaf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:25.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.865+0000 7ff30b400640 1 --2- 172.21.15.83:0/2527775277 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c5f5aab40 0x563c5f5aaf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:25.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.865+0000 7ff30b400640 1 -- 172.21.15.83:0/2527775277 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c5f5b5b30 msgr2=0x563c5f5b7f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:25.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.865+0000 7ff30b400640 1 --2- 172.21.15.83:0/2527775277 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c5f5b5b30 0x563c5f5b7f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:25.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.865+0000 7ff30b400640 1 -- 172.21.15.83:0/2527775277 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c5f573310 con 0x563c5f5ab460 2024-04-24T09:02:25.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.865+0000 7ff303e00640 1 --2- 172.21.15.83:0/147760057 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff2f00a3d20 0x7ff2f00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:25.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.865+0000 7ff303e00640 1 -- 172.21.15.83:0/147760057 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9583+0+0 (secure 0 0 0) 0x7ff3040d0ff0 con 0x7ff30c10d500 2024-04-24T09:02:25.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.865+0000 7ff30aa00640 1 --2- 172.21.15.83:0/147760057 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff2f00a3d20 0x7ff2f00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:25.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.865+0000 7ff30b400640 1 --2- 172.21.15.83:0/2527775277 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c5f5ab460 0x563c5f5b55f0 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc031440 tx=0x7ff2fc032cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f6b3e1f55cc7243 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:02:25.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.866+0000 7ff300c00640 1 -- 172.21.15.83:0/2527775277 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2fc03f070 con 0x563c5f5ab460 2024-04-24T09:02:25.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.866+0000 7ff300c00640 1 -- 172.21.15.83:0/2527775277 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff2fc0316a0 con 0x563c5f5ab460 2024-04-24T09:02:25.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.866+0000 7ff300c00640 1 -- 172.21.15.83:0/2527775277 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2fc035070 con 0x563c5f5ab460 2024-04-24T09:02:25.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.867+0000 7ff315793740 1 -- 172.21.15.83:0/2527775277 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c5f5ab460 msgr2=0x563c5f5b55f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:25.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.867+0000 7ff315793740 1 --2- 172.21.15.83:0/2527775277 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c5f5ab460 0x563c5f5b55f0 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc031440 tx=0x7ff2fc032cc0 comp rx=0 tx=0).stop 2024-04-24T09:02:25.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.867+0000 7ff30aa00640 1 --2- 172.21.15.83:0/147760057 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff2f00a3d20 0x7ff2f00a61e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x563c5f5a7ff0 tx=0x7ff2f8005850 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:02:25.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.867+0000 7ff315793740 1 -- 172.21.15.83:0/2527775277 shutdown_connections 2024-04-24T09:02:25.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.867+0000 7ff315793740 1 --2- 172.21.15.83:0/2527775277 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c5f5b5b30 0x563c5f5b7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:25.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.867+0000 7ff315793740 1 --2- 172.21.15.83:0/2527775277 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c5f5ab460 0x563c5f5b55f0 unknown :-1 s=CLOSED pgs=1773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:25.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.867+0000 7ff315793740 1 --2- 172.21.15.83:0/2527775277 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c5f5aab40 0x563c5f5aaf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:25.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.867+0000 7ff315793740 1 -- 172.21.15.83:0/2527775277 >> 172.21.15.83:0/2527775277 conn(0x563c5f5a6e60 msgr2=0x563c5f5a7240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:02:25.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.867+0000 7ff315793740 1 -- 172.21.15.83:0/2527775277 shutdown_connections 2024-04-24T09:02:25.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.867+0000 7ff315793740 1 -- 172.21.15.83:0/2527775277 wait complete. 2024-04-24T09:02:25.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.867+0000 7ff315793740 1 -- start start 2024-04-24T09:02:25.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.868+0000 7ff315793740 1 Processor -- start 2024-04-24T09:02:25.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.868+0000 7ff315793740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c5f5aab40 0x563c5f5b5590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:25.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.868+0000 7ff315793740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c5f5ab460 0x563c5f5b2880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:25.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.868+0000 7ff315793740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c5f5b5b30 0x563c5f5b2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:25.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.868+0000 7ff315793740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x563c5f5b8960 con 0x563c5f5b5b30 2024-04-24T09:02:25.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.868+0000 7ff315793740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563c5f4c9810 con 0x563c5f5ab460 2024-04-24T09:02:25.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.868+0000 7ff315793740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x563c5f5b8ae0 con 0x563c5f5aab40 2024-04-24T09:02:25.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.868+0000 7ff315793740 10 client.0 _ll_register_callbacks cb 0x563c5f497140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:02:25.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.868+0000 7ff30be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c5f5b5b30 0x563c5f5b2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:25.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.869+0000 7ff30be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c5f5b5b30 0x563c5f5b2fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51372/0 (socket says 172.21.15.83:51372) 2024-04-24T09:02:25.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.869+0000 7ff30be00640 1 -- 172.21.15.83:0/851907105 learned_addr learned my addr 172.21.15.83:0/851907105 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:02:25.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.869+0000 7ff30b400640 1 --2- 172.21.15.83:0/851907105 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c5f5aab40 0x563c5f5b5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:25.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.869+0000 7ff30aa00640 1 --2- 172.21.15.83:0/851907105 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c5f5ab460 0x563c5f5b2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:25.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.869+0000 7ff30be00640 1 -- 172.21.15.83:0/851907105 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c5f5aab40 msgr2=0x563c5f5b5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:25.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.869+0000 7ff30be00640 1 --2- 172.21.15.83:0/851907105 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c5f5aab40 0x563c5f5b5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:25.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.869+0000 7ff30be00640 1 -- 172.21.15.83:0/851907105 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c5f5ab460 msgr2=0x563c5f5b2880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:25.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.869+0000 7ff30be00640 1 --2- 172.21.15.83:0/851907105 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c5f5ab460 0x563c5f5b2880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:25.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.869+0000 7ff30be00640 1 -- 172.21.15.83:0/851907105 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c5f4da3f0 con 0x563c5f5b5b30 2024-04-24T09:02:25.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.869+0000 7ff30aa00640 1 --2- 172.21.15.83:0/851907105 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c5f5ab460 0x563c5f5b2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:02:25.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.869+0000 7ff30be00640 1 --2- 172.21.15.83:0/851907105 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c5f5b5b30 0x563c5f5b2fa0 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7ff304099540 tx=0x7ff30400fee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d8de0a4a4f8c542 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:02:25.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.869+0000 7ff2f5600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:02:25.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.869+0000 7ff2f5600640 1 -- 172.21.15.83:0/851907105 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3040b6060 con 0x563c5f5b5b30 2024-04-24T09:02:25.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.869+0000 7ff2f5600640 1 -- 172.21.15.83:0/851907105 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9583+0+0 (secure 0 0 0) 0x7ff3040a5350 con 0x563c5f5b5b30 2024-04-24T09:02:25.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.869+0000 7ff315793740 1 -- 172.21.15.83:0/851907105 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2fc0316a0 con 0x563c5f5b5b30 2024-04-24T09:02:25.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.870+0000 7ff2f5600640 1 -- 172.21.15.83:0/851907105 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7ff3040a0f20 con 0x563c5f5b5b30 2024-04-24T09:02:25.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.870+0000 7ff315793740 10 client.19482 fetch_fsmap learned FSMap version 41 2024-04-24T09:02:25.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.870+0000 7ff315793740 1 -- 172.21.15.83:0/851907105 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x563c5f2223f0 con 0x563c5f5b5b30 2024-04-24T09:02:25.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.870+0000 7ff2f5600640 1 -- 172.21.15.83:0/851907105 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff2fc0316a0 con 0x563c5f5b5b30 2024-04-24T09:02:25.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.870+0000 7ff2f5600640 1 -- 172.21.15.83:0/851907105 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7ff3040b7020 con 0x563c5f5b5b30 2024-04-24T09:02:25.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.870+0000 7ff315793740 10 client.19482 fetch_fsmap finished waiting for FSMap version 41 2024-04-24T09:02:25.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.870+0000 7ff315793740 10 client.19482 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-24T09:02:25.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.870+0000 7ff315793740 20 client.19482 populate_metadata read hostname 'smithi083' 2024-04-24T09:02:25.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.870+0000 7ff315793740 1 --2- 172.21.15.83:0/851907105 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x563c5f5c5e70 0x563c5f5c8260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:25.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.870+0000 7ff315793740 4 client.19482 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:02:25.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.870+0000 7ff315793740 1 -- 172.21.15.83:0/851907105 --> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563c5f51dbf0 con 0x563c5f5c5e70 2024-04-24T09:02:25.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.871+0000 7ff30b400640 1 --2- 172.21.15.83:0/851907105 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x563c5f5c5e70 0x563c5f5c8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:25.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.872+0000 7ff30b400640 1 --2- 172.21.15.83:0/851907105 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x563c5f5c5e70 0x563c5f5c8260 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=170a6b437de6e54c server_cookie=d39ba8f35276da6f in_seq=0 out_seq=0 2024-04-24T09:02:25.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.872+0000 7ff2f5600640 10 client.19482 ms_handle_connect on v2:172.21.15.138:6834/1909219758 2024-04-24T09:02:25.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.874+0000 7ff2f5600640 1 -- 172.21.15.83:0/851907105 <== mds.2 v2:172.21.15.138:6834/1909219758 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563c5f51dbf0 con 0x563c5f5c5e70 2024-04-24T09:02:25.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.874+0000 7ff2f5600640 10 client.19482 handle_command_reply: tid=0 2024-04-24T09:02:25.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.874+0000 7ff315793740 1 client.19482 shutdown 2024-04-24T09:02:25.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.874+0000 7ff315793740 1 -- 172.21.15.83:0/851907105 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c5f5b5b30 msgr2=0x563c5f5b2fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:25.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.874+0000 7ff315793740 1 --2- 172.21.15.83:0/851907105 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c5f5b5b30 0x563c5f5b2fa0 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7ff304099540 tx=0x7ff30400fee0 comp rx=0 tx=0).stop 2024-04-24T09:02:25.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.874+0000 7ff315793740 1 -- 172.21.15.83:0/851907105 shutdown_connections 2024-04-24T09:02:25.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.874+0000 7ff315793740 1 --2- 172.21.15.83:0/851907105 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x563c5f5c5e70 0x563c5f5c8260 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:25.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.874+0000 7ff315793740 1 --2- 172.21.15.83:0/851907105 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c5f5b5b30 0x563c5f5b2fa0 unknown :-1 s=CLOSED pgs=1774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:25.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.874+0000 7ff315793740 1 --2- 172.21.15.83:0/851907105 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c5f5ab460 0x563c5f5b2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:25.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.874+0000 7ff315793740 1 --2- 172.21.15.83:0/851907105 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c5f5aab40 0x563c5f5b5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:25.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.874+0000 7ff315793740 1 -- 172.21.15.83:0/851907105 >> 172.21.15.83:0/851907105 conn(0x563c5f5a6e60 msgr2=0x563c5f5acb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:02:25.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.874+0000 7ff2f5600640 0 client.19482 ms_handle_reset on v2:172.21.15.138:6834/1909219758 2024-04-24T09:02:25.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.875+0000 7ff315793740 1 -- 172.21.15.83:0/851907105 shutdown_connections 2024-04-24T09:02:25.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.875+0000 7ff315793740 1 -- 172.21.15.83:0/851907105 wait complete. 2024-04-24T09:02:25.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.875+0000 7ff315793740 20 client.19482 trim_cache size 0 max 16384 2024-04-24T09:02:25.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.892+0000 7ff315793740 1 Processor -- start 2024-04-24T09:02:25.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.892+0000 7ff315793740 1 -- start start 2024-04-24T09:02:25.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.893+0000 7ff315793740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c5f5aab40 0x563c5f5c1880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:25.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.893+0000 7ff315793740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c5f5ab460 0x563c5f5b1860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:25.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.893+0000 7ff315793740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c5f5b5b30 0x563c5f5b50c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:25.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.893+0000 7ff315793740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff30c1b5970 con 0x563c5f5aab40 2024-04-24T09:02:25.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.893+0000 7ff315793740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff30c1b75d0 con 0x563c5f5b5b30 2024-04-24T09:02:25.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.893+0000 7ff315793740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff30c1b6040 con 0x563c5f5ab460 2024-04-24T09:02:25.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.893+0000 7ff30b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c5f5aab40 0x563c5f5c1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:25.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.893+0000 7ff30b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c5f5aab40 0x563c5f5c1880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51384/0 (socket says 172.21.15.83:51384) 2024-04-24T09:02:25.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.893+0000 7ff30b400640 1 -- 172.21.15.83:0/4241899749 learned_addr learned my addr 172.21.15.83:0/4241899749 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:02:25.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.893+0000 7ff30aa00640 1 --2- 172.21.15.83:0/4241899749 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c5f5ab460 0x563c5f5b1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:25.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.893+0000 7ff30be00640 1 --2- 172.21.15.83:0/4241899749 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c5f5b5b30 0x563c5f5b50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:25.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.894+0000 7ff30b400640 1 -- 172.21.15.83:0/4241899749 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c5f5ab460 msgr2=0x563c5f5b1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:25.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.894+0000 7ff30b400640 1 --2- 172.21.15.83:0/4241899749 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c5f5ab460 0x563c5f5b1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:25.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.894+0000 7ff30b400640 1 -- 172.21.15.83:0/4241899749 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c5f5b5b30 msgr2=0x563c5f5b50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:25.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.894+0000 7ff30b400640 1 --2- 172.21.15.83:0/4241899749 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c5f5b5b30 0x563c5f5b50c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:25.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.894+0000 7ff30b400640 1 -- 172.21.15.83:0/4241899749 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c5f5ab290 con 0x563c5f5aab40 2024-04-24T09:02:25.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.894+0000 7ff30b400640 1 --2- 172.21.15.83:0/4241899749 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c5f5aab40 0x563c5f5c1880 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc009930 tx=0x7ff2fc005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5940bf37eec7a74 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:02:25.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.894+0000 7ff2f7e00640 1 -- 172.21.15.83:0/4241899749 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2fc03f040 con 0x563c5f5aab40 2024-04-24T09:02:25.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.894+0000 7ff2f7e00640 1 -- 172.21.15.83:0/4241899749 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff2fc005050 con 0x563c5f5aab40 2024-04-24T09:02:25.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.894+0000 7ff2f7e00640 1 -- 172.21.15.83:0/4241899749 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2fc035040 con 0x563c5f5aab40 2024-04-24T09:02:25.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.895+0000 7ff315793740 1 -- 172.21.15.83:0/4241899749 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c5f5aab40 msgr2=0x563c5f5c1880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:25.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.895+0000 7ff315793740 1 --2- 172.21.15.83:0/4241899749 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c5f5aab40 0x563c5f5c1880 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc009930 tx=0x7ff2fc005fd0 comp rx=0 tx=0).stop 2024-04-24T09:02:25.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.895+0000 7ff315793740 1 -- 172.21.15.83:0/4241899749 shutdown_connections 2024-04-24T09:02:25.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.895+0000 7ff315793740 1 --2- 172.21.15.83:0/4241899749 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c5f5b5b30 0x563c5f5b50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:25.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.895+0000 7ff315793740 1 --2- 172.21.15.83:0/4241899749 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c5f5ab460 0x563c5f5b1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:25.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.895+0000 7ff315793740 1 --2- 172.21.15.83:0/4241899749 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c5f5aab40 0x563c5f5c1880 unknown :-1 s=CLOSED pgs=1775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:25.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.895+0000 7ff315793740 1 -- 172.21.15.83:0/4241899749 >> 172.21.15.83:0/4241899749 conn(0x563c5f5a6e60 msgr2=0x563c5f5b1150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:02:25.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.895+0000 7ff315793740 1 -- 172.21.15.83:0/4241899749 shutdown_connections 2024-04-24T09:02:25.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.895+0000 7ff315793740 1 -- 172.21.15.83:0/4241899749 wait complete. 2024-04-24T09:02:25.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.896+0000 7ff315793740 1 -- start start 2024-04-24T09:02:25.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.896+0000 7ff315793740 1 Processor -- start 2024-04-24T09:02:25.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.896+0000 7ff315793740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c5f5aab40 0x563c5f5b6860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:25.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.896+0000 7ff315793740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c5f5ab460 0x563c5f5b6da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:25.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.896+0000 7ff315793740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c5f5b5b30 0x563c5f5b72e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:25.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.896+0000 7ff315793740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff30c1b6ea0 con 0x563c5f5aab40 2024-04-24T09:02:25.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.896+0000 7ff315793740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff30c1b6770 con 0x563c5f5b5b30 2024-04-24T09:02:25.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.896+0000 7ff315793740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff30c1b3970 con 0x563c5f5ab460 2024-04-24T09:02:25.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.896+0000 7ff315793740 10 client.0 _ll_register_callbacks cb 0x7ff304007650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:02:25.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.897+0000 7ff30b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c5f5aab40 0x563c5f5b6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:25.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.897+0000 7ff30b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c5f5aab40 0x563c5f5b6860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51394/0 (socket says 172.21.15.83:51394) 2024-04-24T09:02:25.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.897+0000 7ff30b400640 1 -- 172.21.15.83:0/3162149114 learned_addr learned my addr 172.21.15.83:0/3162149114 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:02:25.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.897+0000 7ff30aa00640 1 --2- 172.21.15.83:0/3162149114 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c5f5ab460 0x563c5f5b6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:25.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.897+0000 7ff30b400640 1 -- 172.21.15.83:0/3162149114 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c5f5ab460 msgr2=0x563c5f5b6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:25.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.897+0000 7ff30be00640 1 --2- 172.21.15.83:0/3162149114 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c5f5b5b30 0x563c5f5b72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:25.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.897+0000 7ff30b400640 1 --2- 172.21.15.83:0/3162149114 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c5f5ab460 0x563c5f5b6da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:25.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.897+0000 7ff30b400640 1 -- 172.21.15.83:0/3162149114 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c5f5b5b30 msgr2=0x563c5f5b72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:25.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.897+0000 7ff30b400640 1 --2- 172.21.15.83:0/3162149114 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c5f5b5b30 0x563c5f5b72e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:25.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.897+0000 7ff30b400640 1 -- 172.21.15.83:0/3162149114 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c5f2ec7d0 con 0x563c5f5aab40 2024-04-24T09:02:25.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.897+0000 7ff30b400640 1 --2- 172.21.15.83:0/3162149114 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c5f5aab40 0x563c5f5b6860 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc031a60 tx=0x7ff2fc031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1271c9c08f2227b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:02:25.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.897+0000 7ff2f5600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:02:25.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.898+0000 7ff2f5600640 1 -- 172.21.15.83:0/3162149114 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2fc05a060 con 0x563c5f5aab40 2024-04-24T09:02:25.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.898+0000 7ff315793740 1 -- 172.21.15.83:0/3162149114 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2fc005050 con 0x563c5f5aab40 2024-04-24T09:02:25.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.898+0000 7ff2f5600640 1 -- 172.21.15.83:0/3162149114 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9583+0+0 (secure 0 0 0) 0x7ff2fc004250 con 0x563c5f5aab40 2024-04-24T09:02:25.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.898+0000 7ff2f5600640 1 -- 172.21.15.83:0/3162149114 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7ff2fc061050 con 0x563c5f5aab40 2024-04-24T09:02:25.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.898+0000 7ff315793740 10 client.19488 fetch_fsmap learned FSMap version 41 2024-04-24T09:02:25.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.898+0000 7ff315793740 1 -- 172.21.15.83:0/3162149114 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x563c5f62c510 con 0x563c5f5aab40 2024-04-24T09:02:25.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.898+0000 7ff2f5600640 1 -- 172.21.15.83:0/3162149114 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff2fc005050 con 0x563c5f5aab40 2024-04-24T09:02:25.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.898+0000 7ff2f5600640 1 -- 172.21.15.83:0/3162149114 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7ff2fc004cc0 con 0x563c5f5aab40 2024-04-24T09:02:25.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.898+0000 7ff315793740 10 client.19488 fetch_fsmap finished waiting for FSMap version 41 2024-04-24T09:02:25.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.898+0000 7ff315793740 10 client.19488 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-24T09:02:25.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.898+0000 7ff315793740 20 client.19488 populate_metadata read hostname 'smithi083' 2024-04-24T09:02:25.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.898+0000 7ff315793740 1 --2- 172.21.15.83:0/3162149114 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x563c5f62c7a0 0x563c5f62eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:25.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.898+0000 7ff315793740 4 client.19488 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:02:25.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.898+0000 7ff315793740 1 -- 172.21.15.83:0/3162149114 --> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563c5f517a60 con 0x563c5f62c7a0 2024-04-24T09:02:25.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.899+0000 7ff30aa00640 1 --2- 172.21.15.83:0/3162149114 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x563c5f62c7a0 0x563c5f62eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:25.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.900+0000 7ff2f5600640 10 client.19488 ms_handle_connect on v2:172.21.15.138:6834/1909219758 2024-04-24T09:02:25.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.900+0000 7ff30aa00640 1 --2- 172.21.15.83:0/3162149114 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x563c5f62c7a0 0x563c5f62eb90 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ea61ceebcd2c1520 server_cookie=8e0115a71a67147a in_seq=0 out_seq=0 2024-04-24T09:02:25.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.901+0000 7ff2f5600640 1 -- 172.21.15.83:0/3162149114 <== mds.2 v2:172.21.15.138:6834/1909219758 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563c5f517a60 con 0x563c5f62c7a0 2024-04-24T09:02:25.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.901+0000 7ff2f5600640 10 client.19488 handle_command_reply: tid=0 2024-04-24T09:02:25.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.901+0000 7ff315793740 1 client.19488 shutdown 2024-04-24T09:02:25.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.901+0000 7ff315793740 1 -- 172.21.15.83:0/3162149114 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c5f5aab40 msgr2=0x563c5f5b6860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:25.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.901+0000 7ff315793740 1 --2- 172.21.15.83:0/3162149114 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c5f5aab40 0x563c5f5b6860 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc031a60 tx=0x7ff2fc031400 comp rx=0 tx=0).stop 2024-04-24T09:02:25.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.901+0000 7ff315793740 1 -- 172.21.15.83:0/3162149114 shutdown_connections 2024-04-24T09:02:25.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.901+0000 7ff315793740 1 --2- 172.21.15.83:0/3162149114 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x563c5f62c7a0 0x563c5f62eb90 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:25.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.901+0000 7ff315793740 1 --2- 172.21.15.83:0/3162149114 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c5f5b5b30 0x563c5f5b72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:25.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.901+0000 7ff315793740 1 --2- 172.21.15.83:0/3162149114 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c5f5ab460 0x563c5f5b6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:25.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.901+0000 7ff2f5600640 0 client.19488 ms_handle_reset on v2:172.21.15.138:6834/1909219758 2024-04-24T09:02:25.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.901+0000 7ff315793740 1 --2- 172.21.15.83:0/3162149114 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c5f5aab40 0x563c5f5b6860 unknown :-1 s=CLOSED pgs=1776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:25.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.901+0000 7ff315793740 1 -- 172.21.15.83:0/3162149114 >> 172.21.15.83:0/3162149114 conn(0x563c5f5a6e60 msgr2=0x563c5f5a56c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:02:25.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.901+0000 7ff315793740 1 -- 172.21.15.83:0/3162149114 shutdown_connections 2024-04-24T09:02:25.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.901+0000 7ff315793740 1 -- 172.21.15.83:0/3162149114 wait complete. 2024-04-24T09:02:25.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.901+0000 7ff315793740 20 client.19488 trim_cache size 0 max 16384 2024-04-24T09:02:25.903 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:02:25.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.902+0000 7ff311200640 1 -- 172.21.15.83:0/147760057 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff2f00a3d20 msgr2=0x7ff2f00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:25.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.902+0000 7ff311200640 1 --2- 172.21.15.83:0/147760057 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff2f00a3d20 0x7ff2f00a61e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x563c5f5a7ff0 tx=0x7ff2f8005850 comp rx=0 tx=0).stop 2024-04-24T09:02:25.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.902+0000 7ff311200640 1 -- 172.21.15.83:0/147760057 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff30c10d500 msgr2=0x7ff30c19eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:25.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.902+0000 7ff311200640 1 --2- 172.21.15.83:0/147760057 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff30c10d500 0x7ff30c19eb40 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7ff30400cab0 tx=0x7ff30400cf70 comp rx=0 tx=0).stop 2024-04-24T09:02:25.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.902+0000 7ff311200640 1 -- 172.21.15.83:0/147760057 shutdown_connections 2024-04-24T09:02:25.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.902+0000 7ff311200640 1 --2- 172.21.15.83:0/147760057 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff2f00a3d20 0x7ff2f00a61e0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:25.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.902+0000 7ff311200640 1 --2- 172.21.15.83:0/147760057 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff30c10d500 0x7ff30c19eb40 unknown :-1 s=CLOSED pgs=1772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:25.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.902+0000 7ff311200640 1 --2- 172.21.15.83:0/147760057 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff30c108f40 0x7ff30c1a4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:25.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.902+0000 7ff311200640 1 --2- 172.21.15.83:0/147760057 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff30c108580 0x7ff30c1a4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:25.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.902+0000 7ff311200640 1 -- 172.21.15.83:0/147760057 >> 172.21.15.83:0/147760057 conn(0x7ff30c077e20 msgr2=0x7ff30c078480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:02:25.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.902+0000 7ff311200640 1 -- 172.21.15.83:0/147760057 shutdown_connections 2024-04-24T09:02:25.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:25.902+0000 7ff311200640 1 -- 172.21.15.83:0/147760057 wait complete. 2024-04-24T09:02:25.917 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:02:25 smithi083 ceph-mon[27907]: pgmap v339: 129 pgs: 129 active+clean; 60 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 273 KiB/s wr, 46 op/s 2024-04-24T09:02:25.917 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:02:25 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/3071260272' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:02:25.920 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T09:02:26.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.136+0000 7ff03f200640 1 -- 172.21.15.83:0/2363663723 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff040103980 msgr2=0x7ff04010be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:26.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.136+0000 7ff03f200640 1 --2- 172.21.15.83:0/2363663723 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff040103980 0x7ff04010be60 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7ff02c009900 tx=0x7ff02c02a480 comp rx=0 tx=0).stop 2024-04-24T09:02:26.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.136+0000 7ff03f200640 1 -- 172.21.15.83:0/2363663723 shutdown_connections 2024-04-24T09:02:26.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.136+0000 7ff03f200640 1 --2- 172.21.15.83:0/2363663723 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff04010c480 0x7ff04010e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:26.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.136+0000 7ff03f200640 1 --2- 172.21.15.83:0/2363663723 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff040103980 0x7ff04010be60 unknown :-1 s=CLOSED pgs=1777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:26.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.136+0000 7ff03f200640 1 --2- 172.21.15.83:0/2363663723 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff040103060 0x7ff040103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:26.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.136+0000 7ff03f200640 1 -- 172.21.15.83:0/2363663723 >> 172.21.15.83:0/2363663723 conn(0x7ff0400fc880 msgr2=0x7ff0400feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:02:26.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.136+0000 7ff03f200640 1 -- 172.21.15.83:0/2363663723 shutdown_connections 2024-04-24T09:02:26.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.136+0000 7ff03f200640 1 -- 172.21.15.83:0/2363663723 wait complete. 2024-04-24T09:02:26.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.137+0000 7ff03f200640 1 Processor -- start 2024-04-24T09:02:26.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.137+0000 7ff03f200640 1 -- start start 2024-04-24T09:02:26.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.138+0000 7ff03f200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff040103060 0x7ff0401a05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:26.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.138+0000 7ff03f200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff040103980 0x7ff0401a0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:26.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.138+0000 7ff03f200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff04010c480 0x7ff04019a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:26.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.138+0000 7ff03f200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff0401120f0 con 0x7ff040103980 2024-04-24T09:02:26.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.138+0000 7ff03f200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff040111f70 con 0x7ff04010c480 2024-04-24T09:02:26.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.138+0000 7ff03f200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff040112270 con 0x7ff040103060 2024-04-24T09:02:26.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.138+0000 7ff03d400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff040103980 0x7ff0401a0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:26.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.138+0000 7ff03d400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff040103980 0x7ff0401a0ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51410/0 (socket says 172.21.15.83:51410) 2024-04-24T09:02:26.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.138+0000 7ff03d400640 1 -- 172.21.15.83:0/3053879695 learned_addr learned my addr 172.21.15.83:0/3053879695 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:02:26.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.138+0000 7ff03de00640 1 --2- 172.21.15.83:0/3053879695 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff040103060 0x7ff0401a05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:26.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.138+0000 7ff03e800640 1 --2- 172.21.15.83:0/3053879695 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff04010c480 0x7ff04019a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:26.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.139+0000 7ff03d400640 1 -- 172.21.15.83:0/3053879695 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff040103060 msgr2=0x7ff0401a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:26.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.139+0000 7ff03d400640 1 --2- 172.21.15.83:0/3053879695 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff040103060 0x7ff0401a05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:26.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.139+0000 7ff03d400640 1 -- 172.21.15.83:0/3053879695 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff04010c480 msgr2=0x7ff04019a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:26.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.139+0000 7ff03d400640 1 --2- 172.21.15.83:0/3053879695 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff04010c480 0x7ff04019a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:26.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.139+0000 7ff03d400640 1 -- 172.21.15.83:0/3053879695 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff04019af70 con 0x7ff040103980 2024-04-24T09:02:26.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.139+0000 7ff03d400640 1 --2- 172.21.15.83:0/3053879695 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff040103980 0x7ff0401a0ae0 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7ff02c006010 tx=0x7ff02c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:02:26.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.139+0000 7ff036a00640 1 -- 172.21.15.83:0/3053879695 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff02c03f030 con 0x7ff040103980 2024-04-24T09:02:26.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.139+0000 7ff036a00640 1 -- 172.21.15.83:0/3053879695 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff02c003800 con 0x7ff040103980 2024-04-24T09:02:26.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.140+0000 7ff03f200640 1 -- 172.21.15.83:0/3053879695 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff04019b200 con 0x7ff040103980 2024-04-24T09:02:26.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.140+0000 7ff036a00640 1 -- 172.21.15.83:0/3053879695 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff02c035070 con 0x7ff040103980 2024-04-24T09:02:26.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.140+0000 7ff03f200640 1 -- 172.21.15.83:0/3053879695 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0401a7340 con 0x7ff040103980 2024-04-24T09:02:26.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.143+0000 7ff047e10740 1 Processor -- start 2024-04-24T09:02:26.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.143+0000 7ff036a00640 1 -- 172.21.15.83:0/3053879695 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff02c031af0 con 0x7ff040103980 2024-04-24T09:02:26.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.143+0000 7ff047e10740 1 -- start start 2024-04-24T09:02:26.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.144+0000 7ff047e10740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5605b9c86b40 0x5605b9c86f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:26.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.144+0000 7ff047e10740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605b9c87460 0x5605b9c915f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:26.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.144+0000 7ff047e10740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5605b9c91b30 0x5605b9c93f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:26.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.144+0000 7ff047e10740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5605b9bec1d0 con 0x5605b9c91b30 2024-04-24T09:02:26.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.144+0000 7ff047e10740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5605b9b92450 con 0x5605b9c87460 2024-04-24T09:02:26.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.144+0000 7ff047e10740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5605b9be8190 con 0x5605b9c86b40 2024-04-24T09:02:26.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.144+0000 7ff03d400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5605b9c91b30 0x5605b9c93f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:26.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.144+0000 7ff03d400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5605b9c91b30 0x5605b9c93f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51426/0 (socket says 172.21.15.83:51426) 2024-04-24T09:02:26.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.144+0000 7ff03d400640 1 -- 172.21.15.83:0/4209708945 learned_addr learned my addr 172.21.15.83:0/4209708945 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:02:26.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.144+0000 7ff03de00640 1 --2- 172.21.15.83:0/4209708945 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605b9c87460 0x5605b9c915f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:26.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.144+0000 7ff03e800640 1 --2- 172.21.15.83:0/4209708945 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5605b9c86b40 0x5605b9c86f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:26.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.144+0000 7ff03d400640 1 -- 172.21.15.83:0/4209708945 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5605b9c86b40 msgr2=0x5605b9c86f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:26.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.144+0000 7ff03d400640 1 --2- 172.21.15.83:0/4209708945 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5605b9c86b40 0x5605b9c86f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:26.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.145+0000 7ff03d400640 1 -- 172.21.15.83:0/4209708945 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605b9c87460 msgr2=0x5605b9c915f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:26.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.145+0000 7ff03d400640 1 --2- 172.21.15.83:0/4209708945 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605b9c87460 0x5605b9c915f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:26.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.145+0000 7ff03d400640 1 -- 172.21.15.83:0/4209708945 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605b9c4f310 con 0x5605b9c91b30 2024-04-24T09:02:26.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.145+0000 7ff03e800640 1 --2- 172.21.15.83:0/4209708945 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5605b9c86b40 0x5605b9c86f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:02:26.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.145+0000 7ff03d400640 1 --2- 172.21.15.83:0/4209708945 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5605b9c91b30 0x5605b9c93f20 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7ff02c0ee800 tx=0x7ff02c0f7800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f41cf2fee0ba873f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:02:26.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.145+0000 7ff036a00640 1 --2- 172.21.15.83:0/3053879695 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff0240a3d20 0x7ff0240a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:26.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.145+0000 7ff02b400640 1 -- 172.21.15.83:0/4209708945 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff02c0b9380 con 0x5605b9c91b30 2024-04-24T09:02:26.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.145+0000 7ff036a00640 1 -- 172.21.15.83:0/3053879695 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9583+0+0 (secure 0 0 0) 0x7ff02c0f1560 con 0x7ff040103980 2024-04-24T09:02:26.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.145+0000 7ff02b400640 1 -- 172.21.15.83:0/4209708945 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff02c0fe030 con 0x5605b9c91b30 2024-04-24T09:02:26.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.145+0000 7ff02b400640 1 -- 172.21.15.83:0/4209708945 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff02c0c1fc0 con 0x5605b9c91b30 2024-04-24T09:02:26.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.145+0000 7ff03e800640 1 --2- 172.21.15.83:0/3053879695 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff0240a3d20 0x7ff0240a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:26.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.146+0000 7ff047e10740 1 -- 172.21.15.83:0/4209708945 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5605b9c91b30 msgr2=0x5605b9c93f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:26.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.146+0000 7ff047e10740 1 --2- 172.21.15.83:0/4209708945 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5605b9c91b30 0x5605b9c93f20 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7ff02c0ee800 tx=0x7ff02c0f7800 comp rx=0 tx=0).stop 2024-04-24T09:02:26.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.146+0000 7ff03e800640 1 --2- 172.21.15.83:0/3053879695 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff0240a3d20 0x7ff0240a61e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7ff03800ae10 tx=0x7ff03800a3c0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:02:26.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.146+0000 7ff047e10740 1 -- 172.21.15.83:0/4209708945 shutdown_connections 2024-04-24T09:02:26.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.146+0000 7ff047e10740 1 --2- 172.21.15.83:0/4209708945 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5605b9c91b30 0x5605b9c93f20 unknown :-1 s=CLOSED pgs=1779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:26.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.146+0000 7ff047e10740 1 --2- 172.21.15.83:0/4209708945 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605b9c87460 0x5605b9c915f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:26.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.146+0000 7ff047e10740 1 --2- 172.21.15.83:0/4209708945 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5605b9c86b40 0x5605b9c86f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:26.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.146+0000 7ff047e10740 1 -- 172.21.15.83:0/4209708945 >> 172.21.15.83:0/4209708945 conn(0x5605b9c82e60 msgr2=0x5605b9c83240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:02:26.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.146+0000 7ff047e10740 1 -- 172.21.15.83:0/4209708945 shutdown_connections 2024-04-24T09:02:26.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.147+0000 7ff047e10740 1 -- 172.21.15.83:0/4209708945 wait complete. 2024-04-24T09:02:26.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.147+0000 7ff047e10740 1 -- start start 2024-04-24T09:02:26.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.147+0000 7ff047e10740 1 Processor -- start 2024-04-24T09:02:26.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.148+0000 7ff047e10740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605b9c86b40 0x5605b9c91500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:26.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.148+0000 7ff047e10740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5605b9c87460 0x5605b9c8e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:26.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.148+0000 7ff047e10740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5605b9c91b30 0x5605b9c8ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:26.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.148+0000 7ff047e10740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5605b9c946e0 con 0x5605b9c87460 2024-04-24T09:02:26.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.148+0000 7ff047e10740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5605b9ba5810 con 0x5605b9c86b40 2024-04-24T09:02:26.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.148+0000 7ff047e10740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5605b9c94860 con 0x5605b9c91b30 2024-04-24T09:02:26.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.148+0000 7ff047e10740 10 client.0 _ll_register_callbacks cb 0x5605b9b73140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:02:26.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.148+0000 7ff03d400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5605b9c87460 0x5605b9c8e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:26.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.148+0000 7ff03d400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5605b9c87460 0x5605b9c8e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51428/0 (socket says 172.21.15.83:51428) 2024-04-24T09:02:26.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.148+0000 7ff03d400640 1 -- 172.21.15.83:0/1431887432 learned_addr learned my addr 172.21.15.83:0/1431887432 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:02:26.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.148+0000 7ff03de00640 1 --2- 172.21.15.83:0/1431887432 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605b9c86b40 0x5605b9c91500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:26.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.148+0000 7ff03e800640 1 --2- 172.21.15.83:0/1431887432 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5605b9c91b30 0x5605b9c8ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:26.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.149+0000 7ff03d400640 1 -- 172.21.15.83:0/1431887432 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5605b9c91b30 msgr2=0x5605b9c8ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:26.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.149+0000 7ff03d400640 1 --2- 172.21.15.83:0/1431887432 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5605b9c91b30 0x5605b9c8ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:26.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.149+0000 7ff03d400640 1 -- 172.21.15.83:0/1431887432 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605b9c86b40 msgr2=0x5605b9c91500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:26.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.149+0000 7ff03d400640 1 --2- 172.21.15.83:0/1431887432 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605b9c86b40 0x5605b9c91500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:26.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.149+0000 7ff03d400640 1 -- 172.21.15.83:0/1431887432 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605b9bb63f0 con 0x5605b9c87460 2024-04-24T09:02:26.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.149+0000 7ff03d400640 1 --2- 172.21.15.83:0/1431887432 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5605b9c87460 0x5605b9c8e7f0 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7ff02c0c2580 tx=0x7ff02c02a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=831e18434d1f9e67 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:02:26.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.149+0000 7ff01fe00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:02:26.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.149+0000 7ff01fe00640 1 -- 172.21.15.83:0/1431887432 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff02c049070 con 0x5605b9c87460 2024-04-24T09:02:26.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.149+0000 7ff01fe00640 1 -- 172.21.15.83:0/1431887432 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9583+0+0 (secure 0 0 0) 0x7ff02c0bb3e0 con 0x5605b9c87460 2024-04-24T09:02:26.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.149+0000 7ff047e10740 1 -- 172.21.15.83:0/1431887432 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff02c0fe030 con 0x5605b9c87460 2024-04-24T09:02:26.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.149+0000 7ff01fe00640 1 -- 172.21.15.83:0/1431887432 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7ff02c0c21e0 con 0x5605b9c87460 2024-04-24T09:02:26.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.149+0000 7ff047e10740 10 client.19500 fetch_fsmap learned FSMap version 41 2024-04-24T09:02:26.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.149+0000 7ff047e10740 1 -- 172.21.15.83:0/1431887432 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5605b98fe3f0 con 0x5605b9c87460 2024-04-24T09:02:26.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.150+0000 7ff01fe00640 1 -- 172.21.15.83:0/1431887432 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff02c0fe030 con 0x5605b9c87460 2024-04-24T09:02:26.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.150+0000 7ff01fe00640 1 -- 172.21.15.83:0/1431887432 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7ff02c0eeb10 con 0x5605b9c87460 2024-04-24T09:02:26.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.150+0000 7ff047e10740 10 client.19500 fetch_fsmap finished waiting for FSMap version 41 2024-04-24T09:02:26.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.150+0000 7ff047e10740 10 client.19500 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-24T09:02:26.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.150+0000 7ff047e10740 20 client.19500 populate_metadata read hostname 'smithi083' 2024-04-24T09:02:26.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.150+0000 7ff047e10740 1 --2- 172.21.15.83:0/1431887432 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x5605b9ca17d0 0x5605b9ca3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:26.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.150+0000 7ff047e10740 4 client.19500 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:02:26.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.150+0000 7ff047e10740 1 -- 172.21.15.83:0/1431887432 --> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5605b9bf9bf0 con 0x5605b9ca17d0 2024-04-24T09:02:26.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.151+0000 7ff03de00640 1 --2- 172.21.15.83:0/1431887432 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x5605b9ca17d0 0x5605b9ca3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:26.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.152+0000 7ff03de00640 1 --2- 172.21.15.83:0/1431887432 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x5605b9ca17d0 0x5605b9ca3bc0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d41a9337e907a03e server_cookie=77c3d78ad5ea3246 in_seq=0 out_seq=0 2024-04-24T09:02:26.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.152+0000 7ff01fe00640 10 client.19500 ms_handle_connect on v2:172.21.15.138:6836/900566754 2024-04-24T09:02:26.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.154+0000 7ff01fe00640 1 -- 172.21.15.83:0/1431887432 <== mds.1 v2:172.21.15.138:6836/900566754 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5605b9bf9bf0 con 0x5605b9ca17d0 2024-04-24T09:02:26.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.154+0000 7ff01fe00640 10 client.19500 handle_command_reply: tid=0 2024-04-24T09:02:26.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.154+0000 7ff047e10740 1 client.19500 shutdown 2024-04-24T09:02:26.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.154+0000 7ff047e10740 1 -- 172.21.15.83:0/1431887432 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5605b9c87460 msgr2=0x5605b9c8e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:26.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.154+0000 7ff047e10740 1 --2- 172.21.15.83:0/1431887432 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5605b9c87460 0x5605b9c8e7f0 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7ff02c0c2580 tx=0x7ff02c02a480 comp rx=0 tx=0).stop 2024-04-24T09:02:26.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.154+0000 7ff047e10740 1 -- 172.21.15.83:0/1431887432 shutdown_connections 2024-04-24T09:02:26.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.154+0000 7ff047e10740 1 --2- 172.21.15.83:0/1431887432 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x5605b9ca17d0 0x5605b9ca3bc0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:26.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.154+0000 7ff047e10740 1 --2- 172.21.15.83:0/1431887432 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5605b9c91b30 0x5605b9c8ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:26.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.154+0000 7ff047e10740 1 --2- 172.21.15.83:0/1431887432 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5605b9c87460 0x5605b9c8e7f0 unknown :-1 s=CLOSED pgs=1780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:26.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.154+0000 7ff047e10740 1 --2- 172.21.15.83:0/1431887432 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605b9c86b40 0x5605b9c91500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:26.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.154+0000 7ff047e10740 1 -- 172.21.15.83:0/1431887432 >> 172.21.15.83:0/1431887432 conn(0x5605b9c82e60 msgr2=0x5605b9c88c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:02:26.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.154+0000 7ff01fe00640 0 client.19500 ms_handle_reset on v2:172.21.15.138:6836/900566754 2024-04-24T09:02:26.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.154+0000 7ff047e10740 1 -- 172.21.15.83:0/1431887432 shutdown_connections 2024-04-24T09:02:26.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.154+0000 7ff047e10740 1 -- 172.21.15.83:0/1431887432 wait complete. 2024-04-24T09:02:26.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.154+0000 7ff047e10740 20 client.19500 trim_cache size 0 max 16384 2024-04-24T09:02:26.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.171+0000 7ff047e10740 1 Processor -- start 2024-04-24T09:02:26.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.171+0000 7ff047e10740 1 -- start start 2024-04-24T09:02:26.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.171+0000 7ff047e10740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5605b9c86b40 0x5605b9c892a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:26.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.171+0000 7ff047e10740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605b9c87460 0x5605b9c81dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:26.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.171+0000 7ff047e10740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5605b9c91b30 0x5605b9c91030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:26.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.171+0000 7ff047e10740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff0401b1550 con 0x5605b9c86b40 2024-04-24T09:02:26.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.171+0000 7ff047e10740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff0401b31b0 con 0x5605b9c87460 2024-04-24T09:02:26.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.171+0000 7ff047e10740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff0401b1c20 con 0x5605b9c91b30 2024-04-24T09:02:26.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.172+0000 7ff03de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5605b9c86b40 0x5605b9c892a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:26.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.172+0000 7ff03de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5605b9c86b40 0x5605b9c892a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51434/0 (socket says 172.21.15.83:51434) 2024-04-24T09:02:26.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.172+0000 7ff03de00640 1 -- 172.21.15.83:0/3538566456 learned_addr learned my addr 172.21.15.83:0/3538566456 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:02:26.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.172+0000 7ff03d400640 1 --2- 172.21.15.83:0/3538566456 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605b9c87460 0x5605b9c81dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:26.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.172+0000 7ff03e800640 1 --2- 172.21.15.83:0/3538566456 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5605b9c91b30 0x5605b9c91030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:26.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.172+0000 7ff03de00640 1 -- 172.21.15.83:0/3538566456 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5605b9c91b30 msgr2=0x5605b9c91030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:26.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.172+0000 7ff03de00640 1 --2- 172.21.15.83:0/3538566456 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5605b9c91b30 0x5605b9c91030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:26.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.172+0000 7ff03de00640 1 -- 172.21.15.83:0/3538566456 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605b9c87460 msgr2=0x5605b9c81dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:26.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.172+0000 7ff03de00640 1 --2- 172.21.15.83:0/3538566456 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605b9c87460 0x5605b9c81dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:26.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.172+0000 7ff03de00640 1 -- 172.21.15.83:0/3538566456 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605b99c87d0 con 0x5605b9c86b40 2024-04-24T09:02:26.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.173+0000 7ff03de00640 1 --2- 172.21.15.83:0/3538566456 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5605b9c86b40 0x5605b9c892a0 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7ff03000ece0 tx=0x7ff03000e3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25199e8d558aa51c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:02:26.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.173+0000 7ff02aa00640 1 -- 172.21.15.83:0/3538566456 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff030013b10 con 0x5605b9c86b40 2024-04-24T09:02:26.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.173+0000 7ff02aa00640 1 -- 172.21.15.83:0/3538566456 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff030013cb0 con 0x5605b9c86b40 2024-04-24T09:02:26.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.173+0000 7ff02aa00640 1 -- 172.21.15.83:0/3538566456 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0300035f0 con 0x5605b9c86b40 2024-04-24T09:02:26.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.174+0000 7ff047e10740 1 -- 172.21.15.83:0/3538566456 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5605b9c86b40 msgr2=0x5605b9c892a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:26.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.174+0000 7ff047e10740 1 --2- 172.21.15.83:0/3538566456 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5605b9c86b40 0x5605b9c892a0 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7ff03000ece0 tx=0x7ff03000e3b0 comp rx=0 tx=0).stop 2024-04-24T09:02:26.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.174+0000 7ff047e10740 1 -- 172.21.15.83:0/3538566456 shutdown_connections 2024-04-24T09:02:26.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.174+0000 7ff047e10740 1 --2- 172.21.15.83:0/3538566456 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5605b9c91b30 0x5605b9c91030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:26.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.174+0000 7ff047e10740 1 --2- 172.21.15.83:0/3538566456 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605b9c87460 0x5605b9c81dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:26.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.174+0000 7ff047e10740 1 --2- 172.21.15.83:0/3538566456 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5605b9c86b40 0x5605b9c892a0 unknown :-1 s=CLOSED pgs=1781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:26.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.174+0000 7ff047e10740 1 -- 172.21.15.83:0/3538566456 >> 172.21.15.83:0/3538566456 conn(0x5605b9c82e60 msgr2=0x5605b9c816c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:02:26.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.174+0000 7ff047e10740 1 -- 172.21.15.83:0/3538566456 shutdown_connections 2024-04-24T09:02:26.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.174+0000 7ff047e10740 1 -- 172.21.15.83:0/3538566456 wait complete. 2024-04-24T09:02:26.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.174+0000 7ff047e10740 1 -- start start 2024-04-24T09:02:26.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.175+0000 7ff047e10740 1 Processor -- start 2024-04-24T09:02:26.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.175+0000 7ff047e10740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5605b9c86b40 0x5605b9c92bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:26.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.175+0000 7ff047e10740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605b9c87460 0x5605b9c93110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:26.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.175+0000 7ff047e10740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5605b9c91b30 0x5605b9c98000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:26.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.175+0000 7ff047e10740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff0401b2a80 con 0x5605b9c91b30 2024-04-24T09:02:26.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.175+0000 7ff047e10740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff0401b2350 con 0x5605b9c87460 2024-04-24T09:02:26.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.175+0000 7ff047e10740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff0401af550 con 0x5605b9c86b40 2024-04-24T09:02:26.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.175+0000 7ff047e10740 10 client.0 _ll_register_callbacks cb 0x7ff02c0f75c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:02:26.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.175+0000 7ff03e800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5605b9c91b30 0x5605b9c98000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:26.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.175+0000 7ff03e800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5605b9c91b30 0x5605b9c98000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51440/0 (socket says 172.21.15.83:51440) 2024-04-24T09:02:26.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.175+0000 7ff03e800640 1 -- 172.21.15.83:0/3876593707 learned_addr learned my addr 172.21.15.83:0/3876593707 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:02:26.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.176+0000 7ff03de00640 1 --2- 172.21.15.83:0/3876593707 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5605b9c86b40 0x5605b9c92bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:26.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.176+0000 7ff03d400640 1 --2- 172.21.15.83:0/3876593707 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605b9c87460 0x5605b9c93110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:26.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.176+0000 7ff03e800640 1 -- 172.21.15.83:0/3876593707 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5605b9c86b40 msgr2=0x5605b9c92bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:26.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.176+0000 7ff03e800640 1 --2- 172.21.15.83:0/3876593707 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5605b9c86b40 0x5605b9c92bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:26.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.176+0000 7ff03e800640 1 -- 172.21.15.83:0/3876593707 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605b9c87460 msgr2=0x5605b9c93110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:26.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.176+0000 7ff03e800640 1 --2- 172.21.15.83:0/3876593707 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605b9c87460 0x5605b9c93110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:26.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.176+0000 7ff03e800640 1 -- 172.21.15.83:0/3876593707 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605b9c939d0 con 0x5605b9c91b30 2024-04-24T09:02:26.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.176+0000 7ff03de00640 1 --2- 172.21.15.83:0/3876593707 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5605b9c86b40 0x5605b9c92bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:02:26.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.176+0000 7ff03e800640 1 --2- 172.21.15.83:0/3876593707 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5605b9c91b30 0x5605b9c98000 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7ff0380119f0 tx=0x7ff038004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21503ba12fb54502 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:02:26.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.176+0000 7ff01fe00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:02:26.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.176+0000 7ff01fe00640 1 -- 172.21.15.83:0/3876593707 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff038010d60 con 0x5605b9c91b30 2024-04-24T09:02:26.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.177+0000 7ff01fe00640 1 -- 172.21.15.83:0/3876593707 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9583+0+0 (secure 0 0 0) 0x7ff038020440 con 0x5605b9c91b30 2024-04-24T09:02:26.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.177+0000 7ff047e10740 1 -- 172.21.15.83:0/3876593707 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff030013cb0 con 0x5605b9c91b30 2024-04-24T09:02:26.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.177+0000 7ff01fe00640 1 -- 172.21.15.83:0/3876593707 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7ff038031050 con 0x5605b9c91b30 2024-04-24T09:02:26.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.177+0000 7ff047e10740 10 client.19506 fetch_fsmap learned FSMap version 41 2024-04-24T09:02:26.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.177+0000 7ff047e10740 1 -- 172.21.15.83:0/3876593707 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5605b9d082e0 con 0x5605b9c91b30 2024-04-24T09:02:26.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.177+0000 7ff01fe00640 1 -- 172.21.15.83:0/3876593707 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff030013cb0 con 0x5605b9c91b30 2024-04-24T09:02:26.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.177+0000 7ff01fe00640 1 -- 172.21.15.83:0/3876593707 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7ff03801f590 con 0x5605b9c91b30 2024-04-24T09:02:26.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.178+0000 7ff047e10740 10 client.19506 fetch_fsmap finished waiting for FSMap version 41 2024-04-24T09:02:26.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.178+0000 7ff047e10740 10 client.19506 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-24T09:02:26.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.178+0000 7ff047e10740 20 client.19506 populate_metadata read hostname 'smithi083' 2024-04-24T09:02:26.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.178+0000 7ff047e10740 1 --2- 172.21.15.83:0/3876593707 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x5605b9d08af0 0x5605b9d0af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:26.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.178+0000 7ff047e10740 4 client.19506 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:02:26.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.178+0000 7ff047e10740 1 -- 172.21.15.83:0/3876593707 --> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5605b9bf3a60 con 0x5605b9d08af0 2024-04-24T09:02:26.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.178+0000 7ff03de00640 1 --2- 172.21.15.83:0/3876593707 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x5605b9d08af0 0x5605b9d0af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:26.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.179+0000 7ff03de00640 1 --2- 172.21.15.83:0/3876593707 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x5605b9d08af0 0x5605b9d0af10 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=dcc941fbcef479ce server_cookie=286dab89ce7b9f03 in_seq=0 out_seq=0 2024-04-24T09:02:26.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.179+0000 7ff01fe00640 10 client.19506 ms_handle_connect on v2:172.21.15.138:6836/900566754 2024-04-24T09:02:26.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.180+0000 7ff01fe00640 1 -- 172.21.15.83:0/3876593707 <== mds.1 v2:172.21.15.138:6836/900566754 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5605b9bf3a60 con 0x5605b9d08af0 2024-04-24T09:02:26.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.180+0000 7ff01fe00640 10 client.19506 handle_command_reply: tid=0 2024-04-24T09:02:26.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.180+0000 7ff047e10740 1 client.19506 shutdown 2024-04-24T09:02:26.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.180+0000 7ff047e10740 1 -- 172.21.15.83:0/3876593707 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5605b9c91b30 msgr2=0x5605b9c98000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:26.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.180+0000 7ff047e10740 1 --2- 172.21.15.83:0/3876593707 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5605b9c91b30 0x5605b9c98000 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7ff0380119f0 tx=0x7ff038004ab0 comp rx=0 tx=0).stop 2024-04-24T09:02:26.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.180+0000 7ff047e10740 1 -- 172.21.15.83:0/3876593707 shutdown_connections 2024-04-24T09:02:26.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.180+0000 7ff047e10740 1 --2- 172.21.15.83:0/3876593707 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5605b9c91b30 0x5605b9c98000 unknown :-1 s=CLOSED pgs=1782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:26.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.180+0000 7ff047e10740 1 --2- 172.21.15.83:0/3876593707 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605b9c87460 0x5605b9c93110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:26.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.180+0000 7ff047e10740 1 --2- 172.21.15.83:0/3876593707 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x5605b9d08af0 0x5605b9d0af10 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:26.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.180+0000 7ff047e10740 1 --2- 172.21.15.83:0/3876593707 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5605b9c86b40 0x5605b9c92bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:26.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.180+0000 7ff047e10740 1 -- 172.21.15.83:0/3876593707 >> 172.21.15.83:0/3876593707 conn(0x5605b9c82e60 msgr2=0x5605b9c81f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:02:26.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.180+0000 7ff01fe00640 0 client.19506 ms_handle_reset on v2:172.21.15.138:6836/900566754 2024-04-24T09:02:26.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.180+0000 7ff047e10740 1 -- 172.21.15.83:0/3876593707 shutdown_connections 2024-04-24T09:02:26.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.180+0000 7ff047e10740 1 -- 172.21.15.83:0/3876593707 wait complete. 2024-04-24T09:02:26.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.180+0000 7ff047e10740 20 client.19506 trim_cache size 0 max 16384 2024-04-24T09:02:26.182 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:02:26.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.181+0000 7ff03f200640 1 -- 172.21.15.83:0/3053879695 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff0240a3d20 msgr2=0x7ff0240a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:26.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.181+0000 7ff03f200640 1 --2- 172.21.15.83:0/3053879695 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff0240a3d20 0x7ff0240a61e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7ff03800ae10 tx=0x7ff03800a3c0 comp rx=0 tx=0).stop 2024-04-24T09:02:26.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.181+0000 7ff03f200640 1 -- 172.21.15.83:0/3053879695 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff040103980 msgr2=0x7ff0401a0ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:26.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.181+0000 7ff03f200640 1 --2- 172.21.15.83:0/3053879695 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff040103980 0x7ff0401a0ae0 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7ff02c006010 tx=0x7ff02c0042f0 comp rx=0 tx=0).stop 2024-04-24T09:02:26.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.181+0000 7ff03f200640 1 -- 172.21.15.83:0/3053879695 shutdown_connections 2024-04-24T09:02:26.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.181+0000 7ff03f200640 1 --2- 172.21.15.83:0/3053879695 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff0240a3d20 0x7ff0240a61e0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:26.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.181+0000 7ff03f200640 1 --2- 172.21.15.83:0/3053879695 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff04010c480 0x7ff04019a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:26.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.181+0000 7ff03f200640 1 --2- 172.21.15.83:0/3053879695 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff040103980 0x7ff0401a0ae0 unknown :-1 s=CLOSED pgs=1778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:26.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.181+0000 7ff03f200640 1 --2- 172.21.15.83:0/3053879695 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff040103060 0x7ff0401a05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:26.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.181+0000 7ff03f200640 1 -- 172.21.15.83:0/3053879695 >> 172.21.15.83:0/3053879695 conn(0x7ff0400fc880 msgr2=0x7ff040100c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:02:26.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.181+0000 7ff03f200640 1 -- 172.21.15.83:0/3053879695 shutdown_connections 2024-04-24T09:02:26.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.181+0000 7ff03f200640 1 -- 172.21.15.83:0/3053879695 wait complete. 2024-04-24T09:02:26.198 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T09:02:26.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:02:25 smithi138 ceph-mon[31441]: pgmap v339: 129 pgs: 129 active+clean; 60 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 273 KiB/s wr, 46 op/s 2024-04-24T09:02:26.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:02:25 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/3071260272' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:02:26.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:02:25 smithi189 ceph-mon[31791]: pgmap v339: 129 pgs: 129 active+clean; 60 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 273 KiB/s wr, 46 op/s 2024-04-24T09:02:26.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:02:25 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/3071260272' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:02:26.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.409+0000 7f6ed2e00640 1 -- 172.21.15.83:0/2458546383 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ecc1071c0 msgr2=0x7f6ecc10ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:26.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.409+0000 7f6ed2e00640 1 --2- 172.21.15.83:0/2458546383 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ecc1071c0 0x7f6ecc10ed90 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f6ec4009900 tx=0x7f6ec402a410 comp rx=0 tx=0).stop 2024-04-24T09:02:26.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.409+0000 7f6ed2e00640 1 -- 172.21.15.83:0/2458546383 shutdown_connections 2024-04-24T09:02:26.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.409+0000 7f6ed2e00640 1 --2- 172.21.15.83:0/2458546383 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ecc1071c0 0x7f6ecc10ed90 unknown :-1 s=CLOSED pgs=1783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:26.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.409+0000 7f6ed2e00640 1 --2- 172.21.15.83:0/2458546383 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6ecc102aa0 0x7f6ecc106ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:26.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.409+0000 7f6ed2e00640 1 --2- 172.21.15.83:0/2458546383 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ecc1020f0 0x7f6ecc1024d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:26.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.409+0000 7f6ed2e00640 1 -- 172.21.15.83:0/2458546383 >> 172.21.15.83:0/2458546383 conn(0x7f6ecc0fd9d0 msgr2=0x7f6ecc0ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:02:26.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.410+0000 7f6ed2e00640 1 -- 172.21.15.83:0/2458546383 shutdown_connections 2024-04-24T09:02:26.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.410+0000 7f6ed2e00640 1 -- 172.21.15.83:0/2458546383 wait complete. 2024-04-24T09:02:26.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.411+0000 7f6ed2e00640 1 Processor -- start 2024-04-24T09:02:26.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.411+0000 7f6ed2e00640 1 -- start start 2024-04-24T09:02:26.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.411+0000 7f6ed2e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6ecc1020f0 0x7f6ecc1a05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:26.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.411+0000 7f6ed2e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ecc1071c0 0x7f6ecc1a0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:26.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.411+0000 7f6ed2e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ecc19a730 0x7f6ecc19abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:26.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.411+0000 7f6ed2e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6ecc112130 con 0x7f6ecc1071c0 2024-04-24T09:02:26.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.411+0000 7f6ed2e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6ecc111fb0 con 0x7f6ecc19a730 2024-04-24T09:02:26.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.411+0000 7f6ed2e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6ecc1122b0 con 0x7f6ecc1020f0 2024-04-24T09:02:26.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.411+0000 7f6ed1000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ecc1071c0 0x7f6ecc1a0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:26.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.412+0000 7f6ed1000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ecc1071c0 0x7f6ecc1a0b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51466/0 (socket says 172.21.15.83:51466) 2024-04-24T09:02:26.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.412+0000 7f6ed1000640 1 -- 172.21.15.83:0/600595459 learned_addr learned my addr 172.21.15.83:0/600595459 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:02:26.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.412+0000 7f6ed1a00640 1 --2- 172.21.15.83:0/600595459 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6ecc1020f0 0x7f6ecc1a05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:26.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.412+0000 7f6ed2400640 1 --2- 172.21.15.83:0/600595459 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ecc19a730 0x7f6ecc19abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:26.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.412+0000 7f6ed1000640 1 -- 172.21.15.83:0/600595459 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6ecc1020f0 msgr2=0x7f6ecc1a05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:26.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.412+0000 7f6ed1000640 1 --2- 172.21.15.83:0/600595459 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6ecc1020f0 0x7f6ecc1a05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:26.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.412+0000 7f6ed1000640 1 -- 172.21.15.83:0/600595459 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ecc19a730 msgr2=0x7f6ecc19abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:26.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.412+0000 7f6ed1000640 1 --2- 172.21.15.83:0/600595459 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ecc19a730 0x7f6ecc19abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:26.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.412+0000 7f6ed1000640 1 -- 172.21.15.83:0/600595459 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ecc19b330 con 0x7f6ecc1071c0 2024-04-24T09:02:26.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.413+0000 7f6ed1000640 1 --2- 172.21.15.83:0/600595459 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ecc1071c0 0x7f6ecc1a0b10 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f6eb800e9f0 tx=0x7f6eb800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:02:26.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.413+0000 7f6eca000640 1 -- 172.21.15.83:0/600595459 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb800ccc0 con 0x7f6ecc1071c0 2024-04-24T09:02:26.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.413+0000 7f6ed2e00640 1 -- 172.21.15.83:0/600595459 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ecc1a7370 con 0x7f6ecc1071c0 2024-04-24T09:02:26.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.413+0000 7f6eca000640 1 -- 172.21.15.83:0/600595459 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6eb80054d0 con 0x7f6ecc1071c0 2024-04-24T09:02:26.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.413+0000 7f6eca000640 1 -- 172.21.15.83:0/600595459 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb8010430 con 0x7f6ecc1071c0 2024-04-24T09:02:26.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.413+0000 7f6ed2e00640 1 -- 172.21.15.83:0/600595459 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ecc1a7770 con 0x7f6ecc1071c0 2024-04-24T09:02:26.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.417+0000 7f6ed7207740 1 Processor -- start 2024-04-24T09:02:26.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.417+0000 7f6ed7207740 1 -- start start 2024-04-24T09:02:26.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.417+0000 7f6ed7207740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556e2ade5b40 0x556e2ade5f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:26.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.417+0000 7f6eca000640 1 -- 172.21.15.83:0/600595459 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6eb8005020 con 0x7f6ecc1071c0 2024-04-24T09:02:26.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.417+0000 7f6ed7207740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556e2ade6460 0x556e2adf05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:26.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.417+0000 7f6ed7207740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e2adf0b30 0x556e2adf2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:26.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.417+0000 7f6ed7207740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556e2ad4b1d0 con 0x556e2adf0b30 2024-04-24T09:02:26.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.417+0000 7f6ed7207740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556e2acf1450 con 0x556e2ade5b40 2024-04-24T09:02:26.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.417+0000 7f6ed7207740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x556e2ad47190 con 0x556e2ade6460 2024-04-24T09:02:26.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.418+0000 7f6ed1000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e2adf0b30 0x556e2adf2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:26.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.418+0000 7f6ed1000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e2adf0b30 0x556e2adf2f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51474/0 (socket says 172.21.15.83:51474) 2024-04-24T09:02:26.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.418+0000 7f6ed1000640 1 -- 172.21.15.83:0/3386160309 learned_addr learned my addr 172.21.15.83:0/3386160309 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:02:26.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.418+0000 7f6ed2400640 1 --2- 172.21.15.83:0/3386160309 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556e2ade5b40 0x556e2ade5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:26.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.418+0000 7f6ed1a00640 1 --2- 172.21.15.83:0/3386160309 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556e2ade6460 0x556e2adf05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:26.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.418+0000 7f6ed1000640 1 -- 172.21.15.83:0/3386160309 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556e2ade6460 msgr2=0x556e2adf05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:26.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.418+0000 7f6ed1000640 1 --2- 172.21.15.83:0/3386160309 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556e2ade6460 0x556e2adf05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:26.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.418+0000 7f6ed1000640 1 -- 172.21.15.83:0/3386160309 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556e2ade5b40 msgr2=0x556e2ade5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:26.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.418+0000 7f6ed1000640 1 --2- 172.21.15.83:0/3386160309 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556e2ade5b40 0x556e2ade5f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:26.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.418+0000 7f6ed1000640 1 -- 172.21.15.83:0/3386160309 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e2adae310 con 0x556e2adf0b30 2024-04-24T09:02:26.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.419+0000 7f6ed1000640 1 --2- 172.21.15.83:0/3386160309 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e2adf0b30 0x556e2adf2f20 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f6eb80ce210 tx=0x7f6eb80d91a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b55de12b775f620 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:02:26.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.419+0000 7f6ebea00640 1 -- 172.21.15.83:0/3386160309 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb80e4070 con 0x556e2adf0b30 2024-04-24T09:02:26.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.419+0000 7f6eca000640 1 --2- 172.21.15.83:0/600595459 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f6eb40a3da0 0x7f6eb40a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:26.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.419+0000 7f6ebea00640 1 -- 172.21.15.83:0/3386160309 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6eb8099160 con 0x556e2adf0b30 2024-04-24T09:02:26.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.419+0000 7f6ebea00640 1 -- 172.21.15.83:0/3386160309 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb809c450 con 0x556e2adf0b30 2024-04-24T09:02:26.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.419+0000 7f6eca000640 1 -- 172.21.15.83:0/600595459 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9583+0+0 (secure 0 0 0) 0x7f6eb80d0f70 con 0x7f6ecc1071c0 2024-04-24T09:02:26.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.420+0000 7f6ed2400640 1 --2- 172.21.15.83:0/600595459 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f6eb40a3da0 0x7f6eb40a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:26.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.420+0000 7f6ed7207740 1 -- 172.21.15.83:0/3386160309 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e2adf0b30 msgr2=0x556e2adf2f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:26.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.420+0000 7f6ed7207740 1 --2- 172.21.15.83:0/3386160309 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e2adf0b30 0x556e2adf2f20 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f6eb80ce210 tx=0x7f6eb80d91a0 comp rx=0 tx=0).stop 2024-04-24T09:02:26.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.420+0000 7f6ed7207740 1 -- 172.21.15.83:0/3386160309 shutdown_connections 2024-04-24T09:02:26.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.420+0000 7f6ed7207740 1 --2- 172.21.15.83:0/3386160309 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e2adf0b30 0x556e2adf2f20 unknown :-1 s=CLOSED pgs=1785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:26.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.420+0000 7f6ed7207740 1 --2- 172.21.15.83:0/3386160309 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556e2ade6460 0x556e2adf05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:26.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.420+0000 7f6ed2400640 1 --2- 172.21.15.83:0/600595459 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f6eb40a3da0 0x7f6eb40a6260 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f6ec4005940 tx=0x7f6ec40058b0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:02:26.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.420+0000 7f6ed7207740 1 --2- 172.21.15.83:0/3386160309 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556e2ade5b40 0x556e2ade5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:26.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.420+0000 7f6ed7207740 1 -- 172.21.15.83:0/3386160309 >> 172.21.15.83:0/3386160309 conn(0x556e2ade1e60 msgr2=0x556e2ade2240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:02:26.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.420+0000 7f6ed7207740 1 -- 172.21.15.83:0/3386160309 shutdown_connections 2024-04-24T09:02:26.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.421+0000 7f6ed7207740 1 -- 172.21.15.83:0/3386160309 wait complete. 2024-04-24T09:02:26.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.421+0000 7f6ed7207740 1 -- start start 2024-04-24T09:02:26.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.421+0000 7f6ed7207740 1 Processor -- start 2024-04-24T09:02:26.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.422+0000 7f6ed7207740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556e2ade5b40 0x556e2adf0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:26.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.422+0000 7f6ed7207740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e2ade6460 0x556e2aded870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:26.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.422+0000 7f6ed7207740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556e2adf0b30 0x556e2adedfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:26.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.422+0000 7f6ed7207740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556e2adf39c0 con 0x556e2ade6460 2024-04-24T09:02:26.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.422+0000 7f6ed7207740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556e2ad04810 con 0x556e2ade5b40 2024-04-24T09:02:26.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.422+0000 7f6ed7207740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x556e2adf3b40 con 0x556e2adf0b30 2024-04-24T09:02:26.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.422+0000 7f6ed7207740 10 client.0 _ll_register_callbacks cb 0x556e2acd2140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:02:26.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.422+0000 7f6ed1000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e2ade6460 0x556e2aded870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:26.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.422+0000 7f6ed1000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e2ade6460 0x556e2aded870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51482/0 (socket says 172.21.15.83:51482) 2024-04-24T09:02:26.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.422+0000 7f6ed1000640 1 -- 172.21.15.83:0/2092335683 learned_addr learned my addr 172.21.15.83:0/2092335683 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:02:26.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.422+0000 7f6ed1a00640 1 --2- 172.21.15.83:0/2092335683 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556e2ade5b40 0x556e2adf0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:26.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.422+0000 7f6ed2400640 1 --2- 172.21.15.83:0/2092335683 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556e2adf0b30 0x556e2adedfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:26.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.422+0000 7f6ed1000640 1 -- 172.21.15.83:0/2092335683 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556e2adf0b30 msgr2=0x556e2adedfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:26.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.422+0000 7f6ed1000640 1 --2- 172.21.15.83:0/2092335683 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556e2adf0b30 0x556e2adedfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:26.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.422+0000 7f6ed1000640 1 -- 172.21.15.83:0/2092335683 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556e2ade5b40 msgr2=0x556e2adf0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:26.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.422+0000 7f6ed1000640 1 --2- 172.21.15.83:0/2092335683 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556e2ade5b40 0x556e2adf0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:26.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.422+0000 7f6ed1000640 1 -- 172.21.15.83:0/2092335683 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e2ad153f0 con 0x556e2ade6460 2024-04-24T09:02:26.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.423+0000 7f6ed1000640 1 --2- 172.21.15.83:0/2092335683 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e2ade6460 0x556e2aded870 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f6eb8098f50 tx=0x7f6eb8004a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e410728bd7623ae5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:02:26.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.423+0000 7f6eb3400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:02:26.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.423+0000 7f6eb3400640 1 -- 172.21.15.83:0/2092335683 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb809ae90 con 0x556e2ade6460 2024-04-24T09:02:26.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.423+0000 7f6eb3400640 1 -- 172.21.15.83:0/2092335683 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9583+0+0 (secure 0 0 0) 0x7f6eb80ce520 con 0x556e2ade6460 2024-04-24T09:02:26.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.423+0000 7f6ed7207740 1 -- 172.21.15.83:0/2092335683 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6eb8099160 con 0x556e2ade6460 2024-04-24T09:02:26.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.423+0000 7f6eb3400640 1 -- 172.21.15.83:0/2092335683 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f6eb809b0b0 con 0x556e2ade6460 2024-04-24T09:02:26.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.423+0000 7f6ed7207740 10 client.19518 fetch_fsmap learned FSMap version 41 2024-04-24T09:02:26.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.423+0000 7f6ed7207740 1 -- 172.21.15.83:0/2092335683 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x556e2aa5d3f0 con 0x556e2ade6460 2024-04-24T09:02:26.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.423+0000 7f6eb3400640 1 -- 172.21.15.83:0/2092335683 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6eb8099160 con 0x556e2ade6460 2024-04-24T09:02:26.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.423+0000 7f6eb3400640 1 -- 172.21.15.83:0/2092335683 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7f6eb809cd50 con 0x556e2ade6460 2024-04-24T09:02:26.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.424+0000 7f6ed7207740 10 client.19518 fetch_fsmap finished waiting for FSMap version 41 2024-04-24T09:02:26.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.424+0000 7f6ed7207740 10 client.19518 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:02:26.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.424+0000 7f6ed7207740 20 client.19518 populate_metadata read hostname 'smithi083' 2024-04-24T09:02:26.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.424+0000 7f6ed7207740 1 --2- 172.21.15.83:0/2092335683 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556e2ae00bc0 0x556e2ae02fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:26.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.424+0000 7f6ed7207740 4 client.19518 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:02:26.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.424+0000 7f6ed7207740 1 -- 172.21.15.83:0/2092335683 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556e2ad58bf0 con 0x556e2ae00bc0 2024-04-24T09:02:26.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.424+0000 7f6ed1a00640 1 --2- 172.21.15.83:0/2092335683 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556e2ae00bc0 0x556e2ae02fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:26.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.425+0000 7f6ed1a00640 1 --2- 172.21.15.83:0/2092335683 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556e2ae00bc0 0x556e2ae02fb0 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9fe82cf2081f852b server_cookie=5e7c9494900c9997 in_seq=0 out_seq=0 2024-04-24T09:02:26.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.425+0000 7f6eb3400640 10 client.19518 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:02:26.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.427+0000 7f6eb3400640 1 -- 172.21.15.83:0/2092335683 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556e2ad58bf0 con 0x556e2ae00bc0 2024-04-24T09:02:26.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.427+0000 7f6eb3400640 10 client.19518 handle_command_reply: tid=0 2024-04-24T09:02:26.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.427+0000 7f6ed7207740 1 client.19518 shutdown 2024-04-24T09:02:26.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.427+0000 7f6ed7207740 1 -- 172.21.15.83:0/2092335683 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e2ade6460 msgr2=0x556e2aded870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:26.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.427+0000 7f6ed7207740 1 --2- 172.21.15.83:0/2092335683 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e2ade6460 0x556e2aded870 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f6eb8098f50 tx=0x7f6eb8004a20 comp rx=0 tx=0).stop 2024-04-24T09:02:26.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.427+0000 7f6ed7207740 1 -- 172.21.15.83:0/2092335683 shutdown_connections 2024-04-24T09:02:26.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.427+0000 7f6ed7207740 1 --2- 172.21.15.83:0/2092335683 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556e2ae00bc0 0x556e2ae02fb0 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:26.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.427+0000 7f6ed7207740 1 --2- 172.21.15.83:0/2092335683 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556e2adf0b30 0x556e2adedfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:26.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.427+0000 7f6ed7207740 1 --2- 172.21.15.83:0/2092335683 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e2ade6460 0x556e2aded870 unknown :-1 s=CLOSED pgs=1786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:26.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.427+0000 7f6ed7207740 1 --2- 172.21.15.83:0/2092335683 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556e2ade5b40 0x556e2adf0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:26.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.427+0000 7f6eb3400640 0 client.19518 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:02:26.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.427+0000 7f6ed7207740 1 -- 172.21.15.83:0/2092335683 >> 172.21.15.83:0/2092335683 conn(0x556e2ade1e60 msgr2=0x556e2ade7c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:02:26.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.428+0000 7f6ed7207740 1 -- 172.21.15.83:0/2092335683 shutdown_connections 2024-04-24T09:02:26.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.428+0000 7f6ed7207740 1 -- 172.21.15.83:0/2092335683 wait complete. 2024-04-24T09:02:26.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.428+0000 7f6ed7207740 20 client.19518 trim_cache size 0 max 16384 2024-04-24T09:02:26.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.444+0000 7f6ed7207740 1 Processor -- start 2024-04-24T09:02:26.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.444+0000 7f6ed7207740 1 -- start start 2024-04-24T09:02:26.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.444+0000 7f6ed7207740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556e2ade5b40 0x556e2adec9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:26.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.444+0000 7f6ed7207740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e2ade6460 0x556e2adf00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:26.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.444+0000 7f6ed7207740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556e2adf0b30 0x556e2adf05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:26.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.444+0000 7f6ed7207740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6ecc1b1640 con 0x556e2ade6460 2024-04-24T09:02:26.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.444+0000 7f6ed7207740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6ecc1b32a0 con 0x556e2adf0b30 2024-04-24T09:02:26.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.444+0000 7f6ed7207740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6ecc1b1d10 con 0x556e2ade5b40 2024-04-24T09:02:26.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.444+0000 7f6ed1000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e2ade6460 0x556e2adf00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:26.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.445+0000 7f6ed1000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e2ade6460 0x556e2adf00b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51496/0 (socket says 172.21.15.83:51496) 2024-04-24T09:02:26.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.445+0000 7f6ed1000640 1 -- 172.21.15.83:0/2088454753 learned_addr learned my addr 172.21.15.83:0/2088454753 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:02:26.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.445+0000 7f6ed1a00640 1 --2- 172.21.15.83:0/2088454753 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556e2ade5b40 0x556e2adec9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:26.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.445+0000 7f6ed1000640 1 -- 172.21.15.83:0/2088454753 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556e2ade5b40 msgr2=0x556e2adec9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:26.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.445+0000 7f6ed1000640 1 --2- 172.21.15.83:0/2088454753 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556e2ade5b40 0x556e2adec9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:26.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.445+0000 7f6ed1000640 1 -- 172.21.15.83:0/2088454753 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556e2adf0b30 msgr2=0x556e2adf05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:26.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.445+0000 7f6ed1000640 1 --2- 172.21.15.83:0/2088454753 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556e2adf0b30 0x556e2adf05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:26.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.445+0000 7f6ed1000640 1 -- 172.21.15.83:0/2088454753 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e2adee320 con 0x556e2ade6460 2024-04-24T09:02:26.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.445+0000 7f6ed1000640 1 --2- 172.21.15.83:0/2088454753 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e2ade6460 0x556e2adf00b0 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f6eb800e9c0 tx=0x7f6eb80d1270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bcc40dc4e3c55e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:02:26.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.445+0000 7f6ebe000640 1 -- 172.21.15.83:0/2088454753 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb809b2a0 con 0x556e2ade6460 2024-04-24T09:02:26.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.445+0000 7f6ebe000640 1 -- 172.21.15.83:0/2088454753 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6eb80b6020 con 0x556e2ade6460 2024-04-24T09:02:26.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.445+0000 7f6ebe000640 1 -- 172.21.15.83:0/2088454753 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb809b620 con 0x556e2ade6460 2024-04-24T09:02:26.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.446+0000 7f6ed7207740 1 -- 172.21.15.83:0/2088454753 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e2ade6460 msgr2=0x556e2adf00b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:26.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.446+0000 7f6ed7207740 1 --2- 172.21.15.83:0/2088454753 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e2ade6460 0x556e2adf00b0 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f6eb800e9c0 tx=0x7f6eb80d1270 comp rx=0 tx=0).stop 2024-04-24T09:02:26.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.446+0000 7f6ed7207740 1 -- 172.21.15.83:0/2088454753 shutdown_connections 2024-04-24T09:02:26.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.446+0000 7f6ed7207740 1 --2- 172.21.15.83:0/2088454753 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556e2adf0b30 0x556e2adf05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:26.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.446+0000 7f6ed7207740 1 --2- 172.21.15.83:0/2088454753 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e2ade6460 0x556e2adf00b0 unknown :-1 s=CLOSED pgs=1787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:26.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.446+0000 7f6ed7207740 1 --2- 172.21.15.83:0/2088454753 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556e2ade5b40 0x556e2adec9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:26.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.446+0000 7f6ed7207740 1 -- 172.21.15.83:0/2088454753 >> 172.21.15.83:0/2088454753 conn(0x556e2ade1e60 msgr2=0x556e2adf2e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:02:26.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.446+0000 7f6ed7207740 1 -- 172.21.15.83:0/2088454753 shutdown_connections 2024-04-24T09:02:26.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.446+0000 7f6ed7207740 1 -- 172.21.15.83:0/2088454753 wait complete. 2024-04-24T09:02:26.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.447+0000 7f6ed7207740 1 -- start start 2024-04-24T09:02:26.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.447+0000 7f6ed7207740 1 Processor -- start 2024-04-24T09:02:26.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.447+0000 7f6ed7207740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e2ade5b40 0x556e2ae01980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:26.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.447+0000 7f6ed7207740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556e2ade6460 0x556e2ae01ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:26.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.447+0000 7f6ed7207740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556e2adf0b30 0x556e2ae02400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:26.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.447+0000 7f6ed7207740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6ecc1b2b70 con 0x556e2ade5b40 2024-04-24T09:02:26.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.447+0000 7f6ed7207740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6ecc1b2440 con 0x556e2ade6460 2024-04-24T09:02:26.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.447+0000 7f6ed7207740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6ecc1af640 con 0x556e2adf0b30 2024-04-24T09:02:26.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.448+0000 7f6ed7207740 10 client.0 _ll_register_callbacks cb 0x7f6eb80e2070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:02:26.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.448+0000 7f6ed1a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e2ade5b40 0x556e2ae01980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:26.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.448+0000 7f6ed1a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e2ade5b40 0x556e2ae01980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51502/0 (socket says 172.21.15.83:51502) 2024-04-24T09:02:26.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.448+0000 7f6ed1a00640 1 -- 172.21.15.83:0/2189404262 learned_addr learned my addr 172.21.15.83:0/2189404262 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:02:26.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.448+0000 7f6ed1000640 1 --2- 172.21.15.83:0/2189404262 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556e2ade6460 0x556e2ae01ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:26.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.448+0000 7f6ed1a00640 1 -- 172.21.15.83:0/2189404262 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556e2adf0b30 msgr2=0x556e2ae02400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:26.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.448+0000 7f6ed1a00640 1 --2- 172.21.15.83:0/2189404262 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556e2adf0b30 0x556e2ae02400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:26.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.448+0000 7f6ed1a00640 1 -- 172.21.15.83:0/2189404262 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556e2ade6460 msgr2=0x556e2ae01ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:26.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.448+0000 7f6ed1a00640 1 --2- 172.21.15.83:0/2189404262 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556e2ade6460 0x556e2ae01ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:26.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.448+0000 7f6ed1a00640 1 -- 172.21.15.83:0/2189404262 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e2ab277d0 con 0x556e2ade5b40 2024-04-24T09:02:26.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.448+0000 7f6ed1a00640 1 --2- 172.21.15.83:0/2189404262 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e2ade5b40 0x556e2ae01980 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f6ec0000f80 tx=0x7f6ec0012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a8d62218617ec42 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:02:26.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.448+0000 7f6eb3400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:02:26.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.449+0000 7f6eb3400640 1 -- 172.21.15.83:0/2189404262 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec001b4c0 con 0x556e2ade5b40 2024-04-24T09:02:26.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.449+0000 7f6eb3400640 1 -- 172.21.15.83:0/2189404262 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9583+0+0 (secure 0 0 0) 0x7f6ec001b660 con 0x556e2ade5b40 2024-04-24T09:02:26.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.449+0000 7f6ed7207740 1 -- 172.21.15.83:0/2189404262 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6eb80b6020 con 0x556e2ade5b40 2024-04-24T09:02:26.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.449+0000 7f6eb3400640 1 -- 172.21.15.83:0/2189404262 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f6ec002c050 con 0x556e2ade5b40 2024-04-24T09:02:26.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.449+0000 7f6ed7207740 10 client.19524 fetch_fsmap learned FSMap version 41 2024-04-24T09:02:26.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.449+0000 7f6ed7207740 1 -- 172.21.15.83:0/2189404262 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x556e2ae66e80 con 0x556e2ade5b40 2024-04-24T09:02:26.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.449+0000 7f6eb3400640 1 -- 172.21.15.83:0/2189404262 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6eb80b6020 con 0x556e2ade5b40 2024-04-24T09:02:26.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.450+0000 7f6eb3400640 1 -- 172.21.15.83:0/2189404262 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7f6ec000f040 con 0x556e2ade5b40 2024-04-24T09:02:26.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.450+0000 7f6ed7207740 10 client.19524 fetch_fsmap finished waiting for FSMap version 41 2024-04-24T09:02:26.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.450+0000 7f6ed7207740 10 client.19524 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:02:26.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.450+0000 7f6ed7207740 20 client.19524 populate_metadata read hostname 'smithi083' 2024-04-24T09:02:26.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.450+0000 7f6ed7207740 1 --2- 172.21.15.83:0/2189404262 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556e2ae67620 0x556e2ae69a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:26.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.450+0000 7f6ed7207740 4 client.19524 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:02:26.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.450+0000 7f6ed7207740 1 -- 172.21.15.83:0/2189404262 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556e2ad52a60 con 0x556e2ae67620 2024-04-24T09:02:26.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.450+0000 7f6ed1000640 1 --2- 172.21.15.83:0/2189404262 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556e2ae67620 0x556e2ae69a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:26.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.452+0000 7f6ed1000640 1 --2- 172.21.15.83:0/2189404262 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556e2ae67620 0x556e2ae69a40 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c73d4d25f6361bd8 server_cookie=aac56d1f63eb65f7 in_seq=0 out_seq=0 2024-04-24T09:02:26.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.452+0000 7f6eb3400640 10 client.19524 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:02:26.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.452+0000 7f6eb3400640 1 -- 172.21.15.83:0/2189404262 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556e2ad52a60 con 0x556e2ae67620 2024-04-24T09:02:26.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.452+0000 7f6eb3400640 10 client.19524 handle_command_reply: tid=0 2024-04-24T09:02:26.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.452+0000 7f6ed7207740 1 client.19524 shutdown 2024-04-24T09:02:26.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.453+0000 7f6ed7207740 1 -- 172.21.15.83:0/2189404262 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e2ade5b40 msgr2=0x556e2ae01980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:26.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.453+0000 7f6ed7207740 1 --2- 172.21.15.83:0/2189404262 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e2ade5b40 0x556e2ae01980 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f6ec0000f80 tx=0x7f6ec0012820 comp rx=0 tx=0).stop 2024-04-24T09:02:26.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.453+0000 7f6ed7207740 1 -- 172.21.15.83:0/2189404262 shutdown_connections 2024-04-24T09:02:26.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.453+0000 7f6ed7207740 1 --2- 172.21.15.83:0/2189404262 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556e2ae67620 0x556e2ae69a40 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:26.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.453+0000 7f6ed7207740 1 --2- 172.21.15.83:0/2189404262 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556e2adf0b30 0x556e2ae02400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:26.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.453+0000 7f6eb3400640 0 client.19524 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:02:26.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.453+0000 7f6ed7207740 1 --2- 172.21.15.83:0/2189404262 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556e2ade6460 0x556e2ae01ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:26.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.453+0000 7f6ed7207740 1 --2- 172.21.15.83:0/2189404262 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e2ade5b40 0x556e2ae01980 unknown :-1 s=CLOSED pgs=1788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:26.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.453+0000 7f6ed7207740 1 -- 172.21.15.83:0/2189404262 >> 172.21.15.83:0/2189404262 conn(0x556e2ade1e60 msgr2=0x556e2adfbdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:02:26.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.453+0000 7f6ed7207740 1 -- 172.21.15.83:0/2189404262 shutdown_connections 2024-04-24T09:02:26.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.453+0000 7f6ed7207740 1 -- 172.21.15.83:0/2189404262 wait complete. 2024-04-24T09:02:26.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.453+0000 7f6ed7207740 20 client.19524 trim_cache size 0 max 16384 2024-04-24T09:02:26.457 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:02:26.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.454+0000 7f6ed2e00640 1 -- 172.21.15.83:0/600595459 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f6eb40a3da0 msgr2=0x7f6eb40a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:26.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.454+0000 7f6ed2e00640 1 --2- 172.21.15.83:0/600595459 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f6eb40a3da0 0x7f6eb40a6260 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f6ec4005940 tx=0x7f6ec40058b0 comp rx=0 tx=0).stop 2024-04-24T09:02:26.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.454+0000 7f6ed2e00640 1 -- 172.21.15.83:0/600595459 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ecc1071c0 msgr2=0x7f6ecc1a0b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:26.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.454+0000 7f6ed2e00640 1 --2- 172.21.15.83:0/600595459 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ecc1071c0 0x7f6ecc1a0b10 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f6eb800e9f0 tx=0x7f6eb800eeb0 comp rx=0 tx=0).stop 2024-04-24T09:02:26.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.454+0000 7f6ed2e00640 1 -- 172.21.15.83:0/600595459 shutdown_connections 2024-04-24T09:02:26.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.454+0000 7f6ed2e00640 1 --2- 172.21.15.83:0/600595459 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f6eb40a3da0 0x7f6eb40a6260 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:26.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.454+0000 7f6ed2e00640 1 --2- 172.21.15.83:0/600595459 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ecc19a730 0x7f6ecc19abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:26.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.454+0000 7f6ed2e00640 1 --2- 172.21.15.83:0/600595459 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ecc1071c0 0x7f6ecc1a0b10 unknown :-1 s=CLOSED pgs=1784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:26.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.454+0000 7f6ed2e00640 1 --2- 172.21.15.83:0/600595459 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6ecc1020f0 0x7f6ecc1a05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:26.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.454+0000 7f6ed2e00640 1 -- 172.21.15.83:0/600595459 >> 172.21.15.83:0/600595459 conn(0x7f6ecc0fd9d0 msgr2=0x7f6ecc0ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:02:26.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.454+0000 7f6ed2e00640 1 -- 172.21.15.83:0/600595459 shutdown_connections 2024-04-24T09:02:26.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:26.454+0000 7f6ed2e00640 1 -- 172.21.15.83:0/600595459 wait complete. 2024-04-24T09:02:26.612 INFO:tasks.workunit.client.0.smithi083.stdout: 1 63499 0.62 MB/sec execute 420 sec latency 47.397 ms 2024-04-24T09:02:27.473 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T09:02:27.474 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T09:02:27.474 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T09:02:27.613 INFO:tasks.workunit.client.0.smithi083.stdout: 1 63616 0.62 MB/sec execute 421 sec latency 51.131 ms 2024-04-24T09:02:27.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.700+0000 7faba6000640 1 -- 172.21.15.83:0/1353717906 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faba0104320 msgr2=0x7faba0104700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:27.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.700+0000 7faba6000640 1 --2- 172.21.15.83:0/1353717906 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faba0104320 0x7faba0104700 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7fab94009900 tx=0x7fab9402a3f0 comp rx=0 tx=0).stop 2024-04-24T09:02:27.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.700+0000 7faba6000640 1 -- 172.21.15.83:0/1353717906 shutdown_connections 2024-04-24T09:02:27.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.700+0000 7faba6000640 1 --2- 172.21.15.83:0/1353717906 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faba01093f0 0x7faba0110fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:27.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.700+0000 7faba6000640 1 --2- 172.21.15.83:0/1353717906 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faba0104cd0 0x7faba0108d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:27.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.700+0000 7faba6000640 1 --2- 172.21.15.83:0/1353717906 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faba0104320 0x7faba0104700 unknown :-1 s=CLOSED pgs=1789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:27.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.700+0000 7faba6000640 1 -- 172.21.15.83:0/1353717906 >> 172.21.15.83:0/1353717906 conn(0x7faba00ffb40 msgr2=0x7faba0101f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:02:27.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.700+0000 7faba6000640 1 -- 172.21.15.83:0/1353717906 shutdown_connections 2024-04-24T09:02:27.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.700+0000 7faba6000640 1 -- 172.21.15.83:0/1353717906 wait complete. 2024-04-24T09:02:27.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.701+0000 7faba6000640 1 Processor -- start 2024-04-24T09:02:27.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.701+0000 7faba6000640 1 -- start start 2024-04-24T09:02:27.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.701+0000 7faba6000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faba0104320 0x7faba01a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:27.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.702+0000 7faba6000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faba01093f0 0x7faba01a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:27.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.702+0000 7faba6000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faba019a6d0 0x7faba019ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:27.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.702+0000 7faba6000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7faba01143b0 con 0x7faba019a6d0 2024-04-24T09:02:27.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.702+0000 7faba6000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7faba0114230 con 0x7faba0104320 2024-04-24T09:02:27.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.702+0000 7faba6000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7faba0114530 con 0x7faba01093f0 2024-04-24T09:02:27.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.702+0000 7faba5600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faba019a6d0 0x7faba019ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:27.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.702+0000 7faba5600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faba019a6d0 0x7faba019ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51520/0 (socket says 172.21.15.83:51520) 2024-04-24T09:02:27.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.702+0000 7faba5600640 1 -- 172.21.15.83:0/4226811508 learned_addr learned my addr 172.21.15.83:0/4226811508 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:02:27.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.702+0000 7faba4c00640 1 --2- 172.21.15.83:0/4226811508 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faba0104320 0x7faba01a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:27.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.702+0000 7fab9fe00640 1 --2- 172.21.15.83:0/4226811508 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faba01093f0 0x7faba01a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:27.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.702+0000 7faba5600640 1 -- 172.21.15.83:0/4226811508 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faba01093f0 msgr2=0x7faba01a0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:27.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.702+0000 7faba5600640 1 --2- 172.21.15.83:0/4226811508 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faba01093f0 0x7faba01a0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:27.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.702+0000 7faba5600640 1 -- 172.21.15.83:0/4226811508 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faba0104320 msgr2=0x7faba01a0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:27.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.702+0000 7faba5600640 1 --2- 172.21.15.83:0/4226811508 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faba0104320 0x7faba01a0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:27.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.702+0000 7faba5600640 1 -- 172.21.15.83:0/4226811508 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faba019b2d0 con 0x7faba019a6d0 2024-04-24T09:02:27.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.702+0000 7faba4c00640 1 --2- 172.21.15.83:0/4226811508 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faba0104320 0x7faba01a0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:02:27.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.703+0000 7faba5600640 1 --2- 172.21.15.83:0/4226811508 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faba019a6d0 0x7faba019ab50 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7fab9800eaa0 tx=0x7fab9800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:02:27.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.703+0000 7fab9d600640 1 -- 172.21.15.83:0/4226811508 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab9800cdb0 con 0x7faba019a6d0 2024-04-24T09:02:27.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.703+0000 7fab9d600640 1 -- 172.21.15.83:0/4226811508 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fab98005020 con 0x7faba019a6d0 2024-04-24T09:02:27.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.703+0000 7faba6000640 1 -- 172.21.15.83:0/4226811508 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faba01a73b0 con 0x7faba019a6d0 2024-04-24T09:02:27.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.703+0000 7fab9d600640 1 -- 172.21.15.83:0/4226811508 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab980116d0 con 0x7faba019a6d0 2024-04-24T09:02:27.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.703+0000 7faba6000640 1 -- 172.21.15.83:0/4226811508 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faba01a77b0 con 0x7faba019a6d0 2024-04-24T09:02:27.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.707+0000 7fabaa417740 1 Processor -- start 2024-04-24T09:02:27.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.707+0000 7fab9d600640 1 -- 172.21.15.83:0/4226811508 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fab980051d0 con 0x7faba019a6d0 2024-04-24T09:02:27.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.707+0000 7fabaa417740 1 -- start start 2024-04-24T09:02:27.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.708+0000 7fabaa417740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5636c574e270 0x5636c574e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:27.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.708+0000 7fabaa417740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636c574eb90 0x5636c5758d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:27.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.708+0000 7fabaa417740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5636c5759260 0x5636c575b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:27.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.708+0000 7fabaa417740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5636c56b3e70 con 0x5636c574eb90 2024-04-24T09:02:27.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.708+0000 7fabaa417740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5636c565a380 con 0x5636c574e270 2024-04-24T09:02:27.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.708+0000 7fabaa417740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5636c56afe30 con 0x5636c5759260 2024-04-24T09:02:27.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.708+0000 7faba4c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636c574eb90 0x5636c5758d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:27.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.708+0000 7faba4c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636c574eb90 0x5636c5758d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51526/0 (socket says 172.21.15.83:51526) 2024-04-24T09:02:27.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.708+0000 7faba4c00640 1 -- 172.21.15.83:0/4058718439 learned_addr learned my addr 172.21.15.83:0/4058718439 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:02:27.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.708+0000 7fab9fe00640 1 --2- 172.21.15.83:0/4058718439 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5636c574e270 0x5636c574e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:27.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.708+0000 7fab9fe00640 1 --2- 172.21.15.83:0/4058718439 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5636c5759260 0x5636c575b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:27.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.709+0000 7faba4c00640 1 -- 172.21.15.83:0/4058718439 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5636c5759260 msgr2=0x5636c575b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:27.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.709+0000 7faba4c00640 1 --2- 172.21.15.83:0/4058718439 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5636c5759260 0x5636c575b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:27.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.709+0000 7faba4c00640 1 -- 172.21.15.83:0/4058718439 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5636c574e270 msgr2=0x5636c574e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:27.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.709+0000 7faba4c00640 1 --2- 172.21.15.83:0/4058718439 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5636c574e270 0x5636c574e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:27.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.709+0000 7faba4c00640 1 -- 172.21.15.83:0/4058718439 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636c548e7e0 con 0x5636c574eb90 2024-04-24T09:02:27.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.709+0000 7fab9fe00640 1 --2- 172.21.15.83:0/4058718439 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5636c574e270 0x5636c574e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:02:27.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.709+0000 7faba4c00640 1 --2- 172.21.15.83:0/4058718439 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636c574eb90 0x5636c5758d20 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7fab94009900 tx=0x7fab940042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=294c284e68a47fda server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:02:27.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.709+0000 7fab9d600640 1 --2- 172.21.15.83:0/4226811508 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fab880a3d20 0x7fab880a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:27.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.709+0000 7fab91c00640 1 -- 172.21.15.83:0/4058718439 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab94035030 con 0x5636c574eb90 2024-04-24T09:02:27.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.709+0000 7fab9d600640 1 -- 172.21.15.83:0/4226811508 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9583+0+0 (secure 0 0 0) 0x7fab980d0e00 con 0x7faba019a6d0 2024-04-24T09:02:27.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.709+0000 7fab9fe00640 1 --2- 172.21.15.83:0/4226811508 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fab880a3d20 0x7fab880a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:27.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.710+0000 7fab91c00640 1 -- 172.21.15.83:0/4058718439 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fab940044d0 con 0x5636c574eb90 2024-04-24T09:02:27.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.710+0000 7fab91c00640 1 -- 172.21.15.83:0/4058718439 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab94003850 con 0x5636c574eb90 2024-04-24T09:02:27.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.710+0000 7fabaa417740 1 -- 172.21.15.83:0/4058718439 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636c574eb90 msgr2=0x5636c5758d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:27.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.710+0000 7fabaa417740 1 --2- 172.21.15.83:0/4058718439 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636c574eb90 0x5636c5758d20 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7fab94009900 tx=0x7fab940042e0 comp rx=0 tx=0).stop 2024-04-24T09:02:27.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.710+0000 7fab9fe00640 1 --2- 172.21.15.83:0/4226811508 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fab880a3d20 0x7fab880a61e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x5636c574b720 tx=0x7fab8c004730 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:02:27.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.711+0000 7fabaa417740 1 -- 172.21.15.83:0/4058718439 shutdown_connections 2024-04-24T09:02:27.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.711+0000 7fabaa417740 1 --2- 172.21.15.83:0/4058718439 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5636c5759260 0x5636c575b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:27.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.711+0000 7fabaa417740 1 --2- 172.21.15.83:0/4058718439 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636c574eb90 0x5636c5758d20 unknown :-1 s=CLOSED pgs=1791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:27.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.711+0000 7fabaa417740 1 --2- 172.21.15.83:0/4058718439 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5636c574e270 0x5636c574e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:27.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.711+0000 7fabaa417740 1 -- 172.21.15.83:0/4058718439 >> 172.21.15.83:0/4058718439 conn(0x5636c574a590 msgr2=0x5636c574a970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:02:27.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.711+0000 7fabaa417740 1 -- 172.21.15.83:0/4058718439 shutdown_connections 2024-04-24T09:02:27.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.711+0000 7fabaa417740 1 -- 172.21.15.83:0/4058718439 wait complete. 2024-04-24T09:02:27.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.711+0000 7fabaa417740 1 -- start start 2024-04-24T09:02:27.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.711+0000 7fabaa417740 1 Processor -- start 2024-04-24T09:02:27.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.712+0000 7fabaa417740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636c574e270 0x5636c57561d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:27.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.712+0000 7fabaa417740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5636c574eb90 0x5636c5756710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:27.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.712+0000 7fabaa417740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5636c5759260 0x5636c5756c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:27.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.712+0000 7fabaa417740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5636c575bff0 con 0x5636c574e270 2024-04-24T09:02:27.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.712+0000 7fabaa417740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5636c566d660 con 0x5636c5759260 2024-04-24T09:02:27.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.712+0000 7fabaa417740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5636c575c170 con 0x5636c574eb90 2024-04-24T09:02:27.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.712+0000 7fabaa417740 10 client.0 _ll_register_callbacks cb 0x5636c565baa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:02:27.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.712+0000 7faba4c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636c574e270 0x5636c57561d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:27.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.712+0000 7faba4c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636c574e270 0x5636c57561d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51540/0 (socket says 172.21.15.83:51540) 2024-04-24T09:02:27.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.712+0000 7faba4c00640 1 -- 172.21.15.83:0/3671836606 learned_addr learned my addr 172.21.15.83:0/3671836606 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:02:27.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.712+0000 7fab9fe00640 1 --2- 172.21.15.83:0/3671836606 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5636c574eb90 0x5636c5756710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:27.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.713+0000 7faba5600640 1 --2- 172.21.15.83:0/3671836606 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5636c5759260 0x5636c5756c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:27.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.713+0000 7faba4c00640 1 -- 172.21.15.83:0/3671836606 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5636c574eb90 msgr2=0x5636c5756710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:27.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.713+0000 7faba4c00640 1 --2- 172.21.15.83:0/3671836606 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5636c574eb90 0x5636c5756710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:27.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.713+0000 7faba4c00640 1 -- 172.21.15.83:0/3671836606 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5636c5759260 msgr2=0x5636c5756c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:27.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.713+0000 7faba4c00640 1 --2- 172.21.15.83:0/3671836606 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5636c5759260 0x5636c5756c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:27.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.713+0000 7faba4c00640 1 -- 172.21.15.83:0/3671836606 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636c54eef70 con 0x5636c574e270 2024-04-24T09:02:27.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.713+0000 7faba4c00640 1 --2- 172.21.15.83:0/3671836606 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636c574e270 0x5636c57561d0 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7fab94000c00 tx=0x7fab94005080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4abb1013534db1be server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:02:27.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.713+0000 7fab86000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:02:27.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.713+0000 7fab86000640 1 -- 172.21.15.83:0/3671836606 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab94035070 con 0x5636c574e270 2024-04-24T09:02:27.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.713+0000 7fab86000640 1 -- 172.21.15.83:0/3671836606 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9583+0+0 (secure 0 0 0) 0x7fab94005480 con 0x5636c574e270 2024-04-24T09:02:27.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.713+0000 7fabaa417740 1 -- 172.21.15.83:0/3671836606 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab940044d0 con 0x5636c574e270 2024-04-24T09:02:27.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.714+0000 7fab86000640 1 -- 172.21.15.83:0/3671836606 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fab940314a0 con 0x5636c574e270 2024-04-24T09:02:27.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.714+0000 7fabaa417740 10 client.19536 fetch_fsmap learned FSMap version 41 2024-04-24T09:02:27.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.714+0000 7fabaa417740 1 -- 172.21.15.83:0/3671836606 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5636c5386170 con 0x5636c574e270 2024-04-24T09:02:27.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.714+0000 7fab86000640 1 -- 172.21.15.83:0/3671836606 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fab940044d0 con 0x5636c574e270 2024-04-24T09:02:27.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.714+0000 7fab86000640 1 -- 172.21.15.83:0/3671836606 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7fab9403b5e0 con 0x5636c574e270 2024-04-24T09:02:27.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.714+0000 7fabaa417740 10 client.19536 fetch_fsmap finished waiting for FSMap version 41 2024-04-24T09:02:27.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.714+0000 7fabaa417740 10 client.19536 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:02:27.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.714+0000 7fabaa417740 20 client.19536 populate_metadata read hostname 'smithi083' 2024-04-24T09:02:27.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.714+0000 7fabaa417740 1 --2- 172.21.15.83:0/3671836606 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5636c5769450 0x5636c576b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:27.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.714+0000 7fabaa417740 4 client.19536 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:02:27.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.714+0000 7fabaa417740 1 -- 172.21.15.83:0/3671836606 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5636c56c1890 con 0x5636c5769450 2024-04-24T09:02:27.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.715+0000 7fab9fe00640 1 --2- 172.21.15.83:0/3671836606 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5636c5769450 0x5636c576b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:27.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.716+0000 7fab9fe00640 1 --2- 172.21.15.83:0/3671836606 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5636c5769450 0x5636c576b840 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa848f6fbfbb997b server_cookie=3c16fbce21803621 in_seq=0 out_seq=0 2024-04-24T09:02:27.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.716+0000 7fab86000640 10 client.19536 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:02:27.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.719+0000 7fab86000640 1 -- 172.21.15.83:0/3671836606 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5636c56c1890 con 0x5636c5769450 2024-04-24T09:02:27.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.719+0000 7fab86000640 10 client.19536 handle_command_reply: tid=0 2024-04-24T09:02:27.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.720+0000 7fabaa417740 1 client.19536 shutdown 2024-04-24T09:02:27.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.720+0000 7fabaa417740 1 -- 172.21.15.83:0/3671836606 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636c574e270 msgr2=0x5636c57561d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:27.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.720+0000 7fabaa417740 1 --2- 172.21.15.83:0/3671836606 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636c574e270 0x5636c57561d0 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7fab94000c00 tx=0x7fab94005080 comp rx=0 tx=0).stop 2024-04-24T09:02:27.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.720+0000 7fabaa417740 1 -- 172.21.15.83:0/3671836606 shutdown_connections 2024-04-24T09:02:27.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.720+0000 7fabaa417740 1 --2- 172.21.15.83:0/3671836606 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5636c5769450 0x5636c576b840 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:27.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.720+0000 7fabaa417740 1 --2- 172.21.15.83:0/3671836606 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5636c5759260 0x5636c5756c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:27.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.720+0000 7fab86000640 0 client.19536 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:02:27.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.720+0000 7fabaa417740 1 --2- 172.21.15.83:0/3671836606 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5636c574eb90 0x5636c5756710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:27.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.720+0000 7fabaa417740 1 --2- 172.21.15.83:0/3671836606 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636c574e270 0x5636c57561d0 unknown :-1 s=CLOSED pgs=1792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:27.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.720+0000 7fabaa417740 1 -- 172.21.15.83:0/3671836606 >> 172.21.15.83:0/3671836606 conn(0x5636c574a590 msgr2=0x5636c5750290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:02:27.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.720+0000 7fabaa417740 1 -- 172.21.15.83:0/3671836606 shutdown_connections 2024-04-24T09:02:27.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.720+0000 7fabaa417740 1 -- 172.21.15.83:0/3671836606 wait complete. 2024-04-24T09:02:27.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.720+0000 7fabaa417740 20 client.19536 trim_cache size 0 max 16384 2024-04-24T09:02:27.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.741+0000 7fabaa417740 1 Processor -- start 2024-04-24T09:02:27.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.741+0000 7fabaa417740 1 -- start start 2024-04-24T09:02:27.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.741+0000 7fabaa417740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5636c574e270 0x5636c5764f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:27.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.741+0000 7fabaa417740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636c574eb90 0x5636c5770ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:27.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.741+0000 7fabaa417740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5636c5759260 0x5636c5771430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:27.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.741+0000 7fabaa417740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7faba01b15e0 con 0x5636c574eb90 2024-04-24T09:02:27.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.741+0000 7fabaa417740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7faba01b3240 con 0x5636c5759260 2024-04-24T09:02:27.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.741+0000 7fabaa417740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7faba01b1cb0 con 0x5636c574e270 2024-04-24T09:02:27.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.741+0000 7fab9fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636c574eb90 0x5636c5770ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:27.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.742+0000 7fab9fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636c574eb90 0x5636c5770ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51542/0 (socket says 172.21.15.83:51542) 2024-04-24T09:02:27.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.742+0000 7fab9fe00640 1 -- 172.21.15.83:0/933962168 learned_addr learned my addr 172.21.15.83:0/933962168 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:02:27.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.742+0000 7faba4c00640 1 --2- 172.21.15.83:0/933962168 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5636c574e270 0x5636c5764f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:27.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.742+0000 7fab9fe00640 1 -- 172.21.15.83:0/933962168 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5636c574e270 msgr2=0x5636c5764f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:27.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.742+0000 7fab9fe00640 1 --2- 172.21.15.83:0/933962168 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5636c574e270 0x5636c5764f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:27.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.742+0000 7fab9fe00640 1 -- 172.21.15.83:0/933962168 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5636c5759260 msgr2=0x5636c5771430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:27.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.742+0000 7fab9fe00640 1 --2- 172.21.15.83:0/933962168 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5636c5759260 0x5636c5771430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:27.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.742+0000 7fab9fe00640 1 -- 172.21.15.83:0/933962168 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636c5757130 con 0x5636c574eb90 2024-04-24T09:02:27.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.742+0000 7fab9fe00640 1 --2- 172.21.15.83:0/933962168 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636c574eb90 0x5636c5770ef0 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7fab8c00e400 tx=0x7fab8c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7558c8f65ab6cf0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:02:27.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.742+0000 7fab91200640 1 -- 172.21.15.83:0/933962168 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab8c004250 con 0x5636c574eb90 2024-04-24T09:02:27.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.743+0000 7fab91200640 1 -- 172.21.15.83:0/933962168 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fab8c030070 con 0x5636c574eb90 2024-04-24T09:02:27.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.743+0000 7fab91200640 1 -- 172.21.15.83:0/933962168 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab8c003a80 con 0x5636c574eb90 2024-04-24T09:02:27.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.743+0000 7fabaa417740 1 -- 172.21.15.83:0/933962168 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636c574eb90 msgr2=0x5636c5770ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:27.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.743+0000 7fabaa417740 1 --2- 172.21.15.83:0/933962168 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636c574eb90 0x5636c5770ef0 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7fab8c00e400 tx=0x7fab8c015000 comp rx=0 tx=0).stop 2024-04-24T09:02:27.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.743+0000 7fabaa417740 1 -- 172.21.15.83:0/933962168 shutdown_connections 2024-04-24T09:02:27.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.743+0000 7fabaa417740 1 --2- 172.21.15.83:0/933962168 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5636c5759260 0x5636c5771430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:27.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.744+0000 7fabaa417740 1 --2- 172.21.15.83:0/933962168 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636c574eb90 0x5636c5770ef0 unknown :-1 s=CLOSED pgs=1793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:27.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.744+0000 7fabaa417740 1 --2- 172.21.15.83:0/933962168 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5636c574e270 0x5636c5764f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:27.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.744+0000 7fabaa417740 1 -- 172.21.15.83:0/933962168 >> 172.21.15.83:0/933962168 conn(0x5636c574a590 msgr2=0x5636c5756a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:02:27.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.744+0000 7fabaa417740 1 -- 172.21.15.83:0/933962168 shutdown_connections 2024-04-24T09:02:27.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.744+0000 7fabaa417740 1 -- 172.21.15.83:0/933962168 wait complete. 2024-04-24T09:02:27.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.744+0000 7fabaa417740 1 -- start start 2024-04-24T09:02:27.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.744+0000 7fabaa417740 1 Processor -- start 2024-04-24T09:02:27.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.745+0000 7fabaa417740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636c574e270 0x5636c575d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:27.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.745+0000 7fabaa417740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5636c574eb90 0x5636c575dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:27.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.745+0000 7fabaa417740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5636c5759260 0x5636c57c8730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:27.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.745+0000 7fabaa417740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7faba01b2b10 con 0x5636c574e270 2024-04-24T09:02:27.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.745+0000 7fabaa417740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7faba01b23e0 con 0x5636c574eb90 2024-04-24T09:02:27.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.745+0000 7fabaa417740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7faba01af5e0 con 0x5636c5759260 2024-04-24T09:02:27.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.745+0000 7fabaa417740 10 client.0 _ll_register_callbacks cb 0x7fab94005e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:02:27.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.745+0000 7faba4c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636c574e270 0x5636c575d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:27.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.745+0000 7faba4c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636c574e270 0x5636c575d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51546/0 (socket says 172.21.15.83:51546) 2024-04-24T09:02:27.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.745+0000 7faba4c00640 1 -- 172.21.15.83:0/2247862544 learned_addr learned my addr 172.21.15.83:0/2247862544 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:02:27.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.745+0000 7faba4c00640 1 -- 172.21.15.83:0/2247862544 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5636c5759260 msgr2=0x5636c57c8730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:27.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.745+0000 7faba4c00640 1 --2- 172.21.15.83:0/2247862544 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5636c5759260 0x5636c57c8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:27.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.745+0000 7faba4c00640 1 -- 172.21.15.83:0/2247862544 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5636c574eb90 msgr2=0x5636c575dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:27.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.745+0000 7fab9fe00640 1 --2- 172.21.15.83:0/2247862544 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5636c574eb90 0x5636c575dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:27.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.746+0000 7faba4c00640 1 --2- 172.21.15.83:0/2247862544 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5636c574eb90 0x5636c575dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:27.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.746+0000 7faba4c00640 1 -- 172.21.15.83:0/2247862544 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636c57c8d70 con 0x5636c574e270 2024-04-24T09:02:27.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.746+0000 7fab9fe00640 1 --2- 172.21.15.83:0/2247862544 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5636c574eb90 0x5636c575dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:02:27.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.746+0000 7faba4c00640 1 --2- 172.21.15.83:0/2247862544 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636c574e270 0x5636c575d6b0 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7fab9402a910 tx=0x7fab94005160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9caf191fe47b0f83 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:02:27.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.746+0000 7fab86000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:02:27.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.746+0000 7fab86000640 1 -- 172.21.15.83:0/2247862544 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab94035040 con 0x5636c574e270 2024-04-24T09:02:27.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.746+0000 7fab86000640 1 -- 172.21.15.83:0/2247862544 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9583+0+0 (secure 0 0 0) 0x7fab9403b640 con 0x5636c574e270 2024-04-24T09:02:27.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.746+0000 7fabaa417740 1 -- 172.21.15.83:0/2247862544 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab8c030070 con 0x5636c574e270 2024-04-24T09:02:27.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.746+0000 7fab86000640 1 -- 172.21.15.83:0/2247862544 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fab940049d0 con 0x5636c574e270 2024-04-24T09:02:27.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.746+0000 7fabaa417740 10 client.19542 fetch_fsmap learned FSMap version 41 2024-04-24T09:02:27.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.746+0000 7fabaa417740 1 -- 172.21.15.83:0/2247862544 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5636c57d0150 con 0x5636c574e270 2024-04-24T09:02:27.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.747+0000 7fab86000640 1 -- 172.21.15.83:0/2247862544 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fab8c030070 con 0x5636c574e270 2024-04-24T09:02:27.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.747+0000 7fab86000640 1 -- 172.21.15.83:0/2247862544 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7fab9404d630 con 0x5636c574e270 2024-04-24T09:02:27.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.747+0000 7fabaa417740 10 client.19542 fetch_fsmap finished waiting for FSMap version 41 2024-04-24T09:02:27.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.747+0000 7fabaa417740 10 client.19542 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:02:27.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.747+0000 7fabaa417740 20 client.19542 populate_metadata read hostname 'smithi083' 2024-04-24T09:02:27.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.747+0000 7fabaa417740 1 --2- 172.21.15.83:0/2247862544 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5636c57d0960 0x5636c57d2d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:27.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.747+0000 7fabaa417740 4 client.19542 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T09:02:27.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.747+0000 7fabaa417740 1 -- 172.21.15.83:0/2247862544 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5636c56bb700 con 0x5636c57d0960 2024-04-24T09:02:27.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.747+0000 7fab9fe00640 1 --2- 172.21.15.83:0/2247862544 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5636c57d0960 0x5636c57d2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:27.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.748+0000 7fab9fe00640 1 --2- 172.21.15.83:0/2247862544 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5636c57d0960 0x5636c57d2d80 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6bda2c55d1fecd2 server_cookie=1f66d3bf321495b8 in_seq=0 out_seq=0 2024-04-24T09:02:27.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.748+0000 7fab86000640 10 client.19542 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:02:27.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.753+0000 7fab86000640 1 -- 172.21.15.83:0/2247862544 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5636c56bb700 con 0x5636c57d0960 2024-04-24T09:02:27.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.753+0000 7fab86000640 10 client.19542 handle_command_reply: tid=0 2024-04-24T09:02:27.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.753+0000 7fabaa417740 1 client.19542 shutdown 2024-04-24T09:02:27.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.753+0000 7fabaa417740 1 -- 172.21.15.83:0/2247862544 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636c574e270 msgr2=0x5636c575d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:27.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.753+0000 7fabaa417740 1 --2- 172.21.15.83:0/2247862544 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636c574e270 0x5636c575d6b0 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7fab9402a910 tx=0x7fab94005160 comp rx=0 tx=0).stop 2024-04-24T09:02:27.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.753+0000 7fabaa417740 1 -- 172.21.15.83:0/2247862544 shutdown_connections 2024-04-24T09:02:27.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.753+0000 7fabaa417740 1 --2- 172.21.15.83:0/2247862544 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5636c57d0960 0x5636c57d2d80 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:27.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.753+0000 7fabaa417740 1 --2- 172.21.15.83:0/2247862544 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5636c5759260 0x5636c57c8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:27.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.753+0000 7fabaa417740 1 --2- 172.21.15.83:0/2247862544 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5636c574eb90 0x5636c575dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:27.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.753+0000 7fabaa417740 1 --2- 172.21.15.83:0/2247862544 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636c574e270 0x5636c575d6b0 unknown :-1 s=CLOSED pgs=1794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:27.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.753+0000 7fab86000640 0 client.19542 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:02:27.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.753+0000 7fabaa417740 1 -- 172.21.15.83:0/2247862544 >> 172.21.15.83:0/2247862544 conn(0x5636c574a590 msgr2=0x5636c574f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:02:27.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.753+0000 7fabaa417740 1 -- 172.21.15.83:0/2247862544 shutdown_connections 2024-04-24T09:02:27.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.753+0000 7fabaa417740 1 -- 172.21.15.83:0/2247862544 wait complete. 2024-04-24T09:02:27.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.753+0000 7fabaa417740 20 client.19542 trim_cache size 0 max 16384 2024-04-24T09:02:27.753 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T09:02:27.753 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-04-24T09:02:27.754 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "da60306d-e739-44b8-a2f4-4ffafa3f26cd", 2024-04-24T09:02:27.754 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-04-24T09:02:27.754 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T09:02:27.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.754+0000 7faba6000640 1 -- 172.21.15.83:0/4226811508 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fab880a3d20 msgr2=0x7fab880a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:27.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.754+0000 7faba6000640 1 --2- 172.21.15.83:0/4226811508 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fab880a3d20 0x7fab880a61e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x5636c574b720 tx=0x7fab8c004730 comp rx=0 tx=0).stop 2024-04-24T09:02:27.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.754+0000 7faba6000640 1 -- 172.21.15.83:0/4226811508 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faba019a6d0 msgr2=0x7faba019ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:27.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.754+0000 7faba6000640 1 --2- 172.21.15.83:0/4226811508 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faba019a6d0 0x7faba019ab50 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7fab9800eaa0 tx=0x7fab9800ef60 comp rx=0 tx=0).stop 2024-04-24T09:02:27.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.754+0000 7faba6000640 1 -- 172.21.15.83:0/4226811508 shutdown_connections 2024-04-24T09:02:27.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.754+0000 7faba6000640 1 --2- 172.21.15.83:0/4226811508 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fab880a3d20 0x7fab880a61e0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:27.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.754+0000 7faba6000640 1 --2- 172.21.15.83:0/4226811508 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faba019a6d0 0x7faba019ab50 unknown :-1 s=CLOSED pgs=1790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:27.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.754+0000 7faba6000640 1 --2- 172.21.15.83:0/4226811508 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faba01093f0 0x7faba01a0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:27.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.754+0000 7faba6000640 1 --2- 172.21.15.83:0/4226811508 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faba0104320 0x7faba01a0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:27.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.754+0000 7faba6000640 1 -- 172.21.15.83:0/4226811508 >> 172.21.15.83:0/4226811508 conn(0x7faba00ffb40 msgr2=0x7faba0101e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:02:27.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.754+0000 7faba6000640 1 -- 172.21.15.83:0/4226811508 shutdown_connections 2024-04-24T09:02:27.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.754+0000 7faba6000640 1 -- 172.21.15.83:0/4226811508 wait complete. 2024-04-24T09:02:27.770 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T09:02:27.771 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T09:02:27.802 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:02:27 smithi083 ceph-mon[27907]: pgmap v340: 129 pgs: 129 active+clean; 60 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 231 KiB/s wr, 39 op/s 2024-04-24T09:02:27.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.975+0000 7fe902000640 1 -- 172.21.15.83:0/2216394379 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8fc103280 msgr2=0x7fe8fc103660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:27.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.975+0000 7fe902000640 1 --2- 172.21.15.83:0/2216394379 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8fc103280 0x7fe8fc103660 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec009900 tx=0x7fe8ec02a510 comp rx=0 tx=0).stop 2024-04-24T09:02:27.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.975+0000 7fe902000640 1 -- 172.21.15.83:0/2216394379 shutdown_connections 2024-04-24T09:02:27.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.975+0000 7fe902000640 1 --2- 172.21.15.83:0/2216394379 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe8fc10c5c0 0x7fe8fc10ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:27.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.975+0000 7fe902000640 1 --2- 172.21.15.83:0/2216394379 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe8fc103ba0 0x7fe8fc10c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:27.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.975+0000 7fe902000640 1 --2- 172.21.15.83:0/2216394379 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8fc103280 0x7fe8fc103660 unknown :-1 s=CLOSED pgs=1795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:27.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.975+0000 7fe902000640 1 -- 172.21.15.83:0/2216394379 >> 172.21.15.83:0/2216394379 conn(0x7fe8fc078000 msgr2=0x7fe8fc078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:02:27.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.975+0000 7fe902000640 1 -- 172.21.15.83:0/2216394379 shutdown_connections 2024-04-24T09:02:27.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.975+0000 7fe902000640 1 -- 172.21.15.83:0/2216394379 wait complete. 2024-04-24T09:02:27.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.976+0000 7fe902000640 1 Processor -- start 2024-04-24T09:02:27.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.976+0000 7fe902000640 1 -- start start 2024-04-24T09:02:27.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.977+0000 7fe902000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8fc103280 0x7fe8fc1a06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:27.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.977+0000 7fe902000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe8fc103ba0 0x7fe8fc1a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:27.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.977+0000 7fe902000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe8fc10c5c0 0x7fe8fc19a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:27.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.977+0000 7fe902000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe8fc112250 con 0x7fe8fc103280 2024-04-24T09:02:27.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.977+0000 7fe902000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe8fc1120d0 con 0x7fe8fc10c5c0 2024-04-24T09:02:27.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.977+0000 7fe902000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe8fc1123d0 con 0x7fe8fc103ba0 2024-04-24T09:02:27.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.977+0000 7fe8fb400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8fc103280 0x7fe8fc1a06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:27.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.977+0000 7fe8fb400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8fc103280 0x7fe8fc1a06e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51572/0 (socket says 172.21.15.83:51572) 2024-04-24T09:02:27.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.977+0000 7fe8fb400640 1 -- 172.21.15.83:0/977643562 learned_addr learned my addr 172.21.15.83:0/977643562 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:02:27.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.977+0000 7fe8f3e00640 1 --2- 172.21.15.83:0/977643562 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe8fc103ba0 0x7fe8fc1a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:27.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.977+0000 7fe8fbe00640 1 --2- 172.21.15.83:0/977643562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe8fc10c5c0 0x7fe8fc19a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:27.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.978+0000 7fe8fb400640 1 -- 172.21.15.83:0/977643562 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe8fc103ba0 msgr2=0x7fe8fc1a0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:27.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.978+0000 7fe8fb400640 1 --2- 172.21.15.83:0/977643562 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe8fc103ba0 0x7fe8fc1a0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:27.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.978+0000 7fe8fb400640 1 -- 172.21.15.83:0/977643562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe8fc10c5c0 msgr2=0x7fe8fc19a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:27.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.978+0000 7fe8fb400640 1 --2- 172.21.15.83:0/977643562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe8fc10c5c0 0x7fe8fc19a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:27.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.978+0000 7fe8fb400640 1 -- 172.21.15.83:0/977643562 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8fc19b020 con 0x7fe8fc103280 2024-04-24T09:02:27.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.978+0000 7fe8f3e00640 1 --2- 172.21.15.83:0/977643562 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe8fc103ba0 0x7fe8fc1a0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:02:27.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.978+0000 7fe8fbe00640 1 --2- 172.21.15.83:0/977643562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe8fc10c5c0 0x7fe8fc19a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:02:27.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.978+0000 7fe8fb400640 1 --2- 172.21.15.83:0/977643562 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8fc103280 0x7fe8fc1a06e0 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec005940 tx=0x7fe8ec004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:02:27.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.978+0000 7fe8f8c00640 1 -- 172.21.15.83:0/977643562 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8ec03f030 con 0x7fe8fc103280 2024-04-24T09:02:27.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.978+0000 7fe902000640 1 -- 172.21.15.83:0/977643562 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8fc19b2b0 con 0x7fe8fc103280 2024-04-24T09:02:27.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.978+0000 7fe8f8c00640 1 -- 172.21.15.83:0/977643562 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe8ec003aa0 con 0x7fe8fc103280 2024-04-24T09:02:27.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.978+0000 7fe8f8c00640 1 -- 172.21.15.83:0/977643562 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8ec035070 con 0x7fe8fc103280 2024-04-24T09:02:27.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.978+0000 7fe902000640 1 -- 172.21.15.83:0/977643562 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8fc1a7530 con 0x7fe8fc103280 2024-04-24T09:02:27.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.982+0000 7fe8f8c00640 1 -- 172.21.15.83:0/977643562 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe8ec042030 con 0x7fe8fc103280 2024-04-24T09:02:27.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.982+0000 7fe904da1740 1 Processor -- start 2024-04-24T09:02:27.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.982+0000 7fe904da1740 1 -- start start 2024-04-24T09:02:27.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.983+0000 7fe904da1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5555894bb860 0x5555894bbc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:27.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.983+0000 7fe904da1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5555894bc180 0x5555894c6310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:27.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.983+0000 7fe904da1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555894c6850 0x5555894c8c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:27.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.983+0000 7fe904da1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5555894211f0 con 0x5555894c6850 2024-04-24T09:02:27.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.983+0000 7fe904da1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5555893c7350 con 0x5555894bc180 2024-04-24T09:02:27.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.983+0000 7fe904da1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55558941d1b0 con 0x5555894bb860 2024-04-24T09:02:27.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.983+0000 7fe8f3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555894c6850 0x5555894c8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:27.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.983+0000 7fe8f3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555894c6850 0x5555894c8c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51576/0 (socket says 172.21.15.83:51576) 2024-04-24T09:02:27.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.983+0000 7fe8f3e00640 1 -- 172.21.15.83:0/3848929675 learned_addr learned my addr 172.21.15.83:0/3848929675 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:02:27.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.983+0000 7fe8fb400640 1 --2- 172.21.15.83:0/3848929675 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5555894bc180 0x5555894c6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:27.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.983+0000 7fe8fbe00640 1 --2- 172.21.15.83:0/3848929675 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5555894bb860 0x5555894bbc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:27.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.984+0000 7fe8f8c00640 1 --2- 172.21.15.83:0/977643562 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe8e00a3da0 0x7fe8e00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:27.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.984+0000 7fe8f3e00640 1 -- 172.21.15.83:0/3848929675 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5555894bb860 msgr2=0x5555894bbc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:27.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.984+0000 7fe8f3e00640 1 --2- 172.21.15.83:0/3848929675 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5555894bb860 0x5555894bbc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:27.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.984+0000 7fe8f3e00640 1 -- 172.21.15.83:0/3848929675 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5555894bc180 msgr2=0x5555894c6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:27.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.984+0000 7fe8f3e00640 1 --2- 172.21.15.83:0/3848929675 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5555894bc180 0x5555894c6310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:27.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.984+0000 7fe8f3e00640 1 -- 172.21.15.83:0/3848929675 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55558941bf50 con 0x5555894c6850 2024-04-24T09:02:27.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.984+0000 7fe8f8c00640 1 -- 172.21.15.83:0/977643562 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9583+0+0 (secure 0 0 0) 0x7fe8ec0f1200 con 0x7fe8fc103280 2024-04-24T09:02:27.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.984+0000 7fe8fbe00640 1 --2- 172.21.15.83:0/977643562 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe8e00a3da0 0x7fe8e00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:27.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.984+0000 7fe8f3e00640 1 --2- 172.21.15.83:0/3848929675 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555894c6850 0x5555894c8c40 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7fe8e4009bf0 tx=0x7fe8e400e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b601a4b9f3c793a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:02:27.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.984+0000 7fe8ebe00640 1 -- 172.21.15.83:0/3848929675 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e4016070 con 0x5555894c6850 2024-04-24T09:02:27.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.985+0000 7fe8ebe00640 1 -- 172.21.15.83:0/3848929675 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe8e4011920 con 0x5555894c6850 2024-04-24T09:02:27.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.985+0000 7fe8ebe00640 1 -- 172.21.15.83:0/3848929675 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e40107f0 con 0x5555894c6850 2024-04-24T09:02:27.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.985+0000 7fe904da1740 1 -- 172.21.15.83:0/3848929675 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555894c6850 msgr2=0x5555894c8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:27.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.985+0000 7fe8fbe00640 1 --2- 172.21.15.83:0/977643562 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe8e00a3da0 0x7fe8e00a6260 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fe8f4005fd0 tx=0x7fe8f4004a40 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:02:27.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.985+0000 7fe904da1740 1 --2- 172.21.15.83:0/3848929675 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555894c6850 0x5555894c8c40 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7fe8e4009bf0 tx=0x7fe8e400e6c0 comp rx=0 tx=0).stop 2024-04-24T09:02:27.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.985+0000 7fe904da1740 1 -- 172.21.15.83:0/3848929675 shutdown_connections 2024-04-24T09:02:27.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.985+0000 7fe904da1740 1 --2- 172.21.15.83:0/3848929675 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555894c6850 0x5555894c8c40 unknown :-1 s=CLOSED pgs=1797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:27.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.985+0000 7fe904da1740 1 --2- 172.21.15.83:0/3848929675 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5555894bc180 0x5555894c6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:27.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.985+0000 7fe904da1740 1 --2- 172.21.15.83:0/3848929675 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5555894bb860 0x5555894bbc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:27.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.985+0000 7fe904da1740 1 -- 172.21.15.83:0/3848929675 >> 172.21.15.83:0/3848929675 conn(0x5555894b7b80 msgr2=0x5555894b7f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:02:27.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.986+0000 7fe904da1740 1 -- 172.21.15.83:0/3848929675 shutdown_connections 2024-04-24T09:02:27.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.986+0000 7fe904da1740 1 -- 172.21.15.83:0/3848929675 wait complete. 2024-04-24T09:02:27.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.986+0000 7fe904da1740 1 -- start start 2024-04-24T09:02:27.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.986+0000 7fe904da1740 1 Processor -- start 2024-04-24T09:02:27.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.987+0000 7fe904da1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5555894bb860 0x5555894c3720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:27.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.987+0000 7fe904da1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555894bc180 0x5555894c3c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:27.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.987+0000 7fe904da1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5555894c6850 0x5555894c41a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:27.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.987+0000 7fe904da1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5555894c9640 con 0x5555894bc180 2024-04-24T09:02:27.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.987+0000 7fe904da1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5555893da690 con 0x5555894bb860 2024-04-24T09:02:27.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.987+0000 7fe904da1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5555894c97c0 con 0x5555894c6850 2024-04-24T09:02:27.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.987+0000 7fe904da1740 10 client.0 _ll_register_callbacks cb 0x555589416650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:02:27.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.987+0000 7fe8f3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555894bc180 0x5555894c3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:27.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.987+0000 7fe8f3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555894bc180 0x5555894c3c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51592/0 (socket says 172.21.15.83:51592) 2024-04-24T09:02:27.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.987+0000 7fe8f3e00640 1 -- 172.21.15.83:0/3973531293 learned_addr learned my addr 172.21.15.83:0/3973531293 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:02:27.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.987+0000 7fe8fb400640 1 --2- 172.21.15.83:0/3973531293 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5555894bb860 0x5555894c3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:27.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.988+0000 7fe8fbe00640 1 --2- 172.21.15.83:0/3973531293 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5555894c6850 0x5555894c41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:27.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.988+0000 7fe8f3e00640 1 -- 172.21.15.83:0/3973531293 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5555894c6850 msgr2=0x5555894c41a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:28.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.988+0000 7fe8f3e00640 1 --2- 172.21.15.83:0/3973531293 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5555894c6850 0x5555894c41a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:28.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.988+0000 7fe8f3e00640 1 -- 172.21.15.83:0/3973531293 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5555894bb860 msgr2=0x5555894c3720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:28.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.988+0000 7fe8f3e00640 1 --2- 172.21.15.83:0/3973531293 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5555894bb860 0x5555894c3720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:28.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.988+0000 7fe8f3e00640 1 -- 172.21.15.83:0/3973531293 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555893d9920 con 0x5555894bc180 2024-04-24T09:02:28.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.988+0000 7fe8fbe00640 1 --2- 172.21.15.83:0/3973531293 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5555894c6850 0x5555894c41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:02:28.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.988+0000 7fe8f3e00640 1 --2- 172.21.15.83:0/3973531293 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555894bc180 0x5555894c3c60 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7fe8e40098a0 tx=0x7fe8e4005900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1f0ea53f790201e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:02:28.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.988+0000 7fe8e8c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:02:28.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.988+0000 7fe8e8c00640 1 -- 172.21.15.83:0/3973531293 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e4016030 con 0x5555894bc180 2024-04-24T09:02:28.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.988+0000 7fe8e8c00640 1 -- 172.21.15.83:0/3973531293 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9583+0+0 (secure 0 0 0) 0x7fe8e4005050 con 0x5555894bc180 2024-04-24T09:02:28.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.988+0000 7fe904da1740 1 -- 172.21.15.83:0/3973531293 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8e4011920 con 0x5555894bc180 2024-04-24T09:02:28.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.989+0000 7fe8e8c00640 1 -- 172.21.15.83:0/3973531293 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fe8e4005250 con 0x5555894bc180 2024-04-24T09:02:28.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.989+0000 7fe904da1740 10 client.19554 fetch_fsmap learned FSMap version 41 2024-04-24T09:02:28.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.989+0000 7fe904da1740 1 -- 172.21.15.83:0/3973531293 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5555891fd7d0 con 0x5555894bc180 2024-04-24T09:02:28.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.989+0000 7fe8e8c00640 1 -- 172.21.15.83:0/3973531293 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe8e4011920 con 0x5555894bc180 2024-04-24T09:02:28.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.989+0000 7fe8e8c00640 1 -- 172.21.15.83:0/3973531293 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7fe8e400f040 con 0x5555894bc180 2024-04-24T09:02:28.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.989+0000 7fe904da1740 10 client.19554 fetch_fsmap finished waiting for FSMap version 41 2024-04-24T09:02:28.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.989+0000 7fe904da1740 10 client.19554 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:02:28.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.989+0000 7fe904da1740 20 client.19554 populate_metadata read hostname 'smithi083' 2024-04-24T09:02:28.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.989+0000 7fe904da1740 1 --2- 172.21.15.83:0/3973531293 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5555894d69f0 0x5555894d8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:28.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.989+0000 7fe904da1740 4 client.19554 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:02:28.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.989+0000 7fe904da1740 1 -- 172.21.15.83:0/3973531293 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55558942ec10 con 0x5555894d69f0 2024-04-24T09:02:28.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.990+0000 7fe8fb400640 1 --2- 172.21.15.83:0/3973531293 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5555894d69f0 0x5555894d8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:28.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.991+0000 7fe8fb400640 1 --2- 172.21.15.83:0/3973531293 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5555894d69f0 0x5555894d8e00 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec93821fde3dc20e server_cookie=aa68abf55242cfce in_seq=0 out_seq=0 2024-04-24T09:02:28.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.991+0000 7fe8e8c00640 10 client.19554 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:02:28.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.993+0000 7fe8e8c00640 1 -- 172.21.15.83:0/3973531293 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55558942ec10 con 0x5555894d69f0 2024-04-24T09:02:28.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.993+0000 7fe8e8c00640 10 client.19554 handle_command_reply: tid=0 2024-04-24T09:02:28.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.993+0000 7fe904da1740 1 client.19554 shutdown 2024-04-24T09:02:28.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.994+0000 7fe904da1740 1 -- 172.21.15.83:0/3973531293 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555894bc180 msgr2=0x5555894c3c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:28.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.994+0000 7fe904da1740 1 --2- 172.21.15.83:0/3973531293 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555894bc180 0x5555894c3c60 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7fe8e40098a0 tx=0x7fe8e4005900 comp rx=0 tx=0).stop 2024-04-24T09:02:28.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.994+0000 7fe904da1740 1 -- 172.21.15.83:0/3973531293 shutdown_connections 2024-04-24T09:02:28.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.994+0000 7fe904da1740 1 --2- 172.21.15.83:0/3973531293 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5555894d69f0 0x5555894d8e00 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:28.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.994+0000 7fe904da1740 1 --2- 172.21.15.83:0/3973531293 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5555894c6850 0x5555894c41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:28.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.994+0000 7fe904da1740 1 --2- 172.21.15.83:0/3973531293 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555894bc180 0x5555894c3c60 unknown :-1 s=CLOSED pgs=1798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:28.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.994+0000 7fe904da1740 1 --2- 172.21.15.83:0/3973531293 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5555894bb860 0x5555894c3720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:28.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.994+0000 7fe904da1740 1 -- 172.21.15.83:0/3973531293 >> 172.21.15.83:0/3973531293 conn(0x5555894b7b80 msgr2=0x5555894bd980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:02:28.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.994+0000 7fe8e8c00640 0 client.19554 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:02:28.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.994+0000 7fe904da1740 1 -- 172.21.15.83:0/3973531293 shutdown_connections 2024-04-24T09:02:28.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.994+0000 7fe904da1740 1 -- 172.21.15.83:0/3973531293 wait complete. 2024-04-24T09:02:28.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:27.994+0000 7fe904da1740 20 client.19554 trim_cache size 0 max 16384 2024-04-24T09:02:28.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:28.014+0000 7fe904da1740 1 Processor -- start 2024-04-24T09:02:28.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:28.015+0000 7fe904da1740 1 -- start start 2024-04-24T09:02:28.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:28.015+0000 7fe904da1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555894bb860 0x5555894d2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:28.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:28.015+0000 7fe904da1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5555894bc180 0x5555894be000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:28.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:28.015+0000 7fe904da1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5555894c6850 0x5555894b6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:28.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:28.015+0000 7fe904da1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe8fc1b1850 con 0x5555894bb860 2024-04-24T09:02:28.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:28.015+0000 7fe904da1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe8fc1b34b0 con 0x5555894c6850 2024-04-24T09:02:28.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:28.015+0000 7fe904da1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe8fc1b1f20 con 0x5555894bc180 2024-04-24T09:02:28.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:28.015+0000 7fe8fb400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555894bb860 0x5555894d2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:28.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:28.015+0000 7fe8fb400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555894bb860 0x5555894d2500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51608/0 (socket says 172.21.15.83:51608) 2024-04-24T09:02:28.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:28.015+0000 7fe8fb400640 1 -- 172.21.15.83:0/3409459797 learned_addr learned my addr 172.21.15.83:0/3409459797 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:02:28.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:28.016+0000 7fe8fb400640 1 -- 172.21.15.83:0/3409459797 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5555894bc180 msgr2=0x5555894be000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:28.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:28.016+0000 7fe8fbe00640 1 --2- 172.21.15.83:0/3409459797 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5555894c6850 0x5555894b6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:28.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:28.016+0000 7fe8f3e00640 1 --2- 172.21.15.83:0/3409459797 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5555894bc180 0x5555894be000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:28.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:28.016+0000 7fe8fb400640 1 --2- 172.21.15.83:0/3409459797 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5555894bc180 0x5555894be000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:28.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:28.016+0000 7fe8fb400640 1 -- 172.21.15.83:0/3409459797 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5555894c6850 msgr2=0x5555894b6aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:28.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:28.016+0000 7fe8fb400640 1 --2- 172.21.15.83:0/3409459797 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5555894c6850 0x5555894b6aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:28.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:28.016+0000 7fe8fb400640 1 -- 172.21.15.83:0/3409459797 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555891333f0 con 0x5555894bb860 2024-04-24T09:02:28.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:28.016+0000 7fe8fbe00640 1 --2- 172.21.15.83:0/3409459797 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5555894c6850 0x5555894b6aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:02:28.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:28.016+0000 7fe8f3e00640 1 --2- 172.21.15.83:0/3409459797 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5555894bc180 0x5555894be000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:02:28.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:28.016+0000 7fe8fb400640 1 --2- 172.21.15.83:0/3409459797 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555894bb860 0x5555894d2500 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec0bcd10 tx=0x7fe8ec02a510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cabf8c05200868e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:02:28.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:28.016+0000 7fe8eb400640 1 -- 172.21.15.83:0/3409459797 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8ec049040 con 0x5555894bb860 2024-04-24T09:02:28.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:28.016+0000 7fe8eb400640 1 -- 172.21.15.83:0/3409459797 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe8ec03b460 con 0x5555894bb860 2024-04-24T09:02:28.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:28.016+0000 7fe8eb400640 1 -- 172.21.15.83:0/3409459797 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8ec003c50 con 0x5555894bb860 2024-04-24T09:02:28.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:28.017+0000 7fe904da1740 1 -- 172.21.15.83:0/3409459797 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555894bb860 msgr2=0x5555894d2500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:28.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:28.017+0000 7fe904da1740 1 --2- 172.21.15.83:0/3409459797 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555894bb860 0x5555894d2500 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec0bcd10 tx=0x7fe8ec02a510 comp rx=0 tx=0).stop 2024-04-24T09:02:28.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:28.017+0000 7fe904da1740 1 -- 172.21.15.83:0/3409459797 shutdown_connections 2024-04-24T09:02:28.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:28.017+0000 7fe904da1740 1 --2- 172.21.15.83:0/3409459797 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5555894c6850 0x5555894b6aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:28.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:28.017+0000 7fe904da1740 1 --2- 172.21.15.83:0/3409459797 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5555894bc180 0x5555894be000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:28.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:28.017+0000 7fe904da1740 1 --2- 172.21.15.83:0/3409459797 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555894bb860 0x5555894d2500 unknown :-1 s=CLOSED pgs=1799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:28.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:28.017+0000 7fe904da1740 1 -- 172.21.15.83:0/3409459797 >> 172.21.15.83:0/3409459797 conn(0x5555894b7b80 msgr2=0x5555894b6390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:02:28.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:28.017+0000 7fe904da1740 1 -- 172.21.15.83:0/3409459797 shutdown_connections 2024-04-24T09:02:28.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:28.017+0000 7fe904da1740 1 -- 172.21.15.83:0/3409459797 wait complete. 2024-04-24T09:02:28.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:28.018+0000 7fe904da1740 1 -- start start 2024-04-24T09:02:28.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:28.018+0000 7fe904da1740 1 Processor -- start 2024-04-24T09:02:28.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:28.018+0000 7fe904da1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5555894bb860 0x5555894cede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:28.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:28.018+0000 7fe904da1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5555894bc180 0x5555894ca100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:28.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:28.018+0000 7fe904da1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555894c6850 0x5555894ca640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:28.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:28.018+0000 7fe904da1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe8fc1b2d80 con 0x5555894c6850 2024-04-24T09:02:28.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:28.018+0000 7fe904da1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe8fc1b2650 con 0x5555894bc180 2024-04-24T09:02:28.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:28.018+0000 7fe904da1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe8fc1af850 con 0x5555894bb860 2024-04-24T09:02:28.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:28.018+0000 7fe904da1740 10 client.0 _ll_register_callbacks cb 0x7fe8e4009a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:02:28.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:28.018+0000 7fe8fbe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555894c6850 0x5555894ca640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:28.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:28.019+0000 7fe8fbe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555894c6850 0x5555894ca640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51618/0 (socket says 172.21.15.83:51618) 2024-04-24T09:02:28.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:28.019+0000 7fe8fbe00640 1 -- 172.21.15.83:0/2327978320 learned_addr learned my addr 172.21.15.83:0/2327978320 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:02:28.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:28.019+0000 7fe8f3e00640 1 --2- 172.21.15.83:0/2327978320 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5555894bc180 0x5555894ca100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:28.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:28.019+0000 7fe8fb400640 1 --2- 172.21.15.83:0/2327978320 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5555894bb860 0x5555894cede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:28.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:28.019+0000 7fe8fbe00640 1 -- 172.21.15.83:0/2327978320 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5555894bb860 msgr2=0x5555894cede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:28.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:28.019+0000 7fe8fbe00640 1 --2- 172.21.15.83:0/2327978320 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5555894bb860 0x5555894cede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:28.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:28.019+0000 7fe8fbe00640 1 -- 172.21.15.83:0/2327978320 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5555894bc180 msgr2=0x5555894ca100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:28.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:28.019+0000 7fe8fbe00640 1 --2- 172.21.15.83:0/2327978320 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5555894bc180 0x5555894ca100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:28.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:28.019+0000 7fe8fbe00640 1 -- 172.21.15.83:0/2327978320 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555890f30f0 con 0x5555894c6850 2024-04-24T09:02:28.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:28.019+0000 7fe8fbe00640 1 --2- 172.21.15.83:0/2327978320 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555894c6850 0x5555894ca640 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7fe8f401c990 tx=0x7fe8f4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69624eba44b5a41b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:02:28.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:28.019+0000 7fe8e8c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:02:28.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:28.020+0000 7fe8e8c00640 1 -- 172.21.15.83:0/2327978320 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8f401c2e0 con 0x5555894c6850 2024-04-24T09:02:28.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:28.020+0000 7fe8e8c00640 1 -- 172.21.15.83:0/2327978320 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9583+0+0 (secure 0 0 0) 0x7fe8f40128f0 con 0x5555894c6850 2024-04-24T09:02:28.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:28.020+0000 7fe904da1740 1 -- 172.21.15.83:0/2327978320 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555589428a80 con 0x5555894c6850 2024-04-24T09:02:28.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:28.020+0000 7fe8e8c00640 1 -- 172.21.15.83:0/2327978320 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fe8f4012af0 con 0x5555894c6850 2024-04-24T09:02:28.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:28.020+0000 7fe904da1740 10 client.19560 fetch_fsmap learned FSMap version 41 2024-04-24T09:02:28.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:28.020+0000 7fe904da1740 1 -- 172.21.15.83:0/2327978320 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5555893e6810 con 0x5555894c6850 2024-04-24T09:02:28.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:28.020+0000 7fe8e8c00640 1 -- 172.21.15.83:0/2327978320 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x555589428a80 con 0x5555894c6850 2024-04-24T09:02:28.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:28.021+0000 7fe8e8c00640 1 -- 172.21.15.83:0/2327978320 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7fe8f401a420 con 0x5555894c6850 2024-04-24T09:02:28.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:28.021+0000 7fe904da1740 10 client.19560 fetch_fsmap finished waiting for FSMap version 41 2024-04-24T09:02:28.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:28.021+0000 7fe904da1740 10 client.19560 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:02:28.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:28.021+0000 7fe904da1740 20 client.19560 populate_metadata read hostname 'smithi083' 2024-04-24T09:02:28.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:28.021+0000 7fe904da1740 1 --2- 172.21.15.83:0/2327978320 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55558953d3e0 0x55558953f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:28.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:28.021+0000 7fe904da1740 4 client.19560 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T09:02:28.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:28.021+0000 7fe904da1740 1 -- 172.21.15.83:0/2327978320 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5555893a4360 con 0x55558953d3e0 2024-04-24T09:02:28.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:28.021+0000 7fe8fb400640 1 --2- 172.21.15.83:0/2327978320 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55558953d3e0 0x55558953f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:28.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:28.022+0000 7fe8fb400640 1 --2- 172.21.15.83:0/2327978320 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55558953d3e0 0x55558953f800 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42cd0523b8e5f1e1 server_cookie=16071acf80c646ba in_seq=0 out_seq=0 2024-04-24T09:02:28.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:28.022+0000 7fe8e8c00640 10 client.19560 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:02:28.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:28.024+0000 7fe8e8c00640 1 -- 172.21.15.83:0/2327978320 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5555893a4360 con 0x55558953d3e0 2024-04-24T09:02:28.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:28.024+0000 7fe8e8c00640 10 client.19560 handle_command_reply: tid=0 2024-04-24T09:02:28.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:28.024+0000 7fe904da1740 1 client.19560 shutdown 2024-04-24T09:02:28.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:28.024+0000 7fe904da1740 1 -- 172.21.15.83:0/2327978320 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555894c6850 msgr2=0x5555894ca640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:28.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:28.024+0000 7fe904da1740 1 --2- 172.21.15.83:0/2327978320 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555894c6850 0x5555894ca640 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7fe8f401c990 tx=0x7fe8f4004ab0 comp rx=0 tx=0).stop 2024-04-24T09:02:28.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:28.024+0000 7fe904da1740 1 -- 172.21.15.83:0/2327978320 shutdown_connections 2024-04-24T09:02:28.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:28.024+0000 7fe904da1740 1 --2- 172.21.15.83:0/2327978320 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55558953d3e0 0x55558953f800 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:28.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:28.024+0000 7fe904da1740 1 --2- 172.21.15.83:0/2327978320 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5555894c6850 0x5555894ca640 unknown :-1 s=CLOSED pgs=1800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:28.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:28.024+0000 7fe904da1740 1 --2- 172.21.15.83:0/2327978320 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5555894bc180 0x5555894ca100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:28.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:28.024+0000 7fe904da1740 1 --2- 172.21.15.83:0/2327978320 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5555894bb860 0x5555894cede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:28.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:28.024+0000 7fe904da1740 1 -- 172.21.15.83:0/2327978320 >> 172.21.15.83:0/2327978320 conn(0x5555894b7b80 msgr2=0x5555894b6c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:02:28.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:28.024+0000 7fe8e8c00640 0 client.19560 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:02:28.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:28.024+0000 7fe904da1740 1 -- 172.21.15.83:0/2327978320 shutdown_connections 2024-04-24T09:02:28.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:28.024+0000 7fe904da1740 1 -- 172.21.15.83:0/2327978320 wait complete. 2024-04-24T09:02:28.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:28.025+0000 7fe904da1740 20 client.19560 trim_cache size 0 max 16384 2024-04-24T09:02:28.027 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T09:02:28.027 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T09:02:28.027 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-04-24T09:02:28.028 INFO:teuthology.orchestra.run.smithi083.stdout: "da60306d-e739-44b8-a2f4-4ffafa3f26cd": { 2024-04-24T09:02:28.028 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-04-24T09:02:28.028 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "da60306d-e739-44b8-a2f4-4ffafa3f26cd", 2024-04-24T09:02:28.028 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-04-24T09:02:28.028 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:02:28.028 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:02:28.028 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T09:02:28.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:28.025+0000 7fe902000640 1 -- 172.21.15.83:0/977643562 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe8e00a3da0 msgr2=0x7fe8e00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:28.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:28.025+0000 7fe902000640 1 --2- 172.21.15.83:0/977643562 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe8e00a3da0 0x7fe8e00a6260 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fe8f4005fd0 tx=0x7fe8f4004a40 comp rx=0 tx=0).stop 2024-04-24T09:02:28.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:28.025+0000 7fe902000640 1 -- 172.21.15.83:0/977643562 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8fc103280 msgr2=0x7fe8fc1a06e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:28.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:28.025+0000 7fe902000640 1 --2- 172.21.15.83:0/977643562 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8fc103280 0x7fe8fc1a06e0 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec005940 tx=0x7fe8ec004270 comp rx=0 tx=0).stop 2024-04-24T09:02:28.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:28.025+0000 7fe902000640 1 -- 172.21.15.83:0/977643562 shutdown_connections 2024-04-24T09:02:28.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:28.025+0000 7fe902000640 1 --2- 172.21.15.83:0/977643562 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe8e00a3da0 0x7fe8e00a6260 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:28.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:28.025+0000 7fe902000640 1 --2- 172.21.15.83:0/977643562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe8fc10c5c0 0x7fe8fc19a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:28.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:28.025+0000 7fe902000640 1 --2- 172.21.15.83:0/977643562 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe8fc103ba0 0x7fe8fc1a0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:28.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:28.025+0000 7fe902000640 1 --2- 172.21.15.83:0/977643562 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8fc103280 0x7fe8fc1a06e0 unknown :-1 s=CLOSED pgs=1796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:28.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:28.025+0000 7fe902000640 1 -- 172.21.15.83:0/977643562 >> 172.21.15.83:0/977643562 conn(0x7fe8fc078000 msgr2=0x7fe8fc1084f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:02:28.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:28.025+0000 7fe902000640 1 -- 172.21.15.83:0/977643562 shutdown_connections 2024-04-24T09:02:28.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:28.026+0000 7fe902000640 1 -- 172.21.15.83:0/977643562 wait complete. 2024-04-24T09:02:28.042 INFO:tasks.cephfs.filesystem:scrub status for tag:da60306d-e739-44b8-a2f4-4ffafa3f26cd - {'path': '/', 'tag': 'da60306d-e739-44b8-a2f4-4ffafa3f26cd', 'options': 'recursive,force'} 2024-04-24T09:02:28.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:02:27 smithi138 ceph-mon[31441]: pgmap v340: 129 pgs: 129 active+clean; 60 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 231 KiB/s wr, 39 op/s 2024-04-24T09:02:28.205 INFO:journalctl@ceph.mds.b.smithi138.stdout:Apr 24 09:02:27 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-b[61515]: 2024-04-24T09:02:27.795+0000 7f7fbbc00700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000208 [...12,head] /volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/WORD/ auth{2=1} v10845 ap=2 f(v408 m2024-04-24T09:02:13.499237+0000 5=5+0) n(v38 rc2024-04-24T09:02:13.499237+0000 b814326 6=5+1) old_inodes=6 (inest sync->mix g=2) caps={15603=pAsLsXsFs/-@9117} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x55cf87274c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000000208:[<0x10000000009/WORD v427>,<0x10000000008/~dmtmp v57>,<0x10000000007/client0 v33>,<0x10000000006/clients v23>,<0x10000000005/tmp v23>,<0x10000000003/client.0 v22>,<0x10000000002/e395bed6-5bba-41b8-a256-3505e4afcca2 v36>,<0x10000000001/sv_0 v55>,<0x10000000000/qa v43>,<0x1/volumes v39>]//[]","memoryvalue":"(2)0x10000000208:[<0x10000000009/WORD v10845>,<0x10000000008/~dmtmp v10271>,<0x10000000007/client0 v1205>,<0x10000000006/clients v1196>,<0x10000000005/tmp v1178>,<0x10000000003/client.0 v1173>,<0x10000000002/e395bed6-5bba-41b8-a256-3505e4afcca2 v1167>,<0x10000000001/sv_0 v1161>,<0x10000000000/qa v1142>,<0x1/volumes v1107>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-04-24T09:02:13.452563+0000 6=6+0)","ondisk_value.rstat":"n(v0 rc2024-04-24T09:02:13.499237+0000 b814326 6=5+1)","memory_value.dirstat":"f(v408 m2024-04-24T09:02:13.499237+0000 5=5+0)","memory_value.rstat":"n(v38 rc2024-04-24T09:02:13.499237+0000 b814326 6=5+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-04-24T09:02:28.206 INFO:journalctl@ceph.mds.b.smithi138.stdout:Apr 24 09:02:27 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-b[61515]: 2024-04-24T09:02:27.811+0000 7f7fbbc00700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000225 [...12,head] /volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/PM/ auth{3=1} v10805 ap=2 f(v1258 m2024-04-24T09:02:07.585268+0000 3=3+0) n(v34 rc2024-04-24T09:02:07.585268+0000 b780288 4=3+1) old_inodes=6 (inest sync->mix g=3) caps={15603=pAsLsXsFs/-@12392} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x55cf8762f180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000000225:[<0x10000000009/PM v381>,<0x10000000008/~dmtmp v57>,<0x10000000007/client0 v33>,<0x10000000006/clients v23>,<0x10000000005/tmp v23>,<0x10000000003/client.0 v22>,<0x10000000002/e395bed6-5bba-41b8-a256-3505e4afcca2 v36>,<0x10000000001/sv_0 v55>,<0x10000000000/qa v43>,<0x1/volumes v39>]//[]","memoryvalue":"(2)0x10000000225:[<0x10000000009/PM v10805>,<0x10000000008/~dmtmp v10277>,<0x10000000007/client0 v1207>,<0x10000000006/clients v1198>,<0x10000000005/tmp v1180>,<0x10000000003/client.0 v1175>,<0x10000000002/e395bed6-5bba-41b8-a256-3505e4afcca2 v1169>,<0x10000000001/sv_0 v1163>,<0x10000000000/qa v1144>,<0x1/volumes v1109>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-04-24T09:02:06.471349+0000 4=4+0)","ondisk_value.rstat":"n(v0 rc2024-04-24T09:02:07.585268+0000 b780288 4=3+1)","memory_value.dirstat":"f(v1258 m2024-04-24T09:02:07.585268+0000 3=3+0)","memory_value.rstat":"n(v34 rc2024-04-24T09:02:07.585268+0000 b780288 4=3+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-04-24T09:02:28.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:02:27 smithi189 ceph-mon[31791]: pgmap v340: 129 pgs: 129 active+clean; 60 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 231 KiB/s wr, 39 op/s 2024-04-24T09:02:28.613 INFO:tasks.workunit.client.0.smithi083.stdout: 1 63649 0.62 MB/sec execute 422 sec latency 69.686 ms 2024-04-24T09:02:29.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:02:28 smithi083 ceph-mon[27907]: scrub queued for path: / 2024-04-24T09:02:29.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:02:28 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:02:29.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:02:28 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:02:29.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:02:28 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:02:29.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:02:28 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:02:29.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:02:28 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:02:29.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:02:28 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:02:29.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:02:28 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:02:29.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:02:28 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:02:29.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:02:28 smithi083 ceph-mon[27907]: Scrub error on inode 0x10000000208 (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/WORD) see mds.b log and `damage ls` output for details 2024-04-24T09:02:29.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:02:28 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:02:29.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:02:28 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:02:29.054 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:02:28 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:02:29.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:02:28 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:02:29.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:02:28 smithi083 ceph-mon[27907]: Scrub error on inode 0x10000000225 (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/PM) see mds.b log and `damage ls` output for details 2024-04-24T09:02:29.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:02:28 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:02:29.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:02:28 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:02:29.055 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:02:28 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:02:29.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:02:28 smithi138 ceph-mon[31441]: scrub queued for path: / 2024-04-24T09:02:29.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:02:28 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:02:29.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:02:28 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:02:29.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:02:28 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:02:29.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:02:28 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:02:29.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:02:28 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:02:29.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:02:28 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:02:29.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:02:28 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:02:29.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:02:28 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:02:29.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:02:28 smithi138 ceph-mon[31441]: Scrub error on inode 0x10000000208 (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/WORD) see mds.b log and `damage ls` output for details 2024-04-24T09:02:29.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:02:28 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:02:29.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:02:28 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:02:29.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:02:28 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:02:29.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:02:28 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:02:29.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:02:28 smithi138 ceph-mon[31441]: Scrub error on inode 0x10000000225 (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/PM) see mds.b log and `damage ls` output for details 2024-04-24T09:02:29.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:02:28 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:02:29.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:02:28 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:02:29.207 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:02:28 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:02:29.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:02:28 smithi189 ceph-mon[31791]: scrub queued for path: / 2024-04-24T09:02:29.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:02:28 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:02:29.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:02:28 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:02:29.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:02:28 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:02:29.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:02:28 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:02:29.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:02:28 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:02:29.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:02:28 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:02:29.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:02:28 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:02:29.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:02:28 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:02:29.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:02:28 smithi189 ceph-mon[31791]: Scrub error on inode 0x10000000208 (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/WORD) see mds.b log and `damage ls` output for details 2024-04-24T09:02:29.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:02:28 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:02:29.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:02:28 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:02:29.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:02:28 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:02:29.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:02:28 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:02:29.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:02:28 smithi189 ceph-mon[31791]: Scrub error on inode 0x10000000225 (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/PM) see mds.b log and `damage ls` output for details 2024-04-24T09:02:29.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:02:28 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:02:29.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:02:28 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:02:29.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:02:28 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:02:29.613 INFO:tasks.workunit.client.0.smithi083.stdout: 1 63702 0.62 MB/sec execute 423 sec latency 50.726 ms 2024-04-24T09:02:30.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:02:29 smithi138 ceph-mon[31441]: pgmap v341: 129 pgs: 129 active+clean; 63 MiB data, 885 MiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 526 KiB/s wr, 66 op/s 2024-04-24T09:02:30.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:02:29 smithi189 ceph-mon[31791]: pgmap v341: 129 pgs: 129 active+clean; 63 MiB data, 885 MiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 526 KiB/s wr, 66 op/s 2024-04-24T09:02:30.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:02:29 smithi083 ceph-mon[27907]: pgmap v341: 129 pgs: 129 active+clean; 63 MiB data, 885 MiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 526 KiB/s wr, 66 op/s 2024-04-24T09:02:30.613 INFO:tasks.workunit.client.0.smithi083.stdout: 1 63879 0.63 MB/sec execute 424 sec latency 51.185 ms 2024-04-24T09:02:31.613 INFO:tasks.workunit.client.0.smithi083.stdout: 1 63943 0.63 MB/sec execute 425 sec latency 590.797 ms 2024-04-24T09:02:32.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:02:31 smithi138 ceph-mon[31441]: pgmap v342: 129 pgs: 129 active+clean; 63 MiB data, 885 MiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 436 KiB/s wr, 44 op/s 2024-04-24T09:02:32.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:02:31 smithi189 ceph-mon[31791]: pgmap v342: 129 pgs: 129 active+clean; 63 MiB data, 885 MiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 436 KiB/s wr, 44 op/s 2024-04-24T09:02:32.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:02:31 smithi083 ceph-mon[27907]: pgmap v342: 129 pgs: 129 active+clean; 63 MiB data, 885 MiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 436 KiB/s wr, 44 op/s 2024-04-24T09:02:32.613 INFO:tasks.workunit.client.0.smithi083.stdout: 1 63943 0.62 MB/sec execute 426 sec latency 1590.962 ms 2024-04-24T09:02:33.613 INFO:tasks.workunit.client.0.smithi083.stdout: 1 63979 0.62 MB/sec execute 427 sec latency 2087.648 ms 2024-04-24T09:02:34.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:02:33 smithi138 ceph-mon[31441]: pgmap v343: 129 pgs: 129 active+clean; 63 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 569 KiB/s wr, 67 op/s 2024-04-24T09:02:34.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:02:33 smithi189 ceph-mon[31791]: pgmap v343: 129 pgs: 129 active+clean; 63 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 569 KiB/s wr, 67 op/s 2024-04-24T09:02:34.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:02:33 smithi083 ceph-mon[27907]: pgmap v343: 129 pgs: 129 active+clean; 63 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 569 KiB/s wr, 67 op/s 2024-04-24T09:02:34.614 INFO:tasks.workunit.client.0.smithi083.stdout: 1 64010 0.62 MB/sec execute 428 sec latency 536.388 ms 2024-04-24T09:02:35.614 INFO:tasks.workunit.client.0.smithi083.stdout: 1 64010 0.62 MB/sec execute 429 sec latency 1536.487 ms 2024-04-24T09:02:36.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:02:35 smithi138 ceph-mon[31441]: pgmap v344: 129 pgs: 129 active+clean; 63 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 428 KiB/s wr, 49 op/s 2024-04-24T09:02:36.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:02:35 smithi189 ceph-mon[31791]: pgmap v344: 129 pgs: 129 active+clean; 63 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 428 KiB/s wr, 49 op/s 2024-04-24T09:02:36.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:02:35 smithi083 ceph-mon[27907]: pgmap v344: 129 pgs: 129 active+clean; 63 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 428 KiB/s wr, 49 op/s 2024-04-24T09:02:36.614 INFO:tasks.workunit.client.0.smithi083.stdout: 1 64010 0.62 MB/sec execute 430 sec latency 2536.582 ms 2024-04-24T09:02:37.614 INFO:tasks.workunit.client.0.smithi083.stdout: 1 64010 0.62 MB/sec execute 431 sec latency 3536.689 ms 2024-04-24T09:02:38.099 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.098+0000 7f737a400640 1 -- 172.21.15.83:0/1035952612 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7374108540 msgr2=0x7f7374108980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:38.099 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.098+0000 7f737a400640 1 --2- 172.21.15.83:0/1035952612 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7374108540 0x7f7374108980 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7f7360009930 tx=0x7f736002a420 comp rx=0 tx=0).stop 2024-04-24T09:02:38.100 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.098+0000 7f737a400640 1 -- 172.21.15.83:0/1035952612 shutdown_connections 2024-04-24T09:02:38.100 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.098+0000 7f737a400640 1 --2- 172.21.15.83:0/1035952612 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7374108ec0 0x7f7374115300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:38.100 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.098+0000 7f737a400640 1 --2- 172.21.15.83:0/1035952612 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7374108540 0x7f7374108980 unknown :-1 s=CLOSED pgs=1801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:38.100 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.098+0000 7f737a400640 1 --2- 172.21.15.83:0/1035952612 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f737410e4d0 0x7f737410e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:38.100 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.098+0000 7f737a400640 1 -- 172.21.15.83:0/1035952612 >> 172.21.15.83:0/1035952612 conn(0x7f7374077de0 msgr2=0x7f7374106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:02:38.100 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.098+0000 7f737a400640 1 -- 172.21.15.83:0/1035952612 shutdown_connections 2024-04-24T09:02:38.100 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.098+0000 7f737a400640 1 -- 172.21.15.83:0/1035952612 wait complete. 2024-04-24T09:02:38.100 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.099+0000 7f737a400640 1 Processor -- start 2024-04-24T09:02:38.100 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.099+0000 7f737a400640 1 -- start start 2024-04-24T09:02:38.100 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.099+0000 7f737a400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7374108540 0x7f73741a48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:38.100 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.099+0000 7f737a400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7374108ec0 0x7f73741a4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:38.100 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.099+0000 7f737a400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f737410e4d0 0x7f737419ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:38.100 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.099+0000 7f737a400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f73741186d0 con 0x7f7374108ec0 2024-04-24T09:02:38.101 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.099+0000 7f737a400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7374118550 con 0x7f7374108540 2024-04-24T09:02:38.101 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.099+0000 7f737a400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7374118850 con 0x7f737410e4d0 2024-04-24T09:02:38.101 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.100+0000 7f7373e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7374108ec0 0x7f73741a4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:38.102 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.100+0000 7f7373e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7374108ec0 0x7f73741a4e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46148/0 (socket says 172.21.15.83:46148) 2024-04-24T09:02:38.102 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.100+0000 7f7373e00640 1 -- 172.21.15.83:0/1889375425 learned_addr learned my addr 172.21.15.83:0/1889375425 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:02:38.102 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.100+0000 7f7379a00640 1 --2- 172.21.15.83:0/1889375425 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f737410e4d0 0x7f737419ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:38.103 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.100+0000 7f7379000640 1 --2- 172.21.15.83:0/1889375425 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7374108540 0x7f73741a48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:38.103 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.101+0000 7f7373e00640 1 -- 172.21.15.83:0/1889375425 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f737410e4d0 msgr2=0x7f737419ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:38.103 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.101+0000 7f7373e00640 1 --2- 172.21.15.83:0/1889375425 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f737410e4d0 0x7f737419ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:38.103 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.101+0000 7f7373e00640 1 -- 172.21.15.83:0/1889375425 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7374108540 msgr2=0x7f73741a48c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:38.103 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.101+0000 7f7373e00640 1 --2- 172.21.15.83:0/1889375425 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7374108540 0x7f73741a48c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:38.103 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.101+0000 7f7373e00640 1 -- 172.21.15.83:0/1889375425 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f737419f1f0 con 0x7f7374108ec0 2024-04-24T09:02:38.103 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.101+0000 7f7373e00640 1 --2- 172.21.15.83:0/1889375425 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7374108ec0 0x7f73741a4e00 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7f7360005ab0 tx=0x7f7360003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:02:38.103 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.102+0000 7f7371600640 1 -- 172.21.15.83:0/1889375425 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f736003f070 con 0x7f7374108ec0 2024-04-24T09:02:38.103 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.102+0000 7f7371600640 1 -- 172.21.15.83:0/1889375425 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7360031d60 con 0x7f7374108ec0 2024-04-24T09:02:38.103 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.102+0000 7f737a400640 1 -- 172.21.15.83:0/1889375425 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f737419f480 con 0x7f7374108ec0 2024-04-24T09:02:38.103 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.102+0000 7f7371600640 1 -- 172.21.15.83:0/1889375425 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7360035030 con 0x7f7374108ec0 2024-04-24T09:02:38.103 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.102+0000 7f737a400640 1 -- 172.21.15.83:0/1889375425 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73741ab780 con 0x7f7374108ec0 2024-04-24T09:02:38.105 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.105+0000 7f7371600640 1 -- 172.21.15.83:0/1889375425 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7360042070 con 0x7f7374108ec0 2024-04-24T09:02:38.105 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.105+0000 7f737e7d5740 1 Processor -- start 2024-04-24T09:02:38.105 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.105+0000 7f737e7d5740 1 -- start start 2024-04-24T09:02:38.109 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.106+0000 7f737e7d5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ae2fa603c0 0x55ae2fa607a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:38.109 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.106+0000 7f737e7d5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae2fa60ce0 0x55ae2fa6b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:38.109 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.106+0000 7f737e7d5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae2fa6b5a0 0x55ae2fa6d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:38.109 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.106+0000 7f737e7d5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ae2f9c6900 con 0x55ae2fa603c0 2024-04-24T09:02:38.109 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.106+0000 7f737e7d5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ae2f96cf20 con 0x55ae2fa60ce0 2024-04-24T09:02:38.109 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.106+0000 7f737e7d5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55ae2f9c28c0 con 0x55ae2fa6b5a0 2024-04-24T09:02:38.110 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.106+0000 7f7379a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ae2fa603c0 0x55ae2fa607a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:38.110 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.106+0000 7f7379a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ae2fa603c0 0x55ae2fa607a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46152/0 (socket says 172.21.15.83:46152) 2024-04-24T09:02:38.110 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.106+0000 7f7379a00640 1 -- 172.21.15.83:0/3046221503 learned_addr learned my addr 172.21.15.83:0/3046221503 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:02:38.110 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.106+0000 7f7373e00640 1 --2- 172.21.15.83:0/3046221503 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae2fa6b5a0 0x55ae2fa6d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:38.110 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.106+0000 7f7379000640 1 --2- 172.21.15.83:0/3046221503 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae2fa60ce0 0x55ae2fa6b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:38.110 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.106+0000 7f7371600640 1 --2- 172.21.15.83:0/1889375425 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f735c0a3d90 0x7f735c0a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:38.110 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.107+0000 7f7379a00640 1 -- 172.21.15.83:0/3046221503 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae2fa6b5a0 msgr2=0x55ae2fa6d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:38.110 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.107+0000 7f7379a00640 1 --2- 172.21.15.83:0/3046221503 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae2fa6b5a0 0x55ae2fa6d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:38.111 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.107+0000 7f7379a00640 1 -- 172.21.15.83:0/3046221503 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae2fa60ce0 msgr2=0x55ae2fa6b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:38.111 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.107+0000 7f7379a00640 1 --2- 172.21.15.83:0/3046221503 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae2fa60ce0 0x55ae2fa6b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:38.111 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.107+0000 7f7379a00640 1 -- 172.21.15.83:0/3046221503 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae2f991e10 con 0x55ae2fa603c0 2024-04-24T09:02:38.111 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.107+0000 7f7379000640 1 --2- 172.21.15.83:0/3046221503 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae2fa60ce0 0x55ae2fa6b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:02:38.111 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.107+0000 7f7371600640 1 -- 172.21.15.83:0/1889375425 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9583+0+0 (secure 0 0 0) 0x7f73600f0fd0 con 0x7f7374108ec0 2024-04-24T09:02:38.111 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.107+0000 7f7379a00640 1 --2- 172.21.15.83:0/1889375425 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f735c0a3d90 0x7f735c0a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:38.111 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.107+0000 7f7379a00640 1 --2- 172.21.15.83:0/3046221503 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ae2fa603c0 0x55ae2fa607a0 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7f736c010370 tx=0x7f736c010830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=814ed21ab64e9b57 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:02:38.111 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.107+0000 7f7366000640 1 -- 172.21.15.83:0/3046221503 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f736c016070 con 0x55ae2fa603c0 2024-04-24T09:02:38.111 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.107+0000 7f7366000640 1 -- 172.21.15.83:0/3046221503 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f736c01c4c0 con 0x55ae2fa603c0 2024-04-24T09:02:38.111 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.108+0000 7f7366000640 1 -- 172.21.15.83:0/3046221503 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f736c01b6c0 con 0x55ae2fa603c0 2024-04-24T09:02:38.111 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.108+0000 7f737e7d5740 1 -- 172.21.15.83:0/3046221503 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ae2fa603c0 msgr2=0x55ae2fa607a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:38.111 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.108+0000 7f7379a00640 1 --2- 172.21.15.83:0/1889375425 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f735c0a3d90 0x7f735c0a6250 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f736c01c000 tx=0x7f736c01be30 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:02:38.112 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.108+0000 7f737e7d5740 1 --2- 172.21.15.83:0/3046221503 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ae2fa603c0 0x55ae2fa607a0 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7f736c010370 tx=0x7f736c010830 comp rx=0 tx=0).stop 2024-04-24T09:02:38.112 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.108+0000 7f737e7d5740 1 -- 172.21.15.83:0/3046221503 shutdown_connections 2024-04-24T09:02:38.112 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.108+0000 7f737e7d5740 1 --2- 172.21.15.83:0/3046221503 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae2fa6b5a0 0x55ae2fa6d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:38.112 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.108+0000 7f737e7d5740 1 --2- 172.21.15.83:0/3046221503 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae2fa60ce0 0x55ae2fa6b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:38.112 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.108+0000 7f737e7d5740 1 --2- 172.21.15.83:0/3046221503 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ae2fa603c0 0x55ae2fa607a0 unknown :-1 s=CLOSED pgs=1803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:38.112 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.108+0000 7f737e7d5740 1 -- 172.21.15.83:0/3046221503 >> 172.21.15.83:0/3046221503 conn(0x55ae2fa5c8d0 msgr2=0x55ae2fa5ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:02:38.112 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.109+0000 7f737e7d5740 1 -- 172.21.15.83:0/3046221503 shutdown_connections 2024-04-24T09:02:38.112 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.109+0000 7f737e7d5740 1 -- 172.21.15.83:0/3046221503 wait complete. 2024-04-24T09:02:38.112 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.109+0000 7f737e7d5740 1 -- start start 2024-04-24T09:02:38.117 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.109+0000 7f737e7d5740 1 Processor -- start 2024-04-24T09:02:38.117 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.110+0000 7f737e7d5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ae2fa603c0 0x55ae2fa68520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:38.117 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.110+0000 7f737e7d5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae2fa60ce0 0x55ae2fa68a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:38.117 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.110+0000 7f737e7d5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae2fa6b5a0 0x55ae2fa68fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:38.117 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.110+0000 7f737e7d5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ae2fa6e0e0 con 0x55ae2fa603c0 2024-04-24T09:02:38.117 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.110+0000 7f737e7d5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ae2f9802c0 con 0x55ae2fa60ce0 2024-04-24T09:02:38.117 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.110+0000 7f737e7d5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55ae2fa6e260 con 0x55ae2fa6b5a0 2024-04-24T09:02:38.117 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.110+0000 7f737e7d5740 10 client.0 _ll_register_callbacks cb 0x55ae2f6d8a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:02:38.117 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.110+0000 7f7379000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ae2fa603c0 0x55ae2fa68520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:38.117 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.110+0000 7f7379000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ae2fa603c0 0x55ae2fa68520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46158/0 (socket says 172.21.15.83:46158) 2024-04-24T09:02:38.118 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.110+0000 7f7379000640 1 -- 172.21.15.83:0/2018838286 learned_addr learned my addr 172.21.15.83:0/2018838286 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:02:38.118 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.110+0000 7f7379000640 1 -- 172.21.15.83:0/2018838286 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae2fa6b5a0 msgr2=0x55ae2fa68fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:38.118 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.110+0000 7f7379000640 1 --2- 172.21.15.83:0/2018838286 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae2fa6b5a0 0x55ae2fa68fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:38.118 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.110+0000 7f7373e00640 1 --2- 172.21.15.83:0/2018838286 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae2fa60ce0 0x55ae2fa68a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:38.118 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.110+0000 7f7379000640 1 -- 172.21.15.83:0/2018838286 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae2fa60ce0 msgr2=0x55ae2fa68a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:38.118 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.110+0000 7f7379000640 1 --2- 172.21.15.83:0/2018838286 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae2fa60ce0 0x55ae2fa68a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:38.118 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.110+0000 7f7379000640 1 -- 172.21.15.83:0/2018838286 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae2f990fd0 con 0x55ae2fa603c0 2024-04-24T09:02:38.118 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.111+0000 7f7379000640 1 --2- 172.21.15.83:0/2018838286 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ae2fa603c0 0x55ae2fa68520 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f7368011690 tx=0x7f7368011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46c600b047979211 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:02:38.118 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.111+0000 7f735aa00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:02:38.118 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.111+0000 7f735aa00640 1 -- 172.21.15.83:0/2018838286 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f736800fb40 con 0x55ae2fa603c0 2024-04-24T09:02:38.118 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.111+0000 7f735aa00640 1 -- 172.21.15.83:0/2018838286 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9583+0+0 (secure 0 0 0) 0x7f736800fce0 con 0x55ae2fa603c0 2024-04-24T09:02:38.118 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.111+0000 7f737e7d5740 1 -- 172.21.15.83:0/2018838286 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f736c01c4c0 con 0x55ae2fa603c0 2024-04-24T09:02:38.119 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.111+0000 7f735aa00640 1 -- 172.21.15.83:0/2018838286 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f7368025050 con 0x55ae2fa603c0 2024-04-24T09:02:38.119 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.111+0000 7f737e7d5740 10 client.19572 fetch_fsmap learned FSMap version 41 2024-04-24T09:02:38.119 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.111+0000 7f737e7d5740 1 -- 172.21.15.83:0/2018838286 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55ae2f6d9210 con 0x55ae2fa603c0 2024-04-24T09:02:38.119 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.112+0000 7f735aa00640 1 -- 172.21.15.83:0/2018838286 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f736c01c4c0 con 0x55ae2fa603c0 2024-04-24T09:02:38.119 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.112+0000 7f735aa00640 1 -- 172.21.15.83:0/2018838286 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7f736801a4d0 con 0x55ae2fa603c0 2024-04-24T09:02:38.119 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.112+0000 7f737e7d5740 10 client.19572 fetch_fsmap finished waiting for FSMap version 41 2024-04-24T09:02:38.119 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.112+0000 7f737e7d5740 10 client.19572 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:02:38.119 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.112+0000 7f737e7d5740 20 client.19572 populate_metadata read hostname 'smithi083' 2024-04-24T09:02:38.119 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.112+0000 7f737e7d5740 1 --2- 172.21.15.83:0/2018838286 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ae2fa7b740 0x55ae2fa7db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:38.119 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.112+0000 7f737e7d5740 4 client.19572 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:02:38.119 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.112+0000 7f737e7d5740 1 -- 172.21.15.83:0/2018838286 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ae2f9d4320 con 0x55ae2fa7b740 2024-04-24T09:02:38.119 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.113+0000 7f7373e00640 1 --2- 172.21.15.83:0/2018838286 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ae2fa7b740 0x55ae2fa7db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:38.119 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.113+0000 7f7373e00640 1 --2- 172.21.15.83:0/2018838286 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ae2fa7b740 0x55ae2fa7db70 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8202a0646cd7d9ce server_cookie=2d5e6a51edb3638a in_seq=0 out_seq=0 2024-04-24T09:02:38.119 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.113+0000 7f735aa00640 10 client.19572 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:02:38.120 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.115+0000 7f735aa00640 1 -- 172.21.15.83:0/2018838286 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ae2f9d4320 con 0x55ae2fa7b740 2024-04-24T09:02:38.120 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.115+0000 7f735aa00640 10 client.19572 handle_command_reply: tid=0 2024-04-24T09:02:38.120 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.115+0000 7f737e7d5740 1 client.19572 shutdown 2024-04-24T09:02:38.120 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.115+0000 7f737e7d5740 1 -- 172.21.15.83:0/2018838286 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ae2fa603c0 msgr2=0x55ae2fa68520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:38.120 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.115+0000 7f737e7d5740 1 --2- 172.21.15.83:0/2018838286 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ae2fa603c0 0x55ae2fa68520 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f7368011690 tx=0x7f7368011b50 comp rx=0 tx=0).stop 2024-04-24T09:02:38.120 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.115+0000 7f737e7d5740 1 -- 172.21.15.83:0/2018838286 shutdown_connections 2024-04-24T09:02:38.120 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.115+0000 7f737e7d5740 1 --2- 172.21.15.83:0/2018838286 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ae2fa7b740 0x55ae2fa7db70 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:38.120 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.115+0000 7f737e7d5740 1 --2- 172.21.15.83:0/2018838286 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae2fa6b5a0 0x55ae2fa68fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:38.120 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.115+0000 7f737e7d5740 1 --2- 172.21.15.83:0/2018838286 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae2fa60ce0 0x55ae2fa68a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:38.120 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.115+0000 7f737e7d5740 1 --2- 172.21.15.83:0/2018838286 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ae2fa603c0 0x55ae2fa68520 unknown :-1 s=CLOSED pgs=1804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:38.120 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.115+0000 7f735aa00640 0 client.19572 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:02:38.120 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.115+0000 7f737e7d5740 1 -- 172.21.15.83:0/2018838286 >> 172.21.15.83:0/2018838286 conn(0x55ae2fa5c8d0 msgr2=0x55ae2fa62650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:02:38.120 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.116+0000 7f737e7d5740 1 -- 172.21.15.83:0/2018838286 shutdown_connections 2024-04-24T09:02:38.120 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.116+0000 7f737e7d5740 1 -- 172.21.15.83:0/2018838286 wait complete. 2024-04-24T09:02:38.121 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.116+0000 7f737e7d5740 20 client.19572 trim_cache size 0 max 16384 2024-04-24T09:02:38.132 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.132+0000 7f737e7d5740 1 Processor -- start 2024-04-24T09:02:38.133 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.133+0000 7f737e7d5740 1 -- start start 2024-04-24T09:02:38.133 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.133+0000 7f737e7d5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae2fa603c0 0x55ae2fa77090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:38.133 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.133+0000 7f737e7d5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ae2fa60ce0 0x55ae2fa62d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:38.133 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.133+0000 7f737e7d5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae2fa6b5a0 0x55ae2fa83480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:38.133 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.133+0000 7f737e7d5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f73741b5aa0 con 0x55ae2fa60ce0 2024-04-24T09:02:38.133 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.133+0000 7f737e7d5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f73741b7700 con 0x55ae2fa603c0 2024-04-24T09:02:38.133 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.133+0000 7f737e7d5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f73741b6170 con 0x55ae2fa6b5a0 2024-04-24T09:02:38.135 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.133+0000 7f7373e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ae2fa60ce0 0x55ae2fa62d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:38.135 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.133+0000 7f7373e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ae2fa60ce0 0x55ae2fa62d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46168/0 (socket says 172.21.15.83:46168) 2024-04-24T09:02:38.135 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.133+0000 7f7373e00640 1 -- 172.21.15.83:0/3618011996 learned_addr learned my addr 172.21.15.83:0/3618011996 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:02:38.135 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.133+0000 7f7379000640 1 --2- 172.21.15.83:0/3618011996 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae2fa603c0 0x55ae2fa77090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:38.135 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.134+0000 7f7373e00640 1 -- 172.21.15.83:0/3618011996 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae2fa6b5a0 msgr2=0x55ae2fa83480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:38.135 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.134+0000 7f7379a00640 1 --2- 172.21.15.83:0/3618011996 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae2fa6b5a0 0x55ae2fa83480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:38.135 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.134+0000 7f7373e00640 1 --2- 172.21.15.83:0/3618011996 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae2fa6b5a0 0x55ae2fa83480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:38.135 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.134+0000 7f7373e00640 1 -- 172.21.15.83:0/3618011996 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae2fa603c0 msgr2=0x55ae2fa77090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:38.135 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.134+0000 7f7373e00640 1 --2- 172.21.15.83:0/3618011996 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae2fa603c0 0x55ae2fa77090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:38.135 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.134+0000 7f7373e00640 1 -- 172.21.15.83:0/3618011996 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae2fa6dd00 con 0x55ae2fa60ce0 2024-04-24T09:02:38.135 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.134+0000 7f7379000640 1 --2- 172.21.15.83:0/3618011996 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae2fa603c0 0x55ae2fa77090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:02:38.136 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.134+0000 7f7373e00640 1 --2- 172.21.15.83:0/3618011996 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ae2fa60ce0 0x55ae2fa62d80 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f7360000c00 tx=0x7f7360009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf89fd447a8014f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:02:38.136 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.134+0000 7f7365600640 1 -- 172.21.15.83:0/3618011996 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7360049040 con 0x55ae2fa60ce0 2024-04-24T09:02:38.136 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.134+0000 7f7365600640 1 -- 172.21.15.83:0/3618011996 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f73600c0ea0 con 0x55ae2fa60ce0 2024-04-24T09:02:38.136 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.134+0000 7f7365600640 1 -- 172.21.15.83:0/3618011996 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f736003b580 con 0x55ae2fa60ce0 2024-04-24T09:02:38.136 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.135+0000 7f737e7d5740 1 -- 172.21.15.83:0/3618011996 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ae2fa60ce0 msgr2=0x55ae2fa62d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:38.136 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.135+0000 7f737e7d5740 1 --2- 172.21.15.83:0/3618011996 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ae2fa60ce0 0x55ae2fa62d80 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f7360000c00 tx=0x7f7360009260 comp rx=0 tx=0).stop 2024-04-24T09:02:38.136 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.135+0000 7f737e7d5740 1 -- 172.21.15.83:0/3618011996 shutdown_connections 2024-04-24T09:02:38.136 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.135+0000 7f737e7d5740 1 --2- 172.21.15.83:0/3618011996 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae2fa6b5a0 0x55ae2fa83480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:38.136 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.135+0000 7f737e7d5740 1 --2- 172.21.15.83:0/3618011996 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ae2fa60ce0 0x55ae2fa62d80 unknown :-1 s=CLOSED pgs=1805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:38.137 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.135+0000 7f737e7d5740 1 --2- 172.21.15.83:0/3618011996 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae2fa603c0 0x55ae2fa77090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:38.137 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.135+0000 7f737e7d5740 1 -- 172.21.15.83:0/3618011996 >> 172.21.15.83:0/3618011996 conn(0x55ae2fa5c8d0 msgr2=0x55ae2fa5ba90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:02:38.137 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.135+0000 7f737e7d5740 1 -- 172.21.15.83:0/3618011996 shutdown_connections 2024-04-24T09:02:38.137 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.135+0000 7f737e7d5740 1 -- 172.21.15.83:0/3618011996 wait complete. 2024-04-24T09:02:38.140 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.136+0000 7f737e7d5740 1 -- start start 2024-04-24T09:02:38.140 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.136+0000 7f737e7d5740 1 Processor -- start 2024-04-24T09:02:38.140 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.136+0000 7f737e7d5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae2fa603c0 0x55ae2fa73800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:38.140 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.136+0000 7f737e7d5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae2fa60ce0 0x55ae2fa70570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:38.140 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.136+0000 7f737e7d5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ae2fa6b5a0 0x55ae2fa70ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:38.140 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.136+0000 7f737e7d5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f73741b6fd0 con 0x55ae2fa6b5a0 2024-04-24T09:02:38.140 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.136+0000 7f737e7d5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f73741b68a0 con 0x55ae2fa603c0 2024-04-24T09:02:38.141 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.136+0000 7f737e7d5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f73741b3aa0 con 0x55ae2fa60ce0 2024-04-24T09:02:38.141 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.136+0000 7f737e7d5740 10 client.0 _ll_register_callbacks cb 0x7f736800a590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:02:38.141 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.136+0000 7f7379a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ae2fa6b5a0 0x55ae2fa70ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:38.141 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.136+0000 7f7379a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ae2fa6b5a0 0x55ae2fa70ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46180/0 (socket says 172.21.15.83:46180) 2024-04-24T09:02:38.141 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.136+0000 7f7379a00640 1 -- 172.21.15.83:0/3599739452 learned_addr learned my addr 172.21.15.83:0/3599739452 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:02:38.141 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.137+0000 7f7373e00640 1 --2- 172.21.15.83:0/3599739452 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae2fa60ce0 0x55ae2fa70570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:38.141 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.137+0000 7f7379000640 1 --2- 172.21.15.83:0/3599739452 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae2fa603c0 0x55ae2fa73800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:38.141 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.137+0000 7f7379a00640 1 -- 172.21.15.83:0/3599739452 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae2fa60ce0 msgr2=0x55ae2fa70570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:38.141 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.137+0000 7f7379a00640 1 --2- 172.21.15.83:0/3599739452 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae2fa60ce0 0x55ae2fa70570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:38.141 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.137+0000 7f7379a00640 1 -- 172.21.15.83:0/3599739452 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae2fa603c0 msgr2=0x55ae2fa73800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:38.141 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.137+0000 7f7379a00640 1 --2- 172.21.15.83:0/3599739452 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae2fa603c0 0x55ae2fa73800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:38.142 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.137+0000 7f7379a00640 1 -- 172.21.15.83:0/3599739452 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae2f7a5620 con 0x55ae2fa6b5a0 2024-04-24T09:02:38.142 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.137+0000 7f7379000640 1 --2- 172.21.15.83:0/3599739452 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae2fa603c0 0x55ae2fa73800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:02:38.142 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.137+0000 7f7379a00640 1 --2- 172.21.15.83:0/3599739452 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ae2fa6b5a0 0x55ae2fa70ab0 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f736c006010 tx=0x7f736c00b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f235de059432c8ac server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:02:38.142 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.137+0000 7f735aa00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:02:38.142 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.138+0000 7f735aa00640 1 -- 172.21.15.83:0/3599739452 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f736c009920 con 0x55ae2fa6b5a0 2024-04-24T09:02:38.142 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.138+0000 7f735aa00640 1 -- 172.21.15.83:0/3599739452 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9583+0+0 (secure 0 0 0) 0x7f736c009bd0 con 0x55ae2fa6b5a0 2024-04-24T09:02:38.142 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.138+0000 7f737e7d5740 1 -- 172.21.15.83:0/3599739452 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73600c0ea0 con 0x55ae2fa6b5a0 2024-04-24T09:02:38.142 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.138+0000 7f735aa00640 1 -- 172.21.15.83:0/3599739452 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f736c03e050 con 0x55ae2fa6b5a0 2024-04-24T09:02:38.142 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.138+0000 7f737e7d5740 10 client.19578 fetch_fsmap learned FSMap version 41 2024-04-24T09:02:38.142 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.138+0000 7f737e7d5740 1 -- 172.21.15.83:0/3599739452 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55ae2fae2930 con 0x55ae2fa6b5a0 2024-04-24T09:02:38.142 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.138+0000 7f735aa00640 1 -- 172.21.15.83:0/3599739452 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f73600c0ea0 con 0x55ae2fa6b5a0 2024-04-24T09:02:38.142 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.138+0000 7f735aa00640 1 -- 172.21.15.83:0/3599739452 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7f736c00f040 con 0x55ae2fa6b5a0 2024-04-24T09:02:38.143 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.139+0000 7f737e7d5740 10 client.19578 fetch_fsmap finished waiting for FSMap version 41 2024-04-24T09:02:38.143 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.139+0000 7f737e7d5740 10 client.19578 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:02:38.143 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.139+0000 7f737e7d5740 20 client.19578 populate_metadata read hostname 'smithi083' 2024-04-24T09:02:38.143 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.139+0000 7f737e7d5740 1 --2- 172.21.15.83:0/3599739452 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ae2fae2f20 0x55ae2fae5360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:38.143 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.139+0000 7f737e7d5740 4 client.19578 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T09:02:38.143 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.139+0000 7f737e7d5740 1 -- 172.21.15.83:0/3599739452 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ae2f9ce190 con 0x55ae2fae2f20 2024-04-24T09:02:38.143 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.139+0000 7f7379000640 1 --2- 172.21.15.83:0/3599739452 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ae2fae2f20 0x55ae2fae5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:38.143 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.140+0000 7f7379000640 1 --2- 172.21.15.83:0/3599739452 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ae2fae2f20 0x55ae2fae5360 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ad24603ba9d5adb server_cookie=e814ce523b9b59e5 in_seq=0 out_seq=0 2024-04-24T09:02:38.143 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.140+0000 7f735aa00640 10 client.19578 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:02:38.199 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.198+0000 7f735aa00640 1 -- 172.21.15.83:0/3599739452 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ae2f9ce190 con 0x55ae2fae2f20 2024-04-24T09:02:38.199 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.198+0000 7f735aa00640 10 client.19578 handle_command_reply: tid=0 2024-04-24T09:02:38.199 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.198+0000 7f737e7d5740 1 client.19578 shutdown 2024-04-24T09:02:38.199 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.199+0000 7f737e7d5740 1 -- 172.21.15.83:0/3599739452 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ae2fa6b5a0 msgr2=0x55ae2fa70ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:38.199 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.199+0000 7f737e7d5740 1 --2- 172.21.15.83:0/3599739452 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ae2fa6b5a0 0x55ae2fa70ab0 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f736c006010 tx=0x7f736c00b000 comp rx=0 tx=0).stop 2024-04-24T09:02:38.200 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.199+0000 7f737e7d5740 1 -- 172.21.15.83:0/3599739452 shutdown_connections 2024-04-24T09:02:38.200 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.199+0000 7f737e7d5740 1 --2- 172.21.15.83:0/3599739452 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ae2fae2f20 0x55ae2fae5360 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:38.200 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.199+0000 7f737e7d5740 1 --2- 172.21.15.83:0/3599739452 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ae2fa6b5a0 0x55ae2fa70ab0 unknown :-1 s=CLOSED pgs=1806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:38.200 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.199+0000 7f737e7d5740 1 --2- 172.21.15.83:0/3599739452 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae2fa60ce0 0x55ae2fa70570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:38.200 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.199+0000 7f737e7d5740 1 --2- 172.21.15.83:0/3599739452 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae2fa603c0 0x55ae2fa73800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:38.200 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.199+0000 7f735aa00640 0 client.19578 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:02:38.200 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.199+0000 7f737e7d5740 1 -- 172.21.15.83:0/3599739452 >> 172.21.15.83:0/3599739452 conn(0x55ae2fa5c8d0 msgr2=0x55ae2fa684c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:02:38.201 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.199+0000 7f737e7d5740 1 -- 172.21.15.83:0/3599739452 shutdown_connections 2024-04-24T09:02:38.201 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.200+0000 7f737e7d5740 1 -- 172.21.15.83:0/3599739452 wait complete. 2024-04-24T09:02:38.201 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.200+0000 7f737e7d5740 20 client.19578 trim_cache size 0 max 16384 2024-04-24T09:02:38.201 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T09:02:38.201 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T09:02:38.201 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T09:02:38.201 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T09:02:38.202 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.201+0000 7f737a400640 1 -- 172.21.15.83:0/1889375425 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f735c0a3d90 msgr2=0x7f735c0a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:38.202 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.201+0000 7f737a400640 1 --2- 172.21.15.83:0/1889375425 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f735c0a3d90 0x7f735c0a6250 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f736c01c000 tx=0x7f736c01be30 comp rx=0 tx=0).stop 2024-04-24T09:02:38.202 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.201+0000 7f737a400640 1 -- 172.21.15.83:0/1889375425 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7374108ec0 msgr2=0x7f73741a4e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:38.202 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.201+0000 7f737a400640 1 --2- 172.21.15.83:0/1889375425 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7374108ec0 0x7f73741a4e00 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7f7360005ab0 tx=0x7f7360003cd0 comp rx=0 tx=0).stop 2024-04-24T09:02:38.202 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.202+0000 7f737a400640 1 -- 172.21.15.83:0/1889375425 shutdown_connections 2024-04-24T09:02:38.202 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.202+0000 7f737a400640 1 --2- 172.21.15.83:0/1889375425 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f735c0a3d90 0x7f735c0a6250 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:38.202 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.202+0000 7f737a400640 1 --2- 172.21.15.83:0/1889375425 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f737410e4d0 0x7f737419ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:38.203 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.202+0000 7f737a400640 1 --2- 172.21.15.83:0/1889375425 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7374108ec0 0x7f73741a4e00 unknown :-1 s=CLOSED pgs=1802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:38.203 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.202+0000 7f737a400640 1 --2- 172.21.15.83:0/1889375425 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7374108540 0x7f73741a48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:38.203 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.202+0000 7f737a400640 1 -- 172.21.15.83:0/1889375425 >> 172.21.15.83:0/1889375425 conn(0x7f7374077de0 msgr2=0x7f7374078440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:02:38.203 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.202+0000 7f737a400640 1 -- 172.21.15.83:0/1889375425 shutdown_connections 2024-04-24T09:02:38.203 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:38.202+0000 7f737a400640 1 -- 172.21.15.83:0/1889375425 wait complete. 2024-04-24T09:02:38.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:02:37 smithi138 ceph-mon[31441]: pgmap v345: 129 pgs: 129 active+clean; 63 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 428 KiB/s wr, 49 op/s 2024-04-24T09:02:38.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:02:37 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:02:38.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:02:37 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:02:38.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:02:37 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:02:38.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:02:37 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:02:38.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:02:37 smithi189 ceph-mon[31791]: pgmap v345: 129 pgs: 129 active+clean; 63 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 428 KiB/s wr, 49 op/s 2024-04-24T09:02:38.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:02:37 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:02:38.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:02:37 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:02:38.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:02:37 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:02:38.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:02:37 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:02:38.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:02:37 smithi083 ceph-mon[27907]: pgmap v345: 129 pgs: 129 active+clean; 63 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 428 KiB/s wr, 49 op/s 2024-04-24T09:02:38.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:02:37 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:02:38.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:02:37 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:02:38.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:02:37 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:02:38.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:02:37 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:02:38.614 INFO:tasks.workunit.client.0.smithi083.stdout: 1 64072 0.62 MB/sec execute 432 sec latency 4022.766 ms 2024-04-24T09:02:39.614 INFO:tasks.workunit.client.0.smithi083.stdout: 1 64128 0.62 MB/sec execute 433 sec latency 50.100 ms 2024-04-24T09:02:40.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:02:39 smithi138 ceph-mon[31441]: pgmap v346: 129 pgs: 129 active+clean; 64 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 466 KiB/s wr, 61 op/s 2024-04-24T09:02:40.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:02:39 smithi189 ceph-mon[31791]: pgmap v346: 129 pgs: 129 active+clean; 64 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 466 KiB/s wr, 61 op/s 2024-04-24T09:02:40.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:02:39 smithi083 ceph-mon[27907]: pgmap v346: 129 pgs: 129 active+clean; 64 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 466 KiB/s wr, 61 op/s 2024-04-24T09:02:40.614 INFO:tasks.workunit.client.0.smithi083.stdout: 1 64187 0.62 MB/sec execute 434 sec latency 49.849 ms 2024-04-24T09:02:41.615 INFO:tasks.workunit.client.0.smithi083.stdout: 1 64253 0.62 MB/sec execute 435 sec latency 51.007 ms 2024-04-24T09:02:42.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:02:41 smithi138 ceph-mon[31441]: pgmap v347: 129 pgs: 129 active+clean; 64 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 170 KiB/s wr, 34 op/s 2024-04-24T09:02:42.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:02:41 smithi189 ceph-mon[31791]: pgmap v347: 129 pgs: 129 active+clean; 64 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 170 KiB/s wr, 34 op/s 2024-04-24T09:02:42.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:02:41 smithi083 ceph-mon[27907]: pgmap v347: 129 pgs: 129 active+clean; 64 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 170 KiB/s wr, 34 op/s 2024-04-24T09:02:42.614 INFO:tasks.workunit.client.0.smithi083.stdout: 1 64308 0.61 MB/sec execute 436 sec latency 49.048 ms 2024-04-24T09:02:43.615 INFO:tasks.workunit.client.0.smithi083.stdout: 1 64369 0.61 MB/sec execute 437 sec latency 54.143 ms 2024-04-24T09:02:44.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:02:43 smithi138 ceph-mon[31441]: pgmap v348: 129 pgs: 129 active+clean; 65 MiB data, 922 MiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 379 KiB/s wr, 59 op/s 2024-04-24T09:02:44.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:02:43 smithi189 ceph-mon[31791]: pgmap v348: 129 pgs: 129 active+clean; 65 MiB data, 922 MiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 379 KiB/s wr, 59 op/s 2024-04-24T09:02:44.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:02:43 smithi083 ceph-mon[27907]: pgmap v348: 129 pgs: 129 active+clean; 65 MiB data, 922 MiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 379 KiB/s wr, 59 op/s 2024-04-24T09:02:44.615 INFO:tasks.workunit.client.0.smithi083.stdout: 1 64438 0.61 MB/sec execute 438 sec latency 58.201 ms 2024-04-24T09:02:45.615 INFO:tasks.workunit.client.0.smithi083.stdout: 1 64527 0.61 MB/sec execute 439 sec latency 62.983 ms 2024-04-24T09:02:46.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:02:45 smithi138 ceph-mon[31441]: pgmap v349: 129 pgs: 129 active+clean; 65 MiB data, 922 MiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 247 KiB/s wr, 36 op/s 2024-04-24T09:02:46.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:02:45 smithi189 ceph-mon[31791]: pgmap v349: 129 pgs: 129 active+clean; 65 MiB data, 922 MiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 247 KiB/s wr, 36 op/s 2024-04-24T09:02:46.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:02:45 smithi083 ceph-mon[27907]: pgmap v349: 129 pgs: 129 active+clean; 65 MiB data, 922 MiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 247 KiB/s wr, 36 op/s 2024-04-24T09:02:46.615 INFO:tasks.workunit.client.0.smithi083.stdout: 1 64650 0.61 MB/sec execute 440 sec latency 54.534 ms 2024-04-24T09:02:47.615 INFO:tasks.workunit.client.0.smithi083.stdout: 1 64760 0.61 MB/sec execute 441 sec latency 54.061 ms 2024-04-24T09:02:48.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:02:47 smithi138 ceph-mon[31441]: pgmap v350: 129 pgs: 129 active+clean; 65 MiB data, 922 MiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 247 KiB/s wr, 36 op/s 2024-04-24T09:02:48.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:02:47 smithi189 ceph-mon[31791]: pgmap v350: 129 pgs: 129 active+clean; 65 MiB data, 922 MiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 247 KiB/s wr, 36 op/s 2024-04-24T09:02:48.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:02:47 smithi083 ceph-mon[27907]: pgmap v350: 129 pgs: 129 active+clean; 65 MiB data, 922 MiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 247 KiB/s wr, 36 op/s 2024-04-24T09:02:48.615 INFO:tasks.workunit.client.0.smithi083.stdout: 1 64862 0.61 MB/sec execute 442 sec latency 48.937 ms 2024-04-24T09:02:49.616 INFO:tasks.workunit.client.0.smithi083.stdout: 1 64980 0.61 MB/sec execute 443 sec latency 49.653 ms 2024-04-24T09:02:50.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:02:49 smithi138 ceph-mon[31441]: pgmap v351: 129 pgs: 129 active+clean; 63 MiB data, 889 MiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 365 KiB/s wr, 63 op/s 2024-04-24T09:02:50.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:02:49 smithi189 ceph-mon[31791]: pgmap v351: 129 pgs: 129 active+clean; 63 MiB data, 889 MiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 365 KiB/s wr, 63 op/s 2024-04-24T09:02:50.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:02:49 smithi083 ceph-mon[27907]: pgmap v351: 129 pgs: 129 active+clean; 63 MiB data, 889 MiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 365 KiB/s wr, 63 op/s 2024-04-24T09:02:50.615 INFO:tasks.workunit.client.0.smithi083.stdout: 1 65083 0.61 MB/sec execute 444 sec latency 50.451 ms 2024-04-24T09:02:51.364 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.364+0000 7f5c3fe00640 1 -- 172.21.15.83:0/855112504 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5c40107140 msgr2=0x7f5c4010ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:51.364 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.364+0000 7f5c3fe00640 1 --2- 172.21.15.83:0/855112504 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5c40107140 0x7f5c4010ed10 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f5c30009900 tx=0x7f5c3002a3f0 comp rx=0 tx=0).stop 2024-04-24T09:02:51.364 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.364+0000 7f5c3fe00640 1 -- 172.21.15.83:0/855112504 shutdown_connections 2024-04-24T09:02:51.364 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.364+0000 7f5c3fe00640 1 --2- 172.21.15.83:0/855112504 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5c40107140 0x7f5c4010ed10 unknown :-1 s=CLOSED pgs=1807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:51.364 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.364+0000 7f5c3fe00640 1 --2- 172.21.15.83:0/855112504 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5c40102a20 0x7f5c40106a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:51.364 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.364+0000 7f5c3fe00640 1 --2- 172.21.15.83:0/855112504 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c40102070 0x7f5c40102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:51.364 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.364+0000 7f5c3fe00640 1 -- 172.21.15.83:0/855112504 >> 172.21.15.83:0/855112504 conn(0x7f5c400fd950 msgr2=0x7f5c400ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:02:51.365 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.364+0000 7f5c3fe00640 1 -- 172.21.15.83:0/855112504 shutdown_connections 2024-04-24T09:02:51.365 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.364+0000 7f5c3fe00640 1 -- 172.21.15.83:0/855112504 wait complete. 2024-04-24T09:02:51.367 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.365+0000 7f5c3fe00640 1 Processor -- start 2024-04-24T09:02:51.367 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.365+0000 7f5c3fe00640 1 -- start start 2024-04-24T09:02:51.368 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.365+0000 7f5c3fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c40102070 0x7f5c40198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:51.368 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.365+0000 7f5c3fe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5c40107140 0x7f5c40198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:51.368 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.365+0000 7f5c3fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5c4019ce20 0x7f5c4019d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:51.368 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.365+0000 7f5c3fe00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5c40112100 con 0x7f5c4019ce20 2024-04-24T09:02:51.368 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.365+0000 7f5c3fe00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5c40111f80 con 0x7f5c40102070 2024-04-24T09:02:51.369 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.365+0000 7f5c3fe00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5c40112280 con 0x7f5c40107140 2024-04-24T09:02:51.369 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.366+0000 7f5c3f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5c4019ce20 0x7f5c4019d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:51.369 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.366+0000 7f5c3f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5c4019ce20 0x7f5c4019d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33682/0 (socket says 172.21.15.83:33682) 2024-04-24T09:02:51.369 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.366+0000 7f5c3f400640 1 -- 172.21.15.83:0/2882602355 learned_addr learned my addr 172.21.15.83:0/2882602355 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:02:51.369 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.366+0000 7f5c3ea00640 1 --2- 172.21.15.83:0/2882602355 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c40102070 0x7f5c40198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:51.370 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.366+0000 7f5c3e000640 1 --2- 172.21.15.83:0/2882602355 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5c40107140 0x7f5c40198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:51.370 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.366+0000 7f5c3f400640 1 -- 172.21.15.83:0/2882602355 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5c40107140 msgr2=0x7f5c40198aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:51.370 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.366+0000 7f5c3f400640 1 --2- 172.21.15.83:0/2882602355 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5c40107140 0x7f5c40198aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:51.370 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.366+0000 7f5c3f400640 1 -- 172.21.15.83:0/2882602355 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c40102070 msgr2=0x7f5c40198560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:51.370 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.366+0000 7f5c3f400640 1 --2- 172.21.15.83:0/2882602355 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c40102070 0x7f5c40198560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:51.371 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.366+0000 7f5c3f400640 1 -- 172.21.15.83:0/2882602355 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c4019d910 con 0x7f5c4019ce20 2024-04-24T09:02:51.371 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.366+0000 7f5c3ea00640 1 --2- 172.21.15.83:0/2882602355 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c40102070 0x7f5c40198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:02:51.371 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.367+0000 7f5c3f400640 1 --2- 172.21.15.83:0/2882602355 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5c4019ce20 0x7f5c4019d2d0 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f5c3002a910 tx=0x7f5c30003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:02:51.371 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.367+0000 7f5c44a00640 1 -- 172.21.15.83:0/2882602355 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c3003f070 con 0x7f5c4019ce20 2024-04-24T09:02:51.371 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.367+0000 7f5c3fe00640 1 -- 172.21.15.83:0/2882602355 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c4019dba0 con 0x7f5c4019ce20 2024-04-24T09:02:51.371 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.367+0000 7f5c44a00640 1 -- 172.21.15.83:0/2882602355 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5c30031d60 con 0x7f5c4019ce20 2024-04-24T09:02:51.372 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.367+0000 7f5c44a00640 1 -- 172.21.15.83:0/2882602355 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c30035030 con 0x7f5c4019ce20 2024-04-24T09:02:51.372 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.367+0000 7f5c3fe00640 1 -- 172.21.15.83:0/2882602355 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c401a53c0 con 0x7f5c4019ce20 2024-04-24T09:02:51.381 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.371+0000 7f5c44a00640 1 -- 172.21.15.83:0/2882602355 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5c30042070 con 0x7f5c4019ce20 2024-04-24T09:02:51.382 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.371+0000 7f5c48c0a740 1 Processor -- start 2024-04-24T09:02:51.382 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.371+0000 7f5c48c0a740 1 -- start start 2024-04-24T09:02:51.382 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.371+0000 7f5c48c0a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b870592e0 0x558b870596c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:51.382 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.371+0000 7f5c48c0a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558b87059c00 0x558b87068470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:51.382 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.372+0000 7f5c48c0a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558b870689b0 0x558b8706ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:51.383 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.372+0000 7f5c48c0a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x558b86fbf900 con 0x558b870592e0 2024-04-24T09:02:51.383 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.372+0000 7f5c48c0a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558b86f65f20 con 0x558b870689b0 2024-04-24T09:02:51.383 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.372+0000 7f5c48c0a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x558b86fbb8c0 con 0x558b87059c00 2024-04-24T09:02:51.383 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.372+0000 7f5c3e000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b870592e0 0x558b870596c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:51.383 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.372+0000 7f5c3e000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b870592e0 0x558b870596c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33690/0 (socket says 172.21.15.83:33690) 2024-04-24T09:02:51.384 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.372+0000 7f5c3e000640 1 -- 172.21.15.83:0/1672942553 learned_addr learned my addr 172.21.15.83:0/1672942553 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:02:51.384 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.372+0000 7f5c3ea00640 1 --2- 172.21.15.83:0/1672942553 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558b87059c00 0x558b87068470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:51.384 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.372+0000 7f5c3e000640 1 -- 172.21.15.83:0/1672942553 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558b87059c00 msgr2=0x558b87068470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:51.384 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.372+0000 7f5c3e000640 1 --2- 172.21.15.83:0/1672942553 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558b87059c00 0x558b87068470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:51.384 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.372+0000 7f5c3e000640 1 -- 172.21.15.83:0/1672942553 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558b870689b0 msgr2=0x558b8706ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:51.384 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.372+0000 7f5c3e000640 1 --2- 172.21.15.83:0/1672942553 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558b870689b0 0x558b8706ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:51.385 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.372+0000 7f5c3e000640 1 -- 172.21.15.83:0/1672942553 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b86f8ae10 con 0x558b870592e0 2024-04-24T09:02:51.385 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.373+0000 7f5c44a00640 1 --2- 172.21.15.83:0/2882602355 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f5c240a3d10 0x7f5c240a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:51.385 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.373+0000 7f5c3e000640 1 --2- 172.21.15.83:0/1672942553 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b870592e0 0x558b870596c0 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f5c2800cb70 tx=0x7f5c28010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=278d4ac83c1ca1ae server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:02:51.385 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.373+0000 7f5c44a00640 1 -- 172.21.15.83:0/2882602355 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9583+0+0 (secure 0 0 0) 0x7f5c300f1380 con 0x7f5c4019ce20 2024-04-24T09:02:51.385 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.373+0000 7f5c3ea00640 1 --2- 172.21.15.83:0/2882602355 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f5c240a3d10 0x7f5c240a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:51.385 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.373+0000 7f5c2fe00640 1 -- 172.21.15.83:0/1672942553 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c28016030 con 0x558b870592e0 2024-04-24T09:02:51.385 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.373+0000 7f5c2fe00640 1 -- 172.21.15.83:0/1672942553 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5c28012930 con 0x558b870592e0 2024-04-24T09:02:51.385 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.373+0000 7f5c2fe00640 1 -- 172.21.15.83:0/1672942553 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c280117e0 con 0x558b870592e0 2024-04-24T09:02:51.385 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.374+0000 7f5c48c0a740 1 -- 172.21.15.83:0/1672942553 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b870592e0 msgr2=0x558b870596c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:51.386 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.374+0000 7f5c3ea00640 1 --2- 172.21.15.83:0/2882602355 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f5c240a3d10 0x7f5c240a61d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x558b8705bee0 tx=0x7f5c38007560 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:02:51.386 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.374+0000 7f5c48c0a740 1 --2- 172.21.15.83:0/1672942553 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b870592e0 0x558b870596c0 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f5c2800cb70 tx=0x7f5c28010740 comp rx=0 tx=0).stop 2024-04-24T09:02:51.386 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.374+0000 7f5c48c0a740 1 -- 172.21.15.83:0/1672942553 shutdown_connections 2024-04-24T09:02:51.386 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.374+0000 7f5c48c0a740 1 --2- 172.21.15.83:0/1672942553 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558b870689b0 0x558b8706ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:51.386 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.374+0000 7f5c48c0a740 1 --2- 172.21.15.83:0/1672942553 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558b87059c00 0x558b87068470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:51.386 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.374+0000 7f5c48c0a740 1 --2- 172.21.15.83:0/1672942553 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b870592e0 0x558b870596c0 unknown :-1 s=CLOSED pgs=1809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:51.386 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.374+0000 7f5c48c0a740 1 -- 172.21.15.83:0/1672942553 >> 172.21.15.83:0/1672942553 conn(0x558b870558d0 msgr2=0x558b87055cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:02:51.386 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.374+0000 7f5c48c0a740 1 -- 172.21.15.83:0/1672942553 shutdown_connections 2024-04-24T09:02:51.387 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.374+0000 7f5c48c0a740 1 -- 172.21.15.83:0/1672942553 wait complete. 2024-04-24T09:02:51.387 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.375+0000 7f5c48c0a740 1 -- start start 2024-04-24T09:02:51.387 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.375+0000 7f5c48c0a740 1 Processor -- start 2024-04-24T09:02:51.387 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.375+0000 7f5c48c0a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b870592e0 0x558b87065970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:51.387 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.375+0000 7f5c48c0a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558b87059c00 0x558b87065eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:51.387 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.376+0000 7f5c48c0a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558b870689b0 0x558b870663f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:51.387 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.376+0000 7f5c48c0a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x558b8706b5e0 con 0x558b870592e0 2024-04-24T09:02:51.388 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.376+0000 7f5c48c0a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558b86f792c0 con 0x558b870689b0 2024-04-24T09:02:51.388 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.376+0000 7f5c48c0a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x558b8706b760 con 0x558b87059c00 2024-04-24T09:02:51.388 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.376+0000 7f5c48c0a740 10 client.0 _ll_register_callbacks cb 0x558b86cd1a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:02:51.388 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.376+0000 7f5c3ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b870592e0 0x558b87065970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:51.388 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.376+0000 7f5c3ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b870592e0 0x558b87065970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33692/0 (socket says 172.21.15.83:33692) 2024-04-24T09:02:51.388 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.376+0000 7f5c3ea00640 1 -- 172.21.15.83:0/4188618041 learned_addr learned my addr 172.21.15.83:0/4188618041 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:02:51.388 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.376+0000 7f5c3e000640 1 --2- 172.21.15.83:0/4188618041 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558b87059c00 0x558b87065eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:51.389 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.376+0000 7f5c3f400640 1 --2- 172.21.15.83:0/4188618041 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558b870689b0 0x558b870663f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:51.389 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.376+0000 7f5c3ea00640 1 -- 172.21.15.83:0/4188618041 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558b87059c00 msgr2=0x558b87065eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:51.389 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.376+0000 7f5c3ea00640 1 --2- 172.21.15.83:0/4188618041 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558b87059c00 0x558b87065eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:51.389 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.376+0000 7f5c3ea00640 1 -- 172.21.15.83:0/4188618041 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558b870689b0 msgr2=0x558b870663f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:51.389 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.376+0000 7f5c3ea00640 1 --2- 172.21.15.83:0/4188618041 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558b870689b0 0x558b870663f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:51.390 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.376+0000 7f5c3ea00640 1 -- 172.21.15.83:0/4188618041 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b86f89fd0 con 0x558b870592e0 2024-04-24T09:02:51.390 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.376+0000 7f5c3f400640 1 --2- 172.21.15.83:0/4188618041 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558b870689b0 0x558b870663f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:02:51.390 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.377+0000 7f5c3ea00640 1 --2- 172.21.15.83:0/4188618041 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b870592e0 0x558b87065970 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7f5c38011ea0 tx=0x7f5c38004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34b7d676cfe39cc6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:02:51.390 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.377+0000 7f5c2cc00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:02:51.390 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.377+0000 7f5c2cc00640 1 -- 172.21.15.83:0/4188618041 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c3801e620 con 0x558b870592e0 2024-04-24T09:02:51.390 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.377+0000 7f5c48c0a740 1 -- 172.21.15.83:0/4188618041 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c28012930 con 0x558b870592e0 2024-04-24T09:02:51.390 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.377+0000 7f5c2cc00640 1 -- 172.21.15.83:0/4188618041 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9583+0+0 (secure 0 0 0) 0x7f5c3801e7c0 con 0x558b870592e0 2024-04-24T09:02:51.391 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.377+0000 7f5c2cc00640 1 -- 172.21.15.83:0/4188618041 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f5c3801e9c0 con 0x558b870592e0 2024-04-24T09:02:51.391 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.377+0000 7f5c48c0a740 10 client.19590 fetch_fsmap learned FSMap version 41 2024-04-24T09:02:51.391 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.377+0000 7f5c48c0a740 1 -- 172.21.15.83:0/4188618041 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x558b86cd2210 con 0x558b870592e0 2024-04-24T09:02:51.391 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.377+0000 7f5c2cc00640 1 -- 172.21.15.83:0/4188618041 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5c28012930 con 0x558b870592e0 2024-04-24T09:02:51.391 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.378+0000 7f5c2cc00640 1 -- 172.21.15.83:0/4188618041 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7f5c38026c80 con 0x558b870592e0 2024-04-24T09:02:51.391 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.378+0000 7f5c48c0a740 10 client.19590 fetch_fsmap finished waiting for FSMap version 41 2024-04-24T09:02:51.392 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.378+0000 7f5c48c0a740 10 client.19590 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:02:51.392 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.378+0000 7f5c48c0a740 20 client.19590 populate_metadata read hostname 'smithi083' 2024-04-24T09:02:51.392 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.378+0000 7f5c48c0a740 1 --2- 172.21.15.83:0/4188618041 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x558b87078c70 0x558b8707b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:51.392 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.378+0000 7f5c48c0a740 4 client.19590 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:02:51.392 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.378+0000 7f5c48c0a740 1 -- 172.21.15.83:0/4188618041 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558b86fcd320 con 0x558b87078c70 2024-04-24T09:02:51.392 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.379+0000 7f5c3e000640 1 --2- 172.21.15.83:0/4188618041 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x558b87078c70 0x558b8707b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:51.392 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.380+0000 7f5c3e000640 1 --2- 172.21.15.83:0/4188618041 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x558b87078c70 0x558b8707b110 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=689529de8f8ca956 server_cookie=342ed4118c0efa62 in_seq=0 out_seq=0 2024-04-24T09:02:51.392 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.380+0000 7f5c2cc00640 10 client.19590 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:02:51.394 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.383+0000 7f5c2cc00640 1 -- 172.21.15.83:0/4188618041 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558b86fcd320 con 0x558b87078c70 2024-04-24T09:02:51.394 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.383+0000 7f5c2cc00640 10 client.19590 handle_command_reply: tid=0 2024-04-24T09:02:51.394 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.383+0000 7f5c48c0a740 1 client.19590 shutdown 2024-04-24T09:02:51.394 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.383+0000 7f5c48c0a740 1 -- 172.21.15.83:0/4188618041 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b870592e0 msgr2=0x558b87065970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:51.394 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.383+0000 7f5c48c0a740 1 --2- 172.21.15.83:0/4188618041 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b870592e0 0x558b87065970 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7f5c38011ea0 tx=0x7f5c38004a40 comp rx=0 tx=0).stop 2024-04-24T09:02:51.394 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.383+0000 7f5c48c0a740 1 -- 172.21.15.83:0/4188618041 shutdown_connections 2024-04-24T09:02:51.394 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.383+0000 7f5c48c0a740 1 --2- 172.21.15.83:0/4188618041 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x558b87078c70 0x558b8707b110 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:51.394 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.383+0000 7f5c48c0a740 1 --2- 172.21.15.83:0/4188618041 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558b870689b0 0x558b870663f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:51.394 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.383+0000 7f5c48c0a740 1 --2- 172.21.15.83:0/4188618041 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558b87059c00 0x558b87065eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:51.394 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.383+0000 7f5c48c0a740 1 --2- 172.21.15.83:0/4188618041 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b870592e0 0x558b87065970 unknown :-1 s=CLOSED pgs=1810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:51.394 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.383+0000 7f5c2cc00640 0 client.19590 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:02:51.395 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.383+0000 7f5c48c0a740 1 -- 172.21.15.83:0/4188618041 >> 172.21.15.83:0/4188618041 conn(0x558b870558d0 msgr2=0x558b8705b740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:02:51.395 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.383+0000 7f5c48c0a740 1 -- 172.21.15.83:0/4188618041 shutdown_connections 2024-04-24T09:02:51.395 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.383+0000 7f5c48c0a740 1 -- 172.21.15.83:0/4188618041 wait complete. 2024-04-24T09:02:51.395 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.383+0000 7f5c48c0a740 20 client.19590 trim_cache size 0 max 16384 2024-04-24T09:02:51.400 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.400+0000 7f5c48c0a740 1 Processor -- start 2024-04-24T09:02:51.400 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.400+0000 7f5c48c0a740 1 -- start start 2024-04-24T09:02:51.400 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.400+0000 7f5c48c0a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b870592e0 0x558b870744a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:51.400 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.400+0000 7f5c48c0a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558b87059c00 0x558b87064970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:51.400 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.400+0000 7f5c48c0a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558b870689b0 0x558b87080a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:51.400 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.400+0000 7f5c48c0a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5c401ad220 con 0x558b870592e0 2024-04-24T09:02:51.401 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.400+0000 7f5c48c0a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5c401aee80 con 0x558b87059c00 2024-04-24T09:02:51.401 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.400+0000 7f5c48c0a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5c401ad8f0 con 0x558b870689b0 2024-04-24T09:02:51.402 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.401+0000 7f5c3ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b870592e0 0x558b870744a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:51.402 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.401+0000 7f5c3ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b870592e0 0x558b870744a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33698/0 (socket says 172.21.15.83:33698) 2024-04-24T09:02:51.402 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.401+0000 7f5c3ea00640 1 -- 172.21.15.83:0/2860552904 learned_addr learned my addr 172.21.15.83:0/2860552904 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:02:51.402 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.401+0000 7f5c3e000640 1 --2- 172.21.15.83:0/2860552904 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558b87059c00 0x558b87064970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:51.402 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.401+0000 7f5c3ea00640 1 -- 172.21.15.83:0/2860552904 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558b870689b0 msgr2=0x558b87080a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:51.402 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.401+0000 7f5c3ea00640 1 --2- 172.21.15.83:0/2860552904 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558b870689b0 0x558b87080a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:51.403 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.401+0000 7f5c3ea00640 1 -- 172.21.15.83:0/2860552904 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558b87059c00 msgr2=0x558b87064970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:51.403 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.401+0000 7f5c3ea00640 1 --2- 172.21.15.83:0/2860552904 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558b87059c00 0x558b87064970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:51.403 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.401+0000 7f5c3ea00640 1 -- 172.21.15.83:0/2860552904 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b86d9e620 con 0x558b870592e0 2024-04-24T09:02:51.403 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.401+0000 7f5c3ea00640 1 --2- 172.21.15.83:0/2860552904 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b870592e0 0x558b870744a0 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f5c38011e70 tx=0x7f5c38011f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58cbd8c97add286d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:02:51.403 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.402+0000 7f5c2f400640 1 -- 172.21.15.83:0/2860552904 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c3801ed60 con 0x558b870592e0 2024-04-24T09:02:51.403 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.402+0000 7f5c2f400640 1 -- 172.21.15.83:0/2860552904 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5c3801e420 con 0x558b870592e0 2024-04-24T09:02:51.403 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.402+0000 7f5c2f400640 1 -- 172.21.15.83:0/2860552904 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c3801cbf0 con 0x558b870592e0 2024-04-24T09:02:51.406 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.403+0000 7f5c48c0a740 1 -- 172.21.15.83:0/2860552904 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b870592e0 msgr2=0x558b870744a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:51.406 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.403+0000 7f5c48c0a740 1 --2- 172.21.15.83:0/2860552904 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b870592e0 0x558b870744a0 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f5c38011e70 tx=0x7f5c38011f90 comp rx=0 tx=0).stop 2024-04-24T09:02:51.406 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.403+0000 7f5c48c0a740 1 -- 172.21.15.83:0/2860552904 shutdown_connections 2024-04-24T09:02:51.407 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.403+0000 7f5c48c0a740 1 --2- 172.21.15.83:0/2860552904 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558b870689b0 0x558b87080a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:51.407 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.403+0000 7f5c48c0a740 1 --2- 172.21.15.83:0/2860552904 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558b87059c00 0x558b87064970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:51.407 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.403+0000 7f5c48c0a740 1 --2- 172.21.15.83:0/2860552904 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b870592e0 0x558b870744a0 unknown :-1 s=CLOSED pgs=1811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:51.407 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.403+0000 7f5c48c0a740 1 -- 172.21.15.83:0/2860552904 >> 172.21.15.83:0/2860552904 conn(0x558b870558d0 msgr2=0x558b87064260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:02:51.407 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.403+0000 7f5c48c0a740 1 -- 172.21.15.83:0/2860552904 shutdown_connections 2024-04-24T09:02:51.407 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.403+0000 7f5c48c0a740 1 -- 172.21.15.83:0/2860552904 wait complete. 2024-04-24T09:02:51.407 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.403+0000 7f5c48c0a740 1 -- start start 2024-04-24T09:02:51.407 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.404+0000 7f5c48c0a740 1 Processor -- start 2024-04-24T09:02:51.407 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.404+0000 7f5c48c0a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558b87059c00 0x558b870709b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:51.407 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.404+0000 7f5c48c0a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b870689b0 0x558b87070ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:51.407 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.404+0000 7f5c48c0a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558b87061de0 0x558b87062260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:51.407 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.404+0000 7f5c48c0a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5c401ae750 con 0x558b870689b0 2024-04-24T09:02:51.407 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.404+0000 7f5c48c0a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5c401ae020 con 0x558b87061de0 2024-04-24T09:02:51.407 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.404+0000 7f5c48c0a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5c401ab220 con 0x558b87059c00 2024-04-24T09:02:51.407 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.404+0000 7f5c48c0a740 10 client.0 _ll_register_callbacks cb 0x7f5c38007ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:02:51.408 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.404+0000 7f5c3e000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b870689b0 0x558b87070ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:51.408 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.404+0000 7f5c3e000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b870689b0 0x558b87070ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33708/0 (socket says 172.21.15.83:33708) 2024-04-24T09:02:51.408 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.404+0000 7f5c3e000640 1 -- 172.21.15.83:0/1312461077 learned_addr learned my addr 172.21.15.83:0/1312461077 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:02:51.408 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.405+0000 7f5c3ea00640 1 --2- 172.21.15.83:0/1312461077 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558b87059c00 0x558b870709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:51.408 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.405+0000 7f5c3f400640 1 --2- 172.21.15.83:0/1312461077 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558b87061de0 0x558b87062260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:51.408 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.405+0000 7f5c3e000640 1 -- 172.21.15.83:0/1312461077 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558b87059c00 msgr2=0x558b870709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:51.408 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.405+0000 7f5c3e000640 1 --2- 172.21.15.83:0/1312461077 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558b87059c00 0x558b870709b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:51.408 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.405+0000 7f5c3e000640 1 -- 172.21.15.83:0/1312461077 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558b87061de0 msgr2=0x558b87062260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:51.408 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.405+0000 7f5c3e000640 1 --2- 172.21.15.83:0/1312461077 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558b87061de0 0x558b87062260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:51.408 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.405+0000 7f5c3e000640 1 -- 172.21.15.83:0/1312461077 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b87062ab0 con 0x558b870689b0 2024-04-24T09:02:51.408 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.405+0000 7f5c3e000640 1 --2- 172.21.15.83:0/1312461077 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b870689b0 0x558b87070ef0 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7f5c2800cb70 tx=0x7f5c28011450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55bcd165e7bd1807 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:02:51.408 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.405+0000 7f5c2cc00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:02:51.408 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.405+0000 7f5c2cc00640 1 -- 172.21.15.83:0/1312461077 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c28016070 con 0x558b870689b0 2024-04-24T09:02:51.408 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.405+0000 7f5c48c0a740 1 -- 172.21.15.83:0/1312461077 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c3801e420 con 0x558b870689b0 2024-04-24T09:02:51.409 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.405+0000 7f5c2cc00640 1 -- 172.21.15.83:0/1312461077 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9583+0+0 (secure 0 0 0) 0x7f5c28004220 con 0x558b870689b0 2024-04-24T09:02:51.409 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.406+0000 7f5c2cc00640 1 -- 172.21.15.83:0/1312461077 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f5c28004420 con 0x558b870689b0 2024-04-24T09:02:51.409 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.406+0000 7f5c48c0a740 10 client.19596 fetch_fsmap learned FSMap version 41 2024-04-24T09:02:51.409 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.406+0000 7f5c48c0a740 1 -- 172.21.15.83:0/1312461077 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x558b870df220 con 0x558b870689b0 2024-04-24T09:02:51.409 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.406+0000 7f5c2cc00640 1 -- 172.21.15.83:0/1312461077 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5c3801e420 con 0x558b870689b0 2024-04-24T09:02:51.409 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.406+0000 7f5c2cc00640 1 -- 172.21.15.83:0/1312461077 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7f5c28004b20 con 0x558b870689b0 2024-04-24T09:02:51.409 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.406+0000 7f5c48c0a740 10 client.19596 fetch_fsmap finished waiting for FSMap version 41 2024-04-24T09:02:51.409 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.406+0000 7f5c48c0a740 10 client.19596 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:02:51.409 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.406+0000 7f5c48c0a740 20 client.19596 populate_metadata read hostname 'smithi083' 2024-04-24T09:02:51.409 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.406+0000 7f5c48c0a740 1 --2- 172.21.15.83:0/1312461077 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x558b870df720 0x558b870e1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:51.409 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.406+0000 7f5c48c0a740 4 client.19596 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T09:02:51.409 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.406+0000 7f5c48c0a740 1 -- 172.21.15.83:0/1312461077 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558b86fc7190 con 0x558b870df720 2024-04-24T09:02:51.410 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.407+0000 7f5c3ea00640 1 --2- 172.21.15.83:0/1312461077 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x558b870df720 0x558b870e1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:51.410 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.408+0000 7f5c3ea00640 1 --2- 172.21.15.83:0/1312461077 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x558b870df720 0x558b870e1b40 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7044c70ba5bf2de server_cookie=e2d5b2206bef980f in_seq=0 out_seq=0 2024-04-24T09:02:51.410 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.408+0000 7f5c2cc00640 10 client.19596 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:02:51.476 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.475+0000 7f5c2cc00640 1 -- 172.21.15.83:0/1312461077 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558b86fc7190 con 0x558b870df720 2024-04-24T09:02:51.477 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.475+0000 7f5c2cc00640 10 client.19596 handle_command_reply: tid=0 2024-04-24T09:02:51.477 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.475+0000 7f5c48c0a740 1 client.19596 shutdown 2024-04-24T09:02:51.477 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.475+0000 7f5c48c0a740 1 -- 172.21.15.83:0/1312461077 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b870689b0 msgr2=0x558b87070ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:51.477 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.475+0000 7f5c48c0a740 1 --2- 172.21.15.83:0/1312461077 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b870689b0 0x558b87070ef0 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7f5c2800cb70 tx=0x7f5c28011450 comp rx=0 tx=0).stop 2024-04-24T09:02:51.477 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.475+0000 7f5c48c0a740 1 -- 172.21.15.83:0/1312461077 shutdown_connections 2024-04-24T09:02:51.478 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.475+0000 7f5c48c0a740 1 --2- 172.21.15.83:0/1312461077 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x558b870df720 0x558b870e1b40 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:51.478 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.475+0000 7f5c48c0a740 1 --2- 172.21.15.83:0/1312461077 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558b87061de0 0x558b87062260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:51.478 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.475+0000 7f5c48c0a740 1 --2- 172.21.15.83:0/1312461077 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b870689b0 0x558b87070ef0 unknown :-1 s=CLOSED pgs=1812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:51.478 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.475+0000 7f5c48c0a740 1 --2- 172.21.15.83:0/1312461077 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558b87059c00 0x558b870709b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:51.478 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.475+0000 7f5c2cc00640 0 client.19596 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:02:51.478 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.475+0000 7f5c48c0a740 1 -- 172.21.15.83:0/1312461077 >> 172.21.15.83:0/1312461077 conn(0x558b870558d0 msgr2=0x558b8705ada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:02:51.478 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.476+0000 7f5c48c0a740 1 -- 172.21.15.83:0/1312461077 shutdown_connections 2024-04-24T09:02:51.479 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.476+0000 7f5c48c0a740 1 -- 172.21.15.83:0/1312461077 wait complete. 2024-04-24T09:02:51.479 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.476+0000 7f5c48c0a740 20 client.19596 trim_cache size 0 max 16384 2024-04-24T09:02:51.479 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T09:02:51.479 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T09:02:51.479 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T09:02:51.479 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T09:02:51.480 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.477+0000 7f5c3fe00640 1 -- 172.21.15.83:0/2882602355 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f5c240a3d10 msgr2=0x7f5c240a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:51.481 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.477+0000 7f5c3fe00640 1 --2- 172.21.15.83:0/2882602355 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f5c240a3d10 0x7f5c240a61d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x558b8705bee0 tx=0x7f5c38007560 comp rx=0 tx=0).stop 2024-04-24T09:02:51.481 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.477+0000 7f5c3fe00640 1 -- 172.21.15.83:0/2882602355 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5c4019ce20 msgr2=0x7f5c4019d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:51.481 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.477+0000 7f5c3fe00640 1 --2- 172.21.15.83:0/2882602355 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5c4019ce20 0x7f5c4019d2d0 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f5c3002a910 tx=0x7f5c30003cd0 comp rx=0 tx=0).stop 2024-04-24T09:02:51.481 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.477+0000 7f5c3fe00640 1 -- 172.21.15.83:0/2882602355 shutdown_connections 2024-04-24T09:02:51.481 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.477+0000 7f5c3fe00640 1 --2- 172.21.15.83:0/2882602355 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f5c240a3d10 0x7f5c240a61d0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:51.481 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.477+0000 7f5c3fe00640 1 --2- 172.21.15.83:0/2882602355 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5c4019ce20 0x7f5c4019d2d0 unknown :-1 s=CLOSED pgs=1808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:51.481 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.477+0000 7f5c3fe00640 1 --2- 172.21.15.83:0/2882602355 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5c40107140 0x7f5c40198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:51.482 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.477+0000 7f5c3fe00640 1 --2- 172.21.15.83:0/2882602355 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c40102070 0x7f5c40198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:51.482 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.477+0000 7f5c3fe00640 1 -- 172.21.15.83:0/2882602355 >> 172.21.15.83:0/2882602355 conn(0x7f5c400fd950 msgr2=0x7f5c400fe490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:02:51.482 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.477+0000 7f5c3fe00640 1 -- 172.21.15.83:0/2882602355 shutdown_connections 2024-04-24T09:02:51.482 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:02:51.477+0000 7f5c3fe00640 1 -- 172.21.15.83:0/2882602355 wait complete. 2024-04-24T09:02:51.616 INFO:tasks.workunit.client.0.smithi083.stdout: 1 65261 0.61 MB/sec execute 445 sec latency 50.906 ms 2024-04-24T09:02:52.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:02:51 smithi138 ceph-mon[31441]: pgmap v352: 129 pgs: 129 active+clean; 63 MiB data, 889 MiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 327 KiB/s wr, 51 op/s 2024-04-24T09:02:52.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:02:51 smithi189 ceph-mon[31791]: pgmap v352: 129 pgs: 129 active+clean; 63 MiB data, 889 MiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 327 KiB/s wr, 51 op/s 2024-04-24T09:02:52.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:02:51 smithi083 ceph-mon[27907]: pgmap v352: 129 pgs: 129 active+clean; 63 MiB data, 889 MiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 327 KiB/s wr, 51 op/s 2024-04-24T09:02:52.616 INFO:tasks.workunit.client.0.smithi083.stdout: 1 65361 0.61 MB/sec execute 446 sec latency 61.888 ms 2024-04-24T09:02:53.616 INFO:tasks.workunit.client.0.smithi083.stdout: 1 65461 0.61 MB/sec execute 447 sec latency 65.258 ms 2024-04-24T09:02:54.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:02:53 smithi138 ceph-mon[31441]: pgmap v353: 129 pgs: 129 active+clean; 60 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 523 KiB/s wr, 79 op/s 2024-04-24T09:02:54.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:02:53 smithi189 ceph-mon[31791]: pgmap v353: 129 pgs: 129 active+clean; 60 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 523 KiB/s wr, 79 op/s 2024-04-24T09:02:54.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:02:53 smithi083 ceph-mon[27907]: pgmap v353: 129 pgs: 129 active+clean; 60 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 523 KiB/s wr, 79 op/s 2024-04-24T09:02:54.616 INFO:tasks.workunit.client.0.smithi083.stdout: 1 65562 0.61 MB/sec execute 448 sec latency 68.837 ms 2024-04-24T09:02:55.616 INFO:tasks.workunit.client.0.smithi083.stdout: 1 65696 0.61 MB/sec execute 449 sec latency 71.021 ms 2024-04-24T09:02:56.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:02:55 smithi138 ceph-mon[31441]: pgmap v354: 129 pgs: 129 active+clean; 60 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 313 KiB/s wr, 54 op/s 2024-04-24T09:02:56.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:02:55 smithi189 ceph-mon[31791]: pgmap v354: 129 pgs: 129 active+clean; 60 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 313 KiB/s wr, 54 op/s 2024-04-24T09:02:56.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:02:55 smithi083 ceph-mon[27907]: pgmap v354: 129 pgs: 129 active+clean; 60 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 313 KiB/s wr, 54 op/s 2024-04-24T09:02:56.616 INFO:tasks.workunit.client.0.smithi083.stdout: 1 65795 0.61 MB/sec execute 450 sec latency 50.994 ms 2024-04-24T09:02:57.616 INFO:tasks.workunit.client.0.smithi083.stdout: 1 65933 0.61 MB/sec execute 451 sec latency 66.822 ms 2024-04-24T09:02:58.043 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T09:02:58.043 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T09:02:58.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:02:57 smithi138 ceph-mon[31441]: pgmap v355: 129 pgs: 129 active+clean; 60 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 314 KiB/s wr, 54 op/s 2024-04-24T09:02:58.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:02:57 smithi189 ceph-mon[31791]: pgmap v355: 129 pgs: 129 active+clean; 60 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 314 KiB/s wr, 54 op/s 2024-04-24T09:02:58.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.255+0000 7faf30e00640 1 -- 172.21.15.83:0/2263996214 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf2c104e60 msgr2=0x7faf2c108ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:58.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.255+0000 7faf30e00640 1 --2- 172.21.15.83:0/2263996214 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf2c104e60 0x7faf2c108ea0 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7faf18009900 tx=0x7faf1802a400 comp rx=0 tx=0).stop 2024-04-24T09:02:58.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.255+0000 7faf30e00640 1 -- 172.21.15.83:0/2263996214 shutdown_connections 2024-04-24T09:02:58.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.255+0000 7faf30e00640 1 --2- 172.21.15.83:0/2263996214 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faf2c109580 0x7faf2c111150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:58.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.255+0000 7faf30e00640 1 --2- 172.21.15.83:0/2263996214 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf2c104e60 0x7faf2c108ea0 unknown :-1 s=CLOSED pgs=1813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:58.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.255+0000 7faf30e00640 1 --2- 172.21.15.83:0/2263996214 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faf2c1044b0 0x7faf2c104890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:58.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.255+0000 7faf30e00640 1 -- 172.21.15.83:0/2263996214 >> 172.21.15.83:0/2263996214 conn(0x7faf2c0ffd90 msgr2=0x7faf2c1021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:02:58.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.255+0000 7faf30e00640 1 -- 172.21.15.83:0/2263996214 shutdown_connections 2024-04-24T09:02:58.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.255+0000 7faf30e00640 1 -- 172.21.15.83:0/2263996214 wait complete. 2024-04-24T09:02:58.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.256+0000 7faf30e00640 1 Processor -- start 2024-04-24T09:02:58.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.256+0000 7faf30e00640 1 -- start start 2024-04-24T09:02:58.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.257+0000 7faf30e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faf2c1044b0 0x7faf2c1a07f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:58.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.257+0000 7faf30e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faf2c109580 0x7faf2c1a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:58.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.257+0000 7faf30e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf2c19a9f0 0x7faf2c19aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:58.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.257+0000 7faf30e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7faf2c1145a0 con 0x7faf2c19a9f0 2024-04-24T09:02:58.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.257+0000 7faf30e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7faf2c114420 con 0x7faf2c109580 2024-04-24T09:02:58.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.257+0000 7faf30e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7faf2c114720 con 0x7faf2c1044b0 2024-04-24T09:02:58.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.257+0000 7faf2a800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf2c19a9f0 0x7faf2c19aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:58.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.257+0000 7faf2a800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf2c19a9f0 0x7faf2c19aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33720/0 (socket says 172.21.15.83:33720) 2024-04-24T09:02:58.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.257+0000 7faf2a800640 1 -- 172.21.15.83:0/3089301565 learned_addr learned my addr 172.21.15.83:0/3089301565 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:02:58.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.257+0000 7faf29400640 1 --2- 172.21.15.83:0/3089301565 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faf2c109580 0x7faf2c1a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:58.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.257+0000 7faf29e00640 1 --2- 172.21.15.83:0/3089301565 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faf2c1044b0 0x7faf2c1a07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:58.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.257+0000 7faf2a800640 1 -- 172.21.15.83:0/3089301565 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faf2c1044b0 msgr2=0x7faf2c1a07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:58.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.257+0000 7faf2a800640 1 --2- 172.21.15.83:0/3089301565 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faf2c1044b0 0x7faf2c1a07f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:58.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.258+0000 7faf2a800640 1 -- 172.21.15.83:0/3089301565 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faf2c109580 msgr2=0x7faf2c1a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:58.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.258+0000 7faf2a800640 1 --2- 172.21.15.83:0/3089301565 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faf2c109580 0x7faf2c1a0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:58.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.258+0000 7faf2a800640 1 -- 172.21.15.83:0/3089301565 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf2c19b510 con 0x7faf2c19a9f0 2024-04-24T09:02:58.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.258+0000 7faf2a800640 1 --2- 172.21.15.83:0/3089301565 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf2c19a9f0 0x7faf2c19aea0 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7faf2400b970 tx=0x7faf2400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:02:58.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.258+0000 7faf22a00640 1 -- 172.21.15.83:0/3089301565 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf24019070 con 0x7faf2c19a9f0 2024-04-24T09:02:58.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.258+0000 7faf30e00640 1 -- 172.21.15.83:0/3089301565 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf2c19b800 con 0x7faf2c19a9f0 2024-04-24T09:02:58.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.258+0000 7faf22a00640 1 -- 172.21.15.83:0/3089301565 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faf24014440 con 0x7faf2c19a9f0 2024-04-24T09:02:58.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.258+0000 7faf22a00640 1 -- 172.21.15.83:0/3089301565 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf2401d5d0 con 0x7faf2c19a9f0 2024-04-24T09:02:58.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.258+0000 7faf30e00640 1 -- 172.21.15.83:0/3089301565 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf2c1a7a00 con 0x7faf2c19a9f0 2024-04-24T09:02:58.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.263+0000 7faf337fd740 1 Processor -- start 2024-04-24T09:02:58.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.263+0000 7faf22a00640 1 -- 172.21.15.83:0/3089301565 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7faf2401c070 con 0x7faf2c19a9f0 2024-04-24T09:02:58.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.264+0000 7faf337fd740 1 -- start start 2024-04-24T09:02:58.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.264+0000 7faf337fd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dca80d9860 0x55dca80d9c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:58.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.264+0000 7faf337fd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dca80da180 0x55dca80e4310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:58.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.264+0000 7faf337fd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dca80e4850 0x55dca80e6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:58.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.264+0000 7faf337fd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55dca803f1f0 con 0x55dca80e4850 2024-04-24T09:02:58.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.264+0000 7faf337fd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55dca7fe5350 con 0x55dca80d9860 2024-04-24T09:02:58.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.264+0000 7faf337fd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55dca803b1b0 con 0x55dca80da180 2024-04-24T09:02:58.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.264+0000 7faf29400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dca80e4850 0x55dca80e6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:58.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.265+0000 7faf29400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dca80e4850 0x55dca80e6c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33732/0 (socket says 172.21.15.83:33732) 2024-04-24T09:02:58.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.265+0000 7faf29400640 1 -- 172.21.15.83:0/919767006 learned_addr learned my addr 172.21.15.83:0/919767006 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:02:58.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.265+0000 7faf29e00640 1 --2- 172.21.15.83:0/919767006 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dca80da180 0x55dca80e4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:58.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.265+0000 7faf29400640 1 --2- 172.21.15.83:0/919767006 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dca80d9860 0x55dca80d9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:58.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.265+0000 7faf29400640 1 -- 172.21.15.83:0/919767006 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dca80da180 msgr2=0x55dca80e4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:58.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.265+0000 7faf29400640 1 --2- 172.21.15.83:0/919767006 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dca80da180 0x55dca80e4310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:58.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.265+0000 7faf29400640 1 -- 172.21.15.83:0/919767006 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dca80d9860 msgr2=0x55dca80d9c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:58.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.265+0000 7faf29400640 1 --2- 172.21.15.83:0/919767006 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dca80d9860 0x55dca80d9c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:58.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.265+0000 7faf29400640 1 -- 172.21.15.83:0/919767006 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dca8039f50 con 0x55dca80e4850 2024-04-24T09:02:58.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.265+0000 7faf22a00640 1 --2- 172.21.15.83:0/3089301565 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7faf100a3d20 0x7faf100a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:58.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.265+0000 7faf29400640 1 --2- 172.21.15.83:0/919767006 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dca80e4850 0x55dca80e6c40 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7faf2c06b750 tx=0x7faf180058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31cf4fe57bd03f3b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:02:58.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.266+0000 7faf22a00640 1 -- 172.21.15.83:0/3089301565 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9583+0+0 (secure 0 0 0) 0x7faf240d0100 con 0x7faf2c19a9f0 2024-04-24T09:02:58.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.266+0000 7faf29e00640 1 --2- 172.21.15.83:0/3089301565 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7faf100a3d20 0x7faf100a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:58.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.266+0000 7faf17400640 1 -- 172.21.15.83:0/919767006 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf18035040 con 0x55dca80e4850 2024-04-24T09:02:58.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.266+0000 7faf17400640 1 -- 172.21.15.83:0/919767006 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faf18005020 con 0x55dca80e4850 2024-04-24T09:02:58.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.266+0000 7faf17400640 1 -- 172.21.15.83:0/919767006 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf18004a00 con 0x55dca80e4850 2024-04-24T09:02:58.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.267+0000 7faf337fd740 1 -- 172.21.15.83:0/919767006 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dca80e4850 msgr2=0x55dca80e6c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:58.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.267+0000 7faf29e00640 1 --2- 172.21.15.83:0/3089301565 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7faf100a3d20 0x7faf100a61e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x55dca80dae20 tx=0x7faf1c008040 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:02:58.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.267+0000 7faf337fd740 1 --2- 172.21.15.83:0/919767006 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dca80e4850 0x55dca80e6c40 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7faf2c06b750 tx=0x7faf180058b0 comp rx=0 tx=0).stop 2024-04-24T09:02:58.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.267+0000 7faf337fd740 1 -- 172.21.15.83:0/919767006 shutdown_connections 2024-04-24T09:02:58.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.267+0000 7faf337fd740 1 --2- 172.21.15.83:0/919767006 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dca80e4850 0x55dca80e6c40 unknown :-1 s=CLOSED pgs=1815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:58.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.267+0000 7faf337fd740 1 --2- 172.21.15.83:0/919767006 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dca80da180 0x55dca80e4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:58.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.267+0000 7faf337fd740 1 --2- 172.21.15.83:0/919767006 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dca80d9860 0x55dca80d9c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:58.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.267+0000 7faf337fd740 1 -- 172.21.15.83:0/919767006 >> 172.21.15.83:0/919767006 conn(0x55dca80d5b80 msgr2=0x55dca80d5f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:02:58.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.267+0000 7faf337fd740 1 -- 172.21.15.83:0/919767006 shutdown_connections 2024-04-24T09:02:58.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.267+0000 7faf337fd740 1 -- 172.21.15.83:0/919767006 wait complete. 2024-04-24T09:02:58.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.268+0000 7faf337fd740 1 -- start start 2024-04-24T09:02:58.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.268+0000 7faf337fd740 1 Processor -- start 2024-04-24T09:02:58.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.268+0000 7faf337fd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dca80d9860 0x55dca80e17e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:58.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.268+0000 7faf337fd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dca80da180 0x55dca80e1d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:58.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.268+0000 7faf337fd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dca80e4850 0x55dca80e2260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:58.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.268+0000 7faf337fd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55dca80e75e0 con 0x55dca80e4850 2024-04-24T09:02:58.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.268+0000 7faf337fd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55dca7ff8690 con 0x55dca80d9860 2024-04-24T09:02:58.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.268+0000 7faf337fd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55dca80e7760 con 0x55dca80da180 2024-04-24T09:02:58.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.269+0000 7faf337fd740 10 client.0 _ll_register_callbacks cb 0x55dca8034650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:02:58.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.269+0000 7faf2a800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dca80e4850 0x55dca80e2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:58.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.269+0000 7faf2a800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dca80e4850 0x55dca80e2260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33748/0 (socket says 172.21.15.83:33748) 2024-04-24T09:02:58.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.269+0000 7faf2a800640 1 -- 172.21.15.83:0/2962151178 learned_addr learned my addr 172.21.15.83:0/2962151178 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:02:58.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.269+0000 7faf29e00640 1 --2- 172.21.15.83:0/2962151178 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dca80d9860 0x55dca80e17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:58.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.269+0000 7faf2a800640 1 -- 172.21.15.83:0/2962151178 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dca80da180 msgr2=0x55dca80e1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:58.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.269+0000 7faf29400640 1 --2- 172.21.15.83:0/2962151178 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dca80da180 0x55dca80e1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:58.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.269+0000 7faf2a800640 1 --2- 172.21.15.83:0/2962151178 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dca80da180 0x55dca80e1d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:58.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.269+0000 7faf2a800640 1 -- 172.21.15.83:0/2962151178 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dca80d9860 msgr2=0x55dca80e17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:58.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.269+0000 7faf2a800640 1 --2- 172.21.15.83:0/2962151178 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dca80d9860 0x55dca80e17e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:58.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.269+0000 7faf2a800640 1 -- 172.21.15.83:0/2962151178 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dca7ff7920 con 0x55dca80e4850 2024-04-24T09:02:58.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.269+0000 7faf29e00640 1 --2- 172.21.15.83:0/2962151178 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dca80d9860 0x55dca80e17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:02:58.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.269+0000 7faf29400640 1 --2- 172.21.15.83:0/2962151178 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dca80da180 0x55dca80e1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:02:58.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.270+0000 7faf2a800640 1 --2- 172.21.15.83:0/2962151178 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dca80e4850 0x55dca80e2260 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7faf240981f0 tx=0x7faf24014aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c790fedee272cb8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:02:58.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.270+0000 7faf0be00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:02:58.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.270+0000 7faf0be00640 1 -- 172.21.15.83:0/2962151178 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf240b5060 con 0x55dca80e4850 2024-04-24T09:02:58.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.270+0000 7faf0be00640 1 -- 172.21.15.83:0/2962151178 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9583+0+0 (secure 0 0 0) 0x7faf240145f0 con 0x55dca80e4850 2024-04-24T09:02:58.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.270+0000 7faf337fd740 1 -- 172.21.15.83:0/2962151178 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf18005020 con 0x55dca80e4850 2024-04-24T09:02:58.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.270+0000 7faf0be00640 1 -- 172.21.15.83:0/2962151178 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7faf240a4a00 con 0x55dca80e4850 2024-04-24T09:02:58.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.270+0000 7faf337fd740 10 client.19608 fetch_fsmap learned FSMap version 41 2024-04-24T09:02:58.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.270+0000 7faf337fd740 1 -- 172.21.15.83:0/2962151178 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55dca7e1b7d0 con 0x55dca80e4850 2024-04-24T09:02:58.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.270+0000 7faf0be00640 1 -- 172.21.15.83:0/2962151178 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faf18005020 con 0x55dca80e4850 2024-04-24T09:02:58.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.271+0000 7faf0be00640 1 -- 172.21.15.83:0/2962151178 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7faf2407abb0 con 0x55dca80e4850 2024-04-24T09:02:58.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.271+0000 7faf337fd740 10 client.19608 fetch_fsmap finished waiting for FSMap version 41 2024-04-24T09:02:58.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.271+0000 7faf337fd740 10 client.19608 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:02:58.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.271+0000 7faf337fd740 20 client.19608 populate_metadata read hostname 'smithi083' 2024-04-24T09:02:58.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.271+0000 7faf337fd740 1 --2- 172.21.15.83:0/2962151178 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55dca80f4bd0 0x55dca80f7050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:58.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.271+0000 7faf337fd740 4 client.19608 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:02:58.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.271+0000 7faf337fd740 1 -- 172.21.15.83:0/2962151178 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dca804cc10 con 0x55dca80f4bd0 2024-04-24T09:02:58.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.271+0000 7faf29e00640 1 --2- 172.21.15.83:0/2962151178 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55dca80f4bd0 0x55dca80f7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:58.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.273+0000 7faf29e00640 1 --2- 172.21.15.83:0/2962151178 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55dca80f4bd0 0x55dca80f7050 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbfee8bc68cb1dba server_cookie=6f097937f1c66c54 in_seq=0 out_seq=0 2024-04-24T09:02:58.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.273+0000 7faf0be00640 10 client.19608 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:02:58.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.274+0000 7faf0be00640 1 -- 172.21.15.83:0/2962151178 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55dca804cc10 con 0x55dca80f4bd0 2024-04-24T09:02:58.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.274+0000 7faf0be00640 10 client.19608 handle_command_reply: tid=0 2024-04-24T09:02:58.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.274+0000 7faf337fd740 1 client.19608 shutdown 2024-04-24T09:02:58.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.275+0000 7faf337fd740 1 -- 172.21.15.83:0/2962151178 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dca80e4850 msgr2=0x55dca80e2260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:58.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.275+0000 7faf337fd740 1 --2- 172.21.15.83:0/2962151178 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dca80e4850 0x55dca80e2260 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7faf240981f0 tx=0x7faf24014aa0 comp rx=0 tx=0).stop 2024-04-24T09:02:58.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.275+0000 7faf337fd740 1 -- 172.21.15.83:0/2962151178 shutdown_connections 2024-04-24T09:02:58.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.275+0000 7faf337fd740 1 --2- 172.21.15.83:0/2962151178 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55dca80f4bd0 0x55dca80f7050 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:58.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.275+0000 7faf337fd740 1 --2- 172.21.15.83:0/2962151178 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dca80e4850 0x55dca80e2260 unknown :-1 s=CLOSED pgs=1816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:58.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.275+0000 7faf0be00640 0 client.19608 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:02:58.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.275+0000 7faf337fd740 1 --2- 172.21.15.83:0/2962151178 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dca80da180 0x55dca80e1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:58.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.275+0000 7faf337fd740 1 --2- 172.21.15.83:0/2962151178 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dca80d9860 0x55dca80e17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:58.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.275+0000 7faf337fd740 1 -- 172.21.15.83:0/2962151178 >> 172.21.15.83:0/2962151178 conn(0x55dca80d5b80 msgr2=0x55dca80db980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:02:58.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.275+0000 7faf337fd740 1 -- 172.21.15.83:0/2962151178 shutdown_connections 2024-04-24T09:02:58.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.275+0000 7faf337fd740 1 -- 172.21.15.83:0/2962151178 wait complete. 2024-04-24T09:02:58.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.275+0000 7faf337fd740 20 client.19608 trim_cache size 0 max 16384 2024-04-24T09:02:58.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.299+0000 7faf337fd740 1 Processor -- start 2024-04-24T09:02:58.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.299+0000 7faf337fd740 1 -- start start 2024-04-24T09:02:58.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.300+0000 7faf337fd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dca80d5b80 0x55dca80f04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:58.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.300+0000 7faf337fd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dca80d9860 0x55dca80d5f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:58.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.300+0000 7faf337fd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dca80da180 0x55dca80dc000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:58.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.300+0000 7faf337fd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7faf2c1b1880 con 0x55dca80da180 2024-04-24T09:02:58.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.300+0000 7faf337fd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7faf2c1b34e0 con 0x55dca80d9860 2024-04-24T09:02:58.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.300+0000 7faf337fd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7faf2c1b1f50 con 0x55dca80d5b80 2024-04-24T09:02:58.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.300+0000 7faf2a800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dca80da180 0x55dca80dc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:58.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.300+0000 7faf2a800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dca80da180 0x55dca80dc000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33750/0 (socket says 172.21.15.83:33750) 2024-04-24T09:02:58.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.300+0000 7faf2a800640 1 -- 172.21.15.83:0/4147711508 learned_addr learned my addr 172.21.15.83:0/4147711508 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:02:58.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.300+0000 7faf29e00640 1 --2- 172.21.15.83:0/4147711508 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dca80d5b80 0x55dca80f04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:58.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.300+0000 7faf29400640 1 --2- 172.21.15.83:0/4147711508 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dca80d9860 0x55dca80d5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:58.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.301+0000 7faf2a800640 1 -- 172.21.15.83:0/4147711508 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dca80d5b80 msgr2=0x55dca80f04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:58.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.301+0000 7faf2a800640 1 --2- 172.21.15.83:0/4147711508 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dca80d5b80 0x55dca80f04a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:58.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.301+0000 7faf2a800640 1 -- 172.21.15.83:0/4147711508 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dca80d9860 msgr2=0x55dca80d5f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:58.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.301+0000 7faf2a800640 1 --2- 172.21.15.83:0/4147711508 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dca80d9860 0x55dca80d5f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:58.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.301+0000 7faf2a800640 1 -- 172.21.15.83:0/4147711508 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dca7d513f0 con 0x55dca80da180 2024-04-24T09:02:58.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.301+0000 7faf29e00640 1 --2- 172.21.15.83:0/4147711508 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dca80d5b80 0x55dca80f04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:02:58.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.301+0000 7faf29400640 1 --2- 172.21.15.83:0/4147711508 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dca80d9860 0x55dca80d5f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:02:58.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.301+0000 7faf2a800640 1 --2- 172.21.15.83:0/4147711508 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dca80da180 0x55dca80dc000 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7faf240cda30 tx=0x7faf2400c590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df0b66c74ee74172 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:02:58.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.301+0000 7faf16a00640 1 -- 172.21.15.83:0/4147711508 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf240b9020 con 0x55dca80da180 2024-04-24T09:02:58.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.301+0000 7faf16a00640 1 -- 172.21.15.83:0/4147711508 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faf240090a0 con 0x55dca80da180 2024-04-24T09:02:58.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.301+0000 7faf16a00640 1 -- 172.21.15.83:0/4147711508 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf240b5050 con 0x55dca80da180 2024-04-24T09:02:58.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.302+0000 7faf337fd740 1 -- 172.21.15.83:0/4147711508 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dca80da180 msgr2=0x55dca80dc000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:58.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.302+0000 7faf337fd740 1 --2- 172.21.15.83:0/4147711508 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dca80da180 0x55dca80dc000 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7faf240cda30 tx=0x7faf2400c590 comp rx=0 tx=0).stop 2024-04-24T09:02:58.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.302+0000 7faf337fd740 1 -- 172.21.15.83:0/4147711508 shutdown_connections 2024-04-24T09:02:58.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.302+0000 7faf337fd740 1 --2- 172.21.15.83:0/4147711508 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dca80da180 0x55dca80dc000 unknown :-1 s=CLOSED pgs=1817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:58.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.302+0000 7faf337fd740 1 --2- 172.21.15.83:0/4147711508 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dca80d9860 0x55dca80d5f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:58.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.302+0000 7faf337fd740 1 --2- 172.21.15.83:0/4147711508 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dca80d5b80 0x55dca80f04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:58.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.302+0000 7faf337fd740 1 -- 172.21.15.83:0/4147711508 >> 172.21.15.83:0/4147711508 conn(0x55dca80dd1b0 msgr2=0x55dca80d4390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:02:58.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.302+0000 7faf337fd740 1 -- 172.21.15.83:0/4147711508 shutdown_connections 2024-04-24T09:02:58.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.302+0000 7faf337fd740 1 -- 172.21.15.83:0/4147711508 wait complete. 2024-04-24T09:02:58.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.302+0000 7faf337fd740 1 -- start start 2024-04-24T09:02:58.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.303+0000 7faf337fd740 1 Processor -- start 2024-04-24T09:02:58.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.303+0000 7faf337fd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dca80d5b80 0x55dca80ecb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:58.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.303+0000 7faf337fd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dca80d9860 0x55dca80e7f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:58.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.303+0000 7faf337fd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dca80da180 0x55dca80e84b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:58.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.303+0000 7faf337fd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7faf2c1b2db0 con 0x55dca80da180 2024-04-24T09:02:58.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.303+0000 7faf337fd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7faf2c1b2680 con 0x55dca80d9860 2024-04-24T09:02:58.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.303+0000 7faf337fd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7faf2c1af880 con 0x55dca80d5b80 2024-04-24T09:02:58.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.303+0000 7faf337fd740 10 client.0 _ll_register_callbacks cb 0x7faf24098030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:02:58.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.303+0000 7faf2a800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dca80da180 0x55dca80e84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:58.306 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:02:57 smithi083 ceph-mon[27907]: pgmap v355: 129 pgs: 129 active+clean; 60 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 314 KiB/s wr, 54 op/s 2024-04-24T09:02:58.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.304+0000 7faf2a800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dca80da180 0x55dca80e84b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33758/0 (socket says 172.21.15.83:33758) 2024-04-24T09:02:58.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.304+0000 7faf2a800640 1 -- 172.21.15.83:0/659542207 learned_addr learned my addr 172.21.15.83:0/659542207 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:02:58.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.304+0000 7faf29e00640 1 --2- 172.21.15.83:0/659542207 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dca80d5b80 0x55dca80ecb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:58.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.304+0000 7faf29400640 1 --2- 172.21.15.83:0/659542207 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dca80d9860 0x55dca80e7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:58.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.304+0000 7faf2a800640 1 -- 172.21.15.83:0/659542207 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dca80d5b80 msgr2=0x55dca80ecb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:58.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.304+0000 7faf2a800640 1 --2- 172.21.15.83:0/659542207 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dca80d5b80 0x55dca80ecb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:58.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.304+0000 7faf2a800640 1 -- 172.21.15.83:0/659542207 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dca80d9860 msgr2=0x55dca80e7f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:58.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.304+0000 7faf2a800640 1 --2- 172.21.15.83:0/659542207 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dca80d9860 0x55dca80e7f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:58.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.304+0000 7faf2a800640 1 -- 172.21.15.83:0/659542207 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dca7d110f0 con 0x55dca80da180 2024-04-24T09:02:58.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.304+0000 7faf2a800640 1 --2- 172.21.15.83:0/659542207 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dca80da180 0x55dca80e84b0 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7faf24014b90 tx=0x7faf240b5410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae13e5007ff23365 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:02:58.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.304+0000 7faf0be00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:02:58.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.304+0000 7faf0be00640 1 -- 172.21.15.83:0/659542207 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf240b9050 con 0x55dca80da180 2024-04-24T09:02:58.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.304+0000 7faf0be00640 1 -- 172.21.15.83:0/659542207 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9583+0+0 (secure 0 0 0) 0x7faf240a4770 con 0x55dca80da180 2024-04-24T09:02:58.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.304+0000 7faf337fd740 1 -- 172.21.15.83:0/659542207 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf240090a0 con 0x55dca80da180 2024-04-24T09:02:58.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.305+0000 7faf0be00640 1 -- 172.21.15.83:0/659542207 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7faf240b5040 con 0x55dca80da180 2024-04-24T09:02:58.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.305+0000 7faf337fd740 10 client.19614 fetch_fsmap learned FSMap version 41 2024-04-24T09:02:58.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.305+0000 7faf337fd740 1 -- 172.21.15.83:0/659542207 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55dca8004810 con 0x55dca80da180 2024-04-24T09:02:58.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.305+0000 7faf0be00640 1 -- 172.21.15.83:0/659542207 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faf240090a0 con 0x55dca80da180 2024-04-24T09:02:58.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.305+0000 7faf0be00640 1 -- 172.21.15.83:0/659542207 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7faf240bc0b0 con 0x55dca80da180 2024-04-24T09:02:58.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.305+0000 7faf337fd740 10 client.19614 fetch_fsmap finished waiting for FSMap version 41 2024-04-24T09:02:58.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.305+0000 7faf337fd740 10 client.19614 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:02:58.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.305+0000 7faf337fd740 20 client.19614 populate_metadata read hostname 'smithi083' 2024-04-24T09:02:58.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.305+0000 7faf337fd740 1 --2- 172.21.15.83:0/659542207 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55dca80e4850 0x55dca815cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:58.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.305+0000 7faf337fd740 4 client.19614 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T09:02:58.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.305+0000 7faf337fd740 1 -- 172.21.15.83:0/659542207 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55dca8046a80 con 0x55dca80e4850 2024-04-24T09:02:58.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.306+0000 7faf29e00640 1 --2- 172.21.15.83:0/659542207 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55dca80e4850 0x55dca815cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:58.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.307+0000 7faf29e00640 1 --2- 172.21.15.83:0/659542207 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55dca80e4850 0x55dca815cf00 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7bebb67d7086c022 server_cookie=61d30a6a4556ed3a in_seq=0 out_seq=0 2024-04-24T09:02:58.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.307+0000 7faf0be00640 10 client.19614 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:02:58.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.309+0000 7faf0be00640 1 -- 172.21.15.83:0/659542207 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55dca8046a80 con 0x55dca80e4850 2024-04-24T09:02:58.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.309+0000 7faf0be00640 10 client.19614 handle_command_reply: tid=0 2024-04-24T09:02:58.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.309+0000 7faf337fd740 1 client.19614 shutdown 2024-04-24T09:02:58.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.309+0000 7faf337fd740 1 -- 172.21.15.83:0/659542207 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dca80da180 msgr2=0x55dca80e84b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:58.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.309+0000 7faf337fd740 1 --2- 172.21.15.83:0/659542207 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dca80da180 0x55dca80e84b0 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7faf24014b90 tx=0x7faf240b5410 comp rx=0 tx=0).stop 2024-04-24T09:02:58.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.309+0000 7faf337fd740 1 -- 172.21.15.83:0/659542207 shutdown_connections 2024-04-24T09:02:58.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.309+0000 7faf337fd740 1 --2- 172.21.15.83:0/659542207 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55dca80e4850 0x55dca815cf00 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:58.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.309+0000 7faf337fd740 1 --2- 172.21.15.83:0/659542207 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dca80da180 0x55dca80e84b0 unknown :-1 s=CLOSED pgs=1818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:58.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.309+0000 7faf337fd740 1 --2- 172.21.15.83:0/659542207 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dca80d9860 0x55dca80e7f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:58.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.309+0000 7faf337fd740 1 --2- 172.21.15.83:0/659542207 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dca80d5b80 0x55dca80ecb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:58.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.309+0000 7faf0be00640 0 client.19614 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:02:58.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.309+0000 7faf337fd740 1 -- 172.21.15.83:0/659542207 >> 172.21.15.83:0/659542207 conn(0x55dca80dd1b0 msgr2=0x55dca80dfc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:02:58.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.310+0000 7faf337fd740 1 -- 172.21.15.83:0/659542207 shutdown_connections 2024-04-24T09:02:58.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.310+0000 7faf337fd740 1 -- 172.21.15.83:0/659542207 wait complete. 2024-04-24T09:02:58.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.310+0000 7faf337fd740 20 client.19614 trim_cache size 0 max 16384 2024-04-24T09:02:58.312 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T09:02:58.312 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-04-24T09:02:58.312 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-04-24T09:02:58.312 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T09:02:58.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.310+0000 7faf30e00640 1 -- 172.21.15.83:0/3089301565 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7faf100a3d20 msgr2=0x7faf100a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:58.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.310+0000 7faf30e00640 1 --2- 172.21.15.83:0/3089301565 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7faf100a3d20 0x7faf100a61e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x55dca80dae20 tx=0x7faf1c008040 comp rx=0 tx=0).stop 2024-04-24T09:02:58.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.310+0000 7faf30e00640 1 -- 172.21.15.83:0/3089301565 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf2c19a9f0 msgr2=0x7faf2c19aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:58.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.310+0000 7faf30e00640 1 --2- 172.21.15.83:0/3089301565 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf2c19a9f0 0x7faf2c19aea0 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7faf2400b970 tx=0x7faf2400be30 comp rx=0 tx=0).stop 2024-04-24T09:02:58.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.311+0000 7faf30e00640 1 -- 172.21.15.83:0/3089301565 shutdown_connections 2024-04-24T09:02:58.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.311+0000 7faf30e00640 1 --2- 172.21.15.83:0/3089301565 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7faf100a3d20 0x7faf100a61e0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:58.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.311+0000 7faf30e00640 1 --2- 172.21.15.83:0/3089301565 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf2c19a9f0 0x7faf2c19aea0 unknown :-1 s=CLOSED pgs=1814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:58.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.311+0000 7faf30e00640 1 --2- 172.21.15.83:0/3089301565 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faf2c109580 0x7faf2c1a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:58.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.311+0000 7faf30e00640 1 --2- 172.21.15.83:0/3089301565 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faf2c1044b0 0x7faf2c1a07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:58.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.311+0000 7faf30e00640 1 -- 172.21.15.83:0/3089301565 >> 172.21.15.83:0/3089301565 conn(0x7faf2c0ffd90 msgr2=0x7faf2c1011f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:02:58.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.311+0000 7faf30e00640 1 -- 172.21.15.83:0/3089301565 shutdown_connections 2024-04-24T09:02:58.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.311+0000 7faf30e00640 1 -- 172.21.15.83:0/3089301565 wait complete. 2024-04-24T09:02:58.328 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T09:02:58.328 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T09:02:58.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.534+0000 7efcee200640 1 -- 172.21.15.83:0/2090867995 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efce8101980 msgr2=0x7efce81059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:58.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.534+0000 7efcee200640 1 --2- 172.21.15.83:0/2090867995 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efce8101980 0x7efce81059c0 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7efcd4009900 tx=0x7efcd402a410 comp rx=0 tx=0).stop 2024-04-24T09:02:58.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.534+0000 7efcee200640 1 -- 172.21.15.83:0/2090867995 shutdown_connections 2024-04-24T09:02:58.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.534+0000 7efcee200640 1 --2- 172.21.15.83:0/2090867995 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efce81060a0 0x7efce810dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:58.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.534+0000 7efcee200640 1 --2- 172.21.15.83:0/2090867995 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efce8101980 0x7efce81059c0 unknown :-1 s=CLOSED pgs=1819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:58.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.534+0000 7efcee200640 1 --2- 172.21.15.83:0/2090867995 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efce8100fd0 0x7efce81013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:58.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.534+0000 7efcee200640 1 -- 172.21.15.83:0/2090867995 >> 172.21.15.83:0/2090867995 conn(0x7efce80fc830 msgr2=0x7efce80fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:02:58.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.534+0000 7efcee200640 1 -- 172.21.15.83:0/2090867995 shutdown_connections 2024-04-24T09:02:58.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.534+0000 7efcee200640 1 -- 172.21.15.83:0/2090867995 wait complete. 2024-04-24T09:02:58.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.535+0000 7efcee200640 1 Processor -- start 2024-04-24T09:02:58.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.535+0000 7efcee200640 1 -- start start 2024-04-24T09:02:58.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.535+0000 7efcee200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efce8100fd0 0x7efce810e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:58.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.536+0000 7efcee200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efce81060a0 0x7efce810eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:58.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.536+0000 7efcee200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efce810f080 0x7efce81a68c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:58.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.536+0000 7efcee200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7efce8112150 con 0x7efce810f080 2024-04-24T09:02:58.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.536+0000 7efcee200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efce8111fd0 con 0x7efce81060a0 2024-04-24T09:02:58.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.536+0000 7efcee200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7efce81122d0 con 0x7efce8100fd0 2024-04-24T09:02:58.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.536+0000 7efced800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efce810f080 0x7efce81a68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:58.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.536+0000 7efced800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efce810f080 0x7efce81a68c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33790/0 (socket says 172.21.15.83:33790) 2024-04-24T09:02:58.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.536+0000 7efced800640 1 -- 172.21.15.83:0/3211832790 learned_addr learned my addr 172.21.15.83:0/3211832790 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:02:58.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.536+0000 7efcece00640 1 --2- 172.21.15.83:0/3211832790 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efce8100fd0 0x7efce810e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:58.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.536+0000 7efce7e00640 1 --2- 172.21.15.83:0/3211832790 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efce81060a0 0x7efce810eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:58.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.536+0000 7efced800640 1 -- 172.21.15.83:0/3211832790 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efce8100fd0 msgr2=0x7efce810e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:58.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.536+0000 7efced800640 1 --2- 172.21.15.83:0/3211832790 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efce8100fd0 0x7efce810e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:58.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.536+0000 7efced800640 1 -- 172.21.15.83:0/3211832790 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efce81060a0 msgr2=0x7efce810eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:58.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.536+0000 7efced800640 1 --2- 172.21.15.83:0/3211832790 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efce81060a0 0x7efce810eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:58.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.536+0000 7efced800640 1 -- 172.21.15.83:0/3211832790 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efce81a6f60 con 0x7efce810f080 2024-04-24T09:02:58.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.536+0000 7efcece00640 1 --2- 172.21.15.83:0/3211832790 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efce8100fd0 0x7efce810e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:02:58.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.536+0000 7efce7e00640 1 --2- 172.21.15.83:0/3211832790 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efce81060a0 0x7efce810eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:02:58.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.537+0000 7efced800640 1 --2- 172.21.15.83:0/3211832790 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efce810f080 0x7efce81a68c0 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7efce000cce0 tx=0x7efce000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:02:58.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.537+0000 7efce5600640 1 -- 172.21.15.83:0/3211832790 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efce000f450 con 0x7efce810f080 2024-04-24T09:02:58.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.537+0000 7efce5600640 1 -- 172.21.15.83:0/3211832790 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efce000faa0 con 0x7efce810f080 2024-04-24T09:02:58.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.537+0000 7efcee200640 1 -- 172.21.15.83:0/3211832790 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efce81a7250 con 0x7efce810f080 2024-04-24T09:02:58.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.537+0000 7efce5600640 1 -- 172.21.15.83:0/3211832790 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efce0018b70 con 0x7efce810f080 2024-04-24T09:02:58.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.537+0000 7efcee200640 1 -- 172.21.15.83:0/3211832790 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efce81a7750 con 0x7efce810f080 2024-04-24T09:02:58.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.538+0000 7efcee200640 1 -- 172.21.15.83:0/3211832790 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efce81024a0 con 0x7efce810f080 2024-04-24T09:02:58.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.540+0000 7efce5600640 1 -- 172.21.15.83:0/3211832790 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7efce0018d10 con 0x7efce810f080 2024-04-24T09:02:58.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.542+0000 7efce5600640 1 --2- 172.21.15.83:0/3211832790 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7efcd00a3d20 0x7efcd00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:58.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.542+0000 7efce5600640 1 -- 172.21.15.83:0/3211832790 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9583+0+0 (secure 0 0 0) 0x7efce00d1130 con 0x7efce810f080 2024-04-24T09:02:58.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.542+0000 7efcece00640 1 --2- 172.21.15.83:0/3211832790 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7efcd00a3d20 0x7efcd00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:58.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.543+0000 7efcece00640 1 --2- 172.21.15.83:0/3211832790 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7efcd00a3d20 0x7efcd00a61e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7efcdc0058c0 tx=0x7efcdc005850 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:02:58.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.547+0000 7efce5600640 1 -- 172.21.15.83:0/3211832790 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efce0014030 con 0x7efce810f080 2024-04-24T09:02:58.616 INFO:tasks.workunit.client.0.smithi083.stdout: 1 66076 0.61 MB/sec execute 452 sec latency 55.498 ms 2024-04-24T09:02:58.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.832+0000 7efcee200640 1 -- 172.21.15.83:0/3211832790 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efce8109330 con 0x7efce810f080 2024-04-24T09:02:58.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.833+0000 7efce5600640 1 -- 172.21.15.83:0/3211832790 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+11953 (secure 0 0 0) 0x7efce009d350 con 0x7efce810f080 2024-04-24T09:02:58.832 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T09:02:58.832 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":41,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/358872257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":358872257},{"type":"v1","addr":"172.21.15.83:6835","nonce":358872257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3157070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3157070851},{"type":"v1","addr":"172.21.15.83:6837","nonce":3157070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":41,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T08:48:29.422576+0000","modified":"2024-04-24T09:02:18.079759+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34520,"mds_1":34493,"mds_2":34451,"mds_3":34427,"mds_4":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14658":{"gid":14658,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6839/4085470844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":4085470844},{"type":"v1","addr":"172.21.15.83:6839","nonce":4085470844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14676":{"gid":14676,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6841/2634754210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":2634754210},{"type":"v1","addr":"172.21.15.83:6841","nonce":2634754210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24382":{"gid":24382,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/1859921310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1859921310},{"type":"v1","addr":"172.21.15.189:6833","nonce":1859921310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/2241923197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2241923197},{"type":"v1","addr":"172.21.15.189:6835","nonce":2241923197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.189:6839/2548965336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":2548965336},{"type":"v1","addr":"172.21.15.189:6839","nonce":2548965336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26686":{"gid":26686,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6837/1678817161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":1678817161},{"type":"v1","addr":"172.21.15.189:6837","nonce":1678817161}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.138:6833/1157967658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1157967658},{"type":"v1","addr":"172.21.15.138:6833","nonce":1157967658}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.138:6835/1909219758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1909219758},{"type":"v1","addr":"172.21.15.138:6835","nonce":1909219758}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34493":{"gid":34493,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.138:6837/900566754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":900566754},{"type":"v1","addr":"172.21.15.138:6837","nonce":900566754}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34520":{"gid":34520,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.138:6839/2236932393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2236932393},{"type":"v1","addr":"172.21.15.138:6839","nonce":2236932393}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34520,"qdb_cluster":[24454,34520,34427,34451,34493]},"id":1}]} 2024-04-24T09:02:58.833 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 41 2024-04-24T09:02:58.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.836+0000 7efcee200640 1 -- 172.21.15.83:0/3211832790 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7efcd00a3d20 msgr2=0x7efcd00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:58.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.836+0000 7efcee200640 1 --2- 172.21.15.83:0/3211832790 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7efcd00a3d20 0x7efcd00a61e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7efcdc0058c0 tx=0x7efcdc005850 comp rx=0 tx=0).stop 2024-04-24T09:02:58.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.836+0000 7efcee200640 1 -- 172.21.15.83:0/3211832790 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efce810f080 msgr2=0x7efce81a68c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:58.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.836+0000 7efcee200640 1 --2- 172.21.15.83:0/3211832790 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efce810f080 0x7efce81a68c0 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7efce000cce0 tx=0x7efce000a620 comp rx=0 tx=0).stop 2024-04-24T09:02:58.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.836+0000 7efcee200640 1 -- 172.21.15.83:0/3211832790 shutdown_connections 2024-04-24T09:02:58.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.836+0000 7efcee200640 1 --2- 172.21.15.83:0/3211832790 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7efcd00a3d20 0x7efcd00a61e0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:58.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.836+0000 7efcee200640 1 --2- 172.21.15.83:0/3211832790 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efce810f080 0x7efce81a68c0 unknown :-1 s=CLOSED pgs=1820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:58.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.836+0000 7efcee200640 1 --2- 172.21.15.83:0/3211832790 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efce81060a0 0x7efce810eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:58.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.836+0000 7efcee200640 1 --2- 172.21.15.83:0/3211832790 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efce8100fd0 0x7efce810e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:58.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.836+0000 7efcee200640 1 -- 172.21.15.83:0/3211832790 >> 172.21.15.83:0/3211832790 conn(0x7efce80fc830 msgr2=0x7efce80feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:02:58.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.837+0000 7efcee200640 1 -- 172.21.15.83:0/3211832790 shutdown_connections 2024-04-24T09:02:58.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:58.837+0000 7efcee200640 1 -- 172.21.15.83:0/3211832790 wait complete. 2024-04-24T09:02:58.855 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-24T09:02:59.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.065+0000 7fef46000640 1 -- 172.21.15.83:0/1264710195 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef40104e60 msgr2=0x7fef40108ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:59.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.065+0000 7fef46000640 1 --2- 172.21.15.83:0/1264710195 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef40104e60 0x7fef40108ea0 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7fef2c009900 tx=0x7fef2c02a410 comp rx=0 tx=0).stop 2024-04-24T09:02:59.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.066+0000 7fef46000640 1 -- 172.21.15.83:0/1264710195 shutdown_connections 2024-04-24T09:02:59.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.066+0000 7fef46000640 1 --2- 172.21.15.83:0/1264710195 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fef40109580 0x7fef40111150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.066+0000 7fef46000640 1 --2- 172.21.15.83:0/1264710195 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef40104e60 0x7fef40108ea0 unknown :-1 s=CLOSED pgs=1821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.066+0000 7fef46000640 1 --2- 172.21.15.83:0/1264710195 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fef401044b0 0x7fef40104890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.066+0000 7fef46000640 1 -- 172.21.15.83:0/1264710195 >> 172.21.15.83:0/1264710195 conn(0x7fef400ffd90 msgr2=0x7fef401021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:02:59.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.066+0000 7fef46000640 1 -- 172.21.15.83:0/1264710195 shutdown_connections 2024-04-24T09:02:59.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.066+0000 7fef46000640 1 -- 172.21.15.83:0/1264710195 wait complete. 2024-04-24T09:02:59.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.067+0000 7fef46000640 1 Processor -- start 2024-04-24T09:02:59.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.067+0000 7fef46000640 1 -- start start 2024-04-24T09:02:59.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.067+0000 7fef46000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fef401044b0 0x7fef401a0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:59.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.067+0000 7fef46000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef40109580 0x7fef401a0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:59.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.067+0000 7fef46000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fef4019a970 0x7fef4019ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:59.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.067+0000 7fef46000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fef40114570 con 0x7fef40109580 2024-04-24T09:02:59.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.067+0000 7fef46000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fef401143f0 con 0x7fef401044b0 2024-04-24T09:02:59.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.067+0000 7fef46000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fef401146f0 con 0x7fef4019a970 2024-04-24T09:02:59.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.068+0000 7fef3ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef40109580 0x7fef401a0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:59.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.068+0000 7fef3ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef40109580 0x7fef401a0cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45344/0 (socket says 172.21.15.83:45344) 2024-04-24T09:02:59.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.068+0000 7fef3ea00640 1 -- 172.21.15.83:0/3860840142 learned_addr learned my addr 172.21.15.83:0/3860840142 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:02:59.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.068+0000 7fef3f400640 1 --2- 172.21.15.83:0/3860840142 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fef401044b0 0x7fef401a0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:59.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.068+0000 7fef3fe00640 1 --2- 172.21.15.83:0/3860840142 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fef4019a970 0x7fef4019ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:59.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.068+0000 7fef3ea00640 1 -- 172.21.15.83:0/3860840142 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fef4019a970 msgr2=0x7fef4019ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:59.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.068+0000 7fef3ea00640 1 --2- 172.21.15.83:0/3860840142 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fef4019a970 0x7fef4019ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.068+0000 7fef3ea00640 1 -- 172.21.15.83:0/3860840142 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fef401044b0 msgr2=0x7fef401a0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:59.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.068+0000 7fef3ea00640 1 --2- 172.21.15.83:0/3860840142 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fef401044b0 0x7fef401a0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.068+0000 7fef3ea00640 1 -- 172.21.15.83:0/3860840142 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef4019b490 con 0x7fef40109580 2024-04-24T09:02:59.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.068+0000 7fef3ea00640 1 --2- 172.21.15.83:0/3860840142 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef40109580 0x7fef401a0cb0 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7fef2c031440 tx=0x7fef2c003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:02:59.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.069+0000 7fef37e00640 1 -- 172.21.15.83:0/3860840142 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef2c03f070 con 0x7fef40109580 2024-04-24T09:02:59.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.069+0000 7fef37e00640 1 -- 172.21.15.83:0/3860840142 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fef2c031d60 con 0x7fef40109580 2024-04-24T09:02:59.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.069+0000 7fef46000640 1 -- 172.21.15.83:0/3860840142 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef4019b720 con 0x7fef40109580 2024-04-24T09:02:59.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.069+0000 7fef37e00640 1 -- 172.21.15.83:0/3860840142 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef2c035030 con 0x7fef40109580 2024-04-24T09:02:59.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.069+0000 7fef46000640 1 -- 172.21.15.83:0/3860840142 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef401a7920 con 0x7fef40109580 2024-04-24T09:02:59.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.072+0000 7fef37e00640 1 -- 172.21.15.83:0/3860840142 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fef2c042070 con 0x7fef40109580 2024-04-24T09:02:59.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.073+0000 7fef48fcd740 1 Processor -- start 2024-04-24T09:02:59.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.073+0000 7fef48fcd740 1 -- start start 2024-04-24T09:02:59.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.073+0000 7fef48fcd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56483c87a950 0x56483c87ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:59.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.073+0000 7fef48fcd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56483c87b270 0x56483c885400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:59.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.073+0000 7fef48fcd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56483c885940 0x56483c887d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:59.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.073+0000 7fef48fcd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56483c7e01d0 con 0x56483c885940 2024-04-24T09:02:59.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.073+0000 7fef48fcd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56483c786450 con 0x56483c87a950 2024-04-24T09:02:59.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.073+0000 7fef48fcd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56483c7dc190 con 0x56483c87b270 2024-04-24T09:02:59.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.074+0000 7fef3ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56483c885940 0x56483c887d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:59.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.074+0000 7fef3ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56483c885940 0x56483c887d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45350/0 (socket says 172.21.15.83:45350) 2024-04-24T09:02:59.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.074+0000 7fef3ea00640 1 -- 172.21.15.83:0/4260793068 learned_addr learned my addr 172.21.15.83:0/4260793068 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:02:59.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.074+0000 7fef3fe00640 1 --2- 172.21.15.83:0/4260793068 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56483c87a950 0x56483c87ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:59.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.074+0000 7fef37e00640 1 --2- 172.21.15.83:0/3860840142 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fef240a3da0 0x7fef240a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:59.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.074+0000 7fef3f400640 1 --2- 172.21.15.83:0/4260793068 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56483c87b270 0x56483c885400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:59.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.074+0000 7fef37e00640 1 -- 172.21.15.83:0/3860840142 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9583+0+0 (secure 0 0 0) 0x7fef2c0f1590 con 0x7fef40109580 2024-04-24T09:02:59.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.074+0000 7fef3ea00640 1 -- 172.21.15.83:0/4260793068 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56483c87b270 msgr2=0x56483c885400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:59.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.074+0000 7fef3ea00640 1 --2- 172.21.15.83:0/4260793068 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56483c87b270 0x56483c885400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.074+0000 7fef3ea00640 1 -- 172.21.15.83:0/4260793068 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56483c87a950 msgr2=0x56483c87ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:59.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.074+0000 7fef3ea00640 1 --2- 172.21.15.83:0/4260793068 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56483c87a950 0x56483c87ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.074+0000 7fef3ea00640 1 -- 172.21.15.83:0/4260793068 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56483c843310 con 0x56483c885940 2024-04-24T09:02:59.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.074+0000 7fef3fe00640 1 --2- 172.21.15.83:0/3860840142 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fef240a3da0 0x7fef240a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:59.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.075+0000 7fef3ea00640 1 --2- 172.21.15.83:0/4260793068 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56483c885940 0x56483c887d30 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7fef2c0ee7a0 tx=0x7fef2c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebf417631c8a2fc8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:02:59.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.075+0000 7fef2be00640 1 -- 172.21.15.83:0/4260793068 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef2c049030 con 0x56483c885940 2024-04-24T09:02:59.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.075+0000 7fef2be00640 1 -- 172.21.15.83:0/4260793068 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fef2c0bd7b0 con 0x56483c885940 2024-04-24T09:02:59.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.075+0000 7fef2be00640 1 -- 172.21.15.83:0/4260793068 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef2c03b680 con 0x56483c885940 2024-04-24T09:02:59.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.075+0000 7fef3fe00640 1 --2- 172.21.15.83:0/3860840142 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fef240a3da0 0x7fef240a6260 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fef38008fc0 tx=0x7fef38004ab0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:02:59.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.076+0000 7fef48fcd740 1 -- 172.21.15.83:0/4260793068 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56483c885940 msgr2=0x56483c887d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:59.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.076+0000 7fef48fcd740 1 --2- 172.21.15.83:0/4260793068 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56483c885940 0x56483c887d30 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7fef2c0ee7a0 tx=0x7fef2c009260 comp rx=0 tx=0).stop 2024-04-24T09:02:59.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.076+0000 7fef48fcd740 1 -- 172.21.15.83:0/4260793068 shutdown_connections 2024-04-24T09:02:59.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.076+0000 7fef48fcd740 1 --2- 172.21.15.83:0/4260793068 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56483c885940 0x56483c887d30 unknown :-1 s=CLOSED pgs=1823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.076+0000 7fef48fcd740 1 --2- 172.21.15.83:0/4260793068 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56483c87b270 0x56483c885400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.076+0000 7fef48fcd740 1 --2- 172.21.15.83:0/4260793068 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56483c87a950 0x56483c87ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.076+0000 7fef48fcd740 1 -- 172.21.15.83:0/4260793068 >> 172.21.15.83:0/4260793068 conn(0x56483c876c70 msgr2=0x56483c877050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:02:59.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.076+0000 7fef48fcd740 1 -- 172.21.15.83:0/4260793068 shutdown_connections 2024-04-24T09:02:59.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.076+0000 7fef48fcd740 1 -- 172.21.15.83:0/4260793068 wait complete. 2024-04-24T09:02:59.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.076+0000 7fef48fcd740 1 -- start start 2024-04-24T09:02:59.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.077+0000 7fef48fcd740 1 Processor -- start 2024-04-24T09:02:59.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.077+0000 7fef48fcd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56483c87a950 0x56483c885400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:59.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.077+0000 7fef48fcd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56483c87b270 0x56483c8826f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:59.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.077+0000 7fef48fcd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56483c885940 0x56483c882e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:59.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.077+0000 7fef48fcd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56483c8884f0 con 0x56483c87a950 2024-04-24T09:02:59.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.077+0000 7fef48fcd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56483c799810 con 0x56483c87b270 2024-04-24T09:02:59.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.077+0000 7fef48fcd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56483c888670 con 0x56483c885940 2024-04-24T09:02:59.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.077+0000 7fef48fcd740 10 client.0 _ll_register_callbacks cb 0x56483c767140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:02:59.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.077+0000 7fef3f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56483c87a950 0x56483c885400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:59.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.077+0000 7fef3f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56483c87a950 0x56483c885400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45354/0 (socket says 172.21.15.83:45354) 2024-04-24T09:02:59.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.077+0000 7fef3f400640 1 -- 172.21.15.83:0/4000373276 learned_addr learned my addr 172.21.15.83:0/4000373276 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:02:59.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.078+0000 7fef3ea00640 1 --2- 172.21.15.83:0/4000373276 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56483c87b270 0x56483c8826f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:59.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.078+0000 7fef3fe00640 1 --2- 172.21.15.83:0/4000373276 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56483c885940 0x56483c882e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:59.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.078+0000 7fef3f400640 1 -- 172.21.15.83:0/4000373276 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56483c885940 msgr2=0x56483c882e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:59.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.078+0000 7fef3f400640 1 --2- 172.21.15.83:0/4000373276 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56483c885940 0x56483c882e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.078+0000 7fef3f400640 1 -- 172.21.15.83:0/4000373276 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56483c87b270 msgr2=0x56483c8826f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:59.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.078+0000 7fef3f400640 1 --2- 172.21.15.83:0/4000373276 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56483c87b270 0x56483c8826f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.078+0000 7fef3f400640 1 -- 172.21.15.83:0/4000373276 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56483c7aa3f0 con 0x56483c87a950 2024-04-24T09:02:59.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.078+0000 7fef3f400640 1 --2- 172.21.15.83:0/4000373276 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56483c87a950 0x56483c885400 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7fef3000bf90 tx=0x7fef3000f670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85e654081196b734 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:02:59.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.078+0000 7fef28c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:02:59.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.079+0000 7fef28c00640 1 -- 172.21.15.83:0/4000373276 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef30018020 con 0x56483c87a950 2024-04-24T09:02:59.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.079+0000 7fef28c00640 1 -- 172.21.15.83:0/4000373276 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9583+0+0 (secure 0 0 0) 0x7fef30014870 con 0x56483c87a950 2024-04-24T09:02:59.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.079+0000 7fef48fcd740 1 -- 172.21.15.83:0/4000373276 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef2c0bd7b0 con 0x56483c87a950 2024-04-24T09:02:59.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.079+0000 7fef28c00640 1 -- 172.21.15.83:0/4000373276 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fef30023050 con 0x56483c87a950 2024-04-24T09:02:59.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.079+0000 7fef48fcd740 10 client.19632 fetch_fsmap learned FSMap version 41 2024-04-24T09:02:59.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.079+0000 7fef48fcd740 1 -- 172.21.15.83:0/4000373276 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x56483c4f23f0 con 0x56483c87a950 2024-04-24T09:02:59.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.079+0000 7fef28c00640 1 -- 172.21.15.83:0/4000373276 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fef2c0bd7b0 con 0x56483c87a950 2024-04-24T09:02:59.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.079+0000 7fef28c00640 1 -- 172.21.15.83:0/4000373276 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7fef30013a60 con 0x56483c87a950 2024-04-24T09:02:59.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.080+0000 7fef48fcd740 10 client.19632 fetch_fsmap finished waiting for FSMap version 41 2024-04-24T09:02:59.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.080+0000 7fef48fcd740 10 client.19632 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-24T09:02:59.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.080+0000 7fef48fcd740 20 client.19632 populate_metadata read hostname 'smithi083' 2024-04-24T09:02:59.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.080+0000 7fef48fcd740 1 --2- 172.21.15.83:0/4000373276 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x56483c8958c0 0x56483c897d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:59.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.080+0000 7fef48fcd740 4 client.19632 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:02:59.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.080+0000 7fef48fcd740 1 -- 172.21.15.83:0/4000373276 --> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56483c7edbf0 con 0x56483c8958c0 2024-04-24T09:02:59.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.080+0000 7fef3ea00640 1 --2- 172.21.15.83:0/4000373276 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x56483c8958c0 0x56483c897d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:59.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.081+0000 7fef3ea00640 1 --2- 172.21.15.83:0/4000373276 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x56483c8958c0 0x56483c897d50 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=df9f34b69c0257c5 server_cookie=44f82666eb6298e in_seq=0 out_seq=0 2024-04-24T09:02:59.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.081+0000 7fef28c00640 10 client.19632 ms_handle_connect on v2:172.21.15.189:6838/2548965336 2024-04-24T09:02:59.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.083+0000 7fef28c00640 1 -- 172.21.15.83:0/4000373276 <== mds.4 v2:172.21.15.189:6838/2548965336 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56483c7edbf0 con 0x56483c8958c0 2024-04-24T09:02:59.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.083+0000 7fef28c00640 10 client.19632 handle_command_reply: tid=0 2024-04-24T09:02:59.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.083+0000 7fef48fcd740 1 client.19632 shutdown 2024-04-24T09:02:59.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.083+0000 7fef48fcd740 1 -- 172.21.15.83:0/4000373276 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56483c87a950 msgr2=0x56483c885400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:59.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.083+0000 7fef48fcd740 1 --2- 172.21.15.83:0/4000373276 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56483c87a950 0x56483c885400 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7fef3000bf90 tx=0x7fef3000f670 comp rx=0 tx=0).stop 2024-04-24T09:02:59.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.083+0000 7fef48fcd740 1 -- 172.21.15.83:0/4000373276 shutdown_connections 2024-04-24T09:02:59.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.083+0000 7fef48fcd740 1 --2- 172.21.15.83:0/4000373276 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x56483c8958c0 0x56483c897d50 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.083+0000 7fef48fcd740 1 --2- 172.21.15.83:0/4000373276 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56483c885940 0x56483c882e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.083+0000 7fef48fcd740 1 --2- 172.21.15.83:0/4000373276 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56483c87b270 0x56483c8826f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.083+0000 7fef48fcd740 1 --2- 172.21.15.83:0/4000373276 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56483c87a950 0x56483c885400 unknown :-1 s=CLOSED pgs=1824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.083+0000 7fef48fcd740 1 -- 172.21.15.83:0/4000373276 >> 172.21.15.83:0/4000373276 conn(0x56483c876c70 msgr2=0x56483c87ca70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:02:59.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.083+0000 7fef28c00640 0 client.19632 ms_handle_reset on v2:172.21.15.189:6838/2548965336 2024-04-24T09:02:59.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.083+0000 7fef48fcd740 1 -- 172.21.15.83:0/4000373276 shutdown_connections 2024-04-24T09:02:59.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.083+0000 7fef48fcd740 1 -- 172.21.15.83:0/4000373276 wait complete. 2024-04-24T09:02:59.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.084+0000 7fef48fcd740 20 client.19632 trim_cache size 0 max 16384 2024-04-24T09:02:59.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.100+0000 7fef48fcd740 1 Processor -- start 2024-04-24T09:02:59.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.100+0000 7fef48fcd740 1 -- start start 2024-04-24T09:02:59.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.101+0000 7fef48fcd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56483c87a950 0x56483c884f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:59.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.101+0000 7fef48fcd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56483c87b270 0x56483c89d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:59.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.101+0000 7fef48fcd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56483c885940 0x56483c89db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:59.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.101+0000 7fef48fcd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fef401b1880 con 0x56483c87b270 2024-04-24T09:02:59.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.101+0000 7fef48fcd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fef401b34e0 con 0x56483c885940 2024-04-24T09:02:59.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.101+0000 7fef48fcd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fef401b1f50 con 0x56483c87a950 2024-04-24T09:02:59.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.101+0000 7fef3ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56483c87b270 0x56483c89d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:59.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.101+0000 7fef3ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56483c87b270 0x56483c89d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45370/0 (socket says 172.21.15.83:45370) 2024-04-24T09:02:59.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.101+0000 7fef3ea00640 1 -- 172.21.15.83:0/3818871781 learned_addr learned my addr 172.21.15.83:0/3818871781 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:02:59.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.101+0000 7fef3fe00640 1 --2- 172.21.15.83:0/3818871781 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56483c885940 0x56483c89db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:59.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.101+0000 7fef3f400640 1 --2- 172.21.15.83:0/3818871781 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56483c87a950 0x56483c884f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:59.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.101+0000 7fef3ea00640 1 -- 172.21.15.83:0/3818871781 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56483c87a950 msgr2=0x56483c884f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:59.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.101+0000 7fef3ea00640 1 --2- 172.21.15.83:0/3818871781 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56483c87a950 0x56483c884f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.101+0000 7fef3ea00640 1 -- 172.21.15.83:0/3818871781 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56483c885940 msgr2=0x56483c89db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:59.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.101+0000 7fef3ea00640 1 --2- 172.21.15.83:0/3818871781 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56483c885940 0x56483c89db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.101+0000 7fef3ea00640 1 -- 172.21.15.83:0/3818871781 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56483c5bc7d0 con 0x56483c87b270 2024-04-24T09:02:59.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.101+0000 7fef3f400640 1 --2- 172.21.15.83:0/3818871781 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56483c87a950 0x56483c884f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:02:59.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.101+0000 7fef3fe00640 1 --2- 172.21.15.83:0/3818871781 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56483c885940 0x56483c89db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:02:59.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.102+0000 7fef3ea00640 1 --2- 172.21.15.83:0/3818871781 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56483c87b270 0x56483c89d620 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7fef2c005ab0 tx=0x7fef2c0bcf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84a818145ceb8565 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:02:59.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.102+0000 7fef2b400640 1 -- 172.21.15.83:0/3818871781 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef2c049040 con 0x56483c87b270 2024-04-24T09:02:59.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.102+0000 7fef2b400640 1 -- 172.21.15.83:0/3818871781 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fef2c0bd0b0 con 0x56483c87b270 2024-04-24T09:02:59.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.102+0000 7fef2b400640 1 -- 172.21.15.83:0/3818871781 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef2c03b680 con 0x56483c87b270 2024-04-24T09:02:59.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.103+0000 7fef48fcd740 1 -- 172.21.15.83:0/3818871781 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56483c87b270 msgr2=0x56483c89d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:59.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.103+0000 7fef48fcd740 1 --2- 172.21.15.83:0/3818871781 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56483c87b270 0x56483c89d620 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7fef2c005ab0 tx=0x7fef2c0bcf80 comp rx=0 tx=0).stop 2024-04-24T09:02:59.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.103+0000 7fef48fcd740 1 -- 172.21.15.83:0/3818871781 shutdown_connections 2024-04-24T09:02:59.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.103+0000 7fef48fcd740 1 --2- 172.21.15.83:0/3818871781 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56483c885940 0x56483c89db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.103+0000 7fef48fcd740 1 --2- 172.21.15.83:0/3818871781 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56483c87b270 0x56483c89d620 unknown :-1 s=CLOSED pgs=1825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.103+0000 7fef48fcd740 1 --2- 172.21.15.83:0/3818871781 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56483c87a950 0x56483c884f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.103+0000 7fef48fcd740 1 -- 172.21.15.83:0/3818871781 >> 172.21.15.83:0/3818871781 conn(0x56483c876c70 msgr2=0x56483c87d980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:02:59.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.103+0000 7fef48fcd740 1 -- 172.21.15.83:0/3818871781 shutdown_connections 2024-04-24T09:02:59.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.103+0000 7fef48fcd740 1 -- 172.21.15.83:0/3818871781 wait complete. 2024-04-24T09:02:59.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.103+0000 7fef48fcd740 1 -- start start 2024-04-24T09:02:59.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.103+0000 7fef48fcd740 1 Processor -- start 2024-04-24T09:02:59.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.104+0000 7fef48fcd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56483c87a950 0x56483c886810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:59.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.104+0000 7fef48fcd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56483c87b270 0x56483c886d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:59.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.104+0000 7fef48fcd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56483c885940 0x56483c887290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:59.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.104+0000 7fef48fcd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fef401b2db0 con 0x56483c87b270 2024-04-24T09:02:59.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.104+0000 7fef48fcd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fef401b2680 con 0x56483c87a950 2024-04-24T09:02:59.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.104+0000 7fef48fcd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fef401af880 con 0x56483c885940 2024-04-24T09:02:59.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.104+0000 7fef48fcd740 10 client.0 _ll_register_callbacks cb 0x7fef3000b2a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:02:59.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.104+0000 7fef3ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56483c87b270 0x56483c886d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:59.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.104+0000 7fef3ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56483c87b270 0x56483c886d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45382/0 (socket says 172.21.15.83:45382) 2024-04-24T09:02:59.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.104+0000 7fef3ea00640 1 -- 172.21.15.83:0/2656886971 learned_addr learned my addr 172.21.15.83:0/2656886971 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:02:59.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.104+0000 7fef3f400640 1 --2- 172.21.15.83:0/2656886971 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56483c87a950 0x56483c886810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:59.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.104+0000 7fef3fe00640 1 --2- 172.21.15.83:0/2656886971 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56483c885940 0x56483c887290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:59.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.104+0000 7fef3ea00640 1 -- 172.21.15.83:0/2656886971 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56483c885940 msgr2=0x56483c887290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:59.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.104+0000 7fef3ea00640 1 --2- 172.21.15.83:0/2656886971 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56483c885940 0x56483c887290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.104+0000 7fef3ea00640 1 -- 172.21.15.83:0/2656886971 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56483c87a950 msgr2=0x56483c886810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:59.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.105+0000 7fef3ea00640 1 --2- 172.21.15.83:0/2656886971 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56483c87a950 0x56483c886810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.105+0000 7fef3ea00640 1 -- 172.21.15.83:0/2656886971 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56483c88be80 con 0x56483c87b270 2024-04-24T09:02:59.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.105+0000 7fef3f400640 1 --2- 172.21.15.83:0/2656886971 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56483c87a950 0x56483c886810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:02:59.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.105+0000 7fef3ea00640 1 --2- 172.21.15.83:0/2656886971 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56483c87b270 0x56483c886d50 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7fef2c0b9a80 tx=0x7fef2c0c1460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c76fb539d00a2c2e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:02:59.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.105+0000 7fef28c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:02:59.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.105+0000 7fef28c00640 1 -- 172.21.15.83:0/2656886971 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef2c0c10d0 con 0x56483c87b270 2024-04-24T09:02:59.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.105+0000 7fef28c00640 1 -- 172.21.15.83:0/2656886971 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9583+0+0 (secure 0 0 0) 0x7fef2c0b9720 con 0x56483c87b270 2024-04-24T09:02:59.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.105+0000 7fef48fcd740 1 -- 172.21.15.83:0/2656886971 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56483c7e7a60 con 0x56483c87b270 2024-04-24T09:02:59.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.105+0000 7fef28c00640 1 -- 172.21.15.83:0/2656886971 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fef2c03b680 con 0x56483c87b270 2024-04-24T09:02:59.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.105+0000 7fef48fcd740 10 client.19638 fetch_fsmap learned FSMap version 41 2024-04-24T09:02:59.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.105+0000 7fef48fcd740 1 -- 172.21.15.83:0/2656886971 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x56483c8fc2e0 con 0x56483c87b270 2024-04-24T09:02:59.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.106+0000 7fef28c00640 1 -- 172.21.15.83:0/2656886971 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x56483c7e7a60 con 0x56483c87b270 2024-04-24T09:02:59.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.106+0000 7fef28c00640 1 -- 172.21.15.83:0/2656886971 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7fef2c0f1170 con 0x56483c87b270 2024-04-24T09:02:59.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.106+0000 7fef48fcd740 10 client.19638 fetch_fsmap finished waiting for FSMap version 41 2024-04-24T09:02:59.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.106+0000 7fef48fcd740 10 client.19638 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-24T09:02:59.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.106+0000 7fef48fcd740 20 client.19638 populate_metadata read hostname 'smithi083' 2024-04-24T09:02:59.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.106+0000 7fef48fcd740 1 --2- 172.21.15.83:0/2656886971 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x56483c8fc560 0x56483c8fe980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:59.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.106+0000 7fef48fcd740 4 client.19638 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:02:59.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.106+0000 7fef48fcd740 1 -- 172.21.15.83:0/2656886971 --> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56483c7634d0 con 0x56483c8fc560 2024-04-24T09:02:59.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.106+0000 7fef3f400640 1 --2- 172.21.15.83:0/2656886971 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x56483c8fc560 0x56483c8fe980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:59.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.107+0000 7fef3f400640 1 --2- 172.21.15.83:0/2656886971 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x56483c8fc560 0x56483c8fe980 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a1a5ae4e821c136 server_cookie=b881e76bcefbf3ab in_seq=0 out_seq=0 2024-04-24T09:02:59.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.107+0000 7fef28c00640 10 client.19638 ms_handle_connect on v2:172.21.15.189:6838/2548965336 2024-04-24T09:02:59.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.108+0000 7fef28c00640 1 -- 172.21.15.83:0/2656886971 <== mds.4 v2:172.21.15.189:6838/2548965336 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56483c7634d0 con 0x56483c8fc560 2024-04-24T09:02:59.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.108+0000 7fef28c00640 10 client.19638 handle_command_reply: tid=0 2024-04-24T09:02:59.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.108+0000 7fef48fcd740 1 client.19638 shutdown 2024-04-24T09:02:59.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.108+0000 7fef48fcd740 1 -- 172.21.15.83:0/2656886971 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56483c87b270 msgr2=0x56483c886d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:59.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.108+0000 7fef48fcd740 1 --2- 172.21.15.83:0/2656886971 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56483c87b270 0x56483c886d50 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7fef2c0b9a80 tx=0x7fef2c0c1460 comp rx=0 tx=0).stop 2024-04-24T09:02:59.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.108+0000 7fef48fcd740 1 -- 172.21.15.83:0/2656886971 shutdown_connections 2024-04-24T09:02:59.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.108+0000 7fef48fcd740 1 --2- 172.21.15.83:0/2656886971 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x56483c8fc560 0x56483c8fe980 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.108+0000 7fef48fcd740 1 --2- 172.21.15.83:0/2656886971 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56483c885940 0x56483c887290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.108+0000 7fef48fcd740 1 --2- 172.21.15.83:0/2656886971 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56483c87b270 0x56483c886d50 unknown :-1 s=CLOSED pgs=1826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.108+0000 7fef48fcd740 1 --2- 172.21.15.83:0/2656886971 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56483c87a950 0x56483c886810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.108+0000 7fef48fcd740 1 -- 172.21.15.83:0/2656886971 >> 172.21.15.83:0/2656886971 conn(0x56483c876c70 msgr2=0x56483c875560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:02:59.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.108+0000 7fef28c00640 0 client.19638 ms_handle_reset on v2:172.21.15.189:6838/2548965336 2024-04-24T09:02:59.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.108+0000 7fef48fcd740 1 -- 172.21.15.83:0/2656886971 shutdown_connections 2024-04-24T09:02:59.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.108+0000 7fef48fcd740 1 -- 172.21.15.83:0/2656886971 wait complete. 2024-04-24T09:02:59.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.108+0000 7fef48fcd740 20 client.19638 trim_cache size 0 max 16384 2024-04-24T09:02:59.111 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:02:59.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.109+0000 7fef46000640 1 -- 172.21.15.83:0/3860840142 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fef240a3da0 msgr2=0x7fef240a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:59.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.109+0000 7fef46000640 1 --2- 172.21.15.83:0/3860840142 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fef240a3da0 0x7fef240a6260 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fef38008fc0 tx=0x7fef38004ab0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.109+0000 7fef46000640 1 -- 172.21.15.83:0/3860840142 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef40109580 msgr2=0x7fef401a0cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:59.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.109+0000 7fef46000640 1 --2- 172.21.15.83:0/3860840142 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef40109580 0x7fef401a0cb0 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7fef2c031440 tx=0x7fef2c003d50 comp rx=0 tx=0).stop 2024-04-24T09:02:59.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.109+0000 7fef46000640 1 -- 172.21.15.83:0/3860840142 shutdown_connections 2024-04-24T09:02:59.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.109+0000 7fef46000640 1 --2- 172.21.15.83:0/3860840142 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fef240a3da0 0x7fef240a6260 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.109+0000 7fef46000640 1 --2- 172.21.15.83:0/3860840142 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fef4019a970 0x7fef4019ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.109+0000 7fef46000640 1 --2- 172.21.15.83:0/3860840142 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef40109580 0x7fef401a0cb0 unknown :-1 s=CLOSED pgs=1822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.109+0000 7fef46000640 1 --2- 172.21.15.83:0/3860840142 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fef401044b0 0x7fef401a0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.109+0000 7fef46000640 1 -- 172.21.15.83:0/3860840142 >> 172.21.15.83:0/3860840142 conn(0x7fef400ffd90 msgr2=0x7fef40101180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:02:59.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.109+0000 7fef46000640 1 -- 172.21.15.83:0/3860840142 shutdown_connections 2024-04-24T09:02:59.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.109+0000 7fef46000640 1 -- 172.21.15.83:0/3860840142 wait complete. 2024-04-24T09:02:59.123 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:02:58 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/3211832790' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:02:59.127 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-24T09:02:59.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:02:58 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/3211832790' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:02:59.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:02:58 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/3211832790' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:02:59.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.339+0000 7f0c31000640 1 -- 172.21.15.83:0/3256978736 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c2c1030c0 msgr2=0x7f0c2c1034a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:59.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.339+0000 7f0c31000640 1 --2- 172.21.15.83:0/3256978736 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c2c1030c0 0x7f0c2c1034a0 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c009900 tx=0x7f0c1c02a480 comp rx=0 tx=0).stop 2024-04-24T09:02:59.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.339+0000 7f0c31000640 1 -- 172.21.15.83:0/3256978736 shutdown_connections 2024-04-24T09:02:59.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.339+0000 7f0c31000640 1 --2- 172.21.15.83:0/3256978736 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0c2c10c490 0x7f0c2c10e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.339+0000 7f0c31000640 1 --2- 172.21.15.83:0/3256978736 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c2c1039e0 0x7f0c2c10bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.339+0000 7f0c31000640 1 --2- 172.21.15.83:0/3256978736 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c2c1030c0 0x7f0c2c1034a0 unknown :-1 s=CLOSED pgs=1827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.339+0000 7f0c31000640 1 -- 172.21.15.83:0/3256978736 >> 172.21.15.83:0/3256978736 conn(0x7f0c2c0fc880 msgr2=0x7f0c2c0feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:02:59.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.339+0000 7f0c31000640 1 -- 172.21.15.83:0/3256978736 shutdown_connections 2024-04-24T09:02:59.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.339+0000 7f0c31000640 1 -- 172.21.15.83:0/3256978736 wait complete. 2024-04-24T09:02:59.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.340+0000 7f0c31000640 1 Processor -- start 2024-04-24T09:02:59.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.340+0000 7f0c31000640 1 -- start start 2024-04-24T09:02:59.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.341+0000 7f0c31000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0c2c1030c0 0x7f0c2c1a03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:59.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.341+0000 7f0c31000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c2c1039e0 0x7f0c2c1a0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:59.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.341+0000 7f0c31000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c2c10c490 0x7f0c2c19a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:59.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.341+0000 7f0c31000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0c2c112140 con 0x7f0c2c1039e0 2024-04-24T09:02:59.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.341+0000 7f0c31000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0c2c111fc0 con 0x7f0c2c10c490 2024-04-24T09:02:59.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.341+0000 7f0c31000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0c2c1122c0 con 0x7f0c2c1030c0 2024-04-24T09:02:59.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.341+0000 7f0c23e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c2c1039e0 0x7f0c2c1a0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:59.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.341+0000 7f0c23e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c2c1039e0 0x7f0c2c1a0930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45398/0 (socket says 172.21.15.83:45398) 2024-04-24T09:02:59.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.341+0000 7f0c23e00640 1 -- 172.21.15.83:0/1068137447 learned_addr learned my addr 172.21.15.83:0/1068137447 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:02:59.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.341+0000 7f0c2be00640 1 --2- 172.21.15.83:0/1068137447 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c2c10c490 0x7f0c2c19a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:59.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.342+0000 7f0c2b400640 1 --2- 172.21.15.83:0/1068137447 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0c2c1030c0 0x7f0c2c1a03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:59.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.342+0000 7f0c23e00640 1 -- 172.21.15.83:0/1068137447 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0c2c1030c0 msgr2=0x7f0c2c1a03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:59.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.342+0000 7f0c23e00640 1 --2- 172.21.15.83:0/1068137447 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0c2c1030c0 0x7f0c2c1a03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.342+0000 7f0c23e00640 1 -- 172.21.15.83:0/1068137447 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c2c10c490 msgr2=0x7f0c2c19a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:59.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.342+0000 7f0c23e00640 1 --2- 172.21.15.83:0/1068137447 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c2c10c490 0x7f0c2c19a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.342+0000 7f0c23e00640 1 -- 172.21.15.83:0/1068137447 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c2c19ad20 con 0x7f0c2c1039e0 2024-04-24T09:02:59.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.342+0000 7f0c23e00640 1 --2- 172.21.15.83:0/1068137447 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c2c1039e0 0x7f0c2c1a0930 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f0c1400c920 tx=0x7f0c1400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:02:59.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.342+0000 7f0c28c00640 1 -- 172.21.15.83:0/1068137447 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c14007c70 con 0x7f0c2c1039e0 2024-04-24T09:02:59.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.342+0000 7f0c31000640 1 -- 172.21.15.83:0/1068137447 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c2c19b010 con 0x7f0c2c1039e0 2024-04-24T09:02:59.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.342+0000 7f0c28c00640 1 -- 172.21.15.83:0/1068137447 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0c14007e10 con 0x7f0c2c1039e0 2024-04-24T09:02:59.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.343+0000 7f0c28c00640 1 -- 172.21.15.83:0/1068137447 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c14018820 con 0x7f0c2c1039e0 2024-04-24T09:02:59.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.343+0000 7f0c31000640 1 -- 172.21.15.83:0/1068137447 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c2c1a72b0 con 0x7f0c2c1039e0 2024-04-24T09:02:59.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.346+0000 7f0c35585740 1 Processor -- start 2024-04-24T09:02:59.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.346+0000 7f0c35585740 1 -- start start 2024-04-24T09:02:59.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.347+0000 7f0c28c00640 1 -- 172.21.15.83:0/1068137447 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0c140189c0 con 0x7f0c2c1039e0 2024-04-24T09:02:59.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.347+0000 7f0c35585740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cc1acd3b40 0x55cc1acd3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:59.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.347+0000 7f0c35585740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cc1acd4460 0x55cc1acde5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:59.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.347+0000 7f0c35585740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cc1acdeb30 0x55cc1ace0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:59.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.347+0000 7f0c35585740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55cc1ac391d0 con 0x55cc1acd3b40 2024-04-24T09:02:59.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.347+0000 7f0c35585740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55cc1abdf450 con 0x55cc1acd4460 2024-04-24T09:02:59.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.347+0000 7f0c35585740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55cc1ac35190 con 0x55cc1acdeb30 2024-04-24T09:02:59.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.347+0000 7f0c2be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cc1acd3b40 0x55cc1acd3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:59.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.347+0000 7f0c2be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cc1acd3b40 0x55cc1acd3f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45408/0 (socket says 172.21.15.83:45408) 2024-04-24T09:02:59.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.347+0000 7f0c2be00640 1 -- 172.21.15.83:0/1540994157 learned_addr learned my addr 172.21.15.83:0/1540994157 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:02:59.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.347+0000 7f0c2b400640 1 --2- 172.21.15.83:0/1540994157 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cc1acd4460 0x55cc1acde5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:59.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.348+0000 7f0c23e00640 1 --2- 172.21.15.83:0/1540994157 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cc1acdeb30 0x55cc1ace0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:59.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.348+0000 7f0c2be00640 1 -- 172.21.15.83:0/1540994157 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cc1acdeb30 msgr2=0x55cc1ace0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:59.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.348+0000 7f0c2be00640 1 --2- 172.21.15.83:0/1540994157 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cc1acdeb30 0x55cc1ace0f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.348+0000 7f0c2be00640 1 -- 172.21.15.83:0/1540994157 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cc1acd4460 msgr2=0x55cc1acde5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:59.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.348+0000 7f0c2be00640 1 --2- 172.21.15.83:0/1540994157 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cc1acd4460 0x55cc1acde5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.348+0000 7f0c2be00640 1 -- 172.21.15.83:0/1540994157 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc1ac9c310 con 0x55cc1acd3b40 2024-04-24T09:02:59.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.348+0000 7f0c2be00640 1 --2- 172.21.15.83:0/1540994157 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cc1acd3b40 0x55cc1acd3f20 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f0c24007dd0 tx=0x7f0c2400e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31cc6a8bff9d26d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:02:59.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.349+0000 7f0c28c00640 1 --2- 172.21.15.83:0/1068137447 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f0c100a3d20 0x7f0c100a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:59.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.349+0000 7f0c20c00640 1 -- 172.21.15.83:0/1540994157 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c24014070 con 0x55cc1acd3b40 2024-04-24T09:02:59.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.349+0000 7f0c20c00640 1 -- 172.21.15.83:0/1540994157 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0c24004d20 con 0x55cc1acd3b40 2024-04-24T09:02:59.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.349+0000 7f0c20c00640 1 -- 172.21.15.83:0/1540994157 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c24010790 con 0x55cc1acd3b40 2024-04-24T09:02:59.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.349+0000 7f0c28c00640 1 -- 172.21.15.83:0/1068137447 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9583+0+0 (secure 0 0 0) 0x7f0c140d0e20 con 0x7f0c2c1039e0 2024-04-24T09:02:59.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.349+0000 7f0c2b400640 1 --2- 172.21.15.83:0/1068137447 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f0c100a3d20 0x7f0c100a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:59.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.350+0000 7f0c35585740 1 -- 172.21.15.83:0/1540994157 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cc1acd3b40 msgr2=0x55cc1acd3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:59.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.350+0000 7f0c35585740 1 --2- 172.21.15.83:0/1540994157 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cc1acd3b40 0x55cc1acd3f20 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f0c24007dd0 tx=0x7f0c2400e720 comp rx=0 tx=0).stop 2024-04-24T09:02:59.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.350+0000 7f0c35585740 1 -- 172.21.15.83:0/1540994157 shutdown_connections 2024-04-24T09:02:59.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.350+0000 7f0c35585740 1 --2- 172.21.15.83:0/1540994157 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cc1acdeb30 0x55cc1ace0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.350+0000 7f0c35585740 1 --2- 172.21.15.83:0/1540994157 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cc1acd4460 0x55cc1acde5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.350+0000 7f0c35585740 1 --2- 172.21.15.83:0/1540994157 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cc1acd3b40 0x55cc1acd3f20 unknown :-1 s=CLOSED pgs=1829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.350+0000 7f0c35585740 1 -- 172.21.15.83:0/1540994157 >> 172.21.15.83:0/1540994157 conn(0x55cc1accfe60 msgr2=0x55cc1acd0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:02:59.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.350+0000 7f0c2b400640 1 --2- 172.21.15.83:0/1068137447 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f0c100a3d20 0x7f0c100a61e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c006010 tx=0x7f0c1c000bc0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:02:59.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.350+0000 7f0c35585740 1 -- 172.21.15.83:0/1540994157 shutdown_connections 2024-04-24T09:02:59.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.350+0000 7f0c35585740 1 -- 172.21.15.83:0/1540994157 wait complete. 2024-04-24T09:02:59.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.351+0000 7f0c35585740 1 -- start start 2024-04-24T09:02:59.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.351+0000 7f0c35585740 1 Processor -- start 2024-04-24T09:02:59.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.352+0000 7f0c35585740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cc1acd3b40 0x55cc1acdb910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:59.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.352+0000 7f0c35585740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cc1acd4460 0x55cc1acdbe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:59.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.352+0000 7f0c35585740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cc1acdeb30 0x55cc1acdc390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:59.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.352+0000 7f0c35585740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55cc1ace19c0 con 0x55cc1acdeb30 2024-04-24T09:02:59.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.352+0000 7f0c35585740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55cc1abf2810 con 0x55cc1acd3b40 2024-04-24T09:02:59.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.352+0000 7f0c35585740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55cc1ace1b40 con 0x55cc1acd4460 2024-04-24T09:02:59.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.352+0000 7f0c35585740 10 client.0 _ll_register_callbacks cb 0x55cc1abc0140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:02:59.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.352+0000 7f0c2be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cc1acdeb30 0x55cc1acdc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:59.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.352+0000 7f0c2be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cc1acdeb30 0x55cc1acdc390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45412/0 (socket says 172.21.15.83:45412) 2024-04-24T09:02:59.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.352+0000 7f0c2be00640 1 -- 172.21.15.83:0/4127837469 learned_addr learned my addr 172.21.15.83:0/4127837469 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:02:59.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.352+0000 7f0c2b400640 1 --2- 172.21.15.83:0/4127837469 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cc1acd3b40 0x55cc1acdb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:59.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.353+0000 7f0c23e00640 1 --2- 172.21.15.83:0/4127837469 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cc1acd4460 0x55cc1acdbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:59.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.353+0000 7f0c2be00640 1 -- 172.21.15.83:0/4127837469 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cc1acd4460 msgr2=0x55cc1acdbe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:59.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.353+0000 7f0c2be00640 1 --2- 172.21.15.83:0/4127837469 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cc1acd4460 0x55cc1acdbe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.353+0000 7f0c2be00640 1 -- 172.21.15.83:0/4127837469 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cc1acd3b40 msgr2=0x55cc1acdb910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:59.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.353+0000 7f0c2be00640 1 --2- 172.21.15.83:0/4127837469 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cc1acd3b40 0x55cc1acdb910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.353+0000 7f0c2be00640 1 -- 172.21.15.83:0/4127837469 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc1ac033f0 con 0x55cc1acdeb30 2024-04-24T09:02:59.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.353+0000 7f0c2b400640 1 --2- 172.21.15.83:0/4127837469 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cc1acd3b40 0x55cc1acdb910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:02:59.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.353+0000 7f0c2be00640 1 --2- 172.21.15.83:0/4127837469 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cc1acdeb30 0x55cc1acdc390 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f0c24005b40 tx=0x7f0c24004ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0a4b22456c6ee6f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:02:59.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.353+0000 7f0c19600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:02:59.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.354+0000 7f0c19600640 1 -- 172.21.15.83:0/4127837469 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c24027060 con 0x55cc1acdeb30 2024-04-24T09:02:59.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.354+0000 7f0c19600640 1 -- 172.21.15.83:0/4127837469 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9583+0+0 (secure 0 0 0) 0x7f0c24010510 con 0x55cc1acdeb30 2024-04-24T09:02:59.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.354+0000 7f0c35585740 1 -- 172.21.15.83:0/4127837469 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c24004d20 con 0x55cc1acdeb30 2024-04-24T09:02:59.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.354+0000 7f0c19600640 1 -- 172.21.15.83:0/4127837469 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f0c2402e050 con 0x55cc1acdeb30 2024-04-24T09:02:59.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.354+0000 7f0c35585740 10 client.19650 fetch_fsmap learned FSMap version 41 2024-04-24T09:02:59.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.354+0000 7f0c35585740 1 -- 172.21.15.83:0/4127837469 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55cc1a94b3f0 con 0x55cc1acdeb30 2024-04-24T09:02:59.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.355+0000 7f0c19600640 1 -- 172.21.15.83:0/4127837469 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0c24004d20 con 0x55cc1acdeb30 2024-04-24T09:02:59.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.355+0000 7f0c19600640 1 -- 172.21.15.83:0/4127837469 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7f0c240186e0 con 0x55cc1acdeb30 2024-04-24T09:02:59.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.355+0000 7f0c35585740 10 client.19650 fetch_fsmap finished waiting for FSMap version 41 2024-04-24T09:02:59.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.355+0000 7f0c35585740 10 client.19650 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-24T09:02:59.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.355+0000 7f0c35585740 20 client.19650 populate_metadata read hostname 'smithi083' 2024-04-24T09:02:59.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.355+0000 7f0c35585740 1 --2- 172.21.15.83:0/4127837469 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55cc1aceec90 0x55cc1acf1080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:59.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.355+0000 7f0c35585740 4 client.19650 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:02:59.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.355+0000 7f0c35585740 1 -- 172.21.15.83:0/4127837469 --> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cc1ac46bf0 con 0x55cc1aceec90 2024-04-24T09:02:59.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.356+0000 7f0c2b400640 1 --2- 172.21.15.83:0/4127837469 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55cc1aceec90 0x55cc1acf1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:59.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.357+0000 7f0c2b400640 1 --2- 172.21.15.83:0/4127837469 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55cc1aceec90 0x55cc1acf1080 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c63b62d1b0bf20e2 server_cookie=61180e29e0350055 in_seq=0 out_seq=0 2024-04-24T09:02:59.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.357+0000 7f0c19600640 10 client.19650 ms_handle_connect on v2:172.21.15.138:6832/1157967658 2024-04-24T09:02:59.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.360+0000 7f0c19600640 1 -- 172.21.15.83:0/4127837469 <== mds.3 v2:172.21.15.138:6832/1157967658 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cc1ac46bf0 con 0x55cc1aceec90 2024-04-24T09:02:59.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.360+0000 7f0c19600640 10 client.19650 handle_command_reply: tid=0 2024-04-24T09:02:59.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.360+0000 7f0c35585740 1 client.19650 shutdown 2024-04-24T09:02:59.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.360+0000 7f0c35585740 1 -- 172.21.15.83:0/4127837469 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cc1acdeb30 msgr2=0x55cc1acdc390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:59.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.360+0000 7f0c35585740 1 --2- 172.21.15.83:0/4127837469 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cc1acdeb30 0x55cc1acdc390 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f0c24005b40 tx=0x7f0c24004ef0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.360+0000 7f0c35585740 1 -- 172.21.15.83:0/4127837469 shutdown_connections 2024-04-24T09:02:59.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.360+0000 7f0c35585740 1 --2- 172.21.15.83:0/4127837469 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cc1acdeb30 0x55cc1acdc390 unknown :-1 s=CLOSED pgs=1830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.360+0000 7f0c35585740 1 --2- 172.21.15.83:0/4127837469 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55cc1aceec90 0x55cc1acf1080 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.360+0000 7f0c35585740 1 --2- 172.21.15.83:0/4127837469 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cc1acd4460 0x55cc1acdbe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.360+0000 7f0c35585740 1 --2- 172.21.15.83:0/4127837469 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cc1acd3b40 0x55cc1acdb910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.360+0000 7f0c19600640 0 client.19650 ms_handle_reset on v2:172.21.15.138:6832/1157967658 2024-04-24T09:02:59.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.360+0000 7f0c35585740 1 -- 172.21.15.83:0/4127837469 >> 172.21.15.83:0/4127837469 conn(0x55cc1accfe60 msgr2=0x55cc1acd5b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:02:59.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.361+0000 7f0c35585740 1 -- 172.21.15.83:0/4127837469 shutdown_connections 2024-04-24T09:02:59.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.361+0000 7f0c35585740 1 -- 172.21.15.83:0/4127837469 wait complete. 2024-04-24T09:02:59.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.361+0000 7f0c35585740 20 client.19650 trim_cache size 0 max 16384 2024-04-24T09:02:59.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.379+0000 7f0c35585740 1 Processor -- start 2024-04-24T09:02:59.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.379+0000 7f0c35585740 1 -- start start 2024-04-24T09:02:59.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.379+0000 7f0c35585740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cc1acd3b40 0x55cc1acea880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:59.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.379+0000 7f0c35585740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cc1acd4460 0x55cc1acdae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:59.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.379+0000 7f0c35585740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cc1acdeb30 0x55cc1acf6650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:59.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.379+0000 7f0c35585740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0c2c1b1530 con 0x55cc1acd4460 2024-04-24T09:02:59.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.379+0000 7f0c35585740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0c2c1b3190 con 0x55cc1acd3b40 2024-04-24T09:02:59.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.379+0000 7f0c35585740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0c2c1b1c00 con 0x55cc1acdeb30 2024-04-24T09:02:59.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.380+0000 7f0c23e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cc1acd4460 0x55cc1acdae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:59.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.380+0000 7f0c23e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cc1acd4460 0x55cc1acdae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45424/0 (socket says 172.21.15.83:45424) 2024-04-24T09:02:59.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.380+0000 7f0c23e00640 1 -- 172.21.15.83:0/1476680417 learned_addr learned my addr 172.21.15.83:0/1476680417 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:02:59.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.380+0000 7f0c23e00640 1 -- 172.21.15.83:0/1476680417 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cc1acdeb30 msgr2=0x55cc1acf6650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:59.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.380+0000 7f0c2be00640 1 --2- 172.21.15.83:0/1476680417 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cc1acdeb30 0x55cc1acf6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:59.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.380+0000 7f0c2b400640 1 --2- 172.21.15.83:0/1476680417 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cc1acd3b40 0x55cc1acea880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:59.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.380+0000 7f0c23e00640 1 --2- 172.21.15.83:0/1476680417 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cc1acdeb30 0x55cc1acf6650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.380+0000 7f0c23e00640 1 -- 172.21.15.83:0/1476680417 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cc1acd3b40 msgr2=0x55cc1acea880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:59.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.380+0000 7f0c23e00640 1 --2- 172.21.15.83:0/1476680417 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cc1acd3b40 0x55cc1acea880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.380+0000 7f0c23e00640 1 -- 172.21.15.83:0/1476680417 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc1aa157d0 con 0x55cc1acd4460 2024-04-24T09:02:59.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.380+0000 7f0c23e00640 1 --2- 172.21.15.83:0/1476680417 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cc1acd4460 0x55cc1acdae70 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f0c14099370 tx=0x7f0c1400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce7df9f32754fd8a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:02:59.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.381+0000 7f0c1be00640 1 -- 172.21.15.83:0/1476680417 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c14018cb0 con 0x55cc1acd4460 2024-04-24T09:02:59.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.381+0000 7f0c1be00640 1 -- 172.21.15.83:0/1476680417 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0c1400fb00 con 0x55cc1acd4460 2024-04-24T09:02:59.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.381+0000 7f0c1be00640 1 -- 172.21.15.83:0/1476680417 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c140a5610 con 0x55cc1acd4460 2024-04-24T09:02:59.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.381+0000 7f0c35585740 1 -- 172.21.15.83:0/1476680417 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cc1acd4460 msgr2=0x55cc1acdae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:59.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.381+0000 7f0c35585740 1 --2- 172.21.15.83:0/1476680417 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cc1acd4460 0x55cc1acdae70 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f0c14099370 tx=0x7f0c1400a620 comp rx=0 tx=0).stop 2024-04-24T09:02:59.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.382+0000 7f0c35585740 1 -- 172.21.15.83:0/1476680417 shutdown_connections 2024-04-24T09:02:59.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.382+0000 7f0c35585740 1 --2- 172.21.15.83:0/1476680417 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cc1acdeb30 0x55cc1acf6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.382+0000 7f0c35585740 1 --2- 172.21.15.83:0/1476680417 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cc1acd4460 0x55cc1acdae70 unknown :-1 s=CLOSED pgs=1831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.382+0000 7f0c35585740 1 --2- 172.21.15.83:0/1476680417 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cc1acd3b40 0x55cc1acea880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.382+0000 7f0c35585740 1 -- 172.21.15.83:0/1476680417 >> 172.21.15.83:0/1476680417 conn(0x55cc1accfe60 msgr2=0x55cc1acdc160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:02:59.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.382+0000 7f0c35585740 1 -- 172.21.15.83:0/1476680417 shutdown_connections 2024-04-24T09:02:59.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.382+0000 7f0c35585740 1 -- 172.21.15.83:0/1476680417 wait complete. 2024-04-24T09:02:59.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.382+0000 7f0c35585740 1 -- start start 2024-04-24T09:02:59.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.382+0000 7f0c35585740 1 Processor -- start 2024-04-24T09:02:59.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.383+0000 7f0c35585740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cc1acd3b40 0x55cc1acdae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:59.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.383+0000 7f0c35585740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cc1acd4460 0x55cc1acdb3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:59.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.383+0000 7f0c35585740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cc1acdeb30 0x55cc1ace5d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:59.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.383+0000 7f0c35585740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0c2c1b2a60 con 0x55cc1acdeb30 2024-04-24T09:02:59.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.383+0000 7f0c35585740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0c2c1b2330 con 0x55cc1acd3b40 2024-04-24T09:02:59.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.383+0000 7f0c35585740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0c2c1af530 con 0x55cc1acd4460 2024-04-24T09:02:59.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.383+0000 7f0c35585740 10 client.0 _ll_register_callbacks cb 0x7f0c24007c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:02:59.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.383+0000 7f0c2be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cc1acdeb30 0x55cc1ace5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:59.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.383+0000 7f0c2be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cc1acdeb30 0x55cc1ace5d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45432/0 (socket says 172.21.15.83:45432) 2024-04-24T09:02:59.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.383+0000 7f0c2be00640 1 -- 172.21.15.83:0/1621716050 learned_addr learned my addr 172.21.15.83:0/1621716050 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:02:59.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.383+0000 7f0c2b400640 1 --2- 172.21.15.83:0/1621716050 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cc1acd3b40 0x55cc1acdae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:59.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.383+0000 7f0c23e00640 1 --2- 172.21.15.83:0/1621716050 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cc1acd4460 0x55cc1acdb3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:59.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.384+0000 7f0c2be00640 1 -- 172.21.15.83:0/1621716050 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cc1acd4460 msgr2=0x55cc1acdb3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:59.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.384+0000 7f0c2be00640 1 --2- 172.21.15.83:0/1621716050 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cc1acd4460 0x55cc1acdb3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.384+0000 7f0c2be00640 1 -- 172.21.15.83:0/1621716050 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cc1acd3b40 msgr2=0x55cc1acdae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:59.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.384+0000 7f0c2be00640 1 --2- 172.21.15.83:0/1621716050 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cc1acd3b40 0x55cc1acdae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.384+0000 7f0c2be00640 1 -- 172.21.15.83:0/1621716050 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc1ace62d0 con 0x55cc1acdeb30 2024-04-24T09:02:59.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.384+0000 7f0c2b400640 1 --2- 172.21.15.83:0/1621716050 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cc1acd3b40 0x55cc1acdae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:02:59.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.384+0000 7f0c2be00640 1 --2- 172.21.15.83:0/1621716050 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cc1acdeb30 0x55cc1ace5d90 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f0c240053e0 tx=0x7f0c24005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c202293bafbcb531 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:02:59.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.384+0000 7f0c19600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:02:59.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.384+0000 7f0c19600640 1 -- 172.21.15.83:0/1621716050 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c24027040 con 0x55cc1acdeb30 2024-04-24T09:02:59.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.384+0000 7f0c19600640 1 -- 172.21.15.83:0/1621716050 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9583+0+0 (secure 0 0 0) 0x7f0c24003b80 con 0x55cc1acdeb30 2024-04-24T09:02:59.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.384+0000 7f0c35585740 1 -- 172.21.15.83:0/1621716050 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55cc1ac40a60 con 0x55cc1acdeb30 2024-04-24T09:02:59.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.384+0000 7f0c19600640 1 -- 172.21.15.83:0/1621716050 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f0c24014070 con 0x55cc1acdeb30 2024-04-24T09:02:59.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.384+0000 7f0c35585740 10 client.19656 fetch_fsmap learned FSMap version 41 2024-04-24T09:02:59.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.384+0000 7f0c35585740 1 -- 172.21.15.83:0/1621716050 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55cc1ace75d0 con 0x55cc1acdeb30 2024-04-24T09:02:59.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.385+0000 7f0c19600640 1 -- 172.21.15.83:0/1621716050 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55cc1ac40a60 con 0x55cc1acdeb30 2024-04-24T09:02:59.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.385+0000 7f0c19600640 1 -- 172.21.15.83:0/1621716050 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7f0c24022690 con 0x55cc1acdeb30 2024-04-24T09:02:59.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.385+0000 7f0c35585740 10 client.19656 fetch_fsmap finished waiting for FSMap version 41 2024-04-24T09:02:59.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.385+0000 7f0c35585740 10 client.19656 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-24T09:02:59.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.385+0000 7f0c35585740 20 client.19656 populate_metadata read hostname 'smithi083' 2024-04-24T09:02:59.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.385+0000 7f0c35585740 1 --2- 172.21.15.83:0/1621716050 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55cc1ad552a0 0x55cc1ad576c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:59.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.385+0000 7f0c35585740 4 client.19656 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:02:59.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.385+0000 7f0c35585740 1 -- 172.21.15.83:0/1621716050 --> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cc1abbc4d0 con 0x55cc1ad552a0 2024-04-24T09:02:59.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.386+0000 7f0c2b400640 1 --2- 172.21.15.83:0/1621716050 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55cc1ad552a0 0x55cc1ad576c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:59.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.387+0000 7f0c2b400640 1 --2- 172.21.15.83:0/1621716050 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55cc1ad552a0 0x55cc1ad576c0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ac7a91d89412b031 server_cookie=21baf16de995780c in_seq=0 out_seq=0 2024-04-24T09:02:59.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.387+0000 7f0c19600640 10 client.19656 ms_handle_connect on v2:172.21.15.138:6832/1157967658 2024-04-24T09:02:59.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.388+0000 7f0c19600640 1 -- 172.21.15.83:0/1621716050 <== mds.3 v2:172.21.15.138:6832/1157967658 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cc1abbc4d0 con 0x55cc1ad552a0 2024-04-24T09:02:59.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.388+0000 7f0c19600640 10 client.19656 handle_command_reply: tid=0 2024-04-24T09:02:59.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.388+0000 7f0c35585740 1 client.19656 shutdown 2024-04-24T09:02:59.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.389+0000 7f0c35585740 1 -- 172.21.15.83:0/1621716050 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cc1acdeb30 msgr2=0x55cc1ace5d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:59.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.389+0000 7f0c35585740 1 --2- 172.21.15.83:0/1621716050 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cc1acdeb30 0x55cc1ace5d90 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f0c240053e0 tx=0x7f0c24005a20 comp rx=0 tx=0).stop 2024-04-24T09:02:59.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.389+0000 7f0c35585740 1 -- 172.21.15.83:0/1621716050 shutdown_connections 2024-04-24T09:02:59.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.389+0000 7f0c35585740 1 --2- 172.21.15.83:0/1621716050 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cc1acdeb30 0x55cc1ace5d90 unknown :-1 s=CLOSED pgs=1832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.389+0000 7f0c35585740 1 --2- 172.21.15.83:0/1621716050 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55cc1ad552a0 0x55cc1ad576c0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.389+0000 7f0c35585740 1 --2- 172.21.15.83:0/1621716050 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cc1acd4460 0x55cc1acdb3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.389+0000 7f0c35585740 1 --2- 172.21.15.83:0/1621716050 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cc1acd3b40 0x55cc1acdae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.389+0000 7f0c35585740 1 -- 172.21.15.83:0/1621716050 >> 172.21.15.83:0/1621716050 conn(0x55cc1accfe60 msgr2=0x55cc1acd5130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:02:59.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.389+0000 7f0c19600640 0 client.19656 ms_handle_reset on v2:172.21.15.138:6832/1157967658 2024-04-24T09:02:59.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.389+0000 7f0c35585740 1 -- 172.21.15.83:0/1621716050 shutdown_connections 2024-04-24T09:02:59.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.389+0000 7f0c35585740 1 -- 172.21.15.83:0/1621716050 wait complete. 2024-04-24T09:02:59.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.389+0000 7f0c35585740 20 client.19656 trim_cache size 0 max 16384 2024-04-24T09:02:59.390 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:02:59.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.390+0000 7f0c31000640 1 -- 172.21.15.83:0/1068137447 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f0c100a3d20 msgr2=0x7f0c100a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:59.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.390+0000 7f0c31000640 1 --2- 172.21.15.83:0/1068137447 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f0c100a3d20 0x7f0c100a61e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c006010 tx=0x7f0c1c000bc0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.390+0000 7f0c31000640 1 -- 172.21.15.83:0/1068137447 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c2c1039e0 msgr2=0x7f0c2c1a0930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:59.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.390+0000 7f0c31000640 1 --2- 172.21.15.83:0/1068137447 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c2c1039e0 0x7f0c2c1a0930 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f0c1400c920 tx=0x7f0c1400cde0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.390+0000 7f0c31000640 1 -- 172.21.15.83:0/1068137447 shutdown_connections 2024-04-24T09:02:59.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.390+0000 7f0c31000640 1 --2- 172.21.15.83:0/1068137447 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f0c100a3d20 0x7f0c100a61e0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.390+0000 7f0c31000640 1 --2- 172.21.15.83:0/1068137447 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c2c10c490 0x7f0c2c19a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.390+0000 7f0c31000640 1 --2- 172.21.15.83:0/1068137447 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c2c1039e0 0x7f0c2c1a0930 unknown :-1 s=CLOSED pgs=1828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.390+0000 7f0c31000640 1 --2- 172.21.15.83:0/1068137447 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0c2c1030c0 0x7f0c2c1a03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.390+0000 7f0c31000640 1 -- 172.21.15.83:0/1068137447 >> 172.21.15.83:0/1068137447 conn(0x7f0c2c0fc880 msgr2=0x7f0c2c101690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:02:59.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.390+0000 7f0c31000640 1 -- 172.21.15.83:0/1068137447 shutdown_connections 2024-04-24T09:02:59.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.390+0000 7f0c31000640 1 -- 172.21.15.83:0/1068137447 wait complete. 2024-04-24T09:02:59.406 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T09:02:59.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.612+0000 7f0c0ee00640 1 -- 172.21.15.83:0/1595374148 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c08103a20 msgr2=0x7f0c0810bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:59.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.612+0000 7f0c0ee00640 1 --2- 172.21.15.83:0/1595374148 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c08103a20 0x7f0c0810bf00 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f0bf4009900 tx=0x7f0bf402a480 comp rx=0 tx=0).stop 2024-04-24T09:02:59.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.612+0000 7f0c0ee00640 1 -- 172.21.15.83:0/1595374148 shutdown_connections 2024-04-24T09:02:59.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.612+0000 7f0c0ee00640 1 --2- 172.21.15.83:0/1595374148 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0c0810c4d0 0x7f0c0810e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.612+0000 7f0c0ee00640 1 --2- 172.21.15.83:0/1595374148 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c08103a20 0x7f0c0810bf00 unknown :-1 s=CLOSED pgs=1833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.612+0000 7f0c0ee00640 1 --2- 172.21.15.83:0/1595374148 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c08103100 0x7f0c081034e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.612+0000 7f0c0ee00640 1 -- 172.21.15.83:0/1595374148 >> 172.21.15.83:0/1595374148 conn(0x7f0c080fc880 msgr2=0x7f0c080feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:02:59.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.612+0000 7f0c0ee00640 1 -- 172.21.15.83:0/1595374148 shutdown_connections 2024-04-24T09:02:59.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.612+0000 7f0c0ee00640 1 -- 172.21.15.83:0/1595374148 wait complete. 2024-04-24T09:02:59.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.613+0000 7f0c0ee00640 1 Processor -- start 2024-04-24T09:02:59.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.613+0000 7f0c0ee00640 1 -- start start 2024-04-24T09:02:59.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.614+0000 7f0c0ee00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c08103100 0x7f0c081a05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:59.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.614+0000 7f0c0ee00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c08103a20 0x7f0c081a0b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:59.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.614+0000 7f0c0ee00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0c0810c4d0 0x7f0c0819a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:59.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.614+0000 7f0c0ee00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0c081120f0 con 0x7f0c08103a20 2024-04-24T09:02:59.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.614+0000 7f0c0ee00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0c08111f70 con 0x7f0c08103100 2024-04-24T09:02:59.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.614+0000 7f0c0ee00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0c08112270 con 0x7f0c0810c4d0 2024-04-24T09:02:59.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.614+0000 7f0c0d000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c08103a20 0x7f0c081a0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:59.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.614+0000 7f0c0d000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c08103a20 0x7f0c081a0b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45460/0 (socket says 172.21.15.83:45460) 2024-04-24T09:02:59.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.614+0000 7f0c0d000640 1 -- 172.21.15.83:0/1261721155 learned_addr learned my addr 172.21.15.83:0/1261721155 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:02:59.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.614+0000 7f0c0da00640 1 --2- 172.21.15.83:0/1261721155 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c08103100 0x7f0c081a05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:59.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.614+0000 7f0c0e400640 1 --2- 172.21.15.83:0/1261721155 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0c0810c4d0 0x7f0c0819a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:59.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.615+0000 7f0c0d000640 1 -- 172.21.15.83:0/1261721155 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0c0810c4d0 msgr2=0x7f0c0819a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:59.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.615+0000 7f0c0d000640 1 --2- 172.21.15.83:0/1261721155 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0c0810c4d0 0x7f0c0819a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.615+0000 7f0c0d000640 1 -- 172.21.15.83:0/1261721155 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c08103100 msgr2=0x7f0c081a05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:59.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.615+0000 7f0c0d000640 1 --2- 172.21.15.83:0/1261721155 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c08103100 0x7f0c081a05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.615+0000 7f0c0d000640 1 -- 172.21.15.83:0/1261721155 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c0819afb0 con 0x7f0c08103a20 2024-04-24T09:02:59.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.615+0000 7f0c0d000640 1 --2- 172.21.15.83:0/1261721155 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c08103a20 0x7f0c081a0b20 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f0bf4005940 tx=0x7f0bf4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:02:59.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.615+0000 7f0c06000640 1 -- 172.21.15.83:0/1261721155 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf403f030 con 0x7f0c08103a20 2024-04-24T09:02:59.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.615+0000 7f0c06000640 1 -- 172.21.15.83:0/1261721155 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0bf4003aa0 con 0x7f0c08103a20 2024-04-24T09:02:59.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.615+0000 7f0c0ee00640 1 -- 172.21.15.83:0/1261721155 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c0819b240 con 0x7f0c08103a20 2024-04-24T09:02:59.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.615+0000 7f0c06000640 1 -- 172.21.15.83:0/1261721155 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf4035070 con 0x7f0c08103a20 2024-04-24T09:02:59.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.615+0000 7f0c0ee00640 1 -- 172.21.15.83:0/1261721155 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c081a7380 con 0x7f0c08103a20 2024-04-24T09:02:59.617 INFO:tasks.workunit.client.0.smithi083.stdout: 1 66243 0.61 MB/sec execute 453 sec latency 46.590 ms 2024-04-24T09:02:59.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.619+0000 7f0c13264740 1 Processor -- start 2024-04-24T09:02:59.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.619+0000 7f0c13264740 1 -- start start 2024-04-24T09:02:59.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.619+0000 7f0c06000640 1 -- 172.21.15.83:0/1261721155 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0bf4042030 con 0x7f0c08103a20 2024-04-24T09:02:59.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.620+0000 7f0c13264740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bfb4588b40 0x55bfb4588f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:59.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.620+0000 7f0c13264740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bfb4589460 0x55bfb45935f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:59.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.620+0000 7f0c13264740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bfb4593b30 0x55bfb4595f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:59.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.620+0000 7f0c13264740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55bfb44ee1d0 con 0x55bfb4588b40 2024-04-24T09:02:59.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.620+0000 7f0c13264740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55bfb4494450 con 0x55bfb4589460 2024-04-24T09:02:59.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.620+0000 7f0c13264740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55bfb44ea190 con 0x55bfb4593b30 2024-04-24T09:02:59.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.620+0000 7f0c0e400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bfb4588b40 0x55bfb4588f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:59.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.620+0000 7f0c0e400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bfb4588b40 0x55bfb4588f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45470/0 (socket says 172.21.15.83:45470) 2024-04-24T09:02:59.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.620+0000 7f0c0e400640 1 -- 172.21.15.83:0/4043383101 learned_addr learned my addr 172.21.15.83:0/4043383101 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:02:59.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.620+0000 7f0c0da00640 1 --2- 172.21.15.83:0/4043383101 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bfb4589460 0x55bfb45935f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:59.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.620+0000 7f0c0d000640 1 --2- 172.21.15.83:0/4043383101 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bfb4593b30 0x55bfb4595f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:59.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.620+0000 7f0c0e400640 1 -- 172.21.15.83:0/4043383101 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bfb4593b30 msgr2=0x55bfb4595f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:59.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.620+0000 7f0c0e400640 1 --2- 172.21.15.83:0/4043383101 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bfb4593b30 0x55bfb4595f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.620+0000 7f0c0e400640 1 -- 172.21.15.83:0/4043383101 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bfb4589460 msgr2=0x55bfb45935f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:59.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.620+0000 7f0c0e400640 1 --2- 172.21.15.83:0/4043383101 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bfb4589460 0x55bfb45935f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.621+0000 7f0c0e400640 1 -- 172.21.15.83:0/4043383101 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bfb4551310 con 0x55bfb4588b40 2024-04-24T09:02:59.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.621+0000 7f0c0d000640 1 --2- 172.21.15.83:0/4043383101 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bfb4593b30 0x55bfb4595f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:02:59.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.621+0000 7f0c0da00640 1 --2- 172.21.15.83:0/4043383101 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bfb4589460 0x55bfb45935f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:02:59.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.621+0000 7f0c0e400640 1 --2- 172.21.15.83:0/4043383101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bfb4588b40 0x55bfb4588f20 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7f0c00009e60 tx=0x7f0c00010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d7f83f0c760d693 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:02:59.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.621+0000 7f0c06000640 1 --2- 172.21.15.83:0/1261721155 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f0bf00a3d20 0x7f0bf00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:59.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.621+0000 7f0bfaa00640 1 -- 172.21.15.83:0/4043383101 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c00016070 con 0x55bfb4588b40 2024-04-24T09:02:59.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.621+0000 7f0bfaa00640 1 -- 172.21.15.83:0/4043383101 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0c00004d20 con 0x55bfb4588b40 2024-04-24T09:02:59.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.621+0000 7f0bfaa00640 1 -- 172.21.15.83:0/4043383101 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c000126e0 con 0x55bfb4588b40 2024-04-24T09:02:59.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.621+0000 7f0c06000640 1 -- 172.21.15.83:0/1261721155 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9583+0+0 (secure 0 0 0) 0x7f0bf40f14a0 con 0x7f0c08103a20 2024-04-24T09:02:59.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.621+0000 7f0c0da00640 1 --2- 172.21.15.83:0/1261721155 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f0bf00a3d20 0x7f0bf00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:59.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.622+0000 7f0c13264740 1 -- 172.21.15.83:0/4043383101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bfb4588b40 msgr2=0x55bfb4588f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:59.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.622+0000 7f0c13264740 1 --2- 172.21.15.83:0/4043383101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bfb4588b40 0x55bfb4588f20 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7f0c00009e60 tx=0x7f0c00010740 comp rx=0 tx=0).stop 2024-04-24T09:02:59.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.622+0000 7f0c13264740 1 -- 172.21.15.83:0/4043383101 shutdown_connections 2024-04-24T09:02:59.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.622+0000 7f0c13264740 1 --2- 172.21.15.83:0/4043383101 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bfb4593b30 0x55bfb4595f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.622+0000 7f0c13264740 1 --2- 172.21.15.83:0/4043383101 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bfb4589460 0x55bfb45935f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.622+0000 7f0c13264740 1 --2- 172.21.15.83:0/4043383101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bfb4588b40 0x55bfb4588f20 unknown :-1 s=CLOSED pgs=1835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.622+0000 7f0c13264740 1 -- 172.21.15.83:0/4043383101 >> 172.21.15.83:0/4043383101 conn(0x55bfb4584e60 msgr2=0x55bfb4585240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:02:59.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.622+0000 7f0c13264740 1 -- 172.21.15.83:0/4043383101 shutdown_connections 2024-04-24T09:02:59.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.622+0000 7f0c0da00640 1 --2- 172.21.15.83:0/1261721155 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f0bf00a3d20 0x7f0bf00a61e0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x55bfb458a100 tx=0x7f0bfc005f50 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:02:59.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.622+0000 7f0c13264740 1 -- 172.21.15.83:0/4043383101 wait complete. 2024-04-24T09:02:59.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.623+0000 7f0c13264740 1 -- start start 2024-04-24T09:02:59.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.623+0000 7f0c13264740 1 Processor -- start 2024-04-24T09:02:59.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.623+0000 7f0c13264740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bfb4588b40 0x55bfb4593480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:59.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.623+0000 7f0c13264740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bfb4589460 0x55bfb4590770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:59.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.623+0000 7f0c13264740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bfb4593b30 0x55bfb4590cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:59.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.623+0000 7f0c13264740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55bfb45969c0 con 0x55bfb4588b40 2024-04-24T09:02:59.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.623+0000 7f0c13264740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55bfb44a7810 con 0x55bfb4589460 2024-04-24T09:02:59.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.623+0000 7f0c13264740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55bfb4596b40 con 0x55bfb4593b30 2024-04-24T09:02:59.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.624+0000 7f0c13264740 10 client.0 _ll_register_callbacks cb 0x55bfb4475140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:02:59.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.624+0000 7f0c0da00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bfb4588b40 0x55bfb4593480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:59.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.624+0000 7f0c0da00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bfb4588b40 0x55bfb4593480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45480/0 (socket says 172.21.15.83:45480) 2024-04-24T09:02:59.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.624+0000 7f0c0da00640 1 -- 172.21.15.83:0/1598189544 learned_addr learned my addr 172.21.15.83:0/1598189544 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:02:59.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.624+0000 7f0c0e400640 1 --2- 172.21.15.83:0/1598189544 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bfb4593b30 0x55bfb4590cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:59.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.624+0000 7f0c0d000640 1 --2- 172.21.15.83:0/1598189544 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bfb4589460 0x55bfb4590770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:59.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.624+0000 7f0c0da00640 1 -- 172.21.15.83:0/1598189544 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bfb4593b30 msgr2=0x55bfb4590cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:59.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.624+0000 7f0c0da00640 1 --2- 172.21.15.83:0/1598189544 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bfb4593b30 0x55bfb4590cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.624+0000 7f0c0da00640 1 -- 172.21.15.83:0/1598189544 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bfb4589460 msgr2=0x55bfb4590770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:59.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.624+0000 7f0c0da00640 1 --2- 172.21.15.83:0/1598189544 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bfb4589460 0x55bfb4590770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.624+0000 7f0c0da00640 1 -- 172.21.15.83:0/1598189544 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bfb44b83f0 con 0x55bfb4588b40 2024-04-24T09:02:59.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.624+0000 7f0c0e400640 1 --2- 172.21.15.83:0/1598189544 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bfb4593b30 0x55bfb4590cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:02:59.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.624+0000 7f0c0da00640 1 --2- 172.21.15.83:0/1598189544 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bfb4588b40 0x55bfb4593480 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc012700 tx=0x7f0bfc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bee1303e68ef6a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:02:59.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.625+0000 7f0bef400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:02:59.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.625+0000 7f0bef400640 1 -- 172.21.15.83:0/1598189544 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bfc023070 con 0x55bfb4588b40 2024-04-24T09:02:59.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.625+0000 7f0bef400640 1 -- 172.21.15.83:0/1598189544 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9583+0+0 (secure 0 0 0) 0x7f0bfc0118b0 con 0x55bfb4588b40 2024-04-24T09:02:59.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.625+0000 7f0c13264740 1 -- 172.21.15.83:0/1598189544 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c00004d20 con 0x55bfb4588b40 2024-04-24T09:02:59.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.625+0000 7f0bef400640 1 -- 172.21.15.83:0/1598189544 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f0bfc02f050 con 0x55bfb4588b40 2024-04-24T09:02:59.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.625+0000 7f0c13264740 10 client.19668 fetch_fsmap learned FSMap version 41 2024-04-24T09:02:59.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.625+0000 7f0c13264740 1 -- 172.21.15.83:0/1598189544 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55bfb42003f0 con 0x55bfb4588b40 2024-04-24T09:02:59.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.625+0000 7f0bef400640 1 -- 172.21.15.83:0/1598189544 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0c00004d20 con 0x55bfb4588b40 2024-04-24T09:02:59.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.626+0000 7f0bef400640 1 -- 172.21.15.83:0/1598189544 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7f0bfc00f040 con 0x55bfb4588b40 2024-04-24T09:02:59.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.626+0000 7f0c13264740 10 client.19668 fetch_fsmap finished waiting for FSMap version 41 2024-04-24T09:02:59.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.626+0000 7f0c13264740 10 client.19668 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-24T09:02:59.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.626+0000 7f0c13264740 20 client.19668 populate_metadata read hostname 'smithi083' 2024-04-24T09:02:59.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.626+0000 7f0c13264740 1 --2- 172.21.15.83:0/1598189544 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55bfb45a3c10 0x55bfb45a6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:59.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.626+0000 7f0c13264740 4 client.19668 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:02:59.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.626+0000 7f0c13264740 1 -- 172.21.15.83:0/1598189544 --> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bfb44fbbf0 con 0x55bfb45a3c10 2024-04-24T09:02:59.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.627+0000 7f0c0d000640 1 --2- 172.21.15.83:0/1598189544 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55bfb45a3c10 0x55bfb45a6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:59.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.628+0000 7f0c0d000640 1 --2- 172.21.15.83:0/1598189544 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55bfb45a3c10 0x55bfb45a6090 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f6bcd77b74de4b6d server_cookie=31a80114d31ec54e in_seq=0 out_seq=0 2024-04-24T09:02:59.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.628+0000 7f0bef400640 10 client.19668 ms_handle_connect on v2:172.21.15.138:6834/1909219758 2024-04-24T09:02:59.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.630+0000 7f0bef400640 1 -- 172.21.15.83:0/1598189544 <== mds.2 v2:172.21.15.138:6834/1909219758 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bfb44fbbf0 con 0x55bfb45a3c10 2024-04-24T09:02:59.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.630+0000 7f0bef400640 10 client.19668 handle_command_reply: tid=0 2024-04-24T09:02:59.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.630+0000 7f0c13264740 1 client.19668 shutdown 2024-04-24T09:02:59.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.630+0000 7f0c13264740 1 -- 172.21.15.83:0/1598189544 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bfb4588b40 msgr2=0x55bfb4593480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:59.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.630+0000 7f0c13264740 1 --2- 172.21.15.83:0/1598189544 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bfb4588b40 0x55bfb4593480 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc012700 tx=0x7f0bfc004730 comp rx=0 tx=0).stop 2024-04-24T09:02:59.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.630+0000 7f0c13264740 1 -- 172.21.15.83:0/1598189544 shutdown_connections 2024-04-24T09:02:59.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.630+0000 7f0c13264740 1 --2- 172.21.15.83:0/1598189544 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55bfb45a3c10 0x55bfb45a6090 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.630+0000 7f0c13264740 1 --2- 172.21.15.83:0/1598189544 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bfb4593b30 0x55bfb4590cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.630+0000 7f0c13264740 1 --2- 172.21.15.83:0/1598189544 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bfb4589460 0x55bfb4590770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.630+0000 7f0bef400640 0 client.19668 ms_handle_reset on v2:172.21.15.138:6834/1909219758 2024-04-24T09:02:59.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.630+0000 7f0c13264740 1 --2- 172.21.15.83:0/1598189544 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bfb4588b40 0x55bfb4593480 unknown :-1 s=CLOSED pgs=1836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.630+0000 7f0c13264740 1 -- 172.21.15.83:0/1598189544 >> 172.21.15.83:0/1598189544 conn(0x55bfb4584e60 msgr2=0x55bfb458ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:02:59.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.630+0000 7f0c13264740 1 -- 172.21.15.83:0/1598189544 shutdown_connections 2024-04-24T09:02:59.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.630+0000 7f0c13264740 1 -- 172.21.15.83:0/1598189544 wait complete. 2024-04-24T09:02:59.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.631+0000 7f0c13264740 20 client.19668 trim_cache size 0 max 16384 2024-04-24T09:02:59.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.646+0000 7f0c13264740 1 Processor -- start 2024-04-24T09:02:59.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.647+0000 7f0c13264740 1 -- start start 2024-04-24T09:02:59.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.647+0000 7f0c13264740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bfb4588b40 0x55bfb459f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:59.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.647+0000 7f0c13264740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bfb4589460 0x55bfb4592fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:59.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.647+0000 7f0c13264740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bfb4593b30 0x55bfb45ab960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:59.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.647+0000 7f0c13264740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0c081b15b0 con 0x55bfb4588b40 2024-04-24T09:02:59.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.647+0000 7f0c13264740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0c081b3210 con 0x55bfb4593b30 2024-04-24T09:02:59.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.647+0000 7f0c13264740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0c081b1c80 con 0x55bfb4589460 2024-04-24T09:02:59.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.647+0000 7f0c0da00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bfb4588b40 0x55bfb459f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:59.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.647+0000 7f0c0da00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bfb4588b40 0x55bfb459f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45486/0 (socket says 172.21.15.83:45486) 2024-04-24T09:02:59.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.647+0000 7f0c0da00640 1 -- 172.21.15.83:0/2267822774 learned_addr learned my addr 172.21.15.83:0/2267822774 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:02:59.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.648+0000 7f0c0e400640 1 --2- 172.21.15.83:0/2267822774 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bfb4593b30 0x55bfb45ab960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:59.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.648+0000 7f0c0da00640 1 -- 172.21.15.83:0/2267822774 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bfb4589460 msgr2=0x55bfb4592fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:59.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.648+0000 7f0c0da00640 1 --2- 172.21.15.83:0/2267822774 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bfb4589460 0x55bfb4592fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.648+0000 7f0c0da00640 1 -- 172.21.15.83:0/2267822774 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bfb4593b30 msgr2=0x55bfb45ab960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:59.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.648+0000 7f0c0da00640 1 --2- 172.21.15.83:0/2267822774 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bfb4593b30 0x55bfb45ab960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.648+0000 7f0c0da00640 1 -- 172.21.15.83:0/2267822774 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bfb4589290 con 0x55bfb4588b40 2024-04-24T09:02:59.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.648+0000 7f0c0da00640 1 --2- 172.21.15.83:0/2267822774 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bfb4588b40 0x55bfb459f8b0 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc000c00 tx=0x7f0bfc0275f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7484aeb093c09048 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:02:59.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.648+0000 7f0bfa000640 1 -- 172.21.15.83:0/2267822774 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bfc037030 con 0x55bfb4588b40 2024-04-24T09:02:59.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.648+0000 7f0bfa000640 1 -- 172.21.15.83:0/2267822774 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0bfc01f900 con 0x55bfb4588b40 2024-04-24T09:02:59.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.648+0000 7f0bfa000640 1 -- 172.21.15.83:0/2267822774 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bfc023050 con 0x55bfb4588b40 2024-04-24T09:02:59.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.649+0000 7f0c13264740 1 -- 172.21.15.83:0/2267822774 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bfb4588b40 msgr2=0x55bfb459f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:59.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.649+0000 7f0c13264740 1 --2- 172.21.15.83:0/2267822774 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bfb4588b40 0x55bfb459f8b0 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc000c00 tx=0x7f0bfc0275f0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.650+0000 7f0c13264740 1 -- 172.21.15.83:0/2267822774 shutdown_connections 2024-04-24T09:02:59.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.650+0000 7f0c13264740 1 --2- 172.21.15.83:0/2267822774 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bfb4593b30 0x55bfb45ab960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.650+0000 7f0c13264740 1 --2- 172.21.15.83:0/2267822774 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bfb4589460 0x55bfb4592fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.650+0000 7f0c13264740 1 --2- 172.21.15.83:0/2267822774 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bfb4588b40 0x55bfb459f8b0 unknown :-1 s=CLOSED pgs=1837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.650+0000 7f0c13264740 1 -- 172.21.15.83:0/2267822774 >> 172.21.15.83:0/2267822774 conn(0x55bfb4584e60 msgr2=0x55bfb458ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:02:59.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.650+0000 7f0c13264740 1 -- 172.21.15.83:0/2267822774 shutdown_connections 2024-04-24T09:02:59.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.650+0000 7f0c13264740 1 -- 172.21.15.83:0/2267822774 wait complete. 2024-04-24T09:02:59.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.650+0000 7f0c13264740 1 -- start start 2024-04-24T09:02:59.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.650+0000 7f0c13264740 1 Processor -- start 2024-04-24T09:02:59.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.651+0000 7f0c13264740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bfb4588b40 0x55bfb4597950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:59.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.651+0000 7f0c13264740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bfb4589460 0x55bfb4597e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:59.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.651+0000 7f0c13264740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bfb4593b30 0x55bfb45983d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:59.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.651+0000 7f0c13264740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0c081b2ae0 con 0x55bfb4588b40 2024-04-24T09:02:59.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.651+0000 7f0c13264740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0c081b23b0 con 0x55bfb4593b30 2024-04-24T09:02:59.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.651+0000 7f0c13264740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0c081af5b0 con 0x55bfb4589460 2024-04-24T09:02:59.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.651+0000 7f0c13264740 10 client.0 _ll_register_callbacks cb 0x7f0bfc016070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:02:59.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.651+0000 7f0c0da00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bfb4588b40 0x55bfb4597950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:59.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.651+0000 7f0c0da00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bfb4588b40 0x55bfb4597950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45494/0 (socket says 172.21.15.83:45494) 2024-04-24T09:02:59.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.651+0000 7f0c0da00640 1 -- 172.21.15.83:0/543041971 learned_addr learned my addr 172.21.15.83:0/543041971 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:02:59.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.651+0000 7f0c0e400640 1 --2- 172.21.15.83:0/543041971 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bfb4593b30 0x55bfb45983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:59.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.652+0000 7f0c0d000640 1 --2- 172.21.15.83:0/543041971 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bfb4589460 0x55bfb4597e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:59.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.652+0000 7f0c0da00640 1 -- 172.21.15.83:0/543041971 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bfb4589460 msgr2=0x55bfb4597e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:59.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.652+0000 7f0c0da00640 1 --2- 172.21.15.83:0/543041971 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bfb4589460 0x55bfb4597e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.652+0000 7f0c0da00640 1 -- 172.21.15.83:0/543041971 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bfb4593b30 msgr2=0x55bfb45983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:59.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.652+0000 7f0c0da00640 1 --2- 172.21.15.83:0/543041971 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bfb4593b30 0x55bfb45983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.652+0000 7f0c0da00640 1 -- 172.21.15.83:0/543041971 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bfb42ca7d0 con 0x55bfb4588b40 2024-04-24T09:02:59.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.652+0000 7f0c0e400640 1 --2- 172.21.15.83:0/543041971 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bfb4593b30 0x55bfb45983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:02:59.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.652+0000 7f0c0da00640 1 --2- 172.21.15.83:0/543041971 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bfb4588b40 0x55bfb4597950 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc01fe40 tx=0x7f0bfc00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d0244724baf227a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:02:59.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.652+0000 7f0bef400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:02:59.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.652+0000 7f0bef400640 1 -- 172.21.15.83:0/543041971 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bfc037040 con 0x55bfb4588b40 2024-04-24T09:02:59.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.652+0000 7f0bef400640 1 -- 172.21.15.83:0/543041971 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9583+0+0 (secure 0 0 0) 0x7f0bfc01f650 con 0x55bfb4588b40 2024-04-24T09:02:59.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.652+0000 7f0c13264740 1 -- 172.21.15.83:0/543041971 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0bfc01f900 con 0x55bfb4588b40 2024-04-24T09:02:59.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.653+0000 7f0bef400640 1 -- 172.21.15.83:0/543041971 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f0bfc04a050 con 0x55bfb4588b40 2024-04-24T09:02:59.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.653+0000 7f0c13264740 10 client.19674 fetch_fsmap learned FSMap version 41 2024-04-24T09:02:59.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.653+0000 7f0c13264740 1 -- 172.21.15.83:0/543041971 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55bfb460a410 con 0x55bfb4588b40 2024-04-24T09:02:59.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.653+0000 7f0bef400640 1 -- 172.21.15.83:0/543041971 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0bfc01f900 con 0x55bfb4588b40 2024-04-24T09:02:59.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.653+0000 7f0bef400640 1 -- 172.21.15.83:0/543041971 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7f0bfc0276f0 con 0x55bfb4588b40 2024-04-24T09:02:59.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.653+0000 7f0c13264740 10 client.19674 fetch_fsmap finished waiting for FSMap version 41 2024-04-24T09:02:59.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.653+0000 7f0c13264740 10 client.19674 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-24T09:02:59.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.653+0000 7f0c13264740 20 client.19674 populate_metadata read hostname 'smithi083' 2024-04-24T09:02:59.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.653+0000 7f0c13264740 1 --2- 172.21.15.83:0/543041971 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55bfb460ab10 0x55bfb460cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:59.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.653+0000 7f0c13264740 4 client.19674 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:02:59.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.653+0000 7f0c13264740 1 -- 172.21.15.83:0/543041971 --> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bfb44f5a60 con 0x55bfb460ab10 2024-04-24T09:02:59.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.654+0000 7f0c0d000640 1 --2- 172.21.15.83:0/543041971 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55bfb460ab10 0x55bfb460cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:59.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.655+0000 7f0bef400640 10 client.19674 ms_handle_connect on v2:172.21.15.138:6834/1909219758 2024-04-24T09:02:59.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.655+0000 7f0c0d000640 1 --2- 172.21.15.83:0/543041971 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55bfb460ab10 0x55bfb460cf30 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ee84092002d9ca0b server_cookie=a462f5b128574df9 in_seq=0 out_seq=0 2024-04-24T09:02:59.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.656+0000 7f0bef400640 1 -- 172.21.15.83:0/543041971 <== mds.2 v2:172.21.15.138:6834/1909219758 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bfb44f5a60 con 0x55bfb460ab10 2024-04-24T09:02:59.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.656+0000 7f0bef400640 10 client.19674 handle_command_reply: tid=0 2024-04-24T09:02:59.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.656+0000 7f0c13264740 1 client.19674 shutdown 2024-04-24T09:02:59.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.656+0000 7f0c13264740 1 -- 172.21.15.83:0/543041971 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bfb4588b40 msgr2=0x55bfb4597950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:59.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.656+0000 7f0c13264740 1 --2- 172.21.15.83:0/543041971 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bfb4588b40 0x55bfb4597950 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc01fe40 tx=0x7f0bfc00bf10 comp rx=0 tx=0).stop 2024-04-24T09:02:59.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.656+0000 7f0c13264740 1 -- 172.21.15.83:0/543041971 shutdown_connections 2024-04-24T09:02:59.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.656+0000 7f0c13264740 1 --2- 172.21.15.83:0/543041971 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55bfb460ab10 0x55bfb460cf30 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.656+0000 7f0c13264740 1 --2- 172.21.15.83:0/543041971 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bfb4593b30 0x55bfb45983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.656+0000 7f0c13264740 1 --2- 172.21.15.83:0/543041971 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bfb4589460 0x55bfb4597e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.656+0000 7f0c13264740 1 --2- 172.21.15.83:0/543041971 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bfb4588b40 0x55bfb4597950 unknown :-1 s=CLOSED pgs=1838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.656+0000 7f0bef400640 0 client.19674 ms_handle_reset on v2:172.21.15.138:6834/1909219758 2024-04-24T09:02:59.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.656+0000 7f0c13264740 1 -- 172.21.15.83:0/543041971 >> 172.21.15.83:0/543041971 conn(0x55bfb4584e60 msgr2=0x55bfb45836c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:02:59.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.656+0000 7f0c13264740 1 -- 172.21.15.83:0/543041971 shutdown_connections 2024-04-24T09:02:59.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.656+0000 7f0c13264740 1 -- 172.21.15.83:0/543041971 wait complete. 2024-04-24T09:02:59.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.656+0000 7f0c13264740 20 client.19674 trim_cache size 0 max 16384 2024-04-24T09:02:59.658 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:02:59.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.657+0000 7f0c0ee00640 1 -- 172.21.15.83:0/1261721155 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f0bf00a3d20 msgr2=0x7f0bf00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:59.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.657+0000 7f0c0ee00640 1 --2- 172.21.15.83:0/1261721155 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f0bf00a3d20 0x7f0bf00a61e0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x55bfb458a100 tx=0x7f0bfc005f50 comp rx=0 tx=0).stop 2024-04-24T09:02:59.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.657+0000 7f0c0ee00640 1 -- 172.21.15.83:0/1261721155 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c08103a20 msgr2=0x7f0c081a0b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:59.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.657+0000 7f0c0ee00640 1 --2- 172.21.15.83:0/1261721155 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c08103a20 0x7f0c081a0b20 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f0bf4005940 tx=0x7f0bf4004270 comp rx=0 tx=0).stop 2024-04-24T09:02:59.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.657+0000 7f0c0ee00640 1 -- 172.21.15.83:0/1261721155 shutdown_connections 2024-04-24T09:02:59.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.657+0000 7f0c0ee00640 1 --2- 172.21.15.83:0/1261721155 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f0bf00a3d20 0x7f0bf00a61e0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.657+0000 7f0c0ee00640 1 --2- 172.21.15.83:0/1261721155 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0c0810c4d0 0x7f0c0819a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.657+0000 7f0c0ee00640 1 --2- 172.21.15.83:0/1261721155 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c08103a20 0x7f0c081a0b20 unknown :-1 s=CLOSED pgs=1834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.657+0000 7f0c0ee00640 1 --2- 172.21.15.83:0/1261721155 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c08103100 0x7f0c081a05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.657+0000 7f0c0ee00640 1 -- 172.21.15.83:0/1261721155 >> 172.21.15.83:0/1261721155 conn(0x7f0c080fc880 msgr2=0x7f0c08101f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:02:59.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.657+0000 7f0c0ee00640 1 -- 172.21.15.83:0/1261721155 shutdown_connections 2024-04-24T09:02:59.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.657+0000 7f0c0ee00640 1 -- 172.21.15.83:0/1261721155 wait complete. 2024-04-24T09:02:59.674 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T09:02:59.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.872+0000 7f558cc00640 1 -- 172.21.15.83:0/960406973 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5588103060 msgr2=0x7f5588103440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:59.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.872+0000 7f558cc00640 1 --2- 172.21.15.83:0/960406973 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5588103060 0x7f5588103440 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7f5578009ca0 tx=0x7f557802a950 comp rx=0 tx=0).stop 2024-04-24T09:02:59.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.872+0000 7f558cc00640 1 -- 172.21.15.83:0/960406973 shutdown_connections 2024-04-24T09:02:59.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.872+0000 7f558cc00640 1 --2- 172.21.15.83:0/960406973 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f558810c480 0x7f558810e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.872+0000 7f558cc00640 1 --2- 172.21.15.83:0/960406973 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5588103980 0x7f558810be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.872+0000 7f558cc00640 1 --2- 172.21.15.83:0/960406973 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5588103060 0x7f5588103440 unknown :-1 s=CLOSED pgs=1839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.872+0000 7f558cc00640 1 -- 172.21.15.83:0/960406973 >> 172.21.15.83:0/960406973 conn(0x7f55880fc880 msgr2=0x7f55880feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:02:59.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.872+0000 7f558cc00640 1 -- 172.21.15.83:0/960406973 shutdown_connections 2024-04-24T09:02:59.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.872+0000 7f558cc00640 1 -- 172.21.15.83:0/960406973 wait complete. 2024-04-24T09:02:59.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.873+0000 7f558cc00640 1 Processor -- start 2024-04-24T09:02:59.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.873+0000 7f558cc00640 1 -- start start 2024-04-24T09:02:59.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.874+0000 7f558cc00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5588103060 0x7f55881a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:59.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.874+0000 7f558cc00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5588103980 0x7f55881a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:59.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.874+0000 7f558cc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f558810c480 0x7f558819a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:59.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.874+0000 7f558cc00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5588112140 con 0x7f558810c480 2024-04-24T09:02:59.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.874+0000 7f558cc00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5588111fc0 con 0x7f5588103060 2024-04-24T09:02:59.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.874+0000 7f558cc00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f55881122c0 con 0x7f5588103980 2024-04-24T09:02:59.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.874+0000 7f5587e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f558810c480 0x7f558819a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:59.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.874+0000 7f5587e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f558810c480 0x7f558819a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45524/0 (socket says 172.21.15.83:45524) 2024-04-24T09:02:59.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.874+0000 7f5587e00640 1 -- 172.21.15.83:0/4157859461 learned_addr learned my addr 172.21.15.83:0/4157859461 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:02:59.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.874+0000 7f5587400640 1 --2- 172.21.15.83:0/4157859461 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5588103060 0x7f55881a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:59.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.874+0000 7f5586a00640 1 --2- 172.21.15.83:0/4157859461 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5588103980 0x7f55881a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:59.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.874+0000 7f5587e00640 1 -- 172.21.15.83:0/4157859461 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5588103980 msgr2=0x7f55881a0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:59.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.874+0000 7f5587e00640 1 --2- 172.21.15.83:0/4157859461 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5588103980 0x7f55881a0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.874+0000 7f5587e00640 1 -- 172.21.15.83:0/4157859461 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5588103060 msgr2=0x7f55881a0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:59.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.874+0000 7f5587e00640 1 --2- 172.21.15.83:0/4157859461 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5588103060 0x7f55881a0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.874+0000 7f5587e00640 1 -- 172.21.15.83:0/4157859461 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f558819b000 con 0x7f558810c480 2024-04-24T09:02:59.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.874+0000 7f5586a00640 1 --2- 172.21.15.83:0/4157859461 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5588103980 0x7f55881a0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:02:59.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.875+0000 7f5587e00640 1 --2- 172.21.15.83:0/4157859461 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f558810c480 0x7f558819a740 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f558000c970 tx=0x7f558000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:02:59.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.875+0000 7f557fe00640 1 -- 172.21.15.83:0/4157859461 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5580007bb0 con 0x7f558810c480 2024-04-24T09:02:59.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.875+0000 7f557fe00640 1 -- 172.21.15.83:0/4157859461 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5580007d50 con 0x7f558810c480 2024-04-24T09:02:59.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.875+0000 7f558cc00640 1 -- 172.21.15.83:0/4157859461 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f558819b2f0 con 0x7f558810c480 2024-04-24T09:02:59.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.875+0000 7f557fe00640 1 -- 172.21.15.83:0/4157859461 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5580004e10 con 0x7f558810c480 2024-04-24T09:02:59.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.875+0000 7f558cc00640 1 -- 172.21.15.83:0/4157859461 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55881a75b0 con 0x7f558810c480 2024-04-24T09:02:59.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.879+0000 7f557fe00640 1 -- 172.21.15.83:0/4157859461 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5580005020 con 0x7f558810c480 2024-04-24T09:02:59.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.879+0000 7f5590d22740 1 Processor -- start 2024-04-24T09:02:59.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.879+0000 7f5590d22740 1 -- start start 2024-04-24T09:02:59.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.879+0000 7f5590d22740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7d0379b40 0x55b7d0379f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:59.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.879+0000 7f5590d22740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b7d037a460 0x55b7d03845f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:59.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.879+0000 7f5590d22740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b7d0384b30 0x55b7d0386f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:59.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.879+0000 7f5590d22740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55b7d02df1d0 con 0x55b7d0379b40 2024-04-24T09:02:59.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.879+0000 7f5590d22740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b7d0285450 con 0x55b7d037a460 2024-04-24T09:02:59.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.879+0000 7f5590d22740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55b7d02db190 con 0x55b7d0384b30 2024-04-24T09:02:59.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.880+0000 7f5586a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7d0379b40 0x55b7d0379f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:59.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.880+0000 7f5586a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7d0379b40 0x55b7d0379f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45526/0 (socket says 172.21.15.83:45526) 2024-04-24T09:02:59.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.880+0000 7f5586a00640 1 -- 172.21.15.83:0/375284532 learned_addr learned my addr 172.21.15.83:0/375284532 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:02:59.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.880+0000 7f5586a00640 1 --2- 172.21.15.83:0/375284532 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b7d0384b30 0x55b7d0386f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:59.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.880+0000 7f5587400640 1 --2- 172.21.15.83:0/375284532 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b7d037a460 0x55b7d03845f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:59.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.880+0000 7f5586a00640 1 -- 172.21.15.83:0/375284532 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b7d0384b30 msgr2=0x55b7d0386f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:59.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.880+0000 7f5586a00640 1 --2- 172.21.15.83:0/375284532 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b7d0384b30 0x55b7d0386f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.880+0000 7f5586a00640 1 -- 172.21.15.83:0/375284532 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b7d037a460 msgr2=0x55b7d03845f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:59.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.880+0000 7f5586a00640 1 --2- 172.21.15.83:0/375284532 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b7d037a460 0x55b7d03845f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.880+0000 7f5586a00640 1 -- 172.21.15.83:0/375284532 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7d0342310 con 0x55b7d0379b40 2024-04-24T09:02:59.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.880+0000 7f5587400640 1 --2- 172.21.15.83:0/375284532 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b7d037a460 0x55b7d03845f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:02:59.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.880+0000 7f557fe00640 1 --2- 172.21.15.83:0/4157859461 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f556c0a3d20 0x7f556c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:59.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.880+0000 7f5586a00640 1 --2- 172.21.15.83:0/375284532 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7d0379b40 0x55b7d0379f20 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f5574009f00 tx=0x7f5574009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e9244098f19de70 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:02:59.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.880+0000 7f557fe00640 1 -- 172.21.15.83:0/4157859461 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9583+0+0 (secure 0 0 0) 0x7f55800d0990 con 0x7f558810c480 2024-04-24T09:02:59.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.881+0000 7f5587400640 1 --2- 172.21.15.83:0/4157859461 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f556c0a3d20 0x7f556c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:59.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.881+0000 7f5573e00640 1 -- 172.21.15.83:0/375284532 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5574017070 con 0x55b7d0379b40 2024-04-24T09:02:59.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.881+0000 7f5573e00640 1 -- 172.21.15.83:0/375284532 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5574010e40 con 0x55b7d0379b40 2024-04-24T09:02:59.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.881+0000 7f5573e00640 1 -- 172.21.15.83:0/375284532 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5574013600 con 0x55b7d0379b40 2024-04-24T09:02:59.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.882+0000 7f5590d22740 1 -- 172.21.15.83:0/375284532 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7d0379b40 msgr2=0x55b7d0379f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:59.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.882+0000 7f5587400640 1 --2- 172.21.15.83:0/4157859461 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f556c0a3d20 0x7f556c0a61e0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x55b7d037b100 tx=0x7f5578004a40 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:02:59.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.882+0000 7f5590d22740 1 --2- 172.21.15.83:0/375284532 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7d0379b40 0x55b7d0379f20 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f5574009f00 tx=0x7f5574009680 comp rx=0 tx=0).stop 2024-04-24T09:02:59.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.882+0000 7f5590d22740 1 -- 172.21.15.83:0/375284532 shutdown_connections 2024-04-24T09:02:59.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.882+0000 7f5590d22740 1 --2- 172.21.15.83:0/375284532 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b7d0384b30 0x55b7d0386f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.882+0000 7f5590d22740 1 --2- 172.21.15.83:0/375284532 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b7d037a460 0x55b7d03845f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.882+0000 7f5590d22740 1 --2- 172.21.15.83:0/375284532 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7d0379b40 0x55b7d0379f20 unknown :-1 s=CLOSED pgs=1841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.882+0000 7f5590d22740 1 -- 172.21.15.83:0/375284532 >> 172.21.15.83:0/375284532 conn(0x55b7d0375e60 msgr2=0x55b7d0376240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:02:59.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.882+0000 7f5590d22740 1 -- 172.21.15.83:0/375284532 shutdown_connections 2024-04-24T09:02:59.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.882+0000 7f5590d22740 1 -- 172.21.15.83:0/375284532 wait complete. 2024-04-24T09:02:59.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.882+0000 7f5590d22740 1 -- start start 2024-04-24T09:02:59.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.882+0000 7f5590d22740 1 Processor -- start 2024-04-24T09:02:59.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.883+0000 7f5590d22740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b7d0379b40 0x55b7d03845f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:59.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.883+0000 7f5590d22740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7d037a460 0x55b7d03818e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:59.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.883+0000 7f5590d22740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b7d0384b30 0x55b7d0382020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:59.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.883+0000 7f5590d22740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55b7d03879c0 con 0x55b7d037a460 2024-04-24T09:02:59.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.883+0000 7f5590d22740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b7d0298810 con 0x55b7d0379b40 2024-04-24T09:02:59.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.883+0000 7f5590d22740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55b7d0387b40 con 0x55b7d0384b30 2024-04-24T09:02:59.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.883+0000 7f5590d22740 10 client.0 _ll_register_callbacks cb 0x55b7d0266140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:02:59.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.883+0000 7f5586a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7d037a460 0x55b7d03818e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:59.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.883+0000 7f5586a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7d037a460 0x55b7d03818e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45532/0 (socket says 172.21.15.83:45532) 2024-04-24T09:02:59.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.883+0000 7f5586a00640 1 -- 172.21.15.83:0/4211359182 learned_addr learned my addr 172.21.15.83:0/4211359182 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:02:59.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.884+0000 7f5587e00640 1 --2- 172.21.15.83:0/4211359182 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b7d0384b30 0x55b7d0382020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:59.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.884+0000 7f5587400640 1 --2- 172.21.15.83:0/4211359182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b7d0379b40 0x55b7d03845f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:59.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.884+0000 7f5586a00640 1 -- 172.21.15.83:0/4211359182 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b7d0384b30 msgr2=0x55b7d0382020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:59.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.884+0000 7f5586a00640 1 --2- 172.21.15.83:0/4211359182 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b7d0384b30 0x55b7d0382020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.884+0000 7f5586a00640 1 -- 172.21.15.83:0/4211359182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b7d0379b40 msgr2=0x55b7d03845f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:59.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.884+0000 7f5586a00640 1 --2- 172.21.15.83:0/4211359182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b7d0379b40 0x55b7d03845f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.884+0000 7f5586a00640 1 -- 172.21.15.83:0/4211359182 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7d02a93f0 con 0x55b7d037a460 2024-04-24T09:02:59.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.884+0000 7f5586a00640 1 --2- 172.21.15.83:0/4211359182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7d037a460 0x55b7d03818e0 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7f5574009ba0 tx=0x7f55740048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2448707f8ece13 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:02:59.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.884+0000 7f5570c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:02:59.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.885+0000 7f5570c00640 1 -- 172.21.15.83:0/4211359182 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5574017040 con 0x55b7d037a460 2024-04-24T09:02:59.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.885+0000 7f5570c00640 1 -- 172.21.15.83:0/4211359182 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9583+0+0 (secure 0 0 0) 0x7f5574005230 con 0x55b7d037a460 2024-04-24T09:02:59.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.885+0000 7f5590d22740 1 -- 172.21.15.83:0/4211359182 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b7d02ecbf0 con 0x55b7d037a460 2024-04-24T09:02:59.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.885+0000 7f5570c00640 1 -- 172.21.15.83:0/4211359182 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f5574005430 con 0x55b7d037a460 2024-04-24T09:02:59.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.885+0000 7f5590d22740 10 client.19686 fetch_fsmap learned FSMap version 41 2024-04-24T09:02:59.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.885+0000 7f5590d22740 1 -- 172.21.15.83:0/4211359182 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55b7cfff13f0 con 0x55b7d037a460 2024-04-24T09:02:59.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.885+0000 7f5570c00640 1 -- 172.21.15.83:0/4211359182 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55b7d02ecbf0 con 0x55b7d037a460 2024-04-24T09:02:59.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.885+0000 7f5570c00640 1 -- 172.21.15.83:0/4211359182 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7f557401b400 con 0x55b7d037a460 2024-04-24T09:02:59.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.885+0000 7f5590d22740 10 client.19686 fetch_fsmap finished waiting for FSMap version 41 2024-04-24T09:02:59.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.886+0000 7f5590d22740 10 client.19686 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-24T09:02:59.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.886+0000 7f5590d22740 20 client.19686 populate_metadata read hostname 'smithi083' 2024-04-24T09:02:59.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.886+0000 7f5590d22740 1 --2- 172.21.15.83:0/4211359182 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x55b7d0394ca0 0x55b7d0397090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:59.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.886+0000 7f5590d22740 4 client.19686 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:02:59.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.886+0000 7f5590d22740 1 -- 172.21.15.83:0/4211359182 --> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b7d02e6a60 con 0x55b7d0394ca0 2024-04-24T09:02:59.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.886+0000 7f5587400640 1 --2- 172.21.15.83:0/4211359182 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x55b7d0394ca0 0x55b7d0397090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:59.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.887+0000 7f5587400640 1 --2- 172.21.15.83:0/4211359182 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x55b7d0394ca0 0x55b7d0397090 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=99d4b0bacded3064 server_cookie=225d94e7523fe652 in_seq=0 out_seq=0 2024-04-24T09:02:59.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.887+0000 7f5570c00640 10 client.19686 ms_handle_connect on v2:172.21.15.138:6836/900566754 2024-04-24T09:02:59.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.890+0000 7f5570c00640 1 -- 172.21.15.83:0/4211359182 <== mds.1 v2:172.21.15.138:6836/900566754 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b7d02e6a60 con 0x55b7d0394ca0 2024-04-24T09:02:59.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.890+0000 7f5570c00640 10 client.19686 handle_command_reply: tid=0 2024-04-24T09:02:59.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.890+0000 7f5590d22740 1 client.19686 shutdown 2024-04-24T09:02:59.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.890+0000 7f5590d22740 1 -- 172.21.15.83:0/4211359182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7d037a460 msgr2=0x55b7d03818e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:59.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.890+0000 7f5590d22740 1 --2- 172.21.15.83:0/4211359182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7d037a460 0x55b7d03818e0 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7f5574009ba0 tx=0x7f55740048f0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.890+0000 7f5590d22740 1 -- 172.21.15.83:0/4211359182 shutdown_connections 2024-04-24T09:02:59.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.890+0000 7f5590d22740 1 --2- 172.21.15.83:0/4211359182 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x55b7d0394ca0 0x55b7d0397090 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.890+0000 7f5590d22740 1 --2- 172.21.15.83:0/4211359182 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b7d0384b30 0x55b7d0382020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.890+0000 7f5590d22740 1 --2- 172.21.15.83:0/4211359182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7d037a460 0x55b7d03818e0 unknown :-1 s=CLOSED pgs=1842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.890+0000 7f5570c00640 0 client.19686 ms_handle_reset on v2:172.21.15.138:6836/900566754 2024-04-24T09:02:59.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.890+0000 7f5590d22740 1 --2- 172.21.15.83:0/4211359182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b7d0379b40 0x55b7d03845f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.890+0000 7f5590d22740 1 -- 172.21.15.83:0/4211359182 >> 172.21.15.83:0/4211359182 conn(0x55b7d0375e60 msgr2=0x55b7d037bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:02:59.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.890+0000 7f5590d22740 1 -- 172.21.15.83:0/4211359182 shutdown_connections 2024-04-24T09:02:59.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.890+0000 7f5590d22740 1 -- 172.21.15.83:0/4211359182 wait complete. 2024-04-24T09:02:59.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.890+0000 7f5590d22740 20 client.19686 trim_cache size 0 max 16384 2024-04-24T09:02:59.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.905+0000 7f5590d22740 1 Processor -- start 2024-04-24T09:02:59.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.905+0000 7f5590d22740 1 -- start start 2024-04-24T09:02:59.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.905+0000 7f5590d22740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b7d0379b40 0x55b7d037c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:59.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.905+0000 7f5590d22740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b7d037a460 0x55b7d0384120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:59.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.905+0000 7f5590d22740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7d0384b30 0x55b7d039c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:59.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.906+0000 7f5590d22740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f55881b1690 con 0x55b7d0384b30 2024-04-24T09:02:59.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.906+0000 7f5590d22740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f55881b32f0 con 0x55b7d0379b40 2024-04-24T09:02:59.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.906+0000 7f5590d22740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f55881b1d60 con 0x55b7d037a460 2024-04-24T09:02:59.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.906+0000 7f5587e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7d0384b30 0x55b7d039c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:59.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.906+0000 7f5587e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7d0384b30 0x55b7d039c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45548/0 (socket says 172.21.15.83:45548) 2024-04-24T09:02:59.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.906+0000 7f5587e00640 1 -- 172.21.15.83:0/3327051321 learned_addr learned my addr 172.21.15.83:0/3327051321 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:02:59.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.906+0000 7f5587400640 1 --2- 172.21.15.83:0/3327051321 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b7d0379b40 0x55b7d037c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:59.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.906+0000 7f5586a00640 1 --2- 172.21.15.83:0/3327051321 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b7d037a460 0x55b7d0384120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:59.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.906+0000 7f5587e00640 1 -- 172.21.15.83:0/3327051321 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b7d037a460 msgr2=0x55b7d0384120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:59.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.906+0000 7f5587e00640 1 --2- 172.21.15.83:0/3327051321 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b7d037a460 0x55b7d0384120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.906+0000 7f5587e00640 1 -- 172.21.15.83:0/3327051321 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b7d0379b40 msgr2=0x55b7d037c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:59.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.906+0000 7f5587e00640 1 --2- 172.21.15.83:0/3327051321 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b7d0379b40 0x55b7d037c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.906+0000 7f5587e00640 1 -- 172.21.15.83:0/3327051321 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7d00bb7d0 con 0x55b7d0384b30 2024-04-24T09:02:59.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.906+0000 7f5586a00640 1 --2- 172.21.15.83:0/3327051321 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b7d037a460 0x55b7d0384120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:02:59.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.907+0000 7f5587e00640 1 --2- 172.21.15.83:0/3327051321 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7d0384b30 0x55b7d039c740 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f558000a3b0 tx=0x7f5580005560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ea0d4aba752e562 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:02:59.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.907+0000 7f5573400640 1 -- 172.21.15.83:0/3327051321 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f558001d040 con 0x55b7d0384b30 2024-04-24T09:02:59.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.907+0000 7f5573400640 1 -- 172.21.15.83:0/3327051321 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f55800b5070 con 0x55b7d0384b30 2024-04-24T09:02:59.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.907+0000 7f5573400640 1 -- 172.21.15.83:0/3327051321 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f558009a840 con 0x55b7d0384b30 2024-04-24T09:02:59.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.908+0000 7f5590d22740 1 -- 172.21.15.83:0/3327051321 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7d0384b30 msgr2=0x55b7d039c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:59.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.908+0000 7f5590d22740 1 --2- 172.21.15.83:0/3327051321 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7d0384b30 0x55b7d039c740 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f558000a3b0 tx=0x7f5580005560 comp rx=0 tx=0).stop 2024-04-24T09:02:59.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.908+0000 7f5590d22740 1 -- 172.21.15.83:0/3327051321 shutdown_connections 2024-04-24T09:02:59.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.908+0000 7f5590d22740 1 --2- 172.21.15.83:0/3327051321 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7d0384b30 0x55b7d039c740 unknown :-1 s=CLOSED pgs=1843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.908+0000 7f5590d22740 1 --2- 172.21.15.83:0/3327051321 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b7d037a460 0x55b7d0384120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.908+0000 7f5590d22740 1 --2- 172.21.15.83:0/3327051321 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b7d0379b40 0x55b7d037c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.908+0000 7f5590d22740 1 -- 172.21.15.83:0/3327051321 >> 172.21.15.83:0/3327051321 conn(0x55b7d0375e60 msgr2=0x55b7d03911f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:02:59.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.908+0000 7f5590d22740 1 -- 172.21.15.83:0/3327051321 shutdown_connections 2024-04-24T09:02:59.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.908+0000 7f5590d22740 1 -- 172.21.15.83:0/3327051321 wait complete. 2024-04-24T09:02:59.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.909+0000 7f5590d22740 1 -- start start 2024-04-24T09:02:59.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.909+0000 7f5590d22740 1 Processor -- start 2024-04-24T09:02:59.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.910+0000 7f5590d22740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b7d0379b40 0x55b7d038d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:59.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.910+0000 7f5590d22740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b7d037a460 0x55b7d03855f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:59.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.910+0000 7f5590d22740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7d0384b30 0x55b7d0385b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:59.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.910+0000 7f5590d22740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f55881b2bc0 con 0x55b7d0384b30 2024-04-24T09:02:59.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.910+0000 7f5590d22740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f55881b2490 con 0x55b7d0379b40 2024-04-24T09:02:59.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.910+0000 7f5590d22740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f55881af690 con 0x55b7d037a460 2024-04-24T09:02:59.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.910+0000 7f5590d22740 10 client.0 _ll_register_callbacks cb 0x7f5574009d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:02:59.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.910+0000 7f5587e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7d0384b30 0x55b7d0385b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:59.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.910+0000 7f5587e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7d0384b30 0x55b7d0385b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45564/0 (socket says 172.21.15.83:45564) 2024-04-24T09:02:59.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.910+0000 7f5587e00640 1 -- 172.21.15.83:0/159726967 learned_addr learned my addr 172.21.15.83:0/159726967 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:02:59.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.910+0000 7f5587400640 1 --2- 172.21.15.83:0/159726967 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b7d0379b40 0x55b7d038d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:59.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.911+0000 7f5586a00640 1 --2- 172.21.15.83:0/159726967 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b7d037a460 0x55b7d03855f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:59.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.911+0000 7f5587e00640 1 -- 172.21.15.83:0/159726967 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b7d037a460 msgr2=0x55b7d03855f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:59.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.911+0000 7f5587e00640 1 --2- 172.21.15.83:0/159726967 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b7d037a460 0x55b7d03855f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.911+0000 7f5587e00640 1 -- 172.21.15.83:0/159726967 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b7d0379b40 msgr2=0x55b7d038d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:02:59.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.911+0000 7f5587e00640 1 --2- 172.21.15.83:0/159726967 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b7d0379b40 0x55b7d038d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.911+0000 7f5587e00640 1 -- 172.21.15.83:0/159726967 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7d03862a0 con 0x55b7d0384b30 2024-04-24T09:02:59.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.911+0000 7f5587e00640 1 --2- 172.21.15.83:0/159726967 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7d0384b30 0x55b7d0385b30 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f55800988c0 tx=0x7f55800058f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ef084a54568d2cb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:02:59.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.911+0000 7f5570c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:02:59.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.911+0000 7f5570c00640 1 -- 172.21.15.83:0/159726967 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55800b9020 con 0x55b7d0384b30 2024-04-24T09:02:59.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.911+0000 7f5570c00640 1 -- 172.21.15.83:0/159726967 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9583+0+0 (secure 0 0 0) 0x7f55800bb2c0 con 0x55b7d0384b30 2024-04-24T09:02:59.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.911+0000 7f5590d22740 1 -- 172.21.15.83:0/159726967 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55800b5070 con 0x55b7d0384b30 2024-04-24T09:02:59.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.912+0000 7f5570c00640 1 -- 172.21.15.83:0/159726967 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f558009c4b0 con 0x55b7d0384b30 2024-04-24T09:02:59.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.912+0000 7f5590d22740 10 client.19692 fetch_fsmap learned FSMap version 41 2024-04-24T09:02:59.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.912+0000 7f5590d22740 1 -- 172.21.15.83:0/159726967 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55b7d03fafb0 con 0x55b7d0384b30 2024-04-24T09:02:59.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.912+0000 7f5570c00640 1 -- 172.21.15.83:0/159726967 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f55800b5070 con 0x55b7d0384b30 2024-04-24T09:02:59.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.912+0000 7f5570c00640 1 -- 172.21.15.83:0/159726967 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7f5580098400 con 0x55b7d0384b30 2024-04-24T09:02:59.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.912+0000 7f5590d22740 10 client.19692 fetch_fsmap finished waiting for FSMap version 41 2024-04-24T09:02:59.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.912+0000 7f5590d22740 10 client.19692 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-24T09:02:59.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.912+0000 7f5590d22740 20 client.19692 populate_metadata read hostname 'smithi083' 2024-04-24T09:02:59.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.912+0000 7f5590d22740 1 --2- 172.21.15.83:0/159726967 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x55b7d03fb5f0 0x55b7d03fda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:02:59.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.912+0000 7f5590d22740 4 client.19692 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:02:59.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.912+0000 7f5590d22740 1 -- 172.21.15.83:0/159726967 --> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b7d02624d0 con 0x55b7d03fb5f0 2024-04-24T09:02:59.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.913+0000 7f5587400640 1 --2- 172.21.15.83:0/159726967 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x55b7d03fb5f0 0x55b7d03fda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:02:59.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.914+0000 7f5587400640 1 --2- 172.21.15.83:0/159726967 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x55b7d03fb5f0 0x55b7d03fda10 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a8c52c13ae8f7271 server_cookie=d2e9888a7864021 in_seq=0 out_seq=0 2024-04-24T09:02:59.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.914+0000 7f5570c00640 10 client.19692 ms_handle_connect on v2:172.21.15.138:6836/900566754 2024-04-24T09:02:59.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.915+0000 7f5570c00640 1 -- 172.21.15.83:0/159726967 <== mds.1 v2:172.21.15.138:6836/900566754 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b7d02624d0 con 0x55b7d03fb5f0 2024-04-24T09:02:59.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.915+0000 7f5570c00640 10 client.19692 handle_command_reply: tid=0 2024-04-24T09:02:59.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.915+0000 7f5590d22740 1 client.19692 shutdown 2024-04-24T09:02:59.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.915+0000 7f5590d22740 1 -- 172.21.15.83:0/159726967 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7d0384b30 msgr2=0x55b7d0385b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:59.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.915+0000 7f5590d22740 1 --2- 172.21.15.83:0/159726967 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7d0384b30 0x55b7d0385b30 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f55800988c0 tx=0x7f55800058f0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.915+0000 7f5590d22740 1 -- 172.21.15.83:0/159726967 shutdown_connections 2024-04-24T09:02:59.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.915+0000 7f5590d22740 1 --2- 172.21.15.83:0/159726967 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7d0384b30 0x55b7d0385b30 unknown :-1 s=CLOSED pgs=1844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.915+0000 7f5590d22740 1 --2- 172.21.15.83:0/159726967 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x55b7d03fb5f0 0x55b7d03fda10 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.915+0000 7f5590d22740 1 --2- 172.21.15.83:0/159726967 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b7d037a460 0x55b7d03855f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.915+0000 7f5590d22740 1 --2- 172.21.15.83:0/159726967 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b7d0379b40 0x55b7d038d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.915+0000 7f5570c00640 0 client.19692 ms_handle_reset on v2:172.21.15.138:6836/900566754 2024-04-24T09:02:59.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.915+0000 7f5590d22740 1 -- 172.21.15.83:0/159726967 >> 172.21.15.83:0/159726967 conn(0x55b7d0375e60 msgr2=0x55b7d03746c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:02:59.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.915+0000 7f5590d22740 1 -- 172.21.15.83:0/159726967 shutdown_connections 2024-04-24T09:02:59.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.915+0000 7f5590d22740 1 -- 172.21.15.83:0/159726967 wait complete. 2024-04-24T09:02:59.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.915+0000 7f5590d22740 20 client.19692 trim_cache size 0 max 16384 2024-04-24T09:02:59.916 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:02:59.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.916+0000 7f558cc00640 1 -- 172.21.15.83:0/4157859461 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f556c0a3d20 msgr2=0x7f556c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:59.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.916+0000 7f558cc00640 1 --2- 172.21.15.83:0/4157859461 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f556c0a3d20 0x7f556c0a61e0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x55b7d037b100 tx=0x7f5578004a40 comp rx=0 tx=0).stop 2024-04-24T09:02:59.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.916+0000 7f558cc00640 1 -- 172.21.15.83:0/4157859461 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f558810c480 msgr2=0x7f558819a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:02:59.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.916+0000 7f558cc00640 1 --2- 172.21.15.83:0/4157859461 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f558810c480 0x7f558819a740 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f558000c970 tx=0x7f558000ce30 comp rx=0 tx=0).stop 2024-04-24T09:02:59.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.916+0000 7f558cc00640 1 -- 172.21.15.83:0/4157859461 shutdown_connections 2024-04-24T09:02:59.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.916+0000 7f558cc00640 1 --2- 172.21.15.83:0/4157859461 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f556c0a3d20 0x7f556c0a61e0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.916+0000 7f558cc00640 1 --2- 172.21.15.83:0/4157859461 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f558810c480 0x7f558819a740 unknown :-1 s=CLOSED pgs=1840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.916+0000 7f558cc00640 1 --2- 172.21.15.83:0/4157859461 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5588103980 0x7f55881a0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.916+0000 7f558cc00640 1 --2- 172.21.15.83:0/4157859461 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5588103060 0x7f55881a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:02:59.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.916+0000 7f558cc00640 1 -- 172.21.15.83:0/4157859461 >> 172.21.15.83:0/4157859461 conn(0x7f55880fc880 msgr2=0x7f5588101d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:02:59.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.916+0000 7f558cc00640 1 -- 172.21.15.83:0/4157859461 shutdown_connections 2024-04-24T09:02:59.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:02:59.916+0000 7f558cc00640 1 -- 172.21.15.83:0/4157859461 wait complete. 2024-04-24T09:02:59.933 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T09:03:00.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.137+0000 7f23d9c00640 1 -- 172.21.15.83:0/2476168899 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23d4102110 msgr2=0x7f23d41024f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:00.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.137+0000 7f23d9c00640 1 --2- 172.21.15.83:0/2476168899 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23d4102110 0x7f23d41024f0 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f23c4009900 tx=0x7f23c402a410 comp rx=0 tx=0).stop 2024-04-24T09:03:00.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.138+0000 7f23d9c00640 1 -- 172.21.15.83:0/2476168899 shutdown_connections 2024-04-24T09:03:00.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.138+0000 7f23d9c00640 1 --2- 172.21.15.83:0/2476168899 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f23d41071e0 0x7f23d410edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:00.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.138+0000 7f23d9c00640 1 --2- 172.21.15.83:0/2476168899 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23d4102ac0 0x7f23d4106b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:00.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.138+0000 7f23d9c00640 1 --2- 172.21.15.83:0/2476168899 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23d4102110 0x7f23d41024f0 unknown :-1 s=CLOSED pgs=1845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:00.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.138+0000 7f23d9c00640 1 -- 172.21.15.83:0/2476168899 >> 172.21.15.83:0/2476168899 conn(0x7f23d40fd9d0 msgr2=0x7f23d40ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:00.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.138+0000 7f23d9c00640 1 -- 172.21.15.83:0/2476168899 shutdown_connections 2024-04-24T09:03:00.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.138+0000 7f23d9c00640 1 -- 172.21.15.83:0/2476168899 wait complete. 2024-04-24T09:03:00.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.139+0000 7f23d9c00640 1 Processor -- start 2024-04-24T09:03:00.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.139+0000 7f23d9c00640 1 -- start start 2024-04-24T09:03:00.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.139+0000 7f23d9c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f23d41071e0 0x7f23d41a07f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:00.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.139+0000 7f23d9c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23d41a0d30 0x7f23d419a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:00.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.139+0000 7f23d9c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23d419aee0 0x7f23d419b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:00.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.139+0000 7f23d9c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f23d41121a0 con 0x7f23d41a0d30 2024-04-24T09:03:00.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.139+0000 7f23d9c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f23d4112020 con 0x7f23d419aee0 2024-04-24T09:03:00.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.139+0000 7f23d9c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f23d4112320 con 0x7f23d41071e0 2024-04-24T09:03:00.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.140+0000 7f23cbe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23d41a0d30 0x7f23d419a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:00.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.140+0000 7f23cbe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23d41a0d30 0x7f23d419a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45570/0 (socket says 172.21.15.83:45570) 2024-04-24T09:03:00.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.140+0000 7f23cbe00640 1 -- 172.21.15.83:0/3159806849 learned_addr learned my addr 172.21.15.83:0/3159806849 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:03:00.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.140+0000 7f23d3e00640 1 --2- 172.21.15.83:0/3159806849 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f23d41071e0 0x7f23d41a07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:00.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.140+0000 7f23d9200640 1 --2- 172.21.15.83:0/3159806849 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23d419aee0 0x7f23d419b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:00.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.140+0000 7f23cbe00640 1 -- 172.21.15.83:0/3159806849 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f23d41071e0 msgr2=0x7f23d41a07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:00.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.140+0000 7f23cbe00640 1 --2- 172.21.15.83:0/3159806849 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f23d41071e0 0x7f23d41a07f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:00.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.140+0000 7f23cbe00640 1 -- 172.21.15.83:0/3159806849 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23d419aee0 msgr2=0x7f23d419b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:00.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.140+0000 7f23cbe00640 1 --2- 172.21.15.83:0/3159806849 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23d419aee0 0x7f23d419b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:00.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.140+0000 7f23cbe00640 1 -- 172.21.15.83:0/3159806849 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23d41a76b0 con 0x7f23d41a0d30 2024-04-24T09:03:00.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.140+0000 7f23d9200640 1 --2- 172.21.15.83:0/3159806849 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23d419aee0 0x7f23d419b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:03:00.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.140+0000 7f23cbe00640 1 --2- 172.21.15.83:0/3159806849 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23d41a0d30 0x7f23d419a9a0 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f23bc00e9c0 tx=0x7f23bc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:00.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.141+0000 7f23d1600640 1 -- 172.21.15.83:0/3159806849 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23bc00cd80 con 0x7f23d41a0d30 2024-04-24T09:03:00.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.141+0000 7f23d1600640 1 -- 172.21.15.83:0/3159806849 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f23bc0054d0 con 0x7f23d41a0d30 2024-04-24T09:03:00.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.141+0000 7f23d9c00640 1 -- 172.21.15.83:0/3159806849 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23d41a78e0 con 0x7f23d41a0d30 2024-04-24T09:03:00.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.141+0000 7f23d1600640 1 -- 172.21.15.83:0/3159806849 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23bc010430 con 0x7f23d41a0d30 2024-04-24T09:03:00.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.141+0000 7f23d9c00640 1 -- 172.21.15.83:0/3159806849 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23d41a7e50 con 0x7f23d41a0d30 2024-04-24T09:03:00.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.144+0000 7f23d1600640 1 -- 172.21.15.83:0/3159806849 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f23bc004fd0 con 0x7f23d41a0d30 2024-04-24T09:03:00.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.144+0000 7f23ddfe8740 1 Processor -- start 2024-04-24T09:03:00.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.145+0000 7f23ddfe8740 1 -- start start 2024-04-24T09:03:00.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.145+0000 7f23ddfe8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c6e4559b40 0x55c6e4559f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:00.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.145+0000 7f23ddfe8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c6e455a460 0x55c6e45645f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:00.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.145+0000 7f23ddfe8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c6e4564b30 0x55c6e4566f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:00.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.145+0000 7f23ddfe8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c6e44bf1d0 con 0x55c6e4559b40 2024-04-24T09:03:00.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.145+0000 7f23ddfe8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c6e4465450 con 0x55c6e4564b30 2024-04-24T09:03:00.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.145+0000 7f23ddfe8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55c6e44bb190 con 0x55c6e455a460 2024-04-24T09:03:00.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.145+0000 7f23d9200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c6e4559b40 0x55c6e4559f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:00.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.145+0000 7f23d9200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c6e4559b40 0x55c6e4559f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45586/0 (socket says 172.21.15.83:45586) 2024-04-24T09:03:00.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.145+0000 7f23d9200640 1 -- 172.21.15.83:0/1825662507 learned_addr learned my addr 172.21.15.83:0/1825662507 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:03:00.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.146+0000 7f23d3e00640 1 --2- 172.21.15.83:0/1825662507 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c6e455a460 0x55c6e45645f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:00.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.146+0000 7f23cbe00640 1 --2- 172.21.15.83:0/1825662507 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c6e4564b30 0x55c6e4566f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:00.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.146+0000 7f23d9200640 1 -- 172.21.15.83:0/1825662507 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c6e455a460 msgr2=0x55c6e45645f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:00.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.146+0000 7f23d9200640 1 --2- 172.21.15.83:0/1825662507 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c6e455a460 0x55c6e45645f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:00.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.146+0000 7f23d9200640 1 -- 172.21.15.83:0/1825662507 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c6e4564b30 msgr2=0x55c6e4566f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:00.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.146+0000 7f23d9200640 1 --2- 172.21.15.83:0/1825662507 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c6e4564b30 0x55c6e4566f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:00.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.146+0000 7f23d9200640 1 -- 172.21.15.83:0/1825662507 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6e4522310 con 0x55c6e4559b40 2024-04-24T09:03:00.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.146+0000 7f23cbe00640 1 --2- 172.21.15.83:0/1825662507 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c6e4564b30 0x55c6e4566f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:03:00.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.146+0000 7f23d1600640 1 --2- 172.21.15.83:0/3159806849 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f23b80a3da0 0x7f23b80a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:00.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.146+0000 7f23d9200640 1 --2- 172.21.15.83:0/1825662507 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c6e4559b40 0x55c6e4559f20 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7f23cc009f10 tx=0x7f23cc00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88f8871844faacfa server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:00.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.146+0000 7f23d1600640 1 -- 172.21.15.83:0/3159806849 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9583+0+0 (secure 0 0 0) 0x7f23bc0d1190 con 0x7f23d41a0d30 2024-04-24T09:03:00.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.146+0000 7f23d3e00640 1 --2- 172.21.15.83:0/3159806849 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f23b80a3da0 0x7f23b80a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:00.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.147+0000 7f23c9600640 1 -- 172.21.15.83:0/1825662507 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23cc003b70 con 0x55c6e4559b40 2024-04-24T09:03:00.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.147+0000 7f23c9600640 1 -- 172.21.15.83:0/1825662507 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f23cc003d10 con 0x55c6e4559b40 2024-04-24T09:03:00.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.147+0000 7f23c9600640 1 -- 172.21.15.83:0/1825662507 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23cc011a00 con 0x55c6e4559b40 2024-04-24T09:03:00.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.147+0000 7f23ddfe8740 1 -- 172.21.15.83:0/1825662507 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c6e4559b40 msgr2=0x55c6e4559f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:00.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.147+0000 7f23d3e00640 1 --2- 172.21.15.83:0/3159806849 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f23b80a3da0 0x7f23b80a6260 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f23c4006010 tx=0x7f23c4005a20 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:00.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.147+0000 7f23ddfe8740 1 --2- 172.21.15.83:0/1825662507 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c6e4559b40 0x55c6e4559f20 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7f23cc009f10 tx=0x7f23cc00f730 comp rx=0 tx=0).stop 2024-04-24T09:03:00.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.147+0000 7f23ddfe8740 1 -- 172.21.15.83:0/1825662507 shutdown_connections 2024-04-24T09:03:00.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.147+0000 7f23ddfe8740 1 --2- 172.21.15.83:0/1825662507 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c6e4564b30 0x55c6e4566f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:00.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.148+0000 7f23ddfe8740 1 --2- 172.21.15.83:0/1825662507 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c6e455a460 0x55c6e45645f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:00.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.148+0000 7f23ddfe8740 1 --2- 172.21.15.83:0/1825662507 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c6e4559b40 0x55c6e4559f20 unknown :-1 s=CLOSED pgs=1847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:00.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.148+0000 7f23ddfe8740 1 -- 172.21.15.83:0/1825662507 >> 172.21.15.83:0/1825662507 conn(0x55c6e4555e60 msgr2=0x55c6e4556240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:00.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.148+0000 7f23ddfe8740 1 -- 172.21.15.83:0/1825662507 shutdown_connections 2024-04-24T09:03:00.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.148+0000 7f23ddfe8740 1 -- 172.21.15.83:0/1825662507 wait complete. 2024-04-24T09:03:00.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.148+0000 7f23ddfe8740 1 -- start start 2024-04-24T09:03:00.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.148+0000 7f23ddfe8740 1 Processor -- start 2024-04-24T09:03:00.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.149+0000 7f23ddfe8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c6e4559b40 0x55c6e45618f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:00.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.149+0000 7f23ddfe8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c6e455a460 0x55c6e4561e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:00.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.149+0000 7f23ddfe8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c6e4564b30 0x55c6e4562370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:00.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.149+0000 7f23ddfe8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c6e45679c0 con 0x55c6e455a460 2024-04-24T09:03:00.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.149+0000 7f23ddfe8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c6e4478810 con 0x55c6e4564b30 2024-04-24T09:03:00.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.149+0000 7f23ddfe8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55c6e4567b40 con 0x55c6e4559b40 2024-04-24T09:03:00.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.149+0000 7f23ddfe8740 10 client.0 _ll_register_callbacks cb 0x55c6e4446140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:03:00.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.149+0000 7f23cbe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c6e455a460 0x55c6e4561e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:00.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.149+0000 7f23cbe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c6e455a460 0x55c6e4561e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45596/0 (socket says 172.21.15.83:45596) 2024-04-24T09:03:00.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.149+0000 7f23cbe00640 1 -- 172.21.15.83:0/2668576131 learned_addr learned my addr 172.21.15.83:0/2668576131 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:03:00.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.150+0000 7f23d3e00640 1 --2- 172.21.15.83:0/2668576131 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c6e4559b40 0x55c6e45618f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:00.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.150+0000 7f23d9200640 1 --2- 172.21.15.83:0/2668576131 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c6e4564b30 0x55c6e4562370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:00.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.150+0000 7f23cbe00640 1 -- 172.21.15.83:0/2668576131 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c6e4559b40 msgr2=0x55c6e45618f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:00.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.150+0000 7f23cbe00640 1 --2- 172.21.15.83:0/2668576131 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c6e4559b40 0x55c6e45618f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:00.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.150+0000 7f23cbe00640 1 -- 172.21.15.83:0/2668576131 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c6e4564b30 msgr2=0x55c6e4562370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:00.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.150+0000 7f23cbe00640 1 --2- 172.21.15.83:0/2668576131 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c6e4564b30 0x55c6e4562370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:00.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.150+0000 7f23cbe00640 1 -- 172.21.15.83:0/2668576131 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6e44893f0 con 0x55c6e455a460 2024-04-24T09:03:00.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.150+0000 7f23cbe00640 1 --2- 172.21.15.83:0/2668576131 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c6e455a460 0x55c6e4561e30 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f23bc09c9b0 tx=0x7f23bc09c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ba4be278aedee65 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:00.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.150+0000 7f23c2000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:03:00.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.150+0000 7f23c2000640 1 -- 172.21.15.83:0/2668576131 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23bc0ce8e0 con 0x55c6e455a460 2024-04-24T09:03:00.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.150+0000 7f23c2000640 1 -- 172.21.15.83:0/2668576131 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9583+0+0 (secure 0 0 0) 0x7f23bc0bd0c0 con 0x55c6e455a460 2024-04-24T09:03:00.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.150+0000 7f23ddfe8740 1 -- 172.21.15.83:0/2668576131 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23cc003d10 con 0x55c6e455a460 2024-04-24T09:03:00.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.151+0000 7f23c2000640 1 -- 172.21.15.83:0/2668576131 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f23bc09ccb0 con 0x55c6e455a460 2024-04-24T09:03:00.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.151+0000 7f23ddfe8740 10 client.19704 fetch_fsmap learned FSMap version 41 2024-04-24T09:03:00.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.151+0000 7f23ddfe8740 1 -- 172.21.15.83:0/2668576131 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55c6e41d13f0 con 0x55c6e455a460 2024-04-24T09:03:00.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.151+0000 7f23c2000640 1 -- 172.21.15.83:0/2668576131 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f23cc003d10 con 0x55c6e455a460 2024-04-24T09:03:00.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.151+0000 7f23c2000640 1 -- 172.21.15.83:0/2668576131 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7f23bc0a5520 con 0x55c6e455a460 2024-04-24T09:03:00.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.151+0000 7f23ddfe8740 10 client.19704 fetch_fsmap finished waiting for FSMap version 41 2024-04-24T09:03:00.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.151+0000 7f23ddfe8740 10 client.19704 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:03:00.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.151+0000 7f23ddfe8740 20 client.19704 populate_metadata read hostname 'smithi083' 2024-04-24T09:03:00.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.151+0000 7f23ddfe8740 1 --2- 172.21.15.83:0/2668576131 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55c6e4574cf0 0x55c6e4577120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:00.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.151+0000 7f23ddfe8740 4 client.19704 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:03:00.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.151+0000 7f23ddfe8740 1 -- 172.21.15.83:0/2668576131 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c6e44ccbf0 con 0x55c6e4574cf0 2024-04-24T09:03:00.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.152+0000 7f23d3e00640 1 --2- 172.21.15.83:0/2668576131 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55c6e4574cf0 0x55c6e4577120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:00.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.153+0000 7f23d3e00640 1 --2- 172.21.15.83:0/2668576131 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55c6e4574cf0 0x55c6e4577120 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac24a0e800eff8d4 server_cookie=4e8177297168059f in_seq=0 out_seq=0 2024-04-24T09:03:00.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.153+0000 7f23c2000640 10 client.19704 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:03:00.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.156+0000 7f23c2000640 1 -- 172.21.15.83:0/2668576131 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c6e44ccbf0 con 0x55c6e4574cf0 2024-04-24T09:03:00.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.156+0000 7f23c2000640 10 client.19704 handle_command_reply: tid=0 2024-04-24T09:03:00.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.156+0000 7f23ddfe8740 1 client.19704 shutdown 2024-04-24T09:03:00.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.157+0000 7f23ddfe8740 1 -- 172.21.15.83:0/2668576131 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c6e455a460 msgr2=0x55c6e4561e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:00.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.157+0000 7f23ddfe8740 1 --2- 172.21.15.83:0/2668576131 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c6e455a460 0x55c6e4561e30 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f23bc09c9b0 tx=0x7f23bc09c8b0 comp rx=0 tx=0).stop 2024-04-24T09:03:00.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.157+0000 7f23ddfe8740 1 -- 172.21.15.83:0/2668576131 shutdown_connections 2024-04-24T09:03:00.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.157+0000 7f23ddfe8740 1 --2- 172.21.15.83:0/2668576131 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55c6e4574cf0 0x55c6e4577120 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:00.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.157+0000 7f23ddfe8740 1 --2- 172.21.15.83:0/2668576131 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c6e4564b30 0x55c6e4562370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:00.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.157+0000 7f23ddfe8740 1 --2- 172.21.15.83:0/2668576131 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c6e455a460 0x55c6e4561e30 unknown :-1 s=CLOSED pgs=1848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:00.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.157+0000 7f23ddfe8740 1 --2- 172.21.15.83:0/2668576131 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c6e4559b40 0x55c6e45618f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:00.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.157+0000 7f23c2000640 0 client.19704 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:03:00.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.157+0000 7f23ddfe8740 1 -- 172.21.15.83:0/2668576131 >> 172.21.15.83:0/2668576131 conn(0x55c6e4555e60 msgr2=0x55c6e455bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:00.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.157+0000 7f23ddfe8740 1 -- 172.21.15.83:0/2668576131 shutdown_connections 2024-04-24T09:03:00.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.157+0000 7f23ddfe8740 1 -- 172.21.15.83:0/2668576131 wait complete. 2024-04-24T09:03:00.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.157+0000 7f23ddfe8740 20 client.19704 trim_cache size 0 max 16384 2024-04-24T09:03:00.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.172+0000 7f23ddfe8740 1 Processor -- start 2024-04-24T09:03:00.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.172+0000 7f23ddfe8740 1 -- start start 2024-04-24T09:03:00.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.172+0000 7f23ddfe8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c6e4559b40 0x55c6e455c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:00.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.172+0000 7f23ddfe8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c6e455a460 0x55c6e4570880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:00.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.172+0000 7f23ddfe8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c6e4564b30 0x55c6e457ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:00.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.172+0000 7f23ddfe8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f23d41b1c10 con 0x55c6e4564b30 2024-04-24T09:03:00.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.172+0000 7f23ddfe8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f23d41b3870 con 0x55c6e455a460 2024-04-24T09:03:00.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.172+0000 7f23ddfe8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f23d41b22e0 con 0x55c6e4559b40 2024-04-24T09:03:00.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.173+0000 7f23d9200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c6e4564b30 0x55c6e457ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:00.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.173+0000 7f23d9200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c6e4564b30 0x55c6e457ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45606/0 (socket says 172.21.15.83:45606) 2024-04-24T09:03:00.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.173+0000 7f23d9200640 1 -- 172.21.15.83:0/2098076372 learned_addr learned my addr 172.21.15.83:0/2098076372 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:03:00.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.173+0000 7f23d9200640 1 -- 172.21.15.83:0/2098076372 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c6e4559b40 msgr2=0x55c6e455c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:00.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.173+0000 7f23d9200640 1 --2- 172.21.15.83:0/2098076372 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c6e4559b40 0x55c6e455c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:00.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.173+0000 7f23d9200640 1 -- 172.21.15.83:0/2098076372 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c6e455a460 msgr2=0x55c6e4570880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:00.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.173+0000 7f23d9200640 1 --2- 172.21.15.83:0/2098076372 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c6e455a460 0x55c6e4570880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:00.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.173+0000 7f23d9200640 1 -- 172.21.15.83:0/2098076372 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6e455a290 con 0x55c6e4564b30 2024-04-24T09:03:00.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.173+0000 7f23d9200640 1 --2- 172.21.15.83:0/2098076372 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c6e4564b30 0x55c6e457ca30 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f23cc009c60 tx=0x7f23cc009c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b01a1e64453347d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:00.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.174+0000 7f23c8c00640 1 -- 172.21.15.83:0/2098076372 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23cc029070 con 0x55c6e4564b30 2024-04-24T09:03:00.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.174+0000 7f23c8c00640 1 -- 172.21.15.83:0/2098076372 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f23cc003cb0 con 0x55c6e4564b30 2024-04-24T09:03:00.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.174+0000 7f23c8c00640 1 -- 172.21.15.83:0/2098076372 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23cc011680 con 0x55c6e4564b30 2024-04-24T09:03:00.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.175+0000 7f23ddfe8740 1 -- 172.21.15.83:0/2098076372 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c6e4564b30 msgr2=0x55c6e457ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:00.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.175+0000 7f23ddfe8740 1 --2- 172.21.15.83:0/2098076372 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c6e4564b30 0x55c6e457ca30 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f23cc009c60 tx=0x7f23cc009c90 comp rx=0 tx=0).stop 2024-04-24T09:03:00.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.175+0000 7f23ddfe8740 1 -- 172.21.15.83:0/2098076372 shutdown_connections 2024-04-24T09:03:00.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.175+0000 7f23ddfe8740 1 --2- 172.21.15.83:0/2098076372 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c6e4564b30 0x55c6e457ca30 unknown :-1 s=CLOSED pgs=1849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:00.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.175+0000 7f23ddfe8740 1 --2- 172.21.15.83:0/2098076372 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c6e455a460 0x55c6e4570880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:00.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.175+0000 7f23ddfe8740 1 --2- 172.21.15.83:0/2098076372 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c6e4559b40 0x55c6e455c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:00.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.175+0000 7f23ddfe8740 1 -- 172.21.15.83:0/2098076372 >> 172.21.15.83:0/2098076372 conn(0x55c6e4555e60 msgr2=0x55c6e4554f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:00.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.175+0000 7f23ddfe8740 1 -- 172.21.15.83:0/2098076372 shutdown_connections 2024-04-24T09:03:00.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.175+0000 7f23ddfe8740 1 -- 172.21.15.83:0/2098076372 wait complete. 2024-04-24T09:03:00.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.175+0000 7f23ddfe8740 1 -- start start 2024-04-24T09:03:00.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.176+0000 7f23ddfe8740 1 Processor -- start 2024-04-24T09:03:00.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.176+0000 7f23ddfe8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c6e4559b40 0x55c6e45659c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:00.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.176+0000 7f23ddfe8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c6e455a460 0x55c6e4565f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:00.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.176+0000 7f23ddfe8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c6e4564b30 0x55c6e4566440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:00.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.176+0000 7f23ddfe8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f23d41b3140 con 0x55c6e4564b30 2024-04-24T09:03:00.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.176+0000 7f23ddfe8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f23d41b2a10 con 0x55c6e455a460 2024-04-24T09:03:00.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.176+0000 7f23ddfe8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f23d41afc10 con 0x55c6e4559b40 2024-04-24T09:03:00.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.176+0000 7f23ddfe8740 10 client.0 _ll_register_callbacks cb 0x7f23bc004840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:03:00.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.176+0000 7f23d9200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c6e4564b30 0x55c6e4566440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:00.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.176+0000 7f23d9200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c6e4564b30 0x55c6e4566440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45614/0 (socket says 172.21.15.83:45614) 2024-04-24T09:03:00.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.176+0000 7f23d9200640 1 -- 172.21.15.83:0/3791616749 learned_addr learned my addr 172.21.15.83:0/3791616749 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:03:00.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.177+0000 7f23d3e00640 1 --2- 172.21.15.83:0/3791616749 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c6e4559b40 0x55c6e45659c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:00.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.177+0000 7f23cbe00640 1 --2- 172.21.15.83:0/3791616749 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c6e455a460 0x55c6e4565f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:00.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.177+0000 7f23d9200640 1 -- 172.21.15.83:0/3791616749 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c6e4559b40 msgr2=0x55c6e45659c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:00.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.177+0000 7f23d9200640 1 --2- 172.21.15.83:0/3791616749 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c6e4559b40 0x55c6e45659c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:00.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.177+0000 7f23d9200640 1 -- 172.21.15.83:0/3791616749 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c6e455a460 msgr2=0x55c6e4565f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:00.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.177+0000 7f23d9200640 1 --2- 172.21.15.83:0/3791616749 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c6e455a460 0x55c6e4565f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:00.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.177+0000 7f23d9200640 1 -- 172.21.15.83:0/3791616749 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6e429b7d0 con 0x55c6e4564b30 2024-04-24T09:03:00.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.177+0000 7f23d9200640 1 --2- 172.21.15.83:0/3791616749 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c6e4564b30 0x55c6e4566440 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7f23cc009d20 tx=0x7f23cc0050a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72eee33a77565c8c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:00.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.177+0000 7f23c2000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:03:00.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.177+0000 7f23c2000640 1 -- 172.21.15.83:0/3791616749 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23cc032060 con 0x55c6e4564b30 2024-04-24T09:03:00.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.177+0000 7f23c2000640 1 -- 172.21.15.83:0/3791616749 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9583+0+0 (secure 0 0 0) 0x7f23cc005210 con 0x55c6e4564b30 2024-04-24T09:03:00.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.177+0000 7f23ddfe8740 1 -- 172.21.15.83:0/3791616749 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23cc003cb0 con 0x55c6e4564b30 2024-04-24T09:03:00.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.178+0000 7f23c2000640 1 -- 172.21.15.83:0/3791616749 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f23cc029040 con 0x55c6e4564b30 2024-04-24T09:03:00.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.178+0000 7f23ddfe8740 10 client.19710 fetch_fsmap learned FSMap version 41 2024-04-24T09:03:00.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.178+0000 7f23ddfe8740 1 -- 172.21.15.83:0/3791616749 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55c6e45db460 con 0x55c6e4564b30 2024-04-24T09:03:00.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.178+0000 7f23c2000640 1 -- 172.21.15.83:0/3791616749 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f23cc003cb0 con 0x55c6e4564b30 2024-04-24T09:03:00.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.178+0000 7f23c2000640 1 -- 172.21.15.83:0/3791616749 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7f23cc02c070 con 0x55c6e4564b30 2024-04-24T09:03:00.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.178+0000 7f23ddfe8740 10 client.19710 fetch_fsmap finished waiting for FSMap version 41 2024-04-24T09:03:00.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.178+0000 7f23ddfe8740 10 client.19710 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:03:00.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.178+0000 7f23ddfe8740 20 client.19710 populate_metadata read hostname 'smithi083' 2024-04-24T09:03:00.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.178+0000 7f23ddfe8740 1 --2- 172.21.15.83:0/3791616749 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55c6e45dba10 0x55c6e45dde30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:00.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.178+0000 7f23ddfe8740 4 client.19710 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:03:00.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.178+0000 7f23ddfe8740 1 -- 172.21.15.83:0/3791616749 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c6e44c6a60 con 0x55c6e45dba10 2024-04-24T09:03:00.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.179+0000 7f23d3e00640 1 --2- 172.21.15.83:0/3791616749 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55c6e45dba10 0x55c6e45dde30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:00.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.180+0000 7f23d3e00640 1 --2- 172.21.15.83:0/3791616749 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55c6e45dba10 0x55c6e45dde30 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3284be2587acde75 server_cookie=83397e57da479df0 in_seq=0 out_seq=0 2024-04-24T09:03:00.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.180+0000 7f23c2000640 10 client.19710 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:03:00.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.182+0000 7f23c2000640 1 -- 172.21.15.83:0/3791616749 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c6e44c6a60 con 0x55c6e45dba10 2024-04-24T09:03:00.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.182+0000 7f23c2000640 10 client.19710 handle_command_reply: tid=0 2024-04-24T09:03:00.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.182+0000 7f23ddfe8740 1 client.19710 shutdown 2024-04-24T09:03:00.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.182+0000 7f23ddfe8740 1 -- 172.21.15.83:0/3791616749 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c6e4564b30 msgr2=0x55c6e4566440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:00.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.182+0000 7f23ddfe8740 1 --2- 172.21.15.83:0/3791616749 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c6e4564b30 0x55c6e4566440 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7f23cc009d20 tx=0x7f23cc0050a0 comp rx=0 tx=0).stop 2024-04-24T09:03:00.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.182+0000 7f23ddfe8740 1 -- 172.21.15.83:0/3791616749 shutdown_connections 2024-04-24T09:03:00.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.182+0000 7f23ddfe8740 1 --2- 172.21.15.83:0/3791616749 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55c6e45dba10 0x55c6e45dde30 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:00.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.182+0000 7f23ddfe8740 1 --2- 172.21.15.83:0/3791616749 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c6e4564b30 0x55c6e4566440 unknown :-1 s=CLOSED pgs=1850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:00.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.182+0000 7f23ddfe8740 1 --2- 172.21.15.83:0/3791616749 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c6e455a460 0x55c6e4565f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:00.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.182+0000 7f23ddfe8740 1 --2- 172.21.15.83:0/3791616749 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c6e4559b40 0x55c6e45659c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:00.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.182+0000 7f23ddfe8740 1 -- 172.21.15.83:0/3791616749 >> 172.21.15.83:0/3791616749 conn(0x55c6e4555e60 msgr2=0x55c6e4561860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:00.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.182+0000 7f23c2000640 0 client.19710 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:03:00.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.182+0000 7f23ddfe8740 1 -- 172.21.15.83:0/3791616749 shutdown_connections 2024-04-24T09:03:00.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.182+0000 7f23ddfe8740 1 -- 172.21.15.83:0/3791616749 wait complete. 2024-04-24T09:03:00.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.182+0000 7f23ddfe8740 20 client.19710 trim_cache size 0 max 16384 2024-04-24T09:03:00.182 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:03:00.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.183+0000 7f23d9c00640 1 -- 172.21.15.83:0/3159806849 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f23b80a3da0 msgr2=0x7f23b80a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:00.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.183+0000 7f23d9c00640 1 --2- 172.21.15.83:0/3159806849 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f23b80a3da0 0x7f23b80a6260 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f23c4006010 tx=0x7f23c4005a20 comp rx=0 tx=0).stop 2024-04-24T09:03:00.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.183+0000 7f23d9c00640 1 -- 172.21.15.83:0/3159806849 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23d41a0d30 msgr2=0x7f23d419a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:00.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.183+0000 7f23d9c00640 1 --2- 172.21.15.83:0/3159806849 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23d41a0d30 0x7f23d419a9a0 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f23bc00e9c0 tx=0x7f23bc00ee80 comp rx=0 tx=0).stop 2024-04-24T09:03:00.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.183+0000 7f23d9c00640 1 -- 172.21.15.83:0/3159806849 shutdown_connections 2024-04-24T09:03:00.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.183+0000 7f23d9c00640 1 --2- 172.21.15.83:0/3159806849 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f23b80a3da0 0x7f23b80a6260 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:00.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.183+0000 7f23d9c00640 1 --2- 172.21.15.83:0/3159806849 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23d419aee0 0x7f23d419b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:00.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.183+0000 7f23d9c00640 1 --2- 172.21.15.83:0/3159806849 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23d41a0d30 0x7f23d419a9a0 unknown :-1 s=CLOSED pgs=1846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:00.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.183+0000 7f23d9c00640 1 --2- 172.21.15.83:0/3159806849 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f23d41071e0 0x7f23d41a07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:00.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.183+0000 7f23d9c00640 1 -- 172.21.15.83:0/3159806849 >> 172.21.15.83:0/3159806849 conn(0x7f23d40fd9d0 msgr2=0x7f23d40fddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:00.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.183+0000 7f23d9c00640 1 -- 172.21.15.83:0/3159806849 shutdown_connections 2024-04-24T09:03:00.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:00.183+0000 7f23d9c00640 1 -- 172.21.15.83:0/3159806849 wait complete. 2024-04-24T09:03:00.199 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:02:59 smithi083 ceph-mon[27907]: pgmap v356: 129 pgs: 129 active+clean; 61 MiB data, 883 MiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 385 KiB/s wr, 73 op/s 2024-04-24T09:03:00.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:02:59 smithi138 ceph-mon[31441]: pgmap v356: 129 pgs: 129 active+clean; 61 MiB data, 883 MiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 385 KiB/s wr, 73 op/s 2024-04-24T09:03:00.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:02:59 smithi189 ceph-mon[31791]: pgmap v356: 129 pgs: 129 active+clean; 61 MiB data, 883 MiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 385 KiB/s wr, 73 op/s 2024-04-24T09:03:00.616 INFO:tasks.workunit.client.0.smithi083.stdout: 1 66339 0.61 MB/sec execute 454 sec latency 57.519 ms 2024-04-24T09:03:01.202 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T09:03:01.203 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T09:03:01.203 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T09:03:01.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.443+0000 7f372f800640 1 -- 172.21.15.83:0/1067106062 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3728102290 msgr2=0x7f3728102670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:01.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.443+0000 7f372f800640 1 --2- 172.21.15.83:0/1067106062 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3728102290 0x7f3728102670 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7f371c009900 tx=0x7f371c02a3f0 comp rx=0 tx=0).stop 2024-04-24T09:03:01.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.443+0000 7f372f800640 1 -- 172.21.15.83:0/1067106062 shutdown_connections 2024-04-24T09:03:01.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.443+0000 7f372f800640 1 --2- 172.21.15.83:0/1067106062 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3728107360 0x7f372810ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:01.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.443+0000 7f372f800640 1 --2- 172.21.15.83:0/1067106062 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3728102c40 0x7f3728106c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:01.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.443+0000 7f372f800640 1 --2- 172.21.15.83:0/1067106062 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3728102290 0x7f3728102670 unknown :-1 s=CLOSED pgs=1851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:01.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.443+0000 7f372f800640 1 -- 172.21.15.83:0/1067106062 >> 172.21.15.83:0/1067106062 conn(0x7f37280fdb10 msgr2=0x7f37280fff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:01.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.443+0000 7f372f800640 1 -- 172.21.15.83:0/1067106062 shutdown_connections 2024-04-24T09:03:01.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.443+0000 7f372f800640 1 -- 172.21.15.83:0/1067106062 wait complete. 2024-04-24T09:03:01.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.444+0000 7f372f800640 1 Processor -- start 2024-04-24T09:03:01.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.444+0000 7f372f800640 1 -- start start 2024-04-24T09:03:01.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.445+0000 7f372f800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3728102290 0x7f3728074300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:01.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.445+0000 7f372f800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3728107360 0x7f3728074840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:01.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.445+0000 7f372f800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3728074d80 0x7f37281a6a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:01.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.445+0000 7f372f800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f37281122d0 con 0x7f3728107360 2024-04-24T09:03:01.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.445+0000 7f372f800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3728112150 con 0x7f3728074d80 2024-04-24T09:03:01.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.445+0000 7f372f800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3728112450 con 0x7f3728102290 2024-04-24T09:03:01.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.445+0000 7f3727e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3728107360 0x7f3728074840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:01.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.445+0000 7f3727e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3728107360 0x7f3728074840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45642/0 (socket says 172.21.15.83:45642) 2024-04-24T09:03:01.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.445+0000 7f3727e00640 1 -- 172.21.15.83:0/3180241918 learned_addr learned my addr 172.21.15.83:0/3180241918 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:03:01.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.445+0000 7f372ce00640 1 --2- 172.21.15.83:0/3180241918 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3728102290 0x7f3728074300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:01.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.445+0000 7f372d800640 1 --2- 172.21.15.83:0/3180241918 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3728074d80 0x7f37281a6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:01.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.446+0000 7f3727e00640 1 -- 172.21.15.83:0/3180241918 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3728102290 msgr2=0x7f3728074300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:01.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.446+0000 7f3727e00640 1 --2- 172.21.15.83:0/3180241918 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3728102290 0x7f3728074300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:01.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.446+0000 7f3727e00640 1 -- 172.21.15.83:0/3180241918 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3728074d80 msgr2=0x7f37281a6a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:01.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.446+0000 7f3727e00640 1 --2- 172.21.15.83:0/3180241918 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3728074d80 0x7f37281a6a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:01.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.446+0000 7f3727e00640 1 -- 172.21.15.83:0/3180241918 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37281a7120 con 0x7f3728107360 2024-04-24T09:03:01.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.446+0000 7f3727e00640 1 --2- 172.21.15.83:0/3180241918 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3728107360 0x7f3728074840 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f371400c970 tx=0x7f371400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:01.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.446+0000 7f3725600640 1 -- 172.21.15.83:0/3180241918 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3714014070 con 0x7f3728107360 2024-04-24T09:03:01.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.446+0000 7f372f800640 1 -- 172.21.15.83:0/3180241918 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37281a7410 con 0x7f3728107360 2024-04-24T09:03:01.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.446+0000 7f3725600640 1 -- 172.21.15.83:0/3180241918 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3714004440 con 0x7f3728107360 2024-04-24T09:03:01.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.447+0000 7f3725600640 1 -- 172.21.15.83:0/3180241918 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3714003e50 con 0x7f3728107360 2024-04-24T09:03:01.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.447+0000 7f372f800640 1 -- 172.21.15.83:0/3180241918 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37281a7960 con 0x7f3728107360 2024-04-24T09:03:01.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.450+0000 7f3732624740 1 Processor -- start 2024-04-24T09:03:01.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.450+0000 7f3732624740 1 -- start start 2024-04-24T09:03:01.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.451+0000 7f3732624740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f26bd1090 0x556f26bd1470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:01.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.451+0000 7f3732624740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f26bd19b0 0x556f26bdbb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:01.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.451+0000 7f3732624740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f26bdc080 0x556f26bde470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:01.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.451+0000 7f3732624740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556f26b36e70 con 0x556f26bdc080 2024-04-24T09:03:01.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.451+0000 7f3732624740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556f26add380 con 0x556f26bd1090 2024-04-24T09:03:01.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.451+0000 7f3732624740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x556f26b32e30 con 0x556f26bd19b0 2024-04-24T09:03:01.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.451+0000 7f3725600640 1 -- 172.21.15.83:0/3180241918 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3714017070 con 0x7f3728107360 2024-04-24T09:03:01.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.451+0000 7f372d800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f26bd1090 0x556f26bd1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:01.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.451+0000 7f372d800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f26bd1090 0x556f26bd1470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:55082/0 (socket says 172.21.15.83:55082) 2024-04-24T09:03:01.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.451+0000 7f372d800640 1 -- 172.21.15.83:0/1780058135 learned_addr learned my addr 172.21.15.83:0/1780058135 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:03:01.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.451+0000 7f372ce00640 1 --2- 172.21.15.83:0/1780058135 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f26bd19b0 0x556f26bdbb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:01.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.451+0000 7f3727e00640 1 --2- 172.21.15.83:0/1780058135 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f26bdc080 0x556f26bde470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:01.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.452+0000 7f3727e00640 1 -- 172.21.15.83:0/1780058135 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f26bd19b0 msgr2=0x556f26bdbb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:01.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.452+0000 7f3727e00640 1 --2- 172.21.15.83:0/1780058135 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f26bd19b0 0x556f26bdbb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:01.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.452+0000 7f3727e00640 1 -- 172.21.15.83:0/1780058135 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f26bd1090 msgr2=0x556f26bd1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:01.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.452+0000 7f3727e00640 1 --2- 172.21.15.83:0/1780058135 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f26bd1090 0x556f26bd1470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:01.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.452+0000 7f3727e00640 1 -- 172.21.15.83:0/1780058135 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f269117e0 con 0x556f26bdc080 2024-04-24T09:03:01.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.452+0000 7f3727e00640 1 --2- 172.21.15.83:0/1780058135 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f26bdc080 0x556f26bde470 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f37140cde70 tx=0x7f37140d9660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ad08c9e783f2511 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:01.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.453+0000 7f371a000640 1 -- 172.21.15.83:0/1780058135 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37140e0020 con 0x556f26bdc080 2024-04-24T09:03:01.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.453+0000 7f3725600640 1 --2- 172.21.15.83:0/3180241918 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f37100a3ec0 0x7f37100a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:01.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.453+0000 7f371a000640 1 -- 172.21.15.83:0/1780058135 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f37140d5040 con 0x556f26bdc080 2024-04-24T09:03:01.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.453+0000 7f371a000640 1 -- 172.21.15.83:0/1780058135 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3714098960 con 0x556f26bdc080 2024-04-24T09:03:01.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.453+0000 7f3725600640 1 -- 172.21.15.83:0/3180241918 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9583+0+0 (secure 0 0 0) 0x7f37140d0ba0 con 0x7f3728107360 2024-04-24T09:03:01.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.453+0000 7f372d800640 1 --2- 172.21.15.83:0/3180241918 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f37100a3ec0 0x7f37100a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:01.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.454+0000 7f3732624740 1 -- 172.21.15.83:0/1780058135 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f26bdc080 msgr2=0x556f26bde470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:01.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.454+0000 7f3732624740 1 --2- 172.21.15.83:0/1780058135 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f26bdc080 0x556f26bde470 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f37140cde70 tx=0x7f37140d9660 comp rx=0 tx=0).stop 2024-04-24T09:03:01.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.454+0000 7f3732624740 1 -- 172.21.15.83:0/1780058135 shutdown_connections 2024-04-24T09:03:01.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.454+0000 7f3732624740 1 --2- 172.21.15.83:0/1780058135 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f26bdc080 0x556f26bde470 unknown :-1 s=CLOSED pgs=1853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:01.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.454+0000 7f3732624740 1 --2- 172.21.15.83:0/1780058135 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f26bd19b0 0x556f26bdbb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:01.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.454+0000 7f3732624740 1 --2- 172.21.15.83:0/1780058135 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f26bd1090 0x556f26bd1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:01.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.454+0000 7f3732624740 1 -- 172.21.15.83:0/1780058135 >> 172.21.15.83:0/1780058135 conn(0x556f26bcd3b0 msgr2=0x556f26bcd790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:01.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.454+0000 7f372d800640 1 --2- 172.21.15.83:0/3180241918 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f37100a3ec0 0x7f37100a6380 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f3720005c40 tx=0x7f3720009760 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:01.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.454+0000 7f3732624740 1 -- 172.21.15.83:0/1780058135 shutdown_connections 2024-04-24T09:03:01.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.454+0000 7f3732624740 1 -- 172.21.15.83:0/1780058135 wait complete. 2024-04-24T09:03:01.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.455+0000 7f3732624740 1 -- start start 2024-04-24T09:03:01.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.455+0000 7f3732624740 1 Processor -- start 2024-04-24T09:03:01.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.455+0000 7f3732624740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f26bd1090 0x556f26bd8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:01.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.455+0000 7f3732624740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f26bd19b0 0x556f26bd9500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:01.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.455+0000 7f3732624740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f26bdc080 0x556f26bd9a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:01.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.455+0000 7f3732624740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556f26bdee70 con 0x556f26bdc080 2024-04-24T09:03:01.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.455+0000 7f3732624740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556f26af0660 con 0x556f26bd1090 2024-04-24T09:03:01.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.455+0000 7f3732624740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x556f26bdeff0 con 0x556f26bd19b0 2024-04-24T09:03:01.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.455+0000 7f3732624740 10 client.0 _ll_register_callbacks cb 0x556f26adeaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:03:01.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.456+0000 7f372d800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f26bdc080 0x556f26bd9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:01.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.456+0000 7f372d800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f26bdc080 0x556f26bd9a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45660/0 (socket says 172.21.15.83:45660) 2024-04-24T09:03:01.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.456+0000 7f372d800640 1 -- 172.21.15.83:0/2386615218 learned_addr learned my addr 172.21.15.83:0/2386615218 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:03:01.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.456+0000 7f372ce00640 1 --2- 172.21.15.83:0/2386615218 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f26bd1090 0x556f26bd8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:01.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.456+0000 7f3727e00640 1 --2- 172.21.15.83:0/2386615218 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f26bd19b0 0x556f26bd9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:01.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.456+0000 7f372d800640 1 -- 172.21.15.83:0/2386615218 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f26bd19b0 msgr2=0x556f26bd9500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:01.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.456+0000 7f372d800640 1 --2- 172.21.15.83:0/2386615218 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f26bd19b0 0x556f26bd9500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:01.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.456+0000 7f372d800640 1 -- 172.21.15.83:0/2386615218 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f26bd1090 msgr2=0x556f26bd8fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:01.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.456+0000 7f372d800640 1 --2- 172.21.15.83:0/2386615218 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f26bd1090 0x556f26bd8fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:01.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.456+0000 7f372d800640 1 -- 172.21.15.83:0/2386615218 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f26971f70 con 0x556f26bdc080 2024-04-24T09:03:01.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.457+0000 7f372d800640 1 --2- 172.21.15.83:0/2386615218 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f26bdc080 0x556f26bd9a40 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f3720010890 tx=0x7f3720004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f077bd20c87464d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:01.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.457+0000 7f370ea00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:03:01.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.457+0000 7f370ea00640 1 -- 172.21.15.83:0/2386615218 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3720027680 con 0x556f26bdc080 2024-04-24T09:03:01.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.457+0000 7f370ea00640 1 -- 172.21.15.83:0/2386615218 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9583+0+0 (secure 0 0 0) 0x7f3720027820 con 0x556f26bdc080 2024-04-24T09:03:01.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.457+0000 7f3732624740 1 -- 172.21.15.83:0/2386615218 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37140d5040 con 0x556f26bdc080 2024-04-24T09:03:01.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.457+0000 7f370ea00640 1 -- 172.21.15.83:0/2386615218 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f3720030050 con 0x556f26bdc080 2024-04-24T09:03:01.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.457+0000 7f3732624740 10 client.19722 fetch_fsmap learned FSMap version 41 2024-04-24T09:03:01.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.457+0000 7f3732624740 1 -- 172.21.15.83:0/2386615218 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x556f26809170 con 0x556f26bdc080 2024-04-24T09:03:01.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.458+0000 7f370ea00640 1 -- 172.21.15.83:0/2386615218 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f37140d5040 con 0x556f26bdc080 2024-04-24T09:03:01.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.458+0000 7f370ea00640 1 -- 172.21.15.83:0/2386615218 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7f372000f040 con 0x556f26bdc080 2024-04-24T09:03:01.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.458+0000 7f3732624740 10 client.19722 fetch_fsmap finished waiting for FSMap version 41 2024-04-24T09:03:01.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.458+0000 7f3732624740 10 client.19722 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:03:01.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.458+0000 7f3732624740 20 client.19722 populate_metadata read hostname 'smithi083' 2024-04-24T09:03:01.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.458+0000 7f3732624740 1 --2- 172.21.15.83:0/2386615218 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556f26bec4e0 0x556f26bee930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:01.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.458+0000 7f3732624740 4 client.19722 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:03:01.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.458+0000 7f3732624740 1 -- 172.21.15.83:0/2386615218 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556f26b44890 con 0x556f26bec4e0 2024-04-24T09:03:01.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.459+0000 7f372ce00640 1 --2- 172.21.15.83:0/2386615218 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556f26bec4e0 0x556f26bee930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:01.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.460+0000 7f372ce00640 1 --2- 172.21.15.83:0/2386615218 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556f26bec4e0 0x556f26bee930 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aee21f78c37a82e8 server_cookie=f2d5ab47086ca2bf in_seq=0 out_seq=0 2024-04-24T09:03:01.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.460+0000 7f370ea00640 10 client.19722 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:03:01.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.462+0000 7f370ea00640 1 -- 172.21.15.83:0/2386615218 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556f26b44890 con 0x556f26bec4e0 2024-04-24T09:03:01.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.462+0000 7f370ea00640 10 client.19722 handle_command_reply: tid=0 2024-04-24T09:03:01.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.463+0000 7f3732624740 1 client.19722 shutdown 2024-04-24T09:03:01.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.463+0000 7f3732624740 1 -- 172.21.15.83:0/2386615218 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f26bdc080 msgr2=0x556f26bd9a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:01.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.463+0000 7f3732624740 1 --2- 172.21.15.83:0/2386615218 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f26bdc080 0x556f26bd9a40 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f3720010890 tx=0x7f3720004a40 comp rx=0 tx=0).stop 2024-04-24T09:03:01.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.463+0000 7f3732624740 1 -- 172.21.15.83:0/2386615218 shutdown_connections 2024-04-24T09:03:01.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.463+0000 7f3732624740 1 --2- 172.21.15.83:0/2386615218 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556f26bec4e0 0x556f26bee930 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:01.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.463+0000 7f3732624740 1 --2- 172.21.15.83:0/2386615218 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f26bdc080 0x556f26bd9a40 unknown :-1 s=CLOSED pgs=1854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:01.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.463+0000 7f3732624740 1 --2- 172.21.15.83:0/2386615218 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f26bd19b0 0x556f26bd9500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:01.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.463+0000 7f3732624740 1 --2- 172.21.15.83:0/2386615218 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f26bd1090 0x556f26bd8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:01.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.463+0000 7f3732624740 1 -- 172.21.15.83:0/2386615218 >> 172.21.15.83:0/2386615218 conn(0x556f26bcd3b0 msgr2=0x556f26bd31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:01.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.463+0000 7f370ea00640 0 client.19722 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:03:01.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.463+0000 7f3732624740 1 -- 172.21.15.83:0/2386615218 shutdown_connections 2024-04-24T09:03:01.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.463+0000 7f3732624740 1 -- 172.21.15.83:0/2386615218 wait complete. 2024-04-24T09:03:01.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.463+0000 7f3732624740 20 client.19722 trim_cache size 0 max 16384 2024-04-24T09:03:01.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.483+0000 7f3732624740 1 Processor -- start 2024-04-24T09:03:01.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.483+0000 7f3732624740 1 -- start start 2024-04-24T09:03:01.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.484+0000 7f3732624740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f26bd1090 0x556f26be7d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:01.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.484+0000 7f3732624740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f26bd19b0 0x556f26bd3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:01.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.484+0000 7f3732624740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f26bdc080 0x556f26bf4240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:01.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.484+0000 7f3732624740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f37281b1860 con 0x556f26bd1090 2024-04-24T09:03:01.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.484+0000 7f3732624740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f37281b34c0 con 0x556f26bdc080 2024-04-24T09:03:01.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.484+0000 7f3732624740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f37281b1f30 con 0x556f26bd19b0 2024-04-24T09:03:01.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.484+0000 7f372ce00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f26bd1090 0x556f26be7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:01.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.484+0000 7f372ce00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f26bd1090 0x556f26be7d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45664/0 (socket says 172.21.15.83:45664) 2024-04-24T09:03:01.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.484+0000 7f372ce00640 1 -- 172.21.15.83:0/898463422 learned_addr learned my addr 172.21.15.83:0/898463422 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:03:01.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.484+0000 7f372d800640 1 --2- 172.21.15.83:0/898463422 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f26bdc080 0x556f26bf4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:01.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.484+0000 7f3727e00640 1 --2- 172.21.15.83:0/898463422 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f26bd19b0 0x556f26bd3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:01.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.484+0000 7f372ce00640 1 -- 172.21.15.83:0/898463422 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f26bd19b0 msgr2=0x556f26bd3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:01.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.484+0000 7f372ce00640 1 --2- 172.21.15.83:0/898463422 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f26bd19b0 0x556f26bd3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:01.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.484+0000 7f372ce00640 1 -- 172.21.15.83:0/898463422 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f26bdc080 msgr2=0x556f26bf4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:01.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.484+0000 7f372ce00640 1 --2- 172.21.15.83:0/898463422 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f26bdc080 0x556f26bf4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:01.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.484+0000 7f372ce00640 1 -- 172.21.15.83:0/898463422 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f26bd4720 con 0x556f26bd1090 2024-04-24T09:03:01.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.484+0000 7f372d800640 1 --2- 172.21.15.83:0/898463422 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f26bdc080 0x556f26bf4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:03:01.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.485+0000 7f372ce00640 1 --2- 172.21.15.83:0/898463422 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f26bd1090 0x556f26be7d60 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f371c009900 tx=0x7f371c005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0ae72a3d71ad86c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:01.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.485+0000 7f3719600640 1 -- 172.21.15.83:0/898463422 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f371c049070 con 0x556f26bd1090 2024-04-24T09:03:01.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.485+0000 7f3719600640 1 -- 172.21.15.83:0/898463422 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f371c044880 con 0x556f26bd1090 2024-04-24T09:03:01.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.485+0000 7f3719600640 1 -- 172.21.15.83:0/898463422 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f371c035040 con 0x556f26bd1090 2024-04-24T09:03:01.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.486+0000 7f3732624740 1 -- 172.21.15.83:0/898463422 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f26bd1090 msgr2=0x556f26be7d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:01.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.486+0000 7f3732624740 1 --2- 172.21.15.83:0/898463422 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f26bd1090 0x556f26be7d60 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f371c009900 tx=0x7f371c005a20 comp rx=0 tx=0).stop 2024-04-24T09:03:01.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.486+0000 7f3732624740 1 -- 172.21.15.83:0/898463422 shutdown_connections 2024-04-24T09:03:01.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.486+0000 7f3732624740 1 --2- 172.21.15.83:0/898463422 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f26bdc080 0x556f26bf4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:01.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.486+0000 7f3732624740 1 --2- 172.21.15.83:0/898463422 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f26bd19b0 0x556f26bd3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:01.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.486+0000 7f3732624740 1 --2- 172.21.15.83:0/898463422 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f26bd1090 0x556f26be7d60 unknown :-1 s=CLOSED pgs=1855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:01.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.486+0000 7f3732624740 1 -- 172.21.15.83:0/898463422 >> 172.21.15.83:0/898463422 conn(0x556f26bcd3b0 msgr2=0x556f26bcc510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:01.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.486+0000 7f3732624740 1 -- 172.21.15.83:0/898463422 shutdown_connections 2024-04-24T09:03:01.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.486+0000 7f3732624740 1 -- 172.21.15.83:0/898463422 wait complete. 2024-04-24T09:03:01.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.486+0000 7f3732624740 1 -- start start 2024-04-24T09:03:01.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.487+0000 7f3732624740 1 Processor -- start 2024-04-24T09:03:01.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.487+0000 7f3732624740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f26bd1090 0x556f26be4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:01.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.487+0000 7f3732624740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f26bd19b0 0x556f26be47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:01.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.487+0000 7f3732624740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f26bdc080 0x556f26be1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:01.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.487+0000 7f3732624740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f37281b2d90 con 0x556f26bd19b0 2024-04-24T09:03:01.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.487+0000 7f3732624740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f37281b2660 con 0x556f26bdc080 2024-04-24T09:03:01.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.487+0000 7f3732624740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f37281af860 con 0x556f26bd1090 2024-04-24T09:03:01.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.487+0000 7f3732624740 10 client.0 _ll_register_callbacks cb 0x7f3720010560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:03:01.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.487+0000 7f3727e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f26bd19b0 0x556f26be47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:01.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.487+0000 7f3727e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f26bd19b0 0x556f26be47b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45666/0 (socket says 172.21.15.83:45666) 2024-04-24T09:03:01.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.487+0000 7f3727e00640 1 -- 172.21.15.83:0/4138480533 learned_addr learned my addr 172.21.15.83:0/4138480533 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:03:01.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.488+0000 7f372ce00640 1 --2- 172.21.15.83:0/4138480533 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f26bd1090 0x556f26be4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:01.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.488+0000 7f372d800640 1 --2- 172.21.15.83:0/4138480533 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f26bdc080 0x556f26be1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:01.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.488+0000 7f3727e00640 1 -- 172.21.15.83:0/4138480533 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f26bd1090 msgr2=0x556f26be4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:01.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.488+0000 7f3727e00640 1 --2- 172.21.15.83:0/4138480533 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f26bd1090 0x556f26be4270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:01.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.488+0000 7f3727e00640 1 -- 172.21.15.83:0/4138480533 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f26bdc080 msgr2=0x556f26be1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:01.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.488+0000 7f3727e00640 1 --2- 172.21.15.83:0/4138480533 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f26bdc080 0x556f26be1240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:01.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.488+0000 7f3727e00640 1 -- 172.21.15.83:0/4138480533 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f26be1a70 con 0x556f26bd19b0 2024-04-24T09:03:01.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.488+0000 7f3727e00640 1 --2- 172.21.15.83:0/4138480533 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f26bd19b0 0x556f26be47b0 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f3714099090 tx=0x7f3714007f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f854061f476fb477 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:01.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.488+0000 7f370ea00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:03:01.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.488+0000 7f370ea00640 1 -- 172.21.15.83:0/4138480533 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f371409be40 con 0x556f26bd19b0 2024-04-24T09:03:01.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.488+0000 7f370ea00640 1 -- 172.21.15.83:0/4138480533 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9583+0+0 (secure 0 0 0) 0x7f371409a760 con 0x556f26bd19b0 2024-04-24T09:03:01.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.488+0000 7f3732624740 1 -- 172.21.15.83:0/4138480533 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f371c044880 con 0x556f26bd19b0 2024-04-24T09:03:01.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.489+0000 7f370ea00640 1 -- 172.21.15.83:0/4138480533 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f371409c020 con 0x556f26bd19b0 2024-04-24T09:03:01.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.489+0000 7f3732624740 10 client.19728 fetch_fsmap learned FSMap version 41 2024-04-24T09:03:01.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.489+0000 7f3732624740 1 -- 172.21.15.83:0/4138480533 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x556f26c53590 con 0x556f26bd19b0 2024-04-24T09:03:01.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.489+0000 7f370ea00640 1 -- 172.21.15.83:0/4138480533 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f371c044880 con 0x556f26bd19b0 2024-04-24T09:03:01.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.489+0000 7f370ea00640 1 -- 172.21.15.83:0/4138480533 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7f371409bb10 con 0x556f26bd19b0 2024-04-24T09:03:01.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.489+0000 7f3732624740 10 client.19728 fetch_fsmap finished waiting for FSMap version 41 2024-04-24T09:03:01.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.489+0000 7f3732624740 10 client.19728 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:03:01.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.489+0000 7f3732624740 20 client.19728 populate_metadata read hostname 'smithi083' 2024-04-24T09:03:01.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.489+0000 7f3732624740 1 --2- 172.21.15.83:0/4138480533 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556f26c53910 0x556f26c55d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:01.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.489+0000 7f3732624740 4 client.19728 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T09:03:01.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.489+0000 7f3732624740 1 -- 172.21.15.83:0/4138480533 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x556f26b3e700 con 0x556f26c53910 2024-04-24T09:03:01.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.490+0000 7f372ce00640 1 --2- 172.21.15.83:0/4138480533 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556f26c53910 0x556f26c55d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:01.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.491+0000 7f372ce00640 1 --2- 172.21.15.83:0/4138480533 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556f26c53910 0x556f26c55d30 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5efc810cab45652 server_cookie=8161cfb4f391b7ee in_seq=0 out_seq=0 2024-04-24T09:03:01.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.491+0000 7f370ea00640 10 client.19728 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:03:01.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.494+0000 7f370ea00640 1 -- 172.21.15.83:0/4138480533 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x556f26b3e700 con 0x556f26c53910 2024-04-24T09:03:01.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.494+0000 7f370ea00640 10 client.19728 handle_command_reply: tid=0 2024-04-24T09:03:01.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.494+0000 7f3732624740 1 client.19728 shutdown 2024-04-24T09:03:01.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.494+0000 7f3732624740 1 -- 172.21.15.83:0/4138480533 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f26bd19b0 msgr2=0x556f26be47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:01.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.494+0000 7f3732624740 1 --2- 172.21.15.83:0/4138480533 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f26bd19b0 0x556f26be47b0 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f3714099090 tx=0x7f3714007f30 comp rx=0 tx=0).stop 2024-04-24T09:03:01.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.494+0000 7f3732624740 1 -- 172.21.15.83:0/4138480533 shutdown_connections 2024-04-24T09:03:01.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.494+0000 7f3732624740 1 --2- 172.21.15.83:0/4138480533 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556f26c53910 0x556f26c55d30 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:01.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.494+0000 7f3732624740 1 --2- 172.21.15.83:0/4138480533 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f26bdc080 0x556f26be1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:01.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.495+0000 7f3732624740 1 --2- 172.21.15.83:0/4138480533 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f26bd19b0 0x556f26be47b0 unknown :-1 s=CLOSED pgs=1856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:01.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.495+0000 7f3732624740 1 --2- 172.21.15.83:0/4138480533 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f26bd1090 0x556f26be4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:01.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.495+0000 7f3732624740 1 -- 172.21.15.83:0/4138480533 >> 172.21.15.83:0/4138480533 conn(0x556f26bcd3b0 msgr2=0x556f26bd8f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:01.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.495+0000 7f370ea00640 0 client.19728 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:03:01.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.495+0000 7f3732624740 1 -- 172.21.15.83:0/4138480533 shutdown_connections 2024-04-24T09:03:01.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.495+0000 7f3732624740 1 -- 172.21.15.83:0/4138480533 wait complete. 2024-04-24T09:03:01.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.495+0000 7f3732624740 20 client.19728 trim_cache size 0 max 16384 2024-04-24T09:03:01.495 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T09:03:01.495 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-04-24T09:03:01.495 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "ffdc5ee0-3237-49bf-acce-c50d21c8100f", 2024-04-24T09:03:01.495 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-04-24T09:03:01.496 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T09:03:01.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.496+0000 7f372f800640 1 -- 172.21.15.83:0/3180241918 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f37100a3ec0 msgr2=0x7f37100a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:01.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.496+0000 7f372f800640 1 --2- 172.21.15.83:0/3180241918 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f37100a3ec0 0x7f37100a6380 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f3720005c40 tx=0x7f3720009760 comp rx=0 tx=0).stop 2024-04-24T09:03:01.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.496+0000 7f372f800640 1 -- 172.21.15.83:0/3180241918 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3728107360 msgr2=0x7f3728074840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:01.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.496+0000 7f372f800640 1 --2- 172.21.15.83:0/3180241918 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3728107360 0x7f3728074840 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f371400c970 tx=0x7f371400ce30 comp rx=0 tx=0).stop 2024-04-24T09:03:01.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.496+0000 7f372f800640 1 -- 172.21.15.83:0/3180241918 shutdown_connections 2024-04-24T09:03:01.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.496+0000 7f372f800640 1 --2- 172.21.15.83:0/3180241918 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f37100a3ec0 0x7f37100a6380 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:01.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.496+0000 7f372f800640 1 --2- 172.21.15.83:0/3180241918 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3728074d80 0x7f37281a6a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:01.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.496+0000 7f372f800640 1 --2- 172.21.15.83:0/3180241918 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3728107360 0x7f3728074840 unknown :-1 s=CLOSED pgs=1852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:01.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.496+0000 7f372f800640 1 --2- 172.21.15.83:0/3180241918 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3728102290 0x7f3728074300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:01.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.496+0000 7f372f800640 1 -- 172.21.15.83:0/3180241918 >> 172.21.15.83:0/3180241918 conn(0x7f37280fdb10 msgr2=0x7f37280fef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:01.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.496+0000 7f372f800640 1 -- 172.21.15.83:0/3180241918 shutdown_connections 2024-04-24T09:03:01.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.496+0000 7f372f800640 1 -- 172.21.15.83:0/3180241918 wait complete. 2024-04-24T09:03:01.513 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T09:03:01.513 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T09:03:01.616 INFO:tasks.workunit.client.0.smithi083.stdout: 1 66466 0.61 MB/sec execute 455 sec latency 52.268 ms 2024-04-24T09:03:01.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.719+0000 7f0513e00640 1 -- 172.21.15.83:0/3405066641 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f050c101990 msgr2=0x7f050c105e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:01.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.719+0000 7f0513e00640 1 --2- 172.21.15.83:0/3405066641 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f050c101990 0x7f050c105e50 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f04f8009900 tx=0x7f04f802a510 comp rx=0 tx=0).stop 2024-04-24T09:03:01.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.719+0000 7f0513e00640 1 -- 172.21.15.83:0/3405066641 shutdown_connections 2024-04-24T09:03:01.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.719+0000 7f0513e00640 1 --2- 172.21.15.83:0/3405066641 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f050c106390 0x7f050c106820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:01.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.719+0000 7f0513e00640 1 --2- 172.21.15.83:0/3405066641 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f050c101990 0x7f050c105e50 unknown :-1 s=CLOSED pgs=1857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:01.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.719+0000 7f0513e00640 1 --2- 172.21.15.83:0/3405066641 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f050c101070 0x7f050c101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:01.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.719+0000 7f0513e00640 1 -- 172.21.15.83:0/3405066641 >> 172.21.15.83:0/3405066641 conn(0x7f050c078000 msgr2=0x7f050c078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:01.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.719+0000 7f0513e00640 1 -- 172.21.15.83:0/3405066641 shutdown_connections 2024-04-24T09:03:01.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.719+0000 7f0513e00640 1 -- 172.21.15.83:0/3405066641 wait complete. 2024-04-24T09:03:01.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.720+0000 7f0513e00640 1 Processor -- start 2024-04-24T09:03:01.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.720+0000 7f0513e00640 1 -- start start 2024-04-24T09:03:01.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.720+0000 7f0513e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f050c101070 0x7f050c19eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:01.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.721+0000 7f0513e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f050c101990 0x7f050c19f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:01.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.721+0000 7f0513e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f050c106390 0x7f050c1a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:01.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.721+0000 7f0513e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f050c118870 con 0x7f050c106390 2024-04-24T09:03:01.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.721+0000 7f0513e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f050c1186f0 con 0x7f050c101070 2024-04-24T09:03:01.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.721+0000 7f0513e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f050c1189f0 con 0x7f050c101990 2024-04-24T09:03:01.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.721+0000 7f0511e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f050c106390 0x7f050c1a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:01.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.721+0000 7f0511e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f050c106390 0x7f050c1a3770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45676/0 (socket says 172.21.15.83:45676) 2024-04-24T09:03:01.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.721+0000 7f0511e00640 1 -- 172.21.15.83:0/2317408198 learned_addr learned my addr 172.21.15.83:0/2317408198 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:03:01.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.721+0000 7f0511400640 1 --2- 172.21.15.83:0/2317408198 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f050c101070 0x7f050c19eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:01.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.721+0000 7f0510a00640 1 --2- 172.21.15.83:0/2317408198 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f050c101990 0x7f050c19f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:01.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.721+0000 7f0511e00640 1 -- 172.21.15.83:0/2317408198 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f050c101990 msgr2=0x7f050c19f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:01.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.721+0000 7f0511e00640 1 --2- 172.21.15.83:0/2317408198 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f050c101990 0x7f050c19f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:01.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.721+0000 7f0511e00640 1 -- 172.21.15.83:0/2317408198 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f050c101070 msgr2=0x7f050c19eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:01.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.721+0000 7f0511e00640 1 --2- 172.21.15.83:0/2317408198 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f050c101070 0x7f050c19eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:01.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.721+0000 7f0511e00640 1 -- 172.21.15.83:0/2317408198 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f050c1a3ea0 con 0x7f050c106390 2024-04-24T09:03:01.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.721+0000 7f0511400640 1 --2- 172.21.15.83:0/2317408198 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f050c101070 0x7f050c19eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:03:01.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.722+0000 7f0511e00640 1 --2- 172.21.15.83:0/2317408198 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f050c106390 0x7f050c1a3770 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f050400e970 tx=0x7f050400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:01.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.722+0000 7f050a000640 1 -- 172.21.15.83:0/2317408198 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f050400ccc0 con 0x7f050c106390 2024-04-24T09:03:01.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.722+0000 7f050a000640 1 -- 172.21.15.83:0/2317408198 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f050400ce60 con 0x7f050c106390 2024-04-24T09:03:01.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.722+0000 7f050a000640 1 -- 172.21.15.83:0/2317408198 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0504010430 con 0x7f050c106390 2024-04-24T09:03:01.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.722+0000 7f0513e00640 1 -- 172.21.15.83:0/2317408198 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f050c1a4190 con 0x7f050c106390 2024-04-24T09:03:01.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.722+0000 7f0513e00640 1 -- 172.21.15.83:0/2317408198 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f050c1aba20 con 0x7f050c106390 2024-04-24T09:03:01.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.726+0000 7f0516d9f740 1 Processor -- start 2024-04-24T09:03:01.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.726+0000 7f050a000640 1 -- 172.21.15.83:0/2317408198 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0504004ff0 con 0x7f050c106390 2024-04-24T09:03:01.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.726+0000 7f0516d9f740 1 -- start start 2024-04-24T09:03:01.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.726+0000 7f0516d9f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db597d3860 0x55db597d3c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:01.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.726+0000 7f0516d9f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55db597d4180 0x55db597de310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:01.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.727+0000 7f0516d9f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db597de850 0x55db597e0c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:01.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.727+0000 7f0516d9f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55db597391f0 con 0x55db597de850 2024-04-24T09:03:01.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.727+0000 7f0516d9f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55db596df350 con 0x55db597d3860 2024-04-24T09:03:01.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.727+0000 7f0516d9f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55db597351b0 con 0x55db597d4180 2024-04-24T09:03:01.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.727+0000 7f0510a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db597de850 0x55db597e0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:01.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.727+0000 7f0510a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db597de850 0x55db597e0c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45678/0 (socket says 172.21.15.83:45678) 2024-04-24T09:03:01.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.727+0000 7f0510a00640 1 -- 172.21.15.83:0/1186974151 learned_addr learned my addr 172.21.15.83:0/1186974151 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:03:01.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.727+0000 7f0511400640 1 --2- 172.21.15.83:0/1186974151 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55db597d4180 0x55db597de310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:01.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.727+0000 7f0510a00640 1 --2- 172.21.15.83:0/1186974151 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db597d3860 0x55db597d3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:01.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.727+0000 7f0510a00640 1 -- 172.21.15.83:0/1186974151 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55db597d4180 msgr2=0x55db597de310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:01.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.727+0000 7f0510a00640 1 --2- 172.21.15.83:0/1186974151 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55db597d4180 0x55db597de310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:01.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.727+0000 7f0510a00640 1 -- 172.21.15.83:0/1186974151 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db597d3860 msgr2=0x55db597d3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:01.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.727+0000 7f0510a00640 1 --2- 172.21.15.83:0/1186974151 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db597d3860 0x55db597d3c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:01.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.728+0000 7f0510a00640 1 -- 172.21.15.83:0/1186974151 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db59733f50 con 0x55db597de850 2024-04-24T09:03:01.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.728+0000 7f050a000640 1 --2- 172.21.15.83:0/2317408198 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f04f40a3da0 0x7f04f40a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:01.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.728+0000 7f0510a00640 1 --2- 172.21.15.83:0/1186974151 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db597de850 0x55db597e0c40 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7f04f8009900 tx=0x7f04f8005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee6aba8f0854e107 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:01.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.728+0000 7f050a000640 1 -- 172.21.15.83:0/2317408198 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9583+0+0 (secure 0 0 0) 0x7f05040d10f0 con 0x7f050c106390 2024-04-24T09:03:01.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.728+0000 7f0511400640 1 --2- 172.21.15.83:0/2317408198 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f04f40a3da0 0x7f04f40a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:01.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.728+0000 7f04fe600640 1 -- 172.21.15.83:0/1186974151 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f8043020 con 0x55db597de850 2024-04-24T09:03:01.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.728+0000 7f04fe600640 1 -- 172.21.15.83:0/1186974151 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f04f80316a0 con 0x55db597de850 2024-04-24T09:03:01.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.728+0000 7f04fe600640 1 -- 172.21.15.83:0/1186974151 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f8035040 con 0x55db597de850 2024-04-24T09:03:01.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.729+0000 7f0516d9f740 1 -- 172.21.15.83:0/1186974151 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db597de850 msgr2=0x55db597e0c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:01.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.729+0000 7f0511400640 1 --2- 172.21.15.83:0/2317408198 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f04f40a3da0 0x7f04f40a6260 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x55db597d4e20 tx=0x7f0500009500 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:01.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.729+0000 7f0516d9f740 1 --2- 172.21.15.83:0/1186974151 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db597de850 0x55db597e0c40 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7f04f8009900 tx=0x7f04f8005b50 comp rx=0 tx=0).stop 2024-04-24T09:03:01.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.730+0000 7f0516d9f740 1 -- 172.21.15.83:0/1186974151 shutdown_connections 2024-04-24T09:03:01.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.730+0000 7f0516d9f740 1 --2- 172.21.15.83:0/1186974151 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db597de850 0x55db597e0c40 unknown :-1 s=CLOSED pgs=1859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:01.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.730+0000 7f0516d9f740 1 --2- 172.21.15.83:0/1186974151 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55db597d4180 0x55db597de310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:01.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.730+0000 7f0516d9f740 1 --2- 172.21.15.83:0/1186974151 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db597d3860 0x55db597d3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:01.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.730+0000 7f0516d9f740 1 -- 172.21.15.83:0/1186974151 >> 172.21.15.83:0/1186974151 conn(0x55db597cfb80 msgr2=0x55db597cff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:01.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.730+0000 7f0516d9f740 1 -- 172.21.15.83:0/1186974151 shutdown_connections 2024-04-24T09:03:01.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.730+0000 7f0516d9f740 1 -- 172.21.15.83:0/1186974151 wait complete. 2024-04-24T09:03:01.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.730+0000 7f0516d9f740 1 -- start start 2024-04-24T09:03:01.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.730+0000 7f0516d9f740 1 Processor -- start 2024-04-24T09:03:01.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.731+0000 7f0516d9f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55db597d3860 0x55db597db7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:01.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.731+0000 7f0516d9f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db597d4180 0x55db597dbd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:01.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.731+0000 7f0516d9f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db597de850 0x55db597dc250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:01.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.731+0000 7f0516d9f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55db597e15e0 con 0x55db597de850 2024-04-24T09:03:01.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.731+0000 7f0516d9f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55db596f2690 con 0x55db597d4180 2024-04-24T09:03:01.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.731+0000 7f0516d9f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55db597e1760 con 0x55db597d3860 2024-04-24T09:03:01.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.731+0000 7f0516d9f740 10 client.0 _ll_register_callbacks cb 0x55db5972e650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:03:01.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.731+0000 7f0511e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db597de850 0x55db597dc250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:01.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.731+0000 7f0511e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db597de850 0x55db597dc250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45686/0 (socket says 172.21.15.83:45686) 2024-04-24T09:03:01.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.731+0000 7f0511e00640 1 -- 172.21.15.83:0/2154104610 learned_addr learned my addr 172.21.15.83:0/2154104610 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:03:01.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.731+0000 7f0511400640 1 --2- 172.21.15.83:0/2154104610 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55db597d3860 0x55db597db7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:01.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.731+0000 7f0510a00640 1 --2- 172.21.15.83:0/2154104610 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db597d4180 0x55db597dbd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:01.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.732+0000 7f0511e00640 1 -- 172.21.15.83:0/2154104610 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55db597d3860 msgr2=0x55db597db7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:01.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.732+0000 7f0511e00640 1 --2- 172.21.15.83:0/2154104610 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55db597d3860 0x55db597db7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:01.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.732+0000 7f0511e00640 1 -- 172.21.15.83:0/2154104610 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db597d4180 msgr2=0x55db597dbd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:01.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.732+0000 7f0511e00640 1 --2- 172.21.15.83:0/2154104610 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db597d4180 0x55db597dbd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:01.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.732+0000 7f0511e00640 1 -- 172.21.15.83:0/2154104610 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db596f1920 con 0x55db597de850 2024-04-24T09:03:01.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.732+0000 7f0511400640 1 --2- 172.21.15.83:0/2154104610 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55db597d3860 0x55db597db7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:03:01.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.732+0000 7f0511e00640 1 --2- 172.21.15.83:0/2154104610 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db597de850 0x55db597dc250 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7f05040a5350 tx=0x7f050409c630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff3439eafebfd6d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:01.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.732+0000 7f04f2a00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:03:01.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.732+0000 7f04f2a00640 1 -- 172.21.15.83:0/2154104610 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05040a0fc0 con 0x55db597de850 2024-04-24T09:03:01.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.732+0000 7f04f2a00640 1 -- 172.21.15.83:0/2154104610 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9583+0+0 (secure 0 0 0) 0x7f05040052e0 con 0x55db597de850 2024-04-24T09:03:01.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.732+0000 7f0516d9f740 1 -- 172.21.15.83:0/2154104610 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04f80316a0 con 0x55db597de850 2024-04-24T09:03:01.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.732+0000 7f04f2a00640 1 -- 172.21.15.83:0/2154104610 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f05040a5880 con 0x55db597de850 2024-04-24T09:03:01.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.732+0000 7f0516d9f740 10 client.19740 fetch_fsmap learned FSMap version 41 2024-04-24T09:03:01.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.732+0000 7f0516d9f740 1 -- 172.21.15.83:0/2154104610 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55db595157d0 con 0x55db597de850 2024-04-24T09:03:01.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.733+0000 7f04f2a00640 1 -- 172.21.15.83:0/2154104610 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f04f80316a0 con 0x55db597de850 2024-04-24T09:03:01.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.733+0000 7f04f2a00640 1 -- 172.21.15.83:0/2154104610 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7f05040a53f0 con 0x55db597de850 2024-04-24T09:03:01.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.733+0000 7f0516d9f740 10 client.19740 fetch_fsmap finished waiting for FSMap version 41 2024-04-24T09:03:01.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.733+0000 7f0516d9f740 10 client.19740 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:03:01.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.733+0000 7f0516d9f740 20 client.19740 populate_metadata read hostname 'smithi083' 2024-04-24T09:03:01.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.733+0000 7f0516d9f740 1 --2- 172.21.15.83:0/2154104610 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55db597eec30 0x55db597f10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:01.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.733+0000 7f0516d9f740 4 client.19740 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:03:01.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.733+0000 7f0516d9f740 1 -- 172.21.15.83:0/2154104610 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55db59746c10 con 0x55db597eec30 2024-04-24T09:03:01.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.734+0000 7f0511400640 1 --2- 172.21.15.83:0/2154104610 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55db597eec30 0x55db597f10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:01.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.735+0000 7f0511400640 1 --2- 172.21.15.83:0/2154104610 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55db597eec30 0x55db597f10a0 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ffcf6b7315f9a02 server_cookie=c1c3b74a296130a2 in_seq=0 out_seq=0 2024-04-24T09:03:01.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.735+0000 7f04f2a00640 10 client.19740 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:03:01.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.737+0000 7f04f2a00640 1 -- 172.21.15.83:0/2154104610 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55db59746c10 con 0x55db597eec30 2024-04-24T09:03:01.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.737+0000 7f04f2a00640 10 client.19740 handle_command_reply: tid=0 2024-04-24T09:03:01.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.737+0000 7f0516d9f740 1 client.19740 shutdown 2024-04-24T09:03:01.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.737+0000 7f0516d9f740 1 -- 172.21.15.83:0/2154104610 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db597de850 msgr2=0x55db597dc250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:01.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.737+0000 7f0516d9f740 1 --2- 172.21.15.83:0/2154104610 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db597de850 0x55db597dc250 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7f05040a5350 tx=0x7f050409c630 comp rx=0 tx=0).stop 2024-04-24T09:03:01.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.737+0000 7f0516d9f740 1 -- 172.21.15.83:0/2154104610 shutdown_connections 2024-04-24T09:03:01.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.737+0000 7f0516d9f740 1 --2- 172.21.15.83:0/2154104610 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55db597eec30 0x55db597f10a0 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:01.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.737+0000 7f0516d9f740 1 --2- 172.21.15.83:0/2154104610 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db597de850 0x55db597dc250 unknown :-1 s=CLOSED pgs=1860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:01.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.737+0000 7f0516d9f740 1 --2- 172.21.15.83:0/2154104610 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db597d4180 0x55db597dbd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:01.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.737+0000 7f0516d9f740 1 --2- 172.21.15.83:0/2154104610 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55db597d3860 0x55db597db7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:01.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.737+0000 7f0516d9f740 1 -- 172.21.15.83:0/2154104610 >> 172.21.15.83:0/2154104610 conn(0x55db597cfb80 msgr2=0x55db597d5980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:01.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.737+0000 7f04f2a00640 0 client.19740 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:03:01.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.737+0000 7f0516d9f740 1 -- 172.21.15.83:0/2154104610 shutdown_connections 2024-04-24T09:03:01.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.738+0000 7f0516d9f740 1 -- 172.21.15.83:0/2154104610 wait complete. 2024-04-24T09:03:01.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.738+0000 7f0516d9f740 20 client.19740 trim_cache size 0 max 16384 2024-04-24T09:03:01.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.758+0000 7f0516d9f740 1 Processor -- start 2024-04-24T09:03:01.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.758+0000 7f0516d9f740 1 -- start start 2024-04-24T09:03:01.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.758+0000 7f0516d9f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db597d3860 0x55db597ea4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:01.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.758+0000 7f0516d9f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55db597d4180 0x55db597da930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:01.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.758+0000 7f0516d9f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db597de850 0x55db597d5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:01.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.758+0000 7f0516d9f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f050c1b5b70 con 0x55db597de850 2024-04-24T09:03:01.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.758+0000 7f0516d9f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f050c1b77d0 con 0x55db597d3860 2024-04-24T09:03:01.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.758+0000 7f0516d9f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f050c1b6240 con 0x55db597d4180 2024-04-24T09:03:01.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.759+0000 7f0511e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db597de850 0x55db597d5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:01.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.759+0000 7f0511e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db597de850 0x55db597d5fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45698/0 (socket says 172.21.15.83:45698) 2024-04-24T09:03:01.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.759+0000 7f0511e00640 1 -- 172.21.15.83:0/2561645534 learned_addr learned my addr 172.21.15.83:0/2561645534 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:03:01.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.759+0000 7f0510a00640 1 --2- 172.21.15.83:0/2561645534 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55db597d4180 0x55db597da930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:01.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.759+0000 7f0511400640 1 --2- 172.21.15.83:0/2561645534 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db597d3860 0x55db597ea4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:01.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.759+0000 7f0511e00640 1 -- 172.21.15.83:0/2561645534 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55db597d4180 msgr2=0x55db597da930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:01.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.759+0000 7f0511e00640 1 --2- 172.21.15.83:0/2561645534 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55db597d4180 0x55db597da930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:01.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.759+0000 7f0511e00640 1 -- 172.21.15.83:0/2561645534 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db597d3860 msgr2=0x55db597ea4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:01.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.759+0000 7f0511e00640 1 --2- 172.21.15.83:0/2561645534 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db597d3860 0x55db597ea4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:01.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.759+0000 7f0511e00640 1 -- 172.21.15.83:0/2561645534 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db5944b3f0 con 0x55db597de850 2024-04-24T09:03:01.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.759+0000 7f0510a00640 1 --2- 172.21.15.83:0/2561645534 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55db597d4180 0x55db597da930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:03:01.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.759+0000 7f0511e00640 1 --2- 172.21.15.83:0/2561645534 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db597de850 0x55db597d5fc0 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7f05040a5a40 tx=0x7f050409b120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fba8282fcb300226 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:01.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.760+0000 7f04fdc00640 1 -- 172.21.15.83:0/2561645534 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f050409c9e0 con 0x55db597de850 2024-04-24T09:03:01.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.760+0000 7f04fdc00640 1 -- 172.21.15.83:0/2561645534 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f05040a0fc0 con 0x55db597de850 2024-04-24T09:03:01.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.760+0000 7f04fdc00640 1 -- 172.21.15.83:0/2561645534 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05040052e0 con 0x55db597de850 2024-04-24T09:03:01.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.760+0000 7f0516d9f740 1 -- 172.21.15.83:0/2561645534 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db597de850 msgr2=0x55db597d5fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:01.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.760+0000 7f0516d9f740 1 --2- 172.21.15.83:0/2561645534 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db597de850 0x55db597d5fc0 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7f05040a5a40 tx=0x7f050409b120 comp rx=0 tx=0).stop 2024-04-24T09:03:01.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.760+0000 7f0516d9f740 1 -- 172.21.15.83:0/2561645534 shutdown_connections 2024-04-24T09:03:01.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.760+0000 7f0516d9f740 1 --2- 172.21.15.83:0/2561645534 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db597de850 0x55db597d5fc0 unknown :-1 s=CLOSED pgs=1861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:01.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.760+0000 7f0516d9f740 1 --2- 172.21.15.83:0/2561645534 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55db597d4180 0x55db597da930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:01.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.760+0000 7f0516d9f740 1 --2- 172.21.15.83:0/2561645534 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db597d3860 0x55db597ea4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:01.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.760+0000 7f0516d9f740 1 -- 172.21.15.83:0/2561645534 >> 172.21.15.83:0/2561645534 conn(0x55db597cfb80 msgr2=0x55db597ce390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:01.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.761+0000 7f0516d9f740 1 -- 172.21.15.83:0/2561645534 shutdown_connections 2024-04-24T09:03:01.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.761+0000 7f0516d9f740 1 -- 172.21.15.83:0/2561645534 wait complete. 2024-04-24T09:03:01.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.761+0000 7f0516d9f740 1 -- start start 2024-04-24T09:03:01.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.761+0000 7f0516d9f740 1 Processor -- start 2024-04-24T09:03:01.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.762+0000 7f0516d9f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55db597d3860 0x55db597e19a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:01.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.762+0000 7f0516d9f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db597d4180 0x55db597e1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:01.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.762+0000 7f0516d9f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db597de850 0x55db59853180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:01.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.762+0000 7f0516d9f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f050c1b70a0 con 0x55db597de850 2024-04-24T09:03:01.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.762+0000 7f0516d9f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f050c1b6970 con 0x55db597d4180 2024-04-24T09:03:01.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.762+0000 7f0516d9f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f050c1b3b70 con 0x55db597d3860 2024-04-24T09:03:01.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.762+0000 7f0516d9f740 10 client.0 _ll_register_callbacks cb 0x7f0504004840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:03:01.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.762+0000 7f0511e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db597de850 0x55db59853180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:01.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.762+0000 7f0511e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db597de850 0x55db59853180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45706/0 (socket says 172.21.15.83:45706) 2024-04-24T09:03:01.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.762+0000 7f0511e00640 1 -- 172.21.15.83:0/1575089303 learned_addr learned my addr 172.21.15.83:0/1575089303 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:03:01.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.762+0000 7f0511400640 1 --2- 172.21.15.83:0/1575089303 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55db597d3860 0x55db597e19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:01.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.762+0000 7f0510a00640 1 --2- 172.21.15.83:0/1575089303 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db597d4180 0x55db597e1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:01.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.762+0000 7f0511e00640 1 -- 172.21.15.83:0/1575089303 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55db597d3860 msgr2=0x55db597e19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:01.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.762+0000 7f0511e00640 1 --2- 172.21.15.83:0/1575089303 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55db597d3860 0x55db597e19a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:01.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.762+0000 7f0511e00640 1 -- 172.21.15.83:0/1575089303 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db597d4180 msgr2=0x55db597e1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:01.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.762+0000 7f0511e00640 1 --2- 172.21.15.83:0/1575089303 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db597d4180 0x55db597e1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:01.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.762+0000 7f0511e00640 1 -- 172.21.15.83:0/1575089303 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db5940b0f0 con 0x55db597de850 2024-04-24T09:03:01.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.762+0000 7f0510a00640 1 --2- 172.21.15.83:0/1575089303 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db597d4180 0x55db597e1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:03:01.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.763+0000 7f0511e00640 1 --2- 172.21.15.83:0/1575089303 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db597de850 0x55db59853180 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7f05040a5350 tx=0x7f0504005510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cff83b972442e476 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:01.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.763+0000 7f04f2a00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:03:01.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.763+0000 7f04f2a00640 1 -- 172.21.15.83:0/1575089303 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f050409c9e0 con 0x55db597de850 2024-04-24T09:03:01.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.763+0000 7f04f2a00640 1 -- 172.21.15.83:0/1575089303 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9583+0+0 (secure 0 0 0) 0x7f050409c260 con 0x55db597de850 2024-04-24T09:03:01.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.763+0000 7f0516d9f740 1 -- 172.21.15.83:0/1575089303 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05040a0fc0 con 0x55db597de850 2024-04-24T09:03:01.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.763+0000 7f04f2a00640 1 -- 172.21.15.83:0/1575089303 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f0504099420 con 0x55db597de850 2024-04-24T09:03:01.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.763+0000 7f0516d9f740 10 client.19746 fetch_fsmap learned FSMap version 41 2024-04-24T09:03:01.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.763+0000 7f0516d9f740 1 -- 172.21.15.83:0/1575089303 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55db596fe810 con 0x55db597de850 2024-04-24T09:03:01.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.763+0000 7f04f2a00640 1 -- 172.21.15.83:0/1575089303 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f05040a0fc0 con 0x55db597de850 2024-04-24T09:03:01.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.764+0000 7f04f2a00640 1 -- 172.21.15.83:0/1575089303 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7f050409bad0 con 0x55db597de850 2024-04-24T09:03:01.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.764+0000 7f0516d9f740 10 client.19746 fetch_fsmap finished waiting for FSMap version 41 2024-04-24T09:03:01.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.764+0000 7f0516d9f740 10 client.19746 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:03:01.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.764+0000 7f0516d9f740 20 client.19746 populate_metadata read hostname 'smithi083' 2024-04-24T09:03:01.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.764+0000 7f0516d9f740 1 --2- 172.21.15.83:0/1575089303 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55db59854910 0x55db59856d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:01.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.764+0000 7f0516d9f740 4 client.19746 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T09:03:01.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.764+0000 7f0516d9f740 1 -- 172.21.15.83:0/1575089303 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55db59740a80 con 0x55db59854910 2024-04-24T09:03:01.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.764+0000 7f0511400640 1 --2- 172.21.15.83:0/1575089303 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55db59854910 0x55db59856d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:01.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.765+0000 7f0511400640 1 --2- 172.21.15.83:0/1575089303 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55db59854910 0x55db59856d30 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8e461d670186a0f server_cookie=39f14ae93915e2bb in_seq=0 out_seq=0 2024-04-24T09:03:01.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.765+0000 7f04f2a00640 10 client.19746 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:03:01.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.769+0000 7f04f2a00640 1 -- 172.21.15.83:0/1575089303 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55db59740a80 con 0x55db59854910 2024-04-24T09:03:01.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.769+0000 7f04f2a00640 10 client.19746 handle_command_reply: tid=0 2024-04-24T09:03:01.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.769+0000 7f0516d9f740 1 client.19746 shutdown 2024-04-24T09:03:01.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.769+0000 7f0516d9f740 1 -- 172.21.15.83:0/1575089303 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db597de850 msgr2=0x55db59853180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:01.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.769+0000 7f0516d9f740 1 --2- 172.21.15.83:0/1575089303 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db597de850 0x55db59853180 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7f05040a5350 tx=0x7f0504005510 comp rx=0 tx=0).stop 2024-04-24T09:03:01.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.769+0000 7f0516d9f740 1 -- 172.21.15.83:0/1575089303 shutdown_connections 2024-04-24T09:03:01.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.769+0000 7f0516d9f740 1 --2- 172.21.15.83:0/1575089303 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55db59854910 0x55db59856d30 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:01.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.769+0000 7f0516d9f740 1 --2- 172.21.15.83:0/1575089303 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db597de850 0x55db59853180 unknown :-1 s=CLOSED pgs=1862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:01.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.769+0000 7f0516d9f740 1 --2- 172.21.15.83:0/1575089303 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db597d4180 0x55db597e1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:01.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.769+0000 7f0516d9f740 1 --2- 172.21.15.83:0/1575089303 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55db597d3860 0x55db597e19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:01.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.769+0000 7f0516d9f740 1 -- 172.21.15.83:0/1575089303 >> 172.21.15.83:0/1575089303 conn(0x55db597cfb80 msgr2=0x55db597db790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:01.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.769+0000 7f04f2a00640 0 client.19746 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:03:01.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.769+0000 7f0516d9f740 1 -- 172.21.15.83:0/1575089303 shutdown_connections 2024-04-24T09:03:01.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.769+0000 7f0516d9f740 1 -- 172.21.15.83:0/1575089303 wait complete. 2024-04-24T09:03:01.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.769+0000 7f0516d9f740 20 client.19746 trim_cache size 0 max 16384 2024-04-24T09:03:01.769 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T09:03:01.769 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T09:03:01.769 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-04-24T09:03:01.770 INFO:teuthology.orchestra.run.smithi083.stdout: "ffdc5ee0-3237-49bf-acce-c50d21c8100f": { 2024-04-24T09:03:01.770 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-04-24T09:03:01.770 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "ffdc5ee0-3237-49bf-acce-c50d21c8100f", 2024-04-24T09:03:01.770 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-04-24T09:03:01.770 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:03:01.770 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:03:01.770 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T09:03:01.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.770+0000 7f0513e00640 1 -- 172.21.15.83:0/2317408198 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f04f40a3da0 msgr2=0x7f04f40a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:01.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.770+0000 7f0513e00640 1 --2- 172.21.15.83:0/2317408198 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f04f40a3da0 0x7f04f40a6260 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x55db597d4e20 tx=0x7f0500009500 comp rx=0 tx=0).stop 2024-04-24T09:03:01.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.770+0000 7f0513e00640 1 -- 172.21.15.83:0/2317408198 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f050c106390 msgr2=0x7f050c1a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:01.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.770+0000 7f0513e00640 1 --2- 172.21.15.83:0/2317408198 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f050c106390 0x7f050c1a3770 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f050400e970 tx=0x7f050400ed70 comp rx=0 tx=0).stop 2024-04-24T09:03:01.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.770+0000 7f0513e00640 1 -- 172.21.15.83:0/2317408198 shutdown_connections 2024-04-24T09:03:01.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.770+0000 7f0513e00640 1 --2- 172.21.15.83:0/2317408198 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f04f40a3da0 0x7f04f40a6260 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:01.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.770+0000 7f0513e00640 1 --2- 172.21.15.83:0/2317408198 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f050c106390 0x7f050c1a3770 unknown :-1 s=CLOSED pgs=1858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:01.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.770+0000 7f0513e00640 1 --2- 172.21.15.83:0/2317408198 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f050c101990 0x7f050c19f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:01.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.770+0000 7f0513e00640 1 --2- 172.21.15.83:0/2317408198 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f050c101070 0x7f050c19eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:01.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.770+0000 7f0513e00640 1 -- 172.21.15.83:0/2317408198 >> 172.21.15.83:0/2317408198 conn(0x7f050c078000 msgr2=0x7f050c103e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:01.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.770+0000 7f0513e00640 1 -- 172.21.15.83:0/2317408198 shutdown_connections 2024-04-24T09:03:01.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:01.770+0000 7f0513e00640 1 -- 172.21.15.83:0/2317408198 wait complete. 2024-04-24T09:03:01.788 INFO:tasks.cephfs.filesystem:scrub status for tag:ffdc5ee0-3237-49bf-acce-c50d21c8100f - {'path': '/', 'tag': 'ffdc5ee0-3237-49bf-acce-c50d21c8100f', 'options': 'recursive,force'} 2024-04-24T09:03:01.934 INFO:journalctl@ceph.mds.b.smithi138.stdout:Apr 24 09:03:01 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-b[61515]: 2024-04-24T09:03:01.544+0000 7f7fbbc00700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000221 [...12,head] /volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/WORDPRO/ auth{3=1} v11332 ap=2 f(v630 m2024-04-24T09:02:44.877334+0000 3=3+0) n(v42 rc2024-04-24T09:02:44.877334+0000 b780288 4=3+1) old_inodes=6 (inest sync->mix g=3) caps={15603=pAsLsXsFs/-@10160} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x55cf87627700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000000221:[<0x10000000009/WORDPRO v375>,<0x10000000008/~dmtmp v57>,<0x10000000007/client0 v33>,<0x10000000006/clients v23>,<0x10000000005/tmp v23>,<0x10000000003/client.0 v22>,<0x10000000002/e395bed6-5bba-41b8-a256-3505e4afcca2 v36>,<0x10000000001/sv_0 v55>,<0x10000000000/qa v43>,<0x1/volumes v39>]//[]","memoryvalue":"(2)0x10000000221:[<0x10000000009/WORDPRO v11332>,<0x10000000008/~dmtmp v10792>,<0x10000000007/client0 v1276>,<0x10000000006/clients v1267>,<0x10000000005/tmp v1249>,<0x10000000003/client.0 v1244>,<0x10000000002/e395bed6-5bba-41b8-a256-3505e4afcca2 v1238>,<0x10000000001/sv_0 v1232>,<0x10000000000/qa v1213>,<0x1/volumes v1175>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-04-24T09:02:44.839123+0000 4=4+0)","ondisk_value.rstat":"n(v0 rc2024-04-24T09:02:44.877334+0000 b780288 4=3+1)","memory_value.dirstat":"f(v630 m2024-04-24T09:02:44.877334+0000 3=3+0)","memory_value.rstat":"n(v42 rc2024-04-24T09:02:44.877334+0000 b780288 4=3+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-04-24T09:03:01.935 INFO:journalctl@ceph.mds.b.smithi138.stdout:Apr 24 09:03:01 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-b[61515]: 2024-04-24T09:03:01.559+0000 7f7fbbc00700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000023e [...12,head] /volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/ACCESS/ auth{3=1} v11338 ap=2 f(v252 m2024-04-24T09:02:45.250520+0000 1=1+0) n(v40 rc2024-04-24T09:02:45.250520+0000 b260096 2=1+1) old_inodes=6 (inest sync->mix g=3) caps={15603=pAsLsXsFs/-@8677} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x55cf87666b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000023e:[<0x10000000009/ACCESS v463>,<0x10000000008/~dmtmp v57>,<0x10000000007/client0 v33>,<0x10000000006/clients v23>,<0x10000000005/tmp v23>,<0x10000000003/client.0 v22>,<0x10000000002/e395bed6-5bba-41b8-a256-3505e4afcca2 v36>,<0x10000000001/sv_0 v55>,<0x10000000000/qa v43>,<0x1/volumes v39>]//[]","memoryvalue":"(2)0x1000000023e:[<0x10000000009/ACCESS v11338>,<0x10000000008/~dmtmp v10798>,<0x10000000007/client0 v1276>,<0x10000000006/clients v1267>,<0x10000000005/tmp v1249>,<0x10000000003/client.0 v1244>,<0x10000000002/e395bed6-5bba-41b8-a256-3505e4afcca2 v1238>,<0x10000000001/sv_0 v1232>,<0x10000000000/qa v1213>,<0x1/volumes v1175>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-04-24T09:02:45.212897+0000 2=2+0)","ondisk_value.rstat":"n(v0 rc2024-04-24T09:02:45.250520+0000 b260096 2=1+1)","memory_value.dirstat":"f(v252 m2024-04-24T09:02:45.250520+0000 1=1+0)","memory_value.rstat":"n(v40 rc2024-04-24T09:02:45.250520+0000 b260096 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-04-24T09:03:02.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:03:01 smithi138 ceph-mon[31441]: pgmap v357: 129 pgs: 129 active+clean; 61 MiB data, 883 MiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 266 KiB/s wr, 47 op/s 2024-04-24T09:03:02.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:03:01 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:03:02.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:03:01 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:03:02.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:03:01 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:03:02.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:03:01 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:03:02.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:03:01 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:03:02.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:03:01 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:03:02.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:03:01 smithi189 ceph-mon[31791]: pgmap v357: 129 pgs: 129 active+clean; 61 MiB data, 883 MiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 266 KiB/s wr, 47 op/s 2024-04-24T09:03:02.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:03:01 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:03:02.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:03:01 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:03:02.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:03:01 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:03:02.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:03:01 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:03:02.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:03:01 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:03:02.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:03:01 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:03:02.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:03:01 smithi083 ceph-mon[27907]: pgmap v357: 129 pgs: 129 active+clean; 61 MiB data, 883 MiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 266 KiB/s wr, 47 op/s 2024-04-24T09:03:02.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:03:01 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:03:02.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:03:01 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:03:02.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:03:01 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:03:02.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:03:01 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:03:02.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:03:01 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:03:02.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:03:01 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:03:02.616 INFO:tasks.workunit.client.0.smithi083.stdout: 1 66649 0.61 MB/sec execute 456 sec latency 54.569 ms 2024-04-24T09:03:03.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:03:02 smithi138 ceph-mon[31441]: scrub queued for path: / 2024-04-24T09:03:03.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:03:02 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:03:03.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:03:02 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:03:03.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:03:02 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:03:03.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:03:02 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:03:03.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:03:02 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:03:03.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:03:02 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:03:03.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:03:02 smithi138 ceph-mon[31441]: Scrub error on inode 0x10000000221 (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/WORDPRO) see mds.b log and `damage ls` output for details 2024-04-24T09:03:03.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:03:02 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:03:03.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:03:02 smithi138 ceph-mon[31441]: Scrub error on inode 0x1000000023e (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/ACCESS) see mds.b log and `damage ls` output for details 2024-04-24T09:03:03.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:03:02 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:03:03.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:03:02 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:03:03.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:03:02 smithi189 ceph-mon[31791]: scrub queued for path: / 2024-04-24T09:03:03.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:03:02 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:03:03.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:03:02 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:03:03.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:03:02 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:03:03.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:03:02 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:03:03.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:03:02 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:03:03.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:03:02 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:03:03.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:03:02 smithi189 ceph-mon[31791]: Scrub error on inode 0x10000000221 (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/WORDPRO) see mds.b log and `damage ls` output for details 2024-04-24T09:03:03.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:03:02 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:03:03.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:03:02 smithi189 ceph-mon[31791]: Scrub error on inode 0x1000000023e (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/ACCESS) see mds.b log and `damage ls` output for details 2024-04-24T09:03:03.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:03:02 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:03:03.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:03:02 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:03:03.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:03:02 smithi083 ceph-mon[27907]: scrub queued for path: / 2024-04-24T09:03:03.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:03:02 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:03:03.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:03:02 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:03:03.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:03:02 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:03:03.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:03:02 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:03:03.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:03:02 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:03:03.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:03:02 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:03:03.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:03:02 smithi083 ceph-mon[27907]: Scrub error on inode 0x10000000221 (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/WORDPRO) see mds.b log and `damage ls` output for details 2024-04-24T09:03:03.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:03:02 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:03:03.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:03:02 smithi083 ceph-mon[27907]: Scrub error on inode 0x1000000023e (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/ACCESS) see mds.b log and `damage ls` output for details 2024-04-24T09:03:03.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:03:02 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:03:03.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:03:02 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:03:03.617 INFO:tasks.workunit.client.0.smithi083.stdout: 1 66857 0.61 MB/sec execute 457 sec latency 57.722 ms 2024-04-24T09:03:04.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:03:03 smithi138 ceph-mon[31441]: pgmap v358: 129 pgs: 129 active+clean; 63 MiB data, 888 MiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 549 KiB/s wr, 93 op/s 2024-04-24T09:03:04.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:03:03 smithi138 ceph-mon[31441]: osdmap e107: 12 total, 12 up, 12 in 2024-04-24T09:03:04.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:03:03 smithi189 ceph-mon[31791]: pgmap v358: 129 pgs: 129 active+clean; 63 MiB data, 888 MiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 549 KiB/s wr, 93 op/s 2024-04-24T09:03:04.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:03:03 smithi189 ceph-mon[31791]: osdmap e107: 12 total, 12 up, 12 in 2024-04-24T09:03:04.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:03:03 smithi083 ceph-mon[27907]: pgmap v358: 129 pgs: 129 active+clean; 63 MiB data, 888 MiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 549 KiB/s wr, 93 op/s 2024-04-24T09:03:04.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:03:03 smithi083 ceph-mon[27907]: osdmap e107: 12 total, 12 up, 12 in 2024-04-24T09:03:04.617 INFO:tasks.workunit.client.0.smithi083.stdout: 1 67080 0.61 MB/sec execute 458 sec latency 50.482 ms 2024-04-24T09:03:04.635 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.636+0000 7f0684a00640 1 -- 172.21.15.83:0/912008512 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0680109370 msgr2=0x7f0680110f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:04.635 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.636+0000 7f0684a00640 1 --2- 172.21.15.83:0/912008512 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0680109370 0x7f0680110f40 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f0670009900 tx=0x7f067002a410 comp rx=0 tx=0).stop 2024-04-24T09:03:04.636 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.636+0000 7f0684a00640 1 -- 172.21.15.83:0/912008512 shutdown_connections 2024-04-24T09:03:04.636 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.636+0000 7f0684a00640 1 --2- 172.21.15.83:0/912008512 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0680109370 0x7f0680110f40 unknown :-1 s=CLOSED pgs=1863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:04.636 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.636+0000 7f0684a00640 1 --2- 172.21.15.83:0/912008512 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0680104c50 0x7f0680108c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:04.636 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.636+0000 7f0684a00640 1 --2- 172.21.15.83:0/912008512 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06801042a0 0x7f0680104680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:04.636 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.636+0000 7f0684a00640 1 -- 172.21.15.83:0/912008512 >> 172.21.15.83:0/912008512 conn(0x7f06800ffae0 msgr2=0x7f0680101f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:04.637 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.637+0000 7f0684a00640 1 -- 172.21.15.83:0/912008512 shutdown_connections 2024-04-24T09:03:04.637 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.637+0000 7f0684a00640 1 -- 172.21.15.83:0/912008512 wait complete. 2024-04-24T09:03:04.638 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.638+0000 7f0684a00640 1 Processor -- start 2024-04-24T09:03:04.638 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.638+0000 7f0684a00640 1 -- start start 2024-04-24T09:03:04.638 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.638+0000 7f0684a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f06801042a0 0x7f06801a0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:04.638 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.638+0000 7f0684a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0680109370 0x7f06801a0970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:04.638 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.638+0000 7f0684a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f068019a630 0x7f068019aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:04.638 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.638+0000 7f0684a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0680114330 con 0x7f0680109370 2024-04-24T09:03:04.638 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.638+0000 7f0684a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f06801141b0 con 0x7f068019a630 2024-04-24T09:03:04.638 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.638+0000 7f0684a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f06801144b0 con 0x7f06801042a0 2024-04-24T09:03:04.638 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.639+0000 7f067de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0680109370 0x7f06801a0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:04.639 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.639+0000 7f067de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0680109370 0x7f06801a0970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45736/0 (socket says 172.21.15.83:45736) 2024-04-24T09:03:04.639 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.639+0000 7f067de00640 1 -- 172.21.15.83:0/92690091 learned_addr learned my addr 172.21.15.83:0/92690091 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:03:04.639 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.639+0000 7f067e800640 1 --2- 172.21.15.83:0/92690091 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f06801042a0 0x7f06801a0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:04.639 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.639+0000 7f067f200640 1 --2- 172.21.15.83:0/92690091 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f068019a630 0x7f068019aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:04.639 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.639+0000 7f067de00640 1 -- 172.21.15.83:0/92690091 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f06801042a0 msgr2=0x7f06801a0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:04.639 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.639+0000 7f067de00640 1 --2- 172.21.15.83:0/92690091 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f06801042a0 0x7f06801a0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:04.639 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.639+0000 7f067de00640 1 -- 172.21.15.83:0/92690091 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f068019a630 msgr2=0x7f068019aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:04.639 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.639+0000 7f067de00640 1 --2- 172.21.15.83:0/92690091 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f068019a630 0x7f068019aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:04.639 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.639+0000 7f067de00640 1 -- 172.21.15.83:0/92690091 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f068019b150 con 0x7f0680109370 2024-04-24T09:03:04.640 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.639+0000 7f067de00640 1 --2- 172.21.15.83:0/92690091 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0680109370 0x7f06801a0970 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f066800d740 tx=0x7f066800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:04.640 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.640+0000 7f0677400640 1 -- 172.21.15.83:0/92690091 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0668017070 con 0x7f0680109370 2024-04-24T09:03:04.640 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.640+0000 7f0677400640 1 -- 172.21.15.83:0/92690091 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f066800de50 con 0x7f0680109370 2024-04-24T09:03:04.640 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.640+0000 7f0677400640 1 -- 172.21.15.83:0/92690091 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0668004d60 con 0x7f0680109370 2024-04-24T09:03:04.640 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.640+0000 7f0684a00640 1 -- 172.21.15.83:0/92690091 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f068019b440 con 0x7f0680109370 2024-04-24T09:03:04.640 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.640+0000 7f0684a00640 1 -- 172.21.15.83:0/92690091 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06801a75f0 con 0x7f0680109370 2024-04-24T09:03:04.642 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.643+0000 7f0687dfd740 1 Processor -- start 2024-04-24T09:03:04.643 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.644+0000 7f0687dfd740 1 -- start start 2024-04-24T09:03:04.643 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.644+0000 7f0677400640 1 -- 172.21.15.83:0/92690091 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0668004f00 con 0x7f0680109370 2024-04-24T09:03:04.643 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.644+0000 7f0687dfd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b69b7fd3c0 0x55b69b7fd7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:04.644 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.644+0000 7f0687dfd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b69b7fdce0 0x55b69b808060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:04.644 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.644+0000 7f0687dfd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b69b8085a0 0x55b69b80a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:04.644 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.644+0000 7f0687dfd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55b69b763900 con 0x55b69b8085a0 2024-04-24T09:03:04.644 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.644+0000 7f0687dfd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b69b709f20 con 0x55b69b7fd3c0 2024-04-24T09:03:04.644 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.644+0000 7f0687dfd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55b69b75f8c0 con 0x55b69b7fdce0 2024-04-24T09:03:04.644 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.644+0000 7f067de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b69b8085a0 0x55b69b80a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:04.645 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.644+0000 7f067de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b69b8085a0 0x55b69b80a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45752/0 (socket says 172.21.15.83:45752) 2024-04-24T09:03:04.645 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.645+0000 7f067de00640 1 -- 172.21.15.83:0/840201954 learned_addr learned my addr 172.21.15.83:0/840201954 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:03:04.645 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.645+0000 7f067f200640 1 --2- 172.21.15.83:0/840201954 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b69b7fd3c0 0x55b69b7fd7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:04.645 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.645+0000 7f067e800640 1 --2- 172.21.15.83:0/840201954 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b69b7fdce0 0x55b69b808060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:04.645 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.645+0000 7f067de00640 1 -- 172.21.15.83:0/840201954 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b69b7fdce0 msgr2=0x55b69b808060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:04.645 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.645+0000 7f067de00640 1 --2- 172.21.15.83:0/840201954 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b69b7fdce0 0x55b69b808060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:04.645 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.645+0000 7f067de00640 1 -- 172.21.15.83:0/840201954 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b69b7fd3c0 msgr2=0x55b69b7fd7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:04.645 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.645+0000 7f067de00640 1 --2- 172.21.15.83:0/840201954 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b69b7fd3c0 0x55b69b7fd7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:04.645 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.645+0000 7f067de00640 1 -- 172.21.15.83:0/840201954 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b69b72ee10 con 0x55b69b8085a0 2024-04-24T09:03:04.645 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.645+0000 7f067de00640 1 --2- 172.21.15.83:0/840201954 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b69b8085a0 0x55b69b80a990 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f06680cd7d0 tx=0x7f06680d8670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab54286397015431 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:04.647 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.646+0000 7f0677400640 1 --2- 172.21.15.83:0/92690091 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f06640a3d90 0x7f06640a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:04.648 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.646+0000 7f066f400640 1 -- 172.21.15.83:0/840201954 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06680df070 con 0x55b69b8085a0 2024-04-24T09:03:04.648 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.646+0000 7f066f400640 1 -- 172.21.15.83:0/840201954 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f066809c7e0 con 0x55b69b8085a0 2024-04-24T09:03:04.648 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.646+0000 7f066f400640 1 -- 172.21.15.83:0/840201954 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f066809bc20 con 0x55b69b8085a0 2024-04-24T09:03:04.648 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.646+0000 7f0677400640 1 -- 172.21.15.83:0/92690091 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9671+0+0 (secure 0 0 0) 0x7f06680d0620 con 0x7f0680109370 2024-04-24T09:03:04.648 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.646+0000 7f067f200640 1 --2- 172.21.15.83:0/92690091 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f06640a3d90 0x7f06640a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:04.648 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.647+0000 7f0687dfd740 1 -- 172.21.15.83:0/840201954 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b69b8085a0 msgr2=0x55b69b80a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:04.648 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.647+0000 7f0687dfd740 1 --2- 172.21.15.83:0/840201954 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b69b8085a0 0x55b69b80a990 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f06680cd7d0 tx=0x7f06680d8670 comp rx=0 tx=0).stop 2024-04-24T09:03:04.648 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.647+0000 7f0687dfd740 1 -- 172.21.15.83:0/840201954 shutdown_connections 2024-04-24T09:03:04.648 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.647+0000 7f0687dfd740 1 --2- 172.21.15.83:0/840201954 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b69b8085a0 0x55b69b80a990 unknown :-1 s=CLOSED pgs=1865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:04.648 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.647+0000 7f0687dfd740 1 --2- 172.21.15.83:0/840201954 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b69b7fdce0 0x55b69b808060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:04.648 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.647+0000 7f0687dfd740 1 --2- 172.21.15.83:0/840201954 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b69b7fd3c0 0x55b69b7fd7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:04.648 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.647+0000 7f0687dfd740 1 -- 172.21.15.83:0/840201954 >> 172.21.15.83:0/840201954 conn(0x55b69b7f98d0 msgr2=0x55b69b7f9cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:04.648 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.647+0000 7f067f200640 1 --2- 172.21.15.83:0/92690091 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f06640a3d90 0x7f06640a6250 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f0670031440 tx=0x7f0670005a20 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:04.648 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.647+0000 7f0687dfd740 1 -- 172.21.15.83:0/840201954 shutdown_connections 2024-04-24T09:03:04.649 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.647+0000 7f0687dfd740 1 -- 172.21.15.83:0/840201954 wait complete. 2024-04-24T09:03:04.649 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.648+0000 7f0687dfd740 1 -- start start 2024-04-24T09:03:04.649 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.648+0000 7f0687dfd740 1 Processor -- start 2024-04-24T09:03:04.649 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.648+0000 7f0687dfd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b69b7fd3c0 0x55b69b805500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:04.649 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.648+0000 7f0687dfd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b69b7fdce0 0x55b69b805a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:04.649 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.648+0000 7f0687dfd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b69b8085a0 0x55b69b805f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:04.649 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.648+0000 7f0687dfd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55b69b80b0e0 con 0x55b69b7fdce0 2024-04-24T09:03:04.649 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.648+0000 7f0687dfd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b69b71d2c0 con 0x55b69b8085a0 2024-04-24T09:03:04.649 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.648+0000 7f0687dfd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55b69b80b260 con 0x55b69b7fd3c0 2024-04-24T09:03:04.649 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.649+0000 7f0687dfd740 10 client.0 _ll_register_callbacks cb 0x55b69b475a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:03:04.649 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.649+0000 7f067de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b69b7fdce0 0x55b69b805a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:04.649 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.649+0000 7f067de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b69b7fdce0 0x55b69b805a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45764/0 (socket says 172.21.15.83:45764) 2024-04-24T09:03:04.649 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.649+0000 7f067de00640 1 -- 172.21.15.83:0/509061638 learned_addr learned my addr 172.21.15.83:0/509061638 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:03:04.652 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.649+0000 7f067e800640 1 --2- 172.21.15.83:0/509061638 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b69b7fd3c0 0x55b69b805500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:04.652 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.649+0000 7f067f200640 1 --2- 172.21.15.83:0/509061638 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b69b8085a0 0x55b69b805f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:04.652 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.649+0000 7f067de00640 1 -- 172.21.15.83:0/509061638 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b69b7fd3c0 msgr2=0x55b69b805500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:04.652 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.649+0000 7f067de00640 1 --2- 172.21.15.83:0/509061638 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b69b7fd3c0 0x55b69b805500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:04.652 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.649+0000 7f067de00640 1 -- 172.21.15.83:0/509061638 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b69b8085a0 msgr2=0x55b69b805f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:04.652 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.649+0000 7f067de00640 1 --2- 172.21.15.83:0/509061638 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b69b8085a0 0x55b69b805f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:04.652 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.649+0000 7f067de00640 1 -- 172.21.15.83:0/509061638 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b69b72dfd0 con 0x55b69b7fdce0 2024-04-24T09:03:04.652 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.650+0000 7f067de00640 1 --2- 172.21.15.83:0/509061638 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b69b7fdce0 0x55b69b805a40 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7f066809bde0 tx=0x7f06680d08a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7e38c43188f9e2e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:04.652 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.650+0000 7f0663e00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:03:04.652 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.650+0000 7f0663e00640 1 -- 172.21.15.83:0/509061638 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06680a8720 con 0x55b69b7fdce0 2024-04-24T09:03:04.652 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.650+0000 7f0663e00640 1 -- 172.21.15.83:0/509061638 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9671+0+0 (secure 0 0 0) 0x7f066809a3e0 con 0x55b69b7fdce0 2024-04-24T09:03:04.652 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.650+0000 7f0687dfd740 1 -- 172.21.15.83:0/509061638 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f066809c7e0 con 0x55b69b7fdce0 2024-04-24T09:03:04.653 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.650+0000 7f0663e00640 1 -- 172.21.15.83:0/509061638 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f06680df030 con 0x55b69b7fdce0 2024-04-24T09:03:04.653 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.650+0000 7f0687dfd740 10 client.19758 fetch_fsmap learned FSMap version 41 2024-04-24T09:03:04.653 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.650+0000 7f0687dfd740 1 -- 172.21.15.83:0/509061638 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55b69b476210 con 0x55b69b7fdce0 2024-04-24T09:03:04.653 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.650+0000 7f0663e00640 1 -- 172.21.15.83:0/509061638 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f066809c7e0 con 0x55b69b7fdce0 2024-04-24T09:03:04.653 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.651+0000 7f0663e00640 1 -- 172.21.15.83:0/509061638 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7f0668098750 con 0x55b69b7fdce0 2024-04-24T09:03:04.653 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.651+0000 7f0687dfd740 10 client.19758 fetch_fsmap finished waiting for FSMap version 41 2024-04-24T09:03:04.653 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.651+0000 7f0687dfd740 10 client.19758 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:03:04.653 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.651+0000 7f0687dfd740 20 client.19758 populate_metadata read hostname 'smithi083' 2024-04-24T09:03:04.653 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.651+0000 7f0687dfd740 1 --2- 172.21.15.83:0/509061638 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55b69b818510 0x55b69b81a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:04.653 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.651+0000 7f0687dfd740 4 client.19758 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:03:04.653 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.651+0000 7f0687dfd740 1 -- 172.21.15.83:0/509061638 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b69b771320 con 0x55b69b818510 2024-04-24T09:03:04.653 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.652+0000 7f067e800640 1 --2- 172.21.15.83:0/509061638 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55b69b818510 0x55b69b81a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:04.653 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.653+0000 7f0663e00640 10 client.19758 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:03:04.653 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.653+0000 7f067e800640 1 --2- 172.21.15.83:0/509061638 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55b69b818510 0x55b69b81a900 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc67b0f8c58d0c3 server_cookie=2547ac2cf3d3758c in_seq=0 out_seq=0 2024-04-24T09:03:04.654 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.654+0000 7f0663e00640 1 -- 172.21.15.83:0/509061638 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b69b771320 con 0x55b69b818510 2024-04-24T09:03:04.655 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.654+0000 7f0663e00640 10 client.19758 handle_command_reply: tid=0 2024-04-24T09:03:04.655 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.655+0000 7f0687dfd740 1 client.19758 shutdown 2024-04-24T09:03:04.655 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.655+0000 7f0687dfd740 1 -- 172.21.15.83:0/509061638 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b69b7fdce0 msgr2=0x55b69b805a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:04.655 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.655+0000 7f0687dfd740 1 --2- 172.21.15.83:0/509061638 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b69b7fdce0 0x55b69b805a40 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7f066809bde0 tx=0x7f06680d08a0 comp rx=0 tx=0).stop 2024-04-24T09:03:04.655 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.655+0000 7f0687dfd740 1 -- 172.21.15.83:0/509061638 shutdown_connections 2024-04-24T09:03:04.655 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.655+0000 7f0687dfd740 1 --2- 172.21.15.83:0/509061638 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55b69b818510 0x55b69b81a900 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:04.655 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.655+0000 7f0687dfd740 1 --2- 172.21.15.83:0/509061638 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b69b8085a0 0x55b69b805f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:04.655 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.655+0000 7f0687dfd740 1 --2- 172.21.15.83:0/509061638 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b69b7fdce0 0x55b69b805a40 unknown :-1 s=CLOSED pgs=1866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:04.655 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.655+0000 7f0687dfd740 1 --2- 172.21.15.83:0/509061638 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b69b7fd3c0 0x55b69b805500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:04.655 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.655+0000 7f0663e00640 0 client.19758 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:03:04.655 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.655+0000 7f0687dfd740 1 -- 172.21.15.83:0/509061638 >> 172.21.15.83:0/509061638 conn(0x55b69b7f98d0 msgr2=0x55b69b7f8a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:04.655 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.655+0000 7f0687dfd740 1 -- 172.21.15.83:0/509061638 shutdown_connections 2024-04-24T09:03:04.655 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.655+0000 7f0687dfd740 1 -- 172.21.15.83:0/509061638 wait complete. 2024-04-24T09:03:04.656 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.655+0000 7f0687dfd740 20 client.19758 trim_cache size 0 max 16384 2024-04-24T09:03:04.673 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.674+0000 7f0687dfd740 1 Processor -- start 2024-04-24T09:03:04.673 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.674+0000 7f0687dfd740 1 -- start start 2024-04-24T09:03:04.673 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.674+0000 7f0687dfd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b69b7fd3c0 0x55b69b807c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:04.673 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.674+0000 7f0687dfd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b69b7fdce0 0x55b69b814080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:04.673 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.674+0000 7f0687dfd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b69b8085a0 0x55b69b7f9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:04.674 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.674+0000 7f0687dfd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f06801b1750 con 0x55b69b7fdce0 2024-04-24T09:03:04.674 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.674+0000 7f0687dfd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f06801b33b0 con 0x55b69b8085a0 2024-04-24T09:03:04.674 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.674+0000 7f0687dfd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f06801b1e20 con 0x55b69b7fd3c0 2024-04-24T09:03:04.674 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.674+0000 7f067de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b69b7fdce0 0x55b69b814080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:04.674 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.674+0000 7f067de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b69b7fdce0 0x55b69b814080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45770/0 (socket says 172.21.15.83:45770) 2024-04-24T09:03:04.674 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.674+0000 7f067de00640 1 -- 172.21.15.83:0/2620133558 learned_addr learned my addr 172.21.15.83:0/2620133558 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:03:04.675 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.675+0000 7f067de00640 1 -- 172.21.15.83:0/2620133558 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b69b7fd3c0 msgr2=0x55b69b807c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:04.675 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.675+0000 7f067de00640 1 --2- 172.21.15.83:0/2620133558 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b69b7fd3c0 0x55b69b807c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:04.675 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.675+0000 7f067de00640 1 -- 172.21.15.83:0/2620133558 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b69b8085a0 msgr2=0x55b69b7f9cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:04.676 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.675+0000 7f067de00640 1 --2- 172.21.15.83:0/2620133558 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b69b8085a0 0x55b69b7f9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:04.676 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.675+0000 7f067de00640 1 -- 172.21.15.83:0/2620133558 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b69b8044b0 con 0x55b69b7fdce0 2024-04-24T09:03:04.676 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.675+0000 7f067de00640 1 --2- 172.21.15.83:0/2620133558 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b69b7fdce0 0x55b69b814080 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7f066809c070 tx=0x7f066809bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=460c9b24330992db server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:04.676 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.675+0000 7f066ea00640 1 -- 172.21.15.83:0/2620133558 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0668020070 con 0x55b69b7fdce0 2024-04-24T09:03:04.676 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.675+0000 7f066ea00640 1 -- 172.21.15.83:0/2620133558 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f06680d6300 con 0x55b69b7fdce0 2024-04-24T09:03:04.676 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.675+0000 7f066ea00640 1 -- 172.21.15.83:0/2620133558 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06680a8910 con 0x55b69b7fdce0 2024-04-24T09:03:04.676 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.676+0000 7f0687dfd740 1 -- 172.21.15.83:0/2620133558 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b69b7fdce0 msgr2=0x55b69b814080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:04.676 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.676+0000 7f0687dfd740 1 --2- 172.21.15.83:0/2620133558 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b69b7fdce0 0x55b69b814080 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7f066809c070 tx=0x7f066809bf70 comp rx=0 tx=0).stop 2024-04-24T09:03:04.676 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.676+0000 7f0687dfd740 1 -- 172.21.15.83:0/2620133558 shutdown_connections 2024-04-24T09:03:04.676 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.676+0000 7f0687dfd740 1 --2- 172.21.15.83:0/2620133558 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b69b8085a0 0x55b69b7f9cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:04.676 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.676+0000 7f0687dfd740 1 --2- 172.21.15.83:0/2620133558 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b69b7fdce0 0x55b69b814080 unknown :-1 s=CLOSED pgs=1867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:04.676 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.676+0000 7f0687dfd740 1 --2- 172.21.15.83:0/2620133558 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b69b7fd3c0 0x55b69b807c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:04.676 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.676+0000 7f0687dfd740 1 -- 172.21.15.83:0/2620133558 >> 172.21.15.83:0/2620133558 conn(0x55b69b7f98d0 msgr2=0x55b69b7f8a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:04.677 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.676+0000 7f0687dfd740 1 -- 172.21.15.83:0/2620133558 shutdown_connections 2024-04-24T09:03:04.677 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.676+0000 7f0687dfd740 1 -- 172.21.15.83:0/2620133558 wait complete. 2024-04-24T09:03:04.677 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.677+0000 7f0687dfd740 1 -- start start 2024-04-24T09:03:04.679 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.677+0000 7f0687dfd740 1 Processor -- start 2024-04-24T09:03:04.679 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.677+0000 7f0687dfd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b69b7fd3c0 0x55b69b80c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:04.679 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.677+0000 7f0687dfd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b69b7fdce0 0x55b69b80c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:04.679 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.677+0000 7f0687dfd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b69b8085a0 0x55b69b87d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:04.680 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.677+0000 7f0687dfd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f06801b2c80 con 0x55b69b7fd3c0 2024-04-24T09:03:04.680 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.677+0000 7f0687dfd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f06801b2550 con 0x55b69b7fdce0 2024-04-24T09:03:04.680 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.678+0000 7f0687dfd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f06801af750 con 0x55b69b8085a0 2024-04-24T09:03:04.680 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.678+0000 7f0687dfd740 10 client.0 _ll_register_callbacks cb 0x7f06680d6a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:03:04.680 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.678+0000 7f067e800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b69b7fd3c0 0x55b69b80c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:04.680 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.678+0000 7f067e800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b69b7fd3c0 0x55b69b80c380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45778/0 (socket says 172.21.15.83:45778) 2024-04-24T09:03:04.680 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.678+0000 7f067e800640 1 -- 172.21.15.83:0/4193023991 learned_addr learned my addr 172.21.15.83:0/4193023991 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:03:04.680 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.678+0000 7f067e800640 1 -- 172.21.15.83:0/4193023991 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b69b8085a0 msgr2=0x55b69b87d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:04.680 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.678+0000 7f067f200640 1 --2- 172.21.15.83:0/4193023991 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b69b8085a0 0x55b69b87d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:04.680 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.678+0000 7f067de00640 1 --2- 172.21.15.83:0/4193023991 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b69b7fdce0 0x55b69b80c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:04.680 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.678+0000 7f067e800640 1 --2- 172.21.15.83:0/4193023991 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b69b8085a0 0x55b69b87d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:04.680 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.678+0000 7f067e800640 1 -- 172.21.15.83:0/4193023991 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b69b7fdce0 msgr2=0x55b69b80c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:04.680 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.678+0000 7f067e800640 1 --2- 172.21.15.83:0/4193023991 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b69b7fdce0 0x55b69b80c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:04.680 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.678+0000 7f067e800640 1 -- 172.21.15.83:0/4193023991 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b69b81ac70 con 0x55b69b7fd3c0 2024-04-24T09:03:04.681 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.678+0000 7f067f200640 1 --2- 172.21.15.83:0/4193023991 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b69b8085a0 0x55b69b87d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:03:04.681 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.678+0000 7f067e800640 1 --2- 172.21.15.83:0/4193023991 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b69b7fd3c0 0x55b69b80c380 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x55b69b81ae40 tx=0x7f0678013830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11bf70fc98283258 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:04.681 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.678+0000 7f0663e00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:03:04.681 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.679+0000 7f0663e00640 1 -- 172.21.15.83:0/4193023991 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f067801a530 con 0x55b69b7fd3c0 2024-04-24T09:03:04.681 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.679+0000 7f0663e00640 1 -- 172.21.15.83:0/4193023991 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9671+0+0 (secure 0 0 0) 0x7f067801a6d0 con 0x55b69b7fd3c0 2024-04-24T09:03:04.681 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.679+0000 7f0687dfd740 1 -- 172.21.15.83:0/4193023991 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06680d6300 con 0x55b69b7fd3c0 2024-04-24T09:03:04.681 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.679+0000 7f0663e00640 1 -- 172.21.15.83:0/4193023991 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f067802a050 con 0x55b69b7fd3c0 2024-04-24T09:03:04.681 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.679+0000 7f0687dfd740 10 client.19764 fetch_fsmap learned FSMap version 41 2024-04-24T09:03:04.681 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.679+0000 7f0687dfd740 1 -- 172.21.15.83:0/4193023991 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55b69b542620 con 0x55b69b7fd3c0 2024-04-24T09:03:04.681 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.679+0000 7f0663e00640 1 -- 172.21.15.83:0/4193023991 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f06680d6300 con 0x55b69b7fd3c0 2024-04-24T09:03:04.681 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.680+0000 7f0663e00640 1 -- 172.21.15.83:0/4193023991 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7f0678010430 con 0x55b69b7fd3c0 2024-04-24T09:03:04.681 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.680+0000 7f0687dfd740 10 client.19764 fetch_fsmap finished waiting for FSMap version 41 2024-04-24T09:03:04.681 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.680+0000 7f0687dfd740 10 client.19764 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:03:04.681 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.680+0000 7f0687dfd740 20 client.19764 populate_metadata read hostname 'smithi083' 2024-04-24T09:03:04.681 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.680+0000 7f0687dfd740 1 --2- 172.21.15.83:0/4193023991 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55b69b87f250 0x55b69b881670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:04.682 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.680+0000 7f0687dfd740 4 client.19764 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T09:03:04.682 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.680+0000 7f0687dfd740 1 -- 172.21.15.83:0/4193023991 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b69b76b190 con 0x55b69b87f250 2024-04-24T09:03:04.682 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.680+0000 7f067de00640 1 --2- 172.21.15.83:0/4193023991 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55b69b87f250 0x55b69b881670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:04.682 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.681+0000 7f067de00640 1 --2- 172.21.15.83:0/4193023991 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55b69b87f250 0x55b69b881670 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6050428ae97fd58 server_cookie=f27d1ae91cf2cd38 in_seq=0 out_seq=0 2024-04-24T09:03:04.682 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.681+0000 7f0663e00640 10 client.19764 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:03:04.744 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.745+0000 7f0663e00640 1 -- 172.21.15.83:0/4193023991 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b69b76b190 con 0x55b69b87f250 2024-04-24T09:03:04.744 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.745+0000 7f0663e00640 10 client.19764 handle_command_reply: tid=0 2024-04-24T09:03:04.744 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.745+0000 7f0687dfd740 1 client.19764 shutdown 2024-04-24T09:03:04.744 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.745+0000 7f0687dfd740 1 -- 172.21.15.83:0/4193023991 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b69b7fd3c0 msgr2=0x55b69b80c380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:04.744 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.745+0000 7f0687dfd740 1 --2- 172.21.15.83:0/4193023991 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b69b7fd3c0 0x55b69b80c380 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x55b69b81ae40 tx=0x7f0678013830 comp rx=0 tx=0).stop 2024-04-24T09:03:04.745 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.745+0000 7f0687dfd740 1 -- 172.21.15.83:0/4193023991 shutdown_connections 2024-04-24T09:03:04.745 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.745+0000 7f0687dfd740 1 --2- 172.21.15.83:0/4193023991 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55b69b87f250 0x55b69b881670 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:04.745 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.745+0000 7f0687dfd740 1 --2- 172.21.15.83:0/4193023991 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b69b8085a0 0x55b69b87d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:04.745 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.745+0000 7f0687dfd740 1 --2- 172.21.15.83:0/4193023991 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b69b7fdce0 0x55b69b80c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:04.745 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.745+0000 7f0687dfd740 1 --2- 172.21.15.83:0/4193023991 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b69b7fd3c0 0x55b69b80c380 unknown :-1 s=CLOSED pgs=1868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:04.745 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.745+0000 7f0663e00640 0 client.19764 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:03:04.745 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.745+0000 7f0687dfd740 1 -- 172.21.15.83:0/4193023991 >> 172.21.15.83:0/4193023991 conn(0x55b69b7f98d0 msgr2=0x55b69b7f8a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:04.746 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.746+0000 7f0687dfd740 1 -- 172.21.15.83:0/4193023991 shutdown_connections 2024-04-24T09:03:04.746 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.746+0000 7f0687dfd740 1 -- 172.21.15.83:0/4193023991 wait complete. 2024-04-24T09:03:04.746 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.746+0000 7f0687dfd740 20 client.19764 trim_cache size 0 max 16384 2024-04-24T09:03:04.746 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T09:03:04.746 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T09:03:04.746 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T09:03:04.746 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T09:03:04.747 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.747+0000 7f0684a00640 1 -- 172.21.15.83:0/92690091 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f06640a3d90 msgr2=0x7f06640a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:04.747 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.747+0000 7f0684a00640 1 --2- 172.21.15.83:0/92690091 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f06640a3d90 0x7f06640a6250 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f0670031440 tx=0x7f0670005a20 comp rx=0 tx=0).stop 2024-04-24T09:03:04.747 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.747+0000 7f0684a00640 1 -- 172.21.15.83:0/92690091 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0680109370 msgr2=0x7f06801a0970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:04.747 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.747+0000 7f0684a00640 1 --2- 172.21.15.83:0/92690091 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0680109370 0x7f06801a0970 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f066800d740 tx=0x7f066800dc00 comp rx=0 tx=0).stop 2024-04-24T09:03:04.747 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.747+0000 7f0684a00640 1 -- 172.21.15.83:0/92690091 shutdown_connections 2024-04-24T09:03:04.747 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.747+0000 7f0684a00640 1 --2- 172.21.15.83:0/92690091 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f06640a3d90 0x7f06640a6250 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:04.747 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.747+0000 7f0684a00640 1 --2- 172.21.15.83:0/92690091 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f068019a630 0x7f068019aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:04.748 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.747+0000 7f0684a00640 1 --2- 172.21.15.83:0/92690091 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0680109370 0x7f06801a0970 unknown :-1 s=CLOSED pgs=1864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:04.748 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.747+0000 7f0684a00640 1 --2- 172.21.15.83:0/92690091 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f06801042a0 0x7f06801a0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:04.748 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.747+0000 7f0684a00640 1 -- 172.21.15.83:0/92690091 >> 172.21.15.83:0/92690091 conn(0x7f06800ffae0 msgr2=0x7f0680101db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:04.748 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.748+0000 7f0684a00640 1 -- 172.21.15.83:0/92690091 shutdown_connections 2024-04-24T09:03:04.748 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:04.748+0000 7f0684a00640 1 -- 172.21.15.83:0/92690091 wait complete. 2024-04-24T09:03:05.617 INFO:tasks.workunit.client.0.smithi083.stdout: 1 67174 0.61 MB/sec execute 459 sec latency 63.115 ms 2024-04-24T09:03:06.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:03:05 smithi189 ceph-mon[31791]: pgmap v360: 129 pgs: 129 active+clean; 63 MiB data, 888 MiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 425 KiB/s wr, 78 op/s 2024-04-24T09:03:06.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:03:05 smithi083 ceph-mon[27907]: pgmap v360: 129 pgs: 129 active+clean; 63 MiB data, 888 MiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 425 KiB/s wr, 78 op/s 2024-04-24T09:03:06.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:03:05 smithi138 ceph-mon[31441]: pgmap v360: 129 pgs: 129 active+clean; 63 MiB data, 888 MiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 425 KiB/s wr, 78 op/s 2024-04-24T09:03:06.617 INFO:tasks.workunit.client.0.smithi083.stdout: 1 67198 0.61 MB/sec execute 460 sec latency 53.039 ms 2024-04-24T09:03:07.617 INFO:tasks.workunit.client.0.smithi083.stdout: 1 67242 0.61 MB/sec execute 461 sec latency 266.022 ms 2024-04-24T09:03:08.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:03:07 smithi083 ceph-mon[27907]: pgmap v361: 129 pgs: 129 active+clean; 63 MiB data, 888 MiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 425 KiB/s wr, 78 op/s 2024-04-24T09:03:08.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:03:07 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:03:08.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:03:07 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:03:08.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:03:07 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:03:08.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:03:07 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:03:08.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:03:07 smithi138 ceph-mon[31441]: pgmap v361: 129 pgs: 129 active+clean; 63 MiB data, 888 MiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 425 KiB/s wr, 78 op/s 2024-04-24T09:03:08.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:03:07 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:03:08.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:03:07 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:03:08.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:03:07 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:03:08.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:03:07 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:03:08.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:03:07 smithi189 ceph-mon[31791]: pgmap v361: 129 pgs: 129 active+clean; 63 MiB data, 888 MiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 425 KiB/s wr, 78 op/s 2024-04-24T09:03:08.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:03:07 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:03:08.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:03:07 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:03:08.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:03:07 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:03:08.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:03:07 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:03:08.617 INFO:tasks.workunit.client.0.smithi083.stdout: 1 67423 0.62 MB/sec execute 462 sec latency 57.296 ms 2024-04-24T09:03:09.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:03:08 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:03:09.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:03:08 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:03:09.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:03:08 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:03:09.617 INFO:tasks.workunit.client.0.smithi083.stdout: 1 67488 0.62 MB/sec execute 463 sec latency 670.649 ms 2024-04-24T09:03:10.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:03:09 smithi083 ceph-mon[27907]: pgmap v362: 129 pgs: 129 active+clean; 65 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 683 KiB/s wr, 80 op/s 2024-04-24T09:03:10.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:03:09 smithi083 ceph-mon[27907]: osdmap e108: 12 total, 12 up, 12 in 2024-04-24T09:03:10.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:03:09 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:03:10.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:03:09 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:03:10.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:03:09 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:03:10.356 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:03:09 smithi138 ceph-mon[31441]: pgmap v362: 129 pgs: 129 active+clean; 65 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 683 KiB/s wr, 80 op/s 2024-04-24T09:03:10.356 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:03:09 smithi138 ceph-mon[31441]: osdmap e108: 12 total, 12 up, 12 in 2024-04-24T09:03:10.356 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:03:09 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:03:10.356 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:03:09 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:03:10.356 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:03:09 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:03:10.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:03:09 smithi189 ceph-mon[31791]: pgmap v362: 129 pgs: 129 active+clean; 65 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 683 KiB/s wr, 80 op/s 2024-04-24T09:03:10.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:03:09 smithi189 ceph-mon[31791]: osdmap e108: 12 total, 12 up, 12 in 2024-04-24T09:03:10.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:03:09 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:03:10.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:03:09 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:03:10.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:03:09 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:03:10.617 INFO:tasks.workunit.client.0.smithi083.stdout: 1 67488 0.61 MB/sec execute 464 sec latency 1670.788 ms 2024-04-24T09:03:11.618 INFO:tasks.workunit.client.0.smithi083.stdout: 1 67488 0.61 MB/sec execute 465 sec latency 2670.925 ms 2024-04-24T09:03:12.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:03:11 smithi083 ceph-mon[27907]: pgmap v364: 129 pgs: 129 active+clean; 65 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 429 KiB/s wr, 31 op/s 2024-04-24T09:03:12.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:03:11 smithi138 ceph-mon[31441]: pgmap v364: 129 pgs: 129 active+clean; 65 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 429 KiB/s wr, 31 op/s 2024-04-24T09:03:12.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:03:11 smithi189 ceph-mon[31791]: pgmap v364: 129 pgs: 129 active+clean; 65 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 429 KiB/s wr, 31 op/s 2024-04-24T09:03:12.618 INFO:tasks.workunit.client.0.smithi083.stdout: 1 67488 0.61 MB/sec execute 466 sec latency 3671.129 ms 2024-04-24T09:03:13.618 INFO:tasks.workunit.client.0.smithi083.stdout: 1 67528 0.61 MB/sec execute 467 sec latency 4160.154 ms 2024-04-24T09:03:14.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:03:13 smithi083 ceph-mon[27907]: pgmap v365: 129 pgs: 129 active+clean; 65 MiB data, 886 MiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 397 KiB/s wr, 41 op/s 2024-04-24T09:03:14.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:03:13 smithi138 ceph-mon[31441]: pgmap v365: 129 pgs: 129 active+clean; 65 MiB data, 886 MiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 397 KiB/s wr, 41 op/s 2024-04-24T09:03:14.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:03:13 smithi189 ceph-mon[31791]: pgmap v365: 129 pgs: 129 active+clean; 65 MiB data, 886 MiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 397 KiB/s wr, 41 op/s 2024-04-24T09:03:14.618 INFO:tasks.workunit.client.0.smithi083.stdout: 1 67556 0.61 MB/sec execute 468 sec latency 626.595 ms 2024-04-24T09:03:15.618 INFO:tasks.workunit.client.0.smithi083.stdout: 1 67556 0.61 MB/sec execute 469 sec latency 1626.772 ms 2024-04-24T09:03:16.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:03:15 smithi083 ceph-mon[27907]: pgmap v366: 129 pgs: 129 active+clean; 65 MiB data, 886 MiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 388 KiB/s wr, 40 op/s 2024-04-24T09:03:16.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:03:15 smithi138 ceph-mon[31441]: pgmap v366: 129 pgs: 129 active+clean; 65 MiB data, 886 MiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 388 KiB/s wr, 40 op/s 2024-04-24T09:03:16.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:03:15 smithi189 ceph-mon[31791]: pgmap v366: 129 pgs: 129 active+clean; 65 MiB data, 886 MiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 388 KiB/s wr, 40 op/s 2024-04-24T09:03:16.618 INFO:tasks.workunit.client.0.smithi083.stdout: 1 67556 0.61 MB/sec execute 470 sec latency 2626.931 ms 2024-04-24T09:03:17.618 INFO:tasks.workunit.client.0.smithi083.stdout: 1 67556 0.61 MB/sec execute 471 sec latency 3627.096 ms 2024-04-24T09:03:17.892 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.892+0000 7fcefac00640 1 -- 172.21.15.83:0/2713464609 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcef4109540 msgr2=0x7fcef4111110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:17.893 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.892+0000 7fcefac00640 1 --2- 172.21.15.83:0/2713464609 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcef4109540 0x7fcef4111110 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7fceec009ca0 tx=0x7fceec02a8c0 comp rx=0 tx=0).stop 2024-04-24T09:03:17.893 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.892+0000 7fcefac00640 1 -- 172.21.15.83:0/2713464609 shutdown_connections 2024-04-24T09:03:17.893 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.892+0000 7fcefac00640 1 --2- 172.21.15.83:0/2713464609 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcef4109540 0x7fcef4111110 unknown :-1 s=CLOSED pgs=1869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:17.893 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.892+0000 7fcefac00640 1 --2- 172.21.15.83:0/2713464609 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcef4104e20 0x7fcef4108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:17.893 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.892+0000 7fcefac00640 1 --2- 172.21.15.83:0/2713464609 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcef4104470 0x7fcef4104850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:17.893 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.892+0000 7fcefac00640 1 -- 172.21.15.83:0/2713464609 >> 172.21.15.83:0/2713464609 conn(0x7fcef40ffcd0 msgr2=0x7fcef41020f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:17.893 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.892+0000 7fcefac00640 1 -- 172.21.15.83:0/2713464609 shutdown_connections 2024-04-24T09:03:17.893 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.892+0000 7fcefac00640 1 -- 172.21.15.83:0/2713464609 wait complete. 2024-04-24T09:03:17.894 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.893+0000 7fcefac00640 1 Processor -- start 2024-04-24T09:03:17.894 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.893+0000 7fcefac00640 1 -- start start 2024-04-24T09:03:17.895 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.894+0000 7fcefac00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcef4104470 0x7fcef41a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:17.895 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.894+0000 7fcefac00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcef4109540 0x7fcef41a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:17.895 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.894+0000 7fcefac00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcef419a7a0 0x7fcef419ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:17.895 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.894+0000 7fcefac00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcef4114500 con 0x7fcef4109540 2024-04-24T09:03:17.895 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.894+0000 7fcefac00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcef4114380 con 0x7fcef4104470 2024-04-24T09:03:17.895 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.894+0000 7fcefac00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fcef4114680 con 0x7fcef419a7a0 2024-04-24T09:03:17.895 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.894+0000 7fceebe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcef4109540 0x7fcef41a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:17.895 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.894+0000 7fceebe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcef4109540 0x7fcef41a0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45052/0 (socket says 172.21.15.83:45052) 2024-04-24T09:03:17.895 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.894+0000 7fceebe00640 1 -- 172.21.15.83:0/1844747730 learned_addr learned my addr 172.21.15.83:0/1844747730 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:03:17.895 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.894+0000 7fcef3e00640 1 --2- 172.21.15.83:0/1844747730 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcef4104470 0x7fcef41a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:17.895 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.894+0000 7fcef8c00640 1 --2- 172.21.15.83:0/1844747730 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcef419a7a0 0x7fcef419ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:17.895 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.895+0000 7fceebe00640 1 -- 172.21.15.83:0/1844747730 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcef419a7a0 msgr2=0x7fcef419ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:17.895 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.895+0000 7fceebe00640 1 --2- 172.21.15.83:0/1844747730 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcef419a7a0 0x7fcef419ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:17.896 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.895+0000 7fceebe00640 1 -- 172.21.15.83:0/1844747730 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcef4104470 msgr2=0x7fcef41a0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:17.896 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.895+0000 7fceebe00640 1 --2- 172.21.15.83:0/1844747730 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcef4104470 0x7fcef41a0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:17.896 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.895+0000 7fceebe00640 1 -- 172.21.15.83:0/1844747730 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcef419b3a0 con 0x7fcef4109540 2024-04-24T09:03:17.896 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.895+0000 7fceebe00640 1 --2- 172.21.15.83:0/1844747730 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcef4109540 0x7fcef41a0bd0 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7fcedc00b450 tx=0x7fcedc00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:17.897 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.895+0000 7fcef1600640 1 -- 172.21.15.83:0/1844747730 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcedc015020 con 0x7fcef4109540 2024-04-24T09:03:17.897 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.895+0000 7fcef1600640 1 -- 172.21.15.83:0/1844747730 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcedc011d40 con 0x7fcef4109540 2024-04-24T09:03:17.897 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.895+0000 7fcefac00640 1 -- 172.21.15.83:0/1844747730 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcef41a7430 con 0x7fcef4109540 2024-04-24T09:03:17.897 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.895+0000 7fcef1600640 1 -- 172.21.15.83:0/1844747730 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcedc00f340 con 0x7fcef4109540 2024-04-24T09:03:17.897 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.895+0000 7fcefac00640 1 -- 172.21.15.83:0/1844747730 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcef41a7890 con 0x7fcef4109540 2024-04-24T09:03:17.898 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.899+0000 7fcefda7f740 1 Processor -- start 2024-04-24T09:03:17.898 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.899+0000 7fcefda7f740 1 -- start start 2024-04-24T09:03:17.900 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.899+0000 7fcef1600640 1 -- 172.21.15.83:0/1844747730 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fcedc018020 con 0x7fcef4109540 2024-04-24T09:03:17.900 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.899+0000 7fcefda7f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5636cba1b1b0 0x5636cba1b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:17.900 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.899+0000 7fcefda7f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636cba1bad0 0x5636cba25e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:17.900 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.899+0000 7fcefda7f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5636cba26390 0x5636cba28780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:17.900 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.899+0000 7fcefda7f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5636cb981900 con 0x5636cba1bad0 2024-04-24T09:03:17.900 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.899+0000 7fcefda7f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5636cb927f20 con 0x5636cba26390 2024-04-24T09:03:17.900 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.899+0000 7fcefda7f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5636cb97d8c0 con 0x5636cba1b1b0 2024-04-24T09:03:17.900 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.899+0000 7fcef3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636cba1bad0 0x5636cba25e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:17.901 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.900+0000 7fcef3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636cba1bad0 0x5636cba25e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45054/0 (socket says 172.21.15.83:45054) 2024-04-24T09:03:17.901 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.900+0000 7fcef3e00640 1 -- 172.21.15.83:0/3879151935 learned_addr learned my addr 172.21.15.83:0/3879151935 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:03:17.901 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.900+0000 7fcef8c00640 1 --2- 172.21.15.83:0/3879151935 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5636cba1b1b0 0x5636cba1b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:17.901 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.900+0000 7fceebe00640 1 --2- 172.21.15.83:0/3879151935 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5636cba26390 0x5636cba28780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:17.901 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.900+0000 7fcef3e00640 1 -- 172.21.15.83:0/3879151935 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5636cba1b1b0 msgr2=0x5636cba1b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:17.901 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.900+0000 7fcef3e00640 1 --2- 172.21.15.83:0/3879151935 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5636cba1b1b0 0x5636cba1b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:17.901 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.900+0000 7fcef3e00640 1 -- 172.21.15.83:0/3879151935 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5636cba26390 msgr2=0x5636cba28780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:17.901 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.900+0000 7fcef3e00640 1 --2- 172.21.15.83:0/3879151935 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5636cba26390 0x5636cba28780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:17.901 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.900+0000 7fcef3e00640 1 -- 172.21.15.83:0/3879151935 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636cb94ce10 con 0x5636cba1bad0 2024-04-24T09:03:17.901 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.900+0000 7fcef8c00640 1 --2- 172.21.15.83:0/3879151935 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5636cba1b1b0 0x5636cba1b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:03:17.904 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.901+0000 7fcef3e00640 1 --2- 172.21.15.83:0/3879151935 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636cba1bad0 0x5636cba25e50 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7fcee400ac20 tx=0x7fcee400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a754274e6e7d415 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:17.904 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.901+0000 7fcee9200640 1 -- 172.21.15.83:0/3879151935 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcee401d070 con 0x5636cba1bad0 2024-04-24T09:03:17.904 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.901+0000 7fcee9200640 1 -- 172.21.15.83:0/3879151935 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcee4018af0 con 0x5636cba1bad0 2024-04-24T09:03:17.904 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.901+0000 7fcee9200640 1 -- 172.21.15.83:0/3879151935 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcee4010af0 con 0x5636cba1bad0 2024-04-24T09:03:17.904 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.901+0000 7fcef1600640 1 --2- 172.21.15.83:0/1844747730 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fced80a3e40 0x7fced80a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:17.904 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.901+0000 7fcef1600640 1 -- 172.21.15.83:0/1844747730 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9583+0+0 (secure 0 0 0) 0x7fcedc0d0530 con 0x7fcef4109540 2024-04-24T09:03:17.904 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.901+0000 7fcef8c00640 1 --2- 172.21.15.83:0/1844747730 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fced80a3e40 0x7fced80a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:17.904 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.902+0000 7fcefda7f740 1 -- 172.21.15.83:0/3879151935 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636cba1bad0 msgr2=0x5636cba25e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:17.904 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.902+0000 7fcefda7f740 1 --2- 172.21.15.83:0/3879151935 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636cba1bad0 0x5636cba25e50 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7fcee400ac20 tx=0x7fcee400f790 comp rx=0 tx=0).stop 2024-04-24T09:03:17.904 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.902+0000 7fcefda7f740 1 -- 172.21.15.83:0/3879151935 shutdown_connections 2024-04-24T09:03:17.904 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.902+0000 7fcefda7f740 1 --2- 172.21.15.83:0/3879151935 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5636cba26390 0x5636cba28780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:17.904 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.902+0000 7fcefda7f740 1 --2- 172.21.15.83:0/3879151935 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636cba1bad0 0x5636cba25e50 unknown :-1 s=CLOSED pgs=1871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:17.904 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.902+0000 7fcefda7f740 1 --2- 172.21.15.83:0/3879151935 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5636cba1b1b0 0x5636cba1b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:17.905 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.902+0000 7fcefda7f740 1 -- 172.21.15.83:0/3879151935 >> 172.21.15.83:0/3879151935 conn(0x5636cba176c0 msgr2=0x5636cba17aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:17.905 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.902+0000 7fcefda7f740 1 -- 172.21.15.83:0/3879151935 shutdown_connections 2024-04-24T09:03:17.905 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.902+0000 7fcefda7f740 1 -- 172.21.15.83:0/3879151935 wait complete. 2024-04-24T09:03:17.905 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.902+0000 7fcef8c00640 1 --2- 172.21.15.83:0/1844747730 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fced80a3e40 0x7fced80a6300 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fceec009430 tx=0x7fceec009800 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:17.905 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.903+0000 7fcefda7f740 1 -- start start 2024-04-24T09:03:17.905 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.903+0000 7fcefda7f740 1 Processor -- start 2024-04-24T09:03:17.905 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.903+0000 7fcefda7f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5636cba1b1b0 0x5636cba23310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:17.905 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.903+0000 7fcefda7f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5636cba1bad0 0x5636cba23850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:17.905 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.903+0000 7fcefda7f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636cba26390 0x5636cba23d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:17.905 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.903+0000 7fcefda7f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5636cba28fc0 con 0x5636cba26390 2024-04-24T09:03:17.905 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.903+0000 7fcefda7f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5636cb93b2c0 con 0x5636cba1b1b0 2024-04-24T09:03:17.905 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.903+0000 7fcefda7f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5636cba29140 con 0x5636cba1bad0 2024-04-24T09:03:17.905 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.903+0000 7fcefda7f740 10 client.0 _ll_register_callbacks cb 0x5636cb693a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:03:17.905 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.904+0000 7fcef8c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636cba26390 0x5636cba23d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:17.905 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.904+0000 7fcef8c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636cba26390 0x5636cba23d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45060/0 (socket says 172.21.15.83:45060) 2024-04-24T09:03:17.906 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.904+0000 7fcef8c00640 1 -- 172.21.15.83:0/858830975 learned_addr learned my addr 172.21.15.83:0/858830975 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:03:17.906 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.904+0000 7fceebe00640 1 --2- 172.21.15.83:0/858830975 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5636cba1bad0 0x5636cba23850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:17.906 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.904+0000 7fcef3e00640 1 --2- 172.21.15.83:0/858830975 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5636cba1b1b0 0x5636cba23310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:17.908 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.904+0000 7fcef8c00640 1 -- 172.21.15.83:0/858830975 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5636cba1bad0 msgr2=0x5636cba23850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:17.908 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.904+0000 7fcef8c00640 1 --2- 172.21.15.83:0/858830975 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5636cba1bad0 0x5636cba23850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:17.908 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.904+0000 7fcef8c00640 1 -- 172.21.15.83:0/858830975 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5636cba1b1b0 msgr2=0x5636cba23310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:17.908 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.904+0000 7fcef8c00640 1 --2- 172.21.15.83:0/858830975 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5636cba1b1b0 0x5636cba23310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:17.908 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.904+0000 7fcef8c00640 1 -- 172.21.15.83:0/858830975 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636cb94bfd0 con 0x5636cba26390 2024-04-24T09:03:17.908 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.905+0000 7fcef8c00640 1 --2- 172.21.15.83:0/858830975 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636cba26390 0x5636cba23d90 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7fceec031870 tx=0x7fceec0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea44807ac58f9558 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:17.908 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.905+0000 7fcee1600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:03:17.908 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.905+0000 7fcee1600640 1 -- 172.21.15.83:0/858830975 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceec04f070 con 0x5636cba26390 2024-04-24T09:03:17.908 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.905+0000 7fcee1600640 1 -- 172.21.15.83:0/858830975 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9583+0+0 (secure 0 0 0) 0x7fceec004450 con 0x5636cba26390 2024-04-24T09:03:17.908 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.905+0000 7fcefda7f740 1 -- 172.21.15.83:0/858830975 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcee4018af0 con 0x5636cba26390 2024-04-24T09:03:17.908 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.905+0000 7fcee1600640 1 -- 172.21.15.83:0/858830975 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fceec05b050 con 0x5636cba26390 2024-04-24T09:03:17.908 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.905+0000 7fcefda7f740 10 client.19776 fetch_fsmap learned FSMap version 41 2024-04-24T09:03:17.908 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.905+0000 7fcefda7f740 1 -- 172.21.15.83:0/858830975 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5636cb694210 con 0x5636cba26390 2024-04-24T09:03:17.908 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.906+0000 7fcee1600640 1 -- 172.21.15.83:0/858830975 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcee4018af0 con 0x5636cba26390 2024-04-24T09:03:17.909 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.906+0000 7fcee1600640 1 -- 172.21.15.83:0/858830975 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7fceec044040 con 0x5636cba26390 2024-04-24T09:03:17.909 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.906+0000 7fcefda7f740 10 client.19776 fetch_fsmap finished waiting for FSMap version 41 2024-04-24T09:03:17.909 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.906+0000 7fcefda7f740 10 client.19776 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:03:17.909 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.906+0000 7fcefda7f740 20 client.19776 populate_metadata read hostname 'smithi083' 2024-04-24T09:03:17.909 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.906+0000 7fcefda7f740 1 --2- 172.21.15.83:0/858830975 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5636cba36580 0x5636cba38a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:17.909 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.906+0000 7fcefda7f740 4 client.19776 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:03:17.909 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.906+0000 7fcefda7f740 1 -- 172.21.15.83:0/858830975 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5636cb98f320 con 0x5636cba36580 2024-04-24T09:03:17.909 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.907+0000 7fcef3e00640 1 --2- 172.21.15.83:0/858830975 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5636cba36580 0x5636cba38a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:17.909 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.908+0000 7fcef3e00640 1 --2- 172.21.15.83:0/858830975 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5636cba36580 0x5636cba38a00 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4492727242e6b7e3 server_cookie=c1db5ca849d617e6 in_seq=0 out_seq=0 2024-04-24T09:03:17.909 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.908+0000 7fcee1600640 10 client.19776 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:03:17.910 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.909+0000 7fcee1600640 1 -- 172.21.15.83:0/858830975 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5636cb98f320 con 0x5636cba36580 2024-04-24T09:03:17.910 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.909+0000 7fcee1600640 10 client.19776 handle_command_reply: tid=0 2024-04-24T09:03:17.910 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.909+0000 7fcefda7f740 1 client.19776 shutdown 2024-04-24T09:03:17.910 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.910+0000 7fcefda7f740 1 -- 172.21.15.83:0/858830975 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636cba26390 msgr2=0x5636cba23d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:17.911 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.910+0000 7fcefda7f740 1 --2- 172.21.15.83:0/858830975 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636cba26390 0x5636cba23d90 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7fceec031870 tx=0x7fceec0095c0 comp rx=0 tx=0).stop 2024-04-24T09:03:17.911 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.910+0000 7fcefda7f740 1 -- 172.21.15.83:0/858830975 shutdown_connections 2024-04-24T09:03:17.911 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.910+0000 7fcefda7f740 1 --2- 172.21.15.83:0/858830975 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5636cba36580 0x5636cba38a00 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:17.911 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.910+0000 7fcefda7f740 1 --2- 172.21.15.83:0/858830975 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636cba26390 0x5636cba23d90 unknown :-1 s=CLOSED pgs=1872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:17.911 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.910+0000 7fcefda7f740 1 --2- 172.21.15.83:0/858830975 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5636cba1bad0 0x5636cba23850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:17.911 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.910+0000 7fcee1600640 0 client.19776 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:03:17.911 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.910+0000 7fcefda7f740 1 --2- 172.21.15.83:0/858830975 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5636cba1b1b0 0x5636cba23310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:17.911 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.910+0000 7fcefda7f740 1 -- 172.21.15.83:0/858830975 >> 172.21.15.83:0/858830975 conn(0x5636cba176c0 msgr2=0x5636cba16840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:17.911 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.910+0000 7fcefda7f740 1 -- 172.21.15.83:0/858830975 shutdown_connections 2024-04-24T09:03:17.911 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.910+0000 7fcefda7f740 1 -- 172.21.15.83:0/858830975 wait complete. 2024-04-24T09:03:17.911 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.910+0000 7fcefda7f740 20 client.19776 trim_cache size 0 max 16384 2024-04-24T09:03:17.927 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.926+0000 7fcefda7f740 1 Processor -- start 2024-04-24T09:03:17.927 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.926+0000 7fcefda7f740 1 -- start start 2024-04-24T09:03:17.927 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.927+0000 7fcefda7f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636cba1b1b0 0x5636cba31e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:17.927 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.927+0000 7fcefda7f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5636cba1bad0 0x5636cba22c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:17.927 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.927+0000 7fcefda7f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5636cba26390 0x5636cba3e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:17.927 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.927+0000 7fcefda7f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcef41b16d0 con 0x5636cba1b1b0 2024-04-24T09:03:17.927 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.927+0000 7fcefda7f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcef41b3330 con 0x5636cba26390 2024-04-24T09:03:17.927 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.927+0000 7fcefda7f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fcef41b1da0 con 0x5636cba1bad0 2024-04-24T09:03:17.927 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.927+0000 7fcef3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636cba1b1b0 0x5636cba31e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:17.927 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.927+0000 7fcef3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636cba1b1b0 0x5636cba31e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45062/0 (socket says 172.21.15.83:45062) 2024-04-24T09:03:17.928 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.927+0000 7fcef3e00640 1 -- 172.21.15.83:0/3861010068 learned_addr learned my addr 172.21.15.83:0/3861010068 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:03:17.929 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.927+0000 7fcef3e00640 1 -- 172.21.15.83:0/3861010068 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5636cba1bad0 msgr2=0x5636cba22c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:17.929 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.927+0000 7fcef3e00640 1 --2- 172.21.15.83:0/3861010068 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5636cba1bad0 0x5636cba22c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:17.929 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.927+0000 7fcef3e00640 1 -- 172.21.15.83:0/3861010068 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5636cba26390 msgr2=0x5636cba3e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:17.930 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.927+0000 7fcef8c00640 1 --2- 172.21.15.83:0/3861010068 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5636cba26390 0x5636cba3e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:17.930 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.928+0000 7fcef3e00640 1 --2- 172.21.15.83:0/3861010068 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5636cba26390 0x5636cba3e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:17.930 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.928+0000 7fcef3e00640 1 -- 172.21.15.83:0/3861010068 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636cba1b900 con 0x5636cba1b1b0 2024-04-24T09:03:17.930 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.928+0000 7fcef8c00640 1 --2- 172.21.15.83:0/3861010068 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5636cba26390 0x5636cba3e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:03:17.930 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.928+0000 7fcef3e00640 1 --2- 172.21.15.83:0/3861010068 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636cba1b1b0 0x5636cba31e80 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7fcee400a790 tx=0x7fcee40058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b59be7b5c6a5e416 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:17.930 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.928+0000 7fcee3e00640 1 -- 172.21.15.83:0/3861010068 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcee401d040 con 0x5636cba1b1b0 2024-04-24T09:03:17.930 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.928+0000 7fcee3e00640 1 -- 172.21.15.83:0/3861010068 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcee401e040 con 0x5636cba1b1b0 2024-04-24T09:03:17.930 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.928+0000 7fcee3e00640 1 -- 172.21.15.83:0/3861010068 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcee4010ca0 con 0x5636cba1b1b0 2024-04-24T09:03:17.930 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.929+0000 7fcefda7f740 1 -- 172.21.15.83:0/3861010068 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636cba1b1b0 msgr2=0x5636cba31e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:17.930 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.929+0000 7fcefda7f740 1 --2- 172.21.15.83:0/3861010068 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636cba1b1b0 0x5636cba31e80 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7fcee400a790 tx=0x7fcee40058a0 comp rx=0 tx=0).stop 2024-04-24T09:03:17.930 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.929+0000 7fcefda7f740 1 -- 172.21.15.83:0/3861010068 shutdown_connections 2024-04-24T09:03:17.930 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.929+0000 7fcefda7f740 1 --2- 172.21.15.83:0/3861010068 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5636cba26390 0x5636cba3e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:17.930 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.929+0000 7fcefda7f740 1 --2- 172.21.15.83:0/3861010068 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5636cba1bad0 0x5636cba22c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:17.930 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.929+0000 7fcefda7f740 1 --2- 172.21.15.83:0/3861010068 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636cba1b1b0 0x5636cba31e80 unknown :-1 s=CLOSED pgs=1873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:17.931 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.929+0000 7fcefda7f740 1 -- 172.21.15.83:0/3861010068 >> 172.21.15.83:0/3861010068 conn(0x5636cba176c0 msgr2=0x5636cba253d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:17.931 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.929+0000 7fcefda7f740 1 -- 172.21.15.83:0/3861010068 shutdown_connections 2024-04-24T09:03:17.931 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.929+0000 7fcefda7f740 1 -- 172.21.15.83:0/3861010068 wait complete. 2024-04-24T09:03:17.931 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.929+0000 7fcefda7f740 1 -- start start 2024-04-24T09:03:17.931 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.930+0000 7fcefda7f740 1 Processor -- start 2024-04-24T09:03:17.933 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.930+0000 7fcefda7f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636cba1b1b0 0x5636cba202c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:17.933 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.930+0000 7fcefda7f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5636cba1bad0 0x5636cba20800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:17.933 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.930+0000 7fcefda7f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5636cba26390 0x5636cba20d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:17.934 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.930+0000 7fcefda7f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcef41b2c00 con 0x5636cba1b1b0 2024-04-24T09:03:17.934 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.930+0000 7fcefda7f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcef41b24d0 con 0x5636cba1bad0 2024-04-24T09:03:17.934 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.930+0000 7fcefda7f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fcef41af6d0 con 0x5636cba26390 2024-04-24T09:03:17.934 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.930+0000 7fcefda7f740 10 client.0 _ll_register_callbacks cb 0x7fceec009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:03:17.934 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.930+0000 7fcef3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636cba1b1b0 0x5636cba202c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:17.934 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.930+0000 7fcef3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636cba1b1b0 0x5636cba202c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45068/0 (socket says 172.21.15.83:45068) 2024-04-24T09:03:17.934 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.930+0000 7fcef3e00640 1 -- 172.21.15.83:0/2973329780 learned_addr learned my addr 172.21.15.83:0/2973329780 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:03:17.934 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.931+0000 7fceebe00640 1 --2- 172.21.15.83:0/2973329780 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5636cba1bad0 0x5636cba20800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:17.934 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.931+0000 7fcef8c00640 1 --2- 172.21.15.83:0/2973329780 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5636cba26390 0x5636cba20d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:17.934 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.931+0000 7fcef3e00640 1 -- 172.21.15.83:0/2973329780 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5636cba26390 msgr2=0x5636cba20d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:17.934 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.931+0000 7fcef3e00640 1 --2- 172.21.15.83:0/2973329780 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5636cba26390 0x5636cba20d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:17.935 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.931+0000 7fcef3e00640 1 -- 172.21.15.83:0/2973329780 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5636cba1bad0 msgr2=0x5636cba20800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:17.935 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.931+0000 7fcef3e00640 1 --2- 172.21.15.83:0/2973329780 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5636cba1bad0 0x5636cba20800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:17.935 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.931+0000 7fcef3e00640 1 -- 172.21.15.83:0/2973329780 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636cb760620 con 0x5636cba1b1b0 2024-04-24T09:03:17.935 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.931+0000 7fcef3e00640 1 --2- 172.21.15.83:0/2973329780 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636cba1b1b0 0x5636cba202c0 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7fcee4010940 tx=0x7fcee4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac812b5276847b7c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:17.935 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.931+0000 7fcee1600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:03:17.935 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.931+0000 7fcee1600640 1 -- 172.21.15.83:0/2973329780 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcee4039060 con 0x5636cba1b1b0 2024-04-24T09:03:17.935 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.931+0000 7fcee1600640 1 -- 172.21.15.83:0/2973329780 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9583+0+0 (secure 0 0 0) 0x7fcee4010df0 con 0x5636cba1b1b0 2024-04-24T09:03:17.935 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.931+0000 7fcefda7f740 1 -- 172.21.15.83:0/2973329780 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcee401e040 con 0x5636cba1b1b0 2024-04-24T09:03:17.935 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.932+0000 7fcee1600640 1 -- 172.21.15.83:0/2973329780 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fcee4040050 con 0x5636cba1b1b0 2024-04-24T09:03:17.935 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.932+0000 7fcefda7f740 10 client.19782 fetch_fsmap learned FSMap version 41 2024-04-24T09:03:17.935 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.932+0000 7fcefda7f740 1 -- 172.21.15.83:0/2973329780 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5636cbaa5680 con 0x5636cba1b1b0 2024-04-24T09:03:17.935 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.932+0000 7fcee1600640 1 -- 172.21.15.83:0/2973329780 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcee401e040 con 0x5636cba1b1b0 2024-04-24T09:03:17.935 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.932+0000 7fcee1600640 1 -- 172.21.15.83:0/2973329780 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7fcee4023040 con 0x5636cba1b1b0 2024-04-24T09:03:17.936 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.932+0000 7fcefda7f740 10 client.19782 fetch_fsmap finished waiting for FSMap version 41 2024-04-24T09:03:17.936 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.932+0000 7fcefda7f740 10 client.19782 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:03:17.936 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.932+0000 7fcefda7f740 20 client.19782 populate_metadata read hostname 'smithi083' 2024-04-24T09:03:17.936 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.932+0000 7fcefda7f740 1 --2- 172.21.15.83:0/2973329780 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5636cbaa5af0 0x5636cbaa7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:17.936 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.932+0000 7fcefda7f740 4 client.19782 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T09:03:17.936 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.932+0000 7fcefda7f740 1 -- 172.21.15.83:0/2973329780 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5636cb989190 con 0x5636cbaa5af0 2024-04-24T09:03:17.936 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.933+0000 7fceebe00640 1 --2- 172.21.15.83:0/2973329780 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5636cbaa5af0 0x5636cbaa7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:17.936 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.934+0000 7fceebe00640 1 --2- 172.21.15.83:0/2973329780 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5636cbaa5af0 0x5636cbaa7ee0 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a94bab75a89f6b4f server_cookie=675b7d8ab1b1dbcc in_seq=0 out_seq=0 2024-04-24T09:03:17.936 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:17.934+0000 7fcee1600640 10 client.19782 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:03:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:18.117+0000 7fcee1600640 1 -- 172.21.15.83:0/2973329780 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5636cb989190 con 0x5636cbaa5af0 2024-04-24T09:03:18.119 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:18.117+0000 7fcee1600640 10 client.19782 handle_command_reply: tid=0 2024-04-24T09:03:18.119 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:18.118+0000 7fcefda7f740 1 client.19782 shutdown 2024-04-24T09:03:18.119 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:18.118+0000 7fcefda7f740 1 -- 172.21.15.83:0/2973329780 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636cba1b1b0 msgr2=0x5636cba202c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:18.119 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:18.118+0000 7fcefda7f740 1 --2- 172.21.15.83:0/2973329780 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636cba1b1b0 0x5636cba202c0 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7fcee4010940 tx=0x7fcee4000bc0 comp rx=0 tx=0).stop 2024-04-24T09:03:18.119 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:18.118+0000 7fcefda7f740 1 -- 172.21.15.83:0/2973329780 shutdown_connections 2024-04-24T09:03:18.119 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:18.118+0000 7fcefda7f740 1 --2- 172.21.15.83:0/2973329780 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5636cbaa5af0 0x5636cbaa7ee0 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:18.119 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:18.118+0000 7fcefda7f740 1 --2- 172.21.15.83:0/2973329780 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5636cba26390 0x5636cba20d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:18.119 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:18.118+0000 7fcefda7f740 1 --2- 172.21.15.83:0/2973329780 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5636cba1bad0 0x5636cba20800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:18.119 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:18.118+0000 7fcee1600640 0 client.19782 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:03:18.119 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:18.118+0000 7fcefda7f740 1 --2- 172.21.15.83:0/2973329780 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636cba1b1b0 0x5636cba202c0 unknown :-1 s=CLOSED pgs=1874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:18.119 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:18.118+0000 7fcefda7f740 1 -- 172.21.15.83:0/2973329780 >> 172.21.15.83:0/2973329780 conn(0x5636cba176c0 msgr2=0x5636cba232d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:18.119 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:18.119+0000 7fcefda7f740 1 -- 172.21.15.83:0/2973329780 shutdown_connections 2024-04-24T09:03:18.119 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:18.119+0000 7fcefda7f740 1 -- 172.21.15.83:0/2973329780 wait complete. 2024-04-24T09:03:18.120 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:18.119+0000 7fcefda7f740 20 client.19782 trim_cache size 0 max 16384 2024-04-24T09:03:18.120 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T09:03:18.120 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T09:03:18.120 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T09:03:18.120 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T09:03:18.121 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:18.120+0000 7fcefac00640 1 -- 172.21.15.83:0/1844747730 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fced80a3e40 msgr2=0x7fced80a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:18.121 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:18.120+0000 7fcefac00640 1 --2- 172.21.15.83:0/1844747730 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fced80a3e40 0x7fced80a6300 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fceec009430 tx=0x7fceec009800 comp rx=0 tx=0).stop 2024-04-24T09:03:18.121 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:18.120+0000 7fcefac00640 1 -- 172.21.15.83:0/1844747730 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcef4109540 msgr2=0x7fcef41a0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:18.121 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:18.120+0000 7fcefac00640 1 --2- 172.21.15.83:0/1844747730 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcef4109540 0x7fcef41a0bd0 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7fcedc00b450 tx=0x7fcedc00b910 comp rx=0 tx=0).stop 2024-04-24T09:03:18.121 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:18.121+0000 7fcefac00640 1 -- 172.21.15.83:0/1844747730 shutdown_connections 2024-04-24T09:03:18.121 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:18.121+0000 7fcefac00640 1 --2- 172.21.15.83:0/1844747730 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fced80a3e40 0x7fced80a6300 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:18.121 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:18.121+0000 7fcefac00640 1 --2- 172.21.15.83:0/1844747730 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcef419a7a0 0x7fcef419ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:18.121 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:18.121+0000 7fcefac00640 1 --2- 172.21.15.83:0/1844747730 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcef4109540 0x7fcef41a0bd0 unknown :-1 s=CLOSED pgs=1870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:18.121 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:18.121+0000 7fcefac00640 1 --2- 172.21.15.83:0/1844747730 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcef4104470 0x7fcef41a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:18.121 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:18.121+0000 7fcefac00640 1 -- 172.21.15.83:0/1844747730 >> 172.21.15.83:0/1844747730 conn(0x7fcef40ffcd0 msgr2=0x7fcef4100820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:18.122 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:18.121+0000 7fcefac00640 1 -- 172.21.15.83:0/1844747730 shutdown_connections 2024-04-24T09:03:18.122 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:18.121+0000 7fcefac00640 1 -- 172.21.15.83:0/1844747730 wait complete. 2024-04-24T09:03:18.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:03:18 smithi083 ceph-mon[27907]: pgmap v367: 129 pgs: 129 active+clean; 65 MiB data, 886 MiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 388 KiB/s wr, 40 op/s 2024-04-24T09:03:18.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:03:18 smithi138 ceph-mon[31441]: pgmap v367: 129 pgs: 129 active+clean; 65 MiB data, 886 MiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 388 KiB/s wr, 40 op/s 2024-04-24T09:03:18.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:03:18 smithi189 ceph-mon[31791]: pgmap v367: 129 pgs: 129 active+clean; 65 MiB data, 886 MiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 388 KiB/s wr, 40 op/s 2024-04-24T09:03:18.618 INFO:tasks.workunit.client.0.smithi083.stdout: 1 67621 0.61 MB/sec execute 472 sec latency 4112.801 ms 2024-04-24T09:03:19.619 INFO:tasks.workunit.client.0.smithi083.stdout: 1 67675 0.61 MB/sec execute 473 sec latency 55.481 ms 2024-04-24T09:03:20.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:03:20 smithi083 ceph-mon[27907]: pgmap v368: 129 pgs: 129 active+clean; 65 MiB data, 887 MiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 81 KiB/s wr, 27 op/s 2024-04-24T09:03:20.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:03:20 smithi138 ceph-mon[31441]: pgmap v368: 129 pgs: 129 active+clean; 65 MiB data, 887 MiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 81 KiB/s wr, 27 op/s 2024-04-24T09:03:20.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:03:20 smithi189 ceph-mon[31791]: pgmap v368: 129 pgs: 129 active+clean; 65 MiB data, 887 MiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 81 KiB/s wr, 27 op/s 2024-04-24T09:03:20.619 INFO:tasks.workunit.client.0.smithi083.stdout: 1 67735 0.60 MB/sec execute 474 sec latency 46.682 ms 2024-04-24T09:03:21.619 INFO:tasks.workunit.client.0.smithi083.stdout: 1 67802 0.60 MB/sec execute 475 sec latency 51.441 ms 2024-04-24T09:03:22.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:03:22 smithi083 ceph-mon[27907]: pgmap v369: 129 pgs: 129 active+clean; 65 MiB data, 887 MiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 73 KiB/s wr, 24 op/s 2024-04-24T09:03:22.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:03:22 smithi138 ceph-mon[31441]: pgmap v369: 129 pgs: 129 active+clean; 65 MiB data, 887 MiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 73 KiB/s wr, 24 op/s 2024-04-24T09:03:22.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:03:22 smithi189 ceph-mon[31791]: pgmap v369: 129 pgs: 129 active+clean; 65 MiB data, 887 MiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 73 KiB/s wr, 24 op/s 2024-04-24T09:03:22.619 INFO:tasks.workunit.client.0.smithi083.stdout: 1 67859 0.60 MB/sec execute 476 sec latency 44.467 ms 2024-04-24T09:03:23.619 INFO:tasks.workunit.client.0.smithi083.stdout: 1 67919 0.60 MB/sec execute 477 sec latency 52.690 ms 2024-04-24T09:03:24.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:03:24 smithi083 ceph-mon[27907]: pgmap v370: 129 pgs: 129 active+clean; 67 MiB data, 891 MiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 281 KiB/s wr, 48 op/s 2024-04-24T09:03:24.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:03:24 smithi138 ceph-mon[31441]: pgmap v370: 129 pgs: 129 active+clean; 67 MiB data, 891 MiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 281 KiB/s wr, 48 op/s 2024-04-24T09:03:24.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:03:24 smithi189 ceph-mon[31791]: pgmap v370: 129 pgs: 129 active+clean; 67 MiB data, 891 MiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 281 KiB/s wr, 48 op/s 2024-04-24T09:03:24.619 INFO:tasks.workunit.client.0.smithi083.stdout: 1 67995 0.60 MB/sec execute 478 sec latency 50.967 ms 2024-04-24T09:03:25.619 INFO:tasks.workunit.client.0.smithi083.stdout: 1 68082 0.60 MB/sec execute 479 sec latency 63.538 ms 2024-04-24T09:03:26.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:03:26 smithi083 ceph-mon[27907]: pgmap v371: 129 pgs: 129 active+clean; 67 MiB data, 891 MiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 244 KiB/s wr, 35 op/s 2024-04-24T09:03:26.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:03:26 smithi138 ceph-mon[31441]: pgmap v371: 129 pgs: 129 active+clean; 67 MiB data, 891 MiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 244 KiB/s wr, 35 op/s 2024-04-24T09:03:26.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:03:26 smithi189 ceph-mon[31791]: pgmap v371: 129 pgs: 129 active+clean; 67 MiB data, 891 MiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 244 KiB/s wr, 35 op/s 2024-04-24T09:03:26.619 INFO:tasks.workunit.client.0.smithi083.stdout: 1 68206 0.60 MB/sec execute 480 sec latency 55.598 ms 2024-04-24T09:03:27.619 INFO:tasks.workunit.client.0.smithi083.stdout: 1 68313 0.60 MB/sec execute 481 sec latency 76.885 ms 2024-04-24T09:03:28.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:03:28 smithi083 ceph-mon[27907]: pgmap v372: 129 pgs: 129 active+clean; 67 MiB data, 891 MiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 244 KiB/s wr, 35 op/s 2024-04-24T09:03:28.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:03:28 smithi138 ceph-mon[31441]: pgmap v372: 129 pgs: 129 active+clean; 67 MiB data, 891 MiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 244 KiB/s wr, 35 op/s 2024-04-24T09:03:28.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:03:28 smithi189 ceph-mon[31791]: pgmap v372: 129 pgs: 129 active+clean; 67 MiB data, 891 MiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 244 KiB/s wr, 35 op/s 2024-04-24T09:03:28.619 INFO:tasks.workunit.client.0.smithi083.stdout: 1 68423 0.60 MB/sec execute 482 sec latency 51.504 ms 2024-04-24T09:03:29.620 INFO:tasks.workunit.client.0.smithi083.stdout: 1 68542 0.60 MB/sec execute 483 sec latency 53.724 ms 2024-04-24T09:03:30.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:03:30 smithi083 ceph-mon[27907]: pgmap v373: 129 pgs: 129 active+clean; 66 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 370 KiB/s wr, 62 op/s 2024-04-24T09:03:30.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:03:30 smithi138 ceph-mon[31441]: pgmap v373: 129 pgs: 129 active+clean; 66 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 370 KiB/s wr, 62 op/s 2024-04-24T09:03:30.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:03:30 smithi189 ceph-mon[31791]: pgmap v373: 129 pgs: 129 active+clean; 66 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 370 KiB/s wr, 62 op/s 2024-04-24T09:03:30.620 INFO:tasks.workunit.client.0.smithi083.stdout: 1 68641 0.60 MB/sec execute 484 sec latency 49.925 ms 2024-04-24T09:03:31.286 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.286+0000 7ff73d600640 1 -- 172.21.15.83:0/958327188 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff738108fb0 msgr2=0x7ff7381154c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:31.287 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.286+0000 7ff73d600640 1 --2- 172.21.15.83:0/958327188 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff738108fb0 0x7ff7381154c0 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7ff730009cd0 tx=0x7ff73002a8e0 comp rx=0 tx=0).stop 2024-04-24T09:03:31.287 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.287+0000 7ff73d600640 1 -- 172.21.15.83:0/958327188 shutdown_connections 2024-04-24T09:03:31.287 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.287+0000 7ff73d600640 1 --2- 172.21.15.83:0/958327188 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff738108fb0 0x7ff7381154c0 unknown :-1 s=CLOSED pgs=1875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:31.287 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.287+0000 7ff73d600640 1 --2- 172.21.15.83:0/958327188 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff7381085f0 0x7ff738108a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:31.287 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.287+0000 7ff73d600640 1 --2- 172.21.15.83:0/958327188 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff73810e690 0x7ff73810ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:31.287 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.287+0000 7ff73d600640 1 -- 172.21.15.83:0/958327188 >> 172.21.15.83:0/958327188 conn(0x7ff738076590 msgr2=0x7ff7380789b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:31.287 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.287+0000 7ff73d600640 1 -- 172.21.15.83:0/958327188 shutdown_connections 2024-04-24T09:03:31.287 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.287+0000 7ff73d600640 1 -- 172.21.15.83:0/958327188 wait complete. 2024-04-24T09:03:31.288 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.288+0000 7ff73d600640 1 Processor -- start 2024-04-24T09:03:31.288 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.288+0000 7ff73d600640 1 -- start start 2024-04-24T09:03:31.289 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.288+0000 7ff73d600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7381085f0 0x7ff7381a4920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:31.289 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.288+0000 7ff73d600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff738108fb0 0x7ff7381a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:31.289 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.288+0000 7ff73d600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff73810e690 0x7ff73819eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:31.289 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.288+0000 7ff73d600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff7381188c0 con 0x7ff73810e690 2024-04-24T09:03:31.290 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.288+0000 7ff73d600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff738118740 con 0x7ff7381085f0 2024-04-24T09:03:31.290 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.288+0000 7ff73d600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff738118a40 con 0x7ff738108fb0 2024-04-24T09:03:31.290 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.289+0000 7ff737400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff73810e690 0x7ff73819eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:31.290 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.289+0000 7ff737400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff73810e690 0x7ff73819eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56174/0 (socket says 172.21.15.83:56174) 2024-04-24T09:03:31.290 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.289+0000 7ff737400640 1 -- 172.21.15.83:0/837914228 learned_addr learned my addr 172.21.15.83:0/837914228 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:03:31.290 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.289+0000 7ff736000640 1 --2- 172.21.15.83:0/837914228 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff738108fb0 0x7ff7381a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:31.290 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.289+0000 7ff736a00640 1 --2- 172.21.15.83:0/837914228 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7381085f0 0x7ff7381a4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:31.290 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.289+0000 7ff737400640 1 -- 172.21.15.83:0/837914228 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff738108fb0 msgr2=0x7ff7381a4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:31.290 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.289+0000 7ff737400640 1 --2- 172.21.15.83:0/837914228 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff738108fb0 0x7ff7381a4e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:31.290 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.289+0000 7ff737400640 1 -- 172.21.15.83:0/837914228 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7381085f0 msgr2=0x7ff7381a4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:31.290 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.289+0000 7ff737400640 1 --2- 172.21.15.83:0/837914228 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7381085f0 0x7ff7381a4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:31.290 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.289+0000 7ff737400640 1 -- 172.21.15.83:0/837914228 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff73819f2a0 con 0x7ff73810e690 2024-04-24T09:03:31.290 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.289+0000 7ff736a00640 1 --2- 172.21.15.83:0/837914228 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7381085f0 0x7ff7381a4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:03:31.291 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.290+0000 7ff737400640 1 --2- 172.21.15.83:0/837914228 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff73810e690 0x7ff73819eb20 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7ff730033040 tx=0x7ff730009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:31.291 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.290+0000 7ff72f400640 1 -- 172.21.15.83:0/837914228 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff730004040 con 0x7ff73810e690 2024-04-24T09:03:31.291 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.290+0000 7ff73d600640 1 -- 172.21.15.83:0/837914228 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff73819f530 con 0x7ff73810e690 2024-04-24T09:03:31.291 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.290+0000 7ff72f400640 1 -- 172.21.15.83:0/837914228 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff730004690 con 0x7ff73810e690 2024-04-24T09:03:31.291 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.290+0000 7ff72f400640 1 -- 172.21.15.83:0/837914228 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff730030da0 con 0x7ff73810e690 2024-04-24T09:03:31.291 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.290+0000 7ff73d600640 1 -- 172.21.15.83:0/837914228 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7381ab7e0 con 0x7ff73810e690 2024-04-24T09:03:31.293 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.293+0000 7ff7403ea740 1 Processor -- start 2024-04-24T09:03:31.293 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.294+0000 7ff72f400640 1 -- 172.21.15.83:0/837914228 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff7300041e0 con 0x7ff73810e690 2024-04-24T09:03:31.294 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.294+0000 7ff7403ea740 1 -- start start 2024-04-24T09:03:31.295 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.294+0000 7ff7403ea740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e3e38e1b0 0x560e3e38e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:31.295 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.294+0000 7ff7403ea740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e3e38ead0 0x560e3e398e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:31.295 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.294+0000 7ff7403ea740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e3e399390 0x560e3e39b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:31.295 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.294+0000 7ff7403ea740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x560e3e2f4900 con 0x560e3e38e1b0 2024-04-24T09:03:31.295 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.294+0000 7ff7403ea740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560e3e29af20 con 0x560e3e399390 2024-04-24T09:03:31.295 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.294+0000 7ff7403ea740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x560e3e2f08c0 con 0x560e3e38ead0 2024-04-24T09:03:31.296 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.294+0000 7ff736000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e3e38e1b0 0x560e3e38e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:31.296 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.295+0000 7ff736000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e3e38e1b0 0x560e3e38e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56184/0 (socket says 172.21.15.83:56184) 2024-04-24T09:03:31.296 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.295+0000 7ff736000640 1 -- 172.21.15.83:0/4122996867 learned_addr learned my addr 172.21.15.83:0/4122996867 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:03:31.296 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.295+0000 7ff736a00640 1 --2- 172.21.15.83:0/4122996867 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e3e38ead0 0x560e3e398e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:31.296 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.295+0000 7ff736000640 1 -- 172.21.15.83:0/4122996867 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e3e38ead0 msgr2=0x560e3e398e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:31.296 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.295+0000 7ff736000640 1 --2- 172.21.15.83:0/4122996867 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e3e38ead0 0x560e3e398e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:31.296 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.295+0000 7ff736000640 1 -- 172.21.15.83:0/4122996867 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e3e399390 msgr2=0x560e3e39b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:31.296 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.295+0000 7ff736000640 1 --2- 172.21.15.83:0/4122996867 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e3e399390 0x560e3e39b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:31.296 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.295+0000 7ff736000640 1 -- 172.21.15.83:0/4122996867 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e3e2bfe10 con 0x560e3e38e1b0 2024-04-24T09:03:31.296 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.295+0000 7ff736a00640 1 --2- 172.21.15.83:0/4122996867 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e3e38ead0 0x560e3e398e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:03:31.296 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.295+0000 7ff736000640 1 --2- 172.21.15.83:0/4122996867 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e3e38e1b0 0x560e3e38e590 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7ff724009c70 tx=0x7ff7240137d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c23e69761607c66 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:31.296 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.296+0000 7ff72f400640 1 --2- 172.21.15.83:0/837914228 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff71c0a3dc0 0x7ff71c0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:31.299 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.296+0000 7ff723400640 1 -- 172.21.15.83:0/4122996867 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff724013070 con 0x560e3e38e1b0 2024-04-24T09:03:31.299 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.296+0000 7ff723400640 1 -- 172.21.15.83:0/4122996867 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff724017070 con 0x560e3e38e1b0 2024-04-24T09:03:31.299 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.296+0000 7ff723400640 1 -- 172.21.15.83:0/4122996867 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff724003c90 con 0x560e3e38e1b0 2024-04-24T09:03:31.299 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.296+0000 7ff72f400640 1 -- 172.21.15.83:0/837914228 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9583+0+0 (secure 0 0 0) 0x7ff73002e030 con 0x7ff73810e690 2024-04-24T09:03:31.300 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.296+0000 7ff736a00640 1 --2- 172.21.15.83:0/837914228 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff71c0a3dc0 0x7ff71c0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:31.300 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.297+0000 7ff7403ea740 1 -- 172.21.15.83:0/4122996867 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e3e38e1b0 msgr2=0x560e3e38e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:31.300 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.297+0000 7ff736a00640 1 --2- 172.21.15.83:0/837914228 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff71c0a3dc0 0x7ff71c0a6280 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x560e3e38f750 tx=0x7ff728005f90 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:31.300 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.297+0000 7ff7403ea740 1 --2- 172.21.15.83:0/4122996867 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e3e38e1b0 0x560e3e38e590 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7ff724009c70 tx=0x7ff7240137d0 comp rx=0 tx=0).stop 2024-04-24T09:03:31.300 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.297+0000 7ff7403ea740 1 -- 172.21.15.83:0/4122996867 shutdown_connections 2024-04-24T09:03:31.300 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.297+0000 7ff7403ea740 1 --2- 172.21.15.83:0/4122996867 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e3e399390 0x560e3e39b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:31.300 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.297+0000 7ff7403ea740 1 --2- 172.21.15.83:0/4122996867 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e3e38ead0 0x560e3e398e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:31.300 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.297+0000 7ff7403ea740 1 --2- 172.21.15.83:0/4122996867 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e3e38e1b0 0x560e3e38e590 unknown :-1 s=CLOSED pgs=1877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:31.300 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.297+0000 7ff7403ea740 1 -- 172.21.15.83:0/4122996867 >> 172.21.15.83:0/4122996867 conn(0x560e3e38a6c0 msgr2=0x560e3e38aaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:31.300 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.297+0000 7ff7403ea740 1 -- 172.21.15.83:0/4122996867 shutdown_connections 2024-04-24T09:03:31.300 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.297+0000 7ff7403ea740 1 -- 172.21.15.83:0/4122996867 wait complete. 2024-04-24T09:03:31.301 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.297+0000 7ff7403ea740 1 -- start start 2024-04-24T09:03:31.301 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.298+0000 7ff7403ea740 1 Processor -- start 2024-04-24T09:03:31.301 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.298+0000 7ff7403ea740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e3e38e1b0 0x560e3e396240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:31.301 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.298+0000 7ff7403ea740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e3e38ead0 0x560e3e396780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:31.301 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.298+0000 7ff7403ea740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e3e399390 0x560e3e396cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:31.301 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.298+0000 7ff7403ea740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x560e3e39bfc0 con 0x560e3e38e1b0 2024-04-24T09:03:31.301 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.298+0000 7ff7403ea740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560e3e2ae2c0 con 0x560e3e399390 2024-04-24T09:03:31.301 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.298+0000 7ff7403ea740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x560e3e39c140 con 0x560e3e38ead0 2024-04-24T09:03:31.301 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.298+0000 7ff7403ea740 10 client.0 _ll_register_callbacks cb 0x560e3e006a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:03:31.301 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.298+0000 7ff736a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e3e38e1b0 0x560e3e396240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:31.301 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.299+0000 7ff736a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e3e38e1b0 0x560e3e396240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56188/0 (socket says 172.21.15.83:56188) 2024-04-24T09:03:31.302 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.299+0000 7ff736a00640 1 -- 172.21.15.83:0/1641896703 learned_addr learned my addr 172.21.15.83:0/1641896703 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:03:31.302 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.299+0000 7ff736000640 1 --2- 172.21.15.83:0/1641896703 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e3e38ead0 0x560e3e396780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:31.302 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.299+0000 7ff737400640 1 --2- 172.21.15.83:0/1641896703 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e3e399390 0x560e3e396cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:31.302 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.299+0000 7ff736a00640 1 -- 172.21.15.83:0/1641896703 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e3e38ead0 msgr2=0x560e3e396780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:31.302 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.299+0000 7ff736a00640 1 --2- 172.21.15.83:0/1641896703 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e3e38ead0 0x560e3e396780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:31.302 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.299+0000 7ff736a00640 1 -- 172.21.15.83:0/1641896703 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e3e399390 msgr2=0x560e3e396cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:31.302 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.299+0000 7ff736a00640 1 --2- 172.21.15.83:0/1641896703 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e3e399390 0x560e3e396cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:31.302 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.299+0000 7ff736a00640 1 -- 172.21.15.83:0/1641896703 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e3e2befd0 con 0x560e3e38e1b0 2024-04-24T09:03:31.302 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.299+0000 7ff737400640 1 --2- 172.21.15.83:0/1641896703 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e3e399390 0x560e3e396cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:03:31.302 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.299+0000 7ff736a00640 1 --2- 172.21.15.83:0/1641896703 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e3e38e1b0 0x560e3e396240 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7ff72801eba0 tx=0x7ff7280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2361d82c4be7a84f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:31.302 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.299+0000 7ff717e00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:03:31.304 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.300+0000 7ff717e00640 1 -- 172.21.15.83:0/1641896703 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff728010cf0 con 0x560e3e38e1b0 2024-04-24T09:03:31.304 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.300+0000 7ff717e00640 1 -- 172.21.15.83:0/1641896703 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9583+0+0 (secure 0 0 0) 0x7ff72801f420 con 0x560e3e38e1b0 2024-04-24T09:03:31.304 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.300+0000 7ff7403ea740 1 -- 172.21.15.83:0/1641896703 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff724017070 con 0x560e3e38e1b0 2024-04-24T09:03:31.304 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.300+0000 7ff717e00640 1 -- 172.21.15.83:0/1641896703 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7ff728032050 con 0x560e3e38e1b0 2024-04-24T09:03:31.304 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.300+0000 7ff7403ea740 10 client.19794 fetch_fsmap learned FSMap version 41 2024-04-24T09:03:31.304 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.300+0000 7ff7403ea740 1 -- 172.21.15.83:0/1641896703 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x560e3e007210 con 0x560e3e38e1b0 2024-04-24T09:03:31.304 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.301+0000 7ff717e00640 1 -- 172.21.15.83:0/1641896703 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff724017070 con 0x560e3e38e1b0 2024-04-24T09:03:31.304 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.301+0000 7ff717e00640 1 -- 172.21.15.83:0/1641896703 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7ff72800f040 con 0x560e3e38e1b0 2024-04-24T09:03:31.304 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.301+0000 7ff7403ea740 10 client.19794 fetch_fsmap finished waiting for FSMap version 41 2024-04-24T09:03:31.304 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.301+0000 7ff7403ea740 10 client.19794 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:03:31.304 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.301+0000 7ff7403ea740 20 client.19794 populate_metadata read hostname 'smithi083' 2024-04-24T09:03:31.305 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.301+0000 7ff7403ea740 1 --2- 172.21.15.83:0/1641896703 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x560e3e3a95b0 0x560e3e3aba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:31.305 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.301+0000 7ff7403ea740 4 client.19794 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:03:31.305 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.301+0000 7ff7403ea740 1 -- 172.21.15.83:0/1641896703 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560e3e302320 con 0x560e3e3a95b0 2024-04-24T09:03:31.305 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.302+0000 7ff736000640 1 --2- 172.21.15.83:0/1641896703 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x560e3e3a95b0 0x560e3e3aba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:31.305 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.303+0000 7ff736000640 1 --2- 172.21.15.83:0/1641896703 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x560e3e3a95b0 0x560e3e3aba10 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4930ef8032b50cac server_cookie=42cc170cbecb5a3a in_seq=0 out_seq=0 2024-04-24T09:03:31.305 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.303+0000 7ff717e00640 10 client.19794 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:03:31.307 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.306+0000 7ff717e00640 1 -- 172.21.15.83:0/1641896703 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560e3e302320 con 0x560e3e3a95b0 2024-04-24T09:03:31.307 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.306+0000 7ff717e00640 10 client.19794 handle_command_reply: tid=0 2024-04-24T09:03:31.307 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.306+0000 7ff7403ea740 1 client.19794 shutdown 2024-04-24T09:03:31.307 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.306+0000 7ff7403ea740 1 -- 172.21.15.83:0/1641896703 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e3e38e1b0 msgr2=0x560e3e396240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:31.307 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.306+0000 7ff7403ea740 1 --2- 172.21.15.83:0/1641896703 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e3e38e1b0 0x560e3e396240 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7ff72801eba0 tx=0x7ff7280046c0 comp rx=0 tx=0).stop 2024-04-24T09:03:31.307 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.306+0000 7ff7403ea740 1 -- 172.21.15.83:0/1641896703 shutdown_connections 2024-04-24T09:03:31.307 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.306+0000 7ff7403ea740 1 --2- 172.21.15.83:0/1641896703 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x560e3e3a95b0 0x560e3e3aba10 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:31.307 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.306+0000 7ff7403ea740 1 --2- 172.21.15.83:0/1641896703 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e3e399390 0x560e3e396cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:31.307 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.306+0000 7ff7403ea740 1 --2- 172.21.15.83:0/1641896703 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e3e38ead0 0x560e3e396780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:31.307 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.306+0000 7ff7403ea740 1 --2- 172.21.15.83:0/1641896703 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e3e38e1b0 0x560e3e396240 unknown :-1 s=CLOSED pgs=1878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:31.307 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.306+0000 7ff7403ea740 1 -- 172.21.15.83:0/1641896703 >> 172.21.15.83:0/1641896703 conn(0x560e3e38a6c0 msgr2=0x560e3e3903f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:31.307 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.306+0000 7ff717e00640 0 client.19794 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:03:31.307 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.306+0000 7ff7403ea740 1 -- 172.21.15.83:0/1641896703 shutdown_connections 2024-04-24T09:03:31.308 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.306+0000 7ff7403ea740 1 -- 172.21.15.83:0/1641896703 wait complete. 2024-04-24T09:03:31.308 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.307+0000 7ff7403ea740 20 client.19794 trim_cache size 0 max 16384 2024-04-24T09:03:31.329 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.327+0000 7ff7403ea740 1 Processor -- start 2024-04-24T09:03:31.329 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.327+0000 7ff7403ea740 1 -- start start 2024-04-24T09:03:31.329 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.328+0000 7ff7403ea740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e3e38e1b0 0x560e3e390b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:31.329 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.328+0000 7ff7403ea740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e3e38ead0 0x560e3e3b1320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:31.329 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.328+0000 7ff7403ea740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e3e399390 0x560e3e3b1860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:31.329 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.328+0000 7ff7403ea740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff7381b5a00 con 0x560e3e38e1b0 2024-04-24T09:03:31.329 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.328+0000 7ff7403ea740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff7381b7660 con 0x560e3e399390 2024-04-24T09:03:31.329 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.328+0000 7ff7403ea740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff7381b60d0 con 0x560e3e38ead0 2024-04-24T09:03:31.330 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.328+0000 7ff736a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e3e38e1b0 0x560e3e390b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:31.330 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.328+0000 7ff736a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e3e38e1b0 0x560e3e390b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56194/0 (socket says 172.21.15.83:56194) 2024-04-24T09:03:31.330 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.328+0000 7ff736a00640 1 -- 172.21.15.83:0/2034092110 learned_addr learned my addr 172.21.15.83:0/2034092110 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:03:31.330 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.328+0000 7ff736000640 1 --2- 172.21.15.83:0/2034092110 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e3e38ead0 0x560e3e3b1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:31.330 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.328+0000 7ff737400640 1 --2- 172.21.15.83:0/2034092110 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e3e399390 0x560e3e3b1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:31.330 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.329+0000 7ff736a00640 1 -- 172.21.15.83:0/2034092110 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e3e38ead0 msgr2=0x560e3e3b1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:31.330 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.329+0000 7ff736a00640 1 --2- 172.21.15.83:0/2034092110 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e3e38ead0 0x560e3e3b1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:31.330 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.329+0000 7ff736a00640 1 -- 172.21.15.83:0/2034092110 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e3e399390 msgr2=0x560e3e3b1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:31.330 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.329+0000 7ff736a00640 1 --2- 172.21.15.83:0/2034092110 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e3e399390 0x560e3e3b1860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:31.330 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.329+0000 7ff736a00640 1 -- 172.21.15.83:0/2034092110 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e3e0d3620 con 0x560e3e38e1b0 2024-04-24T09:03:31.330 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.329+0000 7ff736000640 1 --2- 172.21.15.83:0/2034092110 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e3e38ead0 0x560e3e3b1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:03:31.330 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.329+0000 7ff736a00640 1 --2- 172.21.15.83:0/2034092110 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e3e38e1b0 0x560e3e390b70 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7ff72801c650 tx=0x7ff728009640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b54817e64b2f0d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:31.331 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.329+0000 7ff722a00640 1 -- 172.21.15.83:0/2034092110 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff72801e750 con 0x560e3e38e1b0 2024-04-24T09:03:31.331 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.329+0000 7ff722a00640 1 -- 172.21.15.83:0/2034092110 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff72803b070 con 0x560e3e38e1b0 2024-04-24T09:03:31.331 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.329+0000 7ff722a00640 1 -- 172.21.15.83:0/2034092110 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7280053b0 con 0x560e3e38e1b0 2024-04-24T09:03:31.334 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.330+0000 7ff7403ea740 1 -- 172.21.15.83:0/2034092110 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e3e38e1b0 msgr2=0x560e3e390b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:31.334 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.330+0000 7ff7403ea740 1 --2- 172.21.15.83:0/2034092110 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e3e38e1b0 0x560e3e390b70 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7ff72801c650 tx=0x7ff728009640 comp rx=0 tx=0).stop 2024-04-24T09:03:31.334 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.330+0000 7ff7403ea740 1 -- 172.21.15.83:0/2034092110 shutdown_connections 2024-04-24T09:03:31.334 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.330+0000 7ff7403ea740 1 --2- 172.21.15.83:0/2034092110 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e3e399390 0x560e3e3b1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:31.334 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.330+0000 7ff7403ea740 1 --2- 172.21.15.83:0/2034092110 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e3e38ead0 0x560e3e3b1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:31.334 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.330+0000 7ff7403ea740 1 --2- 172.21.15.83:0/2034092110 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e3e38e1b0 0x560e3e390b70 unknown :-1 s=CLOSED pgs=1879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:31.334 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.330+0000 7ff7403ea740 1 -- 172.21.15.83:0/2034092110 >> 172.21.15.83:0/2034092110 conn(0x560e3e38a6c0 msgr2=0x560e3e3915d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:31.334 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.330+0000 7ff7403ea740 1 -- 172.21.15.83:0/2034092110 shutdown_connections 2024-04-24T09:03:31.334 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.330+0000 7ff7403ea740 1 -- 172.21.15.83:0/2034092110 wait complete. 2024-04-24T09:03:31.335 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.330+0000 7ff7403ea740 1 -- start start 2024-04-24T09:03:31.335 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.331+0000 7ff7403ea740 1 Processor -- start 2024-04-24T09:03:31.335 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.331+0000 7ff7403ea740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e3e38e1b0 0x560e3e39c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:31.335 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.331+0000 7ff7403ea740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e3e38ead0 0x560e3e39c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:31.335 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.331+0000 7ff7403ea740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e3e399390 0x560e3e39cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:31.335 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.331+0000 7ff7403ea740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff7381b6f30 con 0x560e3e38ead0 2024-04-24T09:03:31.335 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.331+0000 7ff7403ea740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff7381b6800 con 0x560e3e38e1b0 2024-04-24T09:03:31.335 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.331+0000 7ff7403ea740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff7381b3a00 con 0x560e3e399390 2024-04-24T09:03:31.335 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.331+0000 7ff7403ea740 10 client.0 _ll_register_callbacks cb 0x7ff728009cd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:03:31.335 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.331+0000 7ff736000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e3e38ead0 0x560e3e39c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:31.335 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.332+0000 7ff736000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e3e38ead0 0x560e3e39c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56196/0 (socket says 172.21.15.83:56196) 2024-04-24T09:03:31.335 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.332+0000 7ff736000640 1 -- 172.21.15.83:0/1802310937 learned_addr learned my addr 172.21.15.83:0/1802310937 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:03:31.335 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.332+0000 7ff737400640 1 --2- 172.21.15.83:0/1802310937 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e3e399390 0x560e3e39cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:31.335 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.332+0000 7ff736a00640 1 --2- 172.21.15.83:0/1802310937 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e3e38e1b0 0x560e3e39c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:31.336 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.332+0000 7ff736000640 1 -- 172.21.15.83:0/1802310937 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e3e399390 msgr2=0x560e3e39cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:31.336 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.332+0000 7ff736000640 1 --2- 172.21.15.83:0/1802310937 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e3e399390 0x560e3e39cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:31.336 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.332+0000 7ff736000640 1 -- 172.21.15.83:0/1802310937 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e3e38e1b0 msgr2=0x560e3e39c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:31.336 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.332+0000 7ff736000640 1 --2- 172.21.15.83:0/1802310937 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e3e38e1b0 0x560e3e39c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:31.336 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.332+0000 7ff736000640 1 -- 172.21.15.83:0/1802310937 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e3e40dce0 con 0x560e3e38ead0 2024-04-24T09:03:31.336 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.332+0000 7ff737400640 1 --2- 172.21.15.83:0/1802310937 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e3e399390 0x560e3e39cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:03:31.336 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.332+0000 7ff736a00640 1 --2- 172.21.15.83:0/1802310937 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e3e38e1b0 0x560e3e39c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:03:31.336 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.332+0000 7ff736000640 1 --2- 172.21.15.83:0/1802310937 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e3e38ead0 0x560e3e39c890 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7ff724009350 tx=0x7ff724009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff427151c9358fac server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:31.336 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.332+0000 7ff717e00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:03:31.336 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.333+0000 7ff717e00640 1 -- 172.21.15.83:0/1802310937 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff724005310 con 0x560e3e38ead0 2024-04-24T09:03:31.336 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.333+0000 7ff717e00640 1 -- 172.21.15.83:0/1802310937 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9583+0+0 (secure 0 0 0) 0x7ff7240054b0 con 0x560e3e38ead0 2024-04-24T09:03:31.336 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.333+0000 7ff7403ea740 1 -- 172.21.15.83:0/1802310937 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff72803b070 con 0x560e3e38ead0 2024-04-24T09:03:31.336 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.333+0000 7ff717e00640 1 -- 172.21.15.83:0/1802310937 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7ff724009910 con 0x560e3e38ead0 2024-04-24T09:03:31.336 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.333+0000 7ff7403ea740 10 client.19800 fetch_fsmap learned FSMap version 41 2024-04-24T09:03:31.336 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.333+0000 7ff7403ea740 1 -- 172.21.15.83:0/1802310937 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x560e3e40ef00 con 0x560e3e38ead0 2024-04-24T09:03:31.337 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.333+0000 7ff717e00640 1 -- 172.21.15.83:0/1802310937 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff72803b070 con 0x560e3e38ead0 2024-04-24T09:03:31.337 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.333+0000 7ff717e00640 1 -- 172.21.15.83:0/1802310937 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7ff724029330 con 0x560e3e38ead0 2024-04-24T09:03:31.337 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.333+0000 7ff7403ea740 10 client.19800 fetch_fsmap finished waiting for FSMap version 41 2024-04-24T09:03:31.337 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.333+0000 7ff7403ea740 10 client.19800 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:03:31.337 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.333+0000 7ff7403ea740 20 client.19800 populate_metadata read hostname 'smithi083' 2024-04-24T09:03:31.337 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.333+0000 7ff7403ea740 1 --2- 172.21.15.83:0/1802310937 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x560e3e40f570 0x560e3e411990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:31.337 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.333+0000 7ff7403ea740 4 client.19800 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T09:03:31.337 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.333+0000 7ff7403ea740 1 -- 172.21.15.83:0/1802310937 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560e3e2fc190 con 0x560e3e40f570 2024-04-24T09:03:31.337 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.334+0000 7ff736a00640 1 --2- 172.21.15.83:0/1802310937 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x560e3e40f570 0x560e3e411990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:31.338 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.335+0000 7ff736a00640 1 --2- 172.21.15.83:0/1802310937 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x560e3e40f570 0x560e3e411990 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a20bccd1613adad2 server_cookie=5c53b48f85990548 in_seq=0 out_seq=0 2024-04-24T09:03:31.338 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.335+0000 7ff717e00640 10 client.19800 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:03:31.404 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.404+0000 7ff717e00640 1 -- 172.21.15.83:0/1802310937 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560e3e2fc190 con 0x560e3e40f570 2024-04-24T09:03:31.404 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.404+0000 7ff717e00640 10 client.19800 handle_command_reply: tid=0 2024-04-24T09:03:31.404 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.404+0000 7ff7403ea740 1 client.19800 shutdown 2024-04-24T09:03:31.404 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.404+0000 7ff7403ea740 1 -- 172.21.15.83:0/1802310937 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e3e38ead0 msgr2=0x560e3e39c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:31.405 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.404+0000 7ff7403ea740 1 --2- 172.21.15.83:0/1802310937 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e3e38ead0 0x560e3e39c890 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7ff724009350 tx=0x7ff724009650 comp rx=0 tx=0).stop 2024-04-24T09:03:31.405 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.405+0000 7ff7403ea740 1 -- 172.21.15.83:0/1802310937 shutdown_connections 2024-04-24T09:03:31.405 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.405+0000 7ff7403ea740 1 --2- 172.21.15.83:0/1802310937 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x560e3e40f570 0x560e3e411990 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:31.405 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.405+0000 7ff7403ea740 1 --2- 172.21.15.83:0/1802310937 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e3e399390 0x560e3e39cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:31.406 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.405+0000 7ff7403ea740 1 --2- 172.21.15.83:0/1802310937 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e3e38ead0 0x560e3e39c890 unknown :-1 s=CLOSED pgs=1880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:31.406 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.405+0000 7ff7403ea740 1 --2- 172.21.15.83:0/1802310937 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e3e38e1b0 0x560e3e39c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:31.406 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.405+0000 7ff717e00640 0 client.19800 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:03:31.406 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.405+0000 7ff7403ea740 1 -- 172.21.15.83:0/1802310937 >> 172.21.15.83:0/1802310937 conn(0x560e3e38a6c0 msgr2=0x560e3e394b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:31.406 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.405+0000 7ff7403ea740 1 -- 172.21.15.83:0/1802310937 shutdown_connections 2024-04-24T09:03:31.406 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.405+0000 7ff7403ea740 1 -- 172.21.15.83:0/1802310937 wait complete. 2024-04-24T09:03:31.406 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.406+0000 7ff7403ea740 20 client.19800 trim_cache size 0 max 16384 2024-04-24T09:03:31.406 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T09:03:31.407 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T09:03:31.407 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T09:03:31.407 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T09:03:31.408 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.407+0000 7ff73d600640 1 -- 172.21.15.83:0/837914228 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff71c0a3dc0 msgr2=0x7ff71c0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:31.408 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.407+0000 7ff73d600640 1 --2- 172.21.15.83:0/837914228 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff71c0a3dc0 0x7ff71c0a6280 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x560e3e38f750 tx=0x7ff728005f90 comp rx=0 tx=0).stop 2024-04-24T09:03:31.408 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.407+0000 7ff73d600640 1 -- 172.21.15.83:0/837914228 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff73810e690 msgr2=0x7ff73819eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:31.408 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.407+0000 7ff73d600640 1 --2- 172.21.15.83:0/837914228 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff73810e690 0x7ff73819eb20 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7ff730033040 tx=0x7ff730009360 comp rx=0 tx=0).stop 2024-04-24T09:03:31.409 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.407+0000 7ff73d600640 1 -- 172.21.15.83:0/837914228 shutdown_connections 2024-04-24T09:03:31.409 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.407+0000 7ff73d600640 1 --2- 172.21.15.83:0/837914228 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff71c0a3dc0 0x7ff71c0a6280 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:31.409 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.407+0000 7ff73d600640 1 --2- 172.21.15.83:0/837914228 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff73810e690 0x7ff73819eb20 unknown :-1 s=CLOSED pgs=1876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:31.409 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.407+0000 7ff73d600640 1 --2- 172.21.15.83:0/837914228 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff738108fb0 0x7ff7381a4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:31.409 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.407+0000 7ff73d600640 1 --2- 172.21.15.83:0/837914228 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7381085f0 0x7ff7381a4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:31.409 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.407+0000 7ff73d600640 1 -- 172.21.15.83:0/837914228 >> 172.21.15.83:0/837914228 conn(0x7ff738076590 msgr2=0x7ff738077be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:31.409 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.408+0000 7ff73d600640 1 -- 172.21.15.83:0/837914228 shutdown_connections 2024-04-24T09:03:31.410 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:31.408+0000 7ff73d600640 1 -- 172.21.15.83:0/837914228 wait complete. 2024-04-24T09:03:31.620 INFO:tasks.workunit.client.0.smithi083.stdout: 1 68820 0.60 MB/sec execute 485 sec latency 55.662 ms 2024-04-24T09:03:31.789 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T09:03:31.789 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T09:03:32.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.016+0000 7f9fbae00640 1 -- 172.21.15.83:0/118341933 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9fb4109580 msgr2=0x7f9fb4111150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:32.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.016+0000 7f9fbae00640 1 --2- 172.21.15.83:0/118341933 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9fb4109580 0x7f9fb4111150 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f9fac009ca0 tx=0x7f9fac02a8c0 comp rx=0 tx=0).stop 2024-04-24T09:03:32.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.016+0000 7f9fbae00640 1 -- 172.21.15.83:0/118341933 shutdown_connections 2024-04-24T09:03:32.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.016+0000 7f9fbae00640 1 --2- 172.21.15.83:0/118341933 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9fb4109580 0x7f9fb4111150 unknown :-1 s=CLOSED pgs=1881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:32.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.016+0000 7f9fbae00640 1 --2- 172.21.15.83:0/118341933 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9fb4104e60 0x7f9fb4108ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:32.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.016+0000 7f9fbae00640 1 --2- 172.21.15.83:0/118341933 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9fb41044b0 0x7f9fb4104890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:32.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.016+0000 7f9fbae00640 1 -- 172.21.15.83:0/118341933 >> 172.21.15.83:0/118341933 conn(0x7f9fb40ffd90 msgr2=0x7f9fb41021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:32.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.016+0000 7f9fbae00640 1 -- 172.21.15.83:0/118341933 shutdown_connections 2024-04-24T09:03:32.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.016+0000 7f9fbae00640 1 -- 172.21.15.83:0/118341933 wait complete. 2024-04-24T09:03:32.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.017+0000 7f9fbae00640 1 Processor -- start 2024-04-24T09:03:32.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.017+0000 7f9fbae00640 1 -- start start 2024-04-24T09:03:32.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.018+0000 7f9fbae00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9fb41044b0 0x7f9fb41a06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:32.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.018+0000 7f9fbae00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9fb4109580 0x7f9fb41a0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:32.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.018+0000 7f9fbae00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9fb419a8a0 0x7f9fb419ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:32.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.018+0000 7f9fbae00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb4114540 con 0x7f9fb419a8a0 2024-04-24T09:03:32.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.018+0000 7f9fbae00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb41143c0 con 0x7f9fb4109580 2024-04-24T09:03:32.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.018+0000 7f9fbae00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb41146c0 con 0x7f9fb41044b0 2024-04-24T09:03:32.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.018+0000 7f9fb8e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9fb419a8a0 0x7f9fb419ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:32.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.018+0000 7f9fb8e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9fb419a8a0 0x7f9fb419ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56210/0 (socket says 172.21.15.83:56210) 2024-04-24T09:03:32.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.018+0000 7f9fb8e00640 1 -- 172.21.15.83:0/1889820203 learned_addr learned my addr 172.21.15.83:0/1889820203 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:03:32.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.018+0000 7f9fb3e00640 1 --2- 172.21.15.83:0/1889820203 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9fb41044b0 0x7f9fb41a06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:32.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.019+0000 7f9fb3400640 1 --2- 172.21.15.83:0/1889820203 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9fb4109580 0x7f9fb41a0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:32.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.019+0000 7f9fb8e00640 1 -- 172.21.15.83:0/1889820203 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9fb41044b0 msgr2=0x7f9fb41a06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:32.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.019+0000 7f9fb8e00640 1 --2- 172.21.15.83:0/1889820203 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9fb41044b0 0x7f9fb41a06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:32.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.019+0000 7f9fb8e00640 1 -- 172.21.15.83:0/1889820203 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9fb4109580 msgr2=0x7f9fb41a0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:32.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.019+0000 7f9fb8e00640 1 --2- 172.21.15.83:0/1889820203 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9fb4109580 0x7f9fb41a0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:32.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.019+0000 7f9fb8e00640 1 -- 172.21.15.83:0/1889820203 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fb419b3c0 con 0x7f9fb419a8a0 2024-04-24T09:03:32.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.019+0000 7f9fb8e00640 1 --2- 172.21.15.83:0/1889820203 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9fb419a8a0 0x7f9fb419ad50 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f9fac031040 tx=0x7f9fac003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:32.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.019+0000 7f9fb0c00640 1 -- 172.21.15.83:0/1889820203 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fac003ce0 con 0x7f9fb419a8a0 2024-04-24T09:03:32.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.019+0000 7f9fb0c00640 1 -- 172.21.15.83:0/1889820203 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9fac0044f0 con 0x7f9fb419a8a0 2024-04-24T09:03:32.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.019+0000 7f9fb0c00640 1 -- 172.21.15.83:0/1889820203 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fac030cb0 con 0x7f9fb419a8a0 2024-04-24T09:03:32.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.020+0000 7f9fbae00640 1 -- 172.21.15.83:0/1889820203 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fb419b650 con 0x7f9fb419a8a0 2024-04-24T09:03:32.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.020+0000 7f9fbae00640 1 -- 172.21.15.83:0/1889820203 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fb41a7850 con 0x7f9fb419a8a0 2024-04-24T09:03:32.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.023+0000 7f9fbdb71740 1 Processor -- start 2024-04-24T09:03:32.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.023+0000 7f9fb0c00640 1 -- 172.21.15.83:0/1889820203 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9fac0046a0 con 0x7f9fb419a8a0 2024-04-24T09:03:32.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.023+0000 7f9fbdb71740 1 -- start start 2024-04-24T09:03:32.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.024+0000 7f9fbdb71740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dcc8535860 0x55dcc8535c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:32.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.024+0000 7f9fbdb71740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dcc8536180 0x55dcc8540310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:32.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.024+0000 7f9fbdb71740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dcc8540850 0x55dcc8542c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:32.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.024+0000 7f9fbdb71740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55dcc849b1f0 con 0x55dcc8535860 2024-04-24T09:03:32.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.024+0000 7f9fbdb71740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55dcc8441350 con 0x55dcc8540850 2024-04-24T09:03:32.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.024+0000 7f9fbdb71740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55dcc84971b0 con 0x55dcc8536180 2024-04-24T09:03:32.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.024+0000 7f9fb3400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dcc8535860 0x55dcc8535c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:32.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.024+0000 7f9fb3400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dcc8535860 0x55dcc8535c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56226/0 (socket says 172.21.15.83:56226) 2024-04-24T09:03:32.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.024+0000 7f9fb3400640 1 -- 172.21.15.83:0/485832921 learned_addr learned my addr 172.21.15.83:0/485832921 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:03:32.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.025+0000 7f9fb3e00640 1 --2- 172.21.15.83:0/485832921 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dcc8536180 0x55dcc8540310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:32.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.025+0000 7f9fb3400640 1 -- 172.21.15.83:0/485832921 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dcc8536180 msgr2=0x55dcc8540310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:32.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.025+0000 7f9fb3400640 1 --2- 172.21.15.83:0/485832921 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dcc8536180 0x55dcc8540310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:32.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.025+0000 7f9fb3400640 1 -- 172.21.15.83:0/485832921 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dcc8540850 msgr2=0x55dcc8542c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:32.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.025+0000 7f9fb3400640 1 --2- 172.21.15.83:0/485832921 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dcc8540850 0x55dcc8542c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:32.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.025+0000 7f9fb3400640 1 -- 172.21.15.83:0/485832921 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dcc8495f50 con 0x55dcc8535860 2024-04-24T09:03:32.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.025+0000 7f9fb0c00640 1 --2- 172.21.15.83:0/1889820203 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f9f980a3e70 0x7f9f980a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:32.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.025+0000 7f9fb0c00640 1 -- 172.21.15.83:0/1889820203 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9583+0+0 (secure 0 0 0) 0x7f9fac0f1ff0 con 0x7f9fb419a8a0 2024-04-24T09:03:32.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.025+0000 7f9fb3400640 1 --2- 172.21.15.83:0/485832921 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dcc8535860 0x55dcc8535c40 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7f9fa0005940 tx=0x7f9fa0010370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8eb35f2f09b8af server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:32.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.025+0000 7f9fb3e00640 1 --2- 172.21.15.83:0/1889820203 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f9f980a3e70 0x7f9f980a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:32.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.026+0000 7f9fa9600640 1 -- 172.21.15.83:0/485832921 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fa0016030 con 0x55dcc8535860 2024-04-24T09:03:32.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.026+0000 7f9fa9600640 1 -- 172.21.15.83:0/485832921 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9fa0010db0 con 0x55dcc8535860 2024-04-24T09:03:32.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.026+0000 7f9fa9600640 1 -- 172.21.15.83:0/485832921 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fa0012480 con 0x55dcc8535860 2024-04-24T09:03:32.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.027+0000 7f9fbdb71740 1 -- 172.21.15.83:0/485832921 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dcc8535860 msgr2=0x55dcc8535c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:32.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.027+0000 7f9fbdb71740 1 --2- 172.21.15.83:0/485832921 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dcc8535860 0x55dcc8535c40 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7f9fa0005940 tx=0x7f9fa0010370 comp rx=0 tx=0).stop 2024-04-24T09:03:32.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.027+0000 7f9fb3e00640 1 --2- 172.21.15.83:0/1889820203 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f9f980a3e70 0x7f9f980a6330 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f9fa4005fd0 tx=0x7f9fa400b650 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:32.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.027+0000 7f9fbdb71740 1 -- 172.21.15.83:0/485832921 shutdown_connections 2024-04-24T09:03:32.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.027+0000 7f9fbdb71740 1 --2- 172.21.15.83:0/485832921 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dcc8540850 0x55dcc8542c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:32.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.027+0000 7f9fbdb71740 1 --2- 172.21.15.83:0/485832921 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dcc8536180 0x55dcc8540310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:32.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.027+0000 7f9fbdb71740 1 --2- 172.21.15.83:0/485832921 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dcc8535860 0x55dcc8535c40 unknown :-1 s=CLOSED pgs=1883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:32.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.027+0000 7f9fbdb71740 1 -- 172.21.15.83:0/485832921 >> 172.21.15.83:0/485832921 conn(0x55dcc8531b80 msgr2=0x55dcc8531f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:32.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.027+0000 7f9fbdb71740 1 -- 172.21.15.83:0/485832921 shutdown_connections 2024-04-24T09:03:32.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.027+0000 7f9fbdb71740 1 -- 172.21.15.83:0/485832921 wait complete. 2024-04-24T09:03:32.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.027+0000 7f9fbdb71740 1 -- start start 2024-04-24T09:03:32.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.028+0000 7f9fbdb71740 1 Processor -- start 2024-04-24T09:03:32.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.028+0000 7f9fbdb71740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dcc8535860 0x55dcc853d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:32.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.028+0000 7f9fbdb71740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dcc8536180 0x55dcc853dde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:32.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.028+0000 7f9fbdb71740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dcc8540850 0x55dcc853e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:32.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.028+0000 7f9fbdb71740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55dcc8543640 con 0x55dcc8536180 2024-04-24T09:03:32.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.028+0000 7f9fbdb71740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55dcc8454690 con 0x55dcc8540850 2024-04-24T09:03:32.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.028+0000 7f9fbdb71740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55dcc85437c0 con 0x55dcc8535860 2024-04-24T09:03:32.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.028+0000 7f9fbdb71740 10 client.0 _ll_register_callbacks cb 0x55dcc8490650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:03:32.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.028+0000 7f9fb3400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dcc8536180 0x55dcc853dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:32.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.028+0000 7f9fb3400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dcc8536180 0x55dcc853dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56228/0 (socket says 172.21.15.83:56228) 2024-04-24T09:03:32.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.028+0000 7f9fb3400640 1 -- 172.21.15.83:0/58266630 learned_addr learned my addr 172.21.15.83:0/58266630 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:03:32.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.029+0000 7f9fb3e00640 1 --2- 172.21.15.83:0/58266630 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dcc8535860 0x55dcc853d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:32.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.029+0000 7f9fb8e00640 1 --2- 172.21.15.83:0/58266630 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dcc8540850 0x55dcc853e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:32.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.029+0000 7f9fb3400640 1 -- 172.21.15.83:0/58266630 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dcc8535860 msgr2=0x55dcc853d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:32.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.029+0000 7f9fb3400640 1 --2- 172.21.15.83:0/58266630 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dcc8535860 0x55dcc853d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:32.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.029+0000 7f9fb3400640 1 -- 172.21.15.83:0/58266630 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dcc8540850 msgr2=0x55dcc853e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:32.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.029+0000 7f9fb3400640 1 --2- 172.21.15.83:0/58266630 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dcc8540850 0x55dcc853e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:32.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.029+0000 7f9fb3400640 1 -- 172.21.15.83:0/58266630 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dcc8453920 con 0x55dcc8536180 2024-04-24T09:03:32.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.029+0000 7f9fb3400640 1 --2- 172.21.15.83:0/58266630 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dcc8536180 0x55dcc853dde0 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7f9fa0005d70 tx=0x7f9fa00049a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bb2b41873d303d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:32.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.029+0000 7f9f9e000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:03:32.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.029+0000 7f9f9e000640 1 -- 172.21.15.83:0/58266630 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fa0016030 con 0x55dcc8536180 2024-04-24T09:03:32.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.030+0000 7f9f9e000640 1 -- 172.21.15.83:0/58266630 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9583+0+0 (secure 0 0 0) 0x7f9fa0005430 con 0x55dcc8536180 2024-04-24T09:03:32.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.030+0000 7f9fbdb71740 1 -- 172.21.15.83:0/58266630 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9fa0010db0 con 0x55dcc8536180 2024-04-24T09:03:32.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.030+0000 7f9f9e000640 1 -- 172.21.15.83:0/58266630 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f9fa0005630 con 0x55dcc8536180 2024-04-24T09:03:32.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.030+0000 7f9fbdb71740 10 client.19812 fetch_fsmap learned FSMap version 41 2024-04-24T09:03:32.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.030+0000 7f9fbdb71740 1 -- 172.21.15.83:0/58266630 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55dcc82777d0 con 0x55dcc8536180 2024-04-24T09:03:32.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.030+0000 7f9f9e000640 1 -- 172.21.15.83:0/58266630 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9fa0010db0 con 0x55dcc8536180 2024-04-24T09:03:32.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.030+0000 7f9f9e000640 1 -- 172.21.15.83:0/58266630 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7f9fa0023070 con 0x55dcc8536180 2024-04-24T09:03:32.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.031+0000 7f9fbdb71740 10 client.19812 fetch_fsmap finished waiting for FSMap version 41 2024-04-24T09:03:32.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.031+0000 7f9fbdb71740 10 client.19812 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:03:32.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.031+0000 7f9fbdb71740 20 client.19812 populate_metadata read hostname 'smithi083' 2024-04-24T09:03:32.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.031+0000 7f9fbdb71740 1 --2- 172.21.15.83:0/58266630 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55dcc8550be0 0x55dcc8552fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:32.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.031+0000 7f9fbdb71740 4 client.19812 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:03:32.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.031+0000 7f9fbdb71740 1 -- 172.21.15.83:0/58266630 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dcc84a8c10 con 0x55dcc8550be0 2024-04-24T09:03:32.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.031+0000 7f9fb3e00640 1 --2- 172.21.15.83:0/58266630 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55dcc8550be0 0x55dcc8552fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:32.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.032+0000 7f9fb3e00640 1 --2- 172.21.15.83:0/58266630 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55dcc8550be0 0x55dcc8552fd0 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf113d372d523bc9 server_cookie=9f9f88b4471450f4 in_seq=0 out_seq=0 2024-04-24T09:03:32.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.032+0000 7f9f9e000640 10 client.19812 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:03:32.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.036+0000 7f9f9e000640 1 -- 172.21.15.83:0/58266630 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55dcc84a8c10 con 0x55dcc8550be0 2024-04-24T09:03:32.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.036+0000 7f9f9e000640 10 client.19812 handle_command_reply: tid=0 2024-04-24T09:03:32.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.036+0000 7f9fbdb71740 1 client.19812 shutdown 2024-04-24T09:03:32.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.036+0000 7f9fbdb71740 1 -- 172.21.15.83:0/58266630 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dcc8536180 msgr2=0x55dcc853dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:32.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.036+0000 7f9fbdb71740 1 --2- 172.21.15.83:0/58266630 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dcc8536180 0x55dcc853dde0 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7f9fa0005d70 tx=0x7f9fa00049a0 comp rx=0 tx=0).stop 2024-04-24T09:03:32.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.036+0000 7f9fbdb71740 1 -- 172.21.15.83:0/58266630 shutdown_connections 2024-04-24T09:03:32.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.036+0000 7f9fbdb71740 1 --2- 172.21.15.83:0/58266630 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55dcc8550be0 0x55dcc8552fd0 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:32.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.036+0000 7f9fbdb71740 1 --2- 172.21.15.83:0/58266630 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dcc8540850 0x55dcc853e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:32.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.036+0000 7f9fbdb71740 1 --2- 172.21.15.83:0/58266630 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dcc8536180 0x55dcc853dde0 unknown :-1 s=CLOSED pgs=1884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:32.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.036+0000 7f9fbdb71740 1 --2- 172.21.15.83:0/58266630 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dcc8535860 0x55dcc853d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:32.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.036+0000 7f9f9e000640 0 client.19812 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:03:32.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.036+0000 7f9fbdb71740 1 -- 172.21.15.83:0/58266630 >> 172.21.15.83:0/58266630 conn(0x55dcc8531b80 msgr2=0x55dcc8537980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:32.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.036+0000 7f9fbdb71740 1 -- 172.21.15.83:0/58266630 shutdown_connections 2024-04-24T09:03:32.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.036+0000 7f9fbdb71740 1 -- 172.21.15.83:0/58266630 wait complete. 2024-04-24T09:03:32.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.037+0000 7f9fbdb71740 20 client.19812 trim_cache size 0 max 16384 2024-04-24T09:03:32.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.060+0000 7f9fbdb71740 1 Processor -- start 2024-04-24T09:03:32.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.060+0000 7f9fbdb71740 1 -- start start 2024-04-24T09:03:32.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.060+0000 7f9fbdb71740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dcc8535860 0x55dcc853c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:32.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.060+0000 7f9fbdb71740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dcc8536180 0x55dcc854c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:32.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.060+0000 7f9fbdb71740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dcc8540850 0x55dcc8538e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:32.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.060+0000 7f9fbdb71740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb41b17d0 con 0x55dcc8540850 2024-04-24T09:03:32.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.060+0000 7f9fbdb71740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb41b3430 con 0x55dcc8536180 2024-04-24T09:03:32.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.060+0000 7f9fbdb71740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb41b1ea0 con 0x55dcc8535860 2024-04-24T09:03:32.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.061+0000 7f9fb8e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dcc8540850 0x55dcc8538e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:32.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.061+0000 7f9fb8e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dcc8540850 0x55dcc8538e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56240/0 (socket says 172.21.15.83:56240) 2024-04-24T09:03:32.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.061+0000 7f9fb8e00640 1 -- 172.21.15.83:0/2073717130 learned_addr learned my addr 172.21.15.83:0/2073717130 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:03:32.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.061+0000 7f9fb3400640 1 --2- 172.21.15.83:0/2073717130 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dcc8536180 0x55dcc854c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:32.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.061+0000 7f9fb3e00640 1 --2- 172.21.15.83:0/2073717130 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dcc8535860 0x55dcc853c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:32.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.061+0000 7f9fb8e00640 1 -- 172.21.15.83:0/2073717130 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dcc8535860 msgr2=0x55dcc853c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:32.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.061+0000 7f9fb8e00640 1 --2- 172.21.15.83:0/2073717130 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dcc8535860 0x55dcc853c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:32.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.061+0000 7f9fb8e00640 1 -- 172.21.15.83:0/2073717130 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dcc8536180 msgr2=0x55dcc854c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:32.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.061+0000 7f9fb8e00640 1 --2- 172.21.15.83:0/2073717130 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dcc8536180 0x55dcc854c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:32.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.061+0000 7f9fb8e00640 1 -- 172.21.15.83:0/2073717130 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dcc81ad3f0 con 0x55dcc8540850 2024-04-24T09:03:32.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.061+0000 7f9fb3400640 1 --2- 172.21.15.83:0/2073717130 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dcc8536180 0x55dcc854c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:03:32.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.062+0000 7f9fb8e00640 1 --2- 172.21.15.83:0/2073717130 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dcc8540850 0x55dcc8538e00 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f9fac0bdbd0 tx=0x7f9fac0bd5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e08786b7c1208ffd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:32.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.062+0000 7f9fa8c00640 1 -- 172.21.15.83:0/2073717130 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fac039e10 con 0x55dcc8540850 2024-04-24T09:03:32.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.062+0000 7f9fa8c00640 1 -- 172.21.15.83:0/2073717130 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9fac036070 con 0x55dcc8540850 2024-04-24T09:03:32.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.062+0000 7f9fa8c00640 1 -- 172.21.15.83:0/2073717130 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fac0bbf40 con 0x55dcc8540850 2024-04-24T09:03:32.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.063+0000 7f9fbdb71740 1 -- 172.21.15.83:0/2073717130 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dcc8540850 msgr2=0x55dcc8538e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:32.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.063+0000 7f9fbdb71740 1 --2- 172.21.15.83:0/2073717130 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dcc8540850 0x55dcc8538e00 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f9fac0bdbd0 tx=0x7f9fac0bd5a0 comp rx=0 tx=0).stop 2024-04-24T09:03:32.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.063+0000 7f9fbdb71740 1 -- 172.21.15.83:0/2073717130 shutdown_connections 2024-04-24T09:03:32.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.063+0000 7f9fbdb71740 1 --2- 172.21.15.83:0/2073717130 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dcc8540850 0x55dcc8538e00 unknown :-1 s=CLOSED pgs=1885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:32.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.063+0000 7f9fbdb71740 1 --2- 172.21.15.83:0/2073717130 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dcc8536180 0x55dcc854c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:32.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.063+0000 7f9fbdb71740 1 --2- 172.21.15.83:0/2073717130 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dcc8535860 0x55dcc853c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:32.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.063+0000 7f9fbdb71740 1 -- 172.21.15.83:0/2073717130 >> 172.21.15.83:0/2073717130 conn(0x55dcc8531b80 msgr2=0x55dcc853e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:32.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.063+0000 7f9fbdb71740 1 -- 172.21.15.83:0/2073717130 shutdown_connections 2024-04-24T09:03:32.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.063+0000 7f9fbdb71740 1 -- 172.21.15.83:0/2073717130 wait complete. 2024-04-24T09:03:32.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.063+0000 7f9fbdb71740 1 -- start start 2024-04-24T09:03:32.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.063+0000 7f9fbdb71740 1 Processor -- start 2024-04-24T09:03:32.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.064+0000 7f9fbdb71740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dcc8535860 0x55dcc853a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:32.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.064+0000 7f9fbdb71740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dcc8536180 0x55dcc853aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:32.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.064+0000 7f9fbdb71740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dcc8540850 0x55dcc853b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:32.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.064+0000 7f9fbdb71740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb41b2d00 con 0x55dcc8536180 2024-04-24T09:03:32.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.064+0000 7f9fbdb71740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb41b25d0 con 0x55dcc8535860 2024-04-24T09:03:32.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.064+0000 7f9fbdb71740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb41af7d0 con 0x55dcc8540850 2024-04-24T09:03:32.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.064+0000 7f9fbdb71740 10 client.0 _ll_register_callbacks cb 0x7f9fa000ca70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:03:32.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.064+0000 7f9fb3400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dcc8536180 0x55dcc853aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:32.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.064+0000 7f9fb3400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dcc8536180 0x55dcc853aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56244/0 (socket says 172.21.15.83:56244) 2024-04-24T09:03:32.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.064+0000 7f9fb3400640 1 -- 172.21.15.83:0/1286249050 learned_addr learned my addr 172.21.15.83:0/1286249050 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:03:32.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.065+0000 7f9fb3e00640 1 --2- 172.21.15.83:0/1286249050 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dcc8535860 0x55dcc853a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:32.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.065+0000 7f9fb3400640 1 -- 172.21.15.83:0/1286249050 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dcc8540850 msgr2=0x55dcc853b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:32.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.065+0000 7f9fb8e00640 1 --2- 172.21.15.83:0/1286249050 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dcc8540850 0x55dcc853b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:32.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.065+0000 7f9fb3400640 1 --2- 172.21.15.83:0/1286249050 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dcc8540850 0x55dcc853b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:32.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.065+0000 7f9fb3400640 1 -- 172.21.15.83:0/1286249050 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dcc8535860 msgr2=0x55dcc853a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:32.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.065+0000 7f9fb3400640 1 --2- 172.21.15.83:0/1286249050 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dcc8535860 0x55dcc853a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:32.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.065+0000 7f9fb3400640 1 -- 172.21.15.83:0/1286249050 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dcc816d0f0 con 0x55dcc8536180 2024-04-24T09:03:32.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.065+0000 7f9fb3400640 1 --2- 172.21.15.83:0/1286249050 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dcc8536180 0x55dcc853aaf0 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x55dcc853d100 tx=0x7f9fa0005560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2625aa1e4d2e397 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:32.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.065+0000 7f9f9e000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:03:32.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.065+0000 7f9f9e000640 1 -- 172.21.15.83:0/1286249050 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fa0016070 con 0x55dcc8536180 2024-04-24T09:03:32.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.065+0000 7f9f9e000640 1 -- 172.21.15.83:0/1286249050 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9583+0+0 (secure 0 0 0) 0x7f9fa0012690 con 0x55dcc8536180 2024-04-24T09:03:32.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.065+0000 7f9fbdb71740 1 -- 172.21.15.83:0/1286249050 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9fac036070 con 0x55dcc8536180 2024-04-24T09:03:32.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.066+0000 7f9f9e000640 1 -- 172.21.15.83:0/1286249050 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f9fa0004a10 con 0x55dcc8536180 2024-04-24T09:03:32.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.066+0000 7f9fbdb71740 10 client.19818 fetch_fsmap learned FSMap version 41 2024-04-24T09:03:32.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.066+0000 7f9fbdb71740 1 -- 172.21.15.83:0/1286249050 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55dcc8460810 con 0x55dcc8536180 2024-04-24T09:03:32.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.066+0000 7f9f9e000640 1 -- 172.21.15.83:0/1286249050 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9fac036070 con 0x55dcc8536180 2024-04-24T09:03:32.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.066+0000 7f9f9e000640 1 -- 172.21.15.83:0/1286249050 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7f9fa0023040 con 0x55dcc8536180 2024-04-24T09:03:32.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.066+0000 7f9fbdb71740 10 client.19818 fetch_fsmap finished waiting for FSMap version 41 2024-04-24T09:03:32.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.066+0000 7f9fbdb71740 10 client.19818 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:03:32.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.066+0000 7f9fbdb71740 20 client.19818 populate_metadata read hostname 'smithi083' 2024-04-24T09:03:32.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.066+0000 7f9fbdb71740 1 --2- 172.21.15.83:0/1286249050 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55dcc85b7690 0x55dcc85b9ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:32.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.066+0000 7f9fbdb71740 4 client.19818 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T09:03:32.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.066+0000 7f9fbdb71740 1 -- 172.21.15.83:0/1286249050 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55dcc84a2a80 con 0x55dcc85b7690 2024-04-24T09:03:32.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.067+0000 7f9fb3e00640 1 --2- 172.21.15.83:0/1286249050 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55dcc85b7690 0x55dcc85b9ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:32.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.068+0000 7f9fb3e00640 1 --2- 172.21.15.83:0/1286249050 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55dcc85b7690 0x55dcc85b9ab0 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd1b48081fc1fb20 server_cookie=323b3823d2d95d3b in_seq=0 out_seq=0 2024-04-24T09:03:32.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.068+0000 7f9f9e000640 10 client.19818 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:03:32.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.068+0000 7f9f9e000640 1 -- 172.21.15.83:0/1286249050 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55dcc84a2a80 con 0x55dcc85b7690 2024-04-24T09:03:32.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.068+0000 7f9f9e000640 10 client.19818 handle_command_reply: tid=0 2024-04-24T09:03:32.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.068+0000 7f9fbdb71740 1 client.19818 shutdown 2024-04-24T09:03:32.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.068+0000 7f9fbdb71740 1 -- 172.21.15.83:0/1286249050 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dcc8536180 msgr2=0x55dcc853aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:32.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.068+0000 7f9fbdb71740 1 --2- 172.21.15.83:0/1286249050 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dcc8536180 0x55dcc853aaf0 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x55dcc853d100 tx=0x7f9fa0005560 comp rx=0 tx=0).stop 2024-04-24T09:03:32.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.069+0000 7f9fbdb71740 1 -- 172.21.15.83:0/1286249050 shutdown_connections 2024-04-24T09:03:32.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.069+0000 7f9fbdb71740 1 --2- 172.21.15.83:0/1286249050 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55dcc85b7690 0x55dcc85b9ab0 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:32.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.069+0000 7f9fbdb71740 1 --2- 172.21.15.83:0/1286249050 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dcc8540850 0x55dcc853b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:32.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.069+0000 7f9fbdb71740 1 --2- 172.21.15.83:0/1286249050 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dcc8536180 0x55dcc853aaf0 unknown :-1 s=CLOSED pgs=1886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:32.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.069+0000 7f9fbdb71740 1 --2- 172.21.15.83:0/1286249050 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dcc8535860 0x55dcc853a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:32.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.069+0000 7f9f9e000640 0 client.19818 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:03:32.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.069+0000 7f9fbdb71740 1 -- 172.21.15.83:0/1286249050 >> 172.21.15.83:0/1286249050 conn(0x55dcc8531b80 msgr2=0x55dcc853d840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:32.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.069+0000 7f9fbdb71740 1 -- 172.21.15.83:0/1286249050 shutdown_connections 2024-04-24T09:03:32.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.069+0000 7f9fbdb71740 1 -- 172.21.15.83:0/1286249050 wait complete. 2024-04-24T09:03:32.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.069+0000 7f9fbdb71740 20 client.19818 trim_cache size 0 max 16384 2024-04-24T09:03:32.072 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T09:03:32.072 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-04-24T09:03:32.072 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-04-24T09:03:32.072 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T09:03:32.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.069+0000 7f9fbae00640 1 -- 172.21.15.83:0/1889820203 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f9f980a3e70 msgr2=0x7f9f980a6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:32.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.069+0000 7f9fbae00640 1 --2- 172.21.15.83:0/1889820203 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f9f980a3e70 0x7f9f980a6330 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f9fa4005fd0 tx=0x7f9fa400b650 comp rx=0 tx=0).stop 2024-04-24T09:03:32.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.069+0000 7f9fbae00640 1 -- 172.21.15.83:0/1889820203 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9fb419a8a0 msgr2=0x7f9fb419ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:32.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.069+0000 7f9fbae00640 1 --2- 172.21.15.83:0/1889820203 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9fb419a8a0 0x7f9fb419ad50 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f9fac031040 tx=0x7f9fac003970 comp rx=0 tx=0).stop 2024-04-24T09:03:32.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.070+0000 7f9fbae00640 1 -- 172.21.15.83:0/1889820203 shutdown_connections 2024-04-24T09:03:32.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.070+0000 7f9fbae00640 1 --2- 172.21.15.83:0/1889820203 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f9f980a3e70 0x7f9f980a6330 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:32.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.070+0000 7f9fbae00640 1 --2- 172.21.15.83:0/1889820203 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9fb419a8a0 0x7f9fb419ad50 unknown :-1 s=CLOSED pgs=1882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:32.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.070+0000 7f9fbae00640 1 --2- 172.21.15.83:0/1889820203 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9fb4109580 0x7f9fb41a0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:32.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.070+0000 7f9fbae00640 1 --2- 172.21.15.83:0/1889820203 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9fb41044b0 0x7f9fb41a06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:32.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.070+0000 7f9fbae00640 1 -- 172.21.15.83:0/1889820203 >> 172.21.15.83:0/1889820203 conn(0x7f9fb40ffd90 msgr2=0x7f9fb4102070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:32.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.070+0000 7f9fbae00640 1 -- 172.21.15.83:0/1889820203 shutdown_connections 2024-04-24T09:03:32.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.070+0000 7f9fbae00640 1 -- 172.21.15.83:0/1889820203 wait complete. 2024-04-24T09:03:32.086 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T09:03:32.087 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T09:03:32.149 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:03:32 smithi083 ceph-mon[27907]: pgmap v374: 129 pgs: 129 active+clean; 66 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 340 KiB/s wr, 52 op/s 2024-04-24T09:03:32.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.284+0000 7fb917200640 1 -- 172.21.15.83:0/1576017009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb918104c20 msgr2=0x7fb918108c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:32.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.284+0000 7fb917200640 1 --2- 172.21.15.83:0/1576017009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb918104c20 0x7fb918108c60 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7fb904009900 tx=0x7fb90402a410 comp rx=0 tx=0).stop 2024-04-24T09:03:32.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.284+0000 7fb917200640 1 -- 172.21.15.83:0/1576017009 shutdown_connections 2024-04-24T09:03:32.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.284+0000 7fb917200640 1 --2- 172.21.15.83:0/1576017009 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb918109340 0x7fb918110f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:32.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.284+0000 7fb917200640 1 --2- 172.21.15.83:0/1576017009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb918104c20 0x7fb918108c60 unknown :-1 s=CLOSED pgs=1887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:32.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.284+0000 7fb917200640 1 --2- 172.21.15.83:0/1576017009 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb918104270 0x7fb918104650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:32.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.284+0000 7fb917200640 1 -- 172.21.15.83:0/1576017009 >> 172.21.15.83:0/1576017009 conn(0x7fb9180ffa90 msgr2=0x7fb918101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:32.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.284+0000 7fb917200640 1 -- 172.21.15.83:0/1576017009 shutdown_connections 2024-04-24T09:03:32.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.284+0000 7fb917200640 1 -- 172.21.15.83:0/1576017009 wait complete. 2024-04-24T09:03:32.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.285+0000 7fb917200640 1 Processor -- start 2024-04-24T09:03:32.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.285+0000 7fb917200640 1 -- start start 2024-04-24T09:03:32.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.285+0000 7fb917200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb918104270 0x7fb9181a05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:32.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.285+0000 7fb917200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb918109340 0x7fb9181a0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:32.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.285+0000 7fb917200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb91819a690 0x7fb91819ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:32.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.285+0000 7fb917200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb9181142d0 con 0x7fb918104270 2024-04-24T09:03:32.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.285+0000 7fb917200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb918114150 con 0x7fb918109340 2024-04-24T09:03:32.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.285+0000 7fb917200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb918114450 con 0x7fb91819a690 2024-04-24T09:03:32.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.286+0000 7fb915e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb918104270 0x7fb9181a05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:32.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.286+0000 7fb915e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb918104270 0x7fb9181a05d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56268/0 (socket says 172.21.15.83:56268) 2024-04-24T09:03:32.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.286+0000 7fb915e00640 1 -- 172.21.15.83:0/1295966910 learned_addr learned my addr 172.21.15.83:0/1295966910 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:03:32.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.286+0000 7fb915e00640 1 -- 172.21.15.83:0/1295966910 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb91819a690 msgr2=0x7fb91819ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:32.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.286+0000 7fb915e00640 1 --2- 172.21.15.83:0/1295966910 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb91819a690 0x7fb91819ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:32.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.286+0000 7fb915e00640 1 -- 172.21.15.83:0/1295966910 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb918109340 msgr2=0x7fb9181a0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:32.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.286+0000 7fb915e00640 1 --2- 172.21.15.83:0/1295966910 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb918109340 0x7fb9181a0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:32.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.286+0000 7fb915e00640 1 -- 172.21.15.83:0/1295966910 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb91819b290 con 0x7fb918104270 2024-04-24T09:03:32.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.286+0000 7fb915e00640 1 --2- 172.21.15.83:0/1295966910 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb918104270 0x7fb9181a05d0 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7fb90800d950 tx=0x7fb90800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:32.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.287+0000 7fb90ea00640 1 -- 172.21.15.83:0/1295966910 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb908017070 con 0x7fb918104270 2024-04-24T09:03:32.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.287+0000 7fb90ea00640 1 -- 172.21.15.83:0/1295966910 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb908009c30 con 0x7fb918104270 2024-04-24T09:03:32.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.287+0000 7fb917200640 1 -- 172.21.15.83:0/1295966910 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9181a7370 con 0x7fb918104270 2024-04-24T09:03:32.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.287+0000 7fb90ea00640 1 -- 172.21.15.83:0/1295966910 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb90800f620 con 0x7fb918104270 2024-04-24T09:03:32.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.287+0000 7fb917200640 1 -- 172.21.15.83:0/1295966910 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9181a7770 con 0x7fb918104270 2024-04-24T09:03:32.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.288+0000 7fb917200640 1 -- 172.21.15.83:0/1295966910 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb918105740 con 0x7fb918104270 2024-04-24T09:03:32.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.291+0000 7fb90ea00640 1 -- 172.21.15.83:0/1295966910 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb9080109a0 con 0x7fb918104270 2024-04-24T09:03:32.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.292+0000 7fb90ea00640 1 --2- 172.21.15.83:0/1295966910 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb8fc0a3d20 0x7fb8fc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:32.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.292+0000 7fb90ea00640 1 -- 172.21.15.83:0/1295966910 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9583+0+0 (secure 0 0 0) 0x7fb9080d0970 con 0x7fb918104270 2024-04-24T09:03:32.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.292+0000 7fb915400640 1 --2- 172.21.15.83:0/1295966910 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb8fc0a3d20 0x7fb8fc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:32.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.293+0000 7fb915400640 1 --2- 172.21.15.83:0/1295966910 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb8fc0a3d20 0x7fb8fc0a61e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fb90402a930 tx=0x7fb904031400 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:32.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.295+0000 7fb90ea00640 1 -- 172.21.15.83:0/1295966910 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb908009de0 con 0x7fb918104270 2024-04-24T09:03:32.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:03:32 smithi138 ceph-mon[31441]: pgmap v374: 129 pgs: 129 active+clean; 66 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 340 KiB/s wr, 52 op/s 2024-04-24T09:03:32.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:03:32 smithi189 ceph-mon[31791]: pgmap v374: 129 pgs: 129 active+clean; 66 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 340 KiB/s wr, 52 op/s 2024-04-24T09:03:32.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.562+0000 7fb917200640 1 -- 172.21.15.83:0/1295966910 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb918104c20 con 0x7fb918104270 2024-04-24T09:03:32.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.564+0000 7fb90ea00640 1 -- 172.21.15.83:0/1295966910 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+11953 (secure 0 0 0) 0x7fb90809cb90 con 0x7fb918104270 2024-04-24T09:03:32.564 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T09:03:32.564 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":41,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/358872257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":358872257},{"type":"v1","addr":"172.21.15.83:6835","nonce":358872257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3157070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3157070851},{"type":"v1","addr":"172.21.15.83:6837","nonce":3157070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":41,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T08:48:29.422576+0000","modified":"2024-04-24T09:02:18.079759+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34520,"mds_1":34493,"mds_2":34451,"mds_3":34427,"mds_4":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14658":{"gid":14658,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6839/4085470844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":4085470844},{"type":"v1","addr":"172.21.15.83:6839","nonce":4085470844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14676":{"gid":14676,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6841/2634754210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":2634754210},{"type":"v1","addr":"172.21.15.83:6841","nonce":2634754210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24382":{"gid":24382,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/1859921310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1859921310},{"type":"v1","addr":"172.21.15.189:6833","nonce":1859921310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/2241923197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2241923197},{"type":"v1","addr":"172.21.15.189:6835","nonce":2241923197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.189:6839/2548965336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":2548965336},{"type":"v1","addr":"172.21.15.189:6839","nonce":2548965336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26686":{"gid":26686,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6837/1678817161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":1678817161},{"type":"v1","addr":"172.21.15.189:6837","nonce":1678817161}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.138:6833/1157967658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1157967658},{"type":"v1","addr":"172.21.15.138:6833","nonce":1157967658}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.138:6835/1909219758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1909219758},{"type":"v1","addr":"172.21.15.138:6835","nonce":1909219758}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34493":{"gid":34493,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.138:6837/900566754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":900566754},{"type":"v1","addr":"172.21.15.138:6837","nonce":900566754}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34520":{"gid":34520,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.138:6839/2236932393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2236932393},{"type":"v1","addr":"172.21.15.138:6839","nonce":2236932393}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34520,"qdb_cluster":[24454,34520,34427,34451,34493]},"id":1}]} 2024-04-24T09:03:32.565 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 41 2024-04-24T09:03:32.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.567+0000 7fb917200640 1 -- 172.21.15.83:0/1295966910 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb8fc0a3d20 msgr2=0x7fb8fc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:32.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.567+0000 7fb917200640 1 --2- 172.21.15.83:0/1295966910 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb8fc0a3d20 0x7fb8fc0a61e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fb90402a930 tx=0x7fb904031400 comp rx=0 tx=0).stop 2024-04-24T09:03:32.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.567+0000 7fb917200640 1 -- 172.21.15.83:0/1295966910 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb918104270 msgr2=0x7fb9181a05d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:32.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.567+0000 7fb917200640 1 --2- 172.21.15.83:0/1295966910 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb918104270 0x7fb9181a05d0 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7fb90800d950 tx=0x7fb90800de10 comp rx=0 tx=0).stop 2024-04-24T09:03:32.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.567+0000 7fb917200640 1 -- 172.21.15.83:0/1295966910 shutdown_connections 2024-04-24T09:03:32.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.567+0000 7fb917200640 1 --2- 172.21.15.83:0/1295966910 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb8fc0a3d20 0x7fb8fc0a61e0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:32.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.567+0000 7fb917200640 1 --2- 172.21.15.83:0/1295966910 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb91819a690 0x7fb91819ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:32.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.567+0000 7fb917200640 1 --2- 172.21.15.83:0/1295966910 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb918109340 0x7fb9181a0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:32.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.567+0000 7fb917200640 1 --2- 172.21.15.83:0/1295966910 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb918104270 0x7fb9181a05d0 unknown :-1 s=CLOSED pgs=1888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:32.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.567+0000 7fb917200640 1 -- 172.21.15.83:0/1295966910 >> 172.21.15.83:0/1295966910 conn(0x7fb9180ffa90 msgr2=0x7fb918100db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:32.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.568+0000 7fb917200640 1 -- 172.21.15.83:0/1295966910 shutdown_connections 2024-04-24T09:03:32.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.568+0000 7fb917200640 1 -- 172.21.15.83:0/1295966910 wait complete. 2024-04-24T09:03:32.585 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-24T09:03:32.620 INFO:tasks.workunit.client.0.smithi083.stdout: 1 68926 0.60 MB/sec execute 486 sec latency 64.426 ms 2024-04-24T09:03:32.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.800+0000 7f5dde000640 1 -- 172.21.15.83:0/3835738183 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dd8109520 msgr2=0x7f5dd81110f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:32.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.800+0000 7f5dde000640 1 --2- 172.21.15.83:0/3835738183 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dd8109520 0x7f5dd81110f0 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7f5dd0009ca0 tx=0x7f5dd002a8c0 comp rx=0 tx=0).stop 2024-04-24T09:03:32.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.800+0000 7f5dde000640 1 -- 172.21.15.83:0/3835738183 shutdown_connections 2024-04-24T09:03:32.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.800+0000 7f5dde000640 1 --2- 172.21.15.83:0/3835738183 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dd8109520 0x7f5dd81110f0 unknown :-1 s=CLOSED pgs=1889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:32.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.800+0000 7f5dde000640 1 --2- 172.21.15.83:0/3835738183 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5dd8104e00 0x7f5dd8108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:32.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.800+0000 7f5dde000640 1 --2- 172.21.15.83:0/3835738183 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5dd8104450 0x7f5dd8104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:32.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.800+0000 7f5dde000640 1 -- 172.21.15.83:0/3835738183 >> 172.21.15.83:0/3835738183 conn(0x7f5dd80ffc70 msgr2=0x7f5dd8102090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:32.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.801+0000 7f5dde000640 1 -- 172.21.15.83:0/3835738183 shutdown_connections 2024-04-24T09:03:32.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.801+0000 7f5dde000640 1 -- 172.21.15.83:0/3835738183 wait complete. 2024-04-24T09:03:32.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.801+0000 7f5dde000640 1 Processor -- start 2024-04-24T09:03:32.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.802+0000 7f5dde000640 1 -- start start 2024-04-24T09:03:32.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.802+0000 7f5dde000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dd8104450 0x7f5dd81a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:32.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.802+0000 7f5dde000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5dd8109520 0x7f5dd81a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:32.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.802+0000 7f5dde000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5dd819a790 0x7f5dd819ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:32.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.802+0000 7f5dde000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5dd8114470 con 0x7f5dd8104450 2024-04-24T09:03:32.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.802+0000 7f5dde000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5dd81142f0 con 0x7f5dd819a790 2024-04-24T09:03:32.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.802+0000 7f5dde000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5dd81145f0 con 0x7f5dd8109520 2024-04-24T09:03:32.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.802+0000 7f5dd7400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dd8104450 0x7f5dd81a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:32.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.802+0000 7f5dd7400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dd8104450 0x7f5dd81a0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56276/0 (socket says 172.21.15.83:56276) 2024-04-24T09:03:32.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.802+0000 7f5dd7400640 1 -- 172.21.15.83:0/2983226383 learned_addr learned my addr 172.21.15.83:0/2983226383 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:03:32.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.803+0000 7f5dd7e00640 1 --2- 172.21.15.83:0/2983226383 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5dd819a790 0x7f5dd819ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:32.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.803+0000 7f5dd6a00640 1 --2- 172.21.15.83:0/2983226383 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5dd8109520 0x7f5dd81a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:32.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.803+0000 7f5dd7400640 1 -- 172.21.15.83:0/2983226383 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5dd8109520 msgr2=0x7f5dd81a0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:32.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.803+0000 7f5dd7400640 1 --2- 172.21.15.83:0/2983226383 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5dd8109520 0x7f5dd81a0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:32.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.803+0000 7f5dd7400640 1 -- 172.21.15.83:0/2983226383 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5dd819a790 msgr2=0x7f5dd819ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:32.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.803+0000 7f5dd7400640 1 --2- 172.21.15.83:0/2983226383 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5dd819a790 0x7f5dd819ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:32.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.803+0000 7f5dd7400640 1 -- 172.21.15.83:0/2983226383 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dd819b320 con 0x7f5dd8104450 2024-04-24T09:03:32.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.803+0000 7f5dd7400640 1 --2- 172.21.15.83:0/2983226383 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dd8104450 0x7f5dd81a0620 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7f5dc800c920 tx=0x7f5dc800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:32.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.803+0000 7f5dcfe00640 1 -- 172.21.15.83:0/2983226383 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dc8007c70 con 0x7f5dd8104450 2024-04-24T09:03:32.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.803+0000 7f5dcfe00640 1 -- 172.21.15.83:0/2983226383 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5dc8007e10 con 0x7f5dd8104450 2024-04-24T09:03:32.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.803+0000 7f5dde000640 1 -- 172.21.15.83:0/2983226383 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dd819b610 con 0x7f5dd8104450 2024-04-24T09:03:32.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.804+0000 7f5dcfe00640 1 -- 172.21.15.83:0/2983226383 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dc8018820 con 0x7f5dd8104450 2024-04-24T09:03:32.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.804+0000 7f5dde000640 1 -- 172.21.15.83:0/2983226383 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dd81a77f0 con 0x7f5dd8104450 2024-04-24T09:03:32.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.807+0000 7f5de0e16740 1 Processor -- start 2024-04-24T09:03:32.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.807+0000 7f5de0e16740 1 -- start start 2024-04-24T09:03:32.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.807+0000 7f5dcfe00640 1 -- 172.21.15.83:0/2983226383 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5dc80189c0 con 0x7f5dd8104450 2024-04-24T09:03:32.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.807+0000 7f5de0e16740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af3a7c6950 0x55af3a7c6d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:32.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.808+0000 7f5de0e16740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55af3a7c7270 0x55af3a7d1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:32.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.808+0000 7f5de0e16740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55af3a7d1940 0x55af3a7d3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:32.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.808+0000 7f5de0e16740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55af3a72c1d0 con 0x55af3a7c7270 2024-04-24T09:03:32.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.808+0000 7f5de0e16740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55af3a6d2450 con 0x55af3a7c6950 2024-04-24T09:03:32.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.808+0000 7f5de0e16740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55af3a728190 con 0x55af3a7d1940 2024-04-24T09:03:32.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.808+0000 7f5dd7400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55af3a7c7270 0x55af3a7d1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:32.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.808+0000 7f5dd7400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55af3a7c7270 0x55af3a7d1400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56288/0 (socket says 172.21.15.83:56288) 2024-04-24T09:03:32.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.808+0000 7f5dd7400640 1 -- 172.21.15.83:0/1255946701 learned_addr learned my addr 172.21.15.83:0/1255946701 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:03:32.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.808+0000 7f5dd7e00640 1 --2- 172.21.15.83:0/1255946701 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af3a7c6950 0x55af3a7c6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:32.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.808+0000 7f5dd6a00640 1 --2- 172.21.15.83:0/1255946701 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55af3a7d1940 0x55af3a7d3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:32.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.808+0000 7f5dd7400640 1 -- 172.21.15.83:0/1255946701 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55af3a7d1940 msgr2=0x55af3a7d3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:32.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.808+0000 7f5dd7400640 1 --2- 172.21.15.83:0/1255946701 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55af3a7d1940 0x55af3a7d3d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:32.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.808+0000 7f5dd7400640 1 -- 172.21.15.83:0/1255946701 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af3a7c6950 msgr2=0x55af3a7c6d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:32.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.808+0000 7f5dd7400640 1 --2- 172.21.15.83:0/1255946701 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af3a7c6950 0x55af3a7c6d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:32.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.808+0000 7f5dd7400640 1 -- 172.21.15.83:0/1255946701 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af3a78f310 con 0x55af3a7c7270 2024-04-24T09:03:32.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.809+0000 7f5dd7400640 1 --2- 172.21.15.83:0/1255946701 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55af3a7c7270 0x55af3a7d1400 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7f5dc80ce080 tx=0x7f5dc80d91a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c610970dbe1aee0e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:32.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.809+0000 7f5dc7e00640 1 -- 172.21.15.83:0/1255946701 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dc80e4070 con 0x55af3a7c7270 2024-04-24T09:03:32.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.809+0000 7f5dcfe00640 1 --2- 172.21.15.83:0/2983226383 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f5dbc0a3d20 0x7f5dbc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:32.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.809+0000 7f5dc7e00640 1 -- 172.21.15.83:0/1255946701 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5dc80d9d80 con 0x55af3a7c7270 2024-04-24T09:03:32.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.809+0000 7f5dc7e00640 1 -- 172.21.15.83:0/1255946701 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dc8098c30 con 0x55af3a7c7270 2024-04-24T09:03:32.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.809+0000 7f5dcfe00640 1 -- 172.21.15.83:0/2983226383 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9583+0+0 (secure 0 0 0) 0x7f5dc80d0e70 con 0x7f5dd8104450 2024-04-24T09:03:32.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.810+0000 7f5dd7e00640 1 --2- 172.21.15.83:0/2983226383 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f5dbc0a3d20 0x7f5dbc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:32.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.810+0000 7f5de0e16740 1 -- 172.21.15.83:0/1255946701 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55af3a7c7270 msgr2=0x55af3a7d1400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:32.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.810+0000 7f5de0e16740 1 --2- 172.21.15.83:0/1255946701 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55af3a7c7270 0x55af3a7d1400 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7f5dc80ce080 tx=0x7f5dc80d91a0 comp rx=0 tx=0).stop 2024-04-24T09:03:32.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.810+0000 7f5de0e16740 1 -- 172.21.15.83:0/1255946701 shutdown_connections 2024-04-24T09:03:32.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.810+0000 7f5de0e16740 1 --2- 172.21.15.83:0/1255946701 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55af3a7d1940 0x55af3a7d3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:32.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.810+0000 7f5de0e16740 1 --2- 172.21.15.83:0/1255946701 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55af3a7c7270 0x55af3a7d1400 unknown :-1 s=CLOSED pgs=1891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:32.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.810+0000 7f5de0e16740 1 --2- 172.21.15.83:0/1255946701 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af3a7c6950 0x55af3a7c6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:32.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.810+0000 7f5de0e16740 1 -- 172.21.15.83:0/1255946701 >> 172.21.15.83:0/1255946701 conn(0x55af3a7c2c70 msgr2=0x55af3a7c3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:32.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.810+0000 7f5de0e16740 1 -- 172.21.15.83:0/1255946701 shutdown_connections 2024-04-24T09:03:32.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.810+0000 7f5dd7e00640 1 --2- 172.21.15.83:0/2983226383 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f5dbc0a3d20 0x7f5dbc0a61e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f5dd0031040 tx=0x7f5dd00093f0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:32.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.810+0000 7f5de0e16740 1 -- 172.21.15.83:0/1255946701 wait complete. 2024-04-24T09:03:32.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.811+0000 7f5de0e16740 1 -- start start 2024-04-24T09:03:32.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.811+0000 7f5de0e16740 1 Processor -- start 2024-04-24T09:03:32.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.811+0000 7f5de0e16740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af3a7c6950 0x55af3a7d13c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:32.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.811+0000 7f5de0e16740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55af3a7c7270 0x55af3a7ce6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:32.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.812+0000 7f5de0e16740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55af3a7d1940 0x55af3a7cedf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:32.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.812+0000 7f5de0e16740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55af3a7d47d0 con 0x55af3a7c7270 2024-04-24T09:03:32.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.812+0000 7f5de0e16740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55af3a6e5810 con 0x55af3a7c6950 2024-04-24T09:03:32.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.812+0000 7f5de0e16740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55af3a7d4950 con 0x55af3a7d1940 2024-04-24T09:03:32.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.812+0000 7f5de0e16740 10 client.0 _ll_register_callbacks cb 0x55af3a6b3140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:03:32.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.812+0000 7f5dd6a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55af3a7c7270 0x55af3a7ce6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:32.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.812+0000 7f5dd6a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55af3a7c7270 0x55af3a7ce6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56292/0 (socket says 172.21.15.83:56292) 2024-04-24T09:03:32.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.812+0000 7f5dd6a00640 1 -- 172.21.15.83:0/2497874036 learned_addr learned my addr 172.21.15.83:0/2497874036 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:03:32.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.812+0000 7f5dd7400640 1 --2- 172.21.15.83:0/2497874036 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af3a7c6950 0x55af3a7d13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:32.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.812+0000 7f5dd6a00640 1 -- 172.21.15.83:0/2497874036 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55af3a7d1940 msgr2=0x55af3a7cedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:32.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.812+0000 7f5dd6a00640 1 --2- 172.21.15.83:0/2497874036 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55af3a7d1940 0x55af3a7cedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:32.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.812+0000 7f5dd6a00640 1 -- 172.21.15.83:0/2497874036 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af3a7c6950 msgr2=0x55af3a7d13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:32.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.812+0000 7f5dd6a00640 1 --2- 172.21.15.83:0/2497874036 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af3a7c6950 0x55af3a7d13c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:32.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.812+0000 7f5dd6a00640 1 -- 172.21.15.83:0/2497874036 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af3a6f63f0 con 0x55af3a7c7270 2024-04-24T09:03:32.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.813+0000 7f5dd6a00640 1 --2- 172.21.15.83:0/2497874036 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55af3a7c7270 0x55af3a7ce6b0 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7f5dc000ae70 tx=0x7f5dc000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5da0c44bd4ed089b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:32.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.813+0000 7f5dc4c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:03:32.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.813+0000 7f5dc4c00640 1 -- 172.21.15.83:0/2497874036 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dc0004220 con 0x55af3a7c7270 2024-04-24T09:03:32.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.813+0000 7f5dc4c00640 1 -- 172.21.15.83:0/2497874036 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9583+0+0 (secure 0 0 0) 0x7f5dc00043c0 con 0x55af3a7c7270 2024-04-24T09:03:32.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.813+0000 7f5de0e16740 1 -- 172.21.15.83:0/2497874036 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5dc80d9d80 con 0x55af3a7c7270 2024-04-24T09:03:32.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.813+0000 7f5dc4c00640 1 -- 172.21.15.83:0/2497874036 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f5dc0011650 con 0x55af3a7c7270 2024-04-24T09:03:32.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.813+0000 7f5de0e16740 10 client.19836 fetch_fsmap learned FSMap version 41 2024-04-24T09:03:32.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.813+0000 7f5de0e16740 1 -- 172.21.15.83:0/2497874036 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55af3a43e3f0 con 0x55af3a7c7270 2024-04-24T09:03:32.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.813+0000 7f5dc4c00640 1 -- 172.21.15.83:0/2497874036 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5dc80d9d80 con 0x55af3a7c7270 2024-04-24T09:03:32.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.814+0000 7f5dc4c00640 1 -- 172.21.15.83:0/2497874036 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7f5dc0003a00 con 0x55af3a7c7270 2024-04-24T09:03:32.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.814+0000 7f5de0e16740 10 client.19836 fetch_fsmap finished waiting for FSMap version 41 2024-04-24T09:03:32.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.814+0000 7f5de0e16740 10 client.19836 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-24T09:03:32.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.814+0000 7f5de0e16740 20 client.19836 populate_metadata read hostname 'smithi083' 2024-04-24T09:03:32.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.814+0000 7f5de0e16740 1 --2- 172.21.15.83:0/2497874036 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55af3a7e1c00 0x55af3a7e3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:32.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.814+0000 7f5de0e16740 4 client.19836 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:03:32.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.814+0000 7f5de0e16740 1 -- 172.21.15.83:0/2497874036 --> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55af3a739bf0 con 0x55af3a7e1c00 2024-04-24T09:03:32.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.814+0000 7f5dd7400640 1 --2- 172.21.15.83:0/2497874036 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55af3a7e1c00 0x55af3a7e3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:32.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.815+0000 7f5dd7400640 1 --2- 172.21.15.83:0/2497874036 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55af3a7e1c00 0x55af3a7e3ff0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d72a52b0da90ee0 server_cookie=ea169e5e3d97a36c in_seq=0 out_seq=0 2024-04-24T09:03:32.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.815+0000 7f5dc4c00640 10 client.19836 ms_handle_connect on v2:172.21.15.189:6838/2548965336 2024-04-24T09:03:32.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.817+0000 7f5dc4c00640 1 -- 172.21.15.83:0/2497874036 <== mds.4 v2:172.21.15.189:6838/2548965336 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55af3a739bf0 con 0x55af3a7e1c00 2024-04-24T09:03:32.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.817+0000 7f5dc4c00640 10 client.19836 handle_command_reply: tid=0 2024-04-24T09:03:32.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.817+0000 7f5de0e16740 1 client.19836 shutdown 2024-04-24T09:03:32.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.817+0000 7f5de0e16740 1 -- 172.21.15.83:0/2497874036 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55af3a7c7270 msgr2=0x55af3a7ce6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:32.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.817+0000 7f5de0e16740 1 --2- 172.21.15.83:0/2497874036 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55af3a7c7270 0x55af3a7ce6b0 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7f5dc000ae70 tx=0x7f5dc000f730 comp rx=0 tx=0).stop 2024-04-24T09:03:32.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.817+0000 7f5de0e16740 1 -- 172.21.15.83:0/2497874036 shutdown_connections 2024-04-24T09:03:32.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.817+0000 7f5de0e16740 1 --2- 172.21.15.83:0/2497874036 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55af3a7e1c00 0x55af3a7e3ff0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:32.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.818+0000 7f5de0e16740 1 --2- 172.21.15.83:0/2497874036 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55af3a7d1940 0x55af3a7cedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:32.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.818+0000 7f5de0e16740 1 --2- 172.21.15.83:0/2497874036 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55af3a7c7270 0x55af3a7ce6b0 unknown :-1 s=CLOSED pgs=1892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:32.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.818+0000 7f5de0e16740 1 --2- 172.21.15.83:0/2497874036 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af3a7c6950 0x55af3a7d13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:32.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.818+0000 7f5dc4c00640 0 client.19836 ms_handle_reset on v2:172.21.15.189:6838/2548965336 2024-04-24T09:03:32.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.818+0000 7f5de0e16740 1 -- 172.21.15.83:0/2497874036 >> 172.21.15.83:0/2497874036 conn(0x55af3a7c2c70 msgr2=0x55af3a7c1df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:32.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.818+0000 7f5de0e16740 1 -- 172.21.15.83:0/2497874036 shutdown_connections 2024-04-24T09:03:32.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.818+0000 7f5de0e16740 1 -- 172.21.15.83:0/2497874036 wait complete. 2024-04-24T09:03:32.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.818+0000 7f5de0e16740 20 client.19836 trim_cache size 0 max 16384 2024-04-24T09:03:32.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.833+0000 7f5de0e16740 1 Processor -- start 2024-04-24T09:03:32.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.833+0000 7f5de0e16740 1 -- start start 2024-04-24T09:03:32.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.833+0000 7f5de0e16740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af3a7c6950 0x55af3a7dd690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:32.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.833+0000 7f5de0e16740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55af3a7c7270 0x55af3a7cd690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:32.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.833+0000 7f5de0e16740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55af3a7d1940 0x55af3a7d0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:32.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.833+0000 7f5de0e16740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5dd81b1880 con 0x55af3a7c7270 2024-04-24T09:03:32.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.833+0000 7f5de0e16740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5dd81b34e0 con 0x55af3a7c6950 2024-04-24T09:03:32.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.833+0000 7f5de0e16740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5dd81b1f50 con 0x55af3a7d1940 2024-04-24T09:03:32.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.834+0000 7f5dd6a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55af3a7c7270 0x55af3a7cd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:32.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.834+0000 7f5dd6a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55af3a7c7270 0x55af3a7cd690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56294/0 (socket says 172.21.15.83:56294) 2024-04-24T09:03:32.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.834+0000 7f5dd6a00640 1 -- 172.21.15.83:0/468870428 learned_addr learned my addr 172.21.15.83:0/468870428 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:03:32.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.834+0000 7f5dd7400640 1 --2- 172.21.15.83:0/468870428 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af3a7c6950 0x55af3a7dd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:32.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.834+0000 7f5dd7e00640 1 --2- 172.21.15.83:0/468870428 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55af3a7d1940 0x55af3a7d0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:32.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.834+0000 7f5dd6a00640 1 -- 172.21.15.83:0/468870428 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55af3a7d1940 msgr2=0x55af3a7d0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:32.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.834+0000 7f5dd6a00640 1 --2- 172.21.15.83:0/468870428 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55af3a7d1940 0x55af3a7d0ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:32.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.834+0000 7f5dd6a00640 1 -- 172.21.15.83:0/468870428 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af3a7c6950 msgr2=0x55af3a7dd690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:32.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.834+0000 7f5dd6a00640 1 --2- 172.21.15.83:0/468870428 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af3a7c6950 0x55af3a7dd690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:32.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.834+0000 7f5dd6a00640 1 -- 172.21.15.83:0/468870428 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af3a5087d0 con 0x55af3a7c7270 2024-04-24T09:03:32.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.834+0000 7f5dd7e00640 1 --2- 172.21.15.83:0/468870428 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55af3a7d1940 0x55af3a7d0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:03:32.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.835+0000 7f5dd6a00640 1 --2- 172.21.15.83:0/468870428 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55af3a7c7270 0x55af3a7cd690 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7f5dc0003e20 tx=0x7f5dc000a4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99bf10323686ef59 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:32.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.835+0000 7f5dc7400640 1 -- 172.21.15.83:0/468870428 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dc00041c0 con 0x55af3a7c7270 2024-04-24T09:03:32.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.835+0000 7f5dc7400640 1 -- 172.21.15.83:0/468870428 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5dc002b070 con 0x55af3a7c7270 2024-04-24T09:03:32.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.835+0000 7f5dc7400640 1 -- 172.21.15.83:0/468870428 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dc0011bf0 con 0x55af3a7c7270 2024-04-24T09:03:32.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.835+0000 7f5de0e16740 1 -- 172.21.15.83:0/468870428 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55af3a7c7270 msgr2=0x55af3a7cd690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:32.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.835+0000 7f5de0e16740 1 --2- 172.21.15.83:0/468870428 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55af3a7c7270 0x55af3a7cd690 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7f5dc0003e20 tx=0x7f5dc000a4a0 comp rx=0 tx=0).stop 2024-04-24T09:03:32.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.836+0000 7f5de0e16740 1 -- 172.21.15.83:0/468870428 shutdown_connections 2024-04-24T09:03:32.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.836+0000 7f5de0e16740 1 --2- 172.21.15.83:0/468870428 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55af3a7d1940 0x55af3a7d0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:32.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.836+0000 7f5de0e16740 1 --2- 172.21.15.83:0/468870428 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55af3a7c7270 0x55af3a7cd690 unknown :-1 s=CLOSED pgs=1893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:32.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.836+0000 7f5de0e16740 1 --2- 172.21.15.83:0/468870428 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af3a7c6950 0x55af3a7dd690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:32.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.836+0000 7f5de0e16740 1 -- 172.21.15.83:0/468870428 >> 172.21.15.83:0/468870428 conn(0x55af3a7c2c70 msgr2=0x55af3a7ccf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:32.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.836+0000 7f5de0e16740 1 -- 172.21.15.83:0/468870428 shutdown_connections 2024-04-24T09:03:32.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.836+0000 7f5de0e16740 1 -- 172.21.15.83:0/468870428 wait complete. 2024-04-24T09:03:32.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.836+0000 7f5de0e16740 1 -- start start 2024-04-24T09:03:32.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.836+0000 7f5de0e16740 1 Processor -- start 2024-04-24T09:03:32.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.837+0000 7f5de0e16740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af3a7c6950 0x55af3a7da050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:32.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.837+0000 7f5de0e16740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55af3a7c7270 0x55af3a7d2420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:32.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.837+0000 7f5de0e16740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55af3a7d1940 0x55af3a7d2980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:32.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.837+0000 7f5de0e16740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5dd81b2db0 con 0x55af3a7d1940 2024-04-24T09:03:32.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.837+0000 7f5de0e16740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5dd81b2680 con 0x55af3a7c6950 2024-04-24T09:03:32.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.837+0000 7f5de0e16740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5dd81af880 con 0x55af3a7c7270 2024-04-24T09:03:32.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.837+0000 7f5de0e16740 10 client.0 _ll_register_callbacks cb 0x7f5dc0005f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:03:32.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.837+0000 7f5dd7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55af3a7d1940 0x55af3a7d2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:32.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.837+0000 7f5dd7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55af3a7d1940 0x55af3a7d2980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56298/0 (socket says 172.21.15.83:56298) 2024-04-24T09:03:32.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.837+0000 7f5dd7e00640 1 -- 172.21.15.83:0/607804869 learned_addr learned my addr 172.21.15.83:0/607804869 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:03:32.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.837+0000 7f5dd7400640 1 --2- 172.21.15.83:0/607804869 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af3a7c6950 0x55af3a7da050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:32.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.837+0000 7f5dd6a00640 1 --2- 172.21.15.83:0/607804869 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55af3a7c7270 0x55af3a7d2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:32.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.838+0000 7f5dd7e00640 1 -- 172.21.15.83:0/607804869 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55af3a7c7270 msgr2=0x55af3a7d2420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:32.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.838+0000 7f5dd7e00640 1 --2- 172.21.15.83:0/607804869 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55af3a7c7270 0x55af3a7d2420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:32.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.838+0000 7f5dd7e00640 1 -- 172.21.15.83:0/607804869 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af3a7c6950 msgr2=0x55af3a7da050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:32.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.838+0000 7f5dd7e00640 1 --2- 172.21.15.83:0/607804869 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af3a7c6950 0x55af3a7da050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:32.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.838+0000 7f5dd7e00640 1 -- 172.21.15.83:0/607804869 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af3a7d31b0 con 0x55af3a7d1940 2024-04-24T09:03:32.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.838+0000 7f5dd7e00640 1 --2- 172.21.15.83:0/607804869 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55af3a7d1940 0x55af3a7d2980 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7f5dd0035000 tx=0x7f5dd002a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa6107f6e7226e5b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:32.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.838+0000 7f5dc4c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:03:32.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.838+0000 7f5dc4c00640 1 -- 172.21.15.83:0/607804869 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd0055060 con 0x55af3a7d1940 2024-04-24T09:03:32.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.838+0000 7f5de0e16740 1 -- 172.21.15.83:0/607804869 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5dc002b070 con 0x55af3a7d1940 2024-04-24T09:03:32.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.838+0000 7f5dc4c00640 1 -- 172.21.15.83:0/607804869 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9583+0+0 (secure 0 0 0) 0x7f5dd0004250 con 0x55af3a7d1940 2024-04-24T09:03:32.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.838+0000 7f5dc4c00640 1 -- 172.21.15.83:0/607804869 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f5dd0004490 con 0x55af3a7d1940 2024-04-24T09:03:32.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.839+0000 7f5de0e16740 10 client.19842 fetch_fsmap learned FSMap version 41 2024-04-24T09:03:32.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.839+0000 7f5de0e16740 1 -- 172.21.15.83:0/607804869 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55af3a8481c0 con 0x55af3a7d1940 2024-04-24T09:03:32.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.839+0000 7f5dc4c00640 1 -- 172.21.15.83:0/607804869 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5dc002b070 con 0x55af3a7d1940 2024-04-24T09:03:32.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.839+0000 7f5dc4c00640 1 -- 172.21.15.83:0/607804869 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7f5dd0044040 con 0x55af3a7d1940 2024-04-24T09:03:32.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.839+0000 7f5de0e16740 10 client.19842 fetch_fsmap finished waiting for FSMap version 41 2024-04-24T09:03:32.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.839+0000 7f5de0e16740 10 client.19842 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-24T09:03:32.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.839+0000 7f5de0e16740 20 client.19842 populate_metadata read hostname 'smithi083' 2024-04-24T09:03:32.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.839+0000 7f5de0e16740 1 --2- 172.21.15.83:0/607804869 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55af3a848920 0x55af3a84ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:32.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.839+0000 7f5de0e16740 4 client.19842 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:03:32.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.839+0000 7f5de0e16740 1 -- 172.21.15.83:0/607804869 --> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55af3a733a60 con 0x55af3a848920 2024-04-24T09:03:32.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.840+0000 7f5dd7400640 1 --2- 172.21.15.83:0/607804869 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55af3a848920 0x55af3a84ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:32.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.840+0000 7f5dd7400640 1 --2- 172.21.15.83:0/607804869 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55af3a848920 0x55af3a84ad40 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d96903ca954f2788 server_cookie=c677ff694665eb6b in_seq=0 out_seq=0 2024-04-24T09:03:32.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.840+0000 7f5dc4c00640 10 client.19842 ms_handle_connect on v2:172.21.15.189:6838/2548965336 2024-04-24T09:03:32.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.841+0000 7f5dc4c00640 1 -- 172.21.15.83:0/607804869 <== mds.4 v2:172.21.15.189:6838/2548965336 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55af3a733a60 con 0x55af3a848920 2024-04-24T09:03:32.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.841+0000 7f5dc4c00640 10 client.19842 handle_command_reply: tid=0 2024-04-24T09:03:32.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.841+0000 7f5de0e16740 1 client.19842 shutdown 2024-04-24T09:03:32.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.841+0000 7f5de0e16740 1 -- 172.21.15.83:0/607804869 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55af3a7d1940 msgr2=0x55af3a7d2980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:32.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.841+0000 7f5de0e16740 1 --2- 172.21.15.83:0/607804869 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55af3a7d1940 0x55af3a7d2980 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7f5dd0035000 tx=0x7f5dd002a8c0 comp rx=0 tx=0).stop 2024-04-24T09:03:32.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.841+0000 7f5de0e16740 1 -- 172.21.15.83:0/607804869 shutdown_connections 2024-04-24T09:03:32.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.841+0000 7f5de0e16740 1 --2- 172.21.15.83:0/607804869 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55af3a848920 0x55af3a84ad40 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:32.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.841+0000 7f5de0e16740 1 --2- 172.21.15.83:0/607804869 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55af3a7d1940 0x55af3a7d2980 unknown :-1 s=CLOSED pgs=1894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:32.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.841+0000 7f5de0e16740 1 --2- 172.21.15.83:0/607804869 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55af3a7c7270 0x55af3a7d2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:32.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.841+0000 7f5de0e16740 1 --2- 172.21.15.83:0/607804869 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af3a7c6950 0x55af3a7da050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:32.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.841+0000 7f5de0e16740 1 -- 172.21.15.83:0/607804869 >> 172.21.15.83:0/607804869 conn(0x55af3a7c2c70 msgr2=0x55af3a7c8930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:32.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.841+0000 7f5dc4c00640 0 client.19842 ms_handle_reset on v2:172.21.15.189:6838/2548965336 2024-04-24T09:03:32.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.842+0000 7f5de0e16740 1 -- 172.21.15.83:0/607804869 shutdown_connections 2024-04-24T09:03:32.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.842+0000 7f5de0e16740 1 -- 172.21.15.83:0/607804869 wait complete. 2024-04-24T09:03:32.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.842+0000 7f5de0e16740 20 client.19842 trim_cache size 0 max 16384 2024-04-24T09:03:32.844 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:03:32.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.842+0000 7f5dde000640 1 -- 172.21.15.83:0/2983226383 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f5dbc0a3d20 msgr2=0x7f5dbc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:32.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.842+0000 7f5dde000640 1 --2- 172.21.15.83:0/2983226383 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f5dbc0a3d20 0x7f5dbc0a61e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f5dd0031040 tx=0x7f5dd00093f0 comp rx=0 tx=0).stop 2024-04-24T09:03:32.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.842+0000 7f5dde000640 1 -- 172.21.15.83:0/2983226383 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dd8104450 msgr2=0x7f5dd81a0620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:32.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.842+0000 7f5dde000640 1 --2- 172.21.15.83:0/2983226383 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dd8104450 0x7f5dd81a0620 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7f5dc800c920 tx=0x7f5dc800cde0 comp rx=0 tx=0).stop 2024-04-24T09:03:32.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.842+0000 7f5dde000640 1 -- 172.21.15.83:0/2983226383 shutdown_connections 2024-04-24T09:03:32.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.842+0000 7f5dde000640 1 --2- 172.21.15.83:0/2983226383 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f5dbc0a3d20 0x7f5dbc0a61e0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:32.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.842+0000 7f5dde000640 1 --2- 172.21.15.83:0/2983226383 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5dd819a790 0x7f5dd819ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:32.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.842+0000 7f5dde000640 1 --2- 172.21.15.83:0/2983226383 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5dd8109520 0x7f5dd81a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:32.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.842+0000 7f5dde000640 1 --2- 172.21.15.83:0/2983226383 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dd8104450 0x7f5dd81a0620 unknown :-1 s=CLOSED pgs=1890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:32.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.843+0000 7f5dde000640 1 -- 172.21.15.83:0/2983226383 >> 172.21.15.83:0/2983226383 conn(0x7f5dd80ffc70 msgr2=0x7f5dd8101f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:32.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.843+0000 7f5dde000640 1 -- 172.21.15.83:0/2983226383 shutdown_connections 2024-04-24T09:03:32.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:32.843+0000 7f5dde000640 1 -- 172.21.15.83:0/2983226383 wait complete. 2024-04-24T09:03:32.860 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-24T09:03:33.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.120+0000 7f2e43e00640 1 -- 172.21.15.83:0/2986447987 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e441015f0 msgr2=0x7f2e4410e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:33.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.120+0000 7f2e43e00640 1 --2- 172.21.15.83:0/2986447987 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e441015f0 0x7f2e4410e2b0 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7f2e30009900 tx=0x7f2e3002a3f0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.120+0000 7f2e43e00640 1 -- 172.21.15.83:0/2986447987 shutdown_connections 2024-04-24T09:03:33.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.120+0000 7f2e43e00640 1 --2- 172.21.15.83:0/2986447987 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e4410e7f0 0x7f2e44110be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.120+0000 7f2e43e00640 1 --2- 172.21.15.83:0/2986447987 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e441015f0 0x7f2e4410e2b0 unknown :-1 s=CLOSED pgs=1895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.120+0000 7f2e43e00640 1 --2- 172.21.15.83:0/2986447987 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2e44100c40 0x7f2e44101020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.120+0000 7f2e43e00640 1 -- 172.21.15.83:0/2986447987 >> 172.21.15.83:0/2986447987 conn(0x7f2e440fc880 msgr2=0x7f2e440feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:33.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.120+0000 7f2e43e00640 1 -- 172.21.15.83:0/2986447987 shutdown_connections 2024-04-24T09:03:33.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.120+0000 7f2e43e00640 1 -- 172.21.15.83:0/2986447987 wait complete. 2024-04-24T09:03:33.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.121+0000 7f2e43e00640 1 Processor -- start 2024-04-24T09:03:33.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.122+0000 7f2e43e00640 1 -- start start 2024-04-24T09:03:33.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.122+0000 7f2e43e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e44100c40 0x7f2e441a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:33.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.122+0000 7f2e43e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e441015f0 0x7f2e441a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:33.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.122+0000 7f2e43e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2e4410e7f0 0x7f2e4419a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:33.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.122+0000 7f2e43e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2e441143f0 con 0x7f2e441015f0 2024-04-24T09:03:33.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.122+0000 7f2e43e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2e44114270 con 0x7f2e44100c40 2024-04-24T09:03:33.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.122+0000 7f2e43e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2e44114570 con 0x7f2e4410e7f0 2024-04-24T09:03:33.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.122+0000 7f2e42000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e441015f0 0x7f2e441a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:33.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.122+0000 7f2e42000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e441015f0 0x7f2e441a0b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56324/0 (socket says 172.21.15.83:56324) 2024-04-24T09:03:33.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.122+0000 7f2e42000640 1 -- 172.21.15.83:0/362983657 learned_addr learned my addr 172.21.15.83:0/362983657 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:03:33.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.122+0000 7f2e42a00640 1 --2- 172.21.15.83:0/362983657 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e44100c40 0x7f2e441a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:33.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.123+0000 7f2e42000640 1 -- 172.21.15.83:0/362983657 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2e4410e7f0 msgr2=0x7f2e4419a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:33.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.123+0000 7f2e42000640 1 --2- 172.21.15.83:0/362983657 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2e4410e7f0 0x7f2e4419a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.123+0000 7f2e42000640 1 -- 172.21.15.83:0/362983657 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e44100c40 msgr2=0x7f2e441a0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:33.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.123+0000 7f2e42000640 1 --2- 172.21.15.83:0/362983657 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e44100c40 0x7f2e441a0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.123+0000 7f2e42000640 1 -- 172.21.15.83:0/362983657 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e4419afd0 con 0x7f2e441015f0 2024-04-24T09:03:33.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.123+0000 7f2e42a00640 1 --2- 172.21.15.83:0/362983657 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e44100c40 0x7f2e441a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:03:33.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.123+0000 7f2e42000640 1 --2- 172.21.15.83:0/362983657 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e441015f0 0x7f2e441a0b90 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7f2e30005ab0 tx=0x7f2e30003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:33.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.123+0000 7f2e48a00640 1 -- 172.21.15.83:0/362983657 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e3003f070 con 0x7f2e441015f0 2024-04-24T09:03:33.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.123+0000 7f2e48a00640 1 -- 172.21.15.83:0/362983657 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2e30031d60 con 0x7f2e441015f0 2024-04-24T09:03:33.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.123+0000 7f2e43e00640 1 -- 172.21.15.83:0/362983657 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e4419b260 con 0x7f2e441015f0 2024-04-24T09:03:33.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.123+0000 7f2e48a00640 1 -- 172.21.15.83:0/362983657 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e30035030 con 0x7f2e441015f0 2024-04-24T09:03:33.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.124+0000 7f2e43e00640 1 -- 172.21.15.83:0/362983657 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e441a73f0 con 0x7f2e441015f0 2024-04-24T09:03:33.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.127+0000 7f2e4cbff740 1 Processor -- start 2024-04-24T09:03:33.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.127+0000 7f2e4cbff740 1 -- start start 2024-04-24T09:03:33.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.128+0000 7f2e4cbff740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbb3621b40 0x55fbb3621f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:33.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.128+0000 7f2e48a00640 1 -- 172.21.15.83:0/362983657 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2e30042070 con 0x7f2e441015f0 2024-04-24T09:03:33.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.128+0000 7f2e4cbff740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fbb3622460 0x55fbb362c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:33.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.128+0000 7f2e4cbff740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fbb362cb30 0x55fbb362ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:33.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.128+0000 7f2e4cbff740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55fbb35871d0 con 0x55fbb3621b40 2024-04-24T09:03:33.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.128+0000 7f2e4cbff740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55fbb352d450 con 0x55fbb3622460 2024-04-24T09:03:33.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.128+0000 7f2e4cbff740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55fbb3583190 con 0x55fbb362cb30 2024-04-24T09:03:33.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.128+0000 7f2e43400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbb3621b40 0x55fbb3621f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:33.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.128+0000 7f2e43400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbb3621b40 0x55fbb3621f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56332/0 (socket says 172.21.15.83:56332) 2024-04-24T09:03:33.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.128+0000 7f2e43400640 1 -- 172.21.15.83:0/3298731247 learned_addr learned my addr 172.21.15.83:0/3298731247 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:03:33.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.128+0000 7f2e42000640 1 --2- 172.21.15.83:0/3298731247 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fbb362cb30 0x55fbb362ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:33.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.128+0000 7f2e42a00640 1 --2- 172.21.15.83:0/3298731247 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fbb3622460 0x55fbb362c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:33.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.129+0000 7f2e43400640 1 -- 172.21.15.83:0/3298731247 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fbb362cb30 msgr2=0x55fbb362ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:33.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.129+0000 7f2e43400640 1 --2- 172.21.15.83:0/3298731247 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fbb362cb30 0x55fbb362ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.129+0000 7f2e43400640 1 -- 172.21.15.83:0/3298731247 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fbb3622460 msgr2=0x55fbb362c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:33.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.129+0000 7f2e43400640 1 --2- 172.21.15.83:0/3298731247 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fbb3622460 0x55fbb362c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.129+0000 7f2e43400640 1 -- 172.21.15.83:0/3298731247 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fbb35ea310 con 0x55fbb3621b40 2024-04-24T09:03:33.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.129+0000 7f2e43400640 1 --2- 172.21.15.83:0/3298731247 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbb3621b40 0x55fbb3621f20 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c007cc0 tx=0x7f2e3c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcc80a066ef70836 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:33.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.130+0000 7f2e2fe00640 1 -- 172.21.15.83:0/3298731247 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e3c014070 con 0x55fbb3621b40 2024-04-24T09:03:33.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.130+0000 7f2e48a00640 1 --2- 172.21.15.83:0/362983657 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f2e280a3d20 0x7f2e280a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:33.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.130+0000 7f2e2fe00640 1 -- 172.21.15.83:0/3298731247 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2e3c00fb00 con 0x55fbb3621b40 2024-04-24T09:03:33.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.130+0000 7f2e2fe00640 1 -- 172.21.15.83:0/3298731247 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e3c00eb00 con 0x55fbb3621b40 2024-04-24T09:03:33.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.130+0000 7f2e48a00640 1 -- 172.21.15.83:0/362983657 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9583+0+0 (secure 0 0 0) 0x7f2e300f16f0 con 0x7f2e441015f0 2024-04-24T09:03:33.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.130+0000 7f2e42a00640 1 --2- 172.21.15.83:0/362983657 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f2e280a3d20 0x7f2e280a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:33.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.131+0000 7f2e4cbff740 1 -- 172.21.15.83:0/3298731247 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbb3621b40 msgr2=0x55fbb3621f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:33.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.131+0000 7f2e4cbff740 1 --2- 172.21.15.83:0/3298731247 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbb3621b40 0x55fbb3621f20 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c007cc0 tx=0x7f2e3c00c6a0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.131+0000 7f2e4cbff740 1 -- 172.21.15.83:0/3298731247 shutdown_connections 2024-04-24T09:03:33.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.131+0000 7f2e4cbff740 1 --2- 172.21.15.83:0/3298731247 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fbb362cb30 0x55fbb362ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.131+0000 7f2e4cbff740 1 --2- 172.21.15.83:0/3298731247 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fbb3622460 0x55fbb362c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.131+0000 7f2e4cbff740 1 --2- 172.21.15.83:0/3298731247 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbb3621b40 0x55fbb3621f20 unknown :-1 s=CLOSED pgs=1897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.131+0000 7f2e4cbff740 1 -- 172.21.15.83:0/3298731247 >> 172.21.15.83:0/3298731247 conn(0x55fbb361de60 msgr2=0x55fbb361e240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:33.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.131+0000 7f2e4cbff740 1 -- 172.21.15.83:0/3298731247 shutdown_connections 2024-04-24T09:03:33.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.131+0000 7f2e4cbff740 1 -- 172.21.15.83:0/3298731247 wait complete. 2024-04-24T09:03:33.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.131+0000 7f2e42a00640 1 --2- 172.21.15.83:0/362983657 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f2e280a3d20 0x7f2e280a61e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x55fbb3623100 tx=0x7f2e34008040 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:33.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.132+0000 7f2e4cbff740 1 -- start start 2024-04-24T09:03:33.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.132+0000 7f2e4cbff740 1 Processor -- start 2024-04-24T09:03:33.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.132+0000 7f2e4cbff740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbb3621b40 0x55fbb362c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:33.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.133+0000 7f2e4cbff740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fbb3622460 0x55fbb36298d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:33.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.133+0000 7f2e4cbff740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fbb362cb30 0x55fbb362a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:33.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.133+0000 7f2e4cbff740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55fbb362f9c0 con 0x55fbb3621b40 2024-04-24T09:03:33.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.133+0000 7f2e4cbff740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55fbb3540810 con 0x55fbb3622460 2024-04-24T09:03:33.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.133+0000 7f2e4cbff740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55fbb362fb40 con 0x55fbb362cb30 2024-04-24T09:03:33.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.133+0000 7f2e4cbff740 10 client.0 _ll_register_callbacks cb 0x55fbb350e140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:03:33.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.133+0000 7f2e42a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbb3621b40 0x55fbb362c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:33.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.133+0000 7f2e42a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbb3621b40 0x55fbb362c5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56348/0 (socket says 172.21.15.83:56348) 2024-04-24T09:03:33.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.133+0000 7f2e42a00640 1 -- 172.21.15.83:0/3129834527 learned_addr learned my addr 172.21.15.83:0/3129834527 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:03:33.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.133+0000 7f2e42000640 1 --2- 172.21.15.83:0/3129834527 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fbb3622460 0x55fbb36298d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:33.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.133+0000 7f2e43400640 1 --2- 172.21.15.83:0/3129834527 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fbb362cb30 0x55fbb362a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:33.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.133+0000 7f2e42a00640 1 -- 172.21.15.83:0/3129834527 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fbb362cb30 msgr2=0x55fbb362a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:33.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.133+0000 7f2e42a00640 1 --2- 172.21.15.83:0/3129834527 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fbb362cb30 0x55fbb362a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.133+0000 7f2e42a00640 1 -- 172.21.15.83:0/3129834527 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fbb3622460 msgr2=0x55fbb36298d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:33.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.133+0000 7f2e42a00640 1 --2- 172.21.15.83:0/3129834527 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fbb3622460 0x55fbb36298d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.133+0000 7f2e42a00640 1 -- 172.21.15.83:0/3129834527 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fbb35513f0 con 0x55fbb3621b40 2024-04-24T09:03:33.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.134+0000 7f2e42a00640 1 --2- 172.21.15.83:0/3129834527 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbb3621b40 0x55fbb362c5e0 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f2e34013700 tx=0x7f2e340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4011c5699d640ed server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:33.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.134+0000 7f2e2cc00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:03:33.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.134+0000 7f2e2cc00640 1 -- 172.21.15.83:0/3129834527 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e3401a050 con 0x55fbb3621b40 2024-04-24T09:03:33.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.134+0000 7f2e2cc00640 1 -- 172.21.15.83:0/3129834527 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9583+0+0 (secure 0 0 0) 0x7f2e34014890 con 0x55fbb3621b40 2024-04-24T09:03:33.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.134+0000 7f2e4cbff740 1 -- 172.21.15.83:0/3129834527 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e3c00fb00 con 0x55fbb3621b40 2024-04-24T09:03:33.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.134+0000 7f2e2cc00640 1 -- 172.21.15.83:0/3129834527 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f2e34014a90 con 0x55fbb3621b40 2024-04-24T09:03:33.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.134+0000 7f2e4cbff740 10 client.19854 fetch_fsmap learned FSMap version 41 2024-04-24T09:03:33.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.134+0000 7f2e4cbff740 1 -- 172.21.15.83:0/3129834527 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55fbb32993f0 con 0x55fbb3621b40 2024-04-24T09:03:33.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.135+0000 7f2e2cc00640 1 -- 172.21.15.83:0/3129834527 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2e3c00fb00 con 0x55fbb3621b40 2024-04-24T09:03:33.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.135+0000 7f2e2cc00640 1 -- 172.21.15.83:0/3129834527 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7f2e34006350 con 0x55fbb3621b40 2024-04-24T09:03:33.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.135+0000 7f2e4cbff740 10 client.19854 fetch_fsmap finished waiting for FSMap version 41 2024-04-24T09:03:33.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.135+0000 7f2e4cbff740 10 client.19854 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-24T09:03:33.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.135+0000 7f2e4cbff740 20 client.19854 populate_metadata read hostname 'smithi083' 2024-04-24T09:03:33.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.135+0000 7f2e4cbff740 1 --2- 172.21.15.83:0/3129834527 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55fbb363ce40 0x55fbb363f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:33.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.135+0000 7f2e4cbff740 4 client.19854 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:03:33.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.135+0000 7f2e4cbff740 1 -- 172.21.15.83:0/3129834527 --> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fbb3594bf0 con 0x55fbb363ce40 2024-04-24T09:03:33.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.136+0000 7f2e42000640 1 --2- 172.21.15.83:0/3129834527 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55fbb363ce40 0x55fbb363f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:33.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.137+0000 7f2e42000640 1 --2- 172.21.15.83:0/3129834527 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55fbb363ce40 0x55fbb363f2b0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=69d83671bc5d861f server_cookie=9127008a6a8f67f8 in_seq=0 out_seq=0 2024-04-24T09:03:33.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.137+0000 7f2e2cc00640 10 client.19854 ms_handle_connect on v2:172.21.15.138:6832/1157967658 2024-04-24T09:03:33.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.139+0000 7f2e2cc00640 1 -- 172.21.15.83:0/3129834527 <== mds.3 v2:172.21.15.138:6832/1157967658 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fbb3594bf0 con 0x55fbb363ce40 2024-04-24T09:03:33.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.139+0000 7f2e2cc00640 10 client.19854 handle_command_reply: tid=0 2024-04-24T09:03:33.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.139+0000 7f2e4cbff740 1 client.19854 shutdown 2024-04-24T09:03:33.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.139+0000 7f2e4cbff740 1 -- 172.21.15.83:0/3129834527 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbb3621b40 msgr2=0x55fbb362c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:33.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.139+0000 7f2e4cbff740 1 --2- 172.21.15.83:0/3129834527 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbb3621b40 0x55fbb362c5e0 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f2e34013700 tx=0x7f2e340046c0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.139+0000 7f2e4cbff740 1 -- 172.21.15.83:0/3129834527 shutdown_connections 2024-04-24T09:03:33.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.139+0000 7f2e4cbff740 1 --2- 172.21.15.83:0/3129834527 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55fbb363ce40 0x55fbb363f2b0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.139+0000 7f2e4cbff740 1 --2- 172.21.15.83:0/3129834527 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fbb362cb30 0x55fbb362a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.139+0000 7f2e4cbff740 1 --2- 172.21.15.83:0/3129834527 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fbb3622460 0x55fbb36298d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.139+0000 7f2e2cc00640 0 client.19854 ms_handle_reset on v2:172.21.15.138:6832/1157967658 2024-04-24T09:03:33.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.139+0000 7f2e4cbff740 1 --2- 172.21.15.83:0/3129834527 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbb3621b40 0x55fbb362c5e0 unknown :-1 s=CLOSED pgs=1898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.139+0000 7f2e4cbff740 1 -- 172.21.15.83:0/3129834527 >> 172.21.15.83:0/3129834527 conn(0x55fbb361de60 msgr2=0x55fbb3623b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:33.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.139+0000 7f2e4cbff740 1 -- 172.21.15.83:0/3129834527 shutdown_connections 2024-04-24T09:03:33.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.139+0000 7f2e4cbff740 1 -- 172.21.15.83:0/3129834527 wait complete. 2024-04-24T09:03:33.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.139+0000 7f2e4cbff740 20 client.19854 trim_cache size 0 max 16384 2024-04-24T09:03:33.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.157+0000 7f2e4cbff740 1 Processor -- start 2024-04-24T09:03:33.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.157+0000 7f2e4cbff740 1 -- start start 2024-04-24T09:03:33.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.157+0000 7f2e4cbff740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbb3621b40 0x55fbb36242a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:33.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.157+0000 7f2e4cbff740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fbb3622460 0x55fbb362c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:33.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.157+0000 7f2e4cbff740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fbb362cb30 0x55fbb3644b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:33.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.157+0000 7f2e4cbff740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2e441b1610 con 0x55fbb3621b40 2024-04-24T09:03:33.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.157+0000 7f2e4cbff740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2e441b3270 con 0x55fbb3622460 2024-04-24T09:03:33.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.157+0000 7f2e4cbff740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2e441b1ce0 con 0x55fbb362cb30 2024-04-24T09:03:33.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.158+0000 7f2e42a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbb3621b40 0x55fbb36242a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:33.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.158+0000 7f2e42a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbb3621b40 0x55fbb36242a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56352/0 (socket says 172.21.15.83:56352) 2024-04-24T09:03:33.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.158+0000 7f2e42a00640 1 -- 172.21.15.83:0/217517058 learned_addr learned my addr 172.21.15.83:0/217517058 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:03:33.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.158+0000 7f2e43400640 1 --2- 172.21.15.83:0/217517058 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fbb362cb30 0x55fbb3644b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:33.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.158+0000 7f2e42a00640 1 -- 172.21.15.83:0/217517058 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fbb362cb30 msgr2=0x55fbb3644b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:33.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.158+0000 7f2e42a00640 1 --2- 172.21.15.83:0/217517058 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fbb362cb30 0x55fbb3644b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.158+0000 7f2e42a00640 1 -- 172.21.15.83:0/217517058 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fbb3622460 msgr2=0x55fbb362c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:33.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.158+0000 7f2e42a00640 1 --2- 172.21.15.83:0/217517058 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fbb3622460 0x55fbb362c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.158+0000 7f2e42a00640 1 -- 172.21.15.83:0/217517058 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fbb3622290 con 0x55fbb3621b40 2024-04-24T09:03:33.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.159+0000 7f2e42a00640 1 --2- 172.21.15.83:0/217517058 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbb3621b40 0x55fbb36242a0 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7f2e3400afe0 tx=0x7f2e340047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4299cf2ef57bf9da server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:33.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.159+0000 7f2e2f400640 1 -- 172.21.15.83:0/217517058 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e34036020 con 0x55fbb3621b40 2024-04-24T09:03:33.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.159+0000 7f2e2f400640 1 -- 172.21.15.83:0/217517058 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2e34024070 con 0x55fbb3621b40 2024-04-24T09:03:33.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.159+0000 7f2e2f400640 1 -- 172.21.15.83:0/217517058 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e3401a040 con 0x55fbb3621b40 2024-04-24T09:03:33.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.160+0000 7f2e4cbff740 1 -- 172.21.15.83:0/217517058 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbb3621b40 msgr2=0x55fbb36242a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:33.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.160+0000 7f2e4cbff740 1 --2- 172.21.15.83:0/217517058 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbb3621b40 0x55fbb36242a0 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7f2e3400afe0 tx=0x7f2e340047d0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.160+0000 7f2e4cbff740 1 -- 172.21.15.83:0/217517058 shutdown_connections 2024-04-24T09:03:33.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.160+0000 7f2e4cbff740 1 --2- 172.21.15.83:0/217517058 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fbb362cb30 0x55fbb3644b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.160+0000 7f2e4cbff740 1 --2- 172.21.15.83:0/217517058 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fbb3622460 0x55fbb362c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.160+0000 7f2e4cbff740 1 --2- 172.21.15.83:0/217517058 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbb3621b40 0x55fbb36242a0 unknown :-1 s=CLOSED pgs=1899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.160+0000 7f2e4cbff740 1 -- 172.21.15.83:0/217517058 >> 172.21.15.83:0/217517058 conn(0x55fbb361de60 msgr2=0x55fbb3624bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:33.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.160+0000 7f2e4cbff740 1 -- 172.21.15.83:0/217517058 shutdown_connections 2024-04-24T09:03:33.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.160+0000 7f2e4cbff740 1 -- 172.21.15.83:0/217517058 wait complete. 2024-04-24T09:03:33.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.160+0000 7f2e4cbff740 1 -- start start 2024-04-24T09:03:33.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.161+0000 7f2e4cbff740 1 Processor -- start 2024-04-24T09:03:33.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.161+0000 7f2e4cbff740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbb3621b40 0x55fbb36352f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:33.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.161+0000 7f2e4cbff740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fbb3622460 0x55fbb362d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:33.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.161+0000 7f2e4cbff740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fbb362cb30 0x55fbb362deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:33.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.161+0000 7f2e4cbff740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2e441b2b40 con 0x55fbb3621b40 2024-04-24T09:03:33.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.161+0000 7f2e4cbff740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2e441b2410 con 0x55fbb362cb30 2024-04-24T09:03:33.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.161+0000 7f2e4cbff740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2e441af610 con 0x55fbb3622460 2024-04-24T09:03:33.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.161+0000 7f2e4cbff740 10 client.0 _ll_register_callbacks cb 0x7f2e3400a780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:03:33.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.161+0000 7f2e42a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbb3621b40 0x55fbb36352f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:33.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.161+0000 7f2e42a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbb3621b40 0x55fbb36352f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56366/0 (socket says 172.21.15.83:56366) 2024-04-24T09:03:33.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.162+0000 7f2e42a00640 1 -- 172.21.15.83:0/1868603995 learned_addr learned my addr 172.21.15.83:0/1868603995 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:03:33.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.162+0000 7f2e42000640 1 --2- 172.21.15.83:0/1868603995 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fbb3622460 0x55fbb362d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:33.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.162+0000 7f2e43400640 1 --2- 172.21.15.83:0/1868603995 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fbb362cb30 0x55fbb362deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:33.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.162+0000 7f2e42a00640 1 -- 172.21.15.83:0/1868603995 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fbb3622460 msgr2=0x55fbb362d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:33.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.162+0000 7f2e42a00640 1 --2- 172.21.15.83:0/1868603995 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fbb3622460 0x55fbb362d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.162+0000 7f2e42a00640 1 -- 172.21.15.83:0/1868603995 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fbb362cb30 msgr2=0x55fbb362deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:33.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.162+0000 7f2e42a00640 1 --2- 172.21.15.83:0/1868603995 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fbb362cb30 0x55fbb362deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.162+0000 7f2e42a00640 1 -- 172.21.15.83:0/1868603995 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fbb33637d0 con 0x55fbb3621b40 2024-04-24T09:03:33.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.162+0000 7f2e42000640 1 --2- 172.21.15.83:0/1868603995 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fbb3622460 0x55fbb362d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:03:33.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.162+0000 7f2e42a00640 1 --2- 172.21.15.83:0/1868603995 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbb3621b40 0x55fbb36352f0 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7f2e3400b010 tx=0x7f2e34032cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa51613b004b1ef4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:33.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.162+0000 7f2e2cc00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:03:33.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.163+0000 7f2e2cc00640 1 -- 172.21.15.83:0/1868603995 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e34042070 con 0x55fbb3621b40 2024-04-24T09:03:33.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.163+0000 7f2e4cbff740 1 -- 172.21.15.83:0/1868603995 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e34024070 con 0x55fbb3621b40 2024-04-24T09:03:33.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.163+0000 7f2e2cc00640 1 -- 172.21.15.83:0/1868603995 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9583+0+0 (secure 0 0 0) 0x7f2e3401f650 con 0x55fbb3621b40 2024-04-24T09:03:33.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.163+0000 7f2e2cc00640 1 -- 172.21.15.83:0/1868603995 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f2e3404e050 con 0x55fbb3621b40 2024-04-24T09:03:33.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.163+0000 7f2e4cbff740 10 client.19860 fetch_fsmap learned FSMap version 41 2024-04-24T09:03:33.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.163+0000 7f2e4cbff740 1 -- 172.21.15.83:0/1868603995 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55fbb36a33f0 con 0x55fbb3621b40 2024-04-24T09:03:33.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.163+0000 7f2e2cc00640 1 -- 172.21.15.83:0/1868603995 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2e34024070 con 0x55fbb3621b40 2024-04-24T09:03:33.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.163+0000 7f2e2cc00640 1 -- 172.21.15.83:0/1868603995 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7f2e3403b750 con 0x55fbb3621b40 2024-04-24T09:03:33.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.163+0000 7f2e4cbff740 10 client.19860 fetch_fsmap finished waiting for FSMap version 41 2024-04-24T09:03:33.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.163+0000 7f2e4cbff740 10 client.19860 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-24T09:03:33.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.163+0000 7f2e4cbff740 20 client.19860 populate_metadata read hostname 'smithi083' 2024-04-24T09:03:33.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.163+0000 7f2e4cbff740 1 --2- 172.21.15.83:0/1868603995 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55fbb36a3900 0x55fbb36a5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:33.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.163+0000 7f2e4cbff740 4 client.19860 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:03:33.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.163+0000 7f2e4cbff740 1 -- 172.21.15.83:0/1868603995 --> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fbb358ea60 con 0x55fbb36a3900 2024-04-24T09:03:33.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.164+0000 7f2e42000640 1 --2- 172.21.15.83:0/1868603995 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55fbb36a3900 0x55fbb36a5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:33.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.165+0000 7f2e42000640 1 --2- 172.21.15.83:0/1868603995 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55fbb36a3900 0x55fbb36a5cf0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=89e104f980997598 server_cookie=919e6a8a2851cacd in_seq=0 out_seq=0 2024-04-24T09:03:33.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.165+0000 7f2e2cc00640 10 client.19860 ms_handle_connect on v2:172.21.15.138:6832/1157967658 2024-04-24T09:03:33.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.165+0000 7f2e2cc00640 1 -- 172.21.15.83:0/1868603995 <== mds.3 v2:172.21.15.138:6832/1157967658 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fbb358ea60 con 0x55fbb36a3900 2024-04-24T09:03:33.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.166+0000 7f2e2cc00640 10 client.19860 handle_command_reply: tid=0 2024-04-24T09:03:33.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.166+0000 7f2e4cbff740 1 client.19860 shutdown 2024-04-24T09:03:33.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.166+0000 7f2e4cbff740 1 -- 172.21.15.83:0/1868603995 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbb3621b40 msgr2=0x55fbb36352f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:33.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.166+0000 7f2e4cbff740 1 --2- 172.21.15.83:0/1868603995 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbb3621b40 0x55fbb36352f0 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7f2e3400b010 tx=0x7f2e34032cb0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.166+0000 7f2e4cbff740 1 -- 172.21.15.83:0/1868603995 shutdown_connections 2024-04-24T09:03:33.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.166+0000 7f2e4cbff740 1 --2- 172.21.15.83:0/1868603995 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fbb362cb30 0x55fbb362deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.166+0000 7f2e4cbff740 1 --2- 172.21.15.83:0/1868603995 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55fbb36a3900 0x55fbb36a5cf0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.166+0000 7f2e4cbff740 1 --2- 172.21.15.83:0/1868603995 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fbb3622460 0x55fbb362d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.166+0000 7f2e4cbff740 1 --2- 172.21.15.83:0/1868603995 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbb3621b40 0x55fbb36352f0 unknown :-1 s=CLOSED pgs=1900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.166+0000 7f2e4cbff740 1 -- 172.21.15.83:0/1868603995 >> 172.21.15.83:0/1868603995 conn(0x55fbb361de60 msgr2=0x55fbb361c6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:33.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.166+0000 7f2e2cc00640 0 client.19860 ms_handle_reset on v2:172.21.15.138:6832/1157967658 2024-04-24T09:03:33.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.166+0000 7f2e4cbff740 1 -- 172.21.15.83:0/1868603995 shutdown_connections 2024-04-24T09:03:33.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.166+0000 7f2e4cbff740 1 -- 172.21.15.83:0/1868603995 wait complete. 2024-04-24T09:03:33.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.166+0000 7f2e4cbff740 20 client.19860 trim_cache size 0 max 16384 2024-04-24T09:03:33.173 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:03:33.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.167+0000 7f2e43e00640 1 -- 172.21.15.83:0/362983657 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f2e280a3d20 msgr2=0x7f2e280a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:33.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.167+0000 7f2e43e00640 1 --2- 172.21.15.83:0/362983657 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f2e280a3d20 0x7f2e280a61e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x55fbb3623100 tx=0x7f2e34008040 comp rx=0 tx=0).stop 2024-04-24T09:03:33.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.167+0000 7f2e43e00640 1 -- 172.21.15.83:0/362983657 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e441015f0 msgr2=0x7f2e441a0b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:33.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.167+0000 7f2e43e00640 1 --2- 172.21.15.83:0/362983657 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e441015f0 0x7f2e441a0b90 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7f2e30005ab0 tx=0x7f2e30003d50 comp rx=0 tx=0).stop 2024-04-24T09:03:33.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.167+0000 7f2e43e00640 1 -- 172.21.15.83:0/362983657 shutdown_connections 2024-04-24T09:03:33.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.167+0000 7f2e43e00640 1 --2- 172.21.15.83:0/362983657 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f2e280a3d20 0x7f2e280a61e0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.167+0000 7f2e43e00640 1 --2- 172.21.15.83:0/362983657 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2e4410e7f0 0x7f2e4419a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.167+0000 7f2e43e00640 1 --2- 172.21.15.83:0/362983657 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e441015f0 0x7f2e441a0b90 unknown :-1 s=CLOSED pgs=1896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.167+0000 7f2e43e00640 1 --2- 172.21.15.83:0/362983657 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e44100c40 0x7f2e441a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.167+0000 7f2e43e00640 1 -- 172.21.15.83:0/362983657 >> 172.21.15.83:0/362983657 conn(0x7f2e440fc880 msgr2=0x7f2e440fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:33.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.167+0000 7f2e43e00640 1 -- 172.21.15.83:0/362983657 shutdown_connections 2024-04-24T09:03:33.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.167+0000 7f2e43e00640 1 -- 172.21.15.83:0/362983657 wait complete. 2024-04-24T09:03:33.182 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:03:33 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/1295966910' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:03:33.185 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T09:03:33.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.375+0000 7f8107200640 1 -- 172.21.15.83:0/2139491215 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81001071e0 msgr2=0x7f810010edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:33.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.375+0000 7f8107200640 1 --2- 172.21.15.83:0/2139491215 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81001071e0 0x7f810010edb0 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7f80fc009ca0 tx=0x7f80fc02a8c0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.375+0000 7f8107200640 1 -- 172.21.15.83:0/2139491215 shutdown_connections 2024-04-24T09:03:33.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.375+0000 7f8107200640 1 --2- 172.21.15.83:0/2139491215 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81001071e0 0x7f810010edb0 unknown :-1 s=CLOSED pgs=1901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.375+0000 7f8107200640 1 --2- 172.21.15.83:0/2139491215 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8100102ac0 0x7f8100106b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.375+0000 7f8107200640 1 --2- 172.21.15.83:0/2139491215 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8100102110 0x7f81001024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.375+0000 7f8107200640 1 -- 172.21.15.83:0/2139491215 >> 172.21.15.83:0/2139491215 conn(0x7f81000fd9d0 msgr2=0x7f81000ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:33.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.375+0000 7f8107200640 1 -- 172.21.15.83:0/2139491215 shutdown_connections 2024-04-24T09:03:33.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.375+0000 7f8107200640 1 -- 172.21.15.83:0/2139491215 wait complete. 2024-04-24T09:03:33.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.376+0000 7f8107200640 1 Processor -- start 2024-04-24T09:03:33.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.377+0000 7f8107200640 1 -- start start 2024-04-24T09:03:33.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.377+0000 7f8107200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8100102110 0x7f81001a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:33.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.377+0000 7f8107200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81001071e0 0x7f81001a0b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:33.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.377+0000 7f8107200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f810019a6b0 0x7f810019ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:33.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.377+0000 7f8107200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8100075920 con 0x7f81001071e0 2024-04-24T09:03:33.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.377+0000 7f8107200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f81000757a0 con 0x7f8100102110 2024-04-24T09:03:33.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.377+0000 7f8107200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8100075aa0 con 0x7f810019a6b0 2024-04-24T09:03:33.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.377+0000 7f8105400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81001071e0 0x7f81001a0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:33.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.377+0000 7f8105400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81001071e0 0x7f81001a0b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56374/0 (socket says 172.21.15.83:56374) 2024-04-24T09:03:33.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.377+0000 7f8105400640 1 -- 172.21.15.83:0/2812959147 learned_addr learned my addr 172.21.15.83:0/2812959147 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:03:33.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.378+0000 7f8106800640 1 --2- 172.21.15.83:0/2812959147 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f810019a6b0 0x7f810019ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:33.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.378+0000 7f8105400640 1 -- 172.21.15.83:0/2812959147 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f810019a6b0 msgr2=0x7f810019ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:33.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.378+0000 7f8105400640 1 --2- 172.21.15.83:0/2812959147 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f810019a6b0 0x7f810019ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.378+0000 7f8105400640 1 -- 172.21.15.83:0/2812959147 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8100102110 msgr2=0x7f81001a05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:33.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.378+0000 7f8105e00640 1 --2- 172.21.15.83:0/2812959147 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8100102110 0x7f81001a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:33.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.378+0000 7f8105400640 1 --2- 172.21.15.83:0/2812959147 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8100102110 0x7f81001a05f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.378+0000 7f8105400640 1 -- 172.21.15.83:0/2812959147 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f810019b2b0 con 0x7f81001071e0 2024-04-24T09:03:33.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.378+0000 7f8105400640 1 --2- 172.21.15.83:0/2812959147 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81001071e0 0x7f81001a0b30 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7f80f000d770 tx=0x7f80f000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:33.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.378+0000 7f80faa00640 1 -- 172.21.15.83:0/2812959147 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80f0017070 con 0x7f81001071e0 2024-04-24T09:03:33.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.379+0000 7f80faa00640 1 -- 172.21.15.83:0/2812959147 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f80f0009c30 con 0x7f81001071e0 2024-04-24T09:03:33.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.379+0000 7f8107200640 1 -- 172.21.15.83:0/2812959147 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81001a7390 con 0x7f81001071e0 2024-04-24T09:03:33.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.379+0000 7f80faa00640 1 -- 172.21.15.83:0/2812959147 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80f000f620 con 0x7f81001071e0 2024-04-24T09:03:33.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.379+0000 7f8107200640 1 -- 172.21.15.83:0/2812959147 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81001a7790 con 0x7f81001071e0 2024-04-24T09:03:33.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.382+0000 7f810b653740 1 Processor -- start 2024-04-24T09:03:33.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.382+0000 7f810b653740 1 -- start start 2024-04-24T09:03:33.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.383+0000 7f80faa00640 1 -- 172.21.15.83:0/2812959147 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f80f001a070 con 0x7f81001071e0 2024-04-24T09:03:33.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.383+0000 7f810b653740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561821395b40 0x561821395f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:33.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.383+0000 7f810b653740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561821396460 0x5618213a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:33.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.383+0000 7f810b653740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5618213a0b30 0x5618213a2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:33.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.383+0000 7f810b653740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5618212fb1d0 con 0x561821396460 2024-04-24T09:03:33.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.383+0000 7f810b653740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5618212a1450 con 0x561821395b40 2024-04-24T09:03:33.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.383+0000 7f810b653740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5618212f7190 con 0x5618213a0b30 2024-04-24T09:03:33.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.383+0000 7f8105e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561821396460 0x5618213a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:33.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.383+0000 7f8105e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561821396460 0x5618213a05f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56380/0 (socket says 172.21.15.83:56380) 2024-04-24T09:03:33.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.383+0000 7f8105e00640 1 -- 172.21.15.83:0/3591715136 learned_addr learned my addr 172.21.15.83:0/3591715136 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:03:33.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.384+0000 7f8106800640 1 --2- 172.21.15.83:0/3591715136 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561821395b40 0x561821395f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:33.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.383+0000 7f8105400640 1 --2- 172.21.15.83:0/3591715136 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5618213a0b30 0x5618213a2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:33.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.384+0000 7f8105e00640 1 -- 172.21.15.83:0/3591715136 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5618213a0b30 msgr2=0x5618213a2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:33.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.384+0000 7f8105e00640 1 --2- 172.21.15.83:0/3591715136 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5618213a0b30 0x5618213a2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.384+0000 7f8105e00640 1 -- 172.21.15.83:0/3591715136 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561821395b40 msgr2=0x561821395f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:33.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.384+0000 7f8105e00640 1 --2- 172.21.15.83:0/3591715136 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561821395b40 0x561821395f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.384+0000 7f8105e00640 1 -- 172.21.15.83:0/3591715136 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56182135e310 con 0x561821396460 2024-04-24T09:03:33.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.384+0000 7f8106800640 1 --2- 172.21.15.83:0/3591715136 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561821395b40 0x561821395f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:03:33.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.384+0000 7f8105e00640 1 --2- 172.21.15.83:0/3591715136 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561821396460 0x5618213a05f0 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7f80f4009b60 tx=0x7f80f400e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e359fae8c36b3c0c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:33.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.384+0000 7f80ef000640 1 -- 172.21.15.83:0/3591715136 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80f4016070 con 0x561821396460 2024-04-24T09:03:33.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.384+0000 7f80ef000640 1 -- 172.21.15.83:0/3591715136 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f80f4009e50 con 0x561821396460 2024-04-24T09:03:33.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.385+0000 7f80ef000640 1 -- 172.21.15.83:0/3591715136 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80f4011470 con 0x561821396460 2024-04-24T09:03:33.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.385+0000 7f80faa00640 1 --2- 172.21.15.83:0/2812959147 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f80e80a3d20 0x7f80e80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:33.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.385+0000 7f80faa00640 1 -- 172.21.15.83:0/2812959147 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9583+0+0 (secure 0 0 0) 0x7f80f00d0750 con 0x7f81001071e0 2024-04-24T09:03:33.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.385+0000 7f8106800640 1 --2- 172.21.15.83:0/2812959147 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f80e80a3d20 0x7f80e80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:33.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.385+0000 7f810b653740 1 -- 172.21.15.83:0/3591715136 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561821396460 msgr2=0x5618213a05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:33.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.385+0000 7f810b653740 1 --2- 172.21.15.83:0/3591715136 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561821396460 0x5618213a05f0 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7f80f4009b60 tx=0x7f80f400e6c0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.386+0000 7f810b653740 1 -- 172.21.15.83:0/3591715136 shutdown_connections 2024-04-24T09:03:33.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.386+0000 7f810b653740 1 --2- 172.21.15.83:0/3591715136 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5618213a0b30 0x5618213a2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.386+0000 7f810b653740 1 --2- 172.21.15.83:0/3591715136 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561821396460 0x5618213a05f0 unknown :-1 s=CLOSED pgs=1903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.386+0000 7f810b653740 1 --2- 172.21.15.83:0/3591715136 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561821395b40 0x561821395f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.386+0000 7f810b653740 1 -- 172.21.15.83:0/3591715136 >> 172.21.15.83:0/3591715136 conn(0x561821391e60 msgr2=0x561821392240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:33.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.386+0000 7f8106800640 1 --2- 172.21.15.83:0/2812959147 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f80e80a3d20 0x7f80e80a61e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f80fc005fa0 tx=0x7f80fc009360 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:33.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.386+0000 7f810b653740 1 -- 172.21.15.83:0/3591715136 shutdown_connections 2024-04-24T09:03:33.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.386+0000 7f810b653740 1 -- 172.21.15.83:0/3591715136 wait complete. 2024-04-24T09:03:33.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.386+0000 7f810b653740 1 -- start start 2024-04-24T09:03:33.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.386+0000 7f810b653740 1 Processor -- start 2024-04-24T09:03:33.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.387+0000 7f810b653740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561821395b40 0x5618213a0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:33.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.387+0000 7f810b653740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561821396460 0x56182139d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:33.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.387+0000 7f810b653740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618213a0b30 0x56182139dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:33.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.387+0000 7f810b653740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5618213a39c0 con 0x561821395b40 2024-04-24T09:03:33.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.387+0000 7f810b653740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5618212b4810 con 0x5618213a0b30 2024-04-24T09:03:33.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.387+0000 7f810b653740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5618213a3b40 con 0x561821396460 2024-04-24T09:03:33.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.387+0000 7f810b653740 10 client.0 _ll_register_callbacks cb 0x561821282140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:03:33.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.387+0000 7f8105e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561821395b40 0x5618213a0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:33.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.388+0000 7f8105e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561821395b40 0x5618213a0590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56386/0 (socket says 172.21.15.83:56386) 2024-04-24T09:03:33.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.388+0000 7f8105e00640 1 -- 172.21.15.83:0/1822833511 learned_addr learned my addr 172.21.15.83:0/1822833511 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:03:33.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.388+0000 7f8105400640 1 --2- 172.21.15.83:0/1822833511 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561821396460 0x56182139d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:33.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.388+0000 7f8105e00640 1 -- 172.21.15.83:0/1822833511 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561821396460 msgr2=0x56182139d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:33.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.388+0000 7f8106800640 1 --2- 172.21.15.83:0/1822833511 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618213a0b30 0x56182139dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:33.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.388+0000 7f8105e00640 1 --2- 172.21.15.83:0/1822833511 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561821396460 0x56182139d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.388+0000 7f8105e00640 1 -- 172.21.15.83:0/1822833511 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618213a0b30 msgr2=0x56182139dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:33.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.388+0000 7f8105e00640 1 --2- 172.21.15.83:0/1822833511 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618213a0b30 0x56182139dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.388+0000 7f8105e00640 1 -- 172.21.15.83:0/1822833511 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618212c53f0 con 0x561821395b40 2024-04-24T09:03:33.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.389+0000 7f8106800640 1 --2- 172.21.15.83:0/1822833511 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618213a0b30 0x56182139dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:03:33.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.389+0000 7f8105e00640 1 --2- 172.21.15.83:0/1822833511 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561821395b40 0x5618213a0590 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f80f4005230 tx=0x7f80f4005260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2569b76fe971973 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:33.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.389+0000 7f80e3400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:03:33.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.389+0000 7f80e3400640 1 -- 172.21.15.83:0/1822833511 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80f4028060 con 0x561821395b40 2024-04-24T09:03:33.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.389+0000 7f80e3400640 1 -- 172.21.15.83:0/1822833511 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9583+0+0 (secure 0 0 0) 0x7f80f4009c50 con 0x561821395b40 2024-04-24T09:03:33.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.389+0000 7f810b653740 1 -- 172.21.15.83:0/1822833511 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80f4009e50 con 0x561821395b40 2024-04-24T09:03:33.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.389+0000 7f80e3400640 1 -- 172.21.15.83:0/1822833511 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f80f402f050 con 0x561821395b40 2024-04-24T09:03:33.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.389+0000 7f810b653740 10 client.19872 fetch_fsmap learned FSMap version 41 2024-04-24T09:03:33.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.389+0000 7f810b653740 1 -- 172.21.15.83:0/1822833511 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x56182100d3f0 con 0x561821395b40 2024-04-24T09:03:33.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.390+0000 7f80e3400640 1 -- 172.21.15.83:0/1822833511 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f80f4009e50 con 0x561821395b40 2024-04-24T09:03:33.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.390+0000 7f80e3400640 1 -- 172.21.15.83:0/1822833511 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7f80f400f040 con 0x561821395b40 2024-04-24T09:03:33.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.390+0000 7f810b653740 10 client.19872 fetch_fsmap finished waiting for FSMap version 41 2024-04-24T09:03:33.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.390+0000 7f810b653740 10 client.19872 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-24T09:03:33.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.390+0000 7f810b653740 20 client.19872 populate_metadata read hostname 'smithi083' 2024-04-24T09:03:33.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.390+0000 7f810b653740 1 --2- 172.21.15.83:0/1822833511 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x5618213b0a20 0x5618213b2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:33.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.390+0000 7f810b653740 4 client.19872 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:03:33.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.390+0000 7f810b653740 1 -- 172.21.15.83:0/1822833511 --> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561821308bf0 con 0x5618213b0a20 2024-04-24T09:03:33.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.391+0000 7f8105400640 1 --2- 172.21.15.83:0/1822833511 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x5618213b0a20 0x5618213b2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:33.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.392+0000 7f8105400640 1 --2- 172.21.15.83:0/1822833511 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x5618213b0a20 0x5618213b2e10 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b8aa414900a204b6 server_cookie=ab1e94fc1d16c18f in_seq=0 out_seq=0 2024-04-24T09:03:33.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.392+0000 7f80e3400640 10 client.19872 ms_handle_connect on v2:172.21.15.138:6834/1909219758 2024-04-24T09:03:33.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.395+0000 7f80e3400640 1 -- 172.21.15.83:0/1822833511 <== mds.2 v2:172.21.15.138:6834/1909219758 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561821308bf0 con 0x5618213b0a20 2024-04-24T09:03:33.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.395+0000 7f80e3400640 10 client.19872 handle_command_reply: tid=0 2024-04-24T09:03:33.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.395+0000 7f810b653740 1 client.19872 shutdown 2024-04-24T09:03:33.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.395+0000 7f810b653740 1 -- 172.21.15.83:0/1822833511 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561821395b40 msgr2=0x5618213a0590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:33.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.395+0000 7f810b653740 1 --2- 172.21.15.83:0/1822833511 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561821395b40 0x5618213a0590 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f80f4005230 tx=0x7f80f4005260 comp rx=0 tx=0).stop 2024-04-24T09:03:33.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.395+0000 7f810b653740 1 -- 172.21.15.83:0/1822833511 shutdown_connections 2024-04-24T09:03:33.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.395+0000 7f810b653740 1 --2- 172.21.15.83:0/1822833511 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x5618213b0a20 0x5618213b2e10 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.395+0000 7f810b653740 1 --2- 172.21.15.83:0/1822833511 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618213a0b30 0x56182139dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.395+0000 7f810b653740 1 --2- 172.21.15.83:0/1822833511 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561821396460 0x56182139d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.395+0000 7f810b653740 1 --2- 172.21.15.83:0/1822833511 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561821395b40 0x5618213a0590 unknown :-1 s=CLOSED pgs=1904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.395+0000 7f80e3400640 0 client.19872 ms_handle_reset on v2:172.21.15.138:6834/1909219758 2024-04-24T09:03:33.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.395+0000 7f810b653740 1 -- 172.21.15.83:0/1822833511 >> 172.21.15.83:0/1822833511 conn(0x561821391e60 msgr2=0x561821397b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:33.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.396+0000 7f810b653740 1 -- 172.21.15.83:0/1822833511 shutdown_connections 2024-04-24T09:03:33.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.396+0000 7f810b653740 1 -- 172.21.15.83:0/1822833511 wait complete. 2024-04-24T09:03:33.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.396+0000 7f810b653740 20 client.19872 trim_cache size 0 max 16384 2024-04-24T09:03:33.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.413+0000 7f810b653740 1 Processor -- start 2024-04-24T09:03:33.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.413+0000 7f810b653740 1 -- start start 2024-04-24T09:03:33.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.414+0000 7f810b653740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561821395b40 0x561821398080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:33.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.414+0000 7f810b653740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561821396460 0x56182139c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:33.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.414+0000 7f810b653740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618213a0b30 0x5618213a00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:33.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.414+0000 7f810b653740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f81001b16f0 con 0x561821395b40 2024-04-24T09:03:33.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.414+0000 7f810b653740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f81001b3350 con 0x5618213a0b30 2024-04-24T09:03:33.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.414+0000 7f810b653740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f81001b1dc0 con 0x561821396460 2024-04-24T09:03:33.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.414+0000 7f8105e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561821395b40 0x561821398080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:33.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.414+0000 7f8105e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561821395b40 0x561821398080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56400/0 (socket says 172.21.15.83:56400) 2024-04-24T09:03:33.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.414+0000 7f8105e00640 1 -- 172.21.15.83:0/2405702833 learned_addr learned my addr 172.21.15.83:0/2405702833 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:03:33.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.414+0000 7f8105e00640 1 -- 172.21.15.83:0/2405702833 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561821396460 msgr2=0x56182139c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:33.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.414+0000 7f8105e00640 1 --2- 172.21.15.83:0/2405702833 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561821396460 0x56182139c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.414+0000 7f8105e00640 1 -- 172.21.15.83:0/2405702833 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618213a0b30 msgr2=0x5618213a00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:33.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.414+0000 7f8105e00640 1 --2- 172.21.15.83:0/2405702833 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618213a0b30 0x5618213a00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.414+0000 7f8105e00640 1 -- 172.21.15.83:0/2405702833 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618210d77d0 con 0x561821395b40 2024-04-24T09:03:33.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.415+0000 7f8105e00640 1 --2- 172.21.15.83:0/2405702833 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561821395b40 0x561821398080 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7f80f401a4a0 tx=0x7f80f4005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b5d4a68060d148f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:33.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.415+0000 7f80ee600640 1 -- 172.21.15.83:0/2405702833 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80f4028070 con 0x561821395b40 2024-04-24T09:03:33.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.415+0000 7f80ee600640 1 -- 172.21.15.83:0/2405702833 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f80f401ae50 con 0x561821395b40 2024-04-24T09:03:33.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.415+0000 7f80ee600640 1 -- 172.21.15.83:0/2405702833 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80f4016040 con 0x561821395b40 2024-04-24T09:03:33.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.416+0000 7f810b653740 1 -- 172.21.15.83:0/2405702833 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561821395b40 msgr2=0x561821398080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:33.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.416+0000 7f810b653740 1 --2- 172.21.15.83:0/2405702833 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561821395b40 0x561821398080 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7f80f401a4a0 tx=0x7f80f4005710 comp rx=0 tx=0).stop 2024-04-24T09:03:33.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.416+0000 7f810b653740 1 -- 172.21.15.83:0/2405702833 shutdown_connections 2024-04-24T09:03:33.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.416+0000 7f810b653740 1 --2- 172.21.15.83:0/2405702833 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618213a0b30 0x5618213a00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.416+0000 7f810b653740 1 --2- 172.21.15.83:0/2405702833 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561821396460 0x56182139c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.416+0000 7f810b653740 1 --2- 172.21.15.83:0/2405702833 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561821395b40 0x561821398080 unknown :-1 s=CLOSED pgs=1905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.416+0000 7f810b653740 1 -- 172.21.15.83:0/2405702833 >> 172.21.15.83:0/2405702833 conn(0x561821391e60 msgr2=0x56182139c150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:33.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.416+0000 7f810b653740 1 -- 172.21.15.83:0/2405702833 shutdown_connections 2024-04-24T09:03:33.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.416+0000 7f810b653740 1 -- 172.21.15.83:0/2405702833 wait complete. 2024-04-24T09:03:33.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.416+0000 7f810b653740 1 -- start start 2024-04-24T09:03:33.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.417+0000 7f810b653740 1 Processor -- start 2024-04-24T09:03:33.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.417+0000 7f810b653740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561821395b40 0x5618213a16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:33.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.417+0000 7f810b653740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561821396460 0x5618213a1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:33.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.417+0000 7f810b653740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5618213a0b30 0x5618213a7d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:33.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.417+0000 7f810b653740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f81001b2c20 con 0x5618213a0b30 2024-04-24T09:03:33.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.417+0000 7f810b653740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f81001b24f0 con 0x561821396460 2024-04-24T09:03:33.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.417+0000 7f810b653740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f81001af6f0 con 0x561821395b40 2024-04-24T09:03:33.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.417+0000 7f810b653740 10 client.0 _ll_register_callbacks cb 0x7f80f40099b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:03:33.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.417+0000 7f8106800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5618213a0b30 0x5618213a7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:33.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.417+0000 7f8106800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5618213a0b30 0x5618213a7d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56414/0 (socket says 172.21.15.83:56414) 2024-04-24T09:03:33.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.417+0000 7f8106800640 1 -- 172.21.15.83:0/3700669604 learned_addr learned my addr 172.21.15.83:0/3700669604 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:03:33.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.418+0000 7f8105400640 1 --2- 172.21.15.83:0/3700669604 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561821396460 0x5618213a1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:33.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.418+0000 7f8105e00640 1 --2- 172.21.15.83:0/3700669604 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561821395b40 0x5618213a16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:33.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.418+0000 7f8106800640 1 -- 172.21.15.83:0/3700669604 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561821395b40 msgr2=0x5618213a16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:33.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.418+0000 7f8106800640 1 --2- 172.21.15.83:0/3700669604 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561821395b40 0x5618213a16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.418+0000 7f8106800640 1 -- 172.21.15.83:0/3700669604 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561821396460 msgr2=0x5618213a1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:33.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.418+0000 7f8106800640 1 --2- 172.21.15.83:0/3700669604 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561821396460 0x5618213a1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.418+0000 7f8106800640 1 -- 172.21.15.83:0/3700669604 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618213a8350 con 0x5618213a0b30 2024-04-24T09:03:33.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.418+0000 7f8106800640 1 --2- 172.21.15.83:0/3700669604 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5618213a0b30 0x5618213a7d90 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7f80fc009ca0 tx=0x7f80fc02a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cea45394d248cc86 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:33.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.418+0000 7f80e3400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:03:33.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.418+0000 7f80e3400640 1 -- 172.21.15.83:0/3700669604 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80fc054060 con 0x5618213a0b30 2024-04-24T09:03:33.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.419+0000 7f810b653740 1 -- 172.21.15.83:0/3700669604 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80f401ae50 con 0x5618213a0b30 2024-04-24T09:03:33.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.419+0000 7f80e3400640 1 -- 172.21.15.83:0/3700669604 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9583+0+0 (secure 0 0 0) 0x7f80fc004490 con 0x5618213a0b30 2024-04-24T09:03:33.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.419+0000 7f80e3400640 1 -- 172.21.15.83:0/3700669604 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f80fc004690 con 0x5618213a0b30 2024-04-24T09:03:33.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.419+0000 7f810b653740 10 client.19878 fetch_fsmap learned FSMap version 41 2024-04-24T09:03:33.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.419+0000 7f810b653740 1 -- 172.21.15.83:0/3700669604 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5618213a9610 con 0x5618213a0b30 2024-04-24T09:03:33.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.419+0000 7f80e3400640 1 -- 172.21.15.83:0/3700669604 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f80f401ae50 con 0x5618213a0b30 2024-04-24T09:03:33.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.419+0000 7f80e3400640 1 -- 172.21.15.83:0/3700669604 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7f80fc043040 con 0x5618213a0b30 2024-04-24T09:03:33.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.419+0000 7f810b653740 10 client.19878 fetch_fsmap finished waiting for FSMap version 41 2024-04-24T09:03:33.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.419+0000 7f810b653740 10 client.19878 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-24T09:03:33.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.419+0000 7f810b653740 20 client.19878 populate_metadata read hostname 'smithi083' 2024-04-24T09:03:33.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.419+0000 7f810b653740 1 --2- 172.21.15.83:0/3700669604 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x561821417290 0x5618214196b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:33.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.419+0000 7f810b653740 4 client.19878 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:03:33.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.419+0000 7f810b653740 1 -- 172.21.15.83:0/3700669604 --> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561821302a60 con 0x561821417290 2024-04-24T09:03:33.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.420+0000 7f8105e00640 1 --2- 172.21.15.83:0/3700669604 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x561821417290 0x5618214196b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:33.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.421+0000 7f8105e00640 1 --2- 172.21.15.83:0/3700669604 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x561821417290 0x5618214196b0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7c3ac27f7a853f07 server_cookie=dd54c9865da587e6 in_seq=0 out_seq=0 2024-04-24T09:03:33.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.421+0000 7f80e3400640 10 client.19878 ms_handle_connect on v2:172.21.15.138:6834/1909219758 2024-04-24T09:03:33.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.421+0000 7f80e3400640 1 -- 172.21.15.83:0/3700669604 <== mds.2 v2:172.21.15.138:6834/1909219758 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561821302a60 con 0x561821417290 2024-04-24T09:03:33.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.421+0000 7f80e3400640 10 client.19878 handle_command_reply: tid=0 2024-04-24T09:03:33.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.421+0000 7f810b653740 1 client.19878 shutdown 2024-04-24T09:03:33.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.422+0000 7f810b653740 1 -- 172.21.15.83:0/3700669604 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5618213a0b30 msgr2=0x5618213a7d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:33.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.422+0000 7f810b653740 1 --2- 172.21.15.83:0/3700669604 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5618213a0b30 0x5618213a7d90 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7f80fc009ca0 tx=0x7f80fc02a8c0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.422+0000 7f810b653740 1 -- 172.21.15.83:0/3700669604 shutdown_connections 2024-04-24T09:03:33.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.422+0000 7f810b653740 1 --2- 172.21.15.83:0/3700669604 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x561821417290 0x5618214196b0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.422+0000 7f810b653740 1 --2- 172.21.15.83:0/3700669604 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5618213a0b30 0x5618213a7d90 unknown :-1 s=CLOSED pgs=1906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.422+0000 7f810b653740 1 --2- 172.21.15.83:0/3700669604 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561821396460 0x5618213a1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.422+0000 7f810b653740 1 --2- 172.21.15.83:0/3700669604 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561821395b40 0x5618213a16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.422+0000 7f810b653740 1 -- 172.21.15.83:0/3700669604 >> 172.21.15.83:0/3700669604 conn(0x561821391e60 msgr2=0x56182139c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:33.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.422+0000 7f80e3400640 0 client.19878 ms_handle_reset on v2:172.21.15.138:6834/1909219758 2024-04-24T09:03:33.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.422+0000 7f810b653740 1 -- 172.21.15.83:0/3700669604 shutdown_connections 2024-04-24T09:03:33.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.422+0000 7f810b653740 1 -- 172.21.15.83:0/3700669604 wait complete. 2024-04-24T09:03:33.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.422+0000 7f810b653740 20 client.19878 trim_cache size 0 max 16384 2024-04-24T09:03:33.425 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:03:33.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.423+0000 7f8107200640 1 -- 172.21.15.83:0/2812959147 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f80e80a3d20 msgr2=0x7f80e80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:33.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.423+0000 7f8107200640 1 --2- 172.21.15.83:0/2812959147 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f80e80a3d20 0x7f80e80a61e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f80fc005fa0 tx=0x7f80fc009360 comp rx=0 tx=0).stop 2024-04-24T09:03:33.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.423+0000 7f8107200640 1 -- 172.21.15.83:0/2812959147 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81001071e0 msgr2=0x7f81001a0b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:33.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.423+0000 7f8107200640 1 --2- 172.21.15.83:0/2812959147 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81001071e0 0x7f81001a0b30 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7f80f000d770 tx=0x7f80f000dc30 comp rx=0 tx=0).stop 2024-04-24T09:03:33.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.423+0000 7f8107200640 1 -- 172.21.15.83:0/2812959147 shutdown_connections 2024-04-24T09:03:33.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.423+0000 7f8107200640 1 --2- 172.21.15.83:0/2812959147 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f80e80a3d20 0x7f80e80a61e0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.423+0000 7f8107200640 1 --2- 172.21.15.83:0/2812959147 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f810019a6b0 0x7f810019ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.423+0000 7f8107200640 1 --2- 172.21.15.83:0/2812959147 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81001071e0 0x7f81001a0b30 unknown :-1 s=CLOSED pgs=1902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.423+0000 7f8107200640 1 --2- 172.21.15.83:0/2812959147 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8100102110 0x7f81001a05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.423+0000 7f8107200640 1 -- 172.21.15.83:0/2812959147 >> 172.21.15.83:0/2812959147 conn(0x7f81000fd9d0 msgr2=0x7f81000fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:33.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.423+0000 7f8107200640 1 -- 172.21.15.83:0/2812959147 shutdown_connections 2024-04-24T09:03:33.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.423+0000 7f8107200640 1 -- 172.21.15.83:0/2812959147 wait complete. 2024-04-24T09:03:33.442 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T09:03:33.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:03:33 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/1295966910' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:03:33.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:03:33 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/1295966910' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:03:33.620 INFO:tasks.workunit.client.0.smithi083.stdout: 1 69070 0.60 MB/sec execute 487 sec latency 52.298 ms 2024-04-24T09:03:33.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.640+0000 7f747e000640 1 -- 172.21.15.83:0/827604879 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f747806af00 msgr2=0x7f747810ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:33.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.640+0000 7f747e000640 1 --2- 172.21.15.83:0/827604879 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f747806af00 0x7f747810ed60 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7f7470009cd0 tx=0x7f747002a8f0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.640+0000 7f747e000640 1 -- 172.21.15.83:0/827604879 shutdown_connections 2024-04-24T09:03:33.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.640+0000 7f747e000640 1 --2- 172.21.15.83:0/827604879 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f747806af00 0x7f747810ed60 unknown :-1 s=CLOSED pgs=1907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.640+0000 7f747e000640 1 --2- 172.21.15.83:0/827604879 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f747806a560 0x7f747806a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.640+0000 7f747e000640 1 --2- 172.21.15.83:0/827604879 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7478107ea0 0x7f7478108280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.640+0000 7f747e000640 1 -- 172.21.15.83:0/827604879 >> 172.21.15.83:0/827604879 conn(0x7f7478077c70 msgr2=0x7f7478078080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:33.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.640+0000 7f747e000640 1 -- 172.21.15.83:0/827604879 shutdown_connections 2024-04-24T09:03:33.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.640+0000 7f747e000640 1 -- 172.21.15.83:0/827604879 wait complete. 2024-04-24T09:03:33.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.641+0000 7f747e000640 1 Processor -- start 2024-04-24T09:03:33.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.641+0000 7f747e000640 1 -- start start 2024-04-24T09:03:33.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.642+0000 7f747e000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f747806a560 0x7f7478198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:33.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.642+0000 7f747e000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f747806af00 0x7f7478198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:33.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.642+0000 7f747e000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7478107ea0 0x7f747819cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:33.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.642+0000 7f747e000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7478112170 con 0x7f747806a560 2024-04-24T09:03:33.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.642+0000 7f747e000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7478111ff0 con 0x7f747806af00 2024-04-24T09:03:33.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.642+0000 7f747e000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f74781122f0 con 0x7f7478107ea0 2024-04-24T09:03:33.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.642+0000 7f747cc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f747806a560 0x7f7478198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:33.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.642+0000 7f747cc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f747806a560 0x7f7478198690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56424/0 (socket says 172.21.15.83:56424) 2024-04-24T09:03:33.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.642+0000 7f747cc00640 1 -- 172.21.15.83:0/4107914302 learned_addr learned my addr 172.21.15.83:0/4107914302 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:03:33.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.642+0000 7f7477e00640 1 --2- 172.21.15.83:0/4107914302 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f747806af00 0x7f7478198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:33.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.642+0000 7f747d600640 1 --2- 172.21.15.83:0/4107914302 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7478107ea0 0x7f747819cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:33.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.642+0000 7f747cc00640 1 -- 172.21.15.83:0/4107914302 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7478107ea0 msgr2=0x7f747819cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:33.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.642+0000 7f747cc00640 1 --2- 172.21.15.83:0/4107914302 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7478107ea0 0x7f747819cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.643+0000 7f747cc00640 1 -- 172.21.15.83:0/4107914302 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f747806af00 msgr2=0x7f7478198bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:33.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.643+0000 7f747cc00640 1 --2- 172.21.15.83:0/4107914302 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f747806af00 0x7f7478198bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.643+0000 7f747cc00640 1 -- 172.21.15.83:0/4107914302 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f747819d630 con 0x7f747806a560 2024-04-24T09:03:33.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.643+0000 7f747cc00640 1 --2- 172.21.15.83:0/4107914302 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f747806a560 0x7f7478198690 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7f746c00d770 tx=0x7f746c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:33.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.643+0000 7f7475600640 1 -- 172.21.15.83:0/4107914302 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f746c017070 con 0x7f747806a560 2024-04-24T09:03:33.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.643+0000 7f7475600640 1 -- 172.21.15.83:0/4107914302 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f746c0054d0 con 0x7f747806a560 2024-04-24T09:03:33.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.643+0000 7f747e000640 1 -- 172.21.15.83:0/4107914302 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f747819d920 con 0x7f747806a560 2024-04-24T09:03:33.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.643+0000 7f7475600640 1 -- 172.21.15.83:0/4107914302 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f746c004cf0 con 0x7f747806a560 2024-04-24T09:03:33.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.643+0000 7f747e000640 1 -- 172.21.15.83:0/4107914302 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74781a51b0 con 0x7f747806a560 2024-04-24T09:03:33.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.648+0000 7f74823fb740 1 Processor -- start 2024-04-24T09:03:33.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.648+0000 7f74823fb740 1 -- start start 2024-04-24T09:03:33.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.648+0000 7f7475600640 1 -- 172.21.15.83:0/4107914302 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f746c004e90 con 0x7f747806a560 2024-04-24T09:03:33.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.648+0000 7f74823fb740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555cffaf9b40 0x555cffaf9f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:33.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.648+0000 7f74823fb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555cffafa460 0x555cffb045f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:33.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.648+0000 7f74823fb740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555cffb04b30 0x555cffb06f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:33.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.648+0000 7f74823fb740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x555cffa5f1d0 con 0x555cffaf9b40 2024-04-24T09:03:33.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.648+0000 7f74823fb740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555cffa05450 con 0x555cffafa460 2024-04-24T09:03:33.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.648+0000 7f74823fb740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x555cffa5b190 con 0x555cffb04b30 2024-04-24T09:03:33.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.649+0000 7f747d600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555cffaf9b40 0x555cffaf9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:33.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.649+0000 7f747d600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555cffaf9b40 0x555cffaf9f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56440/0 (socket says 172.21.15.83:56440) 2024-04-24T09:03:33.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.649+0000 7f747d600640 1 -- 172.21.15.83:0/3369721699 learned_addr learned my addr 172.21.15.83:0/3369721699 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:03:33.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.649+0000 7f747cc00640 1 --2- 172.21.15.83:0/3369721699 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555cffafa460 0x555cffb045f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:33.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.649+0000 7f7477e00640 1 --2- 172.21.15.83:0/3369721699 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555cffb04b30 0x555cffb06f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:33.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.649+0000 7f747d600640 1 -- 172.21.15.83:0/3369721699 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555cffb04b30 msgr2=0x555cffb06f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:33.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.649+0000 7f747d600640 1 --2- 172.21.15.83:0/3369721699 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555cffb04b30 0x555cffb06f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.649+0000 7f747d600640 1 -- 172.21.15.83:0/3369721699 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555cffafa460 msgr2=0x555cffb045f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:33.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.649+0000 7f747d600640 1 --2- 172.21.15.83:0/3369721699 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555cffafa460 0x555cffb045f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.649+0000 7f747d600640 1 -- 172.21.15.83:0/3369721699 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555cffac2310 con 0x555cffaf9b40 2024-04-24T09:03:33.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.650+0000 7f747d600640 1 --2- 172.21.15.83:0/3369721699 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555cffaf9b40 0x555cffaf9f20 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7f7470035000 tx=0x7f7470005fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cda170e70d743b30 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:33.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.650+0000 7f7475600640 1 --2- 172.21.15.83:0/4107914302 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f74600a3d20 0x7f74600a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:33.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.650+0000 7f7469c00640 1 -- 172.21.15.83:0/3369721699 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7470030860 con 0x555cffaf9b40 2024-04-24T09:03:33.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.650+0000 7f7469c00640 1 -- 172.21.15.83:0/3369721699 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7470036040 con 0x555cffaf9b40 2024-04-24T09:03:33.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.650+0000 7f7469c00640 1 -- 172.21.15.83:0/3369721699 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7470004440 con 0x555cffaf9b40 2024-04-24T09:03:33.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.650+0000 7f7475600640 1 -- 172.21.15.83:0/4107914302 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9583+0+0 (secure 0 0 0) 0x7f746c0d06e0 con 0x7f747806a560 2024-04-24T09:03:33.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.650+0000 7f747cc00640 1 --2- 172.21.15.83:0/4107914302 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f74600a3d20 0x7f74600a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:33.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.651+0000 7f74823fb740 1 -- 172.21.15.83:0/3369721699 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555cffaf9b40 msgr2=0x555cffaf9f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:33.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.651+0000 7f747cc00640 1 --2- 172.21.15.83:0/4107914302 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f74600a3d20 0x7f74600a61e0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x555cffafb100 tx=0x7f746c0d4040 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:33.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.651+0000 7f74823fb740 1 --2- 172.21.15.83:0/3369721699 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555cffaf9b40 0x555cffaf9f20 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7f7470035000 tx=0x7f7470005fa0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.651+0000 7f74823fb740 1 -- 172.21.15.83:0/3369721699 shutdown_connections 2024-04-24T09:03:33.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.651+0000 7f74823fb740 1 --2- 172.21.15.83:0/3369721699 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555cffb04b30 0x555cffb06f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.651+0000 7f74823fb740 1 --2- 172.21.15.83:0/3369721699 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555cffafa460 0x555cffb045f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.651+0000 7f74823fb740 1 --2- 172.21.15.83:0/3369721699 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555cffaf9b40 0x555cffaf9f20 unknown :-1 s=CLOSED pgs=1909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.651+0000 7f74823fb740 1 -- 172.21.15.83:0/3369721699 >> 172.21.15.83:0/3369721699 conn(0x555cffaf5e60 msgr2=0x555cffaf6240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:33.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.651+0000 7f74823fb740 1 -- 172.21.15.83:0/3369721699 shutdown_connections 2024-04-24T09:03:33.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.651+0000 7f74823fb740 1 -- 172.21.15.83:0/3369721699 wait complete. 2024-04-24T09:03:33.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.651+0000 7f74823fb740 1 -- start start 2024-04-24T09:03:33.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.652+0000 7f74823fb740 1 Processor -- start 2024-04-24T09:03:33.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.652+0000 7f74823fb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555cffaf9b40 0x555cffb045c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:33.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.652+0000 7f74823fb740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555cffafa460 0x555cffb018b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:33.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.652+0000 7f74823fb740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555cffb04b30 0x555cffb01fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:33.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.652+0000 7f74823fb740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x555cffb07960 con 0x555cffb04b30 2024-04-24T09:03:33.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.652+0000 7f74823fb740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555cffa18810 con 0x555cffaf9b40 2024-04-24T09:03:33.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.652+0000 7f74823fb740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x555cffb07ae0 con 0x555cffafa460 2024-04-24T09:03:33.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.652+0000 7f74823fb740 10 client.0 _ll_register_callbacks cb 0x555cff9e6140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:03:33.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.652+0000 7f747d600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555cffb04b30 0x555cffb01fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:33.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.653+0000 7f747d600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555cffb04b30 0x555cffb01fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56442/0 (socket says 172.21.15.83:56442) 2024-04-24T09:03:33.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.653+0000 7f747d600640 1 -- 172.21.15.83:0/3846685510 learned_addr learned my addr 172.21.15.83:0/3846685510 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:03:33.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.653+0000 7f747d600640 1 --2- 172.21.15.83:0/3846685510 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555cffaf9b40 0x555cffb045c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:33.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.653+0000 7f7477e00640 1 --2- 172.21.15.83:0/3846685510 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555cffafa460 0x555cffb018b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:33.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.653+0000 7f747d600640 1 -- 172.21.15.83:0/3846685510 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555cffafa460 msgr2=0x555cffb018b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:33.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.653+0000 7f747d600640 1 --2- 172.21.15.83:0/3846685510 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555cffafa460 0x555cffb018b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.653+0000 7f747d600640 1 -- 172.21.15.83:0/3846685510 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555cffaf9b40 msgr2=0x555cffb045c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:33.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.653+0000 7f747d600640 1 --2- 172.21.15.83:0/3846685510 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555cffaf9b40 0x555cffb045c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.653+0000 7f747d600640 1 -- 172.21.15.83:0/3846685510 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555cffa293f0 con 0x555cffb04b30 2024-04-24T09:03:33.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.653+0000 7f747d600640 1 --2- 172.21.15.83:0/3846685510 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555cffb04b30 0x555cffb01fa0 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7f7470009960 tx=0x7f74700040e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38af060b72b05f44 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:33.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.653+0000 7f745e000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:03:33.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.654+0000 7f745e000640 1 -- 172.21.15.83:0/3846685510 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7470058060 con 0x555cffb04b30 2024-04-24T09:03:33.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.654+0000 7f745e000640 1 -- 172.21.15.83:0/3846685510 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9583+0+0 (secure 0 0 0) 0x7f74700043c0 con 0x555cffb04b30 2024-04-24T09:03:33.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.654+0000 7f74823fb740 1 -- 172.21.15.83:0/3846685510 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7470036040 con 0x555cffb04b30 2024-04-24T09:03:33.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.654+0000 7f745e000640 1 -- 172.21.15.83:0/3846685510 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f747004a040 con 0x555cffb04b30 2024-04-24T09:03:33.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.654+0000 7f74823fb740 10 client.19890 fetch_fsmap learned FSMap version 41 2024-04-24T09:03:33.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.654+0000 7f74823fb740 1 -- 172.21.15.83:0/3846685510 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x555cff7713f0 con 0x555cffb04b30 2024-04-24T09:03:33.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.654+0000 7f745e000640 1 -- 172.21.15.83:0/3846685510 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7470036040 con 0x555cffb04b30 2024-04-24T09:03:33.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.654+0000 7f745e000640 1 -- 172.21.15.83:0/3846685510 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7f7470050070 con 0x555cffb04b30 2024-04-24T09:03:33.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.655+0000 7f74823fb740 10 client.19890 fetch_fsmap finished waiting for FSMap version 41 2024-04-24T09:03:33.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.655+0000 7f74823fb740 10 client.19890 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-24T09:03:33.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.655+0000 7f74823fb740 20 client.19890 populate_metadata read hostname 'smithi083' 2024-04-24T09:03:33.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.655+0000 7f74823fb740 1 --2- 172.21.15.83:0/3846685510 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x555cffb14bd0 0x555cffb16fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:33.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.655+0000 7f74823fb740 4 client.19890 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:03:33.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.655+0000 7f74823fb740 1 -- 172.21.15.83:0/3846685510 --> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555cffa6cbf0 con 0x555cffb14bd0 2024-04-24T09:03:33.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.655+0000 7f7477e00640 1 --2- 172.21.15.83:0/3846685510 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x555cffb14bd0 0x555cffb16fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:33.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.657+0000 7f7477e00640 1 --2- 172.21.15.83:0/3846685510 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x555cffb14bd0 0x555cffb16fc0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1a7e80ee648c30a server_cookie=2ee034f147e795b1 in_seq=0 out_seq=0 2024-04-24T09:03:33.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.657+0000 7f745e000640 10 client.19890 ms_handle_connect on v2:172.21.15.138:6836/900566754 2024-04-24T09:03:33.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.658+0000 7f745e000640 1 -- 172.21.15.83:0/3846685510 <== mds.1 v2:172.21.15.138:6836/900566754 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555cffa6cbf0 con 0x555cffb14bd0 2024-04-24T09:03:33.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.658+0000 7f745e000640 10 client.19890 handle_command_reply: tid=0 2024-04-24T09:03:33.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.659+0000 7f74823fb740 1 client.19890 shutdown 2024-04-24T09:03:33.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.659+0000 7f74823fb740 1 -- 172.21.15.83:0/3846685510 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555cffb04b30 msgr2=0x555cffb01fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:33.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.659+0000 7f74823fb740 1 --2- 172.21.15.83:0/3846685510 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555cffb04b30 0x555cffb01fa0 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7f7470009960 tx=0x7f74700040e0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.659+0000 7f74823fb740 1 -- 172.21.15.83:0/3846685510 shutdown_connections 2024-04-24T09:03:33.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.659+0000 7f74823fb740 1 --2- 172.21.15.83:0/3846685510 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555cffb04b30 0x555cffb01fa0 unknown :-1 s=CLOSED pgs=1910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.659+0000 7f74823fb740 1 --2- 172.21.15.83:0/3846685510 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x555cffb14bd0 0x555cffb16fc0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.659+0000 7f74823fb740 1 --2- 172.21.15.83:0/3846685510 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555cffafa460 0x555cffb018b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.659+0000 7f74823fb740 1 --2- 172.21.15.83:0/3846685510 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555cffaf9b40 0x555cffb045c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.659+0000 7f745e000640 0 client.19890 ms_handle_reset on v2:172.21.15.138:6836/900566754 2024-04-24T09:03:33.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.659+0000 7f74823fb740 1 -- 172.21.15.83:0/3846685510 >> 172.21.15.83:0/3846685510 conn(0x555cffaf5e60 msgr2=0x555cffafbb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:33.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.659+0000 7f74823fb740 1 -- 172.21.15.83:0/3846685510 shutdown_connections 2024-04-24T09:03:33.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.659+0000 7f74823fb740 1 -- 172.21.15.83:0/3846685510 wait complete. 2024-04-24T09:03:33.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.659+0000 7f74823fb740 20 client.19890 trim_cache size 0 max 16384 2024-04-24T09:03:33.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.677+0000 7f74823fb740 1 Processor -- start 2024-04-24T09:03:33.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.678+0000 7f74823fb740 1 -- start start 2024-04-24T09:03:33.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.678+0000 7f74823fb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555cffaf9b40 0x555cffb104a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:33.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.678+0000 7f74823fb740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555cffafa460 0x555cffb040f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:33.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.678+0000 7f74823fb740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555cffb04b30 0x555cffb19fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:33.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.678+0000 7f74823fb740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f74781af470 con 0x555cffb04b30 2024-04-24T09:03:33.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.678+0000 7f74823fb740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f74781b10d0 con 0x555cffaf9b40 2024-04-24T09:03:33.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.678+0000 7f74823fb740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f74781afb40 con 0x555cffafa460 2024-04-24T09:03:33.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.678+0000 7f747d600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555cffb04b30 0x555cffb19fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:33.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.678+0000 7f747d600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555cffb04b30 0x555cffb19fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56444/0 (socket says 172.21.15.83:56444) 2024-04-24T09:03:33.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.678+0000 7f747d600640 1 -- 172.21.15.83:0/1739688515 learned_addr learned my addr 172.21.15.83:0/1739688515 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:03:33.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.679+0000 7f747d600640 1 --2- 172.21.15.83:0/1739688515 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555cffaf9b40 0x555cffb104a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:33.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.679+0000 7f7477e00640 1 --2- 172.21.15.83:0/1739688515 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555cffafa460 0x555cffb040f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:33.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.679+0000 7f747d600640 1 -- 172.21.15.83:0/1739688515 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555cffafa460 msgr2=0x555cffb040f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:33.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.679+0000 7f747d600640 1 --2- 172.21.15.83:0/1739688515 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555cffafa460 0x555cffb040f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.679+0000 7f747d600640 1 -- 172.21.15.83:0/1739688515 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555cffaf9b40 msgr2=0x555cffb104a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:33.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.679+0000 7f747d600640 1 --2- 172.21.15.83:0/1739688515 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555cffaf9b40 0x555cffb104a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.679+0000 7f747d600640 1 -- 172.21.15.83:0/1739688515 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555cff83b7d0 con 0x555cffb04b30 2024-04-24T09:03:33.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.679+0000 7f747d600640 1 --2- 172.21.15.83:0/1739688515 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555cffb04b30 0x555cffb19fe0 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f7470004730 tx=0x7f7470009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6787f68200737a05 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:33.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.679+0000 7f7469200640 1 -- 172.21.15.83:0/1739688515 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7470058040 con 0x555cffb04b30 2024-04-24T09:03:33.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.679+0000 7f7469200640 1 -- 172.21.15.83:0/1739688515 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7470036070 con 0x555cffb04b30 2024-04-24T09:03:33.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.679+0000 7f7469200640 1 -- 172.21.15.83:0/1739688515 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7470004b40 con 0x555cffb04b30 2024-04-24T09:03:33.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.680+0000 7f74823fb740 1 -- 172.21.15.83:0/1739688515 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555cffb04b30 msgr2=0x555cffb19fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:33.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.680+0000 7f74823fb740 1 --2- 172.21.15.83:0/1739688515 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555cffb04b30 0x555cffb19fe0 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f7470004730 tx=0x7f7470009a20 comp rx=0 tx=0).stop 2024-04-24T09:03:33.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.680+0000 7f74823fb740 1 -- 172.21.15.83:0/1739688515 shutdown_connections 2024-04-24T09:03:33.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.680+0000 7f74823fb740 1 --2- 172.21.15.83:0/1739688515 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555cffb04b30 0x555cffb19fe0 unknown :-1 s=CLOSED pgs=1911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.680+0000 7f74823fb740 1 --2- 172.21.15.83:0/1739688515 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555cffafa460 0x555cffb040f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.680+0000 7f74823fb740 1 --2- 172.21.15.83:0/1739688515 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555cffaf9b40 0x555cffb104a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.680+0000 7f74823fb740 1 -- 172.21.15.83:0/1739688515 >> 172.21.15.83:0/1739688515 conn(0x555cffaf5e60 msgr2=0x555cffb0fd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:33.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.680+0000 7f74823fb740 1 -- 172.21.15.83:0/1739688515 shutdown_connections 2024-04-24T09:03:33.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.680+0000 7f74823fb740 1 -- 172.21.15.83:0/1739688515 wait complete. 2024-04-24T09:03:33.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.681+0000 7f74823fb740 1 -- start start 2024-04-24T09:03:33.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.681+0000 7f74823fb740 1 Processor -- start 2024-04-24T09:03:33.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.681+0000 7f74823fb740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555cffaf9b40 0x555cffb16100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:33.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.681+0000 7f74823fb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555cffafa460 0x555cffb16640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:33.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.681+0000 7f74823fb740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555cffb04b30 0x555cffb79da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:33.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.681+0000 7f74823fb740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f74781b09a0 con 0x555cffaf9b40 2024-04-24T09:03:33.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.681+0000 7f74823fb740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f74781b0270 con 0x555cffafa460 2024-04-24T09:03:33.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.681+0000 7f74823fb740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f74781ad470 con 0x555cffb04b30 2024-04-24T09:03:33.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.681+0000 7f74823fb740 10 client.0 _ll_register_callbacks cb 0x7f7470005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:03:33.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.682+0000 7f747d600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555cffaf9b40 0x555cffb16100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:33.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.682+0000 7f747d600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555cffaf9b40 0x555cffb16100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56450/0 (socket says 172.21.15.83:56450) 2024-04-24T09:03:33.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.682+0000 7f747d600640 1 -- 172.21.15.83:0/2351765169 learned_addr learned my addr 172.21.15.83:0/2351765169 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:03:33.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.682+0000 7f7477e00640 1 --2- 172.21.15.83:0/2351765169 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555cffafa460 0x555cffb16640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:33.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.682+0000 7f747d600640 1 -- 172.21.15.83:0/2351765169 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555cffb04b30 msgr2=0x555cffb79da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:33.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.682+0000 7f747d600640 1 --2- 172.21.15.83:0/2351765169 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555cffb04b30 0x555cffb79da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.682+0000 7f747d600640 1 -- 172.21.15.83:0/2351765169 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555cffafa460 msgr2=0x555cffb16640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:33.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.682+0000 7f747d600640 1 --2- 172.21.15.83:0/2351765169 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555cffafa460 0x555cffb16640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.682+0000 7f747d600640 1 -- 172.21.15.83:0/2351765169 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555cffb7a2e0 con 0x555cffaf9b40 2024-04-24T09:03:33.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.682+0000 7f7477e00640 1 --2- 172.21.15.83:0/2351765169 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555cffafa460 0x555cffb16640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:03:33.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.682+0000 7f747d600640 1 --2- 172.21.15.83:0/2351765169 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555cffaf9b40 0x555cffb16100 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7f7470030bd0 tx=0x7f7470030410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85ebb5c1d426e358 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:33.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.682+0000 7f745e000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:03:33.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.683+0000 7f745e000640 1 -- 172.21.15.83:0/2351765169 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7470058050 con 0x555cffaf9b40 2024-04-24T09:03:33.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.683+0000 7f745e000640 1 -- 172.21.15.83:0/2351765169 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9583+0+0 (secure 0 0 0) 0x7f7470004b40 con 0x555cffaf9b40 2024-04-24T09:03:33.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.683+0000 7f74823fb740 1 -- 172.21.15.83:0/2351765169 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7470036070 con 0x555cffaf9b40 2024-04-24T09:03:33.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.683+0000 7f745e000640 1 -- 172.21.15.83:0/2351765169 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f747006e050 con 0x555cffaf9b40 2024-04-24T09:03:33.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.683+0000 7f74823fb740 10 client.19896 fetch_fsmap learned FSMap version 41 2024-04-24T09:03:33.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.683+0000 7f74823fb740 1 -- 172.21.15.83:0/2351765169 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x555cffb7b590 con 0x555cffaf9b40 2024-04-24T09:03:33.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.683+0000 7f745e000640 1 -- 172.21.15.83:0/2351765169 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7470036070 con 0x555cffaf9b40 2024-04-24T09:03:33.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.684+0000 7f745e000640 1 -- 172.21.15.83:0/2351765169 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7f7470050030 con 0x555cffaf9b40 2024-04-24T09:03:33.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.684+0000 7f74823fb740 10 client.19896 fetch_fsmap finished waiting for FSMap version 41 2024-04-24T09:03:33.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.684+0000 7f74823fb740 10 client.19896 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-24T09:03:33.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.684+0000 7f74823fb740 20 client.19896 populate_metadata read hostname 'smithi083' 2024-04-24T09:03:33.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.684+0000 7f74823fb740 1 --2- 172.21.15.83:0/2351765169 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x555cffb7bbf0 0x555cffb7e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:33.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.684+0000 7f74823fb740 4 client.19896 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:03:33.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.684+0000 7f74823fb740 1 -- 172.21.15.83:0/2351765169 --> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555cffa66a60 con 0x555cffb7bbf0 2024-04-24T09:03:33.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.684+0000 7f7477e00640 1 --2- 172.21.15.83:0/2351765169 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x555cffb7bbf0 0x555cffb7e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:33.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.685+0000 7f7477e00640 1 --2- 172.21.15.83:0/2351765169 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x555cffb7bbf0 0x555cffb7e010 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3b37170aceb80a38 server_cookie=8c64bb18a0ddfa22 in_seq=0 out_seq=0 2024-04-24T09:03:33.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.685+0000 7f745e000640 10 client.19896 ms_handle_connect on v2:172.21.15.138:6836/900566754 2024-04-24T09:03:33.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.686+0000 7f745e000640 1 -- 172.21.15.83:0/2351765169 <== mds.1 v2:172.21.15.138:6836/900566754 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555cffa66a60 con 0x555cffb7bbf0 2024-04-24T09:03:33.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.686+0000 7f745e000640 10 client.19896 handle_command_reply: tid=0 2024-04-24T09:03:33.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.686+0000 7f74823fb740 1 client.19896 shutdown 2024-04-24T09:03:33.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.686+0000 7f74823fb740 1 -- 172.21.15.83:0/2351765169 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555cffaf9b40 msgr2=0x555cffb16100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:33.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.686+0000 7f74823fb740 1 --2- 172.21.15.83:0/2351765169 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555cffaf9b40 0x555cffb16100 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7f7470030bd0 tx=0x7f7470030410 comp rx=0 tx=0).stop 2024-04-24T09:03:33.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.686+0000 7f74823fb740 1 -- 172.21.15.83:0/2351765169 shutdown_connections 2024-04-24T09:03:33.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.686+0000 7f74823fb740 1 --2- 172.21.15.83:0/2351765169 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x555cffb7bbf0 0x555cffb7e010 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.686+0000 7f74823fb740 1 --2- 172.21.15.83:0/2351765169 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555cffb04b30 0x555cffb79da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.686+0000 7f74823fb740 1 --2- 172.21.15.83:0/2351765169 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555cffafa460 0x555cffb16640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.686+0000 7f74823fb740 1 --2- 172.21.15.83:0/2351765169 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555cffaf9b40 0x555cffb16100 unknown :-1 s=CLOSED pgs=1912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.686+0000 7f745e000640 0 client.19896 ms_handle_reset on v2:172.21.15.138:6836/900566754 2024-04-24T09:03:33.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.686+0000 7f74823fb740 1 -- 172.21.15.83:0/2351765169 >> 172.21.15.83:0/2351765169 conn(0x555cffaf5e60 msgr2=0x555cffb10620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:33.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.686+0000 7f74823fb740 1 -- 172.21.15.83:0/2351765169 shutdown_connections 2024-04-24T09:03:33.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.686+0000 7f74823fb740 1 -- 172.21.15.83:0/2351765169 wait complete. 2024-04-24T09:03:33.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.686+0000 7f74823fb740 20 client.19896 trim_cache size 0 max 16384 2024-04-24T09:03:33.690 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:03:33.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.687+0000 7f747e000640 1 -- 172.21.15.83:0/4107914302 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f74600a3d20 msgr2=0x7f74600a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:33.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.687+0000 7f747e000640 1 --2- 172.21.15.83:0/4107914302 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f74600a3d20 0x7f74600a61e0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x555cffafb100 tx=0x7f746c0d4040 comp rx=0 tx=0).stop 2024-04-24T09:03:33.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.687+0000 7f747e000640 1 -- 172.21.15.83:0/4107914302 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f747806a560 msgr2=0x7f7478198690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:33.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.687+0000 7f747e000640 1 --2- 172.21.15.83:0/4107914302 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f747806a560 0x7f7478198690 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7f746c00d770 tx=0x7f746c00dc30 comp rx=0 tx=0).stop 2024-04-24T09:03:33.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.687+0000 7f747e000640 1 -- 172.21.15.83:0/4107914302 shutdown_connections 2024-04-24T09:03:33.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.687+0000 7f747e000640 1 --2- 172.21.15.83:0/4107914302 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f74600a3d20 0x7f74600a61e0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.687+0000 7f747e000640 1 --2- 172.21.15.83:0/4107914302 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7478107ea0 0x7f747819cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.687+0000 7f747e000640 1 --2- 172.21.15.83:0/4107914302 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f747806af00 0x7f7478198bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.687+0000 7f747e000640 1 --2- 172.21.15.83:0/4107914302 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f747806a560 0x7f7478198690 unknown :-1 s=CLOSED pgs=1908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.687+0000 7f747e000640 1 -- 172.21.15.83:0/4107914302 >> 172.21.15.83:0/4107914302 conn(0x7f7478077c70 msgr2=0x7f74780fff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:33.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.687+0000 7f747e000640 1 -- 172.21.15.83:0/4107914302 shutdown_connections 2024-04-24T09:03:33.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.687+0000 7f747e000640 1 -- 172.21.15.83:0/4107914302 wait complete. 2024-04-24T09:03:33.705 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T09:03:33.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.913+0000 7fe853200640 1 -- 172.21.15.83:0/813084481 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe84c103060 msgr2=0x7fe84c103440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:33.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.913+0000 7fe853200640 1 --2- 172.21.15.83:0/813084481 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe84c103060 0x7fe84c103440 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7fe840009900 tx=0x7fe84002a480 comp rx=0 tx=0).stop 2024-04-24T09:03:33.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.913+0000 7fe853200640 1 -- 172.21.15.83:0/813084481 shutdown_connections 2024-04-24T09:03:33.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.913+0000 7fe853200640 1 --2- 172.21.15.83:0/813084481 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe84c10c480 0x7fe84c10e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.913+0000 7fe853200640 1 --2- 172.21.15.83:0/813084481 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe84c103980 0x7fe84c10be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.913+0000 7fe853200640 1 --2- 172.21.15.83:0/813084481 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe84c103060 0x7fe84c103440 unknown :-1 s=CLOSED pgs=1913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.913+0000 7fe853200640 1 -- 172.21.15.83:0/813084481 >> 172.21.15.83:0/813084481 conn(0x7fe84c0fc880 msgr2=0x7fe84c0feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:33.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.913+0000 7fe853200640 1 -- 172.21.15.83:0/813084481 shutdown_connections 2024-04-24T09:03:33.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.913+0000 7fe853200640 1 -- 172.21.15.83:0/813084481 wait complete. 2024-04-24T09:03:33.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.914+0000 7fe853200640 1 Processor -- start 2024-04-24T09:03:33.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.914+0000 7fe853200640 1 -- start start 2024-04-24T09:03:33.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.914+0000 7fe853200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe84c103060 0x7fe84c100cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:33.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.914+0000 7fe853200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe84c103980 0x7fe84c1011f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:33.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.914+0000 7fe853200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe84c10c480 0x7fe84c101730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:33.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.914+0000 7fe853200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe84c112110 con 0x7fe84c103060 2024-04-24T09:03:33.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.914+0000 7fe853200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe84c111f90 con 0x7fe84c103980 2024-04-24T09:03:33.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.914+0000 7fe853200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe84c112290 con 0x7fe84c10c480 2024-04-24T09:03:33.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.915+0000 7fe851e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe84c103060 0x7fe84c100cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:33.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.915+0000 7fe851e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe84c103060 0x7fe84c100cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56470/0 (socket says 172.21.15.83:56470) 2024-04-24T09:03:33.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.915+0000 7fe851e00640 1 -- 172.21.15.83:0/987326575 learned_addr learned my addr 172.21.15.83:0/987326575 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:03:33.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.915+0000 7fe852800640 1 --2- 172.21.15.83:0/987326575 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe84c10c480 0x7fe84c101730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:33.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.915+0000 7fe851400640 1 --2- 172.21.15.83:0/987326575 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe84c103980 0x7fe84c1011f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:33.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.915+0000 7fe851e00640 1 -- 172.21.15.83:0/987326575 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe84c10c480 msgr2=0x7fe84c101730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:33.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.915+0000 7fe851e00640 1 --2- 172.21.15.83:0/987326575 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe84c10c480 0x7fe84c101730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.915+0000 7fe851e00640 1 -- 172.21.15.83:0/987326575 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe84c103980 msgr2=0x7fe84c1011f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:33.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.915+0000 7fe851e00640 1 --2- 172.21.15.83:0/987326575 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe84c103980 0x7fe84c1011f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.915+0000 7fe851e00640 1 -- 172.21.15.83:0/987326575 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe84c1a6b70 con 0x7fe84c103060 2024-04-24T09:03:33.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.915+0000 7fe851400640 1 --2- 172.21.15.83:0/987326575 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe84c103980 0x7fe84c1011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:03:33.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.916+0000 7fe851e00640 1 --2- 172.21.15.83:0/987326575 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe84c103060 0x7fe84c100cb0 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7fe840005940 tx=0x7fe840004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:33.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.916+0000 7fe846a00640 1 -- 172.21.15.83:0/987326575 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe84003f030 con 0x7fe84c103060 2024-04-24T09:03:33.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.916+0000 7fe846a00640 1 -- 172.21.15.83:0/987326575 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe840003aa0 con 0x7fe84c103060 2024-04-24T09:03:33.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.916+0000 7fe853200640 1 -- 172.21.15.83:0/987326575 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe84c1a6e00 con 0x7fe84c103060 2024-04-24T09:03:33.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.916+0000 7fe846a00640 1 -- 172.21.15.83:0/987326575 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe840035070 con 0x7fe84c103060 2024-04-24T09:03:33.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.916+0000 7fe853200640 1 -- 172.21.15.83:0/987326575 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe84c1a7270 con 0x7fe84c103060 2024-04-24T09:03:33.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.919+0000 7fe846a00640 1 -- 172.21.15.83:0/987326575 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe840042030 con 0x7fe84c103060 2024-04-24T09:03:33.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.920+0000 7fe8575c9740 1 Processor -- start 2024-04-24T09:03:33.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.920+0000 7fe8575c9740 1 -- start start 2024-04-24T09:03:33.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.920+0000 7fe8575c9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e6d7deb40 0x560e6d7def20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:33.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.920+0000 7fe8575c9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e6d7df460 0x560e6d7e95f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:33.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.920+0000 7fe8575c9740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e6d7e9b30 0x560e6d7ebf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:33.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.920+0000 7fe8575c9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x560e6d7441d0 con 0x560e6d7df460 2024-04-24T09:03:33.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.920+0000 7fe8575c9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560e6d6ea450 con 0x560e6d7deb40 2024-04-24T09:03:33.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.920+0000 7fe8575c9740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x560e6d740190 con 0x560e6d7e9b30 2024-04-24T09:03:33.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.921+0000 7fe851e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e6d7df460 0x560e6d7e95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:33.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.921+0000 7fe851e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e6d7df460 0x560e6d7e95f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56478/0 (socket says 172.21.15.83:56478) 2024-04-24T09:03:33.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.921+0000 7fe851e00640 1 -- 172.21.15.83:0/1456109693 learned_addr learned my addr 172.21.15.83:0/1456109693 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:03:33.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.921+0000 7fe852800640 1 --2- 172.21.15.83:0/1456109693 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e6d7deb40 0x560e6d7def20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:33.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.921+0000 7fe851400640 1 --2- 172.21.15.83:0/1456109693 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e6d7e9b30 0x560e6d7ebf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:33.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.921+0000 7fe851e00640 1 -- 172.21.15.83:0/1456109693 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e6d7e9b30 msgr2=0x560e6d7ebf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:33.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.921+0000 7fe851e00640 1 --2- 172.21.15.83:0/1456109693 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e6d7e9b30 0x560e6d7ebf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.921+0000 7fe851e00640 1 -- 172.21.15.83:0/1456109693 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e6d7deb40 msgr2=0x560e6d7def20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:33.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.921+0000 7fe851e00640 1 --2- 172.21.15.83:0/1456109693 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e6d7deb40 0x560e6d7def20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.921+0000 7fe851e00640 1 -- 172.21.15.83:0/1456109693 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e6d7a7310 con 0x560e6d7df460 2024-04-24T09:03:33.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.921+0000 7fe852800640 1 --2- 172.21.15.83:0/1456109693 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e6d7deb40 0x560e6d7def20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:03:33.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.921+0000 7fe846a00640 1 --2- 172.21.15.83:0/987326575 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe8340a3dd0 0x7fe8340a6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:33.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.922+0000 7fe846a00640 1 -- 172.21.15.83:0/987326575 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9583+0+0 (secure 0 0 0) 0x7fe8400f1040 con 0x7fe84c103060 2024-04-24T09:03:33.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.922+0000 7fe852800640 1 --2- 172.21.15.83:0/987326575 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe8340a3dd0 0x7fe8340a6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:33.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.922+0000 7fe851e00640 1 --2- 172.21.15.83:0/1456109693 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e6d7df460 0x560e6d7e95f0 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7fe8400ee2e0 tx=0x7fe8400f9540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6925816e161467e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:33.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.922+0000 7fe83b000640 1 -- 172.21.15.83:0/1456109693 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe840049070 con 0x560e6d7df460 2024-04-24T09:03:33.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.922+0000 7fe83b000640 1 -- 172.21.15.83:0/1456109693 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe84003b460 con 0x560e6d7df460 2024-04-24T09:03:33.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.922+0000 7fe83b000640 1 -- 172.21.15.83:0/1456109693 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8400bc700 con 0x560e6d7df460 2024-04-24T09:03:33.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.923+0000 7fe8575c9740 1 -- 172.21.15.83:0/1456109693 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e6d7df460 msgr2=0x560e6d7e95f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:33.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.923+0000 7fe8575c9740 1 --2- 172.21.15.83:0/1456109693 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e6d7df460 0x560e6d7e95f0 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7fe8400ee2e0 tx=0x7fe8400f9540 comp rx=0 tx=0).stop 2024-04-24T09:03:33.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.923+0000 7fe852800640 1 --2- 172.21.15.83:0/987326575 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe8340a3dd0 0x7fe8340a6290 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fe848009d90 tx=0x7fe8480092c0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:33.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.923+0000 7fe8575c9740 1 -- 172.21.15.83:0/1456109693 shutdown_connections 2024-04-24T09:03:33.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.923+0000 7fe8575c9740 1 --2- 172.21.15.83:0/1456109693 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e6d7e9b30 0x560e6d7ebf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.923+0000 7fe8575c9740 1 --2- 172.21.15.83:0/1456109693 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e6d7df460 0x560e6d7e95f0 unknown :-1 s=CLOSED pgs=1915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.923+0000 7fe8575c9740 1 --2- 172.21.15.83:0/1456109693 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e6d7deb40 0x560e6d7def20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.923+0000 7fe8575c9740 1 -- 172.21.15.83:0/1456109693 >> 172.21.15.83:0/1456109693 conn(0x560e6d7dae60 msgr2=0x560e6d7db240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:33.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.923+0000 7fe8575c9740 1 -- 172.21.15.83:0/1456109693 shutdown_connections 2024-04-24T09:03:33.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.923+0000 7fe8575c9740 1 -- 172.21.15.83:0/1456109693 wait complete. 2024-04-24T09:03:33.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.924+0000 7fe8575c9740 1 -- start start 2024-04-24T09:03:33.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.924+0000 7fe8575c9740 1 Processor -- start 2024-04-24T09:03:33.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.924+0000 7fe8575c9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e6d7deb40 0x560e6d7e95c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:33.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.924+0000 7fe8575c9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e6d7df460 0x560e6d7e68b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:33.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.924+0000 7fe8575c9740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e6d7e9b30 0x560e6d7e6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:33.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.924+0000 7fe8575c9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x560e6d7ec6e0 con 0x560e6d7df460 2024-04-24T09:03:33.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.924+0000 7fe8575c9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560e6d6fd810 con 0x560e6d7deb40 2024-04-24T09:03:33.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.924+0000 7fe8575c9740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x560e6d7ec860 con 0x560e6d7e9b30 2024-04-24T09:03:33.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.924+0000 7fe8575c9740 10 client.0 _ll_register_callbacks cb 0x560e6d6cb140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:03:33.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.925+0000 7fe851400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e6d7df460 0x560e6d7e68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:33.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.925+0000 7fe851400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e6d7df460 0x560e6d7e68b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56490/0 (socket says 172.21.15.83:56490) 2024-04-24T09:03:33.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.925+0000 7fe851400640 1 -- 172.21.15.83:0/374784643 learned_addr learned my addr 172.21.15.83:0/374784643 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:03:33.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.925+0000 7fe851e00640 1 --2- 172.21.15.83:0/374784643 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e6d7deb40 0x560e6d7e95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:33.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.925+0000 7fe852800640 1 --2- 172.21.15.83:0/374784643 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e6d7e9b30 0x560e6d7e6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:33.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.925+0000 7fe851400640 1 -- 172.21.15.83:0/374784643 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e6d7e9b30 msgr2=0x560e6d7e6fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:33.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.925+0000 7fe851400640 1 --2- 172.21.15.83:0/374784643 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e6d7e9b30 0x560e6d7e6fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.925+0000 7fe851400640 1 -- 172.21.15.83:0/374784643 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e6d7deb40 msgr2=0x560e6d7e95c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:33.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.925+0000 7fe851400640 1 --2- 172.21.15.83:0/374784643 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e6d7deb40 0x560e6d7e95c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.925+0000 7fe851400640 1 -- 172.21.15.83:0/374784643 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e6d70e3f0 con 0x560e6d7df460 2024-04-24T09:03:33.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.925+0000 7fe851e00640 1 --2- 172.21.15.83:0/374784643 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e6d7deb40 0x560e6d7e95c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:03:33.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.925+0000 7fe851400640 1 --2- 172.21.15.83:0/374784643 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e6d7df460 0x560e6d7e68b0 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7fe83c009e10 tx=0x7fe83c00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffaff7f1f4c25a5c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:33.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.925+0000 7fe82f400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:03:33.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.926+0000 7fe82f400640 1 -- 172.21.15.83:0/374784643 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe83c011850 con 0x560e6d7df460 2024-04-24T09:03:33.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.926+0000 7fe82f400640 1 -- 172.21.15.83:0/374784643 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9583+0+0 (secure 0 0 0) 0x7fe83c0119f0 con 0x560e6d7df460 2024-04-24T09:03:33.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.926+0000 7fe8575c9740 1 -- 172.21.15.83:0/374784643 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560e6d751bf0 con 0x560e6d7df460 2024-04-24T09:03:33.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.926+0000 7fe82f400640 1 -- 172.21.15.83:0/374784643 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fe83c023050 con 0x560e6d7df460 2024-04-24T09:03:33.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.926+0000 7fe8575c9740 10 client.19908 fetch_fsmap learned FSMap version 41 2024-04-24T09:03:33.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.926+0000 7fe8575c9740 1 -- 172.21.15.83:0/374784643 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x560e6d4563f0 con 0x560e6d7df460 2024-04-24T09:03:33.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.926+0000 7fe82f400640 1 -- 172.21.15.83:0/374784643 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x560e6d751bf0 con 0x560e6d7df460 2024-04-24T09:03:33.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.927+0000 7fe82f400640 1 -- 172.21.15.83:0/374784643 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7fe83c0103d0 con 0x560e6d7df460 2024-04-24T09:03:33.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.927+0000 7fe8575c9740 10 client.19908 fetch_fsmap finished waiting for FSMap version 41 2024-04-24T09:03:33.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.927+0000 7fe8575c9740 10 client.19908 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:03:33.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.927+0000 7fe8575c9740 20 client.19908 populate_metadata read hostname 'smithi083' 2024-04-24T09:03:33.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.927+0000 7fe8575c9740 1 --2- 172.21.15.83:0/374784643 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x560e6d7f9a60 0x560e6d7fbeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:33.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.927+0000 7fe8575c9740 4 client.19908 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:03:33.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.927+0000 7fe8575c9740 1 -- 172.21.15.83:0/374784643 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560e6d74ba60 con 0x560e6d7f9a60 2024-04-24T09:03:33.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.928+0000 7fe851e00640 1 --2- 172.21.15.83:0/374784643 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x560e6d7f9a60 0x560e6d7fbeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:33.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.929+0000 7fe851e00640 1 --2- 172.21.15.83:0/374784643 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x560e6d7f9a60 0x560e6d7fbeb0 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1aa0d10c7d763014 server_cookie=c50cee4693ae4e7e in_seq=0 out_seq=0 2024-04-24T09:03:33.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.929+0000 7fe82f400640 10 client.19908 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:03:33.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.931+0000 7fe82f400640 1 -- 172.21.15.83:0/374784643 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560e6d74ba60 con 0x560e6d7f9a60 2024-04-24T09:03:33.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.931+0000 7fe82f400640 10 client.19908 handle_command_reply: tid=0 2024-04-24T09:03:33.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.931+0000 7fe8575c9740 1 client.19908 shutdown 2024-04-24T09:03:33.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.931+0000 7fe8575c9740 1 -- 172.21.15.83:0/374784643 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e6d7df460 msgr2=0x560e6d7e68b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:33.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.931+0000 7fe8575c9740 1 --2- 172.21.15.83:0/374784643 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e6d7df460 0x560e6d7e68b0 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7fe83c009e10 tx=0x7fe83c00f730 comp rx=0 tx=0).stop 2024-04-24T09:03:33.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.931+0000 7fe8575c9740 1 -- 172.21.15.83:0/374784643 shutdown_connections 2024-04-24T09:03:33.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.931+0000 7fe8575c9740 1 --2- 172.21.15.83:0/374784643 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x560e6d7f9a60 0x560e6d7fbeb0 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.931+0000 7fe8575c9740 1 --2- 172.21.15.83:0/374784643 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e6d7e9b30 0x560e6d7e6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.931+0000 7fe8575c9740 1 --2- 172.21.15.83:0/374784643 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e6d7df460 0x560e6d7e68b0 unknown :-1 s=CLOSED pgs=1916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.931+0000 7fe82f400640 0 client.19908 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:03:33.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.931+0000 7fe8575c9740 1 --2- 172.21.15.83:0/374784643 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e6d7deb40 0x560e6d7e95c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.931+0000 7fe8575c9740 1 -- 172.21.15.83:0/374784643 >> 172.21.15.83:0/374784643 conn(0x560e6d7dae60 msgr2=0x560e6d7e0b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:33.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.931+0000 7fe8575c9740 1 -- 172.21.15.83:0/374784643 shutdown_connections 2024-04-24T09:03:33.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.931+0000 7fe8575c9740 1 -- 172.21.15.83:0/374784643 wait complete. 2024-04-24T09:03:33.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.931+0000 7fe8575c9740 20 client.19908 trim_cache size 0 max 16384 2024-04-24T09:03:33.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.947+0000 7fe8575c9740 1 Processor -- start 2024-04-24T09:03:33.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.947+0000 7fe8575c9740 1 -- start start 2024-04-24T09:03:33.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.947+0000 7fe8575c9740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e6d7deb40 0x560e6d7f55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:33.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.947+0000 7fe8575c9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e6d7df460 0x560e6d7ebe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:33.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.947+0000 7fe8575c9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e6d7e9b30 0x560e6d7e90f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:33.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.947+0000 7fe8575c9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe84c1b1600 con 0x560e6d7df460 2024-04-24T09:03:33.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.947+0000 7fe8575c9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe84c1b3260 con 0x560e6d7e9b30 2024-04-24T09:03:33.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.947+0000 7fe8575c9740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe84c1b1cd0 con 0x560e6d7deb40 2024-04-24T09:03:33.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.947+0000 7fe851400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e6d7df460 0x560e6d7ebe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:33.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.947+0000 7fe851400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e6d7df460 0x560e6d7ebe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56498/0 (socket says 172.21.15.83:56498) 2024-04-24T09:03:33.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.947+0000 7fe851400640 1 -- 172.21.15.83:0/2993879053 learned_addr learned my addr 172.21.15.83:0/2993879053 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:03:33.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.948+0000 7fe851e00640 1 --2- 172.21.15.83:0/2993879053 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e6d7deb40 0x560e6d7f55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:33.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.948+0000 7fe852800640 1 --2- 172.21.15.83:0/2993879053 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e6d7e9b30 0x560e6d7e90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:33.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.948+0000 7fe851400640 1 -- 172.21.15.83:0/2993879053 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e6d7deb40 msgr2=0x560e6d7f55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:33.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.948+0000 7fe851400640 1 --2- 172.21.15.83:0/2993879053 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e6d7deb40 0x560e6d7f55d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.948+0000 7fe851400640 1 -- 172.21.15.83:0/2993879053 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e6d7e9b30 msgr2=0x560e6d7e90f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:33.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.948+0000 7fe851400640 1 --2- 172.21.15.83:0/2993879053 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e6d7e9b30 0x560e6d7e90f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.948+0000 7fe851400640 1 -- 172.21.15.83:0/2993879053 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e6d5207d0 con 0x560e6d7df460 2024-04-24T09:03:33.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.948+0000 7fe851400640 1 --2- 172.21.15.83:0/2993879053 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e6d7df460 0x560e6d7ebe30 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7fe83c006010 tx=0x7fe83c009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=348fa73cb41c4e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:33.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.948+0000 7fe83a600640 1 -- 172.21.15.83:0/2993879053 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe83c00fc30 con 0x560e6d7df460 2024-04-24T09:03:33.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.948+0000 7fe83a600640 1 -- 172.21.15.83:0/2993879053 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe83c02b070 con 0x560e6d7df460 2024-04-24T09:03:33.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.948+0000 7fe83a600640 1 -- 172.21.15.83:0/2993879053 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe83c0055e0 con 0x560e6d7df460 2024-04-24T09:03:33.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.949+0000 7fe8575c9740 1 -- 172.21.15.83:0/2993879053 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e6d7df460 msgr2=0x560e6d7ebe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:33.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.949+0000 7fe8575c9740 1 --2- 172.21.15.83:0/2993879053 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e6d7df460 0x560e6d7ebe30 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7fe83c006010 tx=0x7fe83c009620 comp rx=0 tx=0).stop 2024-04-24T09:03:33.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.949+0000 7fe8575c9740 1 -- 172.21.15.83:0/2993879053 shutdown_connections 2024-04-24T09:03:33.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.949+0000 7fe8575c9740 1 --2- 172.21.15.83:0/2993879053 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e6d7e9b30 0x560e6d7e90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.949+0000 7fe8575c9740 1 --2- 172.21.15.83:0/2993879053 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e6d7df460 0x560e6d7ebe30 unknown :-1 s=CLOSED pgs=1917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.949+0000 7fe8575c9740 1 --2- 172.21.15.83:0/2993879053 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e6d7deb40 0x560e6d7f55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.949+0000 7fe8575c9740 1 -- 172.21.15.83:0/2993879053 >> 172.21.15.83:0/2993879053 conn(0x560e6d7dae60 msgr2=0x560e6d7e5180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:33.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.949+0000 7fe8575c9740 1 -- 172.21.15.83:0/2993879053 shutdown_connections 2024-04-24T09:03:33.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.949+0000 7fe8575c9740 1 -- 172.21.15.83:0/2993879053 wait complete. 2024-04-24T09:03:33.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.950+0000 7fe8575c9740 1 -- start start 2024-04-24T09:03:33.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.950+0000 7fe8575c9740 1 Processor -- start 2024-04-24T09:03:33.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.950+0000 7fe8575c9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e6d7deb40 0x560e6d7ea590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:33.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.950+0000 7fe8575c9740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e6d7df460 0x560e6d7eaad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:33.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.950+0000 7fe8575c9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e6d7e9b30 0x560e6d7eb010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:33.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.950+0000 7fe8575c9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe84c1b2b30 con 0x560e6d7e9b30 2024-04-24T09:03:33.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.950+0000 7fe8575c9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe84c1b2400 con 0x560e6d7deb40 2024-04-24T09:03:33.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.950+0000 7fe8575c9740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe84c1af600 con 0x560e6d7df460 2024-04-24T09:03:33.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.950+0000 7fe8575c9740 10 client.0 _ll_register_callbacks cb 0x7fe84c06a1f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:03:33.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.951+0000 7fe852800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e6d7e9b30 0x560e6d7eb010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:33.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.951+0000 7fe852800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e6d7e9b30 0x560e6d7eb010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56510/0 (socket says 172.21.15.83:56510) 2024-04-24T09:03:33.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.951+0000 7fe852800640 1 -- 172.21.15.83:0/356028537 learned_addr learned my addr 172.21.15.83:0/356028537 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:03:33.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.951+0000 7fe851e00640 1 --2- 172.21.15.83:0/356028537 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e6d7deb40 0x560e6d7ea590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:33.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.951+0000 7fe851400640 1 --2- 172.21.15.83:0/356028537 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e6d7df460 0x560e6d7eaad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:33.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.951+0000 7fe852800640 1 -- 172.21.15.83:0/356028537 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e6d7df460 msgr2=0x560e6d7eaad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:33.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.951+0000 7fe852800640 1 --2- 172.21.15.83:0/356028537 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e6d7df460 0x560e6d7eaad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.951+0000 7fe852800640 1 -- 172.21.15.83:0/356028537 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e6d7deb40 msgr2=0x560e6d7ea590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:33.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.951+0000 7fe852800640 1 --2- 172.21.15.83:0/356028537 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e6d7deb40 0x560e6d7ea590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.951+0000 7fe852800640 1 -- 172.21.15.83:0/356028537 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e6d7f0d60 con 0x560e6d7e9b30 2024-04-24T09:03:33.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.951+0000 7fe852800640 1 --2- 172.21.15.83:0/356028537 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e6d7e9b30 0x560e6d7eb010 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7fe84801cdf0 tx=0x7fe848004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f4c88e7562fc3d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:33.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.951+0000 7fe82f400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:03:33.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.952+0000 7fe82f400640 1 -- 172.21.15.83:0/356028537 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe84800fbb0 con 0x560e6d7e9b30 2024-04-24T09:03:33.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.952+0000 7fe8575c9740 1 -- 172.21.15.83:0/356028537 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe83c02b070 con 0x560e6d7e9b30 2024-04-24T09:03:33.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.952+0000 7fe82f400640 1 -- 172.21.15.83:0/356028537 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9583+0+0 (secure 0 0 0) 0x7fe84800fd50 con 0x560e6d7e9b30 2024-04-24T09:03:33.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.952+0000 7fe82f400640 1 -- 172.21.15.83:0/356028537 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fe84802d050 con 0x560e6d7e9b30 2024-04-24T09:03:33.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.952+0000 7fe8575c9740 10 client.19914 fetch_fsmap learned FSMap version 41 2024-04-24T09:03:33.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.952+0000 7fe8575c9740 1 -- 172.21.15.83:0/356028537 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x560e6d7f20c0 con 0x560e6d7e9b30 2024-04-24T09:03:33.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.952+0000 7fe82f400640 1 -- 172.21.15.83:0/356028537 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe83c02b070 con 0x560e6d7e9b30 2024-04-24T09:03:33.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.952+0000 7fe82f400640 1 -- 172.21.15.83:0/356028537 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7fe848024410 con 0x560e6d7e9b30 2024-04-24T09:03:33.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.953+0000 7fe8575c9740 10 client.19914 fetch_fsmap finished waiting for FSMap version 41 2024-04-24T09:03:33.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.953+0000 7fe8575c9740 10 client.19914 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:03:33.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.953+0000 7fe8575c9740 20 client.19914 populate_metadata read hostname 'smithi083' 2024-04-24T09:03:33.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.953+0000 7fe8575c9740 1 --2- 172.21.15.83:0/356028537 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x560e6d860360 0x560e6d862780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:33.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.953+0000 7fe8575c9740 4 client.19914 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:03:33.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.953+0000 7fe8575c9740 1 -- 172.21.15.83:0/356028537 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560e6d6c74d0 con 0x560e6d860360 2024-04-24T09:03:33.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.953+0000 7fe851e00640 1 --2- 172.21.15.83:0/356028537 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x560e6d860360 0x560e6d862780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:33.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.954+0000 7fe851e00640 1 --2- 172.21.15.83:0/356028537 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x560e6d860360 0x560e6d862780 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=432fc2acdc6dd6e0 server_cookie=a257e2dee6822387 in_seq=0 out_seq=0 2024-04-24T09:03:33.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.954+0000 7fe82f400640 10 client.19914 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:03:33.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.956+0000 7fe82f400640 1 -- 172.21.15.83:0/356028537 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560e6d6c74d0 con 0x560e6d860360 2024-04-24T09:03:33.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.956+0000 7fe82f400640 10 client.19914 handle_command_reply: tid=0 2024-04-24T09:03:33.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.956+0000 7fe8575c9740 1 client.19914 shutdown 2024-04-24T09:03:33.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.957+0000 7fe8575c9740 1 -- 172.21.15.83:0/356028537 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e6d7e9b30 msgr2=0x560e6d7eb010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:33.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.957+0000 7fe8575c9740 1 --2- 172.21.15.83:0/356028537 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e6d7e9b30 0x560e6d7eb010 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7fe84801cdf0 tx=0x7fe848004a40 comp rx=0 tx=0).stop 2024-04-24T09:03:33.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.957+0000 7fe8575c9740 1 -- 172.21.15.83:0/356028537 shutdown_connections 2024-04-24T09:03:33.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.957+0000 7fe8575c9740 1 --2- 172.21.15.83:0/356028537 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x560e6d860360 0x560e6d862780 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.957+0000 7fe8575c9740 1 --2- 172.21.15.83:0/356028537 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e6d7e9b30 0x560e6d7eb010 unknown :-1 s=CLOSED pgs=1918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.957+0000 7fe8575c9740 1 --2- 172.21.15.83:0/356028537 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e6d7df460 0x560e6d7eaad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.957+0000 7fe8575c9740 1 --2- 172.21.15.83:0/356028537 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e6d7deb40 0x560e6d7ea590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.957+0000 7fe82f400640 0 client.19914 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:03:33.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.957+0000 7fe8575c9740 1 -- 172.21.15.83:0/356028537 >> 172.21.15.83:0/356028537 conn(0x560e6d7dae60 msgr2=0x560e6d7e55e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:33.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.957+0000 7fe8575c9740 1 -- 172.21.15.83:0/356028537 shutdown_connections 2024-04-24T09:03:33.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.957+0000 7fe8575c9740 1 -- 172.21.15.83:0/356028537 wait complete. 2024-04-24T09:03:33.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.957+0000 7fe8575c9740 20 client.19914 trim_cache size 0 max 16384 2024-04-24T09:03:33.961 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:03:33.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.958+0000 7fe853200640 1 -- 172.21.15.83:0/987326575 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe8340a3dd0 msgr2=0x7fe8340a6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:33.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.958+0000 7fe853200640 1 --2- 172.21.15.83:0/987326575 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe8340a3dd0 0x7fe8340a6290 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fe848009d90 tx=0x7fe8480092c0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.958+0000 7fe853200640 1 -- 172.21.15.83:0/987326575 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe84c103060 msgr2=0x7fe84c100cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:33.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.958+0000 7fe853200640 1 --2- 172.21.15.83:0/987326575 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe84c103060 0x7fe84c100cb0 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7fe840005940 tx=0x7fe840004270 comp rx=0 tx=0).stop 2024-04-24T09:03:33.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.958+0000 7fe853200640 1 -- 172.21.15.83:0/987326575 shutdown_connections 2024-04-24T09:03:33.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.958+0000 7fe853200640 1 --2- 172.21.15.83:0/987326575 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe8340a3dd0 0x7fe8340a6290 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.958+0000 7fe853200640 1 --2- 172.21.15.83:0/987326575 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe84c10c480 0x7fe84c101730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.958+0000 7fe853200640 1 --2- 172.21.15.83:0/987326575 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe84c103980 0x7fe84c1011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.958+0000 7fe853200640 1 --2- 172.21.15.83:0/987326575 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe84c103060 0x7fe84c100cb0 unknown :-1 s=CLOSED pgs=1914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:33.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.958+0000 7fe853200640 1 -- 172.21.15.83:0/987326575 >> 172.21.15.83:0/987326575 conn(0x7fe84c0fc880 msgr2=0x7fe84c0fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:33.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.958+0000 7fe853200640 1 -- 172.21.15.83:0/987326575 shutdown_connections 2024-04-24T09:03:33.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:33.958+0000 7fe853200640 1 -- 172.21.15.83:0/987326575 wait complete. 2024-04-24T09:03:34.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:03:34 smithi083 ceph-mon[27907]: pgmap v375: 129 pgs: 129 active+clean; 62 MiB data, 885 MiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 531 KiB/s wr, 80 op/s 2024-04-24T09:03:34.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:03:34 smithi138 ceph-mon[31441]: pgmap v375: 129 pgs: 129 active+clean; 62 MiB data, 885 MiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 531 KiB/s wr, 80 op/s 2024-04-24T09:03:34.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:03:34 smithi189 ceph-mon[31791]: pgmap v375: 129 pgs: 129 active+clean; 62 MiB data, 885 MiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 531 KiB/s wr, 80 op/s 2024-04-24T09:03:34.620 INFO:tasks.workunit.client.0.smithi083.stdout: 1 69229 0.60 MB/sec execute 488 sec latency 76.018 ms 2024-04-24T09:03:34.978 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T09:03:34.979 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T09:03:34.979 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T09:03:35.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.211+0000 7f7955800640 1 -- 172.21.15.83:0/1773797135 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f795006b220 msgr2=0x7f7950106c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:35.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.211+0000 7f7955800640 1 --2- 172.21.15.83:0/1773797135 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f795006b220 0x7f7950106c50 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7f7938009900 tx=0x7f793802a410 comp rx=0 tx=0).stop 2024-04-24T09:03:35.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.211+0000 7f7955800640 1 -- 172.21.15.83:0/1773797135 shutdown_connections 2024-04-24T09:03:35.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.211+0000 7f7955800640 1 --2- 172.21.15.83:0/1773797135 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7950107330 0x7f795010eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:35.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.211+0000 7f7955800640 1 --2- 172.21.15.83:0/1773797135 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f795006b220 0x7f7950106c50 unknown :-1 s=CLOSED pgs=1919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:35.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.211+0000 7f7955800640 1 --2- 172.21.15.83:0/1773797135 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f795006a870 0x7f795006ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:35.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.211+0000 7f7955800640 1 -- 172.21.15.83:0/1773797135 >> 172.21.15.83:0/1773797135 conn(0x7f7950076d00 msgr2=0x7f7950077110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:35.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.211+0000 7f7955800640 1 -- 172.21.15.83:0/1773797135 shutdown_connections 2024-04-24T09:03:35.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.212+0000 7f7955800640 1 -- 172.21.15.83:0/1773797135 wait complete. 2024-04-24T09:03:35.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.213+0000 7f7955800640 1 Processor -- start 2024-04-24T09:03:35.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.213+0000 7f7955800640 1 -- start start 2024-04-24T09:03:35.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.213+0000 7f7955800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f795006a870 0x7f79501a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:35.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.213+0000 7f7955800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7950107330 0x7f79501a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:35.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.213+0000 7f7955800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f795019a7c0 0x7f795019ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:35.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.213+0000 7f7955800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7950112250 con 0x7f795006a870 2024-04-24T09:03:35.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.213+0000 7f7955800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f79501120d0 con 0x7f795019a7c0 2024-04-24T09:03:35.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.213+0000 7f7955800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f79501123d0 con 0x7f7950107330 2024-04-24T09:03:35.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.214+0000 7f794ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f795006a870 0x7f79501a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:35.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.214+0000 7f794ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f795006a870 0x7f79501a0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56524/0 (socket says 172.21.15.83:56524) 2024-04-24T09:03:35.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.214+0000 7f794ea00640 1 -- 172.21.15.83:0/2457255222 learned_addr learned my addr 172.21.15.83:0/2457255222 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:03:35.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.214+0000 7f794e000640 1 --2- 172.21.15.83:0/2457255222 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7950107330 0x7f79501a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:35.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.214+0000 7f794f400640 1 --2- 172.21.15.83:0/2457255222 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f795019a7c0 0x7f795019ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:35.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.214+0000 7f794ea00640 1 -- 172.21.15.83:0/2457255222 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7950107330 msgr2=0x7f79501a0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:35.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.214+0000 7f794ea00640 1 --2- 172.21.15.83:0/2457255222 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7950107330 0x7f79501a0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:35.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.214+0000 7f794ea00640 1 -- 172.21.15.83:0/2457255222 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f795019a7c0 msgr2=0x7f795019ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:35.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.214+0000 7f794ea00640 1 --2- 172.21.15.83:0/2457255222 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f795019a7c0 0x7f795019ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:35.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.214+0000 7f794ea00640 1 -- 172.21.15.83:0/2457255222 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f795019b350 con 0x7f795006a870 2024-04-24T09:03:35.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.214+0000 7f794f400640 1 --2- 172.21.15.83:0/2457255222 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f795019a7c0 0x7f795019ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:03:35.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.214+0000 7f794ea00640 1 --2- 172.21.15.83:0/2457255222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f795006a870 0x7f79501a0650 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7f794000eab0 tx=0x7f794000ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:35.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.215+0000 7f7947400640 1 -- 172.21.15.83:0/2457255222 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f794000ce40 con 0x7f795006a870 2024-04-24T09:03:35.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.215+0000 7f7955800640 1 -- 172.21.15.83:0/2457255222 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f795019b640 con 0x7f795006a870 2024-04-24T09:03:35.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.215+0000 7f7947400640 1 -- 172.21.15.83:0/2457255222 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7940011920 con 0x7f795006a870 2024-04-24T09:03:35.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.215+0000 7f7947400640 1 -- 172.21.15.83:0/2457255222 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7940010a00 con 0x7f795006a870 2024-04-24T09:03:35.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.215+0000 7f7955800640 1 -- 172.21.15.83:0/2457255222 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79501a77f0 con 0x7f795006a870 2024-04-24T09:03:35.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.219+0000 7f7958790740 1 Processor -- start 2024-04-24T09:03:35.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.220+0000 7f7958790740 1 -- start start 2024-04-24T09:03:35.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.220+0000 7f7947400640 1 -- 172.21.15.83:0/2457255222 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7940011ad0 con 0x7f795006a870 2024-04-24T09:03:35.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.220+0000 7f7958790740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55822179e090 0x55822179e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:35.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.220+0000 7f7958790740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55822179e9b0 0x5582217a8b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:35.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.220+0000 7f7958790740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5582217a9080 0x5582217ab470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:35.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.220+0000 7f7958790740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x558221703e70 con 0x55822179e090 2024-04-24T09:03:35.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.220+0000 7f7958790740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5582216aa380 con 0x5582217a9080 2024-04-24T09:03:35.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.220+0000 7f7958790740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5582216ffe30 con 0x55822179e9b0 2024-04-24T09:03:35.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.220+0000 7f794f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55822179e090 0x55822179e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:35.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.220+0000 7f794f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55822179e090 0x55822179e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56540/0 (socket says 172.21.15.83:56540) 2024-04-24T09:03:35.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.221+0000 7f794f400640 1 -- 172.21.15.83:0/1391329739 learned_addr learned my addr 172.21.15.83:0/1391329739 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:03:35.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.221+0000 7f794ea00640 1 --2- 172.21.15.83:0/1391329739 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55822179e9b0 0x5582217a8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:35.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.221+0000 7f794f400640 1 -- 172.21.15.83:0/1391329739 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55822179e9b0 msgr2=0x5582217a8b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:35.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.221+0000 7f794f400640 1 --2- 172.21.15.83:0/1391329739 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55822179e9b0 0x5582217a8b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:35.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.221+0000 7f794f400640 1 -- 172.21.15.83:0/1391329739 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5582217a9080 msgr2=0x5582217ab470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:35.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.221+0000 7f794f400640 1 --2- 172.21.15.83:0/1391329739 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5582217a9080 0x5582217ab470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:35.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.221+0000 7f794f400640 1 -- 172.21.15.83:0/1391329739 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582214de7e0 con 0x55822179e090 2024-04-24T09:03:35.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.221+0000 7f794ea00640 1 --2- 172.21.15.83:0/1391329739 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55822179e9b0 0x5582217a8b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:03:35.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.221+0000 7f794f400640 1 --2- 172.21.15.83:0/1391329739 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55822179e090 0x55822179e470 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7f7948009bd0 tx=0x7f794800e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3236d52292eafa47 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:35.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.222+0000 7f793fe00640 1 -- 172.21.15.83:0/1391329739 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7948016070 con 0x55822179e090 2024-04-24T09:03:35.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.222+0000 7f793fe00640 1 -- 172.21.15.83:0/1391329739 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7948011920 con 0x55822179e090 2024-04-24T09:03:35.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.222+0000 7f793fe00640 1 -- 172.21.15.83:0/1391329739 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7948010790 con 0x55822179e090 2024-04-24T09:03:35.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.222+0000 7f7947400640 1 --2- 172.21.15.83:0/2457255222 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f79340a3d20 0x7f79340a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:35.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.222+0000 7f7947400640 1 -- 172.21.15.83:0/2457255222 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9583+0+0 (secure 0 0 0) 0x7f79400d1320 con 0x7f795006a870 2024-04-24T09:03:35.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.222+0000 7f794ea00640 1 --2- 172.21.15.83:0/2457255222 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f79340a3d20 0x7f79340a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:35.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.223+0000 7f7958790740 1 -- 172.21.15.83:0/1391329739 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55822179e090 msgr2=0x55822179e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:35.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.223+0000 7f7958790740 1 --2- 172.21.15.83:0/1391329739 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55822179e090 0x55822179e470 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7f7948009bd0 tx=0x7f794800e6c0 comp rx=0 tx=0).stop 2024-04-24T09:03:35.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.223+0000 7f7958790740 1 -- 172.21.15.83:0/1391329739 shutdown_connections 2024-04-24T09:03:35.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.223+0000 7f7958790740 1 --2- 172.21.15.83:0/1391329739 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5582217a9080 0x5582217ab470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:35.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.223+0000 7f7958790740 1 --2- 172.21.15.83:0/1391329739 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55822179e9b0 0x5582217a8b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:35.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.223+0000 7f7958790740 1 --2- 172.21.15.83:0/1391329739 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55822179e090 0x55822179e470 unknown :-1 s=CLOSED pgs=1921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:35.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.223+0000 7f7958790740 1 -- 172.21.15.83:0/1391329739 >> 172.21.15.83:0/1391329739 conn(0x55822179a3b0 msgr2=0x55822179a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:35.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.223+0000 7f7958790740 1 -- 172.21.15.83:0/1391329739 shutdown_connections 2024-04-24T09:03:35.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.223+0000 7f7958790740 1 -- 172.21.15.83:0/1391329739 wait complete. 2024-04-24T09:03:35.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.223+0000 7f794ea00640 1 --2- 172.21.15.83:0/2457255222 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f79340a3d20 0x7f79340a61e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f79400ce700 tx=0x7f79400094a0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:35.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.223+0000 7f7958790740 1 -- start start 2024-04-24T09:03:35.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.224+0000 7f7958790740 1 Processor -- start 2024-04-24T09:03:35.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.224+0000 7f7958790740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55822179e090 0x5582217a6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:35.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.224+0000 7f7958790740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55822179e9b0 0x5582217a65d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:35.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.224+0000 7f7958790740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582217a9080 0x5582217a6b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:35.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.224+0000 7f7958790740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5582217abe70 con 0x5582217a9080 2024-04-24T09:03:35.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.224+0000 7f7958790740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5582216bd660 con 0x55822179e090 2024-04-24T09:03:35.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.224+0000 7f7958790740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5582217abff0 con 0x55822179e9b0 2024-04-24T09:03:35.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.224+0000 7f7958790740 10 client.0 _ll_register_callbacks cb 0x5582216abaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:03:35.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.224+0000 7f794f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582217a9080 0x5582217a6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:35.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.224+0000 7f794f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582217a9080 0x5582217a6b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56552/0 (socket says 172.21.15.83:56552) 2024-04-24T09:03:35.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.224+0000 7f794f400640 1 -- 172.21.15.83:0/4114962187 learned_addr learned my addr 172.21.15.83:0/4114962187 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:03:35.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.225+0000 7f794e000640 1 --2- 172.21.15.83:0/4114962187 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55822179e9b0 0x5582217a65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:35.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.225+0000 7f794f400640 1 --2- 172.21.15.83:0/4114962187 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55822179e090 0x5582217a6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:35.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.225+0000 7f794f400640 1 -- 172.21.15.83:0/4114962187 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55822179e9b0 msgr2=0x5582217a65d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:35.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.225+0000 7f794f400640 1 --2- 172.21.15.83:0/4114962187 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55822179e9b0 0x5582217a65d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:35.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.225+0000 7f794f400640 1 -- 172.21.15.83:0/4114962187 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55822179e090 msgr2=0x5582217a6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:35.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.225+0000 7f794f400640 1 --2- 172.21.15.83:0/4114962187 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55822179e090 0x5582217a6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:35.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.225+0000 7f794f400640 1 -- 172.21.15.83:0/4114962187 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55822153ef70 con 0x5582217a9080 2024-04-24T09:03:35.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.225+0000 7f794e000640 1 --2- 172.21.15.83:0/4114962187 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55822179e9b0 0x5582217a65d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:03:35.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.225+0000 7f794f400640 1 --2- 172.21.15.83:0/4114962187 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582217a9080 0x5582217a6b10 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7f7948009fd0 tx=0x7f79480096e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=182a6dc0b26dd73b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:35.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.225+0000 7f793cc00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:03:35.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.226+0000 7f793cc00640 1 -- 172.21.15.83:0/4114962187 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f794801f070 con 0x5582217a9080 2024-04-24T09:03:35.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.226+0000 7f793cc00640 1 -- 172.21.15.83:0/4114962187 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9583+0+0 (secure 0 0 0) 0x7f7948005050 con 0x5582217a9080 2024-04-24T09:03:35.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.226+0000 7f7958790740 1 -- 172.21.15.83:0/4114962187 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7948011920 con 0x5582217a9080 2024-04-24T09:03:35.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.226+0000 7f793cc00640 1 -- 172.21.15.83:0/4114962187 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f7948031050 con 0x5582217a9080 2024-04-24T09:03:35.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.226+0000 7f7958790740 10 client.19926 fetch_fsmap learned FSMap version 41 2024-04-24T09:03:35.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.226+0000 7f7958790740 1 -- 172.21.15.83:0/4114962187 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5582213d6170 con 0x5582217a9080 2024-04-24T09:03:35.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.226+0000 7f793cc00640 1 -- 172.21.15.83:0/4114962187 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7948011920 con 0x5582217a9080 2024-04-24T09:03:35.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.227+0000 7f793cc00640 1 -- 172.21.15.83:0/4114962187 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7f794800f040 con 0x5582217a9080 2024-04-24T09:03:35.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.227+0000 7f7958790740 10 client.19926 fetch_fsmap finished waiting for FSMap version 41 2024-04-24T09:03:35.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.227+0000 7f7958790740 10 client.19926 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:03:35.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.227+0000 7f7958790740 20 client.19926 populate_metadata read hostname 'smithi083' 2024-04-24T09:03:35.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.227+0000 7f7958790740 1 --2- 172.21.15.83:0/4114962187 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5582217b92f0 0x5582217bb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:35.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.227+0000 7f7958790740 4 client.19926 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:03:35.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.227+0000 7f7958790740 1 -- 172.21.15.83:0/4114962187 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558221711890 con 0x5582217b92f0 2024-04-24T09:03:35.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.227+0000 7f794e000640 1 --2- 172.21.15.83:0/4114962187 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5582217b92f0 0x5582217bb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:35.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.228+0000 7f794e000640 1 --2- 172.21.15.83:0/4114962187 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5582217b92f0 0x5582217bb6e0 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fbcc49c4d2d0f65d server_cookie=8383df047f40173f in_seq=0 out_seq=0 2024-04-24T09:03:35.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.228+0000 7f793cc00640 10 client.19926 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:03:35.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.231+0000 7f793cc00640 1 -- 172.21.15.83:0/4114962187 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558221711890 con 0x5582217b92f0 2024-04-24T09:03:35.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.231+0000 7f793cc00640 10 client.19926 handle_command_reply: tid=0 2024-04-24T09:03:35.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.231+0000 7f7958790740 1 client.19926 shutdown 2024-04-24T09:03:35.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.231+0000 7f7958790740 1 -- 172.21.15.83:0/4114962187 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582217a9080 msgr2=0x5582217a6b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:35.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.231+0000 7f7958790740 1 --2- 172.21.15.83:0/4114962187 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582217a9080 0x5582217a6b10 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7f7948009fd0 tx=0x7f79480096e0 comp rx=0 tx=0).stop 2024-04-24T09:03:35.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.231+0000 7f7958790740 1 -- 172.21.15.83:0/4114962187 shutdown_connections 2024-04-24T09:03:35.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.231+0000 7f7958790740 1 --2- 172.21.15.83:0/4114962187 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5582217b92f0 0x5582217bb6e0 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:35.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.231+0000 7f7958790740 1 --2- 172.21.15.83:0/4114962187 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582217a9080 0x5582217a6b10 unknown :-1 s=CLOSED pgs=1922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:35.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.231+0000 7f7958790740 1 --2- 172.21.15.83:0/4114962187 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55822179e9b0 0x5582217a65d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:35.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.231+0000 7f7958790740 1 --2- 172.21.15.83:0/4114962187 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55822179e090 0x5582217a6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:35.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.231+0000 7f7958790740 1 -- 172.21.15.83:0/4114962187 >> 172.21.15.83:0/4114962187 conn(0x55822179a3b0 msgr2=0x5582217a01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:35.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.231+0000 7f793cc00640 0 client.19926 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:03:35.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.231+0000 7f7958790740 1 -- 172.21.15.83:0/4114962187 shutdown_connections 2024-04-24T09:03:35.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.231+0000 7f7958790740 1 -- 172.21.15.83:0/4114962187 wait complete. 2024-04-24T09:03:35.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.231+0000 7f7958790740 20 client.19926 trim_cache size 0 max 16384 2024-04-24T09:03:35.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.252+0000 7f7958790740 1 Processor -- start 2024-04-24T09:03:35.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.252+0000 7f7958790740 1 -- start start 2024-04-24T09:03:35.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.252+0000 7f7958790740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55822179e090 0x5582217b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:35.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.252+0000 7f7958790740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55822179e9b0 0x5582217a4f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:35.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.252+0000 7f7958790740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5582217a9080 0x5582217c0d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:35.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.252+0000 7f7958790740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f79501b17b0 con 0x55822179e9b0 2024-04-24T09:03:35.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.252+0000 7f7958790740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f79501b3410 con 0x55822179e090 2024-04-24T09:03:35.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.252+0000 7f7958790740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f79501b1e80 con 0x5582217a9080 2024-04-24T09:03:35.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.253+0000 7f794e000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55822179e9b0 0x5582217a4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:35.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.253+0000 7f794e000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55822179e9b0 0x5582217a4f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56556/0 (socket says 172.21.15.83:56556) 2024-04-24T09:03:35.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.253+0000 7f794e000640 1 -- 172.21.15.83:0/3424982236 learned_addr learned my addr 172.21.15.83:0/3424982236 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:03:35.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.253+0000 7f794e000640 1 -- 172.21.15.83:0/3424982236 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5582217a9080 msgr2=0x5582217c0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:35.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.253+0000 7f794e000640 1 --2- 172.21.15.83:0/3424982236 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5582217a9080 0x5582217c0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:35.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.253+0000 7f794e000640 1 -- 172.21.15.83:0/3424982236 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55822179e090 msgr2=0x5582217b4d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:35.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.253+0000 7f794f400640 1 --2- 172.21.15.83:0/3424982236 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55822179e090 0x5582217b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:35.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.253+0000 7f794e000640 1 --2- 172.21.15.83:0/3424982236 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55822179e090 0x5582217b4d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:35.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.253+0000 7f794e000640 1 -- 172.21.15.83:0/3424982236 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55822179e7e0 con 0x55822179e9b0 2024-04-24T09:03:35.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.253+0000 7f794f400640 1 --2- 172.21.15.83:0/3424982236 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55822179e090 0x5582217b4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:03:35.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.254+0000 7f794e000640 1 --2- 172.21.15.83:0/3424982236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55822179e9b0 0x5582217a4f40 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7f7950103480 tx=0x7f79380095f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4651434ae8ee2368 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:35.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.254+0000 7f793f400640 1 -- 172.21.15.83:0/3424982236 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7938035040 con 0x55822179e9b0 2024-04-24T09:03:35.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.254+0000 7f793f400640 1 -- 172.21.15.83:0/3424982236 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7938005040 con 0x55822179e9b0 2024-04-24T09:03:35.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.254+0000 7f793f400640 1 -- 172.21.15.83:0/3424982236 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7938042790 con 0x55822179e9b0 2024-04-24T09:03:35.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.255+0000 7f7958790740 1 -- 172.21.15.83:0/3424982236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55822179e9b0 msgr2=0x5582217a4f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:35.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.255+0000 7f7958790740 1 --2- 172.21.15.83:0/3424982236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55822179e9b0 0x5582217a4f40 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7f7950103480 tx=0x7f79380095f0 comp rx=0 tx=0).stop 2024-04-24T09:03:35.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.255+0000 7f7958790740 1 -- 172.21.15.83:0/3424982236 shutdown_connections 2024-04-24T09:03:35.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.255+0000 7f7958790740 1 --2- 172.21.15.83:0/3424982236 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5582217a9080 0x5582217c0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:35.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.255+0000 7f7958790740 1 --2- 172.21.15.83:0/3424982236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55822179e9b0 0x5582217a4f40 unknown :-1 s=CLOSED pgs=1923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:35.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.255+0000 7f7958790740 1 --2- 172.21.15.83:0/3424982236 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55822179e090 0x5582217b4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:35.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.255+0000 7f7958790740 1 -- 172.21.15.83:0/3424982236 >> 172.21.15.83:0/3424982236 conn(0x55822179a3b0 msgr2=0x5582217a4830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:35.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.255+0000 7f7958790740 1 -- 172.21.15.83:0/3424982236 shutdown_connections 2024-04-24T09:03:35.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.255+0000 7f7958790740 1 -- 172.21.15.83:0/3424982236 wait complete. 2024-04-24T09:03:35.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.256+0000 7f7958790740 1 -- start start 2024-04-24T09:03:35.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.256+0000 7f7958790740 1 Processor -- start 2024-04-24T09:03:35.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.256+0000 7f7958790740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55822179e090 0x5582217ad310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:35.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.257+0000 7f7958790740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55822179e9b0 0x5582217ad850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:35.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.257+0000 7f7958790740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582217a9080 0x55822181e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:35.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.257+0000 7f7958790740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f79501b2ce0 con 0x5582217a9080 2024-04-24T09:03:35.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.257+0000 7f7958790740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f79501b25b0 con 0x55822179e9b0 2024-04-24T09:03:35.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.257+0000 7f7958790740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f79501af7b0 con 0x55822179e090 2024-04-24T09:03:35.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.257+0000 7f7958790740 10 client.0 _ll_register_callbacks cb 0x7f7948009a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:03:35.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.257+0000 7f794f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582217a9080 0x55822181e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:35.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.257+0000 7f794f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582217a9080 0x55822181e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56558/0 (socket says 172.21.15.83:56558) 2024-04-24T09:03:35.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.257+0000 7f794f400640 1 -- 172.21.15.83:0/1981320049 learned_addr learned my addr 172.21.15.83:0/1981320049 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:03:35.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.257+0000 7f794f400640 1 --2- 172.21.15.83:0/1981320049 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55822179e090 0x5582217ad310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:35.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.257+0000 7f794e000640 1 --2- 172.21.15.83:0/1981320049 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55822179e9b0 0x5582217ad850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:35.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.257+0000 7f794f400640 1 -- 172.21.15.83:0/1981320049 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55822179e090 msgr2=0x5582217ad310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:35.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.257+0000 7f794f400640 1 --2- 172.21.15.83:0/1981320049 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55822179e090 0x5582217ad310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:35.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.257+0000 7f794f400640 1 -- 172.21.15.83:0/1981320049 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55822179e9b0 msgr2=0x5582217ad850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:35.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.257+0000 7f794f400640 1 --2- 172.21.15.83:0/1981320049 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55822179e9b0 0x5582217ad850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:35.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.257+0000 7f794f400640 1 -- 172.21.15.83:0/1981320049 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55822181eaf0 con 0x5582217a9080 2024-04-24T09:03:35.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.257+0000 7f794e000640 1 --2- 172.21.15.83:0/1981320049 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55822179e9b0 0x5582217ad850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:03:35.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.258+0000 7f794f400640 1 --2- 172.21.15.83:0/1981320049 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582217a9080 0x55822181e5b0 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7f79480104c0 tx=0x7f7948005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b773f300dce08aa8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:35.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.258+0000 7f793cc00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:03:35.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.258+0000 7f793cc00640 1 -- 172.21.15.83:0/1981320049 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f794803a060 con 0x5582217a9080 2024-04-24T09:03:35.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.258+0000 7f7958790740 1 -- 172.21.15.83:0/1981320049 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7938005040 con 0x5582217a9080 2024-04-24T09:03:35.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.258+0000 7f793cc00640 1 -- 172.21.15.83:0/1981320049 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9583+0+0 (secure 0 0 0) 0x7f7948011600 con 0x5582217a9080 2024-04-24T09:03:35.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.258+0000 7f793cc00640 1 -- 172.21.15.83:0/1981320049 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f7948041050 con 0x5582217a9080 2024-04-24T09:03:35.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.258+0000 7f7958790740 10 client.19932 fetch_fsmap learned FSMap version 41 2024-04-24T09:03:35.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.258+0000 7f7958790740 1 -- 172.21.15.83:0/1981320049 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55822181fcd0 con 0x5582217a9080 2024-04-24T09:03:35.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.259+0000 7f793cc00640 1 -- 172.21.15.83:0/1981320049 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7938005040 con 0x5582217a9080 2024-04-24T09:03:35.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.259+0000 7f793cc00640 1 -- 172.21.15.83:0/1981320049 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7f79480053b0 con 0x5582217a9080 2024-04-24T09:03:35.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.259+0000 7f7958790740 10 client.19932 fetch_fsmap finished waiting for FSMap version 41 2024-04-24T09:03:35.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.259+0000 7f7958790740 10 client.19932 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:03:35.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.259+0000 7f7958790740 20 client.19932 populate_metadata read hostname 'smithi083' 2024-04-24T09:03:35.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.259+0000 7f7958790740 1 --2- 172.21.15.83:0/1981320049 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x558221820480 0x5582218228a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:35.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.259+0000 7f7958790740 4 client.19932 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T09:03:35.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.259+0000 7f7958790740 1 -- 172.21.15.83:0/1981320049 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55822170b700 con 0x558221820480 2024-04-24T09:03:35.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.260+0000 7f794e000640 1 --2- 172.21.15.83:0/1981320049 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x558221820480 0x5582218228a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:35.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.260+0000 7f794e000640 1 --2- 172.21.15.83:0/1981320049 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x558221820480 0x5582218228a0 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=793b3c645b092a8 server_cookie=e6b90d28e33d4601 in_seq=0 out_seq=0 2024-04-24T09:03:35.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.260+0000 7f793cc00640 10 client.19932 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:03:35.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.262+0000 7f793cc00640 1 -- 172.21.15.83:0/1981320049 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55822170b700 con 0x558221820480 2024-04-24T09:03:35.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.262+0000 7f793cc00640 10 client.19932 handle_command_reply: tid=0 2024-04-24T09:03:35.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.262+0000 7f7958790740 1 client.19932 shutdown 2024-04-24T09:03:35.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.263+0000 7f7958790740 1 -- 172.21.15.83:0/1981320049 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582217a9080 msgr2=0x55822181e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:35.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.263+0000 7f7958790740 1 --2- 172.21.15.83:0/1981320049 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582217a9080 0x55822181e5b0 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7f79480104c0 tx=0x7f7948005e50 comp rx=0 tx=0).stop 2024-04-24T09:03:35.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.263+0000 7f7958790740 1 -- 172.21.15.83:0/1981320049 shutdown_connections 2024-04-24T09:03:35.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.263+0000 7f7958790740 1 --2- 172.21.15.83:0/1981320049 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x558221820480 0x5582218228a0 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:35.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.263+0000 7f7958790740 1 --2- 172.21.15.83:0/1981320049 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582217a9080 0x55822181e5b0 unknown :-1 s=CLOSED pgs=1924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:35.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.263+0000 7f7958790740 1 --2- 172.21.15.83:0/1981320049 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55822179e9b0 0x5582217ad850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:35.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.263+0000 7f7958790740 1 --2- 172.21.15.83:0/1981320049 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55822179e090 0x5582217ad310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:35.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.263+0000 7f793cc00640 0 client.19932 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:03:35.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.263+0000 7f7958790740 1 -- 172.21.15.83:0/1981320049 >> 172.21.15.83:0/1981320049 conn(0x55822179a3b0 msgr2=0x5582217a68e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:35.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.263+0000 7f7958790740 1 -- 172.21.15.83:0/1981320049 shutdown_connections 2024-04-24T09:03:35.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.263+0000 7f7958790740 1 -- 172.21.15.83:0/1981320049 wait complete. 2024-04-24T09:03:35.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.263+0000 7f7958790740 20 client.19932 trim_cache size 0 max 16384 2024-04-24T09:03:35.265 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T09:03:35.265 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-04-24T09:03:35.265 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "7404d51c-e420-4277-ba1f-9d551c5c7f66", 2024-04-24T09:03:35.265 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-04-24T09:03:35.265 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T09:03:35.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.264+0000 7f7955800640 1 -- 172.21.15.83:0/2457255222 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f79340a3d20 msgr2=0x7f79340a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:35.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.264+0000 7f7955800640 1 --2- 172.21.15.83:0/2457255222 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f79340a3d20 0x7f79340a61e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f79400ce700 tx=0x7f79400094a0 comp rx=0 tx=0).stop 2024-04-24T09:03:35.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.264+0000 7f7955800640 1 -- 172.21.15.83:0/2457255222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f795006a870 msgr2=0x7f79501a0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:35.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.264+0000 7f7955800640 1 --2- 172.21.15.83:0/2457255222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f795006a870 0x7f79501a0650 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7f794000eab0 tx=0x7f794000ef70 comp rx=0 tx=0).stop 2024-04-24T09:03:35.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.264+0000 7f7955800640 1 -- 172.21.15.83:0/2457255222 shutdown_connections 2024-04-24T09:03:35.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.264+0000 7f7955800640 1 --2- 172.21.15.83:0/2457255222 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f79340a3d20 0x7f79340a61e0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:35.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.264+0000 7f7955800640 1 --2- 172.21.15.83:0/2457255222 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f795019a7c0 0x7f795019ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:35.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.264+0000 7f7955800640 1 --2- 172.21.15.83:0/2457255222 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7950107330 0x7f79501a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:35.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.264+0000 7f7955800640 1 --2- 172.21.15.83:0/2457255222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f795006a870 0x7f79501a0650 unknown :-1 s=CLOSED pgs=1920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:35.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.264+0000 7f7955800640 1 -- 172.21.15.83:0/2457255222 >> 172.21.15.83:0/2457255222 conn(0x7f7950076d00 msgr2=0x7f795010d360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:35.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.264+0000 7f7955800640 1 -- 172.21.15.83:0/2457255222 shutdown_connections 2024-04-24T09:03:35.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.264+0000 7f7955800640 1 -- 172.21.15.83:0/2457255222 wait complete. 2024-04-24T09:03:35.282 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T09:03:35.283 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T09:03:35.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.499+0000 7f672d800640 1 -- 172.21.15.83:0/20628924 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67281093f0 msgr2=0x7f6728110fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:35.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.499+0000 7f672d800640 1 --2- 172.21.15.83:0/20628924 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67281093f0 0x7f6728110fc0 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7f6720009ca0 tx=0x7f672002a8c0 comp rx=0 tx=0).stop 2024-04-24T09:03:35.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.499+0000 7f672d800640 1 -- 172.21.15.83:0/20628924 shutdown_connections 2024-04-24T09:03:35.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.499+0000 7f672d800640 1 --2- 172.21.15.83:0/20628924 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67281093f0 0x7f6728110fc0 unknown :-1 s=CLOSED pgs=1925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:35.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.499+0000 7f672d800640 1 --2- 172.21.15.83:0/20628924 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6728104cd0 0x7f6728108d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:35.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.499+0000 7f672d800640 1 --2- 172.21.15.83:0/20628924 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6728104320 0x7f6728104700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:35.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.499+0000 7f672d800640 1 -- 172.21.15.83:0/20628924 >> 172.21.15.83:0/20628924 conn(0x7f67280ffb40 msgr2=0x7f6728101f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:35.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.499+0000 7f672d800640 1 -- 172.21.15.83:0/20628924 shutdown_connections 2024-04-24T09:03:35.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.499+0000 7f672d800640 1 -- 172.21.15.83:0/20628924 wait complete. 2024-04-24T09:03:35.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.500+0000 7f672d800640 1 Processor -- start 2024-04-24T09:03:35.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.500+0000 7f672d800640 1 -- start start 2024-04-24T09:03:35.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.500+0000 7f672d800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6728104320 0x7f67281a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:35.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.500+0000 7f672d800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f67281093f0 0x7f67281a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:35.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.500+0000 7f672d800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f672819a780 0x7f672819ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:35.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.500+0000 7f672d800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6728114360 con 0x7f672819a780 2024-04-24T09:03:35.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.500+0000 7f672d800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f67281141e0 con 0x7f6728104320 2024-04-24T09:03:35.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.500+0000 7f672d800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f67281144e0 con 0x7f67281093f0 2024-04-24T09:03:35.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.501+0000 7f672ce00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f672819a780 0x7f672819ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:35.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.501+0000 7f672ce00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f672819a780 0x7f672819ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56584/0 (socket says 172.21.15.83:56584) 2024-04-24T09:03:35.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.501+0000 7f672ce00640 1 -- 172.21.15.83:0/2693687986 learned_addr learned my addr 172.21.15.83:0/2693687986 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:03:35.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.501+0000 7f6727400640 1 --2- 172.21.15.83:0/2693687986 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f67281093f0 0x7f67281a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:35.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.501+0000 7f672ce00640 1 -- 172.21.15.83:0/2693687986 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f67281093f0 msgr2=0x7f67281a0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:35.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.501+0000 7f672ce00640 1 --2- 172.21.15.83:0/2693687986 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f67281093f0 0x7f67281a0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:35.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.501+0000 7f6727e00640 1 --2- 172.21.15.83:0/2693687986 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6728104320 0x7f67281a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:35.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.501+0000 7f672ce00640 1 -- 172.21.15.83:0/2693687986 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6728104320 msgr2=0x7f67281a0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:35.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.501+0000 7f672ce00640 1 --2- 172.21.15.83:0/2693687986 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6728104320 0x7f67281a0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:35.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.501+0000 7f672ce00640 1 -- 172.21.15.83:0/2693687986 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f672819b380 con 0x7f672819a780 2024-04-24T09:03:35.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.501+0000 7f6727400640 1 --2- 172.21.15.83:0/2693687986 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f67281093f0 0x7f67281a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:03:35.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.502+0000 7f672ce00640 1 --2- 172.21.15.83:0/2693687986 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f672819a780 0x7f672819ac00 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7f6720009dc0 tx=0x7f6720032ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:35.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.502+0000 7f6724c00640 1 -- 172.21.15.83:0/2693687986 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6720003cf0 con 0x7f672819a780 2024-04-24T09:03:35.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.502+0000 7f672d800640 1 -- 172.21.15.83:0/2693687986 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f672819b610 con 0x7f672819a780 2024-04-24T09:03:35.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.502+0000 7f6724c00640 1 -- 172.21.15.83:0/2693687986 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f67200044f0 con 0x7f672819a780 2024-04-24T09:03:35.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.502+0000 7f6724c00640 1 -- 172.21.15.83:0/2693687986 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6720030bf0 con 0x7f672819a780 2024-04-24T09:03:35.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.502+0000 7f672d800640 1 -- 172.21.15.83:0/2693687986 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67281a77b0 con 0x7f672819a780 2024-04-24T09:03:35.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.506+0000 7f6731c05740 1 Processor -- start 2024-04-24T09:03:35.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.506+0000 7f6724c00640 1 -- 172.21.15.83:0/2693687986 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6720004040 con 0x7f672819a780 2024-04-24T09:03:35.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.506+0000 7f6731c05740 1 -- start start 2024-04-24T09:03:35.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.506+0000 7f6731c05740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e79965fa30 0x55e79965fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:35.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.506+0000 7f6731c05740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e799660350 0x55e79966a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:35.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.506+0000 7f6731c05740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e79966aa20 0x55e79966ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:35.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.506+0000 7f6731c05740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e7995c51f0 con 0x55e79965fa30 2024-04-24T09:03:35.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.506+0000 7f6731c05740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e79956b350 con 0x55e79966aa20 2024-04-24T09:03:35.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.506+0000 7f6731c05740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55e7995c11b0 con 0x55e799660350 2024-04-24T09:03:35.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.507+0000 7f6727400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e79965fa30 0x55e79965fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:35.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.507+0000 7f6727400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e79965fa30 0x55e79965fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56594/0 (socket says 172.21.15.83:56594) 2024-04-24T09:03:35.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.507+0000 7f6727400640 1 -- 172.21.15.83:0/677726215 learned_addr learned my addr 172.21.15.83:0/677726215 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:03:35.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.507+0000 7f6727e00640 1 --2- 172.21.15.83:0/677726215 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e799660350 0x55e79966a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:35.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.507+0000 7f6727400640 1 -- 172.21.15.83:0/677726215 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e799660350 msgr2=0x55e79966a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:35.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.507+0000 7f6727400640 1 --2- 172.21.15.83:0/677726215 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e799660350 0x55e79966a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:35.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.507+0000 7f6727400640 1 -- 172.21.15.83:0/677726215 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e79966aa20 msgr2=0x55e79966ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:35.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.507+0000 7f6727400640 1 --2- 172.21.15.83:0/677726215 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e79966aa20 0x55e79966ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:35.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.507+0000 7f6727400640 1 -- 172.21.15.83:0/677726215 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7995bff50 con 0x55e79965fa30 2024-04-24T09:03:35.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.507+0000 7f6727e00640 1 --2- 172.21.15.83:0/677726215 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e799660350 0x55e79966a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:03:35.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.507+0000 7f6724c00640 1 --2- 172.21.15.83:0/2693687986 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f670c0a3d20 0x7f670c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:35.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.507+0000 7f6727400640 1 --2- 172.21.15.83:0/677726215 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e79965fa30 0x55e79965fe10 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7f6714009c20 tx=0x7f67140106e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6b7ca0fdd8a67d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:35.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.507+0000 7f6724c00640 1 -- 172.21.15.83:0/2693687986 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9583+0+0 (secure 0 0 0) 0x7f67200f2220 con 0x7f672819a780 2024-04-24T09:03:35.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.508+0000 7f6727e00640 1 --2- 172.21.15.83:0/2693687986 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f670c0a3d20 0x7f670c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:35.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.508+0000 7f671d600640 1 -- 172.21.15.83:0/677726215 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6714016070 con 0x55e79965fa30 2024-04-24T09:03:35.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.508+0000 7f671d600640 1 -- 172.21.15.83:0/677726215 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f67140128d0 con 0x55e79965fa30 2024-04-24T09:03:35.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.508+0000 7f671d600640 1 -- 172.21.15.83:0/677726215 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67140116c0 con 0x55e79965fa30 2024-04-24T09:03:35.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.509+0000 7f6731c05740 1 -- 172.21.15.83:0/677726215 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e79965fa30 msgr2=0x55e79965fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:35.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.509+0000 7f6731c05740 1 --2- 172.21.15.83:0/677726215 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e79965fa30 0x55e79965fe10 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7f6714009c20 tx=0x7f67140106e0 comp rx=0 tx=0).stop 2024-04-24T09:03:35.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.509+0000 7f6727e00640 1 --2- 172.21.15.83:0/2693687986 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f670c0a3d20 0x7f670c0a61e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x55e799660ff0 tx=0x7f6718005e90 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:35.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.509+0000 7f6731c05740 1 -- 172.21.15.83:0/677726215 shutdown_connections 2024-04-24T09:03:35.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.509+0000 7f6731c05740 1 --2- 172.21.15.83:0/677726215 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e79966aa20 0x55e79966ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:35.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.509+0000 7f6731c05740 1 --2- 172.21.15.83:0/677726215 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e799660350 0x55e79966a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:35.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.509+0000 7f6731c05740 1 --2- 172.21.15.83:0/677726215 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e79965fa30 0x55e79965fe10 unknown :-1 s=CLOSED pgs=1927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:35.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.509+0000 7f6731c05740 1 -- 172.21.15.83:0/677726215 >> 172.21.15.83:0/677726215 conn(0x55e79965bd50 msgr2=0x55e79965c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:35.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.509+0000 7f6731c05740 1 -- 172.21.15.83:0/677726215 shutdown_connections 2024-04-24T09:03:35.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.509+0000 7f6731c05740 1 -- 172.21.15.83:0/677726215 wait complete. 2024-04-24T09:03:35.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.509+0000 7f6731c05740 1 -- start start 2024-04-24T09:03:35.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.510+0000 7f6731c05740 1 Processor -- start 2024-04-24T09:03:35.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.510+0000 7f6731c05740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e79965fa30 0x55e7996679e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:35.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.510+0000 7f6731c05740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e799660350 0x55e799667f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:35.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.510+0000 7f6731c05740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e79966aa20 0x55e799668460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:35.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.510+0000 7f6731c05740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e79966d810 con 0x55e79966aa20 2024-04-24T09:03:35.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.510+0000 7f6731c05740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e79957e690 con 0x55e799660350 2024-04-24T09:03:35.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.510+0000 7f6731c05740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55e79966d990 con 0x55e79965fa30 2024-04-24T09:03:35.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.510+0000 7f6731c05740 10 client.0 _ll_register_callbacks cb 0x55e7995ba650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:03:35.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.510+0000 7f672ce00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e79966aa20 0x55e799668460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:35.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.510+0000 7f672ce00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e79966aa20 0x55e799668460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56600/0 (socket says 172.21.15.83:56600) 2024-04-24T09:03:35.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.510+0000 7f672ce00640 1 -- 172.21.15.83:0/1671342286 learned_addr learned my addr 172.21.15.83:0/1671342286 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:03:35.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.511+0000 7f6727400640 1 --2- 172.21.15.83:0/1671342286 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e799660350 0x55e799667f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:35.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.511+0000 7f6727e00640 1 --2- 172.21.15.83:0/1671342286 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e79965fa30 0x55e7996679e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:35.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.511+0000 7f672ce00640 1 -- 172.21.15.83:0/1671342286 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e79965fa30 msgr2=0x55e7996679e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:35.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.511+0000 7f672ce00640 1 --2- 172.21.15.83:0/1671342286 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e79965fa30 0x55e7996679e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:35.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.511+0000 7f672ce00640 1 -- 172.21.15.83:0/1671342286 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e799660350 msgr2=0x55e799667f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:35.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.511+0000 7f672ce00640 1 --2- 172.21.15.83:0/1671342286 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e799660350 0x55e799667f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:35.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.511+0000 7f672ce00640 1 -- 172.21.15.83:0/1671342286 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e79957d920 con 0x55e79966aa20 2024-04-24T09:03:35.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.511+0000 7f6727e00640 1 --2- 172.21.15.83:0/1671342286 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e79965fa30 0x55e7996679e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:03:35.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.511+0000 7f6727400640 1 --2- 172.21.15.83:0/1671342286 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e799660350 0x55e799667f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:03:35.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.511+0000 7f672ce00640 1 --2- 172.21.15.83:0/1671342286 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e79966aa20 0x55e799668460 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7f672002f3b0 tx=0x7f6720004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3f80e5d7510c552 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:35.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.511+0000 7f6712000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:03:35.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.511+0000 7f6712000640 1 -- 172.21.15.83:0/1671342286 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67200d7060 con 0x55e79966aa20 2024-04-24T09:03:35.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.511+0000 7f6712000640 1 -- 172.21.15.83:0/1671342286 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9583+0+0 (secure 0 0 0) 0x7f67200046a0 con 0x55e79966aa20 2024-04-24T09:03:35.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.511+0000 7f6731c05740 1 -- 172.21.15.83:0/1671342286 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67140128d0 con 0x55e79966aa20 2024-04-24T09:03:35.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.512+0000 7f6712000640 1 -- 172.21.15.83:0/1671342286 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f67200c2260 con 0x55e79966aa20 2024-04-24T09:03:35.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.512+0000 7f6731c05740 10 client.19944 fetch_fsmap learned FSMap version 41 2024-04-24T09:03:35.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.512+0000 7f6731c05740 1 -- 172.21.15.83:0/1671342286 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55e7993a17d0 con 0x55e79966aa20 2024-04-24T09:03:35.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.512+0000 7f6712000640 1 -- 172.21.15.83:0/1671342286 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f67140128d0 con 0x55e79966aa20 2024-04-24T09:03:35.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.512+0000 7f6712000640 1 -- 172.21.15.83:0/1671342286 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7f67200d7200 con 0x55e79966aa20 2024-04-24T09:03:35.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.512+0000 7f6731c05740 10 client.19944 fetch_fsmap finished waiting for FSMap version 41 2024-04-24T09:03:35.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.512+0000 7f6731c05740 10 client.19944 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:03:35.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.512+0000 7f6731c05740 20 client.19944 populate_metadata read hostname 'smithi083' 2024-04-24T09:03:35.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.512+0000 7f6731c05740 1 --2- 172.21.15.83:0/1671342286 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55e79967ad70 0x55e79967d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:35.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.512+0000 7f6731c05740 4 client.19944 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:03:35.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.512+0000 7f6731c05740 1 -- 172.21.15.83:0/1671342286 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e7995d2c10 con 0x55e79967ad70 2024-04-24T09:03:35.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.513+0000 7f6727e00640 1 --2- 172.21.15.83:0/1671342286 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55e79967ad70 0x55e79967d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:35.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.514+0000 7f6727e00640 1 --2- 172.21.15.83:0/1671342286 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55e79967ad70 0x55e79967d1f0 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7112c1de48a2a20c server_cookie=81eef1fe54e93e3e in_seq=0 out_seq=0 2024-04-24T09:03:35.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.514+0000 7f6712000640 10 client.19944 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:03:35.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.516+0000 7f6712000640 1 -- 172.21.15.83:0/1671342286 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e7995d2c10 con 0x55e79967ad70 2024-04-24T09:03:35.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.516+0000 7f6712000640 10 client.19944 handle_command_reply: tid=0 2024-04-24T09:03:35.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.516+0000 7f6731c05740 1 client.19944 shutdown 2024-04-24T09:03:35.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.517+0000 7f6731c05740 1 -- 172.21.15.83:0/1671342286 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e79966aa20 msgr2=0x55e799668460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:35.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.517+0000 7f6731c05740 1 --2- 172.21.15.83:0/1671342286 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e79966aa20 0x55e799668460 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7f672002f3b0 tx=0x7f6720004470 comp rx=0 tx=0).stop 2024-04-24T09:03:35.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.517+0000 7f6731c05740 1 -- 172.21.15.83:0/1671342286 shutdown_connections 2024-04-24T09:03:35.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.517+0000 7f6731c05740 1 --2- 172.21.15.83:0/1671342286 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55e79967ad70 0x55e79967d1f0 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:35.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.517+0000 7f6731c05740 1 --2- 172.21.15.83:0/1671342286 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e79966aa20 0x55e799668460 unknown :-1 s=CLOSED pgs=1928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:35.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.517+0000 7f6731c05740 1 --2- 172.21.15.83:0/1671342286 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e799660350 0x55e799667f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:35.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.517+0000 7f6731c05740 1 --2- 172.21.15.83:0/1671342286 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e79965fa30 0x55e7996679e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:35.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.517+0000 7f6712000640 0 client.19944 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:03:35.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.517+0000 7f6731c05740 1 -- 172.21.15.83:0/1671342286 >> 172.21.15.83:0/1671342286 conn(0x55e79965bd50 msgr2=0x55e79965afd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:35.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.517+0000 7f6731c05740 1 -- 172.21.15.83:0/1671342286 shutdown_connections 2024-04-24T09:03:35.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.517+0000 7f6731c05740 1 -- 172.21.15.83:0/1671342286 wait complete. 2024-04-24T09:03:35.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.517+0000 7f6731c05740 20 client.19944 trim_cache size 0 max 16384 2024-04-24T09:03:35.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.537+0000 7f6731c05740 1 Processor -- start 2024-04-24T09:03:35.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.537+0000 7f6731c05740 1 -- start start 2024-04-24T09:03:35.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.538+0000 7f6731c05740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e79965fa30 0x55e799676700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:35.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.538+0000 7f6731c05740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e799660350 0x55e79965b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:35.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.538+0000 7f6731c05740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e79966aa20 0x55e7996680b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:35.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.538+0000 7f6731c05740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f67281b15b0 con 0x55e79966aa20 2024-04-24T09:03:35.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.538+0000 7f6731c05740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f67281b3210 con 0x55e799660350 2024-04-24T09:03:35.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.538+0000 7f6731c05740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f67281b1c80 con 0x55e79965fa30 2024-04-24T09:03:35.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.538+0000 7f672ce00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e79966aa20 0x55e7996680b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:35.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.538+0000 7f672ce00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e79966aa20 0x55e7996680b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56610/0 (socket says 172.21.15.83:56610) 2024-04-24T09:03:35.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.538+0000 7f672ce00640 1 -- 172.21.15.83:0/2003041457 learned_addr learned my addr 172.21.15.83:0/2003041457 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:03:35.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.538+0000 7f6727400640 1 --2- 172.21.15.83:0/2003041457 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e799660350 0x55e79965b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:35.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.538+0000 7f672ce00640 1 -- 172.21.15.83:0/2003041457 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e79965fa30 msgr2=0x55e799676700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:35.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.538+0000 7f672ce00640 1 --2- 172.21.15.83:0/2003041457 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e79965fa30 0x55e799676700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:35.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.538+0000 7f672ce00640 1 -- 172.21.15.83:0/2003041457 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e799660350 msgr2=0x55e79965b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:35.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.538+0000 7f672ce00640 1 --2- 172.21.15.83:0/2003041457 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e799660350 0x55e79965b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:35.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.538+0000 7f672ce00640 1 -- 172.21.15.83:0/2003041457 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7992d73f0 con 0x55e79966aa20 2024-04-24T09:03:35.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.538+0000 7f6727400640 1 --2- 172.21.15.83:0/2003041457 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e799660350 0x55e79965b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:03:35.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.539+0000 7f672ce00640 1 --2- 172.21.15.83:0/2003041457 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e79966aa20 0x55e7996680b0 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7f6720005e30 tx=0x7f67200bdca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2103e3bc88a9ef27 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:35.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.539+0000 7f671cc00640 1 -- 172.21.15.83:0/2003041457 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67200ba150 con 0x55e79966aa20 2024-04-24T09:03:35.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.539+0000 7f671cc00640 1 -- 172.21.15.83:0/2003041457 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6720036040 con 0x55e79966aa20 2024-04-24T09:03:35.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.539+0000 7f671cc00640 1 -- 172.21.15.83:0/2003041457 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67200046a0 con 0x55e79966aa20 2024-04-24T09:03:35.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.540+0000 7f6731c05740 1 -- 172.21.15.83:0/2003041457 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e79966aa20 msgr2=0x55e7996680b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:35.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.540+0000 7f6731c05740 1 --2- 172.21.15.83:0/2003041457 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e79966aa20 0x55e7996680b0 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7f6720005e30 tx=0x7f67200bdca0 comp rx=0 tx=0).stop 2024-04-24T09:03:35.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.540+0000 7f6731c05740 1 -- 172.21.15.83:0/2003041457 shutdown_connections 2024-04-24T09:03:35.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.540+0000 7f6731c05740 1 --2- 172.21.15.83:0/2003041457 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e79966aa20 0x55e7996680b0 unknown :-1 s=CLOSED pgs=1929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:35.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.540+0000 7f6731c05740 1 --2- 172.21.15.83:0/2003041457 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e799660350 0x55e79965b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:35.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.540+0000 7f6731c05740 1 --2- 172.21.15.83:0/2003041457 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e79965fa30 0x55e799676700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:35.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.540+0000 7f6731c05740 1 -- 172.21.15.83:0/2003041457 >> 172.21.15.83:0/2003041457 conn(0x55e79965bd50 msgr2=0x55e7996679a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:35.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.540+0000 7f6731c05740 1 -- 172.21.15.83:0/2003041457 shutdown_connections 2024-04-24T09:03:35.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.540+0000 7f6731c05740 1 -- 172.21.15.83:0/2003041457 wait complete. 2024-04-24T09:03:35.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.540+0000 7f6731c05740 1 -- start start 2024-04-24T09:03:35.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.541+0000 7f6731c05740 1 Processor -- start 2024-04-24T09:03:35.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.541+0000 7f6731c05740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e79965fa30 0x55e799664850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:35.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.541+0000 7f6731c05740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e799660350 0x55e799664d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:35.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.541+0000 7f6731c05740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e79966aa20 0x55e7996652d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:35.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.541+0000 7f6731c05740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f67281b2ae0 con 0x55e799660350 2024-04-24T09:03:35.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.541+0000 7f6731c05740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f67281b23b0 con 0x55e79966aa20 2024-04-24T09:03:35.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.541+0000 7f6731c05740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f67281af5b0 con 0x55e79965fa30 2024-04-24T09:03:35.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.541+0000 7f6731c05740 10 client.0 _ll_register_callbacks cb 0x7f6720009900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:03:35.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.541+0000 7f6727400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e799660350 0x55e799664d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:35.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.542+0000 7f6727400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e799660350 0x55e799664d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56620/0 (socket says 172.21.15.83:56620) 2024-04-24T09:03:35.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.542+0000 7f6727400640 1 -- 172.21.15.83:0/582683023 learned_addr learned my addr 172.21.15.83:0/582683023 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:03:35.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.542+0000 7f6727e00640 1 --2- 172.21.15.83:0/582683023 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e79965fa30 0x55e799664850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:35.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.542+0000 7f672ce00640 1 --2- 172.21.15.83:0/582683023 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e79966aa20 0x55e7996652d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:35.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.542+0000 7f6727400640 1 -- 172.21.15.83:0/582683023 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e79965fa30 msgr2=0x55e799664850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:35.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.542+0000 7f6727400640 1 --2- 172.21.15.83:0/582683023 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e79965fa30 0x55e799664850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:35.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.542+0000 7f6727400640 1 -- 172.21.15.83:0/582683023 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e79966aa20 msgr2=0x55e7996652d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:35.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.542+0000 7f6727400640 1 --2- 172.21.15.83:0/582683023 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e79966aa20 0x55e7996652d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:35.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.542+0000 7f6727400640 1 -- 172.21.15.83:0/582683023 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7992970f0 con 0x55e799660350 2024-04-24T09:03:35.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.542+0000 7f6727400640 1 --2- 172.21.15.83:0/582683023 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e799660350 0x55e799664d90 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7f6714009bc0 tx=0x7f67140099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e329a4e7f981d00 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:35.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.542+0000 7f6712000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:03:35.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.543+0000 7f6712000640 1 -- 172.21.15.83:0/582683023 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6714016040 con 0x55e799660350 2024-04-24T09:03:35.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.543+0000 7f6712000640 1 -- 172.21.15.83:0/582683023 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9583+0+0 (secure 0 0 0) 0x7f6714012420 con 0x55e799660350 2024-04-24T09:03:35.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.543+0000 7f6731c05740 1 -- 172.21.15.83:0/582683023 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6720036040 con 0x55e799660350 2024-04-24T09:03:35.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.543+0000 7f6712000640 1 -- 172.21.15.83:0/582683023 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f6714012620 con 0x55e799660350 2024-04-24T09:03:35.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.543+0000 7f6731c05740 10 client.19950 fetch_fsmap learned FSMap version 41 2024-04-24T09:03:35.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.543+0000 7f6731c05740 1 -- 172.21.15.83:0/582683023 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55e79958a810 con 0x55e799660350 2024-04-24T09:03:35.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.543+0000 7f6712000640 1 -- 172.21.15.83:0/582683023 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6720036040 con 0x55e799660350 2024-04-24T09:03:35.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.543+0000 7f6712000640 1 -- 172.21.15.83:0/582683023 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7f671401ba70 con 0x55e799660350 2024-04-24T09:03:35.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.543+0000 7f6731c05740 10 client.19950 fetch_fsmap finished waiting for FSMap version 41 2024-04-24T09:03:35.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.543+0000 7f6731c05740 10 client.19950 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:03:35.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.544+0000 7f6731c05740 20 client.19950 populate_metadata read hostname 'smithi083' 2024-04-24T09:03:35.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.544+0000 7f6731c05740 1 --2- 172.21.15.83:0/582683023 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55e7996e1a40 0x55e7996e3e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:35.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.544+0000 7f6731c05740 4 client.19950 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T09:03:35.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.544+0000 7f6731c05740 1 -- 172.21.15.83:0/582683023 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e7995cca80 con 0x55e7996e1a40 2024-04-24T09:03:35.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.545+0000 7f6727e00640 1 --2- 172.21.15.83:0/582683023 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55e7996e1a40 0x55e7996e3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:35.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.546+0000 7f6727e00640 1 --2- 172.21.15.83:0/582683023 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55e7996e1a40 0x55e7996e3e60 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8070d6a05e74398a server_cookie=df644dd3e6d570ea in_seq=0 out_seq=0 2024-04-24T09:03:35.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.546+0000 7f6712000640 10 client.19950 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:03:35.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.548+0000 7f6712000640 1 -- 172.21.15.83:0/582683023 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55e7995cca80 con 0x55e7996e1a40 2024-04-24T09:03:35.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.548+0000 7f6712000640 10 client.19950 handle_command_reply: tid=0 2024-04-24T09:03:35.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.548+0000 7f6731c05740 1 client.19950 shutdown 2024-04-24T09:03:35.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.548+0000 7f6731c05740 1 -- 172.21.15.83:0/582683023 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e799660350 msgr2=0x55e799664d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:35.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.548+0000 7f6731c05740 1 --2- 172.21.15.83:0/582683023 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e799660350 0x55e799664d90 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7f6714009bc0 tx=0x7f67140099e0 comp rx=0 tx=0).stop 2024-04-24T09:03:35.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.548+0000 7f6731c05740 1 -- 172.21.15.83:0/582683023 shutdown_connections 2024-04-24T09:03:35.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.548+0000 7f6731c05740 1 --2- 172.21.15.83:0/582683023 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55e7996e1a40 0x55e7996e3e60 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:35.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.548+0000 7f6731c05740 1 --2- 172.21.15.83:0/582683023 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e79966aa20 0x55e7996652d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:35.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.548+0000 7f6731c05740 1 --2- 172.21.15.83:0/582683023 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e799660350 0x55e799664d90 unknown :-1 s=CLOSED pgs=1930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:35.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.548+0000 7f6731c05740 1 --2- 172.21.15.83:0/582683023 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e79965fa30 0x55e799664850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:35.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.548+0000 7f6712000640 0 client.19950 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:03:35.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.548+0000 7f6731c05740 1 -- 172.21.15.83:0/582683023 >> 172.21.15.83:0/582683023 conn(0x55e79965bd50 msgr2=0x55e799668230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:35.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.549+0000 7f6731c05740 1 -- 172.21.15.83:0/582683023 shutdown_connections 2024-04-24T09:03:35.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.549+0000 7f6731c05740 1 -- 172.21.15.83:0/582683023 wait complete. 2024-04-24T09:03:35.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.549+0000 7f6731c05740 20 client.19950 trim_cache size 0 max 16384 2024-04-24T09:03:35.550 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T09:03:35.550 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T09:03:35.550 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-04-24T09:03:35.550 INFO:teuthology.orchestra.run.smithi083.stdout: "7404d51c-e420-4277-ba1f-9d551c5c7f66": { 2024-04-24T09:03:35.550 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-04-24T09:03:35.550 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "7404d51c-e420-4277-ba1f-9d551c5c7f66", 2024-04-24T09:03:35.550 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-04-24T09:03:35.550 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:03:35.550 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:03:35.550 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T09:03:35.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.549+0000 7f672d800640 1 -- 172.21.15.83:0/2693687986 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f670c0a3d20 msgr2=0x7f670c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:35.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.549+0000 7f672d800640 1 --2- 172.21.15.83:0/2693687986 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f670c0a3d20 0x7f670c0a61e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x55e799660ff0 tx=0x7f6718005e90 comp rx=0 tx=0).stop 2024-04-24T09:03:35.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.549+0000 7f672d800640 1 -- 172.21.15.83:0/2693687986 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f672819a780 msgr2=0x7f672819ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:35.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.549+0000 7f672d800640 1 --2- 172.21.15.83:0/2693687986 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f672819a780 0x7f672819ac00 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7f6720009dc0 tx=0x7f6720032ff0 comp rx=0 tx=0).stop 2024-04-24T09:03:35.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.550+0000 7f672d800640 1 -- 172.21.15.83:0/2693687986 shutdown_connections 2024-04-24T09:03:35.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.550+0000 7f672d800640 1 --2- 172.21.15.83:0/2693687986 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f670c0a3d20 0x7f670c0a61e0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:35.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.550+0000 7f672d800640 1 --2- 172.21.15.83:0/2693687986 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f672819a780 0x7f672819ac00 unknown :-1 s=CLOSED pgs=1926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:35.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.550+0000 7f672d800640 1 --2- 172.21.15.83:0/2693687986 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f67281093f0 0x7f67281a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:35.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.550+0000 7f672d800640 1 --2- 172.21.15.83:0/2693687986 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6728104320 0x7f67281a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:35.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.550+0000 7f672d800640 1 -- 172.21.15.83:0/2693687986 >> 172.21.15.83:0/2693687986 conn(0x7f67280ffb40 msgr2=0x7f6728100e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:35.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.550+0000 7f672d800640 1 -- 172.21.15.83:0/2693687986 shutdown_connections 2024-04-24T09:03:35.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:03:35.550+0000 7f672d800640 1 -- 172.21.15.83:0/2693687986 wait complete. 2024-04-24T09:03:35.567 INFO:tasks.cephfs.filesystem:scrub status for tag:7404d51c-e420-4277-ba1f-9d551c5c7f66 - {'path': '/', 'tag': '7404d51c-e420-4277-ba1f-9d551c5c7f66', 'options': 'recursive,force'} 2024-04-24T09:03:35.620 INFO:tasks.workunit.client.0.smithi083.stdout: 1 69346 0.60 MB/sec execute 489 sec latency 58.529 ms 2024-04-24T09:03:35.705 INFO:journalctl@ceph.mds.b.smithi138.stdout:Apr 24 09:03:35 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-b[61515]: 2024-04-24T09:03:35.301+0000 7f7fbbc00700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000020c [...14,head] /volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/EXCEL/ auth{3=1} v11956 ap=2 f(v698 m2024-04-24T09:03:24.239339+0000 2=2+0) n(v52 rc2024-04-24T09:03:24.239339+0000 b520192 3=2+1) old_inodes=6 (inest sync->mix g=3) caps={15603=pAsLsXsFs/-@10772} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x55cf8760a580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000020c:[<0x10000000009/EXCEL v347>,<0x10000000008/~dmtmp v57>,<0x10000000007/client0 v33>,<0x10000000006/clients v23>,<0x10000000005/tmp v23>,<0x10000000003/client.0 v22>,<0x10000000002/e395bed6-5bba-41b8-a256-3505e4afcca2 v36>,<0x10000000001/sv_0 v55>,<0x10000000000/qa v43>,<0x1/volumes v39>]//[]","memoryvalue":"(2)0x1000000020c:[<0x10000000009/EXCEL v11956>,<0x10000000008/~dmtmp v11303>,<0x10000000007/client0 v1341>,<0x10000000006/clients v1332>,<0x10000000005/tmp v1314>,<0x10000000003/client.0 v1309>,<0x10000000002/e395bed6-5bba-41b8-a256-3505e4afcca2 v1303>,<0x10000000001/sv_0 v1297>,<0x10000000000/qa v1278>,<0x1/volumes v1240>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-04-24T09:03:24.199347+0000 3=3+0)","ondisk_value.rstat":"n(v0 rc2024-04-24T09:03:24.239339+0000 b520192 3=2+1)","memory_value.dirstat":"f(v698 m2024-04-24T09:03:24.239339+0000 2=2+0)","memory_value.rstat":"n(v52 rc2024-04-24T09:03:24.239339+0000 b520192 3=2+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-04-24T09:03:35.705 INFO:journalctl@ceph.mds.b.smithi138.stdout:Apr 24 09:03:35 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-b[61515]: 2024-04-24T09:03:35.304+0000 7f7fbbc00700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000221 [...14,head] /volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/WORDPRO/ auth{3=1} v11960 ap=2 f(v665 m2024-04-24T09:03:24.805628+0000 3=3+0) n(v45 rc2024-04-24T09:03:24.805628+0000 b780288 4=3+1) old_inodes=6 (inest sync->mix g=3) caps={15603=pAsLsXsFs/-@10614} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x55cf87627700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000000221:[<0x10000000009/WORDPRO v375>,<0x10000000008/~dmtmp v57>,<0x10000000007/client0 v33>,<0x10000000006/clients v23>,<0x10000000005/tmp v23>,<0x10000000003/client.0 v22>,<0x10000000002/e395bed6-5bba-41b8-a256-3505e4afcca2 v36>,<0x10000000001/sv_0 v55>,<0x10000000000/qa v43>,<0x1/volumes v39>]//[]","memoryvalue":"(2)0x10000000221:[<0x10000000009/WORDPRO v11960>,<0x10000000008/~dmtmp v11303>,<0x10000000007/client0 v1341>,<0x10000000006/clients v1332>,<0x10000000005/tmp v1314>,<0x10000000003/client.0 v1309>,<0x10000000002/e395bed6-5bba-41b8-a256-3505e4afcca2 v1303>,<0x10000000001/sv_0 v1297>,<0x10000000000/qa v1278>,<0x1/volumes v1240>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-04-24T09:03:24.766737+0000 4=4+0)","ondisk_value.rstat":"n(v0 rc2024-04-24T09:03:24.805628+0000 b780288 4=3+1)","memory_value.dirstat":"f(v665 m2024-04-24T09:03:24.805628+0000 3=3+0)","memory_value.rstat":"n(v45 rc2024-04-24T09:03:24.805628+0000 b780288 4=3+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-04-24T09:03:35.706 INFO:journalctl@ceph.mds.b.smithi138.stdout:Apr 24 09:03:35 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-b[61515]: 2024-04-24T09:03:35.331+0000 7f7fbbc00700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000023e [...14,head] /volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/ACCESS/ auth{3=1} v11966 ap=2 f(v266 m2024-04-24T09:03:25.193864+0000 1=1+0) n(v43 rc2024-04-24T09:03:25.193864+0000 b260096 2=1+1) old_inodes=6 (inest sync->mix g=3) caps={15603=pAsLsXsFs/-@9050} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x55cf87666b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000023e:[<0x10000000009/ACCESS v463>,<0x10000000008/~dmtmp v57>,<0x10000000007/client0 v33>,<0x10000000006/clients v23>,<0x10000000005/tmp v23>,<0x10000000003/client.0 v22>,<0x10000000002/e395bed6-5bba-41b8-a256-3505e4afcca2 v36>,<0x10000000001/sv_0 v55>,<0x10000000000/qa v43>,<0x1/volumes v39>]//[]","memoryvalue":"(2)0x1000000023e:[<0x10000000009/ACCESS v11966>,<0x10000000008/~dmtmp v11311>,<0x10000000007/client0 v1341>,<0x10000000006/clients v1332>,<0x10000000005/tmp v1314>,<0x10000000003/client.0 v1309>,<0x10000000002/e395bed6-5bba-41b8-a256-3505e4afcca2 v1303>,<0x10000000001/sv_0 v1297>,<0x10000000000/qa v1278>,<0x1/volumes v1240>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-04-24T09:03:25.155346+0000 2=2+0)","ondisk_value.rstat":"n(v0 rc2024-04-24T09:03:25.193864+0000 b260096 2=1+1)","memory_value.dirstat":"f(v266 m2024-04-24T09:03:25.193864+0000 1=1+0)","memory_value.rstat":"n(v43 rc2024-04-24T09:03:25.193864+0000 b260096 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-04-24T09:03:36.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:03:36 smithi138 ceph-mon[31441]: pgmap v376: 129 pgs: 129 active+clean; 62 MiB data, 885 MiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 317 KiB/s wr, 55 op/s 2024-04-24T09:03:36.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:03:36 smithi138 ceph-mon[31441]: scrub queued for path: / 2024-04-24T09:03:36.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:03:36 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:03:36.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:03:36 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:03:36.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:03:36 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:03:36.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:03:36 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:03:36.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:03:36 smithi138 ceph-mon[31441]: Scrub error on inode 0x1000000020c (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/EXCEL) see mds.b log and `damage ls` output for details 2024-04-24T09:03:36.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:03:36 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:03:36.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:03:36 smithi138 ceph-mon[31441]: Scrub error on inode 0x10000000221 (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/WORDPRO) see mds.b log and `damage ls` output for details 2024-04-24T09:03:36.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:03:36 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:03:36.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:03:36 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:03:36.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:03:36 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:03:36.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:03:36 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:03:36.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:03:36 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:03:36.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:03:36 smithi138 ceph-mon[31441]: Scrub error on inode 0x1000000023e (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/ACCESS) see mds.b log and `damage ls` output for details 2024-04-24T09:03:36.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:03:36 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:03:36.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:03:36 smithi189 ceph-mon[31791]: pgmap v376: 129 pgs: 129 active+clean; 62 MiB data, 885 MiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 317 KiB/s wr, 55 op/s 2024-04-24T09:03:36.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:03:36 smithi189 ceph-mon[31791]: scrub queued for path: / 2024-04-24T09:03:36.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:03:36 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:03:36.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:03:36 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:03:36.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:03:36 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:03:36.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:03:36 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:03:36.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:03:36 smithi189 ceph-mon[31791]: Scrub error on inode 0x1000000020c (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/EXCEL) see mds.b log and `damage ls` output for details 2024-04-24T09:03:36.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:03:36 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:03:36.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:03:36 smithi189 ceph-mon[31791]: Scrub error on inode 0x10000000221 (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/WORDPRO) see mds.b log and `damage ls` output for details 2024-04-24T09:03:36.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:03:36 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:03:36.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:03:36 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:03:36.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:03:36 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:03:36.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:03:36 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:03:36.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:03:36 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:03:36.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:03:36 smithi189 ceph-mon[31791]: Scrub error on inode 0x1000000023e (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/ACCESS) see mds.b log and `damage ls` output for details 2024-04-24T09:03:36.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:03:36 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:03:36.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:03:36 smithi083 ceph-mon[27907]: pgmap v376: 129 pgs: 129 active+clean; 62 MiB data, 885 MiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 317 KiB/s wr, 55 op/s 2024-04-24T09:03:36.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:03:36 smithi083 ceph-mon[27907]: scrub queued for path: / 2024-04-24T09:03:36.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:03:36 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:03:36.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:03:36 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:03:36.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:03:36 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:03:36.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:03:36 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:03:36.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:03:36 smithi083 ceph-mon[27907]: Scrub error on inode 0x1000000020c (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/EXCEL) see mds.b log and `damage ls` output for details 2024-04-24T09:03:36.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:03:36 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:03:36.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:03:36 smithi083 ceph-mon[27907]: Scrub error on inode 0x10000000221 (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/WORDPRO) see mds.b log and `damage ls` output for details 2024-04-24T09:03:36.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:03:36 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:03:36.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:03:36 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:03:36.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:03:36 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:03:36.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:03:36 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:03:36.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:03:36 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:03:36.555 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:03:36 smithi083 ceph-mon[27907]: Scrub error on inode 0x1000000023e (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/ACCESS) see mds.b log and `damage ls` output for details 2024-04-24T09:03:36.555 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:03:36 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:03:36.621 INFO:tasks.workunit.client.0.smithi083.stdout: 1 69495 0.60 MB/sec execute 490 sec latency 62.937 ms 2024-04-24T09:03:37.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:03:37 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:03:37.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:03:37 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:03:37.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:03:37 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:03:37.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:03:37 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:03:37.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:03:37 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:03:37.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:03:37 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:03:37.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:03:37 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:03:37.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:03:37 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:03:37.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:03:37 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:03:37.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:03:37 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:03:37.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:03:37 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:03:37.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:03:37 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:03:37.620 INFO:tasks.workunit.client.0.smithi083.stdout: 1 69620 0.60 MB/sec execute 491 sec latency 70.672 ms 2024-04-24T09:03:38.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:03:38 smithi138 ceph-mon[31441]: pgmap v377: 129 pgs: 129 active+clean; 62 MiB data, 885 MiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 317 KiB/s wr, 55 op/s 2024-04-24T09:03:38.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:03:38 smithi189 ceph-mon[31791]: pgmap v377: 129 pgs: 129 active+clean; 62 MiB data, 885 MiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 317 KiB/s wr, 55 op/s 2024-04-24T09:03:38.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:03:38 smithi083 ceph-mon[27907]: pgmap v377: 129 pgs: 129 active+clean; 62 MiB data, 885 MiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 317 KiB/s wr, 55 op/s 2024-04-24T09:03:38.620 INFO:tasks.workunit.client.0.smithi083.stdout: 1 69761 0.60 MB/sec execute 492 sec latency 49.877 ms 2024-04-24T09:03:39.621 INFO:tasks.workunit.client.0.smithi083.stdout: 1 69849 0.60 MB/sec execute 493 sec latency 51.472 ms 2024-04-24T09:03:40.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:03:40 smithi138 ceph-mon[31441]: pgmap v378: 129 pgs: 129 active+clean; 63 MiB data, 887 MiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 395 KiB/s wr, 82 op/s 2024-04-24T09:03:40.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:03:40 smithi189 ceph-mon[31791]: pgmap v378: 129 pgs: 129 active+clean; 63 MiB data, 887 MiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 395 KiB/s wr, 82 op/s 2024-04-24T09:03:40.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:03:40 smithi083 ceph-mon[27907]: pgmap v378: 129 pgs: 129 active+clean; 63 MiB data, 887 MiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 395 KiB/s wr, 82 op/s 2024-04-24T09:03:40.621 INFO:tasks.workunit.client.0.smithi083.stdout: 1 69968 0.60 MB/sec execute 494 sec latency 56.057 ms 2024-04-24T09:03:41.621 INFO:tasks.workunit.client.0.smithi083.stdout: 1 70037 0.60 MB/sec execute 495 sec latency 52.315 ms 2024-04-24T09:03:42.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:03:42 smithi138 ceph-mon[31441]: pgmap v379: 129 pgs: 129 active+clean; 63 MiB data, 887 MiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 269 KiB/s wr, 55 op/s 2024-04-24T09:03:42.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:03:42 smithi189 ceph-mon[31791]: pgmap v379: 129 pgs: 129 active+clean; 63 MiB data, 887 MiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 269 KiB/s wr, 55 op/s 2024-04-24T09:03:42.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:03:42 smithi083 ceph-mon[27907]: pgmap v379: 129 pgs: 129 active+clean; 63 MiB data, 887 MiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 269 KiB/s wr, 55 op/s 2024-04-24T09:03:42.621 INFO:tasks.workunit.client.0.smithi083.stdout: 1 70317 0.60 MB/sec execute 496 sec latency 50.404 ms 2024-04-24T09:03:43.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:03:43 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:03:43.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:03:43 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:03:43.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:03:43 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:03:43.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:03:43 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:03:43.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:03:43 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:03:43.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:03:43 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:03:43.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:03:43 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:03:43.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:03:43 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:03:43.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:03:43 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:03:43.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:03:43 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:03:43.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:03:43 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:03:43.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:03:43 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:03:43.621 INFO:tasks.workunit.client.0.smithi083.stdout: 1 70430 0.60 MB/sec execute 497 sec latency 55.216 ms 2024-04-24T09:03:44.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:03:44 smithi138 ceph-mon[31441]: pgmap v380: 129 pgs: 129 active+clean; 65 MiB data, 899 MiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 498 KiB/s wr, 76 op/s 2024-04-24T09:03:44.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:03:44 smithi189 ceph-mon[31791]: pgmap v380: 129 pgs: 129 active+clean; 65 MiB data, 899 MiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 498 KiB/s wr, 76 op/s 2024-04-24T09:03:44.552 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:03:44 smithi083 ceph-mon[27907]: pgmap v380: 129 pgs: 129 active+clean; 65 MiB data, 899 MiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 498 KiB/s wr, 76 op/s 2024-04-24T09:03:44.564 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.564+0000 7ff2e5000640 1 -- 172.21.15.83:0/670763202 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2e0104450 msgr2=0x7ff2e0104830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:44.564 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.564+0000 7ff2e5000640 1 --2- 172.21.15.83:0/670763202 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2e0104450 0x7ff2e0104830 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7ff2d0009900 tx=0x7ff2d002a3f0 comp rx=0 tx=0).stop 2024-04-24T09:03:44.564 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.564+0000 7ff2e5000640 1 -- 172.21.15.83:0/670763202 shutdown_connections 2024-04-24T09:03:44.564 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.564+0000 7ff2e5000640 1 --2- 172.21.15.83:0/670763202 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff2e0109520 0x7ff2e01110f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:44.564 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.564+0000 7ff2e5000640 1 --2- 172.21.15.83:0/670763202 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff2e0104e00 0x7ff2e0108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:44.564 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.564+0000 7ff2e5000640 1 --2- 172.21.15.83:0/670763202 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2e0104450 0x7ff2e0104830 unknown :-1 s=CLOSED pgs=1931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:44.564 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.564+0000 7ff2e5000640 1 -- 172.21.15.83:0/670763202 >> 172.21.15.83:0/670763202 conn(0x7ff2e00ffcf0 msgr2=0x7ff2e0102110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:44.564 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.565+0000 7ff2e5000640 1 -- 172.21.15.83:0/670763202 shutdown_connections 2024-04-24T09:03:44.564 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.565+0000 7ff2e5000640 1 -- 172.21.15.83:0/670763202 wait complete. 2024-04-24T09:03:44.566 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.566+0000 7ff2e5000640 1 Processor -- start 2024-04-24T09:03:44.566 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.566+0000 7ff2e5000640 1 -- start start 2024-04-24T09:03:44.566 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.566+0000 7ff2e5000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff2e0104450 0x7ff2e01a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:44.566 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.566+0000 7ff2e5000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff2e0109520 0x7ff2e01a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:44.566 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.566+0000 7ff2e5000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2e019a780 0x7ff2e019ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:44.566 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.566+0000 7ff2e5000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e0114460 con 0x7ff2e019a780 2024-04-24T09:03:44.566 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.566+0000 7ff2e5000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e01142e0 con 0x7ff2e0109520 2024-04-24T09:03:44.566 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.566+0000 7ff2e5000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e01145e0 con 0x7ff2e0104450 2024-04-24T09:03:44.566 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.567+0000 7ff2de800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2e019a780 0x7ff2e019ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:44.566 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.567+0000 7ff2de800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2e019a780 0x7ff2e019ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47910/0 (socket says 172.21.15.83:47910) 2024-04-24T09:03:44.566 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.567+0000 7ff2de800640 1 -- 172.21.15.83:0/3405384109 learned_addr learned my addr 172.21.15.83:0/3405384109 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:03:44.566 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.567+0000 7ff2dde00640 1 --2- 172.21.15.83:0/3405384109 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff2e0104450 0x7ff2e01a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:44.567 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.567+0000 7ff2dd400640 1 --2- 172.21.15.83:0/3405384109 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff2e0109520 0x7ff2e01a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:44.567 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.567+0000 7ff2de800640 1 -- 172.21.15.83:0/3405384109 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff2e0104450 msgr2=0x7ff2e01a0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:44.567 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.567+0000 7ff2de800640 1 --2- 172.21.15.83:0/3405384109 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff2e0104450 0x7ff2e01a0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:44.567 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.567+0000 7ff2de800640 1 -- 172.21.15.83:0/3405384109 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff2e0109520 msgr2=0x7ff2e01a0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:44.567 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.567+0000 7ff2de800640 1 --2- 172.21.15.83:0/3405384109 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff2e0109520 0x7ff2e01a0bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:44.567 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.567+0000 7ff2de800640 1 -- 172.21.15.83:0/3405384109 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2e019b380 con 0x7ff2e019a780 2024-04-24T09:03:44.568 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.567+0000 7ff2de800640 1 --2- 172.21.15.83:0/3405384109 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2e019a780 0x7ff2e019ac00 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7ff2d800eaa0 tx=0x7ff2d800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:44.568 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.568+0000 7ff2d6a00640 1 -- 172.21.15.83:0/3405384109 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d800cdb0 con 0x7ff2e019a780 2024-04-24T09:03:44.568 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.568+0000 7ff2d6a00640 1 -- 172.21.15.83:0/3405384109 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff2d8005020 con 0x7ff2e019a780 2024-04-24T09:03:44.568 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.568+0000 7ff2d6a00640 1 -- 172.21.15.83:0/3405384109 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d80116d0 con 0x7ff2e019a780 2024-04-24T09:03:44.568 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.568+0000 7ff2e5000640 1 -- 172.21.15.83:0/3405384109 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2e01a7410 con 0x7ff2e019a780 2024-04-24T09:03:44.568 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.568+0000 7ff2e5000640 1 -- 172.21.15.83:0/3405384109 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2e01a7810 con 0x7ff2e019a780 2024-04-24T09:03:44.570 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.571+0000 7ff2e7d9e740 1 Processor -- start 2024-04-24T09:03:44.570 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.572+0000 7ff2d6a00640 1 -- 172.21.15.83:0/3405384109 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff2d80051d0 con 0x7ff2e019a780 2024-04-24T09:03:44.571 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.572+0000 7ff2e7d9e740 1 -- start start 2024-04-24T09:03:44.572 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.572+0000 7ff2e7d9e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9d38b51b0 0x55d9d38b5590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:44.572 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.572+0000 7ff2e7d9e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d9d38b5ad0 0x55d9d38bfe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:44.573 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.572+0000 7ff2e7d9e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d9d38c0390 0x55d9d38c2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:44.573 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.572+0000 7ff2e7d9e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d9d381b900 con 0x55d9d38b51b0 2024-04-24T09:03:44.573 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.572+0000 7ff2e7d9e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d9d37c1f20 con 0x55d9d38c0390 2024-04-24T09:03:44.573 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.572+0000 7ff2e7d9e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d9d38178c0 con 0x55d9d38b5ad0 2024-04-24T09:03:44.573 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.572+0000 7ff2dd400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9d38b51b0 0x55d9d38b5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:44.573 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.573+0000 7ff2dd400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9d38b51b0 0x55d9d38b5590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47916/0 (socket says 172.21.15.83:47916) 2024-04-24T09:03:44.573 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.573+0000 7ff2dd400640 1 -- 172.21.15.83:0/34748818 learned_addr learned my addr 172.21.15.83:0/34748818 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:03:44.573 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.573+0000 7ff2dde00640 1 --2- 172.21.15.83:0/34748818 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d9d38b5ad0 0x55d9d38bfe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:44.573 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.573+0000 7ff2dd400640 1 -- 172.21.15.83:0/34748818 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d9d38b5ad0 msgr2=0x55d9d38bfe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:44.573 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.573+0000 7ff2dd400640 1 --2- 172.21.15.83:0/34748818 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d9d38b5ad0 0x55d9d38bfe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:44.573 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.573+0000 7ff2dd400640 1 -- 172.21.15.83:0/34748818 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d9d38c0390 msgr2=0x55d9d38c2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:44.573 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.573+0000 7ff2dd400640 1 --2- 172.21.15.83:0/34748818 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d9d38c0390 0x55d9d38c2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:44.573 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.573+0000 7ff2dd400640 1 -- 172.21.15.83:0/34748818 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9d37e6e10 con 0x55d9d38b51b0 2024-04-24T09:03:44.573 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.573+0000 7ff2d6a00640 1 --2- 172.21.15.83:0/3405384109 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff2c40a3e40 0x7ff2c40a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:44.574 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.573+0000 7ff2d6a00640 1 -- 172.21.15.83:0/3405384109 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9583+0+0 (secure 0 0 0) 0x7ff2d80d0e10 con 0x7ff2e019a780 2024-04-24T09:03:44.574 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.573+0000 7ff2dd400640 1 --2- 172.21.15.83:0/34748818 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9d38b51b0 0x55d9d38b5590 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc009bc0 tx=0x7ff2cc0106e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe03d1e2618569aa server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:44.574 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.573+0000 7ff2dde00640 1 --2- 172.21.15.83:0/3405384109 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff2c40a3e40 0x7ff2c40a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:44.574 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.574+0000 7ff2cb400640 1 -- 172.21.15.83:0/34748818 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2cc016070 con 0x55d9d38b51b0 2024-04-24T09:03:44.574 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.574+0000 7ff2cb400640 1 -- 172.21.15.83:0/34748818 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff2cc004c50 con 0x55d9d38b51b0 2024-04-24T09:03:44.574 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.574+0000 7ff2cb400640 1 -- 172.21.15.83:0/34748818 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2cc0056a0 con 0x55d9d38b51b0 2024-04-24T09:03:44.577 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.574+0000 7ff2e7d9e740 1 -- 172.21.15.83:0/34748818 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9d38b51b0 msgr2=0x55d9d38b5590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:44.578 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.574+0000 7ff2e7d9e740 1 --2- 172.21.15.83:0/34748818 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9d38b51b0 0x55d9d38b5590 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc009bc0 tx=0x7ff2cc0106e0 comp rx=0 tx=0).stop 2024-04-24T09:03:44.578 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.574+0000 7ff2dde00640 1 --2- 172.21.15.83:0/3405384109 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff2c40a3e40 0x7ff2c40a6300 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7ff2d0031440 tx=0x7ff2d00058b0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:44.578 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.575+0000 7ff2e7d9e740 1 -- 172.21.15.83:0/34748818 shutdown_connections 2024-04-24T09:03:44.578 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.575+0000 7ff2e7d9e740 1 --2- 172.21.15.83:0/34748818 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d9d38c0390 0x55d9d38c2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:44.578 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.575+0000 7ff2e7d9e740 1 --2- 172.21.15.83:0/34748818 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d9d38b5ad0 0x55d9d38bfe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:44.578 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.575+0000 7ff2e7d9e740 1 --2- 172.21.15.83:0/34748818 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9d38b51b0 0x55d9d38b5590 unknown :-1 s=CLOSED pgs=1933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:44.578 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.575+0000 7ff2e7d9e740 1 -- 172.21.15.83:0/34748818 >> 172.21.15.83:0/34748818 conn(0x55d9d38b16c0 msgr2=0x55d9d38b1aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:44.578 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.575+0000 7ff2e7d9e740 1 -- 172.21.15.83:0/34748818 shutdown_connections 2024-04-24T09:03:44.578 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.575+0000 7ff2e7d9e740 1 -- 172.21.15.83:0/34748818 wait complete. 2024-04-24T09:03:44.578 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.575+0000 7ff2e7d9e740 1 -- start start 2024-04-24T09:03:44.578 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.575+0000 7ff2e7d9e740 1 Processor -- start 2024-04-24T09:03:44.578 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.576+0000 7ff2e7d9e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d9d38b51b0 0x55d9d38bd250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:44.579 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.576+0000 7ff2e7d9e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9d38b5ad0 0x55d9d38bd790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:44.579 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.576+0000 7ff2e7d9e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d9d38c0390 0x55d9d38bdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:44.579 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.576+0000 7ff2e7d9e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d9d38c2fc0 con 0x55d9d38b5ad0 2024-04-24T09:03:44.579 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.576+0000 7ff2e7d9e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d9d37d52c0 con 0x55d9d38c0390 2024-04-24T09:03:44.579 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.576+0000 7ff2e7d9e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d9d38c3140 con 0x55d9d38b51b0 2024-04-24T09:03:44.579 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.576+0000 7ff2e7d9e740 10 client.0 _ll_register_callbacks cb 0x55d9d352da00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:03:44.579 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.576+0000 7ff2dd400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9d38b5ad0 0x55d9d38bd790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:44.579 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.576+0000 7ff2dd400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9d38b5ad0 0x55d9d38bd790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47932/0 (socket says 172.21.15.83:47932) 2024-04-24T09:03:44.579 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.576+0000 7ff2dd400640 1 -- 172.21.15.83:0/2764143536 learned_addr learned my addr 172.21.15.83:0/2764143536 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:03:44.579 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.576+0000 7ff2de800640 1 --2- 172.21.15.83:0/2764143536 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d9d38c0390 0x55d9d38bdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:44.579 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.576+0000 7ff2dde00640 1 --2- 172.21.15.83:0/2764143536 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d9d38b51b0 0x55d9d38bd250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:44.579 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.577+0000 7ff2dd400640 1 -- 172.21.15.83:0/2764143536 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d9d38b51b0 msgr2=0x55d9d38bd250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:44.579 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.577+0000 7ff2dd400640 1 --2- 172.21.15.83:0/2764143536 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d9d38b51b0 0x55d9d38bd250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:44.579 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.577+0000 7ff2dd400640 1 -- 172.21.15.83:0/2764143536 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d9d38c0390 msgr2=0x55d9d38bdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:44.579 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.577+0000 7ff2dd400640 1 --2- 172.21.15.83:0/2764143536 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d9d38c0390 0x55d9d38bdcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:44.580 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.577+0000 7ff2dd400640 1 -- 172.21.15.83:0/2764143536 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9d37e5fd0 con 0x55d9d38b5ad0 2024-04-24T09:03:44.580 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.577+0000 7ff2de800640 1 --2- 172.21.15.83:0/2764143536 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d9d38c0390 0x55d9d38bdcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:03:44.580 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.577+0000 7ff2dd400640 1 --2- 172.21.15.83:0/2764143536 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9d38b5ad0 0x55d9d38bd790 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc005e70 tx=0x7ff2cc009ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76c9fd80b31e0c62 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:44.580 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.577+0000 7ff2bfe00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:03:44.580 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.577+0000 7ff2bfe00640 1 -- 172.21.15.83:0/2764143536 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2cc029060 con 0x55d9d38b5ad0 2024-04-24T09:03:44.580 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.578+0000 7ff2bfe00640 1 -- 172.21.15.83:0/2764143536 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9583+0+0 (secure 0 0 0) 0x7ff2cc0046c0 con 0x55d9d38b5ad0 2024-04-24T09:03:44.580 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.578+0000 7ff2e7d9e740 1 -- 172.21.15.83:0/2764143536 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2cc004c50 con 0x55d9d38b5ad0 2024-04-24T09:03:44.580 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.578+0000 7ff2bfe00640 1 -- 172.21.15.83:0/2764143536 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7ff2cc030050 con 0x55d9d38b5ad0 2024-04-24T09:03:44.580 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.578+0000 7ff2e7d9e740 10 client.19962 fetch_fsmap learned FSMap version 41 2024-04-24T09:03:44.580 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.578+0000 7ff2e7d9e740 1 -- 172.21.15.83:0/2764143536 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55d9d352e210 con 0x55d9d38b5ad0 2024-04-24T09:03:44.580 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.578+0000 7ff2bfe00640 1 -- 172.21.15.83:0/2764143536 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff2cc004c50 con 0x55d9d38b5ad0 2024-04-24T09:03:44.580 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.578+0000 7ff2bfe00640 1 -- 172.21.15.83:0/2764143536 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7ff2cc024c10 con 0x55d9d38b5ad0 2024-04-24T09:03:44.580 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.579+0000 7ff2e7d9e740 10 client.19962 fetch_fsmap finished waiting for FSMap version 41 2024-04-24T09:03:44.580 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.579+0000 7ff2e7d9e740 10 client.19962 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:03:44.581 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.579+0000 7ff2e7d9e740 20 client.19962 populate_metadata read hostname 'smithi083' 2024-04-24T09:03:44.581 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.579+0000 7ff2e7d9e740 1 --2- 172.21.15.83:0/2764143536 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d9d38d03a0 0x55d9d38d2790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:44.581 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.579+0000 7ff2e7d9e740 4 client.19962 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:03:44.581 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.579+0000 7ff2e7d9e740 1 -- 172.21.15.83:0/2764143536 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d9d3829320 con 0x55d9d38d03a0 2024-04-24T09:03:44.582 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.579+0000 7ff2dde00640 1 --2- 172.21.15.83:0/2764143536 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d9d38d03a0 0x55d9d38d2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:44.582 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.580+0000 7ff2dde00640 1 --2- 172.21.15.83:0/2764143536 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d9d38d03a0 0x55d9d38d2790 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d3493c298d1480b server_cookie=2f3e9264ad00724f in_seq=0 out_seq=0 2024-04-24T09:03:44.582 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.580+0000 7ff2bfe00640 10 client.19962 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:03:44.582 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.582+0000 7ff2bfe00640 1 -- 172.21.15.83:0/2764143536 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d9d3829320 con 0x55d9d38d03a0 2024-04-24T09:03:44.582 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.582+0000 7ff2bfe00640 10 client.19962 handle_command_reply: tid=0 2024-04-24T09:03:44.582 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.582+0000 7ff2e7d9e740 1 client.19962 shutdown 2024-04-24T09:03:44.582 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.583+0000 7ff2e7d9e740 1 -- 172.21.15.83:0/2764143536 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9d38b5ad0 msgr2=0x55d9d38bd790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:44.582 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.583+0000 7ff2e7d9e740 1 --2- 172.21.15.83:0/2764143536 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9d38b5ad0 0x55d9d38bd790 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc005e70 tx=0x7ff2cc009ee0 comp rx=0 tx=0).stop 2024-04-24T09:03:44.583 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.583+0000 7ff2e7d9e740 1 -- 172.21.15.83:0/2764143536 shutdown_connections 2024-04-24T09:03:44.583 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.583+0000 7ff2e7d9e740 1 --2- 172.21.15.83:0/2764143536 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d9d38d03a0 0x55d9d38d2790 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:44.583 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.583+0000 7ff2e7d9e740 1 --2- 172.21.15.83:0/2764143536 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d9d38c0390 0x55d9d38bdcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:44.583 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.583+0000 7ff2e7d9e740 1 --2- 172.21.15.83:0/2764143536 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9d38b5ad0 0x55d9d38bd790 unknown :-1 s=CLOSED pgs=1934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:44.583 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.583+0000 7ff2e7d9e740 1 --2- 172.21.15.83:0/2764143536 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d9d38b51b0 0x55d9d38bd250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:44.583 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.583+0000 7ff2bfe00640 0 client.19962 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:03:44.583 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.583+0000 7ff2e7d9e740 1 -- 172.21.15.83:0/2764143536 >> 172.21.15.83:0/2764143536 conn(0x55d9d38b16c0 msgr2=0x55d9d38b73f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:44.583 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.583+0000 7ff2e7d9e740 1 -- 172.21.15.83:0/2764143536 shutdown_connections 2024-04-24T09:03:44.583 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.583+0000 7ff2e7d9e740 1 -- 172.21.15.83:0/2764143536 wait complete. 2024-04-24T09:03:44.583 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.583+0000 7ff2e7d9e740 20 client.19962 trim_cache size 0 max 16384 2024-04-24T09:03:44.603 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.599+0000 7ff2e7d9e740 1 Processor -- start 2024-04-24T09:03:44.603 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.600+0000 7ff2e7d9e740 1 -- start start 2024-04-24T09:03:44.603 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.600+0000 7ff2e7d9e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d9d38b51b0 0x55d9d38b06c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:44.603 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.600+0000 7ff2e7d9e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d9d38b5ad0 0x55d9d38d7e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:44.603 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.600+0000 7ff2e7d9e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9d38c0390 0x55d9d38d8360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:44.604 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.600+0000 7ff2e7d9e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e01b1730 con 0x55d9d38c0390 2024-04-24T09:03:44.604 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.600+0000 7ff2e7d9e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e01b3390 con 0x55d9d38b5ad0 2024-04-24T09:03:44.604 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.600+0000 7ff2e7d9e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e01b1e00 con 0x55d9d38b51b0 2024-04-24T09:03:44.604 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.600+0000 7ff2de800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9d38c0390 0x55d9d38d8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:44.604 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.600+0000 7ff2de800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9d38c0390 0x55d9d38d8360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47940/0 (socket says 172.21.15.83:47940) 2024-04-24T09:03:44.604 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.600+0000 7ff2de800640 1 -- 172.21.15.83:0/1291470432 learned_addr learned my addr 172.21.15.83:0/1291470432 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:03:44.604 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.600+0000 7ff2dde00640 1 --2- 172.21.15.83:0/1291470432 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d9d38b51b0 0x55d9d38b06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:44.604 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.600+0000 7ff2dd400640 1 --2- 172.21.15.83:0/1291470432 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d9d38b5ad0 0x55d9d38d7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:44.604 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.601+0000 7ff2de800640 1 -- 172.21.15.83:0/1291470432 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d9d38b51b0 msgr2=0x55d9d38b06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:44.604 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.601+0000 7ff2de800640 1 --2- 172.21.15.83:0/1291470432 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d9d38b51b0 0x55d9d38b06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:44.604 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.601+0000 7ff2de800640 1 -- 172.21.15.83:0/1291470432 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d9d38b5ad0 msgr2=0x55d9d38d7e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:44.604 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.601+0000 7ff2de800640 1 --2- 172.21.15.83:0/1291470432 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d9d38b5ad0 0x55d9d38d7e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:44.604 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.601+0000 7ff2de800640 1 -- 172.21.15.83:0/1291470432 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9d35fa620 con 0x55d9d38c0390 2024-04-24T09:03:44.604 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.601+0000 7ff2dd400640 1 --2- 172.21.15.83:0/1291470432 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d9d38b5ad0 0x55d9d38d7e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:03:44.604 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.601+0000 7ff2de800640 1 --2- 172.21.15.83:0/1291470432 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9d38c0390 0x55d9d38d8360 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7ff2d80092f0 tx=0x7ff2d809c5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb299ea66639dccd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:44.605 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.601+0000 7ff2caa00640 1 -- 172.21.15.83:0/1291470432 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d801f040 con 0x55d9d38c0390 2024-04-24T09:03:44.605 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.601+0000 7ff2caa00640 1 -- 172.21.15.83:0/1291470432 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff2d80a55a0 con 0x55d9d38c0390 2024-04-24T09:03:44.605 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.601+0000 7ff2caa00640 1 -- 172.21.15.83:0/1291470432 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d80a5110 con 0x55d9d38c0390 2024-04-24T09:03:44.605 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.602+0000 7ff2e7d9e740 1 -- 172.21.15.83:0/1291470432 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9d38c0390 msgr2=0x55d9d38d8360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:44.605 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.602+0000 7ff2e7d9e740 1 --2- 172.21.15.83:0/1291470432 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9d38c0390 0x55d9d38d8360 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7ff2d80092f0 tx=0x7ff2d809c5a0 comp rx=0 tx=0).stop 2024-04-24T09:03:44.605 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.602+0000 7ff2e7d9e740 1 -- 172.21.15.83:0/1291470432 shutdown_connections 2024-04-24T09:03:44.605 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.602+0000 7ff2e7d9e740 1 --2- 172.21.15.83:0/1291470432 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9d38c0390 0x55d9d38d8360 unknown :-1 s=CLOSED pgs=1935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:44.605 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.602+0000 7ff2e7d9e740 1 --2- 172.21.15.83:0/1291470432 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d9d38b5ad0 0x55d9d38d7e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:44.605 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.602+0000 7ff2e7d9e740 1 --2- 172.21.15.83:0/1291470432 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d9d38b51b0 0x55d9d38b06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:44.605 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.602+0000 7ff2e7d9e740 1 -- 172.21.15.83:0/1291470432 >> 172.21.15.83:0/1291470432 conn(0x55d9d38b16c0 msgr2=0x55d9d38affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:44.605 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.602+0000 7ff2e7d9e740 1 -- 172.21.15.83:0/1291470432 shutdown_connections 2024-04-24T09:03:44.605 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.602+0000 7ff2e7d9e740 1 -- 172.21.15.83:0/1291470432 wait complete. 2024-04-24T09:03:44.605 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.603+0000 7ff2e7d9e740 1 -- start start 2024-04-24T09:03:44.605 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.603+0000 7ff2e7d9e740 1 Processor -- start 2024-04-24T09:03:44.606 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.603+0000 7ff2e7d9e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d9d38b51b0 0x55d9d38c8470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:44.606 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.603+0000 7ff2e7d9e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d9d38b5ad0 0x55d9d38c48f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:44.606 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.603+0000 7ff2e7d9e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9d38c0390 0x55d9d38c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:44.606 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.604+0000 7ff2e7d9e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e01b2c60 con 0x55d9d38c0390 2024-04-24T09:03:44.606 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.604+0000 7ff2e7d9e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e01b2530 con 0x55d9d38b51b0 2024-04-24T09:03:44.606 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.604+0000 7ff2e7d9e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e01af730 con 0x55d9d38b5ad0 2024-04-24T09:03:44.606 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.604+0000 7ff2e7d9e740 10 client.0 _ll_register_callbacks cb 0x7ff2cc009ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:03:44.606 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.604+0000 7ff2de800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9d38c0390 0x55d9d38c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:44.606 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.604+0000 7ff2de800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9d38c0390 0x55d9d38c4e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47950/0 (socket says 172.21.15.83:47950) 2024-04-24T09:03:44.606 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.604+0000 7ff2de800640 1 -- 172.21.15.83:0/211939757 learned_addr learned my addr 172.21.15.83:0/211939757 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:03:44.606 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.604+0000 7ff2dde00640 1 --2- 172.21.15.83:0/211939757 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d9d38b51b0 0x55d9d38c8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:44.607 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.604+0000 7ff2de800640 1 -- 172.21.15.83:0/211939757 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d9d38b5ad0 msgr2=0x55d9d38c48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:44.607 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.604+0000 7ff2de800640 1 --2- 172.21.15.83:0/211939757 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d9d38b5ad0 0x55d9d38c48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:44.607 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.604+0000 7ff2de800640 1 -- 172.21.15.83:0/211939757 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d9d38b51b0 msgr2=0x55d9d38c8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:44.607 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.604+0000 7ff2de800640 1 --2- 172.21.15.83:0/211939757 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d9d38b51b0 0x55d9d38c8470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:44.607 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.604+0000 7ff2de800640 1 -- 172.21.15.83:0/211939757 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9d38c5370 con 0x55d9d38c0390 2024-04-24T09:03:44.607 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.604+0000 7ff2de800640 1 --2- 172.21.15.83:0/211939757 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9d38c0390 0x55d9d38c4e30 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7ff2d8098d90 tx=0x7ff2d80ce050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=709b46c10218f28a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:44.607 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.604+0000 7ff2bfe00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:03:44.609 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.605+0000 7ff2bfe00640 1 -- 172.21.15.83:0/211939757 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d80bb070 con 0x55d9d38c0390 2024-04-24T09:03:44.610 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.605+0000 7ff2bfe00640 1 -- 172.21.15.83:0/211939757 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9583+0+0 (secure 0 0 0) 0x7ff2d80a5310 con 0x55d9d38c0390 2024-04-24T09:03:44.610 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.605+0000 7ff2e7d9e740 1 -- 172.21.15.83:0/211939757 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2d80a55a0 con 0x55d9d38c0390 2024-04-24T09:03:44.610 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.605+0000 7ff2bfe00640 1 -- 172.21.15.83:0/211939757 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7ff2d8005620 con 0x55d9d38c0390 2024-04-24T09:03:44.610 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.605+0000 7ff2e7d9e740 10 client.19968 fetch_fsmap learned FSMap version 41 2024-04-24T09:03:44.610 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.605+0000 7ff2e7d9e740 1 -- 172.21.15.83:0/211939757 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55d9d39373c0 con 0x55d9d38c0390 2024-04-24T09:03:44.610 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.605+0000 7ff2bfe00640 1 -- 172.21.15.83:0/211939757 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff2d80a55a0 con 0x55d9d38c0390 2024-04-24T09:03:44.610 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.605+0000 7ff2bfe00640 1 -- 172.21.15.83:0/211939757 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7ff2d800f040 con 0x55d9d38c0390 2024-04-24T09:03:44.610 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.605+0000 7ff2e7d9e740 10 client.19968 fetch_fsmap finished waiting for FSMap version 41 2024-04-24T09:03:44.610 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.605+0000 7ff2e7d9e740 10 client.19968 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:03:44.610 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.605+0000 7ff2e7d9e740 20 client.19968 populate_metadata read hostname 'smithi083' 2024-04-24T09:03:44.610 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.605+0000 7ff2e7d9e740 1 --2- 172.21.15.83:0/211939757 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d9d39379e0 0x55d9d3939e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:44.610 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.605+0000 7ff2e7d9e740 4 client.19968 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T09:03:44.610 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.605+0000 7ff2e7d9e740 1 -- 172.21.15.83:0/211939757 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d9d3823190 con 0x55d9d39379e0 2024-04-24T09:03:44.611 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.606+0000 7ff2dde00640 1 --2- 172.21.15.83:0/211939757 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d9d39379e0 0x55d9d3939e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:44.611 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.607+0000 7ff2dde00640 1 --2- 172.21.15.83:0/211939757 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d9d39379e0 0x55d9d3939e00 crc :-1 s=READY pgs=320 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a92cd9575a3112de server_cookie=b0a78311c880db99 in_seq=0 out_seq=0 2024-04-24T09:03:44.611 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.607+0000 7ff2bfe00640 10 client.19968 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:03:44.621 INFO:tasks.workunit.client.0.smithi083.stdout: 1 70696 0.60 MB/sec execute 498 sec latency 55.167 ms 2024-04-24T09:03:44.663 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.663+0000 7ff2bfe00640 1 -- 172.21.15.83:0/211939757 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d9d3823190 con 0x55d9d39379e0 2024-04-24T09:03:44.664 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.663+0000 7ff2bfe00640 10 client.19968 handle_command_reply: tid=0 2024-04-24T09:03:44.664 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.663+0000 7ff2e7d9e740 1 client.19968 shutdown 2024-04-24T09:03:44.664 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.663+0000 7ff2e7d9e740 1 -- 172.21.15.83:0/211939757 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9d38c0390 msgr2=0x55d9d38c4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:44.664 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.663+0000 7ff2e7d9e740 1 --2- 172.21.15.83:0/211939757 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9d38c0390 0x55d9d38c4e30 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7ff2d8098d90 tx=0x7ff2d80ce050 comp rx=0 tx=0).stop 2024-04-24T09:03:44.664 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.664+0000 7ff2e7d9e740 1 -- 172.21.15.83:0/211939757 shutdown_connections 2024-04-24T09:03:44.664 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.664+0000 7ff2e7d9e740 1 --2- 172.21.15.83:0/211939757 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d9d39379e0 0x55d9d3939e00 crc :-1 s=READY pgs=320 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:44.664 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.664+0000 7ff2e7d9e740 1 --2- 172.21.15.83:0/211939757 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9d38c0390 0x55d9d38c4e30 unknown :-1 s=CLOSED pgs=1936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:44.664 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.664+0000 7ff2e7d9e740 1 --2- 172.21.15.83:0/211939757 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d9d38b5ad0 0x55d9d38c48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:44.664 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.664+0000 7ff2e7d9e740 1 --2- 172.21.15.83:0/211939757 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d9d38b51b0 0x55d9d38c8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:44.664 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.664+0000 7ff2e7d9e740 1 -- 172.21.15.83:0/211939757 >> 172.21.15.83:0/211939757 conn(0x55d9d38b16c0 msgr2=0x55d9d38b73f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:44.664 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.664+0000 7ff2bfe00640 0 client.19968 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:03:44.664 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.664+0000 7ff2e7d9e740 1 -- 172.21.15.83:0/211939757 shutdown_connections 2024-04-24T09:03:44.665 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.664+0000 7ff2e7d9e740 1 -- 172.21.15.83:0/211939757 wait complete. 2024-04-24T09:03:44.665 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.664+0000 7ff2e7d9e740 20 client.19968 trim_cache size 0 max 16384 2024-04-24T09:03:44.665 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T09:03:44.665 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T09:03:44.665 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T09:03:44.665 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T09:03:44.666 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.665+0000 7ff2e5000640 1 -- 172.21.15.83:0/3405384109 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff2c40a3e40 msgr2=0x7ff2c40a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:44.666 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.665+0000 7ff2e5000640 1 --2- 172.21.15.83:0/3405384109 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff2c40a3e40 0x7ff2c40a6300 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7ff2d0031440 tx=0x7ff2d00058b0 comp rx=0 tx=0).stop 2024-04-24T09:03:44.666 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.665+0000 7ff2e5000640 1 -- 172.21.15.83:0/3405384109 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2e019a780 msgr2=0x7ff2e019ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:44.666 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.665+0000 7ff2e5000640 1 --2- 172.21.15.83:0/3405384109 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2e019a780 0x7ff2e019ac00 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7ff2d800eaa0 tx=0x7ff2d800ef60 comp rx=0 tx=0).stop 2024-04-24T09:03:44.666 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.666+0000 7ff2e5000640 1 -- 172.21.15.83:0/3405384109 shutdown_connections 2024-04-24T09:03:44.666 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.666+0000 7ff2e5000640 1 --2- 172.21.15.83:0/3405384109 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff2c40a3e40 0x7ff2c40a6300 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:44.666 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.666+0000 7ff2e5000640 1 --2- 172.21.15.83:0/3405384109 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2e019a780 0x7ff2e019ac00 unknown :-1 s=CLOSED pgs=1932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:44.666 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.666+0000 7ff2e5000640 1 --2- 172.21.15.83:0/3405384109 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff2e0109520 0x7ff2e01a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:44.666 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.666+0000 7ff2e5000640 1 --2- 172.21.15.83:0/3405384109 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff2e0104450 0x7ff2e01a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:44.667 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.666+0000 7ff2e5000640 1 -- 172.21.15.83:0/3405384109 >> 172.21.15.83:0/3405384109 conn(0x7ff2e00ffcf0 msgr2=0x7ff2e0102040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:44.667 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.666+0000 7ff2e5000640 1 -- 172.21.15.83:0/3405384109 shutdown_connections 2024-04-24T09:03:44.667 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:44.666+0000 7ff2e5000640 1 -- 172.21.15.83:0/3405384109 wait complete. 2024-04-24T09:03:45.622 INFO:tasks.workunit.client.0.smithi083.stdout: 1 70726 0.60 MB/sec execute 499 sec latency 60.689 ms 2024-04-24T09:03:46.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:03:46 smithi138 ceph-mon[31441]: pgmap v381: 129 pgs: 129 active+clean; 65 MiB data, 899 MiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 307 KiB/s wr, 48 op/s 2024-04-24T09:03:46.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:03:46 smithi189 ceph-mon[31791]: pgmap v381: 129 pgs: 129 active+clean; 65 MiB data, 899 MiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 307 KiB/s wr, 48 op/s 2024-04-24T09:03:46.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:03:46 smithi083 ceph-mon[27907]: pgmap v381: 129 pgs: 129 active+clean; 65 MiB data, 899 MiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 307 KiB/s wr, 48 op/s 2024-04-24T09:03:46.621 INFO:tasks.workunit.client.0.smithi083.stdout: 1 70749 0.60 MB/sec execute 500 sec latency 60.721 ms 2024-04-24T09:03:47.621 INFO:tasks.workunit.client.0.smithi083.stdout: 1 70907 0.61 MB/sec execute 501 sec latency 50.183 ms 2024-04-24T09:03:48.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:03:48 smithi138 ceph-mon[31441]: pgmap v382: 129 pgs: 129 active+clean; 65 MiB data, 899 MiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 307 KiB/s wr, 48 op/s 2024-04-24T09:03:48.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:03:48 smithi189 ceph-mon[31791]: pgmap v382: 129 pgs: 129 active+clean; 65 MiB data, 899 MiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 307 KiB/s wr, 48 op/s 2024-04-24T09:03:48.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:03:48 smithi083 ceph-mon[27907]: pgmap v382: 129 pgs: 129 active+clean; 65 MiB data, 899 MiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 307 KiB/s wr, 48 op/s 2024-04-24T09:03:48.621 INFO:tasks.workunit.client.0.smithi083.stdout: 1 70998 0.61 MB/sec execute 502 sec latency 57.867 ms 2024-04-24T09:03:49.622 INFO:tasks.workunit.client.0.smithi083.stdout: 1 71034 0.61 MB/sec execute 503 sec latency 924.787 ms 2024-04-24T09:03:50.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:03:50 smithi138 ceph-mon[31441]: pgmap v383: 129 pgs: 129 active+clean; 67 MiB data, 904 MiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 589 KiB/s wr, 69 op/s 2024-04-24T09:03:50.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:03:50 smithi189 ceph-mon[31791]: pgmap v383: 129 pgs: 129 active+clean; 67 MiB data, 904 MiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 589 KiB/s wr, 69 op/s 2024-04-24T09:03:50.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:03:50 smithi083 ceph-mon[27907]: pgmap v383: 129 pgs: 129 active+clean; 67 MiB data, 904 MiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 589 KiB/s wr, 69 op/s 2024-04-24T09:03:50.622 INFO:tasks.workunit.client.0.smithi083.stdout: 1 71034 0.60 MB/sec execute 504 sec latency 1924.888 ms 2024-04-24T09:03:51.622 INFO:tasks.workunit.client.0.smithi083.stdout: 1 71034 0.60 MB/sec execute 505 sec latency 2924.981 ms 2024-04-24T09:03:52.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:03:52 smithi138 ceph-mon[31441]: pgmap v384: 129 pgs: 129 active+clean; 67 MiB data, 904 MiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 511 KiB/s wr, 41 op/s 2024-04-24T09:03:52.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:03:52 smithi189 ceph-mon[31791]: pgmap v384: 129 pgs: 129 active+clean; 67 MiB data, 904 MiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 511 KiB/s wr, 41 op/s 2024-04-24T09:03:52.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:03:52 smithi083 ceph-mon[27907]: pgmap v384: 129 pgs: 129 active+clean; 67 MiB data, 904 MiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 511 KiB/s wr, 41 op/s 2024-04-24T09:03:52.622 INFO:tasks.workunit.client.0.smithi083.stdout: 1 71034 0.60 MB/sec execute 506 sec latency 3925.147 ms 2024-04-24T09:03:53.622 INFO:tasks.workunit.client.0.smithi083.stdout: 1 71073 0.60 MB/sec execute 507 sec latency 4408.808 ms 2024-04-24T09:03:54.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:03:54 smithi138 ceph-mon[31441]: pgmap v385: 129 pgs: 129 active+clean; 67 MiB data, 927 MiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 539 KiB/s wr, 53 op/s 2024-04-24T09:03:54.508 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:03:54 smithi189 ceph-mon[31791]: pgmap v385: 129 pgs: 129 active+clean; 67 MiB data, 927 MiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 539 KiB/s wr, 53 op/s 2024-04-24T09:03:54.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:03:54 smithi083 ceph-mon[27907]: pgmap v385: 129 pgs: 129 active+clean; 67 MiB data, 927 MiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 539 KiB/s wr, 53 op/s 2024-04-24T09:03:54.622 INFO:tasks.workunit.client.0.smithi083.stdout: 1 71101 0.60 MB/sec execute 508 sec latency 610.582 ms 2024-04-24T09:03:55.622 INFO:tasks.workunit.client.0.smithi083.stdout: 1 71101 0.60 MB/sec execute 509 sec latency 1610.721 ms 2024-04-24T09:03:56.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:03:56 smithi138 ceph-mon[31441]: pgmap v386: 129 pgs: 129 active+clean; 67 MiB data, 927 MiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 310 KiB/s wr, 32 op/s 2024-04-24T09:03:56.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:03:56 smithi189 ceph-mon[31791]: pgmap v386: 129 pgs: 129 active+clean; 67 MiB data, 927 MiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 310 KiB/s wr, 32 op/s 2024-04-24T09:03:56.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:03:56 smithi083 ceph-mon[27907]: pgmap v386: 129 pgs: 129 active+clean; 67 MiB data, 927 MiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 310 KiB/s wr, 32 op/s 2024-04-24T09:03:56.622 INFO:tasks.workunit.client.0.smithi083.stdout: 1 71101 0.60 MB/sec execute 510 sec latency 2610.896 ms 2024-04-24T09:03:57.623 INFO:tasks.workunit.client.0.smithi083.stdout: 1 71101 0.60 MB/sec execute 511 sec latency 3611.031 ms 2024-04-24T09:03:57.813 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.813+0000 7f1abf800640 1 -- 172.21.15.83:0/3597603246 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ab8112ea0 msgr2=0x7f1ab8115290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:57.813 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.813+0000 7f1abf800640 1 --2- 172.21.15.83:0/3597603246 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ab8112ea0 0x7f1ab8115290 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7f1ab0009ca0 tx=0x7f1ab002a8b0 comp rx=0 tx=0).stop 2024-04-24T09:03:57.813 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.813+0000 7f1abf800640 1 -- 172.21.15.83:0/3597603246 shutdown_connections 2024-04-24T09:03:57.813 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.813+0000 7f1abf800640 1 --2- 172.21.15.83:0/3597603246 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ab8112ea0 0x7f1ab8115290 unknown :-1 s=CLOSED pgs=1937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:57.813 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.813+0000 7f1abf800640 1 --2- 172.21.15.83:0/3597603246 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ab8076e50 0x7f1ab8112960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:57.813 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.813+0000 7f1abf800640 1 --2- 172.21.15.83:0/3597603246 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1ab8076530 0x7f1ab8076910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:57.814 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.813+0000 7f1abf800640 1 -- 172.21.15.83:0/3597603246 >> 172.21.15.83:0/3597603246 conn(0x7f1ab80ffd30 msgr2=0x7f1ab8102150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:57.814 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.813+0000 7f1abf800640 1 -- 172.21.15.83:0/3597603246 shutdown_connections 2024-04-24T09:03:57.814 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.813+0000 7f1abf800640 1 -- 172.21.15.83:0/3597603246 wait complete. 2024-04-24T09:03:57.816 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.814+0000 7f1abf800640 1 Processor -- start 2024-04-24T09:03:57.816 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.814+0000 7f1abf800640 1 -- start start 2024-04-24T09:03:57.816 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.814+0000 7f1abf800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ab8076530 0x7f1ab81a4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:57.816 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.814+0000 7f1abf800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ab8076e50 0x7f1ab81a4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:57.816 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.814+0000 7f1abf800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1ab8112ea0 0x7f1ab819eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:57.816 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.814+0000 7f1abf800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab81188b0 con 0x7f1ab8076530 2024-04-24T09:03:57.816 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.814+0000 7f1abf800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab8118730 con 0x7f1ab8076e50 2024-04-24T09:03:57.816 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.814+0000 7f1abf800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab8118a30 con 0x7f1ab8112ea0 2024-04-24T09:03:57.816 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.815+0000 7f1abce00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ab8076530 0x7f1ab81a4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:57.816 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.815+0000 7f1abce00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ab8076530 0x7f1ab81a4900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52438/0 (socket says 172.21.15.83:52438) 2024-04-24T09:03:57.816 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.815+0000 7f1abce00640 1 -- 172.21.15.83:0/1720791744 learned_addr learned my addr 172.21.15.83:0/1720791744 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:03:57.817 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.815+0000 7f1abd800640 1 --2- 172.21.15.83:0/1720791744 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1ab8112ea0 0x7f1ab819eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:57.817 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.815+0000 7f1aafe00640 1 --2- 172.21.15.83:0/1720791744 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ab8076e50 0x7f1ab81a4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:57.817 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.815+0000 7f1abce00640 1 -- 172.21.15.83:0/1720791744 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1ab8112ea0 msgr2=0x7f1ab819eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:57.817 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.815+0000 7f1abce00640 1 --2- 172.21.15.83:0/1720791744 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1ab8112ea0 0x7f1ab819eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:57.817 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.815+0000 7f1abce00640 1 -- 172.21.15.83:0/1720791744 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ab8076e50 msgr2=0x7f1ab81a4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:57.817 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.815+0000 7f1abce00640 1 --2- 172.21.15.83:0/1720791744 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ab8076e50 0x7f1ab81a4e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:57.817 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.815+0000 7f1abce00640 1 -- 172.21.15.83:0/1720791744 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ab819f230 con 0x7f1ab8076530 2024-04-24T09:03:57.817 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.816+0000 7f1abce00640 1 --2- 172.21.15.83:0/1720791744 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ab8076530 0x7f1ab81a4900 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7f1aa800c920 tx=0x7f1aa800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:57.818 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.816+0000 7f1ab6000640 1 -- 172.21.15.83:0/1720791744 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa8007c70 con 0x7f1ab8076530 2024-04-24T09:03:57.818 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.816+0000 7f1ab6000640 1 -- 172.21.15.83:0/1720791744 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1aa8007e10 con 0x7f1ab8076530 2024-04-24T09:03:57.818 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.816+0000 7f1ab6000640 1 -- 172.21.15.83:0/1720791744 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa8018820 con 0x7f1ab8076530 2024-04-24T09:03:57.818 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.816+0000 7f1abf800640 1 -- 172.21.15.83:0/1720791744 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ab819f520 con 0x7f1ab8076530 2024-04-24T09:03:57.818 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.816+0000 7f1abf800640 1 -- 172.21.15.83:0/1720791744 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ab81ab7c0 con 0x7f1ab8076530 2024-04-24T09:03:57.819 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.819+0000 7f1ab6000640 1 -- 172.21.15.83:0/1720791744 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1aa80189c0 con 0x7f1ab8076530 2024-04-24T09:03:57.819 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.819+0000 7f1ac2601740 1 Processor -- start 2024-04-24T09:03:57.819 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.819+0000 7f1ac2601740 1 -- start start 2024-04-24T09:03:57.822 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.820+0000 7f1ac2601740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558aa9dac1b0 0x558aa9dac590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:57.822 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.820+0000 7f1ac2601740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558aa9dacad0 0x558aa9db6e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:57.822 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.820+0000 7f1ac2601740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558aa9db7390 0x558aa9db9780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:57.822 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.820+0000 7f1ac2601740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x558aa9d12900 con 0x558aa9dacad0 2024-04-24T09:03:57.822 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.820+0000 7f1ac2601740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558aa9cb8f20 con 0x558aa9dac1b0 2024-04-24T09:03:57.822 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.820+0000 7f1ac2601740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x558aa9d0e8c0 con 0x558aa9db7390 2024-04-24T09:03:57.822 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.820+0000 7f1abce00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558aa9dacad0 0x558aa9db6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:57.822 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.820+0000 7f1abce00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558aa9dacad0 0x558aa9db6e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52446/0 (socket says 172.21.15.83:52446) 2024-04-24T09:03:57.822 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.820+0000 7f1abce00640 1 -- 172.21.15.83:0/1604032141 learned_addr learned my addr 172.21.15.83:0/1604032141 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:03:57.823 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.820+0000 7f1abd800640 1 --2- 172.21.15.83:0/1604032141 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558aa9dac1b0 0x558aa9dac590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:57.823 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.821+0000 7f1aafe00640 1 --2- 172.21.15.83:0/1604032141 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558aa9db7390 0x558aa9db9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:57.823 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.821+0000 7f1abce00640 1 -- 172.21.15.83:0/1604032141 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558aa9db7390 msgr2=0x558aa9db9780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:57.823 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.821+0000 7f1abce00640 1 --2- 172.21.15.83:0/1604032141 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558aa9db7390 0x558aa9db9780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:57.823 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.821+0000 7f1abce00640 1 -- 172.21.15.83:0/1604032141 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558aa9dac1b0 msgr2=0x558aa9dac590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:57.823 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.821+0000 7f1abce00640 1 --2- 172.21.15.83:0/1604032141 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558aa9dac1b0 0x558aa9dac590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:57.823 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.821+0000 7f1abce00640 1 -- 172.21.15.83:0/1604032141 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558aa9cdde10 con 0x558aa9dacad0 2024-04-24T09:03:57.823 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.821+0000 7f1aafe00640 1 --2- 172.21.15.83:0/1604032141 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558aa9db7390 0x558aa9db9780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:03:57.824 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.821+0000 7f1abce00640 1 --2- 172.21.15.83:0/1604032141 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558aa9dacad0 0x558aa9db6e50 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7f1aa80ce090 tx=0x7f1aa80d9220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb359b6a6c0a2fb0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:57.824 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.821+0000 7f1ab6000640 1 --2- 172.21.15.83:0/1720791744 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f1a980a3dc0 0x7f1a980a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:57.824 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.821+0000 7f1aae000640 1 -- 172.21.15.83:0/1604032141 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa80e4070 con 0x558aa9dacad0 2024-04-24T09:03:57.824 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.821+0000 7f1ab6000640 1 -- 172.21.15.83:0/1720791744 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9583+0+0 (secure 0 0 0) 0x7f1aa80d0e80 con 0x7f1ab8076530 2024-04-24T09:03:57.824 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.821+0000 7f1aae000640 1 -- 172.21.15.83:0/1604032141 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1aa800fb00 con 0x558aa9dacad0 2024-04-24T09:03:57.824 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.821+0000 7f1aae000640 1 -- 172.21.15.83:0/1604032141 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa809c770 con 0x558aa9dacad0 2024-04-24T09:03:57.824 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.821+0000 7f1abd800640 1 --2- 172.21.15.83:0/1720791744 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f1a980a3dc0 0x7f1a980a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:57.828 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.822+0000 7f1ac2601740 1 -- 172.21.15.83:0/1604032141 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558aa9dacad0 msgr2=0x558aa9db6e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:57.828 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.822+0000 7f1ac2601740 1 --2- 172.21.15.83:0/1604032141 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558aa9dacad0 0x558aa9db6e50 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7f1aa80ce090 tx=0x7f1aa80d9220 comp rx=0 tx=0).stop 2024-04-24T09:03:57.828 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.822+0000 7f1abd800640 1 --2- 172.21.15.83:0/1720791744 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f1a980a3dc0 0x7f1a980a6280 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f1ab002add0 tx=0x7f1ab0005ef0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:57.828 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.822+0000 7f1ac2601740 1 -- 172.21.15.83:0/1604032141 shutdown_connections 2024-04-24T09:03:57.828 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.822+0000 7f1ac2601740 1 --2- 172.21.15.83:0/1604032141 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558aa9db7390 0x558aa9db9780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:57.828 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.822+0000 7f1ac2601740 1 --2- 172.21.15.83:0/1604032141 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558aa9dacad0 0x558aa9db6e50 unknown :-1 s=CLOSED pgs=1939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:57.828 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.822+0000 7f1ac2601740 1 --2- 172.21.15.83:0/1604032141 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558aa9dac1b0 0x558aa9dac590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:57.828 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.822+0000 7f1ac2601740 1 -- 172.21.15.83:0/1604032141 >> 172.21.15.83:0/1604032141 conn(0x558aa9da86c0 msgr2=0x558aa9da8aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:57.828 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.823+0000 7f1ac2601740 1 -- 172.21.15.83:0/1604032141 shutdown_connections 2024-04-24T09:03:57.828 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.823+0000 7f1ac2601740 1 -- 172.21.15.83:0/1604032141 wait complete. 2024-04-24T09:03:57.828 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.823+0000 7f1ac2601740 1 -- start start 2024-04-24T09:03:57.828 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.823+0000 7f1ac2601740 1 Processor -- start 2024-04-24T09:03:57.829 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.824+0000 7f1ac2601740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558aa9dac1b0 0x558aa9db42f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:57.829 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.824+0000 7f1ac2601740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558aa9dacad0 0x558aa9db4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:57.829 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.824+0000 7f1ac2601740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558aa9db7390 0x558aa9db4d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:57.829 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.824+0000 7f1ac2601740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x558aa9db9fc0 con 0x558aa9dac1b0 2024-04-24T09:03:57.829 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.824+0000 7f1ac2601740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558aa9ccc2c0 con 0x558aa9dacad0 2024-04-24T09:03:57.829 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.824+0000 7f1ac2601740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x558aa9dba140 con 0x558aa9db7390 2024-04-24T09:03:57.829 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.824+0000 7f1ac2601740 10 client.0 _ll_register_callbacks cb 0x558aa9a24a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:03:57.829 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.824+0000 7f1abce00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558aa9dac1b0 0x558aa9db42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:57.829 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.824+0000 7f1abce00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558aa9dac1b0 0x558aa9db42f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52456/0 (socket says 172.21.15.83:52456) 2024-04-24T09:03:57.829 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.824+0000 7f1abce00640 1 -- 172.21.15.83:0/47579156 learned_addr learned my addr 172.21.15.83:0/47579156 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:03:57.829 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.824+0000 7f1aafe00640 1 --2- 172.21.15.83:0/47579156 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558aa9dacad0 0x558aa9db4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:57.829 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.824+0000 7f1abce00640 1 -- 172.21.15.83:0/47579156 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558aa9db7390 msgr2=0x558aa9db4d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:57.829 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.824+0000 7f1abce00640 1 --2- 172.21.15.83:0/47579156 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558aa9db7390 0x558aa9db4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:57.829 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.824+0000 7f1abce00640 1 -- 172.21.15.83:0/47579156 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558aa9dacad0 msgr2=0x558aa9db4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:57.830 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.824+0000 7f1abce00640 1 --2- 172.21.15.83:0/47579156 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558aa9dacad0 0x558aa9db4830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:57.830 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.824+0000 7f1abce00640 1 -- 172.21.15.83:0/47579156 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558aa9cdcfd0 con 0x558aa9dac1b0 2024-04-24T09:03:57.830 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.824+0000 7f1aafe00640 1 --2- 172.21.15.83:0/47579156 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558aa9dacad0 0x558aa9db4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:03:57.830 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.825+0000 7f1abce00640 1 --2- 172.21.15.83:0/47579156 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558aa9dac1b0 0x558aa9db42f0 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7f1aa8098e00 tx=0x7f1aa80d47e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f220ac9eb7891 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:57.830 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.825+0000 7f1aa6a00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:03:57.830 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.825+0000 7f1aa6a00640 1 -- 172.21.15.83:0/47579156 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa805fd60 con 0x558aa9dac1b0 2024-04-24T09:03:57.830 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.825+0000 7f1aa6a00640 1 -- 172.21.15.83:0/47579156 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9583+0+0 (secure 0 0 0) 0x7f1aa809aca0 con 0x558aa9dac1b0 2024-04-24T09:03:57.830 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.825+0000 7f1ac2601740 1 -- 172.21.15.83:0/47579156 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558aa9d20320 con 0x558aa9dac1b0 2024-04-24T09:03:57.830 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.825+0000 7f1aa6a00640 1 -- 172.21.15.83:0/47579156 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f1aa805ff80 con 0x558aa9dac1b0 2024-04-24T09:03:57.830 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.825+0000 7f1ac2601740 10 client.19980 fetch_fsmap learned FSMap version 41 2024-04-24T09:03:57.830 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.825+0000 7f1ac2601740 1 -- 172.21.15.83:0/47579156 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x558aa9a25210 con 0x558aa9dac1b0 2024-04-24T09:03:57.830 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.825+0000 7f1aa6a00640 1 -- 172.21.15.83:0/47579156 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x558aa9d20320 con 0x558aa9dac1b0 2024-04-24T09:03:57.830 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.826+0000 7f1aa6a00640 1 -- 172.21.15.83:0/47579156 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7f1aa80ce3d0 con 0x558aa9dac1b0 2024-04-24T09:03:57.830 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.826+0000 7f1ac2601740 10 client.19980 fetch_fsmap finished waiting for FSMap version 41 2024-04-24T09:03:57.831 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.826+0000 7f1ac2601740 10 client.19980 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:03:57.831 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.826+0000 7f1ac2601740 20 client.19980 populate_metadata read hostname 'smithi083' 2024-04-24T09:03:57.831 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.826+0000 7f1ac2601740 1 --2- 172.21.15.83:0/47579156 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x558aa9dc7290 0x558aa9dc9680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:57.831 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.826+0000 7f1ac2601740 4 client.19980 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:03:57.831 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.826+0000 7f1ac2601740 1 -- 172.21.15.83:0/47579156 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558aa9d1a190 con 0x558aa9dc7290 2024-04-24T09:03:57.831 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.826+0000 7f1aafe00640 1 --2- 172.21.15.83:0/47579156 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x558aa9dc7290 0x558aa9dc9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:57.831 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.828+0000 7f1aa6a00640 10 client.19980 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:03:57.831 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.828+0000 7f1aafe00640 1 --2- 172.21.15.83:0/47579156 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x558aa9dc7290 0x558aa9dc9680 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c6f573411d12ca0 server_cookie=444605ff11508557 in_seq=0 out_seq=0 2024-04-24T09:03:57.832 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.829+0000 7f1aa6a00640 1 -- 172.21.15.83:0/47579156 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558aa9d1a190 con 0x558aa9dc7290 2024-04-24T09:03:57.832 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.829+0000 7f1aa6a00640 10 client.19980 handle_command_reply: tid=0 2024-04-24T09:03:57.832 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.829+0000 7f1ac2601740 1 client.19980 shutdown 2024-04-24T09:03:57.832 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.830+0000 7f1ac2601740 1 -- 172.21.15.83:0/47579156 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558aa9dac1b0 msgr2=0x558aa9db42f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:57.832 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.830+0000 7f1ac2601740 1 --2- 172.21.15.83:0/47579156 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558aa9dac1b0 0x558aa9db42f0 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7f1aa8098e00 tx=0x7f1aa80d47e0 comp rx=0 tx=0).stop 2024-04-24T09:03:57.833 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.830+0000 7f1ac2601740 1 -- 172.21.15.83:0/47579156 shutdown_connections 2024-04-24T09:03:57.833 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.830+0000 7f1ac2601740 1 --2- 172.21.15.83:0/47579156 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x558aa9dc7290 0x558aa9dc9680 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:57.833 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.830+0000 7f1ac2601740 1 --2- 172.21.15.83:0/47579156 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558aa9db7390 0x558aa9db4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:57.833 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.830+0000 7f1ac2601740 1 --2- 172.21.15.83:0/47579156 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558aa9dacad0 0x558aa9db4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:57.833 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.830+0000 7f1aa6a00640 0 client.19980 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:03:57.833 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.830+0000 7f1ac2601740 1 --2- 172.21.15.83:0/47579156 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558aa9dac1b0 0x558aa9db42f0 unknown :-1 s=CLOSED pgs=1940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:57.833 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.830+0000 7f1ac2601740 1 -- 172.21.15.83:0/47579156 >> 172.21.15.83:0/47579156 conn(0x558aa9da86c0 msgr2=0x558aa9dae440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:57.833 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.830+0000 7f1ac2601740 1 -- 172.21.15.83:0/47579156 shutdown_connections 2024-04-24T09:03:57.833 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.830+0000 7f1ac2601740 1 -- 172.21.15.83:0/47579156 wait complete. 2024-04-24T09:03:57.833 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.830+0000 7f1ac2601740 20 client.19980 trim_cache size 0 max 16384 2024-04-24T09:03:57.847 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.846+0000 7f1ac2601740 1 Processor -- start 2024-04-24T09:03:57.847 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.846+0000 7f1ac2601740 1 -- start start 2024-04-24T09:03:57.847 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.847+0000 7f1ac2601740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558aa9dac1b0 0x558aa9daeb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:57.847 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.847+0000 7f1ac2601740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558aa9dacad0 0x558aa9dcc6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:57.847 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.847+0000 7f1ac2601740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558aa9db7390 0x558aa9dccbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:57.847 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.847+0000 7f1ac2601740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab81b5b60 con 0x558aa9dac1b0 2024-04-24T09:03:57.847 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.847+0000 7f1ac2601740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab81b77c0 con 0x558aa9db7390 2024-04-24T09:03:57.847 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.847+0000 7f1ac2601740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab81b6230 con 0x558aa9dacad0 2024-04-24T09:03:57.847 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.847+0000 7f1abce00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558aa9dac1b0 0x558aa9daeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:57.847 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.847+0000 7f1abce00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558aa9dac1b0 0x558aa9daeb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52466/0 (socket says 172.21.15.83:52466) 2024-04-24T09:03:57.847 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.847+0000 7f1abce00640 1 -- 172.21.15.83:0/1596745214 learned_addr learned my addr 172.21.15.83:0/1596745214 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:03:57.849 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.847+0000 7f1abce00640 1 -- 172.21.15.83:0/1596745214 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558aa9dacad0 msgr2=0x558aa9dcc6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:57.849 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.847+0000 7f1abce00640 1 --2- 172.21.15.83:0/1596745214 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558aa9dacad0 0x558aa9dcc6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:57.849 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.847+0000 7f1abce00640 1 -- 172.21.15.83:0/1596745214 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558aa9db7390 msgr2=0x558aa9dccbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:57.849 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.847+0000 7f1abce00640 1 --2- 172.21.15.83:0/1596745214 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558aa9db7390 0x558aa9dccbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:57.849 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.847+0000 7f1abce00640 1 -- 172.21.15.83:0/1596745214 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558aa9af1620 con 0x558aa9dac1b0 2024-04-24T09:03:57.849 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.848+0000 7f1abce00640 1 --2- 172.21.15.83:0/1596745214 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558aa9dac1b0 0x558aa9daeb70 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7f1aa8099370 tx=0x7f1aa809c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8da6cb1dcc296d84 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:57.849 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.848+0000 7f1aad600640 1 -- 172.21.15.83:0/1596745214 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa805eb40 con 0x558aa9dac1b0 2024-04-24T09:03:57.849 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.848+0000 7f1aad600640 1 -- 172.21.15.83:0/1596745214 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1aa800de20 con 0x558aa9dac1b0 2024-04-24T09:03:57.849 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.848+0000 7f1aad600640 1 -- 172.21.15.83:0/1596745214 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa8014210 con 0x558aa9dac1b0 2024-04-24T09:03:57.849 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.848+0000 7f1ac2601740 1 -- 172.21.15.83:0/1596745214 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558aa9dac1b0 msgr2=0x558aa9daeb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:57.849 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.848+0000 7f1ac2601740 1 --2- 172.21.15.83:0/1596745214 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558aa9dac1b0 0x558aa9daeb70 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7f1aa8099370 tx=0x7f1aa809c9a0 comp rx=0 tx=0).stop 2024-04-24T09:03:57.849 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.848+0000 7f1ac2601740 1 -- 172.21.15.83:0/1596745214 shutdown_connections 2024-04-24T09:03:57.850 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.848+0000 7f1ac2601740 1 --2- 172.21.15.83:0/1596745214 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558aa9db7390 0x558aa9dccbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:57.850 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.848+0000 7f1ac2601740 1 --2- 172.21.15.83:0/1596745214 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558aa9dacad0 0x558aa9dcc6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:57.850 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.848+0000 7f1ac2601740 1 --2- 172.21.15.83:0/1596745214 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558aa9dac1b0 0x558aa9daeb70 unknown :-1 s=CLOSED pgs=1941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:57.850 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.848+0000 7f1ac2601740 1 -- 172.21.15.83:0/1596745214 >> 172.21.15.83:0/1596745214 conn(0x558aa9da86c0 msgr2=0x558aa9dc2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:57.850 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.849+0000 7f1ac2601740 1 -- 172.21.15.83:0/1596745214 shutdown_connections 2024-04-24T09:03:57.850 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.849+0000 7f1ac2601740 1 -- 172.21.15.83:0/1596745214 wait complete. 2024-04-24T09:03:57.850 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.849+0000 7f1ac2601740 1 -- start start 2024-04-24T09:03:57.850 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.849+0000 7f1ac2601740 1 Processor -- start 2024-04-24T09:03:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.850+0000 7f1ac2601740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558aa9dac1b0 0x558aa9dca250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.850+0000 7f1ac2601740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558aa9dacad0 0x558aa9dca790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.850+0000 7f1ac2601740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558aa9db7390 0x558aa9dcacd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.850+0000 7f1ac2601740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab81b7090 con 0x558aa9dac1b0 2024-04-24T09:03:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.850+0000 7f1ac2601740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab81b6960 con 0x558aa9db7390 2024-04-24T09:03:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.850+0000 7f1ac2601740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab81b3b60 con 0x558aa9dacad0 2024-04-24T09:03:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.850+0000 7f1ac2601740 10 client.0 _ll_register_callbacks cb 0x7f1aa80e2070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:03:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.850+0000 7f1abce00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558aa9dac1b0 0x558aa9dca250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.850+0000 7f1abce00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558aa9dac1b0 0x558aa9dca250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52474/0 (socket says 172.21.15.83:52474) 2024-04-24T09:03:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.850+0000 7f1abce00640 1 -- 172.21.15.83:0/2287023043 learned_addr learned my addr 172.21.15.83:0/2287023043 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:03:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.850+0000 7f1abce00640 1 -- 172.21.15.83:0/2287023043 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558aa9dacad0 msgr2=0x558aa9dca790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.850+0000 7f1abce00640 1 --2- 172.21.15.83:0/2287023043 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558aa9dacad0 0x558aa9dca790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.850+0000 7f1abce00640 1 -- 172.21.15.83:0/2287023043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558aa9db7390 msgr2=0x558aa9dcacd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:03:57.855 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.850+0000 7f1abce00640 1 --2- 172.21.15.83:0/2287023043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558aa9db7390 0x558aa9dcacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:57.855 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.850+0000 7f1abce00640 1 -- 172.21.15.83:0/2287023043 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558aa9e2c670 con 0x558aa9dac1b0 2024-04-24T09:03:57.855 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.850+0000 7f1abce00640 1 --2- 172.21.15.83:0/2287023043 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558aa9dac1b0 0x558aa9dca250 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7f1aa8007590 tx=0x7f1aa80d4c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ec2c0aacce4fcb3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:03:57.855 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.850+0000 7f1aa6a00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:03:57.855 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.851+0000 7f1aa6a00640 1 -- 172.21.15.83:0/2287023043 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa8066060 con 0x558aa9dac1b0 2024-04-24T09:03:57.855 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.851+0000 7f1aa6a00640 1 -- 172.21.15.83:0/2287023043 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9583+0+0 (secure 0 0 0) 0x7f1aa80d9c60 con 0x558aa9dac1b0 2024-04-24T09:03:57.855 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.851+0000 7f1ac2601740 1 -- 172.21.15.83:0/2287023043 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1aa800de20 con 0x558aa9dac1b0 2024-04-24T09:03:57.855 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.851+0000 7f1aa6a00640 1 -- 172.21.15.83:0/2287023043 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f1aa8018cb0 con 0x558aa9dac1b0 2024-04-24T09:03:57.855 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.851+0000 7f1ac2601740 10 client.19986 fetch_fsmap learned FSMap version 41 2024-04-24T09:03:57.855 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.851+0000 7f1ac2601740 1 -- 172.21.15.83:0/2287023043 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x558aa9e2c670 con 0x558aa9dac1b0 2024-04-24T09:03:57.855 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.851+0000 7f1aa6a00640 1 -- 172.21.15.83:0/2287023043 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1aa800de20 con 0x558aa9dac1b0 2024-04-24T09:03:57.856 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.851+0000 7f1aa6a00640 1 -- 172.21.15.83:0/2287023043 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7f1aa805ece0 con 0x558aa9dac1b0 2024-04-24T09:03:57.856 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.851+0000 7f1ac2601740 10 client.19986 fetch_fsmap finished waiting for FSMap version 41 2024-04-24T09:03:57.856 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.851+0000 7f1ac2601740 10 client.19986 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:03:57.856 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.851+0000 7f1ac2601740 20 client.19986 populate_metadata read hostname 'smithi083' 2024-04-24T09:03:57.856 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.851+0000 7f1ac2601740 1 --2- 172.21.15.83:0/2287023043 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x558aa9e2dcb0 0x558aa9e30110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:03:57.856 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.851+0000 7f1ac2601740 4 client.19986 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T09:03:57.856 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.851+0000 7f1ac2601740 1 -- 172.21.15.83:0/2287023043 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558aa9c964b0 con 0x558aa9e2dcb0 2024-04-24T09:03:57.856 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.852+0000 7f1aafe00640 1 --2- 172.21.15.83:0/2287023043 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x558aa9e2dcb0 0x558aa9e30110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:03:57.856 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.853+0000 7f1aafe00640 1 --2- 172.21.15.83:0/2287023043 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x558aa9e2dcb0 0x558aa9e30110 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=abc3be42d0f405bf server_cookie=9b13fa800cfaa28 in_seq=0 out_seq=0 2024-04-24T09:03:57.856 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.853+0000 7f1aa6a00640 10 client.19986 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:03:57.911 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.911+0000 7f1aa6a00640 1 -- 172.21.15.83:0/2287023043 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558aa9c964b0 con 0x558aa9e2dcb0 2024-04-24T09:03:57.911 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.911+0000 7f1aa6a00640 10 client.19986 handle_command_reply: tid=0 2024-04-24T09:03:57.911 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.911+0000 7f1ac2601740 1 client.19986 shutdown 2024-04-24T09:03:57.912 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.911+0000 7f1ac2601740 1 -- 172.21.15.83:0/2287023043 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558aa9dac1b0 msgr2=0x558aa9dca250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:57.913 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.911+0000 7f1ac2601740 1 --2- 172.21.15.83:0/2287023043 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558aa9dac1b0 0x558aa9dca250 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7f1aa8007590 tx=0x7f1aa80d4c00 comp rx=0 tx=0).stop 2024-04-24T09:03:57.913 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.912+0000 7f1ac2601740 1 -- 172.21.15.83:0/2287023043 shutdown_connections 2024-04-24T09:03:57.913 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.912+0000 7f1ac2601740 1 --2- 172.21.15.83:0/2287023043 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x558aa9e2dcb0 0x558aa9e30110 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:57.913 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.912+0000 7f1ac2601740 1 --2- 172.21.15.83:0/2287023043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558aa9db7390 0x558aa9dcacd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:57.913 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.912+0000 7f1ac2601740 1 --2- 172.21.15.83:0/2287023043 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558aa9dacad0 0x558aa9dca790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:57.913 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.912+0000 7f1ac2601740 1 --2- 172.21.15.83:0/2287023043 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558aa9dac1b0 0x558aa9dca250 unknown :-1 s=CLOSED pgs=1942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:57.913 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.912+0000 7f1ac2601740 1 -- 172.21.15.83:0/2287023043 >> 172.21.15.83:0/2287023043 conn(0x558aa9da86c0 msgr2=0x558aa9dc2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:57.913 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.912+0000 7f1aa6a00640 0 client.19986 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:03:57.913 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.912+0000 7f1ac2601740 1 -- 172.21.15.83:0/2287023043 shutdown_connections 2024-04-24T09:03:57.913 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.912+0000 7f1ac2601740 1 -- 172.21.15.83:0/2287023043 wait complete. 2024-04-24T09:03:57.913 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.912+0000 7f1ac2601740 20 client.19986 trim_cache size 0 max 16384 2024-04-24T09:03:57.914 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T09:03:57.914 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T09:03:57.914 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T09:03:57.914 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T09:03:57.915 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.913+0000 7f1abf800640 1 -- 172.21.15.83:0/1720791744 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f1a980a3dc0 msgr2=0x7f1a980a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:57.915 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.913+0000 7f1abf800640 1 --2- 172.21.15.83:0/1720791744 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f1a980a3dc0 0x7f1a980a6280 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f1ab002add0 tx=0x7f1ab0005ef0 comp rx=0 tx=0).stop 2024-04-24T09:03:57.915 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.913+0000 7f1abf800640 1 -- 172.21.15.83:0/1720791744 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ab8076530 msgr2=0x7f1ab81a4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:03:57.915 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.913+0000 7f1abf800640 1 --2- 172.21.15.83:0/1720791744 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ab8076530 0x7f1ab81a4900 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7f1aa800c920 tx=0x7f1aa800cde0 comp rx=0 tx=0).stop 2024-04-24T09:03:57.915 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.914+0000 7f1abf800640 1 -- 172.21.15.83:0/1720791744 shutdown_connections 2024-04-24T09:03:57.915 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.914+0000 7f1abf800640 1 --2- 172.21.15.83:0/1720791744 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f1a980a3dc0 0x7f1a980a6280 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:57.915 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.914+0000 7f1abf800640 1 --2- 172.21.15.83:0/1720791744 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1ab8112ea0 0x7f1ab819eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:57.915 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.914+0000 7f1abf800640 1 --2- 172.21.15.83:0/1720791744 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ab8076e50 0x7f1ab81a4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:57.915 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.914+0000 7f1abf800640 1 --2- 172.21.15.83:0/1720791744 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ab8076530 0x7f1ab81a4900 unknown :-1 s=CLOSED pgs=1938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:03:57.916 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.914+0000 7f1abf800640 1 -- 172.21.15.83:0/1720791744 >> 172.21.15.83:0/1720791744 conn(0x7f1ab80ffd30 msgr2=0x7f1ab8102120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:03:57.916 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.914+0000 7f1abf800640 1 -- 172.21.15.83:0/1720791744 shutdown_connections 2024-04-24T09:03:57.916 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:03:57.914+0000 7f1abf800640 1 -- 172.21.15.83:0/1720791744 wait complete. 2024-04-24T09:03:58.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:03:58 smithi138 ceph-mon[31441]: pgmap v387: 129 pgs: 129 active+clean; 67 MiB data, 927 MiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 310 KiB/s wr, 32 op/s 2024-04-24T09:03:58.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:03:58 smithi189 ceph-mon[31791]: pgmap v387: 129 pgs: 129 active+clean; 67 MiB data, 927 MiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 310 KiB/s wr, 32 op/s 2024-04-24T09:03:58.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:03:58 smithi083 ceph-mon[27907]: pgmap v387: 129 pgs: 129 active+clean; 67 MiB data, 927 MiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 310 KiB/s wr, 32 op/s 2024-04-24T09:03:58.623 INFO:tasks.workunit.client.0.smithi083.stdout: 1 71165 0.60 MB/sec execute 512 sec latency 4099.960 ms 2024-04-24T09:03:59.623 INFO:tasks.workunit.client.0.smithi083.stdout: 1 71223 0.60 MB/sec execute 513 sec latency 51.130 ms 2024-04-24T09:04:00.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:04:00 smithi138 ceph-mon[31441]: pgmap v388: 129 pgs: 129 active+clean; 67 MiB data, 936 MiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 341 KiB/s wr, 42 op/s 2024-04-24T09:04:00.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:04:00 smithi189 ceph-mon[31791]: pgmap v388: 129 pgs: 129 active+clean; 67 MiB data, 936 MiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 341 KiB/s wr, 42 op/s 2024-04-24T09:04:00.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:04:00 smithi083 ceph-mon[27907]: pgmap v388: 129 pgs: 129 active+clean; 67 MiB data, 936 MiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 341 KiB/s wr, 42 op/s 2024-04-24T09:04:00.623 INFO:tasks.workunit.client.0.smithi083.stdout: 1 71281 0.59 MB/sec execute 514 sec latency 48.447 ms 2024-04-24T09:04:01.623 INFO:tasks.workunit.client.0.smithi083.stdout: 1 71348 0.59 MB/sec execute 515 sec latency 50.081 ms 2024-04-24T09:04:02.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:04:02 smithi189 ceph-mon[31791]: pgmap v389: 129 pgs: 129 active+clean; 67 MiB data, 936 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 59 KiB/s wr, 21 op/s 2024-04-24T09:04:02.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:04:02 smithi083 ceph-mon[27907]: pgmap v389: 129 pgs: 129 active+clean; 67 MiB data, 936 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 59 KiB/s wr, 21 op/s 2024-04-24T09:04:02.623 INFO:tasks.workunit.client.0.smithi083.stdout: 1 71409 0.59 MB/sec execute 516 sec latency 69.037 ms 2024-04-24T09:04:02.704 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:04:02 smithi138 ceph-mon[31441]: pgmap v389: 129 pgs: 129 active+clean; 67 MiB data, 936 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 59 KiB/s wr, 21 op/s 2024-04-24T09:04:03.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:04:03 smithi189 ceph-mon[31791]: osdmap e109: 12 total, 12 up, 12 in 2024-04-24T09:04:03.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:04:03 smithi083 ceph-mon[27907]: osdmap e109: 12 total, 12 up, 12 in 2024-04-24T09:04:03.623 INFO:tasks.workunit.client.0.smithi083.stdout: 1 71470 0.59 MB/sec execute 517 sec latency 53.571 ms 2024-04-24T09:04:03.704 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:04:03 smithi138 ceph-mon[31441]: osdmap e109: 12 total, 12 up, 12 in 2024-04-24T09:04:04.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:04:04 smithi083 ceph-mon[27907]: pgmap v391: 129 pgs: 129 active+clean; 68 MiB data, 944 MiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 348 KiB/s wr, 62 op/s 2024-04-24T09:04:04.624 INFO:tasks.workunit.client.0.smithi083.stdout: 1 71543 0.59 MB/sec execute 518 sec latency 52.262 ms 2024-04-24T09:04:04.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:04:04 smithi138 ceph-mon[31441]: pgmap v391: 129 pgs: 129 active+clean; 68 MiB data, 944 MiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 348 KiB/s wr, 62 op/s 2024-04-24T09:04:04.706 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:04:04 smithi189 ceph-mon[31791]: pgmap v391: 129 pgs: 129 active+clean; 68 MiB data, 944 MiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 348 KiB/s wr, 62 op/s 2024-04-24T09:04:05.568 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T09:04:05.569 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T09:04:05.624 INFO:tasks.workunit.client.0.smithi083.stdout: 1 71629 0.59 MB/sec execute 519 sec latency 59.837 ms 2024-04-24T09:04:05.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.787+0000 7fb957400640 1 -- 172.21.15.83:0/3655794051 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb95010e8f0 msgr2=0x7fb950110d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:05.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.787+0000 7fb957400640 1 --2- 172.21.15.83:0/3655794051 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb95010e8f0 0x7fb950110d00 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7fb948009ca0 tx=0x7fb94802a940 comp rx=0 tx=0).stop 2024-04-24T09:04:05.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.787+0000 7fb957400640 1 -- 172.21.15.83:0/3655794051 shutdown_connections 2024-04-24T09:04:05.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.787+0000 7fb957400640 1 --2- 172.21.15.83:0/3655794051 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb95010e8f0 0x7fb950110d00 unknown :-1 s=CLOSED pgs=1943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:05.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.787+0000 7fb957400640 1 --2- 172.21.15.83:0/3655794051 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb950074cf0 0x7fb95010e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:05.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.787+0000 7fb957400640 1 --2- 172.21.15.83:0/3655794051 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9500743d0 0x7fb9500747b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:05.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.787+0000 7fb957400640 1 -- 172.21.15.83:0/3655794051 >> 172.21.15.83:0/3655794051 conn(0x7fb9500fdb60 msgr2=0x7fb9500fff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:05.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.787+0000 7fb957400640 1 -- 172.21.15.83:0/3655794051 shutdown_connections 2024-04-24T09:04:05.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.787+0000 7fb957400640 1 -- 172.21.15.83:0/3655794051 wait complete. 2024-04-24T09:04:05.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.788+0000 7fb957400640 1 Processor -- start 2024-04-24T09:04:05.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.788+0000 7fb957400640 1 -- start start 2024-04-24T09:04:05.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.788+0000 7fb957400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb9500743d0 0x7fb9501a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:05.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.788+0000 7fb957400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb950074cf0 0x7fb9501a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:05.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.788+0000 7fb957400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb95010e8f0 0x7fb95019a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:05.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.788+0000 7fb957400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb9501144c0 con 0x7fb9500743d0 2024-04-24T09:04:05.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.788+0000 7fb957400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb950114340 con 0x7fb95010e8f0 2024-04-24T09:04:05.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.788+0000 7fb957400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb950114640 con 0x7fb950074cf0 2024-04-24T09:04:05.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.789+0000 7fb954a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb9500743d0 0x7fb9501a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:05.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.789+0000 7fb954a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb9500743d0 0x7fb9501a06b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48668/0 (socket says 172.21.15.83:48668) 2024-04-24T09:04:05.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.789+0000 7fb954a00640 1 -- 172.21.15.83:0/1492191007 learned_addr learned my addr 172.21.15.83:0/1492191007 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:04:05.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.789+0000 7fb94fe00640 1 --2- 172.21.15.83:0/1492191007 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb950074cf0 0x7fb9501a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:05.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.789+0000 7fb955400640 1 --2- 172.21.15.83:0/1492191007 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb95010e8f0 0x7fb95019a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:05.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.789+0000 7fb954a00640 1 -- 172.21.15.83:0/1492191007 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb950074cf0 msgr2=0x7fb9501a0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:05.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.789+0000 7fb954a00640 1 --2- 172.21.15.83:0/1492191007 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb950074cf0 0x7fb9501a0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:05.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.789+0000 7fb954a00640 1 -- 172.21.15.83:0/1492191007 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb95010e8f0 msgr2=0x7fb95019a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:05.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.789+0000 7fb954a00640 1 --2- 172.21.15.83:0/1492191007 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb95010e8f0 0x7fb95019a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:05.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.789+0000 7fb954a00640 1 -- 172.21.15.83:0/1492191007 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb95019aff0 con 0x7fb9500743d0 2024-04-24T09:04:05.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.790+0000 7fb954a00640 1 --2- 172.21.15.83:0/1492191007 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb9500743d0 0x7fb9501a06b0 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7fb94400c890 tx=0x7fb94400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:05.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.790+0000 7fb94d600640 1 -- 172.21.15.83:0/1492191007 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb944004220 con 0x7fb9500743d0 2024-04-24T09:04:05.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.790+0000 7fb94d600640 1 -- 172.21.15.83:0/1492191007 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb9440043c0 con 0x7fb9500743d0 2024-04-24T09:04:05.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.790+0000 7fb94d600640 1 -- 172.21.15.83:0/1492191007 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb944004a00 con 0x7fb9500743d0 2024-04-24T09:04:05.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.790+0000 7fb957400640 1 -- 172.21.15.83:0/1492191007 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb95019b2e0 con 0x7fb9500743d0 2024-04-24T09:04:05.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.790+0000 7fb957400640 1 -- 172.21.15.83:0/1492191007 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9501a7500 con 0x7fb9500743d0 2024-04-24T09:04:05.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.793+0000 7fb95a20b740 1 Processor -- start 2024-04-24T09:04:05.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.793+0000 7fb94d600640 1 -- 172.21.15.83:0/1492191007 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb944004ba0 con 0x7fb9500743d0 2024-04-24T09:04:05.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.794+0000 7fb95a20b740 1 -- start start 2024-04-24T09:04:05.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.794+0000 7fb95a20b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7f7ab5860 0x55b7f7ab5c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:05.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.794+0000 7fb95a20b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b7f7ab6180 0x55b7f7ac0310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:05.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.794+0000 7fb95a20b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b7f7ac0850 0x55b7f7ac2c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:05.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.794+0000 7fb95a20b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55b7f7a1b1f0 con 0x55b7f7ab5860 2024-04-24T09:04:05.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.794+0000 7fb95a20b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b7f79c1350 con 0x55b7f7ac0850 2024-04-24T09:04:05.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.794+0000 7fb95a20b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55b7f7a171b0 con 0x55b7f7ab6180 2024-04-24T09:04:05.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.794+0000 7fb955400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7f7ab5860 0x55b7f7ab5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:05.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.794+0000 7fb955400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7f7ab5860 0x55b7f7ab5c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48676/0 (socket says 172.21.15.83:48676) 2024-04-24T09:04:05.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.794+0000 7fb955400640 1 -- 172.21.15.83:0/1127476528 learned_addr learned my addr 172.21.15.83:0/1127476528 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:04:05.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.794+0000 7fb954a00640 1 --2- 172.21.15.83:0/1127476528 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b7f7ab6180 0x55b7f7ac0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:05.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.794+0000 7fb94fe00640 1 --2- 172.21.15.83:0/1127476528 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b7f7ac0850 0x55b7f7ac2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:05.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.795+0000 7fb955400640 1 -- 172.21.15.83:0/1127476528 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b7f7ab6180 msgr2=0x55b7f7ac0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:05.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.795+0000 7fb955400640 1 --2- 172.21.15.83:0/1127476528 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b7f7ab6180 0x55b7f7ac0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:05.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.795+0000 7fb955400640 1 -- 172.21.15.83:0/1127476528 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b7f7ac0850 msgr2=0x55b7f7ac2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:05.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.795+0000 7fb955400640 1 --2- 172.21.15.83:0/1127476528 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b7f7ac0850 0x55b7f7ac2c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:05.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.795+0000 7fb955400640 1 -- 172.21.15.83:0/1127476528 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7f7a15f50 con 0x55b7f7ab5860 2024-04-24T09:04:05.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.795+0000 7fb955400640 1 --2- 172.21.15.83:0/1127476528 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7f7ab5860 0x55b7f7ab5c40 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7fb948009ca0 tx=0x7fb948009880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ca25fc57df270ac server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:05.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.795+0000 7fb941c00640 1 -- 172.21.15.83:0/1127476528 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb948034040 con 0x55b7f7ab5860 2024-04-24T09:04:05.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.796+0000 7fb941c00640 1 -- 172.21.15.83:0/1127476528 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb94802f590 con 0x55b7f7ab5860 2024-04-24T09:04:05.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.796+0000 7fb941c00640 1 -- 172.21.15.83:0/1127476528 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb948030930 con 0x55b7f7ab5860 2024-04-24T09:04:05.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.796+0000 7fb94d600640 1 --2- 172.21.15.83:0/1492191007 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb9380a3da0 0x7fb9380a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:05.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.796+0000 7fb95a20b740 1 -- 172.21.15.83:0/1127476528 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7f7ab5860 msgr2=0x55b7f7ab5c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:05.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.796+0000 7fb95a20b740 1 --2- 172.21.15.83:0/1127476528 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7f7ab5860 0x55b7f7ab5c40 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7fb948009ca0 tx=0x7fb948009880 comp rx=0 tx=0).stop 2024-04-24T09:04:05.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.796+0000 7fb95a20b740 1 -- 172.21.15.83:0/1127476528 shutdown_connections 2024-04-24T09:04:05.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.796+0000 7fb95a20b740 1 --2- 172.21.15.83:0/1127476528 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b7f7ac0850 0x55b7f7ac2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:05.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.796+0000 7fb95a20b740 1 --2- 172.21.15.83:0/1127476528 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b7f7ab6180 0x55b7f7ac0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:05.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.796+0000 7fb95a20b740 1 --2- 172.21.15.83:0/1127476528 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7f7ab5860 0x55b7f7ab5c40 unknown :-1 s=CLOSED pgs=1945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:05.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.796+0000 7fb95a20b740 1 -- 172.21.15.83:0/1127476528 >> 172.21.15.83:0/1127476528 conn(0x55b7f7ab1b80 msgr2=0x55b7f7ab1f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:05.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.796+0000 7fb94d600640 1 -- 172.21.15.83:0/1492191007 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7fb9440d0950 con 0x7fb9500743d0 2024-04-24T09:04:05.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.797+0000 7fb954a00640 1 --2- 172.21.15.83:0/1492191007 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb9380a3da0 0x7fb9380a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:05.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.797+0000 7fb95a20b740 1 -- 172.21.15.83:0/1127476528 shutdown_connections 2024-04-24T09:04:05.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.797+0000 7fb95a20b740 1 -- 172.21.15.83:0/1127476528 wait complete. 2024-04-24T09:04:05.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.797+0000 7fb95a20b740 1 -- start start 2024-04-24T09:04:05.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.797+0000 7fb954a00640 1 --2- 172.21.15.83:0/1492191007 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb9380a3da0 0x7fb9380a6260 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x55b7f7ab6e20 tx=0x7fb944000bc0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:05.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.798+0000 7fb95a20b740 1 Processor -- start 2024-04-24T09:04:05.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.798+0000 7fb95a20b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b7f7ab5860 0x55b7f7abd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:05.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.798+0000 7fb95a20b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7f7ab6180 0x55b7f7abdcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:05.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.798+0000 7fb95a20b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b7f7ac0850 0x55b7f7abe1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:05.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.798+0000 7fb95a20b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55b7f7ac35e0 con 0x55b7f7ab6180 2024-04-24T09:04:05.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.798+0000 7fb95a20b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b7f79d4690 con 0x55b7f7ab5860 2024-04-24T09:04:05.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.798+0000 7fb95a20b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55b7f7ac3760 con 0x55b7f7ac0850 2024-04-24T09:04:05.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.798+0000 7fb95a20b740 10 client.0 _ll_register_callbacks cb 0x55b7f7a10650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:04:05.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.798+0000 7fb94fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7f7ab6180 0x55b7f7abdcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:05.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.798+0000 7fb94fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7f7ab6180 0x55b7f7abdcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48678/0 (socket says 172.21.15.83:48678) 2024-04-24T09:04:05.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.798+0000 7fb94fe00640 1 -- 172.21.15.83:0/3593052545 learned_addr learned my addr 172.21.15.83:0/3593052545 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:04:05.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.799+0000 7fb955400640 1 --2- 172.21.15.83:0/3593052545 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b7f7ab5860 0x55b7f7abd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:05.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.799+0000 7fb955400640 1 --2- 172.21.15.83:0/3593052545 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b7f7ac0850 0x55b7f7abe1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:05.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.799+0000 7fb94fe00640 1 -- 172.21.15.83:0/3593052545 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b7f7ac0850 msgr2=0x55b7f7abe1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:05.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.799+0000 7fb94fe00640 1 --2- 172.21.15.83:0/3593052545 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b7f7ac0850 0x55b7f7abe1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:05.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.799+0000 7fb94fe00640 1 -- 172.21.15.83:0/3593052545 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b7f7ab5860 msgr2=0x55b7f7abd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:05.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.799+0000 7fb94fe00640 1 --2- 172.21.15.83:0/3593052545 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b7f7ab5860 0x55b7f7abd770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:05.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.799+0000 7fb94fe00640 1 -- 172.21.15.83:0/3593052545 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7f79d3920 con 0x55b7f7ab6180 2024-04-24T09:04:05.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.799+0000 7fb955400640 1 --2- 172.21.15.83:0/3593052545 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b7f7ab5860 0x55b7f7abd770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:04:05.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.799+0000 7fb94fe00640 1 --2- 172.21.15.83:0/3593052545 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7f7ab6180 0x55b7f7abdcb0 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7fb93c010770 tx=0x7fb93c010c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bf5bb78d3f880e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:05.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.799+0000 7fb936000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:04:05.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.799+0000 7fb936000640 1 -- 172.21.15.83:0/3593052545 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb93c00ccc0 con 0x55b7f7ab6180 2024-04-24T09:04:05.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.799+0000 7fb936000640 1 -- 172.21.15.83:0/3593052545 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7fb93c004030 con 0x55b7f7ab6180 2024-04-24T09:04:05.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.799+0000 7fb95a20b740 1 -- 172.21.15.83:0/3593052545 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb94802f590 con 0x55b7f7ab6180 2024-04-24T09:04:05.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.800+0000 7fb936000640 1 -- 172.21.15.83:0/3593052545 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fb93c024050 con 0x55b7f7ab6180 2024-04-24T09:04:05.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.800+0000 7fb95a20b740 10 client.19998 fetch_fsmap learned FSMap version 41 2024-04-24T09:04:05.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.800+0000 7fb95a20b740 1 -- 172.21.15.83:0/3593052545 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55b7f77f77d0 con 0x55b7f7ab6180 2024-04-24T09:04:05.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.800+0000 7fb936000640 1 -- 172.21.15.83:0/3593052545 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb94802f590 con 0x55b7f7ab6180 2024-04-24T09:04:05.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.800+0000 7fb936000640 1 -- 172.21.15.83:0/3593052545 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7fb93c003ca0 con 0x55b7f7ab6180 2024-04-24T09:04:05.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.800+0000 7fb95a20b740 10 client.19998 fetch_fsmap finished waiting for FSMap version 41 2024-04-24T09:04:05.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.800+0000 7fb95a20b740 10 client.19998 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:04:05.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.801+0000 7fb95a20b740 20 client.19998 populate_metadata read hostname 'smithi083' 2024-04-24T09:04:05.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.801+0000 7fb95a20b740 1 --2- 172.21.15.83:0/3593052545 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55b7f7ad0c40 0x55b7f7ad30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:05.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.801+0000 7fb95a20b740 4 client.19998 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:04:05.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.801+0000 7fb95a20b740 1 -- 172.21.15.83:0/3593052545 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b7f7a28c10 con 0x55b7f7ad0c40 2024-04-24T09:04:05.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.801+0000 7fb955400640 1 --2- 172.21.15.83:0/3593052545 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55b7f7ad0c40 0x55b7f7ad30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:05.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.802+0000 7fb955400640 1 --2- 172.21.15.83:0/3593052545 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55b7f7ad0c40 0x55b7f7ad30b0 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=369be1810d74ed9a server_cookie=6ca8412181fabd2 in_seq=0 out_seq=0 2024-04-24T09:04:05.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.802+0000 7fb936000640 10 client.19998 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:04:05.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.805+0000 7fb936000640 1 -- 172.21.15.83:0/3593052545 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b7f7a28c10 con 0x55b7f7ad0c40 2024-04-24T09:04:05.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.805+0000 7fb936000640 10 client.19998 handle_command_reply: tid=0 2024-04-24T09:04:05.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.805+0000 7fb95a20b740 1 client.19998 shutdown 2024-04-24T09:04:05.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.806+0000 7fb95a20b740 1 -- 172.21.15.83:0/3593052545 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7f7ab6180 msgr2=0x55b7f7abdcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:05.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.806+0000 7fb95a20b740 1 --2- 172.21.15.83:0/3593052545 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7f7ab6180 0x55b7f7abdcb0 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7fb93c010770 tx=0x7fb93c010c30 comp rx=0 tx=0).stop 2024-04-24T09:04:05.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.806+0000 7fb95a20b740 1 -- 172.21.15.83:0/3593052545 shutdown_connections 2024-04-24T09:04:05.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.806+0000 7fb95a20b740 1 --2- 172.21.15.83:0/3593052545 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55b7f7ad0c40 0x55b7f7ad30b0 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:05.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.806+0000 7fb95a20b740 1 --2- 172.21.15.83:0/3593052545 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b7f7ac0850 0x55b7f7abe1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:05.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.806+0000 7fb95a20b740 1 --2- 172.21.15.83:0/3593052545 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7f7ab6180 0x55b7f7abdcb0 unknown :-1 s=CLOSED pgs=1946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:05.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.806+0000 7fb95a20b740 1 --2- 172.21.15.83:0/3593052545 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b7f7ab5860 0x55b7f7abd770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:05.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.806+0000 7fb936000640 0 client.19998 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:04:05.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.806+0000 7fb95a20b740 1 -- 172.21.15.83:0/3593052545 >> 172.21.15.83:0/3593052545 conn(0x55b7f7ab1b80 msgr2=0x55b7f7ab7860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:05.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.806+0000 7fb95a20b740 1 -- 172.21.15.83:0/3593052545 shutdown_connections 2024-04-24T09:04:05.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.806+0000 7fb95a20b740 1 -- 172.21.15.83:0/3593052545 wait complete. 2024-04-24T09:04:05.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.806+0000 7fb95a20b740 20 client.19998 trim_cache size 0 max 16384 2024-04-24T09:04:05.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.826+0000 7fb95a20b740 1 Processor -- start 2024-04-24T09:04:05.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.826+0000 7fb95a20b740 1 -- start start 2024-04-24T09:04:05.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.827+0000 7fb95a20b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b7f7ab5860 0x55b7f7acc4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:05.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.827+0000 7fb95a20b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7f7ab6180 0x55b7f7abde40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:05.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.827+0000 7fb95a20b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b7f7ac0850 0x55b7f7ad89c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:05.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.827+0000 7fb95a20b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb9501b1840 con 0x55b7f7ab6180 2024-04-24T09:04:05.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.827+0000 7fb95a20b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb9501b34a0 con 0x55b7f7ab5860 2024-04-24T09:04:05.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.827+0000 7fb95a20b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb9501b1f10 con 0x55b7f7ac0850 2024-04-24T09:04:05.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.827+0000 7fb94fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7f7ab6180 0x55b7f7abde40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:05.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.827+0000 7fb94fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7f7ab6180 0x55b7f7abde40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48684/0 (socket says 172.21.15.83:48684) 2024-04-24T09:04:05.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.827+0000 7fb94fe00640 1 -- 172.21.15.83:0/1849295363 learned_addr learned my addr 172.21.15.83:0/1849295363 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:04:05.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.827+0000 7fb955400640 1 --2- 172.21.15.83:0/1849295363 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b7f7ab5860 0x55b7f7acc4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:05.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.827+0000 7fb94fe00640 1 -- 172.21.15.83:0/1849295363 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b7f7ac0850 msgr2=0x55b7f7ad89c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:05.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.827+0000 7fb955400640 1 --2- 172.21.15.83:0/1849295363 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b7f7ac0850 0x55b7f7ad89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:05.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.827+0000 7fb94fe00640 1 --2- 172.21.15.83:0/1849295363 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b7f7ac0850 0x55b7f7ad89c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:05.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.828+0000 7fb94fe00640 1 -- 172.21.15.83:0/1849295363 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b7f7ab5860 msgr2=0x55b7f7acc4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:05.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.828+0000 7fb94fe00640 1 --2- 172.21.15.83:0/1849295363 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b7f7ab5860 0x55b7f7acc4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:05.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.828+0000 7fb94fe00640 1 -- 172.21.15.83:0/1849295363 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7f772d3f0 con 0x55b7f7ab6180 2024-04-24T09:04:05.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.828+0000 7fb955400640 1 --2- 172.21.15.83:0/1849295363 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b7f7ab5860 0x55b7f7acc4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:04:05.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.828+0000 7fb94fe00640 1 --2- 172.21.15.83:0/1849295363 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7f7ab6180 0x55b7f7abde40 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7fb93c0114e0 tx=0x7fb93c00c700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15e009e7f7d4ed27 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:05.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.828+0000 7fb941200640 1 -- 172.21.15.83:0/1849295363 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb93c02c070 con 0x55b7f7ab6180 2024-04-24T09:04:05.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.828+0000 7fb941200640 1 -- 172.21.15.83:0/1849295363 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb93c010e80 con 0x55b7f7ab6180 2024-04-24T09:04:05.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.828+0000 7fb941200640 1 -- 172.21.15.83:0/1849295363 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb93c00cea0 con 0x55b7f7ab6180 2024-04-24T09:04:05.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.829+0000 7fb95a20b740 1 -- 172.21.15.83:0/1849295363 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7f7ab6180 msgr2=0x55b7f7abde40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:05.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.829+0000 7fb95a20b740 1 --2- 172.21.15.83:0/1849295363 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7f7ab6180 0x55b7f7abde40 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7fb93c0114e0 tx=0x7fb93c00c700 comp rx=0 tx=0).stop 2024-04-24T09:04:05.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.829+0000 7fb95a20b740 1 -- 172.21.15.83:0/1849295363 shutdown_connections 2024-04-24T09:04:05.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.829+0000 7fb95a20b740 1 --2- 172.21.15.83:0/1849295363 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b7f7ac0850 0x55b7f7ad89c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:05.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.829+0000 7fb95a20b740 1 --2- 172.21.15.83:0/1849295363 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7f7ab6180 0x55b7f7abde40 unknown :-1 s=CLOSED pgs=1947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:05.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.829+0000 7fb95a20b740 1 --2- 172.21.15.83:0/1849295363 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b7f7ab5860 0x55b7f7acc4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:05.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.829+0000 7fb95a20b740 1 -- 172.21.15.83:0/1849295363 >> 172.21.15.83:0/1849295363 conn(0x55b7f7ab1b80 msgr2=0x55b7f7abd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:05.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.829+0000 7fb95a20b740 1 -- 172.21.15.83:0/1849295363 shutdown_connections 2024-04-24T09:04:05.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.829+0000 7fb95a20b740 1 -- 172.21.15.83:0/1849295363 wait complete. 2024-04-24T09:04:05.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.829+0000 7fb95a20b740 1 -- start start 2024-04-24T09:04:05.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.830+0000 7fb95a20b740 1 Processor -- start 2024-04-24T09:04:05.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.830+0000 7fb95a20b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b7f7ab5860 0x55b7f7ac8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:05.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.830+0000 7fb95a20b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7f7ab6180 0x55b7f7aba140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:05.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.830+0000 7fb95a20b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b7f7ac0850 0x55b7f7aba680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:05.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.830+0000 7fb95a20b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb9501b2d70 con 0x55b7f7ab6180 2024-04-24T09:04:05.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.830+0000 7fb95a20b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb9501b2640 con 0x55b7f7ab5860 2024-04-24T09:04:05.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.830+0000 7fb95a20b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb9501af840 con 0x55b7f7ac0850 2024-04-24T09:04:05.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.830+0000 7fb95a20b740 10 client.0 _ll_register_callbacks cb 0x7fb93c00e370 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:04:05.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.831+0000 7fb94fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7f7ab6180 0x55b7f7aba140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:05.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.831+0000 7fb94fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7f7ab6180 0x55b7f7aba140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48686/0 (socket says 172.21.15.83:48686) 2024-04-24T09:04:05.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.831+0000 7fb94fe00640 1 -- 172.21.15.83:0/128342216 learned_addr learned my addr 172.21.15.83:0/128342216 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:04:05.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.831+0000 7fb955400640 1 --2- 172.21.15.83:0/128342216 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b7f7ab5860 0x55b7f7ac8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:05.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.831+0000 7fb955400640 1 --2- 172.21.15.83:0/128342216 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b7f7ac0850 0x55b7f7aba680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:05.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.831+0000 7fb94fe00640 1 -- 172.21.15.83:0/128342216 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b7f7ac0850 msgr2=0x55b7f7aba680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:05.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.831+0000 7fb94fe00640 1 --2- 172.21.15.83:0/128342216 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b7f7ac0850 0x55b7f7aba680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:05.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.831+0000 7fb94fe00640 1 -- 172.21.15.83:0/128342216 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b7f7ab5860 msgr2=0x55b7f7ac8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:05.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.831+0000 7fb94fe00640 1 --2- 172.21.15.83:0/128342216 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b7f7ab5860 0x55b7f7ac8db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:05.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.831+0000 7fb94fe00640 1 -- 172.21.15.83:0/128342216 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7f76ed0f0 con 0x55b7f7ab6180 2024-04-24T09:04:05.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.831+0000 7fb955400640 1 --2- 172.21.15.83:0/128342216 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b7f7ab5860 0x55b7f7ac8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:04:05.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.831+0000 7fb94fe00640 1 --2- 172.21.15.83:0/128342216 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7f7ab6180 0x55b7f7aba140 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7fb93c011600 tx=0x7fb93c00cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b772c596c270962 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:05.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.831+0000 7fb936000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:04:05.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.832+0000 7fb936000640 1 -- 172.21.15.83:0/128342216 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb93c035020 con 0x55b7f7ab6180 2024-04-24T09:04:05.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.832+0000 7fb936000640 1 -- 172.21.15.83:0/128342216 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7fb93c0054a0 con 0x55b7f7ab6180 2024-04-24T09:04:05.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.832+0000 7fb95a20b740 1 -- 172.21.15.83:0/128342216 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b7f7a22a80 con 0x55b7f7ab6180 2024-04-24T09:04:05.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.832+0000 7fb936000640 1 -- 172.21.15.83:0/128342216 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fb93c03d050 con 0x55b7f7ab6180 2024-04-24T09:04:05.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.832+0000 7fb95a20b740 10 client.20004 fetch_fsmap learned FSMap version 41 2024-04-24T09:04:05.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.832+0000 7fb95a20b740 1 -- 172.21.15.83:0/128342216 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55b7f79e0810 con 0x55b7f7ab6180 2024-04-24T09:04:05.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.832+0000 7fb936000640 1 -- 172.21.15.83:0/128342216 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55b7f7a22a80 con 0x55b7f7ab6180 2024-04-24T09:04:05.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.833+0000 7fb936000640 1 -- 172.21.15.83:0/128342216 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7fb93c003940 con 0x55b7f7ab6180 2024-04-24T09:04:05.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.833+0000 7fb95a20b740 10 client.20004 fetch_fsmap finished waiting for FSMap version 41 2024-04-24T09:04:05.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.833+0000 7fb95a20b740 10 client.20004 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:04:05.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.833+0000 7fb95a20b740 20 client.20004 populate_metadata read hostname 'smithi083' 2024-04-24T09:04:05.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.833+0000 7fb95a20b740 1 --2- 172.21.15.83:0/128342216 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55b7f7b37510 0x55b7f7b39930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:05.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.833+0000 7fb95a20b740 4 client.20004 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T09:04:05.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.833+0000 7fb95a20b740 1 -- 172.21.15.83:0/128342216 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b7f799e360 con 0x55b7f7b37510 2024-04-24T09:04:05.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.833+0000 7fb955400640 1 --2- 172.21.15.83:0/128342216 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55b7f7b37510 0x55b7f7b39930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:05.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.835+0000 7fb955400640 1 --2- 172.21.15.83:0/128342216 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55b7f7b37510 0x55b7f7b39930 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2d6c4e8f8dada93 server_cookie=70c1836d9e32c20e in_seq=0 out_seq=0 2024-04-24T09:04:05.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.835+0000 7fb936000640 10 client.20004 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:04:05.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.835+0000 7fb936000640 1 -- 172.21.15.83:0/128342216 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55b7f799e360 con 0x55b7f7b37510 2024-04-24T09:04:05.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.835+0000 7fb936000640 10 client.20004 handle_command_reply: tid=0 2024-04-24T09:04:05.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.835+0000 7fb95a20b740 1 client.20004 shutdown 2024-04-24T09:04:05.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.835+0000 7fb95a20b740 1 -- 172.21.15.83:0/128342216 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7f7ab6180 msgr2=0x55b7f7aba140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:05.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.835+0000 7fb95a20b740 1 --2- 172.21.15.83:0/128342216 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7f7ab6180 0x55b7f7aba140 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7fb93c011600 tx=0x7fb93c00cf20 comp rx=0 tx=0).stop 2024-04-24T09:04:05.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.835+0000 7fb95a20b740 1 -- 172.21.15.83:0/128342216 shutdown_connections 2024-04-24T09:04:05.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.835+0000 7fb95a20b740 1 --2- 172.21.15.83:0/128342216 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55b7f7b37510 0x55b7f7b39930 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:05.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.835+0000 7fb95a20b740 1 --2- 172.21.15.83:0/128342216 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b7f7ac0850 0x55b7f7aba680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:05.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.835+0000 7fb95a20b740 1 --2- 172.21.15.83:0/128342216 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7f7ab6180 0x55b7f7aba140 unknown :-1 s=CLOSED pgs=1948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:05.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.835+0000 7fb95a20b740 1 --2- 172.21.15.83:0/128342216 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b7f7ab5860 0x55b7f7ac8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:05.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.835+0000 7fb936000640 0 client.20004 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:04:05.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.835+0000 7fb95a20b740 1 -- 172.21.15.83:0/128342216 >> 172.21.15.83:0/128342216 conn(0x55b7f7ab1b80 msgr2=0x55b7f7ab0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:05.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.836+0000 7fb95a20b740 1 -- 172.21.15.83:0/128342216 shutdown_connections 2024-04-24T09:04:05.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.836+0000 7fb95a20b740 1 -- 172.21.15.83:0/128342216 wait complete. 2024-04-24T09:04:05.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.836+0000 7fb95a20b740 20 client.20004 trim_cache size 0 max 16384 2024-04-24T09:04:05.838 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T09:04:05.839 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-04-24T09:04:05.839 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-04-24T09:04:05.839 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T09:04:05.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.836+0000 7fb957400640 1 -- 172.21.15.83:0/1492191007 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb9380a3da0 msgr2=0x7fb9380a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:05.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.836+0000 7fb957400640 1 --2- 172.21.15.83:0/1492191007 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb9380a3da0 0x7fb9380a6260 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x55b7f7ab6e20 tx=0x7fb944000bc0 comp rx=0 tx=0).stop 2024-04-24T09:04:05.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.836+0000 7fb957400640 1 -- 172.21.15.83:0/1492191007 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb9500743d0 msgr2=0x7fb9501a06b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:05.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.836+0000 7fb957400640 1 --2- 172.21.15.83:0/1492191007 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb9500743d0 0x7fb9501a06b0 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7fb94400c890 tx=0x7fb94400cd50 comp rx=0 tx=0).stop 2024-04-24T09:04:05.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.836+0000 7fb957400640 1 -- 172.21.15.83:0/1492191007 shutdown_connections 2024-04-24T09:04:05.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.836+0000 7fb957400640 1 --2- 172.21.15.83:0/1492191007 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb9380a3da0 0x7fb9380a6260 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:05.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.836+0000 7fb957400640 1 --2- 172.21.15.83:0/1492191007 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb95010e8f0 0x7fb95019a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:05.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.836+0000 7fb957400640 1 --2- 172.21.15.83:0/1492191007 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb950074cf0 0x7fb9501a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:05.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.836+0000 7fb957400640 1 --2- 172.21.15.83:0/1492191007 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb9500743d0 0x7fb9501a06b0 unknown :-1 s=CLOSED pgs=1944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:05.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.836+0000 7fb957400640 1 -- 172.21.15.83:0/1492191007 >> 172.21.15.83:0/1492191007 conn(0x7fb9500fdb60 msgr2=0x7fb9500fe870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:05.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.837+0000 7fb957400640 1 -- 172.21.15.83:0/1492191007 shutdown_connections 2024-04-24T09:04:05.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:05.837+0000 7fb957400640 1 -- 172.21.15.83:0/1492191007 wait complete. 2024-04-24T09:04:05.851 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:04:05 smithi083 ceph-mon[27907]: osdmap e110: 12 total, 12 up, 12 in 2024-04-24T09:04:05.851 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:04:05 smithi083 ceph-mon[27907]: pgmap v393: 129 pgs: 129 active+clean; 68 MiB data, 944 MiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 435 KiB/s wr, 77 op/s 2024-04-24T09:04:05.853 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T09:04:05.854 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T09:04:05.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:04:05 smithi138 ceph-mon[31441]: osdmap e110: 12 total, 12 up, 12 in 2024-04-24T09:04:05.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:04:05 smithi138 ceph-mon[31441]: pgmap v393: 129 pgs: 129 active+clean; 68 MiB data, 944 MiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 435 KiB/s wr, 77 op/s 2024-04-24T09:04:05.956 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:04:05 smithi189 ceph-mon[31791]: osdmap e110: 12 total, 12 up, 12 in 2024-04-24T09:04:05.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:04:05 smithi189 ceph-mon[31791]: pgmap v393: 129 pgs: 129 active+clean; 68 MiB data, 944 MiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 435 KiB/s wr, 77 op/s 2024-04-24T09:04:06.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.057+0000 7f1716000640 1 -- 172.21.15.83:0/935777446 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17101019b0 msgr2=0x7f171010dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:06.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.057+0000 7f1716000640 1 --2- 172.21.15.83:0/935777446 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17101019b0 0x7f171010dbc0 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7f1708009ca0 tx=0x7f170802a950 comp rx=0 tx=0).stop 2024-04-24T09:04:06.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.057+0000 7f1716000640 1 -- 172.21.15.83:0/935777446 shutdown_connections 2024-04-24T09:04:06.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.057+0000 7f1716000640 1 --2- 172.21.15.83:0/935777446 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17101019b0 0x7f171010dbc0 unknown :-1 s=CLOSED pgs=1949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:06.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.057+0000 7f1716000640 1 --2- 172.21.15.83:0/935777446 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1710100ff0 0x7f1710101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:06.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.057+0000 7f1716000640 1 --2- 172.21.15.83:0/935777446 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1710105d30 0x7f1710106110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:06.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.057+0000 7f1716000640 1 -- 172.21.15.83:0/935777446 >> 172.21.15.83:0/935777446 conn(0x7f1710076840 msgr2=0x7f1710076c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:06.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.058+0000 7f1716000640 1 -- 172.21.15.83:0/935777446 shutdown_connections 2024-04-24T09:04:06.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.058+0000 7f1716000640 1 -- 172.21.15.83:0/935777446 wait complete. 2024-04-24T09:04:06.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.059+0000 7f1716000640 1 Processor -- start 2024-04-24T09:04:06.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.059+0000 7f1716000640 1 -- start start 2024-04-24T09:04:06.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.059+0000 7f1716000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1710100ff0 0x7f171010e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:06.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.060+0000 7f1716000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f17101019b0 0x7f171010eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:06.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.060+0000 7f1716000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1710105d30 0x7f171010f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:06.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.060+0000 7f1716000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1710112140 con 0x7f1710100ff0 2024-04-24T09:04:06.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.060+0000 7f1716000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1710111fc0 con 0x7f17101019b0 2024-04-24T09:04:06.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.060+0000 7f1716000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f17101122c0 con 0x7f1710105d30 2024-04-24T09:04:06.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.060+0000 7f1714c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1710100ff0 0x7f171010e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:06.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.060+0000 7f1714c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1710100ff0 0x7f171010e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48696/0 (socket says 172.21.15.83:48696) 2024-04-24T09:04:06.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.060+0000 7f1714c00640 1 -- 172.21.15.83:0/2025356751 learned_addr learned my addr 172.21.15.83:0/2025356751 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:04:06.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.060+0000 7f1715600640 1 --2- 172.21.15.83:0/2025356751 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1710105d30 0x7f171010f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:06.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.060+0000 7f1707e00640 1 --2- 172.21.15.83:0/2025356751 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f17101019b0 0x7f171010eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:06.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.060+0000 7f1714c00640 1 -- 172.21.15.83:0/2025356751 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1710105d30 msgr2=0x7f171010f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:06.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.060+0000 7f1714c00640 1 --2- 172.21.15.83:0/2025356751 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1710105d30 0x7f171010f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:06.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.060+0000 7f1714c00640 1 -- 172.21.15.83:0/2025356751 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f17101019b0 msgr2=0x7f171010eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:06.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.060+0000 7f1714c00640 1 --2- 172.21.15.83:0/2025356751 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f17101019b0 0x7f171010eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:06.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.060+0000 7f1714c00640 1 -- 172.21.15.83:0/2025356751 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17101a6960 con 0x7f1710100ff0 2024-04-24T09:04:06.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.061+0000 7f1714c00640 1 --2- 172.21.15.83:0/2025356751 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1710100ff0 0x7f171010e650 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7f170000d950 tx=0x7f170000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:06.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.061+0000 7f170e000640 1 -- 172.21.15.83:0/2025356751 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1700017070 con 0x7f1710100ff0 2024-04-24T09:04:06.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.061+0000 7f1716000640 1 -- 172.21.15.83:0/2025356751 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17101a6c50 con 0x7f1710100ff0 2024-04-24T09:04:06.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.061+0000 7f170e000640 1 -- 172.21.15.83:0/2025356751 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1700009c30 con 0x7f1710100ff0 2024-04-24T09:04:06.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.061+0000 7f170e000640 1 -- 172.21.15.83:0/2025356751 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f170000f620 con 0x7f1710100ff0 2024-04-24T09:04:06.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.061+0000 7f1716000640 1 -- 172.21.15.83:0/2025356751 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17101a7150 con 0x7f1710100ff0 2024-04-24T09:04:06.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.063+0000 7f1716000640 1 -- 172.21.15.83:0/2025356751 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1710068d40 con 0x7f1710100ff0 2024-04-24T09:04:06.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.065+0000 7f170e000640 1 -- 172.21.15.83:0/2025356751 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f170001a070 con 0x7f1710100ff0 2024-04-24T09:04:06.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.068+0000 7f170e000640 1 --2- 172.21.15.83:0/2025356751 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f16f40a3d50 0x7f16f40a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:06.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.068+0000 7f170e000640 1 -- 172.21.15.83:0/2025356751 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7f17000d0710 con 0x7f1710100ff0 2024-04-24T09:04:06.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.068+0000 7f1707e00640 1 --2- 172.21.15.83:0/2025356751 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f16f40a3d50 0x7f16f40a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:06.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.069+0000 7f1707e00640 1 --2- 172.21.15.83:0/2025356751 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f16f40a3d50 0x7f16f40a6210 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f16fc005e10 tx=0x7f16fc005da0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:06.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.074+0000 7f170e000640 1 -- 172.21.15.83:0/2025356751 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f170009c930 con 0x7f1710100ff0 2024-04-24T09:04:06.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.337+0000 7f1716000640 1 -- 172.21.15.83:0/2025356751 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f17101a7560 con 0x7f1710100ff0 2024-04-24T09:04:06.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.339+0000 7f170e000640 1 -- 172.21.15.83:0/2025356751 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+11953 (secure 0 0 0) 0x7f170009c230 con 0x7f1710100ff0 2024-04-24T09:04:06.339 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T09:04:06.339 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":41,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/358872257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":358872257},{"type":"v1","addr":"172.21.15.83:6835","nonce":358872257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3157070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3157070851},{"type":"v1","addr":"172.21.15.83:6837","nonce":3157070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":41,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T08:48:29.422576+0000","modified":"2024-04-24T09:02:18.079759+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34520,"mds_1":34493,"mds_2":34451,"mds_3":34427,"mds_4":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14658":{"gid":14658,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6839/4085470844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":4085470844},{"type":"v1","addr":"172.21.15.83:6839","nonce":4085470844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14676":{"gid":14676,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6841/2634754210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":2634754210},{"type":"v1","addr":"172.21.15.83:6841","nonce":2634754210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24382":{"gid":24382,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/1859921310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1859921310},{"type":"v1","addr":"172.21.15.189:6833","nonce":1859921310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/2241923197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2241923197},{"type":"v1","addr":"172.21.15.189:6835","nonce":2241923197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.189:6839/2548965336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":2548965336},{"type":"v1","addr":"172.21.15.189:6839","nonce":2548965336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26686":{"gid":26686,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6837/1678817161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":1678817161},{"type":"v1","addr":"172.21.15.189:6837","nonce":1678817161}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.138:6833/1157967658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1157967658},{"type":"v1","addr":"172.21.15.138:6833","nonce":1157967658}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.138:6835/1909219758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1909219758},{"type":"v1","addr":"172.21.15.138:6835","nonce":1909219758}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34493":{"gid":34493,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.138:6837/900566754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":900566754},{"type":"v1","addr":"172.21.15.138:6837","nonce":900566754}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34520":{"gid":34520,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.138:6839/2236932393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2236932393},{"type":"v1","addr":"172.21.15.138:6839","nonce":2236932393}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34520,"qdb_cluster":[24454,34520,34427,34451,34493]},"id":1}]} 2024-04-24T09:04:06.339 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 41 2024-04-24T09:04:06.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.341+0000 7f1716000640 1 -- 172.21.15.83:0/2025356751 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f16f40a3d50 msgr2=0x7f16f40a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:06.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.341+0000 7f1716000640 1 --2- 172.21.15.83:0/2025356751 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f16f40a3d50 0x7f16f40a6210 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f16fc005e10 tx=0x7f16fc005da0 comp rx=0 tx=0).stop 2024-04-24T09:04:06.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.341+0000 7f1716000640 1 -- 172.21.15.83:0/2025356751 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1710100ff0 msgr2=0x7f171010e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:06.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.341+0000 7f1716000640 1 --2- 172.21.15.83:0/2025356751 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1710100ff0 0x7f171010e650 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7f170000d950 tx=0x7f170000de10 comp rx=0 tx=0).stop 2024-04-24T09:04:06.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.341+0000 7f1716000640 1 -- 172.21.15.83:0/2025356751 shutdown_connections 2024-04-24T09:04:06.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.341+0000 7f1716000640 1 --2- 172.21.15.83:0/2025356751 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f16f40a3d50 0x7f16f40a6210 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:06.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.341+0000 7f1716000640 1 --2- 172.21.15.83:0/2025356751 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1710105d30 0x7f171010f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:06.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.341+0000 7f1716000640 1 --2- 172.21.15.83:0/2025356751 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f17101019b0 0x7f171010eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:06.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.341+0000 7f1716000640 1 --2- 172.21.15.83:0/2025356751 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1710100ff0 0x7f171010e650 unknown :-1 s=CLOSED pgs=1950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:06.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.341+0000 7f1716000640 1 -- 172.21.15.83:0/2025356751 >> 172.21.15.83:0/2025356751 conn(0x7f1710076840 msgr2=0x7f17100fef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:06.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.342+0000 7f1716000640 1 -- 172.21.15.83:0/2025356751 shutdown_connections 2024-04-24T09:04:06.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.342+0000 7f1716000640 1 -- 172.21.15.83:0/2025356751 wait complete. 2024-04-24T09:04:06.361 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-24T09:04:06.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.576+0000 7fd84e400640 1 -- 172.21.15.83:0/1377603196 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8481063b0 msgr2=0x7fd848106820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:06.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.576+0000 7fd84e400640 1 --2- 172.21.15.83:0/1377603196 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8481063b0 0x7fd848106820 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7fd840009900 tx=0x7fd84002a480 comp rx=0 tx=0).stop 2024-04-24T09:04:06.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.576+0000 7fd84e400640 1 -- 172.21.15.83:0/1377603196 shutdown_connections 2024-04-24T09:04:06.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.576+0000 7fd84e400640 1 --2- 172.21.15.83:0/1377603196 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8481063b0 0x7fd848106820 unknown :-1 s=CLOSED pgs=1951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:06.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.576+0000 7fd84e400640 1 --2- 172.21.15.83:0/1377603196 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd8481019b0 0x7fd848105e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:06.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.576+0000 7fd84e400640 1 --2- 172.21.15.83:0/1377603196 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd848101090 0x7fd848101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:06.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.576+0000 7fd84e400640 1 -- 172.21.15.83:0/1377603196 >> 172.21.15.83:0/1377603196 conn(0x7fd848078060 msgr2=0x7fd848078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:06.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.576+0000 7fd84e400640 1 -- 172.21.15.83:0/1377603196 shutdown_connections 2024-04-24T09:04:06.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.577+0000 7fd84e400640 1 -- 172.21.15.83:0/1377603196 wait complete. 2024-04-24T09:04:06.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.577+0000 7fd84e400640 1 Processor -- start 2024-04-24T09:04:06.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.578+0000 7fd84e400640 1 -- start start 2024-04-24T09:04:06.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.578+0000 7fd84e400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd848101090 0x7fd84819ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:06.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.578+0000 7fd84e400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd8481019b0 0x7fd84819f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:06.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.578+0000 7fd84e400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8481063b0 0x7fd8481a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:06.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.578+0000 7fd84e400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd8481188c0 con 0x7fd8481063b0 2024-04-24T09:04:06.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.578+0000 7fd84e400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd848118740 con 0x7fd848101090 2024-04-24T09:04:06.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.578+0000 7fd84e400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd848118a40 con 0x7fd8481019b0 2024-04-24T09:04:06.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.578+0000 7fd847e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8481063b0 0x7fd8481a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:06.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.578+0000 7fd847e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8481063b0 0x7fd8481a36c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48708/0 (socket says 172.21.15.83:48708) 2024-04-24T09:04:06.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.578+0000 7fd847e00640 1 -- 172.21.15.83:0/1081626410 learned_addr learned my addr 172.21.15.83:0/1081626410 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:04:06.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.579+0000 7fd847400640 1 --2- 172.21.15.83:0/1081626410 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd848101090 0x7fd84819ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:06.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.579+0000 7fd846a00640 1 --2- 172.21.15.83:0/1081626410 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd8481019b0 0x7fd84819f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:06.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.579+0000 7fd847e00640 1 -- 172.21.15.83:0/1081626410 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd8481019b0 msgr2=0x7fd84819f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:06.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.579+0000 7fd847e00640 1 --2- 172.21.15.83:0/1081626410 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd8481019b0 0x7fd84819f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:06.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.579+0000 7fd847e00640 1 -- 172.21.15.83:0/1081626410 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd848101090 msgr2=0x7fd84819ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:06.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.579+0000 7fd847e00640 1 --2- 172.21.15.83:0/1081626410 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd848101090 0x7fd84819ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:06.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.579+0000 7fd847e00640 1 -- 172.21.15.83:0/1081626410 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8481a3df0 con 0x7fd8481063b0 2024-04-24T09:04:06.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.579+0000 7fd847400640 1 --2- 172.21.15.83:0/1081626410 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd848101090 0x7fd84819ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:04:06.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.579+0000 7fd847e00640 1 --2- 172.21.15.83:0/1081626410 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8481063b0 0x7fd8481a36c0 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7fd84002a9a0 tx=0x7fd840003c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:06.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.580+0000 7fd83fe00640 1 -- 172.21.15.83:0/1081626410 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd84003f070 con 0x7fd8481063b0 2024-04-24T09:04:06.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.580+0000 7fd83fe00640 1 -- 172.21.15.83:0/1081626410 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd840003e20 con 0x7fd8481063b0 2024-04-24T09:04:06.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.580+0000 7fd83fe00640 1 -- 172.21.15.83:0/1081626410 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd840035030 con 0x7fd8481063b0 2024-04-24T09:04:06.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.580+0000 7fd84e400640 1 -- 172.21.15.83:0/1081626410 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8481a4080 con 0x7fd8481063b0 2024-04-24T09:04:06.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.580+0000 7fd84e400640 1 -- 172.21.15.83:0/1081626410 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8481ab970 con 0x7fd8481063b0 2024-04-24T09:04:06.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.583+0000 7fd8513a8740 1 Processor -- start 2024-04-24T09:04:06.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.583+0000 7fd83fe00640 1 -- 172.21.15.83:0/1081626410 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd840042070 con 0x7fd8481063b0 2024-04-24T09:04:06.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.584+0000 7fd8513a8740 1 -- start start 2024-04-24T09:04:06.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.584+0000 7fd8513a8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555bb101d950 0x555bb101dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:06.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.584+0000 7fd8513a8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555bb101e270 0x555bb1028400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:06.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.584+0000 7fd8513a8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555bb1028940 0x555bb102ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:06.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.584+0000 7fd8513a8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x555bb0f831d0 con 0x555bb101d950 2024-04-24T09:04:06.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.584+0000 7fd8513a8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555bb0f29450 con 0x555bb1028940 2024-04-24T09:04:06.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.584+0000 7fd8513a8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x555bb0f7f190 con 0x555bb101e270 2024-04-24T09:04:06.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.584+0000 7fd846a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555bb101d950 0x555bb101dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:06.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.585+0000 7fd846a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555bb101d950 0x555bb101dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48720/0 (socket says 172.21.15.83:48720) 2024-04-24T09:04:06.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.585+0000 7fd846a00640 1 -- 172.21.15.83:0/2089701635 learned_addr learned my addr 172.21.15.83:0/2089701635 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:04:06.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.585+0000 7fd847400640 1 --2- 172.21.15.83:0/2089701635 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555bb101e270 0x555bb1028400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:06.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.585+0000 7fd846a00640 1 -- 172.21.15.83:0/2089701635 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555bb101e270 msgr2=0x555bb1028400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:06.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.585+0000 7fd846a00640 1 --2- 172.21.15.83:0/2089701635 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555bb101e270 0x555bb1028400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:06.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.585+0000 7fd846a00640 1 -- 172.21.15.83:0/2089701635 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555bb1028940 msgr2=0x555bb102ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:06.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.585+0000 7fd846a00640 1 --2- 172.21.15.83:0/2089701635 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555bb1028940 0x555bb102ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:06.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.585+0000 7fd846a00640 1 -- 172.21.15.83:0/2089701635 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555bb0fe6310 con 0x555bb101d950 2024-04-24T09:04:06.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.585+0000 7fd83fe00640 1 --2- 172.21.15.83:0/1081626410 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fd82c0a3d20 0x7fd82c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:06.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.585+0000 7fd846a00640 1 --2- 172.21.15.83:0/2089701635 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555bb101d950 0x555bb101dd30 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7fd830009fd0 tx=0x7fd8300128d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=977ca32091832fd3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:06.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.586+0000 7fd83fe00640 1 -- 172.21.15.83:0/1081626410 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7fd8400f14a0 con 0x7fd8481063b0 2024-04-24T09:04:06.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.586+0000 7fd847400640 1 --2- 172.21.15.83:0/1081626410 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fd82c0a3d20 0x7fd82c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:06.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.586+0000 7fd83cc00640 1 -- 172.21.15.83:0/2089701635 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd830003970 con 0x555bb101d950 2024-04-24T09:04:06.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.586+0000 7fd83cc00640 1 -- 172.21.15.83:0/2089701635 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd830016030 con 0x555bb101d950 2024-04-24T09:04:06.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.586+0000 7fd83cc00640 1 -- 172.21.15.83:0/2089701635 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd830010820 con 0x555bb101d950 2024-04-24T09:04:06.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.587+0000 7fd8513a8740 1 -- 172.21.15.83:0/2089701635 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555bb101d950 msgr2=0x555bb101dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:06.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.587+0000 7fd8513a8740 1 --2- 172.21.15.83:0/2089701635 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555bb101d950 0x555bb101dd30 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7fd830009fd0 tx=0x7fd8300128d0 comp rx=0 tx=0).stop 2024-04-24T09:04:06.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.587+0000 7fd847400640 1 --2- 172.21.15.83:0/1081626410 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fd82c0a3d20 0x7fd82c0a61e0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x555bb101ef10 tx=0x7fd83800b5a0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:06.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.587+0000 7fd8513a8740 1 -- 172.21.15.83:0/2089701635 shutdown_connections 2024-04-24T09:04:06.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.587+0000 7fd8513a8740 1 --2- 172.21.15.83:0/2089701635 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555bb1028940 0x555bb102ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:06.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.587+0000 7fd8513a8740 1 --2- 172.21.15.83:0/2089701635 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555bb101e270 0x555bb1028400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:06.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.587+0000 7fd8513a8740 1 --2- 172.21.15.83:0/2089701635 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555bb101d950 0x555bb101dd30 unknown :-1 s=CLOSED pgs=1953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:06.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.587+0000 7fd8513a8740 1 -- 172.21.15.83:0/2089701635 >> 172.21.15.83:0/2089701635 conn(0x555bb1019c70 msgr2=0x555bb101a050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:06.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.587+0000 7fd8513a8740 1 -- 172.21.15.83:0/2089701635 shutdown_connections 2024-04-24T09:04:06.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.587+0000 7fd8513a8740 1 -- 172.21.15.83:0/2089701635 wait complete. 2024-04-24T09:04:06.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.587+0000 7fd8513a8740 1 -- start start 2024-04-24T09:04:06.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.588+0000 7fd8513a8740 1 Processor -- start 2024-04-24T09:04:06.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.588+0000 7fd8513a8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555bb101d950 0x555bb1025700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:06.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.588+0000 7fd8513a8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555bb101e270 0x555bb1025c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:06.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.588+0000 7fd8513a8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555bb1028940 0x555bb1026180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:06.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.588+0000 7fd8513a8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x555bb102b770 con 0x555bb101d950 2024-04-24T09:04:06.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.588+0000 7fd8513a8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555bb0f3c810 con 0x555bb101e270 2024-04-24T09:04:06.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.588+0000 7fd8513a8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x555bb102b8f0 con 0x555bb1028940 2024-04-24T09:04:06.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.588+0000 7fd8513a8740 10 client.0 _ll_register_callbacks cb 0x555bb0f0a140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:04:06.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.589+0000 7fd847400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555bb101d950 0x555bb1025700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:06.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.589+0000 7fd847400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555bb101d950 0x555bb1025700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48732/0 (socket says 172.21.15.83:48732) 2024-04-24T09:04:06.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.589+0000 7fd847400640 1 -- 172.21.15.83:0/197669124 learned_addr learned my addr 172.21.15.83:0/197669124 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:04:06.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.589+0000 7fd846a00640 1 --2- 172.21.15.83:0/197669124 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555bb101e270 0x555bb1025c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:06.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.589+0000 7fd847e00640 1 --2- 172.21.15.83:0/197669124 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555bb1028940 0x555bb1026180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:06.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.589+0000 7fd847400640 1 -- 172.21.15.83:0/197669124 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555bb1028940 msgr2=0x555bb1026180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:06.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.589+0000 7fd847400640 1 --2- 172.21.15.83:0/197669124 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555bb1028940 0x555bb1026180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:06.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.589+0000 7fd847400640 1 -- 172.21.15.83:0/197669124 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555bb101e270 msgr2=0x555bb1025c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:06.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.589+0000 7fd847400640 1 --2- 172.21.15.83:0/197669124 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555bb101e270 0x555bb1025c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:06.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.589+0000 7fd847400640 1 -- 172.21.15.83:0/197669124 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555bb0f4d3f0 con 0x555bb101d950 2024-04-24T09:04:06.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.589+0000 7fd847e00640 1 --2- 172.21.15.83:0/197669124 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555bb1028940 0x555bb1026180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:04:06.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.589+0000 7fd847400640 1 --2- 172.21.15.83:0/197669124 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555bb101d950 0x555bb1025700 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7fd838005ca0 tx=0x7fd8380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0587b8fe0d17bc4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:06.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.589+0000 7fd835600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:04:06.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.590+0000 7fd835600640 1 -- 172.21.15.83:0/197669124 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd83800fca0 con 0x555bb101d950 2024-04-24T09:04:06.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.590+0000 7fd835600640 1 -- 172.21.15.83:0/197669124 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7fd83800fe40 con 0x555bb101d950 2024-04-24T09:04:06.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.590+0000 7fd8513a8740 1 -- 172.21.15.83:0/197669124 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd830016030 con 0x555bb101d950 2024-04-24T09:04:06.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.590+0000 7fd835600640 1 -- 172.21.15.83:0/197669124 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fd838030050 con 0x555bb101d950 2024-04-24T09:04:06.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.590+0000 7fd8513a8740 10 client.20022 fetch_fsmap learned FSMap version 41 2024-04-24T09:04:06.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.590+0000 7fd8513a8740 1 -- 172.21.15.83:0/197669124 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x555bb0c953f0 con 0x555bb101d950 2024-04-24T09:04:06.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.590+0000 7fd835600640 1 -- 172.21.15.83:0/197669124 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd830016030 con 0x555bb101d950 2024-04-24T09:04:06.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.590+0000 7fd835600640 1 -- 172.21.15.83:0/197669124 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7fd83801d410 con 0x555bb101d950 2024-04-24T09:04:06.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.591+0000 7fd8513a8740 10 client.20022 fetch_fsmap finished waiting for FSMap version 41 2024-04-24T09:04:06.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.591+0000 7fd8513a8740 10 client.20022 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-24T09:04:06.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.591+0000 7fd8513a8740 20 client.20022 populate_metadata read hostname 'smithi083' 2024-04-24T09:04:06.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.591+0000 7fd8513a8740 1 --2- 172.21.15.83:0/197669124 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x555bb1038be0 0x555bb103b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:06.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.591+0000 7fd8513a8740 4 client.20022 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:04:06.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.591+0000 7fd8513a8740 1 -- 172.21.15.83:0/197669124 --> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555bb0f90bf0 con 0x555bb1038be0 2024-04-24T09:04:06.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.591+0000 7fd846a00640 1 --2- 172.21.15.83:0/197669124 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x555bb1038be0 0x555bb103b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:06.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.592+0000 7fd846a00640 1 --2- 172.21.15.83:0/197669124 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x555bb1038be0 0x555bb103b040 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7775549e12391273 server_cookie=2bf3ecc306aae9c0 in_seq=0 out_seq=0 2024-04-24T09:04:06.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.592+0000 7fd835600640 10 client.20022 ms_handle_connect on v2:172.21.15.189:6838/2548965336 2024-04-24T09:04:06.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.594+0000 7fd835600640 1 -- 172.21.15.83:0/197669124 <== mds.4 v2:172.21.15.189:6838/2548965336 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555bb0f90bf0 con 0x555bb1038be0 2024-04-24T09:04:06.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.594+0000 7fd835600640 10 client.20022 handle_command_reply: tid=0 2024-04-24T09:04:06.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.594+0000 7fd8513a8740 1 client.20022 shutdown 2024-04-24T09:04:06.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.594+0000 7fd8513a8740 1 -- 172.21.15.83:0/197669124 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555bb101d950 msgr2=0x555bb1025700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:06.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.594+0000 7fd8513a8740 1 --2- 172.21.15.83:0/197669124 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555bb101d950 0x555bb1025700 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7fd838005ca0 tx=0x7fd8380046c0 comp rx=0 tx=0).stop 2024-04-24T09:04:06.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.594+0000 7fd8513a8740 1 -- 172.21.15.83:0/197669124 shutdown_connections 2024-04-24T09:04:06.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.594+0000 7fd8513a8740 1 --2- 172.21.15.83:0/197669124 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x555bb1038be0 0x555bb103b040 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:06.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.594+0000 7fd8513a8740 1 --2- 172.21.15.83:0/197669124 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555bb1028940 0x555bb1026180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:06.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.594+0000 7fd8513a8740 1 --2- 172.21.15.83:0/197669124 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555bb101e270 0x555bb1025c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:06.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.594+0000 7fd8513a8740 1 --2- 172.21.15.83:0/197669124 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555bb101d950 0x555bb1025700 unknown :-1 s=CLOSED pgs=1954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:06.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.594+0000 7fd8513a8740 1 -- 172.21.15.83:0/197669124 >> 172.21.15.83:0/197669124 conn(0x555bb1019c70 msgr2=0x555bb101fa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:06.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.594+0000 7fd835600640 0 client.20022 ms_handle_reset on v2:172.21.15.189:6838/2548965336 2024-04-24T09:04:06.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.594+0000 7fd8513a8740 1 -- 172.21.15.83:0/197669124 shutdown_connections 2024-04-24T09:04:06.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.594+0000 7fd8513a8740 1 -- 172.21.15.83:0/197669124 wait complete. 2024-04-24T09:04:06.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.594+0000 7fd8513a8740 20 client.20022 trim_cache size 0 max 16384 2024-04-24T09:04:06.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.611+0000 7fd8513a8740 1 Processor -- start 2024-04-24T09:04:06.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.611+0000 7fd8513a8740 1 -- start start 2024-04-24T09:04:06.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.611+0000 7fd8513a8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555bb101d950 0x555bb1034630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:06.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.611+0000 7fd8513a8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555bb101e270 0x555bb10200b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:06.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.611+0000 7fd8513a8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555bb1028940 0x555bb1023ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:06.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.611+0000 7fd8513a8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd8481b5be0 con 0x555bb101d950 2024-04-24T09:04:06.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.611+0000 7fd8513a8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd8481b7840 con 0x555bb1028940 2024-04-24T09:04:06.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.611+0000 7fd8513a8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd8481b62b0 con 0x555bb101e270 2024-04-24T09:04:06.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.611+0000 7fd847400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555bb101d950 0x555bb1034630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:06.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.612+0000 7fd847400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555bb101d950 0x555bb1034630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48734/0 (socket says 172.21.15.83:48734) 2024-04-24T09:04:06.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.612+0000 7fd847400640 1 -- 172.21.15.83:0/2835233293 learned_addr learned my addr 172.21.15.83:0/2835233293 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:04:06.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.612+0000 7fd847e00640 1 --2- 172.21.15.83:0/2835233293 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555bb1028940 0x555bb1023ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:06.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.612+0000 7fd846a00640 1 --2- 172.21.15.83:0/2835233293 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555bb101e270 0x555bb10200b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:06.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.612+0000 7fd847400640 1 -- 172.21.15.83:0/2835233293 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555bb101e270 msgr2=0x555bb10200b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:06.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.612+0000 7fd847400640 1 --2- 172.21.15.83:0/2835233293 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555bb101e270 0x555bb10200b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:06.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.612+0000 7fd847400640 1 -- 172.21.15.83:0/2835233293 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555bb1028940 msgr2=0x555bb1023ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:06.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.612+0000 7fd847400640 1 --2- 172.21.15.83:0/2835233293 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555bb1028940 0x555bb1023ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:06.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.612+0000 7fd847400640 1 -- 172.21.15.83:0/2835233293 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555bb101e0a0 con 0x555bb101d950 2024-04-24T09:04:06.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.613+0000 7fd847400640 1 --2- 172.21.15.83:0/2835233293 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555bb101d950 0x555bb1034630 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7fd838006010 tx=0x7fd838010f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebf5a8ee6a78a49d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:06.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.615+0000 7fd837e00640 1 -- 172.21.15.83:0/2835233293 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd83801d580 con 0x555bb101d950 2024-04-24T09:04:06.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.615+0000 7fd837e00640 1 -- 172.21.15.83:0/2835233293 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd838039070 con 0x555bb101d950 2024-04-24T09:04:06.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.615+0000 7fd837e00640 1 -- 172.21.15.83:0/2835233293 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd83801dea0 con 0x555bb101d950 2024-04-24T09:04:06.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.616+0000 7fd8513a8740 1 -- 172.21.15.83:0/2835233293 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555bb101d950 msgr2=0x555bb1034630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:06.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.616+0000 7fd8513a8740 1 --2- 172.21.15.83:0/2835233293 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555bb101d950 0x555bb1034630 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7fd838006010 tx=0x7fd838010f50 comp rx=0 tx=0).stop 2024-04-24T09:04:06.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.616+0000 7fd8513a8740 1 -- 172.21.15.83:0/2835233293 shutdown_connections 2024-04-24T09:04:06.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.616+0000 7fd8513a8740 1 --2- 172.21.15.83:0/2835233293 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555bb1028940 0x555bb1023ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:06.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.616+0000 7fd8513a8740 1 --2- 172.21.15.83:0/2835233293 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555bb101e270 0x555bb10200b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:06.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.616+0000 7fd8513a8740 1 --2- 172.21.15.83:0/2835233293 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555bb101d950 0x555bb1034630 unknown :-1 s=CLOSED pgs=1955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:06.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.616+0000 7fd8513a8740 1 -- 172.21.15.83:0/2835233293 >> 172.21.15.83:0/2835233293 conn(0x555bb1019c70 msgr2=0x555bb1025f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:06.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.616+0000 7fd8513a8740 1 -- 172.21.15.83:0/2835233293 shutdown_connections 2024-04-24T09:04:06.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.616+0000 7fd8513a8740 1 -- 172.21.15.83:0/2835233293 wait complete. 2024-04-24T09:04:06.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.617+0000 7fd8513a8740 1 -- start start 2024-04-24T09:04:06.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.617+0000 7fd8513a8740 1 Processor -- start 2024-04-24T09:04:06.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.617+0000 7fd8513a8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555bb101d950 0x555bb1029ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:06.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.617+0000 7fd8513a8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555bb101e270 0x555bb102a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:06.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.618+0000 7fd8513a8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555bb1028940 0x555bb102f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:06.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.618+0000 7fd8513a8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd8481b7110 con 0x555bb101d950 2024-04-24T09:04:06.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.618+0000 7fd8513a8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd8481b69e0 con 0x555bb1028940 2024-04-24T09:04:06.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.618+0000 7fd8513a8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd8481b3be0 con 0x555bb101e270 2024-04-24T09:04:06.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.618+0000 7fd8513a8740 10 client.0 _ll_register_callbacks cb 0x7fd83800baa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:04:06.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.618+0000 7fd847400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555bb101d950 0x555bb1029ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:06.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.618+0000 7fd847400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555bb101d950 0x555bb1029ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48740/0 (socket says 172.21.15.83:48740) 2024-04-24T09:04:06.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.618+0000 7fd847400640 1 -- 172.21.15.83:0/1692402194 learned_addr learned my addr 172.21.15.83:0/1692402194 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:04:06.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.618+0000 7fd847e00640 1 --2- 172.21.15.83:0/1692402194 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555bb1028940 0x555bb102f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:06.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.618+0000 7fd847400640 1 -- 172.21.15.83:0/1692402194 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555bb101e270 msgr2=0x555bb102a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:06.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.618+0000 7fd846a00640 1 --2- 172.21.15.83:0/1692402194 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555bb101e270 0x555bb102a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:06.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.618+0000 7fd847400640 1 --2- 172.21.15.83:0/1692402194 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555bb101e270 0x555bb102a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:06.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.618+0000 7fd847400640 1 -- 172.21.15.83:0/1692402194 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555bb1028940 msgr2=0x555bb102f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:06.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.618+0000 7fd847400640 1 --2- 172.21.15.83:0/1692402194 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555bb1028940 0x555bb102f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:06.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.618+0000 7fd847400640 1 -- 172.21.15.83:0/1692402194 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555bb0d5f7d0 con 0x555bb101d950 2024-04-24T09:04:06.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.619+0000 7fd847400640 1 --2- 172.21.15.83:0/1692402194 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555bb101d950 0x555bb1029ee0 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7fd83800bfd0 tx=0x7fd838012d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=632e2665c8feb7d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:06.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.619+0000 7fd835600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:04:06.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.619+0000 7fd835600640 1 -- 172.21.15.83:0/1692402194 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd83800faa0 con 0x555bb101d950 2024-04-24T09:04:06.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.619+0000 7fd835600640 1 -- 172.21.15.83:0/1692402194 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7fd83801d590 con 0x555bb101d950 2024-04-24T09:04:06.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.619+0000 7fd8513a8740 1 -- 172.21.15.83:0/1692402194 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd838039070 con 0x555bb101d950 2024-04-24T09:04:06.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.619+0000 7fd835600640 1 -- 172.21.15.83:0/1692402194 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fd83800fc40 con 0x555bb101d950 2024-04-24T09:04:06.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.619+0000 7fd8513a8740 10 client.20028 fetch_fsmap learned FSMap version 41 2024-04-24T09:04:06.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.619+0000 7fd8513a8740 1 -- 172.21.15.83:0/1692402194 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x555bb109f7c0 con 0x555bb101d950 2024-04-24T09:04:06.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.619+0000 7fd835600640 1 -- 172.21.15.83:0/1692402194 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd838039070 con 0x555bb101d950 2024-04-24T09:04:06.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.620+0000 7fd835600640 1 -- 172.21.15.83:0/1692402194 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7fd838035410 con 0x555bb101d950 2024-04-24T09:04:06.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.620+0000 7fd8513a8740 10 client.20028 fetch_fsmap finished waiting for FSMap version 41 2024-04-24T09:04:06.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.620+0000 7fd8513a8740 10 client.20028 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-24T09:04:06.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.620+0000 7fd8513a8740 20 client.20028 populate_metadata read hostname 'smithi083' 2024-04-24T09:04:06.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.620+0000 7fd8513a8740 1 --2- 172.21.15.83:0/1692402194 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x555bb109fe20 0x555bb10a2240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:06.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.620+0000 7fd8513a8740 4 client.20028 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:04:06.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.620+0000 7fd8513a8740 1 -- 172.21.15.83:0/1692402194 --> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555bb0f8aa60 con 0x555bb109fe20 2024-04-24T09:04:06.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.620+0000 7fd846a00640 1 --2- 172.21.15.83:0/1692402194 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x555bb109fe20 0x555bb10a2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:06.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.621+0000 7fd846a00640 1 --2- 172.21.15.83:0/1692402194 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x555bb109fe20 0x555bb10a2240 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a04f7c8f83ba219c server_cookie=462965cb770fdde4 in_seq=0 out_seq=0 2024-04-24T09:04:06.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.621+0000 7fd835600640 10 client.20028 ms_handle_connect on v2:172.21.15.189:6838/2548965336 2024-04-24T09:04:06.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.622+0000 7fd835600640 1 -- 172.21.15.83:0/1692402194 <== mds.4 v2:172.21.15.189:6838/2548965336 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555bb0f8aa60 con 0x555bb109fe20 2024-04-24T09:04:06.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.622+0000 7fd835600640 10 client.20028 handle_command_reply: tid=0 2024-04-24T09:04:06.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.622+0000 7fd8513a8740 1 client.20028 shutdown 2024-04-24T09:04:06.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.622+0000 7fd8513a8740 1 -- 172.21.15.83:0/1692402194 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555bb101d950 msgr2=0x555bb1029ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:06.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.622+0000 7fd8513a8740 1 --2- 172.21.15.83:0/1692402194 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555bb101d950 0x555bb1029ee0 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7fd83800bfd0 tx=0x7fd838012d00 comp rx=0 tx=0).stop 2024-04-24T09:04:06.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.622+0000 7fd8513a8740 1 -- 172.21.15.83:0/1692402194 shutdown_connections 2024-04-24T09:04:06.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.622+0000 7fd8513a8740 1 --2- 172.21.15.83:0/1692402194 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x555bb109fe20 0x555bb10a2240 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:06.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.622+0000 7fd8513a8740 1 --2- 172.21.15.83:0/1692402194 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555bb1028940 0x555bb102f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:06.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.622+0000 7fd8513a8740 1 --2- 172.21.15.83:0/1692402194 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555bb101e270 0x555bb102a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:06.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.622+0000 7fd8513a8740 1 --2- 172.21.15.83:0/1692402194 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555bb101d950 0x555bb1029ee0 unknown :-1 s=CLOSED pgs=1956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:06.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.622+0000 7fd8513a8740 1 -- 172.21.15.83:0/1692402194 >> 172.21.15.83:0/1692402194 conn(0x555bb1019c70 msgr2=0x555bb1018560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:06.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.622+0000 7fd835600640 0 client.20028 ms_handle_reset on v2:172.21.15.189:6838/2548965336 2024-04-24T09:04:06.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.622+0000 7fd8513a8740 1 -- 172.21.15.83:0/1692402194 shutdown_connections 2024-04-24T09:04:06.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.622+0000 7fd8513a8740 1 -- 172.21.15.83:0/1692402194 wait complete. 2024-04-24T09:04:06.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.622+0000 7fd8513a8740 20 client.20028 trim_cache size 0 max 16384 2024-04-24T09:04:06.625 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:04:06.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.623+0000 7fd84e400640 1 -- 172.21.15.83:0/1081626410 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fd82c0a3d20 msgr2=0x7fd82c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:06.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.623+0000 7fd84e400640 1 --2- 172.21.15.83:0/1081626410 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fd82c0a3d20 0x7fd82c0a61e0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x555bb101ef10 tx=0x7fd83800b5a0 comp rx=0 tx=0).stop 2024-04-24T09:04:06.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.623+0000 7fd84e400640 1 -- 172.21.15.83:0/1081626410 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8481063b0 msgr2=0x7fd8481a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:06.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.623+0000 7fd84e400640 1 --2- 172.21.15.83:0/1081626410 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8481063b0 0x7fd8481a36c0 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7fd84002a9a0 tx=0x7fd840003c00 comp rx=0 tx=0).stop 2024-04-24T09:04:06.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.623+0000 7fd84e400640 1 -- 172.21.15.83:0/1081626410 shutdown_connections 2024-04-24T09:04:06.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.623+0000 7fd84e400640 1 --2- 172.21.15.83:0/1081626410 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fd82c0a3d20 0x7fd82c0a61e0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:06.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.623+0000 7fd84e400640 1 --2- 172.21.15.83:0/1081626410 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8481063b0 0x7fd8481a36c0 unknown :-1 s=CLOSED pgs=1952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:06.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.623+0000 7fd84e400640 1 --2- 172.21.15.83:0/1081626410 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd8481019b0 0x7fd84819f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:06.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.623+0000 7fd84e400640 1 --2- 172.21.15.83:0/1081626410 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd848101090 0x7fd84819ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:06.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.623+0000 7fd84e400640 1 -- 172.21.15.83:0/1081626410 >> 172.21.15.83:0/1081626410 conn(0x7fd848078060 msgr2=0x7fd848103e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:06.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.623+0000 7fd84e400640 1 -- 172.21.15.83:0/1081626410 shutdown_connections 2024-04-24T09:04:06.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.623+0000 7fd84e400640 1 -- 172.21.15.83:0/1081626410 wait complete. 2024-04-24T09:04:06.626 INFO:tasks.workunit.client.0.smithi083.stdout: 1 71787 0.59 MB/sec execute 520 sec latency 45.015 ms 2024-04-24T09:04:06.642 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-24T09:04:06.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.849+0000 7f1c2f200640 1 -- 172.21.15.83:0/3482061259 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c3010c4f0 msgr2=0x7f1c3010e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:06.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.849+0000 7f1c2f200640 1 --2- 172.21.15.83:0/3482061259 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c3010c4f0 0x7f1c3010e920 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7f1c28009ca0 tx=0x7f1c2802a950 comp rx=0 tx=0).stop 2024-04-24T09:04:06.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.849+0000 7f1c2f200640 1 -- 172.21.15.83:0/3482061259 shutdown_connections 2024-04-24T09:04:06.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.849+0000 7f1c2f200640 1 --2- 172.21.15.83:0/3482061259 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c3010c4f0 0x7f1c3010e920 unknown :-1 s=CLOSED pgs=1957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:06.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.849+0000 7f1c2f200640 1 --2- 172.21.15.83:0/3482061259 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c30103a40 0x7f1c3010bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:06.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.849+0000 7f1c2f200640 1 --2- 172.21.15.83:0/3482061259 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1c30103120 0x7f1c30103500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:06.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.849+0000 7f1c2f200640 1 -- 172.21.15.83:0/3482061259 >> 172.21.15.83:0/3482061259 conn(0x7f1c300fc880 msgr2=0x7f1c300feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:06.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.850+0000 7f1c2f200640 1 -- 172.21.15.83:0/3482061259 shutdown_connections 2024-04-24T09:04:06.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.850+0000 7f1c2f200640 1 -- 172.21.15.83:0/3482061259 wait complete. 2024-04-24T09:04:06.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.850+0000 7f1c2f200640 1 Processor -- start 2024-04-24T09:04:06.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.851+0000 7f1c2f200640 1 -- start start 2024-04-24T09:04:06.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.851+0000 7f1c2f200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c30103120 0x7f1c301986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:06.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.851+0000 7f1c2f200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1c30103a40 0x7f1c30198c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:06.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.851+0000 7f1c2f200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c3010c4f0 0x7f1c3019cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:06.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.851+0000 7f1c2f200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1c30112140 con 0x7f1c3010c4f0 2024-04-24T09:04:06.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.851+0000 7f1c2f200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1c30111fc0 con 0x7f1c30103120 2024-04-24T09:04:06.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.851+0000 7f1c2f200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1c301122c0 con 0x7f1c30103a40 2024-04-24T09:04:06.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.851+0000 7f1c2e800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c3010c4f0 0x7f1c3019cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:06.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.851+0000 7f1c2e800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c3010c4f0 0x7f1c3019cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48768/0 (socket says 172.21.15.83:48768) 2024-04-24T09:04:06.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.851+0000 7f1c2e800640 1 -- 172.21.15.83:0/376793642 learned_addr learned my addr 172.21.15.83:0/376793642 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:04:06.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.852+0000 7f1c27e00640 1 --2- 172.21.15.83:0/376793642 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1c30103a40 0x7f1c30198c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:06.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.852+0000 7f1c2de00640 1 --2- 172.21.15.83:0/376793642 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c30103120 0x7f1c301986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:06.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.852+0000 7f1c2e800640 1 -- 172.21.15.83:0/376793642 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1c30103a40 msgr2=0x7f1c30198c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:06.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.852+0000 7f1c2e800640 1 --2- 172.21.15.83:0/376793642 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1c30103a40 0x7f1c30198c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:06.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.852+0000 7f1c2e800640 1 -- 172.21.15.83:0/376793642 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c30103120 msgr2=0x7f1c301986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:06.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.852+0000 7f1c2e800640 1 --2- 172.21.15.83:0/376793642 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c30103120 0x7f1c301986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:06.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.852+0000 7f1c2e800640 1 -- 172.21.15.83:0/376793642 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c3019d660 con 0x7f1c3010c4f0 2024-04-24T09:04:06.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.852+0000 7f1c2de00640 1 --2- 172.21.15.83:0/376793642 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c30103120 0x7f1c301986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:04:06.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.852+0000 7f1c2e800640 1 --2- 172.21.15.83:0/376793642 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c3010c4f0 0x7f1c3019cf30 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7f1c28005e30 tx=0x7f1c28003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:06.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.852+0000 7f1c26a00640 1 -- 172.21.15.83:0/376793642 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c28034040 con 0x7f1c3010c4f0 2024-04-24T09:04:06.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.853+0000 7f1c26a00640 1 -- 172.21.15.83:0/376793642 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1c28030850 con 0x7f1c3010c4f0 2024-04-24T09:04:06.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.853+0000 7f1c2f200640 1 -- 172.21.15.83:0/376793642 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c3019d8f0 con 0x7f1c3010c4f0 2024-04-24T09:04:06.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.853+0000 7f1c26a00640 1 -- 172.21.15.83:0/376793642 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c280046f0 con 0x7f1c3010c4f0 2024-04-24T09:04:06.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.853+0000 7f1c2f200640 1 -- 172.21.15.83:0/376793642 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c30100ca0 con 0x7f1c3010c4f0 2024-04-24T09:04:06.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.856+0000 7f1c26a00640 1 -- 172.21.15.83:0/376793642 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1c28037020 con 0x7f1c3010c4f0 2024-04-24T09:04:06.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.856+0000 7f1c37dab740 1 Processor -- start 2024-04-24T09:04:06.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.856+0000 7f1c37dab740 1 -- start start 2024-04-24T09:04:06.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.857+0000 7f1c37dab740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f610abfb40 0x55f610abff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:06.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.857+0000 7f1c37dab740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f610ac0460 0x55f610aca5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:06.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.857+0000 7f1c37dab740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f610acab30 0x55f610accf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:06.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.857+0000 7f1c37dab740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55f610a251d0 con 0x55f610abfb40 2024-04-24T09:04:06.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.857+0000 7f1c37dab740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f6109cb450 con 0x55f610acab30 2024-04-24T09:04:06.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.857+0000 7f1c37dab740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55f610a21190 con 0x55f610ac0460 2024-04-24T09:04:06.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.857+0000 7f1c27e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f610abfb40 0x55f610abff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:06.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.857+0000 7f1c27e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f610abfb40 0x55f610abff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48776/0 (socket says 172.21.15.83:48776) 2024-04-24T09:04:06.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.857+0000 7f1c27e00640 1 -- 172.21.15.83:0/4030793552 learned_addr learned my addr 172.21.15.83:0/4030793552 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:04:06.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.857+0000 7f1c2de00640 1 --2- 172.21.15.83:0/4030793552 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f610ac0460 0x55f610aca5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:06.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.858+0000 7f1c27e00640 1 --2- 172.21.15.83:0/4030793552 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f610acab30 0x55f610accf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:06.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.858+0000 7f1c27e00640 1 -- 172.21.15.83:0/4030793552 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f610ac0460 msgr2=0x55f610aca5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:06.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.858+0000 7f1c27e00640 1 --2- 172.21.15.83:0/4030793552 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f610ac0460 0x55f610aca5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:06.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.858+0000 7f1c27e00640 1 -- 172.21.15.83:0/4030793552 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f610acab30 msgr2=0x55f610accf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:06.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.858+0000 7f1c27e00640 1 --2- 172.21.15.83:0/4030793552 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f610acab30 0x55f610accf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:06.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.858+0000 7f1c27e00640 1 -- 172.21.15.83:0/4030793552 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f610a88310 con 0x55f610abfb40 2024-04-24T09:04:06.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.858+0000 7f1c26a00640 1 --2- 172.21.15.83:0/376793642 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f1c140a3d20 0x7f1c140a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:06.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.858+0000 7f1c27e00640 1 --2- 172.21.15.83:0/4030793552 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f610abfb40 0x55f610abff20 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7f1c18005fd0 tx=0x7f1c1800b640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f43ea2c8a76b7420 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:06.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.858+0000 7f1c26a00640 1 -- 172.21.15.83:0/376793642 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7f1c280f1c10 con 0x7f1c3010c4f0 2024-04-24T09:04:06.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.858+0000 7f1c2de00640 1 --2- 172.21.15.83:0/376793642 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f1c140a3d20 0x7f1c140a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:06.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.858+0000 7f1c1f400640 1 -- 172.21.15.83:0/4030793552 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c18004030 con 0x55f610abfb40 2024-04-24T09:04:06.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.859+0000 7f1c1f400640 1 -- 172.21.15.83:0/4030793552 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1c180041d0 con 0x55f610abfb40 2024-04-24T09:04:06.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.859+0000 7f1c1f400640 1 -- 172.21.15.83:0/4030793552 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c180116c0 con 0x55f610abfb40 2024-04-24T09:04:06.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.859+0000 7f1c37dab740 1 -- 172.21.15.83:0/4030793552 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f610abfb40 msgr2=0x55f610abff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:06.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.859+0000 7f1c2de00640 1 --2- 172.21.15.83:0/376793642 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f1c140a3d20 0x7f1c140a61e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f1c20005fd0 tx=0x7f1c20009350 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:06.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.859+0000 7f1c37dab740 1 --2- 172.21.15.83:0/4030793552 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f610abfb40 0x55f610abff20 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7f1c18005fd0 tx=0x7f1c1800b640 comp rx=0 tx=0).stop 2024-04-24T09:04:06.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.860+0000 7f1c37dab740 1 -- 172.21.15.83:0/4030793552 shutdown_connections 2024-04-24T09:04:06.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.860+0000 7f1c37dab740 1 --2- 172.21.15.83:0/4030793552 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f610acab30 0x55f610accf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:06.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.860+0000 7f1c37dab740 1 --2- 172.21.15.83:0/4030793552 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f610ac0460 0x55f610aca5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:06.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.860+0000 7f1c37dab740 1 --2- 172.21.15.83:0/4030793552 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f610abfb40 0x55f610abff20 unknown :-1 s=CLOSED pgs=1959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:06.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.860+0000 7f1c37dab740 1 -- 172.21.15.83:0/4030793552 >> 172.21.15.83:0/4030793552 conn(0x55f610abbe60 msgr2=0x55f610abc240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:06.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.860+0000 7f1c37dab740 1 -- 172.21.15.83:0/4030793552 shutdown_connections 2024-04-24T09:04:06.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.860+0000 7f1c37dab740 1 -- 172.21.15.83:0/4030793552 wait complete. 2024-04-24T09:04:06.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.860+0000 7f1c37dab740 1 -- start start 2024-04-24T09:04:06.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.860+0000 7f1c37dab740 1 Processor -- start 2024-04-24T09:04:06.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.861+0000 7f1c37dab740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f610abfb40 0x55f610aca5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:06.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.861+0000 7f1c37dab740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f610ac0460 0x55f610ac78c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:06.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.861+0000 7f1c37dab740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f610acab30 0x55f610ac7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:06.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.861+0000 7f1c37dab740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55f610acd9c0 con 0x55f610ac0460 2024-04-24T09:04:06.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.861+0000 7f1c37dab740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f6109de810 con 0x55f610abfb40 2024-04-24T09:04:06.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.861+0000 7f1c37dab740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55f610acdb40 con 0x55f610acab30 2024-04-24T09:04:06.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.861+0000 7f1c37dab740 10 client.0 _ll_register_callbacks cb 0x55f6109ac140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:04:06.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.861+0000 7f1c27e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f610ac0460 0x55f610ac78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:06.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.861+0000 7f1c27e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f610ac0460 0x55f610ac78c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48792/0 (socket says 172.21.15.83:48792) 2024-04-24T09:04:06.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.861+0000 7f1c27e00640 1 -- 172.21.15.83:0/1059479434 learned_addr learned my addr 172.21.15.83:0/1059479434 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:04:06.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.861+0000 7f1c2de00640 1 --2- 172.21.15.83:0/1059479434 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f610abfb40 0x55f610aca5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:06.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.861+0000 7f1c2e800640 1 --2- 172.21.15.83:0/1059479434 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f610acab30 0x55f610ac7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:06.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.862+0000 7f1c27e00640 1 -- 172.21.15.83:0/1059479434 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f610acab30 msgr2=0x55f610ac7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:06.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.862+0000 7f1c27e00640 1 --2- 172.21.15.83:0/1059479434 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f610acab30 0x55f610ac7fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:06.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.862+0000 7f1c27e00640 1 -- 172.21.15.83:0/1059479434 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f610abfb40 msgr2=0x55f610aca5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:06.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.862+0000 7f1c27e00640 1 --2- 172.21.15.83:0/1059479434 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f610abfb40 0x55f610aca5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:06.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.862+0000 7f1c27e00640 1 -- 172.21.15.83:0/1059479434 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6109ef3f0 con 0x55f610ac0460 2024-04-24T09:04:06.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.862+0000 7f1c27e00640 1 --2- 172.21.15.83:0/1059479434 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f610ac0460 0x55f610ac78c0 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7f1c18003ea0 tx=0x7f1c18004150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cafc8247e3b4999 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:06.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.862+0000 7f1c13e00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:04:06.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.862+0000 7f1c13e00640 1 -- 172.21.15.83:0/1059479434 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c180035f0 con 0x55f610ac0460 2024-04-24T09:04:06.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.862+0000 7f1c13e00640 1 -- 172.21.15.83:0/1059479434 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7f1c18024070 con 0x55f610ac0460 2024-04-24T09:04:06.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.862+0000 7f1c37dab740 1 -- 172.21.15.83:0/1059479434 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c180041d0 con 0x55f610ac0460 2024-04-24T09:04:06.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.863+0000 7f1c13e00640 1 -- 172.21.15.83:0/1059479434 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f1c18032050 con 0x55f610ac0460 2024-04-24T09:04:06.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.863+0000 7f1c37dab740 10 client.20040 fetch_fsmap learned FSMap version 41 2024-04-24T09:04:06.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.863+0000 7f1c37dab740 1 -- 172.21.15.83:0/1059479434 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55f6107373f0 con 0x55f610ac0460 2024-04-24T09:04:06.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.863+0000 7f1c13e00640 1 -- 172.21.15.83:0/1059479434 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1c180041d0 con 0x55f610ac0460 2024-04-24T09:04:06.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.863+0000 7f1c13e00640 1 -- 172.21.15.83:0/1059479434 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7f1c180046c0 con 0x55f610ac0460 2024-04-24T09:04:06.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.863+0000 7f1c37dab740 10 client.20040 fetch_fsmap finished waiting for FSMap version 41 2024-04-24T09:04:06.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.863+0000 7f1c37dab740 10 client.20040 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-24T09:04:06.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.863+0000 7f1c37dab740 20 client.20040 populate_metadata read hostname 'smithi083' 2024-04-24T09:04:06.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.863+0000 7f1c37dab740 1 --2- 172.21.15.83:0/1059479434 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55f610adace0 0x55f610add0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:06.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.863+0000 7f1c37dab740 4 client.20040 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:04:06.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.863+0000 7f1c37dab740 1 -- 172.21.15.83:0/1059479434 --> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f610a32bf0 con 0x55f610adace0 2024-04-24T09:04:06.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.864+0000 7f1c2de00640 1 --2- 172.21.15.83:0/1059479434 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55f610adace0 0x55f610add0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:06.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.865+0000 7f1c2de00640 1 --2- 172.21.15.83:0/1059479434 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55f610adace0 0x55f610add0d0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f3014935e6d99702 server_cookie=2961a412ed2fd43d in_seq=0 out_seq=0 2024-04-24T09:04:06.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.865+0000 7f1c13e00640 10 client.20040 ms_handle_connect on v2:172.21.15.138:6832/1157967658 2024-04-24T09:04:06.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.867+0000 7f1c13e00640 1 -- 172.21.15.83:0/1059479434 <== mds.3 v2:172.21.15.138:6832/1157967658 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f610a32bf0 con 0x55f610adace0 2024-04-24T09:04:06.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.867+0000 7f1c13e00640 10 client.20040 handle_command_reply: tid=0 2024-04-24T09:04:06.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.867+0000 7f1c37dab740 1 client.20040 shutdown 2024-04-24T09:04:06.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.867+0000 7f1c37dab740 1 -- 172.21.15.83:0/1059479434 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f610ac0460 msgr2=0x55f610ac78c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:06.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.867+0000 7f1c37dab740 1 --2- 172.21.15.83:0/1059479434 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f610ac0460 0x55f610ac78c0 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7f1c18003ea0 tx=0x7f1c18004150 comp rx=0 tx=0).stop 2024-04-24T09:04:06.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.867+0000 7f1c37dab740 1 -- 172.21.15.83:0/1059479434 shutdown_connections 2024-04-24T09:04:06.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.867+0000 7f1c37dab740 1 --2- 172.21.15.83:0/1059479434 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55f610adace0 0x55f610add0d0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:06.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.867+0000 7f1c37dab740 1 --2- 172.21.15.83:0/1059479434 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f610acab30 0x55f610ac7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:06.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.867+0000 7f1c37dab740 1 --2- 172.21.15.83:0/1059479434 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f610ac0460 0x55f610ac78c0 unknown :-1 s=CLOSED pgs=1960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:06.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.867+0000 7f1c37dab740 1 --2- 172.21.15.83:0/1059479434 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f610abfb40 0x55f610aca5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:06.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.867+0000 7f1c13e00640 0 client.20040 ms_handle_reset on v2:172.21.15.138:6832/1157967658 2024-04-24T09:04:06.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.867+0000 7f1c37dab740 1 -- 172.21.15.83:0/1059479434 >> 172.21.15.83:0/1059479434 conn(0x55f610abbe60 msgr2=0x55f610ac1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:06.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.867+0000 7f1c37dab740 1 -- 172.21.15.83:0/1059479434 shutdown_connections 2024-04-24T09:04:06.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.867+0000 7f1c37dab740 1 -- 172.21.15.83:0/1059479434 wait complete. 2024-04-24T09:04:06.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.868+0000 7f1c37dab740 20 client.20040 trim_cache size 0 max 16384 2024-04-24T09:04:06.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.883+0000 7f1c37dab740 1 Processor -- start 2024-04-24T09:04:06.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.883+0000 7f1c37dab740 1 -- start start 2024-04-24T09:04:06.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.883+0000 7f1c37dab740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f610abfb40 0x55f610ad6880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:06.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.883+0000 7f1c37dab740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f610ac0460 0x55f610aca100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:06.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.883+0000 7f1c37dab740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f610acab30 0x55f610ae2700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:06.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.883+0000 7f1c37dab740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1c301b15b0 con 0x55f610abfb40 2024-04-24T09:04:06.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.883+0000 7f1c37dab740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1c301b3210 con 0x55f610acab30 2024-04-24T09:04:06.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.883+0000 7f1c37dab740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1c301b1c80 con 0x55f610ac0460 2024-04-24T09:04:06.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.883+0000 7f1c2de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f610abfb40 0x55f610ad6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:06.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.883+0000 7f1c2de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f610abfb40 0x55f610ad6880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48800/0 (socket says 172.21.15.83:48800) 2024-04-24T09:04:06.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.883+0000 7f1c2de00640 1 -- 172.21.15.83:0/2763272572 learned_addr learned my addr 172.21.15.83:0/2763272572 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:04:06.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.884+0000 7f1c2e800640 1 --2- 172.21.15.83:0/2763272572 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f610acab30 0x55f610ae2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:06.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.884+0000 7f1c2de00640 1 -- 172.21.15.83:0/2763272572 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f610ac0460 msgr2=0x55f610aca100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:06.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.884+0000 7f1c27e00640 1 --2- 172.21.15.83:0/2763272572 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f610ac0460 0x55f610aca100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:06.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.884+0000 7f1c2de00640 1 --2- 172.21.15.83:0/2763272572 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f610ac0460 0x55f610aca100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:06.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.884+0000 7f1c2de00640 1 -- 172.21.15.83:0/2763272572 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f610acab30 msgr2=0x55f610ae2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:06.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.884+0000 7f1c2de00640 1 --2- 172.21.15.83:0/2763272572 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f610acab30 0x55f610ae2700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:06.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.884+0000 7f1c2de00640 1 -- 172.21.15.83:0/2763272572 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6108017d0 con 0x55f610abfb40 2024-04-24T09:04:06.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.884+0000 7f1c2e800640 1 --2- 172.21.15.83:0/2763272572 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f610acab30 0x55f610ae2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:04:06.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.884+0000 7f1c2de00640 1 --2- 172.21.15.83:0/2763272572 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f610abfb40 0x55f610ad6880 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x55f610add6a0 tx=0x7f1c20016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ff82ce5bcb91c1e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:06.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.884+0000 7f1c1ea00640 1 -- 172.21.15.83:0/2763272572 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c20027e20 con 0x55f610abfb40 2024-04-24T09:04:06.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.884+0000 7f1c1ea00640 1 -- 172.21.15.83:0/2763272572 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1c20012d30 con 0x55f610abfb40 2024-04-24T09:04:06.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.884+0000 7f1c1ea00640 1 -- 172.21.15.83:0/2763272572 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c200035f0 con 0x55f610abfb40 2024-04-24T09:04:06.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.885+0000 7f1c37dab740 1 -- 172.21.15.83:0/2763272572 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f610abfb40 msgr2=0x55f610ad6880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:06.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.885+0000 7f1c37dab740 1 --2- 172.21.15.83:0/2763272572 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f610abfb40 0x55f610ad6880 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x55f610add6a0 tx=0x7f1c20016000 comp rx=0 tx=0).stop 2024-04-24T09:04:06.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.885+0000 7f1c37dab740 1 -- 172.21.15.83:0/2763272572 shutdown_connections 2024-04-24T09:04:06.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.885+0000 7f1c37dab740 1 --2- 172.21.15.83:0/2763272572 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f610acab30 0x55f610ae2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:06.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.885+0000 7f1c37dab740 1 --2- 172.21.15.83:0/2763272572 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f610ac0460 0x55f610aca100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:06.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.885+0000 7f1c37dab740 1 --2- 172.21.15.83:0/2763272572 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f610abfb40 0x55f610ad6880 unknown :-1 s=CLOSED pgs=1961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:06.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.885+0000 7f1c37dab740 1 -- 172.21.15.83:0/2763272572 >> 172.21.15.83:0/2763272572 conn(0x55f610abbe60 msgr2=0x55f610ac2bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:06.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.885+0000 7f1c37dab740 1 -- 172.21.15.83:0/2763272572 shutdown_connections 2024-04-24T09:04:06.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.885+0000 7f1c37dab740 1 -- 172.21.15.83:0/2763272572 wait complete. 2024-04-24T09:04:06.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.886+0000 7f1c37dab740 1 -- start start 2024-04-24T09:04:06.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.886+0000 7f1c37dab740 1 Processor -- start 2024-04-24T09:04:06.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.886+0000 7f1c37dab740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f610abfb40 0x55f610acb930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:06.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.886+0000 7f1c37dab740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f610ac0460 0x55f610acbe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:06.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.886+0000 7f1c37dab740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f610acab30 0x55f610acc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:06.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.886+0000 7f1c37dab740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1c301b2ae0 con 0x55f610ac0460 2024-04-24T09:04:06.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.886+0000 7f1c37dab740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1c301b23b0 con 0x55f610acab30 2024-04-24T09:04:06.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.887+0000 7f1c37dab740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1c301af5b0 con 0x55f610abfb40 2024-04-24T09:04:06.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.887+0000 7f1c37dab740 10 client.0 _ll_register_callbacks cb 0x7f1c18013400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:04:06.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.887+0000 7f1c27e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f610ac0460 0x55f610acbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:06.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.887+0000 7f1c27e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f610ac0460 0x55f610acbe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48808/0 (socket says 172.21.15.83:48808) 2024-04-24T09:04:06.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.887+0000 7f1c27e00640 1 -- 172.21.15.83:0/3269983949 learned_addr learned my addr 172.21.15.83:0/3269983949 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:04:06.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.887+0000 7f1c2de00640 1 --2- 172.21.15.83:0/3269983949 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f610abfb40 0x55f610acb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:06.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.887+0000 7f1c27e00640 1 -- 172.21.15.83:0/3269983949 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f610abfb40 msgr2=0x55f610acb930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:06.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.887+0000 7f1c27e00640 1 --2- 172.21.15.83:0/3269983949 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f610abfb40 0x55f610acb930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:06.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.887+0000 7f1c27e00640 1 -- 172.21.15.83:0/3269983949 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f610acab30 msgr2=0x55f610acc3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:06.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.887+0000 7f1c27e00640 1 --2- 172.21.15.83:0/3269983949 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f610acab30 0x55f610acc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:06.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.887+0000 7f1c27e00640 1 -- 172.21.15.83:0/3269983949 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f610ad1060 con 0x55f610ac0460 2024-04-24T09:04:06.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.887+0000 7f1c27e00640 1 --2- 172.21.15.83:0/3269983949 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f610ac0460 0x55f610acbe70 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7f1c18004cd0 tx=0x7f1c1800b460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1ef00063c1c1916 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:06.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.888+0000 7f1c13e00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:04:06.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.888+0000 7f1c13e00640 1 -- 172.21.15.83:0/3269983949 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c1803b060 con 0x55f610ac0460 2024-04-24T09:04:06.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.888+0000 7f1c13e00640 1 -- 172.21.15.83:0/3269983949 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7f1c18024040 con 0x55f610ac0460 2024-04-24T09:04:06.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.888+0000 7f1c37dab740 1 -- 172.21.15.83:0/3269983949 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c20012d30 con 0x55f610ac0460 2024-04-24T09:04:06.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.888+0000 7f1c13e00640 1 -- 172.21.15.83:0/3269983949 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f1c180041d0 con 0x55f610ac0460 2024-04-24T09:04:06.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.888+0000 7f1c37dab740 10 client.20046 fetch_fsmap learned FSMap version 41 2024-04-24T09:04:06.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.888+0000 7f1c37dab740 1 -- 172.21.15.83:0/3269983949 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55f610b49a70 con 0x55f610ac0460 2024-04-24T09:04:06.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.888+0000 7f1c13e00640 1 -- 172.21.15.83:0/3269983949 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1c20012d30 con 0x55f610ac0460 2024-04-24T09:04:06.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.889+0000 7f1c13e00640 1 -- 172.21.15.83:0/3269983949 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7f1c18005000 con 0x55f610ac0460 2024-04-24T09:04:06.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.889+0000 7f1c37dab740 10 client.20046 fetch_fsmap finished waiting for FSMap version 41 2024-04-24T09:04:06.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.889+0000 7f1c37dab740 10 client.20046 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-24T09:04:06.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.889+0000 7f1c37dab740 20 client.20046 populate_metadata read hostname 'smithi083' 2024-04-24T09:04:06.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.889+0000 7f1c37dab740 1 --2- 172.21.15.83:0/3269983949 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55f610b4a220 0x55f610b4c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:06.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.889+0000 7f1c37dab740 4 client.20046 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:04:06.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.889+0000 7f1c37dab740 1 -- 172.21.15.83:0/3269983949 --> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f610a2ca60 con 0x55f610b4a220 2024-04-24T09:04:06.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.889+0000 7f1c2de00640 1 --2- 172.21.15.83:0/3269983949 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55f610b4a220 0x55f610b4c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:06.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.890+0000 7f1c2de00640 1 --2- 172.21.15.83:0/3269983949 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55f610b4a220 0x55f610b4c640 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3b552270b73c87d3 server_cookie=1792217c9760bd51 in_seq=0 out_seq=0 2024-04-24T09:04:06.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.890+0000 7f1c13e00640 10 client.20046 ms_handle_connect on v2:172.21.15.138:6832/1157967658 2024-04-24T09:04:06.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.891+0000 7f1c13e00640 1 -- 172.21.15.83:0/3269983949 <== mds.3 v2:172.21.15.138:6832/1157967658 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f610a2ca60 con 0x55f610b4a220 2024-04-24T09:04:06.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.891+0000 7f1c13e00640 10 client.20046 handle_command_reply: tid=0 2024-04-24T09:04:06.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.891+0000 7f1c37dab740 1 client.20046 shutdown 2024-04-24T09:04:06.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.891+0000 7f1c37dab740 1 -- 172.21.15.83:0/3269983949 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f610ac0460 msgr2=0x55f610acbe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:06.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.891+0000 7f1c37dab740 1 --2- 172.21.15.83:0/3269983949 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f610ac0460 0x55f610acbe70 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7f1c18004cd0 tx=0x7f1c1800b460 comp rx=0 tx=0).stop 2024-04-24T09:04:06.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.891+0000 7f1c37dab740 1 -- 172.21.15.83:0/3269983949 shutdown_connections 2024-04-24T09:04:06.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.891+0000 7f1c37dab740 1 --2- 172.21.15.83:0/3269983949 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f610acab30 0x55f610acc3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:06.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.891+0000 7f1c37dab740 1 --2- 172.21.15.83:0/3269983949 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f610ac0460 0x55f610acbe70 unknown :-1 s=CLOSED pgs=1962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:06.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.891+0000 7f1c37dab740 1 --2- 172.21.15.83:0/3269983949 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55f610b4a220 0x55f610b4c640 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:06.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.891+0000 7f1c37dab740 1 --2- 172.21.15.83:0/3269983949 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f610abfb40 0x55f610acb930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:06.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.891+0000 7f1c37dab740 1 -- 172.21.15.83:0/3269983949 >> 172.21.15.83:0/3269983949 conn(0x55f610abbe60 msgr2=0x55f610aba6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:06.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.891+0000 7f1c13e00640 0 client.20046 ms_handle_reset on v2:172.21.15.138:6832/1157967658 2024-04-24T09:04:06.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.891+0000 7f1c37dab740 1 -- 172.21.15.83:0/3269983949 shutdown_connections 2024-04-24T09:04:06.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.891+0000 7f1c37dab740 1 -- 172.21.15.83:0/3269983949 wait complete. 2024-04-24T09:04:06.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.892+0000 7f1c37dab740 20 client.20046 trim_cache size 0 max 16384 2024-04-24T09:04:06.896 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:04:06.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.892+0000 7f1c2f200640 1 -- 172.21.15.83:0/376793642 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f1c140a3d20 msgr2=0x7f1c140a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:06.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.892+0000 7f1c2f200640 1 --2- 172.21.15.83:0/376793642 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f1c140a3d20 0x7f1c140a61e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f1c20005fd0 tx=0x7f1c20009350 comp rx=0 tx=0).stop 2024-04-24T09:04:06.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.892+0000 7f1c2f200640 1 -- 172.21.15.83:0/376793642 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c3010c4f0 msgr2=0x7f1c3019cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:06.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.892+0000 7f1c2f200640 1 --2- 172.21.15.83:0/376793642 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c3010c4f0 0x7f1c3019cf30 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7f1c28005e30 tx=0x7f1c28003f10 comp rx=0 tx=0).stop 2024-04-24T09:04:06.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.892+0000 7f1c2f200640 1 -- 172.21.15.83:0/376793642 shutdown_connections 2024-04-24T09:04:06.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.892+0000 7f1c2f200640 1 --2- 172.21.15.83:0/376793642 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f1c140a3d20 0x7f1c140a61e0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:06.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.892+0000 7f1c2f200640 1 --2- 172.21.15.83:0/376793642 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c3010c4f0 0x7f1c3019cf30 unknown :-1 s=CLOSED pgs=1958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:06.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.892+0000 7f1c2f200640 1 --2- 172.21.15.83:0/376793642 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1c30103a40 0x7f1c30198c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:06.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.892+0000 7f1c2f200640 1 --2- 172.21.15.83:0/376793642 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c30103120 0x7f1c301986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:06.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.892+0000 7f1c2f200640 1 -- 172.21.15.83:0/376793642 >> 172.21.15.83:0/376793642 conn(0x7f1c300fc880 msgr2=0x7f1c300fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:06.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.893+0000 7f1c2f200640 1 -- 172.21.15.83:0/376793642 shutdown_connections 2024-04-24T09:04:06.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:06.893+0000 7f1c2f200640 1 -- 172.21.15.83:0/376793642 wait complete. 2024-04-24T09:04:06.908 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:04:06 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/2025356751' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:04:06.910 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T09:04:06.954 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:04:06 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/2025356751' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:04:06.956 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:04:06 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/2025356751' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:04:07.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.107+0000 7f54c8c00640 1 -- 172.21.15.83:0/4165856823 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54c4102ac0 msgr2=0x7f54c4106b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:07.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.107+0000 7f54c8c00640 1 --2- 172.21.15.83:0/4165856823 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54c4102ac0 0x7f54c4106b00 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7f54b0009900 tx=0x7f54b002a410 comp rx=0 tx=0).stop 2024-04-24T09:04:07.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.107+0000 7f54c8c00640 1 -- 172.21.15.83:0/4165856823 shutdown_connections 2024-04-24T09:04:07.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.107+0000 7f54c8c00640 1 --2- 172.21.15.83:0/4165856823 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f54c41071e0 0x7f54c410edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.107+0000 7f54c8c00640 1 --2- 172.21.15.83:0/4165856823 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54c4102ac0 0x7f54c4106b00 unknown :-1 s=CLOSED pgs=1963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.107+0000 7f54c8c00640 1 --2- 172.21.15.83:0/4165856823 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f54c4102110 0x7f54c41024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.107+0000 7f54c8c00640 1 -- 172.21.15.83:0/4165856823 >> 172.21.15.83:0/4165856823 conn(0x7f54c40fd9d0 msgr2=0x7f54c40ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:07.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.107+0000 7f54c8c00640 1 -- 172.21.15.83:0/4165856823 shutdown_connections 2024-04-24T09:04:07.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.107+0000 7f54c8c00640 1 -- 172.21.15.83:0/4165856823 wait complete. 2024-04-24T09:04:07.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.108+0000 7f54c8c00640 1 Processor -- start 2024-04-24T09:04:07.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.108+0000 7f54c8c00640 1 -- start start 2024-04-24T09:04:07.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.109+0000 7f54c8c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54c4102110 0x7f54c4198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:07.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.109+0000 7f54c8c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f54c41071e0 0x7f54c4198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:07.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.109+0000 7f54c8c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f54c419cff0 0x7f54c419d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:07.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.109+0000 7f54c8c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f54c4112170 con 0x7f54c4102110 2024-04-24T09:04:07.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.109+0000 7f54c8c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f54c4111ff0 con 0x7f54c41071e0 2024-04-24T09:04:07.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.109+0000 7f54c8c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f54c41122f0 con 0x7f54c419cff0 2024-04-24T09:04:07.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.109+0000 7f54c3400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54c4102110 0x7f54c4198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:07.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.109+0000 7f54c3400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54c4102110 0x7f54c4198730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48820/0 (socket says 172.21.15.83:48820) 2024-04-24T09:04:07.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.109+0000 7f54c3400640 1 -- 172.21.15.83:0/3494412614 learned_addr learned my addr 172.21.15.83:0/3494412614 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:04:07.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.109+0000 7f54c3e00640 1 --2- 172.21.15.83:0/3494412614 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f54c419cff0 0x7f54c419d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:07.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.110+0000 7f54c3400640 1 -- 172.21.15.83:0/3494412614 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f54c419cff0 msgr2=0x7f54c419d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:07.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.110+0000 7f54c3400640 1 --2- 172.21.15.83:0/3494412614 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f54c419cff0 0x7f54c419d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.110+0000 7f54c3400640 1 -- 172.21.15.83:0/3494412614 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f54c41071e0 msgr2=0x7f54c4198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:07.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.110+0000 7f54c3400640 1 --2- 172.21.15.83:0/3494412614 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f54c41071e0 0x7f54c4198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.110+0000 7f54c3400640 1 -- 172.21.15.83:0/3494412614 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54c419dae0 con 0x7f54c4102110 2024-04-24T09:04:07.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.110+0000 7f54c3400640 1 --2- 172.21.15.83:0/3494412614 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54c4102110 0x7f54c4198730 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7f54b400d770 tx=0x7f54b400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:07.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.110+0000 7f54bbe00640 1 -- 172.21.15.83:0/3494412614 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b4017070 con 0x7f54c4102110 2024-04-24T09:04:07.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.110+0000 7f54bbe00640 1 -- 172.21.15.83:0/3494412614 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f54b40054d0 con 0x7f54c4102110 2024-04-24T09:04:07.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.110+0000 7f54c8c00640 1 -- 172.21.15.83:0/3494412614 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54c419ddd0 con 0x7f54c4102110 2024-04-24T09:04:07.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.110+0000 7f54bbe00640 1 -- 172.21.15.83:0/3494412614 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b4004cf0 con 0x7f54c4102110 2024-04-24T09:04:07.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.110+0000 7f54c8c00640 1 -- 172.21.15.83:0/3494412614 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54c41a5570 con 0x7f54c4102110 2024-04-24T09:04:07.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.115+0000 7f54ccf8f740 1 Processor -- start 2024-04-24T09:04:07.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.115+0000 7f54ccf8f740 1 -- start start 2024-04-24T09:04:07.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.115+0000 7f54bbe00640 1 -- 172.21.15.83:0/3494412614 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f54b4004e90 con 0x7f54c4102110 2024-04-24T09:04:07.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.115+0000 7f54ccf8f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56154a55bb40 0x56154a55bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:07.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.115+0000 7f54ccf8f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56154a55c460 0x56154a5665f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:07.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.115+0000 7f54ccf8f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56154a566b30 0x56154a568f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:07.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.115+0000 7f54ccf8f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56154a4c11d0 con 0x56154a566b30 2024-04-24T09:04:07.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.115+0000 7f54ccf8f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56154a467450 con 0x56154a55c460 2024-04-24T09:04:07.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.115+0000 7f54ccf8f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56154a4bd190 con 0x56154a55bb40 2024-04-24T09:04:07.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.116+0000 7f54c2a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56154a566b30 0x56154a568f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:07.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.116+0000 7f54c2a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56154a566b30 0x56154a568f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48826/0 (socket says 172.21.15.83:48826) 2024-04-24T09:04:07.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.116+0000 7f54c2a00640 1 -- 172.21.15.83:0/4141613559 learned_addr learned my addr 172.21.15.83:0/4141613559 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:04:07.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.116+0000 7f54c3e00640 1 --2- 172.21.15.83:0/4141613559 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56154a55bb40 0x56154a55bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:07.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.116+0000 7f54c3400640 1 --2- 172.21.15.83:0/4141613559 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56154a55c460 0x56154a5665f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:07.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.116+0000 7f54c2a00640 1 -- 172.21.15.83:0/4141613559 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56154a55bb40 msgr2=0x56154a55bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:07.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.116+0000 7f54c2a00640 1 --2- 172.21.15.83:0/4141613559 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56154a55bb40 0x56154a55bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.116+0000 7f54c2a00640 1 -- 172.21.15.83:0/4141613559 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56154a55c460 msgr2=0x56154a5665f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:07.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.116+0000 7f54c2a00640 1 --2- 172.21.15.83:0/4141613559 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56154a55c460 0x56154a5665f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.116+0000 7f54c2a00640 1 -- 172.21.15.83:0/4141613559 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56154a524310 con 0x56154a566b30 2024-04-24T09:04:07.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.117+0000 7f54bbe00640 1 --2- 172.21.15.83:0/3494412614 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f54a80a3d20 0x7f54a80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:07.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.117+0000 7f54c2a00640 1 --2- 172.21.15.83:0/4141613559 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56154a566b30 0x56154a568f20 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7f54b002a930 tx=0x7f54b0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eba35bf33549b115 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:07.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.117+0000 7f54bbe00640 1 -- 172.21.15.83:0/3494412614 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7f54b40d07b0 con 0x7f54c4102110 2024-04-24T09:04:07.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.117+0000 7f54c3e00640 1 --2- 172.21.15.83:0/3494412614 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f54a80a3d20 0x7f54a80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:07.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.117+0000 7f54afe00640 1 -- 172.21.15.83:0/4141613559 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b003e030 con 0x56154a566b30 2024-04-24T09:04:07.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.117+0000 7f54afe00640 1 -- 172.21.15.83:0/4141613559 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f54b00041f0 con 0x56154a566b30 2024-04-24T09:04:07.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.117+0000 7f54afe00640 1 -- 172.21.15.83:0/4141613559 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b0035040 con 0x56154a566b30 2024-04-24T09:04:07.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.118+0000 7f54ccf8f740 1 -- 172.21.15.83:0/4141613559 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56154a566b30 msgr2=0x56154a568f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:07.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.118+0000 7f54ccf8f740 1 --2- 172.21.15.83:0/4141613559 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56154a566b30 0x56154a568f20 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7f54b002a930 tx=0x7f54b0000bc0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.118+0000 7f54c3e00640 1 --2- 172.21.15.83:0/3494412614 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f54a80a3d20 0x7f54a80a61e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f54bc005fd0 tx=0x7f54bc0096e0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:07.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.118+0000 7f54ccf8f740 1 -- 172.21.15.83:0/4141613559 shutdown_connections 2024-04-24T09:04:07.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.118+0000 7f54ccf8f740 1 --2- 172.21.15.83:0/4141613559 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56154a566b30 0x56154a568f20 unknown :-1 s=CLOSED pgs=1965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.118+0000 7f54ccf8f740 1 --2- 172.21.15.83:0/4141613559 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56154a55c460 0x56154a5665f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.118+0000 7f54ccf8f740 1 --2- 172.21.15.83:0/4141613559 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56154a55bb40 0x56154a55bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.118+0000 7f54ccf8f740 1 -- 172.21.15.83:0/4141613559 >> 172.21.15.83:0/4141613559 conn(0x56154a557e60 msgr2=0x56154a558240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:07.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.119+0000 7f54ccf8f740 1 -- 172.21.15.83:0/4141613559 shutdown_connections 2024-04-24T09:04:07.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.119+0000 7f54ccf8f740 1 -- 172.21.15.83:0/4141613559 wait complete. 2024-04-24T09:04:07.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.119+0000 7f54ccf8f740 1 -- start start 2024-04-24T09:04:07.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.119+0000 7f54ccf8f740 1 Processor -- start 2024-04-24T09:04:07.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.120+0000 7f54ccf8f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56154a55bb40 0x56154a566560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:07.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.120+0000 7f54ccf8f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56154a55c460 0x56154a563850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:07.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.120+0000 7f54ccf8f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56154a566b30 0x56154a563f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:07.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.120+0000 7f54ccf8f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56154a569960 con 0x56154a566b30 2024-04-24T09:04:07.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.120+0000 7f54ccf8f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56154a47a810 con 0x56154a55c460 2024-04-24T09:04:07.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.120+0000 7f54ccf8f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56154a569ae0 con 0x56154a55bb40 2024-04-24T09:04:07.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.120+0000 7f54ccf8f740 10 client.0 _ll_register_callbacks cb 0x56154a448140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:04:07.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.120+0000 7f54c3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56154a566b30 0x56154a563f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:07.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.120+0000 7f54c3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56154a566b30 0x56154a563f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48838/0 (socket says 172.21.15.83:48838) 2024-04-24T09:04:07.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.120+0000 7f54c3e00640 1 -- 172.21.15.83:0/1215106597 learned_addr learned my addr 172.21.15.83:0/1215106597 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:04:07.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.120+0000 7f54c3400640 1 --2- 172.21.15.83:0/1215106597 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56154a55bb40 0x56154a566560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:07.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.121+0000 7f54c2a00640 1 --2- 172.21.15.83:0/1215106597 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56154a55c460 0x56154a563850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:07.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.121+0000 7f54c3e00640 1 -- 172.21.15.83:0/1215106597 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56154a55bb40 msgr2=0x56154a566560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:07.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.121+0000 7f54c3e00640 1 --2- 172.21.15.83:0/1215106597 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56154a55bb40 0x56154a566560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.121+0000 7f54c3e00640 1 -- 172.21.15.83:0/1215106597 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56154a55c460 msgr2=0x56154a563850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:07.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.121+0000 7f54c3e00640 1 --2- 172.21.15.83:0/1215106597 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56154a55c460 0x56154a563850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.121+0000 7f54c3e00640 1 -- 172.21.15.83:0/1215106597 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56154a48b3f0 con 0x56154a566b30 2024-04-24T09:04:07.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.121+0000 7f54c3e00640 1 --2- 172.21.15.83:0/1215106597 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56154a566b30 0x56154a563f90 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7f54bc01d870 tx=0x7f54bc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41b6ec1c99b41fbe server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:07.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.121+0000 7f54acc00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:04:07.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.122+0000 7f54acc00640 1 -- 172.21.15.83:0/1215106597 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54bc01e820 con 0x56154a566b30 2024-04-24T09:04:07.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.122+0000 7f54acc00640 1 -- 172.21.15.83:0/1215106597 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7f54bc01e9c0 con 0x56154a566b30 2024-04-24T09:04:07.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.122+0000 7f54ccf8f740 1 -- 172.21.15.83:0/1215106597 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54b00041f0 con 0x56154a566b30 2024-04-24T09:04:07.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.122+0000 7f54acc00640 1 -- 172.21.15.83:0/1215106597 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f54bc032050 con 0x56154a566b30 2024-04-24T09:04:07.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.122+0000 7f54ccf8f740 10 client.20058 fetch_fsmap learned FSMap version 41 2024-04-24T09:04:07.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.122+0000 7f54ccf8f740 1 -- 172.21.15.83:0/1215106597 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x56154a1d33f0 con 0x56154a566b30 2024-04-24T09:04:07.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.122+0000 7f54acc00640 1 -- 172.21.15.83:0/1215106597 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f54b00041f0 con 0x56154a566b30 2024-04-24T09:04:07.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.122+0000 7f54acc00640 1 -- 172.21.15.83:0/1215106597 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7f54bc00f040 con 0x56154a566b30 2024-04-24T09:04:07.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.122+0000 7f54ccf8f740 10 client.20058 fetch_fsmap finished waiting for FSMap version 41 2024-04-24T09:04:07.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.122+0000 7f54ccf8f740 10 client.20058 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-24T09:04:07.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.122+0000 7f54ccf8f740 20 client.20058 populate_metadata read hostname 'smithi083' 2024-04-24T09:04:07.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.122+0000 7f54ccf8f740 1 --2- 172.21.15.83:0/1215106597 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x56154a57b200 0x56154a57d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:07.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.122+0000 7f54ccf8f740 4 client.20058 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:04:07.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.122+0000 7f54ccf8f740 1 -- 172.21.15.83:0/1215106597 --> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56154a4cebf0 con 0x56154a57b200 2024-04-24T09:04:07.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.123+0000 7f54c3400640 1 --2- 172.21.15.83:0/1215106597 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x56154a57b200 0x56154a57d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:07.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.124+0000 7f54c3400640 1 --2- 172.21.15.83:0/1215106597 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x56154a57b200 0x56154a57d660 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=505871e97bdcc302 server_cookie=9e3e427d9a082049 in_seq=0 out_seq=0 2024-04-24T09:04:07.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.124+0000 7f54acc00640 10 client.20058 ms_handle_connect on v2:172.21.15.138:6834/1909219758 2024-04-24T09:04:07.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.127+0000 7f54acc00640 1 -- 172.21.15.83:0/1215106597 <== mds.2 v2:172.21.15.138:6834/1909219758 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56154a4cebf0 con 0x56154a57b200 2024-04-24T09:04:07.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.127+0000 7f54acc00640 10 client.20058 handle_command_reply: tid=0 2024-04-24T09:04:07.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.127+0000 7f54ccf8f740 1 client.20058 shutdown 2024-04-24T09:04:07.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.128+0000 7f54ccf8f740 1 -- 172.21.15.83:0/1215106597 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56154a566b30 msgr2=0x56154a563f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:07.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.128+0000 7f54ccf8f740 1 --2- 172.21.15.83:0/1215106597 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56154a566b30 0x56154a563f90 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7f54bc01d870 tx=0x7f54bc004a40 comp rx=0 tx=0).stop 2024-04-24T09:04:07.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.128+0000 7f54ccf8f740 1 -- 172.21.15.83:0/1215106597 shutdown_connections 2024-04-24T09:04:07.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.128+0000 7f54ccf8f740 1 --2- 172.21.15.83:0/1215106597 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x56154a57b200 0x56154a57d660 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.128+0000 7f54ccf8f740 1 --2- 172.21.15.83:0/1215106597 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56154a566b30 0x56154a563f90 unknown :-1 s=CLOSED pgs=1966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.128+0000 7f54ccf8f740 1 --2- 172.21.15.83:0/1215106597 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56154a55c460 0x56154a563850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.128+0000 7f54ccf8f740 1 --2- 172.21.15.83:0/1215106597 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56154a55bb40 0x56154a566560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.128+0000 7f54acc00640 0 client.20058 ms_handle_reset on v2:172.21.15.138:6834/1909219758 2024-04-24T09:04:07.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.128+0000 7f54ccf8f740 1 -- 172.21.15.83:0/1215106597 >> 172.21.15.83:0/1215106597 conn(0x56154a557e60 msgr2=0x56154a55dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:07.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.128+0000 7f54ccf8f740 1 -- 172.21.15.83:0/1215106597 shutdown_connections 2024-04-24T09:04:07.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.128+0000 7f54ccf8f740 1 -- 172.21.15.83:0/1215106597 wait complete. 2024-04-24T09:04:07.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.128+0000 7f54ccf8f740 20 client.20058 trim_cache size 0 max 16384 2024-04-24T09:04:07.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.144+0000 7f54ccf8f740 1 Processor -- start 2024-04-24T09:04:07.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.144+0000 7f54ccf8f740 1 -- start start 2024-04-24T09:04:07.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.144+0000 7f54ccf8f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56154a55bb40 0x56154a55e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:07.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.144+0000 7f54ccf8f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56154a55c460 0x56154a566090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:07.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.144+0000 7f54ccf8f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56154a566b30 0x56154a5665d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:07.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.144+0000 7f54ccf8f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f54c41ad180 con 0x56154a55bb40 2024-04-24T09:04:07.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.144+0000 7f54ccf8f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f54c41aede0 con 0x56154a566b30 2024-04-24T09:04:07.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.144+0000 7f54ccf8f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f54c41ad850 con 0x56154a55c460 2024-04-24T09:04:07.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.144+0000 7f54c3400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56154a55bb40 0x56154a55e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:07.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.145+0000 7f54c3400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56154a55bb40 0x56154a55e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48854/0 (socket says 172.21.15.83:48854) 2024-04-24T09:04:07.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.145+0000 7f54c3400640 1 -- 172.21.15.83:0/529763422 learned_addr learned my addr 172.21.15.83:0/529763422 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:04:07.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.145+0000 7f54c2a00640 1 --2- 172.21.15.83:0/529763422 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56154a55c460 0x56154a566090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:07.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.145+0000 7f54c3400640 1 -- 172.21.15.83:0/529763422 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56154a55c460 msgr2=0x56154a566090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:07.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.145+0000 7f54c3400640 1 --2- 172.21.15.83:0/529763422 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56154a55c460 0x56154a566090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.145+0000 7f54c3e00640 1 --2- 172.21.15.83:0/529763422 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56154a566b30 0x56154a5665d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:07.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.145+0000 7f54c3400640 1 -- 172.21.15.83:0/529763422 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56154a566b30 msgr2=0x56154a5665d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:07.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.145+0000 7f54c3400640 1 --2- 172.21.15.83:0/529763422 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56154a566b30 0x56154a5665d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.145+0000 7f54c3400640 1 -- 172.21.15.83:0/529763422 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56154a55c290 con 0x56154a55bb40 2024-04-24T09:04:07.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.145+0000 7f54c3400640 1 --2- 172.21.15.83:0/529763422 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56154a55bb40 0x56154a55e2a0 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7f54b4009610 tx=0x7f54b40d2c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5538315e6382fed4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:07.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.145+0000 7f54af400640 1 -- 172.21.15.83:0/529763422 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b4020040 con 0x56154a55bb40 2024-04-24T09:04:07.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.146+0000 7f54af400640 1 -- 172.21.15.83:0/529763422 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f54b4009c30 con 0x56154a55bb40 2024-04-24T09:04:07.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.146+0000 7f54af400640 1 -- 172.21.15.83:0/529763422 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b4098940 con 0x56154a55bb40 2024-04-24T09:04:07.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.146+0000 7f54ccf8f740 1 -- 172.21.15.83:0/529763422 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56154a55bb40 msgr2=0x56154a55e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:07.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.146+0000 7f54ccf8f740 1 --2- 172.21.15.83:0/529763422 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56154a55bb40 0x56154a55e2a0 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7f54b4009610 tx=0x7f54b40d2c00 comp rx=0 tx=0).stop 2024-04-24T09:04:07.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.146+0000 7f54ccf8f740 1 -- 172.21.15.83:0/529763422 shutdown_connections 2024-04-24T09:04:07.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.146+0000 7f54ccf8f740 1 --2- 172.21.15.83:0/529763422 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56154a566b30 0x56154a5665d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.146+0000 7f54ccf8f740 1 --2- 172.21.15.83:0/529763422 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56154a55c460 0x56154a566090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.146+0000 7f54ccf8f740 1 --2- 172.21.15.83:0/529763422 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56154a55bb40 0x56154a55e2a0 unknown :-1 s=CLOSED pgs=1967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.146+0000 7f54ccf8f740 1 -- 172.21.15.83:0/529763422 >> 172.21.15.83:0/529763422 conn(0x56154a557e60 msgr2=0x56154a562120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:07.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.147+0000 7f54ccf8f740 1 -- 172.21.15.83:0/529763422 shutdown_connections 2024-04-24T09:04:07.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.147+0000 7f54ccf8f740 1 -- 172.21.15.83:0/529763422 wait complete. 2024-04-24T09:04:07.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.147+0000 7f54ccf8f740 1 -- start start 2024-04-24T09:04:07.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.147+0000 7f54ccf8f740 1 Processor -- start 2024-04-24T09:04:07.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.147+0000 7f54ccf8f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56154a55bb40 0x56154a567cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:07.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.147+0000 7f54ccf8f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56154a55c460 0x56154a568230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:07.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.148+0000 7f54ccf8f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56154a566b30 0x56154a571700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:07.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.148+0000 7f54ccf8f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f54c41ae6b0 con 0x56154a566b30 2024-04-24T09:04:07.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.148+0000 7f54ccf8f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f54c41adf80 con 0x56154a55c460 2024-04-24T09:04:07.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.148+0000 7f54ccf8f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f54c41ab180 con 0x56154a55bb40 2024-04-24T09:04:07.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.148+0000 7f54ccf8f740 10 client.0 _ll_register_callbacks cb 0x7f54bc01d520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:04:07.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.148+0000 7f54c3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56154a566b30 0x56154a571700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:07.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.148+0000 7f54c3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56154a566b30 0x56154a571700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48856/0 (socket says 172.21.15.83:48856) 2024-04-24T09:04:07.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.148+0000 7f54c3e00640 1 -- 172.21.15.83:0/1934104067 learned_addr learned my addr 172.21.15.83:0/1934104067 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:04:07.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.148+0000 7f54c2a00640 1 --2- 172.21.15.83:0/1934104067 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56154a55c460 0x56154a568230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:07.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.148+0000 7f54c3e00640 1 -- 172.21.15.83:0/1934104067 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56154a55bb40 msgr2=0x56154a567cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:07.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.148+0000 7f54c3e00640 1 --2- 172.21.15.83:0/1934104067 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56154a55bb40 0x56154a567cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.148+0000 7f54c3e00640 1 -- 172.21.15.83:0/1934104067 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56154a55c460 msgr2=0x56154a568230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:07.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.148+0000 7f54c3e00640 1 --2- 172.21.15.83:0/1934104067 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56154a55c460 0x56154a568230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.148+0000 7f54c3e00640 1 -- 172.21.15.83:0/1934104067 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56154a29d7d0 con 0x56154a566b30 2024-04-24T09:04:07.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.148+0000 7f54c2a00640 1 --2- 172.21.15.83:0/1934104067 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56154a55c460 0x56154a568230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:04:07.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.149+0000 7f54c3e00640 1 --2- 172.21.15.83:0/1934104067 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56154a566b30 0x56154a571700 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7f54bc01d4f0 tx=0x7f54bc01baf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4914f95d5bac3f17 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:07.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.149+0000 7f54acc00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:04:07.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.149+0000 7f54acc00640 1 -- 172.21.15.83:0/1934104067 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54bc03c020 con 0x56154a566b30 2024-04-24T09:04:07.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.149+0000 7f54ccf8f740 1 -- 172.21.15.83:0/1934104067 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56154a5778e0 con 0x56154a566b30 2024-04-24T09:04:07.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.149+0000 7f54acc00640 1 -- 172.21.15.83:0/1934104067 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7f54bc01ea30 con 0x56154a566b30 2024-04-24T09:04:07.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.149+0000 7f54acc00640 1 -- 172.21.15.83:0/1934104067 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f54bc01dd30 con 0x56154a566b30 2024-04-24T09:04:07.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.149+0000 7f54ccf8f740 10 client.20064 fetch_fsmap learned FSMap version 41 2024-04-24T09:04:07.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.149+0000 7f54ccf8f740 1 -- 172.21.15.83:0/1934104067 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x56154a5e1c50 con 0x56154a566b30 2024-04-24T09:04:07.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.149+0000 7f54acc00640 1 -- 172.21.15.83:0/1934104067 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x56154a5778e0 con 0x56154a566b30 2024-04-24T09:04:07.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.150+0000 7f54acc00640 1 -- 172.21.15.83:0/1934104067 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7f54bc035030 con 0x56154a566b30 2024-04-24T09:04:07.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.150+0000 7f54ccf8f740 10 client.20064 fetch_fsmap finished waiting for FSMap version 41 2024-04-24T09:04:07.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.150+0000 7f54ccf8f740 10 client.20064 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-24T09:04:07.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.150+0000 7f54ccf8f740 20 client.20064 populate_metadata read hostname 'smithi083' 2024-04-24T09:04:07.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.150+0000 7f54ccf8f740 1 --2- 172.21.15.83:0/1934104067 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x56154a5e23c0 0x56154a5e47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:07.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.150+0000 7f54ccf8f740 4 client.20064 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:04:07.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.150+0000 7f54ccf8f740 1 -- 172.21.15.83:0/1934104067 --> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56154a4c8a60 con 0x56154a5e23c0 2024-04-24T09:04:07.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.150+0000 7f54c3400640 1 --2- 172.21.15.83:0/1934104067 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x56154a5e23c0 0x56154a5e47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:07.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.152+0000 7f54c3400640 1 --2- 172.21.15.83:0/1934104067 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x56154a5e23c0 0x56154a5e47e0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7a2e9f3c2cf7c290 server_cookie=4999a657e68a6f88 in_seq=0 out_seq=0 2024-04-24T09:04:07.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.152+0000 7f54acc00640 10 client.20064 ms_handle_connect on v2:172.21.15.138:6834/1909219758 2024-04-24T09:04:07.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.152+0000 7f54acc00640 1 -- 172.21.15.83:0/1934104067 <== mds.2 v2:172.21.15.138:6834/1909219758 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56154a4c8a60 con 0x56154a5e23c0 2024-04-24T09:04:07.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.152+0000 7f54acc00640 10 client.20064 handle_command_reply: tid=0 2024-04-24T09:04:07.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.152+0000 7f54ccf8f740 1 client.20064 shutdown 2024-04-24T09:04:07.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.152+0000 7f54ccf8f740 1 -- 172.21.15.83:0/1934104067 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56154a566b30 msgr2=0x56154a571700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:07.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.152+0000 7f54ccf8f740 1 --2- 172.21.15.83:0/1934104067 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56154a566b30 0x56154a571700 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7f54bc01d4f0 tx=0x7f54bc01baf0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.153+0000 7f54ccf8f740 1 -- 172.21.15.83:0/1934104067 shutdown_connections 2024-04-24T09:04:07.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.153+0000 7f54ccf8f740 1 --2- 172.21.15.83:0/1934104067 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x56154a5e23c0 0x56154a5e47e0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.153+0000 7f54ccf8f740 1 --2- 172.21.15.83:0/1934104067 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56154a566b30 0x56154a571700 unknown :-1 s=CLOSED pgs=1968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.153+0000 7f54ccf8f740 1 --2- 172.21.15.83:0/1934104067 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56154a55c460 0x56154a568230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.153+0000 7f54acc00640 0 client.20064 ms_handle_reset on v2:172.21.15.138:6834/1909219758 2024-04-24T09:04:07.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.153+0000 7f54ccf8f740 1 --2- 172.21.15.83:0/1934104067 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56154a55bb40 0x56154a567cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.153+0000 7f54ccf8f740 1 -- 172.21.15.83:0/1934104067 >> 172.21.15.83:0/1934104067 conn(0x56154a557e60 msgr2=0x56154a5566c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:07.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.153+0000 7f54ccf8f740 1 -- 172.21.15.83:0/1934104067 shutdown_connections 2024-04-24T09:04:07.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.153+0000 7f54ccf8f740 1 -- 172.21.15.83:0/1934104067 wait complete. 2024-04-24T09:04:07.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.153+0000 7f54ccf8f740 20 client.20064 trim_cache size 0 max 16384 2024-04-24T09:04:07.158 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:04:07.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.154+0000 7f54c8c00640 1 -- 172.21.15.83:0/3494412614 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f54a80a3d20 msgr2=0x7f54a80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:07.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.154+0000 7f54c8c00640 1 --2- 172.21.15.83:0/3494412614 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f54a80a3d20 0x7f54a80a61e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f54bc005fd0 tx=0x7f54bc0096e0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.154+0000 7f54c8c00640 1 -- 172.21.15.83:0/3494412614 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54c4102110 msgr2=0x7f54c4198730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:07.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.154+0000 7f54c8c00640 1 --2- 172.21.15.83:0/3494412614 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54c4102110 0x7f54c4198730 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7f54b400d770 tx=0x7f54b400dc30 comp rx=0 tx=0).stop 2024-04-24T09:04:07.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.154+0000 7f54c8c00640 1 -- 172.21.15.83:0/3494412614 shutdown_connections 2024-04-24T09:04:07.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.154+0000 7f54c8c00640 1 --2- 172.21.15.83:0/3494412614 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f54a80a3d20 0x7f54a80a61e0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.154+0000 7f54c8c00640 1 --2- 172.21.15.83:0/3494412614 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f54c419cff0 0x7f54c419d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.154+0000 7f54c8c00640 1 --2- 172.21.15.83:0/3494412614 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f54c41071e0 0x7f54c4198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.154+0000 7f54c8c00640 1 --2- 172.21.15.83:0/3494412614 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54c4102110 0x7f54c4198730 unknown :-1 s=CLOSED pgs=1964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.154+0000 7f54c8c00640 1 -- 172.21.15.83:0/3494412614 >> 172.21.15.83:0/3494412614 conn(0x7f54c40fd9d0 msgr2=0x7f54c40fed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:07.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.154+0000 7f54c8c00640 1 -- 172.21.15.83:0/3494412614 shutdown_connections 2024-04-24T09:04:07.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.154+0000 7f54c8c00640 1 -- 172.21.15.83:0/3494412614 wait complete. 2024-04-24T09:04:07.172 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T09:04:07.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.370+0000 7f2e8a000640 1 -- 172.21.15.83:0/860537911 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e7c031880 con 0x7f2e8c0742d0 2024-04-24T09:04:07.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.371+0000 7f2e92000640 1 -- 172.21.15.83:0/860537911 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e8c0742d0 msgr2=0x7f2e8c0746b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:07.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.371+0000 7f2e92000640 1 --2- 172.21.15.83:0/860537911 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e8c0742d0 0x7f2e8c0746b0 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c009900 tx=0x7f2e7c02a480 comp rx=0 tx=0).stop 2024-04-24T09:04:07.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.371+0000 7f2e92000640 1 -- 172.21.15.83:0/860537911 shutdown_connections 2024-04-24T09:04:07.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.371+0000 7f2e92000640 1 --2- 172.21.15.83:0/860537911 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e8c10e8e0 0x7f2e8c110cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.371+0000 7f2e92000640 1 --2- 172.21.15.83:0/860537911 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2e8c074bf0 0x7f2e8c10e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.371+0000 7f2e92000640 1 --2- 172.21.15.83:0/860537911 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e8c0742d0 0x7f2e8c0746b0 unknown :-1 s=CLOSED pgs=1969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.371+0000 7f2e92000640 1 -- 172.21.15.83:0/860537911 >> 172.21.15.83:0/860537911 conn(0x7f2e8c079450 msgr2=0x7f2e8c079860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:07.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.371+0000 7f2e92000640 1 -- 172.21.15.83:0/860537911 shutdown_connections 2024-04-24T09:04:07.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.371+0000 7f2e92000640 1 -- 172.21.15.83:0/860537911 wait complete. 2024-04-24T09:04:07.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.372+0000 7f2e92000640 1 Processor -- start 2024-04-24T09:04:07.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.372+0000 7f2e92000640 1 -- start start 2024-04-24T09:04:07.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.373+0000 7f2e92000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2e8c0742d0 0x7f2e8c1a06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:07.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.373+0000 7f2e92000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e8c074bf0 0x7f2e8c1a0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:07.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.373+0000 7f2e92000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e8c10e8e0 0x7f2e8c19a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:07.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.373+0000 7f2e92000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2e8c1144a0 con 0x7f2e8c10e8e0 2024-04-24T09:04:07.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.373+0000 7f2e92000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2e8c114320 con 0x7f2e8c074bf0 2024-04-24T09:04:07.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.373+0000 7f2e92000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2e8c114620 con 0x7f2e8c0742d0 2024-04-24T09:04:07.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.373+0000 7f2e8be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e8c10e8e0 0x7f2e8c19a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:07.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.373+0000 7f2e8be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e8c10e8e0 0x7f2e8c19a880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48874/0 (socket says 172.21.15.83:48874) 2024-04-24T09:04:07.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.373+0000 7f2e8be00640 1 -- 172.21.15.83:0/1603290905 learned_addr learned my addr 172.21.15.83:0/1603290905 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:04:07.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.373+0000 7f2e8b400640 1 --2- 172.21.15.83:0/1603290905 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2e8c0742d0 0x7f2e8c1a06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:07.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.373+0000 7f2e8aa00640 1 --2- 172.21.15.83:0/1603290905 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e8c074bf0 0x7f2e8c1a0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:07.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.374+0000 7f2e8be00640 1 -- 172.21.15.83:0/1603290905 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2e8c0742d0 msgr2=0x7f2e8c1a06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:07.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.374+0000 7f2e8be00640 1 --2- 172.21.15.83:0/1603290905 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2e8c0742d0 0x7f2e8c1a06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.374+0000 7f2e8be00640 1 -- 172.21.15.83:0/1603290905 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e8c074bf0 msgr2=0x7f2e8c1a0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:07.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.374+0000 7f2e8be00640 1 --2- 172.21.15.83:0/1603290905 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e8c074bf0 0x7f2e8c1a0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.374+0000 7f2e8be00640 1 -- 172.21.15.83:0/1603290905 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e8c19b000 con 0x7f2e8c10e8e0 2024-04-24T09:04:07.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.374+0000 7f2e8aa00640 1 --2- 172.21.15.83:0/1603290905 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e8c074bf0 0x7f2e8c1a0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:04:07.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.374+0000 7f2e8be00640 1 --2- 172.21.15.83:0/1603290905 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e8c10e8e0 0x7f2e8c19a880 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7f2e8400cab0 tx=0x7f2e8400cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:07.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.374+0000 7f2e83e00640 1 -- 172.21.15.83:0/1603290905 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e8400ae20 con 0x7f2e8c10e8e0 2024-04-24T09:04:07.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.374+0000 7f2e83e00640 1 -- 172.21.15.83:0/1603290905 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2e8400f900 con 0x7f2e8c10e8e0 2024-04-24T09:04:07.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.374+0000 7f2e92000640 1 -- 172.21.15.83:0/1603290905 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e8c19b2f0 con 0x7f2e8c10e8e0 2024-04-24T09:04:07.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.374+0000 7f2e83e00640 1 -- 172.21.15.83:0/1603290905 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e840189f0 con 0x7f2e8c10e8e0 2024-04-24T09:04:07.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.374+0000 7f2e92000640 1 -- 172.21.15.83:0/1603290905 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e8c1a7590 con 0x7f2e8c10e8e0 2024-04-24T09:04:07.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.378+0000 7f2e83e00640 1 -- 172.21.15.83:0/1603290905 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2e8400fab0 con 0x7f2e8c10e8e0 2024-04-24T09:04:07.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.378+0000 7f2e94f64740 1 Processor -- start 2024-04-24T09:04:07.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.378+0000 7f2e94f64740 1 -- start start 2024-04-24T09:04:07.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.379+0000 7f2e94f64740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f251df950 0x562f251dfd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:07.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.379+0000 7f2e94f64740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562f251e0270 0x562f251ea400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:07.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.379+0000 7f2e94f64740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562f251ea940 0x562f251ecd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:07.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.379+0000 7f2e94f64740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562f251451d0 con 0x562f251ea940 2024-04-24T09:04:07.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.379+0000 7f2e94f64740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562f250eb450 con 0x562f251df950 2024-04-24T09:04:07.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.379+0000 7f2e94f64740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x562f25141190 con 0x562f251e0270 2024-04-24T09:04:07.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.379+0000 7f2e8aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562f251ea940 0x562f251ecd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:07.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.379+0000 7f2e8aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562f251ea940 0x562f251ecd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48888/0 (socket says 172.21.15.83:48888) 2024-04-24T09:04:07.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.379+0000 7f2e8aa00640 1 -- 172.21.15.83:0/2877696874 learned_addr learned my addr 172.21.15.83:0/2877696874 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:04:07.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.379+0000 7f2e8aa00640 1 --2- 172.21.15.83:0/2877696874 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f251df950 0x562f251dfd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:07.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.379+0000 7f2e8b400640 1 --2- 172.21.15.83:0/2877696874 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562f251e0270 0x562f251ea400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:07.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.379+0000 7f2e8aa00640 1 -- 172.21.15.83:0/2877696874 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562f251e0270 msgr2=0x562f251ea400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:07.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.379+0000 7f2e8aa00640 1 --2- 172.21.15.83:0/2877696874 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562f251e0270 0x562f251ea400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.379+0000 7f2e8aa00640 1 -- 172.21.15.83:0/2877696874 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f251df950 msgr2=0x562f251dfd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:07.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.379+0000 7f2e8aa00640 1 --2- 172.21.15.83:0/2877696874 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f251df950 0x562f251dfd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.379+0000 7f2e8aa00640 1 -- 172.21.15.83:0/2877696874 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f251a8310 con 0x562f251ea940 2024-04-24T09:04:07.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.380+0000 7f2e83e00640 1 --2- 172.21.15.83:0/1603290905 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f2e700a3d20 0x7f2e700a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:07.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.380+0000 7f2e8aa00640 1 --2- 172.21.15.83:0/2877696874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562f251ea940 0x562f251ecd30 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7f2e78012570 tx=0x7f2e7800b640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=868fcb97fac1f782 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:07.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.380+0000 7f2e83e00640 1 -- 172.21.15.83:0/1603290905 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7f2e840d1350 con 0x7f2e8c10e8e0 2024-04-24T09:04:07.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.380+0000 7f2e77e00640 1 -- 172.21.15.83:0/2877696874 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e78018030 con 0x562f251ea940 2024-04-24T09:04:07.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.380+0000 7f2e8b400640 1 --2- 172.21.15.83:0/1603290905 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f2e700a3d20 0x7f2e700a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:07.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.380+0000 7f2e77e00640 1 -- 172.21.15.83:0/2877696874 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2e78010720 con 0x562f251ea940 2024-04-24T09:04:07.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.380+0000 7f2e77e00640 1 -- 172.21.15.83:0/2877696874 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e78003830 con 0x562f251ea940 2024-04-24T09:04:07.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.381+0000 7f2e94f64740 1 -- 172.21.15.83:0/2877696874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562f251ea940 msgr2=0x562f251ecd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:07.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.381+0000 7f2e8b400640 1 --2- 172.21.15.83:0/1603290905 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f2e700a3d20 0x7f2e700a61e0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c000c00 tx=0x7f2e7c005e80 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:07.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.381+0000 7f2e94f64740 1 --2- 172.21.15.83:0/2877696874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562f251ea940 0x562f251ecd30 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7f2e78012570 tx=0x7f2e7800b640 comp rx=0 tx=0).stop 2024-04-24T09:04:07.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.381+0000 7f2e94f64740 1 -- 172.21.15.83:0/2877696874 shutdown_connections 2024-04-24T09:04:07.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.381+0000 7f2e94f64740 1 --2- 172.21.15.83:0/2877696874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562f251ea940 0x562f251ecd30 unknown :-1 s=CLOSED pgs=1971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.381+0000 7f2e94f64740 1 --2- 172.21.15.83:0/2877696874 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562f251e0270 0x562f251ea400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.381+0000 7f2e94f64740 1 --2- 172.21.15.83:0/2877696874 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f251df950 0x562f251dfd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.381+0000 7f2e94f64740 1 -- 172.21.15.83:0/2877696874 >> 172.21.15.83:0/2877696874 conn(0x562f251dbc70 msgr2=0x562f251dc050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:07.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.381+0000 7f2e94f64740 1 -- 172.21.15.83:0/2877696874 shutdown_connections 2024-04-24T09:04:07.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.382+0000 7f2e94f64740 1 -- 172.21.15.83:0/2877696874 wait complete. 2024-04-24T09:04:07.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.382+0000 7f2e94f64740 1 -- start start 2024-04-24T09:04:07.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.382+0000 7f2e94f64740 1 Processor -- start 2024-04-24T09:04:07.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.383+0000 7f2e94f64740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f251df950 0x562f251ea3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:07.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.383+0000 7f2e94f64740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562f251e0270 0x562f251e76c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:07.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.383+0000 7f2e94f64740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562f251ea940 0x562f251e7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:07.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.383+0000 7f2e94f64740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562f251ed7d0 con 0x562f251ea940 2024-04-24T09:04:07.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.383+0000 7f2e94f64740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562f250fe810 con 0x562f251df950 2024-04-24T09:04:07.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.383+0000 7f2e94f64740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x562f251ed950 con 0x562f251e0270 2024-04-24T09:04:07.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.383+0000 7f2e94f64740 10 client.0 _ll_register_callbacks cb 0x562f250cc140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:04:07.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.383+0000 7f2e8be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562f251ea940 0x562f251e7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:07.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.383+0000 7f2e8be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562f251ea940 0x562f251e7db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48894/0 (socket says 172.21.15.83:48894) 2024-04-24T09:04:07.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.383+0000 7f2e8be00640 1 -- 172.21.15.83:0/219809375 learned_addr learned my addr 172.21.15.83:0/219809375 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:04:07.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.383+0000 7f2e8b400640 1 --2- 172.21.15.83:0/219809375 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f251df950 0x562f251ea3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:07.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.383+0000 7f2e8aa00640 1 --2- 172.21.15.83:0/219809375 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562f251e0270 0x562f251e76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:07.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.384+0000 7f2e8be00640 1 -- 172.21.15.83:0/219809375 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562f251e0270 msgr2=0x562f251e76c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:07.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.384+0000 7f2e8be00640 1 --2- 172.21.15.83:0/219809375 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562f251e0270 0x562f251e76c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.384+0000 7f2e8be00640 1 -- 172.21.15.83:0/219809375 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f251df950 msgr2=0x562f251ea3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:07.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.384+0000 7f2e8be00640 1 --2- 172.21.15.83:0/219809375 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f251df950 0x562f251ea3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.384+0000 7f2e8be00640 1 -- 172.21.15.83:0/219809375 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f2510f3f0 con 0x562f251ea940 2024-04-24T09:04:07.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.384+0000 7f2e8be00640 1 --2- 172.21.15.83:0/219809375 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562f251ea940 0x562f251e7db0 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7f2e840a55b0 tx=0x7f2e8400fee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1849051d62b9ed97 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:07.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.384+0000 7f2e74c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:04:07.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.384+0000 7f2e74c00640 1 -- 172.21.15.83:0/219809375 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e840a1220 con 0x562f251ea940 2024-04-24T09:04:07.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.384+0000 7f2e74c00640 1 -- 172.21.15.83:0/219809375 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7f2e840a5650 con 0x562f251ea940 2024-04-24T09:04:07.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.384+0000 7f2e94f64740 1 -- 172.21.15.83:0/219809375 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e78010720 con 0x562f251ea940 2024-04-24T09:04:07.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.385+0000 7f2e74c00640 1 -- 172.21.15.83:0/219809375 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f2e840a5ae0 con 0x562f251ea940 2024-04-24T09:04:07.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.385+0000 7f2e94f64740 10 client.20076 fetch_fsmap learned FSMap version 41 2024-04-24T09:04:07.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.385+0000 7f2e94f64740 1 -- 172.21.15.83:0/219809375 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x562f24e573f0 con 0x562f251ea940 2024-04-24T09:04:07.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.385+0000 7f2e74c00640 1 -- 172.21.15.83:0/219809375 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2e78010720 con 0x562f251ea940 2024-04-24T09:04:07.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.385+0000 7f2e74c00640 1 -- 172.21.15.83:0/219809375 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7f2e8409c2c0 con 0x562f251ea940 2024-04-24T09:04:07.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.385+0000 7f2e94f64740 10 client.20076 fetch_fsmap finished waiting for FSMap version 41 2024-04-24T09:04:07.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.385+0000 7f2e94f64740 10 client.20076 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-24T09:04:07.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.385+0000 7f2e94f64740 20 client.20076 populate_metadata read hostname 'smithi083' 2024-04-24T09:04:07.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.385+0000 7f2e94f64740 1 --2- 172.21.15.83:0/219809375 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x562f251fab80 0x562f251fcfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:07.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.385+0000 7f2e94f64740 4 client.20076 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:04:07.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.385+0000 7f2e94f64740 1 -- 172.21.15.83:0/219809375 --> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562f25152bf0 con 0x562f251fab80 2024-04-24T09:04:07.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.386+0000 7f2e8b400640 1 --2- 172.21.15.83:0/219809375 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x562f251fab80 0x562f251fcfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:07.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.387+0000 7f2e8b400640 1 --2- 172.21.15.83:0/219809375 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x562f251fab80 0x562f251fcfe0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e8584fd25a63492a server_cookie=98101370ae9ad930 in_seq=0 out_seq=0 2024-04-24T09:04:07.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.387+0000 7f2e74c00640 10 client.20076 ms_handle_connect on v2:172.21.15.138:6836/900566754 2024-04-24T09:04:07.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.389+0000 7f2e74c00640 1 -- 172.21.15.83:0/219809375 <== mds.1 v2:172.21.15.138:6836/900566754 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562f25152bf0 con 0x562f251fab80 2024-04-24T09:04:07.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.389+0000 7f2e74c00640 10 client.20076 handle_command_reply: tid=0 2024-04-24T09:04:07.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.389+0000 7f2e94f64740 1 client.20076 shutdown 2024-04-24T09:04:07.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.389+0000 7f2e94f64740 1 -- 172.21.15.83:0/219809375 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562f251ea940 msgr2=0x562f251e7db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:07.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.389+0000 7f2e94f64740 1 --2- 172.21.15.83:0/219809375 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562f251ea940 0x562f251e7db0 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7f2e840a55b0 tx=0x7f2e8400fee0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.389+0000 7f2e94f64740 1 -- 172.21.15.83:0/219809375 shutdown_connections 2024-04-24T09:04:07.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.389+0000 7f2e94f64740 1 --2- 172.21.15.83:0/219809375 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562f251ea940 0x562f251e7db0 unknown :-1 s=CLOSED pgs=1972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.389+0000 7f2e94f64740 1 --2- 172.21.15.83:0/219809375 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x562f251fab80 0x562f251fcfe0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.389+0000 7f2e94f64740 1 --2- 172.21.15.83:0/219809375 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562f251e0270 0x562f251e76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.389+0000 7f2e94f64740 1 --2- 172.21.15.83:0/219809375 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f251df950 0x562f251ea3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.389+0000 7f2e94f64740 1 -- 172.21.15.83:0/219809375 >> 172.21.15.83:0/219809375 conn(0x562f251dbc70 msgr2=0x562f251e1a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:07.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.389+0000 7f2e74c00640 0 client.20076 ms_handle_reset on v2:172.21.15.138:6836/900566754 2024-04-24T09:04:07.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.389+0000 7f2e94f64740 1 -- 172.21.15.83:0/219809375 shutdown_connections 2024-04-24T09:04:07.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.389+0000 7f2e94f64740 1 -- 172.21.15.83:0/219809375 wait complete. 2024-04-24T09:04:07.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.389+0000 7f2e94f64740 20 client.20076 trim_cache size 0 max 16384 2024-04-24T09:04:07.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.406+0000 7f2e94f64740 1 Processor -- start 2024-04-24T09:04:07.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.406+0000 7f2e94f64740 1 -- start start 2024-04-24T09:04:07.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.406+0000 7f2e94f64740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562f251df950 0x562f251f66c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:07.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.406+0000 7f2e94f64740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562f251e0270 0x562f251e9f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:07.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.406+0000 7f2e94f64740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f251ea940 0x562f252028b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:07.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.406+0000 7f2e94f64740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2e8c1b1770 con 0x562f251df950 2024-04-24T09:04:07.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.406+0000 7f2e94f64740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2e8c1b33d0 con 0x562f251ea940 2024-04-24T09:04:07.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.406+0000 7f2e94f64740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2e8c1b1e40 con 0x562f251e0270 2024-04-24T09:04:07.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.406+0000 7f2e8b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562f251df950 0x562f251f66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:07.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.406+0000 7f2e8b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562f251df950 0x562f251f66c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48896/0 (socket says 172.21.15.83:48896) 2024-04-24T09:04:07.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.407+0000 7f2e8b400640 1 -- 172.21.15.83:0/3952094198 learned_addr learned my addr 172.21.15.83:0/3952094198 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:04:07.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.407+0000 7f2e8be00640 1 --2- 172.21.15.83:0/3952094198 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f251ea940 0x562f252028b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:07.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.407+0000 7f2e8aa00640 1 --2- 172.21.15.83:0/3952094198 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562f251e0270 0x562f251e9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:07.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.407+0000 7f2e8b400640 1 -- 172.21.15.83:0/3952094198 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562f251e0270 msgr2=0x562f251e9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:07.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.407+0000 7f2e8b400640 1 --2- 172.21.15.83:0/3952094198 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562f251e0270 0x562f251e9f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.407+0000 7f2e8b400640 1 -- 172.21.15.83:0/3952094198 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f251ea940 msgr2=0x562f252028b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:07.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.407+0000 7f2e8b400640 1 --2- 172.21.15.83:0/3952094198 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f251ea940 0x562f252028b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.407+0000 7f2e8b400640 1 -- 172.21.15.83:0/3952094198 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f24f217d0 con 0x562f251df950 2024-04-24T09:04:07.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.407+0000 7f2e8aa00640 1 --2- 172.21.15.83:0/3952094198 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562f251e0270 0x562f251e9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:04:07.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.407+0000 7f2e8b400640 1 --2- 172.21.15.83:0/3952094198 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562f251df950 0x562f251f66c0 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c02a9a0 tx=0x7f2e7c03e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ce8a613e83eaa46 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:07.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.408+0000 7f2e77400640 1 -- 172.21.15.83:0/3952094198 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e7c035040 con 0x562f251df950 2024-04-24T09:04:07.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.408+0000 7f2e77400640 1 -- 172.21.15.83:0/3952094198 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2e7c056070 con 0x562f251df950 2024-04-24T09:04:07.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.408+0000 7f2e77400640 1 -- 172.21.15.83:0/3952094198 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e7c003850 con 0x562f251df950 2024-04-24T09:04:07.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.408+0000 7f2e94f64740 1 -- 172.21.15.83:0/3952094198 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562f251df950 msgr2=0x562f251f66c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:07.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.408+0000 7f2e94f64740 1 --2- 172.21.15.83:0/3952094198 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562f251df950 0x562f251f66c0 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c02a9a0 tx=0x7f2e7c03e070 comp rx=0 tx=0).stop 2024-04-24T09:04:07.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.408+0000 7f2e94f64740 1 -- 172.21.15.83:0/3952094198 shutdown_connections 2024-04-24T09:04:07.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.408+0000 7f2e94f64740 1 --2- 172.21.15.83:0/3952094198 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f251ea940 0x562f252028b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.408+0000 7f2e94f64740 1 --2- 172.21.15.83:0/3952094198 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562f251e0270 0x562f251e9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.408+0000 7f2e94f64740 1 --2- 172.21.15.83:0/3952094198 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562f251df950 0x562f251f66c0 unknown :-1 s=CLOSED pgs=1973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.408+0000 7f2e94f64740 1 -- 172.21.15.83:0/3952094198 >> 172.21.15.83:0/3952094198 conn(0x562f251dbc70 msgr2=0x562f251dc050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:07.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.409+0000 7f2e94f64740 1 -- 172.21.15.83:0/3952094198 shutdown_connections 2024-04-24T09:04:07.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.409+0000 7f2e94f64740 1 -- 172.21.15.83:0/3952094198 wait complete. 2024-04-24T09:04:07.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.409+0000 7f2e94f64740 1 -- start start 2024-04-24T09:04:07.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.409+0000 7f2e94f64740 1 Processor -- start 2024-04-24T09:04:07.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.410+0000 7f2e94f64740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562f251df950 0x562f251eb780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:07.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.410+0000 7f2e94f64740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562f251e0270 0x562f251ebcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:07.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.410+0000 7f2e94f64740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f251ea940 0x562f251ec200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:07.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.410+0000 7f2e94f64740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2e8c1b2ca0 con 0x562f251df950 2024-04-24T09:04:07.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.410+0000 7f2e94f64740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2e8c1b2570 con 0x562f251ea940 2024-04-24T09:04:07.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.410+0000 7f2e94f64740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2e8c1af770 con 0x562f251e0270 2024-04-24T09:04:07.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.410+0000 7f2e94f64740 10 client.0 _ll_register_callbacks cb 0x7f2e84018400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:04:07.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.410+0000 7f2e8b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562f251df950 0x562f251eb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:07.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.410+0000 7f2e8b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562f251df950 0x562f251eb780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48910/0 (socket says 172.21.15.83:48910) 2024-04-24T09:04:07.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.410+0000 7f2e8b400640 1 -- 172.21.15.83:0/3870818130 learned_addr learned my addr 172.21.15.83:0/3870818130 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:04:07.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.410+0000 7f2e8b400640 1 -- 172.21.15.83:0/3870818130 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562f251e0270 msgr2=0x562f251ebcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:07.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.410+0000 7f2e8b400640 1 --2- 172.21.15.83:0/3870818130 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562f251e0270 0x562f251ebcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.410+0000 7f2e8b400640 1 -- 172.21.15.83:0/3870818130 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f251ea940 msgr2=0x562f251ec200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:07.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.410+0000 7f2e8be00640 1 --2- 172.21.15.83:0/3870818130 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f251ea940 0x562f251ec200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:07.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.410+0000 7f2e8b400640 1 --2- 172.21.15.83:0/3870818130 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f251ea940 0x562f251ec200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.410+0000 7f2e8b400640 1 -- 172.21.15.83:0/3870818130 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f251f0f00 con 0x562f251df950 2024-04-24T09:04:07.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.411+0000 7f2e8b400640 1 --2- 172.21.15.83:0/3870818130 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562f251df950 0x562f251eb780 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c031440 tx=0x7f2e7c031820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c1a10c7d6208679 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:07.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.411+0000 7f2e74c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:04:07.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.411+0000 7f2e74c00640 1 -- 172.21.15.83:0/3870818130 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e7c05f060 con 0x562f251df950 2024-04-24T09:04:07.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.411+0000 7f2e74c00640 1 -- 172.21.15.83:0/3870818130 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7f2e7c0043d0 con 0x562f251df950 2024-04-24T09:04:07.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.411+0000 7f2e94f64740 1 -- 172.21.15.83:0/3870818130 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e7c056070 con 0x562f251df950 2024-04-24T09:04:07.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.411+0000 7f2e74c00640 1 -- 172.21.15.83:0/3870818130 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f2e7c066050 con 0x562f251df950 2024-04-24T09:04:07.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.411+0000 7f2e94f64740 10 client.20082 fetch_fsmap learned FSMap version 41 2024-04-24T09:04:07.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.411+0000 7f2e94f64740 1 -- 172.21.15.83:0/3870818130 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x562f25261410 con 0x562f251df950 2024-04-24T09:04:07.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.412+0000 7f2e74c00640 1 -- 172.21.15.83:0/3870818130 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2e7c056070 con 0x562f251df950 2024-04-24T09:04:07.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.412+0000 7f2e74c00640 1 -- 172.21.15.83:0/3870818130 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7f2e7c03c440 con 0x562f251df950 2024-04-24T09:04:07.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.412+0000 7f2e94f64740 10 client.20082 fetch_fsmap finished waiting for FSMap version 41 2024-04-24T09:04:07.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.412+0000 7f2e94f64740 10 client.20082 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-24T09:04:07.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.412+0000 7f2e94f64740 20 client.20082 populate_metadata read hostname 'smithi083' 2024-04-24T09:04:07.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.412+0000 7f2e94f64740 1 --2- 172.21.15.83:0/3870818130 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x562f25261b30 0x562f25263f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:07.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.412+0000 7f2e94f64740 4 client.20082 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:04:07.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.412+0000 7f2e94f64740 1 -- 172.21.15.83:0/3870818130 --> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562f2514ca60 con 0x562f25261b30 2024-04-24T09:04:07.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.413+0000 7f2e8aa00640 1 --2- 172.21.15.83:0/3870818130 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x562f25261b30 0x562f25263f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:07.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.413+0000 7f2e8aa00640 1 --2- 172.21.15.83:0/3870818130 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x562f25261b30 0x562f25263f50 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3b0699a9f08b28fc server_cookie=fde086a447d0ef9a in_seq=0 out_seq=0 2024-04-24T09:04:07.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.413+0000 7f2e74c00640 10 client.20082 ms_handle_connect on v2:172.21.15.138:6836/900566754 2024-04-24T09:04:07.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.414+0000 7f2e74c00640 1 -- 172.21.15.83:0/3870818130 <== mds.1 v2:172.21.15.138:6836/900566754 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562f2514ca60 con 0x562f25261b30 2024-04-24T09:04:07.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.414+0000 7f2e74c00640 10 client.20082 handle_command_reply: tid=0 2024-04-24T09:04:07.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.414+0000 7f2e94f64740 1 client.20082 shutdown 2024-04-24T09:04:07.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.414+0000 7f2e94f64740 1 -- 172.21.15.83:0/3870818130 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562f251df950 msgr2=0x562f251eb780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:07.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.414+0000 7f2e94f64740 1 --2- 172.21.15.83:0/3870818130 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562f251df950 0x562f251eb780 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c031440 tx=0x7f2e7c031820 comp rx=0 tx=0).stop 2024-04-24T09:04:07.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.414+0000 7f2e94f64740 1 -- 172.21.15.83:0/3870818130 shutdown_connections 2024-04-24T09:04:07.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.414+0000 7f2e94f64740 1 --2- 172.21.15.83:0/3870818130 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f251ea940 0x562f251ec200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.414+0000 7f2e94f64740 1 --2- 172.21.15.83:0/3870818130 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x562f25261b30 0x562f25263f50 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.414+0000 7f2e94f64740 1 --2- 172.21.15.83:0/3870818130 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562f251e0270 0x562f251ebcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.414+0000 7f2e94f64740 1 --2- 172.21.15.83:0/3870818130 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562f251df950 0x562f251eb780 unknown :-1 s=CLOSED pgs=1974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.414+0000 7f2e74c00640 0 client.20082 ms_handle_reset on v2:172.21.15.138:6836/900566754 2024-04-24T09:04:07.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.414+0000 7f2e94f64740 1 -- 172.21.15.83:0/3870818130 >> 172.21.15.83:0/3870818130 conn(0x562f251dbc70 msgr2=0x562f251da560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:07.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.414+0000 7f2e94f64740 1 -- 172.21.15.83:0/3870818130 shutdown_connections 2024-04-24T09:04:07.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.414+0000 7f2e94f64740 1 -- 172.21.15.83:0/3870818130 wait complete. 2024-04-24T09:04:07.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.415+0000 7f2e94f64740 20 client.20082 trim_cache size 0 max 16384 2024-04-24T09:04:07.421 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:04:07.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.415+0000 7f2e92000640 1 -- 172.21.15.83:0/1603290905 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f2e700a3d20 msgr2=0x7f2e700a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:07.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.415+0000 7f2e92000640 1 --2- 172.21.15.83:0/1603290905 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f2e700a3d20 0x7f2e700a61e0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c000c00 tx=0x7f2e7c005e80 comp rx=0 tx=0).stop 2024-04-24T09:04:07.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.415+0000 7f2e92000640 1 -- 172.21.15.83:0/1603290905 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e8c10e8e0 msgr2=0x7f2e8c19a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:07.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.415+0000 7f2e92000640 1 --2- 172.21.15.83:0/1603290905 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e8c10e8e0 0x7f2e8c19a880 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7f2e8400cab0 tx=0x7f2e8400cf70 comp rx=0 tx=0).stop 2024-04-24T09:04:07.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.415+0000 7f2e92000640 1 -- 172.21.15.83:0/1603290905 shutdown_connections 2024-04-24T09:04:07.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.415+0000 7f2e92000640 1 --2- 172.21.15.83:0/1603290905 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f2e700a3d20 0x7f2e700a61e0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.415+0000 7f2e92000640 1 --2- 172.21.15.83:0/1603290905 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e8c10e8e0 0x7f2e8c19a880 unknown :-1 s=CLOSED pgs=1970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.415+0000 7f2e92000640 1 --2- 172.21.15.83:0/1603290905 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e8c074bf0 0x7f2e8c1a0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.415+0000 7f2e92000640 1 --2- 172.21.15.83:0/1603290905 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2e8c0742d0 0x7f2e8c1a06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.415+0000 7f2e92000640 1 -- 172.21.15.83:0/1603290905 >> 172.21.15.83:0/1603290905 conn(0x7f2e8c079450 msgr2=0x7f2e8c0ff010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:07.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.416+0000 7f2e92000640 1 -- 172.21.15.83:0/1603290905 shutdown_connections 2024-04-24T09:04:07.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.416+0000 7f2e92000640 1 -- 172.21.15.83:0/1603290905 wait complete. 2024-04-24T09:04:07.435 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T09:04:07.624 INFO:tasks.workunit.client.0.smithi083.stdout: 1 71931 0.59 MB/sec execute 521 sec latency 45.192 ms 2024-04-24T09:04:07.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.647+0000 7f4184e00640 1 -- 172.21.15.83:0/1387923760 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4180106250 msgr2=0x7f41801066e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:07.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.647+0000 7f4184e00640 1 --2- 172.21.15.83:0/1387923760 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4180106250 0x7f41801066e0 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7f4178009900 tx=0x7f417802a470 comp rx=0 tx=0).stop 2024-04-24T09:04:07.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.648+0000 7f4184e00640 1 -- 172.21.15.83:0/1387923760 shutdown_connections 2024-04-24T09:04:07.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.648+0000 7f4184e00640 1 --2- 172.21.15.83:0/1387923760 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4180106250 0x7f41801066e0 unknown :-1 s=CLOSED pgs=1975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.648+0000 7f4184e00640 1 --2- 172.21.15.83:0/1387923760 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4180101770 0x7f4180105c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.648+0000 7f4184e00640 1 --2- 172.21.15.83:0/1387923760 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4180100e50 0x7f4180101230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.648+0000 7f4184e00640 1 -- 172.21.15.83:0/1387923760 >> 172.21.15.83:0/1387923760 conn(0x7f41800fc880 msgr2=0x7f41800feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:07.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.648+0000 7f4184e00640 1 -- 172.21.15.83:0/1387923760 shutdown_connections 2024-04-24T09:04:07.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.648+0000 7f4184e00640 1 -- 172.21.15.83:0/1387923760 wait complete. 2024-04-24T09:04:07.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.649+0000 7f4184e00640 1 Processor -- start 2024-04-24T09:04:07.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.649+0000 7f4184e00640 1 -- start start 2024-04-24T09:04:07.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.649+0000 7f4184e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4180100e50 0x7f41801a48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:07.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.649+0000 7f4184e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4180101770 0x7f41801a4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:07.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.649+0000 7f4184e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4180106250 0x7f418019ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:07.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.649+0000 7f4184e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4180118750 con 0x7f4180101770 2024-04-24T09:04:07.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.649+0000 7f4184e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f41801185d0 con 0x7f4180100e50 2024-04-24T09:04:07.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.649+0000 7f4184e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f41801188d0 con 0x7f4180106250 2024-04-24T09:04:07.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.650+0000 7f4176a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4180101770 0x7f41801a4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:07.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.650+0000 7f4176a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4180101770 0x7f41801a4e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48930/0 (socket says 172.21.15.83:48930) 2024-04-24T09:04:07.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.650+0000 7f4176a00640 1 -- 172.21.15.83:0/73170746 learned_addr learned my addr 172.21.15.83:0/73170746 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:04:07.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.650+0000 7f417fe00640 1 --2- 172.21.15.83:0/73170746 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4180106250 0x7f418019ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:07.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.650+0000 7f417f400640 1 --2- 172.21.15.83:0/73170746 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4180100e50 0x7f41801a48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:07.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.650+0000 7f4176a00640 1 -- 172.21.15.83:0/73170746 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4180106250 msgr2=0x7f418019ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:07.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.650+0000 7f4176a00640 1 --2- 172.21.15.83:0/73170746 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4180106250 0x7f418019ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.650+0000 7f4176a00640 1 -- 172.21.15.83:0/73170746 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4180100e50 msgr2=0x7f41801a48c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:07.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.650+0000 7f4176a00640 1 --2- 172.21.15.83:0/73170746 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4180100e50 0x7f41801a48c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.650+0000 7f4176a00640 1 -- 172.21.15.83:0/73170746 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f418019f1f0 con 0x7f4180101770 2024-04-24T09:04:07.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.650+0000 7f417fe00640 1 --2- 172.21.15.83:0/73170746 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4180106250 0x7f418019ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:04:07.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.650+0000 7f4176a00640 1 --2- 172.21.15.83:0/73170746 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4180101770 0x7f41801a4e00 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7f416800c940 tx=0x7f416800ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:07.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.651+0000 7f417cc00640 1 -- 172.21.15.83:0/73170746 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4168004220 con 0x7f4180101770 2024-04-24T09:04:07.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.651+0000 7f417cc00640 1 -- 172.21.15.83:0/73170746 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f41680043c0 con 0x7f4180101770 2024-04-24T09:04:07.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.651+0000 7f4184e00640 1 -- 172.21.15.83:0/73170746 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f418019f4e0 con 0x7f4180101770 2024-04-24T09:04:07.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.651+0000 7f417cc00640 1 -- 172.21.15.83:0/73170746 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4168004a00 con 0x7f4180101770 2024-04-24T09:04:07.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.651+0000 7f4184e00640 1 -- 172.21.15.83:0/73170746 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41801ab780 con 0x7f4180101770 2024-04-24T09:04:07.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.655+0000 7f41891a6740 1 Processor -- start 2024-04-24T09:04:07.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.655+0000 7f41891a6740 1 -- start start 2024-04-24T09:04:07.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.655+0000 7f417cc00640 1 -- 172.21.15.83:0/73170746 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f41680079b0 con 0x7f4180101770 2024-04-24T09:04:07.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.656+0000 7f41891a6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5632d158cb40 0x5632d158cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:07.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.656+0000 7f41891a6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5632d158d460 0x5632d15975f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:07.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.656+0000 7f41891a6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5632d1597b30 0x5632d1599f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:07.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.656+0000 7f41891a6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5632d14f21d0 con 0x5632d1597b30 2024-04-24T09:04:07.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.656+0000 7f41891a6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5632d1498450 con 0x5632d158d460 2024-04-24T09:04:07.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.656+0000 7f41891a6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5632d14ee190 con 0x5632d158cb40 2024-04-24T09:04:07.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.656+0000 7f4176a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5632d1597b30 0x5632d1599f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:07.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.656+0000 7f4176a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5632d1597b30 0x5632d1599f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48938/0 (socket says 172.21.15.83:48938) 2024-04-24T09:04:07.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.656+0000 7f4176a00640 1 -- 172.21.15.83:0/4281797422 learned_addr learned my addr 172.21.15.83:0/4281797422 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:04:07.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.656+0000 7f417fe00640 1 --2- 172.21.15.83:0/4281797422 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5632d158cb40 0x5632d158cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:07.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.656+0000 7f417f400640 1 --2- 172.21.15.83:0/4281797422 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5632d158d460 0x5632d15975f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:07.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.657+0000 7f4176a00640 1 -- 172.21.15.83:0/4281797422 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5632d158cb40 msgr2=0x5632d158cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:07.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.657+0000 7f4176a00640 1 --2- 172.21.15.83:0/4281797422 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5632d158cb40 0x5632d158cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.657+0000 7f4176a00640 1 -- 172.21.15.83:0/4281797422 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5632d158d460 msgr2=0x5632d15975f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:07.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.657+0000 7f4176a00640 1 --2- 172.21.15.83:0/4281797422 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5632d158d460 0x5632d15975f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.657+0000 7f4176a00640 1 -- 172.21.15.83:0/4281797422 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632d1555310 con 0x5632d1597b30 2024-04-24T09:04:07.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.657+0000 7f4176a00640 1 --2- 172.21.15.83:0/4281797422 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5632d1597b30 0x5632d1599f20 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7f41680cdeb0 tx=0x7f41680d9380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=843d2ae206a25e84 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:07.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.657+0000 7f4174c00640 1 -- 172.21.15.83:0/4281797422 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41680e3470 con 0x5632d1597b30 2024-04-24T09:04:07.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.657+0000 7f4174c00640 1 -- 172.21.15.83:0/4281797422 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f41680de070 con 0x5632d1597b30 2024-04-24T09:04:07.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.657+0000 7f4174c00640 1 -- 172.21.15.83:0/4281797422 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41680d78d0 con 0x5632d1597b30 2024-04-24T09:04:07.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.657+0000 7f417cc00640 1 --2- 172.21.15.83:0/73170746 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f41640a3d20 0x7f41640a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:07.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.658+0000 7f417cc00640 1 -- 172.21.15.83:0/73170746 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7f41680d0c10 con 0x7f4180101770 2024-04-24T09:04:07.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.658+0000 7f417fe00640 1 --2- 172.21.15.83:0/73170746 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f41640a3d20 0x7f41640a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:07.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.658+0000 7f41891a6740 1 -- 172.21.15.83:0/4281797422 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5632d1597b30 msgr2=0x5632d1599f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:07.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.658+0000 7f41891a6740 1 --2- 172.21.15.83:0/4281797422 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5632d1597b30 0x5632d1599f20 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7f41680cdeb0 tx=0x7f41680d9380 comp rx=0 tx=0).stop 2024-04-24T09:04:07.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.658+0000 7f41891a6740 1 -- 172.21.15.83:0/4281797422 shutdown_connections 2024-04-24T09:04:07.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.658+0000 7f41891a6740 1 --2- 172.21.15.83:0/4281797422 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5632d1597b30 0x5632d1599f20 unknown :-1 s=CLOSED pgs=1977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.658+0000 7f41891a6740 1 --2- 172.21.15.83:0/4281797422 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5632d158d460 0x5632d15975f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.658+0000 7f41891a6740 1 --2- 172.21.15.83:0/4281797422 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5632d158cb40 0x5632d158cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.658+0000 7f41891a6740 1 -- 172.21.15.83:0/4281797422 >> 172.21.15.83:0/4281797422 conn(0x5632d1588e60 msgr2=0x5632d1589240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:07.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.659+0000 7f417fe00640 1 --2- 172.21.15.83:0/73170746 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f41640a3d20 0x7f41640a61e0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f4178031440 tx=0x7f4178009560 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:07.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.659+0000 7f41891a6740 1 -- 172.21.15.83:0/4281797422 shutdown_connections 2024-04-24T09:04:07.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.659+0000 7f41891a6740 1 -- 172.21.15.83:0/4281797422 wait complete. 2024-04-24T09:04:07.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.659+0000 7f41891a6740 1 -- start start 2024-04-24T09:04:07.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.659+0000 7f41891a6740 1 Processor -- start 2024-04-24T09:04:07.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.660+0000 7f41891a6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5632d158cb40 0x5632d15975a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:07.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.660+0000 7f41891a6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5632d158d460 0x5632d1594890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:07.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.660+0000 7f41891a6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5632d1597b30 0x5632d1594fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:07.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.660+0000 7f41891a6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5632d159a9c0 con 0x5632d1597b30 2024-04-24T09:04:07.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.660+0000 7f41891a6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5632d14ab810 con 0x5632d158d460 2024-04-24T09:04:07.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.660+0000 7f41891a6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5632d159ab40 con 0x5632d158cb40 2024-04-24T09:04:07.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.660+0000 7f41891a6740 10 client.0 _ll_register_callbacks cb 0x5632d1479140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:04:07.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.660+0000 7f417fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5632d1597b30 0x5632d1594fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:07.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.660+0000 7f417fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5632d1597b30 0x5632d1594fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48944/0 (socket says 172.21.15.83:48944) 2024-04-24T09:04:07.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.660+0000 7f417fe00640 1 -- 172.21.15.83:0/3295286824 learned_addr learned my addr 172.21.15.83:0/3295286824 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:04:07.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.660+0000 7f4176a00640 1 --2- 172.21.15.83:0/3295286824 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5632d158d460 0x5632d1594890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:07.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.660+0000 7f417f400640 1 --2- 172.21.15.83:0/3295286824 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5632d158cb40 0x5632d15975a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:07.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.661+0000 7f417fe00640 1 -- 172.21.15.83:0/3295286824 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5632d158cb40 msgr2=0x5632d15975a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:07.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.661+0000 7f417fe00640 1 --2- 172.21.15.83:0/3295286824 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5632d158cb40 0x5632d15975a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.661+0000 7f417fe00640 1 -- 172.21.15.83:0/3295286824 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5632d158d460 msgr2=0x5632d1594890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:07.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.661+0000 7f417fe00640 1 --2- 172.21.15.83:0/3295286824 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5632d158d460 0x5632d1594890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.661+0000 7f417fe00640 1 -- 172.21.15.83:0/3295286824 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632d14bc3f0 con 0x5632d1597b30 2024-04-24T09:04:07.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.661+0000 7f417fe00640 1 --2- 172.21.15.83:0/3295286824 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5632d1597b30 0x5632d1594fd0 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7f4178033000 tx=0x7f417802a470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c39eaa7a3d1d0e69 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:07.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.661+0000 7f416d600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:04:07.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.661+0000 7f416d600640 1 -- 172.21.15.83:0/3295286824 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f417804c020 con 0x5632d1597b30 2024-04-24T09:04:07.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.661+0000 7f416d600640 1 -- 172.21.15.83:0/3295286824 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7f41780037f0 con 0x5632d1597b30 2024-04-24T09:04:07.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.661+0000 7f41891a6740 1 -- 172.21.15.83:0/3295286824 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41680de070 con 0x5632d1597b30 2024-04-24T09:04:07.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.662+0000 7f416d600640 1 -- 172.21.15.83:0/3295286824 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f4178057050 con 0x5632d1597b30 2024-04-24T09:04:07.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.662+0000 7f41891a6740 10 client.20094 fetch_fsmap learned FSMap version 41 2024-04-24T09:04:07.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.662+0000 7f41891a6740 1 -- 172.21.15.83:0/3295286824 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5632d12043f0 con 0x5632d1597b30 2024-04-24T09:04:07.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.662+0000 7f416d600640 1 -- 172.21.15.83:0/3295286824 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f41680de070 con 0x5632d1597b30 2024-04-24T09:04:07.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.662+0000 7f416d600640 1 -- 172.21.15.83:0/3295286824 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7f4178032790 con 0x5632d1597b30 2024-04-24T09:04:07.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.662+0000 7f41891a6740 10 client.20094 fetch_fsmap finished waiting for FSMap version 41 2024-04-24T09:04:07.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.662+0000 7f41891a6740 10 client.20094 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:04:07.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.662+0000 7f41891a6740 20 client.20094 populate_metadata read hostname 'smithi083' 2024-04-24T09:04:07.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.662+0000 7f41891a6740 1 --2- 172.21.15.83:0/3295286824 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5632d15a7df0 0x5632d15aa270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:07.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.662+0000 7f41891a6740 4 client.20094 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:04:07.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.662+0000 7f41891a6740 1 -- 172.21.15.83:0/3295286824 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5632d14ffbf0 con 0x5632d15a7df0 2024-04-24T09:04:07.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.663+0000 7f417f400640 1 --2- 172.21.15.83:0/3295286824 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5632d15a7df0 0x5632d15aa270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:07.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.664+0000 7f417f400640 1 --2- 172.21.15.83:0/3295286824 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5632d15a7df0 0x5632d15aa270 crc :-1 s=READY pgs=325 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=faef2833eab80e90 server_cookie=e29a996a9d3592dc in_seq=0 out_seq=0 2024-04-24T09:04:07.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.664+0000 7f416d600640 10 client.20094 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:04:07.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.666+0000 7f416d600640 1 -- 172.21.15.83:0/3295286824 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5632d14ffbf0 con 0x5632d15a7df0 2024-04-24T09:04:07.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.666+0000 7f416d600640 10 client.20094 handle_command_reply: tid=0 2024-04-24T09:04:07.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.667+0000 7f41891a6740 1 client.20094 shutdown 2024-04-24T09:04:07.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.667+0000 7f41891a6740 1 -- 172.21.15.83:0/3295286824 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5632d1597b30 msgr2=0x5632d1594fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:07.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.667+0000 7f41891a6740 1 --2- 172.21.15.83:0/3295286824 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5632d1597b30 0x5632d1594fd0 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7f4178033000 tx=0x7f417802a470 comp rx=0 tx=0).stop 2024-04-24T09:04:07.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.667+0000 7f41891a6740 1 -- 172.21.15.83:0/3295286824 shutdown_connections 2024-04-24T09:04:07.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.667+0000 7f41891a6740 1 --2- 172.21.15.83:0/3295286824 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5632d15a7df0 0x5632d15aa270 crc :-1 s=READY pgs=325 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.667+0000 7f41891a6740 1 --2- 172.21.15.83:0/3295286824 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5632d1597b30 0x5632d1594fd0 unknown :-1 s=CLOSED pgs=1978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.667+0000 7f41891a6740 1 --2- 172.21.15.83:0/3295286824 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5632d158d460 0x5632d1594890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.667+0000 7f41891a6740 1 --2- 172.21.15.83:0/3295286824 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5632d158cb40 0x5632d15975a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.667+0000 7f41891a6740 1 -- 172.21.15.83:0/3295286824 >> 172.21.15.83:0/3295286824 conn(0x5632d1588e60 msgr2=0x5632d158ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:07.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.667+0000 7f416d600640 0 client.20094 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:04:07.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.667+0000 7f41891a6740 1 -- 172.21.15.83:0/3295286824 shutdown_connections 2024-04-24T09:04:07.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.667+0000 7f41891a6740 1 -- 172.21.15.83:0/3295286824 wait complete. 2024-04-24T09:04:07.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.667+0000 7f41891a6740 20 client.20094 trim_cache size 0 max 16384 2024-04-24T09:04:07.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.682+0000 7f41891a6740 1 Processor -- start 2024-04-24T09:04:07.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.682+0000 7f41891a6740 1 -- start start 2024-04-24T09:04:07.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.682+0000 7f41891a6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5632d158cb40 0x5632d15939f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:07.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.682+0000 7f41891a6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5632d158d460 0x5632d15970d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:07.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.682+0000 7f41891a6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5632d1597b30 0x5632d15afb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:07.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.682+0000 7f41891a6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f41801b5aa0 con 0x5632d158d460 2024-04-24T09:04:07.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.682+0000 7f41891a6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f41801b7700 con 0x5632d1597b30 2024-04-24T09:04:07.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.682+0000 7f41891a6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f41801b6170 con 0x5632d158cb40 2024-04-24T09:04:07.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.683+0000 7f4176a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5632d158d460 0x5632d15970d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:07.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.683+0000 7f4176a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5632d158d460 0x5632d15970d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48950/0 (socket says 172.21.15.83:48950) 2024-04-24T09:04:07.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.683+0000 7f4176a00640 1 -- 172.21.15.83:0/2232283321 learned_addr learned my addr 172.21.15.83:0/2232283321 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:04:07.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.683+0000 7f417fe00640 1 --2- 172.21.15.83:0/2232283321 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5632d1597b30 0x5632d15afb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:07.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.683+0000 7f417f400640 1 --2- 172.21.15.83:0/2232283321 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5632d158cb40 0x5632d15939f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:07.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.683+0000 7f4176a00640 1 -- 172.21.15.83:0/2232283321 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5632d158cb40 msgr2=0x5632d15939f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:07.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.683+0000 7f4176a00640 1 --2- 172.21.15.83:0/2232283321 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5632d158cb40 0x5632d15939f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.683+0000 7f4176a00640 1 -- 172.21.15.83:0/2232283321 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5632d1597b30 msgr2=0x5632d15afb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:07.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.683+0000 7f4176a00640 1 --2- 172.21.15.83:0/2232283321 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5632d1597b30 0x5632d15afb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.683+0000 7f4176a00640 1 -- 172.21.15.83:0/2232283321 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632d12ce7d0 con 0x5632d158d460 2024-04-24T09:04:07.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.684+0000 7f4176a00640 1 --2- 172.21.15.83:0/2232283321 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5632d158d460 0x5632d15970d0 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7f4168099160 tx=0x7f416800cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ea66890839d92f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:07.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.684+0000 7f416fe00640 1 -- 172.21.15.83:0/2232283321 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f416801d070 con 0x5632d158d460 2024-04-24T09:04:07.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.684+0000 7f416fe00640 1 -- 172.21.15.83:0/2232283321 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f41680de040 con 0x5632d158d460 2024-04-24T09:04:07.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.684+0000 7f416fe00640 1 -- 172.21.15.83:0/2232283321 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f416809ce90 con 0x5632d158d460 2024-04-24T09:04:07.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.685+0000 7f41891a6740 1 -- 172.21.15.83:0/2232283321 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5632d158d460 msgr2=0x5632d15970d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:07.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.685+0000 7f41891a6740 1 --2- 172.21.15.83:0/2232283321 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5632d158d460 0x5632d15970d0 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7f4168099160 tx=0x7f416800cf30 comp rx=0 tx=0).stop 2024-04-24T09:04:07.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.685+0000 7f41891a6740 1 -- 172.21.15.83:0/2232283321 shutdown_connections 2024-04-24T09:04:07.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.685+0000 7f41891a6740 1 --2- 172.21.15.83:0/2232283321 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5632d1597b30 0x5632d15afb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.685+0000 7f41891a6740 1 --2- 172.21.15.83:0/2232283321 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5632d158d460 0x5632d15970d0 unknown :-1 s=CLOSED pgs=1979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.685+0000 7f41891a6740 1 --2- 172.21.15.83:0/2232283321 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5632d158cb40 0x5632d15939f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.685+0000 7f41891a6740 1 -- 172.21.15.83:0/2232283321 >> 172.21.15.83:0/2232283321 conn(0x5632d1588e60 msgr2=0x5632d15876c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:07.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.685+0000 7f41891a6740 1 -- 172.21.15.83:0/2232283321 shutdown_connections 2024-04-24T09:04:07.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.685+0000 7f41891a6740 1 -- 172.21.15.83:0/2232283321 wait complete. 2024-04-24T09:04:07.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.685+0000 7f41891a6740 1 -- start start 2024-04-24T09:04:07.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.685+0000 7f41891a6740 1 Processor -- start 2024-04-24T09:04:07.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.686+0000 7f41891a6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5632d158cb40 0x5632d15a0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:07.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.686+0000 7f41891a6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5632d158d460 0x5632d1591b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:07.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.686+0000 7f41891a6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5632d1597b30 0x5632d1592050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:07.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.686+0000 7f41891a6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f41801b6fd0 con 0x5632d158d460 2024-04-24T09:04:07.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.686+0000 7f41891a6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f41801b68a0 con 0x5632d1597b30 2024-04-24T09:04:07.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.686+0000 7f41891a6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f41801b3aa0 con 0x5632d158cb40 2024-04-24T09:04:07.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.686+0000 7f41891a6740 10 client.0 _ll_register_callbacks cb 0x7f417803f070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:04:07.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.686+0000 7f4176a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5632d158d460 0x5632d1591b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:07.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.686+0000 7f4176a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5632d158d460 0x5632d1591b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48964/0 (socket says 172.21.15.83:48964) 2024-04-24T09:04:07.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.686+0000 7f4176a00640 1 -- 172.21.15.83:0/3041543955 learned_addr learned my addr 172.21.15.83:0/3041543955 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:04:07.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.686+0000 7f417f400640 1 --2- 172.21.15.83:0/3041543955 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5632d158cb40 0x5632d15a0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:07.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.687+0000 7f4176a00640 1 -- 172.21.15.83:0/3041543955 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5632d158cb40 msgr2=0x5632d15a0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:07.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.687+0000 7f4176a00640 1 --2- 172.21.15.83:0/3041543955 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5632d158cb40 0x5632d15a0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.687+0000 7f4176a00640 1 -- 172.21.15.83:0/3041543955 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5632d1597b30 msgr2=0x5632d1592050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:07.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.687+0000 7f4176a00640 1 --2- 172.21.15.83:0/3041543955 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5632d1597b30 0x5632d1592050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.687+0000 7f4176a00640 1 -- 172.21.15.83:0/3041543955 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632d1592880 con 0x5632d158d460 2024-04-24T09:04:07.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.687+0000 7f417f400640 1 --2- 172.21.15.83:0/3041543955 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5632d158cb40 0x5632d15a0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:04:07.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.687+0000 7f4176a00640 1 --2- 172.21.15.83:0/3041543955 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5632d158d460 0x5632d1591b10 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7f416809c490 tx=0x7f41680d48f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b536ebe2fc981fb0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:07.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.687+0000 7f416d600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:04:07.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.687+0000 7f416d600640 1 -- 172.21.15.83:0/3041543955 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f416801d040 con 0x5632d158d460 2024-04-24T09:04:07.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.687+0000 7f416d600640 1 -- 172.21.15.83:0/3041543955 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7f416809aa90 con 0x5632d158d460 2024-04-24T09:04:07.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.687+0000 7f41891a6740 1 -- 172.21.15.83:0/3041543955 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41680de040 con 0x5632d158d460 2024-04-24T09:04:07.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.687+0000 7f416d600640 1 -- 172.21.15.83:0/3041543955 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f416809c560 con 0x5632d158d460 2024-04-24T09:04:07.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.687+0000 7f41891a6740 10 client.20100 fetch_fsmap learned FSMap version 41 2024-04-24T09:04:07.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.687+0000 7f41891a6740 1 -- 172.21.15.83:0/3041543955 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5632d160e410 con 0x5632d158d460 2024-04-24T09:04:07.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.688+0000 7f416d600640 1 -- 172.21.15.83:0/3041543955 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f41680de040 con 0x5632d158d460 2024-04-24T09:04:07.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.688+0000 7f416d600640 1 -- 172.21.15.83:0/3041543955 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7f41680ce1c0 con 0x5632d158d460 2024-04-24T09:04:07.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.688+0000 7f41891a6740 10 client.20100 fetch_fsmap finished waiting for FSMap version 41 2024-04-24T09:04:07.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.688+0000 7f41891a6740 10 client.20100 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:04:07.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.688+0000 7f41891a6740 20 client.20100 populate_metadata read hostname 'smithi083' 2024-04-24T09:04:07.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.688+0000 7f41891a6740 1 --2- 172.21.15.83:0/3041543955 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5632d160e6a0 0x5632d1610a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:07.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.688+0000 7f41891a6740 4 client.20100 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:04:07.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.688+0000 7f41891a6740 1 -- 172.21.15.83:0/3041543955 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5632d14f9a60 con 0x5632d160e6a0 2024-04-24T09:04:07.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.689+0000 7f417f400640 1 --2- 172.21.15.83:0/3041543955 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5632d160e6a0 0x5632d1610a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:07.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.689+0000 7f417f400640 1 --2- 172.21.15.83:0/3041543955 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5632d160e6a0 0x5632d1610a90 crc :-1 s=READY pgs=326 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3275adaecde578a5 server_cookie=ed5b206b3df1e588 in_seq=0 out_seq=0 2024-04-24T09:04:07.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.689+0000 7f416d600640 10 client.20100 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:04:07.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.690+0000 7f416d600640 1 -- 172.21.15.83:0/3041543955 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5632d14f9a60 con 0x5632d160e6a0 2024-04-24T09:04:07.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.690+0000 7f416d600640 10 client.20100 handle_command_reply: tid=0 2024-04-24T09:04:07.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.690+0000 7f41891a6740 1 client.20100 shutdown 2024-04-24T09:04:07.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.690+0000 7f41891a6740 1 -- 172.21.15.83:0/3041543955 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5632d158d460 msgr2=0x5632d1591b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:07.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.690+0000 7f41891a6740 1 --2- 172.21.15.83:0/3041543955 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5632d158d460 0x5632d1591b10 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7f416809c490 tx=0x7f41680d48f0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.690+0000 7f41891a6740 1 -- 172.21.15.83:0/3041543955 shutdown_connections 2024-04-24T09:04:07.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.690+0000 7f41891a6740 1 --2- 172.21.15.83:0/3041543955 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5632d160e6a0 0x5632d1610a90 crc :-1 s=READY pgs=326 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.690+0000 7f41891a6740 1 --2- 172.21.15.83:0/3041543955 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5632d1597b30 0x5632d1592050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.690+0000 7f41891a6740 1 --2- 172.21.15.83:0/3041543955 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5632d158d460 0x5632d1591b10 unknown :-1 s=CLOSED pgs=1980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.690+0000 7f416d600640 0 client.20100 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:04:07.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.690+0000 7f41891a6740 1 --2- 172.21.15.83:0/3041543955 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5632d158cb40 0x5632d15a0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.690+0000 7f41891a6740 1 -- 172.21.15.83:0/3041543955 >> 172.21.15.83:0/3041543955 conn(0x5632d1588e60 msgr2=0x5632d1587f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:07.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.690+0000 7f41891a6740 1 -- 172.21.15.83:0/3041543955 shutdown_connections 2024-04-24T09:04:07.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.690+0000 7f41891a6740 1 -- 172.21.15.83:0/3041543955 wait complete. 2024-04-24T09:04:07.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.691+0000 7f41891a6740 20 client.20100 trim_cache size 0 max 16384 2024-04-24T09:04:07.693 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:04:07.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.691+0000 7f4184e00640 1 -- 172.21.15.83:0/73170746 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f41640a3d20 msgr2=0x7f41640a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:07.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.691+0000 7f4184e00640 1 --2- 172.21.15.83:0/73170746 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f41640a3d20 0x7f41640a61e0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f4178031440 tx=0x7f4178009560 comp rx=0 tx=0).stop 2024-04-24T09:04:07.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.691+0000 7f4184e00640 1 -- 172.21.15.83:0/73170746 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4180101770 msgr2=0x7f41801a4e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:07.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.691+0000 7f4184e00640 1 --2- 172.21.15.83:0/73170746 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4180101770 0x7f41801a4e00 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7f416800c940 tx=0x7f416800ce00 comp rx=0 tx=0).stop 2024-04-24T09:04:07.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.691+0000 7f4184e00640 1 -- 172.21.15.83:0/73170746 shutdown_connections 2024-04-24T09:04:07.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.691+0000 7f4184e00640 1 --2- 172.21.15.83:0/73170746 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f41640a3d20 0x7f41640a61e0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.691+0000 7f4184e00640 1 --2- 172.21.15.83:0/73170746 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4180106250 0x7f418019ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.691+0000 7f4184e00640 1 --2- 172.21.15.83:0/73170746 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4180101770 0x7f41801a4e00 unknown :-1 s=CLOSED pgs=1976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.691+0000 7f4184e00640 1 --2- 172.21.15.83:0/73170746 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4180100e50 0x7f41801a48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:07.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.691+0000 7f4184e00640 1 -- 172.21.15.83:0/73170746 >> 172.21.15.83:0/73170746 conn(0x7f41800fc880 msgr2=0x7f4180104120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:07.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.692+0000 7f4184e00640 1 -- 172.21.15.83:0/73170746 shutdown_connections 2024-04-24T09:04:07.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:07.692+0000 7f4184e00640 1 -- 172.21.15.83:0/73170746 wait complete. 2024-04-24T09:04:07.707 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:04:07 smithi083 ceph-mon[27907]: pgmap v394: 129 pgs: 129 active+clean; 68 MiB data, 944 MiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 389 KiB/s wr, 63 op/s 2024-04-24T09:04:07.954 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:04:07 smithi138 ceph-mon[31441]: pgmap v394: 129 pgs: 129 active+clean; 68 MiB data, 944 MiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 389 KiB/s wr, 63 op/s 2024-04-24T09:04:07.956 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:04:07 smithi189 ceph-mon[31791]: pgmap v394: 129 pgs: 129 active+clean; 68 MiB data, 944 MiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 389 KiB/s wr, 63 op/s 2024-04-24T09:04:08.624 INFO:tasks.workunit.client.0.smithi083.stdout: 1 72057 0.59 MB/sec execute 522 sec latency 54.740 ms 2024-04-24T09:04:08.710 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T09:04:08.710 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T09:04:08.710 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T09:04:08.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.943+0000 7f3434c00640 1 -- 172.21.15.83:0/2220388465 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3430076530 msgr2=0x7f3430076910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:08.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.943+0000 7f3434c00640 1 --2- 172.21.15.83:0/2220388465 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3430076530 0x7f3430076910 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7f3420009900 tx=0x7f342002a3f0 comp rx=0 tx=0).stop 2024-04-24T09:04:08.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.943+0000 7f3434c00640 1 -- 172.21.15.83:0/2220388465 shutdown_connections 2024-04-24T09:04:08.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.943+0000 7f3434c00640 1 --2- 172.21.15.83:0/2220388465 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3430112ee0 0x7f34301152d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:08.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.943+0000 7f3434c00640 1 --2- 172.21.15.83:0/2220388465 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3430076e50 0x7f34301129a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:08.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.943+0000 7f3434c00640 1 --2- 172.21.15.83:0/2220388465 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3430076530 0x7f3430076910 unknown :-1 s=CLOSED pgs=1981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:08.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.943+0000 7f3434c00640 1 -- 172.21.15.83:0/2220388465 >> 172.21.15.83:0/2220388465 conn(0x7f34300ffd90 msgr2=0x7f34301021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:08.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.943+0000 7f3434c00640 1 -- 172.21.15.83:0/2220388465 shutdown_connections 2024-04-24T09:04:08.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.943+0000 7f3434c00640 1 -- 172.21.15.83:0/2220388465 wait complete. 2024-04-24T09:04:08.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.944+0000 7f3434c00640 1 Processor -- start 2024-04-24T09:04:08.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.944+0000 7f3434c00640 1 -- start start 2024-04-24T09:04:08.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.944+0000 7f3434c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3430076530 0x7f343019eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:08.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.944+0000 7f3434c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3430076e50 0x7f343019f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:08.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.944+0000 7f3434c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3430112ee0 0x7f34301a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:08.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.944+0000 7f3434c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f34301188f0 con 0x7f3430112ee0 2024-04-24T09:04:08.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.944+0000 7f3434c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3430118770 con 0x7f3430076e50 2024-04-24T09:04:08.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.944+0000 7f3434c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3430118a70 con 0x7f3430076530 2024-04-24T09:04:08.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.945+0000 7f342e800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3430112ee0 0x7f34301a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:08.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.945+0000 7f342e800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3430112ee0 0x7f34301a3770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45812/0 (socket says 172.21.15.83:45812) 2024-04-24T09:04:08.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.945+0000 7f342e800640 1 -- 172.21.15.83:0/1967064414 learned_addr learned my addr 172.21.15.83:0/1967064414 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:04:08.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.945+0000 7f342de00640 1 --2- 172.21.15.83:0/1967064414 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3430076530 0x7f343019eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:08.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.945+0000 7f342e800640 1 -- 172.21.15.83:0/1967064414 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3430076530 msgr2=0x7f343019eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:08.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.945+0000 7f342e800640 1 --2- 172.21.15.83:0/1967064414 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3430076530 0x7f343019eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:08.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.945+0000 7f342e800640 1 -- 172.21.15.83:0/1967064414 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3430076e50 msgr2=0x7f343019f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:08.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.945+0000 7f342d400640 1 --2- 172.21.15.83:0/1967064414 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3430076e50 0x7f343019f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:08.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.945+0000 7f342e800640 1 --2- 172.21.15.83:0/1967064414 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3430076e50 0x7f343019f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:08.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.945+0000 7f342e800640 1 -- 172.21.15.83:0/1967064414 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34301a3ea0 con 0x7f3430112ee0 2024-04-24T09:04:08.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.945+0000 7f342de00640 1 --2- 172.21.15.83:0/1967064414 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3430076530 0x7f343019eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:04:08.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.946+0000 7f342e800640 1 --2- 172.21.15.83:0/1967064414 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3430112ee0 0x7f34301a3770 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7f342800ccb0 tx=0x7f342800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:08.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.946+0000 7f3426a00640 1 -- 172.21.15.83:0/1967064414 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f342800f450 con 0x7f3430112ee0 2024-04-24T09:04:08.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.946+0000 7f3434c00640 1 -- 172.21.15.83:0/1967064414 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34301a4190 con 0x7f3430112ee0 2024-04-24T09:04:08.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.946+0000 7f3426a00640 1 -- 172.21.15.83:0/1967064414 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f342800faa0 con 0x7f3430112ee0 2024-04-24T09:04:08.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.946+0000 7f3426a00640 1 -- 172.21.15.83:0/1967064414 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3428018b10 con 0x7f3430112ee0 2024-04-24T09:04:08.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.946+0000 7f3434c00640 1 -- 172.21.15.83:0/1967064414 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34301aba20 con 0x7f3430112ee0 2024-04-24T09:04:08.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.949+0000 7f34379ed740 1 Processor -- start 2024-04-24T09:04:08.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.950+0000 7f3426a00640 1 -- 172.21.15.83:0/1967064414 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f342800f5f0 con 0x7f3430112ee0 2024-04-24T09:04:08.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.950+0000 7f34379ed740 1 -- start start 2024-04-24T09:04:08.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.950+0000 7f34379ed740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563721047090 0x563721047470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:08.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.950+0000 7f34379ed740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5637210479b0 0x563721051b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:08.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.950+0000 7f34379ed740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563721052080 0x563721054470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:08.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.950+0000 7f34379ed740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x563720face70 con 0x563721047090 2024-04-24T09:04:08.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.950+0000 7f34379ed740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563720f53380 con 0x5637210479b0 2024-04-24T09:04:08.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.950+0000 7f34379ed740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x563720fa8e30 con 0x563721052080 2024-04-24T09:04:08.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.951+0000 7f342d400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563721047090 0x563721047470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:08.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.951+0000 7f342d400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563721047090 0x563721047470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45818/0 (socket says 172.21.15.83:45818) 2024-04-24T09:04:08.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.951+0000 7f342d400640 1 -- 172.21.15.83:0/66130897 learned_addr learned my addr 172.21.15.83:0/66130897 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:04:08.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.951+0000 7f342d400640 1 -- 172.21.15.83:0/66130897 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563721052080 msgr2=0x563721054470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:08.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.951+0000 7f342de00640 1 --2- 172.21.15.83:0/66130897 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5637210479b0 0x563721051b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:08.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.951+0000 7f342d400640 1 --2- 172.21.15.83:0/66130897 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563721052080 0x563721054470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:08.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.951+0000 7f342d400640 1 -- 172.21.15.83:0/66130897 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5637210479b0 msgr2=0x563721051b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:08.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.951+0000 7f342d400640 1 --2- 172.21.15.83:0/66130897 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5637210479b0 0x563721051b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:08.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.951+0000 7f342d400640 1 -- 172.21.15.83:0/66130897 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563720d877e0 con 0x563721047090 2024-04-24T09:04:08.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.951+0000 7f3426a00640 1 --2- 172.21.15.83:0/1967064414 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f34140a3da0 0x7f34140a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:08.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.952+0000 7f342d400640 1 --2- 172.21.15.83:0/66130897 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563721047090 0x563721047470 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7f341c0058c0 tx=0x7f341c0106e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1653821b2fb80701 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:08.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.952+0000 7f3426a00640 1 -- 172.21.15.83:0/1967064414 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7f34280d1360 con 0x7f3430112ee0 2024-04-24T09:04:08.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.952+0000 7f342de00640 1 --2- 172.21.15.83:0/1967064414 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f34140a3da0 0x7f34140a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:08.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.952+0000 7f341b000640 1 -- 172.21.15.83:0/66130897 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f341c016070 con 0x563721047090 2024-04-24T09:04:08.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.952+0000 7f341b000640 1 -- 172.21.15.83:0/66130897 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f341c0128d0 con 0x563721047090 2024-04-24T09:04:08.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.952+0000 7f341b000640 1 -- 172.21.15.83:0/66130897 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f341c0115f0 con 0x563721047090 2024-04-24T09:04:08.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.953+0000 7f34379ed740 1 -- 172.21.15.83:0/66130897 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563721047090 msgr2=0x563721047470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:08.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.953+0000 7f34379ed740 1 --2- 172.21.15.83:0/66130897 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563721047090 0x563721047470 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7f341c0058c0 tx=0x7f341c0106e0 comp rx=0 tx=0).stop 2024-04-24T09:04:08.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.953+0000 7f342de00640 1 --2- 172.21.15.83:0/1967064414 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f34140a3da0 0x7f34140a6260 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f3420005ab0 tx=0x7f34200058b0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:08.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.953+0000 7f34379ed740 1 -- 172.21.15.83:0/66130897 shutdown_connections 2024-04-24T09:04:08.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.953+0000 7f34379ed740 1 --2- 172.21.15.83:0/66130897 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563721052080 0x563721054470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:08.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.953+0000 7f34379ed740 1 --2- 172.21.15.83:0/66130897 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5637210479b0 0x563721051b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:08.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.953+0000 7f34379ed740 1 --2- 172.21.15.83:0/66130897 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563721047090 0x563721047470 unknown :-1 s=CLOSED pgs=1983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:08.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.953+0000 7f34379ed740 1 -- 172.21.15.83:0/66130897 >> 172.21.15.83:0/66130897 conn(0x5637210433b0 msgr2=0x563721043790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:08.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.953+0000 7f34379ed740 1 -- 172.21.15.83:0/66130897 shutdown_connections 2024-04-24T09:04:08.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.953+0000 7f34379ed740 1 -- 172.21.15.83:0/66130897 wait complete. 2024-04-24T09:04:08.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.953+0000 7f34379ed740 1 -- start start 2024-04-24T09:04:08.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.954+0000 7f34379ed740 1 Processor -- start 2024-04-24T09:04:08.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.954+0000 7f34379ed740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563721047090 0x56372104ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:08.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.954+0000 7f34379ed740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5637210479b0 0x56372104f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:08.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.954+0000 7f34379ed740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563721052080 0x56372104f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:08.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.954+0000 7f34379ed740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x563721054ec0 con 0x5637210479b0 2024-04-24T09:04:08.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.954+0000 7f34379ed740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563720f66660 con 0x563721052080 2024-04-24T09:04:08.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.954+0000 7f34379ed740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x563721055040 con 0x563721047090 2024-04-24T09:04:08.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.954+0000 7f34379ed740 10 client.0 _ll_register_callbacks cb 0x563720f54aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:04:08.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.954+0000 7f342d400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5637210479b0 0x56372104f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:08.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.954+0000 7f342d400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5637210479b0 0x56372104f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45826/0 (socket says 172.21.15.83:45826) 2024-04-24T09:04:08.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.954+0000 7f342d400640 1 -- 172.21.15.83:0/1833708672 learned_addr learned my addr 172.21.15.83:0/1833708672 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:04:08.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.955+0000 7f342de00640 1 --2- 172.21.15.83:0/1833708672 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563721047090 0x56372104ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:08.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.955+0000 7f342e800640 1 --2- 172.21.15.83:0/1833708672 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563721052080 0x56372104f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:08.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.955+0000 7f342d400640 1 -- 172.21.15.83:0/1833708672 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563721047090 msgr2=0x56372104ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:08.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.955+0000 7f342d400640 1 --2- 172.21.15.83:0/1833708672 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563721047090 0x56372104ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:08.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.955+0000 7f342d400640 1 -- 172.21.15.83:0/1833708672 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563721052080 msgr2=0x56372104f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:08.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.955+0000 7f342d400640 1 --2- 172.21.15.83:0/1833708672 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563721052080 0x56372104f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:08.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.955+0000 7f342d400640 1 -- 172.21.15.83:0/1833708672 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563720de7f70 con 0x5637210479b0 2024-04-24T09:04:08.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.955+0000 7f342e800640 1 --2- 172.21.15.83:0/1833708672 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563721052080 0x56372104f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:04:08.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.955+0000 7f342d400640 1 --2- 172.21.15.83:0/1833708672 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5637210479b0 0x56372104f460 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7f341c005940 tx=0x7f341c009f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7f4d11b73c84e55 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:08.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.955+0000 7f340f400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:04:08.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.956+0000 7f340f400640 1 -- 172.21.15.83:0/1833708672 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f341c029060 con 0x5637210479b0 2024-04-24T09:04:08.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.956+0000 7f340f400640 1 -- 172.21.15.83:0/1833708672 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7f341c012a80 con 0x5637210479b0 2024-04-24T09:04:08.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.956+0000 7f34379ed740 1 -- 172.21.15.83:0/1833708672 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f341c0128d0 con 0x5637210479b0 2024-04-24T09:04:08.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.956+0000 7f340f400640 1 -- 172.21.15.83:0/1833708672 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f341c030050 con 0x5637210479b0 2024-04-24T09:04:08.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.956+0000 7f34379ed740 10 client.20112 fetch_fsmap learned FSMap version 41 2024-04-24T09:04:08.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.956+0000 7f34379ed740 1 -- 172.21.15.83:0/1833708672 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x563720c7f170 con 0x5637210479b0 2024-04-24T09:04:08.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.956+0000 7f340f400640 1 -- 172.21.15.83:0/1833708672 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f341c0128d0 con 0x5637210479b0 2024-04-24T09:04:08.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.957+0000 7f340f400640 1 -- 172.21.15.83:0/1833708672 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7f341c024410 con 0x5637210479b0 2024-04-24T09:04:08.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.957+0000 7f34379ed740 10 client.20112 fetch_fsmap finished waiting for FSMap version 41 2024-04-24T09:04:08.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.957+0000 7f34379ed740 10 client.20112 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:04:08.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.957+0000 7f34379ed740 20 client.20112 populate_metadata read hostname 'smithi083' 2024-04-24T09:04:08.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.957+0000 7f34379ed740 1 --2- 172.21.15.83:0/1833708672 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x563721062370 0x563721064760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:08.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.957+0000 7f34379ed740 4 client.20112 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:04:08.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.957+0000 7f34379ed740 1 -- 172.21.15.83:0/1833708672 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563720fba890 con 0x563721062370 2024-04-24T09:04:08.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.957+0000 7f342de00640 1 --2- 172.21.15.83:0/1833708672 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x563721062370 0x563721064760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:08.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.959+0000 7f342de00640 1 --2- 172.21.15.83:0/1833708672 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x563721062370 0x563721064760 crc :-1 s=READY pgs=327 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=794ed5bf682c8c88 server_cookie=bfce8679a0861f34 in_seq=0 out_seq=0 2024-04-24T09:04:08.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.959+0000 7f340f400640 10 client.20112 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:04:08.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.960+0000 7f340f400640 1 -- 172.21.15.83:0/1833708672 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563720fba890 con 0x563721062370 2024-04-24T09:04:08.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.960+0000 7f340f400640 10 client.20112 handle_command_reply: tid=0 2024-04-24T09:04:08.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.961+0000 7f34379ed740 1 client.20112 shutdown 2024-04-24T09:04:08.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.961+0000 7f34379ed740 1 -- 172.21.15.83:0/1833708672 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5637210479b0 msgr2=0x56372104f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:08.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.961+0000 7f34379ed740 1 --2- 172.21.15.83:0/1833708672 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5637210479b0 0x56372104f460 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7f341c005940 tx=0x7f341c009f70 comp rx=0 tx=0).stop 2024-04-24T09:04:08.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.961+0000 7f34379ed740 1 -- 172.21.15.83:0/1833708672 shutdown_connections 2024-04-24T09:04:08.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.961+0000 7f34379ed740 1 --2- 172.21.15.83:0/1833708672 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x563721062370 0x563721064760 crc :-1 s=READY pgs=327 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:08.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.961+0000 7f34379ed740 1 --2- 172.21.15.83:0/1833708672 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563721052080 0x56372104f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:08.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.961+0000 7f34379ed740 1 --2- 172.21.15.83:0/1833708672 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5637210479b0 0x56372104f460 unknown :-1 s=CLOSED pgs=1984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:08.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.961+0000 7f34379ed740 1 --2- 172.21.15.83:0/1833708672 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563721047090 0x56372104ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:08.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.961+0000 7f34379ed740 1 -- 172.21.15.83:0/1833708672 >> 172.21.15.83:0/1833708672 conn(0x5637210433b0 msgr2=0x5637210490b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:08.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.961+0000 7f340f400640 0 client.20112 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:04:08.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.961+0000 7f34379ed740 1 -- 172.21.15.83:0/1833708672 shutdown_connections 2024-04-24T09:04:08.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.961+0000 7f34379ed740 1 -- 172.21.15.83:0/1833708672 wait complete. 2024-04-24T09:04:08.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.961+0000 7f34379ed740 20 client.20112 trim_cache size 0 max 16384 2024-04-24T09:04:08.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.983+0000 7f34379ed740 1 Processor -- start 2024-04-24T09:04:08.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.983+0000 7f34379ed740 1 -- start start 2024-04-24T09:04:08.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.983+0000 7f34379ed740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563721047090 0x56372105dde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:08.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.983+0000 7f34379ed740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5637210479b0 0x563721049760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:08.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.984+0000 7f34379ed740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563721052080 0x563721042390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:08.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.984+0000 7f34379ed740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f34301b5bb0 con 0x563721052080 2024-04-24T09:04:08.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.984+0000 7f34379ed740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f34301b7810 con 0x5637210479b0 2024-04-24T09:04:08.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.984+0000 7f34379ed740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f34301b6280 con 0x563721047090 2024-04-24T09:04:08.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.984+0000 7f342e800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563721052080 0x563721042390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:08.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.984+0000 7f342e800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563721052080 0x563721042390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45828/0 (socket says 172.21.15.83:45828) 2024-04-24T09:04:08.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.984+0000 7f342e800640 1 -- 172.21.15.83:0/874403515 learned_addr learned my addr 172.21.15.83:0/874403515 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:04:08.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.984+0000 7f342de00640 1 --2- 172.21.15.83:0/874403515 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563721047090 0x56372105dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:08.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.984+0000 7f342d400640 1 --2- 172.21.15.83:0/874403515 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5637210479b0 0x563721049760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:08.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.984+0000 7f342e800640 1 -- 172.21.15.83:0/874403515 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563721047090 msgr2=0x56372105dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:08.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.984+0000 7f342e800640 1 --2- 172.21.15.83:0/874403515 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563721047090 0x56372105dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:08.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.984+0000 7f342e800640 1 -- 172.21.15.83:0/874403515 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5637210479b0 msgr2=0x563721049760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:08.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.984+0000 7f342e800640 1 --2- 172.21.15.83:0/874403515 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5637210479b0 0x563721049760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:08.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.984+0000 7f342e800640 1 -- 172.21.15.83:0/874403515 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56372103ed10 con 0x563721052080 2024-04-24T09:04:08.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.985+0000 7f342e800640 1 --2- 172.21.15.83:0/874403515 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563721052080 0x563721042390 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7f342800ab20 tx=0x7f342809c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af5e4bc5f20afa4f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:08.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.985+0000 7f341a600640 1 -- 172.21.15.83:0/874403515 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34280a1230 con 0x563721052080 2024-04-24T09:04:08.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.985+0000 7f341a600640 1 -- 172.21.15.83:0/874403515 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f34280a5af0 con 0x563721052080 2024-04-24T09:04:08.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.985+0000 7f341a600640 1 -- 172.21.15.83:0/874403515 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34280cea30 con 0x563721052080 2024-04-24T09:04:08.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.986+0000 7f34379ed740 1 -- 172.21.15.83:0/874403515 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563721052080 msgr2=0x563721042390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:08.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.986+0000 7f34379ed740 1 --2- 172.21.15.83:0/874403515 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563721052080 0x563721042390 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7f342800ab20 tx=0x7f342809c910 comp rx=0 tx=0).stop 2024-04-24T09:04:08.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.986+0000 7f34379ed740 1 -- 172.21.15.83:0/874403515 shutdown_connections 2024-04-24T09:04:08.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.986+0000 7f34379ed740 1 --2- 172.21.15.83:0/874403515 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563721052080 0x563721042390 unknown :-1 s=CLOSED pgs=1985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:08.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.986+0000 7f34379ed740 1 --2- 172.21.15.83:0/874403515 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5637210479b0 0x563721049760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:08.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.986+0000 7f34379ed740 1 --2- 172.21.15.83:0/874403515 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563721047090 0x56372105dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:08.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.986+0000 7f34379ed740 1 -- 172.21.15.83:0/874403515 >> 172.21.15.83:0/874403515 conn(0x5637210433b0 msgr2=0x563721041c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:08.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.986+0000 7f34379ed740 1 -- 172.21.15.83:0/874403515 shutdown_connections 2024-04-24T09:04:08.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.986+0000 7f34379ed740 1 -- 172.21.15.83:0/874403515 wait complete. 2024-04-24T09:04:08.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.987+0000 7f34379ed740 1 -- start start 2024-04-24T09:04:08.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.987+0000 7f34379ed740 1 Processor -- start 2024-04-24T09:04:08.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.988+0000 7f34379ed740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563721047090 0x56372104bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:08.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.988+0000 7f34379ed740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5637210479b0 0x56372104c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:08.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.988+0000 7f34379ed740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563721052080 0x56372104c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:08.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.988+0000 7f34379ed740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f34301b70e0 con 0x563721052080 2024-04-24T09:04:08.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.988+0000 7f34379ed740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f34301b69b0 con 0x5637210479b0 2024-04-24T09:04:08.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.988+0000 7f34379ed740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f34301b3bb0 con 0x563721047090 2024-04-24T09:04:08.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.988+0000 7f34379ed740 10 client.0 _ll_register_callbacks cb 0x7f341c009b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:04:08.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.988+0000 7f342e800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563721052080 0x56372104c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:08.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.988+0000 7f342e800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563721052080 0x56372104c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45840/0 (socket says 172.21.15.83:45840) 2024-04-24T09:04:08.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.988+0000 7f342e800640 1 -- 172.21.15.83:0/2306619542 learned_addr learned my addr 172.21.15.83:0/2306619542 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:04:08.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.988+0000 7f342d400640 1 --2- 172.21.15.83:0/2306619542 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5637210479b0 0x56372104c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:08.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.988+0000 7f342de00640 1 --2- 172.21.15.83:0/2306619542 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563721047090 0x56372104bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:08.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.988+0000 7f342e800640 1 -- 172.21.15.83:0/2306619542 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563721047090 msgr2=0x56372104bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:08.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.988+0000 7f342e800640 1 --2- 172.21.15.83:0/2306619542 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563721047090 0x56372104bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:08.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.988+0000 7f342e800640 1 -- 172.21.15.83:0/2306619542 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5637210479b0 msgr2=0x56372104c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:08.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.988+0000 7f342e800640 1 --2- 172.21.15.83:0/2306619542 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5637210479b0 0x56372104c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:08.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.988+0000 7f342e800640 1 -- 172.21.15.83:0/2306619542 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637210c77c0 con 0x563721052080 2024-04-24T09:04:08.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.989+0000 7f342e800640 1 --2- 172.21.15.83:0/2306619542 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563721052080 0x56372104c7f0 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7f3428005fd0 tx=0x7f34280d1230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ef873bbb70b1a2d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:08.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.989+0000 7f340f400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:04:08.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.989+0000 7f340f400640 1 -- 172.21.15.83:0/2306619542 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34280b6270 con 0x563721052080 2024-04-24T09:04:08.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.989+0000 7f340f400640 1 -- 172.21.15.83:0/2306619542 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7f34280994f0 con 0x563721052080 2024-04-24T09:04:08.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.989+0000 7f34379ed740 1 -- 172.21.15.83:0/2306619542 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34280a5af0 con 0x563721052080 2024-04-24T09:04:08.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.989+0000 7f340f400640 1 -- 172.21.15.83:0/2306619542 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f3428086120 con 0x563721052080 2024-04-24T09:04:08.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.989+0000 7f34379ed740 10 client.20118 fetch_fsmap learned FSMap version 41 2024-04-24T09:04:08.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.989+0000 7f34379ed740 1 -- 172.21.15.83:0/2306619542 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5637210c8ae0 con 0x563721052080 2024-04-24T09:04:08.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.990+0000 7f340f400640 1 -- 172.21.15.83:0/2306619542 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f34280a5af0 con 0x563721052080 2024-04-24T09:04:08.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.990+0000 7f340f400640 1 -- 172.21.15.83:0/2306619542 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7f3428083030 con 0x563721052080 2024-04-24T09:04:08.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.990+0000 7f34379ed740 10 client.20118 fetch_fsmap finished waiting for FSMap version 41 2024-04-24T09:04:08.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.990+0000 7f34379ed740 10 client.20118 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:04:08.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.990+0000 7f34379ed740 20 client.20118 populate_metadata read hostname 'smithi083' 2024-04-24T09:04:08.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.990+0000 7f34379ed740 1 --2- 172.21.15.83:0/2306619542 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5637210c9170 0x5637210cb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:08.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.990+0000 7f34379ed740 4 client.20118 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T09:04:08.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.990+0000 7f34379ed740 1 -- 172.21.15.83:0/2306619542 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x563720fb4700 con 0x5637210c9170 2024-04-24T09:04:08.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.991+0000 7f342de00640 1 --2- 172.21.15.83:0/2306619542 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5637210c9170 0x5637210cb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:08.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.992+0000 7f342de00640 1 --2- 172.21.15.83:0/2306619542 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5637210c9170 0x5637210cb590 crc :-1 s=READY pgs=328 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4c8bc41fa5ffe4d server_cookie=d6cc9f40cca8bf4 in_seq=0 out_seq=0 2024-04-24T09:04:08.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.992+0000 7f340f400640 10 client.20118 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:04:08.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.994+0000 7f340f400640 1 -- 172.21.15.83:0/2306619542 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x563720fb4700 con 0x5637210c9170 2024-04-24T09:04:08.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.994+0000 7f340f400640 10 client.20118 handle_command_reply: tid=0 2024-04-24T09:04:08.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.994+0000 7f34379ed740 1 client.20118 shutdown 2024-04-24T09:04:08.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.995+0000 7f34379ed740 1 -- 172.21.15.83:0/2306619542 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563721052080 msgr2=0x56372104c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:08.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.995+0000 7f34379ed740 1 --2- 172.21.15.83:0/2306619542 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563721052080 0x56372104c7f0 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7f3428005fd0 tx=0x7f34280d1230 comp rx=0 tx=0).stop 2024-04-24T09:04:08.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.995+0000 7f34379ed740 1 -- 172.21.15.83:0/2306619542 shutdown_connections 2024-04-24T09:04:08.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.995+0000 7f34379ed740 1 --2- 172.21.15.83:0/2306619542 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5637210c9170 0x5637210cb590 crc :-1 s=READY pgs=328 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:08.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.995+0000 7f34379ed740 1 --2- 172.21.15.83:0/2306619542 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563721052080 0x56372104c7f0 unknown :-1 s=CLOSED pgs=1986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:08.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.995+0000 7f34379ed740 1 --2- 172.21.15.83:0/2306619542 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5637210479b0 0x56372104c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:08.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.995+0000 7f34379ed740 1 --2- 172.21.15.83:0/2306619542 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563721047090 0x56372104bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:08.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.995+0000 7f34379ed740 1 -- 172.21.15.83:0/2306619542 >> 172.21.15.83:0/2306619542 conn(0x5637210433b0 msgr2=0x56372104f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:08.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.995+0000 7f340f400640 0 client.20118 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:04:08.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.995+0000 7f34379ed740 1 -- 172.21.15.83:0/2306619542 shutdown_connections 2024-04-24T09:04:08.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.995+0000 7f34379ed740 1 -- 172.21.15.83:0/2306619542 wait complete. 2024-04-24T09:04:08.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.995+0000 7f34379ed740 20 client.20118 trim_cache size 0 max 16384 2024-04-24T09:04:08.996 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T09:04:08.996 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-04-24T09:04:08.996 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "992d9129-9b62-446e-ad64-b799dcec65ba", 2024-04-24T09:04:08.996 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-04-24T09:04:08.997 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T09:04:08.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.996+0000 7f3434c00640 1 -- 172.21.15.83:0/1967064414 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f34140a3da0 msgr2=0x7f34140a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:08.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.996+0000 7f3434c00640 1 --2- 172.21.15.83:0/1967064414 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f34140a3da0 0x7f34140a6260 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f3420005ab0 tx=0x7f34200058b0 comp rx=0 tx=0).stop 2024-04-24T09:04:08.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.996+0000 7f3434c00640 1 -- 172.21.15.83:0/1967064414 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3430112ee0 msgr2=0x7f34301a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:08.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.996+0000 7f3434c00640 1 --2- 172.21.15.83:0/1967064414 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3430112ee0 0x7f34301a3770 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7f342800ccb0 tx=0x7f342800a620 comp rx=0 tx=0).stop 2024-04-24T09:04:08.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.996+0000 7f3434c00640 1 -- 172.21.15.83:0/1967064414 shutdown_connections 2024-04-24T09:04:08.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.996+0000 7f3434c00640 1 --2- 172.21.15.83:0/1967064414 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f34140a3da0 0x7f34140a6260 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:08.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.996+0000 7f3434c00640 1 --2- 172.21.15.83:0/1967064414 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3430112ee0 0x7f34301a3770 unknown :-1 s=CLOSED pgs=1982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:08.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.996+0000 7f3434c00640 1 --2- 172.21.15.83:0/1967064414 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3430076e50 0x7f343019f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:08.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.996+0000 7f3434c00640 1 --2- 172.21.15.83:0/1967064414 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3430076530 0x7f343019eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:08.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.996+0000 7f3434c00640 1 -- 172.21.15.83:0/1967064414 >> 172.21.15.83:0/1967064414 conn(0x7f34300ffd90 msgr2=0x7f3430113b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:08.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.996+0000 7f3434c00640 1 -- 172.21.15.83:0/1967064414 shutdown_connections 2024-04-24T09:04:08.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:08.996+0000 7f3434c00640 1 -- 172.21.15.83:0/1967064414 wait complete. 2024-04-24T09:04:09.015 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T09:04:09.015 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T09:04:09.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.221+0000 7f7a79a00640 1 -- 172.21.15.83:0/3111310435 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a74102c70 msgr2=0x7f7a74106cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:09.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.221+0000 7f7a79a00640 1 --2- 172.21.15.83:0/3111310435 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a74102c70 0x7f7a74106cb0 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 crypto rx=0x7f7a60009900 tx=0x7f7a6002a410 comp rx=0 tx=0).stop 2024-04-24T09:04:09.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.221+0000 7f7a79a00640 1 -- 172.21.15.83:0/3111310435 shutdown_connections 2024-04-24T09:04:09.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.221+0000 7f7a79a00640 1 --2- 172.21.15.83:0/3111310435 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7a74107390 0x7f7a7410ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:09.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.221+0000 7f7a79a00640 1 --2- 172.21.15.83:0/3111310435 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a74102c70 0x7f7a74106cb0 unknown :-1 s=CLOSED pgs=1987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:09.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.221+0000 7f7a79a00640 1 --2- 172.21.15.83:0/3111310435 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a741022c0 0x7f7a741026a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:09.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.221+0000 7f7a79a00640 1 -- 172.21.15.83:0/3111310435 >> 172.21.15.83:0/3111310435 conn(0x7f7a740fdb60 msgr2=0x7f7a740fff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:09.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.221+0000 7f7a79a00640 1 -- 172.21.15.83:0/3111310435 shutdown_connections 2024-04-24T09:04:09.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.222+0000 7f7a79a00640 1 -- 172.21.15.83:0/3111310435 wait complete. 2024-04-24T09:04:09.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.222+0000 7f7a79a00640 1 Processor -- start 2024-04-24T09:04:09.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.223+0000 7f7a79a00640 1 -- start start 2024-04-24T09:04:09.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.223+0000 7f7a79a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7a741022c0 0x7f7a74074300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:09.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.223+0000 7f7a79a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a74107390 0x7f7a74074840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:09.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.223+0000 7f7a79a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a74074d80 0x7f7a741a69e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:09.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.223+0000 7f7a79a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7a741122d0 con 0x7f7a74107390 2024-04-24T09:04:09.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.223+0000 7f7a79a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7a74112150 con 0x7f7a74074d80 2024-04-24T09:04:09.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.223+0000 7f7a79a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7a74112450 con 0x7f7a741022c0 2024-04-24T09:04:09.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.223+0000 7f7a72000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a74107390 0x7f7a74074840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:09.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.223+0000 7f7a72000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a74107390 0x7f7a74074840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45846/0 (socket says 172.21.15.83:45846) 2024-04-24T09:04:09.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.223+0000 7f7a72000640 1 -- 172.21.15.83:0/2524868128 learned_addr learned my addr 172.21.15.83:0/2524868128 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:04:09.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.223+0000 7f7a72a00640 1 --2- 172.21.15.83:0/2524868128 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7a741022c0 0x7f7a74074300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:09.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.224+0000 7f7a73400640 1 --2- 172.21.15.83:0/2524868128 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a74074d80 0x7f7a741a69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:09.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.224+0000 7f7a72000640 1 -- 172.21.15.83:0/2524868128 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7a741022c0 msgr2=0x7f7a74074300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:09.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.224+0000 7f7a72000640 1 --2- 172.21.15.83:0/2524868128 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7a741022c0 0x7f7a74074300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:09.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.224+0000 7f7a72000640 1 -- 172.21.15.83:0/2524868128 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a74074d80 msgr2=0x7f7a741a69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:09.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.224+0000 7f7a72000640 1 --2- 172.21.15.83:0/2524868128 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a74074d80 0x7f7a741a69e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:09.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.224+0000 7f7a72000640 1 -- 172.21.15.83:0/2524868128 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a741a70e0 con 0x7f7a74107390 2024-04-24T09:04:09.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.224+0000 7f7a72a00640 1 --2- 172.21.15.83:0/2524868128 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7a741022c0 0x7f7a74074300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:04:09.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.224+0000 7f7a72000640 1 --2- 172.21.15.83:0/2524868128 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a74107390 0x7f7a74074840 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7f7a60004270 tx=0x7f7a600042a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:09.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.225+0000 7f7a6b400640 1 -- 172.21.15.83:0/2524868128 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a6003f030 con 0x7f7a74107390 2024-04-24T09:04:09.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.225+0000 7f7a6b400640 1 -- 172.21.15.83:0/2524868128 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7a600044c0 con 0x7f7a74107390 2024-04-24T09:04:09.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.225+0000 7f7a79a00640 1 -- 172.21.15.83:0/2524868128 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a741a7370 con 0x7f7a74107390 2024-04-24T09:04:09.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.225+0000 7f7a6b400640 1 -- 172.21.15.83:0/2524868128 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a60035070 con 0x7f7a74107390 2024-04-24T09:04:09.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.225+0000 7f7a79a00640 1 -- 172.21.15.83:0/2524868128 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a741a7860 con 0x7f7a74107390 2024-04-24T09:04:09.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.228+0000 7f7a7c7d1740 1 Processor -- start 2024-04-24T09:04:09.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.228+0000 7f7a7c7d1740 1 -- start start 2024-04-24T09:04:09.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.229+0000 7f7a7c7d1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563e6587e860 0x563e6587ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:09.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.229+0000 7f7a7c7d1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563e6587f180 0x563e65889310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:09.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.229+0000 7f7a7c7d1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563e65889850 0x563e6588bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:09.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.229+0000 7f7a6b400640 1 -- 172.21.15.83:0/2524868128 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7a60042030 con 0x7f7a74107390 2024-04-24T09:04:09.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.229+0000 7f7a7c7d1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x563e657e41f0 con 0x563e65889850 2024-04-24T09:04:09.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.229+0000 7f7a7c7d1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563e6578a350 con 0x563e6587e860 2024-04-24T09:04:09.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.229+0000 7f7a7c7d1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x563e657e01b0 con 0x563e6587f180 2024-04-24T09:04:09.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.229+0000 7f7a72000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563e65889850 0x563e6588bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:09.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.229+0000 7f7a72000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563e65889850 0x563e6588bc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45856/0 (socket says 172.21.15.83:45856) 2024-04-24T09:04:09.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.229+0000 7f7a72000640 1 -- 172.21.15.83:0/1184204030 learned_addr learned my addr 172.21.15.83:0/1184204030 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:04:09.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.229+0000 7f7a73400640 1 --2- 172.21.15.83:0/1184204030 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563e6587e860 0x563e6587ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:09.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.229+0000 7f7a72a00640 1 --2- 172.21.15.83:0/1184204030 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563e6587f180 0x563e65889310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:09.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.230+0000 7f7a72000640 1 -- 172.21.15.83:0/1184204030 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563e6587f180 msgr2=0x563e65889310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:09.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.230+0000 7f7a72000640 1 --2- 172.21.15.83:0/1184204030 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563e6587f180 0x563e65889310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:09.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.230+0000 7f7a72000640 1 -- 172.21.15.83:0/1184204030 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563e6587e860 msgr2=0x563e6587ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:09.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.230+0000 7f7a72000640 1 --2- 172.21.15.83:0/1184204030 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563e6587e860 0x563e6587ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:09.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.230+0000 7f7a72000640 1 -- 172.21.15.83:0/1184204030 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e657def50 con 0x563e65889850 2024-04-24T09:04:09.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.230+0000 7f7a72000640 1 --2- 172.21.15.83:0/1184204030 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563e65889850 0x563e6588bc40 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7f7a600ee400 tx=0x7f7a600f9540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abd7b7520f2cb129 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:09.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.230+0000 7f7a5fe00640 1 -- 172.21.15.83:0/1184204030 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a600f9910 con 0x563e65889850 2024-04-24T09:04:09.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.230+0000 7f7a5fe00640 1 -- 172.21.15.83:0/1184204030 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7a600fe030 con 0x563e65889850 2024-04-24T09:04:09.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.230+0000 7f7a5fe00640 1 -- 172.21.15.83:0/1184204030 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a601023d0 con 0x563e65889850 2024-04-24T09:04:09.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.230+0000 7f7a6b400640 1 --2- 172.21.15.83:0/2524868128 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f7a580a3d50 0x7f7a580a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:09.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.231+0000 7f7a6b400640 1 -- 172.21.15.83:0/2524868128 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7f7a600f1160 con 0x7f7a74107390 2024-04-24T09:04:09.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.231+0000 7f7a73400640 1 --2- 172.21.15.83:0/2524868128 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f7a580a3d50 0x7f7a580a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:09.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.231+0000 7f7a7c7d1740 1 -- 172.21.15.83:0/1184204030 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563e65889850 msgr2=0x563e6588bc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:09.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.231+0000 7f7a7c7d1740 1 --2- 172.21.15.83:0/1184204030 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563e65889850 0x563e6588bc40 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7f7a600ee400 tx=0x7f7a600f9540 comp rx=0 tx=0).stop 2024-04-24T09:04:09.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.231+0000 7f7a7c7d1740 1 -- 172.21.15.83:0/1184204030 shutdown_connections 2024-04-24T09:04:09.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.231+0000 7f7a73400640 1 --2- 172.21.15.83:0/2524868128 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f7a580a3d50 0x7f7a580a6210 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f7a6c0092f0 tx=0x7f7a6c009280 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:09.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.231+0000 7f7a7c7d1740 1 --2- 172.21.15.83:0/1184204030 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563e65889850 0x563e6588bc40 unknown :-1 s=CLOSED pgs=1989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:09.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.231+0000 7f7a7c7d1740 1 --2- 172.21.15.83:0/1184204030 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563e6587f180 0x563e65889310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:09.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.231+0000 7f7a7c7d1740 1 --2- 172.21.15.83:0/1184204030 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563e6587e860 0x563e6587ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:09.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.231+0000 7f7a7c7d1740 1 -- 172.21.15.83:0/1184204030 >> 172.21.15.83:0/1184204030 conn(0x563e6587ab80 msgr2=0x563e6587af60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:09.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.232+0000 7f7a7c7d1740 1 -- 172.21.15.83:0/1184204030 shutdown_connections 2024-04-24T09:04:09.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.232+0000 7f7a7c7d1740 1 -- 172.21.15.83:0/1184204030 wait complete. 2024-04-24T09:04:09.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.232+0000 7f7a7c7d1740 1 -- start start 2024-04-24T09:04:09.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.232+0000 7f7a7c7d1740 1 Processor -- start 2024-04-24T09:04:09.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.233+0000 7f7a7c7d1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563e6587e860 0x563e658867b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:09.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.233+0000 7f7a7c7d1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563e6587f180 0x563e65886cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:09.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.233+0000 7f7a7c7d1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563e65889850 0x563e65887230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:09.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.233+0000 7f7a7c7d1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x563e6588c400 con 0x563e65889850 2024-04-24T09:04:09.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.233+0000 7f7a7c7d1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563e6579d690 con 0x563e6587f180 2024-04-24T09:04:09.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.233+0000 7f7a7c7d1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x563e6588c580 con 0x563e6587e860 2024-04-24T09:04:09.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.233+0000 7f7a7c7d1740 10 client.0 _ll_register_callbacks cb 0x563e657d9650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:04:09.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.233+0000 7f7a73400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563e65889850 0x563e65887230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:09.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.233+0000 7f7a73400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563e65889850 0x563e65887230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45860/0 (socket says 172.21.15.83:45860) 2024-04-24T09:04:09.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.233+0000 7f7a73400640 1 -- 172.21.15.83:0/689660629 learned_addr learned my addr 172.21.15.83:0/689660629 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:04:09.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.233+0000 7f7a72000640 1 --2- 172.21.15.83:0/689660629 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563e6587f180 0x563e65886cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:09.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.233+0000 7f7a72a00640 1 --2- 172.21.15.83:0/689660629 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563e6587e860 0x563e658867b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:09.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.234+0000 7f7a73400640 1 -- 172.21.15.83:0/689660629 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563e6587e860 msgr2=0x563e658867b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:09.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.234+0000 7f7a73400640 1 --2- 172.21.15.83:0/689660629 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563e6587e860 0x563e658867b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:09.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.234+0000 7f7a73400640 1 -- 172.21.15.83:0/689660629 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563e6587f180 msgr2=0x563e65886cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:09.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.234+0000 7f7a73400640 1 --2- 172.21.15.83:0/689660629 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563e6587f180 0x563e65886cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:09.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.234+0000 7f7a73400640 1 -- 172.21.15.83:0/689660629 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e6579c920 con 0x563e65889850 2024-04-24T09:04:09.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.234+0000 7f7a73400640 1 --2- 172.21.15.83:0/689660629 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563e65889850 0x563e65887230 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7f7a6c012870 tx=0x7f7a6c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c39eb987c4d550d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:09.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.234+0000 7f7a5cc00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:04:09.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.234+0000 7f7a5cc00640 1 -- 172.21.15.83:0/689660629 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a6c00ead0 con 0x563e65889850 2024-04-24T09:04:09.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.234+0000 7f7a5cc00640 1 -- 172.21.15.83:0/689660629 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7f7a6c00ec70 con 0x563e65889850 2024-04-24T09:04:09.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.234+0000 7f7a7c7d1740 1 -- 172.21.15.83:0/689660629 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a600fe030 con 0x563e65889850 2024-04-24T09:04:09.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.235+0000 7f7a5cc00640 1 -- 172.21.15.83:0/689660629 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f7a6c030050 con 0x563e65889850 2024-04-24T09:04:09.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.235+0000 7f7a7c7d1740 10 client.20130 fetch_fsmap learned FSMap version 41 2024-04-24T09:04:09.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.235+0000 7f7a7c7d1740 1 -- 172.21.15.83:0/689660629 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x563e655c07d0 con 0x563e65889850 2024-04-24T09:04:09.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.235+0000 7f7a5cc00640 1 -- 172.21.15.83:0/689660629 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7a600fe030 con 0x563e65889850 2024-04-24T09:04:09.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.235+0000 7f7a5cc00640 1 -- 172.21.15.83:0/689660629 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7f7a6c00f040 con 0x563e65889850 2024-04-24T09:04:09.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.235+0000 7f7a7c7d1740 10 client.20130 fetch_fsmap finished waiting for FSMap version 41 2024-04-24T09:04:09.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.235+0000 7f7a7c7d1740 10 client.20130 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:04:09.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.235+0000 7f7a7c7d1740 20 client.20130 populate_metadata read hostname 'smithi083' 2024-04-24T09:04:09.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.235+0000 7f7a7c7d1740 1 --2- 172.21.15.83:0/689660629 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x563e65899970 0x563e6589bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:09.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.235+0000 7f7a7c7d1740 4 client.20130 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:04:09.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.235+0000 7f7a7c7d1740 1 -- 172.21.15.83:0/689660629 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563e657f1c10 con 0x563e65899970 2024-04-24T09:04:09.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.236+0000 7f7a72a00640 1 --2- 172.21.15.83:0/689660629 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x563e65899970 0x563e6589bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:09.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.237+0000 7f7a5cc00640 10 client.20130 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:04:09.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.237+0000 7f7a72a00640 1 --2- 172.21.15.83:0/689660629 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x563e65899970 0x563e6589bdf0 crc :-1 s=READY pgs=329 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=678bee458962230c server_cookie=90d3d15cc15f7bfc in_seq=0 out_seq=0 2024-04-24T09:04:09.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.239+0000 7f7a5cc00640 1 -- 172.21.15.83:0/689660629 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563e657f1c10 con 0x563e65899970 2024-04-24T09:04:09.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.239+0000 7f7a5cc00640 10 client.20130 handle_command_reply: tid=0 2024-04-24T09:04:09.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.239+0000 7f7a7c7d1740 1 client.20130 shutdown 2024-04-24T09:04:09.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.240+0000 7f7a7c7d1740 1 -- 172.21.15.83:0/689660629 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563e65889850 msgr2=0x563e65887230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:09.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.240+0000 7f7a7c7d1740 1 --2- 172.21.15.83:0/689660629 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563e65889850 0x563e65887230 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7f7a6c012870 tx=0x7f7a6c004a40 comp rx=0 tx=0).stop 2024-04-24T09:04:09.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.240+0000 7f7a7c7d1740 1 -- 172.21.15.83:0/689660629 shutdown_connections 2024-04-24T09:04:09.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.240+0000 7f7a7c7d1740 1 --2- 172.21.15.83:0/689660629 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x563e65899970 0x563e6589bdf0 crc :-1 s=READY pgs=329 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:09.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.240+0000 7f7a7c7d1740 1 --2- 172.21.15.83:0/689660629 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563e65889850 0x563e65887230 unknown :-1 s=CLOSED pgs=1990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:09.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.240+0000 7f7a7c7d1740 1 --2- 172.21.15.83:0/689660629 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563e6587f180 0x563e65886cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:09.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.240+0000 7f7a7c7d1740 1 --2- 172.21.15.83:0/689660629 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563e6587e860 0x563e658867b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:09.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.240+0000 7f7a5cc00640 0 client.20130 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:04:09.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.240+0000 7f7a7c7d1740 1 -- 172.21.15.83:0/689660629 >> 172.21.15.83:0/689660629 conn(0x563e6587ab80 msgr2=0x563e65880980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:09.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.240+0000 7f7a7c7d1740 1 -- 172.21.15.83:0/689660629 shutdown_connections 2024-04-24T09:04:09.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.240+0000 7f7a7c7d1740 1 -- 172.21.15.83:0/689660629 wait complete. 2024-04-24T09:04:09.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.240+0000 7f7a7c7d1740 20 client.20130 trim_cache size 0 max 16384 2024-04-24T09:04:09.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.260+0000 7f7a7c7d1740 1 Processor -- start 2024-04-24T09:04:09.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.260+0000 7f7a7c7d1740 1 -- start start 2024-04-24T09:04:09.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.260+0000 7f7a7c7d1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563e6587e860 0x563e658810d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:09.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.260+0000 7f7a7c7d1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563e6587f180 0x563e65895100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:09.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.260+0000 7f7a7c7d1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563e65889850 0x563e65886e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:09.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.260+0000 7f7a7c7d1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7a741b17c0 con 0x563e6587e860 2024-04-24T09:04:09.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.260+0000 7f7a7c7d1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7a741b3420 con 0x563e65889850 2024-04-24T09:04:09.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.261+0000 7f7a7c7d1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7a741b1e90 con 0x563e6587f180 2024-04-24T09:04:09.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.261+0000 7f7a72a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563e6587e860 0x563e658810d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:09.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.261+0000 7f7a72a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563e6587e860 0x563e658810d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45868/0 (socket says 172.21.15.83:45868) 2024-04-24T09:04:09.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.261+0000 7f7a72a00640 1 -- 172.21.15.83:0/3106752671 learned_addr learned my addr 172.21.15.83:0/3106752671 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:04:09.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.261+0000 7f7a72a00640 1 -- 172.21.15.83:0/3106752671 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563e6587f180 msgr2=0x563e65895100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:09.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.261+0000 7f7a72a00640 1 --2- 172.21.15.83:0/3106752671 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563e6587f180 0x563e65895100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:09.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.261+0000 7f7a72a00640 1 -- 172.21.15.83:0/3106752671 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563e65889850 msgr2=0x563e65886e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:09.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.261+0000 7f7a73400640 1 --2- 172.21.15.83:0/3106752671 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563e65889850 0x563e65886e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:09.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.261+0000 7f7a72a00640 1 --2- 172.21.15.83:0/3106752671 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563e65889850 0x563e65886e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:09.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.261+0000 7f7a72a00640 1 -- 172.21.15.83:0/3106752671 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e654f63f0 con 0x563e6587e860 2024-04-24T09:04:09.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.261+0000 7f7a72a00640 1 --2- 172.21.15.83:0/3106752671 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563e6587e860 0x563e658810d0 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x563e6589c630 tx=0x7f7a64005f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a979fe846a9963b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:09.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.262+0000 7f7a5f400640 1 -- 172.21.15.83:0/3106752671 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a64013a90 con 0x563e6587e860 2024-04-24T09:04:09.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.262+0000 7f7a5f400640 1 -- 172.21.15.83:0/3106752671 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7a6401a7e0 con 0x563e6587e860 2024-04-24T09:04:09.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.262+0000 7f7a5f400640 1 -- 172.21.15.83:0/3106752671 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a64011810 con 0x563e6587e860 2024-04-24T09:04:09.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.263+0000 7f7a7c7d1740 1 -- 172.21.15.83:0/3106752671 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563e6587e860 msgr2=0x563e658810d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:09.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.263+0000 7f7a7c7d1740 1 --2- 172.21.15.83:0/3106752671 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563e6587e860 0x563e658810d0 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x563e6589c630 tx=0x7f7a64005f40 comp rx=0 tx=0).stop 2024-04-24T09:04:09.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.263+0000 7f7a7c7d1740 1 -- 172.21.15.83:0/3106752671 shutdown_connections 2024-04-24T09:04:09.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.263+0000 7f7a7c7d1740 1 --2- 172.21.15.83:0/3106752671 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563e65889850 0x563e65886e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:09.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.263+0000 7f7a7c7d1740 1 --2- 172.21.15.83:0/3106752671 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563e6587f180 0x563e65895100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:09.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.263+0000 7f7a7c7d1740 1 --2- 172.21.15.83:0/3106752671 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563e6587e860 0x563e658810d0 unknown :-1 s=CLOSED pgs=1991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:09.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.263+0000 7f7a7c7d1740 1 -- 172.21.15.83:0/3106752671 >> 172.21.15.83:0/3106752671 conn(0x563e6587ab80 msgr2=0x563e65886770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:09.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.263+0000 7f7a7c7d1740 1 -- 172.21.15.83:0/3106752671 shutdown_connections 2024-04-24T09:04:09.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.263+0000 7f7a7c7d1740 1 -- 172.21.15.83:0/3106752671 wait complete. 2024-04-24T09:04:09.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.263+0000 7f7a7c7d1740 1 -- start start 2024-04-24T09:04:09.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.263+0000 7f7a7c7d1740 1 Processor -- start 2024-04-24T09:04:09.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.264+0000 7f7a7c7d1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563e6587e860 0x563e65891af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:09.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.264+0000 7f7a7c7d1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563e6587f180 0x563e6588ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:09.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.264+0000 7f7a7c7d1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563e65889850 0x563e6588d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:09.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.264+0000 7f7a7c7d1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7a741b2cf0 con 0x563e65889850 2024-04-24T09:04:09.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.264+0000 7f7a7c7d1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7a741b25c0 con 0x563e6587e860 2024-04-24T09:04:09.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.264+0000 7f7a7c7d1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7a741af7c0 con 0x563e6587f180 2024-04-24T09:04:09.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.264+0000 7f7a7c7d1740 10 client.0 _ll_register_callbacks cb 0x7f7a6c009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:04:09.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.264+0000 7f7a73400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563e65889850 0x563e6588d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:09.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.264+0000 7f7a73400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563e65889850 0x563e6588d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45876/0 (socket says 172.21.15.83:45876) 2024-04-24T09:04:09.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.264+0000 7f7a73400640 1 -- 172.21.15.83:0/1898313688 learned_addr learned my addr 172.21.15.83:0/1898313688 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:04:09.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.264+0000 7f7a72a00640 1 --2- 172.21.15.83:0/1898313688 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563e6587e860 0x563e65891af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:09.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.264+0000 7f7a72000640 1 --2- 172.21.15.83:0/1898313688 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563e6587f180 0x563e6588ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:09.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.265+0000 7f7a73400640 1 -- 172.21.15.83:0/1898313688 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563e6587f180 msgr2=0x563e6588ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:09.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.265+0000 7f7a73400640 1 --2- 172.21.15.83:0/1898313688 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563e6587f180 0x563e6588ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:09.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.265+0000 7f7a73400640 1 -- 172.21.15.83:0/1898313688 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563e6587e860 msgr2=0x563e65891af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:09.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.265+0000 7f7a73400640 1 --2- 172.21.15.83:0/1898313688 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563e6587e860 0x563e65891af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:09.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.265+0000 7f7a73400640 1 -- 172.21.15.83:0/1898313688 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e654b60f0 con 0x563e65889850 2024-04-24T09:04:09.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.265+0000 7f7a73400640 1 --2- 172.21.15.83:0/1898313688 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563e65889850 0x563e6588d470 secure :-1 s=READY pgs=1992 cs=0 l=1 rev1=1 crypto rx=0x7f7a6c01d9d0 tx=0x7f7a6c01da00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2022cd26990aa4e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:09.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.265+0000 7f7a5cc00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:04:09.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.265+0000 7f7a5cc00640 1 -- 172.21.15.83:0/1898313688 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a6c039030 con 0x563e65889850 2024-04-24T09:04:09.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.265+0000 7f7a5cc00640 1 -- 172.21.15.83:0/1898313688 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7f7a6c01de20 con 0x563e65889850 2024-04-24T09:04:09.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.265+0000 7f7a7c7d1740 1 -- 172.21.15.83:0/1898313688 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a6401a7e0 con 0x563e65889850 2024-04-24T09:04:09.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.266+0000 7f7a5cc00640 1 -- 172.21.15.83:0/1898313688 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f7a6c00e720 con 0x563e65889850 2024-04-24T09:04:09.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.266+0000 7f7a7c7d1740 10 client.20136 fetch_fsmap learned FSMap version 41 2024-04-24T09:04:09.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.266+0000 7f7a7c7d1740 1 -- 172.21.15.83:0/1898313688 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x563e657a9810 con 0x563e65889850 2024-04-24T09:04:09.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.266+0000 7f7a5cc00640 1 -- 172.21.15.83:0/1898313688 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7a6401a7e0 con 0x563e65889850 2024-04-24T09:04:09.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.266+0000 7f7a5cc00640 1 -- 172.21.15.83:0/1898313688 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7f7a6c033040 con 0x563e65889850 2024-04-24T09:04:09.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.266+0000 7f7a7c7d1740 10 client.20136 fetch_fsmap finished waiting for FSMap version 41 2024-04-24T09:04:09.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.266+0000 7f7a7c7d1740 10 client.20136 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:04:09.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.266+0000 7f7a7c7d1740 20 client.20136 populate_metadata read hostname 'smithi083' 2024-04-24T09:04:09.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.266+0000 7f7a7c7d1740 1 --2- 172.21.15.83:0/1898313688 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x563e65908bb0 0x563e6590afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:09.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.266+0000 7f7a7c7d1740 4 client.20136 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T09:04:09.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.266+0000 7f7a7c7d1740 1 -- 172.21.15.83:0/1898313688 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563e657eba80 con 0x563e65908bb0 2024-04-24T09:04:09.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.267+0000 7f7a72a00640 1 --2- 172.21.15.83:0/1898313688 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x563e65908bb0 0x563e6590afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:09.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.268+0000 7f7a72a00640 1 --2- 172.21.15.83:0/1898313688 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x563e65908bb0 0x563e6590afd0 crc :-1 s=READY pgs=330 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f0d52d16b3bd89f server_cookie=78dd9671540b5588 in_seq=0 out_seq=0 2024-04-24T09:04:09.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.268+0000 7f7a5cc00640 10 client.20136 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:04:09.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.270+0000 7f7a5cc00640 1 -- 172.21.15.83:0/1898313688 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x563e657eba80 con 0x563e65908bb0 2024-04-24T09:04:09.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.270+0000 7f7a5cc00640 10 client.20136 handle_command_reply: tid=0 2024-04-24T09:04:09.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.270+0000 7f7a7c7d1740 1 client.20136 shutdown 2024-04-24T09:04:09.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.270+0000 7f7a7c7d1740 1 -- 172.21.15.83:0/1898313688 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563e65889850 msgr2=0x563e6588d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:09.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.270+0000 7f7a7c7d1740 1 --2- 172.21.15.83:0/1898313688 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563e65889850 0x563e6588d470 secure :-1 s=READY pgs=1992 cs=0 l=1 rev1=1 crypto rx=0x7f7a6c01d9d0 tx=0x7f7a6c01da00 comp rx=0 tx=0).stop 2024-04-24T09:04:09.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.270+0000 7f7a7c7d1740 1 -- 172.21.15.83:0/1898313688 shutdown_connections 2024-04-24T09:04:09.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.270+0000 7f7a7c7d1740 1 --2- 172.21.15.83:0/1898313688 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x563e65908bb0 0x563e6590afd0 crc :-1 s=READY pgs=330 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:09.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.270+0000 7f7a7c7d1740 1 --2- 172.21.15.83:0/1898313688 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563e65889850 0x563e6588d470 unknown :-1 s=CLOSED pgs=1992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:09.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.270+0000 7f7a7c7d1740 1 --2- 172.21.15.83:0/1898313688 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563e6587f180 0x563e6588ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:09.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.270+0000 7f7a7c7d1740 1 --2- 172.21.15.83:0/1898313688 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563e6587e860 0x563e65891af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:09.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.270+0000 7f7a5cc00640 0 client.20136 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:04:09.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.270+0000 7f7a7c7d1740 1 -- 172.21.15.83:0/1898313688 >> 172.21.15.83:0/1898313688 conn(0x563e6587ab80 msgr2=0x563e65879c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:09.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.271+0000 7f7a7c7d1740 1 -- 172.21.15.83:0/1898313688 shutdown_connections 2024-04-24T09:04:09.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.271+0000 7f7a7c7d1740 1 -- 172.21.15.83:0/1898313688 wait complete. 2024-04-24T09:04:09.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.271+0000 7f7a7c7d1740 20 client.20136 trim_cache size 0 max 16384 2024-04-24T09:04:09.272 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T09:04:09.272 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T09:04:09.273 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-04-24T09:04:09.273 INFO:teuthology.orchestra.run.smithi083.stdout: "992d9129-9b62-446e-ad64-b799dcec65ba": { 2024-04-24T09:04:09.273 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-04-24T09:04:09.273 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "992d9129-9b62-446e-ad64-b799dcec65ba", 2024-04-24T09:04:09.273 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-04-24T09:04:09.273 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:04:09.273 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:04:09.273 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T09:04:09.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.271+0000 7f7a79a00640 1 -- 172.21.15.83:0/2524868128 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f7a580a3d50 msgr2=0x7f7a580a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:09.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.271+0000 7f7a79a00640 1 --2- 172.21.15.83:0/2524868128 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f7a580a3d50 0x7f7a580a6210 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f7a6c0092f0 tx=0x7f7a6c009280 comp rx=0 tx=0).stop 2024-04-24T09:04:09.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.271+0000 7f7a79a00640 1 -- 172.21.15.83:0/2524868128 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a74107390 msgr2=0x7f7a74074840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:09.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.271+0000 7f7a79a00640 1 --2- 172.21.15.83:0/2524868128 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a74107390 0x7f7a74074840 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7f7a60004270 tx=0x7f7a600042a0 comp rx=0 tx=0).stop 2024-04-24T09:04:09.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.272+0000 7f7a79a00640 1 -- 172.21.15.83:0/2524868128 shutdown_connections 2024-04-24T09:04:09.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.272+0000 7f7a79a00640 1 --2- 172.21.15.83:0/2524868128 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f7a580a3d50 0x7f7a580a6210 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:09.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.272+0000 7f7a79a00640 1 --2- 172.21.15.83:0/2524868128 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a74074d80 0x7f7a741a69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:09.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.272+0000 7f7a79a00640 1 --2- 172.21.15.83:0/2524868128 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a74107390 0x7f7a74074840 unknown :-1 s=CLOSED pgs=1988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:09.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.272+0000 7f7a79a00640 1 --2- 172.21.15.83:0/2524868128 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7a741022c0 0x7f7a74074300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:09.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.272+0000 7f7a79a00640 1 -- 172.21.15.83:0/2524868128 >> 172.21.15.83:0/2524868128 conn(0x7f7a740fdb60 msgr2=0x7f7a740ffee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:09.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.272+0000 7f7a79a00640 1 -- 172.21.15.83:0/2524868128 shutdown_connections 2024-04-24T09:04:09.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:09.272+0000 7f7a79a00640 1 -- 172.21.15.83:0/2524868128 wait complete. 2024-04-24T09:04:09.291 INFO:tasks.cephfs.filesystem:scrub status for tag:992d9129-9b62-446e-ad64-b799dcec65ba - {'path': '/', 'tag': '992d9129-9b62-446e-ad64-b799dcec65ba', 'options': 'recursive,force'} 2024-04-24T09:04:09.624 INFO:tasks.workunit.client.0.smithi083.stdout: 1 72173 0.59 MB/sec execute 523 sec latency 46.401 ms 2024-04-24T09:04:10.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:04:09 smithi138 ceph-mon[31441]: pgmap v395: 129 pgs: 129 active+clean; 69 MiB data, 946 MiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 616 KiB/s wr, 106 op/s 2024-04-24T09:04:10.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:04:09 smithi138 ceph-mon[31441]: scrub queued for path: / 2024-04-24T09:04:10.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:04:09 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:04:10.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:04:09 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:04:10.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:04:09 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:04:10.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:04:09 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:04:10.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:04:09 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:04:10.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:04:09 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:04:10.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:04:09 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:04:10.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:04:09 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:04:10.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:04:09 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:04:10.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:04:09 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:04:10.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:04:09 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:04:10.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:04:09 smithi189 ceph-mon[31791]: pgmap v395: 129 pgs: 129 active+clean; 69 MiB data, 946 MiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 616 KiB/s wr, 106 op/s 2024-04-24T09:04:10.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:04:09 smithi189 ceph-mon[31791]: scrub queued for path: / 2024-04-24T09:04:10.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:04:09 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:04:10.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:04:09 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:04:10.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:04:09 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:04:10.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:04:09 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:04:10.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:04:09 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:04:10.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:04:09 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:04:10.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:04:09 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:04:10.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:04:09 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:04:10.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:04:09 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:04:10.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:04:09 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:04:10.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:04:09 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:04:10.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:04:09 smithi083 ceph-mon[27907]: pgmap v395: 129 pgs: 129 active+clean; 69 MiB data, 946 MiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 616 KiB/s wr, 106 op/s 2024-04-24T09:04:10.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:04:09 smithi083 ceph-mon[27907]: scrub queued for path: / 2024-04-24T09:04:10.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:04:09 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:04:10.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:04:09 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:04:10.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:04:09 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:04:10.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:04:09 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:04:10.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:04:09 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:04:10.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:04:09 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:04:10.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:04:09 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:04:10.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:04:09 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:04:10.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:04:09 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:04:10.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:04:09 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:04:10.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:04:09 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:04:10.624 INFO:tasks.workunit.client.0.smithi083.stdout: 1 72369 0.59 MB/sec execute 524 sec latency 52.049 ms 2024-04-24T09:04:11.053 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:04:10 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:04:11.053 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:04:10 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:04:11.053 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:04:10 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:04:11.053 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:04:10 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:04:11.054 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:04:10 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:04:11.054 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:04:10 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:04:11.054 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:04:10 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:04:11.075 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.075+0000 7f198f200640 1 -- 172.21.15.83:0/2614966444 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f199010c3e0 msgr2=0x7f199010e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:11.075 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.075+0000 7f198f200640 1 --2- 172.21.15.83:0/2614966444 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f199010c3e0 0x7f199010e880 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7f1988009ca0 tx=0x7f198802a950 comp rx=0 tx=0).stop 2024-04-24T09:04:11.075 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.075+0000 7f198f200640 1 -- 172.21.15.83:0/2614966444 shutdown_connections 2024-04-24T09:04:11.075 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.075+0000 7f198f200640 1 --2- 172.21.15.83:0/2614966444 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f199010c3e0 0x7f199010e880 unknown :-1 s=CLOSED pgs=1993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:11.075 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.075+0000 7f198f200640 1 --2- 172.21.15.83:0/2614966444 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f19901038e0 0x7f199010bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:11.076 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.075+0000 7f198f200640 1 --2- 172.21.15.83:0/2614966444 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1990102fc0 0x7f19901033a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:11.076 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.075+0000 7f198f200640 1 -- 172.21.15.83:0/2614966444 >> 172.21.15.83:0/2614966444 conn(0x7f19900fc820 msgr2=0x7f19900fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:11.076 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.075+0000 7f198f200640 1 -- 172.21.15.83:0/2614966444 shutdown_connections 2024-04-24T09:04:11.076 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.075+0000 7f198f200640 1 -- 172.21.15.83:0/2614966444 wait complete. 2024-04-24T09:04:11.076 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.076+0000 7f198f200640 1 Processor -- start 2024-04-24T09:04:11.076 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.077+0000 7f198f200640 1 -- start start 2024-04-24T09:04:11.078 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.077+0000 7f198f200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1990102fc0 0x7f19901a0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:11.078 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.077+0000 7f198f200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f19901038e0 0x7f19901a0950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:11.078 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.077+0000 7f198f200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f199010c3e0 0x7f199019a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:11.078 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.077+0000 7f198f200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f19901120a0 con 0x7f1990102fc0 2024-04-24T09:04:11.078 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.077+0000 7f198f200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1990111f20 con 0x7f199010c3e0 2024-04-24T09:04:11.078 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.077+0000 7f198f200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1990112220 con 0x7f19901038e0 2024-04-24T09:04:11.078 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.077+0000 7f198de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1990102fc0 0x7f19901a0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:11.079 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.077+0000 7f198de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1990102fc0 0x7f19901a0410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45902/0 (socket says 172.21.15.83:45902) 2024-04-24T09:04:11.079 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.078+0000 7f198de00640 1 -- 172.21.15.83:0/2536187524 learned_addr learned my addr 172.21.15.83:0/2536187524 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:04:11.079 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.078+0000 7f198e800640 1 --2- 172.21.15.83:0/2536187524 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f199010c3e0 0x7f199019a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:11.079 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.078+0000 7f1987e00640 1 --2- 172.21.15.83:0/2536187524 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f19901038e0 0x7f19901a0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:11.079 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.078+0000 7f198de00640 1 -- 172.21.15.83:0/2536187524 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f19901038e0 msgr2=0x7f19901a0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:11.079 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.078+0000 7f198de00640 1 --2- 172.21.15.83:0/2536187524 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f19901038e0 0x7f19901a0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:11.079 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.078+0000 7f198de00640 1 -- 172.21.15.83:0/2536187524 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f199010c3e0 msgr2=0x7f199019a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:11.079 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.078+0000 7f198de00640 1 --2- 172.21.15.83:0/2536187524 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f199010c3e0 0x7f199019a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:11.079 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.078+0000 7f198de00640 1 -- 172.21.15.83:0/2536187524 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f199019ad90 con 0x7f1990102fc0 2024-04-24T09:04:11.080 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.078+0000 7f198de00640 1 --2- 172.21.15.83:0/2536187524 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1990102fc0 0x7f19901a0410 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7f198000e9f0 tx=0x7f198000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:11.080 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.079+0000 7f1986a00640 1 -- 172.21.15.83:0/2536187524 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f198000cd80 con 0x7f1990102fc0 2024-04-24T09:04:11.080 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.079+0000 7f1986a00640 1 -- 172.21.15.83:0/2536187524 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f19800108e0 con 0x7f1990102fc0 2024-04-24T09:04:11.080 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.079+0000 7f198f200640 1 -- 172.21.15.83:0/2536187524 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f199019b080 con 0x7f1990102fc0 2024-04-24T09:04:11.080 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.079+0000 7f1986a00640 1 -- 172.21.15.83:0/2536187524 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1980018920 con 0x7f1990102fc0 2024-04-24T09:04:11.080 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.079+0000 7f198f200640 1 -- 172.21.15.83:0/2536187524 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19901a72d0 con 0x7f1990102fc0 2024-04-24T09:04:11.082 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.082+0000 7f1997d90740 1 Processor -- start 2024-04-24T09:04:11.082 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.082+0000 7f1986a00640 1 -- 172.21.15.83:0/2536187524 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1980010a90 con 0x7f1990102fc0 2024-04-24T09:04:11.082 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.083+0000 7f1997d90740 1 -- start start 2024-04-24T09:04:11.084 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.083+0000 7f1997d90740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5571ba39b3c0 0x5571ba39b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:11.084 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.083+0000 7f1997d90740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5571ba39bce0 0x5571ba3a6060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:11.084 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.083+0000 7f1997d90740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5571ba3a65a0 0x5571ba3a8990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:11.084 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.083+0000 7f1997d90740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5571ba301900 con 0x5571ba3a65a0 2024-04-24T09:04:11.084 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.083+0000 7f1997d90740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5571ba2a7f20 con 0x5571ba39b3c0 2024-04-24T09:04:11.084 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.083+0000 7f1997d90740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5571ba2fd8c0 con 0x5571ba39bce0 2024-04-24T09:04:11.084 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.084+0000 7f1987e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5571ba3a65a0 0x5571ba3a8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:11.084 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.084+0000 7f1987e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5571ba3a65a0 0x5571ba3a8990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45912/0 (socket says 172.21.15.83:45912) 2024-04-24T09:04:11.084 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.084+0000 7f1987e00640 1 -- 172.21.15.83:0/2711800320 learned_addr learned my addr 172.21.15.83:0/2711800320 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:04:11.085 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.084+0000 7f198de00640 1 --2- 172.21.15.83:0/2711800320 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5571ba39bce0 0x5571ba3a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:11.085 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.084+0000 7f198e800640 1 --2- 172.21.15.83:0/2711800320 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5571ba39b3c0 0x5571ba39b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:11.085 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.084+0000 7f1987e00640 1 -- 172.21.15.83:0/2711800320 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5571ba39bce0 msgr2=0x5571ba3a6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:11.085 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.084+0000 7f1987e00640 1 --2- 172.21.15.83:0/2711800320 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5571ba39bce0 0x5571ba3a6060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:11.085 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.084+0000 7f1987e00640 1 -- 172.21.15.83:0/2711800320 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5571ba39b3c0 msgr2=0x5571ba39b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:11.085 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.084+0000 7f1987e00640 1 --2- 172.21.15.83:0/2711800320 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5571ba39b3c0 0x5571ba39b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:11.085 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.084+0000 7f1987e00640 1 -- 172.21.15.83:0/2711800320 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571ba2cce10 con 0x5571ba3a65a0 2024-04-24T09:04:11.086 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.085+0000 7f1987e00640 1 --2- 172.21.15.83:0/2711800320 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5571ba3a65a0 0x5571ba3a8990 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7f197800bbc0 tx=0x7f197800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0810e406f9d9d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:11.086 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.085+0000 7f1986a00640 1 --2- 172.21.15.83:0/2536187524 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f19740a3d90 0x7f19740a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:11.087 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.085+0000 7f1986a00640 1 -- 172.21.15.83:0/2536187524 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7f19800d0f00 con 0x7f1990102fc0 2024-04-24T09:04:11.087 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.085+0000 7f197f400640 1 -- 172.21.15.83:0/2711800320 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1978015070 con 0x5571ba3a65a0 2024-04-24T09:04:11.087 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.085+0000 7f197f400640 1 -- 172.21.15.83:0/2711800320 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f197801a910 con 0x5571ba3a65a0 2024-04-24T09:04:11.087 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.085+0000 7f197f400640 1 -- 172.21.15.83:0/2711800320 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1978011900 con 0x5571ba3a65a0 2024-04-24T09:04:11.087 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.085+0000 7f198e800640 1 --2- 172.21.15.83:0/2536187524 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f19740a3d90 0x7f19740a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:11.087 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.086+0000 7f198e800640 1 --2- 172.21.15.83:0/2536187524 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f19740a3d90 0x7f19740a6250 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f1988006010 tx=0x7f1988005d80 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:11.087 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.086+0000 7f1997d90740 1 -- 172.21.15.83:0/2711800320 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5571ba3a65a0 msgr2=0x5571ba3a8990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:11.087 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.086+0000 7f1997d90740 1 --2- 172.21.15.83:0/2711800320 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5571ba3a65a0 0x5571ba3a8990 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7f197800bbc0 tx=0x7f197800f730 comp rx=0 tx=0).stop 2024-04-24T09:04:11.087 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.086+0000 7f1997d90740 1 -- 172.21.15.83:0/2711800320 shutdown_connections 2024-04-24T09:04:11.087 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.086+0000 7f1997d90740 1 --2- 172.21.15.83:0/2711800320 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5571ba3a65a0 0x5571ba3a8990 unknown :-1 s=CLOSED pgs=1995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:11.087 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.086+0000 7f1997d90740 1 --2- 172.21.15.83:0/2711800320 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5571ba39bce0 0x5571ba3a6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:11.087 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.086+0000 7f1997d90740 1 --2- 172.21.15.83:0/2711800320 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5571ba39b3c0 0x5571ba39b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:11.087 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.086+0000 7f1997d90740 1 -- 172.21.15.83:0/2711800320 >> 172.21.15.83:0/2711800320 conn(0x5571ba3978d0 msgr2=0x5571ba397cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:11.087 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.086+0000 7f1997d90740 1 -- 172.21.15.83:0/2711800320 shutdown_connections 2024-04-24T09:04:11.088 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.086+0000 7f1997d90740 1 -- 172.21.15.83:0/2711800320 wait complete. 2024-04-24T09:04:11.088 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.087+0000 7f1997d90740 1 -- start start 2024-04-24T09:04:11.090 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.087+0000 7f1997d90740 1 Processor -- start 2024-04-24T09:04:11.090 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.087+0000 7f1997d90740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5571ba39b3c0 0x5571ba3a3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:11.091 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.087+0000 7f1997d90740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5571ba39bce0 0x5571ba3a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:11.091 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.087+0000 7f1997d90740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5571ba3a65a0 0x5571ba3a3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:11.091 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.088+0000 7f1997d90740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5571ba3a90e0 con 0x5571ba39b3c0 2024-04-24T09:04:11.091 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.088+0000 7f1997d90740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5571ba2bb2c0 con 0x5571ba3a65a0 2024-04-24T09:04:11.091 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.088+0000 7f1997d90740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5571ba3a9260 con 0x5571ba39bce0 2024-04-24T09:04:11.091 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.088+0000 7f1997d90740 10 client.0 _ll_register_callbacks cb 0x5571ba013a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:04:11.091 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.088+0000 7f198de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5571ba39b3c0 0x5571ba3a3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:11.091 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.088+0000 7f198de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5571ba39b3c0 0x5571ba3a3570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45928/0 (socket says 172.21.15.83:45928) 2024-04-24T09:04:11.091 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.088+0000 7f198de00640 1 -- 172.21.15.83:0/2122921329 learned_addr learned my addr 172.21.15.83:0/2122921329 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:04:11.091 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.088+0000 7f198e800640 1 --2- 172.21.15.83:0/2122921329 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5571ba3a65a0 0x5571ba3a3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:11.091 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.088+0000 7f1987e00640 1 --2- 172.21.15.83:0/2122921329 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5571ba39bce0 0x5571ba3a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:11.091 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.088+0000 7f198de00640 1 -- 172.21.15.83:0/2122921329 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5571ba39bce0 msgr2=0x5571ba3a3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:11.092 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.088+0000 7f198de00640 1 --2- 172.21.15.83:0/2122921329 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5571ba39bce0 0x5571ba3a3ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:11.092 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.088+0000 7f198de00640 1 -- 172.21.15.83:0/2122921329 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5571ba3a65a0 msgr2=0x5571ba3a3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:11.092 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.088+0000 7f198de00640 1 --2- 172.21.15.83:0/2122921329 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5571ba3a65a0 0x5571ba3a3ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:11.092 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.088+0000 7f198de00640 1 -- 172.21.15.83:0/2122921329 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571ba2cbfd0 con 0x5571ba39b3c0 2024-04-24T09:04:11.092 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.088+0000 7f1987e00640 1 --2- 172.21.15.83:0/2122921329 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5571ba39bce0 0x5571ba3a3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:04:11.092 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.088+0000 7f198e800640 1 --2- 172.21.15.83:0/2122921329 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5571ba3a65a0 0x5571ba3a3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:04:11.092 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.088+0000 7f198de00640 1 --2- 172.21.15.83:0/2122921329 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5571ba39b3c0 0x5571ba3a3570 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7f1980000c00 tx=0x7f1980009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5be5e5ebb51d9f15 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:11.092 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.089+0000 7f1973e00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:04:11.092 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.089+0000 7f1973e00640 1 -- 172.21.15.83:0/2122921329 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f198009c3e0 con 0x5571ba39b3c0 2024-04-24T09:04:11.092 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.089+0000 7f1973e00640 1 -- 172.21.15.83:0/2122921329 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7f19800a5460 con 0x5571ba39b3c0 2024-04-24T09:04:11.092 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.089+0000 7f1997d90740 1 -- 172.21.15.83:0/2122921329 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f197801a910 con 0x5571ba39b3c0 2024-04-24T09:04:11.092 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.089+0000 7f1973e00640 1 -- 172.21.15.83:0/2122921329 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f1980010de0 con 0x5571ba39b3c0 2024-04-24T09:04:11.092 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.089+0000 7f1997d90740 10 client.20148 fetch_fsmap learned FSMap version 41 2024-04-24T09:04:11.092 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.089+0000 7f1997d90740 1 -- 172.21.15.83:0/2122921329 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5571ba014210 con 0x5571ba39b3c0 2024-04-24T09:04:11.093 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.089+0000 7f1973e00640 1 -- 172.21.15.83:0/2122921329 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f197801a910 con 0x5571ba39b3c0 2024-04-24T09:04:11.093 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.090+0000 7f1973e00640 1 -- 172.21.15.83:0/2122921329 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7f198009bed0 con 0x5571ba39b3c0 2024-04-24T09:04:11.093 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.090+0000 7f1997d90740 10 client.20148 fetch_fsmap finished waiting for FSMap version 41 2024-04-24T09:04:11.093 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.090+0000 7f1997d90740 10 client.20148 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:04:11.093 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.090+0000 7f1997d90740 20 client.20148 populate_metadata read hostname 'smithi083' 2024-04-24T09:04:11.093 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.090+0000 7f1997d90740 1 --2- 172.21.15.83:0/2122921329 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5571ba3b6820 0x5571ba3b8c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:11.093 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.090+0000 7f1997d90740 4 client.20148 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:04:11.093 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.090+0000 7f1997d90740 1 -- 172.21.15.83:0/2122921329 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5571ba30f320 con 0x5571ba3b6820 2024-04-24T09:04:11.093 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.090+0000 7f1987e00640 1 --2- 172.21.15.83:0/2122921329 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5571ba3b6820 0x5571ba3b8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:11.094 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.091+0000 7f1987e00640 1 --2- 172.21.15.83:0/2122921329 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5571ba3b6820 0x5571ba3b8c50 crc :-1 s=READY pgs=331 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f71bc968261c8151 server_cookie=7d2a192bb90e935a in_seq=0 out_seq=0 2024-04-24T09:04:11.095 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.091+0000 7f1973e00640 10 client.20148 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:04:11.095 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.093+0000 7f1973e00640 1 -- 172.21.15.83:0/2122921329 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5571ba30f320 con 0x5571ba3b6820 2024-04-24T09:04:11.095 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.093+0000 7f1973e00640 10 client.20148 handle_command_reply: tid=0 2024-04-24T09:04:11.095 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.093+0000 7f1997d90740 1 client.20148 shutdown 2024-04-24T09:04:11.095 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.094+0000 7f1997d90740 1 -- 172.21.15.83:0/2122921329 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5571ba39b3c0 msgr2=0x5571ba3a3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:11.095 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.094+0000 7f1997d90740 1 --2- 172.21.15.83:0/2122921329 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5571ba39b3c0 0x5571ba3a3570 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7f1980000c00 tx=0x7f1980009310 comp rx=0 tx=0).stop 2024-04-24T09:04:11.095 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.094+0000 7f1997d90740 1 -- 172.21.15.83:0/2122921329 shutdown_connections 2024-04-24T09:04:11.095 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.094+0000 7f1997d90740 1 --2- 172.21.15.83:0/2122921329 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5571ba3b6820 0x5571ba3b8c50 crc :-1 s=READY pgs=331 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:11.095 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.094+0000 7f1997d90740 1 --2- 172.21.15.83:0/2122921329 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5571ba3a65a0 0x5571ba3a3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:11.095 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.094+0000 7f1997d90740 1 --2- 172.21.15.83:0/2122921329 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5571ba39bce0 0x5571ba3a3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:11.095 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.094+0000 7f1997d90740 1 --2- 172.21.15.83:0/2122921329 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5571ba39b3c0 0x5571ba3a3570 unknown :-1 s=CLOSED pgs=1996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:11.095 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.094+0000 7f1997d90740 1 -- 172.21.15.83:0/2122921329 >> 172.21.15.83:0/2122921329 conn(0x5571ba3978d0 msgr2=0x5571ba396a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:11.095 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.094+0000 7f1973e00640 0 client.20148 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:04:11.096 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.094+0000 7f1997d90740 1 -- 172.21.15.83:0/2122921329 shutdown_connections 2024-04-24T09:04:11.096 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.094+0000 7f1997d90740 1 -- 172.21.15.83:0/2122921329 wait complete. 2024-04-24T09:04:11.096 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.094+0000 7f1997d90740 20 client.20148 trim_cache size 0 max 16384 2024-04-24T09:04:11.114 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.114+0000 7f1997d90740 1 Processor -- start 2024-04-24T09:04:11.114 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.114+0000 7f1997d90740 1 -- start start 2024-04-24T09:04:11.114 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.114+0000 7f1997d90740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5571ba3978d0 0x5571ba3b20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:11.114 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.114+0000 7f1997d90740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5571ba39b3c0 0x5571ba39dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:11.114 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.114+0000 7f1997d90740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5571ba39bce0 0x5571ba3be560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:11.114 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.114+0000 7f1997d90740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f19901b16a0 con 0x5571ba39bce0 2024-04-24T09:04:11.114 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.114+0000 7f1997d90740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f19901b3300 con 0x5571ba39b3c0 2024-04-24T09:04:11.114 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.114+0000 7f1997d90740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f19901b1d70 con 0x5571ba3978d0 2024-04-24T09:04:11.115 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.115+0000 7f198e800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5571ba39bce0 0x5571ba3be560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:11.115 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.115+0000 7f198de00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5571ba3978d0 0x5571ba3b20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:11.116 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.115+0000 7f198e800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5571ba39bce0 0x5571ba3be560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45930/0 (socket says 172.21.15.83:45930) 2024-04-24T09:04:11.116 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.115+0000 7f198e800640 1 -- 172.21.15.83:0/4017125905 learned_addr learned my addr 172.21.15.83:0/4017125905 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:04:11.116 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.115+0000 7f1987e00640 1 --2- 172.21.15.83:0/4017125905 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5571ba39b3c0 0x5571ba39dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:11.116 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.115+0000 7f198e800640 1 -- 172.21.15.83:0/4017125905 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5571ba3978d0 msgr2=0x5571ba3b20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:11.116 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.115+0000 7f198e800640 1 --2- 172.21.15.83:0/4017125905 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5571ba3978d0 0x5571ba3b20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:11.116 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.115+0000 7f198e800640 1 -- 172.21.15.83:0/4017125905 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5571ba39b3c0 msgr2=0x5571ba39dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:11.116 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.115+0000 7f198e800640 1 --2- 172.21.15.83:0/4017125905 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5571ba39b3c0 0x5571ba39dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:11.116 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.115+0000 7f198e800640 1 -- 172.21.15.83:0/4017125905 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571ba3a2520 con 0x5571ba39bce0 2024-04-24T09:04:11.116 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.116+0000 7f198e800640 1 --2- 172.21.15.83:0/4017125905 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5571ba39bce0 0x5571ba3be560 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7f1988035000 tx=0x7f1988045000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dde984db24802e11 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:11.117 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.116+0000 7f197ea00640 1 -- 172.21.15.83:0/4017125905 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1988056070 con 0x5571ba39bce0 2024-04-24T09:04:11.117 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.116+0000 7f197ea00640 1 -- 172.21.15.83:0/4017125905 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1988003e40 con 0x5571ba39bce0 2024-04-24T09:04:11.117 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.116+0000 7f197ea00640 1 -- 172.21.15.83:0/4017125905 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1988034040 con 0x5571ba39bce0 2024-04-24T09:04:11.117 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.117+0000 7f1997d90740 1 -- 172.21.15.83:0/4017125905 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5571ba39bce0 msgr2=0x5571ba3be560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:11.117 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.117+0000 7f1997d90740 1 --2- 172.21.15.83:0/4017125905 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5571ba39bce0 0x5571ba3be560 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7f1988035000 tx=0x7f1988045000 comp rx=0 tx=0).stop 2024-04-24T09:04:11.118 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.117+0000 7f1997d90740 1 -- 172.21.15.83:0/4017125905 shutdown_connections 2024-04-24T09:04:11.118 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.117+0000 7f1997d90740 1 --2- 172.21.15.83:0/4017125905 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5571ba39bce0 0x5571ba3be560 unknown :-1 s=CLOSED pgs=1997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:11.118 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.117+0000 7f1997d90740 1 --2- 172.21.15.83:0/4017125905 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5571ba39b3c0 0x5571ba39dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:11.118 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.117+0000 7f1997d90740 1 --2- 172.21.15.83:0/4017125905 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5571ba3978d0 0x5571ba3b20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:11.118 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.117+0000 7f1997d90740 1 -- 172.21.15.83:0/4017125905 >> 172.21.15.83:0/4017125905 conn(0x5571ba392e60 msgr2=0x5571ba397cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:11.118 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.117+0000 7f1997d90740 1 -- 172.21.15.83:0/4017125905 shutdown_connections 2024-04-24T09:04:11.118 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.117+0000 7f1997d90740 1 -- 172.21.15.83:0/4017125905 wait complete. 2024-04-24T09:04:11.118 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.118+0000 7f1997d90740 1 -- start start 2024-04-24T09:04:11.120 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.118+0000 7f1997d90740 1 Processor -- start 2024-04-24T09:04:11.120 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.118+0000 7f1997d90740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5571ba3978d0 0x5571ba4150d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:11.120 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.118+0000 7f1997d90740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5571ba39b3c0 0x5571ba415610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:11.120 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.118+0000 7f1997d90740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5571ba39bce0 0x5571ba3a97d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:11.120 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.118+0000 7f1997d90740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f19901b2bd0 con 0x5571ba3978d0 2024-04-24T09:04:11.120 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.118+0000 7f1997d90740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f19901b24a0 con 0x5571ba39bce0 2024-04-24T09:04:11.120 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.118+0000 7f1997d90740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f19901af6a0 con 0x5571ba39b3c0 2024-04-24T09:04:11.120 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.118+0000 7f1997d90740 10 client.0 _ll_register_callbacks cb 0x7f19800d4740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:04:11.120 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.119+0000 7f198de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5571ba3978d0 0x5571ba4150d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:11.120 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.119+0000 7f198de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5571ba3978d0 0x5571ba4150d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45944/0 (socket says 172.21.15.83:45944) 2024-04-24T09:04:11.120 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.119+0000 7f198de00640 1 -- 172.21.15.83:0/3181652166 learned_addr learned my addr 172.21.15.83:0/3181652166 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:04:11.121 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.119+0000 7f1987e00640 1 --2- 172.21.15.83:0/3181652166 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5571ba39b3c0 0x5571ba415610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:11.121 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.119+0000 7f198e800640 1 --2- 172.21.15.83:0/3181652166 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5571ba39bce0 0x5571ba3a97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:11.121 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.119+0000 7f198de00640 1 -- 172.21.15.83:0/3181652166 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5571ba39b3c0 msgr2=0x5571ba415610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:11.121 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.119+0000 7f198de00640 1 --2- 172.21.15.83:0/3181652166 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5571ba39b3c0 0x5571ba415610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:11.121 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.119+0000 7f198de00640 1 -- 172.21.15.83:0/3181652166 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5571ba39bce0 msgr2=0x5571ba3a97d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:11.121 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.119+0000 7f198de00640 1 --2- 172.21.15.83:0/3181652166 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5571ba39bce0 0x5571ba3a97d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:11.121 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.119+0000 7f198de00640 1 -- 172.21.15.83:0/3181652166 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571ba0e0620 con 0x5571ba3978d0 2024-04-24T09:04:11.121 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.120+0000 7f198de00640 1 --2- 172.21.15.83:0/3181652166 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5571ba3978d0 0x5571ba4150d0 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7f19800a51c0 tx=0x7f1980099300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b810668d99779f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:11.121 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.120+0000 7f1973e00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:04:11.121 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.120+0000 7f1973e00640 1 -- 172.21.15.83:0/3181652166 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19800a5460 con 0x5571ba3978d0 2024-04-24T09:04:11.121 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.120+0000 7f1973e00640 1 -- 172.21.15.83:0/3181652166 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7f1980010430 con 0x5571ba3978d0 2024-04-24T09:04:11.121 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.120+0000 7f1997d90740 1 -- 172.21.15.83:0/3181652166 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5571ba309190 con 0x5571ba3978d0 2024-04-24T09:04:11.121 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.120+0000 7f1973e00640 1 -- 172.21.15.83:0/3181652166 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f198009ca80 con 0x5571ba3978d0 2024-04-24T09:04:11.121 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.120+0000 7f1997d90740 10 client.20154 fetch_fsmap learned FSMap version 41 2024-04-24T09:04:11.122 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.120+0000 7f1997d90740 1 -- 172.21.15.83:0/3181652166 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5571ba41c6c0 con 0x5571ba3978d0 2024-04-24T09:04:11.122 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.120+0000 7f1973e00640 1 -- 172.21.15.83:0/3181652166 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x5571ba309190 con 0x5571ba3978d0 2024-04-24T09:04:11.123 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.121+0000 7f1973e00640 1 -- 172.21.15.83:0/3181652166 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7f198009b940 con 0x5571ba3978d0 2024-04-24T09:04:11.123 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.121+0000 7f1997d90740 10 client.20154 fetch_fsmap finished waiting for FSMap version 41 2024-04-24T09:04:11.123 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.121+0000 7f1997d90740 10 client.20154 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:04:11.123 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.121+0000 7f1997d90740 20 client.20154 populate_metadata read hostname 'smithi083' 2024-04-24T09:04:11.123 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.121+0000 7f1997d90740 1 --2- 172.21.15.83:0/3181652166 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5571ba3a65a0 0x5571ba41ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:11.123 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.121+0000 7f1997d90740 4 client.20154 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T09:04:11.123 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.121+0000 7f1997d90740 1 -- 172.21.15.83:0/3181652166 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5571ba2854b0 con 0x5571ba3a65a0 2024-04-24T09:04:11.123 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.122+0000 7f1987e00640 1 --2- 172.21.15.83:0/3181652166 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5571ba3a65a0 0x5571ba41ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:11.123 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.122+0000 7f1987e00640 1 --2- 172.21.15.83:0/3181652166 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5571ba3a65a0 0x5571ba41ee80 crc :-1 s=READY pgs=332 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=adec59a638349b93 server_cookie=b06b447d6310c6ee in_seq=0 out_seq=0 2024-04-24T09:04:11.123 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.122+0000 7f1973e00640 10 client.20154 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:04:11.159 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.159+0000 7f1973e00640 1 -- 172.21.15.83:0/3181652166 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5571ba2854b0 con 0x5571ba3a65a0 2024-04-24T09:04:11.159 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.159+0000 7f1973e00640 10 client.20154 handle_command_reply: tid=0 2024-04-24T09:04:11.159 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.159+0000 7f1997d90740 1 client.20154 shutdown 2024-04-24T09:04:11.159 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.159+0000 7f1997d90740 1 -- 172.21.15.83:0/3181652166 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5571ba3978d0 msgr2=0x5571ba4150d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:11.159 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.159+0000 7f1997d90740 1 --2- 172.21.15.83:0/3181652166 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5571ba3978d0 0x5571ba4150d0 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7f19800a51c0 tx=0x7f1980099300 comp rx=0 tx=0).stop 2024-04-24T09:04:11.160 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.159+0000 7f1997d90740 1 -- 172.21.15.83:0/3181652166 shutdown_connections 2024-04-24T09:04:11.160 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.159+0000 7f1997d90740 1 --2- 172.21.15.83:0/3181652166 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5571ba3a65a0 0x5571ba41ee80 crc :-1 s=READY pgs=332 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:11.160 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.159+0000 7f1997d90740 1 --2- 172.21.15.83:0/3181652166 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5571ba39bce0 0x5571ba3a97d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:11.160 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.159+0000 7f1997d90740 1 --2- 172.21.15.83:0/3181652166 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5571ba39b3c0 0x5571ba415610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:11.160 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.159+0000 7f1997d90740 1 --2- 172.21.15.83:0/3181652166 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5571ba3978d0 0x5571ba4150d0 unknown :-1 s=CLOSED pgs=1998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:11.160 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.159+0000 7f1973e00640 0 client.20154 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:04:11.160 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.159+0000 7f1997d90740 1 -- 172.21.15.83:0/3181652166 >> 172.21.15.83:0/3181652166 conn(0x5571ba392e60 msgr2=0x5571ba396200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:11.160 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.159+0000 7f1997d90740 1 -- 172.21.15.83:0/3181652166 shutdown_connections 2024-04-24T09:04:11.160 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.160+0000 7f1997d90740 1 -- 172.21.15.83:0/3181652166 wait complete. 2024-04-24T09:04:11.160 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.160+0000 7f1997d90740 20 client.20154 trim_cache size 0 max 16384 2024-04-24T09:04:11.161 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T09:04:11.161 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T09:04:11.161 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T09:04:11.161 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T09:04:11.161 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.160+0000 7f198f200640 1 -- 172.21.15.83:0/2536187524 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f19740a3d90 msgr2=0x7f19740a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:11.162 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.160+0000 7f198f200640 1 --2- 172.21.15.83:0/2536187524 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f19740a3d90 0x7f19740a6250 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f1988006010 tx=0x7f1988005d80 comp rx=0 tx=0).stop 2024-04-24T09:04:11.162 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.160+0000 7f198f200640 1 -- 172.21.15.83:0/2536187524 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1990102fc0 msgr2=0x7f19901a0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:11.162 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.160+0000 7f198f200640 1 --2- 172.21.15.83:0/2536187524 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1990102fc0 0x7f19901a0410 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7f198000e9f0 tx=0x7f198000eeb0 comp rx=0 tx=0).stop 2024-04-24T09:04:11.162 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.161+0000 7f198f200640 1 -- 172.21.15.83:0/2536187524 shutdown_connections 2024-04-24T09:04:11.162 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.161+0000 7f198f200640 1 --2- 172.21.15.83:0/2536187524 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f19740a3d90 0x7f19740a6250 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:11.162 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.161+0000 7f198f200640 1 --2- 172.21.15.83:0/2536187524 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f199010c3e0 0x7f199019a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:11.162 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.161+0000 7f198f200640 1 --2- 172.21.15.83:0/2536187524 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f19901038e0 0x7f19901a0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:11.162 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.161+0000 7f198f200640 1 --2- 172.21.15.83:0/2536187524 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1990102fc0 0x7f19901a0410 unknown :-1 s=CLOSED pgs=1994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:11.162 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.161+0000 7f198f200640 1 -- 172.21.15.83:0/2536187524 >> 172.21.15.83:0/2536187524 conn(0x7f19900fc820 msgr2=0x7f1990101c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:11.162 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.161+0000 7f198f200640 1 -- 172.21.15.83:0/2536187524 shutdown_connections 2024-04-24T09:04:11.162 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:11.161+0000 7f198f200640 1 -- 172.21.15.83:0/2536187524 wait complete. 2024-04-24T09:04:11.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:04:10 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:04:11.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:04:10 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:04:11.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:04:10 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:04:11.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:04:10 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:04:11.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:04:10 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:04:11.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:04:10 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:04:11.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:04:10 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:04:11.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:04:10 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:04:11.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:04:10 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:04:11.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:04:10 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:04:11.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:04:10 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:04:11.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:04:10 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:04:11.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:04:10 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:04:11.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:04:10 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:04:11.624 INFO:tasks.workunit.client.0.smithi083.stdout: 1 72505 0.59 MB/sec execute 525 sec latency 48.266 ms 2024-04-24T09:04:12.087 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:04:11 smithi138 ceph-mon[31441]: pgmap v396: 129 pgs: 129 active+clean; 69 MiB data, 946 MiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 576 KiB/s wr, 99 op/s 2024-04-24T09:04:12.087 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:04:11 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:04:12.087 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:04:11 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:04:12.087 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:04:11 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:04:12.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:04:11 smithi189 ceph-mon[31791]: pgmap v396: 129 pgs: 129 active+clean; 69 MiB data, 946 MiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 576 KiB/s wr, 99 op/s 2024-04-24T09:04:12.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:04:11 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:04:12.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:04:11 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:04:12.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:04:11 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:04:12.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:04:11 smithi083 ceph-mon[27907]: pgmap v396: 129 pgs: 129 active+clean; 69 MiB data, 946 MiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 576 KiB/s wr, 99 op/s 2024-04-24T09:04:12.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:04:11 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:04:12.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:04:11 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:04:12.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:04:11 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:04:12.624 INFO:tasks.workunit.client.0.smithi083.stdout: 1 72622 0.59 MB/sec execute 526 sec latency 56.214 ms 2024-04-24T09:04:13.624 INFO:tasks.workunit.client.0.smithi083.stdout: 1 72773 0.59 MB/sec execute 527 sec latency 68.548 ms 2024-04-24T09:04:14.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:04:13 smithi138 ceph-mon[31441]: pgmap v397: 129 pgs: 129 active+clean; 67 MiB data, 965 MiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 428 KiB/s wr, 79 op/s 2024-04-24T09:04:14.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:04:13 smithi189 ceph-mon[31791]: pgmap v397: 129 pgs: 129 active+clean; 67 MiB data, 965 MiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 428 KiB/s wr, 79 op/s 2024-04-24T09:04:14.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:04:13 smithi083 ceph-mon[27907]: pgmap v397: 129 pgs: 129 active+clean; 67 MiB data, 965 MiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 428 KiB/s wr, 79 op/s 2024-04-24T09:04:14.625 INFO:tasks.workunit.client.0.smithi083.stdout: 1 72873 0.59 MB/sec execute 528 sec latency 52.667 ms 2024-04-24T09:04:15.625 INFO:tasks.workunit.client.0.smithi083.stdout: 1 73027 0.59 MB/sec execute 529 sec latency 64.977 ms 2024-04-24T09:04:16.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:04:15 smithi138 ceph-mon[31441]: pgmap v398: 129 pgs: 129 active+clean; 67 MiB data, 965 MiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 422 KiB/s wr, 78 op/s 2024-04-24T09:04:16.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:04:15 smithi189 ceph-mon[31791]: pgmap v398: 129 pgs: 129 active+clean; 67 MiB data, 965 MiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 422 KiB/s wr, 78 op/s 2024-04-24T09:04:16.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:04:15 smithi083 ceph-mon[27907]: pgmap v398: 129 pgs: 129 active+clean; 67 MiB data, 965 MiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 422 KiB/s wr, 78 op/s 2024-04-24T09:04:16.625 INFO:tasks.workunit.client.0.smithi083.stdout: 1 73173 0.59 MB/sec execute 530 sec latency 67.191 ms 2024-04-24T09:04:17.625 INFO:tasks.workunit.client.0.smithi083.stdout: 1 73319 0.59 MB/sec execute 531 sec latency 56.557 ms 2024-04-24T09:04:18.106 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:04:17 smithi083 ceph-mon[27907]: pgmap v399: 129 pgs: 129 active+clean; 67 MiB data, 965 MiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 357 KiB/s wr, 66 op/s 2024-04-24T09:04:18.106 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:04:17 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:04:18.106 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:04:17 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:04:18.106 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:04:17 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:04:18.106 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:04:17 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:04:18.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:04:17 smithi138 ceph-mon[31441]: pgmap v399: 129 pgs: 129 active+clean; 67 MiB data, 965 MiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 357 KiB/s wr, 66 op/s 2024-04-24T09:04:18.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:04:17 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:04:18.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:04:17 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:04:18.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:04:17 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:04:18.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:04:17 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:04:18.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:04:17 smithi189 ceph-mon[31791]: pgmap v399: 129 pgs: 129 active+clean; 67 MiB data, 965 MiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 357 KiB/s wr, 66 op/s 2024-04-24T09:04:18.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:04:17 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:04:18.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:04:17 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:04:18.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:04:17 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:04:18.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:04:17 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:04:18.625 INFO:tasks.workunit.client.0.smithi083.stdout: 1 73394 0.59 MB/sec execute 532 sec latency 58.943 ms 2024-04-24T09:04:19.625 INFO:tasks.workunit.client.0.smithi083.stdout: 1 73487 0.59 MB/sec execute 533 sec latency 48.836 ms 2024-04-24T09:04:20.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:04:19 smithi138 ceph-mon[31441]: pgmap v400: 129 pgs: 129 active+clean; 67 MiB data, 966 MiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 417 KiB/s wr, 83 op/s 2024-04-24T09:04:20.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:04:19 smithi189 ceph-mon[31791]: pgmap v400: 129 pgs: 129 active+clean; 67 MiB data, 966 MiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 417 KiB/s wr, 83 op/s 2024-04-24T09:04:20.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:04:19 smithi083 ceph-mon[27907]: pgmap v400: 129 pgs: 129 active+clean; 67 MiB data, 966 MiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 417 KiB/s wr, 83 op/s 2024-04-24T09:04:20.625 INFO:tasks.workunit.client.0.smithi083.stdout: 1 73596 0.59 MB/sec execute 534 sec latency 55.490 ms 2024-04-24T09:04:21.626 INFO:tasks.workunit.client.0.smithi083.stdout: 1 73789 0.59 MB/sec execute 535 sec latency 51.798 ms 2024-04-24T09:04:22.188 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:04:21 smithi138 ceph-mon[31441]: pgmap v401: 129 pgs: 129 active+clean; 67 MiB data, 966 MiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 265 KiB/s wr, 54 op/s 2024-04-24T09:04:22.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:04:21 smithi189 ceph-mon[31791]: pgmap v401: 129 pgs: 129 active+clean; 67 MiB data, 966 MiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 265 KiB/s wr, 54 op/s 2024-04-24T09:04:22.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:04:21 smithi083 ceph-mon[27907]: pgmap v401: 129 pgs: 129 active+clean; 67 MiB data, 966 MiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 265 KiB/s wr, 54 op/s 2024-04-24T09:04:22.625 INFO:tasks.workunit.client.0.smithi083.stdout: 1 73974 0.59 MB/sec execute 536 sec latency 52.504 ms 2024-04-24T09:04:23.626 INFO:tasks.workunit.client.0.smithi083.stdout: 1 74197 0.59 MB/sec execute 537 sec latency 51.286 ms 2024-04-24T09:04:24.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:04:23 smithi138 ceph-mon[31441]: pgmap v402: 129 pgs: 129 active+clean; 70 MiB data, 972 MiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 604 KiB/s wr, 75 op/s 2024-04-24T09:04:24.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:04:23 smithi189 ceph-mon[31791]: pgmap v402: 129 pgs: 129 active+clean; 70 MiB data, 972 MiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 604 KiB/s wr, 75 op/s 2024-04-24T09:04:24.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:04:23 smithi083 ceph-mon[27907]: pgmap v402: 129 pgs: 129 active+clean; 70 MiB data, 972 MiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 604 KiB/s wr, 75 op/s 2024-04-24T09:04:24.315 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.316+0000 7fa337600640 1 -- 172.21.15.83:0/3282663325 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa330101730 msgr2=0x7fa33010e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:24.315 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.316+0000 7fa337600640 1 --2- 172.21.15.83:0/3282663325 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa330101730 0x7fa33010e480 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7fa31c009900 tx=0x7fa31c02a410 comp rx=0 tx=0).stop 2024-04-24T09:04:24.316 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.316+0000 7fa337600640 1 -- 172.21.15.83:0/3282663325 shutdown_connections 2024-04-24T09:04:24.316 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.316+0000 7fa337600640 1 --2- 172.21.15.83:0/3282663325 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa33010e9c0 0x7fa330110db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:24.316 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.316+0000 7fa337600640 1 --2- 172.21.15.83:0/3282663325 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa330101730 0x7fa33010e480 unknown :-1 s=CLOSED pgs=1999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:24.316 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.316+0000 7fa337600640 1 --2- 172.21.15.83:0/3282663325 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa330100e10 0x7fa3301011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:24.316 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.316+0000 7fa337600640 1 -- 172.21.15.83:0/3282663325 >> 172.21.15.83:0/3282663325 conn(0x7fa330078080 msgr2=0x7fa330078490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:24.316 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.317+0000 7fa337600640 1 -- 172.21.15.83:0/3282663325 shutdown_connections 2024-04-24T09:04:24.316 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.317+0000 7fa337600640 1 -- 172.21.15.83:0/3282663325 wait complete. 2024-04-24T09:04:24.317 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.318+0000 7fa337600640 1 Processor -- start 2024-04-24T09:04:24.317 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.318+0000 7fa337600640 1 -- start start 2024-04-24T09:04:24.317 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.318+0000 7fa337600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa330100e10 0x7fa3301a0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:24.317 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.318+0000 7fa337600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa330101730 0x7fa3301a0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:24.317 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.318+0000 7fa337600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa33010e9c0 0x7fa33019a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:24.317 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.318+0000 7fa337600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa3301145d0 con 0x7fa330100e10 2024-04-24T09:04:24.317 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.318+0000 7fa337600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa330114450 con 0x7fa330101730 2024-04-24T09:04:24.317 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.318+0000 7fa337600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa330114750 con 0x7fa33010e9c0 2024-04-24T09:04:24.319 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.318+0000 7fa334c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa330100e10 0x7fa3301a0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:24.319 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.319+0000 7fa334c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa330100e10 0x7fa3301a0780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48138/0 (socket says 172.21.15.83:48138) 2024-04-24T09:04:24.319 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.319+0000 7fa334c00640 1 -- 172.21.15.83:0/2759785250 learned_addr learned my addr 172.21.15.83:0/2759785250 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:04:24.319 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.319+0000 7fa32fe00640 1 --2- 172.21.15.83:0/2759785250 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa330101730 0x7fa3301a0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:24.319 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.319+0000 7fa335600640 1 --2- 172.21.15.83:0/2759785250 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa33010e9c0 0x7fa33019a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:24.319 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.319+0000 7fa334c00640 1 -- 172.21.15.83:0/2759785250 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa33010e9c0 msgr2=0x7fa33019a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:24.320 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.319+0000 7fa334c00640 1 --2- 172.21.15.83:0/2759785250 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa33010e9c0 0x7fa33019a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:24.320 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.319+0000 7fa334c00640 1 -- 172.21.15.83:0/2759785250 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa330101730 msgr2=0x7fa3301a0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:24.320 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.319+0000 7fa334c00640 1 --2- 172.21.15.83:0/2759785250 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa330101730 0x7fa3301a0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:24.320 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.319+0000 7fa334c00640 1 -- 172.21.15.83:0/2759785250 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa33019b0b0 con 0x7fa330100e10 2024-04-24T09:04:24.320 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.319+0000 7fa334c00640 1 --2- 172.21.15.83:0/2759785250 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa330100e10 0x7fa3301a0780 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7fa32400b4a0 tx=0x7fa32400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:24.320 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.320+0000 7fa32d600640 1 -- 172.21.15.83:0/2759785250 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa324014070 con 0x7fa330100e10 2024-04-24T09:04:24.320 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.320+0000 7fa32d600640 1 -- 172.21.15.83:0/2759785250 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa324010ac0 con 0x7fa330100e10 2024-04-24T09:04:24.320 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.320+0000 7fa337600640 1 -- 172.21.15.83:0/2759785250 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa33019b3a0 con 0x7fa330100e10 2024-04-24T09:04:24.320 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.320+0000 7fa32d600640 1 -- 172.21.15.83:0/2759785250 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa324017c30 con 0x7fa330100e10 2024-04-24T09:04:24.320 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.320+0000 7fa337600640 1 -- 172.21.15.83:0/2759785250 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3301a7640 con 0x7fa330100e10 2024-04-24T09:04:24.322 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.323+0000 7fa32d600640 1 -- 172.21.15.83:0/2759785250 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa324017070 con 0x7fa330100e10 2024-04-24T09:04:24.322 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.323+0000 7fa33a587740 1 Processor -- start 2024-04-24T09:04:24.322 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.324+0000 7fa33a587740 1 -- start start 2024-04-24T09:04:24.324 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.324+0000 7fa33a587740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5613f0d331b0 0x5613f0d33590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:24.324 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.324+0000 7fa33a587740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613f0d33ad0 0x5613f0d3de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:24.324 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.324+0000 7fa33a587740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5613f0d3e390 0x5613f0d40780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:24.325 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.324+0000 7fa33a587740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5613f0c99900 con 0x5613f0d331b0 2024-04-24T09:04:24.325 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.324+0000 7fa33a587740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5613f0c3ff20 con 0x5613f0d3e390 2024-04-24T09:04:24.325 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.324+0000 7fa33a587740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5613f0c958c0 con 0x5613f0d33ad0 2024-04-24T09:04:24.325 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.324+0000 7fa335600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5613f0d331b0 0x5613f0d33590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:24.325 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.324+0000 7fa335600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5613f0d331b0 0x5613f0d33590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48150/0 (socket says 172.21.15.83:48150) 2024-04-24T09:04:24.325 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.324+0000 7fa335600640 1 -- 172.21.15.83:0/3299147890 learned_addr learned my addr 172.21.15.83:0/3299147890 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:04:24.325 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.325+0000 7fa334c00640 1 --2- 172.21.15.83:0/3299147890 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613f0d33ad0 0x5613f0d3de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:24.325 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.325+0000 7fa32fe00640 1 --2- 172.21.15.83:0/3299147890 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5613f0d3e390 0x5613f0d40780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:24.325 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.325+0000 7fa335600640 1 -- 172.21.15.83:0/3299147890 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613f0d33ad0 msgr2=0x5613f0d3de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:24.325 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.325+0000 7fa335600640 1 --2- 172.21.15.83:0/3299147890 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613f0d33ad0 0x5613f0d3de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:24.325 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.325+0000 7fa335600640 1 -- 172.21.15.83:0/3299147890 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5613f0d3e390 msgr2=0x5613f0d40780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:24.325 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.325+0000 7fa335600640 1 --2- 172.21.15.83:0/3299147890 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5613f0d3e390 0x5613f0d40780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:24.325 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.325+0000 7fa32d600640 1 --2- 172.21.15.83:0/2759785250 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa3180a3e40 0x7fa3180a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:24.326 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.325+0000 7fa335600640 1 -- 172.21.15.83:0/3299147890 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613f0c64e10 con 0x5613f0d331b0 2024-04-24T09:04:24.326 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.325+0000 7fa32fe00640 1 --2- 172.21.15.83:0/3299147890 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5613f0d3e390 0x5613f0d40780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:04:24.326 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.325+0000 7fa32d600640 1 -- 172.21.15.83:0/2759785250 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7fa3240d0250 con 0x7fa330100e10 2024-04-24T09:04:24.326 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.326+0000 7fa335600640 1 --2- 172.21.15.83:0/2759785250 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa3180a3e40 0x7fa3180a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:24.326 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.326+0000 7fa335600640 1 --2- 172.21.15.83:0/3299147890 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5613f0d331b0 0x5613f0d33590 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7fa328009ef0 tx=0x7fa32800c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=292514445e1796ce server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:24.329 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.326+0000 7fa321c00640 1 -- 172.21.15.83:0/3299147890 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa328007640 con 0x5613f0d331b0 2024-04-24T09:04:24.329 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.326+0000 7fa321c00640 1 -- 172.21.15.83:0/3299147890 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa328017de0 con 0x5613f0d331b0 2024-04-24T09:04:24.329 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.326+0000 7fa321c00640 1 -- 172.21.15.83:0/3299147890 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa328015a50 con 0x5613f0d331b0 2024-04-24T09:04:24.329 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.327+0000 7fa33a587740 1 -- 172.21.15.83:0/3299147890 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5613f0d331b0 msgr2=0x5613f0d33590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:24.330 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.327+0000 7fa33a587740 1 --2- 172.21.15.83:0/3299147890 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5613f0d331b0 0x5613f0d33590 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7fa328009ef0 tx=0x7fa32800c7f0 comp rx=0 tx=0).stop 2024-04-24T09:04:24.330 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.327+0000 7fa335600640 1 --2- 172.21.15.83:0/2759785250 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa3180a3e40 0x7fa3180a6300 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fa32800c3f0 tx=0x7fa32801d000 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:24.330 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.327+0000 7fa33a587740 1 -- 172.21.15.83:0/3299147890 shutdown_connections 2024-04-24T09:04:24.330 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.327+0000 7fa33a587740 1 --2- 172.21.15.83:0/3299147890 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5613f0d3e390 0x5613f0d40780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:24.330 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.327+0000 7fa33a587740 1 --2- 172.21.15.83:0/3299147890 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613f0d33ad0 0x5613f0d3de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:24.330 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.327+0000 7fa33a587740 1 --2- 172.21.15.83:0/3299147890 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5613f0d331b0 0x5613f0d33590 unknown :-1 s=CLOSED pgs=2001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:24.330 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.327+0000 7fa33a587740 1 -- 172.21.15.83:0/3299147890 >> 172.21.15.83:0/3299147890 conn(0x5613f0d2f6c0 msgr2=0x5613f0d2faa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:24.330 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.327+0000 7fa33a587740 1 -- 172.21.15.83:0/3299147890 shutdown_connections 2024-04-24T09:04:24.330 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.327+0000 7fa33a587740 1 -- 172.21.15.83:0/3299147890 wait complete. 2024-04-24T09:04:24.330 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.328+0000 7fa33a587740 1 -- start start 2024-04-24T09:04:24.330 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.328+0000 7fa33a587740 1 Processor -- start 2024-04-24T09:04:24.330 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.328+0000 7fa33a587740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5613f0d331b0 0x5613f0d3b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:24.330 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.328+0000 7fa33a587740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5613f0d33ad0 0x5613f0d3b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:24.331 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.328+0000 7fa33a587740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613f0d3e390 0x5613f0d3bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:24.331 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.328+0000 7fa33a587740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5613f0d40fc0 con 0x5613f0d331b0 2024-04-24T09:04:24.331 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.328+0000 7fa33a587740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5613f0c532c0 con 0x5613f0d33ad0 2024-04-24T09:04:24.331 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.328+0000 7fa33a587740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5613f0d41140 con 0x5613f0d3e390 2024-04-24T09:04:24.331 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.328+0000 7fa33a587740 10 client.0 _ll_register_callbacks cb 0x5613f09aba00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:04:24.332 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.329+0000 7fa334c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5613f0d331b0 0x5613f0d3b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:24.332 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.329+0000 7fa334c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5613f0d331b0 0x5613f0d3b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48160/0 (socket says 172.21.15.83:48160) 2024-04-24T09:04:24.332 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.329+0000 7fa334c00640 1 -- 172.21.15.83:0/1226611378 learned_addr learned my addr 172.21.15.83:0/1226611378 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:04:24.332 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.329+0000 7fa334c00640 1 -- 172.21.15.83:0/1226611378 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613f0d3e390 msgr2=0x5613f0d3bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:24.332 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.329+0000 7fa334c00640 1 --2- 172.21.15.83:0/1226611378 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613f0d3e390 0x5613f0d3bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:24.332 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.329+0000 7fa334c00640 1 -- 172.21.15.83:0/1226611378 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5613f0d33ad0 msgr2=0x5613f0d3b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:24.332 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.329+0000 7fa334c00640 1 --2- 172.21.15.83:0/1226611378 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5613f0d33ad0 0x5613f0d3b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:24.332 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.329+0000 7fa334c00640 1 -- 172.21.15.83:0/1226611378 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613f0c63fd0 con 0x5613f0d331b0 2024-04-24T09:04:24.332 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.329+0000 7fa334c00640 1 --2- 172.21.15.83:0/1226611378 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5613f0d331b0 0x5613f0d3b250 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7fa32400be60 tx=0x7fa32409ba80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c435f578543a406 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:24.332 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.329+0000 7fa316000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:04:24.332 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.329+0000 7fa316000640 1 -- 172.21.15.83:0/1226611378 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa32401e070 con 0x5613f0d331b0 2024-04-24T09:04:24.332 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.330+0000 7fa316000640 1 -- 172.21.15.83:0/1226611378 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7fa324010c70 con 0x5613f0d331b0 2024-04-24T09:04:24.332 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.330+0000 7fa33a587740 1 -- 172.21.15.83:0/1226611378 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa328017de0 con 0x5613f0d331b0 2024-04-24T09:04:24.332 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.330+0000 7fa316000640 1 -- 172.21.15.83:0/1226611378 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fa32409be80 con 0x5613f0d331b0 2024-04-24T09:04:24.333 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.330+0000 7fa33a587740 10 client.20166 fetch_fsmap learned FSMap version 41 2024-04-24T09:04:24.333 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.330+0000 7fa33a587740 1 -- 172.21.15.83:0/1226611378 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5613f09ac210 con 0x5613f0d331b0 2024-04-24T09:04:24.334 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.330+0000 7fa316000640 1 -- 172.21.15.83:0/1226611378 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa328017de0 con 0x5613f0d331b0 2024-04-24T09:04:24.334 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.330+0000 7fa316000640 1 -- 172.21.15.83:0/1226611378 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7fa3240cd7a0 con 0x5613f0d331b0 2024-04-24T09:04:24.334 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.330+0000 7fa33a587740 10 client.20166 fetch_fsmap finished waiting for FSMap version 41 2024-04-24T09:04:24.334 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.330+0000 7fa33a587740 10 client.20166 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:04:24.334 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.331+0000 7fa33a587740 20 client.20166 populate_metadata read hostname 'smithi083' 2024-04-24T09:04:24.334 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.331+0000 7fa33a587740 1 --2- 172.21.15.83:0/1226611378 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5613f0d4e1c0 0x5613f0d50610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:24.334 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.331+0000 7fa33a587740 4 client.20166 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:04:24.334 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.331+0000 7fa33a587740 1 -- 172.21.15.83:0/1226611378 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5613f0ca7320 con 0x5613f0d4e1c0 2024-04-24T09:04:24.336 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.331+0000 7fa32fe00640 1 --2- 172.21.15.83:0/1226611378 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5613f0d4e1c0 0x5613f0d50610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:24.336 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.332+0000 7fa32fe00640 1 --2- 172.21.15.83:0/1226611378 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5613f0d4e1c0 0x5613f0d50610 crc :-1 s=READY pgs=333 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd175e353df9bd2e server_cookie=6ad42ebb75412a86 in_seq=0 out_seq=0 2024-04-24T09:04:24.336 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.332+0000 7fa316000640 10 client.20166 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:04:24.336 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.335+0000 7fa316000640 1 -- 172.21.15.83:0/1226611378 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5613f0ca7320 con 0x5613f0d4e1c0 2024-04-24T09:04:24.336 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.335+0000 7fa316000640 10 client.20166 handle_command_reply: tid=0 2024-04-24T09:04:24.336 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.335+0000 7fa33a587740 1 client.20166 shutdown 2024-04-24T09:04:24.336 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.335+0000 7fa33a587740 1 -- 172.21.15.83:0/1226611378 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5613f0d331b0 msgr2=0x5613f0d3b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:24.336 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.335+0000 7fa33a587740 1 --2- 172.21.15.83:0/1226611378 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5613f0d331b0 0x5613f0d3b250 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7fa32400be60 tx=0x7fa32409ba80 comp rx=0 tx=0).stop 2024-04-24T09:04:24.336 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.335+0000 7fa33a587740 1 -- 172.21.15.83:0/1226611378 shutdown_connections 2024-04-24T09:04:24.336 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.335+0000 7fa33a587740 1 --2- 172.21.15.83:0/1226611378 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5613f0d4e1c0 0x5613f0d50610 crc :-1 s=READY pgs=333 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:24.336 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.335+0000 7fa33a587740 1 --2- 172.21.15.83:0/1226611378 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613f0d3e390 0x5613f0d3bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:24.336 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.335+0000 7fa33a587740 1 --2- 172.21.15.83:0/1226611378 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5613f0d33ad0 0x5613f0d3b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:24.337 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.335+0000 7fa33a587740 1 --2- 172.21.15.83:0/1226611378 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5613f0d331b0 0x5613f0d3b250 unknown :-1 s=CLOSED pgs=2002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:24.337 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.335+0000 7fa316000640 0 client.20166 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:04:24.337 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.335+0000 7fa33a587740 1 -- 172.21.15.83:0/1226611378 >> 172.21.15.83:0/1226611378 conn(0x5613f0d2f6c0 msgr2=0x5613f0d353f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:24.337 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.335+0000 7fa33a587740 1 -- 172.21.15.83:0/1226611378 shutdown_connections 2024-04-24T09:04:24.337 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.335+0000 7fa33a587740 1 -- 172.21.15.83:0/1226611378 wait complete. 2024-04-24T09:04:24.337 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.335+0000 7fa33a587740 20 client.20166 trim_cache size 0 max 16384 2024-04-24T09:04:24.351 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.352+0000 7fa33a587740 1 Processor -- start 2024-04-24T09:04:24.351 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.352+0000 7fa33a587740 1 -- start start 2024-04-24T09:04:24.351 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.352+0000 7fa33a587740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5613f0d331b0 0x5613f0d35b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:24.351 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.352+0000 7fa33a587740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613f0d33ad0 0x5613f0d55f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:24.351 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.352+0000 7fa33a587740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5613f0d3e390 0x5613f0d56460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:24.351 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.352+0000 7fa33a587740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa3301b1850 con 0x5613f0d331b0 2024-04-24T09:04:24.352 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.352+0000 7fa33a587740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa3301b34b0 con 0x5613f0d3e390 2024-04-24T09:04:24.352 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.352+0000 7fa33a587740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa3301b1f20 con 0x5613f0d33ad0 2024-04-24T09:04:24.357 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.353+0000 7fa334c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5613f0d331b0 0x5613f0d35b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:24.357 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.353+0000 7fa334c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5613f0d331b0 0x5613f0d35b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48162/0 (socket says 172.21.15.83:48162) 2024-04-24T09:04:24.357 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.353+0000 7fa334c00640 1 -- 172.21.15.83:0/2609123336 learned_addr learned my addr 172.21.15.83:0/2609123336 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:04:24.357 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.353+0000 7fa335600640 1 --2- 172.21.15.83:0/2609123336 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5613f0d3e390 0x5613f0d56460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:24.357 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.353+0000 7fa32fe00640 1 --2- 172.21.15.83:0/2609123336 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613f0d33ad0 0x5613f0d55f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:24.357 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.353+0000 7fa334c00640 1 -- 172.21.15.83:0/2609123336 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613f0d33ad0 msgr2=0x5613f0d55f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:24.357 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.353+0000 7fa334c00640 1 --2- 172.21.15.83:0/2609123336 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613f0d33ad0 0x5613f0d55f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:24.357 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.353+0000 7fa334c00640 1 -- 172.21.15.83:0/2609123336 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5613f0d3e390 msgr2=0x5613f0d56460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:24.358 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.353+0000 7fa334c00640 1 --2- 172.21.15.83:0/2609123336 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5613f0d3e390 0x5613f0d56460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:24.358 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.353+0000 7fa334c00640 1 -- 172.21.15.83:0/2609123336 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613f0a78620 con 0x5613f0d331b0 2024-04-24T09:04:24.358 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.353+0000 7fa334c00640 1 --2- 172.21.15.83:0/2609123336 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5613f0d331b0 0x5613f0d35b70 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7fa3240982e0 tx=0x7fa32400bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1292d1b8ffc29f2e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:24.358 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.354+0000 7fa321200640 1 -- 172.21.15.83:0/2609123336 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3240ba070 con 0x5613f0d331b0 2024-04-24T09:04:24.358 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.354+0000 7fa321200640 1 -- 172.21.15.83:0/2609123336 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa3240a0230 con 0x5613f0d331b0 2024-04-24T09:04:24.358 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.354+0000 7fa321200640 1 -- 172.21.15.83:0/2609123336 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa32401e070 con 0x5613f0d331b0 2024-04-24T09:04:24.358 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.354+0000 7fa33a587740 1 -- 172.21.15.83:0/2609123336 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5613f0d331b0 msgr2=0x5613f0d35b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:24.358 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.354+0000 7fa33a587740 1 --2- 172.21.15.83:0/2609123336 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5613f0d331b0 0x5613f0d35b70 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7fa3240982e0 tx=0x7fa32400bf90 comp rx=0 tx=0).stop 2024-04-24T09:04:24.358 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.354+0000 7fa33a587740 1 -- 172.21.15.83:0/2609123336 shutdown_connections 2024-04-24T09:04:24.358 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.354+0000 7fa33a587740 1 --2- 172.21.15.83:0/2609123336 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5613f0d3e390 0x5613f0d56460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:24.358 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.354+0000 7fa33a587740 1 --2- 172.21.15.83:0/2609123336 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613f0d33ad0 0x5613f0d55f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:24.358 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.354+0000 7fa33a587740 1 --2- 172.21.15.83:0/2609123336 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5613f0d331b0 0x5613f0d35b70 unknown :-1 s=CLOSED pgs=2003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:24.359 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.354+0000 7fa33a587740 1 -- 172.21.15.83:0/2609123336 >> 172.21.15.83:0/2609123336 conn(0x5613f0d2f6c0 msgr2=0x5613f0d2e840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:24.359 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.355+0000 7fa33a587740 1 -- 172.21.15.83:0/2609123336 shutdown_connections 2024-04-24T09:04:24.359 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.355+0000 7fa33a587740 1 -- 172.21.15.83:0/2609123336 wait complete. 2024-04-24T09:04:24.359 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.355+0000 7fa33a587740 1 -- start start 2024-04-24T09:04:24.359 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.355+0000 7fa33a587740 1 Processor -- start 2024-04-24T09:04:24.359 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.356+0000 7fa33a587740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5613f0d331b0 0x5613f0d37f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:24.359 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.356+0000 7fa33a587740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613f0d33ad0 0x5613f0d38460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:24.359 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.356+0000 7fa33a587740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5613f0d3e390 0x5613f0d389a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:24.359 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.356+0000 7fa33a587740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa3301b2d80 con 0x5613f0d3e390 2024-04-24T09:04:24.359 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.356+0000 7fa33a587740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa3301b2650 con 0x5613f0d331b0 2024-04-24T09:04:24.359 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.356+0000 7fa33a587740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa3301af850 con 0x5613f0d33ad0 2024-04-24T09:04:24.359 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.356+0000 7fa33a587740 10 client.0 _ll_register_callbacks cb 0x7fa3240175f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:04:24.359 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.356+0000 7fa335600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5613f0d3e390 0x5613f0d389a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:24.359 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.356+0000 7fa335600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5613f0d3e390 0x5613f0d389a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48176/0 (socket says 172.21.15.83:48176) 2024-04-24T09:04:24.359 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.356+0000 7fa335600640 1 -- 172.21.15.83:0/3459123943 learned_addr learned my addr 172.21.15.83:0/3459123943 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:04:24.360 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.356+0000 7fa32fe00640 1 --2- 172.21.15.83:0/3459123943 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613f0d33ad0 0x5613f0d38460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:24.360 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.356+0000 7fa334c00640 1 --2- 172.21.15.83:0/3459123943 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5613f0d331b0 0x5613f0d37f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:24.360 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.356+0000 7fa335600640 1 -- 172.21.15.83:0/3459123943 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613f0d33ad0 msgr2=0x5613f0d38460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:24.360 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.356+0000 7fa335600640 1 --2- 172.21.15.83:0/3459123943 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613f0d33ad0 0x5613f0d38460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:24.360 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.356+0000 7fa335600640 1 -- 172.21.15.83:0/3459123943 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5613f0d331b0 msgr2=0x5613f0d37f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:24.360 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.356+0000 7fa335600640 1 --2- 172.21.15.83:0/3459123943 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5613f0d331b0 0x5613f0d37f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:24.360 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.356+0000 7fa335600640 1 -- 172.21.15.83:0/3459123943 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613f0db3790 con 0x5613f0d3e390 2024-04-24T09:04:24.360 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.356+0000 7fa334c00640 1 --2- 172.21.15.83:0/3459123943 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5613f0d331b0 0x5613f0d37f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:04:24.360 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.357+0000 7fa335600640 1 --2- 172.21.15.83:0/3459123943 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5613f0d3e390 0x5613f0d389a0 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7fa328004a70 tx=0x7fa3280097c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54de55c8ce2c04c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:24.360 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.357+0000 7fa316000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:04:24.360 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.357+0000 7fa316000640 1 -- 172.21.15.83:0/3459123943 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa328025050 con 0x5613f0d3e390 2024-04-24T09:04:24.360 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.357+0000 7fa316000640 1 -- 172.21.15.83:0/3459123943 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7fa3280093d0 con 0x5613f0d3e390 2024-04-24T09:04:24.360 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.357+0000 7fa33a587740 1 -- 172.21.15.83:0/3459123943 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3240a0230 con 0x5613f0d3e390 2024-04-24T09:04:24.360 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.357+0000 7fa316000640 1 -- 172.21.15.83:0/3459123943 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fa32803a050 con 0x5613f0d3e390 2024-04-24T09:04:24.361 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.357+0000 7fa33a587740 10 client.20172 fetch_fsmap learned FSMap version 41 2024-04-24T09:04:24.361 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.357+0000 7fa33a587740 1 -- 172.21.15.83:0/3459123943 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5613f0db4a80 con 0x5613f0d3e390 2024-04-24T09:04:24.361 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.358+0000 7fa316000640 1 -- 172.21.15.83:0/3459123943 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa3240a0230 con 0x5613f0d3e390 2024-04-24T09:04:24.361 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.358+0000 7fa316000640 1 -- 172.21.15.83:0/3459123943 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7fa32802c450 con 0x5613f0d3e390 2024-04-24T09:04:24.361 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.358+0000 7fa33a587740 10 client.20172 fetch_fsmap finished waiting for FSMap version 41 2024-04-24T09:04:24.361 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.358+0000 7fa33a587740 10 client.20172 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:04:24.361 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.358+0000 7fa33a587740 20 client.20172 populate_metadata read hostname 'smithi083' 2024-04-24T09:04:24.361 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.358+0000 7fa33a587740 1 --2- 172.21.15.83:0/3459123943 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5613f0db4f60 0x5613f0db7380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:24.361 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.358+0000 7fa33a587740 4 client.20172 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T09:04:24.361 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.358+0000 7fa33a587740 1 -- 172.21.15.83:0/3459123943 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5613f0ca1190 con 0x5613f0db4f60 2024-04-24T09:04:24.362 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.359+0000 7fa334c00640 1 --2- 172.21.15.83:0/3459123943 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5613f0db4f60 0x5613f0db7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:24.362 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.360+0000 7fa334c00640 1 --2- 172.21.15.83:0/3459123943 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5613f0db4f60 0x5613f0db7380 crc :-1 s=READY pgs=334 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ecffac7dd7301ba9 server_cookie=97cdb5b52574ce39 in_seq=0 out_seq=0 2024-04-24T09:04:24.362 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.360+0000 7fa316000640 10 client.20172 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:04:24.427 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.427+0000 7fa316000640 1 -- 172.21.15.83:0/3459123943 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5613f0ca1190 con 0x5613f0db4f60 2024-04-24T09:04:24.427 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.427+0000 7fa316000640 10 client.20172 handle_command_reply: tid=0 2024-04-24T09:04:24.427 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.427+0000 7fa33a587740 1 client.20172 shutdown 2024-04-24T09:04:24.427 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.428+0000 7fa33a587740 1 -- 172.21.15.83:0/3459123943 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5613f0d3e390 msgr2=0x5613f0d389a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:24.427 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.428+0000 7fa33a587740 1 --2- 172.21.15.83:0/3459123943 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5613f0d3e390 0x5613f0d389a0 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7fa328004a70 tx=0x7fa3280097c0 comp rx=0 tx=0).stop 2024-04-24T09:04:24.427 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.428+0000 7fa33a587740 1 -- 172.21.15.83:0/3459123943 shutdown_connections 2024-04-24T09:04:24.427 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.428+0000 7fa33a587740 1 --2- 172.21.15.83:0/3459123943 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5613f0db4f60 0x5613f0db7380 crc :-1 s=READY pgs=334 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:24.427 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.428+0000 7fa33a587740 1 --2- 172.21.15.83:0/3459123943 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5613f0d3e390 0x5613f0d389a0 unknown :-1 s=CLOSED pgs=2004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:24.427 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.428+0000 7fa33a587740 1 --2- 172.21.15.83:0/3459123943 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613f0d33ad0 0x5613f0d38460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:24.427 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.428+0000 7fa33a587740 1 --2- 172.21.15.83:0/3459123943 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5613f0d331b0 0x5613f0d37f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:24.427 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.428+0000 7fa316000640 0 client.20172 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:04:24.427 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.428+0000 7fa33a587740 1 -- 172.21.15.83:0/3459123943 >> 172.21.15.83:0/3459123943 conn(0x5613f0d2f6c0 msgr2=0x5613f0d3b210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:24.428 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.428+0000 7fa33a587740 1 -- 172.21.15.83:0/3459123943 shutdown_connections 2024-04-24T09:04:24.428 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.428+0000 7fa33a587740 1 -- 172.21.15.83:0/3459123943 wait complete. 2024-04-24T09:04:24.428 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.429+0000 7fa33a587740 20 client.20172 trim_cache size 0 max 16384 2024-04-24T09:04:24.429 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T09:04:24.429 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T09:04:24.429 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T09:04:24.429 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T09:04:24.430 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.430+0000 7fa337600640 1 -- 172.21.15.83:0/2759785250 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa3180a3e40 msgr2=0x7fa3180a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:24.430 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.430+0000 7fa337600640 1 --2- 172.21.15.83:0/2759785250 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa3180a3e40 0x7fa3180a6300 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fa32800c3f0 tx=0x7fa32801d000 comp rx=0 tx=0).stop 2024-04-24T09:04:24.430 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.430+0000 7fa337600640 1 -- 172.21.15.83:0/2759785250 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa330100e10 msgr2=0x7fa3301a0780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:24.430 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.430+0000 7fa337600640 1 --2- 172.21.15.83:0/2759785250 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa330100e10 0x7fa3301a0780 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7fa32400b4a0 tx=0x7fa32400b960 comp rx=0 tx=0).stop 2024-04-24T09:04:24.430 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.430+0000 7fa337600640 1 -- 172.21.15.83:0/2759785250 shutdown_connections 2024-04-24T09:04:24.430 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.430+0000 7fa337600640 1 --2- 172.21.15.83:0/2759785250 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa3180a3e40 0x7fa3180a6300 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:24.430 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.430+0000 7fa337600640 1 --2- 172.21.15.83:0/2759785250 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa33010e9c0 0x7fa33019a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:24.430 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.430+0000 7fa337600640 1 --2- 172.21.15.83:0/2759785250 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa330101730 0x7fa3301a0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:24.430 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.430+0000 7fa337600640 1 --2- 172.21.15.83:0/2759785250 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa330100e10 0x7fa3301a0780 unknown :-1 s=CLOSED pgs=2000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:24.430 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.430+0000 7fa337600640 1 -- 172.21.15.83:0/2759785250 >> 172.21.15.83:0/2759785250 conn(0x7fa330078080 msgr2=0x7fa33010ff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:24.430 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.430+0000 7fa337600640 1 -- 172.21.15.83:0/2759785250 shutdown_connections 2024-04-24T09:04:24.430 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:24.430+0000 7fa337600640 1 -- 172.21.15.83:0/2759785250 wait complete. 2024-04-24T09:04:24.626 INFO:tasks.workunit.client.0.smithi083.stdout: 1 74292 0.60 MB/sec execute 538 sec latency 65.833 ms 2024-04-24T09:04:25.626 INFO:tasks.workunit.client.0.smithi083.stdout: 1 74318 0.59 MB/sec execute 539 sec latency 67.215 ms 2024-04-24T09:04:25.954 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:04:25 smithi138 ceph-mon[31441]: pgmap v403: 129 pgs: 129 active+clean; 70 MiB data, 972 MiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 398 KiB/s wr, 37 op/s 2024-04-24T09:04:25.956 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:04:25 smithi189 ceph-mon[31791]: pgmap v403: 129 pgs: 129 active+clean; 70 MiB data, 972 MiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 398 KiB/s wr, 37 op/s 2024-04-24T09:04:26.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:04:25 smithi083 ceph-mon[27907]: pgmap v403: 129 pgs: 129 active+clean; 70 MiB data, 972 MiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 398 KiB/s wr, 37 op/s 2024-04-24T09:04:26.626 INFO:tasks.workunit.client.0.smithi083.stdout: 1 74378 0.59 MB/sec execute 540 sec latency 57.850 ms 2024-04-24T09:04:27.626 INFO:tasks.workunit.client.0.smithi083.stdout: 1 74548 0.60 MB/sec execute 541 sec latency 58.348 ms 2024-04-24T09:04:28.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:04:27 smithi138 ceph-mon[31441]: pgmap v404: 129 pgs: 129 active+clean; 70 MiB data, 972 MiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 398 KiB/s wr, 37 op/s 2024-04-24T09:04:28.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:04:27 smithi189 ceph-mon[31791]: pgmap v404: 129 pgs: 129 active+clean; 70 MiB data, 972 MiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 398 KiB/s wr, 37 op/s 2024-04-24T09:04:28.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:04:27 smithi083 ceph-mon[27907]: pgmap v404: 129 pgs: 129 active+clean; 70 MiB data, 972 MiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 398 KiB/s wr, 37 op/s 2024-04-24T09:04:28.626 INFO:tasks.workunit.client.0.smithi083.stdout: 1 74612 0.60 MB/sec execute 542 sec latency 577.660 ms 2024-04-24T09:04:29.627 INFO:tasks.workunit.client.0.smithi083.stdout: 1 74612 0.60 MB/sec execute 543 sec latency 1577.791 ms 2024-04-24T09:04:30.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:04:29 smithi138 ceph-mon[31441]: pgmap v405: 129 pgs: 129 active+clean; 71 MiB data, 984 MiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 577 KiB/s wr, 58 op/s 2024-04-24T09:04:30.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:04:29 smithi189 ceph-mon[31791]: pgmap v405: 129 pgs: 129 active+clean; 71 MiB data, 984 MiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 577 KiB/s wr, 58 op/s 2024-04-24T09:04:30.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:04:29 smithi083 ceph-mon[27907]: pgmap v405: 129 pgs: 129 active+clean; 71 MiB data, 984 MiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 577 KiB/s wr, 58 op/s 2024-04-24T09:04:30.627 INFO:tasks.workunit.client.0.smithi083.stdout: 1 74612 0.59 MB/sec execute 544 sec latency 2577.956 ms 2024-04-24T09:04:31.627 INFO:tasks.workunit.client.0.smithi083.stdout: 1 74612 0.59 MB/sec execute 545 sec latency 3578.098 ms 2024-04-24T09:04:32.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:04:31 smithi138 ceph-mon[31441]: pgmap v406: 129 pgs: 129 active+clean; 71 MiB data, 984 MiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 518 KiB/s wr, 41 op/s 2024-04-24T09:04:32.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:04:31 smithi189 ceph-mon[31791]: pgmap v406: 129 pgs: 129 active+clean; 71 MiB data, 984 MiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 518 KiB/s wr, 41 op/s 2024-04-24T09:04:32.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:04:31 smithi083 ceph-mon[27907]: pgmap v406: 129 pgs: 129 active+clean; 71 MiB data, 984 MiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 518 KiB/s wr, 41 op/s 2024-04-24T09:04:32.627 INFO:tasks.workunit.client.0.smithi083.stdout: 1 74612 0.59 MB/sec execute 546 sec latency 4578.241 ms 2024-04-24T09:04:33.627 INFO:tasks.workunit.client.0.smithi083.stdout: 1 74651 0.59 MB/sec execute 547 sec latency 5056.446 ms 2024-04-24T09:04:34.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:04:33 smithi138 ceph-mon[31441]: pgmap v407: 129 pgs: 129 active+clean; 71 MiB data, 984 MiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 537 KiB/s wr, 50 op/s 2024-04-24T09:04:34.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:04:33 smithi189 ceph-mon[31791]: pgmap v407: 129 pgs: 129 active+clean; 71 MiB data, 984 MiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 537 KiB/s wr, 50 op/s 2024-04-24T09:04:34.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:04:33 smithi083 ceph-mon[27907]: pgmap v407: 129 pgs: 129 active+clean; 71 MiB data, 984 MiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 537 KiB/s wr, 50 op/s 2024-04-24T09:04:34.627 INFO:tasks.workunit.client.0.smithi083.stdout: 1 74679 0.59 MB/sec execute 548 sec latency 631.049 ms 2024-04-24T09:04:35.627 INFO:tasks.workunit.client.0.smithi083.stdout: 1 74679 0.59 MB/sec execute 549 sec latency 1631.194 ms 2024-04-24T09:04:36.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:04:35 smithi138 ceph-mon[31441]: pgmap v408: 129 pgs: 129 active+clean; 71 MiB data, 984 MiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 198 KiB/s wr, 29 op/s 2024-04-24T09:04:36.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:04:35 smithi189 ceph-mon[31791]: pgmap v408: 129 pgs: 129 active+clean; 71 MiB data, 984 MiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 198 KiB/s wr, 29 op/s 2024-04-24T09:04:36.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:04:35 smithi083 ceph-mon[27907]: pgmap v408: 129 pgs: 129 active+clean; 71 MiB data, 984 MiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 198 KiB/s wr, 29 op/s 2024-04-24T09:04:36.627 INFO:tasks.workunit.client.0.smithi083.stdout: 1 74679 0.59 MB/sec execute 550 sec latency 2631.355 ms 2024-04-24T09:04:37.578 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.577+0000 7f26f6800640 1 -- 172.21.15.83:0/3604089467 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26f0076530 msgr2=0x7f26f0076910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:37.578 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.577+0000 7f26f6800640 1 --2- 172.21.15.83:0/3604089467 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26f0076530 0x7f26f0076910 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7f26e4009900 tx=0x7f26e402a3f0 comp rx=0 tx=0).stop 2024-04-24T09:04:37.578 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.578+0000 7f26f6800640 1 -- 172.21.15.83:0/3604089467 shutdown_connections 2024-04-24T09:04:37.578 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.578+0000 7f26f6800640 1 --2- 172.21.15.83:0/3604089467 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26f0112cc0 0x7f26f01150b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:37.578 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.578+0000 7f26f6800640 1 --2- 172.21.15.83:0/3604089467 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f26f0076e50 0x7f26f0112780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:37.578 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.578+0000 7f26f6800640 1 --2- 172.21.15.83:0/3604089467 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26f0076530 0x7f26f0076910 unknown :-1 s=CLOSED pgs=2005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:37.579 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.578+0000 7f26f6800640 1 -- 172.21.15.83:0/3604089467 >> 172.21.15.83:0/3604089467 conn(0x7f26f00ffb10 msgr2=0x7f26f0101f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:37.579 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.578+0000 7f26f6800640 1 -- 172.21.15.83:0/3604089467 shutdown_connections 2024-04-24T09:04:37.579 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.578+0000 7f26f6800640 1 -- 172.21.15.83:0/3604089467 wait complete. 2024-04-24T09:04:37.581 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.579+0000 7f26f6800640 1 Processor -- start 2024-04-24T09:04:37.581 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.579+0000 7f26f6800640 1 -- start start 2024-04-24T09:04:37.581 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.579+0000 7f26f6800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26f0076530 0x7f26f01a4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:37.581 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.579+0000 7f26f6800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f26f0076e50 0x7f26f01a4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:37.581 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.579+0000 7f26f6800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26f0112cc0 0x7f26f019ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:37.581 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.579+0000 7f26f6800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f26f0118730 con 0x7f26f0076530 2024-04-24T09:04:37.581 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.579+0000 7f26f6800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f26f01185b0 con 0x7f26f0112cc0 2024-04-24T09:04:37.581 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.579+0000 7f26f6800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f26f01188b0 con 0x7f26f0076e50 2024-04-24T09:04:37.581 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.579+0000 7f26f5400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26f0076530 0x7f26f01a4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:37.582 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.580+0000 7f26f5400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26f0076530 0x7f26f01a4850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55994/0 (socket says 172.21.15.83:55994) 2024-04-24T09:04:37.582 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.580+0000 7f26f5400640 1 -- 172.21.15.83:0/3528365720 learned_addr learned my addr 172.21.15.83:0/3528365720 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:04:37.582 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.580+0000 7f26f4a00640 1 --2- 172.21.15.83:0/3528365720 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f26f0076e50 0x7f26f01a4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:37.582 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.580+0000 7f26f5e00640 1 --2- 172.21.15.83:0/3528365720 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26f0112cc0 0x7f26f019ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:37.582 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.580+0000 7f26f5400640 1 -- 172.21.15.83:0/3528365720 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f26f0076e50 msgr2=0x7f26f01a4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:37.582 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.580+0000 7f26f5400640 1 --2- 172.21.15.83:0/3528365720 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f26f0076e50 0x7f26f01a4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:37.582 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.580+0000 7f26f5400640 1 -- 172.21.15.83:0/3528365720 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26f0112cc0 msgr2=0x7f26f019ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:37.582 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.580+0000 7f26f5400640 1 --2- 172.21.15.83:0/3528365720 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26f0112cc0 0x7f26f019ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:37.582 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.580+0000 7f26f5400640 1 -- 172.21.15.83:0/3528365720 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26f019f1d0 con 0x7f26f0076530 2024-04-24T09:04:37.582 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.580+0000 7f26f5e00640 1 --2- 172.21.15.83:0/3528365720 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26f0112cc0 0x7f26f019ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:04:37.582 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.580+0000 7f26f5400640 1 --2- 172.21.15.83:0/3528365720 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26f0076530 0x7f26f01a4850 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7f26e4006010 tx=0x7f26e4003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:37.582 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.581+0000 7f26ee000640 1 -- 172.21.15.83:0/3528365720 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e403f070 con 0x7f26f0076530 2024-04-24T09:04:37.582 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.581+0000 7f26ee000640 1 -- 172.21.15.83:0/3528365720 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f26e403bd20 con 0x7f26f0076530 2024-04-24T09:04:37.582 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.581+0000 7f26f6800640 1 -- 172.21.15.83:0/3528365720 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26f019f460 con 0x7f26f0076530 2024-04-24T09:04:37.583 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.581+0000 7f26ee000640 1 -- 172.21.15.83:0/3528365720 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e4035030 con 0x7f26f0076530 2024-04-24T09:04:37.583 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.581+0000 7f26f6800640 1 -- 172.21.15.83:0/3528365720 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26f01ab710 con 0x7f26f0076530 2024-04-24T09:04:37.585 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.584+0000 7f26fabd0740 1 Processor -- start 2024-04-24T09:04:37.585 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.584+0000 7f26ee000640 1 -- 172.21.15.83:0/3528365720 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f26e4042070 con 0x7f26f0076530 2024-04-24T09:04:37.585 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.584+0000 7f26fabd0740 1 -- start start 2024-04-24T09:04:37.585 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.584+0000 7f26fabd0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561213c453c0 0x561213c457a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:37.585 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.584+0000 7f26fabd0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561213c45ce0 0x561213c50060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:37.586 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.584+0000 7f26fabd0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561213c505a0 0x561213c52990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:37.586 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.584+0000 7f26fabd0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561213bab900 con 0x561213c453c0 2024-04-24T09:04:37.586 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.584+0000 7f26fabd0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x561213b51f20 con 0x561213c45ce0 2024-04-24T09:04:37.586 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.584+0000 7f26fabd0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x561213ba78c0 con 0x561213c505a0 2024-04-24T09:04:37.586 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.585+0000 7f26f5e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561213c453c0 0x561213c457a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:37.586 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.585+0000 7f26f5e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561213c453c0 0x561213c457a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56002/0 (socket says 172.21.15.83:56002) 2024-04-24T09:04:37.586 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.585+0000 7f26f5e00640 1 -- 172.21.15.83:0/3975603083 learned_addr learned my addr 172.21.15.83:0/3975603083 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:04:37.586 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.585+0000 7f26f5400640 1 --2- 172.21.15.83:0/3975603083 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561213c45ce0 0x561213c50060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:37.586 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.585+0000 7f26f4a00640 1 --2- 172.21.15.83:0/3975603083 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561213c505a0 0x561213c52990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:37.586 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.585+0000 7f26f5e00640 1 -- 172.21.15.83:0/3975603083 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561213c505a0 msgr2=0x561213c52990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:37.586 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.585+0000 7f26f5e00640 1 --2- 172.21.15.83:0/3975603083 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561213c505a0 0x561213c52990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:37.586 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.585+0000 7f26f5e00640 1 -- 172.21.15.83:0/3975603083 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561213c45ce0 msgr2=0x561213c50060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:37.587 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.585+0000 7f26f5e00640 1 --2- 172.21.15.83:0/3975603083 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561213c45ce0 0x561213c50060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:37.587 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.585+0000 7f26f5e00640 1 -- 172.21.15.83:0/3975603083 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561213b76e10 con 0x561213c453c0 2024-04-24T09:04:37.587 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.585+0000 7f26f4a00640 1 --2- 172.21.15.83:0/3975603083 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561213c505a0 0x561213c52990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:04:37.591 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.586+0000 7f26ee000640 1 --2- 172.21.15.83:0/3528365720 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f26d80a3d90 0x7f26d80a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:37.591 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.586+0000 7f26f5e00640 1 --2- 172.21.15.83:0/3975603083 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561213c453c0 0x561213c457a0 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 crypto rx=0x7f26e8008db0 tx=0x7f26e800d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35274d9a7a1586e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:37.591 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.586+0000 7f26e2600640 1 -- 172.21.15.83:0/3975603083 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e800f830 con 0x561213c453c0 2024-04-24T09:04:37.591 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.586+0000 7f26ee000640 1 -- 172.21.15.83:0/3528365720 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7f26e40f1610 con 0x7f26f0076530 2024-04-24T09:04:37.592 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.586+0000 7f26f5400640 1 --2- 172.21.15.83:0/3528365720 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f26d80a3d90 0x7f26d80a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:37.592 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.586+0000 7f26e2600640 1 -- 172.21.15.83:0/3975603083 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f26e800de20 con 0x561213c453c0 2024-04-24T09:04:37.592 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.586+0000 7f26e2600640 1 -- 172.21.15.83:0/3975603083 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e8018e30 con 0x561213c453c0 2024-04-24T09:04:37.592 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.587+0000 7f26fabd0740 1 -- 172.21.15.83:0/3975603083 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561213c453c0 msgr2=0x561213c457a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:37.592 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.587+0000 7f26f5400640 1 --2- 172.21.15.83:0/3528365720 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f26d80a3d90 0x7f26d80a6250 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x561213c46960 tx=0x7f26e402a3f0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:37.592 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.587+0000 7f26fabd0740 1 --2- 172.21.15.83:0/3975603083 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561213c453c0 0x561213c457a0 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 crypto rx=0x7f26e8008db0 tx=0x7f26e800d710 comp rx=0 tx=0).stop 2024-04-24T09:04:37.592 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.587+0000 7f26fabd0740 1 -- 172.21.15.83:0/3975603083 shutdown_connections 2024-04-24T09:04:37.592 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.587+0000 7f26fabd0740 1 --2- 172.21.15.83:0/3975603083 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561213c505a0 0x561213c52990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:37.592 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.587+0000 7f26fabd0740 1 --2- 172.21.15.83:0/3975603083 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561213c45ce0 0x561213c50060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:37.592 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.587+0000 7f26fabd0740 1 --2- 172.21.15.83:0/3975603083 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561213c453c0 0x561213c457a0 unknown :-1 s=CLOSED pgs=2007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:37.592 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.587+0000 7f26fabd0740 1 -- 172.21.15.83:0/3975603083 >> 172.21.15.83:0/3975603083 conn(0x561213c418d0 msgr2=0x561213c41cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:37.592 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.587+0000 7f26fabd0740 1 -- 172.21.15.83:0/3975603083 shutdown_connections 2024-04-24T09:04:37.592 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.587+0000 7f26fabd0740 1 -- 172.21.15.83:0/3975603083 wait complete. 2024-04-24T09:04:37.593 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.587+0000 7f26fabd0740 1 -- start start 2024-04-24T09:04:37.593 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.588+0000 7f26fabd0740 1 Processor -- start 2024-04-24T09:04:37.593 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.588+0000 7f26fabd0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561213c453c0 0x561213c4d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:37.593 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.588+0000 7f26fabd0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561213c45ce0 0x561213c4daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:37.593 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.588+0000 7f26fabd0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561213c505a0 0x561213c4dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:37.593 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.588+0000 7f26fabd0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561213c53170 con 0x561213c45ce0 2024-04-24T09:04:37.593 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.588+0000 7f26fabd0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x561213b652c0 con 0x561213c505a0 2024-04-24T09:04:37.593 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.588+0000 7f26fabd0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x561213c532f0 con 0x561213c453c0 2024-04-24T09:04:37.593 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.588+0000 7f26fabd0740 10 client.0 _ll_register_callbacks cb 0x5612138bda00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:04:37.593 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.588+0000 7f26f4a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561213c45ce0 0x561213c4daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:37.593 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.588+0000 7f26f4a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561213c45ce0 0x561213c4daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56014/0 (socket says 172.21.15.83:56014) 2024-04-24T09:04:37.593 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.588+0000 7f26f4a00640 1 -- 172.21.15.83:0/3017085363 learned_addr learned my addr 172.21.15.83:0/3017085363 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:04:37.593 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.589+0000 7f26f5e00640 1 --2- 172.21.15.83:0/3017085363 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561213c453c0 0x561213c4d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:37.593 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.589+0000 7f26f5e00640 1 --2- 172.21.15.83:0/3017085363 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561213c505a0 0x561213c4dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:37.594 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.589+0000 7f26f4a00640 1 -- 172.21.15.83:0/3017085363 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561213c453c0 msgr2=0x561213c4d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:37.594 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.589+0000 7f26f4a00640 1 --2- 172.21.15.83:0/3017085363 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561213c453c0 0x561213c4d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:37.594 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.589+0000 7f26f4a00640 1 -- 172.21.15.83:0/3017085363 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561213c505a0 msgr2=0x561213c4dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:37.594 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.589+0000 7f26f4a00640 1 --2- 172.21.15.83:0/3017085363 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561213c505a0 0x561213c4dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:37.594 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.589+0000 7f26f4a00640 1 -- 172.21.15.83:0/3017085363 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561213b75fd0 con 0x561213c45ce0 2024-04-24T09:04:37.594 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.589+0000 7f26f5e00640 1 --2- 172.21.15.83:0/3017085363 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561213c453c0 0x561213c4d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:04:37.594 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.589+0000 7f26f4a00640 1 --2- 172.21.15.83:0/3017085363 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561213c45ce0 0x561213c4daa0 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7f26dc00afd0 tx=0x7f26dc00f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e375bc69ff767018 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:37.594 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.589+0000 7f26d6a00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:04:37.594 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.590+0000 7f26d6a00640 1 -- 172.21.15.83:0/3017085363 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26dc004220 con 0x561213c45ce0 2024-04-24T09:04:37.594 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.590+0000 7f26d6a00640 1 -- 172.21.15.83:0/3017085363 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7f26dc0043c0 con 0x561213c45ce0 2024-04-24T09:04:37.594 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.590+0000 7f26fabd0740 1 -- 172.21.15.83:0/3017085363 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26e800de20 con 0x561213c45ce0 2024-04-24T09:04:37.594 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.590+0000 7f26d6a00640 1 -- 172.21.15.83:0/3017085363 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f26dc022050 con 0x561213c45ce0 2024-04-24T09:04:37.594 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.590+0000 7f26fabd0740 10 client.20184 fetch_fsmap learned FSMap version 41 2024-04-24T09:04:37.595 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.590+0000 7f26fabd0740 1 -- 172.21.15.83:0/3017085363 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5612138be210 con 0x561213c45ce0 2024-04-24T09:04:37.595 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.590+0000 7f26d6a00640 1 -- 172.21.15.83:0/3017085363 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f26e800de20 con 0x561213c45ce0 2024-04-24T09:04:37.595 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.590+0000 7f26d6a00640 1 -- 172.21.15.83:0/3017085363 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7f26dc0039c0 con 0x561213c45ce0 2024-04-24T09:04:37.595 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.591+0000 7f26fabd0740 10 client.20184 fetch_fsmap finished waiting for FSMap version 41 2024-04-24T09:04:37.595 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.591+0000 7f26fabd0740 10 client.20184 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:04:37.595 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.591+0000 7f26fabd0740 20 client.20184 populate_metadata read hostname 'smithi083' 2024-04-24T09:04:37.595 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.591+0000 7f26fabd0740 1 --2- 172.21.15.83:0/3017085363 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x561213c60720 0x561213c62b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:37.595 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.591+0000 7f26fabd0740 4 client.20184 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:04:37.595 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.591+0000 7f26fabd0740 1 -- 172.21.15.83:0/3017085363 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561213bb9320 con 0x561213c60720 2024-04-24T09:04:37.595 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.591+0000 7f26f5e00640 1 --2- 172.21.15.83:0/3017085363 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x561213c60720 0x561213c62b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:37.596 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.592+0000 7f26f5e00640 1 --2- 172.21.15.83:0/3017085363 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x561213c60720 0x561213c62b10 crc :-1 s=READY pgs=335 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81c020c8d2526417 server_cookie=5d6a733179b9073a in_seq=0 out_seq=0 2024-04-24T09:04:37.596 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.592+0000 7f26d6a00640 10 client.20184 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:04:37.596 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.594+0000 7f26d6a00640 1 -- 172.21.15.83:0/3017085363 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561213bb9320 con 0x561213c60720 2024-04-24T09:04:37.596 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.594+0000 7f26d6a00640 10 client.20184 handle_command_reply: tid=0 2024-04-24T09:04:37.597 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.594+0000 7f26fabd0740 1 client.20184 shutdown 2024-04-24T09:04:37.597 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.594+0000 7f26fabd0740 1 -- 172.21.15.83:0/3017085363 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561213c45ce0 msgr2=0x561213c4daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:37.597 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.594+0000 7f26fabd0740 1 --2- 172.21.15.83:0/3017085363 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561213c45ce0 0x561213c4daa0 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7f26dc00afd0 tx=0x7f26dc00f7f0 comp rx=0 tx=0).stop 2024-04-24T09:04:37.597 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.595+0000 7f26fabd0740 1 -- 172.21.15.83:0/3017085363 shutdown_connections 2024-04-24T09:04:37.597 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.595+0000 7f26fabd0740 1 --2- 172.21.15.83:0/3017085363 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x561213c60720 0x561213c62b10 crc :-1 s=READY pgs=335 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:37.597 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.595+0000 7f26fabd0740 1 --2- 172.21.15.83:0/3017085363 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561213c505a0 0x561213c4dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:37.597 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.595+0000 7f26fabd0740 1 --2- 172.21.15.83:0/3017085363 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561213c45ce0 0x561213c4daa0 unknown :-1 s=CLOSED pgs=2008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:37.597 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.595+0000 7f26fabd0740 1 --2- 172.21.15.83:0/3017085363 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561213c453c0 0x561213c4d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:37.597 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.595+0000 7f26d6a00640 0 client.20184 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:04:37.597 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.595+0000 7f26fabd0740 1 -- 172.21.15.83:0/3017085363 >> 172.21.15.83:0/3017085363 conn(0x561213c418d0 msgr2=0x561213c47650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:37.597 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.595+0000 7f26fabd0740 1 -- 172.21.15.83:0/3017085363 shutdown_connections 2024-04-24T09:04:37.597 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.595+0000 7f26fabd0740 1 -- 172.21.15.83:0/3017085363 wait complete. 2024-04-24T09:04:37.597 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.595+0000 7f26fabd0740 20 client.20184 trim_cache size 0 max 16384 2024-04-24T09:04:37.611 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.611+0000 7f26fabd0740 1 Processor -- start 2024-04-24T09:04:37.611 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.611+0000 7f26fabd0740 1 -- start start 2024-04-24T09:04:37.612 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.611+0000 7f26fabd0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561213c453c0 0x561213c5c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:37.612 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.611+0000 7f26fabd0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561213c45ce0 0x561213c4de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:37.612 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.611+0000 7f26fabd0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561213c505a0 0x561213c68420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:37.612 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.611+0000 7f26fabd0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f26f01b5a50 con 0x561213c453c0 2024-04-24T09:04:37.612 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.611+0000 7f26fabd0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f26f01b76b0 con 0x561213c45ce0 2024-04-24T09:04:37.612 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.611+0000 7f26fabd0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f26f01b6120 con 0x561213c505a0 2024-04-24T09:04:37.612 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.612+0000 7f26f5e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561213c453c0 0x561213c5c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:37.612 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.612+0000 7f26f5e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561213c453c0 0x561213c5c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56018/0 (socket says 172.21.15.83:56018) 2024-04-24T09:04:37.612 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.612+0000 7f26f5e00640 1 -- 172.21.15.83:0/1912749079 learned_addr learned my addr 172.21.15.83:0/1912749079 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:04:37.612 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.612+0000 7f26f5e00640 1 -- 172.21.15.83:0/1912749079 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561213c505a0 msgr2=0x561213c68420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:37.613 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.612+0000 7f26f5e00640 1 --2- 172.21.15.83:0/1912749079 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561213c505a0 0x561213c68420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:37.613 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.612+0000 7f26f5e00640 1 -- 172.21.15.83:0/1912749079 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561213c45ce0 msgr2=0x561213c4de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:37.613 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.612+0000 7f26f5e00640 1 --2- 172.21.15.83:0/1912749079 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561213c45ce0 0x561213c4de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:37.613 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.612+0000 7f26f5e00640 1 -- 172.21.15.83:0/1912749079 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561213c45b10 con 0x561213c453c0 2024-04-24T09:04:37.616 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.612+0000 7f26f5e00640 1 --2- 172.21.15.83:0/1912749079 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561213c453c0 0x561213c5c080 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7f26e8008d80 tx=0x7f26e8018a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daafb3e69fdca695 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:37.616 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.613+0000 7f26e1c00640 1 -- 172.21.15.83:0/1912749079 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e800de20 con 0x561213c453c0 2024-04-24T09:04:37.616 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.613+0000 7f26e1c00640 1 -- 172.21.15.83:0/1912749079 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f26e8030070 con 0x561213c453c0 2024-04-24T09:04:37.617 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.613+0000 7f26e1c00640 1 -- 172.21.15.83:0/1912749079 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e802b1b0 con 0x561213c453c0 2024-04-24T09:04:37.617 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.613+0000 7f26fabd0740 1 -- 172.21.15.83:0/1912749079 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561213c453c0 msgr2=0x561213c5c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:37.617 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.613+0000 7f26fabd0740 1 --2- 172.21.15.83:0/1912749079 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561213c453c0 0x561213c5c080 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7f26e8008d80 tx=0x7f26e8018a10 comp rx=0 tx=0).stop 2024-04-24T09:04:37.617 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.613+0000 7f26fabd0740 1 -- 172.21.15.83:0/1912749079 shutdown_connections 2024-04-24T09:04:37.617 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.613+0000 7f26fabd0740 1 --2- 172.21.15.83:0/1912749079 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561213c505a0 0x561213c68420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:37.617 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.613+0000 7f26fabd0740 1 --2- 172.21.15.83:0/1912749079 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561213c45ce0 0x561213c4de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:37.617 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.613+0000 7f26fabd0740 1 --2- 172.21.15.83:0/1912749079 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561213c453c0 0x561213c5c080 unknown :-1 s=CLOSED pgs=2009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:37.617 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.613+0000 7f26fabd0740 1 -- 172.21.15.83:0/1912749079 >> 172.21.15.83:0/1912749079 conn(0x561213c418d0 msgr2=0x561213c46d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:37.617 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.613+0000 7f26fabd0740 1 -- 172.21.15.83:0/1912749079 shutdown_connections 2024-04-24T09:04:37.617 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.614+0000 7f26fabd0740 1 -- 172.21.15.83:0/1912749079 wait complete. 2024-04-24T09:04:37.617 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.614+0000 7f26fabd0740 1 -- start start 2024-04-24T09:04:37.617 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.614+0000 7f26fabd0740 1 Processor -- start 2024-04-24T09:04:37.617 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.614+0000 7f26fabd0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561213c453c0 0x561213c589d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:37.618 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.614+0000 7f26fabd0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561213c45ce0 0x561213c53d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:37.618 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.614+0000 7f26fabd0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561213c505a0 0x561213c542a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:37.618 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.614+0000 7f26fabd0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f26f01b6f80 con 0x561213c45ce0 2024-04-24T09:04:37.618 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.614+0000 7f26fabd0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f26f01b6850 con 0x561213c453c0 2024-04-24T09:04:37.618 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.614+0000 7f26fabd0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f26f01b3a50 con 0x561213c505a0 2024-04-24T09:04:37.618 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.615+0000 7f26fabd0740 10 client.0 _ll_register_callbacks cb 0x7f26dc00a310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:04:37.618 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.615+0000 7f26f4a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561213c45ce0 0x561213c53d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:37.618 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.615+0000 7f26f4a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561213c45ce0 0x561213c53d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56024/0 (socket says 172.21.15.83:56024) 2024-04-24T09:04:37.618 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.615+0000 7f26f4a00640 1 -- 172.21.15.83:0/3301914072 learned_addr learned my addr 172.21.15.83:0/3301914072 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:04:37.618 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.615+0000 7f26f4a00640 1 -- 172.21.15.83:0/3301914072 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561213c505a0 msgr2=0x561213c542a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:37.618 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.615+0000 7f26f4a00640 1 --2- 172.21.15.83:0/3301914072 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561213c505a0 0x561213c542a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:37.618 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.615+0000 7f26f4a00640 1 -- 172.21.15.83:0/3301914072 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561213c453c0 msgr2=0x561213c589d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:37.618 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.615+0000 7f26f4a00640 1 --2- 172.21.15.83:0/3301914072 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561213c453c0 0x561213c589d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:37.619 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.615+0000 7f26f4a00640 1 -- 172.21.15.83:0/3301914072 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56121398a620 con 0x561213c45ce0 2024-04-24T09:04:37.619 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.615+0000 7f26f4a00640 1 --2- 172.21.15.83:0/3301914072 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561213c45ce0 0x561213c53d40 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x7f26dc003f50 tx=0x7f26dc0056e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44dd199f0c071232 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:37.619 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.615+0000 7f26d6a00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:04:37.619 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.616+0000 7f26d6a00640 1 -- 172.21.15.83:0/3301914072 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26dc01f040 con 0x561213c45ce0 2024-04-24T09:04:37.619 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.616+0000 7f26d6a00640 1 -- 172.21.15.83:0/3301914072 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7f26dc0041d0 con 0x561213c45ce0 2024-04-24T09:04:37.619 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.616+0000 7f26fabd0740 1 -- 172.21.15.83:0/3301914072 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26e8030070 con 0x561213c45ce0 2024-04-24T09:04:37.619 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.616+0000 7f26d6a00640 1 -- 172.21.15.83:0/3301914072 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f26dc038050 con 0x561213c45ce0 2024-04-24T09:04:37.619 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.616+0000 7f26fabd0740 10 client.20190 fetch_fsmap learned FSMap version 41 2024-04-24T09:04:37.619 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.616+0000 7f26fabd0740 1 -- 172.21.15.83:0/3301914072 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x561213cc6b80 con 0x561213c45ce0 2024-04-24T09:04:37.619 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.616+0000 7f26d6a00640 1 -- 172.21.15.83:0/3301914072 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f26e8030070 con 0x561213c45ce0 2024-04-24T09:04:37.619 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.616+0000 7f26d6a00640 1 -- 172.21.15.83:0/3301914072 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7f26dc0119d0 con 0x561213c45ce0 2024-04-24T09:04:37.619 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.616+0000 7f26fabd0740 10 client.20190 fetch_fsmap finished waiting for FSMap version 41 2024-04-24T09:04:37.619 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.616+0000 7f26fabd0740 10 client.20190 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:04:37.619 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.616+0000 7f26fabd0740 20 client.20190 populate_metadata read hostname 'smithi083' 2024-04-24T09:04:37.619 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.616+0000 7f26fabd0740 1 --2- 172.21.15.83:0/3301914072 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x561213cc7390 0x561213cc97b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:37.620 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.616+0000 7f26fabd0740 4 client.20190 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T09:04:37.620 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.616+0000 7f26fabd0740 1 -- 172.21.15.83:0/3301914072 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561213bb3190 con 0x561213cc7390 2024-04-24T09:04:37.620 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.617+0000 7f26f5e00640 1 --2- 172.21.15.83:0/3301914072 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x561213cc7390 0x561213cc97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:37.620 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.618+0000 7f26f5e00640 1 --2- 172.21.15.83:0/3301914072 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x561213cc7390 0x561213cc97b0 crc :-1 s=READY pgs=336 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eec3a87923c86506 server_cookie=a508a35fa01bce94 in_seq=0 out_seq=0 2024-04-24T09:04:37.620 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.618+0000 7f26d6a00640 10 client.20190 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:04:37.627 INFO:tasks.workunit.client.0.smithi083.stdout: 1 74679 0.59 MB/sec execute 551 sec latency 3631.451 ms 2024-04-24T09:04:37.676 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.675+0000 7f26d6a00640 1 -- 172.21.15.83:0/3301914072 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561213bb3190 con 0x561213cc7390 2024-04-24T09:04:37.676 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.675+0000 7f26d6a00640 10 client.20190 handle_command_reply: tid=0 2024-04-24T09:04:37.676 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.675+0000 7f26fabd0740 1 client.20190 shutdown 2024-04-24T09:04:37.676 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.676+0000 7f26fabd0740 1 -- 172.21.15.83:0/3301914072 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561213c45ce0 msgr2=0x561213c53d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:37.676 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.676+0000 7f26fabd0740 1 --2- 172.21.15.83:0/3301914072 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561213c45ce0 0x561213c53d40 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x7f26dc003f50 tx=0x7f26dc0056e0 comp rx=0 tx=0).stop 2024-04-24T09:04:37.676 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.676+0000 7f26fabd0740 1 -- 172.21.15.83:0/3301914072 shutdown_connections 2024-04-24T09:04:37.677 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.676+0000 7f26fabd0740 1 --2- 172.21.15.83:0/3301914072 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x561213cc7390 0x561213cc97b0 crc :-1 s=READY pgs=336 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:37.677 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.676+0000 7f26fabd0740 1 --2- 172.21.15.83:0/3301914072 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561213c505a0 0x561213c542a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:37.677 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.676+0000 7f26fabd0740 1 --2- 172.21.15.83:0/3301914072 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561213c45ce0 0x561213c53d40 unknown :-1 s=CLOSED pgs=2010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:37.677 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.676+0000 7f26fabd0740 1 --2- 172.21.15.83:0/3301914072 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561213c453c0 0x561213c589d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:37.677 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.676+0000 7f26fabd0740 1 -- 172.21.15.83:0/3301914072 >> 172.21.15.83:0/3301914072 conn(0x561213c418d0 msgr2=0x561213c47600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:37.677 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.676+0000 7f26d6a00640 0 client.20190 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:04:37.677 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.676+0000 7f26fabd0740 1 -- 172.21.15.83:0/3301914072 shutdown_connections 2024-04-24T09:04:37.677 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.676+0000 7f26fabd0740 1 -- 172.21.15.83:0/3301914072 wait complete. 2024-04-24T09:04:37.677 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.677+0000 7f26fabd0740 20 client.20190 trim_cache size 0 max 16384 2024-04-24T09:04:37.678 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T09:04:37.678 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T09:04:37.678 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T09:04:37.678 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T09:04:37.679 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.678+0000 7f26f6800640 1 -- 172.21.15.83:0/3528365720 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f26d80a3d90 msgr2=0x7f26d80a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:37.679 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.678+0000 7f26f6800640 1 --2- 172.21.15.83:0/3528365720 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f26d80a3d90 0x7f26d80a6250 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x561213c46960 tx=0x7f26e402a3f0 comp rx=0 tx=0).stop 2024-04-24T09:04:37.679 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.678+0000 7f26f6800640 1 -- 172.21.15.83:0/3528365720 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26f0076530 msgr2=0x7f26f01a4850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:37.679 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.678+0000 7f26f6800640 1 --2- 172.21.15.83:0/3528365720 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26f0076530 0x7f26f01a4850 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7f26e4006010 tx=0x7f26e4003d50 comp rx=0 tx=0).stop 2024-04-24T09:04:37.679 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.678+0000 7f26f6800640 1 -- 172.21.15.83:0/3528365720 shutdown_connections 2024-04-24T09:04:37.679 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.678+0000 7f26f6800640 1 --2- 172.21.15.83:0/3528365720 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f26d80a3d90 0x7f26d80a6250 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:37.679 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.678+0000 7f26f6800640 1 --2- 172.21.15.83:0/3528365720 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26f0112cc0 0x7f26f019ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:37.679 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.678+0000 7f26f6800640 1 --2- 172.21.15.83:0/3528365720 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f26f0076e50 0x7f26f01a4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:37.680 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.678+0000 7f26f6800640 1 --2- 172.21.15.83:0/3528365720 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26f0076530 0x7f26f01a4850 unknown :-1 s=CLOSED pgs=2006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:37.680 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.678+0000 7f26f6800640 1 -- 172.21.15.83:0/3528365720 >> 172.21.15.83:0/3528365720 conn(0x7f26f00ffb10 msgr2=0x7f26f0114310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:37.680 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.678+0000 7f26f6800640 1 -- 172.21.15.83:0/3528365720 shutdown_connections 2024-04-24T09:04:37.680 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:37.678+0000 7f26f6800640 1 -- 172.21.15.83:0/3528365720 wait complete. 2024-04-24T09:04:38.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:04:37 smithi138 ceph-mon[31441]: pgmap v409: 129 pgs: 129 active+clean; 71 MiB data, 984 MiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 198 KiB/s wr, 29 op/s 2024-04-24T09:04:38.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:04:37 smithi189 ceph-mon[31791]: pgmap v409: 129 pgs: 129 active+clean; 71 MiB data, 984 MiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 198 KiB/s wr, 29 op/s 2024-04-24T09:04:38.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:04:37 smithi083 ceph-mon[27907]: pgmap v409: 129 pgs: 129 active+clean; 71 MiB data, 984 MiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 198 KiB/s wr, 29 op/s 2024-04-24T09:04:38.628 INFO:tasks.workunit.client.0.smithi083.stdout: 1 74742 0.59 MB/sec execute 552 sec latency 4110.925 ms 2024-04-24T09:04:39.292 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T09:04:39.292 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T09:04:39.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.518+0000 7f7eb4800640 1 -- 172.21.15.83:0/3241164662 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7eac104450 msgr2=0x7f7eac104830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:39.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.518+0000 7f7eb4800640 1 --2- 172.21.15.83:0/3241164662 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7eac104450 0x7f7eac104830 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7f7ea0009900 tx=0x7f7ea002a410 comp rx=0 tx=0).stop 2024-04-24T09:04:39.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.518+0000 7f7eb4800640 1 -- 172.21.15.83:0/3241164662 shutdown_connections 2024-04-24T09:04:39.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.518+0000 7f7eb4800640 1 --2- 172.21.15.83:0/3241164662 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7eac109520 0x7f7eac1110f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:39.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.518+0000 7f7eb4800640 1 --2- 172.21.15.83:0/3241164662 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7eac104e00 0x7f7eac108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:39.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.518+0000 7f7eb4800640 1 --2- 172.21.15.83:0/3241164662 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7eac104450 0x7f7eac104830 unknown :-1 s=CLOSED pgs=2011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:39.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.518+0000 7f7eb4800640 1 -- 172.21.15.83:0/3241164662 >> 172.21.15.83:0/3241164662 conn(0x7f7eac0ffc70 msgr2=0x7f7eac102090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:39.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.518+0000 7f7eb4800640 1 -- 172.21.15.83:0/3241164662 shutdown_connections 2024-04-24T09:04:39.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.518+0000 7f7eb4800640 1 -- 172.21.15.83:0/3241164662 wait complete. 2024-04-24T09:04:39.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.520+0000 7f7eb4800640 1 Processor -- start 2024-04-24T09:04:39.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.520+0000 7f7eb4800640 1 -- start start 2024-04-24T09:04:39.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.520+0000 7f7eb4800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7eac104450 0x7f7eac1a06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:39.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.520+0000 7f7eb4800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7eac109520 0x7f7eac1a0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:39.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.520+0000 7f7eb4800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7eac19a8a0 0x7f7eac19ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:39.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.520+0000 7f7eb4800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7eac1144e0 con 0x7f7eac109520 2024-04-24T09:04:39.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.520+0000 7f7eb4800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7eac114360 con 0x7f7eac19a8a0 2024-04-24T09:04:39.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.520+0000 7f7eb4800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7eac114660 con 0x7f7eac104450 2024-04-24T09:04:39.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.521+0000 7f7eb1400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7eac109520 0x7f7eac1a0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:39.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.521+0000 7f7eb1400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7eac109520 0x7f7eac1a0c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49904/0 (socket says 172.21.15.83:49904) 2024-04-24T09:04:39.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.521+0000 7f7eb1400640 1 -- 172.21.15.83:0/395210090 learned_addr learned my addr 172.21.15.83:0/395210090 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:04:39.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.521+0000 7f7eb1e00640 1 --2- 172.21.15.83:0/395210090 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7eac104450 0x7f7eac1a06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:39.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.521+0000 7f7eb2800640 1 --2- 172.21.15.83:0/395210090 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7eac19a8a0 0x7f7eac19ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:39.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.521+0000 7f7eb1400640 1 -- 172.21.15.83:0/395210090 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7eac104450 msgr2=0x7f7eac1a06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:39.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.521+0000 7f7eb1400640 1 --2- 172.21.15.83:0/395210090 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7eac104450 0x7f7eac1a06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:39.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.521+0000 7f7eb1400640 1 -- 172.21.15.83:0/395210090 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7eac19a8a0 msgr2=0x7f7eac19ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:39.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.521+0000 7f7eb1400640 1 --2- 172.21.15.83:0/395210090 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7eac19a8a0 0x7f7eac19ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:39.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.521+0000 7f7eb1400640 1 -- 172.21.15.83:0/395210090 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7eac19b3c0 con 0x7f7eac109520 2024-04-24T09:04:39.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.521+0000 7f7eb2800640 1 --2- 172.21.15.83:0/395210090 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7eac19a8a0 0x7f7eac19ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:04:39.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.522+0000 7f7eb1400640 1 --2- 172.21.15.83:0/395210090 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7eac109520 0x7f7eac1a0c30 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c00d770 tx=0x7f7e9c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:39.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.522+0000 7f7ea6a00640 1 -- 172.21.15.83:0/395210090 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e9c017070 con 0x7f7eac109520 2024-04-24T09:04:39.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.522+0000 7f7eb4800640 1 -- 172.21.15.83:0/395210090 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7eac19b6b0 con 0x7f7eac109520 2024-04-24T09:04:39.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.522+0000 7f7ea6a00640 1 -- 172.21.15.83:0/395210090 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7e9c009c30 con 0x7f7eac109520 2024-04-24T09:04:39.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.522+0000 7f7ea6a00640 1 -- 172.21.15.83:0/395210090 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e9c00f620 con 0x7f7eac109520 2024-04-24T09:04:39.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.522+0000 7f7eb4800640 1 -- 172.21.15.83:0/395210090 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7eac1a7950 con 0x7f7eac109520 2024-04-24T09:04:39.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.525+0000 7f7eb77b0740 1 Processor -- start 2024-04-24T09:04:39.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.525+0000 7f7eb77b0740 1 -- start start 2024-04-24T09:04:39.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.526+0000 7f7ea6a00640 1 -- 172.21.15.83:0/395210090 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7e9c01a070 con 0x7f7eac109520 2024-04-24T09:04:39.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.526+0000 7f7eb77b0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56436ed25860 0x56436ed25c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:39.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.526+0000 7f7eb77b0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56436ed26180 0x56436ed30310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:39.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.526+0000 7f7eb77b0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56436ed30850 0x56436ed32c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:39.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.526+0000 7f7eb77b0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56436ec8b1f0 con 0x56436ed26180 2024-04-24T09:04:39.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.526+0000 7f7eb77b0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56436ec31350 con 0x56436ed30850 2024-04-24T09:04:39.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.526+0000 7f7eb77b0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56436ec871b0 con 0x56436ed25860 2024-04-24T09:04:39.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.526+0000 7f7eb1e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56436ed26180 0x56436ed30310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:39.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.526+0000 7f7eb1e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56436ed26180 0x56436ed30310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49916/0 (socket says 172.21.15.83:49916) 2024-04-24T09:04:39.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.526+0000 7f7eb1e00640 1 -- 172.21.15.83:0/4246818705 learned_addr learned my addr 172.21.15.83:0/4246818705 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:04:39.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.526+0000 7f7eb2800640 1 --2- 172.21.15.83:0/4246818705 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56436ed25860 0x56436ed25c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:39.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.527+0000 7f7eb1400640 1 --2- 172.21.15.83:0/4246818705 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56436ed30850 0x56436ed32c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:39.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.527+0000 7f7eb1e00640 1 -- 172.21.15.83:0/4246818705 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56436ed25860 msgr2=0x56436ed25c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:39.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.527+0000 7f7eb1e00640 1 --2- 172.21.15.83:0/4246818705 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56436ed25860 0x56436ed25c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:39.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.527+0000 7f7eb1e00640 1 -- 172.21.15.83:0/4246818705 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56436ed30850 msgr2=0x56436ed32c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:39.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.527+0000 7f7eb1e00640 1 --2- 172.21.15.83:0/4246818705 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56436ed30850 0x56436ed32c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:39.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.527+0000 7f7eb1e00640 1 -- 172.21.15.83:0/4246818705 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56436ec85f50 con 0x56436ed26180 2024-04-24T09:04:39.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.527+0000 7f7eb1400640 1 --2- 172.21.15.83:0/4246818705 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56436ed30850 0x56436ed32c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:04:39.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.527+0000 7f7eb1e00640 1 --2- 172.21.15.83:0/4246818705 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56436ed26180 0x56436ed30310 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7f7ea0031440 tx=0x7f7ea0004180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54e808e647a1e3d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:39.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.527+0000 7f7ea6a00640 1 --2- 172.21.15.83:0/395210090 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f7e940a3da0 0x7f7e940a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:39.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.527+0000 7f7e9b000640 1 -- 172.21.15.83:0/4246818705 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea0035030 con 0x56436ed26180 2024-04-24T09:04:39.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.527+0000 7f7e9b000640 1 -- 172.21.15.83:0/4246818705 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7ea00054f0 con 0x56436ed26180 2024-04-24T09:04:39.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.527+0000 7f7e9b000640 1 -- 172.21.15.83:0/4246818705 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea00035f0 con 0x56436ed26180 2024-04-24T09:04:39.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.528+0000 7f7ea6a00640 1 -- 172.21.15.83:0/395210090 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7f7e9c0d0770 con 0x7f7eac109520 2024-04-24T09:04:39.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.528+0000 7f7eb2800640 1 --2- 172.21.15.83:0/395210090 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f7e940a3da0 0x7f7e940a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:39.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.528+0000 7f7eb77b0740 1 -- 172.21.15.83:0/4246818705 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56436ed26180 msgr2=0x56436ed30310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:39.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.528+0000 7f7eb77b0740 1 --2- 172.21.15.83:0/4246818705 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56436ed26180 0x56436ed30310 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7f7ea0031440 tx=0x7f7ea0004180 comp rx=0 tx=0).stop 2024-04-24T09:04:39.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.528+0000 7f7eb2800640 1 --2- 172.21.15.83:0/395210090 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f7e940a3da0 0x7f7e940a6260 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f7ea8005fd0 tx=0x7f7ea8009530 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:39.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.528+0000 7f7eb77b0740 1 -- 172.21.15.83:0/4246818705 shutdown_connections 2024-04-24T09:04:39.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.528+0000 7f7eb77b0740 1 --2- 172.21.15.83:0/4246818705 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56436ed30850 0x56436ed32c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:39.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.528+0000 7f7eb77b0740 1 --2- 172.21.15.83:0/4246818705 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56436ed26180 0x56436ed30310 unknown :-1 s=CLOSED pgs=2013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:39.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.528+0000 7f7eb77b0740 1 --2- 172.21.15.83:0/4246818705 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56436ed25860 0x56436ed25c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:39.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.528+0000 7f7eb77b0740 1 -- 172.21.15.83:0/4246818705 >> 172.21.15.83:0/4246818705 conn(0x56436ed21b80 msgr2=0x56436ed21f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:39.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.529+0000 7f7eb77b0740 1 -- 172.21.15.83:0/4246818705 shutdown_connections 2024-04-24T09:04:39.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.529+0000 7f7eb77b0740 1 -- 172.21.15.83:0/4246818705 wait complete. 2024-04-24T09:04:39.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.529+0000 7f7eb77b0740 1 -- start start 2024-04-24T09:04:39.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.529+0000 7f7eb77b0740 1 Processor -- start 2024-04-24T09:04:39.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.530+0000 7f7eb77b0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56436ed25860 0x56436ed2d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:39.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.530+0000 7f7eb77b0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56436ed26180 0x56436ed2dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:39.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.530+0000 7f7eb77b0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56436ed30850 0x56436ed2e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:39.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.530+0000 7f7eb77b0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56436ed335e0 con 0x56436ed25860 2024-04-24T09:04:39.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.530+0000 7f7eb77b0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56436ec44690 con 0x56436ed30850 2024-04-24T09:04:39.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.530+0000 7f7eb77b0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56436ed33760 con 0x56436ed26180 2024-04-24T09:04:39.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.530+0000 7f7eb77b0740 10 client.0 _ll_register_callbacks cb 0x56436ec80650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:04:39.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.530+0000 7f7eb1e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56436ed25860 0x56436ed2d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:39.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.530+0000 7f7eb1e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56436ed25860 0x56436ed2d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49920/0 (socket says 172.21.15.83:49920) 2024-04-24T09:04:39.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.530+0000 7f7eb1e00640 1 -- 172.21.15.83:0/3449248539 learned_addr learned my addr 172.21.15.83:0/3449248539 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:04:39.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.530+0000 7f7eb1400640 1 --2- 172.21.15.83:0/3449248539 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56436ed26180 0x56436ed2dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:39.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.530+0000 7f7eb2800640 1 --2- 172.21.15.83:0/3449248539 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56436ed30850 0x56436ed2e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:39.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.530+0000 7f7eb1e00640 1 -- 172.21.15.83:0/3449248539 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56436ed26180 msgr2=0x56436ed2dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:39.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.530+0000 7f7eb1e00640 1 --2- 172.21.15.83:0/3449248539 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56436ed26180 0x56436ed2dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:39.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.530+0000 7f7eb1e00640 1 -- 172.21.15.83:0/3449248539 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56436ed30850 msgr2=0x56436ed2e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:39.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.530+0000 7f7eb1e00640 1 --2- 172.21.15.83:0/3449248539 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56436ed30850 0x56436ed2e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:39.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.530+0000 7f7eb1e00640 1 -- 172.21.15.83:0/3449248539 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56436ec43920 con 0x56436ed25860 2024-04-24T09:04:39.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.531+0000 7f7eb1e00640 1 --2- 172.21.15.83:0/3449248539 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56436ed25860 0x56436ed2d700 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7f7ea002a450 tx=0x7f7ea0004c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2e740926ce6147d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:39.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.531+0000 7f7e8f400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:04:39.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.531+0000 7f7e8f400640 1 -- 172.21.15.83:0/3449248539 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea0035070 con 0x56436ed25860 2024-04-24T09:04:39.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.531+0000 7f7e8f400640 1 -- 172.21.15.83:0/3449248539 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7f7ea0005070 con 0x56436ed25860 2024-04-24T09:04:39.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.531+0000 7f7eb77b0740 1 -- 172.21.15.83:0/3449248539 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ea00054f0 con 0x56436ed25860 2024-04-24T09:04:39.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.531+0000 7f7e8f400640 1 -- 172.21.15.83:0/3449248539 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f7ea0005270 con 0x56436ed25860 2024-04-24T09:04:39.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.531+0000 7f7eb77b0740 10 client.20202 fetch_fsmap learned FSMap version 41 2024-04-24T09:04:39.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.531+0000 7f7eb77b0740 1 -- 172.21.15.83:0/3449248539 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x56436ea677d0 con 0x56436ed25860 2024-04-24T09:04:39.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.532+0000 7f7e8f400640 1 -- 172.21.15.83:0/3449248539 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7ea00054f0 con 0x56436ed25860 2024-04-24T09:04:39.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.532+0000 7f7e8f400640 1 -- 172.21.15.83:0/3449248539 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7f7ea0031700 con 0x56436ed25860 2024-04-24T09:04:39.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.532+0000 7f7eb77b0740 10 client.20202 fetch_fsmap finished waiting for FSMap version 41 2024-04-24T09:04:39.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.532+0000 7f7eb77b0740 10 client.20202 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:04:39.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.532+0000 7f7eb77b0740 20 client.20202 populate_metadata read hostname 'smithi083' 2024-04-24T09:04:39.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.532+0000 7f7eb77b0740 1 --2- 172.21.15.83:0/3449248539 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56436ed408c0 0x56436ed42cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:39.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.532+0000 7f7eb77b0740 4 client.20202 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:04:39.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.532+0000 7f7eb77b0740 1 -- 172.21.15.83:0/3449248539 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56436ec98c10 con 0x56436ed408c0 2024-04-24T09:04:39.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.533+0000 7f7eb1400640 1 --2- 172.21.15.83:0/3449248539 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56436ed408c0 0x56436ed42cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:39.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.533+0000 7f7eb1400640 1 --2- 172.21.15.83:0/3449248539 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56436ed408c0 0x56436ed42cb0 crc :-1 s=READY pgs=337 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea1469f79c240a49 server_cookie=472749baac170efe in_seq=0 out_seq=0 2024-04-24T09:04:39.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.533+0000 7f7e8f400640 10 client.20202 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:04:39.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.536+0000 7f7e8f400640 1 -- 172.21.15.83:0/3449248539 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56436ec98c10 con 0x56436ed408c0 2024-04-24T09:04:39.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.536+0000 7f7e8f400640 10 client.20202 handle_command_reply: tid=0 2024-04-24T09:04:39.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.536+0000 7f7eb77b0740 1 client.20202 shutdown 2024-04-24T09:04:39.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.537+0000 7f7eb77b0740 1 -- 172.21.15.83:0/3449248539 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56436ed25860 msgr2=0x56436ed2d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:39.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.537+0000 7f7eb77b0740 1 --2- 172.21.15.83:0/3449248539 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56436ed25860 0x56436ed2d700 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7f7ea002a450 tx=0x7f7ea0004c70 comp rx=0 tx=0).stop 2024-04-24T09:04:39.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.537+0000 7f7eb77b0740 1 -- 172.21.15.83:0/3449248539 shutdown_connections 2024-04-24T09:04:39.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.537+0000 7f7eb77b0740 1 --2- 172.21.15.83:0/3449248539 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56436ed408c0 0x56436ed42cb0 crc :-1 s=READY pgs=337 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:39.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.537+0000 7f7eb77b0740 1 --2- 172.21.15.83:0/3449248539 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56436ed30850 0x56436ed2e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:39.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.537+0000 7f7eb77b0740 1 --2- 172.21.15.83:0/3449248539 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56436ed26180 0x56436ed2dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:39.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.537+0000 7f7eb77b0740 1 --2- 172.21.15.83:0/3449248539 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56436ed25860 0x56436ed2d700 unknown :-1 s=CLOSED pgs=2014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:39.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.537+0000 7f7eb77b0740 1 -- 172.21.15.83:0/3449248539 >> 172.21.15.83:0/3449248539 conn(0x56436ed21b80 msgr2=0x56436ed27880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:39.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.537+0000 7f7e8f400640 0 client.20202 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:04:39.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.537+0000 7f7eb77b0740 1 -- 172.21.15.83:0/3449248539 shutdown_connections 2024-04-24T09:04:39.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.537+0000 7f7eb77b0740 1 -- 172.21.15.83:0/3449248539 wait complete. 2024-04-24T09:04:39.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.537+0000 7f7eb77b0740 20 client.20202 trim_cache size 0 max 16384 2024-04-24T09:04:39.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.557+0000 7f7eb77b0740 1 Processor -- start 2024-04-24T09:04:39.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.557+0000 7f7eb77b0740 1 -- start start 2024-04-24T09:04:39.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.558+0000 7f7eb77b0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56436ed25860 0x56436ed3c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:39.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.558+0000 7f7eb77b0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56436ed26180 0x56436ed27fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:39.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.558+0000 7f7eb77b0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56436ed30850 0x56436ed2c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:39.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.558+0000 7f7eb77b0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7eac1b1900 con 0x56436ed25860 2024-04-24T09:04:39.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.558+0000 7f7eb77b0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7eac1b3560 con 0x56436ed26180 2024-04-24T09:04:39.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.558+0000 7f7eb77b0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7eac1b1fd0 con 0x56436ed30850 2024-04-24T09:04:39.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.558+0000 7f7eb1e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56436ed25860 0x56436ed3c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:39.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.558+0000 7f7eb1e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56436ed25860 0x56436ed3c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49924/0 (socket says 172.21.15.83:49924) 2024-04-24T09:04:39.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.558+0000 7f7eb1e00640 1 -- 172.21.15.83:0/3429531517 learned_addr learned my addr 172.21.15.83:0/3429531517 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:04:39.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.558+0000 7f7eb2800640 1 --2- 172.21.15.83:0/3429531517 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56436ed30850 0x56436ed2c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:39.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.558+0000 7f7eb1400640 1 --2- 172.21.15.83:0/3429531517 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56436ed26180 0x56436ed27fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:39.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.558+0000 7f7eb1e00640 1 -- 172.21.15.83:0/3429531517 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56436ed30850 msgr2=0x56436ed2c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:39.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.558+0000 7f7eb1e00640 1 --2- 172.21.15.83:0/3429531517 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56436ed30850 0x56436ed2c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:39.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.559+0000 7f7eb1e00640 1 -- 172.21.15.83:0/3429531517 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56436ed26180 msgr2=0x56436ed27fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:39.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.559+0000 7f7eb1e00640 1 --2- 172.21.15.83:0/3429531517 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56436ed26180 0x56436ed27fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:39.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.559+0000 7f7eb1e00640 1 -- 172.21.15.83:0/3429531517 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56436e99d3f0 con 0x56436ed25860 2024-04-24T09:04:39.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.559+0000 7f7eb2800640 1 --2- 172.21.15.83:0/3429531517 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56436ed30850 0x56436ed2c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:04:39.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.559+0000 7f7eb1400640 1 --2- 172.21.15.83:0/3429531517 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56436ed26180 0x56436ed27fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:04:39.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.559+0000 7f7eb1e00640 1 --2- 172.21.15.83:0/3429531517 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56436ed25860 0x56436ed3c4d0 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x7f7ea0031440 tx=0x7f7ea0004180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40d980960bad367a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:39.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.559+0000 7f7e9a600640 1 -- 172.21.15.83:0/3429531517 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea0035070 con 0x56436ed25860 2024-04-24T09:04:39.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.559+0000 7f7e9a600640 1 -- 172.21.15.83:0/3429531517 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7ea0005070 con 0x56436ed25860 2024-04-24T09:04:39.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.559+0000 7f7e9a600640 1 -- 172.21.15.83:0/3429531517 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea00038e0 con 0x56436ed25860 2024-04-24T09:04:39.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.560+0000 7f7eb77b0740 1 -- 172.21.15.83:0/3429531517 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56436ed25860 msgr2=0x56436ed3c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:39.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.560+0000 7f7eb77b0740 1 --2- 172.21.15.83:0/3429531517 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56436ed25860 0x56436ed3c4d0 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x7f7ea0031440 tx=0x7f7ea0004180 comp rx=0 tx=0).stop 2024-04-24T09:04:39.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.560+0000 7f7eb77b0740 1 -- 172.21.15.83:0/3429531517 shutdown_connections 2024-04-24T09:04:39.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.560+0000 7f7eb77b0740 1 --2- 172.21.15.83:0/3429531517 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56436ed30850 0x56436ed2c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:39.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.560+0000 7f7eb77b0740 1 --2- 172.21.15.83:0/3429531517 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56436ed26180 0x56436ed27fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:39.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.560+0000 7f7eb77b0740 1 --2- 172.21.15.83:0/3429531517 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56436ed25860 0x56436ed3c4d0 unknown :-1 s=CLOSED pgs=2015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:39.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.560+0000 7f7eb77b0740 1 -- 172.21.15.83:0/3429531517 >> 172.21.15.83:0/3429531517 conn(0x56436ed21b80 msgr2=0x56436ed20390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:39.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.560+0000 7f7eb77b0740 1 -- 172.21.15.83:0/3429531517 shutdown_connections 2024-04-24T09:04:39.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.561+0000 7f7eb77b0740 1 -- 172.21.15.83:0/3429531517 wait complete. 2024-04-24T09:04:39.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.561+0000 7f7eb77b0740 1 -- start start 2024-04-24T09:04:39.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.561+0000 7f7eb77b0740 1 Processor -- start 2024-04-24T09:04:39.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.561+0000 7f7eb77b0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56436ed25860 0x56436ed38d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:39.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.561+0000 7f7eb77b0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56436ed26180 0x56436ed340a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:39.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.562+0000 7f7eb77b0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56436ed30850 0x56436ed345e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:39.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.562+0000 7f7eb77b0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7eac1b2e30 con 0x56436ed26180 2024-04-24T09:04:39.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.562+0000 7f7eb77b0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7eac1b2700 con 0x56436ed25860 2024-04-24T09:04:39.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.562+0000 7f7eb77b0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7eac1af900 con 0x56436ed30850 2024-04-24T09:04:39.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.562+0000 7f7eb77b0740 10 client.0 _ll_register_callbacks cb 0x7f7ea00040d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:04:39.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.562+0000 7f7eb1400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56436ed26180 0x56436ed340a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:39.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.562+0000 7f7eb1400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56436ed26180 0x56436ed340a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49934/0 (socket says 172.21.15.83:49934) 2024-04-24T09:04:39.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.562+0000 7f7eb1400640 1 -- 172.21.15.83:0/1698919242 learned_addr learned my addr 172.21.15.83:0/1698919242 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:04:39.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.562+0000 7f7eb1e00640 1 --2- 172.21.15.83:0/1698919242 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56436ed25860 0x56436ed38d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:39.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.562+0000 7f7eb2800640 1 --2- 172.21.15.83:0/1698919242 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56436ed30850 0x56436ed345e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:39.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.562+0000 7f7eb1400640 1 -- 172.21.15.83:0/1698919242 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56436ed30850 msgr2=0x56436ed345e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:39.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.562+0000 7f7eb1400640 1 --2- 172.21.15.83:0/1698919242 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56436ed30850 0x56436ed345e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:39.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.562+0000 7f7eb1400640 1 -- 172.21.15.83:0/1698919242 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56436ed25860 msgr2=0x56436ed38d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:39.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.562+0000 7f7eb1400640 1 --2- 172.21.15.83:0/1698919242 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56436ed25860 0x56436ed38d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:39.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.562+0000 7f7eb1400640 1 -- 172.21.15.83:0/1698919242 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56436e95d0f0 con 0x56436ed26180 2024-04-24T09:04:39.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.563+0000 7f7eb1400640 1 --2- 172.21.15.83:0/1698919242 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56436ed26180 0x56436ed340a0 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c098cc0 tx=0x7f7e9c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd633fde4862c565 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:39.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.563+0000 7f7e8f400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:04:39.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.563+0000 7f7e8f400640 1 -- 172.21.15.83:0/1698919242 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e9c021040 con 0x56436ed26180 2024-04-24T09:04:39.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.563+0000 7f7e8f400640 1 -- 172.21.15.83:0/1698919242 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7f7e9c098960 con 0x56436ed26180 2024-04-24T09:04:39.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.563+0000 7f7eb77b0740 1 -- 172.21.15.83:0/1698919242 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ea0005070 con 0x56436ed26180 2024-04-24T09:04:39.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.563+0000 7f7e8f400640 1 -- 172.21.15.83:0/1698919242 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f7e9c096e60 con 0x56436ed26180 2024-04-24T09:04:39.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.563+0000 7f7eb77b0740 10 client.20208 fetch_fsmap learned FSMap version 41 2024-04-24T09:04:39.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.563+0000 7f7eb77b0740 1 -- 172.21.15.83:0/1698919242 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x56436ec50810 con 0x56436ed26180 2024-04-24T09:04:39.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.563+0000 7f7e8f400640 1 -- 172.21.15.83:0/1698919242 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7ea0005070 con 0x56436ed26180 2024-04-24T09:04:39.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.564+0000 7f7e8f400640 1 -- 172.21.15.83:0/1698919242 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7f7e9c0d0360 con 0x56436ed26180 2024-04-24T09:04:39.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.564+0000 7f7eb77b0740 10 client.20208 fetch_fsmap finished waiting for FSMap version 41 2024-04-24T09:04:39.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.564+0000 7f7eb77b0740 10 client.20208 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:04:39.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.564+0000 7f7eb77b0740 20 client.20208 populate_metadata read hostname 'smithi083' 2024-04-24T09:04:39.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.564+0000 7f7eb77b0740 1 --2- 172.21.15.83:0/1698919242 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56436eda6ee0 0x56436eda9300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:39.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.564+0000 7f7eb77b0740 4 client.20208 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T09:04:39.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.564+0000 7f7eb77b0740 1 -- 172.21.15.83:0/1698919242 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56436ec92a80 con 0x56436eda6ee0 2024-04-24T09:04:39.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.564+0000 7f7eb1e00640 1 --2- 172.21.15.83:0/1698919242 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56436eda6ee0 0x56436eda9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:39.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.565+0000 7f7eb1e00640 1 --2- 172.21.15.83:0/1698919242 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56436eda6ee0 0x56436eda9300 crc :-1 s=READY pgs=338 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b86c4fccbc27c29f server_cookie=515b3854b232dff9 in_seq=0 out_seq=0 2024-04-24T09:04:39.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.565+0000 7f7e8f400640 10 client.20208 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:04:39.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.569+0000 7f7e8f400640 1 -- 172.21.15.83:0/1698919242 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56436ec92a80 con 0x56436eda6ee0 2024-04-24T09:04:39.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.569+0000 7f7e8f400640 10 client.20208 handle_command_reply: tid=0 2024-04-24T09:04:39.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.569+0000 7f7eb77b0740 1 client.20208 shutdown 2024-04-24T09:04:39.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.569+0000 7f7eb77b0740 1 -- 172.21.15.83:0/1698919242 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56436ed26180 msgr2=0x56436ed340a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:39.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.569+0000 7f7eb77b0740 1 --2- 172.21.15.83:0/1698919242 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56436ed26180 0x56436ed340a0 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c098cc0 tx=0x7f7e9c0057f0 comp rx=0 tx=0).stop 2024-04-24T09:04:39.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.569+0000 7f7eb77b0740 1 -- 172.21.15.83:0/1698919242 shutdown_connections 2024-04-24T09:04:39.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.569+0000 7f7eb77b0740 1 --2- 172.21.15.83:0/1698919242 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56436eda6ee0 0x56436eda9300 crc :-1 s=READY pgs=338 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:39.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.569+0000 7f7eb77b0740 1 --2- 172.21.15.83:0/1698919242 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56436ed30850 0x56436ed345e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:39.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.569+0000 7f7eb77b0740 1 --2- 172.21.15.83:0/1698919242 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56436ed26180 0x56436ed340a0 unknown :-1 s=CLOSED pgs=2016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:39.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.569+0000 7f7eb77b0740 1 --2- 172.21.15.83:0/1698919242 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56436ed25860 0x56436ed38d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:39.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.569+0000 7f7eb77b0740 1 -- 172.21.15.83:0/1698919242 >> 172.21.15.83:0/1698919242 conn(0x56436ed21b80 msgr2=0x56436ed20390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:39.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.569+0000 7f7e8f400640 0 client.20208 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:04:39.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.569+0000 7f7eb77b0740 1 -- 172.21.15.83:0/1698919242 shutdown_connections 2024-04-24T09:04:39.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.569+0000 7f7eb77b0740 1 -- 172.21.15.83:0/1698919242 wait complete. 2024-04-24T09:04:39.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.569+0000 7f7eb77b0740 20 client.20208 trim_cache size 0 max 16384 2024-04-24T09:04:39.572 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T09:04:39.572 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-04-24T09:04:39.572 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-04-24T09:04:39.573 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T09:04:39.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.570+0000 7f7eb4800640 1 -- 172.21.15.83:0/395210090 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f7e940a3da0 msgr2=0x7f7e940a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:39.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.570+0000 7f7eb4800640 1 --2- 172.21.15.83:0/395210090 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f7e940a3da0 0x7f7e940a6260 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f7ea8005fd0 tx=0x7f7ea8009530 comp rx=0 tx=0).stop 2024-04-24T09:04:39.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.570+0000 7f7eb4800640 1 -- 172.21.15.83:0/395210090 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7eac109520 msgr2=0x7f7eac1a0c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:39.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.570+0000 7f7eb4800640 1 --2- 172.21.15.83:0/395210090 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7eac109520 0x7f7eac1a0c30 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c00d770 tx=0x7f7e9c00dc30 comp rx=0 tx=0).stop 2024-04-24T09:04:39.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.570+0000 7f7eb4800640 1 -- 172.21.15.83:0/395210090 shutdown_connections 2024-04-24T09:04:39.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.570+0000 7f7eb4800640 1 --2- 172.21.15.83:0/395210090 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f7e940a3da0 0x7f7e940a6260 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:39.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.570+0000 7f7eb4800640 1 --2- 172.21.15.83:0/395210090 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7eac19a8a0 0x7f7eac19ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:39.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.570+0000 7f7eb4800640 1 --2- 172.21.15.83:0/395210090 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7eac109520 0x7f7eac1a0c30 unknown :-1 s=CLOSED pgs=2012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:39.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.570+0000 7f7eb4800640 1 --2- 172.21.15.83:0/395210090 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7eac104450 0x7f7eac1a06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:39.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.570+0000 7f7eb4800640 1 -- 172.21.15.83:0/395210090 >> 172.21.15.83:0/395210090 conn(0x7f7eac0ffc70 msgr2=0x7f7eac101090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:39.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.570+0000 7f7eb4800640 1 -- 172.21.15.83:0/395210090 shutdown_connections 2024-04-24T09:04:39.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.570+0000 7f7eb4800640 1 -- 172.21.15.83:0/395210090 wait complete. 2024-04-24T09:04:39.589 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T09:04:39.589 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T09:04:39.628 INFO:tasks.workunit.client.0.smithi083.stdout: 1 74803 0.59 MB/sec execute 553 sec latency 48.591 ms 2024-04-24T09:04:39.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.796+0000 7f530d800640 1 -- 172.21.15.83:0/1161724166 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53081071a0 msgr2=0x7f530810ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:39.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.796+0000 7f530d800640 1 --2- 172.21.15.83:0/1161724166 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53081071a0 0x7f530810ed40 secure :-1 s=READY pgs=2017 cs=0 l=1 rev1=1 crypto rx=0x7f5300009ca0 tx=0x7f530002a8c0 comp rx=0 tx=0).stop 2024-04-24T09:04:39.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.797+0000 7f530d800640 1 -- 172.21.15.83:0/1161724166 shutdown_connections 2024-04-24T09:04:39.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.797+0000 7f530d800640 1 --2- 172.21.15.83:0/1161724166 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53081071a0 0x7f530810ed40 unknown :-1 s=CLOSED pgs=2017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:39.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.797+0000 7f530d800640 1 --2- 172.21.15.83:0/1161724166 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5308069db0 0x7f5308106a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:39.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.797+0000 7f530d800640 1 --2- 172.21.15.83:0/1161724166 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5308069400 0x7f53080697e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:39.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.797+0000 7f530d800640 1 -- 172.21.15.83:0/1161724166 >> 172.21.15.83:0/1161724166 conn(0x7f5308076840 msgr2=0x7f5308076c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:39.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.797+0000 7f530d800640 1 -- 172.21.15.83:0/1161724166 shutdown_connections 2024-04-24T09:04:39.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.797+0000 7f530d800640 1 -- 172.21.15.83:0/1161724166 wait complete. 2024-04-24T09:04:39.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.798+0000 7f530d800640 1 Processor -- start 2024-04-24T09:04:39.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.798+0000 7f530d800640 1 -- start start 2024-04-24T09:04:39.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.799+0000 7f530d800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5308069400 0x7f53081a05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:39.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.799+0000 7f530d800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53081071a0 0x7f53081a0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:39.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.799+0000 7f530d800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f530819a680 0x7f530819ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:39.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.799+0000 7f530d800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5308112110 con 0x7f53081071a0 2024-04-24T09:04:39.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.799+0000 7f530d800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5308111f90 con 0x7f530819a680 2024-04-24T09:04:39.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.799+0000 7f530d800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5308112290 con 0x7f5308069400 2024-04-24T09:04:39.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.799+0000 7f5307400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53081071a0 0x7f53081a0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:39.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.799+0000 7f5307400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53081071a0 0x7f53081a0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49942/0 (socket says 172.21.15.83:49942) 2024-04-24T09:04:39.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.799+0000 7f5307400640 1 -- 172.21.15.83:0/2664974859 learned_addr learned my addr 172.21.15.83:0/2664974859 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:04:39.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.799+0000 7f5307e00640 1 --2- 172.21.15.83:0/2664974859 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5308069400 0x7f53081a05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:39.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.799+0000 7f530ce00640 1 --2- 172.21.15.83:0/2664974859 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f530819a680 0x7f530819ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:39.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.799+0000 7f5307400640 1 -- 172.21.15.83:0/2664974859 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5308069400 msgr2=0x7f53081a05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:39.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.799+0000 7f5307400640 1 --2- 172.21.15.83:0/2664974859 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5308069400 0x7f53081a05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:39.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.799+0000 7f5307400640 1 -- 172.21.15.83:0/2664974859 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f530819a680 msgr2=0x7f530819ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:39.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.799+0000 7f5307400640 1 --2- 172.21.15.83:0/2664974859 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f530819a680 0x7f530819ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:39.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.799+0000 7f5307400640 1 -- 172.21.15.83:0/2664974859 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f530819b280 con 0x7f53081071a0 2024-04-24T09:04:39.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.800+0000 7f5307400640 1 --2- 172.21.15.83:0/2664974859 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53081071a0 0x7f53081a0b00 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7f52f400d740 tx=0x7f52f400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:39.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.800+0000 7f5304c00640 1 -- 172.21.15.83:0/2664974859 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f4017070 con 0x7f53081071a0 2024-04-24T09:04:39.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.800+0000 7f5304c00640 1 -- 172.21.15.83:0/2664974859 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f52f400de50 con 0x7f53081071a0 2024-04-24T09:04:39.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.800+0000 7f5304c00640 1 -- 172.21.15.83:0/2664974859 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f400f620 con 0x7f53081071a0 2024-04-24T09:04:39.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.800+0000 7f530d800640 1 -- 172.21.15.83:0/2664974859 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53081a7360 con 0x7f53081071a0 2024-04-24T09:04:39.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.800+0000 7f530d800640 1 -- 172.21.15.83:0/2664974859 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53081a7710 con 0x7f53081071a0 2024-04-24T09:04:39.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.801+0000 7f530d800640 1 -- 172.21.15.83:0/2664974859 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5308103550 con 0x7f53081071a0 2024-04-24T09:04:39.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.806+0000 7f5304c00640 1 -- 172.21.15.83:0/2664974859 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f52f401a070 con 0x7f53081071a0 2024-04-24T09:04:39.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.807+0000 7f5304c00640 1 --2- 172.21.15.83:0/2664974859 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f52ec0a3d20 0x7f52ec0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:39.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.808+0000 7f5304c00640 1 -- 172.21.15.83:0/2664974859 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7f52f40d06d0 con 0x7f53081071a0 2024-04-24T09:04:39.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.808+0000 7f5307e00640 1 --2- 172.21.15.83:0/2664974859 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f52ec0a3d20 0x7f52ec0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:39.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.809+0000 7f5307e00640 1 --2- 172.21.15.83:0/2664974859 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f52ec0a3d20 0x7f52ec0a61e0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f52f8005fd0 tx=0x7f52f8005850 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:39.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:39.810+0000 7f5304c00640 1 -- 172.21.15.83:0/2664974859 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f52f409c8f0 con 0x7f53081071a0 2024-04-24T09:04:40.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:04:39 smithi083 ceph-mon[27907]: pgmap v410: 129 pgs: 129 active+clean; 72 MiB data, 985 MiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 226 KiB/s wr, 39 op/s 2024-04-24T09:04:40.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.075+0000 7f530d800640 1 -- 172.21.15.83:0/2664974859 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5308068d40 con 0x7f53081071a0 2024-04-24T09:04:40.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.076+0000 7f5304c00640 1 -- 172.21.15.83:0/2664974859 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+11953 (secure 0 0 0) 0x7f52f409c1f0 con 0x7f53081071a0 2024-04-24T09:04:40.076 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T09:04:40.077 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":41,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/358872257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":358872257},{"type":"v1","addr":"172.21.15.83:6835","nonce":358872257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3157070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3157070851},{"type":"v1","addr":"172.21.15.83:6837","nonce":3157070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":41,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T08:48:29.422576+0000","modified":"2024-04-24T09:02:18.079759+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34520,"mds_1":34493,"mds_2":34451,"mds_3":34427,"mds_4":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14658":{"gid":14658,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6839/4085470844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":4085470844},{"type":"v1","addr":"172.21.15.83:6839","nonce":4085470844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14676":{"gid":14676,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6841/2634754210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":2634754210},{"type":"v1","addr":"172.21.15.83:6841","nonce":2634754210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24382":{"gid":24382,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/1859921310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1859921310},{"type":"v1","addr":"172.21.15.189:6833","nonce":1859921310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/2241923197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2241923197},{"type":"v1","addr":"172.21.15.189:6835","nonce":2241923197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.189:6839/2548965336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":2548965336},{"type":"v1","addr":"172.21.15.189:6839","nonce":2548965336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26686":{"gid":26686,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6837/1678817161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":1678817161},{"type":"v1","addr":"172.21.15.189:6837","nonce":1678817161}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.138:6833/1157967658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1157967658},{"type":"v1","addr":"172.21.15.138:6833","nonce":1157967658}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.138:6835/1909219758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1909219758},{"type":"v1","addr":"172.21.15.138:6835","nonce":1909219758}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34493":{"gid":34493,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.138:6837/900566754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":900566754},{"type":"v1","addr":"172.21.15.138:6837","nonce":900566754}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34520":{"gid":34520,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.138:6839/2236932393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2236932393},{"type":"v1","addr":"172.21.15.138:6839","nonce":2236932393}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34520,"qdb_cluster":[24454,34520,34427,34451,34493]},"id":1}]} 2024-04-24T09:04:40.077 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 41 2024-04-24T09:04:40.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.079+0000 7f530d800640 1 -- 172.21.15.83:0/2664974859 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f52ec0a3d20 msgr2=0x7f52ec0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:40.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.079+0000 7f530d800640 1 --2- 172.21.15.83:0/2664974859 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f52ec0a3d20 0x7f52ec0a61e0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f52f8005fd0 tx=0x7f52f8005850 comp rx=0 tx=0).stop 2024-04-24T09:04:40.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.079+0000 7f530d800640 1 -- 172.21.15.83:0/2664974859 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53081071a0 msgr2=0x7f53081a0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:40.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.079+0000 7f530d800640 1 --2- 172.21.15.83:0/2664974859 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53081071a0 0x7f53081a0b00 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7f52f400d740 tx=0x7f52f400dc00 comp rx=0 tx=0).stop 2024-04-24T09:04:40.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.079+0000 7f530d800640 1 -- 172.21.15.83:0/2664974859 shutdown_connections 2024-04-24T09:04:40.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.079+0000 7f530d800640 1 --2- 172.21.15.83:0/2664974859 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f52ec0a3d20 0x7f52ec0a61e0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:40.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.079+0000 7f530d800640 1 --2- 172.21.15.83:0/2664974859 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f530819a680 0x7f530819ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:40.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.079+0000 7f530d800640 1 --2- 172.21.15.83:0/2664974859 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53081071a0 0x7f53081a0b00 unknown :-1 s=CLOSED pgs=2018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:40.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.079+0000 7f530d800640 1 --2- 172.21.15.83:0/2664974859 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5308069400 0x7f53081a05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:40.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.079+0000 7f530d800640 1 -- 172.21.15.83:0/2664974859 >> 172.21.15.83:0/2664974859 conn(0x7f5308076840 msgr2=0x7f53080feef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:40.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.079+0000 7f530d800640 1 -- 172.21.15.83:0/2664974859 shutdown_connections 2024-04-24T09:04:40.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.079+0000 7f530d800640 1 -- 172.21.15.83:0/2664974859 wait complete. 2024-04-24T09:04:40.099 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-24T09:04:40.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:04:39 smithi138 ceph-mon[31441]: pgmap v410: 129 pgs: 129 active+clean; 72 MiB data, 985 MiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 226 KiB/s wr, 39 op/s 2024-04-24T09:04:40.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:04:39 smithi189 ceph-mon[31791]: pgmap v410: 129 pgs: 129 active+clean; 72 MiB data, 985 MiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 226 KiB/s wr, 39 op/s 2024-04-24T09:04:40.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.357+0000 7f1ad2e00640 1 -- 172.21.15.83:0/950408314 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1acc104e60 msgr2=0x7f1acc108ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:40.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.357+0000 7f1ad2e00640 1 --2- 172.21.15.83:0/950408314 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1acc104e60 0x7f1acc108ea0 secure :-1 s=READY pgs=2019 cs=0 l=1 rev1=1 crypto rx=0x7f1ab8009900 tx=0x7f1ab802a410 comp rx=0 tx=0).stop 2024-04-24T09:04:40.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.357+0000 7f1ad2e00640 1 -- 172.21.15.83:0/950408314 shutdown_connections 2024-04-24T09:04:40.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.357+0000 7f1ad2e00640 1 --2- 172.21.15.83:0/950408314 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1acc109580 0x7f1acc111150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:40.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.357+0000 7f1ad2e00640 1 --2- 172.21.15.83:0/950408314 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1acc104e60 0x7f1acc108ea0 unknown :-1 s=CLOSED pgs=2019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:40.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.357+0000 7f1ad2e00640 1 --2- 172.21.15.83:0/950408314 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1acc1044b0 0x7f1acc104890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:40.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.358+0000 7f1ad2e00640 1 -- 172.21.15.83:0/950408314 >> 172.21.15.83:0/950408314 conn(0x7f1acc0ffd90 msgr2=0x7f1acc1021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:40.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.358+0000 7f1ad2e00640 1 -- 172.21.15.83:0/950408314 shutdown_connections 2024-04-24T09:04:40.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.358+0000 7f1ad2e00640 1 -- 172.21.15.83:0/950408314 wait complete. 2024-04-24T09:04:40.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.359+0000 7f1ad2e00640 1 Processor -- start 2024-04-24T09:04:40.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.359+0000 7f1ad2e00640 1 -- start start 2024-04-24T09:04:40.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.359+0000 7f1ad2e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1acc1044b0 0x7f1acc1a06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:40.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.359+0000 7f1ad2e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1acc109580 0x7f1acc1a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:40.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.359+0000 7f1ad2e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1acc19a850 0x7f1acc19ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:40.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.359+0000 7f1ad2e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1acc114510 con 0x7f1acc19a850 2024-04-24T09:04:40.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.359+0000 7f1ad2e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1acc114390 con 0x7f1acc1044b0 2024-04-24T09:04:40.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.359+0000 7f1ad2e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1acc114690 con 0x7f1acc109580 2024-04-24T09:04:40.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.360+0000 7f1ad0e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1acc19a850 0x7f1acc19ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:40.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.360+0000 7f1ad0e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1acc19a850 0x7f1acc19ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49966/0 (socket says 172.21.15.83:49966) 2024-04-24T09:04:40.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.360+0000 7f1ad0e00640 1 -- 172.21.15.83:0/3208209837 learned_addr learned my addr 172.21.15.83:0/3208209837 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:04:40.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.360+0000 7f1acbe00640 1 --2- 172.21.15.83:0/3208209837 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1acc1044b0 0x7f1acc1a06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:40.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.360+0000 7f1acb400640 1 --2- 172.21.15.83:0/3208209837 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1acc109580 0x7f1acc1a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:40.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.360+0000 7f1ad0e00640 1 -- 172.21.15.83:0/3208209837 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1acc109580 msgr2=0x7f1acc1a0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:40.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.360+0000 7f1ad0e00640 1 --2- 172.21.15.83:0/3208209837 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1acc109580 0x7f1acc1a0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:40.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.360+0000 7f1ad0e00640 1 -- 172.21.15.83:0/3208209837 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1acc1044b0 msgr2=0x7f1acc1a06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:40.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.360+0000 7f1ad0e00640 1 --2- 172.21.15.83:0/3208209837 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1acc1044b0 0x7f1acc1a06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:40.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.360+0000 7f1ad0e00640 1 -- 172.21.15.83:0/3208209837 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1acc19b3e0 con 0x7f1acc19a850 2024-04-24T09:04:40.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.360+0000 7f1acbe00640 1 --2- 172.21.15.83:0/3208209837 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1acc1044b0 0x7f1acc1a06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:04:40.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.361+0000 7f1ad0e00640 1 --2- 172.21.15.83:0/3208209837 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1acc19a850 0x7f1acc19ad00 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7f1ac400eaa0 tx=0x7f1ac400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:40.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.361+0000 7f1ac8c00640 1 -- 172.21.15.83:0/3208209837 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac400cdb0 con 0x7f1acc19a850 2024-04-24T09:04:40.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.361+0000 7f1ac8c00640 1 -- 172.21.15.83:0/3208209837 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1ac4005020 con 0x7f1acc19a850 2024-04-24T09:04:40.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.361+0000 7f1ad2e00640 1 -- 172.21.15.83:0/3208209837 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1acc19b6d0 con 0x7f1acc19a850 2024-04-24T09:04:40.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.361+0000 7f1ac8c00640 1 -- 172.21.15.83:0/3208209837 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac40116d0 con 0x7f1acc19a850 2024-04-24T09:04:40.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.361+0000 7f1ad2e00640 1 -- 172.21.15.83:0/3208209837 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1acc1a7880 con 0x7f1acc19a850 2024-04-24T09:04:40.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.365+0000 7f1ac8c00640 1 -- 172.21.15.83:0/3208209837 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1ac40051d0 con 0x7f1acc19a850 2024-04-24T09:04:40.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.365+0000 7f1ad5bf4740 1 Processor -- start 2024-04-24T09:04:40.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.365+0000 7f1ad5bf4740 1 -- start start 2024-04-24T09:04:40.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.365+0000 7f1ad5bf4740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5570cca06950 0x5570cca06d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:40.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.365+0000 7f1ad5bf4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570cca07270 0x5570cca11400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:40.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.365+0000 7f1ad5bf4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570cca11940 0x5570cca13d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:40.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.365+0000 7f1ad5bf4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5570cc96c1d0 con 0x5570cca07270 2024-04-24T09:04:40.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.365+0000 7f1ad5bf4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5570cc912450 con 0x5570cca11940 2024-04-24T09:04:40.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.365+0000 7f1ad5bf4740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5570cc968190 con 0x5570cca06950 2024-04-24T09:04:40.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.366+0000 7f1acbe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570cca07270 0x5570cca11400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:40.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.366+0000 7f1acbe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570cca07270 0x5570cca11400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49968/0 (socket says 172.21.15.83:49968) 2024-04-24T09:04:40.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.366+0000 7f1acbe00640 1 -- 172.21.15.83:0/2161508828 learned_addr learned my addr 172.21.15.83:0/2161508828 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:04:40.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.366+0000 7f1acb400640 1 --2- 172.21.15.83:0/2161508828 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5570cca06950 0x5570cca06d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:40.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.366+0000 7f1acb400640 1 --2- 172.21.15.83:0/2161508828 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570cca11940 0x5570cca13d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:40.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.366+0000 7f1acbe00640 1 -- 172.21.15.83:0/2161508828 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5570cca06950 msgr2=0x5570cca06d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:40.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.366+0000 7f1acbe00640 1 --2- 172.21.15.83:0/2161508828 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5570cca06950 0x5570cca06d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:40.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.366+0000 7f1acbe00640 1 -- 172.21.15.83:0/2161508828 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570cca11940 msgr2=0x5570cca13d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:40.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.366+0000 7f1acbe00640 1 --2- 172.21.15.83:0/2161508828 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570cca11940 0x5570cca13d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:40.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.366+0000 7f1acbe00640 1 -- 172.21.15.83:0/2161508828 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570cc9cf310 con 0x5570cca07270 2024-04-24T09:04:40.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.366+0000 7f1acbe00640 1 --2- 172.21.15.83:0/2161508828 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570cca07270 0x5570cca11400 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7f1abc008c60 tx=0x7f1abc00e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2254dbb9d8f816a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:40.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.367+0000 7f1ac1600640 1 -- 172.21.15.83:0/2161508828 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1abc014070 con 0x5570cca07270 2024-04-24T09:04:40.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.367+0000 7f1ac1600640 1 -- 172.21.15.83:0/2161508828 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1abc0047a0 con 0x5570cca07270 2024-04-24T09:04:40.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.367+0000 7f1ac8c00640 1 --2- 172.21.15.83:0/3208209837 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f1ab00a3d20 0x7f1ab00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:40.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.367+0000 7f1ac1600640 1 -- 172.21.15.83:0/2161508828 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1abc010690 con 0x5570cca07270 2024-04-24T09:04:40.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.367+0000 7f1ac8c00640 1 -- 172.21.15.83:0/3208209837 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7f1ac40d0e70 con 0x7f1acc19a850 2024-04-24T09:04:40.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.367+0000 7f1acb400640 1 --2- 172.21.15.83:0/3208209837 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f1ab00a3d20 0x7f1ab00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:40.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.368+0000 7f1ad5bf4740 1 -- 172.21.15.83:0/2161508828 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570cca07270 msgr2=0x5570cca11400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:40.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.368+0000 7f1ad5bf4740 1 --2- 172.21.15.83:0/2161508828 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570cca07270 0x5570cca11400 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7f1abc008c60 tx=0x7f1abc00e720 comp rx=0 tx=0).stop 2024-04-24T09:04:40.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.368+0000 7f1ad5bf4740 1 -- 172.21.15.83:0/2161508828 shutdown_connections 2024-04-24T09:04:40.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.368+0000 7f1ad5bf4740 1 --2- 172.21.15.83:0/2161508828 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570cca11940 0x5570cca13d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:40.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.368+0000 7f1ad5bf4740 1 --2- 172.21.15.83:0/2161508828 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570cca07270 0x5570cca11400 unknown :-1 s=CLOSED pgs=2021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:40.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.368+0000 7f1ad5bf4740 1 --2- 172.21.15.83:0/2161508828 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5570cca06950 0x5570cca06d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:40.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.368+0000 7f1ad5bf4740 1 -- 172.21.15.83:0/2161508828 >> 172.21.15.83:0/2161508828 conn(0x5570cca02c70 msgr2=0x5570cca03050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:40.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.368+0000 7f1ad5bf4740 1 -- 172.21.15.83:0/2161508828 shutdown_connections 2024-04-24T09:04:40.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.368+0000 7f1ad5bf4740 1 -- 172.21.15.83:0/2161508828 wait complete. 2024-04-24T09:04:40.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.368+0000 7f1acb400640 1 --2- 172.21.15.83:0/3208209837 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f1ab00a3d20 0x7f1ab00a61e0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f1ab8006010 tx=0x7f1ab8031400 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:40.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.369+0000 7f1ad5bf4740 1 -- start start 2024-04-24T09:04:40.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.369+0000 7f1ad5bf4740 1 Processor -- start 2024-04-24T09:04:40.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.369+0000 7f1ad5bf4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570cca06950 0x5570cca11360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:40.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.369+0000 7f1ad5bf4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570cca07270 0x5570cca0e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:40.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.369+0000 7f1ad5bf4740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5570cca11940 0x5570cca0ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:40.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.369+0000 7f1ad5bf4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5570cca147d0 con 0x5570cca07270 2024-04-24T09:04:40.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.369+0000 7f1ad5bf4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5570cc925810 con 0x5570cca06950 2024-04-24T09:04:40.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.369+0000 7f1ad5bf4740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5570cca14950 con 0x5570cca11940 2024-04-24T09:04:40.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.369+0000 7f1ad5bf4740 10 client.0 _ll_register_callbacks cb 0x5570cc8f3140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:04:40.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.370+0000 7f1acb400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570cca07270 0x5570cca0e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:40.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.370+0000 7f1acb400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570cca07270 0x5570cca0e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49976/0 (socket says 172.21.15.83:49976) 2024-04-24T09:04:40.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.370+0000 7f1acb400640 1 -- 172.21.15.83:0/1305462335 learned_addr learned my addr 172.21.15.83:0/1305462335 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:04:40.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.370+0000 7f1acbe00640 1 --2- 172.21.15.83:0/1305462335 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570cca06950 0x5570cca11360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:40.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.370+0000 7f1ad0e00640 1 --2- 172.21.15.83:0/1305462335 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5570cca11940 0x5570cca0ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:40.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.370+0000 7f1acb400640 1 -- 172.21.15.83:0/1305462335 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5570cca11940 msgr2=0x5570cca0ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:40.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.370+0000 7f1acb400640 1 --2- 172.21.15.83:0/1305462335 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5570cca11940 0x5570cca0ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:40.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.370+0000 7f1acb400640 1 -- 172.21.15.83:0/1305462335 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570cca06950 msgr2=0x5570cca11360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:40.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.370+0000 7f1acb400640 1 --2- 172.21.15.83:0/1305462335 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570cca06950 0x5570cca11360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:40.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.370+0000 7f1acb400640 1 -- 172.21.15.83:0/1305462335 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570cc9363f0 con 0x5570cca07270 2024-04-24T09:04:40.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.370+0000 7f1acbe00640 1 --2- 172.21.15.83:0/1305462335 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570cca06950 0x5570cca11360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:04:40.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.371+0000 7f1acb400640 1 --2- 172.21.15.83:0/1305462335 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570cca07270 0x5570cca0e650 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7f1ab8033000 tx=0x7f1ab8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7eb129c547cf51e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:40.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.371+0000 7f1ab6000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:04:40.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.371+0000 7f1ab6000640 1 -- 172.21.15.83:0/1305462335 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab804e070 con 0x5570cca07270 2024-04-24T09:04:40.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.371+0000 7f1ab6000640 1 -- 172.21.15.83:0/1305462335 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7f1ab80035f0 con 0x5570cca07270 2024-04-24T09:04:40.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.371+0000 7f1ad5bf4740 1 -- 172.21.15.83:0/1305462335 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1abc0047a0 con 0x5570cca07270 2024-04-24T09:04:40.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.371+0000 7f1ab6000640 1 -- 172.21.15.83:0/1305462335 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f1ab805a050 con 0x5570cca07270 2024-04-24T09:04:40.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.371+0000 7f1ad5bf4740 10 client.20226 fetch_fsmap learned FSMap version 41 2024-04-24T09:04:40.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.371+0000 7f1ad5bf4740 1 -- 172.21.15.83:0/1305462335 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5570cc67e3f0 con 0x5570cca07270 2024-04-24T09:04:40.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.372+0000 7f1ab6000640 1 -- 172.21.15.83:0/1305462335 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1abc0047a0 con 0x5570cca07270 2024-04-24T09:04:40.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.372+0000 7f1ab6000640 1 -- 172.21.15.83:0/1305462335 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7f1ab804a820 con 0x5570cca07270 2024-04-24T09:04:40.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.372+0000 7f1ad5bf4740 10 client.20226 fetch_fsmap finished waiting for FSMap version 41 2024-04-24T09:04:40.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.372+0000 7f1ad5bf4740 10 client.20226 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-24T09:04:40.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.372+0000 7f1ad5bf4740 20 client.20226 populate_metadata read hostname 'smithi083' 2024-04-24T09:04:40.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.372+0000 7f1ad5bf4740 1 --2- 172.21.15.83:0/1305462335 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x5570cca21a90 0x5570cca23f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:40.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.372+0000 7f1ad5bf4740 4 client.20226 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:04:40.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.372+0000 7f1ad5bf4740 1 -- 172.21.15.83:0/1305462335 --> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5570cc979bf0 con 0x5570cca21a90 2024-04-24T09:04:40.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.373+0000 7f1acbe00640 1 --2- 172.21.15.83:0/1305462335 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x5570cca21a90 0x5570cca23f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:40.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.374+0000 7f1ab6000640 10 client.20226 ms_handle_connect on v2:172.21.15.189:6838/2548965336 2024-04-24T09:04:40.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.374+0000 7f1acbe00640 1 --2- 172.21.15.83:0/1305462335 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x5570cca21a90 0x5570cca23f50 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b9d1f78b2860bc9f server_cookie=d467977368754c40 in_seq=0 out_seq=0 2024-04-24T09:04:40.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.376+0000 7f1ab6000640 1 -- 172.21.15.83:0/1305462335 <== mds.4 v2:172.21.15.189:6838/2548965336 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5570cc979bf0 con 0x5570cca21a90 2024-04-24T09:04:40.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.376+0000 7f1ab6000640 10 client.20226 handle_command_reply: tid=0 2024-04-24T09:04:40.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.376+0000 7f1ad5bf4740 1 client.20226 shutdown 2024-04-24T09:04:40.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.377+0000 7f1ad5bf4740 1 -- 172.21.15.83:0/1305462335 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570cca07270 msgr2=0x5570cca0e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:40.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.377+0000 7f1ad5bf4740 1 --2- 172.21.15.83:0/1305462335 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570cca07270 0x5570cca0e650 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7f1ab8033000 tx=0x7f1ab8000bc0 comp rx=0 tx=0).stop 2024-04-24T09:04:40.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.377+0000 7f1ad5bf4740 1 -- 172.21.15.83:0/1305462335 shutdown_connections 2024-04-24T09:04:40.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.377+0000 7f1ad5bf4740 1 --2- 172.21.15.83:0/1305462335 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x5570cca21a90 0x5570cca23f50 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:40.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.377+0000 7f1ad5bf4740 1 --2- 172.21.15.83:0/1305462335 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5570cca11940 0x5570cca0ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:40.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.377+0000 7f1ad5bf4740 1 --2- 172.21.15.83:0/1305462335 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570cca07270 0x5570cca0e650 unknown :-1 s=CLOSED pgs=2022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:40.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.377+0000 7f1ad5bf4740 1 --2- 172.21.15.83:0/1305462335 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570cca06950 0x5570cca11360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:40.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.377+0000 7f1ab6000640 0 client.20226 ms_handle_reset on v2:172.21.15.189:6838/2548965336 2024-04-24T09:04:40.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.377+0000 7f1ad5bf4740 1 -- 172.21.15.83:0/1305462335 >> 172.21.15.83:0/1305462335 conn(0x5570cca02c70 msgr2=0x5570cca08a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:40.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.377+0000 7f1ad5bf4740 1 -- 172.21.15.83:0/1305462335 shutdown_connections 2024-04-24T09:04:40.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.377+0000 7f1ad5bf4740 1 -- 172.21.15.83:0/1305462335 wait complete. 2024-04-24T09:04:40.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.377+0000 7f1ad5bf4740 20 client.20226 trim_cache size 0 max 16384 2024-04-24T09:04:40.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.393+0000 7f1ad5bf4740 1 Processor -- start 2024-04-24T09:04:40.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.393+0000 7f1ad5bf4740 1 -- start start 2024-04-24T09:04:40.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.393+0000 7f1ad5bf4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570cca06950 0x5570cca090b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:40.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.393+0000 7f1ad5bf4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570cca07270 0x5570cca10e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:40.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.393+0000 7f1ad5bf4740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5570cca11940 0x5570cca29820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:40.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.393+0000 7f1ad5bf4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1acc1b1880 con 0x5570cca07270 2024-04-24T09:04:40.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.393+0000 7f1ad5bf4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1acc1b34e0 con 0x5570cca06950 2024-04-24T09:04:40.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.393+0000 7f1ad5bf4740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1acc1b1f50 con 0x5570cca11940 2024-04-24T09:04:40.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.393+0000 7f1acb400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570cca07270 0x5570cca10e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:40.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.393+0000 7f1acb400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570cca07270 0x5570cca10e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49992/0 (socket says 172.21.15.83:49992) 2024-04-24T09:04:40.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.393+0000 7f1acb400640 1 -- 172.21.15.83:0/1589019281 learned_addr learned my addr 172.21.15.83:0/1589019281 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:04:40.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.394+0000 7f1ad0e00640 1 --2- 172.21.15.83:0/1589019281 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5570cca11940 0x5570cca29820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:40.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.394+0000 7f1acbe00640 1 --2- 172.21.15.83:0/1589019281 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570cca06950 0x5570cca090b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:40.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.394+0000 7f1acb400640 1 -- 172.21.15.83:0/1589019281 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5570cca11940 msgr2=0x5570cca29820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:40.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.394+0000 7f1acb400640 1 --2- 172.21.15.83:0/1589019281 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5570cca11940 0x5570cca29820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:40.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.394+0000 7f1acb400640 1 -- 172.21.15.83:0/1589019281 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570cca06950 msgr2=0x5570cca090b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:40.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.394+0000 7f1acb400640 1 --2- 172.21.15.83:0/1589019281 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570cca06950 0x5570cca090b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:40.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.394+0000 7f1acb400640 1 -- 172.21.15.83:0/1589019281 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570cc7487d0 con 0x5570cca07270 2024-04-24T09:04:40.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.394+0000 7f1acbe00640 1 --2- 172.21.15.83:0/1589019281 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570cca06950 0x5570cca090b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:04:40.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.394+0000 7f1acb400640 1 --2- 172.21.15.83:0/1589019281 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570cca07270 0x5570cca10e90 secure :-1 s=READY pgs=2023 cs=0 l=1 rev1=1 crypto rx=0x7f1ab8005d00 tx=0x7f1ab80318e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a14eca9df28159b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:40.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.394+0000 7f1ac0c00640 1 -- 172.21.15.83:0/1589019281 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab804e070 con 0x5570cca07270 2024-04-24T09:04:40.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.394+0000 7f1ac0c00640 1 -- 172.21.15.83:0/1589019281 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1ab803d520 con 0x5570cca07270 2024-04-24T09:04:40.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.394+0000 7f1ac0c00640 1 -- 172.21.15.83:0/1589019281 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab8035040 con 0x5570cca07270 2024-04-24T09:04:40.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.395+0000 7f1ad5bf4740 1 -- 172.21.15.83:0/1589019281 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570cca07270 msgr2=0x5570cca10e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:40.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.395+0000 7f1ad5bf4740 1 --2- 172.21.15.83:0/1589019281 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570cca07270 0x5570cca10e90 secure :-1 s=READY pgs=2023 cs=0 l=1 rev1=1 crypto rx=0x7f1ab8005d00 tx=0x7f1ab80318e0 comp rx=0 tx=0).stop 2024-04-24T09:04:40.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.395+0000 7f1ad5bf4740 1 -- 172.21.15.83:0/1589019281 shutdown_connections 2024-04-24T09:04:40.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.395+0000 7f1ad5bf4740 1 --2- 172.21.15.83:0/1589019281 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5570cca11940 0x5570cca29820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:40.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.395+0000 7f1ad5bf4740 1 --2- 172.21.15.83:0/1589019281 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570cca07270 0x5570cca10e90 unknown :-1 s=CLOSED pgs=2023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:40.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.395+0000 7f1ad5bf4740 1 --2- 172.21.15.83:0/1589019281 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570cca06950 0x5570cca090b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:40.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.395+0000 7f1ad5bf4740 1 -- 172.21.15.83:0/1589019281 >> 172.21.15.83:0/1589019281 conn(0x5570cca02c70 msgr2=0x5570cca09a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:40.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.395+0000 7f1ad5bf4740 1 -- 172.21.15.83:0/1589019281 shutdown_connections 2024-04-24T09:04:40.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.395+0000 7f1ad5bf4740 1 -- 172.21.15.83:0/1589019281 wait complete. 2024-04-24T09:04:40.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.396+0000 7f1ad5bf4740 1 -- start start 2024-04-24T09:04:40.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.396+0000 7f1ad5bf4740 1 Processor -- start 2024-04-24T09:04:40.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.396+0000 7f1ad5bf4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570cca06950 0x5570cca19f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:40.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.396+0000 7f1ad5bf4740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5570cca07270 0x5570cca0af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:40.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.396+0000 7f1ad5bf4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570cca11940 0x5570cca0b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:40.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.396+0000 7f1ad5bf4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1acc1b2db0 con 0x5570cca06950 2024-04-24T09:04:40.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.396+0000 7f1ad5bf4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1acc1b2680 con 0x5570cca11940 2024-04-24T09:04:40.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.396+0000 7f1ad5bf4740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1acc1af880 con 0x5570cca07270 2024-04-24T09:04:40.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.397+0000 7f1ad5bf4740 10 client.0 _ll_register_callbacks cb 0x7f1ab8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:04:40.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.397+0000 7f1acbe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570cca06950 0x5570cca19f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:40.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.397+0000 7f1acbe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570cca06950 0x5570cca19f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50002/0 (socket says 172.21.15.83:50002) 2024-04-24T09:04:40.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.397+0000 7f1acbe00640 1 -- 172.21.15.83:0/2610589644 learned_addr learned my addr 172.21.15.83:0/2610589644 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:04:40.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.397+0000 7f1acbe00640 1 -- 172.21.15.83:0/2610589644 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5570cca07270 msgr2=0x5570cca0af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:40.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.397+0000 7f1acbe00640 1 --2- 172.21.15.83:0/2610589644 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5570cca07270 0x5570cca0af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:40.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.397+0000 7f1acbe00640 1 -- 172.21.15.83:0/2610589644 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570cca11940 msgr2=0x5570cca0b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:40.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.397+0000 7f1acbe00640 1 --2- 172.21.15.83:0/2610589644 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570cca11940 0x5570cca0b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:40.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.397+0000 7f1acbe00640 1 -- 172.21.15.83:0/2610589644 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570cca0bcf0 con 0x5570cca06950 2024-04-24T09:04:40.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.397+0000 7f1acbe00640 1 --2- 172.21.15.83:0/2610589644 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570cca06950 0x5570cca19f10 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 crypto rx=0x5570cca07f10 tx=0x7f1abc0085e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60e67cb47471d3ab server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:40.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.397+0000 7f1ab6000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:04:40.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.398+0000 7f1ab6000640 1 -- 172.21.15.83:0/2610589644 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1abc014040 con 0x5570cca06950 2024-04-24T09:04:40.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.398+0000 7f1ab6000640 1 -- 172.21.15.83:0/2610589644 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7f1abc004450 con 0x5570cca06950 2024-04-24T09:04:40.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.398+0000 7f1ad5bf4740 1 -- 172.21.15.83:0/2610589644 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ab803d520 con 0x5570cca06950 2024-04-24T09:04:40.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.398+0000 7f1ab6000640 1 -- 172.21.15.83:0/2610589644 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f1abc035050 con 0x5570cca06950 2024-04-24T09:04:40.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.398+0000 7f1ad5bf4740 10 client.20232 fetch_fsmap learned FSMap version 41 2024-04-24T09:04:40.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.398+0000 7f1ad5bf4740 1 -- 172.21.15.83:0/2610589644 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5570cca88090 con 0x5570cca06950 2024-04-24T09:04:40.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.398+0000 7f1ab6000640 1 -- 172.21.15.83:0/2610589644 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1ab803d520 con 0x5570cca06950 2024-04-24T09:04:40.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.399+0000 7f1ab6000640 1 -- 172.21.15.83:0/2610589644 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7f1abc025100 con 0x5570cca06950 2024-04-24T09:04:40.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.399+0000 7f1ad5bf4740 10 client.20232 fetch_fsmap finished waiting for FSMap version 41 2024-04-24T09:04:40.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.399+0000 7f1ad5bf4740 10 client.20232 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-24T09:04:40.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.399+0000 7f1ad5bf4740 20 client.20232 populate_metadata read hostname 'smithi083' 2024-04-24T09:04:40.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.399+0000 7f1ad5bf4740 1 --2- 172.21.15.83:0/2610589644 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x5570cca88460 0x5570cca8a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:40.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.399+0000 7f1ad5bf4740 4 client.20232 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:04:40.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.399+0000 7f1ad5bf4740 1 -- 172.21.15.83:0/2610589644 --> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5570cc973a60 con 0x5570cca88460 2024-04-24T09:04:40.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.399+0000 7f1acb400640 1 --2- 172.21.15.83:0/2610589644 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x5570cca88460 0x5570cca8a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:40.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.400+0000 7f1acb400640 1 --2- 172.21.15.83:0/2610589644 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x5570cca88460 0x5570cca8a8b0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e8c89529ee535d0e server_cookie=fd3657a680935f7f in_seq=0 out_seq=0 2024-04-24T09:04:40.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.400+0000 7f1ab6000640 10 client.20232 ms_handle_connect on v2:172.21.15.189:6838/2548965336 2024-04-24T09:04:40.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.401+0000 7f1ab6000640 1 -- 172.21.15.83:0/2610589644 <== mds.4 v2:172.21.15.189:6838/2548965336 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5570cc973a60 con 0x5570cca88460 2024-04-24T09:04:40.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.401+0000 7f1ab6000640 10 client.20232 handle_command_reply: tid=0 2024-04-24T09:04:40.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.401+0000 7f1ad5bf4740 1 client.20232 shutdown 2024-04-24T09:04:40.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.401+0000 7f1ad5bf4740 1 -- 172.21.15.83:0/2610589644 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570cca06950 msgr2=0x5570cca19f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:40.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.401+0000 7f1ad5bf4740 1 --2- 172.21.15.83:0/2610589644 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570cca06950 0x5570cca19f10 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 crypto rx=0x5570cca07f10 tx=0x7f1abc0085e0 comp rx=0 tx=0).stop 2024-04-24T09:04:40.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.401+0000 7f1ad5bf4740 1 -- 172.21.15.83:0/2610589644 shutdown_connections 2024-04-24T09:04:40.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.401+0000 7f1ad5bf4740 1 --2- 172.21.15.83:0/2610589644 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x5570cca88460 0x5570cca8a8b0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:40.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.401+0000 7f1ad5bf4740 1 --2- 172.21.15.83:0/2610589644 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570cca11940 0x5570cca0b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:40.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.401+0000 7f1ad5bf4740 1 --2- 172.21.15.83:0/2610589644 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5570cca07270 0x5570cca0af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:40.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.401+0000 7f1ad5bf4740 1 --2- 172.21.15.83:0/2610589644 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570cca06950 0x5570cca19f10 unknown :-1 s=CLOSED pgs=2024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:40.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.401+0000 7f1ad5bf4740 1 -- 172.21.15.83:0/2610589644 >> 172.21.15.83:0/2610589644 conn(0x5570cca02c70 msgr2=0x5570cca01560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:40.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.401+0000 7f1ab6000640 0 client.20232 ms_handle_reset on v2:172.21.15.189:6838/2548965336 2024-04-24T09:04:40.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.401+0000 7f1ad5bf4740 1 -- 172.21.15.83:0/2610589644 shutdown_connections 2024-04-24T09:04:40.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.401+0000 7f1ad5bf4740 1 -- 172.21.15.83:0/2610589644 wait complete. 2024-04-24T09:04:40.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.401+0000 7f1ad5bf4740 20 client.20232 trim_cache size 0 max 16384 2024-04-24T09:04:40.404 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:04:40.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.402+0000 7f1ad2e00640 1 -- 172.21.15.83:0/3208209837 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f1ab00a3d20 msgr2=0x7f1ab00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:40.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.402+0000 7f1ad2e00640 1 --2- 172.21.15.83:0/3208209837 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f1ab00a3d20 0x7f1ab00a61e0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f1ab8006010 tx=0x7f1ab8031400 comp rx=0 tx=0).stop 2024-04-24T09:04:40.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.402+0000 7f1ad2e00640 1 -- 172.21.15.83:0/3208209837 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1acc19a850 msgr2=0x7f1acc19ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:40.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.402+0000 7f1ad2e00640 1 --2- 172.21.15.83:0/3208209837 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1acc19a850 0x7f1acc19ad00 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7f1ac400eaa0 tx=0x7f1ac400ef60 comp rx=0 tx=0).stop 2024-04-24T09:04:40.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.402+0000 7f1ad2e00640 1 -- 172.21.15.83:0/3208209837 shutdown_connections 2024-04-24T09:04:40.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.402+0000 7f1ad2e00640 1 --2- 172.21.15.83:0/3208209837 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f1ab00a3d20 0x7f1ab00a61e0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:40.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.402+0000 7f1ad2e00640 1 --2- 172.21.15.83:0/3208209837 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1acc19a850 0x7f1acc19ad00 unknown :-1 s=CLOSED pgs=2020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:40.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.402+0000 7f1ad2e00640 1 --2- 172.21.15.83:0/3208209837 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1acc109580 0x7f1acc1a0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:40.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.402+0000 7f1ad2e00640 1 --2- 172.21.15.83:0/3208209837 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1acc1044b0 0x7f1acc1a06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:40.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.402+0000 7f1ad2e00640 1 -- 172.21.15.83:0/3208209837 >> 172.21.15.83:0/3208209837 conn(0x7f1acc0ffd90 msgr2=0x7f1acc1020c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:40.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.402+0000 7f1ad2e00640 1 -- 172.21.15.83:0/3208209837 shutdown_connections 2024-04-24T09:04:40.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.402+0000 7f1ad2e00640 1 -- 172.21.15.83:0/3208209837 wait complete. 2024-04-24T09:04:40.419 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-24T09:04:40.628 INFO:tasks.workunit.client.0.smithi083.stdout: 1 74880 0.59 MB/sec execute 554 sec latency 37.356 ms 2024-04-24T09:04:40.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.629+0000 7efeefe00640 1 -- 172.21.15.83:0/121956255 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efef0112eb0 msgr2=0x7efef01152a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:40.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.629+0000 7efeefe00640 1 --2- 172.21.15.83:0/121956255 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efef0112eb0 0x7efef01152a0 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 crypto rx=0x7efee8009ca0 tx=0x7efee802a950 comp rx=0 tx=0).stop 2024-04-24T09:04:40.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.630+0000 7efeefe00640 1 -- 172.21.15.83:0/121956255 shutdown_connections 2024-04-24T09:04:40.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.630+0000 7efeefe00640 1 --2- 172.21.15.83:0/121956255 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efef0112eb0 0x7efef01152a0 unknown :-1 s=CLOSED pgs=2025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:40.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.630+0000 7efeefe00640 1 --2- 172.21.15.83:0/121956255 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efef0074cc0 0x7efef0112970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:40.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.630+0000 7efeefe00640 1 --2- 172.21.15.83:0/121956255 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efef00743a0 0x7efef0074780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:40.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.630+0000 7efeefe00640 1 -- 172.21.15.83:0/121956255 >> 172.21.15.83:0/121956255 conn(0x7efef00fd9d0 msgr2=0x7efef00ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:40.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.630+0000 7efeefe00640 1 -- 172.21.15.83:0/121956255 shutdown_connections 2024-04-24T09:04:40.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.630+0000 7efeefe00640 1 -- 172.21.15.83:0/121956255 wait complete. 2024-04-24T09:04:40.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.631+0000 7efeefe00640 1 Processor -- start 2024-04-24T09:04:40.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.631+0000 7efeefe00640 1 -- start start 2024-04-24T09:04:40.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.632+0000 7efeefe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efef00743a0 0x7efef01a4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:40.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.632+0000 7efeefe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efef0074cc0 0x7efef01a4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:40.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.632+0000 7efeefe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efef0112eb0 0x7efef019eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:40.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.632+0000 7efeefe00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7efef0118750 con 0x7efef0112eb0 2024-04-24T09:04:40.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.632+0000 7efeefe00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efef01185d0 con 0x7efef00743a0 2024-04-24T09:04:40.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.632+0000 7efeefe00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7efef01188d0 con 0x7efef0074cc0 2024-04-24T09:04:40.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.632+0000 7efeef400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efef0112eb0 0x7efef019eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:40.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.632+0000 7efeef400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efef0112eb0 0x7efef019eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50018/0 (socket says 172.21.15.83:50018) 2024-04-24T09:04:40.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.632+0000 7efeef400640 1 -- 172.21.15.83:0/3810709995 learned_addr learned my addr 172.21.15.83:0/3810709995 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:04:40.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.632+0000 7efeee000640 1 --2- 172.21.15.83:0/3810709995 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efef0074cc0 0x7efef01a4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:40.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.633+0000 7efeef400640 1 -- 172.21.15.83:0/3810709995 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efef0074cc0 msgr2=0x7efef01a4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:40.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.633+0000 7efeef400640 1 --2- 172.21.15.83:0/3810709995 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efef0074cc0 0x7efef01a4e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:40.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.633+0000 7efeeea00640 1 --2- 172.21.15.83:0/3810709995 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efef00743a0 0x7efef01a4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:40.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.633+0000 7efeef400640 1 -- 172.21.15.83:0/3810709995 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efef00743a0 msgr2=0x7efef01a4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:40.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.633+0000 7efeef400640 1 --2- 172.21.15.83:0/3810709995 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efef00743a0 0x7efef01a4900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:40.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.633+0000 7efeef400640 1 -- 172.21.15.83:0/3810709995 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efef019f230 con 0x7efef0112eb0 2024-04-24T09:04:40.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.633+0000 7efeef400640 1 --2- 172.21.15.83:0/3810709995 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efef0112eb0 0x7efef019eab0 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 crypto rx=0x7efee8005e30 tx=0x7efee8003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:40.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.633+0000 7efee7400640 1 -- 172.21.15.83:0/3810709995 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee8034040 con 0x7efef0112eb0 2024-04-24T09:04:40.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.633+0000 7efee7400640 1 -- 172.21.15.83:0/3810709995 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efee8030850 con 0x7efef0112eb0 2024-04-24T09:04:40.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.633+0000 7efeefe00640 1 -- 172.21.15.83:0/3810709995 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efef019f4c0 con 0x7efef0112eb0 2024-04-24T09:04:40.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.633+0000 7efee7400640 1 -- 172.21.15.83:0/3810709995 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee80046f0 con 0x7efef0112eb0 2024-04-24T09:04:40.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.633+0000 7efeefe00640 1 -- 172.21.15.83:0/3810709995 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efef01ab7c0 con 0x7efef0112eb0 2024-04-24T09:04:40.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.637+0000 7efee7400640 1 -- 172.21.15.83:0/3810709995 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7efee8037020 con 0x7efef0112eb0 2024-04-24T09:04:40.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.637+0000 7efef880d740 1 Processor -- start 2024-04-24T09:04:40.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.637+0000 7efef880d740 1 -- start start 2024-04-24T09:04:40.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.638+0000 7efef880d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55817630ab40 0x55817630af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:40.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.638+0000 7efef880d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55817630b460 0x5581763155f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:40.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.638+0000 7efef880d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558176315b30 0x558176317f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:40.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.638+0000 7efef880d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5581762701d0 con 0x55817630ab40 2024-04-24T09:04:40.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.638+0000 7efef880d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558176216450 con 0x558176315b30 2024-04-24T09:04:40.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.638+0000 7efef880d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55817626c190 con 0x55817630b460 2024-04-24T09:04:40.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.638+0000 7efeee000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55817630ab40 0x55817630af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:40.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.638+0000 7efeee000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55817630ab40 0x55817630af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50032/0 (socket says 172.21.15.83:50032) 2024-04-24T09:04:40.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.638+0000 7efeee000640 1 -- 172.21.15.83:0/2021364565 learned_addr learned my addr 172.21.15.83:0/2021364565 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:04:40.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.638+0000 7efeee000640 1 --2- 172.21.15.83:0/2021364565 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558176315b30 0x558176317f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:40.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.639+0000 7efeee000640 1 -- 172.21.15.83:0/2021364565 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55817630b460 msgr2=0x5581763155f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:40.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.639+0000 7efeee000640 1 --2- 172.21.15.83:0/2021364565 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55817630b460 0x5581763155f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:40.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.639+0000 7efeee000640 1 -- 172.21.15.83:0/2021364565 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558176315b30 msgr2=0x558176317f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:40.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.639+0000 7efeee000640 1 --2- 172.21.15.83:0/2021364565 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558176315b30 0x558176317f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:40.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.639+0000 7efeee000640 1 -- 172.21.15.83:0/2021364565 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581762d3310 con 0x55817630ab40 2024-04-24T09:04:40.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.639+0000 7efee7400640 1 --2- 172.21.15.83:0/3810709995 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7efed40a3da0 0x7efed40a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:40.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.639+0000 7efee7400640 1 -- 172.21.15.83:0/3810709995 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7efee80f1b70 con 0x7efef0112eb0 2024-04-24T09:04:40.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.639+0000 7efeee000640 1 --2- 172.21.15.83:0/2021364565 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55817630ab40 0x55817630af20 secure :-1 s=READY pgs=2027 cs=0 l=1 rev1=1 crypto rx=0x7efedc0134f0 tx=0x7efedc00b600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b4eb2902a20f387 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:40.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.639+0000 7efeeea00640 1 --2- 172.21.15.83:0/3810709995 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7efed40a3da0 0x7efed40a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:40.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.639+0000 7efedbe00640 1 -- 172.21.15.83:0/2021364565 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efedc019030 con 0x55817630ab40 2024-04-24T09:04:40.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.639+0000 7efedbe00640 1 -- 172.21.15.83:0/2021364565 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efedc005bf0 con 0x55817630ab40 2024-04-24T09:04:40.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.639+0000 7efedbe00640 1 -- 172.21.15.83:0/2021364565 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efedc01d600 con 0x55817630ab40 2024-04-24T09:04:40.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.640+0000 7efef880d740 1 -- 172.21.15.83:0/2021364565 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55817630ab40 msgr2=0x55817630af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:40.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.640+0000 7efeeea00640 1 --2- 172.21.15.83:0/3810709995 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7efed40a3da0 0x7efed40a6260 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7efee0005d00 tx=0x7efee0005e50 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:40.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.640+0000 7efef880d740 1 --2- 172.21.15.83:0/2021364565 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55817630ab40 0x55817630af20 secure :-1 s=READY pgs=2027 cs=0 l=1 rev1=1 crypto rx=0x7efedc0134f0 tx=0x7efedc00b600 comp rx=0 tx=0).stop 2024-04-24T09:04:40.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.640+0000 7efef880d740 1 -- 172.21.15.83:0/2021364565 shutdown_connections 2024-04-24T09:04:40.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.640+0000 7efef880d740 1 --2- 172.21.15.83:0/2021364565 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558176315b30 0x558176317f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:40.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.640+0000 7efef880d740 1 --2- 172.21.15.83:0/2021364565 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55817630b460 0x5581763155f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:40.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.640+0000 7efef880d740 1 --2- 172.21.15.83:0/2021364565 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55817630ab40 0x55817630af20 unknown :-1 s=CLOSED pgs=2027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:40.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.640+0000 7efef880d740 1 -- 172.21.15.83:0/2021364565 >> 172.21.15.83:0/2021364565 conn(0x558176306e60 msgr2=0x558176307240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:40.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.640+0000 7efef880d740 1 -- 172.21.15.83:0/2021364565 shutdown_connections 2024-04-24T09:04:40.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.640+0000 7efef880d740 1 -- 172.21.15.83:0/2021364565 wait complete. 2024-04-24T09:04:40.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.641+0000 7efef880d740 1 -- start start 2024-04-24T09:04:40.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.641+0000 7efef880d740 1 Processor -- start 2024-04-24T09:04:40.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.641+0000 7efef880d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55817630ab40 0x558176315560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:40.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.642+0000 7efef880d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55817630b460 0x558176312850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:40.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.642+0000 7efef880d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558176315b30 0x558176312f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:40.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.642+0000 7efef880d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5581763189c0 con 0x55817630ab40 2024-04-24T09:04:40.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.642+0000 7efef880d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558176229810 con 0x55817630b460 2024-04-24T09:04:40.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.642+0000 7efef880d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x558176318b40 con 0x558176315b30 2024-04-24T09:04:40.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.642+0000 7efef880d740 10 client.0 _ll_register_callbacks cb 0x5581761f7140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:04:40.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.642+0000 7efeeea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55817630ab40 0x558176315560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:40.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.642+0000 7efeeea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55817630ab40 0x558176315560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50044/0 (socket says 172.21.15.83:50044) 2024-04-24T09:04:40.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.642+0000 7efeeea00640 1 -- 172.21.15.83:0/4078581563 learned_addr learned my addr 172.21.15.83:0/4078581563 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:04:40.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.642+0000 7efeee000640 1 --2- 172.21.15.83:0/4078581563 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55817630b460 0x558176312850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:40.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.642+0000 7efeeea00640 1 -- 172.21.15.83:0/4078581563 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558176315b30 msgr2=0x558176312f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:40.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.642+0000 7efeef400640 1 --2- 172.21.15.83:0/4078581563 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558176315b30 0x558176312f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:40.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.642+0000 7efeeea00640 1 --2- 172.21.15.83:0/4078581563 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558176315b30 0x558176312f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:40.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.642+0000 7efeeea00640 1 -- 172.21.15.83:0/4078581563 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55817630b460 msgr2=0x558176312850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:40.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.642+0000 7efeeea00640 1 --2- 172.21.15.83:0/4078581563 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55817630b460 0x558176312850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:40.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.642+0000 7efeeea00640 1 -- 172.21.15.83:0/4078581563 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55817623a3f0 con 0x55817630ab40 2024-04-24T09:04:40.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.642+0000 7efeee000640 1 --2- 172.21.15.83:0/4078581563 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55817630b460 0x558176312850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:04:40.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.642+0000 7efeef400640 1 --2- 172.21.15.83:0/4078581563 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558176315b30 0x558176312f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:04:40.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.643+0000 7efeeea00640 1 --2- 172.21.15.83:0/4078581563 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55817630ab40 0x558176315560 secure :-1 s=READY pgs=2028 cs=0 l=1 rev1=1 crypto rx=0x7efee00093b0 tx=0x7efee0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2a910a48865b117 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:40.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.643+0000 7efed8c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:04:40.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.643+0000 7efed8c00640 1 -- 172.21.15.83:0/4078581563 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee0022020 con 0x55817630ab40 2024-04-24T09:04:40.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.643+0000 7efed8c00640 1 -- 172.21.15.83:0/4078581563 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7efee0010690 con 0x55817630ab40 2024-04-24T09:04:40.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.643+0000 7efef880d740 1 -- 172.21.15.83:0/4078581563 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55817627dbf0 con 0x55817630ab40 2024-04-24T09:04:40.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.643+0000 7efed8c00640 1 -- 172.21.15.83:0/4078581563 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7efee002d050 con 0x55817630ab40 2024-04-24T09:04:40.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.643+0000 7efef880d740 10 client.20244 fetch_fsmap learned FSMap version 41 2024-04-24T09:04:40.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.643+0000 7efef880d740 1 -- 172.21.15.83:0/4078581563 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x558175f823f0 con 0x55817630ab40 2024-04-24T09:04:40.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.644+0000 7efed8c00640 1 -- 172.21.15.83:0/4078581563 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55817627dbf0 con 0x55817630ab40 2024-04-24T09:04:40.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.644+0000 7efed8c00640 1 -- 172.21.15.83:0/4078581563 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7efee000f040 con 0x55817630ab40 2024-04-24T09:04:40.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.644+0000 7efef880d740 10 client.20244 fetch_fsmap finished waiting for FSMap version 41 2024-04-24T09:04:40.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.644+0000 7efef880d740 10 client.20244 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-24T09:04:40.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.644+0000 7efef880d740 20 client.20244 populate_metadata read hostname 'smithi083' 2024-04-24T09:04:40.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.644+0000 7efef880d740 1 --2- 172.21.15.83:0/4078581563 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x558176325db0 0x558176328230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:40.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.644+0000 7efef880d740 4 client.20244 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:04:40.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.644+0000 7efef880d740 1 -- 172.21.15.83:0/4078581563 --> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558176277a60 con 0x558176325db0 2024-04-24T09:04:40.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.645+0000 7efeee000640 1 --2- 172.21.15.83:0/4078581563 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x558176325db0 0x558176328230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:40.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.646+0000 7efeee000640 1 --2- 172.21.15.83:0/4078581563 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x558176325db0 0x558176328230 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=216757eb89dd9223 server_cookie=8b88cace5554a5dd in_seq=0 out_seq=0 2024-04-24T09:04:40.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.646+0000 7efed8c00640 10 client.20244 ms_handle_connect on v2:172.21.15.138:6832/1157967658 2024-04-24T09:04:40.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.647+0000 7efed8c00640 1 -- 172.21.15.83:0/4078581563 <== mds.3 v2:172.21.15.138:6832/1157967658 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558176277a60 con 0x558176325db0 2024-04-24T09:04:40.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.647+0000 7efed8c00640 10 client.20244 handle_command_reply: tid=0 2024-04-24T09:04:40.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.648+0000 7efef880d740 1 client.20244 shutdown 2024-04-24T09:04:40.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.648+0000 7efef880d740 1 -- 172.21.15.83:0/4078581563 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55817630ab40 msgr2=0x558176315560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:40.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.648+0000 7efef880d740 1 --2- 172.21.15.83:0/4078581563 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55817630ab40 0x558176315560 secure :-1 s=READY pgs=2028 cs=0 l=1 rev1=1 crypto rx=0x7efee00093b0 tx=0x7efee0004730 comp rx=0 tx=0).stop 2024-04-24T09:04:40.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.648+0000 7efef880d740 1 -- 172.21.15.83:0/4078581563 shutdown_connections 2024-04-24T09:04:40.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.648+0000 7efef880d740 1 --2- 172.21.15.83:0/4078581563 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x558176325db0 0x558176328230 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:40.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.648+0000 7efef880d740 1 --2- 172.21.15.83:0/4078581563 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558176315b30 0x558176312f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:40.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.648+0000 7efef880d740 1 --2- 172.21.15.83:0/4078581563 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55817630b460 0x558176312850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:40.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.648+0000 7efef880d740 1 --2- 172.21.15.83:0/4078581563 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55817630ab40 0x558176315560 unknown :-1 s=CLOSED pgs=2028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:40.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.648+0000 7efef880d740 1 -- 172.21.15.83:0/4078581563 >> 172.21.15.83:0/4078581563 conn(0x558176306e60 msgr2=0x55817630cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:40.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.648+0000 7efed8c00640 0 client.20244 ms_handle_reset on v2:172.21.15.138:6832/1157967658 2024-04-24T09:04:40.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.648+0000 7efef880d740 1 -- 172.21.15.83:0/4078581563 shutdown_connections 2024-04-24T09:04:40.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.648+0000 7efef880d740 1 -- 172.21.15.83:0/4078581563 wait complete. 2024-04-24T09:04:40.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.648+0000 7efef880d740 20 client.20244 trim_cache size 0 max 16384 2024-04-24T09:04:40.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.664+0000 7efef880d740 1 Processor -- start 2024-04-24T09:04:40.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.664+0000 7efef880d740 1 -- start start 2024-04-24T09:04:40.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.664+0000 7efef880d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55817630ab40 0x5581763218b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:40.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.664+0000 7efef880d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55817630b460 0x558176315090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:40.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.664+0000 7efef880d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558176315b30 0x5581763155d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:40.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.664+0000 7efef880d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7efef01b5a20 con 0x558176315b30 2024-04-24T09:04:40.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.664+0000 7efef880d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efef01b7680 con 0x55817630b460 2024-04-24T09:04:40.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.664+0000 7efef880d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7efef01b60f0 con 0x55817630ab40 2024-04-24T09:04:40.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.664+0000 7efeef400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558176315b30 0x5581763155d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:40.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.664+0000 7efeef400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558176315b30 0x5581763155d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50058/0 (socket says 172.21.15.83:50058) 2024-04-24T09:04:40.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.664+0000 7efeef400640 1 -- 172.21.15.83:0/4266869461 learned_addr learned my addr 172.21.15.83:0/4266869461 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:04:40.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.665+0000 7efeee000640 1 --2- 172.21.15.83:0/4266869461 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55817630b460 0x558176315090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:40.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.665+0000 7efeef400640 1 -- 172.21.15.83:0/4266869461 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55817630ab40 msgr2=0x5581763218b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:40.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.665+0000 7efeef400640 1 --2- 172.21.15.83:0/4266869461 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55817630ab40 0x5581763218b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:40.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.665+0000 7efeef400640 1 -- 172.21.15.83:0/4266869461 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55817630b460 msgr2=0x558176315090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:40.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.665+0000 7efeef400640 1 --2- 172.21.15.83:0/4266869461 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55817630b460 0x558176315090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:40.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.665+0000 7efeef400640 1 -- 172.21.15.83:0/4266869461 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55817604c7d0 con 0x558176315b30 2024-04-24T09:04:40.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.665+0000 7efeef400640 1 --2- 172.21.15.83:0/4266869461 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558176315b30 0x5581763155d0 secure :-1 s=READY pgs=2029 cs=0 l=1 rev1=1 crypto rx=0x7efee80eefb0 tx=0x7efee80bd630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63800170c3beaa53 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:40.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.665+0000 7efedb400640 1 -- 172.21.15.83:0/4266869461 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee80bdea0 con 0x558176315b30 2024-04-24T09:04:40.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.665+0000 7efedb400640 1 -- 172.21.15.83:0/4266869461 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efee80bd7a0 con 0x558176315b30 2024-04-24T09:04:40.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.665+0000 7efedb400640 1 -- 172.21.15.83:0/4266869461 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee80b9e10 con 0x558176315b30 2024-04-24T09:04:40.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.666+0000 7efef880d740 1 -- 172.21.15.83:0/4266869461 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558176315b30 msgr2=0x5581763155d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:40.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.666+0000 7efef880d740 1 --2- 172.21.15.83:0/4266869461 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558176315b30 0x5581763155d0 secure :-1 s=READY pgs=2029 cs=0 l=1 rev1=1 crypto rx=0x7efee80eefb0 tx=0x7efee80bd630 comp rx=0 tx=0).stop 2024-04-24T09:04:40.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.666+0000 7efef880d740 1 -- 172.21.15.83:0/4266869461 shutdown_connections 2024-04-24T09:04:40.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.666+0000 7efef880d740 1 --2- 172.21.15.83:0/4266869461 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558176315b30 0x5581763155d0 unknown :-1 s=CLOSED pgs=2029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:40.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.666+0000 7efef880d740 1 --2- 172.21.15.83:0/4266869461 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55817630b460 0x558176315090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:40.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.666+0000 7efef880d740 1 --2- 172.21.15.83:0/4266869461 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55817630ab40 0x5581763218b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:40.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.666+0000 7efef880d740 1 -- 172.21.15.83:0/4266869461 >> 172.21.15.83:0/4266869461 conn(0x558176306e60 msgr2=0x558176307240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:40.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.666+0000 7efef880d740 1 -- 172.21.15.83:0/4266869461 shutdown_connections 2024-04-24T09:04:40.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.666+0000 7efef880d740 1 -- 172.21.15.83:0/4266869461 wait complete. 2024-04-24T09:04:40.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.667+0000 7efef880d740 1 -- start start 2024-04-24T09:04:40.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.667+0000 7efef880d740 1 Processor -- start 2024-04-24T09:04:40.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.667+0000 7efef880d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55817630ab40 0x5581763164f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:40.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.667+0000 7efef880d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55817630b460 0x558176316a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:40.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.667+0000 7efef880d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558176315b30 0x55817631cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:40.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.667+0000 7efef880d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7efef01b6f50 con 0x558176315b30 2024-04-24T09:04:40.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.667+0000 7efef880d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efef01b6820 con 0x55817630ab40 2024-04-24T09:04:40.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.667+0000 7efef880d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7efef01b3a20 con 0x55817630b460 2024-04-24T09:04:40.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.667+0000 7efef880d740 10 client.0 _ll_register_callbacks cb 0x7efee0009690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:04:40.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.668+0000 7efeef400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558176315b30 0x55817631cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:40.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.668+0000 7efeef400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558176315b30 0x55817631cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50062/0 (socket says 172.21.15.83:50062) 2024-04-24T09:04:40.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.668+0000 7efeef400640 1 -- 172.21.15.83:0/1141160401 learned_addr learned my addr 172.21.15.83:0/1141160401 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:04:40.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.668+0000 7efeee000640 1 --2- 172.21.15.83:0/1141160401 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55817630b460 0x558176316a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:40.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.668+0000 7efeeea00640 1 --2- 172.21.15.83:0/1141160401 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55817630ab40 0x5581763164f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:40.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.668+0000 7efeef400640 1 -- 172.21.15.83:0/1141160401 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55817630b460 msgr2=0x558176316a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:40.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.668+0000 7efeef400640 1 --2- 172.21.15.83:0/1141160401 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55817630b460 0x558176316a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:40.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.668+0000 7efeef400640 1 -- 172.21.15.83:0/1141160401 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55817630ab40 msgr2=0x5581763164f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:40.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.668+0000 7efeef400640 1 --2- 172.21.15.83:0/1141160401 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55817630ab40 0x5581763164f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:40.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.668+0000 7efeef400640 1 -- 172.21.15.83:0/1141160401 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55817631d330 con 0x558176315b30 2024-04-24T09:04:40.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.668+0000 7efeeea00640 1 --2- 172.21.15.83:0/1141160401 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55817630ab40 0x5581763164f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:04:40.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.668+0000 7efeef400640 1 --2- 172.21.15.83:0/1141160401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558176315b30 0x55817631cdc0 secure :-1 s=READY pgs=2030 cs=0 l=1 rev1=1 crypto rx=0x7efee8006010 tx=0x7efee802a950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fb86437a6971962 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:40.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.668+0000 7efed8c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:04:40.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.669+0000 7efed8c00640 1 -- 172.21.15.83:0/1141160401 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee80bdea0 con 0x558176315b30 2024-04-24T09:04:40.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.669+0000 7efed8c00640 1 -- 172.21.15.83:0/1141160401 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7efee80ef0c0 con 0x558176315b30 2024-04-24T09:04:40.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.669+0000 7efef880d740 1 -- 172.21.15.83:0/1141160401 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efee80bd7a0 con 0x558176315b30 2024-04-24T09:04:40.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.669+0000 7efed8c00640 1 -- 172.21.15.83:0/1141160401 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7efee80c1b50 con 0x558176315b30 2024-04-24T09:04:40.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.669+0000 7efef880d740 10 client.20250 fetch_fsmap learned FSMap version 41 2024-04-24T09:04:40.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.669+0000 7efef880d740 1 -- 172.21.15.83:0/1141160401 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55817631e610 con 0x558176315b30 2024-04-24T09:04:40.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.669+0000 7efed8c00640 1 -- 172.21.15.83:0/1141160401 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efee80bd7a0 con 0x558176315b30 2024-04-24T09:04:40.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.669+0000 7efed8c00640 1 -- 172.21.15.83:0/1141160401 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7efee80a6ce0 con 0x558176315b30 2024-04-24T09:04:40.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.669+0000 7efef880d740 10 client.20250 fetch_fsmap finished waiting for FSMap version 41 2024-04-24T09:04:40.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.669+0000 7efef880d740 10 client.20250 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-24T09:04:40.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.669+0000 7efef880d740 20 client.20250 populate_metadata read hostname 'smithi083' 2024-04-24T09:04:40.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.669+0000 7efef880d740 1 --2- 172.21.15.83:0/1141160401 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55817638c750 0x55817638eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:40.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.669+0000 7efef880d740 4 client.20250 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:04:40.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.669+0000 7efef880d740 1 -- 172.21.15.83:0/1141160401 --> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5581761f34d0 con 0x55817638c750 2024-04-24T09:04:40.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.670+0000 7efeeea00640 1 --2- 172.21.15.83:0/1141160401 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55817638c750 0x55817638eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:40.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.671+0000 7efeeea00640 1 --2- 172.21.15.83:0/1141160401 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55817638c750 0x55817638eb40 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=bdac67d08cf8c52a server_cookie=54c5d61bbdc195a6 in_seq=0 out_seq=0 2024-04-24T09:04:40.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.671+0000 7efed8c00640 10 client.20250 ms_handle_connect on v2:172.21.15.138:6832/1157967658 2024-04-24T09:04:40.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.671+0000 7efed8c00640 1 -- 172.21.15.83:0/1141160401 <== mds.3 v2:172.21.15.138:6832/1157967658 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5581761f34d0 con 0x55817638c750 2024-04-24T09:04:40.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.672+0000 7efed8c00640 10 client.20250 handle_command_reply: tid=0 2024-04-24T09:04:40.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.672+0000 7efef880d740 1 client.20250 shutdown 2024-04-24T09:04:40.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.672+0000 7efef880d740 1 -- 172.21.15.83:0/1141160401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558176315b30 msgr2=0x55817631cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:40.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.672+0000 7efef880d740 1 --2- 172.21.15.83:0/1141160401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558176315b30 0x55817631cdc0 secure :-1 s=READY pgs=2030 cs=0 l=1 rev1=1 crypto rx=0x7efee8006010 tx=0x7efee802a950 comp rx=0 tx=0).stop 2024-04-24T09:04:40.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.672+0000 7efef880d740 1 -- 172.21.15.83:0/1141160401 shutdown_connections 2024-04-24T09:04:40.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.672+0000 7efef880d740 1 --2- 172.21.15.83:0/1141160401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558176315b30 0x55817631cdc0 unknown :-1 s=CLOSED pgs=2030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:40.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.672+0000 7efef880d740 1 --2- 172.21.15.83:0/1141160401 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55817638c750 0x55817638eb40 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:40.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.672+0000 7efef880d740 1 --2- 172.21.15.83:0/1141160401 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55817630b460 0x558176316a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:40.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.672+0000 7efef880d740 1 --2- 172.21.15.83:0/1141160401 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55817630ab40 0x5581763164f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:40.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.672+0000 7efed8c00640 0 client.20250 ms_handle_reset on v2:172.21.15.138:6832/1157967658 2024-04-24T09:04:40.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.672+0000 7efef880d740 1 -- 172.21.15.83:0/1141160401 >> 172.21.15.83:0/1141160401 conn(0x558176306e60 msgr2=0x55817630dad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:40.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.672+0000 7efef880d740 1 -- 172.21.15.83:0/1141160401 shutdown_connections 2024-04-24T09:04:40.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.672+0000 7efef880d740 1 -- 172.21.15.83:0/1141160401 wait complete. 2024-04-24T09:04:40.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.672+0000 7efef880d740 20 client.20250 trim_cache size 0 max 16384 2024-04-24T09:04:40.675 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:04:40.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.673+0000 7efeefe00640 1 -- 172.21.15.83:0/3810709995 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7efed40a3da0 msgr2=0x7efed40a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:40.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.673+0000 7efeefe00640 1 --2- 172.21.15.83:0/3810709995 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7efed40a3da0 0x7efed40a6260 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7efee0005d00 tx=0x7efee0005e50 comp rx=0 tx=0).stop 2024-04-24T09:04:40.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.673+0000 7efeefe00640 1 -- 172.21.15.83:0/3810709995 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efef0112eb0 msgr2=0x7efef019eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:40.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.673+0000 7efeefe00640 1 --2- 172.21.15.83:0/3810709995 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efef0112eb0 0x7efef019eab0 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 crypto rx=0x7efee8005e30 tx=0x7efee8003f10 comp rx=0 tx=0).stop 2024-04-24T09:04:40.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.673+0000 7efeefe00640 1 -- 172.21.15.83:0/3810709995 shutdown_connections 2024-04-24T09:04:40.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.673+0000 7efeefe00640 1 --2- 172.21.15.83:0/3810709995 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7efed40a3da0 0x7efed40a6260 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:40.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.673+0000 7efeefe00640 1 --2- 172.21.15.83:0/3810709995 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efef0112eb0 0x7efef019eab0 unknown :-1 s=CLOSED pgs=2026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:40.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.673+0000 7efeefe00640 1 --2- 172.21.15.83:0/3810709995 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efef0074cc0 0x7efef01a4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:40.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.673+0000 7efeefe00640 1 --2- 172.21.15.83:0/3810709995 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efef00743a0 0x7efef01a4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:40.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.673+0000 7efeefe00640 1 -- 172.21.15.83:0/3810709995 >> 172.21.15.83:0/3810709995 conn(0x7efef00fd9d0 msgr2=0x7efef00ff230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:40.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.673+0000 7efeefe00640 1 -- 172.21.15.83:0/3810709995 shutdown_connections 2024-04-24T09:04:40.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.673+0000 7efeefe00640 1 -- 172.21.15.83:0/3810709995 wait complete. 2024-04-24T09:04:40.692 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T09:04:40.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.906+0000 7f3319600640 1 -- 172.21.15.83:0/2882307441 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f331410e760 msgr2=0x7f3314110b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:40.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.906+0000 7f3319600640 1 --2- 172.21.15.83:0/2882307441 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f331410e760 0x7f3314110b50 secure :-1 s=READY pgs=2031 cs=0 l=1 rev1=1 crypto rx=0x7f3304009900 tx=0x7f330402a480 comp rx=0 tx=0).stop 2024-04-24T09:04:40.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.906+0000 7f3319600640 1 -- 172.21.15.83:0/2882307441 shutdown_connections 2024-04-24T09:04:40.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.906+0000 7f3319600640 1 --2- 172.21.15.83:0/2882307441 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f331410e760 0x7f3314110b50 unknown :-1 s=CLOSED pgs=2031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:40.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.906+0000 7f3319600640 1 --2- 172.21.15.83:0/2882307441 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3314074cd0 0x7f331410e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:40.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.906+0000 7f3319600640 1 --2- 172.21.15.83:0/2882307441 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f33140743b0 0x7f3314074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:40.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.906+0000 7f3319600640 1 -- 172.21.15.83:0/2882307441 >> 172.21.15.83:0/2882307441 conn(0x7f33140fd9d0 msgr2=0x7f33140ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:40.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.906+0000 7f3319600640 1 -- 172.21.15.83:0/2882307441 shutdown_connections 2024-04-24T09:04:40.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.906+0000 7f3319600640 1 -- 172.21.15.83:0/2882307441 wait complete. 2024-04-24T09:04:40.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.907+0000 7f3319600640 1 Processor -- start 2024-04-24T09:04:40.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.907+0000 7f3319600640 1 -- start start 2024-04-24T09:04:40.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.908+0000 7f3319600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f33140743b0 0x7f33141a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:40.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.908+0000 7f3319600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3314074cd0 0x7f33141a0b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:40.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.908+0000 7f3319600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f331410e760 0x7f331419a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:40.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.908+0000 7f3319600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3314114310 con 0x7f331410e760 2024-04-24T09:04:40.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.908+0000 7f3319600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3314114190 con 0x7f33140743b0 2024-04-24T09:04:40.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.908+0000 7f3319600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3314114490 con 0x7f3314074cd0 2024-04-24T09:04:40.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.908+0000 7f3318c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f331410e760 0x7f331419a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:40.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.908+0000 7f3318c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f331410e760 0x7f331419a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50074/0 (socket says 172.21.15.83:50074) 2024-04-24T09:04:40.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.908+0000 7f3318c00640 1 -- 172.21.15.83:0/666338805 learned_addr learned my addr 172.21.15.83:0/666338805 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:04:40.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.908+0000 7f3313e00640 1 --2- 172.21.15.83:0/666338805 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f33140743b0 0x7f33141a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:40.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.908+0000 7f3313400640 1 --2- 172.21.15.83:0/666338805 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3314074cd0 0x7f33141a0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:40.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.908+0000 7f3318c00640 1 -- 172.21.15.83:0/666338805 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3314074cd0 msgr2=0x7f33141a0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:40.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.908+0000 7f3318c00640 1 --2- 172.21.15.83:0/666338805 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3314074cd0 0x7f33141a0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:40.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.908+0000 7f3318c00640 1 -- 172.21.15.83:0/666338805 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f33140743b0 msgr2=0x7f33141a05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:40.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.908+0000 7f3318c00640 1 --2- 172.21.15.83:0/666338805 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f33140743b0 0x7f33141a05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:40.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.908+0000 7f3318c00640 1 -- 172.21.15.83:0/666338805 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f331419af70 con 0x7f331410e760 2024-04-24T09:04:40.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.908+0000 7f3313400640 1 --2- 172.21.15.83:0/666338805 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3314074cd0 0x7f33141a0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:04:40.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.909+0000 7f3318c00640 1 --2- 172.21.15.83:0/666338805 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f331410e760 0x7f331419a6b0 secure :-1 s=READY pgs=2032 cs=0 l=1 rev1=1 crypto rx=0x7f3304009230 tx=0x7f3304004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:40.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.909+0000 7f3310c00640 1 -- 172.21.15.83:0/666338805 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f330403f030 con 0x7f331410e760 2024-04-24T09:04:40.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.909+0000 7f3310c00640 1 -- 172.21.15.83:0/666338805 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3304003aa0 con 0x7f331410e760 2024-04-24T09:04:40.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.909+0000 7f3319600640 1 -- 172.21.15.83:0/666338805 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f331419b200 con 0x7f331410e760 2024-04-24T09:04:40.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.909+0000 7f3310c00640 1 -- 172.21.15.83:0/666338805 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3304035070 con 0x7f331410e760 2024-04-24T09:04:40.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.909+0000 7f3319600640 1 -- 172.21.15.83:0/666338805 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33141a7390 con 0x7f331410e760 2024-04-24T09:04:40.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.913+0000 7f3310c00640 1 -- 172.21.15.83:0/666338805 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3304042030 con 0x7f331410e760 2024-04-24T09:04:40.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.913+0000 7f331da34740 1 Processor -- start 2024-04-24T09:04:40.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.913+0000 7f331da34740 1 -- start start 2024-04-24T09:04:40.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.914+0000 7f331da34740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56513aab0b40 0x56513aab0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:40.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.914+0000 7f331da34740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56513aab1460 0x56513aabb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:40.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.914+0000 7f331da34740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56513aabbb30 0x56513aabdf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:40.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.914+0000 7f331da34740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56513aa161d0 con 0x56513aab0b40 2024-04-24T09:04:40.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.914+0000 7f331da34740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56513a9bc450 con 0x56513aabbb30 2024-04-24T09:04:40.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.914+0000 7f331da34740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56513aa12190 con 0x56513aab1460 2024-04-24T09:04:40.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.914+0000 7f3313400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56513aab0b40 0x56513aab0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:40.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.914+0000 7f3313400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56513aab0b40 0x56513aab0f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50090/0 (socket says 172.21.15.83:50090) 2024-04-24T09:04:40.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.914+0000 7f3313400640 1 -- 172.21.15.83:0/2076971677 learned_addr learned my addr 172.21.15.83:0/2076971677 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:04:40.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.914+0000 7f3313400640 1 --2- 172.21.15.83:0/2076971677 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56513aabbb30 0x56513aabdf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:40.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.914+0000 7f3313e00640 1 --2- 172.21.15.83:0/2076971677 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56513aab1460 0x56513aabb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:40.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.914+0000 7f3313400640 1 -- 172.21.15.83:0/2076971677 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56513aab1460 msgr2=0x56513aabb5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:40.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.914+0000 7f3313400640 1 --2- 172.21.15.83:0/2076971677 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56513aab1460 0x56513aabb5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:40.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.914+0000 7f3313400640 1 -- 172.21.15.83:0/2076971677 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56513aabbb30 msgr2=0x56513aabdf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:40.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.914+0000 7f3313400640 1 --2- 172.21.15.83:0/2076971677 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56513aabbb30 0x56513aabdf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:40.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.914+0000 7f3313400640 1 -- 172.21.15.83:0/2076971677 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56513aa79310 con 0x56513aab0b40 2024-04-24T09:04:40.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.915+0000 7f3313e00640 1 --2- 172.21.15.83:0/2076971677 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56513aab1460 0x56513aabb5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:04:40.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.915+0000 7f3310c00640 1 --2- 172.21.15.83:0/666338805 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f32f80a3d20 0x7f32f80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:40.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.915+0000 7f3313400640 1 --2- 172.21.15.83:0/2076971677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56513aab0b40 0x56513aab0f20 secure :-1 s=READY pgs=2033 cs=0 l=1 rev1=1 crypto rx=0x7f330000bd40 tx=0x7f330000b4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5f03c9cc6794852 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:40.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.915+0000 7f3310c00640 1 -- 172.21.15.83:0/666338805 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7f33040f0ee0 con 0x7f331410e760 2024-04-24T09:04:40.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.915+0000 7f3313e00640 1 --2- 172.21.15.83:0/666338805 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f32f80a3d20 0x7f32f80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:40.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.915+0000 7f3309200640 1 -- 172.21.15.83:0/2076971677 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3300012650 con 0x56513aab0b40 2024-04-24T09:04:40.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.915+0000 7f3309200640 1 -- 172.21.15.83:0/2076971677 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3300012ca0 con 0x56513aab0b40 2024-04-24T09:04:40.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.915+0000 7f3309200640 1 -- 172.21.15.83:0/2076971677 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3300011c20 con 0x56513aab0b40 2024-04-24T09:04:40.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.916+0000 7f331da34740 1 -- 172.21.15.83:0/2076971677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56513aab0b40 msgr2=0x56513aab0f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:40.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.916+0000 7f331da34740 1 --2- 172.21.15.83:0/2076971677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56513aab0b40 0x56513aab0f20 secure :-1 s=READY pgs=2033 cs=0 l=1 rev1=1 crypto rx=0x7f330000bd40 tx=0x7f330000b4c0 comp rx=0 tx=0).stop 2024-04-24T09:04:40.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.916+0000 7f3313e00640 1 --2- 172.21.15.83:0/666338805 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f32f80a3d20 0x7f32f80a61e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x56513aab2100 tx=0x7f330c009500 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:40.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.916+0000 7f331da34740 1 -- 172.21.15.83:0/2076971677 shutdown_connections 2024-04-24T09:04:40.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.916+0000 7f331da34740 1 --2- 172.21.15.83:0/2076971677 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56513aabbb30 0x56513aabdf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:40.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.916+0000 7f331da34740 1 --2- 172.21.15.83:0/2076971677 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56513aab1460 0x56513aabb5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:40.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.916+0000 7f331da34740 1 --2- 172.21.15.83:0/2076971677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56513aab0b40 0x56513aab0f20 unknown :-1 s=CLOSED pgs=2033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:40.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.916+0000 7f331da34740 1 -- 172.21.15.83:0/2076971677 >> 172.21.15.83:0/2076971677 conn(0x56513aaace60 msgr2=0x56513aaad240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:40.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.916+0000 7f331da34740 1 -- 172.21.15.83:0/2076971677 shutdown_connections 2024-04-24T09:04:40.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.917+0000 7f331da34740 1 -- 172.21.15.83:0/2076971677 wait complete. 2024-04-24T09:04:40.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.917+0000 7f331da34740 1 -- start start 2024-04-24T09:04:40.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.917+0000 7f331da34740 1 Processor -- start 2024-04-24T09:04:40.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.918+0000 7f331da34740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56513aab0b40 0x56513aab8910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:40.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.918+0000 7f331da34740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56513aab1460 0x56513aab8e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:40.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.918+0000 7f331da34740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56513aabbb30 0x56513aab9390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:40.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.918+0000 7f331da34740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56513aabe9c0 con 0x56513aabbb30 2024-04-24T09:04:40.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.918+0000 7f331da34740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56513a9cf810 con 0x56513aab0b40 2024-04-24T09:04:40.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.918+0000 7f331da34740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56513aabeb40 con 0x56513aab1460 2024-04-24T09:04:40.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.918+0000 7f331da34740 10 client.0 _ll_register_callbacks cb 0x56513a99d140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:04:40.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.918+0000 7f3318c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56513aabbb30 0x56513aab9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:40.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.918+0000 7f3318c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56513aabbb30 0x56513aab9390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50104/0 (socket says 172.21.15.83:50104) 2024-04-24T09:04:40.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.918+0000 7f3318c00640 1 -- 172.21.15.83:0/727424464 learned_addr learned my addr 172.21.15.83:0/727424464 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:04:40.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.918+0000 7f3313e00640 1 --2- 172.21.15.83:0/727424464 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56513aab0b40 0x56513aab8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:40.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.918+0000 7f3313400640 1 --2- 172.21.15.83:0/727424464 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56513aab1460 0x56513aab8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:40.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.919+0000 7f3318c00640 1 -- 172.21.15.83:0/727424464 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56513aab1460 msgr2=0x56513aab8e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:40.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.919+0000 7f3318c00640 1 --2- 172.21.15.83:0/727424464 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56513aab1460 0x56513aab8e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:40.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.919+0000 7f3318c00640 1 -- 172.21.15.83:0/727424464 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56513aab0b40 msgr2=0x56513aab8910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:40.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.919+0000 7f3318c00640 1 --2- 172.21.15.83:0/727424464 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56513aab0b40 0x56513aab8910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:40.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.919+0000 7f3318c00640 1 -- 172.21.15.83:0/727424464 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56513a9e03f0 con 0x56513aabbb30 2024-04-24T09:04:40.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.919+0000 7f3313e00640 1 --2- 172.21.15.83:0/727424464 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56513aab0b40 0x56513aab8910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:04:40.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.919+0000 7f3318c00640 1 --2- 172.21.15.83:0/727424464 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56513aabbb30 0x56513aab9390 secure :-1 s=READY pgs=2034 cs=0 l=1 rev1=1 crypto rx=0x7f33040c51a0 tx=0x7f33040bd220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e67d5a5c457cf127 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:40.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.919+0000 7f32fd600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:04:40.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.919+0000 7f32fd600640 1 -- 172.21.15.83:0/727424464 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3304049070 con 0x56513aabbb30 2024-04-24T09:04:40.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.919+0000 7f32fd600640 1 -- 172.21.15.83:0/727424464 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7f3304003c50 con 0x56513aabbb30 2024-04-24T09:04:40.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.919+0000 7f331da34740 1 -- 172.21.15.83:0/727424464 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3300012ca0 con 0x56513aabbb30 2024-04-24T09:04:40.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.919+0000 7f32fd600640 1 -- 172.21.15.83:0/727424464 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f33040c56d0 con 0x56513aabbb30 2024-04-24T09:04:40.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.919+0000 7f331da34740 10 client.20262 fetch_fsmap learned FSMap version 41 2024-04-24T09:04:40.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.919+0000 7f331da34740 1 -- 172.21.15.83:0/727424464 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x56513a7283f0 con 0x56513aabbb30 2024-04-24T09:04:40.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.920+0000 7f32fd600640 1 -- 172.21.15.83:0/727424464 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3300012ca0 con 0x56513aabbb30 2024-04-24T09:04:40.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.920+0000 7f32fd600640 1 -- 172.21.15.83:0/727424464 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7f330404c060 con 0x56513aabbb30 2024-04-24T09:04:40.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.920+0000 7f331da34740 10 client.20262 fetch_fsmap finished waiting for FSMap version 41 2024-04-24T09:04:40.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.920+0000 7f331da34740 10 client.20262 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-24T09:04:40.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.920+0000 7f331da34740 20 client.20262 populate_metadata read hostname 'smithi083' 2024-04-24T09:04:40.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.920+0000 7f331da34740 1 --2- 172.21.15.83:0/727424464 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x56513aacbc70 0x56513aace060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:40.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.920+0000 7f331da34740 4 client.20262 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:04:40.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.920+0000 7f331da34740 1 -- 172.21.15.83:0/727424464 --> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56513aa23bf0 con 0x56513aacbc70 2024-04-24T09:04:40.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.921+0000 7f3313e00640 1 --2- 172.21.15.83:0/727424464 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x56513aacbc70 0x56513aace060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:40.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.922+0000 7f3313e00640 1 --2- 172.21.15.83:0/727424464 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x56513aacbc70 0x56513aace060 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=eae93e06c22d0479 server_cookie=40f5541ad0f52863 in_seq=0 out_seq=0 2024-04-24T09:04:40.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.922+0000 7f32fd600640 10 client.20262 ms_handle_connect on v2:172.21.15.138:6834/1909219758 2024-04-24T09:04:40.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.924+0000 7f32fd600640 1 -- 172.21.15.83:0/727424464 <== mds.2 v2:172.21.15.138:6834/1909219758 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56513aa23bf0 con 0x56513aacbc70 2024-04-24T09:04:40.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.924+0000 7f32fd600640 10 client.20262 handle_command_reply: tid=0 2024-04-24T09:04:40.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.924+0000 7f331da34740 1 client.20262 shutdown 2024-04-24T09:04:40.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.924+0000 7f331da34740 1 -- 172.21.15.83:0/727424464 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56513aabbb30 msgr2=0x56513aab9390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:40.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.924+0000 7f331da34740 1 --2- 172.21.15.83:0/727424464 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56513aabbb30 0x56513aab9390 secure :-1 s=READY pgs=2034 cs=0 l=1 rev1=1 crypto rx=0x7f33040c51a0 tx=0x7f33040bd220 comp rx=0 tx=0).stop 2024-04-24T09:04:40.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.924+0000 7f331da34740 1 -- 172.21.15.83:0/727424464 shutdown_connections 2024-04-24T09:04:40.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.924+0000 7f331da34740 1 --2- 172.21.15.83:0/727424464 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x56513aacbc70 0x56513aace060 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:40.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.924+0000 7f331da34740 1 --2- 172.21.15.83:0/727424464 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56513aabbb30 0x56513aab9390 unknown :-1 s=CLOSED pgs=2034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:40.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.924+0000 7f331da34740 1 --2- 172.21.15.83:0/727424464 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56513aab1460 0x56513aab8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:40.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.924+0000 7f331da34740 1 --2- 172.21.15.83:0/727424464 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56513aab0b40 0x56513aab8910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:40.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.924+0000 7f331da34740 1 -- 172.21.15.83:0/727424464 >> 172.21.15.83:0/727424464 conn(0x56513aaace60 msgr2=0x56513aab2b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:40.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.924+0000 7f32fd600640 0 client.20262 ms_handle_reset on v2:172.21.15.138:6834/1909219758 2024-04-24T09:04:40.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.924+0000 7f331da34740 1 -- 172.21.15.83:0/727424464 shutdown_connections 2024-04-24T09:04:40.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.924+0000 7f331da34740 1 -- 172.21.15.83:0/727424464 wait complete. 2024-04-24T09:04:40.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.925+0000 7f331da34740 20 client.20262 trim_cache size 0 max 16384 2024-04-24T09:04:40.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.941+0000 7f331da34740 1 Processor -- start 2024-04-24T09:04:40.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.941+0000 7f331da34740 1 -- start start 2024-04-24T09:04:40.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.942+0000 7f331da34740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56513aab0b40 0x56513aab32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:40.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.942+0000 7f331da34740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56513aab1460 0x56513aac78b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:40.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.942+0000 7f331da34740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56513aabbb30 0x56513aad3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:40.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.942+0000 7f331da34740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f33141b15a0 con 0x56513aabbb30 2024-04-24T09:04:40.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.942+0000 7f331da34740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f33141b3200 con 0x56513aab0b40 2024-04-24T09:04:40.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.942+0000 7f331da34740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f33141b1c70 con 0x56513aab1460 2024-04-24T09:04:40.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.942+0000 7f3318c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56513aabbb30 0x56513aad3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:40.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.942+0000 7f3318c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56513aabbb30 0x56513aad3950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50118/0 (socket says 172.21.15.83:50118) 2024-04-24T09:04:40.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.942+0000 7f3318c00640 1 -- 172.21.15.83:0/3635306386 learned_addr learned my addr 172.21.15.83:0/3635306386 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:04:40.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.942+0000 7f3318c00640 1 -- 172.21.15.83:0/3635306386 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56513aab1460 msgr2=0x56513aac78b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:40.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.942+0000 7f3318c00640 1 --2- 172.21.15.83:0/3635306386 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56513aab1460 0x56513aac78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:40.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.942+0000 7f3313e00640 1 --2- 172.21.15.83:0/3635306386 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56513aab0b40 0x56513aab32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:40.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.942+0000 7f3318c00640 1 -- 172.21.15.83:0/3635306386 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56513aab0b40 msgr2=0x56513aab32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:40.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.942+0000 7f3318c00640 1 --2- 172.21.15.83:0/3635306386 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56513aab0b40 0x56513aab32a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:40.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.943+0000 7f3318c00640 1 -- 172.21.15.83:0/3635306386 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56513aace3d0 con 0x56513aabbb30 2024-04-24T09:04:40.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.943+0000 7f3318c00640 1 --2- 172.21.15.83:0/3635306386 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56513aabbb30 0x56513aad3950 secure :-1 s=READY pgs=2035 cs=0 l=1 rev1=1 crypto rx=0x7f33040f1290 tx=0x7f33040c0f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ba666c504b1d824 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:40.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.943+0000 7f32ffe00640 1 -- 172.21.15.83:0/3635306386 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3304049030 con 0x56513aabbb30 2024-04-24T09:04:40.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.943+0000 7f32ffe00640 1 -- 172.21.15.83:0/3635306386 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f33040bad60 con 0x56513aabbb30 2024-04-24T09:04:40.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.943+0000 7f32ffe00640 1 -- 172.21.15.83:0/3635306386 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33040bbeb0 con 0x56513aabbb30 2024-04-24T09:04:40.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.944+0000 7f331da34740 1 -- 172.21.15.83:0/3635306386 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56513aabbb30 msgr2=0x56513aad3950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:40.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.944+0000 7f331da34740 1 --2- 172.21.15.83:0/3635306386 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56513aabbb30 0x56513aad3950 secure :-1 s=READY pgs=2035 cs=0 l=1 rev1=1 crypto rx=0x7f33040f1290 tx=0x7f33040c0f90 comp rx=0 tx=0).stop 2024-04-24T09:04:40.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.944+0000 7f331da34740 1 -- 172.21.15.83:0/3635306386 shutdown_connections 2024-04-24T09:04:40.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.944+0000 7f331da34740 1 --2- 172.21.15.83:0/3635306386 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56513aabbb30 0x56513aad3950 unknown :-1 s=CLOSED pgs=2035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:40.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.944+0000 7f331da34740 1 --2- 172.21.15.83:0/3635306386 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56513aab1460 0x56513aac78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:40.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.944+0000 7f331da34740 1 --2- 172.21.15.83:0/3635306386 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56513aab0b40 0x56513aab32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:40.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.944+0000 7f331da34740 1 -- 172.21.15.83:0/3635306386 >> 172.21.15.83:0/3635306386 conn(0x56513aaace60 msgr2=0x56513aab9160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:40.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.944+0000 7f331da34740 1 -- 172.21.15.83:0/3635306386 shutdown_connections 2024-04-24T09:04:40.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.944+0000 7f331da34740 1 -- 172.21.15.83:0/3635306386 wait complete. 2024-04-24T09:04:40.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.944+0000 7f331da34740 1 -- start start 2024-04-24T09:04:40.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.945+0000 7f331da34740 1 Processor -- start 2024-04-24T09:04:40.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.945+0000 7f331da34740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56513aab0b40 0x56513aabc880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:40.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.945+0000 7f331da34740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56513aab1460 0x56513aabcdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:40.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.945+0000 7f331da34740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56513aabbb30 0x56513aabd300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:40.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.945+0000 7f331da34740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f33141b2ad0 con 0x56513aabbb30 2024-04-24T09:04:40.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.945+0000 7f331da34740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f33141b23a0 con 0x56513aab0b40 2024-04-24T09:04:40.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.945+0000 7f331da34740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f33141af5a0 con 0x56513aab1460 2024-04-24T09:04:40.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.945+0000 7f331da34740 10 client.0 _ll_register_callbacks cb 0x7f3304009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:04:40.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.946+0000 7f3318c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56513aabbb30 0x56513aabd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:40.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.946+0000 7f3318c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56513aabbb30 0x56513aabd300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50126/0 (socket says 172.21.15.83:50126) 2024-04-24T09:04:40.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.946+0000 7f3318c00640 1 -- 172.21.15.83:0/3826718321 learned_addr learned my addr 172.21.15.83:0/3826718321 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:04:40.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.946+0000 7f3313e00640 1 --2- 172.21.15.83:0/3826718321 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56513aab0b40 0x56513aabc880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:40.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.946+0000 7f3318c00640 1 -- 172.21.15.83:0/3826718321 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56513aab1460 msgr2=0x56513aabcdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:40.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.946+0000 7f3313400640 1 --2- 172.21.15.83:0/3826718321 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56513aab1460 0x56513aabcdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:40.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.946+0000 7f3318c00640 1 --2- 172.21.15.83:0/3826718321 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56513aab1460 0x56513aabcdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:40.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.946+0000 7f3318c00640 1 -- 172.21.15.83:0/3826718321 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56513aab0b40 msgr2=0x56513aabc880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:40.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.946+0000 7f3318c00640 1 --2- 172.21.15.83:0/3826718321 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56513aab0b40 0x56513aabc880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:40.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.946+0000 7f3318c00640 1 -- 172.21.15.83:0/3826718321 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56513a7f27d0 con 0x56513aabbb30 2024-04-24T09:04:40.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.946+0000 7f3318c00640 1 --2- 172.21.15.83:0/3826718321 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56513aabbb30 0x56513aabd300 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 crypto rx=0x7f33040ee810 tx=0x7f3304005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ac0350b76b58b19 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:40.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.946+0000 7f32fd600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:04:40.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.946+0000 7f32fd600640 1 -- 172.21.15.83:0/3826718321 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3304049070 con 0x56513aabbb30 2024-04-24T09:04:40.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.947+0000 7f32fd600640 1 -- 172.21.15.83:0/3826718321 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7f330403bcf0 con 0x56513aabbb30 2024-04-24T09:04:40.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.947+0000 7f331da34740 1 -- 172.21.15.83:0/3826718321 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33040bad60 con 0x56513aabbb30 2024-04-24T09:04:40.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.947+0000 7f32fd600640 1 -- 172.21.15.83:0/3826718321 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f33040bc3c0 con 0x56513aabbb30 2024-04-24T09:04:40.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.947+0000 7f331da34740 10 client.20268 fetch_fsmap learned FSMap version 41 2024-04-24T09:04:40.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.947+0000 7f331da34740 1 -- 172.21.15.83:0/3826718321 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x56513a7f27d0 con 0x56513aabbb30 2024-04-24T09:04:40.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.947+0000 7f32fd600640 1 -- 172.21.15.83:0/3826718321 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f33040bad60 con 0x56513aabbb30 2024-04-24T09:04:40.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.947+0000 7f32fd600640 1 -- 172.21.15.83:0/3826718321 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7f3304087330 con 0x56513aabbb30 2024-04-24T09:04:40.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.947+0000 7f331da34740 10 client.20268 fetch_fsmap finished waiting for FSMap version 41 2024-04-24T09:04:40.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.947+0000 7f331da34740 10 client.20268 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-24T09:04:40.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.947+0000 7f331da34740 20 client.20268 populate_metadata read hostname 'smithi083' 2024-04-24T09:04:40.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.947+0000 7f331da34740 1 --2- 172.21.15.83:0/3826718321 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x56513ab32550 0x56513ab34970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:40.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.947+0000 7f331da34740 4 client.20268 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:04:40.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.947+0000 7f331da34740 1 -- 172.21.15.83:0/3826718321 --> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56513aa1da60 con 0x56513ab32550 2024-04-24T09:04:40.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.948+0000 7f3313e00640 1 --2- 172.21.15.83:0/3826718321 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x56513ab32550 0x56513ab34970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:40.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.949+0000 7f3313e00640 1 --2- 172.21.15.83:0/3826718321 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x56513ab32550 0x56513ab34970 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9e802447639d814b server_cookie=a5ec27af4e94bcf8 in_seq=0 out_seq=0 2024-04-24T09:04:40.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.949+0000 7f32fd600640 10 client.20268 ms_handle_connect on v2:172.21.15.138:6834/1909219758 2024-04-24T09:04:40.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.949+0000 7f32fd600640 1 -- 172.21.15.83:0/3826718321 <== mds.2 v2:172.21.15.138:6834/1909219758 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56513aa1da60 con 0x56513ab32550 2024-04-24T09:04:40.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.950+0000 7f32fd600640 10 client.20268 handle_command_reply: tid=0 2024-04-24T09:04:40.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.950+0000 7f331da34740 1 client.20268 shutdown 2024-04-24T09:04:40.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.950+0000 7f331da34740 1 -- 172.21.15.83:0/3826718321 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56513aabbb30 msgr2=0x56513aabd300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:40.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.950+0000 7f331da34740 1 --2- 172.21.15.83:0/3826718321 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56513aabbb30 0x56513aabd300 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 crypto rx=0x7f33040ee810 tx=0x7f3304005e80 comp rx=0 tx=0).stop 2024-04-24T09:04:40.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.950+0000 7f331da34740 1 -- 172.21.15.83:0/3826718321 shutdown_connections 2024-04-24T09:04:40.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.950+0000 7f331da34740 1 --2- 172.21.15.83:0/3826718321 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x56513ab32550 0x56513ab34970 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:40.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.950+0000 7f331da34740 1 --2- 172.21.15.83:0/3826718321 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56513aabbb30 0x56513aabd300 unknown :-1 s=CLOSED pgs=2036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:40.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.950+0000 7f331da34740 1 --2- 172.21.15.83:0/3826718321 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56513aab1460 0x56513aabcdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:40.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.950+0000 7f32fd600640 0 client.20268 ms_handle_reset on v2:172.21.15.138:6834/1909219758 2024-04-24T09:04:40.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.950+0000 7f331da34740 1 --2- 172.21.15.83:0/3826718321 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56513aab0b40 0x56513aabc880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:40.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.950+0000 7f331da34740 1 -- 172.21.15.83:0/3826718321 >> 172.21.15.83:0/3826718321 conn(0x56513aaace60 msgr2=0x56513aab88d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:40.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.950+0000 7f331da34740 1 -- 172.21.15.83:0/3826718321 shutdown_connections 2024-04-24T09:04:40.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.950+0000 7f331da34740 1 -- 172.21.15.83:0/3826718321 wait complete. 2024-04-24T09:04:40.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.950+0000 7f331da34740 20 client.20268 trim_cache size 0 max 16384 2024-04-24T09:04:40.953 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:04:40.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.951+0000 7f3319600640 1 -- 172.21.15.83:0/666338805 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f32f80a3d20 msgr2=0x7f32f80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:40.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.951+0000 7f3319600640 1 --2- 172.21.15.83:0/666338805 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f32f80a3d20 0x7f32f80a61e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x56513aab2100 tx=0x7f330c009500 comp rx=0 tx=0).stop 2024-04-24T09:04:40.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.951+0000 7f3319600640 1 -- 172.21.15.83:0/666338805 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f331410e760 msgr2=0x7f331419a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:40.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.951+0000 7f3319600640 1 --2- 172.21.15.83:0/666338805 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f331410e760 0x7f331419a6b0 secure :-1 s=READY pgs=2032 cs=0 l=1 rev1=1 crypto rx=0x7f3304009230 tx=0x7f3304004270 comp rx=0 tx=0).stop 2024-04-24T09:04:40.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.951+0000 7f3319600640 1 -- 172.21.15.83:0/666338805 shutdown_connections 2024-04-24T09:04:40.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.951+0000 7f3319600640 1 --2- 172.21.15.83:0/666338805 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f32f80a3d20 0x7f32f80a61e0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:40.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.951+0000 7f3319600640 1 --2- 172.21.15.83:0/666338805 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f331410e760 0x7f331419a6b0 unknown :-1 s=CLOSED pgs=2032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:40.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.951+0000 7f3319600640 1 --2- 172.21.15.83:0/666338805 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3314074cd0 0x7f33141a0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:40.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.951+0000 7f3319600640 1 --2- 172.21.15.83:0/666338805 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f33140743b0 0x7f33141a05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:40.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.951+0000 7f3319600640 1 -- 172.21.15.83:0/666338805 >> 172.21.15.83:0/666338805 conn(0x7f33140fd9d0 msgr2=0x7f33140fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:40.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.951+0000 7f3319600640 1 -- 172.21.15.83:0/666338805 shutdown_connections 2024-04-24T09:04:40.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:40.951+0000 7f3319600640 1 -- 172.21.15.83:0/666338805 wait complete. 2024-04-24T09:04:40.965 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:04:40 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/2664974859' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:04:40.968 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T09:04:41.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.171+0000 7f2e98e00640 1 -- 172.21.15.83:0/2818100304 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e94101000 msgr2=0x7f2e941013e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:41.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.171+0000 7f2e98e00640 1 --2- 172.21.15.83:0/2818100304 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e94101000 0x7f2e941013e0 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7f2e84009900 tx=0x7f2e8402a3f0 comp rx=0 tx=0).stop 2024-04-24T09:04:41.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.171+0000 7f2e98e00640 1 -- 172.21.15.83:0/2818100304 shutdown_connections 2024-04-24T09:04:41.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.171+0000 7f2e98e00640 1 --2- 172.21.15.83:0/2818100304 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2e941060d0 0x7f2e9410dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:41.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.171+0000 7f2e98e00640 1 --2- 172.21.15.83:0/2818100304 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e941019b0 0x7f2e941059f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:41.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.171+0000 7f2e98e00640 1 --2- 172.21.15.83:0/2818100304 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e94101000 0x7f2e941013e0 unknown :-1 s=CLOSED pgs=2037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:41.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.171+0000 7f2e98e00640 1 -- 172.21.15.83:0/2818100304 >> 172.21.15.83:0/2818100304 conn(0x7f2e940fc880 msgr2=0x7f2e940feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:41.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.172+0000 7f2e98e00640 1 -- 172.21.15.83:0/2818100304 shutdown_connections 2024-04-24T09:04:41.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.172+0000 7f2e98e00640 1 -- 172.21.15.83:0/2818100304 wait complete. 2024-04-24T09:04:41.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.173+0000 7f2e98e00640 1 Processor -- start 2024-04-24T09:04:41.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.173+0000 7f2e98e00640 1 -- start start 2024-04-24T09:04:41.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.173+0000 7f2e98e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e94101000 0x7f2e941a0470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:41.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.173+0000 7f2e98e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2e941060d0 0x7f2e941a09b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:41.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.173+0000 7f2e98e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e9419a620 0x7f2e9419aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:41.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.173+0000 7f2e98e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2e94111010 con 0x7f2e94101000 2024-04-24T09:04:41.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.173+0000 7f2e98e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2e94110e90 con 0x7f2e9419a620 2024-04-24T09:04:41.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.173+0000 7f2e98e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2e94111190 con 0x7f2e941060d0 2024-04-24T09:04:41.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.173+0000 7f2e93400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e94101000 0x7f2e941a0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:41.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.174+0000 7f2e93400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e94101000 0x7f2e941a0470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50144/0 (socket says 172.21.15.83:50144) 2024-04-24T09:04:41.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.174+0000 7f2e93400640 1 -- 172.21.15.83:0/2393536360 learned_addr learned my addr 172.21.15.83:0/2393536360 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:04:41.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.174+0000 7f2e8aa00640 1 --2- 172.21.15.83:0/2393536360 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2e941060d0 0x7f2e941a09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:41.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.174+0000 7f2e93400640 1 -- 172.21.15.83:0/2393536360 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2e941060d0 msgr2=0x7f2e941a09b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:41.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.174+0000 7f2e93400640 1 --2- 172.21.15.83:0/2393536360 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2e941060d0 0x7f2e941a09b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:41.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.174+0000 7f2e93400640 1 -- 172.21.15.83:0/2393536360 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e9419a620 msgr2=0x7f2e9419aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:41.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.174+0000 7f2e93e00640 1 --2- 172.21.15.83:0/2393536360 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e9419a620 0x7f2e9419aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:41.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.174+0000 7f2e93400640 1 --2- 172.21.15.83:0/2393536360 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e9419a620 0x7f2e9419aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:41.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.174+0000 7f2e93400640 1 -- 172.21.15.83:0/2393536360 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e9419b140 con 0x7f2e94101000 2024-04-24T09:04:41.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.174+0000 7f2e93e00640 1 --2- 172.21.15.83:0/2393536360 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e9419a620 0x7f2e9419aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:04:41.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.174+0000 7f2e93400640 1 --2- 172.21.15.83:0/2393536360 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e94101000 0x7f2e941a0470 secure :-1 s=READY pgs=2038 cs=0 l=1 rev1=1 crypto rx=0x7f2e84005940 tx=0x7f2e84004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:41.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.175+0000 7f2e90c00640 1 -- 172.21.15.83:0/2393536360 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e8403f030 con 0x7f2e94101000 2024-04-24T09:04:41.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.175+0000 7f2e90c00640 1 -- 172.21.15.83:0/2393536360 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2e84003aa0 con 0x7f2e94101000 2024-04-24T09:04:41.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.175+0000 7f2e90c00640 1 -- 172.21.15.83:0/2393536360 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e84035070 con 0x7f2e94101000 2024-04-24T09:04:41.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.175+0000 7f2e98e00640 1 -- 172.21.15.83:0/2393536360 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e9419b3d0 con 0x7f2e94101000 2024-04-24T09:04:41.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.175+0000 7f2e98e00640 1 -- 172.21.15.83:0/2393536360 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e941a7570 con 0x7f2e94101000 2024-04-24T09:04:41.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.179+0000 7f2e9d25e740 1 Processor -- start 2024-04-24T09:04:41.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.180+0000 7f2e9d25e740 1 -- start start 2024-04-24T09:04:41.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.180+0000 7f2e90c00640 1 -- 172.21.15.83:0/2393536360 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2e84042030 con 0x7f2e94101000 2024-04-24T09:04:41.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.180+0000 7f2e9d25e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557041d7ab40 0x557041d7af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:41.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.180+0000 7f2e9d25e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557041d7b460 0x557041d855f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:41.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.180+0000 7f2e9d25e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557041d85b30 0x557041d87f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:41.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.180+0000 7f2e9d25e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x557041ce01d0 con 0x557041d7b460 2024-04-24T09:04:41.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.180+0000 7f2e9d25e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x557041c86450 con 0x557041d85b30 2024-04-24T09:04:41.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.180+0000 7f2e9d25e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x557041cdc190 con 0x557041d7ab40 2024-04-24T09:04:41.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.180+0000 7f2e93400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557041d7b460 0x557041d855f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:41.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.180+0000 7f2e93400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557041d7b460 0x557041d855f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50150/0 (socket says 172.21.15.83:50150) 2024-04-24T09:04:41.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.180+0000 7f2e93400640 1 -- 172.21.15.83:0/240706324 learned_addr learned my addr 172.21.15.83:0/240706324 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:04:41.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.181+0000 7f2e93e00640 1 --2- 172.21.15.83:0/240706324 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557041d7ab40 0x557041d7af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:41.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.181+0000 7f2e8aa00640 1 --2- 172.21.15.83:0/240706324 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557041d85b30 0x557041d87f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:41.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.181+0000 7f2e93400640 1 -- 172.21.15.83:0/240706324 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557041d7ab40 msgr2=0x557041d7af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:41.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.181+0000 7f2e93400640 1 --2- 172.21.15.83:0/240706324 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557041d7ab40 0x557041d7af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:41.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.181+0000 7f2e93400640 1 -- 172.21.15.83:0/240706324 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557041d85b30 msgr2=0x557041d87f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:41.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.181+0000 7f2e93400640 1 --2- 172.21.15.83:0/240706324 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557041d85b30 0x557041d87f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:41.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.181+0000 7f2e93400640 1 -- 172.21.15.83:0/240706324 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557041d43310 con 0x557041d7b460 2024-04-24T09:04:41.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.181+0000 7f2e8aa00640 1 --2- 172.21.15.83:0/240706324 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557041d85b30 0x557041d87f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:04:41.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.181+0000 7f2e93400640 1 --2- 172.21.15.83:0/240706324 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557041d7b460 0x557041d855f0 secure :-1 s=READY pgs=2039 cs=0 l=1 rev1=1 crypto rx=0x7f2e840ee680 tx=0x7f2e840f7760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ebc12ab7fed8607 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:41.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.181+0000 7f2e90c00640 1 --2- 172.21.15.83:0/2393536360 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f2e780a3da0 0x7f2e780a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:41.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.182+0000 7f2e88c00640 1 -- 172.21.15.83:0/240706324 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e840b9200 con 0x557041d7b460 2024-04-24T09:04:41.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.182+0000 7f2e88c00640 1 -- 172.21.15.83:0/240706324 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2e840fe030 con 0x557041d7b460 2024-04-24T09:04:41.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.182+0000 7f2e88c00640 1 -- 172.21.15.83:0/240706324 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e840bc3b0 con 0x557041d7b460 2024-04-24T09:04:41.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.182+0000 7f2e93e00640 1 --2- 172.21.15.83:0/2393536360 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f2e780a3da0 0x7f2e780a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:41.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.182+0000 7f2e90c00640 1 -- 172.21.15.83:0/2393536360 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7f2e840f13e0 con 0x7f2e94101000 2024-04-24T09:04:41.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.182+0000 7f2e9d25e740 1 -- 172.21.15.83:0/240706324 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557041d7b460 msgr2=0x557041d855f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:41.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.182+0000 7f2e93e00640 1 --2- 172.21.15.83:0/2393536360 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f2e780a3da0 0x7f2e780a6260 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c005fd0 tx=0x7f2e8c009680 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:41.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.182+0000 7f2e9d25e740 1 --2- 172.21.15.83:0/240706324 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557041d7b460 0x557041d855f0 secure :-1 s=READY pgs=2039 cs=0 l=1 rev1=1 crypto rx=0x7f2e840ee680 tx=0x7f2e840f7760 comp rx=0 tx=0).stop 2024-04-24T09:04:41.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.183+0000 7f2e9d25e740 1 -- 172.21.15.83:0/240706324 shutdown_connections 2024-04-24T09:04:41.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.183+0000 7f2e9d25e740 1 --2- 172.21.15.83:0/240706324 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557041d85b30 0x557041d87f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:41.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.183+0000 7f2e9d25e740 1 --2- 172.21.15.83:0/240706324 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557041d7b460 0x557041d855f0 unknown :-1 s=CLOSED pgs=2039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:41.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.183+0000 7f2e9d25e740 1 --2- 172.21.15.83:0/240706324 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557041d7ab40 0x557041d7af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:41.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.183+0000 7f2e9d25e740 1 -- 172.21.15.83:0/240706324 >> 172.21.15.83:0/240706324 conn(0x557041d76e60 msgr2=0x557041d77240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:41.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.183+0000 7f2e9d25e740 1 -- 172.21.15.83:0/240706324 shutdown_connections 2024-04-24T09:04:41.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.183+0000 7f2e9d25e740 1 -- 172.21.15.83:0/240706324 wait complete. 2024-04-24T09:04:41.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.183+0000 7f2e9d25e740 1 -- start start 2024-04-24T09:04:41.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.184+0000 7f2e9d25e740 1 Processor -- start 2024-04-24T09:04:41.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.184+0000 7f2e9d25e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557041d7ab40 0x557041d85590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:41.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.184+0000 7f2e9d25e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557041d7b460 0x557041d82880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:41.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.184+0000 7f2e9d25e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557041d85b30 0x557041d82fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:41.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.184+0000 7f2e9d25e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x557041d886e0 con 0x557041d7ab40 2024-04-24T09:04:41.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.184+0000 7f2e9d25e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x557041c99810 con 0x557041d85b30 2024-04-24T09:04:41.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.184+0000 7f2e9d25e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x557041d88860 con 0x557041d7b460 2024-04-24T09:04:41.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.184+0000 7f2e9d25e740 10 client.0 _ll_register_callbacks cb 0x557041c67140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:04:41.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.184+0000 7f2e93400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557041d7ab40 0x557041d85590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:41.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.184+0000 7f2e93400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557041d7ab40 0x557041d85590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50160/0 (socket says 172.21.15.83:50160) 2024-04-24T09:04:41.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.184+0000 7f2e93400640 1 -- 172.21.15.83:0/4271756050 learned_addr learned my addr 172.21.15.83:0/4271756050 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:04:41.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.185+0000 7f2e93e00640 1 --2- 172.21.15.83:0/4271756050 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557041d85b30 0x557041d82fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:41.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.185+0000 7f2e8aa00640 1 --2- 172.21.15.83:0/4271756050 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557041d7b460 0x557041d82880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:41.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.185+0000 7f2e93400640 1 -- 172.21.15.83:0/4271756050 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557041d7b460 msgr2=0x557041d82880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:41.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.185+0000 7f2e93400640 1 --2- 172.21.15.83:0/4271756050 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557041d7b460 0x557041d82880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:41.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.185+0000 7f2e93400640 1 -- 172.21.15.83:0/4271756050 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557041d85b30 msgr2=0x557041d82fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:41.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.185+0000 7f2e93400640 1 --2- 172.21.15.83:0/4271756050 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557041d85b30 0x557041d82fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:41.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.185+0000 7f2e93400640 1 -- 172.21.15.83:0/4271756050 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557041caa3f0 con 0x557041d7ab40 2024-04-24T09:04:41.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.185+0000 7f2e93e00640 1 --2- 172.21.15.83:0/4271756050 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557041d85b30 0x557041d82fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:04:41.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.185+0000 7f2e93400640 1 --2- 172.21.15.83:0/4271756050 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557041d7ab40 0x557041d85590 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x7f2e84006010 tx=0x7f2e8404cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e326796059413fc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:41.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.185+0000 7f2e81600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:04:41.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.185+0000 7f2e81600640 1 -- 172.21.15.83:0/4271756050 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e84049070 con 0x557041d7ab40 2024-04-24T09:04:41.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.185+0000 7f2e81600640 1 -- 172.21.15.83:0/4271756050 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7f2e840406b0 con 0x557041d7ab40 2024-04-24T09:04:41.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.185+0000 7f2e9d25e740 1 -- 172.21.15.83:0/4271756050 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e840fe030 con 0x557041d7ab40 2024-04-24T09:04:41.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.186+0000 7f2e81600640 1 -- 172.21.15.83:0/4271756050 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f2e840bc6e0 con 0x557041d7ab40 2024-04-24T09:04:41.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.186+0000 7f2e9d25e740 10 client.20280 fetch_fsmap learned FSMap version 41 2024-04-24T09:04:41.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.186+0000 7f2e9d25e740 1 -- 172.21.15.83:0/4271756050 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5570419f23f0 con 0x557041d7ab40 2024-04-24T09:04:41.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.186+0000 7f2e81600640 1 -- 172.21.15.83:0/4271756050 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2e840fe030 con 0x557041d7ab40 2024-04-24T09:04:41.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.186+0000 7f2e81600640 1 -- 172.21.15.83:0/4271756050 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7f2e840ee990 con 0x557041d7ab40 2024-04-24T09:04:41.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.186+0000 7f2e9d25e740 10 client.20280 fetch_fsmap finished waiting for FSMap version 41 2024-04-24T09:04:41.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.186+0000 7f2e9d25e740 10 client.20280 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-24T09:04:41.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.186+0000 7f2e9d25e740 20 client.20280 populate_metadata read hostname 'smithi083' 2024-04-24T09:04:41.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.186+0000 7f2e9d25e740 1 --2- 172.21.15.83:0/4271756050 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x557041d95840 0x557041d97c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:41.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.186+0000 7f2e9d25e740 4 client.20280 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:04:41.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.186+0000 7f2e9d25e740 1 -- 172.21.15.83:0/4271756050 --> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557041cedbf0 con 0x557041d95840 2024-04-24T09:04:41.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.187+0000 7f2e8aa00640 1 --2- 172.21.15.83:0/4271756050 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x557041d95840 0x557041d97c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:41.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.188+0000 7f2e8aa00640 1 --2- 172.21.15.83:0/4271756050 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x557041d95840 0x557041d97c30 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=71204f4cbbed0f8b server_cookie=d12710e8f2f58561 in_seq=0 out_seq=0 2024-04-24T09:04:41.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.188+0000 7f2e81600640 10 client.20280 ms_handle_connect on v2:172.21.15.138:6836/900566754 2024-04-24T09:04:41.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.190+0000 7f2e81600640 1 -- 172.21.15.83:0/4271756050 <== mds.1 v2:172.21.15.138:6836/900566754 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557041cedbf0 con 0x557041d95840 2024-04-24T09:04:41.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.190+0000 7f2e81600640 10 client.20280 handle_command_reply: tid=0 2024-04-24T09:04:41.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.190+0000 7f2e9d25e740 1 client.20280 shutdown 2024-04-24T09:04:41.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.190+0000 7f2e9d25e740 1 -- 172.21.15.83:0/4271756050 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557041d7ab40 msgr2=0x557041d85590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:41.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.190+0000 7f2e9d25e740 1 --2- 172.21.15.83:0/4271756050 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557041d7ab40 0x557041d85590 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x7f2e84006010 tx=0x7f2e8404cf80 comp rx=0 tx=0).stop 2024-04-24T09:04:41.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.191+0000 7f2e9d25e740 1 -- 172.21.15.83:0/4271756050 shutdown_connections 2024-04-24T09:04:41.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.191+0000 7f2e9d25e740 1 --2- 172.21.15.83:0/4271756050 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557041d85b30 0x557041d82fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:41.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.191+0000 7f2e9d25e740 1 --2- 172.21.15.83:0/4271756050 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x557041d95840 0x557041d97c30 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:41.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.191+0000 7f2e9d25e740 1 --2- 172.21.15.83:0/4271756050 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557041d7b460 0x557041d82880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:41.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.191+0000 7f2e9d25e740 1 --2- 172.21.15.83:0/4271756050 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557041d7ab40 0x557041d85590 unknown :-1 s=CLOSED pgs=2040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:41.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.191+0000 7f2e81600640 0 client.20280 ms_handle_reset on v2:172.21.15.138:6836/900566754 2024-04-24T09:04:41.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.191+0000 7f2e9d25e740 1 -- 172.21.15.83:0/4271756050 >> 172.21.15.83:0/4271756050 conn(0x557041d76e60 msgr2=0x557041d7cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:41.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.191+0000 7f2e9d25e740 1 -- 172.21.15.83:0/4271756050 shutdown_connections 2024-04-24T09:04:41.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.191+0000 7f2e9d25e740 1 -- 172.21.15.83:0/4271756050 wait complete. 2024-04-24T09:04:41.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.191+0000 7f2e9d25e740 20 client.20280 trim_cache size 0 max 16384 2024-04-24T09:04:41.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:04:40 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/2664974859' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:04:41.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:04:40 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/2664974859' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:04:41.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.207+0000 7f2e9d25e740 1 Processor -- start 2024-04-24T09:04:41.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.208+0000 7f2e9d25e740 1 -- start start 2024-04-24T09:04:41.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.208+0000 7f2e9d25e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557041d7ab40 0x557041d915a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:41.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.208+0000 7f2e9d25e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557041d7b460 0x557041d850c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:41.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.208+0000 7f2e9d25e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557041d85b30 0x557041d9d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:41.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.208+0000 7f2e9d25e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2e941b15e0 con 0x557041d7b460 2024-04-24T09:04:41.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.208+0000 7f2e9d25e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2e941b3240 con 0x557041d7ab40 2024-04-24T09:04:41.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.208+0000 7f2e9d25e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2e941b1cb0 con 0x557041d85b30 2024-04-24T09:04:41.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.208+0000 7f2e8aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557041d7b460 0x557041d850c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:41.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.208+0000 7f2e8aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557041d7b460 0x557041d850c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50162/0 (socket says 172.21.15.83:50162) 2024-04-24T09:04:41.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.208+0000 7f2e8aa00640 1 -- 172.21.15.83:0/878637173 learned_addr learned my addr 172.21.15.83:0/878637173 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:04:41.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.208+0000 7f2e93e00640 1 --2- 172.21.15.83:0/878637173 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557041d85b30 0x557041d9d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:41.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.208+0000 7f2e93400640 1 --2- 172.21.15.83:0/878637173 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557041d7ab40 0x557041d915a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:41.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.209+0000 7f2e8aa00640 1 -- 172.21.15.83:0/878637173 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557041d85b30 msgr2=0x557041d9d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:41.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.209+0000 7f2e8aa00640 1 --2- 172.21.15.83:0/878637173 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557041d85b30 0x557041d9d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:41.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.209+0000 7f2e8aa00640 1 -- 172.21.15.83:0/878637173 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557041d7ab40 msgr2=0x557041d915a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:41.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.209+0000 7f2e8aa00640 1 --2- 172.21.15.83:0/878637173 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557041d7ab40 0x557041d915a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:41.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.209+0000 7f2e8aa00640 1 -- 172.21.15.83:0/878637173 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557041d7b290 con 0x557041d7b460 2024-04-24T09:04:41.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.209+0000 7f2e93e00640 1 --2- 172.21.15.83:0/878637173 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557041d85b30 0x557041d9d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:04:41.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.209+0000 7f2e8aa00640 1 --2- 172.21.15.83:0/878637173 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557041d7b460 0x557041d850c0 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c00dae0 tx=0x7f2e7c005cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b69c9526e9abd4d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:41.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.209+0000 7f2e83e00640 1 -- 172.21.15.83:0/878637173 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e7c017330 con 0x557041d7b460 2024-04-24T09:04:41.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.209+0000 7f2e83e00640 1 -- 172.21.15.83:0/878637173 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2e7c010650 con 0x557041d7b460 2024-04-24T09:04:41.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.209+0000 7f2e83e00640 1 -- 172.21.15.83:0/878637173 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e7c00e670 con 0x557041d7b460 2024-04-24T09:04:41.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.210+0000 7f2e9d25e740 1 -- 172.21.15.83:0/878637173 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557041d7b460 msgr2=0x557041d850c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:41.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.210+0000 7f2e9d25e740 1 --2- 172.21.15.83:0/878637173 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557041d7b460 0x557041d850c0 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c00dae0 tx=0x7f2e7c005cd0 comp rx=0 tx=0).stop 2024-04-24T09:04:41.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.210+0000 7f2e9d25e740 1 -- 172.21.15.83:0/878637173 shutdown_connections 2024-04-24T09:04:41.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.210+0000 7f2e9d25e740 1 --2- 172.21.15.83:0/878637173 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557041d85b30 0x557041d9d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:41.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.210+0000 7f2e9d25e740 1 --2- 172.21.15.83:0/878637173 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557041d7b460 0x557041d850c0 unknown :-1 s=CLOSED pgs=2041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:41.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.210+0000 7f2e9d25e740 1 --2- 172.21.15.83:0/878637173 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557041d7ab40 0x557041d915a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:41.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.210+0000 7f2e9d25e740 1 -- 172.21.15.83:0/878637173 >> 172.21.15.83:0/878637173 conn(0x557041d76e60 msgr2=0x557041d818b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:41.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.210+0000 7f2e9d25e740 1 -- 172.21.15.83:0/878637173 shutdown_connections 2024-04-24T09:04:41.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.210+0000 7f2e9d25e740 1 -- 172.21.15.83:0/878637173 wait complete. 2024-04-24T09:04:41.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.211+0000 7f2e9d25e740 1 -- start start 2024-04-24T09:04:41.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.211+0000 7f2e9d25e740 1 Processor -- start 2024-04-24T09:04:41.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.211+0000 7f2e9d25e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557041d7ab40 0x557041d8df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:41.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.211+0000 7f2e9d25e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557041d7b460 0x557041d86700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:41.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.211+0000 7f2e9d25e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557041d85b30 0x557041d86d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:41.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.211+0000 7f2e9d25e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2e941b2b10 con 0x557041d7ab40 2024-04-24T09:04:41.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.211+0000 7f2e9d25e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2e941b23e0 con 0x557041d7b460 2024-04-24T09:04:41.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.211+0000 7f2e9d25e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2e941af5c0 con 0x557041d85b30 2024-04-24T09:04:41.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.211+0000 7f2e9d25e740 10 client.0 _ll_register_callbacks cb 0x7f2e8c009520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:04:41.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.212+0000 7f2e93400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557041d7ab40 0x557041d8df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:41.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.212+0000 7f2e93400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557041d7ab40 0x557041d8df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50174/0 (socket says 172.21.15.83:50174) 2024-04-24T09:04:41.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.212+0000 7f2e93400640 1 -- 172.21.15.83:0/2999498330 learned_addr learned my addr 172.21.15.83:0/2999498330 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:04:41.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.212+0000 7f2e93400640 1 -- 172.21.15.83:0/2999498330 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557041d85b30 msgr2=0x557041d86d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:41.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.212+0000 7f2e93400640 1 --2- 172.21.15.83:0/2999498330 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557041d85b30 0x557041d86d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:41.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.212+0000 7f2e93400640 1 -- 172.21.15.83:0/2999498330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557041d7b460 msgr2=0x557041d86700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:41.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.212+0000 7f2e93400640 1 --2- 172.21.15.83:0/2999498330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557041d7b460 0x557041d86700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:41.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.212+0000 7f2e93400640 1 -- 172.21.15.83:0/2999498330 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557041abc7d0 con 0x557041d7ab40 2024-04-24T09:04:41.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.212+0000 7f2e93400640 1 --2- 172.21.15.83:0/2999498330 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557041d7ab40 0x557041d8df80 secure :-1 s=READY pgs=2042 cs=0 l=1 rev1=1 crypto rx=0x7f2e84032ad0 tx=0x7f2e840f7760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ceb10268538bab8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:41.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.212+0000 7f2e81600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:04:41.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.212+0000 7f2e81600640 1 -- 172.21.15.83:0/2999498330 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e84049040 con 0x557041d7ab40 2024-04-24T09:04:41.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.212+0000 7f2e81600640 1 -- 172.21.15.83:0/2999498330 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7f2e8403bd60 con 0x557041d7ab40 2024-04-24T09:04:41.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.212+0000 7f2e9d25e740 1 -- 172.21.15.83:0/2999498330 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e7c010650 con 0x557041d7ab40 2024-04-24T09:04:41.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.213+0000 7f2e81600640 1 -- 172.21.15.83:0/2999498330 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f2e840eee60 con 0x557041d7ab40 2024-04-24T09:04:41.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.213+0000 7f2e9d25e740 10 client.20286 fetch_fsmap learned FSMap version 41 2024-04-24T09:04:41.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.213+0000 7f2e9d25e740 1 -- 172.21.15.83:0/2999498330 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x557041abc7d0 con 0x557041d7ab40 2024-04-24T09:04:41.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.213+0000 7f2e81600640 1 -- 172.21.15.83:0/2999498330 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2e7c010650 con 0x557041d7ab40 2024-04-24T09:04:41.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.213+0000 7f2e81600640 1 -- 172.21.15.83:0/2999498330 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7f2e840ee990 con 0x557041d7ab40 2024-04-24T09:04:41.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.213+0000 7f2e9d25e740 10 client.20286 fetch_fsmap finished waiting for FSMap version 41 2024-04-24T09:04:41.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.213+0000 7f2e9d25e740 10 client.20286 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-24T09:04:41.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.213+0000 7f2e9d25e740 20 client.20286 populate_metadata read hostname 'smithi083' 2024-04-24T09:04:41.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.213+0000 7f2e9d25e740 1 --2- 172.21.15.83:0/2999498330 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x557041dfc080 0x557041dfe540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:41.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.213+0000 7f2e9d25e740 4 client.20286 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:04:41.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.213+0000 7f2e9d25e740 1 -- 172.21.15.83:0/2999498330 --> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557041ce7a60 con 0x557041dfc080 2024-04-24T09:04:41.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.214+0000 7f2e8aa00640 1 --2- 172.21.15.83:0/2999498330 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x557041dfc080 0x557041dfe540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:41.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.215+0000 7f2e8aa00640 1 --2- 172.21.15.83:0/2999498330 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x557041dfc080 0x557041dfe540 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=54537cb198f23ce7 server_cookie=81a8aeb6cc2c34cb in_seq=0 out_seq=0 2024-04-24T09:04:41.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.215+0000 7f2e81600640 10 client.20286 ms_handle_connect on v2:172.21.15.138:6836/900566754 2024-04-24T09:04:41.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.215+0000 7f2e81600640 1 -- 172.21.15.83:0/2999498330 <== mds.1 v2:172.21.15.138:6836/900566754 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557041ce7a60 con 0x557041dfc080 2024-04-24T09:04:41.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.215+0000 7f2e81600640 10 client.20286 handle_command_reply: tid=0 2024-04-24T09:04:41.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.215+0000 7f2e9d25e740 1 client.20286 shutdown 2024-04-24T09:04:41.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.216+0000 7f2e9d25e740 1 -- 172.21.15.83:0/2999498330 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557041d7ab40 msgr2=0x557041d8df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:41.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.216+0000 7f2e9d25e740 1 --2- 172.21.15.83:0/2999498330 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557041d7ab40 0x557041d8df80 secure :-1 s=READY pgs=2042 cs=0 l=1 rev1=1 crypto rx=0x7f2e84032ad0 tx=0x7f2e840f7760 comp rx=0 tx=0).stop 2024-04-24T09:04:41.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.216+0000 7f2e9d25e740 1 -- 172.21.15.83:0/2999498330 shutdown_connections 2024-04-24T09:04:41.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.216+0000 7f2e9d25e740 1 --2- 172.21.15.83:0/2999498330 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x557041dfc080 0x557041dfe540 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:41.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.216+0000 7f2e9d25e740 1 --2- 172.21.15.83:0/2999498330 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557041d85b30 0x557041d86d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:41.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.216+0000 7f2e9d25e740 1 --2- 172.21.15.83:0/2999498330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557041d7b460 0x557041d86700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:41.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.216+0000 7f2e9d25e740 1 --2- 172.21.15.83:0/2999498330 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557041d7ab40 0x557041d8df80 unknown :-1 s=CLOSED pgs=2042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:41.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.216+0000 7f2e81600640 0 client.20286 ms_handle_reset on v2:172.21.15.138:6836/900566754 2024-04-24T09:04:41.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.216+0000 7f2e9d25e740 1 -- 172.21.15.83:0/2999498330 >> 172.21.15.83:0/2999498330 conn(0x557041d76e60 msgr2=0x557041d756c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:41.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.216+0000 7f2e9d25e740 1 -- 172.21.15.83:0/2999498330 shutdown_connections 2024-04-24T09:04:41.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.216+0000 7f2e9d25e740 1 -- 172.21.15.83:0/2999498330 wait complete. 2024-04-24T09:04:41.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.216+0000 7f2e9d25e740 20 client.20286 trim_cache size 0 max 16384 2024-04-24T09:04:41.219 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:04:41.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.217+0000 7f2e98e00640 1 -- 172.21.15.83:0/2393536360 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f2e780a3da0 msgr2=0x7f2e780a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:41.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.217+0000 7f2e98e00640 1 --2- 172.21.15.83:0/2393536360 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f2e780a3da0 0x7f2e780a6260 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c005fd0 tx=0x7f2e8c009680 comp rx=0 tx=0).stop 2024-04-24T09:04:41.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.217+0000 7f2e98e00640 1 -- 172.21.15.83:0/2393536360 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e94101000 msgr2=0x7f2e941a0470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:41.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.217+0000 7f2e98e00640 1 --2- 172.21.15.83:0/2393536360 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e94101000 0x7f2e941a0470 secure :-1 s=READY pgs=2038 cs=0 l=1 rev1=1 crypto rx=0x7f2e84005940 tx=0x7f2e84004270 comp rx=0 tx=0).stop 2024-04-24T09:04:41.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.217+0000 7f2e98e00640 1 -- 172.21.15.83:0/2393536360 shutdown_connections 2024-04-24T09:04:41.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.217+0000 7f2e98e00640 1 --2- 172.21.15.83:0/2393536360 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f2e780a3da0 0x7f2e780a6260 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:41.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.217+0000 7f2e98e00640 1 --2- 172.21.15.83:0/2393536360 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e9419a620 0x7f2e9419aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:41.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.217+0000 7f2e98e00640 1 --2- 172.21.15.83:0/2393536360 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2e941060d0 0x7f2e941a09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:41.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.217+0000 7f2e98e00640 1 --2- 172.21.15.83:0/2393536360 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e94101000 0x7f2e941a0470 unknown :-1 s=CLOSED pgs=2038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:41.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.217+0000 7f2e98e00640 1 -- 172.21.15.83:0/2393536360 >> 172.21.15.83:0/2393536360 conn(0x7f2e940fc880 msgr2=0x7f2e940fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:41.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.217+0000 7f2e98e00640 1 -- 172.21.15.83:0/2393536360 shutdown_connections 2024-04-24T09:04:41.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.217+0000 7f2e98e00640 1 -- 172.21.15.83:0/2393536360 wait complete. 2024-04-24T09:04:41.236 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T09:04:41.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.445+0000 7efe2be00640 1 -- 172.21.15.83:0/2894890408 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efe2c1093f0 msgr2=0x7efe2c110fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:41.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.445+0000 7efe2be00640 1 --2- 172.21.15.83:0/2894890408 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efe2c1093f0 0x7efe2c110fc0 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x7efe24009ca0 tx=0x7efe2402a8c0 comp rx=0 tx=0).stop 2024-04-24T09:04:41.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.445+0000 7efe2be00640 1 -- 172.21.15.83:0/2894890408 shutdown_connections 2024-04-24T09:04:41.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.445+0000 7efe2be00640 1 --2- 172.21.15.83:0/2894890408 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efe2c1093f0 0x7efe2c110fc0 unknown :-1 s=CLOSED pgs=2043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:41.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.445+0000 7efe2be00640 1 --2- 172.21.15.83:0/2894890408 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe2c104cd0 0x7efe2c108d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:41.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.445+0000 7efe2be00640 1 --2- 172.21.15.83:0/2894890408 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efe2c104320 0x7efe2c104700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:41.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.445+0000 7efe2be00640 1 -- 172.21.15.83:0/2894890408 >> 172.21.15.83:0/2894890408 conn(0x7efe2c0ffb40 msgr2=0x7efe2c101f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:41.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.445+0000 7efe2be00640 1 -- 172.21.15.83:0/2894890408 shutdown_connections 2024-04-24T09:04:41.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.445+0000 7efe2be00640 1 -- 172.21.15.83:0/2894890408 wait complete. 2024-04-24T09:04:41.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.446+0000 7efe2be00640 1 Processor -- start 2024-04-24T09:04:41.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.447+0000 7efe2be00640 1 -- start start 2024-04-24T09:04:41.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.447+0000 7efe2be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efe2c104320 0x7efe2c1a0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:41.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.447+0000 7efe2be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe2c1093f0 0x7efe2c1a0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:41.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.447+0000 7efe2be00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efe2c19a6c0 0x7efe2c19ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:41.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.447+0000 7efe2be00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7efe2c114360 con 0x7efe2c104320 2024-04-24T09:04:41.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.447+0000 7efe2be00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efe2c1141e0 con 0x7efe2c1093f0 2024-04-24T09:04:41.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.447+0000 7efe2be00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7efe2c1144e0 con 0x7efe2c19a6c0 2024-04-24T09:04:41.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.447+0000 7efe2aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efe2c104320 0x7efe2c1a0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:41.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.447+0000 7efe2aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efe2c104320 0x7efe2c1a0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50188/0 (socket says 172.21.15.83:50188) 2024-04-24T09:04:41.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.447+0000 7efe2aa00640 1 -- 172.21.15.83:0/4176728033 learned_addr learned my addr 172.21.15.83:0/4176728033 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:04:41.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.448+0000 7efe2b400640 1 --2- 172.21.15.83:0/4176728033 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efe2c19a6c0 0x7efe2c19ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:41.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.448+0000 7efe2a000640 1 --2- 172.21.15.83:0/4176728033 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe2c1093f0 0x7efe2c1a0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:41.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.448+0000 7efe2aa00640 1 -- 172.21.15.83:0/4176728033 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efe2c19a6c0 msgr2=0x7efe2c19ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:41.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.448+0000 7efe2aa00640 1 --2- 172.21.15.83:0/4176728033 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efe2c19a6c0 0x7efe2c19ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:41.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.448+0000 7efe2aa00640 1 -- 172.21.15.83:0/4176728033 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe2c1093f0 msgr2=0x7efe2c1a0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:41.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.448+0000 7efe2aa00640 1 --2- 172.21.15.83:0/4176728033 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe2c1093f0 0x7efe2c1a0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:41.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.448+0000 7efe2aa00640 1 -- 172.21.15.83:0/4176728033 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe2c19b2c0 con 0x7efe2c104320 2024-04-24T09:04:41.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.448+0000 7efe2aa00640 1 --2- 172.21.15.83:0/4176728033 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efe2c104320 0x7efe2c1a0600 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 crypto rx=0x7efe1c00d770 tx=0x7efe1c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:41.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.449+0000 7efe30a00640 1 -- 172.21.15.83:0/4176728033 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe1c017070 con 0x7efe2c104320 2024-04-24T09:04:41.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.449+0000 7efe30a00640 1 -- 172.21.15.83:0/4176728033 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efe1c0054d0 con 0x7efe2c104320 2024-04-24T09:04:41.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.449+0000 7efe2be00640 1 -- 172.21.15.83:0/4176728033 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe2c1a73a0 con 0x7efe2c104320 2024-04-24T09:04:41.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.449+0000 7efe30a00640 1 -- 172.21.15.83:0/4176728033 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe1c004cf0 con 0x7efe2c104320 2024-04-24T09:04:41.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.449+0000 7efe2be00640 1 -- 172.21.15.83:0/4176728033 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe2c1a77a0 con 0x7efe2c104320 2024-04-24T09:04:41.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.452+0000 7efe34c3a740 1 Processor -- start 2024-04-24T09:04:41.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.452+0000 7efe30a00640 1 -- 172.21.15.83:0/4176728033 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7efe1c004e90 con 0x7efe2c104320 2024-04-24T09:04:41.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.452+0000 7efe34c3a740 1 -- start start 2024-04-24T09:04:41.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.453+0000 7efe34c3a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55813b6c5b40 0x55813b6c5f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:41.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.453+0000 7efe34c3a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55813b6c6460 0x55813b6d05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:41.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.453+0000 7efe34c3a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55813b6d0b30 0x55813b6d2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:41.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.453+0000 7efe34c3a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55813b62b1d0 con 0x55813b6d0b30 2024-04-24T09:04:41.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.453+0000 7efe34c3a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55813b5d1450 con 0x55813b6c6460 2024-04-24T09:04:41.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.453+0000 7efe34c3a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55813b627190 con 0x55813b6c5b40 2024-04-24T09:04:41.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.453+0000 7efe2a000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55813b6d0b30 0x55813b6d2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:41.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.453+0000 7efe2a000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55813b6d0b30 0x55813b6d2f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50196/0 (socket says 172.21.15.83:50196) 2024-04-24T09:04:41.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.453+0000 7efe2a000640 1 -- 172.21.15.83:0/3980735948 learned_addr learned my addr 172.21.15.83:0/3980735948 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:04:41.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.453+0000 7efe2aa00640 1 --2- 172.21.15.83:0/3980735948 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55813b6c6460 0x55813b6d05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:41.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.453+0000 7efe2b400640 1 --2- 172.21.15.83:0/3980735948 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55813b6c5b40 0x55813b6c5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:41.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.454+0000 7efe2a000640 1 -- 172.21.15.83:0/3980735948 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55813b6c5b40 msgr2=0x55813b6c5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:41.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.454+0000 7efe2a000640 1 --2- 172.21.15.83:0/3980735948 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55813b6c5b40 0x55813b6c5f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:41.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.454+0000 7efe2a000640 1 -- 172.21.15.83:0/3980735948 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55813b6c6460 msgr2=0x55813b6d05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:41.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.454+0000 7efe2a000640 1 --2- 172.21.15.83:0/3980735948 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55813b6c6460 0x55813b6d05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:41.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.454+0000 7efe2a000640 1 -- 172.21.15.83:0/3980735948 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55813b68e310 con 0x55813b6d0b30 2024-04-24T09:04:41.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.454+0000 7efe2a000640 1 --2- 172.21.15.83:0/3980735948 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55813b6d0b30 0x55813b6d2f20 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x7efe18009b70 tx=0x7efe180117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1478c27e366f4ab8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:41.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.454+0000 7efe30a00640 1 --2- 172.21.15.83:0/4176728033 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7efe100a3da0 0x7efe100a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:41.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.454+0000 7efe30a00640 1 -- 172.21.15.83:0/4176728033 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7efe1c0d06d0 con 0x7efe2c104320 2024-04-24T09:04:41.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.454+0000 7efe17e00640 1 -- 172.21.15.83:0/3980735948 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe18009e60 con 0x55813b6d0b30 2024-04-24T09:04:41.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.455+0000 7efe17e00640 1 -- 172.21.15.83:0/3980735948 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efe18012900 con 0x55813b6d0b30 2024-04-24T09:04:41.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.455+0000 7efe17e00640 1 -- 172.21.15.83:0/3980735948 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe1800fa70 con 0x55813b6d0b30 2024-04-24T09:04:41.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.455+0000 7efe2b400640 1 --2- 172.21.15.83:0/4176728033 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7efe100a3da0 0x7efe100a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:41.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.455+0000 7efe34c3a740 1 -- 172.21.15.83:0/3980735948 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55813b6d0b30 msgr2=0x55813b6d2f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:41.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.455+0000 7efe34c3a740 1 --2- 172.21.15.83:0/3980735948 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55813b6d0b30 0x55813b6d2f20 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x7efe18009b70 tx=0x7efe180117b0 comp rx=0 tx=0).stop 2024-04-24T09:04:41.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.455+0000 7efe2b400640 1 --2- 172.21.15.83:0/4176728033 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7efe100a3da0 0x7efe100a6260 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7efe24031040 tx=0x7efe24005ef0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:41.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.456+0000 7efe34c3a740 1 -- 172.21.15.83:0/3980735948 shutdown_connections 2024-04-24T09:04:41.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.456+0000 7efe34c3a740 1 --2- 172.21.15.83:0/3980735948 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55813b6d0b30 0x55813b6d2f20 unknown :-1 s=CLOSED pgs=2045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:41.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.456+0000 7efe34c3a740 1 --2- 172.21.15.83:0/3980735948 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55813b6c6460 0x55813b6d05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:41.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.456+0000 7efe34c3a740 1 --2- 172.21.15.83:0/3980735948 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55813b6c5b40 0x55813b6c5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:41.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.456+0000 7efe34c3a740 1 -- 172.21.15.83:0/3980735948 >> 172.21.15.83:0/3980735948 conn(0x55813b6c1e60 msgr2=0x55813b6c2240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:41.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.456+0000 7efe34c3a740 1 -- 172.21.15.83:0/3980735948 shutdown_connections 2024-04-24T09:04:41.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.456+0000 7efe34c3a740 1 -- 172.21.15.83:0/3980735948 wait complete. 2024-04-24T09:04:41.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.456+0000 7efe34c3a740 1 -- start start 2024-04-24T09:04:41.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.456+0000 7efe34c3a740 1 Processor -- start 2024-04-24T09:04:41.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.457+0000 7efe34c3a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55813b6c5b40 0x55813b6d0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:41.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.457+0000 7efe34c3a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55813b6c6460 0x55813b6cd870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:41.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.457+0000 7efe34c3a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55813b6d0b30 0x55813b6cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:41.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.457+0000 7efe34c3a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55813b6d39c0 con 0x55813b6c5b40 2024-04-24T09:04:41.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.457+0000 7efe34c3a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55813b5e4810 con 0x55813b6c6460 2024-04-24T09:04:41.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.457+0000 7efe34c3a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55813b6d3b40 con 0x55813b6d0b30 2024-04-24T09:04:41.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.457+0000 7efe34c3a740 10 client.0 _ll_register_callbacks cb 0x55813b5b2140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:04:41.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.457+0000 7efe2aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55813b6c5b40 0x55813b6d0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:41.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.457+0000 7efe2aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55813b6c5b40 0x55813b6d0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50198/0 (socket says 172.21.15.83:50198) 2024-04-24T09:04:41.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.457+0000 7efe2aa00640 1 -- 172.21.15.83:0/1052408000 learned_addr learned my addr 172.21.15.83:0/1052408000 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:04:41.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.457+0000 7efe2a000640 1 --2- 172.21.15.83:0/1052408000 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55813b6c6460 0x55813b6cd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:41.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.457+0000 7efe2aa00640 1 -- 172.21.15.83:0/1052408000 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55813b6d0b30 msgr2=0x55813b6cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:41.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.457+0000 7efe2b400640 1 --2- 172.21.15.83:0/1052408000 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55813b6d0b30 0x55813b6cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:41.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.458+0000 7efe2aa00640 1 --2- 172.21.15.83:0/1052408000 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55813b6d0b30 0x55813b6cdfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:41.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.458+0000 7efe2aa00640 1 -- 172.21.15.83:0/1052408000 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55813b6c6460 msgr2=0x55813b6cd870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:41.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.458+0000 7efe2aa00640 1 --2- 172.21.15.83:0/1052408000 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55813b6c6460 0x55813b6cd870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:41.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.458+0000 7efe2aa00640 1 -- 172.21.15.83:0/1052408000 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55813b5f53f0 con 0x55813b6c5b40 2024-04-24T09:04:41.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.458+0000 7efe2a000640 1 --2- 172.21.15.83:0/1052408000 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55813b6c6460 0x55813b6cd870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:04:41.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.458+0000 7efe2b400640 1 --2- 172.21.15.83:0/1052408000 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55813b6d0b30 0x55813b6cdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:04:41.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.458+0000 7efe2aa00640 1 --2- 172.21.15.83:0/1052408000 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55813b6c5b40 0x55813b6d0580 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7efe1c0cda60 tx=0x7efe1c0985b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f063a8ce7c3d7b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:41.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.458+0000 7efe14c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:04:41.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.458+0000 7efe14c00640 1 -- 172.21.15.83:0/1052408000 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe1c020040 con 0x55813b6c5b40 2024-04-24T09:04:41.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.458+0000 7efe14c00640 1 -- 172.21.15.83:0/1052408000 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7efe1c0d0b80 con 0x55813b6c5b40 2024-04-24T09:04:41.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.458+0000 7efe34c3a740 1 -- 172.21.15.83:0/1052408000 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe18012900 con 0x55813b6c5b40 2024-04-24T09:04:41.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.458+0000 7efe14c00640 1 -- 172.21.15.83:0/1052408000 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7efe1c005180 con 0x55813b6c5b40 2024-04-24T09:04:41.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.458+0000 7efe34c3a740 10 client.20298 fetch_fsmap learned FSMap version 41 2024-04-24T09:04:41.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.458+0000 7efe34c3a740 1 -- 172.21.15.83:0/1052408000 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55813b33d3f0 con 0x55813b6c5b40 2024-04-24T09:04:41.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.459+0000 7efe14c00640 1 -- 172.21.15.83:0/1052408000 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efe18012900 con 0x55813b6c5b40 2024-04-24T09:04:41.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.459+0000 7efe14c00640 1 -- 172.21.15.83:0/1052408000 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7efe1c0d0240 con 0x55813b6c5b40 2024-04-24T09:04:41.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.459+0000 7efe34c3a740 10 client.20298 fetch_fsmap finished waiting for FSMap version 41 2024-04-24T09:04:41.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.459+0000 7efe34c3a740 10 client.20298 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:04:41.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.459+0000 7efe34c3a740 20 client.20298 populate_metadata read hostname 'smithi083' 2024-04-24T09:04:41.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.459+0000 7efe34c3a740 1 --2- 172.21.15.83:0/1052408000 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55813b6e0da0 0x55813b6e3230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:41.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.459+0000 7efe34c3a740 4 client.20298 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:04:41.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.459+0000 7efe34c3a740 1 -- 172.21.15.83:0/1052408000 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55813b638bf0 con 0x55813b6e0da0 2024-04-24T09:04:41.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.460+0000 7efe2a000640 1 --2- 172.21.15.83:0/1052408000 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55813b6e0da0 0x55813b6e3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:41.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.461+0000 7efe2a000640 1 --2- 172.21.15.83:0/1052408000 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55813b6e0da0 0x55813b6e3230 crc :-1 s=READY pgs=339 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f54b23c9db0c8423 server_cookie=337b87d9983a497a in_seq=0 out_seq=0 2024-04-24T09:04:41.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.461+0000 7efe14c00640 10 client.20298 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:04:41.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.463+0000 7efe14c00640 1 -- 172.21.15.83:0/1052408000 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55813b638bf0 con 0x55813b6e0da0 2024-04-24T09:04:41.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.463+0000 7efe14c00640 10 client.20298 handle_command_reply: tid=0 2024-04-24T09:04:41.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.463+0000 7efe34c3a740 1 client.20298 shutdown 2024-04-24T09:04:41.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.463+0000 7efe34c3a740 1 -- 172.21.15.83:0/1052408000 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55813b6c5b40 msgr2=0x55813b6d0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:41.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.463+0000 7efe34c3a740 1 --2- 172.21.15.83:0/1052408000 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55813b6c5b40 0x55813b6d0580 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7efe1c0cda60 tx=0x7efe1c0985b0 comp rx=0 tx=0).stop 2024-04-24T09:04:41.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.463+0000 7efe34c3a740 1 -- 172.21.15.83:0/1052408000 shutdown_connections 2024-04-24T09:04:41.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.463+0000 7efe34c3a740 1 --2- 172.21.15.83:0/1052408000 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55813b6e0da0 0x55813b6e3230 crc :-1 s=READY pgs=339 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:41.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.463+0000 7efe34c3a740 1 --2- 172.21.15.83:0/1052408000 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55813b6d0b30 0x55813b6cdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:41.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.463+0000 7efe34c3a740 1 --2- 172.21.15.83:0/1052408000 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55813b6c6460 0x55813b6cd870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:41.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.463+0000 7efe34c3a740 1 --2- 172.21.15.83:0/1052408000 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55813b6c5b40 0x55813b6d0580 unknown :-1 s=CLOSED pgs=2046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:41.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.463+0000 7efe14c00640 0 client.20298 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:04:41.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.463+0000 7efe34c3a740 1 -- 172.21.15.83:0/1052408000 >> 172.21.15.83:0/1052408000 conn(0x55813b6c1e60 msgr2=0x55813b6c7c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:41.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.464+0000 7efe34c3a740 1 -- 172.21.15.83:0/1052408000 shutdown_connections 2024-04-24T09:04:41.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.464+0000 7efe34c3a740 1 -- 172.21.15.83:0/1052408000 wait complete. 2024-04-24T09:04:41.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.464+0000 7efe34c3a740 20 client.20298 trim_cache size 0 max 16384 2024-04-24T09:04:41.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.479+0000 7efe34c3a740 1 Processor -- start 2024-04-24T09:04:41.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.479+0000 7efe34c3a740 1 -- start start 2024-04-24T09:04:41.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.479+0000 7efe34c3a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55813b6c5b40 0x55813b6dc880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:41.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.479+0000 7efe34c3a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55813b6c6460 0x55813b6cc9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:41.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.479+0000 7efe34c3a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55813b6d0b30 0x55813b6d00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:41.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.479+0000 7efe34c3a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7efe2c1b15c0 con 0x55813b6d0b30 2024-04-24T09:04:41.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.479+0000 7efe34c3a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efe2c1b3220 con 0x55813b6c5b40 2024-04-24T09:04:41.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.479+0000 7efe34c3a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7efe2c1b1c90 con 0x55813b6c6460 2024-04-24T09:04:41.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.480+0000 7efe2b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55813b6d0b30 0x55813b6d00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:41.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.480+0000 7efe2b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55813b6d0b30 0x55813b6d00b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50206/0 (socket says 172.21.15.83:50206) 2024-04-24T09:04:41.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.480+0000 7efe2b400640 1 -- 172.21.15.83:0/2405573888 learned_addr learned my addr 172.21.15.83:0/2405573888 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:04:41.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.480+0000 7efe2aa00640 1 --2- 172.21.15.83:0/2405573888 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55813b6c5b40 0x55813b6dc880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:41.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.480+0000 7efe2a000640 1 --2- 172.21.15.83:0/2405573888 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55813b6c6460 0x55813b6cc9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:41.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.480+0000 7efe2b400640 1 -- 172.21.15.83:0/2405573888 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55813b6c6460 msgr2=0x55813b6cc9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:41.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.480+0000 7efe2b400640 1 --2- 172.21.15.83:0/2405573888 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55813b6c6460 0x55813b6cc9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:41.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.480+0000 7efe2b400640 1 -- 172.21.15.83:0/2405573888 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55813b6c5b40 msgr2=0x55813b6dc880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:41.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.480+0000 7efe2b400640 1 --2- 172.21.15.83:0/2405573888 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55813b6c5b40 0x55813b6dc880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:41.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.480+0000 7efe2b400640 1 -- 172.21.15.83:0/2405573888 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55813b4077d0 con 0x55813b6d0b30 2024-04-24T09:04:41.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.480+0000 7efe2a000640 1 --2- 172.21.15.83:0/2405573888 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55813b6c6460 0x55813b6cc9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:04:41.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.481+0000 7efe2b400640 1 --2- 172.21.15.83:0/2405573888 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55813b6d0b30 0x55813b6d00b0 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 crypto rx=0x7efe24009ca0 tx=0x7efe24040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c59766df976b7646 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:41.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.481+0000 7efe17400640 1 -- 172.21.15.83:0/2405573888 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe24003970 con 0x55813b6d0b30 2024-04-24T09:04:41.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.481+0000 7efe17400640 1 -- 172.21.15.83:0/2405573888 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efe24036040 con 0x55813b6d0b30 2024-04-24T09:04:41.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.481+0000 7efe17400640 1 -- 172.21.15.83:0/2405573888 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe240305c0 con 0x55813b6d0b30 2024-04-24T09:04:41.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.482+0000 7efe34c3a740 1 -- 172.21.15.83:0/2405573888 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55813b6d0b30 msgr2=0x55813b6d00b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:41.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.482+0000 7efe34c3a740 1 --2- 172.21.15.83:0/2405573888 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55813b6d0b30 0x55813b6d00b0 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 crypto rx=0x7efe24009ca0 tx=0x7efe24040000 comp rx=0 tx=0).stop 2024-04-24T09:04:41.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.482+0000 7efe34c3a740 1 -- 172.21.15.83:0/2405573888 shutdown_connections 2024-04-24T09:04:41.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.482+0000 7efe34c3a740 1 --2- 172.21.15.83:0/2405573888 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55813b6d0b30 0x55813b6d00b0 unknown :-1 s=CLOSED pgs=2047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:41.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.482+0000 7efe34c3a740 1 --2- 172.21.15.83:0/2405573888 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55813b6c6460 0x55813b6cc9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:41.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.482+0000 7efe34c3a740 1 --2- 172.21.15.83:0/2405573888 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55813b6c5b40 0x55813b6dc880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:41.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.482+0000 7efe34c3a740 1 -- 172.21.15.83:0/2405573888 >> 172.21.15.83:0/2405573888 conn(0x55813b6c1e60 msgr2=0x55813b6c7290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:41.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.482+0000 7efe34c3a740 1 -- 172.21.15.83:0/2405573888 shutdown_connections 2024-04-24T09:04:41.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.482+0000 7efe34c3a740 1 -- 172.21.15.83:0/2405573888 wait complete. 2024-04-24T09:04:41.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.482+0000 7efe34c3a740 1 -- start start 2024-04-24T09:04:41.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.483+0000 7efe34c3a740 1 Processor -- start 2024-04-24T09:04:41.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.483+0000 7efe34c3a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55813b6c5b40 0x55813b6d9020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:41.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.483+0000 7efe34c3a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55813b6c6460 0x55813b6d14f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:41.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.483+0000 7efe34c3a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55813b6d0b30 0x55813b6d1a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:41.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.483+0000 7efe34c3a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7efe2c1b2af0 con 0x55813b6d0b30 2024-04-24T09:04:41.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.483+0000 7efe34c3a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efe2c1b23c0 con 0x55813b6c6460 2024-04-24T09:04:41.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.483+0000 7efe34c3a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7efe2c1af5c0 con 0x55813b6c5b40 2024-04-24T09:04:41.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.483+0000 7efe34c3a740 10 client.0 _ll_register_callbacks cb 0x7efe1c00d6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:04:41.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.483+0000 7efe2b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55813b6d0b30 0x55813b6d1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:41.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.484+0000 7efe2b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55813b6d0b30 0x55813b6d1a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50214/0 (socket says 172.21.15.83:50214) 2024-04-24T09:04:41.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.484+0000 7efe2b400640 1 -- 172.21.15.83:0/2994752342 learned_addr learned my addr 172.21.15.83:0/2994752342 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:04:41.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.484+0000 7efe2aa00640 1 --2- 172.21.15.83:0/2994752342 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55813b6c5b40 0x55813b6d9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:41.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.484+0000 7efe2b400640 1 -- 172.21.15.83:0/2994752342 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55813b6c5b40 msgr2=0x55813b6d9020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:41.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.484+0000 7efe2b400640 1 --2- 172.21.15.83:0/2994752342 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55813b6c5b40 0x55813b6d9020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:41.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.484+0000 7efe2b400640 1 -- 172.21.15.83:0/2994752342 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55813b6c6460 msgr2=0x55813b6d14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:41.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.484+0000 7efe2b400640 1 --2- 172.21.15.83:0/2994752342 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55813b6c6460 0x55813b6d14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:41.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.484+0000 7efe2b400640 1 -- 172.21.15.83:0/2994752342 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55813b6d2120 con 0x55813b6d0b30 2024-04-24T09:04:41.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.484+0000 7efe2b400640 1 --2- 172.21.15.83:0/2994752342 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55813b6d0b30 0x55813b6d1a30 secure :-1 s=READY pgs=2048 cs=0 l=1 rev1=1 crypto rx=0x7efe24031fe0 tx=0x7efe240307f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=725fe8a123c55f1a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:41.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.484+0000 7efe14c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:04:41.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.484+0000 7efe14c00640 1 -- 172.21.15.83:0/2994752342 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe24057070 con 0x55813b6d0b30 2024-04-24T09:04:41.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.484+0000 7efe14c00640 1 -- 172.21.15.83:0/2994752342 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7efe24030b70 con 0x55813b6d0b30 2024-04-24T09:04:41.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.484+0000 7efe34c3a740 1 -- 172.21.15.83:0/2994752342 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe24036040 con 0x55813b6d0b30 2024-04-24T09:04:41.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.485+0000 7efe14c00640 1 -- 172.21.15.83:0/2994752342 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7efe24030d90 con 0x55813b6d0b30 2024-04-24T09:04:41.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.485+0000 7efe34c3a740 10 client.20304 fetch_fsmap learned FSMap version 41 2024-04-24T09:04:41.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.485+0000 7efe34c3a740 1 -- 172.21.15.83:0/2994752342 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55813b747370 con 0x55813b6d0b30 2024-04-24T09:04:41.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.485+0000 7efe14c00640 1 -- 172.21.15.83:0/2994752342 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efe24036040 con 0x55813b6d0b30 2024-04-24T09:04:41.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.485+0000 7efe14c00640 1 -- 172.21.15.83:0/2994752342 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7efe24044040 con 0x55813b6d0b30 2024-04-24T09:04:41.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.485+0000 7efe34c3a740 10 client.20304 fetch_fsmap finished waiting for FSMap version 41 2024-04-24T09:04:41.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.485+0000 7efe34c3a740 10 client.20304 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:04:41.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.485+0000 7efe34c3a740 20 client.20304 populate_metadata read hostname 'smithi083' 2024-04-24T09:04:41.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.485+0000 7efe34c3a740 1 --2- 172.21.15.83:0/2994752342 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55813b7478e0 0x55813b749d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:41.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.485+0000 7efe34c3a740 4 client.20304 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:04:41.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.485+0000 7efe34c3a740 1 -- 172.21.15.83:0/2994752342 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55813b632a60 con 0x55813b7478e0 2024-04-24T09:04:41.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.486+0000 7efe2aa00640 1 --2- 172.21.15.83:0/2994752342 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55813b7478e0 0x55813b749d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:41.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.487+0000 7efe2aa00640 1 --2- 172.21.15.83:0/2994752342 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55813b7478e0 0x55813b749d20 crc :-1 s=READY pgs=340 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66c026488723dc19 server_cookie=99184db530ccbf8c in_seq=0 out_seq=0 2024-04-24T09:04:41.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.487+0000 7efe14c00640 10 client.20304 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:04:41.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.488+0000 7efe14c00640 1 -- 172.21.15.83:0/2994752342 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55813b632a60 con 0x55813b7478e0 2024-04-24T09:04:41.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.488+0000 7efe14c00640 10 client.20304 handle_command_reply: tid=0 2024-04-24T09:04:41.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.488+0000 7efe34c3a740 1 client.20304 shutdown 2024-04-24T09:04:41.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.488+0000 7efe34c3a740 1 -- 172.21.15.83:0/2994752342 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55813b6d0b30 msgr2=0x55813b6d1a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:41.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.488+0000 7efe34c3a740 1 --2- 172.21.15.83:0/2994752342 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55813b6d0b30 0x55813b6d1a30 secure :-1 s=READY pgs=2048 cs=0 l=1 rev1=1 crypto rx=0x7efe24031fe0 tx=0x7efe240307f0 comp rx=0 tx=0).stop 2024-04-24T09:04:41.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.488+0000 7efe34c3a740 1 -- 172.21.15.83:0/2994752342 shutdown_connections 2024-04-24T09:04:41.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.488+0000 7efe34c3a740 1 --2- 172.21.15.83:0/2994752342 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55813b7478e0 0x55813b749d20 crc :-1 s=READY pgs=340 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:41.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.489+0000 7efe34c3a740 1 --2- 172.21.15.83:0/2994752342 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55813b6d0b30 0x55813b6d1a30 unknown :-1 s=CLOSED pgs=2048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:41.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.489+0000 7efe14c00640 0 client.20304 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:04:41.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.489+0000 7efe34c3a740 1 --2- 172.21.15.83:0/2994752342 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55813b6c6460 0x55813b6d14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:41.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.489+0000 7efe34c3a740 1 --2- 172.21.15.83:0/2994752342 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55813b6c5b40 0x55813b6d9020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:41.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.489+0000 7efe34c3a740 1 -- 172.21.15.83:0/2994752342 >> 172.21.15.83:0/2994752342 conn(0x55813b6c1e60 msgr2=0x55813b6cc850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:41.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.489+0000 7efe34c3a740 1 -- 172.21.15.83:0/2994752342 shutdown_connections 2024-04-24T09:04:41.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.489+0000 7efe34c3a740 1 -- 172.21.15.83:0/2994752342 wait complete. 2024-04-24T09:04:41.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.489+0000 7efe34c3a740 20 client.20304 trim_cache size 0 max 16384 2024-04-24T09:04:41.491 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:04:41.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.489+0000 7efe2be00640 1 -- 172.21.15.83:0/4176728033 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7efe100a3da0 msgr2=0x7efe100a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:41.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.489+0000 7efe2be00640 1 --2- 172.21.15.83:0/4176728033 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7efe100a3da0 0x7efe100a6260 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7efe24031040 tx=0x7efe24005ef0 comp rx=0 tx=0).stop 2024-04-24T09:04:41.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.489+0000 7efe2be00640 1 -- 172.21.15.83:0/4176728033 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efe2c104320 msgr2=0x7efe2c1a0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:41.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.489+0000 7efe2be00640 1 --2- 172.21.15.83:0/4176728033 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efe2c104320 0x7efe2c1a0600 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 crypto rx=0x7efe1c00d770 tx=0x7efe1c00dc30 comp rx=0 tx=0).stop 2024-04-24T09:04:41.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.490+0000 7efe2be00640 1 -- 172.21.15.83:0/4176728033 shutdown_connections 2024-04-24T09:04:41.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.490+0000 7efe2be00640 1 --2- 172.21.15.83:0/4176728033 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7efe100a3da0 0x7efe100a6260 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:41.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.490+0000 7efe2be00640 1 --2- 172.21.15.83:0/4176728033 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efe2c19a6c0 0x7efe2c19ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:41.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.490+0000 7efe2be00640 1 --2- 172.21.15.83:0/4176728033 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe2c1093f0 0x7efe2c1a0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:41.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.490+0000 7efe2be00640 1 --2- 172.21.15.83:0/4176728033 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efe2c104320 0x7efe2c1a0600 unknown :-1 s=CLOSED pgs=2044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:41.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.490+0000 7efe2be00640 1 -- 172.21.15.83:0/4176728033 >> 172.21.15.83:0/4176728033 conn(0x7efe2c0ffb40 msgr2=0x7efe2c101e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:41.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.490+0000 7efe2be00640 1 -- 172.21.15.83:0/4176728033 shutdown_connections 2024-04-24T09:04:41.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:41.490+0000 7efe2be00640 1 -- 172.21.15.83:0/4176728033 wait complete. 2024-04-24T09:04:41.628 INFO:tasks.workunit.client.0.smithi083.stdout: 1 74951 0.59 MB/sec execute 555 sec latency 42.104 ms 2024-04-24T09:04:42.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:04:41 smithi138 ceph-mon[31441]: pgmap v411: 129 pgs: 129 active+clean; 72 MiB data, 985 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 47 KiB/s wr, 19 op/s 2024-04-24T09:04:42.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:04:41 smithi189 ceph-mon[31791]: pgmap v411: 129 pgs: 129 active+clean; 72 MiB data, 985 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 47 KiB/s wr, 19 op/s 2024-04-24T09:04:42.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:04:41 smithi083 ceph-mon[27907]: pgmap v411: 129 pgs: 129 active+clean; 72 MiB data, 985 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 47 KiB/s wr, 19 op/s 2024-04-24T09:04:42.508 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T09:04:42.508 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T09:04:42.508 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T09:04:42.628 INFO:tasks.workunit.client.0.smithi083.stdout: 1 75011 0.58 MB/sec execute 556 sec latency 42.010 ms 2024-04-24T09:04:42.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.736+0000 7f04dda00640 1 -- 172.21.15.83:0/2561379986 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04d8102120 msgr2=0x7f04d8102500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:42.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.736+0000 7f04dda00640 1 --2- 172.21.15.83:0/2561379986 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04d8102120 0x7f04d8102500 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7f04c8009900 tx=0x7f04c802a3f0 comp rx=0 tx=0).stop 2024-04-24T09:04:42.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.736+0000 7f04dda00640 1 -- 172.21.15.83:0/2561379986 shutdown_connections 2024-04-24T09:04:42.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.737+0000 7f04dda00640 1 --2- 172.21.15.83:0/2561379986 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f04d81071f0 0x7f04d810edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:42.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.737+0000 7f04dda00640 1 --2- 172.21.15.83:0/2561379986 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04d8102ad0 0x7f04d8106b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:42.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.737+0000 7f04dda00640 1 --2- 172.21.15.83:0/2561379986 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04d8102120 0x7f04d8102500 unknown :-1 s=CLOSED pgs=2049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:42.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.737+0000 7f04dda00640 1 -- 172.21.15.83:0/2561379986 >> 172.21.15.83:0/2561379986 conn(0x7f04d80fd9a0 msgr2=0x7f04d80ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:42.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.737+0000 7f04dda00640 1 -- 172.21.15.83:0/2561379986 shutdown_connections 2024-04-24T09:04:42.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.737+0000 7f04dda00640 1 -- 172.21.15.83:0/2561379986 wait complete. 2024-04-24T09:04:42.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.738+0000 7f04dda00640 1 Processor -- start 2024-04-24T09:04:42.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.738+0000 7f04dda00640 1 -- start start 2024-04-24T09:04:42.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.738+0000 7f04dda00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04d8102120 0x7f04d81a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:42.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.738+0000 7f04dda00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f04d81071f0 0x7f04d81a0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:42.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.738+0000 7f04dda00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04d819a700 0x7f04d819ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:42.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.738+0000 7f04dda00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f04d8112130 con 0x7f04d8102120 2024-04-24T09:04:42.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.738+0000 7f04dda00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f04d8111fb0 con 0x7f04d819a700 2024-04-24T09:04:42.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.738+0000 7f04dda00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f04d81122b0 con 0x7f04d81071f0 2024-04-24T09:04:42.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.738+0000 7f04d7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04d8102120 0x7f04d81a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:42.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.739+0000 7f04d7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04d8102120 0x7f04d81a0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50224/0 (socket says 172.21.15.83:50224) 2024-04-24T09:04:42.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.739+0000 7f04d7e00640 1 -- 172.21.15.83:0/3308407131 learned_addr learned my addr 172.21.15.83:0/3308407131 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:04:42.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.739+0000 7f04dd000640 1 --2- 172.21.15.83:0/3308407131 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04d819a700 0x7f04d819ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:42.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.739+0000 7f04d7e00640 1 -- 172.21.15.83:0/3308407131 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f04d81071f0 msgr2=0x7f04d81a0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:42.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.739+0000 7f04d7400640 1 --2- 172.21.15.83:0/3308407131 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f04d81071f0 0x7f04d81a0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:42.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.739+0000 7f04d7e00640 1 --2- 172.21.15.83:0/3308407131 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f04d81071f0 0x7f04d81a0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:42.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.739+0000 7f04d7e00640 1 -- 172.21.15.83:0/3308407131 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04d819a700 msgr2=0x7f04d819ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:42.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.739+0000 7f04d7e00640 1 --2- 172.21.15.83:0/3308407131 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04d819a700 0x7f04d819ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:42.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.739+0000 7f04d7e00640 1 -- 172.21.15.83:0/3308407131 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04d819b300 con 0x7f04d8102120 2024-04-24T09:04:42.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.739+0000 7f04d7400640 1 --2- 172.21.15.83:0/3308407131 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f04d81071f0 0x7f04d81a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:04:42.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.739+0000 7f04dd000640 1 --2- 172.21.15.83:0/3308407131 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04d819a700 0x7f04d819ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:04:42.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.739+0000 7f04d7e00640 1 --2- 172.21.15.83:0/3308407131 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04d8102120 0x7f04d81a0640 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 crypto rx=0x7f04c8006010 tx=0x7f04c8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:42.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.740+0000 7f04d4c00640 1 -- 172.21.15.83:0/3308407131 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c803f030 con 0x7f04d8102120 2024-04-24T09:04:42.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.740+0000 7f04d4c00640 1 -- 172.21.15.83:0/3308407131 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f04c80035f0 con 0x7f04d8102120 2024-04-24T09:04:42.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.740+0000 7f04d4c00640 1 -- 172.21.15.83:0/3308407131 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c8035070 con 0x7f04d8102120 2024-04-24T09:04:42.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.740+0000 7f04dda00640 1 -- 172.21.15.83:0/3308407131 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04d819b590 con 0x7f04d8102120 2024-04-24T09:04:42.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.740+0000 7f04dda00640 1 -- 172.21.15.83:0/3308407131 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04d81a7680 con 0x7f04d8102120 2024-04-24T09:04:42.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.743+0000 7f04d4c00640 1 -- 172.21.15.83:0/3308407131 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f04c8003b10 con 0x7f04d8102120 2024-04-24T09:04:42.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.743+0000 7f04e1fbd740 1 Processor -- start 2024-04-24T09:04:42.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.744+0000 7f04e1fbd740 1 -- start start 2024-04-24T09:04:42.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.744+0000 7f04e1fbd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599e5bd5270 0x5599e5bd5650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:42.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.744+0000 7f04e1fbd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5599e5bd5b90 0x5599e5bdfd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:42.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.744+0000 7f04e1fbd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5599e5be0260 0x5599e5be2650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:42.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.744+0000 7f04e1fbd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5599e5b3ae70 con 0x5599e5bd5270 2024-04-24T09:04:42.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.744+0000 7f04e1fbd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5599e5ae1380 con 0x5599e5bd5b90 2024-04-24T09:04:42.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.744+0000 7f04e1fbd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5599e5b36e30 con 0x5599e5be0260 2024-04-24T09:04:42.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.744+0000 7f04dd000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599e5bd5270 0x5599e5bd5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:42.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.744+0000 7f04dd000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599e5bd5270 0x5599e5bd5650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50240/0 (socket says 172.21.15.83:50240) 2024-04-24T09:04:42.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.744+0000 7f04dd000640 1 -- 172.21.15.83:0/744857126 learned_addr learned my addr 172.21.15.83:0/744857126 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:04:42.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.744+0000 7f04d7e00640 1 --2- 172.21.15.83:0/744857126 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5599e5bd5b90 0x5599e5bdfd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:42.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.745+0000 7f04d7400640 1 --2- 172.21.15.83:0/744857126 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5599e5be0260 0x5599e5be2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:42.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.745+0000 7f04dd000640 1 -- 172.21.15.83:0/744857126 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5599e5be0260 msgr2=0x5599e5be2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:42.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.745+0000 7f04dd000640 1 --2- 172.21.15.83:0/744857126 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5599e5be0260 0x5599e5be2650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:42.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.745+0000 7f04dd000640 1 -- 172.21.15.83:0/744857126 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5599e5bd5b90 msgr2=0x5599e5bdfd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:42.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.745+0000 7f04dd000640 1 --2- 172.21.15.83:0/744857126 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5599e5bd5b90 0x5599e5bdfd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:42.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.745+0000 7f04dd000640 1 -- 172.21.15.83:0/744857126 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599e59157e0 con 0x5599e5bd5270 2024-04-24T09:04:42.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.745+0000 7f04d4c00640 1 --2- 172.21.15.83:0/3308407131 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f04bc0a3da0 0x7f04bc0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:42.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.745+0000 7f04dd000640 1 --2- 172.21.15.83:0/744857126 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599e5bd5270 0x5599e5bd5650 secure :-1 s=READY pgs=2051 cs=0 l=1 rev1=1 crypto rx=0x7f04d0008d50 tx=0x7f04d000d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8909c6241d3a0a84 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:42.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.745+0000 7f04d4c00640 1 -- 172.21.15.83:0/3308407131 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7f04c80f14e0 con 0x7f04d8102120 2024-04-24T09:04:42.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.745+0000 7f04d7e00640 1 --2- 172.21.15.83:0/3308407131 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f04bc0a3da0 0x7f04bc0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:42.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.746+0000 7f04cd600640 1 -- 172.21.15.83:0/744857126 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d000f830 con 0x5599e5bd5270 2024-04-24T09:04:42.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.746+0000 7f04cd600640 1 -- 172.21.15.83:0/744857126 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f04d000de20 con 0x5599e5bd5270 2024-04-24T09:04:42.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.746+0000 7f04cd600640 1 -- 172.21.15.83:0/744857126 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d0018dd0 con 0x5599e5bd5270 2024-04-24T09:04:42.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.746+0000 7f04e1fbd740 1 -- 172.21.15.83:0/744857126 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599e5bd5270 msgr2=0x5599e5bd5650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:42.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.746+0000 7f04d7e00640 1 --2- 172.21.15.83:0/3308407131 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f04bc0a3da0 0x7f04bc0a6260 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f04c80ee960 tx=0x7f04c802a3f0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:42.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.746+0000 7f04e1fbd740 1 --2- 172.21.15.83:0/744857126 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599e5bd5270 0x5599e5bd5650 secure :-1 s=READY pgs=2051 cs=0 l=1 rev1=1 crypto rx=0x7f04d0008d50 tx=0x7f04d000d710 comp rx=0 tx=0).stop 2024-04-24T09:04:42.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.747+0000 7f04e1fbd740 1 -- 172.21.15.83:0/744857126 shutdown_connections 2024-04-24T09:04:42.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.747+0000 7f04e1fbd740 1 --2- 172.21.15.83:0/744857126 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5599e5be0260 0x5599e5be2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:42.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.747+0000 7f04e1fbd740 1 --2- 172.21.15.83:0/744857126 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5599e5bd5b90 0x5599e5bdfd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:42.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.747+0000 7f04e1fbd740 1 --2- 172.21.15.83:0/744857126 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599e5bd5270 0x5599e5bd5650 unknown :-1 s=CLOSED pgs=2051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:42.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.747+0000 7f04e1fbd740 1 -- 172.21.15.83:0/744857126 >> 172.21.15.83:0/744857126 conn(0x5599e5bd1590 msgr2=0x5599e5bd1970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:42.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.747+0000 7f04e1fbd740 1 -- 172.21.15.83:0/744857126 shutdown_connections 2024-04-24T09:04:42.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.747+0000 7f04e1fbd740 1 -- 172.21.15.83:0/744857126 wait complete. 2024-04-24T09:04:42.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.747+0000 7f04e1fbd740 1 -- start start 2024-04-24T09:04:42.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.747+0000 7f04e1fbd740 1 Processor -- start 2024-04-24T09:04:42.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.748+0000 7f04e1fbd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5599e5bd5270 0x5599e5bdd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:42.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.748+0000 7f04e1fbd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599e5bd5b90 0x5599e5bdd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:42.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.748+0000 7f04e1fbd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5599e5be0260 0x5599e5bddcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:42.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.748+0000 7f04e1fbd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5599e5be3050 con 0x5599e5bd5b90 2024-04-24T09:04:42.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.748+0000 7f04e1fbd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5599e5af4660 con 0x5599e5bd5270 2024-04-24T09:04:42.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.748+0000 7f04e1fbd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5599e5be31d0 con 0x5599e5be0260 2024-04-24T09:04:42.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.748+0000 7f04e1fbd740 10 client.0 _ll_register_callbacks cb 0x5599e5ae2aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:04:42.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.748+0000 7f04d7400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599e5bd5b90 0x5599e5bdd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:42.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.748+0000 7f04d7400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599e5bd5b90 0x5599e5bdd780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50244/0 (socket says 172.21.15.83:50244) 2024-04-24T09:04:42.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.748+0000 7f04d7400640 1 -- 172.21.15.83:0/315389181 learned_addr learned my addr 172.21.15.83:0/315389181 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:04:42.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.749+0000 7f04dd000640 1 --2- 172.21.15.83:0/315389181 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5599e5bd5270 0x5599e5bdd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:42.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.749+0000 7f04d7400640 1 -- 172.21.15.83:0/315389181 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5599e5be0260 msgr2=0x5599e5bddcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:42.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.749+0000 7f04d7400640 1 --2- 172.21.15.83:0/315389181 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5599e5be0260 0x5599e5bddcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:42.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.749+0000 7f04d7400640 1 -- 172.21.15.83:0/315389181 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5599e5bd5270 msgr2=0x5599e5bdd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:42.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.749+0000 7f04d7400640 1 --2- 172.21.15.83:0/315389181 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5599e5bd5270 0x5599e5bdd240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:42.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.749+0000 7f04d7400640 1 -- 172.21.15.83:0/315389181 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599e5975f70 con 0x5599e5bd5b90 2024-04-24T09:04:42.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.749+0000 7f04d7400640 1 --2- 172.21.15.83:0/315389181 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599e5bd5b90 0x5599e5bdd780 secure :-1 s=READY pgs=2052 cs=0 l=1 rev1=1 crypto rx=0x7f04c000f760 tx=0x7f04c000fc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b86da9106d391aa3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:42.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.749+0000 7f04c6000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:04:42.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.749+0000 7f04c6000640 1 -- 172.21.15.83:0/315389181 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c0016070 con 0x5599e5bd5b90 2024-04-24T09:04:42.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.749+0000 7f04c6000640 1 -- 172.21.15.83:0/315389181 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7f04c000bd10 con 0x5599e5bd5b90 2024-04-24T09:04:42.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.749+0000 7f04e1fbd740 1 -- 172.21.15.83:0/315389181 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04d000de20 con 0x5599e5bd5b90 2024-04-24T09:04:42.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.750+0000 7f04c6000640 1 -- 172.21.15.83:0/315389181 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f04c0024050 con 0x5599e5bd5b90 2024-04-24T09:04:42.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.750+0000 7f04e1fbd740 10 client.20316 fetch_fsmap learned FSMap version 41 2024-04-24T09:04:42.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.750+0000 7f04e1fbd740 1 -- 172.21.15.83:0/315389181 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5599e580d170 con 0x5599e5bd5b90 2024-04-24T09:04:42.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.750+0000 7f04c6000640 1 -- 172.21.15.83:0/315389181 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f04d000de20 con 0x5599e5bd5b90 2024-04-24T09:04:42.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.750+0000 7f04c6000640 1 -- 172.21.15.83:0/315389181 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7f04c0010430 con 0x5599e5bd5b90 2024-04-24T09:04:42.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.750+0000 7f04e1fbd740 10 client.20316 fetch_fsmap finished waiting for FSMap version 41 2024-04-24T09:04:42.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.750+0000 7f04e1fbd740 10 client.20316 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:04:42.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.750+0000 7f04e1fbd740 20 client.20316 populate_metadata read hostname 'smithi083' 2024-04-24T09:04:42.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.750+0000 7f04e1fbd740 1 --2- 172.21.15.83:0/315389181 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5599e5bf0730 0x5599e5bf2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:42.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.750+0000 7f04e1fbd740 4 client.20316 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:04:42.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.750+0000 7f04e1fbd740 1 -- 172.21.15.83:0/315389181 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5599e5b48890 con 0x5599e5bf0730 2024-04-24T09:04:42.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.751+0000 7f04dd000640 1 --2- 172.21.15.83:0/315389181 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5599e5bf0730 0x5599e5bf2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:42.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.752+0000 7f04dd000640 1 --2- 172.21.15.83:0/315389181 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5599e5bf0730 0x5599e5bf2bb0 crc :-1 s=READY pgs=341 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d833fb1a188851c6 server_cookie=b3e01701aa8f30a1 in_seq=0 out_seq=0 2024-04-24T09:04:42.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.752+0000 7f04c6000640 10 client.20316 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:04:42.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.755+0000 7f04c6000640 1 -- 172.21.15.83:0/315389181 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5599e5b48890 con 0x5599e5bf0730 2024-04-24T09:04:42.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.755+0000 7f04c6000640 10 client.20316 handle_command_reply: tid=0 2024-04-24T09:04:42.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.755+0000 7f04e1fbd740 1 client.20316 shutdown 2024-04-24T09:04:42.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.755+0000 7f04e1fbd740 1 -- 172.21.15.83:0/315389181 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599e5bd5b90 msgr2=0x5599e5bdd780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:42.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.755+0000 7f04e1fbd740 1 --2- 172.21.15.83:0/315389181 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599e5bd5b90 0x5599e5bdd780 secure :-1 s=READY pgs=2052 cs=0 l=1 rev1=1 crypto rx=0x7f04c000f760 tx=0x7f04c000fc20 comp rx=0 tx=0).stop 2024-04-24T09:04:42.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.755+0000 7f04e1fbd740 1 -- 172.21.15.83:0/315389181 shutdown_connections 2024-04-24T09:04:42.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.755+0000 7f04e1fbd740 1 --2- 172.21.15.83:0/315389181 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5599e5bf0730 0x5599e5bf2bb0 crc :-1 s=READY pgs=341 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:42.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.755+0000 7f04e1fbd740 1 --2- 172.21.15.83:0/315389181 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5599e5be0260 0x5599e5bddcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:42.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.755+0000 7f04e1fbd740 1 --2- 172.21.15.83:0/315389181 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599e5bd5b90 0x5599e5bdd780 unknown :-1 s=CLOSED pgs=2052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:42.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.755+0000 7f04e1fbd740 1 --2- 172.21.15.83:0/315389181 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5599e5bd5270 0x5599e5bdd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:42.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.755+0000 7f04c6000640 0 client.20316 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:04:42.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.755+0000 7f04e1fbd740 1 -- 172.21.15.83:0/315389181 >> 172.21.15.83:0/315389181 conn(0x5599e5bd1590 msgr2=0x5599e5bd7390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:42.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.755+0000 7f04e1fbd740 1 -- 172.21.15.83:0/315389181 shutdown_connections 2024-04-24T09:04:42.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.755+0000 7f04e1fbd740 1 -- 172.21.15.83:0/315389181 wait complete. 2024-04-24T09:04:42.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.755+0000 7f04e1fbd740 20 client.20316 trim_cache size 0 max 16384 2024-04-24T09:04:42.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.775+0000 7f04e1fbd740 1 Processor -- start 2024-04-24T09:04:42.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.775+0000 7f04e1fbd740 1 -- start start 2024-04-24T09:04:42.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.775+0000 7f04e1fbd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599e5bd5270 0x5599e5bebf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:42.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.775+0000 7f04e1fbd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5599e5bd5b90 0x5599e5bd79d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:42.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.775+0000 7f04e1fbd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5599e5be0260 0x5599e5bdd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:42.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.775+0000 7f04e1fbd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f04d81b16a0 con 0x5599e5bd5270 2024-04-24T09:04:42.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.775+0000 7f04e1fbd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f04d81b3300 con 0x5599e5bd5b90 2024-04-24T09:04:42.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.775+0000 7f04e1fbd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f04d81b1d70 con 0x5599e5be0260 2024-04-24T09:04:42.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.775+0000 7f04dd000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599e5bd5270 0x5599e5bebf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:42.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.776+0000 7f04dd000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599e5bd5270 0x5599e5bebf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50248/0 (socket says 172.21.15.83:50248) 2024-04-24T09:04:42.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.776+0000 7f04dd000640 1 -- 172.21.15.83:0/1988322366 learned_addr learned my addr 172.21.15.83:0/1988322366 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:04:42.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.776+0000 7f04d7400640 1 --2- 172.21.15.83:0/1988322366 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5599e5bd5b90 0x5599e5bd79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:42.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.776+0000 7f04dd000640 1 --2- 172.21.15.83:0/1988322366 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5599e5be0260 0x5599e5bdd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:42.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.776+0000 7f04dd000640 1 -- 172.21.15.83:0/1988322366 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5599e5be0260 msgr2=0x5599e5bdd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:42.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.776+0000 7f04dd000640 1 --2- 172.21.15.83:0/1988322366 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5599e5be0260 0x5599e5bdd910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:42.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.776+0000 7f04dd000640 1 -- 172.21.15.83:0/1988322366 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5599e5bd5b90 msgr2=0x5599e5bd79d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:42.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.776+0000 7f04dd000640 1 --2- 172.21.15.83:0/1988322366 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5599e5bd5b90 0x5599e5bd79d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:42.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.776+0000 7f04dd000640 1 -- 172.21.15.83:0/1988322366 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599e5bcce80 con 0x5599e5bd5270 2024-04-24T09:04:42.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.777+0000 7f04dd000640 1 --2- 172.21.15.83:0/1988322366 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599e5bd5270 0x5599e5bebf10 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 crypto rx=0x7f04d0012040 tx=0x7f04d00186f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e285891a4fc5aa01 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:42.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.777+0000 7f04ccc00640 1 -- 172.21.15.83:0/1988322366 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d002d070 con 0x5599e5bd5270 2024-04-24T09:04:42.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.777+0000 7f04ccc00640 1 -- 172.21.15.83:0/1988322366 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f04d0018500 con 0x5599e5bd5270 2024-04-24T09:04:42.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.777+0000 7f04ccc00640 1 -- 172.21.15.83:0/1988322366 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d000fd80 con 0x5599e5bd5270 2024-04-24T09:04:42.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.778+0000 7f04e1fbd740 1 -- 172.21.15.83:0/1988322366 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599e5bd5270 msgr2=0x5599e5bebf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:42.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.778+0000 7f04e1fbd740 1 --2- 172.21.15.83:0/1988322366 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599e5bd5270 0x5599e5bebf10 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 crypto rx=0x7f04d0012040 tx=0x7f04d00186f0 comp rx=0 tx=0).stop 2024-04-24T09:04:42.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.778+0000 7f04e1fbd740 1 -- 172.21.15.83:0/1988322366 shutdown_connections 2024-04-24T09:04:42.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.778+0000 7f04e1fbd740 1 --2- 172.21.15.83:0/1988322366 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5599e5be0260 0x5599e5bdd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:42.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.778+0000 7f04e1fbd740 1 --2- 172.21.15.83:0/1988322366 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5599e5bd5b90 0x5599e5bd79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:42.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.778+0000 7f04e1fbd740 1 --2- 172.21.15.83:0/1988322366 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599e5bd5270 0x5599e5bebf10 unknown :-1 s=CLOSED pgs=2053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:42.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.778+0000 7f04e1fbd740 1 -- 172.21.15.83:0/1988322366 >> 172.21.15.83:0/1988322366 conn(0x5599e5bd1590 msgr2=0x5599e5bdd200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:42.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.778+0000 7f04e1fbd740 1 -- 172.21.15.83:0/1988322366 shutdown_connections 2024-04-24T09:04:42.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.778+0000 7f04e1fbd740 1 -- 172.21.15.83:0/1988322366 wait complete. 2024-04-24T09:04:42.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.778+0000 7f04e1fbd740 1 -- start start 2024-04-24T09:04:42.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.779+0000 7f04e1fbd740 1 Processor -- start 2024-04-24T09:04:42.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.779+0000 7f04e1fbd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599e5bd5270 0x5599e5be3f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:42.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.779+0000 7f04e1fbd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5599e5bd5b90 0x5599e5be4490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:42.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.779+0000 7f04e1fbd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5599e5be0260 0x5599e5be49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:42.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.779+0000 7f04e1fbd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f04d81b2bd0 con 0x5599e5bd5270 2024-04-24T09:04:42.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.779+0000 7f04e1fbd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f04d81b24a0 con 0x5599e5be0260 2024-04-24T09:04:42.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.779+0000 7f04e1fbd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f04d81af6a0 con 0x5599e5bd5b90 2024-04-24T09:04:42.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.779+0000 7f04e1fbd740 10 client.0 _ll_register_callbacks cb 0x7f04c000d3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:04:42.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.779+0000 7f04dd000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599e5bd5270 0x5599e5be3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:42.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.779+0000 7f04dd000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599e5bd5270 0x5599e5be3f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50250/0 (socket says 172.21.15.83:50250) 2024-04-24T09:04:42.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.779+0000 7f04dd000640 1 -- 172.21.15.83:0/3173573101 learned_addr learned my addr 172.21.15.83:0/3173573101 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:04:42.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.780+0000 7f04d7400640 1 --2- 172.21.15.83:0/3173573101 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5599e5bd5b90 0x5599e5be4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:42.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.780+0000 7f04dd000640 1 -- 172.21.15.83:0/3173573101 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5599e5bd5b90 msgr2=0x5599e5be4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:42.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.780+0000 7f04dd000640 1 --2- 172.21.15.83:0/3173573101 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5599e5bd5b90 0x5599e5be4490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:42.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.780+0000 7f04dd000640 1 -- 172.21.15.83:0/3173573101 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5599e5be0260 msgr2=0x5599e5be49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:42.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.780+0000 7f04dd000640 1 --2- 172.21.15.83:0/3173573101 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5599e5be0260 0x5599e5be49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:42.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.780+0000 7f04dd000640 1 -- 172.21.15.83:0/3173573101 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599e5c55e30 con 0x5599e5bd5270 2024-04-24T09:04:42.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.780+0000 7f04d7400640 1 --2- 172.21.15.83:0/3173573101 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5599e5bd5b90 0x5599e5be4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:04:42.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.780+0000 7f04dd000640 1 --2- 172.21.15.83:0/3173573101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599e5bd5270 0x5599e5be3f50 secure :-1 s=READY pgs=2054 cs=0 l=1 rev1=1 crypto rx=0x7f04d0008ca0 tx=0x7f04d000ddd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca44b57320b9ac35 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:42.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.780+0000 7f04c6000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:04:42.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.780+0000 7f04c6000640 1 -- 172.21.15.83:0/3173573101 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d0036060 con 0x5599e5bd5270 2024-04-24T09:04:42.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.781+0000 7f04c6000640 1 -- 172.21.15.83:0/3173573101 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7f04d000fa00 con 0x5599e5bd5270 2024-04-24T09:04:42.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.781+0000 7f04e1fbd740 1 -- 172.21.15.83:0/3173573101 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04d0018500 con 0x5599e5bd5270 2024-04-24T09:04:42.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.781+0000 7f04c6000640 1 -- 172.21.15.83:0/3173573101 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f04d002d070 con 0x5599e5bd5270 2024-04-24T09:04:42.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.781+0000 7f04e1fbd740 10 client.20322 fetch_fsmap learned FSMap version 41 2024-04-24T09:04:42.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.781+0000 7f04e1fbd740 1 -- 172.21.15.83:0/3173573101 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5599e5c57170 con 0x5599e5bd5270 2024-04-24T09:04:42.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.781+0000 7f04c6000640 1 -- 172.21.15.83:0/3173573101 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f04d0018500 con 0x5599e5bd5270 2024-04-24T09:04:42.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.781+0000 7f04c6000640 1 -- 172.21.15.83:0/3173573101 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7f04d00290b0 con 0x5599e5bd5270 2024-04-24T09:04:42.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.781+0000 7f04e1fbd740 10 client.20322 fetch_fsmap finished waiting for FSMap version 41 2024-04-24T09:04:42.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.781+0000 7f04e1fbd740 10 client.20322 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:04:42.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.781+0000 7f04e1fbd740 20 client.20322 populate_metadata read hostname 'smithi083' 2024-04-24T09:04:42.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.781+0000 7f04e1fbd740 1 --2- 172.21.15.83:0/3173573101 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5599e5c57650 0x5599e5c59a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:42.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.781+0000 7f04e1fbd740 4 client.20322 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T09:04:42.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.781+0000 7f04e1fbd740 1 -- 172.21.15.83:0/3173573101 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5599e5b42700 con 0x5599e5c57650 2024-04-24T09:04:42.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.782+0000 7f04d7400640 1 --2- 172.21.15.83:0/3173573101 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5599e5c57650 0x5599e5c59a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:42.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.783+0000 7f04d7400640 1 --2- 172.21.15.83:0/3173573101 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5599e5c57650 0x5599e5c59a70 crc :-1 s=READY pgs=342 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db907cfde7bd8d21 server_cookie=1543e434e3676dc9 in_seq=0 out_seq=0 2024-04-24T09:04:42.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.783+0000 7f04c6000640 10 client.20322 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:04:42.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.788+0000 7f04c6000640 1 -- 172.21.15.83:0/3173573101 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5599e5b42700 con 0x5599e5c57650 2024-04-24T09:04:42.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.788+0000 7f04c6000640 10 client.20322 handle_command_reply: tid=0 2024-04-24T09:04:42.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.788+0000 7f04e1fbd740 1 client.20322 shutdown 2024-04-24T09:04:42.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.788+0000 7f04e1fbd740 1 -- 172.21.15.83:0/3173573101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599e5bd5270 msgr2=0x5599e5be3f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:42.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.788+0000 7f04e1fbd740 1 --2- 172.21.15.83:0/3173573101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599e5bd5270 0x5599e5be3f50 secure :-1 s=READY pgs=2054 cs=0 l=1 rev1=1 crypto rx=0x7f04d0008ca0 tx=0x7f04d000ddd0 comp rx=0 tx=0).stop 2024-04-24T09:04:42.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.788+0000 7f04e1fbd740 1 -- 172.21.15.83:0/3173573101 shutdown_connections 2024-04-24T09:04:42.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.788+0000 7f04e1fbd740 1 --2- 172.21.15.83:0/3173573101 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5599e5c57650 0x5599e5c59a70 crc :-1 s=READY pgs=342 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:42.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.788+0000 7f04e1fbd740 1 --2- 172.21.15.83:0/3173573101 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5599e5be0260 0x5599e5be49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:42.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.788+0000 7f04e1fbd740 1 --2- 172.21.15.83:0/3173573101 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5599e5bd5b90 0x5599e5be4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:42.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.788+0000 7f04c6000640 0 client.20322 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:04:42.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.788+0000 7f04e1fbd740 1 --2- 172.21.15.83:0/3173573101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5599e5bd5270 0x5599e5be3f50 unknown :-1 s=CLOSED pgs=2054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:42.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.788+0000 7f04e1fbd740 1 -- 172.21.15.83:0/3173573101 >> 172.21.15.83:0/3173573101 conn(0x5599e5bd1590 msgr2=0x5599e5bdda90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:42.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.788+0000 7f04e1fbd740 1 -- 172.21.15.83:0/3173573101 shutdown_connections 2024-04-24T09:04:42.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.788+0000 7f04e1fbd740 1 -- 172.21.15.83:0/3173573101 wait complete. 2024-04-24T09:04:42.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.789+0000 7f04e1fbd740 20 client.20322 trim_cache size 0 max 16384 2024-04-24T09:04:42.790 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T09:04:42.790 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-04-24T09:04:42.790 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "3a540d73-d94d-43b8-b95c-25e071105ec0", 2024-04-24T09:04:42.790 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-04-24T09:04:42.790 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T09:04:42.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.789+0000 7f04dda00640 1 -- 172.21.15.83:0/3308407131 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f04bc0a3da0 msgr2=0x7f04bc0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:42.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.789+0000 7f04dda00640 1 --2- 172.21.15.83:0/3308407131 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f04bc0a3da0 0x7f04bc0a6260 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f04c80ee960 tx=0x7f04c802a3f0 comp rx=0 tx=0).stop 2024-04-24T09:04:42.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.789+0000 7f04dda00640 1 -- 172.21.15.83:0/3308407131 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04d8102120 msgr2=0x7f04d81a0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:42.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.789+0000 7f04dda00640 1 --2- 172.21.15.83:0/3308407131 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04d8102120 0x7f04d81a0640 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 crypto rx=0x7f04c8006010 tx=0x7f04c8004270 comp rx=0 tx=0).stop 2024-04-24T09:04:42.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.789+0000 7f04dda00640 1 -- 172.21.15.83:0/3308407131 shutdown_connections 2024-04-24T09:04:42.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.789+0000 7f04dda00640 1 --2- 172.21.15.83:0/3308407131 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f04bc0a3da0 0x7f04bc0a6260 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:42.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.789+0000 7f04dda00640 1 --2- 172.21.15.83:0/3308407131 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04d819a700 0x7f04d819ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:42.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.789+0000 7f04dda00640 1 --2- 172.21.15.83:0/3308407131 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f04d81071f0 0x7f04d81a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:42.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.789+0000 7f04dda00640 1 --2- 172.21.15.83:0/3308407131 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04d8102120 0x7f04d81a0640 unknown :-1 s=CLOSED pgs=2050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:42.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.789+0000 7f04dda00640 1 -- 172.21.15.83:0/3308407131 >> 172.21.15.83:0/3308407131 conn(0x7f04d80fd9a0 msgr2=0x7f04d80ffd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:42.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.790+0000 7f04dda00640 1 -- 172.21.15.83:0/3308407131 shutdown_connections 2024-04-24T09:04:42.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:42.790+0000 7f04dda00640 1 -- 172.21.15.83:0/3308407131 wait complete. 2024-04-24T09:04:42.809 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T09:04:42.809 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T09:04:43.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.018+0000 7fb499e00640 1 -- 172.21.15.83:0/3709207852 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb494101140 msgr2=0x7fb494101520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:43.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.018+0000 7fb499e00640 1 --2- 172.21.15.83:0/3709207852 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb494101140 0x7fb494101520 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x7fb484009900 tx=0x7fb48402a3f0 comp rx=0 tx=0).stop 2024-04-24T09:04:43.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.018+0000 7fb499e00640 1 -- 172.21.15.83:0/3709207852 shutdown_connections 2024-04-24T09:04:43.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.018+0000 7fb499e00640 1 --2- 172.21.15.83:0/3709207852 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb494106210 0x7fb49410dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:43.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.018+0000 7fb499e00640 1 --2- 172.21.15.83:0/3709207852 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb494101af0 0x7fb494105b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:43.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.018+0000 7fb499e00640 1 --2- 172.21.15.83:0/3709207852 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb494101140 0x7fb494101520 unknown :-1 s=CLOSED pgs=2055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:43.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.018+0000 7fb499e00640 1 -- 172.21.15.83:0/3709207852 >> 172.21.15.83:0/3709207852 conn(0x7fb494078000 msgr2=0x7fb494078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:43.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.018+0000 7fb499e00640 1 -- 172.21.15.83:0/3709207852 shutdown_connections 2024-04-24T09:04:43.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.018+0000 7fb499e00640 1 -- 172.21.15.83:0/3709207852 wait complete. 2024-04-24T09:04:43.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.019+0000 7fb499e00640 1 Processor -- start 2024-04-24T09:04:43.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.019+0000 7fb499e00640 1 -- start start 2024-04-24T09:04:43.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.019+0000 7fb499e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb494101140 0x7fb4941a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:43.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.019+0000 7fb499e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb494106210 0x7fb4941a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:43.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.019+0000 7fb499e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb49419a7e0 0x7fb49419ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:43.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.019+0000 7fb499e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb494111180 con 0x7fb494101140 2024-04-24T09:04:43.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.019+0000 7fb499e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb494111000 con 0x7fb49419a7e0 2024-04-24T09:04:43.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.019+0000 7fb499e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb494111300 con 0x7fb494106210 2024-04-24T09:04:43.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.020+0000 7fb492a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb494101140 0x7fb4941a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:43.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.020+0000 7fb492a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb494101140 0x7fb4941a0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50262/0 (socket says 172.21.15.83:50262) 2024-04-24T09:04:43.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.020+0000 7fb492a00640 1 -- 172.21.15.83:0/1786687414 learned_addr learned my addr 172.21.15.83:0/1786687414 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:04:43.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.020+0000 7fb492a00640 1 -- 172.21.15.83:0/1786687414 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb494106210 msgr2=0x7fb4941a0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:43.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.020+0000 7fb492a00640 1 --2- 172.21.15.83:0/1786687414 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb494106210 0x7fb4941a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:43.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.020+0000 7fb492a00640 1 -- 172.21.15.83:0/1786687414 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb49419a7e0 msgr2=0x7fb49419ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:43.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.020+0000 7fb493400640 1 --2- 172.21.15.83:0/1786687414 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb49419a7e0 0x7fb49419ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:43.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.020+0000 7fb492a00640 1 --2- 172.21.15.83:0/1786687414 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb49419a7e0 0x7fb49419ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:43.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.020+0000 7fb492a00640 1 -- 172.21.15.83:0/1786687414 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb49419b370 con 0x7fb494101140 2024-04-24T09:04:43.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.020+0000 7fb492a00640 1 --2- 172.21.15.83:0/1786687414 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb494101140 0x7fb4941a0670 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x7fb484006010 tx=0x7fb484004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:43.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.021+0000 7fb498a00640 1 -- 172.21.15.83:0/1786687414 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb48403f030 con 0x7fb494101140 2024-04-24T09:04:43.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.021+0000 7fb498a00640 1 -- 172.21.15.83:0/1786687414 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb4840035f0 con 0x7fb494101140 2024-04-24T09:04:43.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.021+0000 7fb499e00640 1 -- 172.21.15.83:0/1786687414 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb49419b600 con 0x7fb494101140 2024-04-24T09:04:43.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.021+0000 7fb499e00640 1 -- 172.21.15.83:0/1786687414 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4941a77b0 con 0x7fb494101140 2024-04-24T09:04:43.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.021+0000 7fb498a00640 1 -- 172.21.15.83:0/1786687414 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb484049060 con 0x7fb494101140 2024-04-24T09:04:43.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.024+0000 7fb49cbfa740 1 Processor -- start 2024-04-24T09:04:43.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.025+0000 7fb49cbfa740 1 -- start start 2024-04-24T09:04:43.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.025+0000 7fb49cbfa740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d71373f860 0x55d71373fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:43.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.025+0000 7fb49cbfa740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d713740180 0x55d71374a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:43.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.025+0000 7fb49cbfa740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d71374a850 0x55d71374cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:43.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.025+0000 7fb49cbfa740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d7136a51f0 con 0x55d71374a850 2024-04-24T09:04:43.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.025+0000 7fb49cbfa740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d71364b350 con 0x55d71373f860 2024-04-24T09:04:43.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.025+0000 7fb49cbfa740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d7136a11b0 con 0x55d713740180 2024-04-24T09:04:43.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.025+0000 7fb498a00640 1 -- 172.21.15.83:0/1786687414 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb484003b10 con 0x7fb494101140 2024-04-24T09:04:43.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.025+0000 7fb492000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d71374a850 0x55d71374cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:43.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.025+0000 7fb492000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d71374a850 0x55d71374cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50276/0 (socket says 172.21.15.83:50276) 2024-04-24T09:04:43.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.025+0000 7fb492000640 1 -- 172.21.15.83:0/521700667 learned_addr learned my addr 172.21.15.83:0/521700667 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:04:43.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.026+0000 7fb493400640 1 --2- 172.21.15.83:0/521700667 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d71373f860 0x55d71373fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:43.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.026+0000 7fb492000640 1 -- 172.21.15.83:0/521700667 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d713740180 msgr2=0x55d71374a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:43.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.026+0000 7fb492000640 1 --2- 172.21.15.83:0/521700667 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d713740180 0x55d71374a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:43.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.026+0000 7fb492000640 1 -- 172.21.15.83:0/521700667 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d71373f860 msgr2=0x55d71373fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:43.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.026+0000 7fb492000640 1 --2- 172.21.15.83:0/521700667 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d71373f860 0x55d71373fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:43.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.026+0000 7fb492000640 1 -- 172.21.15.83:0/521700667 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d71369ff50 con 0x55d71374a850 2024-04-24T09:04:43.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.026+0000 7fb492000640 1 --2- 172.21.15.83:0/521700667 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d71374a850 0x55d71374cc40 secure :-1 s=READY pgs=2057 cs=0 l=1 rev1=1 crypto rx=0x7fb48000aa10 tx=0x7fb48000aed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bead6c77dcbf385 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:43.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.026+0000 7fb47fe00640 1 -- 172.21.15.83:0/521700667 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb48000cde0 con 0x55d71374a850 2024-04-24T09:04:43.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.027+0000 7fb47fe00640 1 -- 172.21.15.83:0/521700667 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb48000f900 con 0x55d71374a850 2024-04-24T09:04:43.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.027+0000 7fb47fe00640 1 -- 172.21.15.83:0/521700667 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4800189d0 con 0x55d71374a850 2024-04-24T09:04:43.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.027+0000 7fb498a00640 1 --2- 172.21.15.83:0/1786687414 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb4780a3d20 0x7fb4780a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:43.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.027+0000 7fb498a00640 1 -- 172.21.15.83:0/1786687414 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7fb4840f1830 con 0x7fb494101140 2024-04-24T09:04:43.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.027+0000 7fb49cbfa740 1 -- 172.21.15.83:0/521700667 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d71374a850 msgr2=0x55d71374cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:43.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.027+0000 7fb49cbfa740 1 --2- 172.21.15.83:0/521700667 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d71374a850 0x55d71374cc40 secure :-1 s=READY pgs=2057 cs=0 l=1 rev1=1 crypto rx=0x7fb48000aa10 tx=0x7fb48000aed0 comp rx=0 tx=0).stop 2024-04-24T09:04:43.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.027+0000 7fb493400640 1 --2- 172.21.15.83:0/1786687414 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb4780a3d20 0x7fb4780a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:43.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.028+0000 7fb49cbfa740 1 -- 172.21.15.83:0/521700667 shutdown_connections 2024-04-24T09:04:43.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.028+0000 7fb49cbfa740 1 --2- 172.21.15.83:0/521700667 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d71374a850 0x55d71374cc40 unknown :-1 s=CLOSED pgs=2057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:43.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.028+0000 7fb49cbfa740 1 --2- 172.21.15.83:0/521700667 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d713740180 0x55d71374a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:43.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.028+0000 7fb49cbfa740 1 --2- 172.21.15.83:0/521700667 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d71373f860 0x55d71373fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:43.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.028+0000 7fb49cbfa740 1 -- 172.21.15.83:0/521700667 >> 172.21.15.83:0/521700667 conn(0x55d71373bb80 msgr2=0x55d71373bf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:43.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.028+0000 7fb49cbfa740 1 -- 172.21.15.83:0/521700667 shutdown_connections 2024-04-24T09:04:43.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.028+0000 7fb49cbfa740 1 -- 172.21.15.83:0/521700667 wait complete. 2024-04-24T09:04:43.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.028+0000 7fb49cbfa740 1 -- start start 2024-04-24T09:04:43.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.028+0000 7fb493400640 1 --2- 172.21.15.83:0/1786687414 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb4780a3d20 0x7fb4780a61e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fb48c00bd30 tx=0x7fb48c00b300 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:43.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.029+0000 7fb49cbfa740 1 Processor -- start 2024-04-24T09:04:43.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.029+0000 7fb49cbfa740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d71373f860 0x55d7137476d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:43.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.029+0000 7fb49cbfa740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d713740180 0x55d713747c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:43.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.029+0000 7fb49cbfa740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d71374a850 0x55d713748150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:43.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.029+0000 7fb49cbfa740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d71374d640 con 0x55d713740180 2024-04-24T09:04:43.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.029+0000 7fb49cbfa740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d71365e690 con 0x55d71374a850 2024-04-24T09:04:43.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.029+0000 7fb49cbfa740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d71374d7c0 con 0x55d71373f860 2024-04-24T09:04:43.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.030+0000 7fb492000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d713740180 0x55d713747c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:43.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.030+0000 7fb492000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d713740180 0x55d713747c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50286/0 (socket says 172.21.15.83:50286) 2024-04-24T09:04:43.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.030+0000 7fb492000640 1 -- 172.21.15.83:0/3768897274 learned_addr learned my addr 172.21.15.83:0/3768897274 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:04:43.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.030+0000 7fb49cbfa740 10 client.0 _ll_register_callbacks cb 0x55d71369a650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:04:43.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.030+0000 7fb492a00640 1 --2- 172.21.15.83:0/3768897274 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d71373f860 0x55d7137476d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:43.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.030+0000 7fb493400640 1 --2- 172.21.15.83:0/3768897274 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d71374a850 0x55d713748150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:43.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.030+0000 7fb492000640 1 -- 172.21.15.83:0/3768897274 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d71373f860 msgr2=0x55d7137476d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:43.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.030+0000 7fb492000640 1 --2- 172.21.15.83:0/3768897274 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d71373f860 0x55d7137476d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:43.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.030+0000 7fb492000640 1 -- 172.21.15.83:0/3768897274 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d71374a850 msgr2=0x55d713748150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:43.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.030+0000 7fb492000640 1 --2- 172.21.15.83:0/3768897274 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d71374a850 0x55d713748150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:43.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.031+0000 7fb492000640 1 -- 172.21.15.83:0/3768897274 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d71365d920 con 0x55d713740180 2024-04-24T09:04:43.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.031+0000 7fb492000640 1 --2- 172.21.15.83:0/3768897274 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d713740180 0x55d713747c10 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 crypto rx=0x7fb480005b50 tx=0x7fb48000f680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=463dd39971280585 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:43.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.031+0000 7fb47cc00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:04:43.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.031+0000 7fb47cc00640 1 -- 172.21.15.83:0/3768897274 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb480028060 con 0x55d713740180 2024-04-24T09:04:43.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.031+0000 7fb47cc00640 1 -- 172.21.15.83:0/3768897274 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7fb48000fcc0 con 0x55d713740180 2024-04-24T09:04:43.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.031+0000 7fb49cbfa740 1 -- 172.21.15.83:0/3768897274 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb48000f900 con 0x55d713740180 2024-04-24T09:04:43.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.032+0000 7fb47cc00640 1 -- 172.21.15.83:0/3768897274 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fb48002f050 con 0x55d713740180 2024-04-24T09:04:43.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.032+0000 7fb49cbfa740 10 client.20334 fetch_fsmap learned FSMap version 41 2024-04-24T09:04:43.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.032+0000 7fb49cbfa740 1 -- 172.21.15.83:0/3768897274 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55d7134817d0 con 0x55d713740180 2024-04-24T09:04:43.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.032+0000 7fb47cc00640 1 -- 172.21.15.83:0/3768897274 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb48000f900 con 0x55d713740180 2024-04-24T09:04:43.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.032+0000 7fb47cc00640 1 -- 172.21.15.83:0/3768897274 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7fb4800237e0 con 0x55d713740180 2024-04-24T09:04:43.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.032+0000 7fb49cbfa740 10 client.20334 fetch_fsmap finished waiting for FSMap version 41 2024-04-24T09:04:43.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.032+0000 7fb49cbfa740 10 client.20334 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:04:43.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.032+0000 7fb49cbfa740 20 client.20334 populate_metadata read hostname 'smithi083' 2024-04-24T09:04:43.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.032+0000 7fb49cbfa740 1 --2- 172.21.15.83:0/3768897274 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d71375aa90 0x55d71375ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:43.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.032+0000 7fb49cbfa740 4 client.20334 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:04:43.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.032+0000 7fb49cbfa740 1 -- 172.21.15.83:0/3768897274 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d7136b2c10 con 0x55d71375aa90 2024-04-24T09:04:43.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.033+0000 7fb492a00640 1 --2- 172.21.15.83:0/3768897274 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d71375aa90 0x55d71375ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:43.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.034+0000 7fb492a00640 1 --2- 172.21.15.83:0/3768897274 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d71375aa90 0x55d71375ce80 crc :-1 s=READY pgs=343 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7373933183f610c server_cookie=2630524844134b48 in_seq=0 out_seq=0 2024-04-24T09:04:43.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.034+0000 7fb47cc00640 10 client.20334 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:04:43.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.036+0000 7fb47cc00640 1 -- 172.21.15.83:0/3768897274 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d7136b2c10 con 0x55d71375aa90 2024-04-24T09:04:43.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.037+0000 7fb47cc00640 10 client.20334 handle_command_reply: tid=0 2024-04-24T09:04:43.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.037+0000 7fb49cbfa740 1 client.20334 shutdown 2024-04-24T09:04:43.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.037+0000 7fb49cbfa740 1 -- 172.21.15.83:0/3768897274 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d713740180 msgr2=0x55d713747c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:43.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.037+0000 7fb49cbfa740 1 --2- 172.21.15.83:0/3768897274 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d713740180 0x55d713747c10 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 crypto rx=0x7fb480005b50 tx=0x7fb48000f680 comp rx=0 tx=0).stop 2024-04-24T09:04:43.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.037+0000 7fb49cbfa740 1 -- 172.21.15.83:0/3768897274 shutdown_connections 2024-04-24T09:04:43.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.037+0000 7fb49cbfa740 1 --2- 172.21.15.83:0/3768897274 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d71375aa90 0x55d71375ce80 crc :-1 s=READY pgs=343 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:43.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.037+0000 7fb49cbfa740 1 --2- 172.21.15.83:0/3768897274 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d71374a850 0x55d713748150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:43.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.037+0000 7fb49cbfa740 1 --2- 172.21.15.83:0/3768897274 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d713740180 0x55d713747c10 unknown :-1 s=CLOSED pgs=2058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:43.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.037+0000 7fb49cbfa740 1 --2- 172.21.15.83:0/3768897274 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d71373f860 0x55d7137476d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:43.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.037+0000 7fb49cbfa740 1 -- 172.21.15.83:0/3768897274 >> 172.21.15.83:0/3768897274 conn(0x55d71373bb80 msgr2=0x55d713741980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:43.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.037+0000 7fb47cc00640 0 client.20334 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:04:43.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.037+0000 7fb49cbfa740 1 -- 172.21.15.83:0/3768897274 shutdown_connections 2024-04-24T09:04:43.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.037+0000 7fb49cbfa740 1 -- 172.21.15.83:0/3768897274 wait complete. 2024-04-24T09:04:43.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.037+0000 7fb49cbfa740 20 client.20334 trim_cache size 0 max 16384 2024-04-24T09:04:43.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.058+0000 7fb49cbfa740 1 Processor -- start 2024-04-24T09:04:43.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.058+0000 7fb49cbfa740 1 -- start start 2024-04-24T09:04:43.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.058+0000 7fb49cbfa740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d71373f860 0x55d7137420d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:43.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.058+0000 7fb49cbfa740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d713740180 0x55d713756530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:43.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.058+0000 7fb49cbfa740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d71374a850 0x55d71373aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:43.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.058+0000 7fb49cbfa740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb4941b1740 con 0x55d713740180 2024-04-24T09:04:43.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.058+0000 7fb49cbfa740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb4941b33a0 con 0x55d71374a850 2024-04-24T09:04:43.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.058+0000 7fb49cbfa740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb4941b1e10 con 0x55d71373f860 2024-04-24T09:04:43.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.059+0000 7fb492000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d713740180 0x55d713756530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:43.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.059+0000 7fb492000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d713740180 0x55d713756530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50296/0 (socket says 172.21.15.83:50296) 2024-04-24T09:04:43.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.059+0000 7fb492000640 1 -- 172.21.15.83:0/3665051493 learned_addr learned my addr 172.21.15.83:0/3665051493 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:04:43.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.059+0000 7fb492a00640 1 --2- 172.21.15.83:0/3665051493 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d71373f860 0x55d7137420d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:43.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.059+0000 7fb493400640 1 --2- 172.21.15.83:0/3665051493 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d71374a850 0x55d71373aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:43.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.059+0000 7fb492000640 1 -- 172.21.15.83:0/3665051493 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d71373f860 msgr2=0x55d7137420d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:43.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.059+0000 7fb492000640 1 --2- 172.21.15.83:0/3665051493 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d71373f860 0x55d7137420d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:43.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.059+0000 7fb492000640 1 -- 172.21.15.83:0/3665051493 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d71374a850 msgr2=0x55d71373aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:43.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.059+0000 7fb492000640 1 --2- 172.21.15.83:0/3665051493 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d71374a850 0x55d71373aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:43.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.059+0000 7fb492000640 1 -- 172.21.15.83:0/3665051493 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7133b73f0 con 0x55d713740180 2024-04-24T09:04:43.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.059+0000 7fb492000640 1 --2- 172.21.15.83:0/3665051493 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d713740180 0x55d713756530 secure :-1 s=READY pgs=2059 cs=0 l=1 rev1=1 crypto rx=0x7fb480023d00 tx=0x7fb48000c6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d8605b87dac66f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:43.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.059+0000 7fb47f400640 1 -- 172.21.15.83:0/3665051493 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb480028070 con 0x55d713740180 2024-04-24T09:04:43.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.060+0000 7fb47f400640 1 -- 172.21.15.83:0/3665051493 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb48000a280 con 0x55d713740180 2024-04-24T09:04:43.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.060+0000 7fb47f400640 1 -- 172.21.15.83:0/3665051493 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb480005360 con 0x55d713740180 2024-04-24T09:04:43.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.060+0000 7fb49cbfa740 1 -- 172.21.15.83:0/3665051493 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d713740180 msgr2=0x55d713756530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:43.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.060+0000 7fb49cbfa740 1 --2- 172.21.15.83:0/3665051493 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d713740180 0x55d713756530 secure :-1 s=READY pgs=2059 cs=0 l=1 rev1=1 crypto rx=0x7fb480023d00 tx=0x7fb48000c6c0 comp rx=0 tx=0).stop 2024-04-24T09:04:43.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.060+0000 7fb49cbfa740 1 -- 172.21.15.83:0/3665051493 shutdown_connections 2024-04-24T09:04:43.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.060+0000 7fb49cbfa740 1 --2- 172.21.15.83:0/3665051493 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d71374a850 0x55d71373aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:43.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.060+0000 7fb49cbfa740 1 --2- 172.21.15.83:0/3665051493 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d713740180 0x55d713756530 unknown :-1 s=CLOSED pgs=2059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:43.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.060+0000 7fb49cbfa740 1 --2- 172.21.15.83:0/3665051493 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d71373f860 0x55d7137420d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:43.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.060+0000 7fb49cbfa740 1 -- 172.21.15.83:0/3665051493 >> 172.21.15.83:0/3665051493 conn(0x55d71373bb80 msgr2=0x55d71373a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:43.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.061+0000 7fb49cbfa740 1 -- 172.21.15.83:0/3665051493 shutdown_connections 2024-04-24T09:04:43.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.061+0000 7fb49cbfa740 1 -- 172.21.15.83:0/3665051493 wait complete. 2024-04-24T09:04:43.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.061+0000 7fb49cbfa740 1 -- start start 2024-04-24T09:04:43.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.061+0000 7fb49cbfa740 1 Processor -- start 2024-04-24T09:04:43.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.062+0000 7fb49cbfa740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d71373f860 0x55d7137526a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:43.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.062+0000 7fb49cbfa740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d713740180 0x55d713752be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:43.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.062+0000 7fb49cbfa740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d71374a850 0x55d71374db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:43.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.062+0000 7fb49cbfa740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb4941b2c70 con 0x55d71373f860 2024-04-24T09:04:43.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.062+0000 7fb49cbfa740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb4941b2540 con 0x55d71374a850 2024-04-24T09:04:43.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.062+0000 7fb49cbfa740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb4941af740 con 0x55d713740180 2024-04-24T09:04:43.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.062+0000 7fb49cbfa740 10 client.0 _ll_register_callbacks cb 0x7fb48000a640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:04:43.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.062+0000 7fb492a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d71373f860 0x55d7137526a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:43.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.062+0000 7fb492a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d71373f860 0x55d7137526a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50308/0 (socket says 172.21.15.83:50308) 2024-04-24T09:04:43.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.062+0000 7fb492a00640 1 -- 172.21.15.83:0/3515040669 learned_addr learned my addr 172.21.15.83:0/3515040669 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:04:43.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.062+0000 7fb492000640 1 --2- 172.21.15.83:0/3515040669 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d713740180 0x55d713752be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:43.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.062+0000 7fb493400640 1 --2- 172.21.15.83:0/3515040669 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d71374a850 0x55d71374db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:43.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.062+0000 7fb492a00640 1 -- 172.21.15.83:0/3515040669 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d713740180 msgr2=0x55d713752be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:43.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.062+0000 7fb492a00640 1 --2- 172.21.15.83:0/3515040669 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d713740180 0x55d713752be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:43.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.062+0000 7fb492a00640 1 -- 172.21.15.83:0/3515040669 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d71374a850 msgr2=0x55d71374db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:43.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.062+0000 7fb492a00640 1 --2- 172.21.15.83:0/3515040669 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d71374a850 0x55d71374db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:43.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.062+0000 7fb492a00640 1 -- 172.21.15.83:0/3515040669 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7133770f0 con 0x55d71373f860 2024-04-24T09:04:43.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.063+0000 7fb492a00640 1 --2- 172.21.15.83:0/3515040669 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d71373f860 0x55d7137526a0 secure :-1 s=READY pgs=2060 cs=0 l=1 rev1=1 crypto rx=0x7fb4840eebc0 tx=0x7fb484003860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61d2476fe1586150 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:43.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.063+0000 7fb47cc00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:04:43.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.063+0000 7fb47cc00640 1 -- 172.21.15.83:0/3515040669 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4840a6070 con 0x55d71373f860 2024-04-24T09:04:43.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.063+0000 7fb47cc00640 1 -- 172.21.15.83:0/3515040669 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7fb4840eede0 con 0x55d71373f860 2024-04-24T09:04:43.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.063+0000 7fb49cbfa740 1 -- 172.21.15.83:0/3515040669 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb48000a280 con 0x55d71373f860 2024-04-24T09:04:43.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.063+0000 7fb47cc00640 1 -- 172.21.15.83:0/3515040669 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fb484035040 con 0x55d71373f860 2024-04-24T09:04:43.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.063+0000 7fb49cbfa740 10 client.20340 fetch_fsmap learned FSMap version 41 2024-04-24T09:04:43.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.063+0000 7fb49cbfa740 1 -- 172.21.15.83:0/3515040669 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55d71366a810 con 0x55d71373f860 2024-04-24T09:04:43.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.063+0000 7fb47cc00640 1 -- 172.21.15.83:0/3515040669 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb48000a280 con 0x55d71373f860 2024-04-24T09:04:43.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.064+0000 7fb47cc00640 1 -- 172.21.15.83:0/3515040669 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7fb4840a2ce0 con 0x55d71373f860 2024-04-24T09:04:43.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.064+0000 7fb49cbfa740 10 client.20340 fetch_fsmap finished waiting for FSMap version 41 2024-04-24T09:04:43.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.064+0000 7fb49cbfa740 10 client.20340 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:04:43.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.064+0000 7fb49cbfa740 20 client.20340 populate_metadata read hostname 'smithi083' 2024-04-24T09:04:43.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.064+0000 7fb49cbfa740 1 --2- 172.21.15.83:0/3515040669 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d7137c0b90 0x55d7137c2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:43.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.064+0000 7fb49cbfa740 4 client.20340 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T09:04:43.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.064+0000 7fb49cbfa740 1 -- 172.21.15.83:0/3515040669 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d7136aca80 con 0x55d7137c0b90 2024-04-24T09:04:43.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.064+0000 7fb492000640 1 --2- 172.21.15.83:0/3515040669 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d7137c0b90 0x55d7137c2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:43.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.065+0000 7fb492000640 1 --2- 172.21.15.83:0/3515040669 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d7137c0b90 0x55d7137c2fb0 crc :-1 s=READY pgs=344 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45596bac4df37173 server_cookie=2548d63e0cb05177 in_seq=0 out_seq=0 2024-04-24T09:04:43.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.065+0000 7fb47cc00640 10 client.20340 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:04:43.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.067+0000 7fb47cc00640 1 -- 172.21.15.83:0/3515040669 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55d7136aca80 con 0x55d7137c0b90 2024-04-24T09:04:43.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.067+0000 7fb47cc00640 10 client.20340 handle_command_reply: tid=0 2024-04-24T09:04:43.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.067+0000 7fb49cbfa740 1 client.20340 shutdown 2024-04-24T09:04:43.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.067+0000 7fb49cbfa740 1 -- 172.21.15.83:0/3515040669 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d71373f860 msgr2=0x55d7137526a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:43.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.068+0000 7fb49cbfa740 1 --2- 172.21.15.83:0/3515040669 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d71373f860 0x55d7137526a0 secure :-1 s=READY pgs=2060 cs=0 l=1 rev1=1 crypto rx=0x7fb4840eebc0 tx=0x7fb484003860 comp rx=0 tx=0).stop 2024-04-24T09:04:43.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.068+0000 7fb49cbfa740 1 -- 172.21.15.83:0/3515040669 shutdown_connections 2024-04-24T09:04:43.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.068+0000 7fb49cbfa740 1 --2- 172.21.15.83:0/3515040669 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d7137c0b90 0x55d7137c2fb0 crc :-1 s=READY pgs=344 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:43.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.068+0000 7fb49cbfa740 1 --2- 172.21.15.83:0/3515040669 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d71374a850 0x55d71374db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:43.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.068+0000 7fb49cbfa740 1 --2- 172.21.15.83:0/3515040669 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d713740180 0x55d713752be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:43.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.068+0000 7fb49cbfa740 1 --2- 172.21.15.83:0/3515040669 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d71373f860 0x55d7137526a0 unknown :-1 s=CLOSED pgs=2060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:43.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.068+0000 7fb49cbfa740 1 -- 172.21.15.83:0/3515040669 >> 172.21.15.83:0/3515040669 conn(0x55d71373bb80 msgr2=0x55d71373ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:43.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.068+0000 7fb47cc00640 0 client.20340 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:04:43.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.068+0000 7fb49cbfa740 1 -- 172.21.15.83:0/3515040669 shutdown_connections 2024-04-24T09:04:43.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.068+0000 7fb49cbfa740 1 -- 172.21.15.83:0/3515040669 wait complete. 2024-04-24T09:04:43.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.068+0000 7fb49cbfa740 20 client.20340 trim_cache size 0 max 16384 2024-04-24T09:04:43.069 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T09:04:43.070 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T09:04:43.070 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-04-24T09:04:43.070 INFO:teuthology.orchestra.run.smithi083.stdout: "3a540d73-d94d-43b8-b95c-25e071105ec0": { 2024-04-24T09:04:43.070 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-04-24T09:04:43.070 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "3a540d73-d94d-43b8-b95c-25e071105ec0", 2024-04-24T09:04:43.070 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-04-24T09:04:43.070 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:04:43.070 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:04:43.070 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T09:04:43.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.069+0000 7fb499e00640 1 -- 172.21.15.83:0/1786687414 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb4780a3d20 msgr2=0x7fb4780a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:43.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.069+0000 7fb499e00640 1 --2- 172.21.15.83:0/1786687414 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb4780a3d20 0x7fb4780a61e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fb48c00bd30 tx=0x7fb48c00b300 comp rx=0 tx=0).stop 2024-04-24T09:04:43.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.069+0000 7fb499e00640 1 -- 172.21.15.83:0/1786687414 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb494101140 msgr2=0x7fb4941a0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:43.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.069+0000 7fb499e00640 1 --2- 172.21.15.83:0/1786687414 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb494101140 0x7fb4941a0670 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x7fb484006010 tx=0x7fb484004270 comp rx=0 tx=0).stop 2024-04-24T09:04:43.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.069+0000 7fb499e00640 1 -- 172.21.15.83:0/1786687414 shutdown_connections 2024-04-24T09:04:43.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.069+0000 7fb499e00640 1 --2- 172.21.15.83:0/1786687414 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb4780a3d20 0x7fb4780a61e0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:43.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.069+0000 7fb499e00640 1 --2- 172.21.15.83:0/1786687414 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb49419a7e0 0x7fb49419ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:43.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.069+0000 7fb499e00640 1 --2- 172.21.15.83:0/1786687414 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb494106210 0x7fb4941a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:43.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.069+0000 7fb499e00640 1 --2- 172.21.15.83:0/1786687414 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb494101140 0x7fb4941a0670 unknown :-1 s=CLOSED pgs=2056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:43.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.069+0000 7fb499e00640 1 -- 172.21.15.83:0/1786687414 >> 172.21.15.83:0/1786687414 conn(0x7fb494078000 msgr2=0x7fb494100130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:43.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.069+0000 7fb499e00640 1 -- 172.21.15.83:0/1786687414 shutdown_connections 2024-04-24T09:04:43.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:04:43.069+0000 7fb499e00640 1 -- 172.21.15.83:0/1786687414 wait complete. 2024-04-24T09:04:43.088 INFO:tasks.cephfs.filesystem:scrub status for tag:3a540d73-d94d-43b8-b95c-25e071105ec0 - {'path': '/', 'tag': '3a540d73-d94d-43b8-b95c-25e071105ec0', 'options': 'recursive,force'} 2024-04-24T09:04:43.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:04:42 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:04:43.205 INFO:journalctl@ceph.mds.b.smithi138.stdout:Apr 24 09:04:42 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-b[61515]: 2024-04-24T09:04:42.829+0000 7f7fbbc00700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000020c [...16,head] /volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/EXCEL/ auth{3=1} v12897 ap=2 f(v768 m2024-04-24T09:04:20.802310+0000 8=8+0) n(v57 rc2024-04-24T09:04:20.813992+0000 b1283356 9=8+1) old_inodes=6 (inest sync->mix g=3) caps={15603=pAsLsXsFs/-@11719} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x55cf8760a580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000020c:[<0x10000000009/EXCEL v347>,<0x10000000008/~dmtmp v57>,<0x10000000007/client0 v33>,<0x10000000006/clients v23>,<0x10000000005/tmp v23>,<0x10000000003/client.0 v22>,<0x10000000002/e395bed6-5bba-41b8-a256-3505e4afcca2 v36>,<0x10000000001/sv_0 v55>,<0x10000000000/qa v43>,<0x1/volumes v39>]//[]","memoryvalue":"(2)0x1000000020c:[<0x10000000009/EXCEL v12897>,<0x10000000008/~dmtmp v12284>,<0x10000000007/client0 v1471>,<0x10000000006/clients v1463>,<0x10000000005/tmp v1439>,<0x10000000003/client.0 v1428>,<0x10000000002/e395bed6-5bba-41b8-a256-3505e4afcca2 v1420>,<0x10000000001/sv_0 v1412>,<0x10000000000/qa v1392>,<0x1/volumes v1347>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-04-24T09:04:20.741300+0000 7=7+0)","ondisk_value.rstat":"n(v0 rc2024-04-24T09:04:20.813992+0000 b1283356 9=8+1)","memory_value.dirstat":"f(v768 m2024-04-24T09:04:20.802310+0000 8=8+0)","memory_value.rstat":"n(v57 rc2024-04-24T09:04:20.813992+0000 b1283356 9=8+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-04-24T09:04:43.205 INFO:journalctl@ceph.mds.b.smithi138.stdout:Apr 24 09:04:42 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-b[61515]: 2024-04-24T09:04:42.834+0000 7f7fbbc00700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000208 [...16,head] /volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/WORD/ auth{2=1} v12692 ap=2 f(v480 m2024-04-24T09:04:10.226301+0000 5=5+0) n(v45 rc2024-04-24T09:04:10.226301+0000 b814326 6=5+1) old_inodes=6 (inest sync->mix g=2) caps={15603=pAsLsXsFs/-@10760} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x55cf87274c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000000208:[<0x10000000009/WORD v427>,<0x10000000008/~dmtmp v57>,<0x10000000007/client0 v33>,<0x10000000006/clients v23>,<0x10000000005/tmp v23>,<0x10000000003/client.0 v22>,<0x10000000002/e395bed6-5bba-41b8-a256-3505e4afcca2 v36>,<0x10000000001/sv_0 v55>,<0x10000000000/qa v43>,<0x1/volumes v39>]//[]","memoryvalue":"(2)0x10000000208:[<0x10000000009/WORD v12692>,<0x10000000008/~dmtmp v12284>,<0x10000000007/client0 v1471>,<0x10000000006/clients v1463>,<0x10000000005/tmp v1439>,<0x10000000003/client.0 v1428>,<0x10000000002/e395bed6-5bba-41b8-a256-3505e4afcca2 v1420>,<0x10000000001/sv_0 v1412>,<0x10000000000/qa v1392>,<0x1/volumes v1347>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-04-24T09:04:10.175023+0000 6=6+0)","ondisk_value.rstat":"n(v0 rc2024-04-24T09:04:10.226301+0000 b814326 6=5+1)","memory_value.dirstat":"f(v480 m2024-04-24T09:04:10.226301+0000 5=5+0)","memory_value.rstat":"n(v45 rc2024-04-24T09:04:10.226301+0000 b814326 6=5+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-04-24T09:04:43.205 INFO:journalctl@ceph.mds.b.smithi138.stdout:Apr 24 09:04:42 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-b[61515]: 2024-04-24T09:04:42.845+0000 7f7fbbc00700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000229 [...16,head] /volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/PARADOX/ auth{1=1} fragtree_t(*^3) v12860 ap=2 f(v700 m2024-04-24T09:04:17.234128+0000 24=24+0) n(v59 rc2024-04-24T09:04:17.234128+0000 b5230239 25=24+1) old_inodes=6 (inest sync->mix g=1) caps={15603=pAsLsXsFs/-@12823} | dirtyscattered=0 request=0 lock=0 dirfrag=8 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x55cf87642b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000000229:[<0x10000000009/PARADOX v481>,<0x10000000008/~dmtmp v57>,<0x10000000007/client0 v33>,<0x10000000006/clients v23>,<0x10000000005/tmp v23>,<0x10000000003/client.0 v22>,<0x10000000002/e395bed6-5bba-41b8-a256-3505e4afcca2 v36>,<0x10000000001/sv_0 v55>,<0x10000000000/qa v43>,<0x1/volumes v39>]//[]","memoryvalue":"(2)0x10000000229:[<0x10000000009/PARADOX v12860>,<0x10000000008/~dmtmp v12286>,<0x10000000007/client0 v1471>,<0x10000000006/clients v1463>,<0x10000000005/tmp v1439>,<0x10000000003/client.0 v1428>,<0x10000000002/e395bed6-5bba-41b8-a256-3505e4afcca2 v1420>,<0x10000000001/sv_0 v1412>,<0x10000000000/qa v1392>,<0x1/volumes v1347>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-04-24T09:04:17.167887+0000 25=25+0)","ondisk_value.rstat":"n(v0 rc2024-04-24T09:04:17.234128+0000 b5230239 25=24+1)","memory_value.dirstat":"f(v700 m2024-04-24T09:04:17.234128+0000 24=24+0)","memory_value.rstat":"n(v59 rc2024-04-24T09:04:17.234128+0000 b5230239 25=24+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-04-24T09:04:43.205 INFO:journalctl@ceph.mds.b.smithi138.stdout:Apr 24 09:04:42 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-b[61515]: 2024-04-24T09:04:42.850+0000 7f7fbbc00700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000020f [...16,head] /volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/PWRPNT/ auth{1=1} v12738 ap=2 f(v560 m2024-04-24T09:04:07.861675+0000 7=7+0) n(v42 rc2024-04-24T09:04:07.861675+0000 b1462582 8=7+1) old_inodes=6 (inest sync->mix g=1) caps={15603=pAsLsXsFs/-@10707} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x55cf8760b600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000020f:[<0x10000000009/PWRPNT v274>,<0x10000000008/~dmtmp v57>,<0x10000000007/client0 v33>,<0x10000000006/clients v23>,<0x10000000005/tmp v23>,<0x10000000003/client.0 v22>,<0x10000000002/e395bed6-5bba-41b8-a256-3505e4afcca2 v36>,<0x10000000001/sv_0 v55>,<0x10000000000/qa v43>,<0x1/volumes v39>]//[]","memoryvalue":"(2)0x1000000020f:[<0x10000000009/PWRPNT v12738>,<0x10000000008/~dmtmp v12286>,<0x10000000007/client0 v1471>,<0x10000000006/clients v1463>,<0x10000000005/tmp v1439>,<0x10000000003/client.0 v1428>,<0x10000000002/e395bed6-5bba-41b8-a256-3505e4afcca2 v1420>,<0x10000000001/sv_0 v1412>,<0x10000000000/qa v1392>,<0x1/volumes v1347>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-04-24T09:04:07.762987+0000 8=8+0)","ondisk_value.rstat":"n(v0 rc2024-04-24T09:04:07.861675+0000 b1462582 8=7+1)","memory_value.dirstat":"f(v560 m2024-04-24T09:04:07.861675+0000 7=7+0)","memory_value.rstat":"n(v42 rc2024-04-24T09:04:07.861675+0000 b1462582 8=7+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-04-24T09:04:43.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:04:42 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:04:43.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:04:42 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:04:43.628 INFO:tasks.workunit.client.0.smithi083.stdout: 1 75101 0.58 MB/sec execute 557 sec latency 68.935 ms 2024-04-24T09:04:44.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:04:43 smithi138 ceph-mon[31441]: pgmap v412: 129 pgs: 129 active+clean; 73 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 264 KiB/s wr, 45 op/s 2024-04-24T09:04:44.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:04:43 smithi138 ceph-mon[31441]: scrub queued for path: / 2024-04-24T09:04:44.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:04:43 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:04:44.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:04:43 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:04:44.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:04:43 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:04:44.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:04:43 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:04:44.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:04:43 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:04:44.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:04:43 smithi138 ceph-mon[31441]: Scrub error on inode 0x1000000020c (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/EXCEL) see mds.b log and `damage ls` output for details 2024-04-24T09:04:44.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:04:43 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:04:44.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:04:43 smithi138 ceph-mon[31441]: Scrub error on inode 0x10000000208 (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/WORD) see mds.b log and `damage ls` output for details 2024-04-24T09:04:44.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:04:43 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:04:44.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:04:43 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:04:44.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:04:43 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:04:44.207 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:04:43 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:04:44.207 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:04:43 smithi138 ceph-mon[31441]: Scrub error on inode 0x10000000229 (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/PARADOX) see mds.b log and `damage ls` output for details 2024-04-24T09:04:44.207 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:04:43 smithi138 ceph-mon[31441]: Scrub error on inode 0x1000000020f (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/PWRPNT) see mds.b log and `damage ls` output for details 2024-04-24T09:04:44.207 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:04:43 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:04:44.207 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:04:43 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:04:44.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:04:43 smithi189 ceph-mon[31791]: pgmap v412: 129 pgs: 129 active+clean; 73 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 264 KiB/s wr, 45 op/s 2024-04-24T09:04:44.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:04:43 smithi189 ceph-mon[31791]: scrub queued for path: / 2024-04-24T09:04:44.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:04:43 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:04:44.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:04:43 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:04:44.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:04:43 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:04:44.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:04:43 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:04:44.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:04:43 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:04:44.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:04:43 smithi189 ceph-mon[31791]: Scrub error on inode 0x1000000020c (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/EXCEL) see mds.b log and `damage ls` output for details 2024-04-24T09:04:44.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:04:43 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:04:44.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:04:43 smithi189 ceph-mon[31791]: Scrub error on inode 0x10000000208 (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/WORD) see mds.b log and `damage ls` output for details 2024-04-24T09:04:44.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:04:43 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:04:44.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:04:43 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:04:44.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:04:43 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:04:44.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:04:43 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:04:44.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:04:43 smithi189 ceph-mon[31791]: Scrub error on inode 0x10000000229 (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/PARADOX) see mds.b log and `damage ls` output for details 2024-04-24T09:04:44.209 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:04:43 smithi189 ceph-mon[31791]: Scrub error on inode 0x1000000020f (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/PWRPNT) see mds.b log and `damage ls` output for details 2024-04-24T09:04:44.209 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:04:43 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:04:44.209 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:04:43 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:04:44.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:04:43 smithi083 ceph-mon[27907]: pgmap v412: 129 pgs: 129 active+clean; 73 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 264 KiB/s wr, 45 op/s 2024-04-24T09:04:44.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:04:43 smithi083 ceph-mon[27907]: scrub queued for path: / 2024-04-24T09:04:44.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:04:43 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:04:44.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:04:43 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:04:44.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:04:43 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:04:44.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:04:43 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:04:44.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:04:43 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:04:44.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:04:43 smithi083 ceph-mon[27907]: Scrub error on inode 0x1000000020c (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/EXCEL) see mds.b log and `damage ls` output for details 2024-04-24T09:04:44.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:04:43 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:04:44.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:04:43 smithi083 ceph-mon[27907]: Scrub error on inode 0x10000000208 (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/WORD) see mds.b log and `damage ls` output for details 2024-04-24T09:04:44.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:04:43 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:04:44.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:04:43 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:04:44.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:04:43 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:04:44.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:04:43 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:04:44.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:04:43 smithi083 ceph-mon[27907]: Scrub error on inode 0x10000000229 (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/PARADOX) see mds.b log and `damage ls` output for details 2024-04-24T09:04:44.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:04:43 smithi083 ceph-mon[27907]: Scrub error on inode 0x1000000020f (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/PWRPNT) see mds.b log and `damage ls` output for details 2024-04-24T09:04:44.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:04:43 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:04:44.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:04:43 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:04:44.628 INFO:tasks.workunit.client.0.smithi083.stdout: 1 75164 0.58 MB/sec execute 558 sec latency 60.700 ms 2024-04-24T09:04:45.628 INFO:tasks.workunit.client.0.smithi083.stdout: 1 75295 0.58 MB/sec execute 559 sec latency 54.467 ms 2024-04-24T09:04:46.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:04:45 smithi138 ceph-mon[31441]: pgmap v413: 129 pgs: 129 active+clean; 73 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 245 KiB/s wr, 36 op/s 2024-04-24T09:04:46.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:04:45 smithi189 ceph-mon[31791]: pgmap v413: 129 pgs: 129 active+clean; 73 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 245 KiB/s wr, 36 op/s 2024-04-24T09:04:46.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:04:45 smithi083 ceph-mon[27907]: pgmap v413: 129 pgs: 129 active+clean; 73 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 245 KiB/s wr, 36 op/s 2024-04-24T09:04:46.629 INFO:tasks.workunit.client.0.smithi083.stdout: 1 75416 0.58 MB/sec execute 560 sec latency 55.977 ms 2024-04-24T09:04:47.628 INFO:tasks.workunit.client.0.smithi083.stdout: 1 75517 0.58 MB/sec execute 561 sec latency 51.923 ms 2024-04-24T09:04:48.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:04:47 smithi138 ceph-mon[31441]: pgmap v414: 129 pgs: 129 active+clean; 73 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 245 KiB/s wr, 36 op/s 2024-04-24T09:04:48.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:04:47 smithi189 ceph-mon[31791]: pgmap v414: 129 pgs: 129 active+clean; 73 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 245 KiB/s wr, 36 op/s 2024-04-24T09:04:48.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:04:47 smithi083 ceph-mon[27907]: pgmap v414: 129 pgs: 129 active+clean; 73 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 245 KiB/s wr, 36 op/s 2024-04-24T09:04:48.629 INFO:tasks.workunit.client.0.smithi083.stdout: 1 75638 0.58 MB/sec execute 562 sec latency 50.270 ms 2024-04-24T09:04:49.629 INFO:tasks.workunit.client.0.smithi083.stdout: 1 75740 0.58 MB/sec execute 563 sec latency 48.892 ms 2024-04-24T09:04:50.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:04:49 smithi138 ceph-mon[31441]: pgmap v415: 129 pgs: 129 active+clean; 72 MiB data, 999 MiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 379 KiB/s wr, 71 op/s 2024-04-24T09:04:50.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:04:49 smithi138 ceph-mon[31441]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:04:50.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:04:49 smithi189 ceph-mon[31791]: pgmap v415: 129 pgs: 129 active+clean; 72 MiB data, 999 MiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 379 KiB/s wr, 71 op/s 2024-04-24T09:04:50.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:04:49 smithi189 ceph-mon[31791]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:04:50.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:04:49 smithi083 ceph-mon[27907]: pgmap v415: 129 pgs: 129 active+clean; 72 MiB data, 999 MiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 379 KiB/s wr, 71 op/s 2024-04-24T09:04:50.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:04:49 smithi083 ceph-mon[27907]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:04:50.629 INFO:tasks.workunit.client.0.smithi083.stdout: 1 75926 0.59 MB/sec execute 564 sec latency 59.201 ms 2024-04-24T09:04:50.834 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.834+0000 7f63b5e00640 1 -- 172.21.15.83:0/2312691975 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63b010e4d0 msgr2=0x7f63b010e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:50.834 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.834+0000 7f63b5e00640 1 --2- 172.21.15.83:0/2312691975 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63b010e4d0 0x7f63b010e8b0 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 crypto rx=0x7f63a0009930 tx=0x7f63a002a440 comp rx=0 tx=0).stop 2024-04-24T09:04:50.834 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.834+0000 7f63b5e00640 1 -- 172.21.15.83:0/2312691975 shutdown_connections 2024-04-24T09:04:50.834 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.834+0000 7f63b5e00640 1 --2- 172.21.15.83:0/2312691975 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f63b0108ec0 0x7f63b0115300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:50.834 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.834+0000 7f63b5e00640 1 --2- 172.21.15.83:0/2312691975 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f63b0108540 0x7f63b0108980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:50.835 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.834+0000 7f63b5e00640 1 --2- 172.21.15.83:0/2312691975 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63b010e4d0 0x7f63b010e8b0 unknown :-1 s=CLOSED pgs=2061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:50.835 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.834+0000 7f63b5e00640 1 -- 172.21.15.83:0/2312691975 >> 172.21.15.83:0/2312691975 conn(0x7f63b0077de0 msgr2=0x7f63b0106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:50.835 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.834+0000 7f63b5e00640 1 -- 172.21.15.83:0/2312691975 shutdown_connections 2024-04-24T09:04:50.835 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.834+0000 7f63b5e00640 1 -- 172.21.15.83:0/2312691975 wait complete. 2024-04-24T09:04:50.837 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.835+0000 7f63b5e00640 1 Processor -- start 2024-04-24T09:04:50.837 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.835+0000 7f63b5e00640 1 -- start start 2024-04-24T09:04:50.837 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.836+0000 7f63b5e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63b0108540 0x7f63b01a4830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:50.837 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.836+0000 7f63b5e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f63b0108ec0 0x7f63b01a4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:50.837 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.836+0000 7f63b5e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f63b010e4d0 0x7f63b019ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:50.837 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.836+0000 7f63b5e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f63b01186d0 con 0x7f63b0108540 2024-04-24T09:04:50.837 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.836+0000 7f63b5e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f63b0118550 con 0x7f63b0108ec0 2024-04-24T09:04:50.837 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.836+0000 7f63b5e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f63b0118850 con 0x7f63b010e4d0 2024-04-24T09:04:50.837 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.836+0000 7f63b4a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63b0108540 0x7f63b01a4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:50.837 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.836+0000 7f63b4a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63b0108540 0x7f63b01a4830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47118/0 (socket says 172.21.15.83:47118) 2024-04-24T09:04:50.837 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.836+0000 7f63b4a00640 1 -- 172.21.15.83:0/2249064362 learned_addr learned my addr 172.21.15.83:0/2249064362 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:04:50.838 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.836+0000 7f63b5400640 1 --2- 172.21.15.83:0/2249064362 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f63b010e4d0 0x7f63b019ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:50.838 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.837+0000 7f63a7e00640 1 --2- 172.21.15.83:0/2249064362 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f63b0108ec0 0x7f63b01a4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:50.838 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.837+0000 7f63b4a00640 1 -- 172.21.15.83:0/2249064362 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f63b010e4d0 msgr2=0x7f63b019ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:50.838 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.837+0000 7f63b4a00640 1 --2- 172.21.15.83:0/2249064362 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f63b010e4d0 0x7f63b019ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:50.838 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.837+0000 7f63b4a00640 1 -- 172.21.15.83:0/2249064362 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f63b0108ec0 msgr2=0x7f63b01a4d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:50.838 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.837+0000 7f63b4a00640 1 --2- 172.21.15.83:0/2249064362 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f63b0108ec0 0x7f63b01a4d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:50.838 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.837+0000 7f63b4a00640 1 -- 172.21.15.83:0/2249064362 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63b019f1b0 con 0x7f63b0108540 2024-04-24T09:04:50.838 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.837+0000 7f63a7e00640 1 --2- 172.21.15.83:0/2249064362 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f63b0108ec0 0x7f63b01a4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:04:50.838 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.837+0000 7f63b4a00640 1 --2- 172.21.15.83:0/2249064362 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63b0108540 0x7f63b01a4830 secure :-1 s=READY pgs=2062 cs=0 l=1 rev1=1 crypto rx=0x7f63a0031440 tx=0x7f63a0003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:50.839 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.837+0000 7f63ae000640 1 -- 172.21.15.83:0/2249064362 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a003f070 con 0x7f63b0108540 2024-04-24T09:04:50.839 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.837+0000 7f63ae000640 1 -- 172.21.15.83:0/2249064362 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f63a0031d60 con 0x7f63b0108540 2024-04-24T09:04:50.839 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.837+0000 7f63b5e00640 1 -- 172.21.15.83:0/2249064362 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63b019f440 con 0x7f63b0108540 2024-04-24T09:04:50.839 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.837+0000 7f63ae000640 1 -- 172.21.15.83:0/2249064362 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a0035030 con 0x7f63b0108540 2024-04-24T09:04:50.839 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.838+0000 7f63b5e00640 1 -- 172.21.15.83:0/2249064362 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63b01ab6f0 con 0x7f63b0108540 2024-04-24T09:04:50.842 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.841+0000 7f63ba26a740 1 Processor -- start 2024-04-24T09:04:50.842 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.841+0000 7f63ba26a740 1 -- start start 2024-04-24T09:04:50.842 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.841+0000 7f63ae000640 1 -- 172.21.15.83:0/2249064362 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f63a0042070 con 0x7f63b0108540 2024-04-24T09:04:50.842 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.842+0000 7f63ba26a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55582d1e03c0 0x55582d1e07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:50.842 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.842+0000 7f63ba26a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55582d1e0ce0 0x55582d1eb060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:50.842 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.842+0000 7f63ba26a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55582d1eb5a0 0x55582d1ed990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:50.842 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.842+0000 7f63ba26a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55582d146900 con 0x55582d1e0ce0 2024-04-24T09:04:50.842 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.842+0000 7f63ba26a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55582d0ecf20 con 0x55582d1e03c0 2024-04-24T09:04:50.842 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.842+0000 7f63ba26a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55582d1428c0 con 0x55582d1eb5a0 2024-04-24T09:04:50.842 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.842+0000 7f63b4a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55582d1e0ce0 0x55582d1eb060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:50.843 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.842+0000 7f63b4a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55582d1e0ce0 0x55582d1eb060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47122/0 (socket says 172.21.15.83:47122) 2024-04-24T09:04:50.844 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.842+0000 7f63b4a00640 1 -- 172.21.15.83:0/996204061 learned_addr learned my addr 172.21.15.83:0/996204061 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:04:50.844 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.842+0000 7f63a7e00640 1 --2- 172.21.15.83:0/996204061 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55582d1eb5a0 0x55582d1ed990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:50.844 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.842+0000 7f63b5400640 1 --2- 172.21.15.83:0/996204061 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55582d1e03c0 0x55582d1e07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:50.844 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.843+0000 7f63b4a00640 1 -- 172.21.15.83:0/996204061 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55582d1eb5a0 msgr2=0x55582d1ed990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:50.844 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.843+0000 7f63b4a00640 1 --2- 172.21.15.83:0/996204061 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55582d1eb5a0 0x55582d1ed990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:50.844 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.843+0000 7f63b4a00640 1 -- 172.21.15.83:0/996204061 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55582d1e03c0 msgr2=0x55582d1e07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:50.844 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.843+0000 7f63b4a00640 1 --2- 172.21.15.83:0/996204061 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55582d1e03c0 0x55582d1e07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:50.844 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.843+0000 7f63b4a00640 1 -- 172.21.15.83:0/996204061 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55582d111e10 con 0x55582d1e0ce0 2024-04-24T09:04:50.844 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.843+0000 7f63b4a00640 1 --2- 172.21.15.83:0/996204061 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55582d1e0ce0 0x55582d1eb060 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 crypto rx=0x7f63a00ee840 tx=0x7f63a00f7cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86b3d522f60edad5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:50.844 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.843+0000 7f63ae000640 1 --2- 172.21.15.83:0/2249064362 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f63940a3d10 0x7f63940a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:50.844 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.844+0000 7f63a5c00640 1 -- 172.21.15.83:0/996204061 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a0100070 con 0x55582d1e0ce0 2024-04-24T09:04:50.844 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.844+0000 7f63a5c00640 1 -- 172.21.15.83:0/996204061 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f63a00bd820 con 0x55582d1e0ce0 2024-04-24T09:04:50.844 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.844+0000 7f63a5c00640 1 -- 172.21.15.83:0/996204061 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a008e260 con 0x55582d1e0ce0 2024-04-24T09:04:50.844 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.844+0000 7f63ae000640 1 -- 172.21.15.83:0/2249064362 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7f63a00f1580 con 0x7f63b0108540 2024-04-24T09:04:50.845 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.844+0000 7f63b5400640 1 --2- 172.21.15.83:0/2249064362 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f63940a3d10 0x7f63940a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:50.847 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.845+0000 7f63ba26a740 1 -- 172.21.15.83:0/996204061 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55582d1e0ce0 msgr2=0x55582d1eb060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:50.847 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.845+0000 7f63ba26a740 1 --2- 172.21.15.83:0/996204061 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55582d1e0ce0 0x55582d1eb060 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 crypto rx=0x7f63a00ee840 tx=0x7f63a00f7cb0 comp rx=0 tx=0).stop 2024-04-24T09:04:50.847 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.845+0000 7f63b5400640 1 --2- 172.21.15.83:0/2249064362 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f63940a3d10 0x7f63940a61d0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f63a8008dc0 tx=0x7f63a8008570 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:50.847 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.845+0000 7f63ba26a740 1 -- 172.21.15.83:0/996204061 shutdown_connections 2024-04-24T09:04:50.847 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.845+0000 7f63ba26a740 1 --2- 172.21.15.83:0/996204061 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55582d1eb5a0 0x55582d1ed990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:50.847 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.845+0000 7f63ba26a740 1 --2- 172.21.15.83:0/996204061 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55582d1e0ce0 0x55582d1eb060 unknown :-1 s=CLOSED pgs=2063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:50.847 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.845+0000 7f63ba26a740 1 --2- 172.21.15.83:0/996204061 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55582d1e03c0 0x55582d1e07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:50.847 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.845+0000 7f63ba26a740 1 -- 172.21.15.83:0/996204061 >> 172.21.15.83:0/996204061 conn(0x55582d1dc8d0 msgr2=0x55582d1dccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:50.848 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.845+0000 7f63ba26a740 1 -- 172.21.15.83:0/996204061 shutdown_connections 2024-04-24T09:04:50.848 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.845+0000 7f63ba26a740 1 -- 172.21.15.83:0/996204061 wait complete. 2024-04-24T09:04:50.848 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.845+0000 7f63ba26a740 1 -- start start 2024-04-24T09:04:50.848 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.846+0000 7f63ba26a740 1 Processor -- start 2024-04-24T09:04:50.848 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.846+0000 7f63ba26a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55582d1e03c0 0x55582d1e8500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:50.848 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.846+0000 7f63ba26a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55582d1e0ce0 0x55582d1e8a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:50.848 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.846+0000 7f63ba26a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55582d1eb5a0 0x55582d1e8f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:50.848 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.846+0000 7f63ba26a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55582d1edf90 con 0x55582d1eb5a0 2024-04-24T09:04:50.848 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.846+0000 7f63ba26a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55582d1002c0 con 0x55582d1e0ce0 2024-04-24T09:04:50.848 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.846+0000 7f63ba26a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55582d1ee110 con 0x55582d1e03c0 2024-04-24T09:04:50.848 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.846+0000 7f63ba26a740 10 client.0 _ll_register_callbacks cb 0x55582ce58a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:04:50.848 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.847+0000 7f63b5400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55582d1eb5a0 0x55582d1e8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:50.848 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.847+0000 7f63b5400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55582d1eb5a0 0x55582d1e8f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47130/0 (socket says 172.21.15.83:47130) 2024-04-24T09:04:50.848 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.847+0000 7f63b5400640 1 -- 172.21.15.83:0/3711496448 learned_addr learned my addr 172.21.15.83:0/3711496448 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:04:50.849 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.847+0000 7f63b4a00640 1 --2- 172.21.15.83:0/3711496448 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55582d1e03c0 0x55582d1e8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:50.849 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.847+0000 7f63a7e00640 1 --2- 172.21.15.83:0/3711496448 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55582d1e0ce0 0x55582d1e8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:50.849 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.847+0000 7f63b5400640 1 -- 172.21.15.83:0/3711496448 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55582d1e03c0 msgr2=0x55582d1e8500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:50.849 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.847+0000 7f63b5400640 1 --2- 172.21.15.83:0/3711496448 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55582d1e03c0 0x55582d1e8500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:50.849 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.847+0000 7f63b5400640 1 -- 172.21.15.83:0/3711496448 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55582d1e0ce0 msgr2=0x55582d1e8a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:50.849 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.847+0000 7f63b5400640 1 --2- 172.21.15.83:0/3711496448 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55582d1e0ce0 0x55582d1e8a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:50.849 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.847+0000 7f63b5400640 1 -- 172.21.15.83:0/3711496448 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55582d110fd0 con 0x55582d1eb5a0 2024-04-24T09:04:50.849 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.847+0000 7f63b5400640 1 --2- 172.21.15.83:0/3711496448 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55582d1eb5a0 0x55582d1e8f80 secure :-1 s=READY pgs=2064 cs=0 l=1 rev1=1 crypto rx=0x7f63a800edb0 tx=0x7f63a8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c137f653fd3d86f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:50.849 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.847+0000 7f639e000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:04:50.851 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.848+0000 7f639e000640 1 -- 172.21.15.83:0/3711496448 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a801cd30 con 0x55582d1eb5a0 2024-04-24T09:04:50.851 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.848+0000 7f639e000640 1 -- 172.21.15.83:0/3711496448 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7f63a8028480 con 0x55582d1eb5a0 2024-04-24T09:04:50.851 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.848+0000 7f63ba26a740 1 -- 172.21.15.83:0/3711496448 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63a00bd820 con 0x55582d1eb5a0 2024-04-24T09:04:50.851 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.848+0000 7f639e000640 1 -- 172.21.15.83:0/3711496448 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f63a8031050 con 0x55582d1eb5a0 2024-04-24T09:04:50.851 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.848+0000 7f63ba26a740 10 client.20352 fetch_fsmap learned FSMap version 42 2024-04-24T09:04:50.851 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.848+0000 7f63ba26a740 1 -- 172.21.15.83:0/3711496448 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55582ce59210 con 0x55582d1eb5a0 2024-04-24T09:04:50.851 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.848+0000 7f639e000640 1 -- 172.21.15.83:0/3711496448 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f63a00bd820 con 0x55582d1eb5a0 2024-04-24T09:04:50.851 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.849+0000 7f639e000640 1 -- 172.21.15.83:0/3711496448 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 42) ==== 8382+0+0 (secure 0 0 0) 0x7f63a801a390 con 0x55582d1eb5a0 2024-04-24T09:04:50.851 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.849+0000 7f63ba26a740 10 client.20352 fetch_fsmap finished waiting for FSMap version 42 2024-04-24T09:04:50.851 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.849+0000 7f63ba26a740 10 client.20352 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:04:50.851 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.849+0000 7f63ba26a740 20 client.20352 populate_metadata read hostname 'smithi083' 2024-04-24T09:04:50.851 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.849+0000 7f63ba26a740 1 --2- 172.21.15.83:0/3711496448 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55582d1fb4a0 0x55582d1fd940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:50.851 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.849+0000 7f63ba26a740 4 client.20352 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:04:50.852 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.849+0000 7f63ba26a740 1 -- 172.21.15.83:0/3711496448 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55582d154320 con 0x55582d1fb4a0 2024-04-24T09:04:50.852 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.849+0000 7f63b4a00640 1 --2- 172.21.15.83:0/3711496448 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55582d1fb4a0 0x55582d1fd940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:50.852 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.850+0000 7f639e000640 10 client.20352 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:04:50.852 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.850+0000 7f63b4a00640 1 --2- 172.21.15.83:0/3711496448 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55582d1fb4a0 0x55582d1fd940 crc :-1 s=READY pgs=345 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2d31e3959813fce server_cookie=ede5108a7bb16db1 in_seq=0 out_seq=0 2024-04-24T09:04:50.853 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.852+0000 7f639e000640 1 -- 172.21.15.83:0/3711496448 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55582d154320 con 0x55582d1fb4a0 2024-04-24T09:04:50.853 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.852+0000 7f639e000640 10 client.20352 handle_command_reply: tid=0 2024-04-24T09:04:50.853 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.852+0000 7f63ba26a740 1 client.20352 shutdown 2024-04-24T09:04:50.853 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.853+0000 7f63ba26a740 1 -- 172.21.15.83:0/3711496448 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55582d1eb5a0 msgr2=0x55582d1e8f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:50.853 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.853+0000 7f63ba26a740 1 --2- 172.21.15.83:0/3711496448 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55582d1eb5a0 0x55582d1e8f80 secure :-1 s=READY pgs=2064 cs=0 l=1 rev1=1 crypto rx=0x7f63a800edb0 tx=0x7f63a8004a40 comp rx=0 tx=0).stop 2024-04-24T09:04:50.853 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.853+0000 7f63ba26a740 1 -- 172.21.15.83:0/3711496448 shutdown_connections 2024-04-24T09:04:50.853 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.853+0000 7f63ba26a740 1 --2- 172.21.15.83:0/3711496448 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55582d1fb4a0 0x55582d1fd940 crc :-1 s=READY pgs=345 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:50.853 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.853+0000 7f63ba26a740 1 --2- 172.21.15.83:0/3711496448 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55582d1eb5a0 0x55582d1e8f80 unknown :-1 s=CLOSED pgs=2064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:50.853 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.853+0000 7f63ba26a740 1 --2- 172.21.15.83:0/3711496448 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55582d1e0ce0 0x55582d1e8a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:50.854 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.853+0000 7f639e000640 0 client.20352 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:04:50.854 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.853+0000 7f63ba26a740 1 --2- 172.21.15.83:0/3711496448 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55582d1e03c0 0x55582d1e8500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:50.854 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.853+0000 7f63ba26a740 1 -- 172.21.15.83:0/3711496448 >> 172.21.15.83:0/3711496448 conn(0x55582d1dc8d0 msgr2=0x55582d1e2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:50.854 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.853+0000 7f63ba26a740 1 -- 172.21.15.83:0/3711496448 shutdown_connections 2024-04-24T09:04:50.854 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.853+0000 7f63ba26a740 1 -- 172.21.15.83:0/3711496448 wait complete. 2024-04-24T09:04:50.854 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.853+0000 7f63ba26a740 20 client.20352 trim_cache size 0 max 16384 2024-04-24T09:04:50.872 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.871+0000 7f63ba26a740 1 Processor -- start 2024-04-24T09:04:50.872 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.871+0000 7f63ba26a740 1 -- start start 2024-04-24T09:04:50.872 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.871+0000 7f63ba26a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55582d1e03c0 0x55582d1e8d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:50.873 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.871+0000 7f63ba26a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55582d1e0ce0 0x55582d1e2d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:50.873 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.872+0000 7f63ba26a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55582d1eb5a0 0x55582d203260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:50.873 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.872+0000 7f63ba26a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f63b01b5aa0 con 0x55582d1e03c0 2024-04-24T09:04:50.873 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.872+0000 7f63ba26a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f63b01b76d0 con 0x55582d1eb5a0 2024-04-24T09:04:50.873 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.872+0000 7f63ba26a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f63b01b6170 con 0x55582d1e0ce0 2024-04-24T09:04:50.873 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.872+0000 7f63b4a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55582d1e03c0 0x55582d1e8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:50.873 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.872+0000 7f63b4a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55582d1e03c0 0x55582d1e8d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47140/0 (socket says 172.21.15.83:47140) 2024-04-24T09:04:50.873 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.872+0000 7f63b4a00640 1 -- 172.21.15.83:0/846838911 learned_addr learned my addr 172.21.15.83:0/846838911 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:04:50.873 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.872+0000 7f63b5400640 1 --2- 172.21.15.83:0/846838911 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55582d1eb5a0 0x55582d203260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:50.873 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.872+0000 7f63a7e00640 1 --2- 172.21.15.83:0/846838911 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55582d1e0ce0 0x55582d1e2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:50.873 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.872+0000 7f63b4a00640 1 -- 172.21.15.83:0/846838911 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55582d1e0ce0 msgr2=0x55582d1e2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:50.873 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.872+0000 7f63b4a00640 1 --2- 172.21.15.83:0/846838911 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55582d1e0ce0 0x55582d1e2d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:50.873 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.872+0000 7f63b4a00640 1 -- 172.21.15.83:0/846838911 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55582d1eb5a0 msgr2=0x55582d203260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:50.874 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.872+0000 7f63b4a00640 1 --2- 172.21.15.83:0/846838911 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55582d1eb5a0 0x55582d203260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:50.874 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.872+0000 7f63b4a00640 1 -- 172.21.15.83:0/846838911 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55582d1edd00 con 0x55582d1e03c0 2024-04-24T09:04:50.874 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.873+0000 7f63b4a00640 1 --2- 172.21.15.83:0/846838911 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55582d1e03c0 0x55582d1e8d50 secure :-1 s=READY pgs=2065 cs=0 l=1 rev1=1 crypto rx=0x7f63a008e010 tx=0x7f63a0031f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f40a4bd1cfdff95 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:50.877 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.873+0000 7f63a5200640 1 -- 172.21.15.83:0/846838911 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a008cdf0 con 0x55582d1e03c0 2024-04-24T09:04:50.877 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.873+0000 7f63a5200640 1 -- 172.21.15.83:0/846838911 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f63a00bd820 con 0x55582d1e03c0 2024-04-24T09:04:50.877 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.873+0000 7f63a5200640 1 -- 172.21.15.83:0/846838911 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a008cfe0 con 0x55582d1e03c0 2024-04-24T09:04:50.877 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.874+0000 7f63ba26a740 1 -- 172.21.15.83:0/846838911 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55582d1e03c0 msgr2=0x55582d1e8d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:50.878 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.874+0000 7f63ba26a740 1 --2- 172.21.15.83:0/846838911 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55582d1e03c0 0x55582d1e8d50 secure :-1 s=READY pgs=2065 cs=0 l=1 rev1=1 crypto rx=0x7f63a008e010 tx=0x7f63a0031f10 comp rx=0 tx=0).stop 2024-04-24T09:04:50.878 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.874+0000 7f63ba26a740 1 -- 172.21.15.83:0/846838911 shutdown_connections 2024-04-24T09:04:50.878 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.874+0000 7f63ba26a740 1 --2- 172.21.15.83:0/846838911 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55582d1eb5a0 0x55582d203260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:50.878 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.874+0000 7f63ba26a740 1 --2- 172.21.15.83:0/846838911 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55582d1e0ce0 0x55582d1e2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:50.878 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.874+0000 7f63ba26a740 1 --2- 172.21.15.83:0/846838911 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55582d1e03c0 0x55582d1e8d50 unknown :-1 s=CLOSED pgs=2065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:50.878 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.874+0000 7f63ba26a740 1 -- 172.21.15.83:0/846838911 >> 172.21.15.83:0/846838911 conn(0x55582d1dc8d0 msgr2=0x55582d1db200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:50.878 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.874+0000 7f63ba26a740 1 -- 172.21.15.83:0/846838911 shutdown_connections 2024-04-24T09:04:50.878 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.874+0000 7f63ba26a740 1 -- 172.21.15.83:0/846838911 wait complete. 2024-04-24T09:04:50.878 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.874+0000 7f63ba26a740 1 -- start start 2024-04-24T09:04:50.878 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.875+0000 7f63ba26a740 1 Processor -- start 2024-04-24T09:04:50.878 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.875+0000 7f63ba26a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55582d1e03c0 0x55582d1f3270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:50.878 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.875+0000 7f63ba26a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55582d1e0ce0 0x55582d1f37b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:50.878 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.875+0000 7f63ba26a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55582d1eb5a0 0x55582d1f0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:50.878 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.875+0000 7f63ba26a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f63b01b6fa0 con 0x55582d1e03c0 2024-04-24T09:04:50.879 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.875+0000 7f63ba26a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f63b01b6870 con 0x55582d1e0ce0 2024-04-24T09:04:50.879 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.875+0000 7f63ba26a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f63b01b3aa0 con 0x55582d1eb5a0 2024-04-24T09:04:50.879 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.875+0000 7f63ba26a740 10 client.0 _ll_register_callbacks cb 0x7f63a8008f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:04:50.879 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.875+0000 7f63b4a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55582d1e03c0 0x55582d1f3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:50.879 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.875+0000 7f63b4a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55582d1e03c0 0x55582d1f3270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47152/0 (socket says 172.21.15.83:47152) 2024-04-24T09:04:50.879 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.875+0000 7f63b4a00640 1 -- 172.21.15.83:0/940255855 learned_addr learned my addr 172.21.15.83:0/940255855 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:04:50.879 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.876+0000 7f63a7e00640 1 --2- 172.21.15.83:0/940255855 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55582d1e0ce0 0x55582d1f37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:50.879 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.876+0000 7f63b5400640 1 --2- 172.21.15.83:0/940255855 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55582d1eb5a0 0x55582d1f0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:50.879 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.876+0000 7f63b4a00640 1 -- 172.21.15.83:0/940255855 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55582d1eb5a0 msgr2=0x55582d1f0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:50.879 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.876+0000 7f63b4a00640 1 --2- 172.21.15.83:0/940255855 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55582d1eb5a0 0x55582d1f0240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:50.879 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.876+0000 7f63b4a00640 1 -- 172.21.15.83:0/940255855 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55582d1e0ce0 msgr2=0x55582d1f37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:04:50.879 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.876+0000 7f63b4a00640 1 --2- 172.21.15.83:0/940255855 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55582d1e0ce0 0x55582d1f37b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:50.879 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.876+0000 7f63b4a00640 1 -- 172.21.15.83:0/940255855 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55582cf25620 con 0x55582d1e03c0 2024-04-24T09:04:50.879 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.876+0000 7f63b4a00640 1 --2- 172.21.15.83:0/940255855 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55582d1e03c0 0x55582d1f3270 secure :-1 s=READY pgs=2066 cs=0 l=1 rev1=1 crypto rx=0x7f63a0000c00 tx=0x7f63a0005e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da240675fdbbb05b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:04:50.880 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.876+0000 7f639e000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:04:50.880 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.876+0000 7f639e000640 1 -- 172.21.15.83:0/940255855 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a0049070 con 0x55582d1e03c0 2024-04-24T09:04:50.880 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.876+0000 7f639e000640 1 -- 172.21.15.83:0/940255855 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7f63a00bb420 con 0x55582d1e03c0 2024-04-24T09:04:50.880 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.876+0000 7f63ba26a740 1 -- 172.21.15.83:0/940255855 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63a00bd820 con 0x55582d1e03c0 2024-04-24T09:04:50.880 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.877+0000 7f639e000640 1 -- 172.21.15.83:0/940255855 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f63a0100070 con 0x55582d1e03c0 2024-04-24T09:04:50.880 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.877+0000 7f63ba26a740 10 client.20358 fetch_fsmap learned FSMap version 42 2024-04-24T09:04:50.880 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.877+0000 7f63ba26a740 1 -- 172.21.15.83:0/940255855 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55582cf25620 con 0x55582d1e03c0 2024-04-24T09:04:50.880 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.877+0000 7f639e000640 1 -- 172.21.15.83:0/940255855 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f63a00bd820 con 0x55582d1e03c0 2024-04-24T09:04:50.880 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.877+0000 7f639e000640 1 -- 172.21.15.83:0/940255855 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 42) ==== 8382+0+0 (secure 0 0 0) 0x7f63a00c1150 con 0x55582d1e03c0 2024-04-24T09:04:50.880 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.877+0000 7f63ba26a740 10 client.20358 fetch_fsmap finished waiting for FSMap version 42 2024-04-24T09:04:50.880 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.877+0000 7f63ba26a740 10 client.20358 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:04:50.880 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.877+0000 7f63ba26a740 20 client.20358 populate_metadata read hostname 'smithi083' 2024-04-24T09:04:50.880 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.877+0000 7f63ba26a740 1 --2- 172.21.15.83:0/940255855 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55582d2624f0 0x55582d2648e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:04:50.880 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.877+0000 7f63ba26a740 4 client.20358 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T09:04:50.881 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.877+0000 7f63ba26a740 1 -- 172.21.15.83:0/940255855 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55582d14e190 con 0x55582d2624f0 2024-04-24T09:04:50.881 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.878+0000 7f63a7e00640 1 --2- 172.21.15.83:0/940255855 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55582d2624f0 0x55582d2648e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:04:50.881 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.879+0000 7f63a7e00640 1 --2- 172.21.15.83:0/940255855 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55582d2624f0 0x55582d2648e0 crc :-1 s=READY pgs=346 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=564ae760460ea282 server_cookie=58fcf5372a9baa6c in_seq=0 out_seq=0 2024-04-24T09:04:50.881 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.879+0000 7f639e000640 10 client.20358 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:04:50.936 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.936+0000 7f639e000640 1 -- 172.21.15.83:0/940255855 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55582d14e190 con 0x55582d2624f0 2024-04-24T09:04:50.936 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.936+0000 7f639e000640 10 client.20358 handle_command_reply: tid=0 2024-04-24T09:04:50.937 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.937+0000 7f63ba26a740 1 client.20358 shutdown 2024-04-24T09:04:50.937 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.937+0000 7f63ba26a740 1 -- 172.21.15.83:0/940255855 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55582d1e03c0 msgr2=0x55582d1f3270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:50.937 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.937+0000 7f63ba26a740 1 --2- 172.21.15.83:0/940255855 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55582d1e03c0 0x55582d1f3270 secure :-1 s=READY pgs=2066 cs=0 l=1 rev1=1 crypto rx=0x7f63a0000c00 tx=0x7f63a0005e20 comp rx=0 tx=0).stop 2024-04-24T09:04:50.938 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.937+0000 7f63ba26a740 1 -- 172.21.15.83:0/940255855 shutdown_connections 2024-04-24T09:04:50.938 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.937+0000 7f63ba26a740 1 --2- 172.21.15.83:0/940255855 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55582d2624f0 0x55582d2648e0 crc :-1 s=READY pgs=346 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:50.938 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.937+0000 7f63ba26a740 1 --2- 172.21.15.83:0/940255855 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55582d1eb5a0 0x55582d1f0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:50.938 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.937+0000 7f63ba26a740 1 --2- 172.21.15.83:0/940255855 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55582d1e0ce0 0x55582d1f37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:50.938 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.937+0000 7f63ba26a740 1 --2- 172.21.15.83:0/940255855 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55582d1e03c0 0x55582d1f3270 unknown :-1 s=CLOSED pgs=2066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:50.938 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.937+0000 7f639e000640 0 client.20358 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:04:50.938 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.937+0000 7f63ba26a740 1 -- 172.21.15.83:0/940255855 >> 172.21.15.83:0/940255855 conn(0x55582d1dc8d0 msgr2=0x55582d1e8bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:50.939 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.937+0000 7f63ba26a740 1 -- 172.21.15.83:0/940255855 shutdown_connections 2024-04-24T09:04:50.939 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.937+0000 7f63ba26a740 1 -- 172.21.15.83:0/940255855 wait complete. 2024-04-24T09:04:50.939 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.938+0000 7f63ba26a740 20 client.20358 trim_cache size 0 max 16384 2024-04-24T09:04:50.939 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T09:04:50.939 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T09:04:50.939 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T09:04:50.939 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T09:04:50.940 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.938+0000 7f63b5e00640 1 -- 172.21.15.83:0/2249064362 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f63940a3d10 msgr2=0x7f63940a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:50.940 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.938+0000 7f63b5e00640 1 --2- 172.21.15.83:0/2249064362 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f63940a3d10 0x7f63940a61d0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f63a8008dc0 tx=0x7f63a8008570 comp rx=0 tx=0).stop 2024-04-24T09:04:50.940 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.938+0000 7f63b5e00640 1 -- 172.21.15.83:0/2249064362 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63b0108540 msgr2=0x7f63b01a4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:04:50.940 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.938+0000 7f63b5e00640 1 --2- 172.21.15.83:0/2249064362 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63b0108540 0x7f63b01a4830 secure :-1 s=READY pgs=2062 cs=0 l=1 rev1=1 crypto rx=0x7f63a0031440 tx=0x7f63a0003d50 comp rx=0 tx=0).stop 2024-04-24T09:04:50.940 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.939+0000 7f63b5e00640 1 -- 172.21.15.83:0/2249064362 shutdown_connections 2024-04-24T09:04:50.940 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.939+0000 7f63b5e00640 1 --2- 172.21.15.83:0/2249064362 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f63940a3d10 0x7f63940a61d0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:50.940 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.939+0000 7f63b5e00640 1 --2- 172.21.15.83:0/2249064362 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f63b010e4d0 0x7f63b019ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:50.940 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.939+0000 7f63b5e00640 1 --2- 172.21.15.83:0/2249064362 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f63b0108ec0 0x7f63b01a4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:50.940 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.939+0000 7f63b5e00640 1 --2- 172.21.15.83:0/2249064362 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63b0108540 0x7f63b01a4830 unknown :-1 s=CLOSED pgs=2062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:04:50.940 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.939+0000 7f63b5e00640 1 -- 172.21.15.83:0/2249064362 >> 172.21.15.83:0/2249064362 conn(0x7f63b0077de0 msgr2=0x7f63b0078440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:04:50.941 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.939+0000 7f63b5e00640 1 -- 172.21.15.83:0/2249064362 shutdown_connections 2024-04-24T09:04:50.941 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:04:50.939+0000 7f63b5e00640 1 -- 172.21.15.83:0/2249064362 wait complete. 2024-04-24T09:04:51.629 INFO:tasks.workunit.client.0.smithi083.stdout: 1 76015 0.58 MB/sec execute 565 sec latency 66.947 ms 2024-04-24T09:04:52.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:04:51 smithi138 ceph-mon[31441]: pgmap v416: 129 pgs: 129 active+clean; 72 MiB data, 999 MiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 351 KiB/s wr, 61 op/s 2024-04-24T09:04:52.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:04:51 smithi189 ceph-mon[31791]: pgmap v416: 129 pgs: 129 active+clean; 72 MiB data, 999 MiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 351 KiB/s wr, 61 op/s 2024-04-24T09:04:52.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:04:51 smithi083 ceph-mon[27907]: pgmap v416: 129 pgs: 129 active+clean; 72 MiB data, 999 MiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 351 KiB/s wr, 61 op/s 2024-04-24T09:04:52.629 INFO:tasks.workunit.client.0.smithi083.stdout: 1 76107 0.58 MB/sec execute 566 sec latency 70.456 ms 2024-04-24T09:04:53.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:04:52 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:04:53.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:04:52 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:04:53.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:04:52 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:04:53.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:04:52 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:04:53.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:04:52 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:04:53.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:04:52 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:04:53.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:04:52 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:04:53.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:04:52 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:04:53.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:04:52 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:04:53.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:04:52 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:04:53.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:04:52 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:04:53.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:04:52 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:04:53.629 INFO:tasks.workunit.client.0.smithi083.stdout: 1 76211 0.58 MB/sec execute 567 sec latency 73.108 ms 2024-04-24T09:04:54.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:04:53 smithi138 ceph-mon[31441]: pgmap v417: 129 pgs: 129 active+clean; 68 MiB data, 994 MiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 537 KiB/s wr, 89 op/s 2024-04-24T09:04:54.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:04:53 smithi189 ceph-mon[31791]: pgmap v417: 129 pgs: 129 active+clean; 68 MiB data, 994 MiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 537 KiB/s wr, 89 op/s 2024-04-24T09:04:54.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:04:53 smithi083 ceph-mon[27907]: pgmap v417: 129 pgs: 129 active+clean; 68 MiB data, 994 MiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 537 KiB/s wr, 89 op/s 2024-04-24T09:04:54.629 INFO:tasks.workunit.client.0.smithi083.stdout: 1 76354 0.58 MB/sec execute 568 sec latency 63.711 ms 2024-04-24T09:04:55.629 INFO:tasks.workunit.client.0.smithi083.stdout: 1 76457 0.58 MB/sec execute 569 sec latency 54.034 ms 2024-04-24T09:04:56.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:04:55 smithi138 ceph-mon[31441]: pgmap v418: 129 pgs: 129 active+clean; 68 MiB data, 994 MiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 319 KiB/s wr, 63 op/s 2024-04-24T09:04:56.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:04:55 smithi189 ceph-mon[31791]: pgmap v418: 129 pgs: 129 active+clean; 68 MiB data, 994 MiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 319 KiB/s wr, 63 op/s 2024-04-24T09:04:56.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:04:55 smithi083 ceph-mon[27907]: pgmap v418: 129 pgs: 129 active+clean; 68 MiB data, 994 MiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 319 KiB/s wr, 63 op/s 2024-04-24T09:04:56.629 INFO:tasks.workunit.client.0.smithi083.stdout: 1 76599 0.58 MB/sec execute 570 sec latency 71.851 ms 2024-04-24T09:04:57.631 INFO:tasks.workunit.client.0.smithi083.stdout: 1 76721 0.58 MB/sec execute 571 sec latency 63.944 ms 2024-04-24T09:04:58.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:04:57 smithi138 ceph-mon[31441]: pgmap v419: 129 pgs: 129 active+clean; 68 MiB data, 994 MiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 319 KiB/s wr, 63 op/s 2024-04-24T09:04:58.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:04:57 smithi189 ceph-mon[31791]: pgmap v419: 129 pgs: 129 active+clean; 68 MiB data, 994 MiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 319 KiB/s wr, 63 op/s 2024-04-24T09:04:58.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:04:57 smithi083 ceph-mon[27907]: pgmap v419: 129 pgs: 129 active+clean; 68 MiB data, 994 MiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 319 KiB/s wr, 63 op/s 2024-04-24T09:04:58.629 INFO:tasks.workunit.client.0.smithi083.stdout: 1 76872 0.58 MB/sec execute 572 sec latency 57.870 ms 2024-04-24T09:04:59.630 INFO:tasks.workunit.client.0.smithi083.stdout: 1 76946 0.58 MB/sec execute 573 sec latency 57.579 ms 2024-04-24T09:05:00.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:04:59 smithi083 ceph-mon[27907]: pgmap v420: 129 pgs: 129 active+clean; 69 MiB data, 997 MiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 382 KiB/s wr, 81 op/s 2024-04-24T09:05:00.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:04:59 smithi138 ceph-mon[31441]: pgmap v420: 129 pgs: 129 active+clean; 69 MiB data, 997 MiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 382 KiB/s wr, 81 op/s 2024-04-24T09:05:00.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:04:59 smithi189 ceph-mon[31791]: pgmap v420: 129 pgs: 129 active+clean; 69 MiB data, 997 MiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 382 KiB/s wr, 81 op/s 2024-04-24T09:05:00.630 INFO:tasks.workunit.client.0.smithi083.stdout: 1 77037 0.58 MB/sec execute 574 sec latency 52.400 ms 2024-04-24T09:05:01.630 INFO:tasks.workunit.client.0.smithi083.stdout: 1 77147 0.58 MB/sec execute 575 sec latency 63.589 ms 2024-04-24T09:05:02.259 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:05:01 smithi189 ceph-mon[31791]: pgmap v421: 129 pgs: 129 active+clean; 69 MiB data, 997 MiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 249 KiB/s wr, 45 op/s 2024-04-24T09:05:02.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:05:01 smithi083 ceph-mon[27907]: pgmap v421: 129 pgs: 129 active+clean; 69 MiB data, 997 MiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 249 KiB/s wr, 45 op/s 2024-04-24T09:05:02.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:05:01 smithi138 ceph-mon[31441]: pgmap v421: 129 pgs: 129 active+clean; 69 MiB data, 997 MiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 249 KiB/s wr, 45 op/s 2024-04-24T09:05:02.630 INFO:tasks.workunit.client.0.smithi083.stdout: 1 77416 0.58 MB/sec execute 576 sec latency 54.593 ms 2024-04-24T09:05:03.630 INFO:tasks.workunit.client.0.smithi083.stdout: 1 77532 0.59 MB/sec execute 577 sec latency 51.005 ms 2024-04-24T09:05:04.093 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.094+0000 7fad18000640 1 -- 172.21.15.83:0/2482279413 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad10103bb0 msgr2=0x7fad1010c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:04.094 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.094+0000 7fad18000640 1 --2- 172.21.15.83:0/2482279413 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad10103bb0 0x7fad1010c090 secure :-1 s=READY pgs=2067 cs=0 l=1 rev1=1 crypto rx=0x7facfc009900 tx=0x7facfc02a510 comp rx=0 tx=0).stop 2024-04-24T09:05:04.094 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.094+0000 7fad18000640 1 -- 172.21.15.83:0/2482279413 shutdown_connections 2024-04-24T09:05:04.094 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.094+0000 7fad18000640 1 --2- 172.21.15.83:0/2482279413 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad1010c5d0 0x7fad1010ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:04.094 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.094+0000 7fad18000640 1 --2- 172.21.15.83:0/2482279413 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad10103bb0 0x7fad1010c090 unknown :-1 s=CLOSED pgs=2067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:04.094 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.094+0000 7fad18000640 1 --2- 172.21.15.83:0/2482279413 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fad10103290 0x7fad10103670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:04.094 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.094+0000 7fad18000640 1 -- 172.21.15.83:0/2482279413 >> 172.21.15.83:0/2482279413 conn(0x7fad10078080 msgr2=0x7fad10078490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:04.094 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.094+0000 7fad18000640 1 -- 172.21.15.83:0/2482279413 shutdown_connections 2024-04-24T09:05:04.094 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.095+0000 7fad18000640 1 -- 172.21.15.83:0/2482279413 wait complete. 2024-04-24T09:05:04.096 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.095+0000 7fad18000640 1 Processor -- start 2024-04-24T09:05:04.096 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.096+0000 7fad18000640 1 -- start start 2024-04-24T09:05:04.097 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.096+0000 7fad18000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fad10103290 0x7fad10100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:04.097 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.096+0000 7fad18000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad10103bb0 0x7fad10101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:04.097 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.096+0000 7fad18000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad1010c5d0 0x7fad10101960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:04.097 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.096+0000 7fad18000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fad10112260 con 0x7fad1010c5d0 2024-04-24T09:05:04.097 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.096+0000 7fad18000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fad101120e0 con 0x7fad10103bb0 2024-04-24T09:05:04.097 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.096+0000 7fad18000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fad101123e0 con 0x7fad10103290 2024-04-24T09:05:04.097 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.096+0000 7fad16000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad1010c5d0 0x7fad10101960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:04.097 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.096+0000 7fad16000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad1010c5d0 0x7fad10101960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60716/0 (socket says 172.21.15.83:60716) 2024-04-24T09:05:04.097 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.096+0000 7fad16000640 1 -- 172.21.15.83:0/3373050605 learned_addr learned my addr 172.21.15.83:0/3373050605 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:05:04.097 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.097+0000 7fad15600640 1 --2- 172.21.15.83:0/3373050605 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fad10103290 0x7fad10100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:04.097 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.097+0000 7fad14c00640 1 --2- 172.21.15.83:0/3373050605 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad10103bb0 0x7fad10101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:04.097 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.097+0000 7fad16000640 1 -- 172.21.15.83:0/3373050605 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fad10103290 msgr2=0x7fad10100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:04.097 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.097+0000 7fad16000640 1 --2- 172.21.15.83:0/3373050605 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fad10103290 0x7fad10100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:04.097 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.097+0000 7fad16000640 1 -- 172.21.15.83:0/3373050605 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad10103bb0 msgr2=0x7fad10101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:04.097 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.097+0000 7fad16000640 1 --2- 172.21.15.83:0/3373050605 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad10103bb0 0x7fad10101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:04.097 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.097+0000 7fad16000640 1 -- 172.21.15.83:0/3373050605 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad101a6c60 con 0x7fad1010c5d0 2024-04-24T09:05:04.098 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.097+0000 7fad16000640 1 --2- 172.21.15.83:0/3373050605 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad1010c5d0 0x7fad10101960 secure :-1 s=READY pgs=2068 cs=0 l=1 rev1=1 crypto rx=0x7fad0800eaa0 tx=0x7fad0800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:04.098 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.097+0000 7fad0e000640 1 -- 172.21.15.83:0/3373050605 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad0800cdb0 con 0x7fad1010c5d0 2024-04-24T09:05:04.098 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.098+0000 7fad0e000640 1 -- 172.21.15.83:0/3373050605 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fad08005020 con 0x7fad1010c5d0 2024-04-24T09:05:04.098 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.098+0000 7fad18000640 1 -- 172.21.15.83:0/3373050605 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad101a6f50 con 0x7fad1010c5d0 2024-04-24T09:05:04.098 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.098+0000 7fad0e000640 1 -- 172.21.15.83:0/3373050605 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad080116d0 con 0x7fad1010c5d0 2024-04-24T09:05:04.098 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.098+0000 7fad18000640 1 -- 172.21.15.83:0/3373050605 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad101a7420 con 0x7fad1010c5d0 2024-04-24T09:05:04.100 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.101+0000 7fad1ad95740 1 Processor -- start 2024-04-24T09:05:04.100 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.101+0000 7fad0e000640 1 -- 172.21.15.83:0/3373050605 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fad080051d0 con 0x7fad1010c5d0 2024-04-24T09:05:04.100 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.101+0000 7fad1ad95740 1 -- start start 2024-04-24T09:05:04.101 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.102+0000 7fad1ad95740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55744943c1b0 0x55744943c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:04.101 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.102+0000 7fad1ad95740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55744943cad0 0x557449446e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:04.101 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.102+0000 7fad1ad95740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557449447390 0x557449449780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:04.101 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.102+0000 7fad1ad95740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5574493a2900 con 0x55744943c1b0 2024-04-24T09:05:04.101 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.102+0000 7fad1ad95740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x557449348f20 con 0x55744943cad0 2024-04-24T09:05:04.101 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.102+0000 7fad1ad95740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55744939e8c0 con 0x557449447390 2024-04-24T09:05:04.101 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.102+0000 7fad14c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55744943c1b0 0x55744943c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:04.101 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.102+0000 7fad14c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55744943c1b0 0x55744943c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60718/0 (socket says 172.21.15.83:60718) 2024-04-24T09:05:04.101 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.102+0000 7fad14c00640 1 -- 172.21.15.83:0/617506712 learned_addr learned my addr 172.21.15.83:0/617506712 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:05:04.102 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.102+0000 7fad15600640 1 --2- 172.21.15.83:0/617506712 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55744943cad0 0x557449446e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:04.102 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.102+0000 7fad14c00640 1 -- 172.21.15.83:0/617506712 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557449447390 msgr2=0x557449449780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:04.103 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.102+0000 7fad14c00640 1 --2- 172.21.15.83:0/617506712 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557449447390 0x557449449780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:04.103 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.102+0000 7fad14c00640 1 -- 172.21.15.83:0/617506712 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55744943cad0 msgr2=0x557449446e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:04.103 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.102+0000 7fad14c00640 1 --2- 172.21.15.83:0/617506712 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55744943cad0 0x557449446e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:04.103 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.103+0000 7fad14c00640 1 -- 172.21.15.83:0/617506712 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55744936de10 con 0x55744943c1b0 2024-04-24T09:05:04.103 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.103+0000 7fad0e000640 1 --2- 172.21.15.83:0/3373050605 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7facf80a3df0 0x7facf80a62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:04.103 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.103+0000 7fad14c00640 1 --2- 172.21.15.83:0/617506712 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55744943c1b0 0x55744943c590 secure :-1 s=READY pgs=2069 cs=0 l=1 rev1=1 crypto rx=0x7facfc02aa30 tx=0x7facfc032cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ad7d899b63bc7b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:04.103 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.103+0000 7fad0e000640 1 -- 172.21.15.83:0/3373050605 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7fad080d0e60 con 0x7fad1010c5d0 2024-04-24T09:05:04.103 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.103+0000 7fad15600640 1 --2- 172.21.15.83:0/3373050605 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7facf80a3df0 0x7facf80a62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:04.103 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.103+0000 7fad02600640 1 -- 172.21.15.83:0/617506712 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facfc035050 con 0x55744943c1b0 2024-04-24T09:05:04.103 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.103+0000 7fad02600640 1 -- 172.21.15.83:0/617506712 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7facfc032e20 con 0x55744943c1b0 2024-04-24T09:05:04.103 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.104+0000 7fad02600640 1 -- 172.21.15.83:0/617506712 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facfc005570 con 0x55744943c1b0 2024-04-24T09:05:04.107 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.104+0000 7fad1ad95740 1 -- 172.21.15.83:0/617506712 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55744943c1b0 msgr2=0x55744943c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:04.107 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.104+0000 7fad15600640 1 --2- 172.21.15.83:0/3373050605 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7facf80a3df0 0x7facf80a62b0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fad04005fd0 tx=0x7fad0400b650 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:04.107 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.104+0000 7fad1ad95740 1 --2- 172.21.15.83:0/617506712 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55744943c1b0 0x55744943c590 secure :-1 s=READY pgs=2069 cs=0 l=1 rev1=1 crypto rx=0x7facfc02aa30 tx=0x7facfc032cb0 comp rx=0 tx=0).stop 2024-04-24T09:05:04.107 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.104+0000 7fad1ad95740 1 -- 172.21.15.83:0/617506712 shutdown_connections 2024-04-24T09:05:04.107 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.104+0000 7fad1ad95740 1 --2- 172.21.15.83:0/617506712 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557449447390 0x557449449780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:04.107 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.104+0000 7fad1ad95740 1 --2- 172.21.15.83:0/617506712 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55744943cad0 0x557449446e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:04.107 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.104+0000 7fad1ad95740 1 --2- 172.21.15.83:0/617506712 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55744943c1b0 0x55744943c590 unknown :-1 s=CLOSED pgs=2069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:04.107 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.104+0000 7fad1ad95740 1 -- 172.21.15.83:0/617506712 >> 172.21.15.83:0/617506712 conn(0x5574494386c0 msgr2=0x557449438aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:04.107 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.104+0000 7fad1ad95740 1 -- 172.21.15.83:0/617506712 shutdown_connections 2024-04-24T09:05:04.107 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.105+0000 7fad1ad95740 1 -- 172.21.15.83:0/617506712 wait complete. 2024-04-24T09:05:04.107 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.105+0000 7fad1ad95740 1 -- start start 2024-04-24T09:05:04.108 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.105+0000 7fad1ad95740 1 Processor -- start 2024-04-24T09:05:04.108 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.106+0000 7fad1ad95740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55744943c1b0 0x557449444340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:04.108 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.106+0000 7fad1ad95740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55744943cad0 0x557449444880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:04.108 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.106+0000 7fad1ad95740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557449447390 0x557449444dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:04.108 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.106+0000 7fad1ad95740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x557449449f60 con 0x557449447390 2024-04-24T09:05:04.108 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.106+0000 7fad1ad95740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55744935c2c0 con 0x55744943c1b0 2024-04-24T09:05:04.108 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.106+0000 7fad1ad95740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55744944a0e0 con 0x55744943cad0 2024-04-24T09:05:04.108 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.106+0000 7fad1ad95740 10 client.0 _ll_register_callbacks cb 0x5574490b4a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:05:04.108 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.106+0000 7fad16000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557449447390 0x557449444dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:04.108 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.106+0000 7fad16000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557449447390 0x557449444dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60732/0 (socket says 172.21.15.83:60732) 2024-04-24T09:05:04.108 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.106+0000 7fad16000640 1 -- 172.21.15.83:0/3386518756 learned_addr learned my addr 172.21.15.83:0/3386518756 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:05:04.108 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.106+0000 7fad15600640 1 --2- 172.21.15.83:0/3386518756 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55744943c1b0 0x557449444340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:04.108 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.106+0000 7fad14c00640 1 --2- 172.21.15.83:0/3386518756 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55744943cad0 0x557449444880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:04.108 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.106+0000 7fad16000640 1 -- 172.21.15.83:0/3386518756 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55744943cad0 msgr2=0x557449444880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:04.108 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.107+0000 7fad16000640 1 --2- 172.21.15.83:0/3386518756 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55744943cad0 0x557449444880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:04.109 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.107+0000 7fad16000640 1 -- 172.21.15.83:0/3386518756 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55744943c1b0 msgr2=0x557449444340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:04.109 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.107+0000 7fad16000640 1 --2- 172.21.15.83:0/3386518756 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55744943c1b0 0x557449444340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:04.109 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.107+0000 7fad16000640 1 -- 172.21.15.83:0/3386518756 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55744936cfd0 con 0x557449447390 2024-04-24T09:05:04.109 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.107+0000 7fad15600640 1 --2- 172.21.15.83:0/3386518756 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55744943c1b0 0x557449444340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:05:04.109 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.107+0000 7fad16000640 1 --2- 172.21.15.83:0/3386518756 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557449447390 0x557449444dc0 secure :-1 s=READY pgs=2070 cs=0 l=1 rev1=1 crypto rx=0x7fad080ce190 tx=0x7fad0809c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afb167fc918105e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:04.109 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.107+0000 7facf6a00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:05:04.109 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.107+0000 7facf6a00640 1 -- 172.21.15.83:0/3386518756 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad0801f040 con 0x557449447390 2024-04-24T09:05:04.109 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.107+0000 7facf6a00640 1 -- 172.21.15.83:0/3386518756 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7fad080870c0 con 0x557449447390 2024-04-24T09:05:04.109 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.107+0000 7fad1ad95740 1 -- 172.21.15.83:0/3386518756 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7facfc032e20 con 0x557449447390 2024-04-24T09:05:04.109 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.107+0000 7facf6a00640 1 -- 172.21.15.83:0/3386518756 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fad080a55f0 con 0x557449447390 2024-04-24T09:05:04.109 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.107+0000 7fad1ad95740 10 client.20370 fetch_fsmap learned FSMap version 43 2024-04-24T09:05:04.109 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.107+0000 7fad1ad95740 1 -- 172.21.15.83:0/3386518756 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5574490b5210 con 0x557449447390 2024-04-24T09:05:04.110 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.108+0000 7facf6a00640 1 -- 172.21.15.83:0/3386518756 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7facfc032e20 con 0x557449447390 2024-04-24T09:05:04.110 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.108+0000 7facf6a00640 1 -- 172.21.15.83:0/3386518756 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7fad0800f040 con 0x557449447390 2024-04-24T09:05:04.110 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.108+0000 7fad1ad95740 10 client.20370 fetch_fsmap finished waiting for FSMap version 43 2024-04-24T09:05:04.110 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.108+0000 7fad1ad95740 10 client.20370 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:05:04.110 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.108+0000 7fad1ad95740 20 client.20370 populate_metadata read hostname 'smithi083' 2024-04-24T09:05:04.110 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.108+0000 7fad1ad95740 1 --2- 172.21.15.83:0/3386518756 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x557449457540 0x557449459950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:04.110 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.108+0000 7fad1ad95740 4 client.20370 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:05:04.110 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.108+0000 7fad1ad95740 1 -- 172.21.15.83:0/3386518756 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5574493b0320 con 0x557449457540 2024-04-24T09:05:04.111 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.109+0000 7fad15600640 1 --2- 172.21.15.83:0/3386518756 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x557449457540 0x557449459950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:04.111 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.110+0000 7fad15600640 1 --2- 172.21.15.83:0/3386518756 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x557449457540 0x557449459950 crc :-1 s=READY pgs=347 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51d98c3b7ba65641 server_cookie=292b146271e2f980 in_seq=0 out_seq=0 2024-04-24T09:05:04.111 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.110+0000 7facf6a00640 10 client.20370 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:05:04.112 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.113+0000 7facf6a00640 1 -- 172.21.15.83:0/3386518756 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5574493b0320 con 0x557449457540 2024-04-24T09:05:04.113 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.113+0000 7facf6a00640 10 client.20370 handle_command_reply: tid=0 2024-04-24T09:05:04.113 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.113+0000 7fad1ad95740 1 client.20370 shutdown 2024-04-24T09:05:04.113 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.113+0000 7fad1ad95740 1 -- 172.21.15.83:0/3386518756 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557449447390 msgr2=0x557449444dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:04.113 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.113+0000 7fad1ad95740 1 --2- 172.21.15.83:0/3386518756 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557449447390 0x557449444dc0 secure :-1 s=READY pgs=2070 cs=0 l=1 rev1=1 crypto rx=0x7fad080ce190 tx=0x7fad0809c980 comp rx=0 tx=0).stop 2024-04-24T09:05:04.113 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.114+0000 7fad1ad95740 1 -- 172.21.15.83:0/3386518756 shutdown_connections 2024-04-24T09:05:04.113 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.114+0000 7fad1ad95740 1 --2- 172.21.15.83:0/3386518756 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x557449457540 0x557449459950 crc :-1 s=READY pgs=347 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:04.113 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.114+0000 7fad1ad95740 1 --2- 172.21.15.83:0/3386518756 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557449447390 0x557449444dc0 unknown :-1 s=CLOSED pgs=2070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:04.113 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.114+0000 7fad1ad95740 1 --2- 172.21.15.83:0/3386518756 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55744943cad0 0x557449444880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:04.113 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.114+0000 7facf6a00640 0 client.20370 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:05:04.113 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.114+0000 7fad1ad95740 1 --2- 172.21.15.83:0/3386518756 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55744943c1b0 0x557449444340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:04.113 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.114+0000 7fad1ad95740 1 -- 172.21.15.83:0/3386518756 >> 172.21.15.83:0/3386518756 conn(0x5574494386c0 msgr2=0x55744943e440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:04.113 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.114+0000 7fad1ad95740 1 -- 172.21.15.83:0/3386518756 shutdown_connections 2024-04-24T09:05:04.113 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.114+0000 7fad1ad95740 1 -- 172.21.15.83:0/3386518756 wait complete. 2024-04-24T09:05:04.113 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.114+0000 7fad1ad95740 20 client.20370 trim_cache size 0 max 16384 2024-04-24T09:05:04.131 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.132+0000 7fad1ad95740 1 Processor -- start 2024-04-24T09:05:04.131 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.132+0000 7fad1ad95740 1 -- start start 2024-04-24T09:05:04.131 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.132+0000 7fad1ad95740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55744943c1b0 0x557449452e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:04.131 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.132+0000 7fad1ad95740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55744943cad0 0x55744943ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:04.131 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.132+0000 7fad1ad95740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557449447390 0x5574494376c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:04.131 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.132+0000 7fad1ad95740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fad101b1650 con 0x55744943c1b0 2024-04-24T09:05:04.131 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.132+0000 7fad1ad95740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fad101b3250 con 0x557449447390 2024-04-24T09:05:04.131 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.132+0000 7fad1ad95740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fad101b1d20 con 0x55744943cad0 2024-04-24T09:05:04.133 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.133+0000 7fad15600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55744943c1b0 0x557449452e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:04.133 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.133+0000 7fad15600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55744943c1b0 0x557449452e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60740/0 (socket says 172.21.15.83:60740) 2024-04-24T09:05:04.133 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.133+0000 7fad15600640 1 -- 172.21.15.83:0/1724047244 learned_addr learned my addr 172.21.15.83:0/1724047244 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:05:04.133 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.133+0000 7fad14c00640 1 --2- 172.21.15.83:0/1724047244 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55744943cad0 0x55744943ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:04.133 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.133+0000 7fad16000640 1 --2- 172.21.15.83:0/1724047244 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557449447390 0x5574494376c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:04.133 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.133+0000 7fad15600640 1 -- 172.21.15.83:0/1724047244 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55744943cad0 msgr2=0x55744943ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:04.133 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.133+0000 7fad15600640 1 --2- 172.21.15.83:0/1724047244 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55744943cad0 0x55744943ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:04.133 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.133+0000 7fad15600640 1 -- 172.21.15.83:0/1724047244 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557449447390 msgr2=0x5574494376c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:04.133 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.133+0000 7fad15600640 1 --2- 172.21.15.83:0/1724047244 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557449447390 0x5574494376c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:04.133 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.133+0000 7fad15600640 1 -- 172.21.15.83:0/1724047244 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557449181620 con 0x55744943c1b0 2024-04-24T09:05:04.133 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.133+0000 7fad15600640 1 --2- 172.21.15.83:0/1724047244 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55744943c1b0 0x557449452e20 secure :-1 s=READY pgs=2071 cs=0 l=1 rev1=1 crypto rx=0x55744945a200 tx=0x7fad04014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef5b6e4ff9e49ac8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:04.134 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.134+0000 7fad01c00640 1 -- 172.21.15.83:0/1724047244 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad04012d90 con 0x55744943c1b0 2024-04-24T09:05:04.134 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.134+0000 7fad01c00640 1 -- 172.21.15.83:0/1724047244 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fad04029850 con 0x55744943c1b0 2024-04-24T09:05:04.134 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.134+0000 7fad01c00640 1 -- 172.21.15.83:0/1724047244 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad04004a90 con 0x55744943c1b0 2024-04-24T09:05:04.136 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.135+0000 7fad1ad95740 1 -- 172.21.15.83:0/1724047244 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55744943c1b0 msgr2=0x557449452e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:04.136 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.135+0000 7fad1ad95740 1 --2- 172.21.15.83:0/1724047244 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55744943c1b0 0x557449452e20 secure :-1 s=READY pgs=2071 cs=0 l=1 rev1=1 crypto rx=0x55744945a200 tx=0x7fad04014000 comp rx=0 tx=0).stop 2024-04-24T09:05:04.136 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.135+0000 7fad1ad95740 1 -- 172.21.15.83:0/1724047244 shutdown_connections 2024-04-24T09:05:04.136 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.135+0000 7fad1ad95740 1 --2- 172.21.15.83:0/1724047244 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557449447390 0x5574494376c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:04.136 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.135+0000 7fad1ad95740 1 --2- 172.21.15.83:0/1724047244 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55744943cad0 0x55744943ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:04.136 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.135+0000 7fad1ad95740 1 --2- 172.21.15.83:0/1724047244 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55744943c1b0 0x557449452e20 unknown :-1 s=CLOSED pgs=2071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:04.136 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.135+0000 7fad1ad95740 1 -- 172.21.15.83:0/1724047244 >> 172.21.15.83:0/1724047244 conn(0x5574494386c0 msgr2=0x557449436fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:04.136 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.135+0000 7fad1ad95740 1 -- 172.21.15.83:0/1724047244 shutdown_connections 2024-04-24T09:05:04.136 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.135+0000 7fad1ad95740 1 -- 172.21.15.83:0/1724047244 wait complete. 2024-04-24T09:05:04.137 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.135+0000 7fad1ad95740 1 -- start start 2024-04-24T09:05:04.137 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.136+0000 7fad1ad95740 1 Processor -- start 2024-04-24T09:05:04.137 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.136+0000 7fad1ad95740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55744943c1b0 0x55744944f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:04.137 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.136+0000 7fad1ad95740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55744943cad0 0x557449440dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:04.137 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.136+0000 7fad1ad95740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557449447390 0x557449441340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:04.137 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.136+0000 7fad1ad95740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fad101b2b20 con 0x55744943c1b0 2024-04-24T09:05:04.137 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.136+0000 7fad1ad95740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fad101b23f0 con 0x557449447390 2024-04-24T09:05:04.137 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.136+0000 7fad1ad95740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fad101af650 con 0x55744943cad0 2024-04-24T09:05:04.137 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.136+0000 7fad1ad95740 10 client.0 _ll_register_callbacks cb 0x7fad08005830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:05:04.137 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.136+0000 7fad15600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55744943c1b0 0x55744944f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:04.137 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.136+0000 7fad15600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55744943c1b0 0x55744944f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60752/0 (socket says 172.21.15.83:60752) 2024-04-24T09:05:04.137 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.136+0000 7fad15600640 1 -- 172.21.15.83:0/3706612289 learned_addr learned my addr 172.21.15.83:0/3706612289 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:05:04.137 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.137+0000 7fad14c00640 1 --2- 172.21.15.83:0/3706612289 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55744943cad0 0x557449440dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:04.137 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.137+0000 7fad16000640 1 --2- 172.21.15.83:0/3706612289 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557449447390 0x557449441340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:04.138 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.137+0000 7fad15600640 1 -- 172.21.15.83:0/3706612289 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55744943cad0 msgr2=0x557449440dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:04.138 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.137+0000 7fad15600640 1 --2- 172.21.15.83:0/3706612289 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55744943cad0 0x557449440dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:04.138 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.137+0000 7fad15600640 1 -- 172.21.15.83:0/3706612289 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557449447390 msgr2=0x557449441340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:04.138 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.137+0000 7fad15600640 1 --2- 172.21.15.83:0/3706612289 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557449447390 0x557449441340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:04.138 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.137+0000 7fad15600640 1 -- 172.21.15.83:0/3706612289 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557449441b70 con 0x55744943c1b0 2024-04-24T09:05:04.138 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.137+0000 7fad14c00640 1 --2- 172.21.15.83:0/3706612289 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55744943cad0 0x557449440dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:05:04.138 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.137+0000 7fad15600640 1 --2- 172.21.15.83:0/3706612289 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55744943c1b0 0x55744944f810 secure :-1 s=READY pgs=2072 cs=0 l=1 rev1=1 crypto rx=0x7fad0400bec0 tx=0x7fad040057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8191b4b60399af0f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:04.138 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.137+0000 7facf6a00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:05:04.138 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.137+0000 7facf6a00640 1 -- 172.21.15.83:0/3706612289 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad04012e10 con 0x55744943c1b0 2024-04-24T09:05:04.138 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.137+0000 7facf6a00640 1 -- 172.21.15.83:0/3706612289 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7fad040295b0 con 0x55744943c1b0 2024-04-24T09:05:04.138 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.137+0000 7fad1ad95740 1 -- 172.21.15.83:0/3706612289 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad04029850 con 0x55744943c1b0 2024-04-24T09:05:04.139 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.138+0000 7facf6a00640 1 -- 172.21.15.83:0/3706612289 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fad04041050 con 0x55744943c1b0 2024-04-24T09:05:04.140 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.138+0000 7fad1ad95740 10 client.20376 fetch_fsmap learned FSMap version 43 2024-04-24T09:05:04.140 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.138+0000 7fad1ad95740 1 -- 172.21.15.83:0/3706612289 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5574494bdb80 con 0x55744943c1b0 2024-04-24T09:05:04.140 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.138+0000 7facf6a00640 1 -- 172.21.15.83:0/3706612289 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fad04029850 con 0x55744943c1b0 2024-04-24T09:05:04.140 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.138+0000 7facf6a00640 1 -- 172.21.15.83:0/3706612289 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7fad040204c0 con 0x55744943c1b0 2024-04-24T09:05:04.140 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.138+0000 7fad1ad95740 10 client.20376 fetch_fsmap finished waiting for FSMap version 43 2024-04-24T09:05:04.140 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.138+0000 7fad1ad95740 10 client.20376 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:05:04.140 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.138+0000 7fad1ad95740 20 client.20376 populate_metadata read hostname 'smithi083' 2024-04-24T09:05:04.140 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.138+0000 7fad1ad95740 1 --2- 172.21.15.83:0/3706612289 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5574494be260 0x5574494c0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:04.140 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.138+0000 7fad1ad95740 4 client.20376 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T09:05:04.140 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.138+0000 7fad1ad95740 1 -- 172.21.15.83:0/3706612289 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5574493aa190 con 0x5574494be260 2024-04-24T09:05:04.140 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.139+0000 7fad14c00640 1 --2- 172.21.15.83:0/3706612289 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5574494be260 0x5574494c0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:04.140 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.140+0000 7fad14c00640 1 --2- 172.21.15.83:0/3706612289 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5574494be260 0x5574494c0680 crc :-1 s=READY pgs=348 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e138503aeb58d990 server_cookie=197304d036b67187 in_seq=0 out_seq=0 2024-04-24T09:05:04.140 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.140+0000 7facf6a00640 10 client.20376 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:05:04.201 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.202+0000 7facf6a00640 1 -- 172.21.15.83:0/3706612289 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5574493aa190 con 0x5574494be260 2024-04-24T09:05:04.201 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.202+0000 7facf6a00640 10 client.20376 handle_command_reply: tid=0 2024-04-24T09:05:04.202 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.202+0000 7fad1ad95740 1 client.20376 shutdown 2024-04-24T09:05:04.202 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.202+0000 7fad1ad95740 1 -- 172.21.15.83:0/3706612289 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55744943c1b0 msgr2=0x55744944f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:04.202 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.202+0000 7fad1ad95740 1 --2- 172.21.15.83:0/3706612289 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55744943c1b0 0x55744944f810 secure :-1 s=READY pgs=2072 cs=0 l=1 rev1=1 crypto rx=0x7fad0400bec0 tx=0x7fad040057c0 comp rx=0 tx=0).stop 2024-04-24T09:05:04.202 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.202+0000 7fad1ad95740 1 -- 172.21.15.83:0/3706612289 shutdown_connections 2024-04-24T09:05:04.202 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.202+0000 7fad1ad95740 1 --2- 172.21.15.83:0/3706612289 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5574494be260 0x5574494c0680 crc :-1 s=READY pgs=348 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:04.202 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.202+0000 7fad1ad95740 1 --2- 172.21.15.83:0/3706612289 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557449447390 0x557449441340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:04.202 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.202+0000 7fad1ad95740 1 --2- 172.21.15.83:0/3706612289 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55744943cad0 0x557449440dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:04.202 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.202+0000 7fad1ad95740 1 --2- 172.21.15.83:0/3706612289 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55744943c1b0 0x55744944f810 unknown :-1 s=CLOSED pgs=2072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:04.202 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.202+0000 7facf6a00640 0 client.20376 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:05:04.202 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.202+0000 7fad1ad95740 1 -- 172.21.15.83:0/3706612289 >> 172.21.15.83:0/3706612289 conn(0x5574494386c0 msgr2=0x557449437840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:04.202 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.203+0000 7fad1ad95740 1 -- 172.21.15.83:0/3706612289 shutdown_connections 2024-04-24T09:05:04.202 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.203+0000 7fad1ad95740 1 -- 172.21.15.83:0/3706612289 wait complete. 2024-04-24T09:05:04.203 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.203+0000 7fad1ad95740 20 client.20376 trim_cache size 0 max 16384 2024-04-24T09:05:04.203 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T09:05:04.203 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T09:05:04.203 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T09:05:04.203 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T09:05:04.204 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.204+0000 7fad18000640 1 -- 172.21.15.83:0/3373050605 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7facf80a3df0 msgr2=0x7facf80a62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:04.204 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.204+0000 7fad18000640 1 --2- 172.21.15.83:0/3373050605 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7facf80a3df0 0x7facf80a62b0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fad04005fd0 tx=0x7fad0400b650 comp rx=0 tx=0).stop 2024-04-24T09:05:04.204 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.204+0000 7fad18000640 1 -- 172.21.15.83:0/3373050605 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad1010c5d0 msgr2=0x7fad10101960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:04.204 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.204+0000 7fad18000640 1 --2- 172.21.15.83:0/3373050605 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad1010c5d0 0x7fad10101960 secure :-1 s=READY pgs=2068 cs=0 l=1 rev1=1 crypto rx=0x7fad0800eaa0 tx=0x7fad0800ef60 comp rx=0 tx=0).stop 2024-04-24T09:05:04.204 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.205+0000 7fad18000640 1 -- 172.21.15.83:0/3373050605 shutdown_connections 2024-04-24T09:05:04.204 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.205+0000 7fad18000640 1 --2- 172.21.15.83:0/3373050605 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7facf80a3df0 0x7facf80a62b0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:04.205 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.205+0000 7fad18000640 1 --2- 172.21.15.83:0/3373050605 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad1010c5d0 0x7fad10101960 unknown :-1 s=CLOSED pgs=2068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:04.205 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.205+0000 7fad18000640 1 --2- 172.21.15.83:0/3373050605 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad10103bb0 0x7fad10101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:04.205 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.205+0000 7fad18000640 1 --2- 172.21.15.83:0/3373050605 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fad10103290 0x7fad10100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:04.205 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.205+0000 7fad18000640 1 -- 172.21.15.83:0/3373050605 >> 172.21.15.83:0/3373050605 conn(0x7fad10078080 msgr2=0x7fad100fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:04.205 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.205+0000 7fad18000640 1 -- 172.21.15.83:0/3373050605 shutdown_connections 2024-04-24T09:05:04.205 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:04.205+0000 7fad18000640 1 -- 172.21.15.83:0/3373050605 wait complete. 2024-04-24T09:05:04.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:05:03 smithi083 ceph-mon[27907]: pgmap v422: 129 pgs: 129 active+clean; 71 MiB data, 985 MiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 504 KiB/s wr, 80 op/s 2024-04-24T09:05:04.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:05:03 smithi138 ceph-mon[31441]: pgmap v422: 129 pgs: 129 active+clean; 71 MiB data, 985 MiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 504 KiB/s wr, 80 op/s 2024-04-24T09:05:04.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:05:03 smithi189 ceph-mon[31791]: pgmap v422: 129 pgs: 129 active+clean; 71 MiB data, 985 MiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 504 KiB/s wr, 80 op/s 2024-04-24T09:05:04.630 INFO:tasks.workunit.client.0.smithi083.stdout: 1 77779 0.59 MB/sec execute 578 sec latency 55.389 ms 2024-04-24T09:05:05.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:05:05 smithi083 ceph-mon[27907]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:05:05.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:05:05 smithi138 ceph-mon[31441]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:05:05.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:05:05 smithi189 ceph-mon[31791]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:05:05.630 INFO:tasks.workunit.client.0.smithi083.stdout: 1 77844 0.59 MB/sec execute 579 sec latency 54.544 ms 2024-04-24T09:05:06.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:05:06 smithi083 ceph-mon[27907]: pgmap v423: 129 pgs: 129 active+clean; 71 MiB data, 985 MiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 318 KiB/s wr, 52 op/s 2024-04-24T09:05:06.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:05:06 smithi138 ceph-mon[31441]: pgmap v423: 129 pgs: 129 active+clean; 71 MiB data, 985 MiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 318 KiB/s wr, 52 op/s 2024-04-24T09:05:06.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:05:06 smithi189 ceph-mon[31791]: pgmap v423: 129 pgs: 129 active+clean; 71 MiB data, 985 MiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 318 KiB/s wr, 52 op/s 2024-04-24T09:05:06.630 INFO:tasks.workunit.client.0.smithi083.stdout: 1 77868 0.59 MB/sec execute 580 sec latency 66.076 ms 2024-04-24T09:05:07.631 INFO:tasks.workunit.client.0.smithi083.stdout: 1 77934 0.58 MB/sec execute 581 sec latency 55.140 ms 2024-04-24T09:05:08.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:05:08 smithi083 ceph-mon[27907]: pgmap v424: 129 pgs: 129 active+clean; 71 MiB data, 985 MiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 318 KiB/s wr, 52 op/s 2024-04-24T09:05:08.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:05:08 smithi138 ceph-mon[31441]: pgmap v424: 129 pgs: 129 active+clean; 71 MiB data, 985 MiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 318 KiB/s wr, 52 op/s 2024-04-24T09:05:08.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:05:08 smithi189 ceph-mon[31791]: pgmap v424: 129 pgs: 129 active+clean; 71 MiB data, 985 MiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 318 KiB/s wr, 52 op/s 2024-04-24T09:05:08.631 INFO:tasks.workunit.client.0.smithi083.stdout: 1 78106 0.59 MB/sec execute 582 sec latency 62.593 ms 2024-04-24T09:05:09.631 INFO:tasks.workunit.client.0.smithi083.stdout: 1 78159 0.59 MB/sec execute 583 sec latency 715.130 ms 2024-04-24T09:05:10.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:05:10 smithi083 ceph-mon[27907]: pgmap v425: 129 pgs: 129 active+clean; 74 MiB data, 991 MiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 612 KiB/s wr, 74 op/s 2024-04-24T09:05:10.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:05:10 smithi138 ceph-mon[31441]: pgmap v425: 129 pgs: 129 active+clean; 74 MiB data, 991 MiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 612 KiB/s wr, 74 op/s 2024-04-24T09:05:10.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:05:10 smithi189 ceph-mon[31791]: pgmap v425: 129 pgs: 129 active+clean; 74 MiB data, 991 MiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 612 KiB/s wr, 74 op/s 2024-04-24T09:05:10.631 INFO:tasks.workunit.client.0.smithi083.stdout: 1 78159 0.59 MB/sec execute 584 sec latency 1715.223 ms 2024-04-24T09:05:11.631 INFO:tasks.workunit.client.0.smithi083.stdout: 1 78159 0.59 MB/sec execute 585 sec latency 2715.366 ms 2024-04-24T09:05:12.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:05:12 smithi083 ceph-mon[27907]: pgmap v426: 129 pgs: 129 active+clean; 74 MiB data, 991 MiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 549 KiB/s wr, 56 op/s 2024-04-24T09:05:12.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:05:12 smithi138 ceph-mon[31441]: pgmap v426: 129 pgs: 129 active+clean; 74 MiB data, 991 MiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 549 KiB/s wr, 56 op/s 2024-04-24T09:05:12.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:05:12 smithi189 ceph-mon[31791]: pgmap v426: 129 pgs: 129 active+clean; 74 MiB data, 991 MiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 549 KiB/s wr, 56 op/s 2024-04-24T09:05:12.631 INFO:tasks.workunit.client.0.smithi083.stdout: 1 78159 0.58 MB/sec execute 586 sec latency 3715.498 ms 2024-04-24T09:05:13.089 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T09:05:13.090 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T09:05:13.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.298+0000 7ff514a00640 1 -- 172.21.15.83:0/170438129 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff510103a20 msgr2=0x7ff51010bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:13.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.298+0000 7ff514a00640 1 --2- 172.21.15.83:0/170438129 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff510103a20 0x7ff51010bf00 secure :-1 s=READY pgs=2073 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc009900 tx=0x7ff4fc02a480 comp rx=0 tx=0).stop 2024-04-24T09:05:13.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.298+0000 7ff514a00640 1 -- 172.21.15.83:0/170438129 shutdown_connections 2024-04-24T09:05:13.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.298+0000 7ff514a00640 1 --2- 172.21.15.83:0/170438129 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff51010c4d0 0x7ff51010e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:13.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.298+0000 7ff514a00640 1 --2- 172.21.15.83:0/170438129 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff510103a20 0x7ff51010bf00 unknown :-1 s=CLOSED pgs=2073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:13.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.298+0000 7ff514a00640 1 --2- 172.21.15.83:0/170438129 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff510103100 0x7ff5101034e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:13.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.298+0000 7ff514a00640 1 -- 172.21.15.83:0/170438129 >> 172.21.15.83:0/170438129 conn(0x7ff5100fc880 msgr2=0x7ff5100feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:13.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.298+0000 7ff514a00640 1 -- 172.21.15.83:0/170438129 shutdown_connections 2024-04-24T09:05:13.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.298+0000 7ff514a00640 1 -- 172.21.15.83:0/170438129 wait complete. 2024-04-24T09:05:13.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.299+0000 7ff514a00640 1 Processor -- start 2024-04-24T09:05:13.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.299+0000 7ff514a00640 1 -- start start 2024-04-24T09:05:13.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.300+0000 7ff514a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff510103100 0x7ff510100d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:13.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.300+0000 7ff514a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff510103a20 0x7ff510101290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:13.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.300+0000 7ff514a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff51010c4d0 0x7ff5101017d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:13.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.300+0000 7ff514a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff510112160 con 0x7ff510103100 2024-04-24T09:05:13.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.300+0000 7ff514a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff510111fe0 con 0x7ff51010c4d0 2024-04-24T09:05:13.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.300+0000 7ff514a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff5101122e0 con 0x7ff510103a20 2024-04-24T09:05:13.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.300+0000 7ff50e800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff510103100 0x7ff510100d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:13.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.300+0000 7ff50e800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff510103100 0x7ff510100d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35970/0 (socket says 172.21.15.83:35970) 2024-04-24T09:05:13.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.300+0000 7ff50e800640 1 -- 172.21.15.83:0/1715601339 learned_addr learned my addr 172.21.15.83:0/1715601339 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:05:13.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.300+0000 7ff50f200640 1 --2- 172.21.15.83:0/1715601339 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff51010c4d0 0x7ff5101017d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:13.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.300+0000 7ff50de00640 1 --2- 172.21.15.83:0/1715601339 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff510103a20 0x7ff510101290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:13.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.300+0000 7ff50e800640 1 -- 172.21.15.83:0/1715601339 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff510103a20 msgr2=0x7ff510101290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:13.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.300+0000 7ff50e800640 1 --2- 172.21.15.83:0/1715601339 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff510103a20 0x7ff510101290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:13.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.300+0000 7ff50e800640 1 -- 172.21.15.83:0/1715601339 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff51010c4d0 msgr2=0x7ff5101017d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:13.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.300+0000 7ff50e800640 1 --2- 172.21.15.83:0/1715601339 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff51010c4d0 0x7ff5101017d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:13.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.300+0000 7ff50e800640 1 -- 172.21.15.83:0/1715601339 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5101a6be0 con 0x7ff510103100 2024-04-24T09:05:13.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.300+0000 7ff50f200640 1 --2- 172.21.15.83:0/1715601339 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff51010c4d0 0x7ff5101017d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:05:13.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.301+0000 7ff50e800640 1 --2- 172.21.15.83:0/1715601339 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff510103100 0x7ff510100d50 secure :-1 s=READY pgs=2074 cs=0 l=1 rev1=1 crypto rx=0x7ff50000d950 tx=0x7ff50000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:13.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.301+0000 7ff507400640 1 -- 172.21.15.83:0/1715601339 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff500017070 con 0x7ff510103100 2024-04-24T09:05:13.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.301+0000 7ff514a00640 1 -- 172.21.15.83:0/1715601339 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5101a6ed0 con 0x7ff510103100 2024-04-24T09:05:13.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.301+0000 7ff507400640 1 -- 172.21.15.83:0/1715601339 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff500009c30 con 0x7ff510103100 2024-04-24T09:05:13.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.301+0000 7ff507400640 1 -- 172.21.15.83:0/1715601339 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff50000f620 con 0x7ff510103100 2024-04-24T09:05:13.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.301+0000 7ff514a00640 1 -- 172.21.15.83:0/1715601339 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5101a73e0 con 0x7ff510103100 2024-04-24T09:05:13.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:05:13 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:05:13.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.305+0000 7ff5181d8740 1 Processor -- start 2024-04-24T09:05:13.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.306+0000 7ff5181d8740 1 -- start start 2024-04-24T09:05:13.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.306+0000 7ff5181d8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55faa0275a30 0x55faa0275e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:13.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.306+0000 7ff5181d8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55faa0276350 0x55faa02804e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:13.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.306+0000 7ff5181d8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55faa0280a20 0x55faa0282e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:13.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.306+0000 7ff5181d8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55faa01db1f0 con 0x55faa0276350 2024-04-24T09:05:13.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.306+0000 7ff5181d8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55faa0181350 con 0x55faa0275a30 2024-04-24T09:05:13.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.306+0000 7ff5181d8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55faa01d71b0 con 0x55faa0280a20 2024-04-24T09:05:13.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.307+0000 7ff507400640 1 -- 172.21.15.83:0/1715601339 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff5000109a0 con 0x7ff510103100 2024-04-24T09:05:13.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.307+0000 7ff50de00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55faa0280a20 0x55faa0282e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:13.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.307+0000 7ff50de00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55faa0280a20 0x55faa0282e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:43296/0 (socket says 172.21.15.83:43296) 2024-04-24T09:05:13.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.307+0000 7ff50de00640 1 -- 172.21.15.83:0/2881433413 learned_addr learned my addr 172.21.15.83:0/2881433413 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:05:13.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.307+0000 7ff50f200640 1 --2- 172.21.15.83:0/2881433413 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55faa0275a30 0x55faa0275e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:13.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.307+0000 7ff50e800640 1 --2- 172.21.15.83:0/2881433413 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55faa0276350 0x55faa02804e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:13.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.308+0000 7ff50de00640 1 -- 172.21.15.83:0/2881433413 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55faa0275a30 msgr2=0x55faa0275e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:13.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.308+0000 7ff50de00640 1 --2- 172.21.15.83:0/2881433413 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55faa0275a30 0x55faa0275e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:13.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.308+0000 7ff50de00640 1 -- 172.21.15.83:0/2881433413 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55faa0276350 msgr2=0x55faa02804e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:13.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.308+0000 7ff50de00640 1 --2- 172.21.15.83:0/2881433413 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55faa0276350 0x55faa02804e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:13.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.308+0000 7ff50de00640 1 -- 172.21.15.83:0/2881433413 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55faa01d5f50 con 0x55faa0280a20 2024-04-24T09:05:13.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.308+0000 7ff50e800640 1 --2- 172.21.15.83:0/2881433413 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55faa0276350 0x55faa02804e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:05:13.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.308+0000 7ff50de00640 1 --2- 172.21.15.83:0/2881433413 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55faa0280a20 0x55faa0282e10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc005ab0 tx=0x7ff4fc004170 comp rx=0 tx=0).ready entity=mon.1 client_cookie=956788daaaf8051a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:13.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.309+0000 7ff507400640 1 --2- 172.21.15.83:0/1715601339 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff4f40a3da0 0x7ff4f40a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:13.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.309+0000 7ff4fb400640 1 -- 172.21.15.83:0/2881433413 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4fc03f070 con 0x55faa0280a20 2024-04-24T09:05:13.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.309+0000 7ff507400640 1 -- 172.21.15.83:0/1715601339 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7ff5000d0830 con 0x7ff510103100 2024-04-24T09:05:13.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.309+0000 7ff4fb400640 1 -- 172.21.15.83:0/2881433413 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff4fc0054f0 con 0x55faa0280a20 2024-04-24T09:05:13.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.309+0000 7ff4fb400640 1 -- 172.21.15.83:0/2881433413 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4fc035030 con 0x55faa0280a20 2024-04-24T09:05:13.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.309+0000 7ff50f200640 1 --2- 172.21.15.83:0/1715601339 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff4f40a3da0 0x7ff4f40a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:13.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.310+0000 7ff5181d8740 1 -- 172.21.15.83:0/2881433413 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55faa0280a20 msgr2=0x55faa0282e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:13.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.310+0000 7ff50f200640 1 --2- 172.21.15.83:0/1715601339 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff4f40a3da0 0x7ff4f40a6260 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7ff508005fd0 tx=0x7ff5080085b0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:13.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.310+0000 7ff5181d8740 1 --2- 172.21.15.83:0/2881433413 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55faa0280a20 0x55faa0282e10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc005ab0 tx=0x7ff4fc004170 comp rx=0 tx=0).stop 2024-04-24T09:05:13.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.310+0000 7ff5181d8740 1 -- 172.21.15.83:0/2881433413 shutdown_connections 2024-04-24T09:05:13.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.310+0000 7ff5181d8740 1 --2- 172.21.15.83:0/2881433413 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55faa0280a20 0x55faa0282e10 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:13.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.310+0000 7ff5181d8740 1 --2- 172.21.15.83:0/2881433413 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55faa0276350 0x55faa02804e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:13.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.310+0000 7ff5181d8740 1 --2- 172.21.15.83:0/2881433413 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55faa0275a30 0x55faa0275e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:13.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.310+0000 7ff5181d8740 1 -- 172.21.15.83:0/2881433413 >> 172.21.15.83:0/2881433413 conn(0x55faa0271d50 msgr2=0x55faa0272130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:13.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.310+0000 7ff5181d8740 1 -- 172.21.15.83:0/2881433413 shutdown_connections 2024-04-24T09:05:13.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.310+0000 7ff5181d8740 1 -- 172.21.15.83:0/2881433413 wait complete. 2024-04-24T09:05:13.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.311+0000 7ff5181d8740 1 -- start start 2024-04-24T09:05:13.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.311+0000 7ff5181d8740 1 Processor -- start 2024-04-24T09:05:13.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.312+0000 7ff5181d8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55faa0275a30 0x55faa027d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:13.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.312+0000 7ff5181d8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55faa0276350 0x55faa027de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:13.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.312+0000 7ff5181d8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55faa0280a20 0x55faa027e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:13.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.312+0000 7ff5181d8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55faa02837b0 con 0x55faa0275a30 2024-04-24T09:05:13.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.312+0000 7ff5181d8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55faa0194690 con 0x55faa0280a20 2024-04-24T09:05:13.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.312+0000 7ff5181d8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55faa0283930 con 0x55faa0276350 2024-04-24T09:05:13.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.312+0000 7ff5181d8740 10 client.0 _ll_register_callbacks cb 0x55faa01d0650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:05:13.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.312+0000 7ff50e800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55faa0275a30 0x55faa027d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:13.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.312+0000 7ff50e800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55faa0275a30 0x55faa027d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35980/0 (socket says 172.21.15.83:35980) 2024-04-24T09:05:13.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.312+0000 7ff50e800640 1 -- 172.21.15.83:0/743320072 learned_addr learned my addr 172.21.15.83:0/743320072 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:05:13.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.313+0000 7ff50f200640 1 --2- 172.21.15.83:0/743320072 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55faa0280a20 0x55faa027e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:13.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.313+0000 7ff50e800640 1 -- 172.21.15.83:0/743320072 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55faa0276350 msgr2=0x55faa027de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:13.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.313+0000 7ff50e800640 1 --2- 172.21.15.83:0/743320072 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55faa0276350 0x55faa027de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:13.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.313+0000 7ff50e800640 1 -- 172.21.15.83:0/743320072 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55faa0280a20 msgr2=0x55faa027e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:13.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.313+0000 7ff50e800640 1 --2- 172.21.15.83:0/743320072 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55faa0280a20 0x55faa027e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:13.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.313+0000 7ff50e800640 1 -- 172.21.15.83:0/743320072 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55faa0193920 con 0x55faa0275a30 2024-04-24T09:05:13.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.313+0000 7ff50e800640 1 --2- 172.21.15.83:0/743320072 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55faa0275a30 0x55faa027d8c0 secure :-1 s=READY pgs=2075 cs=0 l=1 rev1=1 crypto rx=0x7ff50009c340 tx=0x7ff50009bef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fb2ac3137c493fb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:13.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.313+0000 7ff4efe00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:05:13.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.313+0000 7ff4efe00640 1 -- 172.21.15.83:0/743320072 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff500020070 con 0x55faa0275a30 2024-04-24T09:05:13.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.313+0000 7ff4efe00640 1 -- 172.21.15.83:0/743320072 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7ff5000d03b0 con 0x55faa0275a30 2024-04-24T09:05:13.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.314+0000 7ff5181d8740 1 -- 172.21.15.83:0/743320072 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4fc0054f0 con 0x55faa0275a30 2024-04-24T09:05:13.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.314+0000 7ff4efe00640 1 -- 172.21.15.83:0/743320072 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7ff5000cdfe0 con 0x55faa0275a30 2024-04-24T09:05:13.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.314+0000 7ff5181d8740 10 client.20388 fetch_fsmap learned FSMap version 43 2024-04-24T09:05:13.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.314+0000 7ff5181d8740 1 -- 172.21.15.83:0/743320072 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55fa9ffb77d0 con 0x55faa0275a30 2024-04-24T09:05:13.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.314+0000 7ff4efe00640 1 -- 172.21.15.83:0/743320072 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff4fc0054f0 con 0x55faa0275a30 2024-04-24T09:05:13.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.314+0000 7ff4efe00640 1 -- 172.21.15.83:0/743320072 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7ff500098100 con 0x55faa0275a30 2024-04-24T09:05:13.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.315+0000 7ff5181d8740 10 client.20388 fetch_fsmap finished waiting for FSMap version 43 2024-04-24T09:05:13.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.315+0000 7ff5181d8740 10 client.20388 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:05:13.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.315+0000 7ff5181d8740 20 client.20388 populate_metadata read hostname 'smithi083' 2024-04-24T09:05:13.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.315+0000 7ff5181d8740 1 --2- 172.21.15.83:0/743320072 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55faa0290c70 0x55faa0293130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:13.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.315+0000 7ff5181d8740 4 client.20388 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:05:13.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.315+0000 7ff5181d8740 1 -- 172.21.15.83:0/743320072 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55faa01e8c10 con 0x55faa0290c70 2024-04-24T09:05:13.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.315+0000 7ff50de00640 1 --2- 172.21.15.83:0/743320072 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55faa0290c70 0x55faa0293130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:13.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.316+0000 7ff50de00640 1 --2- 172.21.15.83:0/743320072 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55faa0290c70 0x55faa0293130 crc :-1 s=READY pgs=349 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2caa0d06edd30a23 server_cookie=be5d498ec39e40cb in_seq=0 out_seq=0 2024-04-24T09:05:13.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.316+0000 7ff4efe00640 10 client.20388 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:05:13.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.320+0000 7ff4efe00640 1 -- 172.21.15.83:0/743320072 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55faa01e8c10 con 0x55faa0290c70 2024-04-24T09:05:13.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.320+0000 7ff4efe00640 10 client.20388 handle_command_reply: tid=0 2024-04-24T09:05:13.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.320+0000 7ff5181d8740 1 client.20388 shutdown 2024-04-24T09:05:13.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.321+0000 7ff5181d8740 1 -- 172.21.15.83:0/743320072 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55faa0275a30 msgr2=0x55faa027d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:13.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.321+0000 7ff5181d8740 1 --2- 172.21.15.83:0/743320072 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55faa0275a30 0x55faa027d8c0 secure :-1 s=READY pgs=2075 cs=0 l=1 rev1=1 crypto rx=0x7ff50009c340 tx=0x7ff50009bef0 comp rx=0 tx=0).stop 2024-04-24T09:05:13.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.321+0000 7ff5181d8740 1 -- 172.21.15.83:0/743320072 shutdown_connections 2024-04-24T09:05:13.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.321+0000 7ff5181d8740 1 --2- 172.21.15.83:0/743320072 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55faa0290c70 0x55faa0293130 crc :-1 s=READY pgs=349 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:13.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.321+0000 7ff5181d8740 1 --2- 172.21.15.83:0/743320072 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55faa0280a20 0x55faa027e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:13.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.321+0000 7ff5181d8740 1 --2- 172.21.15.83:0/743320072 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55faa0276350 0x55faa027de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:13.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.321+0000 7ff4efe00640 0 client.20388 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:05:13.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.321+0000 7ff5181d8740 1 --2- 172.21.15.83:0/743320072 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55faa0275a30 0x55faa027d8c0 unknown :-1 s=CLOSED pgs=2075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:13.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.321+0000 7ff5181d8740 1 -- 172.21.15.83:0/743320072 >> 172.21.15.83:0/743320072 conn(0x55faa0271d50 msgr2=0x55faa0270fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:13.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.321+0000 7ff5181d8740 1 -- 172.21.15.83:0/743320072 shutdown_connections 2024-04-24T09:05:13.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.321+0000 7ff5181d8740 1 -- 172.21.15.83:0/743320072 wait complete. 2024-04-24T09:05:13.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.321+0000 7ff5181d8740 20 client.20388 trim_cache size 0 max 16384 2024-04-24T09:05:13.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.342+0000 7ff5181d8740 1 Processor -- start 2024-04-24T09:05:13.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.342+0000 7ff5181d8740 1 -- start start 2024-04-24T09:05:13.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.342+0000 7ff5181d8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55faa0275a30 0x55faa028c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:13.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.343+0000 7ff5181d8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55faa0276350 0x55faa0277890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:13.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.343+0000 7ff5181d8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55faa0280a20 0x55faa0298a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:13.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.343+0000 7ff5181d8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff5101b16e0 con 0x55faa0275a30 2024-04-24T09:05:13.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.343+0000 7ff5181d8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff5101b32e0 con 0x55faa0280a20 2024-04-24T09:05:13.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.343+0000 7ff5181d8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff5101b1db0 con 0x55faa0276350 2024-04-24T09:05:13.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.343+0000 7ff50e800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55faa0275a30 0x55faa028c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:13.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.343+0000 7ff50e800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55faa0275a30 0x55faa028c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35982/0 (socket says 172.21.15.83:35982) 2024-04-24T09:05:13.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.343+0000 7ff50e800640 1 -- 172.21.15.83:0/2021946222 learned_addr learned my addr 172.21.15.83:0/2021946222 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:05:13.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.343+0000 7ff50e800640 1 -- 172.21.15.83:0/2021946222 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55faa0276350 msgr2=0x55faa0277890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:13.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.343+0000 7ff50e800640 1 --2- 172.21.15.83:0/2021946222 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55faa0276350 0x55faa0277890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:13.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.343+0000 7ff50e800640 1 -- 172.21.15.83:0/2021946222 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55faa0280a20 msgr2=0x55faa0298a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:13.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.343+0000 7ff50e800640 1 --2- 172.21.15.83:0/2021946222 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55faa0280a20 0x55faa0298a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:13.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.343+0000 7ff50e800640 1 -- 172.21.15.83:0/2021946222 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa9feed3f0 con 0x55faa0275a30 2024-04-24T09:05:13.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.344+0000 7ff50e800640 1 --2- 172.21.15.83:0/2021946222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55faa0275a30 0x55faa028c670 secure :-1 s=READY pgs=2076 cs=0 l=1 rev1=1 crypto rx=0x7ff50009c0b0 tx=0x7ff500005f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6de4742f2b7f5b8d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:13.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.344+0000 7ff4faa00640 1 -- 172.21.15.83:0/2021946222 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5000ba070 con 0x55faa0275a30 2024-04-24T09:05:13.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.344+0000 7ff4faa00640 1 -- 172.21.15.83:0/2021946222 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff5000b0e50 con 0x55faa0275a30 2024-04-24T09:05:13.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.344+0000 7ff4faa00640 1 -- 172.21.15.83:0/2021946222 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff500014030 con 0x55faa0275a30 2024-04-24T09:05:13.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.345+0000 7ff5181d8740 1 -- 172.21.15.83:0/2021946222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55faa0275a30 msgr2=0x55faa028c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:13.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.345+0000 7ff5181d8740 1 --2- 172.21.15.83:0/2021946222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55faa0275a30 0x55faa028c670 secure :-1 s=READY pgs=2076 cs=0 l=1 rev1=1 crypto rx=0x7ff50009c0b0 tx=0x7ff500005f70 comp rx=0 tx=0).stop 2024-04-24T09:05:13.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.345+0000 7ff5181d8740 1 -- 172.21.15.83:0/2021946222 shutdown_connections 2024-04-24T09:05:13.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.345+0000 7ff5181d8740 1 --2- 172.21.15.83:0/2021946222 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55faa0280a20 0x55faa0298a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:13.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.345+0000 7ff5181d8740 1 --2- 172.21.15.83:0/2021946222 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55faa0276350 0x55faa0277890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:13.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.345+0000 7ff5181d8740 1 --2- 172.21.15.83:0/2021946222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55faa0275a30 0x55faa028c670 unknown :-1 s=CLOSED pgs=2076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:13.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.345+0000 7ff5181d8740 1 -- 172.21.15.83:0/2021946222 >> 172.21.15.83:0/2021946222 conn(0x55faa0271d50 msgr2=0x55faa0277180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:13.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.345+0000 7ff5181d8740 1 -- 172.21.15.83:0/2021946222 shutdown_connections 2024-04-24T09:05:13.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.345+0000 7ff5181d8740 1 -- 172.21.15.83:0/2021946222 wait complete. 2024-04-24T09:05:13.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.345+0000 7ff5181d8740 1 -- start start 2024-04-24T09:05:13.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.345+0000 7ff5181d8740 1 Processor -- start 2024-04-24T09:05:13.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.346+0000 7ff5181d8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55faa0275a30 0x55faa0288e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:13.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.346+0000 7ff5181d8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55faa0276350 0x55faa027a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:13.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.346+0000 7ff5181d8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55faa0280a20 0x55faa027a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:13.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.346+0000 7ff5181d8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff5101b2bb0 con 0x55faa0280a20 2024-04-24T09:05:13.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.346+0000 7ff5181d8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff5101b2480 con 0x55faa0276350 2024-04-24T09:05:13.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.346+0000 7ff5181d8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff5101af6e0 con 0x55faa0275a30 2024-04-24T09:05:13.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.346+0000 7ff5181d8740 10 client.0 _ll_register_callbacks cb 0x7ff50000df40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:05:13.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.346+0000 7ff50f200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55faa0280a20 0x55faa027a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:13.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.346+0000 7ff50f200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55faa0280a20 0x55faa027a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35992/0 (socket says 172.21.15.83:35992) 2024-04-24T09:05:13.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.346+0000 7ff50f200640 1 -- 172.21.15.83:0/2054853385 learned_addr learned my addr 172.21.15.83:0/2054853385 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:05:13.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.347+0000 7ff50de00640 1 --2- 172.21.15.83:0/2054853385 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55faa0276350 0x55faa027a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:13.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.347+0000 7ff50e800640 1 --2- 172.21.15.83:0/2054853385 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55faa0275a30 0x55faa0288e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:13.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.347+0000 7ff50f200640 1 -- 172.21.15.83:0/2054853385 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55faa0275a30 msgr2=0x55faa0288e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:13.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.347+0000 7ff50f200640 1 --2- 172.21.15.83:0/2054853385 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55faa0275a30 0x55faa0288e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:13.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.347+0000 7ff50f200640 1 -- 172.21.15.83:0/2054853385 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55faa0276350 msgr2=0x55faa027a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:13.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.347+0000 7ff50f200640 1 --2- 172.21.15.83:0/2054853385 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55faa0276350 0x55faa027a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:13.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.347+0000 7ff50f200640 1 -- 172.21.15.83:0/2054853385 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa9fead0f0 con 0x55faa0280a20 2024-04-24T09:05:13.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.347+0000 7ff50f200640 1 --2- 172.21.15.83:0/2054853385 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55faa0280a20 0x55faa027a7b0 secure :-1 s=READY pgs=2077 cs=0 l=1 rev1=1 crypto rx=0x7ff50801dd90 tx=0x7ff508004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=633f2ac683279eed server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:13.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.347+0000 7ff4efe00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:05:13.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.347+0000 7ff4efe00640 1 -- 172.21.15.83:0/2054853385 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff50801de80 con 0x55faa0280a20 2024-04-24T09:05:13.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.348+0000 7ff4efe00640 1 -- 172.21.15.83:0/2054853385 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7ff50801e400 con 0x55faa0280a20 2024-04-24T09:05:13.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.348+0000 7ff5181d8740 1 -- 172.21.15.83:0/2054853385 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5000b0e50 con 0x55faa0280a20 2024-04-24T09:05:13.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.348+0000 7ff4efe00640 1 -- 172.21.15.83:0/2054853385 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7ff50800ee80 con 0x55faa0280a20 2024-04-24T09:05:13.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.348+0000 7ff5181d8740 10 client.20394 fetch_fsmap learned FSMap version 43 2024-04-24T09:05:13.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.348+0000 7ff5181d8740 1 -- 172.21.15.83:0/2054853385 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55faa01a0810 con 0x55faa0280a20 2024-04-24T09:05:13.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.348+0000 7ff4efe00640 1 -- 172.21.15.83:0/2054853385 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff5000b0e50 con 0x55faa0280a20 2024-04-24T09:05:13.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.348+0000 7ff4efe00640 1 -- 172.21.15.83:0/2054853385 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7ff508027790 con 0x55faa0280a20 2024-04-24T09:05:13.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.348+0000 7ff5181d8740 10 client.20394 fetch_fsmap finished waiting for FSMap version 43 2024-04-24T09:05:13.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.348+0000 7ff5181d8740 10 client.20394 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:05:13.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.348+0000 7ff5181d8740 20 client.20394 populate_metadata read hostname 'smithi083' 2024-04-24T09:05:13.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.348+0000 7ff5181d8740 1 --2- 172.21.15.83:0/2054853385 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55faa02f76f0 0x55faa02f9b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:13.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.348+0000 7ff5181d8740 4 client.20394 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T09:05:13.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.348+0000 7ff5181d8740 1 -- 172.21.15.83:0/2054853385 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55faa01e2a80 con 0x55faa02f76f0 2024-04-24T09:05:13.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.349+0000 7ff50e800640 1 --2- 172.21.15.83:0/2054853385 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55faa02f76f0 0x55faa02f9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:13.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.350+0000 7ff50e800640 1 --2- 172.21.15.83:0/2054853385 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55faa02f76f0 0x55faa02f9b10 crc :-1 s=READY pgs=350 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b17f05ad4ebd1d0 server_cookie=926e4334c045ee11 in_seq=0 out_seq=0 2024-04-24T09:05:13.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.350+0000 7ff4efe00640 10 client.20394 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:05:13.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.353+0000 7ff4efe00640 1 -- 172.21.15.83:0/2054853385 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55faa01e2a80 con 0x55faa02f76f0 2024-04-24T09:05:13.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.353+0000 7ff4efe00640 10 client.20394 handle_command_reply: tid=0 2024-04-24T09:05:13.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.353+0000 7ff5181d8740 1 client.20394 shutdown 2024-04-24T09:05:13.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.353+0000 7ff5181d8740 1 -- 172.21.15.83:0/2054853385 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55faa0280a20 msgr2=0x55faa027a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:13.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.353+0000 7ff5181d8740 1 --2- 172.21.15.83:0/2054853385 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55faa0280a20 0x55faa027a7b0 secure :-1 s=READY pgs=2077 cs=0 l=1 rev1=1 crypto rx=0x7ff50801dd90 tx=0x7ff508004ab0 comp rx=0 tx=0).stop 2024-04-24T09:05:13.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.353+0000 7ff5181d8740 1 -- 172.21.15.83:0/2054853385 shutdown_connections 2024-04-24T09:05:13.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.353+0000 7ff5181d8740 1 --2- 172.21.15.83:0/2054853385 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55faa02f76f0 0x55faa02f9b10 crc :-1 s=READY pgs=350 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:13.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.353+0000 7ff5181d8740 1 --2- 172.21.15.83:0/2054853385 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55faa0280a20 0x55faa027a7b0 unknown :-1 s=CLOSED pgs=2077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:13.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.353+0000 7ff5181d8740 1 --2- 172.21.15.83:0/2054853385 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55faa0276350 0x55faa027a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:13.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.353+0000 7ff5181d8740 1 --2- 172.21.15.83:0/2054853385 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55faa0275a30 0x55faa0288e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:13.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.353+0000 7ff5181d8740 1 -- 172.21.15.83:0/2054853385 >> 172.21.15.83:0/2054853385 conn(0x55faa0271d50 msgr2=0x55faa027e110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:13.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.353+0000 7ff4efe00640 0 client.20394 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:05:13.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.353+0000 7ff5181d8740 1 -- 172.21.15.83:0/2054853385 shutdown_connections 2024-04-24T09:05:13.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.353+0000 7ff5181d8740 1 -- 172.21.15.83:0/2054853385 wait complete. 2024-04-24T09:05:13.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.353+0000 7ff5181d8740 20 client.20394 trim_cache size 0 max 16384 2024-04-24T09:05:13.357 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T09:05:13.357 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-04-24T09:05:13.357 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-04-24T09:05:13.357 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T09:05:13.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.354+0000 7ff514a00640 1 -- 172.21.15.83:0/1715601339 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff4f40a3da0 msgr2=0x7ff4f40a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:13.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.354+0000 7ff514a00640 1 --2- 172.21.15.83:0/1715601339 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff4f40a3da0 0x7ff4f40a6260 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7ff508005fd0 tx=0x7ff5080085b0 comp rx=0 tx=0).stop 2024-04-24T09:05:13.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.354+0000 7ff514a00640 1 -- 172.21.15.83:0/1715601339 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff510103100 msgr2=0x7ff510100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:13.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.354+0000 7ff514a00640 1 --2- 172.21.15.83:0/1715601339 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff510103100 0x7ff510100d50 secure :-1 s=READY pgs=2074 cs=0 l=1 rev1=1 crypto rx=0x7ff50000d950 tx=0x7ff50000de10 comp rx=0 tx=0).stop 2024-04-24T09:05:13.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.354+0000 7ff514a00640 1 -- 172.21.15.83:0/1715601339 shutdown_connections 2024-04-24T09:05:13.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.354+0000 7ff514a00640 1 --2- 172.21.15.83:0/1715601339 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff4f40a3da0 0x7ff4f40a6260 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:13.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.354+0000 7ff514a00640 1 --2- 172.21.15.83:0/1715601339 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff51010c4d0 0x7ff5101017d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:13.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.354+0000 7ff514a00640 1 --2- 172.21.15.83:0/1715601339 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff510103a20 0x7ff510101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:13.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.354+0000 7ff514a00640 1 --2- 172.21.15.83:0/1715601339 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff510103100 0x7ff510100d50 unknown :-1 s=CLOSED pgs=2074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:13.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.354+0000 7ff514a00640 1 -- 172.21.15.83:0/1715601339 >> 172.21.15.83:0/1715601339 conn(0x7ff5100fc880 msgr2=0x7ff5100fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:13.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.354+0000 7ff514a00640 1 -- 172.21.15.83:0/1715601339 shutdown_connections 2024-04-24T09:05:13.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.354+0000 7ff514a00640 1 -- 172.21.15.83:0/1715601339 wait complete. 2024-04-24T09:05:13.372 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T09:05:13.372 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T09:05:13.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:05:13 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:05:13.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:05:13 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:05:13.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.562+0000 7f2d93a00640 1 -- 172.21.15.83:0/2531094346 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d8c074430 msgr2=0x7f2d8c074810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:13.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.562+0000 7f2d93a00640 1 --2- 172.21.15.83:0/2531094346 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d8c074430 0x7f2d8c074810 secure :-1 s=READY pgs=2078 cs=0 l=1 rev1=1 crypto rx=0x7f2d80009900 tx=0x7f2d8002a480 comp rx=0 tx=0).stop 2024-04-24T09:05:13.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.562+0000 7f2d93a00640 1 -- 172.21.15.83:0/2531094346 shutdown_connections 2024-04-24T09:05:13.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.562+0000 7f2d93a00640 1 --2- 172.21.15.83:0/2531094346 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2d8c10e820 0x7f2d8c110c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:13.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.562+0000 7f2d93a00640 1 --2- 172.21.15.83:0/2531094346 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d8c074d50 0x7f2d8c10e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:13.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.562+0000 7f2d93a00640 1 --2- 172.21.15.83:0/2531094346 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d8c074430 0x7f2d8c074810 unknown :-1 s=CLOSED pgs=2078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:13.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.562+0000 7f2d93a00640 1 -- 172.21.15.83:0/2531094346 >> 172.21.15.83:0/2531094346 conn(0x7f2d8c0fda90 msgr2=0x7f2d8c0ffeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:13.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.562+0000 7f2d93a00640 1 -- 172.21.15.83:0/2531094346 shutdown_connections 2024-04-24T09:05:13.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.562+0000 7f2d93a00640 1 -- 172.21.15.83:0/2531094346 wait complete. 2024-04-24T09:05:13.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.563+0000 7f2d93a00640 1 Processor -- start 2024-04-24T09:05:13.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.563+0000 7f2d93a00640 1 -- start start 2024-04-24T09:05:13.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.564+0000 7f2d93a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d8c074430 0x7f2d8c1a06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:13.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.564+0000 7f2d93a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2d8c074d50 0x7f2d8c1a0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:13.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.564+0000 7f2d93a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d8c10e820 0x7f2d8c19a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:13.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.564+0000 7f2d93a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2d8c1144b0 con 0x7f2d8c10e820 2024-04-24T09:05:13.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.564+0000 7f2d93a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2d8c114330 con 0x7f2d8c074430 2024-04-24T09:05:13.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.564+0000 7f2d93a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2d8c114630 con 0x7f2d8c074d50 2024-04-24T09:05:13.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.564+0000 7f2d91a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d8c10e820 0x7f2d8c19a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:13.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.564+0000 7f2d91000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d8c074430 0x7f2d8c1a06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:13.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.564+0000 7f2d91a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d8c10e820 0x7f2d8c19a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36012/0 (socket says 172.21.15.83:36012) 2024-04-24T09:05:13.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.564+0000 7f2d91000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d8c074430 0x7f2d8c1a06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:51072/0 (socket says 172.21.15.83:51072) 2024-04-24T09:05:13.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.564+0000 7f2d91a00640 1 -- 172.21.15.83:0/3754001237 learned_addr learned my addr 172.21.15.83:0/3754001237 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:05:13.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.565+0000 7f2d8be00640 1 --2- 172.21.15.83:0/3754001237 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2d8c074d50 0x7f2d8c1a0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:13.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.565+0000 7f2d91a00640 1 -- 172.21.15.83:0/3754001237 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2d8c074d50 msgr2=0x7f2d8c1a0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:13.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.565+0000 7f2d91a00640 1 --2- 172.21.15.83:0/3754001237 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2d8c074d50 0x7f2d8c1a0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:13.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.565+0000 7f2d91a00640 1 -- 172.21.15.83:0/3754001237 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d8c074430 msgr2=0x7f2d8c1a06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:13.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.565+0000 7f2d91a00640 1 --2- 172.21.15.83:0/3754001237 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d8c074430 0x7f2d8c1a06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:13.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.565+0000 7f2d91a00640 1 -- 172.21.15.83:0/3754001237 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d8c19b020 con 0x7f2d8c10e820 2024-04-24T09:05:13.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.565+0000 7f2d91a00640 1 --2- 172.21.15.83:0/3754001237 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d8c10e820 0x7f2d8c19a8a0 secure :-1 s=READY pgs=2079 cs=0 l=1 rev1=1 crypto rx=0x7f2d8400eaa0 tx=0x7f2d8400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:13.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.565+0000 7f2d89600640 1 -- 172.21.15.83:0/3754001237 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d8400cdb0 con 0x7f2d8c10e820 2024-04-24T09:05:13.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.565+0000 7f2d93a00640 1 -- 172.21.15.83:0/3754001237 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d8c19b310 con 0x7f2d8c10e820 2024-04-24T09:05:13.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.565+0000 7f2d89600640 1 -- 172.21.15.83:0/3754001237 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2d84005020 con 0x7f2d8c10e820 2024-04-24T09:05:13.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.565+0000 7f2d89600640 1 -- 172.21.15.83:0/3754001237 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d840116d0 con 0x7f2d8c10e820 2024-04-24T09:05:13.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.566+0000 7f2d93a00640 1 -- 172.21.15.83:0/3754001237 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d8c1a75b0 con 0x7f2d8c10e820 2024-04-24T09:05:13.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.567+0000 7f2d93a00640 1 -- 172.21.15.83:0/3754001237 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d8c068e20 con 0x7f2d8c10e820 2024-04-24T09:05:13.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.569+0000 7f2d89600640 1 -- 172.21.15.83:0/3754001237 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2d840051d0 con 0x7f2d8c10e820 2024-04-24T09:05:13.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.571+0000 7f2d89600640 1 --2- 172.21.15.83:0/3754001237 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f2d740a3d20 0x7f2d740a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:13.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.571+0000 7f2d89600640 1 -- 172.21.15.83:0/3754001237 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7f2d840d0db0 con 0x7f2d8c10e820 2024-04-24T09:05:13.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.571+0000 7f2d91000640 1 --2- 172.21.15.83:0/3754001237 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f2d740a3d20 0x7f2d740a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:13.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.572+0000 7f2d91000640 1 --2- 172.21.15.83:0/3754001237 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f2d740a3d20 0x7f2d740a61e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f2d80005ab0 tx=0x7f2d80005a20 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:13.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.576+0000 7f2d89600640 1 -- 172.21.15.83:0/3754001237 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2d840d5070 con 0x7f2d8c10e820 2024-04-24T09:05:13.631 INFO:tasks.workunit.client.0.smithi083.stdout: 1 78214 0.58 MB/sec execute 587 sec latency 4184.857 ms 2024-04-24T09:05:13.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.844+0000 7f2d93a00640 1 -- 172.21.15.83:0/3754001237 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2d8c02d620 con 0x7f2d8c10e820 2024-04-24T09:05:13.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.845+0000 7f2d89600640 1 -- 172.21.15.83:0/3754001237 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+11950 (secure 0 0 0) 0x7f2d84016030 con 0x7f2d8c10e820 2024-04-24T09:05:13.845 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T09:05:13.845 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":43,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/358872257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":358872257},{"type":"v1","addr":"172.21.15.83:6835","nonce":358872257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3157070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3157070851},{"type":"v1","addr":"172.21.15.83:6837","nonce":3157070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":43,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T08:48:29.422576+0000","modified":"2024-04-24T09:05:03.083941+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34520,"mds_1":34493,"mds_2":34451,"mds_3":34427,"mds_4":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14658":{"gid":14658,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6839/4085470844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":4085470844},{"type":"v1","addr":"172.21.15.83:6839","nonce":4085470844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14676":{"gid":14676,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6841/2634754210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":2634754210},{"type":"v1","addr":"172.21.15.83:6841","nonce":2634754210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24382":{"gid":24382,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/1859921310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1859921310},{"type":"v1","addr":"172.21.15.189:6833","nonce":1859921310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/2241923197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2241923197},{"type":"v1","addr":"172.21.15.189:6835","nonce":2241923197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.189:6839/2548965336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":2548965336},{"type":"v1","addr":"172.21.15.189:6839","nonce":2548965336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26686":{"gid":26686,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6837/1678817161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":1678817161},{"type":"v1","addr":"172.21.15.189:6837","nonce":1678817161}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.138:6833/1157967658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1157967658},{"type":"v1","addr":"172.21.15.138:6833","nonce":1157967658}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.138:6835/1909219758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1909219758},{"type":"v1","addr":"172.21.15.138:6835","nonce":1909219758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34493":{"gid":34493,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.138:6837/900566754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":900566754},{"type":"v1","addr":"172.21.15.138:6837","nonce":900566754}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34520":{"gid":34520,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.138:6839/2236932393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2236932393},{"type":"v1","addr":"172.21.15.138:6839","nonce":2236932393}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34520,"qdb_cluster":[34520,24454,34427,34493,34451]},"id":1}]} 2024-04-24T09:05:13.845 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 43 2024-04-24T09:05:13.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.848+0000 7f2d93a00640 1 -- 172.21.15.83:0/3754001237 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f2d740a3d20 msgr2=0x7f2d740a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:13.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.848+0000 7f2d93a00640 1 --2- 172.21.15.83:0/3754001237 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f2d740a3d20 0x7f2d740a61e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f2d80005ab0 tx=0x7f2d80005a20 comp rx=0 tx=0).stop 2024-04-24T09:05:13.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.848+0000 7f2d93a00640 1 -- 172.21.15.83:0/3754001237 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d8c10e820 msgr2=0x7f2d8c19a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:13.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.848+0000 7f2d93a00640 1 --2- 172.21.15.83:0/3754001237 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d8c10e820 0x7f2d8c19a8a0 secure :-1 s=READY pgs=2079 cs=0 l=1 rev1=1 crypto rx=0x7f2d8400eaa0 tx=0x7f2d8400ef60 comp rx=0 tx=0).stop 2024-04-24T09:05:13.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.848+0000 7f2d93a00640 1 -- 172.21.15.83:0/3754001237 shutdown_connections 2024-04-24T09:05:13.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.848+0000 7f2d93a00640 1 --2- 172.21.15.83:0/3754001237 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f2d740a3d20 0x7f2d740a61e0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:13.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.848+0000 7f2d93a00640 1 --2- 172.21.15.83:0/3754001237 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d8c10e820 0x7f2d8c19a8a0 unknown :-1 s=CLOSED pgs=2079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:13.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.848+0000 7f2d93a00640 1 --2- 172.21.15.83:0/3754001237 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2d8c074d50 0x7f2d8c1a0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:13.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.848+0000 7f2d93a00640 1 --2- 172.21.15.83:0/3754001237 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d8c074430 0x7f2d8c1a06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:13.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.848+0000 7f2d93a00640 1 -- 172.21.15.83:0/3754001237 >> 172.21.15.83:0/3754001237 conn(0x7f2d8c0fda90 msgr2=0x7f2d8c0ff320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:13.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.848+0000 7f2d93a00640 1 -- 172.21.15.83:0/3754001237 shutdown_connections 2024-04-24T09:05:13.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:13.848+0000 7f2d93a00640 1 -- 172.21.15.83:0/3754001237 wait complete. 2024-04-24T09:05:13.865 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-24T09:05:14.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.068+0000 7f093ee00640 1 -- 172.21.15.83:0/4114352436 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0938102ad0 msgr2=0x7f0938106b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:14.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.068+0000 7f093ee00640 1 --2- 172.21.15.83:0/4114352436 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0938102ad0 0x7f0938106b10 secure :-1 s=READY pgs=2080 cs=0 l=1 rev1=1 crypto rx=0x7f0924009900 tx=0x7f092402a410 comp rx=0 tx=0).stop 2024-04-24T09:05:14.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.069+0000 7f093ee00640 1 -- 172.21.15.83:0/4114352436 shutdown_connections 2024-04-24T09:05:14.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.069+0000 7f093ee00640 1 --2- 172.21.15.83:0/4114352436 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f09381071f0 0x7f093810edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.069+0000 7f093ee00640 1 --2- 172.21.15.83:0/4114352436 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0938102ad0 0x7f0938106b10 unknown :-1 s=CLOSED pgs=2080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.069+0000 7f093ee00640 1 --2- 172.21.15.83:0/4114352436 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0938102120 0x7f0938102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.069+0000 7f093ee00640 1 -- 172.21.15.83:0/4114352436 >> 172.21.15.83:0/4114352436 conn(0x7f09380fd9a0 msgr2=0x7f09380ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:14.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.069+0000 7f093ee00640 1 -- 172.21.15.83:0/4114352436 shutdown_connections 2024-04-24T09:05:14.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.069+0000 7f093ee00640 1 -- 172.21.15.83:0/4114352436 wait complete. 2024-04-24T09:05:14.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.070+0000 7f093ee00640 1 Processor -- start 2024-04-24T09:05:14.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.070+0000 7f093ee00640 1 -- start start 2024-04-24T09:05:14.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.070+0000 7f093ee00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0938102120 0x7f0938198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:14.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.070+0000 7f093ee00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f09381071f0 0x7f0938198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:14.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.070+0000 7f093ee00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f093819cfb0 0x7f093819d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:14.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.070+0000 7f093ee00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0938112160 con 0x7f09381071f0 2024-04-24T09:05:14.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.070+0000 7f093ee00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0938111fe0 con 0x7f093819cfb0 2024-04-24T09:05:14.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.070+0000 7f093ee00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f09381122e0 con 0x7f0938102120 2024-04-24T09:05:14.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.070+0000 7f093d000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f09381071f0 0x7f0938198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:14.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.071+0000 7f093d000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f09381071f0 0x7f0938198c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36026/0 (socket says 172.21.15.83:36026) 2024-04-24T09:05:14.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.071+0000 7f093d000640 1 -- 172.21.15.83:0/1585836307 learned_addr learned my addr 172.21.15.83:0/1585836307 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:05:14.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.071+0000 7f093da00640 1 --2- 172.21.15.83:0/1585836307 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0938102120 0x7f0938198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:14.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.071+0000 7f093e400640 1 --2- 172.21.15.83:0/1585836307 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f093819cfb0 0x7f093819d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:14.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.071+0000 7f093d000640 1 -- 172.21.15.83:0/1585836307 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0938102120 msgr2=0x7f0938198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:14.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.071+0000 7f093d000640 1 --2- 172.21.15.83:0/1585836307 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0938102120 0x7f0938198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.071+0000 7f093d000640 1 -- 172.21.15.83:0/1585836307 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f093819cfb0 msgr2=0x7f093819d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:14.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.071+0000 7f093d000640 1 --2- 172.21.15.83:0/1585836307 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f093819cfb0 0x7f093819d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.071+0000 7f093d000640 1 -- 172.21.15.83:0/1585836307 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f093819daa0 con 0x7f09381071f0 2024-04-24T09:05:14.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.071+0000 7f093e400640 1 --2- 172.21.15.83:0/1585836307 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f093819cfb0 0x7f093819d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:05:14.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.071+0000 7f093d000640 1 --2- 172.21.15.83:0/1585836307 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f09381071f0 0x7f0938198c80 secure :-1 s=READY pgs=2081 cs=0 l=1 rev1=1 crypto rx=0x7f0924005ab0 tx=0x7f0924003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:14.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.072+0000 7f0936000640 1 -- 172.21.15.83:0/1585836307 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f092403f070 con 0x7f09381071f0 2024-04-24T09:05:14.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.072+0000 7f0936000640 1 -- 172.21.15.83:0/1585836307 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0924031d60 con 0x7f09381071f0 2024-04-24T09:05:14.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.072+0000 7f0936000640 1 -- 172.21.15.83:0/1585836307 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0924035030 con 0x7f09381071f0 2024-04-24T09:05:14.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.072+0000 7f093ee00640 1 -- 172.21.15.83:0/1585836307 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f093819dd30 con 0x7f09381071f0 2024-04-24T09:05:14.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.072+0000 7f093ee00640 1 -- 172.21.15.83:0/1585836307 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09381a54a0 con 0x7f09381071f0 2024-04-24T09:05:14.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.075+0000 7f09431a1740 1 Processor -- start 2024-04-24T09:05:14.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.075+0000 7f0936000640 1 -- 172.21.15.83:0/1585836307 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0924042070 con 0x7f09381071f0 2024-04-24T09:05:14.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.075+0000 7f09431a1740 1 -- start start 2024-04-24T09:05:14.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.076+0000 7f09431a1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d6d1b96b40 0x55d6d1b96f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:14.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.076+0000 7f09431a1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6d1b97460 0x55d6d1ba15f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:14.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.076+0000 7f09431a1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d6d1ba1b30 0x55d6d1ba3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:14.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.076+0000 7f09431a1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d6d1afc1d0 con 0x55d6d1b97460 2024-04-24T09:05:14.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.076+0000 7f09431a1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d6d1aa2450 con 0x55d6d1ba1b30 2024-04-24T09:05:14.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.076+0000 7f09431a1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d6d1af8190 con 0x55d6d1b96b40 2024-04-24T09:05:14.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.076+0000 7f093da00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6d1b97460 0x55d6d1ba15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:14.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.076+0000 7f093da00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6d1b97460 0x55d6d1ba15f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36030/0 (socket says 172.21.15.83:36030) 2024-04-24T09:05:14.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.076+0000 7f093da00640 1 -- 172.21.15.83:0/3657100356 learned_addr learned my addr 172.21.15.83:0/3657100356 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:05:14.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.076+0000 7f093d000640 1 --2- 172.21.15.83:0/3657100356 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d6d1ba1b30 0x55d6d1ba3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:14.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.076+0000 7f093e400640 1 --2- 172.21.15.83:0/3657100356 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d6d1b96b40 0x55d6d1b96f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:14.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.077+0000 7f093da00640 1 -- 172.21.15.83:0/3657100356 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d6d1b96b40 msgr2=0x55d6d1b96f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:14.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.077+0000 7f093da00640 1 --2- 172.21.15.83:0/3657100356 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d6d1b96b40 0x55d6d1b96f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.077+0000 7f093da00640 1 -- 172.21.15.83:0/3657100356 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d6d1ba1b30 msgr2=0x55d6d1ba3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:14.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.077+0000 7f093da00640 1 --2- 172.21.15.83:0/3657100356 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d6d1ba1b30 0x55d6d1ba3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.077+0000 7f093da00640 1 -- 172.21.15.83:0/3657100356 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6d1b5f310 con 0x55d6d1b97460 2024-04-24T09:05:14.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.077+0000 7f093d000640 1 --2- 172.21.15.83:0/3657100356 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d6d1ba1b30 0x55d6d1ba3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:05:14.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.077+0000 7f093da00640 1 --2- 172.21.15.83:0/3657100356 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6d1b97460 0x55d6d1ba15f0 secure :-1 s=READY pgs=2082 cs=0 l=1 rev1=1 crypto rx=0x7f092c00dcb0 tx=0x7f092c00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c61ccffb82c52ce7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:14.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.077+0000 7f0936000640 1 --2- 172.21.15.83:0/1585836307 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f09200a3d20 0x7f09200a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:14.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.077+0000 7f092aa00640 1 -- 172.21.15.83:0/3657100356 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f092c011440 con 0x55d6d1b97460 2024-04-24T09:05:14.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.077+0000 7f092aa00640 1 -- 172.21.15.83:0/3657100356 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f092c011a90 con 0x55d6d1b97460 2024-04-24T09:05:14.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.077+0000 7f092aa00640 1 -- 172.21.15.83:0/3657100356 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f092c01b9d0 con 0x55d6d1b97460 2024-04-24T09:05:14.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.077+0000 7f0936000640 1 -- 172.21.15.83:0/1585836307 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7f09240f1550 con 0x7f09381071f0 2024-04-24T09:05:14.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.077+0000 7f093e400640 1 --2- 172.21.15.83:0/1585836307 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f09200a3d20 0x7f09200a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:14.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.078+0000 7f09431a1740 1 -- 172.21.15.83:0/3657100356 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6d1b97460 msgr2=0x55d6d1ba15f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:14.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.078+0000 7f09431a1740 1 --2- 172.21.15.83:0/3657100356 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6d1b97460 0x55d6d1ba15f0 secure :-1 s=READY pgs=2082 cs=0 l=1 rev1=1 crypto rx=0x7f092c00dcb0 tx=0x7f092c00f730 comp rx=0 tx=0).stop 2024-04-24T09:05:14.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.078+0000 7f093e400640 1 --2- 172.21.15.83:0/1585836307 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f09200a3d20 0x7f09200a61e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f0930005fd0 tx=0x7f0930009530 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:14.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.078+0000 7f09431a1740 1 -- 172.21.15.83:0/3657100356 shutdown_connections 2024-04-24T09:05:14.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.078+0000 7f09431a1740 1 --2- 172.21.15.83:0/3657100356 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d6d1ba1b30 0x55d6d1ba3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.078+0000 7f09431a1740 1 --2- 172.21.15.83:0/3657100356 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6d1b97460 0x55d6d1ba15f0 unknown :-1 s=CLOSED pgs=2082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.078+0000 7f09431a1740 1 --2- 172.21.15.83:0/3657100356 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d6d1b96b40 0x55d6d1b96f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.078+0000 7f09431a1740 1 -- 172.21.15.83:0/3657100356 >> 172.21.15.83:0/3657100356 conn(0x55d6d1b92e60 msgr2=0x55d6d1b93240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:14.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.078+0000 7f09431a1740 1 -- 172.21.15.83:0/3657100356 shutdown_connections 2024-04-24T09:05:14.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.078+0000 7f09431a1740 1 -- 172.21.15.83:0/3657100356 wait complete. 2024-04-24T09:05:14.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.079+0000 7f09431a1740 1 -- start start 2024-04-24T09:05:14.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.079+0000 7f09431a1740 1 Processor -- start 2024-04-24T09:05:14.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.079+0000 7f09431a1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d6d1b96b40 0x55d6d1b9e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:14.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.079+0000 7f09431a1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d6d1b97460 0x55d6d1b9ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:14.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.079+0000 7f09431a1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6d1ba1b30 0x55d6d1b9f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:14.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.079+0000 7f09431a1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d6d1ba49c0 con 0x55d6d1ba1b30 2024-04-24T09:05:14.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.079+0000 7f09431a1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d6d1ab5810 con 0x55d6d1b96b40 2024-04-24T09:05:14.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.079+0000 7f09431a1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d6d1ba4b40 con 0x55d6d1b97460 2024-04-24T09:05:14.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.079+0000 7f09431a1740 10 client.0 _ll_register_callbacks cb 0x55d6d1a83140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:05:14.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.080+0000 7f093e400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6d1ba1b30 0x55d6d1b9f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:14.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.080+0000 7f093e400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6d1ba1b30 0x55d6d1b9f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36032/0 (socket says 172.21.15.83:36032) 2024-04-24T09:05:14.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.080+0000 7f093e400640 1 -- 172.21.15.83:0/1514546266 learned_addr learned my addr 172.21.15.83:0/1514546266 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:05:14.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.080+0000 7f093da00640 1 --2- 172.21.15.83:0/1514546266 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d6d1b96b40 0x55d6d1b9e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:14.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.080+0000 7f093e400640 1 -- 172.21.15.83:0/1514546266 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d6d1b97460 msgr2=0x55d6d1b9ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:14.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.080+0000 7f093e400640 1 --2- 172.21.15.83:0/1514546266 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d6d1b97460 0x55d6d1b9ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.080+0000 7f093e400640 1 -- 172.21.15.83:0/1514546266 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d6d1b96b40 msgr2=0x55d6d1b9e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:14.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.080+0000 7f093e400640 1 --2- 172.21.15.83:0/1514546266 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d6d1b96b40 0x55d6d1b9e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.080+0000 7f093e400640 1 -- 172.21.15.83:0/1514546266 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6d1ac63f0 con 0x55d6d1ba1b30 2024-04-24T09:05:14.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.081+0000 7f093e400640 1 --2- 172.21.15.83:0/1514546266 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6d1ba1b30 0x55d6d1b9f380 secure :-1 s=READY pgs=2083 cs=0 l=1 rev1=1 crypto rx=0x7f0930012930 tx=0x7f0930004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5e37f3f7de30e9e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:14.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.081+0000 7f091f400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:05:14.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.081+0000 7f091f400640 1 -- 172.21.15.83:0/1514546266 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0930022070 con 0x55d6d1ba1b30 2024-04-24T09:05:14.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.081+0000 7f091f400640 1 -- 172.21.15.83:0/1514546266 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7f093000fd60 con 0x55d6d1ba1b30 2024-04-24T09:05:14.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.081+0000 7f09431a1740 1 -- 172.21.15.83:0/1514546266 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f092c011a90 con 0x55d6d1ba1b30 2024-04-24T09:05:14.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.081+0000 7f091f400640 1 -- 172.21.15.83:0/1514546266 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f0930027460 con 0x55d6d1ba1b30 2024-04-24T09:05:14.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.081+0000 7f09431a1740 10 client.20412 fetch_fsmap learned FSMap version 43 2024-04-24T09:05:14.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.081+0000 7f09431a1740 1 -- 172.21.15.83:0/1514546266 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55d6d180e3f0 con 0x55d6d1ba1b30 2024-04-24T09:05:14.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.082+0000 7f091f400640 1 -- 172.21.15.83:0/1514546266 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f092c011a90 con 0x55d6d1ba1b30 2024-04-24T09:05:14.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.082+0000 7f091f400640 1 -- 172.21.15.83:0/1514546266 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7f093001e530 con 0x55d6d1ba1b30 2024-04-24T09:05:14.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.082+0000 7f09431a1740 10 client.20412 fetch_fsmap finished waiting for FSMap version 43 2024-04-24T09:05:14.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.082+0000 7f09431a1740 10 client.20412 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-24T09:05:14.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.082+0000 7f09431a1740 20 client.20412 populate_metadata read hostname 'smithi083' 2024-04-24T09:05:14.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.082+0000 7f09431a1740 1 --2- 172.21.15.83:0/1514546266 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55d6d1bb1e30 0x55d6d1bb4220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:14.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.082+0000 7f09431a1740 4 client.20412 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:05:14.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.082+0000 7f09431a1740 1 -- 172.21.15.83:0/1514546266 --> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d6d1b09bf0 con 0x55d6d1bb1e30 2024-04-24T09:05:14.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.083+0000 7f093da00640 1 --2- 172.21.15.83:0/1514546266 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55d6d1bb1e30 0x55d6d1bb4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:14.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.084+0000 7f093da00640 1 --2- 172.21.15.83:0/1514546266 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55d6d1bb1e30 0x55d6d1bb4220 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3a0b25c47bab39 server_cookie=913ffc29ba8a8107 in_seq=0 out_seq=0 2024-04-24T09:05:14.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.084+0000 7f091f400640 10 client.20412 ms_handle_connect on v2:172.21.15.189:6838/2548965336 2024-04-24T09:05:14.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.086+0000 7f091f400640 1 -- 172.21.15.83:0/1514546266 <== mds.4 v2:172.21.15.189:6838/2548965336 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d6d1b09bf0 con 0x55d6d1bb1e30 2024-04-24T09:05:14.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.086+0000 7f091f400640 10 client.20412 handle_command_reply: tid=0 2024-04-24T09:05:14.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.086+0000 7f09431a1740 1 client.20412 shutdown 2024-04-24T09:05:14.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.086+0000 7f09431a1740 1 -- 172.21.15.83:0/1514546266 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6d1ba1b30 msgr2=0x55d6d1b9f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:14.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.086+0000 7f09431a1740 1 --2- 172.21.15.83:0/1514546266 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6d1ba1b30 0x55d6d1b9f380 secure :-1 s=READY pgs=2083 cs=0 l=1 rev1=1 crypto rx=0x7f0930012930 tx=0x7f0930004a40 comp rx=0 tx=0).stop 2024-04-24T09:05:14.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.086+0000 7f09431a1740 1 -- 172.21.15.83:0/1514546266 shutdown_connections 2024-04-24T09:05:14.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.086+0000 7f09431a1740 1 --2- 172.21.15.83:0/1514546266 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55d6d1bb1e30 0x55d6d1bb4220 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.086+0000 7f09431a1740 1 --2- 172.21.15.83:0/1514546266 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6d1ba1b30 0x55d6d1b9f380 unknown :-1 s=CLOSED pgs=2083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.086+0000 7f09431a1740 1 --2- 172.21.15.83:0/1514546266 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d6d1b97460 0x55d6d1b9ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.086+0000 7f091f400640 0 client.20412 ms_handle_reset on v2:172.21.15.189:6838/2548965336 2024-04-24T09:05:14.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.086+0000 7f09431a1740 1 --2- 172.21.15.83:0/1514546266 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d6d1b96b40 0x55d6d1b9e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.086+0000 7f09431a1740 1 -- 172.21.15.83:0/1514546266 >> 172.21.15.83:0/1514546266 conn(0x55d6d1b92e60 msgr2=0x55d6d1b98b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:14.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.086+0000 7f09431a1740 1 -- 172.21.15.83:0/1514546266 shutdown_connections 2024-04-24T09:05:14.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.087+0000 7f09431a1740 1 -- 172.21.15.83:0/1514546266 wait complete. 2024-04-24T09:05:14.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.087+0000 7f09431a1740 20 client.20412 trim_cache size 0 max 16384 2024-04-24T09:05:14.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.101+0000 7f09431a1740 1 Processor -- start 2024-04-24T09:05:14.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.101+0000 7f09431a1740 1 -- start start 2024-04-24T09:05:14.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.102+0000 7f09431a1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d6d1b96b40 0x55d6d1b992e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:14.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.102+0000 7f09431a1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d6d1b97460 0x55d6d1bad8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:14.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.102+0000 7f09431a1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6d1ba1b30 0x55d6d1bb9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:14.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.102+0000 7f09431a1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f09381af4e0 con 0x55d6d1ba1b30 2024-04-24T09:05:14.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.102+0000 7f09431a1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f09381b10e0 con 0x55d6d1b96b40 2024-04-24T09:05:14.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.102+0000 7f09431a1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f09381afbb0 con 0x55d6d1b97460 2024-04-24T09:05:14.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.102+0000 7f093e400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6d1ba1b30 0x55d6d1bb9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:14.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.102+0000 7f093e400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6d1ba1b30 0x55d6d1bb9aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36048/0 (socket says 172.21.15.83:36048) 2024-04-24T09:05:14.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.102+0000 7f093e400640 1 -- 172.21.15.83:0/1646223529 learned_addr learned my addr 172.21.15.83:0/1646223529 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:05:14.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.102+0000 7f093da00640 1 --2- 172.21.15.83:0/1646223529 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d6d1b96b40 0x55d6d1b992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:14.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.102+0000 7f093d000640 1 --2- 172.21.15.83:0/1646223529 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d6d1b97460 0x55d6d1bad8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:14.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.102+0000 7f093e400640 1 -- 172.21.15.83:0/1646223529 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d6d1b97460 msgr2=0x55d6d1bad8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:14.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.102+0000 7f093e400640 1 --2- 172.21.15.83:0/1646223529 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d6d1b97460 0x55d6d1bad8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.102+0000 7f093e400640 1 -- 172.21.15.83:0/1646223529 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d6d1b96b40 msgr2=0x55d6d1b992e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:14.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.102+0000 7f093e400640 1 --2- 172.21.15.83:0/1646223529 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d6d1b96b40 0x55d6d1b992e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.103+0000 7f093e400640 1 -- 172.21.15.83:0/1646223529 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6d1b97290 con 0x55d6d1ba1b30 2024-04-24T09:05:14.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.103+0000 7f093d000640 1 --2- 172.21.15.83:0/1646223529 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d6d1b97460 0x55d6d1bad8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:05:14.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.103+0000 7f093e400640 1 --2- 172.21.15.83:0/1646223529 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6d1ba1b30 0x55d6d1bb9aa0 secure :-1 s=READY pgs=2084 cs=0 l=1 rev1=1 crypto rx=0x7f0930000f80 tx=0x7f0930012f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=370d752e4cdb66b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:14.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.103+0000 7f092a000640 1 -- 172.21.15.83:0/1646223529 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0930022040 con 0x55d6d1ba1b30 2024-04-24T09:05:14.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.103+0000 7f092a000640 1 -- 172.21.15.83:0/1646223529 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f093001ebc0 con 0x55d6d1ba1b30 2024-04-24T09:05:14.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.103+0000 7f092a000640 1 -- 172.21.15.83:0/1646223529 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09300279c0 con 0x55d6d1ba1b30 2024-04-24T09:05:14.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.104+0000 7f09431a1740 1 -- 172.21.15.83:0/1646223529 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6d1ba1b30 msgr2=0x55d6d1bb9aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:14.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.104+0000 7f09431a1740 1 --2- 172.21.15.83:0/1646223529 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6d1ba1b30 0x55d6d1bb9aa0 secure :-1 s=READY pgs=2084 cs=0 l=1 rev1=1 crypto rx=0x7f0930000f80 tx=0x7f0930012f80 comp rx=0 tx=0).stop 2024-04-24T09:05:14.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.104+0000 7f09431a1740 1 -- 172.21.15.83:0/1646223529 shutdown_connections 2024-04-24T09:05:14.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.104+0000 7f09431a1740 1 --2- 172.21.15.83:0/1646223529 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6d1ba1b30 0x55d6d1bb9aa0 unknown :-1 s=CLOSED pgs=2084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.104+0000 7f09431a1740 1 --2- 172.21.15.83:0/1646223529 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d6d1b97460 0x55d6d1bad8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.104+0000 7f09431a1740 1 --2- 172.21.15.83:0/1646223529 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d6d1b96b40 0x55d6d1b992e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.104+0000 7f09431a1740 1 -- 172.21.15.83:0/1646223529 >> 172.21.15.83:0/1646223529 conn(0x55d6d1b92e60 msgr2=0x55d6d1b91f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:14.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.104+0000 7f09431a1740 1 -- 172.21.15.83:0/1646223529 shutdown_connections 2024-04-24T09:05:14.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.104+0000 7f09431a1740 1 -- 172.21.15.83:0/1646223529 wait complete. 2024-04-24T09:05:14.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.104+0000 7f09431a1740 1 -- start start 2024-04-24T09:05:14.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.105+0000 7f09431a1740 1 Processor -- start 2024-04-24T09:05:14.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.105+0000 7f09431a1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6d1b96b40 0x55d6d1baa1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:14.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.105+0000 7f09431a1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d6d1b97460 0x55d6d1b9bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:14.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.105+0000 7f09431a1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d6d1ba1b30 0x55d6d1b9c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:14.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.105+0000 7f09431a1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f09381b09b0 con 0x55d6d1b96b40 2024-04-24T09:05:14.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.105+0000 7f09431a1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f09381b0280 con 0x55d6d1b97460 2024-04-24T09:05:14.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.105+0000 7f09431a1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f09381ad4e0 con 0x55d6d1ba1b30 2024-04-24T09:05:14.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.105+0000 7f09431a1740 10 client.0 _ll_register_callbacks cb 0x55d6d1ba1a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:05:14.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.105+0000 7f093da00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6d1b96b40 0x55d6d1baa1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:14.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.106+0000 7f093da00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6d1b96b40 0x55d6d1baa1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36050/0 (socket says 172.21.15.83:36050) 2024-04-24T09:05:14.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.106+0000 7f093da00640 1 -- 172.21.15.83:0/3856343031 learned_addr learned my addr 172.21.15.83:0/3856343031 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:05:14.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.106+0000 7f093e400640 1 --2- 172.21.15.83:0/3856343031 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d6d1ba1b30 0x55d6d1b9c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:14.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.106+0000 7f093da00640 1 -- 172.21.15.83:0/3856343031 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d6d1ba1b30 msgr2=0x55d6d1b9c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:14.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.106+0000 7f093d000640 1 --2- 172.21.15.83:0/3856343031 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d6d1b97460 0x55d6d1b9bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:14.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.106+0000 7f093da00640 1 --2- 172.21.15.83:0/3856343031 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d6d1ba1b30 0x55d6d1b9c0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.106+0000 7f093da00640 1 -- 172.21.15.83:0/3856343031 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d6d1b97460 msgr2=0x55d6d1b9bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:14.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.106+0000 7f093da00640 1 --2- 172.21.15.83:0/3856343031 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d6d1b97460 0x55d6d1b9bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.106+0000 7f093da00640 1 -- 172.21.15.83:0/3856343031 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6d18d87d0 con 0x55d6d1b96b40 2024-04-24T09:05:14.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.106+0000 7f093da00640 1 --2- 172.21.15.83:0/3856343031 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6d1b96b40 0x55d6d1baa1b0 secure :-1 s=READY pgs=2085 cs=0 l=1 rev1=1 crypto rx=0x7f092c00dbd0 tx=0x7f092c00d7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45c7c739e6b3514b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:14.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.106+0000 7f091f400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:05:14.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.106+0000 7f091f400640 1 -- 172.21.15.83:0/3856343031 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f092c00de60 con 0x55d6d1b96b40 2024-04-24T09:05:14.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.107+0000 7f091f400640 1 -- 172.21.15.83:0/3856343031 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7f092c004250 con 0x55d6d1b96b40 2024-04-24T09:05:14.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.107+0000 7f09431a1740 1 -- 172.21.15.83:0/3856343031 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f093001ebc0 con 0x55d6d1b96b40 2024-04-24T09:05:14.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.107+0000 7f091f400640 1 -- 172.21.15.83:0/3856343031 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f092c037050 con 0x55d6d1b96b40 2024-04-24T09:05:14.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.107+0000 7f09431a1740 10 client.20418 fetch_fsmap learned FSMap version 43 2024-04-24T09:05:14.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.107+0000 7f09431a1740 1 -- 172.21.15.83:0/3856343031 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55d6d1ba3410 con 0x55d6d1b96b40 2024-04-24T09:05:14.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.107+0000 7f091f400640 1 -- 172.21.15.83:0/3856343031 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f093001ebc0 con 0x55d6d1b96b40 2024-04-24T09:05:14.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.107+0000 7f091f400640 1 -- 172.21.15.83:0/3856343031 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7f092c0048e0 con 0x55d6d1b96b40 2024-04-24T09:05:14.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.107+0000 7f09431a1740 10 client.20418 fetch_fsmap finished waiting for FSMap version 43 2024-04-24T09:05:14.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.107+0000 7f09431a1740 10 client.20418 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-24T09:05:14.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.107+0000 7f09431a1740 20 client.20418 populate_metadata read hostname 'smithi083' 2024-04-24T09:05:14.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.107+0000 7f09431a1740 1 --2- 172.21.15.83:0/3856343031 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55d6d1c185e0 0x55d6d1c1aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:14.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.107+0000 7f09431a1740 4 client.20418 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:05:14.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.107+0000 7f09431a1740 1 -- 172.21.15.83:0/3856343031 --> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d6d1b03a60 con 0x55d6d1c185e0 2024-04-24T09:05:14.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.108+0000 7f093d000640 1 --2- 172.21.15.83:0/3856343031 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55d6d1c185e0 0x55d6d1c1aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:14.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.109+0000 7f093d000640 1 --2- 172.21.15.83:0/3856343031 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55d6d1c185e0 0x55d6d1c1aa00 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=cc7c0ff7180b64a3 server_cookie=ee9668deef283454 in_seq=0 out_seq=0 2024-04-24T09:05:14.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.109+0000 7f091f400640 10 client.20418 ms_handle_connect on v2:172.21.15.189:6838/2548965336 2024-04-24T09:05:14.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.110+0000 7f091f400640 1 -- 172.21.15.83:0/3856343031 <== mds.4 v2:172.21.15.189:6838/2548965336 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d6d1b03a60 con 0x55d6d1c185e0 2024-04-24T09:05:14.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.110+0000 7f091f400640 10 client.20418 handle_command_reply: tid=0 2024-04-24T09:05:14.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.110+0000 7f09431a1740 1 client.20418 shutdown 2024-04-24T09:05:14.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.110+0000 7f09431a1740 1 -- 172.21.15.83:0/3856343031 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6d1b96b40 msgr2=0x55d6d1baa1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:14.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.110+0000 7f09431a1740 1 --2- 172.21.15.83:0/3856343031 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6d1b96b40 0x55d6d1baa1b0 secure :-1 s=READY pgs=2085 cs=0 l=1 rev1=1 crypto rx=0x7f092c00dbd0 tx=0x7f092c00d7c0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.110+0000 7f09431a1740 1 -- 172.21.15.83:0/3856343031 shutdown_connections 2024-04-24T09:05:14.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.110+0000 7f09431a1740 1 --2- 172.21.15.83:0/3856343031 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55d6d1c185e0 0x55d6d1c1aa00 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.110+0000 7f09431a1740 1 --2- 172.21.15.83:0/3856343031 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d6d1ba1b30 0x55d6d1b9c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.110+0000 7f09431a1740 1 --2- 172.21.15.83:0/3856343031 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d6d1b97460 0x55d6d1b9bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.110+0000 7f091f400640 0 client.20418 ms_handle_reset on v2:172.21.15.189:6838/2548965336 2024-04-24T09:05:14.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.110+0000 7f09431a1740 1 --2- 172.21.15.83:0/3856343031 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6d1b96b40 0x55d6d1baa1b0 unknown :-1 s=CLOSED pgs=2085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.110+0000 7f09431a1740 1 -- 172.21.15.83:0/3856343031 >> 172.21.15.83:0/3856343031 conn(0x55d6d1b92e60 msgr2=0x55d6d1b9e8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:14.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.110+0000 7f09431a1740 1 -- 172.21.15.83:0/3856343031 shutdown_connections 2024-04-24T09:05:14.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.110+0000 7f09431a1740 1 -- 172.21.15.83:0/3856343031 wait complete. 2024-04-24T09:05:14.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.110+0000 7f09431a1740 20 client.20418 trim_cache size 0 max 16384 2024-04-24T09:05:14.113 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:05:14.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.111+0000 7f093ee00640 1 -- 172.21.15.83:0/1585836307 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f09200a3d20 msgr2=0x7f09200a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:14.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.111+0000 7f093ee00640 1 --2- 172.21.15.83:0/1585836307 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f09200a3d20 0x7f09200a61e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f0930005fd0 tx=0x7f0930009530 comp rx=0 tx=0).stop 2024-04-24T09:05:14.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.111+0000 7f093ee00640 1 -- 172.21.15.83:0/1585836307 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f09381071f0 msgr2=0x7f0938198c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:14.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.111+0000 7f093ee00640 1 --2- 172.21.15.83:0/1585836307 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f09381071f0 0x7f0938198c80 secure :-1 s=READY pgs=2081 cs=0 l=1 rev1=1 crypto rx=0x7f0924005ab0 tx=0x7f0924003cd0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.111+0000 7f093ee00640 1 -- 172.21.15.83:0/1585836307 shutdown_connections 2024-04-24T09:05:14.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.111+0000 7f093ee00640 1 --2- 172.21.15.83:0/1585836307 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f09200a3d20 0x7f09200a61e0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.111+0000 7f093ee00640 1 --2- 172.21.15.83:0/1585836307 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f093819cfb0 0x7f093819d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.111+0000 7f093ee00640 1 --2- 172.21.15.83:0/1585836307 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f09381071f0 0x7f0938198c80 unknown :-1 s=CLOSED pgs=2081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.111+0000 7f093ee00640 1 --2- 172.21.15.83:0/1585836307 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0938102120 0x7f0938198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.111+0000 7f093ee00640 1 -- 172.21.15.83:0/1585836307 >> 172.21.15.83:0/1585836307 conn(0x7f09380fd9a0 msgr2=0x7f09380fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:14.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.111+0000 7f093ee00640 1 -- 172.21.15.83:0/1585836307 shutdown_connections 2024-04-24T09:05:14.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.111+0000 7f093ee00640 1 -- 172.21.15.83:0/1585836307 wait complete. 2024-04-24T09:05:14.124 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:05:14 smithi083 ceph-mon[27907]: pgmap v427: 129 pgs: 129 active+clean; 74 MiB data, 997 MiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 581 KiB/s wr, 68 op/s 2024-04-24T09:05:14.125 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:05:14 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:05:14.125 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:05:14 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:05:14.125 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:05:14 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:05:14.125 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:05:14 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/3754001237' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:05:14.129 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-24T09:05:14.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.333+0000 7ff8fde00640 1 -- 172.21.15.83:0/1570228691 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8f810e770 msgr2=0x7ff8f8110b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:14.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.333+0000 7ff8fde00640 1 --2- 172.21.15.83:0/1570228691 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8f810e770 0x7ff8f8110b60 secure :-1 s=READY pgs=2086 cs=0 l=1 rev1=1 crypto rx=0x7ff8f0009ca0 tx=0x7ff8f002a950 comp rx=0 tx=0).stop 2024-04-24T09:05:14.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.333+0000 7ff8fde00640 1 -- 172.21.15.83:0/1570228691 shutdown_connections 2024-04-24T09:05:14.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.333+0000 7ff8fde00640 1 --2- 172.21.15.83:0/1570228691 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8f810e770 0x7ff8f8110b60 unknown :-1 s=CLOSED pgs=2086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.333+0000 7ff8fde00640 1 --2- 172.21.15.83:0/1570228691 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff8f8074cd0 0x7ff8f810e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.333+0000 7ff8fde00640 1 --2- 172.21.15.83:0/1570228691 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8f80743b0 0x7ff8f8074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.333+0000 7ff8fde00640 1 -- 172.21.15.83:0/1570228691 >> 172.21.15.83:0/1570228691 conn(0x7ff8f80fd9a0 msgr2=0x7ff8f80ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:14.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.333+0000 7ff8fde00640 1 -- 172.21.15.83:0/1570228691 shutdown_connections 2024-04-24T09:05:14.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.333+0000 7ff8fde00640 1 -- 172.21.15.83:0/1570228691 wait complete. 2024-04-24T09:05:14.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.335+0000 7ff8fde00640 1 Processor -- start 2024-04-24T09:05:14.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.335+0000 7ff8fde00640 1 -- start start 2024-04-24T09:05:14.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.335+0000 7ff8fde00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8f80743b0 0x7ff8f81a0460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:14.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.335+0000 7ff8fde00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8f8074cd0 0x7ff8f81a09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:14.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.335+0000 7ff8fde00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff8f810e770 0x7ff8f819a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:14.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.335+0000 7ff8fde00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f8114300 con 0x7ff8f80743b0 2024-04-24T09:05:14.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.335+0000 7ff8fde00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f8114180 con 0x7ff8f8074cd0 2024-04-24T09:05:14.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.335+0000 7ff8fde00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f8114480 con 0x7ff8f810e770 2024-04-24T09:05:14.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.335+0000 7ff8fca00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8f80743b0 0x7ff8f81a0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:14.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.336+0000 7ff8fca00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8f80743b0 0x7ff8f81a0460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36064/0 (socket says 172.21.15.83:36064) 2024-04-24T09:05:14.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.336+0000 7ff8fca00640 1 -- 172.21.15.83:0/964882315 learned_addr learned my addr 172.21.15.83:0/964882315 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:05:14.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.336+0000 7ff8efe00640 1 --2- 172.21.15.83:0/964882315 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8f8074cd0 0x7ff8f81a09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:14.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.336+0000 7ff8fd400640 1 --2- 172.21.15.83:0/964882315 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff8f810e770 0x7ff8f819a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:14.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.336+0000 7ff8fca00640 1 -- 172.21.15.83:0/964882315 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff8f810e770 msgr2=0x7ff8f819a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:14.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.336+0000 7ff8fca00640 1 --2- 172.21.15.83:0/964882315 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff8f810e770 0x7ff8f819a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.336+0000 7ff8fca00640 1 -- 172.21.15.83:0/964882315 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8f8074cd0 msgr2=0x7ff8f81a09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:14.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.336+0000 7ff8fca00640 1 --2- 172.21.15.83:0/964882315 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8f8074cd0 0x7ff8f81a09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.336+0000 7ff8fca00640 1 -- 172.21.15.83:0/964882315 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8f819ade0 con 0x7ff8f80743b0 2024-04-24T09:05:14.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.336+0000 7ff8efe00640 1 --2- 172.21.15.83:0/964882315 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8f8074cd0 0x7ff8f81a09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:05:14.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.336+0000 7ff8fca00640 1 --2- 172.21.15.83:0/964882315 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8f80743b0 0x7ff8f81a0460 secure :-1 s=READY pgs=2087 cs=0 l=1 rev1=1 crypto rx=0x7ff8e800c920 tx=0x7ff8e800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:14.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.337+0000 7ff8f6000640 1 -- 172.21.15.83:0/964882315 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e8007c70 con 0x7ff8f80743b0 2024-04-24T09:05:14.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.337+0000 7ff8f6000640 1 -- 172.21.15.83:0/964882315 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff8e8007e10 con 0x7ff8f80743b0 2024-04-24T09:05:14.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.337+0000 7ff8f6000640 1 -- 172.21.15.83:0/964882315 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e8018820 con 0x7ff8f80743b0 2024-04-24T09:05:14.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.337+0000 7ff8fde00640 1 -- 172.21.15.83:0/964882315 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8f819b0d0 con 0x7ff8f80743b0 2024-04-24T09:05:14.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.337+0000 7ff8fde00640 1 -- 172.21.15.83:0/964882315 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8f81a7320 con 0x7ff8f80743b0 2024-04-24T09:05:14.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.340+0000 7ff8f6000640 1 -- 172.21.15.83:0/964882315 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff8e80189c0 con 0x7ff8f80743b0 2024-04-24T09:05:14.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.340+0000 7ff902206740 1 Processor -- start 2024-04-24T09:05:14.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.340+0000 7ff902206740 1 -- start start 2024-04-24T09:05:14.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.341+0000 7ff902206740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5598509b5b40 0x5598509b5f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:14.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.341+0000 7ff902206740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5598509b6460 0x5598509c05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:14.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.341+0000 7ff902206740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5598509c0b30 0x5598509c2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:14.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.341+0000 7ff902206740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55985091b1d0 con 0x5598509b5b40 2024-04-24T09:05:14.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.341+0000 7ff902206740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5598508c1450 con 0x5598509b6460 2024-04-24T09:05:14.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.341+0000 7ff902206740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559850917190 con 0x5598509c0b30 2024-04-24T09:05:14.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.341+0000 7ff8fd400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5598509b5b40 0x5598509b5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:14.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.341+0000 7ff8fd400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5598509b5b40 0x5598509b5f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36066/0 (socket says 172.21.15.83:36066) 2024-04-24T09:05:14.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.341+0000 7ff8fd400640 1 -- 172.21.15.83:0/1124517846 learned_addr learned my addr 172.21.15.83:0/1124517846 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:05:14.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.341+0000 7ff8efe00640 1 --2- 172.21.15.83:0/1124517846 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5598509c0b30 0x5598509c2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:14.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.341+0000 7ff8fca00640 1 --2- 172.21.15.83:0/1124517846 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5598509b6460 0x5598509c05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:14.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.341+0000 7ff8fd400640 1 -- 172.21.15.83:0/1124517846 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5598509c0b30 msgr2=0x5598509c2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:14.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.341+0000 7ff8fd400640 1 --2- 172.21.15.83:0/1124517846 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5598509c0b30 0x5598509c2f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.341+0000 7ff8fd400640 1 -- 172.21.15.83:0/1124517846 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5598509b6460 msgr2=0x5598509c05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:14.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.341+0000 7ff8fd400640 1 --2- 172.21.15.83:0/1124517846 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5598509b6460 0x5598509c05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.341+0000 7ff8fd400640 1 -- 172.21.15.83:0/1124517846 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55985097e310 con 0x5598509b5b40 2024-04-24T09:05:14.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.342+0000 7ff8fca00640 1 --2- 172.21.15.83:0/1124517846 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5598509b6460 0x5598509c05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:05:14.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.342+0000 7ff8f6000640 1 --2- 172.21.15.83:0/964882315 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff8dc0a3da0 0x7ff8dc0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:14.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.342+0000 7ff8f6000640 1 -- 172.21.15.83:0/964882315 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7ff8e80d0da0 con 0x7ff8f80743b0 2024-04-24T09:05:14.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.342+0000 7ff8fd400640 1 --2- 172.21.15.83:0/1124517846 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5598509b5b40 0x5598509b5f20 secure :-1 s=READY pgs=2088 cs=0 l=1 rev1=1 crypto rx=0x7ff8f0009420 tx=0x7ff8f0009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cd8dc97ef95d46 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:14.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.342+0000 7ff8fca00640 1 --2- 172.21.15.83:0/964882315 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff8dc0a3da0 0x7ff8dc0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:14.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.342+0000 7ff8edc00640 1 -- 172.21.15.83:0/1124517846 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8f0034050 con 0x5598509b5b40 2024-04-24T09:05:14.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.342+0000 7ff8edc00640 1 -- 172.21.15.83:0/1124517846 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff8f0003e20 con 0x5598509b5b40 2024-04-24T09:05:14.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.342+0000 7ff8edc00640 1 -- 172.21.15.83:0/1124517846 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8f0004280 con 0x5598509b5b40 2024-04-24T09:05:14.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.343+0000 7ff902206740 1 -- 172.21.15.83:0/1124517846 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5598509b5b40 msgr2=0x5598509b5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:14.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.343+0000 7ff8fca00640 1 --2- 172.21.15.83:0/964882315 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff8dc0a3da0 0x7ff8dc0a6260 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7ff8e80ce180 tx=0x7ff8e80bd040 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:14.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.343+0000 7ff902206740 1 --2- 172.21.15.83:0/1124517846 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5598509b5b40 0x5598509b5f20 secure :-1 s=READY pgs=2088 cs=0 l=1 rev1=1 crypto rx=0x7ff8f0009420 tx=0x7ff8f0009a20 comp rx=0 tx=0).stop 2024-04-24T09:05:14.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.343+0000 7ff902206740 1 -- 172.21.15.83:0/1124517846 shutdown_connections 2024-04-24T09:05:14.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.343+0000 7ff902206740 1 --2- 172.21.15.83:0/1124517846 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5598509c0b30 0x5598509c2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.343+0000 7ff902206740 1 --2- 172.21.15.83:0/1124517846 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5598509b6460 0x5598509c05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.343+0000 7ff902206740 1 --2- 172.21.15.83:0/1124517846 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5598509b5b40 0x5598509b5f20 unknown :-1 s=CLOSED pgs=2088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.343+0000 7ff902206740 1 -- 172.21.15.83:0/1124517846 >> 172.21.15.83:0/1124517846 conn(0x5598509b1e60 msgr2=0x5598509b2240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:14.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.343+0000 7ff902206740 1 -- 172.21.15.83:0/1124517846 shutdown_connections 2024-04-24T09:05:14.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.343+0000 7ff902206740 1 -- 172.21.15.83:0/1124517846 wait complete. 2024-04-24T09:05:14.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.344+0000 7ff902206740 1 -- start start 2024-04-24T09:05:14.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.344+0000 7ff902206740 1 Processor -- start 2024-04-24T09:05:14.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.344+0000 7ff902206740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5598509b5b40 0x5598509c05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:14.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.344+0000 7ff902206740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5598509b6460 0x5598509bd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:14.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.345+0000 7ff902206740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5598509c0b30 0x5598509bdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:14.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.345+0000 7ff902206740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5598509c3960 con 0x5598509c0b30 2024-04-24T09:05:14.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.345+0000 7ff902206740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5598508d4810 con 0x5598509b6460 2024-04-24T09:05:14.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.345+0000 7ff902206740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5598509c3ae0 con 0x5598509b5b40 2024-04-24T09:05:14.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.345+0000 7ff902206740 10 client.0 _ll_register_callbacks cb 0x5598508a2140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:05:14.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.345+0000 7ff8fd400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5598509c0b30 0x5598509bdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:14.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.345+0000 7ff8fd400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5598509c0b30 0x5598509bdfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36074/0 (socket says 172.21.15.83:36074) 2024-04-24T09:05:14.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.345+0000 7ff8fd400640 1 -- 172.21.15.83:0/80530049 learned_addr learned my addr 172.21.15.83:0/80530049 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:05:14.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.345+0000 7ff8efe00640 1 --2- 172.21.15.83:0/80530049 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5598509b6460 0x5598509bd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:14.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.345+0000 7ff8fd400640 1 --2- 172.21.15.83:0/80530049 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5598509b5b40 0x5598509c05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:14.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.345+0000 7ff8fd400640 1 -- 172.21.15.83:0/80530049 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5598509b5b40 msgr2=0x5598509c05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:14.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.345+0000 7ff8fd400640 1 --2- 172.21.15.83:0/80530049 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5598509b5b40 0x5598509c05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.345+0000 7ff8fd400640 1 -- 172.21.15.83:0/80530049 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5598509b6460 msgr2=0x5598509bd8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:14.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.345+0000 7ff8fd400640 1 --2- 172.21.15.83:0/80530049 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5598509b6460 0x5598509bd8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.345+0000 7ff8fd400640 1 -- 172.21.15.83:0/80530049 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598508e53f0 con 0x5598509c0b30 2024-04-24T09:05:14.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.346+0000 7ff8fd400640 1 --2- 172.21.15.83:0/80530049 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5598509c0b30 0x5598509bdfe0 secure :-1 s=READY pgs=2089 cs=0 l=1 rev1=1 crypto rx=0x7ff8f002b140 tx=0x7ff8f002b050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6ade73a2b9b4385 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:14.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.346+0000 7ff8e6000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:05:14.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.346+0000 7ff8e6000640 1 -- 172.21.15.83:0/80530049 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8f0054060 con 0x5598509c0b30 2024-04-24T09:05:14.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.346+0000 7ff8e6000640 1 -- 172.21.15.83:0/80530049 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7ff8f0003970 con 0x5598509c0b30 2024-04-24T09:05:14.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.346+0000 7ff902206740 1 -- 172.21.15.83:0/80530049 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8f0003e20 con 0x5598509c0b30 2024-04-24T09:05:14.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.346+0000 7ff8e6000640 1 -- 172.21.15.83:0/80530049 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7ff8f0003b70 con 0x5598509c0b30 2024-04-24T09:05:14.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.346+0000 7ff902206740 10 client.20430 fetch_fsmap learned FSMap version 43 2024-04-24T09:05:14.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.346+0000 7ff902206740 1 -- 172.21.15.83:0/80530049 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55985062d3f0 con 0x5598509c0b30 2024-04-24T09:05:14.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.347+0000 7ff8e6000640 1 -- 172.21.15.83:0/80530049 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff8f0003e20 con 0x5598509c0b30 2024-04-24T09:05:14.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.347+0000 7ff8e6000640 1 -- 172.21.15.83:0/80530049 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7ff8f004e040 con 0x5598509c0b30 2024-04-24T09:05:14.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.347+0000 7ff902206740 10 client.20430 fetch_fsmap finished waiting for FSMap version 43 2024-04-24T09:05:14.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.347+0000 7ff902206740 10 client.20430 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-24T09:05:14.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.347+0000 7ff902206740 20 client.20430 populate_metadata read hostname 'smithi083' 2024-04-24T09:05:14.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.347+0000 7ff902206740 1 --2- 172.21.15.83:0/80530049 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x5598509d0ae0 0x5598509d2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:14.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.347+0000 7ff902206740 4 client.20430 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:05:14.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.347+0000 7ff902206740 1 -- 172.21.15.83:0/80530049 --> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559850928bf0 con 0x5598509d0ae0 2024-04-24T09:05:14.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.348+0000 7ff8efe00640 1 --2- 172.21.15.83:0/80530049 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x5598509d0ae0 0x5598509d2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:14.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.349+0000 7ff8efe00640 1 --2- 172.21.15.83:0/80530049 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x5598509d0ae0 0x5598509d2ed0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=811004b88110240 server_cookie=170ead55acf1762a in_seq=0 out_seq=0 2024-04-24T09:05:14.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.349+0000 7ff8e6000640 10 client.20430 ms_handle_connect on v2:172.21.15.138:6832/1157967658 2024-04-24T09:05:14.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.351+0000 7ff8e6000640 1 -- 172.21.15.83:0/80530049 <== mds.3 v2:172.21.15.138:6832/1157967658 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559850928bf0 con 0x5598509d0ae0 2024-04-24T09:05:14.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.351+0000 7ff8e6000640 10 client.20430 handle_command_reply: tid=0 2024-04-24T09:05:14.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.351+0000 7ff902206740 1 client.20430 shutdown 2024-04-24T09:05:14.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.351+0000 7ff902206740 1 -- 172.21.15.83:0/80530049 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5598509c0b30 msgr2=0x5598509bdfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:14.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.351+0000 7ff902206740 1 --2- 172.21.15.83:0/80530049 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5598509c0b30 0x5598509bdfe0 secure :-1 s=READY pgs=2089 cs=0 l=1 rev1=1 crypto rx=0x7ff8f002b140 tx=0x7ff8f002b050 comp rx=0 tx=0).stop 2024-04-24T09:05:14.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.351+0000 7ff902206740 1 -- 172.21.15.83:0/80530049 shutdown_connections 2024-04-24T09:05:14.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.351+0000 7ff902206740 1 --2- 172.21.15.83:0/80530049 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5598509c0b30 0x5598509bdfe0 unknown :-1 s=CLOSED pgs=2089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.351+0000 7ff902206740 1 --2- 172.21.15.83:0/80530049 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5598509b6460 0x5598509bd8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.351+0000 7ff902206740 1 --2- 172.21.15.83:0/80530049 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x5598509d0ae0 0x5598509d2ed0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.351+0000 7ff902206740 1 --2- 172.21.15.83:0/80530049 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5598509b5b40 0x5598509c05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.351+0000 7ff902206740 1 -- 172.21.15.83:0/80530049 >> 172.21.15.83:0/80530049 conn(0x5598509b1e60 msgr2=0x5598509b7b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:14.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.351+0000 7ff8e6000640 0 client.20430 ms_handle_reset on v2:172.21.15.138:6832/1157967658 2024-04-24T09:05:14.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.351+0000 7ff902206740 1 -- 172.21.15.83:0/80530049 shutdown_connections 2024-04-24T09:05:14.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.351+0000 7ff902206740 1 -- 172.21.15.83:0/80530049 wait complete. 2024-04-24T09:05:14.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.351+0000 7ff902206740 20 client.20430 trim_cache size 0 max 16384 2024-04-24T09:05:14.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.366+0000 7ff902206740 1 Processor -- start 2024-04-24T09:05:14.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.366+0000 7ff902206740 1 -- start start 2024-04-24T09:05:14.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.367+0000 7ff902206740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5598509b5b40 0x5598509cc820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:14.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.367+0000 7ff902206740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5598509b6460 0x5598509bca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:14.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.367+0000 7ff902206740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5598509c0b30 0x5598509c00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:14.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.367+0000 7ff902206740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f81b15c0 con 0x5598509c0b30 2024-04-24T09:05:14.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.367+0000 7ff902206740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f81b31c0 con 0x5598509b5b40 2024-04-24T09:05:14.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.367+0000 7ff902206740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f81b1c90 con 0x5598509b6460 2024-04-24T09:05:14.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.367+0000 7ff8fd400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5598509c0b30 0x5598509c00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:14.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.367+0000 7ff8fd400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5598509c0b30 0x5598509c00e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36086/0 (socket says 172.21.15.83:36086) 2024-04-24T09:05:14.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.367+0000 7ff8fd400640 1 -- 172.21.15.83:0/139529727 learned_addr learned my addr 172.21.15.83:0/139529727 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:05:14.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.367+0000 7ff8efe00640 1 --2- 172.21.15.83:0/139529727 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5598509b6460 0x5598509bca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:14.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.367+0000 7ff8fd400640 1 --2- 172.21.15.83:0/139529727 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5598509b5b40 0x5598509cc820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:14.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.368+0000 7ff8fd400640 1 -- 172.21.15.83:0/139529727 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5598509b6460 msgr2=0x5598509bca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:14.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.368+0000 7ff8fd400640 1 --2- 172.21.15.83:0/139529727 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5598509b6460 0x5598509bca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.368+0000 7ff8fd400640 1 -- 172.21.15.83:0/139529727 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5598509b5b40 msgr2=0x5598509cc820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:14.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.368+0000 7ff8fd400640 1 --2- 172.21.15.83:0/139529727 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5598509b5b40 0x5598509cc820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.368+0000 7ff8fd400640 1 -- 172.21.15.83:0/139529727 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598509d3240 con 0x5598509c0b30 2024-04-24T09:05:14.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.368+0000 7ff8fd400640 1 --2- 172.21.15.83:0/139529727 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5598509c0b30 0x5598509c00e0 secure :-1 s=READY pgs=2090 cs=0 l=1 rev1=1 crypto rx=0x7ff8f002fa30 tx=0x7ff8f0009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97fb4d1a871b4099 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:14.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.368+0000 7ff8ed200640 1 -- 172.21.15.83:0/139529727 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8f0054070 con 0x5598509c0b30 2024-04-24T09:05:14.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.368+0000 7ff8ed200640 1 -- 172.21.15.83:0/139529727 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff8f0043440 con 0x5598509c0b30 2024-04-24T09:05:14.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.368+0000 7ff8ed200640 1 -- 172.21.15.83:0/139529727 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8f0034040 con 0x5598509c0b30 2024-04-24T09:05:14.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.369+0000 7ff902206740 1 -- 172.21.15.83:0/139529727 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5598509c0b30 msgr2=0x5598509c00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:14.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.369+0000 7ff902206740 1 --2- 172.21.15.83:0/139529727 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5598509c0b30 0x5598509c00e0 secure :-1 s=READY pgs=2090 cs=0 l=1 rev1=1 crypto rx=0x7ff8f002fa30 tx=0x7ff8f0009920 comp rx=0 tx=0).stop 2024-04-24T09:05:14.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.369+0000 7ff902206740 1 -- 172.21.15.83:0/139529727 shutdown_connections 2024-04-24T09:05:14.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.369+0000 7ff902206740 1 --2- 172.21.15.83:0/139529727 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5598509c0b30 0x5598509c00e0 unknown :-1 s=CLOSED pgs=2090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.369+0000 7ff902206740 1 --2- 172.21.15.83:0/139529727 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5598509b6460 0x5598509bca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.369+0000 7ff902206740 1 --2- 172.21.15.83:0/139529727 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5598509b5b40 0x5598509cc820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.369+0000 7ff902206740 1 -- 172.21.15.83:0/139529727 >> 172.21.15.83:0/139529727 conn(0x5598509b1e60 msgr2=0x5598509b7070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:14.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.369+0000 7ff902206740 1 -- 172.21.15.83:0/139529727 shutdown_connections 2024-04-24T09:05:14.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.369+0000 7ff902206740 1 -- 172.21.15.83:0/139529727 wait complete. 2024-04-24T09:05:14.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.370+0000 7ff902206740 1 -- start start 2024-04-24T09:05:14.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.370+0000 7ff902206740 1 Processor -- start 2024-04-24T09:05:14.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.370+0000 7ff902206740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5598509b5b40 0x5598509c1b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:14.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.370+0000 7ff902206740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5598509b6460 0x5598509c20d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:14.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.370+0000 7ff902206740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5598509c0b30 0x5598509c7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:14.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.370+0000 7ff902206740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f81b2a90 con 0x5598509c0b30 2024-04-24T09:05:14.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.370+0000 7ff902206740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f81b2360 con 0x5598509b5b40 2024-04-24T09:05:14.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.370+0000 7ff902206740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f81af5c0 con 0x5598509b6460 2024-04-24T09:05:14.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.370+0000 7ff902206740 10 client.0 _ll_register_callbacks cb 0x7ff8f0005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:05:14.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.371+0000 7ff8fd400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5598509c0b30 0x5598509c7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:14.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.371+0000 7ff8fd400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5598509c0b30 0x5598509c7210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36092/0 (socket says 172.21.15.83:36092) 2024-04-24T09:05:14.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.371+0000 7ff8fd400640 1 -- 172.21.15.83:0/3896414482 learned_addr learned my addr 172.21.15.83:0/3896414482 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:05:14.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.371+0000 7ff8efe00640 1 --2- 172.21.15.83:0/3896414482 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5598509b6460 0x5598509c20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:14.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.371+0000 7ff8fd400640 1 -- 172.21.15.83:0/3896414482 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5598509b6460 msgr2=0x5598509c20d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:14.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.371+0000 7ff8fd400640 1 --2- 172.21.15.83:0/3896414482 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5598509b6460 0x5598509c20d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.371+0000 7ff8fd400640 1 -- 172.21.15.83:0/3896414482 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5598509b5b40 msgr2=0x5598509c1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:14.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.371+0000 7ff8fd400640 1 --2- 172.21.15.83:0/3896414482 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5598509b5b40 0x5598509c1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.371+0000 7ff8fd400640 1 -- 172.21.15.83:0/3896414482 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598506f77d0 con 0x5598509c0b30 2024-04-24T09:05:14.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.371+0000 7ff8fd400640 1 --2- 172.21.15.83:0/3896414482 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5598509c0b30 0x5598509c7210 secure :-1 s=READY pgs=2091 cs=0 l=1 rev1=1 crypto rx=0x5598509bedb0 tx=0x7ff8f0004350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3fae7d66b2c8d4f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:14.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.371+0000 7ff8e6000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:05:14.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.372+0000 7ff8e6000640 1 -- 172.21.15.83:0/3896414482 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8f0054070 con 0x5598509c0b30 2024-04-24T09:05:14.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.372+0000 7ff902206740 1 -- 172.21.15.83:0/3896414482 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559850922a60 con 0x5598509c0b30 2024-04-24T09:05:14.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.372+0000 7ff8e6000640 1 -- 172.21.15.83:0/3896414482 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7ff8f0048040 con 0x5598509c0b30 2024-04-24T09:05:14.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.372+0000 7ff8e6000640 1 -- 172.21.15.83:0/3896414482 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7ff8f0034040 con 0x5598509c0b30 2024-04-24T09:05:14.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.372+0000 7ff902206740 10 client.20436 fetch_fsmap learned FSMap version 43 2024-04-24T09:05:14.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.372+0000 7ff902206740 1 -- 172.21.15.83:0/3896414482 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x559850a37120 con 0x5598509c0b30 2024-04-24T09:05:14.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.372+0000 7ff8e6000640 1 -- 172.21.15.83:0/3896414482 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x559850922a60 con 0x5598509c0b30 2024-04-24T09:05:14.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.372+0000 7ff8e6000640 1 -- 172.21.15.83:0/3896414482 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7ff8f0041070 con 0x5598509c0b30 2024-04-24T09:05:14.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.373+0000 7ff902206740 10 client.20436 fetch_fsmap finished waiting for FSMap version 43 2024-04-24T09:05:14.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.373+0000 7ff902206740 10 client.20436 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-24T09:05:14.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.373+0000 7ff902206740 20 client.20436 populate_metadata read hostname 'smithi083' 2024-04-24T09:05:14.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.373+0000 7ff902206740 1 --2- 172.21.15.83:0/3896414482 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x559850a376c0 0x559850a39ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:14.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.373+0000 7ff902206740 4 client.20436 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:05:14.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.373+0000 7ff902206740 1 -- 172.21.15.83:0/3896414482 --> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55985089e4d0 con 0x559850a376c0 2024-04-24T09:05:14.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.373+0000 7ff8efe00640 1 --2- 172.21.15.83:0/3896414482 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x559850a376c0 0x559850a39ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:14.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.374+0000 7ff8efe00640 1 --2- 172.21.15.83:0/3896414482 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x559850a376c0 0x559850a39ae0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1b151a05206ab386 server_cookie=5228a46f8be5f19a in_seq=0 out_seq=0 2024-04-24T09:05:14.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.374+0000 7ff8e6000640 10 client.20436 ms_handle_connect on v2:172.21.15.138:6832/1157967658 2024-04-24T09:05:14.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.375+0000 7ff8e6000640 1 -- 172.21.15.83:0/3896414482 <== mds.3 v2:172.21.15.138:6832/1157967658 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55985089e4d0 con 0x559850a376c0 2024-04-24T09:05:14.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.375+0000 7ff8e6000640 10 client.20436 handle_command_reply: tid=0 2024-04-24T09:05:14.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.375+0000 7ff902206740 1 client.20436 shutdown 2024-04-24T09:05:14.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.375+0000 7ff902206740 1 -- 172.21.15.83:0/3896414482 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5598509c0b30 msgr2=0x5598509c7210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:14.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.375+0000 7ff902206740 1 --2- 172.21.15.83:0/3896414482 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5598509c0b30 0x5598509c7210 secure :-1 s=READY pgs=2091 cs=0 l=1 rev1=1 crypto rx=0x5598509bedb0 tx=0x7ff8f0004350 comp rx=0 tx=0).stop 2024-04-24T09:05:14.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.375+0000 7ff902206740 1 -- 172.21.15.83:0/3896414482 shutdown_connections 2024-04-24T09:05:14.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.375+0000 7ff902206740 1 --2- 172.21.15.83:0/3896414482 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5598509c0b30 0x5598509c7210 unknown :-1 s=CLOSED pgs=2091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.375+0000 7ff902206740 1 --2- 172.21.15.83:0/3896414482 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x559850a376c0 0x559850a39ae0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.375+0000 7ff902206740 1 --2- 172.21.15.83:0/3896414482 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5598509b6460 0x5598509c20d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.375+0000 7ff902206740 1 --2- 172.21.15.83:0/3896414482 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5598509b5b40 0x5598509c1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.375+0000 7ff902206740 1 -- 172.21.15.83:0/3896414482 >> 172.21.15.83:0/3896414482 conn(0x5598509b1e60 msgr2=0x5598509bc880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:14.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.375+0000 7ff8e6000640 0 client.20436 ms_handle_reset on v2:172.21.15.138:6832/1157967658 2024-04-24T09:05:14.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.375+0000 7ff902206740 1 -- 172.21.15.83:0/3896414482 shutdown_connections 2024-04-24T09:05:14.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.375+0000 7ff902206740 1 -- 172.21.15.83:0/3896414482 wait complete. 2024-04-24T09:05:14.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.376+0000 7ff902206740 20 client.20436 trim_cache size 0 max 16384 2024-04-24T09:05:14.380 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:05:14.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.376+0000 7ff8fde00640 1 -- 172.21.15.83:0/964882315 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff8dc0a3da0 msgr2=0x7ff8dc0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:14.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.376+0000 7ff8fde00640 1 --2- 172.21.15.83:0/964882315 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff8dc0a3da0 0x7ff8dc0a6260 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7ff8e80ce180 tx=0x7ff8e80bd040 comp rx=0 tx=0).stop 2024-04-24T09:05:14.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.376+0000 7ff8fde00640 1 -- 172.21.15.83:0/964882315 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8f80743b0 msgr2=0x7ff8f81a0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:14.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.376+0000 7ff8fde00640 1 --2- 172.21.15.83:0/964882315 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8f80743b0 0x7ff8f81a0460 secure :-1 s=READY pgs=2087 cs=0 l=1 rev1=1 crypto rx=0x7ff8e800c920 tx=0x7ff8e800cde0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.376+0000 7ff8fde00640 1 -- 172.21.15.83:0/964882315 shutdown_connections 2024-04-24T09:05:14.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.376+0000 7ff8fde00640 1 --2- 172.21.15.83:0/964882315 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff8dc0a3da0 0x7ff8dc0a6260 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.376+0000 7ff8fde00640 1 --2- 172.21.15.83:0/964882315 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff8f810e770 0x7ff8f819a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.376+0000 7ff8fde00640 1 --2- 172.21.15.83:0/964882315 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8f8074cd0 0x7ff8f81a09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.376+0000 7ff8fde00640 1 --2- 172.21.15.83:0/964882315 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8f80743b0 0x7ff8f81a0460 unknown :-1 s=CLOSED pgs=2087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.376+0000 7ff8fde00640 1 -- 172.21.15.83:0/964882315 >> 172.21.15.83:0/964882315 conn(0x7ff8f80fd9a0 msgr2=0x7ff8f80ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:14.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.377+0000 7ff8fde00640 1 -- 172.21.15.83:0/964882315 shutdown_connections 2024-04-24T09:05:14.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.377+0000 7ff8fde00640 1 -- 172.21.15.83:0/964882315 wait complete. 2024-04-24T09:05:14.396 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T09:05:14.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:05:14 smithi138 ceph-mon[31441]: pgmap v427: 129 pgs: 129 active+clean; 74 MiB data, 997 MiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 581 KiB/s wr, 68 op/s 2024-04-24T09:05:14.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:05:14 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:05:14.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:05:14 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:05:14.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:05:14 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:05:14.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:05:14 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/3754001237' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:05:14.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:05:14 smithi189 ceph-mon[31791]: pgmap v427: 129 pgs: 129 active+clean; 74 MiB data, 997 MiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 581 KiB/s wr, 68 op/s 2024-04-24T09:05:14.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:05:14 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:05:14.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:05:14 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:05:14.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:05:14 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:05:14.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:05:14 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/3754001237' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:05:14.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.593+0000 7f750b000640 1 -- 172.21.15.83:0/197575913 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7504102ad0 msgr2=0x7f7504106b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:14.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.593+0000 7f750b000640 1 --2- 172.21.15.83:0/197575913 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7504102ad0 0x7f7504106b10 secure :-1 s=READY pgs=2092 cs=0 l=1 rev1=1 crypto rx=0x7f74f0009900 tx=0x7f74f002a3f0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.593+0000 7f750b000640 1 -- 172.21.15.83:0/197575913 shutdown_connections 2024-04-24T09:05:14.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.593+0000 7f750b000640 1 --2- 172.21.15.83:0/197575913 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f75041071f0 0x7f750410edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.593+0000 7f750b000640 1 --2- 172.21.15.83:0/197575913 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7504102ad0 0x7f7504106b10 unknown :-1 s=CLOSED pgs=2092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.593+0000 7f750b000640 1 --2- 172.21.15.83:0/197575913 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7504102120 0x7f7504102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.593+0000 7f750b000640 1 -- 172.21.15.83:0/197575913 >> 172.21.15.83:0/197575913 conn(0x7f75040fd9a0 msgr2=0x7f75040ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:14.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.593+0000 7f750b000640 1 -- 172.21.15.83:0/197575913 shutdown_connections 2024-04-24T09:05:14.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.594+0000 7f750b000640 1 -- 172.21.15.83:0/197575913 wait complete. 2024-04-24T09:05:14.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.594+0000 7f750b000640 1 Processor -- start 2024-04-24T09:05:14.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.595+0000 7f750b000640 1 -- start start 2024-04-24T09:05:14.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.595+0000 7f750b000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7504102120 0x7f75041a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:14.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.595+0000 7f750b000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f75041071f0 0x7f75041a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:14.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.595+0000 7f750b000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f750419a720 0x7f750419aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:14.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.595+0000 7f750b000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7504112180 con 0x7f750419a720 2024-04-24T09:05:14.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.595+0000 7f750b000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7504112000 con 0x7f7504102120 2024-04-24T09:05:14.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.595+0000 7f750b000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7504112300 con 0x7f75041071f0 2024-04-24T09:05:14.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.595+0000 7f750a600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f750419a720 0x7f750419aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:14.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.595+0000 7f750a600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f750419a720 0x7f750419aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36114/0 (socket says 172.21.15.83:36114) 2024-04-24T09:05:14.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.595+0000 7f750a600640 1 -- 172.21.15.83:0/1336413957 learned_addr learned my addr 172.21.15.83:0/1336413957 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:05:14.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.595+0000 7f7509c00640 1 --2- 172.21.15.83:0/1336413957 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7504102120 0x7f75041a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:14.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.596+0000 7f7509200640 1 --2- 172.21.15.83:0/1336413957 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f75041071f0 0x7f75041a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:14.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.596+0000 7f750a600640 1 -- 172.21.15.83:0/1336413957 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f75041071f0 msgr2=0x7f75041a0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:14.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.596+0000 7f750a600640 1 --2- 172.21.15.83:0/1336413957 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f75041071f0 0x7f75041a0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.596+0000 7f750a600640 1 -- 172.21.15.83:0/1336413957 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7504102120 msgr2=0x7f75041a0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:14.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.596+0000 7f750a600640 1 --2- 172.21.15.83:0/1336413957 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7504102120 0x7f75041a0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.596+0000 7f750a600640 1 -- 172.21.15.83:0/1336413957 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f750419b320 con 0x7f750419a720 2024-04-24T09:05:14.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.596+0000 7f750a600640 1 --2- 172.21.15.83:0/1336413957 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f750419a720 0x7f750419aba0 secure :-1 s=READY pgs=2093 cs=0 l=1 rev1=1 crypto rx=0x7f74fc00cce0 tx=0x7f74fc00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:14.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.596+0000 7f7502000640 1 -- 172.21.15.83:0/1336413957 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74fc00f450 con 0x7f750419a720 2024-04-24T09:05:14.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.596+0000 7f7502000640 1 -- 172.21.15.83:0/1336413957 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f74fc00faa0 con 0x7f750419a720 2024-04-24T09:05:14.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.596+0000 7f750b000640 1 -- 172.21.15.83:0/1336413957 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75041a7400 con 0x7f750419a720 2024-04-24T09:05:14.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.596+0000 7f7502000640 1 -- 172.21.15.83:0/1336413957 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74fc018b70 con 0x7f750419a720 2024-04-24T09:05:14.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.596+0000 7f750b000640 1 -- 172.21.15.83:0/1336413957 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75041a77b0 con 0x7f750419a720 2024-04-24T09:05:14.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.600+0000 7f750f41e740 1 Processor -- start 2024-04-24T09:05:14.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.600+0000 7f7502000640 1 -- 172.21.15.83:0/1336413957 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f74fc018d10 con 0x7f750419a720 2024-04-24T09:05:14.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.600+0000 7f750f41e740 1 -- start start 2024-04-24T09:05:14.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.600+0000 7f750f41e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db08c6eb40 0x55db08c6ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:14.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.600+0000 7f750f41e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db08c6f460 0x55db08c795f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:14.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.600+0000 7f750f41e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55db08c79b30 0x55db08c7bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:14.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.600+0000 7f750f41e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55db08bd41d0 con 0x55db08c6eb40 2024-04-24T09:05:14.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.600+0000 7f750f41e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55db08b7a450 con 0x55db08c6f460 2024-04-24T09:05:14.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.600+0000 7f750f41e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55db08bd0190 con 0x55db08c79b30 2024-04-24T09:05:14.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.601+0000 7f7509200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db08c6eb40 0x55db08c6ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:14.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.601+0000 7f7509200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db08c6eb40 0x55db08c6ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36130/0 (socket says 172.21.15.83:36130) 2024-04-24T09:05:14.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.601+0000 7f7509200640 1 -- 172.21.15.83:0/3910888332 learned_addr learned my addr 172.21.15.83:0/3910888332 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:05:14.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.601+0000 7f7509c00640 1 --2- 172.21.15.83:0/3910888332 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db08c6f460 0x55db08c795f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:14.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.601+0000 7f7509200640 1 -- 172.21.15.83:0/3910888332 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55db08c79b30 msgr2=0x55db08c7bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:14.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.601+0000 7f7509200640 1 --2- 172.21.15.83:0/3910888332 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55db08c79b30 0x55db08c7bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.601+0000 7f7509200640 1 -- 172.21.15.83:0/3910888332 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db08c6f460 msgr2=0x55db08c795f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:14.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.601+0000 7f7509200640 1 --2- 172.21.15.83:0/3910888332 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db08c6f460 0x55db08c795f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.601+0000 7f7509200640 1 -- 172.21.15.83:0/3910888332 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db08c37310 con 0x55db08c6eb40 2024-04-24T09:05:14.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.601+0000 7f7509c00640 1 --2- 172.21.15.83:0/3910888332 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db08c6f460 0x55db08c795f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:05:14.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.601+0000 7f7509200640 1 --2- 172.21.15.83:0/3910888332 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db08c6eb40 0x55db08c6ef20 secure :-1 s=READY pgs=2094 cs=0 l=1 rev1=1 crypto rx=0x7f74f0005ab0 tx=0x7f74f0005e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31758beff296f968 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:14.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.601+0000 7f7502000640 1 --2- 172.21.15.83:0/1336413957 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f74ec0a3d20 0x7f74ec0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:14.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.602+0000 7f7502000640 1 -- 172.21.15.83:0/1336413957 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7f74fc0d1110 con 0x7f750419a720 2024-04-24T09:05:14.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.602+0000 7f74f6a00640 1 -- 172.21.15.83:0/3910888332 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74f0041020 con 0x55db08c6eb40 2024-04-24T09:05:14.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.602+0000 7f7509c00640 1 --2- 172.21.15.83:0/1336413957 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f74ec0a3d20 0x7f74ec0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:14.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.602+0000 7f74f6a00640 1 -- 172.21.15.83:0/3910888332 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f74f0004490 con 0x55db08c6eb40 2024-04-24T09:05:14.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.602+0000 7f74f6a00640 1 -- 172.21.15.83:0/3910888332 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74f0035070 con 0x55db08c6eb40 2024-04-24T09:05:14.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.602+0000 7f750f41e740 1 -- 172.21.15.83:0/3910888332 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db08c6eb40 msgr2=0x55db08c6ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:14.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.602+0000 7f750f41e740 1 --2- 172.21.15.83:0/3910888332 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db08c6eb40 0x55db08c6ef20 secure :-1 s=READY pgs=2094 cs=0 l=1 rev1=1 crypto rx=0x7f74f0005ab0 tx=0x7f74f0005e70 comp rx=0 tx=0).stop 2024-04-24T09:05:14.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.602+0000 7f7509c00640 1 --2- 172.21.15.83:0/1336413957 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f74ec0a3d20 0x7f74ec0a61e0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x55db08c70100 tx=0x7f74f8005f50 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:14.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.603+0000 7f750f41e740 1 -- 172.21.15.83:0/3910888332 shutdown_connections 2024-04-24T09:05:14.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.603+0000 7f750f41e740 1 --2- 172.21.15.83:0/3910888332 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55db08c79b30 0x55db08c7bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.603+0000 7f750f41e740 1 --2- 172.21.15.83:0/3910888332 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db08c6f460 0x55db08c795f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.603+0000 7f750f41e740 1 --2- 172.21.15.83:0/3910888332 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db08c6eb40 0x55db08c6ef20 unknown :-1 s=CLOSED pgs=2094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.603+0000 7f750f41e740 1 -- 172.21.15.83:0/3910888332 >> 172.21.15.83:0/3910888332 conn(0x55db08c6ae60 msgr2=0x55db08c6b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:14.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.603+0000 7f750f41e740 1 -- 172.21.15.83:0/3910888332 shutdown_connections 2024-04-24T09:05:14.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.603+0000 7f750f41e740 1 -- 172.21.15.83:0/3910888332 wait complete. 2024-04-24T09:05:14.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.603+0000 7f750f41e740 1 -- start start 2024-04-24T09:05:14.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.603+0000 7f750f41e740 1 Processor -- start 2024-04-24T09:05:14.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.604+0000 7f750f41e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55db08c6eb40 0x55db08c795c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:14.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.604+0000 7f750f41e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db08c6f460 0x55db08c768b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:14.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.604+0000 7f750f41e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db08c79b30 0x55db08c76fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:14.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.604+0000 7f750f41e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55db08c7c960 con 0x55db08c6f460 2024-04-24T09:05:14.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.604+0000 7f750f41e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55db08b8d810 con 0x55db08c79b30 2024-04-24T09:05:14.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.604+0000 7f750f41e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55db08c7cae0 con 0x55db08c6eb40 2024-04-24T09:05:14.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.604+0000 7f750f41e740 10 client.0 _ll_register_callbacks cb 0x55db08b5b140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:05:14.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.604+0000 7f7509200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db08c6f460 0x55db08c768b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:14.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.604+0000 7f7509200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db08c6f460 0x55db08c768b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36144/0 (socket says 172.21.15.83:36144) 2024-04-24T09:05:14.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.604+0000 7f7509200640 1 -- 172.21.15.83:0/1317702109 learned_addr learned my addr 172.21.15.83:0/1317702109 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:05:14.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.604+0000 7f7509c00640 1 --2- 172.21.15.83:0/1317702109 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55db08c6eb40 0x55db08c795c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:14.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.604+0000 7f750a600640 1 --2- 172.21.15.83:0/1317702109 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db08c79b30 0x55db08c76fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:14.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.604+0000 7f7509200640 1 -- 172.21.15.83:0/1317702109 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55db08c6eb40 msgr2=0x55db08c795c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:14.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.604+0000 7f7509200640 1 --2- 172.21.15.83:0/1317702109 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55db08c6eb40 0x55db08c795c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.604+0000 7f7509200640 1 -- 172.21.15.83:0/1317702109 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db08c79b30 msgr2=0x55db08c76fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:14.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.604+0000 7f7509200640 1 --2- 172.21.15.83:0/1317702109 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db08c79b30 0x55db08c76fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.604+0000 7f7509200640 1 -- 172.21.15.83:0/1317702109 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db08b9e3f0 con 0x55db08c6f460 2024-04-24T09:05:14.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.604+0000 7f7509c00640 1 --2- 172.21.15.83:0/1317702109 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55db08c6eb40 0x55db08c795c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:05:14.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.605+0000 7f7509200640 1 --2- 172.21.15.83:0/1317702109 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db08c6f460 0x55db08c768b0 secure :-1 s=READY pgs=2095 cs=0 l=1 rev1=1 crypto rx=0x7f74f0005130 tx=0x7f74f0005160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b7aeaef77c127a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:14.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.605+0000 7f74eb400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:05:14.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.605+0000 7f74eb400640 1 -- 172.21.15.83:0/1317702109 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74f0041030 con 0x55db08c6f460 2024-04-24T09:05:14.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.605+0000 7f74eb400640 1 -- 172.21.15.83:0/1317702109 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7f74f00314a0 con 0x55db08c6f460 2024-04-24T09:05:14.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.605+0000 7f750f41e740 1 -- 172.21.15.83:0/1317702109 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74f0004490 con 0x55db08c6f460 2024-04-24T09:05:14.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.605+0000 7f74eb400640 1 -- 172.21.15.83:0/1317702109 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f74f0056050 con 0x55db08c6f460 2024-04-24T09:05:14.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.605+0000 7f750f41e740 10 client.20448 fetch_fsmap learned FSMap version 43 2024-04-24T09:05:14.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.605+0000 7f750f41e740 1 -- 172.21.15.83:0/1317702109 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55db088e63f0 con 0x55db08c6f460 2024-04-24T09:05:14.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.606+0000 7f74eb400640 1 -- 172.21.15.83:0/1317702109 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f74f0004490 con 0x55db08c6f460 2024-04-24T09:05:14.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.606+0000 7f74eb400640 1 -- 172.21.15.83:0/1317702109 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7f74f003d6b0 con 0x55db08c6f460 2024-04-24T09:05:14.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.606+0000 7f750f41e740 10 client.20448 fetch_fsmap finished waiting for FSMap version 43 2024-04-24T09:05:14.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.606+0000 7f750f41e740 10 client.20448 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-24T09:05:14.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.606+0000 7f750f41e740 20 client.20448 populate_metadata read hostname 'smithi083' 2024-04-24T09:05:14.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.606+0000 7f750f41e740 1 --2- 172.21.15.83:0/1317702109 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55db08c89b10 0x55db08c8bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:14.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.606+0000 7f750f41e740 4 client.20448 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:05:14.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.606+0000 7f750f41e740 1 -- 172.21.15.83:0/1317702109 --> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55db08be1bf0 con 0x55db08c89b10 2024-04-24T09:05:14.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.607+0000 7f7509c00640 1 --2- 172.21.15.83:0/1317702109 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55db08c89b10 0x55db08c8bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:14.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.608+0000 7f7509c00640 1 --2- 172.21.15.83:0/1317702109 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55db08c89b10 0x55db08c8bf00 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a1c02482143bc6af server_cookie=ce2e202b33ceb168 in_seq=0 out_seq=0 2024-04-24T09:05:14.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.608+0000 7f74eb400640 10 client.20448 ms_handle_connect on v2:172.21.15.138:6834/1909219758 2024-04-24T09:05:14.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.610+0000 7f74eb400640 1 -- 172.21.15.83:0/1317702109 <== mds.2 v2:172.21.15.138:6834/1909219758 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55db08be1bf0 con 0x55db08c89b10 2024-04-24T09:05:14.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.610+0000 7f74eb400640 10 client.20448 handle_command_reply: tid=0 2024-04-24T09:05:14.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.610+0000 7f750f41e740 1 client.20448 shutdown 2024-04-24T09:05:14.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.610+0000 7f750f41e740 1 -- 172.21.15.83:0/1317702109 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db08c6f460 msgr2=0x55db08c768b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:14.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.610+0000 7f750f41e740 1 --2- 172.21.15.83:0/1317702109 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db08c6f460 0x55db08c768b0 secure :-1 s=READY pgs=2095 cs=0 l=1 rev1=1 crypto rx=0x7f74f0005130 tx=0x7f74f0005160 comp rx=0 tx=0).stop 2024-04-24T09:05:14.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.610+0000 7f750f41e740 1 -- 172.21.15.83:0/1317702109 shutdown_connections 2024-04-24T09:05:14.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.610+0000 7f750f41e740 1 --2- 172.21.15.83:0/1317702109 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55db08c89b10 0x55db08c8bf00 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.610+0000 7f750f41e740 1 --2- 172.21.15.83:0/1317702109 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db08c79b30 0x55db08c76fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.610+0000 7f74eb400640 0 client.20448 ms_handle_reset on v2:172.21.15.138:6834/1909219758 2024-04-24T09:05:14.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.610+0000 7f750f41e740 1 --2- 172.21.15.83:0/1317702109 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db08c6f460 0x55db08c768b0 unknown :-1 s=CLOSED pgs=2095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.610+0000 7f750f41e740 1 --2- 172.21.15.83:0/1317702109 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55db08c6eb40 0x55db08c795c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.610+0000 7f750f41e740 1 -- 172.21.15.83:0/1317702109 >> 172.21.15.83:0/1317702109 conn(0x55db08c6ae60 msgr2=0x55db08c70b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:14.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.610+0000 7f750f41e740 1 -- 172.21.15.83:0/1317702109 shutdown_connections 2024-04-24T09:05:14.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.610+0000 7f750f41e740 1 -- 172.21.15.83:0/1317702109 wait complete. 2024-04-24T09:05:14.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.610+0000 7f750f41e740 20 client.20448 trim_cache size 0 max 16384 2024-04-24T09:05:14.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.625+0000 7f750f41e740 1 Processor -- start 2024-04-24T09:05:14.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.625+0000 7f750f41e740 1 -- start start 2024-04-24T09:05:14.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.625+0000 7f750f41e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db08c6eb40 0x55db08c71140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:14.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.625+0000 7f750f41e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55db08c6f460 0x55db08c790f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:14.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.625+0000 7f750f41e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db08c79b30 0x55db08c914a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:14.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.625+0000 7f750f41e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f75041b14b0 con 0x55db08c79b30 2024-04-24T09:05:14.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.625+0000 7f750f41e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f75041b30b0 con 0x55db08c6eb40 2024-04-24T09:05:14.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.625+0000 7f750f41e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f75041b1b80 con 0x55db08c6f460 2024-04-24T09:05:14.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.626+0000 7f750a600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db08c79b30 0x55db08c914a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:14.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.626+0000 7f750a600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db08c79b30 0x55db08c914a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36150/0 (socket says 172.21.15.83:36150) 2024-04-24T09:05:14.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.626+0000 7f750a600640 1 -- 172.21.15.83:0/3876531406 learned_addr learned my addr 172.21.15.83:0/3876531406 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:05:14.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.626+0000 7f7509c00640 1 --2- 172.21.15.83:0/3876531406 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db08c6eb40 0x55db08c71140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:14.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.626+0000 7f7509200640 1 --2- 172.21.15.83:0/3876531406 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55db08c6f460 0x55db08c790f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:14.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.626+0000 7f750a600640 1 -- 172.21.15.83:0/3876531406 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55db08c6f460 msgr2=0x55db08c790f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:14.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.626+0000 7f750a600640 1 --2- 172.21.15.83:0/3876531406 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55db08c6f460 0x55db08c790f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.626+0000 7f750a600640 1 -- 172.21.15.83:0/3876531406 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db08c6eb40 msgr2=0x55db08c71140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:14.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.626+0000 7f750a600640 1 --2- 172.21.15.83:0/3876531406 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db08c6eb40 0x55db08c71140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.626+0000 7f750a600640 1 -- 172.21.15.83:0/3876531406 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db089b07d0 con 0x55db08c79b30 2024-04-24T09:05:14.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.627+0000 7f750a600640 1 --2- 172.21.15.83:0/3876531406 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db08c79b30 0x55db08c914a0 secure :-1 s=READY pgs=2096 cs=0 l=1 rev1=1 crypto rx=0x7f74fc099600 tx=0x7f74fc00fd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=335862cd921c6fde server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:14.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.627+0000 7f74f6000640 1 -- 172.21.15.83:0/3876531406 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74fc0b6070 con 0x55db08c79b30 2024-04-24T09:05:14.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.627+0000 7f74f6000640 1 -- 172.21.15.83:0/3876531406 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f74fc0a0fe0 con 0x55db08c79b30 2024-04-24T09:05:14.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.627+0000 7f74f6000640 1 -- 172.21.15.83:0/3876531406 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74fc087040 con 0x55db08c79b30 2024-04-24T09:05:14.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.628+0000 7f750f41e740 1 -- 172.21.15.83:0/3876531406 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db08c79b30 msgr2=0x55db08c914a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:14.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.628+0000 7f750f41e740 1 --2- 172.21.15.83:0/3876531406 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db08c79b30 0x55db08c914a0 secure :-1 s=READY pgs=2096 cs=0 l=1 rev1=1 crypto rx=0x7f74fc099600 tx=0x7f74fc00fd10 comp rx=0 tx=0).stop 2024-04-24T09:05:14.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.628+0000 7f750f41e740 1 -- 172.21.15.83:0/3876531406 shutdown_connections 2024-04-24T09:05:14.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.628+0000 7f750f41e740 1 --2- 172.21.15.83:0/3876531406 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db08c79b30 0x55db08c914a0 unknown :-1 s=CLOSED pgs=2096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.628+0000 7f750f41e740 1 --2- 172.21.15.83:0/3876531406 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55db08c6f460 0x55db08c790f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.628+0000 7f750f41e740 1 --2- 172.21.15.83:0/3876531406 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db08c6eb40 0x55db08c71140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.628+0000 7f750f41e740 1 -- 172.21.15.83:0/3876531406 >> 172.21.15.83:0/3876531406 conn(0x55db08c6ae60 msgr2=0x55db08c71bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:14.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.628+0000 7f750f41e740 1 -- 172.21.15.83:0/3876531406 shutdown_connections 2024-04-24T09:05:14.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.628+0000 7f750f41e740 1 -- 172.21.15.83:0/3876531406 wait complete. 2024-04-24T09:05:14.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.628+0000 7f750f41e740 1 -- start start 2024-04-24T09:05:14.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.628+0000 7f750f41e740 1 Processor -- start 2024-04-24T09:05:14.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.629+0000 7f750f41e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db08c6eb40 0x55db08c82280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:14.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.629+0000 7f750f41e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55db08c6f460 0x55db08c74030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:14.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.629+0000 7f750f41e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db08c79b30 0x55db08c7ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:14.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.629+0000 7f750f41e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f75041b2980 con 0x55db08c79b30 2024-04-24T09:05:14.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.629+0000 7f750f41e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f75041b2250 con 0x55db08c6eb40 2024-04-24T09:05:14.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.629+0000 7f750f41e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f75041af4b0 con 0x55db08c6f460 2024-04-24T09:05:14.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.629+0000 7f750f41e740 10 client.0 _ll_register_callbacks cb 0x7f74f0004260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:05:14.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.629+0000 7f750a600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db08c79b30 0x55db08c7ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:14.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.629+0000 7f750a600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db08c79b30 0x55db08c7ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36152/0 (socket says 172.21.15.83:36152) 2024-04-24T09:05:14.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.629+0000 7f750a600640 1 -- 172.21.15.83:0/1638485349 learned_addr learned my addr 172.21.15.83:0/1638485349 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:05:14.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.629+0000 7f7509c00640 1 --2- 172.21.15.83:0/1638485349 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db08c6eb40 0x55db08c82280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:14.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.630+0000 7f750a600640 1 -- 172.21.15.83:0/1638485349 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55db08c6f460 msgr2=0x55db08c74030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:14.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.630+0000 7f7509200640 1 --2- 172.21.15.83:0/1638485349 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55db08c6f460 0x55db08c74030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:14.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.630+0000 7f750a600640 1 --2- 172.21.15.83:0/1638485349 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55db08c6f460 0x55db08c74030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.630+0000 7f750a600640 1 -- 172.21.15.83:0/1638485349 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db08c6eb40 msgr2=0x55db08c82280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:14.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.630+0000 7f750a600640 1 --2- 172.21.15.83:0/1638485349 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db08c6eb40 0x55db08c82280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.630+0000 7f750a600640 1 -- 172.21.15.83:0/1638485349 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db08c7b160 con 0x55db08c79b30 2024-04-24T09:05:14.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.630+0000 7f7509c00640 1 --2- 172.21.15.83:0/1638485349 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db08c6eb40 0x55db08c82280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:05:14.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.630+0000 7f750a600640 1 --2- 172.21.15.83:0/1638485349 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db08c79b30 0x55db08c7ac20 secure :-1 s=READY pgs=2097 cs=0 l=1 rev1=1 crypto rx=0x7f74fc0078b0 tx=0x7f74fc0ce380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93461fd3f49111e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:14.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.630+0000 7f74eb400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:05:14.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.630+0000 7f74eb400640 1 -- 172.21.15.83:0/1638485349 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74fc087030 con 0x55db08c79b30 2024-04-24T09:05:14.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.630+0000 7f74eb400640 1 -- 172.21.15.83:0/1638485349 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7f74fc0bd050 con 0x55db08c79b30 2024-04-24T09:05:14.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.630+0000 7f750f41e740 1 -- 172.21.15.83:0/1638485349 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74fc0a0fe0 con 0x55db08c79b30 2024-04-24T09:05:14.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.630+0000 7f74eb400640 1 -- 172.21.15.83:0/1638485349 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f74fc0b6070 con 0x55db08c79b30 2024-04-24T09:05:14.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.630+0000 7f750f41e740 10 client.20454 fetch_fsmap learned FSMap version 43 2024-04-24T09:05:14.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.630+0000 7f750f41e740 1 -- 172.21.15.83:0/1638485349 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55db08cf8b10 con 0x55db08c79b30 2024-04-24T09:05:14.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.631+0000 7f74eb400640 1 -- 172.21.15.83:0/1638485349 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f74fc0a0fe0 con 0x55db08c79b30 2024-04-24T09:05:14.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.631+0000 7f74eb400640 1 -- 172.21.15.83:0/1638485349 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7f74fc0a54a0 con 0x55db08c79b30 2024-04-24T09:05:14.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.631+0000 7f750f41e740 10 client.20454 fetch_fsmap finished waiting for FSMap version 43 2024-04-24T09:05:14.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.631+0000 7f750f41e740 10 client.20454 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-24T09:05:14.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.631+0000 7f750f41e740 20 client.20454 populate_metadata read hostname 'smithi083' 2024-04-24T09:05:14.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.631+0000 7f750f41e740 1 --2- 172.21.15.83:0/1638485349 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55db08cf9120 0x55db08cfb510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:14.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.631+0000 7f750f41e740 4 client.20454 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:05:14.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.631+0000 7f750f41e740 1 -- 172.21.15.83:0/1638485349 --> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55db08bdba60 con 0x55db08cf9120 2024-04-24T09:05:14.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.632+0000 7f7509c00640 1 --2- 172.21.15.83:0/1638485349 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55db08cf9120 0x55db08cfb510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:14.634 INFO:tasks.workunit.client.0.smithi083.stdout: 1 78226 0.58 MB/sec execute 588 sec latency 862.588 ms 2024-04-24T09:05:14.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.633+0000 7f7509c00640 1 --2- 172.21.15.83:0/1638485349 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55db08cf9120 0x55db08cfb510 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e3c5562052210105 server_cookie=60d3fd2ff7d829cb in_seq=0 out_seq=0 2024-04-24T09:05:14.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.633+0000 7f74eb400640 10 client.20454 ms_handle_connect on v2:172.21.15.138:6834/1909219758 2024-04-24T09:05:14.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.633+0000 7f74eb400640 1 -- 172.21.15.83:0/1638485349 <== mds.2 v2:172.21.15.138:6834/1909219758 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55db08bdba60 con 0x55db08cf9120 2024-04-24T09:05:14.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.633+0000 7f74eb400640 10 client.20454 handle_command_reply: tid=0 2024-04-24T09:05:14.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.633+0000 7f750f41e740 1 client.20454 shutdown 2024-04-24T09:05:14.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.633+0000 7f750f41e740 1 -- 172.21.15.83:0/1638485349 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db08c79b30 msgr2=0x55db08c7ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:14.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.633+0000 7f750f41e740 1 --2- 172.21.15.83:0/1638485349 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db08c79b30 0x55db08c7ac20 secure :-1 s=READY pgs=2097 cs=0 l=1 rev1=1 crypto rx=0x7f74fc0078b0 tx=0x7f74fc0ce380 comp rx=0 tx=0).stop 2024-04-24T09:05:14.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.634+0000 7f750f41e740 1 -- 172.21.15.83:0/1638485349 shutdown_connections 2024-04-24T09:05:14.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.634+0000 7f750f41e740 1 --2- 172.21.15.83:0/1638485349 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55db08cf9120 0x55db08cfb510 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.634+0000 7f750f41e740 1 --2- 172.21.15.83:0/1638485349 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db08c79b30 0x55db08c7ac20 unknown :-1 s=CLOSED pgs=2097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.634+0000 7f750f41e740 1 --2- 172.21.15.83:0/1638485349 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55db08c6f460 0x55db08c74030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.634+0000 7f750f41e740 1 --2- 172.21.15.83:0/1638485349 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db08c6eb40 0x55db08c82280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.634+0000 7f74eb400640 0 client.20454 ms_handle_reset on v2:172.21.15.138:6834/1909219758 2024-04-24T09:05:14.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.634+0000 7f750f41e740 1 -- 172.21.15.83:0/1638485349 >> 172.21.15.83:0/1638485349 conn(0x55db08c6ae60 msgr2=0x55db08c696c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:14.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.634+0000 7f750f41e740 1 -- 172.21.15.83:0/1638485349 shutdown_connections 2024-04-24T09:05:14.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.634+0000 7f750f41e740 1 -- 172.21.15.83:0/1638485349 wait complete. 2024-04-24T09:05:14.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.634+0000 7f750f41e740 20 client.20454 trim_cache size 0 max 16384 2024-04-24T09:05:14.637 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:05:14.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.634+0000 7f750b000640 1 -- 172.21.15.83:0/1336413957 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f74ec0a3d20 msgr2=0x7f74ec0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:14.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.634+0000 7f750b000640 1 --2- 172.21.15.83:0/1336413957 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f74ec0a3d20 0x7f74ec0a61e0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x55db08c70100 tx=0x7f74f8005f50 comp rx=0 tx=0).stop 2024-04-24T09:05:14.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.634+0000 7f750b000640 1 -- 172.21.15.83:0/1336413957 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f750419a720 msgr2=0x7f750419aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:14.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.634+0000 7f750b000640 1 --2- 172.21.15.83:0/1336413957 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f750419a720 0x7f750419aba0 secure :-1 s=READY pgs=2093 cs=0 l=1 rev1=1 crypto rx=0x7f74fc00cce0 tx=0x7f74fc00a620 comp rx=0 tx=0).stop 2024-04-24T09:05:14.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.635+0000 7f750b000640 1 -- 172.21.15.83:0/1336413957 shutdown_connections 2024-04-24T09:05:14.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.635+0000 7f750b000640 1 --2- 172.21.15.83:0/1336413957 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f74ec0a3d20 0x7f74ec0a61e0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.635+0000 7f750b000640 1 --2- 172.21.15.83:0/1336413957 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f750419a720 0x7f750419aba0 unknown :-1 s=CLOSED pgs=2093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.635+0000 7f750b000640 1 --2- 172.21.15.83:0/1336413957 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f75041071f0 0x7f75041a0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.635+0000 7f750b000640 1 --2- 172.21.15.83:0/1336413957 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7504102120 0x7f75041a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.635+0000 7f750b000640 1 -- 172.21.15.83:0/1336413957 >> 172.21.15.83:0/1336413957 conn(0x7f75040fd9a0 msgr2=0x7f75040fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:14.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.635+0000 7f750b000640 1 -- 172.21.15.83:0/1336413957 shutdown_connections 2024-04-24T09:05:14.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.635+0000 7f750b000640 1 -- 172.21.15.83:0/1336413957 wait complete. 2024-04-24T09:05:14.652 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T09:05:14.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.847+0000 7f25f0c00640 1 -- 172.21.15.83:0/424186226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f25ec108f40 msgr2=0x7f25ec1153b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:14.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.847+0000 7f25f0c00640 1 --2- 172.21.15.83:0/424186226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f25ec108f40 0x7f25ec1153b0 secure :-1 s=READY pgs=2098 cs=0 l=1 rev1=1 crypto rx=0x7f25e4009cd0 tx=0x7f25e402a8f0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.847+0000 7f25f0c00640 1 -- 172.21.15.83:0/424186226 shutdown_connections 2024-04-24T09:05:14.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.847+0000 7f25f0c00640 1 --2- 172.21.15.83:0/424186226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f25ec108f40 0x7f25ec1153b0 unknown :-1 s=CLOSED pgs=2098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.847+0000 7f25f0c00640 1 --2- 172.21.15.83:0/424186226 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f25ec108580 0x7f25ec108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.847+0000 7f25f0c00640 1 --2- 172.21.15.83:0/424186226 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25ec10d500 0x7f25ec10d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.847+0000 7f25f0c00640 1 -- 172.21.15.83:0/424186226 >> 172.21.15.83:0/424186226 conn(0x7f25ec077e20 msgr2=0x7f25ec106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:14.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.847+0000 7f25f0c00640 1 -- 172.21.15.83:0/424186226 shutdown_connections 2024-04-24T09:05:14.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.847+0000 7f25f0c00640 1 -- 172.21.15.83:0/424186226 wait complete. 2024-04-24T09:05:14.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.848+0000 7f25f0c00640 1 Processor -- start 2024-04-24T09:05:14.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.848+0000 7f25f0c00640 1 -- start start 2024-04-24T09:05:14.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.848+0000 7f25f0c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f25ec108580 0x7f25ec1a4830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:14.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.848+0000 7f25f0c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25ec108f40 0x7f25ec1a4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:14.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.848+0000 7f25f0c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f25ec10d500 0x7f25ec19ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:14.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.848+0000 7f25f0c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f25ec1187b0 con 0x7f25ec10d500 2024-04-24T09:05:14.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.848+0000 7f25f0c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f25ec118630 con 0x7f25ec108f40 2024-04-24T09:05:14.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.848+0000 7f25f0c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f25ec118930 con 0x7f25ec108580 2024-04-24T09:05:14.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.848+0000 7f25ebe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f25ec10d500 0x7f25ec19ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:14.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.849+0000 7f25ebe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f25ec10d500 0x7f25ec19ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36164/0 (socket says 172.21.15.83:36164) 2024-04-24T09:05:14.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.849+0000 7f25ebe00640 1 -- 172.21.15.83:0/979713259 learned_addr learned my addr 172.21.15.83:0/979713259 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:05:14.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.849+0000 7f25eaa00640 1 --2- 172.21.15.83:0/979713259 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25ec108f40 0x7f25ec1a4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:14.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.849+0000 7f25eb400640 1 --2- 172.21.15.83:0/979713259 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f25ec108580 0x7f25ec1a4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:14.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.849+0000 7f25ebe00640 1 -- 172.21.15.83:0/979713259 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f25ec108580 msgr2=0x7f25ec1a4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:14.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.849+0000 7f25ebe00640 1 --2- 172.21.15.83:0/979713259 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f25ec108580 0x7f25ec1a4830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.849+0000 7f25ebe00640 1 -- 172.21.15.83:0/979713259 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25ec108f40 msgr2=0x7f25ec1a4d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:14.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.849+0000 7f25ebe00640 1 --2- 172.21.15.83:0/979713259 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25ec108f40 0x7f25ec1a4d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.849+0000 7f25ebe00640 1 -- 172.21.15.83:0/979713259 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25ec19f1b0 con 0x7f25ec10d500 2024-04-24T09:05:14.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.849+0000 7f25eb400640 1 --2- 172.21.15.83:0/979713259 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f25ec108580 0x7f25ec1a4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:05:14.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.849+0000 7f25eaa00640 1 --2- 172.21.15.83:0/979713259 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25ec108f40 0x7f25ec1a4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:05:14.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.849+0000 7f25ebe00640 1 --2- 172.21.15.83:0/979713259 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f25ec10d500 0x7f25ec19ea30 secure :-1 s=READY pgs=2099 cs=0 l=1 rev1=1 crypto rx=0x7f25e402ae10 tx=0x7f25e4003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:14.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.849+0000 7f25e3e00640 1 -- 172.21.15.83:0/979713259 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25e4004240 con 0x7f25ec10d500 2024-04-24T09:05:14.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.850+0000 7f25e3e00640 1 -- 172.21.15.83:0/979713259 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f25e4040070 con 0x7f25ec10d500 2024-04-24T09:05:14.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.850+0000 7f25e3e00640 1 -- 172.21.15.83:0/979713259 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25e4031d10 con 0x7f25ec10d500 2024-04-24T09:05:14.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.850+0000 7f25f0c00640 1 -- 172.21.15.83:0/979713259 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25ec19f440 con 0x7f25ec10d500 2024-04-24T09:05:14.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.850+0000 7f25f0c00640 1 -- 172.21.15.83:0/979713259 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25ec1ab6f0 con 0x7f25ec10d500 2024-04-24T09:05:14.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.853+0000 7f25f51db740 1 Processor -- start 2024-04-24T09:05:14.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.853+0000 7f25e3e00640 1 -- 172.21.15.83:0/979713259 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f25e4031440 con 0x7f25ec10d500 2024-04-24T09:05:14.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.853+0000 7f25f51db740 1 -- start start 2024-04-24T09:05:14.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.853+0000 7f25f51db740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5641ecc52b40 0x5641ecc52f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:14.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.853+0000 7f25f51db740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641ecc53460 0x5641ecc5d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:14.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.853+0000 7f25f51db740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641ecc5db30 0x5641ecc5ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:14.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.854+0000 7f25f51db740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5641ecbb81d0 con 0x5641ecc5db30 2024-04-24T09:05:14.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.854+0000 7f25f51db740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5641ecb5e450 con 0x5641ecc53460 2024-04-24T09:05:14.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.854+0000 7f25f51db740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5641ecbb4190 con 0x5641ecc52b40 2024-04-24T09:05:14.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.854+0000 7f25eaa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641ecc5db30 0x5641ecc5ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:14.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.854+0000 7f25eaa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641ecc5db30 0x5641ecc5ff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36166/0 (socket says 172.21.15.83:36166) 2024-04-24T09:05:14.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.854+0000 7f25eaa00640 1 -- 172.21.15.83:0/2905537898 learned_addr learned my addr 172.21.15.83:0/2905537898 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:05:14.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.854+0000 7f25eb400640 1 --2- 172.21.15.83:0/2905537898 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641ecc53460 0x5641ecc5d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:14.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.854+0000 7f25eaa00640 1 --2- 172.21.15.83:0/2905537898 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5641ecc52b40 0x5641ecc52f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:14.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.854+0000 7f25eaa00640 1 -- 172.21.15.83:0/2905537898 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5641ecc52b40 msgr2=0x5641ecc52f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:14.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.854+0000 7f25eaa00640 1 --2- 172.21.15.83:0/2905537898 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5641ecc52b40 0x5641ecc52f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.854+0000 7f25eaa00640 1 -- 172.21.15.83:0/2905537898 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641ecc53460 msgr2=0x5641ecc5d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:14.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.854+0000 7f25eaa00640 1 --2- 172.21.15.83:0/2905537898 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641ecc53460 0x5641ecc5d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.854+0000 7f25eaa00640 1 -- 172.21.15.83:0/2905537898 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641ecc1b310 con 0x5641ecc5db30 2024-04-24T09:05:14.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.855+0000 7f25eaa00640 1 --2- 172.21.15.83:0/2905537898 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641ecc5db30 0x5641ecc5ff20 secure :-1 s=READY pgs=2100 cs=0 l=1 rev1=1 crypto rx=0x7f25d800dfc0 tx=0x7f25d800d600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7615138b6b19ca31 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:14.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.855+0000 7f25e3e00640 1 --2- 172.21.15.83:0/979713259 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f25d00a3d20 0x7f25d00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:14.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.855+0000 7f25d7e00640 1 -- 172.21.15.83:0/2905537898 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d8005220 con 0x5641ecc5db30 2024-04-24T09:05:14.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.855+0000 7f25d7e00640 1 -- 172.21.15.83:0/2905537898 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f25d8004c50 con 0x5641ecc5db30 2024-04-24T09:05:14.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.855+0000 7f25d7e00640 1 -- 172.21.15.83:0/2905537898 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d8012890 con 0x5641ecc5db30 2024-04-24T09:05:14.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.855+0000 7f25e3e00640 1 -- 172.21.15.83:0/979713259 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7f25e40f2070 con 0x7f25ec10d500 2024-04-24T09:05:14.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.855+0000 7f25eb400640 1 --2- 172.21.15.83:0/979713259 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f25d00a3d20 0x7f25d00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:14.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.856+0000 7f25f51db740 1 -- 172.21.15.83:0/2905537898 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641ecc5db30 msgr2=0x5641ecc5ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:14.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.856+0000 7f25f51db740 1 --2- 172.21.15.83:0/2905537898 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641ecc5db30 0x5641ecc5ff20 secure :-1 s=READY pgs=2100 cs=0 l=1 rev1=1 crypto rx=0x7f25d800dfc0 tx=0x7f25d800d600 comp rx=0 tx=0).stop 2024-04-24T09:05:14.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.856+0000 7f25f51db740 1 -- 172.21.15.83:0/2905537898 shutdown_connections 2024-04-24T09:05:14.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.856+0000 7f25f51db740 1 --2- 172.21.15.83:0/2905537898 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641ecc5db30 0x5641ecc5ff20 unknown :-1 s=CLOSED pgs=2100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.856+0000 7f25f51db740 1 --2- 172.21.15.83:0/2905537898 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641ecc53460 0x5641ecc5d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.856+0000 7f25f51db740 1 --2- 172.21.15.83:0/2905537898 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5641ecc52b40 0x5641ecc52f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.856+0000 7f25f51db740 1 -- 172.21.15.83:0/2905537898 >> 172.21.15.83:0/2905537898 conn(0x5641ecc4ee60 msgr2=0x5641ecc4f240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:14.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.856+0000 7f25eb400640 1 --2- 172.21.15.83:0/979713259 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f25d00a3d20 0x7f25d00a61e0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f25dc005fd0 tx=0x7f25dc005dd0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:14.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.856+0000 7f25f51db740 1 -- 172.21.15.83:0/2905537898 shutdown_connections 2024-04-24T09:05:14.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.856+0000 7f25f51db740 1 -- 172.21.15.83:0/2905537898 wait complete. 2024-04-24T09:05:14.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.856+0000 7f25f51db740 1 -- start start 2024-04-24T09:05:14.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.857+0000 7f25f51db740 1 Processor -- start 2024-04-24T09:05:14.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.857+0000 7f25f51db740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641ecc52b40 0x5641ecc5d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:14.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.857+0000 7f25f51db740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641ecc53460 0x5641ecc5a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:14.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.857+0000 7f25f51db740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5641ecc5db30 0x5641ecc5af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:14.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.857+0000 7f25f51db740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5641ecc609c0 con 0x5641ecc52b40 2024-04-24T09:05:14.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.857+0000 7f25f51db740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5641ecb71810 con 0x5641ecc53460 2024-04-24T09:05:14.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.857+0000 7f25f51db740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5641ecc60b40 con 0x5641ecc5db30 2024-04-24T09:05:14.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.857+0000 7f25f51db740 10 client.0 _ll_register_callbacks cb 0x5641ecb3f140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:05:14.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.857+0000 7f25eb400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641ecc52b40 0x5641ecc5d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:14.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.857+0000 7f25eb400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641ecc52b40 0x5641ecc5d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36180/0 (socket says 172.21.15.83:36180) 2024-04-24T09:05:14.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.857+0000 7f25eb400640 1 -- 172.21.15.83:0/1027537506 learned_addr learned my addr 172.21.15.83:0/1027537506 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:05:14.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.858+0000 7f25eb400640 1 -- 172.21.15.83:0/1027537506 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5641ecc5db30 msgr2=0x5641ecc5af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:14.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.858+0000 7f25eaa00640 1 --2- 172.21.15.83:0/1027537506 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641ecc53460 0x5641ecc5a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:14.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.858+0000 7f25eb400640 1 --2- 172.21.15.83:0/1027537506 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5641ecc5db30 0x5641ecc5af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.858+0000 7f25eb400640 1 -- 172.21.15.83:0/1027537506 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641ecc53460 msgr2=0x5641ecc5a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:14.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.858+0000 7f25eb400640 1 --2- 172.21.15.83:0/1027537506 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641ecc53460 0x5641ecc5a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.858+0000 7f25eb400640 1 -- 172.21.15.83:0/1027537506 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641ecb823f0 con 0x5641ecc52b40 2024-04-24T09:05:14.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.858+0000 7f25eb400640 1 --2- 172.21.15.83:0/1027537506 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641ecc52b40 0x5641ecc5d560 secure :-1 s=READY pgs=2101 cs=0 l=1 rev1=1 crypto rx=0x7f25dc010a90 tx=0x7f25dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58f21ccf1abba2ab server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:14.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.858+0000 7f25d4c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:05:14.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.858+0000 7f25d4c00640 1 -- 172.21.15.83:0/1027537506 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25dc00de40 con 0x5641ecc52b40 2024-04-24T09:05:14.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.858+0000 7f25d4c00640 1 -- 172.21.15.83:0/1027537506 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7f25dc01d420 con 0x5641ecc52b40 2024-04-24T09:05:14.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.859+0000 7f25f51db740 1 -- 172.21.15.83:0/1027537506 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25d8004c50 con 0x5641ecc52b40 2024-04-24T09:05:14.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.859+0000 7f25d4c00640 1 -- 172.21.15.83:0/1027537506 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f25dc01d620 con 0x5641ecc52b40 2024-04-24T09:05:14.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.859+0000 7f25f51db740 10 client.20466 fetch_fsmap learned FSMap version 43 2024-04-24T09:05:14.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.859+0000 7f25f51db740 1 -- 172.21.15.83:0/1027537506 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5641ec8ca3f0 con 0x5641ecc52b40 2024-04-24T09:05:14.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.859+0000 7f25d4c00640 1 -- 172.21.15.83:0/1027537506 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f25d8004c50 con 0x5641ecc52b40 2024-04-24T09:05:14.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.859+0000 7f25d4c00640 1 -- 172.21.15.83:0/1027537506 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7f25dc0257a0 con 0x5641ecc52b40 2024-04-24T09:05:14.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.860+0000 7f25f51db740 10 client.20466 fetch_fsmap finished waiting for FSMap version 43 2024-04-24T09:05:14.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.860+0000 7f25f51db740 10 client.20466 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-24T09:05:14.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.860+0000 7f25f51db740 20 client.20466 populate_metadata read hostname 'smithi083' 2024-04-24T09:05:14.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.860+0000 7f25f51db740 1 --2- 172.21.15.83:0/1027537506 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x5641ecc6dde0 0x5641ecc702a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:14.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.860+0000 7f25f51db740 4 client.20466 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:05:14.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.860+0000 7f25f51db740 1 -- 172.21.15.83:0/1027537506 --> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5641ecbc5bf0 con 0x5641ecc6dde0 2024-04-24T09:05:14.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.860+0000 7f25eaa00640 1 --2- 172.21.15.83:0/1027537506 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x5641ecc6dde0 0x5641ecc702a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:14.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.861+0000 7f25eaa00640 1 --2- 172.21.15.83:0/1027537506 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x5641ecc6dde0 0x5641ecc702a0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7cc57fd7a18444ab server_cookie=64fa4f6146fd93c in_seq=0 out_seq=0 2024-04-24T09:05:14.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.861+0000 7f25d4c00640 10 client.20466 ms_handle_connect on v2:172.21.15.138:6836/900566754 2024-04-24T09:05:14.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.863+0000 7f25d4c00640 1 -- 172.21.15.83:0/1027537506 <== mds.1 v2:172.21.15.138:6836/900566754 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5641ecbc5bf0 con 0x5641ecc6dde0 2024-04-24T09:05:14.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.863+0000 7f25d4c00640 10 client.20466 handle_command_reply: tid=0 2024-04-24T09:05:14.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.864+0000 7f25f51db740 1 client.20466 shutdown 2024-04-24T09:05:14.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.864+0000 7f25f51db740 1 -- 172.21.15.83:0/1027537506 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641ecc52b40 msgr2=0x5641ecc5d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:14.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.864+0000 7f25f51db740 1 --2- 172.21.15.83:0/1027537506 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641ecc52b40 0x5641ecc5d560 secure :-1 s=READY pgs=2101 cs=0 l=1 rev1=1 crypto rx=0x7f25dc010a90 tx=0x7f25dc0046c0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.864+0000 7f25f51db740 1 -- 172.21.15.83:0/1027537506 shutdown_connections 2024-04-24T09:05:14.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.864+0000 7f25f51db740 1 --2- 172.21.15.83:0/1027537506 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x5641ecc6dde0 0x5641ecc702a0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.864+0000 7f25f51db740 1 --2- 172.21.15.83:0/1027537506 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5641ecc5db30 0x5641ecc5af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.864+0000 7f25f51db740 1 --2- 172.21.15.83:0/1027537506 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641ecc53460 0x5641ecc5a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.864+0000 7f25f51db740 1 --2- 172.21.15.83:0/1027537506 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641ecc52b40 0x5641ecc5d560 unknown :-1 s=CLOSED pgs=2101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.864+0000 7f25d4c00640 0 client.20466 ms_handle_reset on v2:172.21.15.138:6836/900566754 2024-04-24T09:05:14.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.864+0000 7f25f51db740 1 -- 172.21.15.83:0/1027537506 >> 172.21.15.83:0/1027537506 conn(0x5641ecc4ee60 msgr2=0x5641ecc54c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:14.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.864+0000 7f25f51db740 1 -- 172.21.15.83:0/1027537506 shutdown_connections 2024-04-24T09:05:14.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.864+0000 7f25f51db740 1 -- 172.21.15.83:0/1027537506 wait complete. 2024-04-24T09:05:14.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.864+0000 7f25f51db740 20 client.20466 trim_cache size 0 max 16384 2024-04-24T09:05:14.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.879+0000 7f25f51db740 1 Processor -- start 2024-04-24T09:05:14.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.879+0000 7f25f51db740 1 -- start start 2024-04-24T09:05:14.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.879+0000 7f25f51db740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5641ecc52b40 0x5641ecc4e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:14.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.879+0000 7f25f51db740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641ecc53460 0x5641ecc59830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:14.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.879+0000 7f25f51db740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641ecc5db30 0x5641ecc5d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:14.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.879+0000 7f25f51db740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f25ec1b5940 con 0x5641ecc53460 2024-04-24T09:05:14.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.879+0000 7f25f51db740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f25ec1b7540 con 0x5641ecc5db30 2024-04-24T09:05:14.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.879+0000 7f25f51db740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f25ec1b6010 con 0x5641ecc52b40 2024-04-24T09:05:14.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.879+0000 7f25eaa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641ecc53460 0x5641ecc59830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:14.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.879+0000 7f25eaa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641ecc53460 0x5641ecc59830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36192/0 (socket says 172.21.15.83:36192) 2024-04-24T09:05:14.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.879+0000 7f25eaa00640 1 -- 172.21.15.83:0/413087148 learned_addr learned my addr 172.21.15.83:0/413087148 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:05:14.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.880+0000 7f25eaa00640 1 -- 172.21.15.83:0/413087148 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5641ecc52b40 msgr2=0x5641ecc4e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:14.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.880+0000 7f25eb400640 1 --2- 172.21.15.83:0/413087148 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5641ecc52b40 0x5641ecc4e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:14.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.880+0000 7f25eaa00640 1 --2- 172.21.15.83:0/413087148 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5641ecc52b40 0x5641ecc4e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.880+0000 7f25eaa00640 1 -- 172.21.15.83:0/413087148 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641ecc5db30 msgr2=0x5641ecc5d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:14.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.880+0000 7f25ebe00640 1 --2- 172.21.15.83:0/413087148 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641ecc5db30 0x5641ecc5d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:14.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.880+0000 7f25eaa00640 1 --2- 172.21.15.83:0/413087148 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641ecc5db30 0x5641ecc5d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.880+0000 7f25eaa00640 1 -- 172.21.15.83:0/413087148 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641ec9947d0 con 0x5641ecc53460 2024-04-24T09:05:14.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.880+0000 7f25eaa00640 1 --2- 172.21.15.83:0/413087148 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641ecc53460 0x5641ecc59830 secure :-1 s=READY pgs=2102 cs=0 l=1 rev1=1 crypto rx=0x7f25d80057c0 tx=0x7f25d8005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=699cd62e48f127df server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:14.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.880+0000 7f25d7400640 1 -- 172.21.15.83:0/413087148 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d802d070 con 0x5641ecc53460 2024-04-24T09:05:14.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.880+0000 7f25d7400640 1 -- 172.21.15.83:0/413087148 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f25d8013860 con 0x5641ecc53460 2024-04-24T09:05:14.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.880+0000 7f25d7400640 1 -- 172.21.15.83:0/413087148 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d801d6a0 con 0x5641ecc53460 2024-04-24T09:05:14.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.881+0000 7f25f51db740 1 -- 172.21.15.83:0/413087148 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641ecc53460 msgr2=0x5641ecc59830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:14.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.881+0000 7f25f51db740 1 --2- 172.21.15.83:0/413087148 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641ecc53460 0x5641ecc59830 secure :-1 s=READY pgs=2102 cs=0 l=1 rev1=1 crypto rx=0x7f25d80057c0 tx=0x7f25d8005850 comp rx=0 tx=0).stop 2024-04-24T09:05:14.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.881+0000 7f25f51db740 1 -- 172.21.15.83:0/413087148 shutdown_connections 2024-04-24T09:05:14.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.881+0000 7f25f51db740 1 --2- 172.21.15.83:0/413087148 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641ecc5db30 0x5641ecc5d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.881+0000 7f25f51db740 1 --2- 172.21.15.83:0/413087148 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641ecc53460 0x5641ecc59830 unknown :-1 s=CLOSED pgs=2102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.881+0000 7f25f51db740 1 --2- 172.21.15.83:0/413087148 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5641ecc52b40 0x5641ecc4e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.881+0000 7f25f51db740 1 -- 172.21.15.83:0/413087148 >> 172.21.15.83:0/413087148 conn(0x5641ecc4ee60 msgr2=0x5641ecc59120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:14.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.881+0000 7f25f51db740 1 -- 172.21.15.83:0/413087148 shutdown_connections 2024-04-24T09:05:14.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.881+0000 7f25f51db740 1 -- 172.21.15.83:0/413087148 wait complete. 2024-04-24T09:05:14.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.882+0000 7f25f51db740 1 -- start start 2024-04-24T09:05:14.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.882+0000 7f25f51db740 1 Processor -- start 2024-04-24T09:05:14.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.882+0000 7f25f51db740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641ecc52b40 0x5641ecc5e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:14.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.882+0000 7f25f51db740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641ecc53460 0x5641ecc5e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:14.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.882+0000 7f25f51db740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5641ecc5db30 0x5641ecc64df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:14.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.882+0000 7f25f51db740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f25ec1b6e10 con 0x5641ecc52b40 2024-04-24T09:05:14.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.882+0000 7f25f51db740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f25ec1b66e0 con 0x5641ecc53460 2024-04-24T09:05:14.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.882+0000 7f25f51db740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f25ec1b3940 con 0x5641ecc5db30 2024-04-24T09:05:14.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.882+0000 7f25f51db740 10 client.0 _ll_register_callbacks cb 0x7f25dc00b320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:05:14.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.882+0000 7f25eb400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641ecc52b40 0x5641ecc5e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:14.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.883+0000 7f25eb400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641ecc52b40 0x5641ecc5e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36208/0 (socket says 172.21.15.83:36208) 2024-04-24T09:05:14.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.883+0000 7f25eb400640 1 -- 172.21.15.83:0/1629337680 learned_addr learned my addr 172.21.15.83:0/1629337680 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:05:14.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.883+0000 7f25eb400640 1 -- 172.21.15.83:0/1629337680 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5641ecc5db30 msgr2=0x5641ecc64df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:14.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.883+0000 7f25eb400640 1 --2- 172.21.15.83:0/1629337680 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5641ecc5db30 0x5641ecc64df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.883+0000 7f25eb400640 1 -- 172.21.15.83:0/1629337680 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641ecc53460 msgr2=0x5641ecc5e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:14.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.883+0000 7f25eb400640 1 --2- 172.21.15.83:0/1629337680 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641ecc53460 0x5641ecc5e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.883+0000 7f25eb400640 1 -- 172.21.15.83:0/1629337680 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641ecc5f100 con 0x5641ecc52b40 2024-04-24T09:05:14.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.883+0000 7f25eb400640 1 --2- 172.21.15.83:0/1629337680 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641ecc52b40 0x5641ecc5e2c0 secure :-1 s=READY pgs=2103 cs=0 l=1 rev1=1 crypto rx=0x7f25dc000c00 tx=0x7f25dc0256b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c92c3d01ca213cc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:14.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.883+0000 7f25d4c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:05:14.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.883+0000 7f25d4c00640 1 -- 172.21.15.83:0/1629337680 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25dc037070 con 0x5641ecc52b40 2024-04-24T09:05:14.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.883+0000 7f25d4c00640 1 -- 172.21.15.83:0/1629337680 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7f25dc01d7c0 con 0x5641ecc52b40 2024-04-24T09:05:14.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.883+0000 7f25f51db740 1 -- 172.21.15.83:0/1629337680 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25d8013860 con 0x5641ecc52b40 2024-04-24T09:05:14.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.884+0000 7f25d4c00640 1 -- 172.21.15.83:0/1629337680 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f25dc045050 con 0x5641ecc52b40 2024-04-24T09:05:14.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.884+0000 7f25f51db740 10 client.20472 fetch_fsmap learned FSMap version 43 2024-04-24T09:05:14.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.884+0000 7f25f51db740 1 -- 172.21.15.83:0/1629337680 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5641ecc664f0 con 0x5641ecc52b40 2024-04-24T09:05:14.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.884+0000 7f25d4c00640 1 -- 172.21.15.83:0/1629337680 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f25d8013860 con 0x5641ecc52b40 2024-04-24T09:05:14.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.884+0000 7f25d4c00640 1 -- 172.21.15.83:0/1629337680 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7f25dc01b640 con 0x5641ecc52b40 2024-04-24T09:05:14.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.884+0000 7f25f51db740 10 client.20472 fetch_fsmap finished waiting for FSMap version 43 2024-04-24T09:05:14.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.884+0000 7f25f51db740 10 client.20472 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-24T09:05:14.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.884+0000 7f25f51db740 20 client.20472 populate_metadata read hostname 'smithi083' 2024-04-24T09:05:14.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.884+0000 7f25f51db740 1 --2- 172.21.15.83:0/1629337680 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x5641eccd4860 0x5641eccd6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:14.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.884+0000 7f25f51db740 4 client.20472 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:05:14.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.884+0000 7f25f51db740 1 -- 172.21.15.83:0/1629337680 --> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5641ecbbfa60 con 0x5641eccd4860 2024-04-24T09:05:14.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.885+0000 7f25eaa00640 1 --2- 172.21.15.83:0/1629337680 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x5641eccd4860 0x5641eccd6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:14.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.886+0000 7f25eaa00640 1 --2- 172.21.15.83:0/1629337680 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x5641eccd4860 0x5641eccd6cc0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4252d2970db56f47 server_cookie=3d52bd0d10a9509b in_seq=0 out_seq=0 2024-04-24T09:05:14.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.886+0000 7f25d4c00640 10 client.20472 ms_handle_connect on v2:172.21.15.138:6836/900566754 2024-04-24T09:05:14.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.886+0000 7f25d4c00640 1 -- 172.21.15.83:0/1629337680 <== mds.1 v2:172.21.15.138:6836/900566754 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5641ecbbfa60 con 0x5641eccd4860 2024-04-24T09:05:14.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.886+0000 7f25d4c00640 10 client.20472 handle_command_reply: tid=0 2024-04-24T09:05:14.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.886+0000 7f25f51db740 1 client.20472 shutdown 2024-04-24T09:05:14.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.887+0000 7f25f51db740 1 -- 172.21.15.83:0/1629337680 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641ecc52b40 msgr2=0x5641ecc5e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:14.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.887+0000 7f25f51db740 1 --2- 172.21.15.83:0/1629337680 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641ecc52b40 0x5641ecc5e2c0 secure :-1 s=READY pgs=2103 cs=0 l=1 rev1=1 crypto rx=0x7f25dc000c00 tx=0x7f25dc0256b0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.887+0000 7f25f51db740 1 -- 172.21.15.83:0/1629337680 shutdown_connections 2024-04-24T09:05:14.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.887+0000 7f25f51db740 1 --2- 172.21.15.83:0/1629337680 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x5641eccd4860 0x5641eccd6cc0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.887+0000 7f25f51db740 1 --2- 172.21.15.83:0/1629337680 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5641ecc5db30 0x5641ecc64df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.887+0000 7f25d4c00640 0 client.20472 ms_handle_reset on v2:172.21.15.138:6836/900566754 2024-04-24T09:05:14.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.887+0000 7f25f51db740 1 --2- 172.21.15.83:0/1629337680 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641ecc53460 0x5641ecc5e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.887+0000 7f25f51db740 1 --2- 172.21.15.83:0/1629337680 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5641ecc52b40 0x5641ecc5e2c0 unknown :-1 s=CLOSED pgs=2103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.887+0000 7f25f51db740 1 -- 172.21.15.83:0/1629337680 >> 172.21.15.83:0/1629337680 conn(0x5641ecc4ee60 msgr2=0x5641ecc599b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:14.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.887+0000 7f25f51db740 1 -- 172.21.15.83:0/1629337680 shutdown_connections 2024-04-24T09:05:14.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.887+0000 7f25f51db740 1 -- 172.21.15.83:0/1629337680 wait complete. 2024-04-24T09:05:14.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.887+0000 7f25f51db740 20 client.20472 trim_cache size 0 max 16384 2024-04-24T09:05:14.889 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:05:14.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.888+0000 7f25f0c00640 1 -- 172.21.15.83:0/979713259 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f25d00a3d20 msgr2=0x7f25d00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:14.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.888+0000 7f25f0c00640 1 --2- 172.21.15.83:0/979713259 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f25d00a3d20 0x7f25d00a61e0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f25dc005fd0 tx=0x7f25dc005dd0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.888+0000 7f25f0c00640 1 -- 172.21.15.83:0/979713259 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f25ec10d500 msgr2=0x7f25ec19ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:14.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.888+0000 7f25f0c00640 1 --2- 172.21.15.83:0/979713259 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f25ec10d500 0x7f25ec19ea30 secure :-1 s=READY pgs=2099 cs=0 l=1 rev1=1 crypto rx=0x7f25e402ae10 tx=0x7f25e4003ba0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.888+0000 7f25f0c00640 1 -- 172.21.15.83:0/979713259 shutdown_connections 2024-04-24T09:05:14.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.888+0000 7f25f0c00640 1 --2- 172.21.15.83:0/979713259 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f25d00a3d20 0x7f25d00a61e0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.888+0000 7f25f0c00640 1 --2- 172.21.15.83:0/979713259 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f25ec10d500 0x7f25ec19ea30 unknown :-1 s=CLOSED pgs=2099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.888+0000 7f25f0c00640 1 --2- 172.21.15.83:0/979713259 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25ec108f40 0x7f25ec1a4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.888+0000 7f25f0c00640 1 --2- 172.21.15.83:0/979713259 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f25ec108580 0x7f25ec1a4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:14.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.888+0000 7f25f0c00640 1 -- 172.21.15.83:0/979713259 >> 172.21.15.83:0/979713259 conn(0x7f25ec077e20 msgr2=0x7f25ec078340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:14.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.888+0000 7f25f0c00640 1 -- 172.21.15.83:0/979713259 shutdown_connections 2024-04-24T09:05:14.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:14.888+0000 7f25f0c00640 1 -- 172.21.15.83:0/979713259 wait complete. 2024-04-24T09:05:14.905 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T09:05:15.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.103+0000 7fe0ea800640 1 -- 172.21.15.83:0/431670411 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0e4076e50 msgr2=0x7fe0e41129a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:15.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.103+0000 7fe0ea800640 1 --2- 172.21.15.83:0/431670411 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0e4076e50 0x7fe0e41129a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe0d0009930 tx=0x7fe0d002a420 comp rx=0 tx=0).stop 2024-04-24T09:05:15.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.103+0000 7fe0ea800640 1 -- 172.21.15.83:0/431670411 shutdown_connections 2024-04-24T09:05:15.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.103+0000 7fe0ea800640 1 --2- 172.21.15.83:0/431670411 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0e4112ee0 0x7fe0e41152d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:15.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.103+0000 7fe0ea800640 1 --2- 172.21.15.83:0/431670411 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0e4076e50 0x7fe0e41129a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:15.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.103+0000 7fe0ea800640 1 --2- 172.21.15.83:0/431670411 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe0e4076530 0x7fe0e4076910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:15.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.103+0000 7fe0ea800640 1 -- 172.21.15.83:0/431670411 >> 172.21.15.83:0/431670411 conn(0x7fe0e40ffd90 msgr2=0x7fe0e41021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:15.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.103+0000 7fe0ea800640 1 -- 172.21.15.83:0/431670411 shutdown_connections 2024-04-24T09:05:15.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.103+0000 7fe0ea800640 1 -- 172.21.15.83:0/431670411 wait complete. 2024-04-24T09:05:15.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.104+0000 7fe0ea800640 1 Processor -- start 2024-04-24T09:05:15.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.105+0000 7fe0ea800640 1 -- start start 2024-04-24T09:05:15.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.105+0000 7fe0ea800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0e4076530 0x7fe0e419eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:15.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.105+0000 7fe0ea800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe0e4076e50 0x7fe0e419f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:15.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.105+0000 7fe0ea800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0e4112ee0 0x7fe0e41a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:15.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.105+0000 7fe0ea800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe0e4118940 con 0x7fe0e4076530 2024-04-24T09:05:15.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.105+0000 7fe0ea800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe0e41187c0 con 0x7fe0e4112ee0 2024-04-24T09:05:15.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.105+0000 7fe0ea800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe0e4118ac0 con 0x7fe0e4076e50 2024-04-24T09:05:15.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.105+0000 7fe0e3400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0e4076530 0x7fe0e419eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:15.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.105+0000 7fe0e3400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0e4076530 0x7fe0e419eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36226/0 (socket says 172.21.15.83:36226) 2024-04-24T09:05:15.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.105+0000 7fe0e3400640 1 -- 172.21.15.83:0/1018995534 learned_addr learned my addr 172.21.15.83:0/1018995534 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:05:15.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.106+0000 7fe0e2a00640 1 --2- 172.21.15.83:0/1018995534 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe0e4076e50 0x7fe0e419f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:15.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.106+0000 7fe0e3400640 1 -- 172.21.15.83:0/1018995534 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe0e4076e50 msgr2=0x7fe0e419f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:15.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.106+0000 7fe0e3400640 1 --2- 172.21.15.83:0/1018995534 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe0e4076e50 0x7fe0e419f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:15.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.106+0000 7fe0e3400640 1 -- 172.21.15.83:0/1018995534 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0e4112ee0 msgr2=0x7fe0e41a37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:15.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.106+0000 7fe0e3400640 1 --2- 172.21.15.83:0/1018995534 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0e4112ee0 0x7fe0e41a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:15.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.106+0000 7fe0e3400640 1 -- 172.21.15.83:0/1018995534 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0e41a3ed0 con 0x7fe0e4076530 2024-04-24T09:05:15.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.106+0000 7fe0e3400640 1 --2- 172.21.15.83:0/1018995534 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0e4076530 0x7fe0e419eee0 secure :-1 s=READY pgs=2104 cs=0 l=1 rev1=1 crypto rx=0x7fe0d400e9f0 tx=0x7fe0d400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:15.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.106+0000 7fe0dbe00640 1 -- 172.21.15.83:0/1018995534 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0d400cd80 con 0x7fe0e4076530 2024-04-24T09:05:15.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.107+0000 7fe0ea800640 1 -- 172.21.15.83:0/1018995534 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0e41a41c0 con 0x7fe0e4076530 2024-04-24T09:05:15.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.107+0000 7fe0dbe00640 1 -- 172.21.15.83:0/1018995534 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe0d40054d0 con 0x7fe0e4076530 2024-04-24T09:05:15.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.107+0000 7fe0dbe00640 1 -- 172.21.15.83:0/1018995534 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0d4018630 con 0x7fe0e4076530 2024-04-24T09:05:15.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.107+0000 7fe0ea800640 1 -- 172.21.15.83:0/1018995534 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0e41aba50 con 0x7fe0e4076530 2024-04-24T09:05:15.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.110+0000 7fe0dbe00640 1 -- 172.21.15.83:0/1018995534 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe0d40187d0 con 0x7fe0e4076530 2024-04-24T09:05:15.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.110+0000 7fe0ed781740 1 Processor -- start 2024-04-24T09:05:15.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.110+0000 7fe0ed781740 1 -- start start 2024-04-24T09:05:15.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.111+0000 7fe0ed781740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642226cf950 0x5642226cfd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:15.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.111+0000 7fe0ed781740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642226d0270 0x5642226da400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:15.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.111+0000 7fe0ed781740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5642226da940 0x5642226dcd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:15.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.111+0000 7fe0ed781740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5642226351d0 con 0x5642226cf950 2024-04-24T09:05:15.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.111+0000 7fe0ed781740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5642225db450 con 0x5642226d0270 2024-04-24T09:05:15.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.111+0000 7fe0ed781740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x564222631190 con 0x5642226da940 2024-04-24T09:05:15.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.111+0000 7fe0e3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642226cf950 0x5642226cfd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:15.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.111+0000 7fe0e3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642226cf950 0x5642226cfd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36234/0 (socket says 172.21.15.83:36234) 2024-04-24T09:05:15.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.111+0000 7fe0e3e00640 1 -- 172.21.15.83:0/1936574136 learned_addr learned my addr 172.21.15.83:0/1936574136 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:05:15.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.111+0000 7fe0e3400640 1 --2- 172.21.15.83:0/1936574136 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642226d0270 0x5642226da400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:15.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.111+0000 7fe0e3e00640 1 -- 172.21.15.83:0/1936574136 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5642226da940 msgr2=0x5642226dcd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:15.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.111+0000 7fe0e3e00640 1 --2- 172.21.15.83:0/1936574136 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5642226da940 0x5642226dcd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:15.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.111+0000 7fe0e3e00640 1 -- 172.21.15.83:0/1936574136 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642226d0270 msgr2=0x5642226da400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:15.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.111+0000 7fe0e3e00640 1 --2- 172.21.15.83:0/1936574136 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642226d0270 0x5642226da400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:15.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.111+0000 7fe0e3e00640 1 -- 172.21.15.83:0/1936574136 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564222698310 con 0x5642226cf950 2024-04-24T09:05:15.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.111+0000 7fe0e3400640 1 --2- 172.21.15.83:0/1936574136 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642226d0270 0x5642226da400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:05:15.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.112+0000 7fe0e3e00640 1 --2- 172.21.15.83:0/1936574136 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642226cf950 0x5642226cfd30 secure :-1 s=READY pgs=2105 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc009e00 tx=0x7fe0dc00e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d206ae8ba1433ebf server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:15.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.112+0000 7fe0dbe00640 1 --2- 172.21.15.83:0/1018995534 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe0c80a3d20 0x7fe0c80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:15.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.112+0000 7fe0dbe00640 1 -- 172.21.15.83:0/1018995534 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7fe0d40d12e0 con 0x7fe0e4076530 2024-04-24T09:05:15.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.112+0000 7fe0d8c00640 1 -- 172.21.15.83:0/1936574136 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0dc004d20 con 0x5642226cf950 2024-04-24T09:05:15.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.112+0000 7fe0e3400640 1 --2- 172.21.15.83:0/1018995534 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe0c80a3d20 0x7fe0c80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:15.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.112+0000 7fe0d8c00640 1 -- 172.21.15.83:0/1936574136 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe0dc00ee70 con 0x5642226cf950 2024-04-24T09:05:15.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.112+0000 7fe0d8c00640 1 -- 172.21.15.83:0/1936574136 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0dc005020 con 0x5642226cf950 2024-04-24T09:05:15.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.113+0000 7fe0ed781740 1 -- 172.21.15.83:0/1936574136 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642226cf950 msgr2=0x5642226cfd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:15.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.113+0000 7fe0ed781740 1 --2- 172.21.15.83:0/1936574136 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642226cf950 0x5642226cfd30 secure :-1 s=READY pgs=2105 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc009e00 tx=0x7fe0dc00e720 comp rx=0 tx=0).stop 2024-04-24T09:05:15.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.113+0000 7fe0e3400640 1 --2- 172.21.15.83:0/1018995534 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe0c80a3d20 0x7fe0c80a61e0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x5642226d0f10 tx=0x7fe0d400cf20 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:15.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.113+0000 7fe0ed781740 1 -- 172.21.15.83:0/1936574136 shutdown_connections 2024-04-24T09:05:15.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.113+0000 7fe0ed781740 1 --2- 172.21.15.83:0/1936574136 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5642226da940 0x5642226dcd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:15.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.113+0000 7fe0ed781740 1 --2- 172.21.15.83:0/1936574136 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642226d0270 0x5642226da400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:15.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.113+0000 7fe0ed781740 1 --2- 172.21.15.83:0/1936574136 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642226cf950 0x5642226cfd30 unknown :-1 s=CLOSED pgs=2105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:15.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.113+0000 7fe0ed781740 1 -- 172.21.15.83:0/1936574136 >> 172.21.15.83:0/1936574136 conn(0x5642226cbc70 msgr2=0x5642226cc050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:15.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.113+0000 7fe0ed781740 1 -- 172.21.15.83:0/1936574136 shutdown_connections 2024-04-24T09:05:15.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.113+0000 7fe0ed781740 1 -- 172.21.15.83:0/1936574136 wait complete. 2024-04-24T09:05:15.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.114+0000 7fe0ed781740 1 -- start start 2024-04-24T09:05:15.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.114+0000 7fe0ed781740 1 Processor -- start 2024-04-24T09:05:15.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.114+0000 7fe0ed781740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642226cf950 0x5642226da2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:15.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.115+0000 7fe0ed781740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5642226d0270 0x5642226d7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:15.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.115+0000 7fe0ed781740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642226da940 0x5642226d7c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:15.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.115+0000 7fe0ed781740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5642226dd7d0 con 0x5642226cf950 2024-04-24T09:05:15.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.115+0000 7fe0ed781740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5642225ee810 con 0x5642226da940 2024-04-24T09:05:15.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.115+0000 7fe0ed781740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5642226dd950 con 0x5642226d0270 2024-04-24T09:05:15.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.115+0000 7fe0ed781740 10 client.0 _ll_register_callbacks cb 0x5642225bc140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:05:15.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.115+0000 7fe0e3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642226cf950 0x5642226da2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:15.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.115+0000 7fe0e3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642226cf950 0x5642226da2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36244/0 (socket says 172.21.15.83:36244) 2024-04-24T09:05:15.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.115+0000 7fe0e3e00640 1 -- 172.21.15.83:0/891300232 learned_addr learned my addr 172.21.15.83:0/891300232 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:05:15.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.115+0000 7fe0e2a00640 1 --2- 172.21.15.83:0/891300232 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5642226d0270 0x5642226d7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:15.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.115+0000 7fe0e3e00640 1 -- 172.21.15.83:0/891300232 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5642226d0270 msgr2=0x5642226d7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:15.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.115+0000 7fe0e3e00640 1 --2- 172.21.15.83:0/891300232 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5642226d0270 0x5642226d7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:15.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.115+0000 7fe0e3e00640 1 -- 172.21.15.83:0/891300232 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642226da940 msgr2=0x5642226d7c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:15.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.115+0000 7fe0e3e00640 1 --2- 172.21.15.83:0/891300232 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642226da940 0x5642226d7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:15.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.115+0000 7fe0e3e00640 1 -- 172.21.15.83:0/891300232 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642225ff3f0 con 0x5642226cf950 2024-04-24T09:05:15.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.115+0000 7fe0e2a00640 1 --2- 172.21.15.83:0/891300232 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5642226d0270 0x5642226d7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:05:15.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.116+0000 7fe0e3e00640 1 --2- 172.21.15.83:0/891300232 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642226cf950 0x5642226da2f0 secure :-1 s=READY pgs=2106 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc004a80 tx=0x7fe0dc004900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d93f23589b5a320f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:15.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.116+0000 7fe0cd600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:05:15.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.116+0000 7fe0cd600640 1 -- 172.21.15.83:0/891300232 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0dc029060 con 0x5642226cf950 2024-04-24T09:05:15.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.116+0000 7fe0cd600640 1 -- 172.21.15.83:0/891300232 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7fe0dc003b70 con 0x5642226cf950 2024-04-24T09:05:15.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.116+0000 7fe0ed781740 1 -- 172.21.15.83:0/891300232 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0dc00ee70 con 0x5642226cf950 2024-04-24T09:05:15.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.117+0000 7fe0cd600640 1 -- 172.21.15.83:0/891300232 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fe0dc004e30 con 0x5642226cf950 2024-04-24T09:05:15.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.117+0000 7fe0ed781740 10 client.20484 fetch_fsmap learned FSMap version 43 2024-04-24T09:05:15.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.117+0000 7fe0ed781740 1 -- 172.21.15.83:0/891300232 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5642223473f0 con 0x5642226cf950 2024-04-24T09:05:15.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.117+0000 7fe0cd600640 1 -- 172.21.15.83:0/891300232 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe0dc00ee70 con 0x5642226cf950 2024-04-24T09:05:15.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.117+0000 7fe0cd600640 1 -- 172.21.15.83:0/891300232 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7fe0dc0104c0 con 0x5642226cf950 2024-04-24T09:05:15.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.117+0000 7fe0ed781740 10 client.20484 fetch_fsmap finished waiting for FSMap version 43 2024-04-24T09:05:15.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.117+0000 7fe0ed781740 10 client.20484 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:05:15.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.118+0000 7fe0ed781740 20 client.20484 populate_metadata read hostname 'smithi083' 2024-04-24T09:05:15.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.118+0000 7fe0ed781740 1 --2- 172.21.15.83:0/891300232 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5642226ea980 0x5642226ecd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:15.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.118+0000 7fe0ed781740 4 client.20484 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:05:15.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.118+0000 7fe0ed781740 1 -- 172.21.15.83:0/891300232 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564222642bf0 con 0x5642226ea980 2024-04-24T09:05:15.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.118+0000 7fe0e2a00640 1 --2- 172.21.15.83:0/891300232 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5642226ea980 0x5642226ecd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:15.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.120+0000 7fe0cd600640 10 client.20484 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:05:15.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.120+0000 7fe0e2a00640 1 --2- 172.21.15.83:0/891300232 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5642226ea980 0x5642226ecd70 crc :-1 s=READY pgs=351 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6698824ab2c37a84 server_cookie=a0a942ab0e0d786b in_seq=0 out_seq=0 2024-04-24T09:05:15.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.122+0000 7fe0cd600640 1 -- 172.21.15.83:0/891300232 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564222642bf0 con 0x5642226ea980 2024-04-24T09:05:15.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.122+0000 7fe0cd600640 10 client.20484 handle_command_reply: tid=0 2024-04-24T09:05:15.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.122+0000 7fe0ed781740 1 client.20484 shutdown 2024-04-24T09:05:15.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.122+0000 7fe0ed781740 1 -- 172.21.15.83:0/891300232 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642226cf950 msgr2=0x5642226da2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:15.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.122+0000 7fe0ed781740 1 --2- 172.21.15.83:0/891300232 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642226cf950 0x5642226da2f0 secure :-1 s=READY pgs=2106 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc004a80 tx=0x7fe0dc004900 comp rx=0 tx=0).stop 2024-04-24T09:05:15.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.122+0000 7fe0ed781740 1 -- 172.21.15.83:0/891300232 shutdown_connections 2024-04-24T09:05:15.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.122+0000 7fe0ed781740 1 --2- 172.21.15.83:0/891300232 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5642226ea980 0x5642226ecd70 crc :-1 s=READY pgs=351 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:15.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.122+0000 7fe0ed781740 1 --2- 172.21.15.83:0/891300232 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642226da940 0x5642226d7c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:15.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.122+0000 7fe0ed781740 1 --2- 172.21.15.83:0/891300232 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5642226d0270 0x5642226d7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:15.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.122+0000 7fe0ed781740 1 --2- 172.21.15.83:0/891300232 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642226cf950 0x5642226da2f0 unknown :-1 s=CLOSED pgs=2106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:15.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.122+0000 7fe0ed781740 1 -- 172.21.15.83:0/891300232 >> 172.21.15.83:0/891300232 conn(0x5642226cbc70 msgr2=0x5642226d1970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:15.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.122+0000 7fe0cd600640 0 client.20484 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:05:15.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.122+0000 7fe0ed781740 1 -- 172.21.15.83:0/891300232 shutdown_connections 2024-04-24T09:05:15.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.122+0000 7fe0ed781740 1 -- 172.21.15.83:0/891300232 wait complete. 2024-04-24T09:05:15.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.123+0000 7fe0ed781740 20 client.20484 trim_cache size 0 max 16384 2024-04-24T09:05:15.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.137+0000 7fe0ed781740 1 Processor -- start 2024-04-24T09:05:15.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.137+0000 7fe0ed781740 1 -- start start 2024-04-24T09:05:15.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.138+0000 7fe0ed781740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642226cf950 0x5642226e6690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:15.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.138+0000 7fe0ed781740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5642226d0270 0x5642226d9e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:15.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.138+0000 7fe0ed781740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642226da940 0x5642226f2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:15.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.138+0000 7fe0ed781740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe0e41b5d60 con 0x5642226da940 2024-04-24T09:05:15.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.138+0000 7fe0ed781740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe0e41b7960 con 0x5642226cf950 2024-04-24T09:05:15.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.138+0000 7fe0ed781740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe0e41b6430 con 0x5642226d0270 2024-04-24T09:05:15.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.138+0000 7fe0e3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642226da940 0x5642226f2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:15.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.138+0000 7fe0e3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642226da940 0x5642226f2340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36246/0 (socket says 172.21.15.83:36246) 2024-04-24T09:05:15.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.138+0000 7fe0e3e00640 1 -- 172.21.15.83:0/2517752713 learned_addr learned my addr 172.21.15.83:0/2517752713 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:05:15.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.138+0000 7fe0e3e00640 1 --2- 172.21.15.83:0/2517752713 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642226cf950 0x5642226e6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:15.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.138+0000 7fe0e2a00640 1 --2- 172.21.15.83:0/2517752713 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5642226d0270 0x5642226d9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:15.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.139+0000 7fe0e3e00640 1 -- 172.21.15.83:0/2517752713 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5642226d0270 msgr2=0x5642226d9e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:15.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.139+0000 7fe0e3e00640 1 --2- 172.21.15.83:0/2517752713 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5642226d0270 0x5642226d9e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:15.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.139+0000 7fe0e3e00640 1 -- 172.21.15.83:0/2517752713 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642226cf950 msgr2=0x5642226e6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:15.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.139+0000 7fe0e3e00640 1 --2- 172.21.15.83:0/2517752713 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642226cf950 0x5642226e6690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:15.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.139+0000 7fe0e3e00640 1 -- 172.21.15.83:0/2517752713 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642226d00a0 con 0x5642226da940 2024-04-24T09:05:15.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.139+0000 7fe0e3e00640 1 --2- 172.21.15.83:0/2517752713 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642226da940 0x5642226f2340 secure :-1 s=READY pgs=2107 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc009e00 tx=0x7fe0dc01a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89371f70df0841f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:15.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.139+0000 7fe0cfe00640 1 -- 172.21.15.83:0/2517752713 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0dc03b070 con 0x5642226da940 2024-04-24T09:05:15.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.139+0000 7fe0cfe00640 1 -- 172.21.15.83:0/2517752713 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe0dc003e20 con 0x5642226da940 2024-04-24T09:05:15.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.139+0000 7fe0cfe00640 1 -- 172.21.15.83:0/2517752713 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0dc029070 con 0x5642226da940 2024-04-24T09:05:15.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.140+0000 7fe0ed781740 1 -- 172.21.15.83:0/2517752713 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642226da940 msgr2=0x5642226f2340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:15.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.140+0000 7fe0ed781740 1 --2- 172.21.15.83:0/2517752713 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642226da940 0x5642226f2340 secure :-1 s=READY pgs=2107 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc009e00 tx=0x7fe0dc01a040 comp rx=0 tx=0).stop 2024-04-24T09:05:15.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.140+0000 7fe0ed781740 1 -- 172.21.15.83:0/2517752713 shutdown_connections 2024-04-24T09:05:15.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.140+0000 7fe0ed781740 1 --2- 172.21.15.83:0/2517752713 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642226da940 0x5642226f2340 unknown :-1 s=CLOSED pgs=2107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:15.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.140+0000 7fe0ed781740 1 --2- 172.21.15.83:0/2517752713 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5642226d0270 0x5642226d9e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:15.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.140+0000 7fe0ed781740 1 --2- 172.21.15.83:0/2517752713 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642226cf950 0x5642226e6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:15.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.140+0000 7fe0ed781740 1 -- 172.21.15.83:0/2517752713 >> 172.21.15.83:0/2517752713 conn(0x5642226cbc70 msgr2=0x5642226d2940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:15.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.140+0000 7fe0ed781740 1 -- 172.21.15.83:0/2517752713 shutdown_connections 2024-04-24T09:05:15.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.140+0000 7fe0ed781740 1 -- 172.21.15.83:0/2517752713 wait complete. 2024-04-24T09:05:15.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.140+0000 7fe0ed781740 1 -- start start 2024-04-24T09:05:15.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.141+0000 7fe0ed781740 1 Processor -- start 2024-04-24T09:05:15.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.141+0000 7fe0ed781740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642226cf950 0x5642226dad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:15.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.141+0000 7fe0ed781740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5642226d0270 0x5642226db260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:15.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.141+0000 7fe0ed781740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642226da940 0x5642226db7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:15.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.141+0000 7fe0ed781740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe0e41b7230 con 0x5642226cf950 2024-04-24T09:05:15.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.141+0000 7fe0ed781740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe0e41b6b00 con 0x5642226da940 2024-04-24T09:05:15.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.141+0000 7fe0ed781740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe0e41b3d60 con 0x5642226d0270 2024-04-24T09:05:15.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.141+0000 7fe0ed781740 10 client.0 _ll_register_callbacks cb 0x7fe0dc004590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:05:15.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.141+0000 7fe0e3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642226cf950 0x5642226dad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:15.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.141+0000 7fe0e3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642226cf950 0x5642226dad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36262/0 (socket says 172.21.15.83:36262) 2024-04-24T09:05:15.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.141+0000 7fe0e3e00640 1 -- 172.21.15.83:0/1293054622 learned_addr learned my addr 172.21.15.83:0/1293054622 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:05:15.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.142+0000 7fe0e2a00640 1 --2- 172.21.15.83:0/1293054622 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5642226d0270 0x5642226db260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:15.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.142+0000 7fe0e3e00640 1 -- 172.21.15.83:0/1293054622 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5642226d0270 msgr2=0x5642226db260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:15.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.142+0000 7fe0e3e00640 1 --2- 172.21.15.83:0/1293054622 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5642226d0270 0x5642226db260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:15.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.142+0000 7fe0e3e00640 1 -- 172.21.15.83:0/1293054622 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642226da940 msgr2=0x5642226db7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:15.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.142+0000 7fe0e3e00640 1 --2- 172.21.15.83:0/1293054622 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642226da940 0x5642226db7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:15.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.142+0000 7fe0e3e00640 1 -- 172.21.15.83:0/1293054622 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642224117d0 con 0x5642226cf950 2024-04-24T09:05:15.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.142+0000 7fe0e3e00640 1 --2- 172.21.15.83:0/1293054622 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642226cf950 0x5642226dad20 secure :-1 s=READY pgs=2108 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc003ff0 tx=0x7fe0dc0107c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ca02ffffedfa814 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:15.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.142+0000 7fe0cd600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:05:15.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.142+0000 7fe0cd600640 1 -- 172.21.15.83:0/1293054622 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0dc029050 con 0x5642226cf950 2024-04-24T09:05:15.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.142+0000 7fe0cd600640 1 -- 172.21.15.83:0/1293054622 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7fe0dc003b70 con 0x5642226cf950 2024-04-24T09:05:15.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.142+0000 7fe0ed781740 1 -- 172.21.15.83:0/1293054622 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0dc003e20 con 0x5642226cf950 2024-04-24T09:05:15.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.143+0000 7fe0cd600640 1 -- 172.21.15.83:0/1293054622 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fe0dc03b040 con 0x5642226cf950 2024-04-24T09:05:15.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.143+0000 7fe0ed781740 10 client.20490 fetch_fsmap learned FSMap version 43 2024-04-24T09:05:15.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.143+0000 7fe0ed781740 1 -- 172.21.15.83:0/1293054622 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5642226e2c80 con 0x5642226cf950 2024-04-24T09:05:15.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.143+0000 7fe0cd600640 1 -- 172.21.15.83:0/1293054622 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe0dc003e20 con 0x5642226cf950 2024-04-24T09:05:15.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.143+0000 7fe0cd600640 1 -- 172.21.15.83:0/1293054622 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7fe0dc042480 con 0x5642226cf950 2024-04-24T09:05:15.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.143+0000 7fe0ed781740 10 client.20490 fetch_fsmap finished waiting for FSMap version 43 2024-04-24T09:05:15.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.143+0000 7fe0ed781740 10 client.20490 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:05:15.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.143+0000 7fe0ed781740 20 client.20490 populate_metadata read hostname 'smithi083' 2024-04-24T09:05:15.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.143+0000 7fe0ed781740 1 --2- 172.21.15.83:0/1293054622 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5642226e31d0 0x564222752bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:15.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.143+0000 7fe0ed781740 4 client.20490 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:05:15.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.143+0000 7fe0ed781740 1 -- 172.21.15.83:0/1293054622 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56422263ca60 con 0x5642226e31d0 2024-04-24T09:05:15.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.144+0000 7fe0e2a00640 1 --2- 172.21.15.83:0/1293054622 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5642226e31d0 0x564222752bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:15.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.145+0000 7fe0e2a00640 1 --2- 172.21.15.83:0/1293054622 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5642226e31d0 0x564222752bc0 crc :-1 s=READY pgs=352 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c162c98e73b543e6 server_cookie=acc597470d89cdef in_seq=0 out_seq=0 2024-04-24T09:05:15.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.145+0000 7fe0cd600640 10 client.20490 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:05:15.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.145+0000 7fe0cd600640 1 -- 172.21.15.83:0/1293054622 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56422263ca60 con 0x5642226e31d0 2024-04-24T09:05:15.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.145+0000 7fe0cd600640 10 client.20490 handle_command_reply: tid=0 2024-04-24T09:05:15.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.145+0000 7fe0ed781740 1 client.20490 shutdown 2024-04-24T09:05:15.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.146+0000 7fe0ed781740 1 -- 172.21.15.83:0/1293054622 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642226cf950 msgr2=0x5642226dad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:15.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.146+0000 7fe0ed781740 1 --2- 172.21.15.83:0/1293054622 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642226cf950 0x5642226dad20 secure :-1 s=READY pgs=2108 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc003ff0 tx=0x7fe0dc0107c0 comp rx=0 tx=0).stop 2024-04-24T09:05:15.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.146+0000 7fe0ed781740 1 -- 172.21.15.83:0/1293054622 shutdown_connections 2024-04-24T09:05:15.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.146+0000 7fe0ed781740 1 --2- 172.21.15.83:0/1293054622 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5642226e31d0 0x564222752bc0 crc :-1 s=READY pgs=352 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:15.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.146+0000 7fe0ed781740 1 --2- 172.21.15.83:0/1293054622 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642226da940 0x5642226db7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:15.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.146+0000 7fe0ed781740 1 --2- 172.21.15.83:0/1293054622 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5642226d0270 0x5642226db260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:15.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.146+0000 7fe0cd600640 0 client.20490 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:05:15.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.146+0000 7fe0ed781740 1 --2- 172.21.15.83:0/1293054622 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642226cf950 0x5642226dad20 unknown :-1 s=CLOSED pgs=2108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:15.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.146+0000 7fe0ed781740 1 -- 172.21.15.83:0/1293054622 >> 172.21.15.83:0/1293054622 conn(0x5642226cbc70 msgr2=0x5642226ca560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:15.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.146+0000 7fe0ed781740 1 -- 172.21.15.83:0/1293054622 shutdown_connections 2024-04-24T09:05:15.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.146+0000 7fe0ed781740 1 -- 172.21.15.83:0/1293054622 wait complete. 2024-04-24T09:05:15.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.146+0000 7fe0ed781740 20 client.20490 trim_cache size 0 max 16384 2024-04-24T09:05:15.153 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:05:15.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.147+0000 7fe0ea800640 1 -- 172.21.15.83:0/1018995534 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe0c80a3d20 msgr2=0x7fe0c80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:15.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.147+0000 7fe0ea800640 1 --2- 172.21.15.83:0/1018995534 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe0c80a3d20 0x7fe0c80a61e0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x5642226d0f10 tx=0x7fe0d400cf20 comp rx=0 tx=0).stop 2024-04-24T09:05:15.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.147+0000 7fe0ea800640 1 -- 172.21.15.83:0/1018995534 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0e4076530 msgr2=0x7fe0e419eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:15.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.147+0000 7fe0ea800640 1 --2- 172.21.15.83:0/1018995534 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0e4076530 0x7fe0e419eee0 secure :-1 s=READY pgs=2104 cs=0 l=1 rev1=1 crypto rx=0x7fe0d400e9f0 tx=0x7fe0d400edf0 comp rx=0 tx=0).stop 2024-04-24T09:05:15.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.147+0000 7fe0ea800640 1 -- 172.21.15.83:0/1018995534 shutdown_connections 2024-04-24T09:05:15.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.147+0000 7fe0ea800640 1 --2- 172.21.15.83:0/1018995534 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe0c80a3d20 0x7fe0c80a61e0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:15.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.147+0000 7fe0ea800640 1 --2- 172.21.15.83:0/1018995534 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0e4112ee0 0x7fe0e41a37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:15.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.147+0000 7fe0ea800640 1 --2- 172.21.15.83:0/1018995534 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe0e4076e50 0x7fe0e419f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:15.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.147+0000 7fe0ea800640 1 --2- 172.21.15.83:0/1018995534 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0e4076530 0x7fe0e419eee0 unknown :-1 s=CLOSED pgs=2104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:15.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.147+0000 7fe0ea800640 1 -- 172.21.15.83:0/1018995534 >> 172.21.15.83:0/1018995534 conn(0x7fe0e40ffd90 msgr2=0x7fe0e4102180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:15.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.147+0000 7fe0ea800640 1 -- 172.21.15.83:0/1018995534 shutdown_connections 2024-04-24T09:05:15.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:15.147+0000 7fe0ea800640 1 -- 172.21.15.83:0/1018995534 wait complete. 2024-04-24T09:05:15.632 INFO:tasks.workunit.client.0.smithi083.stdout: 1 78226 0.58 MB/sec execute 589 sec latency 1862.752 ms 2024-04-24T09:05:16.164 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T09:05:16.165 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T09:05:16.165 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T09:05:16.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:05:16 smithi083 ceph-mon[27907]: pgmap v428: 129 pgs: 129 active+clean; 74 MiB data, 997 MiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 326 KiB/s wr, 33 op/s 2024-04-24T09:05:16.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.384+0000 7f71e1c00640 1 -- 172.21.15.83:0/3699524501 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71dc076e50 msgr2=0x7f71dc1129a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:16.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.384+0000 7f71e1c00640 1 --2- 172.21.15.83:0/3699524501 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71dc076e50 0x7f71dc1129a0 secure :-1 s=READY pgs=2109 cs=0 l=1 rev1=1 crypto rx=0x7f71c4009900 tx=0x7f71c402a3f0 comp rx=0 tx=0).stop 2024-04-24T09:05:16.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.384+0000 7f71e1c00640 1 -- 172.21.15.83:0/3699524501 shutdown_connections 2024-04-24T09:05:16.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.384+0000 7f71e1c00640 1 --2- 172.21.15.83:0/3699524501 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f71dc112ee0 0x7f71dc1152d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:16.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.384+0000 7f71e1c00640 1 --2- 172.21.15.83:0/3699524501 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71dc076e50 0x7f71dc1129a0 unknown :-1 s=CLOSED pgs=2109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:16.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.384+0000 7f71e1c00640 1 --2- 172.21.15.83:0/3699524501 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f71dc076530 0x7f71dc076910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:16.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.384+0000 7f71e1c00640 1 -- 172.21.15.83:0/3699524501 >> 172.21.15.83:0/3699524501 conn(0x7f71dc0ffd90 msgr2=0x7f71dc1021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:16.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.385+0000 7f71e1c00640 1 -- 172.21.15.83:0/3699524501 shutdown_connections 2024-04-24T09:05:16.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.385+0000 7f71e1c00640 1 -- 172.21.15.83:0/3699524501 wait complete. 2024-04-24T09:05:16.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.386+0000 7f71e1c00640 1 Processor -- start 2024-04-24T09:05:16.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.386+0000 7f71e1c00640 1 -- start start 2024-04-24T09:05:16.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.386+0000 7f71e1c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f71dc076530 0x7f71dc19eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:16.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.386+0000 7f71e1c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71dc076e50 0x7f71dc19f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:16.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.386+0000 7f71e1c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f71dc112ee0 0x7f71dc1a3720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:16.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.386+0000 7f71e1c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f71dc1188f0 con 0x7f71dc076e50 2024-04-24T09:05:16.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.386+0000 7f71e1c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f71dc118770 con 0x7f71dc112ee0 2024-04-24T09:05:16.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.386+0000 7f71e1c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f71dc118a70 con 0x7f71dc076530 2024-04-24T09:05:16.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.386+0000 7f71d2000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71dc076e50 0x7f71dc19f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:16.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.386+0000 7f71d2000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71dc076e50 0x7f71dc19f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36270/0 (socket says 172.21.15.83:36270) 2024-04-24T09:05:16.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.386+0000 7f71d2000640 1 -- 172.21.15.83:0/2651854513 learned_addr learned my addr 172.21.15.83:0/2651854513 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:05:16.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.387+0000 7f71daa00640 1 --2- 172.21.15.83:0/2651854513 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f71dc076530 0x7f71dc19eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:16.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.387+0000 7f71db400640 1 --2- 172.21.15.83:0/2651854513 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f71dc112ee0 0x7f71dc1a3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:16.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.387+0000 7f71d2000640 1 -- 172.21.15.83:0/2651854513 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f71dc076530 msgr2=0x7f71dc19eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:16.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.387+0000 7f71d2000640 1 --2- 172.21.15.83:0/2651854513 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f71dc076530 0x7f71dc19eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:16.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.387+0000 7f71d2000640 1 -- 172.21.15.83:0/2651854513 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f71dc112ee0 msgr2=0x7f71dc1a3720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:16.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.387+0000 7f71d2000640 1 --2- 172.21.15.83:0/2651854513 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f71dc112ee0 0x7f71dc1a3720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:16.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.387+0000 7f71d2000640 1 -- 172.21.15.83:0/2651854513 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71dc1a3e50 con 0x7f71dc076e50 2024-04-24T09:05:16.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.387+0000 7f71d2000640 1 --2- 172.21.15.83:0/2651854513 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71dc076e50 0x7f71dc19f3f0 secure :-1 s=READY pgs=2110 cs=0 l=1 rev1=1 crypto rx=0x7f71c4006010 tx=0x7f71c40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:16.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.387+0000 7f71d3e00640 1 -- 172.21.15.83:0/2651854513 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c403f070 con 0x7f71dc076e50 2024-04-24T09:05:16.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.387+0000 7f71d3e00640 1 -- 172.21.15.83:0/2651854513 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f71c4003aa0 con 0x7f71dc076e50 2024-04-24T09:05:16.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.388+0000 7f71e1c00640 1 -- 172.21.15.83:0/2651854513 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71dc1a40e0 con 0x7f71dc076e50 2024-04-24T09:05:16.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.388+0000 7f71d3e00640 1 -- 172.21.15.83:0/2651854513 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c4035030 con 0x7f71dc076e50 2024-04-24T09:05:16.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.388+0000 7f71e1c00640 1 -- 172.21.15.83:0/2651854513 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71dc1ab9d0 con 0x7f71dc076e50 2024-04-24T09:05:16.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.392+0000 7f71e4b67740 1 Processor -- start 2024-04-24T09:05:16.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.392+0000 7f71e4b67740 1 -- start start 2024-04-24T09:05:16.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.392+0000 7f71d3e00640 1 -- 172.21.15.83:0/2651854513 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f71c4042070 con 0x7f71dc076e50 2024-04-24T09:05:16.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.392+0000 7f71e4b67740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df0f472090 0x55df0f472470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:16.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.392+0000 7f71e4b67740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df0f4729b0 0x55df0f47cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:16.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.392+0000 7f71e4b67740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55df0f47d080 0x55df0f47f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:16.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.392+0000 7f71e4b67740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55df0f3d7e70 con 0x55df0f4729b0 2024-04-24T09:05:16.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.392+0000 7f71e4b67740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55df0f37e380 con 0x55df0f472090 2024-04-24T09:05:16.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.392+0000 7f71e4b67740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55df0f3d3e30 con 0x55df0f47d080 2024-04-24T09:05:16.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.393+0000 7f71daa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df0f4729b0 0x55df0f47cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:16.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.393+0000 7f71daa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df0f4729b0 0x55df0f47cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36276/0 (socket says 172.21.15.83:36276) 2024-04-24T09:05:16.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.393+0000 7f71daa00640 1 -- 172.21.15.83:0/535186819 learned_addr learned my addr 172.21.15.83:0/535186819 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:05:16.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.393+0000 7f71db400640 1 --2- 172.21.15.83:0/535186819 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df0f472090 0x55df0f472470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:16.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.393+0000 7f71d2000640 1 --2- 172.21.15.83:0/535186819 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55df0f47d080 0x55df0f47f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:16.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.393+0000 7f71daa00640 1 -- 172.21.15.83:0/535186819 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55df0f47d080 msgr2=0x55df0f47f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:16.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.393+0000 7f71daa00640 1 --2- 172.21.15.83:0/535186819 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55df0f47d080 0x55df0f47f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:16.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.393+0000 7f71daa00640 1 -- 172.21.15.83:0/535186819 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df0f472090 msgr2=0x55df0f472470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:16.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.393+0000 7f71daa00640 1 --2- 172.21.15.83:0/535186819 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df0f472090 0x55df0f472470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:16.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.393+0000 7f71daa00640 1 -- 172.21.15.83:0/535186819 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df0f1b27e0 con 0x55df0f4729b0 2024-04-24T09:05:16.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.394+0000 7f71daa00640 1 --2- 172.21.15.83:0/535186819 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df0f4729b0 0x55df0f47cb40 secure :-1 s=READY pgs=2111 cs=0 l=1 rev1=1 crypto rx=0x7f71cc00cc60 tx=0x7f71cc00e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7d60a95cb4ced2f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:16.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.394+0000 7f71d3e00640 1 --2- 172.21.15.83:0/2651854513 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f71c00a3d20 0x7f71c00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:16.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.394+0000 7f71cbe00640 1 -- 172.21.15.83:0/535186819 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71cc014070 con 0x55df0f4729b0 2024-04-24T09:05:16.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.394+0000 7f71cbe00640 1 -- 172.21.15.83:0/535186819 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f71cc0047a0 con 0x55df0f4729b0 2024-04-24T09:05:16.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.394+0000 7f71cbe00640 1 -- 172.21.15.83:0/535186819 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71cc010630 con 0x55df0f4729b0 2024-04-24T09:05:16.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.394+0000 7f71d3e00640 1 -- 172.21.15.83:0/2651854513 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7f71c40f1670 con 0x7f71dc076e50 2024-04-24T09:05:16.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.394+0000 7f71db400640 1 --2- 172.21.15.83:0/2651854513 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f71c00a3d20 0x7f71c00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:16.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.395+0000 7f71e4b67740 1 -- 172.21.15.83:0/535186819 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df0f4729b0 msgr2=0x55df0f47cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:16.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.395+0000 7f71e4b67740 1 --2- 172.21.15.83:0/535186819 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df0f4729b0 0x55df0f47cb40 secure :-1 s=READY pgs=2111 cs=0 l=1 rev1=1 crypto rx=0x7f71cc00cc60 tx=0x7f71cc00e720 comp rx=0 tx=0).stop 2024-04-24T09:05:16.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.395+0000 7f71e4b67740 1 -- 172.21.15.83:0/535186819 shutdown_connections 2024-04-24T09:05:16.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.395+0000 7f71e4b67740 1 --2- 172.21.15.83:0/535186819 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55df0f47d080 0x55df0f47f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:16.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.395+0000 7f71e4b67740 1 --2- 172.21.15.83:0/535186819 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df0f4729b0 0x55df0f47cb40 unknown :-1 s=CLOSED pgs=2111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:16.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.395+0000 7f71e4b67740 1 --2- 172.21.15.83:0/535186819 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df0f472090 0x55df0f472470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:16.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.395+0000 7f71e4b67740 1 -- 172.21.15.83:0/535186819 >> 172.21.15.83:0/535186819 conn(0x55df0f46e3b0 msgr2=0x55df0f46e790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:16.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.395+0000 7f71e4b67740 1 -- 172.21.15.83:0/535186819 shutdown_connections 2024-04-24T09:05:16.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.395+0000 7f71db400640 1 --2- 172.21.15.83:0/2651854513 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f71c00a3d20 0x7f71c00a61e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f71dc06b4e0 tx=0x7f71d400b300 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:16.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.395+0000 7f71e4b67740 1 -- 172.21.15.83:0/535186819 wait complete. 2024-04-24T09:05:16.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.395+0000 7f71e4b67740 1 -- start start 2024-04-24T09:05:16.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.396+0000 7f71e4b67740 1 Processor -- start 2024-04-24T09:05:16.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.396+0000 7f71e4b67740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55df0f472090 0x55df0f47a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:16.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.396+0000 7f71e4b67740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df0f4729b0 0x55df0f47a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:16.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.396+0000 7f71e4b67740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df0f47d080 0x55df0f47aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:16.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.396+0000 7f71e4b67740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55df0f47fe70 con 0x55df0f47d080 2024-04-24T09:05:16.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.396+0000 7f71e4b67740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55df0f391660 con 0x55df0f4729b0 2024-04-24T09:05:16.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.396+0000 7f71e4b67740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55df0f47fff0 con 0x55df0f472090 2024-04-24T09:05:16.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.396+0000 7f71e4b67740 10 client.0 _ll_register_callbacks cb 0x55df0f37faa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:05:16.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.396+0000 7f71db400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df0f47d080 0x55df0f47aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:16.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.396+0000 7f71db400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df0f47d080 0x55df0f47aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36286/0 (socket says 172.21.15.83:36286) 2024-04-24T09:05:16.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.396+0000 7f71db400640 1 -- 172.21.15.83:0/2205472840 learned_addr learned my addr 172.21.15.83:0/2205472840 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:05:16.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.397+0000 7f71daa00640 1 --2- 172.21.15.83:0/2205472840 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55df0f472090 0x55df0f47a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:16.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.397+0000 7f71d2000640 1 --2- 172.21.15.83:0/2205472840 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df0f4729b0 0x55df0f47a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:16.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.397+0000 7f71db400640 1 -- 172.21.15.83:0/2205472840 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55df0f472090 msgr2=0x55df0f47a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:16.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.397+0000 7f71db400640 1 --2- 172.21.15.83:0/2205472840 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55df0f472090 0x55df0f47a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:16.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.397+0000 7f71db400640 1 -- 172.21.15.83:0/2205472840 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df0f4729b0 msgr2=0x55df0f47a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:16.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.397+0000 7f71db400640 1 --2- 172.21.15.83:0/2205472840 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df0f4729b0 0x55df0f47a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:16.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.397+0000 7f71db400640 1 -- 172.21.15.83:0/2205472840 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df0f212f70 con 0x55df0f47d080 2024-04-24T09:05:16.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.397+0000 7f71db400640 1 --2- 172.21.15.83:0/2205472840 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df0f47d080 0x55df0f47aa80 secure :-1 s=READY pgs=2112 cs=0 l=1 rev1=1 crypto rx=0x7f71d4011e40 tx=0x7f71d4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28f66390ec2ca83b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:16.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.397+0000 7f71c8c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:05:16.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.397+0000 7f71c8c00640 1 -- 172.21.15.83:0/2205472840 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d401e420 con 0x55df0f47d080 2024-04-24T09:05:16.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.398+0000 7f71c8c00640 1 -- 172.21.15.83:0/2205472840 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7f71d401e5c0 con 0x55df0f47d080 2024-04-24T09:05:16.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.398+0000 7f71e4b67740 1 -- 172.21.15.83:0/2205472840 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71cc0047a0 con 0x55df0f47d080 2024-04-24T09:05:16.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.398+0000 7f71c8c00640 1 -- 172.21.15.83:0/2205472840 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f71d401e7c0 con 0x55df0f47d080 2024-04-24T09:05:16.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.398+0000 7f71e4b67740 10 client.20502 fetch_fsmap learned FSMap version 43 2024-04-24T09:05:16.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.398+0000 7f71e4b67740 1 -- 172.21.15.83:0/2205472840 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55df0f0aa170 con 0x55df0f47d080 2024-04-24T09:05:16.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.398+0000 7f71c8c00640 1 -- 172.21.15.83:0/2205472840 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f71cc0047a0 con 0x55df0f47d080 2024-04-24T09:05:16.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.398+0000 7f71c8c00640 1 -- 172.21.15.83:0/2205472840 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7f71d4026a80 con 0x55df0f47d080 2024-04-24T09:05:16.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.398+0000 7f71e4b67740 10 client.20502 fetch_fsmap finished waiting for FSMap version 43 2024-04-24T09:05:16.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.398+0000 7f71e4b67740 10 client.20502 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:05:16.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.398+0000 7f71e4b67740 20 client.20502 populate_metadata read hostname 'smithi083' 2024-04-24T09:05:16.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.398+0000 7f71e4b67740 1 --2- 172.21.15.83:0/2205472840 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55df0f48d4c0 0x55df0f48f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:16.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.398+0000 7f71e4b67740 4 client.20502 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:05:16.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.398+0000 7f71e4b67740 1 -- 172.21.15.83:0/2205472840 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55df0f3e5890 con 0x55df0f48d4c0 2024-04-24T09:05:16.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.399+0000 7f71daa00640 1 --2- 172.21.15.83:0/2205472840 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55df0f48d4c0 0x55df0f48f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:16.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.401+0000 7f71daa00640 1 --2- 172.21.15.83:0/2205472840 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55df0f48d4c0 0x55df0f48f910 crc :-1 s=READY pgs=353 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f84fa7ff8a021f32 server_cookie=fe4cbfd1485b7e0b in_seq=0 out_seq=0 2024-04-24T09:05:16.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.401+0000 7f71c8c00640 10 client.20502 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:05:16.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.403+0000 7f71c8c00640 1 -- 172.21.15.83:0/2205472840 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55df0f3e5890 con 0x55df0f48d4c0 2024-04-24T09:05:16.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.403+0000 7f71c8c00640 10 client.20502 handle_command_reply: tid=0 2024-04-24T09:05:16.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.403+0000 7f71e4b67740 1 client.20502 shutdown 2024-04-24T09:05:16.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.403+0000 7f71e4b67740 1 -- 172.21.15.83:0/2205472840 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df0f47d080 msgr2=0x55df0f47aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:16.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.403+0000 7f71e4b67740 1 --2- 172.21.15.83:0/2205472840 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df0f47d080 0x55df0f47aa80 secure :-1 s=READY pgs=2112 cs=0 l=1 rev1=1 crypto rx=0x7f71d4011e40 tx=0x7f71d4004a40 comp rx=0 tx=0).stop 2024-04-24T09:05:16.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.403+0000 7f71e4b67740 1 -- 172.21.15.83:0/2205472840 shutdown_connections 2024-04-24T09:05:16.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.403+0000 7f71e4b67740 1 --2- 172.21.15.83:0/2205472840 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55df0f48d4c0 0x55df0f48f910 crc :-1 s=READY pgs=353 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:16.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.403+0000 7f71e4b67740 1 --2- 172.21.15.83:0/2205472840 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df0f47d080 0x55df0f47aa80 unknown :-1 s=CLOSED pgs=2112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:16.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.403+0000 7f71e4b67740 1 --2- 172.21.15.83:0/2205472840 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df0f4729b0 0x55df0f47a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:16.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.403+0000 7f71e4b67740 1 --2- 172.21.15.83:0/2205472840 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55df0f472090 0x55df0f47a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:16.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.403+0000 7f71c8c00640 0 client.20502 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:05:16.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.404+0000 7f71e4b67740 1 -- 172.21.15.83:0/2205472840 >> 172.21.15.83:0/2205472840 conn(0x55df0f46e3b0 msgr2=0x55df0f46d510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:16.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.404+0000 7f71e4b67740 1 -- 172.21.15.83:0/2205472840 shutdown_connections 2024-04-24T09:05:16.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.404+0000 7f71e4b67740 1 -- 172.21.15.83:0/2205472840 wait complete. 2024-04-24T09:05:16.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.404+0000 7f71e4b67740 20 client.20502 trim_cache size 0 max 16384 2024-04-24T09:05:16.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.423+0000 7f71e4b67740 1 Processor -- start 2024-04-24T09:05:16.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.424+0000 7f71e4b67740 1 -- start start 2024-04-24T09:05:16.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.424+0000 7f71e4b67740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55df0f472090 0x55df0f488d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:16.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.424+0000 7f71e4b67740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df0f4729b0 0x55df0f474830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:16.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.424+0000 7f71e4b67740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df0f47d080 0x55df0f495220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:16.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.424+0000 7f71e4b67740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f71dc1b5c40 con 0x55df0f4729b0 2024-04-24T09:05:16.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.424+0000 7f71e4b67740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f71dc1b7840 con 0x55df0f47d080 2024-04-24T09:05:16.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.424+0000 7f71e4b67740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f71dc1b6310 con 0x55df0f472090 2024-04-24T09:05:16.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.424+0000 7f71d2000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df0f4729b0 0x55df0f474830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:16.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.424+0000 7f71d2000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df0f4729b0 0x55df0f474830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36292/0 (socket says 172.21.15.83:36292) 2024-04-24T09:05:16.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.424+0000 7f71d2000640 1 -- 172.21.15.83:0/4204274079 learned_addr learned my addr 172.21.15.83:0/4204274079 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:05:16.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.424+0000 7f71daa00640 1 --2- 172.21.15.83:0/4204274079 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55df0f472090 0x55df0f488d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:16.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.425+0000 7f71d2000640 1 -- 172.21.15.83:0/4204274079 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55df0f472090 msgr2=0x55df0f488d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:16.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.425+0000 7f71d2000640 1 --2- 172.21.15.83:0/4204274079 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55df0f472090 0x55df0f488d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:16.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.425+0000 7f71db400640 1 --2- 172.21.15.83:0/4204274079 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df0f47d080 0x55df0f495220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:16.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.425+0000 7f71d2000640 1 -- 172.21.15.83:0/4204274079 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df0f47d080 msgr2=0x55df0f495220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:16.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.425+0000 7f71d2000640 1 --2- 172.21.15.83:0/4204274079 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df0f47d080 0x55df0f495220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:16.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.425+0000 7f71d2000640 1 -- 172.21.15.83:0/4204274079 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df0f478550 con 0x55df0f4729b0 2024-04-24T09:05:16.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.425+0000 7f71db400640 1 --2- 172.21.15.83:0/4204274079 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df0f47d080 0x55df0f495220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:05:16.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.425+0000 7f71d2000640 1 --2- 172.21.15.83:0/4204274079 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df0f4729b0 0x55df0f474830 secure :-1 s=READY pgs=2113 cs=0 l=1 rev1=1 crypto rx=0x7f71c4005ab0 tx=0x7f71c40319b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30d8b64b03edc815 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:16.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.425+0000 7f71cb400640 1 -- 172.21.15.83:0/4204274079 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c4049040 con 0x55df0f4729b0 2024-04-24T09:05:16.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.425+0000 7f71cb400640 1 -- 172.21.15.83:0/4204274079 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f71c40bd890 con 0x55df0f4729b0 2024-04-24T09:05:16.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.425+0000 7f71cb400640 1 -- 172.21.15.83:0/4204274079 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c40eebc0 con 0x55df0f4729b0 2024-04-24T09:05:16.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.426+0000 7f71e4b67740 1 -- 172.21.15.83:0/4204274079 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df0f4729b0 msgr2=0x55df0f474830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:16.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.426+0000 7f71e4b67740 1 --2- 172.21.15.83:0/4204274079 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df0f4729b0 0x55df0f474830 secure :-1 s=READY pgs=2113 cs=0 l=1 rev1=1 crypto rx=0x7f71c4005ab0 tx=0x7f71c40319b0 comp rx=0 tx=0).stop 2024-04-24T09:05:16.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.426+0000 7f71e4b67740 1 -- 172.21.15.83:0/4204274079 shutdown_connections 2024-04-24T09:05:16.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.426+0000 7f71e4b67740 1 --2- 172.21.15.83:0/4204274079 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df0f47d080 0x55df0f495220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:16.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.426+0000 7f71e4b67740 1 --2- 172.21.15.83:0/4204274079 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df0f4729b0 0x55df0f474830 unknown :-1 s=CLOSED pgs=2113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:16.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.426+0000 7f71e4b67740 1 --2- 172.21.15.83:0/4204274079 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55df0f472090 0x55df0f488d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:16.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.426+0000 7f71e4b67740 1 -- 172.21.15.83:0/4204274079 >> 172.21.15.83:0/4204274079 conn(0x55df0f46e3b0 msgr2=0x55df0f46d510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:16.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.426+0000 7f71e4b67740 1 -- 172.21.15.83:0/4204274079 shutdown_connections 2024-04-24T09:05:16.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.426+0000 7f71e4b67740 1 -- 172.21.15.83:0/4204274079 wait complete. 2024-04-24T09:05:16.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.426+0000 7f71e4b67740 1 -- start start 2024-04-24T09:05:16.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.427+0000 7f71e4b67740 1 Processor -- start 2024-04-24T09:05:16.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.427+0000 7f71e4b67740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55df0f472090 0x55df0f480d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:16.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.427+0000 7f71e4b67740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df0f4729b0 0x55df0f4812c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:16.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.427+0000 7f71e4b67740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df0f47d080 0x55df0f481800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:16.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.427+0000 7f71e4b67740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f71dc1b7110 con 0x55df0f47d080 2024-04-24T09:05:16.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.427+0000 7f71e4b67740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f71dc1b69e0 con 0x55df0f4729b0 2024-04-24T09:05:16.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.427+0000 7f71e4b67740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f71dc1b3c40 con 0x55df0f472090 2024-04-24T09:05:16.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.427+0000 7f71e4b67740 10 client.0 _ll_register_callbacks cb 0x7f71d400b9c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:05:16.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.427+0000 7f71db400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df0f47d080 0x55df0f481800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:16.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.428+0000 7f71db400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df0f47d080 0x55df0f481800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36306/0 (socket says 172.21.15.83:36306) 2024-04-24T09:05:16.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.428+0000 7f71db400640 1 -- 172.21.15.83:0/280698014 learned_addr learned my addr 172.21.15.83:0/280698014 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:05:16.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.428+0000 7f71daa00640 1 --2- 172.21.15.83:0/280698014 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55df0f472090 0x55df0f480d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:16.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.428+0000 7f71d2000640 1 --2- 172.21.15.83:0/280698014 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df0f4729b0 0x55df0f4812c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:16.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.428+0000 7f71db400640 1 -- 172.21.15.83:0/280698014 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55df0f472090 msgr2=0x55df0f480d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:16.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.428+0000 7f71db400640 1 --2- 172.21.15.83:0/280698014 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55df0f472090 0x55df0f480d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:16.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.428+0000 7f71db400640 1 -- 172.21.15.83:0/280698014 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df0f4729b0 msgr2=0x55df0f4812c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:16.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.428+0000 7f71db400640 1 --2- 172.21.15.83:0/280698014 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df0f4729b0 0x55df0f4812c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:16.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.428+0000 7f71db400640 1 -- 172.21.15.83:0/280698014 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df0f4f2b90 con 0x55df0f47d080 2024-04-24T09:05:16.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.428+0000 7f71d2000640 1 --2- 172.21.15.83:0/280698014 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df0f4729b0 0x55df0f4812c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:05:16.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.428+0000 7f71daa00640 1 --2- 172.21.15.83:0/280698014 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55df0f472090 0x55df0f480d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:05:16.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.428+0000 7f71db400640 1 --2- 172.21.15.83:0/280698014 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df0f47d080 0x55df0f481800 secure :-1 s=READY pgs=2114 cs=0 l=1 rev1=1 crypto rx=0x7f71d400b400 tx=0x7f71d400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7ac3563171bd0cc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:16.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.428+0000 7f71c8c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:05:16.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.429+0000 7f71c8c00640 1 -- 172.21.15.83:0/280698014 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d4004cb0 con 0x55df0f47d080 2024-04-24T09:05:16.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.429+0000 7f71c8c00640 1 -- 172.21.15.83:0/280698014 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7f71d401e960 con 0x55df0f47d080 2024-04-24T09:05:16.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.429+0000 7f71e4b67740 1 -- 172.21.15.83:0/280698014 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71c40bd890 con 0x55df0f47d080 2024-04-24T09:05:16.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.429+0000 7f71c8c00640 1 -- 172.21.15.83:0/280698014 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f71d4004e50 con 0x55df0f47d080 2024-04-24T09:05:16.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.429+0000 7f71e4b67740 10 client.20508 fetch_fsmap learned FSMap version 43 2024-04-24T09:05:16.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.429+0000 7f71e4b67740 1 -- 172.21.15.83:0/280698014 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55df0f4f3e20 con 0x55df0f47d080 2024-04-24T09:05:16.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.429+0000 7f71c8c00640 1 -- 172.21.15.83:0/280698014 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f71c40bd890 con 0x55df0f47d080 2024-04-24T09:05:16.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.429+0000 7f71c8c00640 1 -- 172.21.15.83:0/280698014 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7f71d4026520 con 0x55df0f47d080 2024-04-24T09:05:16.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.429+0000 7f71e4b67740 10 client.20508 fetch_fsmap finished waiting for FSMap version 43 2024-04-24T09:05:16.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.429+0000 7f71e4b67740 10 client.20508 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:05:16.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.429+0000 7f71e4b67740 20 client.20508 populate_metadata read hostname 'smithi083' 2024-04-24T09:05:16.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.429+0000 7f71e4b67740 1 --2- 172.21.15.83:0/280698014 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55df0f4f4600 0x55df0f4f6a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:16.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.429+0000 7f71e4b67740 4 client.20508 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T09:05:16.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.429+0000 7f71e4b67740 1 -- 172.21.15.83:0/280698014 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55df0f3df700 con 0x55df0f4f4600 2024-04-24T09:05:16.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.430+0000 7f71daa00640 1 --2- 172.21.15.83:0/280698014 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55df0f4f4600 0x55df0f4f6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:16.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.431+0000 7f71daa00640 1 --2- 172.21.15.83:0/280698014 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55df0f4f4600 0x55df0f4f6a20 crc :-1 s=READY pgs=354 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4acea947605dd880 server_cookie=9bbea809951c1c31 in_seq=0 out_seq=0 2024-04-24T09:05:16.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.431+0000 7f71c8c00640 10 client.20508 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:05:16.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.433+0000 7f71c8c00640 1 -- 172.21.15.83:0/280698014 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55df0f3df700 con 0x55df0f4f4600 2024-04-24T09:05:16.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.433+0000 7f71c8c00640 10 client.20508 handle_command_reply: tid=0 2024-04-24T09:05:16.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.433+0000 7f71e4b67740 1 client.20508 shutdown 2024-04-24T09:05:16.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.433+0000 7f71e4b67740 1 -- 172.21.15.83:0/280698014 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df0f47d080 msgr2=0x55df0f481800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:16.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.433+0000 7f71e4b67740 1 --2- 172.21.15.83:0/280698014 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df0f47d080 0x55df0f481800 secure :-1 s=READY pgs=2114 cs=0 l=1 rev1=1 crypto rx=0x7f71d400b400 tx=0x7f71d400ed60 comp rx=0 tx=0).stop 2024-04-24T09:05:16.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.433+0000 7f71e4b67740 1 -- 172.21.15.83:0/280698014 shutdown_connections 2024-04-24T09:05:16.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.433+0000 7f71e4b67740 1 --2- 172.21.15.83:0/280698014 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55df0f4f4600 0x55df0f4f6a20 crc :-1 s=READY pgs=354 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:16.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.433+0000 7f71e4b67740 1 --2- 172.21.15.83:0/280698014 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df0f47d080 0x55df0f481800 unknown :-1 s=CLOSED pgs=2114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:16.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.433+0000 7f71e4b67740 1 --2- 172.21.15.83:0/280698014 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df0f4729b0 0x55df0f4812c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:16.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.433+0000 7f71c8c00640 0 client.20508 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:05:16.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.433+0000 7f71e4b67740 1 --2- 172.21.15.83:0/280698014 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55df0f472090 0x55df0f480d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:16.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.433+0000 7f71e4b67740 1 -- 172.21.15.83:0/280698014 >> 172.21.15.83:0/280698014 conn(0x55df0f46e3b0 msgr2=0x55df0f479fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:16.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.433+0000 7f71e4b67740 1 -- 172.21.15.83:0/280698014 shutdown_connections 2024-04-24T09:05:16.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.433+0000 7f71e4b67740 1 -- 172.21.15.83:0/280698014 wait complete. 2024-04-24T09:05:16.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.434+0000 7f71e4b67740 20 client.20508 trim_cache size 0 max 16384 2024-04-24T09:05:16.434 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T09:05:16.435 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-04-24T09:05:16.435 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "68ce8d58-3a68-4916-9ea3-ef66b01ca559", 2024-04-24T09:05:16.435 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-04-24T09:05:16.435 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T09:05:16.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.434+0000 7f71e1c00640 1 -- 172.21.15.83:0/2651854513 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f71c00a3d20 msgr2=0x7f71c00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:16.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.434+0000 7f71e1c00640 1 --2- 172.21.15.83:0/2651854513 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f71c00a3d20 0x7f71c00a61e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f71dc06b4e0 tx=0x7f71d400b300 comp rx=0 tx=0).stop 2024-04-24T09:05:16.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.434+0000 7f71e1c00640 1 -- 172.21.15.83:0/2651854513 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71dc076e50 msgr2=0x7f71dc19f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:16.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.434+0000 7f71e1c00640 1 --2- 172.21.15.83:0/2651854513 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71dc076e50 0x7f71dc19f3f0 secure :-1 s=READY pgs=2110 cs=0 l=1 rev1=1 crypto rx=0x7f71c4006010 tx=0x7f71c40042f0 comp rx=0 tx=0).stop 2024-04-24T09:05:16.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.434+0000 7f71e1c00640 1 -- 172.21.15.83:0/2651854513 shutdown_connections 2024-04-24T09:05:16.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.434+0000 7f71e1c00640 1 --2- 172.21.15.83:0/2651854513 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f71c00a3d20 0x7f71c00a61e0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:16.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.434+0000 7f71e1c00640 1 --2- 172.21.15.83:0/2651854513 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f71dc112ee0 0x7f71dc1a3720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:16.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.434+0000 7f71e1c00640 1 --2- 172.21.15.83:0/2651854513 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71dc076e50 0x7f71dc19f3f0 unknown :-1 s=CLOSED pgs=2110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:16.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.434+0000 7f71e1c00640 1 --2- 172.21.15.83:0/2651854513 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f71dc076530 0x7f71dc19eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:16.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.434+0000 7f71e1c00640 1 -- 172.21.15.83:0/2651854513 >> 172.21.15.83:0/2651854513 conn(0x7f71dc0ffd90 msgr2=0x7f71dc1145a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:16.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.434+0000 7f71e1c00640 1 -- 172.21.15.83:0/2651854513 shutdown_connections 2024-04-24T09:05:16.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.435+0000 7f71e1c00640 1 -- 172.21.15.83:0/2651854513 wait complete. 2024-04-24T09:05:16.453 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T09:05:16.453 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T09:05:16.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:05:16 smithi138 ceph-mon[31441]: pgmap v428: 129 pgs: 129 active+clean; 74 MiB data, 997 MiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 326 KiB/s wr, 33 op/s 2024-04-24T09:05:16.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:05:16 smithi189 ceph-mon[31791]: pgmap v428: 129 pgs: 129 active+clean; 74 MiB data, 997 MiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 326 KiB/s wr, 33 op/s 2024-04-24T09:05:16.632 INFO:tasks.workunit.client.0.smithi083.stdout: 1 78236 0.58 MB/sec execute 590 sec latency 2721.834 ms 2024-04-24T09:05:16.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.671+0000 7fb1bf200640 1 -- 172.21.15.83:0/232442863 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb1c01071f0 msgr2=0x7fb1c010edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:16.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.671+0000 7fb1bf200640 1 --2- 172.21.15.83:0/232442863 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb1c01071f0 0x7fb1c010edc0 secure :-1 s=READY pgs=2115 cs=0 l=1 rev1=1 crypto rx=0x7fb1b8009ca0 tx=0x7fb1b802a8c0 comp rx=0 tx=0).stop 2024-04-24T09:05:16.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.671+0000 7fb1bf200640 1 -- 172.21.15.83:0/232442863 shutdown_connections 2024-04-24T09:05:16.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.671+0000 7fb1bf200640 1 --2- 172.21.15.83:0/232442863 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb1c01071f0 0x7fb1c010edc0 unknown :-1 s=CLOSED pgs=2115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:16.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.671+0000 7fb1bf200640 1 --2- 172.21.15.83:0/232442863 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb1c0102ad0 0x7fb1c0106b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:16.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.671+0000 7fb1bf200640 1 --2- 172.21.15.83:0/232442863 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb1c0102120 0x7fb1c0102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:16.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.671+0000 7fb1bf200640 1 -- 172.21.15.83:0/232442863 >> 172.21.15.83:0/232442863 conn(0x7fb1c00fd9a0 msgr2=0x7fb1c00ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:16.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.671+0000 7fb1bf200640 1 -- 172.21.15.83:0/232442863 shutdown_connections 2024-04-24T09:05:16.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.671+0000 7fb1bf200640 1 -- 172.21.15.83:0/232442863 wait complete. 2024-04-24T09:05:16.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.672+0000 7fb1bf200640 1 Processor -- start 2024-04-24T09:05:16.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.672+0000 7fb1bf200640 1 -- start start 2024-04-24T09:05:16.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.672+0000 7fb1bf200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb1c0102120 0x7fb1c01a05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:16.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.672+0000 7fb1bf200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb1c01071f0 0x7fb1c01a0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:16.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.672+0000 7fb1bf200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb1c019a710 0x7fb1c019ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:16.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.672+0000 7fb1bf200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb1c01121b0 con 0x7fb1c019a710 2024-04-24T09:05:16.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.672+0000 7fb1bf200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb1c0112030 con 0x7fb1c01071f0 2024-04-24T09:05:16.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.672+0000 7fb1bf200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb1c0112330 con 0x7fb1c0102120 2024-04-24T09:05:16.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.673+0000 7fb1be800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb1c019a710 0x7fb1c019ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:16.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.673+0000 7fb1be800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb1c019a710 0x7fb1c019ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36322/0 (socket says 172.21.15.83:36322) 2024-04-24T09:05:16.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.673+0000 7fb1be800640 1 -- 172.21.15.83:0/4032631400 learned_addr learned my addr 172.21.15.83:0/4032631400 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:05:16.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.673+0000 7fb1bde00640 1 --2- 172.21.15.83:0/4032631400 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb1c0102120 0x7fb1c01a05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:16.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.673+0000 7fb1be800640 1 -- 172.21.15.83:0/4032631400 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb1c0102120 msgr2=0x7fb1c01a05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:16.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.673+0000 7fb1be800640 1 --2- 172.21.15.83:0/4032631400 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb1c0102120 0x7fb1c01a05b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:16.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.673+0000 7fb1be800640 1 -- 172.21.15.83:0/4032631400 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb1c01071f0 msgr2=0x7fb1c01a0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:16.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.673+0000 7fb1b7e00640 1 --2- 172.21.15.83:0/4032631400 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb1c01071f0 0x7fb1c01a0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:16.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.673+0000 7fb1be800640 1 --2- 172.21.15.83:0/4032631400 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb1c01071f0 0x7fb1c01a0af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:16.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.673+0000 7fb1be800640 1 -- 172.21.15.83:0/4032631400 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1c019b310 con 0x7fb1c019a710 2024-04-24T09:05:16.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.674+0000 7fb1be800640 1 --2- 172.21.15.83:0/4032631400 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb1c019a710 0x7fb1c019ab90 secure :-1 s=READY pgs=2116 cs=0 l=1 rev1=1 crypto rx=0x7fb1b8031040 tx=0x7fb1b8003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:16.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.674+0000 7fb1b6a00640 1 -- 172.21.15.83:0/4032631400 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1b8003ce0 con 0x7fb1c019a710 2024-04-24T09:05:16.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.674+0000 7fb1b6a00640 1 -- 172.21.15.83:0/4032631400 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb1b80044f0 con 0x7fb1c019a710 2024-04-24T09:05:16.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.674+0000 7fb1bf200640 1 -- 172.21.15.83:0/4032631400 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1c019b5a0 con 0x7fb1c019a710 2024-04-24T09:05:16.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.674+0000 7fb1b6a00640 1 -- 172.21.15.83:0/4032631400 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1b8030c90 con 0x7fb1c019a710 2024-04-24T09:05:16.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.674+0000 7fb1bf200640 1 -- 172.21.15.83:0/4032631400 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1c01a76a0 con 0x7fb1c019a710 2024-04-24T09:05:16.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.677+0000 7fb1c7857740 1 Processor -- start 2024-04-24T09:05:16.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.677+0000 7fb1b6a00640 1 -- 172.21.15.83:0/4032631400 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb1b80046a0 con 0x7fb1c019a710 2024-04-24T09:05:16.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.678+0000 7fb1c7857740 1 -- start start 2024-04-24T09:05:16.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.678+0000 7fb1c7857740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55da67ad4a30 0x55da67ad4e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:16.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.678+0000 7fb1c7857740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da67ad5350 0x55da67adf4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:16.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.678+0000 7fb1c7857740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da67adfa20 0x55da67ae1e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:16.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.678+0000 7fb1c7857740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55da67a3a1f0 con 0x55da67ad4a30 2024-04-24T09:05:16.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.678+0000 7fb1c7857740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55da679e0350 con 0x55da67ad5350 2024-04-24T09:05:16.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.678+0000 7fb1c7857740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55da67a361b0 con 0x55da67adfa20 2024-04-24T09:05:16.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.678+0000 7fb1b7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55da67ad4a30 0x55da67ad4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:16.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.678+0000 7fb1b7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55da67ad4a30 0x55da67ad4e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36336/0 (socket says 172.21.15.83:36336) 2024-04-24T09:05:16.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.678+0000 7fb1b7e00640 1 -- 172.21.15.83:0/1784411135 learned_addr learned my addr 172.21.15.83:0/1784411135 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:05:16.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.679+0000 7fb1bde00640 1 --2- 172.21.15.83:0/1784411135 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da67ad5350 0x55da67adf4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:16.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.679+0000 7fb1b7e00640 1 -- 172.21.15.83:0/1784411135 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da67adfa20 msgr2=0x55da67ae1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:16.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.679+0000 7fb1b7e00640 1 --2- 172.21.15.83:0/1784411135 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da67adfa20 0x55da67ae1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:16.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.679+0000 7fb1b7e00640 1 -- 172.21.15.83:0/1784411135 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da67ad5350 msgr2=0x55da67adf4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:16.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.679+0000 7fb1b7e00640 1 --2- 172.21.15.83:0/1784411135 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da67ad5350 0x55da67adf4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:16.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.679+0000 7fb1b7e00640 1 -- 172.21.15.83:0/1784411135 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da67a34f50 con 0x55da67ad4a30 2024-04-24T09:05:16.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.679+0000 7fb1b6a00640 1 --2- 172.21.15.83:0/4032631400 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb1a40a3d20 0x7fb1a40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:16.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.679+0000 7fb1b7e00640 1 --2- 172.21.15.83:0/1784411135 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55da67ad4a30 0x55da67ad4e10 secure :-1 s=READY pgs=2117 cs=0 l=1 rev1=1 crypto rx=0x7fb1a8009b50 tx=0x7fb1a80106e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f560a383619f5c58 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:16.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.680+0000 7fb1b6a00640 1 -- 172.21.15.83:0/4032631400 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7fb1b80f2230 con 0x7fb1c019a710 2024-04-24T09:05:16.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.680+0000 7fb1af000640 1 -- 172.21.15.83:0/1784411135 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a8016070 con 0x55da67ad4a30 2024-04-24T09:05:16.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.680+0000 7fb1bde00640 1 --2- 172.21.15.83:0/4032631400 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb1a40a3d20 0x7fb1a40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:16.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.680+0000 7fb1af000640 1 -- 172.21.15.83:0/1784411135 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb1a8009e40 con 0x55da67ad4a30 2024-04-24T09:05:16.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.680+0000 7fb1af000640 1 -- 172.21.15.83:0/1784411135 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a8004de0 con 0x55da67ad4a30 2024-04-24T09:05:16.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.680+0000 7fb1c7857740 1 -- 172.21.15.83:0/1784411135 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55da67ad4a30 msgr2=0x55da67ad4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:16.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.680+0000 7fb1c7857740 1 --2- 172.21.15.83:0/1784411135 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55da67ad4a30 0x55da67ad4e10 secure :-1 s=READY pgs=2117 cs=0 l=1 rev1=1 crypto rx=0x7fb1a8009b50 tx=0x7fb1a80106e0 comp rx=0 tx=0).stop 2024-04-24T09:05:16.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.680+0000 7fb1bde00640 1 --2- 172.21.15.83:0/4032631400 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb1a40a3d20 0x7fb1a40a61e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fb1b0005fd0 tx=0x7fb1b0005850 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:16.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.680+0000 7fb1c7857740 1 -- 172.21.15.83:0/1784411135 shutdown_connections 2024-04-24T09:05:16.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.680+0000 7fb1c7857740 1 --2- 172.21.15.83:0/1784411135 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da67adfa20 0x55da67ae1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:16.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.680+0000 7fb1c7857740 1 --2- 172.21.15.83:0/1784411135 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da67ad5350 0x55da67adf4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:16.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.680+0000 7fb1c7857740 1 --2- 172.21.15.83:0/1784411135 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55da67ad4a30 0x55da67ad4e10 unknown :-1 s=CLOSED pgs=2117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:16.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.681+0000 7fb1c7857740 1 -- 172.21.15.83:0/1784411135 >> 172.21.15.83:0/1784411135 conn(0x55da67ad0d50 msgr2=0x55da67ad1130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:16.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.681+0000 7fb1c7857740 1 -- 172.21.15.83:0/1784411135 shutdown_connections 2024-04-24T09:05:16.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.681+0000 7fb1c7857740 1 -- 172.21.15.83:0/1784411135 wait complete. 2024-04-24T09:05:16.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.681+0000 7fb1c7857740 1 -- start start 2024-04-24T09:05:16.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.681+0000 7fb1c7857740 1 Processor -- start 2024-04-24T09:05:16.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.682+0000 7fb1c7857740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da67ad4a30 0x55da67adc9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:16.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.682+0000 7fb1c7857740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da67ad5350 0x55da67adcf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:16.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.682+0000 7fb1c7857740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55da67adfa20 0x55da67add460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:16.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.682+0000 7fb1c7857740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55da67ae2810 con 0x55da67adfa20 2024-04-24T09:05:16.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.682+0000 7fb1c7857740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55da679f3690 con 0x55da67ad5350 2024-04-24T09:05:16.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.682+0000 7fb1c7857740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55da67ae2990 con 0x55da67ad4a30 2024-04-24T09:05:16.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.682+0000 7fb1c7857740 10 client.0 _ll_register_callbacks cb 0x55da67a2f650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:05:16.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.682+0000 7fb1be800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55da67adfa20 0x55da67add460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:16.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.682+0000 7fb1be800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55da67adfa20 0x55da67add460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36344/0 (socket says 172.21.15.83:36344) 2024-04-24T09:05:16.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.682+0000 7fb1be800640 1 -- 172.21.15.83:0/3446410443 learned_addr learned my addr 172.21.15.83:0/3446410443 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:05:16.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.682+0000 7fb1bde00640 1 --2- 172.21.15.83:0/3446410443 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da67ad4a30 0x55da67adc9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:16.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.682+0000 7fb1b7e00640 1 --2- 172.21.15.83:0/3446410443 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da67ad5350 0x55da67adcf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:16.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.683+0000 7fb1be800640 1 -- 172.21.15.83:0/3446410443 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da67ad4a30 msgr2=0x55da67adc9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:16.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.683+0000 7fb1be800640 1 --2- 172.21.15.83:0/3446410443 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da67ad4a30 0x55da67adc9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:16.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.683+0000 7fb1be800640 1 -- 172.21.15.83:0/3446410443 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da67ad5350 msgr2=0x55da67adcf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:16.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.683+0000 7fb1be800640 1 --2- 172.21.15.83:0/3446410443 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da67ad5350 0x55da67adcf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:16.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.683+0000 7fb1be800640 1 -- 172.21.15.83:0/3446410443 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da679f2920 con 0x55da67adfa20 2024-04-24T09:05:16.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.683+0000 7fb1be800640 1 --2- 172.21.15.83:0/3446410443 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55da67adfa20 0x55da67add460 secure :-1 s=READY pgs=2118 cs=0 l=1 rev1=1 crypto rx=0x7fb1b80ba2c0 tx=0x7fb1b8009990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aad40d1be403da83 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:16.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.683+0000 7fb1a3400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:05:16.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.683+0000 7fb1a3400640 1 -- 172.21.15.83:0/3446410443 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1b80be560 con 0x55da67adfa20 2024-04-24T09:05:16.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.683+0000 7fb1a3400640 1 -- 172.21.15.83:0/3446410443 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7fb1b80ba4d0 con 0x55da67adfa20 2024-04-24T09:05:16.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.683+0000 7fb1c7857740 1 -- 172.21.15.83:0/3446410443 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1a8009e40 con 0x55da67adfa20 2024-04-24T09:05:16.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.684+0000 7fb1a3400640 1 -- 172.21.15.83:0/3446410443 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fb1b80bde60 con 0x55da67adfa20 2024-04-24T09:05:16.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.684+0000 7fb1c7857740 10 client.20520 fetch_fsmap learned FSMap version 43 2024-04-24T09:05:16.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.684+0000 7fb1c7857740 1 -- 172.21.15.83:0/3446410443 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55da678167d0 con 0x55da67adfa20 2024-04-24T09:05:16.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.684+0000 7fb1a3400640 1 -- 172.21.15.83:0/3446410443 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb1a8009e40 con 0x55da67adfa20 2024-04-24T09:05:16.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.684+0000 7fb1a3400640 1 -- 172.21.15.83:0/3446410443 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7fb1b80c6640 con 0x55da67adfa20 2024-04-24T09:05:16.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.684+0000 7fb1c7857740 10 client.20520 fetch_fsmap finished waiting for FSMap version 43 2024-04-24T09:05:16.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.684+0000 7fb1c7857740 10 client.20520 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:05:16.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.684+0000 7fb1c7857740 20 client.20520 populate_metadata read hostname 'smithi083' 2024-04-24T09:05:16.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.684+0000 7fb1c7857740 1 --2- 172.21.15.83:0/3446410443 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55da67aefe20 0x55da67af2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:16.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.684+0000 7fb1c7857740 4 client.20520 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:05:16.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.684+0000 7fb1c7857740 1 -- 172.21.15.83:0/3446410443 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55da67a47c10 con 0x55da67aefe20 2024-04-24T09:05:16.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.685+0000 7fb1bde00640 1 --2- 172.21.15.83:0/3446410443 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55da67aefe20 0x55da67af2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:16.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.686+0000 7fb1bde00640 1 --2- 172.21.15.83:0/3446410443 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55da67aefe20 0x55da67af2280 crc :-1 s=READY pgs=355 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18b77d798a3fcca0 server_cookie=b48244e256ea9752 in_seq=0 out_seq=0 2024-04-24T09:05:16.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.686+0000 7fb1a3400640 10 client.20520 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:05:16.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.688+0000 7fb1a3400640 1 -- 172.21.15.83:0/3446410443 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55da67a47c10 con 0x55da67aefe20 2024-04-24T09:05:16.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.688+0000 7fb1a3400640 10 client.20520 handle_command_reply: tid=0 2024-04-24T09:05:16.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.688+0000 7fb1c7857740 1 client.20520 shutdown 2024-04-24T09:05:16.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.688+0000 7fb1c7857740 1 -- 172.21.15.83:0/3446410443 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55da67adfa20 msgr2=0x55da67add460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:16.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.688+0000 7fb1c7857740 1 --2- 172.21.15.83:0/3446410443 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55da67adfa20 0x55da67add460 secure :-1 s=READY pgs=2118 cs=0 l=1 rev1=1 crypto rx=0x7fb1b80ba2c0 tx=0x7fb1b8009990 comp rx=0 tx=0).stop 2024-04-24T09:05:16.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.688+0000 7fb1c7857740 1 -- 172.21.15.83:0/3446410443 shutdown_connections 2024-04-24T09:05:16.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.688+0000 7fb1c7857740 1 --2- 172.21.15.83:0/3446410443 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55da67aefe20 0x55da67af2280 crc :-1 s=READY pgs=355 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:16.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.688+0000 7fb1c7857740 1 --2- 172.21.15.83:0/3446410443 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55da67adfa20 0x55da67add460 unknown :-1 s=CLOSED pgs=2118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:16.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.688+0000 7fb1c7857740 1 --2- 172.21.15.83:0/3446410443 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da67ad5350 0x55da67adcf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:16.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.688+0000 7fb1c7857740 1 --2- 172.21.15.83:0/3446410443 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da67ad4a30 0x55da67adc9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:16.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.688+0000 7fb1c7857740 1 -- 172.21.15.83:0/3446410443 >> 172.21.15.83:0/3446410443 conn(0x55da67ad0d50 msgr2=0x55da67acffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:16.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.688+0000 7fb1a3400640 0 client.20520 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:05:16.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.688+0000 7fb1c7857740 1 -- 172.21.15.83:0/3446410443 shutdown_connections 2024-04-24T09:05:16.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.688+0000 7fb1c7857740 1 -- 172.21.15.83:0/3446410443 wait complete. 2024-04-24T09:05:16.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.689+0000 7fb1c7857740 20 client.20520 trim_cache size 0 max 16384 2024-04-24T09:05:16.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.708+0000 7fb1c7857740 1 Processor -- start 2024-04-24T09:05:16.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.708+0000 7fb1c7857740 1 -- start start 2024-04-24T09:05:16.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.709+0000 7fb1c7857740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da67ad4a30 0x55da67aeb6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:16.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.709+0000 7fb1c7857740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da67ad5350 0x55da67ad6890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:16.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.709+0000 7fb1c7857740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55da67adfa20 0x55da67af7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:16.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.709+0000 7fb1c7857740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb1c01b1570 con 0x55da67adfa20 2024-04-24T09:05:16.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.709+0000 7fb1c7857740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb1c01b3170 con 0x55da67ad5350 2024-04-24T09:05:16.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.709+0000 7fb1c7857740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb1c01b1c40 con 0x55da67ad4a30 2024-04-24T09:05:16.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.709+0000 7fb1be800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55da67adfa20 0x55da67af7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:16.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.709+0000 7fb1be800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55da67adfa20 0x55da67af7b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36358/0 (socket says 172.21.15.83:36358) 2024-04-24T09:05:16.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.709+0000 7fb1be800640 1 -- 172.21.15.83:0/4005620428 learned_addr learned my addr 172.21.15.83:0/4005620428 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:05:16.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.709+0000 7fb1bde00640 1 --2- 172.21.15.83:0/4005620428 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da67ad4a30 0x55da67aeb6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:16.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.710+0000 7fb1b7e00640 1 --2- 172.21.15.83:0/4005620428 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da67ad5350 0x55da67ad6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:16.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.710+0000 7fb1be800640 1 -- 172.21.15.83:0/4005620428 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da67ad4a30 msgr2=0x55da67aeb6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:16.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.710+0000 7fb1be800640 1 --2- 172.21.15.83:0/4005620428 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da67ad4a30 0x55da67aeb6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:16.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.710+0000 7fb1be800640 1 -- 172.21.15.83:0/4005620428 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da67ad5350 msgr2=0x55da67ad6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:16.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.710+0000 7fb1be800640 1 --2- 172.21.15.83:0/4005620428 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da67ad5350 0x55da67ad6890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:16.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.710+0000 7fb1be800640 1 -- 172.21.15.83:0/4005620428 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da6774c3f0 con 0x55da67adfa20 2024-04-24T09:05:16.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.710+0000 7fb1be800640 1 --2- 172.21.15.83:0/4005620428 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55da67adfa20 0x55da67af7b90 secure :-1 s=READY pgs=2119 cs=0 l=1 rev1=1 crypto rx=0x7fb1b802ae80 tx=0x7fb1b80bc370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24734b4688d97e91 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:16.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.710+0000 7fb1ae600640 1 -- 172.21.15.83:0/4005620428 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1b80a1070 con 0x55da67adfa20 2024-04-24T09:05:16.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.710+0000 7fb1ae600640 1 -- 172.21.15.83:0/4005620428 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb1b80d7070 con 0x55da67adfa20 2024-04-24T09:05:16.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.710+0000 7fb1ae600640 1 -- 172.21.15.83:0/4005620428 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1b80c6ad0 con 0x55da67adfa20 2024-04-24T09:05:16.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.711+0000 7fb1c7857740 1 -- 172.21.15.83:0/4005620428 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55da67adfa20 msgr2=0x55da67af7b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:16.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.711+0000 7fb1c7857740 1 --2- 172.21.15.83:0/4005620428 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55da67adfa20 0x55da67af7b90 secure :-1 s=READY pgs=2119 cs=0 l=1 rev1=1 crypto rx=0x7fb1b802ae80 tx=0x7fb1b80bc370 comp rx=0 tx=0).stop 2024-04-24T09:05:16.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.711+0000 7fb1c7857740 1 -- 172.21.15.83:0/4005620428 shutdown_connections 2024-04-24T09:05:16.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.711+0000 7fb1c7857740 1 --2- 172.21.15.83:0/4005620428 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55da67adfa20 0x55da67af7b90 unknown :-1 s=CLOSED pgs=2119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:16.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.711+0000 7fb1c7857740 1 --2- 172.21.15.83:0/4005620428 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da67ad5350 0x55da67ad6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:16.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.711+0000 7fb1c7857740 1 --2- 172.21.15.83:0/4005620428 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da67ad4a30 0x55da67aeb6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:16.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.711+0000 7fb1c7857740 1 -- 172.21.15.83:0/4005620428 >> 172.21.15.83:0/4005620428 conn(0x55da67ad0d50 msgr2=0x55da67ad6180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:16.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.711+0000 7fb1c7857740 1 -- 172.21.15.83:0/4005620428 shutdown_connections 2024-04-24T09:05:16.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.711+0000 7fb1c7857740 1 -- 172.21.15.83:0/4005620428 wait complete. 2024-04-24T09:05:16.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.712+0000 7fb1c7857740 1 -- start start 2024-04-24T09:05:16.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.712+0000 7fb1c7857740 1 Processor -- start 2024-04-24T09:05:16.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.712+0000 7fb1c7857740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da67ad4a30 0x55da67ae7c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:16.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.712+0000 7fb1c7857740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55da67ad5350 0x55da67ae3120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:16.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.712+0000 7fb1c7857740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da67adfa20 0x55da67ae3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:16.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.712+0000 7fb1c7857740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb1c01b2a40 con 0x55da67ad5350 2024-04-24T09:05:16.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.712+0000 7fb1c7857740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb1c01b2310 con 0x55da67adfa20 2024-04-24T09:05:16.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.712+0000 7fb1c7857740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb1c01af570 con 0x55da67ad4a30 2024-04-24T09:05:16.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.713+0000 7fb1c7857740 10 client.0 _ll_register_callbacks cb 0x7fb1b8003e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:05:16.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.713+0000 7fb1b7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55da67ad5350 0x55da67ae3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:16.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.713+0000 7fb1b7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55da67ad5350 0x55da67ae3120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36362/0 (socket says 172.21.15.83:36362) 2024-04-24T09:05:16.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.713+0000 7fb1b7e00640 1 -- 172.21.15.83:0/145010140 learned_addr learned my addr 172.21.15.83:0/145010140 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:05:16.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.713+0000 7fb1bde00640 1 --2- 172.21.15.83:0/145010140 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da67ad4a30 0x55da67ae7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:16.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.713+0000 7fb1be800640 1 --2- 172.21.15.83:0/145010140 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da67adfa20 0x55da67ae3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:16.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.713+0000 7fb1b7e00640 1 -- 172.21.15.83:0/145010140 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da67ad4a30 msgr2=0x55da67ae7c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:16.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.713+0000 7fb1b7e00640 1 --2- 172.21.15.83:0/145010140 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da67ad4a30 0x55da67ae7c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:16.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.713+0000 7fb1b7e00640 1 -- 172.21.15.83:0/145010140 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da67adfa20 msgr2=0x55da67ae3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:16.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.713+0000 7fb1b7e00640 1 --2- 172.21.15.83:0/145010140 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da67adfa20 0x55da67ae3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:16.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.713+0000 7fb1b7e00640 1 -- 172.21.15.83:0/145010140 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da6770c0f0 con 0x55da67ad5350 2024-04-24T09:05:16.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.714+0000 7fb1b7e00640 1 --2- 172.21.15.83:0/145010140 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55da67ad5350 0x55da67ae3120 secure :-1 s=READY pgs=2120 cs=0 l=1 rev1=1 crypto rx=0x7fb1a8011010 tx=0x7fb1a8009ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85b2b4b998257ece server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:16.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.714+0000 7fb1a3400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:05:16.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.714+0000 7fb1a3400640 1 -- 172.21.15.83:0/145010140 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a802b060 con 0x55da67ad5350 2024-04-24T09:05:16.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.714+0000 7fb1a3400640 1 -- 172.21.15.83:0/145010140 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7fb1a8012620 con 0x55da67ad5350 2024-04-24T09:05:16.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.714+0000 7fb1c7857740 1 -- 172.21.15.83:0/145010140 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1b80d7070 con 0x55da67ad5350 2024-04-24T09:05:16.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.715+0000 7fb1a3400640 1 -- 172.21.15.83:0/145010140 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fb1a8032050 con 0x55da67ad5350 2024-04-24T09:05:16.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.715+0000 7fb1c7857740 10 client.20526 fetch_fsmap learned FSMap version 43 2024-04-24T09:05:16.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.715+0000 7fb1c7857740 1 -- 172.21.15.83:0/145010140 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55da679ff810 con 0x55da67ad5350 2024-04-24T09:05:16.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.715+0000 7fb1a3400640 1 -- 172.21.15.83:0/145010140 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb1b80d7070 con 0x55da67ad5350 2024-04-24T09:05:16.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.715+0000 7fb1a3400640 1 -- 172.21.15.83:0/145010140 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7fb1a801a820 con 0x55da67ad5350 2024-04-24T09:05:16.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.715+0000 7fb1c7857740 10 client.20526 fetch_fsmap finished waiting for FSMap version 43 2024-04-24T09:05:16.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.715+0000 7fb1c7857740 10 client.20526 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:05:16.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.715+0000 7fb1c7857740 20 client.20526 populate_metadata read hostname 'smithi083' 2024-04-24T09:05:16.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.715+0000 7fb1c7857740 1 --2- 172.21.15.83:0/145010140 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55da67b562f0 0x55da67b58710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:16.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.715+0000 7fb1c7857740 4 client.20526 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T09:05:16.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.715+0000 7fb1c7857740 1 -- 172.21.15.83:0/145010140 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55da67a41a80 con 0x55da67b562f0 2024-04-24T09:05:16.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.716+0000 7fb1bde00640 1 --2- 172.21.15.83:0/145010140 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55da67b562f0 0x55da67b58710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:16.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.717+0000 7fb1bde00640 1 --2- 172.21.15.83:0/145010140 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55da67b562f0 0x55da67b58710 crc :-1 s=READY pgs=356 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80ba6a371a3f287c server_cookie=1e17f0fef33ec8f in_seq=0 out_seq=0 2024-04-24T09:05:16.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.717+0000 7fb1a3400640 10 client.20526 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:05:16.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.718+0000 7fb1a3400640 1 -- 172.21.15.83:0/145010140 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55da67a41a80 con 0x55da67b562f0 2024-04-24T09:05:16.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.718+0000 7fb1a3400640 10 client.20526 handle_command_reply: tid=0 2024-04-24T09:05:16.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.718+0000 7fb1c7857740 1 client.20526 shutdown 2024-04-24T09:05:16.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.718+0000 7fb1c7857740 1 -- 172.21.15.83:0/145010140 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55da67ad5350 msgr2=0x55da67ae3120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:16.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.718+0000 7fb1c7857740 1 --2- 172.21.15.83:0/145010140 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55da67ad5350 0x55da67ae3120 secure :-1 s=READY pgs=2120 cs=0 l=1 rev1=1 crypto rx=0x7fb1a8011010 tx=0x7fb1a8009ee0 comp rx=0 tx=0).stop 2024-04-24T09:05:16.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.718+0000 7fb1c7857740 1 -- 172.21.15.83:0/145010140 shutdown_connections 2024-04-24T09:05:16.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.718+0000 7fb1c7857740 1 --2- 172.21.15.83:0/145010140 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55da67b562f0 0x55da67b58710 crc :-1 s=READY pgs=356 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:16.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.718+0000 7fb1c7857740 1 --2- 172.21.15.83:0/145010140 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da67adfa20 0x55da67ae3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:16.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.718+0000 7fb1c7857740 1 --2- 172.21.15.83:0/145010140 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55da67ad5350 0x55da67ae3120 unknown :-1 s=CLOSED pgs=2120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:16.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.718+0000 7fb1c7857740 1 --2- 172.21.15.83:0/145010140 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da67ad4a30 0x55da67ae7c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:16.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.718+0000 7fb1a3400640 0 client.20526 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:05:16.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.718+0000 7fb1c7857740 1 -- 172.21.15.83:0/145010140 >> 172.21.15.83:0/145010140 conn(0x55da67ad0d50 msgr2=0x55da67add230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:16.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.718+0000 7fb1c7857740 1 -- 172.21.15.83:0/145010140 shutdown_connections 2024-04-24T09:05:16.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.718+0000 7fb1c7857740 1 -- 172.21.15.83:0/145010140 wait complete. 2024-04-24T09:05:16.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.718+0000 7fb1c7857740 20 client.20526 trim_cache size 0 max 16384 2024-04-24T09:05:16.719 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T09:05:16.719 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T09:05:16.719 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-04-24T09:05:16.719 INFO:teuthology.orchestra.run.smithi083.stdout: "68ce8d58-3a68-4916-9ea3-ef66b01ca559": { 2024-04-24T09:05:16.719 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-04-24T09:05:16.719 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "68ce8d58-3a68-4916-9ea3-ef66b01ca559", 2024-04-24T09:05:16.719 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-04-24T09:05:16.719 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:05:16.720 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:05:16.720 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T09:05:16.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.719+0000 7fb1bf200640 1 -- 172.21.15.83:0/4032631400 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb1a40a3d20 msgr2=0x7fb1a40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:16.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.719+0000 7fb1bf200640 1 --2- 172.21.15.83:0/4032631400 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb1a40a3d20 0x7fb1a40a61e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fb1b0005fd0 tx=0x7fb1b0005850 comp rx=0 tx=0).stop 2024-04-24T09:05:16.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.719+0000 7fb1bf200640 1 -- 172.21.15.83:0/4032631400 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb1c019a710 msgr2=0x7fb1c019ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:16.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.719+0000 7fb1bf200640 1 --2- 172.21.15.83:0/4032631400 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb1c019a710 0x7fb1c019ab90 secure :-1 s=READY pgs=2116 cs=0 l=1 rev1=1 crypto rx=0x7fb1b8031040 tx=0x7fb1b8003970 comp rx=0 tx=0).stop 2024-04-24T09:05:16.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.719+0000 7fb1bf200640 1 -- 172.21.15.83:0/4032631400 shutdown_connections 2024-04-24T09:05:16.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.719+0000 7fb1bf200640 1 --2- 172.21.15.83:0/4032631400 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb1a40a3d20 0x7fb1a40a61e0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:16.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.719+0000 7fb1bf200640 1 --2- 172.21.15.83:0/4032631400 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb1c019a710 0x7fb1c019ab90 unknown :-1 s=CLOSED pgs=2116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:16.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.719+0000 7fb1bf200640 1 --2- 172.21.15.83:0/4032631400 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb1c01071f0 0x7fb1c01a0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:16.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.719+0000 7fb1bf200640 1 --2- 172.21.15.83:0/4032631400 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb1c0102120 0x7fb1c01a05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:16.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.719+0000 7fb1bf200640 1 -- 172.21.15.83:0/4032631400 >> 172.21.15.83:0/4032631400 conn(0x7fb1c00fd9a0 msgr2=0x7fb1c00fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:16.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.719+0000 7fb1bf200640 1 -- 172.21.15.83:0/4032631400 shutdown_connections 2024-04-24T09:05:16.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:16.719+0000 7fb1bf200640 1 -- 172.21.15.83:0/4032631400 wait complete. 2024-04-24T09:05:16.737 INFO:tasks.cephfs.filesystem:scrub status for tag:68ce8d58-3a68-4916-9ea3-ef66b01ca559 - {'path': '/', 'tag': '68ce8d58-3a68-4916-9ea3-ef66b01ca559', 'options': 'recursive,force'} 2024-04-24T09:05:16.955 INFO:journalctl@ceph.mds.b.smithi138.stdout:Apr 24 09:05:16 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-b[61515]: 2024-04-24T09:05:16.468+0000 7f7fbbc00700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000208 [...17,head] /volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/WORD/ auth{2=1} v13401 ap=2 f(v504 m2024-04-24T09:04:50.368133+0000 5=5+0) n(v47 rc2024-04-24T09:04:50.368133+0000 b814326 6=5+1) old_inodes=7 (inest sync->mix g=2) caps={15603=pAsLsXsFs/-@11245} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x55cf87274c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000000208:[<0x10000000009/WORD v427>,<0x10000000008/~dmtmp v57>,<0x10000000007/client0 v33>,<0x10000000006/clients v23>,<0x10000000005/tmp v23>,<0x10000000003/client.0 v22>,<0x10000000002/e395bed6-5bba-41b8-a256-3505e4afcca2 v36>,<0x10000000001/sv_0 v55>,<0x10000000000/qa v43>,<0x1/volumes v39>]//[]","memoryvalue":"(2)0x10000000208:[<0x10000000009/WORD v13401>,<0x10000000008/~dmtmp v12715>,<0x10000000007/client0 v1538>,<0x10000000006/clients v1529>,<0x10000000005/tmp v1500>,<0x10000000003/client.0 v1489>,<0x10000000002/e395bed6-5bba-41b8-a256-3505e4afcca2 v1482>,<0x10000000001/sv_0 v1473>,<0x10000000000/qa v1454>,<0x1/volumes v1405>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-04-24T09:04:50.315778+0000 6=6+0)","ondisk_value.rstat":"n(v0 rc2024-04-24T09:04:50.368133+0000 b814326 6=5+1)","memory_value.dirstat":"f(v504 m2024-04-24T09:04:50.368133+0000 5=5+0)","memory_value.rstat":"n(v47 rc2024-04-24T09:04:50.368133+0000 b814326 6=5+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-04-24T09:05:16.955 INFO:journalctl@ceph.mds.b.smithi138.stdout:Apr 24 09:05:16 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-b[61515]: 2024-04-24T09:05:16.476+0000 7f7fbbc00700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000225 [...16,head] /volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/PM/ auth{3=1} v13237 ap=2 f(v1554 m2024-04-24T09:04:44.122302+0000 3=3+0) n(v43 rc2024-04-24T09:04:44.122302+0000 b780288 4=3+1) old_inodes=6 (inest sync->mix g=3) caps={15603=pAsLsXsFs/-@15316} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x55cf8762f180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000000225:[<0x10000000009/PM v381>,<0x10000000008/~dmtmp v57>,<0x10000000007/client0 v33>,<0x10000000006/clients v23>,<0x10000000005/tmp v23>,<0x10000000003/client.0 v22>,<0x10000000002/e395bed6-5bba-41b8-a256-3505e4afcca2 v36>,<0x10000000001/sv_0 v55>,<0x10000000000/qa v43>,<0x1/volumes v39>]//[]","memoryvalue":"(2)0x10000000225:[<0x10000000009/PM v13237>,<0x10000000008/~dmtmp v12715>,<0x10000000007/client0 v1538>,<0x10000000006/clients v1529>,<0x10000000005/tmp v1500>,<0x10000000003/client.0 v1489>,<0x10000000002/e395bed6-5bba-41b8-a256-3505e4afcca2 v1482>,<0x10000000001/sv_0 v1473>,<0x10000000000/qa v1454>,<0x1/volumes v1405>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-04-24T09:04:42.713779+0000 4=4+0)","ondisk_value.rstat":"n(v0 rc2024-04-24T09:04:44.122302+0000 b780288 4=3+1)","memory_value.dirstat":"f(v1554 m2024-04-24T09:04:44.122302+0000 3=3+0)","memory_value.rstat":"n(v43 rc2024-04-24T09:04:44.122302+0000 b780288 4=3+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-04-24T09:05:16.955 INFO:journalctl@ceph.mds.b.smithi138.stdout:Apr 24 09:05:16 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-b[61515]: 2024-04-24T09:05:16.485+0000 7f7fbbc00700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000020f [...16,head] /volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/PWRPNT/ auth{1=1} v13349 ap=2 f(v588 m2024-04-24T09:04:47.854727+0000 7=7+0) n(v44 rc2024-04-24T09:04:47.854727+0000 b1462582 8=7+1) old_inodes=6 (inest sync->mix g=1) caps={15603=pAsLsXsFs/-@11189} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x55cf8760b600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000020f:[<0x10000000009/PWRPNT v274>,<0x10000000008/~dmtmp v57>,<0x10000000007/client0 v33>,<0x10000000006/clients v23>,<0x10000000005/tmp v23>,<0x10000000003/client.0 v22>,<0x10000000002/e395bed6-5bba-41b8-a256-3505e4afcca2 v36>,<0x10000000001/sv_0 v55>,<0x10000000000/qa v43>,<0x1/volumes v39>]//[]","memoryvalue":"(2)0x1000000020f:[<0x10000000009/PWRPNT v13349>,<0x10000000008/~dmtmp v12717>,<0x10000000007/client0 v1538>,<0x10000000006/clients v1529>,<0x10000000005/tmp v1500>,<0x10000000003/client.0 v1489>,<0x10000000002/e395bed6-5bba-41b8-a256-3505e4afcca2 v1482>,<0x10000000001/sv_0 v1473>,<0x10000000000/qa v1454>,<0x1/volumes v1405>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-04-24T09:04:47.752570+0000 8=8+0)","ondisk_value.rstat":"n(v0 rc2024-04-24T09:04:47.854727+0000 b1462582 8=7+1)","memory_value.dirstat":"f(v588 m2024-04-24T09:04:47.854727+0000 7=7+0)","memory_value.rstat":"n(v44 rc2024-04-24T09:04:47.854727+0000 b1462582 8=7+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-04-24T09:05:17.356 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.356+0000 7f340f200640 1 -- 172.21.15.83:0/1849439412 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f341010c3f0 msgr2=0x7f341010e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:17.357 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.356+0000 7f340f200640 1 --2- 172.21.15.83:0/1849439412 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f341010c3f0 0x7f341010e870 secure :-1 s=READY pgs=2121 cs=0 l=1 rev1=1 crypto rx=0x7f3408009ca0 tx=0x7f340802a950 comp rx=0 tx=0).stop 2024-04-24T09:05:17.357 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.356+0000 7f340f200640 1 -- 172.21.15.83:0/1849439412 shutdown_connections 2024-04-24T09:05:17.357 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.356+0000 7f340f200640 1 --2- 172.21.15.83:0/1849439412 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f341010c3f0 0x7f341010e870 unknown :-1 s=CLOSED pgs=2121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:17.357 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.356+0000 7f340f200640 1 --2- 172.21.15.83:0/1849439412 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3410103940 0x7f341010be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:17.357 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.356+0000 7f340f200640 1 --2- 172.21.15.83:0/1849439412 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3410103020 0x7f3410103400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:17.357 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.356+0000 7f340f200640 1 -- 172.21.15.83:0/1849439412 >> 172.21.15.83:0/1849439412 conn(0x7f34100fc820 msgr2=0x7f34100fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:17.357 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.356+0000 7f340f200640 1 -- 172.21.15.83:0/1849439412 shutdown_connections 2024-04-24T09:05:17.357 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.356+0000 7f340f200640 1 -- 172.21.15.83:0/1849439412 wait complete. 2024-04-24T09:05:17.357 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.357+0000 7f340f200640 1 Processor -- start 2024-04-24T09:05:17.358 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.357+0000 7f340f200640 1 -- start start 2024-04-24T09:05:17.360 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.358+0000 7f340f200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3410103020 0x7f34101a03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:17.360 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.358+0000 7f340f200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3410103940 0x7f34101a0920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:17.360 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.358+0000 7f340f200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f341010c3f0 0x7f341019a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:17.360 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.358+0000 7f340f200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3410112110 con 0x7f3410103940 2024-04-24T09:05:17.360 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.358+0000 7f340f200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3410111f90 con 0x7f341010c3f0 2024-04-24T09:05:17.360 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.358+0000 7f340f200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3410112290 con 0x7f3410103020 2024-04-24T09:05:17.361 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.358+0000 7f340d400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3410103940 0x7f34101a0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:17.361 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.358+0000 7f340d400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3410103940 0x7f34101a0920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36392/0 (socket says 172.21.15.83:36392) 2024-04-24T09:05:17.361 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.358+0000 7f340d400640 1 -- 172.21.15.83:0/2113134440 learned_addr learned my addr 172.21.15.83:0/2113134440 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:05:17.361 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.358+0000 7f340e800640 1 --2- 172.21.15.83:0/2113134440 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f341010c3f0 0x7f341019a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:17.361 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.358+0000 7f340de00640 1 --2- 172.21.15.83:0/2113134440 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3410103020 0x7f34101a03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:17.361 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.359+0000 7f340d400640 1 -- 172.21.15.83:0/2113134440 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3410103020 msgr2=0x7f34101a03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:17.361 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.359+0000 7f340d400640 1 --2- 172.21.15.83:0/2113134440 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3410103020 0x7f34101a03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:17.361 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.359+0000 7f340d400640 1 -- 172.21.15.83:0/2113134440 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f341010c3f0 msgr2=0x7f341019a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:17.361 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.359+0000 7f340d400640 1 --2- 172.21.15.83:0/2113134440 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f341010c3f0 0x7f341019a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:17.361 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.359+0000 7f340d400640 1 -- 172.21.15.83:0/2113134440 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f341019ad10 con 0x7f3410103940 2024-04-24T09:05:17.361 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.359+0000 7f340de00640 1 --2- 172.21.15.83:0/2113134440 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3410103020 0x7f34101a03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:05:17.361 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.359+0000 7f340d400640 1 --2- 172.21.15.83:0/2113134440 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3410103940 0x7f34101a0920 secure :-1 s=READY pgs=2122 cs=0 l=1 rev1=1 crypto rx=0x7f33fc00c920 tx=0x7f33fc00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:17.361 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.359+0000 7f3406a00640 1 -- 172.21.15.83:0/2113134440 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33fc007c70 con 0x7f3410103940 2024-04-24T09:05:17.361 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.359+0000 7f3406a00640 1 -- 172.21.15.83:0/2113134440 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f33fc007e10 con 0x7f3410103940 2024-04-24T09:05:17.362 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.359+0000 7f340f200640 1 -- 172.21.15.83:0/2113134440 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f341019b000 con 0x7f3410103940 2024-04-24T09:05:17.362 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.359+0000 7f3406a00640 1 -- 172.21.15.83:0/2113134440 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33fc018820 con 0x7f3410103940 2024-04-24T09:05:17.362 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.359+0000 7f340f200640 1 -- 172.21.15.83:0/2113134440 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34101a72a0 con 0x7f3410103940 2024-04-24T09:05:17.364 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.363+0000 7f341797c740 1 Processor -- start 2024-04-24T09:05:17.364 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.363+0000 7f3406a00640 1 -- 172.21.15.83:0/2113134440 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f33fc0189c0 con 0x7f3410103940 2024-04-24T09:05:17.364 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.363+0000 7f341797c740 1 -- start start 2024-04-24T09:05:17.364 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.363+0000 7f341797c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c66d0653c0 0x55c66d0657a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:17.365 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.363+0000 7f341797c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c66d065ce0 0x55c66d070060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:17.365 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.363+0000 7f341797c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c66d0705a0 0x55c66d072990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:17.365 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.363+0000 7f341797c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c66cfcb900 con 0x55c66d0705a0 2024-04-24T09:05:17.365 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.363+0000 7f341797c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c66cf71f20 con 0x55c66d0653c0 2024-04-24T09:05:17.365 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.363+0000 7f341797c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55c66cfc78c0 con 0x55c66d065ce0 2024-04-24T09:05:17.365 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.364+0000 7f340d400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c66d0705a0 0x55c66d072990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:17.365 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.364+0000 7f340d400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c66d0705a0 0x55c66d072990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36408/0 (socket says 172.21.15.83:36408) 2024-04-24T09:05:17.365 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.364+0000 7f340d400640 1 -- 172.21.15.83:0/1860991669 learned_addr learned my addr 172.21.15.83:0/1860991669 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:05:17.365 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.364+0000 7f340de00640 1 --2- 172.21.15.83:0/1860991669 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c66d065ce0 0x55c66d070060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:17.365 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.364+0000 7f340e800640 1 --2- 172.21.15.83:0/1860991669 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c66d0653c0 0x55c66d0657a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:17.365 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.364+0000 7f340d400640 1 -- 172.21.15.83:0/1860991669 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c66d065ce0 msgr2=0x55c66d070060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:17.365 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.364+0000 7f340d400640 1 --2- 172.21.15.83:0/1860991669 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c66d065ce0 0x55c66d070060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:17.365 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.364+0000 7f340d400640 1 -- 172.21.15.83:0/1860991669 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c66d0653c0 msgr2=0x55c66d0657a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:17.366 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.364+0000 7f340d400640 1 --2- 172.21.15.83:0/1860991669 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c66d0653c0 0x55c66d0657a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:17.366 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.364+0000 7f340d400640 1 -- 172.21.15.83:0/1860991669 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c66cf96e10 con 0x55c66d0705a0 2024-04-24T09:05:17.366 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.364+0000 7f3406a00640 1 --2- 172.21.15.83:0/2113134440 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f33f40a3d90 0x7f33f40a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:17.366 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.365+0000 7f3406a00640 1 -- 172.21.15.83:0/2113134440 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7f33fc0d0da0 con 0x7f3410103940 2024-04-24T09:05:17.366 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.365+0000 7f340d400640 1 --2- 172.21.15.83:0/1860991669 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c66d0705a0 0x55c66d072990 secure :-1 s=READY pgs=2123 cs=0 l=1 rev1=1 crypto rx=0x7f33fc0ce040 tx=0x7f33fc0d90e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60f2f5f75a1a2a16 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:17.366 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.365+0000 7f340e800640 1 --2- 172.21.15.83:0/2113134440 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f33f40a3d90 0x7f33f40a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:17.366 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.365+0000 7f33fb000640 1 -- 172.21.15.83:0/1860991669 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33fc085e60 con 0x55c66d0705a0 2024-04-24T09:05:17.366 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.365+0000 7f33fb000640 1 -- 172.21.15.83:0/1860991669 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f33fc00fb00 con 0x55c66d0705a0 2024-04-24T09:05:17.366 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.365+0000 7f33fb000640 1 -- 172.21.15.83:0/1860991669 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33fc09c6f0 con 0x55c66d0705a0 2024-04-24T09:05:17.371 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.366+0000 7f341797c740 1 -- 172.21.15.83:0/1860991669 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c66d0705a0 msgr2=0x55c66d072990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:17.371 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.366+0000 7f340e800640 1 --2- 172.21.15.83:0/2113134440 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f33f40a3d90 0x7f33f40a6250 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f3408005fa0 tx=0x7f3408005ef0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:17.372 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.366+0000 7f341797c740 1 --2- 172.21.15.83:0/1860991669 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c66d0705a0 0x55c66d072990 secure :-1 s=READY pgs=2123 cs=0 l=1 rev1=1 crypto rx=0x7f33fc0ce040 tx=0x7f33fc0d90e0 comp rx=0 tx=0).stop 2024-04-24T09:05:17.372 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.366+0000 7f341797c740 1 -- 172.21.15.83:0/1860991669 shutdown_connections 2024-04-24T09:05:17.372 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.366+0000 7f341797c740 1 --2- 172.21.15.83:0/1860991669 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c66d0705a0 0x55c66d072990 unknown :-1 s=CLOSED pgs=2123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:17.372 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.366+0000 7f341797c740 1 --2- 172.21.15.83:0/1860991669 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c66d065ce0 0x55c66d070060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:17.372 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.366+0000 7f341797c740 1 --2- 172.21.15.83:0/1860991669 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c66d0653c0 0x55c66d0657a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:17.372 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.366+0000 7f341797c740 1 -- 172.21.15.83:0/1860991669 >> 172.21.15.83:0/1860991669 conn(0x55c66d0618d0 msgr2=0x55c66d061cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:17.372 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.366+0000 7f341797c740 1 -- 172.21.15.83:0/1860991669 shutdown_connections 2024-04-24T09:05:17.372 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.366+0000 7f341797c740 1 -- 172.21.15.83:0/1860991669 wait complete. 2024-04-24T09:05:17.372 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.367+0000 7f341797c740 1 -- start start 2024-04-24T09:05:17.372 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.367+0000 7f341797c740 1 Processor -- start 2024-04-24T09:05:17.372 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.368+0000 7f341797c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c66d0653c0 0x55c66d06d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:17.372 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.368+0000 7f341797c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c66d065ce0 0x55c66d06db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:17.373 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.368+0000 7f341797c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c66d0705a0 0x55c66d06e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:17.374 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.368+0000 7f341797c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c66d0730e0 con 0x55c66d0653c0 2024-04-24T09:05:17.374 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.368+0000 7f341797c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c66cf852c0 con 0x55c66d065ce0 2024-04-24T09:05:17.374 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.368+0000 7f341797c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55c66d073260 con 0x55c66d0705a0 2024-04-24T09:05:17.374 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.368+0000 7f341797c740 10 client.0 _ll_register_callbacks cb 0x55c66ccdda00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:05:17.374 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.368+0000 7f340de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c66d0653c0 0x55c66d06d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:17.374 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.368+0000 7f340de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c66d0653c0 0x55c66d06d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36414/0 (socket says 172.21.15.83:36414) 2024-04-24T09:05:17.374 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.368+0000 7f340de00640 1 -- 172.21.15.83:0/2316153723 learned_addr learned my addr 172.21.15.83:0/2316153723 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:05:17.374 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.368+0000 7f340d400640 1 --2- 172.21.15.83:0/2316153723 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c66d065ce0 0x55c66d06db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:17.375 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.368+0000 7f340e800640 1 --2- 172.21.15.83:0/2316153723 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c66d0705a0 0x55c66d06e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:17.375 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.368+0000 7f340de00640 1 -- 172.21.15.83:0/2316153723 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c66d0705a0 msgr2=0x55c66d06e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:17.375 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.369+0000 7f340de00640 1 --2- 172.21.15.83:0/2316153723 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c66d0705a0 0x55c66d06e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:17.375 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.369+0000 7f340de00640 1 -- 172.21.15.83:0/2316153723 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c66d065ce0 msgr2=0x55c66d06db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:17.375 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.369+0000 7f340de00640 1 --2- 172.21.15.83:0/2316153723 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c66d065ce0 0x55c66d06db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:17.375 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.369+0000 7f340de00640 1 -- 172.21.15.83:0/2316153723 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c66cf95fd0 con 0x55c66d0653c0 2024-04-24T09:05:17.375 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.369+0000 7f340de00640 1 --2- 172.21.15.83:0/2316153723 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c66d0653c0 0x55c66d06d620 secure :-1 s=READY pgs=2124 cs=0 l=1 rev1=1 crypto rx=0x7f340000e2f0 tx=0x7f340000e7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e35d50d8f95c2515 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:17.375 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.369+0000 7f33ef400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:05:17.375 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.369+0000 7f33ef400640 1 -- 172.21.15.83:0/2316153723 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3400018030 con 0x55c66d0653c0 2024-04-24T09:05:17.375 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.369+0000 7f33ef400640 1 -- 172.21.15.83:0/2316153723 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7f340000f040 con 0x55c66d0653c0 2024-04-24T09:05:17.375 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.369+0000 7f341797c740 1 -- 172.21.15.83:0/2316153723 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c66cfd9320 con 0x55c66d0653c0 2024-04-24T09:05:17.375 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.370+0000 7f33ef400640 1 -- 172.21.15.83:0/2316153723 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f3400022050 con 0x55c66d0653c0 2024-04-24T09:05:17.376 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.370+0000 7f341797c740 10 client.20538 fetch_fsmap learned FSMap version 43 2024-04-24T09:05:17.376 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.370+0000 7f341797c740 1 -- 172.21.15.83:0/2316153723 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55c66ccde210 con 0x55c66d0653c0 2024-04-24T09:05:17.376 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.370+0000 7f33ef400640 1 -- 172.21.15.83:0/2316153723 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55c66cfd9320 con 0x55c66d0653c0 2024-04-24T09:05:17.376 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.370+0000 7f33ef400640 1 -- 172.21.15.83:0/2316153723 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7f340001d960 con 0x55c66d0653c0 2024-04-24T09:05:17.376 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.370+0000 7f341797c740 10 client.20538 fetch_fsmap finished waiting for FSMap version 43 2024-04-24T09:05:17.376 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.370+0000 7f341797c740 10 client.20538 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:05:17.376 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.370+0000 7f341797c740 20 client.20538 populate_metadata read hostname 'smithi083' 2024-04-24T09:05:17.376 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.370+0000 7f341797c740 1 --2- 172.21.15.83:0/2316153723 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55c66d080900 0x55c66d082d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:17.376 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.370+0000 7f341797c740 4 client.20538 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:05:17.376 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.370+0000 7f341797c740 1 -- 172.21.15.83:0/2316153723 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c66cfd3190 con 0x55c66d080900 2024-04-24T09:05:17.376 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.371+0000 7f340d400640 1 --2- 172.21.15.83:0/2316153723 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55c66d080900 0x55c66d082d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:17.376 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.372+0000 7f340d400640 1 --2- 172.21.15.83:0/2316153723 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55c66d080900 0x55c66d082d70 crc :-1 s=READY pgs=357 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59d1f93ce93da55d server_cookie=6f5f46c672c3c35d in_seq=0 out_seq=0 2024-04-24T09:05:17.376 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.372+0000 7f33ef400640 10 client.20538 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:05:17.377 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.376+0000 7f33ef400640 1 -- 172.21.15.83:0/2316153723 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c66cfd3190 con 0x55c66d080900 2024-04-24T09:05:17.378 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.376+0000 7f33ef400640 10 client.20538 handle_command_reply: tid=0 2024-04-24T09:05:17.378 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.376+0000 7f341797c740 1 client.20538 shutdown 2024-04-24T09:05:17.378 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.376+0000 7f341797c740 1 -- 172.21.15.83:0/2316153723 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c66d0653c0 msgr2=0x55c66d06d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:17.378 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.376+0000 7f341797c740 1 --2- 172.21.15.83:0/2316153723 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c66d0653c0 0x55c66d06d620 secure :-1 s=READY pgs=2124 cs=0 l=1 rev1=1 crypto rx=0x7f340000e2f0 tx=0x7f340000e7b0 comp rx=0 tx=0).stop 2024-04-24T09:05:17.378 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.376+0000 7f341797c740 1 -- 172.21.15.83:0/2316153723 shutdown_connections 2024-04-24T09:05:17.378 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.376+0000 7f341797c740 1 --2- 172.21.15.83:0/2316153723 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55c66d080900 0x55c66d082d70 crc :-1 s=READY pgs=357 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:17.378 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.376+0000 7f341797c740 1 --2- 172.21.15.83:0/2316153723 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c66d0705a0 0x55c66d06e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:17.378 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.376+0000 7f341797c740 1 --2- 172.21.15.83:0/2316153723 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c66d065ce0 0x55c66d06db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:17.378 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.376+0000 7f341797c740 1 --2- 172.21.15.83:0/2316153723 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c66d0653c0 0x55c66d06d620 unknown :-1 s=CLOSED pgs=2124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:17.378 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.376+0000 7f33ef400640 0 client.20538 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:05:17.378 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.376+0000 7f341797c740 1 -- 172.21.15.83:0/2316153723 >> 172.21.15.83:0/2316153723 conn(0x55c66d0618d0 msgr2=0x55c66d060a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:17.378 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.376+0000 7f341797c740 1 -- 172.21.15.83:0/2316153723 shutdown_connections 2024-04-24T09:05:17.378 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.377+0000 7f341797c740 1 -- 172.21.15.83:0/2316153723 wait complete. 2024-04-24T09:05:17.379 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.377+0000 7f341797c740 20 client.20538 trim_cache size 0 max 16384 2024-04-24T09:05:17.395 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.393+0000 7f341797c740 1 Processor -- start 2024-04-24T09:05:17.395 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.393+0000 7f341797c740 1 -- start start 2024-04-24T09:05:17.395 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.393+0000 7f341797c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c66d0653c0 0x55c66d07c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:17.395 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.393+0000 7f341797c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c66d065ce0 0x55c66d060910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:17.395 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.393+0000 7f341797c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c66d0705a0 0x55c66d088680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:17.395 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.393+0000 7f341797c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f34101b1580 con 0x55c66d065ce0 2024-04-24T09:05:17.395 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.393+0000 7f341797c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f34101b3180 con 0x55c66d0653c0 2024-04-24T09:05:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.393+0000 7f341797c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f34101b1c50 con 0x55c66d0705a0 2024-04-24T09:05:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.394+0000 7f340d400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c66d065ce0 0x55c66d060910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.394+0000 7f340d400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c66d065ce0 0x55c66d060910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36420/0 (socket says 172.21.15.83:36420) 2024-04-24T09:05:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.394+0000 7f340d400640 1 -- 172.21.15.83:0/1164293115 learned_addr learned my addr 172.21.15.83:0/1164293115 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:05:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.394+0000 7f340de00640 1 --2- 172.21.15.83:0/1164293115 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c66d0653c0 0x55c66d07c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.394+0000 7f340e800640 1 --2- 172.21.15.83:0/1164293115 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c66d0705a0 0x55c66d088680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.394+0000 7f340d400640 1 -- 172.21.15.83:0/1164293115 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c66d0705a0 msgr2=0x55c66d088680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.394+0000 7f340d400640 1 --2- 172.21.15.83:0/1164293115 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c66d0705a0 0x55c66d088680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.394+0000 7f340d400640 1 -- 172.21.15.83:0/1164293115 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c66d0653c0 msgr2=0x55c66d07c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.394+0000 7f340d400640 1 --2- 172.21.15.83:0/1164293115 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c66d0653c0 0x55c66d07c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.394+0000 7f340d400640 1 -- 172.21.15.83:0/1164293115 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c66cdaa620 con 0x55c66d065ce0 2024-04-24T09:05:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.394+0000 7f340de00640 1 --2- 172.21.15.83:0/1164293115 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c66d0653c0 0x55c66d07c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:05:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.395+0000 7f340d400640 1 --2- 172.21.15.83:0/1164293115 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c66d065ce0 0x55c66d060910 secure :-1 s=READY pgs=2125 cs=0 l=1 rev1=1 crypto rx=0x7f33fc0a5060 tx=0x7f33fc000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=155d41b765963d1c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.395+0000 7f33fa600640 1 -- 172.21.15.83:0/1164293115 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33fc085e60 con 0x55c66d065ce0 2024-04-24T09:05:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.395+0000 7f33fa600640 1 -- 172.21.15.83:0/1164293115 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f33fc09d020 con 0x55c66d065ce0 2024-04-24T09:05:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.395+0000 7f33fa600640 1 -- 172.21.15.83:0/1164293115 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33fc018cb0 con 0x55c66d065ce0 2024-04-24T09:05:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.396+0000 7f341797c740 1 -- 172.21.15.83:0/1164293115 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c66d065ce0 msgr2=0x55c66d060910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.396+0000 7f341797c740 1 --2- 172.21.15.83:0/1164293115 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c66d065ce0 0x55c66d060910 secure :-1 s=READY pgs=2125 cs=0 l=1 rev1=1 crypto rx=0x7f33fc0a5060 tx=0x7f33fc000bc0 comp rx=0 tx=0).stop 2024-04-24T09:05:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.396+0000 7f341797c740 1 -- 172.21.15.83:0/1164293115 shutdown_connections 2024-04-24T09:05:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.396+0000 7f341797c740 1 --2- 172.21.15.83:0/1164293115 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c66d0705a0 0x55c66d088680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.396+0000 7f341797c740 1 --2- 172.21.15.83:0/1164293115 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c66d065ce0 0x55c66d060910 unknown :-1 s=CLOSED pgs=2125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.396+0000 7f341797c740 1 --2- 172.21.15.83:0/1164293115 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c66d0653c0 0x55c66d07c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.396+0000 7f341797c740 1 -- 172.21.15.83:0/1164293115 >> 172.21.15.83:0/1164293115 conn(0x55c66d0618d0 msgr2=0x55c66d060200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.396+0000 7f341797c740 1 -- 172.21.15.83:0/1164293115 shutdown_connections 2024-04-24T09:05:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.396+0000 7f341797c740 1 -- 172.21.15.83:0/1164293115 wait complete. 2024-04-24T09:05:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.396+0000 7f341797c740 1 -- start start 2024-04-24T09:05:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.396+0000 7f341797c740 1 Processor -- start 2024-04-24T09:05:17.401 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.397+0000 7f341797c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c66d0653c0 0x55c66d078880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:17.401 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.397+0000 7f341797c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c66d065ce0 0x55c66d073bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:17.401 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.397+0000 7f341797c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c66d0705a0 0x55c66d074130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:17.401 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.397+0000 7f341797c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f34101b2a50 con 0x55c66d065ce0 2024-04-24T09:05:17.401 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.397+0000 7f341797c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f34101b2320 con 0x55c66d0653c0 2024-04-24T09:05:17.402 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.397+0000 7f341797c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f34101af580 con 0x55c66d0705a0 2024-04-24T09:05:17.402 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.397+0000 7f341797c740 10 client.0 _ll_register_callbacks cb 0x7f340000c190 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:05:17.402 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.397+0000 7f340d400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c66d065ce0 0x55c66d073bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:17.402 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.397+0000 7f340d400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c66d065ce0 0x55c66d073bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36432/0 (socket says 172.21.15.83:36432) 2024-04-24T09:05:17.402 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.397+0000 7f340d400640 1 -- 172.21.15.83:0/3293012745 learned_addr learned my addr 172.21.15.83:0/3293012745 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:05:17.403 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.397+0000 7f340d400640 1 -- 172.21.15.83:0/3293012745 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c66d0705a0 msgr2=0x55c66d074130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:17.403 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.397+0000 7f340d400640 1 --2- 172.21.15.83:0/3293012745 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c66d0705a0 0x55c66d074130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:17.403 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.397+0000 7f340d400640 1 -- 172.21.15.83:0/3293012745 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c66d0653c0 msgr2=0x55c66d078880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:17.403 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.397+0000 7f340d400640 1 --2- 172.21.15.83:0/3293012745 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c66d0653c0 0x55c66d078880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:17.403 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.397+0000 7f340d400640 1 -- 172.21.15.83:0/3293012745 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c66d074820 con 0x55c66d065ce0 2024-04-24T09:05:17.404 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.398+0000 7f340d400640 1 --2- 172.21.15.83:0/3293012745 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c66d065ce0 0x55c66d073bf0 secure :-1 s=READY pgs=2126 cs=0 l=1 rev1=1 crypto rx=0x7f33fc09c620 tx=0x7f33fc0a0dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cf93030e3b5395 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:17.404 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.398+0000 7f33ef400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:05:17.404 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.398+0000 7f33ef400640 1 -- 172.21.15.83:0/3293012745 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33fc085e60 con 0x55c66d065ce0 2024-04-24T09:05:17.404 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.398+0000 7f33ef400640 1 -- 172.21.15.83:0/3293012745 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7f33fc0ce350 con 0x55c66d065ce0 2024-04-24T09:05:17.405 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.398+0000 7f341797c740 1 -- 172.21.15.83:0/3293012745 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33fc09d020 con 0x55c66d065ce0 2024-04-24T09:05:17.405 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.398+0000 7f33ef400640 1 -- 172.21.15.83:0/3293012745 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f33fc09c280 con 0x55c66d065ce0 2024-04-24T09:05:17.405 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.398+0000 7f341797c740 10 client.20544 fetch_fsmap learned FSMap version 43 2024-04-24T09:05:17.405 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.398+0000 7f341797c740 1 -- 172.21.15.83:0/3293012745 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55c66d0e6d30 con 0x55c66d065ce0 2024-04-24T09:05:17.405 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.399+0000 7f33ef400640 1 -- 172.21.15.83:0/3293012745 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f33fc0b9070 con 0x55c66d065ce0 2024-04-24T09:05:17.406 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.399+0000 7f33ef400640 1 -- 172.21.15.83:0/3293012745 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7f33fc0d3040 con 0x55c66d065ce0 2024-04-24T09:05:17.406 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.399+0000 7f341797c740 10 client.20544 fetch_fsmap finished waiting for FSMap version 43 2024-04-24T09:05:17.406 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.399+0000 7f341797c740 10 client.20544 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:05:17.406 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.399+0000 7f341797c740 20 client.20544 populate_metadata read hostname 'smithi083' 2024-04-24T09:05:17.406 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.399+0000 7f341797c740 1 --2- 172.21.15.83:0/3293012745 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55c66d0e6fb0 0x55c66d0e93a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:17.406 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.399+0000 7f341797c740 4 client.20544 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T09:05:17.407 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.399+0000 7f341797c740 1 -- 172.21.15.83:0/3293012745 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c66cf4f4b0 con 0x55c66d0e6fb0 2024-04-24T09:05:17.407 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.400+0000 7f340de00640 1 --2- 172.21.15.83:0/3293012745 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55c66d0e6fb0 0x55c66d0e93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:17.407 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.400+0000 7f340de00640 1 --2- 172.21.15.83:0/3293012745 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55c66d0e6fb0 0x55c66d0e93a0 crc :-1 s=READY pgs=358 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5748c9b07a40fea6 server_cookie=15ca80d0ec74876b in_seq=0 out_seq=0 2024-04-24T09:05:17.407 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.400+0000 7f33ef400640 10 client.20544 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:05:17.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:05:17 smithi138 ceph-mon[31441]: scrub queued for path: / 2024-04-24T09:05:17.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:05:17 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:05:17.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:05:17 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:05:17.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:05:17 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:05:17.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:05:17 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:05:17.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:05:17 smithi138 ceph-mon[31441]: Scrub error on inode 0x10000000208 (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/WORD) see mds.b log and `damage ls` output for details 2024-04-24T09:05:17.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:05:17 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:05:17.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:05:17 smithi138 ceph-mon[31441]: Scrub error on inode 0x10000000225 (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/PM) see mds.b log and `damage ls` output for details 2024-04-24T09:05:17.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:05:17 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:05:17.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:05:17 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:05:17.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:05:17 smithi138 ceph-mon[31441]: Scrub error on inode 0x1000000020f (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/PWRPNT) see mds.b log and `damage ls` output for details 2024-04-24T09:05:17.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:05:17 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:05:17.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:05:17 smithi189 ceph-mon[31791]: scrub queued for path: / 2024-04-24T09:05:17.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:05:17 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:05:17.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:05:17 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:05:17.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:05:17 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:05:17.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:05:17 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:05:17.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:05:17 smithi189 ceph-mon[31791]: Scrub error on inode 0x10000000208 (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/WORD) see mds.b log and `damage ls` output for details 2024-04-24T09:05:17.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:05:17 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:05:17.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:05:17 smithi189 ceph-mon[31791]: Scrub error on inode 0x10000000225 (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/PM) see mds.b log and `damage ls` output for details 2024-04-24T09:05:17.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:05:17 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:05:17.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:05:17 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:05:17.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:05:17 smithi189 ceph-mon[31791]: Scrub error on inode 0x1000000020f (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/PWRPNT) see mds.b log and `damage ls` output for details 2024-04-24T09:05:17.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:05:17 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:05:17.464 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.465+0000 7f33ef400640 1 -- 172.21.15.83:0/3293012745 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c66cf4f4b0 con 0x55c66d0e6fb0 2024-04-24T09:05:17.464 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.465+0000 7f33ef400640 10 client.20544 handle_command_reply: tid=0 2024-04-24T09:05:17.464 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.465+0000 7f341797c740 1 client.20544 shutdown 2024-04-24T09:05:17.465 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.465+0000 7f341797c740 1 -- 172.21.15.83:0/3293012745 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c66d065ce0 msgr2=0x55c66d073bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:17.465 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.465+0000 7f341797c740 1 --2- 172.21.15.83:0/3293012745 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c66d065ce0 0x55c66d073bf0 secure :-1 s=READY pgs=2126 cs=0 l=1 rev1=1 crypto rx=0x7f33fc09c620 tx=0x7f33fc0a0dc0 comp rx=0 tx=0).stop 2024-04-24T09:05:17.465 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.466+0000 7f341797c740 1 -- 172.21.15.83:0/3293012745 shutdown_connections 2024-04-24T09:05:17.465 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.466+0000 7f341797c740 1 --2- 172.21.15.83:0/3293012745 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55c66d0e6fb0 0x55c66d0e93a0 crc :-1 s=READY pgs=358 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:17.465 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.466+0000 7f341797c740 1 --2- 172.21.15.83:0/3293012745 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c66d0705a0 0x55c66d074130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:17.465 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.466+0000 7f341797c740 1 --2- 172.21.15.83:0/3293012745 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c66d065ce0 0x55c66d073bf0 unknown :-1 s=CLOSED pgs=2126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:17.465 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.466+0000 7f341797c740 1 --2- 172.21.15.83:0/3293012745 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c66d0653c0 0x55c66d078880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:17.466 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.466+0000 7f341797c740 1 -- 172.21.15.83:0/3293012745 >> 172.21.15.83:0/3293012745 conn(0x55c66d0618d0 msgr2=0x55c66d060a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:17.466 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.466+0000 7f33ef400640 0 client.20544 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:05:17.466 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.466+0000 7f341797c740 1 -- 172.21.15.83:0/3293012745 shutdown_connections 2024-04-24T09:05:17.466 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.466+0000 7f341797c740 1 -- 172.21.15.83:0/3293012745 wait complete. 2024-04-24T09:05:17.466 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.466+0000 7f341797c740 20 client.20544 trim_cache size 0 max 16384 2024-04-24T09:05:17.466 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T09:05:17.466 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T09:05:17.466 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T09:05:17.467 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T09:05:17.467 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.467+0000 7f340f200640 1 -- 172.21.15.83:0/2113134440 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f33f40a3d90 msgr2=0x7f33f40a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:17.467 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.467+0000 7f340f200640 1 --2- 172.21.15.83:0/2113134440 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f33f40a3d90 0x7f33f40a6250 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f3408005fa0 tx=0x7f3408005ef0 comp rx=0 tx=0).stop 2024-04-24T09:05:17.467 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.467+0000 7f340f200640 1 -- 172.21.15.83:0/2113134440 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3410103940 msgr2=0x7f34101a0920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:17.468 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.467+0000 7f340f200640 1 --2- 172.21.15.83:0/2113134440 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3410103940 0x7f34101a0920 secure :-1 s=READY pgs=2122 cs=0 l=1 rev1=1 crypto rx=0x7f33fc00c920 tx=0x7f33fc00cde0 comp rx=0 tx=0).stop 2024-04-24T09:05:17.468 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.468+0000 7f340f200640 1 -- 172.21.15.83:0/2113134440 shutdown_connections 2024-04-24T09:05:17.468 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.468+0000 7f340f200640 1 --2- 172.21.15.83:0/2113134440 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f33f40a3d90 0x7f33f40a6250 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:17.468 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.468+0000 7f340f200640 1 --2- 172.21.15.83:0/2113134440 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f341010c3f0 0x7f341019a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:17.468 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.468+0000 7f340f200640 1 --2- 172.21.15.83:0/2113134440 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3410103940 0x7f34101a0920 unknown :-1 s=CLOSED pgs=2122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:17.468 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.468+0000 7f340f200640 1 --2- 172.21.15.83:0/2113134440 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3410103020 0x7f34101a03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:17.468 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.468+0000 7f340f200640 1 -- 172.21.15.83:0/2113134440 >> 172.21.15.83:0/2113134440 conn(0x7f34100fc820 msgr2=0x7f34100fcc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:17.468 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.468+0000 7f340f200640 1 -- 172.21.15.83:0/2113134440 shutdown_connections 2024-04-24T09:05:17.468 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:17.468+0000 7f340f200640 1 -- 172.21.15.83:0/2113134440 wait complete. 2024-04-24T09:05:17.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:05:17 smithi083 ceph-mon[27907]: scrub queued for path: / 2024-04-24T09:05:17.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:05:17 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:05:17.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:05:17 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:05:17.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:05:17 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:05:17.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:05:17 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:05:17.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:05:17 smithi083 ceph-mon[27907]: Scrub error on inode 0x10000000208 (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/WORD) see mds.b log and `damage ls` output for details 2024-04-24T09:05:17.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:05:17 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:05:17.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:05:17 smithi083 ceph-mon[27907]: Scrub error on inode 0x10000000225 (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/PM) see mds.b log and `damage ls` output for details 2024-04-24T09:05:17.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:05:17 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:05:17.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:05:17 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:05:17.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:05:17 smithi083 ceph-mon[27907]: Scrub error on inode 0x1000000020f (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/PWRPNT) see mds.b log and `damage ls` output for details 2024-04-24T09:05:17.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:05:17 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:05:17.632 INFO:tasks.workunit.client.0.smithi083.stdout: 1 78327 0.58 MB/sec execute 591 sec latency 59.733 ms 2024-04-24T09:05:18.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:05:18 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:05:18.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:05:18 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:05:18.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:05:18 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:05:18.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:05:18 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:05:18.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:05:18 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:05:18.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:05:18 smithi138 ceph-mon[31441]: pgmap v429: 129 pgs: 129 active+clean; 74 MiB data, 997 MiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 326 KiB/s wr, 33 op/s 2024-04-24T09:05:18.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:05:18 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:05:18.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:05:18 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:05:18.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:05:18 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:05:18.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:05:18 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:05:18.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:05:18 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:05:18.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:05:18 smithi189 ceph-mon[31791]: pgmap v429: 129 pgs: 129 active+clean; 74 MiB data, 997 MiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 326 KiB/s wr, 33 op/s 2024-04-24T09:05:18.457 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:05:18 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:05:18.457 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:05:18 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:05:18.457 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:05:18 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:05:18.457 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:05:18 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:05:18.457 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:05:18 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:05:18.457 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:05:18 smithi083 ceph-mon[27907]: pgmap v429: 129 pgs: 129 active+clean; 74 MiB data, 997 MiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 326 KiB/s wr, 33 op/s 2024-04-24T09:05:18.632 INFO:tasks.workunit.client.0.smithi083.stdout: 1 78386 0.58 MB/sec execute 592 sec latency 46.077 ms 2024-04-24T09:05:19.632 INFO:tasks.workunit.client.0.smithi083.stdout: 1 78455 0.58 MB/sec execute 593 sec latency 49.761 ms 2024-04-24T09:05:20.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:05:20 smithi138 ceph-mon[31441]: pgmap v430: 129 pgs: 129 active+clean; 75 MiB data, 999 MiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 482 KiB/s wr, 60 op/s 2024-04-24T09:05:20.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:05:20 smithi189 ceph-mon[31791]: pgmap v430: 129 pgs: 129 active+clean; 75 MiB data, 999 MiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 482 KiB/s wr, 60 op/s 2024-04-24T09:05:20.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:05:20 smithi083 ceph-mon[27907]: pgmap v430: 129 pgs: 129 active+clean; 75 MiB data, 999 MiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 482 KiB/s wr, 60 op/s 2024-04-24T09:05:20.632 INFO:tasks.workunit.client.0.smithi083.stdout: 1 78506 0.58 MB/sec execute 594 sec latency 45.885 ms 2024-04-24T09:05:21.632 INFO:tasks.workunit.client.0.smithi083.stdout: 1 78561 0.58 MB/sec execute 595 sec latency 54.734 ms 2024-04-24T09:05:22.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:05:22 smithi138 ceph-mon[31441]: pgmap v431: 129 pgs: 129 active+clean; 75 MiB data, 999 MiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 188 KiB/s wr, 38 op/s 2024-04-24T09:05:22.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:05:22 smithi189 ceph-mon[31791]: pgmap v431: 129 pgs: 129 active+clean; 75 MiB data, 999 MiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 188 KiB/s wr, 38 op/s 2024-04-24T09:05:22.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:05:22 smithi083 ceph-mon[27907]: pgmap v431: 129 pgs: 129 active+clean; 75 MiB data, 999 MiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 188 KiB/s wr, 38 op/s 2024-04-24T09:05:22.632 INFO:tasks.workunit.client.0.smithi083.stdout: 1 78638 0.58 MB/sec execute 596 sec latency 56.565 ms 2024-04-24T09:05:23.633 INFO:tasks.workunit.client.0.smithi083.stdout: 1 78700 0.58 MB/sec execute 597 sec latency 55.134 ms 2024-04-24T09:05:24.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:05:24 smithi138 ceph-mon[31441]: pgmap v432: 129 pgs: 129 active+clean; 76 MiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 306 KiB/s wr, 58 op/s 2024-04-24T09:05:24.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:05:24 smithi189 ceph-mon[31791]: pgmap v432: 129 pgs: 129 active+clean; 76 MiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 306 KiB/s wr, 58 op/s 2024-04-24T09:05:24.552 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:05:24 smithi083 ceph-mon[27907]: pgmap v432: 129 pgs: 129 active+clean; 76 MiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 306 KiB/s wr, 58 op/s 2024-04-24T09:05:24.633 INFO:tasks.workunit.client.0.smithi083.stdout: 1 78816 0.58 MB/sec execute 598 sec latency 52.856 ms 2024-04-24T09:05:25.633 INFO:tasks.workunit.client.0.smithi083.stdout: 1 78950 0.58 MB/sec execute 599 sec latency 55.654 ms 2024-04-24T09:05:26.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:05:26 smithi138 ceph-mon[31441]: pgmap v433: 129 pgs: 129 active+clean; 76 MiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 275 KiB/s wr, 47 op/s 2024-04-24T09:05:26.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:05:26 smithi189 ceph-mon[31791]: pgmap v433: 129 pgs: 129 active+clean; 76 MiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 275 KiB/s wr, 47 op/s 2024-04-24T09:05:26.552 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:05:26 smithi083 ceph-mon[27907]: pgmap v433: 129 pgs: 129 active+clean; 76 MiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 275 KiB/s wr, 47 op/s 2024-04-24T09:05:26.633 INFO:tasks.workunit.client.0.smithi083.stdout: 1 cleanup 600 sec 2024-04-24T09:05:27.633 INFO:tasks.workunit.client.0.smithi083.stdout: 1 cleanup 601 sec 2024-04-24T09:05:28.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:05:28 smithi138 ceph-mon[31441]: pgmap v434: 129 pgs: 129 active+clean; 76 MiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 275 KiB/s wr, 47 op/s 2024-04-24T09:05:28.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:05:28 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:05:28.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:05:28 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:05:28.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:05:28 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:05:28.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:05:28 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:05:28.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:05:28 smithi189 ceph-mon[31791]: pgmap v434: 129 pgs: 129 active+clean; 76 MiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 275 KiB/s wr, 47 op/s 2024-04-24T09:05:28.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:05:28 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:05:28.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:05:28 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:05:28.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:05:28 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:05:28.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:05:28 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:05:28.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:05:28 smithi083 ceph-mon[27907]: pgmap v434: 129 pgs: 129 active+clean; 76 MiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 275 KiB/s wr, 47 op/s 2024-04-24T09:05:28.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:05:28 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:05:28.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:05:28 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:05:28.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:05:28 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:05:28.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:05:28 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:05:28.633 INFO:tasks.workunit.client.0.smithi083.stdout: 1 cleanup 602 sec 2024-04-24T09:05:29.044 INFO:tasks.workunit.client.0.smithi083.stdout: 0 cleanup 602 sec 2024-04-24T09:05:29.044 INFO:tasks.workunit.client.0.smithi083.stdout: 2024-04-24T09:05:29.044 INFO:tasks.workunit.client.0.smithi083.stdout: Operation Count AvgLat MaxLat 2024-04-24T09:05:29.044 INFO:tasks.workunit.client.0.smithi083.stdout: ---------------------------------------- 2024-04-24T09:05:29.044 INFO:tasks.workunit.client.0.smithi083.stdout: NTCreateX 11115 15.985 279.442 2024-04-24T09:05:29.044 INFO:tasks.workunit.client.0.smithi083.stdout: Close 8172 0.213 2.996 2024-04-24T09:05:29.045 INFO:tasks.workunit.client.0.smithi083.stdout: Rename 471 126.659 5056.416 2024-04-24T09:05:29.045 INFO:tasks.workunit.client.0.smithi083.stdout: Unlink 2242 16.250 421.934 2024-04-24T09:05:29.045 INFO:tasks.workunit.client.0.smithi083.stdout: Qpathinfo 10039 12.141 82.006 2024-04-24T09:05:29.045 INFO:tasks.workunit.client.0.smithi083.stdout: Qfileinfo 1754 0.481 2.614 2024-04-24T09:05:29.045 INFO:tasks.workunit.client.0.smithi083.stdout: Qfsinfo 1852 1.378 15.003 2024-04-24T09:05:29.045 INFO:tasks.workunit.client.0.smithi083.stdout: Sfileinfo 915 5.926 43.248 2024-04-24T09:05:29.045 INFO:tasks.workunit.client.0.smithi083.stdout: Find 3894 46.332 265.994 2024-04-24T09:05:29.045 INFO:tasks.workunit.client.0.smithi083.stdout: WriteX 5498 0.936 15.587 2024-04-24T09:05:29.045 INFO:tasks.workunit.client.0.smithi083.stdout: ReadX 17412 0.013 1.069 2024-04-24T09:05:29.045 INFO:tasks.workunit.client.0.smithi083.stdout: LockX 36 2.099 2.813 2024-04-24T09:05:29.045 INFO:tasks.workunit.client.0.smithi083.stdout: UnlockX 36 2.035 2.775 2024-04-24T09:05:29.045 INFO:tasks.workunit.client.0.smithi083.stdout: Flush 796 8.857 21.887 2024-04-24T09:05:29.045 INFO:tasks.workunit.client.0.smithi083.stdout: 2024-04-24T09:05:29.045 INFO:tasks.workunit.client.0.smithi083.stdout:Throughput 0.575723 MB/sec 1 clients 1 procs max_latency=5056.446 ms 2024-04-24T09:05:29.046 INFO:tasks.workunit.client.0.smithi083.stderr:+ dbench 10 2024-04-24T09:05:29.048 INFO:tasks.workunit.client.0.smithi083.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-04-24T09:05:29.048 INFO:tasks.workunit.client.0.smithi083.stdout: 2024-04-24T09:05:29.048 INFO:tasks.workunit.client.0.smithi083.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-04-24T09:05:29.051 INFO:tasks.workunit.client.0.smithi083.stdout:8 of 10 processes prepared for launch 0 sec 2024-04-24T09:05:29.152 INFO:tasks.workunit.client.0.smithi083.stdout:10 of 10 processes prepared for launch 0 sec 2024-04-24T09:05:29.152 INFO:tasks.workunit.client.0.smithi083.stdout:releasing clients 2024-04-24T09:05:29.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:05:29 smithi138 ceph-mon[31441]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:05:29.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:05:29 smithi189 ceph-mon[31791]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:05:29.552 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:05:29 smithi083 ceph-mon[27907]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:05:30.152 INFO:tasks.workunit.client.0.smithi083.stdout: 10 169 69.59 MB/sec warmup 1 sec latency 69.328 ms 2024-04-24T09:05:30.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:05:30 smithi138 ceph-mon[31441]: pgmap v435: 129 pgs: 129 active+clean; 76 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 431 KiB/s wr, 77 op/s 2024-04-24T09:05:30.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:05:30 smithi189 ceph-mon[31791]: pgmap v435: 129 pgs: 129 active+clean; 76 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 431 KiB/s wr, 77 op/s 2024-04-24T09:05:30.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:05:30 smithi083 ceph-mon[27907]: pgmap v435: 129 pgs: 129 active+clean; 76 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 431 KiB/s wr, 77 op/s 2024-04-24T09:05:30.654 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.654+0000 7f26ac200640 1 -- 172.21.15.83:0/1360772669 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26a4108190 msgr2=0x7f26a4108570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:30.654 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.654+0000 7f26ac200640 1 --2- 172.21.15.83:0/1360772669 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26a4108190 0x7f26a4108570 secure :-1 s=READY pgs=2127 cs=0 l=1 rev1=1 crypto rx=0x7f2698009900 tx=0x7f269802a480 comp rx=0 tx=0).stop 2024-04-24T09:05:30.654 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.654+0000 7f26ac200640 1 -- 172.21.15.83:0/1360772669 shutdown_connections 2024-04-24T09:05:30.654 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.654+0000 7f26ac200640 1 --2- 172.21.15.83:0/1360772669 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26a406b1d0 0x7f26a410eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:30.654 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.654+0000 7f26ac200640 1 --2- 172.21.15.83:0/1360772669 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f26a406a830 0x7f26a406ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:30.654 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.654+0000 7f26ac200640 1 --2- 172.21.15.83:0/1360772669 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26a4108190 0x7f26a4108570 unknown :-1 s=CLOSED pgs=2127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:30.655 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.654+0000 7f26ac200640 1 -- 172.21.15.83:0/1360772669 >> 172.21.15.83:0/1360772669 conn(0x7f26a4076d00 msgr2=0x7f26a4077110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:30.655 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.654+0000 7f26ac200640 1 -- 172.21.15.83:0/1360772669 shutdown_connections 2024-04-24T09:05:30.655 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.654+0000 7f26ac200640 1 -- 172.21.15.83:0/1360772669 wait complete. 2024-04-24T09:05:30.655 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.656+0000 7f26ac200640 1 Processor -- start 2024-04-24T09:05:30.656 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.656+0000 7f26ac200640 1 -- start start 2024-04-24T09:05:30.657 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.657+0000 7f26ac200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26a406a830 0x7f26a410ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:30.657 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.657+0000 7f26ac200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26a406b1d0 0x7f26a410bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:30.657 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.657+0000 7f26ac200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f26a4108190 0x7f26a410c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:30.657 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.657+0000 7f26ac200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f26a4102e80 con 0x7f26a406a830 2024-04-24T09:05:30.657 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.657+0000 7f26ac200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f26a4102d00 con 0x7f26a406b1d0 2024-04-24T09:05:30.657 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.657+0000 7f26ac200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f26a4103000 con 0x7f26a4108190 2024-04-24T09:05:30.658 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.657+0000 7f26a9800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26a406a830 0x7f26a410ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:30.658 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.657+0000 7f26a9800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26a406a830 0x7f26a410ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48012/0 (socket says 172.21.15.83:48012) 2024-04-24T09:05:30.658 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.657+0000 7f26a9800640 1 -- 172.21.15.83:0/940355351 learned_addr learned my addr 172.21.15.83:0/940355351 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:05:30.658 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.658+0000 7f26aa200640 1 --2- 172.21.15.83:0/940355351 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f26a4108190 0x7f26a410c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:30.658 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.658+0000 7f26a8e00640 1 --2- 172.21.15.83:0/940355351 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26a406b1d0 0x7f26a410bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:30.658 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.658+0000 7f26a9800640 1 -- 172.21.15.83:0/940355351 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f26a4108190 msgr2=0x7f26a410c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:30.659 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.658+0000 7f26a9800640 1 --2- 172.21.15.83:0/940355351 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f26a4108190 0x7f26a410c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:30.659 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.658+0000 7f26a9800640 1 -- 172.21.15.83:0/940355351 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26a406b1d0 msgr2=0x7f26a410bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:30.659 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.658+0000 7f26a9800640 1 --2- 172.21.15.83:0/940355351 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26a406b1d0 0x7f26a410bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:30.659 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.658+0000 7f26a9800640 1 -- 172.21.15.83:0/940355351 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26a410cb20 con 0x7f26a406a830 2024-04-24T09:05:30.660 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.659+0000 7f26a9800640 1 --2- 172.21.15.83:0/940355351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26a406a830 0x7f26a410ec80 secure :-1 s=READY pgs=2128 cs=0 l=1 rev1=1 crypto rx=0x7f2698005ab0 tx=0x7f2698003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:30.660 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.660+0000 7f26a2000640 1 -- 172.21.15.83:0/940355351 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f269803f070 con 0x7f26a406a830 2024-04-24T09:05:30.660 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.660+0000 7f26a2000640 1 -- 172.21.15.83:0/940355351 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2698031b60 con 0x7f26a406a830 2024-04-24T09:05:30.660 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.660+0000 7f26a2000640 1 -- 172.21.15.83:0/940355351 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2698035030 con 0x7f26a406a830 2024-04-24T09:05:30.660 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.660+0000 7f26ac200640 1 -- 172.21.15.83:0/940355351 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26a41ab1e0 con 0x7f26a406a830 2024-04-24T09:05:30.660 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.660+0000 7f26ac200640 1 -- 172.21.15.83:0/940355351 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26a41ab6b0 con 0x7f26a406a830 2024-04-24T09:05:30.665 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.665+0000 7f26a2000640 1 -- 172.21.15.83:0/940355351 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2698042070 con 0x7f26a406a830 2024-04-24T09:05:30.666 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.665+0000 7f26af07e740 1 Processor -- start 2024-04-24T09:05:30.666 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.666+0000 7f26af07e740 1 -- start start 2024-04-24T09:05:30.667 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.667+0000 7f26af07e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bc1561e1b0 0x55bc1561e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:30.667 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.667+0000 7f26af07e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc1561ead0 0x55bc15628e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:30.667 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.667+0000 7f26af07e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bc15629390 0x55bc1562b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:30.667 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.667+0000 7f26af07e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55bc15584900 con 0x55bc15629390 2024-04-24T09:05:30.667 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.667+0000 7f26af07e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55bc1552af20 con 0x55bc1561ead0 2024-04-24T09:05:30.668 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.667+0000 7f26af07e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55bc155808c0 con 0x55bc1561e1b0 2024-04-24T09:05:30.668 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.667+0000 7f26a9800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc1561ead0 0x55bc15628e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:30.668 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.667+0000 7f26a9800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc1561ead0 0x55bc15628e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:35530/0 (socket says 172.21.15.83:35530) 2024-04-24T09:05:30.668 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.667+0000 7f26a9800640 1 -- 172.21.15.83:0/1070569628 learned_addr learned my addr 172.21.15.83:0/1070569628 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:05:30.668 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.667+0000 7f26aa200640 1 --2- 172.21.15.83:0/1070569628 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bc1561e1b0 0x55bc1561e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:30.668 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.667+0000 7f26a8e00640 1 --2- 172.21.15.83:0/1070569628 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bc15629390 0x55bc1562b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:30.668 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.668+0000 7f26a9800640 1 -- 172.21.15.83:0/1070569628 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bc1561e1b0 msgr2=0x55bc1561e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:30.669 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.668+0000 7f26a9800640 1 --2- 172.21.15.83:0/1070569628 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bc1561e1b0 0x55bc1561e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:30.669 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.668+0000 7f26a9800640 1 -- 172.21.15.83:0/1070569628 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bc15629390 msgr2=0x55bc1562b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:30.669 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.668+0000 7f26a9800640 1 --2- 172.21.15.83:0/1070569628 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bc15629390 0x55bc1562b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:30.669 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.668+0000 7f26a9800640 1 -- 172.21.15.83:0/1070569628 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc1554fe10 con 0x55bc1561ead0 2024-04-24T09:05:30.669 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.669+0000 7f26a2000640 1 --2- 172.21.15.83:0/940355351 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f268c0a3fa0 0x7f268c0a6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:30.669 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.669+0000 7f26a9800640 1 --2- 172.21.15.83:0/1070569628 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc1561ead0 0x55bc15628e50 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f26980ee390 tx=0x7f26980f93b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d65f2277a626896c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:30.669 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.669+0000 7f26aa200640 1 --2- 172.21.15.83:0/940355351 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f268c0a3fa0 0x7f268c0a6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:30.669 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.669+0000 7f2696a00640 1 -- 172.21.15.83:0/1070569628 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26980dbe60 con 0x55bc1561ead0 2024-04-24T09:05:30.670 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.669+0000 7f26a2000640 1 -- 172.21.15.83:0/940355351 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7f26980f1180 con 0x7f26a406a830 2024-04-24T09:05:30.670 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.669+0000 7f2696a00640 1 -- 172.21.15.83:0/1070569628 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2698031d10 con 0x55bc1561ead0 2024-04-24T09:05:30.670 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.669+0000 7f2696a00640 1 -- 172.21.15.83:0/1070569628 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26980b8f40 con 0x55bc1561ead0 2024-04-24T09:05:30.670 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.670+0000 7f26af07e740 1 -- 172.21.15.83:0/1070569628 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc1561ead0 msgr2=0x55bc15628e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:30.670 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.670+0000 7f26af07e740 1 --2- 172.21.15.83:0/1070569628 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc1561ead0 0x55bc15628e50 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f26980ee390 tx=0x7f26980f93b0 comp rx=0 tx=0).stop 2024-04-24T09:05:30.670 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.670+0000 7f26aa200640 1 --2- 172.21.15.83:0/940355351 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f268c0a3fa0 0x7f268c0a6460 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f26a410f3b0 tx=0x7f269c009500 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:30.671 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.670+0000 7f26af07e740 1 -- 172.21.15.83:0/1070569628 shutdown_connections 2024-04-24T09:05:30.671 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.670+0000 7f26af07e740 1 --2- 172.21.15.83:0/1070569628 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bc15629390 0x55bc1562b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:30.671 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.670+0000 7f26af07e740 1 --2- 172.21.15.83:0/1070569628 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc1561ead0 0x55bc15628e50 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:30.671 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.670+0000 7f26af07e740 1 --2- 172.21.15.83:0/1070569628 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bc1561e1b0 0x55bc1561e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:30.671 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.670+0000 7f26af07e740 1 -- 172.21.15.83:0/1070569628 >> 172.21.15.83:0/1070569628 conn(0x55bc1561a6c0 msgr2=0x55bc1561aaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:30.671 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.670+0000 7f26af07e740 1 -- 172.21.15.83:0/1070569628 shutdown_connections 2024-04-24T09:05:30.671 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.671+0000 7f26af07e740 1 -- 172.21.15.83:0/1070569628 wait complete. 2024-04-24T09:05:30.673 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.671+0000 7f26af07e740 1 -- start start 2024-04-24T09:05:30.673 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.671+0000 7f26af07e740 1 Processor -- start 2024-04-24T09:05:30.673 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.672+0000 7f26af07e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bc1561e1b0 0x55bc15626390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:30.673 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.672+0000 7f26af07e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bc1561ead0 0x55bc156268d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:30.673 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.672+0000 7f26af07e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc15629390 0x55bc15626e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:30.673 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.672+0000 7f26af07e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55bc1562bd80 con 0x55bc1561ead0 2024-04-24T09:05:30.674 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.672+0000 7f26af07e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55bc1553e2c0 con 0x55bc15629390 2024-04-24T09:05:30.674 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.672+0000 7f26af07e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55bc1562bf00 con 0x55bc1561e1b0 2024-04-24T09:05:30.674 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.672+0000 7f26af07e740 10 client.0 _ll_register_callbacks cb 0x55bc15296a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:05:30.674 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.672+0000 7f26aa200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc15629390 0x55bc15626e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:30.674 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.672+0000 7f26a8e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bc1561ead0 0x55bc156268d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:30.674 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.672+0000 7f26aa200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc15629390 0x55bc15626e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:35534/0 (socket says 172.21.15.83:35534) 2024-04-24T09:05:30.674 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.672+0000 7f26aa200640 1 -- 172.21.15.83:0/3071633693 learned_addr learned my addr 172.21.15.83:0/3071633693 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:05:30.674 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.672+0000 7f26a9800640 1 --2- 172.21.15.83:0/3071633693 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bc1561e1b0 0x55bc15626390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:30.674 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.673+0000 7f26aa200640 1 -- 172.21.15.83:0/3071633693 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bc1561e1b0 msgr2=0x55bc15626390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:30.674 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.673+0000 7f26aa200640 1 --2- 172.21.15.83:0/3071633693 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bc1561e1b0 0x55bc15626390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:30.674 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.673+0000 7f26aa200640 1 -- 172.21.15.83:0/3071633693 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bc1561ead0 msgr2=0x55bc156268d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:30.674 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.673+0000 7f26aa200640 1 --2- 172.21.15.83:0/3071633693 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bc1561ead0 0x55bc156268d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:30.674 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.673+0000 7f26aa200640 1 -- 172.21.15.83:0/3071633693 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc1554efd0 con 0x55bc15629390 2024-04-24T09:05:30.674 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.673+0000 7f26a9800640 1 --2- 172.21.15.83:0/3071633693 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bc1561e1b0 0x55bc15626390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:05:30.675 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.673+0000 7f26a8e00640 1 --2- 172.21.15.83:0/3071633693 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bc1561ead0 0x55bc156268d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:05:30.675 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.673+0000 7f26aa200640 1 --2- 172.21.15.83:0/3071633693 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc15629390 0x55bc15626e10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f269c003fc0 tx=0x7f269c004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d44aabe1138d3c49 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:30.675 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.673+0000 7f268b400640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:05:30.675 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.674+0000 7f268b400640 1 -- 172.21.15.83:0/3071633693 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f269c02d060 con 0x55bc15629390 2024-04-24T09:05:30.676 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.674+0000 7f26af07e740 1 -- 172.21.15.83:0/3071633693 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55bc15592320 con 0x55bc15629390 2024-04-24T09:05:30.676 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.674+0000 7f268b400640 1 -- 172.21.15.83:0/3071633693 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7f269c018070 con 0x55bc15629390 2024-04-24T09:05:30.676 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.674+0000 7f268b400640 1 -- 172.21.15.83:0/3071633693 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f269c034050 con 0x55bc15629390 2024-04-24T09:05:30.676 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.674+0000 7f26af07e740 10 client.37706 fetch_fsmap learned FSMap version 44 2024-04-24T09:05:30.676 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.674+0000 7f26af07e740 1 -- 172.21.15.83:0/3071633693 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55bc15297210 con 0x55bc15629390 2024-04-24T09:05:30.676 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.675+0000 7f268b400640 1 -- 172.21.15.83:0/3071633693 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55bc15592320 con 0x55bc15629390 2024-04-24T09:05:30.676 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.676+0000 7f268b400640 1 -- 172.21.15.83:0/3071633693 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 44) ==== 8382+0+0 (secure 0 0 0) 0x7f269c01ea70 con 0x55bc15629390 2024-04-24T09:05:30.677 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.676+0000 7f26af07e740 10 client.37706 fetch_fsmap finished waiting for FSMap version 44 2024-04-24T09:05:30.677 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.676+0000 7f26af07e740 10 client.37706 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:05:30.677 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.676+0000 7f26af07e740 20 client.37706 populate_metadata read hostname 'smithi083' 2024-04-24T09:05:30.677 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.676+0000 7f26af07e740 1 --2- 172.21.15.83:0/3071633693 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55bc15639380 0x55bc1563b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:30.677 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.676+0000 7f26af07e740 4 client.37706 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:05:30.677 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.676+0000 7f26af07e740 1 -- 172.21.15.83:0/3071633693 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bc1558c190 con 0x55bc15639380 2024-04-24T09:05:30.677 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.677+0000 7f26a9800640 1 --2- 172.21.15.83:0/3071633693 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55bc15639380 0x55bc1563b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:30.677 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.677+0000 7f26a9800640 1 --2- 172.21.15.83:0/3071633693 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55bc15639380 0x55bc1563b790 crc :-1 s=READY pgs=359 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8809a23440cc17d server_cookie=fd6d991c570badd4 in_seq=0 out_seq=0 2024-04-24T09:05:30.677 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.678+0000 7f268b400640 10 client.37706 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:05:30.679 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.679+0000 7f268b400640 1 -- 172.21.15.83:0/3071633693 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bc1558c190 con 0x55bc15639380 2024-04-24T09:05:30.679 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.679+0000 7f268b400640 10 client.37706 handle_command_reply: tid=0 2024-04-24T09:05:30.679 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.679+0000 7f26af07e740 1 client.37706 shutdown 2024-04-24T09:05:30.679 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.680+0000 7f26af07e740 1 -- 172.21.15.83:0/3071633693 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc15629390 msgr2=0x55bc15626e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:30.679 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.680+0000 7f26af07e740 1 --2- 172.21.15.83:0/3071633693 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc15629390 0x55bc15626e10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f269c003fc0 tx=0x7f269c004a40 comp rx=0 tx=0).stop 2024-04-24T09:05:30.680 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.680+0000 7f26af07e740 1 -- 172.21.15.83:0/3071633693 shutdown_connections 2024-04-24T09:05:30.681 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.680+0000 7f26af07e740 1 --2- 172.21.15.83:0/3071633693 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55bc15639380 0x55bc1563b790 crc :-1 s=READY pgs=359 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:30.681 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.680+0000 7f26af07e740 1 --2- 172.21.15.83:0/3071633693 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc15629390 0x55bc15626e10 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:30.681 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.680+0000 7f26af07e740 1 --2- 172.21.15.83:0/3071633693 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bc1561ead0 0x55bc156268d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:30.681 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.680+0000 7f26af07e740 1 --2- 172.21.15.83:0/3071633693 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bc1561e1b0 0x55bc15626390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:30.681 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.680+0000 7f268b400640 0 client.37706 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:05:30.681 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.680+0000 7f26af07e740 1 -- 172.21.15.83:0/3071633693 >> 172.21.15.83:0/3071633693 conn(0x55bc1561a6c0 msgr2=0x55bc156203f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:30.681 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.681+0000 7f26af07e740 1 -- 172.21.15.83:0/3071633693 shutdown_connections 2024-04-24T09:05:30.681 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.681+0000 7f26af07e740 1 -- 172.21.15.83:0/3071633693 wait complete. 2024-04-24T09:05:30.681 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.681+0000 7f26af07e740 20 client.37706 trim_cache size 0 max 16384 2024-04-24T09:05:30.711 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.711+0000 7f26af07e740 1 Processor -- start 2024-04-24T09:05:30.711 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.711+0000 7f26af07e740 1 -- start start 2024-04-24T09:05:30.712 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.712+0000 7f26af07e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc1561e1b0 0x55bc15619fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:30.712 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.712+0000 7f26af07e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bc1561ead0 0x55bc15626c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:30.712 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.712+0000 7f26af07e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bc15629390 0x55bc156410f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:30.712 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.712+0000 7f26af07e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f26a41b5ac0 con 0x55bc15629390 2024-04-24T09:05:30.712 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.712+0000 7f26af07e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f26a41b76f0 con 0x55bc1561e1b0 2024-04-24T09:05:30.712 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.712+0000 7f26af07e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f26a41b6190 con 0x55bc1561ead0 2024-04-24T09:05:30.713 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.713+0000 7f26a9800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc1561e1b0 0x55bc15619fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:30.713 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.713+0000 7f26a9800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc1561e1b0 0x55bc15619fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:35540/0 (socket says 172.21.15.83:35540) 2024-04-24T09:05:30.713 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.713+0000 7f26a9800640 1 -- 172.21.15.83:0/2142262712 learned_addr learned my addr 172.21.15.83:0/2142262712 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:05:30.713 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.713+0000 7f26a8e00640 1 --2- 172.21.15.83:0/2142262712 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bc1561ead0 0x55bc15626c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:30.713 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.713+0000 7f26aa200640 1 --2- 172.21.15.83:0/2142262712 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bc15629390 0x55bc156410f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:30.714 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.713+0000 7f26a9800640 1 -- 172.21.15.83:0/2142262712 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bc1561ead0 msgr2=0x55bc15626c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:30.714 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.713+0000 7f26a9800640 1 --2- 172.21.15.83:0/2142262712 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bc1561ead0 0x55bc15626c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:30.714 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.713+0000 7f26a9800640 1 -- 172.21.15.83:0/2142262712 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bc15629390 msgr2=0x55bc156410f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:30.714 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.713+0000 7f26a9800640 1 --2- 172.21.15.83:0/2142262712 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bc15629390 0x55bc156410f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:30.714 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.713+0000 7f26a9800640 1 -- 172.21.15.83:0/2142262712 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc15363620 con 0x55bc1561e1b0 2024-04-24T09:05:30.714 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.713+0000 7f26a9800640 1 --2- 172.21.15.83:0/2142262712 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc1561e1b0 0x55bc15619fc0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2698000c00 tx=0x7f269802a480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58475f068a432c87 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:30.714 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.714+0000 7f2696000640 1 -- 172.21.15.83:0/2142262712 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26980dbe60 con 0x55bc1561e1b0 2024-04-24T09:05:30.714 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.714+0000 7f2696000640 1 -- 172.21.15.83:0/2142262712 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f26980bcca0 con 0x55bc1561e1b0 2024-04-24T09:05:30.715 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.714+0000 7f2696000640 1 -- 172.21.15.83:0/2142262712 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26980bc600 con 0x55bc1561e1b0 2024-04-24T09:05:30.715 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.715+0000 7f26af07e740 1 -- 172.21.15.83:0/2142262712 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc1561e1b0 msgr2=0x55bc15619fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:30.715 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.715+0000 7f26af07e740 1 --2- 172.21.15.83:0/2142262712 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc1561e1b0 0x55bc15619fc0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2698000c00 tx=0x7f269802a480 comp rx=0 tx=0).stop 2024-04-24T09:05:30.715 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.715+0000 7f26af07e740 1 -- 172.21.15.83:0/2142262712 shutdown_connections 2024-04-24T09:05:30.716 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.715+0000 7f26af07e740 1 --2- 172.21.15.83:0/2142262712 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bc15629390 0x55bc156410f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:30.716 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.716+0000 7f26af07e740 1 --2- 172.21.15.83:0/2142262712 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bc1561ead0 0x55bc15626c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:30.716 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.716+0000 7f26af07e740 1 --2- 172.21.15.83:0/2142262712 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc1561e1b0 0x55bc15619fc0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:30.716 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.716+0000 7f26af07e740 1 -- 172.21.15.83:0/2142262712 >> 172.21.15.83:0/2142262712 conn(0x55bc1561a6c0 msgr2=0x55bc156203f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:30.716 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.716+0000 7f26af07e740 1 -- 172.21.15.83:0/2142262712 shutdown_connections 2024-04-24T09:05:30.716 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.716+0000 7f26af07e740 1 -- 172.21.15.83:0/2142262712 wait complete. 2024-04-24T09:05:30.717 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.716+0000 7f26af07e740 1 -- start start 2024-04-24T09:05:30.717 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.716+0000 7f26af07e740 1 Processor -- start 2024-04-24T09:05:30.717 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.717+0000 7f26af07e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bc1561e1b0 0x55bc156314e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:30.717 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.717+0000 7f26af07e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc1561ead0 0x55bc1562c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:30.718 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.717+0000 7f26af07e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bc15629390 0x55bc1562cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:30.718 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.717+0000 7f26af07e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f26a41b6fc0 con 0x55bc1561e1b0 2024-04-24T09:05:30.718 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.717+0000 7f26af07e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f26a41b6890 con 0x55bc1561ead0 2024-04-24T09:05:30.718 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.717+0000 7f26af07e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f26a41b3ac0 con 0x55bc15629390 2024-04-24T09:05:30.718 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.717+0000 7f26af07e740 10 client.0 _ll_register_callbacks cb 0x7f269c009db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:05:30.718 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.717+0000 7f26a8e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc1561ead0 0x55bc1562c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:30.718 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.717+0000 7f26a8e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc1561ead0 0x55bc1562c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:35556/0 (socket says 172.21.15.83:35556) 2024-04-24T09:05:30.718 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.717+0000 7f26a8e00640 1 -- 172.21.15.83:0/3136077119 learned_addr learned my addr 172.21.15.83:0/3136077119 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:05:30.718 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.717+0000 7f26a9800640 1 --2- 172.21.15.83:0/3136077119 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bc1561e1b0 0x55bc156314e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:30.718 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.717+0000 7f26aa200640 1 --2- 172.21.15.83:0/3136077119 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bc15629390 0x55bc1562cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:30.718 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.718+0000 7f26a8e00640 1 -- 172.21.15.83:0/3136077119 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bc15629390 msgr2=0x55bc1562cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:30.718 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.718+0000 7f26a8e00640 1 --2- 172.21.15.83:0/3136077119 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bc15629390 0x55bc1562cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:30.718 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.718+0000 7f26a8e00640 1 -- 172.21.15.83:0/3136077119 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bc1561e1b0 msgr2=0x55bc156314e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:30.718 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.718+0000 7f26a8e00640 1 --2- 172.21.15.83:0/3136077119 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bc1561e1b0 0x55bc156314e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:30.719 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.718+0000 7f26a8e00640 1 -- 172.21.15.83:0/3136077119 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc1562d630 con 0x55bc1561ead0 2024-04-24T09:05:30.719 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.718+0000 7f26aa200640 1 --2- 172.21.15.83:0/3136077119 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bc15629390 0x55bc1562cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:05:30.720 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.718+0000 7f26a8e00640 1 --2- 172.21.15.83:0/3136077119 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc1561ead0 0x55bc1562c850 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f26a410d4e0 tx=0x7f269000f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a74728fdc497c79 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:30.720 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.718+0000 7f268b400640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:05:30.720 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.718+0000 7f268b400640 1 -- 172.21.15.83:0/3136077119 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2690023060 con 0x55bc1561ead0 2024-04-24T09:05:30.720 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.718+0000 7f26af07e740 1 -- 172.21.15.83:0/3136077119 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26980bcca0 con 0x55bc1561ead0 2024-04-24T09:05:30.720 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.718+0000 7f268b400640 1 -- 172.21.15.83:0/3136077119 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7f2690004250 con 0x55bc1561ead0 2024-04-24T09:05:30.720 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.719+0000 7f268b400640 1 -- 172.21.15.83:0/3136077119 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f269002a050 con 0x55bc1561ead0 2024-04-24T09:05:30.720 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.719+0000 7f26af07e740 10 client.37712 fetch_fsmap learned FSMap version 44 2024-04-24T09:05:30.720 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.719+0000 7f26af07e740 1 -- 172.21.15.83:0/3136077119 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55bc1569fa70 con 0x55bc1561ead0 2024-04-24T09:05:30.720 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.719+0000 7f268b400640 1 -- 172.21.15.83:0/3136077119 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f26980bcca0 con 0x55bc1561ead0 2024-04-24T09:05:30.720 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.719+0000 7f268b400640 1 -- 172.21.15.83:0/3136077119 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 44) ==== 8382+0+0 (secure 0 0 0) 0x7f269001e440 con 0x55bc1561ead0 2024-04-24T09:05:30.720 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.719+0000 7f26af07e740 10 client.37712 fetch_fsmap finished waiting for FSMap version 44 2024-04-24T09:05:30.721 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.719+0000 7f26af07e740 10 client.37712 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:05:30.721 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.719+0000 7f26af07e740 20 client.37712 populate_metadata read hostname 'smithi083' 2024-04-24T09:05:30.721 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.719+0000 7f26af07e740 1 --2- 172.21.15.83:0/3136077119 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55bc156a02b0 0x55bc156a26d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:30.721 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.719+0000 7f26af07e740 4 client.37712 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T09:05:30.721 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.719+0000 7f26af07e740 1 -- 172.21.15.83:0/3136077119 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bc155084b0 con 0x55bc156a02b0 2024-04-24T09:05:30.721 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.720+0000 7f26a9800640 1 --2- 172.21.15.83:0/3136077119 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55bc156a02b0 0x55bc156a26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:30.721 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.720+0000 7f268b400640 10 client.37712 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:05:30.721 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.720+0000 7f26a9800640 1 --2- 172.21.15.83:0/3136077119 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55bc156a02b0 0x55bc156a26d0 crc :-1 s=READY pgs=360 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2b00157b7932feb server_cookie=eb95818f9a87a5c4 in_seq=0 out_seq=0 2024-04-24T09:05:30.996 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.995+0000 7f268b400640 1 -- 172.21.15.83:0/3136077119 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bc155084b0 con 0x55bc156a02b0 2024-04-24T09:05:30.996 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.995+0000 7f268b400640 10 client.37712 handle_command_reply: tid=0 2024-04-24T09:05:30.996 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.995+0000 7f26af07e740 1 client.37712 shutdown 2024-04-24T09:05:30.996 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.995+0000 7f26af07e740 1 -- 172.21.15.83:0/3136077119 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc1561ead0 msgr2=0x55bc1562c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:30.996 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.995+0000 7f26af07e740 1 --2- 172.21.15.83:0/3136077119 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc1561ead0 0x55bc1562c850 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f26a410d4e0 tx=0x7f269000f790 comp rx=0 tx=0).stop 2024-04-24T09:05:30.996 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.996+0000 7f26af07e740 1 -- 172.21.15.83:0/3136077119 shutdown_connections 2024-04-24T09:05:30.996 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.996+0000 7f26af07e740 1 --2- 172.21.15.83:0/3136077119 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55bc156a02b0 0x55bc156a26d0 crc :-1 s=READY pgs=360 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:30.997 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.996+0000 7f26af07e740 1 --2- 172.21.15.83:0/3136077119 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bc15629390 0x55bc1562cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:30.997 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.996+0000 7f26af07e740 1 --2- 172.21.15.83:0/3136077119 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc1561ead0 0x55bc1562c850 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:30.997 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.996+0000 7f26af07e740 1 --2- 172.21.15.83:0/3136077119 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bc1561e1b0 0x55bc156314e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:30.997 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.996+0000 7f268b400640 0 client.37712 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:05:30.997 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.996+0000 7f26af07e740 1 -- 172.21.15.83:0/3136077119 >> 172.21.15.83:0/3136077119 conn(0x55bc1561a6c0 msgr2=0x55bc15627bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:30.997 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.996+0000 7f26af07e740 1 -- 172.21.15.83:0/3136077119 shutdown_connections 2024-04-24T09:05:30.997 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.996+0000 7f26af07e740 1 -- 172.21.15.83:0/3136077119 wait complete. 2024-04-24T09:05:30.997 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.996+0000 7f26af07e740 20 client.37712 trim_cache size 0 max 16384 2024-04-24T09:05:30.997 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T09:05:30.997 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T09:05:30.997 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T09:05:30.998 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T09:05:30.998 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.997+0000 7f26ac200640 1 -- 172.21.15.83:0/940355351 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f268c0a3fa0 msgr2=0x7f268c0a6460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:30.998 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.997+0000 7f26ac200640 1 --2- 172.21.15.83:0/940355351 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f268c0a3fa0 0x7f268c0a6460 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f26a410f3b0 tx=0x7f269c009500 comp rx=0 tx=0).stop 2024-04-24T09:05:30.998 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.997+0000 7f26ac200640 1 -- 172.21.15.83:0/940355351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26a406a830 msgr2=0x7f26a410ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:30.998 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.997+0000 7f26ac200640 1 --2- 172.21.15.83:0/940355351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26a406a830 0x7f26a410ec80 secure :-1 s=READY pgs=2128 cs=0 l=1 rev1=1 crypto rx=0x7f2698005ab0 tx=0x7f2698003c10 comp rx=0 tx=0).stop 2024-04-24T09:05:30.999 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.997+0000 7f26ac200640 1 -- 172.21.15.83:0/940355351 shutdown_connections 2024-04-24T09:05:30.999 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.997+0000 7f26ac200640 1 --2- 172.21.15.83:0/940355351 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f268c0a3fa0 0x7f268c0a6460 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:30.999 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.997+0000 7f26ac200640 1 --2- 172.21.15.83:0/940355351 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f26a4108190 0x7f26a410c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:30.999 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.997+0000 7f26ac200640 1 --2- 172.21.15.83:0/940355351 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26a406b1d0 0x7f26a410bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:30.999 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.997+0000 7f26ac200640 1 --2- 172.21.15.83:0/940355351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26a406a830 0x7f26a410ec80 unknown :-1 s=CLOSED pgs=2128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:30.999 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.997+0000 7f26ac200640 1 -- 172.21.15.83:0/940355351 >> 172.21.15.83:0/940355351 conn(0x7f26a4076d00 msgr2=0x7f26a40ffd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:30.999 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.997+0000 7f26ac200640 1 -- 172.21.15.83:0/940355351 shutdown_connections 2024-04-24T09:05:30.999 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:30.997+0000 7f26ac200640 1 -- 172.21.15.83:0/940355351 wait complete. 2024-04-24T09:05:31.152 INFO:tasks.workunit.client.0.smithi083.stdout: 10 307 56.38 MB/sec warmup 2 sec latency 323.934 ms 2024-04-24T09:05:32.152 INFO:tasks.workunit.client.0.smithi083.stdout: 10 473 55.19 MB/sec warmup 3 sec latency 116.907 ms 2024-04-24T09:05:32.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:05:32 smithi138 ceph-mon[31441]: pgmap v436: 129 pgs: 129 active+clean; 76 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 275 KiB/s wr, 50 op/s 2024-04-24T09:05:32.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:05:32 smithi138 ceph-mon[31441]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:05:32.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:05:32 smithi189 ceph-mon[31791]: pgmap v436: 129 pgs: 129 active+clean; 76 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 275 KiB/s wr, 50 op/s 2024-04-24T09:05:32.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:05:32 smithi189 ceph-mon[31791]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:05:32.552 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:05:32 smithi083 ceph-mon[27907]: pgmap v436: 129 pgs: 129 active+clean; 76 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 275 KiB/s wr, 50 op/s 2024-04-24T09:05:32.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:05:32 smithi083 ceph-mon[27907]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:05:33.152 INFO:tasks.workunit.client.0.smithi083.stdout: 10 634 51.27 MB/sec warmup 4 sec latency 128.797 ms 2024-04-24T09:05:34.152 INFO:tasks.workunit.client.0.smithi083.stdout: 10 764 42.73 MB/sec warmup 5 sec latency 133.206 ms 2024-04-24T09:05:34.187 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:05:34 smithi189 ceph-mon[31791]: pgmap v437: 129 pgs: 129 active+clean; 265 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 16 MiB/s wr, 228 op/s 2024-04-24T09:05:34.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:05:34 smithi138 ceph-mon[31441]: pgmap v437: 129 pgs: 129 active+clean; 265 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 16 MiB/s wr, 228 op/s 2024-04-24T09:05:34.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:05:34 smithi138 ceph-mon[31441]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:05:34.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:05:34 smithi189 ceph-mon[31791]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:05:34.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:05:34 smithi083 ceph-mon[27907]: pgmap v437: 129 pgs: 129 active+clean; 265 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 16 MiB/s wr, 228 op/s 2024-04-24T09:05:34.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:05:34 smithi083 ceph-mon[27907]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:05:35.152 INFO:tasks.workunit.client.0.smithi083.stdout: 10 844 35.91 MB/sec warmup 6 sec latency 195.313 ms 2024-04-24T09:05:35.954 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:05:35 smithi138 ceph-mon[31441]: pgmap v438: 129 pgs: 129 active+clean; 265 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 16 MiB/s wr, 208 op/s 2024-04-24T09:05:35.956 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:05:35 smithi189 ceph-mon[31791]: pgmap v438: 129 pgs: 129 active+clean; 265 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 16 MiB/s wr, 208 op/s 2024-04-24T09:05:36.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:05:35 smithi083 ceph-mon[27907]: pgmap v438: 129 pgs: 129 active+clean; 265 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 16 MiB/s wr, 208 op/s 2024-04-24T09:05:36.152 INFO:tasks.workunit.client.0.smithi083.stdout: 10 918 31.06 MB/sec warmup 7 sec latency 197.464 ms 2024-04-24T09:05:37.153 INFO:tasks.workunit.client.0.smithi083.stdout: 10 1013 28.44 MB/sec warmup 8 sec latency 212.766 ms 2024-04-24T09:05:38.153 INFO:tasks.workunit.client.0.smithi083.stdout: 10 1082 25.37 MB/sec warmup 9 sec latency 431.536 ms 2024-04-24T09:05:38.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:05:37 smithi138 ceph-mon[31441]: pgmap v439: 129 pgs: 129 active+clean; 265 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 16 MiB/s wr, 208 op/s 2024-04-24T09:05:38.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:05:37 smithi189 ceph-mon[31791]: pgmap v439: 129 pgs: 129 active+clean; 265 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 16 MiB/s wr, 208 op/s 2024-04-24T09:05:38.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:05:37 smithi083 ceph-mon[27907]: pgmap v439: 129 pgs: 129 active+clean; 265 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 16 MiB/s wr, 208 op/s 2024-04-24T09:05:39.153 INFO:tasks.workunit.client.0.smithi083.stdout: 10 1197 23.07 MB/sec warmup 10 sec latency 90.266 ms 2024-04-24T09:05:40.153 INFO:tasks.workunit.client.0.smithi083.stdout: 10 1355 22.35 MB/sec warmup 11 sec latency 104.000 ms 2024-04-24T09:05:40.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:05:39 smithi138 ceph-mon[31441]: pgmap v440: 129 pgs: 129 active+clean; 280 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 598 KiB/s rd, 18 MiB/s wr, 280 op/s 2024-04-24T09:05:40.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:05:39 smithi189 ceph-mon[31791]: pgmap v440: 129 pgs: 129 active+clean; 280 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 598 KiB/s rd, 18 MiB/s wr, 280 op/s 2024-04-24T09:05:40.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:05:39 smithi083 ceph-mon[27907]: pgmap v440: 129 pgs: 129 active+clean; 280 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 598 KiB/s rd, 18 MiB/s wr, 280 op/s 2024-04-24T09:05:41.153 INFO:tasks.workunit.client.0.smithi083.stdout: 10 1469 21.71 MB/sec warmup 12 sec latency 332.282 ms 2024-04-24T09:05:42.153 INFO:tasks.workunit.client.0.smithi083.stdout: 10 1548 20.08 MB/sec warmup 13 sec latency 453.897 ms 2024-04-24T09:05:42.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:05:41 smithi138 ceph-mon[31441]: pgmap v441: 129 pgs: 129 active+clean; 280 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 510 KiB/s rd, 18 MiB/s wr, 249 op/s 2024-04-24T09:05:42.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:05:41 smithi189 ceph-mon[31791]: pgmap v441: 129 pgs: 129 active+clean; 280 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 510 KiB/s rd, 18 MiB/s wr, 249 op/s 2024-04-24T09:05:42.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:05:41 smithi083 ceph-mon[27907]: pgmap v441: 129 pgs: 129 active+clean; 280 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 510 KiB/s rd, 18 MiB/s wr, 249 op/s 2024-04-24T09:05:43.153 INFO:tasks.workunit.client.0.smithi083.stdout: 10 1662 18.72 MB/sec warmup 14 sec latency 479.483 ms 2024-04-24T09:05:44.153 INFO:tasks.workunit.client.0.smithi083.stdout: 10 1802 17.55 MB/sec warmup 15 sec latency 73.859 ms 2024-04-24T09:05:44.171 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.171+0000 7f9695c00640 1 -- 172.21.15.83:0/3532463622 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9690102fc0 msgr2=0x7f96901033a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:44.171 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.171+0000 7f9695c00640 1 --2- 172.21.15.83:0/3532463622 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9690102fc0 0x7f96901033a0 secure :-1 s=READY pgs=2129 cs=0 l=1 rev1=1 crypto rx=0x7f9680009900 tx=0x7f968002a480 comp rx=0 tx=0).stop 2024-04-24T09:05:44.172 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.171+0000 7f9695c00640 1 -- 172.21.15.83:0/3532463622 shutdown_connections 2024-04-24T09:05:44.172 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.171+0000 7f9695c00640 1 --2- 172.21.15.83:0/3532463622 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f969010c3e0 0x7f969010e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:44.172 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.171+0000 7f9695c00640 1 --2- 172.21.15.83:0/3532463622 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f96901038e0 0x7f969010bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:44.172 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.171+0000 7f9695c00640 1 --2- 172.21.15.83:0/3532463622 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9690102fc0 0x7f96901033a0 unknown :-1 s=CLOSED pgs=2129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:44.172 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.171+0000 7f9695c00640 1 -- 172.21.15.83:0/3532463622 >> 172.21.15.83:0/3532463622 conn(0x7f96900fc820 msgr2=0x7f96900fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:44.172 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.171+0000 7f9695c00640 1 -- 172.21.15.83:0/3532463622 shutdown_connections 2024-04-24T09:05:44.172 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.171+0000 7f9695c00640 1 -- 172.21.15.83:0/3532463622 wait complete. 2024-04-24T09:05:44.172 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.172+0000 7f9695c00640 1 Processor -- start 2024-04-24T09:05:44.173 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.173+0000 7f9695c00640 1 -- start start 2024-04-24T09:05:44.173 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.173+0000 7f9695c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9690102fc0 0x7f96901a05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:44.173 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.173+0000 7f9695c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96901038e0 0x7f96901a0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:44.173 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.173+0000 7f9695c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f969010c3e0 0x7f969019a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:44.174 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.173+0000 7f9695c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f96901120a0 con 0x7f969010c3e0 2024-04-24T09:05:44.174 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.173+0000 7f9695c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9690111f20 con 0x7f96901038e0 2024-04-24T09:05:44.174 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.173+0000 7f9695c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9690112220 con 0x7f9690102fc0 2024-04-24T09:05:44.174 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.173+0000 7f9687e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96901038e0 0x7f96901a0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:44.174 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.173+0000 7f9687e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96901038e0 0x7f96901a0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:60508/0 (socket says 172.21.15.83:60508) 2024-04-24T09:05:44.174 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.174+0000 7f9687e00640 1 -- 172.21.15.83:0/2192272101 learned_addr learned my addr 172.21.15.83:0/2192272101 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:05:44.174 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.174+0000 7f9695200640 1 --2- 172.21.15.83:0/2192272101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f969010c3e0 0x7f969019a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:44.174 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.174+0000 7f968fe00640 1 --2- 172.21.15.83:0/2192272101 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9690102fc0 0x7f96901a05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:44.175 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.174+0000 7f9687e00640 1 -- 172.21.15.83:0/2192272101 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9690102fc0 msgr2=0x7f96901a05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:44.175 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.174+0000 7f9687e00640 1 --2- 172.21.15.83:0/2192272101 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9690102fc0 0x7f96901a05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:44.175 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.174+0000 7f9687e00640 1 -- 172.21.15.83:0/2192272101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f969010c3e0 msgr2=0x7f969019a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:44.175 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.174+0000 7f9687e00640 1 --2- 172.21.15.83:0/2192272101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f969010c3e0 0x7f969019a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:44.175 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.174+0000 7f9687e00640 1 -- 172.21.15.83:0/2192272101 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f969019af40 con 0x7f96901038e0 2024-04-24T09:05:44.175 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.174+0000 7f9687e00640 1 --2- 172.21.15.83:0/2192272101 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96901038e0 0x7f96901a0b00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f967800d740 tx=0x7f967800dc00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:44.175 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.175+0000 7f968d600640 1 -- 172.21.15.83:0/2192272101 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9678017070 con 0x7f96901038e0 2024-04-24T09:05:44.175 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.175+0000 7f9695c00640 1 -- 172.21.15.83:0/2192272101 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f969019b230 con 0x7f96901038e0 2024-04-24T09:05:44.176 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.175+0000 7f968d600640 1 -- 172.21.15.83:0/2192272101 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f967800de50 con 0x7f96901038e0 2024-04-24T09:05:44.176 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.175+0000 7f9695c00640 1 -- 172.21.15.83:0/2192272101 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96901a7540 con 0x7f96901038e0 2024-04-24T09:05:44.176 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.175+0000 7f968d600640 1 -- 172.21.15.83:0/2192272101 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f967800f620 con 0x7f96901038e0 2024-04-24T09:05:44.178 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.179+0000 7f969a036740 1 Processor -- start 2024-04-24T09:05:44.179 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.179+0000 7f969a036740 1 -- start start 2024-04-24T09:05:44.179 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.179+0000 7f969a036740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560149a2f3c0 0x560149a2f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:44.179 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.179+0000 7f969a036740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560149a2fce0 0x560149a3a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:44.179 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.179+0000 7f969a036740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560149a3a5a0 0x560149a3c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:44.179 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.179+0000 7f969a036740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x560149995900 con 0x560149a2fce0 2024-04-24T09:05:44.179 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.179+0000 7f969a036740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56014993bf20 con 0x560149a3a5a0 2024-04-24T09:05:44.179 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.179+0000 7f969a036740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5601499918c0 con 0x560149a2f3c0 2024-04-24T09:05:44.180 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.179+0000 7f968d600640 1 -- 172.21.15.83:0/2192272101 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9678009c30 con 0x7f96901038e0 2024-04-24T09:05:44.180 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.180+0000 7f9695200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560149a2f3c0 0x560149a2f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:44.180 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.180+0000 7f9695200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560149a2f3c0 0x560149a2f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:57482/0 (socket says 172.21.15.83:57482) 2024-04-24T09:05:44.180 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.180+0000 7f9695200640 1 -- 172.21.15.83:0/3571668403 learned_addr learned my addr 172.21.15.83:0/3571668403 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:05:44.180 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.180+0000 7f968fe00640 1 --2- 172.21.15.83:0/3571668403 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560149a2fce0 0x560149a3a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:44.181 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.180+0000 7f968fe00640 1 -- 172.21.15.83:0/3571668403 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560149a2f3c0 msgr2=0x560149a2f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:44.181 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.180+0000 7f968fe00640 1 --2- 172.21.15.83:0/3571668403 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560149a2f3c0 0x560149a2f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:44.181 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.181+0000 7f968fe00640 1 -- 172.21.15.83:0/3571668403 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560149a3a5a0 msgr2=0x560149a3c990 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-24T09:05:44.181 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.181+0000 7f968fe00640 1 --2- 172.21.15.83:0/3571668403 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560149a3a5a0 0x560149a3c990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:44.181 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.181+0000 7f968fe00640 1 -- 172.21.15.83:0/3571668403 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560149960e10 con 0x560149a2fce0 2024-04-24T09:05:44.181 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.181+0000 7f968fe00640 1 --2- 172.21.15.83:0/3571668403 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560149a2fce0 0x560149a3a060 secure :-1 s=READY pgs=2130 cs=0 l=1 rev1=1 crypto rx=0x7f968002aa70 tx=0x7f9680004190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ff3ac592da26345 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:44.181 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.182+0000 7f9685600640 1 -- 172.21.15.83:0/3571668403 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9680035040 con 0x560149a2fce0 2024-04-24T09:05:44.182 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.182+0000 7f9685600640 1 -- 172.21.15.83:0/3571668403 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f96800054f0 con 0x560149a2fce0 2024-04-24T09:05:44.183 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.182+0000 7f9685600640 1 -- 172.21.15.83:0/3571668403 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96800035f0 con 0x560149a2fce0 2024-04-24T09:05:44.183 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.182+0000 7f968d600640 1 --2- 172.21.15.83:0/2192272101 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f96740a3e10 0x7f96740a62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:44.183 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.182+0000 7f968d600640 1 -- 172.21.15.83:0/2192272101 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7f96780d06c0 con 0x7f96901038e0 2024-04-24T09:05:44.183 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.182+0000 7f9695200640 1 --2- 172.21.15.83:0/2192272101 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f96740a3e10 0x7f96740a62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:44.183 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.183+0000 7f969a036740 1 -- 172.21.15.83:0/3571668403 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560149a2fce0 msgr2=0x560149a3a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:44.183 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.183+0000 7f969a036740 1 --2- 172.21.15.83:0/3571668403 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560149a2fce0 0x560149a3a060 secure :-1 s=READY pgs=2130 cs=0 l=1 rev1=1 crypto rx=0x7f968002aa70 tx=0x7f9680004190 comp rx=0 tx=0).stop 2024-04-24T09:05:44.184 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.183+0000 7f969a036740 1 -- 172.21.15.83:0/3571668403 shutdown_connections 2024-04-24T09:05:44.184 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.183+0000 7f969a036740 1 --2- 172.21.15.83:0/3571668403 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560149a3a5a0 0x560149a3c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:44.184 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.183+0000 7f969a036740 1 --2- 172.21.15.83:0/3571668403 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560149a2fce0 0x560149a3a060 unknown :-1 s=CLOSED pgs=2130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:44.184 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.183+0000 7f969a036740 1 --2- 172.21.15.83:0/3571668403 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560149a2f3c0 0x560149a2f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:44.184 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.183+0000 7f969a036740 1 -- 172.21.15.83:0/3571668403 >> 172.21.15.83:0/3571668403 conn(0x560149a2b8d0 msgr2=0x560149a2bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:44.184 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.183+0000 7f969a036740 1 -- 172.21.15.83:0/3571668403 shutdown_connections 2024-04-24T09:05:44.184 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.184+0000 7f969a036740 1 -- 172.21.15.83:0/3571668403 wait complete. 2024-04-24T09:05:44.185 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.184+0000 7f969a036740 1 -- start start 2024-04-24T09:05:44.185 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.185+0000 7f969a036740 1 Processor -- start 2024-04-24T09:05:44.185 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.185+0000 7f9695200640 1 --2- 172.21.15.83:0/2192272101 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f96740a3e10 0x7f96740a62d0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f9688011630 tx=0x7f9688009500 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:44.186 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.185+0000 7f969a036740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560149a2f3c0 0x560149a37810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:44.186 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.185+0000 7f969a036740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560149a3a5a0 0x560149a37d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:44.186 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.185+0000 7f969a036740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560149a383f0 0x560149a4ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:44.186 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.185+0000 7f969a036740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x560149a3d020 con 0x560149a2f3c0 2024-04-24T09:05:44.186 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.185+0000 7f969a036740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56014994f2c0 con 0x560149a3a5a0 2024-04-24T09:05:44.186 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.185+0000 7f969a036740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x560149a3d1a0 con 0x560149a383f0 2024-04-24T09:05:44.186 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.185+0000 7f969a036740 10 client.0 _ll_register_callbacks cb 0x5601496a7a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:05:44.187 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.186+0000 7f9687e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560149a3a5a0 0x560149a37d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:44.187 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.186+0000 7f9687e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560149a3a5a0 0x560149a37d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:60532/0 (socket says 172.21.15.83:60532) 2024-04-24T09:05:44.187 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.186+0000 7f9687e00640 1 -- 172.21.15.83:0/2862487538 learned_addr learned my addr 172.21.15.83:0/2862487538 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:05:44.187 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.186+0000 7f968fe00640 1 --2- 172.21.15.83:0/2862487538 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560149a2f3c0 0x560149a37810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:44.187 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.186+0000 7f9695200640 1 --2- 172.21.15.83:0/2862487538 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560149a383f0 0x560149a4ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:44.187 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.186+0000 7f9687e00640 1 -- 172.21.15.83:0/2862487538 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560149a383f0 msgr2=0x560149a4ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:44.187 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.186+0000 7f9687e00640 1 --2- 172.21.15.83:0/2862487538 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560149a383f0 0x560149a4ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:44.187 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.186+0000 7f9687e00640 1 -- 172.21.15.83:0/2862487538 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560149a2f3c0 msgr2=0x560149a37810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:44.187 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.186+0000 7f9687e00640 1 --2- 172.21.15.83:0/2862487538 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560149a2f3c0 0x560149a37810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:44.187 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.186+0000 7f9687e00640 1 -- 172.21.15.83:0/2862487538 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56014995ffd0 con 0x560149a3a5a0 2024-04-24T09:05:44.189 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.187+0000 7f9687e00640 1 --2- 172.21.15.83:0/2862487538 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560149a3a5a0 0x560149a37d50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9678005fd0 tx=0x7f967809bcb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=252f871f11e29567 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:44.189 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.187+0000 7f967e000640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:05:44.189 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.187+0000 7f967e000640 1 -- 172.21.15.83:0/2862487538 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9678020050 con 0x560149a3a5a0 2024-04-24T09:05:44.189 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.187+0000 7f967e000640 1 -- 172.21.15.83:0/2862487538 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7f96780b56d0 con 0x560149a3a5a0 2024-04-24T09:05:44.189 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.187+0000 7f969a036740 1 -- 172.21.15.83:0/2862487538 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96800054f0 con 0x560149a3a5a0 2024-04-24T09:05:44.189 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.187+0000 7f967e000640 1 -- 172.21.15.83:0/2862487538 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f96780cde10 con 0x560149a3a5a0 2024-04-24T09:05:44.189 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.188+0000 7f969a036740 10 client.37721 fetch_fsmap learned FSMap version 47 2024-04-24T09:05:44.189 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.188+0000 7f969a036740 1 -- 172.21.15.83:0/2862487538 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5601496a8210 con 0x560149a3a5a0 2024-04-24T09:05:44.189 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.188+0000 7f967e000640 1 -- 172.21.15.83:0/2862487538 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f96800054f0 con 0x560149a3a5a0 2024-04-24T09:05:44.189 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.188+0000 7f967e000640 1 -- 172.21.15.83:0/2862487538 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 47) ==== 8398+0+0 (secure 0 0 0) 0x7f96780a49c0 con 0x560149a3a5a0 2024-04-24T09:05:44.189 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.188+0000 7f969a036740 10 client.37721 fetch_fsmap finished waiting for FSMap version 47 2024-04-24T09:05:44.189 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.188+0000 7f969a036740 10 client.37721 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:05:44.189 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.188+0000 7f969a036740 20 client.37721 populate_metadata read hostname 'smithi083' 2024-04-24T09:05:44.189 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.188+0000 7f969a036740 1 --2- 172.21.15.83:0/2862487538 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x560149a536a0 0x560149a55ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:44.190 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.188+0000 7f969a036740 4 client.37721 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:05:44.190 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.188+0000 7f969a036740 1 -- 172.21.15.83:0/2862487538 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5601499a3320 con 0x560149a536a0 2024-04-24T09:05:44.190 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.189+0000 7f968fe00640 1 --2- 172.21.15.83:0/2862487538 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x560149a536a0 0x560149a55ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:44.190 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.189+0000 7f967e000640 10 client.37721 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:05:44.190 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.189+0000 7f968fe00640 1 --2- 172.21.15.83:0/2862487538 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x560149a536a0 0x560149a55ab0 crc :-1 s=READY pgs=361 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7543721f68e6dc22 server_cookie=d850225708bfe085 in_seq=0 out_seq=0 2024-04-24T09:05:44.191 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.191+0000 7f967e000640 1 -- 172.21.15.83:0/2862487538 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5601499a3320 con 0x560149a536a0 2024-04-24T09:05:44.191 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.191+0000 7f967e000640 10 client.37721 handle_command_reply: tid=0 2024-04-24T09:05:44.191 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.191+0000 7f969a036740 1 client.37721 shutdown 2024-04-24T09:05:44.192 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.192+0000 7f969a036740 1 -- 172.21.15.83:0/2862487538 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560149a3a5a0 msgr2=0x560149a37d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:44.192 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.192+0000 7f969a036740 1 --2- 172.21.15.83:0/2862487538 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560149a3a5a0 0x560149a37d50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9678005fd0 tx=0x7f967809bcb0 comp rx=0 tx=0).stop 2024-04-24T09:05:44.192 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.192+0000 7f969a036740 1 -- 172.21.15.83:0/2862487538 shutdown_connections 2024-04-24T09:05:44.192 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.192+0000 7f969a036740 1 --2- 172.21.15.83:0/2862487538 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x560149a536a0 0x560149a55ab0 crc :-1 s=READY pgs=361 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:44.192 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.192+0000 7f969a036740 1 --2- 172.21.15.83:0/2862487538 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560149a383f0 0x560149a4ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:44.192 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.192+0000 7f969a036740 1 --2- 172.21.15.83:0/2862487538 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560149a3a5a0 0x560149a37d50 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:44.192 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.192+0000 7f967e000640 0 client.37721 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:05:44.193 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.192+0000 7f969a036740 1 --2- 172.21.15.83:0/2862487538 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560149a2f3c0 0x560149a37810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:44.193 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.192+0000 7f969a036740 1 -- 172.21.15.83:0/2862487538 >> 172.21.15.83:0/2862487538 conn(0x560149a2b8d0 msgr2=0x560149a31600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:44.193 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.192+0000 7f969a036740 1 -- 172.21.15.83:0/2862487538 shutdown_connections 2024-04-24T09:05:44.193 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.192+0000 7f969a036740 1 -- 172.21.15.83:0/2862487538 wait complete. 2024-04-24T09:05:44.193 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.192+0000 7f969a036740 20 client.37721 trim_cache size 0 max 16384 2024-04-24T09:05:44.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:05:43 smithi138 ceph-mon[31441]: pgmap v442: 129 pgs: 129 active+clean; 283 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 992 KiB/s rd, 19 MiB/s wr, 316 op/s 2024-04-24T09:05:44.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:05:43 smithi138 ceph-mon[31441]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:05:44.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:05:43 smithi189 ceph-mon[31791]: pgmap v442: 129 pgs: 129 active+clean; 283 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 992 KiB/s rd, 19 MiB/s wr, 316 op/s 2024-04-24T09:05:44.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:05:43 smithi189 ceph-mon[31791]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:05:44.218 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.218+0000 7f969a036740 1 Processor -- start 2024-04-24T09:05:44.218 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.218+0000 7f969a036740 1 -- start start 2024-04-24T09:05:44.218 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.218+0000 7f969a036740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560149a2f3c0 0x560149a2b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:44.219 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.218+0000 7f969a036740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560149a383f0 0x560149a4e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:44.219 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.218+0000 7f969a036740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560149a3a5a0 0x560149a5b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:44.219 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.218+0000 7f969a036740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f96901ad2d0 con 0x560149a383f0 2024-04-24T09:05:44.219 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.218+0000 7f969a036740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f96901aef90 con 0x560149a3a5a0 2024-04-24T09:05:44.219 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.218+0000 7f969a036740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f96901ad9a0 con 0x560149a2f3c0 2024-04-24T09:05:44.219 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.219+0000 7f9687e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560149a383f0 0x560149a4e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:44.219 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.219+0000 7f9695200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560149a3a5a0 0x560149a5b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:44.219 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.219+0000 7f9695200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560149a3a5a0 0x560149a5b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:60548/0 (socket says 172.21.15.83:60548) 2024-04-24T09:05:44.219 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.219+0000 7f9687e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560149a383f0 0x560149a4e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49914/0 (socket says 172.21.15.83:49914) 2024-04-24T09:05:44.219 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.219+0000 7f9695200640 1 -- 172.21.15.83:0/3975599782 learned_addr learned my addr 172.21.15.83:0/3975599782 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:05:44.219 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.219+0000 7f968fe00640 1 --2- 172.21.15.83:0/3975599782 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560149a2f3c0 0x560149a2b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:44.220 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.219+0000 7f9695200640 1 -- 172.21.15.83:0/3975599782 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560149a2f3c0 msgr2=0x560149a2b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:44.220 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.219+0000 7f9695200640 1 --2- 172.21.15.83:0/3975599782 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560149a2f3c0 0x560149a2b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:44.220 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.219+0000 7f9695200640 1 -- 172.21.15.83:0/3975599782 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560149a383f0 msgr2=0x560149a4e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:44.220 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.219+0000 7f9695200640 1 --2- 172.21.15.83:0/3975599782 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560149a383f0 0x560149a4e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:44.220 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.219+0000 7f9695200640 1 -- 172.21.15.83:0/3975599782 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560149774620 con 0x560149a3a5a0 2024-04-24T09:05:44.220 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.219+0000 7f9687e00640 1 --2- 172.21.15.83:0/3975599782 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560149a383f0 0x560149a4e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:05:44.220 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.220+0000 7f9695200640 1 --2- 172.21.15.83:0/3975599782 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560149a3a5a0 0x560149a5b380 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9688003c90 tx=0x7f9688011f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4492cadaf03f8b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:44.221 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.220+0000 7f9684c00640 1 -- 172.21.15.83:0/3975599782 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9688003de0 con 0x560149a3a5a0 2024-04-24T09:05:44.221 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.220+0000 7f9684c00640 1 -- 172.21.15.83:0/3975599782 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f968800f560 con 0x560149a3a5a0 2024-04-24T09:05:44.221 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.220+0000 7f9684c00640 1 -- 172.21.15.83:0/3975599782 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f968801c410 con 0x560149a3a5a0 2024-04-24T09:05:44.221 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.221+0000 7f969a036740 1 -- 172.21.15.83:0/3975599782 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560149a3a5a0 msgr2=0x560149a5b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:44.222 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.221+0000 7f969a036740 1 --2- 172.21.15.83:0/3975599782 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560149a3a5a0 0x560149a5b380 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9688003c90 tx=0x7f9688011f90 comp rx=0 tx=0).stop 2024-04-24T09:05:44.222 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.221+0000 7f969a036740 1 -- 172.21.15.83:0/3975599782 shutdown_connections 2024-04-24T09:05:44.222 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.221+0000 7f969a036740 1 --2- 172.21.15.83:0/3975599782 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560149a3a5a0 0x560149a5b380 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:44.222 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.221+0000 7f969a036740 1 --2- 172.21.15.83:0/3975599782 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560149a383f0 0x560149a4e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:44.222 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.221+0000 7f969a036740 1 --2- 172.21.15.83:0/3975599782 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560149a2f3c0 0x560149a2b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:44.222 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.221+0000 7f969a036740 1 -- 172.21.15.83:0/3975599782 >> 172.21.15.83:0/3975599782 conn(0x560149a2b8d0 msgr2=0x560149a398d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:44.222 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.221+0000 7f969a036740 1 -- 172.21.15.83:0/3975599782 shutdown_connections 2024-04-24T09:05:44.222 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.222+0000 7f969a036740 1 -- 172.21.15.83:0/3975599782 wait complete. 2024-04-24T09:05:44.222 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.222+0000 7f969a036740 1 -- start start 2024-04-24T09:05:44.223 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.222+0000 7f969a036740 1 Processor -- start 2024-04-24T09:05:44.223 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.223+0000 7f969a036740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560149a2f3c0 0x560149a4adc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:44.223 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.223+0000 7f969a036740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560149a383f0 0x560149a4b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:44.223 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.223+0000 7f969a036740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560149a3a5a0 0x560149a33eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:44.224 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.223+0000 7f969a036740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f96901ae800 con 0x560149a2f3c0 2024-04-24T09:05:44.224 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.223+0000 7f969a036740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f96901ae0d0 con 0x560149a383f0 2024-04-24T09:05:44.224 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.223+0000 7f969a036740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f96901ab2f0 con 0x560149a3a5a0 2024-04-24T09:05:44.224 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.223+0000 7f969a036740 10 client.0 _ll_register_callbacks cb 0x7f9678010be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:05:44.224 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.223+0000 7f968fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560149a2f3c0 0x560149a4adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:44.224 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.223+0000 7f968fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560149a2f3c0 0x560149a4adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49918/0 (socket says 172.21.15.83:49918) 2024-04-24T09:05:44.224 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.223+0000 7f968fe00640 1 -- 172.21.15.83:0/863394326 learned_addr learned my addr 172.21.15.83:0/863394326 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:05:44.224 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.223+0000 7f9687e00640 1 --2- 172.21.15.83:0/863394326 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560149a383f0 0x560149a4b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:44.224 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.224+0000 7f9695200640 1 --2- 172.21.15.83:0/863394326 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560149a3a5a0 0x560149a33eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:44.225 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.224+0000 7f968fe00640 1 -- 172.21.15.83:0/863394326 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560149a3a5a0 msgr2=0x560149a33eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:44.225 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.224+0000 7f968fe00640 1 --2- 172.21.15.83:0/863394326 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560149a3a5a0 0x560149a33eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:44.225 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.224+0000 7f968fe00640 1 -- 172.21.15.83:0/863394326 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560149a383f0 msgr2=0x560149a4b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:44.225 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.224+0000 7f968fe00640 1 --2- 172.21.15.83:0/863394326 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560149a383f0 0x560149a4b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:44.225 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.224+0000 7f968fe00640 1 -- 172.21.15.83:0/863394326 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560149a346c0 con 0x560149a2f3c0 2024-04-24T09:05:44.225 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.224+0000 7f9695200640 1 --2- 172.21.15.83:0/863394326 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560149a3a5a0 0x560149a33eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:05:44.225 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.225+0000 7f968fe00640 1 --2- 172.21.15.83:0/863394326 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560149a2f3c0 0x560149a4adc0 secure :-1 s=READY pgs=2131 cs=0 l=1 rev1=1 crypto rx=0x7f9680009900 tx=0x7f9680000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a5b271aa35e369f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:44.225 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.225+0000 7f967e000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:05:44.225 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.225+0000 7f967e000640 1 -- 172.21.15.83:0/863394326 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9680035040 con 0x560149a2f3c0 2024-04-24T09:05:44.226 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.225+0000 7f969a036740 1 -- 172.21.15.83:0/863394326 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f968800f560 con 0x560149a2f3c0 2024-04-24T09:05:44.227 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.225+0000 7f967e000640 1 -- 172.21.15.83:0/863394326 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7f96800522f0 con 0x560149a2f3c0 2024-04-24T09:05:44.227 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.226+0000 7f967e000640 1 -- 172.21.15.83:0/863394326 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f9680061050 con 0x560149a2f3c0 2024-04-24T09:05:44.227 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.226+0000 7f969a036740 10 client.20580 fetch_fsmap learned FSMap version 47 2024-04-24T09:05:44.227 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.226+0000 7f969a036740 1 -- 172.21.15.83:0/863394326 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x560149ab96a0 con 0x560149a2f3c0 2024-04-24T09:05:44.227 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.227+0000 7f967e000640 1 -- 172.21.15.83:0/863394326 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f968800f560 con 0x560149a2f3c0 2024-04-24T09:05:44.227 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.227+0000 7f967e000640 1 -- 172.21.15.83:0/863394326 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 47) ==== 8398+0+0 (secure 0 0 0) 0x7f96800039d0 con 0x560149a2f3c0 2024-04-24T09:05:44.227 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.227+0000 7f969a036740 10 client.20580 fetch_fsmap finished waiting for FSMap version 47 2024-04-24T09:05:44.227 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.227+0000 7f969a036740 10 client.20580 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:05:44.227 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.227+0000 7f969a036740 20 client.20580 populate_metadata read hostname 'smithi083' 2024-04-24T09:05:44.227 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.227+0000 7f969a036740 1 --2- 172.21.15.83:0/863394326 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x560149ab99e0 0x560149abbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:44.227 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.227+0000 7f969a036740 4 client.20580 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T09:05:44.227 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.227+0000 7f969a036740 1 -- 172.21.15.83:0/863394326 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56014999d190 con 0x560149ab99e0 2024-04-24T09:05:44.228 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.227+0000 7f9687e00640 1 --2- 172.21.15.83:0/863394326 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x560149ab99e0 0x560149abbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:44.228 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.228+0000 7f9687e00640 1 --2- 172.21.15.83:0/863394326 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x560149ab99e0 0x560149abbdd0 crc :-1 s=READY pgs=362 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1775fd3928aff8d7 server_cookie=a63970687122af67 in_seq=0 out_seq=0 2024-04-24T09:05:44.228 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.228+0000 7f967e000640 10 client.20580 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:05:44.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:05:43 smithi083 ceph-mon[27907]: pgmap v442: 129 pgs: 129 active+clean; 283 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 992 KiB/s rd, 19 MiB/s wr, 316 op/s 2024-04-24T09:05:44.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:05:43 smithi083 ceph-mon[27907]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:05:44.432 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.432+0000 7f967e000640 1 -- 172.21.15.83:0/863394326 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56014999d190 con 0x560149ab99e0 2024-04-24T09:05:44.432 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.432+0000 7f967e000640 10 client.20580 handle_command_reply: tid=0 2024-04-24T09:05:44.432 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.433+0000 7f969a036740 1 client.20580 shutdown 2024-04-24T09:05:44.432 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.433+0000 7f969a036740 1 -- 172.21.15.83:0/863394326 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560149a2f3c0 msgr2=0x560149a4adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:44.433 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.433+0000 7f969a036740 1 --2- 172.21.15.83:0/863394326 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560149a2f3c0 0x560149a4adc0 secure :-1 s=READY pgs=2131 cs=0 l=1 rev1=1 crypto rx=0x7f9680009900 tx=0x7f9680000bc0 comp rx=0 tx=0).stop 2024-04-24T09:05:44.433 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.433+0000 7f969a036740 1 -- 172.21.15.83:0/863394326 shutdown_connections 2024-04-24T09:05:44.433 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.433+0000 7f969a036740 1 --2- 172.21.15.83:0/863394326 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x560149ab99e0 0x560149abbdd0 crc :-1 s=READY pgs=362 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:44.433 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.433+0000 7f969a036740 1 --2- 172.21.15.83:0/863394326 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560149a3a5a0 0x560149a33eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:44.433 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.433+0000 7f967e000640 0 client.20580 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:05:44.433 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.433+0000 7f969a036740 1 --2- 172.21.15.83:0/863394326 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560149a383f0 0x560149a4b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:44.433 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.433+0000 7f969a036740 1 --2- 172.21.15.83:0/863394326 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560149a2f3c0 0x560149a4adc0 unknown :-1 s=CLOSED pgs=2131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:44.433 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.433+0000 7f969a036740 1 -- 172.21.15.83:0/863394326 >> 172.21.15.83:0/863394326 conn(0x560149a2b8d0 msgr2=0x560149a30d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:44.433 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.433+0000 7f969a036740 1 -- 172.21.15.83:0/863394326 shutdown_connections 2024-04-24T09:05:44.433 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.433+0000 7f969a036740 1 -- 172.21.15.83:0/863394326 wait complete. 2024-04-24T09:05:44.434 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.434+0000 7f969a036740 20 client.20580 trim_cache size 0 max 16384 2024-04-24T09:05:44.434 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T09:05:44.434 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T09:05:44.434 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T09:05:44.434 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T09:05:44.435 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.436+0000 7f9695c00640 1 -- 172.21.15.83:0/2192272101 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f96740a3e10 msgr2=0x7f96740a62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:44.435 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.436+0000 7f9695c00640 1 --2- 172.21.15.83:0/2192272101 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f96740a3e10 0x7f96740a62d0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f9688011630 tx=0x7f9688009500 comp rx=0 tx=0).stop 2024-04-24T09:05:44.435 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.436+0000 7f9695c00640 1 -- 172.21.15.83:0/2192272101 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96901038e0 msgr2=0x7f96901a0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:44.435 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.436+0000 7f9695c00640 1 --2- 172.21.15.83:0/2192272101 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96901038e0 0x7f96901a0b00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f967800d740 tx=0x7f967800dc00 comp rx=0 tx=0).stop 2024-04-24T09:05:44.436 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.436+0000 7f9695c00640 1 -- 172.21.15.83:0/2192272101 shutdown_connections 2024-04-24T09:05:44.436 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.436+0000 7f9695c00640 1 --2- 172.21.15.83:0/2192272101 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f96740a3e10 0x7f96740a62d0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:44.436 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.436+0000 7f9695c00640 1 --2- 172.21.15.83:0/2192272101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f969010c3e0 0x7f969019a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:44.436 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.436+0000 7f9695c00640 1 --2- 172.21.15.83:0/2192272101 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96901038e0 0x7f96901a0b00 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:44.436 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.436+0000 7f9695c00640 1 --2- 172.21.15.83:0/2192272101 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9690102fc0 0x7f96901a05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:44.436 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.436+0000 7f9695c00640 1 -- 172.21.15.83:0/2192272101 >> 172.21.15.83:0/2192272101 conn(0x7f96900fc820 msgr2=0x7f9690101bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:44.436 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.436+0000 7f9695c00640 1 -- 172.21.15.83:0/2192272101 shutdown_connections 2024-04-24T09:05:44.436 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:44.436+0000 7f9695c00640 1 -- 172.21.15.83:0/2192272101 wait complete. 2024-04-24T09:05:45.153 INFO:tasks.workunit.client.0.smithi083.stdout: 10 1930 16.53 MB/sec warmup 16 sec latency 146.008 ms 2024-04-24T09:05:46.153 INFO:tasks.workunit.client.0.smithi083.stdout: 10 2078 15.76 MB/sec warmup 17 sec latency 76.075 ms 2024-04-24T09:05:46.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:05:45 smithi138 ceph-mon[31441]: pgmap v443: 129 pgs: 129 active+clean; 283 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 735 KiB/s rd, 2.7 MiB/s wr, 138 op/s 2024-04-24T09:05:46.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:05:45 smithi189 ceph-mon[31791]: pgmap v443: 129 pgs: 129 active+clean; 283 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 735 KiB/s rd, 2.7 MiB/s wr, 138 op/s 2024-04-24T09:05:46.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:05:45 smithi083 ceph-mon[27907]: pgmap v443: 129 pgs: 129 active+clean; 283 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 735 KiB/s rd, 2.7 MiB/s wr, 138 op/s 2024-04-24T09:05:46.738 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T09:05:46.739 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T09:05:46.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:46.992+0000 7f29da800640 1 -- 172.21.15.83:0/4244582345 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29d410c880 msgr2=0x7f29d410cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:46.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:46.992+0000 7f29da800640 1 --2- 172.21.15.83:0/4244582345 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29d410c880 0x7f29d410cd00 secure :-1 s=READY pgs=2132 cs=0 l=1 rev1=1 crypto rx=0x7f29c0009930 tx=0x7f29c002a420 comp rx=0 tx=0).stop 2024-04-24T09:05:46.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:46.992+0000 7f29da800640 1 -- 172.21.15.83:0/4244582345 shutdown_connections 2024-04-24T09:05:46.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:46.992+0000 7f29da800640 1 --2- 172.21.15.83:0/4244582345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f29d410d240 0x7f29d4119750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:46.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:46.992+0000 7f29da800640 1 --2- 172.21.15.83:0/4244582345 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29d410c880 0x7f29d410cd00 unknown :-1 s=CLOSED pgs=2132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:46.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:46.992+0000 7f29da800640 1 --2- 172.21.15.83:0/4244582345 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f29d4112890 0x7f29d4112c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:46.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:46.992+0000 7f29da800640 1 -- 172.21.15.83:0/4244582345 >> 172.21.15.83:0/4244582345 conn(0x7f29d406ce90 msgr2=0x7f29d410b010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:46.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:46.992+0000 7f29da800640 1 -- 172.21.15.83:0/4244582345 shutdown_connections 2024-04-24T09:05:46.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:46.992+0000 7f29da800640 1 -- 172.21.15.83:0/4244582345 wait complete. 2024-04-24T09:05:46.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:46.993+0000 7f29da800640 1 Processor -- start 2024-04-24T09:05:46.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:46.993+0000 7f29da800640 1 -- start start 2024-04-24T09:05:46.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:46.993+0000 7f29da800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29d410c880 0x7f29d41a3100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:46.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:46.993+0000 7f29da800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f29d410d240 0x7f29d41a3640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:46.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:46.994+0000 7f29da800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f29d4112890 0x7f29d41a7970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:46.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:46.994+0000 7f29da800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f29d411cba0 con 0x7f29d410c880 2024-04-24T09:05:46.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:46.994+0000 7f29da800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f29d411ca20 con 0x7f29d4112890 2024-04-24T09:05:46.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:46.994+0000 7f29da800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f29d411cd20 con 0x7f29d410d240 2024-04-24T09:05:46.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:46.994+0000 7f29d9400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29d410c880 0x7f29d41a3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:46.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:46.994+0000 7f29d9400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29d410c880 0x7f29d41a3100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49948/0 (socket says 172.21.15.83:49948) 2024-04-24T09:05:46.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:46.994+0000 7f29d9400640 1 -- 172.21.15.83:0/1414321325 learned_addr learned my addr 172.21.15.83:0/1414321325 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:05:46.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:46.994+0000 7f29d8a00640 1 --2- 172.21.15.83:0/1414321325 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f29d410d240 0x7f29d41a3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:46.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:46.994+0000 7f29d9e00640 1 --2- 172.21.15.83:0/1414321325 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f29d4112890 0x7f29d41a7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:46.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:46.995+0000 7f29d9400640 1 -- 172.21.15.83:0/1414321325 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f29d410d240 msgr2=0x7f29d41a3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:46.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:46.995+0000 7f29d9400640 1 --2- 172.21.15.83:0/1414321325 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f29d410d240 0x7f29d41a3640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:46.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:46.995+0000 7f29d9400640 1 -- 172.21.15.83:0/1414321325 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f29d4112890 msgr2=0x7f29d41a7970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:46.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:46.995+0000 7f29d9400640 1 --2- 172.21.15.83:0/1414321325 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f29d4112890 0x7f29d41a7970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:46.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:46.995+0000 7f29d9400640 1 -- 172.21.15.83:0/1414321325 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29d41a80a0 con 0x7f29d410c880 2024-04-24T09:05:46.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:46.995+0000 7f29d8a00640 1 --2- 172.21.15.83:0/1414321325 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f29d410d240 0x7f29d41a3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:05:46.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:46.995+0000 7f29d9400640 1 --2- 172.21.15.83:0/1414321325 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29d410c880 0x7f29d41a3100 secure :-1 s=READY pgs=2133 cs=0 l=1 rev1=1 crypto rx=0x7f29c800d770 tx=0x7f29c800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:46.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:46.996+0000 7f29d2000640 1 -- 172.21.15.83:0/1414321325 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29c8017070 con 0x7f29d410c880 2024-04-24T09:05:46.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:46.996+0000 7f29d2000640 1 -- 172.21.15.83:0/1414321325 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f29c80054d0 con 0x7f29d410c880 2024-04-24T09:05:46.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:46.996+0000 7f29d2000640 1 -- 172.21.15.83:0/1414321325 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29c8004cf0 con 0x7f29d410c880 2024-04-24T09:05:46.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:46.996+0000 7f29da800640 1 -- 172.21.15.83:0/1414321325 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29d41a8390 con 0x7f29d410c880 2024-04-24T09:05:46.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:46.996+0000 7f29da800640 1 -- 172.21.15.83:0/1414321325 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29d41afc20 con 0x7f29d410c880 2024-04-24T09:05:46.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.000+0000 7f29ded68740 1 Processor -- start 2024-04-24T09:05:47.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.000+0000 7f29ded68740 1 -- start start 2024-04-24T09:05:47.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.000+0000 7f29d2000640 1 -- 172.21.15.83:0/1414321325 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f29c8004e90 con 0x7f29d410c880 2024-04-24T09:05:47.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.001+0000 7f29ded68740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e93d100240 0x55e93d100620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:47.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.001+0000 7f29ded68740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e93d100b60 0x55e93d10acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:47.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.001+0000 7f29ded68740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e93d10b230 0x55e93d10d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:47.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.001+0000 7f29ded68740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e93d05d1f0 con 0x55e93d100240 2024-04-24T09:05:47.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.001+0000 7f29ded68740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e93d003350 con 0x55e93d10b230 2024-04-24T09:05:47.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.001+0000 7f29ded68740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55e93d0591b0 con 0x55e93d100b60 2024-04-24T09:05:47.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.001+0000 7f29d9e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e93d100240 0x55e93d100620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:47.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.002+0000 7f29d8a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e93d10b230 0x55e93d10d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:47.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.002+0000 7f29d9e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e93d100240 0x55e93d100620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49956/0 (socket says 172.21.15.83:49956) 2024-04-24T09:05:47.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.002+0000 7f29d9e00640 1 -- 172.21.15.83:0/1077257505 learned_addr learned my addr 172.21.15.83:0/1077257505 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:05:47.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.002+0000 7f29d9400640 1 --2- 172.21.15.83:0/1077257505 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e93d100b60 0x55e93d10acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:47.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.002+0000 7f29d9e00640 1 -- 172.21.15.83:0/1077257505 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e93d100b60 msgr2=0x55e93d10acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:47.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.002+0000 7f29d9e00640 1 --2- 172.21.15.83:0/1077257505 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e93d100b60 0x55e93d10acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:47.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.002+0000 7f29d9e00640 1 -- 172.21.15.83:0/1077257505 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e93d10b230 msgr2=0x55e93d10d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:47.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.002+0000 7f29d9e00640 1 --2- 172.21.15.83:0/1077257505 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e93d10b230 0x55e93d10d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:47.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.002+0000 7f29d9e00640 1 -- 172.21.15.83:0/1077257505 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e93d057f50 con 0x55e93d100240 2024-04-24T09:05:47.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.002+0000 7f29d2000640 1 --2- 172.21.15.83:0/1414321325 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f29bc0a3de0 0x7f29bc0a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:47.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.003+0000 7f29d2000640 1 -- 172.21.15.83:0/1414321325 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7f29c80d0580 con 0x7f29d410c880 2024-04-24T09:05:47.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.003+0000 7f29d9e00640 1 --2- 172.21.15.83:0/1077257505 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e93d100240 0x55e93d100620 secure :-1 s=READY pgs=2134 cs=0 l=1 rev1=1 crypto rx=0x7f29cc009fd0 tx=0x7f29cc0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9180cef9f928230f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:47.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.003+0000 7f29c6600640 1 -- 172.21.15.83:0/1077257505 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29cc004d20 con 0x55e93d100240 2024-04-24T09:05:47.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.003+0000 7f29c6600640 1 -- 172.21.15.83:0/1077257505 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f29cc004050 con 0x55e93d100240 2024-04-24T09:05:47.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.003+0000 7f29c6600640 1 -- 172.21.15.83:0/1077257505 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29cc005280 con 0x55e93d100240 2024-04-24T09:05:47.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.004+0000 7f29d9400640 1 --2- 172.21.15.83:0/1414321325 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f29bc0a3de0 0x7f29bc0a62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:47.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.005+0000 7f29ded68740 1 -- 172.21.15.83:0/1077257505 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e93d100240 msgr2=0x55e93d100620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:47.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.005+0000 7f29ded68740 1 --2- 172.21.15.83:0/1077257505 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e93d100240 0x55e93d100620 secure :-1 s=READY pgs=2134 cs=0 l=1 rev1=1 crypto rx=0x7f29cc009fd0 tx=0x7f29cc0118d0 comp rx=0 tx=0).stop 2024-04-24T09:05:47.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.005+0000 7f29d9400640 1 --2- 172.21.15.83:0/1414321325 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f29bc0a3de0 0x7f29bc0a62a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f29c80cd960 tx=0x7f29c80d3e00 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:47.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.005+0000 7f29ded68740 1 -- 172.21.15.83:0/1077257505 shutdown_connections 2024-04-24T09:05:47.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.005+0000 7f29ded68740 1 --2- 172.21.15.83:0/1077257505 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e93d10b230 0x55e93d10d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:47.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.005+0000 7f29ded68740 1 --2- 172.21.15.83:0/1077257505 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e93d100b60 0x55e93d10acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:47.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.005+0000 7f29ded68740 1 --2- 172.21.15.83:0/1077257505 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e93d100240 0x55e93d100620 unknown :-1 s=CLOSED pgs=2134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:47.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.005+0000 7f29ded68740 1 -- 172.21.15.83:0/1077257505 >> 172.21.15.83:0/1077257505 conn(0x55e93d0fc560 msgr2=0x55e93d0fc940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:47.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.006+0000 7f29ded68740 1 -- 172.21.15.83:0/1077257505 shutdown_connections 2024-04-24T09:05:47.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.006+0000 7f29ded68740 1 -- 172.21.15.83:0/1077257505 wait complete. 2024-04-24T09:05:47.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.007+0000 7f29ded68740 1 -- start start 2024-04-24T09:05:47.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.007+0000 7f29ded68740 1 Processor -- start 2024-04-24T09:05:47.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.008+0000 7f29ded68740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e93d100240 0x55e93d1081e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:47.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.008+0000 7f29ded68740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e93d100b60 0x55e93d108720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:47.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.008+0000 7f29ded68740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e93d10b230 0x55e93d108c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:47.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.008+0000 7f29ded68740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e93d10e020 con 0x55e93d100b60 2024-04-24T09:05:47.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.008+0000 7f29ded68740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e93d016690 con 0x55e93d100240 2024-04-24T09:05:47.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.008+0000 7f29ded68740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55e93d10e1a0 con 0x55e93d10b230 2024-04-24T09:05:47.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.008+0000 7f29ded68740 10 client.0 _ll_register_callbacks cb 0x55e93d052650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:05:47.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.008+0000 7f29d8a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e93d100b60 0x55e93d108720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:47.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.008+0000 7f29d8a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e93d100b60 0x55e93d108720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49966/0 (socket says 172.21.15.83:49966) 2024-04-24T09:05:47.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.008+0000 7f29d8a00640 1 -- 172.21.15.83:0/3476584233 learned_addr learned my addr 172.21.15.83:0/3476584233 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:05:47.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.008+0000 7f29d9e00640 1 --2- 172.21.15.83:0/3476584233 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e93d100240 0x55e93d1081e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:47.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.009+0000 7f29d8a00640 1 -- 172.21.15.83:0/3476584233 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e93d10b230 msgr2=0x55e93d108c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:47.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.009+0000 7f29d8a00640 1 --2- 172.21.15.83:0/3476584233 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e93d10b230 0x55e93d108c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:47.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.009+0000 7f29d8a00640 1 -- 172.21.15.83:0/3476584233 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e93d100240 msgr2=0x55e93d1081e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:47.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.009+0000 7f29d8a00640 1 --2- 172.21.15.83:0/3476584233 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e93d100240 0x55e93d1081e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:47.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.009+0000 7f29d8a00640 1 -- 172.21.15.83:0/3476584233 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e93d015920 con 0x55e93d100b60 2024-04-24T09:05:47.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.009+0000 7f29d8a00640 1 --2- 172.21.15.83:0/3476584233 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e93d100b60 0x55e93d108720 secure :-1 s=READY pgs=2135 cs=0 l=1 rev1=1 crypto rx=0x7f29c0005ab0 tx=0x7f29c0005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb35a62db18eb295 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:47.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.009+0000 7f29baa00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:05:47.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.010+0000 7f29baa00640 1 -- 172.21.15.83:0/3476584233 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29c0035040 con 0x55e93d100b60 2024-04-24T09:05:47.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.010+0000 7f29baa00640 1 -- 172.21.15.83:0/3476584233 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7f29c00048c0 con 0x55e93d100b60 2024-04-24T09:05:47.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.010+0000 7f29ded68740 1 -- 172.21.15.83:0/3476584233 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29cc004050 con 0x55e93d100b60 2024-04-24T09:05:47.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.010+0000 7f29baa00640 1 -- 172.21.15.83:0/3476584233 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f29c0004ac0 con 0x55e93d100b60 2024-04-24T09:05:47.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.010+0000 7f29ded68740 10 client.20592 fetch_fsmap learned FSMap version 47 2024-04-24T09:05:47.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.011+0000 7f29ded68740 1 -- 172.21.15.83:0/3476584233 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55e93ce397d0 con 0x55e93d100b60 2024-04-24T09:05:47.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.011+0000 7f29baa00640 1 -- 172.21.15.83:0/3476584233 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f29cc004050 con 0x55e93d100b60 2024-04-24T09:05:47.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.012+0000 7f29baa00640 1 -- 172.21.15.83:0/3476584233 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 47) ==== 8398+0+0 (secure 0 0 0) 0x7f29c00035f0 con 0x55e93d100b60 2024-04-24T09:05:47.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.012+0000 7f29ded68740 10 client.20592 fetch_fsmap finished waiting for FSMap version 47 2024-04-24T09:05:47.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.013+0000 7f29ded68740 10 client.20592 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:05:47.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.013+0000 7f29ded68740 20 client.20592 populate_metadata read hostname 'smithi083' 2024-04-24T09:05:47.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.013+0000 7f29ded68740 1 --2- 172.21.15.83:0/3476584233 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55e93d11b600 0x55e93d11da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:47.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.013+0000 7f29ded68740 4 client.20592 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:05:47.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.013+0000 7f29ded68740 1 -- 172.21.15.83:0/3476584233 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e93d06ac10 con 0x55e93d11b600 2024-04-24T09:05:47.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.014+0000 7f29d9e00640 1 --2- 172.21.15.83:0/3476584233 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55e93d11b600 0x55e93d11da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:47.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.015+0000 7f29d9e00640 1 --2- 172.21.15.83:0/3476584233 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55e93d11b600 0x55e93d11da30 crc :-1 s=READY pgs=363 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3fe06a9b4d6d2845 server_cookie=49624cde03c1240b in_seq=0 out_seq=0 2024-04-24T09:05:47.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.015+0000 7f29baa00640 10 client.20592 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:05:47.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.016+0000 7f29baa00640 1 -- 172.21.15.83:0/3476584233 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e93d06ac10 con 0x55e93d11b600 2024-04-24T09:05:47.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.016+0000 7f29baa00640 10 client.20592 handle_command_reply: tid=0 2024-04-24T09:05:47.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.017+0000 7f29ded68740 1 client.20592 shutdown 2024-04-24T09:05:47.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.017+0000 7f29ded68740 1 -- 172.21.15.83:0/3476584233 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e93d100b60 msgr2=0x55e93d108720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:47.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.017+0000 7f29ded68740 1 --2- 172.21.15.83:0/3476584233 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e93d100b60 0x55e93d108720 secure :-1 s=READY pgs=2135 cs=0 l=1 rev1=1 crypto rx=0x7f29c0005ab0 tx=0x7f29c0005270 comp rx=0 tx=0).stop 2024-04-24T09:05:47.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.017+0000 7f29ded68740 1 -- 172.21.15.83:0/3476584233 shutdown_connections 2024-04-24T09:05:47.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.017+0000 7f29ded68740 1 --2- 172.21.15.83:0/3476584233 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55e93d11b600 0x55e93d11da30 crc :-1 s=READY pgs=363 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:47.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.017+0000 7f29ded68740 1 --2- 172.21.15.83:0/3476584233 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e93d10b230 0x55e93d108c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:47.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.017+0000 7f29ded68740 1 --2- 172.21.15.83:0/3476584233 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e93d100b60 0x55e93d108720 unknown :-1 s=CLOSED pgs=2135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:47.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.017+0000 7f29ded68740 1 --2- 172.21.15.83:0/3476584233 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e93d100240 0x55e93d1081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:47.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.017+0000 7f29ded68740 1 -- 172.21.15.83:0/3476584233 >> 172.21.15.83:0/3476584233 conn(0x55e93d0fc560 msgr2=0x55e93d0f2fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:47.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.017+0000 7f29baa00640 0 client.20592 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:05:47.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.017+0000 7f29ded68740 1 -- 172.21.15.83:0/3476584233 shutdown_connections 2024-04-24T09:05:47.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.017+0000 7f29ded68740 1 -- 172.21.15.83:0/3476584233 wait complete. 2024-04-24T09:05:47.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.018+0000 7f29ded68740 20 client.20592 trim_cache size 0 max 16384 2024-04-24T09:05:47.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.046+0000 7f29ded68740 1 Processor -- start 2024-04-24T09:05:47.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.047+0000 7f29ded68740 1 -- start start 2024-04-24T09:05:47.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.047+0000 7f29ded68740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e93d100240 0x55e93d0eee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:47.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.047+0000 7f29ded68740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e93d10b230 0x55e93d116f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:47.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.047+0000 7f29ded68740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e93d1029a0 0x55e93d123340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:47.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.047+0000 7f29ded68740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f29d41b5a30 con 0x55e93d100240 2024-04-24T09:05:47.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.047+0000 7f29ded68740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f29d41b76f0 con 0x55e93d1029a0 2024-04-24T09:05:47.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.047+0000 7f29ded68740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f29d41b6100 con 0x55e93d10b230 2024-04-24T09:05:47.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.047+0000 7f29d9e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e93d100240 0x55e93d0eee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:47.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.047+0000 7f29d9e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e93d100240 0x55e93d0eee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49970/0 (socket says 172.21.15.83:49970) 2024-04-24T09:05:47.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.047+0000 7f29d9e00640 1 -- 172.21.15.83:0/959653763 learned_addr learned my addr 172.21.15.83:0/959653763 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:05:47.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.048+0000 7f29d9e00640 1 --2- 172.21.15.83:0/959653763 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e93d1029a0 0x55e93d123340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:47.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.048+0000 7f29d8a00640 1 --2- 172.21.15.83:0/959653763 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e93d10b230 0x55e93d116f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:47.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.048+0000 7f29d9e00640 1 -- 172.21.15.83:0/959653763 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e93d10b230 msgr2=0x55e93d116f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:47.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.048+0000 7f29d9e00640 1 --2- 172.21.15.83:0/959653763 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e93d10b230 0x55e93d116f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:47.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.048+0000 7f29d9e00640 1 -- 172.21.15.83:0/959653763 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e93d1029a0 msgr2=0x55e93d123340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:47.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.048+0000 7f29d9e00640 1 --2- 172.21.15.83:0/959653763 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e93d1029a0 0x55e93d123340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:47.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.048+0000 7f29d9e00640 1 -- 172.21.15.83:0/959653763 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e93d107190 con 0x55e93d100240 2024-04-24T09:05:47.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.048+0000 7f29d9e00640 1 --2- 172.21.15.83:0/959653763 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e93d100240 0x55e93d0eee60 secure :-1 s=READY pgs=2136 cs=0 l=1 rev1=1 crypto rx=0x7f29cc011dd0 tx=0x7f29cc003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0c4dbdeb56d1bea server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:47.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.049+0000 7f29c5c00640 1 -- 172.21.15.83:0/959653763 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29cc031070 con 0x55e93d100240 2024-04-24T09:05:47.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.049+0000 7f29c5c00640 1 -- 172.21.15.83:0/959653763 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f29cc028030 con 0x55e93d100240 2024-04-24T09:05:47.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.049+0000 7f29c5c00640 1 -- 172.21.15.83:0/959653763 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29cc004380 con 0x55e93d100240 2024-04-24T09:05:47.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.049+0000 7f29ded68740 1 -- 172.21.15.83:0/959653763 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e93d100240 msgr2=0x55e93d0eee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:47.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.049+0000 7f29ded68740 1 --2- 172.21.15.83:0/959653763 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e93d100240 0x55e93d0eee60 secure :-1 s=READY pgs=2136 cs=0 l=1 rev1=1 crypto rx=0x7f29cc011dd0 tx=0x7f29cc003f90 comp rx=0 tx=0).stop 2024-04-24T09:05:47.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.050+0000 7f29ded68740 1 -- 172.21.15.83:0/959653763 shutdown_connections 2024-04-24T09:05:47.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.050+0000 7f29ded68740 1 --2- 172.21.15.83:0/959653763 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e93d1029a0 0x55e93d123340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:47.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.050+0000 7f29ded68740 1 --2- 172.21.15.83:0/959653763 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e93d10b230 0x55e93d116f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:47.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.050+0000 7f29ded68740 1 --2- 172.21.15.83:0/959653763 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e93d100240 0x55e93d0eee60 unknown :-1 s=CLOSED pgs=2136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:47.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.050+0000 7f29ded68740 1 -- 172.21.15.83:0/959653763 >> 172.21.15.83:0/959653763 conn(0x55e93d0fc560 msgr2=0x55e93d0f2fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:47.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.050+0000 7f29ded68740 1 -- 172.21.15.83:0/959653763 shutdown_connections 2024-04-24T09:05:47.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.050+0000 7f29ded68740 1 -- 172.21.15.83:0/959653763 wait complete. 2024-04-24T09:05:47.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.050+0000 7f29ded68740 1 -- start start 2024-04-24T09:05:47.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.051+0000 7f29ded68740 1 Processor -- start 2024-04-24T09:05:47.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.051+0000 7f29ded68740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e93d100240 0x55e93d113420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:47.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.051+0000 7f29ded68740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e93d1029a0 0x55e93d113960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:47.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.051+0000 7f29ded68740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e93d10b230 0x55e93d1103f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:47.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.051+0000 7f29ded68740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f29d41b6f60 con 0x55e93d100240 2024-04-24T09:05:47.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.051+0000 7f29ded68740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f29d41b6830 con 0x55e93d10b230 2024-04-24T09:05:47.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.051+0000 7f29ded68740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f29d41b3a50 con 0x55e93d1029a0 2024-04-24T09:05:47.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.051+0000 7f29ded68740 10 client.0 _ll_register_callbacks cb 0x7f29c0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:05:47.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.051+0000 7f29d9e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e93d100240 0x55e93d113420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:47.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.052+0000 7f29d9e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e93d100240 0x55e93d113420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49976/0 (socket says 172.21.15.83:49976) 2024-04-24T09:05:47.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.052+0000 7f29d9e00640 1 -- 172.21.15.83:0/2644063125 learned_addr learned my addr 172.21.15.83:0/2644063125 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:05:47.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.052+0000 7f29d8a00640 1 --2- 172.21.15.83:0/2644063125 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e93d1029a0 0x55e93d113960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:47.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.052+0000 7f29d9e00640 1 --2- 172.21.15.83:0/2644063125 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e93d10b230 0x55e93d1103f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:47.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.052+0000 7f29d9e00640 1 -- 172.21.15.83:0/2644063125 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e93d1029a0 msgr2=0x55e93d113960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:47.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.052+0000 7f29d9e00640 1 --2- 172.21.15.83:0/2644063125 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e93d1029a0 0x55e93d113960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:47.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.052+0000 7f29d9e00640 1 -- 172.21.15.83:0/2644063125 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e93d10b230 msgr2=0x55e93d1103f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:47.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.052+0000 7f29d9e00640 1 --2- 172.21.15.83:0/2644063125 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e93d10b230 0x55e93d1103f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:47.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.052+0000 7f29d9e00640 1 -- 172.21.15.83:0/2644063125 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e93cd6f3f0 con 0x55e93d100240 2024-04-24T09:05:47.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.053+0000 7f29d9e00640 1 --2- 172.21.15.83:0/2644063125 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e93d100240 0x55e93d113420 secure :-1 s=READY pgs=2137 cs=0 l=1 rev1=1 crypto rx=0x7f29cc009de0 tx=0x7f29cc005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f28f2176db835c57 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:47.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.053+0000 7f29baa00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:05:47.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.053+0000 7f29baa00640 1 -- 172.21.15.83:0/2644063125 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29cc040060 con 0x55e93d100240 2024-04-24T09:05:47.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.053+0000 7f29baa00640 1 -- 172.21.15.83:0/2644063125 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7f29cc004000 con 0x55e93d100240 2024-04-24T09:05:47.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.053+0000 7f29ded68740 1 -- 172.21.15.83:0/2644063125 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29cc028030 con 0x55e93d100240 2024-04-24T09:05:47.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.053+0000 7f29baa00640 1 -- 172.21.15.83:0/2644063125 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f29cc047050 con 0x55e93d100240 2024-04-24T09:05:47.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.053+0000 7f29ded68740 10 client.20598 fetch_fsmap learned FSMap version 47 2024-04-24T09:05:47.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.053+0000 7f29ded68740 1 -- 172.21.15.83:0/2644063125 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55e93cd2f0f0 con 0x55e93d100240 2024-04-24T09:05:47.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.054+0000 7f29baa00640 1 -- 172.21.15.83:0/2644063125 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f29cc028030 con 0x55e93d100240 2024-04-24T09:05:47.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.054+0000 7f29baa00640 1 -- 172.21.15.83:0/2644063125 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 47) ==== 8398+0+0 (secure 0 0 0) 0x7f29cc02b540 con 0x55e93d100240 2024-04-24T09:05:47.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.054+0000 7f29ded68740 10 client.20598 fetch_fsmap finished waiting for FSMap version 47 2024-04-24T09:05:47.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.054+0000 7f29ded68740 10 client.20598 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:05:47.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.054+0000 7f29ded68740 20 client.20598 populate_metadata read hostname 'smithi083' 2024-04-24T09:05:47.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.054+0000 7f29ded68740 1 --2- 172.21.15.83:0/2644063125 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55e93d182770 0x55e93d184b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:47.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.054+0000 7f29ded68740 4 client.20598 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T09:05:47.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.054+0000 7f29ded68740 1 -- 172.21.15.83:0/2644063125 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e93d064a80 con 0x55e93d182770 2024-04-24T09:05:47.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.055+0000 7f29d8a00640 1 --2- 172.21.15.83:0/2644063125 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55e93d182770 0x55e93d184b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:47.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.055+0000 7f29d8a00640 1 --2- 172.21.15.83:0/2644063125 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55e93d182770 0x55e93d184b90 crc :-1 s=READY pgs=364 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c91e76a1b9864df8 server_cookie=eb8cb12395542283 in_seq=0 out_seq=0 2024-04-24T09:05:47.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.055+0000 7f29baa00640 10 client.20598 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:05:47.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.057+0000 7f29baa00640 1 -- 172.21.15.83:0/2644063125 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55e93d064a80 con 0x55e93d182770 2024-04-24T09:05:47.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.057+0000 7f29baa00640 10 client.20598 handle_command_reply: tid=0 2024-04-24T09:05:47.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.057+0000 7f29ded68740 1 client.20598 shutdown 2024-04-24T09:05:47.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.057+0000 7f29ded68740 1 -- 172.21.15.83:0/2644063125 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e93d100240 msgr2=0x55e93d113420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:47.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.057+0000 7f29ded68740 1 --2- 172.21.15.83:0/2644063125 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e93d100240 0x55e93d113420 secure :-1 s=READY pgs=2137 cs=0 l=1 rev1=1 crypto rx=0x7f29cc009de0 tx=0x7f29cc005f10 comp rx=0 tx=0).stop 2024-04-24T09:05:47.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.057+0000 7f29ded68740 1 -- 172.21.15.83:0/2644063125 shutdown_connections 2024-04-24T09:05:47.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.057+0000 7f29ded68740 1 --2- 172.21.15.83:0/2644063125 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55e93d182770 0x55e93d184b90 crc :-1 s=READY pgs=364 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:47.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.057+0000 7f29ded68740 1 --2- 172.21.15.83:0/2644063125 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e93d10b230 0x55e93d1103f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:47.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.057+0000 7f29ded68740 1 --2- 172.21.15.83:0/2644063125 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e93d1029a0 0x55e93d113960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:47.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.057+0000 7f29ded68740 1 --2- 172.21.15.83:0/2644063125 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e93d100240 0x55e93d113420 unknown :-1 s=CLOSED pgs=2137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:47.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.057+0000 7f29baa00640 0 client.20598 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:05:47.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.057+0000 7f29ded68740 1 -- 172.21.15.83:0/2644063125 >> 172.21.15.83:0/2644063125 conn(0x55e93d0fc560 msgr2=0x55e93d108180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:47.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.057+0000 7f29ded68740 1 -- 172.21.15.83:0/2644063125 shutdown_connections 2024-04-24T09:05:47.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.057+0000 7f29ded68740 1 -- 172.21.15.83:0/2644063125 wait complete. 2024-04-24T09:05:47.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.058+0000 7f29ded68740 20 client.20598 trim_cache size 0 max 16384 2024-04-24T09:05:47.058 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T09:05:47.058 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-04-24T09:05:47.058 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-04-24T09:05:47.058 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T09:05:47.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.058+0000 7f29da800640 1 -- 172.21.15.83:0/1414321325 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f29bc0a3de0 msgr2=0x7f29bc0a62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:47.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.058+0000 7f29da800640 1 --2- 172.21.15.83:0/1414321325 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f29bc0a3de0 0x7f29bc0a62a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f29c80cd960 tx=0x7f29c80d3e00 comp rx=0 tx=0).stop 2024-04-24T09:05:47.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.058+0000 7f29da800640 1 -- 172.21.15.83:0/1414321325 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29d410c880 msgr2=0x7f29d41a3100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:47.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.058+0000 7f29da800640 1 --2- 172.21.15.83:0/1414321325 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29d410c880 0x7f29d41a3100 secure :-1 s=READY pgs=2133 cs=0 l=1 rev1=1 crypto rx=0x7f29c800d770 tx=0x7f29c800dc30 comp rx=0 tx=0).stop 2024-04-24T09:05:47.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.059+0000 7f29da800640 1 -- 172.21.15.83:0/1414321325 shutdown_connections 2024-04-24T09:05:47.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.059+0000 7f29da800640 1 --2- 172.21.15.83:0/1414321325 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f29bc0a3de0 0x7f29bc0a62a0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:47.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.059+0000 7f29da800640 1 --2- 172.21.15.83:0/1414321325 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f29d4112890 0x7f29d41a7970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:47.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.059+0000 7f29da800640 1 --2- 172.21.15.83:0/1414321325 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f29d410d240 0x7f29d41a3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:47.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.059+0000 7f29da800640 1 --2- 172.21.15.83:0/1414321325 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29d410c880 0x7f29d41a3100 unknown :-1 s=CLOSED pgs=2133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:47.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.059+0000 7f29da800640 1 -- 172.21.15.83:0/1414321325 >> 172.21.15.83:0/1414321325 conn(0x7f29d406ce90 msgr2=0x7f29d410f7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:47.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.059+0000 7f29da800640 1 -- 172.21.15.83:0/1414321325 shutdown_connections 2024-04-24T09:05:47.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.059+0000 7f29da800640 1 -- 172.21.15.83:0/1414321325 wait complete. 2024-04-24T09:05:47.079 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T09:05:47.079 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T09:05:47.153 INFO:tasks.workunit.client.0.smithi083.stdout: 10 2197 15.05 MB/sec warmup 18 sec latency 83.850 ms 2024-04-24T09:05:47.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.330+0000 7f16b2c00640 1 -- 172.21.15.83:0/965818696 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f16ac104c40 msgr2=0x7f16ac108c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:47.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.330+0000 7f16b2c00640 1 --2- 172.21.15.83:0/965818696 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f16ac104c40 0x7f16ac108c80 secure :-1 s=READY pgs=2138 cs=0 l=1 rev1=1 crypto rx=0x7f1698009900 tx=0x7f169802a3f0 comp rx=0 tx=0).stop 2024-04-24T09:05:47.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.330+0000 7f16b2c00640 1 -- 172.21.15.83:0/965818696 shutdown_connections 2024-04-24T09:05:47.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.330+0000 7f16b2c00640 1 --2- 172.21.15.83:0/965818696 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16ac109360 0x7f16ac110f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:47.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.330+0000 7f16b2c00640 1 --2- 172.21.15.83:0/965818696 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f16ac104c40 0x7f16ac108c80 unknown :-1 s=CLOSED pgs=2138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:47.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.330+0000 7f16b2c00640 1 --2- 172.21.15.83:0/965818696 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f16ac104290 0x7f16ac104670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:47.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.330+0000 7f16b2c00640 1 -- 172.21.15.83:0/965818696 >> 172.21.15.83:0/965818696 conn(0x7f16ac0ffad0 msgr2=0x7f16ac101ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:47.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.331+0000 7f16b2c00640 1 -- 172.21.15.83:0/965818696 shutdown_connections 2024-04-24T09:05:47.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.331+0000 7f16b2c00640 1 -- 172.21.15.83:0/965818696 wait complete. 2024-04-24T09:05:47.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.332+0000 7f16b2c00640 1 Processor -- start 2024-04-24T09:05:47.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.333+0000 7f16b2c00640 1 -- start start 2024-04-24T09:05:47.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.333+0000 7f16b2c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f16ac104290 0x7f16ac1a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:47.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.333+0000 7f16b2c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16ac109360 0x7f16ac1a0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:47.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.333+0000 7f16b2c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f16ac19a870 0x7f16ac19ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:47.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.333+0000 7f16b2c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f16ac1143d0 con 0x7f16ac104290 2024-04-24T09:05:47.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.333+0000 7f16b2c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f16ac114250 con 0x7f16ac109360 2024-04-24T09:05:47.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.333+0000 7f16b2c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f16ac114550 con 0x7f16ac19a870 2024-04-24T09:05:47.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.334+0000 7f16b1800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f16ac104290 0x7f16ac1a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:47.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.334+0000 7f16b0e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16ac109360 0x7f16ac1a0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:47.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.334+0000 7f16b1800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f16ac104290 0x7f16ac1a06c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50004/0 (socket says 172.21.15.83:50004) 2024-04-24T09:05:47.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.334+0000 7f16b1800640 1 -- 172.21.15.83:0/1468512877 learned_addr learned my addr 172.21.15.83:0/1468512877 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:05:47.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.334+0000 7f16b2200640 1 --2- 172.21.15.83:0/1468512877 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f16ac19a870 0x7f16ac19ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:47.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.334+0000 7f16b1800640 1 -- 172.21.15.83:0/1468512877 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f16ac19a870 msgr2=0x7f16ac19ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:47.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.334+0000 7f16b1800640 1 --2- 172.21.15.83:0/1468512877 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f16ac19a870 0x7f16ac19ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:47.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.334+0000 7f16b1800640 1 -- 172.21.15.83:0/1468512877 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16ac109360 msgr2=0x7f16ac1a0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:47.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.334+0000 7f16b1800640 1 --2- 172.21.15.83:0/1468512877 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16ac109360 0x7f16ac1a0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:47.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.334+0000 7f16b1800640 1 -- 172.21.15.83:0/1468512877 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16ac19b390 con 0x7f16ac104290 2024-04-24T09:05:47.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.335+0000 7f16b2200640 1 --2- 172.21.15.83:0/1468512877 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f16ac19a870 0x7f16ac19ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:05:47.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.335+0000 7f16b1800640 1 --2- 172.21.15.83:0/1468512877 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f16ac104290 0x7f16ac1a06c0 secure :-1 s=READY pgs=2139 cs=0 l=1 rev1=1 crypto rx=0x7f16a000b4a0 tx=0x7f16a000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:47.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.335+0000 7f16aa000640 1 -- 172.21.15.83:0/1468512877 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a0014070 con 0x7f16ac104290 2024-04-24T09:05:47.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.335+0000 7f16aa000640 1 -- 172.21.15.83:0/1468512877 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f16a0010ac0 con 0x7f16ac104290 2024-04-24T09:05:47.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.335+0000 7f16aa000640 1 -- 172.21.15.83:0/1468512877 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a0017c30 con 0x7f16ac104290 2024-04-24T09:05:47.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.336+0000 7f16b2c00640 1 -- 172.21.15.83:0/1468512877 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16ac19b680 con 0x7f16ac104290 2024-04-24T09:05:47.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.336+0000 7f16b2c00640 1 -- 172.21.15.83:0/1468512877 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16ac1a7880 con 0x7f16ac104290 2024-04-24T09:05:47.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.337+0000 7f16b2c00640 1 -- 172.21.15.83:0/1468512877 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16ac068d40 con 0x7f16ac104290 2024-04-24T09:05:47.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.340+0000 7f16aa000640 1 -- 172.21.15.83:0/1468512877 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f16a0017070 con 0x7f16ac104290 2024-04-24T09:05:47.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.341+0000 7f16aa000640 1 --2- 172.21.15.83:0/1468512877 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f16940a3ea0 0x7f16940a6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:47.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.341+0000 7f16aa000640 1 -- 172.21.15.83:0/1468512877 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7f16a00cfe00 con 0x7f16ac104290 2024-04-24T09:05:47.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.342+0000 7f16b0e00640 1 --2- 172.21.15.83:0/1468512877 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f16940a3ea0 0x7f16940a6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:47.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.343+0000 7f16b0e00640 1 --2- 172.21.15.83:0/1468512877 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f16940a3ea0 0x7f16940a6360 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f1698031440 tx=0x7f1698000bc0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:47.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.348+0000 7f16aa000640 1 -- 172.21.15.83:0/1468512877 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f16a009c190 con 0x7f16ac104290 2024-04-24T09:05:47.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.780+0000 7f16b2c00640 1 -- 172.21.15.83:0/1468512877 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f16ac19bda0 con 0x7f16ac104290 2024-04-24T09:05:47.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.781+0000 7f16aa000640 1 -- 172.21.15.83:0/1468512877 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+11958 (secure 0 0 0) 0x7f16a009ba90 con 0x7f16ac104290 2024-04-24T09:05:47.781 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T09:05:47.781 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":47,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/358872257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":358872257},{"type":"v1","addr":"172.21.15.83:6835","nonce":358872257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3157070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3157070851},{"type":"v1","addr":"172.21.15.83:6837","nonce":3157070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":47,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T08:48:29.422576+0000","modified":"2024-04-24T09:05:42.118600+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34520,"mds_1":34493,"mds_2":34451,"mds_3":34427,"mds_4":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14658":{"gid":14658,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6839/4085470844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":4085470844},{"type":"v1","addr":"172.21.15.83:6839","nonce":4085470844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14676":{"gid":14676,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6841/2634754210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":2634754210},{"type":"v1","addr":"172.21.15.83:6841","nonce":2634754210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24382":{"gid":24382,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/1859921310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1859921310},{"type":"v1","addr":"172.21.15.189:6833","nonce":1859921310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/2241923197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2241923197},{"type":"v1","addr":"172.21.15.189:6835","nonce":2241923197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.189:6839/2548965336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":2548965336},{"type":"v1","addr":"172.21.15.189:6839","nonce":2548965336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26686":{"gid":26686,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6837/1678817161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":1678817161},{"type":"v1","addr":"172.21.15.189:6837","nonce":1678817161}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.138:6833/1157967658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1157967658},{"type":"v1","addr":"172.21.15.138:6833","nonce":1157967658}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.138:6835/1909219758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1909219758},{"type":"v1","addr":"172.21.15.138:6835","nonce":1909219758}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34493":{"gid":34493,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.138:6837/900566754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":900566754},{"type":"v1","addr":"172.21.15.138:6837","nonce":900566754}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34520":{"gid":34520,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.138:6839/2236932393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2236932393},{"type":"v1","addr":"172.21.15.138:6839","nonce":2236932393}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34520,"qdb_cluster":[34520,24454,34427,34493,34451]},"id":1}]} 2024-04-24T09:05:47.781 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 47 2024-04-24T09:05:47.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.785+0000 7f16b2c00640 1 -- 172.21.15.83:0/1468512877 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f16940a3ea0 msgr2=0x7f16940a6360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:47.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.785+0000 7f16b2c00640 1 --2- 172.21.15.83:0/1468512877 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f16940a3ea0 0x7f16940a6360 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f1698031440 tx=0x7f1698000bc0 comp rx=0 tx=0).stop 2024-04-24T09:05:47.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.785+0000 7f16b2c00640 1 -- 172.21.15.83:0/1468512877 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f16ac104290 msgr2=0x7f16ac1a06c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:47.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.785+0000 7f16b2c00640 1 --2- 172.21.15.83:0/1468512877 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f16ac104290 0x7f16ac1a06c0 secure :-1 s=READY pgs=2139 cs=0 l=1 rev1=1 crypto rx=0x7f16a000b4a0 tx=0x7f16a000b960 comp rx=0 tx=0).stop 2024-04-24T09:05:47.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.785+0000 7f16b2c00640 1 -- 172.21.15.83:0/1468512877 shutdown_connections 2024-04-24T09:05:47.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.785+0000 7f16b2c00640 1 --2- 172.21.15.83:0/1468512877 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f16940a3ea0 0x7f16940a6360 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:47.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.785+0000 7f16b2c00640 1 --2- 172.21.15.83:0/1468512877 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f16ac19a870 0x7f16ac19ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:47.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.785+0000 7f16b2c00640 1 --2- 172.21.15.83:0/1468512877 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16ac109360 0x7f16ac1a0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:47.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.785+0000 7f16b2c00640 1 --2- 172.21.15.83:0/1468512877 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f16ac104290 0x7f16ac1a06c0 unknown :-1 s=CLOSED pgs=2139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:47.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.785+0000 7f16b2c00640 1 -- 172.21.15.83:0/1468512877 >> 172.21.15.83:0/1468512877 conn(0x7f16ac0ffad0 msgr2=0x7f16ac100e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:47.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.785+0000 7f16b2c00640 1 -- 172.21.15.83:0/1468512877 shutdown_connections 2024-04-24T09:05:47.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:47.785+0000 7f16b2c00640 1 -- 172.21.15.83:0/1468512877 wait complete. 2024-04-24T09:05:47.808 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-24T09:05:48.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.067+0000 7f9417e00640 1 -- 172.21.15.83:0/1327559533 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9418076e50 msgr2=0x7f9418112810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:48.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.067+0000 7f9417e00640 1 --2- 172.21.15.83:0/1327559533 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9418076e50 0x7f9418112810 secure :-1 s=READY pgs=2140 cs=0 l=1 rev1=1 crypto rx=0x7f9404009900 tx=0x7f940402a3f0 comp rx=0 tx=0).stop 2024-04-24T09:05:48.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.067+0000 7f9417e00640 1 -- 172.21.15.83:0/1327559533 shutdown_connections 2024-04-24T09:05:48.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.067+0000 7f9417e00640 1 --2- 172.21.15.83:0/1327559533 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9418112d50 0x7f9418115140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:48.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.067+0000 7f9417e00640 1 --2- 172.21.15.83:0/1327559533 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9418076e50 0x7f9418112810 unknown :-1 s=CLOSED pgs=2140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:48.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.067+0000 7f9417e00640 1 --2- 172.21.15.83:0/1327559533 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9418076530 0x7f9418076910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:48.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.067+0000 7f9417e00640 1 -- 172.21.15.83:0/1327559533 >> 172.21.15.83:0/1327559533 conn(0x7f94180ffb40 msgr2=0x7f9418101f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:48.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.068+0000 7f9417e00640 1 -- 172.21.15.83:0/1327559533 shutdown_connections 2024-04-24T09:05:48.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.068+0000 7f9417e00640 1 -- 172.21.15.83:0/1327559533 wait complete. 2024-04-24T09:05:48.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.068+0000 7f9417e00640 1 Processor -- start 2024-04-24T09:05:48.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.069+0000 7f9417e00640 1 -- start start 2024-04-24T09:05:48.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.069+0000 7f9417e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9418076530 0x7f94181a48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:48.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.069+0000 7f9417e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9418076e50 0x7f94181a4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:48.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.069+0000 7f9417e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9418112d50 0x7f941819ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:48.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.069+0000 7f9417e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f94181187a0 con 0x7f9418076e50 2024-04-24T09:05:48.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.069+0000 7f9417e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9418118620 con 0x7f9418076530 2024-04-24T09:05:48.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.069+0000 7f9417e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9418118920 con 0x7f9418112d50 2024-04-24T09:05:48.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.070+0000 7f9416a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9418076530 0x7f94181a48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:48.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.070+0000 7f9416a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9418076530 0x7f94181a48b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:60638/0 (socket says 172.21.15.83:60638) 2024-04-24T09:05:48.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.070+0000 7f9416a00640 1 -- 172.21.15.83:0/2859316536 learned_addr learned my addr 172.21.15.83:0/2859316536 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:05:48.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.070+0000 7f9416000640 1 --2- 172.21.15.83:0/2859316536 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9418076e50 0x7f94181a4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:48.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.070+0000 7f9416a00640 1 -- 172.21.15.83:0/2859316536 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9418112d50 msgr2=0x7f941819ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:48.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.070+0000 7f9416a00640 1 --2- 172.21.15.83:0/2859316536 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9418112d50 0x7f941819ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:48.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.070+0000 7f9416a00640 1 -- 172.21.15.83:0/2859316536 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9418076e50 msgr2=0x7f94181a4df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:48.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.070+0000 7f9416a00640 1 --2- 172.21.15.83:0/2859316536 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9418076e50 0x7f94181a4df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:48.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.070+0000 7f9416a00640 1 -- 172.21.15.83:0/2859316536 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f941819f1e0 con 0x7f9418076530 2024-04-24T09:05:48.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.071+0000 7f9416a00640 1 --2- 172.21.15.83:0/2859316536 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9418076530 0x7f94181a48b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f940800c920 tx=0x7f940800cde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:48.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.071+0000 7f940f400640 1 -- 172.21.15.83:0/2859316536 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9408007c70 con 0x7f9418076530 2024-04-24T09:05:48.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.071+0000 7f940f400640 1 -- 172.21.15.83:0/2859316536 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9408007e10 con 0x7f9418076530 2024-04-24T09:05:48.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.071+0000 7f9417e00640 1 -- 172.21.15.83:0/2859316536 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f941819f4d0 con 0x7f9418076530 2024-04-24T09:05:48.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.071+0000 7f940f400640 1 -- 172.21.15.83:0/2859316536 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9408018820 con 0x7f9418076530 2024-04-24T09:05:48.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.071+0000 7f9417e00640 1 -- 172.21.15.83:0/2859316536 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94181ab770 con 0x7f9418076530 2024-04-24T09:05:48.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.076+0000 7f942087e740 1 Processor -- start 2024-04-24T09:05:48.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.076+0000 7f942087e740 1 -- start start 2024-04-24T09:05:48.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.077+0000 7f942087e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4684deb40 0x55f4684def20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:48.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.077+0000 7f942087e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f4684df460 0x55f4684e95f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:48.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.077+0000 7f942087e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f4684e9b30 0x55f4684ebf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:48.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.077+0000 7f942087e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55f4684441d0 con 0x55f4684df460 2024-04-24T09:05:48.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.077+0000 7f942087e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f4683ea450 con 0x55f4684deb40 2024-04-24T09:05:48.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.077+0000 7f942087e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55f468440190 con 0x55f4684e9b30 2024-04-24T09:05:48.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.077+0000 7f9417400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4684deb40 0x55f4684def20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:48.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.077+0000 7f9417400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4684deb40 0x55f4684def20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:60646/0 (socket says 172.21.15.83:60646) 2024-04-24T09:05:48.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.077+0000 7f9417400640 1 -- 172.21.15.83:0/2012868944 learned_addr learned my addr 172.21.15.83:0/2012868944 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:05:48.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.077+0000 7f940f400640 1 -- 172.21.15.83:0/2859316536 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f94080189c0 con 0x7f9418076530 2024-04-24T09:05:48.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.078+0000 7f9417400640 1 -- 172.21.15.83:0/2012868944 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f4684e9b30 msgr2=0x55f4684ebf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:48.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.078+0000 7f9417400640 1 --2- 172.21.15.83:0/2012868944 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f4684e9b30 0x55f4684ebf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:48.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.078+0000 7f9417400640 1 -- 172.21.15.83:0/2012868944 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f4684df460 msgr2=0x55f4684e95f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:48.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.078+0000 7f9417400640 1 --2- 172.21.15.83:0/2012868944 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f4684df460 0x55f4684e95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:48.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.078+0000 7f9417400640 1 -- 172.21.15.83:0/2012868944 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4684a7310 con 0x55f4684deb40 2024-04-24T09:05:48.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.078+0000 7f9417400640 1 --2- 172.21.15.83:0/2012868944 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4684deb40 0x55f4684def20 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9410009e00 tx=0x7f941000e720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22f3e17ae6e1840f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:48.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.079+0000 7f9403e00640 1 -- 172.21.15.83:0/2012868944 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9410004d20 con 0x55f4684deb40 2024-04-24T09:05:48.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.079+0000 7f9403e00640 1 -- 172.21.15.83:0/2012868944 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f941000ee70 con 0x55f4684deb40 2024-04-24T09:05:48.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.079+0000 7f9403e00640 1 -- 172.21.15.83:0/2012868944 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9410004850 con 0x55f4684deb40 2024-04-24T09:05:48.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.079+0000 7f940f400640 1 --2- 172.21.15.83:0/2859316536 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f93fc0a3da0 0x7f93fc0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:48.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.079+0000 7f940f400640 1 -- 172.21.15.83:0/2859316536 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7f94080d0db0 con 0x7f9418076530 2024-04-24T09:05:48.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.080+0000 7f942087e740 1 -- 172.21.15.83:0/2012868944 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4684deb40 msgr2=0x55f4684def20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:48.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.080+0000 7f942087e740 1 --2- 172.21.15.83:0/2012868944 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4684deb40 0x55f4684def20 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9410009e00 tx=0x7f941000e720 comp rx=0 tx=0).stop 2024-04-24T09:05:48.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.080+0000 7f942087e740 1 -- 172.21.15.83:0/2012868944 shutdown_connections 2024-04-24T09:05:48.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.080+0000 7f942087e740 1 --2- 172.21.15.83:0/2012868944 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f4684e9b30 0x55f4684ebf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:48.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.080+0000 7f942087e740 1 --2- 172.21.15.83:0/2012868944 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f4684df460 0x55f4684e95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:48.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.080+0000 7f942087e740 1 --2- 172.21.15.83:0/2012868944 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4684deb40 0x55f4684def20 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:48.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.080+0000 7f942087e740 1 -- 172.21.15.83:0/2012868944 >> 172.21.15.83:0/2012868944 conn(0x55f4684dae60 msgr2=0x55f4684db240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:48.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.080+0000 7f9416a00640 1 --2- 172.21.15.83:0/2859316536 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f93fc0a3da0 0x7f93fc0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:48.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.081+0000 7f9416a00640 1 --2- 172.21.15.83:0/2859316536 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f93fc0a3da0 0x7f93fc0a6260 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x55f4684e0100 tx=0x7f94080d1000 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:48.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.081+0000 7f942087e740 1 -- 172.21.15.83:0/2012868944 shutdown_connections 2024-04-24T09:05:48.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.081+0000 7f942087e740 1 -- 172.21.15.83:0/2012868944 wait complete. 2024-04-24T09:05:48.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.082+0000 7f942087e740 1 -- start start 2024-04-24T09:05:48.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.083+0000 7f942087e740 1 Processor -- start 2024-04-24T09:05:48.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.083+0000 7f942087e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4684deb40 0x55f4684e9470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:48.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.083+0000 7f942087e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f4684df460 0x55f4684e6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:48.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.083+0000 7f942087e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f4684e9b30 0x55f4684e6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:48.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.083+0000 7f942087e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55f4684ec9c0 con 0x55f4684df460 2024-04-24T09:05:48.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.083+0000 7f942087e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f4683fd810 con 0x55f4684deb40 2024-04-24T09:05:48.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.083+0000 7f942087e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55f4684ecb40 con 0x55f4684e9b30 2024-04-24T09:05:48.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.083+0000 7f942087e740 10 client.0 _ll_register_callbacks cb 0x55f4683cb140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:05:48.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.084+0000 7f9417400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4684deb40 0x55f4684e9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:48.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.084+0000 7f9417400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4684deb40 0x55f4684e9470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:60656/0 (socket says 172.21.15.83:60656) 2024-04-24T09:05:48.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.084+0000 7f9417400640 1 -- 172.21.15.83:0/3604033848 learned_addr learned my addr 172.21.15.83:0/3604033848 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:05:48.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.084+0000 7f9416000640 1 --2- 172.21.15.83:0/3604033848 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f4684df460 0x55f4684e6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:48.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.084+0000 7f9417400640 1 --2- 172.21.15.83:0/3604033848 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f4684e9b30 0x55f4684e6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:48.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.084+0000 7f9417400640 1 -- 172.21.15.83:0/3604033848 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f4684e9b30 msgr2=0x55f4684e6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:48.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.084+0000 7f9417400640 1 --2- 172.21.15.83:0/3604033848 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f4684e9b30 0x55f4684e6ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:48.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.084+0000 7f9417400640 1 -- 172.21.15.83:0/3604033848 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f4684df460 msgr2=0x55f4684e6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:48.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.084+0000 7f9417400640 1 --2- 172.21.15.83:0/3604033848 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f4684df460 0x55f4684e6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:48.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.084+0000 7f9417400640 1 -- 172.21.15.83:0/3604033848 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f46840e3f0 con 0x55f4684deb40 2024-04-24T09:05:48.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.085+0000 7f9417400640 1 --2- 172.21.15.83:0/3604033848 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4684deb40 0x55f4684e9470 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9410005f40 tx=0x7f9410009f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f178bf6e97038b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:48.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.085+0000 7f9400c00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:05:48.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.085+0000 7f9400c00640 1 -- 172.21.15.83:0/3604033848 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9410003e80 con 0x55f4684deb40 2024-04-24T09:05:48.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.085+0000 7f9400c00640 1 -- 172.21.15.83:0/3604033848 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7f9410010430 con 0x55f4684deb40 2024-04-24T09:05:48.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.085+0000 7f942087e740 1 -- 172.21.15.83:0/3604033848 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f941000ee70 con 0x55f4684deb40 2024-04-24T09:05:48.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.086+0000 7f9400c00640 1 -- 172.21.15.83:0/3604033848 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f9410010630 con 0x55f4684deb40 2024-04-24T09:05:48.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.086+0000 7f942087e740 10 client.37760 fetch_fsmap learned FSMap version 47 2024-04-24T09:05:48.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.086+0000 7f942087e740 1 -- 172.21.15.83:0/3604033848 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55f4681563f0 con 0x55f4684deb40 2024-04-24T09:05:48.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:05:47 smithi138 ceph-mon[31441]: pgmap v444: 129 pgs: 129 active+clean; 283 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 735 KiB/s rd, 2.7 MiB/s wr, 138 op/s 2024-04-24T09:05:48.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:05:47 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/1468512877' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:05:48.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:05:47 smithi189 ceph-mon[31791]: pgmap v444: 129 pgs: 129 active+clean; 283 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 735 KiB/s rd, 2.7 MiB/s wr, 138 op/s 2024-04-24T09:05:48.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:05:47 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/1468512877' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:05:48.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.086+0000 7f9400c00640 1 -- 172.21.15.83:0/3604033848 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f941000ee70 con 0x55f4684deb40 2024-04-24T09:05:48.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.087+0000 7f9400c00640 1 -- 172.21.15.83:0/3604033848 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 47) ==== 8398+0+0 (secure 0 0 0) 0x7f9410018880 con 0x55f4684deb40 2024-04-24T09:05:48.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.087+0000 7f942087e740 10 client.37760 fetch_fsmap finished waiting for FSMap version 47 2024-04-24T09:05:48.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.087+0000 7f942087e740 10 client.37760 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-24T09:05:48.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.087+0000 7f942087e740 20 client.37760 populate_metadata read hostname 'smithi083' 2024-04-24T09:05:48.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.087+0000 7f942087e740 1 --2- 172.21.15.83:0/3604033848 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55f4684f9ba0 0x55f4684fbf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:48.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.087+0000 7f942087e740 4 client.37760 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:05:48.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.087+0000 7f942087e740 1 -- 172.21.15.83:0/3604033848 --> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f468451bf0 con 0x55f4684f9ba0 2024-04-24T09:05:48.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.087+0000 7f9416000640 1 --2- 172.21.15.83:0/3604033848 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55f4684f9ba0 0x55f4684fbf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:48.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.089+0000 7f9416000640 1 --2- 172.21.15.83:0/3604033848 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55f4684f9ba0 0x55f4684fbf90 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c66eeb571e80e82e server_cookie=618bd36abf9c4896 in_seq=0 out_seq=0 2024-04-24T09:05:48.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.089+0000 7f9400c00640 10 client.37760 ms_handle_connect on v2:172.21.15.189:6838/2548965336 2024-04-24T09:05:48.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.091+0000 7f9400c00640 1 -- 172.21.15.83:0/3604033848 <== mds.4 v2:172.21.15.189:6838/2548965336 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f468451bf0 con 0x55f4684f9ba0 2024-04-24T09:05:48.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.091+0000 7f9400c00640 10 client.37760 handle_command_reply: tid=0 2024-04-24T09:05:48.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.091+0000 7f942087e740 1 client.37760 shutdown 2024-04-24T09:05:48.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.091+0000 7f942087e740 1 -- 172.21.15.83:0/3604033848 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4684deb40 msgr2=0x55f4684e9470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:48.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.091+0000 7f942087e740 1 --2- 172.21.15.83:0/3604033848 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4684deb40 0x55f4684e9470 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9410005f40 tx=0x7f9410009f50 comp rx=0 tx=0).stop 2024-04-24T09:05:48.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.091+0000 7f942087e740 1 -- 172.21.15.83:0/3604033848 shutdown_connections 2024-04-24T09:05:48.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.091+0000 7f942087e740 1 --2- 172.21.15.83:0/3604033848 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55f4684f9ba0 0x55f4684fbf90 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:48.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.091+0000 7f942087e740 1 --2- 172.21.15.83:0/3604033848 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f4684e9b30 0x55f4684e6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:48.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.091+0000 7f942087e740 1 --2- 172.21.15.83:0/3604033848 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f4684df460 0x55f4684e6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:48.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.091+0000 7f942087e740 1 --2- 172.21.15.83:0/3604033848 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4684deb40 0x55f4684e9470 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:48.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.091+0000 7f9400c00640 0 client.37760 ms_handle_reset on v2:172.21.15.189:6838/2548965336 2024-04-24T09:05:48.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.091+0000 7f942087e740 1 -- 172.21.15.83:0/3604033848 >> 172.21.15.83:0/3604033848 conn(0x55f4684dae60 msgr2=0x55f4684e0b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:48.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.091+0000 7f942087e740 1 -- 172.21.15.83:0/3604033848 shutdown_connections 2024-04-24T09:05:48.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.092+0000 7f942087e740 1 -- 172.21.15.83:0/3604033848 wait complete. 2024-04-24T09:05:48.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.092+0000 7f942087e740 20 client.37760 trim_cache size 0 max 16384 2024-04-24T09:05:48.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.118+0000 7f942087e740 1 Processor -- start 2024-04-24T09:05:48.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.118+0000 7f942087e740 1 -- start start 2024-04-24T09:05:48.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.119+0000 7f942087e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4684deb40 0x55f4684f5880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:48.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.119+0000 7f942087e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f4684df460 0x55f4684d9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:48.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.119+0000 7f942087e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f4684e9b30 0x55f4684e8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:48.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.119+0000 7f942087e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f94181b5a40 con 0x55f4684e9b30 2024-04-24T09:05:48.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.119+0000 7f942087e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f94181b7700 con 0x55f4684deb40 2024-04-24T09:05:48.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.119+0000 7f942087e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f94181b6110 con 0x55f4684df460 2024-04-24T09:05:48.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.119+0000 7f9417400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4684deb40 0x55f4684f5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:48.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.119+0000 7f9417400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f4684e9b30 0x55f4684e8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:48.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.119+0000 7f9417400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f4684e9b30 0x55f4684e8fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50062/0 (socket says 172.21.15.83:50062) 2024-04-24T09:05:48.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.119+0000 7f9417400640 1 -- 172.21.15.83:0/3501704347 learned_addr learned my addr 172.21.15.83:0/3501704347 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:05:48.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.119+0000 7f9416000640 1 --2- 172.21.15.83:0/3501704347 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f4684df460 0x55f4684d9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:48.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.120+0000 7f9417400640 1 -- 172.21.15.83:0/3501704347 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f4684df460 msgr2=0x55f4684d9dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:48.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.120+0000 7f9417400640 1 --2- 172.21.15.83:0/3501704347 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f4684df460 0x55f4684d9dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:48.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.120+0000 7f9417400640 1 -- 172.21.15.83:0/3501704347 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4684deb40 msgr2=0x55f4684f5880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:48.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.120+0000 7f9417400640 1 --2- 172.21.15.83:0/3501704347 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4684deb40 0x55f4684f5880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:48.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.120+0000 7f9417400640 1 -- 172.21.15.83:0/3501704347 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4682207d0 con 0x55f4684e9b30 2024-04-24T09:05:48.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.120+0000 7f9417400640 1 --2- 172.21.15.83:0/3501704347 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f4684e9b30 0x55f4684e8fa0 secure :-1 s=READY pgs=2141 cs=0 l=1 rev1=1 crypto rx=0x7f9410018c50 tx=0x7f9410000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cd7cda4649d5059 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:48.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.120+0000 7f9403400640 1 -- 172.21.15.83:0/3501704347 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9410003c80 con 0x55f4684e9b30 2024-04-24T09:05:48.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.120+0000 7f9403400640 1 -- 172.21.15.83:0/3501704347 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9410003e20 con 0x55f4684e9b30 2024-04-24T09:05:48.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.120+0000 7f9403400640 1 -- 172.21.15.83:0/3501704347 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9410010cc0 con 0x55f4684e9b30 2024-04-24T09:05:48.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.122+0000 7f942087e740 1 -- 172.21.15.83:0/3501704347 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f4684e9b30 msgr2=0x55f4684e8fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:48.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.122+0000 7f942087e740 1 --2- 172.21.15.83:0/3501704347 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f4684e9b30 0x55f4684e8fa0 secure :-1 s=READY pgs=2141 cs=0 l=1 rev1=1 crypto rx=0x7f9410018c50 tx=0x7f9410000f40 comp rx=0 tx=0).stop 2024-04-24T09:05:48.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.122+0000 7f942087e740 1 -- 172.21.15.83:0/3501704347 shutdown_connections 2024-04-24T09:05:48.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.122+0000 7f942087e740 1 --2- 172.21.15.83:0/3501704347 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f4684e9b30 0x55f4684e8fa0 unknown :-1 s=CLOSED pgs=2141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:48.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.122+0000 7f942087e740 1 --2- 172.21.15.83:0/3501704347 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f4684df460 0x55f4684d9dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:48.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.122+0000 7f942087e740 1 --2- 172.21.15.83:0/3501704347 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4684deb40 0x55f4684f5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:48.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.122+0000 7f942087e740 1 -- 172.21.15.83:0/3501704347 >> 172.21.15.83:0/3501704347 conn(0x55f4684dae60 msgr2=0x55f4684d96c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:48.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.122+0000 7f942087e740 1 -- 172.21.15.83:0/3501704347 shutdown_connections 2024-04-24T09:05:48.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.122+0000 7f942087e740 1 -- 172.21.15.83:0/3501704347 wait complete. 2024-04-24T09:05:48.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.123+0000 7f942087e740 1 -- start start 2024-04-24T09:05:48.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.123+0000 7f942087e740 1 Processor -- start 2024-04-24T09:05:48.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.124+0000 7f942087e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f4684deb40 0x55f4684ecd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:48.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.124+0000 7f942087e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f4684df460 0x55f4684ed290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:48.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.124+0000 7f942087e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4684e9b30 0x55f46855dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:48.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.124+0000 7f942087e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f94181b6f70 con 0x55f4684deb40 2024-04-24T09:05:48.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.124+0000 7f942087e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f94181b6840 con 0x55f4684e9b30 2024-04-24T09:05:48.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.124+0000 7f942087e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f94181b3a40 con 0x55f4684df460 2024-04-24T09:05:48.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.124+0000 7f942087e740 10 client.0 _ll_register_callbacks cb 0x7f9410004c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:05:48.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.124+0000 7f9417400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f4684deb40 0x55f4684ecd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:48.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.124+0000 7f9417400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f4684deb40 0x55f4684ecd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50070/0 (socket says 172.21.15.83:50070) 2024-04-24T09:05:48.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.124+0000 7f9417400640 1 -- 172.21.15.83:0/696347664 learned_addr learned my addr 172.21.15.83:0/696347664 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:05:48.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.125+0000 7f9417400640 1 --2- 172.21.15.83:0/696347664 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4684e9b30 0x55f46855dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:48.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.125+0000 7f9417400640 1 -- 172.21.15.83:0/696347664 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f4684df460 msgr2=0x55f4684ed290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:48.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.125+0000 7f9417400640 1 --2- 172.21.15.83:0/696347664 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f4684df460 0x55f4684ed290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:48.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.125+0000 7f9417400640 1 -- 172.21.15.83:0/696347664 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4684e9b30 msgr2=0x55f46855dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:48.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.125+0000 7f9417400640 1 --2- 172.21.15.83:0/696347664 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4684e9b30 0x55f46855dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:48.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.125+0000 7f9417400640 1 -- 172.21.15.83:0/696347664 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f46855e4f0 con 0x55f4684deb40 2024-04-24T09:05:48.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.126+0000 7f9417400640 1 --2- 172.21.15.83:0/696347664 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f4684deb40 0x55f4684ecd50 secure :-1 s=READY pgs=2142 cs=0 l=1 rev1=1 crypto rx=0x7f9410009920 tx=0x7f9410004570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed27e582f7fe2644 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:48.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.126+0000 7f9400c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:05:48.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.126+0000 7f9400c00640 1 -- 172.21.15.83:0/696347664 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9410042070 con 0x55f4684deb40 2024-04-24T09:05:48.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.126+0000 7f9400c00640 1 -- 172.21.15.83:0/696347664 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7f9410018430 con 0x55f4684deb40 2024-04-24T09:05:48.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.126+0000 7f942087e740 1 -- 172.21.15.83:0/696347664 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9410003e20 con 0x55f4684deb40 2024-04-24T09:05:48.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.127+0000 7f9400c00640 1 -- 172.21.15.83:0/696347664 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f9410010e50 con 0x55f4684deb40 2024-04-24T09:05:48.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.127+0000 7f942087e740 10 client.20619 fetch_fsmap learned FSMap version 47 2024-04-24T09:05:48.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.127+0000 7f942087e740 1 -- 172.21.15.83:0/696347664 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55f46855f570 con 0x55f4684deb40 2024-04-24T09:05:48.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.127+0000 7f9400c00640 1 -- 172.21.15.83:0/696347664 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9410003e20 con 0x55f4684deb40 2024-04-24T09:05:48.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.128+0000 7f9400c00640 1 -- 172.21.15.83:0/696347664 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 47) ==== 8398+0+0 (secure 0 0 0) 0x7f9410027830 con 0x55f4684deb40 2024-04-24T09:05:48.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.128+0000 7f942087e740 10 client.20619 fetch_fsmap finished waiting for FSMap version 47 2024-04-24T09:05:48.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.128+0000 7f942087e740 10 client.20619 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-24T09:05:48.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.128+0000 7f942087e740 20 client.20619 populate_metadata read hostname 'smithi083' 2024-04-24T09:05:48.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.128+0000 7f942087e740 1 --2- 172.21.15.83:0/696347664 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55f46855fbd0 0x55f468561fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:48.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.128+0000 7f942087e740 4 client.20619 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:05:48.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.128+0000 7f942087e740 1 -- 172.21.15.83:0/696347664 --> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f4684e7030 con 0x55f46855fbd0 2024-04-24T09:05:48.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.128+0000 7f9416000640 1 --2- 172.21.15.83:0/696347664 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55f46855fbd0 0x55f468561fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:48.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.129+0000 7f9416000640 1 --2- 172.21.15.83:0/696347664 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55f46855fbd0 0x55f468561fc0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=edb400b7d26b6b7 server_cookie=ae334b939a831303 in_seq=0 out_seq=0 2024-04-24T09:05:48.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.129+0000 7f9400c00640 10 client.20619 ms_handle_connect on v2:172.21.15.189:6838/2548965336 2024-04-24T09:05:48.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.130+0000 7f9400c00640 1 -- 172.21.15.83:0/696347664 <== mds.4 v2:172.21.15.189:6838/2548965336 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f4684e7030 con 0x55f46855fbd0 2024-04-24T09:05:48.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.130+0000 7f9400c00640 10 client.20619 handle_command_reply: tid=0 2024-04-24T09:05:48.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.130+0000 7f942087e740 1 client.20619 shutdown 2024-04-24T09:05:48.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.130+0000 7f942087e740 1 -- 172.21.15.83:0/696347664 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f4684deb40 msgr2=0x55f4684ecd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:48.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.130+0000 7f942087e740 1 --2- 172.21.15.83:0/696347664 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f4684deb40 0x55f4684ecd50 secure :-1 s=READY pgs=2142 cs=0 l=1 rev1=1 crypto rx=0x7f9410009920 tx=0x7f9410004570 comp rx=0 tx=0).stop 2024-04-24T09:05:48.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.130+0000 7f942087e740 1 -- 172.21.15.83:0/696347664 shutdown_connections 2024-04-24T09:05:48.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.130+0000 7f942087e740 1 --2- 172.21.15.83:0/696347664 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55f46855fbd0 0x55f468561fc0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:48.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.130+0000 7f942087e740 1 --2- 172.21.15.83:0/696347664 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4684e9b30 0x55f46855dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:48.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.130+0000 7f942087e740 1 --2- 172.21.15.83:0/696347664 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f4684df460 0x55f4684ed290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:48.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.130+0000 7f942087e740 1 --2- 172.21.15.83:0/696347664 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f4684deb40 0x55f4684ecd50 unknown :-1 s=CLOSED pgs=2142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:48.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.130+0000 7f942087e740 1 -- 172.21.15.83:0/696347664 >> 172.21.15.83:0/696347664 conn(0x55f4684dae60 msgr2=0x55f4684d9f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:48.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.131+0000 7f9400c00640 0 client.20619 ms_handle_reset on v2:172.21.15.189:6838/2548965336 2024-04-24T09:05:48.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.131+0000 7f942087e740 1 -- 172.21.15.83:0/696347664 shutdown_connections 2024-04-24T09:05:48.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.131+0000 7f942087e740 1 -- 172.21.15.83:0/696347664 wait complete. 2024-04-24T09:05:48.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.131+0000 7f942087e740 20 client.20619 trim_cache size 0 max 16384 2024-04-24T09:05:48.319 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:05:48.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.132+0000 7f9417e00640 1 -- 172.21.15.83:0/2859316536 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f93fc0a3da0 msgr2=0x7f93fc0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:48.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.132+0000 7f9417e00640 1 --2- 172.21.15.83:0/2859316536 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f93fc0a3da0 0x7f93fc0a6260 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x55f4684e0100 tx=0x7f94080d1000 comp rx=0 tx=0).stop 2024-04-24T09:05:48.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.132+0000 7f9417e00640 1 -- 172.21.15.83:0/2859316536 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9418076530 msgr2=0x7f94181a48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:48.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.132+0000 7f9417e00640 1 --2- 172.21.15.83:0/2859316536 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9418076530 0x7f94181a48b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f940800c920 tx=0x7f940800cde0 comp rx=0 tx=0).stop 2024-04-24T09:05:48.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.132+0000 7f9417e00640 1 -- 172.21.15.83:0/2859316536 shutdown_connections 2024-04-24T09:05:48.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.132+0000 7f9417e00640 1 --2- 172.21.15.83:0/2859316536 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f93fc0a3da0 0x7f93fc0a6260 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:48.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.132+0000 7f9417e00640 1 --2- 172.21.15.83:0/2859316536 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9418112d50 0x7f941819ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:48.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.132+0000 7f9417e00640 1 --2- 172.21.15.83:0/2859316536 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9418076e50 0x7f94181a4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:48.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.132+0000 7f9417e00640 1 --2- 172.21.15.83:0/2859316536 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9418076530 0x7f94181a48b0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:48.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.132+0000 7f9417e00640 1 -- 172.21.15.83:0/2859316536 >> 172.21.15.83:0/2859316536 conn(0x7f94180ffb40 msgr2=0x7f9418101f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:48.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.132+0000 7f9417e00640 1 -- 172.21.15.83:0/2859316536 shutdown_connections 2024-04-24T09:05:48.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.132+0000 7f9417e00640 1 -- 172.21.15.83:0/2859316536 wait complete. 2024-04-24T09:05:48.321 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:05:47 smithi083 ceph-mon[27907]: pgmap v444: 129 pgs: 129 active+clean; 283 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 735 KiB/s rd, 2.7 MiB/s wr, 138 op/s 2024-04-24T09:05:48.321 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:05:47 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/1468512877' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:05:48.322 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-24T09:05:48.322 INFO:tasks.workunit.client.0.smithi083.stdout: 10 2329 14.50 MB/sec warmup 19 sec latency 103.897 ms 2024-04-24T09:05:48.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.574+0000 7f4fad400640 1 -- 172.21.15.83:0/2352007149 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4fa806d9b0 msgr2=0x7f4fa806dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:48.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.574+0000 7f4fad400640 1 --2- 172.21.15.83:0/2352007149 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4fa806d9b0 0x7f4fa806dd90 secure :-1 s=READY pgs=2143 cs=0 l=1 rev1=1 crypto rx=0x7f4f90009900 tx=0x7f4f9002a3f0 comp rx=0 tx=0).stop 2024-04-24T09:05:48.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.575+0000 7f4fad400640 1 -- 172.21.15.83:0/2352007149 shutdown_connections 2024-04-24T09:05:48.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.575+0000 7f4fad400640 1 --2- 172.21.15.83:0/2352007149 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4fa811b520 0x7f4fa811d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:48.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.575+0000 7f4fad400640 1 --2- 172.21.15.83:0/2352007149 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4fa806e2d0 0x7f4fa811afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:48.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.575+0000 7f4fad400640 1 --2- 172.21.15.83:0/2352007149 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4fa806d9b0 0x7f4fa806dd90 unknown :-1 s=CLOSED pgs=2143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:48.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.575+0000 7f4fad400640 1 -- 172.21.15.83:0/2352007149 >> 172.21.15.83:0/2352007149 conn(0x7f4fa806ce90 msgr2=0x7f4fa810b010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:48.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.575+0000 7f4fad400640 1 -- 172.21.15.83:0/2352007149 shutdown_connections 2024-04-24T09:05:48.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.575+0000 7f4fad400640 1 -- 172.21.15.83:0/2352007149 wait complete. 2024-04-24T09:05:48.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.576+0000 7f4fad400640 1 Processor -- start 2024-04-24T09:05:48.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.576+0000 7f4fad400640 1 -- start start 2024-04-24T09:05:48.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.577+0000 7f4fad400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4fa806d9b0 0x7f4fa81a7550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:48.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.577+0000 7f4fad400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4fa806e2d0 0x7f4fa81a7a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:48.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.577+0000 7f4fad400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4fa811b520 0x7f4fa81abdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:48.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.577+0000 7f4fad400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4fa8120f90 con 0x7f4fa806e2d0 2024-04-24T09:05:48.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.577+0000 7f4fad400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4fa8120e10 con 0x7f4fa806d9b0 2024-04-24T09:05:48.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.577+0000 7f4fad400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4fa8121110 con 0x7f4fa811b520 2024-04-24T09:05:48.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.577+0000 7f4fa7e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4fa806d9b0 0x7f4fa81a7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:48.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.577+0000 7f4fa7e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4fa806d9b0 0x7f4fa81a7550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:60698/0 (socket says 172.21.15.83:60698) 2024-04-24T09:05:48.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.577+0000 7f4fa7400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4fa806e2d0 0x7f4fa81a7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:48.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.577+0000 7f4fa7e00640 1 -- 172.21.15.83:0/2868886262 learned_addr learned my addr 172.21.15.83:0/2868886262 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:05:48.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.577+0000 7f4faca00640 1 --2- 172.21.15.83:0/2868886262 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4fa811b520 0x7f4fa81abdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:48.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.578+0000 7f4fa7400640 1 -- 172.21.15.83:0/2868886262 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4fa811b520 msgr2=0x7f4fa81abdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:48.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.578+0000 7f4fa7400640 1 --2- 172.21.15.83:0/2868886262 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4fa811b520 0x7f4fa81abdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:48.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.578+0000 7f4fa7400640 1 -- 172.21.15.83:0/2868886262 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4fa806d9b0 msgr2=0x7f4fa81a7550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:48.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.578+0000 7f4fa7400640 1 --2- 172.21.15.83:0/2868886262 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4fa806d9b0 0x7f4fa81a7550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:48.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.578+0000 7f4fa7400640 1 -- 172.21.15.83:0/2868886262 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fa81ac4f0 con 0x7f4fa806e2d0 2024-04-24T09:05:48.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.578+0000 7f4fa7400640 1 --2- 172.21.15.83:0/2868886262 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4fa806e2d0 0x7f4fa81a7a90 secure :-1 s=READY pgs=2144 cs=0 l=1 rev1=1 crypto rx=0x7f4f9800d720 tx=0x7f4f9800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:48.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.578+0000 7f4fa4c00640 1 -- 172.21.15.83:0/2868886262 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f98017070 con 0x7f4fa806e2d0 2024-04-24T09:05:48.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.578+0000 7f4fad400640 1 -- 172.21.15.83:0/2868886262 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fa81ac7e0 con 0x7f4fa806e2d0 2024-04-24T09:05:48.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.578+0000 7f4fa4c00640 1 -- 172.21.15.83:0/2868886262 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4f9800de30 con 0x7f4fa806e2d0 2024-04-24T09:05:48.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.578+0000 7f4fad400640 1 -- 172.21.15.83:0/2868886262 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fa81b4070 con 0x7f4fa806e2d0 2024-04-24T09:05:48.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.579+0000 7f4fa4c00640 1 -- 172.21.15.83:0/2868886262 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f980047a0 con 0x7f4fa806e2d0 2024-04-24T09:05:48.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.582+0000 7f4fa4c00640 1 -- 172.21.15.83:0/2868886262 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4f980099d0 con 0x7f4fa806e2d0 2024-04-24T09:05:48.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.583+0000 7f4fb19e8740 1 Processor -- start 2024-04-24T09:05:48.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.583+0000 7f4fb19e8740 1 -- start start 2024-04-24T09:05:48.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.583+0000 7f4fb19e8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559f7c7eeb40 0x559f7c7eef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:48.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.583+0000 7f4fb19e8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f7c7ef460 0x559f7c7f95f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:48.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.583+0000 7f4fb19e8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f7c7f9b30 0x559f7c7fbf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:48.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.583+0000 7f4fb19e8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559f7c7541d0 con 0x559f7c7f9b30 2024-04-24T09:05:48.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.583+0000 7f4fb19e8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559f7c6fa450 con 0x559f7c7ef460 2024-04-24T09:05:48.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.583+0000 7f4fb19e8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559f7c750190 con 0x559f7c7eeb40 2024-04-24T09:05:48.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.584+0000 7f4fa7e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f7c7ef460 0x559f7c7f95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:48.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.584+0000 7f4fa7e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f7c7ef460 0x559f7c7f95f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:60712/0 (socket says 172.21.15.83:60712) 2024-04-24T09:05:48.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.584+0000 7f4fa7e00640 1 -- 172.21.15.83:0/643206050 learned_addr learned my addr 172.21.15.83:0/643206050 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:05:48.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.584+0000 7f4faca00640 1 --2- 172.21.15.83:0/643206050 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559f7c7eeb40 0x559f7c7eef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:48.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.584+0000 7f4fa7400640 1 --2- 172.21.15.83:0/643206050 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f7c7f9b30 0x559f7c7fbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:48.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.584+0000 7f4fa7e00640 1 -- 172.21.15.83:0/643206050 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559f7c7eeb40 msgr2=0x559f7c7eef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:48.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.584+0000 7f4fa7e00640 1 --2- 172.21.15.83:0/643206050 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559f7c7eeb40 0x559f7c7eef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:48.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.584+0000 7f4fa7e00640 1 -- 172.21.15.83:0/643206050 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f7c7f9b30 msgr2=0x559f7c7fbf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:48.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.584+0000 7f4fa7e00640 1 --2- 172.21.15.83:0/643206050 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f7c7f9b30 0x559f7c7fbf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:48.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.584+0000 7f4fa7e00640 1 -- 172.21.15.83:0/643206050 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f7c7b7310 con 0x559f7c7ef460 2024-04-24T09:05:48.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.585+0000 7f4fa4c00640 1 --2- 172.21.15.83:0/2868886262 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f4f8c0a3da0 0x7f4f8c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:48.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.585+0000 7f4fa7e00640 1 --2- 172.21.15.83:0/643206050 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f7c7ef460 0x559f7c7f95f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4f90004200 tx=0x7f4f90004230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6bc802d80147113d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:48.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.585+0000 7f4fa4c00640 1 -- 172.21.15.83:0/2868886262 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7f4f980d0910 con 0x7f4fa806e2d0 2024-04-24T09:05:48.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.586+0000 7f4faca00640 1 --2- 172.21.15.83:0/2868886262 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f4f8c0a3da0 0x7f4f8c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:48.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.586+0000 7f4f9d200640 1 -- 172.21.15.83:0/643206050 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f90040070 con 0x559f7c7ef460 2024-04-24T09:05:48.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.586+0000 7f4f9d200640 1 -- 172.21.15.83:0/643206050 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4f900047a0 con 0x559f7c7ef460 2024-04-24T09:05:48.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.586+0000 7f4f9d200640 1 -- 172.21.15.83:0/643206050 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f90035040 con 0x559f7c7ef460 2024-04-24T09:05:48.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.587+0000 7f4fb19e8740 1 -- 172.21.15.83:0/643206050 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f7c7ef460 msgr2=0x559f7c7f95f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:48.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.587+0000 7f4faca00640 1 --2- 172.21.15.83:0/2868886262 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f4f8c0a3da0 0x7f4f8c0a6260 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f4fa00092f0 tx=0x7f4fa0009280 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:48.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.587+0000 7f4fb19e8740 1 --2- 172.21.15.83:0/643206050 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f7c7ef460 0x559f7c7f95f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4f90004200 tx=0x7f4f90004230 comp rx=0 tx=0).stop 2024-04-24T09:05:48.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.587+0000 7f4fb19e8740 1 -- 172.21.15.83:0/643206050 shutdown_connections 2024-04-24T09:05:48.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.587+0000 7f4fb19e8740 1 --2- 172.21.15.83:0/643206050 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f7c7f9b30 0x559f7c7fbf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:48.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.587+0000 7f4fb19e8740 1 --2- 172.21.15.83:0/643206050 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f7c7ef460 0x559f7c7f95f0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:48.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.587+0000 7f4fb19e8740 1 --2- 172.21.15.83:0/643206050 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559f7c7eeb40 0x559f7c7eef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:48.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.587+0000 7f4fb19e8740 1 -- 172.21.15.83:0/643206050 >> 172.21.15.83:0/643206050 conn(0x559f7c7eae60 msgr2=0x559f7c7eb240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:48.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.587+0000 7f4fb19e8740 1 -- 172.21.15.83:0/643206050 shutdown_connections 2024-04-24T09:05:48.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.587+0000 7f4fb19e8740 1 -- 172.21.15.83:0/643206050 wait complete. 2024-04-24T09:05:48.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.587+0000 7f4fb19e8740 1 -- start start 2024-04-24T09:05:48.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.588+0000 7f4fb19e8740 1 Processor -- start 2024-04-24T09:05:48.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.588+0000 7f4fb19e8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f7c7eeb40 0x559f7c7f6910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:48.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.588+0000 7f4fb19e8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f7c7ef460 0x559f7c7f6e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:48.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.588+0000 7f4fb19e8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559f7c7f9b30 0x559f7c7f7390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:48.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.588+0000 7f4fb19e8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559f7c7fc960 con 0x559f7c7ef460 2024-04-24T09:05:48.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.588+0000 7f4fb19e8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559f7c70d810 con 0x559f7c7eeb40 2024-04-24T09:05:48.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.588+0000 7f4fb19e8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559f7c7fcae0 con 0x559f7c7f9b30 2024-04-24T09:05:48.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.589+0000 7f4fb19e8740 10 client.0 _ll_register_callbacks cb 0x559f7c6db140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:05:48.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.589+0000 7f4fa7e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f7c7eeb40 0x559f7c7f6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:48.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.589+0000 7f4fa7e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f7c7eeb40 0x559f7c7f6910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:60714/0 (socket says 172.21.15.83:60714) 2024-04-24T09:05:48.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.589+0000 7f4fa7e00640 1 -- 172.21.15.83:0/293848804 learned_addr learned my addr 172.21.15.83:0/293848804 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:05:48.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.590+0000 7f4faca00640 1 --2- 172.21.15.83:0/293848804 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559f7c7f9b30 0x559f7c7f7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:48.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.590+0000 7f4fa7400640 1 --2- 172.21.15.83:0/293848804 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f7c7ef460 0x559f7c7f6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:48.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.591+0000 7f4fa7e00640 1 -- 172.21.15.83:0/293848804 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559f7c7f9b30 msgr2=0x559f7c7f7390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:48.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.591+0000 7f4fa7e00640 1 --2- 172.21.15.83:0/293848804 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559f7c7f9b30 0x559f7c7f7390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:48.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.591+0000 7f4fa7e00640 1 -- 172.21.15.83:0/293848804 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f7c7ef460 msgr2=0x559f7c7f6e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:48.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.591+0000 7f4fa7e00640 1 --2- 172.21.15.83:0/293848804 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f7c7ef460 0x559f7c7f6e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:48.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.591+0000 7f4fa7e00640 1 -- 172.21.15.83:0/293848804 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f7c71e3f0 con 0x559f7c7eeb40 2024-04-24T09:05:48.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.591+0000 7f4faca00640 1 --2- 172.21.15.83:0/293848804 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559f7c7f9b30 0x559f7c7f7390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:05:48.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.592+0000 7f4fa7e00640 1 --2- 172.21.15.83:0/293848804 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f7c7eeb40 0x559f7c7f6910 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4f90000c00 tx=0x7f4f90004b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1568d1ec53b90b7a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:48.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.592+0000 7f4f95600640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:05:48.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.592+0000 7f4f95600640 1 -- 172.21.15.83:0/293848804 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f90035070 con 0x559f7c7eeb40 2024-04-24T09:05:48.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.592+0000 7f4f95600640 1 -- 172.21.15.83:0/293848804 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7f4f90005430 con 0x559f7c7eeb40 2024-04-24T09:05:48.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.593+0000 7f4fb19e8740 1 -- 172.21.15.83:0/293848804 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f900047a0 con 0x559f7c7eeb40 2024-04-24T09:05:48.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.593+0000 7f4f95600640 1 -- 172.21.15.83:0/293848804 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f4f90040040 con 0x559f7c7eeb40 2024-04-24T09:05:48.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.593+0000 7f4fb19e8740 10 client.37778 fetch_fsmap learned FSMap version 47 2024-04-24T09:05:48.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.593+0000 7f4fb19e8740 1 -- 172.21.15.83:0/293848804 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x559f7c4663f0 con 0x559f7c7eeb40 2024-04-24T09:05:48.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.594+0000 7f4f95600640 1 -- 172.21.15.83:0/293848804 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4f900047a0 con 0x559f7c7eeb40 2024-04-24T09:05:48.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.594+0000 7f4f95600640 1 -- 172.21.15.83:0/293848804 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 47) ==== 8398+0+0 (secure 0 0 0) 0x7f4f90057480 con 0x559f7c7eeb40 2024-04-24T09:05:48.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.594+0000 7f4fb19e8740 10 client.37778 fetch_fsmap finished waiting for FSMap version 47 2024-04-24T09:05:48.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.594+0000 7f4fb19e8740 10 client.37778 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-24T09:05:48.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.594+0000 7f4fb19e8740 20 client.37778 populate_metadata read hostname 'smithi083' 2024-04-24T09:05:48.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.594+0000 7f4fb19e8740 1 --2- 172.21.15.83:0/293848804 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x559f7c809bd0 0x559f7c80bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:48.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.594+0000 7f4fb19e8740 4 client.37778 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:05:48.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.594+0000 7f4fb19e8740 1 -- 172.21.15.83:0/293848804 --> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559f7c761bf0 con 0x559f7c809bd0 2024-04-24T09:05:48.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.595+0000 7f4fa7400640 1 --2- 172.21.15.83:0/293848804 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x559f7c809bd0 0x559f7c80bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:48.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.598+0000 7f4f95600640 10 client.37778 ms_handle_connect on v2:172.21.15.138:6832/1157967658 2024-04-24T09:05:48.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.598+0000 7f4fa7400640 1 --2- 172.21.15.83:0/293848804 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x559f7c809bd0 0x559f7c80bfc0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6aa499638e13f6e5 server_cookie=ce2686914e02be82 in_seq=0 out_seq=0 2024-04-24T09:05:48.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.599+0000 7f4f95600640 1 -- 172.21.15.83:0/293848804 <== mds.3 v2:172.21.15.138:6832/1157967658 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559f7c761bf0 con 0x559f7c809bd0 2024-04-24T09:05:48.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.599+0000 7f4f95600640 10 client.37778 handle_command_reply: tid=0 2024-04-24T09:05:48.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.599+0000 7f4fb19e8740 1 client.37778 shutdown 2024-04-24T09:05:48.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.599+0000 7f4fb19e8740 1 -- 172.21.15.83:0/293848804 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f7c7eeb40 msgr2=0x559f7c7f6910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:48.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.599+0000 7f4fb19e8740 1 --2- 172.21.15.83:0/293848804 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f7c7eeb40 0x559f7c7f6910 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4f90000c00 tx=0x7f4f90004b80 comp rx=0 tx=0).stop 2024-04-24T09:05:48.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.600+0000 7f4fb19e8740 1 -- 172.21.15.83:0/293848804 shutdown_connections 2024-04-24T09:05:48.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.600+0000 7f4fb19e8740 1 --2- 172.21.15.83:0/293848804 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x559f7c809bd0 0x559f7c80bfc0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:48.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.600+0000 7f4fb19e8740 1 --2- 172.21.15.83:0/293848804 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559f7c7f9b30 0x559f7c7f7390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:48.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.600+0000 7f4fb19e8740 1 --2- 172.21.15.83:0/293848804 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f7c7ef460 0x559f7c7f6e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:48.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.600+0000 7f4f95600640 0 client.37778 ms_handle_reset on v2:172.21.15.138:6832/1157967658 2024-04-24T09:05:48.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.600+0000 7f4fb19e8740 1 --2- 172.21.15.83:0/293848804 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f7c7eeb40 0x559f7c7f6910 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:48.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.600+0000 7f4fb19e8740 1 -- 172.21.15.83:0/293848804 >> 172.21.15.83:0/293848804 conn(0x559f7c7eae60 msgr2=0x559f7c7f0c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:48.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.600+0000 7f4fb19e8740 1 -- 172.21.15.83:0/293848804 shutdown_connections 2024-04-24T09:05:48.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.600+0000 7f4fb19e8740 1 -- 172.21.15.83:0/293848804 wait complete. 2024-04-24T09:05:48.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.600+0000 7f4fb19e8740 20 client.37778 trim_cache size 0 max 16384 2024-04-24T09:05:48.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.617+0000 7f4fb19e8740 1 Processor -- start 2024-04-24T09:05:48.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.617+0000 7f4fb19e8740 1 -- start start 2024-04-24T09:05:48.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.617+0000 7f4fb19e8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f7c7eeb40 0x559f7c7f5a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:48.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.617+0000 7f4fb19e8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559f7c7ef460 0x559f7c805440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:48.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.617+0000 7f4fb19e8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f7c7f9b30 0x559f7c7f3b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:48.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.617+0000 7f4fb19e8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4fa81be2f0 con 0x559f7c7eeb40 2024-04-24T09:05:48.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.617+0000 7f4fb19e8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4fa81bffb0 con 0x559f7c7f9b30 2024-04-24T09:05:48.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.617+0000 7f4fb19e8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4fa81be9c0 con 0x559f7c7ef460 2024-04-24T09:05:48.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.618+0000 7f4fa7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f7c7eeb40 0x559f7c7f5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:48.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.618+0000 7f4faca00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f7c7f9b30 0x559f7c7f3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:48.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.618+0000 7f4faca00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f7c7f9b30 0x559f7c7f3b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:60726/0 (socket says 172.21.15.83:60726) 2024-04-24T09:05:48.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.618+0000 7f4faca00640 1 -- 172.21.15.83:0/1424877670 learned_addr learned my addr 172.21.15.83:0/1424877670 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:05:48.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.618+0000 7f4fa7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f7c7eeb40 0x559f7c7f5a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50124/0 (socket says 172.21.15.83:50124) 2024-04-24T09:05:48.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.618+0000 7f4fa7400640 1 --2- 172.21.15.83:0/1424877670 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559f7c7ef460 0x559f7c805440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:48.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.618+0000 7f4fa7e00640 1 -- 172.21.15.83:0/1424877670 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559f7c7ef460 msgr2=0x559f7c805440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:48.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.619+0000 7f4fa7e00640 1 --2- 172.21.15.83:0/1424877670 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559f7c7ef460 0x559f7c805440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:48.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.619+0000 7f4fa7e00640 1 -- 172.21.15.83:0/1424877670 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f7c7f9b30 msgr2=0x559f7c7f3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:48.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.619+0000 7f4fa7e00640 1 --2- 172.21.15.83:0/1424877670 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f7c7f9b30 0x559f7c7f3b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:48.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.619+0000 7f4fa7e00640 1 -- 172.21.15.83:0/1424877670 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f7c5307d0 con 0x559f7c7eeb40 2024-04-24T09:05:48.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.619+0000 7f4faca00640 1 --2- 172.21.15.83:0/1424877670 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f7c7f9b30 0x559f7c7f3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:05:48.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.619+0000 7f4fa7400640 1 --2- 172.21.15.83:0/1424877670 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559f7c7ef460 0x559f7c805440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:05:48.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.619+0000 7f4fa7e00640 1 --2- 172.21.15.83:0/1424877670 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f7c7eeb40 0x559f7c7f5a70 secure :-1 s=READY pgs=2145 cs=0 l=1 rev1=1 crypto rx=0x7f4f90005ec0 tx=0x7f4f90005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2eb5a5f8749b269 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:48.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.619+0000 7f4f97e00640 1 -- 172.21.15.83:0/1424877670 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f9005c030 con 0x559f7c7eeb40 2024-04-24T09:05:48.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.619+0000 7f4f97e00640 1 -- 172.21.15.83:0/1424877670 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4f900046c0 con 0x559f7c7eeb40 2024-04-24T09:05:48.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.619+0000 7f4f97e00640 1 -- 172.21.15.83:0/1424877670 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f90035030 con 0x559f7c7eeb40 2024-04-24T09:05:48.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.620+0000 7f4fb19e8740 1 -- 172.21.15.83:0/1424877670 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f7c7eeb40 msgr2=0x559f7c7f5a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:48.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.620+0000 7f4fb19e8740 1 --2- 172.21.15.83:0/1424877670 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f7c7eeb40 0x559f7c7f5a70 secure :-1 s=READY pgs=2145 cs=0 l=1 rev1=1 crypto rx=0x7f4f90005ec0 tx=0x7f4f90005c00 comp rx=0 tx=0).stop 2024-04-24T09:05:48.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.620+0000 7f4fb19e8740 1 -- 172.21.15.83:0/1424877670 shutdown_connections 2024-04-24T09:05:48.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.620+0000 7f4fb19e8740 1 --2- 172.21.15.83:0/1424877670 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f7c7f9b30 0x559f7c7f3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:48.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.620+0000 7f4fb19e8740 1 --2- 172.21.15.83:0/1424877670 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559f7c7ef460 0x559f7c805440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:48.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.620+0000 7f4fb19e8740 1 --2- 172.21.15.83:0/1424877670 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f7c7eeb40 0x559f7c7f5a70 unknown :-1 s=CLOSED pgs=2145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:48.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.620+0000 7f4fb19e8740 1 -- 172.21.15.83:0/1424877670 >> 172.21.15.83:0/1424877670 conn(0x559f7c7eae60 msgr2=0x559f7c804d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:48.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.621+0000 7f4fb19e8740 1 -- 172.21.15.83:0/1424877670 shutdown_connections 2024-04-24T09:05:48.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.621+0000 7f4fb19e8740 1 -- 172.21.15.83:0/1424877670 wait complete. 2024-04-24T09:05:48.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.621+0000 7f4fb19e8740 1 -- start start 2024-04-24T09:05:48.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.621+0000 7f4fb19e8740 1 Processor -- start 2024-04-24T09:05:48.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.622+0000 7f4fb19e8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f7c7eeb40 0x559f7c80f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:48.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.622+0000 7f4fb19e8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f7c7ef460 0x559f7c80a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:48.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.622+0000 7f4fb19e8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559f7c7f9b30 0x559f7c80ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:48.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.622+0000 7f4fb19e8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4fa81bf820 con 0x559f7c7eeb40 2024-04-24T09:05:48.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.622+0000 7f4fb19e8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4fa81bf0f0 con 0x559f7c7ef460 2024-04-24T09:05:48.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.622+0000 7f4fb19e8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4fa81bc2f0 con 0x559f7c7f9b30 2024-04-24T09:05:48.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.622+0000 7f4fb19e8740 10 client.0 _ll_register_callbacks cb 0x7f4f900040d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:05:48.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.622+0000 7f4fa7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f7c7eeb40 0x559f7c80f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:48.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.623+0000 7f4fa7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f7c7eeb40 0x559f7c80f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50136/0 (socket says 172.21.15.83:50136) 2024-04-24T09:05:48.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.623+0000 7f4fa7e00640 1 -- 172.21.15.83:0/3569208594 learned_addr learned my addr 172.21.15.83:0/3569208594 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:05:48.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.623+0000 7f4fa7400640 1 --2- 172.21.15.83:0/3569208594 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f7c7ef460 0x559f7c80a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:48.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.623+0000 7f4faca00640 1 --2- 172.21.15.83:0/3569208594 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559f7c7f9b30 0x559f7c80ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:48.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.623+0000 7f4fa7e00640 1 -- 172.21.15.83:0/3569208594 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559f7c7f9b30 msgr2=0x559f7c80ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:48.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.623+0000 7f4fa7e00640 1 --2- 172.21.15.83:0/3569208594 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559f7c7f9b30 0x559f7c80ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:48.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.623+0000 7f4fa7e00640 1 -- 172.21.15.83:0/3569208594 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f7c7ef460 msgr2=0x559f7c80a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:48.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.623+0000 7f4fa7e00640 1 --2- 172.21.15.83:0/3569208594 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f7c7ef460 0x559f7c80a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:48.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.623+0000 7f4fa7e00640 1 -- 172.21.15.83:0/3569208594 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f7c80b1f0 con 0x559f7c7eeb40 2024-04-24T09:05:48.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.623+0000 7f4fa7400640 1 --2- 172.21.15.83:0/3569208594 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f7c7ef460 0x559f7c80a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:05:48.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.623+0000 7f4faca00640 1 --2- 172.21.15.83:0/3569208594 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559f7c7f9b30 0x559f7c80ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:05:48.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.624+0000 7f4fa7e00640 1 --2- 172.21.15.83:0/3569208594 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f7c7eeb40 0x559f7c80f270 secure :-1 s=READY pgs=2146 cs=0 l=1 rev1=1 crypto rx=0x7f4f90006010 tx=0x7f4f90057730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16d88e00eb43e860 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:48.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.624+0000 7f4f95600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:05:48.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.624+0000 7f4f95600640 1 -- 172.21.15.83:0/3569208594 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f90065060 con 0x559f7c7eeb40 2024-04-24T09:05:48.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.624+0000 7f4fb19e8740 1 -- 172.21.15.83:0/3569208594 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f900046c0 con 0x559f7c7eeb40 2024-04-24T09:05:48.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.624+0000 7f4f95600640 1 -- 172.21.15.83:0/3569208594 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7f4f90004d60 con 0x559f7c7eeb40 2024-04-24T09:05:48.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.625+0000 7f4f95600640 1 -- 172.21.15.83:0/3569208594 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f4f9006c050 con 0x559f7c7eeb40 2024-04-24T09:05:48.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.625+0000 7f4fb19e8740 10 client.20637 fetch_fsmap learned FSMap version 47 2024-04-24T09:05:48.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.625+0000 7f4fb19e8740 1 -- 172.21.15.83:0/3569208594 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x559f7c86fbc0 con 0x559f7c7eeb40 2024-04-24T09:05:48.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.626+0000 7f4f95600640 1 -- 172.21.15.83:0/3569208594 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4f900046c0 con 0x559f7c7eeb40 2024-04-24T09:05:48.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.626+0000 7f4f95600640 1 -- 172.21.15.83:0/3569208594 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 47) ==== 8398+0+0 (secure 0 0 0) 0x7f4f90003bd0 con 0x559f7c7eeb40 2024-04-24T09:05:48.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.626+0000 7f4fb19e8740 10 client.20637 fetch_fsmap finished waiting for FSMap version 47 2024-04-24T09:05:48.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.626+0000 7f4fb19e8740 10 client.20637 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-24T09:05:48.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.626+0000 7f4fb19e8740 20 client.20637 populate_metadata read hostname 'smithi083' 2024-04-24T09:05:48.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.626+0000 7f4fb19e8740 1 --2- 172.21.15.83:0/3569208594 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x559f7c870020 0x559f7c872440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:48.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.626+0000 7f4fb19e8740 4 client.20637 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:05:48.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.626+0000 7f4fb19e8740 1 -- 172.21.15.83:0/3569208594 --> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559f7c75ba60 con 0x559f7c870020 2024-04-24T09:05:48.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.627+0000 7f4fa7400640 1 --2- 172.21.15.83:0/3569208594 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x559f7c870020 0x559f7c872440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:48.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.628+0000 7f4fa7400640 1 --2- 172.21.15.83:0/3569208594 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x559f7c870020 0x559f7c872440 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=751509d51c1e96e9 server_cookie=626f69f6f76fb39d in_seq=0 out_seq=0 2024-04-24T09:05:48.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.628+0000 7f4f95600640 10 client.20637 ms_handle_connect on v2:172.21.15.138:6832/1157967658 2024-04-24T09:05:48.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.629+0000 7f4f95600640 1 -- 172.21.15.83:0/3569208594 <== mds.3 v2:172.21.15.138:6832/1157967658 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559f7c75ba60 con 0x559f7c870020 2024-04-24T09:05:48.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.629+0000 7f4f95600640 10 client.20637 handle_command_reply: tid=0 2024-04-24T09:05:48.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.629+0000 7f4fb19e8740 1 client.20637 shutdown 2024-04-24T09:05:48.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.629+0000 7f4fb19e8740 1 -- 172.21.15.83:0/3569208594 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f7c7eeb40 msgr2=0x559f7c80f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:48.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.629+0000 7f4fb19e8740 1 --2- 172.21.15.83:0/3569208594 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f7c7eeb40 0x559f7c80f270 secure :-1 s=READY pgs=2146 cs=0 l=1 rev1=1 crypto rx=0x7f4f90006010 tx=0x7f4f90057730 comp rx=0 tx=0).stop 2024-04-24T09:05:48.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.629+0000 7f4fb19e8740 1 -- 172.21.15.83:0/3569208594 shutdown_connections 2024-04-24T09:05:48.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.629+0000 7f4fb19e8740 1 --2- 172.21.15.83:0/3569208594 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x559f7c870020 0x559f7c872440 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:48.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.630+0000 7f4fb19e8740 1 --2- 172.21.15.83:0/3569208594 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559f7c7f9b30 0x559f7c80ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:48.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.630+0000 7f4f95600640 0 client.20637 ms_handle_reset on v2:172.21.15.138:6832/1157967658 2024-04-24T09:05:48.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.630+0000 7f4fb19e8740 1 --2- 172.21.15.83:0/3569208594 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f7c7ef460 0x559f7c80a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:48.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.630+0000 7f4fb19e8740 1 --2- 172.21.15.83:0/3569208594 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f7c7eeb40 0x559f7c80f270 unknown :-1 s=CLOSED pgs=2146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:48.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.630+0000 7f4fb19e8740 1 -- 172.21.15.83:0/3569208594 >> 172.21.15.83:0/3569208594 conn(0x559f7c7eae60 msgr2=0x559f7c804d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:48.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.630+0000 7f4fb19e8740 1 -- 172.21.15.83:0/3569208594 shutdown_connections 2024-04-24T09:05:48.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.630+0000 7f4fb19e8740 1 -- 172.21.15.83:0/3569208594 wait complete. 2024-04-24T09:05:48.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.630+0000 7f4fb19e8740 20 client.20637 trim_cache size 0 max 16384 2024-04-24T09:05:48.630 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:05:48.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.631+0000 7f4fad400640 1 -- 172.21.15.83:0/2868886262 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f4f8c0a3da0 msgr2=0x7f4f8c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:48.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.631+0000 7f4fad400640 1 --2- 172.21.15.83:0/2868886262 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f4f8c0a3da0 0x7f4f8c0a6260 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f4fa00092f0 tx=0x7f4fa0009280 comp rx=0 tx=0).stop 2024-04-24T09:05:48.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.631+0000 7f4fad400640 1 -- 172.21.15.83:0/2868886262 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4fa806e2d0 msgr2=0x7f4fa81a7a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:48.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.631+0000 7f4fad400640 1 --2- 172.21.15.83:0/2868886262 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4fa806e2d0 0x7f4fa81a7a90 secure :-1 s=READY pgs=2144 cs=0 l=1 rev1=1 crypto rx=0x7f4f9800d720 tx=0x7f4f9800dbe0 comp rx=0 tx=0).stop 2024-04-24T09:05:48.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.631+0000 7f4fad400640 1 -- 172.21.15.83:0/2868886262 shutdown_connections 2024-04-24T09:05:48.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.631+0000 7f4fad400640 1 --2- 172.21.15.83:0/2868886262 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f4f8c0a3da0 0x7f4f8c0a6260 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:48.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.631+0000 7f4fad400640 1 --2- 172.21.15.83:0/2868886262 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4fa811b520 0x7f4fa81abdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:48.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.631+0000 7f4fad400640 1 --2- 172.21.15.83:0/2868886262 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4fa806e2d0 0x7f4fa81a7a90 unknown :-1 s=CLOSED pgs=2144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:48.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.631+0000 7f4fad400640 1 --2- 172.21.15.83:0/2868886262 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4fa806d9b0 0x7f4fa81a7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:48.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.631+0000 7f4fad400640 1 -- 172.21.15.83:0/2868886262 >> 172.21.15.83:0/2868886262 conn(0x7f4fa806ce90 msgr2=0x7f4fa810b010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:48.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.631+0000 7f4fad400640 1 -- 172.21.15.83:0/2868886262 shutdown_connections 2024-04-24T09:05:48.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.631+0000 7f4fad400640 1 -- 172.21.15.83:0/2868886262 wait complete. 2024-04-24T09:05:48.652 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T09:05:48.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.916+0000 7f58e8e00640 1 -- 172.21.15.83:0/1923179253 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58e41044b0 msgr2=0x7f58e4104890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:48.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.916+0000 7f58e8e00640 1 --2- 172.21.15.83:0/1923179253 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58e41044b0 0x7f58e4104890 secure :-1 s=READY pgs=2147 cs=0 l=1 rev1=1 crypto rx=0x7f58d0009860 tx=0x7f58d002a340 comp rx=0 tx=0).stop 2024-04-24T09:05:48.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.916+0000 7f58e8e00640 1 -- 172.21.15.83:0/1923179253 shutdown_connections 2024-04-24T09:05:48.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.916+0000 7f58e8e00640 1 --2- 172.21.15.83:0/1923179253 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f58e4109580 0x7f58e4111150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:48.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.916+0000 7f58e8e00640 1 --2- 172.21.15.83:0/1923179253 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f58e4104e60 0x7f58e4108ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:48.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.916+0000 7f58e8e00640 1 --2- 172.21.15.83:0/1923179253 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58e41044b0 0x7f58e4104890 unknown :-1 s=CLOSED pgs=2147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:48.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.916+0000 7f58e8e00640 1 -- 172.21.15.83:0/1923179253 >> 172.21.15.83:0/1923179253 conn(0x7f58e40ffd90 msgr2=0x7f58e41021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:48.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.917+0000 7f58e8e00640 1 -- 172.21.15.83:0/1923179253 shutdown_connections 2024-04-24T09:05:48.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.917+0000 7f58e8e00640 1 -- 172.21.15.83:0/1923179253 wait complete. 2024-04-24T09:05:48.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.918+0000 7f58e8e00640 1 Processor -- start 2024-04-24T09:05:48.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.919+0000 7f58e8e00640 1 -- start start 2024-04-24T09:05:48.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.919+0000 7f58e8e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58e41044b0 0x7f58e41a07b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:48.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.919+0000 7f58e8e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f58e4109580 0x7f58e41a0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:48.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.919+0000 7f58e8e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f58e419a960 0x7f58e419ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:48.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.919+0000 7f58e8e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f58e4114540 con 0x7f58e41044b0 2024-04-24T09:05:48.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.919+0000 7f58e8e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f58e41143c0 con 0x7f58e4109580 2024-04-24T09:05:48.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.919+0000 7f58e8e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f58e41146c0 con 0x7f58e419a960 2024-04-24T09:05:48.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.920+0000 7f58e1400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f58e4109580 0x7f58e41a0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:48.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.920+0000 7f58e1400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f58e4109580 0x7f58e41a0cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:34790/0 (socket says 172.21.15.83:34790) 2024-04-24T09:05:48.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.920+0000 7f58e1400640 1 -- 172.21.15.83:0/3271463313 learned_addr learned my addr 172.21.15.83:0/3271463313 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:05:48.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.920+0000 7f58e1e00640 1 --2- 172.21.15.83:0/3271463313 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58e41044b0 0x7f58e41a07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:48.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.920+0000 7f58e1400640 1 -- 172.21.15.83:0/3271463313 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f58e419a960 msgr2=0x7f58e419ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:48.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.920+0000 7f58e1400640 1 --2- 172.21.15.83:0/3271463313 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f58e419a960 0x7f58e419ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:48.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.920+0000 7f58e1400640 1 -- 172.21.15.83:0/3271463313 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58e41044b0 msgr2=0x7f58e41a07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:48.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.920+0000 7f58e1400640 1 --2- 172.21.15.83:0/3271463313 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58e41044b0 0x7f58e41a07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:48.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.920+0000 7f58e1400640 1 -- 172.21.15.83:0/3271463313 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58e419b480 con 0x7f58e4109580 2024-04-24T09:05:48.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.920+0000 7f58e1e00640 1 --2- 172.21.15.83:0/3271463313 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58e41044b0 0x7f58e41a07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:05:48.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.921+0000 7f58e1400640 1 --2- 172.21.15.83:0/3271463313 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f58e4109580 0x7f58e41a0cf0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f58d400e9a0 tx=0x7f58d400ee60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:48.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.921+0000 7f58daa00640 1 -- 172.21.15.83:0/3271463313 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d400cd80 con 0x7f58e4109580 2024-04-24T09:05:48.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.921+0000 7f58daa00640 1 -- 172.21.15.83:0/3271463313 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f58d40035f0 con 0x7f58e4109580 2024-04-24T09:05:48.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.921+0000 7f58daa00640 1 -- 172.21.15.83:0/3271463313 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d4010690 con 0x7f58e4109580 2024-04-24T09:05:48.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.921+0000 7f58e8e00640 1 -- 172.21.15.83:0/3271463313 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58e419b770 con 0x7f58e4109580 2024-04-24T09:05:48.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.922+0000 7f58e8e00640 1 -- 172.21.15.83:0/3271463313 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58e406bd20 con 0x7f58e4109580 2024-04-24T09:05:48.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.926+0000 7f58ebc17740 1 Processor -- start 2024-04-24T09:05:48.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.927+0000 7f58ebc17740 1 -- start start 2024-04-24T09:05:48.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.927+0000 7f58daa00640 1 -- 172.21.15.83:0/3271463313 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f58d40037a0 con 0x7f58e4109580 2024-04-24T09:05:48.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.927+0000 7f58ebc17740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3d68b3950 0x55c3d68b3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:48.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.927+0000 7f58ebc17740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3d68b4270 0x55c3d68be400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:48.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.927+0000 7f58ebc17740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c3d68be940 0x55c3d68c0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:48.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.927+0000 7f58ebc17740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c3d68191d0 con 0x55c3d68b4270 2024-04-24T09:05:48.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.927+0000 7f58ebc17740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c3d67bf450 con 0x55c3d68b3950 2024-04-24T09:05:48.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.927+0000 7f58ebc17740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55c3d6815190 con 0x55c3d68be940 2024-04-24T09:05:48.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.928+0000 7f58e2800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3d68b3950 0x55c3d68b3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:48.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.928+0000 7f58e2800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3d68b3950 0x55c3d68b3d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:34800/0 (socket says 172.21.15.83:34800) 2024-04-24T09:05:48.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.928+0000 7f58e2800640 1 -- 172.21.15.83:0/2677546761 learned_addr learned my addr 172.21.15.83:0/2677546761 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:05:48.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.928+0000 7f58e1e00640 1 --2- 172.21.15.83:0/2677546761 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3d68b4270 0x55c3d68be400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:48.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.928+0000 7f58e2800640 1 -- 172.21.15.83:0/2677546761 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c3d68be940 msgr2=0x55c3d68c0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:48.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.928+0000 7f58e2800640 1 --2- 172.21.15.83:0/2677546761 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c3d68be940 0x55c3d68c0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:48.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.928+0000 7f58e2800640 1 -- 172.21.15.83:0/2677546761 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3d68b4270 msgr2=0x55c3d68be400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:48.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.928+0000 7f58e2800640 1 --2- 172.21.15.83:0/2677546761 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3d68b4270 0x55c3d68be400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:48.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.928+0000 7f58e2800640 1 -- 172.21.15.83:0/2677546761 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3d687c310 con 0x55c3d68b3950 2024-04-24T09:05:48.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.929+0000 7f58e2800640 1 --2- 172.21.15.83:0/2677546761 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3d68b3950 0x55c3d68b3d30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f58dc009d00 tx=0x7f58dc00b420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce01f8578ea9f398 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:48.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.929+0000 7f58daa00640 1 --2- 172.21.15.83:0/3271463313 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f58c80a3da0 0x7f58c80a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:48.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.929+0000 7f58cf400640 1 -- 172.21.15.83:0/2677546761 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58dc00bd40 con 0x55c3d68b3950 2024-04-24T09:05:48.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.929+0000 7f58cf400640 1 -- 172.21.15.83:0/2677546761 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f58dc017070 con 0x55c3d68b3950 2024-04-24T09:05:48.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.929+0000 7f58cf400640 1 -- 172.21.15.83:0/2677546761 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58dc0115f0 con 0x55c3d68b3950 2024-04-24T09:05:48.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.929+0000 7f58daa00640 1 -- 172.21.15.83:0/3271463313 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7f58d40d0f50 con 0x7f58e4109580 2024-04-24T09:05:48.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.929+0000 7f58e1e00640 1 --2- 172.21.15.83:0/3271463313 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f58c80a3da0 0x7f58c80a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:48.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.930+0000 7f58ebc17740 1 -- 172.21.15.83:0/2677546761 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3d68b3950 msgr2=0x55c3d68b3d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:48.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.930+0000 7f58ebc17740 1 --2- 172.21.15.83:0/2677546761 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3d68b3950 0x55c3d68b3d30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f58dc009d00 tx=0x7f58dc00b420 comp rx=0 tx=0).stop 2024-04-24T09:05:48.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.930+0000 7f58ebc17740 1 -- 172.21.15.83:0/2677546761 shutdown_connections 2024-04-24T09:05:48.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.930+0000 7f58ebc17740 1 --2- 172.21.15.83:0/2677546761 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c3d68be940 0x55c3d68c0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:48.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.930+0000 7f58ebc17740 1 --2- 172.21.15.83:0/2677546761 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3d68b4270 0x55c3d68be400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:48.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.930+0000 7f58ebc17740 1 --2- 172.21.15.83:0/2677546761 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3d68b3950 0x55c3d68b3d30 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:48.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.930+0000 7f58ebc17740 1 -- 172.21.15.83:0/2677546761 >> 172.21.15.83:0/2677546761 conn(0x55c3d68afc70 msgr2=0x55c3d68b0050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:48.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.931+0000 7f58ebc17740 1 -- 172.21.15.83:0/2677546761 shutdown_connections 2024-04-24T09:05:48.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.931+0000 7f58ebc17740 1 -- 172.21.15.83:0/2677546761 wait complete. 2024-04-24T09:05:48.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.931+0000 7f58ebc17740 1 -- start start 2024-04-24T09:05:48.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.931+0000 7f58ebc17740 1 Processor -- start 2024-04-24T09:05:48.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.931+0000 7f58e1e00640 1 --2- 172.21.15.83:0/3271463313 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f58c80a3da0 0x7f58c80a6260 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f58d0009780 tx=0x7f58d0005ab0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:48.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.932+0000 7f58ebc17740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3d68b3950 0x55c3d68bb780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:48.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.932+0000 7f58e1e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3d68b3950 0x55c3d68bb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:48.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.932+0000 7f58ebc17740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c3d68b4270 0x55c3d68bbcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:48.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.932+0000 7f58ebc17740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3d68be940 0x55c3d68bc200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:48.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.933+0000 7f58e1e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3d68b3950 0x55c3d68bb780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43132/0 (socket says 172.21.15.83:43132) 2024-04-24T09:05:48.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.933+0000 7f58e2800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3d68be940 0x55c3d68bc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:48.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.933+0000 7f58e2800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3d68be940 0x55c3d68bc200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:34812/0 (socket says 172.21.15.83:34812) 2024-04-24T09:05:48.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.933+0000 7f58e2800640 1 -- 172.21.15.83:0/3658437679 learned_addr learned my addr 172.21.15.83:0/3658437679 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:05:48.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.933+0000 7f58ebc17740 1 -- 172.21.15.83:0/3658437679 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c3d68c17d0 con 0x55c3d68b3950 2024-04-24T09:05:48.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.933+0000 7f58e1400640 1 --2- 172.21.15.83:0/3658437679 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c3d68b4270 0x55c3d68bbcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:48.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.933+0000 7f58ebc17740 1 -- 172.21.15.83:0/3658437679 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c3d67d2810 con 0x55c3d68be940 2024-04-24T09:05:48.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.933+0000 7f58ebc17740 1 -- 172.21.15.83:0/3658437679 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55c3d68c1950 con 0x55c3d68b4270 2024-04-24T09:05:48.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.934+0000 7f58ebc17740 10 client.0 _ll_register_callbacks cb 0x55c3d67a0140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:05:48.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.934+0000 7f58e2800640 1 -- 172.21.15.83:0/3658437679 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c3d68b4270 msgr2=0x55c3d68bbcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:48.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.934+0000 7f58e2800640 1 --2- 172.21.15.83:0/3658437679 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c3d68b4270 0x55c3d68bbcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:48.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.934+0000 7f58e2800640 1 -- 172.21.15.83:0/3658437679 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3d68b3950 msgr2=0x55c3d68bb780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:48.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.934+0000 7f58e2800640 1 --2- 172.21.15.83:0/3658437679 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3d68b3950 0x55c3d68bb780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:48.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.935+0000 7f58e2800640 1 -- 172.21.15.83:0/3658437679 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3d67e33f0 con 0x55c3d68be940 2024-04-24T09:05:48.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.935+0000 7f58e1e00640 1 --2- 172.21.15.83:0/3658437679 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3d68b3950 0x55c3d68bb780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:05:48.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.935+0000 7f58ebc17740 1 -- 172.21.15.83:0/3658437679 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58dc017070 con 0x55c3d68be940 2024-04-24T09:05:48.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.935+0000 7f58e2800640 1 --2- 172.21.15.83:0/3658437679 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3d68be940 0x55c3d68bc200 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f58dc012f00 tx=0x7f58dc009830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c4f43c9fe9bbe25 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:48.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.935+0000 7f58c3e00640 10 client.37796 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:05:48.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.936+0000 7f58c3e00640 1 -- 172.21.15.83:0/3658437679 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58dc021070 con 0x55c3d68be940 2024-04-24T09:05:48.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.936+0000 7f58c3e00640 1 -- 172.21.15.83:0/3658437679 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7f58dc01b3f0 con 0x55c3d68be940 2024-04-24T09:05:48.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.936+0000 7f58c3e00640 1 -- 172.21.15.83:0/3658437679 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f58dc01b630 con 0x55c3d68be940 2024-04-24T09:05:48.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.936+0000 7f58ebc17740 10 client.37796 fetch_fsmap learned FSMap version 47 2024-04-24T09:05:48.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.936+0000 7f58ebc17740 1 -- 172.21.15.83:0/3658437679 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({fsmap=47}) -- 0x55c3d652b3f0 con 0x55c3d68be940 2024-04-24T09:05:48.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.937+0000 7f58c3e00640 1 -- 172.21.15.83:0/3658437679 <== mon.2 v2:172.21.15.138:3300/0 4 ==== fsmap(e 47) ==== 8398+0+0 (secure 0 0 0) 0x7f58dc011070 con 0x55c3d68be940 2024-04-24T09:05:48.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.937+0000 7f58ebc17740 10 client.37796 fetch_fsmap finished waiting for FSMap version 47 2024-04-24T09:05:48.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.937+0000 7f58ebc17740 10 client.37796 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-24T09:05:48.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.937+0000 7f58ebc17740 20 client.37796 populate_metadata read hostname 'smithi083' 2024-04-24T09:05:48.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.937+0000 7f58ebc17740 1 --2- 172.21.15.83:0/3658437679 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55c3d68d71b0 0x55c3d68d95a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:48.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.937+0000 7f58ebc17740 4 client.37796 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:05:48.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.937+0000 7f58ebc17740 1 -- 172.21.15.83:0/3658437679 --> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c3d6826bf0 con 0x55c3d68d71b0 2024-04-24T09:05:48.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.938+0000 7f58e1e00640 1 --2- 172.21.15.83:0/3658437679 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55c3d68d71b0 0x55c3d68d95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:48.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.939+0000 7f58e1e00640 1 --2- 172.21.15.83:0/3658437679 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55c3d68d71b0 0x55c3d68d95a0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8b4c72dc62e263ab server_cookie=9b7271c87f52f5b in_seq=0 out_seq=0 2024-04-24T09:05:48.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.939+0000 7f58c3e00640 10 client.37796 ms_handle_connect on v2:172.21.15.138:6834/1909219758 2024-04-24T09:05:48.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.941+0000 7f58c3e00640 1 -- 172.21.15.83:0/3658437679 <== mds.2 v2:172.21.15.138:6834/1909219758 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c3d6826bf0 con 0x55c3d68d71b0 2024-04-24T09:05:48.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.941+0000 7f58c3e00640 10 client.37796 handle_command_reply: tid=0 2024-04-24T09:05:48.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.942+0000 7f58ebc17740 1 client.37796 shutdown 2024-04-24T09:05:48.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.942+0000 7f58ebc17740 1 -- 172.21.15.83:0/3658437679 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3d68be940 msgr2=0x55c3d68bc200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:48.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.942+0000 7f58ebc17740 1 --2- 172.21.15.83:0/3658437679 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3d68be940 0x55c3d68bc200 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f58dc012f00 tx=0x7f58dc009830 comp rx=0 tx=0).stop 2024-04-24T09:05:48.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.942+0000 7f58ebc17740 1 -- 172.21.15.83:0/3658437679 shutdown_connections 2024-04-24T09:05:48.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.942+0000 7f58ebc17740 1 --2- 172.21.15.83:0/3658437679 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55c3d68d71b0 0x55c3d68d95a0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:48.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.942+0000 7f58ebc17740 1 --2- 172.21.15.83:0/3658437679 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3d68be940 0x55c3d68bc200 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:48.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.942+0000 7f58ebc17740 1 --2- 172.21.15.83:0/3658437679 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c3d68b4270 0x55c3d68bbcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:48.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.942+0000 7f58ebc17740 1 --2- 172.21.15.83:0/3658437679 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3d68b3950 0x55c3d68bb780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:48.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.942+0000 7f58ebc17740 1 -- 172.21.15.83:0/3658437679 >> 172.21.15.83:0/3658437679 conn(0x55c3d68afc70 msgr2=0x55c3d68b5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:48.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.943+0000 7f58c3e00640 0 client.37796 ms_handle_reset on v2:172.21.15.138:6834/1909219758 2024-04-24T09:05:48.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.943+0000 7f58ebc17740 1 -- 172.21.15.83:0/3658437679 shutdown_connections 2024-04-24T09:05:48.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.943+0000 7f58ebc17740 1 -- 172.21.15.83:0/3658437679 wait complete. 2024-04-24T09:05:48.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.943+0000 7f58ebc17740 20 client.37796 trim_cache size 0 max 16384 2024-04-24T09:05:48.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.964+0000 7f58ebc17740 1 Processor -- start 2024-04-24T09:05:48.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.964+0000 7f58ebc17740 1 -- start start 2024-04-24T09:05:48.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.964+0000 7f58ebc17740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3d68b3950 0x55c3d68b61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:48.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.964+0000 7f58ebc17740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3d68b4270 0x55c3d68b9f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:48.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.964+0000 7f58ebc17740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c3d68be940 0x55c3d68bc020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:48.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.964+0000 7f58ebc17740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f58e4071910 con 0x55c3d68b4270 2024-04-24T09:05:48.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.964+0000 7f58ebc17740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f58e40735d0 con 0x55c3d68b3950 2024-04-24T09:05:48.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.964+0000 7f58ebc17740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f58e4071fe0 con 0x55c3d68be940 2024-04-24T09:05:48.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.964+0000 7f58e1400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3d68b4270 0x55c3d68b9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:48.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.964+0000 7f58e1400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3d68b4270 0x55c3d68b9f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43144/0 (socket says 172.21.15.83:43144) 2024-04-24T09:05:48.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.964+0000 7f58e1400640 1 -- 172.21.15.83:0/3993430534 learned_addr learned my addr 172.21.15.83:0/3993430534 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:05:48.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.965+0000 7f58e1e00640 1 --2- 172.21.15.83:0/3993430534 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3d68b3950 0x55c3d68b61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:48.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.965+0000 7f58e2800640 1 --2- 172.21.15.83:0/3993430534 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c3d68be940 0x55c3d68bc020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:48.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.965+0000 7f58e1400640 1 -- 172.21.15.83:0/3993430534 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c3d68be940 msgr2=0x55c3d68bc020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:48.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.965+0000 7f58e1400640 1 --2- 172.21.15.83:0/3993430534 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c3d68be940 0x55c3d68bc020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:48.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.965+0000 7f58e1400640 1 -- 172.21.15.83:0/3993430534 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3d68b3950 msgr2=0x55c3d68b61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:48.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.965+0000 7f58e1400640 1 --2- 172.21.15.83:0/3993430534 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3d68b3950 0x55c3d68b61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:48.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.965+0000 7f58e1400640 1 -- 172.21.15.83:0/3993430534 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3d65f57d0 con 0x55c3d68b4270 2024-04-24T09:05:48.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.965+0000 7f58e1400640 1 --2- 172.21.15.83:0/3993430534 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3d68b4270 0x55c3d68b9f20 secure :-1 s=READY pgs=2148 cs=0 l=1 rev1=1 crypto rx=0x7f58d40ce2e0 tx=0x7f58d40d1270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2af7123159bb462e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:48.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.966+0000 7f58cea00640 1 -- 172.21.15.83:0/3993430534 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d409d1d0 con 0x55c3d68b4270 2024-04-24T09:05:48.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.966+0000 7f58cea00640 1 -- 172.21.15.83:0/3993430534 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f58d409cad0 con 0x55c3d68b4270 2024-04-24T09:05:48.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.966+0000 7f58cea00640 1 -- 172.21.15.83:0/3993430534 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d40ce500 con 0x55c3d68b4270 2024-04-24T09:05:48.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.967+0000 7f58ebc17740 1 -- 172.21.15.83:0/3993430534 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3d68b4270 msgr2=0x55c3d68b9f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:48.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.967+0000 7f58ebc17740 1 --2- 172.21.15.83:0/3993430534 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3d68b4270 0x55c3d68b9f20 secure :-1 s=READY pgs=2148 cs=0 l=1 rev1=1 crypto rx=0x7f58d40ce2e0 tx=0x7f58d40d1270 comp rx=0 tx=0).stop 2024-04-24T09:05:48.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.967+0000 7f58ebc17740 1 -- 172.21.15.83:0/3993430534 shutdown_connections 2024-04-24T09:05:48.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.967+0000 7f58ebc17740 1 --2- 172.21.15.83:0/3993430534 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c3d68be940 0x55c3d68bc020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:48.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.967+0000 7f58ebc17740 1 --2- 172.21.15.83:0/3993430534 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3d68b4270 0x55c3d68b9f20 unknown :-1 s=CLOSED pgs=2148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:48.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.967+0000 7f58ebc17740 1 --2- 172.21.15.83:0/3993430534 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3d68b3950 0x55c3d68b61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:48.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.967+0000 7f58ebc17740 1 -- 172.21.15.83:0/3993430534 >> 172.21.15.83:0/3993430534 conn(0x55c3d68afc70 msgr2=0x55c3d68aedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:48.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.967+0000 7f58ebc17740 1 -- 172.21.15.83:0/3993430534 shutdown_connections 2024-04-24T09:05:48.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.967+0000 7f58ebc17740 1 -- 172.21.15.83:0/3993430534 wait complete. 2024-04-24T09:05:48.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.967+0000 7f58ebc17740 1 -- start start 2024-04-24T09:05:48.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.968+0000 7f58ebc17740 1 Processor -- start 2024-04-24T09:05:48.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.968+0000 7f58ebc17740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3d68b3950 0x55c3d68cf430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:48.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.968+0000 7f58ebc17740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3d68b4270 0x55c3d68cf970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:48.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.968+0000 7f58ebc17740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c3d68be940 0x55c3d68b7de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:48.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.968+0000 7f58ebc17740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f58e4072e40 con 0x55c3d68b4270 2024-04-24T09:05:48.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.968+0000 7f58ebc17740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f58e4072710 con 0x55c3d68b3950 2024-04-24T09:05:48.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.968+0000 7f58ebc17740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f58e406f910 con 0x55c3d68be940 2024-04-24T09:05:48.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.968+0000 7f58ebc17740 10 client.0 _ll_register_callbacks cb 0x7f58d0035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:05:48.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.968+0000 7f58e1400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3d68b4270 0x55c3d68cf970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:48.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.968+0000 7f58e1e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3d68b3950 0x55c3d68cf430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:48.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.968+0000 7f58e1e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3d68b3950 0x55c3d68cf430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:34828/0 (socket says 172.21.15.83:34828) 2024-04-24T09:05:48.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.969+0000 7f58e1e00640 1 -- 172.21.15.83:0/2184670204 learned_addr learned my addr 172.21.15.83:0/2184670204 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:05:48.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.969+0000 7f58e2800640 1 --2- 172.21.15.83:0/2184670204 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c3d68be940 0x55c3d68b7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:48.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.969+0000 7f58e1400640 1 -- 172.21.15.83:0/2184670204 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c3d68be940 msgr2=0x55c3d68b7de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:48.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.969+0000 7f58e1400640 1 --2- 172.21.15.83:0/2184670204 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c3d68be940 0x55c3d68b7de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:48.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.969+0000 7f58e1400640 1 -- 172.21.15.83:0/2184670204 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3d68b3950 msgr2=0x55c3d68cf430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:48.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.969+0000 7f58e1400640 1 --2- 172.21.15.83:0/2184670204 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3d68b3950 0x55c3d68cf430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:48.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.969+0000 7f58e1400640 1 -- 172.21.15.83:0/2184670204 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3d68b8610 con 0x55c3d68b4270 2024-04-24T09:05:48.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.969+0000 7f58e1400640 1 --2- 172.21.15.83:0/2184670204 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3d68b4270 0x55c3d68cf970 secure :-1 s=READY pgs=2149 cs=0 l=1 rev1=1 crypto rx=0x55c3d68d43d0 tx=0x7f58d409c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f31c916c64b05a5f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:48.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.969+0000 7f58c3e00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:05:48.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.970+0000 7f58c3e00640 1 -- 172.21.15.83:0/2184670204 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d40bd0b0 con 0x55c3d68b4270 2024-04-24T09:05:48.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.970+0000 7f58c3e00640 1 -- 172.21.15.83:0/2184670204 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7f58d4099140 con 0x55c3d68b4270 2024-04-24T09:05:48.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.970+0000 7f58ebc17740 1 -- 172.21.15.83:0/2184670204 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58d409cad0 con 0x55c3d68b4270 2024-04-24T09:05:48.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.970+0000 7f58c3e00640 1 -- 172.21.15.83:0/2184670204 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f58d409d1d0 con 0x55c3d68b4270 2024-04-24T09:05:48.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.970+0000 7f58ebc17740 10 client.20652 fetch_fsmap learned FSMap version 47 2024-04-24T09:05:48.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.970+0000 7f58ebc17740 1 -- 172.21.15.83:0/2184670204 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55c3d693d100 con 0x55c3d68b4270 2024-04-24T09:05:48.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.970+0000 7f58c3e00640 1 -- 172.21.15.83:0/2184670204 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f58d409cad0 con 0x55c3d68b4270 2024-04-24T09:05:48.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.971+0000 7f58c3e00640 1 -- 172.21.15.83:0/2184670204 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 47) ==== 8398+0+0 (secure 0 0 0) 0x7f58d40b8070 con 0x55c3d68b4270 2024-04-24T09:05:48.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.971+0000 7f58ebc17740 10 client.20652 fetch_fsmap finished waiting for FSMap version 47 2024-04-24T09:05:48.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.971+0000 7f58ebc17740 10 client.20652 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-24T09:05:48.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.971+0000 7f58ebc17740 20 client.20652 populate_metadata read hostname 'smithi083' 2024-04-24T09:05:48.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.971+0000 7f58ebc17740 1 --2- 172.21.15.83:0/2184670204 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55c3d693d780 0x55c3d693fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:48.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.971+0000 7f58ebc17740 4 client.20652 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:05:48.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.971+0000 7f58ebc17740 1 -- 172.21.15.83:0/2184670204 --> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c3d6820a60 con 0x55c3d693d780 2024-04-24T09:05:48.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.971+0000 7f58e1e00640 1 --2- 172.21.15.83:0/2184670204 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55c3d693d780 0x55c3d693fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:48.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.972+0000 7f58e1e00640 1 --2- 172.21.15.83:0/2184670204 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55c3d693d780 0x55c3d693fb70 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fd4cadc6f51ae67b server_cookie=e799f1dc39473d4b in_seq=0 out_seq=0 2024-04-24T09:05:48.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.972+0000 7f58c3e00640 10 client.20652 ms_handle_connect on v2:172.21.15.138:6834/1909219758 2024-04-24T09:05:48.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.973+0000 7f58c3e00640 1 -- 172.21.15.83:0/2184670204 <== mds.2 v2:172.21.15.138:6834/1909219758 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c3d6820a60 con 0x55c3d693d780 2024-04-24T09:05:48.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.973+0000 7f58c3e00640 10 client.20652 handle_command_reply: tid=0 2024-04-24T09:05:48.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.973+0000 7f58ebc17740 1 client.20652 shutdown 2024-04-24T09:05:48.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.973+0000 7f58ebc17740 1 -- 172.21.15.83:0/2184670204 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3d68b4270 msgr2=0x55c3d68cf970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:48.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.973+0000 7f58ebc17740 1 --2- 172.21.15.83:0/2184670204 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3d68b4270 0x55c3d68cf970 secure :-1 s=READY pgs=2149 cs=0 l=1 rev1=1 crypto rx=0x55c3d68d43d0 tx=0x7f58d409c530 comp rx=0 tx=0).stop 2024-04-24T09:05:48.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.973+0000 7f58ebc17740 1 -- 172.21.15.83:0/2184670204 shutdown_connections 2024-04-24T09:05:48.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.973+0000 7f58ebc17740 1 --2- 172.21.15.83:0/2184670204 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55c3d693d780 0x55c3d693fb70 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:48.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.973+0000 7f58ebc17740 1 --2- 172.21.15.83:0/2184670204 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c3d68be940 0x55c3d68b7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:48.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.973+0000 7f58c3e00640 0 client.20652 ms_handle_reset on v2:172.21.15.138:6834/1909219758 2024-04-24T09:05:48.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.973+0000 7f58ebc17740 1 --2- 172.21.15.83:0/2184670204 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3d68b4270 0x55c3d68cf970 unknown :-1 s=CLOSED pgs=2149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:48.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.973+0000 7f58ebc17740 1 --2- 172.21.15.83:0/2184670204 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3d68b3950 0x55c3d68cf430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:48.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.973+0000 7f58ebc17740 1 -- 172.21.15.83:0/2184670204 >> 172.21.15.83:0/2184670204 conn(0x55c3d68afc70 msgr2=0x55c3d68bcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:48.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.974+0000 7f58ebc17740 1 -- 172.21.15.83:0/2184670204 shutdown_connections 2024-04-24T09:05:48.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.974+0000 7f58ebc17740 1 -- 172.21.15.83:0/2184670204 wait complete. 2024-04-24T09:05:48.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.974+0000 7f58ebc17740 20 client.20652 trim_cache size 0 max 16384 2024-04-24T09:05:48.975 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:05:48.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.975+0000 7f58e8e00640 1 -- 172.21.15.83:0/3271463313 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f58c80a3da0 msgr2=0x7f58c80a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:48.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.975+0000 7f58e8e00640 1 --2- 172.21.15.83:0/3271463313 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f58c80a3da0 0x7f58c80a6260 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f58d0009780 tx=0x7f58d0005ab0 comp rx=0 tx=0).stop 2024-04-24T09:05:48.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.975+0000 7f58e8e00640 1 -- 172.21.15.83:0/3271463313 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f58e4109580 msgr2=0x7f58e41a0cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:48.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.975+0000 7f58e8e00640 1 --2- 172.21.15.83:0/3271463313 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f58e4109580 0x7f58e41a0cf0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f58d400e9a0 tx=0x7f58d400ee60 comp rx=0 tx=0).stop 2024-04-24T09:05:48.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.975+0000 7f58e8e00640 1 -- 172.21.15.83:0/3271463313 shutdown_connections 2024-04-24T09:05:48.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.975+0000 7f58e8e00640 1 --2- 172.21.15.83:0/3271463313 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f58c80a3da0 0x7f58c80a6260 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:48.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.975+0000 7f58e8e00640 1 --2- 172.21.15.83:0/3271463313 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f58e419a960 0x7f58e419ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:48.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.975+0000 7f58e8e00640 1 --2- 172.21.15.83:0/3271463313 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f58e4109580 0x7f58e41a0cf0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:48.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.975+0000 7f58e8e00640 1 --2- 172.21.15.83:0/3271463313 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58e41044b0 0x7f58e41a07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:48.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.975+0000 7f58e8e00640 1 -- 172.21.15.83:0/3271463313 >> 172.21.15.83:0/3271463313 conn(0x7f58e40ffd90 msgr2=0x7f58e4101200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:48.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.975+0000 7f58e8e00640 1 -- 172.21.15.83:0/3271463313 shutdown_connections 2024-04-24T09:05:48.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:48.975+0000 7f58e8e00640 1 -- 172.21.15.83:0/3271463313 wait complete. 2024-04-24T09:05:48.995 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T09:05:49.154 INFO:tasks.workunit.client.0.smithi083.stdout: 10 2484 14.46 MB/sec warmup 20 sec latency 86.466 ms 2024-04-24T09:05:49.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.256+0000 7f7de6400640 1 -- 172.21.15.83:0/133773041 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7de0104e60 msgr2=0x7f7de0108ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:49.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.256+0000 7f7de6400640 1 --2- 172.21.15.83:0/133773041 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7de0104e60 0x7f7de0108ea0 secure :-1 s=READY pgs=2150 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc009900 tx=0x7f7dcc02a400 comp rx=0 tx=0).stop 2024-04-24T09:05:49.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.256+0000 7f7de6400640 1 -- 172.21.15.83:0/133773041 shutdown_connections 2024-04-24T09:05:49.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.256+0000 7f7de6400640 1 --2- 172.21.15.83:0/133773041 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7de0109580 0x7f7de0111150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:49.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.256+0000 7f7de6400640 1 --2- 172.21.15.83:0/133773041 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7de0104e60 0x7f7de0108ea0 unknown :-1 s=CLOSED pgs=2150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:49.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.256+0000 7f7de6400640 1 --2- 172.21.15.83:0/133773041 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7de01044b0 0x7f7de0104890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:49.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.256+0000 7f7de6400640 1 -- 172.21.15.83:0/133773041 >> 172.21.15.83:0/133773041 conn(0x7f7de00ffd90 msgr2=0x7f7de01021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:49.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.257+0000 7f7de6400640 1 -- 172.21.15.83:0/133773041 shutdown_connections 2024-04-24T09:05:49.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.257+0000 7f7de6400640 1 -- 172.21.15.83:0/133773041 wait complete. 2024-04-24T09:05:49.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.258+0000 7f7de6400640 1 Processor -- start 2024-04-24T09:05:49.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.258+0000 7f7de6400640 1 -- start start 2024-04-24T09:05:49.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.258+0000 7f7de6400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7de01044b0 0x7f7de01128c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:49.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.258+0000 7f7de6400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7de0109580 0x7f7de0112e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:49.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.258+0000 7f7de6400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7de01134a0 0x7f7de0119940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:49.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.258+0000 7f7de6400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7de0078070 con 0x7f7de01044b0 2024-04-24T09:05:49.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.258+0000 7f7de6400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7de0077ef0 con 0x7f7de0109580 2024-04-24T09:05:49.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.258+0000 7f7de6400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7de00781f0 con 0x7f7de01134a0 2024-04-24T09:05:49.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.259+0000 7f7ddf400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7de01044b0 0x7f7de01128c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:49.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.259+0000 7f7ddf400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7de01044b0 0x7f7de01128c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43160/0 (socket says 172.21.15.83:43160) 2024-04-24T09:05:49.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.259+0000 7f7ddf400640 1 -- 172.21.15.83:0/4211274727 learned_addr learned my addr 172.21.15.83:0/4211274727 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:05:49.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.259+0000 7f7ddea00640 1 --2- 172.21.15.83:0/4211274727 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7de0109580 0x7f7de0112e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:49.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.259+0000 7f7ddfe00640 1 --2- 172.21.15.83:0/4211274727 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7de01134a0 0x7f7de0119940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:49.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.259+0000 7f7ddf400640 1 -- 172.21.15.83:0/4211274727 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7de01134a0 msgr2=0x7f7de0119940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:49.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.259+0000 7f7ddf400640 1 --2- 172.21.15.83:0/4211274727 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7de01134a0 0x7f7de0119940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:49.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.259+0000 7f7ddf400640 1 -- 172.21.15.83:0/4211274727 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7de0109580 msgr2=0x7f7de0112e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:49.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.259+0000 7f7ddf400640 1 --2- 172.21.15.83:0/4211274727 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7de0109580 0x7f7de0112e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:49.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.259+0000 7f7ddf400640 1 -- 172.21.15.83:0/4211274727 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7de011a200 con 0x7f7de01044b0 2024-04-24T09:05:49.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.260+0000 7f7ddf400640 1 --2- 172.21.15.83:0/4211274727 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7de01044b0 0x7f7de01128c0 secure :-1 s=READY pgs=2151 cs=0 l=1 rev1=1 crypto rx=0x7f7dd000e9f0 tx=0x7f7dd000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:49.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.260+0000 7f7dd7e00640 1 -- 172.21.15.83:0/4211274727 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd000cd80 con 0x7f7de01044b0 2024-04-24T09:05:49.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.260+0000 7f7dd7e00640 1 -- 172.21.15.83:0/4211274727 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7dd00108e0 con 0x7f7de01044b0 2024-04-24T09:05:49.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.260+0000 7f7dd7e00640 1 -- 172.21.15.83:0/4211274727 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd0018920 con 0x7f7de01044b0 2024-04-24T09:05:49.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.261+0000 7f7de6400640 1 -- 172.21.15.83:0/4211274727 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7de011a4f0 con 0x7f7de01044b0 2024-04-24T09:05:49.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.262+0000 7f7de6400640 1 -- 172.21.15.83:0/4211274727 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7de011a9c0 con 0x7f7de01044b0 2024-04-24T09:05:49.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.266+0000 7f7de919e740 1 Processor -- start 2024-04-24T09:05:49.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.266+0000 7f7de919e740 1 -- start start 2024-04-24T09:05:49.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.266+0000 7f7dd7e00640 1 -- 172.21.15.83:0/4211274727 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7dd0010a90 con 0x7f7de01044b0 2024-04-24T09:05:49.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.267+0000 7f7de919e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55da2c1e9950 0x55da2c1e9d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:49.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.267+0000 7f7de919e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da2c1ea270 0x55da2c1f4400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:49.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.267+0000 7f7de919e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da2c1f4940 0x55da2c1f6d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:49.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.267+0000 7f7de919e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55da2c14f1d0 con 0x55da2c1e9950 2024-04-24T09:05:49.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.267+0000 7f7de919e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55da2c0f5450 con 0x55da2c1ea270 2024-04-24T09:05:49.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.267+0000 7f7de919e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55da2c14b190 con 0x55da2c1f4940 2024-04-24T09:05:49.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.267+0000 7f7ddf400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da2c1ea270 0x55da2c1f4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:49.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.267+0000 7f7ddf400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da2c1ea270 0x55da2c1f4400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:34856/0 (socket says 172.21.15.83:34856) 2024-04-24T09:05:49.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.267+0000 7f7ddf400640 1 -- 172.21.15.83:0/2054278179 learned_addr learned my addr 172.21.15.83:0/2054278179 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:05:49.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.267+0000 7f7ddfe00640 1 --2- 172.21.15.83:0/2054278179 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55da2c1e9950 0x55da2c1e9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:49.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.268+0000 7f7ddf400640 1 -- 172.21.15.83:0/2054278179 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da2c1f4940 msgr2=0x55da2c1f6d30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-24T09:05:49.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.268+0000 7f7ddf400640 1 --2- 172.21.15.83:0/2054278179 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da2c1f4940 0x55da2c1f6d30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:49.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.268+0000 7f7ddf400640 1 -- 172.21.15.83:0/2054278179 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55da2c1e9950 msgr2=0x55da2c1e9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:49.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.268+0000 7f7ddf400640 1 --2- 172.21.15.83:0/2054278179 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55da2c1e9950 0x55da2c1e9d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:49.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.268+0000 7f7ddf400640 1 -- 172.21.15.83:0/2054278179 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da2c1b2310 con 0x55da2c1ea270 2024-04-24T09:05:49.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.268+0000 7f7dd7e00640 1 --2- 172.21.15.83:0/4211274727 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f7dc40a3d20 0x7f7dc40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:49.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.268+0000 7f7ddf400640 1 --2- 172.21.15.83:0/2054278179 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da2c1ea270 0x55da2c1f4400 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7dd00ce3d0 tx=0x7f7dd00d9910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69b9b8ee1952f37d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:49.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.269+0000 7f7dd7e00640 1 -- 172.21.15.83:0/4211274727 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7f7dd00d1130 con 0x7f7de01044b0 2024-04-24T09:05:49.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.269+0000 7f7ddfe00640 1 --2- 172.21.15.83:0/4211274727 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f7dc40a3d20 0x7f7dc40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:49.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.269+0000 7f7dd4c00640 1 -- 172.21.15.83:0/2054278179 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd0098f50 con 0x55da2c1ea270 2024-04-24T09:05:49.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.269+0000 7f7dd4c00640 1 -- 172.21.15.83:0/2054278179 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7dd0010de0 con 0x55da2c1ea270 2024-04-24T09:05:49.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.269+0000 7f7dd4c00640 1 -- 172.21.15.83:0/2054278179 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd0098f50 con 0x55da2c1ea270 2024-04-24T09:05:49.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.271+0000 7f7de919e740 1 -- 172.21.15.83:0/2054278179 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da2c1ea270 msgr2=0x55da2c1f4400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:49.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.271+0000 7f7ddfe00640 1 --2- 172.21.15.83:0/4211274727 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f7dc40a3d20 0x7f7dc40a61e0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f7de01024d0 tx=0x7f7dd80096e0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:49.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.271+0000 7f7de919e740 1 --2- 172.21.15.83:0/2054278179 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da2c1ea270 0x55da2c1f4400 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7dd00ce3d0 tx=0x7f7dd00d9910 comp rx=0 tx=0).stop 2024-04-24T09:05:49.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.271+0000 7f7de919e740 1 -- 172.21.15.83:0/2054278179 shutdown_connections 2024-04-24T09:05:49.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.271+0000 7f7de919e740 1 --2- 172.21.15.83:0/2054278179 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da2c1f4940 0x55da2c1f6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:49.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.271+0000 7f7de919e740 1 --2- 172.21.15.83:0/2054278179 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da2c1ea270 0x55da2c1f4400 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:49.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.271+0000 7f7de919e740 1 --2- 172.21.15.83:0/2054278179 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55da2c1e9950 0x55da2c1e9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:49.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.271+0000 7f7de919e740 1 -- 172.21.15.83:0/2054278179 >> 172.21.15.83:0/2054278179 conn(0x55da2c1e5c70 msgr2=0x55da2c1e6050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:49.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.271+0000 7f7de919e740 1 -- 172.21.15.83:0/2054278179 shutdown_connections 2024-04-24T09:05:49.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.271+0000 7f7de919e740 1 -- 172.21.15.83:0/2054278179 wait complete. 2024-04-24T09:05:49.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.272+0000 7f7de919e740 1 -- start start 2024-04-24T09:05:49.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.272+0000 7f7de919e740 1 Processor -- start 2024-04-24T09:05:49.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.272+0000 7f7de919e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da2c1e9950 0x55da2c1f4400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:49.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.273+0000 7f7de919e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da2c1ea270 0x55da2c1f16f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:49.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.273+0000 7f7de919e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55da2c1f4940 0x55da2c1f1e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:49.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.273+0000 7f7de919e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55da2c1f77d0 con 0x55da2c1f4940 2024-04-24T09:05:49.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.273+0000 7f7de919e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55da2c108810 con 0x55da2c1ea270 2024-04-24T09:05:49.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.273+0000 7f7de919e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55da2c1f7950 con 0x55da2c1e9950 2024-04-24T09:05:49.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.273+0000 7f7de919e740 10 client.0 _ll_register_callbacks cb 0x55da2c0d6140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:05:49.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.273+0000 7f7ddea00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da2c1ea270 0x55da2c1f16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:49.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.273+0000 7f7ddf400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da2c1e9950 0x55da2c1f4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:49.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.273+0000 7f7ddfe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55da2c1f4940 0x55da2c1f1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:49.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.273+0000 7f7ddf400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da2c1e9950 0x55da2c1f4400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:39382/0 (socket says 172.21.15.83:39382) 2024-04-24T09:05:49.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.273+0000 7f7ddf400640 1 -- 172.21.15.83:0/2168663427 learned_addr learned my addr 172.21.15.83:0/2168663427 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:05:49.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.274+0000 7f7ddfe00640 1 -- 172.21.15.83:0/2168663427 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da2c1e9950 msgr2=0x55da2c1f4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:49.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.274+0000 7f7ddfe00640 1 --2- 172.21.15.83:0/2168663427 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da2c1e9950 0x55da2c1f4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:49.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.274+0000 7f7ddfe00640 1 -- 172.21.15.83:0/2168663427 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da2c1ea270 msgr2=0x55da2c1f16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:49.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.274+0000 7f7ddfe00640 1 --2- 172.21.15.83:0/2168663427 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da2c1ea270 0x55da2c1f16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:49.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.274+0000 7f7ddfe00640 1 -- 172.21.15.83:0/2168663427 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da2c1193f0 con 0x55da2c1f4940 2024-04-24T09:05:49.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.274+0000 7f7ddea00640 1 --2- 172.21.15.83:0/2168663427 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da2c1ea270 0x55da2c1f16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:05:49.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.275+0000 7f7ddfe00640 1 --2- 172.21.15.83:0/2168663427 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55da2c1f4940 0x55da2c1f1e30 secure :-1 s=READY pgs=2152 cs=0 l=1 rev1=1 crypto rx=0x7f7dd8005fd0 tx=0x7f7dd8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48e0db41a13336c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:49.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.275+0000 7f7dc9600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:05:49.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.275+0000 7f7dc9600640 1 -- 172.21.15.83:0/2168663427 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd800eab0 con 0x55da2c1f4940 2024-04-24T09:05:49.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.276+0000 7f7de919e740 1 -- 172.21.15.83:0/2168663427 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55da2c15cbf0 con 0x55da2c1f4940 2024-04-24T09:05:49.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.276+0000 7f7dc9600640 1 -- 172.21.15.83:0/2168663427 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7f7dd800ec50 con 0x55da2c1f4940 2024-04-24T09:05:49.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.277+0000 7f7dc9600640 1 -- 172.21.15.83:0/2168663427 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f7dd800ee50 con 0x55da2c1f4940 2024-04-24T09:05:49.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.277+0000 7f7de919e740 10 client.20664 fetch_fsmap learned FSMap version 47 2024-04-24T09:05:49.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.277+0000 7f7de919e740 1 -- 172.21.15.83:0/2168663427 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55da2be613f0 con 0x55da2c1f4940 2024-04-24T09:05:49.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.277+0000 7f7dc9600640 1 -- 172.21.15.83:0/2168663427 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55da2c15cbf0 con 0x55da2c1f4940 2024-04-24T09:05:49.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.277+0000 7f7dc9600640 1 -- 172.21.15.83:0/2168663427 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 47) ==== 8398+0+0 (secure 0 0 0) 0x7f7dd800f040 con 0x55da2c1f4940 2024-04-24T09:05:49.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.278+0000 7f7de919e740 10 client.20664 fetch_fsmap finished waiting for FSMap version 47 2024-04-24T09:05:49.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.278+0000 7f7de919e740 10 client.20664 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-24T09:05:49.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.278+0000 7f7de919e740 20 client.20664 populate_metadata read hostname 'smithi083' 2024-04-24T09:05:49.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.278+0000 7f7de919e740 1 --2- 172.21.15.83:0/2168663427 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x55da2c204cc0 0x55da2c2070b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:49.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.278+0000 7f7de919e740 4 client.20664 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:05:49.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.278+0000 7f7de919e740 1 -- 172.21.15.83:0/2168663427 --> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55da2c156a60 con 0x55da2c204cc0 2024-04-24T09:05:49.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.278+0000 7f7ddf400640 1 --2- 172.21.15.83:0/2168663427 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x55da2c204cc0 0x55da2c2070b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:49.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.279+0000 7f7ddf400640 1 --2- 172.21.15.83:0/2168663427 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x55da2c204cc0 0x55da2c2070b0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9c2485c45304a402 server_cookie=339ba4e2905e4935 in_seq=0 out_seq=0 2024-04-24T09:05:49.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.279+0000 7f7dc9600640 10 client.20664 ms_handle_connect on v2:172.21.15.138:6836/900566754 2024-04-24T09:05:49.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.283+0000 7f7dc9600640 1 -- 172.21.15.83:0/2168663427 <== mds.1 v2:172.21.15.138:6836/900566754 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55da2c156a60 con 0x55da2c204cc0 2024-04-24T09:05:49.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.283+0000 7f7dc9600640 10 client.20664 handle_command_reply: tid=0 2024-04-24T09:05:49.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.283+0000 7f7de919e740 1 client.20664 shutdown 2024-04-24T09:05:49.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.284+0000 7f7de919e740 1 -- 172.21.15.83:0/2168663427 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55da2c1f4940 msgr2=0x55da2c1f1e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:49.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.284+0000 7f7de919e740 1 --2- 172.21.15.83:0/2168663427 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55da2c1f4940 0x55da2c1f1e30 secure :-1 s=READY pgs=2152 cs=0 l=1 rev1=1 crypto rx=0x7f7dd8005fd0 tx=0x7f7dd8004a40 comp rx=0 tx=0).stop 2024-04-24T09:05:49.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.284+0000 7f7de919e740 1 -- 172.21.15.83:0/2168663427 shutdown_connections 2024-04-24T09:05:49.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.284+0000 7f7de919e740 1 --2- 172.21.15.83:0/2168663427 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55da2c1f4940 0x55da2c1f1e30 unknown :-1 s=CLOSED pgs=2152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:49.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.284+0000 7f7de919e740 1 --2- 172.21.15.83:0/2168663427 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da2c1ea270 0x55da2c1f16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:49.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.284+0000 7f7de919e740 1 --2- 172.21.15.83:0/2168663427 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x55da2c204cc0 0x55da2c2070b0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:49.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.284+0000 7f7de919e740 1 --2- 172.21.15.83:0/2168663427 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da2c1e9950 0x55da2c1f4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:49.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.284+0000 7f7de919e740 1 -- 172.21.15.83:0/2168663427 >> 172.21.15.83:0/2168663427 conn(0x55da2c1e5c70 msgr2=0x55da2c1eba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:49.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.284+0000 7f7dc9600640 0 client.20664 ms_handle_reset on v2:172.21.15.138:6836/900566754 2024-04-24T09:05:49.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.284+0000 7f7de919e740 1 -- 172.21.15.83:0/2168663427 shutdown_connections 2024-04-24T09:05:49.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.284+0000 7f7de919e740 1 -- 172.21.15.83:0/2168663427 wait complete. 2024-04-24T09:05:49.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.284+0000 7f7de919e740 20 client.20664 trim_cache size 0 max 16384 2024-04-24T09:05:49.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.307+0000 7f7de919e740 1 Processor -- start 2024-04-24T09:05:49.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.307+0000 7f7de919e740 1 -- start start 2024-04-24T09:05:49.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.307+0000 7f7de919e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da2c1e9950 0x55da2c2006e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:49.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.307+0000 7f7de919e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55da2c1ea270 0x55da2c1f3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:49.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.307+0000 7f7de919e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da2c1f4940 0x55da2c20c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:49.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.307+0000 7f7de919e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7de01bea40 con 0x55da2c1ea270 2024-04-24T09:05:49.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.307+0000 7f7de919e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7de01c0700 con 0x55da2c1f4940 2024-04-24T09:05:49.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.307+0000 7f7de919e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7de01bf110 con 0x55da2c1e9950 2024-04-24T09:05:49.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.308+0000 7f7ddea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55da2c1ea270 0x55da2c1f3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:49.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.308+0000 7f7ddfe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da2c1f4940 0x55da2c20c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:49.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.308+0000 7f7ddea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55da2c1ea270 0x55da2c1f3f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43192/0 (socket says 172.21.15.83:43192) 2024-04-24T09:05:49.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.308+0000 7f7ddea00640 1 -- 172.21.15.83:0/336811533 learned_addr learned my addr 172.21.15.83:0/336811533 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:05:49.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.308+0000 7f7ddf400640 1 --2- 172.21.15.83:0/336811533 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da2c1e9950 0x55da2c2006e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:49.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.308+0000 7f7ddfe00640 1 -- 172.21.15.83:0/336811533 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da2c1e9950 msgr2=0x55da2c2006e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:49.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.308+0000 7f7ddfe00640 1 --2- 172.21.15.83:0/336811533 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da2c1e9950 0x55da2c2006e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:49.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.308+0000 7f7ddfe00640 1 -- 172.21.15.83:0/336811533 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55da2c1ea270 msgr2=0x55da2c1f3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:49.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.308+0000 7f7ddfe00640 1 --2- 172.21.15.83:0/336811533 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55da2c1ea270 0x55da2c1f3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:49.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.308+0000 7f7ddfe00640 1 -- 172.21.15.83:0/336811533 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da2bf2b7d0 con 0x55da2c1f4940 2024-04-24T09:05:49.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.309+0000 7f7ddfe00640 1 --2- 172.21.15.83:0/336811533 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da2c1f4940 0x55da2c20c9c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7dd801f710 tx=0x7f7dd8004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3fecdecc307bd17c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:49.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.309+0000 7f7dcbe00640 1 -- 172.21.15.83:0/336811533 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd800ec50 con 0x55da2c1f4940 2024-04-24T09:05:49.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.309+0000 7f7dcbe00640 1 -- 172.21.15.83:0/336811533 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7dd800edf0 con 0x55da2c1f4940 2024-04-24T09:05:49.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.309+0000 7f7dcbe00640 1 -- 172.21.15.83:0/336811533 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd8011240 con 0x55da2c1f4940 2024-04-24T09:05:49.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.310+0000 7f7de919e740 1 -- 172.21.15.83:0/336811533 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da2c1f4940 msgr2=0x55da2c20c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:49.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.310+0000 7f7de919e740 1 --2- 172.21.15.83:0/336811533 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da2c1f4940 0x55da2c20c9c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7dd801f710 tx=0x7f7dd8004ab0 comp rx=0 tx=0).stop 2024-04-24T09:05:49.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.310+0000 7f7de919e740 1 -- 172.21.15.83:0/336811533 shutdown_connections 2024-04-24T09:05:49.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.310+0000 7f7de919e740 1 --2- 172.21.15.83:0/336811533 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da2c1f4940 0x55da2c20c9c0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:49.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.310+0000 7f7de919e740 1 --2- 172.21.15.83:0/336811533 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55da2c1ea270 0x55da2c1f3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:49.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.310+0000 7f7de919e740 1 --2- 172.21.15.83:0/336811533 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da2c1e9950 0x55da2c2006e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:49.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.310+0000 7f7de919e740 1 -- 172.21.15.83:0/336811533 >> 172.21.15.83:0/336811533 conn(0x55da2c1e5c70 msgr2=0x55da2c1e6050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:49.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.310+0000 7f7de919e740 1 -- 172.21.15.83:0/336811533 shutdown_connections 2024-04-24T09:05:49.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.311+0000 7f7de919e740 1 -- 172.21.15.83:0/336811533 wait complete. 2024-04-24T09:05:49.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.311+0000 7f7de919e740 1 -- start start 2024-04-24T09:05:49.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.311+0000 7f7de919e740 1 Processor -- start 2024-04-24T09:05:49.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.313+0000 7f7de919e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55da2c1e9950 0x55da2c1fcbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:49.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.313+0000 7f7de919e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da2c1ea270 0x55da2c1fd130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:49.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.313+0000 7f7de919e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da2c1f4940 0x55da2c1ee990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:49.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.313+0000 7f7de919e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7de01bff70 con 0x55da2c1e9950 2024-04-24T09:05:49.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.313+0000 7f7de919e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7de01bf840 con 0x55da2c1ea270 2024-04-24T09:05:49.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.313+0000 7f7de919e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7de01bca40 con 0x55da2c1f4940 2024-04-24T09:05:49.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.313+0000 7f7de919e740 10 client.0 _ll_register_callbacks cb 0x7f7dd80104a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:05:49.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.315+0000 7f7ddf400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55da2c1e9950 0x55da2c1fcbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:49.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.315+0000 7f7ddf400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55da2c1e9950 0x55da2c1fcbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43208/0 (socket says 172.21.15.83:43208) 2024-04-24T09:05:49.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.315+0000 7f7ddf400640 1 -- 172.21.15.83:0/2385897653 learned_addr learned my addr 172.21.15.83:0/2385897653 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:05:49.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.315+0000 7f7ddea00640 1 --2- 172.21.15.83:0/2385897653 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da2c1ea270 0x55da2c1fd130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:49.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.316+0000 7f7ddfe00640 1 --2- 172.21.15.83:0/2385897653 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da2c1f4940 0x55da2c1ee990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:49.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.316+0000 7f7ddf400640 1 -- 172.21.15.83:0/2385897653 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da2c1f4940 msgr2=0x55da2c1ee990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:49.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.316+0000 7f7ddf400640 1 --2- 172.21.15.83:0/2385897653 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da2c1f4940 0x55da2c1ee990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:49.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.316+0000 7f7ddf400640 1 -- 172.21.15.83:0/2385897653 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da2c1ea270 msgr2=0x55da2c1fd130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:49.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.316+0000 7f7ddf400640 1 --2- 172.21.15.83:0/2385897653 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da2c1ea270 0x55da2c1fd130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:49.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.316+0000 7f7ddf400640 1 -- 172.21.15.83:0/2385897653 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da2c1ef020 con 0x55da2c1e9950 2024-04-24T09:05:49.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.316+0000 7f7ddf400640 1 --2- 172.21.15.83:0/2385897653 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55da2c1e9950 0x55da2c1fcbf0 secure :-1 s=READY pgs=2153 cs=0 l=1 rev1=1 crypto rx=0x7f7dd000c6a0 tx=0x7f7dd00d4f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ad8b3d766ad090c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:49.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.316+0000 7f7dc9600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:05:49.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.317+0000 7f7dc9600640 1 -- 172.21.15.83:0/2385897653 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd009c610 con 0x55da2c1e9950 2024-04-24T09:05:49.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.317+0000 7f7dc9600640 1 -- 172.21.15.83:0/2385897653 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7f7dd00ce6e0 con 0x55da2c1e9950 2024-04-24T09:05:49.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.317+0000 7f7de919e740 1 -- 172.21.15.83:0/2385897653 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7dd800edf0 con 0x55da2c1e9950 2024-04-24T09:05:49.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.317+0000 7f7dc9600640 1 -- 172.21.15.83:0/2385897653 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f7dd009c7f0 con 0x55da2c1e9950 2024-04-24T09:05:49.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.317+0000 7f7de919e740 10 client.20670 fetch_fsmap learned FSMap version 47 2024-04-24T09:05:49.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.317+0000 7f7de919e740 1 -- 172.21.15.83:0/2385897653 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55da2c1f5780 con 0x55da2c1e9950 2024-04-24T09:05:49.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.318+0000 7f7dc9600640 1 -- 172.21.15.83:0/2385897653 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7dd800edf0 con 0x55da2c1e9950 2024-04-24T09:05:49.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.318+0000 7f7dc9600640 1 -- 172.21.15.83:0/2385897653 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 47) ==== 8398+0+0 (secure 0 0 0) 0x7f7dd00bd0b0 con 0x55da2c1e9950 2024-04-24T09:05:49.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.318+0000 7f7de919e740 10 client.20670 fetch_fsmap finished waiting for FSMap version 47 2024-04-24T09:05:49.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.318+0000 7f7de919e740 10 client.20670 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-24T09:05:49.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.318+0000 7f7de919e740 20 client.20670 populate_metadata read hostname 'smithi083' 2024-04-24T09:05:49.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.318+0000 7f7de919e740 1 --2- 172.21.15.83:0/2385897653 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x55da2c26b230 0x55da2c26d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:49.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.318+0000 7f7de919e740 4 client.20670 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:05:49.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.318+0000 7f7de919e740 1 -- 172.21.15.83:0/2385897653 --> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55da2c0d24d0 con 0x55da2c26b230 2024-04-24T09:05:49.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.319+0000 7f7ddea00640 1 --2- 172.21.15.83:0/2385897653 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x55da2c26b230 0x55da2c26d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:49.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.319+0000 7f7dc9600640 10 client.20670 ms_handle_connect on v2:172.21.15.138:6836/900566754 2024-04-24T09:05:49.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.319+0000 7f7ddea00640 1 --2- 172.21.15.83:0/2385897653 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x55da2c26b230 0x55da2c26d620 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a43b3f78ca9db7fb server_cookie=e756e8b1d835d73d in_seq=0 out_seq=0 2024-04-24T09:05:49.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.320+0000 7f7dc9600640 1 -- 172.21.15.83:0/2385897653 <== mds.1 v2:172.21.15.138:6836/900566754 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55da2c0d24d0 con 0x55da2c26b230 2024-04-24T09:05:49.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.320+0000 7f7dc9600640 10 client.20670 handle_command_reply: tid=0 2024-04-24T09:05:49.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.320+0000 7f7de919e740 1 client.20670 shutdown 2024-04-24T09:05:49.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.320+0000 7f7de919e740 1 -- 172.21.15.83:0/2385897653 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55da2c1e9950 msgr2=0x55da2c1fcbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:49.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.320+0000 7f7de919e740 1 --2- 172.21.15.83:0/2385897653 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55da2c1e9950 0x55da2c1fcbf0 secure :-1 s=READY pgs=2153 cs=0 l=1 rev1=1 crypto rx=0x7f7dd000c6a0 tx=0x7f7dd00d4f50 comp rx=0 tx=0).stop 2024-04-24T09:05:49.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.321+0000 7f7de919e740 1 -- 172.21.15.83:0/2385897653 shutdown_connections 2024-04-24T09:05:49.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.321+0000 7f7de919e740 1 --2- 172.21.15.83:0/2385897653 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x55da2c26b230 0x55da2c26d620 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:49.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.321+0000 7f7de919e740 1 --2- 172.21.15.83:0/2385897653 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da2c1f4940 0x55da2c1ee990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:49.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.321+0000 7f7de919e740 1 --2- 172.21.15.83:0/2385897653 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da2c1ea270 0x55da2c1fd130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:49.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.321+0000 7f7dc9600640 0 client.20670 ms_handle_reset on v2:172.21.15.138:6836/900566754 2024-04-24T09:05:49.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.321+0000 7f7de919e740 1 --2- 172.21.15.83:0/2385897653 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55da2c1e9950 0x55da2c1fcbf0 unknown :-1 s=CLOSED pgs=2153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:49.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.321+0000 7f7de919e740 1 -- 172.21.15.83:0/2385897653 >> 172.21.15.83:0/2385897653 conn(0x55da2c1e5c70 msgr2=0x55da2c1effa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:49.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.321+0000 7f7de919e740 1 -- 172.21.15.83:0/2385897653 shutdown_connections 2024-04-24T09:05:49.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.322+0000 7f7de919e740 1 -- 172.21.15.83:0/2385897653 wait complete. 2024-04-24T09:05:49.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.322+0000 7f7de919e740 20 client.20670 trim_cache size 0 max 16384 2024-04-24T09:05:49.321 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:05:49.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.323+0000 7f7de6400640 1 -- 172.21.15.83:0/4211274727 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f7dc40a3d20 msgr2=0x7f7dc40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:49.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.323+0000 7f7de6400640 1 --2- 172.21.15.83:0/4211274727 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f7dc40a3d20 0x7f7dc40a61e0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f7de01024d0 tx=0x7f7dd80096e0 comp rx=0 tx=0).stop 2024-04-24T09:05:49.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.323+0000 7f7de6400640 1 -- 172.21.15.83:0/4211274727 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7de01044b0 msgr2=0x7f7de01128c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:49.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.323+0000 7f7de6400640 1 --2- 172.21.15.83:0/4211274727 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7de01044b0 0x7f7de01128c0 secure :-1 s=READY pgs=2151 cs=0 l=1 rev1=1 crypto rx=0x7f7dd000e9f0 tx=0x7f7dd000eeb0 comp rx=0 tx=0).stop 2024-04-24T09:05:49.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.324+0000 7f7de6400640 1 -- 172.21.15.83:0/4211274727 shutdown_connections 2024-04-24T09:05:49.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.324+0000 7f7de6400640 1 --2- 172.21.15.83:0/4211274727 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f7dc40a3d20 0x7f7dc40a61e0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:49.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.324+0000 7f7de6400640 1 --2- 172.21.15.83:0/4211274727 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7de01134a0 0x7f7de0119940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:49.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.324+0000 7f7de6400640 1 --2- 172.21.15.83:0/4211274727 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7de0109580 0x7f7de0112e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:49.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.324+0000 7f7de6400640 1 --2- 172.21.15.83:0/4211274727 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7de01044b0 0x7f7de01128c0 unknown :-1 s=CLOSED pgs=2151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:49.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.324+0000 7f7de6400640 1 -- 172.21.15.83:0/4211274727 >> 172.21.15.83:0/4211274727 conn(0x7f7de00ffd90 msgr2=0x7f7de0078ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:49.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.324+0000 7f7de6400640 1 -- 172.21.15.83:0/4211274727 shutdown_connections 2024-04-24T09:05:49.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.324+0000 7f7de6400640 1 -- 172.21.15.83:0/4211274727 wait complete. 2024-04-24T09:05:49.343 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T09:05:49.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.600+0000 7ff757200640 1 -- 172.21.15.83:0/768659952 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff758109920 msgr2=0x7ff758114c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:49.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.600+0000 7ff757200640 1 --2- 172.21.15.83:0/768659952 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff758109920 0x7ff758114c10 secure :-1 s=READY pgs=2154 cs=0 l=1 rev1=1 crypto rx=0x7ff740009900 tx=0x7ff74002a400 comp rx=0 tx=0).stop 2024-04-24T09:05:49.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.600+0000 7ff757200640 1 -- 172.21.15.83:0/768659952 shutdown_connections 2024-04-24T09:05:49.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.600+0000 7ff757200640 1 --2- 172.21.15.83:0/768659952 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff758115150 0x7ff758117540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:49.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.600+0000 7ff757200640 1 --2- 172.21.15.83:0/768659952 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff758109920 0x7ff758114c10 unknown :-1 s=CLOSED pgs=2154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:49.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.600+0000 7ff757200640 1 --2- 172.21.15.83:0/768659952 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff758109000 0x7ff7581093e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:49.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.600+0000 7ff757200640 1 -- 172.21.15.83:0/768659952 >> 172.21.15.83:0/768659952 conn(0x7ff75806cf80 msgr2=0x7ff75806d390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:49.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.600+0000 7ff757200640 1 -- 172.21.15.83:0/768659952 shutdown_connections 2024-04-24T09:05:49.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.600+0000 7ff757200640 1 -- 172.21.15.83:0/768659952 wait complete. 2024-04-24T09:05:49.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.601+0000 7ff757200640 1 Processor -- start 2024-04-24T09:05:49.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.602+0000 7ff757200640 1 -- start start 2024-04-24T09:05:49.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.602+0000 7ff757200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff758109000 0x7ff75810aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:49.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.602+0000 7ff757200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff758109920 0x7ff75810afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:49.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.602+0000 7ff757200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff758115150 0x7ff75810b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:49.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.602+0000 7ff757200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff75811a970 con 0x7ff758109000 2024-04-24T09:05:49.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.602+0000 7ff757200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff75811a7f0 con 0x7ff758109920 2024-04-24T09:05:49.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.602+0000 7ff757200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff75811aaf0 con 0x7ff758115150 2024-04-24T09:05:49.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.602+0000 7ff74fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff758109920 0x7ff75810afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:49.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.602+0000 7ff74fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff758109920 0x7ff75810afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:34892/0 (socket says 172.21.15.83:34892) 2024-04-24T09:05:49.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.602+0000 7ff74fe00640 1 -- 172.21.15.83:0/2385648756 learned_addr learned my addr 172.21.15.83:0/2385648756 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:05:49.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.602+0000 7ff755e00640 1 --2- 172.21.15.83:0/2385648756 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff758109000 0x7ff75810aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:49.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.603+0000 7ff756800640 1 --2- 172.21.15.83:0/2385648756 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff758115150 0x7ff75810b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:49.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.603+0000 7ff74fe00640 1 -- 172.21.15.83:0/2385648756 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff758115150 msgr2=0x7ff75810b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:49.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.603+0000 7ff74fe00640 1 --2- 172.21.15.83:0/2385648756 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff758115150 0x7ff75810b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:49.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.603+0000 7ff74fe00640 1 -- 172.21.15.83:0/2385648756 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff758109000 msgr2=0x7ff75810aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:49.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.603+0000 7ff74fe00640 1 --2- 172.21.15.83:0/2385648756 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff758109000 0x7ff75810aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:49.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.603+0000 7ff74fe00640 1 -- 172.21.15.83:0/2385648756 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7581af390 con 0x7ff758109920 2024-04-24T09:05:49.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.604+0000 7ff74fe00640 1 --2- 172.21.15.83:0/2385648756 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff758109920 0x7ff75810afb0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff740005ab0 tx=0x7ff740003c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:49.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.604+0000 7ff74ea00640 1 -- 172.21.15.83:0/2385648756 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff74003f070 con 0x7ff758109920 2024-04-24T09:05:49.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.604+0000 7ff74ea00640 1 -- 172.21.15.83:0/2385648756 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff740031d60 con 0x7ff758109920 2024-04-24T09:05:49.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.604+0000 7ff74ea00640 1 -- 172.21.15.83:0/2385648756 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff740035030 con 0x7ff758109920 2024-04-24T09:05:49.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.605+0000 7ff757200640 1 -- 172.21.15.83:0/2385648756 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7581af620 con 0x7ff758109920 2024-04-24T09:05:49.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.605+0000 7ff757200640 1 -- 172.21.15.83:0/2385648756 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7581afac0 con 0x7ff758109920 2024-04-24T09:05:49.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.608+0000 7ff75f7e5740 1 Processor -- start 2024-04-24T09:05:49.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.609+0000 7ff75f7e5740 1 -- start start 2024-04-24T09:05:49.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.609+0000 7ff75f7e5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d74ecf1890 0x55d74ecf1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:49.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.609+0000 7ff75f7e5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d74ecf2240 0x55d74ed04e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:49.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.609+0000 7ff74ea00640 1 -- 172.21.15.83:0/2385648756 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff740042070 con 0x7ff758109920 2024-04-24T09:05:49.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.609+0000 7ff75f7e5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d74ed05340 0x55d74ed07730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:49.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.609+0000 7ff75f7e5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d74ec571d0 con 0x55d74ecf1890 2024-04-24T09:05:49.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.609+0000 7ff75f7e5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d74ebfd450 con 0x55d74ecf2240 2024-04-24T09:05:49.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.609+0000 7ff75f7e5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d74ec53190 con 0x55d74ed05340 2024-04-24T09:05:49.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.610+0000 7ff756800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d74ecf1890 0x55d74ecf1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:49.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.610+0000 7ff756800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d74ecf1890 0x55d74ecf1c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43230/0 (socket says 172.21.15.83:43230) 2024-04-24T09:05:49.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.610+0000 7ff756800640 1 -- 172.21.15.83:0/3719591293 learned_addr learned my addr 172.21.15.83:0/3719591293 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:05:49.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.610+0000 7ff755e00640 1 --2- 172.21.15.83:0/3719591293 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d74ecf2240 0x55d74ed04e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:49.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.610+0000 7ff74fe00640 1 --2- 172.21.15.83:0/3719591293 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d74ed05340 0x55d74ed07730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:49.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.610+0000 7ff756800640 1 -- 172.21.15.83:0/3719591293 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d74ed05340 msgr2=0x55d74ed07730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:49.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.610+0000 7ff756800640 1 --2- 172.21.15.83:0/3719591293 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d74ed05340 0x55d74ed07730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:49.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.610+0000 7ff756800640 1 -- 172.21.15.83:0/3719591293 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d74ecf2240 msgr2=0x55d74ed04e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:49.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.610+0000 7ff756800640 1 --2- 172.21.15.83:0/3719591293 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d74ecf2240 0x55d74ed04e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:49.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.610+0000 7ff756800640 1 -- 172.21.15.83:0/3719591293 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d74ecba310 con 0x55d74ecf1890 2024-04-24T09:05:49.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.611+0000 7ff755e00640 1 --2- 172.21.15.83:0/3719591293 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d74ecf2240 0x55d74ed04e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:05:49.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.611+0000 7ff756800640 1 --2- 172.21.15.83:0/3719591293 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d74ecf1890 0x55d74ecf1c70 secure :-1 s=READY pgs=2155 cs=0 l=1 rev1=1 crypto rx=0x7ff750009e00 tx=0x7ff750010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=711daf586350b55b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:49.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.611+0000 7ff747000640 1 -- 172.21.15.83:0/3719591293 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff750016070 con 0x55d74ecf1890 2024-04-24T09:05:49.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.611+0000 7ff747000640 1 -- 172.21.15.83:0/3719591293 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff750004d20 con 0x55d74ecf1890 2024-04-24T09:05:49.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.611+0000 7ff747000640 1 -- 172.21.15.83:0/3719591293 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff750012690 con 0x55d74ecf1890 2024-04-24T09:05:49.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.611+0000 7ff74ea00640 1 --2- 172.21.15.83:0/2385648756 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff73c0a3dd0 0x7ff73c0a6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:49.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.612+0000 7ff74ea00640 1 -- 172.21.15.83:0/2385648756 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7ff7400f0f50 con 0x7ff758109920 2024-04-24T09:05:49.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.612+0000 7ff755e00640 1 --2- 172.21.15.83:0/2385648756 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff73c0a3dd0 0x7ff73c0a6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:49.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.612+0000 7ff75f7e5740 1 -- 172.21.15.83:0/3719591293 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d74ecf1890 msgr2=0x55d74ecf1c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:49.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.612+0000 7ff75f7e5740 1 --2- 172.21.15.83:0/3719591293 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d74ecf1890 0x55d74ecf1c70 secure :-1 s=READY pgs=2155 cs=0 l=1 rev1=1 crypto rx=0x7ff750009e00 tx=0x7ff750010740 comp rx=0 tx=0).stop 2024-04-24T09:05:49.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.613+0000 7ff75f7e5740 1 -- 172.21.15.83:0/3719591293 shutdown_connections 2024-04-24T09:05:49.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.613+0000 7ff75f7e5740 1 --2- 172.21.15.83:0/3719591293 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d74ed05340 0x55d74ed07730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:49.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.613+0000 7ff75f7e5740 1 --2- 172.21.15.83:0/3719591293 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d74ecf2240 0x55d74ed04e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:49.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.613+0000 7ff75f7e5740 1 --2- 172.21.15.83:0/3719591293 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d74ecf1890 0x55d74ecf1c70 unknown :-1 s=CLOSED pgs=2155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:49.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.613+0000 7ff75f7e5740 1 -- 172.21.15.83:0/3719591293 >> 172.21.15.83:0/3719591293 conn(0x55d74ecede60 msgr2=0x55d74ecee240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:49.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.613+0000 7ff755e00640 1 --2- 172.21.15.83:0/2385648756 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff73c0a3dd0 0x7ff73c0a6290 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7ff748009db0 tx=0x7ff748005f90 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:49.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.613+0000 7ff75f7e5740 1 -- 172.21.15.83:0/3719591293 shutdown_connections 2024-04-24T09:05:49.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.613+0000 7ff75f7e5740 1 -- 172.21.15.83:0/3719591293 wait complete. 2024-04-24T09:05:49.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.614+0000 7ff75f7e5740 1 -- start start 2024-04-24T09:05:49.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.614+0000 7ff75f7e5740 1 Processor -- start 2024-04-24T09:05:49.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.615+0000 7ff75f7e5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d74ecf1890 0x55d74ed02110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:49.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.615+0000 7ff75f7e5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d74ecf2240 0x55d74ed02650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:49.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.616+0000 7ff75f7e5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d74ed05340 0x55d74ed02b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:49.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.616+0000 7ff75f7e5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d74ed081a0 con 0x55d74ecf1890 2024-04-24T09:05:49.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.616+0000 7ff75f7e5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d74ec10810 con 0x55d74ecf2240 2024-04-24T09:05:49.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.616+0000 7ff75f7e5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d74ed08320 con 0x55d74ed05340 2024-04-24T09:05:49.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.616+0000 7ff755e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d74ecf1890 0x55d74ed02110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:49.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.616+0000 7ff755e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d74ecf1890 0x55d74ed02110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43236/0 (socket says 172.21.15.83:43236) 2024-04-24T09:05:49.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.616+0000 7ff755e00640 1 -- 172.21.15.83:0/1771785726 learned_addr learned my addr 172.21.15.83:0/1771785726 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:05:49.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.616+0000 7ff75f7e5740 10 client.0 _ll_register_callbacks cb 0x55d74ebde140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:05:49.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.616+0000 7ff74fe00640 1 --2- 172.21.15.83:0/1771785726 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d74ecf2240 0x55d74ed02650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:49.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.616+0000 7ff756800640 1 --2- 172.21.15.83:0/1771785726 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d74ed05340 0x55d74ed02b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:49.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.616+0000 7ff755e00640 1 -- 172.21.15.83:0/1771785726 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d74ed05340 msgr2=0x55d74ed02b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:49.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.616+0000 7ff755e00640 1 --2- 172.21.15.83:0/1771785726 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d74ed05340 0x55d74ed02b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:49.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.616+0000 7ff755e00640 1 -- 172.21.15.83:0/1771785726 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d74ecf2240 msgr2=0x55d74ed02650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:49.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.616+0000 7ff755e00640 1 --2- 172.21.15.83:0/1771785726 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d74ecf2240 0x55d74ed02650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:49.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.617+0000 7ff755e00640 1 -- 172.21.15.83:0/1771785726 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d74ec213f0 con 0x55d74ecf1890 2024-04-24T09:05:49.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.617+0000 7ff756800640 1 --2- 172.21.15.83:0/1771785726 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d74ed05340 0x55d74ed02b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:05:49.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.617+0000 7ff755e00640 1 --2- 172.21.15.83:0/1771785726 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d74ecf1890 0x55d74ed02110 secure :-1 s=READY pgs=2156 cs=0 l=1 rev1=1 crypto rx=0x7ff748005f60 tx=0x7ff748004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e578f71bbb6e3df7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:49.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.617+0000 7ff73b400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:05:49.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.618+0000 7ff73b400640 1 -- 172.21.15.83:0/1771785726 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff74802e060 con 0x55d74ecf1890 2024-04-24T09:05:49.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.618+0000 7ff73b400640 1 -- 172.21.15.83:0/1771785726 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7ff748018030 con 0x55d74ecf1890 2024-04-24T09:05:49.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.618+0000 7ff75f7e5740 1 -- 172.21.15.83:0/1771785726 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff750004d20 con 0x55d74ecf1890 2024-04-24T09:05:49.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.619+0000 7ff73b400640 1 -- 172.21.15.83:0/1771785726 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7ff748035050 con 0x55d74ecf1890 2024-04-24T09:05:49.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.619+0000 7ff75f7e5740 10 client.20682 fetch_fsmap learned FSMap version 47 2024-04-24T09:05:49.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.619+0000 7ff75f7e5740 1 -- 172.21.15.83:0/1771785726 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55d74e9693f0 con 0x55d74ecf1890 2024-04-24T09:05:49.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.619+0000 7ff73b400640 1 -- 172.21.15.83:0/1771785726 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff750004d20 con 0x55d74ecf1890 2024-04-24T09:05:49.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.620+0000 7ff73b400640 1 -- 172.21.15.83:0/1771785726 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 47) ==== 8398+0+0 (secure 0 0 0) 0x7ff74801c3a0 con 0x55d74ecf1890 2024-04-24T09:05:49.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.621+0000 7ff75f7e5740 10 client.20682 fetch_fsmap finished waiting for FSMap version 47 2024-04-24T09:05:49.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.621+0000 7ff75f7e5740 10 client.20682 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:05:49.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.621+0000 7ff75f7e5740 20 client.20682 populate_metadata read hostname 'smithi083' 2024-04-24T09:05:49.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.621+0000 7ff75f7e5740 1 --2- 172.21.15.83:0/1771785726 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d74ed156a0 0x55d74ed17b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:49.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.621+0000 7ff75f7e5740 4 client.20682 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:05:49.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.621+0000 7ff75f7e5740 1 -- 172.21.15.83:0/1771785726 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d74ec64bf0 con 0x55d74ed156a0 2024-04-24T09:05:49.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.622+0000 7ff74fe00640 1 --2- 172.21.15.83:0/1771785726 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d74ed156a0 0x55d74ed17b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:49.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.623+0000 7ff74fe00640 1 --2- 172.21.15.83:0/1771785726 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d74ed156a0 0x55d74ed17b60 crc :-1 s=READY pgs=365 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7518d0218be87fd server_cookie=e7943dd3cc80d950 in_seq=0 out_seq=0 2024-04-24T09:05:49.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.623+0000 7ff73b400640 10 client.20682 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:05:49.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.625+0000 7ff73b400640 1 -- 172.21.15.83:0/1771785726 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d74ec64bf0 con 0x55d74ed156a0 2024-04-24T09:05:49.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.625+0000 7ff73b400640 10 client.20682 handle_command_reply: tid=0 2024-04-24T09:05:49.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.625+0000 7ff75f7e5740 1 client.20682 shutdown 2024-04-24T09:05:49.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.626+0000 7ff75f7e5740 1 -- 172.21.15.83:0/1771785726 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d74ecf1890 msgr2=0x55d74ed02110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:49.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.626+0000 7ff75f7e5740 1 --2- 172.21.15.83:0/1771785726 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d74ecf1890 0x55d74ed02110 secure :-1 s=READY pgs=2156 cs=0 l=1 rev1=1 crypto rx=0x7ff748005f60 tx=0x7ff748004730 comp rx=0 tx=0).stop 2024-04-24T09:05:49.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.626+0000 7ff75f7e5740 1 -- 172.21.15.83:0/1771785726 shutdown_connections 2024-04-24T09:05:49.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.626+0000 7ff75f7e5740 1 --2- 172.21.15.83:0/1771785726 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d74ed156a0 0x55d74ed17b60 crc :-1 s=READY pgs=365 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:49.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.626+0000 7ff75f7e5740 1 --2- 172.21.15.83:0/1771785726 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d74ed05340 0x55d74ed02b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:49.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.626+0000 7ff75f7e5740 1 --2- 172.21.15.83:0/1771785726 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d74ecf2240 0x55d74ed02650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:49.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.626+0000 7ff75f7e5740 1 --2- 172.21.15.83:0/1771785726 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d74ecf1890 0x55d74ed02110 unknown :-1 s=CLOSED pgs=2156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:49.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.626+0000 7ff75f7e5740 1 -- 172.21.15.83:0/1771785726 >> 172.21.15.83:0/1771785726 conn(0x55d74ecede60 msgr2=0x55d74ecf3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:49.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.626+0000 7ff73b400640 0 client.20682 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:05:49.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.627+0000 7ff75f7e5740 1 -- 172.21.15.83:0/1771785726 shutdown_connections 2024-04-24T09:05:49.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.627+0000 7ff75f7e5740 1 -- 172.21.15.83:0/1771785726 wait complete. 2024-04-24T09:05:49.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.627+0000 7ff75f7e5740 20 client.20682 trim_cache size 0 max 16384 2024-04-24T09:05:49.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.651+0000 7ff75f7e5740 1 Processor -- start 2024-04-24T09:05:49.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.651+0000 7ff75f7e5740 1 -- start start 2024-04-24T09:05:49.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.653+0000 7ff75f7e5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d74ecf1890 0x55d74ed110a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:49.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.653+0000 7ff75f7e5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d74ecf2240 0x55d74ed1d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:49.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.653+0000 7ff75f7e5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d74ed05340 0x55d74ed1d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:49.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.653+0000 7ff75f7e5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff7581b5a20 con 0x55d74ed05340 2024-04-24T09:05:49.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.653+0000 7ff75f7e5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff7581b76e0 con 0x55d74ecf2240 2024-04-24T09:05:49.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.653+0000 7ff75f7e5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff7581b60f0 con 0x55d74ecf1890 2024-04-24T09:05:49.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.654+0000 7ff74fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d74ecf2240 0x55d74ed1d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:49.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.654+0000 7ff756800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d74ed05340 0x55d74ed1d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:49.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.654+0000 7ff74fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d74ecf2240 0x55d74ed1d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:34920/0 (socket says 172.21.15.83:34920) 2024-04-24T09:05:49.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.654+0000 7ff74fe00640 1 -- 172.21.15.83:0/3862759675 learned_addr learned my addr 172.21.15.83:0/3862759675 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:05:49.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.654+0000 7ff755e00640 1 --2- 172.21.15.83:0/3862759675 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d74ecf1890 0x55d74ed110a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:49.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.655+0000 7ff74fe00640 1 -- 172.21.15.83:0/3862759675 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d74ecf1890 msgr2=0x55d74ed110a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:49.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.655+0000 7ff74fe00640 1 --2- 172.21.15.83:0/3862759675 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d74ecf1890 0x55d74ed110a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:49.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.655+0000 7ff74fe00640 1 -- 172.21.15.83:0/3862759675 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d74ed05340 msgr2=0x55d74ed1d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:49.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.655+0000 7ff74fe00640 1 --2- 172.21.15.83:0/3862759675 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d74ed05340 0x55d74ed1d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:49.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.655+0000 7ff74fe00640 1 -- 172.21.15.83:0/3862759675 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d74ea337d0 con 0x55d74ecf2240 2024-04-24T09:05:49.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.655+0000 7ff756800640 1 --2- 172.21.15.83:0/3862759675 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d74ed05340 0x55d74ed1d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:05:49.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.656+0000 7ff74fe00640 1 --2- 172.21.15.83:0/3862759675 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d74ecf2240 0x55d74ed1d470 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff740005ec0 tx=0x7ff7400dd040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa9d65a2dc848651 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:49.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.657+0000 7ff746600640 1 -- 172.21.15.83:0/3862759675 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff740049040 con 0x55d74ecf2240 2024-04-24T09:05:49.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.657+0000 7ff746600640 1 -- 172.21.15.83:0/3862759675 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff7400dd0b0 con 0x55d74ecf2240 2024-04-24T09:05:49.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.657+0000 7ff746600640 1 -- 172.21.15.83:0/3862759675 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff74003b580 con 0x55d74ecf2240 2024-04-24T09:05:49.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.658+0000 7ff75f7e5740 1 -- 172.21.15.83:0/3862759675 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d74ecf2240 msgr2=0x55d74ed1d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:49.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.659+0000 7ff75f7e5740 1 --2- 172.21.15.83:0/3862759675 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d74ecf2240 0x55d74ed1d470 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff740005ec0 tx=0x7ff7400dd040 comp rx=0 tx=0).stop 2024-04-24T09:05:49.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.659+0000 7ff75f7e5740 1 -- 172.21.15.83:0/3862759675 shutdown_connections 2024-04-24T09:05:49.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.659+0000 7ff75f7e5740 1 --2- 172.21.15.83:0/3862759675 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d74ed05340 0x55d74ed1d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:49.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.659+0000 7ff75f7e5740 1 --2- 172.21.15.83:0/3862759675 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d74ecf2240 0x55d74ed1d470 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:49.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.659+0000 7ff75f7e5740 1 --2- 172.21.15.83:0/3862759675 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d74ecf1890 0x55d74ed110a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:49.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.659+0000 7ff75f7e5740 1 -- 172.21.15.83:0/3862759675 >> 172.21.15.83:0/3862759675 conn(0x55d74ecede60 msgr2=0x55d74ecec6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:49.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.659+0000 7ff75f7e5740 1 -- 172.21.15.83:0/3862759675 shutdown_connections 2024-04-24T09:05:49.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.659+0000 7ff75f7e5740 1 -- 172.21.15.83:0/3862759675 wait complete. 2024-04-24T09:05:49.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.660+0000 7ff75f7e5740 1 -- start start 2024-04-24T09:05:49.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.660+0000 7ff75f7e5740 1 Processor -- start 2024-04-24T09:05:49.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.661+0000 7ff75f7e5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d74ecf1890 0x55d74ecff770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:49.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.662+0000 7ff75f7e5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d74ecf2240 0x55d74ecffcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:49.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.662+0000 7ff75f7e5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d74ed05340 0x55d74ed06320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:49.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.662+0000 7ff755e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d74ecf1890 0x55d74ecff770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:49.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.662+0000 7ff755e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d74ecf1890 0x55d74ecff770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43264/0 (socket says 172.21.15.83:43264) 2024-04-24T09:05:49.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.662+0000 7ff755e00640 1 -- 172.21.15.83:0/952777873 learned_addr learned my addr 172.21.15.83:0/952777873 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:05:49.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.662+0000 7ff75f7e5740 1 -- 172.21.15.83:0/952777873 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff7581b6f50 con 0x55d74ecf1890 2024-04-24T09:05:49.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.662+0000 7ff75f7e5740 1 -- 172.21.15.83:0/952777873 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff7581b6820 con 0x55d74ed05340 2024-04-24T09:05:49.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.662+0000 7ff75f7e5740 1 -- 172.21.15.83:0/952777873 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff7581b3a00 con 0x55d74ecf2240 2024-04-24T09:05:49.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.662+0000 7ff75f7e5740 10 client.0 _ll_register_callbacks cb 0x55d74ed05290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:05:49.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.662+0000 7ff756800640 1 --2- 172.21.15.83:0/952777873 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d74ed05340 0x55d74ed06320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:49.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.663+0000 7ff74fe00640 1 --2- 172.21.15.83:0/952777873 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d74ecf2240 0x55d74ecffcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:49.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.663+0000 7ff755e00640 1 -- 172.21.15.83:0/952777873 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d74ecf2240 msgr2=0x55d74ecffcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:49.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.663+0000 7ff755e00640 1 --2- 172.21.15.83:0/952777873 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d74ecf2240 0x55d74ecffcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:49.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.663+0000 7ff755e00640 1 -- 172.21.15.83:0/952777873 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d74ed05340 msgr2=0x55d74ed06320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:49.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.663+0000 7ff755e00640 1 --2- 172.21.15.83:0/952777873 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d74ed05340 0x55d74ed06320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:49.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.663+0000 7ff755e00640 1 -- 172.21.15.83:0/952777873 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d74ed0de80 con 0x55d74ecf1890 2024-04-24T09:05:49.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.663+0000 7ff756800640 1 --2- 172.21.15.83:0/952777873 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d74ed05340 0x55d74ed06320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:05:49.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.663+0000 7ff755e00640 1 --2- 172.21.15.83:0/952777873 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d74ecf1890 0x55d74ecff770 secure :-1 s=READY pgs=2157 cs=0 l=1 rev1=1 crypto rx=0x7ff74801e950 tx=0x7ff748009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca3bfb07a6cec699 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:49.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.663+0000 7ff73b400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:05:49.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.663+0000 7ff73b400640 1 -- 172.21.15.83:0/952777873 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff748035050 con 0x55d74ecf1890 2024-04-24T09:05:49.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.663+0000 7ff73b400640 1 -- 172.21.15.83:0/952777873 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7ff748024040 con 0x55d74ecf1890 2024-04-24T09:05:49.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.664+0000 7ff75f7e5740 1 -- 172.21.15.83:0/952777873 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7400dd0b0 con 0x55d74ecf1890 2024-04-24T09:05:49.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.664+0000 7ff73b400640 1 -- 172.21.15.83:0/952777873 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7ff748045050 con 0x55d74ecf1890 2024-04-24T09:05:49.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.664+0000 7ff75f7e5740 10 client.20688 fetch_fsmap learned FSMap version 47 2024-04-24T09:05:49.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.664+0000 7ff75f7e5740 1 -- 172.21.15.83:0/952777873 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55d74ed7c250 con 0x55d74ecf1890 2024-04-24T09:05:49.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.665+0000 7ff73b400640 1 -- 172.21.15.83:0/952777873 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff7400dd0b0 con 0x55d74ecf1890 2024-04-24T09:05:49.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.665+0000 7ff73b400640 1 -- 172.21.15.83:0/952777873 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 47) ==== 8398+0+0 (secure 0 0 0) 0x7ff748039a40 con 0x55d74ecf1890 2024-04-24T09:05:49.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.665+0000 7ff75f7e5740 10 client.20688 fetch_fsmap finished waiting for FSMap version 47 2024-04-24T09:05:49.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.665+0000 7ff75f7e5740 10 client.20688 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:05:49.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.665+0000 7ff75f7e5740 20 client.20688 populate_metadata read hostname 'smithi083' 2024-04-24T09:05:49.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.665+0000 7ff75f7e5740 1 --2- 172.21.15.83:0/952777873 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d74ed7c9b0 0x55d74ed7edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:49.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.665+0000 7ff75f7e5740 4 client.20688 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:05:49.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.665+0000 7ff75f7e5740 1 -- 172.21.15.83:0/952777873 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d74ec5ea60 con 0x55d74ed7c9b0 2024-04-24T09:05:49.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.666+0000 7ff74fe00640 1 --2- 172.21.15.83:0/952777873 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d74ed7c9b0 0x55d74ed7edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:49.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.666+0000 7ff74fe00640 1 --2- 172.21.15.83:0/952777873 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d74ed7c9b0 0x55d74ed7edd0 crc :-1 s=READY pgs=366 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b033e3817c7539b server_cookie=35848e364d17b973 in_seq=0 out_seq=0 2024-04-24T09:05:49.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.666+0000 7ff73b400640 10 client.20688 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:05:49.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.667+0000 7ff73b400640 1 -- 172.21.15.83:0/952777873 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d74ec5ea60 con 0x55d74ed7c9b0 2024-04-24T09:05:49.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.667+0000 7ff73b400640 10 client.20688 handle_command_reply: tid=0 2024-04-24T09:05:49.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.667+0000 7ff75f7e5740 1 client.20688 shutdown 2024-04-24T09:05:49.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.667+0000 7ff75f7e5740 1 -- 172.21.15.83:0/952777873 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d74ecf1890 msgr2=0x55d74ecff770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:49.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.667+0000 7ff75f7e5740 1 --2- 172.21.15.83:0/952777873 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d74ecf1890 0x55d74ecff770 secure :-1 s=READY pgs=2157 cs=0 l=1 rev1=1 crypto rx=0x7ff74801e950 tx=0x7ff748009c30 comp rx=0 tx=0).stop 2024-04-24T09:05:49.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.667+0000 7ff75f7e5740 1 -- 172.21.15.83:0/952777873 shutdown_connections 2024-04-24T09:05:49.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.667+0000 7ff75f7e5740 1 --2- 172.21.15.83:0/952777873 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d74ed7c9b0 0x55d74ed7edd0 crc :-1 s=READY pgs=366 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:49.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.667+0000 7ff75f7e5740 1 --2- 172.21.15.83:0/952777873 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d74ed05340 0x55d74ed06320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:49.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.667+0000 7ff75f7e5740 1 --2- 172.21.15.83:0/952777873 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d74ecf2240 0x55d74ecffcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:49.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.667+0000 7ff75f7e5740 1 --2- 172.21.15.83:0/952777873 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d74ecf1890 0x55d74ecff770 unknown :-1 s=CLOSED pgs=2157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:49.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.667+0000 7ff75f7e5740 1 -- 172.21.15.83:0/952777873 >> 172.21.15.83:0/952777873 conn(0x55d74ecede60 msgr2=0x55d74ecf3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:49.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.667+0000 7ff73b400640 0 client.20688 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:05:49.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.668+0000 7ff75f7e5740 1 -- 172.21.15.83:0/952777873 shutdown_connections 2024-04-24T09:05:49.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.668+0000 7ff75f7e5740 1 -- 172.21.15.83:0/952777873 wait complete. 2024-04-24T09:05:49.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.668+0000 7ff75f7e5740 20 client.20688 trim_cache size 0 max 16384 2024-04-24T09:05:49.668 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:05:49.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.669+0000 7ff757200640 1 -- 172.21.15.83:0/2385648756 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff73c0a3dd0 msgr2=0x7ff73c0a6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:49.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.669+0000 7ff757200640 1 --2- 172.21.15.83:0/2385648756 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff73c0a3dd0 0x7ff73c0a6290 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7ff748009db0 tx=0x7ff748005f90 comp rx=0 tx=0).stop 2024-04-24T09:05:49.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.669+0000 7ff757200640 1 -- 172.21.15.83:0/2385648756 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff758109920 msgr2=0x7ff75810afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:49.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.669+0000 7ff757200640 1 --2- 172.21.15.83:0/2385648756 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff758109920 0x7ff75810afb0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff740005ab0 tx=0x7ff740003c80 comp rx=0 tx=0).stop 2024-04-24T09:05:49.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.669+0000 7ff757200640 1 -- 172.21.15.83:0/2385648756 shutdown_connections 2024-04-24T09:05:49.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.669+0000 7ff757200640 1 --2- 172.21.15.83:0/2385648756 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff73c0a3dd0 0x7ff73c0a6290 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:49.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.669+0000 7ff757200640 1 --2- 172.21.15.83:0/2385648756 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff758115150 0x7ff75810b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:49.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.669+0000 7ff757200640 1 --2- 172.21.15.83:0/2385648756 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff758109920 0x7ff75810afb0 secure :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff740005ab0 tx=0x7ff740003c80 comp rx=0 tx=0).stop 2024-04-24T09:05:49.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.669+0000 7ff757200640 1 --2- 172.21.15.83:0/2385648756 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff758109000 0x7ff75810aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:49.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.669+0000 7ff757200640 1 -- 172.21.15.83:0/2385648756 >> 172.21.15.83:0/2385648756 conn(0x7ff75806cf80 msgr2=0x7ff758070c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:49.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.670+0000 7ff757200640 1 -- 172.21.15.83:0/2385648756 shutdown_connections 2024-04-24T09:05:49.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:49.670+0000 7ff757200640 1 -- 172.21.15.83:0/2385648756 wait complete. 2024-04-24T09:05:50.052 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:05:49 smithi083 ceph-mon[27907]: pgmap v445: 129 pgs: 129 active+clean; 279 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 988 KiB/s rd, 3.4 MiB/s wr, 291 op/s 2024-04-24T09:05:50.153 INFO:tasks.workunit.client.0.smithi083.stdout: 10 2606 14.05 MB/sec warmup 21 sec latency 82.137 ms 2024-04-24T09:05:50.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:05:49 smithi138 ceph-mon[31441]: pgmap v445: 129 pgs: 129 active+clean; 279 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 988 KiB/s rd, 3.4 MiB/s wr, 291 op/s 2024-04-24T09:05:50.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:05:49 smithi189 ceph-mon[31791]: pgmap v445: 129 pgs: 129 active+clean; 279 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 988 KiB/s rd, 3.4 MiB/s wr, 291 op/s 2024-04-24T09:05:50.695 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T09:05:50.696 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T09:05:50.696 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T09:05:50.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.956+0000 7ff042200640 1 -- 172.21.15.83:0/1176478525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff03c103980 msgr2=0x7ff03c10be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:50.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.956+0000 7ff042200640 1 --2- 172.21.15.83:0/1176478525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff03c103980 0x7ff03c10be60 secure :-1 s=READY pgs=2158 cs=0 l=1 rev1=1 crypto rx=0x7ff02c009900 tx=0x7ff02c02a480 comp rx=0 tx=0).stop 2024-04-24T09:05:50.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.956+0000 7ff042200640 1 -- 172.21.15.83:0/1176478525 shutdown_connections 2024-04-24T09:05:50.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.956+0000 7ff042200640 1 --2- 172.21.15.83:0/1176478525 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff03c10c480 0x7ff03c10e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:50.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.956+0000 7ff042200640 1 --2- 172.21.15.83:0/1176478525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff03c103980 0x7ff03c10be60 unknown :-1 s=CLOSED pgs=2158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:50.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.956+0000 7ff042200640 1 --2- 172.21.15.83:0/1176478525 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff03c103060 0x7ff03c103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:50.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.956+0000 7ff042200640 1 -- 172.21.15.83:0/1176478525 >> 172.21.15.83:0/1176478525 conn(0x7ff03c0fc880 msgr2=0x7ff03c0feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:50.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.956+0000 7ff042200640 1 -- 172.21.15.83:0/1176478525 shutdown_connections 2024-04-24T09:05:50.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.956+0000 7ff042200640 1 -- 172.21.15.83:0/1176478525 wait complete. 2024-04-24T09:05:50.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.957+0000 7ff042200640 1 Processor -- start 2024-04-24T09:05:50.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.957+0000 7ff042200640 1 -- start start 2024-04-24T09:05:50.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.958+0000 7ff042200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff03c103060 0x7ff03c100cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:50.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.958+0000 7ff042200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff03c103980 0x7ff03c1011f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:50.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.958+0000 7ff042200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff03c10c480 0x7ff03c101730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:50.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.958+0000 7ff042200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff03c112140 con 0x7ff03c10c480 2024-04-24T09:05:50.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.958+0000 7ff042200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff03c111fc0 con 0x7ff03c103060 2024-04-24T09:05:50.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.958+0000 7ff042200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff03c1122c0 con 0x7ff03c103980 2024-04-24T09:05:50.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.958+0000 7ff040e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff03c103060 0x7ff03c100cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:50.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.958+0000 7ff040e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff03c103060 0x7ff03c100cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:34958/0 (socket says 172.21.15.83:34958) 2024-04-24T09:05:50.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.958+0000 7ff040e00640 1 -- 172.21.15.83:0/1345406910 learned_addr learned my addr 172.21.15.83:0/1345406910 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:05:50.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.958+0000 7ff041800640 1 --2- 172.21.15.83:0/1345406910 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff03c10c480 0x7ff03c101730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:50.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.958+0000 7ff03be00640 1 --2- 172.21.15.83:0/1345406910 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff03c103980 0x7ff03c1011f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:50.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.959+0000 7ff040e00640 1 -- 172.21.15.83:0/1345406910 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff03c103980 msgr2=0x7ff03c1011f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:50.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.959+0000 7ff040e00640 1 --2- 172.21.15.83:0/1345406910 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff03c103980 0x7ff03c1011f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:50.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.959+0000 7ff040e00640 1 -- 172.21.15.83:0/1345406910 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff03c10c480 msgr2=0x7ff03c101730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:50.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.959+0000 7ff040e00640 1 --2- 172.21.15.83:0/1345406910 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff03c10c480 0x7ff03c101730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:50.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.959+0000 7ff040e00640 1 -- 172.21.15.83:0/1345406910 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff03c1a6be0 con 0x7ff03c103060 2024-04-24T09:05:50.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.959+0000 7ff040e00640 1 --2- 172.21.15.83:0/1345406910 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff03c103060 0x7ff03c100cb0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff02400eab0 tx=0x7ff02400ef70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:50.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.960+0000 7ff039600640 1 -- 172.21.15.83:0/1345406910 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff02400ce40 con 0x7ff03c103060 2024-04-24T09:05:50.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.960+0000 7ff039600640 1 -- 172.21.15.83:0/1345406910 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff024011920 con 0x7ff03c103060 2024-04-24T09:05:50.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.960+0000 7ff039600640 1 -- 172.21.15.83:0/1345406910 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff02401aa00 con 0x7ff03c103060 2024-04-24T09:05:50.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.960+0000 7ff042200640 1 -- 172.21.15.83:0/1345406910 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff03c1a6ed0 con 0x7ff03c103060 2024-04-24T09:05:50.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.960+0000 7ff042200640 1 -- 172.21.15.83:0/1345406910 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff03c1a7340 con 0x7ff03c103060 2024-04-24T09:05:50.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.964+0000 7ff04677b740 1 Processor -- start 2024-04-24T09:05:50.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.965+0000 7ff04677b740 1 -- start start 2024-04-24T09:05:50.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.965+0000 7ff04677b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f5dd4c270 0x555f5dd4c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:50.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.965+0000 7ff04677b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555f5dd4cb90 0x555f5dd56d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:50.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.965+0000 7ff04677b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f5dd57260 0x555f5dd59650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:50.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.965+0000 7ff04677b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x555f5dcb1e70 con 0x555f5dd4c270 2024-04-24T09:05:50.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.965+0000 7ff04677b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555f5dc58380 con 0x555f5dd57260 2024-04-24T09:05:50.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.965+0000 7ff04677b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x555f5dcade30 con 0x555f5dd4cb90 2024-04-24T09:05:50.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.966+0000 7ff03be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f5dd57260 0x555f5dd59650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:50.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.966+0000 7ff03be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f5dd57260 0x555f5dd59650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:34972/0 (socket says 172.21.15.83:34972) 2024-04-24T09:05:50.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.966+0000 7ff03be00640 1 -- 172.21.15.83:0/1753498563 learned_addr learned my addr 172.21.15.83:0/1753498563 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:05:50.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.966+0000 7ff041800640 1 --2- 172.21.15.83:0/1753498563 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f5dd4c270 0x555f5dd4c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:50.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.966+0000 7ff03be00640 1 -- 172.21.15.83:0/1753498563 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555f5dd4cb90 msgr2=0x555f5dd56d20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T09:05:50.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.966+0000 7ff03be00640 1 --2- 172.21.15.83:0/1753498563 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555f5dd4cb90 0x555f5dd56d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:50.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.966+0000 7ff03be00640 1 -- 172.21.15.83:0/1753498563 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f5dd4c270 msgr2=0x555f5dd4c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:50.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.966+0000 7ff03be00640 1 --2- 172.21.15.83:0/1753498563 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f5dd4c270 0x555f5dd4c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:50.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.966+0000 7ff03be00640 1 -- 172.21.15.83:0/1753498563 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f5da8c7e0 con 0x555f5dd57260 2024-04-24T09:05:50.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.967+0000 7ff03be00640 1 --2- 172.21.15.83:0/1753498563 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f5dd57260 0x555f5dd59650 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff02c009900 tx=0x7ff02c0040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1c8672df0be2771 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:50.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.967+0000 7ff032000640 1 -- 172.21.15.83:0/1753498563 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff02c035030 con 0x555f5dd57260 2024-04-24T09:05:50.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.967+0000 7ff039600640 1 -- 172.21.15.83:0/1345406910 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff02401abe0 con 0x7ff03c103060 2024-04-24T09:05:50.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.967+0000 7ff032000640 1 -- 172.21.15.83:0/1753498563 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff02c0054f0 con 0x555f5dd57260 2024-04-24T09:05:50.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.967+0000 7ff032000640 1 -- 172.21.15.83:0/1753498563 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff02c035030 con 0x555f5dd57260 2024-04-24T09:05:50.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.968+0000 7ff04677b740 1 -- 172.21.15.83:0/1753498563 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f5dd57260 msgr2=0x555f5dd59650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:50.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.969+0000 7ff04677b740 1 --2- 172.21.15.83:0/1753498563 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f5dd57260 0x555f5dd59650 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff02c009900 tx=0x7ff02c0040c0 comp rx=0 tx=0).stop 2024-04-24T09:05:50.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.969+0000 7ff04677b740 1 -- 172.21.15.83:0/1753498563 shutdown_connections 2024-04-24T09:05:50.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.969+0000 7ff04677b740 1 --2- 172.21.15.83:0/1753498563 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f5dd57260 0x555f5dd59650 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:50.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.969+0000 7ff04677b740 1 --2- 172.21.15.83:0/1753498563 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555f5dd4cb90 0x555f5dd56d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:50.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.969+0000 7ff04677b740 1 --2- 172.21.15.83:0/1753498563 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f5dd4c270 0x555f5dd4c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:50.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.969+0000 7ff04677b740 1 -- 172.21.15.83:0/1753498563 >> 172.21.15.83:0/1753498563 conn(0x555f5dd48590 msgr2=0x555f5dd48970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:50.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.970+0000 7ff04677b740 1 -- 172.21.15.83:0/1753498563 shutdown_connections 2024-04-24T09:05:50.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.970+0000 7ff04677b740 1 -- 172.21.15.83:0/1753498563 wait complete. 2024-04-24T09:05:50.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.970+0000 7ff039600640 1 --2- 172.21.15.83:0/1345406910 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff0200a3cb0 0x7ff0200a6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:50.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.970+0000 7ff04677b740 1 -- start start 2024-04-24T09:05:50.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.970+0000 7ff039600640 1 -- 172.21.15.83:0/1345406910 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7ff0240d1610 con 0x7ff03c103060 2024-04-24T09:05:50.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.971+0000 7ff04677b740 1 Processor -- start 2024-04-24T09:05:50.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.971+0000 7ff041800640 1 --2- 172.21.15.83:0/1345406910 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff0200a3cb0 0x7ff0200a6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:50.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.972+0000 7ff041800640 1 --2- 172.21.15.83:0/1345406910 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff0200a3cb0 0x7ff0200a6170 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7ff034009c80 tx=0x7ff034005f90 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:50.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.972+0000 7ff04677b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f5dd4c270 0x555f5dd54270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:50.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.973+0000 7ff04677b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555f5dd4cb90 0x555f5dd547b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:50.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.973+0000 7ff04677b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f5dd57260 0x555f5dd54cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:50.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.973+0000 7ff04677b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x555f5dd62800 con 0x555f5dd4c270 2024-04-24T09:05:50.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.973+0000 7ff04677b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555f5dc6b660 con 0x555f5dd57260 2024-04-24T09:05:50.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.973+0000 7ff04677b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x555f5dd62980 con 0x555f5dd4cb90 2024-04-24T09:05:50.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.973+0000 7ff04677b740 10 client.0 _ll_register_callbacks cb 0x555f5dc59aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:05:50.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.973+0000 7ff041800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f5dd57260 0x555f5dd54cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:50.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.973+0000 7ff041800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f5dd57260 0x555f5dd54cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:34974/0 (socket says 172.21.15.83:34974) 2024-04-24T09:05:50.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.973+0000 7ff041800640 1 -- 172.21.15.83:0/756954124 learned_addr learned my addr 172.21.15.83:0/756954124 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:05:50.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.973+0000 7ff040e00640 1 --2- 172.21.15.83:0/756954124 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f5dd4c270 0x555f5dd54270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:50.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.974+0000 7ff03be00640 1 --2- 172.21.15.83:0/756954124 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555f5dd4cb90 0x555f5dd547b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:50.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.974+0000 7ff041800640 1 -- 172.21.15.83:0/756954124 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555f5dd4cb90 msgr2=0x555f5dd547b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:50.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.974+0000 7ff041800640 1 --2- 172.21.15.83:0/756954124 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555f5dd4cb90 0x555f5dd547b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:50.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.974+0000 7ff041800640 1 -- 172.21.15.83:0/756954124 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f5dd4c270 msgr2=0x555f5dd54270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:50.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.974+0000 7ff041800640 1 --2- 172.21.15.83:0/756954124 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f5dd4c270 0x555f5dd54270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:50.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.974+0000 7ff041800640 1 -- 172.21.15.83:0/756954124 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f5daecf70 con 0x555f5dd57260 2024-04-24T09:05:50.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.974+0000 7ff03be00640 1 --2- 172.21.15.83:0/756954124 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555f5dd4cb90 0x555f5dd547b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:05:50.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.974+0000 7ff040e00640 1 --2- 172.21.15.83:0/756954124 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f5dd4c270 0x555f5dd54270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:05:50.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.975+0000 7ff041800640 1 --2- 172.21.15.83:0/756954124 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f5dd57260 0x555f5dd54cf0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff0340045a0 tx=0x7ff034004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb4fcccc7ae0a5f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:50.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.975+0000 7ff02aa00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:05:50.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.975+0000 7ff02aa00640 1 -- 172.21.15.83:0/756954124 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff034029440 con 0x555f5dd57260 2024-04-24T09:05:50.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.976+0000 7ff02aa00640 1 -- 172.21.15.83:0/756954124 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7ff03401f430 con 0x555f5dd57260 2024-04-24T09:05:50.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.976+0000 7ff04677b740 1 -- 172.21.15.83:0/756954124 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff02c0054f0 con 0x555f5dd57260 2024-04-24T09:05:50.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.976+0000 7ff02aa00640 1 -- 172.21.15.83:0/756954124 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7ff03401f630 con 0x555f5dd57260 2024-04-24T09:05:50.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.976+0000 7ff04677b740 10 client.37850 fetch_fsmap learned FSMap version 47 2024-04-24T09:05:50.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.976+0000 7ff04677b740 1 -- 172.21.15.83:0/756954124 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x555f5d984170 con 0x555f5dd57260 2024-04-24T09:05:50.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.977+0000 7ff02aa00640 1 -- 172.21.15.83:0/756954124 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff02c0054f0 con 0x555f5dd57260 2024-04-24T09:05:50.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.977+0000 7ff02aa00640 1 -- 172.21.15.83:0/756954124 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 47) ==== 8398+0+0 (secure 0 0 0) 0x7ff034027070 con 0x555f5dd57260 2024-04-24T09:05:50.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.977+0000 7ff04677b740 10 client.37850 fetch_fsmap finished waiting for FSMap version 47 2024-04-24T09:05:50.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.977+0000 7ff04677b740 10 client.37850 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:05:50.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.977+0000 7ff04677b740 20 client.37850 populate_metadata read hostname 'smithi083' 2024-04-24T09:05:50.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.977+0000 7ff04677b740 1 --2- 172.21.15.83:0/756954124 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x555f5dd6ff20 0x555f5dd723c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:50.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.977+0000 7ff04677b740 4 client.37850 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:05:50.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.977+0000 7ff04677b740 1 -- 172.21.15.83:0/756954124 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555f5dcbf890 con 0x555f5dd6ff20 2024-04-24T09:05:50.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.978+0000 7ff040e00640 1 --2- 172.21.15.83:0/756954124 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x555f5dd6ff20 0x555f5dd723c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:50.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.978+0000 7ff040e00640 1 --2- 172.21.15.83:0/756954124 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x555f5dd6ff20 0x555f5dd723c0 crc :-1 s=READY pgs=367 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b826af1906ed8f4 server_cookie=f068d7d324c2e32 in_seq=0 out_seq=0 2024-04-24T09:05:50.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.978+0000 7ff02aa00640 10 client.37850 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:05:50.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.981+0000 7ff02aa00640 1 -- 172.21.15.83:0/756954124 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555f5dcbf890 con 0x555f5dd6ff20 2024-04-24T09:05:50.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.981+0000 7ff02aa00640 10 client.37850 handle_command_reply: tid=0 2024-04-24T09:05:50.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.981+0000 7ff04677b740 1 client.37850 shutdown 2024-04-24T09:05:50.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.981+0000 7ff04677b740 1 -- 172.21.15.83:0/756954124 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f5dd57260 msgr2=0x555f5dd54cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:50.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.981+0000 7ff04677b740 1 --2- 172.21.15.83:0/756954124 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f5dd57260 0x555f5dd54cf0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff0340045a0 tx=0x7ff034004730 comp rx=0 tx=0).stop 2024-04-24T09:05:50.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.982+0000 7ff04677b740 1 -- 172.21.15.83:0/756954124 shutdown_connections 2024-04-24T09:05:50.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.982+0000 7ff04677b740 1 --2- 172.21.15.83:0/756954124 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x555f5dd6ff20 0x555f5dd723c0 crc :-1 s=READY pgs=367 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:50.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.982+0000 7ff04677b740 1 --2- 172.21.15.83:0/756954124 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f5dd57260 0x555f5dd54cf0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:50.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.982+0000 7ff04677b740 1 --2- 172.21.15.83:0/756954124 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555f5dd4cb90 0x555f5dd547b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:50.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.982+0000 7ff04677b740 1 --2- 172.21.15.83:0/756954124 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f5dd4c270 0x555f5dd54270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:50.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.982+0000 7ff04677b740 1 -- 172.21.15.83:0/756954124 >> 172.21.15.83:0/756954124 conn(0x555f5dd48590 msgr2=0x555f5dd4e290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:50.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.982+0000 7ff02aa00640 0 client.37850 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:05:50.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.982+0000 7ff04677b740 1 -- 172.21.15.83:0/756954124 shutdown_connections 2024-04-24T09:05:50.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.983+0000 7ff04677b740 1 -- 172.21.15.83:0/756954124 wait complete. 2024-04-24T09:05:50.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:50.983+0000 7ff04677b740 20 client.37850 trim_cache size 0 max 16384 2024-04-24T09:05:51.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.015+0000 7ff04677b740 1 Processor -- start 2024-04-24T09:05:51.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.015+0000 7ff04677b740 1 -- start start 2024-04-24T09:05:51.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.015+0000 7ff04677b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f5dd4c270 0x555f5dd6b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:51.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.015+0000 7ff04677b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f5dd4cb90 0x555f5dd4ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:51.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.015+0000 7ff04677b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555f5dd57260 0x555f5dd47570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:51.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.015+0000 7ff04677b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff03c1ad290 con 0x555f5dd4c270 2024-04-24T09:05:51.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.015+0000 7ff04677b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff03c1aef50 con 0x555f5dd4cb90 2024-04-24T09:05:51.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.015+0000 7ff04677b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff03c1ad960 con 0x555f5dd57260 2024-04-24T09:05:51.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.016+0000 7ff03be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f5dd4cb90 0x555f5dd4ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:51.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.016+0000 7ff041800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555f5dd57260 0x555f5dd47570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:51.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.016+0000 7ff040e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f5dd4c270 0x555f5dd6b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:51.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.016+0000 7ff041800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555f5dd57260 0x555f5dd47570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:39506/0 (socket says 172.21.15.83:39506) 2024-04-24T09:05:51.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.016+0000 7ff040e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f5dd4c270 0x555f5dd6b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43322/0 (socket says 172.21.15.83:43322) 2024-04-24T09:05:51.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.016+0000 7ff041800640 1 -- 172.21.15.83:0/372383606 learned_addr learned my addr 172.21.15.83:0/372383606 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:05:51.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.017+0000 7ff040e00640 1 -- 172.21.15.83:0/372383606 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555f5dd57260 msgr2=0x555f5dd47570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:51.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.017+0000 7ff040e00640 1 --2- 172.21.15.83:0/372383606 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555f5dd57260 0x555f5dd47570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:51.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.017+0000 7ff040e00640 1 -- 172.21.15.83:0/372383606 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f5dd4cb90 msgr2=0x555f5dd4ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:51.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.017+0000 7ff040e00640 1 --2- 172.21.15.83:0/372383606 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f5dd4cb90 0x555f5dd4ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:51.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.017+0000 7ff040e00640 1 -- 172.21.15.83:0/372383606 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f5dd599c0 con 0x555f5dd4c270 2024-04-24T09:05:51.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.018+0000 7ff040e00640 1 --2- 172.21.15.83:0/372383606 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f5dd4c270 0x555f5dd6b6f0 secure :-1 s=READY pgs=2159 cs=0 l=1 rev1=1 crypto rx=0x7ff0240ce8e0 tx=0x7ff024098d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f94231576c65db9b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:51.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.018+0000 7ff031600640 1 -- 172.21.15.83:0/372383606 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0240bb070 con 0x555f5dd4c270 2024-04-24T09:05:51.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.018+0000 7ff031600640 1 -- 172.21.15.83:0/372383606 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff02409d830 con 0x555f5dd4c270 2024-04-24T09:05:51.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.018+0000 7ff031600640 1 -- 172.21.15.83:0/372383606 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0240ceb60 con 0x555f5dd4c270 2024-04-24T09:05:51.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.020+0000 7ff04677b740 1 -- 172.21.15.83:0/372383606 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f5dd4c270 msgr2=0x555f5dd6b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:51.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.020+0000 7ff04677b740 1 --2- 172.21.15.83:0/372383606 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f5dd4c270 0x555f5dd6b6f0 secure :-1 s=READY pgs=2159 cs=0 l=1 rev1=1 crypto rx=0x7ff0240ce8e0 tx=0x7ff024098d30 comp rx=0 tx=0).stop 2024-04-24T09:05:51.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.020+0000 7ff04677b740 1 -- 172.21.15.83:0/372383606 shutdown_connections 2024-04-24T09:05:51.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.020+0000 7ff04677b740 1 --2- 172.21.15.83:0/372383606 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555f5dd57260 0x555f5dd47570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:51.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.020+0000 7ff04677b740 1 --2- 172.21.15.83:0/372383606 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f5dd4cb90 0x555f5dd4ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:51.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.020+0000 7ff04677b740 1 --2- 172.21.15.83:0/372383606 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f5dd4c270 0x555f5dd6b6f0 unknown :-1 s=CLOSED pgs=2159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:51.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.020+0000 7ff04677b740 1 -- 172.21.15.83:0/372383606 >> 172.21.15.83:0/372383606 conn(0x555f5dd48590 msgr2=0x555f5dd46e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:51.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.021+0000 7ff04677b740 1 -- 172.21.15.83:0/372383606 shutdown_connections 2024-04-24T09:05:51.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.021+0000 7ff04677b740 1 -- 172.21.15.83:0/372383606 wait complete. 2024-04-24T09:05:51.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.021+0000 7ff04677b740 1 -- start start 2024-04-24T09:05:51.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.022+0000 7ff04677b740 1 Processor -- start 2024-04-24T09:05:51.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.022+0000 7ff04677b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555f5dd4c270 0x555f5dd51620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:51.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.022+0000 7ff04677b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f5dd4cb90 0x555f5dd51b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:51.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.022+0000 7ff04677b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f5dd57260 0x555f5ddd5500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:51.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.023+0000 7ff04677b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff03c1ae7c0 con 0x555f5dd57260 2024-04-24T09:05:51.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.023+0000 7ff04677b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff03c1ae090 con 0x555f5dd4cb90 2024-04-24T09:05:51.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.023+0000 7ff04677b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff03c1ab290 con 0x555f5dd4c270 2024-04-24T09:05:51.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.023+0000 7ff04677b740 10 client.0 _ll_register_callbacks cb 0x7ff0340098a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:05:51.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.023+0000 7ff03be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f5dd4cb90 0x555f5dd51b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:51.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.023+0000 7ff03be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f5dd4cb90 0x555f5dd51b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:34992/0 (socket says 172.21.15.83:34992) 2024-04-24T09:05:51.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.023+0000 7ff03be00640 1 -- 172.21.15.83:0/2176053337 learned_addr learned my addr 172.21.15.83:0/2176053337 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:05:51.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.023+0000 7ff040e00640 1 --2- 172.21.15.83:0/2176053337 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555f5dd4c270 0x555f5dd51620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:51.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.023+0000 7ff041800640 1 --2- 172.21.15.83:0/2176053337 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f5dd57260 0x555f5ddd5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:51.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.023+0000 7ff03be00640 1 -- 172.21.15.83:0/2176053337 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555f5dd4c270 msgr2=0x555f5dd51620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:51.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.023+0000 7ff03be00640 1 --2- 172.21.15.83:0/2176053337 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555f5dd4c270 0x555f5dd51620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:51.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.023+0000 7ff03be00640 1 -- 172.21.15.83:0/2176053337 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f5dd57260 msgr2=0x555f5ddd5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:51.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.024+0000 7ff03be00640 1 --2- 172.21.15.83:0/2176053337 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f5dd57260 0x555f5ddd5500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:51.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.024+0000 7ff03be00640 1 -- 172.21.15.83:0/2176053337 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f5ddd5a40 con 0x555f5dd4cb90 2024-04-24T09:05:51.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.024+0000 7ff041800640 1 --2- 172.21.15.83:0/2176053337 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f5dd57260 0x555f5ddd5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:05:51.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.024+0000 7ff03be00640 1 --2- 172.21.15.83:0/2176053337 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f5dd4cb90 0x555f5dd51b60 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff02c02a4c0 tx=0x7ff02c009230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22ea40c3f6dbabc5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:51.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.024+0000 7ff02aa00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:05:51.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.024+0000 7ff02aa00640 1 -- 172.21.15.83:0/2176053337 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff02c035040 con 0x555f5dd4cb90 2024-04-24T09:05:51.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.024+0000 7ff02aa00640 1 -- 172.21.15.83:0/2176053337 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7ff02c004ad0 con 0x555f5dd4cb90 2024-04-24T09:05:51.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.024+0000 7ff04677b740 1 -- 172.21.15.83:0/2176053337 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff02409d830 con 0x555f5dd4cb90 2024-04-24T09:05:51.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.025+0000 7ff02aa00640 1 -- 172.21.15.83:0/2176053337 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7ff02c004cf0 con 0x555f5dd4cb90 2024-04-24T09:05:51.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.025+0000 7ff04677b740 10 client.37856 fetch_fsmap learned FSMap version 47 2024-04-24T09:05:51.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.025+0000 7ff04677b740 1 -- 172.21.15.83:0/2176053337 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x555f5ddd6cf0 con 0x555f5dd4cb90 2024-04-24T09:05:51.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.025+0000 7ff02aa00640 1 -- 172.21.15.83:0/2176053337 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff02409d830 con 0x555f5dd4cb90 2024-04-24T09:05:51.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.026+0000 7ff02aa00640 1 -- 172.21.15.83:0/2176053337 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 47) ==== 8398+0+0 (secure 0 0 0) 0x7ff02c0314a0 con 0x555f5dd4cb90 2024-04-24T09:05:51.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.026+0000 7ff04677b740 10 client.37856 fetch_fsmap finished waiting for FSMap version 47 2024-04-24T09:05:51.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.026+0000 7ff04677b740 10 client.37856 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:05:51.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.026+0000 7ff04677b740 20 client.37856 populate_metadata read hostname 'smithi083' 2024-04-24T09:05:51.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.026+0000 7ff04677b740 1 --2- 172.21.15.83:0/2176053337 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x555f5ddd7040 0x555f5ddd9460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:51.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.026+0000 7ff04677b740 4 client.37856 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T09:05:51.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.026+0000 7ff04677b740 1 -- 172.21.15.83:0/2176053337 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x555f5dcb9700 con 0x555f5ddd7040 2024-04-24T09:05:51.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.027+0000 7ff040e00640 1 --2- 172.21.15.83:0/2176053337 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x555f5ddd7040 0x555f5ddd9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:51.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.028+0000 7ff040e00640 1 --2- 172.21.15.83:0/2176053337 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x555f5ddd7040 0x555f5ddd9460 crc :-1 s=READY pgs=368 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a818a611341d6d87 server_cookie=23af3b26fbecb62d in_seq=0 out_seq=0 2024-04-24T09:05:51.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.028+0000 7ff02aa00640 10 client.37856 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:05:51.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.029+0000 7ff02aa00640 1 -- 172.21.15.83:0/2176053337 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x555f5dcb9700 con 0x555f5ddd7040 2024-04-24T09:05:51.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.029+0000 7ff02aa00640 10 client.37856 handle_command_reply: tid=0 2024-04-24T09:05:51.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.029+0000 7ff04677b740 1 client.37856 shutdown 2024-04-24T09:05:51.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.029+0000 7ff04677b740 1 -- 172.21.15.83:0/2176053337 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f5dd4cb90 msgr2=0x555f5dd51b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:51.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.029+0000 7ff04677b740 1 --2- 172.21.15.83:0/2176053337 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f5dd4cb90 0x555f5dd51b60 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff02c02a4c0 tx=0x7ff02c009230 comp rx=0 tx=0).stop 2024-04-24T09:05:51.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.030+0000 7ff04677b740 1 -- 172.21.15.83:0/2176053337 shutdown_connections 2024-04-24T09:05:51.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.030+0000 7ff04677b740 1 --2- 172.21.15.83:0/2176053337 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x555f5ddd7040 0x555f5ddd9460 crc :-1 s=READY pgs=368 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:51.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.030+0000 7ff04677b740 1 --2- 172.21.15.83:0/2176053337 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f5dd57260 0x555f5ddd5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:51.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.030+0000 7ff04677b740 1 --2- 172.21.15.83:0/2176053337 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f5dd4cb90 0x555f5dd51b60 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:51.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.030+0000 7ff04677b740 1 --2- 172.21.15.83:0/2176053337 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555f5dd4c270 0x555f5dd51620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:51.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.030+0000 7ff02aa00640 0 client.37856 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:05:51.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.030+0000 7ff04677b740 1 -- 172.21.15.83:0/2176053337 >> 172.21.15.83:0/2176053337 conn(0x555f5dd48590 msgr2=0x555f5dd54ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:51.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.030+0000 7ff04677b740 1 -- 172.21.15.83:0/2176053337 shutdown_connections 2024-04-24T09:05:51.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.030+0000 7ff04677b740 1 -- 172.21.15.83:0/2176053337 wait complete. 2024-04-24T09:05:51.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.030+0000 7ff04677b740 20 client.37856 trim_cache size 0 max 16384 2024-04-24T09:05:51.030 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T09:05:51.030 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-04-24T09:05:51.030 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "a5c8b530-1a53-40d0-ab6c-94c416568360", 2024-04-24T09:05:51.030 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-04-24T09:05:51.030 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T09:05:51.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.031+0000 7ff042200640 1 -- 172.21.15.83:0/1345406910 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff0200a3cb0 msgr2=0x7ff0200a6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:51.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.031+0000 7ff042200640 1 --2- 172.21.15.83:0/1345406910 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff0200a3cb0 0x7ff0200a6170 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7ff034009c80 tx=0x7ff034005f90 comp rx=0 tx=0).stop 2024-04-24T09:05:51.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.031+0000 7ff042200640 1 -- 172.21.15.83:0/1345406910 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff03c103060 msgr2=0x7ff03c100cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:51.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.031+0000 7ff042200640 1 --2- 172.21.15.83:0/1345406910 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff03c103060 0x7ff03c100cb0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff02400eab0 tx=0x7ff02400ef70 comp rx=0 tx=0).stop 2024-04-24T09:05:51.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.031+0000 7ff042200640 1 -- 172.21.15.83:0/1345406910 shutdown_connections 2024-04-24T09:05:51.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.031+0000 7ff042200640 1 --2- 172.21.15.83:0/1345406910 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff0200a3cb0 0x7ff0200a6170 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:51.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.031+0000 7ff042200640 1 --2- 172.21.15.83:0/1345406910 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff03c10c480 0x7ff03c101730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:51.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.031+0000 7ff042200640 1 --2- 172.21.15.83:0/1345406910 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff03c103980 0x7ff03c1011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:51.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.031+0000 7ff042200640 1 --2- 172.21.15.83:0/1345406910 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff03c103060 0x7ff03c100cb0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:51.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.031+0000 7ff042200640 1 -- 172.21.15.83:0/1345406910 >> 172.21.15.83:0/1345406910 conn(0x7ff03c0fc880 msgr2=0x7ff03c0fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:51.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.031+0000 7ff042200640 1 -- 172.21.15.83:0/1345406910 shutdown_connections 2024-04-24T09:05:51.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.031+0000 7ff042200640 1 -- 172.21.15.83:0/1345406910 wait complete. 2024-04-24T09:05:51.050 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T09:05:51.050 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T09:05:51.154 INFO:tasks.workunit.client.0.smithi083.stdout: 10 2778 14.03 MB/sec warmup 22 sec latency 107.470 ms 2024-04-24T09:05:51.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.300+0000 7f62f1000640 1 -- 172.21.15.83:0/1601076272 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f62ec1011a0 msgr2=0x7f62ec101580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:51.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.300+0000 7f62f1000640 1 --2- 172.21.15.83:0/1601076272 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f62ec1011a0 0x7f62ec101580 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f62dc009900 tx=0x7f62dc02a3f0 comp rx=0 tx=0).stop 2024-04-24T09:05:51.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.301+0000 7f62f1000640 1 -- 172.21.15.83:0/1601076272 shutdown_connections 2024-04-24T09:05:51.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.301+0000 7f62f1000640 1 --2- 172.21.15.83:0/1601076272 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62ec106220 0x7f62ec10ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:51.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.301+0000 7f62f1000640 1 --2- 172.21.15.83:0/1601076272 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f62ec101b50 0x7f62ec105b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:51.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.301+0000 7f62f1000640 1 --2- 172.21.15.83:0/1601076272 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f62ec1011a0 0x7f62ec101580 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:51.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.301+0000 7f62f1000640 1 -- 172.21.15.83:0/1601076272 >> 172.21.15.83:0/1601076272 conn(0x7f62ec078060 msgr2=0x7f62ec078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:51.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.301+0000 7f62f1000640 1 -- 172.21.15.83:0/1601076272 shutdown_connections 2024-04-24T09:05:51.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.301+0000 7f62f1000640 1 -- 172.21.15.83:0/1601076272 wait complete. 2024-04-24T09:05:51.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.303+0000 7f62f1000640 1 Processor -- start 2024-04-24T09:05:51.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.304+0000 7f62f1000640 1 -- start start 2024-04-24T09:05:51.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.304+0000 7f62f1000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f62ec1011a0 0x7f62ec1a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:51.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.304+0000 7f62f1000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62ec106220 0x7f62ec1a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:51.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.304+0000 7f62f1000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f62ec19ae20 0x7f62ec19b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:51.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.304+0000 7f62f1000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f62ec10ffb0 con 0x7f62ec106220 2024-04-24T09:05:51.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.304+0000 7f62f1000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f62ec10fe30 con 0x7f62ec1011a0 2024-04-24T09:05:51.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.304+0000 7f62f1000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f62ec110130 con 0x7f62ec19ae20 2024-04-24T09:05:51.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.305+0000 7f62e9e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f62ec1011a0 0x7f62ec1a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:51.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.305+0000 7f62e9e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f62ec1011a0 0x7f62ec1a0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:35002/0 (socket says 172.21.15.83:35002) 2024-04-24T09:05:51.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.305+0000 7f62e9e00640 1 -- 172.21.15.83:0/1733726767 learned_addr learned my addr 172.21.15.83:0/1733726767 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:05:51.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.305+0000 7f62e9400640 1 --2- 172.21.15.83:0/1733726767 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62ec106220 0x7f62ec1a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:51.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.305+0000 7f62ea800640 1 --2- 172.21.15.83:0/1733726767 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f62ec19ae20 0x7f62ec19b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:51.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.305+0000 7f62e9e00640 1 -- 172.21.15.83:0/1733726767 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f62ec19ae20 msgr2=0x7f62ec19b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:51.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.305+0000 7f62e9e00640 1 --2- 172.21.15.83:0/1733726767 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f62ec19ae20 0x7f62ec19b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:51.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.305+0000 7f62e9e00640 1 -- 172.21.15.83:0/1733726767 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62ec106220 msgr2=0x7f62ec1a0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:51.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.305+0000 7f62e9e00640 1 --2- 172.21.15.83:0/1733726767 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62ec106220 0x7f62ec1a0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:51.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.305+0000 7f62e9e00640 1 -- 172.21.15.83:0/1733726767 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62ec19b940 con 0x7f62ec1011a0 2024-04-24T09:05:51.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.306+0000 7f62e9e00640 1 --2- 172.21.15.83:0/1733726767 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f62ec1011a0 0x7f62ec1a0680 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f62dc005ab0 tx=0x7f62dc003c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:51.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.306+0000 7f62e2a00640 1 -- 172.21.15.83:0/1733726767 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62dc03f070 con 0x7f62ec1011a0 2024-04-24T09:05:51.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.306+0000 7f62e2a00640 1 -- 172.21.15.83:0/1733726767 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f62dc031d60 con 0x7f62ec1011a0 2024-04-24T09:05:51.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.306+0000 7f62e2a00640 1 -- 172.21.15.83:0/1733726767 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62dc035030 con 0x7f62ec1011a0 2024-04-24T09:05:51.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.306+0000 7f62f1000640 1 -- 172.21.15.83:0/1733726767 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62ec19bbd0 con 0x7f62ec1011a0 2024-04-24T09:05:51.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.307+0000 7f62f1000640 1 -- 172.21.15.83:0/1733726767 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62ec111d00 con 0x7f62ec1011a0 2024-04-24T09:05:51.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.311+0000 7f62f3fc8740 1 Processor -- start 2024-04-24T09:05:51.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.311+0000 7f62e2a00640 1 -- 172.21.15.83:0/1733726767 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f62dc042070 con 0x7f62ec1011a0 2024-04-24T09:05:51.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.311+0000 7f62f3fc8740 1 -- start start 2024-04-24T09:05:51.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.311+0000 7f62f3fc8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560cb673d860 0x560cb673dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:51.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.311+0000 7f62f3fc8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560cb673e180 0x560cb6748310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:51.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.311+0000 7f62f3fc8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560cb6748850 0x560cb674ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:51.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.311+0000 7f62f3fc8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x560cb66a31f0 con 0x560cb673e180 2024-04-24T09:05:51.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.311+0000 7f62f3fc8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560cb6649350 con 0x560cb673d860 2024-04-24T09:05:51.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.311+0000 7f62f3fc8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x560cb669f1b0 con 0x560cb6748850 2024-04-24T09:05:51.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.312+0000 7f62ea800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560cb673d860 0x560cb673dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:51.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.312+0000 7f62ea800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560cb673d860 0x560cb673dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:35004/0 (socket says 172.21.15.83:35004) 2024-04-24T09:05:51.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.312+0000 7f62ea800640 1 -- 172.21.15.83:0/4231434899 learned_addr learned my addr 172.21.15.83:0/4231434899 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:05:51.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.312+0000 7f62e9400640 1 --2- 172.21.15.83:0/4231434899 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560cb6748850 0x560cb674ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:51.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.312+0000 7f62e9e00640 1 --2- 172.21.15.83:0/4231434899 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560cb673e180 0x560cb6748310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:51.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.313+0000 7f62ea800640 1 -- 172.21.15.83:0/4231434899 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560cb6748850 msgr2=0x560cb674ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:51.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.313+0000 7f62ea800640 1 --2- 172.21.15.83:0/4231434899 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560cb6748850 0x560cb674ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:51.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.313+0000 7f62ea800640 1 -- 172.21.15.83:0/4231434899 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560cb673e180 msgr2=0x560cb6748310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:51.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.313+0000 7f62ea800640 1 --2- 172.21.15.83:0/4231434899 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560cb673e180 0x560cb6748310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:51.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.313+0000 7f62ea800640 1 -- 172.21.15.83:0/4231434899 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560cb669df50 con 0x560cb673d860 2024-04-24T09:05:51.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.313+0000 7f62e9400640 1 --2- 172.21.15.83:0/4231434899 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560cb6748850 0x560cb674ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:05:51.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.313+0000 7f62e2a00640 1 --2- 172.21.15.83:0/1733726767 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f62d00ac700 0x7f62d00aebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:51.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.314+0000 7f62e2a00640 1 -- 172.21.15.83:0/1733726767 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7f62dc0f1370 con 0x7f62ec1011a0 2024-04-24T09:05:51.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.314+0000 7f62e9e00640 1 --2- 172.21.15.83:0/1733726767 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f62d00ac700 0x7f62d00aebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:51.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.314+0000 7f62ea800640 1 --2- 172.21.15.83:0/4231434899 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560cb673d860 0x560cb673dc40 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f62e4009ef0 tx=0x7f62e40117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c786bc389f46aefc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:51.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.315+0000 7f62e9e00640 1 --2- 172.21.15.83:0/1733726767 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f62d00ac700 0x7f62d00aebc0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x560cb673ee20 tx=0x7f62dc0f4da0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:51.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.315+0000 7f62d7400640 1 -- 172.21.15.83:0/4231434899 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e4003b70 con 0x560cb673d860 2024-04-24T09:05:51.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.315+0000 7f62d7400640 1 -- 172.21.15.83:0/4231434899 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f62e4003d10 con 0x560cb673d860 2024-04-24T09:05:51.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.315+0000 7f62d7400640 1 -- 172.21.15.83:0/4231434899 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e40129a0 con 0x560cb673d860 2024-04-24T09:05:51.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.316+0000 7f62f3fc8740 1 -- 172.21.15.83:0/4231434899 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560cb673d860 msgr2=0x560cb673dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:51.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.316+0000 7f62f3fc8740 1 --2- 172.21.15.83:0/4231434899 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560cb673d860 0x560cb673dc40 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f62e4009ef0 tx=0x7f62e40117b0 comp rx=0 tx=0).stop 2024-04-24T09:05:51.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.316+0000 7f62f3fc8740 1 -- 172.21.15.83:0/4231434899 shutdown_connections 2024-04-24T09:05:51.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.316+0000 7f62f3fc8740 1 --2- 172.21.15.83:0/4231434899 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560cb6748850 0x560cb674ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:51.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.316+0000 7f62f3fc8740 1 --2- 172.21.15.83:0/4231434899 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560cb673e180 0x560cb6748310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:51.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.316+0000 7f62f3fc8740 1 --2- 172.21.15.83:0/4231434899 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560cb673d860 0x560cb673dc40 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:51.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.316+0000 7f62f3fc8740 1 -- 172.21.15.83:0/4231434899 >> 172.21.15.83:0/4231434899 conn(0x560cb6739b80 msgr2=0x560cb6739f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:51.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.316+0000 7f62f3fc8740 1 -- 172.21.15.83:0/4231434899 shutdown_connections 2024-04-24T09:05:51.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.317+0000 7f62f3fc8740 1 -- 172.21.15.83:0/4231434899 wait complete. 2024-04-24T09:05:51.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.317+0000 7f62f3fc8740 1 -- start start 2024-04-24T09:05:51.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.317+0000 7f62f3fc8740 1 Processor -- start 2024-04-24T09:05:51.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.318+0000 7f62f3fc8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560cb673d860 0x560cb6745860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:51.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.318+0000 7f62f3fc8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560cb673e180 0x560cb6745da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:51.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.318+0000 7f62f3fc8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560cb6748850 0x560cb67462e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:51.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.318+0000 7f62f3fc8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x560cb674b640 con 0x560cb673d860 2024-04-24T09:05:51.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.318+0000 7f62f3fc8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560cb665c690 con 0x560cb6748850 2024-04-24T09:05:51.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.318+0000 7f62f3fc8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x560cb674b7c0 con 0x560cb673e180 2024-04-24T09:05:51.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.318+0000 7f62f3fc8740 10 client.0 _ll_register_callbacks cb 0x560cb6698650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:05:51.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.318+0000 7f62ea800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560cb673d860 0x560cb6745860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:51.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.318+0000 7f62ea800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560cb673d860 0x560cb6745860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43372/0 (socket says 172.21.15.83:43372) 2024-04-24T09:05:51.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.318+0000 7f62ea800640 1 -- 172.21.15.83:0/4068502999 learned_addr learned my addr 172.21.15.83:0/4068502999 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:05:51.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.318+0000 7f62ea800640 1 -- 172.21.15.83:0/4068502999 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560cb673e180 msgr2=0x560cb6745da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:51.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.318+0000 7f62ea800640 1 --2- 172.21.15.83:0/4068502999 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560cb673e180 0x560cb6745da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:51.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.318+0000 7f62ea800640 1 -- 172.21.15.83:0/4068502999 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560cb6748850 msgr2=0x560cb67462e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:51.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.318+0000 7f62ea800640 1 --2- 172.21.15.83:0/4068502999 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560cb6748850 0x560cb67462e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:51.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.318+0000 7f62ea800640 1 -- 172.21.15.83:0/4068502999 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560cb665b920 con 0x560cb673d860 2024-04-24T09:05:51.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.319+0000 7f62ea800640 1 --2- 172.21.15.83:0/4068502999 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560cb673d860 0x560cb6745860 secure :-1 s=READY pgs=2160 cs=0 l=1 rev1=1 crypto rx=0x7f62e4012520 tx=0x7f62e4009cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecb0978cc89c7908 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:51.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.319+0000 7f62cbe00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:05:51.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.319+0000 7f62cbe00640 1 -- 172.21.15.83:0/4068502999 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e4028020 con 0x560cb673d860 2024-04-24T09:05:51.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.319+0000 7f62f3fc8740 1 -- 172.21.15.83:0/4068502999 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62e4003d10 con 0x560cb673d860 2024-04-24T09:05:51.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.319+0000 7f62cbe00640 1 -- 172.21.15.83:0/4068502999 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7f62e4012b60 con 0x560cb673d860 2024-04-24T09:05:51.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.320+0000 7f62cbe00640 1 -- 172.21.15.83:0/4068502999 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f62e401f040 con 0x560cb673d860 2024-04-24T09:05:51.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.320+0000 7f62f3fc8740 10 client.20715 fetch_fsmap learned FSMap version 47 2024-04-24T09:05:51.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.320+0000 7f62f3fc8740 1 -- 172.21.15.83:0/4068502999 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x560cb647f7d0 con 0x560cb673d860 2024-04-24T09:05:51.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.320+0000 7f62cbe00640 1 -- 172.21.15.83:0/4068502999 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f62e4003d10 con 0x560cb673d860 2024-04-24T09:05:51.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.320+0000 7f62cbe00640 1 -- 172.21.15.83:0/4068502999 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 47) ==== 8398+0+0 (secure 0 0 0) 0x7f62e40244a0 con 0x560cb673d860 2024-04-24T09:05:51.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.320+0000 7f62f3fc8740 10 client.20715 fetch_fsmap finished waiting for FSMap version 47 2024-04-24T09:05:51.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.320+0000 7f62f3fc8740 10 client.20715 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:05:51.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.320+0000 7f62f3fc8740 20 client.20715 populate_metadata read hostname 'smithi083' 2024-04-24T09:05:51.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.320+0000 7f62f3fc8740 1 --2- 172.21.15.83:0/4068502999 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x560cb6758fa0 0x560cb675b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:51.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.320+0000 7f62f3fc8740 4 client.20715 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:05:51.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.320+0000 7f62f3fc8740 1 -- 172.21.15.83:0/4068502999 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560cb66b0c10 con 0x560cb6758fa0 2024-04-24T09:05:51.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.321+0000 7f62e9400640 1 --2- 172.21.15.83:0/4068502999 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x560cb6758fa0 0x560cb675b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:51.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.322+0000 7f62cbe00640 10 client.20715 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:05:51.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.322+0000 7f62e9400640 1 --2- 172.21.15.83:0/4068502999 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x560cb6758fa0 0x560cb675b440 crc :-1 s=READY pgs=369 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d68823c12da9c6fb server_cookie=1fd6a5fb8afc5128 in_seq=0 out_seq=0 2024-04-24T09:05:51.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.323+0000 7f62cbe00640 1 -- 172.21.15.83:0/4068502999 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560cb66b0c10 con 0x560cb6758fa0 2024-04-24T09:05:51.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.323+0000 7f62cbe00640 10 client.20715 handle_command_reply: tid=0 2024-04-24T09:05:51.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.323+0000 7f62f3fc8740 1 client.20715 shutdown 2024-04-24T09:05:51.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.323+0000 7f62f3fc8740 1 -- 172.21.15.83:0/4068502999 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560cb673d860 msgr2=0x560cb6745860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:51.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.323+0000 7f62f3fc8740 1 --2- 172.21.15.83:0/4068502999 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560cb673d860 0x560cb6745860 secure :-1 s=READY pgs=2160 cs=0 l=1 rev1=1 crypto rx=0x7f62e4012520 tx=0x7f62e4009cb0 comp rx=0 tx=0).stop 2024-04-24T09:05:51.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.323+0000 7f62f3fc8740 1 -- 172.21.15.83:0/4068502999 shutdown_connections 2024-04-24T09:05:51.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.323+0000 7f62f3fc8740 1 --2- 172.21.15.83:0/4068502999 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x560cb6758fa0 0x560cb675b440 crc :-1 s=READY pgs=369 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:51.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.323+0000 7f62f3fc8740 1 --2- 172.21.15.83:0/4068502999 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560cb6748850 0x560cb67462e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:51.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.323+0000 7f62f3fc8740 1 --2- 172.21.15.83:0/4068502999 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560cb673e180 0x560cb6745da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:51.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.323+0000 7f62f3fc8740 1 --2- 172.21.15.83:0/4068502999 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560cb673d860 0x560cb6745860 unknown :-1 s=CLOSED pgs=2160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:51.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.323+0000 7f62f3fc8740 1 -- 172.21.15.83:0/4068502999 >> 172.21.15.83:0/4068502999 conn(0x560cb6739b80 msgr2=0x560cb673f880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:51.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.323+0000 7f62cbe00640 0 client.20715 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:05:51.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.324+0000 7f62f3fc8740 1 -- 172.21.15.83:0/4068502999 shutdown_connections 2024-04-24T09:05:51.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.324+0000 7f62f3fc8740 1 -- 172.21.15.83:0/4068502999 wait complete. 2024-04-24T09:05:51.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.324+0000 7f62f3fc8740 20 client.20715 trim_cache size 0 max 16384 2024-04-24T09:05:51.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.355+0000 7f62f3fc8740 1 Processor -- start 2024-04-24T09:05:51.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.355+0000 7f62f3fc8740 1 -- start start 2024-04-24T09:05:51.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.356+0000 7f62f3fc8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560cb673d860 0x560cb6754500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:51.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.356+0000 7f62f3fc8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560cb673e180 0x560cb6760d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:51.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.356+0000 7f62f3fc8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560cb6748850 0x560cb6761290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:51.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.356+0000 7f62f3fc8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f62ec071a00 con 0x560cb673d860 2024-04-24T09:05:51.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.356+0000 7f62f3fc8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f62ec0736c0 con 0x560cb673e180 2024-04-24T09:05:51.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.356+0000 7f62f3fc8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f62ec0720d0 con 0x560cb6748850 2024-04-24T09:05:51.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.356+0000 7f62ea800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560cb673d860 0x560cb6754500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:51.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.356+0000 7f62ea800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560cb673d860 0x560cb6754500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43382/0 (socket says 172.21.15.83:43382) 2024-04-24T09:05:51.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.356+0000 7f62ea800640 1 -- 172.21.15.83:0/164871170 learned_addr learned my addr 172.21.15.83:0/164871170 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:05:51.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.356+0000 7f62ea800640 1 -- 172.21.15.83:0/164871170 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560cb6748850 msgr2=0x560cb6761290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:51.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.357+0000 7f62ea800640 1 --2- 172.21.15.83:0/164871170 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560cb6748850 0x560cb6761290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:51.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.357+0000 7f62ea800640 1 -- 172.21.15.83:0/164871170 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560cb673e180 msgr2=0x560cb6760d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:51.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.357+0000 7f62ea800640 1 --2- 172.21.15.83:0/164871170 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560cb673e180 0x560cb6760d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:51.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.357+0000 7f62ea800640 1 -- 172.21.15.83:0/164871170 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560cb674afb0 con 0x560cb673d860 2024-04-24T09:05:51.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.357+0000 7f62ea800640 1 --2- 172.21.15.83:0/164871170 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560cb673d860 0x560cb6754500 secure :-1 s=READY pgs=2161 cs=0 l=1 rev1=1 crypto rx=0x7f62e40189c0 tx=0x7f62e40126b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ed2b11a1988d6f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:51.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.357+0000 7f62d6a00640 1 -- 172.21.15.83:0/164871170 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e401f040 con 0x560cb673d860 2024-04-24T09:05:51.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.357+0000 7f62d6a00640 1 -- 172.21.15.83:0/164871170 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f62e4023370 con 0x560cb673d860 2024-04-24T09:05:51.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.357+0000 7f62d6a00640 1 -- 172.21.15.83:0/164871170 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e4018340 con 0x560cb673d860 2024-04-24T09:05:51.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.358+0000 7f62f3fc8740 1 -- 172.21.15.83:0/164871170 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560cb673d860 msgr2=0x560cb6754500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:51.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.358+0000 7f62f3fc8740 1 --2- 172.21.15.83:0/164871170 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560cb673d860 0x560cb6754500 secure :-1 s=READY pgs=2161 cs=0 l=1 rev1=1 crypto rx=0x7f62e40189c0 tx=0x7f62e40126b0 comp rx=0 tx=0).stop 2024-04-24T09:05:51.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.360+0000 7f62f3fc8740 1 -- 172.21.15.83:0/164871170 shutdown_connections 2024-04-24T09:05:51.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.360+0000 7f62f3fc8740 1 --2- 172.21.15.83:0/164871170 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560cb6748850 0x560cb6761290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:51.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.360+0000 7f62f3fc8740 1 --2- 172.21.15.83:0/164871170 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560cb673e180 0x560cb6760d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:51.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.360+0000 7f62f3fc8740 1 --2- 172.21.15.83:0/164871170 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560cb673d860 0x560cb6754500 unknown :-1 s=CLOSED pgs=2161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:51.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.360+0000 7f62f3fc8740 1 -- 172.21.15.83:0/164871170 >> 172.21.15.83:0/164871170 conn(0x560cb6739b80 msgr2=0x560cb6738390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:51.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.362+0000 7f62f3fc8740 1 -- 172.21.15.83:0/164871170 shutdown_connections 2024-04-24T09:05:51.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.363+0000 7f62f3fc8740 1 -- 172.21.15.83:0/164871170 wait complete. 2024-04-24T09:05:51.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.363+0000 7f62f3fc8740 1 -- start start 2024-04-24T09:05:51.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.363+0000 7f62f3fc8740 1 Processor -- start 2024-04-24T09:05:51.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.365+0000 7f62f3fc8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560cb673d860 0x560cb6750f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:51.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.365+0000 7f62f3fc8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560cb673e180 0x560cb6742330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:51.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.365+0000 7f62f3fc8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560cb6748850 0x560cb6742a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:51.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.365+0000 7f62f3fc8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f62ec072f30 con 0x560cb673d860 2024-04-24T09:05:51.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.365+0000 7f62f3fc8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f62ec072800 con 0x560cb6748850 2024-04-24T09:05:51.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.365+0000 7f62f3fc8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f62ec06f9c0 con 0x560cb673e180 2024-04-24T09:05:51.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.365+0000 7f62f3fc8740 10 client.0 _ll_register_callbacks cb 0x7f62dc0f4c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:05:51.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.365+0000 7f62e9400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560cb673e180 0x560cb6742330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:51.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.365+0000 7f62e9400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560cb673e180 0x560cb6742330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:39580/0 (socket says 172.21.15.83:39580) 2024-04-24T09:05:51.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.365+0000 7f62e9400640 1 -- 172.21.15.83:0/2040211661 learned_addr learned my addr 172.21.15.83:0/2040211661 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:05:51.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.366+0000 7f62ea800640 1 --2- 172.21.15.83:0/2040211661 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560cb673d860 0x560cb6750f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:51.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.366+0000 7f62ea800640 1 -- 172.21.15.83:0/2040211661 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560cb673e180 msgr2=0x560cb6742330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:51.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.366+0000 7f62ea800640 1 --2- 172.21.15.83:0/2040211661 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560cb673e180 0x560cb6742330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:51.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.366+0000 7f62ea800640 1 -- 172.21.15.83:0/2040211661 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560cb6748850 msgr2=0x560cb6742a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:51.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.366+0000 7f62ea800640 1 --2- 172.21.15.83:0/2040211661 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560cb6748850 0x560cb6742a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:51.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.366+0000 7f62ea800640 1 -- 172.21.15.83:0/2040211661 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560cb63b53f0 con 0x560cb673d860 2024-04-24T09:05:51.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.366+0000 7f62ea800640 1 --2- 172.21.15.83:0/2040211661 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560cb673d860 0x560cb6750f10 secure :-1 s=READY pgs=2162 cs=0 l=1 rev1=1 crypto rx=0x7f62e4012870 tx=0x7f62e40096a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87568533e52e7366 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:51.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.366+0000 7f62cbe00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:05:51.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.367+0000 7f62cbe00640 1 -- 172.21.15.83:0/2040211661 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e403a030 con 0x560cb673d860 2024-04-24T09:05:51.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.367+0000 7f62cbe00640 1 -- 172.21.15.83:0/2040211661 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7f62e4018d60 con 0x560cb673d860 2024-04-24T09:05:51.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.367+0000 7f62f3fc8740 1 -- 172.21.15.83:0/2040211661 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62e4023370 con 0x560cb673d860 2024-04-24T09:05:51.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.367+0000 7f62cbe00640 1 -- 172.21.15.83:0/2040211661 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f62e4047050 con 0x560cb673d860 2024-04-24T09:05:51.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.368+0000 7f62f3fc8740 10 client.20721 fetch_fsmap learned FSMap version 47 2024-04-24T09:05:51.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.368+0000 7f62f3fc8740 1 -- 172.21.15.83:0/2040211661 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x560cb63750f0 con 0x560cb673d860 2024-04-24T09:05:51.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.368+0000 7f62cbe00640 1 -- 172.21.15.83:0/2040211661 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f62e4023370 con 0x560cb673d860 2024-04-24T09:05:51.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.368+0000 7f62cbe00640 1 -- 172.21.15.83:0/2040211661 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 47) ==== 8398+0+0 (secure 0 0 0) 0x7f62e40244a0 con 0x560cb673d860 2024-04-24T09:05:51.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.368+0000 7f62f3fc8740 10 client.20721 fetch_fsmap finished waiting for FSMap version 47 2024-04-24T09:05:51.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.368+0000 7f62f3fc8740 10 client.20721 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:05:51.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.368+0000 7f62f3fc8740 20 client.20721 populate_metadata read hostname 'smithi083' 2024-04-24T09:05:51.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.368+0000 7f62f3fc8740 1 --2- 172.21.15.83:0/2040211661 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x560cb67bfab0 0x560cb67c1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:51.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.369+0000 7f62f3fc8740 4 client.20721 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T09:05:51.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.369+0000 7f62f3fc8740 1 -- 172.21.15.83:0/2040211661 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560cb66aaa80 con 0x560cb67bfab0 2024-04-24T09:05:51.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.369+0000 7f62e9400640 1 --2- 172.21.15.83:0/2040211661 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x560cb67bfab0 0x560cb67c1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:51.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.370+0000 7f62e9400640 1 --2- 172.21.15.83:0/2040211661 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x560cb67bfab0 0x560cb67c1ed0 crc :-1 s=READY pgs=370 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0b15396b10adf06 server_cookie=e406855ec025a4d9 in_seq=0 out_seq=0 2024-04-24T09:05:51.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.370+0000 7f62cbe00640 10 client.20721 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:05:51.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.372+0000 7f62cbe00640 1 -- 172.21.15.83:0/2040211661 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x560cb66aaa80 con 0x560cb67bfab0 2024-04-24T09:05:51.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.372+0000 7f62cbe00640 10 client.20721 handle_command_reply: tid=0 2024-04-24T09:05:51.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.372+0000 7f62f3fc8740 1 client.20721 shutdown 2024-04-24T09:05:51.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.372+0000 7f62f3fc8740 1 -- 172.21.15.83:0/2040211661 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560cb673d860 msgr2=0x560cb6750f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:51.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.372+0000 7f62f3fc8740 1 --2- 172.21.15.83:0/2040211661 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560cb673d860 0x560cb6750f10 secure :-1 s=READY pgs=2162 cs=0 l=1 rev1=1 crypto rx=0x7f62e4012870 tx=0x7f62e40096a0 comp rx=0 tx=0).stop 2024-04-24T09:05:51.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.372+0000 7f62f3fc8740 1 -- 172.21.15.83:0/2040211661 shutdown_connections 2024-04-24T09:05:51.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.372+0000 7f62f3fc8740 1 --2- 172.21.15.83:0/2040211661 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x560cb67bfab0 0x560cb67c1ed0 crc :-1 s=READY pgs=370 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:51.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.372+0000 7f62f3fc8740 1 --2- 172.21.15.83:0/2040211661 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560cb6748850 0x560cb6742a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:51.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.372+0000 7f62f3fc8740 1 --2- 172.21.15.83:0/2040211661 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560cb673e180 0x560cb6742330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:51.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.372+0000 7f62f3fc8740 1 --2- 172.21.15.83:0/2040211661 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560cb673d860 0x560cb6750f10 unknown :-1 s=CLOSED pgs=2162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:51.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.372+0000 7f62f3fc8740 1 -- 172.21.15.83:0/2040211661 >> 172.21.15.83:0/2040211661 conn(0x560cb6739b80 msgr2=0x560cb6738c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:51.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.372+0000 7f62cbe00640 0 client.20721 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:05:51.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.373+0000 7f62f3fc8740 1 -- 172.21.15.83:0/2040211661 shutdown_connections 2024-04-24T09:05:51.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.373+0000 7f62f3fc8740 1 -- 172.21.15.83:0/2040211661 wait complete. 2024-04-24T09:05:51.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.373+0000 7f62f3fc8740 20 client.20721 trim_cache size 0 max 16384 2024-04-24T09:05:51.373 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T09:05:51.373 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T09:05:51.373 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-04-24T09:05:51.373 INFO:teuthology.orchestra.run.smithi083.stdout: "a5c8b530-1a53-40d0-ab6c-94c416568360": { 2024-04-24T09:05:51.373 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-04-24T09:05:51.373 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "a5c8b530-1a53-40d0-ab6c-94c416568360", 2024-04-24T09:05:51.373 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-04-24T09:05:51.373 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:05:51.373 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:05:51.374 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T09:05:51.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.374+0000 7f62f1000640 1 -- 172.21.15.83:0/1733726767 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f62d00ac700 msgr2=0x7f62d00aebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:51.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.374+0000 7f62f1000640 1 --2- 172.21.15.83:0/1733726767 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f62d00ac700 0x7f62d00aebc0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x560cb673ee20 tx=0x7f62dc0f4da0 comp rx=0 tx=0).stop 2024-04-24T09:05:51.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.374+0000 7f62f1000640 1 -- 172.21.15.83:0/1733726767 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f62ec1011a0 msgr2=0x7f62ec1a0680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:51.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.374+0000 7f62f1000640 1 --2- 172.21.15.83:0/1733726767 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f62ec1011a0 0x7f62ec1a0680 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f62dc005ab0 tx=0x7f62dc003c80 comp rx=0 tx=0).stop 2024-04-24T09:05:51.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.374+0000 7f62f1000640 1 -- 172.21.15.83:0/1733726767 shutdown_connections 2024-04-24T09:05:51.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.374+0000 7f62f1000640 1 --2- 172.21.15.83:0/1733726767 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f62d00ac700 0x7f62d00aebc0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:51.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.374+0000 7f62f1000640 1 --2- 172.21.15.83:0/1733726767 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f62ec19ae20 0x7f62ec19b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:51.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.374+0000 7f62f1000640 1 --2- 172.21.15.83:0/1733726767 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62ec106220 0x7f62ec1a0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:51.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.374+0000 7f62f1000640 1 --2- 172.21.15.83:0/1733726767 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f62ec1011a0 0x7f62ec1a0680 secure :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f62dc005ab0 tx=0x7f62dc003c80 comp rx=0 tx=0).stop 2024-04-24T09:05:51.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.374+0000 7f62f1000640 1 -- 172.21.15.83:0/1733726767 >> 172.21.15.83:0/1733726767 conn(0x7f62ec078060 msgr2=0x7f62ec104b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:51.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.375+0000 7f62f1000640 1 -- 172.21.15.83:0/1733726767 shutdown_connections 2024-04-24T09:05:51.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:05:51.375+0000 7f62f1000640 1 -- 172.21.15.83:0/1733726767 wait complete. 2024-04-24T09:05:51.398 INFO:tasks.cephfs.filesystem:scrub status for tag:a5c8b530-1a53-40d0-ab6c-94c416568360 - {'path': '/', 'tag': 'a5c8b530-1a53-40d0-ab6c-94c416568360', 'options': 'recursive,force'} 2024-04-24T09:05:52.154 INFO:tasks.workunit.client.0.smithi083.stdout: 10 2952 14.38 MB/sec warmup 23 sec latency 80.676 ms 2024-04-24T09:05:52.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:05:51 smithi138 ceph-mon[31441]: pgmap v446: 129 pgs: 129 active+clean; 279 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 735 KiB/s rd, 1.8 MiB/s wr, 219 op/s 2024-04-24T09:05:52.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:05:51 smithi138 ceph-mon[31441]: scrub queued for path: / 2024-04-24T09:05:52.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:05:51 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:05:52.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:05:51 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:05:52.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:05:51 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:05:52.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:05:51 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:05:52.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:05:51 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:05:52.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:05:51 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:05:52.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:05:51 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:05:52.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:05:51 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:05:52.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:05:51 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:05:52.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:05:51 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:05:52.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:05:51 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:05:52.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:05:51 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:05:52.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:05:51 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:05:52.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:05:51 smithi189 ceph-mon[31791]: pgmap v446: 129 pgs: 129 active+clean; 279 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 735 KiB/s rd, 1.8 MiB/s wr, 219 op/s 2024-04-24T09:05:52.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:05:51 smithi189 ceph-mon[31791]: scrub queued for path: / 2024-04-24T09:05:52.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:05:51 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:05:52.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:05:51 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:05:52.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:05:51 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:05:52.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:05:51 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:05:52.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:05:51 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:05:52.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:05:51 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:05:52.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:05:51 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:05:52.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:05:51 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:05:52.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:05:51 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:05:52.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:05:51 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:05:52.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:05:51 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:05:52.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:05:51 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:05:52.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:05:51 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:05:52.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:05:51 smithi083 ceph-mon[27907]: pgmap v446: 129 pgs: 129 active+clean; 279 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 735 KiB/s rd, 1.8 MiB/s wr, 219 op/s 2024-04-24T09:05:52.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:05:51 smithi083 ceph-mon[27907]: scrub queued for path: / 2024-04-24T09:05:52.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:05:51 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:05:52.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:05:51 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:05:52.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:05:51 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:05:52.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:05:51 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:05:52.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:05:51 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:05:52.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:05:51 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:05:52.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:05:51 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:05:52.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:05:51 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:05:52.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:05:51 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:05:52.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:05:51 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:05:52.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:05:51 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:05:52.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:05:51 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:05:52.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:05:51 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:05:53.154 INFO:tasks.workunit.client.0.smithi083.stdout: 10 3132 14.40 MB/sec warmup 24 sec latency 173.461 ms 2024-04-24T09:05:54.154 INFO:tasks.workunit.client.0.smithi083.stdout: 10 3338 14.27 MB/sec warmup 25 sec latency 98.218 ms 2024-04-24T09:05:54.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:05:53 smithi138 ceph-mon[31441]: pgmap v447: 129 pgs: 129 active+clean; 297 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.2 MiB/s wr, 344 op/s 2024-04-24T09:05:54.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:05:53 smithi189 ceph-mon[31791]: pgmap v447: 129 pgs: 129 active+clean; 297 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.2 MiB/s wr, 344 op/s 2024-04-24T09:05:54.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:05:53 smithi083 ceph-mon[27907]: pgmap v447: 129 pgs: 129 active+clean; 297 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.2 MiB/s wr, 344 op/s 2024-04-24T09:05:55.154 INFO:tasks.workunit.client.0.smithi083.stdout: 10 3427 13.95 MB/sec warmup 26 sec latency 75.967 ms 2024-04-24T09:05:56.154 INFO:tasks.workunit.client.0.smithi083.stdout: 10 3524 13.70 MB/sec warmup 27 sec latency 65.311 ms 2024-04-24T09:05:56.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:05:55 smithi138 ceph-mon[31441]: pgmap v448: 129 pgs: 129 active+clean; 297 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 721 KiB/s rd, 3.0 MiB/s wr, 277 op/s 2024-04-24T09:05:56.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:05:55 smithi189 ceph-mon[31791]: pgmap v448: 129 pgs: 129 active+clean; 297 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 721 KiB/s rd, 3.0 MiB/s wr, 277 op/s 2024-04-24T09:05:56.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:05:55 smithi083 ceph-mon[27907]: pgmap v448: 129 pgs: 129 active+clean; 297 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 721 KiB/s rd, 3.0 MiB/s wr, 277 op/s 2024-04-24T09:05:57.154 INFO:tasks.workunit.client.0.smithi083.stdout: 10 3650 13.62 MB/sec warmup 28 sec latency 64.221 ms 2024-04-24T09:05:57.618 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.619+0000 7ff5b5c00640 1 -- 172.21.15.83:0/3802549543 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff5b0102c00 msgr2=0x7ff5b0106c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:57.618 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.619+0000 7ff5b5c00640 1 --2- 172.21.15.83:0/3802549543 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff5b0102c00 0x7ff5b0106c40 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff5a0009ca0 tx=0x7ff5a002a8c0 comp rx=0 tx=0).stop 2024-04-24T09:05:57.618 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.619+0000 7ff5b5c00640 1 -- 172.21.15.83:0/3802549543 shutdown_connections 2024-04-24T09:05:57.618 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.619+0000 7ff5b5c00640 1 --2- 172.21.15.83:0/3802549543 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff5b0107320 0x7ff5b010eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:57.618 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.619+0000 7ff5b5c00640 1 --2- 172.21.15.83:0/3802549543 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff5b0102c00 0x7ff5b0106c40 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:57.618 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.619+0000 7ff5b5c00640 1 --2- 172.21.15.83:0/3802549543 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5b0102250 0x7ff5b0102630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:57.619 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.619+0000 7ff5b5c00640 1 -- 172.21.15.83:0/3802549543 >> 172.21.15.83:0/3802549543 conn(0x7ff5b00fdab0 msgr2=0x7ff5b00ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:57.619 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.619+0000 7ff5b5c00640 1 -- 172.21.15.83:0/3802549543 shutdown_connections 2024-04-24T09:05:57.619 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.619+0000 7ff5b5c00640 1 -- 172.21.15.83:0/3802549543 wait complete. 2024-04-24T09:05:57.620 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.621+0000 7ff5b5c00640 1 Processor -- start 2024-04-24T09:05:57.620 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.622+0000 7ff5b5c00640 1 -- start start 2024-04-24T09:05:57.621 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.622+0000 7ff5b5c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff5b0102250 0x7ff5b0196600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:57.622 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.622+0000 7ff5b5c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff5b0107320 0x7ff5b0196b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:57.622 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.622+0000 7ff5b5c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5b019aec0 0x7ff5b019b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:57.622 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.623+0000 7ff5aea00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff5b0102250 0x7ff5b0196600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:57.622 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.623+0000 7ff5aea00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff5b0102250 0x7ff5b0196600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:35048/0 (socket says 172.21.15.83:35048) 2024-04-24T09:05:57.622 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.623+0000 7ff5ae000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff5b0107320 0x7ff5b0196b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:57.623 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.623+0000 7ff5ae000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff5b0107320 0x7ff5b0196b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:39588/0 (socket says 172.21.15.83:39588) 2024-04-24T09:05:57.623 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.623+0000 7ff5af400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5b019aec0 0x7ff5b019b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:57.623 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.623+0000 7ff5af400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5b019aec0 0x7ff5b019b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43392/0 (socket says 172.21.15.83:43392) 2024-04-24T09:05:57.623 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.623+0000 7ff5aea00640 1 -- 172.21.15.83:0/2817185233 learned_addr learned my addr 172.21.15.83:0/2817185233 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:05:57.623 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.623+0000 7ff5b5c00640 1 -- 172.21.15.83:0/2817185233 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b01100c0 con 0x7ff5b019aec0 2024-04-24T09:05:57.623 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.623+0000 7ff5b5c00640 1 -- 172.21.15.83:0/2817185233 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b010ff40 con 0x7ff5b0102250 2024-04-24T09:05:57.623 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.623+0000 7ff5b5c00640 1 -- 172.21.15.83:0/2817185233 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b0110240 con 0x7ff5b0107320 2024-04-24T09:05:57.623 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.624+0000 7ff5ae000640 1 -- 172.21.15.83:0/2817185233 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff5b0102250 msgr2=0x7ff5b0196600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:57.623 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.624+0000 7ff5ae000640 1 --2- 172.21.15.83:0/2817185233 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff5b0102250 0x7ff5b0196600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:57.623 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.624+0000 7ff5ae000640 1 -- 172.21.15.83:0/2817185233 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5b019aec0 msgr2=0x7ff5b019b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:57.624 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.625+0000 7ff5ae000640 1 --2- 172.21.15.83:0/2817185233 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5b019aec0 0x7ff5b019b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:57.624 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.625+0000 7ff5ae000640 1 -- 172.21.15.83:0/2817185233 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5b019b9b0 con 0x7ff5b0107320 2024-04-24T09:05:57.624 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.625+0000 7ff5af400640 1 --2- 172.21.15.83:0/2817185233 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5b019aec0 0x7ff5b019b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:05:57.624 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.625+0000 7ff5ae000640 1 --2- 172.21.15.83:0/2817185233 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff5b0107320 0x7ff5b0196b40 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff5a002aeb0 tx=0x7ff5a0004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:57.626 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.626+0000 7ff5a7400640 1 -- 172.21.15.83:0/2817185233 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a0003b70 con 0x7ff5b0107320 2024-04-24T09:05:57.626 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.626+0000 7ff5a7400640 1 -- 172.21.15.83:0/2817185233 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff5a0003d10 con 0x7ff5b0107320 2024-04-24T09:05:57.626 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.626+0000 7ff5a7400640 1 -- 172.21.15.83:0/2817185233 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a00309e0 con 0x7ff5b0107320 2024-04-24T09:05:57.626 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.626+0000 7ff5b5c00640 1 -- 172.21.15.83:0/2817185233 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5b019bc40 con 0x7ff5b0107320 2024-04-24T09:05:57.626 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.627+0000 7ff5b5c00640 1 -- 172.21.15.83:0/2817185233 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5b01a33b0 con 0x7ff5b0107320 2024-04-24T09:05:57.634 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.635+0000 7ff5a7400640 1 -- 172.21.15.83:0/2817185233 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff5a0030b80 con 0x7ff5b0107320 2024-04-24T09:05:57.634 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.635+0000 7ff5b8b81740 1 Processor -- start 2024-04-24T09:05:57.634 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.636+0000 7ff5b8b81740 1 -- start start 2024-04-24T09:05:57.637 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.636+0000 7ff5b8b81740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5632daef21b0 0x5632daef2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:57.637 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.636+0000 7ff5b8b81740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5632daef2ad0 0x5632daefce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:57.637 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.636+0000 7ff5b8b81740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5632daefd390 0x5632daeff780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:57.637 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.636+0000 7ff5b8b81740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5632dae58900 con 0x5632daef2ad0 2024-04-24T09:05:57.637 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.636+0000 7ff5b8b81740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5632dadfef20 con 0x5632daef21b0 2024-04-24T09:05:57.637 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.636+0000 7ff5b8b81740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5632dae548c0 con 0x5632daefd390 2024-04-24T09:05:57.637 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.637+0000 7ff5af400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5632daef21b0 0x5632daef2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:57.637 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.637+0000 7ff5af400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5632daef21b0 0x5632daef2590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:35050/0 (socket says 172.21.15.83:35050) 2024-04-24T09:05:57.637 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.637+0000 7ff5af400640 1 -- 172.21.15.83:0/1943255588 learned_addr learned my addr 172.21.15.83:0/1943255588 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:05:57.638 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.637+0000 7ff5aea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5632daef2ad0 0x5632daefce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:57.638 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.637+0000 7ff5ae000640 1 --2- 172.21.15.83:0/1943255588 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5632daefd390 0x5632daeff780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:57.638 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.638+0000 7ff5af400640 1 -- 172.21.15.83:0/1943255588 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5632daefd390 msgr2=0x5632daeff780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:57.638 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.638+0000 7ff5af400640 1 --2- 172.21.15.83:0/1943255588 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5632daefd390 0x5632daeff780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:57.638 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.638+0000 7ff5af400640 1 -- 172.21.15.83:0/1943255588 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5632daef2ad0 msgr2=0x5632daefce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:57.638 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.638+0000 7ff5af400640 1 --2- 172.21.15.83:0/1943255588 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5632daef2ad0 0x5632daefce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:57.638 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.638+0000 7ff5af400640 1 -- 172.21.15.83:0/1943255588 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632dae23e10 con 0x5632daef21b0 2024-04-24T09:05:57.638 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.638+0000 7ff5a7400640 1 --2- 172.21.15.83:0/2817185233 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff5940a3ed0 0x7ff5940a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:57.638 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.638+0000 7ff5aea00640 1 --2- 172.21.15.83:0/2817185233 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff5940a3ed0 0x7ff5940a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:57.638 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.638+0000 7ff5a7400640 1 -- 172.21.15.83:0/2817185233 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7ff5a00f1ea0 con 0x7ff5b0107320 2024-04-24T09:05:57.640 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.639+0000 7ff5af400640 1 --2- 172.21.15.83:0/1943255588 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5632daef21b0 0x5632daef2590 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff598011440 tx=0x7ff598011900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86bf6ef2691e7f5c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:57.640 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.639+0000 7ff59fe00640 1 -- 172.21.15.83:0/1943255588 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff598004020 con 0x5632daef21b0 2024-04-24T09:05:57.640 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.639+0000 7ff59fe00640 1 -- 172.21.15.83:0/1943255588 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff5980041c0 con 0x5632daef21b0 2024-04-24T09:05:57.640 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.639+0000 7ff59fe00640 1 -- 172.21.15.83:0/1943255588 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff59800f7a0 con 0x5632daef21b0 2024-04-24T09:05:57.640 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.640+0000 7ff5aea00640 1 --2- 172.21.15.83:0/2817185233 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff5940a3ed0 0x7ff5940a6390 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x5632daef3750 tx=0x7ff5a8005f90 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:57.640 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.640+0000 7ff5b8b81740 1 -- 172.21.15.83:0/1943255588 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5632daef21b0 msgr2=0x5632daef2590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:57.640 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.640+0000 7ff5b8b81740 1 --2- 172.21.15.83:0/1943255588 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5632daef21b0 0x5632daef2590 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff598011440 tx=0x7ff598011900 comp rx=0 tx=0).stop 2024-04-24T09:05:57.640 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.640+0000 7ff5b8b81740 1 -- 172.21.15.83:0/1943255588 shutdown_connections 2024-04-24T09:05:57.640 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.640+0000 7ff5b8b81740 1 --2- 172.21.15.83:0/1943255588 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5632daefd390 0x5632daeff780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:57.640 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.640+0000 7ff5b8b81740 1 --2- 172.21.15.83:0/1943255588 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5632daef2ad0 0x5632daefce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:57.641 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.640+0000 7ff5b8b81740 1 --2- 172.21.15.83:0/1943255588 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5632daef21b0 0x5632daef2590 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:57.641 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.640+0000 7ff5b8b81740 1 -- 172.21.15.83:0/1943255588 >> 172.21.15.83:0/1943255588 conn(0x5632daeee6c0 msgr2=0x5632daeeeaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:57.641 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.640+0000 7ff5b8b81740 1 -- 172.21.15.83:0/1943255588 shutdown_connections 2024-04-24T09:05:57.641 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.641+0000 7ff5b8b81740 1 -- 172.21.15.83:0/1943255588 wait complete. 2024-04-24T09:05:57.641 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.641+0000 7ff5b8b81740 1 -- start start 2024-04-24T09:05:57.642 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.641+0000 7ff5b8b81740 1 Processor -- start 2024-04-24T09:05:57.642 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.642+0000 7ff5b8b81740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5632daef21b0 0x5632daefa3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:57.642 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.642+0000 7ff5b8b81740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5632daef2ad0 0x5632daefa910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:57.642 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.642+0000 7ff5b8b81740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5632daefd390 0x5632daefae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:57.642 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.642+0000 7ff5b8b81740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5632daf087d0 con 0x5632daefd390 2024-04-24T09:05:57.642 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.642+0000 7ff5b8b81740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5632dae122c0 con 0x5632daef21b0 2024-04-24T09:05:57.642 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.642+0000 7ff5b8b81740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5632daf08950 con 0x5632daef2ad0 2024-04-24T09:05:57.642 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.642+0000 7ff5b8b81740 10 client.0 _ll_register_callbacks cb 0x5632dab6aa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:05:57.642 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.642+0000 7ff5aea00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5632daef21b0 0x5632daefa3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:57.643 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.642+0000 7ff5ae000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5632daef2ad0 0x5632daefa910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:57.643 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.642+0000 7ff5aea00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5632daef21b0 0x5632daefa3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:35054/0 (socket says 172.21.15.83:35054) 2024-04-24T09:05:57.643 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.642+0000 7ff5aea00640 1 -- 172.21.15.83:0/883094193 learned_addr learned my addr 172.21.15.83:0/883094193 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:05:57.643 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.643+0000 7ff5af400640 1 --2- 172.21.15.83:0/883094193 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5632daefd390 0x5632daefae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:57.643 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.643+0000 7ff5aea00640 1 -- 172.21.15.83:0/883094193 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5632daef2ad0 msgr2=0x5632daefa910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:57.643 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.643+0000 7ff5aea00640 1 --2- 172.21.15.83:0/883094193 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5632daef2ad0 0x5632daefa910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:57.643 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.643+0000 7ff5aea00640 1 -- 172.21.15.83:0/883094193 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5632daefd390 msgr2=0x5632daefae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:57.643 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.643+0000 7ff5aea00640 1 --2- 172.21.15.83:0/883094193 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5632daefd390 0x5632daefae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:57.643 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.643+0000 7ff5aea00640 1 -- 172.21.15.83:0/883094193 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632dae22fd0 con 0x5632daef21b0 2024-04-24T09:05:57.643 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.643+0000 7ff5af400640 1 --2- 172.21.15.83:0/883094193 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5632daefd390 0x5632daefae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:05:57.645 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.643+0000 7ff5aea00640 1 --2- 172.21.15.83:0/883094193 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5632daef21b0 0x5632daefa3d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff5a801e9d0 tx=0x7ff5a8004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f712eb1d19cb9cf4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:57.645 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.643+0000 7ff59cc00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:05:57.645 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.644+0000 7ff59cc00640 1 -- 172.21.15.83:0/883094193 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a8004020 con 0x5632daef21b0 2024-04-24T09:05:57.645 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.644+0000 7ff59cc00640 1 -- 172.21.15.83:0/883094193 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7ff5a8018030 con 0x5632daef21b0 2024-04-24T09:05:57.645 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.644+0000 7ff5b8b81740 1 -- 172.21.15.83:0/883094193 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5980041c0 con 0x5632daef21b0 2024-04-24T09:05:57.645 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.644+0000 7ff59cc00640 1 -- 172.21.15.83:0/883094193 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7ff5a8034050 con 0x5632daef21b0 2024-04-24T09:05:57.645 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.644+0000 7ff5b8b81740 10 client.37877 fetch_fsmap learned FSMap version 47 2024-04-24T09:05:57.645 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.644+0000 7ff5b8b81740 1 -- 172.21.15.83:0/883094193 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5632dab6b210 con 0x5632daef21b0 2024-04-24T09:05:57.645 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.644+0000 7ff59cc00640 1 -- 172.21.15.83:0/883094193 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff5980041c0 con 0x5632daef21b0 2024-04-24T09:05:57.645 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.645+0000 7ff59cc00640 1 -- 172.21.15.83:0/883094193 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 47) ==== 8398+0+0 (secure 0 0 0) 0x7ff5a8029420 con 0x5632daef21b0 2024-04-24T09:05:57.645 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.645+0000 7ff5b8b81740 10 client.37877 fetch_fsmap finished waiting for FSMap version 47 2024-04-24T09:05:57.645 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.645+0000 7ff5b8b81740 10 client.37877 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:05:57.646 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.645+0000 7ff5b8b81740 20 client.37877 populate_metadata read hostname 'smithi083' 2024-04-24T09:05:57.646 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.645+0000 7ff5b8b81740 1 --2- 172.21.15.83:0/883094193 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5632daf26f10 0x5632daf29340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:57.646 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.645+0000 7ff5b8b81740 4 client.37877 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:05:57.646 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.645+0000 7ff5b8b81740 1 -- 172.21.15.83:0/883094193 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5632dae66320 con 0x5632daf26f10 2024-04-24T09:05:57.646 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.645+0000 7ff5ae000640 1 --2- 172.21.15.83:0/883094193 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5632daf26f10 0x5632daf29340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:57.646 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.646+0000 7ff5ae000640 1 --2- 172.21.15.83:0/883094193 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5632daf26f10 0x5632daf29340 crc :-1 s=READY pgs=371 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ebf908e9a58fd42e server_cookie=31ee5b0ce77eaf46 in_seq=0 out_seq=0 2024-04-24T09:05:57.646 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.646+0000 7ff59cc00640 10 client.37877 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:05:57.647 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.647+0000 7ff59cc00640 1 -- 172.21.15.83:0/883094193 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5632dae66320 con 0x5632daf26f10 2024-04-24T09:05:57.647 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.647+0000 7ff59cc00640 10 client.37877 handle_command_reply: tid=0 2024-04-24T09:05:57.647 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.647+0000 7ff5b8b81740 1 client.37877 shutdown 2024-04-24T09:05:57.647 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.648+0000 7ff5b8b81740 1 -- 172.21.15.83:0/883094193 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5632daef21b0 msgr2=0x5632daefa3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:57.647 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.648+0000 7ff5b8b81740 1 --2- 172.21.15.83:0/883094193 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5632daef21b0 0x5632daefa3d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff5a801e9d0 tx=0x7ff5a8004730 comp rx=0 tx=0).stop 2024-04-24T09:05:57.647 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.648+0000 7ff5b8b81740 1 -- 172.21.15.83:0/883094193 shutdown_connections 2024-04-24T09:05:57.647 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.648+0000 7ff5b8b81740 1 --2- 172.21.15.83:0/883094193 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5632daf26f10 0x5632daf29340 crc :-1 s=READY pgs=371 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:57.647 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.648+0000 7ff5b8b81740 1 --2- 172.21.15.83:0/883094193 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5632daefd390 0x5632daefae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:57.648 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.648+0000 7ff5b8b81740 1 --2- 172.21.15.83:0/883094193 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5632daef2ad0 0x5632daefa910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:57.648 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.648+0000 7ff59cc00640 0 client.37877 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:05:57.648 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.648+0000 7ff5b8b81740 1 --2- 172.21.15.83:0/883094193 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5632daef21b0 0x5632daefa3d0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:57.648 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.648+0000 7ff5b8b81740 1 -- 172.21.15.83:0/883094193 >> 172.21.15.83:0/883094193 conn(0x5632daeee6c0 msgr2=0x5632daef4440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:57.648 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.648+0000 7ff5b8b81740 1 -- 172.21.15.83:0/883094193 shutdown_connections 2024-04-24T09:05:57.648 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.648+0000 7ff5b8b81740 1 -- 172.21.15.83:0/883094193 wait complete. 2024-04-24T09:05:57.648 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.648+0000 7ff5b8b81740 20 client.37877 trim_cache size 0 max 16384 2024-04-24T09:05:57.675 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.675+0000 7ff5b8b81740 1 Processor -- start 2024-04-24T09:05:57.675 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.675+0000 7ff5b8b81740 1 -- start start 2024-04-24T09:05:57.675 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.675+0000 7ff5b8b81740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5632daef21b0 0x5632daf11690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:57.675 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.675+0000 7ff5b8b81740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5632daef2ad0 0x5632daf2ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:57.676 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.675+0000 7ff5b8b81740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5632daefd390 0x5632daf2f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:57.676 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.675+0000 7ff5b8b81740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b01a4b70 con 0x5632daef21b0 2024-04-24T09:05:57.676 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.675+0000 7ff5b8b81740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b01a6830 con 0x5632daefd390 2024-04-24T09:05:57.676 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.675+0000 7ff5b8b81740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b01a5240 con 0x5632daef2ad0 2024-04-24T09:05:57.676 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.676+0000 7ff5aea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5632daef21b0 0x5632daf11690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:57.676 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.676+0000 7ff5af400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5632daefd390 0x5632daf2f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:57.676 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.676+0000 7ff5aea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5632daef21b0 0x5632daf11690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43424/0 (socket says 172.21.15.83:43424) 2024-04-24T09:05:57.676 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.676+0000 7ff5aea00640 1 -- 172.21.15.83:0/1544292456 learned_addr learned my addr 172.21.15.83:0/1544292456 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:05:57.676 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.676+0000 7ff5ae000640 1 --2- 172.21.15.83:0/1544292456 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5632daef2ad0 0x5632daf2ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:57.676 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.676+0000 7ff5aea00640 1 -- 172.21.15.83:0/1544292456 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5632daef2ad0 msgr2=0x5632daf2ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:57.676 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.676+0000 7ff5aea00640 1 --2- 172.21.15.83:0/1544292456 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5632daef2ad0 0x5632daf2ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:57.676 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.676+0000 7ff5aea00640 1 -- 172.21.15.83:0/1544292456 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5632daefd390 msgr2=0x5632daf2f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:57.676 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.676+0000 7ff5aea00640 1 --2- 172.21.15.83:0/1544292456 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5632daefd390 0x5632daf2f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:57.677 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.676+0000 7ff5aea00640 1 -- 172.21.15.83:0/1544292456 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632dac37620 con 0x5632daef21b0 2024-04-24T09:05:57.678 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.677+0000 7ff5aea00640 1 --2- 172.21.15.83:0/1544292456 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5632daef21b0 0x5632daf11690 secure :-1 s=READY pgs=2163 cs=0 l=1 rev1=1 crypto rx=0x7ff5a8019040 tx=0x7ff5a80092a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3a0d3d298e5adf9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:57.678 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.677+0000 7ff59f400640 1 -- 172.21.15.83:0/1544292456 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a8029420 con 0x5632daef21b0 2024-04-24T09:05:57.678 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.677+0000 7ff59f400640 1 -- 172.21.15.83:0/1544292456 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff5a80295c0 con 0x5632daef21b0 2024-04-24T09:05:57.678 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.678+0000 7ff59f400640 1 -- 172.21.15.83:0/1544292456 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a80052f0 con 0x5632daef21b0 2024-04-24T09:05:57.678 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.679+0000 7ff5b8b81740 1 -- 172.21.15.83:0/1544292456 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5632daef21b0 msgr2=0x5632daf11690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:57.678 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.679+0000 7ff5b8b81740 1 --2- 172.21.15.83:0/1544292456 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5632daef21b0 0x5632daf11690 secure :-1 s=READY pgs=2163 cs=0 l=1 rev1=1 crypto rx=0x7ff5a8019040 tx=0x7ff5a80092a0 comp rx=0 tx=0).stop 2024-04-24T09:05:57.678 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.679+0000 7ff5b8b81740 1 -- 172.21.15.83:0/1544292456 shutdown_connections 2024-04-24T09:05:57.678 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.679+0000 7ff5b8b81740 1 --2- 172.21.15.83:0/1544292456 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5632daefd390 0x5632daf2f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:57.678 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.679+0000 7ff5b8b81740 1 --2- 172.21.15.83:0/1544292456 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5632daef2ad0 0x5632daf2ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:57.678 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.679+0000 7ff5b8b81740 1 --2- 172.21.15.83:0/1544292456 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5632daef21b0 0x5632daf11690 unknown :-1 s=CLOSED pgs=2163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:57.678 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.679+0000 7ff5b8b81740 1 -- 172.21.15.83:0/1544292456 >> 172.21.15.83:0/1544292456 conn(0x5632daeee6c0 msgr2=0x5632daef4440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:57.679 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.679+0000 7ff5b8b81740 1 -- 172.21.15.83:0/1544292456 shutdown_connections 2024-04-24T09:05:57.679 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.679+0000 7ff5b8b81740 1 -- 172.21.15.83:0/1544292456 wait complete. 2024-04-24T09:05:57.679 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.680+0000 7ff5b8b81740 1 -- start start 2024-04-24T09:05:57.679 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.680+0000 7ff5b8b81740 1 Processor -- start 2024-04-24T09:05:57.680 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.681+0000 7ff5b8b81740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5632daef21b0 0x5632daf0e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:57.680 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.681+0000 7ff5b8b81740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5632daef2ad0 0x5632daf093b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:57.680 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.681+0000 7ff5b8b81740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5632daefd390 0x5632daf099c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:57.681 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.681+0000 7ff5b8b81740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b01a60a0 con 0x5632daefd390 2024-04-24T09:05:57.681 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.681+0000 7ff5b8b81740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b01a5970 con 0x5632daef21b0 2024-04-24T09:05:57.681 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.681+0000 7ff5b8b81740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b00736c0 con 0x5632daef2ad0 2024-04-24T09:05:57.681 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.681+0000 7ff5b8b81740 10 client.0 _ll_register_callbacks cb 0x7ff5a8009f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:05:57.681 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.681+0000 7ff5aea00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5632daef21b0 0x5632daf0e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:57.681 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.681+0000 7ff5aea00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5632daef21b0 0x5632daf0e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:35066/0 (socket says 172.21.15.83:35066) 2024-04-24T09:05:57.681 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.681+0000 7ff5aea00640 1 -- 172.21.15.83:0/1100420472 learned_addr learned my addr 172.21.15.83:0/1100420472 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:05:57.681 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.681+0000 7ff5af400640 1 --2- 172.21.15.83:0/1100420472 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5632daefd390 0x5632daf099c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:57.681 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.682+0000 7ff5ae000640 1 --2- 172.21.15.83:0/1100420472 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5632daef2ad0 0x5632daf093b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:57.681 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.682+0000 7ff5aea00640 1 -- 172.21.15.83:0/1100420472 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5632daef2ad0 msgr2=0x5632daf093b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:57.682 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.682+0000 7ff5aea00640 1 --2- 172.21.15.83:0/1100420472 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5632daef2ad0 0x5632daf093b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:57.682 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.682+0000 7ff5aea00640 1 -- 172.21.15.83:0/1100420472 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5632daefd390 msgr2=0x5632daf099c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:05:57.682 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.682+0000 7ff5aea00640 1 --2- 172.21.15.83:0/1100420472 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5632daefd390 0x5632daf099c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:57.682 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.682+0000 7ff5aea00640 1 -- 172.21.15.83:0/1100420472 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632daf0a190 con 0x5632daef21b0 2024-04-24T09:05:57.682 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.682+0000 7ff5ae000640 1 --2- 172.21.15.83:0/1100420472 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5632daef2ad0 0x5632daf093b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:05:57.682 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.682+0000 7ff5af400640 1 --2- 172.21.15.83:0/1100420472 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5632daefd390 0x5632daf099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:05:57.682 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.682+0000 7ff5aea00640 1 --2- 172.21.15.83:0/1100420472 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5632daef21b0 0x5632daf0e040 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff5a801e9d0 tx=0x7ff5a8029c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2bc426250638a5e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:05:57.682 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.682+0000 7ff59cc00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:05:57.683 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.683+0000 7ff59cc00640 1 -- 172.21.15.83:0/1100420472 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a8046020 con 0x5632daef21b0 2024-04-24T09:05:57.683 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.684+0000 7ff59cc00640 1 -- 172.21.15.83:0/1100420472 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9583+0+0 (secure 0 0 0) 0x7ff5a8005230 con 0x5632daef21b0 2024-04-24T09:05:57.683 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.684+0000 7ff5b8b81740 1 -- 172.21.15.83:0/1100420472 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5a80295c0 con 0x5632daef21b0 2024-04-24T09:05:57.683 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.684+0000 7ff59cc00640 1 -- 172.21.15.83:0/1100420472 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7ff5a8029420 con 0x5632daef21b0 2024-04-24T09:05:57.683 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.684+0000 7ff5b8b81740 10 client.37883 fetch_fsmap learned FSMap version 47 2024-04-24T09:05:57.683 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.684+0000 7ff5b8b81740 1 -- 172.21.15.83:0/1100420472 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5632daf8d5a0 con 0x5632daef21b0 2024-04-24T09:05:57.684 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.685+0000 7ff59cc00640 1 -- 172.21.15.83:0/1100420472 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff5a80295c0 con 0x5632daef21b0 2024-04-24T09:05:57.684 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.685+0000 7ff59cc00640 1 -- 172.21.15.83:0/1100420472 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 47) ==== 8398+0+0 (secure 0 0 0) 0x7ff5a8042400 con 0x5632daef21b0 2024-04-24T09:05:57.685 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.686+0000 7ff5b8b81740 10 client.37883 fetch_fsmap finished waiting for FSMap version 47 2024-04-24T09:05:57.685 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.686+0000 7ff5b8b81740 10 client.37883 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:05:57.685 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.686+0000 7ff5b8b81740 20 client.37883 populate_metadata read hostname 'smithi083' 2024-04-24T09:05:57.685 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.686+0000 7ff5b8b81740 1 --2- 172.21.15.83:0/1100420472 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5632daf8dc90 0x5632daf90080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:05:57.685 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.686+0000 7ff5b8b81740 4 client.37883 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T09:05:57.685 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.686+0000 7ff5b8b81740 1 -- 172.21.15.83:0/1100420472 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5632dae60190 con 0x5632daf8dc90 2024-04-24T09:05:57.686 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.687+0000 7ff5ae000640 1 --2- 172.21.15.83:0/1100420472 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5632daf8dc90 0x5632daf90080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:05:57.686 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.687+0000 7ff5ae000640 1 --2- 172.21.15.83:0/1100420472 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5632daf8dc90 0x5632daf90080 crc :-1 s=READY pgs=372 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d3f89fa57fa67bf server_cookie=34f5015a2e846d48 in_seq=0 out_seq=0 2024-04-24T09:05:57.686 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.687+0000 7ff59cc00640 10 client.37883 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:05:57.783 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.784+0000 7ff59cc00640 1 -- 172.21.15.83:0/1100420472 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5632dae60190 con 0x5632daf8dc90 2024-04-24T09:05:57.783 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.784+0000 7ff59cc00640 10 client.37883 handle_command_reply: tid=0 2024-04-24T09:05:57.783 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.784+0000 7ff5b8b81740 1 client.37883 shutdown 2024-04-24T09:05:57.784 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.785+0000 7ff5b8b81740 1 -- 172.21.15.83:0/1100420472 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5632daef21b0 msgr2=0x5632daf0e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:57.784 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.785+0000 7ff5b8b81740 1 --2- 172.21.15.83:0/1100420472 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5632daef21b0 0x5632daf0e040 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff5a801e9d0 tx=0x7ff5a8029c90 comp rx=0 tx=0).stop 2024-04-24T09:05:57.784 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.785+0000 7ff5b8b81740 1 -- 172.21.15.83:0/1100420472 shutdown_connections 2024-04-24T09:05:57.784 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.785+0000 7ff5b8b81740 1 --2- 172.21.15.83:0/1100420472 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5632daf8dc90 0x5632daf90080 crc :-1 s=READY pgs=372 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:57.784 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.785+0000 7ff5b8b81740 1 --2- 172.21.15.83:0/1100420472 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5632daefd390 0x5632daf099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:57.784 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.785+0000 7ff5b8b81740 1 --2- 172.21.15.83:0/1100420472 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5632daef2ad0 0x5632daf093b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:57.785 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.785+0000 7ff5b8b81740 1 --2- 172.21.15.83:0/1100420472 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5632daef21b0 0x5632daf0e040 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:57.785 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.785+0000 7ff59cc00640 0 client.37883 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:05:57.785 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.785+0000 7ff5b8b81740 1 -- 172.21.15.83:0/1100420472 >> 172.21.15.83:0/1100420472 conn(0x5632daeee6c0 msgr2=0x5632daefa370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:57.786 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.786+0000 7ff5b8b81740 1 -- 172.21.15.83:0/1100420472 shutdown_connections 2024-04-24T09:05:57.786 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.787+0000 7ff5b8b81740 1 -- 172.21.15.83:0/1100420472 wait complete. 2024-04-24T09:05:57.786 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.787+0000 7ff5b8b81740 20 client.37883 trim_cache size 0 max 16384 2024-04-24T09:05:57.787 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T09:05:57.787 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T09:05:57.787 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T09:05:57.787 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T09:05:57.788 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.788+0000 7ff5b5c00640 1 -- 172.21.15.83:0/2817185233 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff5940a3ed0 msgr2=0x7ff5940a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:57.788 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.788+0000 7ff5b5c00640 1 --2- 172.21.15.83:0/2817185233 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff5940a3ed0 0x7ff5940a6390 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x5632daef3750 tx=0x7ff5a8005f90 comp rx=0 tx=0).stop 2024-04-24T09:05:57.788 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.788+0000 7ff5b5c00640 1 -- 172.21.15.83:0/2817185233 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff5b0107320 msgr2=0x7ff5b0196b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:05:57.788 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.788+0000 7ff5b5c00640 1 --2- 172.21.15.83:0/2817185233 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff5b0107320 0x7ff5b0196b40 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff5a002aeb0 tx=0x7ff5a0004a40 comp rx=0 tx=0).stop 2024-04-24T09:05:57.788 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.789+0000 7ff5b5c00640 1 -- 172.21.15.83:0/2817185233 shutdown_connections 2024-04-24T09:05:57.788 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.789+0000 7ff5b5c00640 1 --2- 172.21.15.83:0/2817185233 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff5940a3ed0 0x7ff5940a6390 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:57.788 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.789+0000 7ff5b5c00640 1 --2- 172.21.15.83:0/2817185233 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5b019aec0 0x7ff5b019b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:57.788 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.789+0000 7ff5b5c00640 1 --2- 172.21.15.83:0/2817185233 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff5b0107320 0x7ff5b0196b40 secure :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff5a002aeb0 tx=0x7ff5a0004a40 comp rx=0 tx=0).stop 2024-04-24T09:05:57.788 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.789+0000 7ff5b5c00640 1 --2- 172.21.15.83:0/2817185233 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff5b0102250 0x7ff5b0196600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:05:57.788 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.789+0000 7ff5b5c00640 1 -- 172.21.15.83:0/2817185233 >> 172.21.15.83:0/2817185233 conn(0x7ff5b00fdab0 msgr2=0x7ff5b00ffe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:05:57.789 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.790+0000 7ff5b5c00640 1 -- 172.21.15.83:0/2817185233 shutdown_connections 2024-04-24T09:05:57.789 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:05:57.790+0000 7ff5b5c00640 1 -- 172.21.15.83:0/2817185233 wait complete. 2024-04-24T09:05:58.154 INFO:tasks.workunit.client.0.smithi083.stdout: 10 3787 13.57 MB/sec warmup 29 sec latency 115.696 ms 2024-04-24T09:05:58.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:05:57 smithi138 ceph-mon[31441]: pgmap v449: 129 pgs: 129 active+clean; 297 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 721 KiB/s rd, 3.0 MiB/s wr, 277 op/s 2024-04-24T09:05:58.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:05:57 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:05:58.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:05:57 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:05:58.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:05:57 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:05:58.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:05:57 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:05:58.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:05:57 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:05:58.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:05:57 smithi189 ceph-mon[31791]: pgmap v449: 129 pgs: 129 active+clean; 297 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 721 KiB/s rd, 3.0 MiB/s wr, 277 op/s 2024-04-24T09:05:58.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:05:57 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:05:58.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:05:57 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:05:58.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:05:57 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:05:58.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:05:57 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:05:58.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:05:57 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:05:58.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:05:57 smithi083 ceph-mon[27907]: pgmap v449: 129 pgs: 129 active+clean; 297 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 721 KiB/s rd, 3.0 MiB/s wr, 277 op/s 2024-04-24T09:05:58.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:05:57 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:05:58.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:05:57 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:05:58.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:05:57 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:05:58.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:05:57 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:05:58.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:05:57 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:05:59.154 INFO:tasks.workunit.client.0.smithi083.stdout: 10 3903 13.21 MB/sec warmup 30 sec latency 76.988 ms 2024-04-24T09:06:00.155 INFO:tasks.workunit.client.0.smithi083.stdout: 10 4017 12.93 MB/sec warmup 31 sec latency 78.177 ms 2024-04-24T09:06:00.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:05:59 smithi138 ceph-mon[31441]: pgmap v450: 129 pgs: 129 active+clean; 309 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 5.1 MiB/s wr, 355 op/s 2024-04-24T09:06:00.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:05:59 smithi189 ceph-mon[31791]: pgmap v450: 129 pgs: 129 active+clean; 309 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 5.1 MiB/s wr, 355 op/s 2024-04-24T09:06:00.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:05:59 smithi083 ceph-mon[27907]: pgmap v450: 129 pgs: 129 active+clean; 309 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 5.1 MiB/s wr, 355 op/s 2024-04-24T09:06:01.155 INFO:tasks.workunit.client.0.smithi083.stdout: 10 4117 12.58 MB/sec warmup 32 sec latency 126.869 ms 2024-04-24T09:06:02.155 INFO:tasks.workunit.client.0.smithi083.stdout: 10 4233 12.30 MB/sec warmup 33 sec latency 137.624 ms 2024-04-24T09:06:02.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:06:01 smithi138 ceph-mon[31441]: pgmap v451: 129 pgs: 129 active+clean; 309 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 968 KiB/s rd, 4.4 MiB/s wr, 203 op/s 2024-04-24T09:06:02.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:06:01 smithi189 ceph-mon[31791]: pgmap v451: 129 pgs: 129 active+clean; 309 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 968 KiB/s rd, 4.4 MiB/s wr, 203 op/s 2024-04-24T09:06:02.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:06:01 smithi083 ceph-mon[27907]: pgmap v451: 129 pgs: 129 active+clean; 309 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 968 KiB/s rd, 4.4 MiB/s wr, 203 op/s 2024-04-24T09:06:03.155 INFO:tasks.workunit.client.0.smithi083.stdout: 10 4349 12.07 MB/sec warmup 34 sec latency 95.462 ms 2024-04-24T09:06:04.155 INFO:tasks.workunit.client.0.smithi083.stdout: 10 4463 11.89 MB/sec warmup 35 sec latency 84.184 ms 2024-04-24T09:06:04.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:06:03 smithi138 ceph-mon[31441]: pgmap v452: 129 pgs: 129 active+clean; 314 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 6.0 MiB/s wr, 324 op/s 2024-04-24T09:06:04.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:06:03 smithi138 ceph-mon[31441]: osdmap e111: 12 total, 12 up, 12 in 2024-04-24T09:06:04.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:06:03 smithi189 ceph-mon[31791]: pgmap v452: 129 pgs: 129 active+clean; 314 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 6.0 MiB/s wr, 324 op/s 2024-04-24T09:06:04.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:06:03 smithi189 ceph-mon[31791]: osdmap e111: 12 total, 12 up, 12 in 2024-04-24T09:06:04.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:06:03 smithi083 ceph-mon[27907]: pgmap v452: 129 pgs: 129 active+clean; 314 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 6.0 MiB/s wr, 324 op/s 2024-04-24T09:06:04.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:06:03 smithi083 ceph-mon[27907]: osdmap e111: 12 total, 12 up, 12 in 2024-04-24T09:06:05.155 INFO:tasks.workunit.client.0.smithi083.stdout: 10 4571 11.62 MB/sec warmup 36 sec latency 180.291 ms 2024-04-24T09:06:06.155 INFO:tasks.workunit.client.0.smithi083.stdout: 10 4673 11.38 MB/sec warmup 37 sec latency 307.957 ms 2024-04-24T09:06:06.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:06:05 smithi189 ceph-mon[31791]: pgmap v454: 129 pgs: 129 active+clean; 314 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.3 MiB/s wr, 239 op/s 2024-04-24T09:06:06.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:06:05 smithi083 ceph-mon[27907]: pgmap v454: 129 pgs: 129 active+clean; 314 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.3 MiB/s wr, 239 op/s 2024-04-24T09:06:06.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:06:05 smithi138 ceph-mon[31441]: pgmap v454: 129 pgs: 129 active+clean; 314 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.3 MiB/s wr, 239 op/s 2024-04-24T09:06:07.155 INFO:tasks.workunit.client.0.smithi083.stdout: 10 4817 11.28 MB/sec warmup 38 sec latency 95.659 ms 2024-04-24T09:06:08.155 INFO:tasks.workunit.client.0.smithi083.stdout: 10 4963 11.20 MB/sec warmup 39 sec latency 83.018 ms 2024-04-24T09:06:08.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:06:07 smithi083 ceph-mon[27907]: pgmap v455: 129 pgs: 129 active+clean; 314 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.3 MiB/s wr, 239 op/s 2024-04-24T09:06:08.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:06:07 smithi138 ceph-mon[31441]: pgmap v455: 129 pgs: 129 active+clean; 314 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.3 MiB/s wr, 239 op/s 2024-04-24T09:06:08.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:06:07 smithi189 ceph-mon[31791]: pgmap v455: 129 pgs: 129 active+clean; 314 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.3 MiB/s wr, 239 op/s 2024-04-24T09:06:09.155 INFO:tasks.workunit.client.0.smithi083.stdout: 10 5093 11.03 MB/sec warmup 40 sec latency 72.516 ms 2024-04-24T09:06:10.155 INFO:tasks.workunit.client.0.smithi083.stdout: 10 5206 10.88 MB/sec warmup 41 sec latency 293.038 ms 2024-04-24T09:06:10.256 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:06:09 smithi138 ceph-mon[31441]: pgmap v456: 129 pgs: 129 active+clean; 317 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.2 MiB/s wr, 280 op/s 2024-04-24T09:06:10.257 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:06:09 smithi138 ceph-mon[31441]: osdmap e112: 12 total, 12 up, 12 in 2024-04-24T09:06:10.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:06:09 smithi083 ceph-mon[27907]: pgmap v456: 129 pgs: 129 active+clean; 317 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.2 MiB/s wr, 280 op/s 2024-04-24T09:06:10.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:06:09 smithi083 ceph-mon[27907]: osdmap e112: 12 total, 12 up, 12 in 2024-04-24T09:06:10.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:06:09 smithi189 ceph-mon[31791]: pgmap v456: 129 pgs: 129 active+clean; 317 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.2 MiB/s wr, 280 op/s 2024-04-24T09:06:10.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:06:09 smithi189 ceph-mon[31791]: osdmap e112: 12 total, 12 up, 12 in 2024-04-24T09:06:10.969 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.969+0000 7ffa02600640 1 -- 172.21.15.83:0/1823266927 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff9fc108540 msgr2=0x7ff9fc108980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:10.969 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.969+0000 7ffa02600640 1 --2- 172.21.15.83:0/1823266927 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff9fc108540 0x7ff9fc108980 secure :-1 s=READY pgs=2164 cs=0 l=1 rev1=1 crypto rx=0x7ff9e8009930 tx=0x7ff9e802a440 comp rx=0 tx=0).stop 2024-04-24T09:06:10.969 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.969+0000 7ffa02600640 1 -- 172.21.15.83:0/1823266927 shutdown_connections 2024-04-24T09:06:10.969 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.969+0000 7ffa02600640 1 --2- 172.21.15.83:0/1823266927 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff9fc108ec0 0x7ff9fc115300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:10.970 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.969+0000 7ffa02600640 1 --2- 172.21.15.83:0/1823266927 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff9fc108540 0x7ff9fc108980 unknown :-1 s=CLOSED pgs=2164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:10.970 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.969+0000 7ffa02600640 1 --2- 172.21.15.83:0/1823266927 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9fc10e4d0 0x7ff9fc10e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:10.970 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.969+0000 7ffa02600640 1 -- 172.21.15.83:0/1823266927 >> 172.21.15.83:0/1823266927 conn(0x7ff9fc077de0 msgr2=0x7ff9fc106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:10.970 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.970+0000 7ffa02600640 1 -- 172.21.15.83:0/1823266927 shutdown_connections 2024-04-24T09:06:10.970 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.970+0000 7ffa02600640 1 -- 172.21.15.83:0/1823266927 wait complete. 2024-04-24T09:06:10.971 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.971+0000 7ffa02600640 1 Processor -- start 2024-04-24T09:06:10.971 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.971+0000 7ffa02600640 1 -- start start 2024-04-24T09:06:10.972 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.972+0000 7ffa02600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff9fc108540 0x7ff9fc1a4a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:10.972 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.972+0000 7ffa02600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff9fc108ec0 0x7ff9fc1a4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:10.972 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.972+0000 7ffa02600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9fc10e4d0 0x7ff9fc19eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:10.973 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.972+0000 7ffa02600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff9fc118790 con 0x7ff9fc108540 2024-04-24T09:06:10.973 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.972+0000 7ffa02600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff9fc118610 con 0x7ff9fc10e4d0 2024-04-24T09:06:10.973 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.972+0000 7ffa02600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff9fc118910 con 0x7ff9fc108ec0 2024-04-24T09:06:10.973 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.973+0000 7ff9fbe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff9fc108ec0 0x7ff9fc1a4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:10.973 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.973+0000 7ff9fbe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff9fc108ec0 0x7ff9fc1a4fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:59390/0 (socket says 172.21.15.83:59390) 2024-04-24T09:06:10.973 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.973+0000 7ff9fbe00640 1 -- 172.21.15.83:0/2800369980 learned_addr learned my addr 172.21.15.83:0/2800369980 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:06:10.973 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.973+0000 7ffa01200640 1 --2- 172.21.15.83:0/2800369980 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff9fc108540 0x7ff9fc1a4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:10.974 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.974+0000 7ff9fbe00640 1 -- 172.21.15.83:0/2800369980 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9fc10e4d0 msgr2=0x7ff9fc19eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:10.974 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.974+0000 7ff9fbe00640 1 --2- 172.21.15.83:0/2800369980 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9fc10e4d0 0x7ff9fc19eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:10.974 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.974+0000 7ff9fbe00640 1 -- 172.21.15.83:0/2800369980 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff9fc108540 msgr2=0x7ff9fc1a4a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:10.974 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.974+0000 7ff9fbe00640 1 --2- 172.21.15.83:0/2800369980 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff9fc108540 0x7ff9fc1a4a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:10.974 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.974+0000 7ff9fbe00640 1 -- 172.21.15.83:0/2800369980 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9fc19f3f0 con 0x7ff9fc108ec0 2024-04-24T09:06:10.974 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.974+0000 7ffa01200640 1 --2- 172.21.15.83:0/2800369980 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff9fc108540 0x7ff9fc1a4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:06:10.975 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.974+0000 7ff9fbe00640 1 --2- 172.21.15.83:0/2800369980 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff9fc108ec0 0x7ff9fc1a4fb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff9e8031440 tx=0x7ff9e8003c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:10.975 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.975+0000 7ff9f9600640 1 -- 172.21.15.83:0/2800369980 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9e803f070 con 0x7ff9fc108ec0 2024-04-24T09:06:10.975 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.975+0000 7ffa02600640 1 -- 172.21.15.83:0/2800369980 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9fc19f680 con 0x7ff9fc108ec0 2024-04-24T09:06:10.975 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.975+0000 7ffa02600640 1 -- 172.21.15.83:0/2800369980 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9fc1ab9f0 con 0x7ff9fc108ec0 2024-04-24T09:06:10.975 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.975+0000 7ff9f9600640 1 -- 172.21.15.83:0/2800369980 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff9e8031b50 con 0x7ff9fc108ec0 2024-04-24T09:06:10.975 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.975+0000 7ff9f9600640 1 -- 172.21.15.83:0/2800369980 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9e8035030 con 0x7ff9fc108ec0 2024-04-24T09:06:10.980 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.978+0000 7ffa06ba6740 1 Processor -- start 2024-04-24T09:06:10.980 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.979+0000 7ffa06ba6740 1 -- start start 2024-04-24T09:06:10.980 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.979+0000 7ffa06ba6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622a4b212a0 0x5622a4b21680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:10.980 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.979+0000 7ffa06ba6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622a4b21bc0 0x5622a4b3d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:10.980 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.979+0000 7ffa06ba6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622a4b3d5b0 0x5622a4b3f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:10.980 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.979+0000 7ffa06ba6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5622a4a87900 con 0x5622a4b3d5b0 2024-04-24T09:06:10.980 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.979+0000 7ffa06ba6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5622a4a2df20 con 0x5622a4b212a0 2024-04-24T09:06:10.980 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.979+0000 7ffa06ba6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5622a4a838c0 con 0x5622a4b21bc0 2024-04-24T09:06:10.980 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.979+0000 7ffa01c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622a4b212a0 0x5622a4b21680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:10.980 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.979+0000 7ffa01c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622a4b212a0 0x5622a4b21680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:58262/0 (socket says 172.21.15.83:58262) 2024-04-24T09:06:10.980 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.980+0000 7ffa01c00640 1 -- 172.21.15.83:0/1595162484 learned_addr learned my addr 172.21.15.83:0/1595162484 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:06:10.980 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.980+0000 7ffa01200640 1 --2- 172.21.15.83:0/1595162484 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622a4b21bc0 0x5622a4b3d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:10.981 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.980+0000 7ffa01c00640 1 -- 172.21.15.83:0/1595162484 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622a4b21bc0 msgr2=0x5622a4b3d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:10.981 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.980+0000 7ffa01c00640 1 --2- 172.21.15.83:0/1595162484 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622a4b21bc0 0x5622a4b3d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:10.981 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.980+0000 7ffa01c00640 1 -- 172.21.15.83:0/1595162484 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622a4b3d5b0 msgr2=0x5622a4b3f9a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T09:06:10.982 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.980+0000 7ffa01c00640 1 --2- 172.21.15.83:0/1595162484 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622a4b3d5b0 0x5622a4b3f9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:10.982 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.980+0000 7ffa01c00640 1 -- 172.21.15.83:0/1595162484 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622a4a52e10 con 0x5622a4b212a0 2024-04-24T09:06:10.982 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.981+0000 7ffa01c00640 1 --2- 172.21.15.83:0/1595162484 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622a4b212a0 0x5622a4b21680 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff9f4009ed0 tx=0x7ff9f4030730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3981739fddcce4c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:10.982 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.981+0000 7ff9ee000640 1 -- 172.21.15.83:0/1595162484 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f4004d20 con 0x5622a4b212a0 2024-04-24T09:06:10.982 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.981+0000 7ff9ee000640 1 -- 172.21.15.83:0/1595162484 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff9f40054d0 con 0x5622a4b212a0 2024-04-24T09:06:10.982 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.981+0000 7ff9ee000640 1 -- 172.21.15.83:0/1595162484 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f40328b0 con 0x5622a4b212a0 2024-04-24T09:06:10.982 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.982+0000 7ff9f9600640 1 -- 172.21.15.83:0/2800369980 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff9e8042070 con 0x7ff9fc108ec0 2024-04-24T09:06:10.983 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.982+0000 7ffa06ba6740 1 -- 172.21.15.83:0/1595162484 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622a4b212a0 msgr2=0x5622a4b21680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:10.983 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.982+0000 7ffa06ba6740 1 --2- 172.21.15.83:0/1595162484 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622a4b212a0 0x5622a4b21680 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff9f4009ed0 tx=0x7ff9f4030730 comp rx=0 tx=0).stop 2024-04-24T09:06:10.984 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.982+0000 7ffa06ba6740 1 -- 172.21.15.83:0/1595162484 shutdown_connections 2024-04-24T09:06:10.984 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.982+0000 7ffa06ba6740 1 --2- 172.21.15.83:0/1595162484 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622a4b3d5b0 0x5622a4b3f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:10.984 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.982+0000 7ffa06ba6740 1 --2- 172.21.15.83:0/1595162484 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622a4b21bc0 0x5622a4b3d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:10.984 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.982+0000 7ffa06ba6740 1 --2- 172.21.15.83:0/1595162484 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622a4b212a0 0x5622a4b21680 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:10.984 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.982+0000 7ffa06ba6740 1 -- 172.21.15.83:0/1595162484 >> 172.21.15.83:0/1595162484 conn(0x5622a4b1d8d0 msgr2=0x5622a4b1dcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:10.984 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.984+0000 7ffa06ba6740 1 -- 172.21.15.83:0/1595162484 shutdown_connections 2024-04-24T09:06:10.985 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.984+0000 7ffa06ba6740 1 -- 172.21.15.83:0/1595162484 wait complete. 2024-04-24T09:06:10.985 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.985+0000 7ffa06ba6740 1 -- start start 2024-04-24T09:06:10.985 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.985+0000 7ff9f9600640 1 --2- 172.21.15.83:0/2800369980 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff9e40a3d70 0x7ff9e40a6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:10.985 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.985+0000 7ff9f9600640 1 -- 172.21.15.83:0/2800369980 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9583+0+0 (secure 0 0 0) 0x7ff9e80f12e0 con 0x7ff9fc108ec0 2024-04-24T09:06:10.985 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.986+0000 7ffa06ba6740 1 Processor -- start 2024-04-24T09:06:10.985 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.986+0000 7ffa01c00640 1 --2- 172.21.15.83:0/2800369980 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff9e40a3d70 0x7ff9e40a6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:10.987 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.986+0000 7ffa06ba6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622a4b212a0 0x5622a4b2e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:10.987 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.986+0000 7ffa06ba6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622a4b21bc0 0x5622a4b2edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:10.987 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.986+0000 7ffa06ba6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622a4b3d5b0 0x5622a4b29c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:10.987 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.986+0000 7ffa06ba6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5622a4b24240 con 0x5622a4b212a0 2024-04-24T09:06:10.987 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.986+0000 7ffa06ba6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5622a4a412c0 con 0x5622a4b3d5b0 2024-04-24T09:06:10.987 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.986+0000 7ffa06ba6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5622a4b243c0 con 0x5622a4b21bc0 2024-04-24T09:06:10.987 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.986+0000 7ffa06ba6740 10 client.0 _ll_register_callbacks cb 0x5622a4799a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:06:10.987 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.987+0000 7ffa01c00640 1 --2- 172.21.15.83:0/2800369980 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff9e40a3d70 0x7ff9e40a6230 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7ff9f4009ea0 tx=0x7ff9f4004a40 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:10.987 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.987+0000 7ffa01200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622a4b212a0 0x5622a4b2e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:10.987 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.987+0000 7ffa01c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622a4b3d5b0 0x5622a4b29c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:10.988 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.987+0000 7ffa01200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622a4b212a0 0x5622a4b2e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53466/0 (socket says 172.21.15.83:53466) 2024-04-24T09:06:10.988 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.987+0000 7ffa01200640 1 -- 172.21.15.83:0/306268977 learned_addr learned my addr 172.21.15.83:0/306268977 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:06:10.989 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.987+0000 7ffa01c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622a4b3d5b0 0x5622a4b29c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:58268/0 (socket says 172.21.15.83:58268) 2024-04-24T09:06:10.989 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.987+0000 7ff9fbe00640 1 --2- 172.21.15.83:0/306268977 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622a4b21bc0 0x5622a4b2edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:10.989 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.988+0000 7ff9fbe00640 1 -- 172.21.15.83:0/306268977 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622a4b3d5b0 msgr2=0x5622a4b29c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:10.989 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.988+0000 7ff9fbe00640 1 --2- 172.21.15.83:0/306268977 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622a4b3d5b0 0x5622a4b29c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:10.989 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.988+0000 7ff9fbe00640 1 -- 172.21.15.83:0/306268977 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622a4b212a0 msgr2=0x5622a4b2e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:10.989 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.989+0000 7ff9fbe00640 1 --2- 172.21.15.83:0/306268977 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622a4b212a0 0x5622a4b2e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:10.989 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.989+0000 7ff9fbe00640 1 -- 172.21.15.83:0/306268977 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622a4a51fd0 con 0x5622a4b21bc0 2024-04-24T09:06:10.990 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.989+0000 7ffa01200640 1 --2- 172.21.15.83:0/306268977 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622a4b212a0 0x5622a4b2e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:06:10.990 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.990+0000 7ff9fbe00640 1 --2- 172.21.15.83:0/306268977 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622a4b21bc0 0x5622a4b2edf0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff9e80ee5b0 tx=0x7ff9e80bcc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=350a8606b7a10cbb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:10.990 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.990+0000 7ff9e2a00640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-04-24T09:06:10.991 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.990+0000 7ff9e2a00640 1 -- 172.21.15.83:0/306268977 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9e80bce00 con 0x5622a4b21bc0 2024-04-24T09:06:10.991 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.990+0000 7ffa06ba6740 1 -- 172.21.15.83:0/306268977 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9f40054d0 con 0x5622a4b21bc0 2024-04-24T09:06:10.991 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.991+0000 7ff9e2a00640 1 -- 172.21.15.83:0/306268977 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9583+0+0 (secure 0 0 0) 0x7ff9e80ee830 con 0x5622a4b21bc0 2024-04-24T09:06:10.991 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.991+0000 7ff9e2a00640 1 -- 172.21.15.83:0/306268977 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7ff9e80eea30 con 0x5622a4b21bc0 2024-04-24T09:06:10.991 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.991+0000 7ffa06ba6740 10 client.28048 fetch_fsmap learned FSMap version 47 2024-04-24T09:06:10.991 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.991+0000 7ffa06ba6740 1 -- 172.21.15.83:0/306268977 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5622a479a210 con 0x5622a4b21bc0 2024-04-24T09:06:10.992 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.992+0000 7ff9e2a00640 1 -- 172.21.15.83:0/306268977 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff9f40054d0 con 0x5622a4b21bc0 2024-04-24T09:06:10.992 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.992+0000 7ff9e2a00640 1 -- 172.21.15.83:0/306268977 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 47) ==== 8398+0+0 (secure 0 0 0) 0x7ff9e80c55e0 con 0x5622a4b21bc0 2024-04-24T09:06:10.993 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.992+0000 7ffa06ba6740 10 client.28048 fetch_fsmap finished waiting for FSMap version 47 2024-04-24T09:06:10.993 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.992+0000 7ffa06ba6740 10 client.28048 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:06:10.993 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.993+0000 7ffa06ba6740 20 client.28048 populate_metadata read hostname 'smithi083' 2024-04-24T09:06:10.993 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.993+0000 7ffa06ba6740 1 --2- 172.21.15.83:0/306268977 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5622a4b39ea0 0x5622a4b3c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:10.993 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.993+0000 7ffa06ba6740 4 client.28048 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:06:10.993 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.993+0000 7ffa06ba6740 1 -- 172.21.15.83:0/306268977 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5622a4a95320 con 0x5622a4b39ea0 2024-04-24T09:06:10.993 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.993+0000 7ffa01200640 1 --2- 172.21.15.83:0/306268977 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5622a4b39ea0 0x5622a4b3c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:10.994 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.994+0000 7ffa01200640 1 --2- 172.21.15.83:0/306268977 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5622a4b39ea0 0x5622a4b3c2c0 crc :-1 s=READY pgs=373 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=200cf80a66ad7510 server_cookie=43c12eed70dc239a in_seq=0 out_seq=0 2024-04-24T09:06:10.994 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.994+0000 7ff9e2a00640 10 client.28048 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:06:10.996 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.996+0000 7ff9e2a00640 1 -- 172.21.15.83:0/306268977 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5622a4a95320 con 0x5622a4b39ea0 2024-04-24T09:06:10.996 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.996+0000 7ff9e2a00640 10 client.28048 handle_command_reply: tid=0 2024-04-24T09:06:10.996 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.996+0000 7ffa06ba6740 1 client.28048 shutdown 2024-04-24T09:06:10.997 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.996+0000 7ffa06ba6740 1 -- 172.21.15.83:0/306268977 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622a4b21bc0 msgr2=0x5622a4b2edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:10.997 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.996+0000 7ffa06ba6740 1 --2- 172.21.15.83:0/306268977 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622a4b21bc0 0x5622a4b2edf0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff9e80ee5b0 tx=0x7ff9e80bcc90 comp rx=0 tx=0).stop 2024-04-24T09:06:10.997 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.996+0000 7ffa06ba6740 1 -- 172.21.15.83:0/306268977 shutdown_connections 2024-04-24T09:06:10.997 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.996+0000 7ffa06ba6740 1 --2- 172.21.15.83:0/306268977 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5622a4b39ea0 0x5622a4b3c2c0 crc :-1 s=READY pgs=373 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:10.997 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.996+0000 7ffa06ba6740 1 --2- 172.21.15.83:0/306268977 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622a4b3d5b0 0x5622a4b29c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:10.997 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.996+0000 7ffa06ba6740 1 --2- 172.21.15.83:0/306268977 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622a4b21bc0 0x5622a4b2edf0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:10.997 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.996+0000 7ffa06ba6740 1 --2- 172.21.15.83:0/306268977 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622a4b212a0 0x5622a4b2e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:10.997 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.996+0000 7ffa06ba6740 1 -- 172.21.15.83:0/306268977 >> 172.21.15.83:0/306268977 conn(0x5622a4b1d8d0 msgr2=0x5622a4b236e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:10.997 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.996+0000 7ff9e2a00640 0 client.28048 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:06:10.997 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.996+0000 7ffa06ba6740 1 -- 172.21.15.83:0/306268977 shutdown_connections 2024-04-24T09:06:10.997 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.997+0000 7ffa06ba6740 1 -- 172.21.15.83:0/306268977 wait complete. 2024-04-24T09:06:10.997 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:10.997+0000 7ffa06ba6740 20 client.28048 trim_cache size 0 max 16384 2024-04-24T09:06:11.017 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:11.017+0000 7ffa06ba6740 1 Processor -- start 2024-04-24T09:06:11.017 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:11.017+0000 7ffa06ba6740 1 -- start start 2024-04-24T09:06:11.017 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:11.017+0000 7ffa06ba6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622a4b212a0 0x5622a4b23520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:11.017 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:11.017+0000 7ffa06ba6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622a4b21bc0 0x5622a4b23a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:11.017 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:11.017+0000 7ffa06ba6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622a4b3d5b0 0x5622a4b2e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:11.017 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:11.017+0000 7ffa06ba6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff9dc00ffd0 con 0x5622a4b3d5b0 2024-04-24T09:06:11.018 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:11.017+0000 7ffa06ba6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff9dc011c90 con 0x5622a4b21bc0 2024-04-24T09:06:11.018 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:11.017+0000 7ffa06ba6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff9dc0106a0 con 0x5622a4b212a0 2024-04-24T09:06:11.019 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:11.018+0000 7ff9fbe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622a4b21bc0 0x5622a4b23a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:11.019 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:11.018+0000 7ffa01c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622a4b3d5b0 0x5622a4b2e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:11.019 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:11.018+0000 7ff9fbe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622a4b21bc0 0x5622a4b23a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:58272/0 (socket says 172.21.15.83:58272) 2024-04-24T09:06:11.019 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:11.018+0000 7ff9fbe00640 1 -- 172.21.15.83:0/523943755 learned_addr learned my addr 172.21.15.83:0/523943755 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:06:11.019 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:11.018+0000 7ffa01200640 1 --2- 172.21.15.83:0/523943755 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622a4b212a0 0x5622a4b23520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:11.019 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:11.018+0000 7ff9fbe00640 1 -- 172.21.15.83:0/523943755 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622a4b212a0 msgr2=0x5622a4b23520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:11.019 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:11.018+0000 7ff9fbe00640 1 --2- 172.21.15.83:0/523943755 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622a4b212a0 0x5622a4b23520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:11.019 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:11.018+0000 7ff9fbe00640 1 -- 172.21.15.83:0/523943755 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622a4b3d5b0 msgr2=0x5622a4b2e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:11.019 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:11.018+0000 7ff9fbe00640 1 --2- 172.21.15.83:0/523943755 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622a4b3d5b0 0x5622a4b2e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:11.019 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:11.018+0000 7ff9fbe00640 1 -- 172.21.15.83:0/523943755 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622a4866620 con 0x5622a4b21bc0 2024-04-24T09:06:11.019 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:11.019+0000 7ff9fbe00640 1 --2- 172.21.15.83:0/523943755 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622a4b21bc0 0x5622a4b23a60 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff9e80bcb00 tx=0x7ff9e80bcd00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7628aab5771dfa69 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:11.020 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:11.019+0000 7ff9ed600640 1 -- 172.21.15.83:0/523943755 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9e80c11b0 con 0x5622a4b21bc0 2024-04-24T09:06:11.020 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:11.019+0000 7ff9ed600640 1 -- 172.21.15.83:0/523943755 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff9e80c5a70 con 0x5622a4b21bc0 2024-04-24T09:06:11.020 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:11.019+0000 7ff9ed600640 1 -- 172.21.15.83:0/523943755 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9e80ee830 con 0x5622a4b21bc0 2024-04-24T09:06:11.020 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:11.020+0000 7ffa06ba6740 1 -- 172.21.15.83:0/523943755 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622a4b21bc0 msgr2=0x5622a4b23a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:11.020 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:11.020+0000 7ffa06ba6740 1 --2- 172.21.15.83:0/523943755 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622a4b21bc0 0x5622a4b23a60 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff9e80bcb00 tx=0x7ff9e80bcd00 comp rx=0 tx=0).stop 2024-04-24T09:06:11.020 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:11.020+0000 7ffa06ba6740 1 -- 172.21.15.83:0/523943755 shutdown_connections 2024-04-24T09:06:11.020 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:11.020+0000 7ffa06ba6740 1 --2- 172.21.15.83:0/523943755 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622a4b3d5b0 0x5622a4b2e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:11.021 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:11.020+0000 7ffa06ba6740 1 --2- 172.21.15.83:0/523943755 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622a4b21bc0 0x5622a4b23a60 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:11.021 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:11.020+0000 7ffa06ba6740 1 --2- 172.21.15.83:0/523943755 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622a4b212a0 0x5622a4b23520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:11.021 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:11.020+0000 7ffa06ba6740 1 -- 172.21.15.83:0/523943755 >> 172.21.15.83:0/523943755 conn(0x5622a4b1d8d0 msgr2=0x5622a4b22e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:11.021 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:11.021+0000 7ffa06ba6740 1 -- 172.21.15.83:0/523943755 shutdown_connections 2024-04-24T09:06:11.022 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:11.021+0000 7ffa06ba6740 1 -- 172.21.15.83:0/523943755 wait complete. 2024-04-24T09:06:11.022 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:11.021+0000 7ffa06ba6740 1 -- start start 2024-04-24T09:06:11.022 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:11.021+0000 7ffa06ba6740 1 Processor -- start 2024-04-24T09:06:11.022 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:11.022+0000 7ffa06ba6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622a4b212a0 0x5622a4b316a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:11.022 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:11.022+0000 7ffa06ba6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622a4b21bc0 0x5622a4b31be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:11.022 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:11.022+0000 7ffa06ba6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622a4b3d5b0 0x5622a4ba1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:11.022 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:11.022+0000 7ffa06ba6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff9dc011500 con 0x5622a4b3d5b0 2024-04-24T09:06:11.025 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:11.022+0000 7ffa06ba6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff9dc010dd0 con 0x5622a4b212a0 2024-04-24T09:06:11.025 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:11.022+0000 7ffa06ba6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff9dc00dfd0 con 0x5622a4b21bc0 2024-04-24T09:06:11.025 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:11.022+0000 7ffa06ba6740 10 client.0 _ll_register_callbacks cb 0x7ff9e803b460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:06:11.027 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:11.022+0000 7ffa01c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622a4b3d5b0 0x5622a4ba1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:11.027 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:11.022+0000 7ffa01c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622a4b3d5b0 0x5622a4ba1830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53478/0 (socket says 172.21.15.83:53478) 2024-04-24T09:06:11.027 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:11.022+0000 7ffa01c00640 1 -- 172.21.15.83:0/66193921 learned_addr learned my addr 172.21.15.83:0/66193921 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:06:11.027 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:11.023+0000 7ffa01200640 1 --2- 172.21.15.83:0/66193921 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622a4b212a0 0x5622a4b316a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:11.028 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:11.023+0000 7ffa01200640 1 -- 172.21.15.83:0/66193921 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622a4b21bc0 msgr2=0x5622a4b31be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:11.028 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:11.023+0000 7ffa01200640 1 --2- 172.21.15.83:0/66193921 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622a4b21bc0 0x5622a4b31be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:11.028 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:11.023+0000 7ffa01200640 1 -- 172.21.15.83:0/66193921 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622a4b3d5b0 msgr2=0x5622a4ba1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:11.028 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:11.023+0000 7ffa01200640 1 --2- 172.21.15.83:0/66193921 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622a4b3d5b0 0x5622a4ba1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:11.028 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:11.023+0000 7ffa01200640 1 -- 172.21.15.83:0/66193921 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622a4ba1e20 con 0x5622a4b212a0 2024-04-24T09:06:11.028 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:11.024+0000 7ffa01200640 1 --2- 172.21.15.83:0/66193921 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622a4b212a0 0x5622a4b316a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff9f0006fd0 tx=0x7ff9f0013870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20a1b43d2df9f567 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:11.028 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:11.024+0000 7ff9e2a00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:06:11.028 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:11.025+0000 7ff9e2a00640 1 -- 172.21.15.83:0/66193921 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f0021070 con 0x5622a4b212a0 2024-04-24T09:06:11.028 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:11.025+0000 7ff9e2a00640 1 -- 172.21.15.83:0/66193921 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9583+0+0 (secure 0 0 0) 0x7ff9f001ddd0 con 0x5622a4b212a0 2024-04-24T09:06:11.028 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:11.026+0000 7ffa06ba6740 1 -- 172.21.15.83:0/66193921 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9e80c5a70 con 0x5622a4b212a0 2024-04-24T09:06:11.028 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:11.026+0000 7ff9e2a00640 1 -- 172.21.15.83:0/66193921 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7ff9f0004020 con 0x5622a4b212a0 2024-04-24T09:06:11.028 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:11.026+0000 7ffa06ba6740 10 client.37895 fetch_fsmap learned FSMap version 47 2024-04-24T09:06:11.028 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:11.026+0000 7ffa06ba6740 1 -- 172.21.15.83:0/66193921 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5622a4ba3130 con 0x5622a4b212a0 2024-04-24T09:06:11.028 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:11.026+0000 7ff9e2a00640 1 -- 172.21.15.83:0/66193921 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff9e80c5a70 con 0x5622a4b212a0 2024-04-24T09:06:11.029 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:11.027+0000 7ff9e2a00640 1 -- 172.21.15.83:0/66193921 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 47) ==== 8398+0+0 (secure 0 0 0) 0x7ff9f0003690 con 0x5622a4b212a0 2024-04-24T09:06:11.029 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:11.027+0000 7ffa06ba6740 10 client.37895 fetch_fsmap finished waiting for FSMap version 47 2024-04-24T09:06:11.029 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:11.027+0000 7ffa06ba6740 10 client.37895 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:06:11.029 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:11.027+0000 7ffa06ba6740 20 client.37895 populate_metadata read hostname 'smithi083' 2024-04-24T09:06:11.029 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:11.027+0000 7ffa06ba6740 1 --2- 172.21.15.83:0/66193921 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5622a4ba3940 0x5622a4ba5d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:11.029 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:11.027+0000 7ffa06ba6740 4 client.37895 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T09:06:11.029 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:11.027+0000 7ffa06ba6740 1 -- 172.21.15.83:0/66193921 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5622a4a8f190 con 0x5622a4ba3940 2024-04-24T09:06:11.029 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:11.027+0000 7ff9fbe00640 1 --2- 172.21.15.83:0/66193921 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5622a4ba3940 0x5622a4ba5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:11.029 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:11.028+0000 7ff9fbe00640 1 --2- 172.21.15.83:0/66193921 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5622a4ba3940 0x5622a4ba5d60 crc :-1 s=READY pgs=374 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9edbb914db3bad90 server_cookie=1a5479795694a648 in_seq=0 out_seq=0 2024-04-24T09:06:11.029 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:11.028+0000 7ff9e2a00640 10 client.37895 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:06:11.084 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:11.084+0000 7ff9e2a00640 1 -- 172.21.15.83:0/66193921 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5622a4a8f190 con 0x5622a4ba3940 2024-04-24T09:06:11.084 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:11.084+0000 7ff9e2a00640 10 client.37895 handle_command_reply: tid=0 2024-04-24T09:06:11.085 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:11.085+0000 7ffa06ba6740 1 client.37895 shutdown 2024-04-24T09:06:11.086 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:11.086+0000 7ffa06ba6740 1 -- 172.21.15.83:0/66193921 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622a4b212a0 msgr2=0x5622a4b316a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:11.086 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:11.086+0000 7ffa06ba6740 1 --2- 172.21.15.83:0/66193921 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622a4b212a0 0x5622a4b316a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff9f0006fd0 tx=0x7ff9f0013870 comp rx=0 tx=0).stop 2024-04-24T09:06:11.086 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:11.087+0000 7ffa06ba6740 1 -- 172.21.15.83:0/66193921 shutdown_connections 2024-04-24T09:06:11.087 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:11.087+0000 7ffa06ba6740 1 --2- 172.21.15.83:0/66193921 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5622a4ba3940 0x5622a4ba5d60 crc :-1 s=READY pgs=374 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:11.087 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:11.087+0000 7ffa06ba6740 1 --2- 172.21.15.83:0/66193921 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622a4b3d5b0 0x5622a4ba1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:11.087 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:11.087+0000 7ffa06ba6740 1 --2- 172.21.15.83:0/66193921 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622a4b21bc0 0x5622a4b31be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:11.087 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:11.087+0000 7ffa06ba6740 1 --2- 172.21.15.83:0/66193921 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622a4b212a0 0x5622a4b316a0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:11.087 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:11.087+0000 7ffa06ba6740 1 -- 172.21.15.83:0/66193921 >> 172.21.15.83:0/66193921 conn(0x5622a4b1d8d0 msgr2=0x5622a4b1ca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:11.087 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:11.087+0000 7ff9e2a00640 0 client.37895 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:06:11.088 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:11.088+0000 7ffa06ba6740 1 -- 172.21.15.83:0/66193921 shutdown_connections 2024-04-24T09:06:11.088 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:11.088+0000 7ffa06ba6740 1 -- 172.21.15.83:0/66193921 wait complete. 2024-04-24T09:06:11.089 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:11.089+0000 7ffa06ba6740 20 client.37895 trim_cache size 0 max 16384 2024-04-24T09:06:11.090 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T09:06:11.090 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T09:06:11.090 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T09:06:11.090 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T09:06:11.091 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:11.091+0000 7ffa02600640 1 -- 172.21.15.83:0/2800369980 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff9e40a3d70 msgr2=0x7ff9e40a6230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:11.091 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:11.091+0000 7ffa02600640 1 --2- 172.21.15.83:0/2800369980 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff9e40a3d70 0x7ff9e40a6230 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7ff9f4009ea0 tx=0x7ff9f4004a40 comp rx=0 tx=0).stop 2024-04-24T09:06:11.091 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:11.091+0000 7ffa02600640 1 -- 172.21.15.83:0/2800369980 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff9fc108ec0 msgr2=0x7ff9fc1a4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:11.091 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:11.091+0000 7ffa02600640 1 --2- 172.21.15.83:0/2800369980 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff9fc108ec0 0x7ff9fc1a4fb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff9e8031440 tx=0x7ff9e8003c00 comp rx=0 tx=0).stop 2024-04-24T09:06:11.092 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:11.092+0000 7ffa02600640 1 -- 172.21.15.83:0/2800369980 shutdown_connections 2024-04-24T09:06:11.092 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:11.092+0000 7ffa02600640 1 --2- 172.21.15.83:0/2800369980 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff9e40a3d70 0x7ff9e40a6230 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:11.092 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:11.092+0000 7ffa02600640 1 --2- 172.21.15.83:0/2800369980 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9fc10e4d0 0x7ff9fc19eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:11.092 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:11.092+0000 7ffa02600640 1 --2- 172.21.15.83:0/2800369980 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff9fc108ec0 0x7ff9fc1a4fb0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:11.092 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:11.092+0000 7ffa02600640 1 --2- 172.21.15.83:0/2800369980 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff9fc108540 0x7ff9fc1a4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:11.092 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:11.092+0000 7ffa02600640 1 -- 172.21.15.83:0/2800369980 >> 172.21.15.83:0/2800369980 conn(0x7ff9fc077de0 msgr2=0x7ff9fc0784a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:11.092 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:11.092+0000 7ffa02600640 1 -- 172.21.15.83:0/2800369980 shutdown_connections 2024-04-24T09:06:11.093 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:11.092+0000 7ffa02600640 1 -- 172.21.15.83:0/2800369980 wait complete. 2024-04-24T09:06:11.155 INFO:tasks.workunit.client.0.smithi083.stdout: 10 5330 10.67 MB/sec warmup 42 sec latency 102.540 ms 2024-04-24T09:06:12.155 INFO:tasks.workunit.client.0.smithi083.stdout: 10 5482 10.68 MB/sec warmup 43 sec latency 70.583 ms 2024-04-24T09:06:12.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:06:11 smithi083 ceph-mon[27907]: pgmap v458: 129 pgs: 129 active+clean; 317 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 708 KiB/s rd, 1.6 MiB/s wr, 169 op/s 2024-04-24T09:06:12.457 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:06:11 smithi138 ceph-mon[31441]: pgmap v458: 129 pgs: 129 active+clean; 317 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 708 KiB/s rd, 1.6 MiB/s wr, 169 op/s 2024-04-24T09:06:12.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:06:11 smithi189 ceph-mon[31791]: pgmap v458: 129 pgs: 129 active+clean; 317 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 708 KiB/s rd, 1.6 MiB/s wr, 169 op/s 2024-04-24T09:06:13.155 INFO:tasks.workunit.client.0.smithi083.stdout: 10 5621 10.58 MB/sec warmup 44 sec latency 82.609 ms 2024-04-24T09:06:14.155 INFO:tasks.workunit.client.0.smithi083.stdout: 10 5773 10.52 MB/sec warmup 45 sec latency 81.101 ms 2024-04-24T09:06:14.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:06:13 smithi083 ceph-mon[27907]: pgmap v459: 129 pgs: 129 active+clean; 322 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.6 MiB/s wr, 260 op/s 2024-04-24T09:06:14.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:06:13 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:06:14.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:06:13 smithi138 ceph-mon[31441]: pgmap v459: 129 pgs: 129 active+clean; 322 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.6 MiB/s wr, 260 op/s 2024-04-24T09:06:14.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:06:13 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:06:14.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:06:13 smithi189 ceph-mon[31791]: pgmap v459: 129 pgs: 129 active+clean; 322 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.6 MiB/s wr, 260 op/s 2024-04-24T09:06:14.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:06:13 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:06:15.156 INFO:tasks.workunit.client.0.smithi083.stdout: 10 5863 10.37 MB/sec warmup 46 sec latency 409.494 ms 2024-04-24T09:06:16.156 INFO:tasks.workunit.client.0.smithi083.stdout: 10 6002 10.27 MB/sec warmup 47 sec latency 421.535 ms 2024-04-24T09:06:16.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:06:15 smithi083 ceph-mon[27907]: pgmap v460: 129 pgs: 129 active+clean; 322 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.6 MiB/s wr, 256 op/s 2024-04-24T09:06:16.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:06:15 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:06:16.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:06:15 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:06:16.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:06:15 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:06:16.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:06:15 smithi138 ceph-mon[31441]: pgmap v460: 129 pgs: 129 active+clean; 322 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.6 MiB/s wr, 256 op/s 2024-04-24T09:06:16.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:06:15 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:06:16.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:06:15 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:06:16.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:06:15 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:06:16.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:06:15 smithi189 ceph-mon[31791]: pgmap v460: 129 pgs: 129 active+clean; 322 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.6 MiB/s wr, 256 op/s 2024-04-24T09:06:16.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:06:15 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:06:16.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:06:15 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:06:16.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:06:15 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:06:17.156 INFO:tasks.workunit.client.0.smithi083.stdout: 10 6168 10.31 MB/sec warmup 48 sec latency 73.620 ms 2024-04-24T09:06:18.156 INFO:tasks.workunit.client.0.smithi083.stdout: 10 6337 10.30 MB/sec warmup 49 sec latency 109.104 ms 2024-04-24T09:06:18.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:06:17 smithi083 ceph-mon[27907]: pgmap v461: 129 pgs: 129 active+clean; 322 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.6 MiB/s wr, 256 op/s 2024-04-24T09:06:18.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:06:17 smithi138 ceph-mon[31441]: pgmap v461: 129 pgs: 129 active+clean; 322 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.6 MiB/s wr, 256 op/s 2024-04-24T09:06:18.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:06:17 smithi189 ceph-mon[31791]: pgmap v461: 129 pgs: 129 active+clean; 322 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.6 MiB/s wr, 256 op/s 2024-04-24T09:06:19.156 INFO:tasks.workunit.client.0.smithi083.stdout: 10 6526 10.41 MB/sec warmup 50 sec latency 66.262 ms 2024-04-24T09:06:20.156 INFO:tasks.workunit.client.0.smithi083.stdout: 10 6620 10.30 MB/sec warmup 51 sec latency 73.931 ms 2024-04-24T09:06:20.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:06:19 smithi083 ceph-mon[27907]: pgmap v462: 129 pgs: 129 active+clean; 335 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 774 KiB/s rd, 3.1 MiB/s wr, 215 op/s 2024-04-24T09:06:20.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:06:19 smithi138 ceph-mon[31441]: pgmap v462: 129 pgs: 129 active+clean; 335 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 774 KiB/s rd, 3.1 MiB/s wr, 215 op/s 2024-04-24T09:06:20.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:06:19 smithi189 ceph-mon[31791]: pgmap v462: 129 pgs: 129 active+clean; 335 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 774 KiB/s rd, 3.1 MiB/s wr, 215 op/s 2024-04-24T09:06:21.156 INFO:tasks.workunit.client.0.smithi083.stdout: 10 6707 10.20 MB/sec warmup 52 sec latency 432.239 ms 2024-04-24T09:06:21.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:06:20 smithi083 ceph-mon[27907]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:06:21.399 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T09:06:21.400 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T09:06:21.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:06:20 smithi138 ceph-mon[31441]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:06:21.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:06:20 smithi189 ceph-mon[31791]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:06:21.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.653+0000 7f7b5ea00640 1 -- 172.21.15.83:0/341674915 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b58108610 msgr2=0x7f7b58108a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:21.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.653+0000 7f7b5ea00640 1 --2- 172.21.15.83:0/341674915 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b58108610 0x7f7b58108a90 secure :-1 s=READY pgs=2165 cs=0 l=1 rev1=1 crypto rx=0x7f7b44009930 tx=0x7f7b4402a460 comp rx=0 tx=0).stop 2024-04-24T09:06:21.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.653+0000 7f7b5ea00640 1 -- 172.21.15.83:0/341674915 shutdown_connections 2024-04-24T09:06:21.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.653+0000 7f7b5ea00640 1 --2- 172.21.15.83:0/341674915 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7b58108fd0 0x7f7b58115490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:21.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.653+0000 7f7b5ea00640 1 --2- 172.21.15.83:0/341674915 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b58108610 0x7f7b58108a90 unknown :-1 s=CLOSED pgs=2165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:21.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.653+0000 7f7b5ea00640 1 --2- 172.21.15.83:0/341674915 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7b5810e660 0x7f7b5810ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:21.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.653+0000 7f7b5ea00640 1 -- 172.21.15.83:0/341674915 >> 172.21.15.83:0/341674915 conn(0x7f7b58076540 msgr2=0x7f7b58078960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:21.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.653+0000 7f7b5ea00640 1 -- 172.21.15.83:0/341674915 shutdown_connections 2024-04-24T09:06:21.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.653+0000 7f7b5ea00640 1 -- 172.21.15.83:0/341674915 wait complete. 2024-04-24T09:06:21.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.654+0000 7f7b5ea00640 1 Processor -- start 2024-04-24T09:06:21.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.654+0000 7f7b5ea00640 1 -- start start 2024-04-24T09:06:21.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.655+0000 7f7b5ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b58108610 0x7f7b5819ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:21.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.655+0000 7f7b5ea00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7b58108fd0 0x7f7b5819f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:21.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.655+0000 7f7b5ea00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7b5810e660 0x7f7b581a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:21.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.655+0000 7f7b5ea00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7b58118850 con 0x7f7b58108610 2024-04-24T09:06:21.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.655+0000 7f7b5ea00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7b581186d0 con 0x7f7b5810e660 2024-04-24T09:06:21.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.655+0000 7f7b5ea00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7b581189d0 con 0x7f7b58108fd0 2024-04-24T09:06:21.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.655+0000 7f7b57e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b58108610 0x7f7b5819ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:21.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.655+0000 7f7b57e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b58108610 0x7f7b5819ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55662/0 (socket says 172.21.15.83:55662) 2024-04-24T09:06:21.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.655+0000 7f7b57e00640 1 -- 172.21.15.83:0/1408880452 learned_addr learned my addr 172.21.15.83:0/1408880452 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:06:21.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.655+0000 7f7b5ca00640 1 --2- 172.21.15.83:0/1408880452 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7b5810e660 0x7f7b581a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:21.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.655+0000 7f7b57400640 1 --2- 172.21.15.83:0/1408880452 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7b58108fd0 0x7f7b5819f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:21.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.656+0000 7f7b5ca00640 1 -- 172.21.15.83:0/1408880452 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7b58108fd0 msgr2=0x7f7b5819f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:21.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.656+0000 7f7b5ca00640 1 --2- 172.21.15.83:0/1408880452 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7b58108fd0 0x7f7b5819f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:21.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.656+0000 7f7b5ca00640 1 -- 172.21.15.83:0/1408880452 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b58108610 msgr2=0x7f7b5819ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:21.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.656+0000 7f7b5ca00640 1 --2- 172.21.15.83:0/1408880452 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b58108610 0x7f7b5819ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:21.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.656+0000 7f7b5ca00640 1 -- 172.21.15.83:0/1408880452 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b581a3e00 con 0x7f7b5810e660 2024-04-24T09:06:21.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.656+0000 7f7b5ca00640 1 --2- 172.21.15.83:0/1408880452 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7b5810e660 0x7f7b581a36d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7b50009fd0 tx=0x7f7b5000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:21.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.656+0000 7f7b54c00640 1 -- 172.21.15.83:0/1408880452 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b5000ce20 con 0x7f7b5810e660 2024-04-24T09:06:21.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.656+0000 7f7b54c00640 1 -- 172.21.15.83:0/1408880452 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7b50005020 con 0x7f7b5810e660 2024-04-24T09:06:21.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.656+0000 7f7b54c00640 1 -- 172.21.15.83:0/1408880452 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b50010670 con 0x7f7b5810e660 2024-04-24T09:06:21.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.657+0000 7f7b5ea00640 1 -- 172.21.15.83:0/1408880452 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b581a40f0 con 0x7f7b5810e660 2024-04-24T09:06:21.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.657+0000 7f7b5ea00640 1 -- 172.21.15.83:0/1408880452 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b581ab980 con 0x7f7b5810e660 2024-04-24T09:06:21.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.661+0000 7f7b6198d740 1 Processor -- start 2024-04-24T09:06:21.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.661+0000 7f7b54c00640 1 -- 172.21.15.83:0/1408880452 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7b500051d0 con 0x7f7b5810e660 2024-04-24T09:06:21.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.662+0000 7f7b6198d740 1 -- start start 2024-04-24T09:06:21.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.662+0000 7f7b6198d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ca2852f860 0x55ca2852fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:21.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.662+0000 7f7b6198d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ca28530180 0x55ca2853a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:21.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.662+0000 7f7b6198d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ca2853a850 0x55ca2853cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:21.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.662+0000 7f7b6198d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ca284951f0 con 0x55ca28530180 2024-04-24T09:06:21.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.662+0000 7f7b6198d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ca2843b350 con 0x55ca2852f860 2024-04-24T09:06:21.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.662+0000 7f7b6198d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55ca284911b0 con 0x55ca2853a850 2024-04-24T09:06:21.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.662+0000 7f7b57400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ca2852f860 0x55ca2852fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:21.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.662+0000 7f7b57400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ca2852f860 0x55ca2852fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:38702/0 (socket says 172.21.15.83:38702) 2024-04-24T09:06:21.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.662+0000 7f7b57400640 1 -- 172.21.15.83:0/1875731448 learned_addr learned my addr 172.21.15.83:0/1875731448 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:06:21.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.663+0000 7f7b57400640 1 --2- 172.21.15.83:0/1875731448 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ca2853a850 0x55ca2853cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:21.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.663+0000 7f7b57e00640 1 --2- 172.21.15.83:0/1875731448 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ca28530180 0x55ca2853a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:21.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.663+0000 7f7b54c00640 1 --2- 172.21.15.83:0/1408880452 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f7b3c0a3da0 0x7f7b3c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:21.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.663+0000 7f7b57400640 1 -- 172.21.15.83:0/1875731448 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ca2853a850 msgr2=0x55ca2853cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:21.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.663+0000 7f7b57400640 1 --2- 172.21.15.83:0/1875731448 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ca2853a850 0x55ca2853cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:21.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.663+0000 7f7b57400640 1 -- 172.21.15.83:0/1875731448 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ca28530180 msgr2=0x55ca2853a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:21.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.663+0000 7f7b54c00640 1 -- 172.21.15.83:0/1408880452 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9583+0+0 (secure 0 0 0) 0x7f7b500d12c0 con 0x7f7b5810e660 2024-04-24T09:06:21.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.664+0000 7f7b57400640 1 --2- 172.21.15.83:0/1875731448 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ca28530180 0x55ca2853a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:21.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.664+0000 7f7b57400640 1 -- 172.21.15.83:0/1875731448 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca2848ff50 con 0x55ca2852f860 2024-04-24T09:06:21.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.664+0000 7f7b57e00640 1 --2- 172.21.15.83:0/1875731448 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ca28530180 0x55ca2853a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:06:21.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.664+0000 7f7b5ca00640 1 --2- 172.21.15.83:0/1408880452 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f7b3c0a3da0 0x7f7b3c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:21.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.664+0000 7f7b57400640 1 --2- 172.21.15.83:0/1875731448 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ca2852f860 0x55ca2852fc40 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7b44005ec0 tx=0x7f7b44005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5cb8e4486a447c2a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:21.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.665+0000 7f7b5ca00640 1 --2- 172.21.15.83:0/1408880452 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f7b3c0a3da0 0x7f7b3c0a6260 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f7b500ce5f0 tx=0x7f7b50005f70 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:21.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.665+0000 7f7b4d200640 1 -- 172.21.15.83:0/1875731448 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b440035f0 con 0x55ca2852f860 2024-04-24T09:06:21.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.665+0000 7f7b4d200640 1 -- 172.21.15.83:0/1875731448 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7b440054b0 con 0x55ca2852f860 2024-04-24T09:06:21.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.665+0000 7f7b4d200640 1 -- 172.21.15.83:0/1875731448 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b4403fce0 con 0x55ca2852f860 2024-04-24T09:06:21.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.666+0000 7f7b6198d740 1 -- 172.21.15.83:0/1875731448 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ca2852f860 msgr2=0x55ca2852fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:21.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.666+0000 7f7b6198d740 1 --2- 172.21.15.83:0/1875731448 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ca2852f860 0x55ca2852fc40 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7b44005ec0 tx=0x7f7b44005850 comp rx=0 tx=0).stop 2024-04-24T09:06:21.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.666+0000 7f7b6198d740 1 -- 172.21.15.83:0/1875731448 shutdown_connections 2024-04-24T09:06:21.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.666+0000 7f7b6198d740 1 --2- 172.21.15.83:0/1875731448 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ca2853a850 0x55ca2853cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:21.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.666+0000 7f7b6198d740 1 --2- 172.21.15.83:0/1875731448 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ca28530180 0x55ca2853a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:21.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.666+0000 7f7b6198d740 1 --2- 172.21.15.83:0/1875731448 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ca2852f860 0x55ca2852fc40 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:21.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.666+0000 7f7b6198d740 1 -- 172.21.15.83:0/1875731448 >> 172.21.15.83:0/1875731448 conn(0x55ca2852bb80 msgr2=0x55ca2852bf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:21.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.666+0000 7f7b6198d740 1 -- 172.21.15.83:0/1875731448 shutdown_connections 2024-04-24T09:06:21.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.666+0000 7f7b6198d740 1 -- 172.21.15.83:0/1875731448 wait complete. 2024-04-24T09:06:21.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.667+0000 7f7b6198d740 1 -- start start 2024-04-24T09:06:21.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.667+0000 7f7b6198d740 1 Processor -- start 2024-04-24T09:06:21.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.668+0000 7f7b6198d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ca2852f860 0x55ca28537790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:21.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.668+0000 7f7b6198d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ca28530180 0x55ca28537cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:21.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.668+0000 7f7b6198d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ca2853a850 0x55ca28538210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:21.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.668+0000 7f7b6198d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ca2853d5e0 con 0x55ca2853a850 2024-04-24T09:06:21.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.668+0000 7f7b6198d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ca2844e690 con 0x55ca2852f860 2024-04-24T09:06:21.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.668+0000 7f7b6198d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55ca2853d760 con 0x55ca28530180 2024-04-24T09:06:21.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.668+0000 7f7b6198d740 10 client.0 _ll_register_callbacks cb 0x55ca2848a650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:06:21.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.669+0000 7f7b57400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ca2853a850 0x55ca28538210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:21.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.669+0000 7f7b57400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ca2852f860 0x55ca28537790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:21.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.669+0000 7f7b57400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ca2852f860 0x55ca28537790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:38714/0 (socket says 172.21.15.83:38714) 2024-04-24T09:06:21.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.669+0000 7f7b57400640 1 -- 172.21.15.83:0/3401107903 learned_addr learned my addr 172.21.15.83:0/3401107903 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:06:21.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.669+0000 7f7b57e00640 1 --2- 172.21.15.83:0/3401107903 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ca28530180 0x55ca28537cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:21.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.669+0000 7f7b57400640 1 -- 172.21.15.83:0/3401107903 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ca28530180 msgr2=0x55ca28537cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:21.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.669+0000 7f7b57400640 1 --2- 172.21.15.83:0/3401107903 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ca28530180 0x55ca28537cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:21.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.669+0000 7f7b57400640 1 -- 172.21.15.83:0/3401107903 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ca2852f860 msgr2=0x55ca28537790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:21.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.669+0000 7f7b57400640 1 --2- 172.21.15.83:0/3401107903 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ca2852f860 0x55ca28537790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:21.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.669+0000 7f7b57400640 1 -- 172.21.15.83:0/3401107903 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca2844d920 con 0x55ca2853a850 2024-04-24T09:06:21.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.670+0000 7f7b57400640 1 --2- 172.21.15.83:0/3401107903 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ca2853a850 0x55ca28538210 secure :-1 s=READY pgs=2166 cs=0 l=1 rev1=1 crypto rx=0x7f7b4403f9c0 tx=0x7f7b44043040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a471e8edd335520 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:21.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.670+0000 7f7b41600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:06:21.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.670+0000 7f7b41600640 1 -- 172.21.15.83:0/3401107903 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b44052020 con 0x55ca2853a850 2024-04-24T09:06:21.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.670+0000 7f7b41600640 1 -- 172.21.15.83:0/3401107903 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9583+0+0 (secure 0 0 0) 0x7f7b4403f640 con 0x55ca2853a850 2024-04-24T09:06:21.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.670+0000 7f7b6198d740 1 -- 172.21.15.83:0/3401107903 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b440054b0 con 0x55ca2853a850 2024-04-24T09:06:21.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.671+0000 7f7b41600640 1 -- 172.21.15.83:0/3401107903 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f7b4403fa60 con 0x55ca2853a850 2024-04-24T09:06:21.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.671+0000 7f7b6198d740 10 client.20763 fetch_fsmap learned FSMap version 48 2024-04-24T09:06:21.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.671+0000 7f7b6198d740 1 -- 172.21.15.83:0/3401107903 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55ca282717d0 con 0x55ca2853a850 2024-04-24T09:06:21.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.671+0000 7f7b41600640 1 -- 172.21.15.83:0/3401107903 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7b440054b0 con 0x55ca2853a850 2024-04-24T09:06:21.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.672+0000 7f7b41600640 1 -- 172.21.15.83:0/3401107903 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 48) ==== 8402+0+0 (secure 0 0 0) 0x7f7b44031cc0 con 0x55ca2853a850 2024-04-24T09:06:21.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.672+0000 7f7b6198d740 10 client.20763 fetch_fsmap finished waiting for FSMap version 48 2024-04-24T09:06:21.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.672+0000 7f7b6198d740 10 client.20763 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:06:21.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.672+0000 7f7b6198d740 20 client.20763 populate_metadata read hostname 'smithi083' 2024-04-24T09:06:21.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.672+0000 7f7b6198d740 1 --2- 172.21.15.83:0/3401107903 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ca2855ba40 0x55ca2855de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:21.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.672+0000 7f7b6198d740 4 client.20763 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:06:21.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.672+0000 7f7b6198d740 1 -- 172.21.15.83:0/3401107903 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ca284a2c10 con 0x55ca2855ba40 2024-04-24T09:06:21.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.673+0000 7f7b57e00640 1 --2- 172.21.15.83:0/3401107903 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ca2855ba40 0x55ca2855de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:21.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.674+0000 7f7b41600640 10 client.20763 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:06:21.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.674+0000 7f7b57e00640 1 --2- 172.21.15.83:0/3401107903 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ca2855ba40 0x55ca2855de30 crc :-1 s=READY pgs=375 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d19b4d038f507501 server_cookie=bd5413d3c07606ad in_seq=0 out_seq=0 2024-04-24T09:06:21.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.675+0000 7f7b41600640 1 -- 172.21.15.83:0/3401107903 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ca284a2c10 con 0x55ca2855ba40 2024-04-24T09:06:21.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.675+0000 7f7b41600640 10 client.20763 handle_command_reply: tid=0 2024-04-24T09:06:21.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.675+0000 7f7b6198d740 1 client.20763 shutdown 2024-04-24T09:06:21.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.676+0000 7f7b6198d740 1 -- 172.21.15.83:0/3401107903 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ca2853a850 msgr2=0x55ca28538210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:21.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.676+0000 7f7b6198d740 1 --2- 172.21.15.83:0/3401107903 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ca2853a850 0x55ca28538210 secure :-1 s=READY pgs=2166 cs=0 l=1 rev1=1 crypto rx=0x7f7b4403f9c0 tx=0x7f7b44043040 comp rx=0 tx=0).stop 2024-04-24T09:06:21.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.676+0000 7f7b6198d740 1 -- 172.21.15.83:0/3401107903 shutdown_connections 2024-04-24T09:06:21.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.676+0000 7f7b6198d740 1 --2- 172.21.15.83:0/3401107903 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ca2855ba40 0x55ca2855de30 crc :-1 s=READY pgs=375 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:21.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.676+0000 7f7b41600640 0 client.20763 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:06:21.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.676+0000 7f7b6198d740 1 --2- 172.21.15.83:0/3401107903 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ca2853a850 0x55ca28538210 unknown :-1 s=CLOSED pgs=2166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:21.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.676+0000 7f7b6198d740 1 --2- 172.21.15.83:0/3401107903 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ca28530180 0x55ca28537cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:21.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.676+0000 7f7b6198d740 1 --2- 172.21.15.83:0/3401107903 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ca2852f860 0x55ca28537790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:21.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.676+0000 7f7b6198d740 1 -- 172.21.15.83:0/3401107903 >> 172.21.15.83:0/3401107903 conn(0x55ca2852bb80 msgr2=0x55ca28531880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:21.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.676+0000 7f7b6198d740 1 -- 172.21.15.83:0/3401107903 shutdown_connections 2024-04-24T09:06:21.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.676+0000 7f7b6198d740 1 -- 172.21.15.83:0/3401107903 wait complete. 2024-04-24T09:06:21.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.676+0000 7f7b6198d740 20 client.20763 trim_cache size 0 max 16384 2024-04-24T09:06:21.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.702+0000 7f7b6198d740 1 Processor -- start 2024-04-24T09:06:21.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.702+0000 7f7b6198d740 1 -- start start 2024-04-24T09:06:21.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.702+0000 7f7b6198d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ca2852f860 0x55ca285366d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:21.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.702+0000 7f7b6198d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ca28530180 0x55ca28538030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:21.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.702+0000 7f7b6198d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ca28560e50 0x55ca285612d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:21.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.702+0000 7f7b6198d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7b581ad4d0 con 0x55ca28530180 2024-04-24T09:06:21.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.702+0000 7f7b6198d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7b581af190 con 0x55ca2852f860 2024-04-24T09:06:21.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.702+0000 7f7b6198d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7b581adba0 con 0x55ca28560e50 2024-04-24T09:06:21.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.703+0000 7f7b57e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ca28530180 0x55ca28538030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:21.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.703+0000 7f7b57e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ca28530180 0x55ca28538030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55702/0 (socket says 172.21.15.83:55702) 2024-04-24T09:06:21.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.703+0000 7f7b57e00640 1 -- 172.21.15.83:0/2813924276 learned_addr learned my addr 172.21.15.83:0/2813924276 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:06:21.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.703+0000 7f7b57400640 1 --2- 172.21.15.83:0/2813924276 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ca2852f860 0x55ca285366d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:21.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.703+0000 7f7b57400640 1 --2- 172.21.15.83:0/2813924276 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ca28560e50 0x55ca285612d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:21.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.703+0000 7f7b57400640 1 -- 172.21.15.83:0/2813924276 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ca28560e50 msgr2=0x55ca285612d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:21.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.703+0000 7f7b57400640 1 --2- 172.21.15.83:0/2813924276 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ca28560e50 0x55ca285612d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:21.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.703+0000 7f7b57400640 1 -- 172.21.15.83:0/2813924276 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ca28530180 msgr2=0x55ca28538030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:21.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.703+0000 7f7b57400640 1 --2- 172.21.15.83:0/2813924276 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ca28530180 0x55ca28538030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:21.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.703+0000 7f7b57400640 1 -- 172.21.15.83:0/2813924276 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca281a73f0 con 0x55ca2852f860 2024-04-24T09:06:21.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.703+0000 7f7b57e00640 1 --2- 172.21.15.83:0/2813924276 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ca28530180 0x55ca28538030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:06:21.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.704+0000 7f7b57400640 1 --2- 172.21.15.83:0/2813924276 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ca2852f860 0x55ca285366d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7b44003820 tx=0x7f7b44009600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb5185e6bd196ba4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:21.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.704+0000 7f7b43e00640 1 -- 172.21.15.83:0/2813924276 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b44052020 con 0x55ca2852f860 2024-04-24T09:06:21.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.704+0000 7f7b43e00640 1 -- 172.21.15.83:0/2813924276 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7b44005310 con 0x55ca2852f860 2024-04-24T09:06:21.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.704+0000 7f7b43e00640 1 -- 172.21.15.83:0/2813924276 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b4403cdb0 con 0x55ca2852f860 2024-04-24T09:06:21.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.705+0000 7f7b6198d740 1 -- 172.21.15.83:0/2813924276 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ca2852f860 msgr2=0x55ca285366d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:21.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.705+0000 7f7b6198d740 1 --2- 172.21.15.83:0/2813924276 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ca2852f860 0x55ca285366d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7b44003820 tx=0x7f7b44009600 comp rx=0 tx=0).stop 2024-04-24T09:06:21.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.705+0000 7f7b6198d740 1 -- 172.21.15.83:0/2813924276 shutdown_connections 2024-04-24T09:06:21.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.705+0000 7f7b6198d740 1 --2- 172.21.15.83:0/2813924276 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ca28560e50 0x55ca285612d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:21.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.705+0000 7f7b6198d740 1 --2- 172.21.15.83:0/2813924276 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ca28530180 0x55ca28538030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:21.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.705+0000 7f7b6198d740 1 --2- 172.21.15.83:0/2813924276 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ca2852f860 0x55ca285366d0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:21.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.705+0000 7f7b6198d740 1 -- 172.21.15.83:0/2813924276 >> 172.21.15.83:0/2813924276 conn(0x55ca2852bb80 msgr2=0x55ca2852ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:21.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.706+0000 7f7b6198d740 1 -- 172.21.15.83:0/2813924276 shutdown_connections 2024-04-24T09:06:21.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.706+0000 7f7b6198d740 1 -- 172.21.15.83:0/2813924276 wait complete. 2024-04-24T09:06:21.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.706+0000 7f7b6198d740 1 -- start start 2024-04-24T09:06:21.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.706+0000 7f7b6198d740 1 Processor -- start 2024-04-24T09:06:21.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.707+0000 7f7b6198d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ca2852f860 0x55ca2855fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:21.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.707+0000 7f7b6198d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ca28530180 0x55ca285603a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:21.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.707+0000 7f7b6198d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ca28560e50 0x55ca285608e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:21.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.707+0000 7f7b6198d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7b581aea00 con 0x55ca28530180 2024-04-24T09:06:21.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.707+0000 7f7b6198d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7b581ae2d0 con 0x55ca2852f860 2024-04-24T09:06:21.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.707+0000 7f7b6198d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7b58073bb0 con 0x55ca28560e50 2024-04-24T09:06:21.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.707+0000 7f7b6198d740 10 client.0 _ll_register_callbacks cb 0x7f7b44005d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:06:21.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.707+0000 7f7b57400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ca2852f860 0x55ca2855fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:21.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.707+0000 7f7b57400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ca2852f860 0x55ca2855fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:38734/0 (socket says 172.21.15.83:38734) 2024-04-24T09:06:21.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.707+0000 7f7b57400640 1 -- 172.21.15.83:0/4005756606 learned_addr learned my addr 172.21.15.83:0/4005756606 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:06:21.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.707+0000 7f7b57e00640 1 --2- 172.21.15.83:0/4005756606 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ca28530180 0x55ca285603a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:21.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.708+0000 7f7b57400640 1 --2- 172.21.15.83:0/4005756606 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ca28560e50 0x55ca285608e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:21.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.708+0000 7f7b57e00640 1 -- 172.21.15.83:0/4005756606 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ca28560e50 msgr2=0x55ca285608e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:21.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.708+0000 7f7b57e00640 1 --2- 172.21.15.83:0/4005756606 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ca28560e50 0x55ca285608e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:21.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.708+0000 7f7b57e00640 1 -- 172.21.15.83:0/4005756606 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ca2852f860 msgr2=0x55ca2855fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:21.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.708+0000 7f7b57e00640 1 --2- 172.21.15.83:0/4005756606 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ca2852f860 0x55ca2855fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:21.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.708+0000 7f7b57e00640 1 -- 172.21.15.83:0/4005756606 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca281670f0 con 0x55ca28530180 2024-04-24T09:06:21.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.709+0000 7f7b57e00640 1 --2- 172.21.15.83:0/4005756606 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ca28530180 0x55ca285603a0 secure :-1 s=READY pgs=2167 cs=0 l=1 rev1=1 crypto rx=0x7f7b4800bda0 tx=0x7f7b48011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39903d1af2f73926 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:21.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.709+0000 7f7b41600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:06:21.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.709+0000 7f7b41600640 1 -- 172.21.15.83:0/4005756606 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b48005220 con 0x55ca28530180 2024-04-24T09:06:21.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.709+0000 7f7b41600640 1 -- 172.21.15.83:0/4005756606 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9583+0+0 (secure 0 0 0) 0x7f7b480053c0 con 0x55ca28530180 2024-04-24T09:06:21.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.709+0000 7f7b6198d740 1 -- 172.21.15.83:0/4005756606 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b44005310 con 0x55ca28530180 2024-04-24T09:06:21.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.709+0000 7f7b41600640 1 -- 172.21.15.83:0/4005756606 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f7b480055c0 con 0x55ca28530180 2024-04-24T09:06:21.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.710+0000 7f7b6198d740 10 client.20769 fetch_fsmap learned FSMap version 48 2024-04-24T09:06:21.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.710+0000 7f7b6198d740 1 -- 172.21.15.83:0/4005756606 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55ca2845a810 con 0x55ca28530180 2024-04-24T09:06:21.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.710+0000 7f7b41600640 1 -- 172.21.15.83:0/4005756606 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7b48004c50 con 0x55ca28530180 2024-04-24T09:06:21.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.710+0000 7f7b41600640 1 -- 172.21.15.83:0/4005756606 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 48) ==== 8402+0+0 (secure 0 0 0) 0x7f7b48028480 con 0x55ca28530180 2024-04-24T09:06:21.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.710+0000 7f7b6198d740 10 client.20769 fetch_fsmap finished waiting for FSMap version 48 2024-04-24T09:06:21.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.710+0000 7f7b6198d740 10 client.20769 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:06:21.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.710+0000 7f7b6198d740 20 client.20769 populate_metadata read hostname 'smithi083' 2024-04-24T09:06:21.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.710+0000 7f7b6198d740 1 --2- 172.21.15.83:0/4005756606 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ca285d3440 0x55ca285d5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:21.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.710+0000 7f7b6198d740 4 client.20769 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T09:06:21.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.710+0000 7f7b6198d740 1 -- 172.21.15.83:0/4005756606 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f7b44005310 con 0x55ca285d3440 2024-04-24T09:06:21.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.711+0000 7f7b57400640 1 --2- 172.21.15.83:0/4005756606 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ca285d3440 0x55ca285d5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:21.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.711+0000 7f7b57400640 1 --2- 172.21.15.83:0/4005756606 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ca285d3440 0x55ca285d5860 crc :-1 s=READY pgs=376 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6aca07b1c057c397 server_cookie=5017afd6c5d531e7 in_seq=0 out_seq=0 2024-04-24T09:06:21.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.711+0000 7f7b41600640 10 client.20769 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:06:21.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.712+0000 7f7b41600640 1 -- 172.21.15.83:0/4005756606 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7f7b44005310 con 0x55ca285d3440 2024-04-24T09:06:21.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.712+0000 7f7b41600640 10 client.20769 handle_command_reply: tid=0 2024-04-24T09:06:21.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.712+0000 7f7b6198d740 1 client.20769 shutdown 2024-04-24T09:06:21.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.712+0000 7f7b6198d740 1 -- 172.21.15.83:0/4005756606 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ca28530180 msgr2=0x55ca285603a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:21.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.712+0000 7f7b6198d740 1 --2- 172.21.15.83:0/4005756606 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ca28530180 0x55ca285603a0 secure :-1 s=READY pgs=2167 cs=0 l=1 rev1=1 crypto rx=0x7f7b4800bda0 tx=0x7f7b48011b00 comp rx=0 tx=0).stop 2024-04-24T09:06:21.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.712+0000 7f7b6198d740 1 -- 172.21.15.83:0/4005756606 shutdown_connections 2024-04-24T09:06:21.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.712+0000 7f7b6198d740 1 --2- 172.21.15.83:0/4005756606 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ca285d3440 0x55ca285d5860 crc :-1 s=READY pgs=376 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:21.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.712+0000 7f7b6198d740 1 --2- 172.21.15.83:0/4005756606 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ca28560e50 0x55ca285608e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:21.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.712+0000 7f7b6198d740 1 --2- 172.21.15.83:0/4005756606 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ca28530180 0x55ca285603a0 unknown :-1 s=CLOSED pgs=2167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:21.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.712+0000 7f7b6198d740 1 --2- 172.21.15.83:0/4005756606 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ca2852f860 0x55ca2855fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:21.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.712+0000 7f7b6198d740 1 -- 172.21.15.83:0/4005756606 >> 172.21.15.83:0/4005756606 conn(0x55ca2852bb80 msgr2=0x55ca28545a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:21.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.712+0000 7f7b41600640 0 client.20769 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:06:21.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.713+0000 7f7b6198d740 1 -- 172.21.15.83:0/4005756606 shutdown_connections 2024-04-24T09:06:21.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.713+0000 7f7b6198d740 1 -- 172.21.15.83:0/4005756606 wait complete. 2024-04-24T09:06:21.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.713+0000 7f7b6198d740 20 client.20769 trim_cache size 0 max 16384 2024-04-24T09:06:21.715 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T09:06:21.715 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-04-24T09:06:21.715 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-04-24T09:06:21.715 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T09:06:21.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.714+0000 7f7b5ea00640 1 -- 172.21.15.83:0/1408880452 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f7b3c0a3da0 msgr2=0x7f7b3c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:21.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.714+0000 7f7b5ea00640 1 --2- 172.21.15.83:0/1408880452 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f7b3c0a3da0 0x7f7b3c0a6260 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f7b500ce5f0 tx=0x7f7b50005f70 comp rx=0 tx=0).stop 2024-04-24T09:06:21.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.714+0000 7f7b5ea00640 1 -- 172.21.15.83:0/1408880452 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7b5810e660 msgr2=0x7f7b581a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:21.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.714+0000 7f7b5ea00640 1 --2- 172.21.15.83:0/1408880452 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7b5810e660 0x7f7b581a36d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7b50009fd0 tx=0x7f7b5000ef10 comp rx=0 tx=0).stop 2024-04-24T09:06:21.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.714+0000 7f7b5ea00640 1 -- 172.21.15.83:0/1408880452 shutdown_connections 2024-04-24T09:06:21.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.714+0000 7f7b5ea00640 1 --2- 172.21.15.83:0/1408880452 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f7b3c0a3da0 0x7f7b3c0a6260 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:21.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.714+0000 7f7b5ea00640 1 --2- 172.21.15.83:0/1408880452 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7b5810e660 0x7f7b581a36d0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:21.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.714+0000 7f7b5ea00640 1 --2- 172.21.15.83:0/1408880452 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7b58108fd0 0x7f7b5819f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:21.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.714+0000 7f7b5ea00640 1 --2- 172.21.15.83:0/1408880452 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b58108610 0x7f7b5819ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:21.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.714+0000 7f7b5ea00640 1 -- 172.21.15.83:0/1408880452 >> 172.21.15.83:0/1408880452 conn(0x7f7b58076540 msgr2=0x7f7b58077e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:21.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.714+0000 7f7b5ea00640 1 -- 172.21.15.83:0/1408880452 shutdown_connections 2024-04-24T09:06:21.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.715+0000 7f7b5ea00640 1 -- 172.21.15.83:0/1408880452 wait complete. 2024-04-24T09:06:21.736 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T09:06:21.736 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T09:06:21.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.982+0000 7f5830800640 1 -- 172.21.15.83:0/2784457112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5828100e20 msgr2=0x7f5828104e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:21.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.982+0000 7f5830800640 1 --2- 172.21.15.83:0/2784457112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5828100e20 0x7f5828104e60 secure :-1 s=READY pgs=2168 cs=0 l=1 rev1=1 crypto rx=0x7f58180074e0 tx=0x7f581802b070 comp rx=0 tx=0).stop 2024-04-24T09:06:21.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.983+0000 7f5830800640 1 -- 172.21.15.83:0/2784457112 shutdown_connections 2024-04-24T09:06:21.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.983+0000 7f5830800640 1 --2- 172.21.15.83:0/2784457112 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5828105540 0x7f582810d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:21.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.983+0000 7f5830800640 1 --2- 172.21.15.83:0/2784457112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5828100e20 0x7f5828104e60 unknown :-1 s=CLOSED pgs=2168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:21.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.983+0000 7f5830800640 1 --2- 172.21.15.83:0/2784457112 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5828100470 0x7f5828100850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:21.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.983+0000 7f5830800640 1 -- 172.21.15.83:0/2784457112 >> 172.21.15.83:0/2784457112 conn(0x7f5828077060 msgr2=0x7f58280fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:21.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.983+0000 7f5830800640 1 -- 172.21.15.83:0/2784457112 shutdown_connections 2024-04-24T09:06:21.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.983+0000 7f5830800640 1 -- 172.21.15.83:0/2784457112 wait complete. 2024-04-24T09:06:21.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.985+0000 7f5830800640 1 Processor -- start 2024-04-24T09:06:21.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.985+0000 7f5830800640 1 -- start start 2024-04-24T09:06:21.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.986+0000 7f5830800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5828100470 0x7f58280738a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:21.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.986+0000 7f5830800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5828105540 0x7f582806c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:21.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.986+0000 7f5830800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f582806ceb0 0x7f582806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:21.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.986+0000 7f582de00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5828100470 0x7f58280738a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:21.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.986+0000 7f5830800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5828110080 con 0x7f582806ceb0 2024-04-24T09:06:21.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.986+0000 7f582e800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f582806ceb0 0x7f582806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:21.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.986+0000 7f582e800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f582806ceb0 0x7f582806d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55718/0 (socket says 172.21.15.83:55718) 2024-04-24T09:06:21.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.986+0000 7f582e800640 1 -- 172.21.15.83:0/4116579888 learned_addr learned my addr 172.21.15.83:0/4116579888 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:06:21.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.986+0000 7f5830800640 1 -- 172.21.15.83:0/4116579888 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f582810ff00 con 0x7f5828100470 2024-04-24T09:06:21.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.986+0000 7f5830800640 1 -- 172.21.15.83:0/4116579888 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5828110200 con 0x7f5828105540 2024-04-24T09:06:21.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.986+0000 7f582de00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5828100470 0x7f58280738a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:38752/0 (socket says 172.21.15.83:38752) 2024-04-24T09:06:21.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.986+0000 7f582d400640 1 --2- 172.21.15.83:0/4116579888 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5828105540 0x7f582806c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:21.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.987+0000 7f582e800640 1 -- 172.21.15.83:0/4116579888 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5828105540 msgr2=0x7f582806c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:21.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.987+0000 7f582e800640 1 --2- 172.21.15.83:0/4116579888 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5828105540 0x7f582806c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:21.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.987+0000 7f582e800640 1 -- 172.21.15.83:0/4116579888 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5828100470 msgr2=0x7f58280738a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:21.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.987+0000 7f582e800640 1 --2- 172.21.15.83:0/4116579888 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5828100470 0x7f58280738a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:21.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.987+0000 7f582e800640 1 -- 172.21.15.83:0/4116579888 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58280713c0 con 0x7f582806ceb0 2024-04-24T09:06:21.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.987+0000 7f582d400640 1 --2- 172.21.15.83:0/4116579888 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5828105540 0x7f582806c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:06:21.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.988+0000 7f582e800640 1 --2- 172.21.15.83:0/4116579888 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f582806ceb0 0x7f582806d360 secure :-1 s=READY pgs=2169 cs=0 l=1 rev1=1 crypto rx=0x7f582400eaa0 tx=0x7f582400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:21.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.988+0000 7f5822a00640 1 -- 172.21.15.83:0/4116579888 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f582400cdb0 con 0x7f582806ceb0 2024-04-24T09:06:21.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.988+0000 7f5830800640 1 -- 172.21.15.83:0/4116579888 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58280716b0 con 0x7f582806ceb0 2024-04-24T09:06:21.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.988+0000 7f5830800640 1 -- 172.21.15.83:0/4116579888 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5828071c00 con 0x7f582806ceb0 2024-04-24T09:06:21.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.988+0000 7f5822a00640 1 -- 172.21.15.83:0/4116579888 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5824005020 con 0x7f582806ceb0 2024-04-24T09:06:21.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.988+0000 7f5822a00640 1 -- 172.21.15.83:0/4116579888 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58240116d0 con 0x7f582806ceb0 2024-04-24T09:06:21.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.990+0000 7f5830800640 1 -- 172.21.15.83:0/4116579888 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f580c005290 con 0x7f582806ceb0 2024-04-24T09:06:21.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.994+0000 7f5822a00640 1 -- 172.21.15.83:0/4116579888 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5824004a40 con 0x7f582806ceb0 2024-04-24T09:06:21.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.995+0000 7f5822a00640 1 --2- 172.21.15.83:0/4116579888 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f58100a3da0 0x7f58100a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:21.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.996+0000 7f5822a00640 1 -- 172.21.15.83:0/4116579888 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9583+0+0 (secure 0 0 0) 0x7f58240d0c40 con 0x7f582806ceb0 2024-04-24T09:06:21.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.996+0000 7f582de00640 1 --2- 172.21.15.83:0/4116579888 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f58100a3da0 0x7f58100a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:21.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:21.997+0000 7f582de00640 1 --2- 172.21.15.83:0/4116579888 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f58100a3da0 0x7f58100a6260 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f581c005fd0 tx=0x7f581c00b390 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:22.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.002+0000 7f5822a00640 1 -- 172.21.15.83:0/4116579888 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5824016030 con 0x7f582806ceb0 2024-04-24T09:06:22.004 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:06:22 smithi083 ceph-mon[27907]: pgmap v463: 129 pgs: 129 active+clean; 335 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 693 KiB/s rd, 2.7 MiB/s wr, 192 op/s 2024-04-24T09:06:22.157 INFO:tasks.workunit.client.0.smithi083.stdout: 10 6860 10.30 MB/sec warmup 53 sec latency 86.114 ms 2024-04-24T09:06:22.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.437+0000 7f5830800640 1 -- 172.21.15.83:0/4116579888 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f580c005850 con 0x7f582806ceb0 2024-04-24T09:06:22.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.438+0000 7f5822a00640 1 -- 172.21.15.83:0/4116579888 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+11959 (secure 0 0 0) 0x7f582409ce60 con 0x7f582806ceb0 2024-04-24T09:06:22.438 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T09:06:22.438 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":48,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/358872257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":358872257},{"type":"v1","addr":"172.21.15.83:6835","nonce":358872257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3157070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3157070851},{"type":"v1","addr":"172.21.15.83:6837","nonce":3157070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":48,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T08:48:29.422576+0000","modified":"2024-04-24T09:06:19.458421+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34520,"mds_1":34493,"mds_2":34451,"mds_3":34427,"mds_4":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14658":{"gid":14658,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6839/4085470844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":4085470844},{"type":"v1","addr":"172.21.15.83:6839","nonce":4085470844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14676":{"gid":14676,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6841/2634754210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":2634754210},{"type":"v1","addr":"172.21.15.83:6841","nonce":2634754210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24382":{"gid":24382,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/1859921310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1859921310},{"type":"v1","addr":"172.21.15.189:6833","nonce":1859921310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/2241923197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2241923197},{"type":"v1","addr":"172.21.15.189:6835","nonce":2241923197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.189:6839/2548965336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":2548965336},{"type":"v1","addr":"172.21.15.189:6839","nonce":2548965336}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26686":{"gid":26686,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6837/1678817161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":1678817161},{"type":"v1","addr":"172.21.15.189:6837","nonce":1678817161}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.138:6833/1157967658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1157967658},{"type":"v1","addr":"172.21.15.138:6833","nonce":1157967658}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.138:6835/1909219758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1909219758},{"type":"v1","addr":"172.21.15.138:6835","nonce":1909219758}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34493":{"gid":34493,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.138:6837/900566754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":900566754},{"type":"v1","addr":"172.21.15.138:6837","nonce":900566754}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34520":{"gid":34520,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.138:6839/2236932393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2236932393},{"type":"v1","addr":"172.21.15.138:6839","nonce":2236932393}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34520,"qdb_cluster":[34451,34493,34427,34520,24454]},"id":1}]} 2024-04-24T09:06:22.439 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 48 2024-04-24T09:06:22.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.441+0000 7f5830800640 1 -- 172.21.15.83:0/4116579888 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f58100a3da0 msgr2=0x7f58100a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:22.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.441+0000 7f5830800640 1 --2- 172.21.15.83:0/4116579888 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f58100a3da0 0x7f58100a6260 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f581c005fd0 tx=0x7f581c00b390 comp rx=0 tx=0).stop 2024-04-24T09:06:22.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.441+0000 7f5830800640 1 -- 172.21.15.83:0/4116579888 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f582806ceb0 msgr2=0x7f582806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:22.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.441+0000 7f5830800640 1 --2- 172.21.15.83:0/4116579888 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f582806ceb0 0x7f582806d360 secure :-1 s=READY pgs=2169 cs=0 l=1 rev1=1 crypto rx=0x7f582400eaa0 tx=0x7f582400ef60 comp rx=0 tx=0).stop 2024-04-24T09:06:22.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.441+0000 7f5830800640 1 -- 172.21.15.83:0/4116579888 shutdown_connections 2024-04-24T09:06:22.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.441+0000 7f5830800640 1 --2- 172.21.15.83:0/4116579888 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f58100a3da0 0x7f58100a6260 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:22.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.441+0000 7f5830800640 1 --2- 172.21.15.83:0/4116579888 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f582806ceb0 0x7f582806d360 unknown :-1 s=CLOSED pgs=2169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:22.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.441+0000 7f5830800640 1 --2- 172.21.15.83:0/4116579888 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5828105540 0x7f582806c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:22.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.441+0000 7f5830800640 1 --2- 172.21.15.83:0/4116579888 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5828100470 0x7f58280738a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:22.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.441+0000 7f5830800640 1 -- 172.21.15.83:0/4116579888 >> 172.21.15.83:0/4116579888 conn(0x7f5828077060 msgr2=0x7f5828103840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:22.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.442+0000 7f5830800640 1 -- 172.21.15.83:0/4116579888 shutdown_connections 2024-04-24T09:06:22.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.442+0000 7f5830800640 1 -- 172.21.15.83:0/4116579888 wait complete. 2024-04-24T09:06:22.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:06:22 smithi138 ceph-mon[31441]: pgmap v463: 129 pgs: 129 active+clean; 335 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 693 KiB/s rd, 2.7 MiB/s wr, 192 op/s 2024-04-24T09:06:22.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:06:22 smithi189 ceph-mon[31791]: pgmap v463: 129 pgs: 129 active+clean; 335 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 693 KiB/s rd, 2.7 MiB/s wr, 192 op/s 2024-04-24T09:06:22.464 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-24T09:06:22.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.713+0000 7f8020000640 1 -- 172.21.15.83:0/4007667819 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8018109580 msgr2=0x7f8018111150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:22.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.713+0000 7f8020000640 1 --2- 172.21.15.83:0/4007667819 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8018109580 0x7f8018111150 secure :-1 s=READY pgs=2170 cs=0 l=1 rev1=1 crypto rx=0x7f8010009ca0 tx=0x7f801002a8c0 comp rx=0 tx=0).stop 2024-04-24T09:06:22.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.713+0000 7f8020000640 1 -- 172.21.15.83:0/4007667819 shutdown_connections 2024-04-24T09:06:22.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.713+0000 7f8020000640 1 --2- 172.21.15.83:0/4007667819 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8018109580 0x7f8018111150 unknown :-1 s=CLOSED pgs=2170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:22.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.713+0000 7f8020000640 1 --2- 172.21.15.83:0/4007667819 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8018104e60 0x7f8018108ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:22.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.713+0000 7f8020000640 1 --2- 172.21.15.83:0/4007667819 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f80181044b0 0x7f8018104890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:22.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.713+0000 7f8020000640 1 -- 172.21.15.83:0/4007667819 >> 172.21.15.83:0/4007667819 conn(0x7f80180ffd90 msgr2=0x7f80181021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:22.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.713+0000 7f8020000640 1 -- 172.21.15.83:0/4007667819 shutdown_connections 2024-04-24T09:06:22.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.713+0000 7f8020000640 1 -- 172.21.15.83:0/4007667819 wait complete. 2024-04-24T09:06:22.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.714+0000 7f8020000640 1 Processor -- start 2024-04-24T09:06:22.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.715+0000 7f8020000640 1 -- start start 2024-04-24T09:06:22.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.715+0000 7f8020000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f80181044b0 0x7f80181a0720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:22.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.716+0000 7f8020000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8018109580 0x7f80181a0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:22.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.716+0000 7f8020000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f801819a920 0x7f801819add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:22.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.716+0000 7f8020000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8018114540 con 0x7f80181044b0 2024-04-24T09:06:22.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.716+0000 7f8020000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f80181143c0 con 0x7f801819a920 2024-04-24T09:06:22.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.716+0000 7f8020000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f80181146c0 con 0x7f8018109580 2024-04-24T09:06:22.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.716+0000 7f801e000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f801819a920 0x7f801819add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:22.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.716+0000 7f801e000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f801819a920 0x7f801819add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:38774/0 (socket says 172.21.15.83:38774) 2024-04-24T09:06:22.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.716+0000 7f801e000640 1 -- 172.21.15.83:0/675389110 learned_addr learned my addr 172.21.15.83:0/675389110 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:06:22.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.716+0000 7f801d600640 1 --2- 172.21.15.83:0/675389110 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f80181044b0 0x7f80181a0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:22.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.716+0000 7f801cc00640 1 --2- 172.21.15.83:0/675389110 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8018109580 0x7f80181a0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:22.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.717+0000 7f801e000640 1 -- 172.21.15.83:0/675389110 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8018109580 msgr2=0x7f80181a0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:22.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.717+0000 7f801e000640 1 --2- 172.21.15.83:0/675389110 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8018109580 0x7f80181a0c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:22.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.717+0000 7f801e000640 1 -- 172.21.15.83:0/675389110 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f80181044b0 msgr2=0x7f80181a0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:22.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.717+0000 7f801e000640 1 --2- 172.21.15.83:0/675389110 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f80181044b0 0x7f80181a0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:22.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.717+0000 7f801e000640 1 -- 172.21.15.83:0/675389110 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f801819b440 con 0x7f801819a920 2024-04-24T09:06:22.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.717+0000 7f801e000640 1 --2- 172.21.15.83:0/675389110 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f801819a920 0x7f801819add0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f8010031040 tx=0x7f8010032f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:22.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.717+0000 7f8016000640 1 -- 172.21.15.83:0/675389110 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8010003b70 con 0x7f801819a920 2024-04-24T09:06:22.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.718+0000 7f8020000640 1 -- 172.21.15.83:0/675389110 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f801819b6d0 con 0x7f801819a920 2024-04-24T09:06:22.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.718+0000 7f8016000640 1 -- 172.21.15.83:0/675389110 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8010003d10 con 0x7f801819a920 2024-04-24T09:06:22.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.718+0000 7f8016000640 1 -- 172.21.15.83:0/675389110 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8010030b20 con 0x7f801819a920 2024-04-24T09:06:22.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.718+0000 7f8020000640 1 -- 172.21.15.83:0/675389110 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80181a7860 con 0x7f801819a920 2024-04-24T09:06:22.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.721+0000 7f8016000640 1 -- 172.21.15.83:0/675389110 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8010030cc0 con 0x7f801819a920 2024-04-24T09:06:22.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.722+0000 7f8022f7f740 1 Processor -- start 2024-04-24T09:06:22.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.723+0000 7f8022f7f740 1 -- start start 2024-04-24T09:06:22.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.723+0000 7f8022f7f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5590a73ae160 0x5590a73ae540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:22.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.723+0000 7f8022f7f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590a73aea80 0x5590a73b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:22.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.723+0000 7f8022f7f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5590a73b9150 0x5590a73bb540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:22.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.723+0000 7f8022f7f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5590a730b1d0 con 0x5590a73b9150 2024-04-24T09:06:22.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.723+0000 7f8022f7f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5590a72b1450 con 0x5590a73aea80 2024-04-24T09:06:22.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.723+0000 7f8022f7f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5590a7307190 con 0x5590a73ae160 2024-04-24T09:06:22.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.723+0000 7f8016000640 1 --2- 172.21.15.83:0/675389110 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f80000a3da0 0x7f80000a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:22.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.723+0000 7f801d600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590a73aea80 0x5590a73b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:22.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.723+0000 7f801d600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590a73aea80 0x5590a73b8c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:38790/0 (socket says 172.21.15.83:38790) 2024-04-24T09:06:22.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.723+0000 7f801d600640 1 -- 172.21.15.83:0/3658765745 learned_addr learned my addr 172.21.15.83:0/3658765745 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:06:22.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.723+0000 7f8016000640 1 -- 172.21.15.83:0/675389110 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9583+0+0 (secure 0 0 0) 0x7f80100f1e10 con 0x7f801819a920 2024-04-24T09:06:22.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.723+0000 7f801cc00640 1 --2- 172.21.15.83:0/3658765745 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5590a73b9150 0x5590a73bb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:22.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.724+0000 7f801cc00640 1 --2- 172.21.15.83:0/3658765745 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5590a73ae160 0x5590a73ae540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:22.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.724+0000 7f801e000640 1 --2- 172.21.15.83:0/675389110 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f80000a3da0 0x7f80000a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:22.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.724+0000 7f801d600640 1 -- 172.21.15.83:0/3658765745 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5590a73ae160 msgr2=0x5590a73ae540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:22.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.724+0000 7f801d600640 1 --2- 172.21.15.83:0/3658765745 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5590a73ae160 0x5590a73ae540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:22.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.724+0000 7f801d600640 1 -- 172.21.15.83:0/3658765745 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5590a73b9150 msgr2=0x5590a73bb540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:22.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.724+0000 7f801d600640 1 --2- 172.21.15.83:0/3658765745 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5590a73b9150 0x5590a73bb540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:22.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.724+0000 7f801d600640 1 -- 172.21.15.83:0/3658765745 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590a736e310 con 0x5590a73aea80 2024-04-24T09:06:22.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.725+0000 7f801d600640 1 --2- 172.21.15.83:0/3658765745 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590a73aea80 0x5590a73b8c10 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f800c009c90 tx=0x7f800c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e6bc1dc8f5b98bf server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:22.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.725+0000 7f800a600640 1 -- 172.21.15.83:0/3658765745 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f800c005220 con 0x5590a73aea80 2024-04-24T09:06:22.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.725+0000 7f801e000640 1 --2- 172.21.15.83:0/675389110 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f80000a3da0 0x7f80000a6260 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f80100c6180 tx=0x7f8010005ef0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:22.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.725+0000 7f800a600640 1 -- 172.21.15.83:0/3658765745 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f800c004c50 con 0x5590a73aea80 2024-04-24T09:06:22.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.725+0000 7f800a600640 1 -- 172.21.15.83:0/3658765745 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f800c019670 con 0x5590a73aea80 2024-04-24T09:06:22.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.726+0000 7f8022f7f740 1 -- 172.21.15.83:0/3658765745 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590a73aea80 msgr2=0x5590a73b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:22.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.726+0000 7f8022f7f740 1 --2- 172.21.15.83:0/3658765745 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590a73aea80 0x5590a73b8c10 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f800c009c90 tx=0x7f800c011810 comp rx=0 tx=0).stop 2024-04-24T09:06:22.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.726+0000 7f8022f7f740 1 -- 172.21.15.83:0/3658765745 shutdown_connections 2024-04-24T09:06:22.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.726+0000 7f8022f7f740 1 --2- 172.21.15.83:0/3658765745 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5590a73b9150 0x5590a73bb540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:22.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.726+0000 7f8022f7f740 1 --2- 172.21.15.83:0/3658765745 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590a73aea80 0x5590a73b8c10 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:22.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.726+0000 7f8022f7f740 1 --2- 172.21.15.83:0/3658765745 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5590a73ae160 0x5590a73ae540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:22.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.726+0000 7f8022f7f740 1 -- 172.21.15.83:0/3658765745 >> 172.21.15.83:0/3658765745 conn(0x5590a73aa480 msgr2=0x5590a73aa860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:22.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.726+0000 7f8022f7f740 1 -- 172.21.15.83:0/3658765745 shutdown_connections 2024-04-24T09:06:22.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.726+0000 7f8022f7f740 1 -- 172.21.15.83:0/3658765745 wait complete. 2024-04-24T09:06:22.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.727+0000 7f8022f7f740 1 -- start start 2024-04-24T09:06:22.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.727+0000 7f8022f7f740 1 Processor -- start 2024-04-24T09:06:22.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.728+0000 7f8022f7f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5590a73ae160 0x5590a73b8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:22.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.728+0000 7f8022f7f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5590a73aea80 0x5590a73b5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:22.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.728+0000 7f8022f7f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590a73b9150 0x5590a73b65d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:22.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.728+0000 7f8022f7f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5590a73bbfe0 con 0x5590a73aea80 2024-04-24T09:06:22.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.728+0000 7f8022f7f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5590a72c4810 con 0x5590a73b9150 2024-04-24T09:06:22.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.728+0000 7f8022f7f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5590a73bc160 con 0x5590a73ae160 2024-04-24T09:06:22.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.728+0000 7f8022f7f740 10 client.0 _ll_register_callbacks cb 0x5590a7292140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:06:22.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.729+0000 7f801d600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5590a73aea80 0x5590a73b5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:22.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.729+0000 7f801d600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5590a73aea80 0x5590a73b5ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55766/0 (socket says 172.21.15.83:55766) 2024-04-24T09:06:22.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.729+0000 7f801d600640 1 -- 172.21.15.83:0/1212156797 learned_addr learned my addr 172.21.15.83:0/1212156797 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:06:22.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.729+0000 7f801cc00640 1 --2- 172.21.15.83:0/1212156797 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5590a73ae160 0x5590a73b8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:22.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.729+0000 7f801d600640 1 -- 172.21.15.83:0/1212156797 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5590a73ae160 msgr2=0x5590a73b8bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:22.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.729+0000 7f801d600640 1 --2- 172.21.15.83:0/1212156797 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5590a73ae160 0x5590a73b8bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:22.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.729+0000 7f801d600640 1 -- 172.21.15.83:0/1212156797 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590a73b9150 msgr2=0x5590a73b65d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:22.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.729+0000 7f801d600640 1 --2- 172.21.15.83:0/1212156797 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590a73b9150 0x5590a73b65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:22.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.729+0000 7f801d600640 1 -- 172.21.15.83:0/1212156797 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590a72d53f0 con 0x5590a73aea80 2024-04-24T09:06:22.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.729+0000 7f801cc00640 1 --2- 172.21.15.83:0/1212156797 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5590a73ae160 0x5590a73b8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:06:22.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.729+0000 7f801d600640 1 --2- 172.21.15.83:0/1212156797 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5590a73aea80 0x5590a73b5ee0 secure :-1 s=READY pgs=2171 cs=0 l=1 rev1=1 crypto rx=0x7f800c01b040 tx=0x7f800c005230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceaed863b748dbc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:22.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.729+0000 7f7ffea00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:06:22.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.730+0000 7f7ffea00640 1 -- 172.21.15.83:0/1212156797 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f800c029070 con 0x5590a73aea80 2024-04-24T09:06:22.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.730+0000 7f7ffea00640 1 -- 172.21.15.83:0/1212156797 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9583+0+0 (secure 0 0 0) 0x7f800c023840 con 0x5590a73aea80 2024-04-24T09:06:22.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.730+0000 7f8022f7f740 1 -- 172.21.15.83:0/1212156797 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f800c004c50 con 0x5590a73aea80 2024-04-24T09:06:22.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.730+0000 7f7ffea00640 1 -- 172.21.15.83:0/1212156797 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f800c035050 con 0x5590a73aea80 2024-04-24T09:06:22.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.730+0000 7f8022f7f740 10 client.20787 fetch_fsmap learned FSMap version 48 2024-04-24T09:06:22.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.730+0000 7f8022f7f740 1 -- 172.21.15.83:0/1212156797 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x5590a701d3f0 con 0x5590a73aea80 2024-04-24T09:06:22.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.731+0000 7f7ffea00640 1 -- 172.21.15.83:0/1212156797 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f800c004c50 con 0x5590a73aea80 2024-04-24T09:06:22.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.731+0000 7f7ffea00640 1 -- 172.21.15.83:0/1212156797 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 48) ==== 8402+0+0 (secure 0 0 0) 0x7f800c019430 con 0x5590a73aea80 2024-04-24T09:06:22.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.731+0000 7f8022f7f740 10 client.20787 fetch_fsmap finished waiting for FSMap version 48 2024-04-24T09:06:22.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.731+0000 7f8022f7f740 10 client.20787 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-24T09:06:22.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.731+0000 7f8022f7f740 20 client.20787 populate_metadata read hostname 'smithi083' 2024-04-24T09:06:22.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.731+0000 7f8022f7f740 1 --2- 172.21.15.83:0/1212156797 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x5590a73c9300 0x5590a73cb6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:22.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.731+0000 7f8022f7f740 4 client.20787 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:06:22.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.731+0000 7f8022f7f740 1 -- 172.21.15.83:0/1212156797 --> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5590a7318bf0 con 0x5590a73c9300 2024-04-24T09:06:22.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.732+0000 7f801cc00640 1 --2- 172.21.15.83:0/1212156797 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x5590a73c9300 0x5590a73cb6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:22.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.733+0000 7f801cc00640 1 --2- 172.21.15.83:0/1212156797 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x5590a73c9300 0x5590a73cb6f0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c5f119fb980090ba server_cookie=2b971b946ccba670 in_seq=0 out_seq=0 2024-04-24T09:06:22.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.733+0000 7f7ffea00640 10 client.20787 ms_handle_connect on v2:172.21.15.189:6838/2548965336 2024-04-24T09:06:22.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.736+0000 7f7ffea00640 1 -- 172.21.15.83:0/1212156797 <== mds.4 v2:172.21.15.189:6838/2548965336 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5590a7318bf0 con 0x5590a73c9300 2024-04-24T09:06:22.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.736+0000 7f7ffea00640 10 client.20787 handle_command_reply: tid=0 2024-04-24T09:06:22.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.736+0000 7f8022f7f740 1 client.20787 shutdown 2024-04-24T09:06:22.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.737+0000 7f8022f7f740 1 -- 172.21.15.83:0/1212156797 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5590a73aea80 msgr2=0x5590a73b5ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:22.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.737+0000 7f8022f7f740 1 --2- 172.21.15.83:0/1212156797 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5590a73aea80 0x5590a73b5ee0 secure :-1 s=READY pgs=2171 cs=0 l=1 rev1=1 crypto rx=0x7f800c01b040 tx=0x7f800c005230 comp rx=0 tx=0).stop 2024-04-24T09:06:22.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.737+0000 7f8022f7f740 1 -- 172.21.15.83:0/1212156797 shutdown_connections 2024-04-24T09:06:22.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.737+0000 7f8022f7f740 1 --2- 172.21.15.83:0/1212156797 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x5590a73c9300 0x5590a73cb6f0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:22.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.737+0000 7f8022f7f740 1 --2- 172.21.15.83:0/1212156797 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590a73b9150 0x5590a73b65d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:22.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.737+0000 7f8022f7f740 1 --2- 172.21.15.83:0/1212156797 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5590a73aea80 0x5590a73b5ee0 unknown :-1 s=CLOSED pgs=2171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:22.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.737+0000 7f8022f7f740 1 --2- 172.21.15.83:0/1212156797 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5590a73ae160 0x5590a73b8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:22.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.737+0000 7f7ffea00640 0 client.20787 ms_handle_reset on v2:172.21.15.189:6838/2548965336 2024-04-24T09:06:22.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.737+0000 7f8022f7f740 1 -- 172.21.15.83:0/1212156797 >> 172.21.15.83:0/1212156797 conn(0x5590a73aa480 msgr2=0x5590a73b0280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:22.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.737+0000 7f8022f7f740 1 -- 172.21.15.83:0/1212156797 shutdown_connections 2024-04-24T09:06:22.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.737+0000 7f8022f7f740 1 -- 172.21.15.83:0/1212156797 wait complete. 2024-04-24T09:06:22.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.737+0000 7f8022f7f740 20 client.20787 trim_cache size 0 max 16384 2024-04-24T09:06:22.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.760+0000 7f8022f7f740 1 Processor -- start 2024-04-24T09:06:22.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.760+0000 7f8022f7f740 1 -- start start 2024-04-24T09:06:22.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.761+0000 7f8022f7f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5590a73ae160 0x5590a73b0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:22.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.761+0000 7f8022f7f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590a73aea80 0x5590a73b8720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:22.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.761+0000 7f8022f7f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5590a73b9150 0x5590a73d0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:22.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.761+0000 7f8022f7f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f80180713e0 con 0x5590a73b9150 2024-04-24T09:06:22.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.761+0000 7f8022f7f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f80180730a0 con 0x5590a73aea80 2024-04-24T09:06:22.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.761+0000 7f8022f7f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8018071ab0 con 0x5590a73ae160 2024-04-24T09:06:22.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.761+0000 7f801d600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590a73aea80 0x5590a73b8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:22.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.761+0000 7f801cc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5590a73b9150 0x5590a73d0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:22.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.761+0000 7f801d600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590a73aea80 0x5590a73b8720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:38806/0 (socket says 172.21.15.83:38806) 2024-04-24T09:06:22.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.761+0000 7f801d600640 1 -- 172.21.15.83:0/3467010467 learned_addr learned my addr 172.21.15.83:0/3467010467 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:06:22.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.761+0000 7f801cc00640 1 --2- 172.21.15.83:0/3467010467 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5590a73ae160 0x5590a73b0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:22.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.762+0000 7f801cc00640 1 -- 172.21.15.83:0/3467010467 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5590a73ae160 msgr2=0x5590a73b0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:22.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.762+0000 7f801cc00640 1 --2- 172.21.15.83:0/3467010467 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5590a73ae160 0x5590a73b0900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:22.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.762+0000 7f801cc00640 1 -- 172.21.15.83:0/3467010467 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590a73aea80 msgr2=0x5590a73b8720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:22.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.762+0000 7f801cc00640 1 --2- 172.21.15.83:0/3467010467 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590a73aea80 0x5590a73b8720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:22.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.762+0000 7f801cc00640 1 -- 172.21.15.83:0/3467010467 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590a70e77d0 con 0x5590a73b9150 2024-04-24T09:06:22.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.762+0000 7f801cc00640 1 --2- 172.21.15.83:0/3467010467 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5590a73b9150 0x5590a73d0ec0 secure :-1 s=READY pgs=2172 cs=0 l=1 rev1=1 crypto rx=0x5590a73cbe60 tx=0x7f8004004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89db14b0f4ece71d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:22.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.762+0000 7f8009c00640 1 -- 172.21.15.83:0/3467010467 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8004023990 con 0x5590a73b9150 2024-04-24T09:06:22.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.763+0000 7f8009c00640 1 -- 172.21.15.83:0/3467010467 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8004005020 con 0x5590a73b9150 2024-04-24T09:06:22.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.763+0000 7f8009c00640 1 -- 172.21.15.83:0/3467010467 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8004004c30 con 0x5590a73b9150 2024-04-24T09:06:22.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.763+0000 7f8022f7f740 1 -- 172.21.15.83:0/3467010467 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5590a73b9150 msgr2=0x5590a73d0ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:22.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.763+0000 7f8022f7f740 1 --2- 172.21.15.83:0/3467010467 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5590a73b9150 0x5590a73d0ec0 secure :-1 s=READY pgs=2172 cs=0 l=1 rev1=1 crypto rx=0x5590a73cbe60 tx=0x7f8004004730 comp rx=0 tx=0).stop 2024-04-24T09:06:22.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.764+0000 7f8022f7f740 1 -- 172.21.15.83:0/3467010467 shutdown_connections 2024-04-24T09:06:22.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.764+0000 7f8022f7f740 1 --2- 172.21.15.83:0/3467010467 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5590a73b9150 0x5590a73d0ec0 secure :-1 s=CLOSED pgs=2172 cs=0 l=1 rev1=1 crypto rx=0x5590a73cbe60 tx=0x7f8004004730 comp rx=0 tx=0).stop 2024-04-24T09:06:22.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.764+0000 7f8022f7f740 1 --2- 172.21.15.83:0/3467010467 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590a73aea80 0x5590a73b8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:22.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.764+0000 7f8022f7f740 1 --2- 172.21.15.83:0/3467010467 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5590a73ae160 0x5590a73b0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:22.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.764+0000 7f8022f7f740 1 -- 172.21.15.83:0/3467010467 >> 172.21.15.83:0/3467010467 conn(0x5590a73aa480 msgr2=0x5590a73b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:22.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.764+0000 7f8022f7f740 1 -- 172.21.15.83:0/3467010467 shutdown_connections 2024-04-24T09:06:22.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.764+0000 7f8022f7f740 1 -- 172.21.15.83:0/3467010467 wait complete. 2024-04-24T09:06:22.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.764+0000 7f8022f7f740 1 -- start start 2024-04-24T09:06:22.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.765+0000 7f8022f7f740 1 Processor -- start 2024-04-24T09:06:22.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.765+0000 7f8022f7f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590a73ae160 0x5590a73b2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:22.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.765+0000 7f8022f7f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5590a73aea80 0x5590a73b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:22.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.765+0000 7f8022f7f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5590a73b3ab0 0x5590a73bf8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:22.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.765+0000 7f8022f7f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8018072910 con 0x5590a73b3ab0 2024-04-24T09:06:22.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.765+0000 7f8022f7f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f80180721e0 con 0x5590a73ae160 2024-04-24T09:06:22.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.765+0000 7f8022f7f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f801806f3e0 con 0x5590a73aea80 2024-04-24T09:06:22.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.765+0000 7f8022f7f740 10 client.0 _ll_register_callbacks cb 0x7f8004009990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:06:22.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.766+0000 7f801cc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5590a73b3ab0 0x5590a73bf8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:22.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.766+0000 7f801cc00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590a73ae160 0x5590a73b2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:22.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.766+0000 7f801d600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5590a73aea80 0x5590a73b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:22.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.766+0000 7f801cc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5590a73b3ab0 0x5590a73bf8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55788/0 (socket says 172.21.15.83:55788) 2024-04-24T09:06:22.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.766+0000 7f801cc00640 1 -- 172.21.15.83:0/3563596801 learned_addr learned my addr 172.21.15.83:0/3563596801 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:06:22.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.766+0000 7f801cc00640 1 -- 172.21.15.83:0/3563596801 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5590a73aea80 msgr2=0x5590a73b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:22.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.766+0000 7f801cc00640 1 --2- 172.21.15.83:0/3563596801 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5590a73aea80 0x5590a73b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:22.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.766+0000 7f801cc00640 1 -- 172.21.15.83:0/3563596801 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5590a73b3ab0 msgr2=0x5590a73bf8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:22.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.766+0000 7f801cc00640 1 --2- 172.21.15.83:0/3563596801 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5590a73b3ab0 0x5590a73bf8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:22.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.766+0000 7f801cc00640 1 -- 172.21.15.83:0/3563596801 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590a73c00c0 con 0x5590a73ae160 2024-04-24T09:06:22.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.767+0000 7f801cc00640 1 --2- 172.21.15.83:0/3563596801 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590a73ae160 0x5590a73b2ed0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x5590a73071c0 tx=0x7f8004009a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=351e2975c9f017e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:22.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.767+0000 7f7ffea00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:06:22.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.767+0000 7f7ffea00640 1 -- 172.21.15.83:0/3563596801 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8004023310 con 0x5590a73ae160 2024-04-24T09:06:22.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.767+0000 7f7ffea00640 1 -- 172.21.15.83:0/3563596801 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9583+0+0 (secure 0 0 0) 0x7f80040051d0 con 0x5590a73ae160 2024-04-24T09:06:22.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.767+0000 7f8022f7f740 1 -- 172.21.15.83:0/3563596801 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8004005020 con 0x5590a73ae160 2024-04-24T09:06:22.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.768+0000 7f7ffea00640 1 -- 172.21.15.83:0/3563596801 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f80040053d0 con 0x5590a73ae160 2024-04-24T09:06:22.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.768+0000 7f8022f7f740 10 client.37934 fetch_fsmap learned FSMap version 48 2024-04-24T09:06:22.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.768+0000 7f8022f7f740 1 -- 172.21.15.83:0/3563596801 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x5590a74301d0 con 0x5590a73ae160 2024-04-24T09:06:22.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.768+0000 7f7ffea00640 1 -- 172.21.15.83:0/3563596801 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8004005020 con 0x5590a73ae160 2024-04-24T09:06:22.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.769+0000 7f7ffea00640 1 -- 172.21.15.83:0/3563596801 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 48) ==== 8402+0+0 (secure 0 0 0) 0x7f800403e6b0 con 0x5590a73ae160 2024-04-24T09:06:22.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.769+0000 7f8022f7f740 10 client.37934 fetch_fsmap finished waiting for FSMap version 48 2024-04-24T09:06:22.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.769+0000 7f8022f7f740 10 client.37934 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-24T09:06:22.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.769+0000 7f8022f7f740 20 client.37934 populate_metadata read hostname 'smithi083' 2024-04-24T09:06:22.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.769+0000 7f8022f7f740 1 --2- 172.21.15.83:0/3563596801 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x5590a74307c0 0x5590a7432be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:22.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.769+0000 7f8022f7f740 4 client.37934 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:06:22.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.769+0000 7f8022f7f740 1 -- 172.21.15.83:0/3563596801 --> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5590a7312a60 con 0x5590a74307c0 2024-04-24T09:06:22.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.770+0000 7f801d600640 1 --2- 172.21.15.83:0/3563596801 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x5590a74307c0 0x5590a7432be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:22.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.771+0000 7f801d600640 1 --2- 172.21.15.83:0/3563596801 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x5590a74307c0 0x5590a7432be0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b10cf5514f4a60dc server_cookie=e26d354955d97c7e in_seq=0 out_seq=0 2024-04-24T09:06:22.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.771+0000 7f7ffea00640 10 client.37934 ms_handle_connect on v2:172.21.15.189:6838/2548965336 2024-04-24T09:06:22.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.771+0000 7f7ffea00640 1 -- 172.21.15.83:0/3563596801 <== mds.4 v2:172.21.15.189:6838/2548965336 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5590a7312a60 con 0x5590a74307c0 2024-04-24T09:06:22.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.771+0000 7f7ffea00640 10 client.37934 handle_command_reply: tid=0 2024-04-24T09:06:22.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.772+0000 7f8022f7f740 1 client.37934 shutdown 2024-04-24T09:06:22.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.772+0000 7f8022f7f740 1 -- 172.21.15.83:0/3563596801 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590a73ae160 msgr2=0x5590a73b2ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:22.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.772+0000 7f8022f7f740 1 --2- 172.21.15.83:0/3563596801 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590a73ae160 0x5590a73b2ed0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x5590a73071c0 tx=0x7f8004009a40 comp rx=0 tx=0).stop 2024-04-24T09:06:22.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.772+0000 7f8022f7f740 1 -- 172.21.15.83:0/3563596801 shutdown_connections 2024-04-24T09:06:22.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.772+0000 7f8022f7f740 1 --2- 172.21.15.83:0/3563596801 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x5590a74307c0 0x5590a7432be0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:22.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.772+0000 7f8022f7f740 1 --2- 172.21.15.83:0/3563596801 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5590a73b3ab0 0x5590a73bf8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:22.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.772+0000 7f8022f7f740 1 --2- 172.21.15.83:0/3563596801 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5590a73aea80 0x5590a73b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:22.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.772+0000 7f7ffea00640 0 client.37934 ms_handle_reset on v2:172.21.15.189:6838/2548965336 2024-04-24T09:06:22.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.772+0000 7f8022f7f740 1 --2- 172.21.15.83:0/3563596801 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590a73ae160 0x5590a73b2ed0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:22.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.772+0000 7f8022f7f740 1 -- 172.21.15.83:0/3563596801 >> 172.21.15.83:0/3563596801 conn(0x5590a73aa480 msgr2=0x5590a73a0560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:22.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.773+0000 7f8022f7f740 1 -- 172.21.15.83:0/3563596801 shutdown_connections 2024-04-24T09:06:22.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.773+0000 7f8022f7f740 1 -- 172.21.15.83:0/3563596801 wait complete. 2024-04-24T09:06:22.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.773+0000 7f8022f7f740 20 client.37934 trim_cache size 0 max 16384 2024-04-24T09:06:22.774 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:06:22.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.774+0000 7f8020000640 1 -- 172.21.15.83:0/675389110 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f80000a3da0 msgr2=0x7f80000a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:22.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.774+0000 7f8020000640 1 --2- 172.21.15.83:0/675389110 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f80000a3da0 0x7f80000a6260 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f80100c6180 tx=0x7f8010005ef0 comp rx=0 tx=0).stop 2024-04-24T09:06:22.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.774+0000 7f8020000640 1 -- 172.21.15.83:0/675389110 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f801819a920 msgr2=0x7f801819add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:22.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.774+0000 7f8020000640 1 --2- 172.21.15.83:0/675389110 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f801819a920 0x7f801819add0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f8010031040 tx=0x7f8010032f80 comp rx=0 tx=0).stop 2024-04-24T09:06:22.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.774+0000 7f8020000640 1 -- 172.21.15.83:0/675389110 shutdown_connections 2024-04-24T09:06:22.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.774+0000 7f8020000640 1 --2- 172.21.15.83:0/675389110 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f80000a3da0 0x7f80000a6260 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:22.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.774+0000 7f8020000640 1 --2- 172.21.15.83:0/675389110 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f801819a920 0x7f801819add0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:22.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.774+0000 7f8020000640 1 --2- 172.21.15.83:0/675389110 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8018109580 0x7f80181a0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:22.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.774+0000 7f8020000640 1 --2- 172.21.15.83:0/675389110 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f80181044b0 0x7f80181a0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:22.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.774+0000 7f8020000640 1 -- 172.21.15.83:0/675389110 >> 172.21.15.83:0/675389110 conn(0x7f80180ffd90 msgr2=0x7f8018101160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:22.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.774+0000 7f8020000640 1 -- 172.21.15.83:0/675389110 shutdown_connections 2024-04-24T09:06:22.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:22.774+0000 7f8020000640 1 -- 172.21.15.83:0/675389110 wait complete. 2024-04-24T09:06:22.796 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-24T09:06:23.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:06:23 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/4116579888' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:06:23.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.058+0000 7fec0a200640 1 -- 172.21.15.83:0/867955753 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec0411b7e0 msgr2=0x7fec0411dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:23.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.058+0000 7fec0a200640 1 --2- 172.21.15.83:0/867955753 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec0411b7e0 0x7fec0411dbd0 secure :-1 s=READY pgs=2173 cs=0 l=1 rev1=1 crypto rx=0x7febfc009cd0 tx=0x7febfc02a8f0 comp rx=0 tx=0).stop 2024-04-24T09:06:23.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.058+0000 7fec0a200640 1 -- 172.21.15.83:0/867955753 shutdown_connections 2024-04-24T09:06:23.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.058+0000 7fec0a200640 1 --2- 172.21.15.83:0/867955753 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec0411b7e0 0x7fec0411dbd0 unknown :-1 s=CLOSED pgs=2173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:23.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.058+0000 7fec0a200640 1 --2- 172.21.15.83:0/867955753 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec0410e880 0x7fec0410ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:23.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.058+0000 7fec0a200640 1 --2- 172.21.15.83:0/867955753 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fec04110850 0x7fec0410e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:23.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.058+0000 7fec0a200640 1 -- 172.21.15.83:0/867955753 >> 172.21.15.83:0/867955753 conn(0x7fec0406d110 msgr2=0x7fec0406d520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:23.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.058+0000 7fec0a200640 1 -- 172.21.15.83:0/867955753 shutdown_connections 2024-04-24T09:06:23.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.059+0000 7fec0a200640 1 -- 172.21.15.83:0/867955753 wait complete. 2024-04-24T09:06:23.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.060+0000 7fec0a200640 1 Processor -- start 2024-04-24T09:06:23.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.060+0000 7fec0a200640 1 -- start start 2024-04-24T09:06:23.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.061+0000 7fec0a200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec0410e880 0x7fec041a75b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:23.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.061+0000 7fec0a200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fec04110850 0x7fec041a7af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:23.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.061+0000 7fec0a200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec0411b7e0 0x7fec041abe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:23.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.061+0000 7fec0a200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fec04120ff0 con 0x7fec0410e880 2024-04-24T09:06:23.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.061+0000 7fec0a200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fec04120e70 con 0x7fec0411b7e0 2024-04-24T09:06:23.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.061+0000 7fec0a200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fec04121170 con 0x7fec04110850 2024-04-24T09:06:23.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.062+0000 7fec03400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec0410e880 0x7fec041a75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:23.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.062+0000 7fec03400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec0410e880 0x7fec041a75b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55808/0 (socket says 172.21.15.83:55808) 2024-04-24T09:06:23.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.062+0000 7fec03400640 1 -- 172.21.15.83:0/4000688479 learned_addr learned my addr 172.21.15.83:0/4000688479 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:06:23.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.062+0000 7fec03e00640 1 --2- 172.21.15.83:0/4000688479 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec0411b7e0 0x7fec041abe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:23.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.062+0000 7fec02a00640 1 --2- 172.21.15.83:0/4000688479 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fec04110850 0x7fec041a7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:23.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.062+0000 7fec03e00640 1 -- 172.21.15.83:0/4000688479 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fec04110850 msgr2=0x7fec041a7af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:23.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.062+0000 7fec03e00640 1 --2- 172.21.15.83:0/4000688479 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fec04110850 0x7fec041a7af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:23.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.062+0000 7fec03e00640 1 -- 172.21.15.83:0/4000688479 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec0410e880 msgr2=0x7fec041a75b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:23.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.062+0000 7fec03e00640 1 --2- 172.21.15.83:0/4000688479 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec0410e880 0x7fec041a75b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:23.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.062+0000 7fec03e00640 1 -- 172.21.15.83:0/4000688479 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec041ac5a0 con 0x7fec0411b7e0 2024-04-24T09:06:23.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.062+0000 7fec03400640 1 --2- 172.21.15.83:0/4000688479 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec0410e880 0x7fec041a75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:06:23.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.063+0000 7fec03e00640 1 --2- 172.21.15.83:0/4000688479 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec0411b7e0 0x7fec041abe70 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7febfc009bf0 tx=0x7febfc003ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:23.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.065+0000 7febfbe00640 1 -- 172.21.15.83:0/4000688479 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febfc004240 con 0x7fec0411b7e0 2024-04-24T09:06:23.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.065+0000 7febfbe00640 1 -- 172.21.15.83:0/4000688479 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7febfc040070 con 0x7fec0411b7e0 2024-04-24T09:06:23.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.065+0000 7febfbe00640 1 -- 172.21.15.83:0/4000688479 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febfc045b50 con 0x7fec0411b7e0 2024-04-24T09:06:23.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.065+0000 7fec0a200640 1 -- 172.21.15.83:0/4000688479 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec041ac830 con 0x7fec0411b7e0 2024-04-24T09:06:23.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.066+0000 7fec0a200640 1 -- 172.21.15.83:0/4000688479 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec041b4120 con 0x7fec0411b7e0 2024-04-24T09:06:23.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.073+0000 7febfbe00640 1 -- 172.21.15.83:0/4000688479 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7febfc045cf0 con 0x7fec0411b7e0 2024-04-24T09:06:23.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.073+0000 7fec0d024740 1 Processor -- start 2024-04-24T09:06:23.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.074+0000 7fec0d024740 1 -- start start 2024-04-24T09:06:23.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.074+0000 7fec0d024740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ed65a34950 0x55ed65a34d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:23.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.074+0000 7fec0d024740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ed65a35270 0x55ed65a3f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:23.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.074+0000 7fec0d024740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ed65a3f940 0x55ed65a41d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:23.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.074+0000 7fec0d024740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ed6599a1d0 con 0x55ed65a3f940 2024-04-24T09:06:23.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.074+0000 7fec0d024740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ed65940450 con 0x55ed65a35270 2024-04-24T09:06:23.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.074+0000 7fec0d024740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55ed65996190 con 0x55ed65a34950 2024-04-24T09:06:23.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.075+0000 7fec02a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ed65a3f940 0x55ed65a41d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:23.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.075+0000 7fec02a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ed65a3f940 0x55ed65a41d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55812/0 (socket says 172.21.15.83:55812) 2024-04-24T09:06:23.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.075+0000 7fec02a00640 1 -- 172.21.15.83:0/3743865913 learned_addr learned my addr 172.21.15.83:0/3743865913 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:06:23.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.075+0000 7fec03400640 1 --2- 172.21.15.83:0/3743865913 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ed65a35270 0x55ed65a3f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:23.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.076+0000 7fec02a00640 1 -- 172.21.15.83:0/3743865913 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ed65a34950 msgr2=0x55ed65a34d30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-24T09:06:23.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.076+0000 7fec02a00640 1 --2- 172.21.15.83:0/3743865913 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ed65a34950 0x55ed65a34d30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:23.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.076+0000 7fec02a00640 1 -- 172.21.15.83:0/3743865913 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ed65a35270 msgr2=0x55ed65a3f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:23.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.076+0000 7fec02a00640 1 --2- 172.21.15.83:0/3743865913 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ed65a35270 0x55ed65a3f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:23.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.076+0000 7fec02a00640 1 -- 172.21.15.83:0/3743865913 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed659fd310 con 0x55ed65a3f940 2024-04-24T09:06:23.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.076+0000 7fec02a00640 1 --2- 172.21.15.83:0/3743865913 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ed65a3f940 0x55ed65a41d30 secure :-1 s=READY pgs=2174 cs=0 l=1 rev1=1 crypto rx=0x7febf000bcd0 tx=0x7febf00107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=718786f82b3fa862 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:23.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.076+0000 7febefe00640 1 -- 172.21.15.83:0/3743865913 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf0004030 con 0x55ed65a3f940 2024-04-24T09:06:23.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.077+0000 7febefe00640 1 -- 172.21.15.83:0/3743865913 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7febf00041d0 con 0x55ed65a3f940 2024-04-24T09:06:23.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.077+0000 7febefe00640 1 -- 172.21.15.83:0/3743865913 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf00117a0 con 0x55ed65a3f940 2024-04-24T09:06:23.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.078+0000 7febfbe00640 1 --2- 172.21.15.83:0/4000688479 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7febe80a3e60 0x7febe80a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:23.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.078+0000 7fec0d024740 1 -- 172.21.15.83:0/3743865913 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ed65a3f940 msgr2=0x55ed65a41d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:23.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.078+0000 7fec0d024740 1 --2- 172.21.15.83:0/3743865913 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ed65a3f940 0x55ed65a41d30 secure :-1 s=READY pgs=2174 cs=0 l=1 rev1=1 crypto rx=0x7febf000bcd0 tx=0x7febf00107a0 comp rx=0 tx=0).stop 2024-04-24T09:06:23.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.079+0000 7febfbe00640 1 -- 172.21.15.83:0/4000688479 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9583+0+0 (secure 0 0 0) 0x7febfc0f1c00 con 0x7fec0411b7e0 2024-04-24T09:06:23.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.079+0000 7fec0d024740 1 -- 172.21.15.83:0/3743865913 shutdown_connections 2024-04-24T09:06:23.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.079+0000 7fec0d024740 1 --2- 172.21.15.83:0/3743865913 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ed65a3f940 0x55ed65a41d30 unknown :-1 s=CLOSED pgs=2174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:23.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.079+0000 7fec0d024740 1 --2- 172.21.15.83:0/3743865913 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ed65a35270 0x55ed65a3f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:23.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.079+0000 7fec0d024740 1 --2- 172.21.15.83:0/3743865913 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ed65a34950 0x55ed65a34d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:23.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.079+0000 7fec0d024740 1 -- 172.21.15.83:0/3743865913 >> 172.21.15.83:0/3743865913 conn(0x55ed65a30c70 msgr2=0x55ed65a31050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:23.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.079+0000 7fec03400640 1 --2- 172.21.15.83:0/4000688479 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7febe80a3e60 0x7febe80a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:23.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.079+0000 7fec0d024740 1 -- 172.21.15.83:0/3743865913 shutdown_connections 2024-04-24T09:06:23.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.081+0000 7fec03400640 1 --2- 172.21.15.83:0/4000688479 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7febe80a3e60 0x7febe80a6320 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x55ed65a35f10 tx=0x7febf400b480 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:23.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.081+0000 7fec0d024740 1 -- 172.21.15.83:0/3743865913 wait complete. 2024-04-24T09:06:23.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.081+0000 7fec0d024740 1 -- start start 2024-04-24T09:06:23.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.082+0000 7fec0d024740 1 Processor -- start 2024-04-24T09:06:23.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.082+0000 7fec0d024740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ed65a34950 0x55ed65a3f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:23.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.083+0000 7fec0d024740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ed65a35270 0x55ed65a3c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:23.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.083+0000 7fec0d024740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ed65a3f940 0x55ed65a3cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:23.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.083+0000 7fec0d024740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ed65a427d0 con 0x55ed65a34950 2024-04-24T09:06:23.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.083+0000 7fec0d024740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ed65953810 con 0x55ed65a3f940 2024-04-24T09:06:23.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.083+0000 7fec0d024740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55ed65a42950 con 0x55ed65a35270 2024-04-24T09:06:23.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.083+0000 7fec0d024740 10 client.0 _ll_register_callbacks cb 0x55ed65921140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:06:23.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.083+0000 7fec03400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ed65a34950 0x55ed65a3f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:23.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.083+0000 7fec03e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ed65a3f940 0x55ed65a3cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:23.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.083+0000 7fec03400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ed65a34950 0x55ed65a3f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55824/0 (socket says 172.21.15.83:55824) 2024-04-24T09:06:23.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.083+0000 7fec03400640 1 -- 172.21.15.83:0/1825702702 learned_addr learned my addr 172.21.15.83:0/1825702702 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:06:23.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.083+0000 7fec02a00640 1 --2- 172.21.15.83:0/1825702702 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ed65a35270 0x55ed65a3c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:23.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.084+0000 7fec03e00640 1 -- 172.21.15.83:0/1825702702 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ed65a35270 msgr2=0x55ed65a3c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:23.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.084+0000 7fec03e00640 1 --2- 172.21.15.83:0/1825702702 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ed65a35270 0x55ed65a3c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:23.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.084+0000 7fec03e00640 1 -- 172.21.15.83:0/1825702702 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ed65a34950 msgr2=0x55ed65a3f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:23.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.084+0000 7fec03e00640 1 --2- 172.21.15.83:0/1825702702 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ed65a34950 0x55ed65a3f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:23.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.084+0000 7fec03e00640 1 -- 172.21.15.83:0/1825702702 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed659643f0 con 0x55ed65a3f940 2024-04-24T09:06:23.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.084+0000 7fec03400640 1 --2- 172.21.15.83:0/1825702702 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ed65a34950 0x55ed65a3f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:06:23.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.084+0000 7fec03e00640 1 --2- 172.21.15.83:0/1825702702 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ed65a3f940 0x55ed65a3cdc0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7febfc0eefe0 tx=0x7febfc0bd190 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bbc0256798b1d5d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:23.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.084+0000 7febecc00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:06:23.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.085+0000 7febecc00640 1 -- 172.21.15.83:0/1825702702 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febfc0c1be0 con 0x55ed65a3f940 2024-04-24T09:06:23.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.085+0000 7febecc00640 1 -- 172.21.15.83:0/1825702702 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9583+0+0 (secure 0 0 0) 0x7febfc0ef150 con 0x55ed65a3f940 2024-04-24T09:06:23.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.085+0000 7fec0d024740 1 -- 172.21.15.83:0/1825702702 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7febf00041d0 con 0x55ed65a3f940 2024-04-24T09:06:23.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.085+0000 7febecc00640 1 -- 172.21.15.83:0/1825702702 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7febfc0db050 con 0x55ed65a3f940 2024-04-24T09:06:23.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.085+0000 7fec0d024740 10 client.37946 fetch_fsmap learned FSMap version 48 2024-04-24T09:06:23.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.085+0000 7fec0d024740 1 -- 172.21.15.83:0/1825702702 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55ed656ac3f0 con 0x55ed65a3f940 2024-04-24T09:06:23.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.086+0000 7febecc00640 1 -- 172.21.15.83:0/1825702702 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7febf00041d0 con 0x55ed65a3f940 2024-04-24T09:06:23.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.087+0000 7febecc00640 1 -- 172.21.15.83:0/1825702702 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 48) ==== 8402+0+0 (secure 0 0 0) 0x7febfc0c6010 con 0x55ed65a3f940 2024-04-24T09:06:23.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.087+0000 7fec0d024740 10 client.37946 fetch_fsmap finished waiting for FSMap version 48 2024-04-24T09:06:23.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.087+0000 7fec0d024740 10 client.37946 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-24T09:06:23.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.087+0000 7fec0d024740 20 client.37946 populate_metadata read hostname 'smithi083' 2024-04-24T09:06:23.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.087+0000 7fec0d024740 1 --2- 172.21.15.83:0/1825702702 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55ed65a4fb80 0x55ed65a51fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:23.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.087+0000 7fec0d024740 4 client.37946 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:06:23.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.087+0000 7fec0d024740 1 -- 172.21.15.83:0/1825702702 --> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ed659a7bf0 con 0x55ed65a4fb80 2024-04-24T09:06:23.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.088+0000 7fec03400640 1 --2- 172.21.15.83:0/1825702702 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55ed65a4fb80 0x55ed65a51fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:23.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.089+0000 7febecc00640 10 client.37946 ms_handle_connect on v2:172.21.15.138:6832/1157967658 2024-04-24T09:06:23.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.089+0000 7fec03400640 1 --2- 172.21.15.83:0/1825702702 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55ed65a4fb80 0x55ed65a51fd0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=dfd9439f626ba0b5 server_cookie=a75fc629e4197f04 in_seq=0 out_seq=0 2024-04-24T09:06:23.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.092+0000 7febecc00640 1 -- 172.21.15.83:0/1825702702 <== mds.3 v2:172.21.15.138:6832/1157967658 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ed659a7bf0 con 0x55ed65a4fb80 2024-04-24T09:06:23.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.092+0000 7febecc00640 10 client.37946 handle_command_reply: tid=0 2024-04-24T09:06:23.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.093+0000 7fec0d024740 1 client.37946 shutdown 2024-04-24T09:06:23.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.093+0000 7fec0d024740 1 -- 172.21.15.83:0/1825702702 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ed65a3f940 msgr2=0x55ed65a3cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:23.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.093+0000 7fec0d024740 1 --2- 172.21.15.83:0/1825702702 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ed65a3f940 0x55ed65a3cdc0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7febfc0eefe0 tx=0x7febfc0bd190 comp rx=0 tx=0).stop 2024-04-24T09:06:23.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.093+0000 7fec0d024740 1 -- 172.21.15.83:0/1825702702 shutdown_connections 2024-04-24T09:06:23.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.093+0000 7fec0d024740 1 --2- 172.21.15.83:0/1825702702 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ed65a3f940 0x55ed65a3cdc0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:23.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.093+0000 7fec0d024740 1 --2- 172.21.15.83:0/1825702702 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55ed65a4fb80 0x55ed65a51fd0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:23.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.093+0000 7fec0d024740 1 --2- 172.21.15.83:0/1825702702 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ed65a35270 0x55ed65a3c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:23.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.093+0000 7fec0d024740 1 --2- 172.21.15.83:0/1825702702 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ed65a34950 0x55ed65a3f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:23.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.093+0000 7fec0d024740 1 -- 172.21.15.83:0/1825702702 >> 172.21.15.83:0/1825702702 conn(0x55ed65a30c70 msgr2=0x55ed65a36a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:23.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.093+0000 7febecc00640 0 client.37946 ms_handle_reset on v2:172.21.15.138:6832/1157967658 2024-04-24T09:06:23.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.094+0000 7fec0d024740 1 -- 172.21.15.83:0/1825702702 shutdown_connections 2024-04-24T09:06:23.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.094+0000 7fec0d024740 1 -- 172.21.15.83:0/1825702702 wait complete. 2024-04-24T09:06:23.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.094+0000 7fec0d024740 20 client.37946 trim_cache size 0 max 16384 2024-04-24T09:06:23.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.113+0000 7fec0d024740 1 Processor -- start 2024-04-24T09:06:23.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.113+0000 7fec0d024740 1 -- start start 2024-04-24T09:06:23.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.113+0000 7fec0d024740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ed65a34950 0x55ed65a370b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:23.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.113+0000 7fec0d024740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ed65a35270 0x55ed65a3eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:23.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.113+0000 7fec0d024740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ed65a3f940 0x55ed65a3f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:23.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.113+0000 7fec0d024740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fec041be3c0 con 0x55ed65a34950 2024-04-24T09:06:23.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.113+0000 7fec0d024740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fec041c0080 con 0x55ed65a3f940 2024-04-24T09:06:23.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.113+0000 7fec0d024740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fec041bea90 con 0x55ed65a35270 2024-04-24T09:06:23.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.114+0000 7fec02a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ed65a35270 0x55ed65a3eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:23.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.114+0000 7fec03400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ed65a34950 0x55ed65a370b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:23.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.114+0000 7fec03400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ed65a34950 0x55ed65a370b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55836/0 (socket says 172.21.15.83:55836) 2024-04-24T09:06:23.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.114+0000 7fec03400640 1 -- 172.21.15.83:0/3337583759 learned_addr learned my addr 172.21.15.83:0/3337583759 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:06:23.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.114+0000 7fec03e00640 1 --2- 172.21.15.83:0/3337583759 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ed65a3f940 0x55ed65a3f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:23.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.114+0000 7fec03e00640 1 -- 172.21.15.83:0/3337583759 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ed65a35270 msgr2=0x55ed65a3eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:23.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.114+0000 7fec03e00640 1 --2- 172.21.15.83:0/3337583759 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ed65a35270 0x55ed65a3eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:23.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.114+0000 7fec03e00640 1 -- 172.21.15.83:0/3337583759 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ed65a34950 msgr2=0x55ed65a370b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:23.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.114+0000 7fec03e00640 1 --2- 172.21.15.83:0/3337583759 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ed65a34950 0x55ed65a370b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:23.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.114+0000 7fec03e00640 1 -- 172.21.15.83:0/3337583759 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed65a350a0 con 0x55ed65a3f940 2024-04-24T09:06:23.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.114+0000 7fec03400640 1 --2- 172.21.15.83:0/3337583759 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ed65a34950 0x55ed65a370b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:06:23.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.114+0000 7fec02a00640 1 --2- 172.21.15.83:0/3337583759 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ed65a35270 0x55ed65a3eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:06:23.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.115+0000 7fec03e00640 1 --2- 172.21.15.83:0/3337583759 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ed65a3f940 0x55ed65a3f400 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7febfc0b9c90 tx=0x7febfc0b9410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e79f4a72f7281df7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:23.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.115+0000 7febef400640 1 -- 172.21.15.83:0/3337583759 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febfc0b9ad0 con 0x55ed65a3f940 2024-04-24T09:06:23.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.115+0000 7febef400640 1 -- 172.21.15.83:0/3337583759 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7febfc036070 con 0x55ed65a3f940 2024-04-24T09:06:23.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.115+0000 7febef400640 1 -- 172.21.15.83:0/3337583759 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febfc0bedc0 con 0x55ed65a3f940 2024-04-24T09:06:23.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.116+0000 7fec0d024740 1 -- 172.21.15.83:0/3337583759 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ed65a3f940 msgr2=0x55ed65a3f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:23.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.116+0000 7fec0d024740 1 --2- 172.21.15.83:0/3337583759 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ed65a3f940 0x55ed65a3f400 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7febfc0b9c90 tx=0x7febfc0b9410 comp rx=0 tx=0).stop 2024-04-24T09:06:23.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.116+0000 7fec0d024740 1 -- 172.21.15.83:0/3337583759 shutdown_connections 2024-04-24T09:06:23.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.116+0000 7fec0d024740 1 --2- 172.21.15.83:0/3337583759 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ed65a3f940 0x55ed65a3f400 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:23.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.116+0000 7fec0d024740 1 --2- 172.21.15.83:0/3337583759 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ed65a35270 0x55ed65a3eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:23.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.116+0000 7fec0d024740 1 --2- 172.21.15.83:0/3337583759 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ed65a34950 0x55ed65a370b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:23.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.117+0000 7fec0d024740 1 -- 172.21.15.83:0/3337583759 >> 172.21.15.83:0/3337583759 conn(0x55ed65a30c70 msgr2=0x55ed65a31050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:23.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.117+0000 7fec0d024740 1 -- 172.21.15.83:0/3337583759 shutdown_connections 2024-04-24T09:06:23.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.117+0000 7fec0d024740 1 -- 172.21.15.83:0/3337583759 wait complete. 2024-04-24T09:06:23.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.117+0000 7fec0d024740 1 -- start start 2024-04-24T09:06:23.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.118+0000 7fec0d024740 1 Processor -- start 2024-04-24T09:06:23.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.118+0000 7fec0d024740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ed65a34950 0x55ed65a40d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:23.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.118+0000 7fec0d024740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ed65a35270 0x55ed65a41250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:23.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.118+0000 7fec0d024740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ed65a3f940 0x55ed65a461e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:23.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.118+0000 7fec0d024740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fec041bf8f0 con 0x55ed65a3f940 2024-04-24T09:06:23.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.118+0000 7fec0d024740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fec041bf1c0 con 0x55ed65a34950 2024-04-24T09:06:23.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.118+0000 7fec0d024740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fec041bc3c0 con 0x55ed65a35270 2024-04-24T09:06:23.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.118+0000 7fec0d024740 10 client.0 _ll_register_callbacks cb 0x7febfc003ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:06:23.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.118+0000 7fec03e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ed65a3f940 0x55ed65a461e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:23.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.119+0000 7fec02a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ed65a35270 0x55ed65a41250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:23.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.119+0000 7fec03400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ed65a34950 0x55ed65a40d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:23.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.119+0000 7fec03e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ed65a3f940 0x55ed65a461e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55844/0 (socket says 172.21.15.83:55844) 2024-04-24T09:06:23.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.119+0000 7fec03e00640 1 -- 172.21.15.83:0/115349018 learned_addr learned my addr 172.21.15.83:0/115349018 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:06:23.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.119+0000 7fec03e00640 1 -- 172.21.15.83:0/115349018 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ed65a35270 msgr2=0x55ed65a41250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:23.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.119+0000 7fec03e00640 1 --2- 172.21.15.83:0/115349018 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ed65a35270 0x55ed65a41250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:23.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.119+0000 7fec03e00640 1 -- 172.21.15.83:0/115349018 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ed65a34950 msgr2=0x55ed65a40d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:23.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.119+0000 7fec03e00640 1 --2- 172.21.15.83:0/115349018 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ed65a34950 0x55ed65a40d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:23.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.119+0000 7fec03e00640 1 -- 172.21.15.83:0/115349018 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed657767d0 con 0x55ed65a3f940 2024-04-24T09:06:23.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.119+0000 7fec02a00640 1 --2- 172.21.15.83:0/115349018 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ed65a35270 0x55ed65a41250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:06:23.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.119+0000 7fec03e00640 1 --2- 172.21.15.83:0/115349018 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ed65a3f940 0x55ed65a461e0 secure :-1 s=READY pgs=2175 cs=0 l=1 rev1=1 crypto rx=0x7febfc0b9510 tx=0x7febfc0bd9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fb21b244d61f567 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:23.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.119+0000 7febecc00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:06:23.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.120+0000 7febecc00640 1 -- 172.21.15.83:0/115349018 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febfc0a0050 con 0x55ed65a3f940 2024-04-24T09:06:23.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.120+0000 7febecc00640 1 -- 172.21.15.83:0/115349018 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9583+0+0 (secure 0 0 0) 0x7febfc0beb60 con 0x55ed65a3f940 2024-04-24T09:06:23.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.120+0000 7fec0d024740 1 -- 172.21.15.83:0/115349018 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7febfc036070 con 0x55ed65a3f940 2024-04-24T09:06:23.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.120+0000 7febecc00640 1 -- 172.21.15.83:0/115349018 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7febfc0c64a0 con 0x55ed65a3f940 2024-04-24T09:06:23.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.120+0000 7fec0d024740 10 client.20811 fetch_fsmap learned FSMap version 48 2024-04-24T09:06:23.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.120+0000 7fec0d024740 1 -- 172.21.15.83:0/115349018 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55ed657767d0 con 0x55ed65a3f940 2024-04-24T09:06:23.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.121+0000 7febecc00640 1 -- 172.21.15.83:0/115349018 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7febfc036070 con 0x55ed65a3f940 2024-04-24T09:06:23.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.121+0000 7febecc00640 1 -- 172.21.15.83:0/115349018 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 48) ==== 8402+0+0 (secure 0 0 0) 0x7febfc0ef1e0 con 0x55ed65a3f940 2024-04-24T09:06:23.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.121+0000 7fec0d024740 10 client.20811 fetch_fsmap finished waiting for FSMap version 48 2024-04-24T09:06:23.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.121+0000 7fec0d024740 10 client.20811 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-24T09:06:23.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.121+0000 7fec0d024740 20 client.20811 populate_metadata read hostname 'smithi083' 2024-04-24T09:06:23.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.121+0000 7fec0d024740 1 --2- 172.21.15.83:0/115349018 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55ed65ab68f0 0x55ed65ab8d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:23.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.121+0000 7fec0d024740 4 client.20811 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:06:23.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.121+0000 7fec0d024740 1 -- 172.21.15.83:0/115349018 --> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ed659a1a60 con 0x55ed65ab68f0 2024-04-24T09:06:23.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.121+0000 7fec03400640 1 --2- 172.21.15.83:0/115349018 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55ed65ab68f0 0x55ed65ab8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:23.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.122+0000 7fec03400640 1 --2- 172.21.15.83:0/115349018 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55ed65ab68f0 0x55ed65ab8d10 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=897caa1a0916d216 server_cookie=a2afe68e09d6031d in_seq=0 out_seq=0 2024-04-24T09:06:23.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.122+0000 7febecc00640 10 client.20811 ms_handle_connect on v2:172.21.15.138:6832/1157967658 2024-04-24T09:06:23.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.123+0000 7febecc00640 1 -- 172.21.15.83:0/115349018 <== mds.3 v2:172.21.15.138:6832/1157967658 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ed659a1a60 con 0x55ed65ab68f0 2024-04-24T09:06:23.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.123+0000 7febecc00640 10 client.20811 handle_command_reply: tid=0 2024-04-24T09:06:23.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.123+0000 7fec0d024740 1 client.20811 shutdown 2024-04-24T09:06:23.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.123+0000 7fec0d024740 1 -- 172.21.15.83:0/115349018 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ed65a3f940 msgr2=0x55ed65a461e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:23.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.123+0000 7fec0d024740 1 --2- 172.21.15.83:0/115349018 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ed65a3f940 0x55ed65a461e0 secure :-1 s=READY pgs=2175 cs=0 l=1 rev1=1 crypto rx=0x7febfc0b9510 tx=0x7febfc0bd9b0 comp rx=0 tx=0).stop 2024-04-24T09:06:23.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.123+0000 7fec0d024740 1 -- 172.21.15.83:0/115349018 shutdown_connections 2024-04-24T09:06:23.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.123+0000 7fec0d024740 1 --2- 172.21.15.83:0/115349018 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ed65a3f940 0x55ed65a461e0 unknown :-1 s=CLOSED pgs=2175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:23.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.123+0000 7fec0d024740 1 --2- 172.21.15.83:0/115349018 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55ed65ab68f0 0x55ed65ab8d10 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:23.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.123+0000 7fec0d024740 1 --2- 172.21.15.83:0/115349018 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ed65a35270 0x55ed65a41250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:23.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.123+0000 7fec0d024740 1 --2- 172.21.15.83:0/115349018 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ed65a34950 0x55ed65a40d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:23.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.123+0000 7fec0d024740 1 -- 172.21.15.83:0/115349018 >> 172.21.15.83:0/115349018 conn(0x55ed65a30c70 msgr2=0x55ed65a2f560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:23.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.123+0000 7febecc00640 0 client.20811 ms_handle_reset on v2:172.21.15.138:6832/1157967658 2024-04-24T09:06:23.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.123+0000 7fec0d024740 1 -- 172.21.15.83:0/115349018 shutdown_connections 2024-04-24T09:06:23.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.124+0000 7fec0d024740 1 -- 172.21.15.83:0/115349018 wait complete. 2024-04-24T09:06:23.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.124+0000 7fec0d024740 20 client.20811 trim_cache size 0 max 16384 2024-04-24T09:06:23.125 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:06:23.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.124+0000 7fec0a200640 1 -- 172.21.15.83:0/4000688479 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7febe80a3e60 msgr2=0x7febe80a6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:23.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.124+0000 7fec0a200640 1 --2- 172.21.15.83:0/4000688479 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7febe80a3e60 0x7febe80a6320 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x55ed65a35f10 tx=0x7febf400b480 comp rx=0 tx=0).stop 2024-04-24T09:06:23.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.124+0000 7fec0a200640 1 -- 172.21.15.83:0/4000688479 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec0411b7e0 msgr2=0x7fec041abe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:23.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.124+0000 7fec0a200640 1 --2- 172.21.15.83:0/4000688479 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec0411b7e0 0x7fec041abe70 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7febfc009bf0 tx=0x7febfc003ba0 comp rx=0 tx=0).stop 2024-04-24T09:06:23.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.125+0000 7fec0a200640 1 -- 172.21.15.83:0/4000688479 shutdown_connections 2024-04-24T09:06:23.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.125+0000 7fec0a200640 1 --2- 172.21.15.83:0/4000688479 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7febe80a3e60 0x7febe80a6320 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:23.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.125+0000 7fec0a200640 1 --2- 172.21.15.83:0/4000688479 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec0411b7e0 0x7fec041abe70 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:23.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.125+0000 7fec0a200640 1 --2- 172.21.15.83:0/4000688479 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fec04110850 0x7fec041a7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:23.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.125+0000 7fec0a200640 1 --2- 172.21.15.83:0/4000688479 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec0410e880 0x7fec041a75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:23.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.125+0000 7fec0a200640 1 -- 172.21.15.83:0/4000688479 >> 172.21.15.83:0/4000688479 conn(0x7fec0406d110 msgr2=0x7fec04072b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:23.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.125+0000 7fec0a200640 1 -- 172.21.15.83:0/4000688479 shutdown_connections 2024-04-24T09:06:23.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.125+0000 7fec0a200640 1 -- 172.21.15.83:0/4000688479 wait complete. 2024-04-24T09:06:23.145 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T09:06:23.156 INFO:tasks.workunit.client.0.smithi083.stdout: 10 7019 10.34 MB/sec warmup 54 sec latency 78.075 ms 2024-04-24T09:06:23.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.384+0000 7f0600e00640 1 -- 172.21.15.83:0/562601509 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05fc109800 msgr2=0x7f05fc115360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:23.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.384+0000 7f0600e00640 1 --2- 172.21.15.83:0/562601509 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05fc109800 0x7f05fc115360 secure :-1 s=READY pgs=2176 cs=0 l=1 rev1=1 crypto rx=0x7f05f4009be0 tx=0x7f05f402a6c0 comp rx=0 tx=0).stop 2024-04-24T09:06:23.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.385+0000 7f0600e00640 1 -- 172.21.15.83:0/562601509 shutdown_connections 2024-04-24T09:06:23.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.385+0000 7f0600e00640 1 --2- 172.21.15.83:0/562601509 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05fc109800 0x7f05fc115360 unknown :-1 s=CLOSED pgs=2176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:23.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.385+0000 7f0600e00640 1 --2- 172.21.15.83:0/562601509 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f05fc108e40 0x7f05fc1092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:23.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.385+0000 7f0600e00640 1 --2- 172.21.15.83:0/562601509 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05fc10e5a0 0x7f05fc10e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:23.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.385+0000 7f0600e00640 1 -- 172.21.15.83:0/562601509 >> 172.21.15.83:0/562601509 conn(0x7f05fc077a20 msgr2=0x7f05fc079e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:23.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.385+0000 7f0600e00640 1 -- 172.21.15.83:0/562601509 shutdown_connections 2024-04-24T09:06:23.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.385+0000 7f0600e00640 1 -- 172.21.15.83:0/562601509 wait complete. 2024-04-24T09:06:23.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.386+0000 7f0600e00640 1 Processor -- start 2024-04-24T09:06:23.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.386+0000 7f0600e00640 1 -- start start 2024-04-24T09:06:23.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.386+0000 7f0600e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05fc108e40 0x7f05fc19ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:23.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.386+0000 7f0600e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05fc109800 0x7f05fc19f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:23.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.387+0000 7f0600e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f05fc10e5a0 0x7f05fc1a3610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:23.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.387+0000 7f0600e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f05fc118790 con 0x7f05fc108e40 2024-04-24T09:06:23.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.387+0000 7f0600e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f05fc118610 con 0x7f05fc109800 2024-04-24T09:06:23.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.387+0000 7f0600e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f05fc118910 con 0x7f05fc10e5a0 2024-04-24T09:06:23.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.387+0000 7f05fb400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05fc108e40 0x7f05fc19ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:23.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.387+0000 7f05faa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05fc109800 0x7f05fc19f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:23.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.387+0000 7f05faa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05fc109800 0x7f05fc19f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:38884/0 (socket says 172.21.15.83:38884) 2024-04-24T09:06:23.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.387+0000 7f05faa00640 1 -- 172.21.15.83:0/4073523884 learned_addr learned my addr 172.21.15.83:0/4073523884 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:06:23.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.387+0000 7f05fb400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05fc108e40 0x7f05fc19ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55860/0 (socket says 172.21.15.83:55860) 2024-04-24T09:06:23.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.387+0000 7f05fbe00640 1 --2- 172.21.15.83:0/4073523884 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f05fc10e5a0 0x7f05fc1a3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:23.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.388+0000 7f05faa00640 1 -- 172.21.15.83:0/4073523884 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f05fc10e5a0 msgr2=0x7f05fc1a3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:23.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.388+0000 7f05faa00640 1 --2- 172.21.15.83:0/4073523884 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f05fc10e5a0 0x7f05fc1a3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:23.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.388+0000 7f05faa00640 1 -- 172.21.15.83:0/4073523884 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05fc108e40 msgr2=0x7f05fc19ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:23.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.388+0000 7f05faa00640 1 --2- 172.21.15.83:0/4073523884 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05fc108e40 0x7f05fc19ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:23.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.388+0000 7f05faa00640 1 -- 172.21.15.83:0/4073523884 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05fc1a3d40 con 0x7f05fc109800 2024-04-24T09:06:23.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.388+0000 7f05fb400640 1 --2- 172.21.15.83:0/4073523884 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05fc108e40 0x7f05fc19ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:06:23.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.388+0000 7f05faa00640 1 --2- 172.21.15.83:0/4073523884 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05fc109800 0x7f05fc19f290 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f05e800b4a0 tx=0x7f05e800b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:23.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.388+0000 7f05f3e00640 1 -- 172.21.15.83:0/4073523884 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e8014070 con 0x7f05fc109800 2024-04-24T09:06:23.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.388+0000 7f0600e00640 1 -- 172.21.15.83:0/4073523884 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05fc1a4030 con 0x7f05fc109800 2024-04-24T09:06:23.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.389+0000 7f0600e00640 1 -- 172.21.15.83:0/4073523884 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05fc1ab8c0 con 0x7f05fc109800 2024-04-24T09:06:23.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.389+0000 7f05f3e00640 1 -- 172.21.15.83:0/4073523884 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f05e8010ac0 con 0x7f05fc109800 2024-04-24T09:06:23.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.389+0000 7f05f3e00640 1 -- 172.21.15.83:0/4073523884 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e8017b60 con 0x7f05fc109800 2024-04-24T09:06:23.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.393+0000 7f060536f740 1 Processor -- start 2024-04-24T09:06:23.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.393+0000 7f060536f740 1 -- start start 2024-04-24T09:06:23.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.393+0000 7f05f3e00640 1 -- 172.21.15.83:0/4073523884 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f05e8017070 con 0x7f05fc109800 2024-04-24T09:06:23.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.393+0000 7f060536f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e869299b40 0x55e869299f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:23.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.393+0000 7f060536f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e86929a460 0x55e8692a45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:23.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.394+0000 7f060536f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e8692a4b30 0x55e8692a6f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:23.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.394+0000 7f060536f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e8691ff1d0 con 0x55e869299b40 2024-04-24T09:06:23.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.394+0000 7f060536f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e8691a5450 con 0x55e86929a460 2024-04-24T09:06:23.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.394+0000 7f060536f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55e8691fb190 con 0x55e8692a4b30 2024-04-24T09:06:23.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.394+0000 7f05fb400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e86929a460 0x55e8692a45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:23.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.394+0000 7f05fbe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e869299b40 0x55e869299f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:23.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.394+0000 7f05fb400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e86929a460 0x55e8692a45f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:38898/0 (socket says 172.21.15.83:38898) 2024-04-24T09:06:23.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.394+0000 7f05fb400640 1 -- 172.21.15.83:0/2946761491 learned_addr learned my addr 172.21.15.83:0/2946761491 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:06:23.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.395+0000 7f05fbe00640 1 -- 172.21.15.83:0/2946761491 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e8692a4b30 msgr2=0x55e8692a6f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:23.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.395+0000 7f05fbe00640 1 --2- 172.21.15.83:0/2946761491 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e8692a4b30 0x55e8692a6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:23.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.395+0000 7f05fbe00640 1 -- 172.21.15.83:0/2946761491 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e86929a460 msgr2=0x55e8692a45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:23.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.395+0000 7f05fbe00640 1 --2- 172.21.15.83:0/2946761491 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e86929a460 0x55e8692a45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:23.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.395+0000 7f05fbe00640 1 -- 172.21.15.83:0/2946761491 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e869262310 con 0x55e869299b40 2024-04-24T09:06:23.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.395+0000 7f05f3e00640 1 --2- 172.21.15.83:0/4073523884 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f05e00a3e60 0x7f05e00a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:23.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.395+0000 7f05f3e00640 1 -- 172.21.15.83:0/4073523884 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9583+0+0 (secure 0 0 0) 0x7f05e80cfb90 con 0x7f05fc109800 2024-04-24T09:06:23.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.395+0000 7f05fbe00640 1 --2- 172.21.15.83:0/2946761491 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e869299b40 0x55e869299f20 secure :-1 s=READY pgs=2177 cs=0 l=1 rev1=1 crypto rx=0x7f05f4009b30 tx=0x7f05f40305c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99523796e438bf76 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:23.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.396+0000 7f05fb400640 1 --2- 172.21.15.83:0/4073523884 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f05e00a3e60 0x7f05e00a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:23.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.396+0000 7f05f0c00640 1 -- 172.21.15.83:0/2946761491 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f4041030 con 0x55e869299b40 2024-04-24T09:06:23.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.396+0000 7f05f0c00640 1 -- 172.21.15.83:0/2946761491 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f05f4037040 con 0x55e869299b40 2024-04-24T09:06:23.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.396+0000 7f05f0c00640 1 -- 172.21.15.83:0/2946761491 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f4036040 con 0x55e869299b40 2024-04-24T09:06:23.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.397+0000 7f060536f740 1 -- 172.21.15.83:0/2946761491 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e869299b40 msgr2=0x55e869299f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:23.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.397+0000 7f05fb400640 1 --2- 172.21.15.83:0/4073523884 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f05e00a3e60 0x7f05e00a6320 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x55e86929b100 tx=0x7f05ec009260 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:23.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.397+0000 7f060536f740 1 --2- 172.21.15.83:0/2946761491 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e869299b40 0x55e869299f20 secure :-1 s=READY pgs=2177 cs=0 l=1 rev1=1 crypto rx=0x7f05f4009b30 tx=0x7f05f40305c0 comp rx=0 tx=0).stop 2024-04-24T09:06:23.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.397+0000 7f060536f740 1 -- 172.21.15.83:0/2946761491 shutdown_connections 2024-04-24T09:06:23.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.397+0000 7f060536f740 1 --2- 172.21.15.83:0/2946761491 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e8692a4b30 0x55e8692a6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:23.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.397+0000 7f060536f740 1 --2- 172.21.15.83:0/2946761491 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e86929a460 0x55e8692a45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:23.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.397+0000 7f060536f740 1 --2- 172.21.15.83:0/2946761491 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e869299b40 0x55e869299f20 unknown :-1 s=CLOSED pgs=2177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:23.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.397+0000 7f060536f740 1 -- 172.21.15.83:0/2946761491 >> 172.21.15.83:0/2946761491 conn(0x55e869295e60 msgr2=0x55e869296240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:23.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.397+0000 7f060536f740 1 -- 172.21.15.83:0/2946761491 shutdown_connections 2024-04-24T09:06:23.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.397+0000 7f060536f740 1 -- 172.21.15.83:0/2946761491 wait complete. 2024-04-24T09:06:23.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.398+0000 7f060536f740 1 -- start start 2024-04-24T09:06:23.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.398+0000 7f060536f740 1 Processor -- start 2024-04-24T09:06:23.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.398+0000 7f060536f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e869299b40 0x55e8692a45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:23.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.398+0000 7f060536f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e86929a460 0x55e8692a18d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:23.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.398+0000 7f060536f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e8692a4b30 0x55e8692a2010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:23.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.398+0000 7f060536f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e8692a7960 con 0x55e869299b40 2024-04-24T09:06:23.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.399+0000 7f060536f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e8691b8810 con 0x55e86929a460 2024-04-24T09:06:23.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.399+0000 7f060536f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55e8692a7ae0 con 0x55e8692a4b30 2024-04-24T09:06:23.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.399+0000 7f060536f740 10 client.0 _ll_register_callbacks cb 0x55e869186140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:06:23.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.399+0000 7f05faa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e86929a460 0x55e8692a18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:23.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.399+0000 7f05faa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e86929a460 0x55e8692a18d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:38902/0 (socket says 172.21.15.83:38902) 2024-04-24T09:06:23.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.399+0000 7f05faa00640 1 -- 172.21.15.83:0/1132395975 learned_addr learned my addr 172.21.15.83:0/1132395975 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:06:23.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.399+0000 7f05fb400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e869299b40 0x55e8692a45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:23.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.399+0000 7f05faa00640 1 -- 172.21.15.83:0/1132395975 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e8692a4b30 msgr2=0x55e8692a2010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:23.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.399+0000 7f05faa00640 1 --2- 172.21.15.83:0/1132395975 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e8692a4b30 0x55e8692a2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:23.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.399+0000 7f05faa00640 1 -- 172.21.15.83:0/1132395975 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e869299b40 msgr2=0x55e8692a45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:23.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.399+0000 7f05faa00640 1 --2- 172.21.15.83:0/1132395975 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e869299b40 0x55e8692a45e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:23.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.400+0000 7f05faa00640 1 -- 172.21.15.83:0/1132395975 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8691c93f0 con 0x55e86929a460 2024-04-24T09:06:23.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.400+0000 7f05fb400640 1 --2- 172.21.15.83:0/1132395975 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e869299b40 0x55e8692a45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:06:23.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.400+0000 7f05faa00640 1 --2- 172.21.15.83:0/1132395975 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e86929a460 0x55e8692a18d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f05e800be60 tx=0x7f05e809b2c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0c44be16b95caba server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:23.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.400+0000 7f05e5600640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:06:23.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.400+0000 7f05e5600640 1 -- 172.21.15.83:0/1132395975 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e801e040 con 0x55e86929a460 2024-04-24T09:06:23.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.401+0000 7f05e5600640 1 -- 172.21.15.83:0/1132395975 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9583+0+0 (secure 0 0 0) 0x7f05e8097ee0 con 0x55e86929a460 2024-04-24T09:06:23.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.401+0000 7f060536f740 1 -- 172.21.15.83:0/1132395975 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05f4037040 con 0x55e86929a460 2024-04-24T09:06:23.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.401+0000 7f05e5600640 1 -- 172.21.15.83:0/1132395975 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f05e80a4430 con 0x55e86929a460 2024-04-24T09:06:23.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.401+0000 7f060536f740 10 client.37964 fetch_fsmap learned FSMap version 48 2024-04-24T09:06:23.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.401+0000 7f060536f740 1 -- 172.21.15.83:0/1132395975 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55e868f113f0 con 0x55e86929a460 2024-04-24T09:06:23.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.402+0000 7f05e5600640 1 -- 172.21.15.83:0/1132395975 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f05f4037040 con 0x55e86929a460 2024-04-24T09:06:23.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.402+0000 7f05e5600640 1 -- 172.21.15.83:0/1132395975 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 48) ==== 8402+0+0 (secure 0 0 0) 0x7f05e80a3fa0 con 0x55e86929a460 2024-04-24T09:06:23.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.402+0000 7f060536f740 10 client.37964 fetch_fsmap finished waiting for FSMap version 48 2024-04-24T09:06:23.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.402+0000 7f060536f740 10 client.37964 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-24T09:06:23.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.402+0000 7f060536f740 20 client.37964 populate_metadata read hostname 'smithi083' 2024-04-24T09:06:23.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.402+0000 7f060536f740 1 --2- 172.21.15.83:0/1132395975 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55e8692b4e60 0x55e8692b7250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:23.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.402+0000 7f060536f740 4 client.37964 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:06:23.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.402+0000 7f060536f740 1 -- 172.21.15.83:0/1132395975 --> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e86920cbf0 con 0x55e8692b4e60 2024-04-24T09:06:23.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.402+0000 7f05fb400640 1 --2- 172.21.15.83:0/1132395975 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55e8692b4e60 0x55e8692b7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:23.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.403+0000 7f05fb400640 1 --2- 172.21.15.83:0/1132395975 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55e8692b4e60 0x55e8692b7250 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ba69f6921bd10a41 server_cookie=6733d6645bb2d702 in_seq=0 out_seq=0 2024-04-24T09:06:23.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.403+0000 7f05e5600640 10 client.37964 ms_handle_connect on v2:172.21.15.138:6834/1909219758 2024-04-24T09:06:23.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.404+0000 7f05e5600640 1 -- 172.21.15.83:0/1132395975 <== mds.2 v2:172.21.15.138:6834/1909219758 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e86920cbf0 con 0x55e8692b4e60 2024-04-24T09:06:23.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.404+0000 7f05e5600640 10 client.37964 handle_command_reply: tid=0 2024-04-24T09:06:23.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.404+0000 7f060536f740 1 client.37964 shutdown 2024-04-24T09:06:23.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.405+0000 7f060536f740 1 -- 172.21.15.83:0/1132395975 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e86929a460 msgr2=0x55e8692a18d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:23.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.405+0000 7f060536f740 1 --2- 172.21.15.83:0/1132395975 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e86929a460 0x55e8692a18d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f05e800be60 tx=0x7f05e809b2c0 comp rx=0 tx=0).stop 2024-04-24T09:06:23.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.405+0000 7f060536f740 1 -- 172.21.15.83:0/1132395975 shutdown_connections 2024-04-24T09:06:23.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.405+0000 7f060536f740 1 --2- 172.21.15.83:0/1132395975 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55e8692b4e60 0x55e8692b7250 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:23.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.405+0000 7f060536f740 1 --2- 172.21.15.83:0/1132395975 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e8692a4b30 0x55e8692a2010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:23.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.405+0000 7f060536f740 1 --2- 172.21.15.83:0/1132395975 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e86929a460 0x55e8692a18d0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:23.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.405+0000 7f060536f740 1 --2- 172.21.15.83:0/1132395975 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e869299b40 0x55e8692a45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:23.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.405+0000 7f060536f740 1 -- 172.21.15.83:0/1132395975 >> 172.21.15.83:0/1132395975 conn(0x55e869295e60 msgr2=0x55e86929bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:23.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.405+0000 7f05e5600640 0 client.37964 ms_handle_reset on v2:172.21.15.138:6834/1909219758 2024-04-24T09:06:23.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.405+0000 7f060536f740 1 -- 172.21.15.83:0/1132395975 shutdown_connections 2024-04-24T09:06:23.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.405+0000 7f060536f740 1 -- 172.21.15.83:0/1132395975 wait complete. 2024-04-24T09:06:23.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.405+0000 7f060536f740 20 client.37964 trim_cache size 0 max 16384 2024-04-24T09:06:23.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.427+0000 7f060536f740 1 Processor -- start 2024-04-24T09:06:23.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.427+0000 7f060536f740 1 -- start start 2024-04-24T09:06:23.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.428+0000 7f060536f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e869299b40 0x55e8692b0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:23.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.428+0000 7f060536f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e86929a460 0x55e8692a4110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:23.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.428+0000 7f060536f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e8692a4b30 0x55e8692bcb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:23.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.428+0000 7f060536f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f05dc010070 con 0x55e869299b40 2024-04-24T09:06:23.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.428+0000 7f060536f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f05dc011d30 con 0x55e86929a460 2024-04-24T09:06:23.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.428+0000 7f060536f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f05dc010740 con 0x55e8692a4b30 2024-04-24T09:06:23.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.428+0000 7f05faa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e86929a460 0x55e8692a4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:23.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.428+0000 7f05fb400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e869299b40 0x55e8692b0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:23.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.428+0000 7f05faa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e86929a460 0x55e8692a4110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:38908/0 (socket says 172.21.15.83:38908) 2024-04-24T09:06:23.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.428+0000 7f05faa00640 1 -- 172.21.15.83:0/2069657309 learned_addr learned my addr 172.21.15.83:0/2069657309 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:06:23.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.428+0000 7f05fbe00640 1 --2- 172.21.15.83:0/2069657309 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e8692a4b30 0x55e8692bcb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:23.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.429+0000 7f05fb400640 1 -- 172.21.15.83:0/2069657309 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e8692a4b30 msgr2=0x55e8692bcb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:23.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.429+0000 7f05fb400640 1 --2- 172.21.15.83:0/2069657309 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e8692a4b30 0x55e8692bcb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:23.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.429+0000 7f05fb400640 1 -- 172.21.15.83:0/2069657309 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e86929a460 msgr2=0x55e8692a4110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:23.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.429+0000 7f05fb400640 1 --2- 172.21.15.83:0/2069657309 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e86929a460 0x55e8692a4110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:23.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.429+0000 7f05fb400640 1 -- 172.21.15.83:0/2069657309 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e86929a290 con 0x55e869299b40 2024-04-24T09:06:23.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.429+0000 7f05faa00640 1 --2- 172.21.15.83:0/2069657309 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e86929a460 0x55e8692a4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:06:23.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.429+0000 7f05fb400640 1 --2- 172.21.15.83:0/2069657309 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e869299b40 0x55e8692b0880 secure :-1 s=READY pgs=2178 cs=0 l=1 rev1=1 crypto rx=0x55e8692b7a90 tx=0x7f05ec01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0fb359af50f36a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:23.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.429+0000 7f05e7e00640 1 -- 172.21.15.83:0/2069657309 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05ec02a4f0 con 0x55e869299b40 2024-04-24T09:06:23.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.429+0000 7f05e7e00640 1 -- 172.21.15.83:0/2069657309 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f05ec02a690 con 0x55e869299b40 2024-04-24T09:06:23.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.429+0000 7f05e7e00640 1 -- 172.21.15.83:0/2069657309 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05ec0039a0 con 0x55e869299b40 2024-04-24T09:06:23.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.430+0000 7f060536f740 1 -- 172.21.15.83:0/2069657309 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e869299b40 msgr2=0x55e8692b0880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:23.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.430+0000 7f060536f740 1 --2- 172.21.15.83:0/2069657309 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e869299b40 0x55e8692b0880 secure :-1 s=READY pgs=2178 cs=0 l=1 rev1=1 crypto rx=0x55e8692b7a90 tx=0x7f05ec01a000 comp rx=0 tx=0).stop 2024-04-24T09:06:23.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.430+0000 7f060536f740 1 -- 172.21.15.83:0/2069657309 shutdown_connections 2024-04-24T09:06:23.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.430+0000 7f060536f740 1 --2- 172.21.15.83:0/2069657309 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e8692a4b30 0x55e8692bcb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:23.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.430+0000 7f060536f740 1 --2- 172.21.15.83:0/2069657309 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e86929a460 0x55e8692a4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:23.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.430+0000 7f060536f740 1 --2- 172.21.15.83:0/2069657309 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e869299b40 0x55e8692b0880 unknown :-1 s=CLOSED pgs=2178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:23.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.430+0000 7f060536f740 1 -- 172.21.15.83:0/2069657309 >> 172.21.15.83:0/2069657309 conn(0x55e869295e60 msgr2=0x55e869296240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:23.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.431+0000 7f060536f740 1 -- 172.21.15.83:0/2069657309 shutdown_connections 2024-04-24T09:06:23.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.431+0000 7f060536f740 1 -- 172.21.15.83:0/2069657309 wait complete. 2024-04-24T09:06:23.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.431+0000 7f060536f740 1 -- start start 2024-04-24T09:06:23.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.431+0000 7f060536f740 1 Processor -- start 2024-04-24T09:06:23.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.432+0000 7f060536f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e869299b40 0x55e8692a5320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:23.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.432+0000 7f060536f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e86929a460 0x55e8692a5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:23.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.432+0000 7f060536f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e8692a4b30 0x55e8692a5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:23.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.432+0000 7f060536f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f05dc0115a0 con 0x55e86929a460 2024-04-24T09:06:23.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.432+0000 7f060536f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f05dc010e70 con 0x55e869299b40 2024-04-24T09:06:23.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.432+0000 7f060536f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f05dc00e070 con 0x55e8692a4b30 2024-04-24T09:06:23.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.432+0000 7f060536f740 10 client.0 _ll_register_callbacks cb 0x7f05e80092f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:06:23.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.432+0000 7f05fb400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e869299b40 0x55e8692a5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:23.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.433+0000 7f05fb400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e869299b40 0x55e8692a5320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:38922/0 (socket says 172.21.15.83:38922) 2024-04-24T09:06:23.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.433+0000 7f05fb400640 1 -- 172.21.15.83:0/476260192 learned_addr learned my addr 172.21.15.83:0/476260192 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:06:23.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.433+0000 7f05faa00640 1 --2- 172.21.15.83:0/476260192 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e86929a460 0x55e8692a5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:23.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.433+0000 7f05fbe00640 1 --2- 172.21.15.83:0/476260192 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e8692a4b30 0x55e8692a5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:23.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.433+0000 7f05fb400640 1 -- 172.21.15.83:0/476260192 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e8692a4b30 msgr2=0x55e8692a5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:23.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.433+0000 7f05fb400640 1 --2- 172.21.15.83:0/476260192 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e8692a4b30 0x55e8692a5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:23.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.433+0000 7f05fb400640 1 -- 172.21.15.83:0/476260192 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e86929a460 msgr2=0x55e8692a5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:23.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.433+0000 7f05fb400640 1 --2- 172.21.15.83:0/476260192 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e86929a460 0x55e8692a5860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:23.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.433+0000 7f05fb400640 1 -- 172.21.15.83:0/476260192 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e868fdb7d0 con 0x55e869299b40 2024-04-24T09:06:23.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.433+0000 7f05faa00640 1 --2- 172.21.15.83:0/476260192 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e86929a460 0x55e8692a5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:06:23.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.434+0000 7f05fb400640 1 --2- 172.21.15.83:0/476260192 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e869299b40 0x55e8692a5320 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f05ec019040 tx=0x7f05ec02a4f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f6e49af05988a50 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:23.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.434+0000 7f05e5600640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:06:23.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.434+0000 7f05e5600640 1 -- 172.21.15.83:0/476260192 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05ec039020 con 0x55e869299b40 2024-04-24T09:06:23.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.434+0000 7f060536f740 1 -- 172.21.15.83:0/476260192 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05ec02a690 con 0x55e869299b40 2024-04-24T09:06:23.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.434+0000 7f05e5600640 1 -- 172.21.15.83:0/476260192 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9583+0+0 (secure 0 0 0) 0x7f05ec005040 con 0x55e869299b40 2024-04-24T09:06:23.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.435+0000 7f05e5600640 1 -- 172.21.15.83:0/476260192 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f05ec044050 con 0x55e869299b40 2024-04-24T09:06:23.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.435+0000 7f060536f740 10 client.37970 fetch_fsmap learned FSMap version 48 2024-04-24T09:06:23.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.435+0000 7f060536f740 1 -- 172.21.15.83:0/476260192 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55e8692acf10 con 0x55e869299b40 2024-04-24T09:06:23.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.435+0000 7f05e5600640 1 -- 172.21.15.83:0/476260192 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f05ec02a690 con 0x55e869299b40 2024-04-24T09:06:23.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.435+0000 7f05e5600640 1 -- 172.21.15.83:0/476260192 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 48) ==== 8402+0+0 (secure 0 0 0) 0x7f05ec004c30 con 0x55e869299b40 2024-04-24T09:06:23.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.435+0000 7f060536f740 10 client.37970 fetch_fsmap finished waiting for FSMap version 48 2024-04-24T09:06:23.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.435+0000 7f060536f740 10 client.37970 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-24T09:06:23.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.435+0000 7f060536f740 20 client.37970 populate_metadata read hostname 'smithi083' 2024-04-24T09:06:23.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.435+0000 7f060536f740 1 --2- 172.21.15.83:0/476260192 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55e86931b390 0x55e86931d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:23.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.435+0000 7f060536f740 4 client.37970 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:06:23.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.435+0000 7f060536f740 1 -- 172.21.15.83:0/476260192 --> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e869206a60 con 0x55e86931b390 2024-04-24T09:06:23.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.436+0000 7f05faa00640 1 --2- 172.21.15.83:0/476260192 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55e86931b390 0x55e86931d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:23.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.437+0000 7f05faa00640 1 --2- 172.21.15.83:0/476260192 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55e86931b390 0x55e86931d780 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=70c99975c3cf7a0c server_cookie=503e132675bf9589 in_seq=0 out_seq=0 2024-04-24T09:06:23.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.437+0000 7f05e5600640 10 client.37970 ms_handle_connect on v2:172.21.15.138:6834/1909219758 2024-04-24T09:06:23.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.439+0000 7f05e5600640 1 -- 172.21.15.83:0/476260192 <== mds.2 v2:172.21.15.138:6834/1909219758 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e869206a60 con 0x55e86931b390 2024-04-24T09:06:23.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.439+0000 7f05e5600640 10 client.37970 handle_command_reply: tid=0 2024-04-24T09:06:23.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.439+0000 7f060536f740 1 client.37970 shutdown 2024-04-24T09:06:23.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.439+0000 7f060536f740 1 -- 172.21.15.83:0/476260192 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e869299b40 msgr2=0x55e8692a5320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:23.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.439+0000 7f060536f740 1 --2- 172.21.15.83:0/476260192 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e869299b40 0x55e8692a5320 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f05ec019040 tx=0x7f05ec02a4f0 comp rx=0 tx=0).stop 2024-04-24T09:06:23.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.439+0000 7f060536f740 1 -- 172.21.15.83:0/476260192 shutdown_connections 2024-04-24T09:06:23.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.439+0000 7f060536f740 1 --2- 172.21.15.83:0/476260192 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55e86931b390 0x55e86931d780 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:23.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.439+0000 7f060536f740 1 --2- 172.21.15.83:0/476260192 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e8692a4b30 0x55e8692a5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:23.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.439+0000 7f060536f740 1 --2- 172.21.15.83:0/476260192 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e86929a460 0x55e8692a5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:23.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.439+0000 7f060536f740 1 --2- 172.21.15.83:0/476260192 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e869299b40 0x55e8692a5320 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:23.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.439+0000 7f060536f740 1 -- 172.21.15.83:0/476260192 >> 172.21.15.83:0/476260192 conn(0x55e869295e60 msgr2=0x55e86929cb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:23.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.440+0000 7f05e5600640 0 client.37970 ms_handle_reset on v2:172.21.15.138:6834/1909219758 2024-04-24T09:06:23.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.440+0000 7f060536f740 1 -- 172.21.15.83:0/476260192 shutdown_connections 2024-04-24T09:06:23.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.440+0000 7f060536f740 1 -- 172.21.15.83:0/476260192 wait complete. 2024-04-24T09:06:23.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.440+0000 7f060536f740 20 client.37970 trim_cache size 0 max 16384 2024-04-24T09:06:23.441 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:06:23.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.441+0000 7f0600e00640 1 -- 172.21.15.83:0/4073523884 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f05e00a3e60 msgr2=0x7f05e00a6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:23.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.441+0000 7f0600e00640 1 --2- 172.21.15.83:0/4073523884 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f05e00a3e60 0x7f05e00a6320 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x55e86929b100 tx=0x7f05ec009260 comp rx=0 tx=0).stop 2024-04-24T09:06:23.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.441+0000 7f0600e00640 1 -- 172.21.15.83:0/4073523884 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05fc109800 msgr2=0x7f05fc19f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:23.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.441+0000 7f0600e00640 1 --2- 172.21.15.83:0/4073523884 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05fc109800 0x7f05fc19f290 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f05e800b4a0 tx=0x7f05e800b960 comp rx=0 tx=0).stop 2024-04-24T09:06:23.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.441+0000 7f0600e00640 1 -- 172.21.15.83:0/4073523884 shutdown_connections 2024-04-24T09:06:23.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.441+0000 7f0600e00640 1 --2- 172.21.15.83:0/4073523884 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f05e00a3e60 0x7f05e00a6320 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:23.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.441+0000 7f0600e00640 1 --2- 172.21.15.83:0/4073523884 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f05fc10e5a0 0x7f05fc1a3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:23.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.441+0000 7f0600e00640 1 --2- 172.21.15.83:0/4073523884 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05fc109800 0x7f05fc19f290 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:23.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.441+0000 7f0600e00640 1 --2- 172.21.15.83:0/4073523884 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05fc108e40 0x7f05fc19ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:23.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.441+0000 7f0600e00640 1 -- 172.21.15.83:0/4073523884 >> 172.21.15.83:0/4073523884 conn(0x7f05fc077a20 msgr2=0x7f05fc07a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:23.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.442+0000 7f0600e00640 1 -- 172.21.15.83:0/4073523884 shutdown_connections 2024-04-24T09:06:23.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.442+0000 7f0600e00640 1 -- 172.21.15.83:0/4073523884 wait complete. 2024-04-24T09:06:23.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:06:23 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/4116579888' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:06:23.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:06:23 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/4116579888' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:06:23.465 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T09:06:23.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.756+0000 7f7388600640 1 -- 172.21.15.83:0/364561353 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f73801044b0 msgr2=0x7f7380104890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:23.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.756+0000 7f7388600640 1 --2- 172.21.15.83:0/364561353 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f73801044b0 0x7f7380104890 secure :-1 s=READY pgs=2179 cs=0 l=1 rev1=1 crypto rx=0x7f7374009900 tx=0x7f737402a410 comp rx=0 tx=0).stop 2024-04-24T09:06:23.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.756+0000 7f7388600640 1 -- 172.21.15.83:0/364561353 shutdown_connections 2024-04-24T09:06:23.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.756+0000 7f7388600640 1 --2- 172.21.15.83:0/364561353 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7380109580 0x7f7380111150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:23.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.756+0000 7f7388600640 1 --2- 172.21.15.83:0/364561353 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7380104e60 0x7f7380108ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:23.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.756+0000 7f7388600640 1 --2- 172.21.15.83:0/364561353 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f73801044b0 0x7f7380104890 unknown :-1 s=CLOSED pgs=2179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:23.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.756+0000 7f7388600640 1 -- 172.21.15.83:0/364561353 >> 172.21.15.83:0/364561353 conn(0x7f73800ffd90 msgr2=0x7f73801021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:23.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.756+0000 7f7388600640 1 -- 172.21.15.83:0/364561353 shutdown_connections 2024-04-24T09:06:23.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.756+0000 7f7388600640 1 -- 172.21.15.83:0/364561353 wait complete. 2024-04-24T09:06:23.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.757+0000 7f7388600640 1 Processor -- start 2024-04-24T09:06:23.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.758+0000 7f7388600640 1 -- start start 2024-04-24T09:06:23.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.758+0000 7f7388600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f73801044b0 0x7f73801a0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:23.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.758+0000 7f7388600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7380109580 0x7f73801a0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:23.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.758+0000 7f7388600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f738019a970 0x7f738019ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:23.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.758+0000 7f7388600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7380114590 con 0x7f73801044b0 2024-04-24T09:06:23.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.758+0000 7f7388600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7380114410 con 0x7f738019a970 2024-04-24T09:06:23.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.758+0000 7f7388600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7380114710 con 0x7f7380109580 2024-04-24T09:06:23.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.758+0000 7f7385c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f73801044b0 0x7f73801a0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:23.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.759+0000 7f7385c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f73801044b0 0x7f73801a0770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55918/0 (socket says 172.21.15.83:55918) 2024-04-24T09:06:23.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.759+0000 7f7385c00640 1 -- 172.21.15.83:0/80249375 learned_addr learned my addr 172.21.15.83:0/80249375 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:06:23.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.759+0000 7f7386600640 1 --2- 172.21.15.83:0/80249375 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f738019a970 0x7f738019ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:23.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.759+0000 7f7385200640 1 --2- 172.21.15.83:0/80249375 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7380109580 0x7f73801a0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:23.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.759+0000 7f7385c00640 1 -- 172.21.15.83:0/80249375 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7380109580 msgr2=0x7f73801a0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:23.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.759+0000 7f7385c00640 1 --2- 172.21.15.83:0/80249375 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7380109580 0x7f73801a0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:23.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.759+0000 7f7385c00640 1 -- 172.21.15.83:0/80249375 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f738019a970 msgr2=0x7f738019ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:23.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.759+0000 7f7385c00640 1 --2- 172.21.15.83:0/80249375 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f738019a970 0x7f738019ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:23.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.759+0000 7f7385c00640 1 -- 172.21.15.83:0/80249375 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f738019b490 con 0x7f73801044b0 2024-04-24T09:06:23.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.760+0000 7f7385200640 1 --2- 172.21.15.83:0/80249375 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7380109580 0x7f73801a0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:06:23.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.760+0000 7f7385c00640 1 --2- 172.21.15.83:0/80249375 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f73801044b0 0x7f73801a0770 secure :-1 s=READY pgs=2180 cs=0 l=1 rev1=1 crypto rx=0x7f7374005ab0 tx=0x7f7374003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:23.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.760+0000 7f737e000640 1 -- 172.21.15.83:0/80249375 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f737403f070 con 0x7f73801044b0 2024-04-24T09:06:23.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.760+0000 7f7388600640 1 -- 172.21.15.83:0/80249375 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f738019b720 con 0x7f73801044b0 2024-04-24T09:06:23.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.760+0000 7f7388600640 1 -- 172.21.15.83:0/80249375 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73801a7920 con 0x7f73801044b0 2024-04-24T09:06:23.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.760+0000 7f737e000640 1 -- 172.21.15.83:0/80249375 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7374031b50 con 0x7f73801044b0 2024-04-24T09:06:23.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.760+0000 7f737e000640 1 -- 172.21.15.83:0/80249375 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7374049060 con 0x7f73801044b0 2024-04-24T09:06:23.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.765+0000 7f738b45a740 1 Processor -- start 2024-04-24T09:06:23.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.765+0000 7f738b45a740 1 -- start start 2024-04-24T09:06:23.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.765+0000 7f738b45a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f30bce0950 0x55f30bce0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:23.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.765+0000 7f738b45a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f30bce1270 0x55f30bceb400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:23.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.765+0000 7f738b45a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f30bceb940 0x55f30bcedd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:23.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.765+0000 7f738b45a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55f30bc461d0 con 0x55f30bce0950 2024-04-24T09:06:23.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.765+0000 7f738b45a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f30bbec450 con 0x55f30bceb940 2024-04-24T09:06:23.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.765+0000 7f738b45a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55f30bc42190 con 0x55f30bce1270 2024-04-24T09:06:23.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.766+0000 7f737e000640 1 -- 172.21.15.83:0/80249375 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7374042070 con 0x7f73801044b0 2024-04-24T09:06:23.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.766+0000 7f7386600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f30bce0950 0x55f30bce0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:23.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.766+0000 7f7386600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f30bce0950 0x55f30bce0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55920/0 (socket says 172.21.15.83:55920) 2024-04-24T09:06:23.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.766+0000 7f7386600640 1 -- 172.21.15.83:0/2709978788 learned_addr learned my addr 172.21.15.83:0/2709978788 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:06:23.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.766+0000 7f7385200640 1 --2- 172.21.15.83:0/2709978788 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f30bceb940 0x55f30bcedd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:23.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.766+0000 7f7385c00640 1 --2- 172.21.15.83:0/2709978788 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f30bce1270 0x55f30bceb400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:23.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.766+0000 7f7386600640 1 -- 172.21.15.83:0/2709978788 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f30bce1270 msgr2=0x55f30bceb400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:23.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.766+0000 7f7386600640 1 --2- 172.21.15.83:0/2709978788 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f30bce1270 0x55f30bceb400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:23.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.766+0000 7f7386600640 1 -- 172.21.15.83:0/2709978788 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f30bceb940 msgr2=0x55f30bcedd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:23.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.766+0000 7f7386600640 1 --2- 172.21.15.83:0/2709978788 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f30bceb940 0x55f30bcedd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:23.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.766+0000 7f7386600640 1 -- 172.21.15.83:0/2709978788 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f30bca9310 con 0x55f30bce0950 2024-04-24T09:06:23.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.767+0000 7f7386600640 1 --2- 172.21.15.83:0/2709978788 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f30bce0950 0x55f30bce0d30 secure :-1 s=READY pgs=2181 cs=0 l=1 rev1=1 crypto rx=0x7f737800a430 tx=0x7f7378009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5736c93ebed2c945 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:23.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.767+0000 7f7372a00640 1 -- 172.21.15.83:0/2709978788 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7378013790 con 0x55f30bce0950 2024-04-24T09:06:23.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.767+0000 7f7372a00640 1 -- 172.21.15.83:0/2709978788 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7378019070 con 0x55f30bce0950 2024-04-24T09:06:23.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.767+0000 7f7372a00640 1 -- 172.21.15.83:0/2709978788 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7378014d00 con 0x55f30bce0950 2024-04-24T09:06:23.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.768+0000 7f737e000640 1 --2- 172.21.15.83:0/80249375 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f73680a3e60 0x7f73680a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:23.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.768+0000 7f737e000640 1 -- 172.21.15.83:0/80249375 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9583+0+0 (secure 0 0 0) 0x7f73740f0fd0 con 0x7f73801044b0 2024-04-24T09:06:23.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.768+0000 7f7385c00640 1 --2- 172.21.15.83:0/80249375 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f73680a3e60 0x7f73680a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:23.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.768+0000 7f738b45a740 1 -- 172.21.15.83:0/2709978788 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f30bce0950 msgr2=0x55f30bce0d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:23.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.768+0000 7f738b45a740 1 --2- 172.21.15.83:0/2709978788 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f30bce0950 0x55f30bce0d30 secure :-1 s=READY pgs=2181 cs=0 l=1 rev1=1 crypto rx=0x7f737800a430 tx=0x7f7378009650 comp rx=0 tx=0).stop 2024-04-24T09:06:23.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.769+0000 7f738b45a740 1 -- 172.21.15.83:0/2709978788 shutdown_connections 2024-04-24T09:06:23.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.769+0000 7f738b45a740 1 --2- 172.21.15.83:0/2709978788 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f30bceb940 0x55f30bcedd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:23.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.769+0000 7f738b45a740 1 --2- 172.21.15.83:0/2709978788 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f30bce1270 0x55f30bceb400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:23.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.769+0000 7f738b45a740 1 --2- 172.21.15.83:0/2709978788 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f30bce0950 0x55f30bce0d30 unknown :-1 s=CLOSED pgs=2181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:23.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.769+0000 7f738b45a740 1 -- 172.21.15.83:0/2709978788 >> 172.21.15.83:0/2709978788 conn(0x55f30bcdcc70 msgr2=0x55f30bcdd050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:23.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.769+0000 7f7385c00640 1 --2- 172.21.15.83:0/80249375 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f73680a3e60 0x7f73680a6320 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x55f30bce1f10 tx=0x7f7374009260 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:23.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.769+0000 7f738b45a740 1 -- 172.21.15.83:0/2709978788 shutdown_connections 2024-04-24T09:06:23.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.770+0000 7f738b45a740 1 -- 172.21.15.83:0/2709978788 wait complete. 2024-04-24T09:06:23.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.770+0000 7f738b45a740 1 -- start start 2024-04-24T09:06:23.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.770+0000 7f738b45a740 1 Processor -- start 2024-04-24T09:06:23.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.771+0000 7f738b45a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f30bce0950 0x55f30bceb350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:23.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.771+0000 7f738b45a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f30bce1270 0x55f30bce8640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:23.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.771+0000 7f738b45a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f30bceb940 0x55f30bce8c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:23.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.771+0000 7f738b45a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55f30bcee7d0 con 0x55f30bce0950 2024-04-24T09:06:23.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.771+0000 7f738b45a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f30bbff810 con 0x55f30bce1270 2024-04-24T09:06:23.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.771+0000 7f738b45a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55f30bcee950 con 0x55f30bceb940 2024-04-24T09:06:23.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.771+0000 7f738b45a740 10 client.0 _ll_register_callbacks cb 0x55f30bbcd140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:06:23.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.771+0000 7f7386600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f30bce0950 0x55f30bceb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:23.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.772+0000 7f7386600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f30bce0950 0x55f30bceb350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55922/0 (socket says 172.21.15.83:55922) 2024-04-24T09:06:23.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.772+0000 7f7386600640 1 -- 172.21.15.83:0/2011734989 learned_addr learned my addr 172.21.15.83:0/2011734989 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:06:23.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.772+0000 7f7385200640 1 --2- 172.21.15.83:0/2011734989 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f30bce1270 0x55f30bce8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:23.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.772+0000 7f7386600640 1 -- 172.21.15.83:0/2011734989 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f30bceb940 msgr2=0x55f30bce8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:23.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.772+0000 7f7386600640 1 --2- 172.21.15.83:0/2011734989 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f30bceb940 0x55f30bce8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:23.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.772+0000 7f7386600640 1 -- 172.21.15.83:0/2011734989 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f30bce1270 msgr2=0x55f30bce8640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:23.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.772+0000 7f7386600640 1 --2- 172.21.15.83:0/2011734989 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f30bce1270 0x55f30bce8640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:23.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.772+0000 7f7386600640 1 -- 172.21.15.83:0/2011734989 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f30bc103f0 con 0x55f30bce0950 2024-04-24T09:06:23.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.772+0000 7f7386600640 1 --2- 172.21.15.83:0/2011734989 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f30bce0950 0x55f30bceb350 secure :-1 s=READY pgs=2182 cs=0 l=1 rev1=1 crypto rx=0x7f7378007090 tx=0x7f7378014c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0cd3c6a21417dc7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:23.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.772+0000 7f7367400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:06:23.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.773+0000 7f7367400640 1 -- 172.21.15.83:0/2011734989 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7378014e60 con 0x55f30bce0950 2024-04-24T09:06:23.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.773+0000 7f7367400640 1 -- 172.21.15.83:0/2011734989 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9583+0+0 (secure 0 0 0) 0x7f73780139a0 con 0x55f30bce0950 2024-04-24T09:06:23.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.773+0000 7f738b45a740 1 -- 172.21.15.83:0/2011734989 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7378019070 con 0x55f30bce0950 2024-04-24T09:06:23.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.773+0000 7f7367400640 1 -- 172.21.15.83:0/2011734989 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f7378014420 con 0x55f30bce0950 2024-04-24T09:06:23.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.774+0000 7f738b45a740 10 client.20841 fetch_fsmap learned FSMap version 48 2024-04-24T09:06:23.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.774+0000 7f738b45a740 1 -- 172.21.15.83:0/2011734989 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55f30b9583f0 con 0x55f30bce0950 2024-04-24T09:06:23.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.774+0000 7f7367400640 1 -- 172.21.15.83:0/2011734989 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7378019070 con 0x55f30bce0950 2024-04-24T09:06:23.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.774+0000 7f7367400640 1 -- 172.21.15.83:0/2011734989 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 48) ==== 8402+0+0 (secure 0 0 0) 0x7f7378032b20 con 0x55f30bce0950 2024-04-24T09:06:23.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.774+0000 7f738b45a740 10 client.20841 fetch_fsmap finished waiting for FSMap version 48 2024-04-24T09:06:23.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.774+0000 7f738b45a740 10 client.20841 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-24T09:06:23.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.774+0000 7f738b45a740 20 client.20841 populate_metadata read hostname 'smithi083' 2024-04-24T09:06:23.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.774+0000 7f738b45a740 1 --2- 172.21.15.83:0/2011734989 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x55f30bcfbaa0 0x55f30bcfde90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:23.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.774+0000 7f738b45a740 4 client.20841 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:06:23.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.774+0000 7f738b45a740 1 -- 172.21.15.83:0/2011734989 --> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f30bc53bf0 con 0x55f30bcfbaa0 2024-04-24T09:06:23.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.775+0000 7f7385200640 1 --2- 172.21.15.83:0/2011734989 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x55f30bcfbaa0 0x55f30bcfde90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:23.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.776+0000 7f7367400640 10 client.20841 ms_handle_connect on v2:172.21.15.138:6836/900566754 2024-04-24T09:06:23.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.776+0000 7f7385200640 1 --2- 172.21.15.83:0/2011734989 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x55f30bcfbaa0 0x55f30bcfde90 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=be8044bdb443e985 server_cookie=66a45c4c46bf437f in_seq=0 out_seq=0 2024-04-24T09:06:23.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.777+0000 7f7367400640 1 -- 172.21.15.83:0/2011734989 <== mds.1 v2:172.21.15.138:6836/900566754 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f30bc53bf0 con 0x55f30bcfbaa0 2024-04-24T09:06:23.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.777+0000 7f7367400640 10 client.20841 handle_command_reply: tid=0 2024-04-24T09:06:23.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.777+0000 7f738b45a740 1 client.20841 shutdown 2024-04-24T09:06:23.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.777+0000 7f738b45a740 1 -- 172.21.15.83:0/2011734989 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f30bce0950 msgr2=0x55f30bceb350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:23.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.777+0000 7f738b45a740 1 --2- 172.21.15.83:0/2011734989 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f30bce0950 0x55f30bceb350 secure :-1 s=READY pgs=2182 cs=0 l=1 rev1=1 crypto rx=0x7f7378007090 tx=0x7f7378014c70 comp rx=0 tx=0).stop 2024-04-24T09:06:23.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.778+0000 7f738b45a740 1 -- 172.21.15.83:0/2011734989 shutdown_connections 2024-04-24T09:06:23.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.778+0000 7f738b45a740 1 --2- 172.21.15.83:0/2011734989 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x55f30bcfbaa0 0x55f30bcfde90 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:23.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.778+0000 7f738b45a740 1 --2- 172.21.15.83:0/2011734989 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f30bceb940 0x55f30bce8c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:23.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.778+0000 7f738b45a740 1 --2- 172.21.15.83:0/2011734989 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f30bce1270 0x55f30bce8640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:23.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.778+0000 7f738b45a740 1 --2- 172.21.15.83:0/2011734989 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f30bce0950 0x55f30bceb350 unknown :-1 s=CLOSED pgs=2182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:23.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.778+0000 7f738b45a740 1 -- 172.21.15.83:0/2011734989 >> 172.21.15.83:0/2011734989 conn(0x55f30bcdcc70 msgr2=0x55f30bce2950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:23.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.778+0000 7f7367400640 0 client.20841 ms_handle_reset on v2:172.21.15.138:6836/900566754 2024-04-24T09:06:23.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.778+0000 7f738b45a740 1 -- 172.21.15.83:0/2011734989 shutdown_connections 2024-04-24T09:06:23.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.778+0000 7f738b45a740 1 -- 172.21.15.83:0/2011734989 wait complete. 2024-04-24T09:06:23.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.779+0000 7f738b45a740 20 client.20841 trim_cache size 0 max 16384 2024-04-24T09:06:23.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.807+0000 7f738b45a740 1 Processor -- start 2024-04-24T09:06:23.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.807+0000 7f738b45a740 1 -- start start 2024-04-24T09:06:23.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.808+0000 7f738b45a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f30bce0950 0x55f30bcf7720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:23.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.808+0000 7f738b45a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f30bce1270 0x55f30bce7620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:23.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.808+0000 7f738b45a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f30bceb940 0x55f30bceae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:23.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.808+0000 7f7386600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f30bce0950 0x55f30bcf7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:23.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.809+0000 7f7386600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f30bce0950 0x55f30bcf7720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:43106/0 (socket says 172.21.15.83:43106) 2024-04-24T09:06:23.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.809+0000 7f7386600640 1 -- 172.21.15.83:0/359184391 learned_addr learned my addr 172.21.15.83:0/359184391 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:06:23.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.809+0000 7f738b45a740 1 -- 172.21.15.83:0/359184391 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7360010100 con 0x55f30bce1270 2024-04-24T09:06:23.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.809+0000 7f738b45a740 1 -- 172.21.15.83:0/359184391 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7360011dc0 con 0x55f30bceb940 2024-04-24T09:06:23.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.809+0000 7f738b45a740 1 -- 172.21.15.83:0/359184391 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f73600107d0 con 0x55f30bce0950 2024-04-24T09:06:23.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.809+0000 7f7386600640 1 --2- 172.21.15.83:0/359184391 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f30bceb940 0x55f30bceae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:23.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.809+0000 7f7385200640 1 --2- 172.21.15.83:0/359184391 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f30bce1270 0x55f30bce7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:23.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.809+0000 7f7386600640 1 -- 172.21.15.83:0/359184391 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f30bceb940 msgr2=0x55f30bceae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:23.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.809+0000 7f7386600640 1 --2- 172.21.15.83:0/359184391 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f30bceb940 0x55f30bceae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:23.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.809+0000 7f7386600640 1 -- 172.21.15.83:0/359184391 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f30bce1270 msgr2=0x55f30bce7620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:23.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.809+0000 7f7386600640 1 --2- 172.21.15.83:0/359184391 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f30bce1270 0x55f30bce7620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:23.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.809+0000 7f7386600640 1 -- 172.21.15.83:0/359184391 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f30ba227d0 con 0x55f30bce0950 2024-04-24T09:06:23.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.810+0000 7f7386600640 1 --2- 172.21.15.83:0/359184391 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f30bce0950 0x55f30bcf7720 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7378009af0 tx=0x7f73780021d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f724f481d9d10339 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:23.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.810+0000 7f7372000640 1 -- 172.21.15.83:0/359184391 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7378028bd0 con 0x55f30bce0950 2024-04-24T09:06:23.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.810+0000 7f7372000640 1 -- 172.21.15.83:0/359184391 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7378039070 con 0x55f30bce0950 2024-04-24T09:06:23.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.810+0000 7f7372000640 1 -- 172.21.15.83:0/359184391 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7378032420 con 0x55f30bce0950 2024-04-24T09:06:23.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.811+0000 7f738b45a740 1 -- 172.21.15.83:0/359184391 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f30bce0950 msgr2=0x55f30bcf7720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:23.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.811+0000 7f738b45a740 1 --2- 172.21.15.83:0/359184391 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f30bce0950 0x55f30bcf7720 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7378009af0 tx=0x7f73780021d0 comp rx=0 tx=0).stop 2024-04-24T09:06:23.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.812+0000 7f738b45a740 1 -- 172.21.15.83:0/359184391 shutdown_connections 2024-04-24T09:06:23.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.812+0000 7f738b45a740 1 --2- 172.21.15.83:0/359184391 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f30bceb940 0x55f30bceae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:23.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.812+0000 7f738b45a740 1 --2- 172.21.15.83:0/359184391 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f30bce1270 0x55f30bce7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:23.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.812+0000 7f738b45a740 1 --2- 172.21.15.83:0/359184391 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f30bce0950 0x55f30bcf7720 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:23.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.812+0000 7f738b45a740 1 -- 172.21.15.83:0/359184391 >> 172.21.15.83:0/359184391 conn(0x55f30bcdcc70 msgr2=0x55f30bce6f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:23.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.812+0000 7f738b45a740 1 -- 172.21.15.83:0/359184391 shutdown_connections 2024-04-24T09:06:23.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.813+0000 7f738b45a740 1 -- 172.21.15.83:0/359184391 wait complete. 2024-04-24T09:06:23.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.813+0000 7f738b45a740 1 -- start start 2024-04-24T09:06:23.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.813+0000 7f738b45a740 1 Processor -- start 2024-04-24T09:06:23.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.814+0000 7f738b45a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f30bce0950 0x55f30bcf3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:23.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.814+0000 7f738b45a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f30bce1270 0x55f30bce58c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:23.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.814+0000 7f738b45a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f30bceb940 0x55f30bce5e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:23.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.814+0000 7f738b45a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7360011630 con 0x55f30bce0950 2024-04-24T09:06:23.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.814+0000 7f738b45a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7360010f00 con 0x55f30bce1270 2024-04-24T09:06:23.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.814+0000 7f738b45a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f736000e100 con 0x55f30bceb940 2024-04-24T09:06:23.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.814+0000 7f738b45a740 10 client.0 _ll_register_callbacks cb 0x7f736c0095d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:06:23.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.814+0000 7f7386600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f30bce0950 0x55f30bcf3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:23.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.814+0000 7f7386600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f30bce0950 0x55f30bcf3e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55938/0 (socket says 172.21.15.83:55938) 2024-04-24T09:06:23.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.814+0000 7f7386600640 1 -- 172.21.15.83:0/2003448852 learned_addr learned my addr 172.21.15.83:0/2003448852 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:06:23.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.814+0000 7f7385200640 1 --2- 172.21.15.83:0/2003448852 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f30bce1270 0x55f30bce58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:23.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.815+0000 7f7386600640 1 -- 172.21.15.83:0/2003448852 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f30bceb940 msgr2=0x55f30bce5e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:23.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.815+0000 7f7386600640 1 --2- 172.21.15.83:0/2003448852 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f30bceb940 0x55f30bce5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:23.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.815+0000 7f7386600640 1 -- 172.21.15.83:0/2003448852 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f30bce1270 msgr2=0x55f30bce58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:23.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.815+0000 7f7386600640 1 --2- 172.21.15.83:0/2003448852 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f30bce1270 0x55f30bce58c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:23.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.815+0000 7f7386600640 1 -- 172.21.15.83:0/2003448852 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f30bce6340 con 0x55f30bce0950 2024-04-24T09:06:23.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.815+0000 7f7386600640 1 --2- 172.21.15.83:0/2003448852 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f30bce0950 0x55f30bcf3e20 secure :-1 s=READY pgs=2183 cs=0 l=1 rev1=1 crypto rx=0x7f7378014cb0 tx=0x7f7378009760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d3ff1a44d23b8c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:23.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.815+0000 7f7367400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:06:23.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.815+0000 7f7367400640 1 -- 172.21.15.83:0/2003448852 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7378042060 con 0x55f30bce0950 2024-04-24T09:06:23.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.816+0000 7f7367400640 1 -- 172.21.15.83:0/2003448852 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9583+0+0 (secure 0 0 0) 0x7f7378013bb0 con 0x55f30bce0950 2024-04-24T09:06:23.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.816+0000 7f738b45a740 1 -- 172.21.15.83:0/2003448852 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7378039070 con 0x55f30bce0950 2024-04-24T09:06:23.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.816+0000 7f7367400640 1 -- 172.21.15.83:0/2003448852 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f7378049050 con 0x55f30bce0950 2024-04-24T09:06:23.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.816+0000 7f738b45a740 10 client.20847 fetch_fsmap learned FSMap version 48 2024-04-24T09:06:23.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.816+0000 7f738b45a740 1 -- 172.21.15.83:0/2003448852 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55f30bcecec0 con 0x55f30bce0950 2024-04-24T09:06:23.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.817+0000 7f7367400640 1 -- 172.21.15.83:0/2003448852 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7378039070 con 0x55f30bce0950 2024-04-24T09:06:23.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.817+0000 7f7367400640 1 -- 172.21.15.83:0/2003448852 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 48) ==== 8402+0+0 (secure 0 0 0) 0x7f7378014ce0 con 0x55f30bce0950 2024-04-24T09:06:23.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.817+0000 7f738b45a740 10 client.20847 fetch_fsmap finished waiting for FSMap version 48 2024-04-24T09:06:23.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.817+0000 7f738b45a740 10 client.20847 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-24T09:06:23.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.817+0000 7f738b45a740 20 client.20847 populate_metadata read hostname 'smithi083' 2024-04-24T09:06:23.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.817+0000 7f738b45a740 1 --2- 172.21.15.83:0/2003448852 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x55f30bd621c0 0x55f30bd645e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:23.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.817+0000 7f738b45a740 4 client.20847 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:06:23.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.817+0000 7f738b45a740 1 -- 172.21.15.83:0/2003448852 --> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f30bc4da60 con 0x55f30bd621c0 2024-04-24T09:06:23.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.818+0000 7f7385200640 1 --2- 172.21.15.83:0/2003448852 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x55f30bd621c0 0x55f30bd645e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:23.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.818+0000 7f7385200640 1 --2- 172.21.15.83:0/2003448852 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x55f30bd621c0 0x55f30bd645e0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1fc1fbd571f16e71 server_cookie=1928ac791604216f in_seq=0 out_seq=0 2024-04-24T09:06:23.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.818+0000 7f7367400640 10 client.20847 ms_handle_connect on v2:172.21.15.138:6836/900566754 2024-04-24T09:06:23.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.819+0000 7f7367400640 1 -- 172.21.15.83:0/2003448852 <== mds.1 v2:172.21.15.138:6836/900566754 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f30bc4da60 con 0x55f30bd621c0 2024-04-24T09:06:23.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.819+0000 7f7367400640 10 client.20847 handle_command_reply: tid=0 2024-04-24T09:06:23.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.819+0000 7f738b45a740 1 client.20847 shutdown 2024-04-24T09:06:23.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.819+0000 7f738b45a740 1 -- 172.21.15.83:0/2003448852 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f30bce0950 msgr2=0x55f30bcf3e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:23.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.819+0000 7f738b45a740 1 --2- 172.21.15.83:0/2003448852 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f30bce0950 0x55f30bcf3e20 secure :-1 s=READY pgs=2183 cs=0 l=1 rev1=1 crypto rx=0x7f7378014cb0 tx=0x7f7378009760 comp rx=0 tx=0).stop 2024-04-24T09:06:23.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.820+0000 7f738b45a740 1 -- 172.21.15.83:0/2003448852 shutdown_connections 2024-04-24T09:06:23.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.820+0000 7f738b45a740 1 --2- 172.21.15.83:0/2003448852 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x55f30bd621c0 0x55f30bd645e0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:23.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.820+0000 7f738b45a740 1 --2- 172.21.15.83:0/2003448852 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f30bceb940 0x55f30bce5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:23.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.820+0000 7f738b45a740 1 --2- 172.21.15.83:0/2003448852 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f30bce1270 0x55f30bce58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:23.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.820+0000 7f738b45a740 1 --2- 172.21.15.83:0/2003448852 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f30bce0950 0x55f30bcf3e20 unknown :-1 s=CLOSED pgs=2183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:23.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.820+0000 7f7367400640 0 client.20847 ms_handle_reset on v2:172.21.15.138:6836/900566754 2024-04-24T09:06:23.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.820+0000 7f738b45a740 1 -- 172.21.15.83:0/2003448852 >> 172.21.15.83:0/2003448852 conn(0x55f30bcdcc70 msgr2=0x55f30bcdb560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:23.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.820+0000 7f738b45a740 1 -- 172.21.15.83:0/2003448852 shutdown_connections 2024-04-24T09:06:23.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.820+0000 7f738b45a740 1 -- 172.21.15.83:0/2003448852 wait complete. 2024-04-24T09:06:23.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.820+0000 7f738b45a740 20 client.20847 trim_cache size 0 max 16384 2024-04-24T09:06:23.821 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:06:23.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.822+0000 7f7388600640 1 -- 172.21.15.83:0/80249375 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f73680a3e60 msgr2=0x7f73680a6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:23.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.822+0000 7f7388600640 1 --2- 172.21.15.83:0/80249375 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f73680a3e60 0x7f73680a6320 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x55f30bce1f10 tx=0x7f7374009260 comp rx=0 tx=0).stop 2024-04-24T09:06:23.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.822+0000 7f7388600640 1 -- 172.21.15.83:0/80249375 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f73801044b0 msgr2=0x7f73801a0770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:23.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.822+0000 7f7388600640 1 --2- 172.21.15.83:0/80249375 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f73801044b0 0x7f73801a0770 secure :-1 s=READY pgs=2180 cs=0 l=1 rev1=1 crypto rx=0x7f7374005ab0 tx=0x7f7374003c20 comp rx=0 tx=0).stop 2024-04-24T09:06:23.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.822+0000 7f7388600640 1 -- 172.21.15.83:0/80249375 shutdown_connections 2024-04-24T09:06:23.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.822+0000 7f7388600640 1 --2- 172.21.15.83:0/80249375 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f73680a3e60 0x7f73680a6320 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:23.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.822+0000 7f7388600640 1 --2- 172.21.15.83:0/80249375 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f738019a970 0x7f738019ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:23.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.822+0000 7f7388600640 1 --2- 172.21.15.83:0/80249375 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7380109580 0x7f73801a0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:23.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.822+0000 7f7388600640 1 --2- 172.21.15.83:0/80249375 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f73801044b0 0x7f73801a0770 unknown :-1 s=CLOSED pgs=2180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:23.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.822+0000 7f7388600640 1 -- 172.21.15.83:0/80249375 >> 172.21.15.83:0/80249375 conn(0x7f73800ffd90 msgr2=0x7f73801010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:23.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.822+0000 7f7388600640 1 -- 172.21.15.83:0/80249375 shutdown_connections 2024-04-24T09:06:23.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:23.822+0000 7f7388600640 1 -- 172.21.15.83:0/80249375 wait complete. 2024-04-24T09:06:23.846 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T09:06:24.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.026+0000 7f787dc00640 1 -- 172.21.15.83:0/277937034 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78780742d0 msgr2=0x7f78780746b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:24.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.026+0000 7f787dc00640 1 --2- 172.21.15.83:0/277937034 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78780742d0 0x7f78780746b0 secure :-1 s=READY pgs=2184 cs=0 l=1 rev1=1 crypto rx=0x7f7868009900 tx=0x7f786802a480 comp rx=0 tx=0).stop 2024-04-24T09:06:24.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.026+0000 7f787dc00640 1 -- 172.21.15.83:0/277937034 shutdown_connections 2024-04-24T09:06:24.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.026+0000 7f787dc00640 1 --2- 172.21.15.83:0/277937034 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f787810e8e0 0x7f7878110cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:24.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.026+0000 7f787dc00640 1 --2- 172.21.15.83:0/277937034 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7878074bf0 0x7f787810e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:24.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.026+0000 7f787dc00640 1 --2- 172.21.15.83:0/277937034 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78780742d0 0x7f78780746b0 unknown :-1 s=CLOSED pgs=2184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:24.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.026+0000 7f787dc00640 1 -- 172.21.15.83:0/277937034 >> 172.21.15.83:0/277937034 conn(0x7f7878079450 msgr2=0x7f7878079860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:24.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.026+0000 7f787dc00640 1 -- 172.21.15.83:0/277937034 shutdown_connections 2024-04-24T09:06:24.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.026+0000 7f787dc00640 1 -- 172.21.15.83:0/277937034 wait complete. 2024-04-24T09:06:24.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.027+0000 7f787dc00640 1 Processor -- start 2024-04-24T09:06:24.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.027+0000 7f787dc00640 1 -- start start 2024-04-24T09:06:24.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.027+0000 7f787dc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78780742d0 0x7f78781a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:24.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.027+0000 7f787dc00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7878074bf0 0x7f78781a0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:24.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.027+0000 7f787dc00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f787810e8e0 0x7f787819a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:24.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.027+0000 7f787dc00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f78781144f0 con 0x7f78780742d0 2024-04-24T09:06:24.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.027+0000 7f787dc00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7878114370 con 0x7f7878074bf0 2024-04-24T09:06:24.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.027+0000 7f787dc00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7878114670 con 0x7f787810e8e0 2024-04-24T09:06:24.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.027+0000 7f7876a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78780742d0 0x7f78781a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:24.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.027+0000 7f7876a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78780742d0 0x7f78781a06c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55946/0 (socket says 172.21.15.83:55946) 2024-04-24T09:06:24.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.027+0000 7f7876a00640 1 -- 172.21.15.83:0/2893681165 learned_addr learned my addr 172.21.15.83:0/2893681165 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:06:24.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.028+0000 7f7876000640 1 --2- 172.21.15.83:0/2893681165 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7878074bf0 0x7f78781a0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:24.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.028+0000 7f7877400640 1 --2- 172.21.15.83:0/2893681165 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f787810e8e0 0x7f787819a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:24.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.028+0000 7f7876a00640 1 -- 172.21.15.83:0/2893681165 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f787810e8e0 msgr2=0x7f787819a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:24.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.028+0000 7f7876a00640 1 --2- 172.21.15.83:0/2893681165 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f787810e8e0 0x7f787819a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:24.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.028+0000 7f7876a00640 1 -- 172.21.15.83:0/2893681165 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7878074bf0 msgr2=0x7f78781a0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:24.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.028+0000 7f7876a00640 1 --2- 172.21.15.83:0/2893681165 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7878074bf0 0x7f78781a0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:24.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.028+0000 7f7876a00640 1 -- 172.21.15.83:0/2893681165 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f787819aff0 con 0x7f78780742d0 2024-04-24T09:06:24.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.028+0000 7f7876a00640 1 --2- 172.21.15.83:0/2893681165 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78780742d0 0x7f78781a06c0 secure :-1 s=READY pgs=2185 cs=0 l=1 rev1=1 crypto rx=0x7f7868005ab0 tx=0x7f7868003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:24.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.028+0000 7f786f400640 1 -- 172.21.15.83:0/2893681165 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f786803f070 con 0x7f78780742d0 2024-04-24T09:06:24.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.028+0000 7f786f400640 1 -- 172.21.15.83:0/2893681165 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7868031d60 con 0x7f78780742d0 2024-04-24T09:06:24.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.028+0000 7f786f400640 1 -- 172.21.15.83:0/2893681165 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7868035030 con 0x7f78780742d0 2024-04-24T09:06:24.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.028+0000 7f787dc00640 1 -- 172.21.15.83:0/2893681165 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f787819b280 con 0x7f78780742d0 2024-04-24T09:06:24.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.029+0000 7f787dc00640 1 -- 172.21.15.83:0/2893681165 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78781a7580 con 0x7f78780742d0 2024-04-24T09:06:24.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.031+0000 7f7880a2d740 1 Processor -- start 2024-04-24T09:06:24.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.032+0000 7f7880a2d740 1 -- start start 2024-04-24T09:06:24.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.032+0000 7f786f400640 1 -- 172.21.15.83:0/2893681165 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7868042070 con 0x7f78780742d0 2024-04-24T09:06:24.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.032+0000 7f7880a2d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a5f245950 0x563a5f245d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:24.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.032+0000 7f7880a2d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a5f246270 0x563a5f250400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:24.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.032+0000 7f7880a2d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563a5f250940 0x563a5f252d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:24.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.032+0000 7f7880a2d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x563a5f1ab1d0 con 0x563a5f246270 2024-04-24T09:06:24.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.032+0000 7f7880a2d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563a5f151450 con 0x563a5f245950 2024-04-24T09:06:24.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.032+0000 7f7880a2d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x563a5f1a7190 con 0x563a5f250940 2024-04-24T09:06:24.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.032+0000 7f7876a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a5f246270 0x563a5f250400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:24.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.032+0000 7f7876a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a5f246270 0x563a5f250400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55960/0 (socket says 172.21.15.83:55960) 2024-04-24T09:06:24.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.032+0000 7f7876a00640 1 -- 172.21.15.83:0/3914946164 learned_addr learned my addr 172.21.15.83:0/3914946164 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:06:24.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.032+0000 7f7877400640 1 --2- 172.21.15.83:0/3914946164 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a5f245950 0x563a5f245d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:24.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.033+0000 7f7876000640 1 --2- 172.21.15.83:0/3914946164 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563a5f250940 0x563a5f252d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:24.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.033+0000 7f7876a00640 1 -- 172.21.15.83:0/3914946164 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563a5f250940 msgr2=0x563a5f252d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:24.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.033+0000 7f7876a00640 1 --2- 172.21.15.83:0/3914946164 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563a5f250940 0x563a5f252d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:24.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.033+0000 7f7876a00640 1 -- 172.21.15.83:0/3914946164 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a5f245950 msgr2=0x563a5f245d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:24.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.033+0000 7f7876a00640 1 --2- 172.21.15.83:0/3914946164 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a5f245950 0x563a5f245d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:24.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.033+0000 7f7876a00640 1 -- 172.21.15.83:0/3914946164 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a5f20e310 con 0x563a5f246270 2024-04-24T09:06:24.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.033+0000 7f7877400640 1 --2- 172.21.15.83:0/3914946164 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a5f245950 0x563a5f245d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:06:24.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.033+0000 7f7876a00640 1 --2- 172.21.15.83:0/3914946164 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a5f246270 0x563a5f250400 secure :-1 s=READY pgs=2186 cs=0 l=1 rev1=1 crypto rx=0x7f78680ee830 tx=0x7f78680f7cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bfd7ba32d8d464 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:24.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.033+0000 7f786f400640 1 --2- 172.21.15.83:0/2893681165 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f785c0a3d20 0x7f785c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:24.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.033+0000 7f7863e00640 1 -- 172.21.15.83:0/3914946164 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7868100070 con 0x563a5f246270 2024-04-24T09:06:24.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.033+0000 7f786f400640 1 -- 172.21.15.83:0/2893681165 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9583+0+0 (secure 0 0 0) 0x7f78680f1590 con 0x7f78780742d0 2024-04-24T09:06:24.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.033+0000 7f7877400640 1 --2- 172.21.15.83:0/2893681165 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f785c0a3d20 0x7f785c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:24.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.033+0000 7f7863e00640 1 -- 172.21.15.83:0/3914946164 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f78680bd810 con 0x563a5f246270 2024-04-24T09:06:24.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.033+0000 7f7863e00640 1 -- 172.21.15.83:0/3914946164 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78680b93b0 con 0x563a5f246270 2024-04-24T09:06:24.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.034+0000 7f7880a2d740 1 -- 172.21.15.83:0/3914946164 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a5f246270 msgr2=0x563a5f250400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:24.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.034+0000 7f7877400640 1 --2- 172.21.15.83:0/2893681165 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f785c0a3d20 0x7f785c0a61e0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f787806b8d0 tx=0x7f78700093e0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:24.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.034+0000 7f7880a2d740 1 --2- 172.21.15.83:0/3914946164 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a5f246270 0x563a5f250400 secure :-1 s=READY pgs=2186 cs=0 l=1 rev1=1 crypto rx=0x7f78680ee830 tx=0x7f78680f7cd0 comp rx=0 tx=0).stop 2024-04-24T09:06:24.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.034+0000 7f7880a2d740 1 -- 172.21.15.83:0/3914946164 shutdown_connections 2024-04-24T09:06:24.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.034+0000 7f7880a2d740 1 --2- 172.21.15.83:0/3914946164 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563a5f250940 0x563a5f252d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:24.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.034+0000 7f7880a2d740 1 --2- 172.21.15.83:0/3914946164 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a5f246270 0x563a5f250400 unknown :-1 s=CLOSED pgs=2186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:24.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.034+0000 7f7880a2d740 1 --2- 172.21.15.83:0/3914946164 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a5f245950 0x563a5f245d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:24.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.034+0000 7f7880a2d740 1 -- 172.21.15.83:0/3914946164 >> 172.21.15.83:0/3914946164 conn(0x563a5f241c70 msgr2=0x563a5f242050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:24.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.034+0000 7f7880a2d740 1 -- 172.21.15.83:0/3914946164 shutdown_connections 2024-04-24T09:06:24.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.035+0000 7f7880a2d740 1 -- 172.21.15.83:0/3914946164 wait complete. 2024-04-24T09:06:24.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.035+0000 7f7880a2d740 1 -- start start 2024-04-24T09:06:24.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.035+0000 7f7880a2d740 1 Processor -- start 2024-04-24T09:06:24.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.035+0000 7f7880a2d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a5f245950 0x563a5f250310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:24.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.035+0000 7f7880a2d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563a5f246270 0x563a5f24d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:24.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.035+0000 7f7880a2d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a5f250940 0x563a5f24dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:24.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.035+0000 7f7880a2d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x563a5f2534f0 con 0x563a5f250940 2024-04-24T09:06:24.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.036+0000 7f7880a2d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563a5f164810 con 0x563a5f245950 2024-04-24T09:06:24.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.036+0000 7f7880a2d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x563a5f253670 con 0x563a5f246270 2024-04-24T09:06:24.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.036+0000 7f7880a2d740 10 client.0 _ll_register_callbacks cb 0x563a5f132140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:06:24.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.036+0000 7f7877400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a5f250940 0x563a5f24dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:24.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.036+0000 7f7877400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a5f250940 0x563a5f24dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55962/0 (socket says 172.21.15.83:55962) 2024-04-24T09:06:24.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.036+0000 7f7877400640 1 -- 172.21.15.83:0/3331197181 learned_addr learned my addr 172.21.15.83:0/3331197181 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:06:24.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.036+0000 7f7876a00640 1 --2- 172.21.15.83:0/3331197181 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a5f245950 0x563a5f250310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:24.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.036+0000 7f7876000640 1 --2- 172.21.15.83:0/3331197181 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563a5f246270 0x563a5f24d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:24.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.036+0000 7f7877400640 1 -- 172.21.15.83:0/3331197181 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563a5f246270 msgr2=0x563a5f24d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:24.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.036+0000 7f7877400640 1 --2- 172.21.15.83:0/3331197181 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563a5f246270 0x563a5f24d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:24.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.036+0000 7f7877400640 1 -- 172.21.15.83:0/3331197181 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a5f245950 msgr2=0x563a5f250310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:24.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.036+0000 7f7877400640 1 --2- 172.21.15.83:0/3331197181 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a5f245950 0x563a5f250310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:24.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.036+0000 7f7877400640 1 -- 172.21.15.83:0/3331197181 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a5f1753f0 con 0x563a5f250940 2024-04-24T09:06:24.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.036+0000 7f7876a00640 1 --2- 172.21.15.83:0/3331197181 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a5f245950 0x563a5f250310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:06:24.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.036+0000 7f7877400640 1 --2- 172.21.15.83:0/3331197181 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a5f250940 0x563a5f24dc10 secure :-1 s=READY pgs=2187 cs=0 l=1 rev1=1 crypto rx=0x7f78700127a0 tx=0x7f7870004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bd2eff8fbc6272b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:24.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.036+0000 7f7860c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:06:24.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.037+0000 7f7860c00640 1 -- 172.21.15.83:0/3331197181 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f787000fad0 con 0x563a5f250940 2024-04-24T09:06:24.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.037+0000 7f7860c00640 1 -- 172.21.15.83:0/3331197181 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9583+0+0 (secure 0 0 0) 0x7f787000fc70 con 0x563a5f250940 2024-04-24T09:06:24.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.037+0000 7f7880a2d740 1 -- 172.21.15.83:0/3331197181 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78680bd810 con 0x563a5f250940 2024-04-24T09:06:24.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.037+0000 7f7860c00640 1 -- 172.21.15.83:0/3331197181 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f7870030050 con 0x563a5f250940 2024-04-24T09:06:24.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.037+0000 7f7880a2d740 10 client.20859 fetch_fsmap learned FSMap version 48 2024-04-24T09:06:24.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.037+0000 7f7880a2d740 1 -- 172.21.15.83:0/3331197181 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x563a5eebd3f0 con 0x563a5f250940 2024-04-24T09:06:24.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.037+0000 7f7860c00640 1 -- 172.21.15.83:0/3331197181 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f78680bd810 con 0x563a5f250940 2024-04-24T09:06:24.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.037+0000 7f7860c00640 1 -- 172.21.15.83:0/3331197181 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 48) ==== 8402+0+0 (secure 0 0 0) 0x7f7870027400 con 0x563a5f250940 2024-04-24T09:06:24.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.037+0000 7f7880a2d740 10 client.20859 fetch_fsmap finished waiting for FSMap version 48 2024-04-24T09:06:24.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.038+0000 7f7880a2d740 10 client.20859 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:06:24.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.038+0000 7f7880a2d740 20 client.20859 populate_metadata read hostname 'smithi083' 2024-04-24T09:06:24.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.038+0000 7f7880a2d740 1 --2- 172.21.15.83:0/3331197181 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x563a5f260920 0x563a5f262dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:24.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.038+0000 7f7880a2d740 4 client.20859 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:06:24.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.038+0000 7f7880a2d740 1 -- 172.21.15.83:0/3331197181 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563a5f1b8bf0 con 0x563a5f260920 2024-04-24T09:06:24.157 INFO:tasks.workunit.client.0.smithi083.stdout: 10 7108 10.27 MB/sec warmup 55 sec latency 324.879 ms 2024-04-24T09:06:24.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.252+0000 7f7876a00640 1 --2- 172.21.15.83:0/3331197181 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x563a5f260920 0x563a5f262dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:24.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.254+0000 7f7876a00640 1 --2- 172.21.15.83:0/3331197181 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x563a5f260920 0x563a5f262dc0 crc :-1 s=READY pgs=377 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc7d1af200f77514 server_cookie=9edd8cd44b1de16d in_seq=0 out_seq=0 2024-04-24T09:06:24.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.254+0000 7f7860c00640 10 client.20859 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:06:24.263 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.263+0000 7fb5b9000640 1 -- 172.21.15.83:0/3381292149 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5b4102a30 msgr2=0x7fb5b4106a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:24.263 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.263+0000 7fb5b9000640 1 --2- 172.21.15.83:0/3381292149 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5b4102a30 0x7fb5b4106a70 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb59c009900 tx=0x7fb59c02a3f0 comp rx=0 tx=0).stop 2024-04-24T09:06:24.264 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.263+0000 7fb5b9000640 1 -- 172.21.15.83:0/3381292149 shutdown_connections 2024-04-24T09:06:24.264 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.263+0000 7fb5b9000640 1 --2- 172.21.15.83:0/3381292149 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5b4107150 0x7fb5b410ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:24.264 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.263+0000 7fb5b9000640 1 --2- 172.21.15.83:0/3381292149 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5b4102a30 0x7fb5b4106a70 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:24.264 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.263+0000 7fb5b9000640 1 --2- 172.21.15.83:0/3381292149 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb5b4102080 0x7fb5b4102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:24.264 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.263+0000 7fb5b9000640 1 -- 172.21.15.83:0/3381292149 >> 172.21.15.83:0/3381292149 conn(0x7fb5b40fd980 msgr2=0x7fb5b40ffda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:24.264 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.264+0000 7fb5b9000640 1 -- 172.21.15.83:0/3381292149 shutdown_connections 2024-04-24T09:06:24.264 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.264+0000 7fb5b9000640 1 -- 172.21.15.83:0/3381292149 wait complete. 2024-04-24T09:06:24.266 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.266+0000 7fb5b9000640 1 Processor -- start 2024-04-24T09:06:24.267 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.267+0000 7fb5b9000640 1 -- start start 2024-04-24T09:06:24.268 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.268+0000 7fb5b9000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5b4102080 0x7fb5b41a0440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:24.268 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.268+0000 7fb5b9000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb5b4107150 0x7fb5b41a0980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:24.268 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.268+0000 7fb5b9000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5b419a5f0 0x7fb5b419aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:24.268 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.268+0000 7fb5b9000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b41120e0 con 0x7fb5b419a5f0 2024-04-24T09:06:24.268 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.268+0000 7fb5b9000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b4111f60 con 0x7fb5b4102080 2024-04-24T09:06:24.268 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.268+0000 7fb5b9000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b4112260 con 0x7fb5b4107150 2024-04-24T09:06:24.270 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.268+0000 7fb5b3400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5b4102080 0x7fb5b41a0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:24.270 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.268+0000 7fb5b3400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5b4102080 0x7fb5b41a0440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:39000/0 (socket says 172.21.15.83:39000) 2024-04-24T09:06:24.270 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.268+0000 7fb5b3400640 1 -- 172.21.15.83:0/1090675069 learned_addr learned my addr 172.21.15.83:0/1090675069 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:06:24.270 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.268+0000 7fb5abe00640 1 --2- 172.21.15.83:0/1090675069 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb5b4107150 0x7fb5b41a0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:24.270 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.269+0000 7fb5b3e00640 1 --2- 172.21.15.83:0/1090675069 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5b419a5f0 0x7fb5b419aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:24.270 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.269+0000 7fb5b3400640 1 -- 172.21.15.83:0/1090675069 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb5b4107150 msgr2=0x7fb5b41a0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:24.271 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.269+0000 7fb5b3400640 1 --2- 172.21.15.83:0/1090675069 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb5b4107150 0x7fb5b41a0980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:24.271 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.269+0000 7fb5b3400640 1 -- 172.21.15.83:0/1090675069 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5b419a5f0 msgr2=0x7fb5b419aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:24.271 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.269+0000 7fb5b3400640 1 --2- 172.21.15.83:0/1090675069 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5b419a5f0 0x7fb5b419aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:24.271 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.269+0000 7fb5b3400640 1 -- 172.21.15.83:0/1090675069 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5b419b110 con 0x7fb5b4102080 2024-04-24T09:06:24.271 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.269+0000 7fb5b3e00640 1 --2- 172.21.15.83:0/1090675069 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5b419a5f0 0x7fb5b419aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:06:24.271 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.270+0000 7fb5b3400640 1 --2- 172.21.15.83:0/1090675069 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5b4102080 0x7fb5b41a0440 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb5a400e9f0 tx=0x7fb5a400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:24.271 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.270+0000 7fb5b0c00640 1 -- 172.21.15.83:0/1090675069 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5a400cd80 con 0x7fb5b4102080 2024-04-24T09:06:24.271 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.270+0000 7fb5b9000640 1 -- 172.21.15.83:0/1090675069 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5b419b400 con 0x7fb5b4102080 2024-04-24T09:06:24.271 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.270+0000 7fb5b0c00640 1 -- 172.21.15.83:0/1090675069 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb5a40054d0 con 0x7fb5b4102080 2024-04-24T09:06:24.271 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.270+0000 7fb5b0c00640 1 -- 172.21.15.83:0/1090675069 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5a4010490 con 0x7fb5b4102080 2024-04-24T09:06:24.271 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.270+0000 7fb5b9000640 1 -- 172.21.15.83:0/1090675069 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5b41a7650 con 0x7fb5b4102080 2024-04-24T09:06:24.275 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.275+0000 7fb5bd440740 1 Processor -- start 2024-04-24T09:06:24.275 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.275+0000 7fb5bd440740 1 -- start start 2024-04-24T09:06:24.276 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.276+0000 7fb5bd440740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac5480cbd0 0x55ac5480cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:24.276 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.276+0000 7fb5bd440740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ac5480d4f0 0x55ac54817870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:24.276 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.276+0000 7fb5bd440740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ac54817db0 0x55ac5481a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:24.276 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.276+0000 7fb5bd440740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ac5476a900 con 0x55ac5480cbd0 2024-04-24T09:06:24.276 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.276+0000 7fb5bd440740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ac54710f20 con 0x55ac54817db0 2024-04-24T09:06:24.276 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.276+0000 7fb5bd440740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55ac547668c0 con 0x55ac5480d4f0 2024-04-24T09:06:24.276 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.276+0000 7fb5b3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac5480cbd0 0x55ac5480cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:24.276 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.276+0000 7fb5b3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac5480cbd0 0x55ac5480cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56000/0 (socket says 172.21.15.83:56000) 2024-04-24T09:06:24.276 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.276+0000 7fb5b3e00640 1 -- 172.21.15.83:0/1436518461 learned_addr learned my addr 172.21.15.83:0/1436518461 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:06:24.277 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.276+0000 7fb5abe00640 1 --2- 172.21.15.83:0/1436518461 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ac54817db0 0x55ac5481a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:24.278 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.277+0000 7fb5b3e00640 1 -- 172.21.15.83:0/1436518461 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ac5480d4f0 msgr2=0x55ac54817870 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T09:06:24.278 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.277+0000 7fb5b3e00640 1 --2- 172.21.15.83:0/1436518461 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ac5480d4f0 0x55ac54817870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:24.278 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.277+0000 7fb5b3e00640 1 -- 172.21.15.83:0/1436518461 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ac54817db0 msgr2=0x55ac5481a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:24.278 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.277+0000 7fb5b3e00640 1 --2- 172.21.15.83:0/1436518461 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ac54817db0 0x55ac5481a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:24.278 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.277+0000 7fb5b3e00640 1 -- 172.21.15.83:0/1436518461 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac54735e10 con 0x55ac5480cbd0 2024-04-24T09:06:24.278 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.277+0000 7fb5b0c00640 1 -- 172.21.15.83:0/1090675069 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb5a4005020 con 0x7fb5b4102080 2024-04-24T09:06:24.278 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.277+0000 7fb5b3e00640 1 --2- 172.21.15.83:0/1436518461 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac5480cbd0 0x55ac5480cfb0 secure :-1 s=READY pgs=2188 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac007e60 tx=0x7fb5ac00e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59e1456841a6beb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:24.278 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.277+0000 7fb5a8c00640 1 -- 172.21.15.83:0/1436518461 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5ac014070 con 0x55ac5480cbd0 2024-04-24T09:06:24.278 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.277+0000 7fb5a8c00640 1 -- 172.21.15.83:0/1436518461 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb5ac003e20 con 0x55ac5480cbd0 2024-04-24T09:06:24.278 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.277+0000 7fb5a8c00640 1 -- 172.21.15.83:0/1436518461 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5ac004730 con 0x55ac5480cbd0 2024-04-24T09:06:24.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.278+0000 7f7860c00640 1 -- 172.21.15.83:0/3331197181 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563a5f1b8bf0 con 0x563a5f260920 2024-04-24T09:06:24.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.278+0000 7f7860c00640 10 client.20859 handle_command_reply: tid=0 2024-04-24T09:06:24.279 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.278+0000 7fb5bd440740 1 -- 172.21.15.83:0/1436518461 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac5480cbd0 msgr2=0x55ac5480cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:24.279 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.278+0000 7fb5bd440740 1 --2- 172.21.15.83:0/1436518461 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac5480cbd0 0x55ac5480cfb0 secure :-1 s=READY pgs=2188 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac007e60 tx=0x7fb5ac00e720 comp rx=0 tx=0).stop 2024-04-24T09:06:24.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.278+0000 7f7880a2d740 1 client.20859 shutdown 2024-04-24T09:06:24.280 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.279+0000 7fb5bd440740 1 -- 172.21.15.83:0/1436518461 shutdown_connections 2024-04-24T09:06:24.280 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.279+0000 7fb5bd440740 1 --2- 172.21.15.83:0/1436518461 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ac54817db0 0x55ac5481a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:24.280 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.279+0000 7fb5bd440740 1 --2- 172.21.15.83:0/1436518461 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ac5480d4f0 0x55ac54817870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:24.280 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.279+0000 7fb5bd440740 1 --2- 172.21.15.83:0/1436518461 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac5480cbd0 0x55ac5480cfb0 unknown :-1 s=CLOSED pgs=2188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:24.280 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.279+0000 7fb5bd440740 1 -- 172.21.15.83:0/1436518461 >> 172.21.15.83:0/1436518461 conn(0x55ac548090e0 msgr2=0x55ac548094c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:24.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.279+0000 7f7880a2d740 1 -- 172.21.15.83:0/3331197181 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a5f250940 msgr2=0x563a5f24dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:24.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.279+0000 7f7880a2d740 1 --2- 172.21.15.83:0/3331197181 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a5f250940 0x563a5f24dc10 secure :-1 s=READY pgs=2187 cs=0 l=1 rev1=1 crypto rx=0x7f78700127a0 tx=0x7f7870004a40 comp rx=0 tx=0).stop 2024-04-24T09:06:24.281 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.279+0000 7fb5bd440740 1 -- 172.21.15.83:0/1436518461 shutdown_connections 2024-04-24T09:06:24.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.279+0000 7f7880a2d740 1 -- 172.21.15.83:0/3331197181 shutdown_connections 2024-04-24T09:06:24.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.279+0000 7f7880a2d740 1 --2- 172.21.15.83:0/3331197181 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x563a5f260920 0x563a5f262dc0 crc :-1 s=READY pgs=377 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:24.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.279+0000 7f7880a2d740 1 --2- 172.21.15.83:0/3331197181 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a5f250940 0x563a5f24dc10 unknown :-1 s=CLOSED pgs=2187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:24.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.279+0000 7f7880a2d740 1 --2- 172.21.15.83:0/3331197181 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563a5f246270 0x563a5f24d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:24.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.279+0000 7f7880a2d740 1 --2- 172.21.15.83:0/3331197181 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a5f245950 0x563a5f250310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:24.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.279+0000 7f7880a2d740 1 -- 172.21.15.83:0/3331197181 >> 172.21.15.83:0/3331197181 conn(0x563a5f241c70 msgr2=0x563a5f247930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:24.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.279+0000 7f7860c00640 0 client.20859 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:06:24.283 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.279+0000 7fb5bd440740 1 -- 172.21.15.83:0/1436518461 wait complete. 2024-04-24T09:06:24.283 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.279+0000 7fb5b0c00640 1 --2- 172.21.15.83:0/1090675069 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb5980a3ca0 0x7fb5980a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:24.283 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.279+0000 7fb5b0c00640 1 -- 172.21.15.83:0/1090675069 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9583+0+0 (secure 0 0 0) 0x7fb5a40d1060 con 0x7fb5b4102080 2024-04-24T09:06:24.283 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.279+0000 7fb5bd440740 1 -- start start 2024-04-24T09:06:24.283 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.280+0000 7fb5bd440740 1 Processor -- start 2024-04-24T09:06:24.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.280+0000 7f7880a2d740 1 -- 172.21.15.83:0/3331197181 shutdown_connections 2024-04-24T09:06:24.283 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.281+0000 7fb5bd440740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ac5480cbd0 0x55ac54814e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:24.284 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.281+0000 7fb5bd440740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac5480d4f0 0x55ac54815350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:24.284 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.281+0000 7fb5bd440740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ac54817db0 0x55ac54815890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:24.284 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.281+0000 7fb5bd440740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ac5481a9e0 con 0x55ac5480d4f0 2024-04-24T09:06:24.284 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.281+0000 7fb5bd440740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ac547242c0 con 0x55ac54817db0 2024-04-24T09:06:24.284 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.281+0000 7fb5bd440740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55ac5481ab60 con 0x55ac5480cbd0 2024-04-24T09:06:24.284 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.281+0000 7fb5bd440740 10 client.0 _ll_register_callbacks cb 0x55ac5447ca00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:06:24.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.281+0000 7f7880a2d740 1 -- 172.21.15.83:0/3331197181 wait complete. 2024-04-24T09:06:24.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.281+0000 7f7880a2d740 20 client.20859 trim_cache size 0 max 16384 2024-04-24T09:06:24.287 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.283+0000 7fb5b3e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ac54817db0 0x55ac54815890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:24.287 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.283+0000 7fb5abe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac5480d4f0 0x55ac54815350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:24.287 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.283+0000 7fb5abe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac5480d4f0 0x55ac54815350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56012/0 (socket says 172.21.15.83:56012) 2024-04-24T09:06:24.287 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.283+0000 7fb5abe00640 1 -- 172.21.15.83:0/4261775292 learned_addr learned my addr 172.21.15.83:0/4261775292 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:06:24.287 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.283+0000 7fb5abe00640 1 --2- 172.21.15.83:0/1090675069 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb5980a3ca0 0x7fb5980a6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:24.287 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.284+0000 7fb5b3400640 1 --2- 172.21.15.83:0/4261775292 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ac5480cbd0 0x55ac54814e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:24.287 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.285+0000 7fb5abe00640 1 -- 172.21.15.83:0/4261775292 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ac5480cbd0 msgr2=0x55ac54814e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:24.287 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.285+0000 7fb5abe00640 1 --2- 172.21.15.83:0/4261775292 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ac5480cbd0 0x55ac54814e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:24.287 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.285+0000 7fb5abe00640 1 -- 172.21.15.83:0/4261775292 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ac54817db0 msgr2=0x55ac54815890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:24.287 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.285+0000 7fb5abe00640 1 --2- 172.21.15.83:0/4261775292 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ac54817db0 0x55ac54815890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:24.287 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.285+0000 7fb5abe00640 1 -- 172.21.15.83:0/4261775292 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac54734fd0 con 0x55ac5480d4f0 2024-04-24T09:06:24.288 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.288+0000 7fb5abe00640 1 --2- 172.21.15.83:0/4261775292 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac5480d4f0 0x55ac54815350 secure :-1 s=READY pgs=2189 cs=0 l=1 rev1=1 crypto rx=0x7fb59c005ab0 tx=0x7fb59c004a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0c9bc01158841f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:24.288 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.288+0000 7fb5a1600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:06:24.289 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.289+0000 7fb5abe00640 1 --2- 172.21.15.83:0/1090675069 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb5980a3ca0 0x7fb5980a6160 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fb59c000c00 tx=0x7fb59c042000 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:24.289 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.289+0000 7fb5a1600640 1 -- 172.21.15.83:0/4261775292 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb59c035040 con 0x55ac5480d4f0 2024-04-24T09:06:24.289 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.289+0000 7fb5a1600640 1 -- 172.21.15.83:0/4261775292 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9583+0+0 (secure 0 0 0) 0x7fb59c031a80 con 0x55ac5480d4f0 2024-04-24T09:06:24.289 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.289+0000 7fb5bd440740 1 -- 172.21.15.83:0/4261775292 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5ac003e20 con 0x55ac5480d4f0 2024-04-24T09:06:24.289 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.289+0000 7fb5a1600640 1 -- 172.21.15.83:0/4261775292 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fb59c004e50 con 0x55ac5480d4f0 2024-04-24T09:06:24.289 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.289+0000 7fb5bd440740 10 client.20868 fetch_fsmap learned FSMap version 48 2024-04-24T09:06:24.289 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.289+0000 7fb5bd440740 1 -- 172.21.15.83:0/4261775292 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55ac5447d210 con 0x55ac5480d4f0 2024-04-24T09:06:24.291 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.290+0000 7fb5a1600640 1 -- 172.21.15.83:0/4261775292 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb5ac003e20 con 0x55ac5480d4f0 2024-04-24T09:06:24.291 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.290+0000 7fb5a1600640 1 -- 172.21.15.83:0/4261775292 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 48) ==== 8402+0+0 (secure 0 0 0) 0x7fb59c003780 con 0x55ac5480d4f0 2024-04-24T09:06:24.291 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.290+0000 7fb5bd440740 10 client.20868 fetch_fsmap finished waiting for FSMap version 48 2024-04-24T09:06:24.291 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.290+0000 7fb5bd440740 10 client.20868 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:06:24.291 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.290+0000 7fb5bd440740 20 client.20868 populate_metadata read hostname 'smithi083' 2024-04-24T09:06:24.291 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.290+0000 7fb5bd440740 1 --2- 172.21.15.83:0/4261775292 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ac54828120 0x55ac5482a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:24.291 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.290+0000 7fb5bd440740 4 client.20868 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:06:24.291 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.290+0000 7fb5bd440740 1 -- 172.21.15.83:0/4261775292 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ac54778320 con 0x55ac54828120 2024-04-24T09:06:24.291 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.291+0000 7fb5b3400640 1 --2- 172.21.15.83:0/4261775292 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ac54828120 0x55ac5482a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:24.293 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.293+0000 7fb5b3400640 1 --2- 172.21.15.83:0/4261775292 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ac54828120 0x55ac5482a560 crc :-1 s=READY pgs=378 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b43eb1cf2f1f5a8f server_cookie=6adfa3fdd592174b in_seq=0 out_seq=0 2024-04-24T09:06:24.294 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.294+0000 7fb5a1600640 10 client.20868 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:06:24.296 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.296+0000 7fb5a1600640 1 -- 172.21.15.83:0/4261775292 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ac54778320 con 0x55ac54828120 2024-04-24T09:06:24.296 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.297+0000 7fb5a1600640 10 client.20868 handle_command_reply: tid=0 2024-04-24T09:06:24.297 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.297+0000 7fb5bd440740 1 client.20868 shutdown 2024-04-24T09:06:24.300 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.298+0000 7fb5bd440740 1 -- 172.21.15.83:0/4261775292 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac5480d4f0 msgr2=0x55ac54815350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:24.300 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.298+0000 7fb5bd440740 1 --2- 172.21.15.83:0/4261775292 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac5480d4f0 0x55ac54815350 secure :-1 s=READY pgs=2189 cs=0 l=1 rev1=1 crypto rx=0x7fb59c005ab0 tx=0x7fb59c004a00 comp rx=0 tx=0).stop 2024-04-24T09:06:24.300 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.300+0000 7fb5bd440740 1 -- 172.21.15.83:0/4261775292 shutdown_connections 2024-04-24T09:06:24.300 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.300+0000 7fb5bd440740 1 --2- 172.21.15.83:0/4261775292 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ac54828120 0x55ac5482a560 crc :-1 s=READY pgs=378 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:24.300 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.300+0000 7fb5bd440740 1 --2- 172.21.15.83:0/4261775292 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ac54817db0 0x55ac54815890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:24.300 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.300+0000 7fb5bd440740 1 --2- 172.21.15.83:0/4261775292 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac5480d4f0 0x55ac54815350 unknown :-1 s=CLOSED pgs=2189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:24.300 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.300+0000 7fb5bd440740 1 --2- 172.21.15.83:0/4261775292 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ac5480cbd0 0x55ac54814e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:24.300 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.300+0000 7fb5a1600640 0 client.20868 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:06:24.300 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.300+0000 7fb5bd440740 1 -- 172.21.15.83:0/4261775292 >> 172.21.15.83:0/4261775292 conn(0x55ac548090e0 msgr2=0x55ac547ffa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:24.301 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.301+0000 7fb5bd440740 1 -- 172.21.15.83:0/4261775292 shutdown_connections 2024-04-24T09:06:24.301 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.301+0000 7fb5bd440740 1 -- 172.21.15.83:0/4261775292 wait complete. 2024-04-24T09:06:24.301 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.301+0000 7fb5bd440740 20 client.20868 trim_cache size 0 max 16384 2024-04-24T09:06:24.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:06:24 smithi083 ceph-mon[27907]: pgmap v464: 129 pgs: 129 active+clean; 348 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.6 MiB/s wr, 265 op/s 2024-04-24T09:06:24.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.317+0000 7f7880a2d740 1 Processor -- start 2024-04-24T09:06:24.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.317+0000 7f7880a2d740 1 -- start start 2024-04-24T09:06:24.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.318+0000 7f7880a2d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563a5f245950 0x563a5f24fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:24.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.318+0000 7f7880a2d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a5f246270 0x563a5f240c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:24.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.318+0000 7f7880a2d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a5f250940 0x563a5f2411b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:24.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.318+0000 7f7880a2d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f78781b1860 con 0x563a5f246270 2024-04-24T09:06:24.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.318+0000 7f7880a2d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f78781b3520 con 0x563a5f250940 2024-04-24T09:06:24.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.318+0000 7f7880a2d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f78781b1f30 con 0x563a5f245950 2024-04-24T09:06:24.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.318+0000 7f7876a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563a5f245950 0x563a5f24fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:24.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.318+0000 7f7877400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a5f250940 0x563a5f2411b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:24.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.318+0000 7f7876a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563a5f245950 0x563a5f24fe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:43168/0 (socket says 172.21.15.83:43168) 2024-04-24T09:06:24.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.318+0000 7f7876a00640 1 -- 172.21.15.83:0/3674472280 learned_addr learned my addr 172.21.15.83:0/3674472280 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:06:24.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.319+0000 7f7876000640 1 --2- 172.21.15.83:0/3674472280 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a5f246270 0x563a5f240c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:24.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.319+0000 7f7876a00640 1 -- 172.21.15.83:0/3674472280 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a5f250940 msgr2=0x563a5f2411b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:24.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.319+0000 7f7876a00640 1 --2- 172.21.15.83:0/3674472280 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a5f250940 0x563a5f2411b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:24.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.319+0000 7f7876a00640 1 -- 172.21.15.83:0/3674472280 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a5f246270 msgr2=0x563a5f240c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:24.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.319+0000 7f7876a00640 1 --2- 172.21.15.83:0/3674472280 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a5f246270 0x563a5f240c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:24.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.319+0000 7f7876a00640 1 -- 172.21.15.83:0/3674472280 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a5ef877d0 con 0x563a5f245950 2024-04-24T09:06:24.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.319+0000 7f7876000640 1 --2- 172.21.15.83:0/3674472280 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a5f246270 0x563a5f240c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:06:24.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.319+0000 7f7876a00640 1 --2- 172.21.15.83:0/3674472280 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563a5f245950 0x563a5f24fe40 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f78680ee970 tx=0x7f7868005c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75c9b7abc54ddcc9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:24.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.319+0000 7f7863400640 1 -- 172.21.15.83:0/3674472280 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7868049040 con 0x563a5f245950 2024-04-24T09:06:24.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.320+0000 7f7863400640 1 -- 172.21.15.83:0/3674472280 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f786803be50 con 0x563a5f245950 2024-04-24T09:06:24.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.320+0000 7f7863400640 1 -- 172.21.15.83:0/3674472280 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7868100040 con 0x563a5f245950 2024-04-24T09:06:24.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.321+0000 7f7880a2d740 1 -- 172.21.15.83:0/3674472280 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563a5f245950 msgr2=0x563a5f24fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:24.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.321+0000 7f7880a2d740 1 --2- 172.21.15.83:0/3674472280 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563a5f245950 0x563a5f24fe40 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f78680ee970 tx=0x7f7868005c90 comp rx=0 tx=0).stop 2024-04-24T09:06:24.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.321+0000 7f7880a2d740 1 -- 172.21.15.83:0/3674472280 shutdown_connections 2024-04-24T09:06:24.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.321+0000 7f7880a2d740 1 --2- 172.21.15.83:0/3674472280 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a5f250940 0x563a5f2411b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:24.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.321+0000 7f7880a2d740 1 --2- 172.21.15.83:0/3674472280 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a5f246270 0x563a5f240c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:24.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.321+0000 7f7880a2d740 1 --2- 172.21.15.83:0/3674472280 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563a5f245950 0x563a5f24fe40 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:24.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.321+0000 7f7880a2d740 1 -- 172.21.15.83:0/3674472280 >> 172.21.15.83:0/3674472280 conn(0x563a5f241c70 msgr2=0x563a5f240560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:24.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.321+0000 7f7880a2d740 1 -- 172.21.15.83:0/3674472280 shutdown_connections 2024-04-24T09:06:24.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.321+0000 7f7880a2d740 1 -- 172.21.15.83:0/3674472280 wait complete. 2024-04-24T09:06:24.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.322+0000 7f7880a2d740 1 -- start start 2024-04-24T09:06:24.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.322+0000 7f7880a2d740 1 Processor -- start 2024-04-24T09:06:24.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.323+0000 7f7880a2d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a5f245950 0x563a5f251800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:24.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.323+0000 7f7880a2d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a5f246270 0x563a5f251d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:24.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.323+0000 7f7880a2d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563a5f250940 0x563a5f252280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:24.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.323+0000 7f7880a2d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f78781b2d90 con 0x563a5f245950 2024-04-24T09:06:24.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.323+0000 7f7880a2d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f78781b2660 con 0x563a5f246270 2024-04-24T09:06:24.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.323+0000 7f7880a2d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f78781af860 con 0x563a5f250940 2024-04-24T09:06:24.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.323+0000 7f7880a2d740 10 client.0 _ll_register_callbacks cb 0x7f7870009e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:06:24.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.323+0000 7f7876000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a5f246270 0x563a5f251d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:24.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.323+0000 7f7876a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a5f245950 0x563a5f251800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:24.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.323+0000 7f7876000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a5f246270 0x563a5f251d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:39036/0 (socket says 172.21.15.83:39036) 2024-04-24T09:06:24.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.323+0000 7f7876000640 1 -- 172.21.15.83:0/2010974968 learned_addr learned my addr 172.21.15.83:0/2010974968 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:06:24.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.323+0000 7f7877400640 1 --2- 172.21.15.83:0/2010974968 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563a5f250940 0x563a5f252280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:24.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.324+0000 7f7876a00640 1 -- 172.21.15.83:0/2010974968 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563a5f250940 msgr2=0x563a5f252280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:24.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.324+0000 7f7876a00640 1 --2- 172.21.15.83:0/2010974968 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563a5f250940 0x563a5f252280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:24.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.324+0000 7f7876a00640 1 -- 172.21.15.83:0/2010974968 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a5f246270 msgr2=0x563a5f251d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:24.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.324+0000 7f7876a00640 1 --2- 172.21.15.83:0/2010974968 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a5f246270 0x563a5f251d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:24.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.324+0000 7f7876a00640 1 -- 172.21.15.83:0/2010974968 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a5f256d40 con 0x563a5f245950 2024-04-24T09:06:24.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.324+0000 7f7876000640 1 --2- 172.21.15.83:0/2010974968 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a5f246270 0x563a5f251d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:06:24.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.324+0000 7f7876a00640 1 --2- 172.21.15.83:0/2010974968 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a5f245950 0x563a5f251800 secure :-1 s=READY pgs=2190 cs=0 l=1 rev1=1 crypto rx=0x7f78680bce10 tx=0x7f786802a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e3884a4a838f2f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:24.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.324+0000 7f7860c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:06:24.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.325+0000 7f7860c00640 1 -- 172.21.15.83:0/2010974968 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7868049040 con 0x563a5f245950 2024-04-24T09:06:24.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.325+0000 7f7880a2d740 1 -- 172.21.15.83:0/2010974968 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f786803be50 con 0x563a5f245950 2024-04-24T09:06:24.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.325+0000 7f7860c00640 1 -- 172.21.15.83:0/2010974968 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9583+0+0 (secure 0 0 0) 0x7f78680bb1b0 con 0x563a5f245950 2024-04-24T09:06:24.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.325+0000 7f7860c00640 1 -- 172.21.15.83:0/2010974968 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f7868049040 con 0x563a5f245950 2024-04-24T09:06:24.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.325+0000 7f7880a2d740 10 client.20871 fetch_fsmap learned FSMap version 48 2024-04-24T09:06:24.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.325+0000 7f7880a2d740 1 -- 172.21.15.83:0/2010974968 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x563a5f2c7260 con 0x563a5f245950 2024-04-24T09:06:24.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.326+0000 7f7860c00640 1 -- 172.21.15.83:0/2010974968 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f786803be50 con 0x563a5f245950 2024-04-24T09:06:24.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.326+0000 7f7860c00640 1 -- 172.21.15.83:0/2010974968 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 48) ==== 8402+0+0 (secure 0 0 0) 0x7f78680b9780 con 0x563a5f245950 2024-04-24T09:06:24.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.326+0000 7f7880a2d740 10 client.20871 fetch_fsmap finished waiting for FSMap version 48 2024-04-24T09:06:24.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.326+0000 7f7880a2d740 10 client.20871 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:06:24.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.326+0000 7f7880a2d740 20 client.20871 populate_metadata read hostname 'smithi083' 2024-04-24T09:06:24.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.326+0000 7f7880a2d740 1 --2- 172.21.15.83:0/2010974968 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x563a5f2c7680 0x563a5f2c9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:24.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.326+0000 7f7880a2d740 4 client.20871 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:06:24.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.326+0000 7f7880a2d740 1 -- 172.21.15.83:0/2010974968 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563a5f1b2a60 con 0x563a5f2c7680 2024-04-24T09:06:24.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.326+0000 7f7876000640 1 --2- 172.21.15.83:0/2010974968 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x563a5f2c7680 0x563a5f2c9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:24.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.327+0000 7f7860c00640 10 client.20871 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:06:24.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.327+0000 7f7876000640 1 --2- 172.21.15.83:0/2010974968 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x563a5f2c7680 0x563a5f2c9aa0 crc :-1 s=READY pgs=379 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e2603d6caf4d3c9 server_cookie=22d675b5d03a8dff in_seq=0 out_seq=0 2024-04-24T09:06:24.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.328+0000 7f7860c00640 1 -- 172.21.15.83:0/2010974968 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563a5f1b2a60 con 0x563a5f2c7680 2024-04-24T09:06:24.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.328+0000 7f7860c00640 10 client.20871 handle_command_reply: tid=0 2024-04-24T09:06:24.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.328+0000 7f7880a2d740 1 client.20871 shutdown 2024-04-24T09:06:24.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.328+0000 7f7880a2d740 1 -- 172.21.15.83:0/2010974968 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a5f245950 msgr2=0x563a5f251800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:24.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.328+0000 7f7880a2d740 1 --2- 172.21.15.83:0/2010974968 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a5f245950 0x563a5f251800 secure :-1 s=READY pgs=2190 cs=0 l=1 rev1=1 crypto rx=0x7f78680bce10 tx=0x7f786802a480 comp rx=0 tx=0).stop 2024-04-24T09:06:24.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.328+0000 7f7880a2d740 1 -- 172.21.15.83:0/2010974968 shutdown_connections 2024-04-24T09:06:24.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.328+0000 7f7880a2d740 1 --2- 172.21.15.83:0/2010974968 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x563a5f2c7680 0x563a5f2c9aa0 crc :-1 s=READY pgs=379 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:24.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.328+0000 7f7880a2d740 1 --2- 172.21.15.83:0/2010974968 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563a5f250940 0x563a5f252280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:24.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.329+0000 7f7860c00640 0 client.20871 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:06:24.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.329+0000 7f7880a2d740 1 --2- 172.21.15.83:0/2010974968 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a5f246270 0x563a5f251d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:24.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.329+0000 7f7880a2d740 1 --2- 172.21.15.83:0/2010974968 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a5f245950 0x563a5f251800 unknown :-1 s=CLOSED pgs=2190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:24.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.329+0000 7f7880a2d740 1 -- 172.21.15.83:0/2010974968 >> 172.21.15.83:0/2010974968 conn(0x563a5f241c70 msgr2=0x563a5f240df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:24.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.329+0000 7f7880a2d740 1 -- 172.21.15.83:0/2010974968 shutdown_connections 2024-04-24T09:06:24.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.329+0000 7f7880a2d740 1 -- 172.21.15.83:0/2010974968 wait complete. 2024-04-24T09:06:24.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.330+0000 7f7880a2d740 20 client.20871 trim_cache size 0 max 16384 2024-04-24T09:06:24.332 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:06:24.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.331+0000 7f787dc00640 1 -- 172.21.15.83:0/2893681165 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f785c0a3d20 msgr2=0x7f785c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:24.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.331+0000 7f787dc00640 1 --2- 172.21.15.83:0/2893681165 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f785c0a3d20 0x7f785c0a61e0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f787806b8d0 tx=0x7f78700093e0 comp rx=0 tx=0).stop 2024-04-24T09:06:24.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.331+0000 7f787dc00640 1 -- 172.21.15.83:0/2893681165 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78780742d0 msgr2=0x7f78781a06c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:24.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.331+0000 7f787dc00640 1 --2- 172.21.15.83:0/2893681165 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78780742d0 0x7f78781a06c0 secure :-1 s=READY pgs=2185 cs=0 l=1 rev1=1 crypto rx=0x7f7868005ab0 tx=0x7f7868003cd0 comp rx=0 tx=0).stop 2024-04-24T09:06:24.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.331+0000 7f787dc00640 1 -- 172.21.15.83:0/2893681165 shutdown_connections 2024-04-24T09:06:24.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.331+0000 7f787dc00640 1 --2- 172.21.15.83:0/2893681165 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f785c0a3d20 0x7f785c0a61e0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:24.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.331+0000 7f787dc00640 1 --2- 172.21.15.83:0/2893681165 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f787810e8e0 0x7f787819a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:24.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.331+0000 7f787dc00640 1 --2- 172.21.15.83:0/2893681165 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7878074bf0 0x7f78781a0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:24.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.331+0000 7f787dc00640 1 --2- 172.21.15.83:0/2893681165 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78780742d0 0x7f78781a06c0 unknown :-1 s=CLOSED pgs=2185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:24.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.331+0000 7f787dc00640 1 -- 172.21.15.83:0/2893681165 >> 172.21.15.83:0/2893681165 conn(0x7f7878079450 msgr2=0x7f78780fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:24.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.332+0000 7f787dc00640 1 -- 172.21.15.83:0/2893681165 shutdown_connections 2024-04-24T09:06:24.334 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.332+0000 7fb5bd440740 1 Processor -- start 2024-04-24T09:06:24.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:24.332+0000 7f787dc00640 1 -- 172.21.15.83:0/2893681165 wait complete. 2024-04-24T09:06:24.335 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.332+0000 7fb5bd440740 1 -- start start 2024-04-24T09:06:24.335 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.332+0000 7fb5bd440740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ac5480cbd0 0x55ac548238a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:24.335 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.332+0000 7fb5bd440740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ac5480d4f0 0x55ac548156b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:24.336 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.332+0000 7fb5bd440740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac54817db0 0x55ac5482fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:24.336 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.332+0000 7fb5bd440740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b41ad1a0 con 0x55ac54817db0 2024-04-24T09:06:24.336 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.332+0000 7fb5bd440740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b41aee60 con 0x55ac5480d4f0 2024-04-24T09:06:24.336 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.332+0000 7fb5bd440740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b41ad870 con 0x55ac5480cbd0 2024-04-24T09:06:24.336 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.333+0000 7fb5abe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ac5480d4f0 0x55ac548156b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:24.336 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.333+0000 7fb5abe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ac5480d4f0 0x55ac548156b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:39052/0 (socket says 172.21.15.83:39052) 2024-04-24T09:06:24.336 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.333+0000 7fb5abe00640 1 -- 172.21.15.83:0/1715350344 learned_addr learned my addr 172.21.15.83:0/1715350344 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:06:24.336 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.333+0000 7fb5b3400640 1 --2- 172.21.15.83:0/1715350344 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ac5480cbd0 0x55ac548238a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:24.336 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.333+0000 7fb5b3e00640 1 --2- 172.21.15.83:0/1715350344 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac54817db0 0x55ac5482fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:24.336 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.333+0000 7fb5abe00640 1 -- 172.21.15.83:0/1715350344 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ac5480cbd0 msgr2=0x55ac548238a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:24.336 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.333+0000 7fb5abe00640 1 --2- 172.21.15.83:0/1715350344 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ac5480cbd0 0x55ac548238a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:24.336 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.333+0000 7fb5abe00640 1 -- 172.21.15.83:0/1715350344 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac54817db0 msgr2=0x55ac5482fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:24.336 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.333+0000 7fb5abe00640 1 --2- 172.21.15.83:0/1715350344 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac54817db0 0x55ac5482fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:24.336 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.333+0000 7fb5abe00640 1 -- 172.21.15.83:0/1715350344 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac54549620 con 0x55ac5480d4f0 2024-04-24T09:06:24.337 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.333+0000 7fb5b3400640 1 --2- 172.21.15.83:0/1715350344 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ac5480cbd0 0x55ac548238a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:06:24.337 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.333+0000 7fb5b3e00640 1 --2- 172.21.15.83:0/1715350344 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac54817db0 0x55ac5482fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:06:24.337 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.334+0000 7fb5abe00640 1 --2- 172.21.15.83:0/1715350344 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ac5480d4f0 0x55ac548156b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb59c04e030 tx=0x7fb59c005610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=259350233e97bcf5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:24.337 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.334+0000 7fb5a3e00640 1 -- 172.21.15.83:0/1715350344 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb59c0037f0 con 0x55ac5480d4f0 2024-04-24T09:06:24.337 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.335+0000 7fb5a3e00640 1 -- 172.21.15.83:0/1715350344 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb59c004e50 con 0x55ac5480d4f0 2024-04-24T09:06:24.337 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.335+0000 7fb5a3e00640 1 -- 172.21.15.83:0/1715350344 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb59c0041e0 con 0x55ac5480d4f0 2024-04-24T09:06:24.338 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.336+0000 7fb5bd440740 1 -- 172.21.15.83:0/1715350344 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ac5480d4f0 msgr2=0x55ac548156b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:24.338 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.336+0000 7fb5bd440740 1 --2- 172.21.15.83:0/1715350344 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ac5480d4f0 0x55ac548156b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb59c04e030 tx=0x7fb59c005610 comp rx=0 tx=0).stop 2024-04-24T09:06:24.338 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.336+0000 7fb5bd440740 1 -- 172.21.15.83:0/1715350344 shutdown_connections 2024-04-24T09:06:24.338 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.336+0000 7fb5bd440740 1 --2- 172.21.15.83:0/1715350344 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac54817db0 0x55ac5482fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:24.338 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.336+0000 7fb5bd440740 1 --2- 172.21.15.83:0/1715350344 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ac5480d4f0 0x55ac548156b0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:24.338 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.336+0000 7fb5bd440740 1 --2- 172.21.15.83:0/1715350344 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ac5480cbd0 0x55ac548238a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:24.338 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.336+0000 7fb5bd440740 1 -- 172.21.15.83:0/1715350344 >> 172.21.15.83:0/1715350344 conn(0x55ac548090e0 msgr2=0x55ac547ffa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:24.338 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.337+0000 7fb5bd440740 1 -- 172.21.15.83:0/1715350344 shutdown_connections 2024-04-24T09:06:24.338 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.337+0000 7fb5bd440740 1 -- 172.21.15.83:0/1715350344 wait complete. 2024-04-24T09:06:24.338 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.338+0000 7fb5bd440740 1 -- start start 2024-04-24T09:06:24.339 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.338+0000 7fb5bd440740 1 Processor -- start 2024-04-24T09:06:24.339 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.339+0000 7fb5bd440740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ac5480cbd0 0x55ac54889f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:24.339 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.339+0000 7fb5bd440740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac5480d4f0 0x55ac5481fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:24.339 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.339+0000 7fb5bd440740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ac54817db0 0x55ac548202f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:24.339 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.339+0000 7fb5bd440740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b41ae6d0 con 0x55ac5480d4f0 2024-04-24T09:06:24.340 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.339+0000 7fb5bd440740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b41adfa0 con 0x55ac5480cbd0 2024-04-24T09:06:24.340 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.339+0000 7fb5bd440740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b41ab1c0 con 0x55ac54817db0 2024-04-24T09:06:24.340 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.339+0000 7fb5bd440740 10 client.0 _ll_register_callbacks cb 0x7fb59c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:06:24.340 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.339+0000 7fb5b3e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ac54817db0 0x55ac548202f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:24.340 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.340+0000 7fb5b3e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ac54817db0 0x55ac548202f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:43192/0 (socket says 172.21.15.83:43192) 2024-04-24T09:06:24.340 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.340+0000 7fb5b3e00640 1 -- 172.21.15.83:0/3816092562 learned_addr learned my addr 172.21.15.83:0/3816092562 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:06:24.341 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.340+0000 7fb5b3e00640 1 -- 172.21.15.83:0/3816092562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ac5480cbd0 msgr2=0x55ac54889f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:24.341 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.340+0000 7fb5b3e00640 1 --2- 172.21.15.83:0/3816092562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ac5480cbd0 0x55ac54889f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:24.341 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.340+0000 7fb5b3e00640 1 -- 172.21.15.83:0/3816092562 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac5480d4f0 msgr2=0x55ac5481fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:24.341 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.340+0000 7fb5b3e00640 1 --2- 172.21.15.83:0/3816092562 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac5480d4f0 0x55ac5481fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:24.341 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.340+0000 7fb5b3e00640 1 -- 172.21.15.83:0/3816092562 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac54811f30 con 0x55ac54817db0 2024-04-24T09:06:24.341 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.341+0000 7fb5b3e00640 1 --2- 172.21.15.83:0/3816092562 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ac54817db0 0x55ac548202f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac007e60 tx=0x7fb5ac007830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8016db69b8e4b7ed server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:24.341 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.341+0000 7fb5a1600640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-04-24T09:06:24.342 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.341+0000 7fb5a1600640 1 -- 172.21.15.83:0/3816092562 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5ac026020 con 0x55ac54817db0 2024-04-24T09:06:24.342 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.341+0000 7fb5bd440740 1 -- 172.21.15.83:0/3816092562 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb59c004e50 con 0x55ac54817db0 2024-04-24T09:06:24.342 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.341+0000 7fb5a1600640 1 -- 172.21.15.83:0/3816092562 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9583+0+0 (secure 0 0 0) 0x7fb5ac003d80 con 0x55ac54817db0 2024-04-24T09:06:24.342 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.341+0000 7fb5a1600640 1 -- 172.21.15.83:0/3816092562 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fb5ac031050 con 0x55ac54817db0 2024-04-24T09:06:24.342 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.341+0000 7fb5bd440740 10 client.28150 fetch_fsmap learned FSMap version 48 2024-04-24T09:06:24.342 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.341+0000 7fb5bd440740 1 -- 172.21.15.83:0/3816092562 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55ac5489fbe0 con 0x55ac54817db0 2024-04-24T09:06:24.342 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.342+0000 7fb5a1600640 1 -- 172.21.15.83:0/3816092562 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb59c004e50 con 0x55ac54817db0 2024-04-24T09:06:24.343 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.342+0000 7fb5a1600640 1 -- 172.21.15.83:0/3816092562 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 48) ==== 8402+0+0 (secure 0 0 0) 0x7fb5ac018410 con 0x55ac54817db0 2024-04-24T09:06:24.343 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.342+0000 7fb5bd440740 10 client.28150 fetch_fsmap finished waiting for FSMap version 48 2024-04-24T09:06:24.343 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.342+0000 7fb5bd440740 10 client.28150 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:06:24.343 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.342+0000 7fb5bd440740 20 client.28150 populate_metadata read hostname 'smithi083' 2024-04-24T09:06:24.343 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.342+0000 7fb5bd440740 1 --2- 172.21.15.83:0/3816092562 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ac548a0150 0x55ac548a2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:24.343 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.342+0000 7fb5bd440740 4 client.28150 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T09:06:24.343 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.342+0000 7fb5bd440740 1 -- 172.21.15.83:0/3816092562 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ac54772190 con 0x55ac548a0150 2024-04-24T09:06:24.343 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.343+0000 7fb5b3400640 1 --2- 172.21.15.83:0/3816092562 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ac548a0150 0x55ac548a2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:24.344 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.343+0000 7fb5b3400640 1 --2- 172.21.15.83:0/3816092562 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ac548a0150 0x55ac548a2590 crc :-1 s=READY pgs=380 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f0100aa79a3df6d server_cookie=d38b32ba613582e4 in_seq=0 out_seq=0 2024-04-24T09:06:24.344 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.343+0000 7fb5a1600640 10 client.28150 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:06:24.425 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.425+0000 7fb5a1600640 1 -- 172.21.15.83:0/3816092562 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ac54772190 con 0x55ac548a0150 2024-04-24T09:06:24.425 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.425+0000 7fb5a1600640 10 client.28150 handle_command_reply: tid=0 2024-04-24T09:06:24.425 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.425+0000 7fb5bd440740 1 client.28150 shutdown 2024-04-24T09:06:24.425 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.425+0000 7fb5bd440740 1 -- 172.21.15.83:0/3816092562 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ac54817db0 msgr2=0x55ac548202f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:24.425 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.425+0000 7fb5bd440740 1 --2- 172.21.15.83:0/3816092562 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ac54817db0 0x55ac548202f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac007e60 tx=0x7fb5ac007830 comp rx=0 tx=0).stop 2024-04-24T09:06:24.425 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.425+0000 7fb5bd440740 1 -- 172.21.15.83:0/3816092562 shutdown_connections 2024-04-24T09:06:24.425 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.425+0000 7fb5bd440740 1 --2- 172.21.15.83:0/3816092562 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ac548a0150 0x55ac548a2590 crc :-1 s=READY pgs=380 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:24.426 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.425+0000 7fb5bd440740 1 --2- 172.21.15.83:0/3816092562 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ac54817db0 0x55ac548202f0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:24.426 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.425+0000 7fb5a1600640 0 client.28150 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:06:24.427 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.425+0000 7fb5bd440740 1 --2- 172.21.15.83:0/3816092562 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac5480d4f0 0x55ac5481fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:24.427 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.425+0000 7fb5bd440740 1 --2- 172.21.15.83:0/3816092562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ac5480cbd0 0x55ac54889f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:24.427 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.425+0000 7fb5bd440740 1 -- 172.21.15.83:0/3816092562 >> 172.21.15.83:0/3816092562 conn(0x55ac548090e0 msgr2=0x55ac5480e580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:24.427 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.426+0000 7fb5bd440740 1 -- 172.21.15.83:0/3816092562 shutdown_connections 2024-04-24T09:06:24.427 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.426+0000 7fb5bd440740 1 -- 172.21.15.83:0/3816092562 wait complete. 2024-04-24T09:06:24.427 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.426+0000 7fb5bd440740 20 client.28150 trim_cache size 0 max 16384 2024-04-24T09:06:24.427 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T09:06:24.427 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T09:06:24.427 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T09:06:24.427 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T09:06:24.428 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.427+0000 7fb5b9000640 1 -- 172.21.15.83:0/1090675069 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb5980a3ca0 msgr2=0x7fb5980a6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:24.428 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.427+0000 7fb5b9000640 1 --2- 172.21.15.83:0/1090675069 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb5980a3ca0 0x7fb5980a6160 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fb59c000c00 tx=0x7fb59c042000 comp rx=0 tx=0).stop 2024-04-24T09:06:24.428 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.427+0000 7fb5b9000640 1 -- 172.21.15.83:0/1090675069 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5b4102080 msgr2=0x7fb5b41a0440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:24.428 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.427+0000 7fb5b9000640 1 --2- 172.21.15.83:0/1090675069 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5b4102080 0x7fb5b41a0440 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb5a400e9f0 tx=0x7fb5a400edf0 comp rx=0 tx=0).stop 2024-04-24T09:06:24.428 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.427+0000 7fb5b9000640 1 -- 172.21.15.83:0/1090675069 shutdown_connections 2024-04-24T09:06:24.428 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.427+0000 7fb5b9000640 1 --2- 172.21.15.83:0/1090675069 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb5980a3ca0 0x7fb5980a6160 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:24.428 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.427+0000 7fb5b9000640 1 --2- 172.21.15.83:0/1090675069 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5b419a5f0 0x7fb5b419aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:24.429 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.427+0000 7fb5b9000640 1 --2- 172.21.15.83:0/1090675069 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb5b4107150 0x7fb5b41a0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:24.429 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.427+0000 7fb5b9000640 1 --2- 172.21.15.83:0/1090675069 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5b4102080 0x7fb5b41a0440 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:24.429 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.427+0000 7fb5b9000640 1 -- 172.21.15.83:0/1090675069 >> 172.21.15.83:0/1090675069 conn(0x7fb5b40fd980 msgr2=0x7fb5b40fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:24.429 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.427+0000 7fb5b9000640 1 -- 172.21.15.83:0/1090675069 shutdown_connections 2024-04-24T09:06:24.429 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:24.427+0000 7fb5b9000640 1 -- 172.21.15.83:0/1090675069 wait complete. 2024-04-24T09:06:24.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:06:24 smithi138 ceph-mon[31441]: pgmap v464: 129 pgs: 129 active+clean; 348 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.6 MiB/s wr, 265 op/s 2024-04-24T09:06:24.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:06:24 smithi189 ceph-mon[31791]: pgmap v464: 129 pgs: 129 active+clean; 348 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.6 MiB/s wr, 265 op/s 2024-04-24T09:06:25.156 INFO:tasks.workunit.client.0.smithi083.stdout: 10 7230 10.17 MB/sec warmup 56 sec latency 494.680 ms 2024-04-24T09:06:25.354 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T09:06:25.354 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T09:06:25.355 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T09:06:25.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.604+0000 7f88f5a00640 1 -- 172.21.15.83:0/2606829305 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88f01015f0 msgr2=0x7f88f010e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:25.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.604+0000 7f88f5a00640 1 --2- 172.21.15.83:0/2606829305 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88f01015f0 0x7f88f010e2b0 secure :-1 s=READY pgs=2191 cs=0 l=1 rev1=1 crypto rx=0x7f88dc009900 tx=0x7f88dc02a3f0 comp rx=0 tx=0).stop 2024-04-24T09:06:25.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.605+0000 7f88f5a00640 1 -- 172.21.15.83:0/2606829305 shutdown_connections 2024-04-24T09:06:25.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.605+0000 7f88f5a00640 1 --2- 172.21.15.83:0/2606829305 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88f010e7f0 0x7f88f0110be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:25.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.605+0000 7f88f5a00640 1 --2- 172.21.15.83:0/2606829305 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88f01015f0 0x7f88f010e2b0 unknown :-1 s=CLOSED pgs=2191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:25.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.605+0000 7f88f5a00640 1 --2- 172.21.15.83:0/2606829305 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f88f0100c40 0x7f88f0101020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:25.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.605+0000 7f88f5a00640 1 -- 172.21.15.83:0/2606829305 >> 172.21.15.83:0/2606829305 conn(0x7f88f00fc880 msgr2=0x7f88f00feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:25.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.605+0000 7f88f5a00640 1 -- 172.21.15.83:0/2606829305 shutdown_connections 2024-04-24T09:06:25.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.605+0000 7f88f5a00640 1 -- 172.21.15.83:0/2606829305 wait complete. 2024-04-24T09:06:25.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.607+0000 7f88f5a00640 1 Processor -- start 2024-04-24T09:06:25.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.607+0000 7f88f5a00640 1 -- start start 2024-04-24T09:06:25.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.607+0000 7f88f5a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88f0100c40 0x7f88f01a05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:25.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.607+0000 7f88f5a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88f01015f0 0x7f88f01a0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:25.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.607+0000 7f88f5a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f88f010e7f0 0x7f88f019a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:25.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.607+0000 7f88f5a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f88f0114370 con 0x7f88f0100c40 2024-04-24T09:06:25.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.607+0000 7f88f5a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f88f01141f0 con 0x7f88f01015f0 2024-04-24T09:06:25.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.607+0000 7f88f5a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f88f01144f0 con 0x7f88f010e7f0 2024-04-24T09:06:25.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.608+0000 7f88efe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88f0100c40 0x7f88f01a05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:25.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.608+0000 7f88ef400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88f01015f0 0x7f88f01a0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:25.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.608+0000 7f88efe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88f0100c40 0x7f88f01a05d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56074/0 (socket says 172.21.15.83:56074) 2024-04-24T09:06:25.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.608+0000 7f88efe00640 1 -- 172.21.15.83:0/860644664 learned_addr learned my addr 172.21.15.83:0/860644664 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:06:25.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.608+0000 7f88f5000640 1 --2- 172.21.15.83:0/860644664 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f88f010e7f0 0x7f88f019a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:25.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.608+0000 7f88efe00640 1 -- 172.21.15.83:0/860644664 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f88f010e7f0 msgr2=0x7f88f019a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:25.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.608+0000 7f88efe00640 1 --2- 172.21.15.83:0/860644664 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f88f010e7f0 0x7f88f019a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:25.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.608+0000 7f88efe00640 1 -- 172.21.15.83:0/860644664 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88f01015f0 msgr2=0x7f88f01a0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:25.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.608+0000 7f88efe00640 1 --2- 172.21.15.83:0/860644664 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88f01015f0 0x7f88f01a0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:25.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.608+0000 7f88efe00640 1 -- 172.21.15.83:0/860644664 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88f019af50 con 0x7f88f0100c40 2024-04-24T09:06:25.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.609+0000 7f88efe00640 1 --2- 172.21.15.83:0/860644664 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88f0100c40 0x7f88f01a05d0 secure :-1 s=READY pgs=2192 cs=0 l=1 rev1=1 crypto rx=0x7f88e800cce0 tx=0x7f88e800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:25.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.609+0000 7f88ecc00640 1 -- 172.21.15.83:0/860644664 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88e800f650 con 0x7f88f0100c40 2024-04-24T09:06:25.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.609+0000 7f88f5a00640 1 -- 172.21.15.83:0/860644664 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88f019b240 con 0x7f88f0100c40 2024-04-24T09:06:25.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.609+0000 7f88ecc00640 1 -- 172.21.15.83:0/860644664 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f88e800fca0 con 0x7f88f0100c40 2024-04-24T09:06:25.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.609+0000 7f88ecc00640 1 -- 172.21.15.83:0/860644664 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88e8018b90 con 0x7f88f0100c40 2024-04-24T09:06:25.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.609+0000 7f88f5a00640 1 -- 172.21.15.83:0/860644664 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88f01a7550 con 0x7f88f0100c40 2024-04-24T09:06:25.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.613+0000 7f88ecc00640 1 -- 172.21.15.83:0/860644664 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f88e800f7f0 con 0x7f88f0100c40 2024-04-24T09:06:25.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.613+0000 7f88f9fd3740 1 Processor -- start 2024-04-24T09:06:25.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.614+0000 7f88f9fd3740 1 -- start start 2024-04-24T09:06:25.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.614+0000 7f88f9fd3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dbd3856fc0 0x55dbd38573a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:25.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.614+0000 7f88f9fd3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dbd3857970 0x55dbd386a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:25.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.614+0000 7f88f9fd3740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dbd386aa70 0x55dbd386ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:25.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.614+0000 7f88f9fd3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55dbd37bce70 con 0x55dbd3857970 2024-04-24T09:06:25.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.614+0000 7f88f9fd3740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55dbd3763380 con 0x55dbd386aa70 2024-04-24T09:06:25.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.614+0000 7f88f9fd3740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55dbd37b8e30 con 0x55dbd3856fc0 2024-04-24T09:06:25.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.614+0000 7f88ecc00640 1 --2- 172.21.15.83:0/860644664 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f88d40a3da0 0x7f88d40a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:25.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.614+0000 7f88ef400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dbd386aa70 0x55dbd386ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:25.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.614+0000 7f88efe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dbd3857970 0x55dbd386a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:25.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.615+0000 7f88ef400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dbd386aa70 0x55dbd386ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:39080/0 (socket says 172.21.15.83:39080) 2024-04-24T09:06:25.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.615+0000 7f88ef400640 1 -- 172.21.15.83:0/455626933 learned_addr learned my addr 172.21.15.83:0/455626933 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:06:25.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.615+0000 7f88ecc00640 1 -- 172.21.15.83:0/860644664 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9583+0+0 (secure 0 0 0) 0x7f88e80d10c0 con 0x7f88f0100c40 2024-04-24T09:06:25.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.615+0000 7f88ef400640 1 -- 172.21.15.83:0/455626933 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dbd3856fc0 msgr2=0x55dbd38573a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:25.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.615+0000 7f88ef400640 1 --2- 172.21.15.83:0/455626933 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dbd3856fc0 0x55dbd38573a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:25.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.615+0000 7f88ef400640 1 -- 172.21.15.83:0/455626933 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dbd3857970 msgr2=0x55dbd386a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:25.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.615+0000 7f88ef400640 1 --2- 172.21.15.83:0/455626933 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dbd3857970 0x55dbd386a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:25.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.615+0000 7f88ef400640 1 -- 172.21.15.83:0/455626933 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dbd35977e0 con 0x55dbd386aa70 2024-04-24T09:06:25.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.615+0000 7f88efe00640 1 --2- 172.21.15.83:0/455626933 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dbd3857970 0x55dbd386a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:06:25.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.616+0000 7f88f5000640 1 --2- 172.21.15.83:0/860644664 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f88d40a3da0 0x7f88d40a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:25.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.616+0000 7f88ef400640 1 --2- 172.21.15.83:0/455626933 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dbd386aa70 0x55dbd386ce60 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f88dc009230 tx=0x7f88dc004180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9c5c28ac5badd76 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:25.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.616+0000 7f88e5600640 1 -- 172.21.15.83:0/455626933 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88dc035040 con 0x55dbd386aa70 2024-04-24T09:06:25.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.616+0000 7f88e5600640 1 -- 172.21.15.83:0/455626933 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f88dc0054f0 con 0x55dbd386aa70 2024-04-24T09:06:25.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.616+0000 7f88e5600640 1 -- 172.21.15.83:0/455626933 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88dc003850 con 0x55dbd386aa70 2024-04-24T09:06:25.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.617+0000 7f88f9fd3740 1 -- 172.21.15.83:0/455626933 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dbd386aa70 msgr2=0x55dbd386ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:25.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.617+0000 7f88f9fd3740 1 --2- 172.21.15.83:0/455626933 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dbd386aa70 0x55dbd386ce60 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f88dc009230 tx=0x7f88dc004180 comp rx=0 tx=0).stop 2024-04-24T09:06:25.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.617+0000 7f88f5000640 1 --2- 172.21.15.83:0/860644664 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f88d40a3da0 0x7f88d40a6260 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f88f0102e20 tx=0x7f88e0009530 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:25.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.617+0000 7f88f9fd3740 1 -- 172.21.15.83:0/455626933 shutdown_connections 2024-04-24T09:06:25.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.617+0000 7f88f9fd3740 1 --2- 172.21.15.83:0/455626933 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dbd386aa70 0x55dbd386ce60 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:25.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.617+0000 7f88f9fd3740 1 --2- 172.21.15.83:0/455626933 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dbd3857970 0x55dbd386a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:25.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.617+0000 7f88f9fd3740 1 --2- 172.21.15.83:0/455626933 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dbd3856fc0 0x55dbd38573a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:25.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.617+0000 7f88f9fd3740 1 -- 172.21.15.83:0/455626933 >> 172.21.15.83:0/455626933 conn(0x55dbd3853590 msgr2=0x55dbd3853970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:25.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.618+0000 7f88f9fd3740 1 -- 172.21.15.83:0/455626933 shutdown_connections 2024-04-24T09:06:25.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.618+0000 7f88f9fd3740 1 -- 172.21.15.83:0/455626933 wait complete. 2024-04-24T09:06:25.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.618+0000 7f88f9fd3740 1 -- start start 2024-04-24T09:06:25.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.618+0000 7f88f9fd3740 1 Processor -- start 2024-04-24T09:06:25.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.619+0000 7f88f9fd3740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dbd3856fc0 0x55dbd3867a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:25.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.619+0000 7f88f9fd3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dbd3857970 0x55dbd3867f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:25.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.619+0000 7f88f9fd3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dbd386aa70 0x55dbd38684a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:25.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.619+0000 7f88f9fd3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55dbd386d7d0 con 0x55dbd386aa70 2024-04-24T09:06:25.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.619+0000 7f88f9fd3740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55dbd3776660 con 0x55dbd3856fc0 2024-04-24T09:06:25.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.619+0000 7f88f9fd3740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55dbd386d950 con 0x55dbd3857970 2024-04-24T09:06:25.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.619+0000 7f88f9fd3740 10 client.0 _ll_register_callbacks cb 0x55dbd3764aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:06:25.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.619+0000 7f88efe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dbd3856fc0 0x55dbd3867a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:25.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.619+0000 7f88efe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dbd3856fc0 0x55dbd3867a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:39094/0 (socket says 172.21.15.83:39094) 2024-04-24T09:06:25.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.619+0000 7f88efe00640 1 -- 172.21.15.83:0/4196497861 learned_addr learned my addr 172.21.15.83:0/4196497861 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:06:25.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.620+0000 7f88efe00640 1 -- 172.21.15.83:0/4196497861 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dbd3857970 msgr2=0x55dbd3867f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:25.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.620+0000 7f88efe00640 1 --2- 172.21.15.83:0/4196497861 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dbd3857970 0x55dbd3867f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:25.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.620+0000 7f88efe00640 1 -- 172.21.15.83:0/4196497861 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dbd386aa70 msgr2=0x55dbd38684a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:25.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.620+0000 7f88f5000640 1 --2- 172.21.15.83:0/4196497861 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dbd386aa70 0x55dbd38684a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:25.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.620+0000 7f88efe00640 1 --2- 172.21.15.83:0/4196497861 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dbd386aa70 0x55dbd38684a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:25.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.620+0000 7f88efe00640 1 -- 172.21.15.83:0/4196497861 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dbd35f7f70 con 0x55dbd3856fc0 2024-04-24T09:06:25.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.620+0000 7f88f5000640 1 --2- 172.21.15.83:0/4196497861 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dbd386aa70 0x55dbd38684a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:06:25.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.620+0000 7f88efe00640 1 --2- 172.21.15.83:0/4196497861 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dbd3856fc0 0x55dbd3867a20 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f88e800afd0 tx=0x7f88e8000f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29624c80c3908c64 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:25.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.620+0000 7f88da000640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:06:25.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.621+0000 7f88da000640 1 -- 172.21.15.83:0/4196497861 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88e809d2e0 con 0x55dbd3856fc0 2024-04-24T09:06:25.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.621+0000 7f88f9fd3740 1 -- 172.21.15.83:0/4196497861 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88dc0054f0 con 0x55dbd3856fc0 2024-04-24T09:06:25.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.621+0000 7f88da000640 1 -- 172.21.15.83:0/4196497861 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9583+0+0 (secure 0 0 0) 0x7f88e809aee0 con 0x55dbd3856fc0 2024-04-24T09:06:25.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.621+0000 7f88da000640 1 -- 172.21.15.83:0/4196497861 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f88e80b6150 con 0x55dbd3856fc0 2024-04-24T09:06:25.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.621+0000 7f88f9fd3740 10 client.38033 fetch_fsmap learned FSMap version 48 2024-04-24T09:06:25.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.621+0000 7f88f9fd3740 1 -- 172.21.15.83:0/4196497861 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55dbd348f170 con 0x55dbd3856fc0 2024-04-24T09:06:25.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.621+0000 7f88da000640 1 -- 172.21.15.83:0/4196497861 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f88dc0054f0 con 0x55dbd3856fc0 2024-04-24T09:06:25.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.622+0000 7f88da000640 1 -- 172.21.15.83:0/4196497861 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 48) ==== 8402+0+0 (secure 0 0 0) 0x7f88e80ce610 con 0x55dbd3856fc0 2024-04-24T09:06:25.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.623+0000 7f88f9fd3740 10 client.38033 fetch_fsmap finished waiting for FSMap version 48 2024-04-24T09:06:25.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.623+0000 7f88f9fd3740 10 client.38033 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:06:25.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.623+0000 7f88f9fd3740 20 client.38033 populate_metadata read hostname 'smithi083' 2024-04-24T09:06:25.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.623+0000 7f88f9fd3740 1 --2- 172.21.15.83:0/4196497861 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55dbd387ad40 0x55dbd387d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:25.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.623+0000 7f88f9fd3740 4 client.38033 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:06:25.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.623+0000 7f88f9fd3740 1 -- 172.21.15.83:0/4196497861 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dbd37ca890 con 0x55dbd387ad40 2024-04-24T09:06:25.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.623+0000 7f88ef400640 1 --2- 172.21.15.83:0/4196497861 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55dbd387ad40 0x55dbd387d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:25.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.624+0000 7f88ef400640 1 --2- 172.21.15.83:0/4196497861 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55dbd387ad40 0x55dbd387d200 crc :-1 s=READY pgs=381 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8cee7263417815b8 server_cookie=cdd3230a09f9a281 in_seq=0 out_seq=0 2024-04-24T09:06:25.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.624+0000 7f88da000640 10 client.38033 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:06:25.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.625+0000 7f88da000640 1 -- 172.21.15.83:0/4196497861 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55dbd37ca890 con 0x55dbd387ad40 2024-04-24T09:06:25.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.625+0000 7f88da000640 10 client.38033 handle_command_reply: tid=0 2024-04-24T09:06:25.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.625+0000 7f88f9fd3740 1 client.38033 shutdown 2024-04-24T09:06:25.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.625+0000 7f88f9fd3740 1 -- 172.21.15.83:0/4196497861 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dbd3856fc0 msgr2=0x55dbd3867a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:25.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.625+0000 7f88f9fd3740 1 --2- 172.21.15.83:0/4196497861 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dbd3856fc0 0x55dbd3867a20 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f88e800afd0 tx=0x7f88e8000f40 comp rx=0 tx=0).stop 2024-04-24T09:06:25.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.625+0000 7f88f9fd3740 1 -- 172.21.15.83:0/4196497861 shutdown_connections 2024-04-24T09:06:25.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.625+0000 7f88f9fd3740 1 --2- 172.21.15.83:0/4196497861 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55dbd387ad40 0x55dbd387d200 crc :-1 s=READY pgs=381 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:25.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.625+0000 7f88da000640 0 client.38033 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:06:25.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.625+0000 7f88f9fd3740 1 --2- 172.21.15.83:0/4196497861 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dbd386aa70 0x55dbd38684a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:25.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.625+0000 7f88f9fd3740 1 --2- 172.21.15.83:0/4196497861 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dbd3857970 0x55dbd3867f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:25.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.625+0000 7f88f9fd3740 1 --2- 172.21.15.83:0/4196497861 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dbd3856fc0 0x55dbd3867a20 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:25.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.625+0000 7f88f9fd3740 1 -- 172.21.15.83:0/4196497861 >> 172.21.15.83:0/4196497861 conn(0x55dbd3853590 msgr2=0x55dbd3859390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:25.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.625+0000 7f88f9fd3740 1 -- 172.21.15.83:0/4196497861 shutdown_connections 2024-04-24T09:06:25.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.626+0000 7f88f9fd3740 1 -- 172.21.15.83:0/4196497861 wait complete. 2024-04-24T09:06:25.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.626+0000 7f88f9fd3740 20 client.38033 trim_cache size 0 max 16384 2024-04-24T09:06:25.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.651+0000 7f88f9fd3740 1 Processor -- start 2024-04-24T09:06:25.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.651+0000 7f88f9fd3740 1 -- start start 2024-04-24T09:06:25.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.651+0000 7f88f9fd3740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dbd3856fc0 0x55dbd3876700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:25.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.651+0000 7f88f9fd3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dbd3857970 0x55dbd3882b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:25.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.651+0000 7f88f9fd3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dbd386aa70 0x55dbd3883050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:25.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.651+0000 7f88f9fd3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f88f01ad210 con 0x55dbd386aa70 2024-04-24T09:06:25.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.651+0000 7f88f9fd3740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f88f01aeed0 con 0x55dbd3856fc0 2024-04-24T09:06:25.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.651+0000 7f88f9fd3740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f88f01ad8e0 con 0x55dbd3857970 2024-04-24T09:06:25.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.651+0000 7f88f5000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dbd386aa70 0x55dbd3883050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:25.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.651+0000 7f88efe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dbd3856fc0 0x55dbd3876700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:25.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.651+0000 7f88f5000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dbd386aa70 0x55dbd3883050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56092/0 (socket says 172.21.15.83:56092) 2024-04-24T09:06:25.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.651+0000 7f88f5000640 1 -- 172.21.15.83:0/707062170 learned_addr learned my addr 172.21.15.83:0/707062170 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:06:25.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.652+0000 7f88ef400640 1 --2- 172.21.15.83:0/707062170 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dbd3857970 0x55dbd3882b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:25.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.652+0000 7f88f5000640 1 -- 172.21.15.83:0/707062170 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dbd3857970 msgr2=0x55dbd3882b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:25.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.652+0000 7f88f5000640 1 --2- 172.21.15.83:0/707062170 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dbd3857970 0x55dbd3882b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:25.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.652+0000 7f88f5000640 1 -- 172.21.15.83:0/707062170 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dbd3856fc0 msgr2=0x55dbd3876700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:25.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.652+0000 7f88f5000640 1 --2- 172.21.15.83:0/707062170 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dbd3856fc0 0x55dbd3876700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:25.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.652+0000 7f88f5000640 1 -- 172.21.15.83:0/707062170 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dbd3857710 con 0x55dbd386aa70 2024-04-24T09:06:25.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.652+0000 7f88f5000640 1 --2- 172.21.15.83:0/707062170 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dbd386aa70 0x55dbd3883050 secure :-1 s=READY pgs=2193 cs=0 l=1 rev1=1 crypto rx=0x7f88e0004650 tx=0x7f88e0015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74875346eebd274c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:25.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.652+0000 7f88e4c00640 1 -- 172.21.15.83:0/707062170 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88e0010950 con 0x55dbd386aa70 2024-04-24T09:06:25.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.652+0000 7f88e4c00640 1 -- 172.21.15.83:0/707062170 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f88e001a410 con 0x55dbd386aa70 2024-04-24T09:06:25.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.652+0000 7f88e4c00640 1 -- 172.21.15.83:0/707062170 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88e0025620 con 0x55dbd386aa70 2024-04-24T09:06:25.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.653+0000 7f88f9fd3740 1 -- 172.21.15.83:0/707062170 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dbd386aa70 msgr2=0x55dbd3883050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:25.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.653+0000 7f88f9fd3740 1 --2- 172.21.15.83:0/707062170 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dbd386aa70 0x55dbd3883050 secure :-1 s=READY pgs=2193 cs=0 l=1 rev1=1 crypto rx=0x7f88e0004650 tx=0x7f88e0015000 comp rx=0 tx=0).stop 2024-04-24T09:06:25.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.653+0000 7f88f9fd3740 1 -- 172.21.15.83:0/707062170 shutdown_connections 2024-04-24T09:06:25.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.653+0000 7f88f9fd3740 1 --2- 172.21.15.83:0/707062170 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dbd386aa70 0x55dbd3883050 unknown :-1 s=CLOSED pgs=2193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:25.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.653+0000 7f88f9fd3740 1 --2- 172.21.15.83:0/707062170 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dbd3857970 0x55dbd3882b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:25.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.653+0000 7f88f9fd3740 1 --2- 172.21.15.83:0/707062170 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dbd3856fc0 0x55dbd3876700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:25.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.653+0000 7f88f9fd3740 1 -- 172.21.15.83:0/707062170 >> 172.21.15.83:0/707062170 conn(0x55dbd3853590 msgr2=0x55dbd38526f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:25.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.653+0000 7f88f9fd3740 1 -- 172.21.15.83:0/707062170 shutdown_connections 2024-04-24T09:06:25.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.653+0000 7f88f9fd3740 1 -- 172.21.15.83:0/707062170 wait complete. 2024-04-24T09:06:25.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.654+0000 7f88f9fd3740 1 -- start start 2024-04-24T09:06:25.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.654+0000 7f88f9fd3740 1 Processor -- start 2024-04-24T09:06:25.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.654+0000 7f88f9fd3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dbd3856fc0 0x55dbd3872ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:25.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.654+0000 7f88f9fd3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dbd3857970 0x55dbd386f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:25.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.655+0000 7f88f9fd3740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dbd386aa70 0x55dbd386f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:25.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.655+0000 7f88f9fd3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f88f01ae740 con 0x55dbd3857970 2024-04-24T09:06:25.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.655+0000 7f88f9fd3740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f88f01ae010 con 0x55dbd386aa70 2024-04-24T09:06:25.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.655+0000 7f88f9fd3740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f88f01ab210 con 0x55dbd3856fc0 2024-04-24T09:06:25.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.655+0000 7f88f9fd3740 10 client.0 _ll_register_callbacks cb 0x7f88e809c720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:06:25.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.655+0000 7f88ef400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dbd3857970 0x55dbd386f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:25.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.655+0000 7f88ef400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dbd3857970 0x55dbd386f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56094/0 (socket says 172.21.15.83:56094) 2024-04-24T09:06:25.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.655+0000 7f88ef400640 1 -- 172.21.15.83:0/2569023621 learned_addr learned my addr 172.21.15.83:0/2569023621 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:06:25.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.655+0000 7f88f5000640 1 --2- 172.21.15.83:0/2569023621 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dbd386aa70 0x55dbd386f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:25.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.655+0000 7f88efe00640 1 --2- 172.21.15.83:0/2569023621 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dbd3856fc0 0x55dbd3872ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:25.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.655+0000 7f88ef400640 1 -- 172.21.15.83:0/2569023621 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dbd3856fc0 msgr2=0x55dbd3872ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:25.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.655+0000 7f88ef400640 1 --2- 172.21.15.83:0/2569023621 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dbd3856fc0 0x55dbd3872ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:25.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.655+0000 7f88ef400640 1 -- 172.21.15.83:0/2569023621 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dbd386aa70 msgr2=0x55dbd386f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:25.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.655+0000 7f88ef400640 1 --2- 172.21.15.83:0/2569023621 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dbd386aa70 0x55dbd386f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:25.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.655+0000 7f88ef400640 1 -- 172.21.15.83:0/2569023621 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dbd386fae0 con 0x55dbd3857970 2024-04-24T09:06:25.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.656+0000 7f88ef400640 1 --2- 172.21.15.83:0/2569023621 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dbd3857970 0x55dbd386f060 secure :-1 s=READY pgs=2194 cs=0 l=1 rev1=1 crypto rx=0x7f88dc02a9e0 tx=0x7f88dc02a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaf75e2300b0e245 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:25.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.656+0000 7f88da000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:06:25.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.656+0000 7f88da000640 1 -- 172.21.15.83:0/2569023621 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88dc035040 con 0x55dbd3857970 2024-04-24T09:06:25.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.656+0000 7f88da000640 1 -- 172.21.15.83:0/2569023621 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9583+0+0 (secure 0 0 0) 0x7f88dc004f90 con 0x55dbd3857970 2024-04-24T09:06:25.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.656+0000 7f88f9fd3740 1 -- 172.21.15.83:0/2569023621 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88e001a410 con 0x55dbd3857970 2024-04-24T09:06:25.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.656+0000 7f88da000640 1 -- 172.21.15.83:0/2569023621 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f88dc0044b0 con 0x55dbd3857970 2024-04-24T09:06:25.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.656+0000 7f88f9fd3740 10 client.20889 fetch_fsmap learned FSMap version 48 2024-04-24T09:06:25.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.656+0000 7f88f9fd3740 1 -- 172.21.15.83:0/2569023621 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55dbd38e20e0 con 0x55dbd3857970 2024-04-24T09:06:25.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.656+0000 7f88da000640 1 -- 172.21.15.83:0/2569023621 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f88e001a410 con 0x55dbd3857970 2024-04-24T09:06:25.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.657+0000 7f88da000640 1 -- 172.21.15.83:0/2569023621 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 48) ==== 8402+0+0 (secure 0 0 0) 0x7f88dc003990 con 0x55dbd3857970 2024-04-24T09:06:25.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.657+0000 7f88f9fd3740 10 client.20889 fetch_fsmap finished waiting for FSMap version 48 2024-04-24T09:06:25.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.657+0000 7f88f9fd3740 10 client.20889 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:06:25.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.657+0000 7f88f9fd3740 20 client.20889 populate_metadata read hostname 'smithi083' 2024-04-24T09:06:25.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.657+0000 7f88f9fd3740 1 --2- 172.21.15.83:0/2569023621 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55dbd38e2530 0x55dbd38e4920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:25.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.657+0000 7f88f9fd3740 4 client.20889 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T09:06:25.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.657+0000 7f88f9fd3740 1 -- 172.21.15.83:0/2569023621 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55dbd37c4700 con 0x55dbd38e2530 2024-04-24T09:06:25.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.657+0000 7f88efe00640 1 --2- 172.21.15.83:0/2569023621 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55dbd38e2530 0x55dbd38e4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:25.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.658+0000 7f88efe00640 1 --2- 172.21.15.83:0/2569023621 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55dbd38e2530 0x55dbd38e4920 crc :-1 s=READY pgs=382 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c80ac439d58982cd server_cookie=7ebfd44c4d184ddf in_seq=0 out_seq=0 2024-04-24T09:06:25.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.658+0000 7f88da000640 10 client.20889 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:06:25.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.659+0000 7f88da000640 1 -- 172.21.15.83:0/2569023621 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55dbd37c4700 con 0x55dbd38e2530 2024-04-24T09:06:25.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.659+0000 7f88da000640 10 client.20889 handle_command_reply: tid=0 2024-04-24T09:06:25.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.659+0000 7f88f9fd3740 1 client.20889 shutdown 2024-04-24T09:06:25.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.659+0000 7f88f9fd3740 1 -- 172.21.15.83:0/2569023621 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dbd3857970 msgr2=0x55dbd386f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:25.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.659+0000 7f88f9fd3740 1 --2- 172.21.15.83:0/2569023621 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dbd3857970 0x55dbd386f060 secure :-1 s=READY pgs=2194 cs=0 l=1 rev1=1 crypto rx=0x7f88dc02a9e0 tx=0x7f88dc02a3f0 comp rx=0 tx=0).stop 2024-04-24T09:06:25.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.659+0000 7f88f9fd3740 1 -- 172.21.15.83:0/2569023621 shutdown_connections 2024-04-24T09:06:25.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.659+0000 7f88f9fd3740 1 --2- 172.21.15.83:0/2569023621 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55dbd38e2530 0x55dbd38e4920 crc :-1 s=READY pgs=382 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:25.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.659+0000 7f88f9fd3740 1 --2- 172.21.15.83:0/2569023621 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dbd386aa70 0x55dbd386f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:25.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.659+0000 7f88f9fd3740 1 --2- 172.21.15.83:0/2569023621 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dbd3857970 0x55dbd386f060 unknown :-1 s=CLOSED pgs=2194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:25.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.659+0000 7f88f9fd3740 1 --2- 172.21.15.83:0/2569023621 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dbd3856fc0 0x55dbd3872ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:25.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.659+0000 7f88da000640 0 client.20889 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:06:25.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.659+0000 7f88f9fd3740 1 -- 172.21.15.83:0/2569023621 >> 172.21.15.83:0/2569023621 conn(0x55dbd3853590 msgr2=0x55dbd3858b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:25.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.659+0000 7f88f9fd3740 1 -- 172.21.15.83:0/2569023621 shutdown_connections 2024-04-24T09:06:25.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.659+0000 7f88f9fd3740 1 -- 172.21.15.83:0/2569023621 wait complete. 2024-04-24T09:06:25.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.659+0000 7f88f9fd3740 20 client.20889 trim_cache size 0 max 16384 2024-04-24T09:06:25.662 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T09:06:25.662 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-04-24T09:06:25.662 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "fbb102a6-193a-4cfb-bbb2-ca4e68e5dfe8", 2024-04-24T09:06:25.662 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-04-24T09:06:25.662 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T09:06:25.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.660+0000 7f88f5a00640 1 -- 172.21.15.83:0/860644664 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f88d40a3da0 msgr2=0x7f88d40a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:25.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.660+0000 7f88f5a00640 1 --2- 172.21.15.83:0/860644664 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f88d40a3da0 0x7f88d40a6260 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f88f0102e20 tx=0x7f88e0009530 comp rx=0 tx=0).stop 2024-04-24T09:06:25.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.660+0000 7f88f5a00640 1 -- 172.21.15.83:0/860644664 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88f0100c40 msgr2=0x7f88f01a05d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:25.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.660+0000 7f88f5a00640 1 --2- 172.21.15.83:0/860644664 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88f0100c40 0x7f88f01a05d0 secure :-1 s=READY pgs=2192 cs=0 l=1 rev1=1 crypto rx=0x7f88e800cce0 tx=0x7f88e800a620 comp rx=0 tx=0).stop 2024-04-24T09:06:25.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.660+0000 7f88f5a00640 1 -- 172.21.15.83:0/860644664 shutdown_connections 2024-04-24T09:06:25.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.660+0000 7f88f5a00640 1 --2- 172.21.15.83:0/860644664 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f88d40a3da0 0x7f88d40a6260 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:25.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.660+0000 7f88f5a00640 1 --2- 172.21.15.83:0/860644664 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f88f010e7f0 0x7f88f019a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:25.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.660+0000 7f88f5a00640 1 --2- 172.21.15.83:0/860644664 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88f01015f0 0x7f88f01a0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:25.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.660+0000 7f88f5a00640 1 --2- 172.21.15.83:0/860644664 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88f0100c40 0x7f88f01a05d0 unknown :-1 s=CLOSED pgs=2192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:25.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.660+0000 7f88f5a00640 1 -- 172.21.15.83:0/860644664 >> 172.21.15.83:0/860644664 conn(0x7f88f00fc880 msgr2=0x7f88f00fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:25.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.660+0000 7f88f5a00640 1 -- 172.21.15.83:0/860644664 shutdown_connections 2024-04-24T09:06:25.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.660+0000 7f88f5a00640 1 -- 172.21.15.83:0/860644664 wait complete. 2024-04-24T09:06:25.675 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T09:06:25.676 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T09:06:25.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.892+0000 7f511ca00640 1 -- 172.21.15.83:0/2133693107 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5118109540 msgr2=0x7f51181099a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:25.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.892+0000 7f511ca00640 1 --2- 172.21.15.83:0/2133693107 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5118109540 0x7f51181099a0 secure :-1 s=READY pgs=2195 cs=0 l=1 rev1=1 crypto rx=0x7f5104009900 tx=0x7f510402a3f0 comp rx=0 tx=0).stop 2024-04-24T09:06:25.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.892+0000 7f511ca00640 1 -- 172.21.15.83:0/2133693107 shutdown_connections 2024-04-24T09:06:25.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.892+0000 7f511ca00640 1 --2- 172.21.15.83:0/2133693107 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5118117000 0x7f51181193f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:25.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.892+0000 7f511ca00640 1 --2- 172.21.15.83:0/2133693107 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5118109540 0x7f51181099a0 unknown :-1 s=CLOSED pgs=2195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:25.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.892+0000 7f511ca00640 1 --2- 172.21.15.83:0/2133693107 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f511810c5b0 0x7f5118109000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:25.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.892+0000 7f511ca00640 1 -- 172.21.15.83:0/2133693107 >> 172.21.15.83:0/2133693107 conn(0x7f511806cf80 msgr2=0x7f511806d390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:25.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.892+0000 7f511ca00640 1 -- 172.21.15.83:0/2133693107 shutdown_connections 2024-04-24T09:06:25.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.892+0000 7f511ca00640 1 -- 172.21.15.83:0/2133693107 wait complete. 2024-04-24T09:06:25.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.893+0000 7f511ca00640 1 Processor -- start 2024-04-24T09:06:25.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.894+0000 7f511ca00640 1 -- start start 2024-04-24T09:06:25.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.894+0000 7f511ca00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5118109540 0x7f51181a32a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:25.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.894+0000 7f511ca00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f511810c5b0 0x7f51181a37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:25.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.894+0000 7f511ca00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5118117000 0x7f51181a7b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:25.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.894+0000 7f511ca00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f511811cc20 con 0x7f5118117000 2024-04-24T09:06:25.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.894+0000 7f511ca00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f511811caa0 con 0x7f5118109540 2024-04-24T09:06:25.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.894+0000 7f511ca00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f511811cda0 con 0x7f511810c5b0 2024-04-24T09:06:25.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.895+0000 7f5117200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5118117000 0x7f51181a7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:25.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.895+0000 7f5117200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5118117000 0x7f51181a7b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56110/0 (socket says 172.21.15.83:56110) 2024-04-24T09:06:25.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.895+0000 7f5117200640 1 -- 172.21.15.83:0/624529597 learned_addr learned my addr 172.21.15.83:0/624529597 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:06:25.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.895+0000 7f5115e00640 1 --2- 172.21.15.83:0/624529597 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f511810c5b0 0x7f51181a37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:25.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.895+0000 7f5117200640 1 -- 172.21.15.83:0/624529597 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f511810c5b0 msgr2=0x7f51181a37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:25.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.895+0000 7f5117200640 1 --2- 172.21.15.83:0/624529597 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f511810c5b0 0x7f51181a37e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:25.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.895+0000 7f5117200640 1 -- 172.21.15.83:0/624529597 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5118109540 msgr2=0x7f51181a32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:25.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.895+0000 7f5117200640 1 --2- 172.21.15.83:0/624529597 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5118109540 0x7f51181a32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:25.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.895+0000 7f5117200640 1 -- 172.21.15.83:0/624529597 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51181a8290 con 0x7f5118117000 2024-04-24T09:06:25.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.896+0000 7f5117200640 1 --2- 172.21.15.83:0/624529597 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5118117000 0x7f51181a7b60 secure :-1 s=READY pgs=2196 cs=0 l=1 rev1=1 crypto rx=0x7f511000caf0 tx=0x7f5110002170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:25.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.897+0000 7f510f400640 1 -- 172.21.15.83:0/624529597 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5110016030 con 0x7f5118117000 2024-04-24T09:06:25.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.897+0000 7f510f400640 1 -- 172.21.15.83:0/624529597 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5110003970 con 0x7f5118117000 2024-04-24T09:06:25.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.897+0000 7f510f400640 1 -- 172.21.15.83:0/624529597 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5110004660 con 0x7f5118117000 2024-04-24T09:06:25.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.897+0000 7f511ca00640 1 -- 172.21.15.83:0/624529597 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51181a8580 con 0x7f5118117000 2024-04-24T09:06:25.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.897+0000 7f511ca00640 1 -- 172.21.15.83:0/624529597 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51181afe10 con 0x7f5118117000 2024-04-24T09:06:25.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.902+0000 7f510f400640 1 -- 172.21.15.83:0/624529597 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5110019070 con 0x7f5118117000 2024-04-24T09:06:25.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.904+0000 7f510f400640 1 --2- 172.21.15.83:0/624529597 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f50fc0a3e70 0x7f50fc0a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:25.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.904+0000 7f510f400640 1 -- 172.21.15.83:0/624529597 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9583+0+0 (secure 0 0 0) 0x7f51100d0590 con 0x7f5118117000 2024-04-24T09:06:25.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.905+0000 7f5116800640 1 --2- 172.21.15.83:0/624529597 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f50fc0a3e70 0x7f50fc0a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:25.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.905+0000 7f51201e7740 1 Processor -- start 2024-04-24T09:06:25.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.905+0000 7f51201e7740 1 -- start start 2024-04-24T09:06:25.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.906+0000 7f51201e7740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56307b7b0240 0x56307b7b0620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:25.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.906+0000 7f51201e7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56307b7b0b60 0x56307b7bacf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:25.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.906+0000 7f51201e7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56307b7bb230 0x56307b7bd620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:25.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.906+0000 7f51201e7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56307b70d1f0 con 0x56307b7bb230 2024-04-24T09:06:25.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.906+0000 7f51201e7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56307b6b3350 con 0x56307b7b0b60 2024-04-24T09:06:25.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.906+0000 7f51201e7740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56307b7091b0 con 0x56307b7b0240 2024-04-24T09:06:25.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.906+0000 7f5116800640 1 --2- 172.21.15.83:0/624529597 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f50fc0a3e70 0x7f50fc0a6330 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f5108009760 tx=0x7f5108006c90 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:25.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.906+0000 7f5117200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56307b7bb230 0x56307b7bd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:25.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.906+0000 7f5117200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56307b7bb230 0x56307b7bd620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56126/0 (socket says 172.21.15.83:56126) 2024-04-24T09:06:25.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.906+0000 7f5117200640 1 -- 172.21.15.83:0/1098923762 learned_addr learned my addr 172.21.15.83:0/1098923762 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:06:25.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.906+0000 7f5116800640 1 --2- 172.21.15.83:0/1098923762 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56307b7b0240 0x56307b7b0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:25.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.907+0000 7f5115e00640 1 --2- 172.21.15.83:0/1098923762 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56307b7b0b60 0x56307b7bacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:25.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.907+0000 7f5117200640 1 -- 172.21.15.83:0/1098923762 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56307b7b0240 msgr2=0x56307b7b0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:25.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.907+0000 7f5117200640 1 --2- 172.21.15.83:0/1098923762 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56307b7b0240 0x56307b7b0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:25.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.907+0000 7f5117200640 1 -- 172.21.15.83:0/1098923762 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56307b7b0b60 msgr2=0x56307b7bacf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:25.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.907+0000 7f5117200640 1 --2- 172.21.15.83:0/1098923762 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56307b7b0b60 0x56307b7bacf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:25.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.907+0000 7f5117200640 1 -- 172.21.15.83:0/1098923762 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56307b707f50 con 0x56307b7bb230 2024-04-24T09:06:25.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.907+0000 7f5116800640 1 --2- 172.21.15.83:0/1098923762 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56307b7b0240 0x56307b7b0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:06:25.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.908+0000 7f5117200640 1 --2- 172.21.15.83:0/1098923762 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56307b7bb230 0x56307b7bd620 secure :-1 s=READY pgs=2197 cs=0 l=1 rev1=1 crypto rx=0x7f5110098680 tx=0x7f51100048c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc61c9cebe8cb562 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:25.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.909+0000 7f5103400640 1 -- 172.21.15.83:0/1098923762 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51100b4020 con 0x56307b7bb230 2024-04-24T09:06:25.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.909+0000 7f5103400640 1 -- 172.21.15.83:0/1098923762 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f51100a05d0 con 0x56307b7bb230 2024-04-24T09:06:25.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.909+0000 7f5103400640 1 -- 172.21.15.83:0/1098923762 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51100b4020 con 0x56307b7bb230 2024-04-24T09:06:25.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.911+0000 7f51201e7740 1 -- 172.21.15.83:0/1098923762 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56307b7bb230 msgr2=0x56307b7bd620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:25.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.911+0000 7f51201e7740 1 --2- 172.21.15.83:0/1098923762 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56307b7bb230 0x56307b7bd620 secure :-1 s=READY pgs=2197 cs=0 l=1 rev1=1 crypto rx=0x7f5110098680 tx=0x7f51100048c0 comp rx=0 tx=0).stop 2024-04-24T09:06:25.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.911+0000 7f51201e7740 1 -- 172.21.15.83:0/1098923762 shutdown_connections 2024-04-24T09:06:25.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.911+0000 7f51201e7740 1 --2- 172.21.15.83:0/1098923762 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56307b7bb230 0x56307b7bd620 unknown :-1 s=CLOSED pgs=2197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:25.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.911+0000 7f51201e7740 1 --2- 172.21.15.83:0/1098923762 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56307b7b0b60 0x56307b7bacf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:25.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.911+0000 7f51201e7740 1 --2- 172.21.15.83:0/1098923762 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56307b7b0240 0x56307b7b0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:25.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.911+0000 7f51201e7740 1 -- 172.21.15.83:0/1098923762 >> 172.21.15.83:0/1098923762 conn(0x56307b7ac560 msgr2=0x56307b7ac940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:25.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.912+0000 7f51201e7740 1 -- 172.21.15.83:0/1098923762 shutdown_connections 2024-04-24T09:06:25.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.912+0000 7f51201e7740 1 -- 172.21.15.83:0/1098923762 wait complete. 2024-04-24T09:06:25.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.912+0000 7f51201e7740 1 -- start start 2024-04-24T09:06:25.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.912+0000 7f51201e7740 1 Processor -- start 2024-04-24T09:06:25.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.913+0000 7f51201e7740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56307b7b0240 0x56307b7a95e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:25.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.913+0000 7f51201e7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56307b7b0b60 0x56307b7a9b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:25.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.913+0000 7f51201e7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56307b7bb230 0x56307b7b9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:25.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.913+0000 7f51201e7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56307b7a3940 con 0x56307b7b0b60 2024-04-24T09:06:25.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.913+0000 7f51201e7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56307b6c6690 con 0x56307b7bb230 2024-04-24T09:06:25.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.913+0000 7f51201e7740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56307b7a3ac0 con 0x56307b7b0240 2024-04-24T09:06:25.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.913+0000 7f51201e7740 10 client.0 _ll_register_callbacks cb 0x56307b702650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:06:25.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.913+0000 7f5116800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56307b7b0240 0x56307b7a95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:25.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.913+0000 7f5116800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56307b7b0240 0x56307b7a95e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:43304/0 (socket says 172.21.15.83:43304) 2024-04-24T09:06:25.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.913+0000 7f5116800640 1 -- 172.21.15.83:0/1311521355 learned_addr learned my addr 172.21.15.83:0/1311521355 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:06:25.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.915+0000 7f5115e00640 1 --2- 172.21.15.83:0/1311521355 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56307b7b0b60 0x56307b7a9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:25.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.915+0000 7f5116800640 1 -- 172.21.15.83:0/1311521355 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56307b7bb230 msgr2=0x56307b7b9ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:25.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.915+0000 7f5116800640 1 --2- 172.21.15.83:0/1311521355 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56307b7bb230 0x56307b7b9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:25.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.915+0000 7f5116800640 1 -- 172.21.15.83:0/1311521355 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56307b7b0b60 msgr2=0x56307b7a9b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:25.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.915+0000 7f5116800640 1 --2- 172.21.15.83:0/1311521355 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56307b7b0b60 0x56307b7a9b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:25.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.915+0000 7f5116800640 1 -- 172.21.15.83:0/1311521355 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56307b6c5920 con 0x56307b7b0240 2024-04-24T09:06:25.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.915+0000 7f5116800640 1 --2- 172.21.15.83:0/1311521355 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56307b7b0240 0x56307b7a95e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f51080156a0 tx=0x7f510800e000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6482efbe0e893644 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:25.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.915+0000 7f50f7e00640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-04-24T09:06:25.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.916+0000 7f50f7e00640 1 -- 172.21.15.83:0/1311521355 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5108026060 con 0x56307b7b0240 2024-04-24T09:06:25.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.916+0000 7f50f7e00640 1 -- 172.21.15.83:0/1311521355 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9583+0+0 (secure 0 0 0) 0x7f5108021680 con 0x56307b7b0240 2024-04-24T09:06:25.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.916+0000 7f51201e7740 1 -- 172.21.15.83:0/1311521355 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51100a05d0 con 0x56307b7b0240 2024-04-24T09:06:25.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.916+0000 7f50f7e00640 1 -- 172.21.15.83:0/1311521355 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f5108005a30 con 0x56307b7b0240 2024-04-24T09:06:25.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.917+0000 7f51201e7740 10 client.28168 fetch_fsmap learned FSMap version 48 2024-04-24T09:06:25.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.917+0000 7f51201e7740 1 -- 172.21.15.83:0/1311521355 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x56307b4e97d0 con 0x56307b7b0240 2024-04-24T09:06:25.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.917+0000 7f50f7e00640 1 -- 172.21.15.83:0/1311521355 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f51100a05d0 con 0x56307b7b0240 2024-04-24T09:06:25.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.918+0000 7f50f7e00640 1 -- 172.21.15.83:0/1311521355 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 48) ==== 8402+0+0 (secure 0 0 0) 0x7f5108006020 con 0x56307b7b0240 2024-04-24T09:06:25.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.918+0000 7f51201e7740 10 client.28168 fetch_fsmap finished waiting for FSMap version 48 2024-04-24T09:06:25.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.918+0000 7f51201e7740 10 client.28168 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:06:25.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.918+0000 7f51201e7740 20 client.28168 populate_metadata read hostname 'smithi083' 2024-04-24T09:06:25.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.918+0000 7f51201e7740 1 --2- 172.21.15.83:0/1311521355 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56307b7d3db0 0x56307b7d6230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:25.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.918+0000 7f51201e7740 4 client.28168 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:06:25.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.918+0000 7f51201e7740 1 -- 172.21.15.83:0/1311521355 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56307b71ac10 con 0x56307b7d3db0 2024-04-24T09:06:25.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.918+0000 7f5115e00640 1 --2- 172.21.15.83:0/1311521355 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56307b7d3db0 0x56307b7d6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:25.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.919+0000 7f5115e00640 1 --2- 172.21.15.83:0/1311521355 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56307b7d3db0 0x56307b7d6230 crc :-1 s=READY pgs=383 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e26dd734a8e3cd12 server_cookie=177ecb9c2cd84a7e in_seq=0 out_seq=0 2024-04-24T09:06:25.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.919+0000 7f50f7e00640 10 client.28168 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:06:25.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.921+0000 7f50f7e00640 1 -- 172.21.15.83:0/1311521355 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56307b71ac10 con 0x56307b7d3db0 2024-04-24T09:06:25.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.921+0000 7f50f7e00640 10 client.28168 handle_command_reply: tid=0 2024-04-24T09:06:25.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.921+0000 7f51201e7740 1 client.28168 shutdown 2024-04-24T09:06:25.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.922+0000 7f51201e7740 1 -- 172.21.15.83:0/1311521355 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56307b7b0240 msgr2=0x56307b7a95e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:25.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.922+0000 7f51201e7740 1 --2- 172.21.15.83:0/1311521355 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56307b7b0240 0x56307b7a95e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f51080156a0 tx=0x7f510800e000 comp rx=0 tx=0).stop 2024-04-24T09:06:25.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.922+0000 7f51201e7740 1 -- 172.21.15.83:0/1311521355 shutdown_connections 2024-04-24T09:06:25.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.922+0000 7f51201e7740 1 --2- 172.21.15.83:0/1311521355 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56307b7d3db0 0x56307b7d6230 crc :-1 s=READY pgs=383 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:25.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.922+0000 7f51201e7740 1 --2- 172.21.15.83:0/1311521355 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56307b7bb230 0x56307b7b9ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:25.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.922+0000 7f51201e7740 1 --2- 172.21.15.83:0/1311521355 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56307b7b0b60 0x56307b7a9b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:25.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.922+0000 7f51201e7740 1 --2- 172.21.15.83:0/1311521355 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56307b7b0240 0x56307b7a95e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:25.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.922+0000 7f51201e7740 1 -- 172.21.15.83:0/1311521355 >> 172.21.15.83:0/1311521355 conn(0x56307b7ac560 msgr2=0x56307b7a2fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:25.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.922+0000 7f50f7e00640 0 client.28168 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:06:25.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.922+0000 7f51201e7740 1 -- 172.21.15.83:0/1311521355 shutdown_connections 2024-04-24T09:06:25.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.922+0000 7f51201e7740 1 -- 172.21.15.83:0/1311521355 wait complete. 2024-04-24T09:06:25.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.922+0000 7f51201e7740 20 client.28168 trim_cache size 0 max 16384 2024-04-24T09:06:25.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.956+0000 7f51201e7740 1 Processor -- start 2024-04-24T09:06:25.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.956+0000 7f51201e7740 1 -- start start 2024-04-24T09:06:25.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.956+0000 7f51201e7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56307b7b0240 0x56307b7a7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:25.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.956+0000 7f51201e7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56307b7b0b60 0x56307b7b9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:25.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.956+0000 7f51201e7740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56307b7bb230 0x56307b7a9940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:25.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.956+0000 7f51201e7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f51181b5bc0 con 0x56307b7b0b60 2024-04-24T09:06:25.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.956+0000 7f51201e7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f51181b7880 con 0x56307b7b0240 2024-04-24T09:06:25.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.956+0000 7f51201e7740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f51181b6290 con 0x56307b7bb230 2024-04-24T09:06:25.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.957+0000 7f5116800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56307b7b0240 0x56307b7a7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:25.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.957+0000 7f5115e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56307b7b0b60 0x56307b7b9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:25.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.957+0000 7f5116800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56307b7b0240 0x56307b7a7cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:39148/0 (socket says 172.21.15.83:39148) 2024-04-24T09:06:25.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.957+0000 7f5116800640 1 -- 172.21.15.83:0/3171424518 learned_addr learned my addr 172.21.15.83:0/3171424518 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:06:25.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.957+0000 7f5117200640 1 --2- 172.21.15.83:0/3171424518 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56307b7bb230 0x56307b7a9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:25.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.957+0000 7f5115e00640 1 -- 172.21.15.83:0/3171424518 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56307b7bb230 msgr2=0x56307b7a9940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:25.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.957+0000 7f5115e00640 1 --2- 172.21.15.83:0/3171424518 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56307b7bb230 0x56307b7a9940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:25.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.957+0000 7f5115e00640 1 -- 172.21.15.83:0/3171424518 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56307b7b0240 msgr2=0x56307b7a7cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:25.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.957+0000 7f5115e00640 1 --2- 172.21.15.83:0/3171424518 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56307b7b0240 0x56307b7a7cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:25.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.957+0000 7f5115e00640 1 -- 172.21.15.83:0/3171424518 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56307b41f3f0 con 0x56307b7b0b60 2024-04-24T09:06:25.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.958+0000 7f5115e00640 1 --2- 172.21.15.83:0/3171424518 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56307b7b0b60 0x56307b7b9cd0 secure :-1 s=READY pgs=2198 cs=0 l=1 rev1=1 crypto rx=0x7f5104009230 tx=0x7f510402a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62258c5515abbc7d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:25.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.958+0000 7f5102a00640 1 -- 172.21.15.83:0/3171424518 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5104049070 con 0x56307b7b0b60 2024-04-24T09:06:25.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.958+0000 7f5102a00640 1 -- 172.21.15.83:0/3171424518 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x56307b71ac10 con 0x56307b7b0b60 2024-04-24T09:06:25.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.958+0000 7f5102a00640 1 -- 172.21.15.83:0/3171424518 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5104035040 con 0x56307b7b0b60 2024-04-24T09:06:25.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.959+0000 7f51201e7740 1 -- 172.21.15.83:0/3171424518 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56307b7b0b60 msgr2=0x56307b7b9cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:25.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.959+0000 7f51201e7740 1 --2- 172.21.15.83:0/3171424518 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56307b7b0b60 0x56307b7b9cd0 secure :-1 s=READY pgs=2198 cs=0 l=1 rev1=1 crypto rx=0x7f5104009230 tx=0x7f510402a3f0 comp rx=0 tx=0).stop 2024-04-24T09:06:25.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.960+0000 7f51201e7740 1 -- 172.21.15.83:0/3171424518 shutdown_connections 2024-04-24T09:06:25.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.960+0000 7f51201e7740 1 --2- 172.21.15.83:0/3171424518 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56307b7bb230 0x56307b7a9940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:25.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.960+0000 7f51201e7740 1 --2- 172.21.15.83:0/3171424518 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56307b7b0b60 0x56307b7b9cd0 unknown :-1 s=CLOSED pgs=2198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:25.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.960+0000 7f51201e7740 1 --2- 172.21.15.83:0/3171424518 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56307b7b0240 0x56307b7a7cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:25.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.960+0000 7f51201e7740 1 -- 172.21.15.83:0/3171424518 >> 172.21.15.83:0/3171424518 conn(0x56307b7ac560 msgr2=0x56307b7a2fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:25.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.960+0000 7f51201e7740 1 -- 172.21.15.83:0/3171424518 shutdown_connections 2024-04-24T09:06:25.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.960+0000 7f51201e7740 1 -- 172.21.15.83:0/3171424518 wait complete. 2024-04-24T09:06:25.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.962+0000 7f51201e7740 1 -- start start 2024-04-24T09:06:25.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.962+0000 7f51201e7740 1 Processor -- start 2024-04-24T09:06:25.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.963+0000 7f51201e7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56307b7b0240 0x56307b7b51a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:25.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.963+0000 7f51201e7740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56307b7b0b60 0x56307b7b56e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:25.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.963+0000 7f51201e7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56307b7bb230 0x56307b7b5c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:25.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.963+0000 7f51201e7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f51181b70f0 con 0x56307b7bb230 2024-04-24T09:06:25.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.963+0000 7f51201e7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f51181b69c0 con 0x56307b7b0240 2024-04-24T09:06:25.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.963+0000 7f51201e7740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f51181b3be0 con 0x56307b7b0b60 2024-04-24T09:06:25.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.963+0000 7f51201e7740 10 client.0 _ll_register_callbacks cb 0x7f510800b590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:06:25.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.964+0000 7f5116800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56307b7b0240 0x56307b7b51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:25.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.964+0000 7f5116800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56307b7b0240 0x56307b7b51a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:39154/0 (socket says 172.21.15.83:39154) 2024-04-24T09:06:25.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.964+0000 7f5116800640 1 -- 172.21.15.83:0/3068666968 learned_addr learned my addr 172.21.15.83:0/3068666968 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:06:25.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.964+0000 7f5117200640 1 --2- 172.21.15.83:0/3068666968 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56307b7bb230 0x56307b7b5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:25.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.964+0000 7f5115e00640 1 --2- 172.21.15.83:0/3068666968 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56307b7b0b60 0x56307b7b56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:25.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.964+0000 7f5115e00640 1 -- 172.21.15.83:0/3068666968 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56307b7b0240 msgr2=0x56307b7b51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:25.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.964+0000 7f5115e00640 1 --2- 172.21.15.83:0/3068666968 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56307b7b0240 0x56307b7b51a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:25.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.964+0000 7f5115e00640 1 -- 172.21.15.83:0/3068666968 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56307b7bb230 msgr2=0x56307b7b5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:25.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.964+0000 7f5115e00640 1 --2- 172.21.15.83:0/3068666968 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56307b7bb230 0x56307b7b5c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:25.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.964+0000 7f5115e00640 1 -- 172.21.15.83:0/3068666968 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56307b3df0f0 con 0x56307b7b0b60 2024-04-24T09:06:25.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.964+0000 7f5116800640 1 --2- 172.21.15.83:0/3068666968 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56307b7b0240 0x56307b7b51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:06:25.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.965+0000 7f5115e00640 1 --2- 172.21.15.83:0/3068666968 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56307b7b0b60 0x56307b7b56e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x56307b7af990 tx=0x7f5104031400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e516ab6bd2494cb2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:25.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.965+0000 7f50f7e00640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-04-24T09:06:25.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.965+0000 7f50f7e00640 1 -- 172.21.15.83:0/3068666968 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5104035070 con 0x56307b7b0b60 2024-04-24T09:06:25.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.965+0000 7f50f7e00640 1 -- 172.21.15.83:0/3068666968 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9583+0+0 (secure 0 0 0) 0x7f51040316a0 con 0x56307b7b0b60 2024-04-24T09:06:25.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.965+0000 7f51201e7740 1 -- 172.21.15.83:0/3068666968 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56307b71ac10 con 0x56307b7b0b60 2024-04-24T09:06:25.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.966+0000 7f50f7e00640 1 -- 172.21.15.83:0/3068666968 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f5104049040 con 0x56307b7b0b60 2024-04-24T09:06:25.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.966+0000 7f51201e7740 10 client.28174 fetch_fsmap learned FSMap version 48 2024-04-24T09:06:25.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.966+0000 7f51201e7740 1 -- 172.21.15.83:0/3068666968 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x56307b6d2810 con 0x56307b7b0b60 2024-04-24T09:06:25.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.966+0000 7f50f7e00640 1 -- 172.21.15.83:0/3068666968 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x56307b71ac10 con 0x56307b7b0b60 2024-04-24T09:06:25.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.966+0000 7f50f7e00640 1 -- 172.21.15.83:0/3068666968 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 48) ==== 8402+0+0 (secure 0 0 0) 0x7f51040049d0 con 0x56307b7b0b60 2024-04-24T09:06:25.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.967+0000 7f51201e7740 10 client.28174 fetch_fsmap finished waiting for FSMap version 48 2024-04-24T09:06:25.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.967+0000 7f51201e7740 10 client.28174 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:06:25.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.967+0000 7f51201e7740 20 client.28174 populate_metadata read hostname 'smithi083' 2024-04-24T09:06:25.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.967+0000 7f51201e7740 1 --2- 172.21.15.83:0/3068666968 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56307b83aa50 0x56307b83ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:25.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.967+0000 7f51201e7740 4 client.28174 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T09:06:25.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.967+0000 7f51201e7740 1 -- 172.21.15.83:0/3068666968 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56307b714a80 con 0x56307b83aa50 2024-04-24T09:06:25.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.969+0000 7f5116800640 1 --2- 172.21.15.83:0/3068666968 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56307b83aa50 0x56307b83ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:25.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.970+0000 7f5116800640 1 --2- 172.21.15.83:0/3068666968 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56307b83aa50 0x56307b83ce70 crc :-1 s=READY pgs=384 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=deaaa841b172615 server_cookie=9a43e3c7553157f3 in_seq=0 out_seq=0 2024-04-24T09:06:25.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.970+0000 7f50f7e00640 10 client.28174 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:06:25.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.973+0000 7f50f7e00640 1 -- 172.21.15.83:0/3068666968 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56307b714a80 con 0x56307b83aa50 2024-04-24T09:06:25.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.973+0000 7f50f7e00640 10 client.28174 handle_command_reply: tid=0 2024-04-24T09:06:25.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.973+0000 7f51201e7740 1 client.28174 shutdown 2024-04-24T09:06:25.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.973+0000 7f51201e7740 1 -- 172.21.15.83:0/3068666968 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56307b7b0b60 msgr2=0x56307b7b56e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:25.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.973+0000 7f51201e7740 1 --2- 172.21.15.83:0/3068666968 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56307b7b0b60 0x56307b7b56e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x56307b7af990 tx=0x7f5104031400 comp rx=0 tx=0).stop 2024-04-24T09:06:25.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.973+0000 7f51201e7740 1 -- 172.21.15.83:0/3068666968 shutdown_connections 2024-04-24T09:06:25.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.973+0000 7f51201e7740 1 --2- 172.21.15.83:0/3068666968 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56307b83aa50 0x56307b83ce70 crc :-1 s=READY pgs=384 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:25.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.973+0000 7f51201e7740 1 --2- 172.21.15.83:0/3068666968 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56307b7bb230 0x56307b7b5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:25.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.973+0000 7f51201e7740 1 --2- 172.21.15.83:0/3068666968 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56307b7b0b60 0x56307b7b56e0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:25.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.973+0000 7f51201e7740 1 --2- 172.21.15.83:0/3068666968 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56307b7b0240 0x56307b7b51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:25.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.973+0000 7f51201e7740 1 -- 172.21.15.83:0/3068666968 >> 172.21.15.83:0/3068666968 conn(0x56307b7ac560 msgr2=0x56307b7a2fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:25.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.973+0000 7f50f7e00640 0 client.28174 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:06:25.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.973+0000 7f51201e7740 1 -- 172.21.15.83:0/3068666968 shutdown_connections 2024-04-24T09:06:25.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.973+0000 7f51201e7740 1 -- 172.21.15.83:0/3068666968 wait complete. 2024-04-24T09:06:25.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.974+0000 7f51201e7740 20 client.28174 trim_cache size 0 max 16384 2024-04-24T09:06:25.975 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T09:06:25.975 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T09:06:25.975 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-04-24T09:06:25.975 INFO:teuthology.orchestra.run.smithi083.stdout: "fbb102a6-193a-4cfb-bbb2-ca4e68e5dfe8": { 2024-04-24T09:06:25.975 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-04-24T09:06:25.975 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "fbb102a6-193a-4cfb-bbb2-ca4e68e5dfe8", 2024-04-24T09:06:25.975 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-04-24T09:06:25.975 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:06:25.975 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:06:25.975 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T09:06:25.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.975+0000 7f511ca00640 1 -- 172.21.15.83:0/624529597 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f50fc0a3e70 msgr2=0x7f50fc0a6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:25.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.975+0000 7f511ca00640 1 --2- 172.21.15.83:0/624529597 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f50fc0a3e70 0x7f50fc0a6330 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f5108009760 tx=0x7f5108006c90 comp rx=0 tx=0).stop 2024-04-24T09:06:25.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.975+0000 7f511ca00640 1 -- 172.21.15.83:0/624529597 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5118117000 msgr2=0x7f51181a7b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:25.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.975+0000 7f511ca00640 1 --2- 172.21.15.83:0/624529597 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5118117000 0x7f51181a7b60 secure :-1 s=READY pgs=2196 cs=0 l=1 rev1=1 crypto rx=0x7f511000caf0 tx=0x7f5110002170 comp rx=0 tx=0).stop 2024-04-24T09:06:25.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.975+0000 7f511ca00640 1 -- 172.21.15.83:0/624529597 shutdown_connections 2024-04-24T09:06:25.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.975+0000 7f511ca00640 1 --2- 172.21.15.83:0/624529597 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f50fc0a3e70 0x7f50fc0a6330 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:25.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.975+0000 7f511ca00640 1 --2- 172.21.15.83:0/624529597 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5118117000 0x7f51181a7b60 unknown :-1 s=CLOSED pgs=2196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:25.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.975+0000 7f511ca00640 1 --2- 172.21.15.83:0/624529597 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f511810c5b0 0x7f51181a37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:25.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.975+0000 7f511ca00640 1 --2- 172.21.15.83:0/624529597 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5118109540 0x7f51181a32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:25.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.975+0000 7f511ca00640 1 -- 172.21.15.83:0/624529597 >> 172.21.15.83:0/624529597 conn(0x7f511806cf80 msgr2=0x7f51181173e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:25.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.975+0000 7f511ca00640 1 -- 172.21.15.83:0/624529597 shutdown_connections 2024-04-24T09:06:25.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:25.975+0000 7f511ca00640 1 -- 172.21.15.83:0/624529597 wait complete. 2024-04-24T09:06:25.997 INFO:tasks.cephfs.filesystem:scrub status for tag:fbb102a6-193a-4cfb-bbb2-ca4e68e5dfe8 - {'path': '/', 'tag': 'fbb102a6-193a-4cfb-bbb2-ca4e68e5dfe8', 'options': 'recursive,force'} 2024-04-24T09:06:26.157 INFO:tasks.workunit.client.0.smithi083.stdout: 10 7315 10.04 MB/sec warmup 57 sec latency 259.225 ms 2024-04-24T09:06:26.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:06:26 smithi083 ceph-mon[27907]: pgmap v465: 129 pgs: 129 active+clean; 348 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 739 KiB/s rd, 3.6 MiB/s wr, 164 op/s 2024-04-24T09:06:26.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:06:26 smithi083 ceph-mon[27907]: scrub queued for path: / 2024-04-24T09:06:26.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:06:26 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:06:26.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:06:26 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:06:26.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:06:26 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:06:26.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:06:26 smithi138 ceph-mon[31441]: pgmap v465: 129 pgs: 129 active+clean; 348 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 739 KiB/s rd, 3.6 MiB/s wr, 164 op/s 2024-04-24T09:06:26.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:06:26 smithi138 ceph-mon[31441]: scrub queued for path: / 2024-04-24T09:06:26.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:06:26 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:06:26.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:06:26 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:06:26.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:06:26 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:06:26.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:06:26 smithi189 ceph-mon[31791]: pgmap v465: 129 pgs: 129 active+clean; 348 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 739 KiB/s rd, 3.6 MiB/s wr, 164 op/s 2024-04-24T09:06:26.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:06:26 smithi189 ceph-mon[31791]: scrub queued for path: / 2024-04-24T09:06:26.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:06:26 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:06:26.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:06:26 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:06:26.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:06:26 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:06:27.157 INFO:tasks.workunit.client.0.smithi083.stdout: 10 7437 9.98 MB/sec warmup 58 sec latency 56.833 ms 2024-04-24T09:06:27.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:06:27 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:06:27.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:06:27 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:06:27.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:06:27 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:06:27.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:06:27 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:06:27.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:06:27 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:06:27.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:06:27 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:06:27.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:06:27 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:06:27.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:06:27 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:06:27.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:06:27 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:06:27.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:06:27 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:06:27.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:06:27 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:06:27.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:06:27 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:06:27.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:06:27 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:06:27.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:06:27 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:06:27.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:06:27 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:06:27.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:06:27 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:06:27.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:06:27 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:06:27.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:06:27 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:06:27.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:06:27 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:06:27.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:06:27 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:06:27.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:06:27 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:06:27.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:06:27 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:06:27.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:06:27 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:06:27.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:06:27 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:06:27.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:06:27 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:06:27.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:06:27 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:06:27.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:06:27 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:06:27.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:06:27 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:06:27.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:06:27 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:06:27.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:06:27 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:06:28.157 INFO:tasks.workunit.client.0.smithi083.stdout: 10 7548 9.85 MB/sec warmup 59 sec latency 134.420 ms 2024-04-24T09:06:28.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:06:28 smithi083 ceph-mon[27907]: pgmap v466: 129 pgs: 129 active+clean; 348 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 739 KiB/s rd, 3.6 MiB/s wr, 164 op/s 2024-04-24T09:06:28.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:06:28 smithi138 ceph-mon[31441]: pgmap v466: 129 pgs: 129 active+clean; 348 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 739 KiB/s rd, 3.6 MiB/s wr, 164 op/s 2024-04-24T09:06:28.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:06:28 smithi189 ceph-mon[31791]: pgmap v466: 129 pgs: 129 active+clean; 348 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 739 KiB/s rd, 3.6 MiB/s wr, 164 op/s 2024-04-24T09:06:29.157 INFO:tasks.workunit.client.0.smithi083.stdout: 10 7702 9.88 MB/sec warmup 60 sec latency 80.856 ms 2024-04-24T09:06:30.157 INFO:tasks.workunit.client.0.smithi083.stdout: 10 7833 9.84 MB/sec warmup 61 sec latency 79.675 ms 2024-04-24T09:06:30.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:06:30 smithi083 ceph-mon[27907]: pgmap v467: 129 pgs: 129 active+clean; 353 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.1 MiB/s wr, 310 op/s 2024-04-24T09:06:30.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:06:30 smithi138 ceph-mon[31441]: pgmap v467: 129 pgs: 129 active+clean; 353 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.1 MiB/s wr, 310 op/s 2024-04-24T09:06:30.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:06:30 smithi189 ceph-mon[31791]: pgmap v467: 129 pgs: 129 active+clean; 353 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.1 MiB/s wr, 310 op/s 2024-04-24T09:06:31.157 INFO:tasks.workunit.client.0.smithi083.stdout: 10 7960 9.80 MB/sec warmup 62 sec latency 96.935 ms 2024-04-24T09:06:32.157 INFO:tasks.workunit.client.0.smithi083.stdout: 10 8080 9.68 MB/sec warmup 63 sec latency 79.085 ms 2024-04-24T09:06:32.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:06:32 smithi083 ceph-mon[27907]: pgmap v468: 129 pgs: 129 active+clean; 353 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.6 MiB/s wr, 232 op/s 2024-04-24T09:06:32.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:06:32 smithi138 ceph-mon[31441]: pgmap v468: 129 pgs: 129 active+clean; 353 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.6 MiB/s wr, 232 op/s 2024-04-24T09:06:32.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:06:32 smithi189 ceph-mon[31791]: pgmap v468: 129 pgs: 129 active+clean; 353 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.6 MiB/s wr, 232 op/s 2024-04-24T09:06:33.157 INFO:tasks.workunit.client.0.smithi083.stdout: 10 8238 9.73 MB/sec warmup 64 sec latency 79.046 ms 2024-04-24T09:06:34.157 INFO:tasks.workunit.client.0.smithi083.stdout: 10 8363 9.60 MB/sec warmup 65 sec latency 64.622 ms 2024-04-24T09:06:34.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:06:34 smithi083 ceph-mon[27907]: pgmap v469: 129 pgs: 129 active+clean; 350 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.3 MiB/s wr, 345 op/s 2024-04-24T09:06:34.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:06:34 smithi138 ceph-mon[31441]: pgmap v469: 129 pgs: 129 active+clean; 350 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.3 MiB/s wr, 345 op/s 2024-04-24T09:06:34.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:06:34 smithi189 ceph-mon[31791]: pgmap v469: 129 pgs: 129 active+clean; 350 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.3 MiB/s wr, 345 op/s 2024-04-24T09:06:35.157 INFO:tasks.workunit.client.0.smithi083.stdout: 10 8497 9.49 MB/sec warmup 66 sec latency 304.240 ms 2024-04-24T09:06:36.157 INFO:tasks.workunit.client.0.smithi083.stdout: 10 8635 9.42 MB/sec warmup 67 sec latency 519.948 ms 2024-04-24T09:06:36.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:06:36 smithi083 ceph-mon[27907]: pgmap v470: 129 pgs: 129 active+clean; 350 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.2 MiB/s wr, 259 op/s 2024-04-24T09:06:36.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:06:36 smithi138 ceph-mon[31441]: pgmap v470: 129 pgs: 129 active+clean; 350 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.2 MiB/s wr, 259 op/s 2024-04-24T09:06:36.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:06:36 smithi189 ceph-mon[31791]: pgmap v470: 129 pgs: 129 active+clean; 350 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.2 MiB/s wr, 259 op/s 2024-04-24T09:06:37.157 INFO:tasks.workunit.client.0.smithi083.stdout: 10 8771 9.31 MB/sec warmup 68 sec latency 83.183 ms 2024-04-24T09:06:37.611 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.612+0000 7f92ab000640 1 -- 172.21.15.83:0/610990176 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92a4076530 msgr2=0x7f92a4076910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:37.611 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.612+0000 7f92ab000640 1 --2- 172.21.15.83:0/610990176 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92a4076530 0x7f92a4076910 secure :-1 s=READY pgs=2199 cs=0 l=1 rev1=1 crypto rx=0x7f9298009900 tx=0x7f929802a3f0 comp rx=0 tx=0).stop 2024-04-24T09:06:37.611 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.612+0000 7f92ab000640 1 -- 172.21.15.83:0/610990176 shutdown_connections 2024-04-24T09:06:37.612 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.612+0000 7f92ab000640 1 --2- 172.21.15.83:0/610990176 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f92a4112cd0 0x7f92a41150c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:37.612 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.612+0000 7f92ab000640 1 --2- 172.21.15.83:0/610990176 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f92a4076e50 0x7f92a4112790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:37.612 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.612+0000 7f92ab000640 1 --2- 172.21.15.83:0/610990176 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92a4076530 0x7f92a4076910 unknown :-1 s=CLOSED pgs=2199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:37.612 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.612+0000 7f92ab000640 1 -- 172.21.15.83:0/610990176 >> 172.21.15.83:0/610990176 conn(0x7f92a40ffae0 msgr2=0x7f92a4101f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:37.612 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.612+0000 7f92ab000640 1 -- 172.21.15.83:0/610990176 shutdown_connections 2024-04-24T09:06:37.612 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.612+0000 7f92ab000640 1 -- 172.21.15.83:0/610990176 wait complete. 2024-04-24T09:06:37.613 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.613+0000 7f92ab000640 1 Processor -- start 2024-04-24T09:06:37.613 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.614+0000 7f92ab000640 1 -- start start 2024-04-24T09:06:37.613 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.614+0000 7f92ab000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f92a4076530 0x7f92a41a4910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:37.613 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.614+0000 7f92ab000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92a4076e50 0x7f92a41a4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:37.613 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.614+0000 7f92ab000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f92a4112cd0 0x7f92a419eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:37.613 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.614+0000 7f92ab000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f92a4118740 con 0x7f92a4076e50 2024-04-24T09:06:37.614 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.614+0000 7f92ab000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f92a41185c0 con 0x7f92a4112cd0 2024-04-24T09:06:37.614 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.614+0000 7f92ab000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f92a41188c0 con 0x7f92a4076530 2024-04-24T09:06:37.614 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.615+0000 7f92a9c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f92a4076530 0x7f92a41a4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:37.614 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.615+0000 7f92a9200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92a4076e50 0x7f92a41a4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:37.614 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.615+0000 7f92aa600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f92a4112cd0 0x7f92a419eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:37.615 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.615+0000 7f92a9c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f92a4076530 0x7f92a41a4910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:42074/0 (socket says 172.21.15.83:42074) 2024-04-24T09:06:37.615 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.615+0000 7f92a9c00640 1 -- 172.21.15.83:0/2659501389 learned_addr learned my addr 172.21.15.83:0/2659501389 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:06:37.615 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.615+0000 7f92aa600640 1 -- 172.21.15.83:0/2659501389 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f92a4076530 msgr2=0x7f92a41a4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:37.615 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.615+0000 7f92aa600640 1 --2- 172.21.15.83:0/2659501389 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f92a4076530 0x7f92a41a4910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:37.615 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.616+0000 7f92aa600640 1 -- 172.21.15.83:0/2659501389 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92a4076e50 msgr2=0x7f92a41a4e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:37.615 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.616+0000 7f92aa600640 1 --2- 172.21.15.83:0/2659501389 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92a4076e50 0x7f92a41a4e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:37.615 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.616+0000 7f92aa600640 1 -- 172.21.15.83:0/2659501389 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92a419f240 con 0x7f92a4112cd0 2024-04-24T09:06:37.615 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.616+0000 7f92a9c00640 1 --2- 172.21.15.83:0/2659501389 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f92a4076530 0x7f92a41a4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:06:37.616 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.616+0000 7f92a9200640 1 --2- 172.21.15.83:0/2659501389 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92a4076e50 0x7f92a41a4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:06:37.616 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.616+0000 7f92aa600640 1 --2- 172.21.15.83:0/2659501389 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f92a4112cd0 0x7f92a419eac0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f929c005fd0 tx=0x7f929c00edd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:37.616 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.617+0000 7f92a2000640 1 -- 172.21.15.83:0/2659501389 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f929c004d20 con 0x7f92a4112cd0 2024-04-24T09:06:37.616 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.617+0000 7f92a2000640 1 -- 172.21.15.83:0/2659501389 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f929c005020 con 0x7f92a4112cd0 2024-04-24T09:06:37.616 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.617+0000 7f92ab000640 1 -- 172.21.15.83:0/2659501389 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92a419f530 con 0x7f92a4112cd0 2024-04-24T09:06:37.616 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.617+0000 7f92a2000640 1 -- 172.21.15.83:0/2659501389 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f929c0116b0 con 0x7f92a4112cd0 2024-04-24T09:06:37.617 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.617+0000 7f92ab000640 1 -- 172.21.15.83:0/2659501389 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92a41ab7d0 con 0x7f92a4112cd0 2024-04-24T09:06:37.620 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.622+0000 7f92a2000640 1 -- 172.21.15.83:0/2659501389 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f929c00cb20 con 0x7f92a4112cd0 2024-04-24T09:06:37.622 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.624+0000 7f92a2000640 1 --2- 172.21.15.83:0/2659501389 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f928c0a3d90 0x7f928c0a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:37.623 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.624+0000 7f92a2000640 1 -- 172.21.15.83:0/2659501389 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9583+0+0 (secure 0 0 0) 0x7f929c0d0c90 con 0x7f92a4112cd0 2024-04-24T09:06:37.623 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.624+0000 7f92a9c00640 1 --2- 172.21.15.83:0/2659501389 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f928c0a3d90 0x7f928c0a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:37.624 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.625+0000 7f92af467740 1 Processor -- start 2024-04-24T09:06:37.624 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.625+0000 7f92af467740 1 -- start start 2024-04-24T09:06:37.625 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.625+0000 7f92af467740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56264dff63c0 0x56264dff67a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:37.625 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.625+0000 7f92a9c00640 1 --2- 172.21.15.83:0/2659501389 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f928c0a3d90 0x7f928c0a6250 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f9298000c00 tx=0x7f9298005e00 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:37.625 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.625+0000 7f92af467740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56264dff6ce0 0x56264e001060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:37.625 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.625+0000 7f92af467740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56264e0015a0 0x56264e003990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:37.625 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.625+0000 7f92af467740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56264df5c900 con 0x56264dff6ce0 2024-04-24T09:06:37.625 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.625+0000 7f92af467740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56264df02f20 con 0x56264e0015a0 2024-04-24T09:06:37.625 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.625+0000 7f92af467740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56264df588c0 con 0x56264dff63c0 2024-04-24T09:06:37.625 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.626+0000 7f92a9200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56264dff6ce0 0x56264e001060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:37.625 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.626+0000 7f92a9200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56264dff6ce0 0x56264e001060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36782/0 (socket says 172.21.15.83:36782) 2024-04-24T09:06:37.625 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.626+0000 7f92a9200640 1 -- 172.21.15.83:0/392843749 learned_addr learned my addr 172.21.15.83:0/392843749 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:06:37.625 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.626+0000 7f92aa600640 1 --2- 172.21.15.83:0/392843749 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56264e0015a0 0x56264e003990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:37.625 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.626+0000 7f92a9c00640 1 --2- 172.21.15.83:0/392843749 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56264dff63c0 0x56264dff67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:37.626 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.626+0000 7f92a9200640 1 -- 172.21.15.83:0/392843749 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56264dff63c0 msgr2=0x56264dff67a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:37.626 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.626+0000 7f92a9200640 1 --2- 172.21.15.83:0/392843749 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56264dff63c0 0x56264dff67a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:37.627 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.626+0000 7f92a9200640 1 -- 172.21.15.83:0/392843749 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56264e0015a0 msgr2=0x56264e003990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:37.627 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.626+0000 7f92a9200640 1 --2- 172.21.15.83:0/392843749 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56264e0015a0 0x56264e003990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:37.627 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.626+0000 7f92a9200640 1 -- 172.21.15.83:0/392843749 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56264df27e10 con 0x56264dff6ce0 2024-04-24T09:06:37.627 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.627+0000 7f92a9200640 1 --2- 172.21.15.83:0/392843749 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56264dff6ce0 0x56264e001060 secure :-1 s=READY pgs=2200 cs=0 l=1 rev1=1 crypto rx=0x7f92900114f0 tx=0x7f92900119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5186624cd5272b7b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:37.627 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.627+0000 7f9296a00640 1 -- 172.21.15.83:0/392843749 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9290005020 con 0x56264dff6ce0 2024-04-24T09:06:37.627 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.627+0000 7f9296a00640 1 -- 172.21.15.83:0/392843749 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f92900047a0 con 0x56264dff6ce0 2024-04-24T09:06:37.627 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.627+0000 7f9296a00640 1 -- 172.21.15.83:0/392843749 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f929000f880 con 0x56264dff6ce0 2024-04-24T09:06:37.628 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.628+0000 7f92af467740 1 -- 172.21.15.83:0/392843749 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56264dff6ce0 msgr2=0x56264e001060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:37.628 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.628+0000 7f92af467740 1 --2- 172.21.15.83:0/392843749 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56264dff6ce0 0x56264e001060 secure :-1 s=READY pgs=2200 cs=0 l=1 rev1=1 crypto rx=0x7f92900114f0 tx=0x7f92900119b0 comp rx=0 tx=0).stop 2024-04-24T09:06:37.628 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.628+0000 7f92af467740 1 -- 172.21.15.83:0/392843749 shutdown_connections 2024-04-24T09:06:37.628 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.628+0000 7f92af467740 1 --2- 172.21.15.83:0/392843749 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56264e0015a0 0x56264e003990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:37.628 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.628+0000 7f92af467740 1 --2- 172.21.15.83:0/392843749 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56264dff6ce0 0x56264e001060 unknown :-1 s=CLOSED pgs=2200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:37.629 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.628+0000 7f92af467740 1 --2- 172.21.15.83:0/392843749 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56264dff63c0 0x56264dff67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:37.629 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.628+0000 7f92af467740 1 -- 172.21.15.83:0/392843749 >> 172.21.15.83:0/392843749 conn(0x56264dff28d0 msgr2=0x56264dff2cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:37.629 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.628+0000 7f92af467740 1 -- 172.21.15.83:0/392843749 shutdown_connections 2024-04-24T09:06:37.629 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.628+0000 7f92af467740 1 -- 172.21.15.83:0/392843749 wait complete. 2024-04-24T09:06:37.629 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.629+0000 7f92af467740 1 -- start start 2024-04-24T09:06:37.629 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.629+0000 7f92af467740 1 Processor -- start 2024-04-24T09:06:37.631 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.630+0000 7f92af467740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56264dff63c0 0x56264dffe480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:37.631 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.630+0000 7f92af467740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56264dff6ce0 0x56264dffe9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:37.631 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.630+0000 7f92af467740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56264e0015a0 0x56264dffef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:37.631 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.630+0000 7f92af467740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56264e004140 con 0x56264e0015a0 2024-04-24T09:06:37.631 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.630+0000 7f92af467740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56264df162c0 con 0x56264dff63c0 2024-04-24T09:06:37.631 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.630+0000 7f92af467740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56264e0042c0 con 0x56264dff6ce0 2024-04-24T09:06:37.631 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.630+0000 7f92af467740 10 client.0 _ll_register_callbacks cb 0x56264dc6ea00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:06:37.631 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.630+0000 7f92a9c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56264dff63c0 0x56264dffe480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:37.632 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.630+0000 7f92a9c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56264dff63c0 0x56264dffe480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:50728/0 (socket says 172.21.15.83:50728) 2024-04-24T09:06:37.632 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.630+0000 7f92a9c00640 1 -- 172.21.15.83:0/3072250871 learned_addr learned my addr 172.21.15.83:0/3072250871 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:06:37.632 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.631+0000 7f92a9200640 1 --2- 172.21.15.83:0/3072250871 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56264dff6ce0 0x56264dffe9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:37.632 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.631+0000 7f92aa600640 1 --2- 172.21.15.83:0/3072250871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56264e0015a0 0x56264dffef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:37.632 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.631+0000 7f92a9c00640 1 -- 172.21.15.83:0/3072250871 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56264dff6ce0 msgr2=0x56264dffe9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:37.632 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.631+0000 7f92a9c00640 1 --2- 172.21.15.83:0/3072250871 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56264dff6ce0 0x56264dffe9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:37.632 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.631+0000 7f92a9c00640 1 -- 172.21.15.83:0/3072250871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56264e0015a0 msgr2=0x56264dffef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:37.632 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.631+0000 7f92a9c00640 1 --2- 172.21.15.83:0/3072250871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56264e0015a0 0x56264dffef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:37.632 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.631+0000 7f92a9c00640 1 -- 172.21.15.83:0/3072250871 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56264df26fd0 con 0x56264dff63c0 2024-04-24T09:06:37.633 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.632+0000 7f92a9c00640 1 --2- 172.21.15.83:0/3072250871 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56264dff63c0 0x56264dffe480 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9298004540 tx=0x7f9298009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1052867972ab5e04 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:37.633 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.632+0000 7f928b400640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:06:37.633 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.632+0000 7f928b400640 1 -- 172.21.15.83:0/3072250871 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f929804d070 con 0x56264dff63c0 2024-04-24T09:06:37.633 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.632+0000 7f92af467740 1 -- 172.21.15.83:0/3072250871 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92900047a0 con 0x56264dff63c0 2024-04-24T09:06:37.633 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.632+0000 7f928b400640 1 -- 172.21.15.83:0/3072250871 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9583+0+0 (secure 0 0 0) 0x7f9298004ad0 con 0x56264dff63c0 2024-04-24T09:06:37.634 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.632+0000 7f928b400640 1 -- 172.21.15.83:0/3072250871 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f9298059050 con 0x56264dff63c0 2024-04-24T09:06:37.634 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.632+0000 7f92af467740 10 client.38060 fetch_fsmap learned FSMap version 48 2024-04-24T09:06:37.634 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.632+0000 7f92af467740 1 -- 172.21.15.83:0/3072250871 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x56264dc6f210 con 0x56264dff63c0 2024-04-24T09:06:37.634 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.633+0000 7f928b400640 1 -- 172.21.15.83:0/3072250871 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f92900047a0 con 0x56264dff63c0 2024-04-24T09:06:37.634 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.633+0000 7f928b400640 1 -- 172.21.15.83:0/3072250871 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 48) ==== 8402+0+0 (secure 0 0 0) 0x7f9298049460 con 0x56264dff63c0 2024-04-24T09:06:37.634 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.633+0000 7f92af467740 10 client.38060 fetch_fsmap finished waiting for FSMap version 48 2024-04-24T09:06:37.634 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.633+0000 7f92af467740 10 client.38060 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:06:37.635 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.633+0000 7f92af467740 20 client.38060 populate_metadata read hostname 'smithi083' 2024-04-24T09:06:37.635 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.633+0000 7f92af467740 1 --2- 172.21.15.83:0/3072250871 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56264e011730 0x56264e013b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:37.635 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.633+0000 7f92af467740 4 client.38060 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:06:37.635 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.633+0000 7f92af467740 1 -- 172.21.15.83:0/3072250871 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56264df6a320 con 0x56264e011730 2024-04-24T09:06:37.635 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.634+0000 7f92a9200640 1 --2- 172.21.15.83:0/3072250871 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56264e011730 0x56264e013b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:37.635 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.634+0000 7f92a9200640 1 --2- 172.21.15.83:0/3072250871 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56264e011730 0x56264e013b90 crc :-1 s=READY pgs=385 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89092ee5cfa956f2 server_cookie=b45c34bf51aa52f in_seq=0 out_seq=0 2024-04-24T09:06:37.635 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.634+0000 7f928b400640 10 client.38060 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:06:37.635 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.635+0000 7f928b400640 1 -- 172.21.15.83:0/3072250871 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56264df6a320 con 0x56264e011730 2024-04-24T09:06:37.635 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.635+0000 7f928b400640 10 client.38060 handle_command_reply: tid=0 2024-04-24T09:06:37.636 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.636+0000 7f92af467740 1 client.38060 shutdown 2024-04-24T09:06:37.636 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.636+0000 7f92af467740 1 -- 172.21.15.83:0/3072250871 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56264dff63c0 msgr2=0x56264dffe480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:37.636 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.636+0000 7f92af467740 1 --2- 172.21.15.83:0/3072250871 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56264dff63c0 0x56264dffe480 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9298004540 tx=0x7f9298009260 comp rx=0 tx=0).stop 2024-04-24T09:06:37.636 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.636+0000 7f92af467740 1 -- 172.21.15.83:0/3072250871 shutdown_connections 2024-04-24T09:06:37.636 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.636+0000 7f92af467740 1 --2- 172.21.15.83:0/3072250871 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56264e011730 0x56264e013b90 crc :-1 s=READY pgs=385 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:37.637 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.636+0000 7f92af467740 1 --2- 172.21.15.83:0/3072250871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56264e0015a0 0x56264dffef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:37.637 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.636+0000 7f92af467740 1 --2- 172.21.15.83:0/3072250871 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56264dff6ce0 0x56264dffe9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:37.637 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.636+0000 7f92af467740 1 --2- 172.21.15.83:0/3072250871 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56264dff63c0 0x56264dffe480 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:37.637 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.636+0000 7f92af467740 1 -- 172.21.15.83:0/3072250871 >> 172.21.15.83:0/3072250871 conn(0x56264dff28d0 msgr2=0x56264dff8600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:37.637 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.636+0000 7f928b400640 0 client.38060 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:06:37.637 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.637+0000 7f92af467740 1 -- 172.21.15.83:0/3072250871 shutdown_connections 2024-04-24T09:06:37.637 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.637+0000 7f92af467740 1 -- 172.21.15.83:0/3072250871 wait complete. 2024-04-24T09:06:37.637 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.637+0000 7f92af467740 20 client.38060 trim_cache size 0 max 16384 2024-04-24T09:06:37.662 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.664+0000 7f92af467740 1 Processor -- start 2024-04-24T09:06:37.663 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.664+0000 7f92af467740 1 -- start start 2024-04-24T09:06:37.663 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.664+0000 7f92af467740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56264dff63c0 0x56264e000bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:37.664 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.664+0000 7f92af467740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56264dff6ce0 0x56264dff8d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:37.664 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.664+0000 7f92af467740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56264e0015a0 0x56264dff1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:37.664 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.664+0000 7f92af467740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f92a41b5b70 con 0x56264dff6ce0 2024-04-24T09:06:37.664 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.664+0000 7f92af467740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f92a41b7830 con 0x56264dff63c0 2024-04-24T09:06:37.664 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.664+0000 7f92af467740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f92a41b6240 con 0x56264e0015a0 2024-04-24T09:06:37.664 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.664+0000 7f92a9c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56264dff63c0 0x56264e000bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:37.664 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.665+0000 7f92a9c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56264dff63c0 0x56264e000bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:50732/0 (socket says 172.21.15.83:50732) 2024-04-24T09:06:37.664 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.665+0000 7f92a9200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56264dff6ce0 0x56264dff8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:37.664 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.665+0000 7f92a9c00640 1 -- 172.21.15.83:0/2006325091 learned_addr learned my addr 172.21.15.83:0/2006325091 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:06:37.665 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.665+0000 7f92aa600640 1 --2- 172.21.15.83:0/2006325091 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56264e0015a0 0x56264dff1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:37.666 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.667+0000 7f92a9200640 1 -- 172.21.15.83:0/2006325091 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56264e0015a0 msgr2=0x56264dff1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:37.668 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.667+0000 7f92a9200640 1 --2- 172.21.15.83:0/2006325091 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56264e0015a0 0x56264dff1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:37.668 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.667+0000 7f92a9200640 1 -- 172.21.15.83:0/2006325091 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56264dff63c0 msgr2=0x56264e000bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:37.668 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.667+0000 7f92a9200640 1 --2- 172.21.15.83:0/2006325091 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56264dff63c0 0x56264e000bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:37.668 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.667+0000 7f92a9200640 1 -- 172.21.15.83:0/2006325091 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56264dff6b10 con 0x56264dff6ce0 2024-04-24T09:06:37.668 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.668+0000 7f92aa600640 1 --2- 172.21.15.83:0/2006325091 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56264e0015a0 0x56264dff1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:06:37.668 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.668+0000 7f92a9c00640 1 --2- 172.21.15.83:0/2006325091 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56264dff63c0 0x56264e000bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:06:37.668 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.669+0000 7f92a9200640 1 --2- 172.21.15.83:0/2006325091 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56264dff6ce0 0x56264dff8d80 secure :-1 s=READY pgs=2201 cs=0 l=1 rev1=1 crypto rx=0x7f92900114c0 tx=0x7f9290004ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=549668f06b2ec462 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:37.669 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.669+0000 7f9296000640 1 -- 172.21.15.83:0/2006325091 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9290029da0 con 0x56264dff6ce0 2024-04-24T09:06:37.669 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.671+0000 7f9296000640 1 -- 172.21.15.83:0/2006325091 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9290029700 con 0x56264dff6ce0 2024-04-24T09:06:37.669 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.671+0000 7f9296000640 1 -- 172.21.15.83:0/2006325091 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9290005210 con 0x56264dff6ce0 2024-04-24T09:06:37.671 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.672+0000 7f92af467740 1 -- 172.21.15.83:0/2006325091 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56264dff6ce0 msgr2=0x56264dff8d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:37.671 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.672+0000 7f92af467740 1 --2- 172.21.15.83:0/2006325091 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56264dff6ce0 0x56264dff8d80 secure :-1 s=READY pgs=2201 cs=0 l=1 rev1=1 crypto rx=0x7f92900114c0 tx=0x7f9290004ed0 comp rx=0 tx=0).stop 2024-04-24T09:06:37.673 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.673+0000 7f92af467740 1 -- 172.21.15.83:0/2006325091 shutdown_connections 2024-04-24T09:06:37.673 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.673+0000 7f92af467740 1 --2- 172.21.15.83:0/2006325091 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56264e0015a0 0x56264dff1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:37.673 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.673+0000 7f92af467740 1 --2- 172.21.15.83:0/2006325091 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56264dff6ce0 0x56264dff8d80 unknown :-1 s=CLOSED pgs=2201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:37.673 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.673+0000 7f92af467740 1 --2- 172.21.15.83:0/2006325091 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56264dff63c0 0x56264e000bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:37.673 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.673+0000 7f92af467740 1 -- 172.21.15.83:0/2006325091 >> 172.21.15.83:0/2006325091 conn(0x56264dff28d0 msgr2=0x56264dffecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:37.673 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.674+0000 7f92af467740 1 -- 172.21.15.83:0/2006325091 shutdown_connections 2024-04-24T09:06:37.673 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.674+0000 7f92af467740 1 -- 172.21.15.83:0/2006325091 wait complete. 2024-04-24T09:06:37.673 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.674+0000 7f92af467740 1 -- start start 2024-04-24T09:06:37.674 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.675+0000 7f92af467740 1 Processor -- start 2024-04-24T09:06:37.675 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.676+0000 7f92af467740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56264dff63c0 0x56264e073580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:37.676 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.677+0000 7f92af467740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56264dff6ce0 0x56264e009600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:37.677 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.677+0000 7f92af467740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56264e0015a0 0x56264e009b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:37.677 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.677+0000 7f92af467740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f92a41b70a0 con 0x56264dff6ce0 2024-04-24T09:06:37.677 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.677+0000 7f92af467740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f92a41b6970 con 0x56264e0015a0 2024-04-24T09:06:37.677 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.677+0000 7f92af467740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f92a41b3b70 con 0x56264dff63c0 2024-04-24T09:06:37.677 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.677+0000 7f92af467740 10 client.0 _ll_register_callbacks cb 0x7f929803f070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:06:37.677 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.678+0000 7f92aa600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56264e0015a0 0x56264e009b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:37.677 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.678+0000 7f92aa600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56264e0015a0 0x56264e009b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:50742/0 (socket says 172.21.15.83:50742) 2024-04-24T09:06:37.677 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.678+0000 7f92aa600640 1 -- 172.21.15.83:0/1076696371 learned_addr learned my addr 172.21.15.83:0/1076696371 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:06:37.678 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.678+0000 7f92a9200640 1 --2- 172.21.15.83:0/1076696371 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56264dff6ce0 0x56264e009600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:37.678 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.678+0000 7f92aa600640 1 -- 172.21.15.83:0/1076696371 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56264dff63c0 msgr2=0x56264e073580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:37.678 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.678+0000 7f92aa600640 1 --2- 172.21.15.83:0/1076696371 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56264dff63c0 0x56264e073580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:37.679 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.679+0000 7f92aa600640 1 -- 172.21.15.83:0/1076696371 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56264dff6ce0 msgr2=0x56264e009600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:37.679 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.679+0000 7f92aa600640 1 --2- 172.21.15.83:0/1076696371 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56264dff6ce0 0x56264e009600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:37.679 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.679+0000 7f92aa600640 1 -- 172.21.15.83:0/1076696371 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56264dd3b620 con 0x56264e0015a0 2024-04-24T09:06:37.679 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.679+0000 7f92aa600640 1 --2- 172.21.15.83:0/1076696371 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56264e0015a0 0x56264e009b40 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f929c098c10 tx=0x7f929c09c240 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dbda59a3490005c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:37.679 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.679+0000 7f928b400640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:06:37.679 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.679+0000 7f928b400640 1 -- 172.21.15.83:0/1076696371 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f929c01f040 con 0x56264e0015a0 2024-04-24T09:06:37.679 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.679+0000 7f928b400640 1 -- 172.21.15.83:0/1076696371 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9583+0+0 (secure 0 0 0) 0x7f929c0ce1e0 con 0x56264e0015a0 2024-04-24T09:06:37.679 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.679+0000 7f92af467740 1 -- 172.21.15.83:0/1076696371 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9290029700 con 0x56264e0015a0 2024-04-24T09:06:37.679 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.680+0000 7f928b400640 1 -- 172.21.15.83:0/1076696371 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f929c09ceb0 con 0x56264e0015a0 2024-04-24T09:06:37.680 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.680+0000 7f92af467740 10 client.38066 fetch_fsmap learned FSMap version 48 2024-04-24T09:06:37.680 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.680+0000 7f92af467740 1 -- 172.21.15.83:0/1076696371 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x56264e088f90 con 0x56264e0015a0 2024-04-24T09:06:37.680 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.681+0000 7f928b400640 1 -- 172.21.15.83:0/1076696371 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9290029700 con 0x56264e0015a0 2024-04-24T09:06:37.680 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.681+0000 7f928b400640 1 -- 172.21.15.83:0/1076696371 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 48) ==== 8402+0+0 (secure 0 0 0) 0x7f929c00f040 con 0x56264e0015a0 2024-04-24T09:06:37.681 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.681+0000 7f92af467740 10 client.38066 fetch_fsmap finished waiting for FSMap version 48 2024-04-24T09:06:37.681 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.681+0000 7f92af467740 10 client.38066 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:06:37.681 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.682+0000 7f92af467740 20 client.38066 populate_metadata read hostname 'smithi083' 2024-04-24T09:06:37.681 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.682+0000 7f92af467740 1 --2- 172.21.15.83:0/1076696371 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56264e089770 0x56264e08bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:37.681 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.682+0000 7f92af467740 4 client.38066 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T09:06:37.681 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.682+0000 7f92af467740 1 -- 172.21.15.83:0/1076696371 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56264df64190 con 0x56264e089770 2024-04-24T09:06:37.681 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.682+0000 7f92a9c00640 1 --2- 172.21.15.83:0/1076696371 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56264e089770 0x56264e08bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:37.682 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.683+0000 7f92a9c00640 1 --2- 172.21.15.83:0/1076696371 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56264e089770 0x56264e08bb90 crc :-1 s=READY pgs=386 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ad25e4f8712a0b4 server_cookie=51aeccf50f69f834 in_seq=0 out_seq=0 2024-04-24T09:06:37.682 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.683+0000 7f928b400640 10 client.38066 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:06:37.783 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.783+0000 7f928b400640 1 -- 172.21.15.83:0/1076696371 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56264df64190 con 0x56264e089770 2024-04-24T09:06:37.783 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.783+0000 7f928b400640 10 client.38066 handle_command_reply: tid=0 2024-04-24T09:06:37.783 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.784+0000 7f92af467740 1 client.38066 shutdown 2024-04-24T09:06:37.784 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.784+0000 7f92af467740 1 -- 172.21.15.83:0/1076696371 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56264e0015a0 msgr2=0x56264e009b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:37.784 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.784+0000 7f92af467740 1 --2- 172.21.15.83:0/1076696371 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56264e0015a0 0x56264e009b40 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f929c098c10 tx=0x7f929c09c240 comp rx=0 tx=0).stop 2024-04-24T09:06:37.784 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.784+0000 7f92af467740 1 -- 172.21.15.83:0/1076696371 shutdown_connections 2024-04-24T09:06:37.784 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.784+0000 7f92af467740 1 --2- 172.21.15.83:0/1076696371 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56264e089770 0x56264e08bb90 crc :-1 s=READY pgs=386 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:37.784 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.784+0000 7f92af467740 1 --2- 172.21.15.83:0/1076696371 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56264e0015a0 0x56264e009b40 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:37.784 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.784+0000 7f92af467740 1 --2- 172.21.15.83:0/1076696371 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56264dff6ce0 0x56264e009600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:37.784 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.784+0000 7f92af467740 1 --2- 172.21.15.83:0/1076696371 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56264dff63c0 0x56264e073580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:37.784 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.784+0000 7f92af467740 1 -- 172.21.15.83:0/1076696371 >> 172.21.15.83:0/1076696371 conn(0x56264dff28d0 msgr2=0x56264dff7d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:37.784 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.784+0000 7f928b400640 0 client.38066 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:06:37.784 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.785+0000 7f92af467740 1 -- 172.21.15.83:0/1076696371 shutdown_connections 2024-04-24T09:06:37.784 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.785+0000 7f92af467740 1 -- 172.21.15.83:0/1076696371 wait complete. 2024-04-24T09:06:37.784 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.785+0000 7f92af467740 20 client.38066 trim_cache size 0 max 16384 2024-04-24T09:06:37.785 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T09:06:37.785 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T09:06:37.785 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T09:06:37.785 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T09:06:37.786 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.786+0000 7f92ab000640 1 -- 172.21.15.83:0/2659501389 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f928c0a3d90 msgr2=0x7f928c0a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:37.786 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.786+0000 7f92ab000640 1 --2- 172.21.15.83:0/2659501389 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f928c0a3d90 0x7f928c0a6250 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f9298000c00 tx=0x7f9298005e00 comp rx=0 tx=0).stop 2024-04-24T09:06:37.786 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.786+0000 7f92ab000640 1 -- 172.21.15.83:0/2659501389 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f92a4112cd0 msgr2=0x7f92a419eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:37.786 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.786+0000 7f92ab000640 1 --2- 172.21.15.83:0/2659501389 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f92a4112cd0 0x7f92a419eac0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f929c005fd0 tx=0x7f929c00edd0 comp rx=0 tx=0).stop 2024-04-24T09:06:37.786 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.786+0000 7f92ab000640 1 -- 172.21.15.83:0/2659501389 shutdown_connections 2024-04-24T09:06:37.786 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.786+0000 7f92ab000640 1 --2- 172.21.15.83:0/2659501389 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f928c0a3d90 0x7f928c0a6250 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:37.786 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.786+0000 7f92ab000640 1 --2- 172.21.15.83:0/2659501389 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f92a4112cd0 0x7f92a419eac0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:37.786 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.786+0000 7f92ab000640 1 --2- 172.21.15.83:0/2659501389 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92a4076e50 0x7f92a41a4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:37.786 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.786+0000 7f92ab000640 1 --2- 172.21.15.83:0/2659501389 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f92a4076530 0x7f92a41a4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:37.787 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.786+0000 7f92ab000640 1 -- 172.21.15.83:0/2659501389 >> 172.21.15.83:0/2659501389 conn(0x7f92a40ffae0 msgr2=0x7f92a4113940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:37.787 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.786+0000 7f92ab000640 1 -- 172.21.15.83:0/2659501389 shutdown_connections 2024-04-24T09:06:37.787 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:37.786+0000 7f92ab000640 1 -- 172.21.15.83:0/2659501389 wait complete. 2024-04-24T09:06:38.157 INFO:tasks.workunit.client.0.smithi083.stdout: 10 8915 9.33 MB/sec warmup 69 sec latency 106.941 ms 2024-04-24T09:06:38.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:06:38 smithi138 ceph-mon[31441]: pgmap v471: 129 pgs: 129 active+clean; 350 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.2 MiB/s wr, 260 op/s 2024-04-24T09:06:38.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:06:38 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:06:38.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:06:38 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:06:38.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:06:38 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:06:38.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:06:38 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:06:38.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:06:38 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:06:38.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:06:38 smithi189 ceph-mon[31791]: pgmap v471: 129 pgs: 129 active+clean; 350 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.2 MiB/s wr, 260 op/s 2024-04-24T09:06:38.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:06:38 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:06:38.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:06:38 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:06:38.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:06:38 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:06:38.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:06:38 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:06:38.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:06:38 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:06:38.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:06:38 smithi083 ceph-mon[27907]: pgmap v471: 129 pgs: 129 active+clean; 350 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.2 MiB/s wr, 260 op/s 2024-04-24T09:06:38.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:06:38 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:06:38.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:06:38 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:06:38.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:06:38 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:06:38.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:06:38 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:06:38.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:06:38 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:06:39.157 INFO:tasks.workunit.client.0.smithi083.stdout: 10 9076 9.33 MB/sec warmup 70 sec latency 65.922 ms 2024-04-24T09:06:40.157 INFO:tasks.workunit.client.0.smithi083.stdout: 10 9211 9.32 MB/sec warmup 71 sec latency 67.242 ms 2024-04-24T09:06:40.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:06:40 smithi138 ceph-mon[31441]: pgmap v472: 129 pgs: 129 active+clean; 351 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.3 MiB/s wr, 366 op/s 2024-04-24T09:06:40.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:06:40 smithi189 ceph-mon[31791]: pgmap v472: 129 pgs: 129 active+clean; 351 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.3 MiB/s wr, 366 op/s 2024-04-24T09:06:40.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:06:40 smithi083 ceph-mon[27907]: pgmap v472: 129 pgs: 129 active+clean; 351 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.3 MiB/s wr, 366 op/s 2024-04-24T09:06:41.158 INFO:tasks.workunit.client.0.smithi083.stdout: 10 9368 9.33 MB/sec warmup 72 sec latency 75.247 ms 2024-04-24T09:06:42.158 INFO:tasks.workunit.client.0.smithi083.stdout: 10 9553 9.46 MB/sec warmup 73 sec latency 81.100 ms 2024-04-24T09:06:42.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:06:42 smithi138 ceph-mon[31441]: pgmap v473: 129 pgs: 129 active+clean; 351 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 968 KiB/s rd, 2.8 MiB/s wr, 220 op/s 2024-04-24T09:06:42.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:06:42 smithi189 ceph-mon[31791]: pgmap v473: 129 pgs: 129 active+clean; 351 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 968 KiB/s rd, 2.8 MiB/s wr, 220 op/s 2024-04-24T09:06:42.552 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:06:42 smithi083 ceph-mon[27907]: pgmap v473: 129 pgs: 129 active+clean; 351 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 968 KiB/s rd, 2.8 MiB/s wr, 220 op/s 2024-04-24T09:06:43.158 INFO:tasks.workunit.client.0.smithi083.stdout: 10 9734 9.40 MB/sec warmup 74 sec latency 73.625 ms 2024-04-24T09:06:44.158 INFO:tasks.workunit.client.0.smithi083.stdout: 10 9855 9.39 MB/sec warmup 75 sec latency 124.597 ms 2024-04-24T09:06:44.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:06:44 smithi138 ceph-mon[31441]: pgmap v474: 129 pgs: 129 active+clean; 357 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.8 MiB/s wr, 346 op/s 2024-04-24T09:06:44.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:06:44 smithi189 ceph-mon[31791]: pgmap v474: 129 pgs: 129 active+clean; 357 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.8 MiB/s wr, 346 op/s 2024-04-24T09:06:44.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:06:44 smithi083 ceph-mon[27907]: pgmap v474: 129 pgs: 129 active+clean; 357 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.8 MiB/s wr, 346 op/s 2024-04-24T09:06:45.158 INFO:tasks.workunit.client.0.smithi083.stdout: 10 9913 9.29 MB/sec warmup 76 sec latency 422.469 ms 2024-04-24T09:06:46.159 INFO:tasks.workunit.client.0.smithi083.stdout: 10 10010 9.23 MB/sec warmup 77 sec latency 67.286 ms 2024-04-24T09:06:46.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:06:46 smithi138 ceph-mon[31441]: pgmap v475: 129 pgs: 129 active+clean; 357 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 888 KiB/s rd, 3.1 MiB/s wr, 232 op/s 2024-04-24T09:06:46.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:06:46 smithi189 ceph-mon[31791]: pgmap v475: 129 pgs: 129 active+clean; 357 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 888 KiB/s rd, 3.1 MiB/s wr, 232 op/s 2024-04-24T09:06:46.552 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:06:46 smithi083 ceph-mon[27907]: pgmap v475: 129 pgs: 129 active+clean; 357 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 888 KiB/s rd, 3.1 MiB/s wr, 232 op/s 2024-04-24T09:06:47.158 INFO:tasks.workunit.client.0.smithi083.stdout: 10 10166 9.30 MB/sec warmup 78 sec latency 84.061 ms 2024-04-24T09:06:48.159 INFO:tasks.workunit.client.0.smithi083.stdout: 10 10307 9.28 MB/sec warmup 79 sec latency 76.525 ms 2024-04-24T09:06:48.354 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:06:48 smithi083 ceph-mon[27907]: pgmap v476: 129 pgs: 129 active+clean; 357 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 888 KiB/s rd, 3.1 MiB/s wr, 232 op/s 2024-04-24T09:06:48.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:06:48 smithi138 ceph-mon[31441]: pgmap v476: 129 pgs: 129 active+clean; 357 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 888 KiB/s rd, 3.1 MiB/s wr, 232 op/s 2024-04-24T09:06:48.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:06:48 smithi189 ceph-mon[31791]: pgmap v476: 129 pgs: 129 active+clean; 357 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 888 KiB/s rd, 3.1 MiB/s wr, 232 op/s 2024-04-24T09:06:49.159 INFO:tasks.workunit.client.0.smithi083.stdout: 10 10428 9.23 MB/sec warmup 80 sec latency 83.131 ms 2024-04-24T09:06:50.159 INFO:tasks.workunit.client.0.smithi083.stdout: 10 10566 9.17 MB/sec warmup 81 sec latency 70.318 ms 2024-04-24T09:06:50.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:06:50 smithi138 ceph-mon[31441]: pgmap v477: 129 pgs: 129 active+clean; 364 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.5 MiB/s wr, 317 op/s 2024-04-24T09:06:50.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:06:50 smithi189 ceph-mon[31791]: pgmap v477: 129 pgs: 129 active+clean; 364 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.5 MiB/s wr, 317 op/s 2024-04-24T09:06:50.552 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:06:50 smithi083 ceph-mon[27907]: pgmap v477: 129 pgs: 129 active+clean; 364 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.5 MiB/s wr, 317 op/s 2024-04-24T09:06:50.983 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:50.983+0000 7f7bf7200640 1 -- 172.21.15.83:0/3659987193 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7bf8109380 msgr2=0x7f7bf8110f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:50.984 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:50.983+0000 7f7bf7200640 1 --2- 172.21.15.83:0/3659987193 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7bf8109380 0x7f7bf8110f50 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7bf0009be0 tx=0x7f7bf002a6c0 comp rx=0 tx=0).stop 2024-04-24T09:06:50.984 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:50.983+0000 7f7bf7200640 1 -- 172.21.15.83:0/3659987193 shutdown_connections 2024-04-24T09:06:50.984 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:50.983+0000 7f7bf7200640 1 --2- 172.21.15.83:0/3659987193 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7bf8109380 0x7f7bf8110f50 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:50.984 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:50.983+0000 7f7bf7200640 1 --2- 172.21.15.83:0/3659987193 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7bf8104c60 0x7f7bf8108ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:50.984 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:50.983+0000 7f7bf7200640 1 --2- 172.21.15.83:0/3659987193 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7bf81042b0 0x7f7bf8104690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:50.984 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:50.983+0000 7f7bf7200640 1 -- 172.21.15.83:0/3659987193 >> 172.21.15.83:0/3659987193 conn(0x7f7bf80ffaf0 msgr2=0x7f7bf8101f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:50.984 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:50.983+0000 7f7bf7200640 1 -- 172.21.15.83:0/3659987193 shutdown_connections 2024-04-24T09:06:50.984 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:50.983+0000 7f7bf7200640 1 -- 172.21.15.83:0/3659987193 wait complete. 2024-04-24T09:06:50.985 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:50.985+0000 7f7bf7200640 1 Processor -- start 2024-04-24T09:06:50.985 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:50.985+0000 7f7bf7200640 1 -- start start 2024-04-24T09:06:50.985 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:50.985+0000 7f7bf7200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7bf81042b0 0x7f7bf81a0800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:50.986 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:50.985+0000 7f7bf7200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7bf8109380 0x7f7bf81a0d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:50.986 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:50.985+0000 7f7bf7200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7bf819a9b0 0x7f7bf819ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:50.986 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:50.985+0000 7f7bf7200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf81142d0 con 0x7f7bf819a9b0 2024-04-24T09:06:50.986 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:50.985+0000 7f7bf7200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf8114150 con 0x7f7bf8109380 2024-04-24T09:06:50.986 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:50.985+0000 7f7bf7200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf8114450 con 0x7f7bf81042b0 2024-04-24T09:06:50.986 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:50.986+0000 7f7bf5400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7bf8109380 0x7f7bf81a0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:50.986 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:50.986+0000 7f7bf6800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7bf819a9b0 0x7f7bf819ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:50.986 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:50.986+0000 7f7bf6800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7bf819a9b0 0x7f7bf819ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50942/0 (socket says 172.21.15.83:50942) 2024-04-24T09:06:50.986 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:50.986+0000 7f7bf6800640 1 -- 172.21.15.83:0/3397161054 learned_addr learned my addr 172.21.15.83:0/3397161054 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:06:50.987 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:50.986+0000 7f7bf5e00640 1 --2- 172.21.15.83:0/3397161054 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7bf81042b0 0x7f7bf81a0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:50.988 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:50.986+0000 7f7bf6800640 1 -- 172.21.15.83:0/3397161054 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7bf81042b0 msgr2=0x7f7bf81a0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:50.988 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:50.986+0000 7f7bf6800640 1 --2- 172.21.15.83:0/3397161054 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7bf81042b0 0x7f7bf81a0800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:50.988 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:50.986+0000 7f7bf6800640 1 -- 172.21.15.83:0/3397161054 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7bf8109380 msgr2=0x7f7bf81a0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:50.988 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:50.986+0000 7f7bf6800640 1 --2- 172.21.15.83:0/3397161054 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7bf8109380 0x7f7bf81a0d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:50.988 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:50.986+0000 7f7bf6800640 1 -- 172.21.15.83:0/3397161054 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bf819b4d0 con 0x7f7bf819a9b0 2024-04-24T09:06:50.988 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:50.986+0000 7f7bf5400640 1 --2- 172.21.15.83:0/3397161054 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7bf8109380 0x7f7bf81a0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:06:50.988 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:50.987+0000 7f7bf6800640 1 --2- 172.21.15.83:0/3397161054 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7bf819a9b0 0x7f7bf819ae60 secure :-1 s=READY pgs=2202 cs=0 l=1 rev1=1 crypto rx=0x7f7bf0004af0 tx=0x7f7bf0003d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:50.988 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:50.987+0000 7f7beea00640 1 -- 172.21.15.83:0/3397161054 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bf0041020 con 0x7f7bf819a9b0 2024-04-24T09:06:50.988 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:50.988+0000 7f7beea00640 1 -- 172.21.15.83:0/3397161054 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7bf0037040 con 0x7f7bf819a9b0 2024-04-24T09:06:50.988 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:50.988+0000 7f7beea00640 1 -- 172.21.15.83:0/3397161054 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bf0036070 con 0x7f7bf819a9b0 2024-04-24T09:06:50.988 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:50.988+0000 7f7bf7200640 1 -- 172.21.15.83:0/3397161054 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bf819b760 con 0x7f7bf819a9b0 2024-04-24T09:06:50.988 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:50.988+0000 7f7bf7200640 1 -- 172.21.15.83:0/3397161054 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bf81a7900 con 0x7f7bf819a9b0 2024-04-24T09:06:50.992 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:50.992+0000 7f7bfffaf740 1 Processor -- start 2024-04-24T09:06:50.992 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:50.992+0000 7f7beea00640 1 -- 172.21.15.83:0/3397161054 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7bf0044020 con 0x7f7bf819a9b0 2024-04-24T09:06:50.993 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:50.993+0000 7f7bfffaf740 1 -- start start 2024-04-24T09:06:50.993 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:50.993+0000 7f7bfffaf740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561e474a7bd0 0x561e474a7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:50.993 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:50.993+0000 7f7bfffaf740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e474a84f0 0x561e474b2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:50.993 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:50.993+0000 7f7bfffaf740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561e474b2db0 0x561e474b51a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:50.994 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:50.993+0000 7f7bfffaf740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561e47405900 con 0x561e474b2db0 2024-04-24T09:06:50.994 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:50.993+0000 7f7bfffaf740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x561e473abf20 con 0x561e474a84f0 2024-04-24T09:06:50.994 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:50.993+0000 7f7bfffaf740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x561e474018c0 con 0x561e474a7bd0 2024-04-24T09:06:50.994 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:50.994+0000 7f7bf5e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e474a84f0 0x561e474b2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:50.995 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:50.994+0000 7f7bf5e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e474a84f0 0x561e474b2870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:56678/0 (socket says 172.21.15.83:56678) 2024-04-24T09:06:50.995 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:50.994+0000 7f7bf5400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561e474a7bd0 0x561e474a7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:50.995 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:50.994+0000 7f7bf5e00640 1 -- 172.21.15.83:0/2962750350 learned_addr learned my addr 172.21.15.83:0/2962750350 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:06:50.995 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:50.994+0000 7f7bf5e00640 1 -- 172.21.15.83:0/2962750350 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561e474a7bd0 msgr2=0x561e474a7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:50.995 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:50.994+0000 7f7bf5e00640 1 --2- 172.21.15.83:0/2962750350 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561e474a7bd0 0x561e474a7fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:50.995 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:50.994+0000 7f7bf5e00640 1 -- 172.21.15.83:0/2962750350 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561e474b2db0 msgr2=0x561e474b51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:50.995 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:50.994+0000 7f7bf5e00640 1 --2- 172.21.15.83:0/2962750350 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561e474b2db0 0x561e474b51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:50.995 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:50.994+0000 7f7bf5e00640 1 -- 172.21.15.83:0/2962750350 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e473d0e10 con 0x561e474a84f0 2024-04-24T09:06:50.995 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:50.995+0000 7f7beea00640 1 --2- 172.21.15.83:0/3397161054 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f7bdc0a3d90 0x7f7bdc0a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:50.995 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:50.995+0000 7f7beea00640 1 -- 172.21.15.83:0/3397161054 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9583+0+0 (secure 0 0 0) 0x7f7bf00f1490 con 0x7f7bf819a9b0 2024-04-24T09:06:50.996 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:50.995+0000 7f7bf5400640 1 --2- 172.21.15.83:0/3397161054 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f7bdc0a3d90 0x7f7bdc0a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:50.996 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:50.995+0000 7f7bf5e00640 1 --2- 172.21.15.83:0/2962750350 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e474a84f0 0x561e474b2870 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f7be800bc90 tx=0x7f7be800f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3a73ce7a2856a99 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:50.996 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:50.995+0000 7f7be3400640 1 -- 172.21.15.83:0/2962750350 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be8015070 con 0x561e474a84f0 2024-04-24T09:06:50.997 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:50.995+0000 7f7be3400640 1 -- 172.21.15.83:0/2962750350 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7be801a910 con 0x561e474a84f0 2024-04-24T09:06:50.997 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:50.996+0000 7f7be3400640 1 -- 172.21.15.83:0/2962750350 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be8011900 con 0x561e474a84f0 2024-04-24T09:06:50.997 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:50.997+0000 7f7bfffaf740 1 -- 172.21.15.83:0/2962750350 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e474a84f0 msgr2=0x561e474b2870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:50.997 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:50.997+0000 7f7bf5400640 1 --2- 172.21.15.83:0/3397161054 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f7bdc0a3d90 0x7f7bdc0a6250 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x561e474a5500 tx=0x7f7be4014040 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:50.997 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:50.997+0000 7f7bfffaf740 1 --2- 172.21.15.83:0/2962750350 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e474a84f0 0x561e474b2870 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f7be800bc90 tx=0x7f7be800f730 comp rx=0 tx=0).stop 2024-04-24T09:06:50.998 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:50.997+0000 7f7bfffaf740 1 -- 172.21.15.83:0/2962750350 shutdown_connections 2024-04-24T09:06:50.998 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:50.997+0000 7f7bfffaf740 1 --2- 172.21.15.83:0/2962750350 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561e474b2db0 0x561e474b51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:50.998 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:50.997+0000 7f7bfffaf740 1 --2- 172.21.15.83:0/2962750350 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e474a84f0 0x561e474b2870 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:50.998 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:50.997+0000 7f7bfffaf740 1 --2- 172.21.15.83:0/2962750350 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561e474a7bd0 0x561e474a7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:50.998 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:50.997+0000 7f7bfffaf740 1 -- 172.21.15.83:0/2962750350 >> 172.21.15.83:0/2962750350 conn(0x561e474a40e0 msgr2=0x561e474a44c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:50.998 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:50.997+0000 7f7bfffaf740 1 -- 172.21.15.83:0/2962750350 shutdown_connections 2024-04-24T09:06:50.998 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:50.998+0000 7f7bfffaf740 1 -- 172.21.15.83:0/2962750350 wait complete. 2024-04-24T09:06:50.998 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:50.998+0000 7f7bfffaf740 1 -- start start 2024-04-24T09:06:50.999 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:50.998+0000 7f7bfffaf740 1 Processor -- start 2024-04-24T09:06:50.999 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:50.999+0000 7f7bfffaf740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561e474a7bd0 0x561e474afd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:50.999 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:50.999+0000 7f7bfffaf740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561e474a84f0 0x561e474b0250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:50.999 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:50.999+0000 7f7bfffaf740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e474b2db0 0x561e474b0790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:50.999 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:50.999+0000 7f7bfffaf740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561e474b59e0 con 0x561e474a7bd0 2024-04-24T09:06:50.999 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:50.999+0000 7f7bfffaf740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x561e473bf2c0 con 0x561e474b2db0 2024-04-24T09:06:50.999 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:50.999+0000 7f7bfffaf740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x561e474b5b60 con 0x561e474a84f0 2024-04-24T09:06:51.000 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:50.999+0000 7f7bfffaf740 10 client.0 _ll_register_callbacks cb 0x561e47117a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:06:51.000 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:50.999+0000 7f7bf5e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561e474a7bd0 0x561e474afd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:51.000 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.000+0000 7f7bf5e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561e474a7bd0 0x561e474afd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50968/0 (socket says 172.21.15.83:50968) 2024-04-24T09:06:51.001 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.000+0000 7f7bf5e00640 1 -- 172.21.15.83:0/2054537721 learned_addr learned my addr 172.21.15.83:0/2054537721 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:06:51.001 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.000+0000 7f7bf6800640 1 --2- 172.21.15.83:0/2054537721 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e474b2db0 0x561e474b0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:51.001 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.000+0000 7f7bf5400640 1 --2- 172.21.15.83:0/2054537721 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561e474a84f0 0x561e474b0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:51.001 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.000+0000 7f7bf5e00640 1 -- 172.21.15.83:0/2054537721 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561e474a84f0 msgr2=0x561e474b0250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:51.001 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.000+0000 7f7bf5e00640 1 --2- 172.21.15.83:0/2054537721 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561e474a84f0 0x561e474b0250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:51.001 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.000+0000 7f7bf5e00640 1 -- 172.21.15.83:0/2054537721 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e474b2db0 msgr2=0x561e474b0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:51.001 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.000+0000 7f7bf5e00640 1 --2- 172.21.15.83:0/2054537721 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e474b2db0 0x561e474b0790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:51.001 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.000+0000 7f7bf5e00640 1 -- 172.21.15.83:0/2054537721 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e473cffd0 con 0x561e474a7bd0 2024-04-24T09:06:51.001 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.000+0000 7f7bf6800640 1 --2- 172.21.15.83:0/2054537721 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e474b2db0 0x561e474b0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:06:51.001 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.001+0000 7f7bf5e00640 1 --2- 172.21.15.83:0/2054537721 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561e474a7bd0 0x561e474afd10 secure :-1 s=READY pgs=2203 cs=0 l=1 rev1=1 crypto rx=0x7f7be8011480 tx=0x7f7be8005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3aeb0d57cfc6a32c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:51.001 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.001+0000 7f7bd7e00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:06:51.002 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.001+0000 7f7bd7e00640 1 -- 172.21.15.83:0/2054537721 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be8029060 con 0x561e474a7bd0 2024-04-24T09:06:51.002 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.001+0000 7f7bd7e00640 1 -- 172.21.15.83:0/2054537721 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9583+0+0 (secure 0 0 0) 0x7f7be801aac0 con 0x561e474a7bd0 2024-04-24T09:06:51.002 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.001+0000 7f7bfffaf740 1 -- 172.21.15.83:0/2054537721 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7be801a910 con 0x561e474a7bd0 2024-04-24T09:06:51.002 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.002+0000 7f7bd7e00640 1 -- 172.21.15.83:0/2054537721 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f7be8030050 con 0x561e474a7bd0 2024-04-24T09:06:51.002 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.002+0000 7f7bfffaf740 10 client.20934 fetch_fsmap learned FSMap version 49 2024-04-24T09:06:51.002 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.002+0000 7f7bfffaf740 1 -- 172.21.15.83:0/2054537721 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x561e47118210 con 0x561e474a7bd0 2024-04-24T09:06:51.003 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.002+0000 7f7bd7e00640 1 -- 172.21.15.83:0/2054537721 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7be801a910 con 0x561e474a7bd0 2024-04-24T09:06:51.003 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.003+0000 7f7bd7e00640 1 -- 172.21.15.83:0/2054537721 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 49) ==== 8406+0+0 (secure 0 0 0) 0x7f7be8023030 con 0x561e474a7bd0 2024-04-24T09:06:51.003 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.003+0000 7f7bfffaf740 10 client.20934 fetch_fsmap finished waiting for FSMap version 49 2024-04-24T09:06:51.003 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.003+0000 7f7bfffaf740 10 client.20934 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:06:51.003 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.003+0000 7f7bfffaf740 20 client.20934 populate_metadata read hostname 'smithi083' 2024-04-24T09:06:51.003 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.003+0000 7f7bfffaf740 1 --2- 172.21.15.83:0/2054537721 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x561e474c2e20 0x561e474c5210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:51.003 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.003+0000 7f7bfffaf740 4 client.20934 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:06:51.003 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.003+0000 7f7bfffaf740 1 -- 172.21.15.83:0/2054537721 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561e47413320 con 0x561e474c2e20 2024-04-24T09:06:51.003 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.003+0000 7f7bf5400640 1 --2- 172.21.15.83:0/2054537721 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x561e474c2e20 0x561e474c5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:51.004 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.004+0000 7f7bf5400640 1 --2- 172.21.15.83:0/2054537721 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x561e474c2e20 0x561e474c5210 crc :-1 s=READY pgs=387 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f94a40ff0ec6124 server_cookie=512b014c7819df78 in_seq=0 out_seq=0 2024-04-24T09:06:51.004 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.004+0000 7f7bd7e00640 10 client.20934 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:06:51.005 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.005+0000 7f7bd7e00640 1 -- 172.21.15.83:0/2054537721 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561e47413320 con 0x561e474c2e20 2024-04-24T09:06:51.005 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.005+0000 7f7bd7e00640 10 client.20934 handle_command_reply: tid=0 2024-04-24T09:06:51.005 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.005+0000 7f7bfffaf740 1 client.20934 shutdown 2024-04-24T09:06:51.006 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.006+0000 7f7bfffaf740 1 -- 172.21.15.83:0/2054537721 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561e474a7bd0 msgr2=0x561e474afd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:51.006 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.006+0000 7f7bfffaf740 1 --2- 172.21.15.83:0/2054537721 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561e474a7bd0 0x561e474afd10 secure :-1 s=READY pgs=2203 cs=0 l=1 rev1=1 crypto rx=0x7f7be8011480 tx=0x7f7be8005740 comp rx=0 tx=0).stop 2024-04-24T09:06:51.006 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.006+0000 7f7bfffaf740 1 -- 172.21.15.83:0/2054537721 shutdown_connections 2024-04-24T09:06:51.006 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.006+0000 7f7bfffaf740 1 --2- 172.21.15.83:0/2054537721 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x561e474c2e20 0x561e474c5210 crc :-1 s=READY pgs=387 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:51.007 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.006+0000 7f7bfffaf740 1 --2- 172.21.15.83:0/2054537721 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e474b2db0 0x561e474b0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:51.007 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.006+0000 7f7bfffaf740 1 --2- 172.21.15.83:0/2054537721 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561e474a84f0 0x561e474b0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:51.007 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.006+0000 7f7bfffaf740 1 --2- 172.21.15.83:0/2054537721 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561e474a7bd0 0x561e474afd10 unknown :-1 s=CLOSED pgs=2203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:51.007 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.006+0000 7f7bfffaf740 1 -- 172.21.15.83:0/2054537721 >> 172.21.15.83:0/2054537721 conn(0x561e474a40e0 msgr2=0x561e4749aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:51.007 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.006+0000 7f7bd7e00640 0 client.20934 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:06:51.007 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.006+0000 7f7bfffaf740 1 -- 172.21.15.83:0/2054537721 shutdown_connections 2024-04-24T09:06:51.007 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.006+0000 7f7bfffaf740 1 -- 172.21.15.83:0/2054537721 wait complete. 2024-04-24T09:06:51.007 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.007+0000 7f7bfffaf740 20 client.20934 trim_cache size 0 max 16384 2024-04-24T09:06:51.035 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.034+0000 7f7bfffaf740 1 Processor -- start 2024-04-24T09:06:51.035 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.034+0000 7f7bfffaf740 1 -- start start 2024-04-24T09:06:51.035 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.034+0000 7f7bfffaf740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561e474a7bd0 0x561e474be8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:51.035 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.034+0000 7f7bfffaf740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e474a84f0 0x561e4749a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:51.035 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.034+0000 7f7bfffaf740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561e474b2db0 0x561e474ca810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:51.035 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.034+0000 7f7bfffaf740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf81ad370 con 0x561e474a7bd0 2024-04-24T09:06:51.035 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.034+0000 7f7bfffaf740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf81af030 con 0x561e474a84f0 2024-04-24T09:06:51.035 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.034+0000 7f7bfffaf740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf81ada40 con 0x561e474b2db0 2024-04-24T09:06:51.035 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.035+0000 7f7bf5e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561e474a7bd0 0x561e474be8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:51.035 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.035+0000 7f7bf5e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561e474a7bd0 0x561e474be8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50974/0 (socket says 172.21.15.83:50974) 2024-04-24T09:06:51.035 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.035+0000 7f7bf5e00640 1 -- 172.21.15.83:0/2896562523 learned_addr learned my addr 172.21.15.83:0/2896562523 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:06:51.036 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.035+0000 7f7bf5400640 1 --2- 172.21.15.83:0/2896562523 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e474a84f0 0x561e4749a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:51.036 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.035+0000 7f7bf6800640 1 --2- 172.21.15.83:0/2896562523 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561e474b2db0 0x561e474ca810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:51.036 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.035+0000 7f7bf5e00640 1 -- 172.21.15.83:0/2896562523 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561e474b2db0 msgr2=0x561e474ca810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:51.036 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.035+0000 7f7bf5e00640 1 --2- 172.21.15.83:0/2896562523 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561e474b2db0 0x561e474ca810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:51.037 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.035+0000 7f7bf5e00640 1 -- 172.21.15.83:0/2896562523 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e474a84f0 msgr2=0x561e4749a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:51.037 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.035+0000 7f7bf5e00640 1 --2- 172.21.15.83:0/2896562523 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e474a84f0 0x561e4749a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:51.037 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.035+0000 7f7bf5e00640 1 -- 172.21.15.83:0/2896562523 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e471e4620 con 0x561e474a7bd0 2024-04-24T09:06:51.037 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.035+0000 7f7bf6800640 1 --2- 172.21.15.83:0/2896562523 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561e474b2db0 0x561e474ca810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:06:51.037 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.035+0000 7f7bf5e00640 1 --2- 172.21.15.83:0/2896562523 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561e474a7bd0 0x561e474be8a0 secure :-1 s=READY pgs=2204 cs=0 l=1 rev1=1 crypto rx=0x7f7be800f770 tx=0x7f7be80059b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fe6d51817c97e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:51.037 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.036+0000 7f7be2a00640 1 -- 172.21.15.83:0/2896562523 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be8029070 con 0x561e474a7bd0 2024-04-24T09:06:51.037 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.036+0000 7f7be2a00640 1 -- 172.21.15.83:0/2896562523 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7be80118b0 con 0x561e474a7bd0 2024-04-24T09:06:51.037 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.036+0000 7f7be2a00640 1 -- 172.21.15.83:0/2896562523 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be8015050 con 0x561e474a7bd0 2024-04-24T09:06:51.038 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.037+0000 7f7bfffaf740 1 -- 172.21.15.83:0/2896562523 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561e474a7bd0 msgr2=0x561e474be8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:51.038 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.037+0000 7f7bfffaf740 1 --2- 172.21.15.83:0/2896562523 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561e474a7bd0 0x561e474be8a0 secure :-1 s=READY pgs=2204 cs=0 l=1 rev1=1 crypto rx=0x7f7be800f770 tx=0x7f7be80059b0 comp rx=0 tx=0).stop 2024-04-24T09:06:51.038 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.037+0000 7f7bfffaf740 1 -- 172.21.15.83:0/2896562523 shutdown_connections 2024-04-24T09:06:51.038 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.037+0000 7f7bfffaf740 1 --2- 172.21.15.83:0/2896562523 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561e474b2db0 0x561e474ca810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:51.039 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.037+0000 7f7bfffaf740 1 --2- 172.21.15.83:0/2896562523 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e474a84f0 0x561e4749a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:51.039 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.037+0000 7f7bfffaf740 1 --2- 172.21.15.83:0/2896562523 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561e474a7bd0 0x561e474be8a0 unknown :-1 s=CLOSED pgs=2204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:51.039 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.037+0000 7f7bfffaf740 1 -- 172.21.15.83:0/2896562523 >> 172.21.15.83:0/2896562523 conn(0x561e474a40e0 msgr2=0x561e4749a200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:51.039 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.037+0000 7f7bfffaf740 1 -- 172.21.15.83:0/2896562523 shutdown_connections 2024-04-24T09:06:51.039 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.037+0000 7f7bfffaf740 1 -- 172.21.15.83:0/2896562523 wait complete. 2024-04-24T09:06:51.039 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.038+0000 7f7bfffaf740 1 -- start start 2024-04-24T09:06:51.039 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.038+0000 7f7bfffaf740 1 Processor -- start 2024-04-24T09:06:51.040 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.039+0000 7f7bfffaf740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561e474a7bd0 0x561e474bae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:51.040 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.039+0000 7f7bfffaf740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561e474a84f0 0x561e474bb340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:51.040 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.039+0000 7f7bfffaf740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e474b2db0 0x561e474b64b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:51.040 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.039+0000 7f7bfffaf740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf81ae8a0 con 0x561e474a84f0 2024-04-24T09:06:51.040 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.039+0000 7f7bfffaf740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf81ae170 con 0x561e474b2db0 2024-04-24T09:06:51.040 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.039+0000 7f7bfffaf740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf81ab370 con 0x561e474a7bd0 2024-04-24T09:06:51.040 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.039+0000 7f7bfffaf740 10 client.0 _ll_register_callbacks cb 0x7f7be40096e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:06:51.040 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.039+0000 7f7bf6800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e474b2db0 0x561e474b64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:51.041 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.039+0000 7f7bf6800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e474b2db0 0x561e474b64b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:56704/0 (socket says 172.21.15.83:56704) 2024-04-24T09:06:51.041 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.039+0000 7f7bf6800640 1 -- 172.21.15.83:0/295955036 learned_addr learned my addr 172.21.15.83:0/295955036 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:06:51.041 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.039+0000 7f7bf6800640 1 -- 172.21.15.83:0/295955036 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561e474a7bd0 msgr2=0x561e474bae00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-24T09:06:51.041 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.039+0000 7f7bf6800640 1 --2- 172.21.15.83:0/295955036 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561e474a7bd0 0x561e474bae00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:51.041 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.039+0000 7f7bf6800640 1 -- 172.21.15.83:0/295955036 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561e474a84f0 msgr2=0x561e474bb340 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T09:06:51.041 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.040+0000 7f7bf5400640 1 --2- 172.21.15.83:0/295955036 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561e474a84f0 0x561e474bb340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:51.041 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.040+0000 7f7bf6800640 1 --2- 172.21.15.83:0/295955036 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561e474a84f0 0x561e474bb340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:51.041 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.040+0000 7f7bf6800640 1 -- 172.21.15.83:0/295955036 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e474b6cb0 con 0x561e474b2db0 2024-04-24T09:06:51.041 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.040+0000 7f7bf5400640 1 --2- 172.21.15.83:0/295955036 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561e474a84f0 0x561e474bb340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:06:51.043 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.041+0000 7f7bf6800640 1 --2- 172.21.15.83:0/295955036 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e474b2db0 0x561e474b64b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7bf00bcf40 tx=0x7f7bf00bc910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53dfc4ea407447eb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:51.043 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.041+0000 7f7bd7e00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:06:51.043 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.041+0000 7f7bd7e00640 1 -- 172.21.15.83:0/295955036 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bf00db060 con 0x561e474b2db0 2024-04-24T09:06:51.043 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.041+0000 7f7bfffaf740 1 -- 172.21.15.83:0/295955036 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7be80118b0 con 0x561e474b2db0 2024-04-24T09:06:51.043 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.041+0000 7f7bd7e00640 1 -- 172.21.15.83:0/295955036 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9583+0+0 (secure 0 0 0) 0x7f7bf00b9620 con 0x561e474b2db0 2024-04-24T09:06:51.043 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.041+0000 7f7bd7e00640 1 -- 172.21.15.83:0/295955036 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f7bf00ca050 con 0x561e474b2db0 2024-04-24T09:06:51.043 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.041+0000 7f7bfffaf740 10 client.38084 fetch_fsmap learned FSMap version 49 2024-04-24T09:06:51.043 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.041+0000 7f7bfffaf740 1 -- 172.21.15.83:0/295955036 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x561e47528cf0 con 0x561e474b2db0 2024-04-24T09:06:51.043 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.042+0000 7f7bd7e00640 1 -- 172.21.15.83:0/295955036 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7be80118b0 con 0x561e474b2db0 2024-04-24T09:06:51.043 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.042+0000 7f7bd7e00640 1 -- 172.21.15.83:0/295955036 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 49) ==== 8406+0+0 (secure 0 0 0) 0x7f7bf00bd210 con 0x561e474b2db0 2024-04-24T09:06:51.043 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.042+0000 7f7bfffaf740 10 client.38084 fetch_fsmap finished waiting for FSMap version 49 2024-04-24T09:06:51.043 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.042+0000 7f7bfffaf740 10 client.38084 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:06:51.043 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.042+0000 7f7bfffaf740 20 client.38084 populate_metadata read hostname 'smithi083' 2024-04-24T09:06:51.044 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.042+0000 7f7bfffaf740 1 --2- 172.21.15.83:0/295955036 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x561e475294d0 0x561e4752b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:51.044 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.042+0000 7f7bfffaf740 4 client.38084 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T09:06:51.044 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.042+0000 7f7bfffaf740 1 -- 172.21.15.83:0/295955036 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561e4740d190 con 0x561e475294d0 2024-04-24T09:06:51.044 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.043+0000 7f7bf5e00640 1 --2- 172.21.15.83:0/295955036 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x561e475294d0 0x561e4752b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:51.044 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.043+0000 7f7bd7e00640 10 client.38084 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:06:51.044 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.043+0000 7f7bf5e00640 1 --2- 172.21.15.83:0/295955036 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x561e475294d0 0x561e4752b910 crc :-1 s=READY pgs=388 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3c60d042fec8aee server_cookie=cec71396a4951198 in_seq=0 out_seq=0 2024-04-24T09:06:51.155 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.154+0000 7f7bd7e00640 1 -- 172.21.15.83:0/295955036 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561e4740d190 con 0x561e475294d0 2024-04-24T09:06:51.155 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.154+0000 7f7bd7e00640 10 client.38084 handle_command_reply: tid=0 2024-04-24T09:06:51.155 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.155+0000 7f7bfffaf740 1 client.38084 shutdown 2024-04-24T09:06:51.155 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.155+0000 7f7bfffaf740 1 -- 172.21.15.83:0/295955036 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e474b2db0 msgr2=0x561e474b64b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:51.156 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.155+0000 7f7bfffaf740 1 --2- 172.21.15.83:0/295955036 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e474b2db0 0x561e474b64b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7bf00bcf40 tx=0x7f7bf00bc910 comp rx=0 tx=0).stop 2024-04-24T09:06:51.156 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.155+0000 7f7bfffaf740 1 -- 172.21.15.83:0/295955036 shutdown_connections 2024-04-24T09:06:51.156 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.155+0000 7f7bfffaf740 1 --2- 172.21.15.83:0/295955036 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x561e475294d0 0x561e4752b910 crc :-1 s=READY pgs=388 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:51.156 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.155+0000 7f7bfffaf740 1 --2- 172.21.15.83:0/295955036 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e474b2db0 0x561e474b64b0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:51.156 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.155+0000 7f7bfffaf740 1 --2- 172.21.15.83:0/295955036 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561e474a84f0 0x561e474bb340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:51.156 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.155+0000 7f7bd7e00640 0 client.38084 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:06:51.156 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.155+0000 7f7bfffaf740 1 --2- 172.21.15.83:0/295955036 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561e474a7bd0 0x561e474bae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:51.156 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.155+0000 7f7bfffaf740 1 -- 172.21.15.83:0/295955036 >> 172.21.15.83:0/295955036 conn(0x561e474a40e0 msgr2=0x561e4749aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:51.156 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.155+0000 7f7bfffaf740 1 -- 172.21.15.83:0/295955036 shutdown_connections 2024-04-24T09:06:51.156 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.155+0000 7f7bfffaf740 1 -- 172.21.15.83:0/295955036 wait complete. 2024-04-24T09:06:51.157 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.156+0000 7f7bfffaf740 20 client.38084 trim_cache size 0 max 16384 2024-04-24T09:06:51.157 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T09:06:51.157 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T09:06:51.157 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T09:06:51.157 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T09:06:51.158 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.157+0000 7f7bf7200640 1 -- 172.21.15.83:0/3397161054 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f7bdc0a3d90 msgr2=0x7f7bdc0a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:51.158 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.157+0000 7f7bf7200640 1 --2- 172.21.15.83:0/3397161054 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f7bdc0a3d90 0x7f7bdc0a6250 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x561e474a5500 tx=0x7f7be4014040 comp rx=0 tx=0).stop 2024-04-24T09:06:51.158 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.157+0000 7f7bf7200640 1 -- 172.21.15.83:0/3397161054 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7bf819a9b0 msgr2=0x7f7bf819ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:51.158 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.157+0000 7f7bf7200640 1 --2- 172.21.15.83:0/3397161054 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7bf819a9b0 0x7f7bf819ae60 secure :-1 s=READY pgs=2202 cs=0 l=1 rev1=1 crypto rx=0x7f7bf0004af0 tx=0x7f7bf0003d60 comp rx=0 tx=0).stop 2024-04-24T09:06:51.158 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.157+0000 7f7bf7200640 1 -- 172.21.15.83:0/3397161054 shutdown_connections 2024-04-24T09:06:51.158 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.157+0000 7f7bf7200640 1 --2- 172.21.15.83:0/3397161054 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f7bdc0a3d90 0x7f7bdc0a6250 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:51.158 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.157+0000 7f7bf7200640 1 --2- 172.21.15.83:0/3397161054 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7bf819a9b0 0x7f7bf819ae60 unknown :-1 s=CLOSED pgs=2202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:51.158 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.157+0000 7f7bf7200640 1 --2- 172.21.15.83:0/3397161054 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7bf8109380 0x7f7bf81a0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:51.158 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.157+0000 7f7bf7200640 1 --2- 172.21.15.83:0/3397161054 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7bf81042b0 0x7f7bf81a0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:51.159 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.157+0000 7f7bf7200640 1 -- 172.21.15.83:0/3397161054 >> 172.21.15.83:0/3397161054 conn(0x7f7bf80ffaf0 msgr2=0x7f7bf8101e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:51.159 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.157+0000 7f7bf7200640 1 -- 172.21.15.83:0/3397161054 shutdown_connections 2024-04-24T09:06:51.159 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:06:51.157+0000 7f7bf7200640 1 -- 172.21.15.83:0/3397161054 wait complete. 2024-04-24T09:06:51.159 INFO:tasks.workunit.client.0.smithi083.stdout: 10 10669 9.12 MB/sec warmup 82 sec latency 127.166 ms 2024-04-24T09:06:51.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:06:51 smithi138 ceph-mon[31441]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:06:51.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:06:51 smithi189 ceph-mon[31791]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:06:51.552 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:06:51 smithi083 ceph-mon[27907]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:06:52.159 INFO:tasks.workunit.client.0.smithi083.stdout: 10 10810 9.10 MB/sec warmup 83 sec latency 93.993 ms 2024-04-24T09:06:52.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:06:52 smithi138 ceph-mon[31441]: pgmap v478: 129 pgs: 129 active+clean; 364 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 850 KiB/s rd, 3.4 MiB/s wr, 210 op/s 2024-04-24T09:06:52.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:06:52 smithi189 ceph-mon[31791]: pgmap v478: 129 pgs: 129 active+clean; 364 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 850 KiB/s rd, 3.4 MiB/s wr, 210 op/s 2024-04-24T09:06:52.552 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:06:52 smithi083 ceph-mon[27907]: pgmap v478: 129 pgs: 129 active+clean; 364 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 850 KiB/s rd, 3.4 MiB/s wr, 210 op/s 2024-04-24T09:06:53.159 INFO:tasks.workunit.client.0.smithi083.stdout: 10 10962 9.09 MB/sec warmup 84 sec latency 73.417 ms 2024-04-24T09:06:54.159 INFO:tasks.workunit.client.0.smithi083.stdout: 10 11116 9.05 MB/sec warmup 85 sec latency 80.488 ms 2024-04-24T09:06:54.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:06:54 smithi138 ceph-mon[31441]: pgmap v479: 129 pgs: 129 active+clean; 372 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.3 MiB/s wr, 333 op/s 2024-04-24T09:06:54.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:06:54 smithi189 ceph-mon[31791]: pgmap v479: 129 pgs: 129 active+clean; 372 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.3 MiB/s wr, 333 op/s 2024-04-24T09:06:54.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:06:54 smithi083 ceph-mon[27907]: pgmap v479: 129 pgs: 129 active+clean; 372 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.3 MiB/s wr, 333 op/s 2024-04-24T09:06:55.159 INFO:tasks.workunit.client.0.smithi083.stdout: 10 11233 9.03 MB/sec warmup 86 sec latency 110.364 ms 2024-04-24T09:06:55.998 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T09:06:55.998 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T09:06:56.159 INFO:tasks.workunit.client.0.smithi083.stdout: 10 11345 8.97 MB/sec warmup 87 sec latency 75.128 ms 2024-04-24T09:06:56.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.257+0000 7f0f03800640 1 -- 172.21.15.83:0/2108392114 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0efc0743a0 msgr2=0x7f0efc074780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:56.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.257+0000 7f0f03800640 1 --2- 172.21.15.83:0/2108392114 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0efc0743a0 0x7f0efc074780 secure :-1 s=READY pgs=2205 cs=0 l=1 rev1=1 crypto rx=0x7f0ef0009900 tx=0x7f0ef002a480 comp rx=0 tx=0).stop 2024-04-24T09:06:56.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.257+0000 7f0f03800640 1 -- 172.21.15.83:0/2108392114 shutdown_connections 2024-04-24T09:06:56.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.257+0000 7f0f03800640 1 --2- 172.21.15.83:0/2108392114 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0efc113030 0x7f0efc115420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:56.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.257+0000 7f0f03800640 1 --2- 172.21.15.83:0/2108392114 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0efc074cc0 0x7f0efc112af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:56.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.257+0000 7f0f03800640 1 --2- 172.21.15.83:0/2108392114 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0efc0743a0 0x7f0efc074780 unknown :-1 s=CLOSED pgs=2205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:56.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.257+0000 7f0f03800640 1 -- 172.21.15.83:0/2108392114 >> 172.21.15.83:0/2108392114 conn(0x7f0efc0fdb10 msgr2=0x7f0efc0fff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:56.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.258+0000 7f0f03800640 1 -- 172.21.15.83:0/2108392114 shutdown_connections 2024-04-24T09:06:56.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.258+0000 7f0f03800640 1 -- 172.21.15.83:0/2108392114 wait complete. 2024-04-24T09:06:56.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.259+0000 7f0f03800640 1 Processor -- start 2024-04-24T09:06:56.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.259+0000 7f0f03800640 1 -- start start 2024-04-24T09:06:56.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.260+0000 7f0f03800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0efc0743a0 0x7f0efc073a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:56.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.260+0000 7f0f03800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0efc074cc0 0x7f0efc06daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:56.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.260+0000 7f0f03800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0efc113030 0x7f0efc06dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:56.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.260+0000 7f0f03800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0efc06c2e0 con 0x7f0efc074cc0 2024-04-24T09:06:56.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.260+0000 7f0f03800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0efc06c160 con 0x7f0efc113030 2024-04-24T09:06:56.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.260+0000 7f0f03800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0efc06c460 con 0x7f0efc0743a0 2024-04-24T09:06:56.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.260+0000 7f0efbe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0efc074cc0 0x7f0efc06daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:56.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.260+0000 7f0efbe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0efc074cc0 0x7f0efc06daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51004/0 (socket says 172.21.15.83:51004) 2024-04-24T09:06:56.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.260+0000 7f0efbe00640 1 -- 172.21.15.83:0/742216016 learned_addr learned my addr 172.21.15.83:0/742216016 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:06:56.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.260+0000 7f0f00e00640 1 --2- 172.21.15.83:0/742216016 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0efc0743a0 0x7f0efc073a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:56.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.260+0000 7f0f01800640 1 --2- 172.21.15.83:0/742216016 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0efc113030 0x7f0efc06dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:56.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.261+0000 7f0f01800640 1 -- 172.21.15.83:0/742216016 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0efc0743a0 msgr2=0x7f0efc073a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:56.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.261+0000 7f0f01800640 1 --2- 172.21.15.83:0/742216016 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0efc0743a0 0x7f0efc073a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:56.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.261+0000 7f0f01800640 1 -- 172.21.15.83:0/742216016 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0efc074cc0 msgr2=0x7f0efc06daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:56.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.261+0000 7f0f01800640 1 --2- 172.21.15.83:0/742216016 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0efc074cc0 0x7f0efc06daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:56.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.261+0000 7f0f01800640 1 -- 172.21.15.83:0/742216016 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0efc06e6e0 con 0x7f0efc113030 2024-04-24T09:06:56.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.261+0000 7f0efbe00640 1 --2- 172.21.15.83:0/742216016 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0efc074cc0 0x7f0efc06daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:06:56.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.262+0000 7f0f01800640 1 --2- 172.21.15.83:0/742216016 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0efc113030 0x7f0efc06dfe0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0ef4005fd0 tx=0x7f0ef400c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:56.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.262+0000 7f0ef9600640 1 -- 172.21.15.83:0/742216016 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef4003b70 con 0x7f0efc113030 2024-04-24T09:06:56.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.262+0000 7f0f03800640 1 -- 172.21.15.83:0/742216016 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0efc1b3dc0 con 0x7f0efc113030 2024-04-24T09:06:56.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.262+0000 7f0ef9600640 1 -- 172.21.15.83:0/742216016 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0ef4003d10 con 0x7f0efc113030 2024-04-24T09:06:56.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.262+0000 7f0ef9600640 1 -- 172.21.15.83:0/742216016 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef4010780 con 0x7f0efc113030 2024-04-24T09:06:56.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.262+0000 7f0f03800640 1 -- 172.21.15.83:0/742216016 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0efc1b41d0 con 0x7f0efc113030 2024-04-24T09:06:56.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.266+0000 7f0f06765740 1 Processor -- start 2024-04-24T09:06:56.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.267+0000 7f0ef9600640 1 -- 172.21.15.83:0/742216016 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0ef4003fc0 con 0x7f0efc113030 2024-04-24T09:06:56.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.268+0000 7f0f06765740 1 -- start start 2024-04-24T09:06:56.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.268+0000 7f0f06765740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561fc6374860 0x561fc6374c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:56.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.268+0000 7f0f06765740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561fc6375180 0x561fc637f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:56.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.268+0000 7f0f06765740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561fc637f850 0x561fc6381c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:56.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.268+0000 7f0f06765740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561fc62da1f0 con 0x561fc6374860 2024-04-24T09:06:56.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.268+0000 7f0f06765740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x561fc6280350 con 0x561fc637f850 2024-04-24T09:06:56.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.268+0000 7f0f06765740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x561fc62d61b0 con 0x561fc6375180 2024-04-24T09:06:56.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.269+0000 7f0efbe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561fc6374860 0x561fc6374c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:56.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.269+0000 7f0f00e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561fc6375180 0x561fc637f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:56.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.269+0000 7f0efbe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561fc6374860 0x561fc6374c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51012/0 (socket says 172.21.15.83:51012) 2024-04-24T09:06:56.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.269+0000 7f0efbe00640 1 -- 172.21.15.83:0/3385660612 learned_addr learned my addr 172.21.15.83:0/3385660612 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:06:56.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.269+0000 7f0ef9600640 1 --2- 172.21.15.83:0/742216016 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f0ee40a3d20 0x7f0ee40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:56.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.269+0000 7f0efbe00640 1 --2- 172.21.15.83:0/3385660612 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561fc637f850 0x561fc6381c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:56.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.269+0000 7f0ef9600640 1 -- 172.21.15.83:0/742216016 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9583+0+0 (secure 0 0 0) 0x7f0ef40d1040 con 0x7f0efc113030 2024-04-24T09:06:56.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.270+0000 7f0efbe00640 1 -- 172.21.15.83:0/3385660612 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561fc6375180 msgr2=0x561fc637f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:56.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.270+0000 7f0efbe00640 1 --2- 172.21.15.83:0/3385660612 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561fc6375180 0x561fc637f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:56.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.270+0000 7f0efbe00640 1 -- 172.21.15.83:0/3385660612 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561fc637f850 msgr2=0x561fc6381c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:56.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.270+0000 7f0f01800640 1 --2- 172.21.15.83:0/742216016 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f0ee40a3d20 0x7f0ee40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:56.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.270+0000 7f0efbe00640 1 --2- 172.21.15.83:0/3385660612 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561fc637f850 0x561fc6381c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:56.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.270+0000 7f0efbe00640 1 -- 172.21.15.83:0/3385660612 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561fc62d4f50 con 0x561fc6374860 2024-04-24T09:06:56.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.270+0000 7f0f00e00640 1 --2- 172.21.15.83:0/3385660612 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561fc6375180 0x561fc637f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:06:56.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.270+0000 7f0f01800640 1 --2- 172.21.15.83:0/742216016 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f0ee40a3d20 0x7f0ee40a61e0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f0ef4099590 tx=0x7f0ef4004a40 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:56.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.271+0000 7f0efbe00640 1 --2- 172.21.15.83:0/3385660612 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561fc6374860 0x561fc6374c40 secure :-1 s=READY pgs=2206 cs=0 l=1 rev1=1 crypto rx=0x7f0ee8012c20 tx=0x7f0ee800be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebab3b2d3ecae5e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:56.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.271+0000 7f0eee000640 1 -- 172.21.15.83:0/3385660612 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee8010c00 con 0x561fc6374860 2024-04-24T09:06:56.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.271+0000 7f0eee000640 1 -- 172.21.15.83:0/3385660612 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0ee8010da0 con 0x561fc6374860 2024-04-24T09:06:56.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.271+0000 7f0eee000640 1 -- 172.21.15.83:0/3385660612 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee80047a0 con 0x561fc6374860 2024-04-24T09:06:56.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.272+0000 7f0f06765740 1 -- 172.21.15.83:0/3385660612 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561fc6374860 msgr2=0x561fc6374c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:56.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.272+0000 7f0f06765740 1 --2- 172.21.15.83:0/3385660612 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561fc6374860 0x561fc6374c40 secure :-1 s=READY pgs=2206 cs=0 l=1 rev1=1 crypto rx=0x7f0ee8012c20 tx=0x7f0ee800be60 comp rx=0 tx=0).stop 2024-04-24T09:06:56.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.272+0000 7f0f06765740 1 -- 172.21.15.83:0/3385660612 shutdown_connections 2024-04-24T09:06:56.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.272+0000 7f0f06765740 1 --2- 172.21.15.83:0/3385660612 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561fc637f850 0x561fc6381c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:56.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.273+0000 7f0f06765740 1 --2- 172.21.15.83:0/3385660612 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561fc6375180 0x561fc637f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:56.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.273+0000 7f0f06765740 1 --2- 172.21.15.83:0/3385660612 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561fc6374860 0x561fc6374c40 unknown :-1 s=CLOSED pgs=2206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:56.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.273+0000 7f0f06765740 1 -- 172.21.15.83:0/3385660612 >> 172.21.15.83:0/3385660612 conn(0x561fc6370b80 msgr2=0x561fc6370f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:56.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.273+0000 7f0f06765740 1 -- 172.21.15.83:0/3385660612 shutdown_connections 2024-04-24T09:06:56.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.273+0000 7f0f06765740 1 -- 172.21.15.83:0/3385660612 wait complete. 2024-04-24T09:06:56.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.273+0000 7f0f06765740 1 -- start start 2024-04-24T09:06:56.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.273+0000 7f0f06765740 1 Processor -- start 2024-04-24T09:06:56.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.274+0000 7f0f06765740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561fc6374860 0x561fc637c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:56.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.274+0000 7f0f06765740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561fc6375180 0x561fc637cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:56.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.274+0000 7f0f06765740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561fc637f850 0x561fc637d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:56.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.274+0000 7f0f06765740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561fc63825e0 con 0x561fc637f850 2024-04-24T09:06:56.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.274+0000 7f0f06765740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x561fc6293690 con 0x561fc6375180 2024-04-24T09:06:56.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.274+0000 7f0f06765740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x561fc6382760 con 0x561fc6374860 2024-04-24T09:06:56.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.274+0000 7f0f06765740 10 client.0 _ll_register_callbacks cb 0x561fc62cf650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:06:56.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.275+0000 7f0f00e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561fc6375180 0x561fc637cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:56.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.275+0000 7f0f00e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561fc6375180 0x561fc637cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:56736/0 (socket says 172.21.15.83:56736) 2024-04-24T09:06:56.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.275+0000 7f0f00e00640 1 -- 172.21.15.83:0/3854153129 learned_addr learned my addr 172.21.15.83:0/3854153129 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:06:56.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.275+0000 7f0efbe00640 1 --2- 172.21.15.83:0/3854153129 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561fc637f850 0x561fc637d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:56.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.275+0000 7f0efbe00640 1 --2- 172.21.15.83:0/3854153129 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561fc6374860 0x561fc637c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:56.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.276+0000 7f0efbe00640 1 -- 172.21.15.83:0/3854153129 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561fc6374860 msgr2=0x561fc637c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:56.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.276+0000 7f0efbe00640 1 --2- 172.21.15.83:0/3854153129 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561fc6374860 0x561fc637c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:56.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.276+0000 7f0efbe00640 1 -- 172.21.15.83:0/3854153129 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561fc6375180 msgr2=0x561fc637cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:56.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.276+0000 7f0efbe00640 1 --2- 172.21.15.83:0/3854153129 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561fc6375180 0x561fc637cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:56.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.276+0000 7f0efbe00640 1 -- 172.21.15.83:0/3854153129 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561fc6292920 con 0x561fc637f850 2024-04-24T09:06:56.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.276+0000 7f0efbe00640 1 --2- 172.21.15.83:0/3854153129 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561fc637f850 0x561fc637d260 secure :-1 s=READY pgs=2207 cs=0 l=1 rev1=1 crypto rx=0x7f0ee80125b0 tx=0x7f0ee800b700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4af8543bbee41f32 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:56.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.276+0000 7f0ee2a00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:06:56.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.276+0000 7f0ee2a00640 1 -- 172.21.15.83:0/3854153129 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee8004020 con 0x561fc637f850 2024-04-24T09:06:56.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.276+0000 7f0ee2a00640 1 -- 172.21.15.83:0/3854153129 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9583+0+0 (secure 0 0 0) 0x7f0ee80041c0 con 0x561fc637f850 2024-04-24T09:06:56.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.276+0000 7f0f06765740 1 -- 172.21.15.83:0/3854153129 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ee8010da0 con 0x561fc637f850 2024-04-24T09:06:56.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.277+0000 7f0ee2a00640 1 -- 172.21.15.83:0/3854153129 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f0ee80043c0 con 0x561fc637f850 2024-04-24T09:06:56.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.277+0000 7f0f06765740 10 client.20949 fetch_fsmap learned FSMap version 49 2024-04-24T09:06:56.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.277+0000 7f0f06765740 1 -- 172.21.15.83:0/3854153129 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x561fc60b67d0 con 0x561fc637f850 2024-04-24T09:06:56.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.277+0000 7f0ee2a00640 1 -- 172.21.15.83:0/3854153129 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0ee8010da0 con 0x561fc637f850 2024-04-24T09:06:56.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.278+0000 7f0ee2a00640 1 -- 172.21.15.83:0/3854153129 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 49) ==== 8406+0+0 (secure 0 0 0) 0x7f0ee802b020 con 0x561fc637f850 2024-04-24T09:06:56.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.278+0000 7f0f06765740 10 client.20949 fetch_fsmap finished waiting for FSMap version 49 2024-04-24T09:06:56.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.278+0000 7f0f06765740 10 client.20949 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:06:56.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.278+0000 7f0f06765740 20 client.20949 populate_metadata read hostname 'smithi083' 2024-04-24T09:06:56.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.278+0000 7f0f06765740 1 --2- 172.21.15.83:0/3854153129 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x561fc638fa00 0x561fc6391df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:56.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.278+0000 7f0f06765740 4 client.20949 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:06:56.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.278+0000 7f0f06765740 1 -- 172.21.15.83:0/3854153129 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561fc62e7c10 con 0x561fc638fa00 2024-04-24T09:06:56.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.278+0000 7f0f00e00640 1 --2- 172.21.15.83:0/3854153129 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x561fc638fa00 0x561fc6391df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:56.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.279+0000 7f0f00e00640 1 --2- 172.21.15.83:0/3854153129 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x561fc638fa00 0x561fc6391df0 crc :-1 s=READY pgs=389 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21ae290fdb5b72d3 server_cookie=fc01ae87b4cfdff2 in_seq=0 out_seq=0 2024-04-24T09:06:56.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.279+0000 7f0ee2a00640 10 client.20949 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:06:56.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.280+0000 7f0ee2a00640 1 -- 172.21.15.83:0/3854153129 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561fc62e7c10 con 0x561fc638fa00 2024-04-24T09:06:56.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.280+0000 7f0ee2a00640 10 client.20949 handle_command_reply: tid=0 2024-04-24T09:06:56.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.280+0000 7f0f06765740 1 client.20949 shutdown 2024-04-24T09:06:56.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.281+0000 7f0f06765740 1 -- 172.21.15.83:0/3854153129 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561fc637f850 msgr2=0x561fc637d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:56.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.281+0000 7f0f06765740 1 --2- 172.21.15.83:0/3854153129 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561fc637f850 0x561fc637d260 secure :-1 s=READY pgs=2207 cs=0 l=1 rev1=1 crypto rx=0x7f0ee80125b0 tx=0x7f0ee800b700 comp rx=0 tx=0).stop 2024-04-24T09:06:56.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.281+0000 7f0f06765740 1 -- 172.21.15.83:0/3854153129 shutdown_connections 2024-04-24T09:06:56.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.281+0000 7f0f06765740 1 --2- 172.21.15.83:0/3854153129 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x561fc638fa00 0x561fc6391df0 crc :-1 s=READY pgs=389 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:56.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.281+0000 7f0f06765740 1 --2- 172.21.15.83:0/3854153129 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561fc637f850 0x561fc637d260 unknown :-1 s=CLOSED pgs=2207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:56.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.281+0000 7f0f06765740 1 --2- 172.21.15.83:0/3854153129 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561fc6375180 0x561fc637cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:56.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.281+0000 7f0ee2a00640 0 client.20949 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:06:56.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.281+0000 7f0f06765740 1 --2- 172.21.15.83:0/3854153129 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561fc6374860 0x561fc637c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:56.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.281+0000 7f0f06765740 1 -- 172.21.15.83:0/3854153129 >> 172.21.15.83:0/3854153129 conn(0x561fc6370b80 msgr2=0x561fc6376980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:56.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.281+0000 7f0f06765740 1 -- 172.21.15.83:0/3854153129 shutdown_connections 2024-04-24T09:06:56.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.281+0000 7f0f06765740 1 -- 172.21.15.83:0/3854153129 wait complete. 2024-04-24T09:06:56.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.281+0000 7f0f06765740 20 client.20949 trim_cache size 0 max 16384 2024-04-24T09:06:56.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.312+0000 7f0f06765740 1 Processor -- start 2024-04-24T09:06:56.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.312+0000 7f0f06765740 1 -- start start 2024-04-24T09:06:56.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.312+0000 7f0f06765740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561fc6374860 0x561fc637d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:56.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.312+0000 7f0f06765740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561fc6375180 0x561fc636faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:56.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.312+0000 7f0f06765740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561fc637f850 0x561fc63975e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:56.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.312+0000 7f0f06765740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0efc1be350 con 0x561fc6374860 2024-04-24T09:06:56.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.312+0000 7f0f06765740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0efc1c0010 con 0x561fc6375180 2024-04-24T09:06:56.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.312+0000 7f0f06765740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0efc1bea20 con 0x561fc637f850 2024-04-24T09:06:56.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.314+0000 7f0efbe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561fc6374860 0x561fc637d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:56.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.314+0000 7f0f00e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561fc6375180 0x561fc636faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:56.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.314+0000 7f0efbe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561fc6374860 0x561fc637d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51034/0 (socket says 172.21.15.83:51034) 2024-04-24T09:06:56.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.314+0000 7f0efbe00640 1 -- 172.21.15.83:0/1079102272 learned_addr learned my addr 172.21.15.83:0/1079102272 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:06:56.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.315+0000 7f0efbe00640 1 -- 172.21.15.83:0/1079102272 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561fc637f850 msgr2=0x561fc63975e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:56.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.315+0000 7f0efbe00640 1 --2- 172.21.15.83:0/1079102272 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561fc637f850 0x561fc63975e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:56.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.315+0000 7f0efbe00640 1 -- 172.21.15.83:0/1079102272 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561fc6375180 msgr2=0x561fc636faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:56.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.315+0000 7f0efbe00640 1 --2- 172.21.15.83:0/1079102272 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561fc6375180 0x561fc636faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:56.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.315+0000 7f0efbe00640 1 -- 172.21.15.83:0/1079102272 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561fc5fec3f0 con 0x561fc6374860 2024-04-24T09:06:56.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.316+0000 7f0efbe00640 1 --2- 172.21.15.83:0/1079102272 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561fc6374860 0x561fc637d080 secure :-1 s=READY pgs=2208 cs=0 l=1 rev1=1 crypto rx=0x7f0ee8012780 tx=0x7f0ee8005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94423d3ac07d47ad server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:56.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.316+0000 7f0eed600640 1 -- 172.21.15.83:0/1079102272 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee8010d50 con 0x561fc6374860 2024-04-24T09:06:56.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.316+0000 7f0eed600640 1 -- 172.21.15.83:0/1079102272 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0ee80041b0 con 0x561fc6374860 2024-04-24T09:06:56.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.316+0000 7f0eed600640 1 -- 172.21.15.83:0/1079102272 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee8003aa0 con 0x561fc6374860 2024-04-24T09:06:56.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.317+0000 7f0f06765740 1 -- 172.21.15.83:0/1079102272 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561fc6374860 msgr2=0x561fc637d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:56.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.317+0000 7f0f06765740 1 --2- 172.21.15.83:0/1079102272 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561fc6374860 0x561fc637d080 secure :-1 s=READY pgs=2208 cs=0 l=1 rev1=1 crypto rx=0x7f0ee8012780 tx=0x7f0ee8005bd0 comp rx=0 tx=0).stop 2024-04-24T09:06:56.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.318+0000 7f0f06765740 1 -- 172.21.15.83:0/1079102272 shutdown_connections 2024-04-24T09:06:56.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.318+0000 7f0f06765740 1 --2- 172.21.15.83:0/1079102272 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561fc637f850 0x561fc63975e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:56.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.318+0000 7f0f06765740 1 --2- 172.21.15.83:0/1079102272 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561fc6375180 0x561fc636faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:56.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.318+0000 7f0f06765740 1 --2- 172.21.15.83:0/1079102272 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561fc6374860 0x561fc637d080 unknown :-1 s=CLOSED pgs=2208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:56.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.318+0000 7f0f06765740 1 -- 172.21.15.83:0/1079102272 >> 172.21.15.83:0/1079102272 conn(0x561fc6370b80 msgr2=0x561fc636f390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:56.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.318+0000 7f0f06765740 1 -- 172.21.15.83:0/1079102272 shutdown_connections 2024-04-24T09:06:56.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.318+0000 7f0f06765740 1 -- 172.21.15.83:0/1079102272 wait complete. 2024-04-24T09:06:56.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.319+0000 7f0f06765740 1 -- start start 2024-04-24T09:06:56.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.320+0000 7f0f06765740 1 Processor -- start 2024-04-24T09:06:56.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.320+0000 7f0f06765740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561fc6374860 0x561fc6383560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:56.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.320+0000 7f0f06765740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561fc6375180 0x561fc6383aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:56.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.320+0000 7f0f06765740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561fc637f850 0x561fc6383fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:56.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.320+0000 7f0f06765740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0efc1bf880 con 0x561fc6374860 2024-04-24T09:06:56.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.320+0000 7f0f06765740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0efc1bf150 con 0x561fc637f850 2024-04-24T09:06:56.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.320+0000 7f0f06765740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0efc1bc350 con 0x561fc6375180 2024-04-24T09:06:56.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.320+0000 7f0f06765740 10 client.0 _ll_register_callbacks cb 0x7f0ee8012610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:06:56.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.321+0000 7f0f00e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561fc6375180 0x561fc6383aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:56.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.321+0000 7f0f00e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561fc6375180 0x561fc6383aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:49488/0 (socket says 172.21.15.83:49488) 2024-04-24T09:06:56.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.321+0000 7f0f00e00640 1 -- 172.21.15.83:0/1394009843 learned_addr learned my addr 172.21.15.83:0/1394009843 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:06:56.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.321+0000 7f0efbe00640 1 --2- 172.21.15.83:0/1394009843 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561fc637f850 0x561fc6383fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:56.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.322+0000 7f0efbe00640 1 --2- 172.21.15.83:0/1394009843 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561fc6374860 0x561fc6383560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:56.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.322+0000 7f0f00e00640 1 -- 172.21.15.83:0/1394009843 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561fc637f850 msgr2=0x561fc6383fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:56.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.322+0000 7f0f00e00640 1 --2- 172.21.15.83:0/1394009843 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561fc637f850 0x561fc6383fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:56.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.322+0000 7f0f00e00640 1 -- 172.21.15.83:0/1394009843 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561fc6374860 msgr2=0x561fc6383560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:56.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.322+0000 7f0f00e00640 1 --2- 172.21.15.83:0/1394009843 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561fc6374860 0x561fc6383560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:56.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.322+0000 7f0f00e00640 1 -- 172.21.15.83:0/1394009843 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561fc5fac0f0 con 0x561fc6375180 2024-04-24T09:06:56.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.322+0000 7f0efbe00640 1 --2- 172.21.15.83:0/1394009843 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561fc6374860 0x561fc6383560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:06:56.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.323+0000 7f0f00e00640 1 --2- 172.21.15.83:0/1394009843 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561fc6375180 0x561fc6383aa0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0ef002a9a0 tx=0x7f0ef0004550 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e6a947734c823efa server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:56.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.323+0000 7f0ee2a00640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-04-24T09:06:56.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.323+0000 7f0ee2a00640 1 -- 172.21.15.83:0/1394009843 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef0035040 con 0x561fc6375180 2024-04-24T09:06:56.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.323+0000 7f0ee2a00640 1 -- 172.21.15.83:0/1394009843 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9583+0+0 (secure 0 0 0) 0x7f0ef0005040 con 0x561fc6375180 2024-04-24T09:06:56.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.323+0000 7f0f06765740 1 -- 172.21.15.83:0/1394009843 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ee80041b0 con 0x561fc6375180 2024-04-24T09:06:56.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.324+0000 7f0ee2a00640 1 -- 172.21.15.83:0/1394009843 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f0ef0005240 con 0x561fc6375180 2024-04-24T09:06:56.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.324+0000 7f0f06765740 10 client.28213 fetch_fsmap learned FSMap version 49 2024-04-24T09:06:56.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.324+0000 7f0f06765740 1 -- 172.21.15.83:0/1394009843 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x561fc629f810 con 0x561fc6375180 2024-04-24T09:06:56.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.324+0000 7f0ee2a00640 1 -- 172.21.15.83:0/1394009843 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0ee80041b0 con 0x561fc6375180 2024-04-24T09:06:56.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.325+0000 7f0ee2a00640 1 -- 172.21.15.83:0/1394009843 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 49) ==== 8406+0+0 (secure 0 0 0) 0x7f0ef0004800 con 0x561fc6375180 2024-04-24T09:06:56.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.325+0000 7f0f06765740 10 client.28213 fetch_fsmap finished waiting for FSMap version 49 2024-04-24T09:06:56.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.325+0000 7f0f06765740 10 client.28213 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:06:56.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.325+0000 7f0f06765740 20 client.28213 populate_metadata read hostname 'smithi083' 2024-04-24T09:06:56.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.325+0000 7f0f06765740 1 --2- 172.21.15.83:0/1394009843 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x561fc6407700 0x561fc6409b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:56.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.325+0000 7f0f06765740 4 client.28213 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T09:06:56.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.325+0000 7f0f06765740 1 -- 172.21.15.83:0/1394009843 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561fc63783e0 con 0x561fc6407700 2024-04-24T09:06:56.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.325+0000 7f0efbe00640 1 --2- 172.21.15.83:0/1394009843 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x561fc6407700 0x561fc6409b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:56.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.326+0000 7f0efbe00640 1 --2- 172.21.15.83:0/1394009843 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x561fc6407700 0x561fc6409b20 crc :-1 s=READY pgs=390 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14ffd4d17bdb1040 server_cookie=1445699161164f01 in_seq=0 out_seq=0 2024-04-24T09:06:56.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.326+0000 7f0ee2a00640 10 client.28213 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:06:56.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.326+0000 7f0ee2a00640 1 -- 172.21.15.83:0/1394009843 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x561fc63783e0 con 0x561fc6407700 2024-04-24T09:06:56.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.327+0000 7f0ee2a00640 10 client.28213 handle_command_reply: tid=0 2024-04-24T09:06:56.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.327+0000 7f0f06765740 1 client.28213 shutdown 2024-04-24T09:06:56.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.327+0000 7f0f06765740 1 -- 172.21.15.83:0/1394009843 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561fc6375180 msgr2=0x561fc6383aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:56.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.327+0000 7f0f06765740 1 --2- 172.21.15.83:0/1394009843 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561fc6375180 0x561fc6383aa0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0ef002a9a0 tx=0x7f0ef0004550 comp rx=0 tx=0).stop 2024-04-24T09:06:56.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.327+0000 7f0f06765740 1 -- 172.21.15.83:0/1394009843 shutdown_connections 2024-04-24T09:06:56.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.327+0000 7f0f06765740 1 --2- 172.21.15.83:0/1394009843 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x561fc6407700 0x561fc6409b20 crc :-1 s=READY pgs=390 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:56.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.327+0000 7f0f06765740 1 --2- 172.21.15.83:0/1394009843 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561fc637f850 0x561fc6383fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:56.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.327+0000 7f0ee2a00640 0 client.28213 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:06:56.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.327+0000 7f0f06765740 1 --2- 172.21.15.83:0/1394009843 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561fc6375180 0x561fc6383aa0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:56.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.327+0000 7f0f06765740 1 --2- 172.21.15.83:0/1394009843 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561fc6374860 0x561fc6383560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:56.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.327+0000 7f0f06765740 1 -- 172.21.15.83:0/1394009843 >> 172.21.15.83:0/1394009843 conn(0x561fc6370b80 msgr2=0x561fc636fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:56.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.327+0000 7f0f06765740 1 -- 172.21.15.83:0/1394009843 shutdown_connections 2024-04-24T09:06:56.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.327+0000 7f0f06765740 1 -- 172.21.15.83:0/1394009843 wait complete. 2024-04-24T09:06:56.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.327+0000 7f0f06765740 20 client.28213 trim_cache size 0 max 16384 2024-04-24T09:06:56.330 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T09:06:56.330 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-04-24T09:06:56.330 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-04-24T09:06:56.331 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T09:06:56.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.329+0000 7f0f03800640 1 -- 172.21.15.83:0/742216016 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f0ee40a3d20 msgr2=0x7f0ee40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:56.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.329+0000 7f0f03800640 1 --2- 172.21.15.83:0/742216016 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f0ee40a3d20 0x7f0ee40a61e0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f0ef4099590 tx=0x7f0ef4004a40 comp rx=0 tx=0).stop 2024-04-24T09:06:56.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.329+0000 7f0f03800640 1 -- 172.21.15.83:0/742216016 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0efc113030 msgr2=0x7f0efc06dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:56.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.329+0000 7f0f03800640 1 --2- 172.21.15.83:0/742216016 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0efc113030 0x7f0efc06dfe0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0ef4005fd0 tx=0x7f0ef400c6a0 comp rx=0 tx=0).stop 2024-04-24T09:06:56.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.330+0000 7f0f03800640 1 -- 172.21.15.83:0/742216016 shutdown_connections 2024-04-24T09:06:56.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.330+0000 7f0f03800640 1 --2- 172.21.15.83:0/742216016 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f0ee40a3d20 0x7f0ee40a61e0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:56.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.330+0000 7f0f03800640 1 --2- 172.21.15.83:0/742216016 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0efc113030 0x7f0efc06dfe0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:56.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.330+0000 7f0f03800640 1 --2- 172.21.15.83:0/742216016 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0efc074cc0 0x7f0efc06daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:56.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.330+0000 7f0f03800640 1 --2- 172.21.15.83:0/742216016 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0efc0743a0 0x7f0efc073a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:56.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.330+0000 7f0f03800640 1 -- 172.21.15.83:0/742216016 >> 172.21.15.83:0/742216016 conn(0x7f0efc0fdb10 msgr2=0x7f0efc0ff8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:56.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.330+0000 7f0f03800640 1 -- 172.21.15.83:0/742216016 shutdown_connections 2024-04-24T09:06:56.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.331+0000 7f0f03800640 1 -- 172.21.15.83:0/742216016 wait complete. 2024-04-24T09:06:56.355 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:06:56 smithi083 ceph-mon[27907]: pgmap v480: 129 pgs: 129 active+clean; 372 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 985 KiB/s rd, 3.3 MiB/s wr, 208 op/s 2024-04-24T09:06:56.357 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T09:06:56.357 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T09:06:56.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:06:56 smithi138 ceph-mon[31441]: pgmap v480: 129 pgs: 129 active+clean; 372 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 985 KiB/s rd, 3.3 MiB/s wr, 208 op/s 2024-04-24T09:06:56.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:06:56 smithi189 ceph-mon[31791]: pgmap v480: 129 pgs: 129 active+clean; 372 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 985 KiB/s rd, 3.3 MiB/s wr, 208 op/s 2024-04-24T09:06:56.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.596+0000 7fc00a400640 1 -- 172.21.15.83:0/2161955257 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc00410e660 msgr2=0x7fc004110a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:56.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.596+0000 7fc00a400640 1 --2- 172.21.15.83:0/2161955257 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc00410e660 0x7fc004110a70 secure :-1 s=READY pgs=2209 cs=0 l=1 rev1=1 crypto rx=0x7fbffc009ca0 tx=0x7fbffc02a8b0 comp rx=0 tx=0).stop 2024-04-24T09:06:56.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.596+0000 7fc00a400640 1 -- 172.21.15.83:0/2161955257 shutdown_connections 2024-04-24T09:06:56.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.596+0000 7fc00a400640 1 --2- 172.21.15.83:0/2161955257 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc00410e660 0x7fc004110a70 unknown :-1 s=CLOSED pgs=2209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:56.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.596+0000 7fc00a400640 1 --2- 172.21.15.83:0/2161955257 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0040696d0 0x7fc004069b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:56.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.596+0000 7fc00a400640 1 --2- 172.21.15.83:0/2161955257 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc004102d00 0x7fc004069190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:56.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.596+0000 7fc00a400640 1 -- 172.21.15.83:0/2161955257 >> 172.21.15.83:0/2161955257 conn(0x7fc004076840 msgr2=0x7fc004076c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:56.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.597+0000 7fc00a400640 1 -- 172.21.15.83:0/2161955257 shutdown_connections 2024-04-24T09:06:56.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.597+0000 7fc00a400640 1 -- 172.21.15.83:0/2161955257 wait complete. 2024-04-24T09:06:56.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.598+0000 7fc00a400640 1 Processor -- start 2024-04-24T09:06:56.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.598+0000 7fc00a400640 1 -- start start 2024-04-24T09:06:56.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.599+0000 7fc00a400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0040696d0 0x7fc00410e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:56.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.599+0000 7fc00a400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc004102d00 0x7fc00410b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:56.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.599+0000 7fc00a400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc00410e660 0x7fc00410b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:56.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.599+0000 7fc00a400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc004114240 con 0x7fc004102d00 2024-04-24T09:06:56.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.599+0000 7fc00a400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc0041140c0 con 0x7fc0040696d0 2024-04-24T09:06:56.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.599+0000 7fc00a400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc0041143c0 con 0x7fc00410e660 2024-04-24T09:06:56.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.599+0000 7fc003e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc004102d00 0x7fc00410b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:56.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.600+0000 7fc003e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc004102d00 0x7fc00410b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51054/0 (socket says 172.21.15.83:51054) 2024-04-24T09:06:56.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.600+0000 7fc003e00640 1 -- 172.21.15.83:0/4205323512 learned_addr learned my addr 172.21.15.83:0/4205323512 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:06:56.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.600+0000 7fc009000640 1 --2- 172.21.15.83:0/4205323512 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0040696d0 0x7fc00410e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:56.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.600+0000 7fc003e00640 1 -- 172.21.15.83:0/4205323512 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc00410e660 msgr2=0x7fc00410b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:56.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.600+0000 7fc003e00640 1 --2- 172.21.15.83:0/4205323512 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc00410e660 0x7fc00410b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:56.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.600+0000 7fc003e00640 1 -- 172.21.15.83:0/4205323512 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0040696d0 msgr2=0x7fc00410e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:56.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.600+0000 7fc003e00640 1 --2- 172.21.15.83:0/4205323512 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0040696d0 0x7fc00410e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:56.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.600+0000 7fc003e00640 1 -- 172.21.15.83:0/4205323512 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0041aaf80 con 0x7fc004102d00 2024-04-24T09:06:56.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.600+0000 7fc009000640 1 --2- 172.21.15.83:0/4205323512 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0040696d0 0x7fc00410e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:06:56.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.601+0000 7fc003e00640 1 --2- 172.21.15.83:0/4205323512 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc004102d00 0x7fc00410b130 secure :-1 s=READY pgs=2210 cs=0 l=1 rev1=1 crypto rx=0x7fbff000d950 tx=0x7fbff000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:56.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.601+0000 7fc001600640 1 -- 172.21.15.83:0/4205323512 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbff0017030 con 0x7fc004102d00 2024-04-24T09:06:56.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.601+0000 7fc001600640 1 -- 172.21.15.83:0/4205323512 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbff00099b0 con 0x7fc004102d00 2024-04-24T09:06:56.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.601+0000 7fc001600640 1 -- 172.21.15.83:0/4205323512 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbff00047a0 con 0x7fc004102d00 2024-04-24T09:06:56.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.601+0000 7fc00a400640 1 -- 172.21.15.83:0/4205323512 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0041ab150 con 0x7fc004102d00 2024-04-24T09:06:56.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.602+0000 7fc00a400640 1 -- 172.21.15.83:0/4205323512 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0041ab5f0 con 0x7fc004102d00 2024-04-24T09:06:56.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.603+0000 7fc00a400640 1 -- 172.21.15.83:0/4205323512 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc004101370 con 0x7fc004102d00 2024-04-24T09:06:56.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.605+0000 7fc001600640 1 -- 172.21.15.83:0/4205323512 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fbff0009b60 con 0x7fc004102d00 2024-04-24T09:06:56.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.607+0000 7fc001600640 1 --2- 172.21.15.83:0/4205323512 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fbfec0a3d60 0x7fbfec0a6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:56.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.607+0000 7fc001600640 1 -- 172.21.15.83:0/4205323512 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9583+0+0 (secure 0 0 0) 0x7fbff00d06c0 con 0x7fc004102d00 2024-04-24T09:06:56.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.608+0000 7fc009000640 1 --2- 172.21.15.83:0/4205323512 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fbfec0a3d60 0x7fbfec0a6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:56.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.609+0000 7fc009000640 1 --2- 172.21.15.83:0/4205323512 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fbfec0a3d60 0x7fbfec0a6220 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fbff8005d90 tx=0x7fbff8005d20 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:56.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:56.615+0000 7fc001600640 1 -- 172.21.15.83:0/4205323512 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbff009c8e0 con 0x7fc004102d00 2024-04-24T09:06:57.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.038+0000 7fc00a400640 1 -- 172.21.15.83:0/4205323512 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc00410c610 con 0x7fc004102d00 2024-04-24T09:06:57.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.040+0000 7fc001600640 1 -- 172.21.15.83:0/4205323512 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) ==== 94+0+11961 (secure 0 0 0) 0x7fbff009c1e0 con 0x7fc004102d00 2024-04-24T09:06:57.041 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T09:06:57.041 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":49,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/358872257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":358872257},{"type":"v1","addr":"172.21.15.83:6835","nonce":358872257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3157070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3157070851},{"type":"v1","addr":"172.21.15.83:6837","nonce":3157070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":49,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T08:48:29.422576+0000","modified":"2024-04-24T09:06:49.458485+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34520,"mds_1":34493,"mds_2":34451,"mds_3":34427,"mds_4":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14658":{"gid":14658,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6839/4085470844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":4085470844},{"type":"v1","addr":"172.21.15.83:6839","nonce":4085470844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14676":{"gid":14676,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6841/2634754210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":2634754210},{"type":"v1","addr":"172.21.15.83:6841","nonce":2634754210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24382":{"gid":24382,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/1859921310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1859921310},{"type":"v1","addr":"172.21.15.189:6833","nonce":1859921310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/2241923197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2241923197},{"type":"v1","addr":"172.21.15.189:6835","nonce":2241923197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.189:6839/2548965336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":2548965336},{"type":"v1","addr":"172.21.15.189:6839","nonce":2548965336}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26686":{"gid":26686,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6837/1678817161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":1678817161},{"type":"v1","addr":"172.21.15.189:6837","nonce":1678817161}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.138:6833/1157967658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1157967658},{"type":"v1","addr":"172.21.15.138:6833","nonce":1157967658}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.138:6835/1909219758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1909219758},{"type":"v1","addr":"172.21.15.138:6835","nonce":1909219758}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34493":{"gid":34493,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.138:6837/900566754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":900566754},{"type":"v1","addr":"172.21.15.138:6837","nonce":900566754}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34520":{"gid":34520,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.138:6839/2236932393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2236932393},{"type":"v1","addr":"172.21.15.138:6839","nonce":2236932393}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34520,"qdb_cluster":[24454,34520,34427,34451,34493]},"id":1}]} 2024-04-24T09:06:57.041 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 49 2024-04-24T09:06:57.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.043+0000 7fc00a400640 1 -- 172.21.15.83:0/4205323512 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fbfec0a3d60 msgr2=0x7fbfec0a6220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:57.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.043+0000 7fc00a400640 1 --2- 172.21.15.83:0/4205323512 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fbfec0a3d60 0x7fbfec0a6220 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fbff8005d90 tx=0x7fbff8005d20 comp rx=0 tx=0).stop 2024-04-24T09:06:57.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.043+0000 7fc00a400640 1 -- 172.21.15.83:0/4205323512 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc004102d00 msgr2=0x7fc00410b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:57.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.043+0000 7fc00a400640 1 --2- 172.21.15.83:0/4205323512 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc004102d00 0x7fc00410b130 secure :-1 s=READY pgs=2210 cs=0 l=1 rev1=1 crypto rx=0x7fbff000d950 tx=0x7fbff000de10 comp rx=0 tx=0).stop 2024-04-24T09:06:57.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.043+0000 7fc00a400640 1 -- 172.21.15.83:0/4205323512 shutdown_connections 2024-04-24T09:06:57.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.043+0000 7fc00a400640 1 --2- 172.21.15.83:0/4205323512 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fbfec0a3d60 0x7fbfec0a6220 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:57.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.043+0000 7fc00a400640 1 --2- 172.21.15.83:0/4205323512 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc00410e660 0x7fc00410b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:57.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.043+0000 7fc00a400640 1 --2- 172.21.15.83:0/4205323512 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc004102d00 0x7fc00410b130 unknown :-1 s=CLOSED pgs=2210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:57.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.043+0000 7fc00a400640 1 --2- 172.21.15.83:0/4205323512 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0040696d0 0x7fc00410e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:57.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.043+0000 7fc00a400640 1 -- 172.21.15.83:0/4205323512 >> 172.21.15.83:0/4205323512 conn(0x7fc004076840 msgr2=0x7fc0040fffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:57.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.043+0000 7fc00a400640 1 -- 172.21.15.83:0/4205323512 shutdown_connections 2024-04-24T09:06:57.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.044+0000 7fc00a400640 1 -- 172.21.15.83:0/4205323512 wait complete. 2024-04-24T09:06:57.065 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-24T09:06:57.159 INFO:tasks.workunit.client.0.smithi083.stdout: 10 11483 8.97 MB/sec warmup 88 sec latency 75.205 ms 2024-04-24T09:06:57.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.305+0000 7fb4cd400640 1 -- 172.21.15.83:0/501520269 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4c8104320 msgr2=0x7fb4c8104700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:57.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.305+0000 7fb4cd400640 1 --2- 172.21.15.83:0/501520269 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4c8104320 0x7fb4c8104700 secure :-1 s=READY pgs=2211 cs=0 l=1 rev1=1 crypto rx=0x7fb4b8009900 tx=0x7fb4b802a410 comp rx=0 tx=0).stop 2024-04-24T09:06:57.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.305+0000 7fb4cd400640 1 -- 172.21.15.83:0/501520269 shutdown_connections 2024-04-24T09:06:57.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.305+0000 7fb4cd400640 1 --2- 172.21.15.83:0/501520269 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4c81093f0 0x7fb4c8110fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:57.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.305+0000 7fb4cd400640 1 --2- 172.21.15.83:0/501520269 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb4c8104cd0 0x7fb4c8108d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:57.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.305+0000 7fb4cd400640 1 --2- 172.21.15.83:0/501520269 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4c8104320 0x7fb4c8104700 unknown :-1 s=CLOSED pgs=2211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:57.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.305+0000 7fb4cd400640 1 -- 172.21.15.83:0/501520269 >> 172.21.15.83:0/501520269 conn(0x7fb4c80ffb40 msgr2=0x7fb4c8101f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:57.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.305+0000 7fb4cd400640 1 -- 172.21.15.83:0/501520269 shutdown_connections 2024-04-24T09:06:57.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.305+0000 7fb4cd400640 1 -- 172.21.15.83:0/501520269 wait complete. 2024-04-24T09:06:57.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.307+0000 7fb4cd400640 1 Processor -- start 2024-04-24T09:06:57.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.307+0000 7fb4cd400640 1 -- start start 2024-04-24T09:06:57.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.307+0000 7fb4cd400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4c8104320 0x7fb4c81a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:57.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.307+0000 7fb4cd400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4c81093f0 0x7fb4c81a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:57.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.307+0000 7fb4cd400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb4c819a720 0x7fb4c819aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:57.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.307+0000 7fb4cd400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c81143a0 con 0x7fb4c81093f0 2024-04-24T09:06:57.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.307+0000 7fb4cd400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c8114220 con 0x7fb4c8104320 2024-04-24T09:06:57.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.307+0000 7fb4cd400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c8114520 con 0x7fb4c819a720 2024-04-24T09:06:57.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.308+0000 7fb4c7e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4c8104320 0x7fb4c81a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:57.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.308+0000 7fb4c7e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4c8104320 0x7fb4c81a0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:56778/0 (socket says 172.21.15.83:56778) 2024-04-24T09:06:57.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.308+0000 7fb4c7e00640 1 -- 172.21.15.83:0/2005479343 learned_addr learned my addr 172.21.15.83:0/2005479343 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:06:57.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.308+0000 7fb4c7400640 1 --2- 172.21.15.83:0/2005479343 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4c81093f0 0x7fb4c81a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:57.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.308+0000 7fb4cca00640 1 --2- 172.21.15.83:0/2005479343 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb4c819a720 0x7fb4c819aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:57.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.309+0000 7fb4c7400640 1 -- 172.21.15.83:0/2005479343 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb4c819a720 msgr2=0x7fb4c819aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:57.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.309+0000 7fb4c7400640 1 --2- 172.21.15.83:0/2005479343 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb4c819a720 0x7fb4c819aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:57.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.309+0000 7fb4c7400640 1 -- 172.21.15.83:0/2005479343 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4c8104320 msgr2=0x7fb4c81a0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:57.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.309+0000 7fb4c7400640 1 --2- 172.21.15.83:0/2005479343 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4c8104320 0x7fb4c81a0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:57.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.309+0000 7fb4c7400640 1 -- 172.21.15.83:0/2005479343 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4c819b320 con 0x7fb4c81093f0 2024-04-24T09:06:57.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.310+0000 7fb4c7400640 1 --2- 172.21.15.83:0/2005479343 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4c81093f0 0x7fb4c81a0ba0 secure :-1 s=READY pgs=2212 cs=0 l=1 rev1=1 crypto rx=0x7fb4c806b560 tx=0x7fb4b400dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:57.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.310+0000 7fb4c4c00640 1 -- 172.21.15.83:0/2005479343 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b4017070 con 0x7fb4c81093f0 2024-04-24T09:06:57.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.310+0000 7fb4cd400640 1 -- 172.21.15.83:0/2005479343 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4c81a7400 con 0x7fb4c81093f0 2024-04-24T09:06:57.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.311+0000 7fb4cd400640 1 -- 172.21.15.83:0/2005479343 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4c81a7800 con 0x7fb4c81093f0 2024-04-24T09:06:57.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.311+0000 7fb4c4c00640 1 -- 172.21.15.83:0/2005479343 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb4b40099d0 con 0x7fb4c81093f0 2024-04-24T09:06:57.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.311+0000 7fb4c4c00640 1 -- 172.21.15.83:0/2005479343 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b40047a0 con 0x7fb4c81093f0 2024-04-24T09:06:57.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.315+0000 7fb4d1830740 1 Processor -- start 2024-04-24T09:06:57.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.316+0000 7fb4d1830740 1 -- start start 2024-04-24T09:06:57.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.319+0000 7fb4d1830740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0c9805350 0x55f0c9805730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:57.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.319+0000 7fb4d1830740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0c9805c70 0x55f0c980fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:57.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.319+0000 7fb4d1830740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0c9810340 0x55f0c9812730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:57.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.319+0000 7fb4d1830740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55f0c97621d0 con 0x55f0c9810340 2024-04-24T09:06:57.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.319+0000 7fb4d1830740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f0c9708450 con 0x55f0c9805c70 2024-04-24T09:06:57.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.319+0000 7fb4d1830740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55f0c975e190 con 0x55f0c9805350 2024-04-24T09:06:57.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.319+0000 7fb4c4c00640 1 -- 172.21.15.83:0/2005479343 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb4b4004940 con 0x7fb4c81093f0 2024-04-24T09:06:57.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.320+0000 7fb4c7e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0c9805c70 0x55f0c980fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:57.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.320+0000 7fb4cca00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0c9805350 0x55f0c9805730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:57.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.321+0000 7fb4cca00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0c9805350 0x55f0c9805730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:49538/0 (socket says 172.21.15.83:49538) 2024-04-24T09:06:57.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.321+0000 7fb4c7e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0c9805c70 0x55f0c980fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:56794/0 (socket says 172.21.15.83:56794) 2024-04-24T09:06:57.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.321+0000 7fb4cca00640 1 -- 172.21.15.83:0/3527301474 learned_addr learned my addr 172.21.15.83:0/3527301474 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:06:57.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.321+0000 7fb4c7400640 1 --2- 172.21.15.83:0/3527301474 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0c9810340 0x55f0c9812730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:57.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.322+0000 7fb4cca00640 1 -- 172.21.15.83:0/3527301474 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0c9805c70 msgr2=0x55f0c980fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:57.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.322+0000 7fb4cca00640 1 --2- 172.21.15.83:0/3527301474 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0c9805c70 0x55f0c980fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:57.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.322+0000 7fb4cca00640 1 -- 172.21.15.83:0/3527301474 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0c9810340 msgr2=0x55f0c9812730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:57.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.322+0000 7fb4cca00640 1 --2- 172.21.15.83:0/3527301474 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0c9810340 0x55f0c9812730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:57.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.322+0000 7fb4cca00640 1 -- 172.21.15.83:0/3527301474 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0c97c5310 con 0x55f0c9805350 2024-04-24T09:06:57.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.323+0000 7fb4c4c00640 1 --2- 172.21.15.83:0/2005479343 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb4ac0a3e20 0x7fb4ac0a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:57.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.323+0000 7fb4c4c00640 1 -- 172.21.15.83:0/2005479343 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9583+0+0 (secure 0 0 0) 0x7fb4b40d0700 con 0x7fb4c81093f0 2024-04-24T09:06:57.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.324+0000 7fb4cca00640 1 --2- 172.21.15.83:0/3527301474 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0c9805350 0x55f0c9805730 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb4c0009f50 tx=0x7fb4c00118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46fcd2b164e70b39 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:57.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.325+0000 7fb4bd200640 1 -- 172.21.15.83:0/3527301474 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4c0004d20 con 0x55f0c9805350 2024-04-24T09:06:57.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.325+0000 7fb4bd200640 1 -- 172.21.15.83:0/3527301474 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb4c0004050 con 0x55f0c9805350 2024-04-24T09:06:57.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.326+0000 7fb4c7e00640 1 --2- 172.21.15.83:0/2005479343 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb4ac0a3e20 0x7fb4ac0a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:57.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.326+0000 7fb4bd200640 1 -- 172.21.15.83:0/3527301474 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4c0005280 con 0x55f0c9805350 2024-04-24T09:06:57.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.326+0000 7fb4d1830740 1 -- 172.21.15.83:0/3527301474 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0c9805350 msgr2=0x55f0c9805730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:57.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.326+0000 7fb4d1830740 1 --2- 172.21.15.83:0/3527301474 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0c9805350 0x55f0c9805730 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb4c0009f50 tx=0x7fb4c00118d0 comp rx=0 tx=0).stop 2024-04-24T09:06:57.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.327+0000 7fb4d1830740 1 -- 172.21.15.83:0/3527301474 shutdown_connections 2024-04-24T09:06:57.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.327+0000 7fb4d1830740 1 --2- 172.21.15.83:0/3527301474 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0c9810340 0x55f0c9812730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:57.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.327+0000 7fb4d1830740 1 --2- 172.21.15.83:0/3527301474 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0c9805c70 0x55f0c980fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:57.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.327+0000 7fb4d1830740 1 --2- 172.21.15.83:0/3527301474 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0c9805350 0x55f0c9805730 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:57.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.327+0000 7fb4d1830740 1 -- 172.21.15.83:0/3527301474 >> 172.21.15.83:0/3527301474 conn(0x55f0c9801670 msgr2=0x55f0c9801a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:57.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.327+0000 7fb4d1830740 1 -- 172.21.15.83:0/3527301474 shutdown_connections 2024-04-24T09:06:57.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.328+0000 7fb4d1830740 1 -- 172.21.15.83:0/3527301474 wait complete. 2024-04-24T09:06:57.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.328+0000 7fb4c7e00640 1 --2- 172.21.15.83:0/2005479343 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb4ac0a3e20 0x7fb4ac0a62e0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fb4b80096c0 tx=0x7fb4b80046c0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:57.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.328+0000 7fb4d1830740 1 -- start start 2024-04-24T09:06:57.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.329+0000 7fb4d1830740 1 Processor -- start 2024-04-24T09:06:57.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.330+0000 7fb4d1830740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0c9805350 0x55f0c980fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:57.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.330+0000 7fb4d1830740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0c9805c70 0x55f0c980cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:57.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.330+0000 7fb4d1830740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0c9810340 0x55f0c980d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:57.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.330+0000 7fb4d1830740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55f0c9813270 con 0x55f0c9805350 2024-04-24T09:06:57.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.330+0000 7fb4d1830740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f0c971b810 con 0x55f0c9810340 2024-04-24T09:06:57.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.330+0000 7fb4d1830740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55f0c98133f0 con 0x55f0c9805c70 2024-04-24T09:06:57.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.330+0000 7fb4d1830740 10 client.0 _ll_register_callbacks cb 0x55f0c96e9140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:06:57.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.330+0000 7fb4cca00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0c9810340 0x55f0c980d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:57.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.330+0000 7fb4cca00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0c9810340 0x55f0c980d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:56798/0 (socket says 172.21.15.83:56798) 2024-04-24T09:06:57.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.330+0000 7fb4cca00640 1 -- 172.21.15.83:0/873394023 learned_addr learned my addr 172.21.15.83:0/873394023 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:06:57.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.331+0000 7fb4c7400640 1 --2- 172.21.15.83:0/873394023 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0c9805c70 0x55f0c980cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:57.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.331+0000 7fb4c7e00640 1 --2- 172.21.15.83:0/873394023 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0c9805350 0x55f0c980fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:57.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.331+0000 7fb4c7e00640 1 -- 172.21.15.83:0/873394023 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0c9805c70 msgr2=0x55f0c980cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:57.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.331+0000 7fb4c7e00640 1 --2- 172.21.15.83:0/873394023 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0c9805c70 0x55f0c980cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:57.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.331+0000 7fb4c7e00640 1 -- 172.21.15.83:0/873394023 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0c9810340 msgr2=0x55f0c980d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:57.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.331+0000 7fb4c7e00640 1 --2- 172.21.15.83:0/873394023 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0c9810340 0x55f0c980d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:57.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.332+0000 7fb4c7e00640 1 -- 172.21.15.83:0/873394023 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0c972c3f0 con 0x55f0c9805350 2024-04-24T09:06:57.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.332+0000 7fb4c7400640 1 --2- 172.21.15.83:0/873394023 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0c9805c70 0x55f0c980cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:06:57.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.332+0000 7fb4c7e00640 1 --2- 172.21.15.83:0/873394023 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0c9805350 0x55f0c980fcd0 secure :-1 s=READY pgs=2213 cs=0 l=1 rev1=1 crypto rx=0x7fb4b8000c00 tx=0x7fb4b8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d47cf37c5cba81f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:57.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.332+0000 7fb4b1600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:06:57.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.332+0000 7fb4b1600640 1 -- 172.21.15.83:0/873394023 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b8035040 con 0x55f0c9805350 2024-04-24T09:06:57.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.332+0000 7fb4b1600640 1 -- 172.21.15.83:0/873394023 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9583+0+0 (secure 0 0 0) 0x7fb4b803d450 con 0x55f0c9805350 2024-04-24T09:06:57.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.333+0000 7fb4d1830740 1 -- 172.21.15.83:0/873394023 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4c0004050 con 0x55f0c9805350 2024-04-24T09:06:57.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.333+0000 7fb4b1600640 1 -- 172.21.15.83:0/873394023 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fb4b803d650 con 0x55f0c9805350 2024-04-24T09:06:57.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.333+0000 7fb4d1830740 10 client.20970 fetch_fsmap learned FSMap version 49 2024-04-24T09:06:57.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.333+0000 7fb4d1830740 1 -- 172.21.15.83:0/873394023 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55f0c94743f0 con 0x55f0c9805350 2024-04-24T09:06:57.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.334+0000 7fb4b1600640 1 -- 172.21.15.83:0/873394023 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb4c0004050 con 0x55f0c9805350 2024-04-24T09:06:57.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.335+0000 7fb4b1600640 1 -- 172.21.15.83:0/873394023 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 49) ==== 8406+0+0 (secure 0 0 0) 0x7fb4b8044030 con 0x55f0c9805350 2024-04-24T09:06:57.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.335+0000 7fb4d1830740 10 client.20970 fetch_fsmap finished waiting for FSMap version 49 2024-04-24T09:06:57.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.335+0000 7fb4d1830740 10 client.20970 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-24T09:06:57.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.335+0000 7fb4d1830740 20 client.20970 populate_metadata read hostname 'smithi083' 2024-04-24T09:06:57.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.335+0000 7fb4d1830740 1 --2- 172.21.15.83:0/873394023 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55f0c9820460 0x55f0c98228a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:57.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.335+0000 7fb4d1830740 4 client.20970 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:06:57.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.335+0000 7fb4d1830740 1 -- 172.21.15.83:0/873394023 --> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f0c976fbf0 con 0x55f0c9820460 2024-04-24T09:06:57.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.337+0000 7fb4c7400640 1 --2- 172.21.15.83:0/873394023 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55f0c9820460 0x55f0c98228a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:57.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.339+0000 7fb4b1600640 10 client.20970 ms_handle_connect on v2:172.21.15.189:6838/2548965336 2024-04-24T09:06:57.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.339+0000 7fb4c7400640 1 --2- 172.21.15.83:0/873394023 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55f0c9820460 0x55f0c98228a0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a2db30903d386ddd server_cookie=9338a6178b606bd9 in_seq=0 out_seq=0 2024-04-24T09:06:57.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.341+0000 7fb4b1600640 1 -- 172.21.15.83:0/873394023 <== mds.4 v2:172.21.15.189:6838/2548965336 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f0c976fbf0 con 0x55f0c9820460 2024-04-24T09:06:57.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.341+0000 7fb4b1600640 10 client.20970 handle_command_reply: tid=0 2024-04-24T09:06:57.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.342+0000 7fb4d1830740 1 client.20970 shutdown 2024-04-24T09:06:57.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.343+0000 7fb4d1830740 1 -- 172.21.15.83:0/873394023 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0c9805350 msgr2=0x55f0c980fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:57.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.343+0000 7fb4d1830740 1 --2- 172.21.15.83:0/873394023 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0c9805350 0x55f0c980fcd0 secure :-1 s=READY pgs=2213 cs=0 l=1 rev1=1 crypto rx=0x7fb4b8000c00 tx=0x7fb4b8009260 comp rx=0 tx=0).stop 2024-04-24T09:06:57.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.343+0000 7fb4d1830740 1 -- 172.21.15.83:0/873394023 shutdown_connections 2024-04-24T09:06:57.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.343+0000 7fb4d1830740 1 --2- 172.21.15.83:0/873394023 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55f0c9820460 0x55f0c98228a0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:57.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.343+0000 7fb4d1830740 1 --2- 172.21.15.83:0/873394023 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0c9810340 0x55f0c980d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:57.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.343+0000 7fb4d1830740 1 --2- 172.21.15.83:0/873394023 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0c9805c70 0x55f0c980cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:57.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.343+0000 7fb4d1830740 1 --2- 172.21.15.83:0/873394023 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0c9805350 0x55f0c980fcd0 unknown :-1 s=CLOSED pgs=2213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:57.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.343+0000 7fb4d1830740 1 -- 172.21.15.83:0/873394023 >> 172.21.15.83:0/873394023 conn(0x55f0c9801670 msgr2=0x55f0c9807370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:57.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.343+0000 7fb4b1600640 0 client.20970 ms_handle_reset on v2:172.21.15.189:6838/2548965336 2024-04-24T09:06:57.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.343+0000 7fb4d1830740 1 -- 172.21.15.83:0/873394023 shutdown_connections 2024-04-24T09:06:57.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.343+0000 7fb4d1830740 1 -- 172.21.15.83:0/873394023 wait complete. 2024-04-24T09:06:57.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.344+0000 7fb4d1830740 20 client.20970 trim_cache size 0 max 16384 2024-04-24T09:06:57.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.370+0000 7fb4d1830740 1 Processor -- start 2024-04-24T09:06:57.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.371+0000 7fb4d1830740 1 -- start start 2024-04-24T09:06:57.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.371+0000 7fb4d1830740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0c9805350 0x55f0c981c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:57.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.371+0000 7fb4d1830740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0c9805c70 0x55f0c980f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:57.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.371+0000 7fb4d1830740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0c9810340 0x55f0c98281b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:57.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.371+0000 7fb4d1830740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a8010100 con 0x55f0c9810340 2024-04-24T09:06:57.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.371+0000 7fb4d1830740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a8011dc0 con 0x55f0c9805350 2024-04-24T09:06:57.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.371+0000 7fb4d1830740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a80107d0 con 0x55f0c9805c70 2024-04-24T09:06:57.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.371+0000 7fb4cca00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0c9810340 0x55f0c98281b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:57.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.371+0000 7fb4c7e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0c9805350 0x55f0c981c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:57.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.371+0000 7fb4cca00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0c9810340 0x55f0c98281b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51098/0 (socket says 172.21.15.83:51098) 2024-04-24T09:06:57.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.371+0000 7fb4cca00640 1 -- 172.21.15.83:0/2999016631 learned_addr learned my addr 172.21.15.83:0/2999016631 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:06:57.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.371+0000 7fb4c7400640 1 --2- 172.21.15.83:0/2999016631 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0c9805c70 0x55f0c980f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:57.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.372+0000 7fb4c7e00640 1 -- 172.21.15.83:0/2999016631 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0c9805c70 msgr2=0x55f0c980f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:57.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.372+0000 7fb4c7e00640 1 --2- 172.21.15.83:0/2999016631 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0c9805c70 0x55f0c980f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:57.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.372+0000 7fb4c7e00640 1 -- 172.21.15.83:0/2999016631 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0c9810340 msgr2=0x55f0c98281b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:57.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.372+0000 7fb4c7e00640 1 --2- 172.21.15.83:0/2999016631 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0c9810340 0x55f0c98281b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:57.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.372+0000 7fb4c7e00640 1 -- 172.21.15.83:0/2999016631 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0c980d8b0 con 0x55f0c9805350 2024-04-24T09:06:57.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.372+0000 7fb4cca00640 1 --2- 172.21.15.83:0/2999016631 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0c9810340 0x55f0c98281b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:06:57.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.373+0000 7fb4c7e00640 1 --2- 172.21.15.83:0/2999016631 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0c9805350 0x55f0c981c210 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb4b802aa00 tx=0x7fb4b802a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f68a3a757fe2d57d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:57.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.373+0000 7fb4b3e00640 1 -- 172.21.15.83:0/2999016631 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b8004160 con 0x55f0c9805350 2024-04-24T09:06:57.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.373+0000 7fb4b3e00640 1 -- 172.21.15.83:0/2999016631 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb4b804c070 con 0x55f0c9805350 2024-04-24T09:06:57.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.373+0000 7fb4b3e00640 1 -- 172.21.15.83:0/2999016631 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b8031430 con 0x55f0c9805350 2024-04-24T09:06:57.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.374+0000 7fb4d1830740 1 -- 172.21.15.83:0/2999016631 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0c9805350 msgr2=0x55f0c981c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:57.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.374+0000 7fb4d1830740 1 --2- 172.21.15.83:0/2999016631 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0c9805350 0x55f0c981c210 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb4b802aa00 tx=0x7fb4b802a410 comp rx=0 tx=0).stop 2024-04-24T09:06:57.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.374+0000 7fb4d1830740 1 -- 172.21.15.83:0/2999016631 shutdown_connections 2024-04-24T09:06:57.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.374+0000 7fb4d1830740 1 --2- 172.21.15.83:0/2999016631 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0c9810340 0x55f0c98281b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:57.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.374+0000 7fb4d1830740 1 --2- 172.21.15.83:0/2999016631 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0c9805c70 0x55f0c980f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:57.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.374+0000 7fb4d1830740 1 --2- 172.21.15.83:0/2999016631 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0c9805350 0x55f0c981c210 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:57.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.374+0000 7fb4d1830740 1 -- 172.21.15.83:0/2999016631 >> 172.21.15.83:0/2999016631 conn(0x55f0c9801670 msgr2=0x55f0c98082c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:57.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.374+0000 7fb4d1830740 1 -- 172.21.15.83:0/2999016631 shutdown_connections 2024-04-24T09:06:57.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.374+0000 7fb4d1830740 1 -- 172.21.15.83:0/2999016631 wait complete. 2024-04-24T09:06:57.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.375+0000 7fb4d1830740 1 -- start start 2024-04-24T09:06:57.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.375+0000 7fb4d1830740 1 Processor -- start 2024-04-24T09:06:57.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.376+0000 7fb4d1830740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0c9805350 0x55f0c980a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:57.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.376+0000 7fb4d1830740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0c9805c70 0x55f0c980a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:57.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.376+0000 7fb4d1830740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0c9810340 0x55f0c980acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:57.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.376+0000 7fb4d1830740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a8011630 con 0x55f0c9805c70 2024-04-24T09:06:57.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.376+0000 7fb4d1830740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a8010f00 con 0x55f0c9805350 2024-04-24T09:06:57.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.376+0000 7fb4d1830740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a800e100 con 0x55f0c9810340 2024-04-24T09:06:57.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.376+0000 7fb4d1830740 10 client.0 _ll_register_callbacks cb 0x7fb4b8004490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:06:57.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.376+0000 7fb4c7e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0c9805350 0x55f0c980a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:57.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.376+0000 7fb4c7e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0c9805350 0x55f0c980a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:56830/0 (socket says 172.21.15.83:56830) 2024-04-24T09:06:57.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.376+0000 7fb4c7400640 1 --2- 172.21.15.83:0/986795226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0c9805c70 0x55f0c980a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:57.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.376+0000 7fb4c7e00640 1 -- 172.21.15.83:0/986795226 learned_addr learned my addr 172.21.15.83:0/986795226 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:06:57.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.376+0000 7fb4cca00640 1 --2- 172.21.15.83:0/986795226 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0c9810340 0x55f0c980acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:57.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.377+0000 7fb4c7400640 1 -- 172.21.15.83:0/986795226 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0c9810340 msgr2=0x55f0c980acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:57.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.377+0000 7fb4c7400640 1 --2- 172.21.15.83:0/986795226 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0c9810340 0x55f0c980acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:57.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.377+0000 7fb4c7400640 1 -- 172.21.15.83:0/986795226 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0c9805350 msgr2=0x55f0c980a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:57.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.377+0000 7fb4c7400640 1 --2- 172.21.15.83:0/986795226 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0c9805350 0x55f0c980a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:57.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.377+0000 7fb4c7400640 1 -- 172.21.15.83:0/986795226 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0c953e7d0 con 0x55f0c9805c70 2024-04-24T09:06:57.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.377+0000 7fb4c7400640 1 --2- 172.21.15.83:0/986795226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0c9805c70 0x55f0c980a780 secure :-1 s=READY pgs=2214 cs=0 l=1 rev1=1 crypto rx=0x7fb4b40cd9d0 tx=0x7fb4b40d2ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0658f9b55da8585 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:57.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.377+0000 7fb4b1600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:06:57.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.378+0000 7fb4b1600640 1 -- 172.21.15.83:0/986795226 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b40a05d0 con 0x55f0c9805c70 2024-04-24T09:06:57.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.378+0000 7fb4b1600640 1 -- 172.21.15.83:0/986795226 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9583+0+0 (secure 0 0 0) 0x7fb4b4098890 con 0x55f0c9805c70 2024-04-24T09:06:57.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.378+0000 7fb4d1830740 1 -- 172.21.15.83:0/986795226 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4b804c070 con 0x55f0c9805c70 2024-04-24T09:06:57.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.378+0000 7fb4b1600640 1 -- 172.21.15.83:0/986795226 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fb4b40a4e90 con 0x55f0c9805c70 2024-04-24T09:06:57.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.379+0000 7fb4d1830740 10 client.20976 fetch_fsmap learned FSMap version 49 2024-04-24T09:06:57.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.379+0000 7fb4d1830740 1 -- 172.21.15.83:0/986795226 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55f0c9810aa0 con 0x55f0c9805c70 2024-04-24T09:06:57.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.379+0000 7fb4b1600640 1 -- 172.21.15.83:0/986795226 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb4b804c070 con 0x55f0c9805c70 2024-04-24T09:06:57.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.379+0000 7fb4b1600640 1 -- 172.21.15.83:0/986795226 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 49) ==== 8406+0+0 (secure 0 0 0) 0x7fb4b40d2bb0 con 0x55f0c9805c70 2024-04-24T09:06:57.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.379+0000 7fb4d1830740 10 client.20976 fetch_fsmap finished waiting for FSMap version 49 2024-04-24T09:06:57.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.379+0000 7fb4d1830740 10 client.20976 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-24T09:06:57.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.379+0000 7fb4d1830740 20 client.20976 populate_metadata read hostname 'smithi083' 2024-04-24T09:06:57.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.379+0000 7fb4d1830740 1 --2- 172.21.15.83:0/986795226 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55f0c98111b0 0x55f0c9817710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:57.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.379+0000 7fb4d1830740 4 client.20976 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:06:57.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.379+0000 7fb4d1830740 1 -- 172.21.15.83:0/986795226 --> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f0c9769a60 con 0x55f0c98111b0 2024-04-24T09:06:57.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.380+0000 7fb4c7e00640 1 --2- 172.21.15.83:0/986795226 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55f0c98111b0 0x55f0c9817710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:57.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.381+0000 7fb4c7e00640 1 --2- 172.21.15.83:0/986795226 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55f0c98111b0 0x55f0c9817710 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9aa7debd9b84071b server_cookie=2729af4226bc1a8c in_seq=0 out_seq=0 2024-04-24T09:06:57.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.381+0000 7fb4b1600640 10 client.20976 ms_handle_connect on v2:172.21.15.189:6838/2548965336 2024-04-24T09:06:57.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.381+0000 7fb4b1600640 1 -- 172.21.15.83:0/986795226 <== mds.4 v2:172.21.15.189:6838/2548965336 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f0c9769a60 con 0x55f0c98111b0 2024-04-24T09:06:57.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.382+0000 7fb4b1600640 10 client.20976 handle_command_reply: tid=0 2024-04-24T09:06:57.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.382+0000 7fb4d1830740 1 client.20976 shutdown 2024-04-24T09:06:57.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.382+0000 7fb4d1830740 1 -- 172.21.15.83:0/986795226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0c9805c70 msgr2=0x55f0c980a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:57.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.382+0000 7fb4d1830740 1 --2- 172.21.15.83:0/986795226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0c9805c70 0x55f0c980a780 secure :-1 s=READY pgs=2214 cs=0 l=1 rev1=1 crypto rx=0x7fb4b40cd9d0 tx=0x7fb4b40d2ad0 comp rx=0 tx=0).stop 2024-04-24T09:06:57.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.382+0000 7fb4d1830740 1 -- 172.21.15.83:0/986795226 shutdown_connections 2024-04-24T09:06:57.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.382+0000 7fb4d1830740 1 --2- 172.21.15.83:0/986795226 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55f0c98111b0 0x55f0c9817710 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:57.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.383+0000 7fb4d1830740 1 --2- 172.21.15.83:0/986795226 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0c9810340 0x55f0c980acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:57.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.383+0000 7fb4d1830740 1 --2- 172.21.15.83:0/986795226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0c9805c70 0x55f0c980a780 unknown :-1 s=CLOSED pgs=2214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:57.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.383+0000 7fb4d1830740 1 --2- 172.21.15.83:0/986795226 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0c9805350 0x55f0c980a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:57.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.383+0000 7fb4d1830740 1 -- 172.21.15.83:0/986795226 >> 172.21.15.83:0/986795226 conn(0x55f0c9801670 msgr2=0x55f0c980b780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:57.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.383+0000 7fb4b1600640 0 client.20976 ms_handle_reset on v2:172.21.15.189:6838/2548965336 2024-04-24T09:06:57.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.383+0000 7fb4d1830740 1 -- 172.21.15.83:0/986795226 shutdown_connections 2024-04-24T09:06:57.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.383+0000 7fb4d1830740 1 -- 172.21.15.83:0/986795226 wait complete. 2024-04-24T09:06:57.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.383+0000 7fb4d1830740 20 client.20976 trim_cache size 0 max 16384 2024-04-24T09:06:57.387 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:06:57.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.384+0000 7fb4cd400640 1 -- 172.21.15.83:0/2005479343 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb4ac0a3e20 msgr2=0x7fb4ac0a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:57.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.384+0000 7fb4cd400640 1 --2- 172.21.15.83:0/2005479343 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb4ac0a3e20 0x7fb4ac0a62e0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fb4b80096c0 tx=0x7fb4b80046c0 comp rx=0 tx=0).stop 2024-04-24T09:06:57.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.384+0000 7fb4cd400640 1 -- 172.21.15.83:0/2005479343 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4c81093f0 msgr2=0x7fb4c81a0ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:57.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.384+0000 7fb4cd400640 1 --2- 172.21.15.83:0/2005479343 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4c81093f0 0x7fb4c81a0ba0 secure :-1 s=READY pgs=2212 cs=0 l=1 rev1=1 crypto rx=0x7fb4c806b560 tx=0x7fb4b400dbb0 comp rx=0 tx=0).stop 2024-04-24T09:06:57.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.384+0000 7fb4cd400640 1 -- 172.21.15.83:0/2005479343 shutdown_connections 2024-04-24T09:06:57.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.384+0000 7fb4cd400640 1 --2- 172.21.15.83:0/2005479343 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb4ac0a3e20 0x7fb4ac0a62e0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:57.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.384+0000 7fb4cd400640 1 --2- 172.21.15.83:0/2005479343 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb4c819a720 0x7fb4c819aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:57.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.384+0000 7fb4cd400640 1 --2- 172.21.15.83:0/2005479343 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4c81093f0 0x7fb4c81a0ba0 unknown :-1 s=CLOSED pgs=2212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:57.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.384+0000 7fb4cd400640 1 --2- 172.21.15.83:0/2005479343 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4c8104320 0x7fb4c81a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:57.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.384+0000 7fb4cd400640 1 -- 172.21.15.83:0/2005479343 >> 172.21.15.83:0/2005479343 conn(0x7fb4c80ffb40 msgr2=0x7fb4c8100e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:57.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.385+0000 7fb4cd400640 1 -- 172.21.15.83:0/2005479343 shutdown_connections 2024-04-24T09:06:57.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.385+0000 7fb4cd400640 1 -- 172.21.15.83:0/2005479343 wait complete. 2024-04-24T09:06:57.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:06:57 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/4205323512' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:06:57.407 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-24T09:06:57.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:06:57 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/4205323512' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:06:57.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:06:57 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/4205323512' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:06:57.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.662+0000 7f394ca00640 1 -- 172.21.15.83:0/3472730620 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39481019b0 msgr2=0x7f39481059f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:57.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.662+0000 7f394ca00640 1 --2- 172.21.15.83:0/3472730620 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39481019b0 0x7f39481059f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f3934009900 tx=0x7f393402a400 comp rx=0 tx=0).stop 2024-04-24T09:06:57.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.662+0000 7f394ca00640 1 -- 172.21.15.83:0/3472730620 shutdown_connections 2024-04-24T09:06:57.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.662+0000 7f394ca00640 1 --2- 172.21.15.83:0/3472730620 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f39481060d0 0x7f394810dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:57.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.662+0000 7f394ca00640 1 --2- 172.21.15.83:0/3472730620 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39481019b0 0x7f39481059f0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:57.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.662+0000 7f394ca00640 1 --2- 172.21.15.83:0/3472730620 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3948101000 0x7f39481013e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:57.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.662+0000 7f394ca00640 1 -- 172.21.15.83:0/3472730620 >> 172.21.15.83:0/3472730620 conn(0x7f39480fc880 msgr2=0x7f39480feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:57.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.662+0000 7f394ca00640 1 -- 172.21.15.83:0/3472730620 shutdown_connections 2024-04-24T09:06:57.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.662+0000 7f394ca00640 1 -- 172.21.15.83:0/3472730620 wait complete. 2024-04-24T09:06:57.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.663+0000 7f394ca00640 1 Processor -- start 2024-04-24T09:06:57.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.663+0000 7f394ca00640 1 -- start start 2024-04-24T09:06:57.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.664+0000 7f394ca00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3948101000 0x7f3948198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:57.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.664+0000 7f394ca00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f39481060d0 0x7f3948198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:57.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.664+0000 7f394ca00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f394819d000 0x7f394819d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:57.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.664+0000 7f394ca00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3948110ff0 con 0x7f3948101000 2024-04-24T09:06:57.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.664+0000 7f394ca00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3948110e70 con 0x7f394819d000 2024-04-24T09:06:57.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.664+0000 7f394ca00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3948111170 con 0x7f39481060d0 2024-04-24T09:06:57.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.664+0000 7f3947400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3948101000 0x7f3948198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:57.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.664+0000 7f3947e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f394819d000 0x7f394819d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:57.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.664+0000 7f3947e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f394819d000 0x7f394819d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:56850/0 (socket says 172.21.15.83:56850) 2024-04-24T09:06:57.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.664+0000 7f3947400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3948101000 0x7f3948198740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51140/0 (socket says 172.21.15.83:51140) 2024-04-24T09:06:57.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.664+0000 7f3947e00640 1 -- 172.21.15.83:0/4245177735 learned_addr learned my addr 172.21.15.83:0/4245177735 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:06:57.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.665+0000 7f3946a00640 1 --2- 172.21.15.83:0/4245177735 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f39481060d0 0x7f3948198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:57.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.665+0000 7f3947400640 1 -- 172.21.15.83:0/4245177735 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f39481060d0 msgr2=0x7f3948198c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:57.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.665+0000 7f3947400640 1 --2- 172.21.15.83:0/4245177735 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f39481060d0 0x7f3948198c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:57.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.665+0000 7f3947400640 1 -- 172.21.15.83:0/4245177735 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f394819d000 msgr2=0x7f394819d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:57.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.665+0000 7f3947400640 1 --2- 172.21.15.83:0/4245177735 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f394819d000 0x7f394819d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:57.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.665+0000 7f3947400640 1 -- 172.21.15.83:0/4245177735 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f394819daf0 con 0x7f3948101000 2024-04-24T09:06:57.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.665+0000 7f3947400640 1 --2- 172.21.15.83:0/4245177735 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3948101000 0x7f3948198740 secure :-1 s=READY pgs=2215 cs=0 l=1 rev1=1 crypto rx=0x7f393800e9f0 tx=0x7f393800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:57.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.666+0000 7f393fe00640 1 -- 172.21.15.83:0/4245177735 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f393800ccc0 con 0x7f3948101000 2024-04-24T09:06:57.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.666+0000 7f393fe00640 1 -- 172.21.15.83:0/4245177735 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f39380108e0 con 0x7f3948101000 2024-04-24T09:06:57.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.666+0000 7f393fe00640 1 -- 172.21.15.83:0/4245177735 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3938018880 con 0x7f3948101000 2024-04-24T09:06:57.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.666+0000 7f394ca00640 1 -- 172.21.15.83:0/4245177735 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f394819dde0 con 0x7f3948101000 2024-04-24T09:06:57.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.666+0000 7f394ca00640 1 -- 172.21.15.83:0/4245177735 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39481a5550 con 0x7f3948101000 2024-04-24T09:06:57.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.671+0000 7f3950d9d740 1 Processor -- start 2024-04-24T09:06:57.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.671+0000 7f3950d9d740 1 -- start start 2024-04-24T09:06:57.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.672+0000 7f3950d9d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ca87a7b40 0x562ca87a7f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:57.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.672+0000 7f3950d9d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562ca87a8460 0x562ca87b25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:57.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.672+0000 7f3950d9d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562ca87b2b30 0x562ca87b4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:57.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.672+0000 7f3950d9d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562ca870d1d0 con 0x562ca87b2b30 2024-04-24T09:06:57.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.672+0000 7f3950d9d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562ca86b3450 con 0x562ca87a7b40 2024-04-24T09:06:57.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.672+0000 7f3950d9d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x562ca8709190 con 0x562ca87a8460 2024-04-24T09:06:57.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.672+0000 7f393fe00640 1 -- 172.21.15.83:0/4245177735 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3938010a90 con 0x7f3948101000 2024-04-24T09:06:57.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.672+0000 7f3947e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ca87a7b40 0x562ca87a7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:57.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.672+0000 7f3947e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ca87a7b40 0x562ca87a7f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:56860/0 (socket says 172.21.15.83:56860) 2024-04-24T09:06:57.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.672+0000 7f3947e00640 1 -- 172.21.15.83:0/2849159676 learned_addr learned my addr 172.21.15.83:0/2849159676 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:06:57.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.672+0000 7f3946a00640 1 --2- 172.21.15.83:0/2849159676 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562ca87b2b30 0x562ca87b4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:57.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.672+0000 7f3947400640 1 --2- 172.21.15.83:0/2849159676 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562ca87a8460 0x562ca87b25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:57.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.673+0000 7f3947e00640 1 -- 172.21.15.83:0/2849159676 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562ca87a8460 msgr2=0x562ca87b25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:57.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.673+0000 7f3947e00640 1 --2- 172.21.15.83:0/2849159676 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562ca87a8460 0x562ca87b25f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:57.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.673+0000 7f3947e00640 1 -- 172.21.15.83:0/2849159676 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562ca87b2b30 msgr2=0x562ca87b4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:57.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.673+0000 7f3947e00640 1 --2- 172.21.15.83:0/2849159676 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562ca87b2b30 0x562ca87b4f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:57.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.673+0000 7f3947e00640 1 -- 172.21.15.83:0/2849159676 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ca8770310 con 0x562ca87a7b40 2024-04-24T09:06:57.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.673+0000 7f3947400640 1 --2- 172.21.15.83:0/2849159676 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562ca87a8460 0x562ca87b25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:06:57.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.673+0000 7f3946a00640 1 --2- 172.21.15.83:0/2849159676 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562ca87b2b30 0x562ca87b4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:06:57.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.673+0000 7f3947e00640 1 --2- 172.21.15.83:0/2849159676 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ca87a7b40 0x562ca87a7f20 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3940009fb0 tx=0x7f3940011870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55c5a07fec434394 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:57.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.674+0000 7f393fe00640 1 --2- 172.21.15.83:0/4245177735 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f392c0a3da0 0x7f392c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:57.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.674+0000 7f3933e00640 1 -- 172.21.15.83:0/2849159676 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39400127d0 con 0x562ca87a7b40 2024-04-24T09:06:57.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.674+0000 7f3933e00640 1 -- 172.21.15.83:0/2849159676 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3940012e20 con 0x562ca87a7b40 2024-04-24T09:06:57.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.674+0000 7f3933e00640 1 -- 172.21.15.83:0/2849159676 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f394001ae40 con 0x562ca87a7b40 2024-04-24T09:06:57.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.674+0000 7f3947400640 1 --2- 172.21.15.83:0/4245177735 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f392c0a3da0 0x7f392c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:57.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.674+0000 7f393fe00640 1 -- 172.21.15.83:0/4245177735 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9583+0+0 (secure 0 0 0) 0x7f39380d1120 con 0x7f3948101000 2024-04-24T09:06:57.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.675+0000 7f3950d9d740 1 -- 172.21.15.83:0/2849159676 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ca87a7b40 msgr2=0x562ca87a7f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:57.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.675+0000 7f3950d9d740 1 --2- 172.21.15.83:0/2849159676 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ca87a7b40 0x562ca87a7f20 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3940009fb0 tx=0x7f3940011870 comp rx=0 tx=0).stop 2024-04-24T09:06:57.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.675+0000 7f3947400640 1 --2- 172.21.15.83:0/4245177735 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f392c0a3da0 0x7f392c0a6260 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x562ca87a9100 tx=0x7f3938009700 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:57.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.675+0000 7f3950d9d740 1 -- 172.21.15.83:0/2849159676 shutdown_connections 2024-04-24T09:06:57.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.675+0000 7f3950d9d740 1 --2- 172.21.15.83:0/2849159676 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562ca87b2b30 0x562ca87b4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:57.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.675+0000 7f3950d9d740 1 --2- 172.21.15.83:0/2849159676 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562ca87a8460 0x562ca87b25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:57.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.675+0000 7f3950d9d740 1 --2- 172.21.15.83:0/2849159676 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ca87a7b40 0x562ca87a7f20 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:57.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.675+0000 7f3950d9d740 1 -- 172.21.15.83:0/2849159676 >> 172.21.15.83:0/2849159676 conn(0x562ca87a3e60 msgr2=0x562ca87a4240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:57.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.675+0000 7f3950d9d740 1 -- 172.21.15.83:0/2849159676 shutdown_connections 2024-04-24T09:06:57.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.676+0000 7f3950d9d740 1 -- 172.21.15.83:0/2849159676 wait complete. 2024-04-24T09:06:57.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.676+0000 7f3950d9d740 1 -- start start 2024-04-24T09:06:57.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.676+0000 7f3950d9d740 1 Processor -- start 2024-04-24T09:06:57.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.677+0000 7f3950d9d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ca87a7b40 0x562ca87b25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:57.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.677+0000 7f3950d9d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562ca87a8460 0x562ca87af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:57.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.677+0000 7f3950d9d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562ca87b2b30 0x562ca87b0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:57.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.677+0000 7f3950d9d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562ca87b59c0 con 0x562ca87a8460 2024-04-24T09:06:57.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.677+0000 7f3950d9d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562ca86c6810 con 0x562ca87a7b40 2024-04-24T09:06:57.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.677+0000 7f3950d9d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x562ca87b5b40 con 0x562ca87b2b30 2024-04-24T09:06:57.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.677+0000 7f3950d9d740 10 client.0 _ll_register_callbacks cb 0x562ca8694140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:06:57.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.677+0000 7f3947e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ca87a7b40 0x562ca87b25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:57.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.677+0000 7f3947e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ca87a7b40 0x562ca87b25e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:56862/0 (socket says 172.21.15.83:56862) 2024-04-24T09:06:57.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.677+0000 7f3947e00640 1 -- 172.21.15.83:0/4180410202 learned_addr learned my addr 172.21.15.83:0/4180410202 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:06:57.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.677+0000 7f3946a00640 1 --2- 172.21.15.83:0/4180410202 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562ca87a8460 0x562ca87af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:57.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.677+0000 7f3947e00640 1 --2- 172.21.15.83:0/4180410202 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562ca87b2b30 0x562ca87b0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:57.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.678+0000 7f3946a00640 1 -- 172.21.15.83:0/4180410202 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562ca87b2b30 msgr2=0x562ca87b0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:57.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.678+0000 7f3946a00640 1 --2- 172.21.15.83:0/4180410202 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562ca87b2b30 0x562ca87b0010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:57.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.678+0000 7f3946a00640 1 -- 172.21.15.83:0/4180410202 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ca87a7b40 msgr2=0x562ca87b25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:57.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.678+0000 7f3946a00640 1 --2- 172.21.15.83:0/4180410202 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ca87a7b40 0x562ca87b25e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:57.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.678+0000 7f3946a00640 1 -- 172.21.15.83:0/4180410202 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ca86d73f0 con 0x562ca87a8460 2024-04-24T09:06:57.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.678+0000 7f3947e00640 1 --2- 172.21.15.83:0/4180410202 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562ca87b2b30 0x562ca87b0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:06:57.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.678+0000 7f3946a00640 1 --2- 172.21.15.83:0/4180410202 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562ca87a8460 0x562ca87af8d0 secure :-1 s=READY pgs=2216 cs=0 l=1 rev1=1 crypto rx=0x7f3934006010 tx=0x7f3934031740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81c7c017bb0bddf5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:57.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.679+0000 7f3930c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:06:57.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.679+0000 7f3930c00640 1 -- 172.21.15.83:0/4180410202 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3934049020 con 0x562ca87a8460 2024-04-24T09:06:57.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.679+0000 7f3930c00640 1 -- 172.21.15.83:0/4180410202 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9583+0+0 (secure 0 0 0) 0x7f3934004450 con 0x562ca87a8460 2024-04-24T09:06:57.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.679+0000 7f3950d9d740 1 -- 172.21.15.83:0/4180410202 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3940012e20 con 0x562ca87a8460 2024-04-24T09:06:57.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.679+0000 7f3930c00640 1 -- 172.21.15.83:0/4180410202 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f3934051050 con 0x562ca87a8460 2024-04-24T09:06:57.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.679+0000 7f3950d9d740 10 client.20988 fetch_fsmap learned FSMap version 49 2024-04-24T09:06:57.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.679+0000 7f3950d9d740 1 -- 172.21.15.83:0/4180410202 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x562ca841f3f0 con 0x562ca87a8460 2024-04-24T09:06:57.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.680+0000 7f3930c00640 1 -- 172.21.15.83:0/4180410202 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3940012e20 con 0x562ca87a8460 2024-04-24T09:06:57.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.680+0000 7f3930c00640 1 -- 172.21.15.83:0/4180410202 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 49) ==== 8406+0+0 (secure 0 0 0) 0x7f393403bbb0 con 0x562ca87a8460 2024-04-24T09:06:57.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.680+0000 7f3950d9d740 10 client.20988 fetch_fsmap finished waiting for FSMap version 49 2024-04-24T09:06:57.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.680+0000 7f3950d9d740 10 client.20988 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-24T09:06:57.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.680+0000 7f3950d9d740 20 client.20988 populate_metadata read hostname 'smithi083' 2024-04-24T09:06:57.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.680+0000 7f3950d9d740 1 --2- 172.21.15.83:0/4180410202 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x562ca87c2cd0 0x562ca87c50c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:57.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.680+0000 7f3950d9d740 4 client.20988 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:06:57.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.680+0000 7f3950d9d740 1 -- 172.21.15.83:0/4180410202 --> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562ca871abf0 con 0x562ca87c2cd0 2024-04-24T09:06:57.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.681+0000 7f3947e00640 1 --2- 172.21.15.83:0/4180410202 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x562ca87c2cd0 0x562ca87c50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:57.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.681+0000 7f3947e00640 1 --2- 172.21.15.83:0/4180410202 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x562ca87c2cd0 0x562ca87c50c0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1ae7ec89f5155b24 server_cookie=5bc417325e8ef0d3 in_seq=0 out_seq=0 2024-04-24T09:06:57.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.681+0000 7f3930c00640 10 client.20988 ms_handle_connect on v2:172.21.15.138:6832/1157967658 2024-04-24T09:06:57.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.682+0000 7f3930c00640 1 -- 172.21.15.83:0/4180410202 <== mds.3 v2:172.21.15.138:6832/1157967658 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562ca871abf0 con 0x562ca87c2cd0 2024-04-24T09:06:57.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.682+0000 7f3930c00640 10 client.20988 handle_command_reply: tid=0 2024-04-24T09:06:57.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.682+0000 7f3950d9d740 1 client.20988 shutdown 2024-04-24T09:06:57.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.682+0000 7f3950d9d740 1 -- 172.21.15.83:0/4180410202 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562ca87a8460 msgr2=0x562ca87af8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:57.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.682+0000 7f3950d9d740 1 --2- 172.21.15.83:0/4180410202 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562ca87a8460 0x562ca87af8d0 secure :-1 s=READY pgs=2216 cs=0 l=1 rev1=1 crypto rx=0x7f3934006010 tx=0x7f3934031740 comp rx=0 tx=0).stop 2024-04-24T09:06:57.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.683+0000 7f3950d9d740 1 -- 172.21.15.83:0/4180410202 shutdown_connections 2024-04-24T09:06:57.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.683+0000 7f3950d9d740 1 --2- 172.21.15.83:0/4180410202 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x562ca87c2cd0 0x562ca87c50c0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:57.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.683+0000 7f3950d9d740 1 --2- 172.21.15.83:0/4180410202 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562ca87b2b30 0x562ca87b0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:57.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.683+0000 7f3950d9d740 1 --2- 172.21.15.83:0/4180410202 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562ca87a8460 0x562ca87af8d0 secure :-1 s=CLOSED pgs=2216 cs=0 l=1 rev1=1 crypto rx=0x7f3934006010 tx=0x7f3934031740 comp rx=0 tx=0).stop 2024-04-24T09:06:57.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.683+0000 7f3930c00640 0 client.20988 ms_handle_reset on v2:172.21.15.138:6832/1157967658 2024-04-24T09:06:57.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.683+0000 7f3950d9d740 1 --2- 172.21.15.83:0/4180410202 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ca87a7b40 0x562ca87b25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:57.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.683+0000 7f3950d9d740 1 -- 172.21.15.83:0/4180410202 >> 172.21.15.83:0/4180410202 conn(0x562ca87a3e60 msgr2=0x562ca87a9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:57.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.683+0000 7f3950d9d740 1 -- 172.21.15.83:0/4180410202 shutdown_connections 2024-04-24T09:06:57.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.683+0000 7f3950d9d740 1 -- 172.21.15.83:0/4180410202 wait complete. 2024-04-24T09:06:57.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.683+0000 7f3950d9d740 20 client.20988 trim_cache size 0 max 16384 2024-04-24T09:06:57.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.709+0000 7f3950d9d740 1 Processor -- start 2024-04-24T09:06:57.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.709+0000 7f3950d9d740 1 -- start start 2024-04-24T09:06:57.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.709+0000 7f3950d9d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562ca87a7b40 0x562ca87be880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:57.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.709+0000 7f3950d9d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ca87b2b30 0x562ca87b2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:57.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.709+0000 7f3950d9d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562ca87b2650 0x562ca87a2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:57.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.709+0000 7f3950d9d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f39481ad230 con 0x562ca87b2650 2024-04-24T09:06:57.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.709+0000 7f3950d9d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f39481aeef0 con 0x562ca87b2b30 2024-04-24T09:06:57.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.709+0000 7f3950d9d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f39481ad900 con 0x562ca87a7b40 2024-04-24T09:06:57.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.710+0000 7f3946a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ca87b2b30 0x562ca87b2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:57.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.710+0000 7f3946a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ca87b2b30 0x562ca87b2110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:56870/0 (socket says 172.21.15.83:56870) 2024-04-24T09:06:57.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.710+0000 7f3946a00640 1 -- 172.21.15.83:0/3221072546 learned_addr learned my addr 172.21.15.83:0/3221072546 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:06:57.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.710+0000 7f3947e00640 1 --2- 172.21.15.83:0/3221072546 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562ca87b2650 0x562ca87a2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:57.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.711+0000 7f3947e00640 1 --2- 172.21.15.83:0/3221072546 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562ca87a7b40 0x562ca87be880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:57.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.711+0000 7f3946a00640 1 -- 172.21.15.83:0/3221072546 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562ca87a7b40 msgr2=0x562ca87be880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:57.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.711+0000 7f3946a00640 1 --2- 172.21.15.83:0/3221072546 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562ca87a7b40 0x562ca87be880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:57.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.711+0000 7f3946a00640 1 -- 172.21.15.83:0/3221072546 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562ca87b2650 msgr2=0x562ca87a2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:57.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.711+0000 7f3946a00640 1 --2- 172.21.15.83:0/3221072546 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562ca87b2650 0x562ca87a2dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:57.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.711+0000 7f3946a00640 1 -- 172.21.15.83:0/3221072546 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ca84e97d0 con 0x562ca87b2b30 2024-04-24T09:06:57.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.711+0000 7f3947e00640 1 --2- 172.21.15.83:0/3221072546 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562ca87b2650 0x562ca87a2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:06:57.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.712+0000 7f3946a00640 1 --2- 172.21.15.83:0/3221072546 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ca87b2b30 0x562ca87b2110 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3934009900 tx=0x7f3934001e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ceb1b847b0af3fac server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:57.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.713+0000 7f3933400640 1 -- 172.21.15.83:0/3221072546 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3934049070 con 0x562ca87b2b30 2024-04-24T09:06:57.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.713+0000 7f3933400640 1 -- 172.21.15.83:0/3221072546 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3934031e20 con 0x562ca87b2b30 2024-04-24T09:06:57.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.713+0000 7f3933400640 1 -- 172.21.15.83:0/3221072546 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3934035050 con 0x562ca87b2b30 2024-04-24T09:06:57.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.714+0000 7f3950d9d740 1 -- 172.21.15.83:0/3221072546 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ca87b2b30 msgr2=0x562ca87b2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:57.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.714+0000 7f3950d9d740 1 --2- 172.21.15.83:0/3221072546 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ca87b2b30 0x562ca87b2110 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3934009900 tx=0x7f3934001e70 comp rx=0 tx=0).stop 2024-04-24T09:06:57.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.715+0000 7f3950d9d740 1 -- 172.21.15.83:0/3221072546 shutdown_connections 2024-04-24T09:06:57.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.715+0000 7f3950d9d740 1 --2- 172.21.15.83:0/3221072546 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562ca87b2650 0x562ca87a2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:57.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.715+0000 7f3950d9d740 1 --2- 172.21.15.83:0/3221072546 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ca87b2b30 0x562ca87b2110 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:57.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.715+0000 7f3950d9d740 1 --2- 172.21.15.83:0/3221072546 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562ca87a7b40 0x562ca87be880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:57.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.715+0000 7f3950d9d740 1 -- 172.21.15.83:0/3221072546 >> 172.21.15.83:0/3221072546 conn(0x562ca87a3e60 msgr2=0x562ca87a26c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:57.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.715+0000 7f3950d9d740 1 -- 172.21.15.83:0/3221072546 shutdown_connections 2024-04-24T09:06:57.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.715+0000 7f3950d9d740 1 -- 172.21.15.83:0/3221072546 wait complete. 2024-04-24T09:06:57.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.716+0000 7f3950d9d740 1 -- start start 2024-04-24T09:06:57.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.716+0000 7f3950d9d740 1 Processor -- start 2024-04-24T09:06:57.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.717+0000 7f3950d9d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562ca87a7b40 0x562ca87baf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:57.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.717+0000 7f3950d9d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ca87b2650 0x562ca87abba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:57.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.717+0000 7f3950d9d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562ca87b2b30 0x562ca87ac0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:57.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.717+0000 7f3950d9d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f39481ae760 con 0x562ca87b2b30 2024-04-24T09:06:57.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.718+0000 7f3950d9d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f39481ae030 con 0x562ca87b2650 2024-04-24T09:06:57.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.718+0000 7f3950d9d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f39481ab230 con 0x562ca87a7b40 2024-04-24T09:06:57.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.718+0000 7f3950d9d740 10 client.0 _ll_register_callbacks cb 0x7f3934009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:06:57.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.718+0000 7f3946a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ca87b2650 0x562ca87abba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:57.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.718+0000 7f3946a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ca87b2650 0x562ca87abba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:56880/0 (socket says 172.21.15.83:56880) 2024-04-24T09:06:57.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.718+0000 7f3946a00640 1 -- 172.21.15.83:0/2818199843 learned_addr learned my addr 172.21.15.83:0/2818199843 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:06:57.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.718+0000 7f3947e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562ca87a7b40 0x562ca87baf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:57.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.718+0000 7f3947e00640 1 --2- 172.21.15.83:0/2818199843 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562ca87b2b30 0x562ca87ac0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:57.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.719+0000 7f3946a00640 1 -- 172.21.15.83:0/2818199843 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562ca87a7b40 msgr2=0x562ca87baf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:57.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.719+0000 7f3946a00640 1 --2- 172.21.15.83:0/2818199843 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562ca87a7b40 0x562ca87baf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:57.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.719+0000 7f3946a00640 1 -- 172.21.15.83:0/2818199843 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562ca87b2b30 msgr2=0x562ca87ac0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:57.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.719+0000 7f3946a00640 1 --2- 172.21.15.83:0/2818199843 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562ca87b2b30 0x562ca87ac0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:57.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.719+0000 7f3946a00640 1 -- 172.21.15.83:0/2818199843 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ca87ac6a0 con 0x562ca87b2650 2024-04-24T09:06:57.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.719+0000 7f3947e00640 1 --2- 172.21.15.83:0/2818199843 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562ca87b2b30 0x562ca87ac0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:06:57.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.719+0000 7f3946a00640 1 --2- 172.21.15.83:0/2818199843 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ca87b2650 0x562ca87abba0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3934000c00 tx=0x7f3934004a80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ed612b2157330cb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:57.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.719+0000 7f3930c00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:06:57.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.720+0000 7f3930c00640 1 -- 172.21.15.83:0/2818199843 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3934035070 con 0x562ca87b2650 2024-04-24T09:06:57.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.720+0000 7f3930c00640 1 -- 172.21.15.83:0/2818199843 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9583+0+0 (secure 0 0 0) 0x7f3934031970 con 0x562ca87b2650 2024-04-24T09:06:57.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.720+0000 7f3950d9d740 1 -- 172.21.15.83:0/2818199843 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3934031e20 con 0x562ca87b2650 2024-04-24T09:06:57.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.721+0000 7f3930c00640 1 -- 172.21.15.83:0/2818199843 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f3934049040 con 0x562ca87b2650 2024-04-24T09:06:57.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.721+0000 7f3950d9d740 10 client.38144 fetch_fsmap learned FSMap version 49 2024-04-24T09:06:57.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.722+0000 7f3950d9d740 1 -- 172.21.15.83:0/2818199843 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x562ca87ad9a0 con 0x562ca87b2650 2024-04-24T09:06:57.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.723+0000 7f3930c00640 1 -- 172.21.15.83:0/2818199843 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3934031e20 con 0x562ca87b2650 2024-04-24T09:06:57.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.723+0000 7f3930c00640 1 -- 172.21.15.83:0/2818199843 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 49) ==== 8406+0+0 (secure 0 0 0) 0x7f3934003910 con 0x562ca87b2650 2024-04-24T09:06:57.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.724+0000 7f3950d9d740 10 client.38144 fetch_fsmap finished waiting for FSMap version 49 2024-04-24T09:06:57.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.724+0000 7f3950d9d740 10 client.38144 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-24T09:06:57.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.724+0000 7f3950d9d740 20 client.38144 populate_metadata read hostname 'smithi083' 2024-04-24T09:06:57.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.724+0000 7f3950d9d740 1 --2- 172.21.15.83:0/2818199843 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x562ca882a640 0x562ca882ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:57.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.724+0000 7f3950d9d740 4 client.38144 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:06:57.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.724+0000 7f3950d9d740 1 -- 172.21.15.83:0/2818199843 --> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562ca8714a60 con 0x562ca882a640 2024-04-24T09:06:57.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.724+0000 7f3947e00640 1 --2- 172.21.15.83:0/2818199843 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x562ca882a640 0x562ca882ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:57.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.725+0000 7f3947e00640 1 --2- 172.21.15.83:0/2818199843 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x562ca882a640 0x562ca882ca60 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6081b34cc43fa4d9 server_cookie=14ad312c1a3a1194 in_seq=0 out_seq=0 2024-04-24T09:06:57.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.725+0000 7f3930c00640 10 client.38144 ms_handle_connect on v2:172.21.15.138:6832/1157967658 2024-04-24T09:06:57.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.726+0000 7f3930c00640 1 -- 172.21.15.83:0/2818199843 <== mds.3 v2:172.21.15.138:6832/1157967658 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562ca8714a60 con 0x562ca882a640 2024-04-24T09:06:57.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.726+0000 7f3930c00640 10 client.38144 handle_command_reply: tid=0 2024-04-24T09:06:57.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.726+0000 7f3950d9d740 1 client.38144 shutdown 2024-04-24T09:06:57.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.726+0000 7f3950d9d740 1 -- 172.21.15.83:0/2818199843 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ca87b2650 msgr2=0x562ca87abba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:57.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.726+0000 7f3950d9d740 1 --2- 172.21.15.83:0/2818199843 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ca87b2650 0x562ca87abba0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3934000c00 tx=0x7f3934004a80 comp rx=0 tx=0).stop 2024-04-24T09:06:57.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.726+0000 7f3950d9d740 1 -- 172.21.15.83:0/2818199843 shutdown_connections 2024-04-24T09:06:57.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.726+0000 7f3950d9d740 1 --2- 172.21.15.83:0/2818199843 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562ca87b2b30 0x562ca87ac0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:57.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.726+0000 7f3950d9d740 1 --2- 172.21.15.83:0/2818199843 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ca87b2650 0x562ca87abba0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:57.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.726+0000 7f3950d9d740 1 --2- 172.21.15.83:0/2818199843 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x562ca882a640 0x562ca882ca60 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:57.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.726+0000 7f3950d9d740 1 --2- 172.21.15.83:0/2818199843 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562ca87a7b40 0x562ca87baf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:57.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.726+0000 7f3950d9d740 1 -- 172.21.15.83:0/2818199843 >> 172.21.15.83:0/2818199843 conn(0x562ca87a3e60 msgr2=0x562ca87a2f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:57.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.727+0000 7f3930c00640 0 client.38144 ms_handle_reset on v2:172.21.15.138:6832/1157967658 2024-04-24T09:06:57.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.727+0000 7f3950d9d740 1 -- 172.21.15.83:0/2818199843 shutdown_connections 2024-04-24T09:06:57.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.727+0000 7f3950d9d740 1 -- 172.21.15.83:0/2818199843 wait complete. 2024-04-24T09:06:57.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.727+0000 7f3950d9d740 20 client.38144 trim_cache size 0 max 16384 2024-04-24T09:06:57.729 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:06:57.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.728+0000 7f394ca00640 1 -- 172.21.15.83:0/4245177735 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f392c0a3da0 msgr2=0x7f392c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:57.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.728+0000 7f394ca00640 1 --2- 172.21.15.83:0/4245177735 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f392c0a3da0 0x7f392c0a6260 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x562ca87a9100 tx=0x7f3938009700 comp rx=0 tx=0).stop 2024-04-24T09:06:57.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.728+0000 7f394ca00640 1 -- 172.21.15.83:0/4245177735 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3948101000 msgr2=0x7f3948198740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:57.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.728+0000 7f394ca00640 1 --2- 172.21.15.83:0/4245177735 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3948101000 0x7f3948198740 secure :-1 s=READY pgs=2215 cs=0 l=1 rev1=1 crypto rx=0x7f393800e9f0 tx=0x7f393800eeb0 comp rx=0 tx=0).stop 2024-04-24T09:06:57.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.728+0000 7f394ca00640 1 -- 172.21.15.83:0/4245177735 shutdown_connections 2024-04-24T09:06:57.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.728+0000 7f394ca00640 1 --2- 172.21.15.83:0/4245177735 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f392c0a3da0 0x7f392c0a6260 secure :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0x562ca87a9100 tx=0x7f3938009700 comp rx=0 tx=0).stop 2024-04-24T09:06:57.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.728+0000 7f394ca00640 1 --2- 172.21.15.83:0/4245177735 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f394819d000 0x7f394819d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:57.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.728+0000 7f394ca00640 1 --2- 172.21.15.83:0/4245177735 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f39481060d0 0x7f3948198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:57.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.728+0000 7f394ca00640 1 --2- 172.21.15.83:0/4245177735 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3948101000 0x7f3948198740 secure :-1 s=CLOSED pgs=2215 cs=0 l=1 rev1=1 crypto rx=0x7f393800e9f0 tx=0x7f393800eeb0 comp rx=0 tx=0).stop 2024-04-24T09:06:57.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.728+0000 7f394ca00640 1 -- 172.21.15.83:0/4245177735 >> 172.21.15.83:0/4245177735 conn(0x7f39480fc880 msgr2=0x7f3948069610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:57.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.729+0000 7f394ca00640 1 -- 172.21.15.83:0/4245177735 shutdown_connections 2024-04-24T09:06:57.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.729+0000 7f394ca00640 1 -- 172.21.15.83:0/4245177735 wait complete. 2024-04-24T09:06:57.750 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T09:06:57.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.993+0000 7f8fbbe00640 1 -- 172.21.15.83:0/356345306 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8fbc102110 msgr2=0x7f8fbc1024f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:57.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.993+0000 7f8fbbe00640 1 --2- 172.21.15.83:0/356345306 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8fbc102110 0x7f8fbc1024f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8fac009900 tx=0x7f8fac02a3f0 comp rx=0 tx=0).stop 2024-04-24T09:06:57.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.993+0000 7f8fbbe00640 1 -- 172.21.15.83:0/356345306 shutdown_connections 2024-04-24T09:06:57.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.993+0000 7f8fbbe00640 1 --2- 172.21.15.83:0/356345306 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8fbc1071e0 0x7f8fbc10edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:57.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.993+0000 7f8fbbe00640 1 --2- 172.21.15.83:0/356345306 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8fbc102ac0 0x7f8fbc106b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:57.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.993+0000 7f8fbbe00640 1 --2- 172.21.15.83:0/356345306 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8fbc102110 0x7f8fbc1024f0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:57.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.993+0000 7f8fbbe00640 1 -- 172.21.15.83:0/356345306 >> 172.21.15.83:0/356345306 conn(0x7f8fbc0fd9d0 msgr2=0x7f8fbc0ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:57.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.994+0000 7f8fbbe00640 1 -- 172.21.15.83:0/356345306 shutdown_connections 2024-04-24T09:06:57.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.994+0000 7f8fbbe00640 1 -- 172.21.15.83:0/356345306 wait complete. 2024-04-24T09:06:57.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.995+0000 7f8fbbe00640 1 Processor -- start 2024-04-24T09:06:57.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.995+0000 7f8fbbe00640 1 -- start start 2024-04-24T09:06:57.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.995+0000 7f8fbbe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8fbc102110 0x7f8fbc1987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:57.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.995+0000 7f8fbbe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8fbc1071e0 0x7f8fbc198ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:57.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.995+0000 7f8fbbe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8fbc19d010 0x7f8fbc19d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:57.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.995+0000 7f8fbbe00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8fbc112120 con 0x7f8fbc19d010 2024-04-24T09:06:57.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.995+0000 7f8fbbe00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8fbc111fa0 con 0x7f8fbc102110 2024-04-24T09:06:57.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.995+0000 7f8fbbe00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8fbc1122a0 con 0x7f8fbc1071e0 2024-04-24T09:06:57.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.996+0000 7f8fbaa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8fbc102110 0x7f8fbc1987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:57.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.996+0000 7f8fbaa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8fbc102110 0x7f8fbc1987a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:56896/0 (socket says 172.21.15.83:56896) 2024-04-24T09:06:57.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.996+0000 7f8fbaa00640 1 -- 172.21.15.83:0/470697867 learned_addr learned my addr 172.21.15.83:0/470697867 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:06:57.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.996+0000 7f8fbb400640 1 --2- 172.21.15.83:0/470697867 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8fbc19d010 0x7f8fbc19d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:57.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.996+0000 7f8fba000640 1 --2- 172.21.15.83:0/470697867 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8fbc1071e0 0x7f8fbc198ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:57.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.996+0000 7f8fbaa00640 1 -- 172.21.15.83:0/470697867 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8fbc1071e0 msgr2=0x7f8fbc198ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:57.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.996+0000 7f8fbaa00640 1 --2- 172.21.15.83:0/470697867 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8fbc1071e0 0x7f8fbc198ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:57.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.996+0000 7f8fbaa00640 1 -- 172.21.15.83:0/470697867 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8fbc19d010 msgr2=0x7f8fbc19d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:57.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.996+0000 7f8fbaa00640 1 --2- 172.21.15.83:0/470697867 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8fbc19d010 0x7f8fbc19d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:57.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.996+0000 7f8fbaa00640 1 -- 172.21.15.83:0/470697867 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fbc19db00 con 0x7f8fbc102110 2024-04-24T09:06:57.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.997+0000 7f8fbaa00640 1 --2- 172.21.15.83:0/470697867 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8fbc102110 0x7f8fbc1987a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f8fac005940 tx=0x7f8fac005e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:57.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.997+0000 7f8fc0a00640 1 -- 172.21.15.83:0/470697867 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fac035070 con 0x7f8fbc102110 2024-04-24T09:06:57.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.997+0000 7f8fc0a00640 1 -- 172.21.15.83:0/470697867 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8fac03dc10 con 0x7f8fbc102110 2024-04-24T09:06:57.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.997+0000 7f8fbbe00640 1 -- 172.21.15.83:0/470697867 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fbc19dd90 con 0x7f8fbc102110 2024-04-24T09:06:57.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.997+0000 7f8fc0a00640 1 -- 172.21.15.83:0/470697867 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fac003db0 con 0x7f8fbc102110 2024-04-24T09:06:57.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:57.997+0000 7f8fbbe00640 1 -- 172.21.15.83:0/470697867 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fbc074520 con 0x7f8fbc102110 2024-04-24T09:06:58.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.001+0000 7f8fc4b72740 1 Processor -- start 2024-04-24T09:06:58.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.001+0000 7f8fc0a00640 1 -- 172.21.15.83:0/470697867 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8fac0035f0 con 0x7f8fbc102110 2024-04-24T09:06:58.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.001+0000 7f8fc4b72740 1 -- start start 2024-04-24T09:06:58.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.002+0000 7f8fc4b72740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5594f561d870 0x5594f561dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:58.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.002+0000 7f8fc4b72740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594f561e220 0x5594f5630e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:58.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.002+0000 7f8fc4b72740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5594f5631340 0x5594f5633730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:58.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.002+0000 7f8fc4b72740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5594f55831d0 con 0x5594f561e220 2024-04-24T09:06:58.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.002+0000 7f8fc4b72740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5594f5529450 con 0x5594f561d870 2024-04-24T09:06:58.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.002+0000 7f8fc4b72740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5594f557f190 con 0x5594f5631340 2024-04-24T09:06:58.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.002+0000 7f8fbaa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594f561e220 0x5594f5630e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:58.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.002+0000 7f8fbb400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5594f561d870 0x5594f561dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:58.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.002+0000 7f8fbaa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594f561e220 0x5594f5630e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51192/0 (socket says 172.21.15.83:51192) 2024-04-24T09:06:58.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.002+0000 7f8fbaa00640 1 -- 172.21.15.83:0/941443795 learned_addr learned my addr 172.21.15.83:0/941443795 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:06:58.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.002+0000 7f8fba000640 1 --2- 172.21.15.83:0/941443795 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5594f5631340 0x5594f5633730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:58.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.003+0000 7f8fbb400640 1 -- 172.21.15.83:0/941443795 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5594f5631340 msgr2=0x5594f5633730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:58.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.003+0000 7f8fbb400640 1 --2- 172.21.15.83:0/941443795 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5594f5631340 0x5594f5633730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:58.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.003+0000 7f8fbb400640 1 -- 172.21.15.83:0/941443795 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594f561e220 msgr2=0x5594f5630e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:58.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.003+0000 7f8fbb400640 1 --2- 172.21.15.83:0/941443795 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594f561e220 0x5594f5630e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:58.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.003+0000 7f8fbb400640 1 -- 172.21.15.83:0/941443795 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594f55e6310 con 0x5594f561d870 2024-04-24T09:06:58.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.003+0000 7f8fbaa00640 1 --2- 172.21.15.83:0/941443795 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594f561e220 0x5594f5630e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:06:58.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.003+0000 7f8fc0a00640 1 --2- 172.21.15.83:0/470697867 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f8fa00a3da0 0x7f8fa00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:58.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.003+0000 7f8fc0a00640 1 -- 172.21.15.83:0/470697867 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9583+0+0 (secure 0 0 0) 0x7f8fac0f5680 con 0x7f8fbc102110 2024-04-24T09:06:58.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.003+0000 7f8fbb400640 1 --2- 172.21.15.83:0/941443795 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5594f561d870 0x5594f561dc50 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4009f30 tx=0x7f8fb40118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=695ef9268ef46cf5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:58.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.004+0000 7f8fbaa00640 1 --2- 172.21.15.83:0/470697867 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f8fa00a3da0 0x7f8fa00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:58.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.004+0000 7f8fa7e00640 1 -- 172.21.15.83:0/941443795 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb4004d20 con 0x5594f561d870 2024-04-24T09:06:58.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.004+0000 7f8fa7e00640 1 -- 172.21.15.83:0/941443795 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8fb4004050 con 0x5594f561d870 2024-04-24T09:06:58.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.004+0000 7f8fa7e00640 1 -- 172.21.15.83:0/941443795 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb4005280 con 0x5594f561d870 2024-04-24T09:06:58.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.005+0000 7f8fc4b72740 1 -- 172.21.15.83:0/941443795 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5594f561d870 msgr2=0x5594f561dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:58.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.005+0000 7f8fbaa00640 1 --2- 172.21.15.83:0/470697867 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f8fa00a3da0 0x7f8fa00a6260 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f8fac0bd770 tx=0x7f8fac03d4f0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:58.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.005+0000 7f8fc4b72740 1 --2- 172.21.15.83:0/941443795 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5594f561d870 0x5594f561dc50 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4009f30 tx=0x7f8fb40118d0 comp rx=0 tx=0).stop 2024-04-24T09:06:58.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.005+0000 7f8fc4b72740 1 -- 172.21.15.83:0/941443795 shutdown_connections 2024-04-24T09:06:58.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.005+0000 7f8fc4b72740 1 --2- 172.21.15.83:0/941443795 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5594f5631340 0x5594f5633730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:58.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.005+0000 7f8fc4b72740 1 --2- 172.21.15.83:0/941443795 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594f561e220 0x5594f5630e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:58.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.005+0000 7f8fc4b72740 1 --2- 172.21.15.83:0/941443795 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5594f561d870 0x5594f561dc50 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:58.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.005+0000 7f8fc4b72740 1 -- 172.21.15.83:0/941443795 >> 172.21.15.83:0/941443795 conn(0x5594f5619e60 msgr2=0x5594f561a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:58.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.005+0000 7f8fc4b72740 1 -- 172.21.15.83:0/941443795 shutdown_connections 2024-04-24T09:06:58.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.005+0000 7f8fc4b72740 1 -- 172.21.15.83:0/941443795 wait complete. 2024-04-24T09:06:58.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.006+0000 7f8fc4b72740 1 -- start start 2024-04-24T09:06:58.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.006+0000 7f8fc4b72740 1 Processor -- start 2024-04-24T09:06:58.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.007+0000 7f8fc4b72740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5594f561d870 0x5594f562e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:58.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.007+0000 7f8fc4b72740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594f561e220 0x5594f562e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:58.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.007+0000 7f8fc4b72740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5594f5631340 0x5594f562ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:58.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.007+0000 7f8fc4b72740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5594f56341a0 con 0x5594f561e220 2024-04-24T09:06:58.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.007+0000 7f8fc4b72740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5594f553c810 con 0x5594f5631340 2024-04-24T09:06:58.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.007+0000 7f8fc4b72740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5594f5634320 con 0x5594f561d870 2024-04-24T09:06:58.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.007+0000 7f8fc4b72740 10 client.0 _ll_register_callbacks cb 0x5594f550a140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:06:58.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.007+0000 7f8fba000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594f561e220 0x5594f562e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:58.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.007+0000 7f8fba000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594f561e220 0x5594f562e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51206/0 (socket says 172.21.15.83:51206) 2024-04-24T09:06:58.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.007+0000 7f8fba000640 1 -- 172.21.15.83:0/3255280009 learned_addr learned my addr 172.21.15.83:0/3255280009 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:06:58.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.008+0000 7f8fba000640 1 -- 172.21.15.83:0/3255280009 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5594f561d870 msgr2=0x5594f562e130 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-24T09:06:58.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.008+0000 7f8fba000640 1 --2- 172.21.15.83:0/3255280009 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5594f561d870 0x5594f562e130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:58.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.008+0000 7f8fba000640 1 -- 172.21.15.83:0/3255280009 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5594f5631340 msgr2=0x5594f562ebb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-24T09:06:58.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.008+0000 7f8fba000640 1 --2- 172.21.15.83:0/3255280009 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5594f5631340 0x5594f562ebb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:58.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.008+0000 7f8fba000640 1 -- 172.21.15.83:0/3255280009 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594f554d3f0 con 0x5594f561e220 2024-04-24T09:06:58.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.008+0000 7f8fba000640 1 --2- 172.21.15.83:0/3255280009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594f561e220 0x5594f562e670 secure :-1 s=READY pgs=2217 cs=0 l=1 rev1=1 crypto rx=0x7f8fa800bdd0 tx=0x7f8fa80117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f81b69a53f000af9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:58.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.008+0000 7f8fa4c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:06:58.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.008+0000 7f8fa4c00640 1 -- 172.21.15.83:0/3255280009 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa8004230 con 0x5594f561e220 2024-04-24T09:06:58.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.008+0000 7f8fc4b72740 1 -- 172.21.15.83:0/3255280009 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8fb4004050 con 0x5594f561e220 2024-04-24T09:06:58.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.009+0000 7f8fa4c00640 1 -- 172.21.15.83:0/3255280009 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9583+0+0 (secure 0 0 0) 0x7f8fa80043d0 con 0x5594f561e220 2024-04-24T09:06:58.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.009+0000 7f8fa4c00640 1 -- 172.21.15.83:0/3255280009 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f8fa8035050 con 0x5594f561e220 2024-04-24T09:06:58.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.009+0000 7f8fc4b72740 10 client.21006 fetch_fsmap learned FSMap version 49 2024-04-24T09:06:58.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.009+0000 7f8fc4b72740 1 -- 172.21.15.83:0/3255280009 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x5594f52953f0 con 0x5594f561e220 2024-04-24T09:06:58.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.009+0000 7f8fa4c00640 1 -- 172.21.15.83:0/3255280009 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8fb4004050 con 0x5594f561e220 2024-04-24T09:06:58.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.009+0000 7f8fa4c00640 1 -- 172.21.15.83:0/3255280009 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 49) ==== 8406+0+0 (secure 0 0 0) 0x7f8fa802c5c0 con 0x5594f561e220 2024-04-24T09:06:58.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.009+0000 7f8fc4b72740 10 client.21006 fetch_fsmap finished waiting for FSMap version 49 2024-04-24T09:06:58.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.009+0000 7f8fc4b72740 10 client.21006 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-24T09:06:58.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.010+0000 7f8fc4b72740 20 client.21006 populate_metadata read hostname 'smithi083' 2024-04-24T09:06:58.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.010+0000 7f8fc4b72740 1 --2- 172.21.15.83:0/3255280009 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x5594f5621920 0x5594f5623d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:58.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.010+0000 7f8fc4b72740 4 client.21006 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:06:58.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.010+0000 7f8fc4b72740 1 -- 172.21.15.83:0/3255280009 --> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5594f5590bf0 con 0x5594f5621920 2024-04-24T09:06:58.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.010+0000 7f8fbb400640 1 --2- 172.21.15.83:0/3255280009 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x5594f5621920 0x5594f5623d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:58.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.011+0000 7f8fbb400640 1 --2- 172.21.15.83:0/3255280009 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x5594f5621920 0x5594f5623d60 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=acd62ead8fcf0521 server_cookie=d812a6c780bfba4b in_seq=0 out_seq=0 2024-04-24T09:06:58.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.011+0000 7f8fa4c00640 10 client.21006 ms_handle_connect on v2:172.21.15.138:6834/1909219758 2024-04-24T09:06:58.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.011+0000 7f8fa4c00640 1 -- 172.21.15.83:0/3255280009 <== mds.2 v2:172.21.15.138:6834/1909219758 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5594f5590bf0 con 0x5594f5621920 2024-04-24T09:06:58.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.011+0000 7f8fa4c00640 10 client.21006 handle_command_reply: tid=0 2024-04-24T09:06:58.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.012+0000 7f8fc4b72740 1 client.21006 shutdown 2024-04-24T09:06:58.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.012+0000 7f8fc4b72740 1 -- 172.21.15.83:0/3255280009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594f561e220 msgr2=0x5594f562e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:58.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.012+0000 7f8fc4b72740 1 --2- 172.21.15.83:0/3255280009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594f561e220 0x5594f562e670 secure :-1 s=READY pgs=2217 cs=0 l=1 rev1=1 crypto rx=0x7f8fa800bdd0 tx=0x7f8fa80117b0 comp rx=0 tx=0).stop 2024-04-24T09:06:58.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.012+0000 7f8fc4b72740 1 -- 172.21.15.83:0/3255280009 shutdown_connections 2024-04-24T09:06:58.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.012+0000 7f8fc4b72740 1 --2- 172.21.15.83:0/3255280009 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x5594f5621920 0x5594f5623d60 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:58.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.012+0000 7f8fc4b72740 1 --2- 172.21.15.83:0/3255280009 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5594f5631340 0x5594f562ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:58.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.012+0000 7f8fc4b72740 1 --2- 172.21.15.83:0/3255280009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594f561e220 0x5594f562e670 unknown :-1 s=CLOSED pgs=2217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:58.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.012+0000 7f8fc4b72740 1 --2- 172.21.15.83:0/3255280009 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5594f561d870 0x5594f562e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:58.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.012+0000 7f8fc4b72740 1 -- 172.21.15.83:0/3255280009 >> 172.21.15.83:0/3255280009 conn(0x5594f5619e60 msgr2=0x5594f561fcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:58.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.013+0000 7f8fa4c00640 0 client.21006 ms_handle_reset on v2:172.21.15.138:6834/1909219758 2024-04-24T09:06:58.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.013+0000 7f8fc4b72740 1 -- 172.21.15.83:0/3255280009 shutdown_connections 2024-04-24T09:06:58.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.013+0000 7f8fc4b72740 1 -- 172.21.15.83:0/3255280009 wait complete. 2024-04-24T09:06:58.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.013+0000 7f8fc4b72740 20 client.21006 trim_cache size 0 max 16384 2024-04-24T09:06:58.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.034+0000 7f8fc4b72740 1 Processor -- start 2024-04-24T09:06:58.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.034+0000 7f8fc4b72740 1 -- start start 2024-04-24T09:06:58.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.035+0000 7f8fc4b72740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5594f561d870 0x5594f562b390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:58.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.035+0000 7f8fc4b72740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594f561e220 0x5594f562b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:58.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.035+0000 7f8fc4b72740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5594f5631340 0x5594f562c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:58.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.035+0000 7f8fc4b72740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8fbc1ad280 con 0x5594f561e220 2024-04-24T09:06:58.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.035+0000 7f8fc4b72740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8fbc1aef40 con 0x5594f5631340 2024-04-24T09:06:58.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.035+0000 7f8fc4b72740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8fbc1ad950 con 0x5594f561d870 2024-04-24T09:06:58.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.035+0000 7f8fbb400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5594f5631340 0x5594f562c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:58.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.035+0000 7f8fba000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594f561e220 0x5594f562b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:58.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.035+0000 7f8fbb400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5594f5631340 0x5594f562c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:56926/0 (socket says 172.21.15.83:56926) 2024-04-24T09:06:58.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.035+0000 7f8fbb400640 1 -- 172.21.15.83:0/2175560664 learned_addr learned my addr 172.21.15.83:0/2175560664 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:06:58.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.036+0000 7f8fbb400640 1 --2- 172.21.15.83:0/2175560664 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5594f561d870 0x5594f562b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:58.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.036+0000 7f8fba000640 1 -- 172.21.15.83:0/2175560664 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5594f561d870 msgr2=0x5594f562b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:58.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.036+0000 7f8fba000640 1 --2- 172.21.15.83:0/2175560664 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5594f561d870 0x5594f562b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:58.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.036+0000 7f8fba000640 1 -- 172.21.15.83:0/2175560664 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5594f5631340 msgr2=0x5594f562c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:58.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.036+0000 7f8fba000640 1 --2- 172.21.15.83:0/2175560664 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5594f5631340 0x5594f562c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:58.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.036+0000 7f8fba000640 1 -- 172.21.15.83:0/2175560664 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594f535f7d0 con 0x5594f561e220 2024-04-24T09:06:58.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.037+0000 7f8fba000640 1 --2- 172.21.15.83:0/2175560664 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594f561e220 0x5594f562b8d0 secure :-1 s=READY pgs=2218 cs=0 l=1 rev1=1 crypto rx=0x7f8fa8000c00 tx=0x7f8fa800b9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11bb74108c94521c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:58.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.037+0000 7f8fa7400640 1 -- 172.21.15.83:0/2175560664 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa803d070 con 0x5594f561e220 2024-04-24T09:06:58.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.037+0000 7f8fa7400640 1 -- 172.21.15.83:0/2175560664 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8fa802ce00 con 0x5594f561e220 2024-04-24T09:06:58.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.037+0000 7f8fa7400640 1 -- 172.21.15.83:0/2175560664 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa8032040 con 0x5594f561e220 2024-04-24T09:06:58.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.038+0000 7f8fc4b72740 1 -- 172.21.15.83:0/2175560664 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594f561e220 msgr2=0x5594f562b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:58.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.038+0000 7f8fc4b72740 1 --2- 172.21.15.83:0/2175560664 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594f561e220 0x5594f562b8d0 secure :-1 s=READY pgs=2218 cs=0 l=1 rev1=1 crypto rx=0x7f8fa8000c00 tx=0x7f8fa800b9c0 comp rx=0 tx=0).stop 2024-04-24T09:06:58.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.038+0000 7f8fc4b72740 1 -- 172.21.15.83:0/2175560664 shutdown_connections 2024-04-24T09:06:58.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.038+0000 7f8fc4b72740 1 --2- 172.21.15.83:0/2175560664 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5594f5631340 0x5594f562c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:58.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.038+0000 7f8fc4b72740 1 --2- 172.21.15.83:0/2175560664 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594f561e220 0x5594f562b8d0 unknown :-1 s=CLOSED pgs=2218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:58.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.038+0000 7f8fc4b72740 1 --2- 172.21.15.83:0/2175560664 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5594f561d870 0x5594f562b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:58.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.038+0000 7f8fc4b72740 1 -- 172.21.15.83:0/2175560664 >> 172.21.15.83:0/2175560664 conn(0x5594f5619e60 msgr2=0x5594f561fcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:58.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.039+0000 7f8fc4b72740 1 -- 172.21.15.83:0/2175560664 shutdown_connections 2024-04-24T09:06:58.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.039+0000 7f8fc4b72740 1 -- 172.21.15.83:0/2175560664 wait complete. 2024-04-24T09:06:58.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.039+0000 7f8fc4b72740 1 -- start start 2024-04-24T09:06:58.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.040+0000 7f8fc4b72740 1 Processor -- start 2024-04-24T09:06:58.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.040+0000 7f8fc4b72740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5594f561d870 0x5594f5640180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:58.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.040+0000 7f8fc4b72740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5594f561e220 0x5594f56406c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:58.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.040+0000 7f8fc4b72740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594f5631340 0x5594f569db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:58.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.040+0000 7f8fc4b72740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8fbc1ae7b0 con 0x5594f5631340 2024-04-24T09:06:58.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.040+0000 7f8fc4b72740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8fbc1ae080 con 0x5594f561d870 2024-04-24T09:06:58.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.040+0000 7f8fc4b72740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8fbc1ab280 con 0x5594f561e220 2024-04-24T09:06:58.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.040+0000 7f8fc4b72740 10 client.0 _ll_register_callbacks cb 0x7f8fa8005f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:06:58.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.040+0000 7f8fbb400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5594f561d870 0x5594f5640180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:58.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.040+0000 7f8fbb400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5594f561d870 0x5594f5640180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:56942/0 (socket says 172.21.15.83:56942) 2024-04-24T09:06:58.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.040+0000 7f8fbb400640 1 -- 172.21.15.83:0/354124094 learned_addr learned my addr 172.21.15.83:0/354124094 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:06:58.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.040+0000 7f8fbb400640 1 --2- 172.21.15.83:0/354124094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594f5631340 0x5594f569db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:58.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.041+0000 7f8fba000640 1 --2- 172.21.15.83:0/354124094 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5594f561e220 0x5594f56406c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:58.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.041+0000 7f8fbb400640 1 -- 172.21.15.83:0/354124094 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5594f561e220 msgr2=0x5594f56406c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:58.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.041+0000 7f8fbb400640 1 --2- 172.21.15.83:0/354124094 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5594f561e220 0x5594f56406c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:58.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.041+0000 7f8fbb400640 1 -- 172.21.15.83:0/354124094 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5594f561d870 msgr2=0x5594f5640180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:58.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.041+0000 7f8fbb400640 1 --2- 172.21.15.83:0/354124094 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5594f561d870 0x5594f5640180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:58.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.041+0000 7f8fbb400640 1 -- 172.21.15.83:0/354124094 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594f5640f80 con 0x5594f5631340 2024-04-24T09:06:58.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.041+0000 7f8fbb400640 1 --2- 172.21.15.83:0/354124094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594f5631340 0x5594f569db50 secure :-1 s=READY pgs=2219 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4005cc0 tx=0x7f8fa800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d422c8ca9ed1141f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:58.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.041+0000 7f8fa4c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:06:58.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.042+0000 7f8fa4c00640 1 -- 172.21.15.83:0/354124094 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb4005230 con 0x5594f5631340 2024-04-24T09:06:58.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.042+0000 7f8fc4b72740 1 -- 172.21.15.83:0/354124094 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8fa802ce00 con 0x5594f5631340 2024-04-24T09:06:58.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.042+0000 7f8fa4c00640 1 -- 172.21.15.83:0/354124094 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9583+0+0 (secure 0 0 0) 0x7f8fb40053d0 con 0x5594f5631340 2024-04-24T09:06:58.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.042+0000 7f8fa4c00640 1 -- 172.21.15.83:0/354124094 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f8fb403c050 con 0x5594f5631340 2024-04-24T09:06:58.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.042+0000 7f8fc4b72740 10 client.21012 fetch_fsmap learned FSMap version 49 2024-04-24T09:06:58.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.042+0000 7f8fc4b72740 1 -- 172.21.15.83:0/354124094 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x5594f569f1c0 con 0x5594f5631340 2024-04-24T09:06:58.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.043+0000 7f8fa4c00640 1 -- 172.21.15.83:0/354124094 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8fa802ce00 con 0x5594f5631340 2024-04-24T09:06:58.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.044+0000 7f8fa4c00640 1 -- 172.21.15.83:0/354124094 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 49) ==== 8406+0+0 (secure 0 0 0) 0x7f8fb402a0d0 con 0x5594f5631340 2024-04-24T09:06:58.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.044+0000 7f8fc4b72740 10 client.21012 fetch_fsmap finished waiting for FSMap version 49 2024-04-24T09:06:58.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.044+0000 7f8fc4b72740 10 client.21012 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-24T09:06:58.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.044+0000 7f8fc4b72740 20 client.21012 populate_metadata read hostname 'smithi083' 2024-04-24T09:06:58.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.044+0000 7f8fc4b72740 1 --2- 172.21.15.83:0/354124094 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x5594f569f760 0x5594f56a1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:58.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.044+0000 7f8fc4b72740 4 client.21012 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:06:58.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.044+0000 7f8fc4b72740 1 -- 172.21.15.83:0/354124094 --> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5594f558aa60 con 0x5594f569f760 2024-04-24T09:06:58.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.045+0000 7f8fba000640 1 --2- 172.21.15.83:0/354124094 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x5594f569f760 0x5594f56a1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:58.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.045+0000 7f8fba000640 1 --2- 172.21.15.83:0/354124094 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x5594f569f760 0x5594f56a1b80 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1f6dd766cafa5aaa server_cookie=ff6aed318c4cfe96 in_seq=0 out_seq=0 2024-04-24T09:06:58.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.045+0000 7f8fa4c00640 10 client.21012 ms_handle_connect on v2:172.21.15.138:6834/1909219758 2024-04-24T09:06:58.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.046+0000 7f8fa4c00640 1 -- 172.21.15.83:0/354124094 <== mds.2 v2:172.21.15.138:6834/1909219758 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5594f558aa60 con 0x5594f569f760 2024-04-24T09:06:58.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.046+0000 7f8fa4c00640 10 client.21012 handle_command_reply: tid=0 2024-04-24T09:06:58.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.046+0000 7f8fc4b72740 1 client.21012 shutdown 2024-04-24T09:06:58.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.046+0000 7f8fc4b72740 1 -- 172.21.15.83:0/354124094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594f5631340 msgr2=0x5594f569db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:58.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.046+0000 7f8fc4b72740 1 --2- 172.21.15.83:0/354124094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594f5631340 0x5594f569db50 secure :-1 s=READY pgs=2219 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4005cc0 tx=0x7f8fa800b8d0 comp rx=0 tx=0).stop 2024-04-24T09:06:58.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.046+0000 7f8fc4b72740 1 -- 172.21.15.83:0/354124094 shutdown_connections 2024-04-24T09:06:58.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.046+0000 7f8fc4b72740 1 --2- 172.21.15.83:0/354124094 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x5594f569f760 0x5594f56a1b80 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:58.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.046+0000 7f8fc4b72740 1 --2- 172.21.15.83:0/354124094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5594f5631340 0x5594f569db50 unknown :-1 s=CLOSED pgs=2219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:58.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.046+0000 7f8fc4b72740 1 --2- 172.21.15.83:0/354124094 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5594f561e220 0x5594f56406c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:58.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.046+0000 7f8fc4b72740 1 --2- 172.21.15.83:0/354124094 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5594f561d870 0x5594f5640180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:58.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.046+0000 7f8fc4b72740 1 -- 172.21.15.83:0/354124094 >> 172.21.15.83:0/354124094 conn(0x5594f5619e60 msgr2=0x5594f562c8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:58.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.046+0000 7f8fa4c00640 0 client.21012 ms_handle_reset on v2:172.21.15.138:6834/1909219758 2024-04-24T09:06:58.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.046+0000 7f8fc4b72740 1 -- 172.21.15.83:0/354124094 shutdown_connections 2024-04-24T09:06:58.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.047+0000 7f8fc4b72740 1 -- 172.21.15.83:0/354124094 wait complete. 2024-04-24T09:06:58.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.047+0000 7f8fc4b72740 20 client.21012 trim_cache size 0 max 16384 2024-04-24T09:06:58.049 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:06:58.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.047+0000 7f8fbbe00640 1 -- 172.21.15.83:0/470697867 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f8fa00a3da0 msgr2=0x7f8fa00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:58.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.047+0000 7f8fbbe00640 1 --2- 172.21.15.83:0/470697867 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f8fa00a3da0 0x7f8fa00a6260 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f8fac0bd770 tx=0x7f8fac03d4f0 comp rx=0 tx=0).stop 2024-04-24T09:06:58.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.047+0000 7f8fbbe00640 1 -- 172.21.15.83:0/470697867 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8fbc102110 msgr2=0x7f8fbc1987a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:58.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.047+0000 7f8fbbe00640 1 --2- 172.21.15.83:0/470697867 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8fbc102110 0x7f8fbc1987a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f8fac005940 tx=0x7f8fac005e50 comp rx=0 tx=0).stop 2024-04-24T09:06:58.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.048+0000 7f8fbbe00640 1 -- 172.21.15.83:0/470697867 shutdown_connections 2024-04-24T09:06:58.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.048+0000 7f8fbbe00640 1 --2- 172.21.15.83:0/470697867 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f8fa00a3da0 0x7f8fa00a6260 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:58.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.048+0000 7f8fbbe00640 1 --2- 172.21.15.83:0/470697867 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8fbc19d010 0x7f8fbc19d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:58.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.048+0000 7f8fbbe00640 1 --2- 172.21.15.83:0/470697867 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8fbc1071e0 0x7f8fbc198ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:58.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.048+0000 7f8fbbe00640 1 --2- 172.21.15.83:0/470697867 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8fbc102110 0x7f8fbc1987a0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:58.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.048+0000 7f8fbbe00640 1 -- 172.21.15.83:0/470697867 >> 172.21.15.83:0/470697867 conn(0x7f8fbc0fd9d0 msgr2=0x7f8fbc0fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:58.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.048+0000 7f8fbbe00640 1 -- 172.21.15.83:0/470697867 shutdown_connections 2024-04-24T09:06:58.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.048+0000 7f8fbbe00640 1 -- 172.21.15.83:0/470697867 wait complete. 2024-04-24T09:06:58.070 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T09:06:58.159 INFO:tasks.workunit.client.0.smithi083.stdout: 10 11614 9.00 MB/sec warmup 89 sec latency 95.579 ms 2024-04-24T09:06:58.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.317+0000 7f2c9e000640 1 -- 172.21.15.83:0/469408862 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c981019b0 msgr2=0x7f2c981059f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:58.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.317+0000 7f2c9e000640 1 --2- 172.21.15.83:0/469408862 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c981019b0 0x7f2c981059f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f2c88009900 tx=0x7f2c8802a410 comp rx=0 tx=0).stop 2024-04-24T09:06:58.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.317+0000 7f2c9e000640 1 -- 172.21.15.83:0/469408862 shutdown_connections 2024-04-24T09:06:58.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.317+0000 7f2c9e000640 1 --2- 172.21.15.83:0/469408862 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c981060d0 0x7f2c9810dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:58.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.317+0000 7f2c9e000640 1 --2- 172.21.15.83:0/469408862 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c981019b0 0x7f2c981059f0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:58.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.317+0000 7f2c9e000640 1 --2- 172.21.15.83:0/469408862 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2c98101000 0x7f2c981013e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:58.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.317+0000 7f2c9e000640 1 -- 172.21.15.83:0/469408862 >> 172.21.15.83:0/469408862 conn(0x7f2c980fc880 msgr2=0x7f2c980feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:58.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.317+0000 7f2c9e000640 1 -- 172.21.15.83:0/469408862 shutdown_connections 2024-04-24T09:06:58.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.317+0000 7f2c9e000640 1 -- 172.21.15.83:0/469408862 wait complete. 2024-04-24T09:06:58.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.319+0000 7f2c9e000640 1 Processor -- start 2024-04-24T09:06:58.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.319+0000 7f2c9e000640 1 -- start start 2024-04-24T09:06:58.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.320+0000 7f2c9e000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c98101000 0x7f2c98196500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:58.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.320+0000 7f2c9e000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c981060d0 0x7f2c98196a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:58.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.320+0000 7f2c9e000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2c9819adc0 0x7f2c9819b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:58.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.320+0000 7f2c9e000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2c9810ff80 con 0x7f2c981060d0 2024-04-24T09:06:58.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.320+0000 7f2c9e000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2c9810fe00 con 0x7f2c98101000 2024-04-24T09:06:58.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.320+0000 7f2c9e000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2c98110100 con 0x7f2c9819adc0 2024-04-24T09:06:58.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.321+0000 7f2c9cc00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c98101000 0x7f2c98196500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:58.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.321+0000 7f2c9cc00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c98101000 0x7f2c98196500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:56972/0 (socket says 172.21.15.83:56972) 2024-04-24T09:06:58.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.321+0000 7f2c9cc00640 1 -- 172.21.15.83:0/3414165503 learned_addr learned my addr 172.21.15.83:0/3414165503 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:06:58.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.322+0000 7f2c97e00640 1 --2- 172.21.15.83:0/3414165503 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c981060d0 0x7f2c98196a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:58.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.322+0000 7f2c9cc00640 1 -- 172.21.15.83:0/3414165503 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2c9819adc0 msgr2=0x7f2c9819b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:58.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.322+0000 7f2c9cc00640 1 --2- 172.21.15.83:0/3414165503 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2c9819adc0 0x7f2c9819b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:58.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.322+0000 7f2c9cc00640 1 -- 172.21.15.83:0/3414165503 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c981060d0 msgr2=0x7f2c98196a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:58.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.322+0000 7f2c9cc00640 1 --2- 172.21.15.83:0/3414165503 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c981060d0 0x7f2c98196a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:58.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.322+0000 7f2c9cc00640 1 -- 172.21.15.83:0/3414165503 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c9819b8b0 con 0x7f2c98101000 2024-04-24T09:06:58.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.322+0000 7f2c97e00640 1 --2- 172.21.15.83:0/3414165503 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c981060d0 0x7f2c98196a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:06:58.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.324+0000 7f2c9cc00640 1 --2- 172.21.15.83:0/3414165503 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c98101000 0x7f2c98196500 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c00d950 tx=0x7f2c8c00de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:58.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.325+0000 7f2c95600640 1 -- 172.21.15.83:0/3414165503 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c8c017070 con 0x7f2c98101000 2024-04-24T09:06:58.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.325+0000 7f2c95600640 1 -- 172.21.15.83:0/3414165503 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2c8c009c30 con 0x7f2c98101000 2024-04-24T09:06:58.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.325+0000 7f2c95600640 1 -- 172.21.15.83:0/3414165503 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c8c00f620 con 0x7f2c98101000 2024-04-24T09:06:58.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.325+0000 7f2c9e000640 1 -- 172.21.15.83:0/3414165503 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c9819bba0 con 0x7f2c98101000 2024-04-24T09:06:58.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.326+0000 7f2c9e000640 1 -- 172.21.15.83:0/3414165503 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c981abb50 con 0x7f2c98101000 2024-04-24T09:06:58.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.333+0000 7f2c95600640 1 -- 172.21.15.83:0/3414165503 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2c8c010460 con 0x7f2c98101000 2024-04-24T09:06:58.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.334+0000 7f2ca2400740 1 Processor -- start 2024-04-24T09:06:58.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.334+0000 7f2ca2400740 1 -- start start 2024-04-24T09:06:58.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.336+0000 7f2c95600640 1 --2- 172.21.15.83:0/3414165503 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f2c800a3ee0 0x7f2c800a63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:58.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.336+0000 7f2c95600640 1 -- 172.21.15.83:0/3414165503 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9583+0+0 (secure 0 0 0) 0x7f2c8c0d0850 con 0x7f2c98101000 2024-04-24T09:06:58.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.336+0000 7f2c9d600640 1 --2- 172.21.15.83:0/3414165503 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f2c800a3ee0 0x7f2c800a63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:58.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.338+0000 7f2ca2400740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560935e47b40 0x560935e47f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:58.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.338+0000 7f2ca2400740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560935e48460 0x560935e525f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:58.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.338+0000 7f2ca2400740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560935e52b30 0x560935e54f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:58.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.338+0000 7f2ca2400740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x560935dad1d0 con 0x560935e48460 2024-04-24T09:06:58.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.338+0000 7f2ca2400740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560935d53450 con 0x560935e52b30 2024-04-24T09:06:58.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.338+0000 7f2ca2400740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x560935da9190 con 0x560935e47b40 2024-04-24T09:06:58.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.340+0000 7f2c9d600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560935e52b30 0x560935e54f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:58.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.340+0000 7f2c9d600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560935e52b30 0x560935e54f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:56988/0 (socket says 172.21.15.83:56988) 2024-04-24T09:06:58.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.340+0000 7f2c9d600640 1 -- 172.21.15.83:0/113091419 learned_addr learned my addr 172.21.15.83:0/113091419 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:06:58.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.340+0000 7f2c9d600640 1 --2- 172.21.15.83:0/3414165503 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f2c800a3ee0 0x7f2c800a63a0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f2c90009370 tx=0x7f2c90009240 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:58.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.340+0000 7f2c9cc00640 1 --2- 172.21.15.83:0/113091419 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560935e47b40 0x560935e47f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:58.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.341+0000 7f2c9d600640 1 -- 172.21.15.83:0/113091419 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560935e47b40 msgr2=0x560935e47f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:58.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.341+0000 7f2c9d600640 1 --2- 172.21.15.83:0/113091419 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560935e47b40 0x560935e47f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:58.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.341+0000 7f2c9d600640 1 -- 172.21.15.83:0/113091419 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560935e48460 msgr2=0x560935e525f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T09:06:58.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.343+0000 7f2c97e00640 1 --2- 172.21.15.83:0/113091419 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560935e48460 0x560935e525f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:58.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.343+0000 7f2c9d600640 1 --2- 172.21.15.83:0/113091419 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560935e48460 0x560935e525f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:58.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.343+0000 7f2c9d600640 1 -- 172.21.15.83:0/113091419 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560935e10310 con 0x560935e52b30 2024-04-24T09:06:58.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.343+0000 7f2c97e00640 1 --2- 172.21.15.83:0/113091419 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560935e48460 0x560935e525f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:06:58.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.344+0000 7f2c9d600640 1 --2- 172.21.15.83:0/113091419 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560935e52b30 0x560935e54f20 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f2c90009fb0 tx=0x7f2c90015000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc16ca2aa80d919e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:58.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.345+0000 7f2c85c00640 1 -- 172.21.15.83:0/113091419 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c90016070 con 0x560935e52b30 2024-04-24T09:06:58.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.345+0000 7f2c85c00640 1 -- 172.21.15.83:0/113091419 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2c90003e20 con 0x560935e52b30 2024-04-24T09:06:58.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.345+0000 7f2c85c00640 1 -- 172.21.15.83:0/113091419 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c90005020 con 0x560935e52b30 2024-04-24T09:06:58.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.346+0000 7f2ca2400740 1 -- 172.21.15.83:0/113091419 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560935e52b30 msgr2=0x560935e54f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:58.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.346+0000 7f2ca2400740 1 --2- 172.21.15.83:0/113091419 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560935e52b30 0x560935e54f20 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f2c90009fb0 tx=0x7f2c90015000 comp rx=0 tx=0).stop 2024-04-24T09:06:58.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.346+0000 7f2ca2400740 1 -- 172.21.15.83:0/113091419 shutdown_connections 2024-04-24T09:06:58.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.346+0000 7f2ca2400740 1 --2- 172.21.15.83:0/113091419 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560935e52b30 0x560935e54f20 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:58.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.346+0000 7f2ca2400740 1 --2- 172.21.15.83:0/113091419 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560935e48460 0x560935e525f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:58.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.346+0000 7f2ca2400740 1 --2- 172.21.15.83:0/113091419 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560935e47b40 0x560935e47f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:58.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.346+0000 7f2ca2400740 1 -- 172.21.15.83:0/113091419 >> 172.21.15.83:0/113091419 conn(0x560935e43e60 msgr2=0x560935e44240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:58.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.347+0000 7f2ca2400740 1 -- 172.21.15.83:0/113091419 shutdown_connections 2024-04-24T09:06:58.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.347+0000 7f2ca2400740 1 -- 172.21.15.83:0/113091419 wait complete. 2024-04-24T09:06:58.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.348+0000 7f2ca2400740 1 -- start start 2024-04-24T09:06:58.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.348+0000 7f2ca2400740 1 Processor -- start 2024-04-24T09:06:58.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.350+0000 7f2ca2400740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560935e47b40 0x560935e4f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:58.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.350+0000 7f2ca2400740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560935e48460 0x560935e4fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:58.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.350+0000 7f2ca2400740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560935e52b30 0x560935e503a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:58.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.350+0000 7f2ca2400740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x560935e559c0 con 0x560935e48460 2024-04-24T09:06:58.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.350+0000 7f2ca2400740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560935d66810 con 0x560935e47b40 2024-04-24T09:06:58.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.350+0000 7f2ca2400740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x560935e55b40 con 0x560935e52b30 2024-04-24T09:06:58.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.351+0000 7f2ca2400740 10 client.0 _ll_register_callbacks cb 0x560935d34140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:06:58.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.351+0000 7f2c9d600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560935e52b30 0x560935e503a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:58.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.351+0000 7f2c9d600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560935e52b30 0x560935e503a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:49722/0 (socket says 172.21.15.83:49722) 2024-04-24T09:06:58.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.351+0000 7f2c97e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560935e48460 0x560935e4fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:58.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.351+0000 7f2c9d600640 1 -- 172.21.15.83:0/575781852 learned_addr learned my addr 172.21.15.83:0/575781852 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:06:58.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.351+0000 7f2c9cc00640 1 --2- 172.21.15.83:0/575781852 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560935e47b40 0x560935e4f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:58.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.353+0000 7f2c97e00640 1 -- 172.21.15.83:0/575781852 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560935e52b30 msgr2=0x560935e503a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:58.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.353+0000 7f2c97e00640 1 --2- 172.21.15.83:0/575781852 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560935e52b30 0x560935e503a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:58.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.353+0000 7f2c97e00640 1 -- 172.21.15.83:0/575781852 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560935e47b40 msgr2=0x560935e4f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:58.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.353+0000 7f2c97e00640 1 --2- 172.21.15.83:0/575781852 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560935e47b40 0x560935e4f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:58.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.353+0000 7f2c97e00640 1 -- 172.21.15.83:0/575781852 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560935d773f0 con 0x560935e48460 2024-04-24T09:06:58.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.354+0000 7f2c9d600640 1 --2- 172.21.15.83:0/575781852 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560935e52b30 0x560935e503a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:06:58.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.354+0000 7f2c9cc00640 1 --2- 172.21.15.83:0/575781852 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560935e47b40 0x560935e4f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:06:58.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.355+0000 7f2c97e00640 1 --2- 172.21.15.83:0/575781852 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560935e48460 0x560935e4fe60 secure :-1 s=READY pgs=2220 cs=0 l=1 rev1=1 crypto rx=0x560935e49100 tx=0x7f2c88005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4ff5ab43d3e00a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:58.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.355+0000 7f2c7a000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:06:58.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.356+0000 7f2c7a000640 1 -- 172.21.15.83:0/575781852 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c88040070 con 0x560935e48460 2024-04-24T09:06:58.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.357+0000 7f2c7a000640 1 -- 172.21.15.83:0/575781852 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9583+0+0 (secure 0 0 0) 0x7f2c880046c0 con 0x560935e48460 2024-04-24T09:06:58.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.357+0000 7f2ca2400740 1 -- 172.21.15.83:0/575781852 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c90003e20 con 0x560935e48460 2024-04-24T09:06:58.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.359+0000 7f2c7a000640 1 -- 172.21.15.83:0/575781852 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f2c8804a050 con 0x560935e48460 2024-04-24T09:06:58.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.359+0000 7f2ca2400740 10 client.21018 fetch_fsmap learned FSMap version 49 2024-04-24T09:06:58.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.359+0000 7f2ca2400740 1 -- 172.21.15.83:0/575781852 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x560935abf3f0 con 0x560935e48460 2024-04-24T09:06:58.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.360+0000 7f2c7a000640 1 -- 172.21.15.83:0/575781852 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2c90003e20 con 0x560935e48460 2024-04-24T09:06:58.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.361+0000 7f2c7a000640 1 -- 172.21.15.83:0/575781852 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 49) ==== 8406+0+0 (secure 0 0 0) 0x7f2c8803a040 con 0x560935e48460 2024-04-24T09:06:58.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.362+0000 7f2ca2400740 10 client.21018 fetch_fsmap finished waiting for FSMap version 49 2024-04-24T09:06:58.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.362+0000 7f2ca2400740 10 client.21018 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-24T09:06:58.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.362+0000 7f2ca2400740 20 client.21018 populate_metadata read hostname 'smithi083' 2024-04-24T09:06:58.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.362+0000 7f2ca2400740 1 --2- 172.21.15.83:0/575781852 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x560935e62d20 0x560935e65170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:58.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.362+0000 7f2ca2400740 4 client.21018 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:06:58.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.362+0000 7f2ca2400740 1 -- 172.21.15.83:0/575781852 --> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560935dbabf0 con 0x560935e62d20 2024-04-24T09:06:58.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.362+0000 7f2c9cc00640 1 --2- 172.21.15.83:0/575781852 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x560935e62d20 0x560935e65170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:58.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.363+0000 7f2c7a000640 10 client.21018 ms_handle_connect on v2:172.21.15.138:6836/900566754 2024-04-24T09:06:58.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.363+0000 7f2c9cc00640 1 --2- 172.21.15.83:0/575781852 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x560935e62d20 0x560935e65170 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c83eae7f756d0d3f server_cookie=d538f071c5834228 in_seq=0 out_seq=0 2024-04-24T09:06:58.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.364+0000 7f2c7a000640 1 -- 172.21.15.83:0/575781852 <== mds.1 v2:172.21.15.138:6836/900566754 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560935dbabf0 con 0x560935e62d20 2024-04-24T09:06:58.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.364+0000 7f2c7a000640 10 client.21018 handle_command_reply: tid=0 2024-04-24T09:06:58.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.364+0000 7f2ca2400740 1 client.21018 shutdown 2024-04-24T09:06:58.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.364+0000 7f2ca2400740 1 -- 172.21.15.83:0/575781852 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560935e48460 msgr2=0x560935e4fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:58.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.364+0000 7f2ca2400740 1 --2- 172.21.15.83:0/575781852 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560935e48460 0x560935e4fe60 secure :-1 s=READY pgs=2220 cs=0 l=1 rev1=1 crypto rx=0x560935e49100 tx=0x7f2c88005270 comp rx=0 tx=0).stop 2024-04-24T09:06:58.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.365+0000 7f2ca2400740 1 -- 172.21.15.83:0/575781852 shutdown_connections 2024-04-24T09:06:58.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.365+0000 7f2ca2400740 1 --2- 172.21.15.83:0/575781852 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x560935e62d20 0x560935e65170 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:58.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.365+0000 7f2ca2400740 1 --2- 172.21.15.83:0/575781852 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560935e52b30 0x560935e503a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:58.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.365+0000 7f2c7a000640 0 client.21018 ms_handle_reset on v2:172.21.15.138:6836/900566754 2024-04-24T09:06:58.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.365+0000 7f2ca2400740 1 --2- 172.21.15.83:0/575781852 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560935e48460 0x560935e4fe60 unknown :-1 s=CLOSED pgs=2220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:58.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.365+0000 7f2ca2400740 1 --2- 172.21.15.83:0/575781852 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560935e47b40 0x560935e4f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:58.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.365+0000 7f2ca2400740 1 -- 172.21.15.83:0/575781852 >> 172.21.15.83:0/575781852 conn(0x560935e43e60 msgr2=0x560935e49c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:58.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.365+0000 7f2ca2400740 1 -- 172.21.15.83:0/575781852 shutdown_connections 2024-04-24T09:06:58.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.366+0000 7f2ca2400740 1 -- 172.21.15.83:0/575781852 wait complete. 2024-04-24T09:06:58.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.366+0000 7f2ca2400740 20 client.21018 trim_cache size 0 max 16384 2024-04-24T09:06:58.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.393+0000 7f2ca2400740 1 Processor -- start 2024-04-24T09:06:58.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.393+0000 7f2ca2400740 1 -- start start 2024-04-24T09:06:58.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.393+0000 7f2ca2400740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560935e47b40 0x560935e5e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:58.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.393+0000 7f2ca2400740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560935e48460 0x560935e4eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:58.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.393+0000 7f2ca2400740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560935e52b30 0x560935e42dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:58.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.393+0000 7f2ca2400740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2c981b5ab0 con 0x560935e52b30 2024-04-24T09:06:58.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.393+0000 7f2ca2400740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2c981b7770 con 0x560935e47b40 2024-04-24T09:06:58.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.393+0000 7f2ca2400740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2c981b6180 con 0x560935e48460 2024-04-24T09:06:58.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.394+0000 7f2c9cc00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560935e47b40 0x560935e5e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:58.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.394+0000 7f2c9d600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560935e52b30 0x560935e42dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:58.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.394+0000 7f2c9d600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560935e52b30 0x560935e42dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51278/0 (socket says 172.21.15.83:51278) 2024-04-24T09:06:58.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.394+0000 7f2c9d600640 1 -- 172.21.15.83:0/1675380613 learned_addr learned my addr 172.21.15.83:0/1675380613 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:06:58.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.394+0000 7f2c97e00640 1 --2- 172.21.15.83:0/1675380613 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560935e48460 0x560935e4eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:58.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.395+0000 7f2c9d600640 1 -- 172.21.15.83:0/1675380613 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560935e48460 msgr2=0x560935e4eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:58.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.395+0000 7f2c9d600640 1 --2- 172.21.15.83:0/1675380613 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560935e48460 0x560935e4eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:58.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.395+0000 7f2c9d600640 1 -- 172.21.15.83:0/1675380613 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560935e47b40 msgr2=0x560935e5e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:58.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.395+0000 7f2c9d600640 1 --2- 172.21.15.83:0/1675380613 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560935e47b40 0x560935e5e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:58.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.395+0000 7f2c9d600640 1 -- 172.21.15.83:0/1675380613 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560935e48290 con 0x560935e52b30 2024-04-24T09:06:58.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.395+0000 7f2c9d600640 1 --2- 172.21.15.83:0/1675380613 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560935e52b30 0x560935e42dd0 secure :-1 s=READY pgs=2221 cs=0 l=1 rev1=1 crypto rx=0x7f2c9000e010 tx=0x7f2c90012040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3cf3160202bbf17 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:58.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.395+0000 7f2c85200640 1 -- 172.21.15.83:0/1675380613 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c90004550 con 0x560935e52b30 2024-04-24T09:06:58.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.395+0000 7f2c85200640 1 -- 172.21.15.83:0/1675380613 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2c900046f0 con 0x560935e52b30 2024-04-24T09:06:58.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.395+0000 7f2c85200640 1 -- 172.21.15.83:0/1675380613 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c90005280 con 0x560935e52b30 2024-04-24T09:06:58.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.396+0000 7f2ca2400740 1 -- 172.21.15.83:0/1675380613 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560935e52b30 msgr2=0x560935e42dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:58.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.396+0000 7f2ca2400740 1 --2- 172.21.15.83:0/1675380613 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560935e52b30 0x560935e42dd0 secure :-1 s=READY pgs=2221 cs=0 l=1 rev1=1 crypto rx=0x7f2c9000e010 tx=0x7f2c90012040 comp rx=0 tx=0).stop 2024-04-24T09:06:58.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.396+0000 7f2ca2400740 1 -- 172.21.15.83:0/1675380613 shutdown_connections 2024-04-24T09:06:58.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.396+0000 7f2ca2400740 1 --2- 172.21.15.83:0/1675380613 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560935e52b30 0x560935e42dd0 unknown :-1 s=CLOSED pgs=2221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:58.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.396+0000 7f2ca2400740 1 --2- 172.21.15.83:0/1675380613 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560935e48460 0x560935e4eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:58.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.396+0000 7f2ca2400740 1 --2- 172.21.15.83:0/1675380613 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560935e47b40 0x560935e5e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:58.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.396+0000 7f2ca2400740 1 -- 172.21.15.83:0/1675380613 >> 172.21.15.83:0/1675380613 conn(0x560935e43e60 msgr2=0x560935e426c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:58.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.397+0000 7f2ca2400740 1 -- 172.21.15.83:0/1675380613 shutdown_connections 2024-04-24T09:06:58.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.397+0000 7f2ca2400740 1 -- 172.21.15.83:0/1675380613 wait complete. 2024-04-24T09:06:58.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.397+0000 7f2ca2400740 1 -- start start 2024-04-24T09:06:58.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.397+0000 7f2ca2400740 1 Processor -- start 2024-04-24T09:06:58.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.398+0000 7f2ca2400740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560935e47b40 0x560935e4bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:58.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.398+0000 7f2ca2400740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560935e48460 0x560935e4c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:58.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.398+0000 7f2ca2400740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560935e52b30 0x560935e59d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:58.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.398+0000 7f2ca2400740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2c981b6fe0 con 0x560935e52b30 2024-04-24T09:06:58.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.398+0000 7f2ca2400740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2c981b68b0 con 0x560935e48460 2024-04-24T09:06:58.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.398+0000 7f2ca2400740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2c981b3ab0 con 0x560935e47b40 2024-04-24T09:06:58.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.398+0000 7f2ca2400740 10 client.0 _ll_register_callbacks cb 0x7f2c88009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:06:58.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.398+0000 7f2c9d600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560935e52b30 0x560935e59d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:58.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.398+0000 7f2c9d600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560935e52b30 0x560935e59d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51290/0 (socket says 172.21.15.83:51290) 2024-04-24T09:06:58.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.398+0000 7f2c9d600640 1 -- 172.21.15.83:0/706788522 learned_addr learned my addr 172.21.15.83:0/706788522 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:06:58.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.399+0000 7f2c97e00640 1 --2- 172.21.15.83:0/706788522 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560935e48460 0x560935e4c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:58.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.399+0000 7f2c9cc00640 1 --2- 172.21.15.83:0/706788522 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560935e47b40 0x560935e4bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:58.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.399+0000 7f2c9d600640 1 -- 172.21.15.83:0/706788522 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560935e47b40 msgr2=0x560935e4bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:58.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.399+0000 7f2c9d600640 1 --2- 172.21.15.83:0/706788522 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560935e47b40 0x560935e4bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:58.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.399+0000 7f2c9d600640 1 -- 172.21.15.83:0/706788522 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560935e48460 msgr2=0x560935e4c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:58.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.399+0000 7f2c9d600640 1 --2- 172.21.15.83:0/706788522 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560935e48460 0x560935e4c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:58.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.399+0000 7f2c9d600640 1 -- 172.21.15.83:0/706788522 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560935b897d0 con 0x560935e52b30 2024-04-24T09:06:58.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.399+0000 7f2c97e00640 1 --2- 172.21.15.83:0/706788522 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560935e48460 0x560935e4c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:06:58.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.400+0000 7f2c9d600640 1 --2- 172.21.15.83:0/706788522 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560935e52b30 0x560935e59d90 secure :-1 s=READY pgs=2222 cs=0 l=1 rev1=1 crypto rx=0x7f2c90005c40 tx=0x7f2c90005e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0d0f7548afa7e7c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:58.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.400+0000 7f2c7a000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:06:58.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.400+0000 7f2c7a000640 1 -- 172.21.15.83:0/706788522 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c90035030 con 0x560935e52b30 2024-04-24T09:06:58.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.400+0000 7f2ca2400740 1 -- 172.21.15.83:0/706788522 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c900046f0 con 0x560935e52b30 2024-04-24T09:06:58.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.400+0000 7f2c7a000640 1 -- 172.21.15.83:0/706788522 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9583+0+0 (secure 0 0 0) 0x7f2c90004b20 con 0x560935e52b30 2024-04-24T09:06:58.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.400+0000 7f2c7a000640 1 -- 172.21.15.83:0/706788522 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f2c9003f050 con 0x560935e52b30 2024-04-24T09:06:58.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.400+0000 7f2ca2400740 10 client.21024 fetch_fsmap learned FSMap version 49 2024-04-24T09:06:58.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.400+0000 7f2ca2400740 1 -- 172.21.15.83:0/706788522 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x560935e5b210 con 0x560935e52b30 2024-04-24T09:06:58.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.401+0000 7f2c7a000640 1 -- 172.21.15.83:0/706788522 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2c900046f0 con 0x560935e52b30 2024-04-24T09:06:58.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.401+0000 7f2c7a000640 1 -- 172.21.15.83:0/706788522 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 49) ==== 8406+0+0 (secure 0 0 0) 0x7f2c90026460 con 0x560935e52b30 2024-04-24T09:06:58.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.401+0000 7f2ca2400740 10 client.21024 fetch_fsmap finished waiting for FSMap version 49 2024-04-24T09:06:58.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.401+0000 7f2ca2400740 10 client.21024 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-24T09:06:58.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.401+0000 7f2ca2400740 20 client.21024 populate_metadata read hostname 'smithi083' 2024-04-24T09:06:58.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.401+0000 7f2ca2400740 1 --2- 172.21.15.83:0/706788522 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x560935e62d20 0x560935ecb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:58.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.401+0000 7f2ca2400740 4 client.21024 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:06:58.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.401+0000 7f2ca2400740 1 -- 172.21.15.83:0/706788522 --> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560935db4a60 con 0x560935e62d20 2024-04-24T09:06:58.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.402+0000 7f2c9cc00640 1 --2- 172.21.15.83:0/706788522 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x560935e62d20 0x560935ecb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:58.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.402+0000 7f2c9cc00640 1 --2- 172.21.15.83:0/706788522 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x560935e62d20 0x560935ecb2c0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4584a2f6f7bcce4e server_cookie=e5017487f111045a in_seq=0 out_seq=0 2024-04-24T09:06:58.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.402+0000 7f2c7a000640 10 client.21024 ms_handle_connect on v2:172.21.15.138:6836/900566754 2024-04-24T09:06:58.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.403+0000 7f2c7a000640 1 -- 172.21.15.83:0/706788522 <== mds.1 v2:172.21.15.138:6836/900566754 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560935db4a60 con 0x560935e62d20 2024-04-24T09:06:58.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.403+0000 7f2c7a000640 10 client.21024 handle_command_reply: tid=0 2024-04-24T09:06:58.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.403+0000 7f2ca2400740 1 client.21024 shutdown 2024-04-24T09:06:58.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.403+0000 7f2ca2400740 1 -- 172.21.15.83:0/706788522 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560935e52b30 msgr2=0x560935e59d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:58.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.403+0000 7f2ca2400740 1 --2- 172.21.15.83:0/706788522 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560935e52b30 0x560935e59d90 secure :-1 s=READY pgs=2222 cs=0 l=1 rev1=1 crypto rx=0x7f2c90005c40 tx=0x7f2c90005e30 comp rx=0 tx=0).stop 2024-04-24T09:06:58.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.403+0000 7f2ca2400740 1 -- 172.21.15.83:0/706788522 shutdown_connections 2024-04-24T09:06:58.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.403+0000 7f2ca2400740 1 --2- 172.21.15.83:0/706788522 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560935e52b30 0x560935e59d90 unknown :-1 s=CLOSED pgs=2222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:58.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.403+0000 7f2ca2400740 1 --2- 172.21.15.83:0/706788522 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560935e48460 0x560935e4c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:58.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.403+0000 7f2ca2400740 1 --2- 172.21.15.83:0/706788522 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x560935e62d20 0x560935ecb2c0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:58.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.403+0000 7f2ca2400740 1 --2- 172.21.15.83:0/706788522 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560935e47b40 0x560935e4bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:58.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.403+0000 7f2ca2400740 1 -- 172.21.15.83:0/706788522 >> 172.21.15.83:0/706788522 conn(0x560935e43e60 msgr2=0x560935e4e920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:58.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.403+0000 7f2c7a000640 0 client.21024 ms_handle_reset on v2:172.21.15.138:6836/900566754 2024-04-24T09:06:58.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.403+0000 7f2ca2400740 1 -- 172.21.15.83:0/706788522 shutdown_connections 2024-04-24T09:06:58.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.404+0000 7f2ca2400740 1 -- 172.21.15.83:0/706788522 wait complete. 2024-04-24T09:06:58.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.404+0000 7f2ca2400740 20 client.21024 trim_cache size 0 max 16384 2024-04-24T09:06:58.406 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:06:58.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.404+0000 7f2c9e000640 1 -- 172.21.15.83:0/3414165503 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f2c800a3ee0 msgr2=0x7f2c800a63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:58.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.404+0000 7f2c9e000640 1 --2- 172.21.15.83:0/3414165503 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f2c800a3ee0 0x7f2c800a63a0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f2c90009370 tx=0x7f2c90009240 comp rx=0 tx=0).stop 2024-04-24T09:06:58.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.404+0000 7f2c9e000640 1 -- 172.21.15.83:0/3414165503 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c98101000 msgr2=0x7f2c98196500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:58.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.404+0000 7f2c9e000640 1 --2- 172.21.15.83:0/3414165503 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c98101000 0x7f2c98196500 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c00d950 tx=0x7f2c8c00de10 comp rx=0 tx=0).stop 2024-04-24T09:06:58.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.405+0000 7f2c9e000640 1 -- 172.21.15.83:0/3414165503 shutdown_connections 2024-04-24T09:06:58.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.405+0000 7f2c9e000640 1 --2- 172.21.15.83:0/3414165503 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f2c800a3ee0 0x7f2c800a63a0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:58.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.405+0000 7f2c9e000640 1 --2- 172.21.15.83:0/3414165503 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2c9819adc0 0x7f2c9819b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:58.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.405+0000 7f2c9e000640 1 --2- 172.21.15.83:0/3414165503 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c981060d0 0x7f2c98196a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:58.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.405+0000 7f2c9e000640 1 --2- 172.21.15.83:0/3414165503 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c98101000 0x7f2c98196500 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:58.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.405+0000 7f2c9e000640 1 -- 172.21.15.83:0/3414165503 >> 172.21.15.83:0/3414165503 conn(0x7f2c980fc880 msgr2=0x7f2c980fdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:58.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.405+0000 7f2c9e000640 1 -- 172.21.15.83:0/3414165503 shutdown_connections 2024-04-24T09:06:58.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.405+0000 7f2c9e000640 1 -- 172.21.15.83:0/3414165503 wait complete. 2024-04-24T09:06:58.427 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:06:58 smithi083 ceph-mon[27907]: pgmap v481: 129 pgs: 129 active+clean; 372 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 985 KiB/s rd, 3.3 MiB/s wr, 208 op/s 2024-04-24T09:06:58.430 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T09:06:58.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:06:58 smithi138 ceph-mon[31441]: pgmap v481: 129 pgs: 129 active+clean; 372 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 985 KiB/s rd, 3.3 MiB/s wr, 208 op/s 2024-04-24T09:06:58.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:06:58 smithi189 ceph-mon[31791]: pgmap v481: 129 pgs: 129 active+clean; 372 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 985 KiB/s rd, 3.3 MiB/s wr, 208 op/s 2024-04-24T09:06:58.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.680+0000 7f81eb200640 1 -- 172.21.15.83:0/3065936708 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81ec103060 msgr2=0x7f81ec103440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:58.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.680+0000 7f81eb200640 1 --2- 172.21.15.83:0/3065936708 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81ec103060 0x7f81ec103440 secure :-1 s=READY pgs=2223 cs=0 l=1 rev1=1 crypto rx=0x7f81dc00dc90 tx=0x7f81dc02e3f0 comp rx=0 tx=0).stop 2024-04-24T09:06:58.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.680+0000 7f81eb200640 1 -- 172.21.15.83:0/3065936708 shutdown_connections 2024-04-24T09:06:58.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.680+0000 7f81eb200640 1 --2- 172.21.15.83:0/3065936708 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f81ec10c480 0x7f81ec10e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:58.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.680+0000 7f81eb200640 1 --2- 172.21.15.83:0/3065936708 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81ec103980 0x7f81ec10be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:58.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.680+0000 7f81eb200640 1 --2- 172.21.15.83:0/3065936708 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81ec103060 0x7f81ec103440 unknown :-1 s=CLOSED pgs=2223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:58.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.680+0000 7f81eb200640 1 -- 172.21.15.83:0/3065936708 >> 172.21.15.83:0/3065936708 conn(0x7f81ec0fc880 msgr2=0x7f81ec0feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:58.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.680+0000 7f81eb200640 1 -- 172.21.15.83:0/3065936708 shutdown_connections 2024-04-24T09:06:58.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.680+0000 7f81eb200640 1 -- 172.21.15.83:0/3065936708 wait complete. 2024-04-24T09:06:58.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.681+0000 7f81eb200640 1 Processor -- start 2024-04-24T09:06:58.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.681+0000 7f81eb200640 1 -- start start 2024-04-24T09:06:58.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.682+0000 7f81eb200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81ec103060 0x7f81ec1a48d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:58.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.682+0000 7f81eb200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81ec103980 0x7f81ec1a4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:58.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.682+0000 7f81eb200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f81ec10c480 0x7f81ec19ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:58.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.682+0000 7f81eb200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f81ec10ff10 con 0x7f81ec103060 2024-04-24T09:06:58.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.682+0000 7f81eb200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f81ec10fd90 con 0x7f81ec103980 2024-04-24T09:06:58.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.682+0000 7f81eb200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f81ec110090 con 0x7f81ec10c480 2024-04-24T09:06:58.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.682+0000 7f81e9e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81ec103060 0x7f81ec1a48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:58.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.682+0000 7f81e9e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81ec103060 0x7f81ec1a48d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51294/0 (socket says 172.21.15.83:51294) 2024-04-24T09:06:58.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.682+0000 7f81e9e00640 1 -- 172.21.15.83:0/3744867197 learned_addr learned my addr 172.21.15.83:0/3744867197 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:06:58.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.682+0000 7f81e9400640 1 --2- 172.21.15.83:0/3744867197 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81ec103980 0x7f81ec1a4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:58.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.683+0000 7f81e9e00640 1 -- 172.21.15.83:0/3744867197 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f81ec10c480 msgr2=0x7f81ec19ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:58.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.683+0000 7f81ea800640 1 --2- 172.21.15.83:0/3744867197 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f81ec10c480 0x7f81ec19ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:58.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.683+0000 7f81e9e00640 1 --2- 172.21.15.83:0/3744867197 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f81ec10c480 0x7f81ec19ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:58.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.683+0000 7f81e9e00640 1 -- 172.21.15.83:0/3744867197 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81ec103980 msgr2=0x7f81ec1a4e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:58.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.683+0000 7f81e9e00640 1 --2- 172.21.15.83:0/3744867197 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81ec103980 0x7f81ec1a4e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:58.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.683+0000 7f81e9e00640 1 -- 172.21.15.83:0/3744867197 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81ec19f200 con 0x7f81ec103060 2024-04-24T09:06:58.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.683+0000 7f81ea800640 1 --2- 172.21.15.83:0/3744867197 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f81ec10c480 0x7f81ec19ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:06:58.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.683+0000 7f81e9e00640 1 --2- 172.21.15.83:0/3744867197 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81ec103060 0x7f81ec1a48d0 secure :-1 s=READY pgs=2224 cs=0 l=1 rev1=1 crypto rx=0x7f81dc005ab0 tx=0x7f81dc0038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:58.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.684+0000 7f81e2a00640 1 -- 172.21.15.83:0/3744867197 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81dc03d410 con 0x7f81ec103060 2024-04-24T09:06:58.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.684+0000 7f81e2a00640 1 -- 172.21.15.83:0/3744867197 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f81dc042070 con 0x7f81ec103060 2024-04-24T09:06:58.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.684+0000 7f81eb200640 1 -- 172.21.15.83:0/3744867197 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81ec19f490 con 0x7f81ec103060 2024-04-24T09:06:58.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.684+0000 7f81e2a00640 1 -- 172.21.15.83:0/3744867197 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81dc035660 con 0x7f81ec103060 2024-04-24T09:06:58.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.684+0000 7f81eb200640 1 -- 172.21.15.83:0/3744867197 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81ec1ab790 con 0x7f81ec103060 2024-04-24T09:06:58.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.689+0000 7f81f3fd8740 1 Processor -- start 2024-04-24T09:06:58.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.689+0000 7f81f3fd8740 1 -- start start 2024-04-24T09:06:58.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.689+0000 7f81f3fd8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d5a2e5b40 0x558d5a2e5f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:58.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.689+0000 7f81f3fd8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d5a2e6460 0x558d5a2f05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:58.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.690+0000 7f81f3fd8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558d5a2f0b30 0x558d5a2f2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:58.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.690+0000 7f81f3fd8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x558d5a24b1d0 con 0x558d5a2e5b40 2024-04-24T09:06:58.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.690+0000 7f81f3fd8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558d5a1f1450 con 0x558d5a2e6460 2024-04-24T09:06:58.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.690+0000 7f81f3fd8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x558d5a247190 con 0x558d5a2f0b30 2024-04-24T09:06:58.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.690+0000 7f81ea800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d5a2e5b40 0x558d5a2e5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:58.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.690+0000 7f81ea800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d5a2e5b40 0x558d5a2e5f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51298/0 (socket says 172.21.15.83:51298) 2024-04-24T09:06:58.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.690+0000 7f81ea800640 1 -- 172.21.15.83:0/3124731788 learned_addr learned my addr 172.21.15.83:0/3124731788 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:06:58.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.690+0000 7f81e9400640 1 --2- 172.21.15.83:0/3124731788 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558d5a2f0b30 0x558d5a2f2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:58.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.691+0000 7f81ea800640 1 -- 172.21.15.83:0/3124731788 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558d5a2f0b30 msgr2=0x558d5a2f2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:58.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.691+0000 7f81ea800640 1 --2- 172.21.15.83:0/3124731788 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558d5a2f0b30 0x558d5a2f2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:58.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.691+0000 7f81ea800640 1 -- 172.21.15.83:0/3124731788 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d5a2e6460 msgr2=0x558d5a2f05f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T09:06:58.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.691+0000 7f81ea800640 1 --2- 172.21.15.83:0/3124731788 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d5a2e6460 0x558d5a2f05f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:58.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.691+0000 7f81ea800640 1 -- 172.21.15.83:0/3124731788 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d5a2ae310 con 0x558d5a2e5b40 2024-04-24T09:06:58.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.691+0000 7f81e2a00640 1 -- 172.21.15.83:0/3744867197 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f81dc03d5b0 con 0x7f81ec103060 2024-04-24T09:06:58.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.691+0000 7f81ea800640 1 --2- 172.21.15.83:0/3124731788 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d5a2e5b40 0x558d5a2e5f20 secure :-1 s=READY pgs=2225 cs=0 l=1 rev1=1 crypto rx=0x7f81e4008d50 tx=0x7f81e400d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55878f1e976d58a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:58.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.692+0000 7f81d7400640 1 -- 172.21.15.83:0/3124731788 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81e400f830 con 0x558d5a2e5b40 2024-04-24T09:06:58.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.692+0000 7f81d7400640 1 -- 172.21.15.83:0/3124731788 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f81e400de20 con 0x558d5a2e5b40 2024-04-24T09:06:58.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.692+0000 7f81d7400640 1 -- 172.21.15.83:0/3124731788 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81e4018e10 con 0x558d5a2e5b40 2024-04-24T09:06:58.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.692+0000 7f81f3fd8740 1 -- 172.21.15.83:0/3124731788 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d5a2e5b40 msgr2=0x558d5a2e5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:58.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.692+0000 7f81f3fd8740 1 --2- 172.21.15.83:0/3124731788 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d5a2e5b40 0x558d5a2e5f20 secure :-1 s=READY pgs=2225 cs=0 l=1 rev1=1 crypto rx=0x7f81e4008d50 tx=0x7f81e400d710 comp rx=0 tx=0).stop 2024-04-24T09:06:58.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.694+0000 7f81f3fd8740 1 -- 172.21.15.83:0/3124731788 shutdown_connections 2024-04-24T09:06:58.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.694+0000 7f81f3fd8740 1 --2- 172.21.15.83:0/3124731788 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558d5a2f0b30 0x558d5a2f2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:58.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.694+0000 7f81f3fd8740 1 --2- 172.21.15.83:0/3124731788 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d5a2e6460 0x558d5a2f05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:58.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.694+0000 7f81f3fd8740 1 --2- 172.21.15.83:0/3124731788 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d5a2e5b40 0x558d5a2e5f20 unknown :-1 s=CLOSED pgs=2225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:58.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.694+0000 7f81f3fd8740 1 -- 172.21.15.83:0/3124731788 >> 172.21.15.83:0/3124731788 conn(0x558d5a2e1e60 msgr2=0x558d5a2e2240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:58.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.694+0000 7f81f3fd8740 1 -- 172.21.15.83:0/3124731788 shutdown_connections 2024-04-24T09:06:58.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.694+0000 7f81f3fd8740 1 -- 172.21.15.83:0/3124731788 wait complete. 2024-04-24T09:06:58.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.695+0000 7f81f3fd8740 1 -- start start 2024-04-24T09:06:58.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.695+0000 7f81f3fd8740 1 Processor -- start 2024-04-24T09:06:58.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.695+0000 7f81e2a00640 1 --2- 172.21.15.83:0/3744867197 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f81d00a3d80 0x7f81d00a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:58.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.696+0000 7f81e2a00640 1 -- 172.21.15.83:0/3744867197 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9583+0+0 (secure 0 0 0) 0x7f81dc0f13a0 con 0x7f81ec103060 2024-04-24T09:06:58.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.696+0000 7f81ea800640 1 --2- 172.21.15.83:0/3744867197 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f81d00a3d80 0x7f81d00a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:58.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.697+0000 7f81f3fd8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d5a2e5b40 0x558d5a2ed940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:58.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.697+0000 7f81f3fd8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d5a2e6460 0x558d5a2ede80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:58.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.697+0000 7f81f3fd8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558d5a2f0b30 0x558d5a2ee3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:58.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.698+0000 7f81f3fd8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x558d5a2f39c0 con 0x558d5a2e6460 2024-04-24T09:06:58.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.698+0000 7f81e9e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d5a2e5b40 0x558d5a2ed940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:58.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.698+0000 7f81f3fd8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558d5a204810 con 0x558d5a2e5b40 2024-04-24T09:06:58.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.698+0000 7f81f3fd8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x558d5a2f3b40 con 0x558d5a2f0b30 2024-04-24T09:06:58.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.698+0000 7f81e9400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d5a2e6460 0x558d5a2ede80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:58.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.698+0000 7f81e9e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d5a2e5b40 0x558d5a2ed940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:57048/0 (socket says 172.21.15.83:57048) 2024-04-24T09:06:58.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.698+0000 7f81e9e00640 1 -- 172.21.15.83:0/2529706314 learned_addr learned my addr 172.21.15.83:0/2529706314 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:06:58.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.698+0000 7f81f3fd8740 10 client.0 _ll_register_callbacks cb 0x558d5a1d2140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:06:58.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.698+0000 7f81ea800640 1 --2- 172.21.15.83:0/3744867197 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f81d00a3d80 0x7f81d00a6240 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f81e4008fa0 tx=0x7f81e4018990 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:58.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.698+0000 7f81e9400640 1 -- 172.21.15.83:0/2529706314 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558d5a2f0b30 msgr2=0x558d5a2ee3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:58.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.698+0000 7f81e9400640 1 --2- 172.21.15.83:0/2529706314 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558d5a2f0b30 0x558d5a2ee3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:58.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.698+0000 7f81e9400640 1 -- 172.21.15.83:0/2529706314 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d5a2e5b40 msgr2=0x558d5a2ed940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:58.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.698+0000 7f81e9400640 1 --2- 172.21.15.83:0/2529706314 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d5a2e5b40 0x558d5a2ed940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:58.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.698+0000 7f81e9400640 1 -- 172.21.15.83:0/2529706314 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d5a2153f0 con 0x558d5a2e6460 2024-04-24T09:06:58.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.698+0000 7f81e9e00640 1 --2- 172.21.15.83:0/2529706314 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d5a2e5b40 0x558d5a2ed940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:06:58.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.698+0000 7f81e9400640 1 --2- 172.21.15.83:0/2529706314 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d5a2e6460 0x558d5a2ede80 secure :-1 s=READY pgs=2226 cs=0 l=1 rev1=1 crypto rx=0x7f81ec0fef80 tx=0x7f81d800bf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc22efcd82d2ec32 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:58.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.699+0000 7f81cbe00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:06:58.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.699+0000 7f81cbe00640 1 -- 172.21.15.83:0/2529706314 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81d8003d30 con 0x558d5a2e6460 2024-04-24T09:06:58.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.699+0000 7f81cbe00640 1 -- 172.21.15.83:0/2529706314 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9583+0+0 (secure 0 0 0) 0x7f81d8005020 con 0x558d5a2e6460 2024-04-24T09:06:58.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.699+0000 7f81f3fd8740 1 -- 172.21.15.83:0/2529706314 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81e400de20 con 0x558d5a2e6460 2024-04-24T09:06:58.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.700+0000 7f81cbe00640 1 -- 172.21.15.83:0/2529706314 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f81d8005260 con 0x558d5a2e6460 2024-04-24T09:06:58.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.700+0000 7f81f3fd8740 10 client.21036 fetch_fsmap learned FSMap version 49 2024-04-24T09:06:58.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.700+0000 7f81f3fd8740 1 -- 172.21.15.83:0/2529706314 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x558d59f5d3f0 con 0x558d5a2e6460 2024-04-24T09:06:58.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.700+0000 7f81cbe00640 1 -- 172.21.15.83:0/2529706314 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f81e400de20 con 0x558d5a2e6460 2024-04-24T09:06:58.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.701+0000 7f81cbe00640 1 -- 172.21.15.83:0/2529706314 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 49) ==== 8406+0+0 (secure 0 0 0) 0x7f81d801b570 con 0x558d5a2e6460 2024-04-24T09:06:58.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.701+0000 7f81f3fd8740 10 client.21036 fetch_fsmap finished waiting for FSMap version 49 2024-04-24T09:06:58.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.701+0000 7f81f3fd8740 10 client.21036 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:06:58.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.701+0000 7f81f3fd8740 20 client.21036 populate_metadata read hostname 'smithi083' 2024-04-24T09:06:58.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.702+0000 7f81f3fd8740 1 --2- 172.21.15.83:0/2529706314 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x558d5a3011b0 0x558d5a303630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:58.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.702+0000 7f81f3fd8740 4 client.21036 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:06:58.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.702+0000 7f81f3fd8740 1 -- 172.21.15.83:0/2529706314 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558d5a258bf0 con 0x558d5a3011b0 2024-04-24T09:06:58.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.702+0000 7f81e9e00640 1 --2- 172.21.15.83:0/2529706314 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x558d5a3011b0 0x558d5a303630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:58.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.703+0000 7f81cbe00640 10 client.21036 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:06:58.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.703+0000 7f81e9e00640 1 --2- 172.21.15.83:0/2529706314 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x558d5a3011b0 0x558d5a303630 crc :-1 s=READY pgs=391 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b68c4c178f80cdbb server_cookie=d873ba8549e9a99b in_seq=0 out_seq=0 2024-04-24T09:06:58.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.705+0000 7f81cbe00640 1 -- 172.21.15.83:0/2529706314 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558d5a258bf0 con 0x558d5a3011b0 2024-04-24T09:06:58.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.705+0000 7f81cbe00640 10 client.21036 handle_command_reply: tid=0 2024-04-24T09:06:58.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.705+0000 7f81f3fd8740 1 client.21036 shutdown 2024-04-24T09:06:58.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.705+0000 7f81f3fd8740 1 -- 172.21.15.83:0/2529706314 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d5a2e6460 msgr2=0x558d5a2ede80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:58.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.705+0000 7f81f3fd8740 1 --2- 172.21.15.83:0/2529706314 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d5a2e6460 0x558d5a2ede80 secure :-1 s=READY pgs=2226 cs=0 l=1 rev1=1 crypto rx=0x7f81ec0fef80 tx=0x7f81d800bf20 comp rx=0 tx=0).stop 2024-04-24T09:06:58.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.706+0000 7f81f3fd8740 1 -- 172.21.15.83:0/2529706314 shutdown_connections 2024-04-24T09:06:58.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.706+0000 7f81f3fd8740 1 --2- 172.21.15.83:0/2529706314 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x558d5a3011b0 0x558d5a303630 crc :-1 s=READY pgs=391 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:58.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.706+0000 7f81f3fd8740 1 --2- 172.21.15.83:0/2529706314 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558d5a2f0b30 0x558d5a2ee3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:58.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.706+0000 7f81f3fd8740 1 --2- 172.21.15.83:0/2529706314 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d5a2e6460 0x558d5a2ede80 unknown :-1 s=CLOSED pgs=2226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:58.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.706+0000 7f81cbe00640 0 client.21036 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:06:58.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.706+0000 7f81f3fd8740 1 --2- 172.21.15.83:0/2529706314 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d5a2e5b40 0x558d5a2ed940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:58.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.706+0000 7f81f3fd8740 1 -- 172.21.15.83:0/2529706314 >> 172.21.15.83:0/2529706314 conn(0x558d5a2e1e60 msgr2=0x558d5a2e7b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:58.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.706+0000 7f81f3fd8740 1 -- 172.21.15.83:0/2529706314 shutdown_connections 2024-04-24T09:06:58.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.706+0000 7f81f3fd8740 1 -- 172.21.15.83:0/2529706314 wait complete. 2024-04-24T09:06:58.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.706+0000 7f81f3fd8740 20 client.21036 trim_cache size 0 max 16384 2024-04-24T09:06:58.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.730+0000 7f81f3fd8740 1 Processor -- start 2024-04-24T09:06:58.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.731+0000 7f81f3fd8740 1 -- start start 2024-04-24T09:06:58.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.731+0000 7f81f3fd8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558d5a2e5b40 0x558d5a2fc880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:58.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.731+0000 7f81f3fd8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d5a2e6460 0x558d5a2ecac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:58.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.731+0000 7f81f3fd8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d5a2f0b30 0x558d5a2ee010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:58.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.731+0000 7f81f3fd8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f81ec1b59f0 con 0x558d5a2f0b30 2024-04-24T09:06:58.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.731+0000 7f81f3fd8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f81ec1b76b0 con 0x558d5a2e6460 2024-04-24T09:06:58.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.731+0000 7f81f3fd8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f81ec1b60c0 con 0x558d5a2e5b40 2024-04-24T09:06:58.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.732+0000 7f81e9400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d5a2e6460 0x558d5a2ecac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:58.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.732+0000 7f81e9e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558d5a2e5b40 0x558d5a2fc880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:58.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.732+0000 7f81e9400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d5a2e6460 0x558d5a2ecac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:57060/0 (socket says 172.21.15.83:57060) 2024-04-24T09:06:58.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.732+0000 7f81e9400640 1 -- 172.21.15.83:0/1102651014 learned_addr learned my addr 172.21.15.83:0/1102651014 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:06:58.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.732+0000 7f81ea800640 1 --2- 172.21.15.83:0/1102651014 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d5a2f0b30 0x558d5a2ee010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:58.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.733+0000 7f81e9400640 1 -- 172.21.15.83:0/1102651014 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558d5a2e5b40 msgr2=0x558d5a2fc880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:58.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.733+0000 7f81e9400640 1 --2- 172.21.15.83:0/1102651014 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558d5a2e5b40 0x558d5a2fc880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:58.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.733+0000 7f81e9400640 1 -- 172.21.15.83:0/1102651014 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d5a2f0b30 msgr2=0x558d5a2ee010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:58.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.733+0000 7f81e9400640 1 --2- 172.21.15.83:0/1102651014 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d5a2f0b30 0x558d5a2ee010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:58.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.733+0000 7f81e9400640 1 -- 172.21.15.83:0/1102651014 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d5a0277d0 con 0x558d5a2e6460 2024-04-24T09:06:58.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.733+0000 7f81e9400640 1 --2- 172.21.15.83:0/1102651014 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d5a2e6460 0x558d5a2ecac0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f81d8001eb0 tx=0x7f81d80043a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10a70bd04b556e81 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:58.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.734+0000 7f81d6a00640 1 -- 172.21.15.83:0/1102651014 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81d8003d30 con 0x558d5a2e6460 2024-04-24T09:06:58.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.734+0000 7f81d6a00640 1 -- 172.21.15.83:0/1102651014 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f81d8005400 con 0x558d5a2e6460 2024-04-24T09:06:58.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.734+0000 7f81d6a00640 1 -- 172.21.15.83:0/1102651014 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81d801bde0 con 0x558d5a2e6460 2024-04-24T09:06:58.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.735+0000 7f81f3fd8740 1 -- 172.21.15.83:0/1102651014 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d5a2e6460 msgr2=0x558d5a2ecac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:58.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.735+0000 7f81f3fd8740 1 --2- 172.21.15.83:0/1102651014 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d5a2e6460 0x558d5a2ecac0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f81d8001eb0 tx=0x7f81d80043a0 comp rx=0 tx=0).stop 2024-04-24T09:06:58.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.735+0000 7f81f3fd8740 1 -- 172.21.15.83:0/1102651014 shutdown_connections 2024-04-24T09:06:58.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.736+0000 7f81f3fd8740 1 --2- 172.21.15.83:0/1102651014 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d5a2f0b30 0x558d5a2ee010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:58.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.736+0000 7f81f3fd8740 1 --2- 172.21.15.83:0/1102651014 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d5a2e6460 0x558d5a2ecac0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:58.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.736+0000 7f81f3fd8740 1 --2- 172.21.15.83:0/1102651014 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558d5a2e5b40 0x558d5a2fc880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:58.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.736+0000 7f81f3fd8740 1 -- 172.21.15.83:0/1102651014 >> 172.21.15.83:0/1102651014 conn(0x558d5a2e1e60 msgr2=0x558d5a2ed900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:58.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.736+0000 7f81f3fd8740 1 -- 172.21.15.83:0/1102651014 shutdown_connections 2024-04-24T09:06:58.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.736+0000 7f81f3fd8740 1 -- 172.21.15.83:0/1102651014 wait complete. 2024-04-24T09:06:58.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.737+0000 7f81f3fd8740 1 -- start start 2024-04-24T09:06:58.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.737+0000 7f81f3fd8740 1 Processor -- start 2024-04-24T09:06:58.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.739+0000 7f81f3fd8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558d5a2e5b40 0x558d5a2f9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:58.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.739+0000 7f81f3fd8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d5a2e6460 0x558d5a2f1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:58.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.739+0000 7f81f3fd8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d5a2f0b30 0x558d5a2f1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:58.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.739+0000 7f81f3fd8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f81ec1b6f20 con 0x558d5a2e6460 2024-04-24T09:06:58.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.739+0000 7f81f3fd8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f81ec1b67f0 con 0x558d5a2f0b30 2024-04-24T09:06:58.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.739+0000 7f81f3fd8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f81ec1b39f0 con 0x558d5a2e5b40 2024-04-24T09:06:58.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.739+0000 7f81f3fd8740 10 client.0 _ll_register_callbacks cb 0x7f81d8004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:06:58.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.739+0000 7f81e9400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d5a2e6460 0x558d5a2f1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:58.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.740+0000 7f81e9400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d5a2e6460 0x558d5a2f1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51324/0 (socket says 172.21.15.83:51324) 2024-04-24T09:06:58.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.740+0000 7f81e9400640 1 -- 172.21.15.83:0/2461224305 learned_addr learned my addr 172.21.15.83:0/2461224305 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:06:58.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.740+0000 7f81ea800640 1 --2- 172.21.15.83:0/2461224305 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d5a2f0b30 0x558d5a2f1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:58.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.740+0000 7f81e9e00640 1 --2- 172.21.15.83:0/2461224305 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558d5a2e5b40 0x558d5a2f9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:58.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.741+0000 7f81e9400640 1 -- 172.21.15.83:0/2461224305 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558d5a2e5b40 msgr2=0x558d5a2f9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:58.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.741+0000 7f81e9400640 1 --2- 172.21.15.83:0/2461224305 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558d5a2e5b40 0x558d5a2f9070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:58.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.741+0000 7f81e9400640 1 -- 172.21.15.83:0/2461224305 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d5a2f0b30 msgr2=0x558d5a2f1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:06:58.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.741+0000 7f81e9400640 1 --2- 172.21.15.83:0/2461224305 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d5a2f0b30 0x558d5a2f1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:58.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.741+0000 7f81e9400640 1 -- 172.21.15.83:0/2461224305 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d5a2f2190 con 0x558d5a2e6460 2024-04-24T09:06:58.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.741+0000 7f81e9400640 1 --2- 172.21.15.83:0/2461224305 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d5a2e6460 0x558d5a2f1510 secure :-1 s=READY pgs=2227 cs=0 l=1 rev1=1 crypto rx=0x7f81d801ba80 tx=0x7f81d80057e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=786b1a731d804201 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:06:58.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.741+0000 7f81cbe00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:06:58.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.741+0000 7f81cbe00640 1 -- 172.21.15.83:0/2461224305 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81d801bd90 con 0x558d5a2e6460 2024-04-24T09:06:58.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.742+0000 7f81cbe00640 1 -- 172.21.15.83:0/2461224305 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9583+0+0 (secure 0 0 0) 0x7f81d802f480 con 0x558d5a2e6460 2024-04-24T09:06:58.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.742+0000 7f81f3fd8740 1 -- 172.21.15.83:0/2461224305 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81d8005400 con 0x558d5a2e6460 2024-04-24T09:06:58.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.742+0000 7f81cbe00640 1 -- 172.21.15.83:0/2461224305 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f81d802f6a0 con 0x558d5a2e6460 2024-04-24T09:06:58.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.742+0000 7f81f3fd8740 10 client.21042 fetch_fsmap learned FSMap version 49 2024-04-24T09:06:58.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.742+0000 7f81f3fd8740 1 -- 172.21.15.83:0/2461224305 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x558d5a3677b0 con 0x558d5a2e6460 2024-04-24T09:06:58.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.742+0000 7f81cbe00640 1 -- 172.21.15.83:0/2461224305 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f81d8005400 con 0x558d5a2e6460 2024-04-24T09:06:58.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.743+0000 7f81cbe00640 1 -- 172.21.15.83:0/2461224305 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 49) ==== 8406+0+0 (secure 0 0 0) 0x7f81d8011b00 con 0x558d5a2e6460 2024-04-24T09:06:58.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.743+0000 7f81f3fd8740 10 client.21042 fetch_fsmap finished waiting for FSMap version 49 2024-04-24T09:06:58.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.743+0000 7f81f3fd8740 10 client.21042 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:06:58.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.743+0000 7f81f3fd8740 20 client.21042 populate_metadata read hostname 'smithi083' 2024-04-24T09:06:58.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.743+0000 7f81f3fd8740 1 --2- 172.21.15.83:0/2461224305 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x558d5a367e70 0x558d5a36a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:06:58.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.743+0000 7f81f3fd8740 4 client.21042 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:06:58.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.743+0000 7f81f3fd8740 1 -- 172.21.15.83:0/2461224305 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558d5a252a60 con 0x558d5a367e70 2024-04-24T09:06:58.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.744+0000 7f81e9e00640 1 --2- 172.21.15.83:0/2461224305 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x558d5a367e70 0x558d5a36a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:06:58.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.745+0000 7f81e9e00640 1 --2- 172.21.15.83:0/2461224305 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x558d5a367e70 0x558d5a36a290 crc :-1 s=READY pgs=392 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b66e32c11fd8c7a server_cookie=e7fd13078fe816c4 in_seq=0 out_seq=0 2024-04-24T09:06:58.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.745+0000 7f81cbe00640 10 client.21042 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:06:58.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.746+0000 7f81cbe00640 1 -- 172.21.15.83:0/2461224305 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558d5a252a60 con 0x558d5a367e70 2024-04-24T09:06:58.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.746+0000 7f81cbe00640 10 client.21042 handle_command_reply: tid=0 2024-04-24T09:06:58.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.746+0000 7f81f3fd8740 1 client.21042 shutdown 2024-04-24T09:06:58.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.746+0000 7f81f3fd8740 1 -- 172.21.15.83:0/2461224305 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d5a2e6460 msgr2=0x558d5a2f1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:58.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.746+0000 7f81f3fd8740 1 --2- 172.21.15.83:0/2461224305 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d5a2e6460 0x558d5a2f1510 secure :-1 s=READY pgs=2227 cs=0 l=1 rev1=1 crypto rx=0x7f81d801ba80 tx=0x7f81d80057e0 comp rx=0 tx=0).stop 2024-04-24T09:06:58.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.747+0000 7f81f3fd8740 1 -- 172.21.15.83:0/2461224305 shutdown_connections 2024-04-24T09:06:58.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.747+0000 7f81f3fd8740 1 --2- 172.21.15.83:0/2461224305 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x558d5a367e70 0x558d5a36a290 crc :-1 s=READY pgs=392 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:58.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.747+0000 7f81f3fd8740 1 --2- 172.21.15.83:0/2461224305 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d5a2f0b30 0x558d5a2f1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:58.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.747+0000 7f81cbe00640 0 client.21042 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:06:58.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.747+0000 7f81f3fd8740 1 --2- 172.21.15.83:0/2461224305 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d5a2e6460 0x558d5a2f1510 unknown :-1 s=CLOSED pgs=2227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:58.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.747+0000 7f81f3fd8740 1 --2- 172.21.15.83:0/2461224305 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558d5a2e5b40 0x558d5a2f9070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:58.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.747+0000 7f81f3fd8740 1 -- 172.21.15.83:0/2461224305 >> 172.21.15.83:0/2461224305 conn(0x558d5a2e1e60 msgr2=0x558d5a2ed8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:58.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.747+0000 7f81f3fd8740 1 -- 172.21.15.83:0/2461224305 shutdown_connections 2024-04-24T09:06:58.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.747+0000 7f81f3fd8740 1 -- 172.21.15.83:0/2461224305 wait complete. 2024-04-24T09:06:58.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.748+0000 7f81f3fd8740 20 client.21042 trim_cache size 0 max 16384 2024-04-24T09:06:58.749 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:06:58.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.749+0000 7f81eb200640 1 -- 172.21.15.83:0/3744867197 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f81d00a3d80 msgr2=0x7f81d00a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:58.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.749+0000 7f81eb200640 1 --2- 172.21.15.83:0/3744867197 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f81d00a3d80 0x7f81d00a6240 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f81e4008fa0 tx=0x7f81e4018990 comp rx=0 tx=0).stop 2024-04-24T09:06:58.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.749+0000 7f81eb200640 1 -- 172.21.15.83:0/3744867197 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81ec103060 msgr2=0x7f81ec1a48d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:06:58.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.749+0000 7f81eb200640 1 --2- 172.21.15.83:0/3744867197 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81ec103060 0x7f81ec1a48d0 secure :-1 s=READY pgs=2224 cs=0 l=1 rev1=1 crypto rx=0x7f81dc005ab0 tx=0x7f81dc0038a0 comp rx=0 tx=0).stop 2024-04-24T09:06:58.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.750+0000 7f81eb200640 1 -- 172.21.15.83:0/3744867197 shutdown_connections 2024-04-24T09:06:58.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.750+0000 7f81eb200640 1 --2- 172.21.15.83:0/3744867197 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f81d00a3d80 0x7f81d00a6240 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:58.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.750+0000 7f81eb200640 1 --2- 172.21.15.83:0/3744867197 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f81ec10c480 0x7f81ec19ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:58.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.750+0000 7f81eb200640 1 --2- 172.21.15.83:0/3744867197 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81ec103980 0x7f81ec1a4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:58.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.750+0000 7f81eb200640 1 --2- 172.21.15.83:0/3744867197 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81ec103060 0x7f81ec1a48d0 unknown :-1 s=CLOSED pgs=2224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:06:58.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.750+0000 7f81eb200640 1 -- 172.21.15.83:0/3744867197 >> 172.21.15.83:0/3744867197 conn(0x7f81ec0fc880 msgr2=0x7f81ec0fe0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:06:58.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.750+0000 7f81eb200640 1 -- 172.21.15.83:0/3744867197 shutdown_connections 2024-04-24T09:06:58.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:06:58.750+0000 7f81eb200640 1 -- 172.21.15.83:0/3744867197 wait complete. 2024-04-24T09:06:59.159 INFO:tasks.workunit.client.0.smithi083.stdout: 10 11731 8.95 MB/sec warmup 90 sec latency 171.279 ms 2024-04-24T09:06:59.774 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T09:06:59.774 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T09:06:59.774 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T09:07:00.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.022+0000 7f6d75200640 1 -- 172.21.15.83:0/1529801150 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d70104e60 msgr2=0x7f6d70108ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:00.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.022+0000 7f6d75200640 1 --2- 172.21.15.83:0/1529801150 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d70104e60 0x7f6d70108ea0 secure :-1 s=READY pgs=2228 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c009900 tx=0x7f6d5c02a3f0 comp rx=0 tx=0).stop 2024-04-24T09:07:00.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.022+0000 7f6d75200640 1 -- 172.21.15.83:0/1529801150 shutdown_connections 2024-04-24T09:07:00.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.022+0000 7f6d75200640 1 --2- 172.21.15.83:0/1529801150 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6d70109580 0x7f6d70111150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:00.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.022+0000 7f6d75200640 1 --2- 172.21.15.83:0/1529801150 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d70104e60 0x7f6d70108ea0 unknown :-1 s=CLOSED pgs=2228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:00.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.022+0000 7f6d75200640 1 --2- 172.21.15.83:0/1529801150 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d701044b0 0x7f6d70104890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:00.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.022+0000 7f6d75200640 1 -- 172.21.15.83:0/1529801150 >> 172.21.15.83:0/1529801150 conn(0x7f6d700ffd90 msgr2=0x7f6d701021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:00.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.023+0000 7f6d75200640 1 -- 172.21.15.83:0/1529801150 shutdown_connections 2024-04-24T09:07:00.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.023+0000 7f6d75200640 1 -- 172.21.15.83:0/1529801150 wait complete. 2024-04-24T09:07:00.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.024+0000 7f6d75200640 1 Processor -- start 2024-04-24T09:07:00.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.024+0000 7f6d75200640 1 -- start start 2024-04-24T09:07:00.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.024+0000 7f6d75200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6d701044b0 0x7f6d701a0700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:00.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.025+0000 7f6d75200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d70109580 0x7f6d701a0c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:00.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.025+0000 7f6d75200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d7019a8b0 0x7f6d7019ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:00.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.025+0000 7f6d75200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6d701144f0 con 0x7f6d7019a8b0 2024-04-24T09:07:00.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.025+0000 7f6d75200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6d70114370 con 0x7f6d70109580 2024-04-24T09:07:00.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.025+0000 7f6d75200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6d70114670 con 0x7f6d701044b0 2024-04-24T09:07:00.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.025+0000 7f6d6e800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d7019a8b0 0x7f6d7019ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:00.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.025+0000 7f6d6d400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d70109580 0x7f6d701a0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:00.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.025+0000 7f6d6e800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d7019a8b0 0x7f6d7019ad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46316/0 (socket says 172.21.15.83:46316) 2024-04-24T09:07:00.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.025+0000 7f6d6e800640 1 -- 172.21.15.83:0/561798581 learned_addr learned my addr 172.21.15.83:0/561798581 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:07:00.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.025+0000 7f6d6e800640 1 -- 172.21.15.83:0/561798581 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6d701044b0 msgr2=0x7f6d701a0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:00.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.025+0000 7f6d6e800640 1 --2- 172.21.15.83:0/561798581 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6d701044b0 0x7f6d701a0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:00.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.025+0000 7f6d6e800640 1 -- 172.21.15.83:0/561798581 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d70109580 msgr2=0x7f6d701a0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:00.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.025+0000 7f6d6e800640 1 --2- 172.21.15.83:0/561798581 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d70109580 0x7f6d701a0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:00.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.025+0000 7f6d6e800640 1 -- 172.21.15.83:0/561798581 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d7019b3d0 con 0x7f6d7019a8b0 2024-04-24T09:07:00.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.026+0000 7f6d6e800640 1 --2- 172.21.15.83:0/561798581 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d7019a8b0 0x7f6d7019ad60 secure :-1 s=READY pgs=2229 cs=0 l=1 rev1=1 crypto rx=0x7f6d6800d750 tx=0x7f6d6800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:00.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.026+0000 7f6d66a00640 1 -- 172.21.15.83:0/561798581 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d68017070 con 0x7f6d7019a8b0 2024-04-24T09:07:00.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.026+0000 7f6d66a00640 1 -- 172.21.15.83:0/561798581 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6d680054d0 con 0x7f6d7019a8b0 2024-04-24T09:07:00.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.026+0000 7f6d66a00640 1 -- 172.21.15.83:0/561798581 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d68004c60 con 0x7f6d7019a8b0 2024-04-24T09:07:00.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.026+0000 7f6d75200640 1 -- 172.21.15.83:0/561798581 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d7019b6c0 con 0x7f6d7019a8b0 2024-04-24T09:07:00.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.026+0000 7f6d75200640 1 -- 172.21.15.83:0/561798581 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d701a78c0 con 0x7f6d7019a8b0 2024-04-24T09:07:00.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.031+0000 7f6d7800b740 1 Processor -- start 2024-04-24T09:07:00.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.031+0000 7f6d66a00640 1 -- 172.21.15.83:0/561798581 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6d68004e00 con 0x7f6d7019a8b0 2024-04-24T09:07:00.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.031+0000 7f6d7800b740 1 -- start start 2024-04-24T09:07:00.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.032+0000 7f6d7800b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd9719d090 0x55dd9719d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:00.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.032+0000 7f6d7800b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dd9719d9b0 0x55dd971a7b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:00.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.032+0000 7f6d7800b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd971a8080 0x55dd971aa470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:00.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.032+0000 7f6d7800b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55dd97102e70 con 0x55dd971a8080 2024-04-24T09:07:00.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.032+0000 7f6d7800b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55dd970a9380 con 0x55dd9719d9b0 2024-04-24T09:07:00.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.032+0000 7f6d7800b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55dd970fee30 con 0x55dd9719d090 2024-04-24T09:07:00.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.032+0000 7f6d6de00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dd9719d9b0 0x55dd971a7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:00.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.032+0000 7f6d6d400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd971a8080 0x55dd971aa470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:00.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.032+0000 7f6d6de00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dd9719d9b0 0x55dd971a7b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:34226/0 (socket says 172.21.15.83:34226) 2024-04-24T09:07:00.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.032+0000 7f6d6de00640 1 -- 172.21.15.83:0/3069161848 learned_addr learned my addr 172.21.15.83:0/3069161848 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:07:00.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.033+0000 7f6d6d400640 1 --2- 172.21.15.83:0/3069161848 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd9719d090 0x55dd9719d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:00.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.033+0000 7f6d6de00640 1 -- 172.21.15.83:0/3069161848 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd9719d090 msgr2=0x55dd9719d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:00.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.033+0000 7f6d6de00640 1 --2- 172.21.15.83:0/3069161848 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd9719d090 0x55dd9719d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:00.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.033+0000 7f6d6de00640 1 -- 172.21.15.83:0/3069161848 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd971a8080 msgr2=0x55dd971aa470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:00.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.033+0000 7f6d6de00640 1 --2- 172.21.15.83:0/3069161848 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd971a8080 0x55dd971aa470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:00.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.033+0000 7f6d6de00640 1 -- 172.21.15.83:0/3069161848 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd96edd7e0 con 0x55dd9719d9b0 2024-04-24T09:07:00.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.033+0000 7f6d6d400640 1 --2- 172.21.15.83:0/3069161848 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd971a8080 0x55dd971aa470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:07:00.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.033+0000 7f6d66a00640 1 --2- 172.21.15.83:0/561798581 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f6d540a3d20 0x7f6d540a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:00.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.033+0000 7f6d66a00640 1 -- 172.21.15.83:0/561798581 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9583+0+0 (secure 0 0 0) 0x7f6d680d04f0 con 0x7f6d7019a8b0 2024-04-24T09:07:00.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.033+0000 7f6d6de00640 1 --2- 172.21.15.83:0/3069161848 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dd9719d9b0 0x55dd971a7b40 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6d60009dd0 tx=0x7f6d6000e6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1cf2523b5e20970b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:00.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.034+0000 7f6d6d400640 1 --2- 172.21.15.83:0/561798581 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f6d540a3d20 0x7f6d540a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:00.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.034+0000 7f6d5b400640 1 -- 172.21.15.83:0/3069161848 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d60016070 con 0x55dd9719d9b0 2024-04-24T09:07:00.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.034+0000 7f6d5b400640 1 -- 172.21.15.83:0/3069161848 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6d60005020 con 0x55dd9719d9b0 2024-04-24T09:07:00.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.034+0000 7f6d5b400640 1 -- 172.21.15.83:0/3069161848 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d600116b0 con 0x55dd9719d9b0 2024-04-24T09:07:00.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.035+0000 7f6d7800b740 1 -- 172.21.15.83:0/3069161848 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dd9719d9b0 msgr2=0x55dd971a7b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:00.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.035+0000 7f6d7800b740 1 --2- 172.21.15.83:0/3069161848 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dd9719d9b0 0x55dd971a7b40 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6d60009dd0 tx=0x7f6d6000e6c0 comp rx=0 tx=0).stop 2024-04-24T09:07:00.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.035+0000 7f6d7800b740 1 -- 172.21.15.83:0/3069161848 shutdown_connections 2024-04-24T09:07:00.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.035+0000 7f6d7800b740 1 --2- 172.21.15.83:0/3069161848 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd971a8080 0x55dd971aa470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:00.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.035+0000 7f6d7800b740 1 --2- 172.21.15.83:0/3069161848 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dd9719d9b0 0x55dd971a7b40 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:00.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.035+0000 7f6d7800b740 1 --2- 172.21.15.83:0/3069161848 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd9719d090 0x55dd9719d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:00.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.035+0000 7f6d7800b740 1 -- 172.21.15.83:0/3069161848 >> 172.21.15.83:0/3069161848 conn(0x55dd971993b0 msgr2=0x55dd97199790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:00.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.035+0000 7f6d6d400640 1 --2- 172.21.15.83:0/561798581 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f6d540a3d20 0x7f6d540a61e0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c031440 tx=0x7f6d5c005780 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:00.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.036+0000 7f6d7800b740 1 -- 172.21.15.83:0/3069161848 shutdown_connections 2024-04-24T09:07:00.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.036+0000 7f6d7800b740 1 -- 172.21.15.83:0/3069161848 wait complete. 2024-04-24T09:07:00.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.036+0000 7f6d7800b740 1 -- start start 2024-04-24T09:07:00.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.037+0000 7f6d7800b740 1 Processor -- start 2024-04-24T09:07:00.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.037+0000 7f6d7800b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd9719d090 0x55dd971a4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:00.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.037+0000 7f6d7800b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dd9719d9b0 0x55dd971a5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:00.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.037+0000 7f6d7800b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd971a8080 0x55dd971a5a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:00.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.037+0000 7f6d7800b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55dd971aae70 con 0x55dd971a8080 2024-04-24T09:07:00.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.037+0000 7f6d7800b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55dd970bc660 con 0x55dd9719d9b0 2024-04-24T09:07:00.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.037+0000 7f6d7800b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55dd971aaff0 con 0x55dd9719d090 2024-04-24T09:07:00.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.037+0000 7f6d7800b740 10 client.0 _ll_register_callbacks cb 0x55dd970aaaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:07:00.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.038+0000 7f6d6d400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dd9719d9b0 0x55dd971a5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:00.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.038+0000 7f6d6d400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dd9719d9b0 0x55dd971a5530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:34238/0 (socket says 172.21.15.83:34238) 2024-04-24T09:07:00.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.038+0000 7f6d6d400640 1 -- 172.21.15.83:0/1631932167 learned_addr learned my addr 172.21.15.83:0/1631932167 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:07:00.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.038+0000 7f6d6e800640 1 --2- 172.21.15.83:0/1631932167 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd971a8080 0x55dd971a5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:00.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.038+0000 7f6d6de00640 1 --2- 172.21.15.83:0/1631932167 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd9719d090 0x55dd971a4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:00.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.038+0000 7f6d6d400640 1 -- 172.21.15.83:0/1631932167 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd9719d090 msgr2=0x55dd971a4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:00.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.038+0000 7f6d6d400640 1 --2- 172.21.15.83:0/1631932167 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd9719d090 0x55dd971a4ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:00.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.038+0000 7f6d6d400640 1 -- 172.21.15.83:0/1631932167 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd971a8080 msgr2=0x55dd971a5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:00.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.038+0000 7f6d6d400640 1 --2- 172.21.15.83:0/1631932167 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd971a8080 0x55dd971a5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:00.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.038+0000 7f6d6d400640 1 -- 172.21.15.83:0/1631932167 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd96f3df70 con 0x55dd9719d9b0 2024-04-24T09:07:00.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.038+0000 7f6d6d400640 1 --2- 172.21.15.83:0/1631932167 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dd9719d9b0 0x55dd971a5530 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c004650 tx=0x7f6d5c02a3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8bd3e9615bb0b5a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:00.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.038+0000 7f6d4fe00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:07:00.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.039+0000 7f6d4fe00640 1 -- 172.21.15.83:0/1631932167 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d5c035040 con 0x55dd9719d9b0 2024-04-24T09:07:00.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.039+0000 7f6d4fe00640 1 -- 172.21.15.83:0/1631932167 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9583+0+0 (secure 0 0 0) 0x7f6d5c031a30 con 0x55dd9719d9b0 2024-04-24T09:07:00.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.039+0000 7f6d7800b740 1 -- 172.21.15.83:0/1631932167 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d60005020 con 0x55dd9719d9b0 2024-04-24T09:07:00.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.039+0000 7f6d4fe00640 1 -- 172.21.15.83:0/1631932167 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f6d5c031c30 con 0x55dd9719d9b0 2024-04-24T09:07:00.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.039+0000 7f6d7800b740 10 client.38201 fetch_fsmap learned FSMap version 49 2024-04-24T09:07:00.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.040+0000 7f6d7800b740 1 -- 172.21.15.83:0/1631932167 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55dd96dd5170 con 0x55dd9719d9b0 2024-04-24T09:07:00.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.040+0000 7f6d4fe00640 1 -- 172.21.15.83:0/1631932167 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6d60005020 con 0x55dd9719d9b0 2024-04-24T09:07:00.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.040+0000 7f6d4fe00640 1 -- 172.21.15.83:0/1631932167 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 49) ==== 8406+0+0 (secure 0 0 0) 0x7f6d5c04d440 con 0x55dd9719d9b0 2024-04-24T09:07:00.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.041+0000 7f6d7800b740 10 client.38201 fetch_fsmap finished waiting for FSMap version 49 2024-04-24T09:07:00.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.041+0000 7f6d7800b740 10 client.38201 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:07:00.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.041+0000 7f6d7800b740 20 client.38201 populate_metadata read hostname 'smithi083' 2024-04-24T09:07:00.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.041+0000 7f6d7800b740 1 --2- 172.21.15.83:0/1631932167 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55dd971b84e0 0x55dd971ba910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:00.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.041+0000 7f6d7800b740 4 client.38201 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:07:00.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.041+0000 7f6d7800b740 1 -- 172.21.15.83:0/1631932167 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dd97110890 con 0x55dd971b84e0 2024-04-24T09:07:00.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.041+0000 7f6d6de00640 1 --2- 172.21.15.83:0/1631932167 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55dd971b84e0 0x55dd971ba910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:00.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.042+0000 7f6d4fe00640 10 client.38201 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:07:00.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.042+0000 7f6d6de00640 1 --2- 172.21.15.83:0/1631932167 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55dd971b84e0 0x55dd971ba910 crc :-1 s=READY pgs=393 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f76b4c09ecf0e8d1 server_cookie=5be795639264bfc7 in_seq=0 out_seq=0 2024-04-24T09:07:00.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.043+0000 7f6d4fe00640 1 -- 172.21.15.83:0/1631932167 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55dd97110890 con 0x55dd971b84e0 2024-04-24T09:07:00.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.043+0000 7f6d4fe00640 10 client.38201 handle_command_reply: tid=0 2024-04-24T09:07:00.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.044+0000 7f6d7800b740 1 client.38201 shutdown 2024-04-24T09:07:00.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.044+0000 7f6d7800b740 1 -- 172.21.15.83:0/1631932167 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dd9719d9b0 msgr2=0x55dd971a5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:00.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.044+0000 7f6d7800b740 1 --2- 172.21.15.83:0/1631932167 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dd9719d9b0 0x55dd971a5530 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c004650 tx=0x7f6d5c02a3f0 comp rx=0 tx=0).stop 2024-04-24T09:07:00.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.044+0000 7f6d7800b740 1 -- 172.21.15.83:0/1631932167 shutdown_connections 2024-04-24T09:07:00.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.044+0000 7f6d7800b740 1 --2- 172.21.15.83:0/1631932167 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55dd971b84e0 0x55dd971ba910 crc :-1 s=READY pgs=393 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:00.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.044+0000 7f6d7800b740 1 --2- 172.21.15.83:0/1631932167 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd971a8080 0x55dd971a5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:00.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.044+0000 7f6d7800b740 1 --2- 172.21.15.83:0/1631932167 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dd9719d9b0 0x55dd971a5530 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:00.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.044+0000 7f6d4fe00640 0 client.38201 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:07:00.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.044+0000 7f6d7800b740 1 --2- 172.21.15.83:0/1631932167 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd9719d090 0x55dd971a4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:00.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.044+0000 7f6d7800b740 1 -- 172.21.15.83:0/1631932167 >> 172.21.15.83:0/1631932167 conn(0x55dd971993b0 msgr2=0x55dd9719f1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:00.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.045+0000 7f6d7800b740 1 -- 172.21.15.83:0/1631932167 shutdown_connections 2024-04-24T09:07:00.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.045+0000 7f6d7800b740 1 -- 172.21.15.83:0/1631932167 wait complete. 2024-04-24T09:07:00.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.045+0000 7f6d7800b740 20 client.38201 trim_cache size 0 max 16384 2024-04-24T09:07:00.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.077+0000 7f6d7800b740 1 Processor -- start 2024-04-24T09:07:00.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.077+0000 7f6d7800b740 1 -- start start 2024-04-24T09:07:00.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.078+0000 7f6d7800b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd9719d090 0x55dd971a4910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:00.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.078+0000 7f6d7800b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dd9719d9b0 0x55dd9719f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:00.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.078+0000 7f6d7800b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd971a8080 0x55dd97198390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:00.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.078+0000 7f6d7800b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6d701b18c0 con 0x55dd9719d090 2024-04-24T09:07:00.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.078+0000 7f6d7800b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6d701b3580 con 0x55dd9719d9b0 2024-04-24T09:07:00.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.078+0000 7f6d7800b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6d701b1f90 con 0x55dd971a8080 2024-04-24T09:07:00.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.078+0000 7f6d6de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd9719d090 0x55dd971a4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:00.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.078+0000 7f6d6de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd9719d090 0x55dd971a4910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46340/0 (socket says 172.21.15.83:46340) 2024-04-24T09:07:00.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.078+0000 7f6d6de00640 1 -- 172.21.15.83:0/1994953082 learned_addr learned my addr 172.21.15.83:0/1994953082 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:07:00.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.078+0000 7f6d6d400640 1 --2- 172.21.15.83:0/1994953082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dd9719d9b0 0x55dd9719f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:00.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.079+0000 7f6d6e800640 1 --2- 172.21.15.83:0/1994953082 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd971a8080 0x55dd97198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:00.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.079+0000 7f6d6de00640 1 -- 172.21.15.83:0/1994953082 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd971a8080 msgr2=0x55dd97198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:00.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.079+0000 7f6d6de00640 1 --2- 172.21.15.83:0/1994953082 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd971a8080 0x55dd97198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:00.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.079+0000 7f6d6de00640 1 -- 172.21.15.83:0/1994953082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dd9719d9b0 msgr2=0x55dd9719f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:00.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.079+0000 7f6d6de00640 1 --2- 172.21.15.83:0/1994953082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dd9719d9b0 0x55dd9719f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:00.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.079+0000 7f6d6de00640 1 -- 172.21.15.83:0/1994953082 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd9719d7e0 con 0x55dd9719d090 2024-04-24T09:07:00.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.079+0000 7f6d6d400640 1 --2- 172.21.15.83:0/1994953082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dd9719d9b0 0x55dd9719f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:07:00.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.079+0000 7f6d6de00640 1 --2- 172.21.15.83:0/1994953082 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd9719d090 0x55dd971a4910 secure :-1 s=READY pgs=2230 cs=0 l=1 rev1=1 crypto rx=0x7f6d60009dd0 tx=0x7f6d60000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b2257abeacab934 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:00.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.079+0000 7f6d5aa00640 1 -- 172.21.15.83:0/1994953082 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d60016040 con 0x55dd9719d090 2024-04-24T09:07:00.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.080+0000 7f6d5aa00640 1 -- 172.21.15.83:0/1994953082 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6d60009640 con 0x55dd9719d090 2024-04-24T09:07:00.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.080+0000 7f6d5aa00640 1 -- 172.21.15.83:0/1994953082 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d60028280 con 0x55dd9719d090 2024-04-24T09:07:00.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.080+0000 7f6d7800b740 1 -- 172.21.15.83:0/1994953082 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd9719d090 msgr2=0x55dd971a4910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:00.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.080+0000 7f6d7800b740 1 --2- 172.21.15.83:0/1994953082 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd9719d090 0x55dd971a4910 secure :-1 s=READY pgs=2230 cs=0 l=1 rev1=1 crypto rx=0x7f6d60009dd0 tx=0x7f6d60000f40 comp rx=0 tx=0).stop 2024-04-24T09:07:00.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.081+0000 7f6d7800b740 1 -- 172.21.15.83:0/1994953082 shutdown_connections 2024-04-24T09:07:00.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.081+0000 7f6d7800b740 1 --2- 172.21.15.83:0/1994953082 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd971a8080 0x55dd97198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:00.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.081+0000 7f6d7800b740 1 --2- 172.21.15.83:0/1994953082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dd9719d9b0 0x55dd9719f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:00.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.081+0000 7f6d7800b740 1 --2- 172.21.15.83:0/1994953082 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd9719d090 0x55dd971a4910 unknown :-1 s=CLOSED pgs=2230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:00.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.081+0000 7f6d7800b740 1 -- 172.21.15.83:0/1994953082 >> 172.21.15.83:0/1994953082 conn(0x55dd971993b0 msgr2=0x55dd97197c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:00.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.081+0000 7f6d7800b740 1 -- 172.21.15.83:0/1994953082 shutdown_connections 2024-04-24T09:07:00.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.081+0000 7f6d7800b740 1 -- 172.21.15.83:0/1994953082 wait complete. 2024-04-24T09:07:00.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.082+0000 7f6d7800b740 1 -- start start 2024-04-24T09:07:00.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.082+0000 7f6d7800b740 1 Processor -- start 2024-04-24T09:07:00.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.082+0000 7f6d7800b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dd9719d090 0x55dd971a1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:00.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.082+0000 7f6d7800b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd9719d9b0 0x55dd971a2230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:00.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.082+0000 7f6d7800b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd971a8080 0x55dd971a2770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:00.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.082+0000 7f6d7800b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6d701b2df0 con 0x55dd9719d9b0 2024-04-24T09:07:00.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.082+0000 7f6d7800b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6d701b26c0 con 0x55dd9719d090 2024-04-24T09:07:00.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.082+0000 7f6d7800b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6d701af8c0 con 0x55dd971a8080 2024-04-24T09:07:00.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.082+0000 7f6d7800b740 10 client.0 _ll_register_callbacks cb 0x7f6d5c0041c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:07:00.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.083+0000 7f6d6de00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dd9719d090 0x55dd971a1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:00.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.083+0000 7f6d6de00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dd9719d090 0x55dd971a1cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:34254/0 (socket says 172.21.15.83:34254) 2024-04-24T09:07:00.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.083+0000 7f6d6de00640 1 -- 172.21.15.83:0/3876833701 learned_addr learned my addr 172.21.15.83:0/3876833701 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:07:00.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.083+0000 7f6d6d400640 1 --2- 172.21.15.83:0/3876833701 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd9719d9b0 0x55dd971a2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:00.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.083+0000 7f6d6de00640 1 -- 172.21.15.83:0/3876833701 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd971a8080 msgr2=0x55dd971a2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:00.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.083+0000 7f6d6de00640 1 --2- 172.21.15.83:0/3876833701 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd971a8080 0x55dd971a2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:00.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.083+0000 7f6d6de00640 1 -- 172.21.15.83:0/3876833701 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd9719d9b0 msgr2=0x55dd971a2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:00.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.083+0000 7f6d6de00640 1 --2- 172.21.15.83:0/3876833701 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd9719d9b0 0x55dd971a2230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:00.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.083+0000 7f6d6de00640 1 -- 172.21.15.83:0/3876833701 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd9721da50 con 0x55dd9719d090 2024-04-24T09:07:00.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.083+0000 7f6d6d400640 1 --2- 172.21.15.83:0/3876833701 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd9719d9b0 0x55dd971a2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:07:00.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.084+0000 7f6d6de00640 1 --2- 172.21.15.83:0/3876833701 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dd9719d090 0x55dd971a1cf0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f6d60004860 tx=0x7f6d60004890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a611c3499813c748 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:00.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.084+0000 7f6d4fe00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:07:00.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.084+0000 7f6d4fe00640 1 -- 172.21.15.83:0/3876833701 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d60016030 con 0x55dd9719d090 2024-04-24T09:07:00.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.084+0000 7f6d7800b740 1 -- 172.21.15.83:0/3876833701 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d60009640 con 0x55dd9719d090 2024-04-24T09:07:00.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.084+0000 7f6d4fe00640 1 -- 172.21.15.83:0/3876833701 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9583+0+0 (secure 0 0 0) 0x7f6d60028430 con 0x55dd9719d090 2024-04-24T09:07:00.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.085+0000 7f6d4fe00640 1 -- 172.21.15.83:0/3876833701 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f6d6003f050 con 0x55dd9719d090 2024-04-24T09:07:00.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.085+0000 7f6d7800b740 10 client.38207 fetch_fsmap learned FSMap version 49 2024-04-24T09:07:00.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.085+0000 7f6d7800b740 1 -- 172.21.15.83:0/3876833701 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55dd9721ed70 con 0x55dd9719d090 2024-04-24T09:07:00.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.085+0000 7f6d4fe00640 1 -- 172.21.15.83:0/3876833701 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6d60009640 con 0x55dd9719d090 2024-04-24T09:07:00.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.085+0000 7f6d4fe00640 1 -- 172.21.15.83:0/3876833701 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 49) ==== 8406+0+0 (secure 0 0 0) 0x7f6d6000f040 con 0x55dd9719d090 2024-04-24T09:07:00.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.086+0000 7f6d7800b740 10 client.38207 fetch_fsmap finished waiting for FSMap version 49 2024-04-24T09:07:00.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.086+0000 7f6d7800b740 10 client.38207 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:07:00.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.086+0000 7f6d7800b740 20 client.38207 populate_metadata read hostname 'smithi083' 2024-04-24T09:07:00.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.086+0000 7f6d7800b740 1 --2- 172.21.15.83:0/3876833701 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55dd9721f3b0 0x55dd972217d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:00.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.086+0000 7f6d7800b740 4 client.38207 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T09:07:00.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.086+0000 7f6d7800b740 1 -- 172.21.15.83:0/3876833701 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55dd9710a700 con 0x55dd9721f3b0 2024-04-24T09:07:00.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.087+0000 7f6d6d400640 1 --2- 172.21.15.83:0/3876833701 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55dd9721f3b0 0x55dd972217d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:00.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.087+0000 7f6d4fe00640 10 client.38207 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:07:00.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.087+0000 7f6d6d400640 1 --2- 172.21.15.83:0/3876833701 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55dd9721f3b0 0x55dd972217d0 crc :-1 s=READY pgs=394 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=884b476e6b5288ca server_cookie=1109bcb4e8617e73 in_seq=0 out_seq=0 2024-04-24T09:07:00.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.089+0000 7f6d4fe00640 1 -- 172.21.15.83:0/3876833701 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55dd9710a700 con 0x55dd9721f3b0 2024-04-24T09:07:00.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.089+0000 7f6d4fe00640 10 client.38207 handle_command_reply: tid=0 2024-04-24T09:07:00.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.089+0000 7f6d7800b740 1 client.38207 shutdown 2024-04-24T09:07:00.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.089+0000 7f6d7800b740 1 -- 172.21.15.83:0/3876833701 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dd9719d090 msgr2=0x55dd971a1cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:00.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.089+0000 7f6d7800b740 1 --2- 172.21.15.83:0/3876833701 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dd9719d090 0x55dd971a1cf0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f6d60004860 tx=0x7f6d60004890 comp rx=0 tx=0).stop 2024-04-24T09:07:00.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.090+0000 7f6d7800b740 1 -- 172.21.15.83:0/3876833701 shutdown_connections 2024-04-24T09:07:00.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.090+0000 7f6d7800b740 1 --2- 172.21.15.83:0/3876833701 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55dd9721f3b0 0x55dd972217d0 crc :-1 s=READY pgs=394 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:00.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.090+0000 7f6d7800b740 1 --2- 172.21.15.83:0/3876833701 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd971a8080 0x55dd971a2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:00.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.090+0000 7f6d7800b740 1 --2- 172.21.15.83:0/3876833701 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd9719d9b0 0x55dd971a2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:00.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.090+0000 7f6d7800b740 1 --2- 172.21.15.83:0/3876833701 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dd9719d090 0x55dd971a1cf0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:00.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.090+0000 7f6d7800b740 1 -- 172.21.15.83:0/3876833701 >> 172.21.15.83:0/3876833701 conn(0x55dd971993b0 msgr2=0x55dd97198510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:00.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.090+0000 7f6d4fe00640 0 client.38207 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:07:00.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.090+0000 7f6d7800b740 1 -- 172.21.15.83:0/3876833701 shutdown_connections 2024-04-24T09:07:00.091 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T09:07:00.092 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-04-24T09:07:00.092 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "15186edf-ffec-4fed-a6d2-061e3019a571", 2024-04-24T09:07:00.092 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-04-24T09:07:00.092 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T09:07:00.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.090+0000 7f6d7800b740 1 -- 172.21.15.83:0/3876833701 wait complete. 2024-04-24T09:07:00.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.090+0000 7f6d7800b740 20 client.38207 trim_cache size 0 max 16384 2024-04-24T09:07:00.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.091+0000 7f6d75200640 1 -- 172.21.15.83:0/561798581 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f6d540a3d20 msgr2=0x7f6d540a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:00.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.091+0000 7f6d75200640 1 --2- 172.21.15.83:0/561798581 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f6d540a3d20 0x7f6d540a61e0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c031440 tx=0x7f6d5c005780 comp rx=0 tx=0).stop 2024-04-24T09:07:00.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.091+0000 7f6d75200640 1 -- 172.21.15.83:0/561798581 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d7019a8b0 msgr2=0x7f6d7019ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:00.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.091+0000 7f6d75200640 1 --2- 172.21.15.83:0/561798581 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d7019a8b0 0x7f6d7019ad60 secure :-1 s=READY pgs=2229 cs=0 l=1 rev1=1 crypto rx=0x7f6d6800d750 tx=0x7f6d6800dc10 comp rx=0 tx=0).stop 2024-04-24T09:07:00.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.091+0000 7f6d75200640 1 -- 172.21.15.83:0/561798581 shutdown_connections 2024-04-24T09:07:00.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.091+0000 7f6d75200640 1 --2- 172.21.15.83:0/561798581 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f6d540a3d20 0x7f6d540a61e0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:00.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.091+0000 7f6d75200640 1 --2- 172.21.15.83:0/561798581 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d7019a8b0 0x7f6d7019ad60 unknown :-1 s=CLOSED pgs=2229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:00.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.091+0000 7f6d75200640 1 --2- 172.21.15.83:0/561798581 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d70109580 0x7f6d701a0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:00.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.091+0000 7f6d75200640 1 --2- 172.21.15.83:0/561798581 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6d701044b0 0x7f6d701a0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:00.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.091+0000 7f6d75200640 1 -- 172.21.15.83:0/561798581 >> 172.21.15.83:0/561798581 conn(0x7f6d700ffd90 msgr2=0x7f6d70101110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:00.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.092+0000 7f6d75200640 1 -- 172.21.15.83:0/561798581 shutdown_connections 2024-04-24T09:07:00.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.092+0000 7f6d75200640 1 -- 172.21.15.83:0/561798581 wait complete. 2024-04-24T09:07:00.114 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T09:07:00.114 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T09:07:00.159 INFO:tasks.workunit.client.0.smithi083.stdout: 10 11861 8.90 MB/sec warmup 91 sec latency 67.548 ms 2024-04-24T09:07:00.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.363+0000 7fd657600640 1 -- 172.21.15.83:0/2311336269 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd650074bf0 msgr2=0x7fd65010e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:00.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.363+0000 7fd657600640 1 --2- 172.21.15.83:0/2311336269 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd650074bf0 0x7fd65010e3a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd644009860 tx=0x7fd64402a3f0 comp rx=0 tx=0).stop 2024-04-24T09:07:00.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.364+0000 7fd657600640 1 -- 172.21.15.83:0/2311336269 shutdown_connections 2024-04-24T09:07:00.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.364+0000 7fd657600640 1 --2- 172.21.15.83:0/2311336269 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd65010e8e0 0x7fd650110cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:00.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.364+0000 7fd657600640 1 --2- 172.21.15.83:0/2311336269 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd650074bf0 0x7fd65010e3a0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:00.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.364+0000 7fd657600640 1 --2- 172.21.15.83:0/2311336269 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6500742d0 0x7fd6500746b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:00.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.364+0000 7fd657600640 1 -- 172.21.15.83:0/2311336269 >> 172.21.15.83:0/2311336269 conn(0x7fd650079450 msgr2=0x7fd650079860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:00.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.364+0000 7fd657600640 1 -- 172.21.15.83:0/2311336269 shutdown_connections 2024-04-24T09:07:00.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.364+0000 7fd657600640 1 -- 172.21.15.83:0/2311336269 wait complete. 2024-04-24T09:07:00.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.366+0000 7fd657600640 1 Processor -- start 2024-04-24T09:07:00.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.366+0000 7fd657600640 1 -- start start 2024-04-24T09:07:00.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.367+0000 7fd657600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6500742d0 0x7fd65010e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:00.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.367+0000 7fd657600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd650074bf0 0x7fd65010b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:00.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.367+0000 7fd657600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd65010e8e0 0x7fd65010b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:00.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.367+0000 7fd657600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd6501144d0 con 0x7fd65010e8e0 2024-04-24T09:07:00.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.367+0000 7fd657600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd650114350 con 0x7fd6500742d0 2024-04-24T09:07:00.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.367+0000 7fd657600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd650114650 con 0x7fd650074bf0 2024-04-24T09:07:00.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.368+0000 7fd64fe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd650074bf0 0x7fd65010b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:00.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.368+0000 7fd64fe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd650074bf0 0x7fd65010b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:50620/0 (socket says 172.21.15.83:50620) 2024-04-24T09:07:00.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.368+0000 7fd64fe00640 1 -- 172.21.15.83:0/3613694796 learned_addr learned my addr 172.21.15.83:0/3613694796 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:07:00.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.368+0000 7fd654c00640 1 --2- 172.21.15.83:0/3613694796 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6500742d0 0x7fd65010e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:00.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.368+0000 7fd655600640 1 --2- 172.21.15.83:0/3613694796 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd65010e8e0 0x7fd65010b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:00.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.368+0000 7fd64fe00640 1 -- 172.21.15.83:0/3613694796 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6500742d0 msgr2=0x7fd65010e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:00.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.368+0000 7fd64fe00640 1 --2- 172.21.15.83:0/3613694796 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6500742d0 0x7fd65010e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:00.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.368+0000 7fd64fe00640 1 -- 172.21.15.83:0/3613694796 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd65010e8e0 msgr2=0x7fd65010b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:00.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.369+0000 7fd64fe00640 1 --2- 172.21.15.83:0/3613694796 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd65010e8e0 0x7fd65010b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:00.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.369+0000 7fd64fe00640 1 -- 172.21.15.83:0/3613694796 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6501ab210 con 0x7fd650074bf0 2024-04-24T09:07:00.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.369+0000 7fd654c00640 1 --2- 172.21.15.83:0/3613694796 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6500742d0 0x7fd65010e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:07:00.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.369+0000 7fd655600640 1 --2- 172.21.15.83:0/3613694796 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd65010e8e0 0x7fd65010b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:07:00.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.369+0000 7fd64fe00640 1 --2- 172.21.15.83:0/3613694796 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd650074bf0 0x7fd65010b390 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd644000c00 tx=0x7fd644030ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:00.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.370+0000 7fd64d600640 1 -- 172.21.15.83:0/3613694796 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd644040070 con 0x7fd650074bf0 2024-04-24T09:07:00.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.370+0000 7fd64d600640 1 -- 172.21.15.83:0/3613694796 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd6440055f0 con 0x7fd650074bf0 2024-04-24T09:07:00.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.370+0000 7fd64d600640 1 -- 172.21.15.83:0/3613694796 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6440046c0 con 0x7fd650074bf0 2024-04-24T09:07:00.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.370+0000 7fd657600640 1 -- 172.21.15.83:0/3613694796 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6501ab3e0 con 0x7fd650074bf0 2024-04-24T09:07:00.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.370+0000 7fd657600640 1 -- 172.21.15.83:0/3613694796 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6501ab7f0 con 0x7fd650074bf0 2024-04-24T09:07:00.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.375+0000 7fd65a56b740 1 Processor -- start 2024-04-24T09:07:00.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.375+0000 7fd65a56b740 1 -- start start 2024-04-24T09:07:00.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.376+0000 7fd65a56b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5578a5820860 0x5578a5820c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:00.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.376+0000 7fd65a56b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5578a5821180 0x5578a582b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:00.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.376+0000 7fd65a56b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5578a582b850 0x5578a582dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:00.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.376+0000 7fd65a56b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5578a57861f0 con 0x5578a582b850 2024-04-24T09:07:00.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.376+0000 7fd65a56b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5578a572c350 con 0x5578a5821180 2024-04-24T09:07:00.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.376+0000 7fd65a56b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5578a57821b0 con 0x5578a5820860 2024-04-24T09:07:00.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.376+0000 7fd64d600640 1 -- 172.21.15.83:0/3613694796 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd644043070 con 0x7fd650074bf0 2024-04-24T09:07:00.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.376+0000 7fd654c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5578a5821180 0x5578a582b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:00.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.377+0000 7fd654c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5578a5821180 0x5578a582b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:34272/0 (socket says 172.21.15.83:34272) 2024-04-24T09:07:00.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.377+0000 7fd654c00640 1 -- 172.21.15.83:0/213920872 learned_addr learned my addr 172.21.15.83:0/213920872 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:07:00.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.377+0000 7fd655600640 1 --2- 172.21.15.83:0/213920872 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5578a5820860 0x5578a5820c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:00.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.377+0000 7fd654c00640 1 -- 172.21.15.83:0/213920872 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5578a5820860 msgr2=0x5578a5820c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:00.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.377+0000 7fd654c00640 1 --2- 172.21.15.83:0/213920872 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5578a5820860 0x5578a5820c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:00.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.377+0000 7fd654c00640 1 -- 172.21.15.83:0/213920872 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5578a582b850 msgr2=0x5578a582dc40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T09:07:00.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.377+0000 7fd654c00640 1 --2- 172.21.15.83:0/213920872 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5578a582b850 0x5578a582dc40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:00.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.377+0000 7fd654c00640 1 -- 172.21.15.83:0/213920872 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578a5780f50 con 0x5578a5821180 2024-04-24T09:07:00.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.377+0000 7fd654c00640 1 --2- 172.21.15.83:0/213920872 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5578a5821180 0x5578a582b310 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd63c0104d0 tx=0x7fd63c010990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c4ff0ff976f08b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:00.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.378+0000 7fd641c00640 1 -- 172.21.15.83:0/213920872 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd63c011760 con 0x5578a5821180 2024-04-24T09:07:00.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.378+0000 7fd641c00640 1 -- 172.21.15.83:0/213920872 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd63c011db0 con 0x5578a5821180 2024-04-24T09:07:00.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.378+0000 7fd641c00640 1 -- 172.21.15.83:0/213920872 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd63c00e430 con 0x5578a5821180 2024-04-24T09:07:00.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.379+0000 7fd65a56b740 1 -- 172.21.15.83:0/213920872 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5578a5821180 msgr2=0x5578a582b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:00.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.379+0000 7fd65a56b740 1 --2- 172.21.15.83:0/213920872 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5578a5821180 0x5578a582b310 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd63c0104d0 tx=0x7fd63c010990 comp rx=0 tx=0).stop 2024-04-24T09:07:00.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.380+0000 7fd64d600640 1 --2- 172.21.15.83:0/3613694796 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fd6380a3e60 0x7fd6380a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:00.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.380+0000 7fd65a56b740 1 -- 172.21.15.83:0/213920872 shutdown_connections 2024-04-24T09:07:00.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.380+0000 7fd65a56b740 1 --2- 172.21.15.83:0/213920872 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5578a582b850 0x5578a582dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:00.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.380+0000 7fd65a56b740 1 --2- 172.21.15.83:0/213920872 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5578a5821180 0x5578a582b310 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:00.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.380+0000 7fd65a56b740 1 --2- 172.21.15.83:0/213920872 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5578a5820860 0x5578a5820c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:00.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.380+0000 7fd65a56b740 1 -- 172.21.15.83:0/213920872 >> 172.21.15.83:0/213920872 conn(0x5578a581cb80 msgr2=0x5578a581cf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:00.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.380+0000 7fd64d600640 1 -- 172.21.15.83:0/3613694796 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9583+0+0 (secure 0 0 0) 0x7fd6440f18b0 con 0x7fd650074bf0 2024-04-24T09:07:00.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.381+0000 7fd655600640 1 --2- 172.21.15.83:0/3613694796 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fd6380a3e60 0x7fd6380a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:00.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.381+0000 7fd65a56b740 1 -- 172.21.15.83:0/213920872 shutdown_connections 2024-04-24T09:07:00.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.382+0000 7fd65a56b740 1 -- 172.21.15.83:0/213920872 wait complete. 2024-04-24T09:07:00.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.382+0000 7fd655600640 1 --2- 172.21.15.83:0/3613694796 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fd6380a3e60 0x7fd6380a6320 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fd6480093b0 tx=0x7fd6480092c0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:00.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.382+0000 7fd65a56b740 1 -- start start 2024-04-24T09:07:00.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.383+0000 7fd65a56b740 1 Processor -- start 2024-04-24T09:07:00.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.383+0000 7fd65a56b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5578a5820860 0x5578a58287d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:00.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.383+0000 7fd65a56b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5578a5821180 0x5578a5828d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:00.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.383+0000 7fd65a56b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5578a582b850 0x5578a5829250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:00.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.383+0000 7fd65a56b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5578a582e640 con 0x5578a5821180 2024-04-24T09:07:00.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.383+0000 7fd65a56b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5578a573f690 con 0x5578a582b850 2024-04-24T09:07:00.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.383+0000 7fd65a56b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5578a582e7c0 con 0x5578a5820860 2024-04-24T09:07:00.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.383+0000 7fd65a56b740 10 client.0 _ll_register_callbacks cb 0x5578a577b650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:07:00.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.384+0000 7fd654c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5578a5820860 0x5578a58287d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:00.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.384+0000 7fd654c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5578a5820860 0x5578a58287d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:50632/0 (socket says 172.21.15.83:50632) 2024-04-24T09:07:00.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.384+0000 7fd654c00640 1 -- 172.21.15.83:0/1934006203 learned_addr learned my addr 172.21.15.83:0/1934006203 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:07:00.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.384+0000 7fd655600640 1 --2- 172.21.15.83:0/1934006203 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5578a582b850 0x5578a5829250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:00.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.384+0000 7fd64fe00640 1 --2- 172.21.15.83:0/1934006203 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5578a5821180 0x5578a5828d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:00.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.384+0000 7fd655600640 1 -- 172.21.15.83:0/1934006203 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5578a5820860 msgr2=0x5578a58287d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:00.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.384+0000 7fd655600640 1 --2- 172.21.15.83:0/1934006203 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5578a5820860 0x5578a58287d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:00.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.384+0000 7fd655600640 1 -- 172.21.15.83:0/1934006203 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5578a5821180 msgr2=0x5578a5828d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:00.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.384+0000 7fd655600640 1 --2- 172.21.15.83:0/1934006203 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5578a5821180 0x5578a5828d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:00.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.385+0000 7fd655600640 1 -- 172.21.15.83:0/1934006203 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578a573e920 con 0x5578a582b850 2024-04-24T09:07:00.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.385+0000 7fd655600640 1 --2- 172.21.15.83:0/1934006203 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5578a582b850 0x5578a5829250 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd65006b8d0 tx=0x7fd648004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=968cfa9da905072 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:00.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.385+0000 7fd636000640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:07:00.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.385+0000 7fd636000640 1 -- 172.21.15.83:0/1934006203 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd64802a060 con 0x5578a582b850 2024-04-24T09:07:00.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.385+0000 7fd636000640 1 -- 172.21.15.83:0/1934006203 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9583+0+0 (secure 0 0 0) 0x7fd648020030 con 0x5578a582b850 2024-04-24T09:07:00.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.386+0000 7fd65a56b740 1 -- 172.21.15.83:0/1934006203 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd63c011db0 con 0x5578a582b850 2024-04-24T09:07:00.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.386+0000 7fd636000640 1 -- 172.21.15.83:0/1934006203 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fd64802a060 con 0x5578a582b850 2024-04-24T09:07:00.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.386+0000 7fd65a56b740 10 client.38219 fetch_fsmap learned FSMap version 50 2024-04-24T09:07:00.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.386+0000 7fd65a56b740 1 -- 172.21.15.83:0/1934006203 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x5578a55627d0 con 0x5578a582b850 2024-04-24T09:07:00.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.387+0000 7fd636000640 1 -- 172.21.15.83:0/1934006203 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd63c011db0 con 0x5578a582b850 2024-04-24T09:07:00.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.387+0000 7fd636000640 1 -- 172.21.15.83:0/1934006203 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 50) ==== 8410+0+0 (secure 0 0 0) 0x7fd64800b040 con 0x5578a582b850 2024-04-24T09:07:00.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.387+0000 7fd65a56b740 10 client.38219 fetch_fsmap finished waiting for FSMap version 50 2024-04-24T09:07:00.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.387+0000 7fd65a56b740 10 client.38219 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:07:00.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.387+0000 7fd65a56b740 20 client.38219 populate_metadata read hostname 'smithi083' 2024-04-24T09:07:00.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.387+0000 7fd65a56b740 1 --2- 172.21.15.83:0/1934006203 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5578a583bc00 0x5578a583e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:00.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.387+0000 7fd65a56b740 4 client.38219 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:07:00.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.387+0000 7fd65a56b740 1 -- 172.21.15.83:0/1934006203 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5578a5793c10 con 0x5578a583bc00 2024-04-24T09:07:00.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.387+0000 7fd654c00640 1 --2- 172.21.15.83:0/1934006203 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5578a583bc00 0x5578a583e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:00.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.388+0000 7fd636000640 10 client.38219 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:07:00.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.388+0000 7fd654c00640 1 --2- 172.21.15.83:0/1934006203 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5578a583bc00 0x5578a583e010 crc :-1 s=READY pgs=395 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6be4b63cb95f59e7 server_cookie=7ea9284bac865423 in_seq=0 out_seq=0 2024-04-24T09:07:00.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.389+0000 7fd636000640 1 -- 172.21.15.83:0/1934006203 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5578a5793c10 con 0x5578a583bc00 2024-04-24T09:07:00.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.389+0000 7fd636000640 10 client.38219 handle_command_reply: tid=0 2024-04-24T09:07:00.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.389+0000 7fd65a56b740 1 client.38219 shutdown 2024-04-24T09:07:00.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.390+0000 7fd65a56b740 1 -- 172.21.15.83:0/1934006203 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5578a582b850 msgr2=0x5578a5829250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:00.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.390+0000 7fd65a56b740 1 --2- 172.21.15.83:0/1934006203 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5578a582b850 0x5578a5829250 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd65006b8d0 tx=0x7fd648004ab0 comp rx=0 tx=0).stop 2024-04-24T09:07:00.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.390+0000 7fd65a56b740 1 -- 172.21.15.83:0/1934006203 shutdown_connections 2024-04-24T09:07:00.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.390+0000 7fd65a56b740 1 --2- 172.21.15.83:0/1934006203 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5578a583bc00 0x5578a583e010 crc :-1 s=READY pgs=395 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:00.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.390+0000 7fd65a56b740 1 --2- 172.21.15.83:0/1934006203 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5578a582b850 0x5578a5829250 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:00.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.390+0000 7fd65a56b740 1 --2- 172.21.15.83:0/1934006203 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5578a5821180 0x5578a5828d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:00.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.390+0000 7fd65a56b740 1 --2- 172.21.15.83:0/1934006203 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5578a5820860 0x5578a58287d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:00.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.390+0000 7fd65a56b740 1 -- 172.21.15.83:0/1934006203 >> 172.21.15.83:0/1934006203 conn(0x5578a581cb80 msgr2=0x5578a5822980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:00.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.390+0000 7fd636000640 0 client.38219 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:07:00.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.390+0000 7fd65a56b740 1 -- 172.21.15.83:0/1934006203 shutdown_connections 2024-04-24T09:07:00.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.390+0000 7fd65a56b740 1 -- 172.21.15.83:0/1934006203 wait complete. 2024-04-24T09:07:00.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.390+0000 7fd65a56b740 20 client.38219 trim_cache size 0 max 16384 2024-04-24T09:07:00.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.420+0000 7fd65a56b740 1 Processor -- start 2024-04-24T09:07:00.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.420+0000 7fd65a56b740 1 -- start start 2024-04-24T09:07:00.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.420+0000 7fd65a56b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5578a5820860 0x5578a5837500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:00.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.420+0000 7fd65a56b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5578a5821180 0x5578a5843920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:00.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.420+0000 7fd65a56b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5578a582b850 0x5578a5843e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:00.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.420+0000 7fd65a56b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd6501be610 con 0x5578a5821180 2024-04-24T09:07:00.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.420+0000 7fd65a56b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd6501c02d0 con 0x5578a582b850 2024-04-24T09:07:00.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.420+0000 7fd65a56b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd6501bece0 con 0x5578a5820860 2024-04-24T09:07:00.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.421+0000 7fd64fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5578a5821180 0x5578a5843920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:00.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.421+0000 7fd655600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5578a582b850 0x5578a5843e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:00.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.421+0000 7fd655600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5578a582b850 0x5578a5843e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:34292/0 (socket says 172.21.15.83:34292) 2024-04-24T09:07:00.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.421+0000 7fd655600640 1 -- 172.21.15.83:0/1814139360 learned_addr learned my addr 172.21.15.83:0/1814139360 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:07:00.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.421+0000 7fd654c00640 1 --2- 172.21.15.83:0/1814139360 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5578a5820860 0x5578a5837500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:00.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.421+0000 7fd655600640 1 -- 172.21.15.83:0/1814139360 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5578a5820860 msgr2=0x5578a5837500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:00.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.421+0000 7fd655600640 1 --2- 172.21.15.83:0/1814139360 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5578a5820860 0x5578a5837500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:00.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.421+0000 7fd655600640 1 -- 172.21.15.83:0/1814139360 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5578a5821180 msgr2=0x5578a5843920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:00.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.421+0000 7fd655600640 1 --2- 172.21.15.83:0/1814139360 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5578a5821180 0x5578a5843920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:00.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.421+0000 7fd655600640 1 -- 172.21.15.83:0/1814139360 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578a54983f0 con 0x5578a582b850 2024-04-24T09:07:00.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.421+0000 7fd64fe00640 1 --2- 172.21.15.83:0/1814139360 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5578a5821180 0x5578a5843920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:07:00.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.422+0000 7fd655600640 1 --2- 172.21.15.83:0/1814139360 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5578a582b850 0x5578a5843e60 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd648009540 tx=0x7fd648025890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20c7dc9f3fabc7e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:00.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.422+0000 7fd641200640 1 -- 172.21.15.83:0/1814139360 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd648034080 con 0x5578a582b850 2024-04-24T09:07:00.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.422+0000 7fd641200640 1 -- 172.21.15.83:0/1814139360 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd648025990 con 0x5578a582b850 2024-04-24T09:07:00.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.422+0000 7fd641200640 1 -- 172.21.15.83:0/1814139360 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd648018ba0 con 0x5578a582b850 2024-04-24T09:07:00.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.423+0000 7fd65a56b740 1 -- 172.21.15.83:0/1814139360 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5578a582b850 msgr2=0x5578a5843e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:00.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.423+0000 7fd65a56b740 1 --2- 172.21.15.83:0/1814139360 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5578a582b850 0x5578a5843e60 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd648009540 tx=0x7fd648025890 comp rx=0 tx=0).stop 2024-04-24T09:07:00.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.423+0000 7fd65a56b740 1 -- 172.21.15.83:0/1814139360 shutdown_connections 2024-04-24T09:07:00.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.423+0000 7fd65a56b740 1 --2- 172.21.15.83:0/1814139360 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5578a582b850 0x5578a5843e60 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:00.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.423+0000 7fd65a56b740 1 --2- 172.21.15.83:0/1814139360 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5578a5821180 0x5578a5843920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:00.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.423+0000 7fd65a56b740 1 --2- 172.21.15.83:0/1814139360 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5578a5820860 0x5578a5837500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:00.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.423+0000 7fd65a56b740 1 -- 172.21.15.83:0/1814139360 >> 172.21.15.83:0/1814139360 conn(0x5578a581cb80 msgr2=0x5578a58270a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:00.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.423+0000 7fd65a56b740 1 -- 172.21.15.83:0/1814139360 shutdown_connections 2024-04-24T09:07:00.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.423+0000 7fd65a56b740 1 -- 172.21.15.83:0/1814139360 wait complete. 2024-04-24T09:07:00.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.424+0000 7fd65a56b740 1 -- start start 2024-04-24T09:07:00.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.424+0000 7fd65a56b740 1 Processor -- start 2024-04-24T09:07:00.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.424+0000 7fd65a56b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5578a5820860 0x5578a582ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:00.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.424+0000 7fd65a56b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5578a5821180 0x5578a582f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:00.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.424+0000 7fd65a56b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5578a582b850 0x5578a589a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:00.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.424+0000 7fd65a56b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd6501bfb40 con 0x5578a582b850 2024-04-24T09:07:00.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.424+0000 7fd65a56b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd6501bf410 con 0x5578a5821180 2024-04-24T09:07:00.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.424+0000 7fd65a56b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd6501a1c30 con 0x5578a5820860 2024-04-24T09:07:00.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.424+0000 7fd65a56b740 10 client.0 _ll_register_callbacks cb 0x7fd648009a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:07:00.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.425+0000 7fd655600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5578a582b850 0x5578a589a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:00.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.425+0000 7fd655600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5578a582b850 0x5578a589a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46388/0 (socket says 172.21.15.83:46388) 2024-04-24T09:07:00.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.425+0000 7fd655600640 1 -- 172.21.15.83:0/136232404 learned_addr learned my addr 172.21.15.83:0/136232404 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:07:00.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.425+0000 7fd64fe00640 1 --2- 172.21.15.83:0/136232404 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5578a5821180 0x5578a582f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:00.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.425+0000 7fd654c00640 1 --2- 172.21.15.83:0/136232404 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5578a5820860 0x5578a582ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:00.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.425+0000 7fd64fe00640 1 -- 172.21.15.83:0/136232404 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5578a5820860 msgr2=0x5578a582ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:00.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.425+0000 7fd64fe00640 1 --2- 172.21.15.83:0/136232404 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5578a5820860 0x5578a582ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:00.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.425+0000 7fd64fe00640 1 -- 172.21.15.83:0/136232404 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5578a582b850 msgr2=0x5578a589a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:00.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.425+0000 7fd64fe00640 1 --2- 172.21.15.83:0/136232404 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5578a582b850 0x5578a589a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:00.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.425+0000 7fd64fe00640 1 -- 172.21.15.83:0/136232404 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578a54580f0 con 0x5578a5821180 2024-04-24T09:07:00.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.425+0000 7fd655600640 1 --2- 172.21.15.83:0/136232404 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5578a582b850 0x5578a589a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:07:00.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.426+0000 7fd64fe00640 1 --2- 172.21.15.83:0/136232404 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5578a5821180 0x5578a582f1c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd6440eeb80 tx=0x7fd6440094e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e68a906f8c12a85 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:00.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.426+0000 7fd636000640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:07:00.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.426+0000 7fd636000640 1 -- 172.21.15.83:0/136232404 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6440c1780 con 0x5578a5821180 2024-04-24T09:07:00.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.426+0000 7fd636000640 1 -- 172.21.15.83:0/136232404 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9583+0+0 (secure 0 0 0) 0x7fd6440eee00 con 0x5578a5821180 2024-04-24T09:07:00.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.426+0000 7fd65a56b740 1 -- 172.21.15.83:0/136232404 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd648025990 con 0x5578a5821180 2024-04-24T09:07:00.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.426+0000 7fd636000640 1 -- 172.21.15.83:0/136232404 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fd6440ef000 con 0x5578a5821180 2024-04-24T09:07:00.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.426+0000 7fd65a56b740 10 client.38225 fetch_fsmap learned FSMap version 50 2024-04-24T09:07:00.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.426+0000 7fd65a56b740 1 -- 172.21.15.83:0/136232404 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x5578a574b810 con 0x5578a5821180 2024-04-24T09:07:00.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.427+0000 7fd636000640 1 -- 172.21.15.83:0/136232404 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd648025990 con 0x5578a5821180 2024-04-24T09:07:00.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.427+0000 7fd636000640 1 -- 172.21.15.83:0/136232404 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 50) ==== 8410+0+0 (secure 0 0 0) 0x7fd6440dc080 con 0x5578a5821180 2024-04-24T09:07:00.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.427+0000 7fd65a56b740 10 client.38225 fetch_fsmap finished waiting for FSMap version 50 2024-04-24T09:07:00.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.427+0000 7fd65a56b740 10 client.38225 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:07:00.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.427+0000 7fd65a56b740 20 client.38225 populate_metadata read hostname 'smithi083' 2024-04-24T09:07:00.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.427+0000 7fd65a56b740 1 --2- 172.21.15.83:0/136232404 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5578a58a2020 0x5578a58a4440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:00.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.427+0000 7fd65a56b740 4 client.38225 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T09:07:00.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.427+0000 7fd65a56b740 1 -- 172.21.15.83:0/136232404 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5578a578da80 con 0x5578a58a2020 2024-04-24T09:07:00.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.428+0000 7fd654c00640 1 --2- 172.21.15.83:0/136232404 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5578a58a2020 0x5578a58a4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:00.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.428+0000 7fd654c00640 1 --2- 172.21.15.83:0/136232404 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5578a58a2020 0x5578a58a4440 crc :-1 s=READY pgs=396 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4d8af04a835b216 server_cookie=99937b0fdc9d2bb3 in_seq=0 out_seq=0 2024-04-24T09:07:00.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.428+0000 7fd636000640 10 client.38225 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:07:00.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.430+0000 7fd636000640 1 -- 172.21.15.83:0/136232404 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5578a578da80 con 0x5578a58a2020 2024-04-24T09:07:00.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.430+0000 7fd636000640 10 client.38225 handle_command_reply: tid=0 2024-04-24T09:07:00.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.430+0000 7fd65a56b740 1 client.38225 shutdown 2024-04-24T09:07:00.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.431+0000 7fd65a56b740 1 -- 172.21.15.83:0/136232404 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5578a5821180 msgr2=0x5578a582f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:00.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.431+0000 7fd65a56b740 1 --2- 172.21.15.83:0/136232404 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5578a5821180 0x5578a582f1c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd6440eeb80 tx=0x7fd6440094e0 comp rx=0 tx=0).stop 2024-04-24T09:07:00.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.431+0000 7fd65a56b740 1 -- 172.21.15.83:0/136232404 shutdown_connections 2024-04-24T09:07:00.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.431+0000 7fd65a56b740 1 --2- 172.21.15.83:0/136232404 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5578a58a2020 0x5578a58a4440 crc :-1 s=READY pgs=396 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:00.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.431+0000 7fd65a56b740 1 --2- 172.21.15.83:0/136232404 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5578a582b850 0x5578a589a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:00.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.431+0000 7fd65a56b740 1 --2- 172.21.15.83:0/136232404 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5578a5821180 0x5578a582f1c0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:00.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.431+0000 7fd65a56b740 1 --2- 172.21.15.83:0/136232404 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5578a5820860 0x5578a582ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:00.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.431+0000 7fd636000640 0 client.38225 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:07:00.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.431+0000 7fd65a56b740 1 -- 172.21.15.83:0/136232404 >> 172.21.15.83:0/136232404 conn(0x5578a581cb80 msgr2=0x5578a581b390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:00.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.432+0000 7fd65a56b740 1 -- 172.21.15.83:0/136232404 shutdown_connections 2024-04-24T09:07:00.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.432+0000 7fd65a56b740 1 -- 172.21.15.83:0/136232404 wait complete. 2024-04-24T09:07:00.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.432+0000 7fd65a56b740 20 client.38225 trim_cache size 0 max 16384 2024-04-24T09:07:00.433 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T09:07:00.433 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T09:07:00.434 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-04-24T09:07:00.434 INFO:teuthology.orchestra.run.smithi083.stdout: "15186edf-ffec-4fed-a6d2-061e3019a571": { 2024-04-24T09:07:00.434 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-04-24T09:07:00.434 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "15186edf-ffec-4fed-a6d2-061e3019a571", 2024-04-24T09:07:00.434 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-04-24T09:07:00.434 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:07:00.434 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:07:00.434 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T09:07:00.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.434+0000 7fd657600640 1 -- 172.21.15.83:0/3613694796 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fd6380a3e60 msgr2=0x7fd6380a6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:00.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.434+0000 7fd657600640 1 --2- 172.21.15.83:0/3613694796 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fd6380a3e60 0x7fd6380a6320 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fd6480093b0 tx=0x7fd6480092c0 comp rx=0 tx=0).stop 2024-04-24T09:07:00.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.434+0000 7fd657600640 1 -- 172.21.15.83:0/3613694796 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd650074bf0 msgr2=0x7fd65010b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:00.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.434+0000 7fd657600640 1 --2- 172.21.15.83:0/3613694796 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd650074bf0 0x7fd65010b390 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd644000c00 tx=0x7fd644030ef0 comp rx=0 tx=0).stop 2024-04-24T09:07:00.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.434+0000 7fd657600640 1 -- 172.21.15.83:0/3613694796 shutdown_connections 2024-04-24T09:07:00.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.434+0000 7fd657600640 1 --2- 172.21.15.83:0/3613694796 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fd6380a3e60 0x7fd6380a6320 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:00.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.434+0000 7fd657600640 1 --2- 172.21.15.83:0/3613694796 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd65010e8e0 0x7fd65010b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:00.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.434+0000 7fd657600640 1 --2- 172.21.15.83:0/3613694796 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd650074bf0 0x7fd65010b390 secure :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd644000c00 tx=0x7fd644030ef0 comp rx=0 tx=0).stop 2024-04-24T09:07:00.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.434+0000 7fd657600640 1 --2- 172.21.15.83:0/3613694796 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6500742d0 0x7fd65010e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:00.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.434+0000 7fd657600640 1 -- 172.21.15.83:0/3613694796 >> 172.21.15.83:0/3613694796 conn(0x7fd650079450 msgr2=0x7fd6500ff010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:00.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.435+0000 7fd657600640 1 -- 172.21.15.83:0/3613694796 shutdown_connections 2024-04-24T09:07:00.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:00.435+0000 7fd657600640 1 -- 172.21.15.83:0/3613694796 wait complete. 2024-04-24T09:07:00.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:07:00 smithi138 ceph-mon[31441]: pgmap v482: 129 pgs: 129 active+clean; 368 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.7 MiB/s wr, 326 op/s 2024-04-24T09:07:00.455 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:07:00 smithi083 ceph-mon[27907]: pgmap v482: 129 pgs: 129 active+clean; 368 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.7 MiB/s wr, 326 op/s 2024-04-24T09:07:00.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:07:00 smithi189 ceph-mon[31791]: pgmap v482: 129 pgs: 129 active+clean; 368 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.7 MiB/s wr, 326 op/s 2024-04-24T09:07:00.458 INFO:tasks.cephfs.filesystem:scrub status for tag:15186edf-ffec-4fed-a6d2-061e3019a571 - {'path': '/', 'tag': '15186edf-ffec-4fed-a6d2-061e3019a571', 'options': 'recursive,force'} 2024-04-24T09:07:01.159 INFO:tasks.workunit.client.0.smithi083.stdout: 10 11982 8.84 MB/sec warmup 92 sec latency 190.224 ms 2024-04-24T09:07:01.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:07:01 smithi138 ceph-mon[31441]: scrub queued for path: / 2024-04-24T09:07:01.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:07:01 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:07:01.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:07:01 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:07:01.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:07:01 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:07:01.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:07:01 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:07:01.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:07:01 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:07:01.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:07:01 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:07:01.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:07:01 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:07:01.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:07:01 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:07:01.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:07:01 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:07:01.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:07:01 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:07:01.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:07:01 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:07:01.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:07:01 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:07:01.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:07:01 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:07:01.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:07:01 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:07:01.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:07:01 smithi138 ceph-mon[31441]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:07:01.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:07:01 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:07:01.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:07:01 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:07:01.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:07:01 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:07:01.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:07:01 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:07:01.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:07:01 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:07:01.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:07:01 smithi189 ceph-mon[31791]: scrub queued for path: / 2024-04-24T09:07:01.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:07:01 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:07:01.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:07:01 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:07:01.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:07:01 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:07:01.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:07:01 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:07:01.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:07:01 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:07:01.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:07:01 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:07:01.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:07:01 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:07:01.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:07:01 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:07:01.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:07:01 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:07:01.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:07:01 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:07:01.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:07:01 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:07:01.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:07:01 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:07:01.459 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:07:01 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:07:01.459 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:07:01 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:07:01.459 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:07:01 smithi189 ceph-mon[31791]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:07:01.459 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:07:01 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:07:01.459 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:07:01 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:07:01.459 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:07:01 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:07:01.459 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:07:01 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:07:01.459 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:07:01 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:07:01.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:07:01 smithi083 ceph-mon[27907]: scrub queued for path: / 2024-04-24T09:07:01.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:07:01 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:07:01.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:07:01 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:07:01.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:07:01 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:07:01.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:07:01 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:07:01.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:07:01 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:07:01.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:07:01 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:07:01.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:07:01 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:07:01.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:07:01 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:07:01.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:07:01 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:07:01.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:07:01 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:07:01.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:07:01 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:07:01.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:07:01 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:07:01.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:07:01 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:07:01.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:07:01 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:07:01.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:07:01 smithi083 ceph-mon[27907]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:07:01.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:07:01 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:07:01.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:07:01 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:07:01.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:07:01 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:07:01.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:07:01 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:07:01.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:07:01 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:07:02.159 INFO:tasks.workunit.client.0.smithi083.stdout: 10 12109 8.78 MB/sec warmup 93 sec latency 71.053 ms 2024-04-24T09:07:02.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:07:02 smithi138 ceph-mon[31441]: pgmap v483: 129 pgs: 129 active+clean; 368 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.4 MiB/s wr, 241 op/s 2024-04-24T09:07:02.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:07:02 smithi189 ceph-mon[31791]: pgmap v483: 129 pgs: 129 active+clean; 368 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.4 MiB/s wr, 241 op/s 2024-04-24T09:07:02.552 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:07:02 smithi083 ceph-mon[27907]: pgmap v483: 129 pgs: 129 active+clean; 368 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.4 MiB/s wr, 241 op/s 2024-04-24T09:07:03.159 INFO:tasks.workunit.client.0.smithi083.stdout: 10 12239 8.78 MB/sec warmup 94 sec latency 99.659 ms 2024-04-24T09:07:03.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:07:03 smithi138 ceph-mon[31441]: osdmap e113: 12 total, 12 up, 12 in 2024-04-24T09:07:03.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:07:03 smithi189 ceph-mon[31791]: osdmap e113: 12 total, 12 up, 12 in 2024-04-24T09:07:03.552 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:07:03 smithi083 ceph-mon[27907]: osdmap e113: 12 total, 12 up, 12 in 2024-04-24T09:07:04.160 INFO:tasks.workunit.client.0.smithi083.stdout: 10 12363 8.76 MB/sec warmup 95 sec latency 211.529 ms 2024-04-24T09:07:04.298 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.297+0000 7fcebac00640 1 -- 172.21.15.83:0/888840320 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fceb4076e50 msgr2=0x7fceb4112940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:04.298 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.297+0000 7fcebac00640 1 --2- 172.21.15.83:0/888840320 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fceb4076e50 0x7fceb4112940 secure :-1 s=READY pgs=2231 cs=0 l=1 rev1=1 crypto rx=0x7fcea0009900 tx=0x7fcea002a3f0 comp rx=0 tx=0).stop 2024-04-24T09:07:04.298 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.297+0000 7fcebac00640 1 -- 172.21.15.83:0/888840320 shutdown_connections 2024-04-24T09:07:04.298 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.297+0000 7fcebac00640 1 --2- 172.21.15.83:0/888840320 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fceb4112e80 0x7fceb4115270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:04.298 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.297+0000 7fcebac00640 1 --2- 172.21.15.83:0/888840320 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fceb4076e50 0x7fceb4112940 unknown :-1 s=CLOSED pgs=2231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:04.298 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.297+0000 7fcebac00640 1 --2- 172.21.15.83:0/888840320 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fceb4076530 0x7fceb4076910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:04.298 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.297+0000 7fcebac00640 1 -- 172.21.15.83:0/888840320 >> 172.21.15.83:0/888840320 conn(0x7fceb40ffcf0 msgr2=0x7fceb4102110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:04.298 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.298+0000 7fcebac00640 1 -- 172.21.15.83:0/888840320 shutdown_connections 2024-04-24T09:07:04.298 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.298+0000 7fcebac00640 1 -- 172.21.15.83:0/888840320 wait complete. 2024-04-24T09:07:04.300 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.298+0000 7fcebac00640 1 Processor -- start 2024-04-24T09:07:04.301 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.299+0000 7fcebac00640 1 -- start start 2024-04-24T09:07:04.301 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.299+0000 7fcebac00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fceb4076530 0x7fceb41a4a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:04.301 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.299+0000 7fcebac00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fceb4076e50 0x7fceb41a4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:04.301 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.299+0000 7fcebac00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fceb4112e80 0x7fceb419eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:04.301 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.299+0000 7fcebac00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fceb41188e0 con 0x7fceb4076530 2024-04-24T09:07:04.301 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.299+0000 7fcebac00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fceb4118760 con 0x7fceb4112e80 2024-04-24T09:07:04.301 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.299+0000 7fcebac00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fceb4118a60 con 0x7fceb4076e50 2024-04-24T09:07:04.301 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.299+0000 7fceb3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fceb4076530 0x7fceb41a4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:04.301 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.299+0000 7fceb3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fceb4076530 0x7fceb41a4a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46398/0 (socket says 172.21.15.83:46398) 2024-04-24T09:07:04.301 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.299+0000 7fceb3e00640 1 -- 172.21.15.83:0/2130396820 learned_addr learned my addr 172.21.15.83:0/2130396820 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:07:04.301 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.300+0000 7fceb8c00640 1 --2- 172.21.15.83:0/2130396820 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fceb4112e80 0x7fceb419eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:04.301 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.300+0000 7fceb3400640 1 --2- 172.21.15.83:0/2130396820 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fceb4076e50 0x7fceb41a4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:04.301 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.300+0000 7fceb3e00640 1 -- 172.21.15.83:0/2130396820 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fceb4076e50 msgr2=0x7fceb41a4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:04.302 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.300+0000 7fceb3e00640 1 --2- 172.21.15.83:0/2130396820 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fceb4076e50 0x7fceb41a4fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:04.302 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.300+0000 7fceb3e00640 1 -- 172.21.15.83:0/2130396820 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fceb4112e80 msgr2=0x7fceb419eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:04.302 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.300+0000 7fceb3e00640 1 --2- 172.21.15.83:0/2130396820 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fceb4112e80 0x7fceb419eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:04.302 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.300+0000 7fceb3e00640 1 -- 172.21.15.83:0/2130396820 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fceb419f3e0 con 0x7fceb4076530 2024-04-24T09:07:04.302 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.300+0000 7fceb8c00640 1 --2- 172.21.15.83:0/2130396820 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fceb4112e80 0x7fceb419eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:07:04.302 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.300+0000 7fceb3e00640 1 --2- 172.21.15.83:0/2130396820 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fceb4076530 0x7fceb41a4a60 secure :-1 s=READY pgs=2232 cs=0 l=1 rev1=1 crypto rx=0x7fceac00cce0 tx=0x7fceac00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:04.302 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.300+0000 7fceb0c00640 1 -- 172.21.15.83:0/2130396820 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceac00f650 con 0x7fceb4076530 2024-04-24T09:07:04.302 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.300+0000 7fcebac00640 1 -- 172.21.15.83:0/2130396820 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fceb419f6d0 con 0x7fceb4076530 2024-04-24T09:07:04.302 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.300+0000 7fceb0c00640 1 -- 172.21.15.83:0/2130396820 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fceac00fca0 con 0x7fceb4076530 2024-04-24T09:07:04.302 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.300+0000 7fceb0c00640 1 -- 172.21.15.83:0/2130396820 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceac004d20 con 0x7fceb4076530 2024-04-24T09:07:04.302 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.300+0000 7fcebac00640 1 -- 172.21.15.83:0/2130396820 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fceb41ab9e0 con 0x7fceb4076530 2024-04-24T09:07:04.303 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.303+0000 7fceb0c00640 1 -- 172.21.15.83:0/2130396820 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fceac00f7f0 con 0x7fceb4076530 2024-04-24T09:07:04.304 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.304+0000 7fcebd99f740 1 Processor -- start 2024-04-24T09:07:04.304 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.304+0000 7fcebd99f740 1 -- start start 2024-04-24T09:07:04.305 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.304+0000 7fcebd99f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558443ffe1b0 0x558443ffe590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:04.305 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.304+0000 7fcebd99f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558443ffead0 0x558444008e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:04.306 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.304+0000 7fcebd99f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558444009390 0x55844400b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:04.306 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.304+0000 7fcebd99f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x558443f64900 con 0x558443ffead0 2024-04-24T09:07:04.306 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.304+0000 7fcebd99f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558443f0af20 con 0x558443ffe1b0 2024-04-24T09:07:04.306 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.304+0000 7fcebd99f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x558443f608c0 con 0x558444009390 2024-04-24T09:07:04.306 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.304+0000 7fceb3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558443ffead0 0x558444008e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:04.306 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.304+0000 7fceb3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558443ffead0 0x558444008e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46402/0 (socket says 172.21.15.83:46402) 2024-04-24T09:07:04.306 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.304+0000 7fceb3e00640 1 -- 172.21.15.83:0/38598135 learned_addr learned my addr 172.21.15.83:0/38598135 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:07:04.306 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.305+0000 7fceb8c00640 1 --2- 172.21.15.83:0/38598135 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558443ffe1b0 0x558443ffe590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:04.306 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.305+0000 7fceb3400640 1 --2- 172.21.15.83:0/38598135 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558444009390 0x55844400b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:04.306 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.305+0000 7fceb3e00640 1 -- 172.21.15.83:0/38598135 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558444009390 msgr2=0x55844400b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:04.306 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.305+0000 7fceb3e00640 1 --2- 172.21.15.83:0/38598135 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558444009390 0x55844400b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:04.306 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.305+0000 7fceb3e00640 1 -- 172.21.15.83:0/38598135 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558443ffe1b0 msgr2=0x558443ffe590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:04.306 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.305+0000 7fceb3e00640 1 --2- 172.21.15.83:0/38598135 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558443ffe1b0 0x558443ffe590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:04.307 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.305+0000 7fceb3e00640 1 -- 172.21.15.83:0/38598135 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558443f2fe10 con 0x558443ffead0 2024-04-24T09:07:04.307 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.305+0000 7fceb0c00640 1 --2- 172.21.15.83:0/2130396820 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fce980a3dc0 0x7fce980a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:04.307 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.305+0000 7fceb0c00640 1 -- 172.21.15.83:0/2130396820 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9671+0+0 (secure 0 0 0) 0x7fceac0d1250 con 0x7fceb4076530 2024-04-24T09:07:04.307 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.305+0000 7fceb8c00640 1 --2- 172.21.15.83:0/2130396820 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fce980a3dc0 0x7fce980a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:04.307 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.305+0000 7fceb3e00640 1 --2- 172.21.15.83:0/38598135 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558443ffead0 0x558444008e50 secure :-1 s=READY pgs=2233 cs=0 l=1 rev1=1 crypto rx=0x7fceac0ce400 tx=0x7fceac0d95c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0c9278b696547e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:04.307 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.306+0000 7fcea9200640 1 -- 172.21.15.83:0/38598135 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceac09cae0 con 0x558443ffead0 2024-04-24T09:07:04.307 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.306+0000 7fcea9200640 1 -- 172.21.15.83:0/38598135 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fceac00fae0 con 0x558443ffead0 2024-04-24T09:07:04.307 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.306+0000 7fcea9200640 1 -- 172.21.15.83:0/38598135 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceac09c670 con 0x558443ffead0 2024-04-24T09:07:04.310 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.306+0000 7fcebd99f740 1 -- 172.21.15.83:0/38598135 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558443ffead0 msgr2=0x558444008e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:04.310 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.306+0000 7fceb8c00640 1 --2- 172.21.15.83:0/2130396820 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fce980a3dc0 0x7fce980a6280 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fceb406b4e0 tx=0x7fcea400c5e0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:04.310 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.306+0000 7fcebd99f740 1 --2- 172.21.15.83:0/38598135 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558443ffead0 0x558444008e50 secure :-1 s=READY pgs=2233 cs=0 l=1 rev1=1 crypto rx=0x7fceac0ce400 tx=0x7fceac0d95c0 comp rx=0 tx=0).stop 2024-04-24T09:07:04.310 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.306+0000 7fcebd99f740 1 -- 172.21.15.83:0/38598135 shutdown_connections 2024-04-24T09:07:04.310 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.306+0000 7fcebd99f740 1 --2- 172.21.15.83:0/38598135 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558444009390 0x55844400b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:04.311 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.306+0000 7fcebd99f740 1 --2- 172.21.15.83:0/38598135 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558443ffead0 0x558444008e50 unknown :-1 s=CLOSED pgs=2233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:04.311 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.306+0000 7fcebd99f740 1 --2- 172.21.15.83:0/38598135 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558443ffe1b0 0x558443ffe590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:04.311 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.306+0000 7fcebd99f740 1 -- 172.21.15.83:0/38598135 >> 172.21.15.83:0/38598135 conn(0x558443ffa6c0 msgr2=0x558443ffaaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:04.311 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.306+0000 7fcebd99f740 1 -- 172.21.15.83:0/38598135 shutdown_connections 2024-04-24T09:07:04.311 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.307+0000 7fcebd99f740 1 -- 172.21.15.83:0/38598135 wait complete. 2024-04-24T09:07:04.311 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.307+0000 7fcebd99f740 1 -- start start 2024-04-24T09:07:04.311 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.307+0000 7fcebd99f740 1 Processor -- start 2024-04-24T09:07:04.311 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.307+0000 7fcebd99f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558443ffe1b0 0x5584440062f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:04.311 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.308+0000 7fcebd99f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558443ffead0 0x558444006830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:04.311 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.308+0000 7fcebd99f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558444009390 0x558444006d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:04.311 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.308+0000 7fcebd99f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55844400bfc0 con 0x558443ffe1b0 2024-04-24T09:07:04.311 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.308+0000 7fcebd99f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558443f1e2c0 con 0x558443ffead0 2024-04-24T09:07:04.311 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.308+0000 7fcebd99f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55844400c140 con 0x558444009390 2024-04-24T09:07:04.312 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.308+0000 7fcebd99f740 10 client.0 _ll_register_callbacks cb 0x558443c76a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:07:04.312 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.308+0000 7fceb3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558443ffe1b0 0x5584440062f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:04.312 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.308+0000 7fceb3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558443ffe1b0 0x5584440062f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46404/0 (socket says 172.21.15.83:46404) 2024-04-24T09:07:04.312 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.308+0000 7fceb3e00640 1 -- 172.21.15.83:0/3600791423 learned_addr learned my addr 172.21.15.83:0/3600791423 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:07:04.312 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.308+0000 7fceb3400640 1 --2- 172.21.15.83:0/3600791423 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558443ffead0 0x558444006830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:04.312 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.308+0000 7fceb3e00640 1 -- 172.21.15.83:0/3600791423 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558444009390 msgr2=0x558444006d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:04.312 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.308+0000 7fceb3e00640 1 --2- 172.21.15.83:0/3600791423 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558444009390 0x558444006d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:04.312 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.308+0000 7fceb3e00640 1 -- 172.21.15.83:0/3600791423 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558443ffead0 msgr2=0x558444006830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:04.312 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.308+0000 7fceb3e00640 1 --2- 172.21.15.83:0/3600791423 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558443ffead0 0x558444006830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:04.312 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.308+0000 7fceb3e00640 1 -- 172.21.15.83:0/3600791423 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558443f2efd0 con 0x558443ffe1b0 2024-04-24T09:07:04.312 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.308+0000 7fceb3e00640 1 --2- 172.21.15.83:0/3600791423 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558443ffe1b0 0x5584440062f0 secure :-1 s=READY pgs=2234 cs=0 l=1 rev1=1 crypto rx=0x7fceac004af0 tx=0x7fceac0d4f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9573002b0c567245 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:04.312 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.308+0000 7fce9d600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:07:04.312 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.309+0000 7fce9d600640 1 -- 172.21.15.83:0/3600791423 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceac09c850 con 0x558443ffe1b0 2024-04-24T09:07:04.312 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.309+0000 7fce9d600640 1 -- 172.21.15.83:0/3600791423 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9671+0+0 (secure 0 0 0) 0x7fceac0bd110 con 0x558443ffe1b0 2024-04-24T09:07:04.312 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.309+0000 7fcebd99f740 1 -- 172.21.15.83:0/3600791423 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558443f72320 con 0x558443ffe1b0 2024-04-24T09:07:04.313 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.309+0000 7fce9d600640 1 -- 172.21.15.83:0/3600791423 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fceac09c670 con 0x558443ffe1b0 2024-04-24T09:07:04.313 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.309+0000 7fcebd99f740 10 client.21084 fetch_fsmap learned FSMap version 50 2024-04-24T09:07:04.313 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.309+0000 7fcebd99f740 1 -- 172.21.15.83:0/3600791423 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x558443c77210 con 0x558443ffe1b0 2024-04-24T09:07:04.313 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.309+0000 7fce9d600640 1 -- 172.21.15.83:0/3600791423 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x558443f72320 con 0x558443ffe1b0 2024-04-24T09:07:04.313 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.309+0000 7fce9d600640 1 -- 172.21.15.83:0/3600791423 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 50) ==== 8410+0+0 (secure 0 0 0) 0x7fceac09cf70 con 0x558443ffe1b0 2024-04-24T09:07:04.313 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.309+0000 7fcebd99f740 10 client.21084 fetch_fsmap finished waiting for FSMap version 50 2024-04-24T09:07:04.313 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.309+0000 7fcebd99f740 10 client.21084 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:07:04.313 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.309+0000 7fcebd99f740 20 client.21084 populate_metadata read hostname 'smithi083' 2024-04-24T09:07:04.313 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.310+0000 7fcebd99f740 1 --2- 172.21.15.83:0/3600791423 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x558444019320 0x55844401b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:04.313 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.310+0000 7fcebd99f740 4 client.21084 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:07:04.313 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.310+0000 7fcebd99f740 1 -- 172.21.15.83:0/3600791423 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558443f6c190 con 0x558444019320 2024-04-24T09:07:04.381 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.381+0000 7fceb3400640 1 --2- 172.21.15.83:0/3600791423 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x558444019320 0x55844401b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:04.385 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.385+0000 7fceb3400640 1 --2- 172.21.15.83:0/3600791423 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x558444019320 0x55844401b710 crc :-1 s=READY pgs=397 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9670a6b221dfab4f server_cookie=c3afea0d15a1a7f9 in_seq=0 out_seq=0 2024-04-24T09:07:04.385 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.385+0000 7fce9d600640 10 client.21084 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:07:04.390 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.390+0000 7fce9d600640 1 -- 172.21.15.83:0/3600791423 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558443f6c190 con 0x558444019320 2024-04-24T09:07:04.390 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.390+0000 7fce9d600640 10 client.21084 handle_command_reply: tid=0 2024-04-24T09:07:04.391 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.390+0000 7fcebd99f740 1 client.21084 shutdown 2024-04-24T09:07:04.391 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.391+0000 7fcebd99f740 1 -- 172.21.15.83:0/3600791423 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558443ffe1b0 msgr2=0x5584440062f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:04.391 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.391+0000 7fcebd99f740 1 --2- 172.21.15.83:0/3600791423 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558443ffe1b0 0x5584440062f0 secure :-1 s=READY pgs=2234 cs=0 l=1 rev1=1 crypto rx=0x7fceac004af0 tx=0x7fceac0d4f60 comp rx=0 tx=0).stop 2024-04-24T09:07:04.392 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.392+0000 7fcebd99f740 1 -- 172.21.15.83:0/3600791423 shutdown_connections 2024-04-24T09:07:04.392 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.392+0000 7fcebd99f740 1 --2- 172.21.15.83:0/3600791423 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x558444019320 0x55844401b710 crc :-1 s=READY pgs=397 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:04.392 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.392+0000 7fcebd99f740 1 --2- 172.21.15.83:0/3600791423 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558444009390 0x558444006d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:04.392 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.392+0000 7fcebd99f740 1 --2- 172.21.15.83:0/3600791423 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558443ffead0 0x558444006830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:04.393 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.392+0000 7fcebd99f740 1 --2- 172.21.15.83:0/3600791423 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558443ffe1b0 0x5584440062f0 unknown :-1 s=CLOSED pgs=2234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:04.393 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.392+0000 7fcebd99f740 1 -- 172.21.15.83:0/3600791423 >> 172.21.15.83:0/3600791423 conn(0x558443ffa6c0 msgr2=0x558444000440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:04.393 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.392+0000 7fce9d600640 0 client.21084 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:07:04.393 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.393+0000 7fcebd99f740 1 -- 172.21.15.83:0/3600791423 shutdown_connections 2024-04-24T09:07:04.393 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.393+0000 7fcebd99f740 1 -- 172.21.15.83:0/3600791423 wait complete. 2024-04-24T09:07:04.394 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.393+0000 7fcebd99f740 20 client.21084 trim_cache size 0 max 16384 2024-04-24T09:07:04.434 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.434+0000 7fcebd99f740 1 Processor -- start 2024-04-24T09:07:04.434 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.434+0000 7fcebd99f740 1 -- start start 2024-04-24T09:07:04.435 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.434+0000 7fcebd99f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558443ffe1b0 0x558444014e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:04.435 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.434+0000 7fcebd99f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558443ffead0 0x558444000b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:04.435 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.434+0000 7fcebd99f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558444009390 0x558444020cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:04.435 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.434+0000 7fcebd99f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fceb41b5c50 con 0x558443ffe1b0 2024-04-24T09:07:04.435 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.434+0000 7fcebd99f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fceb41b7910 con 0x558444009390 2024-04-24T09:07:04.435 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.434+0000 7fcebd99f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fceb41b6320 con 0x558443ffead0 2024-04-24T09:07:04.435 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.435+0000 7fceb8c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558444009390 0x558444020cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:04.435 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.435+0000 7fceb8c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558444009390 0x558444020cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:34354/0 (socket says 172.21.15.83:34354) 2024-04-24T09:07:04.435 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.435+0000 7fceb8c00640 1 -- 172.21.15.83:0/429009221 learned_addr learned my addr 172.21.15.83:0/429009221 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:07:04.436 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.435+0000 7fceb3400640 1 --2- 172.21.15.83:0/429009221 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558443ffead0 0x558444000b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:04.436 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.436+0000 7fceb3e00640 1 --2- 172.21.15.83:0/429009221 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558443ffe1b0 0x558444014e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:04.436 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.436+0000 7fceb8c00640 1 -- 172.21.15.83:0/429009221 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558443ffead0 msgr2=0x558444000b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:04.436 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.436+0000 7fceb8c00640 1 --2- 172.21.15.83:0/429009221 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558443ffead0 0x558444000b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:04.437 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.436+0000 7fceb8c00640 1 -- 172.21.15.83:0/429009221 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558443ffe1b0 msgr2=0x558444014e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:04.437 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.436+0000 7fceb8c00640 1 --2- 172.21.15.83:0/429009221 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558443ffe1b0 0x558444014e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:04.437 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.436+0000 7fceb8c00640 1 -- 172.21.15.83:0/429009221 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55844401ba80 con 0x558444009390 2024-04-24T09:07:04.437 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.437+0000 7fceb3e00640 1 --2- 172.21.15.83:0/429009221 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558443ffe1b0 0x558444014e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:07:04.437 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.437+0000 7fceb3400640 1 --2- 172.21.15.83:0/429009221 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558443ffead0 0x558444000b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:07:04.438 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.437+0000 7fceb8c00640 1 --2- 172.21.15.83:0/429009221 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558444009390 0x558444020cb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fcea4005ec0 tx=0x7fcea40046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9bbea93e95ba9483 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:04.438 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.437+0000 7fce9fe00640 1 -- 172.21.15.83:0/429009221 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea4023070 con 0x558444009390 2024-04-24T09:07:04.438 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.438+0000 7fce9fe00640 1 -- 172.21.15.83:0/429009221 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcea401f900 con 0x558444009390 2024-04-24T09:07:04.438 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.438+0000 7fce9fe00640 1 -- 172.21.15.83:0/429009221 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea4011810 con 0x558444009390 2024-04-24T09:07:04.438 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.438+0000 7fcebd99f740 1 -- 172.21.15.83:0/429009221 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558444009390 msgr2=0x558444020cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:04.438 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.438+0000 7fcebd99f740 1 --2- 172.21.15.83:0/429009221 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558444009390 0x558444020cb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fcea4005ec0 tx=0x7fcea40046c0 comp rx=0 tx=0).stop 2024-04-24T09:07:04.439 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.439+0000 7fcebd99f740 1 -- 172.21.15.83:0/429009221 shutdown_connections 2024-04-24T09:07:04.439 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.439+0000 7fcebd99f740 1 --2- 172.21.15.83:0/429009221 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558444009390 0x558444020cb0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:04.439 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.439+0000 7fcebd99f740 1 --2- 172.21.15.83:0/429009221 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558443ffead0 0x558444000b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:04.439 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.439+0000 7fcebd99f740 1 --2- 172.21.15.83:0/429009221 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558443ffe1b0 0x558444014e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:04.439 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.439+0000 7fcebd99f740 1 -- 172.21.15.83:0/429009221 >> 172.21.15.83:0/429009221 conn(0x558443ffa6c0 msgr2=0x558443ff9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:04.440 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.439+0000 7fcebd99f740 1 -- 172.21.15.83:0/429009221 shutdown_connections 2024-04-24T09:07:04.440 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.439+0000 7fcebd99f740 1 -- 172.21.15.83:0/429009221 wait complete. 2024-04-24T09:07:04.440 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.439+0000 7fcebd99f740 1 -- start start 2024-04-24T09:07:04.440 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.440+0000 7fcebd99f740 1 Processor -- start 2024-04-24T09:07:04.440 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.440+0000 7fcebd99f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558443ffe1b0 0x558444011390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:04.440 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.440+0000 7fcebd99f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558443ffead0 0x5584440118d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:04.441 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.440+0000 7fcebd99f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558444009390 0x55844400e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:04.441 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.441+0000 7fcebd99f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fceb41b7180 con 0x558444009390 2024-04-24T09:07:04.441 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.441+0000 7fcebd99f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fceb41b6a50 con 0x558443ffe1b0 2024-04-24T09:07:04.441 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.441+0000 7fcebd99f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fceb41b3c50 con 0x558443ffead0 2024-04-24T09:07:04.442 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.441+0000 7fcebd99f740 10 client.0 _ll_register_callbacks cb 0x7fceac0d9380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:07:04.442 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.441+0000 7fceb3e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558443ffe1b0 0x558444011390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:04.442 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.441+0000 7fceb3e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558443ffe1b0 0x558444011390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:34362/0 (socket says 172.21.15.83:34362) 2024-04-24T09:07:04.442 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.441+0000 7fceb3e00640 1 -- 172.21.15.83:0/2415303382 learned_addr learned my addr 172.21.15.83:0/2415303382 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:07:04.442 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.441+0000 7fceb8c00640 1 --2- 172.21.15.83:0/2415303382 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558444009390 0x55844400e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:04.442 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.441+0000 7fceb3400640 1 --2- 172.21.15.83:0/2415303382 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558443ffead0 0x5584440118d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:04.443 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.442+0000 7fceb3e00640 1 -- 172.21.15.83:0/2415303382 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558443ffead0 msgr2=0x5584440118d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:04.443 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.442+0000 7fceb3e00640 1 --2- 172.21.15.83:0/2415303382 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558443ffead0 0x5584440118d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:04.443 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.442+0000 7fceb3e00640 1 -- 172.21.15.83:0/2415303382 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558444009390 msgr2=0x55844400e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:04.443 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.442+0000 7fceb3e00640 1 --2- 172.21.15.83:0/2415303382 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558444009390 0x55844400e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:04.443 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.442+0000 7fceb3e00640 1 -- 172.21.15.83:0/2415303382 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558443d43620 con 0x558443ffe1b0 2024-04-24T09:07:04.443 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.442+0000 7fceb8c00640 1 --2- 172.21.15.83:0/2415303382 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558444009390 0x55844400e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:07:04.443 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.442+0000 7fceb3400640 1 --2- 172.21.15.83:0/2415303382 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558443ffead0 0x5584440118d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:07:04.443 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.442+0000 7fceb3e00640 1 --2- 172.21.15.83:0/2415303382 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558443ffe1b0 0x558444011390 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fceac0a5450 tx=0x7fceac018510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27cfe63264e50e3c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:04.443 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.442+0000 7fce9d600640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:07:04.443 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.442+0000 7fce9d600640 1 -- 172.21.15.83:0/2415303382 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceac0ce950 con 0x558443ffe1b0 2024-04-24T09:07:04.444 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.442+0000 7fce9d600640 1 -- 172.21.15.83:0/2415303382 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9671+0+0 (secure 0 0 0) 0x7fceac0bd310 con 0x558443ffe1b0 2024-04-24T09:07:04.444 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.442+0000 7fcebd99f740 1 -- 172.21.15.83:0/2415303382 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcea401f900 con 0x558443ffe1b0 2024-04-24T09:07:04.444 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.443+0000 7fce9d600640 1 -- 172.21.15.83:0/2415303382 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fceac09cae0 con 0x558443ffe1b0 2024-04-24T09:07:04.444 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.443+0000 7fcebd99f740 10 client.38237 fetch_fsmap learned FSMap version 50 2024-04-24T09:07:04.444 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.443+0000 7fcebd99f740 1 -- 172.21.15.83:0/2415303382 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55844407fb60 con 0x558443ffe1b0 2024-04-24T09:07:04.444 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.443+0000 7fce9d600640 1 -- 172.21.15.83:0/2415303382 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcea401f900 con 0x558443ffe1b0 2024-04-24T09:07:04.444 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.444+0000 7fce9d600640 1 -- 172.21.15.83:0/2415303382 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 50) ==== 8410+0+0 (secure 0 0 0) 0x7fceac0a54f0 con 0x558443ffe1b0 2024-04-24T09:07:04.445 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.444+0000 7fcebd99f740 10 client.38237 fetch_fsmap finished waiting for FSMap version 50 2024-04-24T09:07:04.445 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.444+0000 7fcebd99f740 10 client.38237 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:07:04.445 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.444+0000 7fcebd99f740 20 client.38237 populate_metadata read hostname 'smithi083' 2024-04-24T09:07:04.445 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.444+0000 7fcebd99f740 1 --2- 172.21.15.83:0/2415303382 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x558444080170 0x558444082590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:04.445 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.444+0000 7fcebd99f740 4 client.38237 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T09:07:04.445 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.444+0000 7fcebd99f740 1 -- 172.21.15.83:0/2415303382 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558443ee84b0 con 0x558444080170 2024-04-24T09:07:04.445 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.445+0000 7fceb3400640 1 --2- 172.21.15.83:0/2415303382 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x558444080170 0x558444082590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:04.446 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.446+0000 7fceb3400640 1 --2- 172.21.15.83:0/2415303382 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x558444080170 0x558444082590 crc :-1 s=READY pgs=398 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=992a7faa059f4dcb server_cookie=1d5b316126a18a73 in_seq=0 out_seq=0 2024-04-24T09:07:04.446 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.446+0000 7fce9d600640 10 client.38237 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:07:04.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:07:04 smithi138 ceph-mon[31441]: pgmap v485: 129 pgs: 129 active+clean; 371 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.2 MiB/s wr, 328 op/s 2024-04-24T09:07:04.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:07:04 smithi189 ceph-mon[31791]: pgmap v485: 129 pgs: 129 active+clean; 371 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.2 MiB/s wr, 328 op/s 2024-04-24T09:07:04.552 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:07:04 smithi083 ceph-mon[27907]: pgmap v485: 129 pgs: 129 active+clean; 371 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.2 MiB/s wr, 328 op/s 2024-04-24T09:07:04.574 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.574+0000 7fce9d600640 1 -- 172.21.15.83:0/2415303382 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558443ee84b0 con 0x558444080170 2024-04-24T09:07:04.574 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.574+0000 7fce9d600640 10 client.38237 handle_command_reply: tid=0 2024-04-24T09:07:04.574 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.574+0000 7fcebd99f740 1 client.38237 shutdown 2024-04-24T09:07:04.575 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.575+0000 7fcebd99f740 1 -- 172.21.15.83:0/2415303382 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558443ffe1b0 msgr2=0x558444011390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:04.575 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.575+0000 7fcebd99f740 1 --2- 172.21.15.83:0/2415303382 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558443ffe1b0 0x558444011390 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fceac0a5450 tx=0x7fceac018510 comp rx=0 tx=0).stop 2024-04-24T09:07:04.575 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.575+0000 7fcebd99f740 1 -- 172.21.15.83:0/2415303382 shutdown_connections 2024-04-24T09:07:04.576 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.575+0000 7fcebd99f740 1 --2- 172.21.15.83:0/2415303382 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x558444080170 0x558444082590 crc :-1 s=READY pgs=398 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:04.576 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.575+0000 7fcebd99f740 1 --2- 172.21.15.83:0/2415303382 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558444009390 0x55844400e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:04.576 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.575+0000 7fcebd99f740 1 --2- 172.21.15.83:0/2415303382 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558443ffead0 0x5584440118d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:04.576 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.575+0000 7fce9d600640 0 client.38237 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:07:04.576 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.575+0000 7fcebd99f740 1 --2- 172.21.15.83:0/2415303382 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558443ffe1b0 0x558444011390 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:04.576 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.575+0000 7fcebd99f740 1 -- 172.21.15.83:0/2415303382 >> 172.21.15.83:0/2415303382 conn(0x558443ffa6c0 msgr2=0x558444006290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:04.576 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.575+0000 7fcebd99f740 1 -- 172.21.15.83:0/2415303382 shutdown_connections 2024-04-24T09:07:04.576 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.575+0000 7fcebd99f740 1 -- 172.21.15.83:0/2415303382 wait complete. 2024-04-24T09:07:04.576 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.576+0000 7fcebd99f740 20 client.38237 trim_cache size 0 max 16384 2024-04-24T09:07:04.577 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T09:07:04.577 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T09:07:04.577 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T09:07:04.577 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T09:07:04.577 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.577+0000 7fcebac00640 1 -- 172.21.15.83:0/2130396820 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fce980a3dc0 msgr2=0x7fce980a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:04.577 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.577+0000 7fcebac00640 1 --2- 172.21.15.83:0/2130396820 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fce980a3dc0 0x7fce980a6280 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fceb406b4e0 tx=0x7fcea400c5e0 comp rx=0 tx=0).stop 2024-04-24T09:07:04.578 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.577+0000 7fcebac00640 1 -- 172.21.15.83:0/2130396820 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fceb4076530 msgr2=0x7fceb41a4a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:04.578 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.577+0000 7fcebac00640 1 --2- 172.21.15.83:0/2130396820 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fceb4076530 0x7fceb41a4a60 secure :-1 s=READY pgs=2232 cs=0 l=1 rev1=1 crypto rx=0x7fceac00cce0 tx=0x7fceac00a620 comp rx=0 tx=0).stop 2024-04-24T09:07:04.578 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.577+0000 7fcebac00640 1 -- 172.21.15.83:0/2130396820 shutdown_connections 2024-04-24T09:07:04.578 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.577+0000 7fcebac00640 1 --2- 172.21.15.83:0/2130396820 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fce980a3dc0 0x7fce980a6280 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:04.578 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.577+0000 7fcebac00640 1 --2- 172.21.15.83:0/2130396820 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fceb4112e80 0x7fceb419eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:04.578 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.577+0000 7fcebac00640 1 --2- 172.21.15.83:0/2130396820 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fceb4076e50 0x7fceb41a4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:04.578 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.577+0000 7fcebac00640 1 --2- 172.21.15.83:0/2130396820 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fceb4076530 0x7fceb41a4a60 unknown :-1 s=CLOSED pgs=2232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:04.578 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.577+0000 7fcebac00640 1 -- 172.21.15.83:0/2130396820 >> 172.21.15.83:0/2130396820 conn(0x7fceb40ffcf0 msgr2=0x7fceb41020e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:04.578 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.577+0000 7fcebac00640 1 -- 172.21.15.83:0/2130396820 shutdown_connections 2024-04-24T09:07:04.578 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:04.577+0000 7fcebac00640 1 -- 172.21.15.83:0/2130396820 wait complete. 2024-04-24T09:07:05.160 INFO:tasks.workunit.client.0.smithi083.stdout: 10 12445 8.71 MB/sec warmup 96 sec latency 497.613 ms 2024-04-24T09:07:05.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:07:05 smithi138 ceph-mon[31441]: osdmap e114: 12 total, 12 up, 12 in 2024-04-24T09:07:05.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:07:05 smithi138 ceph-mon[31441]: pgmap v487: 129 pgs: 129 active+clean; 371 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.0 MiB/s wr, 410 op/s 2024-04-24T09:07:05.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:07:05 smithi189 ceph-mon[31791]: osdmap e114: 12 total, 12 up, 12 in 2024-04-24T09:07:05.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:07:05 smithi189 ceph-mon[31791]: pgmap v487: 129 pgs: 129 active+clean; 371 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.0 MiB/s wr, 410 op/s 2024-04-24T09:07:06.052 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:07:05 smithi083 ceph-mon[27907]: osdmap e114: 12 total, 12 up, 12 in 2024-04-24T09:07:06.052 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:07:05 smithi083 ceph-mon[27907]: pgmap v487: 129 pgs: 129 active+clean; 371 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.0 MiB/s wr, 410 op/s 2024-04-24T09:07:06.160 INFO:tasks.workunit.client.0.smithi083.stdout: 10 12585 8.73 MB/sec warmup 97 sec latency 73.866 ms 2024-04-24T09:07:07.160 INFO:tasks.workunit.client.0.smithi083.stdout: 10 12748 8.79 MB/sec warmup 98 sec latency 64.175 ms 2024-04-24T09:07:08.160 INFO:tasks.workunit.client.0.smithi083.stdout: 10 12913 8.81 MB/sec warmup 99 sec latency 72.348 ms 2024-04-24T09:07:08.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:07:07 smithi138 ceph-mon[31441]: pgmap v488: 129 pgs: 129 active+clean; 371 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 712 KiB/s rd, 1.9 MiB/s wr, 233 op/s 2024-04-24T09:07:08.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:07:07 smithi189 ceph-mon[31791]: pgmap v488: 129 pgs: 129 active+clean; 371 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 712 KiB/s rd, 1.9 MiB/s wr, 233 op/s 2024-04-24T09:07:08.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:07:07 smithi083 ceph-mon[27907]: pgmap v488: 129 pgs: 129 active+clean; 371 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 712 KiB/s rd, 1.9 MiB/s wr, 233 op/s 2024-04-24T09:07:09.160 INFO:tasks.workunit.client.0.smithi083.stdout: 10 13093 8.83 MB/sec warmup 100 sec latency 93.915 ms 2024-04-24T09:07:10.160 INFO:tasks.workunit.client.0.smithi083.stdout: 10 13179 8.78 MB/sec warmup 101 sec latency 77.934 ms 2024-04-24T09:07:10.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:07:09 smithi138 ceph-mon[31441]: pgmap v489: 129 pgs: 129 active+clean; 388 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.6 MiB/s wr, 426 op/s 2024-04-24T09:07:10.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:07:09 smithi189 ceph-mon[31791]: pgmap v489: 129 pgs: 129 active+clean; 388 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.6 MiB/s wr, 426 op/s 2024-04-24T09:07:10.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:07:09 smithi083 ceph-mon[27907]: pgmap v489: 129 pgs: 129 active+clean; 388 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.6 MiB/s wr, 426 op/s 2024-04-24T09:07:11.160 INFO:tasks.workunit.client.0.smithi083.stdout: 10 13272 8.74 MB/sec warmup 102 sec latency 170.396 ms 2024-04-24T09:07:12.160 INFO:tasks.workunit.client.0.smithi083.stdout: 10 13416 8.76 MB/sec warmup 103 sec latency 69.831 ms 2024-04-24T09:07:12.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:07:11 smithi138 ceph-mon[31441]: pgmap v490: 129 pgs: 129 active+clean; 388 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.3 MiB/s wr, 394 op/s 2024-04-24T09:07:12.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:07:11 smithi189 ceph-mon[31791]: pgmap v490: 129 pgs: 129 active+clean; 388 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.3 MiB/s wr, 394 op/s 2024-04-24T09:07:12.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:07:11 smithi083 ceph-mon[27907]: pgmap v490: 129 pgs: 129 active+clean; 388 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.3 MiB/s wr, 394 op/s 2024-04-24T09:07:13.160 INFO:tasks.workunit.client.0.smithi083.stdout: 10 13507 8.74 MB/sec warmup 104 sec latency 342.699 ms 2024-04-24T09:07:13.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:07:12 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:07:13.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:07:12 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:07:13.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:07:12 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:07:13.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:07:12 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:07:13.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:07:12 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:07:13.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:07:12 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:07:13.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:07:12 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:07:13.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:07:12 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:07:13.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:07:12 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:07:13.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:07:13 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:07:13.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:07:13 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:07:13.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:07:13 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:07:14.160 INFO:tasks.workunit.client.0.smithi083.stdout: 10 13625 8.71 MB/sec warmup 105 sec latency 76.702 ms 2024-04-24T09:07:14.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:07:13 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:07:14.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:07:13 smithi138 ceph-mon[31441]: pgmap v491: 129 pgs: 129 active+clean; 400 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.5 MiB/s wr, 284 op/s 2024-04-24T09:07:14.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:07:13 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:07:14.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:07:13 smithi189 ceph-mon[31791]: pgmap v491: 129 pgs: 129 active+clean; 400 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.5 MiB/s wr, 284 op/s 2024-04-24T09:07:14.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:07:13 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:07:14.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:07:13 smithi083 ceph-mon[27907]: pgmap v491: 129 pgs: 129 active+clean; 400 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.5 MiB/s wr, 284 op/s 2024-04-24T09:07:15.160 INFO:tasks.workunit.client.0.smithi083.stdout: 10 13705 8.65 MB/sec warmup 106 sec latency 351.141 ms 2024-04-24T09:07:16.161 INFO:tasks.workunit.client.0.smithi083.stdout: 10 13826 8.62 MB/sec warmup 107 sec latency 308.200 ms 2024-04-24T09:07:16.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:07:15 smithi138 ceph-mon[31441]: pgmap v492: 129 pgs: 129 active+clean; 400 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.4 MiB/s wr, 279 op/s 2024-04-24T09:07:16.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:07:15 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:07:16.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:07:15 smithi189 ceph-mon[31791]: pgmap v492: 129 pgs: 129 active+clean; 400 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.4 MiB/s wr, 279 op/s 2024-04-24T09:07:16.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:07:15 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:07:16.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:07:15 smithi083 ceph-mon[27907]: pgmap v492: 129 pgs: 129 active+clean; 400 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.4 MiB/s wr, 279 op/s 2024-04-24T09:07:16.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:07:15 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:07:17.161 INFO:tasks.workunit.client.0.smithi083.stdout: 10 13929 8.56 MB/sec warmup 108 sec latency 96.552 ms 2024-04-24T09:07:17.749 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.750+0000 7f9ab4a00640 1 -- 172.21.15.83:0/795494864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ab0074cf0 msgr2=0x7f9ab010e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:17.750 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.750+0000 7f9ab4a00640 1 --2- 172.21.15.83:0/795494864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ab0074cf0 0x7f9ab010e180 secure :-1 s=READY pgs=2235 cs=0 l=1 rev1=1 crypto rx=0x7f9a9c009900 tx=0x7f9a9c02a480 comp rx=0 tx=0).stop 2024-04-24T09:07:17.750 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.750+0000 7f9ab4a00640 1 -- 172.21.15.83:0/795494864 shutdown_connections 2024-04-24T09:07:17.750 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.750+0000 7f9ab4a00640 1 --2- 172.21.15.83:0/795494864 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9ab010e6c0 0x7f9ab0110ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:17.750 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.750+0000 7f9ab4a00640 1 --2- 172.21.15.83:0/795494864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ab0074cf0 0x7f9ab010e180 unknown :-1 s=CLOSED pgs=2235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:17.750 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.750+0000 7f9ab4a00640 1 --2- 172.21.15.83:0/795494864 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9ab00743d0 0x7f9ab00747b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:17.750 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.750+0000 7f9ab4a00640 1 -- 172.21.15.83:0/795494864 >> 172.21.15.83:0/795494864 conn(0x7f9ab00fd9b0 msgr2=0x7f9ab00ffdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:17.750 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.750+0000 7f9ab4a00640 1 -- 172.21.15.83:0/795494864 shutdown_connections 2024-04-24T09:07:17.750 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.751+0000 7f9ab4a00640 1 -- 172.21.15.83:0/795494864 wait complete. 2024-04-24T09:07:17.751 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.752+0000 7f9ab4a00640 1 Processor -- start 2024-04-24T09:07:17.751 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.752+0000 7f9ab4a00640 1 -- start start 2024-04-24T09:07:17.751 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.752+0000 7f9ab4a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9ab00743d0 0x7f9ab01a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:17.751 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.752+0000 7f9ab4a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9ab0074cf0 0x7f9ab01a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:17.751 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.752+0000 7f9ab4a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ab010e6c0 0x7f9ab019a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:17.751 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.752+0000 7f9ab4a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab0114340 con 0x7f9ab010e6c0 2024-04-24T09:07:17.752 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.753+0000 7f9ab4a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab01141c0 con 0x7f9ab0074cf0 2024-04-24T09:07:17.752 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.753+0000 7f9ab4a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab01144c0 con 0x7f9ab00743d0 2024-04-24T09:07:17.752 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.753+0000 7f9aafe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ab010e6c0 0x7f9ab019a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:17.752 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.753+0000 7f9aaea00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9ab0074cf0 0x7f9ab01a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:17.752 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.753+0000 7f9aafe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ab010e6c0 0x7f9ab019a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44584/0 (socket says 172.21.15.83:44584) 2024-04-24T09:07:17.753 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.753+0000 7f9aafe00640 1 -- 172.21.15.83:0/2661134954 learned_addr learned my addr 172.21.15.83:0/2661134954 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:07:17.753 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.753+0000 7f9aaf400640 1 --2- 172.21.15.83:0/2661134954 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9ab00743d0 0x7f9ab01a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:17.753 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.754+0000 7f9aafe00640 1 -- 172.21.15.83:0/2661134954 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9ab00743d0 msgr2=0x7f9ab01a0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:17.753 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.754+0000 7f9aafe00640 1 --2- 172.21.15.83:0/2661134954 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9ab00743d0 0x7f9ab01a0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:17.753 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.754+0000 7f9aafe00640 1 -- 172.21.15.83:0/2661134954 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9ab0074cf0 msgr2=0x7f9ab01a0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:17.753 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.754+0000 7f9aafe00640 1 --2- 172.21.15.83:0/2661134954 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9ab0074cf0 0x7f9ab01a0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:17.753 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.754+0000 7f9aafe00640 1 -- 172.21.15.83:0/2661134954 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ab019b000 con 0x7f9ab010e6c0 2024-04-24T09:07:17.754 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.755+0000 7f9aafe00640 1 --2- 172.21.15.83:0/2661134954 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ab010e6c0 0x7f9ab019a740 secure :-1 s=READY pgs=2236 cs=0 l=1 rev1=1 crypto rx=0x7f9aa8009fd0 tx=0x7f9aa800ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:17.754 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.755+0000 7f9aa7e00640 1 -- 172.21.15.83:0/2661134954 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa8004d20 con 0x7f9ab010e6c0 2024-04-24T09:07:17.754 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.755+0000 7f9aa7e00640 1 -- 172.21.15.83:0/2661134954 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9aa8005020 con 0x7f9ab010e6c0 2024-04-24T09:07:17.754 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.755+0000 7f9aa7e00640 1 -- 172.21.15.83:0/2661134954 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa80116d0 con 0x7f9ab010e6c0 2024-04-24T09:07:17.754 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.755+0000 7f9ab4a00640 1 -- 172.21.15.83:0/2661134954 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ab019b2f0 con 0x7f9ab010e6c0 2024-04-24T09:07:17.755 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.756+0000 7f9ab4a00640 1 -- 172.21.15.83:0/2661134954 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ab01a75b0 con 0x7f9ab010e6c0 2024-04-24T09:07:17.759 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.760+0000 7f9ab89fd740 1 Processor -- start 2024-04-24T09:07:17.760 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.761+0000 7f9aa7e00640 1 -- 172.21.15.83:0/2661134954 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9aa80051d0 con 0x7f9ab010e6c0 2024-04-24T09:07:17.760 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.761+0000 7f9ab89fd740 1 -- start start 2024-04-24T09:07:17.761 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.761+0000 7f9ab89fd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627972f62e0 0x5627972f66c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:17.761 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.761+0000 7f9ab89fd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5627972f6c00 0x562797309870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:17.761 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.761+0000 7f9ab89fd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562797309db0 0x56279730c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:17.761 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.761+0000 7f9ab89fd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56279725c900 con 0x5627972f6c00 2024-04-24T09:07:17.761 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.761+0000 7f9ab89fd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562797202f20 con 0x5627972f62e0 2024-04-24T09:07:17.761 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.761+0000 7f9ab89fd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5627972588c0 con 0x562797309db0 2024-04-24T09:07:17.762 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.762+0000 7f9aaea00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627972f62e0 0x5627972f66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:17.762 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.762+0000 7f9aaea00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627972f62e0 0x5627972f66c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:40692/0 (socket says 172.21.15.83:40692) 2024-04-24T09:07:17.762 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.762+0000 7f9aaea00640 1 -- 172.21.15.83:0/97926278 learned_addr learned my addr 172.21.15.83:0/97926278 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:07:17.762 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.762+0000 7f9aaea00640 1 --2- 172.21.15.83:0/97926278 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562797309db0 0x56279730c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:17.762 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.763+0000 7f9aaea00640 1 -- 172.21.15.83:0/97926278 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562797309db0 msgr2=0x56279730c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:17.762 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.763+0000 7f9aa7e00640 1 --2- 172.21.15.83:0/2661134954 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f9a940a3e60 0x7f9a940a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:17.762 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.763+0000 7f9aaea00640 1 --2- 172.21.15.83:0/97926278 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562797309db0 0x56279730c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:17.762 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.763+0000 7f9aaea00640 1 -- 172.21.15.83:0/97926278 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5627972f6c00 msgr2=0x562797309870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:17.763 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.763+0000 7f9aaea00640 1 --2- 172.21.15.83:0/97926278 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5627972f6c00 0x562797309870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:17.763 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.763+0000 7f9aaea00640 1 -- 172.21.15.83:0/97926278 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562797227e10 con 0x5627972f62e0 2024-04-24T09:07:17.763 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.763+0000 7f9aa7e00640 1 -- 172.21.15.83:0/2661134954 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 9583+0+0 (secure 0 0 0) 0x7f9aa80d0cb0 con 0x7f9ab010e6c0 2024-04-24T09:07:17.763 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.763+0000 7f9aaea00640 1 --2- 172.21.15.83:0/97926278 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627972f62e0 0x5627972f66c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9a9c006010 tx=0x7f9a9c005780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f90ed74f967bbd16 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:17.763 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.764+0000 7f9aafe00640 1 --2- 172.21.15.83:0/2661134954 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f9a940a3e60 0x7f9a940a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:17.763 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.764+0000 7f9a9be00640 1 -- 172.21.15.83:0/97926278 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a9c035030 con 0x5627972f62e0 2024-04-24T09:07:17.764 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.764+0000 7f9a9be00640 1 -- 172.21.15.83:0/97926278 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9a9c0314a0 con 0x5627972f62e0 2024-04-24T09:07:17.764 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.764+0000 7f9a9be00640 1 -- 172.21.15.83:0/97926278 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a9c03d6b0 con 0x5627972f62e0 2024-04-24T09:07:17.764 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.765+0000 7f9ab89fd740 1 -- 172.21.15.83:0/97926278 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627972f62e0 msgr2=0x5627972f66c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:17.764 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.765+0000 7f9ab89fd740 1 --2- 172.21.15.83:0/97926278 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627972f62e0 0x5627972f66c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9a9c006010 tx=0x7f9a9c005780 comp rx=0 tx=0).stop 2024-04-24T09:07:17.765 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.765+0000 7f9aafe00640 1 --2- 172.21.15.83:0/2661134954 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f9a940a3e60 0x7f9a940a6320 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f9aa8004af0 tx=0x7f9aa8004a40 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:17.765 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.765+0000 7f9ab89fd740 1 -- 172.21.15.83:0/97926278 shutdown_connections 2024-04-24T09:07:17.765 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.765+0000 7f9ab89fd740 1 --2- 172.21.15.83:0/97926278 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562797309db0 0x56279730c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:17.765 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.765+0000 7f9ab89fd740 1 --2- 172.21.15.83:0/97926278 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5627972f6c00 0x562797309870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:17.765 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.765+0000 7f9ab89fd740 1 --2- 172.21.15.83:0/97926278 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627972f62e0 0x5627972f66c0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:17.765 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.765+0000 7f9ab89fd740 1 -- 172.21.15.83:0/97926278 >> 172.21.15.83:0/97926278 conn(0x5627972f28d0 msgr2=0x5627972f2cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:17.765 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.765+0000 7f9ab89fd740 1 -- 172.21.15.83:0/97926278 shutdown_connections 2024-04-24T09:07:17.765 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.765+0000 7f9ab89fd740 1 -- 172.21.15.83:0/97926278 wait complete. 2024-04-24T09:07:17.765 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.766+0000 7f9ab89fd740 1 -- start start 2024-04-24T09:07:17.765 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.766+0000 7f9ab89fd740 1 Processor -- start 2024-04-24T09:07:17.767 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.766+0000 7f9ab89fd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5627972f62e0 0x562797306e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:17.767 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.766+0000 7f9ab89fd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5627972f6c00 0x562797307370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:17.767 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.766+0000 7f9ab89fd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562797309db0 0x5627973078b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:17.767 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.766+0000 7f9ab89fd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56279730c980 con 0x5627972f62e0 2024-04-24T09:07:17.767 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.766+0000 7f9ab89fd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5627972162c0 con 0x562797309db0 2024-04-24T09:07:17.767 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.766+0000 7f9ab89fd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56279730cb00 con 0x5627972f6c00 2024-04-24T09:07:17.767 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.766+0000 7f9ab89fd740 10 client.0 _ll_register_callbacks cb 0x562796f6ea00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:07:17.767 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.767+0000 7f9aaea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5627972f62e0 0x562797306e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:17.767 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.767+0000 7f9aaea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5627972f62e0 0x562797306e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44594/0 (socket says 172.21.15.83:44594) 2024-04-24T09:07:17.767 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.767+0000 7f9aaea00640 1 -- 172.21.15.83:0/2183306270 learned_addr learned my addr 172.21.15.83:0/2183306270 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:07:17.767 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.767+0000 7f9aaf400640 1 --2- 172.21.15.83:0/2183306270 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5627972f6c00 0x562797307370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:17.767 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.767+0000 7f9aaea00640 1 --2- 172.21.15.83:0/2183306270 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562797309db0 0x5627973078b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:17.767 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.767+0000 7f9aaea00640 1 -- 172.21.15.83:0/2183306270 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5627972f6c00 msgr2=0x562797307370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:17.767 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.767+0000 7f9aaea00640 1 --2- 172.21.15.83:0/2183306270 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5627972f6c00 0x562797307370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:17.768 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.767+0000 7f9aaea00640 1 -- 172.21.15.83:0/2183306270 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562797309db0 msgr2=0x5627973078b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:17.768 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.767+0000 7f9aaea00640 1 --2- 172.21.15.83:0/2183306270 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562797309db0 0x5627973078b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:17.768 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.767+0000 7f9aaea00640 1 -- 172.21.15.83:0/2183306270 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562797226fd0 con 0x5627972f62e0 2024-04-24T09:07:17.768 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.768+0000 7f9aaea00640 1 --2- 172.21.15.83:0/2183306270 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5627972f62e0 0x562797306e30 secure :-1 s=READY pgs=2237 cs=0 l=1 rev1=1 crypto rx=0x7f9a9c0092a0 tx=0x7f9a9c031e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=857d31e326b9e916 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:17.768 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.768+0000 7f9a98c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:07:17.769 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.768+0000 7f9a98c00640 1 -- 172.21.15.83:0/2183306270 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a9c041070 con 0x5627972f62e0 2024-04-24T09:07:17.769 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.768+0000 7f9a98c00640 1 -- 172.21.15.83:0/2183306270 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9583+0+0 (secure 0 0 0) 0x7f9a9c0038c0 con 0x5627972f62e0 2024-04-24T09:07:17.769 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.768+0000 7f9ab89fd740 1 -- 172.21.15.83:0/2183306270 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a9c0314a0 con 0x5627972f62e0 2024-04-24T09:07:17.769 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.769+0000 7f9a98c00640 1 -- 172.21.15.83:0/2183306270 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f9a9c035040 con 0x5627972f62e0 2024-04-24T09:07:17.769 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.769+0000 7f9ab89fd740 10 client.21096 fetch_fsmap learned FSMap version 50 2024-04-24T09:07:17.769 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.769+0000 7f9ab89fd740 1 -- 172.21.15.83:0/2183306270 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x562796f6f210 con 0x5627972f62e0 2024-04-24T09:07:17.769 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.770+0000 7f9a98c00640 1 -- 172.21.15.83:0/2183306270 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9a9c0314a0 con 0x5627972f62e0 2024-04-24T09:07:17.769 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.770+0000 7f9a98c00640 1 -- 172.21.15.83:0/2183306270 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 50) ==== 8410+0+0 (secure 0 0 0) 0x7f9a9c03dc50 con 0x5627972f62e0 2024-04-24T09:07:17.770 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.770+0000 7f9ab89fd740 10 client.21096 fetch_fsmap finished waiting for FSMap version 50 2024-04-24T09:07:17.770 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.771+0000 7f9ab89fd740 10 client.21096 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:07:17.770 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.771+0000 7f9ab89fd740 20 client.21096 populate_metadata read hostname 'smithi083' 2024-04-24T09:07:17.770 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.771+0000 7f9ab89fd740 1 --2- 172.21.15.83:0/2183306270 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x562797319e90 0x56279731c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:17.770 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.771+0000 7f9ab89fd740 4 client.21096 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:07:17.770 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.771+0000 7f9ab89fd740 1 -- 172.21.15.83:0/2183306270 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56279726a320 con 0x562797319e90 2024-04-24T09:07:17.770 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.771+0000 7f9aaf400640 1 --2- 172.21.15.83:0/2183306270 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x562797319e90 0x56279731c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:17.771 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.772+0000 7f9a98c00640 10 client.21096 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:07:17.771 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.772+0000 7f9aaf400640 1 --2- 172.21.15.83:0/2183306270 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x562797319e90 0x56279731c280 crc :-1 s=READY pgs=399 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=977e21a0d5ad827a server_cookie=2401fafe6c032fa7 in_seq=0 out_seq=0 2024-04-24T09:07:17.772 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.774+0000 7f9a98c00640 1 -- 172.21.15.83:0/2183306270 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56279726a320 con 0x562797319e90 2024-04-24T09:07:17.772 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.774+0000 7f9a98c00640 10 client.21096 handle_command_reply: tid=0 2024-04-24T09:07:17.773 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.774+0000 7f9ab89fd740 1 client.21096 shutdown 2024-04-24T09:07:17.773 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.774+0000 7f9ab89fd740 1 -- 172.21.15.83:0/2183306270 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5627972f62e0 msgr2=0x562797306e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:17.773 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.774+0000 7f9ab89fd740 1 --2- 172.21.15.83:0/2183306270 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5627972f62e0 0x562797306e30 secure :-1 s=READY pgs=2237 cs=0 l=1 rev1=1 crypto rx=0x7f9a9c0092a0 tx=0x7f9a9c031e00 comp rx=0 tx=0).stop 2024-04-24T09:07:17.773 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.774+0000 7f9ab89fd740 1 -- 172.21.15.83:0/2183306270 shutdown_connections 2024-04-24T09:07:17.773 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.774+0000 7f9ab89fd740 1 --2- 172.21.15.83:0/2183306270 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x562797319e90 0x56279731c280 crc :-1 s=READY pgs=399 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:17.773 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.774+0000 7f9ab89fd740 1 --2- 172.21.15.83:0/2183306270 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562797309db0 0x5627973078b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:17.774 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.774+0000 7f9ab89fd740 1 --2- 172.21.15.83:0/2183306270 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5627972f6c00 0x562797307370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:17.774 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.774+0000 7f9ab89fd740 1 --2- 172.21.15.83:0/2183306270 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5627972f62e0 0x562797306e30 secure :-1 s=CLOSED pgs=2237 cs=0 l=1 rev1=1 crypto rx=0x7f9a9c0092a0 tx=0x7f9a9c031e00 comp rx=0 tx=0).stop 2024-04-24T09:07:17.774 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.774+0000 7f9ab89fd740 1 -- 172.21.15.83:0/2183306270 >> 172.21.15.83:0/2183306270 conn(0x5627972f28d0 msgr2=0x5627972f8740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:17.774 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.774+0000 7f9a98c00640 0 client.21096 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:07:17.774 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.775+0000 7f9ab89fd740 1 -- 172.21.15.83:0/2183306270 shutdown_connections 2024-04-24T09:07:17.774 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.775+0000 7f9ab89fd740 1 -- 172.21.15.83:0/2183306270 wait complete. 2024-04-24T09:07:17.774 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.775+0000 7f9ab89fd740 20 client.21096 trim_cache size 0 max 16384 2024-04-24T09:07:17.800 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.801+0000 7f9ab89fd740 1 Processor -- start 2024-04-24T09:07:17.801 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.802+0000 7f9ab89fd740 1 -- start start 2024-04-24T09:07:17.801 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.802+0000 7f9ab89fd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627972f6c00 0x5627972f2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:17.801 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.802+0000 7f9ab89fd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562797309db0 0x5627972f8500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:17.801 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.802+0000 7f9ab89fd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562797305ba0 0x5627973040f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:17.801 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.802+0000 7f9ab89fd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab01ad270 con 0x562797305ba0 2024-04-24T09:07:17.801 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.802+0000 7f9ab89fd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab01aef30 con 0x5627972f6c00 2024-04-24T09:07:17.801 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.802+0000 7f9ab89fd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab01ad940 con 0x562797309db0 2024-04-24T09:07:17.802 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.803+0000 7f9aaea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562797305ba0 0x5627973040f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:17.802 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.803+0000 7f9aaea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562797305ba0 0x5627973040f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44604/0 (socket says 172.21.15.83:44604) 2024-04-24T09:07:17.803 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.803+0000 7f9aaea00640 1 -- 172.21.15.83:0/2801497545 learned_addr learned my addr 172.21.15.83:0/2801497545 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:07:17.803 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.803+0000 7f9aaf400640 1 --2- 172.21.15.83:0/2801497545 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562797309db0 0x5627972f8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:17.803 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.803+0000 7f9aaea00640 1 --2- 172.21.15.83:0/2801497545 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627972f6c00 0x5627972f2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:17.803 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.803+0000 7f9aaea00640 1 -- 172.21.15.83:0/2801497545 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562797309db0 msgr2=0x5627972f8500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:17.803 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.803+0000 7f9aaea00640 1 --2- 172.21.15.83:0/2801497545 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562797309db0 0x5627972f8500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:17.803 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.803+0000 7f9aaea00640 1 -- 172.21.15.83:0/2801497545 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627972f6c00 msgr2=0x5627972f2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:17.803 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.803+0000 7f9aaea00640 1 --2- 172.21.15.83:0/2801497545 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627972f6c00 0x5627972f2210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:17.803 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.803+0000 7f9aaea00640 1 -- 172.21.15.83:0/2801497545 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56279703b620 con 0x562797305ba0 2024-04-24T09:07:17.803 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.804+0000 7f9aaea00640 1 --2- 172.21.15.83:0/2801497545 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562797305ba0 0x5627973040f0 secure :-1 s=READY pgs=2238 cs=0 l=1 rev1=1 crypto rx=0x7f9a9c005ab0 tx=0x7f9a9c02a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea2bcaf01986840 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:17.804 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.804+0000 7f9a9b400640 1 -- 172.21.15.83:0/2801497545 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a9c041040 con 0x562797305ba0 2024-04-24T09:07:17.804 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.804+0000 7f9a9b400640 1 -- 172.21.15.83:0/2801497545 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9a9c053070 con 0x562797305ba0 2024-04-24T09:07:17.804 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.804+0000 7f9a9b400640 1 -- 172.21.15.83:0/2801497545 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a9c005280 con 0x562797305ba0 2024-04-24T09:07:17.804 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.805+0000 7f9ab89fd740 1 -- 172.21.15.83:0/2801497545 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562797305ba0 msgr2=0x5627973040f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:17.804 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.805+0000 7f9ab89fd740 1 --2- 172.21.15.83:0/2801497545 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562797305ba0 0x5627973040f0 secure :-1 s=READY pgs=2238 cs=0 l=1 rev1=1 crypto rx=0x7f9a9c005ab0 tx=0x7f9a9c02a480 comp rx=0 tx=0).stop 2024-04-24T09:07:17.804 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.805+0000 7f9ab89fd740 1 -- 172.21.15.83:0/2801497545 shutdown_connections 2024-04-24T09:07:17.804 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.805+0000 7f9ab89fd740 1 --2- 172.21.15.83:0/2801497545 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562797305ba0 0x5627973040f0 unknown :-1 s=CLOSED pgs=2238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:17.805 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.805+0000 7f9ab89fd740 1 --2- 172.21.15.83:0/2801497545 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562797309db0 0x5627972f8500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:17.805 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.805+0000 7f9ab89fd740 1 --2- 172.21.15.83:0/2801497545 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627972f6c00 0x5627972f2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:17.805 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.805+0000 7f9ab89fd740 1 -- 172.21.15.83:0/2801497545 >> 172.21.15.83:0/2801497545 conn(0x5627972f28d0 msgr2=0x5627972f7df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:17.805 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.806+0000 7f9ab89fd740 1 -- 172.21.15.83:0/2801497545 shutdown_connections 2024-04-24T09:07:17.805 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.806+0000 7f9ab89fd740 1 -- 172.21.15.83:0/2801497545 wait complete. 2024-04-24T09:07:17.805 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.806+0000 7f9ab89fd740 1 -- start start 2024-04-24T09:07:17.806 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.807+0000 7f9ab89fd740 1 Processor -- start 2024-04-24T09:07:17.806 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.807+0000 7f9ab89fd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627972f6c00 0x56279737b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:17.806 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.807+0000 7f9ab89fd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562797305ba0 0x56279737b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:17.806 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.807+0000 7f9ab89fd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562797309db0 0x56279737bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:17.806 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.807+0000 7f9ab89fd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab01ae7a0 con 0x562797309db0 2024-04-24T09:07:17.807 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.807+0000 7f9ab89fd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab01ae070 con 0x5627972f6c00 2024-04-24T09:07:17.807 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.807+0000 7f9ab89fd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab01ab270 con 0x562797305ba0 2024-04-24T09:07:17.807 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.807+0000 7f9ab89fd740 10 client.0 _ll_register_callbacks cb 0x7f9a9c0042e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:07:17.807 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.808+0000 7f9aaea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562797309db0 0x56279737bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:17.808 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.808+0000 7f9aaea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562797309db0 0x56279737bad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44612/0 (socket says 172.21.15.83:44612) 2024-04-24T09:07:17.808 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.808+0000 7f9aaea00640 1 -- 172.21.15.83:0/1828590131 learned_addr learned my addr 172.21.15.83:0/1828590131 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:07:17.808 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.808+0000 7f9aaf400640 1 --2- 172.21.15.83:0/1828590131 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562797305ba0 0x56279737b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:17.808 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.808+0000 7f9aaea00640 1 --2- 172.21.15.83:0/1828590131 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627972f6c00 0x56279737b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:17.808 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.808+0000 7f9aaea00640 1 -- 172.21.15.83:0/1828590131 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562797305ba0 msgr2=0x56279737b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:17.808 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.808+0000 7f9aaea00640 1 --2- 172.21.15.83:0/1828590131 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562797305ba0 0x56279737b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:17.808 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.808+0000 7f9aaea00640 1 -- 172.21.15.83:0/1828590131 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627972f6c00 msgr2=0x56279737b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:17.808 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.808+0000 7f9aaea00640 1 --2- 172.21.15.83:0/1828590131 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627972f6c00 0x56279737b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:17.808 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.808+0000 7f9aaea00640 1 -- 172.21.15.83:0/1828590131 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56279731d1e0 con 0x562797309db0 2024-04-24T09:07:17.809 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.809+0000 7f9aaea00640 1 --2- 172.21.15.83:0/1828590131 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562797309db0 0x56279737bad0 secure :-1 s=READY pgs=2239 cs=0 l=1 rev1=1 crypto rx=0x7f9a9c004200 tx=0x7f9a9c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25223e0641a3a91f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:17.809 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.809+0000 7f9a98c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:07:17.809 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.809+0000 7f9a98c00640 1 -- 172.21.15.83:0/1828590131 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a9c041070 con 0x562797309db0 2024-04-24T09:07:17.810 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.809+0000 7f9a98c00640 1 -- 172.21.15.83:0/1828590131 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9583+0+0 (secure 0 0 0) 0x7f9a9c031bd0 con 0x562797309db0 2024-04-24T09:07:17.810 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.809+0000 7f9ab89fd740 1 -- 172.21.15.83:0/1828590131 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a9c053070 con 0x562797309db0 2024-04-24T09:07:17.810 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.809+0000 7f9a98c00640 1 -- 172.21.15.83:0/1828590131 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f9a9c0314a0 con 0x562797309db0 2024-04-24T09:07:17.810 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.809+0000 7f9ab89fd740 10 client.21102 fetch_fsmap learned FSMap version 50 2024-04-24T09:07:17.810 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.809+0000 7f9ab89fd740 1 -- 172.21.15.83:0/1828590131 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x5627973915b0 con 0x562797309db0 2024-04-24T09:07:17.810 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.810+0000 7f9a98c00640 1 -- 172.21.15.83:0/1828590131 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9a9c053070 con 0x562797309db0 2024-04-24T09:07:17.810 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.810+0000 7f9a98c00640 1 -- 172.21.15.83:0/1828590131 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 50) ==== 8410+0+0 (secure 0 0 0) 0x7f9a9c06b020 con 0x562797309db0 2024-04-24T09:07:17.810 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.810+0000 7f9ab89fd740 10 client.21102 fetch_fsmap finished waiting for FSMap version 50 2024-04-24T09:07:17.810 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.810+0000 7f9ab89fd740 10 client.21102 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:07:17.810 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.810+0000 7f9ab89fd740 20 client.21102 populate_metadata read hostname 'smithi083' 2024-04-24T09:07:17.810 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.810+0000 7f9ab89fd740 1 --2- 172.21.15.83:0/1828590131 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x562797391ae0 0x562797393f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:17.810 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.810+0000 7f9ab89fd740 4 client.21102 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T09:07:17.810 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.810+0000 7f9ab89fd740 1 -- 172.21.15.83:0/1828590131 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562797264190 con 0x562797391ae0 2024-04-24T09:07:17.811 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.811+0000 7f9aaf400640 1 --2- 172.21.15.83:0/1828590131 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x562797391ae0 0x562797393f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:17.811 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.812+0000 7f9aaf400640 1 --2- 172.21.15.83:0/1828590131 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x562797391ae0 0x562797393f00 crc :-1 s=READY pgs=400 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c3711b7f5fcf7d7 server_cookie=81cf3dd6928ef433 in_seq=0 out_seq=0 2024-04-24T09:07:17.811 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.812+0000 7f9a98c00640 10 client.21102 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:07:17.952 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.953+0000 7f9a98c00640 1 -- 172.21.15.83:0/1828590131 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562797264190 con 0x562797391ae0 2024-04-24T09:07:17.952 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.953+0000 7f9a98c00640 10 client.21102 handle_command_reply: tid=0 2024-04-24T09:07:17.952 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.953+0000 7f9ab89fd740 1 client.21102 shutdown 2024-04-24T09:07:17.953 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.954+0000 7f9ab89fd740 1 -- 172.21.15.83:0/1828590131 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562797309db0 msgr2=0x56279737bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:17.953 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.954+0000 7f9ab89fd740 1 --2- 172.21.15.83:0/1828590131 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562797309db0 0x56279737bad0 secure :-1 s=READY pgs=2239 cs=0 l=1 rev1=1 crypto rx=0x7f9a9c004200 tx=0x7f9a9c000bc0 comp rx=0 tx=0).stop 2024-04-24T09:07:17.954 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.954+0000 7f9ab89fd740 1 -- 172.21.15.83:0/1828590131 shutdown_connections 2024-04-24T09:07:17.954 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.954+0000 7f9ab89fd740 1 --2- 172.21.15.83:0/1828590131 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x562797391ae0 0x562797393f00 crc :-1 s=READY pgs=400 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:17.954 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.954+0000 7f9ab89fd740 1 --2- 172.21.15.83:0/1828590131 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562797309db0 0x56279737bad0 unknown :-1 s=CLOSED pgs=2239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:17.954 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.954+0000 7f9ab89fd740 1 --2- 172.21.15.83:0/1828590131 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562797305ba0 0x56279737b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:17.954 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.954+0000 7f9ab89fd740 1 --2- 172.21.15.83:0/1828590131 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627972f6c00 0x56279737b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:17.954 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.954+0000 7f9ab89fd740 1 -- 172.21.15.83:0/1828590131 >> 172.21.15.83:0/1828590131 conn(0x5627972f28d0 msgr2=0x562797306c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:17.954 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.954+0000 7f9a98c00640 0 client.21102 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:07:17.955 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.954+0000 7f9ab89fd740 1 -- 172.21.15.83:0/1828590131 shutdown_connections 2024-04-24T09:07:17.955 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.954+0000 7f9ab89fd740 1 -- 172.21.15.83:0/1828590131 wait complete. 2024-04-24T09:07:17.955 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.954+0000 7f9ab89fd740 20 client.21102 trim_cache size 0 max 16384 2024-04-24T09:07:17.955 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T09:07:17.955 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T09:07:17.955 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T09:07:17.955 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T09:07:17.956 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.955+0000 7f9ab4a00640 1 -- 172.21.15.83:0/2661134954 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f9a940a3e60 msgr2=0x7f9a940a6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:17.956 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.955+0000 7f9ab4a00640 1 --2- 172.21.15.83:0/2661134954 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f9a940a3e60 0x7f9a940a6320 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f9aa8004af0 tx=0x7f9aa8004a40 comp rx=0 tx=0).stop 2024-04-24T09:07:17.956 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.955+0000 7f9ab4a00640 1 -- 172.21.15.83:0/2661134954 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ab010e6c0 msgr2=0x7f9ab019a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:17.956 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.955+0000 7f9ab4a00640 1 --2- 172.21.15.83:0/2661134954 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ab010e6c0 0x7f9ab019a740 secure :-1 s=READY pgs=2236 cs=0 l=1 rev1=1 crypto rx=0x7f9aa8009fd0 tx=0x7f9aa800ee60 comp rx=0 tx=0).stop 2024-04-24T09:07:17.957 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.956+0000 7f9ab4a00640 1 -- 172.21.15.83:0/2661134954 shutdown_connections 2024-04-24T09:07:17.957 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.956+0000 7f9ab4a00640 1 --2- 172.21.15.83:0/2661134954 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f9a940a3e60 0x7f9a940a6320 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:17.957 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.956+0000 7f9ab4a00640 1 --2- 172.21.15.83:0/2661134954 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ab010e6c0 0x7f9ab019a740 unknown :-1 s=CLOSED pgs=2236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:17.957 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.956+0000 7f9ab4a00640 1 --2- 172.21.15.83:0/2661134954 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9ab0074cf0 0x7f9ab01a0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:17.957 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.956+0000 7f9ab4a00640 1 --2- 172.21.15.83:0/2661134954 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9ab00743d0 0x7f9ab01a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:17.957 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.956+0000 7f9ab4a00640 1 -- 172.21.15.83:0/2661134954 >> 172.21.15.83:0/2661134954 conn(0x7f9ab00fd9b0 msgr2=0x7f9ab00fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:17.957 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.956+0000 7f9ab4a00640 1 -- 172.21.15.83:0/2661134954 shutdown_connections 2024-04-24T09:07:17.958 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:17.956+0000 7f9ab4a00640 1 -- 172.21.15.83:0/2661134954 wait complete. 2024-04-24T09:07:18.160 INFO:tasks.workunit.client.0.smithi083.stdout: 10 14063 8.52 MB/sec warmup 109 sec latency 144.486 ms 2024-04-24T09:07:18.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:07:17 smithi138 ceph-mon[31441]: pgmap v493: 129 pgs: 129 active+clean; 400 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1004 KiB/s rd, 3.8 MiB/s wr, 237 op/s 2024-04-24T09:07:18.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:07:17 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:07:18.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:07:17 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:07:18.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:07:17 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:07:18.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:07:17 smithi189 ceph-mon[31791]: pgmap v493: 129 pgs: 129 active+clean; 400 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1004 KiB/s rd, 3.8 MiB/s wr, 237 op/s 2024-04-24T09:07:18.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:07:17 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:07:18.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:07:17 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:07:18.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:07:17 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:07:18.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:07:17 smithi083 ceph-mon[27907]: pgmap v493: 129 pgs: 129 active+clean; 400 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1004 KiB/s rd, 3.8 MiB/s wr, 237 op/s 2024-04-24T09:07:18.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:07:17 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:07:18.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:07:17 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:07:18.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:07:17 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:07:19.161 INFO:tasks.workunit.client.0.smithi083.stdout: 10 14198 8.49 MB/sec warmup 110 sec latency 73.222 ms 2024-04-24T09:07:20.161 INFO:tasks.workunit.client.0.smithi083.stdout: 10 14325 8.46 MB/sec warmup 111 sec latency 78.117 ms 2024-04-24T09:07:20.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:07:19 smithi138 ceph-mon[31441]: pgmap v494: 129 pgs: 129 active+clean; 404 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.1 MiB/s wr, 328 op/s 2024-04-24T09:07:20.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:07:19 smithi189 ceph-mon[31791]: pgmap v494: 129 pgs: 129 active+clean; 404 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.1 MiB/s wr, 328 op/s 2024-04-24T09:07:20.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:07:19 smithi083 ceph-mon[27907]: pgmap v494: 129 pgs: 129 active+clean; 404 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.1 MiB/s wr, 328 op/s 2024-04-24T09:07:21.162 INFO:tasks.workunit.client.0.smithi083.stdout: 10 14468 8.47 MB/sec warmup 112 sec latency 91.403 ms 2024-04-24T09:07:22.161 INFO:tasks.workunit.client.0.smithi083.stdout: 10 14602 8.44 MB/sec warmup 113 sec latency 74.788 ms 2024-04-24T09:07:22.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:07:21 smithi138 ceph-mon[31441]: pgmap v495: 129 pgs: 129 active+clean; 404 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.2 MiB/s wr, 199 op/s 2024-04-24T09:07:22.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:07:21 smithi189 ceph-mon[31791]: pgmap v495: 129 pgs: 129 active+clean; 404 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.2 MiB/s wr, 199 op/s 2024-04-24T09:07:22.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:07:21 smithi083 ceph-mon[27907]: pgmap v495: 129 pgs: 129 active+clean; 404 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.2 MiB/s wr, 199 op/s 2024-04-24T09:07:23.161 INFO:tasks.workunit.client.0.smithi083.stdout: 10 14755 8.49 MB/sec warmup 114 sec latency 76.612 ms 2024-04-24T09:07:24.161 INFO:tasks.workunit.client.0.smithi083.stdout: 10 14881 8.43 MB/sec warmup 115 sec latency 64.950 ms 2024-04-24T09:07:24.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:07:23 smithi138 ceph-mon[31441]: pgmap v496: 129 pgs: 129 active+clean; 400 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.7 MiB/s wr, 345 op/s 2024-04-24T09:07:24.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:07:23 smithi189 ceph-mon[31791]: pgmap v496: 129 pgs: 129 active+clean; 400 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.7 MiB/s wr, 345 op/s 2024-04-24T09:07:24.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:07:23 smithi083 ceph-mon[27907]: pgmap v496: 129 pgs: 129 active+clean; 400 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.7 MiB/s wr, 345 op/s 2024-04-24T09:07:25.161 INFO:tasks.workunit.client.0.smithi083.stdout: 10 14975 8.38 MB/sec warmup 116 sec latency 302.804 ms 2024-04-24T09:07:26.162 INFO:tasks.workunit.client.0.smithi083.stdout: 10 15138 8.38 MB/sec warmup 117 sec latency 77.121 ms 2024-04-24T09:07:26.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:07:25 smithi138 ceph-mon[31441]: pgmap v497: 129 pgs: 129 active+clean; 400 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.9 MiB/s wr, 237 op/s 2024-04-24T09:07:26.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:07:25 smithi189 ceph-mon[31791]: pgmap v497: 129 pgs: 129 active+clean; 400 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.9 MiB/s wr, 237 op/s 2024-04-24T09:07:26.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:07:25 smithi083 ceph-mon[27907]: pgmap v497: 129 pgs: 129 active+clean; 400 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.9 MiB/s wr, 237 op/s 2024-04-24T09:07:27.162 INFO:tasks.workunit.client.0.smithi083.stdout: 10 15279 8.35 MB/sec warmup 118 sec latency 65.424 ms 2024-04-24T09:07:27.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:07:26 smithi138 ceph-mon[31441]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:07:27.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:07:26 smithi189 ceph-mon[31791]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:07:27.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:07:26 smithi083 ceph-mon[27907]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:07:28.162 INFO:tasks.workunit.client.0.smithi083.stdout: 10 15456 8.35 MB/sec warmup 119 sec latency 74.469 ms 2024-04-24T09:07:28.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:07:27 smithi138 ceph-mon[31441]: pgmap v498: 129 pgs: 129 active+clean; 400 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.9 MiB/s wr, 237 op/s 2024-04-24T09:07:28.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:07:27 smithi189 ceph-mon[31791]: pgmap v498: 129 pgs: 129 active+clean; 400 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.9 MiB/s wr, 237 op/s 2024-04-24T09:07:28.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:07:27 smithi083 ceph-mon[27907]: pgmap v498: 129 pgs: 129 active+clean; 400 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.9 MiB/s wr, 237 op/s 2024-04-24T09:07:30.162 INFO:tasks.workunit.client.0.smithi083.stdout: 10 15704 4.80 MB/sec execute 1 sec latency 364.349 ms 2024-04-24T09:07:30.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:07:29 smithi189 ceph-mon[31791]: pgmap v499: 129 pgs: 129 active+clean; 411 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.1 MiB/s wr, 328 op/s 2024-04-24T09:07:30.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:07:29 smithi083 ceph-mon[27907]: pgmap v499: 129 pgs: 129 active+clean; 411 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.1 MiB/s wr, 328 op/s 2024-04-24T09:07:30.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:07:29 smithi138 ceph-mon[31441]: pgmap v499: 129 pgs: 129 active+clean; 411 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.1 MiB/s wr, 328 op/s 2024-04-24T09:07:30.460 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T09:07:30.460 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T09:07:30.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.721+0000 7f0dbfc00640 1 -- 172.21.15.83:0/3491027770 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0db80743a0 msgr2=0x7f0db8074780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:30.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.721+0000 7f0dbfc00640 1 --2- 172.21.15.83:0/3491027770 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0db80743a0 0x7f0db8074780 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0da4009900 tx=0x7f0da402a490 comp rx=0 tx=0).stop 2024-04-24T09:07:30.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.721+0000 7f0dbfc00640 1 -- 172.21.15.83:0/3491027770 shutdown_connections 2024-04-24T09:07:30.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.721+0000 7f0dbfc00640 1 --2- 172.21.15.83:0/3491027770 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0db8112fd0 0x7f0db81153c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:30.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.721+0000 7f0dbfc00640 1 --2- 172.21.15.83:0/3491027770 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0db8074cc0 0x7f0db8112a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:30.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.721+0000 7f0dbfc00640 1 --2- 172.21.15.83:0/3491027770 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0db80743a0 0x7f0db8074780 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:30.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.721+0000 7f0dbfc00640 1 -- 172.21.15.83:0/3491027770 >> 172.21.15.83:0/3491027770 conn(0x7f0db80fda70 msgr2=0x7f0db80ffe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:30.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.721+0000 7f0dbfc00640 1 -- 172.21.15.83:0/3491027770 shutdown_connections 2024-04-24T09:07:30.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.721+0000 7f0dbfc00640 1 -- 172.21.15.83:0/3491027770 wait complete. 2024-04-24T09:07:30.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.722+0000 7f0dbfc00640 1 Processor -- start 2024-04-24T09:07:30.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.723+0000 7f0dbfc00640 1 -- start start 2024-04-24T09:07:30.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.723+0000 7f0dbfc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0db80743a0 0x7f0db819ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:30.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.723+0000 7f0dbfc00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0db8074cc0 0x7f0db819f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:30.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.723+0000 7f0dbfc00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0db8112fd0 0x7f0db81a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:30.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.723+0000 7f0dbfc00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0db81188d0 con 0x7f0db80743a0 2024-04-24T09:07:30.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.723+0000 7f0dbfc00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0db8118750 con 0x7f0db8112fd0 2024-04-24T09:07:30.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.723+0000 7f0dbfc00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0db8118a50 con 0x7f0db8074cc0 2024-04-24T09:07:30.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.723+0000 7f0dbd200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0db80743a0 0x7f0db819ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:30.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.724+0000 7f0dbdc00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0db8112fd0 0x7f0db81a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:30.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.724+0000 7f0db7e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0db8074cc0 0x7f0db819f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:30.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.724+0000 7f0dbdc00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0db8112fd0 0x7f0db81a3710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:42694/0 (socket says 172.21.15.83:42694) 2024-04-24T09:07:30.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.724+0000 7f0dbdc00640 1 -- 172.21.15.83:0/1914752427 learned_addr learned my addr 172.21.15.83:0/1914752427 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:07:30.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.724+0000 7f0dbdc00640 1 -- 172.21.15.83:0/1914752427 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0db8074cc0 msgr2=0x7f0db819f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:30.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.724+0000 7f0dbdc00640 1 --2- 172.21.15.83:0/1914752427 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0db8074cc0 0x7f0db819f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:30.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.724+0000 7f0dbdc00640 1 -- 172.21.15.83:0/1914752427 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0db80743a0 msgr2=0x7f0db819ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:30.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.725+0000 7f0dbdc00640 1 --2- 172.21.15.83:0/1914752427 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0db80743a0 0x7f0db819ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:30.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.725+0000 7f0dbdc00640 1 -- 172.21.15.83:0/1914752427 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0db81a3e40 con 0x7f0db8112fd0 2024-04-24T09:07:30.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.725+0000 7f0db7e00640 1 --2- 172.21.15.83:0/1914752427 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0db8074cc0 0x7f0db819f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:07:30.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.725+0000 7f0dbd200640 1 --2- 172.21.15.83:0/1914752427 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0db80743a0 0x7f0db819ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:07:30.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.725+0000 7f0dbdc00640 1 --2- 172.21.15.83:0/1914752427 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0db8112fd0 0x7f0db81a3710 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0db000e9d0 tx=0x7f0db000ee90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:30.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.725+0000 7f0db5600640 1 -- 172.21.15.83:0/1914752427 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db000cd80 con 0x7f0db8112fd0 2024-04-24T09:07:30.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.725+0000 7f0db5600640 1 -- 172.21.15.83:0/1914752427 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0db0005020 con 0x7f0db8112fd0 2024-04-24T09:07:30.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.726+0000 7f0db5600640 1 -- 172.21.15.83:0/1914752427 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db00116d0 con 0x7f0db8112fd0 2024-04-24T09:07:30.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.726+0000 7f0dbfc00640 1 -- 172.21.15.83:0/1914752427 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0db81a4130 con 0x7f0db8112fd0 2024-04-24T09:07:30.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.726+0000 7f0dbfc00640 1 -- 172.21.15.83:0/1914752427 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0db81ab9c0 con 0x7f0db8112fd0 2024-04-24T09:07:30.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.731+0000 7f0dc299a740 1 Processor -- start 2024-04-24T09:07:30.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.731+0000 7f0db5600640 1 -- 172.21.15.83:0/1914752427 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0db0011870 con 0x7f0db8112fd0 2024-04-24T09:07:30.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.731+0000 7f0dc299a740 1 -- start start 2024-04-24T09:07:30.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.732+0000 7f0dc299a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e1a27cf860 0x55e1a27cfc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:30.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.732+0000 7f0dc299a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e1a27d0180 0x55e1a27da310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:30.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.732+0000 7f0dc299a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e1a27da850 0x55e1a27dcc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:30.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.732+0000 7f0dc299a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e1a27351f0 con 0x55e1a27d0180 2024-04-24T09:07:30.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.732+0000 7f0dc299a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e1a26db350 con 0x55e1a27cf860 2024-04-24T09:07:30.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.732+0000 7f0dc299a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55e1a27311b0 con 0x55e1a27da850 2024-04-24T09:07:30.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.733+0000 7f0dbd200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e1a27d0180 0x55e1a27da310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:30.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.733+0000 7f0dbd200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e1a27d0180 0x55e1a27da310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39608/0 (socket says 172.21.15.83:39608) 2024-04-24T09:07:30.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.733+0000 7f0dbd200640 1 -- 172.21.15.83:0/2629758035 learned_addr learned my addr 172.21.15.83:0/2629758035 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:07:30.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.733+0000 7f0db7e00640 1 --2- 172.21.15.83:0/2629758035 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e1a27cf860 0x55e1a27cfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:30.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.733+0000 7f0db5600640 1 --2- 172.21.15.83:0/1914752427 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f0da00a3da0 0x7f0da00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:30.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.733+0000 7f0db7e00640 1 --2- 172.21.15.83:0/2629758035 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e1a27da850 0x55e1a27dcc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:30.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.733+0000 7f0db5600640 1 -- 172.21.15.83:0/1914752427 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 9583+0+0 (secure 0 0 0) 0x7f0db00d0cf0 con 0x7f0db8112fd0 2024-04-24T09:07:30.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.733+0000 7f0dbdc00640 1 --2- 172.21.15.83:0/1914752427 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f0da00a3da0 0x7f0da00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:30.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.733+0000 7f0dbd200640 1 -- 172.21.15.83:0/2629758035 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e1a27da850 msgr2=0x55e1a27dcc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:30.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.733+0000 7f0dbd200640 1 --2- 172.21.15.83:0/2629758035 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e1a27da850 0x55e1a27dcc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:30.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.733+0000 7f0dbd200640 1 -- 172.21.15.83:0/2629758035 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e1a27cf860 msgr2=0x55e1a27cfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:30.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.733+0000 7f0dbd200640 1 --2- 172.21.15.83:0/2629758035 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e1a27cf860 0x55e1a27cfc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:30.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.733+0000 7f0dbd200640 1 -- 172.21.15.83:0/2629758035 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1a272ff50 con 0x55e1a27d0180 2024-04-24T09:07:30.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.733+0000 7f0db7e00640 1 --2- 172.21.15.83:0/2629758035 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e1a27da850 0x55e1a27dcc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:07:30.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.734+0000 7f0dbdc00640 1 --2- 172.21.15.83:0/1914752427 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f0da00a3da0 0x7f0da00a6260 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f0db000c880 tx=0x7f0db000c7f0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:30.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.734+0000 7f0dbd200640 1 --2- 172.21.15.83:0/2629758035 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e1a27d0180 0x55e1a27da310 secure :-1 s=READY pgs=2240 cs=0 l=1 rev1=1 crypto rx=0x7f0da4009230 tx=0x7f0da4005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d0f9dec24546dcc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:30.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.735+0000 7f0daa000640 1 -- 172.21.15.83:0/2629758035 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da4035040 con 0x55e1a27d0180 2024-04-24T09:07:30.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.735+0000 7f0daa000640 1 -- 172.21.15.83:0/2629758035 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0da4004c20 con 0x55e1a27d0180 2024-04-24T09:07:30.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.735+0000 7f0daa000640 1 -- 172.21.15.83:0/2629758035 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da4003a80 con 0x55e1a27d0180 2024-04-24T09:07:30.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.736+0000 7f0dc299a740 1 -- 172.21.15.83:0/2629758035 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e1a27d0180 msgr2=0x55e1a27da310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:30.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.736+0000 7f0dc299a740 1 --2- 172.21.15.83:0/2629758035 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e1a27d0180 0x55e1a27da310 secure :-1 s=READY pgs=2240 cs=0 l=1 rev1=1 crypto rx=0x7f0da4009230 tx=0x7f0da4005c00 comp rx=0 tx=0).stop 2024-04-24T09:07:30.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.736+0000 7f0dc299a740 1 -- 172.21.15.83:0/2629758035 shutdown_connections 2024-04-24T09:07:30.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.736+0000 7f0dc299a740 1 --2- 172.21.15.83:0/2629758035 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e1a27da850 0x55e1a27dcc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:30.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.736+0000 7f0dc299a740 1 --2- 172.21.15.83:0/2629758035 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e1a27d0180 0x55e1a27da310 unknown :-1 s=CLOSED pgs=2240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:30.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.736+0000 7f0dc299a740 1 --2- 172.21.15.83:0/2629758035 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e1a27cf860 0x55e1a27cfc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:30.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.736+0000 7f0dc299a740 1 -- 172.21.15.83:0/2629758035 >> 172.21.15.83:0/2629758035 conn(0x55e1a27cbb80 msgr2=0x55e1a27cbf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:30.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.737+0000 7f0dc299a740 1 -- 172.21.15.83:0/2629758035 shutdown_connections 2024-04-24T09:07:30.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.737+0000 7f0dc299a740 1 -- 172.21.15.83:0/2629758035 wait complete. 2024-04-24T09:07:30.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.738+0000 7f0dc299a740 1 -- start start 2024-04-24T09:07:30.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.738+0000 7f0dc299a740 1 Processor -- start 2024-04-24T09:07:30.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.738+0000 7f0dc299a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e1a27cf860 0x55e1a27d7850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:30.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.738+0000 7f0dc299a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e1a27d0180 0x55e1a27d7d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:30.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.738+0000 7f0dc299a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e1a27da850 0x55e1a27d82d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:30.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.738+0000 7f0dc299a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e1a27dd5e0 con 0x55e1a27cf860 2024-04-24T09:07:30.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.738+0000 7f0dc299a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e1a26ee690 con 0x55e1a27d0180 2024-04-24T09:07:30.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.738+0000 7f0dc299a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55e1a27dd760 con 0x55e1a27da850 2024-04-24T09:07:30.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.738+0000 7f0dc299a740 10 client.0 _ll_register_callbacks cb 0x55e1a272a650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:07:30.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.739+0000 7f0db7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e1a27cf860 0x55e1a27d7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:30.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.739+0000 7f0dbd200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e1a27d0180 0x55e1a27d7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:30.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.739+0000 7f0db7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e1a27cf860 0x55e1a27d7850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39624/0 (socket says 172.21.15.83:39624) 2024-04-24T09:07:30.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.739+0000 7f0db7e00640 1 -- 172.21.15.83:0/1760811660 learned_addr learned my addr 172.21.15.83:0/1760811660 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:07:30.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.739+0000 7f0db7e00640 1 --2- 172.21.15.83:0/1760811660 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e1a27da850 0x55e1a27d82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:30.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.739+0000 7f0db7e00640 1 -- 172.21.15.83:0/1760811660 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e1a27da850 msgr2=0x55e1a27d82d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:30.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.739+0000 7f0db7e00640 1 --2- 172.21.15.83:0/1760811660 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e1a27da850 0x55e1a27d82d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:30.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.739+0000 7f0db7e00640 1 -- 172.21.15.83:0/1760811660 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e1a27d0180 msgr2=0x55e1a27d7d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:30.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.739+0000 7f0db7e00640 1 --2- 172.21.15.83:0/1760811660 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e1a27d0180 0x55e1a27d7d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:30.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.739+0000 7f0db7e00640 1 -- 172.21.15.83:0/1760811660 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1a26ed920 con 0x55e1a27cf860 2024-04-24T09:07:30.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.740+0000 7f0db7e00640 1 --2- 172.21.15.83:0/1760811660 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e1a27cf860 0x55e1a27d7850 secure :-1 s=READY pgs=2241 cs=0 l=1 rev1=1 crypto rx=0x7f0dac00f3f0 tx=0x7f0dac004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=490b91a308dfd3a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:30.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.740+0000 7f0d9ea00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:07:30.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.740+0000 7f0d9ea00640 1 -- 172.21.15.83:0/1760811660 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dac004020 con 0x55e1a27cf860 2024-04-24T09:07:30.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.740+0000 7f0d9ea00640 1 -- 172.21.15.83:0/1760811660 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9583+0+0 (secure 0 0 0) 0x7f0dac0041c0 con 0x55e1a27cf860 2024-04-24T09:07:30.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.741+0000 7f0dc299a740 1 -- 172.21.15.83:0/1760811660 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0da4004c20 con 0x55e1a27cf860 2024-04-24T09:07:30.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.741+0000 7f0d9ea00640 1 -- 172.21.15.83:0/1760811660 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f0dac0043c0 con 0x55e1a27cf860 2024-04-24T09:07:30.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.741+0000 7f0dc299a740 10 client.21114 fetch_fsmap learned FSMap version 51 2024-04-24T09:07:30.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.741+0000 7f0dc299a740 1 -- 172.21.15.83:0/1760811660 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x55e1a25117d0 con 0x55e1a27cf860 2024-04-24T09:07:30.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.742+0000 7f0d9ea00640 1 -- 172.21.15.83:0/1760811660 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0da4004c20 con 0x55e1a27cf860 2024-04-24T09:07:30.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.742+0000 7f0d9ea00640 1 -- 172.21.15.83:0/1760811660 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 51) ==== 8406+0+0 (secure 0 0 0) 0x7f0dac02a020 con 0x55e1a27cf860 2024-04-24T09:07:30.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.742+0000 7f0dc299a740 10 client.21114 fetch_fsmap finished waiting for FSMap version 51 2024-04-24T09:07:30.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.742+0000 7f0dc299a740 10 client.21114 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:07:30.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.742+0000 7f0dc299a740 20 client.21114 populate_metadata read hostname 'smithi083' 2024-04-24T09:07:30.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.742+0000 7f0dc299a740 1 --2- 172.21.15.83:0/1760811660 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55e1a27eacd0 0x55e1a27ed110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:30.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.742+0000 7f0dc299a740 4 client.21114 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:07:30.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.742+0000 7f0dc299a740 1 -- 172.21.15.83:0/1760811660 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e1a2742c10 con 0x55e1a27eacd0 2024-04-24T09:07:30.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.742+0000 7f0dbd200640 1 --2- 172.21.15.83:0/1760811660 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55e1a27eacd0 0x55e1a27ed110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:30.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.743+0000 7f0dbd200640 1 --2- 172.21.15.83:0/1760811660 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55e1a27eacd0 0x55e1a27ed110 crc :-1 s=READY pgs=401 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e34245851abee2b1 server_cookie=5f2599c03e8d5b57 in_seq=0 out_seq=0 2024-04-24T09:07:30.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.743+0000 7f0d9ea00640 10 client.21114 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:07:30.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.746+0000 7f0d9ea00640 1 -- 172.21.15.83:0/1760811660 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e1a2742c10 con 0x55e1a27eacd0 2024-04-24T09:07:30.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.746+0000 7f0d9ea00640 10 client.21114 handle_command_reply: tid=0 2024-04-24T09:07:30.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.746+0000 7f0dc299a740 1 client.21114 shutdown 2024-04-24T09:07:30.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.747+0000 7f0dc299a740 1 -- 172.21.15.83:0/1760811660 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e1a27cf860 msgr2=0x55e1a27d7850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:30.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.747+0000 7f0dc299a740 1 --2- 172.21.15.83:0/1760811660 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e1a27cf860 0x55e1a27d7850 secure :-1 s=READY pgs=2241 cs=0 l=1 rev1=1 crypto rx=0x7f0dac00f3f0 tx=0x7f0dac004730 comp rx=0 tx=0).stop 2024-04-24T09:07:30.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.747+0000 7f0dc299a740 1 -- 172.21.15.83:0/1760811660 shutdown_connections 2024-04-24T09:07:30.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.747+0000 7f0dc299a740 1 --2- 172.21.15.83:0/1760811660 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55e1a27eacd0 0x55e1a27ed110 crc :-1 s=READY pgs=401 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:30.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.747+0000 7f0dc299a740 1 --2- 172.21.15.83:0/1760811660 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e1a27da850 0x55e1a27d82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:30.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.747+0000 7f0dc299a740 1 --2- 172.21.15.83:0/1760811660 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e1a27d0180 0x55e1a27d7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:30.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.747+0000 7f0dc299a740 1 --2- 172.21.15.83:0/1760811660 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e1a27cf860 0x55e1a27d7850 unknown :-1 s=CLOSED pgs=2241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:30.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.747+0000 7f0dc299a740 1 -- 172.21.15.83:0/1760811660 >> 172.21.15.83:0/1760811660 conn(0x55e1a27cbb80 msgr2=0x55e1a27d1880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:30.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.747+0000 7f0d9ea00640 0 client.21114 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:07:30.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.747+0000 7f0dc299a740 1 -- 172.21.15.83:0/1760811660 shutdown_connections 2024-04-24T09:07:30.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.747+0000 7f0dc299a740 1 -- 172.21.15.83:0/1760811660 wait complete. 2024-04-24T09:07:30.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.748+0000 7f0dc299a740 20 client.21114 trim_cache size 0 max 16384 2024-04-24T09:07:30.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.780+0000 7f0dc299a740 1 Processor -- start 2024-04-24T09:07:30.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.780+0000 7f0dc299a740 1 -- start start 2024-04-24T09:07:30.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.781+0000 7f0dc299a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e1a27cf860 0x55e1a27e64a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:30.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.781+0000 7f0dc299a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e1a27d0180 0x55e1a27d69d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:30.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.781+0000 7f0dc299a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e1a27da850 0x55e1a27caaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:30.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.781+0000 7f0dc299a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0db81b5ce0 con 0x55e1a27cf860 2024-04-24T09:07:30.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.781+0000 7f0dc299a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0db81b7970 con 0x55e1a27d0180 2024-04-24T09:07:30.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.781+0000 7f0dc299a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0db81b63b0 con 0x55e1a27da850 2024-04-24T09:07:30.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.781+0000 7f0db7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e1a27cf860 0x55e1a27e64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:30.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.781+0000 7f0db7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e1a27cf860 0x55e1a27e64a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39634/0 (socket says 172.21.15.83:39634) 2024-04-24T09:07:30.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.781+0000 7f0db7e00640 1 -- 172.21.15.83:0/2459812325 learned_addr learned my addr 172.21.15.83:0/2459812325 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:07:30.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.781+0000 7f0dbd200640 1 --2- 172.21.15.83:0/2459812325 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e1a27d0180 0x55e1a27d69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:30.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.781+0000 7f0db7e00640 1 --2- 172.21.15.83:0/2459812325 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e1a27da850 0x55e1a27caaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:30.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.782+0000 7f0dbd200640 1 -- 172.21.15.83:0/2459812325 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e1a27da850 msgr2=0x55e1a27caaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:30.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.782+0000 7f0dbd200640 1 --2- 172.21.15.83:0/2459812325 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e1a27da850 0x55e1a27caaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:30.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.782+0000 7f0dbd200640 1 -- 172.21.15.83:0/2459812325 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e1a27cf860 msgr2=0x55e1a27e64a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:30.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.782+0000 7f0dbd200640 1 --2- 172.21.15.83:0/2459812325 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e1a27cf860 0x55e1a27e64a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:30.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.782+0000 7f0dbd200640 1 -- 172.21.15.83:0/2459812325 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1a24473f0 con 0x55e1a27d0180 2024-04-24T09:07:30.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.782+0000 7f0db7e00640 1 --2- 172.21.15.83:0/2459812325 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e1a27cf860 0x55e1a27e64a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:07:30.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.782+0000 7f0dbd200640 1 --2- 172.21.15.83:0/2459812325 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e1a27d0180 0x55e1a27d69d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0da4005ab0 tx=0x7f0da402a490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3ffeff07fc632d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:30.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.783+0000 7f0da9600640 1 -- 172.21.15.83:0/2459812325 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da4035040 con 0x55e1a27d0180 2024-04-24T09:07:30.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.783+0000 7f0da9600640 1 -- 172.21.15.83:0/2459812325 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0da4005630 con 0x55e1a27d0180 2024-04-24T09:07:30.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.783+0000 7f0da9600640 1 -- 172.21.15.83:0/2459812325 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da4053460 con 0x55e1a27d0180 2024-04-24T09:07:30.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.784+0000 7f0dc299a740 1 -- 172.21.15.83:0/2459812325 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e1a27d0180 msgr2=0x55e1a27d69d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:30.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.784+0000 7f0dc299a740 1 --2- 172.21.15.83:0/2459812325 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e1a27d0180 0x55e1a27d69d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0da4005ab0 tx=0x7f0da402a490 comp rx=0 tx=0).stop 2024-04-24T09:07:30.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.784+0000 7f0dc299a740 1 -- 172.21.15.83:0/2459812325 shutdown_connections 2024-04-24T09:07:30.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.784+0000 7f0dc299a740 1 --2- 172.21.15.83:0/2459812325 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e1a27da850 0x55e1a27caaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:30.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.784+0000 7f0dc299a740 1 --2- 172.21.15.83:0/2459812325 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e1a27d0180 0x55e1a27d69d0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:30.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.784+0000 7f0dc299a740 1 --2- 172.21.15.83:0/2459812325 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e1a27cf860 0x55e1a27e64a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:30.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.784+0000 7f0dc299a740 1 -- 172.21.15.83:0/2459812325 >> 172.21.15.83:0/2459812325 conn(0x55e1a27cbb80 msgr2=0x55e1a27ca390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:30.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.784+0000 7f0dc299a740 1 -- 172.21.15.83:0/2459812325 shutdown_connections 2024-04-24T09:07:30.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.784+0000 7f0dc299a740 1 -- 172.21.15.83:0/2459812325 wait complete. 2024-04-24T09:07:30.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.784+0000 7f0dc299a740 1 -- start start 2024-04-24T09:07:30.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.785+0000 7f0dc299a740 1 Processor -- start 2024-04-24T09:07:30.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.785+0000 7f0dc299a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e1a27cf860 0x55e1a27e2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:30.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.785+0000 7f0dc299a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e1a27d0180 0x55e1a27d4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:30.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.785+0000 7f0dc299a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e1a27da850 0x55e1a27d4a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:30.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.785+0000 7f0dc299a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0db81b71e0 con 0x55e1a27da850 2024-04-24T09:07:30.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.785+0000 7f0dc299a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0db81b6ae0 con 0x55e1a27cf860 2024-04-24T09:07:30.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.785+0000 7f0dc299a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0db81b3ce0 con 0x55e1a27d0180 2024-04-24T09:07:30.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.785+0000 7f0dc299a740 10 client.0 _ll_register_callbacks cb 0x7f0dac011650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:07:30.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.786+0000 7f0db7e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e1a27cf860 0x55e1a27e2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:30.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.786+0000 7f0db7e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e1a27cf860 0x55e1a27e2f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:42724/0 (socket says 172.21.15.83:42724) 2024-04-24T09:07:30.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.786+0000 7f0db7e00640 1 -- 172.21.15.83:0/2226637661 learned_addr learned my addr 172.21.15.83:0/2226637661 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:07:30.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.786+0000 7f0db7e00640 1 --2- 172.21.15.83:0/2226637661 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e1a27da850 0x55e1a27d4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:30.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.786+0000 7f0dbd200640 1 --2- 172.21.15.83:0/2226637661 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e1a27d0180 0x55e1a27d4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:30.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.786+0000 7f0db7e00640 1 -- 172.21.15.83:0/2226637661 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e1a27d0180 msgr2=0x55e1a27d4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:30.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.786+0000 7f0db7e00640 1 --2- 172.21.15.83:0/2226637661 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e1a27d0180 0x55e1a27d4360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:30.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.786+0000 7f0db7e00640 1 -- 172.21.15.83:0/2226637661 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e1a27da850 msgr2=0x55e1a27d4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:30.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.786+0000 7f0db7e00640 1 --2- 172.21.15.83:0/2226637661 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e1a27da850 0x55e1a27d4a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:30.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.786+0000 7f0db7e00640 1 -- 172.21.15.83:0/2226637661 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1a24070f0 con 0x55e1a27cf860 2024-04-24T09:07:30.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.787+0000 7f0db7e00640 1 --2- 172.21.15.83:0/2226637661 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e1a27cf860 0x55e1a27e2f00 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0dac0058c0 tx=0x7f0dac005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b552020ce0c405f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:30.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.787+0000 7f0d9ea00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:07:30.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.787+0000 7f0d9ea00640 1 -- 172.21.15.83:0/2226637661 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dac01a040 con 0x55e1a27cf860 2024-04-24T09:07:30.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.787+0000 7f0dc299a740 1 -- 172.21.15.83:0/2226637661 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e1a273ca80 con 0x55e1a27cf860 2024-04-24T09:07:30.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.787+0000 7f0d9ea00640 1 -- 172.21.15.83:0/2226637661 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9583+0+0 (secure 0 0 0) 0x7f0dac011260 con 0x55e1a27cf860 2024-04-24T09:07:30.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.787+0000 7f0d9ea00640 1 -- 172.21.15.83:0/2226637661 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f0dac004e90 con 0x55e1a27cf860 2024-04-24T09:07:30.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.787+0000 7f0dc299a740 10 client.38273 fetch_fsmap learned FSMap version 51 2024-04-24T09:07:30.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.787+0000 7f0dc299a740 1 -- 172.21.15.83:0/2226637661 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x55e1a26fa810 con 0x55e1a27cf860 2024-04-24T09:07:30.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.788+0000 7f0d9ea00640 1 -- 172.21.15.83:0/2226637661 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55e1a273ca80 con 0x55e1a27cf860 2024-04-24T09:07:30.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.788+0000 7f0d9ea00640 1 -- 172.21.15.83:0/2226637661 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 51) ==== 8406+0+0 (secure 0 0 0) 0x7f0dac02a040 con 0x55e1a27cf860 2024-04-24T09:07:30.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.788+0000 7f0dc299a740 10 client.38273 fetch_fsmap finished waiting for FSMap version 51 2024-04-24T09:07:30.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.788+0000 7f0dc299a740 10 client.38273 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:07:30.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.788+0000 7f0dc299a740 20 client.38273 populate_metadata read hostname 'smithi083' 2024-04-24T09:07:30.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.788+0000 7f0dc299a740 1 --2- 172.21.15.83:0/2226637661 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55e1a2862780 0x55e1a2864ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:30.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.788+0000 7f0dc299a740 4 client.38273 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T09:07:30.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.788+0000 7f0dc299a740 1 -- 172.21.15.83:0/2226637661 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e1a26b8360 con 0x55e1a2862780 2024-04-24T09:07:30.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.788+0000 7f0dbd200640 1 --2- 172.21.15.83:0/2226637661 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55e1a2862780 0x55e1a2864ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:30.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.789+0000 7f0dbd200640 1 --2- 172.21.15.83:0/2226637661 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55e1a2862780 0x55e1a2864ba0 crc :-1 s=READY pgs=402 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88f46b2ea9821ef7 server_cookie=4c1c3f6bf49fede3 in_seq=0 out_seq=0 2024-04-24T09:07:30.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.789+0000 7f0d9ea00640 10 client.38273 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:07:30.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.789+0000 7f0d9ea00640 1 -- 172.21.15.83:0/2226637661 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55e1a26b8360 con 0x55e1a2862780 2024-04-24T09:07:30.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.789+0000 7f0d9ea00640 10 client.38273 handle_command_reply: tid=0 2024-04-24T09:07:30.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.789+0000 7f0dc299a740 1 client.38273 shutdown 2024-04-24T09:07:30.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.790+0000 7f0dc299a740 1 -- 172.21.15.83:0/2226637661 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e1a27cf860 msgr2=0x55e1a27e2f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:30.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.790+0000 7f0dc299a740 1 --2- 172.21.15.83:0/2226637661 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e1a27cf860 0x55e1a27e2f00 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0dac0058c0 tx=0x7f0dac005850 comp rx=0 tx=0).stop 2024-04-24T09:07:30.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.790+0000 7f0dc299a740 1 -- 172.21.15.83:0/2226637661 shutdown_connections 2024-04-24T09:07:30.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.790+0000 7f0dc299a740 1 --2- 172.21.15.83:0/2226637661 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55e1a2862780 0x55e1a2864ba0 crc :-1 s=READY pgs=402 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:30.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.790+0000 7f0dc299a740 1 --2- 172.21.15.83:0/2226637661 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e1a27da850 0x55e1a27d4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:30.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.790+0000 7f0dc299a740 1 --2- 172.21.15.83:0/2226637661 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e1a27d0180 0x55e1a27d4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:30.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.790+0000 7f0dc299a740 1 --2- 172.21.15.83:0/2226637661 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e1a27cf860 0x55e1a27e2f00 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:30.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.790+0000 7f0dc299a740 1 -- 172.21.15.83:0/2226637661 >> 172.21.15.83:0/2226637661 conn(0x55e1a27cbb80 msgr2=0x55e1a27cac20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:30.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.790+0000 7f0d9ea00640 0 client.38273 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:07:30.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.790+0000 7f0dc299a740 1 -- 172.21.15.83:0/2226637661 shutdown_connections 2024-04-24T09:07:30.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.790+0000 7f0dc299a740 1 -- 172.21.15.83:0/2226637661 wait complete. 2024-04-24T09:07:30.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.790+0000 7f0dc299a740 20 client.38273 trim_cache size 0 max 16384 2024-04-24T09:07:30.792 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T09:07:30.792 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-04-24T09:07:30.793 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-04-24T09:07:30.793 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T09:07:30.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.791+0000 7f0dbfc00640 1 -- 172.21.15.83:0/1914752427 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f0da00a3da0 msgr2=0x7f0da00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:30.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.791+0000 7f0dbfc00640 1 --2- 172.21.15.83:0/1914752427 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f0da00a3da0 0x7f0da00a6260 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f0db000c880 tx=0x7f0db000c7f0 comp rx=0 tx=0).stop 2024-04-24T09:07:30.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.791+0000 7f0dbfc00640 1 -- 172.21.15.83:0/1914752427 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0db8112fd0 msgr2=0x7f0db81a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:30.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.791+0000 7f0dbfc00640 1 --2- 172.21.15.83:0/1914752427 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0db8112fd0 0x7f0db81a3710 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0db000e9d0 tx=0x7f0db000ee90 comp rx=0 tx=0).stop 2024-04-24T09:07:30.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.791+0000 7f0dbfc00640 1 -- 172.21.15.83:0/1914752427 shutdown_connections 2024-04-24T09:07:30.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.791+0000 7f0dbfc00640 1 --2- 172.21.15.83:0/1914752427 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f0da00a3da0 0x7f0da00a6260 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:30.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.791+0000 7f0dbfc00640 1 --2- 172.21.15.83:0/1914752427 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0db8112fd0 0x7f0db81a3710 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:30.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.791+0000 7f0dbfc00640 1 --2- 172.21.15.83:0/1914752427 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0db8074cc0 0x7f0db819f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:30.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.791+0000 7f0dbfc00640 1 --2- 172.21.15.83:0/1914752427 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0db80743a0 0x7f0db819ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:30.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.791+0000 7f0dbfc00640 1 -- 172.21.15.83:0/1914752427 >> 172.21.15.83:0/1914752427 conn(0x7f0db80fda70 msgr2=0x7f0db80ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:30.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.792+0000 7f0dbfc00640 1 -- 172.21.15.83:0/1914752427 shutdown_connections 2024-04-24T09:07:30.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:30.792+0000 7f0dbfc00640 1 -- 172.21.15.83:0/1914752427 wait complete. 2024-04-24T09:07:30.812 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T09:07:30.813 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T09:07:31.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.076+0000 7f7c22800640 1 -- 172.21.15.83:0/369184914 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c1c101000 msgr2=0x7f7c1c10d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:31.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.076+0000 7f7c22800640 1 --2- 172.21.15.83:0/369184914 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c1c101000 0x7f7c1c10d510 secure :-1 s=READY pgs=2242 cs=0 l=1 rev1=1 crypto rx=0x7f7c14009ca0 tx=0x7f7c1402a8c0 comp rx=0 tx=0).stop 2024-04-24T09:07:31.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.077+0000 7f7c22800640 1 -- 172.21.15.83:0/369184914 shutdown_connections 2024-04-24T09:07:31.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.077+0000 7f7c22800640 1 --2- 172.21.15.83:0/369184914 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c1c101000 0x7f7c1c10d510 unknown :-1 s=CLOSED pgs=2242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:31.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.077+0000 7f7c22800640 1 --2- 172.21.15.83:0/369184914 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7c1c100640 0x7f7c1c100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:31.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.077+0000 7f7c22800640 1 --2- 172.21.15.83:0/369184914 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c1c106650 0x7f7c1c106a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:31.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.077+0000 7f7c22800640 1 -- 172.21.15.83:0/369184914 >> 172.21.15.83:0/369184914 conn(0x7f7c1c0774a0 msgr2=0x7f7c1c0fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:31.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.078+0000 7f7c22800640 1 -- 172.21.15.83:0/369184914 shutdown_connections 2024-04-24T09:07:31.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.079+0000 7f7c22800640 1 -- 172.21.15.83:0/369184914 wait complete. 2024-04-24T09:07:31.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.080+0000 7f7c22800640 1 Processor -- start 2024-04-24T09:07:31.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.080+0000 7f7c22800640 1 -- start start 2024-04-24T09:07:31.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.081+0000 7f7c22800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7c1c100640 0x7f7c1c10ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:31.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.081+0000 7f7c22800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c1c101000 0x7f7c1c10f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:31.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.081+0000 7f7c22800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c1c106650 0x7f7c1c10f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:31.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.081+0000 7f7c22800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7c1c112270 con 0x7f7c1c106650 2024-04-24T09:07:31.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.081+0000 7f7c22800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7c1c1120f0 con 0x7f7c1c101000 2024-04-24T09:07:31.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.081+0000 7f7c22800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7c1c1123f0 con 0x7f7c1c100640 2024-04-24T09:07:31.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.082+0000 7f7c1aa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c1c101000 0x7f7c1c10f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:31.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.082+0000 7f7c1aa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c1c101000 0x7f7c1c10f170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:42748/0 (socket says 172.21.15.83:42748) 2024-04-24T09:07:31.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.082+0000 7f7c1aa00640 1 -- 172.21.15.83:0/3565885439 learned_addr learned my addr 172.21.15.83:0/3565885439 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:07:31.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.082+0000 7f7c1b400640 1 --2- 172.21.15.83:0/3565885439 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7c1c100640 0x7f7c1c10ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:31.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.082+0000 7f7c1be00640 1 --2- 172.21.15.83:0/3565885439 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c1c106650 0x7f7c1c10f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:31.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.082+0000 7f7c1aa00640 1 -- 172.21.15.83:0/3565885439 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7c1c100640 msgr2=0x7f7c1c10ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:31.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.082+0000 7f7c1aa00640 1 --2- 172.21.15.83:0/3565885439 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7c1c100640 0x7f7c1c10ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:31.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.082+0000 7f7c1aa00640 1 -- 172.21.15.83:0/3565885439 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c1c106650 msgr2=0x7f7c1c10f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:31.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.083+0000 7f7c1aa00640 1 --2- 172.21.15.83:0/3565885439 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c1c106650 0x7f7c1c10f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:31.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.083+0000 7f7c1aa00640 1 -- 172.21.15.83:0/3565885439 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c1c1a6c70 con 0x7f7c1c101000 2024-04-24T09:07:31.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.083+0000 7f7c1b400640 1 --2- 172.21.15.83:0/3565885439 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7c1c100640 0x7f7c1c10ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:07:31.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.083+0000 7f7c1be00640 1 --2- 172.21.15.83:0/3565885439 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c1c106650 0x7f7c1c10f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:07:31.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.083+0000 7f7c1aa00640 1 --2- 172.21.15.83:0/3565885439 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c1c101000 0x7f7c1c10f170 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7c0800c920 tx=0x7f7c0800cde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:31.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.083+0000 7f7c13e00640 1 -- 172.21.15.83:0/3565885439 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c08007c70 con 0x7f7c1c101000 2024-04-24T09:07:31.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.083+0000 7f7c13e00640 1 -- 172.21.15.83:0/3565885439 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7c08007e10 con 0x7f7c1c101000 2024-04-24T09:07:31.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.083+0000 7f7c13e00640 1 -- 172.21.15.83:0/3565885439 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c08018820 con 0x7f7c1c101000 2024-04-24T09:07:31.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.084+0000 7f7c22800640 1 -- 172.21.15.83:0/3565885439 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c1c1a6f60 con 0x7f7c1c101000 2024-04-24T09:07:31.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.084+0000 7f7c22800640 1 -- 172.21.15.83:0/3565885439 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c1c1a7370 con 0x7f7c1c101000 2024-04-24T09:07:31.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.085+0000 7f7c22800640 1 -- 172.21.15.83:0/3565885439 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c1c068590 con 0x7f7c1c101000 2024-04-24T09:07:31.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.090+0000 7f7c13e00640 1 -- 172.21.15.83:0/3565885439 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7c080189c0 con 0x7f7c1c101000 2024-04-24T09:07:31.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.091+0000 7f7c13e00640 1 --2- 172.21.15.83:0/3565885439 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f7c000a3e70 0x7f7c000a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:31.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.092+0000 7f7c13e00640 1 -- 172.21.15.83:0/3565885439 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 9583+0+0 (secure 0 0 0) 0x7f7c080d0c90 con 0x7f7c1c101000 2024-04-24T09:07:31.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.092+0000 7f7c1b400640 1 --2- 172.21.15.83:0/3565885439 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f7c000a3e70 0x7f7c000a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:31.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.093+0000 7f7c1b400640 1 --2- 172.21.15.83:0/3565885439 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f7c000a3e70 0x7f7c000a6330 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c009ba0 tx=0x7f7c0c005f90 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:31.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.101+0000 7f7c13e00640 1 -- 172.21.15.83:0/3565885439 <== mon.2 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7c08014030 con 0x7f7c1c101000 2024-04-24T09:07:31.147 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.147+0000 7efc12c00640 1 -- 172.21.15.83:0/3407570432 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc0c104450 msgr2=0x7efc0c104830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:31.147 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.147+0000 7efc12c00640 1 --2- 172.21.15.83:0/3407570432 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc0c104450 0x7efc0c104830 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7efbfc009900 tx=0x7efbfc02a3f0 comp rx=0 tx=0).stop 2024-04-24T09:07:31.147 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.147+0000 7efc12c00640 1 -- 172.21.15.83:0/3407570432 shutdown_connections 2024-04-24T09:07:31.147 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.147+0000 7efc12c00640 1 --2- 172.21.15.83:0/3407570432 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc0c109520 0x7efc0c1110f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:31.147 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.147+0000 7efc12c00640 1 --2- 172.21.15.83:0/3407570432 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efc0c104e00 0x7efc0c108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:31.147 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.147+0000 7efc12c00640 1 --2- 172.21.15.83:0/3407570432 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc0c104450 0x7efc0c104830 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:31.147 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.147+0000 7efc12c00640 1 -- 172.21.15.83:0/3407570432 >> 172.21.15.83:0/3407570432 conn(0x7efc0c0ffcf0 msgr2=0x7efc0c102110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:31.147 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.147+0000 7efc12c00640 1 -- 172.21.15.83:0/3407570432 shutdown_connections 2024-04-24T09:07:31.148 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.147+0000 7efc12c00640 1 -- 172.21.15.83:0/3407570432 wait complete. 2024-04-24T09:07:31.148 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.148+0000 7efc12c00640 1 Processor -- start 2024-04-24T09:07:31.148 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.148+0000 7efc12c00640 1 -- start start 2024-04-24T09:07:31.149 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.149+0000 7efc12c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efc0c104450 0x7efc0c076510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:31.149 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.149+0000 7efc12c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc0c109520 0x7efc0c076a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:31.149 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.149+0000 7efc12c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc0c076f90 0x7efc0c1aafb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:31.149 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.149+0000 7efc12c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7efc0c114480 con 0x7efc0c109520 2024-04-24T09:07:31.149 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.149+0000 7efc12c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efc0c114300 con 0x7efc0c076f90 2024-04-24T09:07:31.149 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.149+0000 7efc12c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7efc0c114600 con 0x7efc0c104450 2024-04-24T09:07:31.150 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.149+0000 7efc10c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc0c076f90 0x7efc0c1aafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:31.150 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.149+0000 7efc10c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc0c076f90 0x7efc0c1aafb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:42772/0 (socket says 172.21.15.83:42772) 2024-04-24T09:07:31.150 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.149+0000 7efc10c00640 1 -- 172.21.15.83:0/3998550175 learned_addr learned my addr 172.21.15.83:0/3998550175 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:07:31.150 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.150+0000 7efc0b400640 1 --2- 172.21.15.83:0/3998550175 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc0c109520 0x7efc0c076a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:31.150 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.150+0000 7efc0be00640 1 --2- 172.21.15.83:0/3998550175 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efc0c104450 0x7efc0c076510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:31.151 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.150+0000 7efc0b400640 1 -- 172.21.15.83:0/3998550175 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efc0c104450 msgr2=0x7efc0c076510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:31.151 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.150+0000 7efc0b400640 1 --2- 172.21.15.83:0/3998550175 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efc0c104450 0x7efc0c076510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:31.151 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.150+0000 7efc0b400640 1 -- 172.21.15.83:0/3998550175 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc0c076f90 msgr2=0x7efc0c1aafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:31.151 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.150+0000 7efc0b400640 1 --2- 172.21.15.83:0/3998550175 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc0c076f90 0x7efc0c1aafb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:31.151 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.150+0000 7efc0b400640 1 -- 172.21.15.83:0/3998550175 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc0c1ab4f0 con 0x7efc0c109520 2024-04-24T09:07:31.151 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.151+0000 7efc0b400640 1 --2- 172.21.15.83:0/3998550175 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc0c109520 0x7efc0c076a50 secure :-1 s=READY pgs=2243 cs=0 l=1 rev1=1 crypto rx=0x7efc0c06b770 tx=0x7efbf800dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:31.152 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.151+0000 7efc08c00640 1 -- 172.21.15.83:0/3998550175 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf8017070 con 0x7efc0c109520 2024-04-24T09:07:31.152 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.151+0000 7efc08c00640 1 -- 172.21.15.83:0/3998550175 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efbf8005020 con 0x7efc0c109520 2024-04-24T09:07:31.152 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.151+0000 7efc08c00640 1 -- 172.21.15.83:0/3998550175 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf8004c30 con 0x7efc0c109520 2024-04-24T09:07:31.152 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.151+0000 7efc12c00640 1 -- 172.21.15.83:0/3998550175 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc0c1ab780 con 0x7efc0c109520 2024-04-24T09:07:31.153 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.153+0000 7efc12c00640 1 -- 172.21.15.83:0/3998550175 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc0c1abc50 con 0x7efc0c109520 2024-04-24T09:07:31.157 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.157+0000 7efc08c00640 1 -- 172.21.15.83:0/3998550175 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7efbf80051d0 con 0x7efc0c109520 2024-04-24T09:07:31.158 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.158+0000 7efc159e1740 1 Processor -- start 2024-04-24T09:07:31.158 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.158+0000 7efc159e1740 1 -- start start 2024-04-24T09:07:31.159 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.159+0000 7efc159e1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ad3e0701b0 0x55ad3e070590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:31.159 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.159+0000 7efc159e1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ad3e070ad0 0x55ad3e07ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:31.160 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.159+0000 7efc159e1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ad3e07b390 0x55ad3e07d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:31.160 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.160+0000 7efc159e1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ad3dfd6900 con 0x55ad3e07b390 2024-04-24T09:07:31.160 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.160+0000 7efc159e1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ad3df7cf20 con 0x55ad3e070ad0 2024-04-24T09:07:31.160 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.160+0000 7efc159e1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55ad3dfd28c0 con 0x55ad3e0701b0 2024-04-24T09:07:31.161 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.160+0000 7efc10c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ad3e0701b0 0x55ad3e070590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:31.161 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.160+0000 7efc08c00640 1 --2- 172.21.15.83:0/3998550175 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7efbf00a3dc0 0x7efbf00a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:31.161 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.160+0000 7efc0b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ad3e07b390 0x55ad3e07d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:31.161 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.160+0000 7efc08c00640 1 -- 172.21.15.83:0/3998550175 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 9583+0+0 (secure 0 0 0) 0x7efbf80d3170 con 0x7efc0c109520 2024-04-24T09:07:31.161 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.160+0000 7efc0b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ad3e07b390 0x55ad3e07d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39700/0 (socket says 172.21.15.83:39700) 2024-04-24T09:07:31.161 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.160+0000 7efc0be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ad3e070ad0 0x55ad3e07ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:31.162 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.160+0000 7efc0be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ad3e070ad0 0x55ad3e07ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:42784/0 (socket says 172.21.15.83:42784) 2024-04-24T09:07:31.162 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.160+0000 7efc0be00640 1 -- 172.21.15.83:0/755137897 learned_addr learned my addr 172.21.15.83:0/755137897 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:07:31.162 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.160+0000 7efc10c00640 1 --2- 172.21.15.83:0/3998550175 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7efbf00a3dc0 0x7efbf00a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:31.162 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.161+0000 7efc0b400640 1 -- 172.21.15.83:0/755137897 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ad3e0701b0 msgr2=0x55ad3e070590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:31.162 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.161+0000 7efc0b400640 1 --2- 172.21.15.83:0/755137897 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ad3e0701b0 0x55ad3e070590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:31.162 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.161+0000 7efc0b400640 1 -- 172.21.15.83:0/755137897 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ad3e070ad0 msgr2=0x55ad3e07ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:31.162 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.161+0000 7efc0b400640 1 --2- 172.21.15.83:0/755137897 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ad3e070ad0 0x55ad3e07ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:31.162 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.161+0000 7efc0b400640 1 -- 172.21.15.83:0/755137897 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad3dfa1e10 con 0x55ad3e07b390 2024-04-24T09:07:31.163 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.161+0000 7efc0b400640 1 --2- 172.21.15.83:0/755137897 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ad3e07b390 0x55ad3e07d780 secure :-1 s=READY pgs=2244 cs=0 l=1 rev1=1 crypto rx=0x7efbf80cccf0 tx=0x7efbf806e030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3241799a08619ff server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:31.163 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.161+0000 7efc10c00640 1 --2- 172.21.15.83:0/3998550175 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7efbf00a3dc0 0x7efbf00a6280 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7efc04009fd0 tx=0x7efc040091e0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:31.163 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.161+0000 7efc01200640 1 -- 172.21.15.83:0/755137897 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf80d4060 con 0x55ad3e07b390 2024-04-24T09:07:31.163 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.162+0000 7efc01200640 1 -- 172.21.15.83:0/755137897 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efbf809bc40 con 0x55ad3e07b390 2024-04-24T09:07:31.163 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.162+0000 7efc01200640 1 -- 172.21.15.83:0/755137897 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf8097bb0 con 0x55ad3e07b390 2024-04-24T09:07:31.163 INFO:tasks.workunit.client.0.smithi083.stdout: 10 15815 5.70 MB/sec execute 2 sec latency 80.723 ms 2024-04-24T09:07:31.164 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.163+0000 7efc159e1740 1 -- 172.21.15.83:0/755137897 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ad3e07b390 msgr2=0x55ad3e07d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:31.164 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.163+0000 7efc159e1740 1 --2- 172.21.15.83:0/755137897 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ad3e07b390 0x55ad3e07d780 secure :-1 s=READY pgs=2244 cs=0 l=1 rev1=1 crypto rx=0x7efbf80cccf0 tx=0x7efbf806e030 comp rx=0 tx=0).stop 2024-04-24T09:07:31.164 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.163+0000 7efc159e1740 1 -- 172.21.15.83:0/755137897 shutdown_connections 2024-04-24T09:07:31.164 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.163+0000 7efc159e1740 1 --2- 172.21.15.83:0/755137897 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ad3e07b390 0x55ad3e07d780 unknown :-1 s=CLOSED pgs=2244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:31.164 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.163+0000 7efc159e1740 1 --2- 172.21.15.83:0/755137897 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ad3e070ad0 0x55ad3e07ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:31.164 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.163+0000 7efc159e1740 1 --2- 172.21.15.83:0/755137897 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ad3e0701b0 0x55ad3e070590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:31.164 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.163+0000 7efc159e1740 1 -- 172.21.15.83:0/755137897 >> 172.21.15.83:0/755137897 conn(0x55ad3e06c6c0 msgr2=0x55ad3e06caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:31.164 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.164+0000 7efc159e1740 1 -- 172.21.15.83:0/755137897 shutdown_connections 2024-04-24T09:07:31.164 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.164+0000 7efc159e1740 1 -- 172.21.15.83:0/755137897 wait complete. 2024-04-24T09:07:31.165 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.165+0000 7efc159e1740 1 -- start start 2024-04-24T09:07:31.165 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.165+0000 7efc159e1740 1 Processor -- start 2024-04-24T09:07:31.167 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.166+0000 7efc159e1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ad3e0701b0 0x55ad3e078290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:31.167 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.166+0000 7efc159e1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ad3e070ad0 0x55ad3e0787d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:31.167 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.166+0000 7efc159e1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ad3e07b390 0x55ad3e078d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:31.167 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.166+0000 7efc159e1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ad3e07df60 con 0x55ad3e07b390 2024-04-24T09:07:31.167 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.166+0000 7efc159e1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ad3df902c0 con 0x55ad3e0701b0 2024-04-24T09:07:31.167 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.166+0000 7efc159e1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55ad3e07e0e0 con 0x55ad3e070ad0 2024-04-24T09:07:31.167 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.166+0000 7efc159e1740 10 client.0 _ll_register_callbacks cb 0x55ad3dce8a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:07:31.168 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.166+0000 7efc10c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ad3e07b390 0x55ad3e078d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:31.168 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.167+0000 7efc0be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ad3e0701b0 0x55ad3e078290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:31.168 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.167+0000 7efc0be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ad3e0701b0 0x55ad3e078290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:42792/0 (socket says 172.21.15.83:42792) 2024-04-24T09:07:31.168 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.167+0000 7efc0be00640 1 -- 172.21.15.83:0/3024326131 learned_addr learned my addr 172.21.15.83:0/3024326131 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:07:31.169 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.167+0000 7efc0b400640 1 --2- 172.21.15.83:0/3024326131 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ad3e070ad0 0x55ad3e0787d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:31.169 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.168+0000 7efc0be00640 1 -- 172.21.15.83:0/3024326131 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ad3e070ad0 msgr2=0x55ad3e0787d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:31.169 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.168+0000 7efc0be00640 1 --2- 172.21.15.83:0/3024326131 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ad3e070ad0 0x55ad3e0787d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:31.169 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.168+0000 7efc0be00640 1 -- 172.21.15.83:0/3024326131 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ad3e07b390 msgr2=0x55ad3e078d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:31.169 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.168+0000 7efc0be00640 1 --2- 172.21.15.83:0/3024326131 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ad3e07b390 0x55ad3e078d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:31.169 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.168+0000 7efc0be00640 1 -- 172.21.15.83:0/3024326131 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad3dfa0fd0 con 0x55ad3e0701b0 2024-04-24T09:07:31.169 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.168+0000 7efc10c00640 1 --2- 172.21.15.83:0/3024326131 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ad3e07b390 0x55ad3e078d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:07:31.170 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.169+0000 7efc0be00640 1 --2- 172.21.15.83:0/3024326131 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ad3e0701b0 0x55ad3e078290 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7efbfc009230 tx=0x7efbfc0316d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9bc69083a38d3bb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:31.170 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.169+0000 7efbf5600640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:07:31.170 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.170+0000 7efbf5600640 1 -- 172.21.15.83:0/3024326131 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbfc035040 con 0x55ad3e0701b0 2024-04-24T09:07:31.171 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.170+0000 7efbf5600640 1 -- 172.21.15.83:0/3024326131 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9583+0+0 (secure 0 0 0) 0x7efbfc004240 con 0x55ad3e0701b0 2024-04-24T09:07:31.171 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.170+0000 7efc159e1740 1 -- 172.21.15.83:0/3024326131 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efbf809bc40 con 0x55ad3e0701b0 2024-04-24T09:07:31.171 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.171+0000 7efbf5600640 1 -- 172.21.15.83:0/3024326131 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7efbfc031900 con 0x55ad3e0701b0 2024-04-24T09:07:31.171 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.171+0000 7efc159e1740 10 client.38291 fetch_fsmap learned FSMap version 51 2024-04-24T09:07:31.171 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.171+0000 7efc159e1740 1 -- 172.21.15.83:0/3024326131 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x55ad3dce9210 con 0x55ad3e0701b0 2024-04-24T09:07:31.172 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.172+0000 7efbf5600640 1 -- 172.21.15.83:0/3024326131 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efbf809bc40 con 0x55ad3e0701b0 2024-04-24T09:07:31.172 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.172+0000 7efbf5600640 1 -- 172.21.15.83:0/3024326131 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 51) ==== 8406+0+0 (secure 0 0 0) 0x7efbfc004ad0 con 0x55ad3e0701b0 2024-04-24T09:07:31.173 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.172+0000 7efc159e1740 10 client.38291 fetch_fsmap finished waiting for FSMap version 51 2024-04-24T09:07:31.173 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.172+0000 7efc159e1740 10 client.38291 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:07:31.173 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.173+0000 7efc159e1740 20 client.38291 populate_metadata read hostname 'smithi083' 2024-04-24T09:07:31.173 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.173+0000 7efc159e1740 1 --2- 172.21.15.83:0/3024326131 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ad3e09c3e0 0x55ad3e09e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:31.173 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.173+0000 7efc159e1740 4 client.38291 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:07:31.173 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.173+0000 7efc159e1740 1 -- 172.21.15.83:0/3024326131 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ad3dfe4320 con 0x55ad3e09c3e0 2024-04-24T09:07:31.174 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.173+0000 7efc0b400640 1 --2- 172.21.15.83:0/3024326131 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ad3e09c3e0 0x55ad3e09e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:31.174 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.174+0000 7efc0b400640 1 --2- 172.21.15.83:0/3024326131 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ad3e09c3e0 0x55ad3e09e7f0 crc :-1 s=READY pgs=403 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93c5e94bfd99b717 server_cookie=77bf29cfefa94933 in_seq=0 out_seq=0 2024-04-24T09:07:31.174 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.174+0000 7efbf5600640 10 client.38291 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:07:31.176 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.176+0000 7efbf5600640 1 -- 172.21.15.83:0/3024326131 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ad3dfe4320 con 0x55ad3e09c3e0 2024-04-24T09:07:31.177 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.177+0000 7efbf5600640 10 client.38291 handle_command_reply: tid=0 2024-04-24T09:07:31.177 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.177+0000 7efc159e1740 1 client.38291 shutdown 2024-04-24T09:07:31.177 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.177+0000 7efc159e1740 1 -- 172.21.15.83:0/3024326131 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ad3e0701b0 msgr2=0x55ad3e078290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:31.177 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.177+0000 7efc159e1740 1 --2- 172.21.15.83:0/3024326131 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ad3e0701b0 0x55ad3e078290 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7efbfc009230 tx=0x7efbfc0316d0 comp rx=0 tx=0).stop 2024-04-24T09:07:31.177 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.178+0000 7efc159e1740 1 -- 172.21.15.83:0/3024326131 shutdown_connections 2024-04-24T09:07:31.177 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.178+0000 7efc159e1740 1 --2- 172.21.15.83:0/3024326131 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ad3e09c3e0 0x55ad3e09e7f0 crc :-1 s=READY pgs=403 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:31.178 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.178+0000 7efc159e1740 1 --2- 172.21.15.83:0/3024326131 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ad3e07b390 0x55ad3e078d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:31.178 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.178+0000 7efc159e1740 1 --2- 172.21.15.83:0/3024326131 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ad3e070ad0 0x55ad3e0787d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:31.178 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.178+0000 7efc159e1740 1 --2- 172.21.15.83:0/3024326131 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ad3e0701b0 0x55ad3e078290 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:31.178 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.178+0000 7efc159e1740 1 -- 172.21.15.83:0/3024326131 >> 172.21.15.83:0/3024326131 conn(0x55ad3e06c6c0 msgr2=0x55ad3e0723f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:31.178 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.178+0000 7efbf5600640 0 client.38291 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:07:31.179 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.178+0000 7efc159e1740 1 -- 172.21.15.83:0/3024326131 shutdown_connections 2024-04-24T09:07:31.179 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.178+0000 7efc159e1740 1 -- 172.21.15.83:0/3024326131 wait complete. 2024-04-24T09:07:31.179 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.178+0000 7efc159e1740 20 client.38291 trim_cache size 0 max 16384 2024-04-24T09:07:31.203 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.203+0000 7efc159e1740 1 Processor -- start 2024-04-24T09:07:31.203 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.203+0000 7efc159e1740 1 -- start start 2024-04-24T09:07:31.204 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.204+0000 7efc159e1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ad3e070ad0 0x55ad3e072b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:31.204 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.204+0000 7efc159e1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ad3e07b390 0x55ad3e0773f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:31.204 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.204+0000 7efc159e1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ad3e086e20 0x55ad3e07a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:31.204 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.204+0000 7efc159e1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7efc0c1ad490 con 0x55ad3e07b390 2024-04-24T09:07:31.204 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.204+0000 7efc159e1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efc0c1af120 con 0x55ad3e086e20 2024-04-24T09:07:31.204 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.204+0000 7efc159e1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7efc0c1adb60 con 0x55ad3e070ad0 2024-04-24T09:07:31.205 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.204+0000 7efc0b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ad3e07b390 0x55ad3e0773f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:31.205 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.204+0000 7efc10c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ad3e086e20 0x55ad3e07a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:31.205 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.204+0000 7efc10c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ad3e086e20 0x55ad3e07a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:42796/0 (socket says 172.21.15.83:42796) 2024-04-24T09:07:31.205 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.204+0000 7efc10c00640 1 -- 172.21.15.83:0/650759351 learned_addr learned my addr 172.21.15.83:0/650759351 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:07:31.205 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.204+0000 7efc0b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ad3e07b390 0x55ad3e0773f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39722/0 (socket says 172.21.15.83:39722) 2024-04-24T09:07:31.205 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.204+0000 7efc0be00640 1 --2- 172.21.15.83:0/650759351 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ad3e070ad0 0x55ad3e072b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:31.205 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.205+0000 7efc0be00640 1 -- 172.21.15.83:0/650759351 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ad3e086e20 msgr2=0x55ad3e07a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:31.206 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.205+0000 7efc0be00640 1 --2- 172.21.15.83:0/650759351 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ad3e086e20 0x55ad3e07a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:31.206 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.205+0000 7efc0be00640 1 -- 172.21.15.83:0/650759351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ad3e07b390 msgr2=0x55ad3e0773f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:31.206 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.205+0000 7efc0be00640 1 --2- 172.21.15.83:0/650759351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ad3e07b390 0x55ad3e0773f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:31.206 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.205+0000 7efc0be00640 1 -- 172.21.15.83:0/650759351 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad3ddb5620 con 0x55ad3e070ad0 2024-04-24T09:07:31.206 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.205+0000 7efc0b400640 1 --2- 172.21.15.83:0/650759351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ad3e07b390 0x55ad3e0773f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:07:31.206 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.206+0000 7efc0be00640 1 --2- 172.21.15.83:0/650759351 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ad3e070ad0 0x55ad3e072b70 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7efbfc004ed0 tx=0x7efbfc005c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b2128c34f743dae server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:31.207 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.207+0000 7efbf7e00640 1 -- 172.21.15.83:0/650759351 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbfc035050 con 0x55ad3e070ad0 2024-04-24T09:07:31.207 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.207+0000 7efbf7e00640 1 -- 172.21.15.83:0/650759351 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efbfc005000 con 0x55ad3e070ad0 2024-04-24T09:07:31.207 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.207+0000 7efbf7e00640 1 -- 172.21.15.83:0/650759351 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbfc005d50 con 0x55ad3e070ad0 2024-04-24T09:07:31.209 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.208+0000 7efc159e1740 1 -- 172.21.15.83:0/650759351 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ad3e070ad0 msgr2=0x55ad3e072b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:31.209 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.208+0000 7efc159e1740 1 --2- 172.21.15.83:0/650759351 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ad3e070ad0 0x55ad3e072b70 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7efbfc004ed0 tx=0x7efbfc005c00 comp rx=0 tx=0).stop 2024-04-24T09:07:31.209 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.209+0000 7efc159e1740 1 -- 172.21.15.83:0/650759351 shutdown_connections 2024-04-24T09:07:31.209 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.209+0000 7efc159e1740 1 --2- 172.21.15.83:0/650759351 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ad3e086e20 0x55ad3e07a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:31.209 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.209+0000 7efc159e1740 1 --2- 172.21.15.83:0/650759351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ad3e07b390 0x55ad3e0773f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:31.209 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.209+0000 7efc159e1740 1 --2- 172.21.15.83:0/650759351 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ad3e070ad0 0x55ad3e072b70 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:31.209 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.209+0000 7efc159e1740 1 -- 172.21.15.83:0/650759351 >> 172.21.15.83:0/650759351 conn(0x55ad3e06c6c0 msgr2=0x55ad3e079ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:31.209 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.209+0000 7efc159e1740 1 -- 172.21.15.83:0/650759351 shutdown_connections 2024-04-24T09:07:31.209 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.209+0000 7efc159e1740 1 -- 172.21.15.83:0/650759351 wait complete. 2024-04-24T09:07:31.210 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.210+0000 7efc159e1740 1 -- start start 2024-04-24T09:07:31.210 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.210+0000 7efc159e1740 1 Processor -- start 2024-04-24T09:07:31.210 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.210+0000 7efc159e1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ad3e070ad0 0x55ad3e07eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:31.211 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.210+0000 7efc159e1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ad3e07b390 0x55ad3e07f420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:31.211 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.210+0000 7efc159e1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ad3e086e20 0x55ad3e07f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:31.211 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.210+0000 7efc159e1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7efc0c1ae990 con 0x55ad3e070ad0 2024-04-24T09:07:31.211 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.210+0000 7efc159e1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efc0c1ae290 con 0x55ad3e086e20 2024-04-24T09:07:31.211 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.210+0000 7efc159e1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7efc0c1a2570 con 0x55ad3e07b390 2024-04-24T09:07:31.211 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.210+0000 7efc159e1740 10 client.0 _ll_register_callbacks cb 0x7efbfc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:07:31.212 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.211+0000 7efc0be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ad3e070ad0 0x55ad3e07eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:31.212 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.211+0000 7efc0be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ad3e070ad0 0x55ad3e07eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39732/0 (socket says 172.21.15.83:39732) 2024-04-24T09:07:31.212 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.211+0000 7efc0be00640 1 -- 172.21.15.83:0/1181446267 learned_addr learned my addr 172.21.15.83:0/1181446267 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:07:31.212 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.211+0000 7efc10c00640 1 --2- 172.21.15.83:0/1181446267 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ad3e086e20 0x55ad3e07f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:31.212 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.211+0000 7efc0be00640 1 -- 172.21.15.83:0/1181446267 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ad3e07b390 msgr2=0x55ad3e07f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:31.212 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.211+0000 7efc0be00640 1 --2- 172.21.15.83:0/1181446267 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ad3e07b390 0x55ad3e07f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:31.212 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.211+0000 7efc0be00640 1 -- 172.21.15.83:0/1181446267 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ad3e086e20 msgr2=0x55ad3e07f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:31.213 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.211+0000 7efc0be00640 1 --2- 172.21.15.83:0/1181446267 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ad3e086e20 0x55ad3e07f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:31.213 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.211+0000 7efc0be00640 1 -- 172.21.15.83:0/1181446267 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad3e101aa0 con 0x55ad3e070ad0 2024-04-24T09:07:31.213 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.212+0000 7efc0be00640 1 --2- 172.21.15.83:0/1181446267 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ad3e070ad0 0x55ad3e07eee0 secure :-1 s=READY pgs=2245 cs=0 l=1 rev1=1 crypto rx=0x7efbfc031440 tx=0x7efbfc02aa70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12a85147063f2850 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:31.213 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.212+0000 7efbf5600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:07:31.213 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.212+0000 7efbf5600640 1 -- 172.21.15.83:0/1181446267 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbfc035040 con 0x55ad3e070ad0 2024-04-24T09:07:31.213 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.212+0000 7efbf5600640 1 -- 172.21.15.83:0/1181446267 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9583+0+0 (secure 0 0 0) 0x7efbfc0048d0 con 0x55ad3e070ad0 2024-04-24T09:07:31.214 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.213+0000 7efc159e1740 1 -- 172.21.15.83:0/1181446267 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efbfc005000 con 0x55ad3e070ad0 2024-04-24T09:07:31.214 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.213+0000 7efbf5600640 1 -- 172.21.15.83:0/1181446267 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7efbfc005d50 con 0x55ad3e070ad0 2024-04-24T09:07:31.214 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.213+0000 7efc159e1740 10 client.21144 fetch_fsmap learned FSMap version 51 2024-04-24T09:07:31.214 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.213+0000 7efc159e1740 1 -- 172.21.15.83:0/1181446267 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x55ad3e102d90 con 0x55ad3e070ad0 2024-04-24T09:07:31.214 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.213+0000 7efbf5600640 1 -- 172.21.15.83:0/1181446267 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efbfc004ad0 con 0x55ad3e070ad0 2024-04-24T09:07:31.214 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.213+0000 7efbf5600640 1 -- 172.21.15.83:0/1181446267 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 51) ==== 8406+0+0 (secure 0 0 0) 0x7efbfc045c80 con 0x55ad3e070ad0 2024-04-24T09:07:31.214 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.214+0000 7efc159e1740 10 client.21144 fetch_fsmap finished waiting for FSMap version 51 2024-04-24T09:07:31.214 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.214+0000 7efc159e1740 10 client.21144 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:07:31.214 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.214+0000 7efc159e1740 20 client.21144 populate_metadata read hostname 'smithi083' 2024-04-24T09:07:31.214 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.214+0000 7efc159e1740 1 --2- 172.21.15.83:0/1181446267 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ad3e1033b0 0x55ad3e105870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:31.214 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.214+0000 7efc159e1740 4 client.21144 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T09:07:31.215 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.214+0000 7efc159e1740 1 -- 172.21.15.83:0/1181446267 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7efbfc005000 con 0x55ad3e1033b0 2024-04-24T09:07:31.215 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.215+0000 7efc0b400640 1 --2- 172.21.15.83:0/1181446267 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ad3e1033b0 0x55ad3e105870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:31.216 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.216+0000 7efc0b400640 1 --2- 172.21.15.83:0/1181446267 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ad3e1033b0 0x55ad3e105870 crc :-1 s=READY pgs=404 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45842dfe3bbf9353 server_cookie=610f47755ffd329f in_seq=0 out_seq=0 2024-04-24T09:07:31.216 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.216+0000 7efbf5600640 10 client.21144 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:07:31.316 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.316+0000 7efbf5600640 1 -- 172.21.15.83:0/1181446267 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7efbfc005000 con 0x55ad3e1033b0 2024-04-24T09:07:31.316 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.316+0000 7efbf5600640 10 client.21144 handle_command_reply: tid=0 2024-04-24T09:07:31.317 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.316+0000 7efc159e1740 1 client.21144 shutdown 2024-04-24T09:07:31.317 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.316+0000 7efc159e1740 1 -- 172.21.15.83:0/1181446267 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ad3e070ad0 msgr2=0x55ad3e07eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:31.317 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.317+0000 7efc159e1740 1 --2- 172.21.15.83:0/1181446267 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ad3e070ad0 0x55ad3e07eee0 secure :-1 s=READY pgs=2245 cs=0 l=1 rev1=1 crypto rx=0x7efbfc031440 tx=0x7efbfc02aa70 comp rx=0 tx=0).stop 2024-04-24T09:07:31.317 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.317+0000 7efc159e1740 1 -- 172.21.15.83:0/1181446267 shutdown_connections 2024-04-24T09:07:31.317 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.317+0000 7efc159e1740 1 --2- 172.21.15.83:0/1181446267 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ad3e1033b0 0x55ad3e105870 crc :-1 s=READY pgs=404 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:31.318 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.317+0000 7efc159e1740 1 --2- 172.21.15.83:0/1181446267 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ad3e086e20 0x55ad3e07f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:31.318 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.317+0000 7efc159e1740 1 --2- 172.21.15.83:0/1181446267 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ad3e07b390 0x55ad3e07f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:31.318 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.317+0000 7efc159e1740 1 --2- 172.21.15.83:0/1181446267 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ad3e070ad0 0x55ad3e07eee0 unknown :-1 s=CLOSED pgs=2245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:31.318 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.317+0000 7efc159e1740 1 -- 172.21.15.83:0/1181446267 >> 172.21.15.83:0/1181446267 conn(0x55ad3e06c6c0 msgr2=0x55ad3e0866a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:31.318 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.317+0000 7efbf5600640 0 client.21144 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:07:31.318 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.317+0000 7efc159e1740 1 -- 172.21.15.83:0/1181446267 shutdown_connections 2024-04-24T09:07:31.318 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.317+0000 7efc159e1740 1 -- 172.21.15.83:0/1181446267 wait complete. 2024-04-24T09:07:31.318 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.317+0000 7efc159e1740 20 client.21144 trim_cache size 0 max 16384 2024-04-24T09:07:31.319 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T09:07:31.319 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T09:07:31.319 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T09:07:31.319 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T09:07:31.319 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.318+0000 7efc12c00640 1 -- 172.21.15.83:0/3998550175 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7efbf00a3dc0 msgr2=0x7efbf00a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:31.319 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.318+0000 7efc12c00640 1 --2- 172.21.15.83:0/3998550175 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7efbf00a3dc0 0x7efbf00a6280 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7efc04009fd0 tx=0x7efc040091e0 comp rx=0 tx=0).stop 2024-04-24T09:07:31.319 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.318+0000 7efc12c00640 1 -- 172.21.15.83:0/3998550175 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc0c109520 msgr2=0x7efc0c076a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:31.319 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.318+0000 7efc12c00640 1 --2- 172.21.15.83:0/3998550175 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc0c109520 0x7efc0c076a50 secure :-1 s=READY pgs=2243 cs=0 l=1 rev1=1 crypto rx=0x7efc0c06b770 tx=0x7efbf800dbb0 comp rx=0 tx=0).stop 2024-04-24T09:07:31.320 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.318+0000 7efc12c00640 1 -- 172.21.15.83:0/3998550175 shutdown_connections 2024-04-24T09:07:31.320 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.318+0000 7efc12c00640 1 --2- 172.21.15.83:0/3998550175 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7efbf00a3dc0 0x7efbf00a6280 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:31.320 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.318+0000 7efc12c00640 1 --2- 172.21.15.83:0/3998550175 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc0c076f90 0x7efc0c1aafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:31.320 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.318+0000 7efc12c00640 1 --2- 172.21.15.83:0/3998550175 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc0c109520 0x7efc0c076a50 unknown :-1 s=CLOSED pgs=2243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:31.320 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.318+0000 7efc12c00640 1 --2- 172.21.15.83:0/3998550175 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efc0c104450 0x7efc0c076510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:31.320 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.318+0000 7efc12c00640 1 -- 172.21.15.83:0/3998550175 >> 172.21.15.83:0/3998550175 conn(0x7efc0c0ffcf0 msgr2=0x7efc0c1010a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:31.320 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.319+0000 7efc12c00640 1 -- 172.21.15.83:0/3998550175 shutdown_connections 2024-04-24T09:07:31.320 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:31.319+0000 7efc12c00640 1 -- 172.21.15.83:0/3998550175 wait complete. 2024-04-24T09:07:31.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.544+0000 7f7c22800640 1 -- 172.21.15.83:0/3565885439 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7c1c110160 con 0x7f7c1c101000 2024-04-24T09:07:31.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.545+0000 7f7c13e00640 1 -- 172.21.15.83:0/3565885439 <== mon.2 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) ==== 94+0+11961 (secure 0 0 0) 0x7f7c0809cf10 con 0x7f7c1c101000 2024-04-24T09:07:31.545 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T09:07:31.545 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":51,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/358872257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":358872257},{"type":"v1","addr":"172.21.15.83:6835","nonce":358872257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3157070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3157070851},{"type":"v1","addr":"172.21.15.83:6837","nonce":3157070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":51,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T08:48:29.422576+0000","modified":"2024-04-24T09:07:25.415732+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34520,"mds_1":34493,"mds_2":34451,"mds_3":34427,"mds_4":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14658":{"gid":14658,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6839/4085470844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":4085470844},{"type":"v1","addr":"172.21.15.83:6839","nonce":4085470844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14676":{"gid":14676,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6841/2634754210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":2634754210},{"type":"v1","addr":"172.21.15.83:6841","nonce":2634754210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24382":{"gid":24382,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/1859921310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1859921310},{"type":"v1","addr":"172.21.15.189:6833","nonce":1859921310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/2241923197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2241923197},{"type":"v1","addr":"172.21.15.189:6835","nonce":2241923197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.189:6839/2548965336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":2548965336},{"type":"v1","addr":"172.21.15.189:6839","nonce":2548965336}]},"join_fscid":-1,"export_targets":[0,1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26686":{"gid":26686,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6837/1678817161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":1678817161},{"type":"v1","addr":"172.21.15.189:6837","nonce":1678817161}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.138:6833/1157967658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1157967658},{"type":"v1","addr":"172.21.15.138:6833","nonce":1157967658}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.138:6835/1909219758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1909219758},{"type":"v1","addr":"172.21.15.138:6835","nonce":1909219758}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34493":{"gid":34493,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.138:6837/900566754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":900566754},{"type":"v1","addr":"172.21.15.138:6837","nonce":900566754}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34520":{"gid":34520,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.138:6839/2236932393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2236932393},{"type":"v1","addr":"172.21.15.138:6839","nonce":2236932393}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34520,"qdb_cluster":[34520,24454,34427,34493,34451]},"id":1}]} 2024-04-24T09:07:31.546 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 51 2024-04-24T09:07:31.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.548+0000 7f7c22800640 1 -- 172.21.15.83:0/3565885439 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f7c000a3e70 msgr2=0x7f7c000a6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:31.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.548+0000 7f7c22800640 1 --2- 172.21.15.83:0/3565885439 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f7c000a3e70 0x7f7c000a6330 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c009ba0 tx=0x7f7c0c005f90 comp rx=0 tx=0).stop 2024-04-24T09:07:31.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.548+0000 7f7c22800640 1 -- 172.21.15.83:0/3565885439 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c1c101000 msgr2=0x7f7c1c10f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:31.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.548+0000 7f7c22800640 1 --2- 172.21.15.83:0/3565885439 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c1c101000 0x7f7c1c10f170 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7c0800c920 tx=0x7f7c0800cde0 comp rx=0 tx=0).stop 2024-04-24T09:07:31.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.548+0000 7f7c22800640 1 -- 172.21.15.83:0/3565885439 shutdown_connections 2024-04-24T09:07:31.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.548+0000 7f7c22800640 1 --2- 172.21.15.83:0/3565885439 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f7c000a3e70 0x7f7c000a6330 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:31.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.548+0000 7f7c22800640 1 --2- 172.21.15.83:0/3565885439 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c1c106650 0x7f7c1c10f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:31.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.548+0000 7f7c22800640 1 --2- 172.21.15.83:0/3565885439 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c1c101000 0x7f7c1c10f170 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:31.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.548+0000 7f7c22800640 1 --2- 172.21.15.83:0/3565885439 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7c1c100640 0x7f7c1c10ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:31.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.549+0000 7f7c22800640 1 -- 172.21.15.83:0/3565885439 >> 172.21.15.83:0/3565885439 conn(0x7f7c1c0774a0 msgr2=0x7f7c1c103560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:31.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.549+0000 7f7c22800640 1 -- 172.21.15.83:0/3565885439 shutdown_connections 2024-04-24T09:07:31.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.549+0000 7f7c22800640 1 -- 172.21.15.83:0/3565885439 wait complete. 2024-04-24T09:07:31.571 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-24T09:07:31.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.813+0000 7f6a65200640 1 -- 172.21.15.83:0/1482184460 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6a60104e60 msgr2=0x7f6a60108ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:31.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.813+0000 7f6a65200640 1 --2- 172.21.15.83:0/1482184460 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6a60104e60 0x7f6a60108ea0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c009900 tx=0x7f6a4c02a410 comp rx=0 tx=0).stop 2024-04-24T09:07:31.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.813+0000 7f6a65200640 1 -- 172.21.15.83:0/1482184460 shutdown_connections 2024-04-24T09:07:31.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.813+0000 7f6a65200640 1 --2- 172.21.15.83:0/1482184460 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6a60109580 0x7f6a60111150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:31.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.813+0000 7f6a65200640 1 --2- 172.21.15.83:0/1482184460 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6a60104e60 0x7f6a60108ea0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:31.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.813+0000 7f6a65200640 1 --2- 172.21.15.83:0/1482184460 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a601044b0 0x7f6a60104890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:31.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.813+0000 7f6a65200640 1 -- 172.21.15.83:0/1482184460 >> 172.21.15.83:0/1482184460 conn(0x7f6a600ffd90 msgr2=0x7f6a601021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:31.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.813+0000 7f6a65200640 1 -- 172.21.15.83:0/1482184460 shutdown_connections 2024-04-24T09:07:31.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.813+0000 7f6a65200640 1 -- 172.21.15.83:0/1482184460 wait complete. 2024-04-24T09:07:31.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.814+0000 7f6a65200640 1 Processor -- start 2024-04-24T09:07:31.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.815+0000 7f6a65200640 1 -- start start 2024-04-24T09:07:31.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.815+0000 7f6a65200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6a601044b0 0x7f6a60196b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:31.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.815+0000 7f6a65200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6a60109580 0x7f6a601970d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:31.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.816+0000 7f6a65200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a6019af30 0x7f6a6019b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:31.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.816+0000 7f6a65200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6a60078cd0 con 0x7f6a6019af30 2024-04-24T09:07:31.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.816+0000 7f6a65200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6a60078b50 con 0x7f6a60109580 2024-04-24T09:07:31.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.816+0000 7f6a65200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6a60078e50 con 0x7f6a601044b0 2024-04-24T09:07:31.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.816+0000 7f6a5e800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a6019af30 0x7f6a6019b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:31.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.816+0000 7f6a5e800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a6019af30 0x7f6a6019b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39750/0 (socket says 172.21.15.83:39750) 2024-04-24T09:07:31.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.816+0000 7f6a5e800640 1 -- 172.21.15.83:0/2017691240 learned_addr learned my addr 172.21.15.83:0/2017691240 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:07:31.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.816+0000 7f6a5de00640 1 --2- 172.21.15.83:0/2017691240 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6a601044b0 0x7f6a60196b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:31.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.816+0000 7f6a5d400640 1 --2- 172.21.15.83:0/2017691240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6a60109580 0x7f6a601970d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:31.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.817+0000 7f6a5e800640 1 -- 172.21.15.83:0/2017691240 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6a601044b0 msgr2=0x7f6a60196b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:31.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.817+0000 7f6a5e800640 1 --2- 172.21.15.83:0/2017691240 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6a601044b0 0x7f6a60196b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:31.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.817+0000 7f6a5e800640 1 -- 172.21.15.83:0/2017691240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6a60109580 msgr2=0x7f6a601970d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:31.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.817+0000 7f6a5e800640 1 --2- 172.21.15.83:0/2017691240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6a60109580 0x7f6a601970d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:31.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.817+0000 7f6a5e800640 1 -- 172.21.15.83:0/2017691240 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a6019ba20 con 0x7f6a6019af30 2024-04-24T09:07:31.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.818+0000 7f6a5e800640 1 --2- 172.21.15.83:0/2017691240 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a6019af30 0x7f6a6019b3e0 secure :-1 s=READY pgs=2246 cs=0 l=1 rev1=1 crypto rx=0x7f6a5800cce0 tx=0x7f6a5800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:31.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.818+0000 7f6a56a00640 1 -- 172.21.15.83:0/2017691240 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a5800f450 con 0x7f6a6019af30 2024-04-24T09:07:31.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.819+0000 7f6a56a00640 1 -- 172.21.15.83:0/2017691240 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6a5800faa0 con 0x7f6a6019af30 2024-04-24T09:07:31.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.819+0000 7f6a56a00640 1 -- 172.21.15.83:0/2017691240 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a58018b70 con 0x7f6a6019af30 2024-04-24T09:07:31.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.819+0000 7f6a65200640 1 -- 172.21.15.83:0/2017691240 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a6019bd10 con 0x7f6a6019af30 2024-04-24T09:07:31.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.819+0000 7f6a65200640 1 -- 172.21.15.83:0/2017691240 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a601b4490 con 0x7f6a6019af30 2024-04-24T09:07:31.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.823+0000 7f6a56a00640 1 -- 172.21.15.83:0/2017691240 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6a58018d10 con 0x7f6a6019af30 2024-04-24T09:07:31.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.824+0000 7f6a681d2740 1 Processor -- start 2024-04-24T09:07:31.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.826+0000 7f6a56a00640 1 --2- 172.21.15.83:0/2017691240 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f6a440a3de0 0x7f6a440a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:31.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.826+0000 7f6a56a00640 1 -- 172.21.15.83:0/2017691240 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 9583+0+0 (secure 0 0 0) 0x7f6a580d0fb0 con 0x7f6a6019af30 2024-04-24T09:07:31.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.827+0000 7f6a5d400640 1 --2- 172.21.15.83:0/2017691240 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f6a440a3de0 0x7f6a440a62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:31.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.827+0000 7f6a681d2740 1 -- start start 2024-04-24T09:07:31.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.827+0000 7f6a681d2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610fc340950 0x5610fc340d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:31.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.827+0000 7f6a681d2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610fc341270 0x5610fc34b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:31.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.827+0000 7f6a681d2740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610fc34b940 0x5610fc34dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:31.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.827+0000 7f6a681d2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5610fc2a61d0 con 0x5610fc340950 2024-04-24T09:07:31.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.827+0000 7f6a681d2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5610fc24c450 con 0x5610fc341270 2024-04-24T09:07:31.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.827+0000 7f6a681d2740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5610fc2a2190 con 0x5610fc34b940 2024-04-24T09:07:31.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.828+0000 7f6a5d400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610fc341270 0x5610fc34b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:31.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.828+0000 7f6a5de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610fc340950 0x5610fc340d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:31.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.828+0000 7f6a5e800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610fc34b940 0x5610fc34dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:31.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.828+0000 7f6a5e800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610fc34b940 0x5610fc34dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:34586/0 (socket says 172.21.15.83:34586) 2024-04-24T09:07:31.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.828+0000 7f6a5e800640 1 -- 172.21.15.83:0/3999085839 learned_addr learned my addr 172.21.15.83:0/3999085839 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:07:31.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.828+0000 7f6a5de00640 1 -- 172.21.15.83:0/3999085839 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610fc34b940 msgr2=0x5610fc34dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:31.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.828+0000 7f6a5de00640 1 --2- 172.21.15.83:0/3999085839 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610fc34b940 0x5610fc34dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:31.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.828+0000 7f6a5de00640 1 -- 172.21.15.83:0/3999085839 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610fc341270 msgr2=0x5610fc34b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:31.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.828+0000 7f6a5de00640 1 --2- 172.21.15.83:0/3999085839 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610fc341270 0x5610fc34b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:31.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.828+0000 7f6a5de00640 1 -- 172.21.15.83:0/3999085839 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610fc309310 con 0x5610fc340950 2024-04-24T09:07:31.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.829+0000 7f6a5d400640 1 --2- 172.21.15.83:0/2017691240 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f6a440a3de0 0x7f6a440a62a0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c02a980 tx=0x7f6a4c004260 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:31.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.829+0000 7f6a5de00640 1 --2- 172.21.15.83:0/3999085839 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610fc340950 0x5610fc340d30 secure :-1 s=READY pgs=2247 cs=0 l=1 rev1=1 crypto rx=0x7f6a50005fd0 tx=0x7f6a5000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b75a47cf32dab9bf server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:31.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.829+0000 7f6a4b400640 1 -- 172.21.15.83:0/3999085839 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a5001e070 con 0x5610fc340950 2024-04-24T09:07:31.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.829+0000 7f6a4b400640 1 -- 172.21.15.83:0/3999085839 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6a50019af0 con 0x5610fc340950 2024-04-24T09:07:31.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.829+0000 7f6a4b400640 1 -- 172.21.15.83:0/3999085839 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a50010a70 con 0x5610fc340950 2024-04-24T09:07:31.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.830+0000 7f6a681d2740 1 -- 172.21.15.83:0/3999085839 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610fc340950 msgr2=0x5610fc340d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:31.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.830+0000 7f6a681d2740 1 --2- 172.21.15.83:0/3999085839 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610fc340950 0x5610fc340d30 secure :-1 s=READY pgs=2247 cs=0 l=1 rev1=1 crypto rx=0x7f6a50005fd0 tx=0x7f6a5000f790 comp rx=0 tx=0).stop 2024-04-24T09:07:31.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.830+0000 7f6a681d2740 1 -- 172.21.15.83:0/3999085839 shutdown_connections 2024-04-24T09:07:31.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.830+0000 7f6a681d2740 1 --2- 172.21.15.83:0/3999085839 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610fc34b940 0x5610fc34dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:31.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.830+0000 7f6a681d2740 1 --2- 172.21.15.83:0/3999085839 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610fc341270 0x5610fc34b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:31.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.830+0000 7f6a681d2740 1 --2- 172.21.15.83:0/3999085839 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610fc340950 0x5610fc340d30 unknown :-1 s=CLOSED pgs=2247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:31.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.830+0000 7f6a681d2740 1 -- 172.21.15.83:0/3999085839 >> 172.21.15.83:0/3999085839 conn(0x5610fc33cc70 msgr2=0x5610fc33d050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:31.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.831+0000 7f6a681d2740 1 -- 172.21.15.83:0/3999085839 shutdown_connections 2024-04-24T09:07:31.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.831+0000 7f6a681d2740 1 -- 172.21.15.83:0/3999085839 wait complete. 2024-04-24T09:07:31.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.831+0000 7f6a681d2740 1 -- start start 2024-04-24T09:07:31.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.831+0000 7f6a681d2740 1 Processor -- start 2024-04-24T09:07:31.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.832+0000 7f6a681d2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610fc341270 0x5610fc348930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:31.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.832+0000 7f6a681d2740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610fc34b940 0x5610fc348e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:31.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.832+0000 7f6a681d2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610fc3493b0 0x5610fc357c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:31.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.832+0000 7f6a681d2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5610fc34e7d0 con 0x5610fc3493b0 2024-04-24T09:07:31.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.832+0000 7f6a681d2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5610fc25f810 con 0x5610fc341270 2024-04-24T09:07:31.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.832+0000 7f6a681d2740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5610fc34e950 con 0x5610fc34b940 2024-04-24T09:07:31.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.832+0000 7f6a681d2740 10 client.0 _ll_register_callbacks cb 0x5610fc22d140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:07:31.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.833+0000 7f6a5e800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610fc3493b0 0x5610fc357c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:31.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.833+0000 7f6a5e800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610fc3493b0 0x5610fc357c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39768/0 (socket says 172.21.15.83:39768) 2024-04-24T09:07:31.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.833+0000 7f6a5e800640 1 -- 172.21.15.83:0/3283738610 learned_addr learned my addr 172.21.15.83:0/3283738610 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:07:31.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.833+0000 7f6a5de00640 1 --2- 172.21.15.83:0/3283738610 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610fc341270 0x5610fc348930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:31.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.833+0000 7f6a5e800640 1 -- 172.21.15.83:0/3283738610 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610fc34b940 msgr2=0x5610fc348e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:31.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.833+0000 7f6a5e800640 1 --2- 172.21.15.83:0/3283738610 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610fc34b940 0x5610fc348e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:31.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.833+0000 7f6a5e800640 1 -- 172.21.15.83:0/3283738610 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610fc341270 msgr2=0x5610fc348930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:31.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.833+0000 7f6a5e800640 1 --2- 172.21.15.83:0/3283738610 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610fc341270 0x5610fc348930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:31.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.833+0000 7f6a5e800640 1 -- 172.21.15.83:0/3283738610 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610fc2703f0 con 0x5610fc3493b0 2024-04-24T09:07:31.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.834+0000 7f6a5e800640 1 --2- 172.21.15.83:0/3283738610 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610fc3493b0 0x5610fc357c40 secure :-1 s=READY pgs=2248 cs=0 l=1 rev1=1 crypto rx=0x7f6a5800ab20 tx=0x7f6a5809c6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d6ba2953550f68d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:31.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.834+0000 7f6a3fe00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:07:31.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.834+0000 7f6a3fe00640 1 -- 172.21.15.83:0/3283738610 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a5809cad0 con 0x5610fc3493b0 2024-04-24T09:07:31.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.834+0000 7f6a3fe00640 1 -- 172.21.15.83:0/3283738610 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9583+0+0 (secure 0 0 0) 0x7f6a58099140 con 0x5610fc3493b0 2024-04-24T09:07:31.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.834+0000 7f6a681d2740 1 -- 172.21.15.83:0/3283738610 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a50019af0 con 0x5610fc3493b0 2024-04-24T09:07:31.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.834+0000 7f6a3fe00640 1 -- 172.21.15.83:0/3283738610 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f6a580ad050 con 0x5610fc3493b0 2024-04-24T09:07:31.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.835+0000 7f6a681d2740 10 client.21156 fetch_fsmap learned FSMap version 51 2024-04-24T09:07:31.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.835+0000 7f6a681d2740 1 -- 172.21.15.83:0/3283738610 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x5610fbfb83f0 con 0x5610fc3493b0 2024-04-24T09:07:31.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.835+0000 7f6a3fe00640 1 -- 172.21.15.83:0/3283738610 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6a50019af0 con 0x5610fc3493b0 2024-04-24T09:07:31.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.835+0000 7f6a3fe00640 1 -- 172.21.15.83:0/3283738610 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 51) ==== 8406+0+0 (secure 0 0 0) 0x7f6a580b8020 con 0x5610fc3493b0 2024-04-24T09:07:31.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.835+0000 7f6a681d2740 10 client.21156 fetch_fsmap finished waiting for FSMap version 51 2024-04-24T09:07:31.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.835+0000 7f6a681d2740 10 client.21156 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-24T09:07:31.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.835+0000 7f6a681d2740 20 client.21156 populate_metadata read hostname 'smithi083' 2024-04-24T09:07:31.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.835+0000 7f6a681d2740 1 --2- 172.21.15.83:0/3283738610 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x5610fc35c310 0x5610fc35e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:31.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.835+0000 7f6a681d2740 4 client.21156 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:07:31.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.835+0000 7f6a681d2740 1 -- 172.21.15.83:0/3283738610 --> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5610fc2b3bf0 con 0x5610fc35c310 2024-04-24T09:07:31.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.836+0000 7f6a5de00640 1 --2- 172.21.15.83:0/3283738610 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x5610fc35c310 0x5610fc35e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:31.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.837+0000 7f6a3fe00640 10 client.21156 ms_handle_connect on v2:172.21.15.189:6838/2548965336 2024-04-24T09:07:31.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.837+0000 7f6a5de00640 1 --2- 172.21.15.83:0/3283738610 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x5610fc35c310 0x5610fc35e790 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e1c3f16987307eb6 server_cookie=3a9309b6ac7a3348 in_seq=0 out_seq=0 2024-04-24T09:07:31.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.840+0000 7f6a3fe00640 1 -- 172.21.15.83:0/3283738610 <== mds.4 v2:172.21.15.189:6838/2548965336 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5610fc2b3bf0 con 0x5610fc35c310 2024-04-24T09:07:31.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.840+0000 7f6a3fe00640 10 client.21156 handle_command_reply: tid=0 2024-04-24T09:07:31.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.840+0000 7f6a681d2740 1 client.21156 shutdown 2024-04-24T09:07:31.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.840+0000 7f6a681d2740 1 -- 172.21.15.83:0/3283738610 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610fc3493b0 msgr2=0x5610fc357c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:31.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.840+0000 7f6a681d2740 1 --2- 172.21.15.83:0/3283738610 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610fc3493b0 0x5610fc357c40 secure :-1 s=READY pgs=2248 cs=0 l=1 rev1=1 crypto rx=0x7f6a5800ab20 tx=0x7f6a5809c6d0 comp rx=0 tx=0).stop 2024-04-24T09:07:31.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.841+0000 7f6a681d2740 1 -- 172.21.15.83:0/3283738610 shutdown_connections 2024-04-24T09:07:31.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.841+0000 7f6a681d2740 1 --2- 172.21.15.83:0/3283738610 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x5610fc35c310 0x5610fc35e790 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:31.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.841+0000 7f6a681d2740 1 --2- 172.21.15.83:0/3283738610 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610fc3493b0 0x5610fc357c40 unknown :-1 s=CLOSED pgs=2248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:31.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.841+0000 7f6a681d2740 1 --2- 172.21.15.83:0/3283738610 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610fc34b940 0x5610fc348e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:31.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.841+0000 7f6a681d2740 1 --2- 172.21.15.83:0/3283738610 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610fc341270 0x5610fc348930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:31.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.841+0000 7f6a3fe00640 0 client.21156 ms_handle_reset on v2:172.21.15.189:6838/2548965336 2024-04-24T09:07:31.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.841+0000 7f6a681d2740 1 -- 172.21.15.83:0/3283738610 >> 172.21.15.83:0/3283738610 conn(0x5610fc33cc70 msgr2=0x5610fc342a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:31.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.841+0000 7f6a681d2740 1 -- 172.21.15.83:0/3283738610 shutdown_connections 2024-04-24T09:07:31.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.841+0000 7f6a681d2740 1 -- 172.21.15.83:0/3283738610 wait complete. 2024-04-24T09:07:31.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.841+0000 7f6a681d2740 20 client.21156 trim_cache size 0 max 16384 2024-04-24T09:07:31.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.864+0000 7f6a681d2740 1 Processor -- start 2024-04-24T09:07:31.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.864+0000 7f6a681d2740 1 -- start start 2024-04-24T09:07:31.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.864+0000 7f6a681d2740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610fc341270 0x5610fc347a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:31.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.864+0000 7f6a681d2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610fc3493b0 0x5610fc364060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:31.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.864+0000 7f6a681d2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610fc34b940 0x5610fc3645a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:31.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.864+0000 7f6a681d2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6a601be3b0 con 0x5610fc3493b0 2024-04-24T09:07:31.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.864+0000 7f6a681d2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6a601c0040 con 0x5610fc34b940 2024-04-24T09:07:31.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.864+0000 7f6a681d2740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6a601bea80 con 0x5610fc341270 2024-04-24T09:07:31.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.865+0000 7f6a5d400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610fc3493b0 0x5610fc364060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:31.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.865+0000 7f6a5d400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610fc3493b0 0x5610fc364060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39772/0 (socket says 172.21.15.83:39772) 2024-04-24T09:07:31.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.865+0000 7f6a5d400640 1 -- 172.21.15.83:0/1378401023 learned_addr learned my addr 172.21.15.83:0/1378401023 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:07:31.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.865+0000 7f6a5e800640 1 --2- 172.21.15.83:0/1378401023 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610fc34b940 0x5610fc3645a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:31.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.865+0000 7f6a5de00640 1 --2- 172.21.15.83:0/1378401023 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610fc341270 0x5610fc347a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:31.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.865+0000 7f6a5d400640 1 -- 172.21.15.83:0/1378401023 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610fc341270 msgr2=0x5610fc347a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:31.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.865+0000 7f6a5d400640 1 --2- 172.21.15.83:0/1378401023 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610fc341270 0x5610fc347a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:31.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.865+0000 7f6a5d400640 1 -- 172.21.15.83:0/1378401023 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610fc34b940 msgr2=0x5610fc3645a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:31.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.865+0000 7f6a5d400640 1 --2- 172.21.15.83:0/1378401023 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610fc34b940 0x5610fc3645a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:31.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.865+0000 7f6a5d400640 1 -- 172.21.15.83:0/1378401023 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610fc0827d0 con 0x5610fc3493b0 2024-04-24T09:07:31.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.866+0000 7f6a5d400640 1 --2- 172.21.15.83:0/1378401023 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610fc3493b0 0x5610fc364060 secure :-1 s=READY pgs=2249 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c005ab0 tx=0x7f6a4c040040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a8eb3e93350dae4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:31.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.867+0000 7f6a4aa00640 1 -- 172.21.15.83:0/1378401023 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a4c04d070 con 0x5610fc3493b0 2024-04-24T09:07:31.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.867+0000 7f6a4aa00640 1 -- 172.21.15.83:0/1378401023 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6a4c0054c0 con 0x5610fc3493b0 2024-04-24T09:07:31.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.867+0000 7f6a4aa00640 1 -- 172.21.15.83:0/1378401023 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a4c035040 con 0x5610fc3493b0 2024-04-24T09:07:31.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.868+0000 7f6a681d2740 1 -- 172.21.15.83:0/1378401023 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610fc3493b0 msgr2=0x5610fc364060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:31.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.868+0000 7f6a681d2740 1 --2- 172.21.15.83:0/1378401023 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610fc3493b0 0x5610fc364060 secure :-1 s=READY pgs=2249 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c005ab0 tx=0x7f6a4c040040 comp rx=0 tx=0).stop 2024-04-24T09:07:31.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.868+0000 7f6a681d2740 1 -- 172.21.15.83:0/1378401023 shutdown_connections 2024-04-24T09:07:31.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.868+0000 7f6a681d2740 1 --2- 172.21.15.83:0/1378401023 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610fc34b940 0x5610fc3645a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:31.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.868+0000 7f6a681d2740 1 --2- 172.21.15.83:0/1378401023 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610fc3493b0 0x5610fc364060 unknown :-1 s=CLOSED pgs=2249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:31.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.868+0000 7f6a681d2740 1 --2- 172.21.15.83:0/1378401023 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610fc341270 0x5610fc347a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:31.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.868+0000 7f6a681d2740 1 -- 172.21.15.83:0/1378401023 >> 172.21.15.83:0/1378401023 conn(0x5610fc33cc70 msgr2=0x5610fc33b560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:31.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.868+0000 7f6a681d2740 1 -- 172.21.15.83:0/1378401023 shutdown_connections 2024-04-24T09:07:31.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.868+0000 7f6a681d2740 1 -- 172.21.15.83:0/1378401023 wait complete. 2024-04-24T09:07:31.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.869+0000 7f6a681d2740 1 -- start start 2024-04-24T09:07:31.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.869+0000 7f6a681d2740 1 Processor -- start 2024-04-24T09:07:31.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.869+0000 7f6a681d2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610fc341270 0x5610fc353e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:31.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.869+0000 7f6a681d2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610fc3493b0 0x5610fc34f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:31.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.869+0000 7f6a681d2740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610fc34b940 0x5610fc34f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:31.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.869+0000 7f6a681d2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6a601bf8b0 con 0x5610fc341270 2024-04-24T09:07:31.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.869+0000 7f6a681d2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6a601bf1b0 con 0x5610fc3493b0 2024-04-24T09:07:31.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.870+0000 7f6a681d2740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6a601bc3b0 con 0x5610fc34b940 2024-04-24T09:07:31.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.870+0000 7f6a681d2740 10 client.0 _ll_register_callbacks cb 0x7f6a5800af30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:07:31.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.870+0000 7f6a5d400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610fc3493b0 0x5610fc34f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:31.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.870+0000 7f6a5de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610fc341270 0x5610fc353e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:31.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.870+0000 7f6a5d400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610fc3493b0 0x5610fc34f200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:42862/0 (socket says 172.21.15.83:42862) 2024-04-24T09:07:31.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.870+0000 7f6a5d400640 1 -- 172.21.15.83:0/4022697521 learned_addr learned my addr 172.21.15.83:0/4022697521 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:07:31.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.870+0000 7f6a5e800640 1 --2- 172.21.15.83:0/4022697521 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610fc34b940 0x5610fc34f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:31.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.870+0000 7f6a5d400640 1 -- 172.21.15.83:0/4022697521 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610fc34b940 msgr2=0x5610fc34f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:31.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.870+0000 7f6a5d400640 1 --2- 172.21.15.83:0/4022697521 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610fc34b940 0x5610fc34f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:31.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.870+0000 7f6a5d400640 1 -- 172.21.15.83:0/4022697521 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610fc341270 msgr2=0x5610fc353e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:31.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.870+0000 7f6a5d400640 1 --2- 172.21.15.83:0/4022697521 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610fc341270 0x5610fc353e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:31.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.870+0000 7f6a5d400640 1 -- 172.21.15.83:0/4022697521 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610fc34fe80 con 0x5610fc3493b0 2024-04-24T09:07:31.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.871+0000 7f6a5d400640 1 --2- 172.21.15.83:0/4022697521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610fc3493b0 0x5610fc34f200 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c0040d0 tx=0x7f6a4c004970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99fe763b3a110934 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:31.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.871+0000 7f6a3fe00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:07:31.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.871+0000 7f6a3fe00640 1 -- 172.21.15.83:0/4022697521 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a4c04d040 con 0x5610fc3493b0 2024-04-24T09:07:31.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.871+0000 7f6a3fe00640 1 -- 172.21.15.83:0/4022697521 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9583+0+0 (secure 0 0 0) 0x7f6a4c004d70 con 0x5610fc3493b0 2024-04-24T09:07:31.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.872+0000 7f6a681d2740 1 -- 172.21.15.83:0/4022697521 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a4c0054c0 con 0x5610fc3493b0 2024-04-24T09:07:31.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.872+0000 7f6a3fe00640 1 -- 172.21.15.83:0/4022697521 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f6a4c0035f0 con 0x5610fc3493b0 2024-04-24T09:07:31.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.872+0000 7f6a681d2740 10 client.38315 fetch_fsmap learned FSMap version 51 2024-04-24T09:07:31.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.872+0000 7f6a681d2740 1 -- 172.21.15.83:0/4022697521 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x5610fc3d3810 con 0x5610fc3493b0 2024-04-24T09:07:31.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.872+0000 7f6a3fe00640 1 -- 172.21.15.83:0/4022697521 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6a4c0054c0 con 0x5610fc3493b0 2024-04-24T09:07:31.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.873+0000 7f6a3fe00640 1 -- 172.21.15.83:0/4022697521 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 51) ==== 8406+0+0 (secure 0 0 0) 0x7f6a4c064020 con 0x5610fc3493b0 2024-04-24T09:07:31.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.873+0000 7f6a681d2740 10 client.38315 fetch_fsmap finished waiting for FSMap version 51 2024-04-24T09:07:31.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.873+0000 7f6a681d2740 10 client.38315 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-24T09:07:31.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.873+0000 7f6a681d2740 20 client.38315 populate_metadata read hostname 'smithi083' 2024-04-24T09:07:31.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.873+0000 7f6a681d2740 1 --2- 172.21.15.83:0/4022697521 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x5610fc3d3ff0 0x5610fc3d6410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:31.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.873+0000 7f6a681d2740 4 client.38315 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:07:31.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.873+0000 7f6a681d2740 1 -- 172.21.15.83:0/4022697521 --> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5610fc344490 con 0x5610fc3d3ff0 2024-04-24T09:07:31.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.874+0000 7f6a5de00640 1 --2- 172.21.15.83:0/4022697521 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x5610fc3d3ff0 0x5610fc3d6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:31.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.875+0000 7f6a3fe00640 10 client.38315 ms_handle_connect on v2:172.21.15.189:6838/2548965336 2024-04-24T09:07:31.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.875+0000 7f6a5de00640 1 --2- 172.21.15.83:0/4022697521 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x5610fc3d3ff0 0x5610fc3d6410 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=bab61894a73a26c8 server_cookie=8ff5318e953e9de6 in_seq=0 out_seq=0 2024-04-24T09:07:31.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.875+0000 7f6a3fe00640 1 -- 172.21.15.83:0/4022697521 <== mds.4 v2:172.21.15.189:6838/2548965336 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5610fc344490 con 0x5610fc3d3ff0 2024-04-24T09:07:31.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.875+0000 7f6a3fe00640 10 client.38315 handle_command_reply: tid=0 2024-04-24T09:07:31.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.875+0000 7f6a681d2740 1 client.38315 shutdown 2024-04-24T09:07:31.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.876+0000 7f6a681d2740 1 -- 172.21.15.83:0/4022697521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610fc3493b0 msgr2=0x5610fc34f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:31.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.876+0000 7f6a681d2740 1 --2- 172.21.15.83:0/4022697521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610fc3493b0 0x5610fc34f200 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c0040d0 tx=0x7f6a4c004970 comp rx=0 tx=0).stop 2024-04-24T09:07:31.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.876+0000 7f6a681d2740 1 -- 172.21.15.83:0/4022697521 shutdown_connections 2024-04-24T09:07:31.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.876+0000 7f6a681d2740 1 --2- 172.21.15.83:0/4022697521 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x5610fc3d3ff0 0x5610fc3d6410 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:31.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.876+0000 7f6a681d2740 1 --2- 172.21.15.83:0/4022697521 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610fc34b940 0x5610fc34f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:31.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.876+0000 7f6a681d2740 1 --2- 172.21.15.83:0/4022697521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610fc3493b0 0x5610fc34f200 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:31.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.876+0000 7f6a3fe00640 0 client.38315 ms_handle_reset on v2:172.21.15.189:6838/2548965336 2024-04-24T09:07:31.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.876+0000 7f6a681d2740 1 --2- 172.21.15.83:0/4022697521 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610fc341270 0x5610fc353e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:31.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.876+0000 7f6a681d2740 1 -- 172.21.15.83:0/4022697521 >> 172.21.15.83:0/4022697521 conn(0x5610fc33cc70 msgr2=0x5610fc33b560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:31.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.876+0000 7f6a681d2740 1 -- 172.21.15.83:0/4022697521 shutdown_connections 2024-04-24T09:07:31.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.876+0000 7f6a681d2740 1 -- 172.21.15.83:0/4022697521 wait complete. 2024-04-24T09:07:31.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.877+0000 7f6a681d2740 20 client.38315 trim_cache size 0 max 16384 2024-04-24T09:07:31.878 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:07:31.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.877+0000 7f6a65200640 1 -- 172.21.15.83:0/2017691240 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f6a440a3de0 msgr2=0x7f6a440a62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:31.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.877+0000 7f6a65200640 1 --2- 172.21.15.83:0/2017691240 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f6a440a3de0 0x7f6a440a62a0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c02a980 tx=0x7f6a4c004260 comp rx=0 tx=0).stop 2024-04-24T09:07:31.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.877+0000 7f6a65200640 1 -- 172.21.15.83:0/2017691240 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a6019af30 msgr2=0x7f6a6019b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:31.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.877+0000 7f6a65200640 1 --2- 172.21.15.83:0/2017691240 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a6019af30 0x7f6a6019b3e0 secure :-1 s=READY pgs=2246 cs=0 l=1 rev1=1 crypto rx=0x7f6a5800cce0 tx=0x7f6a5800a620 comp rx=0 tx=0).stop 2024-04-24T09:07:31.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.878+0000 7f6a65200640 1 -- 172.21.15.83:0/2017691240 shutdown_connections 2024-04-24T09:07:31.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.878+0000 7f6a65200640 1 --2- 172.21.15.83:0/2017691240 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f6a440a3de0 0x7f6a440a62a0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:31.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.878+0000 7f6a65200640 1 --2- 172.21.15.83:0/2017691240 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a6019af30 0x7f6a6019b3e0 unknown :-1 s=CLOSED pgs=2246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:31.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.878+0000 7f6a65200640 1 --2- 172.21.15.83:0/2017691240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6a60109580 0x7f6a601970d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:31.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.878+0000 7f6a65200640 1 --2- 172.21.15.83:0/2017691240 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6a601044b0 0x7f6a60196b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:31.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.878+0000 7f6a65200640 1 -- 172.21.15.83:0/2017691240 >> 172.21.15.83:0/2017691240 conn(0x7f6a600ffd90 msgr2=0x7f6a60107e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:31.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.879+0000 7f6a65200640 1 -- 172.21.15.83:0/2017691240 shutdown_connections 2024-04-24T09:07:31.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:31.879+0000 7f6a65200640 1 -- 172.21.15.83:0/2017691240 wait complete. 2024-04-24T09:07:31.900 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-24T09:07:32.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.147+0000 7ff54e800640 1 -- 172.21.15.83:0/3076194399 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff54810e820 msgr2=0x7ff548110c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:32.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.147+0000 7ff54e800640 1 --2- 172.21.15.83:0/3076194399 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff54810e820 0x7ff548110c10 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff540009be0 tx=0x7ff54002a6c0 comp rx=0 tx=0).stop 2024-04-24T09:07:32.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.148+0000 7ff54e800640 1 -- 172.21.15.83:0/3076194399 shutdown_connections 2024-04-24T09:07:32.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.148+0000 7ff54e800640 1 --2- 172.21.15.83:0/3076194399 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff54810e820 0x7ff548110c10 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:32.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.148+0000 7ff54e800640 1 --2- 172.21.15.83:0/3076194399 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff548101620 0x7ff54810e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:32.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.148+0000 7ff54e800640 1 --2- 172.21.15.83:0/3076194399 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff548100c70 0x7ff548101050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:32.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.148+0000 7ff54e800640 1 -- 172.21.15.83:0/3076194399 >> 172.21.15.83:0/3076194399 conn(0x7ff5480fc880 msgr2=0x7ff5480feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:32.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.148+0000 7ff54e800640 1 -- 172.21.15.83:0/3076194399 shutdown_connections 2024-04-24T09:07:32.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.148+0000 7ff54e800640 1 -- 172.21.15.83:0/3076194399 wait complete. 2024-04-24T09:07:32.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.150+0000 7ff54e800640 1 Processor -- start 2024-04-24T09:07:32.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.150+0000 7ff54e800640 1 -- start start 2024-04-24T09:07:32.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.150+0000 7ff54e800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff548100c70 0x7ff5481a05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:32.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.150+0000 7ff54e800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff548101620 0x7ff5481a0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:32.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.150+0000 7ff54e800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff54810e820 0x7ff54819a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:32.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.150+0000 7ff54e800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff548114300 con 0x7ff54810e820 2024-04-24T09:07:32.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.150+0000 7ff54e800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff548114180 con 0x7ff548100c70 2024-04-24T09:07:32.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.150+0000 7ff54e800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff548114480 con 0x7ff548101620 2024-04-24T09:07:32.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.151+0000 7ff54d400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff548100c70 0x7ff5481a05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:32.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.151+0000 7ff54de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff54810e820 0x7ff54819a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:32.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.151+0000 7ff54d400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff548100c70 0x7ff5481a05b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:42882/0 (socket says 172.21.15.83:42882) 2024-04-24T09:07:32.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.151+0000 7ff54d400640 1 -- 172.21.15.83:0/2079903006 learned_addr learned my addr 172.21.15.83:0/2079903006 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:07:32.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.151+0000 7ff54ca00640 1 --2- 172.21.15.83:0/2079903006 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff548101620 0x7ff5481a0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:32.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.151+0000 7ff54d400640 1 -- 172.21.15.83:0/2079903006 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff548101620 msgr2=0x7ff5481a0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:32.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.151+0000 7ff54d400640 1 --2- 172.21.15.83:0/2079903006 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff548101620 0x7ff5481a0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:32.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.151+0000 7ff54d400640 1 -- 172.21.15.83:0/2079903006 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff54810e820 msgr2=0x7ff54819a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:32.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.151+0000 7ff54d400640 1 --2- 172.21.15.83:0/2079903006 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff54810e820 0x7ff54819a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:32.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.151+0000 7ff54d400640 1 -- 172.21.15.83:0/2079903006 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff54819af80 con 0x7ff548100c70 2024-04-24T09:07:32.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.152+0000 7ff54d400640 1 --2- 172.21.15.83:0/2079903006 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff548100c70 0x7ff5481a05b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff53c00e9f0 tx=0x7ff53c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:32.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.152+0000 7ff546000640 1 -- 172.21.15.83:0/2079903006 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff53c00cd80 con 0x7ff548100c70 2024-04-24T09:07:32.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.152+0000 7ff546000640 1 -- 172.21.15.83:0/2079903006 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff53c0108e0 con 0x7ff548100c70 2024-04-24T09:07:32.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.152+0000 7ff54e800640 1 -- 172.21.15.83:0/2079903006 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff54819b270 con 0x7ff548100c70 2024-04-24T09:07:32.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.152+0000 7ff546000640 1 -- 172.21.15.83:0/2079903006 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff53c018920 con 0x7ff548100c70 2024-04-24T09:07:32.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.152+0000 7ff54e800640 1 -- 172.21.15.83:0/2079903006 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5481a7530 con 0x7ff548100c70 2024-04-24T09:07:32.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.156+0000 7ff552c5d740 1 Processor -- start 2024-04-24T09:07:32.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.156+0000 7ff552c5d740 1 -- start start 2024-04-24T09:07:32.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.157+0000 7ff552c5d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560a9811a350 0x560a9811a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:32.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.157+0000 7ff552c5d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560a9811ac70 0x560a98124e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:32.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.157+0000 7ff552c5d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a98125340 0x560a98127730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:32.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.157+0000 7ff552c5d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x560a980771d0 con 0x560a98125340 2024-04-24T09:07:32.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.157+0000 7ff552c5d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560a9801d450 con 0x560a9811a350 2024-04-24T09:07:32.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.157+0000 7ff552c5d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x560a98073190 con 0x560a9811ac70 2024-04-24T09:07:32.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.157+0000 7ff546000640 1 -- 172.21.15.83:0/2079903006 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff53c010a90 con 0x7ff548100c70 2024-04-24T09:07:32.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.157+0000 7ff54de00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560a9811a350 0x560a9811a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:32.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.157+0000 7ff54ca00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a98125340 0x560a98127730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:32.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.157+0000 7ff54de00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560a9811a350 0x560a9811a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:42884/0 (socket says 172.21.15.83:42884) 2024-04-24T09:07:32.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.157+0000 7ff54de00640 1 -- 172.21.15.83:0/3879451923 learned_addr learned my addr 172.21.15.83:0/3879451923 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:07:32.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.158+0000 7ff54d400640 1 --2- 172.21.15.83:0/3879451923 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560a9811ac70 0x560a98124e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:32.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.158+0000 7ff54ca00640 1 -- 172.21.15.83:0/3879451923 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560a9811ac70 msgr2=0x560a98124e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:32.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.158+0000 7ff54ca00640 1 --2- 172.21.15.83:0/3879451923 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560a9811ac70 0x560a98124e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:32.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.158+0000 7ff54ca00640 1 -- 172.21.15.83:0/3879451923 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560a9811a350 msgr2=0x560a9811a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:32.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.158+0000 7ff54ca00640 1 --2- 172.21.15.83:0/3879451923 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560a9811a350 0x560a9811a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:32.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.158+0000 7ff54ca00640 1 -- 172.21.15.83:0/3879451923 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a980da310 con 0x560a98125340 2024-04-24T09:07:32.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.158+0000 7ff54ca00640 1 --2- 172.21.15.83:0/3879451923 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a98125340 0x560a98127730 secure :-1 s=READY pgs=2250 cs=0 l=1 rev1=1 crypto rx=0x7ff534009530 tx=0x7ff534011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=effd617a8823a537 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:32.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.159+0000 7ff53a600640 1 -- 172.21.15.83:0/3879451923 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff534009db0 con 0x560a98125340 2024-04-24T09:07:32.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.159+0000 7ff53a600640 1 -- 172.21.15.83:0/3879451923 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff5340035f0 con 0x560a98125340 2024-04-24T09:07:32.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.159+0000 7ff546000640 1 --2- 172.21.15.83:0/2079903006 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff5300a3da0 0x7ff5300a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:32.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.159+0000 7ff552c5d740 1 -- 172.21.15.83:0/3879451923 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a98125340 msgr2=0x560a98127730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:32.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.159+0000 7ff552c5d740 1 --2- 172.21.15.83:0/3879451923 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a98125340 0x560a98127730 secure :-1 s=READY pgs=2250 cs=0 l=1 rev1=1 crypto rx=0x7ff534009530 tx=0x7ff534011810 comp rx=0 tx=0).stop 2024-04-24T09:07:32.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.160+0000 7ff546000640 1 -- 172.21.15.83:0/2079903006 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 9583+0+0 (secure 0 0 0) 0x7ff53c0d0eb0 con 0x7ff548100c70 2024-04-24T09:07:32.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.160+0000 7ff552c5d740 1 -- 172.21.15.83:0/3879451923 shutdown_connections 2024-04-24T09:07:32.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.160+0000 7ff552c5d740 1 --2- 172.21.15.83:0/3879451923 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a98125340 0x560a98127730 unknown :-1 s=CLOSED pgs=2250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:32.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.160+0000 7ff552c5d740 1 --2- 172.21.15.83:0/3879451923 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560a9811ac70 0x560a98124e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:32.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.160+0000 7ff552c5d740 1 --2- 172.21.15.83:0/3879451923 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560a9811a350 0x560a9811a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:32.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.160+0000 7ff552c5d740 1 -- 172.21.15.83:0/3879451923 >> 172.21.15.83:0/3879451923 conn(0x560a98116670 msgr2=0x560a98116a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:32.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.160+0000 7ff54de00640 1 --2- 172.21.15.83:0/2079903006 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff5300a3da0 0x7ff5300a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:32.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.160+0000 7ff552c5d740 1 -- 172.21.15.83:0/3879451923 shutdown_connections 2024-04-24T09:07:32.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.160+0000 7ff552c5d740 1 -- 172.21.15.83:0/3879451923 wait complete. 2024-04-24T09:07:32.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.161+0000 7ff54de00640 1 --2- 172.21.15.83:0/2079903006 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff5300a3da0 0x7ff5300a6260 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7ff5400099a0 tx=0x7ff540005fd0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:32.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.161+0000 7ff552c5d740 1 -- start start 2024-04-24T09:07:32.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.161+0000 7ff552c5d740 1 Processor -- start 2024-04-24T09:07:32.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.162+0000 7ff552c5d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a9811a350 0x560a98124d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:32.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.162+0000 7ff552c5d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560a9811ac70 0x560a98122000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:32.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.162+0000 7ff552c5d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560a98125340 0x560a98122610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:32.163 INFO:tasks.workunit.client.0.smithi083.stdout: 10 15931 7.40 MB/sec execute 3 sec latency 91.935 ms 2024-04-24T09:07:32.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.162+0000 7ff552c5d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x560a98128220 con 0x560a9811a350 2024-04-24T09:07:32.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.163+0000 7ff54d400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a9811a350 0x560a98124d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:32.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.163+0000 7ff54de00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560a98125340 0x560a98122610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:32.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.163+0000 7ff54d400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a9811a350 0x560a98124d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39818/0 (socket says 172.21.15.83:39818) 2024-04-24T09:07:32.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.163+0000 7ff54d400640 1 -- 172.21.15.83:0/389683581 learned_addr learned my addr 172.21.15.83:0/389683581 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:07:32.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.163+0000 7ff54de00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560a98125340 0x560a98122610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:42900/0 (socket says 172.21.15.83:42900) 2024-04-24T09:07:32.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.163+0000 7ff54ca00640 1 --2- 172.21.15.83:0/389683581 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560a9811ac70 0x560a98122000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:32.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.163+0000 7ff552c5d740 1 -- 172.21.15.83:0/389683581 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560a98030810 con 0x560a98125340 2024-04-24T09:07:32.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.163+0000 7ff552c5d740 1 -- 172.21.15.83:0/389683581 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x560a981283a0 con 0x560a9811ac70 2024-04-24T09:07:32.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.163+0000 7ff552c5d740 10 client.0 _ll_register_callbacks cb 0x560a97ffe140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:07:32.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.163+0000 7ff54d400640 1 -- 172.21.15.83:0/389683581 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560a9811ac70 msgr2=0x560a98122000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:32.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.163+0000 7ff54d400640 1 --2- 172.21.15.83:0/389683581 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560a9811ac70 0x560a98122000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:32.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.163+0000 7ff54d400640 1 -- 172.21.15.83:0/389683581 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560a98125340 msgr2=0x560a98122610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:32.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.163+0000 7ff54d400640 1 --2- 172.21.15.83:0/389683581 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560a98125340 0x560a98122610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:32.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.163+0000 7ff54d400640 1 -- 172.21.15.83:0/389683581 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a980413f0 con 0x560a9811a350 2024-04-24T09:07:32.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.163+0000 7ff54de00640 1 --2- 172.21.15.83:0/389683581 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560a98125340 0x560a98122610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:07:32.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.163+0000 7ff552c5d740 1 -- 172.21.15.83:0/389683581 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5340035f0 con 0x560a9811a350 2024-04-24T09:07:32.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.164+0000 7ff54d400640 1 --2- 172.21.15.83:0/389683581 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a9811a350 0x560a98124d10 secure :-1 s=READY pgs=2251 cs=0 l=1 rev1=1 crypto rx=0x7ff53c00e9c0 tx=0x7ff53c0ce6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b12594bdeffcfbd2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:32.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.164+0000 7ff52ea00640 10 client.21174 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:07:32.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.164+0000 7ff52ea00640 1 -- 172.21.15.83:0/389683581 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff53c0bbd90 con 0x560a9811a350 2024-04-24T09:07:32.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.164+0000 7ff52ea00640 1 -- 172.21.15.83:0/389683581 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9583+0+0 (secure 0 0 0) 0x7ff53c010430 con 0x560a9811a350 2024-04-24T09:07:32.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.164+0000 7ff52ea00640 1 -- 172.21.15.83:0/389683581 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7ff53c010670 con 0x560a9811a350 2024-04-24T09:07:32.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.164+0000 7ff552c5d740 10 client.21174 fetch_fsmap learned FSMap version 51 2024-04-24T09:07:32.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.164+0000 7ff552c5d740 1 -- 172.21.15.83:0/389683581 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({fsmap=51}) -- 0x560a97d893f0 con 0x560a9811a350 2024-04-24T09:07:32.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.165+0000 7ff52ea00640 1 -- 172.21.15.83:0/389683581 <== mon.0 v2:172.21.15.83:3300/0 4 ==== fsmap(e 51) ==== 8406+0+0 (secure 0 0 0) 0x7ff53c09bed0 con 0x560a9811a350 2024-04-24T09:07:32.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.165+0000 7ff552c5d740 10 client.21174 fetch_fsmap finished waiting for FSMap version 51 2024-04-24T09:07:32.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.165+0000 7ff552c5d740 10 client.21174 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-24T09:07:32.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.165+0000 7ff552c5d740 20 client.21174 populate_metadata read hostname 'smithi083' 2024-04-24T09:07:32.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.165+0000 7ff552c5d740 1 --2- 172.21.15.83:0/389683581 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x560a98135330 0x560a98137720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:32.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.165+0000 7ff552c5d740 4 client.21174 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:07:32.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.165+0000 7ff552c5d740 1 -- 172.21.15.83:0/389683581 --> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560a98084bf0 con 0x560a98135330 2024-04-24T09:07:32.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.166+0000 7ff54ca00640 1 --2- 172.21.15.83:0/389683581 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x560a98135330 0x560a98137720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:32.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.166+0000 7ff54ca00640 1 --2- 172.21.15.83:0/389683581 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x560a98135330 0x560a98137720 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3b0d68da6e88de88 server_cookie=8a47a51da2362c69 in_seq=0 out_seq=0 2024-04-24T09:07:32.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.166+0000 7ff52ea00640 10 client.21174 ms_handle_connect on v2:172.21.15.138:6832/1157967658 2024-04-24T09:07:32.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.167+0000 7ff52ea00640 1 -- 172.21.15.83:0/389683581 <== mds.3 v2:172.21.15.138:6832/1157967658 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560a98084bf0 con 0x560a98135330 2024-04-24T09:07:32.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.167+0000 7ff52ea00640 10 client.21174 handle_command_reply: tid=0 2024-04-24T09:07:32.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.167+0000 7ff552c5d740 1 client.21174 shutdown 2024-04-24T09:07:32.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.168+0000 7ff552c5d740 1 -- 172.21.15.83:0/389683581 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a9811a350 msgr2=0x560a98124d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:32.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.168+0000 7ff552c5d740 1 --2- 172.21.15.83:0/389683581 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a9811a350 0x560a98124d10 secure :-1 s=READY pgs=2251 cs=0 l=1 rev1=1 crypto rx=0x7ff53c00e9c0 tx=0x7ff53c0ce6f0 comp rx=0 tx=0).stop 2024-04-24T09:07:32.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.168+0000 7ff552c5d740 1 -- 172.21.15.83:0/389683581 shutdown_connections 2024-04-24T09:07:32.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.168+0000 7ff552c5d740 1 --2- 172.21.15.83:0/389683581 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560a98125340 0x560a98122610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:32.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.168+0000 7ff552c5d740 1 --2- 172.21.15.83:0/389683581 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x560a98135330 0x560a98137720 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:32.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.168+0000 7ff552c5d740 1 --2- 172.21.15.83:0/389683581 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560a9811ac70 0x560a98122000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:32.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.168+0000 7ff552c5d740 1 --2- 172.21.15.83:0/389683581 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a9811a350 0x560a98124d10 secure :-1 s=CLOSED pgs=2251 cs=0 l=1 rev1=1 crypto rx=0x7ff53c00e9c0 tx=0x7ff53c0ce6f0 comp rx=0 tx=0).stop 2024-04-24T09:07:32.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.168+0000 7ff552c5d740 1 -- 172.21.15.83:0/389683581 >> 172.21.15.83:0/389683581 conn(0x560a98116670 msgr2=0x560a9811c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:32.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.168+0000 7ff52ea00640 0 client.21174 ms_handle_reset on v2:172.21.15.138:6832/1157967658 2024-04-24T09:07:32.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.168+0000 7ff552c5d740 1 -- 172.21.15.83:0/389683581 shutdown_connections 2024-04-24T09:07:32.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.168+0000 7ff552c5d740 1 -- 172.21.15.83:0/389683581 wait complete. 2024-04-24T09:07:32.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.168+0000 7ff552c5d740 20 client.21174 trim_cache size 0 max 16384 2024-04-24T09:07:32.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.192+0000 7ff552c5d740 1 Processor -- start 2024-04-24T09:07:32.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.192+0000 7ff552c5d740 1 -- start start 2024-04-24T09:07:32.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.193+0000 7ff552c5d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a9811ac70 0x560a9811cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:32.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.193+0000 7ff552c5d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560a98125340 0x560a98124840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:32.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.193+0000 7ff552c5d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560a98124d80 0x560a9810cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:32.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.193+0000 7ff552c5d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff5481ad240 con 0x560a9811ac70 2024-04-24T09:07:32.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.193+0000 7ff552c5d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff5481aeed0 con 0x560a98125340 2024-04-24T09:07:32.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.193+0000 7ff552c5d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff5481ad910 con 0x560a98124d80 2024-04-24T09:07:32.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.193+0000 7ff54ca00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560a98125340 0x560a98124840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:32.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.193+0000 7ff54ca00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560a98125340 0x560a98124840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:42908/0 (socket says 172.21.15.83:42908) 2024-04-24T09:07:32.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.193+0000 7ff54d400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a9811ac70 0x560a9811cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:32.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.193+0000 7ff54ca00640 1 -- 172.21.15.83:0/397028095 learned_addr learned my addr 172.21.15.83:0/397028095 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:07:32.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.194+0000 7ff54de00640 1 --2- 172.21.15.83:0/397028095 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560a98124d80 0x560a9810cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:32.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.194+0000 7ff54ca00640 1 -- 172.21.15.83:0/397028095 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560a98124d80 msgr2=0x560a9810cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:32.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.194+0000 7ff54ca00640 1 --2- 172.21.15.83:0/397028095 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560a98124d80 0x560a9810cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:32.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.194+0000 7ff54ca00640 1 -- 172.21.15.83:0/397028095 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a9811ac70 msgr2=0x560a9811cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:32.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.194+0000 7ff54ca00640 1 --2- 172.21.15.83:0/397028095 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a9811ac70 0x560a9811cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:32.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.194+0000 7ff54ca00640 1 -- 172.21.15.83:0/397028095 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a98137a90 con 0x560a98125340 2024-04-24T09:07:32.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.194+0000 7ff54d400640 1 --2- 172.21.15.83:0/397028095 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a9811ac70 0x560a9811cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:07:32.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.195+0000 7ff54ca00640 1 --2- 172.21.15.83:0/397028095 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560a98125340 0x560a98124840 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x560a98117800 tx=0x7ff534003930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51d7eef9a7b61b2b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:32.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.195+0000 7ff539c00640 1 -- 172.21.15.83:0/397028095 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff534030070 con 0x560a98125340 2024-04-24T09:07:32.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.195+0000 7ff539c00640 1 -- 172.21.15.83:0/397028095 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff534003aa0 con 0x560a98125340 2024-04-24T09:07:32.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.196+0000 7ff539c00640 1 -- 172.21.15.83:0/397028095 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff53401d040 con 0x560a98125340 2024-04-24T09:07:32.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.197+0000 7ff552c5d740 1 -- 172.21.15.83:0/397028095 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560a98125340 msgr2=0x560a98124840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:32.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.197+0000 7ff552c5d740 1 --2- 172.21.15.83:0/397028095 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560a98125340 0x560a98124840 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x560a98117800 tx=0x7ff534003930 comp rx=0 tx=0).stop 2024-04-24T09:07:32.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.197+0000 7ff552c5d740 1 -- 172.21.15.83:0/397028095 shutdown_connections 2024-04-24T09:07:32.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.197+0000 7ff552c5d740 1 --2- 172.21.15.83:0/397028095 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560a98124d80 0x560a9810cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:32.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.197+0000 7ff552c5d740 1 --2- 172.21.15.83:0/397028095 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560a98125340 0x560a98124840 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:32.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.197+0000 7ff552c5d740 1 --2- 172.21.15.83:0/397028095 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a9811ac70 0x560a9811cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:32.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.197+0000 7ff552c5d740 1 -- 172.21.15.83:0/397028095 >> 172.21.15.83:0/397028095 conn(0x560a98116670 msgr2=0x560a9810c6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:32.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.197+0000 7ff552c5d740 1 -- 172.21.15.83:0/397028095 shutdown_connections 2024-04-24T09:07:32.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.197+0000 7ff552c5d740 1 -- 172.21.15.83:0/397028095 wait complete. 2024-04-24T09:07:32.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.198+0000 7ff552c5d740 1 -- start start 2024-04-24T09:07:32.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.198+0000 7ff552c5d740 1 Processor -- start 2024-04-24T09:07:32.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.199+0000 7ff552c5d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560a9811ac70 0x560a9812d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:32.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.199+0000 7ff552c5d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560a98124d80 0x560a9812dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:32.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.199+0000 7ff552c5d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a98125340 0x560a98198cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:32.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.199+0000 7ff552c5d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff5481ae740 con 0x560a98125340 2024-04-24T09:07:32.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.199+0000 7ff552c5d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff5481ae040 con 0x560a98124d80 2024-04-24T09:07:32.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.199+0000 7ff552c5d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff5481ab260 con 0x560a9811ac70 2024-04-24T09:07:32.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.199+0000 7ff552c5d740 10 client.0 _ll_register_callbacks cb 0x7ff53c00cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:07:32.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.199+0000 7ff54ca00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560a98124d80 0x560a9812dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:32.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.199+0000 7ff54ca00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560a98124d80 0x560a9812dbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:42922/0 (socket says 172.21.15.83:42922) 2024-04-24T09:07:32.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.199+0000 7ff54ca00640 1 -- 172.21.15.83:0/282919418 learned_addr learned my addr 172.21.15.83:0/282919418 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:07:32.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.200+0000 7ff54d400640 1 --2- 172.21.15.83:0/282919418 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560a9811ac70 0x560a9812d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:32.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.200+0000 7ff54de00640 1 --2- 172.21.15.83:0/282919418 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a98125340 0x560a98198cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:32.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.200+0000 7ff54ca00640 1 -- 172.21.15.83:0/282919418 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560a9811ac70 msgr2=0x560a9812d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:32.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.200+0000 7ff54ca00640 1 --2- 172.21.15.83:0/282919418 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560a9811ac70 0x560a9812d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:32.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.200+0000 7ff54ca00640 1 -- 172.21.15.83:0/282919418 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a98125340 msgr2=0x560a98198cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:32.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.200+0000 7ff54ca00640 1 --2- 172.21.15.83:0/282919418 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a98125340 0x560a98198cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:32.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.200+0000 7ff54ca00640 1 -- 172.21.15.83:0/282919418 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a97e537d0 con 0x560a98124d80 2024-04-24T09:07:32.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.200+0000 7ff54d400640 1 --2- 172.21.15.83:0/282919418 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560a9811ac70 0x560a9812d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:07:32.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.200+0000 7ff54ca00640 1 --2- 172.21.15.83:0/282919418 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560a98124d80 0x560a9812dbb0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff534013040 tx=0x7ff534011e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80057dc1d0f2ad4b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:32.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.200+0000 7ff52ea00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:07:32.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.201+0000 7ff52ea00640 1 -- 172.21.15.83:0/282919418 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff534030030 con 0x560a98124d80 2024-04-24T09:07:32.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.201+0000 7ff52ea00640 1 -- 172.21.15.83:0/282919418 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9583+0+0 (secure 0 0 0) 0x7ff534031070 con 0x560a98124d80 2024-04-24T09:07:32.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.201+0000 7ff552c5d740 1 -- 172.21.15.83:0/282919418 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff534003aa0 con 0x560a98124d80 2024-04-24T09:07:32.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.201+0000 7ff52ea00640 1 -- 172.21.15.83:0/282919418 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7ff53401d040 con 0x560a98124d80 2024-04-24T09:07:32.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.202+0000 7ff552c5d740 10 client.38333 fetch_fsmap learned FSMap version 51 2024-04-24T09:07:32.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.202+0000 7ff552c5d740 1 -- 172.21.15.83:0/282919418 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x560a9811f750 con 0x560a98124d80 2024-04-24T09:07:32.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.202+0000 7ff52ea00640 1 -- 172.21.15.83:0/282919418 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff534003aa0 con 0x560a98124d80 2024-04-24T09:07:32.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.202+0000 7ff52ea00640 1 -- 172.21.15.83:0/282919418 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 51) ==== 8406+0+0 (secure 0 0 0) 0x7ff53402b040 con 0x560a98124d80 2024-04-24T09:07:32.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.202+0000 7ff552c5d740 10 client.38333 fetch_fsmap finished waiting for FSMap version 51 2024-04-24T09:07:32.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.202+0000 7ff552c5d740 10 client.38333 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-24T09:07:32.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.202+0000 7ff552c5d740 20 client.38333 populate_metadata read hostname 'smithi083' 2024-04-24T09:07:32.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.202+0000 7ff552c5d740 1 --2- 172.21.15.83:0/282919418 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x560a98110a80 0x560a98112e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:32.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.202+0000 7ff552c5d740 4 client.38333 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:07:32.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.202+0000 7ff552c5d740 1 -- 172.21.15.83:0/282919418 --> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560a9807ea60 con 0x560a98110a80 2024-04-24T09:07:32.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.203+0000 7ff54d400640 1 --2- 172.21.15.83:0/282919418 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x560a98110a80 0x560a98112e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:32.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.204+0000 7ff54d400640 1 --2- 172.21.15.83:0/282919418 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x560a98110a80 0x560a98112e70 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c15f20a01334a517 server_cookie=6cc04df7d77e297c in_seq=0 out_seq=0 2024-04-24T09:07:32.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.204+0000 7ff52ea00640 10 client.38333 ms_handle_connect on v2:172.21.15.138:6832/1157967658 2024-04-24T09:07:32.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.204+0000 7ff52ea00640 1 -- 172.21.15.83:0/282919418 <== mds.3 v2:172.21.15.138:6832/1157967658 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560a9807ea60 con 0x560a98110a80 2024-04-24T09:07:32.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.204+0000 7ff52ea00640 10 client.38333 handle_command_reply: tid=0 2024-04-24T09:07:32.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.204+0000 7ff552c5d740 1 client.38333 shutdown 2024-04-24T09:07:32.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.205+0000 7ff552c5d740 1 -- 172.21.15.83:0/282919418 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560a98124d80 msgr2=0x560a9812dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:32.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.205+0000 7ff552c5d740 1 --2- 172.21.15.83:0/282919418 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560a98124d80 0x560a9812dbb0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff534013040 tx=0x7ff534011e20 comp rx=0 tx=0).stop 2024-04-24T09:07:32.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.205+0000 7ff552c5d740 1 -- 172.21.15.83:0/282919418 shutdown_connections 2024-04-24T09:07:32.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.205+0000 7ff552c5d740 1 --2- 172.21.15.83:0/282919418 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a98125340 0x560a98198cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:32.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.205+0000 7ff552c5d740 1 --2- 172.21.15.83:0/282919418 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560a98124d80 0x560a9812dbb0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:32.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.205+0000 7ff552c5d740 1 --2- 172.21.15.83:0/282919418 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x560a98110a80 0x560a98112e70 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:32.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.205+0000 7ff552c5d740 1 --2- 172.21.15.83:0/282919418 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560a9811ac70 0x560a9812d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:32.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.205+0000 7ff52ea00640 0 client.38333 ms_handle_reset on v2:172.21.15.138:6832/1157967658 2024-04-24T09:07:32.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.205+0000 7ff552c5d740 1 -- 172.21.15.83:0/282919418 >> 172.21.15.83:0/282919418 conn(0x560a98116670 msgr2=0x560a981240c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:32.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.205+0000 7ff552c5d740 1 -- 172.21.15.83:0/282919418 shutdown_connections 2024-04-24T09:07:32.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.205+0000 7ff552c5d740 1 -- 172.21.15.83:0/282919418 wait complete. 2024-04-24T09:07:32.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.205+0000 7ff552c5d740 20 client.38333 trim_cache size 0 max 16384 2024-04-24T09:07:32.211 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:07:32.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.206+0000 7ff54e800640 1 -- 172.21.15.83:0/2079903006 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff5300a3da0 msgr2=0x7ff5300a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:32.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.206+0000 7ff54e800640 1 --2- 172.21.15.83:0/2079903006 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff5300a3da0 0x7ff5300a6260 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7ff5400099a0 tx=0x7ff540005fd0 comp rx=0 tx=0).stop 2024-04-24T09:07:32.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.206+0000 7ff54e800640 1 -- 172.21.15.83:0/2079903006 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff548100c70 msgr2=0x7ff5481a05b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:32.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.206+0000 7ff54e800640 1 --2- 172.21.15.83:0/2079903006 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff548100c70 0x7ff5481a05b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff53c00e9f0 tx=0x7ff53c00eeb0 comp rx=0 tx=0).stop 2024-04-24T09:07:32.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.206+0000 7ff54e800640 1 -- 172.21.15.83:0/2079903006 shutdown_connections 2024-04-24T09:07:32.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.206+0000 7ff54e800640 1 --2- 172.21.15.83:0/2079903006 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff5300a3da0 0x7ff5300a6260 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:32.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.206+0000 7ff54e800640 1 --2- 172.21.15.83:0/2079903006 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff54810e820 0x7ff54819a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:32.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.206+0000 7ff54e800640 1 --2- 172.21.15.83:0/2079903006 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff548101620 0x7ff5481a0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:32.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.206+0000 7ff54e800640 1 --2- 172.21.15.83:0/2079903006 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff548100c70 0x7ff5481a05b0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:32.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.206+0000 7ff54e800640 1 -- 172.21.15.83:0/2079903006 >> 172.21.15.83:0/2079903006 conn(0x7ff5480fc880 msgr2=0x7ff5480fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:32.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.207+0000 7ff54e800640 1 -- 172.21.15.83:0/2079903006 shutdown_connections 2024-04-24T09:07:32.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.207+0000 7ff54e800640 1 -- 172.21.15.83:0/2079903006 wait complete. 2024-04-24T09:07:32.225 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:07:31 smithi083 ceph-mon[27907]: pgmap v500: 129 pgs: 129 active+clean; 411 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.8 MiB/s wr, 237 op/s 2024-04-24T09:07:32.226 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:07:31 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/3565885439' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:07:32.229 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T09:07:32.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:07:31 smithi189 ceph-mon[31791]: pgmap v500: 129 pgs: 129 active+clean; 411 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.8 MiB/s wr, 237 op/s 2024-04-24T09:07:32.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:07:31 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/3565885439' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:07:32.460 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:07:31 smithi138 ceph-mon[31441]: pgmap v500: 129 pgs: 129 active+clean; 411 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.8 MiB/s wr, 237 op/s 2024-04-24T09:07:32.460 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:07:31 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/3565885439' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:07:32.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.487+0000 7fa23cc00640 1 -- 172.21.15.83:0/914296621 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa238104cd0 msgr2=0x7fa238108d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:32.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.487+0000 7fa23cc00640 1 --2- 172.21.15.83:0/914296621 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa238104cd0 0x7fa238108d10 secure :-1 s=READY pgs=2252 cs=0 l=1 rev1=1 crypto rx=0x7fa224009900 tx=0x7fa22402a3f0 comp rx=0 tx=0).stop 2024-04-24T09:07:32.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.487+0000 7fa23cc00640 1 -- 172.21.15.83:0/914296621 shutdown_connections 2024-04-24T09:07:32.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.487+0000 7fa23cc00640 1 --2- 172.21.15.83:0/914296621 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2381093f0 0x7fa238110fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:32.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.487+0000 7fa23cc00640 1 --2- 172.21.15.83:0/914296621 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa238104cd0 0x7fa238108d10 unknown :-1 s=CLOSED pgs=2252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:32.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.487+0000 7fa23cc00640 1 --2- 172.21.15.83:0/914296621 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa238104320 0x7fa238104700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:32.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.487+0000 7fa23cc00640 1 -- 172.21.15.83:0/914296621 >> 172.21.15.83:0/914296621 conn(0x7fa2380ffb40 msgr2=0x7fa238101f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:32.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.488+0000 7fa23cc00640 1 -- 172.21.15.83:0/914296621 shutdown_connections 2024-04-24T09:07:32.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.488+0000 7fa23cc00640 1 -- 172.21.15.83:0/914296621 wait complete. 2024-04-24T09:07:32.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.489+0000 7fa23cc00640 1 Processor -- start 2024-04-24T09:07:32.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.489+0000 7fa23cc00640 1 -- start start 2024-04-24T09:07:32.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.489+0000 7fa23cc00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa238104320 0x7fa2381a0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:32.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.489+0000 7fa23cc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2381093f0 0x7fa2381a0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:32.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.489+0000 7fa23cc00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa23819a970 0x7fa23819ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:32.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.489+0000 7fa23cc00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa238114380 con 0x7fa2381093f0 2024-04-24T09:07:32.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.489+0000 7fa23cc00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa238114200 con 0x7fa238104320 2024-04-24T09:07:32.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.489+0000 7fa23cc00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa238114500 con 0x7fa23819a970 2024-04-24T09:07:32.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.490+0000 7fa237400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa238104320 0x7fa2381a0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:32.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.490+0000 7fa237400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa238104320 0x7fa2381a0770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:42952/0 (socket says 172.21.15.83:42952) 2024-04-24T09:07:32.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.490+0000 7fa237400640 1 -- 172.21.15.83:0/2821923815 learned_addr learned my addr 172.21.15.83:0/2821923815 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:07:32.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.490+0000 7fa236a00640 1 --2- 172.21.15.83:0/2821923815 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2381093f0 0x7fa2381a0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:32.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.490+0000 7fa237e00640 1 --2- 172.21.15.83:0/2821923815 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa23819a970 0x7fa23819ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:32.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.490+0000 7fa237400640 1 -- 172.21.15.83:0/2821923815 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa23819a970 msgr2=0x7fa23819ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:32.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.490+0000 7fa237400640 1 --2- 172.21.15.83:0/2821923815 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa23819a970 0x7fa23819ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:32.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.490+0000 7fa237400640 1 -- 172.21.15.83:0/2821923815 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2381093f0 msgr2=0x7fa2381a0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:32.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.490+0000 7fa237400640 1 --2- 172.21.15.83:0/2821923815 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2381093f0 0x7fa2381a0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:32.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.490+0000 7fa237400640 1 -- 172.21.15.83:0/2821923815 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa23819b490 con 0x7fa238104320 2024-04-24T09:07:32.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.491+0000 7fa237400640 1 --2- 172.21.15.83:0/2821923815 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa238104320 0x7fa2381a0770 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa22800d770 tx=0x7fa22800dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:32.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.491+0000 7fa22fe00640 1 -- 172.21.15.83:0/2821923815 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa228017070 con 0x7fa238104320 2024-04-24T09:07:32.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.491+0000 7fa22fe00640 1 -- 172.21.15.83:0/2821923815 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa2280054d0 con 0x7fa238104320 2024-04-24T09:07:32.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.491+0000 7fa22fe00640 1 -- 172.21.15.83:0/2821923815 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa228004cf0 con 0x7fa238104320 2024-04-24T09:07:32.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.491+0000 7fa23cc00640 1 -- 172.21.15.83:0/2821923815 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa23819b780 con 0x7fa238104320 2024-04-24T09:07:32.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.491+0000 7fa23cc00640 1 -- 172.21.15.83:0/2821923815 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2381a7930 con 0x7fa238104320 2024-04-24T09:07:32.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.496+0000 7fa24118a740 1 Processor -- start 2024-04-24T09:07:32.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.496+0000 7fa24118a740 1 -- start start 2024-04-24T09:07:32.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.496+0000 7fa24118a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559b85f05350 0x559b85f05730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:32.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.496+0000 7fa24118a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b85f05c70 0x559b85f0fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:32.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.496+0000 7fa24118a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559b85f10340 0x559b85f12730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:32.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.496+0000 7fa24118a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559b85e621d0 con 0x559b85f05c70 2024-04-24T09:07:32.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.496+0000 7fa24118a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559b85e08450 con 0x559b85f10340 2024-04-24T09:07:32.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.496+0000 7fa24118a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559b85e5e190 con 0x559b85f05350 2024-04-24T09:07:32.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.497+0000 7fa237400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b85f05c70 0x559b85f0fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:32.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.497+0000 7fa236a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559b85f10340 0x559b85f12730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:32.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.497+0000 7fa236a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559b85f10340 0x559b85f12730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:42960/0 (socket says 172.21.15.83:42960) 2024-04-24T09:07:32.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.497+0000 7fa237400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b85f05c70 0x559b85f0fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39860/0 (socket says 172.21.15.83:39860) 2024-04-24T09:07:32.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.497+0000 7fa236a00640 1 -- 172.21.15.83:0/2665900024 learned_addr learned my addr 172.21.15.83:0/2665900024 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:07:32.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.497+0000 7fa22fe00640 1 -- 172.21.15.83:0/2821923815 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa228004e90 con 0x7fa238104320 2024-04-24T09:07:32.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.497+0000 7fa237e00640 1 --2- 172.21.15.83:0/2665900024 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559b85f05350 0x559b85f05730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:32.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.498+0000 7fa236a00640 1 -- 172.21.15.83:0/2665900024 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559b85f05350 msgr2=0x559b85f05730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:32.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.498+0000 7fa236a00640 1 --2- 172.21.15.83:0/2665900024 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559b85f05350 0x559b85f05730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:32.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.498+0000 7fa236a00640 1 -- 172.21.15.83:0/2665900024 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b85f05c70 msgr2=0x559b85f0fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:32.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.498+0000 7fa236a00640 1 --2- 172.21.15.83:0/2665900024 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b85f05c70 0x559b85f0fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:32.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.498+0000 7fa236a00640 1 -- 172.21.15.83:0/2665900024 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b85ec5310 con 0x559b85f10340 2024-04-24T09:07:32.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.498+0000 7fa236a00640 1 --2- 172.21.15.83:0/2665900024 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559b85f10340 0x559b85f12730 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa224000c00 tx=0x7fa224004170 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e276ab61b004291d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:32.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.499+0000 7fa223e00640 1 -- 172.21.15.83:0/2665900024 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa224040070 con 0x559b85f10340 2024-04-24T09:07:32.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.499+0000 7fa223e00640 1 -- 172.21.15.83:0/2665900024 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa2240054f0 con 0x559b85f10340 2024-04-24T09:07:32.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.499+0000 7fa223e00640 1 -- 172.21.15.83:0/2665900024 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa224035040 con 0x559b85f10340 2024-04-24T09:07:32.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.499+0000 7fa22fe00640 1 --2- 172.21.15.83:0/2821923815 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa21c0a3da0 0x7fa21c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:32.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.499+0000 7fa237e00640 1 --2- 172.21.15.83:0/2821923815 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa21c0a3da0 0x7fa21c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:32.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.500+0000 7fa22fe00640 1 -- 172.21.15.83:0/2821923815 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 9583+0+0 (secure 0 0 0) 0x7fa2280d0710 con 0x7fa238104320 2024-04-24T09:07:32.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.500+0000 7fa24118a740 1 -- 172.21.15.83:0/2665900024 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559b85f10340 msgr2=0x559b85f12730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:32.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.500+0000 7fa24118a740 1 --2- 172.21.15.83:0/2665900024 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559b85f10340 0x559b85f12730 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa224000c00 tx=0x7fa224004170 comp rx=0 tx=0).stop 2024-04-24T09:07:32.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.500+0000 7fa237e00640 1 --2- 172.21.15.83:0/2821923815 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa21c0a3da0 0x7fa21c0a6260 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fa230005fd0 tx=0x7fa230009500 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:32.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.500+0000 7fa24118a740 1 -- 172.21.15.83:0/2665900024 shutdown_connections 2024-04-24T09:07:32.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.500+0000 7fa24118a740 1 --2- 172.21.15.83:0/2665900024 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559b85f10340 0x559b85f12730 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:32.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.500+0000 7fa24118a740 1 --2- 172.21.15.83:0/2665900024 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b85f05c70 0x559b85f0fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:32.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.500+0000 7fa24118a740 1 --2- 172.21.15.83:0/2665900024 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559b85f05350 0x559b85f05730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:32.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.500+0000 7fa24118a740 1 -- 172.21.15.83:0/2665900024 >> 172.21.15.83:0/2665900024 conn(0x559b85f01670 msgr2=0x559b85f01a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:32.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.501+0000 7fa24118a740 1 -- 172.21.15.83:0/2665900024 shutdown_connections 2024-04-24T09:07:32.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.501+0000 7fa24118a740 1 -- 172.21.15.83:0/2665900024 wait complete. 2024-04-24T09:07:32.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.501+0000 7fa24118a740 1 -- start start 2024-04-24T09:07:32.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.501+0000 7fa24118a740 1 Processor -- start 2024-04-24T09:07:32.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.502+0000 7fa24118a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559b85f05350 0x559b85f0fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:32.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.502+0000 7fa24118a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559b85f05c70 0x559b85f0d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:32.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.502+0000 7fa24118a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b85f10340 0x559b85f0d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:32.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.502+0000 7fa24118a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559b85f13170 con 0x559b85f10340 2024-04-24T09:07:32.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.502+0000 7fa24118a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559b85e1b810 con 0x559b85f05350 2024-04-24T09:07:32.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.502+0000 7fa24118a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559b85f132f0 con 0x559b85f05c70 2024-04-24T09:07:32.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.502+0000 7fa24118a740 10 client.0 _ll_register_callbacks cb 0x559b85de9140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:07:32.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.502+0000 7fa237e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b85f10340 0x559b85f0d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:32.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.502+0000 7fa237400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559b85f05350 0x559b85f0fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:32.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.502+0000 7fa237e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b85f10340 0x559b85f0d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39864/0 (socket says 172.21.15.83:39864) 2024-04-24T09:07:32.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.502+0000 7fa237e00640 1 -- 172.21.15.83:0/1169893906 learned_addr learned my addr 172.21.15.83:0/1169893906 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:07:32.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.503+0000 7fa236a00640 1 --2- 172.21.15.83:0/1169893906 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559b85f05c70 0x559b85f0d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:32.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.503+0000 7fa237e00640 1 -- 172.21.15.83:0/1169893906 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559b85f05c70 msgr2=0x559b85f0d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:32.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.503+0000 7fa237e00640 1 --2- 172.21.15.83:0/1169893906 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559b85f05c70 0x559b85f0d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:32.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.503+0000 7fa237e00640 1 -- 172.21.15.83:0/1169893906 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559b85f05350 msgr2=0x559b85f0fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:32.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.503+0000 7fa237e00640 1 --2- 172.21.15.83:0/1169893906 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559b85f05350 0x559b85f0fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:32.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.503+0000 7fa237e00640 1 -- 172.21.15.83:0/1169893906 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b85e2c3f0 con 0x559b85f10340 2024-04-24T09:07:32.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.503+0000 7fa236a00640 1 --2- 172.21.15.83:0/1169893906 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559b85f05c70 0x559b85f0d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:07:32.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.504+0000 7fa237e00640 1 --2- 172.21.15.83:0/1169893906 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b85f10340 0x559b85f0d700 secure :-1 s=READY pgs=2253 cs=0 l=1 rev1=1 crypto rx=0x7fa2300049d0 tx=0x7fa230004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=906ffe0b0f6234e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:32.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.504+0000 7fa220c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:07:32.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.504+0000 7fa220c00640 1 -- 172.21.15.83:0/1169893906 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa23002c060 con 0x559b85f10340 2024-04-24T09:07:32.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.504+0000 7fa220c00640 1 -- 172.21.15.83:0/1169893906 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9583+0+0 (secure 0 0 0) 0x7fa230017030 con 0x559b85f10340 2024-04-24T09:07:32.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.504+0000 7fa24118a740 1 -- 172.21.15.83:0/1169893906 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2240054f0 con 0x559b85f10340 2024-04-24T09:07:32.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.505+0000 7fa220c00640 1 -- 172.21.15.83:0/1169893906 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7fa230011790 con 0x559b85f10340 2024-04-24T09:07:32.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.505+0000 7fa24118a740 10 client.21192 fetch_fsmap learned FSMap version 51 2024-04-24T09:07:32.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.505+0000 7fa24118a740 1 -- 172.21.15.83:0/1169893906 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x559b85b743f0 con 0x559b85f10340 2024-04-24T09:07:32.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.506+0000 7fa220c00640 1 -- 172.21.15.83:0/1169893906 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa2240054f0 con 0x559b85f10340 2024-04-24T09:07:32.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.506+0000 7fa220c00640 1 -- 172.21.15.83:0/1169893906 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 51) ==== 8406+0+0 (secure 0 0 0) 0x7fa230027b30 con 0x559b85f10340 2024-04-24T09:07:32.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.506+0000 7fa24118a740 10 client.21192 fetch_fsmap finished waiting for FSMap version 51 2024-04-24T09:07:32.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.506+0000 7fa24118a740 10 client.21192 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-24T09:07:32.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.506+0000 7fa24118a740 20 client.21192 populate_metadata read hostname 'smithi083' 2024-04-24T09:07:32.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.506+0000 7fa24118a740 1 --2- 172.21.15.83:0/1169893906 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x559b85f20510 0x559b85f22920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:32.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.506+0000 7fa24118a740 4 client.21192 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:07:32.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.506+0000 7fa24118a740 1 -- 172.21.15.83:0/1169893906 --> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559b85e6fbf0 con 0x559b85f20510 2024-04-24T09:07:32.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.507+0000 7fa237400640 1 --2- 172.21.15.83:0/1169893906 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x559b85f20510 0x559b85f22920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:32.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.508+0000 7fa237400640 1 --2- 172.21.15.83:0/1169893906 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x559b85f20510 0x559b85f22920 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9c76aaa373a169cc server_cookie=7caf93bae35b2d32 in_seq=0 out_seq=0 2024-04-24T09:07:32.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.508+0000 7fa220c00640 10 client.21192 ms_handle_connect on v2:172.21.15.138:6834/1909219758 2024-04-24T09:07:32.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.509+0000 7fa220c00640 1 -- 172.21.15.83:0/1169893906 <== mds.2 v2:172.21.15.138:6834/1909219758 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559b85e6fbf0 con 0x559b85f20510 2024-04-24T09:07:32.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.509+0000 7fa220c00640 10 client.21192 handle_command_reply: tid=0 2024-04-24T09:07:32.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.509+0000 7fa24118a740 1 client.21192 shutdown 2024-04-24T09:07:32.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.510+0000 7fa24118a740 1 -- 172.21.15.83:0/1169893906 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b85f10340 msgr2=0x559b85f0d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:32.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.510+0000 7fa24118a740 1 --2- 172.21.15.83:0/1169893906 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b85f10340 0x559b85f0d700 secure :-1 s=READY pgs=2253 cs=0 l=1 rev1=1 crypto rx=0x7fa2300049d0 tx=0x7fa230004ab0 comp rx=0 tx=0).stop 2024-04-24T09:07:32.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.510+0000 7fa24118a740 1 -- 172.21.15.83:0/1169893906 shutdown_connections 2024-04-24T09:07:32.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.510+0000 7fa24118a740 1 --2- 172.21.15.83:0/1169893906 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x559b85f20510 0x559b85f22920 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:32.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.510+0000 7fa24118a740 1 --2- 172.21.15.83:0/1169893906 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b85f10340 0x559b85f0d700 unknown :-1 s=CLOSED pgs=2253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:32.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.510+0000 7fa24118a740 1 --2- 172.21.15.83:0/1169893906 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559b85f05c70 0x559b85f0d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:32.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.510+0000 7fa24118a740 1 --2- 172.21.15.83:0/1169893906 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559b85f05350 0x559b85f0fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:32.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.510+0000 7fa24118a740 1 -- 172.21.15.83:0/1169893906 >> 172.21.15.83:0/1169893906 conn(0x559b85f01670 msgr2=0x559b85f07470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:32.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.510+0000 7fa220c00640 0 client.21192 ms_handle_reset on v2:172.21.15.138:6834/1909219758 2024-04-24T09:07:32.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.510+0000 7fa24118a740 1 -- 172.21.15.83:0/1169893906 shutdown_connections 2024-04-24T09:07:32.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.510+0000 7fa24118a740 1 -- 172.21.15.83:0/1169893906 wait complete. 2024-04-24T09:07:32.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.510+0000 7fa24118a740 20 client.21192 trim_cache size 0 max 16384 2024-04-24T09:07:32.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.533+0000 7fa24118a740 1 Processor -- start 2024-04-24T09:07:32.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.533+0000 7fa24118a740 1 -- start start 2024-04-24T09:07:32.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.533+0000 7fa24118a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b85f05350 0x559b85f1c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:32.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.533+0000 7fa24118a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559b85f05c70 0x559b85f0c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:32.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.533+0000 7fa24118a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559b85f10340 0x559b85f28230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:32.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.533+0000 7fa24118a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa2381ad7c0 con 0x559b85f05350 2024-04-24T09:07:32.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.533+0000 7fa24118a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa2381af450 con 0x559b85f05c70 2024-04-24T09:07:32.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.533+0000 7fa24118a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa2381ade90 con 0x559b85f10340 2024-04-24T09:07:32.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.534+0000 7fa237400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b85f05350 0x559b85f1c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:32.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.534+0000 7fa237400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b85f05350 0x559b85f1c030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39872/0 (socket says 172.21.15.83:39872) 2024-04-24T09:07:32.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.534+0000 7fa237400640 1 -- 172.21.15.83:0/1047754396 learned_addr learned my addr 172.21.15.83:0/1047754396 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:07:32.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.534+0000 7fa236a00640 1 --2- 172.21.15.83:0/1047754396 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559b85f05c70 0x559b85f0c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:32.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.534+0000 7fa237e00640 1 --2- 172.21.15.83:0/1047754396 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559b85f10340 0x559b85f28230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:32.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.534+0000 7fa237400640 1 -- 172.21.15.83:0/1047754396 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559b85f10340 msgr2=0x559b85f28230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:32.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.534+0000 7fa237400640 1 --2- 172.21.15.83:0/1047754396 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559b85f10340 0x559b85f28230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:32.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.534+0000 7fa237400640 1 -- 172.21.15.83:0/1047754396 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559b85f05c70 msgr2=0x559b85f0c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:32.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.534+0000 7fa237400640 1 --2- 172.21.15.83:0/1047754396 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559b85f05c70 0x559b85f0c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:32.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.534+0000 7fa237400640 1 -- 172.21.15.83:0/1047754396 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b85c3e7d0 con 0x559b85f05350 2024-04-24T09:07:32.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.535+0000 7fa237400640 1 --2- 172.21.15.83:0/1047754396 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b85f05350 0x559b85f1c030 secure :-1 s=READY pgs=2254 cs=0 l=1 rev1=1 crypto rx=0x7fa22809bf30 tx=0x7fa2280d2040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b033dec5fa00792a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:32.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.535+0000 7fa223400640 1 -- 172.21.15.83:0/1047754396 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa228020040 con 0x559b85f05350 2024-04-24T09:07:32.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.535+0000 7fa223400640 1 -- 172.21.15.83:0/1047754396 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa228009c30 con 0x559b85f05350 2024-04-24T09:07:32.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.535+0000 7fa223400640 1 -- 172.21.15.83:0/1047754396 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2280a4ea0 con 0x559b85f05350 2024-04-24T09:07:32.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.536+0000 7fa24118a740 1 -- 172.21.15.83:0/1047754396 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b85f05350 msgr2=0x559b85f1c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:32.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.536+0000 7fa24118a740 1 --2- 172.21.15.83:0/1047754396 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b85f05350 0x559b85f1c030 secure :-1 s=READY pgs=2254 cs=0 l=1 rev1=1 crypto rx=0x7fa22809bf30 tx=0x7fa2280d2040 comp rx=0 tx=0).stop 2024-04-24T09:07:32.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.537+0000 7fa24118a740 1 -- 172.21.15.83:0/1047754396 shutdown_connections 2024-04-24T09:07:32.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.537+0000 7fa24118a740 1 --2- 172.21.15.83:0/1047754396 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559b85f10340 0x559b85f28230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:32.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.537+0000 7fa24118a740 1 --2- 172.21.15.83:0/1047754396 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559b85f05c70 0x559b85f0c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:32.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.537+0000 7fa24118a740 1 --2- 172.21.15.83:0/1047754396 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b85f05350 0x559b85f1c030 unknown :-1 s=CLOSED pgs=2254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:32.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.537+0000 7fa24118a740 1 -- 172.21.15.83:0/1047754396 >> 172.21.15.83:0/1047754396 conn(0x559b85f01670 msgr2=0x559b85f082e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:32.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.537+0000 7fa24118a740 1 -- 172.21.15.83:0/1047754396 shutdown_connections 2024-04-24T09:07:32.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.537+0000 7fa24118a740 1 -- 172.21.15.83:0/1047754396 wait complete. 2024-04-24T09:07:32.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.538+0000 7fa24118a740 1 -- start start 2024-04-24T09:07:32.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.538+0000 7fa24118a740 1 Processor -- start 2024-04-24T09:07:32.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.539+0000 7fa24118a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559b85f05350 0x559b85f10720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:32.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.539+0000 7fa24118a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559b85f05c70 0x559b85f10c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:32.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.539+0000 7fa24118a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b85f10340 0x559b85f111a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:32.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.539+0000 7fa24118a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa2381aecc0 con 0x559b85f10340 2024-04-24T09:07:32.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.539+0000 7fa24118a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa2381ae5c0 con 0x559b85f05350 2024-04-24T09:07:32.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.539+0000 7fa237400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559b85f05350 0x559b85f10720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:32.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.539+0000 7fa237400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559b85f05350 0x559b85f10720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:42992/0 (socket says 172.21.15.83:42992) 2024-04-24T09:07:32.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.539+0000 7fa237400640 1 -- 172.21.15.83:0/858598542 learned_addr learned my addr 172.21.15.83:0/858598542 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:07:32.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.540+0000 7fa24118a740 1 -- 172.21.15.83:0/858598542 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa2381ab7e0 con 0x559b85f05c70 2024-04-24T09:07:32.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.540+0000 7fa24118a740 10 client.0 _ll_register_callbacks cb 0x7fa230009d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:07:32.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.540+0000 7fa237e00640 1 --2- 172.21.15.83:0/858598542 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b85f10340 0x559b85f111a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:32.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.540+0000 7fa236a00640 1 --2- 172.21.15.83:0/858598542 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559b85f05c70 0x559b85f10c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:32.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.540+0000 7fa237400640 1 -- 172.21.15.83:0/858598542 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559b85f05c70 msgr2=0x559b85f10c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:32.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.540+0000 7fa237400640 1 --2- 172.21.15.83:0/858598542 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559b85f05c70 0x559b85f10c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:32.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.540+0000 7fa237400640 1 -- 172.21.15.83:0/858598542 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b85f10340 msgr2=0x559b85f111a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:32.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.540+0000 7fa237400640 1 --2- 172.21.15.83:0/858598542 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b85f10340 0x559b85f111a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:32.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.540+0000 7fa237400640 1 -- 172.21.15.83:0/858598542 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b85f17740 con 0x559b85f05350 2024-04-24T09:07:32.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.541+0000 7fa237400640 1 --2- 172.21.15.83:0/858598542 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559b85f05350 0x559b85f10720 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa228012040 tx=0x7fa228005180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=449c4aa84f62d989 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:32.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.541+0000 7fa220c00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:07:32.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.541+0000 7fa220c00640 1 -- 172.21.15.83:0/858598542 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2280b5020 con 0x559b85f05350 2024-04-24T09:07:32.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.541+0000 7fa24118a740 1 -- 172.21.15.83:0/858598542 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559b85e69a60 con 0x559b85f05350 2024-04-24T09:07:32.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.541+0000 7fa220c00640 1 -- 172.21.15.83:0/858598542 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9583+0+0 (secure 0 0 0) 0x7fa2280cdc60 con 0x559b85f05350 2024-04-24T09:07:32.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.542+0000 7fa220c00640 1 -- 172.21.15.83:0/858598542 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7fa228020070 con 0x559b85f05350 2024-04-24T09:07:32.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.542+0000 7fa24118a740 10 client.38351 fetch_fsmap learned FSMap version 51 2024-04-24T09:07:32.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.542+0000 7fa24118a740 1 -- 172.21.15.83:0/858598542 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x559b85f18a70 con 0x559b85f05350 2024-04-24T09:07:32.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.542+0000 7fa220c00640 1 -- 172.21.15.83:0/858598542 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x559b85e69a60 con 0x559b85f05350 2024-04-24T09:07:32.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.542+0000 7fa220c00640 1 -- 172.21.15.83:0/858598542 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 51) ==== 8406+0+0 (secure 0 0 0) 0x7fa2280d0210 con 0x559b85f05350 2024-04-24T09:07:32.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.543+0000 7fa24118a740 10 client.38351 fetch_fsmap finished waiting for FSMap version 51 2024-04-24T09:07:32.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.543+0000 7fa24118a740 10 client.38351 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-24T09:07:32.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.543+0000 7fa24118a740 20 client.38351 populate_metadata read hostname 'smithi083' 2024-04-24T09:07:32.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.543+0000 7fa24118a740 1 --2- 172.21.15.83:0/858598542 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x559b85f86c80 0x559b85f890a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:32.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.543+0000 7fa24118a740 4 client.38351 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:07:32.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.543+0000 7fa24118a740 1 -- 172.21.15.83:0/858598542 --> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559b85de54d0 con 0x559b85f86c80 2024-04-24T09:07:32.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.543+0000 7fa236a00640 1 --2- 172.21.15.83:0/858598542 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x559b85f86c80 0x559b85f890a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:32.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.544+0000 7fa236a00640 1 --2- 172.21.15.83:0/858598542 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x559b85f86c80 0x559b85f890a0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=64acbc6d7e5c2faa server_cookie=72c4a7d14eba7507 in_seq=0 out_seq=0 2024-04-24T09:07:32.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.544+0000 7fa220c00640 10 client.38351 ms_handle_connect on v2:172.21.15.138:6834/1909219758 2024-04-24T09:07:32.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.546+0000 7fa220c00640 1 -- 172.21.15.83:0/858598542 <== mds.2 v2:172.21.15.138:6834/1909219758 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559b85de54d0 con 0x559b85f86c80 2024-04-24T09:07:32.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.546+0000 7fa220c00640 10 client.38351 handle_command_reply: tid=0 2024-04-24T09:07:32.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.546+0000 7fa24118a740 1 client.38351 shutdown 2024-04-24T09:07:32.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.547+0000 7fa24118a740 1 -- 172.21.15.83:0/858598542 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559b85f05350 msgr2=0x559b85f10720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:32.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.547+0000 7fa24118a740 1 --2- 172.21.15.83:0/858598542 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559b85f05350 0x559b85f10720 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa228012040 tx=0x7fa228005180 comp rx=0 tx=0).stop 2024-04-24T09:07:32.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.547+0000 7fa24118a740 1 -- 172.21.15.83:0/858598542 shutdown_connections 2024-04-24T09:07:32.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.547+0000 7fa24118a740 1 --2- 172.21.15.83:0/858598542 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x559b85f86c80 0x559b85f890a0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:32.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.547+0000 7fa24118a740 1 --2- 172.21.15.83:0/858598542 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b85f10340 0x559b85f111a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:32.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.547+0000 7fa24118a740 1 --2- 172.21.15.83:0/858598542 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559b85f05c70 0x559b85f10c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:32.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.547+0000 7fa24118a740 1 --2- 172.21.15.83:0/858598542 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559b85f05350 0x559b85f10720 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:32.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.547+0000 7fa24118a740 1 -- 172.21.15.83:0/858598542 >> 172.21.15.83:0/858598542 conn(0x559b85f01670 msgr2=0x559b85f0e8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:32.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.547+0000 7fa220c00640 0 client.38351 ms_handle_reset on v2:172.21.15.138:6834/1909219758 2024-04-24T09:07:32.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.547+0000 7fa24118a740 1 -- 172.21.15.83:0/858598542 shutdown_connections 2024-04-24T09:07:32.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.547+0000 7fa24118a740 1 -- 172.21.15.83:0/858598542 wait complete. 2024-04-24T09:07:32.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.547+0000 7fa24118a740 20 client.38351 trim_cache size 0 max 16384 2024-04-24T09:07:32.737 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:07:32.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.548+0000 7fa23cc00640 1 -- 172.21.15.83:0/2821923815 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa21c0a3da0 msgr2=0x7fa21c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:32.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.548+0000 7fa23cc00640 1 --2- 172.21.15.83:0/2821923815 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa21c0a3da0 0x7fa21c0a6260 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fa230005fd0 tx=0x7fa230009500 comp rx=0 tx=0).stop 2024-04-24T09:07:32.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.548+0000 7fa23cc00640 1 -- 172.21.15.83:0/2821923815 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa238104320 msgr2=0x7fa2381a0770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:32.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.548+0000 7fa23cc00640 1 --2- 172.21.15.83:0/2821923815 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa238104320 0x7fa2381a0770 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa22800d770 tx=0x7fa22800dc30 comp rx=0 tx=0).stop 2024-04-24T09:07:32.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.548+0000 7fa23cc00640 1 -- 172.21.15.83:0/2821923815 shutdown_connections 2024-04-24T09:07:32.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.548+0000 7fa23cc00640 1 --2- 172.21.15.83:0/2821923815 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa21c0a3da0 0x7fa21c0a6260 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:32.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.548+0000 7fa23cc00640 1 --2- 172.21.15.83:0/2821923815 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa23819a970 0x7fa23819ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:32.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.548+0000 7fa23cc00640 1 --2- 172.21.15.83:0/2821923815 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2381093f0 0x7fa2381a0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:32.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.548+0000 7fa23cc00640 1 --2- 172.21.15.83:0/2821923815 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa238104320 0x7fa2381a0770 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:32.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.548+0000 7fa23cc00640 1 -- 172.21.15.83:0/2821923815 >> 172.21.15.83:0/2821923815 conn(0x7fa2380ffb40 msgr2=0x7fa2381005e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:32.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.548+0000 7fa23cc00640 1 -- 172.21.15.83:0/2821923815 shutdown_connections 2024-04-24T09:07:32.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.548+0000 7fa23cc00640 1 -- 172.21.15.83:0/2821923815 wait complete. 2024-04-24T09:07:32.739 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T09:07:32.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.987+0000 7f9221000640 1 -- 172.21.15.83:0/2524876169 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f921c103060 msgr2=0x7f921c103440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:32.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.987+0000 7f9221000640 1 --2- 172.21.15.83:0/2524876169 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f921c103060 0x7f921c103440 secure :-1 s=READY pgs=2255 cs=0 l=1 rev1=1 crypto rx=0x7f920c009900 tx=0x7f920c02a480 comp rx=0 tx=0).stop 2024-04-24T09:07:32.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.988+0000 7f9221000640 1 -- 172.21.15.83:0/2524876169 shutdown_connections 2024-04-24T09:07:32.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.988+0000 7f9221000640 1 --2- 172.21.15.83:0/2524876169 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f921c10c480 0x7f921c10e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:32.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.988+0000 7f9221000640 1 --2- 172.21.15.83:0/2524876169 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f921c103980 0x7f921c10be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:32.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.988+0000 7f9221000640 1 --2- 172.21.15.83:0/2524876169 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f921c103060 0x7f921c103440 unknown :-1 s=CLOSED pgs=2255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:32.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.988+0000 7f9221000640 1 -- 172.21.15.83:0/2524876169 >> 172.21.15.83:0/2524876169 conn(0x7f921c0fc880 msgr2=0x7f921c0feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:32.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.988+0000 7f9221000640 1 -- 172.21.15.83:0/2524876169 shutdown_connections 2024-04-24T09:07:32.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.988+0000 7f9221000640 1 -- 172.21.15.83:0/2524876169 wait complete. 2024-04-24T09:07:32.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.989+0000 7f9221000640 1 Processor -- start 2024-04-24T09:07:32.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.989+0000 7f9221000640 1 -- start start 2024-04-24T09:07:32.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.990+0000 7f9221000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f921c103060 0x7f921c100cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:32.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.990+0000 7f9221000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f921c103980 0x7f921c1011f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:32.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.990+0000 7f9221000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f921c10c480 0x7f921c101730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:32.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.990+0000 7f9221000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f921c112140 con 0x7f921c103980 2024-04-24T09:07:32.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.990+0000 7f9221000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f921c111fc0 con 0x7f921c103060 2024-04-24T09:07:32.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.990+0000 7f9221000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f921c1122c0 con 0x7f921c10c480 2024-04-24T09:07:32.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.990+0000 7f921aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f921c103980 0x7f921c1011f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:32.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.990+0000 7f921b400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f921c103060 0x7f921c100cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:32.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.990+0000 7f921aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f921c103980 0x7f921c1011f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39900/0 (socket says 172.21.15.83:39900) 2024-04-24T09:07:32.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.990+0000 7f921aa00640 1 -- 172.21.15.83:0/3963667123 learned_addr learned my addr 172.21.15.83:0/3963667123 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:07:32.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.990+0000 7f921be00640 1 --2- 172.21.15.83:0/3963667123 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f921c10c480 0x7f921c101730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:32.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.990+0000 7f921aa00640 1 -- 172.21.15.83:0/3963667123 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f921c10c480 msgr2=0x7f921c101730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:32.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.990+0000 7f921aa00640 1 --2- 172.21.15.83:0/3963667123 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f921c10c480 0x7f921c101730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:32.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.991+0000 7f921aa00640 1 -- 172.21.15.83:0/3963667123 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f921c103060 msgr2=0x7f921c100cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:32.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.991+0000 7f921aa00640 1 --2- 172.21.15.83:0/3963667123 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f921c103060 0x7f921c100cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:32.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.991+0000 7f921aa00640 1 -- 172.21.15.83:0/3963667123 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f921c1a6b20 con 0x7f921c103980 2024-04-24T09:07:32.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.991+0000 7f921be00640 1 --2- 172.21.15.83:0/3963667123 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f921c10c480 0x7f921c101730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:07:32.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.991+0000 7f921aa00640 1 --2- 172.21.15.83:0/3963667123 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f921c103980 0x7f921c1011f0 secure :-1 s=READY pgs=2256 cs=0 l=1 rev1=1 crypto rx=0x7f920800e9f0 tx=0x7f920800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:32.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.991+0000 7f9213e00640 1 -- 172.21.15.83:0/3963667123 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f920800cd80 con 0x7f921c103980 2024-04-24T09:07:32.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.991+0000 7f9213e00640 1 -- 172.21.15.83:0/3963667123 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f92080035f0 con 0x7f921c103980 2024-04-24T09:07:32.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.991+0000 7f9213e00640 1 -- 172.21.15.83:0/3963667123 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9208010690 con 0x7f921c103980 2024-04-24T09:07:32.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.991+0000 7f9221000640 1 -- 172.21.15.83:0/3963667123 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f921c1a6e10 con 0x7f921c103980 2024-04-24T09:07:32.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.991+0000 7f9221000640 1 -- 172.21.15.83:0/3963667123 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f921c1a7320 con 0x7f921c103980 2024-04-24T09:07:33.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.996+0000 7f922543b740 1 Processor -- start 2024-04-24T09:07:33.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.996+0000 7f922543b740 1 -- start start 2024-04-24T09:07:33.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.997+0000 7f922543b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5579fb7aeb40 0x5579fb7aef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:33.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.997+0000 7f922543b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5579fb7af460 0x5579fb7b95f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:33.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.997+0000 7f922543b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5579fb7b9b30 0x5579fb7bbf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:33.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.997+0000 7f922543b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5579fb7141d0 con 0x5579fb7b9b30 2024-04-24T09:07:33.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.997+0000 7f922543b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5579fb6ba450 con 0x5579fb7af460 2024-04-24T09:07:33.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.997+0000 7f922543b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5579fb710190 con 0x5579fb7aeb40 2024-04-24T09:07:33.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.997+0000 7f921b400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5579fb7af460 0x5579fb7b95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:33.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.997+0000 7f921b400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5579fb7af460 0x5579fb7b95f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:43012/0 (socket says 172.21.15.83:43012) 2024-04-24T09:07:33.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.997+0000 7f921b400640 1 -- 172.21.15.83:0/154740436 learned_addr learned my addr 172.21.15.83:0/154740436 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:07:33.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.997+0000 7f9213e00640 1 -- 172.21.15.83:0/3963667123 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f92080037a0 con 0x7f921c103980 2024-04-24T09:07:33.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.997+0000 7f921be00640 1 --2- 172.21.15.83:0/154740436 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5579fb7aeb40 0x5579fb7aef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:33.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.998+0000 7f921b400640 1 -- 172.21.15.83:0/154740436 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5579fb7aeb40 msgr2=0x5579fb7aef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:33.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.998+0000 7f921b400640 1 --2- 172.21.15.83:0/154740436 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5579fb7aeb40 0x5579fb7aef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:33.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.998+0000 7f921aa00640 1 --2- 172.21.15.83:0/154740436 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5579fb7b9b30 0x5579fb7bbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:33.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.998+0000 7f921b400640 1 -- 172.21.15.83:0/154740436 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5579fb7b9b30 msgr2=0x5579fb7bbf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:33.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.998+0000 7f921b400640 1 --2- 172.21.15.83:0/154740436 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5579fb7b9b30 0x5579fb7bbf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:33.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.998+0000 7f921b400640 1 -- 172.21.15.83:0/154740436 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579fb777310 con 0x5579fb7af460 2024-04-24T09:07:33.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.998+0000 7f921aa00640 1 --2- 172.21.15.83:0/154740436 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5579fb7b9b30 0x5579fb7bbf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:07:33.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.998+0000 7f921b400640 1 --2- 172.21.15.83:0/154740436 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5579fb7af460 0x5579fb7b95f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f920c009230 tx=0x7f920c004180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5ff09765d93cbc0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:33.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.999+0000 7f9213e00640 1 --2- 172.21.15.83:0/3963667123 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f92000a3dd0 0x7f92000a6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:33.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.999+0000 7f9213e00640 1 -- 172.21.15.83:0/3963667123 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 9583+0+0 (secure 0 0 0) 0x7f92080d0f10 con 0x7f921c103980 2024-04-24T09:07:33.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:32.999+0000 7f921be00640 1 --2- 172.21.15.83:0/3963667123 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f92000a3dd0 0x7f92000a6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:33.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.000+0000 7f9210c00640 1 -- 172.21.15.83:0/154740436 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f920c035040 con 0x5579fb7af460 2024-04-24T09:07:33.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.000+0000 7f9210c00640 1 -- 172.21.15.83:0/154740436 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f920c0054f0 con 0x5579fb7af460 2024-04-24T09:07:33.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.000+0000 7f9210c00640 1 -- 172.21.15.83:0/154740436 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f920c003850 con 0x5579fb7af460 2024-04-24T09:07:33.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.001+0000 7f922543b740 1 -- 172.21.15.83:0/154740436 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5579fb7af460 msgr2=0x5579fb7b95f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:33.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.001+0000 7f922543b740 1 --2- 172.21.15.83:0/154740436 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5579fb7af460 0x5579fb7b95f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f920c009230 tx=0x7f920c004180 comp rx=0 tx=0).stop 2024-04-24T09:07:33.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.001+0000 7f921be00640 1 --2- 172.21.15.83:0/3963667123 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f92000a3dd0 0x7f92000a6290 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f9214005cc0 tx=0x7f92140083f0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:33.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.001+0000 7f922543b740 1 -- 172.21.15.83:0/154740436 shutdown_connections 2024-04-24T09:07:33.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.001+0000 7f922543b740 1 --2- 172.21.15.83:0/154740436 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5579fb7b9b30 0x5579fb7bbf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:33.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.001+0000 7f922543b740 1 --2- 172.21.15.83:0/154740436 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5579fb7af460 0x5579fb7b95f0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:33.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.001+0000 7f922543b740 1 --2- 172.21.15.83:0/154740436 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5579fb7aeb40 0x5579fb7aef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:33.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.001+0000 7f922543b740 1 -- 172.21.15.83:0/154740436 >> 172.21.15.83:0/154740436 conn(0x5579fb7aae60 msgr2=0x5579fb7ab240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:33.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.001+0000 7f922543b740 1 -- 172.21.15.83:0/154740436 shutdown_connections 2024-04-24T09:07:33.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.001+0000 7f922543b740 1 -- 172.21.15.83:0/154740436 wait complete. 2024-04-24T09:07:33.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.002+0000 7f922543b740 1 -- start start 2024-04-24T09:07:33.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.002+0000 7f922543b740 1 Processor -- start 2024-04-24T09:07:33.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.003+0000 7f922543b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5579fb7aeb40 0x5579fb7b9570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:33.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.003+0000 7f922543b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5579fb7af460 0x5579fb7b6860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:33.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.003+0000 7f922543b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5579fb7b9b30 0x5579fb7b6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:33.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.003+0000 7f922543b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5579fb7bc9b0 con 0x5579fb7b9b30 2024-04-24T09:07:33.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.003+0000 7f922543b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5579fb6cd810 con 0x5579fb7af460 2024-04-24T09:07:33.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.003+0000 7f922543b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5579fb7bcb30 con 0x5579fb7aeb40 2024-04-24T09:07:33.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.003+0000 7f922543b740 10 client.0 _ll_register_callbacks cb 0x5579fb69b140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:07:33.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.003+0000 7f921aa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5579fb7af460 0x5579fb7b6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:33.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.003+0000 7f921aa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5579fb7af460 0x5579fb7b6860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:43028/0 (socket says 172.21.15.83:43028) 2024-04-24T09:07:33.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.003+0000 7f921aa00640 1 -- 172.21.15.83:0/1050483032 learned_addr learned my addr 172.21.15.83:0/1050483032 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:07:33.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.003+0000 7f921be00640 1 --2- 172.21.15.83:0/1050483032 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5579fb7b9b30 0x5579fb7b6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:33.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.004+0000 7f921aa00640 1 -- 172.21.15.83:0/1050483032 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5579fb7aeb40 msgr2=0x5579fb7b9570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:33.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.004+0000 7f921aa00640 1 --2- 172.21.15.83:0/1050483032 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5579fb7aeb40 0x5579fb7b9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:33.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.004+0000 7f921aa00640 1 -- 172.21.15.83:0/1050483032 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5579fb7b9b30 msgr2=0x5579fb7b6fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:33.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.004+0000 7f921aa00640 1 --2- 172.21.15.83:0/1050483032 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5579fb7b9b30 0x5579fb7b6fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:33.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.004+0000 7f921aa00640 1 -- 172.21.15.83:0/1050483032 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579fb6de3f0 con 0x5579fb7af460 2024-04-24T09:07:33.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.004+0000 7f921be00640 1 --2- 172.21.15.83:0/1050483032 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5579fb7b9b30 0x5579fb7b6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:07:33.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.004+0000 7f921aa00640 1 --2- 172.21.15.83:0/1050483032 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5579fb7af460 0x5579fb7b6860 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f920800c6a0 tx=0x7f920809c390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc9688cb0b863a50 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:33.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.004+0000 7f9205600640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:07:33.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.004+0000 7f9205600640 1 -- 172.21.15.83:0/1050483032 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92080ce660 con 0x5579fb7af460 2024-04-24T09:07:33.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.004+0000 7f9205600640 1 -- 172.21.15.83:0/1050483032 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9583+0+0 (secure 0 0 0) 0x7f92080d4020 con 0x5579fb7af460 2024-04-24T09:07:33.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.004+0000 7f922543b740 1 -- 172.21.15.83:0/1050483032 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f920c0054f0 con 0x5579fb7af460 2024-04-24T09:07:33.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.005+0000 7f9205600640 1 -- 172.21.15.83:0/1050483032 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f920809d130 con 0x5579fb7af460 2024-04-24T09:07:33.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.005+0000 7f922543b740 10 client.38363 fetch_fsmap learned FSMap version 51 2024-04-24T09:07:33.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.005+0000 7f922543b740 1 -- 172.21.15.83:0/1050483032 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x5579fb4263f0 con 0x5579fb7af460 2024-04-24T09:07:33.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.005+0000 7f9205600640 1 -- 172.21.15.83:0/1050483032 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f920c0054f0 con 0x5579fb7af460 2024-04-24T09:07:33.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.005+0000 7f9205600640 1 -- 172.21.15.83:0/1050483032 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 51) ==== 8406+0+0 (secure 0 0 0) 0x7f92080870b0 con 0x5579fb7af460 2024-04-24T09:07:33.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.006+0000 7f922543b740 10 client.38363 fetch_fsmap finished waiting for FSMap version 51 2024-04-24T09:07:33.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.006+0000 7f922543b740 10 client.38363 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-24T09:07:33.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.006+0000 7f922543b740 20 client.38363 populate_metadata read hostname 'smithi083' 2024-04-24T09:07:33.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.006+0000 7f922543b740 1 --2- 172.21.15.83:0/1050483032 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x5579fb7d2530 0x5579fb7d49c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:33.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.006+0000 7f922543b740 4 client.38363 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:07:33.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.006+0000 7f922543b740 1 -- 172.21.15.83:0/1050483032 --> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5579fb721bf0 con 0x5579fb7d2530 2024-04-24T09:07:33.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.006+0000 7f921b400640 1 --2- 172.21.15.83:0/1050483032 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x5579fb7d2530 0x5579fb7d49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:33.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.007+0000 7f921b400640 1 --2- 172.21.15.83:0/1050483032 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x5579fb7d2530 0x5579fb7d49c0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8bf034aa5e64d3df server_cookie=66bb12ac6ca35cfc in_seq=0 out_seq=0 2024-04-24T09:07:33.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.007+0000 7f9205600640 10 client.38363 ms_handle_connect on v2:172.21.15.138:6836/900566754 2024-04-24T09:07:33.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.008+0000 7f9205600640 1 -- 172.21.15.83:0/1050483032 <== mds.1 v2:172.21.15.138:6836/900566754 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5579fb721bf0 con 0x5579fb7d2530 2024-04-24T09:07:33.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.008+0000 7f9205600640 10 client.38363 handle_command_reply: tid=0 2024-04-24T09:07:33.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.008+0000 7f922543b740 1 client.38363 shutdown 2024-04-24T09:07:33.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.008+0000 7f922543b740 1 -- 172.21.15.83:0/1050483032 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5579fb7af460 msgr2=0x5579fb7b6860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:33.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.008+0000 7f922543b740 1 --2- 172.21.15.83:0/1050483032 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5579fb7af460 0x5579fb7b6860 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f920800c6a0 tx=0x7f920809c390 comp rx=0 tx=0).stop 2024-04-24T09:07:33.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.008+0000 7f922543b740 1 -- 172.21.15.83:0/1050483032 shutdown_connections 2024-04-24T09:07:33.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.008+0000 7f922543b740 1 --2- 172.21.15.83:0/1050483032 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5579fb7b9b30 0x5579fb7b6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:33.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.008+0000 7f922543b740 1 --2- 172.21.15.83:0/1050483032 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5579fb7af460 0x5579fb7b6860 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:33.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.008+0000 7f922543b740 1 --2- 172.21.15.83:0/1050483032 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x5579fb7d2530 0x5579fb7d49c0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:33.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.008+0000 7f922543b740 1 --2- 172.21.15.83:0/1050483032 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5579fb7aeb40 0x5579fb7b9570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:33.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.008+0000 7f9205600640 0 client.38363 ms_handle_reset on v2:172.21.15.138:6836/900566754 2024-04-24T09:07:33.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.008+0000 7f922543b740 1 -- 172.21.15.83:0/1050483032 >> 172.21.15.83:0/1050483032 conn(0x5579fb7aae60 msgr2=0x5579fb7b0c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:33.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.008+0000 7f922543b740 1 -- 172.21.15.83:0/1050483032 shutdown_connections 2024-04-24T09:07:33.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.008+0000 7f922543b740 1 -- 172.21.15.83:0/1050483032 wait complete. 2024-04-24T09:07:33.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.009+0000 7f922543b740 20 client.38363 trim_cache size 0 max 16384 2024-04-24T09:07:33.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.031+0000 7f922543b740 1 Processor -- start 2024-04-24T09:07:33.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.031+0000 7f922543b740 1 -- start start 2024-04-24T09:07:33.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.031+0000 7f922543b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5579fb7aeb40 0x5579fb7b12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:33.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.031+0000 7f922543b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5579fb7b9b30 0x5579fb7b90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:33.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.032+0000 7f922543b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5579fb7b95e0 0x5579fb7da2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:33.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.032+0000 7f922543b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f921c1ad1f0 con 0x5579fb7b9b30 2024-04-24T09:07:33.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.032+0000 7f922543b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f921c1aee80 con 0x5579fb7b95e0 2024-04-24T09:07:33.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.032+0000 7f922543b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f921c1ad8c0 con 0x5579fb7aeb40 2024-04-24T09:07:33.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.032+0000 7f921aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5579fb7b9b30 0x5579fb7b90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:33.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.032+0000 7f921aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5579fb7b9b30 0x5579fb7b90a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39922/0 (socket says 172.21.15.83:39922) 2024-04-24T09:07:33.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.032+0000 7f921aa00640 1 -- 172.21.15.83:0/3265668865 learned_addr learned my addr 172.21.15.83:0/3265668865 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:07:33.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.032+0000 7f921be00640 1 --2- 172.21.15.83:0/3265668865 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5579fb7b95e0 0x5579fb7da2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:33.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.032+0000 7f921b400640 1 --2- 172.21.15.83:0/3265668865 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5579fb7aeb40 0x5579fb7b12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:33.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.032+0000 7f921aa00640 1 -- 172.21.15.83:0/3265668865 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5579fb7aeb40 msgr2=0x5579fb7b12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:33.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.032+0000 7f921aa00640 1 --2- 172.21.15.83:0/3265668865 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5579fb7aeb40 0x5579fb7b12e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:33.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.032+0000 7f921aa00640 1 -- 172.21.15.83:0/3265668865 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5579fb7b95e0 msgr2=0x5579fb7da2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:33.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.032+0000 7f921aa00640 1 --2- 172.21.15.83:0/3265668865 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5579fb7b95e0 0x5579fb7da2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:33.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.032+0000 7f921aa00640 1 -- 172.21.15.83:0/3265668865 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579fb4f07d0 con 0x5579fb7b9b30 2024-04-24T09:07:33.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.033+0000 7f921aa00640 1 --2- 172.21.15.83:0/3265668865 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5579fb7b9b30 0x5579fb7b90a0 secure :-1 s=READY pgs=2257 cs=0 l=1 rev1=1 crypto rx=0x7f92080a5170 tx=0x7f92080bd050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8402f04baa042b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:33.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.033+0000 7f9207e00640 1 -- 172.21.15.83:0/3265668865 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f920809ca30 con 0x5579fb7b9b30 2024-04-24T09:07:33.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.033+0000 7f9207e00640 1 -- 172.21.15.83:0/3265668865 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f92080a0de0 con 0x5579fb7b9b30 2024-04-24T09:07:33.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.033+0000 7f9207e00640 1 -- 172.21.15.83:0/3265668865 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9208098b00 con 0x5579fb7b9b30 2024-04-24T09:07:33.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.034+0000 7f922543b740 1 -- 172.21.15.83:0/3265668865 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5579fb7b9b30 msgr2=0x5579fb7b90a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:33.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.034+0000 7f922543b740 1 --2- 172.21.15.83:0/3265668865 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5579fb7b9b30 0x5579fb7b90a0 secure :-1 s=READY pgs=2257 cs=0 l=1 rev1=1 crypto rx=0x7f92080a5170 tx=0x7f92080bd050 comp rx=0 tx=0).stop 2024-04-24T09:07:33.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.034+0000 7f922543b740 1 -- 172.21.15.83:0/3265668865 shutdown_connections 2024-04-24T09:07:33.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.034+0000 7f922543b740 1 --2- 172.21.15.83:0/3265668865 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5579fb7b95e0 0x5579fb7da2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:33.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.034+0000 7f922543b740 1 --2- 172.21.15.83:0/3265668865 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5579fb7b9b30 0x5579fb7b90a0 unknown :-1 s=CLOSED pgs=2257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:33.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.034+0000 7f922543b740 1 --2- 172.21.15.83:0/3265668865 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5579fb7aeb40 0x5579fb7b12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:33.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.034+0000 7f922543b740 1 -- 172.21.15.83:0/3265668865 >> 172.21.15.83:0/3265668865 conn(0x5579fb7aae60 msgr2=0x5579fb7b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:33.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.035+0000 7f922543b740 1 -- 172.21.15.83:0/3265668865 shutdown_connections 2024-04-24T09:07:33.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.035+0000 7f922543b740 1 -- 172.21.15.83:0/3265668865 wait complete. 2024-04-24T09:07:33.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.035+0000 7f922543b740 1 -- start start 2024-04-24T09:07:33.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.035+0000 7f922543b740 1 Processor -- start 2024-04-24T09:07:33.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.036+0000 7f922543b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5579fb7aeb40 0x5579fb7ca640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:33.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.036+0000 7f922543b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5579fb7b95e0 0x5579fb7cab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:33.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.036+0000 7f922543b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5579fb7b9b30 0x5579fb835b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:33.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.036+0000 7f922543b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f921c1ae6f0 con 0x5579fb7b9b30 2024-04-24T09:07:33.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.036+0000 7f922543b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f921c1adff0 con 0x5579fb7aeb40 2024-04-24T09:07:33.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.036+0000 7f922543b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f921c1ab1f0 con 0x5579fb7b95e0 2024-04-24T09:07:33.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.036+0000 7f922543b740 10 client.0 _ll_register_callbacks cb 0x7f9208003d00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:07:33.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.036+0000 7f921b400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5579fb7aeb40 0x5579fb7ca640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:33.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.036+0000 7f921be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5579fb7b9b30 0x5579fb835b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:33.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.036+0000 7f921b400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5579fb7aeb40 0x5579fb7ca640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:43044/0 (socket says 172.21.15.83:43044) 2024-04-24T09:07:33.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.036+0000 7f921b400640 1 -- 172.21.15.83:0/636487644 learned_addr learned my addr 172.21.15.83:0/636487644 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:07:33.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.037+0000 7f921aa00640 1 --2- 172.21.15.83:0/636487644 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5579fb7b95e0 0x5579fb7cab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:33.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.037+0000 7f921be00640 1 -- 172.21.15.83:0/636487644 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5579fb7b95e0 msgr2=0x5579fb7cab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:33.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.037+0000 7f921be00640 1 --2- 172.21.15.83:0/636487644 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5579fb7b95e0 0x5579fb7cab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:33.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.037+0000 7f921be00640 1 -- 172.21.15.83:0/636487644 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5579fb7aeb40 msgr2=0x5579fb7ca640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:33.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.037+0000 7f921be00640 1 --2- 172.21.15.83:0/636487644 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5579fb7aeb40 0x5579fb7ca640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:33.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.037+0000 7f921be00640 1 -- 172.21.15.83:0/636487644 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579fb8361f0 con 0x5579fb7b9b30 2024-04-24T09:07:33.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.037+0000 7f921be00640 1 --2- 172.21.15.83:0/636487644 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5579fb7b9b30 0x5579fb835b30 secure :-1 s=READY pgs=2258 cs=0 l=1 rev1=1 crypto rx=0x7f921400fc20 tx=0x7f9214004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e280fd1c7ad27a36 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:33.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.037+0000 7f9205600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:07:33.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.038+0000 7f9205600640 1 -- 172.21.15.83:0/636487644 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f921401d630 con 0x5579fb7b9b30 2024-04-24T09:07:33.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.038+0000 7f9205600640 1 -- 172.21.15.83:0/636487644 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9583+0+0 (secure 0 0 0) 0x7f921401d7d0 con 0x5579fb7b9b30 2024-04-24T09:07:33.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.038+0000 7f922543b740 1 -- 172.21.15.83:0/636487644 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92080a0de0 con 0x5579fb7b9b30 2024-04-24T09:07:33.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.038+0000 7f9205600640 1 -- 172.21.15.83:0/636487644 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f921401d9d0 con 0x5579fb7b9b30 2024-04-24T09:07:33.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.038+0000 7f922543b740 10 client.21213 fetch_fsmap learned FSMap version 51 2024-04-24T09:07:33.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.038+0000 7f922543b740 1 -- 172.21.15.83:0/636487644 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x5579fb7b34f0 con 0x5579fb7b9b30 2024-04-24T09:07:33.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.039+0000 7f9205600640 1 -- 172.21.15.83:0/636487644 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f92080a0de0 con 0x5579fb7b9b30 2024-04-24T09:07:33.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.039+0000 7f9205600640 1 -- 172.21.15.83:0/636487644 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 51) ==== 8406+0+0 (secure 0 0 0) 0x7f9214028420 con 0x5579fb7b9b30 2024-04-24T09:07:33.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.039+0000 7f922543b740 10 client.21213 fetch_fsmap finished waiting for FSMap version 51 2024-04-24T09:07:33.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.039+0000 7f922543b740 10 client.21213 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-24T09:07:33.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.039+0000 7f922543b740 20 client.21213 populate_metadata read hostname 'smithi083' 2024-04-24T09:07:33.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.039+0000 7f922543b740 1 --2- 172.21.15.83:0/636487644 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x5579fb7b3cd0 0x5579fb7b40e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:33.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.039+0000 7f922543b740 4 client.21213 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:07:33.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.039+0000 7f922543b740 1 -- 172.21.15.83:0/636487644 --> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5579fb71ba60 con 0x5579fb7b3cd0 2024-04-24T09:07:33.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.039+0000 7f921b400640 1 --2- 172.21.15.83:0/636487644 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x5579fb7b3cd0 0x5579fb7b40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:33.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.040+0000 7f9205600640 10 client.21213 ms_handle_connect on v2:172.21.15.138:6836/900566754 2024-04-24T09:07:33.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.040+0000 7f921b400640 1 --2- 172.21.15.83:0/636487644 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x5579fb7b3cd0 0x5579fb7b40e0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=833d5de456cb2211 server_cookie=eb94dea946fc9ba2 in_seq=0 out_seq=0 2024-04-24T09:07:33.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.041+0000 7f9205600640 1 -- 172.21.15.83:0/636487644 <== mds.1 v2:172.21.15.138:6836/900566754 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5579fb71ba60 con 0x5579fb7b3cd0 2024-04-24T09:07:33.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.041+0000 7f9205600640 10 client.21213 handle_command_reply: tid=0 2024-04-24T09:07:33.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.041+0000 7f922543b740 1 client.21213 shutdown 2024-04-24T09:07:33.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.041+0000 7f922543b740 1 -- 172.21.15.83:0/636487644 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5579fb7b9b30 msgr2=0x5579fb835b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:33.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.041+0000 7f922543b740 1 --2- 172.21.15.83:0/636487644 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5579fb7b9b30 0x5579fb835b30 secure :-1 s=READY pgs=2258 cs=0 l=1 rev1=1 crypto rx=0x7f921400fc20 tx=0x7f9214004a40 comp rx=0 tx=0).stop 2024-04-24T09:07:33.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.041+0000 7f922543b740 1 -- 172.21.15.83:0/636487644 shutdown_connections 2024-04-24T09:07:33.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.041+0000 7f922543b740 1 --2- 172.21.15.83:0/636487644 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5579fb7b9b30 0x5579fb835b30 unknown :-1 s=CLOSED pgs=2258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:33.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.041+0000 7f922543b740 1 --2- 172.21.15.83:0/636487644 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x5579fb7b3cd0 0x5579fb7b40e0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:33.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.041+0000 7f922543b740 1 --2- 172.21.15.83:0/636487644 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5579fb7b95e0 0x5579fb7cab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:33.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.041+0000 7f922543b740 1 --2- 172.21.15.83:0/636487644 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5579fb7aeb40 0x5579fb7ca640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:33.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.041+0000 7f922543b740 1 -- 172.21.15.83:0/636487644 >> 172.21.15.83:0/636487644 conn(0x5579fb7aae60 msgr2=0x5579fb7a96c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:33.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.041+0000 7f9205600640 0 client.21213 ms_handle_reset on v2:172.21.15.138:6836/900566754 2024-04-24T09:07:33.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.042+0000 7f922543b740 1 -- 172.21.15.83:0/636487644 shutdown_connections 2024-04-24T09:07:33.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.042+0000 7f922543b740 1 -- 172.21.15.83:0/636487644 wait complete. 2024-04-24T09:07:33.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.042+0000 7f922543b740 20 client.21213 trim_cache size 0 max 16384 2024-04-24T09:07:33.049 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:07:33.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.043+0000 7f9221000640 1 -- 172.21.15.83:0/3963667123 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f92000a3dd0 msgr2=0x7f92000a6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:33.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.043+0000 7f9221000640 1 --2- 172.21.15.83:0/3963667123 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f92000a3dd0 0x7f92000a6290 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f9214005cc0 tx=0x7f92140083f0 comp rx=0 tx=0).stop 2024-04-24T09:07:33.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.043+0000 7f9221000640 1 -- 172.21.15.83:0/3963667123 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f921c103980 msgr2=0x7f921c1011f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:33.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.043+0000 7f9221000640 1 --2- 172.21.15.83:0/3963667123 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f921c103980 0x7f921c1011f0 secure :-1 s=READY pgs=2256 cs=0 l=1 rev1=1 crypto rx=0x7f920800e9f0 tx=0x7f920800eeb0 comp rx=0 tx=0).stop 2024-04-24T09:07:33.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.043+0000 7f9221000640 1 -- 172.21.15.83:0/3963667123 shutdown_connections 2024-04-24T09:07:33.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.043+0000 7f9221000640 1 --2- 172.21.15.83:0/3963667123 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f92000a3dd0 0x7f92000a6290 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:33.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.043+0000 7f9221000640 1 --2- 172.21.15.83:0/3963667123 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f921c10c480 0x7f921c101730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:33.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.043+0000 7f9221000640 1 --2- 172.21.15.83:0/3963667123 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f921c103980 0x7f921c1011f0 unknown :-1 s=CLOSED pgs=2256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:33.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.043+0000 7f9221000640 1 --2- 172.21.15.83:0/3963667123 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f921c103060 0x7f921c100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:33.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.043+0000 7f9221000640 1 -- 172.21.15.83:0/3963667123 >> 172.21.15.83:0/3963667123 conn(0x7f921c0fc880 msgr2=0x7f921c0fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:33.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.043+0000 7f9221000640 1 -- 172.21.15.83:0/3963667123 shutdown_connections 2024-04-24T09:07:33.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.044+0000 7f9221000640 1 -- 172.21.15.83:0/3963667123 wait complete. 2024-04-24T09:07:33.067 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T09:07:33.164 INFO:tasks.workunit.client.0.smithi083.stdout: 10 16119 10.90 MB/sec execute 4 sec latency 81.645 ms 2024-04-24T09:07:33.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.320+0000 7fb6e7200640 1 -- 172.21.15.83:0/1752286228 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6e81060d0 msgr2=0x7fb6e810dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:33.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.320+0000 7fb6e7200640 1 --2- 172.21.15.83:0/1752286228 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6e81060d0 0x7fb6e810dca0 secure :-1 s=READY pgs=2259 cs=0 l=1 rev1=1 crypto rx=0x7fb6e000bd70 tx=0x7fb6e002c430 comp rx=0 tx=0).stop 2024-04-24T09:07:33.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.320+0000 7fb6e7200640 1 -- 172.21.15.83:0/1752286228 shutdown_connections 2024-04-24T09:07:33.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.320+0000 7fb6e7200640 1 --2- 172.21.15.83:0/1752286228 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6e81060d0 0x7fb6e810dca0 unknown :-1 s=CLOSED pgs=2259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:33.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.320+0000 7fb6e7200640 1 --2- 172.21.15.83:0/1752286228 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb6e81019b0 0x7fb6e81059f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:33.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.320+0000 7fb6e7200640 1 --2- 172.21.15.83:0/1752286228 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6e8101000 0x7fb6e81013e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:33.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.320+0000 7fb6e7200640 1 -- 172.21.15.83:0/1752286228 >> 172.21.15.83:0/1752286228 conn(0x7fb6e80fc880 msgr2=0x7fb6e80feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:33.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.320+0000 7fb6e7200640 1 -- 172.21.15.83:0/1752286228 shutdown_connections 2024-04-24T09:07:33.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.320+0000 7fb6e7200640 1 -- 172.21.15.83:0/1752286228 wait complete. 2024-04-24T09:07:33.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.321+0000 7fb6e7200640 1 Processor -- start 2024-04-24T09:07:33.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.322+0000 7fb6e7200640 1 -- start start 2024-04-24T09:07:33.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.322+0000 7fb6e7200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb6e8101000 0x7fb6e8073800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:33.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.322+0000 7fb6e7200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6e81060d0 0x7fb6e806c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:33.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.322+0000 7fb6e7200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6e8073e50 0x7fb6e806ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:33.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.322+0000 7fb6e7200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e810ff30 con 0x7fb6e81060d0 2024-04-24T09:07:33.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.322+0000 7fb6e7200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e810fdb0 con 0x7fb6e8073e50 2024-04-24T09:07:33.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.322+0000 7fb6e7200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e81100b0 con 0x7fb6e8101000 2024-04-24T09:07:33.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.323+0000 7fb6e5400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6e81060d0 0x7fb6e806c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:33.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.323+0000 7fb6e5400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6e81060d0 0x7fb6e806c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39964/0 (socket says 172.21.15.83:39964) 2024-04-24T09:07:33.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.323+0000 7fb6e6800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6e8073e50 0x7fb6e806ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:33.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.323+0000 7fb6e5400640 1 -- 172.21.15.83:0/756402847 learned_addr learned my addr 172.21.15.83:0/756402847 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:07:33.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.323+0000 7fb6e5e00640 1 --2- 172.21.15.83:0/756402847 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb6e8101000 0x7fb6e8073800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:33.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.323+0000 7fb6e6800640 1 -- 172.21.15.83:0/756402847 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb6e8101000 msgr2=0x7fb6e8073800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:33.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.323+0000 7fb6e6800640 1 --2- 172.21.15.83:0/756402847 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb6e8101000 0x7fb6e8073800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:33.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.323+0000 7fb6e6800640 1 -- 172.21.15.83:0/756402847 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6e81060d0 msgr2=0x7fb6e806c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:33.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.323+0000 7fb6e6800640 1 --2- 172.21.15.83:0/756402847 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6e81060d0 0x7fb6e806c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:33.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.323+0000 7fb6e6800640 1 -- 172.21.15.83:0/756402847 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6e8071320 con 0x7fb6e8073e50 2024-04-24T09:07:33.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.323+0000 7fb6e5400640 1 --2- 172.21.15.83:0/756402847 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6e81060d0 0x7fb6e806c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:07:33.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.324+0000 7fb6e6800640 1 --2- 172.21.15.83:0/756402847 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6e8073e50 0x7fb6e806ce90 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb6e002c950 tx=0x7fb6e0033a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:33.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.324+0000 7fb6dea00640 1 -- 172.21.15.83:0/756402847 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e0042030 con 0x7fb6e8073e50 2024-04-24T09:07:33.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.324+0000 7fb6e7200640 1 -- 172.21.15.83:0/756402847 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6e80715b0 con 0x7fb6e8073e50 2024-04-24T09:07:33.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.324+0000 7fb6dea00640 1 -- 172.21.15.83:0/756402847 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb6e0033c20 con 0x7fb6e8073e50 2024-04-24T09:07:33.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.324+0000 7fb6dea00640 1 -- 172.21.15.83:0/756402847 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e0038040 con 0x7fb6e8073e50 2024-04-24T09:07:33.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.324+0000 7fb6e7200640 1 -- 172.21.15.83:0/756402847 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6e8071ac0 con 0x7fb6e8073e50 2024-04-24T09:07:33.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.328+0000 7fb6f0027740 1 Processor -- start 2024-04-24T09:07:33.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.329+0000 7fb6dea00640 1 -- 172.21.15.83:0/756402847 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb6e0045030 con 0x7fb6e8073e50 2024-04-24T09:07:33.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.329+0000 7fb6f0027740 1 -- start start 2024-04-24T09:07:33.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.330+0000 7fb6f0027740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d30261cb40 0x55d30261cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:33.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.330+0000 7fb6f0027740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d30261d460 0x55d3026275f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:33.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.330+0000 7fb6f0027740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d302627b30 0x55d302629f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:33.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.330+0000 7fb6f0027740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d3025821d0 con 0x55d30261cb40 2024-04-24T09:07:33.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.330+0000 7fb6f0027740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d302528450 con 0x55d30261d460 2024-04-24T09:07:33.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.330+0000 7fb6f0027740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d30257e190 con 0x55d302627b30 2024-04-24T09:07:33.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.330+0000 7fb6e5400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d30261cb40 0x55d30261cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:33.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.330+0000 7fb6e5400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d30261cb40 0x55d30261cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39980/0 (socket says 172.21.15.83:39980) 2024-04-24T09:07:33.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.330+0000 7fb6e5400640 1 -- 172.21.15.83:0/654142881 learned_addr learned my addr 172.21.15.83:0/654142881 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:07:33.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.330+0000 7fb6e5e00640 1 --2- 172.21.15.83:0/654142881 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d30261d460 0x55d3026275f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:33.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.331+0000 7fb6e5400640 1 --2- 172.21.15.83:0/654142881 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d302627b30 0x55d302629f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:33.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.331+0000 7fb6dea00640 1 --2- 172.21.15.83:0/756402847 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb6cc0a3da0 0x7fb6cc0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:33.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.331+0000 7fb6e5400640 1 -- 172.21.15.83:0/654142881 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d302627b30 msgr2=0x55d302629f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:33.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.331+0000 7fb6e5400640 1 --2- 172.21.15.83:0/654142881 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d302627b30 0x55d302629f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:33.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.331+0000 7fb6e5400640 1 -- 172.21.15.83:0/654142881 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d30261d460 msgr2=0x55d3026275f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:33.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.331+0000 7fb6e5400640 1 --2- 172.21.15.83:0/654142881 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d30261d460 0x55d3026275f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:33.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.331+0000 7fb6e5400640 1 -- 172.21.15.83:0/654142881 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3025e5310 con 0x55d30261cb40 2024-04-24T09:07:33.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.331+0000 7fb6dea00640 1 -- 172.21.15.83:0/756402847 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 9583+0+0 (secure 0 0 0) 0x7fb6e00f4770 con 0x7fb6e8073e50 2024-04-24T09:07:33.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.331+0000 7fb6e6800640 1 --2- 172.21.15.83:0/756402847 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb6cc0a3da0 0x7fb6cc0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:33.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.332+0000 7fb6e5400640 1 --2- 172.21.15.83:0/654142881 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d30261cb40 0x55d30261cf20 secure :-1 s=READY pgs=2260 cs=0 l=1 rev1=1 crypto rx=0x7fb6d4012900 tx=0x7fb6d4005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26903a42f381cb39 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:33.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.332+0000 7fb6d3400640 1 -- 172.21.15.83:0/654142881 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6d400e8d0 con 0x55d30261cb40 2024-04-24T09:07:33.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.332+0000 7fb6d3400640 1 -- 172.21.15.83:0/654142881 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb6d4005020 con 0x55d30261cb40 2024-04-24T09:07:33.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.332+0000 7fb6d3400640 1 -- 172.21.15.83:0/654142881 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6d4004a00 con 0x55d30261cb40 2024-04-24T09:07:33.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.333+0000 7fb6f0027740 1 -- 172.21.15.83:0/654142881 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d30261cb40 msgr2=0x55d30261cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:33.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.333+0000 7fb6f0027740 1 --2- 172.21.15.83:0/654142881 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d30261cb40 0x55d30261cf20 secure :-1 s=READY pgs=2260 cs=0 l=1 rev1=1 crypto rx=0x7fb6d4012900 tx=0x7fb6d4005850 comp rx=0 tx=0).stop 2024-04-24T09:07:33.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.333+0000 7fb6e6800640 1 --2- 172.21.15.83:0/756402847 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb6cc0a3da0 0x7fb6cc0a6260 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fb6e00c89d0 tx=0x7fb6e0005e50 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:33.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.333+0000 7fb6f0027740 1 -- 172.21.15.83:0/654142881 shutdown_connections 2024-04-24T09:07:33.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.333+0000 7fb6f0027740 1 --2- 172.21.15.83:0/654142881 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d302627b30 0x55d302629f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:33.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.333+0000 7fb6f0027740 1 --2- 172.21.15.83:0/654142881 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d30261d460 0x55d3026275f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:33.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.333+0000 7fb6f0027740 1 --2- 172.21.15.83:0/654142881 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d30261cb40 0x55d30261cf20 unknown :-1 s=CLOSED pgs=2260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:33.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.333+0000 7fb6f0027740 1 -- 172.21.15.83:0/654142881 >> 172.21.15.83:0/654142881 conn(0x55d302618e60 msgr2=0x55d302619240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:33.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.334+0000 7fb6f0027740 1 -- 172.21.15.83:0/654142881 shutdown_connections 2024-04-24T09:07:33.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.334+0000 7fb6f0027740 1 -- 172.21.15.83:0/654142881 wait complete. 2024-04-24T09:07:33.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.334+0000 7fb6f0027740 1 -- start start 2024-04-24T09:07:33.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.334+0000 7fb6f0027740 1 Processor -- start 2024-04-24T09:07:33.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.335+0000 7fb6f0027740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d30261cb40 0x55d302627550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:33.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.335+0000 7fb6f0027740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d30261d460 0x55d3026248b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:33.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.335+0000 7fb6f0027740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d302627b30 0x55d302624f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:33.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.335+0000 7fb6f0027740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d30262a9b0 con 0x55d30261cb40 2024-04-24T09:07:33.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.335+0000 7fb6f0027740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d30253b810 con 0x55d30261d460 2024-04-24T09:07:33.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.335+0000 7fb6f0027740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d30262ab30 con 0x55d302627b30 2024-04-24T09:07:33.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.335+0000 7fb6f0027740 10 client.0 _ll_register_callbacks cb 0x55d302509140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:07:33.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.335+0000 7fb6e5400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d30261cb40 0x55d302627550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:33.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.335+0000 7fb6e5400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d30261cb40 0x55d302627550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39988/0 (socket says 172.21.15.83:39988) 2024-04-24T09:07:33.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.335+0000 7fb6e5400640 1 -- 172.21.15.83:0/1152604066 learned_addr learned my addr 172.21.15.83:0/1152604066 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:07:33.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.335+0000 7fb6e5e00640 1 --2- 172.21.15.83:0/1152604066 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d30261d460 0x55d3026248b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:33.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.336+0000 7fb6e5400640 1 --2- 172.21.15.83:0/1152604066 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d302627b30 0x55d302624f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:33.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.336+0000 7fb6e5e00640 1 -- 172.21.15.83:0/1152604066 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d302627b30 msgr2=0x55d302624f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:33.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.336+0000 7fb6e5e00640 1 --2- 172.21.15.83:0/1152604066 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d302627b30 0x55d302624f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:33.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.336+0000 7fb6e5e00640 1 -- 172.21.15.83:0/1152604066 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d30261cb40 msgr2=0x55d302627550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:33.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.336+0000 7fb6e5e00640 1 --2- 172.21.15.83:0/1152604066 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d30261cb40 0x55d302627550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:33.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.336+0000 7fb6e5e00640 1 -- 172.21.15.83:0/1152604066 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d30254c3f0 con 0x55d30261d460 2024-04-24T09:07:33.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.336+0000 7fb6e5400640 1 --2- 172.21.15.83:0/1152604066 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d302627b30 0x55d302624f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:07:33.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.336+0000 7fb6e5e00640 1 --2- 172.21.15.83:0/1152604066 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d30261d460 0x55d3026248b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb6d8005fd0 tx=0x7fb6d8012b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=419d723f88facec2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:33.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.336+0000 7fb6c7e00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:07:33.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.337+0000 7fb6c7e00640 1 -- 172.21.15.83:0/1152604066 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6d800f920 con 0x55d30261d460 2024-04-24T09:07:33.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.337+0000 7fb6c7e00640 1 -- 172.21.15.83:0/1152604066 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9583+0+0 (secure 0 0 0) 0x7fb6d800fac0 con 0x55d30261d460 2024-04-24T09:07:33.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.337+0000 7fb6f0027740 1 -- 172.21.15.83:0/1152604066 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6d4005020 con 0x55d30261d460 2024-04-24T09:07:33.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.337+0000 7fb6c7e00640 1 -- 172.21.15.83:0/1152604066 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7fb6d800fcc0 con 0x55d30261d460 2024-04-24T09:07:33.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.338+0000 7fb6f0027740 10 client.38381 fetch_fsmap learned FSMap version 51 2024-04-24T09:07:33.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.338+0000 7fb6f0027740 1 -- 172.21.15.83:0/1152604066 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x55d3022943f0 con 0x55d30261d460 2024-04-24T09:07:33.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.339+0000 7fb6c7e00640 1 -- 172.21.15.83:0/1152604066 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb6d4005020 con 0x55d30261d460 2024-04-24T09:07:33.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.339+0000 7fb6c7e00640 1 -- 172.21.15.83:0/1152604066 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 51) ==== 8406+0+0 (secure 0 0 0) 0x7fb6d801c420 con 0x55d30261d460 2024-04-24T09:07:33.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.339+0000 7fb6f0027740 10 client.38381 fetch_fsmap finished waiting for FSMap version 51 2024-04-24T09:07:33.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.339+0000 7fb6f0027740 10 client.38381 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:07:33.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.339+0000 7fb6f0027740 20 client.38381 populate_metadata read hostname 'smithi083' 2024-04-24T09:07:33.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.339+0000 7fb6f0027740 1 --2- 172.21.15.83:0/1152604066 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d302640640 0x55d302642a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:33.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.339+0000 7fb6f0027740 4 client.38381 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:07:33.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.339+0000 7fb6f0027740 1 -- 172.21.15.83:0/1152604066 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d30258fbf0 con 0x55d302640640 2024-04-24T09:07:33.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.340+0000 7fb6e5400640 1 --2- 172.21.15.83:0/1152604066 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d302640640 0x55d302642a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:33.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.341+0000 7fb6c7e00640 10 client.38381 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:07:33.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.341+0000 7fb6e5400640 1 --2- 172.21.15.83:0/1152604066 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d302640640 0x55d302642a90 crc :-1 s=READY pgs=405 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26281ba78dbf5f5b server_cookie=79bb1acb6ecda66f in_seq=0 out_seq=0 2024-04-24T09:07:33.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.342+0000 7fb6c7e00640 1 -- 172.21.15.83:0/1152604066 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d30258fbf0 con 0x55d302640640 2024-04-24T09:07:33.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.342+0000 7fb6c7e00640 10 client.38381 handle_command_reply: tid=0 2024-04-24T09:07:33.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.342+0000 7fb6f0027740 1 client.38381 shutdown 2024-04-24T09:07:33.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.342+0000 7fb6f0027740 1 -- 172.21.15.83:0/1152604066 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d30261d460 msgr2=0x55d3026248b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:33.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.342+0000 7fb6f0027740 1 --2- 172.21.15.83:0/1152604066 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d30261d460 0x55d3026248b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb6d8005fd0 tx=0x7fb6d8012b40 comp rx=0 tx=0).stop 2024-04-24T09:07:33.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.343+0000 7fb6f0027740 1 -- 172.21.15.83:0/1152604066 shutdown_connections 2024-04-24T09:07:33.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.343+0000 7fb6f0027740 1 --2- 172.21.15.83:0/1152604066 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d302640640 0x55d302642a90 crc :-1 s=READY pgs=405 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:33.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.343+0000 7fb6f0027740 1 --2- 172.21.15.83:0/1152604066 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d302627b30 0x55d302624f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:33.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.343+0000 7fb6f0027740 1 --2- 172.21.15.83:0/1152604066 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d30261d460 0x55d3026248b0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:33.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.343+0000 7fb6f0027740 1 --2- 172.21.15.83:0/1152604066 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d30261cb40 0x55d302627550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:33.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.343+0000 7fb6c7e00640 0 client.38381 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:07:33.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.343+0000 7fb6f0027740 1 -- 172.21.15.83:0/1152604066 >> 172.21.15.83:0/1152604066 conn(0x55d302618e60 msgr2=0x55d30261eb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:33.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.343+0000 7fb6f0027740 1 -- 172.21.15.83:0/1152604066 shutdown_connections 2024-04-24T09:07:33.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.343+0000 7fb6f0027740 1 -- 172.21.15.83:0/1152604066 wait complete. 2024-04-24T09:07:33.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.343+0000 7fb6f0027740 20 client.38381 trim_cache size 0 max 16384 2024-04-24T09:07:33.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.367+0000 7fb6f0027740 1 Processor -- start 2024-04-24T09:07:33.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.367+0000 7fb6f0027740 1 -- start start 2024-04-24T09:07:33.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.368+0000 7fb6f0027740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d30261cb40 0x55d3026483a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:33.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.368+0000 7fb6f0027740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d30261d460 0x55d3026488e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:33.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.368+0000 7fb6f0027740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d302627b30 0x55d30261fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:33.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.368+0000 7fb6f0027740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e81b5930 con 0x55d302627b30 2024-04-24T09:07:33.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.368+0000 7fb6f0027740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e81b75c0 con 0x55d30261cb40 2024-04-24T09:07:33.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.368+0000 7fb6f0027740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e81b6000 con 0x55d30261d460 2024-04-24T09:07:33.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.368+0000 7fb6e5400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d30261cb40 0x55d3026483a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:33.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.368+0000 7fb6e5400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d30261cb40 0x55d3026483a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:43094/0 (socket says 172.21.15.83:43094) 2024-04-24T09:07:33.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.368+0000 7fb6e5400640 1 -- 172.21.15.83:0/4029216495 learned_addr learned my addr 172.21.15.83:0/4029216495 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:07:33.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.368+0000 7fb6e5400640 1 --2- 172.21.15.83:0/4029216495 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d302627b30 0x55d30261fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:33.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.368+0000 7fb6e5e00640 1 --2- 172.21.15.83:0/4029216495 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d30261d460 0x55d3026488e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:33.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.369+0000 7fb6e5400640 1 -- 172.21.15.83:0/4029216495 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d30261d460 msgr2=0x55d3026488e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:33.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.369+0000 7fb6e5400640 1 --2- 172.21.15.83:0/4029216495 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d30261d460 0x55d3026488e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:33.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.369+0000 7fb6e5400640 1 -- 172.21.15.83:0/4029216495 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d302627b30 msgr2=0x55d30261fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:33.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.369+0000 7fb6e5400640 1 --2- 172.21.15.83:0/4029216495 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d302627b30 0x55d30261fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:33.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.369+0000 7fb6e5400640 1 -- 172.21.15.83:0/4029216495 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3026252a0 con 0x55d30261cb40 2024-04-24T09:07:33.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.369+0000 7fb6e5400640 1 --2- 172.21.15.83:0/4029216495 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d30261cb40 0x55d3026483a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb6d4005d80 tx=0x7fb6d4012450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27d98f0359128c0a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:33.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.370+0000 7fb6d2a00640 1 -- 172.21.15.83:0/4029216495 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6d400bbc0 con 0x55d30261cb40 2024-04-24T09:07:33.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.370+0000 7fb6d2a00640 1 -- 172.21.15.83:0/4029216495 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb6d402b820 con 0x55d30261cb40 2024-04-24T09:07:33.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.370+0000 7fb6d2a00640 1 -- 172.21.15.83:0/4029216495 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6d400eea0 con 0x55d30261cb40 2024-04-24T09:07:33.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.370+0000 7fb6f0027740 1 -- 172.21.15.83:0/4029216495 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d30261cb40 msgr2=0x55d3026483a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:33.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.370+0000 7fb6f0027740 1 --2- 172.21.15.83:0/4029216495 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d30261cb40 0x55d3026483a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb6d4005d80 tx=0x7fb6d4012450 comp rx=0 tx=0).stop 2024-04-24T09:07:33.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.371+0000 7fb6f0027740 1 -- 172.21.15.83:0/4029216495 shutdown_connections 2024-04-24T09:07:33.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.371+0000 7fb6f0027740 1 --2- 172.21.15.83:0/4029216495 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d302627b30 0x55d30261fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:33.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.371+0000 7fb6f0027740 1 --2- 172.21.15.83:0/4029216495 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d30261d460 0x55d3026488e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:33.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.371+0000 7fb6f0027740 1 --2- 172.21.15.83:0/4029216495 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d30261cb40 0x55d3026483a0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:33.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.371+0000 7fb6f0027740 1 -- 172.21.15.83:0/4029216495 >> 172.21.15.83:0/4029216495 conn(0x55d302618e60 msgr2=0x55d302623e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:33.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.371+0000 7fb6f0027740 1 -- 172.21.15.83:0/4029216495 shutdown_connections 2024-04-24T09:07:33.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.371+0000 7fb6f0027740 1 -- 172.21.15.83:0/4029216495 wait complete. 2024-04-24T09:07:33.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.372+0000 7fb6f0027740 1 -- start start 2024-04-24T09:07:33.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.372+0000 7fb6f0027740 1 Processor -- start 2024-04-24T09:07:33.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.373+0000 7fb6f0027740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d30261cb40 0x55d302638610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:33.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.373+0000 7fb6f0027740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d30261d460 0x55d302638b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:33.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.373+0000 7fb6f0027740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d302627b30 0x55d302621b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:33.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.373+0000 7fb6f0027740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e81b6e30 con 0x55d30261cb40 2024-04-24T09:07:33.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.373+0000 7fb6f0027740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e81b6730 con 0x55d302627b30 2024-04-24T09:07:33.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.373+0000 7fb6f0027740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e81b3930 con 0x55d30261d460 2024-04-24T09:07:33.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.373+0000 7fb6f0027740 10 client.0 _ll_register_callbacks cb 0x7fb6d8009890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:07:33.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.374+0000 7fb6e5400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d30261cb40 0x55d302638610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:33.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.374+0000 7fb6e5400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d30261cb40 0x55d302638610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40012/0 (socket says 172.21.15.83:40012) 2024-04-24T09:07:33.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.374+0000 7fb6e5400640 1 -- 172.21.15.83:0/1832179170 learned_addr learned my addr 172.21.15.83:0/1832179170 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:07:33.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.374+0000 7fb6e5e00640 1 --2- 172.21.15.83:0/1832179170 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d30261d460 0x55d302638b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:33.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.374+0000 7fb6e5400640 1 -- 172.21.15.83:0/1832179170 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d30261d460 msgr2=0x55d302638b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:33.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.374+0000 7fb6e5400640 1 --2- 172.21.15.83:0/1832179170 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d30261d460 0x55d302638b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:33.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.374+0000 7fb6e5400640 1 -- 172.21.15.83:0/1832179170 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d302627b30 msgr2=0x55d302621b50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-24T09:07:33.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.374+0000 7fb6e5400640 1 --2- 172.21.15.83:0/1832179170 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d302627b30 0x55d302621b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:33.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.374+0000 7fb6e5400640 1 -- 172.21.15.83:0/1832179170 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d30235e7d0 con 0x55d30261cb40 2024-04-24T09:07:33.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.375+0000 7fb6e5400640 1 --2- 172.21.15.83:0/1832179170 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d30261cb40 0x55d302638610 secure :-1 s=READY pgs=2261 cs=0 l=1 rev1=1 crypto rx=0x7fb6d40127b0 tx=0x7fb6d4012570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3892dcb701662f3a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:33.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.375+0000 7fb6c7e00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:07:33.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.375+0000 7fb6c7e00640 1 -- 172.21.15.83:0/1832179170 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6d402b570 con 0x55d30261cb40 2024-04-24T09:07:33.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.375+0000 7fb6c7e00640 1 -- 172.21.15.83:0/1832179170 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9583+0+0 (secure 0 0 0) 0x7fb6d400e8d0 con 0x55d30261cb40 2024-04-24T09:07:33.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.375+0000 7fb6f0027740 1 -- 172.21.15.83:0/1832179170 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6d402b820 con 0x55d30261cb40 2024-04-24T09:07:33.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.376+0000 7fb6c7e00640 1 -- 172.21.15.83:0/1832179170 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7fb6d403e050 con 0x55d30261cb40 2024-04-24T09:07:33.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.376+0000 7fb6f0027740 10 client.21231 fetch_fsmap learned FSMap version 51 2024-04-24T09:07:33.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.376+0000 7fb6f0027740 1 -- 172.21.15.83:0/1832179170 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x55d302628830 con 0x55d30261cb40 2024-04-24T09:07:33.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.377+0000 7fb6c7e00640 1 -- 172.21.15.83:0/1832179170 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb6d402b820 con 0x55d30261cb40 2024-04-24T09:07:33.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.377+0000 7fb6c7e00640 1 -- 172.21.15.83:0/1832179170 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 51) ==== 8406+0+0 (secure 0 0 0) 0x7fb6d401c020 con 0x55d30261cb40 2024-04-24T09:07:33.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.377+0000 7fb6f0027740 10 client.21231 fetch_fsmap finished waiting for FSMap version 51 2024-04-24T09:07:33.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.377+0000 7fb6f0027740 10 client.21231 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:07:33.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.377+0000 7fb6f0027740 20 client.21231 populate_metadata read hostname 'smithi083' 2024-04-24T09:07:33.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.377+0000 7fb6f0027740 1 --2- 172.21.15.83:0/1832179170 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d3026a6c10 0x55d3026a9000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:33.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.378+0000 7fb6f0027740 4 client.21231 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:07:33.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.378+0000 7fb6f0027740 1 -- 172.21.15.83:0/1832179170 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d302589a60 con 0x55d3026a6c10 2024-04-24T09:07:33.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.378+0000 7fb6e5e00640 1 --2- 172.21.15.83:0/1832179170 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d3026a6c10 0x55d3026a9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:33.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.379+0000 7fb6e5e00640 1 --2- 172.21.15.83:0/1832179170 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d3026a6c10 0x55d3026a9000 crc :-1 s=READY pgs=406 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c33feb1025cf337 server_cookie=6ad92297ff797dc8 in_seq=0 out_seq=0 2024-04-24T09:07:33.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.379+0000 7fb6c7e00640 10 client.21231 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:07:33.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.379+0000 7fb6c7e00640 1 -- 172.21.15.83:0/1832179170 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d302589a60 con 0x55d3026a6c10 2024-04-24T09:07:33.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.379+0000 7fb6c7e00640 10 client.21231 handle_command_reply: tid=0 2024-04-24T09:07:33.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.379+0000 7fb6f0027740 1 client.21231 shutdown 2024-04-24T09:07:33.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.380+0000 7fb6f0027740 1 -- 172.21.15.83:0/1832179170 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d30261cb40 msgr2=0x55d302638610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:33.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.380+0000 7fb6f0027740 1 --2- 172.21.15.83:0/1832179170 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d30261cb40 0x55d302638610 secure :-1 s=READY pgs=2261 cs=0 l=1 rev1=1 crypto rx=0x7fb6d40127b0 tx=0x7fb6d4012570 comp rx=0 tx=0).stop 2024-04-24T09:07:33.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.380+0000 7fb6f0027740 1 -- 172.21.15.83:0/1832179170 shutdown_connections 2024-04-24T09:07:33.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.380+0000 7fb6f0027740 1 --2- 172.21.15.83:0/1832179170 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d3026a6c10 0x55d3026a9000 crc :-1 s=READY pgs=406 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:33.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.380+0000 7fb6f0027740 1 --2- 172.21.15.83:0/1832179170 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d302627b30 0x55d302621b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:33.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.380+0000 7fb6f0027740 1 --2- 172.21.15.83:0/1832179170 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d30261d460 0x55d302638b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:33.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.380+0000 7fb6f0027740 1 --2- 172.21.15.83:0/1832179170 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d30261cb40 0x55d302638610 unknown :-1 s=CLOSED pgs=2261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:33.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.380+0000 7fb6f0027740 1 -- 172.21.15.83:0/1832179170 >> 172.21.15.83:0/1832179170 conn(0x55d302618e60 msgr2=0x55d30261fb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:33.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.380+0000 7fb6c7e00640 0 client.21231 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:07:33.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.380+0000 7fb6f0027740 1 -- 172.21.15.83:0/1832179170 shutdown_connections 2024-04-24T09:07:33.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.381+0000 7fb6f0027740 1 -- 172.21.15.83:0/1832179170 wait complete. 2024-04-24T09:07:33.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.381+0000 7fb6f0027740 20 client.21231 trim_cache size 0 max 16384 2024-04-24T09:07:33.382 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:07:33.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.382+0000 7fb6e7200640 1 -- 172.21.15.83:0/756402847 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb6cc0a3da0 msgr2=0x7fb6cc0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:33.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.382+0000 7fb6e7200640 1 --2- 172.21.15.83:0/756402847 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb6cc0a3da0 0x7fb6cc0a6260 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fb6e00c89d0 tx=0x7fb6e0005e50 comp rx=0 tx=0).stop 2024-04-24T09:07:33.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.382+0000 7fb6e7200640 1 -- 172.21.15.83:0/756402847 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6e8073e50 msgr2=0x7fb6e806ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:33.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.382+0000 7fb6e7200640 1 --2- 172.21.15.83:0/756402847 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6e8073e50 0x7fb6e806ce90 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb6e002c950 tx=0x7fb6e0033a30 comp rx=0 tx=0).stop 2024-04-24T09:07:33.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.382+0000 7fb6e7200640 1 -- 172.21.15.83:0/756402847 shutdown_connections 2024-04-24T09:07:33.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.382+0000 7fb6e7200640 1 --2- 172.21.15.83:0/756402847 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb6cc0a3da0 0x7fb6cc0a6260 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:33.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.382+0000 7fb6e7200640 1 --2- 172.21.15.83:0/756402847 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6e8073e50 0x7fb6e806ce90 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:33.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.382+0000 7fb6e7200640 1 --2- 172.21.15.83:0/756402847 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6e81060d0 0x7fb6e806c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:33.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.382+0000 7fb6e7200640 1 --2- 172.21.15.83:0/756402847 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb6e8101000 0x7fb6e8073800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:33.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.382+0000 7fb6e7200640 1 -- 172.21.15.83:0/756402847 >> 172.21.15.83:0/756402847 conn(0x7fb6e80fc880 msgr2=0x7fb6e80feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:33.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.382+0000 7fb6e7200640 1 -- 172.21.15.83:0/756402847 shutdown_connections 2024-04-24T09:07:33.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:33.382+0000 7fb6e7200640 1 -- 172.21.15.83:0/756402847 wait complete. 2024-04-24T09:07:34.162 INFO:tasks.workunit.client.0.smithi083.stdout: 10 16317 11.33 MB/sec execute 5 sec latency 77.475 ms 2024-04-24T09:07:34.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:07:33 smithi083 ceph-mon[27907]: pgmap v501: 129 pgs: 129 active+clean; 417 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.7 MiB/s wr, 340 op/s 2024-04-24T09:07:34.404 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T09:07:34.405 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T09:07:34.405 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T09:07:34.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:07:33 smithi138 ceph-mon[31441]: pgmap v501: 129 pgs: 129 active+clean; 417 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.7 MiB/s wr, 340 op/s 2024-04-24T09:07:34.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:07:33 smithi189 ceph-mon[31791]: pgmap v501: 129 pgs: 129 active+clean; 417 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.7 MiB/s wr, 340 op/s 2024-04-24T09:07:34.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.575+0000 7f3c8d400640 1 -- 172.21.15.83:0/2797749158 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c8810e660 msgr2=0x7f3c8810ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:34.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.575+0000 7f3c8d400640 1 --2- 172.21.15.83:0/2797749158 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c8810e660 0x7f3c8810ea40 secure :-1 s=READY pgs=2262 cs=0 l=1 rev1=1 crypto rx=0x7f3c78009930 tx=0x7f3c7802a420 comp rx=0 tx=0).stop 2024-04-24T09:07:34.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.576+0000 7f3c8d400640 1 -- 172.21.15.83:0/2797749158 shutdown_connections 2024-04-24T09:07:34.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.576+0000 7f3c8d400640 1 --2- 172.21.15.83:0/2797749158 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c88108fd0 0x7f3c88115490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:34.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.576+0000 7f3c8d400640 1 --2- 172.21.15.83:0/2797749158 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3c88108610 0x7f3c88108a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:34.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.576+0000 7f3c8d400640 1 --2- 172.21.15.83:0/2797749158 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c8810e660 0x7f3c8810ea40 unknown :-1 s=CLOSED pgs=2262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:34.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.576+0000 7f3c8d400640 1 -- 172.21.15.83:0/2797749158 >> 172.21.15.83:0/2797749158 conn(0x7f3c88076540 msgr2=0x7f3c88078960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:34.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.576+0000 7f3c8d400640 1 -- 172.21.15.83:0/2797749158 shutdown_connections 2024-04-24T09:07:34.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.576+0000 7f3c8d400640 1 -- 172.21.15.83:0/2797749158 wait complete. 2024-04-24T09:07:34.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.577+0000 7f3c8d400640 1 Processor -- start 2024-04-24T09:07:34.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.577+0000 7f3c8d400640 1 -- start start 2024-04-24T09:07:34.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.577+0000 7f3c8d400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c88108610 0x7f3c8819ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:34.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.577+0000 7f3c8d400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3c88108fd0 0x7f3c8819f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:34.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.577+0000 7f3c8d400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c8810e660 0x7f3c881a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:34.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.577+0000 7f3c8d400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3c88118890 con 0x7f3c8810e660 2024-04-24T09:07:34.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.577+0000 7f3c8d400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3c88118710 con 0x7f3c88108610 2024-04-24T09:07:34.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.577+0000 7f3c8d400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3c88118a10 con 0x7f3c88108fd0 2024-04-24T09:07:34.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.578+0000 7f3c87200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c8810e660 0x7f3c881a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:34.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.578+0000 7f3c87200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c8810e660 0x7f3c881a3650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40034/0 (socket says 172.21.15.83:40034) 2024-04-24T09:07:34.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.578+0000 7f3c87200640 1 -- 172.21.15.83:0/4094457495 learned_addr learned my addr 172.21.15.83:0/4094457495 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:07:34.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.578+0000 7f3c85e00640 1 --2- 172.21.15.83:0/4094457495 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3c88108fd0 0x7f3c8819f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:34.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.578+0000 7f3c86800640 1 --2- 172.21.15.83:0/4094457495 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c88108610 0x7f3c8819ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:34.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.578+0000 7f3c87200640 1 -- 172.21.15.83:0/4094457495 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3c88108fd0 msgr2=0x7f3c8819f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:34.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.578+0000 7f3c87200640 1 --2- 172.21.15.83:0/4094457495 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3c88108fd0 0x7f3c8819f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:34.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.578+0000 7f3c87200640 1 -- 172.21.15.83:0/4094457495 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c88108610 msgr2=0x7f3c8819ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:34.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.578+0000 7f3c87200640 1 --2- 172.21.15.83:0/4094457495 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c88108610 0x7f3c8819ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:34.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.578+0000 7f3c87200640 1 -- 172.21.15.83:0/4094457495 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c881a3d80 con 0x7f3c8810e660 2024-04-24T09:07:34.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.578+0000 7f3c87200640 1 --2- 172.21.15.83:0/4094457495 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c8810e660 0x7f3c881a3650 secure :-1 s=READY pgs=2263 cs=0 l=1 rev1=1 crypto rx=0x7f3c8000cce0 tx=0x7f3c8000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:34.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.578+0000 7f3c7f400640 1 -- 172.21.15.83:0/4094457495 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c8000f450 con 0x7f3c8810e660 2024-04-24T09:07:34.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.579+0000 7f3c7f400640 1 -- 172.21.15.83:0/4094457495 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3c8000faa0 con 0x7f3c8810e660 2024-04-24T09:07:34.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.579+0000 7f3c8d400640 1 -- 172.21.15.83:0/4094457495 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c881a4070 con 0x7f3c8810e660 2024-04-24T09:07:34.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.579+0000 7f3c7f400640 1 -- 172.21.15.83:0/4094457495 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c80018b70 con 0x7f3c8810e660 2024-04-24T09:07:34.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.579+0000 7f3c8d400640 1 -- 172.21.15.83:0/4094457495 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c881ab900 con 0x7f3c8810e660 2024-04-24T09:07:34.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.582+0000 7f3c901fc740 1 Processor -- start 2024-04-24T09:07:34.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.582+0000 7f3c7f400640 1 -- 172.21.15.83:0/4094457495 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3c80018d10 con 0x7f3c8810e660 2024-04-24T09:07:34.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.582+0000 7f3c901fc740 1 -- start start 2024-04-24T09:07:34.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.582+0000 7f3c901fc740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5612e0265090 0x5612e0265470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:34.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.582+0000 7f3c901fc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612e02659b0 0x5612e026fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:34.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.582+0000 7f3c901fc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5612e0270080 0x5612e0272470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:34.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.582+0000 7f3c901fc740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5612e01cae70 con 0x5612e02659b0 2024-04-24T09:07:34.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.582+0000 7f3c901fc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5612e0171380 con 0x5612e0270080 2024-04-24T09:07:34.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.582+0000 7f3c901fc740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5612e01c6e30 con 0x5612e0265090 2024-04-24T09:07:34.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.582+0000 7f3c86800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612e02659b0 0x5612e026fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:34.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.583+0000 7f3c86800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612e02659b0 0x5612e026fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40040/0 (socket says 172.21.15.83:40040) 2024-04-24T09:07:34.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.583+0000 7f3c86800640 1 -- 172.21.15.83:0/2058739899 learned_addr learned my addr 172.21.15.83:0/2058739899 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:07:34.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.583+0000 7f3c85e00640 1 --2- 172.21.15.83:0/2058739899 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5612e0265090 0x5612e0265470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:34.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.583+0000 7f3c86800640 1 -- 172.21.15.83:0/2058739899 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5612e0265090 msgr2=0x5612e0265470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:34.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.583+0000 7f3c86800640 1 --2- 172.21.15.83:0/2058739899 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5612e0265090 0x5612e0265470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:34.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.583+0000 7f3c85e00640 1 --2- 172.21.15.83:0/2058739899 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5612e0270080 0x5612e0272470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:34.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.583+0000 7f3c86800640 1 -- 172.21.15.83:0/2058739899 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5612e0270080 msgr2=0x5612e0272470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:34.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.583+0000 7f3c86800640 1 --2- 172.21.15.83:0/2058739899 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5612e0270080 0x5612e0272470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:34.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.583+0000 7f3c86800640 1 -- 172.21.15.83:0/2058739899 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612dffa57e0 con 0x5612e02659b0 2024-04-24T09:07:34.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.583+0000 7f3c85e00640 1 --2- 172.21.15.83:0/2058739899 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5612e0270080 0x5612e0272470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:07:34.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.583+0000 7f3c86800640 1 --2- 172.21.15.83:0/2058739899 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612e02659b0 0x5612e026fb40 secure :-1 s=READY pgs=2264 cs=0 l=1 rev1=1 crypto rx=0x7f3c78031440 tx=0x7f3c78032cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7a6905b7207ea3b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:34.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.583+0000 7f3c7f400640 1 --2- 172.21.15.83:0/4094457495 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f3c6c0a3d20 0x7f3c6c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:34.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.584+0000 7f3c73400640 1 -- 172.21.15.83:0/2058739899 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c7803f070 con 0x5612e02659b0 2024-04-24T09:07:34.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.584+0000 7f3c7f400640 1 -- 172.21.15.83:0/4094457495 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 9583+0+0 (secure 0 0 0) 0x7f3c800d1160 con 0x7f3c8810e660 2024-04-24T09:07:34.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.584+0000 7f3c73400640 1 -- 172.21.15.83:0/2058739899 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3c780316a0 con 0x5612e02659b0 2024-04-24T09:07:34.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.584+0000 7f3c73400640 1 -- 172.21.15.83:0/2058739899 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c78035070 con 0x5612e02659b0 2024-04-24T09:07:34.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.584+0000 7f3c85e00640 1 --2- 172.21.15.83:0/4094457495 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f3c6c0a3d20 0x7f3c6c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:34.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.584+0000 7f3c901fc740 1 -- 172.21.15.83:0/2058739899 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612e02659b0 msgr2=0x5612e026fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:34.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.584+0000 7f3c901fc740 1 --2- 172.21.15.83:0/2058739899 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612e02659b0 0x5612e026fb40 secure :-1 s=READY pgs=2264 cs=0 l=1 rev1=1 crypto rx=0x7f3c78031440 tx=0x7f3c78032cc0 comp rx=0 tx=0).stop 2024-04-24T09:07:34.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.584+0000 7f3c85e00640 1 --2- 172.21.15.83:0/4094457495 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f3c6c0a3d20 0x7f3c6c0a61e0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x5612e0262540 tx=0x7f3c740046c0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:34.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.585+0000 7f3c901fc740 1 -- 172.21.15.83:0/2058739899 shutdown_connections 2024-04-24T09:07:34.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.585+0000 7f3c901fc740 1 --2- 172.21.15.83:0/2058739899 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5612e0270080 0x5612e0272470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:34.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.585+0000 7f3c901fc740 1 --2- 172.21.15.83:0/2058739899 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612e02659b0 0x5612e026fb40 unknown :-1 s=CLOSED pgs=2264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:34.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.585+0000 7f3c901fc740 1 --2- 172.21.15.83:0/2058739899 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5612e0265090 0x5612e0265470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:34.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.585+0000 7f3c901fc740 1 -- 172.21.15.83:0/2058739899 >> 172.21.15.83:0/2058739899 conn(0x5612e02613b0 msgr2=0x5612e0261790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:34.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.585+0000 7f3c901fc740 1 -- 172.21.15.83:0/2058739899 shutdown_connections 2024-04-24T09:07:34.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.585+0000 7f3c901fc740 1 -- 172.21.15.83:0/2058739899 wait complete. 2024-04-24T09:07:34.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.585+0000 7f3c901fc740 1 -- start start 2024-04-24T09:07:34.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.585+0000 7f3c901fc740 1 Processor -- start 2024-04-24T09:07:34.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.586+0000 7f3c901fc740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5612e0265090 0x5612e026d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:34.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.586+0000 7f3c901fc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612e02659b0 0x5612e026d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:34.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.586+0000 7f3c901fc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5612e0270080 0x5612e026da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:34.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.586+0000 7f3c901fc740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5612e0272e10 con 0x5612e02659b0 2024-04-24T09:07:34.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.586+0000 7f3c901fc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5612e0184660 con 0x5612e0270080 2024-04-24T09:07:34.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.586+0000 7f3c901fc740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5612e0272f90 con 0x5612e0265090 2024-04-24T09:07:34.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.586+0000 7f3c901fc740 10 client.0 _ll_register_callbacks cb 0x5612e0172aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:07:34.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.586+0000 7f3c85e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612e02659b0 0x5612e026d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:34.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.586+0000 7f3c85e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612e02659b0 0x5612e026d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40048/0 (socket says 172.21.15.83:40048) 2024-04-24T09:07:34.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.586+0000 7f3c85e00640 1 -- 172.21.15.83:0/2932572182 learned_addr learned my addr 172.21.15.83:0/2932572182 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:07:34.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.586+0000 7f3c85e00640 1 -- 172.21.15.83:0/2932572182 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5612e0265090 msgr2=0x5612e026d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:34.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.586+0000 7f3c85e00640 1 --2- 172.21.15.83:0/2932572182 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5612e0265090 0x5612e026d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:34.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.586+0000 7f3c85e00640 1 -- 172.21.15.83:0/2932572182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5612e0270080 msgr2=0x5612e026da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:34.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.586+0000 7f3c85e00640 1 --2- 172.21.15.83:0/2932572182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5612e0270080 0x5612e026da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:34.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.586+0000 7f3c85e00640 1 -- 172.21.15.83:0/2932572182 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612e0005f70 con 0x5612e02659b0 2024-04-24T09:07:34.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.587+0000 7f3c85e00640 1 --2- 172.21.15.83:0/2932572182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612e02659b0 0x5612e026d540 secure :-1 s=READY pgs=2265 cs=0 l=1 rev1=1 crypto rx=0x7f3c74011c00 tx=0x7f3c74004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d5ae4e0df66a1fc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:34.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.587+0000 7f3c67e00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:07:34.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.587+0000 7f3c67e00640 1 -- 172.21.15.83:0/2932572182 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c7400fe50 con 0x5612e02659b0 2024-04-24T09:07:34.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.587+0000 7f3c67e00640 1 -- 172.21.15.83:0/2932572182 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9583+0+0 (secure 0 0 0) 0x7f3c74027400 con 0x5612e02659b0 2024-04-24T09:07:34.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.587+0000 7f3c901fc740 1 -- 172.21.15.83:0/2932572182 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c780316a0 con 0x5612e02659b0 2024-04-24T09:07:34.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.587+0000 7f3c67e00640 1 -- 172.21.15.83:0/2932572182 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f3c74030050 con 0x5612e02659b0 2024-04-24T09:07:34.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.587+0000 7f3c901fc740 10 client.21243 fetch_fsmap learned FSMap version 51 2024-04-24T09:07:34.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.587+0000 7f3c901fc740 1 -- 172.21.15.83:0/2932572182 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x5612dfe9d170 con 0x5612e02659b0 2024-04-24T09:07:34.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.587+0000 7f3c67e00640 1 -- 172.21.15.83:0/2932572182 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3c780316a0 con 0x5612e02659b0 2024-04-24T09:07:34.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.588+0000 7f3c67e00640 1 -- 172.21.15.83:0/2932572182 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 51) ==== 8406+0+0 (secure 0 0 0) 0x7f3c74010590 con 0x5612e02659b0 2024-04-24T09:07:34.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.588+0000 7f3c901fc740 10 client.21243 fetch_fsmap finished waiting for FSMap version 51 2024-04-24T09:07:34.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.588+0000 7f3c901fc740 10 client.21243 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:07:34.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.588+0000 7f3c901fc740 20 client.21243 populate_metadata read hostname 'smithi083' 2024-04-24T09:07:34.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.588+0000 7f3c901fc740 1 --2- 172.21.15.83:0/2932572182 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5612e0280490 0x5612e0282920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:34.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.588+0000 7f3c901fc740 4 client.21243 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:07:34.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.588+0000 7f3c901fc740 1 -- 172.21.15.83:0/2932572182 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5612e01d8890 con 0x5612e0280490 2024-04-24T09:07:34.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.630+0000 7f3c86800640 1 --2- 172.21.15.83:0/2932572182 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5612e0280490 0x5612e0282920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:34.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.636+0000 7f3c86800640 1 --2- 172.21.15.83:0/2932572182 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5612e0280490 0x5612e0282920 crc :-1 s=READY pgs=407 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=210821d1b5a8a03d server_cookie=f27bcf9bf30aa1f5 in_seq=0 out_seq=0 2024-04-24T09:07:34.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.636+0000 7f3c67e00640 10 client.21243 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:07:34.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.641+0000 7f3c67e00640 1 -- 172.21.15.83:0/2932572182 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5612e01d8890 con 0x5612e0280490 2024-04-24T09:07:34.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.641+0000 7f3c67e00640 10 client.21243 handle_command_reply: tid=0 2024-04-24T09:07:34.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.641+0000 7f3c901fc740 1 client.21243 shutdown 2024-04-24T09:07:34.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.642+0000 7f3c901fc740 1 -- 172.21.15.83:0/2932572182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612e02659b0 msgr2=0x5612e026d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:34.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.642+0000 7f3c901fc740 1 --2- 172.21.15.83:0/2932572182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612e02659b0 0x5612e026d540 secure :-1 s=READY pgs=2265 cs=0 l=1 rev1=1 crypto rx=0x7f3c74011c00 tx=0x7f3c74004730 comp rx=0 tx=0).stop 2024-04-24T09:07:34.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.642+0000 7f3c901fc740 1 -- 172.21.15.83:0/2932572182 shutdown_connections 2024-04-24T09:07:34.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.642+0000 7f3c901fc740 1 --2- 172.21.15.83:0/2932572182 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5612e0280490 0x5612e0282920 crc :-1 s=READY pgs=407 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:34.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.642+0000 7f3c901fc740 1 --2- 172.21.15.83:0/2932572182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5612e0270080 0x5612e026da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:34.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.642+0000 7f3c901fc740 1 --2- 172.21.15.83:0/2932572182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612e02659b0 0x5612e026d540 unknown :-1 s=CLOSED pgs=2265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:34.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.642+0000 7f3c901fc740 1 --2- 172.21.15.83:0/2932572182 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5612e0265090 0x5612e026d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:34.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.642+0000 7f3c901fc740 1 -- 172.21.15.83:0/2932572182 >> 172.21.15.83:0/2932572182 conn(0x5612e02613b0 msgr2=0x5612e0260510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:34.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.642+0000 7f3c67e00640 0 client.21243 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:07:34.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.642+0000 7f3c901fc740 1 -- 172.21.15.83:0/2932572182 shutdown_connections 2024-04-24T09:07:34.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.642+0000 7f3c901fc740 1 -- 172.21.15.83:0/2932572182 wait complete. 2024-04-24T09:07:34.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.643+0000 7f3c901fc740 20 client.21243 trim_cache size 0 max 16384 2024-04-24T09:07:34.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.687+0000 7f3c901fc740 1 Processor -- start 2024-04-24T09:07:34.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.687+0000 7f3c901fc740 1 -- start start 2024-04-24T09:07:34.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.687+0000 7f3c901fc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5612e0265090 0x5612e027bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:34.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.687+0000 7f3c901fc740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5612e02659b0 0x5612e026c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:34.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.687+0000 7f3c901fc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612e0270080 0x5612e0260390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:34.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.687+0000 7f3c901fc740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3c881b5b50 con 0x5612e0270080 2024-04-24T09:07:34.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.687+0000 7f3c901fc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3c881b77e0 con 0x5612e0265090 2024-04-24T09:07:34.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.687+0000 7f3c901fc740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3c881b6220 con 0x5612e02659b0 2024-04-24T09:07:34.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.688+0000 7f3c86800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5612e0265090 0x5612e027bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:34.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.688+0000 7f3c85e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5612e02659b0 0x5612e026c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:34.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.688+0000 7f3c86800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5612e0265090 0x5612e027bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:43124/0 (socket says 172.21.15.83:43124) 2024-04-24T09:07:34.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.688+0000 7f3c86800640 1 -- 172.21.15.83:0/1850759639 learned_addr learned my addr 172.21.15.83:0/1850759639 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:07:34.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.689+0000 7f3c87200640 1 --2- 172.21.15.83:0/1850759639 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612e0270080 0x5612e0260390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:34.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.689+0000 7f3c86800640 1 -- 172.21.15.83:0/1850759639 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5612e02659b0 msgr2=0x5612e026c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:34.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.689+0000 7f3c86800640 1 --2- 172.21.15.83:0/1850759639 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5612e02659b0 0x5612e026c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:34.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.689+0000 7f3c86800640 1 -- 172.21.15.83:0/1850759639 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612e0270080 msgr2=0x5612e0260390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:34.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.689+0000 7f3c86800640 1 --2- 172.21.15.83:0/1850759639 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612e0270080 0x5612e0260390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:34.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.689+0000 7f3c86800640 1 -- 172.21.15.83:0/1850759639 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612e027eef0 con 0x5612e0265090 2024-04-24T09:07:34.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.690+0000 7f3c86800640 1 --2- 172.21.15.83:0/1850759639 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5612e0265090 0x5612e027bd00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x5612e0266650 tx=0x7f3c78000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=135239b0a94a53fd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:34.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.691+0000 7f3c72a00640 1 -- 172.21.15.83:0/1850759639 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c78035040 con 0x5612e0265090 2024-04-24T09:07:34.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.691+0000 7f3c72a00640 1 -- 172.21.15.83:0/1850759639 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3c780316a0 con 0x5612e0265090 2024-04-24T09:07:34.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.691+0000 7f3c72a00640 1 -- 172.21.15.83:0/1850759639 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c7803f040 con 0x5612e0265090 2024-04-24T09:07:34.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.693+0000 7f3c901fc740 1 -- 172.21.15.83:0/1850759639 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5612e0265090 msgr2=0x5612e027bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:34.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.693+0000 7f3c901fc740 1 --2- 172.21.15.83:0/1850759639 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5612e0265090 0x5612e027bd00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x5612e0266650 tx=0x7f3c78000bc0 comp rx=0 tx=0).stop 2024-04-24T09:07:34.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.694+0000 7f3c901fc740 1 -- 172.21.15.83:0/1850759639 shutdown_connections 2024-04-24T09:07:34.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.694+0000 7f3c901fc740 1 --2- 172.21.15.83:0/1850759639 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612e0270080 0x5612e0260390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:34.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.694+0000 7f3c901fc740 1 --2- 172.21.15.83:0/1850759639 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5612e02659b0 0x5612e026c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:34.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.694+0000 7f3c901fc740 1 --2- 172.21.15.83:0/1850759639 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5612e0265090 0x5612e027bd00 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:34.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.694+0000 7f3c901fc740 1 -- 172.21.15.83:0/1850759639 >> 172.21.15.83:0/1850759639 conn(0x5612e02613b0 msgr2=0x5612e025fc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:34.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.694+0000 7f3c901fc740 1 -- 172.21.15.83:0/1850759639 shutdown_connections 2024-04-24T09:07:34.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.694+0000 7f3c901fc740 1 -- 172.21.15.83:0/1850759639 wait complete. 2024-04-24T09:07:34.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.694+0000 7f3c901fc740 1 -- start start 2024-04-24T09:07:34.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.695+0000 7f3c901fc740 1 Processor -- start 2024-04-24T09:07:34.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.696+0000 7f3c901fc740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5612e0265090 0x5612e0274000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:34.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.696+0000 7f3c901fc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612e02659b0 0x5612e0274540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:34.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.696+0000 7f3c901fc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5612e0270080 0x5612e02e5a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:34.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.696+0000 7f3c901fc740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3c881b7050 con 0x5612e02659b0 2024-04-24T09:07:34.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.696+0000 7f3c901fc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3c881b6950 con 0x5612e0270080 2024-04-24T09:07:34.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.696+0000 7f3c901fc740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3c881b3b50 con 0x5612e0265090 2024-04-24T09:07:34.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.696+0000 7f3c901fc740 10 client.0 _ll_register_callbacks cb 0x7f3c7400ce60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:07:34.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.697+0000 7f3c86800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5612e0265090 0x5612e0274000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:34.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.697+0000 7f3c86800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5612e0265090 0x5612e0274000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:34850/0 (socket says 172.21.15.83:34850) 2024-04-24T09:07:34.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.697+0000 7f3c86800640 1 -- 172.21.15.83:0/1618523806 learned_addr learned my addr 172.21.15.83:0/1618523806 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:07:34.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.697+0000 7f3c87200640 1 --2- 172.21.15.83:0/1618523806 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5612e0270080 0x5612e02e5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:34.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.697+0000 7f3c85e00640 1 --2- 172.21.15.83:0/1618523806 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612e02659b0 0x5612e0274540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:34.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.698+0000 7f3c85e00640 1 -- 172.21.15.83:0/1618523806 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5612e0265090 msgr2=0x5612e0274000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:34.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.698+0000 7f3c85e00640 1 --2- 172.21.15.83:0/1618523806 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5612e0265090 0x5612e0274000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:34.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.698+0000 7f3c85e00640 1 -- 172.21.15.83:0/1618523806 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5612e0270080 msgr2=0x5612e02e5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:34.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.698+0000 7f3c85e00640 1 --2- 172.21.15.83:0/1618523806 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5612e0270080 0x5612e02e5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:34.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.698+0000 7f3c85e00640 1 -- 172.21.15.83:0/1618523806 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612e02f6fb0 con 0x5612e02659b0 2024-04-24T09:07:34.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.699+0000 7f3c85e00640 1 --2- 172.21.15.83:0/1618523806 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612e02659b0 0x5612e0274540 secure :-1 s=READY pgs=2266 cs=0 l=1 rev1=1 crypto rx=0x7f3c74012d20 tx=0x7f3c7400ff00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=168ba6ef7acb2882 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:34.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.699+0000 7f3c67e00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:07:34.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.699+0000 7f3c67e00640 1 -- 172.21.15.83:0/1618523806 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c740117e0 con 0x5612e02659b0 2024-04-24T09:07:34.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.699+0000 7f3c67e00640 1 -- 172.21.15.83:0/1618523806 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9583+0+0 (secure 0 0 0) 0x7f3c740105d0 con 0x5612e02659b0 2024-04-24T09:07:34.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.699+0000 7f3c901fc740 1 -- 172.21.15.83:0/1618523806 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c780316a0 con 0x5612e02659b0 2024-04-24T09:07:34.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.700+0000 7f3c67e00640 1 -- 172.21.15.83:0/1618523806 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f3c74049050 con 0x5612e02659b0 2024-04-24T09:07:34.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.700+0000 7f3c901fc740 10 client.21249 fetch_fsmap learned FSMap version 51 2024-04-24T09:07:34.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.700+0000 7f3c901fc740 1 -- 172.21.15.83:0/1618523806 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x5612e02f8180 con 0x5612e02659b0 2024-04-24T09:07:34.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.700+0000 7f3c67e00640 1 -- 172.21.15.83:0/1618523806 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3c780316a0 con 0x5612e02659b0 2024-04-24T09:07:34.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.700+0000 7f3c67e00640 1 -- 172.21.15.83:0/1618523806 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 51) ==== 8406+0+0 (secure 0 0 0) 0x7f3c74035410 con 0x5612e02659b0 2024-04-24T09:07:34.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.701+0000 7f3c901fc740 10 client.21249 fetch_fsmap finished waiting for FSMap version 51 2024-04-24T09:07:34.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.701+0000 7f3c901fc740 10 client.21249 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:07:34.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.701+0000 7f3c901fc740 20 client.21249 populate_metadata read hostname 'smithi083' 2024-04-24T09:07:34.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.701+0000 7f3c901fc740 1 --2- 172.21.15.83:0/1618523806 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5612e02f89a0 0x5612e02fadc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:34.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.701+0000 7f3c901fc740 4 client.21249 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T09:07:34.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.701+0000 7f3c901fc740 1 -- 172.21.15.83:0/1618523806 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5612e01d2700 con 0x5612e02f89a0 2024-04-24T09:07:34.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.701+0000 7f3c86800640 1 --2- 172.21.15.83:0/1618523806 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5612e02f89a0 0x5612e02fadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:34.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.702+0000 7f3c86800640 1 --2- 172.21.15.83:0/1618523806 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5612e02f89a0 0x5612e02fadc0 crc :-1 s=READY pgs=408 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b9637ef380dd2cf server_cookie=ae1ed20f1a692521 in_seq=0 out_seq=0 2024-04-24T09:07:34.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.702+0000 7f3c67e00640 10 client.21249 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:07:34.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.704+0000 7f3c67e00640 1 -- 172.21.15.83:0/1618523806 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5612e01d2700 con 0x5612e02f89a0 2024-04-24T09:07:34.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.704+0000 7f3c67e00640 10 client.21249 handle_command_reply: tid=0 2024-04-24T09:07:34.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.704+0000 7f3c901fc740 1 client.21249 shutdown 2024-04-24T09:07:34.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.705+0000 7f3c901fc740 1 -- 172.21.15.83:0/1618523806 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612e02659b0 msgr2=0x5612e0274540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:34.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.705+0000 7f3c901fc740 1 --2- 172.21.15.83:0/1618523806 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612e02659b0 0x5612e0274540 secure :-1 s=READY pgs=2266 cs=0 l=1 rev1=1 crypto rx=0x7f3c74012d20 tx=0x7f3c7400ff00 comp rx=0 tx=0).stop 2024-04-24T09:07:34.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.705+0000 7f3c901fc740 1 -- 172.21.15.83:0/1618523806 shutdown_connections 2024-04-24T09:07:34.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.705+0000 7f3c901fc740 1 --2- 172.21.15.83:0/1618523806 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5612e02f89a0 0x5612e02fadc0 crc :-1 s=READY pgs=408 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:34.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.705+0000 7f3c901fc740 1 --2- 172.21.15.83:0/1618523806 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5612e0270080 0x5612e02e5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:34.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.705+0000 7f3c901fc740 1 --2- 172.21.15.83:0/1618523806 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612e02659b0 0x5612e0274540 unknown :-1 s=CLOSED pgs=2266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:34.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.705+0000 7f3c901fc740 1 --2- 172.21.15.83:0/1618523806 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5612e0265090 0x5612e0274000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:34.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.705+0000 7f3c901fc740 1 -- 172.21.15.83:0/1618523806 >> 172.21.15.83:0/1618523806 conn(0x5612e02613b0 msgr2=0x5612e0260510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:34.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.705+0000 7f3c67e00640 0 client.21249 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:07:34.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.705+0000 7f3c901fc740 1 -- 172.21.15.83:0/1618523806 shutdown_connections 2024-04-24T09:07:34.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.705+0000 7f3c901fc740 1 -- 172.21.15.83:0/1618523806 wait complete. 2024-04-24T09:07:34.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.705+0000 7f3c901fc740 20 client.21249 trim_cache size 0 max 16384 2024-04-24T09:07:34.706 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T09:07:34.706 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-04-24T09:07:34.707 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "e6be1d0f-9778-4e3b-8bb5-23235cddd735", 2024-04-24T09:07:34.707 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-04-24T09:07:34.707 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T09:07:34.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.706+0000 7f3c8d400640 1 -- 172.21.15.83:0/4094457495 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f3c6c0a3d20 msgr2=0x7f3c6c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:34.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.706+0000 7f3c8d400640 1 --2- 172.21.15.83:0/4094457495 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f3c6c0a3d20 0x7f3c6c0a61e0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x5612e0262540 tx=0x7f3c740046c0 comp rx=0 tx=0).stop 2024-04-24T09:07:34.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.706+0000 7f3c8d400640 1 -- 172.21.15.83:0/4094457495 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c8810e660 msgr2=0x7f3c881a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:34.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.706+0000 7f3c8d400640 1 --2- 172.21.15.83:0/4094457495 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c8810e660 0x7f3c881a3650 secure :-1 s=READY pgs=2263 cs=0 l=1 rev1=1 crypto rx=0x7f3c8000cce0 tx=0x7f3c8000a620 comp rx=0 tx=0).stop 2024-04-24T09:07:34.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.706+0000 7f3c8d400640 1 -- 172.21.15.83:0/4094457495 shutdown_connections 2024-04-24T09:07:34.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.706+0000 7f3c8d400640 1 --2- 172.21.15.83:0/4094457495 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f3c6c0a3d20 0x7f3c6c0a61e0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:34.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.706+0000 7f3c8d400640 1 --2- 172.21.15.83:0/4094457495 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c8810e660 0x7f3c881a3650 unknown :-1 s=CLOSED pgs=2263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:34.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.706+0000 7f3c8d400640 1 --2- 172.21.15.83:0/4094457495 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3c88108fd0 0x7f3c8819f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:34.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.706+0000 7f3c8d400640 1 --2- 172.21.15.83:0/4094457495 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c88108610 0x7f3c8819ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:34.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.706+0000 7f3c8d400640 1 -- 172.21.15.83:0/4094457495 >> 172.21.15.83:0/4094457495 conn(0x7f3c88076540 msgr2=0x7f3c88077cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:34.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.707+0000 7f3c8d400640 1 -- 172.21.15.83:0/4094457495 shutdown_connections 2024-04-24T09:07:34.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.707+0000 7f3c8d400640 1 -- 172.21.15.83:0/4094457495 wait complete. 2024-04-24T09:07:34.727 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T09:07:34.727 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T09:07:34.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.965+0000 7f8182000640 1 -- 172.21.15.83:0/2968966649 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f817c103980 msgr2=0x7f817c10be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:34.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.965+0000 7f8182000640 1 --2- 172.21.15.83:0/2968966649 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f817c103980 0x7f817c10be60 secure :-1 s=READY pgs=2267 cs=0 l=1 rev1=1 crypto rx=0x7f8168009900 tx=0x7f816802a480 comp rx=0 tx=0).stop 2024-04-24T09:07:34.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.966+0000 7f8182000640 1 -- 172.21.15.83:0/2968966649 shutdown_connections 2024-04-24T09:07:34.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.966+0000 7f8182000640 1 --2- 172.21.15.83:0/2968966649 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f817c10c480 0x7f817c10e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:34.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.966+0000 7f8182000640 1 --2- 172.21.15.83:0/2968966649 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f817c103980 0x7f817c10be60 unknown :-1 s=CLOSED pgs=2267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:34.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.966+0000 7f8182000640 1 --2- 172.21.15.83:0/2968966649 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f817c103060 0x7f817c103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:34.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.966+0000 7f8182000640 1 -- 172.21.15.83:0/2968966649 >> 172.21.15.83:0/2968966649 conn(0x7f817c0fc880 msgr2=0x7f817c0feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:34.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.966+0000 7f8182000640 1 -- 172.21.15.83:0/2968966649 shutdown_connections 2024-04-24T09:07:34.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.966+0000 7f8182000640 1 -- 172.21.15.83:0/2968966649 wait complete. 2024-04-24T09:07:34.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.967+0000 7f8182000640 1 Processor -- start 2024-04-24T09:07:34.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.967+0000 7f8182000640 1 -- start start 2024-04-24T09:07:34.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.968+0000 7f8182000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f817c103060 0x7f817c1a38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:34.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.968+0000 7f8182000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f817c103980 0x7f817c1a3e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:34.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.968+0000 7f8182000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f817c10c480 0x7f817c10b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:34.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.968+0000 7f8182000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f817c112160 con 0x7f817c103980 2024-04-24T09:07:34.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.968+0000 7f8182000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f817c111fe0 con 0x7f817c10c480 2024-04-24T09:07:34.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.968+0000 7f8182000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f817c1122e0 con 0x7f817c103060 2024-04-24T09:07:34.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.968+0000 7f8181600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f817c10c480 0x7f817c10b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:34.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.968+0000 7f8181600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f817c10c480 0x7f817c10b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:43170/0 (socket says 172.21.15.83:43170) 2024-04-24T09:07:34.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.968+0000 7f8181600640 1 -- 172.21.15.83:0/3968334835 learned_addr learned my addr 172.21.15.83:0/3968334835 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:07:34.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.968+0000 7f8180c00640 1 --2- 172.21.15.83:0/3968334835 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f817c103060 0x7f817c1a38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:34.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.969+0000 7f8181600640 1 -- 172.21.15.83:0/3968334835 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f817c103060 msgr2=0x7f817c1a38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:34.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.969+0000 7f8181600640 1 --2- 172.21.15.83:0/3968334835 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f817c103060 0x7f817c1a38f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:34.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.969+0000 7f8181600640 1 -- 172.21.15.83:0/3968334835 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f817c103980 msgr2=0x7f817c1a3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:34.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.969+0000 7f8181600640 1 --2- 172.21.15.83:0/3968334835 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f817c103980 0x7f817c1a3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:34.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.969+0000 7f8181600640 1 -- 172.21.15.83:0/3968334835 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f817c10b920 con 0x7f817c10c480 2024-04-24T09:07:34.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.969+0000 7f8181600640 1 --2- 172.21.15.83:0/3968334835 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f817c10c480 0x7f817c10b1a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f817400eb10 tx=0x7f817400c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:34.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.970+0000 7f8179600640 1 -- 172.21.15.83:0/3968334835 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8174004b20 con 0x7f817c10c480 2024-04-24T09:07:34.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.970+0000 7f8179600640 1 -- 172.21.15.83:0/3968334835 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8174004cc0 con 0x7f817c10c480 2024-04-24T09:07:34.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.970+0000 7f8179600640 1 -- 172.21.15.83:0/3968334835 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8174010690 con 0x7f817c10c480 2024-04-24T09:07:34.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.970+0000 7f8182000640 1 -- 172.21.15.83:0/3968334835 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f817c10bc10 con 0x7f817c10c480 2024-04-24T09:07:34.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.970+0000 7f8182000640 1 -- 172.21.15.83:0/3968334835 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f817c1ab860 con 0x7f817c10c480 2024-04-24T09:07:34.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.974+0000 7f8179600640 1 -- 172.21.15.83:0/3968334835 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8174005020 con 0x7f817c10c480 2024-04-24T09:07:34.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.975+0000 7f818640d740 1 Processor -- start 2024-04-24T09:07:34.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.976+0000 7f818640d740 1 -- start start 2024-04-24T09:07:34.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.977+0000 7f818640d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5544e3a30 0x55f5544e3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:34.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.977+0000 7f818640d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f5544e4350 0x55f5544ee4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:34.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.977+0000 7f818640d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f5544eea20 0x55f5544f0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:34.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.977+0000 7f818640d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55f5544491f0 con 0x55f5544e3a30 2024-04-24T09:07:34.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.977+0000 7f818640d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f5543ef350 con 0x55f5544e4350 2024-04-24T09:07:34.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.977+0000 7f818640d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55f5544451b0 con 0x55f5544eea20 2024-04-24T09:07:34.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.977+0000 7f8180c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f5544e4350 0x55f5544ee4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:34.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.978+0000 7f8180c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f5544e4350 0x55f5544ee4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:43176/0 (socket says 172.21.15.83:43176) 2024-04-24T09:07:34.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.978+0000 7f8180c00640 1 -- 172.21.15.83:0/1382776533 learned_addr learned my addr 172.21.15.83:0/1382776533 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:07:34.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.978+0000 7f8179600640 1 --2- 172.21.15.83:0/3968334835 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f81640a3da0 0x7f81640a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:34.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.978+0000 7f8180c00640 1 -- 172.21.15.83:0/1382776533 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f5544eea20 msgr2=0x55f5544f0e10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T09:07:34.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.978+0000 7f8180c00640 1 --2- 172.21.15.83:0/1382776533 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f5544eea20 0x55f5544f0e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:34.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.978+0000 7f8180c00640 1 -- 172.21.15.83:0/1382776533 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5544e3a30 msgr2=0x55f5544e3e10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T09:07:34.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.979+0000 7f817be00640 1 --2- 172.21.15.83:0/1382776533 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5544e3a30 0x55f5544e3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:34.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.979+0000 7f8180c00640 1 --2- 172.21.15.83:0/1382776533 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5544e3a30 0x55f5544e3e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:34.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.979+0000 7f8180c00640 1 -- 172.21.15.83:0/1382776533 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f554443f50 con 0x55f5544e4350 2024-04-24T09:07:34.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.980+0000 7f817be00640 1 --2- 172.21.15.83:0/1382776533 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5544e3a30 0x55f5544e3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:07:34.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.980+0000 7f8180c00640 1 --2- 172.21.15.83:0/1382776533 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f5544e4350 0x55f5544ee4e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f817000bc90 tx=0x7f81700107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=287a83d6e4e1cdf8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:34.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.980+0000 7f8179600640 1 -- 172.21.15.83:0/3968334835 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 9583+0+0 (secure 0 0 0) 0x7f81740d1330 con 0x7f817c10c480 2024-04-24T09:07:34.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.980+0000 7f8181600640 1 --2- 172.21.15.83:0/3968334835 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f81640a3da0 0x7f81640a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:34.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.981+0000 7f816dc00640 1 -- 172.21.15.83:0/1382776533 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8170005220 con 0x55f5544e4350 2024-04-24T09:07:34.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.981+0000 7f816dc00640 1 -- 172.21.15.83:0/1382776533 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f817000be40 con 0x55f5544e4350 2024-04-24T09:07:34.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.981+0000 7f816dc00640 1 -- 172.21.15.83:0/1382776533 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8170011770 con 0x55f5544e4350 2024-04-24T09:07:34.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.982+0000 7f818640d740 1 -- 172.21.15.83:0/1382776533 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f5544e4350 msgr2=0x55f5544ee4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:34.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.982+0000 7f818640d740 1 --2- 172.21.15.83:0/1382776533 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f5544e4350 0x55f5544ee4e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f817000bc90 tx=0x7f81700107a0 comp rx=0 tx=0).stop 2024-04-24T09:07:34.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.982+0000 7f8181600640 1 --2- 172.21.15.83:0/3968334835 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f81640a3da0 0x7f81640a6260 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f81740049d0 tx=0x7f817400efd0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:34.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.982+0000 7f818640d740 1 -- 172.21.15.83:0/1382776533 shutdown_connections 2024-04-24T09:07:34.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.982+0000 7f818640d740 1 --2- 172.21.15.83:0/1382776533 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f5544eea20 0x55f5544f0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:34.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.982+0000 7f818640d740 1 --2- 172.21.15.83:0/1382776533 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f5544e4350 0x55f5544ee4e0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:34.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.982+0000 7f818640d740 1 --2- 172.21.15.83:0/1382776533 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5544e3a30 0x55f5544e3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:34.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.982+0000 7f818640d740 1 -- 172.21.15.83:0/1382776533 >> 172.21.15.83:0/1382776533 conn(0x55f5544dfd50 msgr2=0x55f5544e0130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:34.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.983+0000 7f818640d740 1 -- 172.21.15.83:0/1382776533 shutdown_connections 2024-04-24T09:07:34.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.983+0000 7f818640d740 1 -- 172.21.15.83:0/1382776533 wait complete. 2024-04-24T09:07:34.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.983+0000 7f818640d740 1 -- start start 2024-04-24T09:07:34.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.984+0000 7f818640d740 1 Processor -- start 2024-04-24T09:07:34.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.984+0000 7f818640d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5544e3a30 0x55f5544eba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:34.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.984+0000 7f818640d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f5544e4350 0x55f5544ebfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:34.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.984+0000 7f818640d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f5544eea20 0x55f5544ec500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:34.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.984+0000 7f818640d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55f5544f1810 con 0x55f5544e3a30 2024-04-24T09:07:34.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.984+0000 7f818640d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f554402690 con 0x55f5544eea20 2024-04-24T09:07:34.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.984+0000 7f818640d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55f5544f1990 con 0x55f5544e4350 2024-04-24T09:07:34.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.984+0000 7f818640d740 10 client.0 _ll_register_callbacks cb 0x55f55443e650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:07:34.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.985+0000 7f8180c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f5544e4350 0x55f5544ebfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:34.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.985+0000 7f8180c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f5544e4350 0x55f5544ebfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:34878/0 (socket says 172.21.15.83:34878) 2024-04-24T09:07:34.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.985+0000 7f8180c00640 1 -- 172.21.15.83:0/493889791 learned_addr learned my addr 172.21.15.83:0/493889791 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:07:34.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.985+0000 7f817be00640 1 --2- 172.21.15.83:0/493889791 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5544e3a30 0x55f5544eba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:34.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.985+0000 7f8180c00640 1 -- 172.21.15.83:0/493889791 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f5544eea20 msgr2=0x55f5544ec500 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T09:07:34.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.985+0000 7f8180c00640 1 --2- 172.21.15.83:0/493889791 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f5544eea20 0x55f5544ec500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:34.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.985+0000 7f8180c00640 1 -- 172.21.15.83:0/493889791 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5544e3a30 msgr2=0x55f5544eba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:34.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.985+0000 7f8180c00640 1 --2- 172.21.15.83:0/493889791 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5544e3a30 0x55f5544eba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:34.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.985+0000 7f8180c00640 1 -- 172.21.15.83:0/493889791 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f554401920 con 0x55f5544e4350 2024-04-24T09:07:34.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.985+0000 7f817be00640 1 --2- 172.21.15.83:0/493889791 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5544e3a30 0x55f5544eba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:07:34.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.986+0000 7f8180c00640 1 --2- 172.21.15.83:0/493889791 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f5544e4350 0x55f5544ebfc0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8170005890 tx=0x7f8170011820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73e9e881b3997ab9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:34.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.986+0000 7f8162000640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-04-24T09:07:34.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.986+0000 7f8162000640 1 -- 172.21.15.83:0/493889791 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f817002a020 con 0x55f5544e4350 2024-04-24T09:07:34.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.986+0000 7f818640d740 1 -- 172.21.15.83:0/493889791 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f554456c10 con 0x55f5544e4350 2024-04-24T09:07:34.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.986+0000 7f8162000640 1 -- 172.21.15.83:0/493889791 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9583+0+0 (secure 0 0 0) 0x7f8170004020 con 0x55f5544e4350 2024-04-24T09:07:34.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.987+0000 7f8162000640 1 -- 172.21.15.83:0/493889791 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f8170032050 con 0x55f5544e4350 2024-04-24T09:07:34.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.987+0000 7f818640d740 10 client.28453 fetch_fsmap learned FSMap version 51 2024-04-24T09:07:34.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.987+0000 7f818640d740 1 -- 172.21.15.83:0/493889791 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x55f5542257d0 con 0x55f5544e4350 2024-04-24T09:07:34.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.987+0000 7f8162000640 1 -- 172.21.15.83:0/493889791 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55f554456c10 con 0x55f5544e4350 2024-04-24T09:07:34.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.988+0000 7f8162000640 1 -- 172.21.15.83:0/493889791 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 51) ==== 8406+0+0 (secure 0 0 0) 0x7f817001b430 con 0x55f5544e4350 2024-04-24T09:07:34.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.988+0000 7f818640d740 10 client.28453 fetch_fsmap finished waiting for FSMap version 51 2024-04-24T09:07:34.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.988+0000 7f818640d740 10 client.28453 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:07:34.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.988+0000 7f818640d740 20 client.28453 populate_metadata read hostname 'smithi083' 2024-04-24T09:07:34.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.988+0000 7f818640d740 1 --2- 172.21.15.83:0/493889791 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55f5544fed10 0x55f554501100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:34.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.988+0000 7f818640d740 4 client.28453 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:07:34.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.988+0000 7f818640d740 1 -- 172.21.15.83:0/493889791 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f554450a80 con 0x55f5544fed10 2024-04-24T09:07:34.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.988+0000 7f817be00640 1 --2- 172.21.15.83:0/493889791 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55f5544fed10 0x55f554501100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:34.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.989+0000 7f817be00640 1 --2- 172.21.15.83:0/493889791 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55f5544fed10 0x55f554501100 crc :-1 s=READY pgs=409 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86f189219b001e25 server_cookie=1c6aceda5eb9f976 in_seq=0 out_seq=0 2024-04-24T09:07:34.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.989+0000 7f8162000640 10 client.28453 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:07:34.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.990+0000 7f8162000640 1 -- 172.21.15.83:0/493889791 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f554450a80 con 0x55f5544fed10 2024-04-24T09:07:34.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.990+0000 7f8162000640 10 client.28453 handle_command_reply: tid=0 2024-04-24T09:07:34.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.990+0000 7f818640d740 1 client.28453 shutdown 2024-04-24T09:07:34.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.991+0000 7f818640d740 1 -- 172.21.15.83:0/493889791 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f5544e4350 msgr2=0x55f5544ebfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:34.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.991+0000 7f818640d740 1 --2- 172.21.15.83:0/493889791 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f5544e4350 0x55f5544ebfc0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8170005890 tx=0x7f8170011820 comp rx=0 tx=0).stop 2024-04-24T09:07:34.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.991+0000 7f818640d740 1 -- 172.21.15.83:0/493889791 shutdown_connections 2024-04-24T09:07:34.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.991+0000 7f818640d740 1 --2- 172.21.15.83:0/493889791 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55f5544fed10 0x55f554501100 crc :-1 s=READY pgs=409 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:34.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.991+0000 7f818640d740 1 --2- 172.21.15.83:0/493889791 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f5544eea20 0x55f5544ec500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:34.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.991+0000 7f818640d740 1 --2- 172.21.15.83:0/493889791 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f5544e4350 0x55f5544ebfc0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:34.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.991+0000 7f818640d740 1 --2- 172.21.15.83:0/493889791 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5544e3a30 0x55f5544eba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:34.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.991+0000 7f818640d740 1 -- 172.21.15.83:0/493889791 >> 172.21.15.83:0/493889791 conn(0x55f5544dfd50 msgr2=0x55f5544defd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:34.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.991+0000 7f8162000640 0 client.28453 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:07:34.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.991+0000 7f818640d740 1 -- 172.21.15.83:0/493889791 shutdown_connections 2024-04-24T09:07:34.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.991+0000 7f818640d740 1 -- 172.21.15.83:0/493889791 wait complete. 2024-04-24T09:07:34.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:34.991+0000 7f818640d740 20 client.28453 trim_cache size 0 max 16384 2024-04-24T09:07:35.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:35.023+0000 7f818640d740 1 Processor -- start 2024-04-24T09:07:35.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:35.023+0000 7f818640d740 1 -- start start 2024-04-24T09:07:35.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:35.023+0000 7f818640d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f5544e3a30 0x55f5544fa700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:35.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:35.023+0000 7f818640d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5544e4350 0x55f5545067f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:35.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:35.023+0000 7f818640d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f5544eea20 0x55f554506d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:35.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:35.023+0000 7f818640d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f817c1b5ae0 con 0x55f5544e4350 2024-04-24T09:07:35.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:35.023+0000 7f818640d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f817c1b7770 con 0x55f5544e3a30 2024-04-24T09:07:35.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:35.023+0000 7f818640d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f817c1b61b0 con 0x55f5544eea20 2024-04-24T09:07:35.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:35.024+0000 7f8180c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5544e4350 0x55f5545067f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:35.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:35.024+0000 7f817be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f5544e3a30 0x55f5544fa700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:35.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:35.024+0000 7f817be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f5544e3a30 0x55f5544fa700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:43184/0 (socket says 172.21.15.83:43184) 2024-04-24T09:07:35.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:35.024+0000 7f817be00640 1 -- 172.21.15.83:0/2597319769 learned_addr learned my addr 172.21.15.83:0/2597319769 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:07:35.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:35.024+0000 7f817be00640 1 --2- 172.21.15.83:0/2597319769 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f5544eea20 0x55f554506d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:35.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:35.024+0000 7f817be00640 1 -- 172.21.15.83:0/2597319769 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f5544eea20 msgr2=0x55f554506d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:35.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:35.024+0000 7f817be00640 1 --2- 172.21.15.83:0/2597319769 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f5544eea20 0x55f554506d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:35.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:35.024+0000 7f817be00640 1 -- 172.21.15.83:0/2597319769 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5544e4350 msgr2=0x55f5545067f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:35.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:35.024+0000 7f817be00640 1 --2- 172.21.15.83:0/2597319769 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5544e4350 0x55f5545067f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:35.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:35.024+0000 7f817be00640 1 -- 172.21.15.83:0/2597319769 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f55415b3f0 con 0x55f5544e3a30 2024-04-24T09:07:35.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:35.024+0000 7f817be00640 1 --2- 172.21.15.83:0/2597319769 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f5544e3a30 0x55f5544fa700 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f81680092a0 tx=0x7f8168000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3be6b6df6045fbd9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:35.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:35.025+0000 7f816d200640 1 -- 172.21.15.83:0/2597319769 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8168035040 con 0x55f5544e3a30 2024-04-24T09:07:35.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:35.025+0000 7f816d200640 1 -- 172.21.15.83:0/2597319769 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f81680046c0 con 0x55f5544e3a30 2024-04-24T09:07:35.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:35.025+0000 7f816d200640 1 -- 172.21.15.83:0/2597319769 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8168003850 con 0x55f5544e3a30 2024-04-24T09:07:35.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:35.026+0000 7f818640d740 1 -- 172.21.15.83:0/2597319769 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f5544e3a30 msgr2=0x55f5544fa700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:35.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:35.026+0000 7f818640d740 1 --2- 172.21.15.83:0/2597319769 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f5544e3a30 0x55f5544fa700 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f81680092a0 tx=0x7f8168000bc0 comp rx=0 tx=0).stop 2024-04-24T09:07:35.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:35.026+0000 7f818640d740 1 -- 172.21.15.83:0/2597319769 shutdown_connections 2024-04-24T09:07:35.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:35.026+0000 7f818640d740 1 --2- 172.21.15.83:0/2597319769 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f5544eea20 0x55f554506d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:35.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:35.026+0000 7f818640d740 1 --2- 172.21.15.83:0/2597319769 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5544e4350 0x55f5545067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:35.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:35.026+0000 7f818640d740 1 --2- 172.21.15.83:0/2597319769 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f5544e3a30 0x55f5544fa700 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:35.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:35.026+0000 7f818640d740 1 -- 172.21.15.83:0/2597319769 >> 172.21.15.83:0/2597319769 conn(0x55f5544dfd50 msgr2=0x55f5544defd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:35.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:35.026+0000 7f818640d740 1 -- 172.21.15.83:0/2597319769 shutdown_connections 2024-04-24T09:07:35.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:35.026+0000 7f818640d740 1 -- 172.21.15.83:0/2597319769 wait complete. 2024-04-24T09:07:35.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:35.027+0000 7f818640d740 1 -- start start 2024-04-24T09:07:35.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:35.027+0000 7f818640d740 1 Processor -- start 2024-04-24T09:07:35.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:35.028+0000 7f818640d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f5544e3a30 0x55f5544f1e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:35.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:35.028+0000 7f818640d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f5544e4350 0x55f5544f2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:35.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:35.028+0000 7f818640d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5544eea20 0x55f554563050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:35.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:35.028+0000 7f818640d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f817c1b6fe0 con 0x55f5544eea20 2024-04-24T09:07:35.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:35.028+0000 7f818640d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f817c1b68e0 con 0x55f5544e3a30 2024-04-24T09:07:35.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:35.028+0000 7f818640d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f817c1b3ae0 con 0x55f5544e4350 2024-04-24T09:07:35.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:35.028+0000 7f818640d740 10 client.0 _ll_register_callbacks cb 0x7f817000bae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:07:35.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:35.028+0000 7f817be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f5544e3a30 0x55f5544f1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:35.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:35.028+0000 7f817be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f5544e3a30 0x55f5544f1e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:43192/0 (socket says 172.21.15.83:43192) 2024-04-24T09:07:35.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:35.028+0000 7f817be00640 1 -- 172.21.15.83:0/735987870 learned_addr learned my addr 172.21.15.83:0/735987870 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:07:35.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:35.028+0000 7f817be00640 1 --2- 172.21.15.83:0/735987870 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5544eea20 0x55f554563050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:35.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:35.028+0000 7f8180c00640 1 --2- 172.21.15.83:0/735987870 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f5544e4350 0x55f5544f2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:35.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:35.029+0000 7f817be00640 1 -- 172.21.15.83:0/735987870 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f5544e4350 msgr2=0x55f5544f2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:35.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:35.029+0000 7f817be00640 1 --2- 172.21.15.83:0/735987870 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f5544e4350 0x55f5544f2340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:35.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:35.029+0000 7f817be00640 1 -- 172.21.15.83:0/735987870 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5544eea20 msgr2=0x55f554563050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:35.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:35.029+0000 7f817be00640 1 --2- 172.21.15.83:0/735987870 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5544eea20 0x55f554563050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:35.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:35.029+0000 7f817be00640 1 -- 172.21.15.83:0/735987870 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f55411b0f0 con 0x55f5544e3a30 2024-04-24T09:07:35.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:35.029+0000 7f817be00640 1 --2- 172.21.15.83:0/735987870 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f5544e3a30 0x55f5544f1e00 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8168003920 tx=0x7f816803d040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9455c659c54dc7b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:35.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:35.029+0000 7f8162000640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:07:35.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:35.029+0000 7f8162000640 1 -- 172.21.15.83:0/735987870 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8168035070 con 0x55f5544e3a30 2024-04-24T09:07:35.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:35.029+0000 7f818640d740 1 -- 172.21.15.83:0/735987870 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81680046c0 con 0x55f5544e3a30 2024-04-24T09:07:35.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:35.029+0000 7f8162000640 1 -- 172.21.15.83:0/735987870 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9583+0+0 (secure 0 0 0) 0x7f8168003c40 con 0x55f5544e3a30 2024-04-24T09:07:35.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:35.030+0000 7f8162000640 1 -- 172.21.15.83:0/735987870 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f8168003e40 con 0x55f5544e3a30 2024-04-24T09:07:35.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:35.030+0000 7f818640d740 10 client.38408 fetch_fsmap learned FSMap version 51 2024-04-24T09:07:35.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:35.030+0000 7f818640d740 1 -- 172.21.15.83:0/735987870 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x55f55440e810 con 0x55f5544e3a30 2024-04-24T09:07:35.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:35.030+0000 7f8162000640 1 -- 172.21.15.83:0/735987870 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f81680046c0 con 0x55f5544e3a30 2024-04-24T09:07:35.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:35.031+0000 7f8162000640 1 -- 172.21.15.83:0/735987870 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 51) ==== 8406+0+0 (secure 0 0 0) 0x7f8168005450 con 0x55f5544e3a30 2024-04-24T09:07:35.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:35.031+0000 7f818640d740 10 client.38408 fetch_fsmap finished waiting for FSMap version 51 2024-04-24T09:07:35.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:35.031+0000 7f818640d740 10 client.38408 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:07:35.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:35.031+0000 7f818640d740 20 client.38408 populate_metadata read hostname 'smithi083' 2024-04-24T09:07:35.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:35.031+0000 7f818640d740 1 --2- 172.21.15.83:0/735987870 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55f554564cd0 0x55f5545670f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:35.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:35.031+0000 7f818640d740 4 client.38408 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T09:07:35.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:35.031+0000 7f818640d740 1 -- 172.21.15.83:0/735987870 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f5543cc360 con 0x55f554564cd0 2024-04-24T09:07:35.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:35.031+0000 7f8180c00640 1 --2- 172.21.15.83:0/735987870 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55f554564cd0 0x55f5545670f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:35.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:35.032+0000 7f8180c00640 1 --2- 172.21.15.83:0/735987870 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55f554564cd0 0x55f5545670f0 crc :-1 s=READY pgs=410 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6b0f42bdc1c230 server_cookie=6d8a5dd778061f66 in_seq=0 out_seq=0 2024-04-24T09:07:35.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:35.032+0000 7f8162000640 10 client.38408 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:07:35.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:35.033+0000 7f8162000640 1 -- 172.21.15.83:0/735987870 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55f5543cc360 con 0x55f554564cd0 2024-04-24T09:07:35.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:35.033+0000 7f8162000640 10 client.38408 handle_command_reply: tid=0 2024-04-24T09:07:35.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:35.033+0000 7f818640d740 1 client.38408 shutdown 2024-04-24T09:07:35.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:35.033+0000 7f818640d740 1 -- 172.21.15.83:0/735987870 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f5544e3a30 msgr2=0x55f5544f1e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:35.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:35.033+0000 7f818640d740 1 --2- 172.21.15.83:0/735987870 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f5544e3a30 0x55f5544f1e00 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8168003920 tx=0x7f816803d040 comp rx=0 tx=0).stop 2024-04-24T09:07:35.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:35.033+0000 7f818640d740 1 -- 172.21.15.83:0/735987870 shutdown_connections 2024-04-24T09:07:35.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:35.033+0000 7f818640d740 1 --2- 172.21.15.83:0/735987870 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55f554564cd0 0x55f5545670f0 crc :-1 s=READY pgs=410 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:35.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:35.033+0000 7f818640d740 1 --2- 172.21.15.83:0/735987870 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5544eea20 0x55f554563050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:35.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:35.033+0000 7f818640d740 1 --2- 172.21.15.83:0/735987870 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f5544e4350 0x55f5544f2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:35.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:35.033+0000 7f818640d740 1 --2- 172.21.15.83:0/735987870 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f5544e3a30 0x55f5544f1e00 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:35.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:35.033+0000 7f818640d740 1 -- 172.21.15.83:0/735987870 >> 172.21.15.83:0/735987870 conn(0x55f5544dfd50 msgr2=0x55f5544eba40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:35.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:35.033+0000 7f8162000640 0 client.38408 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:07:35.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:35.034+0000 7f818640d740 1 -- 172.21.15.83:0/735987870 shutdown_connections 2024-04-24T09:07:35.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:35.034+0000 7f818640d740 1 -- 172.21.15.83:0/735987870 wait complete. 2024-04-24T09:07:35.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:35.034+0000 7f818640d740 20 client.38408 trim_cache size 0 max 16384 2024-04-24T09:07:35.036 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T09:07:35.036 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T09:07:35.036 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-04-24T09:07:35.036 INFO:teuthology.orchestra.run.smithi083.stdout: "e6be1d0f-9778-4e3b-8bb5-23235cddd735": { 2024-04-24T09:07:35.036 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-04-24T09:07:35.036 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "e6be1d0f-9778-4e3b-8bb5-23235cddd735", 2024-04-24T09:07:35.036 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-04-24T09:07:35.036 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:07:35.036 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:07:35.036 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T09:07:35.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:35.035+0000 7f8182000640 1 -- 172.21.15.83:0/3968334835 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f81640a3da0 msgr2=0x7f81640a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:35.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:35.035+0000 7f8182000640 1 --2- 172.21.15.83:0/3968334835 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f81640a3da0 0x7f81640a6260 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f81740049d0 tx=0x7f817400efd0 comp rx=0 tx=0).stop 2024-04-24T09:07:35.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:35.035+0000 7f8182000640 1 -- 172.21.15.83:0/3968334835 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f817c10c480 msgr2=0x7f817c10b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:35.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:35.035+0000 7f8182000640 1 --2- 172.21.15.83:0/3968334835 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f817c10c480 0x7f817c10b1a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f817400eb10 tx=0x7f817400c2d0 comp rx=0 tx=0).stop 2024-04-24T09:07:35.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:35.035+0000 7f8182000640 1 -- 172.21.15.83:0/3968334835 shutdown_connections 2024-04-24T09:07:35.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:35.035+0000 7f8182000640 1 --2- 172.21.15.83:0/3968334835 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f81640a3da0 0x7f81640a6260 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:35.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:35.035+0000 7f8182000640 1 --2- 172.21.15.83:0/3968334835 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f817c10c480 0x7f817c10b1a0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:35.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:35.035+0000 7f8182000640 1 --2- 172.21.15.83:0/3968334835 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f817c103980 0x7f817c1a3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:35.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:35.035+0000 7f8182000640 1 --2- 172.21.15.83:0/3968334835 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f817c103060 0x7f817c1a38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:35.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:35.035+0000 7f8182000640 1 -- 172.21.15.83:0/3968334835 >> 172.21.15.83:0/3968334835 conn(0x7f817c0fc880 msgr2=0x7f817c101de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:35.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:35.035+0000 7f8182000640 1 -- 172.21.15.83:0/3968334835 shutdown_connections 2024-04-24T09:07:35.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:07:35.035+0000 7f8182000640 1 -- 172.21.15.83:0/3968334835 wait complete. 2024-04-24T09:07:35.057 INFO:tasks.cephfs.filesystem:scrub status for tag:e6be1d0f-9778-4e3b-8bb5-23235cddd735 - {'path': '/', 'tag': 'e6be1d0f-9778-4e3b-8bb5-23235cddd735', 'options': 'recursive,force'} 2024-04-24T09:07:35.162 INFO:tasks.workunit.client.0.smithi083.stdout: 10 16398 10.44 MB/sec execute 6 sec latency 322.083 ms 2024-04-24T09:07:35.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:07:34 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:07:35.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:07:34 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:07:35.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:07:34 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:07:35.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:07:34 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:07:35.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:07:34 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:07:35.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:07:34 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:07:36.162 INFO:tasks.workunit.client.0.smithi083.stdout: 10 16486 9.27 MB/sec execute 7 sec latency 64.758 ms 2024-04-24T09:07:36.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:07:35 smithi083 ceph-mon[27907]: scrub queued for path: / 2024-04-24T09:07:36.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:07:35 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:07:36.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:07:35 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:07:36.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:07:35 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:07:36.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:07:35 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:07:36.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:07:35 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:07:36.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:07:35 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:07:36.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:07:35 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:07:36.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:07:35 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:07:36.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:07:35 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:07:36.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:07:35 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:07:36.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:07:35 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:07:36.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:07:35 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:07:36.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:07:35 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:07:36.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:07:35 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:07:36.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:07:35 smithi083 ceph-mon[27907]: pgmap v502: 129 pgs: 129 active+clean; 417 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 823 KiB/s rd, 3.2 MiB/s wr, 194 op/s 2024-04-24T09:07:36.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:07:35 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:07:36.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:07:35 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:07:36.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:07:35 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:07:36.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:07:35 smithi138 ceph-mon[31441]: scrub queued for path: / 2024-04-24T09:07:36.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:07:35 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:07:36.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:07:35 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:07:36.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:07:35 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:07:36.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:07:35 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:07:36.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:07:35 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:07:36.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:07:35 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:07:36.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:07:35 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:07:36.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:07:35 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:07:36.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:07:35 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:07:36.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:07:35 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:07:36.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:07:35 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:07:36.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:07:35 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:07:36.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:07:35 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:07:36.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:07:35 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:07:36.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:07:35 smithi138 ceph-mon[31441]: pgmap v502: 129 pgs: 129 active+clean; 417 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 823 KiB/s rd, 3.2 MiB/s wr, 194 op/s 2024-04-24T09:07:36.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:07:35 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:07:36.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:07:35 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:07:36.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:07:35 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:07:36.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:07:35 smithi189 ceph-mon[31791]: scrub queued for path: / 2024-04-24T09:07:36.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:07:35 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:07:36.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:07:35 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:07:36.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:07:35 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:07:36.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:07:35 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:07:36.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:07:35 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:07:36.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:07:35 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:07:36.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:07:35 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:07:36.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:07:35 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:07:36.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:07:35 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:07:36.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:07:35 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:07:36.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:07:35 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:07:36.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:07:35 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:07:36.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:07:35 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:07:36.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:07:35 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:07:36.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:07:35 smithi189 ceph-mon[31791]: pgmap v502: 129 pgs: 129 active+clean; 417 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 823 KiB/s rd, 3.2 MiB/s wr, 194 op/s 2024-04-24T09:07:36.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:07:35 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:07:36.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:07:35 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:07:36.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:07:35 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:07:37.162 INFO:tasks.workunit.client.0.smithi083.stdout: 10 16626 9.90 MB/sec execute 8 sec latency 64.849 ms 2024-04-24T09:07:38.162 INFO:tasks.workunit.client.0.smithi083.stdout: 10 16745 9.11 MB/sec execute 9 sec latency 80.397 ms 2024-04-24T09:07:38.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:07:37 smithi083 ceph-mon[27907]: pgmap v503: 129 pgs: 129 active+clean; 417 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 823 KiB/s rd, 3.2 MiB/s wr, 194 op/s 2024-04-24T09:07:38.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:07:37 smithi138 ceph-mon[31441]: pgmap v503: 129 pgs: 129 active+clean; 417 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 823 KiB/s rd, 3.2 MiB/s wr, 194 op/s 2024-04-24T09:07:38.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:07:37 smithi189 ceph-mon[31791]: pgmap v503: 129 pgs: 129 active+clean; 417 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 823 KiB/s rd, 3.2 MiB/s wr, 194 op/s 2024-04-24T09:07:39.162 INFO:tasks.workunit.client.0.smithi083.stdout: 10 16852 8.56 MB/sec execute 10 sec latency 58.298 ms 2024-04-24T09:07:40.163 INFO:tasks.workunit.client.0.smithi083.stdout: 10 16968 8.32 MB/sec execute 11 sec latency 78.006 ms 2024-04-24T09:07:40.256 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:07:40 smithi138 ceph-mon[31441]: pgmap v504: 129 pgs: 129 active+clean; 430 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.1 MiB/s wr, 333 op/s 2024-04-24T09:07:40.256 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:07:40 smithi138 ceph-mon[31441]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:07:40.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:07:40 smithi083 ceph-mon[27907]: pgmap v504: 129 pgs: 129 active+clean; 430 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.1 MiB/s wr, 333 op/s 2024-04-24T09:07:40.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:07:40 smithi083 ceph-mon[27907]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:07:40.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:07:40 smithi189 ceph-mon[31791]: pgmap v504: 129 pgs: 129 active+clean; 430 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.1 MiB/s wr, 333 op/s 2024-04-24T09:07:40.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:07:40 smithi189 ceph-mon[31791]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:07:41.162 INFO:tasks.workunit.client.0.smithi083.stdout: 10 17072 7.80 MB/sec execute 12 sec latency 75.161 ms 2024-04-24T09:07:42.163 INFO:tasks.workunit.client.0.smithi083.stdout: 10 17182 7.43 MB/sec execute 13 sec latency 98.677 ms 2024-04-24T09:07:42.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:07:42 smithi083 ceph-mon[27907]: pgmap v505: 129 pgs: 129 active+clean; 430 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 967 KiB/s rd, 3.8 MiB/s wr, 242 op/s 2024-04-24T09:07:42.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:07:42 smithi138 ceph-mon[31441]: pgmap v505: 129 pgs: 129 active+clean; 430 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 967 KiB/s rd, 3.8 MiB/s wr, 242 op/s 2024-04-24T09:07:42.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:07:42 smithi189 ceph-mon[31791]: pgmap v505: 129 pgs: 129 active+clean; 430 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 967 KiB/s rd, 3.8 MiB/s wr, 242 op/s 2024-04-24T09:07:43.163 INFO:tasks.workunit.client.0.smithi083.stdout: 10 17252 7.02 MB/sec execute 14 sec latency 453.021 ms 2024-04-24T09:07:44.163 INFO:tasks.workunit.client.0.smithi083.stdout: 10 17397 7.14 MB/sec execute 15 sec latency 97.507 ms 2024-04-24T09:07:44.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:07:44 smithi083 ceph-mon[27907]: pgmap v506: 129 pgs: 129 active+clean; 434 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.3 MiB/s wr, 335 op/s 2024-04-24T09:07:44.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:07:44 smithi138 ceph-mon[31441]: pgmap v506: 129 pgs: 129 active+clean; 434 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.3 MiB/s wr, 335 op/s 2024-04-24T09:07:44.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:07:44 smithi189 ceph-mon[31791]: pgmap v506: 129 pgs: 129 active+clean; 434 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.3 MiB/s wr, 335 op/s 2024-04-24T09:07:44.529 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.528+0000 7f966fa00640 1 -- 172.21.15.83:0/164552118 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9668109540 msgr2=0x7f9668111110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:44.529 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.528+0000 7f966fa00640 1 --2- 172.21.15.83:0/164552118 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9668109540 0x7f9668111110 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9660009ca0 tx=0x7f966002a8c0 comp rx=0 tx=0).stop 2024-04-24T09:07:44.529 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.528+0000 7f966fa00640 1 -- 172.21.15.83:0/164552118 shutdown_connections 2024-04-24T09:07:44.529 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.528+0000 7f966fa00640 1 --2- 172.21.15.83:0/164552118 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9668109540 0x7f9668111110 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:44.529 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.528+0000 7f966fa00640 1 --2- 172.21.15.83:0/164552118 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9668104e20 0x7f9668108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:44.529 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.529+0000 7f966fa00640 1 --2- 172.21.15.83:0/164552118 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9668104470 0x7f9668104850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:44.529 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.529+0000 7f966fa00640 1 -- 172.21.15.83:0/164552118 >> 172.21.15.83:0/164552118 conn(0x7f96680ffcd0 msgr2=0x7f96681020f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:44.529 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.529+0000 7f966fa00640 1 -- 172.21.15.83:0/164552118 shutdown_connections 2024-04-24T09:07:44.529 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.529+0000 7f966fa00640 1 -- 172.21.15.83:0/164552118 wait complete. 2024-04-24T09:07:44.530 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.530+0000 7f966fa00640 1 Processor -- start 2024-04-24T09:07:44.530 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.531+0000 7f966fa00640 1 -- start start 2024-04-24T09:07:44.531 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.531+0000 7f966fa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9668104470 0x7f9668110f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:44.531 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.531+0000 7f966fa00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9668109540 0x7f9668073990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:44.531 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.531+0000 7f966fa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f966810dff0 0x7f966810e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:44.532 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.531+0000 7f966fa00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f966806c180 con 0x7f966810dff0 2024-04-24T09:07:44.532 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.531+0000 7f966fa00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f966806c000 con 0x7f9668104470 2024-04-24T09:07:44.532 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.531+0000 7f966fa00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f966806c300 con 0x7f9668109540 2024-04-24T09:07:44.532 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.532+0000 7f966da00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f966810dff0 0x7f966810e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:44.532 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.532+0000 7f9667e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9668109540 0x7f9668073990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:44.532 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.532+0000 7f966da00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f966810dff0 0x7f966810e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35436/0 (socket says 172.21.15.83:35436) 2024-04-24T09:07:44.532 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.532+0000 7f966da00640 1 -- 172.21.15.83:0/3448071030 learned_addr learned my addr 172.21.15.83:0/3448071030 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:07:44.532 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.532+0000 7f966d000640 1 --2- 172.21.15.83:0/3448071030 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9668104470 0x7f9668110f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:44.532 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.532+0000 7f966da00640 1 -- 172.21.15.83:0/3448071030 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9668109540 msgr2=0x7f9668073990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:44.532 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.532+0000 7f966da00640 1 --2- 172.21.15.83:0/3448071030 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9668109540 0x7f9668073990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:44.533 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.532+0000 7f966da00640 1 -- 172.21.15.83:0/3448071030 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9668104470 msgr2=0x7f9668110f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:44.533 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.532+0000 7f966da00640 1 --2- 172.21.15.83:0/3448071030 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9668104470 0x7f9668110f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:44.533 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.532+0000 7f966da00640 1 -- 172.21.15.83:0/3448071030 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f966810e9e0 con 0x7f966810dff0 2024-04-24T09:07:44.533 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.532+0000 7f966d000640 1 --2- 172.21.15.83:0/3448071030 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9668104470 0x7f9668110f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:07:44.533 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.533+0000 7f966da00640 1 --2- 172.21.15.83:0/3448071030 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f966810dff0 0x7f966810e4a0 secure :-1 s=READY pgs=2268 cs=0 l=1 rev1=1 crypto rx=0x7f9660005e30 tx=0x7f96600093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:44.534 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.533+0000 7f9665600640 1 -- 172.21.15.83:0/3448071030 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9660003c70 con 0x7f966810dff0 2024-04-24T09:07:44.534 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.534+0000 7f9665600640 1 -- 172.21.15.83:0/3448071030 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9660003e10 con 0x7f966810dff0 2024-04-24T09:07:44.534 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.534+0000 7f9665600640 1 -- 172.21.15.83:0/3448071030 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9660030c10 con 0x7f966810dff0 2024-04-24T09:07:44.534 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.534+0000 7f966fa00640 1 -- 172.21.15.83:0/3448071030 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f966810ec70 con 0x7f966810dff0 2024-04-24T09:07:44.534 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.534+0000 7f966fa00640 1 -- 172.21.15.83:0/3448071030 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96681b44c0 con 0x7f966810dff0 2024-04-24T09:07:44.537 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.538+0000 7f96727e8740 1 Processor -- start 2024-04-24T09:07:44.538 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.538+0000 7f9665600640 1 -- 172.21.15.83:0/3448071030 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9660004040 con 0x7f966810dff0 2024-04-24T09:07:44.538 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.538+0000 7f96727e8740 1 -- start start 2024-04-24T09:07:44.539 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.539+0000 7f96727e8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561bd77491b0 0x561bd7749590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:44.539 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.539+0000 7f96727e8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561bd7749ad0 0x561bd7753e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:44.539 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.539+0000 7f96727e8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561bd7754390 0x561bd7756780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:44.539 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.539+0000 7f96727e8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561bd76af900 con 0x561bd77491b0 2024-04-24T09:07:44.539 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.539+0000 7f96727e8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x561bd7655f20 con 0x561bd7749ad0 2024-04-24T09:07:44.539 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.539+0000 7f96727e8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x561bd76ab8c0 con 0x561bd7754390 2024-04-24T09:07:44.541 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.539+0000 7f966d000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561bd7749ad0 0x561bd7753e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:44.541 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.539+0000 7f966d000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561bd7749ad0 0x561bd7753e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:59994/0 (socket says 172.21.15.83:59994) 2024-04-24T09:07:44.541 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.539+0000 7f966d000640 1 -- 172.21.15.83:0/1350496384 learned_addr learned my addr 172.21.15.83:0/1350496384 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:07:44.541 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.539+0000 7f9667e00640 1 --2- 172.21.15.83:0/1350496384 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561bd77491b0 0x561bd7749590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:44.541 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.540+0000 7f9667e00640 1 --2- 172.21.15.83:0/1350496384 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561bd7754390 0x561bd7756780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:44.541 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.540+0000 7f9667e00640 1 -- 172.21.15.83:0/1350496384 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561bd7754390 msgr2=0x561bd7756780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:44.541 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.540+0000 7f9667e00640 1 --2- 172.21.15.83:0/1350496384 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561bd7754390 0x561bd7756780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:44.541 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.540+0000 7f9667e00640 1 -- 172.21.15.83:0/1350496384 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561bd7749ad0 msgr2=0x561bd7753e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:44.541 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.540+0000 7f9667e00640 1 --2- 172.21.15.83:0/1350496384 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561bd7749ad0 0x561bd7753e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:44.541 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.540+0000 7f9667e00640 1 -- 172.21.15.83:0/1350496384 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561bd767ae10 con 0x561bd77491b0 2024-04-24T09:07:44.541 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.540+0000 7f9665600640 1 --2- 172.21.15.83:0/3448071030 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f96500a3e40 0x7f96500a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:44.541 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.540+0000 7f9667e00640 1 --2- 172.21.15.83:0/1350496384 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561bd77491b0 0x561bd7749590 secure :-1 s=READY pgs=2269 cs=0 l=1 rev1=1 crypto rx=0x7f9654009710 tx=0x7f96540096a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c28c57300d34ac7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:44.541 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.541+0000 7f9665600640 1 -- 172.21.15.83:0/3448071030 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 9583+0+0 (secure 0 0 0) 0x7f96600f2050 con 0x7f966810dff0 2024-04-24T09:07:44.542 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.541+0000 7f966d000640 1 --2- 172.21.15.83:0/3448071030 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f96500a3e40 0x7f96500a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:44.543 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.541+0000 7f965a000640 1 -- 172.21.15.83:0/1350496384 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9654004050 con 0x561bd77491b0 2024-04-24T09:07:44.543 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.541+0000 7f965a000640 1 -- 172.21.15.83:0/1350496384 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f96540041f0 con 0x561bd77491b0 2024-04-24T09:07:44.543 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.542+0000 7f965a000640 1 -- 172.21.15.83:0/1350496384 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96540106c0 con 0x561bd77491b0 2024-04-24T09:07:44.543 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.542+0000 7f96727e8740 1 -- 172.21.15.83:0/1350496384 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561bd77491b0 msgr2=0x561bd7749590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:44.543 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.542+0000 7f96727e8740 1 --2- 172.21.15.83:0/1350496384 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561bd77491b0 0x561bd7749590 secure :-1 s=READY pgs=2269 cs=0 l=1 rev1=1 crypto rx=0x7f9654009710 tx=0x7f96540096a0 comp rx=0 tx=0).stop 2024-04-24T09:07:44.543 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.542+0000 7f96727e8740 1 -- 172.21.15.83:0/1350496384 shutdown_connections 2024-04-24T09:07:44.543 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.542+0000 7f96727e8740 1 --2- 172.21.15.83:0/1350496384 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561bd7754390 0x561bd7756780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:44.543 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.542+0000 7f96727e8740 1 --2- 172.21.15.83:0/1350496384 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561bd7749ad0 0x561bd7753e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:44.543 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.542+0000 7f96727e8740 1 --2- 172.21.15.83:0/1350496384 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561bd77491b0 0x561bd7749590 unknown :-1 s=CLOSED pgs=2269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:44.544 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.542+0000 7f96727e8740 1 -- 172.21.15.83:0/1350496384 >> 172.21.15.83:0/1350496384 conn(0x561bd77456c0 msgr2=0x561bd7745aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:44.544 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.542+0000 7f966d000640 1 --2- 172.21.15.83:0/3448071030 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f96500a3e40 0x7f96500a6300 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x561bd774a750 tx=0x7f965c00b5e0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:44.544 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.542+0000 7f96727e8740 1 -- 172.21.15.83:0/1350496384 shutdown_connections 2024-04-24T09:07:44.544 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.542+0000 7f96727e8740 1 -- 172.21.15.83:0/1350496384 wait complete. 2024-04-24T09:07:44.544 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.543+0000 7f96727e8740 1 -- start start 2024-04-24T09:07:44.544 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.543+0000 7f96727e8740 1 Processor -- start 2024-04-24T09:07:44.546 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.543+0000 7f96727e8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561bd77491b0 0x561bd7751250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:44.546 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.543+0000 7f96727e8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561bd7749ad0 0x561bd7751790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:44.546 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.543+0000 7f96727e8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561bd7754390 0x561bd7751cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:44.546 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.543+0000 7f96727e8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561bd7756fc0 con 0x561bd77491b0 2024-04-24T09:07:44.546 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.543+0000 7f96727e8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x561bd76692c0 con 0x561bd7754390 2024-04-24T09:07:44.546 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.543+0000 7f96727e8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x561bd7757140 con 0x561bd7749ad0 2024-04-24T09:07:44.546 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.544+0000 7f96727e8740 10 client.0 _ll_register_callbacks cb 0x561bd73c1a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:07:44.547 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.544+0000 7f966da00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561bd7754390 0x561bd7751cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:44.547 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.544+0000 7f966da00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561bd7754390 0x561bd7751cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:59996/0 (socket says 172.21.15.83:59996) 2024-04-24T09:07:44.547 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.544+0000 7f966da00640 1 -- 172.21.15.83:0/3387699923 learned_addr learned my addr 172.21.15.83:0/3387699923 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:07:44.547 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.544+0000 7f9667e00640 1 --2- 172.21.15.83:0/3387699923 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561bd7749ad0 0x561bd7751790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:44.547 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.544+0000 7f966d000640 1 --2- 172.21.15.83:0/3387699923 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561bd77491b0 0x561bd7751250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:44.547 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.545+0000 7f9667e00640 1 -- 172.21.15.83:0/3387699923 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561bd7754390 msgr2=0x561bd7751cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:44.547 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.545+0000 7f9667e00640 1 --2- 172.21.15.83:0/3387699923 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561bd7754390 0x561bd7751cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:44.547 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.545+0000 7f9667e00640 1 -- 172.21.15.83:0/3387699923 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561bd77491b0 msgr2=0x561bd7751250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:44.547 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.545+0000 7f9667e00640 1 --2- 172.21.15.83:0/3387699923 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561bd77491b0 0x561bd7751250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:44.547 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.545+0000 7f9667e00640 1 -- 172.21.15.83:0/3387699923 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561bd7679fd0 con 0x561bd7749ad0 2024-04-24T09:07:44.547 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.545+0000 7f966da00640 1 --2- 172.21.15.83:0/3387699923 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561bd7754390 0x561bd7751cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:07:44.547 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.545+0000 7f9667e00640 1 --2- 172.21.15.83:0/3387699923 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561bd7749ad0 0x561bd7751790 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9654003f50 tx=0x7f9654003d40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dfd43fdfd00d8cd5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:44.547 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.545+0000 7f964ea00640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-04-24T09:07:44.547 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.545+0000 7f964ea00640 1 -- 172.21.15.83:0/3387699923 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9654005c50 con 0x561bd7749ad0 2024-04-24T09:07:44.548 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.546+0000 7f96727e8740 1 -- 172.21.15.83:0/3387699923 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96540041f0 con 0x561bd7749ad0 2024-04-24T09:07:44.548 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.546+0000 7f964ea00640 1 -- 172.21.15.83:0/3387699923 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9583+0+0 (secure 0 0 0) 0x7f96540131b0 con 0x561bd7749ad0 2024-04-24T09:07:44.548 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.546+0000 7f964ea00640 1 -- 172.21.15.83:0/3387699923 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f9654031050 con 0x561bd7749ad0 2024-04-24T09:07:44.548 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.546+0000 7f96727e8740 10 client.28468 fetch_fsmap learned FSMap version 52 2024-04-24T09:07:44.548 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.546+0000 7f96727e8740 1 -- 172.21.15.83:0/3387699923 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x561bd73c2210 con 0x561bd7749ad0 2024-04-24T09:07:44.549 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.547+0000 7f964ea00640 1 -- 172.21.15.83:0/3387699923 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f96540041f0 con 0x561bd7749ad0 2024-04-24T09:07:44.549 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.547+0000 7f964ea00640 1 -- 172.21.15.83:0/3387699923 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 52) ==== 8402+0+0 (secure 0 0 0) 0x7f9654028420 con 0x561bd7749ad0 2024-04-24T09:07:44.549 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.547+0000 7f96727e8740 10 client.28468 fetch_fsmap finished waiting for FSMap version 52 2024-04-24T09:07:44.549 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.547+0000 7f96727e8740 10 client.28468 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:07:44.549 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.547+0000 7f96727e8740 20 client.28468 populate_metadata read hostname 'smithi083' 2024-04-24T09:07:44.549 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.547+0000 7f96727e8740 1 --2- 172.21.15.83:0/3387699923 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x561bd7764380 0x561bd7766770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:44.549 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.547+0000 7f96727e8740 4 client.28468 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:07:44.550 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.547+0000 7f96727e8740 1 -- 172.21.15.83:0/3387699923 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561bd76bd320 con 0x561bd7764380 2024-04-24T09:07:44.550 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.548+0000 7f966d000640 1 --2- 172.21.15.83:0/3387699923 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x561bd7764380 0x561bd7766770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:44.550 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.549+0000 7f964ea00640 10 client.28468 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:07:44.550 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.549+0000 7f966d000640 1 --2- 172.21.15.83:0/3387699923 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x561bd7764380 0x561bd7766770 crc :-1 s=READY pgs=411 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=539c4ef427970ae4 server_cookie=baf2fbe661269115 in_seq=0 out_seq=0 2024-04-24T09:07:44.551 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.550+0000 7f964ea00640 1 -- 172.21.15.83:0/3387699923 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561bd76bd320 con 0x561bd7764380 2024-04-24T09:07:44.551 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.550+0000 7f964ea00640 10 client.28468 handle_command_reply: tid=0 2024-04-24T09:07:44.551 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.550+0000 7f96727e8740 1 client.28468 shutdown 2024-04-24T09:07:44.551 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.550+0000 7f96727e8740 1 -- 172.21.15.83:0/3387699923 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561bd7749ad0 msgr2=0x561bd7751790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:44.551 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.550+0000 7f96727e8740 1 --2- 172.21.15.83:0/3387699923 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561bd7749ad0 0x561bd7751790 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9654003f50 tx=0x7f9654003d40 comp rx=0 tx=0).stop 2024-04-24T09:07:44.551 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.550+0000 7f96727e8740 1 -- 172.21.15.83:0/3387699923 shutdown_connections 2024-04-24T09:07:44.551 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.550+0000 7f96727e8740 1 --2- 172.21.15.83:0/3387699923 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x561bd7764380 0x561bd7766770 crc :-1 s=READY pgs=411 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:44.551 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.550+0000 7f96727e8740 1 --2- 172.21.15.83:0/3387699923 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561bd7754390 0x561bd7751cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:44.551 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.550+0000 7f96727e8740 1 --2- 172.21.15.83:0/3387699923 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561bd7749ad0 0x561bd7751790 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:44.552 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.550+0000 7f96727e8740 1 --2- 172.21.15.83:0/3387699923 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561bd77491b0 0x561bd7751250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:44.552 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.550+0000 7f964ea00640 0 client.28468 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:07:44.552 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.550+0000 7f96727e8740 1 -- 172.21.15.83:0/3387699923 >> 172.21.15.83:0/3387699923 conn(0x561bd77456c0 msgr2=0x561bd774b440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:44.552 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.551+0000 7f96727e8740 1 -- 172.21.15.83:0/3387699923 shutdown_connections 2024-04-24T09:07:44.552 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.551+0000 7f96727e8740 1 -- 172.21.15.83:0/3387699923 wait complete. 2024-04-24T09:07:44.552 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.551+0000 7f96727e8740 20 client.28468 trim_cache size 0 max 16384 2024-04-24T09:07:44.574 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.574+0000 7f96727e8740 1 Processor -- start 2024-04-24T09:07:44.574 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.574+0000 7f96727e8740 1 -- start start 2024-04-24T09:07:44.575 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.574+0000 7f96727e8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561bd77491b0 0x561bd775fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:44.575 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.574+0000 7f96727e8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561bd7749ad0 0x561bd776bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:44.575 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.574+0000 7f96727e8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561bd7754390 0x561bd776c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:44.575 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.574+0000 7f96727e8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f96681be370 con 0x561bd7754390 2024-04-24T09:07:44.575 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.574+0000 7f96727e8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f96681bffd0 con 0x561bd7749ad0 2024-04-24T09:07:44.576 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.574+0000 7f96727e8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f96681bea40 con 0x561bd77491b0 2024-04-24T09:07:44.576 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.575+0000 7f966da00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561bd7754390 0x561bd776c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:44.576 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.575+0000 7f966da00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561bd7754390 0x561bd776c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35478/0 (socket says 172.21.15.83:35478) 2024-04-24T09:07:44.576 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.575+0000 7f966da00640 1 -- 172.21.15.83:0/2759242306 learned_addr learned my addr 172.21.15.83:0/2759242306 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:07:44.576 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.575+0000 7f9667e00640 1 --2- 172.21.15.83:0/2759242306 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561bd7749ad0 0x561bd776bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:44.576 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.575+0000 7f966da00640 1 -- 172.21.15.83:0/2759242306 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561bd77491b0 msgr2=0x561bd775fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:44.576 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.575+0000 7f966da00640 1 --2- 172.21.15.83:0/2759242306 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561bd77491b0 0x561bd775fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:44.576 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.575+0000 7f966da00640 1 -- 172.21.15.83:0/2759242306 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561bd7749ad0 msgr2=0x561bd776bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:44.576 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.575+0000 7f966da00640 1 --2- 172.21.15.83:0/2759242306 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561bd7749ad0 0x561bd776bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:44.576 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.575+0000 7f966da00640 1 -- 172.21.15.83:0/2759242306 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561bd748e620 con 0x561bd7754390 2024-04-24T09:07:44.577 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.576+0000 7f966da00640 1 --2- 172.21.15.83:0/2759242306 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561bd7754390 0x561bd776c2e0 secure :-1 s=READY pgs=2270 cs=0 l=1 rev1=1 crypto rx=0x7f96600ba6b0 tx=0x7f96600de040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce24d0cb6d7fe144 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:44.577 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.576+0000 7f9659600640 1 -- 172.21.15.83:0/2759242306 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96600d6070 con 0x561bd7754390 2024-04-24T09:07:44.577 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.576+0000 7f9659600640 1 -- 172.21.15.83:0/2759242306 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9660036030 con 0x561bd7754390 2024-04-24T09:07:44.577 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.576+0000 7f9659600640 1 -- 172.21.15.83:0/2759242306 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96600ef600 con 0x561bd7754390 2024-04-24T09:07:44.577 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.577+0000 7f96727e8740 1 -- 172.21.15.83:0/2759242306 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561bd7754390 msgr2=0x561bd776c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:44.578 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.577+0000 7f96727e8740 1 --2- 172.21.15.83:0/2759242306 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561bd7754390 0x561bd776c2e0 secure :-1 s=READY pgs=2270 cs=0 l=1 rev1=1 crypto rx=0x7f96600ba6b0 tx=0x7f96600de040 comp rx=0 tx=0).stop 2024-04-24T09:07:44.578 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.577+0000 7f96727e8740 1 -- 172.21.15.83:0/2759242306 shutdown_connections 2024-04-24T09:07:44.578 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.577+0000 7f96727e8740 1 --2- 172.21.15.83:0/2759242306 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561bd7754390 0x561bd776c2e0 unknown :-1 s=CLOSED pgs=2270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:44.578 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.577+0000 7f96727e8740 1 --2- 172.21.15.83:0/2759242306 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561bd7749ad0 0x561bd776bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:44.578 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.577+0000 7f96727e8740 1 --2- 172.21.15.83:0/2759242306 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561bd77491b0 0x561bd775fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:44.578 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.577+0000 7f96727e8740 1 -- 172.21.15.83:0/2759242306 >> 172.21.15.83:0/2759242306 conn(0x561bd77456c0 msgr2=0x561bd774c4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:44.578 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.577+0000 7f96727e8740 1 -- 172.21.15.83:0/2759242306 shutdown_connections 2024-04-24T09:07:44.578 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.577+0000 7f96727e8740 1 -- 172.21.15.83:0/2759242306 wait complete. 2024-04-24T09:07:44.580 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.578+0000 7f96727e8740 1 -- start start 2024-04-24T09:07:44.580 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.578+0000 7f96727e8740 1 Processor -- start 2024-04-24T09:07:44.580 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.579+0000 7f96727e8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561bd77491b0 0x561bd774e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:44.580 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.579+0000 7f96727e8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561bd7749ad0 0x561bd774e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:44.580 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.579+0000 7f96727e8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561bd7754390 0x561bd774eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:44.580 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.579+0000 7f96727e8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f96681bf840 con 0x561bd77491b0 2024-04-24T09:07:44.580 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.579+0000 7f96727e8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f96681bf140 con 0x561bd7749ad0 2024-04-24T09:07:44.580 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.579+0000 7f96727e8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f96681bc370 con 0x561bd7754390 2024-04-24T09:07:44.580 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.579+0000 7f96727e8740 10 client.0 _ll_register_callbacks cb 0x7f965c00b480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:07:44.580 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.579+0000 7f966d000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561bd77491b0 0x561bd774e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:44.580 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.579+0000 7f966d000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561bd77491b0 0x561bd774e030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35492/0 (socket says 172.21.15.83:35492) 2024-04-24T09:07:44.580 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.579+0000 7f966d000640 1 -- 172.21.15.83:0/751746431 learned_addr learned my addr 172.21.15.83:0/751746431 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:07:44.580 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.579+0000 7f966da00640 1 --2- 172.21.15.83:0/751746431 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561bd7754390 0x561bd774eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:44.580 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.579+0000 7f9667e00640 1 --2- 172.21.15.83:0/751746431 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561bd7749ad0 0x561bd774e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:44.581 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.580+0000 7f966d000640 1 -- 172.21.15.83:0/751746431 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561bd7754390 msgr2=0x561bd774eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:44.581 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.580+0000 7f966d000640 1 --2- 172.21.15.83:0/751746431 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561bd7754390 0x561bd774eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:44.581 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.580+0000 7f966d000640 1 -- 172.21.15.83:0/751746431 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561bd7749ad0 msgr2=0x561bd774e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:44.581 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.580+0000 7f966d000640 1 --2- 172.21.15.83:0/751746431 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561bd7749ad0 0x561bd774e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:44.581 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.580+0000 7f966d000640 1 -- 172.21.15.83:0/751746431 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561bd77c9740 con 0x561bd77491b0 2024-04-24T09:07:44.582 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.580+0000 7f966d000640 1 --2- 172.21.15.83:0/751746431 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561bd77491b0 0x561bd774e030 secure :-1 s=READY pgs=2271 cs=0 l=1 rev1=1 crypto rx=0x7f965c0042e0 tx=0x7f965c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=416dd7b01f20b296 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:44.582 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.580+0000 7f964ea00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:07:44.582 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.580+0000 7f964ea00640 1 -- 172.21.15.83:0/751746431 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f965c02e060 con 0x561bd77491b0 2024-04-24T09:07:44.582 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.580+0000 7f96727e8740 1 -- 172.21.15.83:0/751746431 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9660036030 con 0x561bd77491b0 2024-04-24T09:07:44.583 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.580+0000 7f964ea00640 1 -- 172.21.15.83:0/751746431 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9583+0+0 (secure 0 0 0) 0x7f965c01ed00 con 0x561bd77491b0 2024-04-24T09:07:44.583 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.581+0000 7f964ea00640 1 -- 172.21.15.83:0/751746431 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f965c035050 con 0x561bd77491b0 2024-04-24T09:07:44.583 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.581+0000 7f96727e8740 10 client.21279 fetch_fsmap learned FSMap version 52 2024-04-24T09:07:44.583 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.581+0000 7f96727e8740 1 -- 172.21.15.83:0/751746431 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x561bd77caae0 con 0x561bd77491b0 2024-04-24T09:07:44.583 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.581+0000 7f964ea00640 1 -- 172.21.15.83:0/751746431 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9660036030 con 0x561bd77491b0 2024-04-24T09:07:44.583 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.581+0000 7f964ea00640 1 -- 172.21.15.83:0/751746431 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 52) ==== 8402+0+0 (secure 0 0 0) 0x7f965c029400 con 0x561bd77491b0 2024-04-24T09:07:44.583 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.582+0000 7f96727e8740 10 client.21279 fetch_fsmap finished waiting for FSMap version 52 2024-04-24T09:07:44.583 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.582+0000 7f96727e8740 10 client.21279 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:07:44.583 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.582+0000 7f96727e8740 20 client.21279 populate_metadata read hostname 'smithi083' 2024-04-24T09:07:44.583 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.582+0000 7f96727e8740 1 --2- 172.21.15.83:0/751746431 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x561bd77cb2d0 0x561bd77cd6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:44.583 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.582+0000 7f96727e8740 4 client.21279 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T09:07:44.583 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.582+0000 7f96727e8740 1 -- 172.21.15.83:0/751746431 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561bd76b7190 con 0x561bd77cb2d0 2024-04-24T09:07:44.583 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.582+0000 7f9667e00640 1 --2- 172.21.15.83:0/751746431 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x561bd77cb2d0 0x561bd77cd6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:44.584 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.583+0000 7f9667e00640 1 --2- 172.21.15.83:0/751746431 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x561bd77cb2d0 0x561bd77cd6f0 crc :-1 s=READY pgs=412 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=547aabbc3158773e server_cookie=8b1d188a99fa51c6 in_seq=0 out_seq=0 2024-04-24T09:07:44.584 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.583+0000 7f964ea00640 10 client.21279 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:07:44.834 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.834+0000 7f964ea00640 1 -- 172.21.15.83:0/751746431 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561bd76b7190 con 0x561bd77cb2d0 2024-04-24T09:07:44.834 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.834+0000 7f964ea00640 10 client.21279 handle_command_reply: tid=0 2024-04-24T09:07:44.834 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.834+0000 7f96727e8740 1 client.21279 shutdown 2024-04-24T09:07:44.834 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.834+0000 7f96727e8740 1 -- 172.21.15.83:0/751746431 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561bd77491b0 msgr2=0x561bd774e030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:44.834 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.834+0000 7f96727e8740 1 --2- 172.21.15.83:0/751746431 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561bd77491b0 0x561bd774e030 secure :-1 s=READY pgs=2271 cs=0 l=1 rev1=1 crypto rx=0x7f965c0042e0 tx=0x7f965c004730 comp rx=0 tx=0).stop 2024-04-24T09:07:44.834 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.834+0000 7f96727e8740 1 -- 172.21.15.83:0/751746431 shutdown_connections 2024-04-24T09:07:44.835 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.834+0000 7f96727e8740 1 --2- 172.21.15.83:0/751746431 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x561bd77cb2d0 0x561bd77cd6f0 crc :-1 s=READY pgs=412 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:44.835 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.834+0000 7f96727e8740 1 --2- 172.21.15.83:0/751746431 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561bd7754390 0x561bd774eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:44.835 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.834+0000 7f96727e8740 1 --2- 172.21.15.83:0/751746431 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561bd7749ad0 0x561bd774e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:44.835 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.834+0000 7f96727e8740 1 --2- 172.21.15.83:0/751746431 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561bd77491b0 0x561bd774e030 unknown :-1 s=CLOSED pgs=2271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:44.835 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.834+0000 7f964ea00640 0 client.21279 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:07:44.835 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.834+0000 7f96727e8740 1 -- 172.21.15.83:0/751746431 >> 172.21.15.83:0/751746431 conn(0x561bd77456c0 msgr2=0x561bd7752a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:44.835 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.835+0000 7f96727e8740 1 -- 172.21.15.83:0/751746431 shutdown_connections 2024-04-24T09:07:44.835 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.835+0000 7f96727e8740 1 -- 172.21.15.83:0/751746431 wait complete. 2024-04-24T09:07:44.836 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.835+0000 7f96727e8740 20 client.21279 trim_cache size 0 max 16384 2024-04-24T09:07:44.836 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T09:07:44.836 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T09:07:44.836 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T09:07:44.836 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T09:07:44.836 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.836+0000 7f966fa00640 1 -- 172.21.15.83:0/3448071030 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f96500a3e40 msgr2=0x7f96500a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:44.837 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.836+0000 7f966fa00640 1 --2- 172.21.15.83:0/3448071030 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f96500a3e40 0x7f96500a6300 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x561bd774a750 tx=0x7f965c00b5e0 comp rx=0 tx=0).stop 2024-04-24T09:07:44.837 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.836+0000 7f966fa00640 1 -- 172.21.15.83:0/3448071030 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f966810dff0 msgr2=0x7f966810e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:44.837 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.836+0000 7f966fa00640 1 --2- 172.21.15.83:0/3448071030 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f966810dff0 0x7f966810e4a0 secure :-1 s=READY pgs=2268 cs=0 l=1 rev1=1 crypto rx=0x7f9660005e30 tx=0x7f96600093d0 comp rx=0 tx=0).stop 2024-04-24T09:07:44.837 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.836+0000 7f966fa00640 1 -- 172.21.15.83:0/3448071030 shutdown_connections 2024-04-24T09:07:44.837 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.836+0000 7f966fa00640 1 --2- 172.21.15.83:0/3448071030 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f96500a3e40 0x7f96500a6300 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:44.837 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.836+0000 7f966fa00640 1 --2- 172.21.15.83:0/3448071030 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f966810dff0 0x7f966810e4a0 unknown :-1 s=CLOSED pgs=2268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:44.837 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.836+0000 7f966fa00640 1 --2- 172.21.15.83:0/3448071030 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9668109540 0x7f9668073990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:44.837 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.836+0000 7f966fa00640 1 --2- 172.21.15.83:0/3448071030 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9668104470 0x7f9668110f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:44.837 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.836+0000 7f966fa00640 1 -- 172.21.15.83:0/3448071030 >> 172.21.15.83:0/3448071030 conn(0x7f96680ffcd0 msgr2=0x7f9668101f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:44.837 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.838+0000 7f966fa00640 1 -- 172.21.15.83:0/3448071030 shutdown_connections 2024-04-24T09:07:44.838 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:44.838+0000 7f966fa00640 1 -- 172.21.15.83:0/3448071030 wait complete. 2024-04-24T09:07:45.163 INFO:tasks.workunit.client.0.smithi083.stdout: 10 17505 7.04 MB/sec execute 16 sec latency 265.942 ms 2024-04-24T09:07:46.163 INFO:tasks.workunit.client.0.smithi083.stdout: 10 17635 6.99 MB/sec execute 17 sec latency 98.889 ms 2024-04-24T09:07:46.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:07:46 smithi083 ceph-mon[27907]: pgmap v507: 129 pgs: 129 active+clean; 434 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.4 MiB/s wr, 232 op/s 2024-04-24T09:07:46.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:07:46 smithi138 ceph-mon[31441]: pgmap v507: 129 pgs: 129 active+clean; 434 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.4 MiB/s wr, 232 op/s 2024-04-24T09:07:46.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:07:46 smithi189 ceph-mon[31791]: pgmap v507: 129 pgs: 129 active+clean; 434 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.4 MiB/s wr, 232 op/s 2024-04-24T09:07:47.163 INFO:tasks.workunit.client.0.smithi083.stdout: 10 17753 6.82 MB/sec execute 18 sec latency 62.146 ms 2024-04-24T09:07:48.163 INFO:tasks.workunit.client.0.smithi083.stdout: 10 17853 6.65 MB/sec execute 19 sec latency 271.294 ms 2024-04-24T09:07:48.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:07:48 smithi083 ceph-mon[27907]: pgmap v508: 129 pgs: 129 active+clean; 434 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.4 MiB/s wr, 232 op/s 2024-04-24T09:07:48.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:07:48 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:07:48.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:07:48 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:07:48.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:07:48 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:07:48.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:07:48 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:07:48.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:07:48 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:07:48.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:07:48 smithi138 ceph-mon[31441]: pgmap v508: 129 pgs: 129 active+clean; 434 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.4 MiB/s wr, 232 op/s 2024-04-24T09:07:48.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:07:48 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:07:48.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:07:48 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:07:48.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:07:48 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:07:48.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:07:48 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:07:48.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:07:48 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:07:48.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:07:48 smithi189 ceph-mon[31791]: pgmap v508: 129 pgs: 129 active+clean; 434 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.4 MiB/s wr, 232 op/s 2024-04-24T09:07:48.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:07:48 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:07:48.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:07:48 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:07:48.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:07:48 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:07:48.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:07:48 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:07:48.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:07:48 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:07:49.163 INFO:tasks.workunit.client.0.smithi083.stdout: 10 18017 7.23 MB/sec execute 20 sec latency 70.834 ms 2024-04-24T09:07:50.163 INFO:tasks.workunit.client.0.smithi083.stdout: 10 18146 6.99 MB/sec execute 21 sec latency 69.569 ms 2024-04-24T09:07:50.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:07:50 smithi083 ceph-mon[27907]: pgmap v509: 129 pgs: 129 active+clean; 422 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.0 MiB/s wr, 392 op/s 2024-04-24T09:07:50.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:07:50 smithi138 ceph-mon[31441]: pgmap v509: 129 pgs: 129 active+clean; 422 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.0 MiB/s wr, 392 op/s 2024-04-24T09:07:50.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:07:50 smithi189 ceph-mon[31791]: pgmap v509: 129 pgs: 129 active+clean; 422 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.0 MiB/s wr, 392 op/s 2024-04-24T09:07:51.163 INFO:tasks.workunit.client.0.smithi083.stdout: 10 18268 6.74 MB/sec execute 22 sec latency 73.468 ms 2024-04-24T09:07:52.163 INFO:tasks.workunit.client.0.smithi083.stdout: 10 18415 6.62 MB/sec execute 23 sec latency 70.239 ms 2024-04-24T09:07:52.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:07:52 smithi083 ceph-mon[27907]: pgmap v510: 129 pgs: 129 active+clean; 422 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.0 MiB/s wr, 253 op/s 2024-04-24T09:07:52.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:07:52 smithi138 ceph-mon[31441]: pgmap v510: 129 pgs: 129 active+clean; 422 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.0 MiB/s wr, 253 op/s 2024-04-24T09:07:52.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:07:52 smithi189 ceph-mon[31791]: pgmap v510: 129 pgs: 129 active+clean; 422 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.0 MiB/s wr, 253 op/s 2024-04-24T09:07:53.163 INFO:tasks.workunit.client.0.smithi083.stdout: 10 18551 6.57 MB/sec execute 24 sec latency 75.473 ms 2024-04-24T09:07:54.163 INFO:tasks.workunit.client.0.smithi083.stdout: 10 18704 6.51 MB/sec execute 25 sec latency 68.636 ms 2024-04-24T09:07:54.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:07:54 smithi083 ceph-mon[27907]: pgmap v511: 129 pgs: 129 active+clean; 425 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.0 MiB/s wr, 357 op/s 2024-04-24T09:07:54.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:07:54 smithi138 ceph-mon[31441]: pgmap v511: 129 pgs: 129 active+clean; 425 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.0 MiB/s wr, 357 op/s 2024-04-24T09:07:54.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:07:54 smithi189 ceph-mon[31791]: pgmap v511: 129 pgs: 129 active+clean; 425 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.0 MiB/s wr, 357 op/s 2024-04-24T09:07:55.164 INFO:tasks.workunit.client.0.smithi083.stdout: 10 18869 6.66 MB/sec execute 26 sec latency 61.644 ms 2024-04-24T09:07:56.164 INFO:tasks.workunit.client.0.smithi083.stdout: 10 19000 6.61 MB/sec execute 27 sec latency 66.938 ms 2024-04-24T09:07:56.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:07:56 smithi083 ceph-mon[27907]: pgmap v512: 129 pgs: 129 active+clean; 425 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 264 op/s 2024-04-24T09:07:56.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:07:56 smithi138 ceph-mon[31441]: pgmap v512: 129 pgs: 129 active+clean; 425 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 264 op/s 2024-04-24T09:07:56.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:07:56 smithi189 ceph-mon[31791]: pgmap v512: 129 pgs: 129 active+clean; 425 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 264 op/s 2024-04-24T09:07:57.163 INFO:tasks.workunit.client.0.smithi083.stdout: 10 19133 6.73 MB/sec execute 28 sec latency 68.495 ms 2024-04-24T09:07:58.031 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.031+0000 7fdda5600640 1 -- 172.21.15.83:0/4212642408 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd98031880 con 0x7fdda81011a0 2024-04-24T09:07:58.031 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.031+0000 7fddade00640 1 -- 172.21.15.83:0/4212642408 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdda81011a0 msgr2=0x7fdda8101580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:58.031 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.031+0000 7fddade00640 1 --2- 172.21.15.83:0/4212642408 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdda81011a0 0x7fdda8101580 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fdd98009900 tx=0x7fdd9802a3f0 comp rx=0 tx=0).stop 2024-04-24T09:07:58.031 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.032+0000 7fddade00640 1 -- 172.21.15.83:0/4212642408 shutdown_connections 2024-04-24T09:07:58.031 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.032+0000 7fddade00640 1 --2- 172.21.15.83:0/4212642408 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdda8106220 0x7fdda810ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:58.031 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.032+0000 7fddade00640 1 --2- 172.21.15.83:0/4212642408 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdda8101b50 0x7fdda8105b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:58.031 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.032+0000 7fddade00640 1 --2- 172.21.15.83:0/4212642408 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdda81011a0 0x7fdda8101580 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:58.031 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.032+0000 7fddade00640 1 -- 172.21.15.83:0/4212642408 >> 172.21.15.83:0/4212642408 conn(0x7fdda8078080 msgr2=0x7fdda8078490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:58.031 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.032+0000 7fddade00640 1 -- 172.21.15.83:0/4212642408 shutdown_connections 2024-04-24T09:07:58.032 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.032+0000 7fddade00640 1 -- 172.21.15.83:0/4212642408 wait complete. 2024-04-24T09:07:58.032 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.033+0000 7fddade00640 1 Processor -- start 2024-04-24T09:07:58.032 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.033+0000 7fddade00640 1 -- start start 2024-04-24T09:07:58.033 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.033+0000 7fddade00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdda81011a0 0x7fdda81a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:58.033 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.034+0000 7fddade00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdda8106220 0x7fdda81a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:58.033 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.034+0000 7fddade00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdda819a7c0 0x7fdda819ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:58.033 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.034+0000 7fddade00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdda81111e0 con 0x7fdda8106220 2024-04-24T09:07:58.033 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.034+0000 7fddade00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdda8111060 con 0x7fdda81011a0 2024-04-24T09:07:58.033 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.034+0000 7fddade00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdda8111360 con 0x7fdda819a7c0 2024-04-24T09:07:58.034 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.034+0000 7fdda6000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdda8106220 0x7fdda81a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:58.034 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.034+0000 7fdda6000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdda8106220 0x7fdda81a0b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55408/0 (socket says 172.21.15.83:55408) 2024-04-24T09:07:58.034 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.034+0000 7fdda6000640 1 -- 172.21.15.83:0/2828679479 learned_addr learned my addr 172.21.15.83:0/2828679479 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:07:58.034 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.035+0000 7fdda7400640 1 --2- 172.21.15.83:0/2828679479 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdda819a7c0 0x7fdda819ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:58.034 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.035+0000 7fdda6a00640 1 --2- 172.21.15.83:0/2828679479 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdda81011a0 0x7fdda81a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:58.034 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.035+0000 7fdda6000640 1 -- 172.21.15.83:0/2828679479 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdda819a7c0 msgr2=0x7fdda819ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:58.034 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.035+0000 7fdda6000640 1 --2- 172.21.15.83:0/2828679479 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdda819a7c0 0x7fdda819ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:58.034 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.035+0000 7fdda6000640 1 -- 172.21.15.83:0/2828679479 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdda81011a0 msgr2=0x7fdda81a0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:58.034 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.035+0000 7fdda6000640 1 --2- 172.21.15.83:0/2828679479 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdda81011a0 0x7fdda81a0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:58.035 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.035+0000 7fdda6000640 1 -- 172.21.15.83:0/2828679479 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdda819b350 con 0x7fdda8106220 2024-04-24T09:07:58.035 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.035+0000 7fdda6000640 1 --2- 172.21.15.83:0/2828679479 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdda8106220 0x7fdda81a0b90 secure :-1 s=READY pgs=2272 cs=0 l=1 rev1=1 crypto rx=0x7fdd9400e9c0 tx=0x7fdd9400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:58.035 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.036+0000 7fddaca00640 1 -- 172.21.15.83:0/2828679479 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd9400cd80 con 0x7fdda8106220 2024-04-24T09:07:58.035 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.036+0000 7fddaca00640 1 -- 172.21.15.83:0/2828679479 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdd940054d0 con 0x7fdda8106220 2024-04-24T09:07:58.036 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.036+0000 7fddaca00640 1 -- 172.21.15.83:0/2828679479 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd94010430 con 0x7fdda8106220 2024-04-24T09:07:58.036 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.036+0000 7fddade00640 1 -- 172.21.15.83:0/2828679479 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdda819b640 con 0x7fdda8106220 2024-04-24T09:07:58.036 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.036+0000 7fddade00640 1 -- 172.21.15.83:0/2828679479 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdda8108070 con 0x7fdda8106220 2024-04-24T09:07:58.041 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.042+0000 7fddb0c49740 1 Processor -- start 2024-04-24T09:07:58.041 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.042+0000 7fddb0c49740 1 -- start start 2024-04-24T09:07:58.041 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.042+0000 7fddaca00640 1 -- 172.21.15.83:0/2828679479 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fdd94004fd0 con 0x7fdda8106220 2024-04-24T09:07:58.043 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.043+0000 7fddb0c49740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606dc9771b0 0x5606dc977590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:58.043 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.043+0000 7fddb0c49740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5606dc977ad0 0x5606dc981e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:58.043 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.043+0000 7fddb0c49740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606dc982390 0x5606dc984780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:58.044 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.044+0000 7fddb0c49740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5606dc8dd900 con 0x5606dc9771b0 2024-04-24T09:07:58.044 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.044+0000 7fddb0c49740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5606dc883f20 con 0x5606dc982390 2024-04-24T09:07:58.044 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.044+0000 7fddb0c49740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5606dc8d98c0 con 0x5606dc977ad0 2024-04-24T09:07:58.044 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.044+0000 7fdda7400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606dc9771b0 0x5606dc977590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:58.044 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.044+0000 7fdda7400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606dc9771b0 0x5606dc977590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55412/0 (socket says 172.21.15.83:55412) 2024-04-24T09:07:58.044 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.044+0000 7fdda7400640 1 -- 172.21.15.83:0/2942089993 learned_addr learned my addr 172.21.15.83:0/2942089993 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:07:58.044 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.044+0000 7fdda6a00640 1 --2- 172.21.15.83:0/2942089993 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5606dc977ad0 0x5606dc981e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:58.044 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.044+0000 7fdda6000640 1 --2- 172.21.15.83:0/2942089993 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606dc982390 0x5606dc984780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:58.044 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.044+0000 7fdda7400640 1 -- 172.21.15.83:0/2942089993 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5606dc977ad0 msgr2=0x5606dc981e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:58.044 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.044+0000 7fdda7400640 1 --2- 172.21.15.83:0/2942089993 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5606dc977ad0 0x5606dc981e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:58.044 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.044+0000 7fdda7400640 1 -- 172.21.15.83:0/2942089993 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606dc982390 msgr2=0x5606dc984780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:58.044 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.044+0000 7fdda7400640 1 --2- 172.21.15.83:0/2942089993 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606dc982390 0x5606dc984780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:58.044 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.044+0000 7fdda7400640 1 -- 172.21.15.83:0/2942089993 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606dc8a8e10 con 0x5606dc9771b0 2024-04-24T09:07:58.045 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.045+0000 7fddaca00640 1 --2- 172.21.15.83:0/2828679479 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fdd8c0a3e40 0x7fdd8c0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:58.045 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.045+0000 7fddaca00640 1 -- 172.21.15.83:0/2828679479 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 9583+0+0 (secure 0 0 0) 0x7fdd940d10f0 con 0x7fdda8106220 2024-04-24T09:07:58.045 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.046+0000 7fdda6a00640 1 --2- 172.21.15.83:0/2828679479 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fdd8c0a3e40 0x7fdd8c0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:58.046 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.047+0000 7fdda6a00640 1 --2- 172.21.15.83:0/2828679479 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fdd8c0a3e40 0x7fdd8c0a6300 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fdd98031440 tx=0x7fdd98005a20 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:58.046 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.047+0000 7fdda7400640 1 --2- 172.21.15.83:0/2942089993 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606dc9771b0 0x5606dc977590 secure :-1 s=READY pgs=2273 cs=0 l=1 rev1=1 crypto rx=0x7fdda0008d40 tx=0x7fdda000e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efec1a17ab9d5987 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:58.048 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.049+0000 7fdd93e00640 1 -- 172.21.15.83:0/2942089993 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda0014070 con 0x5606dc9771b0 2024-04-24T09:07:58.048 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.049+0000 7fdd93e00640 1 -- 172.21.15.83:0/2942089993 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdda0003b70 con 0x5606dc9771b0 2024-04-24T09:07:58.048 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.049+0000 7fdd93e00640 1 -- 172.21.15.83:0/2942089993 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda0010700 con 0x5606dc9771b0 2024-04-24T09:07:58.049 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.050+0000 7fddb0c49740 1 -- 172.21.15.83:0/2942089993 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606dc9771b0 msgr2=0x5606dc977590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:58.049 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.050+0000 7fddb0c49740 1 --2- 172.21.15.83:0/2942089993 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606dc9771b0 0x5606dc977590 secure :-1 s=READY pgs=2273 cs=0 l=1 rev1=1 crypto rx=0x7fdda0008d40 tx=0x7fdda000e720 comp rx=0 tx=0).stop 2024-04-24T09:07:58.049 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.050+0000 7fddb0c49740 1 -- 172.21.15.83:0/2942089993 shutdown_connections 2024-04-24T09:07:58.049 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.050+0000 7fddb0c49740 1 --2- 172.21.15.83:0/2942089993 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606dc982390 0x5606dc984780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:58.049 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.050+0000 7fddb0c49740 1 --2- 172.21.15.83:0/2942089993 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5606dc977ad0 0x5606dc981e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:58.049 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.050+0000 7fddb0c49740 1 --2- 172.21.15.83:0/2942089993 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606dc9771b0 0x5606dc977590 unknown :-1 s=CLOSED pgs=2273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:58.049 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.050+0000 7fddb0c49740 1 -- 172.21.15.83:0/2942089993 >> 172.21.15.83:0/2942089993 conn(0x5606dc9736c0 msgr2=0x5606dc973aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:58.050 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.050+0000 7fddb0c49740 1 -- 172.21.15.83:0/2942089993 shutdown_connections 2024-04-24T09:07:58.050 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.051+0000 7fddb0c49740 1 -- 172.21.15.83:0/2942089993 wait complete. 2024-04-24T09:07:58.050 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.051+0000 7fddb0c49740 1 -- start start 2024-04-24T09:07:58.051 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.052+0000 7fddb0c49740 1 Processor -- start 2024-04-24T09:07:58.052 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.052+0000 7fddb0c49740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606dc9771b0 0x5606dc97f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:58.052 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.052+0000 7fddb0c49740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606dc977ad0 0x5606dc97f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:58.052 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.052+0000 7fddb0c49740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5606dc982390 0x5606dc97fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:58.052 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.052+0000 7fddb0c49740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5606dc984fc0 con 0x5606dc9771b0 2024-04-24T09:07:58.052 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.052+0000 7fddb0c49740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5606dc8972c0 con 0x5606dc977ad0 2024-04-24T09:07:58.052 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.052+0000 7fddb0c49740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5606dc985140 con 0x5606dc982390 2024-04-24T09:07:58.052 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.053+0000 7fddb0c49740 10 client.0 _ll_register_callbacks cb 0x5606dc5efa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:07:58.052 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.053+0000 7fdda6000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606dc977ad0 0x5606dc97f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:58.052 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.053+0000 7fdda6000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606dc977ad0 0x5606dc97f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:38306/0 (socket says 172.21.15.83:38306) 2024-04-24T09:07:58.052 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.053+0000 7fdda6000640 1 -- 172.21.15.83:0/1237675169 learned_addr learned my addr 172.21.15.83:0/1237675169 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:07:58.053 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.053+0000 7fdda6a00640 1 --2- 172.21.15.83:0/1237675169 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606dc9771b0 0x5606dc97f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:58.054 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.053+0000 7fdda6000640 1 -- 172.21.15.83:0/1237675169 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5606dc982390 msgr2=0x5606dc97fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:58.054 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.053+0000 7fdda6000640 1 --2- 172.21.15.83:0/1237675169 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5606dc982390 0x5606dc97fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:58.054 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.053+0000 7fdda6000640 1 -- 172.21.15.83:0/1237675169 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606dc9771b0 msgr2=0x5606dc97f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:58.054 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.053+0000 7fdda6000640 1 --2- 172.21.15.83:0/1237675169 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606dc9771b0 0x5606dc97f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:58.054 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.053+0000 7fdda6000640 1 -- 172.21.15.83:0/1237675169 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606dc8a7fd0 con 0x5606dc977ad0 2024-04-24T09:07:58.054 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.054+0000 7fdda6000640 1 --2- 172.21.15.83:0/1237675169 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606dc977ad0 0x5606dc97f870 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fdd940990d0 tx=0x7fdd94098f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9094f14c028a03d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:58.054 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.054+0000 7fdd90c00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:07:58.054 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.054+0000 7fdd90c00640 1 -- 172.21.15.83:0/1237675169 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd9409c5d0 con 0x5606dc977ad0 2024-04-24T09:07:58.054 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.054+0000 7fddb0c49740 1 -- 172.21.15.83:0/1237675169 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdda0003b70 con 0x5606dc977ad0 2024-04-24T09:07:58.054 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.054+0000 7fdd90c00640 1 -- 172.21.15.83:0/1237675169 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9583+0+0 (secure 0 0 0) 0x7fdd940bd0c0 con 0x5606dc977ad0 2024-04-24T09:07:58.054 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.054+0000 7fdd90c00640 1 -- 172.21.15.83:0/1237675169 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fdd940bd2c0 con 0x5606dc977ad0 2024-04-24T09:07:58.054 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.054+0000 7fddb0c49740 10 client.38438 fetch_fsmap learned FSMap version 52 2024-04-24T09:07:58.054 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.054+0000 7fddb0c49740 1 -- 172.21.15.83:0/1237675169 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x5606dc5f0210 con 0x5606dc977ad0 2024-04-24T09:07:58.054 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.055+0000 7fdd90c00640 1 -- 172.21.15.83:0/1237675169 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdda0003b70 con 0x5606dc977ad0 2024-04-24T09:07:58.055 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.055+0000 7fdd90c00640 1 -- 172.21.15.83:0/1237675169 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 52) ==== 8402+0+0 (secure 0 0 0) 0x7fdd940a5450 con 0x5606dc977ad0 2024-04-24T09:07:58.056 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.055+0000 7fddb0c49740 10 client.38438 fetch_fsmap finished waiting for FSMap version 52 2024-04-24T09:07:58.056 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.055+0000 7fddb0c49740 10 client.38438 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:07:58.056 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.055+0000 7fddb0c49740 20 client.38438 populate_metadata read hostname 'smithi083' 2024-04-24T09:07:58.056 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.055+0000 7fddb0c49740 1 --2- 172.21.15.83:0/1237675169 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5606dc992540 0x5606dc9949e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:58.056 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.055+0000 7fddb0c49740 4 client.38438 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:07:58.056 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.055+0000 7fddb0c49740 1 -- 172.21.15.83:0/1237675169 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5606dc8eb320 con 0x5606dc992540 2024-04-24T09:07:58.056 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.056+0000 7fdda6a00640 1 --2- 172.21.15.83:0/1237675169 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5606dc992540 0x5606dc9949e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:58.056 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.056+0000 7fdda6a00640 1 --2- 172.21.15.83:0/1237675169 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5606dc992540 0x5606dc9949e0 crc :-1 s=READY pgs=413 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a711b2d059a0d8af server_cookie=2ec8aa3c351e9ca1 in_seq=0 out_seq=0 2024-04-24T09:07:58.056 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.056+0000 7fdd90c00640 10 client.38438 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:07:58.058 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.059+0000 7fdd90c00640 1 -- 172.21.15.83:0/1237675169 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5606dc8eb320 con 0x5606dc992540 2024-04-24T09:07:58.058 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.059+0000 7fdd90c00640 10 client.38438 handle_command_reply: tid=0 2024-04-24T09:07:58.058 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.059+0000 7fddb0c49740 1 client.38438 shutdown 2024-04-24T09:07:58.058 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.059+0000 7fddb0c49740 1 -- 172.21.15.83:0/1237675169 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606dc977ad0 msgr2=0x5606dc97f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:58.058 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.059+0000 7fddb0c49740 1 --2- 172.21.15.83:0/1237675169 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606dc977ad0 0x5606dc97f870 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fdd940990d0 tx=0x7fdd94098f10 comp rx=0 tx=0).stop 2024-04-24T09:07:58.059 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.059+0000 7fddb0c49740 1 -- 172.21.15.83:0/1237675169 shutdown_connections 2024-04-24T09:07:58.059 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.059+0000 7fddb0c49740 1 --2- 172.21.15.83:0/1237675169 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5606dc992540 0x5606dc9949e0 crc :-1 s=READY pgs=413 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:58.059 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.059+0000 7fddb0c49740 1 --2- 172.21.15.83:0/1237675169 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5606dc982390 0x5606dc97fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:58.059 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.059+0000 7fddb0c49740 1 --2- 172.21.15.83:0/1237675169 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606dc977ad0 0x5606dc97f870 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:58.059 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.059+0000 7fddb0c49740 1 --2- 172.21.15.83:0/1237675169 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606dc9771b0 0x5606dc97f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:58.059 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.059+0000 7fddb0c49740 1 -- 172.21.15.83:0/1237675169 >> 172.21.15.83:0/1237675169 conn(0x5606dc9736c0 msgr2=0x5606dc979440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:58.059 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.059+0000 7fdd90c00640 0 client.38438 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:07:58.059 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.059+0000 7fddb0c49740 1 -- 172.21.15.83:0/1237675169 shutdown_connections 2024-04-24T09:07:58.059 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.059+0000 7fddb0c49740 1 -- 172.21.15.83:0/1237675169 wait complete. 2024-04-24T09:07:58.059 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.060+0000 7fddb0c49740 20 client.38438 trim_cache size 0 max 16384 2024-04-24T09:07:58.086 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.087+0000 7fddb0c49740 1 Processor -- start 2024-04-24T09:07:58.086 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.087+0000 7fddb0c49740 1 -- start start 2024-04-24T09:07:58.087 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.088+0000 7fddb0c49740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606dc9771b0 0x5606dc98de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:58.087 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.088+0000 7fddb0c49740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5606dc977ad0 0x5606dc981270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:58.087 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.088+0000 7fddb0c49740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606dc982390 0x5606dc99a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:58.087 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.088+0000 7fddb0c49740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdda81b5c30 con 0x5606dc982390 2024-04-24T09:07:58.087 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.088+0000 7fddb0c49740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdda81b7890 con 0x5606dc9771b0 2024-04-24T09:07:58.087 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.088+0000 7fddb0c49740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdda81b6300 con 0x5606dc977ad0 2024-04-24T09:07:58.088 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.088+0000 7fdda7400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606dc982390 0x5606dc99a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:58.088 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.088+0000 7fdda7400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606dc982390 0x5606dc99a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55432/0 (socket says 172.21.15.83:55432) 2024-04-24T09:07:58.088 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.088+0000 7fdda7400640 1 -- 172.21.15.83:0/3292905593 learned_addr learned my addr 172.21.15.83:0/3292905593 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:07:58.088 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.089+0000 7fdda6000640 1 --2- 172.21.15.83:0/3292905593 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5606dc977ad0 0x5606dc981270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:58.088 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.089+0000 7fdda6a00640 1 --2- 172.21.15.83:0/3292905593 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606dc9771b0 0x5606dc98de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:58.088 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.089+0000 7fdda7400640 1 -- 172.21.15.83:0/3292905593 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5606dc977ad0 msgr2=0x5606dc981270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:58.088 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.089+0000 7fdda7400640 1 --2- 172.21.15.83:0/3292905593 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5606dc977ad0 0x5606dc981270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:58.088 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.089+0000 7fdda7400640 1 -- 172.21.15.83:0/3292905593 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606dc9771b0 msgr2=0x5606dc98de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:58.088 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.089+0000 7fdda7400640 1 --2- 172.21.15.83:0/3292905593 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606dc9771b0 0x5606dc98de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:58.088 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.089+0000 7fdda7400640 1 -- 172.21.15.83:0/3292905593 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606dc977900 con 0x5606dc982390 2024-04-24T09:07:58.088 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.089+0000 7fdda6a00640 1 --2- 172.21.15.83:0/3292905593 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606dc9771b0 0x5606dc98de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:07:58.088 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.089+0000 7fdda6000640 1 --2- 172.21.15.83:0/3292905593 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5606dc977ad0 0x5606dc981270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:07:58.089 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.089+0000 7fdda7400640 1 --2- 172.21.15.83:0/3292905593 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606dc982390 0x5606dc99a2f0 secure :-1 s=READY pgs=2274 cs=0 l=1 rev1=1 crypto rx=0x7fdda0008500 tx=0x7fdda0005c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7be79781537ad746 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:58.089 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.090+0000 7fdd93400640 1 -- 172.21.15.83:0/3292905593 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda0014040 con 0x5606dc982390 2024-04-24T09:07:58.089 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.090+0000 7fdd93400640 1 -- 172.21.15.83:0/3292905593 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdda0008850 con 0x5606dc982390 2024-04-24T09:07:58.089 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.090+0000 7fdd93400640 1 -- 172.21.15.83:0/3292905593 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda0005920 con 0x5606dc982390 2024-04-24T09:07:58.090 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.091+0000 7fddb0c49740 1 -- 172.21.15.83:0/3292905593 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606dc982390 msgr2=0x5606dc99a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:58.090 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.091+0000 7fddb0c49740 1 --2- 172.21.15.83:0/3292905593 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606dc982390 0x5606dc99a2f0 secure :-1 s=READY pgs=2274 cs=0 l=1 rev1=1 crypto rx=0x7fdda0008500 tx=0x7fdda0005c70 comp rx=0 tx=0).stop 2024-04-24T09:07:58.090 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.091+0000 7fddb0c49740 1 -- 172.21.15.83:0/3292905593 shutdown_connections 2024-04-24T09:07:58.090 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.091+0000 7fddb0c49740 1 --2- 172.21.15.83:0/3292905593 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606dc982390 0x5606dc99a2f0 unknown :-1 s=CLOSED pgs=2274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:58.090 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.091+0000 7fddb0c49740 1 --2- 172.21.15.83:0/3292905593 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5606dc977ad0 0x5606dc981270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:58.090 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.091+0000 7fddb0c49740 1 --2- 172.21.15.83:0/3292905593 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606dc9771b0 0x5606dc98de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:58.091 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.091+0000 7fddb0c49740 1 -- 172.21.15.83:0/3292905593 >> 172.21.15.83:0/3292905593 conn(0x5606dc9736c0 msgr2=0x5606dc980b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:58.091 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.091+0000 7fddb0c49740 1 -- 172.21.15.83:0/3292905593 shutdown_connections 2024-04-24T09:07:58.091 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.091+0000 7fddb0c49740 1 -- 172.21.15.83:0/3292905593 wait complete. 2024-04-24T09:07:58.091 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.092+0000 7fddb0c49740 1 -- start start 2024-04-24T09:07:58.091 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.092+0000 7fddb0c49740 1 Processor -- start 2024-04-24T09:07:58.092 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.092+0000 7fddb0c49740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606dc9771b0 0x5606dc98a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:58.092 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.093+0000 7fddb0c49740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5606dc977ad0 0x5606dc985970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:58.092 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.093+0000 7fddb0c49740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606dc982390 0x5606dc985eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:58.092 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.093+0000 7fddb0c49740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdda81b7100 con 0x5606dc982390 2024-04-24T09:07:58.092 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.093+0000 7fddb0c49740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdda81b6a00 con 0x5606dc9771b0 2024-04-24T09:07:58.092 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.093+0000 7fddb0c49740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdda81b3c90 con 0x5606dc977ad0 2024-04-24T09:07:58.092 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.093+0000 7fddb0c49740 10 client.0 _ll_register_callbacks cb 0x7fdd94004840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:07:58.093 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.093+0000 7fdda6a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606dc9771b0 0x5606dc98a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:58.093 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.093+0000 7fdda6a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606dc9771b0 0x5606dc98a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:38328/0 (socket says 172.21.15.83:38328) 2024-04-24T09:07:58.093 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.093+0000 7fdda6a00640 1 -- 172.21.15.83:0/1147122762 learned_addr learned my addr 172.21.15.83:0/1147122762 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:07:58.093 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.093+0000 7fdda7400640 1 --2- 172.21.15.83:0/1147122762 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606dc982390 0x5606dc985eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:58.094 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.093+0000 7fdda6000640 1 --2- 172.21.15.83:0/1147122762 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5606dc977ad0 0x5606dc985970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:58.094 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.093+0000 7fdda6a00640 1 -- 172.21.15.83:0/1147122762 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5606dc977ad0 msgr2=0x5606dc985970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:58.094 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.094+0000 7fdda6a00640 1 --2- 172.21.15.83:0/1147122762 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5606dc977ad0 0x5606dc985970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:58.094 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.094+0000 7fdda6a00640 1 -- 172.21.15.83:0/1147122762 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606dc982390 msgr2=0x5606dc985eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:07:58.094 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.094+0000 7fdda6a00640 1 --2- 172.21.15.83:0/1147122762 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606dc982390 0x5606dc985eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:58.094 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.094+0000 7fdda6a00640 1 -- 172.21.15.83:0/1147122762 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606dc6bc620 con 0x5606dc9771b0 2024-04-24T09:07:58.094 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.094+0000 7fdda6a00640 1 --2- 172.21.15.83:0/1147122762 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606dc9771b0 0x5606dc98a540 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fdd98032fd0 tx=0x7fdd98009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9da90f1d159ce44b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:07:58.094 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.094+0000 7fdd90c00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:07:58.094 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.094+0000 7fdd90c00640 1 -- 172.21.15.83:0/1147122762 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd98035050 con 0x5606dc9771b0 2024-04-24T09:07:58.094 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.094+0000 7fddb0c49740 1 -- 172.21.15.83:0/1147122762 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdda0008850 con 0x5606dc9771b0 2024-04-24T09:07:58.094 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.094+0000 7fdd90c00640 1 -- 172.21.15.83:0/1147122762 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9583+0+0 (secure 0 0 0) 0x7fdd98005040 con 0x5606dc9771b0 2024-04-24T09:07:58.095 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.095+0000 7fdd90c00640 1 -- 172.21.15.83:0/1147122762 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fdd9805d050 con 0x5606dc9771b0 2024-04-24T09:07:58.095 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.095+0000 7fddb0c49740 10 client.38441 fetch_fsmap learned FSMap version 52 2024-04-24T09:07:58.095 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.095+0000 7fddb0c49740 1 -- 172.21.15.83:0/1147122762 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x5606dc9f8720 con 0x5606dc9771b0 2024-04-24T09:07:58.095 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.095+0000 7fdd90c00640 1 -- 172.21.15.83:0/1147122762 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdda0008850 con 0x5606dc9771b0 2024-04-24T09:07:58.096 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.095+0000 7fdd90c00640 1 -- 172.21.15.83:0/1147122762 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 52) ==== 8402+0+0 (secure 0 0 0) 0x7fdd980035f0 con 0x5606dc9771b0 2024-04-24T09:07:58.096 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.095+0000 7fddb0c49740 10 client.38441 fetch_fsmap finished waiting for FSMap version 52 2024-04-24T09:07:58.096 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.095+0000 7fddb0c49740 10 client.38441 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:07:58.096 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.095+0000 7fddb0c49740 20 client.38441 populate_metadata read hostname 'smithi083' 2024-04-24T09:07:58.096 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.095+0000 7fddb0c49740 1 --2- 172.21.15.83:0/1147122762 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5606dc9f8f90 0x5606dc9fb3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:07:58.096 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.095+0000 7fddb0c49740 4 client.38441 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T09:07:58.096 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.095+0000 7fddb0c49740 1 -- 172.21.15.83:0/1147122762 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5606dc8e5190 con 0x5606dc9f8f90 2024-04-24T09:07:58.096 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.096+0000 7fdda6000640 1 --2- 172.21.15.83:0/1147122762 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5606dc9f8f90 0x5606dc9fb3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:07:58.096 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.097+0000 7fdda6000640 1 --2- 172.21.15.83:0/1147122762 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5606dc9f8f90 0x5606dc9fb3b0 crc :-1 s=READY pgs=414 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d75a38936b30f4e server_cookie=ef6deb6bf54218f3 in_seq=0 out_seq=0 2024-04-24T09:07:58.096 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.097+0000 7fdd90c00640 10 client.38441 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:07:58.164 INFO:tasks.workunit.client.0.smithi083.stdout: 10 19308 7.09 MB/sec execute 29 sec latency 93.130 ms 2024-04-24T09:07:58.188 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.189+0000 7fdd90c00640 1 -- 172.21.15.83:0/1147122762 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5606dc8e5190 con 0x5606dc9f8f90 2024-04-24T09:07:58.189 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.189+0000 7fdd90c00640 10 client.38441 handle_command_reply: tid=0 2024-04-24T09:07:58.189 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.189+0000 7fddb0c49740 1 client.38441 shutdown 2024-04-24T09:07:58.189 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.189+0000 7fddb0c49740 1 -- 172.21.15.83:0/1147122762 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606dc9771b0 msgr2=0x5606dc98a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:58.189 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.189+0000 7fddb0c49740 1 --2- 172.21.15.83:0/1147122762 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606dc9771b0 0x5606dc98a540 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fdd98032fd0 tx=0x7fdd98009260 comp rx=0 tx=0).stop 2024-04-24T09:07:58.189 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.190+0000 7fddb0c49740 1 -- 172.21.15.83:0/1147122762 shutdown_connections 2024-04-24T09:07:58.189 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.190+0000 7fddb0c49740 1 --2- 172.21.15.83:0/1147122762 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5606dc9f8f90 0x5606dc9fb3b0 crc :-1 s=READY pgs=414 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:58.189 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.190+0000 7fddb0c49740 1 --2- 172.21.15.83:0/1147122762 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606dc982390 0x5606dc985eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:58.190 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.190+0000 7fddb0c49740 1 --2- 172.21.15.83:0/1147122762 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5606dc977ad0 0x5606dc985970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:58.190 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.190+0000 7fddb0c49740 1 --2- 172.21.15.83:0/1147122762 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606dc9771b0 0x5606dc98a540 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:58.190 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.190+0000 7fdd90c00640 0 client.38441 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:07:58.190 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.190+0000 7fddb0c49740 1 -- 172.21.15.83:0/1147122762 >> 172.21.15.83:0/1147122762 conn(0x5606dc9736c0 msgr2=0x5606dc9813f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:58.190 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.190+0000 7fddb0c49740 1 -- 172.21.15.83:0/1147122762 shutdown_connections 2024-04-24T09:07:58.190 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.190+0000 7fddb0c49740 1 -- 172.21.15.83:0/1147122762 wait complete. 2024-04-24T09:07:58.190 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.190+0000 7fddb0c49740 20 client.38441 trim_cache size 0 max 16384 2024-04-24T09:07:58.190 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T09:07:58.190 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T09:07:58.191 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T09:07:58.191 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T09:07:58.191 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.191+0000 7fddade00640 1 -- 172.21.15.83:0/2828679479 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fdd8c0a3e40 msgr2=0x7fdd8c0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:58.191 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.191+0000 7fddade00640 1 --2- 172.21.15.83:0/2828679479 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fdd8c0a3e40 0x7fdd8c0a6300 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fdd98031440 tx=0x7fdd98005a20 comp rx=0 tx=0).stop 2024-04-24T09:07:58.191 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.191+0000 7fddade00640 1 -- 172.21.15.83:0/2828679479 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdda8106220 msgr2=0x7fdda81a0b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:07:58.192 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.191+0000 7fddade00640 1 --2- 172.21.15.83:0/2828679479 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdda8106220 0x7fdda81a0b90 secure :-1 s=READY pgs=2272 cs=0 l=1 rev1=1 crypto rx=0x7fdd9400e9c0 tx=0x7fdd9400ee80 comp rx=0 tx=0).stop 2024-04-24T09:07:58.192 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.191+0000 7fddade00640 1 -- 172.21.15.83:0/2828679479 shutdown_connections 2024-04-24T09:07:58.192 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.191+0000 7fddade00640 1 --2- 172.21.15.83:0/2828679479 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fdd8c0a3e40 0x7fdd8c0a6300 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:58.192 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.191+0000 7fddade00640 1 --2- 172.21.15.83:0/2828679479 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdda819a7c0 0x7fdda819ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:58.192 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.191+0000 7fddade00640 1 --2- 172.21.15.83:0/2828679479 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdda8106220 0x7fdda81a0b90 unknown :-1 s=CLOSED pgs=2272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:58.192 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.191+0000 7fddade00640 1 --2- 172.21.15.83:0/2828679479 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdda81011a0 0x7fdda81a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:07:58.192 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.191+0000 7fddade00640 1 -- 172.21.15.83:0/2828679479 >> 172.21.15.83:0/2828679479 conn(0x7fdda8078080 msgr2=0x7fdda80696b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:07:58.192 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.191+0000 7fddade00640 1 -- 172.21.15.83:0/2828679479 shutdown_connections 2024-04-24T09:07:58.192 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:07:58.191+0000 7fddade00640 1 -- 172.21.15.83:0/2828679479 wait complete. 2024-04-24T09:07:58.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:07:58 smithi083 ceph-mon[27907]: pgmap v513: 129 pgs: 129 active+clean; 425 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 264 op/s 2024-04-24T09:07:58.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:07:58 smithi138 ceph-mon[31441]: pgmap v513: 129 pgs: 129 active+clean; 425 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 264 op/s 2024-04-24T09:07:58.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:07:58 smithi189 ceph-mon[31791]: pgmap v513: 129 pgs: 129 active+clean; 425 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 264 op/s 2024-04-24T09:07:59.164 INFO:tasks.workunit.client.0.smithi083.stdout: 10 19506 7.30 MB/sec execute 30 sec latency 100.349 ms 2024-04-24T09:08:00.164 INFO:tasks.workunit.client.0.smithi083.stdout: 10 19604 7.27 MB/sec execute 31 sec latency 376.608 ms 2024-04-24T09:08:00.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:08:00 smithi083 ceph-mon[27907]: pgmap v514: 129 pgs: 129 active+clean; 439 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.4 MiB/s wr, 353 op/s 2024-04-24T09:08:00.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:08:00 smithi138 ceph-mon[31441]: pgmap v514: 129 pgs: 129 active+clean; 439 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.4 MiB/s wr, 353 op/s 2024-04-24T09:08:00.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:08:00 smithi189 ceph-mon[31791]: pgmap v514: 129 pgs: 129 active+clean; 439 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.4 MiB/s wr, 353 op/s 2024-04-24T09:08:01.164 INFO:tasks.workunit.client.0.smithi083.stdout: 10 19710 7.26 MB/sec execute 32 sec latency 109.591 ms 2024-04-24T09:08:02.164 INFO:tasks.workunit.client.0.smithi083.stdout: 10 19834 7.35 MB/sec execute 33 sec latency 62.536 ms 2024-04-24T09:08:02.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:08:02 smithi083 ceph-mon[27907]: pgmap v515: 129 pgs: 129 active+clean; 439 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 970 KiB/s rd, 2.8 MiB/s wr, 194 op/s 2024-04-24T09:08:02.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:08:02 smithi138 ceph-mon[31441]: pgmap v515: 129 pgs: 129 active+clean; 439 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 970 KiB/s rd, 2.8 MiB/s wr, 194 op/s 2024-04-24T09:08:02.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:08:02 smithi189 ceph-mon[31791]: pgmap v515: 129 pgs: 129 active+clean; 439 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 970 KiB/s rd, 2.8 MiB/s wr, 194 op/s 2024-04-24T09:08:03.164 INFO:tasks.workunit.client.0.smithi083.stdout: 10 20005 7.48 MB/sec execute 34 sec latency 101.244 ms 2024-04-24T09:08:04.164 INFO:tasks.workunit.client.0.smithi083.stdout: 10 20113 7.42 MB/sec execute 35 sec latency 125.637 ms 2024-04-24T09:08:04.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:08:04 smithi138 ceph-mon[31441]: pgmap v516: 129 pgs: 129 active+clean; 452 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.9 MiB/s wr, 325 op/s 2024-04-24T09:08:04.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:08:04 smithi138 ceph-mon[31441]: osdmap e115: 12 total, 12 up, 12 in 2024-04-24T09:08:04.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:08:04 smithi189 ceph-mon[31791]: pgmap v516: 129 pgs: 129 active+clean; 452 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.9 MiB/s wr, 325 op/s 2024-04-24T09:08:04.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:08:04 smithi189 ceph-mon[31791]: osdmap e115: 12 total, 12 up, 12 in 2024-04-24T09:08:04.566 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:08:04 smithi083 ceph-mon[27907]: pgmap v516: 129 pgs: 129 active+clean; 452 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.9 MiB/s wr, 325 op/s 2024-04-24T09:08:04.566 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:08:04 smithi083 ceph-mon[27907]: osdmap e115: 12 total, 12 up, 12 in 2024-04-24T09:08:05.057 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T09:08:05.058 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T09:08:05.164 INFO:tasks.workunit.client.0.smithi083.stdout: 10 20213 7.25 MB/sec execute 36 sec latency 58.606 ms 2024-04-24T09:08:05.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.309+0000 7f96aa400640 1 -- 172.21.15.83:0/3803526878 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96a41044b0 msgr2=0x7f96a4104890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:05.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.309+0000 7f96aa400640 1 --2- 172.21.15.83:0/3803526878 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96a41044b0 0x7f96a4104890 secure :-1 s=READY pgs=2275 cs=0 l=1 rev1=1 crypto rx=0x7f969c009900 tx=0x7f969c02a3f0 comp rx=0 tx=0).stop 2024-04-24T09:08:05.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.309+0000 7f96aa400640 1 -- 172.21.15.83:0/3803526878 shutdown_connections 2024-04-24T09:08:05.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.309+0000 7f96aa400640 1 --2- 172.21.15.83:0/3803526878 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96a4109580 0x7f96a4111150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:05.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.309+0000 7f96aa400640 1 --2- 172.21.15.83:0/3803526878 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f96a4104e60 0x7f96a4108ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:05.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.309+0000 7f96aa400640 1 --2- 172.21.15.83:0/3803526878 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96a41044b0 0x7f96a4104890 unknown :-1 s=CLOSED pgs=2275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:05.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.309+0000 7f96aa400640 1 -- 172.21.15.83:0/3803526878 >> 172.21.15.83:0/3803526878 conn(0x7f96a40ffd90 msgr2=0x7f96a41021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:05.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.310+0000 7f96aa400640 1 -- 172.21.15.83:0/3803526878 shutdown_connections 2024-04-24T09:08:05.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.310+0000 7f96aa400640 1 -- 172.21.15.83:0/3803526878 wait complete. 2024-04-24T09:08:05.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.311+0000 7f96aa400640 1 Processor -- start 2024-04-24T09:08:05.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.311+0000 7f96aa400640 1 -- start start 2024-04-24T09:08:05.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.312+0000 7f96aa400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96a41044b0 0x7f96a4111110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:05.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.312+0000 7f96aa400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96a4109580 0x7f96a410e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:05.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.312+0000 7f96aa400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f96a410e8a0 0x7f96a41ab2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:05.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.312+0000 7f96aa400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f96a4114590 con 0x7f96a41044b0 2024-04-24T09:08:05.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.312+0000 7f96aa400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f96a4114410 con 0x7f96a4109580 2024-04-24T09:08:05.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.312+0000 7f96aa400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f96a4114710 con 0x7f96a410e8a0 2024-04-24T09:08:05.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.312+0000 7f96a3400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96a41044b0 0x7f96a4111110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:05.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.312+0000 7f96a3400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96a41044b0 0x7f96a4111110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55162/0 (socket says 172.21.15.83:55162) 2024-04-24T09:08:05.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.312+0000 7f96a3400640 1 -- 172.21.15.83:0/1949228568 learned_addr learned my addr 172.21.15.83:0/1949228568 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:08:05.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.312+0000 7f96a2a00640 1 --2- 172.21.15.83:0/1949228568 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96a4109580 0x7f96a410e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:05.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.312+0000 7f96a3e00640 1 --2- 172.21.15.83:0/1949228568 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f96a410e8a0 0x7f96a41ab2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:05.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.313+0000 7f96a2a00640 1 -- 172.21.15.83:0/1949228568 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f96a410e8a0 msgr2=0x7f96a41ab2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:05.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.313+0000 7f96a2a00640 1 --2- 172.21.15.83:0/1949228568 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f96a410e8a0 0x7f96a41ab2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:05.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.313+0000 7f96a2a00640 1 -- 172.21.15.83:0/1949228568 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96a41044b0 msgr2=0x7f96a4111110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:05.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.313+0000 7f96a2a00640 1 --2- 172.21.15.83:0/1949228568 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96a41044b0 0x7f96a4111110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:05.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.313+0000 7f96a2a00640 1 -- 172.21.15.83:0/1949228568 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96a41ab810 con 0x7f96a4109580 2024-04-24T09:08:05.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.313+0000 7f96a3400640 1 --2- 172.21.15.83:0/1949228568 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96a41044b0 0x7f96a4111110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:08:05.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.313+0000 7f96a2a00640 1 --2- 172.21.15.83:0/1949228568 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96a4109580 0x7f96a410e250 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f968c00b3c0 tx=0x7f968c00b880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:05.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.314+0000 7f969be00640 1 -- 172.21.15.83:0/1949228568 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f968c011830 con 0x7f96a4109580 2024-04-24T09:08:05.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.314+0000 7f969be00640 1 -- 172.21.15.83:0/1949228568 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f968c016070 con 0x7f96a4109580 2024-04-24T09:08:05.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.314+0000 7f969be00640 1 -- 172.21.15.83:0/1949228568 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f968c010e10 con 0x7f96a4109580 2024-04-24T09:08:05.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.314+0000 7f96aa400640 1 -- 172.21.15.83:0/1949228568 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96a41ab9e0 con 0x7f96a4109580 2024-04-24T09:08:05.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.314+0000 7f96aa400640 1 -- 172.21.15.83:0/1949228568 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96a41abdf0 con 0x7f96a4109580 2024-04-24T09:08:05.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.318+0000 7f96ad1b5740 1 Processor -- start 2024-04-24T09:08:05.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.318+0000 7f96ad1b5740 1 -- start start 2024-04-24T09:08:05.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.318+0000 7f969be00640 1 -- 172.21.15.83:0/1949228568 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f968c0119d0 con 0x7f96a4109580 2024-04-24T09:08:05.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.318+0000 7f96ad1b5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564f111b4860 0x564f111b4c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:05.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.319+0000 7f96ad1b5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564f111b5180 0x564f111bf310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:05.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.319+0000 7f96ad1b5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f111bf850 0x564f111c1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:05.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.319+0000 7f96ad1b5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x564f1111a1f0 con 0x564f111b4860 2024-04-24T09:08:05.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.319+0000 7f96ad1b5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x564f110c0350 con 0x564f111bf850 2024-04-24T09:08:05.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.319+0000 7f96ad1b5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x564f111161b0 con 0x564f111b5180 2024-04-24T09:08:05.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.319+0000 7f96a3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564f111b4860 0x564f111b4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:05.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.319+0000 7f96a3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564f111b4860 0x564f111b4c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55170/0 (socket says 172.21.15.83:55170) 2024-04-24T09:08:05.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.319+0000 7f96a3e00640 1 -- 172.21.15.83:0/429382707 learned_addr learned my addr 172.21.15.83:0/429382707 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:08:05.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.319+0000 7f96a2a00640 1 --2- 172.21.15.83:0/429382707 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f111bf850 0x564f111c1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:05.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.320+0000 7f96a3e00640 1 -- 172.21.15.83:0/429382707 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564f111b5180 msgr2=0x564f111bf310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:05.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.320+0000 7f96a3e00640 1 --2- 172.21.15.83:0/429382707 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564f111b5180 0x564f111bf310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:05.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.320+0000 7f96a3e00640 1 -- 172.21.15.83:0/429382707 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f111bf850 msgr2=0x564f111c1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:05.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.320+0000 7f96a3e00640 1 --2- 172.21.15.83:0/429382707 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f111bf850 0x564f111c1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:05.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.320+0000 7f96a3e00640 1 -- 172.21.15.83:0/429382707 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f11114f50 con 0x564f111b4860 2024-04-24T09:08:05.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.320+0000 7f969be00640 1 --2- 172.21.15.83:0/1949228568 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f96880a3e70 0x7f96880a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:05.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.321+0000 7f96a3e00640 1 --2- 172.21.15.83:0/429382707 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564f111b4860 0x564f111b4c40 secure :-1 s=READY pgs=2276 cs=0 l=1 rev1=1 crypto rx=0x7f9694010370 tx=0x7f9694010830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e878f9581a93dc08 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:05.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.321+0000 7f969be00640 1 -- 172.21.15.83:0/1949228568 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9671+0+0 (secure 0 0 0) 0x7f968c0d00a0 con 0x7f96a4109580 2024-04-24T09:08:05.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.321+0000 7f96a3400640 1 --2- 172.21.15.83:0/1949228568 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f96880a3e70 0x7f96880a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:05.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.321+0000 7f9698c00640 1 -- 172.21.15.83:0/429382707 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9694016070 con 0x564f111b4860 2024-04-24T09:08:05.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.321+0000 7f9698c00640 1 -- 172.21.15.83:0/429382707 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9694004d20 con 0x564f111b4860 2024-04-24T09:08:05.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.321+0000 7f9698c00640 1 -- 172.21.15.83:0/429382707 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96940053c0 con 0x564f111b4860 2024-04-24T09:08:05.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.322+0000 7f96ad1b5740 1 -- 172.21.15.83:0/429382707 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564f111b4860 msgr2=0x564f111b4c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:05.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.322+0000 7f96ad1b5740 1 --2- 172.21.15.83:0/429382707 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564f111b4860 0x564f111b4c40 secure :-1 s=READY pgs=2276 cs=0 l=1 rev1=1 crypto rx=0x7f9694010370 tx=0x7f9694010830 comp rx=0 tx=0).stop 2024-04-24T09:08:05.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.322+0000 7f96ad1b5740 1 -- 172.21.15.83:0/429382707 shutdown_connections 2024-04-24T09:08:05.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.322+0000 7f96ad1b5740 1 --2- 172.21.15.83:0/429382707 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f111bf850 0x564f111c1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:05.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.322+0000 7f96ad1b5740 1 --2- 172.21.15.83:0/429382707 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564f111b5180 0x564f111bf310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:05.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.322+0000 7f96ad1b5740 1 --2- 172.21.15.83:0/429382707 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564f111b4860 0x564f111b4c40 unknown :-1 s=CLOSED pgs=2276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:05.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.322+0000 7f96ad1b5740 1 -- 172.21.15.83:0/429382707 >> 172.21.15.83:0/429382707 conn(0x564f111b0b80 msgr2=0x564f111b0f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:05.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.322+0000 7f96a3400640 1 --2- 172.21.15.83:0/1949228568 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f96880a3e70 0x7f96880a6330 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x564f111b5e20 tx=0x7f969c009580 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:05.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.322+0000 7f96ad1b5740 1 -- 172.21.15.83:0/429382707 shutdown_connections 2024-04-24T09:08:05.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.322+0000 7f96ad1b5740 1 -- 172.21.15.83:0/429382707 wait complete. 2024-04-24T09:08:05.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.323+0000 7f96ad1b5740 1 -- start start 2024-04-24T09:08:05.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.323+0000 7f96ad1b5740 1 Processor -- start 2024-04-24T09:08:05.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.324+0000 7f96ad1b5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f111b4860 0x564f111bc840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:05.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.324+0000 7f96ad1b5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564f111b5180 0x564f111bcd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:05.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.324+0000 7f96ad1b5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564f111bf850 0x564f111bd2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:05.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.324+0000 7f96ad1b5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x564f111c2640 con 0x564f111b5180 2024-04-24T09:08:05.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.324+0000 7f96ad1b5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x564f110d3690 con 0x564f111b4860 2024-04-24T09:08:05.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.324+0000 7f96ad1b5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x564f111c27c0 con 0x564f111bf850 2024-04-24T09:08:05.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.324+0000 7f96a3400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f111b4860 0x564f111bc840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:05.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.324+0000 7f96ad1b5740 10 client.0 _ll_register_callbacks cb 0x564f1110f650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:08:05.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.324+0000 7f96a3400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f111b4860 0x564f111bc840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:58368/0 (socket says 172.21.15.83:58368) 2024-04-24T09:08:05.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.324+0000 7f96a3400640 1 -- 172.21.15.83:0/2150334478 learned_addr learned my addr 172.21.15.83:0/2150334478 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:08:05.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.324+0000 7f96a2a00640 1 --2- 172.21.15.83:0/2150334478 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564f111b5180 0x564f111bcd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:05.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.324+0000 7f96a3e00640 1 --2- 172.21.15.83:0/2150334478 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564f111bf850 0x564f111bd2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:05.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.325+0000 7f96a3400640 1 -- 172.21.15.83:0/2150334478 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564f111bf850 msgr2=0x564f111bd2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:05.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.325+0000 7f96a3400640 1 --2- 172.21.15.83:0/2150334478 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564f111bf850 0x564f111bd2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:05.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.325+0000 7f96a3400640 1 -- 172.21.15.83:0/2150334478 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564f111b5180 msgr2=0x564f111bcd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:05.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.325+0000 7f96a3400640 1 --2- 172.21.15.83:0/2150334478 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564f111b5180 0x564f111bcd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:05.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.325+0000 7f96a3400640 1 -- 172.21.15.83:0/2150334478 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f110d2920 con 0x564f111b4860 2024-04-24T09:08:05.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.325+0000 7f96a2a00640 1 --2- 172.21.15.83:0/2150334478 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564f111b5180 0x564f111bcd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:08:05.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.326+0000 7f96a3400640 1 --2- 172.21.15.83:0/2150334478 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f111b4860 0x564f111bc840 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f969c02a9e0 tx=0x7f969c02a3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28636f8c8ed3d0d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:05.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.326+0000 7f9691600640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:08:05.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.326+0000 7f9691600640 1 -- 172.21.15.83:0/2150334478 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f969c04d020 con 0x564f111b4860 2024-04-24T09:08:05.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.326+0000 7f9691600640 1 -- 172.21.15.83:0/2150334478 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9671+0+0 (secure 0 0 0) 0x7f969c031a30 con 0x564f111b4860 2024-04-24T09:08:05.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.326+0000 7f96ad1b5740 1 -- 172.21.15.83:0/2150334478 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9694004d20 con 0x564f111b4860 2024-04-24T09:08:05.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.326+0000 7f9691600640 1 -- 172.21.15.83:0/2150334478 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f969c058050 con 0x564f111b4860 2024-04-24T09:08:05.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.327+0000 7f96ad1b5740 10 client.38453 fetch_fsmap learned FSMap version 52 2024-04-24T09:08:05.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.327+0000 7f96ad1b5740 1 -- 172.21.15.83:0/2150334478 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x564f10ef67d0 con 0x564f111b4860 2024-04-24T09:08:05.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.327+0000 7f9691600640 1 -- 172.21.15.83:0/2150334478 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9694004d20 con 0x564f111b4860 2024-04-24T09:08:05.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.327+0000 7f9691600640 1 -- 172.21.15.83:0/2150334478 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 52) ==== 8402+0+0 (secure 0 0 0) 0x7f969c049cd0 con 0x564f111b4860 2024-04-24T09:08:05.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.327+0000 7f96ad1b5740 10 client.38453 fetch_fsmap finished waiting for FSMap version 52 2024-04-24T09:08:05.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.328+0000 7f96ad1b5740 10 client.38453 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:08:05.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.328+0000 7f96ad1b5740 20 client.38453 populate_metadata read hostname 'smithi083' 2024-04-24T09:08:05.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.328+0000 7f96ad1b5740 1 --2- 172.21.15.83:0/2150334478 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x564f111cfcd0 0x564f111d2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:05.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.328+0000 7f96ad1b5740 4 client.38453 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:08:05.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.328+0000 7f96ad1b5740 1 -- 172.21.15.83:0/2150334478 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564f11127c10 con 0x564f111cfcd0 2024-04-24T09:08:05.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.328+0000 7f96a2a00640 1 --2- 172.21.15.83:0/2150334478 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x564f111cfcd0 0x564f111d2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:05.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.329+0000 7f96a2a00640 1 --2- 172.21.15.83:0/2150334478 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x564f111cfcd0 0x564f111d2100 crc :-1 s=READY pgs=415 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79d725ff50a47f94 server_cookie=61695841109e71f4 in_seq=0 out_seq=0 2024-04-24T09:08:05.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.329+0000 7f9691600640 10 client.38453 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:08:05.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.330+0000 7f9691600640 1 -- 172.21.15.83:0/2150334478 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564f11127c10 con 0x564f111cfcd0 2024-04-24T09:08:05.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.330+0000 7f9691600640 10 client.38453 handle_command_reply: tid=0 2024-04-24T09:08:05.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.330+0000 7f96ad1b5740 1 client.38453 shutdown 2024-04-24T09:08:05.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.330+0000 7f96ad1b5740 1 -- 172.21.15.83:0/2150334478 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f111b4860 msgr2=0x564f111bc840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:05.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.330+0000 7f96ad1b5740 1 --2- 172.21.15.83:0/2150334478 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f111b4860 0x564f111bc840 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f969c02a9e0 tx=0x7f969c02a3f0 comp rx=0 tx=0).stop 2024-04-24T09:08:05.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.330+0000 7f96ad1b5740 1 -- 172.21.15.83:0/2150334478 shutdown_connections 2024-04-24T09:08:05.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.330+0000 7f96ad1b5740 1 --2- 172.21.15.83:0/2150334478 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x564f111cfcd0 0x564f111d2100 crc :-1 s=READY pgs=415 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:05.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.330+0000 7f96ad1b5740 1 --2- 172.21.15.83:0/2150334478 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564f111bf850 0x564f111bd2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:05.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.330+0000 7f96ad1b5740 1 --2- 172.21.15.83:0/2150334478 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564f111b5180 0x564f111bcd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:05.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.330+0000 7f9691600640 0 client.38453 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:08:05.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.330+0000 7f96ad1b5740 1 --2- 172.21.15.83:0/2150334478 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f111b4860 0x564f111bc840 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:05.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.330+0000 7f96ad1b5740 1 -- 172.21.15.83:0/2150334478 >> 172.21.15.83:0/2150334478 conn(0x564f111b0b80 msgr2=0x564f111b6980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:05.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.331+0000 7f96ad1b5740 1 -- 172.21.15.83:0/2150334478 shutdown_connections 2024-04-24T09:08:05.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.331+0000 7f96ad1b5740 1 -- 172.21.15.83:0/2150334478 wait complete. 2024-04-24T09:08:05.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.331+0000 7f96ad1b5740 20 client.38453 trim_cache size 0 max 16384 2024-04-24T09:08:05.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.360+0000 7f96ad1b5740 1 Processor -- start 2024-04-24T09:08:05.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.360+0000 7f96ad1b5740 1 -- start start 2024-04-24T09:08:05.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.360+0000 7f96ad1b5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564f111b4860 0x564f111bb780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:05.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.360+0000 7f96ad1b5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564f111b5180 0x564f111cb500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:05.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.360+0000 7f96ad1b5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f111bf850 0x564f111b7000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:05.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.360+0000 7f96ad1b5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f96a41b5e50 con 0x564f111b5180 2024-04-24T09:08:05.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.360+0000 7f96ad1b5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f96a41b7ab0 con 0x564f111bf850 2024-04-24T09:08:05.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.360+0000 7f96ad1b5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f96a41b6520 con 0x564f111b4860 2024-04-24T09:08:05.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.361+0000 7f96a2a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564f111b5180 0x564f111cb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:05.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.361+0000 7f96a2a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564f111b5180 0x564f111cb500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55190/0 (socket says 172.21.15.83:55190) 2024-04-24T09:08:05.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.361+0000 7f96a2a00640 1 -- 172.21.15.83:0/3493587199 learned_addr learned my addr 172.21.15.83:0/3493587199 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:08:05.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.361+0000 7f96a3e00640 1 --2- 172.21.15.83:0/3493587199 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f111bf850 0x564f111b7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:05.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.361+0000 7f96a3400640 1 --2- 172.21.15.83:0/3493587199 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564f111b4860 0x564f111bb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:05.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.361+0000 7f96a2a00640 1 -- 172.21.15.83:0/3493587199 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564f111b4860 msgr2=0x564f111bb780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:05.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.361+0000 7f96a2a00640 1 --2- 172.21.15.83:0/3493587199 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564f111b4860 0x564f111bb780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:05.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.361+0000 7f96a2a00640 1 -- 172.21.15.83:0/3493587199 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f111bf850 msgr2=0x564f111b7000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:05.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.361+0000 7f96a2a00640 1 --2- 172.21.15.83:0/3493587199 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f111bf850 0x564f111b7000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:05.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.361+0000 7f96a2a00640 1 -- 172.21.15.83:0/3493587199 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f10e2c3f0 con 0x564f111b5180 2024-04-24T09:08:05.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.361+0000 7f96a3400640 1 --2- 172.21.15.83:0/3493587199 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564f111b4860 0x564f111bb780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:08:05.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.362+0000 7f96a2a00640 1 --2- 172.21.15.83:0/3493587199 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564f111b5180 0x564f111cb500 secure :-1 s=READY pgs=2277 cs=0 l=1 rev1=1 crypto rx=0x7f968c0cd480 tx=0x7f968c013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd9948175e3640ad server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:05.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.362+0000 7f9693e00640 1 -- 172.21.15.83:0/3493587199 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f968c09b5d0 con 0x564f111b5180 2024-04-24T09:08:05.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.362+0000 7f9693e00640 1 -- 172.21.15.83:0/3493587199 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f968c010510 con 0x564f111b5180 2024-04-24T09:08:05.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.362+0000 7f9693e00640 1 -- 172.21.15.83:0/3493587199 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f968c0982e0 con 0x564f111b5180 2024-04-24T09:08:05.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.363+0000 7f96ad1b5740 1 -- 172.21.15.83:0/3493587199 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564f111b5180 msgr2=0x564f111cb500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:05.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.363+0000 7f96ad1b5740 1 --2- 172.21.15.83:0/3493587199 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564f111b5180 0x564f111cb500 secure :-1 s=READY pgs=2277 cs=0 l=1 rev1=1 crypto rx=0x7f968c0cd480 tx=0x7f968c013040 comp rx=0 tx=0).stop 2024-04-24T09:08:05.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.363+0000 7f96ad1b5740 1 -- 172.21.15.83:0/3493587199 shutdown_connections 2024-04-24T09:08:05.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.363+0000 7f96ad1b5740 1 --2- 172.21.15.83:0/3493587199 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f111bf850 0x564f111b7000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:05.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.363+0000 7f96ad1b5740 1 --2- 172.21.15.83:0/3493587199 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564f111b5180 0x564f111cb500 unknown :-1 s=CLOSED pgs=2277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:05.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.363+0000 7f96ad1b5740 1 --2- 172.21.15.83:0/3493587199 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564f111b4860 0x564f111bb780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:05.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.363+0000 7f96ad1b5740 1 -- 172.21.15.83:0/3493587199 >> 172.21.15.83:0/3493587199 conn(0x564f111b0b80 msgr2=0x564f111afc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:05.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.363+0000 7f96ad1b5740 1 -- 172.21.15.83:0/3493587199 shutdown_connections 2024-04-24T09:08:05.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.363+0000 7f96ad1b5740 1 -- 172.21.15.83:0/3493587199 wait complete. 2024-04-24T09:08:05.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.364+0000 7f96ad1b5740 1 -- start start 2024-04-24T09:08:05.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.364+0000 7f96ad1b5740 1 Processor -- start 2024-04-24T09:08:05.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.364+0000 7f96ad1b5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f111b4860 0x564f111c7de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:05.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.364+0000 7f96ad1b5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564f111b5180 0x564f111b91f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:05.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.364+0000 7f96ad1b5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564f111bf850 0x564f111b9730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:05.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.364+0000 7f96ad1b5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f96a41b7320 con 0x564f111b5180 2024-04-24T09:08:05.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.364+0000 7f96ad1b5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f96a41b6c20 con 0x564f111b4860 2024-04-24T09:08:05.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.364+0000 7f96ad1b5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f96a41b3e50 con 0x564f111bf850 2024-04-24T09:08:05.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.365+0000 7f96ad1b5740 10 client.0 _ll_register_callbacks cb 0x7f969c040070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:08:05.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.365+0000 7f96a2a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564f111b5180 0x564f111b91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:05.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.365+0000 7f96a3400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f111b4860 0x564f111c7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:05.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.365+0000 7f96a3400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f111b4860 0x564f111c7de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:58378/0 (socket says 172.21.15.83:58378) 2024-04-24T09:08:05.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.365+0000 7f96a3400640 1 -- 172.21.15.83:0/1777260494 learned_addr learned my addr 172.21.15.83:0/1777260494 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:08:05.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.365+0000 7f96a2a00640 1 -- 172.21.15.83:0/1777260494 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564f111bf850 msgr2=0x564f111b9730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:05.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.365+0000 7f96a2a00640 1 --2- 172.21.15.83:0/1777260494 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564f111bf850 0x564f111b9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:05.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.365+0000 7f96a2a00640 1 -- 172.21.15.83:0/1777260494 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f111b4860 msgr2=0x564f111c7de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:05.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.365+0000 7f96a2a00640 1 --2- 172.21.15.83:0/1777260494 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f111b4860 0x564f111c7de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:05.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.365+0000 7f96a2a00640 1 -- 172.21.15.83:0/1777260494 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f10dec0f0 con 0x564f111b5180 2024-04-24T09:08:05.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.366+0000 7f96a2a00640 1 --2- 172.21.15.83:0/1777260494 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564f111b5180 0x564f111b91f0 secure :-1 s=READY pgs=2278 cs=0 l=1 rev1=1 crypto rx=0x7f968c0cd480 tx=0x7f968c0a0110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22d825a32dbdf128 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:05.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.366+0000 7f9691600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:08:05.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.366+0000 7f9691600640 1 -- 172.21.15.83:0/1777260494 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f968c09bc70 con 0x564f111b5180 2024-04-24T09:08:05.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.366+0000 7f9691600640 1 -- 172.21.15.83:0/1777260494 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9671+0+0 (secure 0 0 0) 0x7f968c0cd590 con 0x564f111b5180 2024-04-24T09:08:05.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.366+0000 7f96ad1b5740 1 -- 172.21.15.83:0/1777260494 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564f11121a80 con 0x564f111b5180 2024-04-24T09:08:05.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.367+0000 7f9691600640 1 -- 172.21.15.83:0/1777260494 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f968c0a48e0 con 0x564f111b5180 2024-04-24T09:08:05.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.367+0000 7f96ad1b5740 10 client.21315 fetch_fsmap learned FSMap version 52 2024-04-24T09:08:05.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.367+0000 7f96ad1b5740 1 -- 172.21.15.83:0/1777260494 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x564f110df810 con 0x564f111b5180 2024-04-24T09:08:05.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.367+0000 7f9691600640 1 -- 172.21.15.83:0/1777260494 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x564f11121a80 con 0x564f111b5180 2024-04-24T09:08:05.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.367+0000 7f9691600640 1 -- 172.21.15.83:0/1777260494 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 52) ==== 8402+0+0 (secure 0 0 0) 0x7f968c0979c0 con 0x564f111b5180 2024-04-24T09:08:05.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.368+0000 7f96ad1b5740 10 client.21315 fetch_fsmap finished waiting for FSMap version 52 2024-04-24T09:08:05.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.368+0000 7f96ad1b5740 10 client.21315 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:08:05.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.368+0000 7f96ad1b5740 20 client.21315 populate_metadata read hostname 'smithi083' 2024-04-24T09:08:05.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.368+0000 7f96ad1b5740 1 --2- 172.21.15.83:0/1777260494 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x564f11236460 0x564f11238880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:05.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.368+0000 7f96ad1b5740 4 client.21315 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T09:08:05.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.368+0000 7f96ad1b5740 1 -- 172.21.15.83:0/1777260494 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564f1109d360 con 0x564f11236460 2024-04-24T09:08:05.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.368+0000 7f96a3400640 1 --2- 172.21.15.83:0/1777260494 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x564f11236460 0x564f11238880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:05.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.369+0000 7f9691600640 10 client.21315 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:08:05.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.369+0000 7f96a3400640 1 --2- 172.21.15.83:0/1777260494 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x564f11236460 0x564f11238880 crc :-1 s=READY pgs=416 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ad68fa00d86ce9e server_cookie=8b74758ce7d37c59 in_seq=0 out_seq=0 2024-04-24T09:08:05.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.369+0000 7f9691600640 1 -- 172.21.15.83:0/1777260494 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x564f1109d360 con 0x564f11236460 2024-04-24T09:08:05.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.369+0000 7f9691600640 10 client.21315 handle_command_reply: tid=0 2024-04-24T09:08:05.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.369+0000 7f96ad1b5740 1 client.21315 shutdown 2024-04-24T09:08:05.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.369+0000 7f96ad1b5740 1 -- 172.21.15.83:0/1777260494 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564f111b5180 msgr2=0x564f111b91f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:05.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.369+0000 7f96ad1b5740 1 --2- 172.21.15.83:0/1777260494 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564f111b5180 0x564f111b91f0 secure :-1 s=READY pgs=2278 cs=0 l=1 rev1=1 crypto rx=0x7f968c0cd480 tx=0x7f968c0a0110 comp rx=0 tx=0).stop 2024-04-24T09:08:05.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.370+0000 7f96ad1b5740 1 -- 172.21.15.83:0/1777260494 shutdown_connections 2024-04-24T09:08:05.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.370+0000 7f96ad1b5740 1 --2- 172.21.15.83:0/1777260494 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x564f11236460 0x564f11238880 crc :-1 s=READY pgs=416 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:05.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.370+0000 7f96ad1b5740 1 --2- 172.21.15.83:0/1777260494 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564f111bf850 0x564f111b9730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:05.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.370+0000 7f96ad1b5740 1 --2- 172.21.15.83:0/1777260494 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564f111b5180 0x564f111b91f0 unknown :-1 s=CLOSED pgs=2278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:05.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.370+0000 7f96ad1b5740 1 --2- 172.21.15.83:0/1777260494 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f111b4860 0x564f111c7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:05.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.370+0000 7f96ad1b5740 1 -- 172.21.15.83:0/1777260494 >> 172.21.15.83:0/1777260494 conn(0x564f111b0b80 msgr2=0x564f111bc730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:05.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.370+0000 7f9691600640 0 client.21315 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:08:05.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.370+0000 7f96ad1b5740 1 -- 172.21.15.83:0/1777260494 shutdown_connections 2024-04-24T09:08:05.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.370+0000 7f96ad1b5740 1 -- 172.21.15.83:0/1777260494 wait complete. 2024-04-24T09:08:05.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.370+0000 7f96ad1b5740 20 client.21315 trim_cache size 0 max 16384 2024-04-24T09:08:05.371 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T09:08:05.371 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-04-24T09:08:05.371 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-04-24T09:08:05.371 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T09:08:05.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.371+0000 7f96aa400640 1 -- 172.21.15.83:0/1949228568 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f96880a3e70 msgr2=0x7f96880a6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:05.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.371+0000 7f96aa400640 1 --2- 172.21.15.83:0/1949228568 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f96880a3e70 0x7f96880a6330 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x564f111b5e20 tx=0x7f969c009580 comp rx=0 tx=0).stop 2024-04-24T09:08:05.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.371+0000 7f96aa400640 1 -- 172.21.15.83:0/1949228568 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96a4109580 msgr2=0x7f96a410e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:05.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.371+0000 7f96aa400640 1 --2- 172.21.15.83:0/1949228568 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96a4109580 0x7f96a410e250 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f968c00b3c0 tx=0x7f968c00b880 comp rx=0 tx=0).stop 2024-04-24T09:08:05.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.371+0000 7f96aa400640 1 -- 172.21.15.83:0/1949228568 shutdown_connections 2024-04-24T09:08:05.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.371+0000 7f96aa400640 1 --2- 172.21.15.83:0/1949228568 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f96880a3e70 0x7f96880a6330 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:05.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.371+0000 7f96aa400640 1 --2- 172.21.15.83:0/1949228568 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f96a410e8a0 0x7f96a41ab2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:05.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.371+0000 7f96aa400640 1 --2- 172.21.15.83:0/1949228568 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96a4109580 0x7f96a410e250 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:05.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.371+0000 7f96aa400640 1 --2- 172.21.15.83:0/1949228568 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96a41044b0 0x7f96a4111110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:05.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.371+0000 7f96aa400640 1 -- 172.21.15.83:0/1949228568 >> 172.21.15.83:0/1949228568 conn(0x7f96a40ffd90 msgr2=0x7f96a4101160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:05.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.371+0000 7f96aa400640 1 -- 172.21.15.83:0/1949228568 shutdown_connections 2024-04-24T09:08:05.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.371+0000 7f96aa400640 1 -- 172.21.15.83:0/1949228568 wait complete. 2024-04-24T09:08:05.389 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T09:08:05.390 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T09:08:05.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.644+0000 7f1c5fe00640 1 -- 172.21.15.83:0/4194653710 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c60102a40 msgr2=0x7f1c60106a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:05.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.644+0000 7f1c5fe00640 1 --2- 172.21.15.83:0/4194653710 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c60102a40 0x7f1c60106a80 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1c48009900 tx=0x7f1c4802a3f0 comp rx=0 tx=0).stop 2024-04-24T09:08:05.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.644+0000 7f1c5fe00640 1 -- 172.21.15.83:0/4194653710 shutdown_connections 2024-04-24T09:08:05.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.644+0000 7f1c5fe00640 1 --2- 172.21.15.83:0/4194653710 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1c60107160 0x7f1c6010ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:05.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.644+0000 7f1c5fe00640 1 --2- 172.21.15.83:0/4194653710 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c60102a40 0x7f1c60106a80 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:05.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.644+0000 7f1c5fe00640 1 --2- 172.21.15.83:0/4194653710 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c60102090 0x7f1c60102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:05.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.644+0000 7f1c5fe00640 1 -- 172.21.15.83:0/4194653710 >> 172.21.15.83:0/4194653710 conn(0x7f1c600fd930 msgr2=0x7f1c600ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:05.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.644+0000 7f1c5fe00640 1 -- 172.21.15.83:0/4194653710 shutdown_connections 2024-04-24T09:08:05.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.644+0000 7f1c5fe00640 1 -- 172.21.15.83:0/4194653710 wait complete. 2024-04-24T09:08:05.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.645+0000 7f1c5fe00640 1 Processor -- start 2024-04-24T09:08:05.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.645+0000 7f1c5fe00640 1 -- start start 2024-04-24T09:08:05.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.646+0000 7f1c5fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c60102090 0x7f1c601a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:05.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.646+0000 7f1c5fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c60107160 0x7f1c601a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:05.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.646+0000 7f1c5fe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1c6019a720 0x7f1c6019aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:05.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.646+0000 7f1c5fe00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1c60112080 con 0x7f1c60107160 2024-04-24T09:08:05.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.646+0000 7f1c5fe00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1c60111f00 con 0x7f1c60102090 2024-04-24T09:08:05.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.646+0000 7f1c5fe00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1c60112200 con 0x7f1c6019a720 2024-04-24T09:08:05.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.646+0000 7f1c5ea00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c60102090 0x7f1c601a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:05.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.646+0000 7f1c5ea00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c60102090 0x7f1c601a0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:58392/0 (socket says 172.21.15.83:58392) 2024-04-24T09:08:05.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.646+0000 7f1c5ea00640 1 -- 172.21.15.83:0/3810335009 learned_addr learned my addr 172.21.15.83:0/3810335009 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:08:05.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.646+0000 7f1c56000640 1 --2- 172.21.15.83:0/3810335009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c60107160 0x7f1c601a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:05.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.647+0000 7f1c5f400640 1 --2- 172.21.15.83:0/3810335009 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1c6019a720 0x7f1c6019aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:05.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.647+0000 7f1c5ea00640 1 -- 172.21.15.83:0/3810335009 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1c6019a720 msgr2=0x7f1c6019aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:05.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.647+0000 7f1c5ea00640 1 --2- 172.21.15.83:0/3810335009 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1c6019a720 0x7f1c6019aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:05.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.647+0000 7f1c5ea00640 1 -- 172.21.15.83:0/3810335009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c60107160 msgr2=0x7f1c601a0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:05.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.647+0000 7f1c5ea00640 1 --2- 172.21.15.83:0/3810335009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c60107160 0x7f1c601a0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:05.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.647+0000 7f1c5ea00640 1 -- 172.21.15.83:0/3810335009 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c6019b320 con 0x7f1c60102090 2024-04-24T09:08:05.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.647+0000 7f1c5f400640 1 --2- 172.21.15.83:0/3810335009 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1c6019a720 0x7f1c6019aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:08:05.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.647+0000 7f1c5ea00640 1 --2- 172.21.15.83:0/3810335009 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c60102090 0x7f1c601a0660 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1c5000ebc0 tx=0x7f1c5000c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:05.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.648+0000 7f1c57e00640 1 -- 172.21.15.83:0/3810335009 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c50010630 con 0x7f1c60102090 2024-04-24T09:08:05.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.648+0000 7f1c57e00640 1 -- 172.21.15.83:0/3810335009 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1c50010c80 con 0x7f1c60102090 2024-04-24T09:08:05.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.648+0000 7f1c57e00640 1 -- 172.21.15.83:0/3810335009 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c50018b30 con 0x7f1c60102090 2024-04-24T09:08:05.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.648+0000 7f1c5fe00640 1 -- 172.21.15.83:0/3810335009 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c601a7400 con 0x7f1c60102090 2024-04-24T09:08:05.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.648+0000 7f1c5fe00640 1 -- 172.21.15.83:0/3810335009 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c601a77b0 con 0x7f1c60102090 2024-04-24T09:08:05.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.649+0000 7f1c5fe00640 1 -- 172.21.15.83:0/3810335009 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c60103560 con 0x7f1c60102090 2024-04-24T09:08:05.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.652+0000 7f1c57e00640 1 -- 172.21.15.83:0/3810335009 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1c50018cd0 con 0x7f1c60102090 2024-04-24T09:08:05.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.654+0000 7f1c57e00640 1 --2- 172.21.15.83:0/3810335009 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f1c440a3da0 0x7f1c440a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:05.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.654+0000 7f1c57e00640 1 -- 172.21.15.83:0/3810335009 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9671+0+0 (secure 0 0 0) 0x7f1c500d1280 con 0x7f1c60102090 2024-04-24T09:08:05.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.654+0000 7f1c56000640 1 --2- 172.21.15.83:0/3810335009 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f1c440a3da0 0x7f1c440a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:05.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.655+0000 7f1c56000640 1 --2- 172.21.15.83:0/3810335009 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f1c440a3da0 0x7f1c440a6260 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f1c48000c00 tx=0x7f1c48031400 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:05.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:05.661+0000 7f1c57e00640 1 -- 172.21.15.83:0/3810335009 <== mon.2 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1c5009cdf0 con 0x7f1c60102090 2024-04-24T09:08:06.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.092+0000 7f1c5fe00640 1 -- 172.21.15.83:0/3810335009 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1c60068d40 con 0x7f1c60102090 2024-04-24T09:08:06.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.093+0000 7f1c57e00640 1 -- 172.21.15.83:0/3810335009 <== mon.2 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) ==== 94+0+11958 (secure 0 0 0) 0x7f1c60068d40 con 0x7f1c60102090 2024-04-24T09:08:06.093 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T09:08:06.093 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":53,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/358872257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":358872257},{"type":"v1","addr":"172.21.15.83:6835","nonce":358872257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3157070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3157070851},{"type":"v1","addr":"172.21.15.83:6837","nonce":3157070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":53,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T08:48:29.422576+0000","modified":"2024-04-24T09:08:05.893450+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34520,"mds_1":34493,"mds_2":34451,"mds_3":34427,"mds_4":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14658":{"gid":14658,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6839/4085470844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":4085470844},{"type":"v1","addr":"172.21.15.83:6839","nonce":4085470844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14676":{"gid":14676,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6841/2634754210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":2634754210},{"type":"v1","addr":"172.21.15.83:6841","nonce":2634754210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24382":{"gid":24382,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/1859921310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1859921310},{"type":"v1","addr":"172.21.15.189:6833","nonce":1859921310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/2241923197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2241923197},{"type":"v1","addr":"172.21.15.189:6835","nonce":2241923197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.189:6839/2548965336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":2548965336},{"type":"v1","addr":"172.21.15.189:6839","nonce":2548965336}]},"join_fscid":-1,"export_targets":[0,1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26686":{"gid":26686,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6837/1678817161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":1678817161},{"type":"v1","addr":"172.21.15.189:6837","nonce":1678817161}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.138:6833/1157967658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1157967658},{"type":"v1","addr":"172.21.15.138:6833","nonce":1157967658}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.138:6835/1909219758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1909219758},{"type":"v1","addr":"172.21.15.138:6835","nonce":1909219758}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34493":{"gid":34493,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.138:6837/900566754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":900566754},{"type":"v1","addr":"172.21.15.138:6837","nonce":900566754}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34520":{"gid":34520,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.138:6839/2236932393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2236932393},{"type":"v1","addr":"172.21.15.138:6839","nonce":2236932393}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34520,"qdb_cluster":[24454,34520,34427,34451,34493]},"id":1}]} 2024-04-24T09:08:06.093 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 53 2024-04-24T09:08:06.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.097+0000 7f1c5fe00640 1 -- 172.21.15.83:0/3810335009 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f1c440a3da0 msgr2=0x7f1c440a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:06.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.097+0000 7f1c5fe00640 1 --2- 172.21.15.83:0/3810335009 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f1c440a3da0 0x7f1c440a6260 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f1c48000c00 tx=0x7f1c48031400 comp rx=0 tx=0).stop 2024-04-24T09:08:06.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.097+0000 7f1c5fe00640 1 -- 172.21.15.83:0/3810335009 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c60102090 msgr2=0x7f1c601a0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:06.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.097+0000 7f1c5fe00640 1 --2- 172.21.15.83:0/3810335009 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c60102090 0x7f1c601a0660 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1c5000ebc0 tx=0x7f1c5000c6a0 comp rx=0 tx=0).stop 2024-04-24T09:08:06.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.097+0000 7f1c5fe00640 1 -- 172.21.15.83:0/3810335009 shutdown_connections 2024-04-24T09:08:06.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.097+0000 7f1c5fe00640 1 --2- 172.21.15.83:0/3810335009 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f1c440a3da0 0x7f1c440a6260 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:06.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.097+0000 7f1c5fe00640 1 --2- 172.21.15.83:0/3810335009 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1c6019a720 0x7f1c6019aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:06.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.097+0000 7f1c5fe00640 1 --2- 172.21.15.83:0/3810335009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c60107160 0x7f1c601a0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:06.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.097+0000 7f1c5fe00640 1 --2- 172.21.15.83:0/3810335009 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c60102090 0x7f1c601a0660 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:06.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.097+0000 7f1c5fe00640 1 -- 172.21.15.83:0/3810335009 >> 172.21.15.83:0/3810335009 conn(0x7f1c600fd930 msgr2=0x7f1c600ffd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:06.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.097+0000 7f1c5fe00640 1 -- 172.21.15.83:0/3810335009 shutdown_connections 2024-04-24T09:08:06.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.097+0000 7f1c5fe00640 1 -- 172.21.15.83:0/3810335009 wait complete. 2024-04-24T09:08:06.120 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-24T09:08:06.164 INFO:tasks.workunit.client.0.smithi083.stdout: 10 20310 7.15 MB/sec execute 37 sec latency 69.281 ms 2024-04-24T09:08:06.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.374+0000 7f753d400640 1 -- 172.21.15.83:0/1752948001 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75380743b0 msgr2=0x7f7538074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:06.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.374+0000 7f753d400640 1 --2- 172.21.15.83:0/1752948001 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75380743b0 0x7f7538074790 secure :-1 s=READY pgs=2279 cs=0 l=1 rev1=1 crypto rx=0x7f7528009ca0 tx=0x7f752802a950 comp rx=0 tx=0).stop 2024-04-24T09:08:06.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.375+0000 7f753d400640 1 -- 172.21.15.83:0/1752948001 shutdown_connections 2024-04-24T09:08:06.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.375+0000 7f753d400640 1 --2- 172.21.15.83:0/1752948001 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f753810e890 0x7f7538110c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:06.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.375+0000 7f753d400640 1 --2- 172.21.15.83:0/1752948001 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7538074cd0 0x7f753810e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:06.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.375+0000 7f753d400640 1 --2- 172.21.15.83:0/1752948001 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75380743b0 0x7f7538074790 unknown :-1 s=CLOSED pgs=2279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:06.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.375+0000 7f753d400640 1 -- 172.21.15.83:0/1752948001 >> 172.21.15.83:0/1752948001 conn(0x7f75380fdaa0 msgr2=0x7f75380ffec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:06.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.376+0000 7f753d400640 1 -- 172.21.15.83:0/1752948001 shutdown_connections 2024-04-24T09:08:06.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.376+0000 7f753d400640 1 -- 172.21.15.83:0/1752948001 wait complete. 2024-04-24T09:08:06.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.377+0000 7f753d400640 1 Processor -- start 2024-04-24T09:08:06.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.378+0000 7f753d400640 1 -- start start 2024-04-24T09:08:06.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.379+0000 7f753d400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f75380743b0 0x7f7538073890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:06.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.379+0000 7f753d400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7538074cd0 0x7f753806c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:06.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.379+0000 7f753d400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f753810e890 0x7f753806ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:06.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.379+0000 7f753d400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7538114490 con 0x7f7538074cd0 2024-04-24T09:08:06.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.379+0000 7f753d400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7538114310 con 0x7f75380743b0 2024-04-24T09:08:06.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.379+0000 7f753d400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7538114610 con 0x7f753810e890 2024-04-24T09:08:06.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.379+0000 7f7535e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7538074cd0 0x7f753806c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:06.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.379+0000 7f7535e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7538074cd0 0x7f753806c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55236/0 (socket says 172.21.15.83:55236) 2024-04-24T09:08:06.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.379+0000 7f7535e00640 1 -- 172.21.15.83:0/2923182941 learned_addr learned my addr 172.21.15.83:0/2923182941 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:08:06.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.379+0000 7f7536800640 1 --2- 172.21.15.83:0/2923182941 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f75380743b0 0x7f7538073890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:06.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.379+0000 7f7537200640 1 --2- 172.21.15.83:0/2923182941 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f753810e890 0x7f753806ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:06.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.380+0000 7f7535e00640 1 -- 172.21.15.83:0/2923182941 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f753810e890 msgr2=0x7f753806ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:06.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.380+0000 7f7535e00640 1 --2- 172.21.15.83:0/2923182941 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f753810e890 0x7f753806ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:06.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.380+0000 7f7535e00640 1 -- 172.21.15.83:0/2923182941 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f75380743b0 msgr2=0x7f7538073890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:06.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.380+0000 7f7535e00640 1 --2- 172.21.15.83:0/2923182941 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f75380743b0 0x7f7538073890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:06.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.380+0000 7f7535e00640 1 -- 172.21.15.83:0/2923182941 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f753806d490 con 0x7f7538074cd0 2024-04-24T09:08:06.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.380+0000 7f7536800640 1 --2- 172.21.15.83:0/2923182941 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f75380743b0 0x7f7538073890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:08:06.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.380+0000 7f7535e00640 1 --2- 172.21.15.83:0/2923182941 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7538074cd0 0x7f753806c910 secure :-1 s=READY pgs=2280 cs=0 l=1 rev1=1 crypto rx=0x7f752400d950 tx=0x7f752400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:06.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.381+0000 7f752f400640 1 -- 172.21.15.83:0/2923182941 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7524017030 con 0x7f7538074cd0 2024-04-24T09:08:06.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.381+0000 7f752f400640 1 -- 172.21.15.83:0/2923182941 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7524004020 con 0x7f7538074cd0 2024-04-24T09:08:06.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.381+0000 7f753d400640 1 -- 172.21.15.83:0/2923182941 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75380713b0 con 0x7f7538074cd0 2024-04-24T09:08:06.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.381+0000 7f752f400640 1 -- 172.21.15.83:0/2923182941 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75240047a0 con 0x7f7538074cd0 2024-04-24T09:08:06.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.381+0000 7f753d400640 1 -- 172.21.15.83:0/2923182941 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7538071950 con 0x7f7538074cd0 2024-04-24T09:08:06.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.385+0000 7f75403a2740 1 Processor -- start 2024-04-24T09:08:06.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.385+0000 7f75403a2740 1 -- start start 2024-04-24T09:08:06.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.386+0000 7f75403a2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557a51688950 0x557a51688d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:06.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.386+0000 7f75403a2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557a51689270 0x557a5169bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:06.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.386+0000 7f75403a2740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557a5169c150 0x557a5169e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:06.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.386+0000 7f75403a2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x557a515ee1d0 con 0x557a51689270 2024-04-24T09:08:06.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.386+0000 7f75403a2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x557a51594450 con 0x557a51688950 2024-04-24T09:08:06.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.386+0000 7f75403a2740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x557a515ea190 con 0x557a5169c150 2024-04-24T09:08:06.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.386+0000 7f7537200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557a51688950 0x557a51688d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:06.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.386+0000 7f7537200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557a51688950 0x557a51688d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:58424/0 (socket says 172.21.15.83:58424) 2024-04-24T09:08:06.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.386+0000 7f7537200640 1 -- 172.21.15.83:0/1009353878 learned_addr learned my addr 172.21.15.83:0/1009353878 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:08:06.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.386+0000 7f752f400640 1 -- 172.21.15.83:0/2923182941 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f75240041d0 con 0x7f7538074cd0 2024-04-24T09:08:06.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.387+0000 7f7537200640 1 -- 172.21.15.83:0/1009353878 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557a5169c150 msgr2=0x557a5169e540 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-24T09:08:06.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.387+0000 7f7537200640 1 --2- 172.21.15.83:0/1009353878 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557a5169c150 0x557a5169e540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:06.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.387+0000 7f7537200640 1 -- 172.21.15.83:0/1009353878 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557a51689270 msgr2=0x557a5169bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:06.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.387+0000 7f7537200640 1 --2- 172.21.15.83:0/1009353878 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557a51689270 0x557a5169bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:06.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.387+0000 7f7537200640 1 -- 172.21.15.83:0/1009353878 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a51651310 con 0x557a51688950 2024-04-24T09:08:06.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.388+0000 7f7537200640 1 --2- 172.21.15.83:0/1009353878 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557a51688950 0x557a51688d30 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7530009c20 tx=0x7f7530011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83be37eceb9c828a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:06.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.388+0000 7f7523400640 1 -- 172.21.15.83:0/1009353878 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7530019620 con 0x557a51688950 2024-04-24T09:08:06.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.388+0000 7f752f400640 1 --2- 172.21.15.83:0/2923182941 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f751c0a3de0 0x7f751c0a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:06.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.388+0000 7f752f400640 1 -- 172.21.15.83:0/2923182941 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9671+0+0 (secure 0 0 0) 0x7f75240d0550 con 0x7f7538074cd0 2024-04-24T09:08:06.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.388+0000 7f7536800640 1 --2- 172.21.15.83:0/2923182941 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f751c0a3de0 0x7f751c0a62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:06.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.389+0000 7f7523400640 1 -- 172.21.15.83:0/1009353878 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7530019c70 con 0x557a51688950 2024-04-24T09:08:06.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.389+0000 7f7523400640 1 -- 172.21.15.83:0/1009353878 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7530018a90 con 0x557a51688950 2024-04-24T09:08:06.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.389+0000 7f75403a2740 1 -- 172.21.15.83:0/1009353878 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557a51688950 msgr2=0x557a51688d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:06.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.389+0000 7f75403a2740 1 --2- 172.21.15.83:0/1009353878 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557a51688950 0x557a51688d30 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7530009c20 tx=0x7f7530011810 comp rx=0 tx=0).stop 2024-04-24T09:08:06.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.390+0000 7f7536800640 1 --2- 172.21.15.83:0/2923182941 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f751c0a3de0 0x7f751c0a62a0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f7528000f80 tx=0x7f7528009800 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:06.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.390+0000 7f75403a2740 1 -- 172.21.15.83:0/1009353878 shutdown_connections 2024-04-24T09:08:06.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.390+0000 7f75403a2740 1 --2- 172.21.15.83:0/1009353878 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557a5169c150 0x557a5169e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:06.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.390+0000 7f75403a2740 1 --2- 172.21.15.83:0/1009353878 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557a51689270 0x557a5169bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:06.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.390+0000 7f75403a2740 1 --2- 172.21.15.83:0/1009353878 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557a51688950 0x557a51688d30 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:06.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.390+0000 7f75403a2740 1 -- 172.21.15.83:0/1009353878 >> 172.21.15.83:0/1009353878 conn(0x557a51684c70 msgr2=0x557a51685050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:06.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.390+0000 7f75403a2740 1 -- 172.21.15.83:0/1009353878 shutdown_connections 2024-04-24T09:08:06.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.390+0000 7f75403a2740 1 -- 172.21.15.83:0/1009353878 wait complete. 2024-04-24T09:08:06.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.391+0000 7f75403a2740 1 -- start start 2024-04-24T09:08:06.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.391+0000 7f75403a2740 1 Processor -- start 2024-04-24T09:08:06.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.392+0000 7f75403a2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557a51688950 0x557a51698f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:06.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.392+0000 7f75403a2740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557a51689270 0x557a51699480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:06.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.392+0000 7f75403a2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557a5169c150 0x557a516999c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:06.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.392+0000 7f75403a2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x557a5169efe0 con 0x557a5169c150 2024-04-24T09:08:06.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.392+0000 7f75403a2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x557a515a7810 con 0x557a51688950 2024-04-24T09:08:06.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.392+0000 7f75403a2740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x557a5169f160 con 0x557a51689270 2024-04-24T09:08:06.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.392+0000 7f75403a2740 10 client.0 _ll_register_callbacks cb 0x557a51575140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:08:06.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.392+0000 7f7537200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557a5169c150 0x557a516999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:06.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.392+0000 7f7536800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557a51688950 0x557a51698f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:06.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.392+0000 7f7537200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557a5169c150 0x557a516999c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55248/0 (socket says 172.21.15.83:55248) 2024-04-24T09:08:06.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.392+0000 7f7536800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557a51688950 0x557a51698f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:58440/0 (socket says 172.21.15.83:58440) 2024-04-24T09:08:06.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.392+0000 7f7537200640 1 -- 172.21.15.83:0/1993252009 learned_addr learned my addr 172.21.15.83:0/1993252009 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:08:06.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.392+0000 7f7535e00640 1 --2- 172.21.15.83:0/1993252009 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557a51689270 0x557a51699480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:06.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.393+0000 7f7537200640 1 -- 172.21.15.83:0/1993252009 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557a51689270 msgr2=0x557a51699480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:06.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.393+0000 7f7537200640 1 --2- 172.21.15.83:0/1993252009 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557a51689270 0x557a51699480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:06.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.393+0000 7f7537200640 1 -- 172.21.15.83:0/1993252009 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557a51688950 msgr2=0x557a51698f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:06.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.393+0000 7f7537200640 1 --2- 172.21.15.83:0/1993252009 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557a51688950 0x557a51698f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:06.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.393+0000 7f7537200640 1 -- 172.21.15.83:0/1993252009 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a515b83f0 con 0x557a5169c150 2024-04-24T09:08:06.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.393+0000 7f7535e00640 1 --2- 172.21.15.83:0/1993252009 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557a51689270 0x557a51699480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:08:06.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.393+0000 7f7536800640 1 --2- 172.21.15.83:0/1993252009 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557a51688950 0x557a51698f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:08:06.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.393+0000 7f7537200640 1 --2- 172.21.15.83:0/1993252009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557a5169c150 0x557a516999c0 secure :-1 s=READY pgs=2281 cs=0 l=1 rev1=1 crypto rx=0x7f7530018410 tx=0x7f75300196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94b6bdd07ff6f3db server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:06.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.393+0000 7f7517e00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:08:06.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.394+0000 7f7517e00640 1 -- 172.21.15.83:0/1993252009 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7530019ad0 con 0x557a5169c150 2024-04-24T09:08:06.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.394+0000 7f7517e00640 1 -- 172.21.15.83:0/1993252009 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9671+0+0 (secure 0 0 0) 0x7f7530004220 con 0x557a5169c150 2024-04-24T09:08:06.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.394+0000 7f75403a2740 1 -- 172.21.15.83:0/1993252009 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7530019c70 con 0x557a5169c150 2024-04-24T09:08:06.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.394+0000 7f7517e00640 1 -- 172.21.15.83:0/1993252009 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f7530032050 con 0x557a5169c150 2024-04-24T09:08:06.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.394+0000 7f75403a2740 10 client.21330 fetch_fsmap learned FSMap version 53 2024-04-24T09:08:06.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.394+0000 7f75403a2740 1 -- 172.21.15.83:0/1993252009 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x557a513003f0 con 0x557a5169c150 2024-04-24T09:08:06.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.394+0000 7f7517e00640 1 -- 172.21.15.83:0/1993252009 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7530019c70 con 0x557a5169c150 2024-04-24T09:08:06.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.395+0000 7f7517e00640 1 -- 172.21.15.83:0/1993252009 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 53) ==== 8398+0+0 (secure 0 0 0) 0x7f7530018c40 con 0x557a5169c150 2024-04-24T09:08:06.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.395+0000 7f75403a2740 10 client.21330 fetch_fsmap finished waiting for FSMap version 53 2024-04-24T09:08:06.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.395+0000 7f75403a2740 10 client.21330 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-24T09:08:06.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.395+0000 7f75403a2740 20 client.21330 populate_metadata read hostname 'smithi083' 2024-04-24T09:08:06.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.396+0000 7f75403a2740 1 --2- 172.21.15.83:0/1993252009 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x557a516ac320 0x557a516ae710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:06.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.396+0000 7f75403a2740 4 client.21330 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:08:06.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.396+0000 7f75403a2740 1 -- 172.21.15.83:0/1993252009 --> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557a515fbbf0 con 0x557a516ac320 2024-04-24T09:08:06.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.396+0000 7f7536800640 1 --2- 172.21.15.83:0/1993252009 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x557a516ac320 0x557a516ae710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:06.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.397+0000 7f7536800640 1 --2- 172.21.15.83:0/1993252009 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x557a516ac320 0x557a516ae710 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=cddfe0559285fce1 server_cookie=23008dbce6b11530 in_seq=0 out_seq=0 2024-04-24T09:08:06.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.398+0000 7f7517e00640 10 client.21330 ms_handle_connect on v2:172.21.15.189:6838/2548965336 2024-04-24T09:08:06.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.400+0000 7f7517e00640 1 -- 172.21.15.83:0/1993252009 <== mds.4 v2:172.21.15.189:6838/2548965336 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557a515fbbf0 con 0x557a516ac320 2024-04-24T09:08:06.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.400+0000 7f7517e00640 10 client.21330 handle_command_reply: tid=0 2024-04-24T09:08:06.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.400+0000 7f75403a2740 1 client.21330 shutdown 2024-04-24T09:08:06.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.401+0000 7f75403a2740 1 -- 172.21.15.83:0/1993252009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557a5169c150 msgr2=0x557a516999c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:06.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.401+0000 7f75403a2740 1 --2- 172.21.15.83:0/1993252009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557a5169c150 0x557a516999c0 secure :-1 s=READY pgs=2281 cs=0 l=1 rev1=1 crypto rx=0x7f7530018410 tx=0x7f75300196d0 comp rx=0 tx=0).stop 2024-04-24T09:08:06.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.401+0000 7f75403a2740 1 -- 172.21.15.83:0/1993252009 shutdown_connections 2024-04-24T09:08:06.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.401+0000 7f75403a2740 1 --2- 172.21.15.83:0/1993252009 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x557a516ac320 0x557a516ae710 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:06.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.401+0000 7f75403a2740 1 --2- 172.21.15.83:0/1993252009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557a5169c150 0x557a516999c0 unknown :-1 s=CLOSED pgs=2281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:06.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.401+0000 7f75403a2740 1 --2- 172.21.15.83:0/1993252009 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557a51689270 0x557a51699480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:06.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.401+0000 7f75403a2740 1 --2- 172.21.15.83:0/1993252009 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557a51688950 0x557a51698f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:06.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.401+0000 7f75403a2740 1 -- 172.21.15.83:0/1993252009 >> 172.21.15.83:0/1993252009 conn(0x557a51684c70 msgr2=0x557a5168a970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:06.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.401+0000 7f7517e00640 0 client.21330 ms_handle_reset on v2:172.21.15.189:6838/2548965336 2024-04-24T09:08:06.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.401+0000 7f75403a2740 1 -- 172.21.15.83:0/1993252009 shutdown_connections 2024-04-24T09:08:06.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.401+0000 7f75403a2740 1 -- 172.21.15.83:0/1993252009 wait complete. 2024-04-24T09:08:06.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.402+0000 7f75403a2740 20 client.21330 trim_cache size 0 max 16384 2024-04-24T09:08:06.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.424+0000 7f75403a2740 1 Processor -- start 2024-04-24T09:08:06.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.424+0000 7f75403a2740 1 -- start start 2024-04-24T09:08:06.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.424+0000 7f75403a2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557a51684c70 0x557a5169b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:06.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.424+0000 7f75403a2740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557a51688950 0x557a51684570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:06.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.424+0000 7f75403a2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557a51689270 0x557a516997e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:06.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.424+0000 7f75403a2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f75381b5e60 con 0x557a51689270 2024-04-24T09:08:06.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.424+0000 7f75403a2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f75381b7ac0 con 0x557a51684c70 2024-04-24T09:08:06.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.424+0000 7f75403a2740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f75381b6530 con 0x557a51688950 2024-04-24T09:08:06.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.425+0000 7f7537200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557a51689270 0x557a516997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:06.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.425+0000 7f7537200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557a51689270 0x557a516997e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55264/0 (socket says 172.21.15.83:55264) 2024-04-24T09:08:06.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.425+0000 7f7537200640 1 -- 172.21.15.83:0/401804969 learned_addr learned my addr 172.21.15.83:0/401804969 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:08:06.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.425+0000 7f7536800640 1 --2- 172.21.15.83:0/401804969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557a51684c70 0x557a5169b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:06.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.425+0000 7f7535e00640 1 --2- 172.21.15.83:0/401804969 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557a51688950 0x557a51684570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:06.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.425+0000 7f7537200640 1 -- 172.21.15.83:0/401804969 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557a51688950 msgr2=0x557a51684570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:06.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.425+0000 7f7537200640 1 --2- 172.21.15.83:0/401804969 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557a51688950 0x557a51684570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:06.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.425+0000 7f7537200640 1 -- 172.21.15.83:0/401804969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557a51684c70 msgr2=0x557a5169b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:06.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.425+0000 7f7537200640 1 --2- 172.21.15.83:0/401804969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557a51684c70 0x557a5169b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:06.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.425+0000 7f7537200640 1 -- 172.21.15.83:0/401804969 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a513ca7d0 con 0x557a51689270 2024-04-24T09:08:06.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.426+0000 7f7537200640 1 --2- 172.21.15.83:0/401804969 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557a51689270 0x557a516997e0 secure :-1 s=READY pgs=2282 cs=0 l=1 rev1=1 crypto rx=0x7f7530011850 tx=0x7f7530004680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ff2c5e257727a2c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:06.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.426+0000 7f7522a00640 1 -- 172.21.15.83:0/401804969 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7530009e90 con 0x557a51689270 2024-04-24T09:08:06.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.426+0000 7f7522a00640 1 -- 172.21.15.83:0/401804969 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f75300054b0 con 0x557a51689270 2024-04-24T09:08:06.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.426+0000 7f7522a00640 1 -- 172.21.15.83:0/401804969 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7530018660 con 0x557a51689270 2024-04-24T09:08:06.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.427+0000 7f75403a2740 1 -- 172.21.15.83:0/401804969 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557a51689270 msgr2=0x557a516997e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:06.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.427+0000 7f75403a2740 1 --2- 172.21.15.83:0/401804969 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557a51689270 0x557a516997e0 secure :-1 s=READY pgs=2282 cs=0 l=1 rev1=1 crypto rx=0x7f7530011850 tx=0x7f7530004680 comp rx=0 tx=0).stop 2024-04-24T09:08:06.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.427+0000 7f75403a2740 1 -- 172.21.15.83:0/401804969 shutdown_connections 2024-04-24T09:08:06.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.427+0000 7f75403a2740 1 --2- 172.21.15.83:0/401804969 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557a51689270 0x557a516997e0 unknown :-1 s=CLOSED pgs=2282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:06.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.427+0000 7f75403a2740 1 --2- 172.21.15.83:0/401804969 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557a51688950 0x557a51684570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:06.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.427+0000 7f75403a2740 1 --2- 172.21.15.83:0/401804969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557a51684c70 0x557a5169b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:06.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.427+0000 7f75403a2740 1 -- 172.21.15.83:0/401804969 >> 172.21.15.83:0/401804969 conn(0x557a516983c0 msgr2=0x557a5169b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:06.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.427+0000 7f75403a2740 1 -- 172.21.15.83:0/401804969 shutdown_connections 2024-04-24T09:08:06.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.427+0000 7f75403a2740 1 -- 172.21.15.83:0/401804969 wait complete. 2024-04-24T09:08:06.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.428+0000 7f75403a2740 1 -- start start 2024-04-24T09:08:06.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.428+0000 7f75403a2740 1 Processor -- start 2024-04-24T09:08:06.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.429+0000 7f75403a2740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557a51684c70 0x557a5169c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:06.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.429+0000 7f75403a2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557a51688950 0x557a5169ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:06.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.429+0000 7f75403a2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557a51689270 0x557a516a3380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:06.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.429+0000 7f75403a2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f75381b7330 con 0x557a51689270 2024-04-24T09:08:06.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.429+0000 7f75403a2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f75381b6c30 con 0x557a51688950 2024-04-24T09:08:06.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.429+0000 7f75403a2740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f75381b3e60 con 0x557a51684c70 2024-04-24T09:08:06.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.429+0000 7f75403a2740 10 client.0 _ll_register_callbacks cb 0x7f7530009af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:08:06.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.429+0000 7f7535e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557a51688950 0x557a5169ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:06.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.429+0000 7f7537200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557a51689270 0x557a516a3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:06.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.429+0000 7f7536800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557a51684c70 0x557a5169c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:06.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.429+0000 7f7537200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557a51689270 0x557a516a3380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55278/0 (socket says 172.21.15.83:55278) 2024-04-24T09:08:06.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.429+0000 7f7537200640 1 -- 172.21.15.83:0/1083957657 learned_addr learned my addr 172.21.15.83:0/1083957657 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:08:06.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.429+0000 7f7536800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557a51684c70 0x557a5169c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:38218/0 (socket says 172.21.15.83:38218) 2024-04-24T09:08:06.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.430+0000 7f7537200640 1 -- 172.21.15.83:0/1083957657 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557a51684c70 msgr2=0x557a5169c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:06.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.430+0000 7f7537200640 1 --2- 172.21.15.83:0/1083957657 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557a51684c70 0x557a5169c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:06.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.430+0000 7f7537200640 1 -- 172.21.15.83:0/1083957657 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557a51688950 msgr2=0x557a5169ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:06.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.430+0000 7f7537200640 1 --2- 172.21.15.83:0/1083957657 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557a51688950 0x557a5169ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:06.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.430+0000 7f7537200640 1 -- 172.21.15.83:0/1083957657 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a516a38c0 con 0x557a51689270 2024-04-24T09:08:06.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.430+0000 7f7535e00640 1 --2- 172.21.15.83:0/1083957657 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557a51688950 0x557a5169ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:08:06.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.430+0000 7f7537200640 1 --2- 172.21.15.83:0/1083957657 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557a51689270 0x557a516a3380 secure :-1 s=READY pgs=2283 cs=0 l=1 rev1=1 crypto rx=0x7f7530004770 tx=0x7f75300047a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd19485f595b3d63 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:06.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.430+0000 7f7517e00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:08:06.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.431+0000 7f7517e00640 1 -- 172.21.15.83:0/1083957657 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7530004e30 con 0x557a51689270 2024-04-24T09:08:06.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.431+0000 7f7517e00640 1 -- 172.21.15.83:0/1083957657 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9671+0+0 (secure 0 0 0) 0x7f7530019ad0 con 0x557a51689270 2024-04-24T09:08:06.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.431+0000 7f75403a2740 1 -- 172.21.15.83:0/1083957657 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75300054b0 con 0x557a51689270 2024-04-24T09:08:06.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.431+0000 7f7517e00640 1 -- 172.21.15.83:0/1083957657 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f7530019cf0 con 0x557a51689270 2024-04-24T09:08:06.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.431+0000 7f75403a2740 10 client.21336 fetch_fsmap learned FSMap version 53 2024-04-24T09:08:06.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.431+0000 7f75403a2740 1 -- 172.21.15.83:0/1083957657 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x557a516a4b30 con 0x557a51689270 2024-04-24T09:08:06.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.432+0000 7f7517e00640 1 -- 172.21.15.83:0/1083957657 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f75300054b0 con 0x557a51689270 2024-04-24T09:08:06.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.432+0000 7f7517e00640 1 -- 172.21.15.83:0/1083957657 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 53) ==== 8398+0+0 (secure 0 0 0) 0x7f7530018410 con 0x557a51689270 2024-04-24T09:08:06.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.432+0000 7f75403a2740 10 client.21336 fetch_fsmap finished waiting for FSMap version 53 2024-04-24T09:08:06.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.432+0000 7f75403a2740 10 client.21336 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-24T09:08:06.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.432+0000 7f75403a2740 20 client.21336 populate_metadata read hostname 'smithi083' 2024-04-24T09:08:06.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.432+0000 7f75403a2740 1 --2- 172.21.15.83:0/1083957657 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x557a5169c150 0x557a51714a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:06.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.432+0000 7f75403a2740 4 client.21336 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:08:06.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.432+0000 7f75403a2740 1 -- 172.21.15.83:0/1083957657 --> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557a515f5a60 con 0x557a5169c150 2024-04-24T09:08:06.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.433+0000 7f7536800640 1 --2- 172.21.15.83:0/1083957657 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x557a5169c150 0x557a51714a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:06.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.434+0000 7f7536800640 1 --2- 172.21.15.83:0/1083957657 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x557a5169c150 0x557a51714a50 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=af57f3ec31410744 server_cookie=c79c9bf857185aed in_seq=0 out_seq=0 2024-04-24T09:08:06.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.434+0000 7f7517e00640 10 client.21336 ms_handle_connect on v2:172.21.15.189:6838/2548965336 2024-04-24T09:08:06.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.435+0000 7f7517e00640 1 -- 172.21.15.83:0/1083957657 <== mds.4 v2:172.21.15.189:6838/2548965336 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557a515f5a60 con 0x557a5169c150 2024-04-24T09:08:06.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.435+0000 7f7517e00640 10 client.21336 handle_command_reply: tid=0 2024-04-24T09:08:06.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.435+0000 7f75403a2740 1 client.21336 shutdown 2024-04-24T09:08:06.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.436+0000 7f75403a2740 1 -- 172.21.15.83:0/1083957657 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557a51689270 msgr2=0x557a516a3380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:06.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.436+0000 7f75403a2740 1 --2- 172.21.15.83:0/1083957657 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557a51689270 0x557a516a3380 secure :-1 s=READY pgs=2283 cs=0 l=1 rev1=1 crypto rx=0x7f7530004770 tx=0x7f75300047a0 comp rx=0 tx=0).stop 2024-04-24T09:08:06.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.436+0000 7f75403a2740 1 -- 172.21.15.83:0/1083957657 shutdown_connections 2024-04-24T09:08:06.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.436+0000 7f75403a2740 1 --2- 172.21.15.83:0/1083957657 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x557a5169c150 0x557a51714a50 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:06.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.436+0000 7f75403a2740 1 --2- 172.21.15.83:0/1083957657 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557a51689270 0x557a516a3380 unknown :-1 s=CLOSED pgs=2283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:06.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.436+0000 7f75403a2740 1 --2- 172.21.15.83:0/1083957657 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557a51688950 0x557a5169ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:06.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.436+0000 7f75403a2740 1 --2- 172.21.15.83:0/1083957657 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557a51684c70 0x557a5169c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:06.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.436+0000 7f7517e00640 0 client.21336 ms_handle_reset on v2:172.21.15.189:6838/2548965336 2024-04-24T09:08:06.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.436+0000 7f75403a2740 1 -- 172.21.15.83:0/1083957657 >> 172.21.15.83:0/1083957657 conn(0x557a516983c0 msgr2=0x557a516a6e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:06.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.436+0000 7f75403a2740 1 -- 172.21.15.83:0/1083957657 shutdown_connections 2024-04-24T09:08:06.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.436+0000 7f75403a2740 1 -- 172.21.15.83:0/1083957657 wait complete. 2024-04-24T09:08:06.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.436+0000 7f75403a2740 20 client.21336 trim_cache size 0 max 16384 2024-04-24T09:08:06.436 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:08:06.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.437+0000 7f753d400640 1 -- 172.21.15.83:0/2923182941 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f751c0a3de0 msgr2=0x7f751c0a62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:06.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.437+0000 7f753d400640 1 --2- 172.21.15.83:0/2923182941 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f751c0a3de0 0x7f751c0a62a0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f7528000f80 tx=0x7f7528009800 comp rx=0 tx=0).stop 2024-04-24T09:08:06.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.437+0000 7f753d400640 1 -- 172.21.15.83:0/2923182941 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7538074cd0 msgr2=0x7f753806c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:06.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.437+0000 7f753d400640 1 --2- 172.21.15.83:0/2923182941 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7538074cd0 0x7f753806c910 secure :-1 s=READY pgs=2280 cs=0 l=1 rev1=1 crypto rx=0x7f752400d950 tx=0x7f752400de10 comp rx=0 tx=0).stop 2024-04-24T09:08:06.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.438+0000 7f753d400640 1 -- 172.21.15.83:0/2923182941 shutdown_connections 2024-04-24T09:08:06.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.438+0000 7f753d400640 1 --2- 172.21.15.83:0/2923182941 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f751c0a3de0 0x7f751c0a62a0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:06.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.438+0000 7f753d400640 1 --2- 172.21.15.83:0/2923182941 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f753810e890 0x7f753806ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:06.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.438+0000 7f753d400640 1 --2- 172.21.15.83:0/2923182941 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7538074cd0 0x7f753806c910 unknown :-1 s=CLOSED pgs=2280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:06.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.438+0000 7f753d400640 1 --2- 172.21.15.83:0/2923182941 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f75380743b0 0x7f7538073890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:06.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.438+0000 7f753d400640 1 -- 172.21.15.83:0/2923182941 >> 172.21.15.83:0/2923182941 conn(0x7f75380fdaa0 msgr2=0x7f75380fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:06.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.438+0000 7f753d400640 1 -- 172.21.15.83:0/2923182941 shutdown_connections 2024-04-24T09:08:06.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.438+0000 7f753d400640 1 -- 172.21.15.83:0/2923182941 wait complete. 2024-04-24T09:08:06.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:08:06 smithi138 ceph-mon[31441]: pgmap v518: 129 pgs: 129 active+clean; 452 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.8 MiB/s wr, 265 op/s 2024-04-24T09:08:06.455 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:08:06 smithi083 ceph-mon[27907]: pgmap v518: 129 pgs: 129 active+clean; 452 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.8 MiB/s wr, 265 op/s 2024-04-24T09:08:06.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:08:06 smithi189 ceph-mon[31791]: pgmap v518: 129 pgs: 129 active+clean; 452 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.8 MiB/s wr, 265 op/s 2024-04-24T09:08:06.458 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-24T09:08:06.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.743+0000 7f312ea00640 1 -- 172.21.15.83:0/1568081084 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3128103980 msgr2=0x7f312810be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:06.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.743+0000 7f312ea00640 1 --2- 172.21.15.83:0/1568081084 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3128103980 0x7f312810be60 secure :-1 s=READY pgs=2284 cs=0 l=1 rev1=1 crypto rx=0x7f3114009900 tx=0x7f311402a480 comp rx=0 tx=0).stop 2024-04-24T09:08:06.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.743+0000 7f312ea00640 1 -- 172.21.15.83:0/1568081084 shutdown_connections 2024-04-24T09:08:06.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.743+0000 7f312ea00640 1 --2- 172.21.15.83:0/1568081084 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f312810c480 0x7f312810e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:06.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.743+0000 7f312ea00640 1 --2- 172.21.15.83:0/1568081084 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3128103980 0x7f312810be60 unknown :-1 s=CLOSED pgs=2284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:06.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.743+0000 7f312ea00640 1 --2- 172.21.15.83:0/1568081084 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3128103060 0x7f3128103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:06.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.743+0000 7f312ea00640 1 -- 172.21.15.83:0/1568081084 >> 172.21.15.83:0/1568081084 conn(0x7f31280fc880 msgr2=0x7f31280feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:06.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.743+0000 7f312ea00640 1 -- 172.21.15.83:0/1568081084 shutdown_connections 2024-04-24T09:08:06.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.743+0000 7f312ea00640 1 -- 172.21.15.83:0/1568081084 wait complete. 2024-04-24T09:08:06.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.745+0000 7f312ea00640 1 Processor -- start 2024-04-24T09:08:06.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.745+0000 7f312ea00640 1 -- start start 2024-04-24T09:08:06.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.745+0000 7f312ea00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3128103060 0x7f31281a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:06.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.745+0000 7f312ea00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3128103980 0x7f31281a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:06.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.746+0000 7f312ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f312810c480 0x7f312819a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:06.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.746+0000 7f312ea00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3128112110 con 0x7f312810c480 2024-04-24T09:08:06.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.746+0000 7f312ea00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3128111f90 con 0x7f3128103980 2024-04-24T09:08:06.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.746+0000 7f312ea00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3128112290 con 0x7f3128103060 2024-04-24T09:08:06.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.746+0000 7f312cc00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3128103980 0x7f31281a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:06.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.746+0000 7f312cc00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3128103980 0x7f31281a0b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:58454/0 (socket says 172.21.15.83:58454) 2024-04-24T09:08:06.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.746+0000 7f312cc00640 1 -- 172.21.15.83:0/3085807302 learned_addr learned my addr 172.21.15.83:0/3085807302 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:08:06.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.746+0000 7f312e000640 1 --2- 172.21.15.83:0/3085807302 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f312810c480 0x7f312819a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:06.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.746+0000 7f312d600640 1 --2- 172.21.15.83:0/3085807302 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3128103060 0x7f31281a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:06.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.747+0000 7f312e000640 1 -- 172.21.15.83:0/3085807302 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3128103060 msgr2=0x7f31281a0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:06.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.747+0000 7f312e000640 1 --2- 172.21.15.83:0/3085807302 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3128103060 0x7f31281a0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:06.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.747+0000 7f312e000640 1 -- 172.21.15.83:0/3085807302 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3128103980 msgr2=0x7f31281a0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:06.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.747+0000 7f312e000640 1 --2- 172.21.15.83:0/3085807302 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3128103980 0x7f31281a0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:06.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.747+0000 7f312e000640 1 -- 172.21.15.83:0/3085807302 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f312819afb0 con 0x7f312810c480 2024-04-24T09:08:06.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.747+0000 7f312e000640 1 --2- 172.21.15.83:0/3085807302 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f312810c480 0x7f312819a6f0 secure :-1 s=READY pgs=2285 cs=0 l=1 rev1=1 crypto rx=0x7f3120009fd0 tx=0x7f312000ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:06.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.748+0000 7f3126000640 1 -- 172.21.15.83:0/3085807302 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f312000ce20 con 0x7f312810c480 2024-04-24T09:08:06.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.748+0000 7f3126000640 1 -- 172.21.15.83:0/3085807302 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3120005020 con 0x7f312810c480 2024-04-24T09:08:06.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.748+0000 7f3126000640 1 -- 172.21.15.83:0/3085807302 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3120010690 con 0x7f312810c480 2024-04-24T09:08:06.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.748+0000 7f312ea00640 1 -- 172.21.15.83:0/3085807302 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f312819b2a0 con 0x7f312810c480 2024-04-24T09:08:06.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.748+0000 7f312ea00640 1 -- 172.21.15.83:0/3085807302 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31281a75b0 con 0x7f312810c480 2024-04-24T09:08:06.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.754+0000 7f3126000640 1 -- 172.21.15.83:0/3085807302 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f31200051d0 con 0x7f312810c480 2024-04-24T09:08:06.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.756+0000 7f3132e29740 1 Processor -- start 2024-04-24T09:08:06.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.756+0000 7f3126000640 1 --2- 172.21.15.83:0/3085807302 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f31100a3da0 0x7f31100a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:06.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.756+0000 7f3132e29740 1 -- start start 2024-04-24T09:08:06.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.757+0000 7f3126000640 1 -- 172.21.15.83:0/3085807302 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9671+0+0 (secure 0 0 0) 0x7f31200d1230 con 0x7f312810c480 2024-04-24T09:08:06.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.757+0000 7f3132e29740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561b777a1b40 0x561b777a1f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:06.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.757+0000 7f3132e29740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561b777a2460 0x561b777ac5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:06.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.757+0000 7f3132e29740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561b777b5340 0x561b777b7730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:06.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.757+0000 7f3132e29740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561b777071d0 con 0x561b777a1b40 2024-04-24T09:08:06.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.757+0000 7f3132e29740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x561b776ad450 con 0x561b777a2460 2024-04-24T09:08:06.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.757+0000 7f3132e29740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x561b77703190 con 0x561b777b5340 2024-04-24T09:08:06.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.757+0000 7f312cc00640 1 --2- 172.21.15.83:0/3085807302 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f31100a3da0 0x7f31100a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:06.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.757+0000 7f312d600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561b777a1b40 0x561b777a1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:06.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.758+0000 7f312d600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561b777a1b40 0x561b777a1f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55302/0 (socket says 172.21.15.83:55302) 2024-04-24T09:08:06.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.758+0000 7f312d600640 1 -- 172.21.15.83:0/3920752715 learned_addr learned my addr 172.21.15.83:0/3920752715 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:08:06.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.758+0000 7f312e000640 1 --2- 172.21.15.83:0/3920752715 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561b777b5340 0x561b777b7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:06.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.759+0000 7f312cc00640 1 --2- 172.21.15.83:0/3085807302 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f31100a3da0 0x7f31100a6260 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f3114031440 tx=0x7f31140058d0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:06.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.759+0000 7f312e000640 1 -- 172.21.15.83:0/3920752715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561b777a2460 msgr2=0x561b777ac5f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-24T09:08:06.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.759+0000 7f312e000640 1 --2- 172.21.15.83:0/3920752715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561b777a2460 0x561b777ac5f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:06.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.759+0000 7f312e000640 1 -- 172.21.15.83:0/3920752715 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561b777a1b40 msgr2=0x561b777a1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:06.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.759+0000 7f312e000640 1 --2- 172.21.15.83:0/3920752715 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561b777a1b40 0x561b777a1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:06.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.759+0000 7f312e000640 1 -- 172.21.15.83:0/3920752715 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b7776a310 con 0x561b777b5340 2024-04-24T09:08:06.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.759+0000 7f312d600640 1 --2- 172.21.15.83:0/3920752715 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561b777a1b40 0x561b777a1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:08:06.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.760+0000 7f312e000640 1 --2- 172.21.15.83:0/3920752715 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561b777b5340 0x561b777b7730 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f31200ce500 tx=0x7f312009c7e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6337237d651ae725 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:06.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.760+0000 7f311a600640 1 -- 172.21.15.83:0/3920752715 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f312009b050 con 0x561b777b5340 2024-04-24T09:08:06.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.760+0000 7f311a600640 1 -- 172.21.15.83:0/3920752715 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3120005540 con 0x561b777b5340 2024-04-24T09:08:06.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.760+0000 7f311a600640 1 -- 172.21.15.83:0/3920752715 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31200a5530 con 0x561b777b5340 2024-04-24T09:08:06.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.762+0000 7f3132e29740 1 -- 172.21.15.83:0/3920752715 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561b777b5340 msgr2=0x561b777b7730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:06.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.762+0000 7f3132e29740 1 --2- 172.21.15.83:0/3920752715 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561b777b5340 0x561b777b7730 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f31200ce500 tx=0x7f312009c7e0 comp rx=0 tx=0).stop 2024-04-24T09:08:06.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.762+0000 7f3132e29740 1 -- 172.21.15.83:0/3920752715 shutdown_connections 2024-04-24T09:08:06.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.762+0000 7f3132e29740 1 --2- 172.21.15.83:0/3920752715 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561b777b5340 0x561b777b7730 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:06.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.762+0000 7f3132e29740 1 --2- 172.21.15.83:0/3920752715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561b777a2460 0x561b777ac5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:06.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.762+0000 7f3132e29740 1 --2- 172.21.15.83:0/3920752715 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561b777a1b40 0x561b777a1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:06.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.762+0000 7f3132e29740 1 -- 172.21.15.83:0/3920752715 >> 172.21.15.83:0/3920752715 conn(0x561b7779de60 msgr2=0x561b7779e240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:06.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.762+0000 7f3132e29740 1 -- 172.21.15.83:0/3920752715 shutdown_connections 2024-04-24T09:08:06.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.762+0000 7f3132e29740 1 -- 172.21.15.83:0/3920752715 wait complete. 2024-04-24T09:08:06.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.762+0000 7f3132e29740 1 -- start start 2024-04-24T09:08:06.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.763+0000 7f3132e29740 1 Processor -- start 2024-04-24T09:08:06.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.763+0000 7f3132e29740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561b777a1b40 0x561b777a9910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:06.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.763+0000 7f3132e29740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561b777a2460 0x561b777a9e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:06.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.763+0000 7f3132e29740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561b777b5340 0x561b777aa390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:06.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.763+0000 7f3132e29740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561b777b81d0 con 0x561b777a2460 2024-04-24T09:08:06.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.763+0000 7f3132e29740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x561b776c0810 con 0x561b777b5340 2024-04-24T09:08:06.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.763+0000 7f3132e29740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x561b777b8350 con 0x561b777a1b40 2024-04-24T09:08:06.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.764+0000 7f3132e29740 10 client.0 _ll_register_callbacks cb 0x561b7768e140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:08:06.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.764+0000 7f312cc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561b777a2460 0x561b777a9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:06.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.764+0000 7f312cc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561b777a2460 0x561b777a9e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55316/0 (socket says 172.21.15.83:55316) 2024-04-24T09:08:06.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.764+0000 7f312cc00640 1 -- 172.21.15.83:0/901478653 learned_addr learned my addr 172.21.15.83:0/901478653 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:08:06.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.764+0000 7f312e000640 1 --2- 172.21.15.83:0/901478653 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561b777b5340 0x561b777aa390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:06.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.764+0000 7f312d600640 1 --2- 172.21.15.83:0/901478653 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561b777a1b40 0x561b777a9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:06.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.764+0000 7f312cc00640 1 -- 172.21.15.83:0/901478653 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561b777a1b40 msgr2=0x561b777a9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:06.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.764+0000 7f312cc00640 1 --2- 172.21.15.83:0/901478653 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561b777a1b40 0x561b777a9910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:06.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.764+0000 7f312cc00640 1 -- 172.21.15.83:0/901478653 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561b777b5340 msgr2=0x561b777aa390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:06.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.764+0000 7f312cc00640 1 --2- 172.21.15.83:0/901478653 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561b777b5340 0x561b777aa390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:06.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.764+0000 7f312cc00640 1 -- 172.21.15.83:0/901478653 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b776d13f0 con 0x561b777a2460 2024-04-24T09:08:06.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.765+0000 7f312cc00640 1 --2- 172.21.15.83:0/901478653 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561b777a2460 0x561b777a9e50 secure :-1 s=READY pgs=2286 cs=0 l=1 rev1=1 crypto rx=0x7f3114006010 tx=0x7f3114009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec544f3ff4f06d0d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:06.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.765+0000 7f310ea00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:08:06.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.765+0000 7f310ea00640 1 -- 172.21.15.83:0/901478653 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f311404c070 con 0x561b777a2460 2024-04-24T09:08:06.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.765+0000 7f310ea00640 1 -- 172.21.15.83:0/901478653 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9671+0+0 (secure 0 0 0) 0x7f31140043d0 con 0x561b777a2460 2024-04-24T09:08:06.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.766+0000 7f3132e29740 1 -- 172.21.15.83:0/901478653 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3120005540 con 0x561b777a2460 2024-04-24T09:08:06.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.766+0000 7f310ea00640 1 -- 172.21.15.83:0/901478653 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f3114056050 con 0x561b777a2460 2024-04-24T09:08:06.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.766+0000 7f3132e29740 10 client.21348 fetch_fsmap learned FSMap version 53 2024-04-24T09:08:06.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.766+0000 7f3132e29740 1 -- 172.21.15.83:0/901478653 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x561b774193f0 con 0x561b777a2460 2024-04-24T09:08:06.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.767+0000 7f310ea00640 1 -- 172.21.15.83:0/901478653 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3120005540 con 0x561b777a2460 2024-04-24T09:08:06.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.767+0000 7f310ea00640 1 -- 172.21.15.83:0/901478653 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 53) ==== 8398+0+0 (secure 0 0 0) 0x7f3114051600 con 0x561b777a2460 2024-04-24T09:08:06.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.767+0000 7f3132e29740 10 client.21348 fetch_fsmap finished waiting for FSMap version 53 2024-04-24T09:08:06.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.767+0000 7f3132e29740 10 client.21348 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-24T09:08:06.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.767+0000 7f3132e29740 20 client.21348 populate_metadata read hostname 'smithi083' 2024-04-24T09:08:06.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.767+0000 7f3132e29740 1 --2- 172.21.15.83:0/901478653 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x561b777c5630 0x561b777c7a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:06.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.767+0000 7f3132e29740 4 client.21348 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:08:06.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.767+0000 7f3132e29740 1 -- 172.21.15.83:0/901478653 --> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561b77714bf0 con 0x561b777c5630 2024-04-24T09:08:06.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.768+0000 7f312d600640 1 --2- 172.21.15.83:0/901478653 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x561b777c5630 0x561b777c7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:06.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.769+0000 7f312d600640 1 --2- 172.21.15.83:0/901478653 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x561b777c5630 0x561b777c7a50 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8755e25011a7006 server_cookie=63a42f9a51d33ef7 in_seq=0 out_seq=0 2024-04-24T09:08:06.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.769+0000 7f310ea00640 10 client.21348 ms_handle_connect on v2:172.21.15.138:6832/1157967658 2024-04-24T09:08:06.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.770+0000 7f310ea00640 1 -- 172.21.15.83:0/901478653 <== mds.3 v2:172.21.15.138:6832/1157967658 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561b77714bf0 con 0x561b777c5630 2024-04-24T09:08:06.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.770+0000 7f310ea00640 10 client.21348 handle_command_reply: tid=0 2024-04-24T09:08:06.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.770+0000 7f3132e29740 1 client.21348 shutdown 2024-04-24T09:08:06.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.771+0000 7f3132e29740 1 -- 172.21.15.83:0/901478653 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561b777a2460 msgr2=0x561b777a9e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:06.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.771+0000 7f3132e29740 1 --2- 172.21.15.83:0/901478653 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561b777a2460 0x561b777a9e50 secure :-1 s=READY pgs=2286 cs=0 l=1 rev1=1 crypto rx=0x7f3114006010 tx=0x7f3114009260 comp rx=0 tx=0).stop 2024-04-24T09:08:06.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.771+0000 7f3132e29740 1 -- 172.21.15.83:0/901478653 shutdown_connections 2024-04-24T09:08:06.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.771+0000 7f3132e29740 1 --2- 172.21.15.83:0/901478653 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561b777b5340 0x561b777aa390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:06.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.771+0000 7f3132e29740 1 --2- 172.21.15.83:0/901478653 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561b777a2460 0x561b777a9e50 unknown :-1 s=CLOSED pgs=2286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:06.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.771+0000 7f3132e29740 1 --2- 172.21.15.83:0/901478653 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x561b777c5630 0x561b777c7a50 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:06.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.771+0000 7f3132e29740 1 --2- 172.21.15.83:0/901478653 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561b777a1b40 0x561b777a9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:06.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.771+0000 7f3132e29740 1 -- 172.21.15.83:0/901478653 >> 172.21.15.83:0/901478653 conn(0x561b7779de60 msgr2=0x561b777a3b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:06.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.771+0000 7f310ea00640 0 client.21348 ms_handle_reset on v2:172.21.15.138:6832/1157967658 2024-04-24T09:08:06.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.771+0000 7f3132e29740 1 -- 172.21.15.83:0/901478653 shutdown_connections 2024-04-24T09:08:06.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.771+0000 7f3132e29740 1 -- 172.21.15.83:0/901478653 wait complete. 2024-04-24T09:08:06.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.771+0000 7f3132e29740 20 client.21348 trim_cache size 0 max 16384 2024-04-24T09:08:06.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.794+0000 7f3132e29740 1 Processor -- start 2024-04-24T09:08:06.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.794+0000 7f3132e29740 1 -- start start 2024-04-24T09:08:06.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.795+0000 7f3132e29740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561b777a1b40 0x561b777c10c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:06.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.795+0000 7f3132e29740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561b777a2460 0x561b777a8a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:06.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.795+0000 7f3132e29740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561b777b5340 0x561b777aa1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:06.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.795+0000 7f3132e29740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f31281ad3b0 con 0x561b777a2460 2024-04-24T09:08:06.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.795+0000 7f3132e29740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f31281af010 con 0x561b777a1b40 2024-04-24T09:08:06.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.795+0000 7f3132e29740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f31281ada80 con 0x561b777b5340 2024-04-24T09:08:06.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.795+0000 7f312cc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561b777a2460 0x561b777a8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:06.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.795+0000 7f312cc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561b777a2460 0x561b777a8a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55326/0 (socket says 172.21.15.83:55326) 2024-04-24T09:08:06.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.795+0000 7f312cc00640 1 -- 172.21.15.83:0/1113207485 learned_addr learned my addr 172.21.15.83:0/1113207485 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:08:06.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.795+0000 7f312d600640 1 --2- 172.21.15.83:0/1113207485 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561b777a1b40 0x561b777c10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:06.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.795+0000 7f312cc00640 1 -- 172.21.15.83:0/1113207485 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561b777b5340 msgr2=0x561b777aa1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:06.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.795+0000 7f312cc00640 1 --2- 172.21.15.83:0/1113207485 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561b777b5340 0x561b777aa1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:06.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.795+0000 7f312cc00640 1 -- 172.21.15.83:0/1113207485 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561b777a1b40 msgr2=0x561b777c10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:06.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.795+0000 7f312cc00640 1 --2- 172.21.15.83:0/1113207485 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561b777a1b40 0x561b777c10c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:06.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.795+0000 7f312cc00640 1 -- 172.21.15.83:0/1113207485 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b774e37d0 con 0x561b777a2460 2024-04-24T09:08:06.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.796+0000 7f312cc00640 1 --2- 172.21.15.83:0/1113207485 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561b777a2460 0x561b777a8a70 secure :-1 s=READY pgs=2287 cs=0 l=1 rev1=1 crypto rx=0x7f3114051c10 tx=0x7f3114009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6bf36816c2d72b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:06.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.796+0000 7f3119c00640 1 -- 172.21.15.83:0/1113207485 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3114061020 con 0x561b777a2460 2024-04-24T09:08:06.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.796+0000 7f3119c00640 1 -- 172.21.15.83:0/1113207485 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3114051640 con 0x561b777a2460 2024-04-24T09:08:06.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.796+0000 7f3119c00640 1 -- 172.21.15.83:0/1113207485 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f311404c040 con 0x561b777a2460 2024-04-24T09:08:06.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.797+0000 7f3132e29740 1 -- 172.21.15.83:0/1113207485 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561b777a2460 msgr2=0x561b777a8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:06.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.797+0000 7f3132e29740 1 --2- 172.21.15.83:0/1113207485 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561b777a2460 0x561b777a8a70 secure :-1 s=READY pgs=2287 cs=0 l=1 rev1=1 crypto rx=0x7f3114051c10 tx=0x7f3114009780 comp rx=0 tx=0).stop 2024-04-24T09:08:06.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.797+0000 7f3132e29740 1 -- 172.21.15.83:0/1113207485 shutdown_connections 2024-04-24T09:08:06.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.797+0000 7f3132e29740 1 --2- 172.21.15.83:0/1113207485 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561b777b5340 0x561b777aa1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:06.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.797+0000 7f3132e29740 1 --2- 172.21.15.83:0/1113207485 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561b777a2460 0x561b777a8a70 unknown :-1 s=CLOSED pgs=2287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:06.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.797+0000 7f3132e29740 1 --2- 172.21.15.83:0/1113207485 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561b777a1b40 0x561b777c10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:06.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.797+0000 7f3132e29740 1 -- 172.21.15.83:0/1113207485 >> 172.21.15.83:0/1113207485 conn(0x561b7779de60 msgr2=0x561b777a32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:06.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.798+0000 7f3132e29740 1 -- 172.21.15.83:0/1113207485 shutdown_connections 2024-04-24T09:08:06.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.798+0000 7f3132e29740 1 -- 172.21.15.83:0/1113207485 wait complete. 2024-04-24T09:08:06.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.798+0000 7f3132e29740 1 -- start start 2024-04-24T09:08:06.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.799+0000 7f3132e29740 1 Processor -- start 2024-04-24T09:08:06.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.799+0000 7f3132e29740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561b777a1b40 0x561b777a5b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:06.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.799+0000 7f3132e29740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561b777a2460 0x561b777a60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:06.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.799+0000 7f3132e29740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561b777b5340 0x561b777a65e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:06.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.799+0000 7f3132e29740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f31281ae880 con 0x561b777b5340 2024-04-24T09:08:06.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.799+0000 7f3132e29740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f31281ae180 con 0x561b777a1b40 2024-04-24T09:08:06.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.799+0000 7f3132e29740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f31281ab3d0 con 0x561b777a2460 2024-04-24T09:08:06.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.799+0000 7f3132e29740 10 client.0 _ll_register_callbacks cb 0x7f3114040070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:08:06.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.800+0000 7f312d600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561b777a1b40 0x561b777a5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:06.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.800+0000 7f312e000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561b777b5340 0x561b777a65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:06.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.800+0000 7f312d600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561b777a1b40 0x561b777a5b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:58484/0 (socket says 172.21.15.83:58484) 2024-04-24T09:08:06.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.800+0000 7f312d600640 1 -- 172.21.15.83:0/3132083958 learned_addr learned my addr 172.21.15.83:0/3132083958 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:08:06.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.800+0000 7f312cc00640 1 --2- 172.21.15.83:0/3132083958 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561b777a2460 0x561b777a60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:06.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.800+0000 7f312e000640 1 -- 172.21.15.83:0/3132083958 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561b777a2460 msgr2=0x561b777a60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:06.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.800+0000 7f312e000640 1 --2- 172.21.15.83:0/3132083958 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561b777a2460 0x561b777a60a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:06.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.800+0000 7f312e000640 1 -- 172.21.15.83:0/3132083958 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561b777a1b40 msgr2=0x561b777a5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:06.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.800+0000 7f312e000640 1 --2- 172.21.15.83:0/3132083958 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561b777a1b40 0x561b777a5b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:06.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.800+0000 7f312e000640 1 -- 172.21.15.83:0/3132083958 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b777bc5d0 con 0x561b777b5340 2024-04-24T09:08:06.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.801+0000 7f312e000640 1 --2- 172.21.15.83:0/3132083958 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561b777b5340 0x561b777a65e0 secure :-1 s=READY pgs=2288 cs=0 l=1 rev1=1 crypto rx=0x7f31200991b0 tx=0x7f31200096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4052670a428d2f77 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:06.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.801+0000 7f310ea00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:08:06.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.801+0000 7f310ea00640 1 -- 172.21.15.83:0/3132083958 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f312007f070 con 0x561b777b5340 2024-04-24T09:08:06.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.801+0000 7f310ea00640 1 -- 172.21.15.83:0/3132083958 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9671+0+0 (secure 0 0 0) 0x7f31200a5530 con 0x561b777b5340 2024-04-24T09:08:06.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.801+0000 7f3132e29740 1 -- 172.21.15.83:0/3132083958 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3114051640 con 0x561b777b5340 2024-04-24T09:08:06.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.801+0000 7f310ea00640 1 -- 172.21.15.83:0/3132083958 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f3120098ff0 con 0x561b777b5340 2024-04-24T09:08:06.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.801+0000 7f3132e29740 10 client.21354 fetch_fsmap learned FSMap version 53 2024-04-24T09:08:06.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.801+0000 7f3132e29740 1 -- 172.21.15.83:0/3132083958 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x561b777bd8d0 con 0x561b777b5340 2024-04-24T09:08:06.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.803+0000 7f310ea00640 1 -- 172.21.15.83:0/3132083958 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3114051640 con 0x561b777b5340 2024-04-24T09:08:06.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.803+0000 7f310ea00640 1 -- 172.21.15.83:0/3132083958 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 53) ==== 8398+0+0 (secure 0 0 0) 0x7f3120082020 con 0x561b777b5340 2024-04-24T09:08:06.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.803+0000 7f3132e29740 10 client.21354 fetch_fsmap finished waiting for FSMap version 53 2024-04-24T09:08:06.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.803+0000 7f3132e29740 10 client.21354 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-24T09:08:06.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.803+0000 7f3132e29740 20 client.21354 populate_metadata read hostname 'smithi083' 2024-04-24T09:08:06.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.803+0000 7f3132e29740 1 --2- 172.21.15.83:0/3132083958 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x561b777bdbb0 0x561b7782db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:06.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.803+0000 7f3132e29740 4 client.21354 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:08:06.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.803+0000 7f3132e29740 1 -- 172.21.15.83:0/3132083958 --> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561b7770ea60 con 0x561b777bdbb0 2024-04-24T09:08:06.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.804+0000 7f312d600640 1 --2- 172.21.15.83:0/3132083958 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x561b777bdbb0 0x561b7782db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:06.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.805+0000 7f312d600640 1 --2- 172.21.15.83:0/3132083958 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x561b777bdbb0 0x561b7782db80 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=bf57f37c5107e8f3 server_cookie=8a5320a385bd7331 in_seq=0 out_seq=0 2024-04-24T09:08:06.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.805+0000 7f310ea00640 10 client.21354 ms_handle_connect on v2:172.21.15.138:6832/1157967658 2024-04-24T09:08:06.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.809+0000 7f310ea00640 1 -- 172.21.15.83:0/3132083958 <== mds.3 v2:172.21.15.138:6832/1157967658 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561b7770ea60 con 0x561b777bdbb0 2024-04-24T09:08:06.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.809+0000 7f310ea00640 10 client.21354 handle_command_reply: tid=0 2024-04-24T09:08:06.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.809+0000 7f3132e29740 1 client.21354 shutdown 2024-04-24T09:08:06.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.809+0000 7f3132e29740 1 -- 172.21.15.83:0/3132083958 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561b777b5340 msgr2=0x561b777a65e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:06.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.809+0000 7f3132e29740 1 --2- 172.21.15.83:0/3132083958 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561b777b5340 0x561b777a65e0 secure :-1 s=READY pgs=2288 cs=0 l=1 rev1=1 crypto rx=0x7f31200991b0 tx=0x7f31200096c0 comp rx=0 tx=0).stop 2024-04-24T09:08:06.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.809+0000 7f3132e29740 1 -- 172.21.15.83:0/3132083958 shutdown_connections 2024-04-24T09:08:06.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.809+0000 7f3132e29740 1 --2- 172.21.15.83:0/3132083958 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561b777b5340 0x561b777a65e0 unknown :-1 s=CLOSED pgs=2288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:06.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.809+0000 7f3132e29740 1 --2- 172.21.15.83:0/3132083958 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x561b777bdbb0 0x561b7782db80 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:06.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.809+0000 7f3132e29740 1 --2- 172.21.15.83:0/3132083958 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561b777a2460 0x561b777a60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:06.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.809+0000 7f3132e29740 1 --2- 172.21.15.83:0/3132083958 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561b777a1b40 0x561b777a5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:06.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.809+0000 7f3132e29740 1 -- 172.21.15.83:0/3132083958 >> 172.21.15.83:0/3132083958 conn(0x561b7779de60 msgr2=0x561b777a3b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:06.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.809+0000 7f310ea00640 0 client.21354 ms_handle_reset on v2:172.21.15.138:6832/1157967658 2024-04-24T09:08:06.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.810+0000 7f3132e29740 1 -- 172.21.15.83:0/3132083958 shutdown_connections 2024-04-24T09:08:06.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.810+0000 7f3132e29740 1 -- 172.21.15.83:0/3132083958 wait complete. 2024-04-24T09:08:06.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.810+0000 7f3132e29740 20 client.21354 trim_cache size 0 max 16384 2024-04-24T09:08:06.810 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:08:06.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.811+0000 7f312ea00640 1 -- 172.21.15.83:0/3085807302 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f31100a3da0 msgr2=0x7f31100a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:06.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.811+0000 7f312ea00640 1 --2- 172.21.15.83:0/3085807302 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f31100a3da0 0x7f31100a6260 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f3114031440 tx=0x7f31140058d0 comp rx=0 tx=0).stop 2024-04-24T09:08:06.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.811+0000 7f312ea00640 1 -- 172.21.15.83:0/3085807302 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f312810c480 msgr2=0x7f312819a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:06.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.811+0000 7f312ea00640 1 --2- 172.21.15.83:0/3085807302 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f312810c480 0x7f312819a6f0 secure :-1 s=READY pgs=2285 cs=0 l=1 rev1=1 crypto rx=0x7f3120009fd0 tx=0x7f312000ef10 comp rx=0 tx=0).stop 2024-04-24T09:08:06.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.811+0000 7f312ea00640 1 -- 172.21.15.83:0/3085807302 shutdown_connections 2024-04-24T09:08:06.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.811+0000 7f312ea00640 1 --2- 172.21.15.83:0/3085807302 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f31100a3da0 0x7f31100a6260 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:06.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.811+0000 7f312ea00640 1 --2- 172.21.15.83:0/3085807302 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f312810c480 0x7f312819a6f0 unknown :-1 s=CLOSED pgs=2285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:06.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.811+0000 7f312ea00640 1 --2- 172.21.15.83:0/3085807302 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3128103980 0x7f31281a0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:06.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.811+0000 7f312ea00640 1 --2- 172.21.15.83:0/3085807302 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3128103060 0x7f31281a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:06.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.811+0000 7f312ea00640 1 -- 172.21.15.83:0/3085807302 >> 172.21.15.83:0/3085807302 conn(0x7f31280fc880 msgr2=0x7f3128101e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:06.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.812+0000 7f312ea00640 1 -- 172.21.15.83:0/3085807302 shutdown_connections 2024-04-24T09:08:06.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:06.812+0000 7f312ea00640 1 -- 172.21.15.83:0/3085807302 wait complete. 2024-04-24T09:08:06.832 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T09:08:07.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.123+0000 7fcd59600640 1 -- 172.21.15.83:0/1308685017 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd5410c490 msgr2=0x7fcd5410e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:07.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.123+0000 7fcd59600640 1 --2- 172.21.15.83:0/1308685017 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd5410c490 0x7fcd5410e910 secure :-1 s=READY pgs=2289 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c009ca0 tx=0x7fcd4c02a950 comp rx=0 tx=0).stop 2024-04-24T09:08:07.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.124+0000 7fcd59600640 1 -- 172.21.15.83:0/1308685017 shutdown_connections 2024-04-24T09:08:07.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.124+0000 7fcd59600640 1 --2- 172.21.15.83:0/1308685017 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd5410c490 0x7fcd5410e910 unknown :-1 s=CLOSED pgs=2289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.124+0000 7fcd59600640 1 --2- 172.21.15.83:0/1308685017 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd541039e0 0x7fcd5410bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.124+0000 7fcd59600640 1 --2- 172.21.15.83:0/1308685017 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcd541030c0 0x7fcd541034a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.124+0000 7fcd59600640 1 -- 172.21.15.83:0/1308685017 >> 172.21.15.83:0/1308685017 conn(0x7fcd540fc880 msgr2=0x7fcd540feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:07.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.124+0000 7fcd59600640 1 -- 172.21.15.83:0/1308685017 shutdown_connections 2024-04-24T09:08:07.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.124+0000 7fcd59600640 1 -- 172.21.15.83:0/1308685017 wait complete. 2024-04-24T09:08:07.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.125+0000 7fcd59600640 1 Processor -- start 2024-04-24T09:08:07.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.126+0000 7fcd59600640 1 -- start start 2024-04-24T09:08:07.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.126+0000 7fcd59600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd541030c0 0x7fcd541a05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:07.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.126+0000 7fcd59600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcd541039e0 0x7fcd541a0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:07.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.126+0000 7fcd59600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd5410c490 0x7fcd5419a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:07.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.126+0000 7fcd59600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcd54112150 con 0x7fcd541030c0 2024-04-24T09:08:07.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.126+0000 7fcd59600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcd54111fd0 con 0x7fcd5410c490 2024-04-24T09:08:07.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.126+0000 7fcd59600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fcd541122d0 con 0x7fcd541039e0 2024-04-24T09:08:07.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.127+0000 7fcd53e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd541030c0 0x7fcd541a05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:07.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.127+0000 7fcd53e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd541030c0 0x7fcd541a05b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55346/0 (socket says 172.21.15.83:55346) 2024-04-24T09:08:07.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.127+0000 7fcd53e00640 1 -- 172.21.15.83:0/213091265 learned_addr learned my addr 172.21.15.83:0/213091265 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:08:07.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.127+0000 7fcd58c00640 1 --2- 172.21.15.83:0/213091265 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd5410c490 0x7fcd5419a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:07.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.127+0000 7fcd53e00640 1 -- 172.21.15.83:0/213091265 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcd541039e0 msgr2=0x7fcd541a0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:07.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.127+0000 7fcd53e00640 1 --2- 172.21.15.83:0/213091265 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcd541039e0 0x7fcd541a0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.127+0000 7fcd53e00640 1 -- 172.21.15.83:0/213091265 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd5410c490 msgr2=0x7fcd5419a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:07.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.127+0000 7fcd53e00640 1 --2- 172.21.15.83:0/213091265 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd5410c490 0x7fcd5419a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.127+0000 7fcd53e00640 1 -- 172.21.15.83:0/213091265 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd5419af30 con 0x7fcd541030c0 2024-04-24T09:08:07.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.127+0000 7fcd53e00640 1 --2- 172.21.15.83:0/213091265 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd541030c0 0x7fcd541a05b0 secure :-1 s=READY pgs=2290 cs=0 l=1 rev1=1 crypto rx=0x7fcd4400d770 tx=0x7fcd4400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:07.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.128+0000 7fcd50c00640 1 -- 172.21.15.83:0/213091265 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd44017070 con 0x7fcd541030c0 2024-04-24T09:08:07.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.128+0000 7fcd50c00640 1 -- 172.21.15.83:0/213091265 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcd440054d0 con 0x7fcd541030c0 2024-04-24T09:08:07.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.128+0000 7fcd59600640 1 -- 172.21.15.83:0/213091265 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd5419b220 con 0x7fcd541030c0 2024-04-24T09:08:07.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.128+0000 7fcd50c00640 1 -- 172.21.15.83:0/213091265 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd44004cf0 con 0x7fcd541030c0 2024-04-24T09:08:07.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.128+0000 7fcd59600640 1 -- 172.21.15.83:0/213091265 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd541a7530 con 0x7fcd541030c0 2024-04-24T09:08:07.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.132+0000 7fcd50c00640 1 -- 172.21.15.83:0/213091265 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fcd44004e90 con 0x7fcd541030c0 2024-04-24T09:08:07.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.133+0000 7fcd5da13740 1 Processor -- start 2024-04-24T09:08:07.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.133+0000 7fcd5da13740 1 -- start start 2024-04-24T09:08:07.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.133+0000 7fcd5da13740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560b780f0b40 0x560b780f0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:07.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.133+0000 7fcd5da13740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560b780f1460 0x560b780fb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:07.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.133+0000 7fcd5da13740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560b780fbb30 0x560b780fdf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:07.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.133+0000 7fcd5da13740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x560b780561d0 con 0x560b780f0b40 2024-04-24T09:08:07.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.133+0000 7fcd5da13740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560b77ffc450 con 0x560b780f1460 2024-04-24T09:08:07.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.133+0000 7fcd5da13740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x560b78052190 con 0x560b780fbb30 2024-04-24T09:08:07.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.134+0000 7fcd58c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560b780f0b40 0x560b780f0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:07.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.134+0000 7fcd58c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560b780f0b40 0x560b780f0f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55360/0 (socket says 172.21.15.83:55360) 2024-04-24T09:08:07.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.134+0000 7fcd58c00640 1 -- 172.21.15.83:0/959910934 learned_addr learned my addr 172.21.15.83:0/959910934 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:08:07.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.134+0000 7fcd53400640 1 --2- 172.21.15.83:0/959910934 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560b780fbb30 0x560b780fdf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:07.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.134+0000 7fcd53e00640 1 --2- 172.21.15.83:0/959910934 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560b780f1460 0x560b780fb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:07.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.135+0000 7fcd58c00640 1 -- 172.21.15.83:0/959910934 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560b780fbb30 msgr2=0x560b780fdf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:07.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.135+0000 7fcd58c00640 1 --2- 172.21.15.83:0/959910934 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560b780fbb30 0x560b780fdf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.135+0000 7fcd58c00640 1 -- 172.21.15.83:0/959910934 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560b780f1460 msgr2=0x560b780fb5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:07.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.135+0000 7fcd58c00640 1 --2- 172.21.15.83:0/959910934 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560b780f1460 0x560b780fb5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.135+0000 7fcd50c00640 1 --2- 172.21.15.83:0/213091265 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fcd380a3da0 0x7fcd380a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:07.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.135+0000 7fcd58c00640 1 -- 172.21.15.83:0/959910934 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b780b9310 con 0x560b780f0b40 2024-04-24T09:08:07.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.135+0000 7fcd53400640 1 --2- 172.21.15.83:0/959910934 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560b780fbb30 0x560b780fdf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:08:07.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.135+0000 7fcd50c00640 1 -- 172.21.15.83:0/213091265 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9671+0+0 (secure 0 0 0) 0x7fcd440d0350 con 0x7fcd541030c0 2024-04-24T09:08:07.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.135+0000 7fcd58c00640 1 --2- 172.21.15.83:0/959910934 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560b780f0b40 0x560b780f0f20 secure :-1 s=READY pgs=2291 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c009ca0 tx=0x7fcd4c02f450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6b0d1e3ca87623 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:07.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.136+0000 7fcd53e00640 1 --2- 172.21.15.83:0/213091265 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fcd380a3da0 0x7fcd380a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:07.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.136+0000 7fcd49200640 1 -- 172.21.15.83:0/959910934 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd4c048070 con 0x560b780f0b40 2024-04-24T09:08:07.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.136+0000 7fcd49200640 1 -- 172.21.15.83:0/959910934 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcd4c003e20 con 0x560b780f0b40 2024-04-24T09:08:07.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.136+0000 7fcd49200640 1 -- 172.21.15.83:0/959910934 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd4c034040 con 0x560b780f0b40 2024-04-24T09:08:07.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.137+0000 7fcd5da13740 1 -- 172.21.15.83:0/959910934 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560b780f0b40 msgr2=0x560b780f0f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:07.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.137+0000 7fcd5da13740 1 --2- 172.21.15.83:0/959910934 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560b780f0b40 0x560b780f0f20 secure :-1 s=READY pgs=2291 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c009ca0 tx=0x7fcd4c02f450 comp rx=0 tx=0).stop 2024-04-24T09:08:07.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.137+0000 7fcd53e00640 1 --2- 172.21.15.83:0/213091265 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fcd380a3da0 0x7fcd380a6260 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x560b780f2100 tx=0x7fcd440bc040 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:07.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.137+0000 7fcd5da13740 1 -- 172.21.15.83:0/959910934 shutdown_connections 2024-04-24T09:08:07.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.137+0000 7fcd5da13740 1 --2- 172.21.15.83:0/959910934 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560b780fbb30 0x560b780fdf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.137+0000 7fcd5da13740 1 --2- 172.21.15.83:0/959910934 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560b780f1460 0x560b780fb5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.137+0000 7fcd5da13740 1 --2- 172.21.15.83:0/959910934 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560b780f0b40 0x560b780f0f20 unknown :-1 s=CLOSED pgs=2291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.137+0000 7fcd5da13740 1 -- 172.21.15.83:0/959910934 >> 172.21.15.83:0/959910934 conn(0x560b780ece60 msgr2=0x560b780ed240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:07.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.137+0000 7fcd5da13740 1 -- 172.21.15.83:0/959910934 shutdown_connections 2024-04-24T09:08:07.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.138+0000 7fcd5da13740 1 -- 172.21.15.83:0/959910934 wait complete. 2024-04-24T09:08:07.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.138+0000 7fcd5da13740 1 -- start start 2024-04-24T09:08:07.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.138+0000 7fcd5da13740 1 Processor -- start 2024-04-24T09:08:07.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.139+0000 7fcd5da13740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560b780f0b40 0x560b780f88f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:07.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.139+0000 7fcd5da13740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560b780f1460 0x560b780f8e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:07.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.139+0000 7fcd5da13740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560b780fbb30 0x560b780f9370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:07.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.139+0000 7fcd5da13740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x560b780fe960 con 0x560b780fbb30 2024-04-24T09:08:07.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.139+0000 7fcd5da13740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560b7800f810 con 0x560b780f1460 2024-04-24T09:08:07.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.139+0000 7fcd5da13740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x560b780feae0 con 0x560b780f0b40 2024-04-24T09:08:07.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.139+0000 7fcd5da13740 10 client.0 _ll_register_callbacks cb 0x560b77fdd140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:08:07.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.140+0000 7fcd58c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560b780f0b40 0x560b780f88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:07.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.140+0000 7fcd58c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560b780f0b40 0x560b780f88f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:38308/0 (socket says 172.21.15.83:38308) 2024-04-24T09:08:07.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.140+0000 7fcd58c00640 1 -- 172.21.15.83:0/1687017345 learned_addr learned my addr 172.21.15.83:0/1687017345 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:08:07.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.140+0000 7fcd53400640 1 --2- 172.21.15.83:0/1687017345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560b780f1460 0x560b780f8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:07.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.140+0000 7fcd58c00640 1 --2- 172.21.15.83:0/1687017345 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560b780fbb30 0x560b780f9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:07.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.141+0000 7fcd53400640 1 -- 172.21.15.83:0/1687017345 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560b780f0b40 msgr2=0x560b780f88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:07.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.141+0000 7fcd53400640 1 --2- 172.21.15.83:0/1687017345 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560b780f0b40 0x560b780f88f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.141+0000 7fcd53400640 1 -- 172.21.15.83:0/1687017345 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560b780fbb30 msgr2=0x560b780f9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:07.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.141+0000 7fcd53400640 1 --2- 172.21.15.83:0/1687017345 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560b780fbb30 0x560b780f9370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.141+0000 7fcd53400640 1 -- 172.21.15.83:0/1687017345 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b780203f0 con 0x560b780f1460 2024-04-24T09:08:07.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.141+0000 7fcd58c00640 1 --2- 172.21.15.83:0/1687017345 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560b780fbb30 0x560b780f9370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:08:07.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.142+0000 7fcd53400640 1 --2- 172.21.15.83:0/1687017345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560b780f1460 0x560b780f8e30 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fcd5419bec0 tx=0x7fcd400118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ffbd3be9425fa675 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:07.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.142+0000 7fcd3d600640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:08:07.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.143+0000 7fcd3d600640 1 -- 172.21.15.83:0/1687017345 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd40005020 con 0x560b780f1460 2024-04-24T09:08:07.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.143+0000 7fcd3d600640 1 -- 172.21.15.83:0/1687017345 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9671+0+0 (secure 0 0 0) 0x7fcd400051c0 con 0x560b780f1460 2024-04-24T09:08:07.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.143+0000 7fcd5da13740 1 -- 172.21.15.83:0/1687017345 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd4c003e20 con 0x560b780f1460 2024-04-24T09:08:07.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.144+0000 7fcd3d600640 1 -- 172.21.15.83:0/1687017345 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7fcd40024050 con 0x560b780f1460 2024-04-24T09:08:07.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.144+0000 7fcd5da13740 10 client.38510 fetch_fsmap learned FSMap version 53 2024-04-24T09:08:07.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.144+0000 7fcd5da13740 1 -- 172.21.15.83:0/1687017345 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x560b77d683f0 con 0x560b780f1460 2024-04-24T09:08:07.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.144+0000 7fcd3d600640 1 -- 172.21.15.83:0/1687017345 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcd4c003e20 con 0x560b780f1460 2024-04-24T09:08:07.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.144+0000 7fcd3d600640 1 -- 172.21.15.83:0/1687017345 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 53) ==== 8398+0+0 (secure 0 0 0) 0x7fcd40003610 con 0x560b780f1460 2024-04-24T09:08:07.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.145+0000 7fcd5da13740 10 client.38510 fetch_fsmap finished waiting for FSMap version 53 2024-04-24T09:08:07.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.145+0000 7fcd5da13740 10 client.38510 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-24T09:08:07.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.145+0000 7fcd5da13740 20 client.38510 populate_metadata read hostname 'smithi083' 2024-04-24T09:08:07.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.145+0000 7fcd5da13740 1 --2- 172.21.15.83:0/1687017345 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x560b78114660 0x560b78116a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:07.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.145+0000 7fcd5da13740 4 client.38510 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:08:07.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.145+0000 7fcd5da13740 1 -- 172.21.15.83:0/1687017345 --> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560b78063bf0 con 0x560b78114660 2024-04-24T09:08:07.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.145+0000 7fcd58c00640 1 --2- 172.21.15.83:0/1687017345 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x560b78114660 0x560b78116a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:07.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.146+0000 7fcd58c00640 1 --2- 172.21.15.83:0/1687017345 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x560b78114660 0x560b78116a80 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9b02868b1996b15f server_cookie=c88ed29c8f1254f3 in_seq=0 out_seq=0 2024-04-24T09:08:07.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.146+0000 7fcd3d600640 10 client.38510 ms_handle_connect on v2:172.21.15.138:6834/1909219758 2024-04-24T09:08:07.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.147+0000 7fcd3d600640 1 -- 172.21.15.83:0/1687017345 <== mds.2 v2:172.21.15.138:6834/1909219758 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560b78063bf0 con 0x560b78114660 2024-04-24T09:08:07.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.147+0000 7fcd3d600640 10 client.38510 handle_command_reply: tid=0 2024-04-24T09:08:07.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.147+0000 7fcd5da13740 1 client.38510 shutdown 2024-04-24T09:08:07.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.148+0000 7fcd5da13740 1 -- 172.21.15.83:0/1687017345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560b780f1460 msgr2=0x560b780f8e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:07.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.148+0000 7fcd5da13740 1 --2- 172.21.15.83:0/1687017345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560b780f1460 0x560b780f8e30 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fcd5419bec0 tx=0x7fcd400118d0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.148+0000 7fcd5da13740 1 -- 172.21.15.83:0/1687017345 shutdown_connections 2024-04-24T09:08:07.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.148+0000 7fcd5da13740 1 --2- 172.21.15.83:0/1687017345 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x560b78114660 0x560b78116a80 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.148+0000 7fcd5da13740 1 --2- 172.21.15.83:0/1687017345 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560b780fbb30 0x560b780f9370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.148+0000 7fcd5da13740 1 --2- 172.21.15.83:0/1687017345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560b780f1460 0x560b780f8e30 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.148+0000 7fcd5da13740 1 --2- 172.21.15.83:0/1687017345 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560b780f0b40 0x560b780f88f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.148+0000 7fcd5da13740 1 -- 172.21.15.83:0/1687017345 >> 172.21.15.83:0/1687017345 conn(0x560b780ece60 msgr2=0x560b780f2b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:07.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.148+0000 7fcd3d600640 0 client.38510 ms_handle_reset on v2:172.21.15.138:6834/1909219758 2024-04-24T09:08:07.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.148+0000 7fcd5da13740 1 -- 172.21.15.83:0/1687017345 shutdown_connections 2024-04-24T09:08:07.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.148+0000 7fcd5da13740 1 -- 172.21.15.83:0/1687017345 wait complete. 2024-04-24T09:08:07.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.148+0000 7fcd5da13740 20 client.38510 trim_cache size 0 max 16384 2024-04-24T09:08:07.164 INFO:tasks.workunit.client.0.smithi083.stdout: 10 20420 7.13 MB/sec execute 38 sec latency 90.817 ms 2024-04-24T09:08:07.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.170+0000 7fcd5da13740 1 Processor -- start 2024-04-24T09:08:07.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.170+0000 7fcd5da13740 1 -- start start 2024-04-24T09:08:07.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.170+0000 7fcd5da13740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560b780f0b40 0x560b780f9190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:07.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.170+0000 7fcd5da13740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560b780f1460 0x560b7811c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:07.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.170+0000 7fcd5da13740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560b780fbb30 0x560b7811c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:07.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.170+0000 7fcd5da13740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcd541ad310 con 0x560b780fbb30 2024-04-24T09:08:07.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.170+0000 7fcd5da13740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcd541aef70 con 0x560b780f0b40 2024-04-24T09:08:07.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.170+0000 7fcd5da13740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fcd541ad9e0 con 0x560b780f1460 2024-04-24T09:08:07.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.171+0000 7fcd58c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560b780f0b40 0x560b780f9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:07.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.171+0000 7fcd53400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560b780f1460 0x560b7811c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:07.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.171+0000 7fcd58c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560b780f0b40 0x560b780f9190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:58514/0 (socket says 172.21.15.83:58514) 2024-04-24T09:08:07.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.171+0000 7fcd58c00640 1 -- 172.21.15.83:0/2064335886 learned_addr learned my addr 172.21.15.83:0/2064335886 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:08:07.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.171+0000 7fcd58c00640 1 --2- 172.21.15.83:0/2064335886 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560b780fbb30 0x560b7811c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:07.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.171+0000 7fcd58c00640 1 -- 172.21.15.83:0/2064335886 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560b780f1460 msgr2=0x560b7811c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:07.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.171+0000 7fcd58c00640 1 --2- 172.21.15.83:0/2064335886 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560b780f1460 0x560b7811c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.171+0000 7fcd58c00640 1 -- 172.21.15.83:0/2064335886 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560b780fbb30 msgr2=0x560b7811c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:07.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.171+0000 7fcd58c00640 1 --2- 172.21.15.83:0/2064335886 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560b780fbb30 0x560b7811c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.171+0000 7fcd58c00640 1 -- 172.21.15.83:0/2064335886 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b77e327d0 con 0x560b780f0b40 2024-04-24T09:08:07.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.171+0000 7fcd53400640 1 --2- 172.21.15.83:0/2064335886 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560b780f1460 0x560b7811c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:08:07.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.172+0000 7fcd58c00640 1 --2- 172.21.15.83:0/2064335886 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560b780f0b40 0x560b780f9190 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c02fd00 tx=0x7fcd4c005df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f421219e05cb5e39 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:07.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.172+0000 7fcd3fe00640 1 -- 172.21.15.83:0/2064335886 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd4c048040 con 0x560b780f0b40 2024-04-24T09:08:07.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.172+0000 7fcd3fe00640 1 -- 172.21.15.83:0/2064335886 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcd4c003e20 con 0x560b780f0b40 2024-04-24T09:08:07.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.172+0000 7fcd3fe00640 1 -- 172.21.15.83:0/2064335886 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd4c004770 con 0x560b780f0b40 2024-04-24T09:08:07.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.173+0000 7fcd5da13740 1 -- 172.21.15.83:0/2064335886 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560b780f0b40 msgr2=0x560b780f9190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:07.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.173+0000 7fcd5da13740 1 --2- 172.21.15.83:0/2064335886 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560b780f0b40 0x560b780f9190 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c02fd00 tx=0x7fcd4c005df0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.173+0000 7fcd5da13740 1 -- 172.21.15.83:0/2064335886 shutdown_connections 2024-04-24T09:08:07.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.173+0000 7fcd5da13740 1 --2- 172.21.15.83:0/2064335886 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560b780fbb30 0x560b7811c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.173+0000 7fcd5da13740 1 --2- 172.21.15.83:0/2064335886 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560b780f1460 0x560b7811c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.173+0000 7fcd5da13740 1 --2- 172.21.15.83:0/2064335886 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560b780f0b40 0x560b780f9190 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.173+0000 7fcd5da13740 1 -- 172.21.15.83:0/2064335886 >> 172.21.15.83:0/2064335886 conn(0x560b780ece60 msgr2=0x560b780f7920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:07.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.174+0000 7fcd5da13740 1 -- 172.21.15.83:0/2064335886 shutdown_connections 2024-04-24T09:08:07.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.174+0000 7fcd5da13740 1 -- 172.21.15.83:0/2064335886 wait complete. 2024-04-24T09:08:07.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.174+0000 7fcd5da13740 1 -- start start 2024-04-24T09:08:07.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.174+0000 7fcd5da13740 1 Processor -- start 2024-04-24T09:08:07.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.175+0000 7fcd5da13740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560b780f0b40 0x560b780fc700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:07.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.175+0000 7fcd5da13740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560b780f1460 0x560b780fcc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:07.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.175+0000 7fcd5da13740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560b780fbb30 0x560b7810b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:07.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.175+0000 7fcd5da13740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcd541ae7e0 con 0x560b780fbb30 2024-04-24T09:08:07.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.175+0000 7fcd5da13740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcd541ae0e0 con 0x560b780f1460 2024-04-24T09:08:07.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.175+0000 7fcd5da13740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fcd541ab310 con 0x560b780f0b40 2024-04-24T09:08:07.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.175+0000 7fcd5da13740 10 client.0 _ll_register_callbacks cb 0x7fcd40009650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:08:07.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.175+0000 7fcd53400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560b780f1460 0x560b780fcc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:07.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.175+0000 7fcd58c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560b780f0b40 0x560b780fc700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:07.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.175+0000 7fcd53400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560b780f1460 0x560b780fcc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:58526/0 (socket says 172.21.15.83:58526) 2024-04-24T09:08:07.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.175+0000 7fcd53400640 1 -- 172.21.15.83:0/2132415657 learned_addr learned my addr 172.21.15.83:0/2132415657 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:08:07.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.175+0000 7fcd58c00640 1 --2- 172.21.15.83:0/2132415657 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560b780fbb30 0x560b7810b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:07.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.176+0000 7fcd58c00640 1 -- 172.21.15.83:0/2132415657 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560b780f1460 msgr2=0x560b780fcc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:07.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.176+0000 7fcd58c00640 1 --2- 172.21.15.83:0/2132415657 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560b780f1460 0x560b780fcc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.176+0000 7fcd58c00640 1 -- 172.21.15.83:0/2132415657 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560b780fbb30 msgr2=0x560b7810b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:07.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.176+0000 7fcd58c00640 1 --2- 172.21.15.83:0/2132415657 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560b780fbb30 0x560b7810b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.176+0000 7fcd58c00640 1 -- 172.21.15.83:0/2132415657 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b7810bc30 con 0x560b780f0b40 2024-04-24T09:08:07.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.176+0000 7fcd58c00640 1 --2- 172.21.15.83:0/2132415657 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560b780f0b40 0x560b780fc700 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c009890 tx=0x7fcd4c02f450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ae165ebdc510082 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:07.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.176+0000 7fcd3d600640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-04-24T09:08:07.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.177+0000 7fcd3d600640 1 -- 172.21.15.83:0/2132415657 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd4c034040 con 0x560b780f0b40 2024-04-24T09:08:07.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.177+0000 7fcd5da13740 1 -- 172.21.15.83:0/2132415657 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd4c003e20 con 0x560b780f0b40 2024-04-24T09:08:07.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.177+0000 7fcd3d600640 1 -- 172.21.15.83:0/2132415657 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9671+0+0 (secure 0 0 0) 0x7fcd4c040040 con 0x560b780f0b40 2024-04-24T09:08:07.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.177+0000 7fcd3d600640 1 -- 172.21.15.83:0/2132415657 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7fcd4c048040 con 0x560b780f0b40 2024-04-24T09:08:07.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.177+0000 7fcd5da13740 10 client.28534 fetch_fsmap learned FSMap version 53 2024-04-24T09:08:07.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.177+0000 7fcd5da13740 1 -- 172.21.15.83:0/2132415657 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x560b7817abc0 con 0x560b780f0b40 2024-04-24T09:08:07.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.178+0000 7fcd3d600640 1 -- 172.21.15.83:0/2132415657 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcd4c003e20 con 0x560b780f0b40 2024-04-24T09:08:07.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.178+0000 7fcd3d600640 1 -- 172.21.15.83:0/2132415657 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 53) ==== 8398+0+0 (secure 0 0 0) 0x7fcd4c04e040 con 0x560b780f0b40 2024-04-24T09:08:07.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.178+0000 7fcd5da13740 10 client.28534 fetch_fsmap finished waiting for FSMap version 53 2024-04-24T09:08:07.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.178+0000 7fcd5da13740 10 client.28534 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-24T09:08:07.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.178+0000 7fcd5da13740 20 client.28534 populate_metadata read hostname 'smithi083' 2024-04-24T09:08:07.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.178+0000 7fcd5da13740 1 --2- 172.21.15.83:0/2132415657 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x560b7817ad90 0x560b7817d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:07.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.178+0000 7fcd5da13740 4 client.28534 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:08:07.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.178+0000 7fcd5da13740 1 -- 172.21.15.83:0/2132415657 --> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560b7805da60 con 0x560b7817ad90 2024-04-24T09:08:07.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.179+0000 7fcd53400640 1 --2- 172.21.15.83:0/2132415657 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x560b7817ad90 0x560b7817d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:07.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.180+0000 7fcd53400640 1 --2- 172.21.15.83:0/2132415657 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x560b7817ad90 0x560b7817d180 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b4796fcbcf527b48 server_cookie=ef9275ddb2aa3267 in_seq=0 out_seq=0 2024-04-24T09:08:07.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.180+0000 7fcd3d600640 10 client.28534 ms_handle_connect on v2:172.21.15.138:6834/1909219758 2024-04-24T09:08:07.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.180+0000 7fcd3d600640 1 -- 172.21.15.83:0/2132415657 <== mds.2 v2:172.21.15.138:6834/1909219758 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560b7805da60 con 0x560b7817ad90 2024-04-24T09:08:07.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.180+0000 7fcd3d600640 10 client.28534 handle_command_reply: tid=0 2024-04-24T09:08:07.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.180+0000 7fcd5da13740 1 client.28534 shutdown 2024-04-24T09:08:07.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.180+0000 7fcd5da13740 1 -- 172.21.15.83:0/2132415657 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560b780f0b40 msgr2=0x560b780fc700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:07.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.180+0000 7fcd5da13740 1 --2- 172.21.15.83:0/2132415657 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560b780f0b40 0x560b780fc700 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c009890 tx=0x7fcd4c02f450 comp rx=0 tx=0).stop 2024-04-24T09:08:07.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.180+0000 7fcd5da13740 1 -- 172.21.15.83:0/2132415657 shutdown_connections 2024-04-24T09:08:07.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.180+0000 7fcd5da13740 1 --2- 172.21.15.83:0/2132415657 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x560b7817ad90 0x560b7817d180 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.180+0000 7fcd5da13740 1 --2- 172.21.15.83:0/2132415657 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560b780fbb30 0x560b7810b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.180+0000 7fcd5da13740 1 --2- 172.21.15.83:0/2132415657 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560b780f1460 0x560b780fcc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.180+0000 7fcd5da13740 1 --2- 172.21.15.83:0/2132415657 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560b780f0b40 0x560b780fc700 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.180+0000 7fcd3d600640 0 client.28534 ms_handle_reset on v2:172.21.15.138:6834/1909219758 2024-04-24T09:08:07.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.180+0000 7fcd5da13740 1 -- 172.21.15.83:0/2132415657 >> 172.21.15.83:0/2132415657 conn(0x560b780ece60 msgr2=0x560b780f2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:07.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.181+0000 7fcd5da13740 1 -- 172.21.15.83:0/2132415657 shutdown_connections 2024-04-24T09:08:07.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.181+0000 7fcd5da13740 1 -- 172.21.15.83:0/2132415657 wait complete. 2024-04-24T09:08:07.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.181+0000 7fcd5da13740 20 client.28534 trim_cache size 0 max 16384 2024-04-24T09:08:07.183 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:08:07.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.182+0000 7fcd59600640 1 -- 172.21.15.83:0/213091265 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fcd380a3da0 msgr2=0x7fcd380a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:07.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.182+0000 7fcd59600640 1 --2- 172.21.15.83:0/213091265 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fcd380a3da0 0x7fcd380a6260 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x560b780f2100 tx=0x7fcd440bc040 comp rx=0 tx=0).stop 2024-04-24T09:08:07.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.182+0000 7fcd59600640 1 -- 172.21.15.83:0/213091265 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd541030c0 msgr2=0x7fcd541a05b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:07.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.182+0000 7fcd59600640 1 --2- 172.21.15.83:0/213091265 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd541030c0 0x7fcd541a05b0 secure :-1 s=READY pgs=2290 cs=0 l=1 rev1=1 crypto rx=0x7fcd4400d770 tx=0x7fcd4400dc30 comp rx=0 tx=0).stop 2024-04-24T09:08:07.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.182+0000 7fcd59600640 1 -- 172.21.15.83:0/213091265 shutdown_connections 2024-04-24T09:08:07.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.182+0000 7fcd59600640 1 --2- 172.21.15.83:0/213091265 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fcd380a3da0 0x7fcd380a6260 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.182+0000 7fcd59600640 1 --2- 172.21.15.83:0/213091265 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd5410c490 0x7fcd5419a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.182+0000 7fcd59600640 1 --2- 172.21.15.83:0/213091265 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcd541039e0 0x7fcd541a0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.182+0000 7fcd59600640 1 --2- 172.21.15.83:0/213091265 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd541030c0 0x7fcd541a05b0 secure :-1 s=CLOSED pgs=2290 cs=0 l=1 rev1=1 crypto rx=0x7fcd4400d770 tx=0x7fcd4400dc30 comp rx=0 tx=0).stop 2024-04-24T09:08:07.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.182+0000 7fcd59600640 1 -- 172.21.15.83:0/213091265 >> 172.21.15.83:0/213091265 conn(0x7fcd540fc880 msgr2=0x7fcd54100c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:07.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.182+0000 7fcd59600640 1 -- 172.21.15.83:0/213091265 shutdown_connections 2024-04-24T09:08:07.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.182+0000 7fcd59600640 1 -- 172.21.15.83:0/213091265 wait complete. 2024-04-24T09:08:07.204 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T09:08:07.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:08:07 smithi138 ceph-mon[31441]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:08:07.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:08:07 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/3810335009' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:08:07.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:08:07 smithi189 ceph-mon[31791]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:08:07.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:08:07 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/3810335009' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:08:07.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.464+0000 7ff925a00640 1 -- 172.21.15.83:0/1380342639 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff920115150 msgr2=0x7ff920117540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:07.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.464+0000 7ff925a00640 1 --2- 172.21.15.83:0/1380342639 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff920115150 0x7ff920117540 secure :-1 s=READY pgs=2292 cs=0 l=1 rev1=1 crypto rx=0x7ff918009900 tx=0x7ff91802a410 comp rx=0 tx=0).stop 2024-04-24T09:08:07.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.464+0000 7ff925a00640 1 -- 172.21.15.83:0/1380342639 shutdown_connections 2024-04-24T09:08:07.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.464+0000 7ff925a00640 1 --2- 172.21.15.83:0/1380342639 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff920115150 0x7ff920117540 unknown :-1 s=CLOSED pgs=2292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.464+0000 7ff925a00640 1 --2- 172.21.15.83:0/1380342639 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff920109920 0x7ff920114c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.464+0000 7ff925a00640 1 --2- 172.21.15.83:0/1380342639 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff920109000 0x7ff9201093e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.464+0000 7ff925a00640 1 -- 172.21.15.83:0/1380342639 >> 172.21.15.83:0/1380342639 conn(0x7ff92006cf80 msgr2=0x7ff92006d390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:07.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.464+0000 7ff925a00640 1 -- 172.21.15.83:0/1380342639 shutdown_connections 2024-04-24T09:08:07.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.465+0000 7ff925a00640 1 -- 172.21.15.83:0/1380342639 wait complete. 2024-04-24T09:08:07.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.466+0000 7ff925a00640 1 Processor -- start 2024-04-24T09:08:07.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.466+0000 7ff925a00640 1 -- start start 2024-04-24T09:08:07.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.466+0000 7ff925a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff920109000 0x7ff9201a3100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:07.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.466+0000 7ff925a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff920109920 0x7ff9201a3640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:07.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.466+0000 7ff925a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff920115150 0x7ff9201a7970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:07.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.466+0000 7ff925a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff92011a920 con 0x7ff920109920 2024-04-24T09:08:07.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.466+0000 7ff925a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff92011a7a0 con 0x7ff920115150 2024-04-24T09:08:07.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.466+0000 7ff925a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff92011aaa0 con 0x7ff920109000 2024-04-24T09:08:07.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.467+0000 7ff917400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff920109920 0x7ff9201a3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:07.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.467+0000 7ff917400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff920109920 0x7ff9201a3640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55398/0 (socket says 172.21.15.83:55398) 2024-04-24T09:08:07.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.467+0000 7ff917400640 1 -- 172.21.15.83:0/2155025335 learned_addr learned my addr 172.21.15.83:0/2155025335 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:08:07.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.467+0000 7ff925000640 1 --2- 172.21.15.83:0/2155025335 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff920115150 0x7ff9201a7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:07.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.467+0000 7ff91fe00640 1 --2- 172.21.15.83:0/2155025335 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff920109000 0x7ff9201a3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:07.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.467+0000 7ff917400640 1 -- 172.21.15.83:0/2155025335 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff920109000 msgr2=0x7ff9201a3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:07.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.467+0000 7ff917400640 1 --2- 172.21.15.83:0/2155025335 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff920109000 0x7ff9201a3100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.467+0000 7ff917400640 1 -- 172.21.15.83:0/2155025335 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff920115150 msgr2=0x7ff9201a7970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:07.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.467+0000 7ff917400640 1 --2- 172.21.15.83:0/2155025335 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff920115150 0x7ff9201a7970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.467+0000 7ff917400640 1 -- 172.21.15.83:0/2155025335 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9201a80a0 con 0x7ff920109920 2024-04-24T09:08:07.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.467+0000 7ff925000640 1 --2- 172.21.15.83:0/2155025335 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff920115150 0x7ff9201a7970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:08:07.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.468+0000 7ff91fe00640 1 --2- 172.21.15.83:0/2155025335 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff920109000 0x7ff9201a3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:08:07.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.468+0000 7ff917400640 1 --2- 172.21.15.83:0/2155025335 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff920109920 0x7ff9201a3640 secure :-1 s=READY pgs=2293 cs=0 l=1 rev1=1 crypto rx=0x7ff90800ebc0 tx=0x7ff90800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:07.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.468+0000 7ff91d600640 1 -- 172.21.15.83:0/2155025335 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff908010630 con 0x7ff920109920 2024-04-24T09:08:07.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.468+0000 7ff91d600640 1 -- 172.21.15.83:0/2155025335 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff908010c80 con 0x7ff920109920 2024-04-24T09:08:07.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.468+0000 7ff91d600640 1 -- 172.21.15.83:0/2155025335 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff908018b30 con 0x7ff920109920 2024-04-24T09:08:07.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.468+0000 7ff925a00640 1 -- 172.21.15.83:0/2155025335 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9201a8390 con 0x7ff920109920 2024-04-24T09:08:07.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.468+0000 7ff925a00640 1 -- 172.21.15.83:0/2155025335 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9201afc20 con 0x7ff920109920 2024-04-24T09:08:07.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.473+0000 7ff929f96740 1 Processor -- start 2024-04-24T09:08:07.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.473+0000 7ff929f96740 1 -- start start 2024-04-24T09:08:07.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.473+0000 7ff929f96740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5581ff8c2890 0x5581ff8c2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:07.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.473+0000 7ff929f96740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5581ff8c3240 0x5581ff8d5e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:07.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.473+0000 7ff929f96740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581ff8d6340 0x5581ff8d8730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:07.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.474+0000 7ff929f96740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5581ff8281d0 con 0x5581ff8d6340 2024-04-24T09:08:07.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.474+0000 7ff929f96740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5581ff7ce450 con 0x5581ff8c2890 2024-04-24T09:08:07.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.474+0000 7ff929f96740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5581ff824190 con 0x5581ff8c3240 2024-04-24T09:08:07.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.474+0000 7ff91d600640 1 -- 172.21.15.83:0/2155025335 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff908018cd0 con 0x7ff920109920 2024-04-24T09:08:07.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.474+0000 7ff917400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581ff8d6340 0x5581ff8d8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:07.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.474+0000 7ff917400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581ff8d6340 0x5581ff8d8730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55410/0 (socket says 172.21.15.83:55410) 2024-04-24T09:08:07.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.474+0000 7ff917400640 1 -- 172.21.15.83:0/2673176587 learned_addr learned my addr 172.21.15.83:0/2673176587 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:08:07.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.474+0000 7ff925000640 1 --2- 172.21.15.83:0/2673176587 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5581ff8c2890 0x5581ff8c2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:07.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.475+0000 7ff91fe00640 1 --2- 172.21.15.83:0/2673176587 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5581ff8c3240 0x5581ff8d5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:07.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.475+0000 7ff917400640 1 -- 172.21.15.83:0/2673176587 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5581ff8c3240 msgr2=0x5581ff8d5e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:07.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.475+0000 7ff917400640 1 --2- 172.21.15.83:0/2673176587 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5581ff8c3240 0x5581ff8d5e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.475+0000 7ff917400640 1 -- 172.21.15.83:0/2673176587 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5581ff8c2890 msgr2=0x5581ff8c2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:07.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.475+0000 7ff917400640 1 --2- 172.21.15.83:0/2673176587 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5581ff8c2890 0x5581ff8c2c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.475+0000 7ff917400640 1 -- 172.21.15.83:0/2673176587 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581ff88b310 con 0x5581ff8d6340 2024-04-24T09:08:07.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.475+0000 7ff91fe00640 1 --2- 172.21.15.83:0/2673176587 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5581ff8c3240 0x5581ff8d5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:08:07.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.476+0000 7ff91d600640 1 --2- 172.21.15.83:0/2155025335 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff9000a3da0 0x7ff9000a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:07.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.476+0000 7ff917400640 1 --2- 172.21.15.83:0/2673176587 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581ff8d6340 0x5581ff8d8730 secure :-1 s=READY pgs=2294 cs=0 l=1 rev1=1 crypto rx=0x7ff9080ce540 tx=0x7ff9080d9970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b075c86f47c2e2d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:07.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.476+0000 7ff925000640 1 --2- 172.21.15.83:0/2155025335 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff9000a3da0 0x7ff9000a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:07.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.477+0000 7ff915600640 1 -- 172.21.15.83:0/2673176587 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9080e5070 con 0x5581ff8d6340 2024-04-24T09:08:07.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.477+0000 7ff915600640 1 -- 172.21.15.83:0/2673176587 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff90809d550 con 0x5581ff8d6340 2024-04-24T09:08:07.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.477+0000 7ff915600640 1 -- 172.21.15.83:0/2673176587 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9080a1200 con 0x5581ff8d6340 2024-04-24T09:08:07.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.477+0000 7ff91d600640 1 -- 172.21.15.83:0/2155025335 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9671+0+0 (secure 0 0 0) 0x7ff9080d1390 con 0x7ff920109920 2024-04-24T09:08:07.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.478+0000 7ff925000640 1 --2- 172.21.15.83:0/2155025335 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff9000a3da0 0x7ff9000a6260 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7ff918006010 tx=0x7ff9180058b0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:07.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.479+0000 7ff929f96740 1 -- 172.21.15.83:0/2673176587 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581ff8d6340 msgr2=0x5581ff8d8730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:07.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.479+0000 7ff929f96740 1 --2- 172.21.15.83:0/2673176587 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581ff8d6340 0x5581ff8d8730 secure :-1 s=READY pgs=2294 cs=0 l=1 rev1=1 crypto rx=0x7ff9080ce540 tx=0x7ff9080d9970 comp rx=0 tx=0).stop 2024-04-24T09:08:07.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.479+0000 7ff929f96740 1 -- 172.21.15.83:0/2673176587 shutdown_connections 2024-04-24T09:08:07.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.479+0000 7ff929f96740 1 --2- 172.21.15.83:0/2673176587 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581ff8d6340 0x5581ff8d8730 unknown :-1 s=CLOSED pgs=2294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.479+0000 7ff929f96740 1 --2- 172.21.15.83:0/2673176587 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5581ff8c3240 0x5581ff8d5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.479+0000 7ff929f96740 1 --2- 172.21.15.83:0/2673176587 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5581ff8c2890 0x5581ff8c2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.479+0000 7ff929f96740 1 -- 172.21.15.83:0/2673176587 >> 172.21.15.83:0/2673176587 conn(0x5581ff8bee60 msgr2=0x5581ff8bf240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:07.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.479+0000 7ff929f96740 1 -- 172.21.15.83:0/2673176587 shutdown_connections 2024-04-24T09:08:07.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.479+0000 7ff929f96740 1 -- 172.21.15.83:0/2673176587 wait complete. 2024-04-24T09:08:07.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.479+0000 7ff929f96740 1 -- start start 2024-04-24T09:08:07.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.480+0000 7ff929f96740 1 Processor -- start 2024-04-24T09:08:07.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.480+0000 7ff929f96740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581ff8c2890 0x5581ff8cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:07.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.480+0000 7ff929f96740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5581ff8c3240 0x5581ff8d0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:07.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.480+0000 7ff929f96740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5581ff8d6340 0x5581ff8d5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:07.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.480+0000 7ff929f96740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5581ff8c59b0 con 0x5581ff8c2890 2024-04-24T09:08:07.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.480+0000 7ff929f96740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5581ff7e1810 con 0x5581ff8c3240 2024-04-24T09:08:07.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.480+0000 7ff929f96740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5581ff8c5b30 con 0x5581ff8d6340 2024-04-24T09:08:07.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.480+0000 7ff929f96740 10 client.0 _ll_register_callbacks cb 0x5581ff7af140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:08:07.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.481+0000 7ff91fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581ff8c2890 0x5581ff8cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:07.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.481+0000 7ff91fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581ff8c2890 0x5581ff8cfdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55414/0 (socket says 172.21.15.83:55414) 2024-04-24T09:08:07.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.481+0000 7ff91fe00640 1 -- 172.21.15.83:0/3754447918 learned_addr learned my addr 172.21.15.83:0/3754447918 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:08:07.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.481+0000 7ff917400640 1 --2- 172.21.15.83:0/3754447918 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5581ff8c3240 0x5581ff8d0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:07.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.481+0000 7ff925000640 1 --2- 172.21.15.83:0/3754447918 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5581ff8d6340 0x5581ff8d5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:07.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.481+0000 7ff91fe00640 1 -- 172.21.15.83:0/3754447918 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5581ff8d6340 msgr2=0x5581ff8d5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:07.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.481+0000 7ff91fe00640 1 --2- 172.21.15.83:0/3754447918 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5581ff8d6340 0x5581ff8d5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.481+0000 7ff91fe00640 1 -- 172.21.15.83:0/3754447918 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5581ff8c3240 msgr2=0x5581ff8d0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:07.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.481+0000 7ff91fe00640 1 --2- 172.21.15.83:0/3754447918 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5581ff8c3240 0x5581ff8d0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.481+0000 7ff91fe00640 1 -- 172.21.15.83:0/3754447918 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581ff7f23f0 con 0x5581ff8c2890 2024-04-24T09:08:07.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.482+0000 7ff91fe00640 1 --2- 172.21.15.83:0/3754447918 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581ff8c2890 0x5581ff8cfdc0 secure :-1 s=READY pgs=2295 cs=0 l=1 rev1=1 crypto rx=0x7ff9100126f0 tx=0x7ff910012bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41c81017e411f75d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:07.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.482+0000 7ff90e000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:08:07.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.482+0000 7ff90e000640 1 -- 172.21.15.83:0/3754447918 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff91001f060 con 0x5581ff8c2890 2024-04-24T09:08:07.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.482+0000 7ff90e000640 1 -- 172.21.15.83:0/3754447918 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9671+0+0 (secure 0 0 0) 0x7ff910010d60 con 0x5581ff8c2890 2024-04-24T09:08:07.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.482+0000 7ff929f96740 1 -- 172.21.15.83:0/3754447918 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff90809d550 con 0x5581ff8c2890 2024-04-24T09:08:07.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.483+0000 7ff90e000640 1 -- 172.21.15.83:0/3754447918 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7ff91001a3f0 con 0x5581ff8c2890 2024-04-24T09:08:07.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.483+0000 7ff929f96740 10 client.21384 fetch_fsmap learned FSMap version 53 2024-04-24T09:08:07.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.483+0000 7ff929f96740 1 -- 172.21.15.83:0/3754447918 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x5581ff53a3f0 con 0x5581ff8c2890 2024-04-24T09:08:07.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.483+0000 7ff90e000640 1 -- 172.21.15.83:0/3754447918 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff90809d550 con 0x5581ff8c2890 2024-04-24T09:08:07.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.484+0000 7ff90e000640 1 -- 172.21.15.83:0/3754447918 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 53) ==== 8398+0+0 (secure 0 0 0) 0x7ff910019030 con 0x5581ff8c2890 2024-04-24T09:08:07.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.484+0000 7ff929f96740 10 client.21384 fetch_fsmap finished waiting for FSMap version 53 2024-04-24T09:08:07.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.484+0000 7ff929f96740 10 client.21384 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-24T09:08:07.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.484+0000 7ff929f96740 20 client.21384 populate_metadata read hostname 'smithi083' 2024-04-24T09:08:07.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.484+0000 7ff929f96740 1 --2- 172.21.15.83:0/3754447918 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x5581ff8eed40 0x5581ff8f1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:07.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.484+0000 7ff929f96740 4 client.21384 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:08:07.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.484+0000 7ff929f96740 1 -- 172.21.15.83:0/3754447918 --> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5581ff835bf0 con 0x5581ff8eed40 2024-04-24T09:08:07.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.484+0000 7ff917400640 1 --2- 172.21.15.83:0/3754447918 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x5581ff8eed40 0x5581ff8f1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:07.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.485+0000 7ff917400640 1 --2- 172.21.15.83:0/3754447918 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x5581ff8eed40 0x5581ff8f1130 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d23e4b66269a65a1 server_cookie=c78d33ddf05ef55a in_seq=0 out_seq=0 2024-04-24T09:08:07.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.485+0000 7ff90e000640 10 client.21384 ms_handle_connect on v2:172.21.15.138:6836/900566754 2024-04-24T09:08:07.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.486+0000 7ff90e000640 1 -- 172.21.15.83:0/3754447918 <== mds.1 v2:172.21.15.138:6836/900566754 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5581ff835bf0 con 0x5581ff8eed40 2024-04-24T09:08:07.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.486+0000 7ff90e000640 10 client.21384 handle_command_reply: tid=0 2024-04-24T09:08:07.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.486+0000 7ff929f96740 1 client.21384 shutdown 2024-04-24T09:08:07.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.487+0000 7ff929f96740 1 -- 172.21.15.83:0/3754447918 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581ff8c2890 msgr2=0x5581ff8cfdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:07.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.487+0000 7ff929f96740 1 --2- 172.21.15.83:0/3754447918 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581ff8c2890 0x5581ff8cfdc0 secure :-1 s=READY pgs=2295 cs=0 l=1 rev1=1 crypto rx=0x7ff9100126f0 tx=0x7ff910012bb0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.487+0000 7ff929f96740 1 -- 172.21.15.83:0/3754447918 shutdown_connections 2024-04-24T09:08:07.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.487+0000 7ff929f96740 1 --2- 172.21.15.83:0/3754447918 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x5581ff8eed40 0x5581ff8f1130 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.487+0000 7ff90e000640 0 client.21384 ms_handle_reset on v2:172.21.15.138:6836/900566754 2024-04-24T09:08:07.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.487+0000 7ff929f96740 1 --2- 172.21.15.83:0/3754447918 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5581ff8d6340 0x5581ff8d5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.487+0000 7ff929f96740 1 --2- 172.21.15.83:0/3754447918 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5581ff8c3240 0x5581ff8d0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.487+0000 7ff929f96740 1 --2- 172.21.15.83:0/3754447918 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581ff8c2890 0x5581ff8cfdc0 unknown :-1 s=CLOSED pgs=2295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.487+0000 7ff929f96740 1 -- 172.21.15.83:0/3754447918 >> 172.21.15.83:0/3754447918 conn(0x5581ff8bee60 msgr2=0x5581ff8c4cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:07.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.487+0000 7ff929f96740 1 -- 172.21.15.83:0/3754447918 shutdown_connections 2024-04-24T09:08:07.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.488+0000 7ff929f96740 1 -- 172.21.15.83:0/3754447918 wait complete. 2024-04-24T09:08:07.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.488+0000 7ff929f96740 20 client.21384 trim_cache size 0 max 16384 2024-04-24T09:08:07.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.511+0000 7ff929f96740 1 Processor -- start 2024-04-24T09:08:07.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.512+0000 7ff929f96740 1 -- start start 2024-04-24T09:08:07.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.512+0000 7ff929f96740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5581ff8c2890 0x5581ff8bddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:07.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.512+0000 7ff929f96740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581ff8c3240 0x5581ff8f69d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:07.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.512+0000 7ff929f96740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5581ff8d6340 0x5581ff8f6f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:07.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.512+0000 7ff929f96740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff9201b5cd0 con 0x5581ff8c3240 2024-04-24T09:08:07.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.512+0000 7ff929f96740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff9201b7930 con 0x5581ff8c2890 2024-04-24T09:08:07.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.512+0000 7ff929f96740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff9201b63a0 con 0x5581ff8d6340 2024-04-24T09:08:07.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.512+0000 7ff91fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5581ff8c2890 0x5581ff8bddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:07.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.512+0000 7ff91fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5581ff8c2890 0x5581ff8bddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:58578/0 (socket says 172.21.15.83:58578) 2024-04-24T09:08:07.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.512+0000 7ff91fe00640 1 -- 172.21.15.83:0/453612454 learned_addr learned my addr 172.21.15.83:0/453612454 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:08:07.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.513+0000 7ff917400640 1 --2- 172.21.15.83:0/453612454 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581ff8c3240 0x5581ff8f69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:07.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.513+0000 7ff925000640 1 --2- 172.21.15.83:0/453612454 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5581ff8d6340 0x5581ff8f6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:07.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.513+0000 7ff91fe00640 1 -- 172.21.15.83:0/453612454 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5581ff8d6340 msgr2=0x5581ff8f6f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:07.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.513+0000 7ff91fe00640 1 --2- 172.21.15.83:0/453612454 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5581ff8d6340 0x5581ff8f6f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.513+0000 7ff91fe00640 1 -- 172.21.15.83:0/453612454 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581ff8c3240 msgr2=0x5581ff8f69d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:07.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.513+0000 7ff91fe00640 1 --2- 172.21.15.83:0/453612454 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581ff8c3240 0x5581ff8f69d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.513+0000 7ff91fe00640 1 -- 172.21.15.83:0/453612454 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581ff6047d0 con 0x5581ff8c2890 2024-04-24T09:08:07.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.513+0000 7ff91fe00640 1 --2- 172.21.15.83:0/453612454 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5581ff8c2890 0x5581ff8bddd0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff910015040 tx=0x7ff910009bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb02943a609c9c05 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:07.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.514+0000 7ff914c00640 1 -- 172.21.15.83:0/453612454 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff910029080 con 0x5581ff8c2890 2024-04-24T09:08:07.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.514+0000 7ff914c00640 1 -- 172.21.15.83:0/453612454 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff9100236b0 con 0x5581ff8c2890 2024-04-24T09:08:07.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.514+0000 7ff914c00640 1 -- 172.21.15.83:0/453612454 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff910005310 con 0x5581ff8c2890 2024-04-24T09:08:07.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.514+0000 7ff929f96740 1 -- 172.21.15.83:0/453612454 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5581ff8c2890 msgr2=0x5581ff8bddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:07.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.514+0000 7ff929f96740 1 --2- 172.21.15.83:0/453612454 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5581ff8c2890 0x5581ff8bddd0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff910015040 tx=0x7ff910009bd0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.515+0000 7ff929f96740 1 -- 172.21.15.83:0/453612454 shutdown_connections 2024-04-24T09:08:07.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.515+0000 7ff929f96740 1 --2- 172.21.15.83:0/453612454 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5581ff8d6340 0x5581ff8f6f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.515+0000 7ff929f96740 1 --2- 172.21.15.83:0/453612454 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581ff8c3240 0x5581ff8f69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.515+0000 7ff929f96740 1 --2- 172.21.15.83:0/453612454 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5581ff8c2890 0x5581ff8bddd0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.515+0000 7ff929f96740 1 -- 172.21.15.83:0/453612454 >> 172.21.15.83:0/453612454 conn(0x5581ff8bee60 msgr2=0x5581ff8bd6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:07.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.515+0000 7ff929f96740 1 -- 172.21.15.83:0/453612454 shutdown_connections 2024-04-24T09:08:07.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.515+0000 7ff929f96740 1 -- 172.21.15.83:0/453612454 wait complete. 2024-04-24T09:08:07.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.515+0000 7ff929f96740 1 -- start start 2024-04-24T09:08:07.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.516+0000 7ff929f96740 1 Processor -- start 2024-04-24T09:08:07.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.517+0000 7ff929f96740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581ff8c2890 0x5581ff901fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:07.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.517+0000 7ff929f96740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5581ff8c3240 0x5581ff902510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:07.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.517+0000 7ff929f96740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5581ff8d6340 0x5581ff8fcf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:07.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.517+0000 7ff929f96740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff9201b71a0 con 0x5581ff8c2890 2024-04-24T09:08:07.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.517+0000 7ff929f96740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff9201b6aa0 con 0x5581ff8c3240 2024-04-24T09:08:07.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.517+0000 7ff929f96740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff9201b3d30 con 0x5581ff8d6340 2024-04-24T09:08:07.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.517+0000 7ff929f96740 10 client.0 _ll_register_callbacks cb 0x7ff910009530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:08:07.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.518+0000 7ff917400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5581ff8c3240 0x5581ff902510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:07.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.518+0000 7ff917400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5581ff8c3240 0x5581ff902510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:58582/0 (socket says 172.21.15.83:58582) 2024-04-24T09:08:07.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.518+0000 7ff917400640 1 -- 172.21.15.83:0/4129004949 learned_addr learned my addr 172.21.15.83:0/4129004949 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:08:07.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.518+0000 7ff925000640 1 --2- 172.21.15.83:0/4129004949 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5581ff8d6340 0x5581ff8fcf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:07.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.518+0000 7ff91fe00640 1 --2- 172.21.15.83:0/4129004949 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581ff8c2890 0x5581ff901fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:07.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.518+0000 7ff917400640 1 -- 172.21.15.83:0/4129004949 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5581ff8d6340 msgr2=0x5581ff8fcf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:07.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.518+0000 7ff917400640 1 --2- 172.21.15.83:0/4129004949 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5581ff8d6340 0x5581ff8fcf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.518+0000 7ff917400640 1 -- 172.21.15.83:0/4129004949 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581ff8c2890 msgr2=0x5581ff901fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:07.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.518+0000 7ff917400640 1 --2- 172.21.15.83:0/4129004949 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581ff8c2890 0x5581ff901fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.518+0000 7ff917400640 1 -- 172.21.15.83:0/4129004949 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581ff8fd780 con 0x5581ff8c3240 2024-04-24T09:08:07.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.518+0000 7ff917400640 1 --2- 172.21.15.83:0/4129004949 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5581ff8c3240 0x5581ff902510 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff908000f80 tx=0x7ff90801f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=736eec5aa104604f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:07.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.518+0000 7ff90e000640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:08:07.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.519+0000 7ff90e000640 1 -- 172.21.15.83:0/4129004949 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9080990f0 con 0x5581ff8c3240 2024-04-24T09:08:07.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.519+0000 7ff90e000640 1 -- 172.21.15.83:0/4129004949 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9671+0+0 (secure 0 0 0) 0x7ff90809aef0 con 0x5581ff8c3240 2024-04-24T09:08:07.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.519+0000 7ff929f96740 1 -- 172.21.15.83:0/4129004949 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9100236b0 con 0x5581ff8c3240 2024-04-24T09:08:07.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.519+0000 7ff90e000640 1 -- 172.21.15.83:0/4129004949 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7ff90809c990 con 0x5581ff8c3240 2024-04-24T09:08:07.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.519+0000 7ff929f96740 10 client.38534 fetch_fsmap learned FSMap version 53 2024-04-24T09:08:07.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.519+0000 7ff929f96740 1 -- 172.21.15.83:0/4129004949 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x5581ff9557e0 con 0x5581ff8c3240 2024-04-24T09:08:07.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.519+0000 7ff90e000640 1 -- 172.21.15.83:0/4129004949 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff9100236b0 con 0x5581ff8c3240 2024-04-24T09:08:07.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.520+0000 7ff90e000640 1 -- 172.21.15.83:0/4129004949 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 53) ==== 8398+0+0 (secure 0 0 0) 0x7ff9080994c0 con 0x5581ff8c3240 2024-04-24T09:08:07.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.520+0000 7ff929f96740 10 client.38534 fetch_fsmap finished waiting for FSMap version 53 2024-04-24T09:08:07.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.520+0000 7ff929f96740 10 client.38534 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-24T09:08:07.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.520+0000 7ff929f96740 20 client.38534 populate_metadata read hostname 'smithi083' 2024-04-24T09:08:07.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.520+0000 7ff929f96740 1 --2- 172.21.15.83:0/4129004949 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x5581ff955a60 0x5581ff957e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:07.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.520+0000 7ff929f96740 4 client.38534 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:08:07.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.520+0000 7ff929f96740 1 -- 172.21.15.83:0/4129004949 --> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5581ff82fa60 con 0x5581ff955a60 2024-04-24T09:08:07.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.521+0000 7ff91fe00640 1 --2- 172.21.15.83:0/4129004949 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x5581ff955a60 0x5581ff957e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:07.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.522+0000 7ff91fe00640 1 --2- 172.21.15.83:0/4129004949 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x5581ff955a60 0x5581ff957e50 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f834ca0ec2abe612 server_cookie=139abfaddad1c80f in_seq=0 out_seq=0 2024-04-24T09:08:07.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.522+0000 7ff90e000640 10 client.38534 ms_handle_connect on v2:172.21.15.138:6836/900566754 2024-04-24T09:08:07.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.522+0000 7ff90e000640 1 -- 172.21.15.83:0/4129004949 <== mds.1 v2:172.21.15.138:6836/900566754 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5581ff82fa60 con 0x5581ff955a60 2024-04-24T09:08:07.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.522+0000 7ff90e000640 10 client.38534 handle_command_reply: tid=0 2024-04-24T09:08:07.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.522+0000 7ff929f96740 1 client.38534 shutdown 2024-04-24T09:08:07.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.523+0000 7ff929f96740 1 -- 172.21.15.83:0/4129004949 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5581ff8c3240 msgr2=0x5581ff902510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:07.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.523+0000 7ff929f96740 1 --2- 172.21.15.83:0/4129004949 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5581ff8c3240 0x5581ff902510 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff908000f80 tx=0x7ff90801f000 comp rx=0 tx=0).stop 2024-04-24T09:08:07.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.523+0000 7ff929f96740 1 -- 172.21.15.83:0/4129004949 shutdown_connections 2024-04-24T09:08:07.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.523+0000 7ff929f96740 1 --2- 172.21.15.83:0/4129004949 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x5581ff955a60 0x5581ff957e50 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.523+0000 7ff929f96740 1 --2- 172.21.15.83:0/4129004949 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5581ff8d6340 0x5581ff8fcf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.523+0000 7ff929f96740 1 --2- 172.21.15.83:0/4129004949 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5581ff8c3240 0x5581ff902510 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.523+0000 7ff929f96740 1 --2- 172.21.15.83:0/4129004949 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581ff8c2890 0x5581ff901fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.523+0000 7ff929f96740 1 -- 172.21.15.83:0/4129004949 >> 172.21.15.83:0/4129004949 conn(0x5581ff8bee60 msgr2=0x5581ff8bd6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:07.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.523+0000 7ff90e000640 0 client.38534 ms_handle_reset on v2:172.21.15.138:6836/900566754 2024-04-24T09:08:07.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.523+0000 7ff929f96740 1 -- 172.21.15.83:0/4129004949 shutdown_connections 2024-04-24T09:08:07.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.523+0000 7ff929f96740 1 -- 172.21.15.83:0/4129004949 wait complete. 2024-04-24T09:08:07.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.523+0000 7ff929f96740 20 client.38534 trim_cache size 0 max 16384 2024-04-24T09:08:07.524 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:08:07.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.524+0000 7ff925a00640 1 -- 172.21.15.83:0/2155025335 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff9000a3da0 msgr2=0x7ff9000a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:07.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.524+0000 7ff925a00640 1 --2- 172.21.15.83:0/2155025335 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff9000a3da0 0x7ff9000a6260 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7ff918006010 tx=0x7ff9180058b0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.524+0000 7ff925a00640 1 -- 172.21.15.83:0/2155025335 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff920109920 msgr2=0x7ff9201a3640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:07.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.524+0000 7ff925a00640 1 --2- 172.21.15.83:0/2155025335 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff920109920 0x7ff9201a3640 secure :-1 s=READY pgs=2293 cs=0 l=1 rev1=1 crypto rx=0x7ff90800ebc0 tx=0x7ff90800c6a0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.524+0000 7ff925a00640 1 -- 172.21.15.83:0/2155025335 shutdown_connections 2024-04-24T09:08:07.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.524+0000 7ff925a00640 1 --2- 172.21.15.83:0/2155025335 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff9000a3da0 0x7ff9000a6260 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.524+0000 7ff925a00640 1 --2- 172.21.15.83:0/2155025335 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff920115150 0x7ff9201a7970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.524+0000 7ff925a00640 1 --2- 172.21.15.83:0/2155025335 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff920109920 0x7ff9201a3640 unknown :-1 s=CLOSED pgs=2293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.524+0000 7ff925a00640 1 --2- 172.21.15.83:0/2155025335 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff920109000 0x7ff9201a3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.524+0000 7ff925a00640 1 -- 172.21.15.83:0/2155025335 >> 172.21.15.83:0/2155025335 conn(0x7ff92006cf80 msgr2=0x7ff92010aa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:07.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.525+0000 7ff925a00640 1 -- 172.21.15.83:0/2155025335 shutdown_connections 2024-04-24T09:08:07.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.525+0000 7ff925a00640 1 -- 172.21.15.83:0/2155025335 wait complete. 2024-04-24T09:08:07.543 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:08:07 smithi083 ceph-mon[27907]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:08:07.543 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:08:07 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/3810335009' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:08:07.546 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T09:08:07.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.787+0000 7fbffce00640 1 -- 172.21.15.83:0/3221615782 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbff8104320 msgr2=0x7fbff8104700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:07.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.787+0000 7fbffce00640 1 --2- 172.21.15.83:0/3221615782 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbff8104320 0x7fbff8104700 secure :-1 s=READY pgs=2296 cs=0 l=1 rev1=1 crypto rx=0x7fbfe8009900 tx=0x7fbfe802a410 comp rx=0 tx=0).stop 2024-04-24T09:08:07.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.788+0000 7fbffce00640 1 -- 172.21.15.83:0/3221615782 shutdown_connections 2024-04-24T09:08:07.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.788+0000 7fbffce00640 1 --2- 172.21.15.83:0/3221615782 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbff81093f0 0x7fbff8110fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.788+0000 7fbffce00640 1 --2- 172.21.15.83:0/3221615782 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbff8104cd0 0x7fbff8108d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.788+0000 7fbffce00640 1 --2- 172.21.15.83:0/3221615782 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbff8104320 0x7fbff8104700 unknown :-1 s=CLOSED pgs=2296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.788+0000 7fbffce00640 1 -- 172.21.15.83:0/3221615782 >> 172.21.15.83:0/3221615782 conn(0x7fbff80ffb40 msgr2=0x7fbff8101f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:07.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.789+0000 7fbffce00640 1 -- 172.21.15.83:0/3221615782 shutdown_connections 2024-04-24T09:08:07.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.789+0000 7fbffce00640 1 -- 172.21.15.83:0/3221615782 wait complete. 2024-04-24T09:08:07.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.790+0000 7fbffce00640 1 Processor -- start 2024-04-24T09:08:07.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.790+0000 7fbffce00640 1 -- start start 2024-04-24T09:08:07.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.791+0000 7fbffce00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbff8104320 0x7fbff81a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:07.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.791+0000 7fbffce00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbff81093f0 0x7fbff81a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:07.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.791+0000 7fbffce00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbff819a720 0x7fbff819aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:07.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.791+0000 7fbffce00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbff81143a0 con 0x7fbff8104320 2024-04-24T09:08:07.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.791+0000 7fbffce00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbff8114220 con 0x7fbff81093f0 2024-04-24T09:08:07.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.791+0000 7fbffce00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fbff8114520 con 0x7fbff819a720 2024-04-24T09:08:07.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.791+0000 7fbfefe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbff81093f0 0x7fbff81a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:07.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.791+0000 7fbfefe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbff81093f0 0x7fbff81a0ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:58602/0 (socket says 172.21.15.83:58602) 2024-04-24T09:08:07.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.791+0000 7fbfefe00640 1 -- 172.21.15.83:0/1945926505 learned_addr learned my addr 172.21.15.83:0/1945926505 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:08:07.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.791+0000 7fbff7e00640 1 --2- 172.21.15.83:0/1945926505 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbff819a720 0x7fbff819aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:07.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.792+0000 7fbff7400640 1 --2- 172.21.15.83:0/1945926505 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbff8104320 0x7fbff81a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:07.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.792+0000 7fbff7e00640 1 -- 172.21.15.83:0/1945926505 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbff81093f0 msgr2=0x7fbff81a0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:07.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.792+0000 7fbff7e00640 1 --2- 172.21.15.83:0/1945926505 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbff81093f0 0x7fbff81a0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.792+0000 7fbff7e00640 1 -- 172.21.15.83:0/1945926505 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbff8104320 msgr2=0x7fbff81a0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:07.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.792+0000 7fbff7e00640 1 --2- 172.21.15.83:0/1945926505 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbff8104320 0x7fbff81a0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.792+0000 7fbff7e00640 1 -- 172.21.15.83:0/1945926505 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbff819b320 con 0x7fbff819a720 2024-04-24T09:08:07.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.793+0000 7fbff7e00640 1 --2- 172.21.15.83:0/1945926505 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbff819a720 0x7fbff819aba0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fbff000e940 tx=0x7fbff000ee00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:07.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.793+0000 7fbff4c00640 1 -- 172.21.15.83:0/1945926505 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbff0004d20 con 0x7fbff819a720 2024-04-24T09:08:07.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.793+0000 7fbffce00640 1 -- 172.21.15.83:0/1945926505 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbff81a7400 con 0x7fbff819a720 2024-04-24T09:08:07.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.794+0000 7fbffce00640 1 -- 172.21.15.83:0/1945926505 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbff81a7800 con 0x7fbff819a720 2024-04-24T09:08:07.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.794+0000 7fbff4c00640 1 -- 172.21.15.83:0/1945926505 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbff0005020 con 0x7fbff819a720 2024-04-24T09:08:07.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.794+0000 7fbff4c00640 1 -- 172.21.15.83:0/1945926505 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbff00116d0 con 0x7fbff819a720 2024-04-24T09:08:07.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.797+0000 7fc0013b3740 1 Processor -- start 2024-04-24T09:08:07.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.797+0000 7fc0013b3740 1 -- start start 2024-04-24T09:08:07.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.798+0000 7fc0013b3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5583d8dae890 0x5583d8daec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:07.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.798+0000 7fc0013b3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5583d8daf240 0x5583d8dc1e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:07.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.798+0000 7fc0013b3740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5583d8dc2340 0x5583d8dc4730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:07.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.798+0000 7fc0013b3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5583d8d141d0 con 0x5583d8dae890 2024-04-24T09:08:07.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.798+0000 7fc0013b3740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5583d8cba450 con 0x5583d8dc2340 2024-04-24T09:08:07.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.798+0000 7fc0013b3740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5583d8d10190 con 0x5583d8daf240 2024-04-24T09:08:07.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.799+0000 7fbfefe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5583d8dae890 0x5583d8daec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:07.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.799+0000 7fbfefe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5583d8dae890 0x5583d8daec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55462/0 (socket says 172.21.15.83:55462) 2024-04-24T09:08:07.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.799+0000 7fbfefe00640 1 -- 172.21.15.83:0/342362309 learned_addr learned my addr 172.21.15.83:0/342362309 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:08:07.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.799+0000 7fbfefe00640 1 --2- 172.21.15.83:0/342362309 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5583d8dc2340 0x5583d8dc4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:07.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.799+0000 7fbff7400640 1 --2- 172.21.15.83:0/342362309 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5583d8daf240 0x5583d8dc1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:07.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.799+0000 7fbfefe00640 1 -- 172.21.15.83:0/342362309 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5583d8daf240 msgr2=0x5583d8dc1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:07.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.799+0000 7fbfefe00640 1 --2- 172.21.15.83:0/342362309 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5583d8daf240 0x5583d8dc1e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.799+0000 7fbfefe00640 1 -- 172.21.15.83:0/342362309 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5583d8dc2340 msgr2=0x5583d8dc4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:07.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.799+0000 7fbfefe00640 1 --2- 172.21.15.83:0/342362309 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5583d8dc2340 0x5583d8dc4730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.799+0000 7fbfefe00640 1 -- 172.21.15.83:0/342362309 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583d8d77310 con 0x5583d8dae890 2024-04-24T09:08:07.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.802+0000 7fbff4c00640 1 -- 172.21.15.83:0/1945926505 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fbff000cb20 con 0x7fbff819a720 2024-04-24T09:08:07.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.802+0000 7fbfefe00640 1 --2- 172.21.15.83:0/342362309 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5583d8dae890 0x5583d8daec70 secure :-1 s=READY pgs=2297 cs=0 l=1 rev1=1 crypto rx=0x7fbfe0012640 tx=0x7fbfe000b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=885cd49a6ba2db94 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:07.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.803+0000 7fbfecc00640 1 -- 172.21.15.83:0/342362309 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfe0010730 con 0x5583d8dae890 2024-04-24T09:08:07.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.803+0000 7fbfecc00640 1 -- 172.21.15.83:0/342362309 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbfe0010d80 con 0x5583d8dae890 2024-04-24T09:08:07.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.803+0000 7fbfecc00640 1 -- 172.21.15.83:0/342362309 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfe001dcb0 con 0x5583d8dae890 2024-04-24T09:08:07.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.804+0000 7fc0013b3740 1 -- 172.21.15.83:0/342362309 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5583d8dae890 msgr2=0x5583d8daec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:07.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.804+0000 7fc0013b3740 1 --2- 172.21.15.83:0/342362309 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5583d8dae890 0x5583d8daec70 secure :-1 s=READY pgs=2297 cs=0 l=1 rev1=1 crypto rx=0x7fbfe0012640 tx=0x7fbfe000b8c0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.804+0000 7fbff4c00640 1 --2- 172.21.15.83:0/1945926505 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fbfdc0a3de0 0x7fbfdc0a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:07.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.805+0000 7fc0013b3740 1 -- 172.21.15.83:0/342362309 shutdown_connections 2024-04-24T09:08:07.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.805+0000 7fc0013b3740 1 --2- 172.21.15.83:0/342362309 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5583d8dc2340 0x5583d8dc4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.805+0000 7fc0013b3740 1 --2- 172.21.15.83:0/342362309 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5583d8daf240 0x5583d8dc1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.805+0000 7fc0013b3740 1 --2- 172.21.15.83:0/342362309 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5583d8dae890 0x5583d8daec70 unknown :-1 s=CLOSED pgs=2297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.805+0000 7fc0013b3740 1 -- 172.21.15.83:0/342362309 >> 172.21.15.83:0/342362309 conn(0x5583d8daae60 msgr2=0x5583d8dab240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:07.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.805+0000 7fbff4c00640 1 -- 172.21.15.83:0/1945926505 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9671+0+0 (secure 0 0 0) 0x7fbff00d0af0 con 0x7fbff819a720 2024-04-24T09:08:07.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.805+0000 7fc0013b3740 1 -- 172.21.15.83:0/342362309 shutdown_connections 2024-04-24T09:08:07.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.805+0000 7fbfefe00640 1 --2- 172.21.15.83:0/1945926505 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fbfdc0a3de0 0x7fbfdc0a62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:07.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.805+0000 7fc0013b3740 1 -- 172.21.15.83:0/342362309 wait complete. 2024-04-24T09:08:07.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.806+0000 7fc0013b3740 1 -- start start 2024-04-24T09:08:07.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.806+0000 7fc0013b3740 1 Processor -- start 2024-04-24T09:08:07.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.806+0000 7fc0013b3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5583d8dae890 0x5583d8dc1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:07.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.806+0000 7fc0013b3740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5583d8daf240 0x5583d8dbf090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:07.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.806+0000 7fc0013b3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5583d8dc2340 0x5583d8dbf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:07.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.806+0000 7fc0013b3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5583d8dc51a0 con 0x5583d8dae890 2024-04-24T09:08:07.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.806+0000 7fc0013b3740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5583d8ccd810 con 0x5583d8daf240 2024-04-24T09:08:07.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.806+0000 7fc0013b3740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5583d8dc5320 con 0x5583d8dc2340 2024-04-24T09:08:07.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.807+0000 7fc0013b3740 10 client.0 _ll_register_callbacks cb 0x5583d8c9b140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:08:07.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.807+0000 7fbff7400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5583d8dae890 0x5583d8dc1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:07.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.807+0000 7fbff7400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5583d8dae890 0x5583d8dc1da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55476/0 (socket says 172.21.15.83:55476) 2024-04-24T09:08:07.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.807+0000 7fbff7400640 1 -- 172.21.15.83:0/2927831994 learned_addr learned my addr 172.21.15.83:0/2927831994 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:08:07.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.808+0000 7fbff7400640 1 -- 172.21.15.83:0/2927831994 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5583d8dc2340 msgr2=0x5583d8dbf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:07.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.808+0000 7fbff7400640 1 --2- 172.21.15.83:0/2927831994 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5583d8dc2340 0x5583d8dbf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.808+0000 7fbff7400640 1 -- 172.21.15.83:0/2927831994 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5583d8daf240 msgr2=0x5583d8dbf090 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T09:08:07.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.808+0000 7fbff7400640 1 --2- 172.21.15.83:0/2927831994 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5583d8daf240 0x5583d8dbf090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.808+0000 7fbff7400640 1 -- 172.21.15.83:0/2927831994 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583d8cde3f0 con 0x5583d8dae890 2024-04-24T09:08:07.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.808+0000 7fbfefe00640 1 --2- 172.21.15.83:0/1945926505 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fbfdc0a3de0 0x7fbfdc0a62a0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fbfe001d920 tx=0x7fbfe001d890 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:07.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.808+0000 7fbff7400640 1 --2- 172.21.15.83:0/2927831994 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5583d8dae890 0x5583d8dc1da0 secure :-1 s=READY pgs=2298 cs=0 l=1 rev1=1 crypto rx=0x7fbfe8009230 tx=0x7fbfe8004440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=593c8a9046696167 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:07.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.808+0000 7fbfe5600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:08:07.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.808+0000 7fbfe5600640 1 -- 172.21.15.83:0/2927831994 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfe8035040 con 0x5583d8dae890 2024-04-24T09:08:07.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.808+0000 7fc0013b3740 1 -- 172.21.15.83:0/2927831994 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbfe0010d80 con 0x5583d8dae890 2024-04-24T09:08:07.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.809+0000 7fbfe5600640 1 -- 172.21.15.83:0/2927831994 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9671+0+0 (secure 0 0 0) 0x7fbfe8005450 con 0x5583d8dae890 2024-04-24T09:08:07.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.809+0000 7fbfe5600640 1 -- 172.21.15.83:0/2927831994 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7fbfe8004780 con 0x5583d8dae890 2024-04-24T09:08:07.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.809+0000 7fc0013b3740 10 client.21402 fetch_fsmap learned FSMap version 53 2024-04-24T09:08:07.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.809+0000 7fc0013b3740 1 -- 172.21.15.83:0/2927831994 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x5583d8a263f0 con 0x5583d8dae890 2024-04-24T09:08:07.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.810+0000 7fbfe5600640 1 -- 172.21.15.83:0/2927831994 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbfe0010d80 con 0x5583d8dae890 2024-04-24T09:08:07.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.810+0000 7fbfe5600640 1 -- 172.21.15.83:0/2927831994 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 53) ==== 8398+0+0 (secure 0 0 0) 0x7fbfe8003b70 con 0x5583d8dae890 2024-04-24T09:08:07.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.810+0000 7fc0013b3740 10 client.21402 fetch_fsmap finished waiting for FSMap version 53 2024-04-24T09:08:07.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.810+0000 7fc0013b3740 10 client.21402 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:08:07.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.810+0000 7fc0013b3740 20 client.21402 populate_metadata read hostname 'smithi083' 2024-04-24T09:08:07.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.810+0000 7fc0013b3740 1 --2- 172.21.15.83:0/2927831994 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5583d8dd2b70 0x5583d8dd5030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:07.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.810+0000 7fc0013b3740 4 client.21402 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:08:07.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.810+0000 7fc0013b3740 1 -- 172.21.15.83:0/2927831994 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5583d8d21bf0 con 0x5583d8dd2b70 2024-04-24T09:08:07.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.810+0000 7fbfefe00640 1 --2- 172.21.15.83:0/2927831994 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5583d8dd2b70 0x5583d8dd5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:07.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.811+0000 7fbfefe00640 1 --2- 172.21.15.83:0/2927831994 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5583d8dd2b70 0x5583d8dd5030 crc :-1 s=READY pgs=417 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5985ae3e48cfbbfe server_cookie=185fb27a03fe5feb in_seq=0 out_seq=0 2024-04-24T09:08:07.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.812+0000 7fbfe5600640 10 client.21402 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:08:07.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.813+0000 7fbfe5600640 1 -- 172.21.15.83:0/2927831994 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5583d8d21bf0 con 0x5583d8dd2b70 2024-04-24T09:08:07.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.813+0000 7fbfe5600640 10 client.21402 handle_command_reply: tid=0 2024-04-24T09:08:07.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.813+0000 7fc0013b3740 1 client.21402 shutdown 2024-04-24T09:08:07.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.813+0000 7fc0013b3740 1 -- 172.21.15.83:0/2927831994 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5583d8dae890 msgr2=0x5583d8dc1da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:07.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.813+0000 7fc0013b3740 1 --2- 172.21.15.83:0/2927831994 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5583d8dae890 0x5583d8dc1da0 secure :-1 s=READY pgs=2298 cs=0 l=1 rev1=1 crypto rx=0x7fbfe8009230 tx=0x7fbfe8004440 comp rx=0 tx=0).stop 2024-04-24T09:08:07.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.813+0000 7fc0013b3740 1 -- 172.21.15.83:0/2927831994 shutdown_connections 2024-04-24T09:08:07.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.813+0000 7fc0013b3740 1 --2- 172.21.15.83:0/2927831994 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5583d8dd2b70 0x5583d8dd5030 crc :-1 s=READY pgs=417 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.813+0000 7fc0013b3740 1 --2- 172.21.15.83:0/2927831994 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5583d8dc2340 0x5583d8dbf7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.813+0000 7fc0013b3740 1 --2- 172.21.15.83:0/2927831994 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5583d8daf240 0x5583d8dbf090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.813+0000 7fc0013b3740 1 --2- 172.21.15.83:0/2927831994 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5583d8dae890 0x5583d8dc1da0 secure :-1 s=CLOSED pgs=2298 cs=0 l=1 rev1=1 crypto rx=0x7fbfe8009230 tx=0x7fbfe8004440 comp rx=0 tx=0).stop 2024-04-24T09:08:07.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.813+0000 7fc0013b3740 1 -- 172.21.15.83:0/2927831994 >> 172.21.15.83:0/2927831994 conn(0x5583d8daae60 msgr2=0x5583d8db0cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:07.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.814+0000 7fbfe5600640 0 client.21402 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:08:07.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.814+0000 7fc0013b3740 1 -- 172.21.15.83:0/2927831994 shutdown_connections 2024-04-24T09:08:07.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.814+0000 7fc0013b3740 1 -- 172.21.15.83:0/2927831994 wait complete. 2024-04-24T09:08:07.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.814+0000 7fc0013b3740 20 client.21402 trim_cache size 0 max 16384 2024-04-24T09:08:07.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.834+0000 7fc0013b3740 1 Processor -- start 2024-04-24T09:08:07.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.834+0000 7fc0013b3740 1 -- start start 2024-04-24T09:08:07.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.835+0000 7fc0013b3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5583d8daf240 0x5583d8dce130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:07.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.835+0000 7fc0013b3740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5583d8dc2340 0x5583d8dc18d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:07.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.835+0000 7fc0013b3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5583d8dc1e10 0x5583d8dda940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:07.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.835+0000 7fc0013b3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd4010220 con 0x5583d8dc1e10 2024-04-24T09:08:07.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.835+0000 7fc0013b3740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd4011e80 con 0x5583d8dc2340 2024-04-24T09:08:07.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.835+0000 7fc0013b3740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd40108f0 con 0x5583d8daf240 2024-04-24T09:08:07.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.835+0000 7fbfefe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5583d8dc2340 0x5583d8dc18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:07.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.835+0000 7fbfefe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5583d8dc2340 0x5583d8dc18d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:58620/0 (socket says 172.21.15.83:58620) 2024-04-24T09:08:07.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.835+0000 7fbfefe00640 1 -- 172.21.15.83:0/4064071195 learned_addr learned my addr 172.21.15.83:0/4064071195 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:08:07.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.835+0000 7fbff7400640 1 --2- 172.21.15.83:0/4064071195 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5583d8daf240 0x5583d8dce130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:07.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.835+0000 7fbff7e00640 1 --2- 172.21.15.83:0/4064071195 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5583d8dc1e10 0x5583d8dda940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:07.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.836+0000 7fbfefe00640 1 -- 172.21.15.83:0/4064071195 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5583d8daf240 msgr2=0x5583d8dce130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:07.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.836+0000 7fbfefe00640 1 --2- 172.21.15.83:0/4064071195 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5583d8daf240 0x5583d8dce130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.836+0000 7fbfefe00640 1 -- 172.21.15.83:0/4064071195 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5583d8dc1e10 msgr2=0x5583d8dda940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:07.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.836+0000 7fbfefe00640 1 --2- 172.21.15.83:0/4064071195 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5583d8dc1e10 0x5583d8dda940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.836+0000 7fbfefe00640 1 -- 172.21.15.83:0/4064071195 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583d8dbfb20 con 0x5583d8dc2340 2024-04-24T09:08:07.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.836+0000 7fbfefe00640 1 --2- 172.21.15.83:0/4064071195 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5583d8dc2340 0x5583d8dc18d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fbfe0012f90 tx=0x7fbfe0029000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fdf018c244168656 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:07.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.837+0000 7fbfe7e00640 1 -- 172.21.15.83:0/4064071195 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfe002f040 con 0x5583d8dc2340 2024-04-24T09:08:07.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.837+0000 7fbfe7e00640 1 -- 172.21.15.83:0/4064071195 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbfe001de50 con 0x5583d8dc2340 2024-04-24T09:08:07.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.837+0000 7fbfe7e00640 1 -- 172.21.15.83:0/4064071195 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfe0023040 con 0x5583d8dc2340 2024-04-24T09:08:07.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.837+0000 7fc0013b3740 1 -- 172.21.15.83:0/4064071195 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5583d8dc2340 msgr2=0x5583d8dc18d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:07.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.837+0000 7fc0013b3740 1 --2- 172.21.15.83:0/4064071195 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5583d8dc2340 0x5583d8dc18d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fbfe0012f90 tx=0x7fbfe0029000 comp rx=0 tx=0).stop 2024-04-24T09:08:07.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.838+0000 7fc0013b3740 1 -- 172.21.15.83:0/4064071195 shutdown_connections 2024-04-24T09:08:07.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.838+0000 7fc0013b3740 1 --2- 172.21.15.83:0/4064071195 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5583d8dc1e10 0x5583d8dda940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.838+0000 7fc0013b3740 1 --2- 172.21.15.83:0/4064071195 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5583d8dc2340 0x5583d8dc18d0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.838+0000 7fc0013b3740 1 --2- 172.21.15.83:0/4064071195 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5583d8daf240 0x5583d8dce130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.838+0000 7fc0013b3740 1 -- 172.21.15.83:0/4064071195 >> 172.21.15.83:0/4064071195 conn(0x5583d8daae60 msgr2=0x5583d8dab240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:07.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.838+0000 7fc0013b3740 1 -- 172.21.15.83:0/4064071195 shutdown_connections 2024-04-24T09:08:07.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.838+0000 7fc0013b3740 1 -- 172.21.15.83:0/4064071195 wait complete. 2024-04-24T09:08:07.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.838+0000 7fc0013b3740 1 -- start start 2024-04-24T09:08:07.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.839+0000 7fc0013b3740 1 Processor -- start 2024-04-24T09:08:07.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.839+0000 7fc0013b3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5583d8daf240 0x5583d8dcab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:07.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.839+0000 7fc0013b3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5583d8dc1e10 0x5583d8dc2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:07.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.839+0000 7fc0013b3740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5583d8dc2340 0x5583d8dc3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:07.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.839+0000 7fc0013b3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd40116f0 con 0x5583d8dc1e10 2024-04-24T09:08:07.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.839+0000 7fc0013b3740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd4010ff0 con 0x5583d8dc2340 2024-04-24T09:08:07.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.839+0000 7fc0013b3740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd400e220 con 0x5583d8daf240 2024-04-24T09:08:07.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.839+0000 7fc0013b3740 10 client.0 _ll_register_callbacks cb 0x7fbfe8005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:08:07.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.840+0000 7fbfefe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5583d8dc1e10 0x5583d8dc2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:07.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.840+0000 7fbff7e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5583d8dc2340 0x5583d8dc3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:07.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.840+0000 7fbfefe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5583d8dc1e10 0x5583d8dc2e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55500/0 (socket says 172.21.15.83:55500) 2024-04-24T09:08:07.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.840+0000 7fbfefe00640 1 -- 172.21.15.83:0/3453629504 learned_addr learned my addr 172.21.15.83:0/3453629504 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:08:07.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.840+0000 7fbff7400640 1 --2- 172.21.15.83:0/3453629504 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5583d8daf240 0x5583d8dcab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:07.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.840+0000 7fbfefe00640 1 -- 172.21.15.83:0/3453629504 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5583d8daf240 msgr2=0x5583d8dcab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:07.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.840+0000 7fbfefe00640 1 --2- 172.21.15.83:0/3453629504 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5583d8daf240 0x5583d8dcab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.840+0000 7fbfefe00640 1 -- 172.21.15.83:0/3453629504 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5583d8dc2340 msgr2=0x5583d8dc3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:07.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.840+0000 7fbfefe00640 1 --2- 172.21.15.83:0/3453629504 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5583d8dc2340 0x5583d8dc3430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.840+0000 7fbfefe00640 1 -- 172.21.15.83:0/3453629504 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583d8af07d0 con 0x5583d8dc1e10 2024-04-24T09:08:07.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.841+0000 7fbfefe00640 1 --2- 172.21.15.83:0/3453629504 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5583d8dc1e10 0x5583d8dc2e90 secure :-1 s=READY pgs=2299 cs=0 l=1 rev1=1 crypto rx=0x7fbfe001d4f0 tx=0x7fbfe000b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec63526fa7e8a757 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:07.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.841+0000 7fbfe5600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:08:07.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.841+0000 7fbfe5600640 1 -- 172.21.15.83:0/3453629504 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfe0023070 con 0x5583d8dc1e10 2024-04-24T09:08:07.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.841+0000 7fbfe5600640 1 -- 172.21.15.83:0/3453629504 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9671+0+0 (secure 0 0 0) 0x7fbfe0010a00 con 0x5583d8dc1e10 2024-04-24T09:08:07.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.841+0000 7fc0013b3740 1 -- 172.21.15.83:0/3453629504 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbfe001de50 con 0x5583d8dc1e10 2024-04-24T09:08:07.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.842+0000 7fbfe5600640 1 -- 172.21.15.83:0/3453629504 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7fbfe004b050 con 0x5583d8dc1e10 2024-04-24T09:08:07.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.842+0000 7fc0013b3740 10 client.21408 fetch_fsmap learned FSMap version 53 2024-04-24T09:08:07.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.842+0000 7fc0013b3740 1 -- 172.21.15.83:0/3453629504 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x5583d8e391b0 con 0x5583d8dc1e10 2024-04-24T09:08:07.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.842+0000 7fbfe5600640 1 -- 172.21.15.83:0/3453629504 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbfe001de50 con 0x5583d8dc1e10 2024-04-24T09:08:07.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.843+0000 7fbfe5600640 1 -- 172.21.15.83:0/3453629504 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 53) ==== 8398+0+0 (secure 0 0 0) 0x7fbfe0026030 con 0x5583d8dc1e10 2024-04-24T09:08:07.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.843+0000 7fc0013b3740 10 client.21408 fetch_fsmap finished waiting for FSMap version 53 2024-04-24T09:08:07.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.843+0000 7fc0013b3740 10 client.21408 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:08:07.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.843+0000 7fc0013b3740 20 client.21408 populate_metadata read hostname 'smithi083' 2024-04-24T09:08:07.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.843+0000 7fc0013b3740 1 --2- 172.21.15.83:0/3453629504 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5583d8e39630 0x5583d8e3ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:07.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.843+0000 7fc0013b3740 4 client.21408 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:08:07.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.843+0000 7fc0013b3740 1 -- 172.21.15.83:0/3453629504 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5583d8d1ba60 con 0x5583d8e39630 2024-04-24T09:08:07.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.844+0000 7fbff7400640 1 --2- 172.21.15.83:0/3453629504 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5583d8e39630 0x5583d8e3ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:07.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.845+0000 7fbff7400640 1 --2- 172.21.15.83:0/3453629504 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5583d8e39630 0x5583d8e3ba50 crc :-1 s=READY pgs=418 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1392d92acd941d29 server_cookie=824895a5cd914385 in_seq=0 out_seq=0 2024-04-24T09:08:07.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.845+0000 7fbfe5600640 10 client.21408 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:08:07.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.845+0000 7fbfe5600640 1 -- 172.21.15.83:0/3453629504 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5583d8d1ba60 con 0x5583d8e39630 2024-04-24T09:08:07.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.845+0000 7fbfe5600640 10 client.21408 handle_command_reply: tid=0 2024-04-24T09:08:07.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.845+0000 7fc0013b3740 1 client.21408 shutdown 2024-04-24T09:08:07.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.846+0000 7fc0013b3740 1 -- 172.21.15.83:0/3453629504 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5583d8dc1e10 msgr2=0x5583d8dc2e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:07.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.846+0000 7fc0013b3740 1 --2- 172.21.15.83:0/3453629504 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5583d8dc1e10 0x5583d8dc2e90 secure :-1 s=READY pgs=2299 cs=0 l=1 rev1=1 crypto rx=0x7fbfe001d4f0 tx=0x7fbfe000b810 comp rx=0 tx=0).stop 2024-04-24T09:08:07.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.846+0000 7fc0013b3740 1 -- 172.21.15.83:0/3453629504 shutdown_connections 2024-04-24T09:08:07.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.846+0000 7fc0013b3740 1 --2- 172.21.15.83:0/3453629504 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5583d8e39630 0x5583d8e3ba50 crc :-1 s=READY pgs=418 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.846+0000 7fc0013b3740 1 --2- 172.21.15.83:0/3453629504 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5583d8dc2340 0x5583d8dc3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.846+0000 7fc0013b3740 1 --2- 172.21.15.83:0/3453629504 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5583d8dc1e10 0x5583d8dc2e90 unknown :-1 s=CLOSED pgs=2299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.846+0000 7fc0013b3740 1 --2- 172.21.15.83:0/3453629504 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5583d8daf240 0x5583d8dcab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.846+0000 7fc0013b3740 1 -- 172.21.15.83:0/3453629504 >> 172.21.15.83:0/3453629504 conn(0x5583d8daae60 msgr2=0x5583d8dbd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:07.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.846+0000 7fbfe5600640 0 client.21408 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:08:07.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.847+0000 7fc0013b3740 1 -- 172.21.15.83:0/3453629504 shutdown_connections 2024-04-24T09:08:07.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.847+0000 7fc0013b3740 1 -- 172.21.15.83:0/3453629504 wait complete. 2024-04-24T09:08:07.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.847+0000 7fc0013b3740 20 client.21408 trim_cache size 0 max 16384 2024-04-24T09:08:07.848 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:08:07.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.849+0000 7fbffce00640 1 -- 172.21.15.83:0/1945926505 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fbfdc0a3de0 msgr2=0x7fbfdc0a62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:07.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.849+0000 7fbffce00640 1 --2- 172.21.15.83:0/1945926505 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fbfdc0a3de0 0x7fbfdc0a62a0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fbfe001d920 tx=0x7fbfe001d890 comp rx=0 tx=0).stop 2024-04-24T09:08:07.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.849+0000 7fbffce00640 1 -- 172.21.15.83:0/1945926505 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbff819a720 msgr2=0x7fbff819aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:07.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.849+0000 7fbffce00640 1 --2- 172.21.15.83:0/1945926505 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbff819a720 0x7fbff819aba0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fbff000e940 tx=0x7fbff000ee00 comp rx=0 tx=0).stop 2024-04-24T09:08:07.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.849+0000 7fbffce00640 1 -- 172.21.15.83:0/1945926505 shutdown_connections 2024-04-24T09:08:07.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.849+0000 7fbffce00640 1 --2- 172.21.15.83:0/1945926505 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fbfdc0a3de0 0x7fbfdc0a62a0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.849+0000 7fbffce00640 1 --2- 172.21.15.83:0/1945926505 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbff819a720 0x7fbff819aba0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.849+0000 7fbffce00640 1 --2- 172.21.15.83:0/1945926505 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbff81093f0 0x7fbff81a0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.849+0000 7fbffce00640 1 --2- 172.21.15.83:0/1945926505 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbff8104320 0x7fbff81a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:07.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.849+0000 7fbffce00640 1 -- 172.21.15.83:0/1945926505 >> 172.21.15.83:0/1945926505 conn(0x7fbff80ffb40 msgr2=0x7fbff8100e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:07.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.849+0000 7fbffce00640 1 -- 172.21.15.83:0/1945926505 shutdown_connections 2024-04-24T09:08:07.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:07.850+0000 7fbffce00640 1 -- 172.21.15.83:0/1945926505 wait complete. 2024-04-24T09:08:08.164 INFO:tasks.workunit.client.0.smithi083.stdout: 10 20556 7.11 MB/sec execute 39 sec latency 79.449 ms 2024-04-24T09:08:08.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:08:08 smithi083 ceph-mon[27907]: pgmap v519: 129 pgs: 129 active+clean; 452 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.8 MiB/s wr, 265 op/s 2024-04-24T09:08:08.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:08:08 smithi138 ceph-mon[31441]: pgmap v519: 129 pgs: 129 active+clean; 452 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.8 MiB/s wr, 265 op/s 2024-04-24T09:08:08.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:08:08 smithi189 ceph-mon[31791]: pgmap v519: 129 pgs: 129 active+clean; 452 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.8 MiB/s wr, 265 op/s 2024-04-24T09:08:08.884 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T09:08:08.885 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T09:08:08.885 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T09:08:09.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.159+0000 7fa787e00640 1 -- 172.21.15.83:0/3142159200 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa788104cd0 msgr2=0x7fa788108d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:09.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.159+0000 7fa787e00640 1 --2- 172.21.15.83:0/3142159200 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa788104cd0 0x7fa788108d10 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa774009900 tx=0x7fa77402a410 comp rx=0 tx=0).stop 2024-04-24T09:08:09.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.160+0000 7fa787e00640 1 -- 172.21.15.83:0/3142159200 shutdown_connections 2024-04-24T09:08:09.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.160+0000 7fa787e00640 1 --2- 172.21.15.83:0/3142159200 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa7881093f0 0x7fa788110fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:09.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.160+0000 7fa787e00640 1 --2- 172.21.15.83:0/3142159200 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa788104cd0 0x7fa788108d10 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:09.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.160+0000 7fa787e00640 1 --2- 172.21.15.83:0/3142159200 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa788104320 0x7fa788104700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:09.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.160+0000 7fa787e00640 1 -- 172.21.15.83:0/3142159200 >> 172.21.15.83:0/3142159200 conn(0x7fa7880ffb40 msgr2=0x7fa788101f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:09.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.160+0000 7fa787e00640 1 -- 172.21.15.83:0/3142159200 shutdown_connections 2024-04-24T09:08:09.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.160+0000 7fa787e00640 1 -- 172.21.15.83:0/3142159200 wait complete. 2024-04-24T09:08:09.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.161+0000 7fa787e00640 1 Processor -- start 2024-04-24T09:08:09.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.161+0000 7fa787e00640 1 -- start start 2024-04-24T09:08:09.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.162+0000 7fa787e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa788104320 0x7fa7881a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:09.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.162+0000 7fa787e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7881093f0 0x7fa7881a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:09.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.162+0000 7fa787e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa78819a810 0x7fa78819acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:09.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.162+0000 7fa787e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa788114330 con 0x7fa788104320 2024-04-24T09:08:09.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.162+0000 7fa787e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa7881141b0 con 0x7fa7881093f0 2024-04-24T09:08:09.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.162+0000 7fa787e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa7881144b0 con 0x7fa78819a810 2024-04-24T09:08:09.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.162+0000 7fa786a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa788104320 0x7fa7881a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:09.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.162+0000 7fa786a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa788104320 0x7fa7881a0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41976/0 (socket says 172.21.15.83:41976) 2024-04-24T09:08:09.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.162+0000 7fa786a00640 1 -- 172.21.15.83:0/3224925908 learned_addr learned my addr 172.21.15.83:0/3224925908 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:08:09.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.162+0000 7fa786000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7881093f0 0x7fa7881a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:09.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.163+0000 7fa787400640 1 --2- 172.21.15.83:0/3224925908 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa78819a810 0x7fa78819acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:09.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.163+0000 7fa786000640 1 -- 172.21.15.83:0/3224925908 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa78819a810 msgr2=0x7fa78819acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:09.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.163+0000 7fa786000640 1 --2- 172.21.15.83:0/3224925908 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa78819a810 0x7fa78819acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:09.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.163+0000 7fa786000640 1 -- 172.21.15.83:0/3224925908 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa788104320 msgr2=0x7fa7881a0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:09.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.163+0000 7fa786000640 1 --2- 172.21.15.83:0/3224925908 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa788104320 0x7fa7881a0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:09.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.163+0000 7fa786000640 1 -- 172.21.15.83:0/3224925908 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa78819b3a0 con 0x7fa7881093f0 2024-04-24T09:08:09.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.163+0000 7fa786000640 1 --2- 172.21.15.83:0/3224925908 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7881093f0 0x7fa7881a0b90 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa77402a930 tx=0x7fa774004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:09.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.164+0000 7fa77f400640 1 -- 172.21.15.83:0/3224925908 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa77403f070 con 0x7fa7881093f0 2024-04-24T09:08:09.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.164+0000 7fa787e00640 1 -- 172.21.15.83:0/3224925908 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa78819b630 con 0x7fa7881093f0 2024-04-24T09:08:09.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.164+0000 7fa77f400640 1 -- 172.21.15.83:0/3224925908 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa774003ca0 con 0x7fa7881093f0 2024-04-24T09:08:09.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.164+0000 7fa77f400640 1 -- 172.21.15.83:0/3224925908 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa774035070 con 0x7fa7881093f0 2024-04-24T09:08:09.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.164+0000 7fa787e00640 1 -- 172.21.15.83:0/3224925908 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7881a77c0 con 0x7fa7881093f0 2024-04-24T09:08:09.165 INFO:tasks.workunit.client.0.smithi083.stdout: 10 20688 7.13 MB/sec execute 40 sec latency 79.818 ms 2024-04-24T09:08:09.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.168+0000 7fa79041e740 1 Processor -- start 2024-04-24T09:08:09.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.168+0000 7fa79041e740 1 -- start start 2024-04-24T09:08:09.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.168+0000 7fa77f400640 1 -- 172.21.15.83:0/3224925908 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa77403b460 con 0x7fa7881093f0 2024-04-24T09:08:09.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.168+0000 7fa79041e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5623859e9270 0x5623859e9650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:09.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.169+0000 7fa79041e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5623859e9b90 0x5623859f3d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:09.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.169+0000 7fa79041e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5623859f4260 0x5623859f6650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:09.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.169+0000 7fa79041e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56238594ee70 con 0x5623859e9b90 2024-04-24T09:08:09.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.169+0000 7fa79041e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5623858f5380 con 0x5623859f4260 2024-04-24T09:08:09.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.169+0000 7fa79041e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56238594ae30 con 0x5623859e9270 2024-04-24T09:08:09.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.169+0000 7fa786a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5623859e9b90 0x5623859f3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:09.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.169+0000 7fa786a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5623859e9b90 0x5623859f3d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41988/0 (socket says 172.21.15.83:41988) 2024-04-24T09:08:09.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.169+0000 7fa786a00640 1 -- 172.21.15.83:0/3721881832 learned_addr learned my addr 172.21.15.83:0/3721881832 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:08:09.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.169+0000 7fa787400640 1 --2- 172.21.15.83:0/3721881832 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5623859e9270 0x5623859e9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:09.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.169+0000 7fa786000640 1 --2- 172.21.15.83:0/3721881832 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5623859f4260 0x5623859f6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:09.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.170+0000 7fa786000640 1 -- 172.21.15.83:0/3721881832 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5623859e9270 msgr2=0x5623859e9650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:09.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.170+0000 7fa786000640 1 --2- 172.21.15.83:0/3721881832 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5623859e9270 0x5623859e9650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:09.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.170+0000 7fa786000640 1 -- 172.21.15.83:0/3721881832 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5623859e9b90 msgr2=0x5623859f3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:09.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.170+0000 7fa786000640 1 --2- 172.21.15.83:0/3721881832 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5623859e9b90 0x5623859f3d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:09.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.170+0000 7fa786000640 1 -- 172.21.15.83:0/3721881832 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623857297e0 con 0x5623859f4260 2024-04-24T09:08:09.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.170+0000 7fa786a00640 1 --2- 172.21.15.83:0/3721881832 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5623859e9b90 0x5623859f3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:08:09.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.170+0000 7fa786000640 1 --2- 172.21.15.83:0/3721881832 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5623859f4260 0x5623859f6650 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa7740ee490 tx=0x7fa7740f9540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d061d10cca437aff server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:09.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.170+0000 7fa77f400640 1 --2- 172.21.15.83:0/3224925908 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa76c0a3da0 0x7fa76c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:09.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.171+0000 7fa773400640 1 -- 172.21.15.83:0/3721881832 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7740b8f80 con 0x5623859f4260 2024-04-24T09:08:09.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.171+0000 7fa773400640 1 -- 172.21.15.83:0/3721881832 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa7740fe030 con 0x5623859f4260 2024-04-24T09:08:09.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.171+0000 7fa77f400640 1 -- 172.21.15.83:0/3224925908 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9671+0+0 (secure 0 0 0) 0x7fa7740f1220 con 0x7fa7881093f0 2024-04-24T09:08:09.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.171+0000 7fa787400640 1 --2- 172.21.15.83:0/3224925908 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa76c0a3da0 0x7fa76c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:09.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.172+0000 7fa79041e740 1 -- 172.21.15.83:0/3721881832 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5623859f4260 msgr2=0x5623859f6650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:09.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.172+0000 7fa79041e740 1 --2- 172.21.15.83:0/3721881832 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5623859f4260 0x5623859f6650 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa7740ee490 tx=0x7fa7740f9540 comp rx=0 tx=0).stop 2024-04-24T09:08:09.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.172+0000 7fa79041e740 1 -- 172.21.15.83:0/3721881832 shutdown_connections 2024-04-24T09:08:09.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.172+0000 7fa79041e740 1 --2- 172.21.15.83:0/3721881832 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5623859f4260 0x5623859f6650 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:09.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.172+0000 7fa79041e740 1 --2- 172.21.15.83:0/3721881832 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5623859e9b90 0x5623859f3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:09.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.172+0000 7fa79041e740 1 --2- 172.21.15.83:0/3721881832 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5623859e9270 0x5623859e9650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:09.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.172+0000 7fa79041e740 1 -- 172.21.15.83:0/3721881832 >> 172.21.15.83:0/3721881832 conn(0x5623859e5590 msgr2=0x5623859e5970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:09.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.172+0000 7fa787400640 1 --2- 172.21.15.83:0/3224925908 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa76c0a3da0 0x7fa76c0a6260 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fa780005fd0 tx=0x7fa780009500 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:09.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.173+0000 7fa79041e740 1 -- 172.21.15.83:0/3721881832 shutdown_connections 2024-04-24T09:08:09.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.173+0000 7fa79041e740 1 -- 172.21.15.83:0/3721881832 wait complete. 2024-04-24T09:08:09.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.174+0000 7fa79041e740 1 -- start start 2024-04-24T09:08:09.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.174+0000 7fa79041e740 1 Processor -- start 2024-04-24T09:08:09.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.175+0000 7fa79041e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5623859e9270 0x5623859f3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:09.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.175+0000 7fa79041e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5623859e9b90 0x5623859f0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:09.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.175+0000 7fa79041e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5623859f4260 0x5623859f16d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:09.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.175+0000 7fa79041e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5623859f6e10 con 0x5623859f4260 2024-04-24T09:08:09.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.175+0000 7fa79041e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562385908660 con 0x5623859e9270 2024-04-24T09:08:09.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.175+0000 7fa79041e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5623859f6f90 con 0x5623859e9b90 2024-04-24T09:08:09.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.175+0000 7fa79041e740 10 client.0 _ll_register_callbacks cb 0x5623858f6aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:08:09.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.176+0000 7fa787400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5623859f4260 0x5623859f16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:09.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.176+0000 7fa786a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5623859e9270 0x5623859f3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:09.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.176+0000 7fa786a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5623859e9270 0x5623859f3cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:33204/0 (socket says 172.21.15.83:33204) 2024-04-24T09:08:09.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.176+0000 7fa786a00640 1 -- 172.21.15.83:0/1607914867 learned_addr learned my addr 172.21.15.83:0/1607914867 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:08:09.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.176+0000 7fa786000640 1 --2- 172.21.15.83:0/1607914867 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5623859e9b90 0x5623859f0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:09.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.177+0000 7fa786a00640 1 -- 172.21.15.83:0/1607914867 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5623859e9b90 msgr2=0x5623859f0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:09.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.177+0000 7fa786a00640 1 --2- 172.21.15.83:0/1607914867 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5623859e9b90 0x5623859f0fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:09.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.177+0000 7fa786a00640 1 -- 172.21.15.83:0/1607914867 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5623859f4260 msgr2=0x5623859f16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:09.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.177+0000 7fa786a00640 1 --2- 172.21.15.83:0/1607914867 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5623859f4260 0x5623859f16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:09.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.177+0000 7fa786a00640 1 -- 172.21.15.83:0/1607914867 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562385789f70 con 0x5623859e9270 2024-04-24T09:08:09.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.177+0000 7fa786a00640 1 --2- 172.21.15.83:0/1607914867 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5623859e9270 0x5623859f3cc0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa7780116f0 tx=0x7fa778011bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=badf0cd7280ec9ab server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:09.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.177+0000 7fa767e00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:08:09.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.178+0000 7fa767e00640 1 -- 172.21.15.83:0/1607914867 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa77801f060 con 0x5623859e9270 2024-04-24T09:08:09.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.178+0000 7fa767e00640 1 -- 172.21.15.83:0/1607914867 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9671+0+0 (secure 0 0 0) 0x7fa778010ba0 con 0x5623859e9270 2024-04-24T09:08:09.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.178+0000 7fa79041e740 1 -- 172.21.15.83:0/1607914867 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7740fe030 con 0x5623859e9270 2024-04-24T09:08:09.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.178+0000 7fa767e00640 1 -- 172.21.15.83:0/1607914867 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fa778026050 con 0x5623859e9270 2024-04-24T09:08:09.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.178+0000 7fa79041e740 10 client.38561 fetch_fsmap learned FSMap version 54 2024-04-24T09:08:09.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.178+0000 7fa79041e740 1 -- 172.21.15.83:0/1607914867 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x562385621170 con 0x5623859e9270 2024-04-24T09:08:09.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.179+0000 7fa767e00640 1 -- 172.21.15.83:0/1607914867 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa7740fe030 con 0x5623859e9270 2024-04-24T09:08:09.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.179+0000 7fa767e00640 1 -- 172.21.15.83:0/1607914867 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 54) ==== 8394+0+0 (secure 0 0 0) 0x7fa778011ce0 con 0x5623859e9270 2024-04-24T09:08:09.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.179+0000 7fa79041e740 10 client.38561 fetch_fsmap finished waiting for FSMap version 54 2024-04-24T09:08:09.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.179+0000 7fa79041e740 10 client.38561 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:08:09.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.180+0000 7fa79041e740 20 client.38561 populate_metadata read hostname 'smithi083' 2024-04-24T09:08:09.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.180+0000 7fa79041e740 1 --2- 172.21.15.83:0/1607914867 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x562385a0ca80 0x562385a0eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:09.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.180+0000 7fa79041e740 4 client.38561 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:08:09.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.180+0000 7fa79041e740 1 -- 172.21.15.83:0/1607914867 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56238595c890 con 0x562385a0ca80 2024-04-24T09:08:09.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.180+0000 7fa786000640 1 --2- 172.21.15.83:0/1607914867 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x562385a0ca80 0x562385a0eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:09.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.181+0000 7fa786000640 1 --2- 172.21.15.83:0/1607914867 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x562385a0ca80 0x562385a0eef0 crc :-1 s=READY pgs=419 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ff7a994d960a34a server_cookie=7254cc3171021854 in_seq=0 out_seq=0 2024-04-24T09:08:09.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.181+0000 7fa767e00640 10 client.38561 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:08:09.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.182+0000 7fa767e00640 1 -- 172.21.15.83:0/1607914867 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56238595c890 con 0x562385a0ca80 2024-04-24T09:08:09.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.182+0000 7fa767e00640 10 client.38561 handle_command_reply: tid=0 2024-04-24T09:08:09.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.183+0000 7fa79041e740 1 client.38561 shutdown 2024-04-24T09:08:09.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.183+0000 7fa79041e740 1 -- 172.21.15.83:0/1607914867 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5623859e9270 msgr2=0x5623859f3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:09.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.183+0000 7fa79041e740 1 --2- 172.21.15.83:0/1607914867 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5623859e9270 0x5623859f3cc0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa7780116f0 tx=0x7fa778011bb0 comp rx=0 tx=0).stop 2024-04-24T09:08:09.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.183+0000 7fa79041e740 1 -- 172.21.15.83:0/1607914867 shutdown_connections 2024-04-24T09:08:09.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.183+0000 7fa79041e740 1 --2- 172.21.15.83:0/1607914867 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x562385a0ca80 0x562385a0eef0 crc :-1 s=READY pgs=419 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:09.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.183+0000 7fa79041e740 1 --2- 172.21.15.83:0/1607914867 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5623859f4260 0x5623859f16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:09.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.183+0000 7fa79041e740 1 --2- 172.21.15.83:0/1607914867 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5623859e9b90 0x5623859f0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:09.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.183+0000 7fa79041e740 1 --2- 172.21.15.83:0/1607914867 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5623859e9270 0x5623859f3cc0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:09.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.183+0000 7fa79041e740 1 -- 172.21.15.83:0/1607914867 >> 172.21.15.83:0/1607914867 conn(0x5623859e5590 msgr2=0x5623859eb390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:09.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.183+0000 7fa767e00640 0 client.38561 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:08:09.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.183+0000 7fa79041e740 1 -- 172.21.15.83:0/1607914867 shutdown_connections 2024-04-24T09:08:09.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.183+0000 7fa79041e740 1 -- 172.21.15.83:0/1607914867 wait complete. 2024-04-24T09:08:09.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.183+0000 7fa79041e740 20 client.38561 trim_cache size 0 max 16384 2024-04-24T09:08:09.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.217+0000 7fa79041e740 1 Processor -- start 2024-04-24T09:08:09.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.217+0000 7fa79041e740 1 -- start start 2024-04-24T09:08:09.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.217+0000 7fa79041e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5623859e9270 0x5623859f37f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:09.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.217+0000 7fa79041e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5623859e9b90 0x562385a147c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:09.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.217+0000 7fa79041e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5623859f4260 0x562385a14d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:09.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.217+0000 7fa79041e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa7881ad3f0 con 0x5623859e9b90 2024-04-24T09:08:09.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.218+0000 7fa79041e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa7881af020 con 0x5623859e9270 2024-04-24T09:08:09.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.218+0000 7fa79041e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa7881adac0 con 0x5623859f4260 2024-04-24T09:08:09.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.218+0000 7fa786000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5623859e9b90 0x562385a147c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:09.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.218+0000 7fa786000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5623859e9b90 0x562385a147c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41998/0 (socket says 172.21.15.83:41998) 2024-04-24T09:08:09.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.218+0000 7fa786000640 1 -- 172.21.15.83:0/1354006388 learned_addr learned my addr 172.21.15.83:0/1354006388 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:08:09.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.218+0000 7fa786a00640 1 --2- 172.21.15.83:0/1354006388 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5623859e9270 0x5623859f37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:09.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.218+0000 7fa787400640 1 --2- 172.21.15.83:0/1354006388 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5623859f4260 0x562385a14d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:09.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.218+0000 7fa786000640 1 -- 172.21.15.83:0/1354006388 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5623859f4260 msgr2=0x562385a14d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:09.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.218+0000 7fa786000640 1 --2- 172.21.15.83:0/1354006388 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5623859f4260 0x562385a14d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:09.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.218+0000 7fa786000640 1 -- 172.21.15.83:0/1354006388 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5623859e9270 msgr2=0x5623859f37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:09.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.218+0000 7fa786000640 1 --2- 172.21.15.83:0/1354006388 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5623859e9270 0x5623859f37f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:09.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.218+0000 7fa786000640 1 -- 172.21.15.83:0/1354006388 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562385a0f260 con 0x5623859e9b90 2024-04-24T09:08:09.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.219+0000 7fa787400640 1 --2- 172.21.15.83:0/1354006388 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5623859f4260 0x562385a14d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:08:09.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.219+0000 7fa786000640 1 --2- 172.21.15.83:0/1354006388 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5623859e9b90 0x562385a147c0 secure :-1 s=READY pgs=2300 cs=0 l=1 rev1=1 crypto rx=0x7fa7740dd010 tx=0x7fa77402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7013ee4e98f1c49 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:09.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.219+0000 7fa772a00640 1 -- 172.21.15.83:0/1354006388 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa774048040 con 0x5623859e9b90 2024-04-24T09:08:09.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.219+0000 7fa772a00640 1 -- 172.21.15.83:0/1354006388 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa7740fe040 con 0x5623859e9b90 2024-04-24T09:08:09.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.219+0000 7fa772a00640 1 -- 172.21.15.83:0/1354006388 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7740bc460 con 0x5623859e9b90 2024-04-24T09:08:09.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.220+0000 7fa79041e740 1 -- 172.21.15.83:0/1354006388 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5623859e9b90 msgr2=0x562385a147c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:09.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.220+0000 7fa79041e740 1 --2- 172.21.15.83:0/1354006388 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5623859e9b90 0x562385a147c0 secure :-1 s=READY pgs=2300 cs=0 l=1 rev1=1 crypto rx=0x7fa7740dd010 tx=0x7fa77402a410 comp rx=0 tx=0).stop 2024-04-24T09:08:09.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.220+0000 7fa79041e740 1 -- 172.21.15.83:0/1354006388 shutdown_connections 2024-04-24T09:08:09.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.220+0000 7fa79041e740 1 --2- 172.21.15.83:0/1354006388 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5623859f4260 0x562385a14d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:09.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.220+0000 7fa79041e740 1 --2- 172.21.15.83:0/1354006388 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5623859e9b90 0x562385a147c0 unknown :-1 s=CLOSED pgs=2300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:09.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.220+0000 7fa79041e740 1 --2- 172.21.15.83:0/1354006388 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5623859e9270 0x5623859f37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:09.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.220+0000 7fa79041e740 1 -- 172.21.15.83:0/1354006388 >> 172.21.15.83:0/1354006388 conn(0x5623859e5590 msgr2=0x5623859e46f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:09.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.221+0000 7fa79041e740 1 -- 172.21.15.83:0/1354006388 shutdown_connections 2024-04-24T09:08:09.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.221+0000 7fa79041e740 1 -- 172.21.15.83:0/1354006388 wait complete. 2024-04-24T09:08:09.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.221+0000 7fa79041e740 1 -- start start 2024-04-24T09:08:09.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.221+0000 7fa79041e740 1 Processor -- start 2024-04-24T09:08:09.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.222+0000 7fa79041e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5623859e9270 0x5623859ee540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:09.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.222+0000 7fa79041e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5623859e9b90 0x5623859eea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:09.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.222+0000 7fa79041e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5623859f4260 0x5623859f51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:09.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.222+0000 7fa79041e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa7881ae890 con 0x5623859e9b90 2024-04-24T09:08:09.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.222+0000 7fa79041e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa7881ae190 con 0x5623859e9270 2024-04-24T09:08:09.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.222+0000 7fa79041e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa7881ab3f0 con 0x5623859f4260 2024-04-24T09:08:09.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.222+0000 7fa79041e740 10 client.0 _ll_register_callbacks cb 0x7fa77800b9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:08:09.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.222+0000 7fa786a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5623859e9270 0x5623859ee540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:09.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.222+0000 7fa786000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5623859e9b90 0x5623859eea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:09.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.222+0000 7fa786a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5623859e9270 0x5623859ee540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:33218/0 (socket says 172.21.15.83:33218) 2024-04-24T09:08:09.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.222+0000 7fa786a00640 1 -- 172.21.15.83:0/1127946972 learned_addr learned my addr 172.21.15.83:0/1127946972 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:08:09.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.223+0000 7fa786000640 1 -- 172.21.15.83:0/1127946972 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5623859f4260 msgr2=0x5623859f51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:09.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.223+0000 7fa786000640 1 --2- 172.21.15.83:0/1127946972 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5623859f4260 0x5623859f51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:09.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.223+0000 7fa786000640 1 -- 172.21.15.83:0/1127946972 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5623859e9270 msgr2=0x5623859ee540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:09.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.223+0000 7fa786000640 1 --2- 172.21.15.83:0/1127946972 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5623859e9270 0x5623859ee540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:09.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.223+0000 7fa786000640 1 -- 172.21.15.83:0/1127946972 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562385a051b0 con 0x5623859e9b90 2024-04-24T09:08:09.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.223+0000 7fa786000640 1 --2- 172.21.15.83:0/1127946972 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5623859e9b90 0x5623859eea80 secure :-1 s=READY pgs=2301 cs=0 l=1 rev1=1 crypto rx=0x5623859d7150 tx=0x7fa7740bcd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1aa07cdcd2498bd9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:09.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.223+0000 7fa767e00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:08:09.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.223+0000 7fa767e00640 1 -- 172.21.15.83:0/1127946972 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7740d5070 con 0x5623859e9b90 2024-04-24T09:08:09.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.224+0000 7fa767e00640 1 -- 172.21.15.83:0/1127946972 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9671+0+0 (secure 0 0 0) 0x7fa7740b9350 con 0x5623859e9b90 2024-04-24T09:08:09.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.224+0000 7fa79041e740 1 -- 172.21.15.83:0/1127946972 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7740fe040 con 0x5623859e9b90 2024-04-24T09:08:09.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.224+0000 7fa767e00640 1 -- 172.21.15.83:0/1127946972 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fa774048040 con 0x5623859e9b90 2024-04-24T09:08:09.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.224+0000 7fa79041e740 10 client.21426 fetch_fsmap learned FSMap version 54 2024-04-24T09:08:09.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.224+0000 7fa79041e740 1 -- 172.21.15.83:0/1127946972 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x562385a734d0 con 0x5623859e9b90 2024-04-24T09:08:09.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.224+0000 7fa767e00640 1 -- 172.21.15.83:0/1127946972 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa7740fe040 con 0x5623859e9b90 2024-04-24T09:08:09.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.224+0000 7fa767e00640 1 -- 172.21.15.83:0/1127946972 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 54) ==== 8394+0+0 (secure 0 0 0) 0x7fa77404b060 con 0x5623859e9b90 2024-04-24T09:08:09.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.224+0000 7fa79041e740 10 client.21426 fetch_fsmap finished waiting for FSMap version 54 2024-04-24T09:08:09.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.224+0000 7fa79041e740 10 client.21426 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:08:09.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.224+0000 7fa79041e740 20 client.21426 populate_metadata read hostname 'smithi083' 2024-04-24T09:08:09.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.224+0000 7fa79041e740 1 --2- 172.21.15.83:0/1127946972 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x562385a73910 0x562385a75d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:09.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.225+0000 7fa79041e740 4 client.21426 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T09:08:09.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.225+0000 7fa79041e740 1 -- 172.21.15.83:0/1127946972 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x562385956700 con 0x562385a73910 2024-04-24T09:08:09.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.225+0000 7fa786a00640 1 --2- 172.21.15.83:0/1127946972 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x562385a73910 0x562385a75d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:09.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.226+0000 7fa786a00640 1 --2- 172.21.15.83:0/1127946972 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x562385a73910 0x562385a75d30 crc :-1 s=READY pgs=420 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a195658e284d9ed server_cookie=b2a3b0d5e6d1644a in_seq=0 out_seq=0 2024-04-24T09:08:09.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.226+0000 7fa767e00640 10 client.21426 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:08:09.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.229+0000 7fa767e00640 1 -- 172.21.15.83:0/1127946972 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562385956700 con 0x562385a73910 2024-04-24T09:08:09.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.229+0000 7fa767e00640 10 client.21426 handle_command_reply: tid=0 2024-04-24T09:08:09.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.229+0000 7fa79041e740 1 client.21426 shutdown 2024-04-24T09:08:09.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.229+0000 7fa79041e740 1 -- 172.21.15.83:0/1127946972 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5623859e9b90 msgr2=0x5623859eea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:09.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.229+0000 7fa79041e740 1 --2- 172.21.15.83:0/1127946972 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5623859e9b90 0x5623859eea80 secure :-1 s=READY pgs=2301 cs=0 l=1 rev1=1 crypto rx=0x5623859d7150 tx=0x7fa7740bcd50 comp rx=0 tx=0).stop 2024-04-24T09:08:09.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.229+0000 7fa79041e740 1 -- 172.21.15.83:0/1127946972 shutdown_connections 2024-04-24T09:08:09.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.229+0000 7fa79041e740 1 --2- 172.21.15.83:0/1127946972 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x562385a73910 0x562385a75d30 crc :-1 s=READY pgs=420 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:09.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.229+0000 7fa79041e740 1 --2- 172.21.15.83:0/1127946972 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5623859f4260 0x5623859f51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:09.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.229+0000 7fa79041e740 1 --2- 172.21.15.83:0/1127946972 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5623859e9b90 0x5623859eea80 unknown :-1 s=CLOSED pgs=2301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:09.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.229+0000 7fa79041e740 1 --2- 172.21.15.83:0/1127946972 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5623859e9270 0x5623859ee540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:09.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.229+0000 7fa79041e740 1 -- 172.21.15.83:0/1127946972 >> 172.21.15.83:0/1127946972 conn(0x5623859e5590 msgr2=0x5623859ef770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:09.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.229+0000 7fa767e00640 0 client.21426 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:08:09.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.229+0000 7fa79041e740 1 -- 172.21.15.83:0/1127946972 shutdown_connections 2024-04-24T09:08:09.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.229+0000 7fa79041e740 1 -- 172.21.15.83:0/1127946972 wait complete. 2024-04-24T09:08:09.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.230+0000 7fa79041e740 20 client.21426 trim_cache size 0 max 16384 2024-04-24T09:08:09.231 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T09:08:09.231 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-04-24T09:08:09.231 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "096f8830-ef5c-410e-8526-5ab1dcac4bf4", 2024-04-24T09:08:09.231 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-04-24T09:08:09.231 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T09:08:09.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.231+0000 7fa787e00640 1 -- 172.21.15.83:0/3224925908 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa76c0a3da0 msgr2=0x7fa76c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:09.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.231+0000 7fa787e00640 1 --2- 172.21.15.83:0/3224925908 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa76c0a3da0 0x7fa76c0a6260 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fa780005fd0 tx=0x7fa780009500 comp rx=0 tx=0).stop 2024-04-24T09:08:09.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.231+0000 7fa787e00640 1 -- 172.21.15.83:0/3224925908 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7881093f0 msgr2=0x7fa7881a0b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:09.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.231+0000 7fa787e00640 1 --2- 172.21.15.83:0/3224925908 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7881093f0 0x7fa7881a0b90 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa77402a930 tx=0x7fa774004040 comp rx=0 tx=0).stop 2024-04-24T09:08:09.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.231+0000 7fa787e00640 1 -- 172.21.15.83:0/3224925908 shutdown_connections 2024-04-24T09:08:09.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.231+0000 7fa787e00640 1 --2- 172.21.15.83:0/3224925908 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa76c0a3da0 0x7fa76c0a6260 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:09.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.231+0000 7fa787e00640 1 --2- 172.21.15.83:0/3224925908 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa78819a810 0x7fa78819acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:09.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.231+0000 7fa787e00640 1 --2- 172.21.15.83:0/3224925908 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7881093f0 0x7fa7881a0b90 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:09.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.231+0000 7fa787e00640 1 --2- 172.21.15.83:0/3224925908 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa788104320 0x7fa7881a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:09.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.231+0000 7fa787e00640 1 -- 172.21.15.83:0/3224925908 >> 172.21.15.83:0/3224925908 conn(0x7fa7880ffb40 msgr2=0x7fa788100fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:09.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.231+0000 7fa787e00640 1 -- 172.21.15.83:0/3224925908 shutdown_connections 2024-04-24T09:08:09.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.231+0000 7fa787e00640 1 -- 172.21.15.83:0/3224925908 wait complete. 2024-04-24T09:08:09.251 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T09:08:09.251 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T09:08:09.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.505+0000 7f866fc00640 1 -- 172.21.15.83:0/2313677457 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f866811b7e0 msgr2=0x7f866811dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:09.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.505+0000 7f866fc00640 1 --2- 172.21.15.83:0/2313677457 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f866811b7e0 0x7f866811dbd0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8658009930 tx=0x7f865802a440 comp rx=0 tx=0).stop 2024-04-24T09:08:09.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.505+0000 7f866fc00640 1 -- 172.21.15.83:0/2313677457 shutdown_connections 2024-04-24T09:08:09.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.505+0000 7f866fc00640 1 --2- 172.21.15.83:0/2313677457 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f866811b7e0 0x7f866811dbd0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:09.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.505+0000 7f866fc00640 1 --2- 172.21.15.83:0/2313677457 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f866810e880 0x7f866810ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:09.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.505+0000 7f866fc00640 1 --2- 172.21.15.83:0/2313677457 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8668110850 0x7f866810e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:09.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.505+0000 7f866fc00640 1 -- 172.21.15.83:0/2313677457 >> 172.21.15.83:0/2313677457 conn(0x7f866806d110 msgr2=0x7f866806d520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:09.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.505+0000 7f866fc00640 1 -- 172.21.15.83:0/2313677457 shutdown_connections 2024-04-24T09:08:09.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.505+0000 7f866fc00640 1 -- 172.21.15.83:0/2313677457 wait complete. 2024-04-24T09:08:09.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.507+0000 7f866fc00640 1 Processor -- start 2024-04-24T09:08:09.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.507+0000 7f866fc00640 1 -- start start 2024-04-24T09:08:09.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.507+0000 7f866fc00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f866810e880 0x7f86681a75c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:09.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.507+0000 7f866fc00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8668110850 0x7f86681a7b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:09.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.507+0000 7f866fc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f866811b7e0 0x7f86681abe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:09.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.507+0000 7f866fc00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8668121040 con 0x7f866811b7e0 2024-04-24T09:08:09.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.507+0000 7f866fc00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8668120ec0 con 0x7f8668110850 2024-04-24T09:08:09.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.507+0000 7f866fc00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f86681211c0 con 0x7f866810e880 2024-04-24T09:08:09.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.508+0000 7f8667e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8668110850 0x7f86681a7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:09.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.508+0000 7f866d200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f866810e880 0x7f86681a75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:09.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.508+0000 7f8667e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8668110850 0x7f86681a7b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:33240/0 (socket says 172.21.15.83:33240) 2024-04-24T09:08:09.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.508+0000 7f866dc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f866811b7e0 0x7f86681abe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:09.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.508+0000 7f866d200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f866810e880 0x7f86681a75c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:56592/0 (socket says 172.21.15.83:56592) 2024-04-24T09:08:09.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.508+0000 7f866d200640 1 -- 172.21.15.83:0/1660854461 learned_addr learned my addr 172.21.15.83:0/1660854461 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:08:09.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.508+0000 7f866d200640 1 -- 172.21.15.83:0/1660854461 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8668110850 msgr2=0x7f86681a7b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:09.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.508+0000 7f866d200640 1 --2- 172.21.15.83:0/1660854461 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8668110850 0x7f86681a7b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:09.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.509+0000 7f866d200640 1 -- 172.21.15.83:0/1660854461 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f866811b7e0 msgr2=0x7f86681abe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:09.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.509+0000 7f866d200640 1 --2- 172.21.15.83:0/1660854461 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f866811b7e0 0x7f86681abe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:09.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.509+0000 7f866d200640 1 -- 172.21.15.83:0/1660854461 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86681ac5b0 con 0x7f866810e880 2024-04-24T09:08:09.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.509+0000 7f866dc00640 1 --2- 172.21.15.83:0/1660854461 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f866811b7e0 0x7f86681abe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:08:09.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.509+0000 7f866d200640 1 --2- 172.21.15.83:0/1660854461 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f866810e880 0x7f86681a75c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8660009d50 tx=0x7f866000c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:09.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.510+0000 7f8665600640 1 -- 172.21.15.83:0/1660854461 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8660003b70 con 0x7f866810e880 2024-04-24T09:08:09.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.510+0000 7f866fc00640 1 -- 172.21.15.83:0/1660854461 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86681ac8a0 con 0x7f866810e880 2024-04-24T09:08:09.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.510+0000 7f8665600640 1 -- 172.21.15.83:0/1660854461 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8660003d10 con 0x7f866810e880 2024-04-24T09:08:09.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.510+0000 7f866fc00640 1 -- 172.21.15.83:0/1660854461 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86681b4130 con 0x7f866810e880 2024-04-24T09:08:09.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.510+0000 7f8665600640 1 -- 172.21.15.83:0/1660854461 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86600108f0 con 0x7f866810e880 2024-04-24T09:08:09.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.514+0000 7f8672a4e740 1 Processor -- start 2024-04-24T09:08:09.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.515+0000 7f8672a4e740 1 -- start start 2024-04-24T09:08:09.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.515+0000 7f8672a4e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558b3ceef860 0x558b3ceefc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:09.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.515+0000 7f8672a4e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b3cef0180 0x558b3cefa310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:09.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.515+0000 7f8672a4e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558b3cefa850 0x558b3cefcc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:09.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.515+0000 7f8672a4e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x558b3ce551f0 con 0x558b3cef0180 2024-04-24T09:08:09.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.515+0000 7f8672a4e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558b3cdfb350 con 0x558b3cefa850 2024-04-24T09:08:09.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.515+0000 7f8672a4e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x558b3ce511b0 con 0x558b3ceef860 2024-04-24T09:08:09.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.516+0000 7f866dc00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558b3ceef860 0x558b3ceefc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:09.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.516+0000 7f8667e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558b3cefa850 0x558b3cefcc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:09.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.516+0000 7f866dc00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558b3ceef860 0x558b3ceefc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:56598/0 (socket says 172.21.15.83:56598) 2024-04-24T09:08:09.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.516+0000 7f866dc00640 1 -- 172.21.15.83:0/2200754291 learned_addr learned my addr 172.21.15.83:0/2200754291 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:08:09.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.516+0000 7f8665600640 1 -- 172.21.15.83:0/1660854461 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8660003ec0 con 0x7f866810e880 2024-04-24T09:08:09.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.516+0000 7f8667e00640 1 -- 172.21.15.83:0/2200754291 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558b3ceef860 msgr2=0x558b3ceefc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:09.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.516+0000 7f8667e00640 1 --2- 172.21.15.83:0/2200754291 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558b3ceef860 0x558b3ceefc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:09.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.516+0000 7f8667e00640 1 -- 172.21.15.83:0/2200754291 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b3cef0180 msgr2=0x558b3cefa310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:09.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.516+0000 7f8667e00640 1 --2- 172.21.15.83:0/2200754291 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b3cef0180 0x558b3cefa310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:09.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.516+0000 7f8667e00640 1 -- 172.21.15.83:0/2200754291 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b3ce4ff50 con 0x558b3cefa850 2024-04-24T09:08:09.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.517+0000 7f866dc00640 1 --2- 172.21.15.83:0/2200754291 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558b3ceef860 0x558b3ceefc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:08:09.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.517+0000 7f8667e00640 1 --2- 172.21.15.83:0/2200754291 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558b3cefa850 0x558b3cefcc40 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f865000bbc0 tx=0x7f86500107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ecb56fc2c4a69aa9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:09.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.518+0000 7f865e000640 1 -- 172.21.15.83:0/2200754291 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8650004030 con 0x558b3cefa850 2024-04-24T09:08:09.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.518+0000 7f865e000640 1 -- 172.21.15.83:0/2200754291 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f86500035f0 con 0x558b3cefa850 2024-04-24T09:08:09.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.518+0000 7f865e000640 1 -- 172.21.15.83:0/2200754291 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8650011680 con 0x558b3cefa850 2024-04-24T09:08:09.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.518+0000 7f8665600640 1 --2- 172.21.15.83:0/1660854461 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f864c0a3d20 0x7f864c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:09.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.518+0000 7f8665600640 1 -- 172.21.15.83:0/1660854461 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9671+0+0 (secure 0 0 0) 0x7f86600d1300 con 0x7f866810e880 2024-04-24T09:08:09.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.519+0000 7f866dc00640 1 --2- 172.21.15.83:0/1660854461 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f864c0a3d20 0x7f864c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:09.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.519+0000 7f8672a4e740 1 -- 172.21.15.83:0/2200754291 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558b3cefa850 msgr2=0x558b3cefcc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:09.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.519+0000 7f8672a4e740 1 --2- 172.21.15.83:0/2200754291 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558b3cefa850 0x558b3cefcc40 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f865000bbc0 tx=0x7f86500107a0 comp rx=0 tx=0).stop 2024-04-24T09:08:09.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.520+0000 7f866dc00640 1 --2- 172.21.15.83:0/1660854461 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f864c0a3d20 0x7f864c0a61e0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f8658004650 tx=0x7f8658005a40 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:09.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.520+0000 7f8672a4e740 1 -- 172.21.15.83:0/2200754291 shutdown_connections 2024-04-24T09:08:09.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.520+0000 7f8672a4e740 1 --2- 172.21.15.83:0/2200754291 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558b3cefa850 0x558b3cefcc40 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:09.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.520+0000 7f8672a4e740 1 --2- 172.21.15.83:0/2200754291 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b3cef0180 0x558b3cefa310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:09.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.520+0000 7f8672a4e740 1 --2- 172.21.15.83:0/2200754291 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558b3ceef860 0x558b3ceefc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:09.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.520+0000 7f8672a4e740 1 -- 172.21.15.83:0/2200754291 >> 172.21.15.83:0/2200754291 conn(0x558b3ceebb80 msgr2=0x558b3ceebf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:09.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.520+0000 7f8672a4e740 1 -- 172.21.15.83:0/2200754291 shutdown_connections 2024-04-24T09:08:09.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.520+0000 7f8672a4e740 1 -- 172.21.15.83:0/2200754291 wait complete. 2024-04-24T09:08:09.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.520+0000 7f8672a4e740 1 -- start start 2024-04-24T09:08:09.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.520+0000 7f8672a4e740 1 Processor -- start 2024-04-24T09:08:09.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.521+0000 7f8672a4e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558b3ceef860 0x558b3cef7820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:09.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.521+0000 7f8672a4e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b3cef0180 0x558b3cef7d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:09.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.521+0000 7f8672a4e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558b3cefa850 0x558b3cef82a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:09.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.521+0000 7f8672a4e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x558b3cefd640 con 0x558b3cef0180 2024-04-24T09:08:09.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.521+0000 7f8672a4e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558b3ce0e690 con 0x558b3ceef860 2024-04-24T09:08:09.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.521+0000 7f8672a4e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x558b3cefd7c0 con 0x558b3cefa850 2024-04-24T09:08:09.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.521+0000 7f8672a4e740 10 client.0 _ll_register_callbacks cb 0x558b3ce4a650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:08:09.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.521+0000 7f8667e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b3cef0180 0x558b3cef7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:09.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.521+0000 7f8667e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b3cef0180 0x558b3cef7d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42044/0 (socket says 172.21.15.83:42044) 2024-04-24T09:08:09.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.521+0000 7f8667e00640 1 -- 172.21.15.83:0/3938312479 learned_addr learned my addr 172.21.15.83:0/3938312479 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:08:09.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.522+0000 7f866d200640 1 --2- 172.21.15.83:0/3938312479 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558b3ceef860 0x558b3cef7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:09.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.522+0000 7f866dc00640 1 --2- 172.21.15.83:0/3938312479 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558b3cefa850 0x558b3cef82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:09.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.522+0000 7f8667e00640 1 -- 172.21.15.83:0/3938312479 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558b3cefa850 msgr2=0x558b3cef82a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:09.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.522+0000 7f8667e00640 1 --2- 172.21.15.83:0/3938312479 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558b3cefa850 0x558b3cef82a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:09.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.522+0000 7f8667e00640 1 -- 172.21.15.83:0/3938312479 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558b3ceef860 msgr2=0x558b3cef7820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:09.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.522+0000 7f8667e00640 1 --2- 172.21.15.83:0/3938312479 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558b3ceef860 0x558b3cef7820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:09.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.522+0000 7f8667e00640 1 -- 172.21.15.83:0/3938312479 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b3ce0d920 con 0x558b3cef0180 2024-04-24T09:08:09.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.522+0000 7f8667e00640 1 --2- 172.21.15.83:0/3938312479 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b3cef0180 0x558b3cef7d60 secure :-1 s=READY pgs=2302 cs=0 l=1 rev1=1 crypto rx=0x7f8650011010 tx=0x7f865000bfa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e17401c508740b57 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:09.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.522+0000 7f8656a00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:08:09.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.523+0000 7f8656a00640 1 -- 172.21.15.83:0/3938312479 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f865002a060 con 0x558b3cef0180 2024-04-24T09:08:09.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.523+0000 7f8656a00640 1 -- 172.21.15.83:0/3938312479 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9671+0+0 (secure 0 0 0) 0x7f86500041e0 con 0x558b3cef0180 2024-04-24T09:08:09.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.523+0000 7f8672a4e740 1 -- 172.21.15.83:0/3938312479 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86500035f0 con 0x558b3cef0180 2024-04-24T09:08:09.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.523+0000 7f8656a00640 1 -- 172.21.15.83:0/3938312479 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f8650031050 con 0x558b3cef0180 2024-04-24T09:08:09.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.524+0000 7f8672a4e740 10 client.21435 fetch_fsmap learned FSMap version 54 2024-04-24T09:08:09.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.524+0000 7f8672a4e740 1 -- 172.21.15.83:0/3938312479 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x558b3cc317d0 con 0x558b3cef0180 2024-04-24T09:08:09.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.524+0000 7f8656a00640 1 -- 172.21.15.83:0/3938312479 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f86500035f0 con 0x558b3cef0180 2024-04-24T09:08:09.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.524+0000 7f8656a00640 1 -- 172.21.15.83:0/3938312479 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 54) ==== 8394+0+0 (secure 0 0 0) 0x7f86500048f0 con 0x558b3cef0180 2024-04-24T09:08:09.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.524+0000 7f8672a4e740 10 client.21435 fetch_fsmap finished waiting for FSMap version 54 2024-04-24T09:08:09.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.524+0000 7f8672a4e740 10 client.21435 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:08:09.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.524+0000 7f8672a4e740 20 client.21435 populate_metadata read hostname 'smithi083' 2024-04-24T09:08:09.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.524+0000 7f8672a4e740 1 --2- 172.21.15.83:0/3938312479 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x558b3cf0ab60 0x558b3cf0cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:09.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.524+0000 7f8672a4e740 4 client.21435 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:08:09.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.524+0000 7f8672a4e740 1 -- 172.21.15.83:0/3938312479 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558b3ce62c10 con 0x558b3cf0ab60 2024-04-24T09:08:09.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.525+0000 7f866d200640 1 --2- 172.21.15.83:0/3938312479 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x558b3cf0ab60 0x558b3cf0cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:09.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.525+0000 7f866d200640 1 --2- 172.21.15.83:0/3938312479 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x558b3cf0ab60 0x558b3cf0cf50 crc :-1 s=READY pgs=421 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41b62fb73915b9e6 server_cookie=5a647cad8cda35f8 in_seq=0 out_seq=0 2024-04-24T09:08:09.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.525+0000 7f8656a00640 10 client.21435 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:08:09.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.526+0000 7f8656a00640 1 -- 172.21.15.83:0/3938312479 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558b3ce62c10 con 0x558b3cf0ab60 2024-04-24T09:08:09.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.526+0000 7f8656a00640 10 client.21435 handle_command_reply: tid=0 2024-04-24T09:08:09.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.526+0000 7f8672a4e740 1 client.21435 shutdown 2024-04-24T09:08:09.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.526+0000 7f8672a4e740 1 -- 172.21.15.83:0/3938312479 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b3cef0180 msgr2=0x558b3cef7d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:09.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.526+0000 7f8672a4e740 1 --2- 172.21.15.83:0/3938312479 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b3cef0180 0x558b3cef7d60 secure :-1 s=READY pgs=2302 cs=0 l=1 rev1=1 crypto rx=0x7f8650011010 tx=0x7f865000bfa0 comp rx=0 tx=0).stop 2024-04-24T09:08:09.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.527+0000 7f8672a4e740 1 -- 172.21.15.83:0/3938312479 shutdown_connections 2024-04-24T09:08:09.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.527+0000 7f8672a4e740 1 --2- 172.21.15.83:0/3938312479 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x558b3cf0ab60 0x558b3cf0cf50 crc :-1 s=READY pgs=421 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:09.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.527+0000 7f8672a4e740 1 --2- 172.21.15.83:0/3938312479 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558b3cefa850 0x558b3cef82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:09.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.527+0000 7f8672a4e740 1 --2- 172.21.15.83:0/3938312479 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b3cef0180 0x558b3cef7d60 unknown :-1 s=CLOSED pgs=2302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:09.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.527+0000 7f8672a4e740 1 --2- 172.21.15.83:0/3938312479 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558b3ceef860 0x558b3cef7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:09.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.527+0000 7f8656a00640 0 client.21435 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:08:09.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.527+0000 7f8672a4e740 1 -- 172.21.15.83:0/3938312479 >> 172.21.15.83:0/3938312479 conn(0x558b3ceebb80 msgr2=0x558b3cef1980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:09.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.527+0000 7f8672a4e740 1 -- 172.21.15.83:0/3938312479 shutdown_connections 2024-04-24T09:08:09.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.527+0000 7f8672a4e740 1 -- 172.21.15.83:0/3938312479 wait complete. 2024-04-24T09:08:09.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.527+0000 7f8672a4e740 20 client.21435 trim_cache size 0 max 16384 2024-04-24T09:08:09.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.553+0000 7f8672a4e740 1 Processor -- start 2024-04-24T09:08:09.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.554+0000 7f8672a4e740 1 -- start start 2024-04-24T09:08:09.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.554+0000 7f8672a4e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558b3ceef860 0x558b3cef6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:09.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.554+0000 7f8672a4e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558b3cef0180 0x558b3cf06500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:09.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.554+0000 7f8672a4e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b3cefa850 0x558b3cf125b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:09.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.554+0000 7f8672a4e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f86681be640 con 0x558b3cefa850 2024-04-24T09:08:09.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.554+0000 7f8672a4e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f86681c0270 con 0x558b3cef0180 2024-04-24T09:08:09.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.554+0000 7f8672a4e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f86681bed10 con 0x558b3ceef860 2024-04-24T09:08:09.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.554+0000 7f866dc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b3cefa850 0x558b3cf125b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:09.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.555+0000 7f8667e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558b3cef0180 0x558b3cf06500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:09.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.555+0000 7f866dc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b3cefa850 0x558b3cf125b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42058/0 (socket says 172.21.15.83:42058) 2024-04-24T09:08:09.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.555+0000 7f866dc00640 1 -- 172.21.15.83:0/2861949461 learned_addr learned my addr 172.21.15.83:0/2861949461 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:08:09.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.555+0000 7f866d200640 1 --2- 172.21.15.83:0/2861949461 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558b3ceef860 0x558b3cef6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:09.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.555+0000 7f8667e00640 1 -- 172.21.15.83:0/2861949461 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558b3ceef860 msgr2=0x558b3cef6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:09.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.555+0000 7f8667e00640 1 --2- 172.21.15.83:0/2861949461 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558b3ceef860 0x558b3cef6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:09.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.555+0000 7f8667e00640 1 -- 172.21.15.83:0/2861949461 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b3cefa850 msgr2=0x558b3cf125b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:09.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.555+0000 7f8667e00640 1 --2- 172.21.15.83:0/2861949461 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b3cefa850 0x558b3cf125b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:09.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.555+0000 7f8667e00640 1 -- 172.21.15.83:0/2861949461 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b3ceeffb0 con 0x558b3cef0180 2024-04-24T09:08:09.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.556+0000 7f8667e00640 1 --2- 172.21.15.83:0/2861949461 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558b3cef0180 0x558b3cf06500 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8650000c00 tx=0x7f8650010ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e444a6211cdf69d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:09.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.556+0000 7f865d600640 1 -- 172.21.15.83:0/2861949461 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f865003a070 con 0x558b3cef0180 2024-04-24T09:08:09.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.556+0000 7f865d600640 1 -- 172.21.15.83:0/2861949461 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f865000ba40 con 0x558b3cef0180 2024-04-24T09:08:09.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.556+0000 7f865d600640 1 -- 172.21.15.83:0/2861949461 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f865002a040 con 0x558b3cef0180 2024-04-24T09:08:09.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.557+0000 7f8672a4e740 1 -- 172.21.15.83:0/2861949461 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558b3cef0180 msgr2=0x558b3cf06500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:09.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.557+0000 7f8672a4e740 1 --2- 172.21.15.83:0/2861949461 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558b3cef0180 0x558b3cf06500 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8650000c00 tx=0x7f8650010ce0 comp rx=0 tx=0).stop 2024-04-24T09:08:09.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.557+0000 7f8672a4e740 1 -- 172.21.15.83:0/2861949461 shutdown_connections 2024-04-24T09:08:09.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.557+0000 7f8672a4e740 1 --2- 172.21.15.83:0/2861949461 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b3cefa850 0x558b3cf125b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:09.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.557+0000 7f8672a4e740 1 --2- 172.21.15.83:0/2861949461 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558b3cef0180 0x558b3cf06500 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:09.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.557+0000 7f8672a4e740 1 --2- 172.21.15.83:0/2861949461 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558b3ceef860 0x558b3cef6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:09.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.557+0000 7f8672a4e740 1 -- 172.21.15.83:0/2861949461 >> 172.21.15.83:0/2861949461 conn(0x558b3ceebb80 msgr2=0x558b3ceeac20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:09.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.557+0000 7f8672a4e740 1 -- 172.21.15.83:0/2861949461 shutdown_connections 2024-04-24T09:08:09.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.557+0000 7f8672a4e740 1 -- 172.21.15.83:0/2861949461 wait complete. 2024-04-24T09:08:09.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.558+0000 7f8672a4e740 1 -- start start 2024-04-24T09:08:09.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.558+0000 7f8672a4e740 1 Processor -- start 2024-04-24T09:08:09.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.559+0000 7f8672a4e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558b3ceef860 0x558b3cefe740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:09.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.559+0000 7f8672a4e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558b3cef0180 0x558b3cefec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:09.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.559+0000 7f8672a4e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b3cefa850 0x558b3cf6fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:09.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.559+0000 7f8672a4e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f86681bfae0 con 0x558b3cefa850 2024-04-24T09:08:09.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.559+0000 7f8672a4e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f86681bf3e0 con 0x558b3ceef860 2024-04-24T09:08:09.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.559+0000 7f8672a4e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f86681bc640 con 0x558b3cef0180 2024-04-24T09:08:09.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.559+0000 7f8672a4e740 10 client.0 _ll_register_callbacks cb 0x7f8658009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:08:09.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.559+0000 7f866dc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b3cefa850 0x558b3cf6fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:09.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.559+0000 7f866d200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558b3ceef860 0x558b3cefe740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:09.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.559+0000 7f866dc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b3cefa850 0x558b3cf6fe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42070/0 (socket says 172.21.15.83:42070) 2024-04-24T09:08:09.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.559+0000 7f866dc00640 1 -- 172.21.15.83:0/2058810134 learned_addr learned my addr 172.21.15.83:0/2058810134 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:08:09.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.559+0000 7f866d200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558b3ceef860 0x558b3cefe740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:33278/0 (socket says 172.21.15.83:33278) 2024-04-24T09:08:09.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.559+0000 7f8667e00640 1 --2- 172.21.15.83:0/2058810134 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558b3cef0180 0x558b3cefec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:09.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.560+0000 7f866dc00640 1 -- 172.21.15.83:0/2058810134 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558b3cef0180 msgr2=0x558b3cefec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:09.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.560+0000 7f866dc00640 1 --2- 172.21.15.83:0/2058810134 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558b3cef0180 0x558b3cefec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:09.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.560+0000 7f866dc00640 1 -- 172.21.15.83:0/2058810134 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558b3ceef860 msgr2=0x558b3cefe740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:09.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.560+0000 7f866dc00640 1 --2- 172.21.15.83:0/2058810134 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558b3ceef860 0x558b3cefe740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:09.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.560+0000 7f866dc00640 1 -- 172.21.15.83:0/2058810134 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b3cb673f0 con 0x558b3cefa850 2024-04-24T09:08:09.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.560+0000 7f866d200640 1 --2- 172.21.15.83:0/2058810134 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558b3ceef860 0x558b3cefe740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:08:09.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.560+0000 7f866dc00640 1 --2- 172.21.15.83:0/2058810134 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b3cefa850 0x558b3cf6fe20 secure :-1 s=READY pgs=2303 cs=0 l=1 rev1=1 crypto rx=0x7f865802a960 tx=0x7f8658009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f8ca2f7160c240f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:09.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.560+0000 7f8656a00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:08:09.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.561+0000 7f8656a00640 1 -- 172.21.15.83:0/2058810134 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8658035040 con 0x558b3cefa850 2024-04-24T09:08:09.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.561+0000 7f8656a00640 1 -- 172.21.15.83:0/2058810134 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9671+0+0 (secure 0 0 0) 0x7f8658031b80 con 0x558b3cefa850 2024-04-24T09:08:09.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.561+0000 7f8672a4e740 1 -- 172.21.15.83:0/2058810134 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f865000ba40 con 0x558b3cefa850 2024-04-24T09:08:09.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.561+0000 7f8656a00640 1 -- 172.21.15.83:0/2058810134 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f8658031d80 con 0x558b3cefa850 2024-04-24T09:08:09.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.561+0000 7f8672a4e740 10 client.21441 fetch_fsmap learned FSMap version 54 2024-04-24T09:08:09.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.561+0000 7f8672a4e740 1 -- 172.21.15.83:0/2058810134 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x558b3cb270f0 con 0x558b3cefa850 2024-04-24T09:08:09.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.562+0000 7f8656a00640 1 -- 172.21.15.83:0/2058810134 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f865000ba40 con 0x558b3cefa850 2024-04-24T09:08:09.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.562+0000 7f8656a00640 1 -- 172.21.15.83:0/2058810134 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 54) ==== 8394+0+0 (secure 0 0 0) 0x7f8658044040 con 0x558b3cefa850 2024-04-24T09:08:09.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.562+0000 7f8672a4e740 10 client.21441 fetch_fsmap finished waiting for FSMap version 54 2024-04-24T09:08:09.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.562+0000 7f8672a4e740 10 client.21441 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:08:09.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.562+0000 7f8672a4e740 20 client.21441 populate_metadata read hostname 'smithi083' 2024-04-24T09:08:09.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.562+0000 7f8672a4e740 1 --2- 172.21.15.83:0/2058810134 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x558b3cf71980 0x558b3cf73da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:09.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.562+0000 7f8672a4e740 4 client.21441 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T09:08:09.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.562+0000 7f8672a4e740 1 -- 172.21.15.83:0/2058810134 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558b3ce5ca80 con 0x558b3cf71980 2024-04-24T09:08:09.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.563+0000 7f866d200640 1 --2- 172.21.15.83:0/2058810134 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x558b3cf71980 0x558b3cf73da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:09.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.563+0000 7f866d200640 1 --2- 172.21.15.83:0/2058810134 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x558b3cf71980 0x558b3cf73da0 crc :-1 s=READY pgs=422 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23b231d28101c643 server_cookie=98b52dfd5aea6bf6 in_seq=0 out_seq=0 2024-04-24T09:08:09.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.563+0000 7f8656a00640 10 client.21441 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:08:09.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.564+0000 7f8656a00640 1 -- 172.21.15.83:0/2058810134 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x558b3ce5ca80 con 0x558b3cf71980 2024-04-24T09:08:09.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.564+0000 7f8656a00640 10 client.21441 handle_command_reply: tid=0 2024-04-24T09:08:09.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.564+0000 7f8672a4e740 1 client.21441 shutdown 2024-04-24T09:08:09.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.564+0000 7f8672a4e740 1 -- 172.21.15.83:0/2058810134 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b3cefa850 msgr2=0x558b3cf6fe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:09.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.564+0000 7f8672a4e740 1 --2- 172.21.15.83:0/2058810134 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b3cefa850 0x558b3cf6fe20 secure :-1 s=READY pgs=2303 cs=0 l=1 rev1=1 crypto rx=0x7f865802a960 tx=0x7f8658009260 comp rx=0 tx=0).stop 2024-04-24T09:08:09.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.565+0000 7f8672a4e740 1 -- 172.21.15.83:0/2058810134 shutdown_connections 2024-04-24T09:08:09.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.565+0000 7f8672a4e740 1 --2- 172.21.15.83:0/2058810134 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x558b3cf71980 0x558b3cf73da0 crc :-1 s=READY pgs=422 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:09.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.565+0000 7f8672a4e740 1 --2- 172.21.15.83:0/2058810134 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b3cefa850 0x558b3cf6fe20 unknown :-1 s=CLOSED pgs=2303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:09.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.565+0000 7f8672a4e740 1 --2- 172.21.15.83:0/2058810134 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558b3cef0180 0x558b3cefec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:09.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.565+0000 7f8672a4e740 1 --2- 172.21.15.83:0/2058810134 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558b3ceef860 0x558b3cefe740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:09.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.565+0000 7f8656a00640 0 client.21441 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:08:09.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.565+0000 7f8672a4e740 1 -- 172.21.15.83:0/2058810134 >> 172.21.15.83:0/2058810134 conn(0x558b3ceebb80 msgr2=0x558b3cef0fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:09.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.565+0000 7f8672a4e740 1 -- 172.21.15.83:0/2058810134 shutdown_connections 2024-04-24T09:08:09.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.565+0000 7f8672a4e740 1 -- 172.21.15.83:0/2058810134 wait complete. 2024-04-24T09:08:09.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.566+0000 7f8672a4e740 20 client.21441 trim_cache size 0 max 16384 2024-04-24T09:08:09.567 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T09:08:09.568 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T09:08:09.568 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-04-24T09:08:09.568 INFO:teuthology.orchestra.run.smithi083.stdout: "096f8830-ef5c-410e-8526-5ab1dcac4bf4": { 2024-04-24T09:08:09.568 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-04-24T09:08:09.568 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "096f8830-ef5c-410e-8526-5ab1dcac4bf4", 2024-04-24T09:08:09.568 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-04-24T09:08:09.568 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:08:09.568 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:08:09.568 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T09:08:09.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.566+0000 7f866fc00640 1 -- 172.21.15.83:0/1660854461 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f864c0a3d20 msgr2=0x7f864c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:09.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.566+0000 7f866fc00640 1 --2- 172.21.15.83:0/1660854461 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f864c0a3d20 0x7f864c0a61e0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f8658004650 tx=0x7f8658005a40 comp rx=0 tx=0).stop 2024-04-24T09:08:09.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.566+0000 7f866fc00640 1 -- 172.21.15.83:0/1660854461 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f866810e880 msgr2=0x7f86681a75c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:09.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.566+0000 7f866fc00640 1 --2- 172.21.15.83:0/1660854461 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f866810e880 0x7f86681a75c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8660009d50 tx=0x7f866000c6a0 comp rx=0 tx=0).stop 2024-04-24T09:08:09.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.567+0000 7f866fc00640 1 -- 172.21.15.83:0/1660854461 shutdown_connections 2024-04-24T09:08:09.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.567+0000 7f866fc00640 1 --2- 172.21.15.83:0/1660854461 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f864c0a3d20 0x7f864c0a61e0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:09.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.567+0000 7f866fc00640 1 --2- 172.21.15.83:0/1660854461 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f866811b7e0 0x7f86681abe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:09.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.567+0000 7f866fc00640 1 --2- 172.21.15.83:0/1660854461 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8668110850 0x7f86681a7b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:09.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.567+0000 7f866fc00640 1 --2- 172.21.15.83:0/1660854461 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f866810e880 0x7f86681a75c0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:09.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.567+0000 7f866fc00640 1 -- 172.21.15.83:0/1660854461 >> 172.21.15.83:0/1660854461 conn(0x7f866806d110 msgr2=0x7f866811c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:09.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.567+0000 7f866fc00640 1 -- 172.21.15.83:0/1660854461 shutdown_connections 2024-04-24T09:08:09.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:09.567+0000 7f866fc00640 1 -- 172.21.15.83:0/1660854461 wait complete. 2024-04-24T09:08:09.589 INFO:tasks.cephfs.filesystem:scrub status for tag:096f8830-ef5c-410e-8526-5ab1dcac4bf4 - {'path': '/', 'tag': '096f8830-ef5c-410e-8526-5ab1dcac4bf4', 'options': 'recursive,force'} 2024-04-24T09:08:10.166 INFO:tasks.workunit.client.0.smithi083.stdout: 10 20809 7.09 MB/sec execute 41 sec latency 135.915 ms 2024-04-24T09:08:10.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:08:10 smithi138 ceph-mon[31441]: pgmap v520: 129 pgs: 129 active+clean; 465 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.3 MiB/s wr, 263 op/s 2024-04-24T09:08:10.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:08:10 smithi138 ceph-mon[31441]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:08:10.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:08:10 smithi138 ceph-mon[31441]: scrub queued for path: / 2024-04-24T09:08:10.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:08:10 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:08:10.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:08:10 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:08:10.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:08:10 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:08:10.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:08:10 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:08:10.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:08:10 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:08:10.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:08:10 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:08:10.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:08:10 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:08:10.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:08:10 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:08:10.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:08:10 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:08:10.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:08:10 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:08:10.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:08:10 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:08:10.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:08:10 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:08:10.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:08:10 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:08:10.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:08:10 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:08:10.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:08:10 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:08:10.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:08:10 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:08:10.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:08:10 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:08:10.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:08:10 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:08:10.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:08:10 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:08:10.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:08:10 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:08:10.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:08:10 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:08:10.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:08:10 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:08:10.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:08:10 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:08:10.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:08:10 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:08:10.457 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:08:10 smithi138 ceph-mon[31441]: osdmap e116: 12 total, 12 up, 12 in 2024-04-24T09:08:10.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:08:10 smithi189 ceph-mon[31791]: pgmap v520: 129 pgs: 129 active+clean; 465 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.3 MiB/s wr, 263 op/s 2024-04-24T09:08:10.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:08:10 smithi189 ceph-mon[31791]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:08:10.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:08:10 smithi189 ceph-mon[31791]: scrub queued for path: / 2024-04-24T09:08:10.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:08:10 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:08:10.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:08:10 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:08:10.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:08:10 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:08:10.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:08:10 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:08:10.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:08:10 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:08:10.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:08:10 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:08:10.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:08:10 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:08:10.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:08:10 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:08:10.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:08:10 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:08:10.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:08:10 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:08:10.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:08:10 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:08:10.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:08:10 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:08:10.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:08:10 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:08:10.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:08:10 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:08:10.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:08:10 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:08:10.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:08:10 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:08:10.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:08:10 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:08:10.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:08:10 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:08:10.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:08:10 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:08:10.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:08:10 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:08:10.459 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:08:10 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:08:10.459 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:08:10 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:08:10.459 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:08:10 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:08:10.459 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:08:10 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:08:10.459 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:08:10 smithi189 ceph-mon[31791]: osdmap e116: 12 total, 12 up, 12 in 2024-04-24T09:08:10.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:08:10 smithi083 ceph-mon[27907]: pgmap v520: 129 pgs: 129 active+clean; 465 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.3 MiB/s wr, 263 op/s 2024-04-24T09:08:10.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:08:10 smithi083 ceph-mon[27907]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:08:10.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:08:10 smithi083 ceph-mon[27907]: scrub queued for path: / 2024-04-24T09:08:10.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:08:10 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:08:10.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:08:10 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:08:10.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:08:10 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:08:10.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:08:10 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:08:10.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:08:10 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:08:10.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:08:10 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:08:10.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:08:10 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:08:10.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:08:10 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:08:10.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:08:10 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:08:10.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:08:10 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:08:10.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:08:10 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:08:10.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:08:10 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:08:10.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:08:10 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:08:10.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:08:10 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:08:10.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:08:10 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:08:10.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:08:10 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:08:10.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:08:10 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:08:10.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:08:10 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:08:10.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:08:10 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:08:10.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:08:10 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:08:10.555 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:08:10 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:08:10.555 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:08:10 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:08:10.555 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:08:10 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:08:10.555 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:08:10 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:08:10.555 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:08:10 smithi083 ceph-mon[27907]: osdmap e116: 12 total, 12 up, 12 in 2024-04-24T09:08:11.164 INFO:tasks.workunit.client.0.smithi083.stdout: 10 20926 7.04 MB/sec execute 42 sec latency 84.028 ms 2024-04-24T09:08:11.372 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.372+0000 7f756fe00640 1 -- 172.21.15.83:0/2478295924 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7570100f60 msgr2=0x7f7570101340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:11.372 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.372+0000 7f756fe00640 1 --2- 172.21.15.83:0/2478295924 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7570100f60 0x7f7570101340 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7568009ca0 tx=0x7f756802a8b0 comp rx=0 tx=0).stop 2024-04-24T09:08:11.372 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.372+0000 7f756fe00640 1 -- 172.21.15.83:0/2478295924 shutdown_connections 2024-04-24T09:08:11.372 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.372+0000 7f756fe00640 1 --2- 172.21.15.83:0/2478295924 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7570106030 0x7f757010dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:11.372 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.372+0000 7f756fe00640 1 --2- 172.21.15.83:0/2478295924 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7570101910 0x7f7570105950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:11.372 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.372+0000 7f756fe00640 1 --2- 172.21.15.83:0/2478295924 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7570100f60 0x7f7570101340 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:11.372 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.372+0000 7f756fe00640 1 -- 172.21.15.83:0/2478295924 >> 172.21.15.83:0/2478295924 conn(0x7f75700fc820 msgr2=0x7f75700fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:11.373 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.372+0000 7f756fe00640 1 -- 172.21.15.83:0/2478295924 shutdown_connections 2024-04-24T09:08:11.373 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.372+0000 7f756fe00640 1 -- 172.21.15.83:0/2478295924 wait complete. 2024-04-24T09:08:11.374 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.374+0000 7f756fe00640 1 Processor -- start 2024-04-24T09:08:11.374 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.374+0000 7f756fe00640 1 -- start start 2024-04-24T09:08:11.375 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.374+0000 7f756fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7570100f60 0x7f75701a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:11.375 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.374+0000 7f756fe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7570106030 0x7f75701a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:11.375 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.374+0000 7f756fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f757010bb70 0x7f757019a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:11.375 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.374+0000 7f756fe00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f75701120e0 con 0x7f7570100f60 2024-04-24T09:08:11.375 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.375+0000 7f756fe00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7570111f60 con 0x7f757010bb70 2024-04-24T09:08:11.375 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.375+0000 7f756fe00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7570112260 con 0x7f7570106030 2024-04-24T09:08:11.375 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.375+0000 7f756ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7570100f60 0x7f75701a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:11.376 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.375+0000 7f756f400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f757010bb70 0x7f757019a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:11.376 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.375+0000 7f756e000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7570106030 0x7f75701a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:11.376 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.375+0000 7f756f400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f757010bb70 0x7f757019a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:33292/0 (socket says 172.21.15.83:33292) 2024-04-24T09:08:11.376 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.375+0000 7f756f400640 1 -- 172.21.15.83:0/1552020503 learned_addr learned my addr 172.21.15.83:0/1552020503 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:08:11.376 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.376+0000 7f756f400640 1 -- 172.21.15.83:0/1552020503 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7570106030 msgr2=0x7f75701a0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:11.376 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.376+0000 7f756f400640 1 --2- 172.21.15.83:0/1552020503 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7570106030 0x7f75701a0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:11.377 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.376+0000 7f756f400640 1 -- 172.21.15.83:0/1552020503 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7570100f60 msgr2=0x7f75701a0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:11.377 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.376+0000 7f756f400640 1 --2- 172.21.15.83:0/1552020503 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7570100f60 0x7f75701a0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:11.377 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.376+0000 7f756f400640 1 -- 172.21.15.83:0/1552020503 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f757019aef0 con 0x7f757010bb70 2024-04-24T09:08:11.377 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.376+0000 7f756f400640 1 --2- 172.21.15.83:0/1552020503 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f757010bb70 0x7f757019a720 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f756000d950 tx=0x7f756000de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:11.378 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.377+0000 7f7567400640 1 -- 172.21.15.83:0/1552020503 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7560017030 con 0x7f757010bb70 2024-04-24T09:08:11.378 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.377+0000 7f756fe00640 1 -- 172.21.15.83:0/1552020503 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f757019b1e0 con 0x7f757010bb70 2024-04-24T09:08:11.378 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.377+0000 7f756fe00640 1 -- 172.21.15.83:0/1552020503 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75701a13d0 con 0x7f757010bb70 2024-04-24T09:08:11.378 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.379+0000 7f7567400640 1 -- 172.21.15.83:0/1552020503 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7560004020 con 0x7f757010bb70 2024-04-24T09:08:11.379 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.379+0000 7f7567400640 1 -- 172.21.15.83:0/1552020503 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7560003a80 con 0x7f757010bb70 2024-04-24T09:08:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.382+0000 7f75789ca740 1 Processor -- start 2024-04-24T09:08:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.382+0000 7f7567400640 1 -- 172.21.15.83:0/1552020503 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f756001a070 con 0x7f757010bb70 2024-04-24T09:08:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.382+0000 7f75789ca740 1 -- start start 2024-04-24T09:08:11.383 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.382+0000 7f75789ca740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a23a4e3c0 0x563a23a4e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:11.383 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.383+0000 7f75789ca740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a23a4ece0 0x563a23a59060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:11.383 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.383+0000 7f75789ca740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563a23a595a0 0x563a23a5b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:11.383 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.383+0000 7f75789ca740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x563a239b4900 con 0x563a23a4ece0 2024-04-24T09:08:11.383 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.383+0000 7f75789ca740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563a2395af20 con 0x563a23a4e3c0 2024-04-24T09:08:11.383 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.383+0000 7f75789ca740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x563a239b08c0 con 0x563a23a595a0 2024-04-24T09:08:11.384 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.383+0000 7f756ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a23a4ece0 0x563a23a59060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:11.384 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.383+0000 7f756e000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a23a4e3c0 0x563a23a4e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:11.384 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.383+0000 7f756ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a23a4ece0 0x563a23a59060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42098/0 (socket says 172.21.15.83:42098) 2024-04-24T09:08:11.384 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.383+0000 7f756ea00640 1 -- 172.21.15.83:0/3097671942 learned_addr learned my addr 172.21.15.83:0/3097671942 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:08:11.384 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.383+0000 7f756e000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a23a4e3c0 0x563a23a4e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:33300/0 (socket says 172.21.15.83:33300) 2024-04-24T09:08:11.384 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.384+0000 7f756e000640 1 --2- 172.21.15.83:0/3097671942 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563a23a595a0 0x563a23a5b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:11.384 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.384+0000 7f756ea00640 1 -- 172.21.15.83:0/3097671942 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563a23a595a0 msgr2=0x563a23a5b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:11.384 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.384+0000 7f756ea00640 1 --2- 172.21.15.83:0/3097671942 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563a23a595a0 0x563a23a5b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:11.384 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.384+0000 7f756ea00640 1 -- 172.21.15.83:0/3097671942 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a23a4e3c0 msgr2=0x563a23a4e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:11.384 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.384+0000 7f756ea00640 1 --2- 172.21.15.83:0/3097671942 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a23a4e3c0 0x563a23a4e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:11.385 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.384+0000 7f756ea00640 1 -- 172.21.15.83:0/3097671942 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a2397fe10 con 0x563a23a4ece0 2024-04-24T09:08:11.385 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.384+0000 7f7567400640 1 --2- 172.21.15.83:0/1552020503 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f75540a3d90 0x7f75540a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:11.386 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.384+0000 7f756ea00640 1 --2- 172.21.15.83:0/3097671942 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a23a4ece0 0x563a23a59060 secure :-1 s=READY pgs=2304 cs=0 l=1 rev1=1 crypto rx=0x7f7568009890 tx=0x7f7568030450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce5327643134f308 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:11.386 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.384+0000 7f7567400640 1 -- 172.21.15.83:0/1552020503 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 9583+0+0 (secure 0 0 0) 0x7f75600cfef0 con 0x7f757010bb70 2024-04-24T09:08:11.386 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.385+0000 7f756e000640 1 --2- 172.21.15.83:0/1552020503 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f75540a3d90 0x7f75540a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:11.386 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.385+0000 7f755be00640 1 -- 172.21.15.83:0/3097671942 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7568030a60 con 0x563a23a4ece0 2024-04-24T09:08:11.386 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.385+0000 7f755be00640 1 -- 172.21.15.83:0/3097671942 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7568036030 con 0x563a23a4ece0 2024-04-24T09:08:11.386 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.385+0000 7f755be00640 1 -- 172.21.15.83:0/3097671942 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75680044f0 con 0x563a23a4ece0 2024-04-24T09:08:11.387 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.386+0000 7f756e000640 1 --2- 172.21.15.83:0/1552020503 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f75540a3d90 0x7f75540a6250 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x563a23a4bcf0 tx=0x7f755c005c90 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:11.387 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.386+0000 7f75789ca740 1 -- 172.21.15.83:0/3097671942 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a23a4ece0 msgr2=0x563a23a59060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:11.387 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.386+0000 7f75789ca740 1 --2- 172.21.15.83:0/3097671942 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a23a4ece0 0x563a23a59060 secure :-1 s=READY pgs=2304 cs=0 l=1 rev1=1 crypto rx=0x7f7568009890 tx=0x7f7568030450 comp rx=0 tx=0).stop 2024-04-24T09:08:11.387 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.386+0000 7f75789ca740 1 -- 172.21.15.83:0/3097671942 shutdown_connections 2024-04-24T09:08:11.387 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.386+0000 7f75789ca740 1 --2- 172.21.15.83:0/3097671942 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563a23a595a0 0x563a23a5b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:11.387 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.386+0000 7f75789ca740 1 --2- 172.21.15.83:0/3097671942 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a23a4ece0 0x563a23a59060 unknown :-1 s=CLOSED pgs=2304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:11.387 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.386+0000 7f75789ca740 1 --2- 172.21.15.83:0/3097671942 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a23a4e3c0 0x563a23a4e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:11.387 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.386+0000 7f75789ca740 1 -- 172.21.15.83:0/3097671942 >> 172.21.15.83:0/3097671942 conn(0x563a23a4a8d0 msgr2=0x563a23a4acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:11.387 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.386+0000 7f75789ca740 1 -- 172.21.15.83:0/3097671942 shutdown_connections 2024-04-24T09:08:11.387 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.386+0000 7f75789ca740 1 -- 172.21.15.83:0/3097671942 wait complete. 2024-04-24T09:08:11.389 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.387+0000 7f75789ca740 1 -- start start 2024-04-24T09:08:11.389 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.387+0000 7f75789ca740 1 Processor -- start 2024-04-24T09:08:11.389 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.388+0000 7f75789ca740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a23a4e3c0 0x563a23a564e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:11.389 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.388+0000 7f75789ca740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a23a4ece0 0x563a23a56a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:11.389 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.388+0000 7f75789ca740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563a23a595a0 0x563a23a56f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:11.389 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.388+0000 7f75789ca740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x563a23a5c0e0 con 0x563a23a4ece0 2024-04-24T09:08:11.389 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.388+0000 7f75789ca740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563a2396e2c0 con 0x563a23a4e3c0 2024-04-24T09:08:11.389 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.388+0000 7f75789ca740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x563a23a5c260 con 0x563a23a595a0 2024-04-24T09:08:11.389 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.388+0000 7f75789ca740 10 client.0 _ll_register_callbacks cb 0x563a236c6a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:08:11.390 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.388+0000 7f756e000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a23a4ece0 0x563a23a56a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:11.390 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.388+0000 7f756e000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a23a4ece0 0x563a23a56a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42102/0 (socket says 172.21.15.83:42102) 2024-04-24T09:08:11.390 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.388+0000 7f756e000640 1 -- 172.21.15.83:0/3173477754 learned_addr learned my addr 172.21.15.83:0/3173477754 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:08:11.390 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.388+0000 7f756ea00640 1 --2- 172.21.15.83:0/3173477754 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a23a4e3c0 0x563a23a564e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:11.390 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.388+0000 7f756f400640 1 --2- 172.21.15.83:0/3173477754 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563a23a595a0 0x563a23a56f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:11.390 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.389+0000 7f756e000640 1 -- 172.21.15.83:0/3173477754 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563a23a595a0 msgr2=0x563a23a56f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:11.390 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.389+0000 7f756e000640 1 --2- 172.21.15.83:0/3173477754 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563a23a595a0 0x563a23a56f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:11.390 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.389+0000 7f756e000640 1 -- 172.21.15.83:0/3173477754 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a23a4e3c0 msgr2=0x563a23a564e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:11.390 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.389+0000 7f756e000640 1 --2- 172.21.15.83:0/3173477754 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a23a4e3c0 0x563a23a564e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:11.390 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.389+0000 7f756e000640 1 -- 172.21.15.83:0/3173477754 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a2397efd0 con 0x563a23a4ece0 2024-04-24T09:08:11.390 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.389+0000 7f756f400640 1 --2- 172.21.15.83:0/3173477754 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563a23a595a0 0x563a23a56f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:08:11.390 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.389+0000 7f756e000640 1 --2- 172.21.15.83:0/3173477754 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a23a4ece0 0x563a23a56a20 secure :-1 s=READY pgs=2305 cs=0 l=1 rev1=1 crypto rx=0x7f755c003fa0 tx=0x7f755c00b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7016c72d0e83553a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:11.390 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.389+0000 7f7558c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:08:11.393 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.389+0000 7f7558c00640 1 -- 172.21.15.83:0/3173477754 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f755c01e870 con 0x563a23a4ece0 2024-04-24T09:08:11.393 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.389+0000 7f7558c00640 1 -- 172.21.15.83:0/3173477754 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9583+0+0 (secure 0 0 0) 0x7f755c01ea10 con 0x563a23a4ece0 2024-04-24T09:08:11.394 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.389+0000 7f75789ca740 1 -- 172.21.15.83:0/3173477754 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7568036030 con 0x563a23a4ece0 2024-04-24T09:08:11.394 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.390+0000 7f7558c00640 1 -- 172.21.15.83:0/3173477754 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f755c01ec10 con 0x563a23a4ece0 2024-04-24T09:08:11.394 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.390+0000 7f75789ca740 10 client.21453 fetch_fsmap learned FSMap version 55 2024-04-24T09:08:11.394 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.390+0000 7f75789ca740 1 -- 172.21.15.83:0/3173477754 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x563a236c7210 con 0x563a23a4ece0 2024-04-24T09:08:11.394 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.390+0000 7f7558c00640 1 -- 172.21.15.83:0/3173477754 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7568036030 con 0x563a23a4ece0 2024-04-24T09:08:11.394 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.391+0000 7f7558c00640 1 -- 172.21.15.83:0/3173477754 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 55) ==== 8390+0+0 (secure 0 0 0) 0x7f755c027070 con 0x563a23a4ece0 2024-04-24T09:08:11.394 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.391+0000 7f75789ca740 10 client.21453 fetch_fsmap finished waiting for FSMap version 55 2024-04-24T09:08:11.394 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.391+0000 7f75789ca740 10 client.21453 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:08:11.394 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.391+0000 7f75789ca740 20 client.21453 populate_metadata read hostname 'smithi083' 2024-04-24T09:08:11.394 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.391+0000 7f75789ca740 1 --2- 172.21.15.83:0/3173477754 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x563a23a69880 0x563a23a6bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:11.394 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.391+0000 7f75789ca740 4 client.21453 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:08:11.394 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.391+0000 7f75789ca740 1 -- 172.21.15.83:0/3173477754 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563a239c2320 con 0x563a23a69880 2024-04-24T09:08:11.394 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.391+0000 7f756ea00640 1 --2- 172.21.15.83:0/3173477754 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x563a23a69880 0x563a23a6bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:11.394 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.392+0000 7f756ea00640 1 --2- 172.21.15.83:0/3173477754 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x563a23a69880 0x563a23a6bd20 crc :-1 s=READY pgs=423 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=562971329ce8ef13 server_cookie=e296ae919ae6a51e in_seq=0 out_seq=0 2024-04-24T09:08:11.395 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.392+0000 7f7558c00640 10 client.21453 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:08:11.396 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.393+0000 7f7558c00640 1 -- 172.21.15.83:0/3173477754 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563a239c2320 con 0x563a23a69880 2024-04-24T09:08:11.396 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.393+0000 7f7558c00640 10 client.21453 handle_command_reply: tid=0 2024-04-24T09:08:11.396 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.393+0000 7f75789ca740 1 client.21453 shutdown 2024-04-24T09:08:11.396 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.393+0000 7f75789ca740 1 -- 172.21.15.83:0/3173477754 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a23a4ece0 msgr2=0x563a23a56a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:11.396 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.393+0000 7f75789ca740 1 --2- 172.21.15.83:0/3173477754 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a23a4ece0 0x563a23a56a20 secure :-1 s=READY pgs=2305 cs=0 l=1 rev1=1 crypto rx=0x7f755c003fa0 tx=0x7f755c00b390 comp rx=0 tx=0).stop 2024-04-24T09:08:11.396 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.393+0000 7f75789ca740 1 -- 172.21.15.83:0/3173477754 shutdown_connections 2024-04-24T09:08:11.396 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.393+0000 7f75789ca740 1 --2- 172.21.15.83:0/3173477754 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x563a23a69880 0x563a23a6bd20 crc :-1 s=READY pgs=423 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:11.397 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.393+0000 7f7558c00640 0 client.21453 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:08:11.397 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.393+0000 7f75789ca740 1 --2- 172.21.15.83:0/3173477754 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563a23a595a0 0x563a23a56f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:11.397 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.393+0000 7f75789ca740 1 --2- 172.21.15.83:0/3173477754 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a23a4ece0 0x563a23a56a20 unknown :-1 s=CLOSED pgs=2305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:11.397 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.393+0000 7f75789ca740 1 --2- 172.21.15.83:0/3173477754 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a23a4e3c0 0x563a23a564e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:11.397 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.393+0000 7f75789ca740 1 -- 172.21.15.83:0/3173477754 >> 172.21.15.83:0/3173477754 conn(0x563a23a4a8d0 msgr2=0x563a23a50650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:11.397 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.394+0000 7f75789ca740 1 -- 172.21.15.83:0/3173477754 shutdown_connections 2024-04-24T09:08:11.397 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.394+0000 7f75789ca740 1 -- 172.21.15.83:0/3173477754 wait complete. 2024-04-24T09:08:11.397 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.394+0000 7f75789ca740 20 client.21453 trim_cache size 0 max 16384 2024-04-24T09:08:11.421 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.421+0000 7f75789ca740 1 Processor -- start 2024-04-24T09:08:11.421 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.421+0000 7f75789ca740 1 -- start start 2024-04-24T09:08:11.423 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.422+0000 7f75789ca740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a23a4e3c0 0x563a23a58c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:11.423 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.422+0000 7f75789ca740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a23a4ece0 0x563a23a715f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:11.423 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.422+0000 7f75789ca740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563a23a595a0 0x563a23a71b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:11.423 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.422+0000 7f75789ca740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7550010110 con 0x563a23a4e3c0 2024-04-24T09:08:11.423 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.422+0000 7f75789ca740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7550011d10 con 0x563a23a4ece0 2024-04-24T09:08:11.423 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.422+0000 7f75789ca740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f75500107e0 con 0x563a23a595a0 2024-04-24T09:08:11.423 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.422+0000 7f756ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a23a4e3c0 0x563a23a58c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:11.423 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.422+0000 7f756ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a23a4e3c0 0x563a23a58c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42116/0 (socket says 172.21.15.83:42116) 2024-04-24T09:08:11.423 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.422+0000 7f756ea00640 1 -- 172.21.15.83:0/3270884807 learned_addr learned my addr 172.21.15.83:0/3270884807 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:08:11.423 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.422+0000 7f756e000640 1 --2- 172.21.15.83:0/3270884807 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a23a4ece0 0x563a23a715f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:11.423 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.423+0000 7f756f400640 1 --2- 172.21.15.83:0/3270884807 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563a23a595a0 0x563a23a71b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:11.424 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.423+0000 7f756ea00640 1 -- 172.21.15.83:0/3270884807 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563a23a595a0 msgr2=0x563a23a71b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:11.424 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.423+0000 7f756ea00640 1 --2- 172.21.15.83:0/3270884807 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563a23a595a0 0x563a23a71b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:11.424 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.423+0000 7f756ea00640 1 -- 172.21.15.83:0/3270884807 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a23a4ece0 msgr2=0x563a23a715f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:11.424 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.423+0000 7f756ea00640 1 --2- 172.21.15.83:0/3270884807 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a23a4ece0 0x563a23a715f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:11.424 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.423+0000 7f756ea00640 1 -- 172.21.15.83:0/3270884807 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a23793620 con 0x563a23a4e3c0 2024-04-24T09:08:11.424 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.423+0000 7f756e000640 1 --2- 172.21.15.83:0/3270884807 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a23a4ece0 0x563a23a715f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:08:11.424 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.423+0000 7f756ea00640 1 --2- 172.21.15.83:0/3270884807 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a23a4e3c0 0x563a23a58c10 secure :-1 s=READY pgs=2306 cs=0 l=1 rev1=1 crypto rx=0x7f7568004af0 tx=0x7f7568009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=938daf80a7c5df8a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:11.425 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.423+0000 7f755b400640 1 -- 172.21.15.83:0/3270884807 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f756805c070 con 0x563a23a4e3c0 2024-04-24T09:08:11.425 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.423+0000 7f755b400640 1 -- 172.21.15.83:0/3270884807 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7568056030 con 0x563a23a4e3c0 2024-04-24T09:08:11.425 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.423+0000 7f755b400640 1 -- 172.21.15.83:0/3270884807 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75680574e0 con 0x563a23a4e3c0 2024-04-24T09:08:11.425 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.424+0000 7f75789ca740 1 -- 172.21.15.83:0/3270884807 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a23a4e3c0 msgr2=0x563a23a58c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:11.425 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.424+0000 7f75789ca740 1 --2- 172.21.15.83:0/3270884807 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a23a4e3c0 0x563a23a58c10 secure :-1 s=READY pgs=2306 cs=0 l=1 rev1=1 crypto rx=0x7f7568004af0 tx=0x7f7568009800 comp rx=0 tx=0).stop 2024-04-24T09:08:11.425 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.425+0000 7f75789ca740 1 -- 172.21.15.83:0/3270884807 shutdown_connections 2024-04-24T09:08:11.425 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.425+0000 7f75789ca740 1 --2- 172.21.15.83:0/3270884807 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563a23a595a0 0x563a23a71b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:11.425 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.425+0000 7f75789ca740 1 --2- 172.21.15.83:0/3270884807 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a23a4ece0 0x563a23a715f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:11.426 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.425+0000 7f75789ca740 1 --2- 172.21.15.83:0/3270884807 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a23a4e3c0 0x563a23a58c10 unknown :-1 s=CLOSED pgs=2306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:11.426 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.425+0000 7f75789ca740 1 -- 172.21.15.83:0/3270884807 >> 172.21.15.83:0/3270884807 conn(0x563a23a4a8d0 msgr2=0x563a23a56d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:11.426 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.425+0000 7f75789ca740 1 -- 172.21.15.83:0/3270884807 shutdown_connections 2024-04-24T09:08:11.426 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.425+0000 7f75789ca740 1 -- 172.21.15.83:0/3270884807 wait complete. 2024-04-24T09:08:11.426 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.425+0000 7f75789ca740 1 -- start start 2024-04-24T09:08:11.427 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.425+0000 7f75789ca740 1 Processor -- start 2024-04-24T09:08:11.427 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.426+0000 7f75789ca740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563a23a4e3c0 0x563a23a61970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:11.427 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.426+0000 7f75789ca740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a23a4ece0 0x563a23a5cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:11.427 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.426+0000 7f75789ca740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a23a595a0 0x563a23a5d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:11.427 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.426+0000 7f75789ca740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7550011580 con 0x563a23a4ece0 2024-04-24T09:08:11.427 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.426+0000 7f75789ca740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7550010eb0 con 0x563a23a595a0 2024-04-24T09:08:11.427 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.426+0000 7f75789ca740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f755000e110 con 0x563a23a4e3c0 2024-04-24T09:08:11.427 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.426+0000 7f75789ca740 10 client.0 _ll_register_callbacks cb 0x7f755c00b570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:08:11.427 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.426+0000 7f756e000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a23a4ece0 0x563a23a5cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:11.427 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.426+0000 7f756e000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a23a4ece0 0x563a23a5cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42120/0 (socket says 172.21.15.83:42120) 2024-04-24T09:08:11.427 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.426+0000 7f756e000640 1 -- 172.21.15.83:0/406428695 learned_addr learned my addr 172.21.15.83:0/406428695 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:08:11.428 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.427+0000 7f756f400640 1 --2- 172.21.15.83:0/406428695 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a23a595a0 0x563a23a5d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:11.428 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.427+0000 7f756ea00640 1 --2- 172.21.15.83:0/406428695 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563a23a4e3c0 0x563a23a61970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:11.428 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.427+0000 7f756e000640 1 -- 172.21.15.83:0/406428695 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563a23a4e3c0 msgr2=0x563a23a61970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:11.428 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.427+0000 7f756e000640 1 --2- 172.21.15.83:0/406428695 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563a23a4e3c0 0x563a23a61970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:11.428 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.427+0000 7f756e000640 1 -- 172.21.15.83:0/406428695 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a23a595a0 msgr2=0x563a23a5d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:11.428 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.427+0000 7f756e000640 1 --2- 172.21.15.83:0/406428695 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a23a595a0 0x563a23a5d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:11.428 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.427+0000 7f756e000640 1 -- 172.21.15.83:0/406428695 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a23a5d9a0 con 0x563a23a4ece0 2024-04-24T09:08:11.429 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.427+0000 7f756ea00640 1 --2- 172.21.15.83:0/406428695 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563a23a4e3c0 0x563a23a61970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:08:11.429 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.427+0000 7f756e000640 1 --2- 172.21.15.83:0/406428695 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a23a4ece0 0x563a23a5cce0 secure :-1 s=READY pgs=2307 cs=0 l=1 rev1=1 crypto rx=0x7f755c01a040 tx=0x7f755c012d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69a97568c2121e55 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:11.429 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.427+0000 7f7558c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:08:11.429 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.428+0000 7f7558c00640 1 -- 172.21.15.83:0/406428695 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f755c01f990 con 0x563a23a4ece0 2024-04-24T09:08:11.430 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.428+0000 7f7558c00640 1 -- 172.21.15.83:0/406428695 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9583+0+0 (secure 0 0 0) 0x7f755c01fb30 con 0x563a23a4ece0 2024-04-24T09:08:11.430 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.428+0000 7f75789ca740 1 -- 172.21.15.83:0/406428695 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7568056030 con 0x563a23a4ece0 2024-04-24T09:08:11.430 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.428+0000 7f7558c00640 1 -- 172.21.15.83:0/406428695 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f755c01fd30 con 0x563a23a4ece0 2024-04-24T09:08:11.430 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.428+0000 7f75789ca740 10 client.21459 fetch_fsmap learned FSMap version 55 2024-04-24T09:08:11.430 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.428+0000 7f75789ca740 1 -- 172.21.15.83:0/406428695 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x563a23ae0e10 con 0x563a23a4ece0 2024-04-24T09:08:11.430 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.429+0000 7f7558c00640 1 -- 172.21.15.83:0/406428695 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7568056030 con 0x563a23a4ece0 2024-04-24T09:08:11.430 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.429+0000 7f7558c00640 1 -- 172.21.15.83:0/406428695 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 55) ==== 8390+0+0 (secure 0 0 0) 0x7f755c027040 con 0x563a23a4ece0 2024-04-24T09:08:11.430 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.429+0000 7f75789ca740 10 client.21459 fetch_fsmap finished waiting for FSMap version 55 2024-04-24T09:08:11.430 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.429+0000 7f75789ca740 10 client.21459 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:08:11.430 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.429+0000 7f75789ca740 20 client.21459 populate_metadata read hostname 'smithi083' 2024-04-24T09:08:11.430 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.429+0000 7f75789ca740 1 --2- 172.21.15.83:0/406428695 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x563a23ae15f0 0x563a23ae3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:11.430 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.429+0000 7f75789ca740 4 client.21459 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T09:08:11.430 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.429+0000 7f75789ca740 1 -- 172.21.15.83:0/406428695 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563a239bc190 con 0x563a23ae15f0 2024-04-24T09:08:11.431 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.430+0000 7f756ea00640 1 --2- 172.21.15.83:0/406428695 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x563a23ae15f0 0x563a23ae3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:11.431 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.431+0000 7f756ea00640 1 --2- 172.21.15.83:0/406428695 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x563a23ae15f0 0x563a23ae3a10 crc :-1 s=READY pgs=424 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6be076560b71edc2 server_cookie=6f1a8f72b51852c in_seq=0 out_seq=0 2024-04-24T09:08:11.431 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.431+0000 7f7558c00640 10 client.21459 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:08:11.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:08:11 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:08:11.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:08:11 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:08:11.552 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:08:11 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:08:11.558 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.558+0000 7f7558c00640 1 -- 172.21.15.83:0/406428695 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563a239bc190 con 0x563a23ae15f0 2024-04-24T09:08:11.558 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.558+0000 7f7558c00640 10 client.21459 handle_command_reply: tid=0 2024-04-24T09:08:11.558 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.558+0000 7f75789ca740 1 client.21459 shutdown 2024-04-24T09:08:11.559 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.559+0000 7f75789ca740 1 -- 172.21.15.83:0/406428695 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a23a4ece0 msgr2=0x563a23a5cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:11.559 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.559+0000 7f75789ca740 1 --2- 172.21.15.83:0/406428695 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a23a4ece0 0x563a23a5cce0 secure :-1 s=READY pgs=2307 cs=0 l=1 rev1=1 crypto rx=0x7f755c01a040 tx=0x7f755c012d30 comp rx=0 tx=0).stop 2024-04-24T09:08:11.559 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.559+0000 7f75789ca740 1 -- 172.21.15.83:0/406428695 shutdown_connections 2024-04-24T09:08:11.559 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.559+0000 7f75789ca740 1 --2- 172.21.15.83:0/406428695 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x563a23ae15f0 0x563a23ae3a10 crc :-1 s=READY pgs=424 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:11.559 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.559+0000 7f75789ca740 1 --2- 172.21.15.83:0/406428695 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a23a595a0 0x563a23a5d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:11.559 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.559+0000 7f75789ca740 1 --2- 172.21.15.83:0/406428695 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a23a4ece0 0x563a23a5cce0 unknown :-1 s=CLOSED pgs=2307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:11.559 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.559+0000 7f75789ca740 1 --2- 172.21.15.83:0/406428695 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563a23a4e3c0 0x563a23a61970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:11.559 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.559+0000 7f75789ca740 1 -- 172.21.15.83:0/406428695 >> 172.21.15.83:0/406428695 conn(0x563a23a4a8d0 msgr2=0x563a23a564a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:11.560 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.559+0000 7f7558c00640 0 client.21459 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:08:11.560 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.559+0000 7f75789ca740 1 -- 172.21.15.83:0/406428695 shutdown_connections 2024-04-24T09:08:11.560 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.559+0000 7f75789ca740 1 -- 172.21.15.83:0/406428695 wait complete. 2024-04-24T09:08:11.560 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.559+0000 7f75789ca740 20 client.21459 trim_cache size 0 max 16384 2024-04-24T09:08:11.560 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T09:08:11.560 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T09:08:11.561 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T09:08:11.561 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T09:08:11.561 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.561+0000 7f756fe00640 1 -- 172.21.15.83:0/1552020503 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f75540a3d90 msgr2=0x7f75540a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:11.561 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.561+0000 7f756fe00640 1 --2- 172.21.15.83:0/1552020503 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f75540a3d90 0x7f75540a6250 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x563a23a4bcf0 tx=0x7f755c005c90 comp rx=0 tx=0).stop 2024-04-24T09:08:11.561 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.561+0000 7f756fe00640 1 -- 172.21.15.83:0/1552020503 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f757010bb70 msgr2=0x7f757019a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:11.562 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.561+0000 7f756fe00640 1 --2- 172.21.15.83:0/1552020503 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f757010bb70 0x7f757019a720 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f756000d950 tx=0x7f756000de10 comp rx=0 tx=0).stop 2024-04-24T09:08:11.562 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.561+0000 7f756fe00640 1 -- 172.21.15.83:0/1552020503 shutdown_connections 2024-04-24T09:08:11.562 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.561+0000 7f756fe00640 1 --2- 172.21.15.83:0/1552020503 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f75540a3d90 0x7f75540a6250 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:11.562 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.561+0000 7f756fe00640 1 --2- 172.21.15.83:0/1552020503 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f757010bb70 0x7f757019a720 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:11.562 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.561+0000 7f756fe00640 1 --2- 172.21.15.83:0/1552020503 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7570106030 0x7f75701a0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:11.562 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.561+0000 7f756fe00640 1 --2- 172.21.15.83:0/1552020503 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7570100f60 0x7f75701a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:11.562 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.561+0000 7f756fe00640 1 -- 172.21.15.83:0/1552020503 >> 172.21.15.83:0/1552020503 conn(0x7f75700fc820 msgr2=0x7f75700fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:11.562 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.561+0000 7f756fe00640 1 -- 172.21.15.83:0/1552020503 shutdown_connections 2024-04-24T09:08:11.562 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:11.561+0000 7f756fe00640 1 -- 172.21.15.83:0/1552020503 wait complete. 2024-04-24T09:08:12.164 INFO:tasks.workunit.client.0.smithi083.stdout: 10 21040 7.02 MB/sec execute 43 sec latency 129.233 ms 2024-04-24T09:08:12.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:08:12 smithi138 ceph-mon[31441]: pgmap v522: 129 pgs: 129 active+clean; 465 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 894 KiB/s rd, 2.3 MiB/s wr, 132 op/s 2024-04-24T09:08:12.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:08:12 smithi138 ceph-mon[31441]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:08:12.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:08:12 smithi189 ceph-mon[31791]: pgmap v522: 129 pgs: 129 active+clean; 465 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 894 KiB/s rd, 2.3 MiB/s wr, 132 op/s 2024-04-24T09:08:12.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:08:12 smithi189 ceph-mon[31791]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:08:12.552 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:08:12 smithi083 ceph-mon[27907]: pgmap v522: 129 pgs: 129 active+clean; 465 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 894 KiB/s rd, 2.3 MiB/s wr, 132 op/s 2024-04-24T09:08:12.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:08:12 smithi083 ceph-mon[27907]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:08:13.165 INFO:tasks.workunit.client.0.smithi083.stdout: 10 21185 7.05 MB/sec execute 44 sec latency 79.997 ms 2024-04-24T09:08:14.165 INFO:tasks.workunit.client.0.smithi083.stdout: 10 21336 7.14 MB/sec execute 45 sec latency 83.587 ms 2024-04-24T09:08:14.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:08:14 smithi138 ceph-mon[31441]: pgmap v523: 129 pgs: 129 active+clean; 471 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.1 MiB/s wr, 346 op/s 2024-04-24T09:08:14.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:08:14 smithi189 ceph-mon[31791]: pgmap v523: 129 pgs: 129 active+clean; 471 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.1 MiB/s wr, 346 op/s 2024-04-24T09:08:14.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:08:14 smithi083 ceph-mon[27907]: pgmap v523: 129 pgs: 129 active+clean; 471 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.1 MiB/s wr, 346 op/s 2024-04-24T09:08:15.165 INFO:tasks.workunit.client.0.smithi083.stdout: 10 21407 7.02 MB/sec execute 46 sec latency 426.687 ms 2024-04-24T09:08:16.165 INFO:tasks.workunit.client.0.smithi083.stdout: 10 21515 6.90 MB/sec execute 47 sec latency 93.174 ms 2024-04-24T09:08:16.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:08:16 smithi138 ceph-mon[31441]: pgmap v524: 129 pgs: 129 active+clean; 471 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.0 MiB/s wr, 338 op/s 2024-04-24T09:08:16.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:08:16 smithi189 ceph-mon[31791]: pgmap v524: 129 pgs: 129 active+clean; 471 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.0 MiB/s wr, 338 op/s 2024-04-24T09:08:16.552 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:08:16 smithi083 ceph-mon[27907]: pgmap v524: 129 pgs: 129 active+clean; 471 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.0 MiB/s wr, 338 op/s 2024-04-24T09:08:17.165 INFO:tasks.workunit.client.0.smithi083.stdout: 10 21650 6.84 MB/sec execute 48 sec latency 74.902 ms 2024-04-24T09:08:18.165 INFO:tasks.workunit.client.0.smithi083.stdout: 10 21774 6.85 MB/sec execute 49 sec latency 95.317 ms 2024-04-24T09:08:18.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:08:18 smithi138 ceph-mon[31441]: pgmap v525: 129 pgs: 129 active+clean; 471 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.0 MiB/s wr, 338 op/s 2024-04-24T09:08:18.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:08:18 smithi138 ceph-mon[31441]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:08:18.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:08:18 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:08:18.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:08:18 smithi189 ceph-mon[31791]: pgmap v525: 129 pgs: 129 active+clean; 471 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.0 MiB/s wr, 338 op/s 2024-04-24T09:08:18.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:08:18 smithi189 ceph-mon[31791]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:08:18.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:08:18 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:08:18.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:08:18 smithi083 ceph-mon[27907]: pgmap v525: 129 pgs: 129 active+clean; 471 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.0 MiB/s wr, 338 op/s 2024-04-24T09:08:18.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:08:18 smithi083 ceph-mon[27907]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:08:18.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:08:18 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:08:19.165 INFO:tasks.workunit.client.0.smithi083.stdout: 10 21901 6.80 MB/sec execute 50 sec latency 88.158 ms 2024-04-24T09:08:20.165 INFO:tasks.workunit.client.0.smithi083.stdout: 10 21995 6.80 MB/sec execute 51 sec latency 453.515 ms 2024-04-24T09:08:20.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:08:20 smithi138 ceph-mon[31441]: pgmap v526: 129 pgs: 129 active+clean; 479 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.5 MiB/s wr, 352 op/s 2024-04-24T09:08:20.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:08:20 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:08:20.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:08:20 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:08:20.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:08:20 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:08:20.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:08:20 smithi189 ceph-mon[31791]: pgmap v526: 129 pgs: 129 active+clean; 479 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.5 MiB/s wr, 352 op/s 2024-04-24T09:08:20.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:08:20 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:08:20.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:08:20 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:08:20.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:08:20 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:08:20.552 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:08:20 smithi083 ceph-mon[27907]: pgmap v526: 129 pgs: 129 active+clean; 479 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.5 MiB/s wr, 352 op/s 2024-04-24T09:08:20.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:08:20 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:08:20.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:08:20 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:08:20.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:08:20 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:08:21.165 INFO:tasks.workunit.client.0.smithi083.stdout: 10 22124 6.82 MB/sec execute 52 sec latency 67.707 ms 2024-04-24T09:08:22.165 INFO:tasks.workunit.client.0.smithi083.stdout: 10 22246 6.78 MB/sec execute 53 sec latency 69.300 ms 2024-04-24T09:08:22.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:08:22 smithi138 ceph-mon[31441]: pgmap v527: 129 pgs: 129 active+clean; 479 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.1 MiB/s wr, 315 op/s 2024-04-24T09:08:22.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:08:22 smithi189 ceph-mon[31791]: pgmap v527: 129 pgs: 129 active+clean; 479 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.1 MiB/s wr, 315 op/s 2024-04-24T09:08:22.552 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:08:22 smithi083 ceph-mon[27907]: pgmap v527: 129 pgs: 129 active+clean; 479 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.1 MiB/s wr, 315 op/s 2024-04-24T09:08:23.161 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:08:23 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:08:23.162 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:08:23 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:08:23.162 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:08:23 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:08:23.165 INFO:tasks.workunit.client.0.smithi083.stdout: 10 22407 6.99 MB/sec execute 54 sec latency 65.437 ms 2024-04-24T09:08:23.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:08:23 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:08:23.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:08:23 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:08:23.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:08:23 smithi138 ceph-mon[31441]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:08:23.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:08:23 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:08:23.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:08:23 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:08:23.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:08:23 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:08:23.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:08:23 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:08:23.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:08:23 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:08:23.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:08:23 smithi189 ceph-mon[31791]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:08:23.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:08:23 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:08:23.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:08:23 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:08:23.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:08:23 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:08:23.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:08:23 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:08:23.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:08:23 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:08:23.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:08:23 smithi083 ceph-mon[27907]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:08:24.166 INFO:tasks.workunit.client.0.smithi083.stdout: 10 22580 7.08 MB/sec execute 55 sec latency 64.501 ms 2024-04-24T09:08:24.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:08:24 smithi138 ceph-mon[31441]: pgmap v528: 129 pgs: 129 active+clean; 488 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.7 MiB/s wr, 391 op/s 2024-04-24T09:08:24.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:08:24 smithi189 ceph-mon[31791]: pgmap v528: 129 pgs: 129 active+clean; 488 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.7 MiB/s wr, 391 op/s 2024-04-24T09:08:24.552 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:08:24 smithi083 ceph-mon[27907]: pgmap v528: 129 pgs: 129 active+clean; 488 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.7 MiB/s wr, 391 op/s 2024-04-24T09:08:24.746 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.746+0000 7f6af9600640 1 -- 172.21.15.83:0/4189989466 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6af4104450 msgr2=0x7f6af4104830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:24.746 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.746+0000 7f6af9600640 1 --2- 172.21.15.83:0/4189989466 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6af4104450 0x7f6af4104830 secure :-1 s=READY pgs=2308 cs=0 l=1 rev1=1 crypto rx=0x7f6ae4009900 tx=0x7f6ae402a410 comp rx=0 tx=0).stop 2024-04-24T09:08:24.747 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.746+0000 7f6af9600640 1 -- 172.21.15.83:0/4189989466 shutdown_connections 2024-04-24T09:08:24.747 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.746+0000 7f6af9600640 1 --2- 172.21.15.83:0/4189989466 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6af4109520 0x7f6af41110f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:24.747 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.746+0000 7f6af9600640 1 --2- 172.21.15.83:0/4189989466 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6af4104e00 0x7f6af4108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:24.747 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.746+0000 7f6af9600640 1 --2- 172.21.15.83:0/4189989466 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6af4104450 0x7f6af4104830 unknown :-1 s=CLOSED pgs=2308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:24.747 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.746+0000 7f6af9600640 1 -- 172.21.15.83:0/4189989466 >> 172.21.15.83:0/4189989466 conn(0x7f6af40ffcf0 msgr2=0x7f6af4102110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:24.747 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.746+0000 7f6af9600640 1 -- 172.21.15.83:0/4189989466 shutdown_connections 2024-04-24T09:08:24.747 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.747+0000 7f6af9600640 1 -- 172.21.15.83:0/4189989466 wait complete. 2024-04-24T09:08:24.748 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.748+0000 7f6af9600640 1 Processor -- start 2024-04-24T09:08:24.750 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.748+0000 7f6af9600640 1 -- start start 2024-04-24T09:08:24.750 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.748+0000 7f6af9600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6af4104450 0x7f6af4076510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:24.750 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.749+0000 7f6af9600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6af4109520 0x7f6af4076a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:24.750 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.749+0000 7f6af9600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6af4076f90 0x7f6af41aaff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:24.750 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.749+0000 7f6af9600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6af41144e0 con 0x7f6af4076f90 2024-04-24T09:08:24.750 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.749+0000 7f6af9600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6af4114360 con 0x7f6af4109520 2024-04-24T09:08:24.750 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.749+0000 7f6af9600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6af4114660 con 0x7f6af4104450 2024-04-24T09:08:24.750 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.749+0000 7f6af2a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6af4104450 0x7f6af4076510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:24.750 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.749+0000 7f6af2a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6af4104450 0x7f6af4076510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:45770/0 (socket says 172.21.15.83:45770) 2024-04-24T09:08:24.750 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.749+0000 7f6af2a00640 1 -- 172.21.15.83:0/3426098797 learned_addr learned my addr 172.21.15.83:0/3426098797 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:08:24.750 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.749+0000 7f6af2000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6af4109520 0x7f6af4076a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:24.751 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.750+0000 7f6af3400640 1 --2- 172.21.15.83:0/3426098797 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6af4076f90 0x7f6af41aaff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:24.751 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.750+0000 7f6af2a00640 1 -- 172.21.15.83:0/3426098797 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6af4109520 msgr2=0x7f6af4076a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:24.752 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.750+0000 7f6af2a00640 1 --2- 172.21.15.83:0/3426098797 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6af4109520 0x7f6af4076a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:24.752 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.750+0000 7f6af2a00640 1 -- 172.21.15.83:0/3426098797 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6af4076f90 msgr2=0x7f6af41aaff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:24.752 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.750+0000 7f6af2a00640 1 --2- 172.21.15.83:0/3426098797 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6af4076f90 0x7f6af41aaff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:24.752 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.750+0000 7f6af2a00640 1 -- 172.21.15.83:0/3426098797 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6af419a790 con 0x7f6af4104450 2024-04-24T09:08:24.752 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.751+0000 7f6af2a00640 1 --2- 172.21.15.83:0/3426098797 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6af4104450 0x7f6af4076510 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6ae402aa00 tx=0x7f6ae4003c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:24.752 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.752+0000 7f6aeb400640 1 -- 172.21.15.83:0/3426098797 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae403f070 con 0x7f6af4104450 2024-04-24T09:08:24.753 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.752+0000 7f6af9600640 1 -- 172.21.15.83:0/3426098797 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6af419aa20 con 0x7f6af4104450 2024-04-24T09:08:24.753 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.752+0000 7f6af9600640 1 -- 172.21.15.83:0/3426098797 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6af419af60 con 0x7f6af4104450 2024-04-24T09:08:24.753 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.753+0000 7f6aeb400640 1 -- 172.21.15.83:0/3426098797 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6ae4031b50 con 0x7f6af4104450 2024-04-24T09:08:24.753 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.753+0000 7f6aeb400640 1 -- 172.21.15.83:0/3426098797 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae4035030 con 0x7f6af4104450 2024-04-24T09:08:24.757 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.757+0000 7f6afc568740 1 Processor -- start 2024-04-24T09:08:24.758 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.758+0000 7f6afc568740 1 -- start start 2024-04-24T09:08:24.759 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.758+0000 7f6afc568740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5564c59811b0 0x5564c5981590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:24.759 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.758+0000 7f6afc568740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5564c5981ad0 0x5564c598be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:24.759 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.758+0000 7f6afc568740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5564c598c390 0x5564c598e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:24.759 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.758+0000 7f6afc568740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5564c58e7900 con 0x5564c5981ad0 2024-04-24T09:08:24.759 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.758+0000 7f6afc568740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5564c588df20 con 0x5564c598c390 2024-04-24T09:08:24.759 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.758+0000 7f6afc568740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5564c58e38c0 con 0x5564c59811b0 2024-04-24T09:08:24.759 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.759+0000 7f6af3400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5564c59811b0 0x5564c5981590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:24.759 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.759+0000 7f6af3400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5564c59811b0 0x5564c5981590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:45780/0 (socket says 172.21.15.83:45780) 2024-04-24T09:08:24.760 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.759+0000 7f6af3400640 1 -- 172.21.15.83:0/1531481046 learned_addr learned my addr 172.21.15.83:0/1531481046 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:08:24.760 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.760+0000 7f6af3400640 1 -- 172.21.15.83:0/1531481046 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5564c598c390 msgr2=0x5564c598e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:24.760 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.760+0000 7f6af3400640 1 --2- 172.21.15.83:0/1531481046 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5564c598c390 0x5564c598e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:24.760 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.760+0000 7f6af3400640 1 -- 172.21.15.83:0/1531481046 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5564c5981ad0 msgr2=0x5564c598be50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T09:08:24.760 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.760+0000 7f6af3400640 1 --2- 172.21.15.83:0/1531481046 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5564c5981ad0 0x5564c598be50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:24.760 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.760+0000 7f6af3400640 1 -- 172.21.15.83:0/1531481046 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564c58b2e10 con 0x5564c59811b0 2024-04-24T09:08:24.761 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.760+0000 7f6af3400640 1 --2- 172.21.15.83:0/1531481046 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5564c59811b0 0x5564c5981590 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6aec009f70 tx=0x7f6aec010800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=586b9ba79fc357dc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:24.761 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.761+0000 7f6aeb400640 1 -- 172.21.15.83:0/3426098797 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6ae4042070 con 0x7f6af4104450 2024-04-24T09:08:24.761 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.761+0000 7f6adf400640 1 -- 172.21.15.83:0/1531481046 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aec016070 con 0x5564c59811b0 2024-04-24T09:08:24.762 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.761+0000 7f6adf400640 1 -- 172.21.15.83:0/1531481046 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6aec004d20 con 0x5564c59811b0 2024-04-24T09:08:24.762 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.761+0000 7f6adf400640 1 -- 172.21.15.83:0/1531481046 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aec0127d0 con 0x5564c59811b0 2024-04-24T09:08:24.763 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.763+0000 7f6afc568740 1 -- 172.21.15.83:0/1531481046 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5564c59811b0 msgr2=0x5564c5981590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:24.763 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.763+0000 7f6afc568740 1 --2- 172.21.15.83:0/1531481046 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5564c59811b0 0x5564c5981590 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6aec009f70 tx=0x7f6aec010800 comp rx=0 tx=0).stop 2024-04-24T09:08:24.763 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.763+0000 7f6afc568740 1 -- 172.21.15.83:0/1531481046 shutdown_connections 2024-04-24T09:08:24.764 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.763+0000 7f6afc568740 1 --2- 172.21.15.83:0/1531481046 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5564c598c390 0x5564c598e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:24.764 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.763+0000 7f6afc568740 1 --2- 172.21.15.83:0/1531481046 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5564c5981ad0 0x5564c598be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:24.764 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.763+0000 7f6afc568740 1 --2- 172.21.15.83:0/1531481046 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5564c59811b0 0x5564c5981590 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:24.764 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.763+0000 7f6afc568740 1 -- 172.21.15.83:0/1531481046 >> 172.21.15.83:0/1531481046 conn(0x5564c597d6c0 msgr2=0x5564c597daa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:24.764 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.763+0000 7f6aeb400640 1 --2- 172.21.15.83:0/3426098797 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f6ad80a3e40 0x7f6ad80a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:24.765 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.764+0000 7f6aeb400640 1 -- 172.21.15.83:0/3426098797 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 9583+0+0 (secure 0 0 0) 0x7f6ae40f11a0 con 0x7f6af4104450 2024-04-24T09:08:24.765 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.764+0000 7f6afc568740 1 -- 172.21.15.83:0/1531481046 shutdown_connections 2024-04-24T09:08:24.765 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.764+0000 7f6af2000640 1 --2- 172.21.15.83:0/3426098797 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f6ad80a3e40 0x7f6ad80a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:24.765 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.764+0000 7f6afc568740 1 -- 172.21.15.83:0/1531481046 wait complete. 2024-04-24T09:08:24.765 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.765+0000 7f6afc568740 1 -- start start 2024-04-24T09:08:24.765 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.765+0000 7f6af2000640 1 --2- 172.21.15.83:0/3426098797 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f6ad80a3e40 0x7f6ad80a6300 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x5564c597eae0 tx=0x7f6ae00046c0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:24.765 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.765+0000 7f6afc568740 1 Processor -- start 2024-04-24T09:08:24.766 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.766+0000 7f6afc568740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5564c59811b0 0x5564c5989370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:24.767 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.766+0000 7f6afc568740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5564c5981ad0 0x5564c59898b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:24.767 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.766+0000 7f6afc568740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5564c598c390 0x5564c5989df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:24.767 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.766+0000 7f6afc568740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5564c598efc0 con 0x5564c5981ad0 2024-04-24T09:08:24.767 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.766+0000 7f6afc568740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5564c58a12c0 con 0x5564c59811b0 2024-04-24T09:08:24.767 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.766+0000 7f6afc568740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5564c598f140 con 0x5564c598c390 2024-04-24T09:08:24.767 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.766+0000 7f6afc568740 10 client.0 _ll_register_callbacks cb 0x5564c55f9a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:08:24.768 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.767+0000 7f6af3400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5564c598c390 0x5564c5989df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:24.768 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.767+0000 7f6af3400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5564c598c390 0x5564c5989df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:45790/0 (socket says 172.21.15.83:45790) 2024-04-24T09:08:24.768 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.767+0000 7f6af3400640 1 -- 172.21.15.83:0/1475763353 learned_addr learned my addr 172.21.15.83:0/1475763353 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:08:24.768 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.767+0000 7f6af2a00640 1 --2- 172.21.15.83:0/1475763353 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5564c59811b0 0x5564c5989370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:24.768 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.767+0000 7f6af2000640 1 --2- 172.21.15.83:0/1475763353 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5564c5981ad0 0x5564c59898b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:24.768 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.768+0000 7f6af3400640 1 -- 172.21.15.83:0/1475763353 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5564c59811b0 msgr2=0x5564c5989370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:24.768 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.768+0000 7f6af3400640 1 --2- 172.21.15.83:0/1475763353 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5564c59811b0 0x5564c5989370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:24.768 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.768+0000 7f6af3400640 1 -- 172.21.15.83:0/1475763353 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5564c5981ad0 msgr2=0x5564c59898b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:24.768 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.768+0000 7f6af3400640 1 --2- 172.21.15.83:0/1475763353 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5564c5981ad0 0x5564c59898b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:24.768 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.768+0000 7f6af3400640 1 -- 172.21.15.83:0/1475763353 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564c58b1fd0 con 0x5564c598c390 2024-04-24T09:08:24.769 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.768+0000 7f6af3400640 1 --2- 172.21.15.83:0/1475763353 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5564c598c390 0x5564c5989df0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6aec004cf0 tx=0x7f6aec005000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a55f0ab1700b756 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:24.769 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.768+0000 7f6ad3e00640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-04-24T09:08:24.769 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.769+0000 7f6ad3e00640 1 -- 172.21.15.83:0/1475763353 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aec029060 con 0x5564c598c390 2024-04-24T09:08:24.770 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.769+0000 7f6afc568740 1 -- 172.21.15.83:0/1475763353 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6aec004d20 con 0x5564c598c390 2024-04-24T09:08:24.770 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.769+0000 7f6ad3e00640 1 -- 172.21.15.83:0/1475763353 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9583+0+0 (secure 0 0 0) 0x7f6aec012530 con 0x5564c598c390 2024-04-24T09:08:24.770 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.769+0000 7f6ad3e00640 1 -- 172.21.15.83:0/1475763353 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f6aec030050 con 0x5564c598c390 2024-04-24T09:08:24.770 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.769+0000 7f6afc568740 10 client.28618 fetch_fsmap learned FSMap version 57 2024-04-24T09:08:24.770 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.769+0000 7f6afc568740 1 -- 172.21.15.83:0/1475763353 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x5564c55fa210 con 0x5564c598c390 2024-04-24T09:08:24.770 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.770+0000 7f6ad3e00640 1 -- 172.21.15.83:0/1475763353 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6aec004d20 con 0x5564c598c390 2024-04-24T09:08:24.771 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.771+0000 7f6ad3e00640 1 -- 172.21.15.83:0/1475763353 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 57) ==== 8382+0+0 (secure 0 0 0) 0x7f6aec00f040 con 0x5564c598c390 2024-04-24T09:08:24.771 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.771+0000 7f6afc568740 10 client.28618 fetch_fsmap finished waiting for FSMap version 57 2024-04-24T09:08:24.771 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.771+0000 7f6afc568740 10 client.28618 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:08:24.771 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.771+0000 7f6afc568740 20 client.28618 populate_metadata read hostname 'smithi083' 2024-04-24T09:08:24.771 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.771+0000 7f6afc568740 1 --2- 172.21.15.83:0/1475763353 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5564c599c550 0x5564c599e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:24.771 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.771+0000 7f6afc568740 4 client.28618 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:08:24.771 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.771+0000 7f6afc568740 1 -- 172.21.15.83:0/1475763353 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5564c58f5320 con 0x5564c599c550 2024-04-24T09:08:24.772 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.771+0000 7f6af2a00640 1 --2- 172.21.15.83:0/1475763353 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5564c599c550 0x5564c599e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:24.772 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.772+0000 7f6af2a00640 1 --2- 172.21.15.83:0/1475763353 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5564c599c550 0x5564c599e940 crc :-1 s=READY pgs=425 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30795aec1c9d4b3 server_cookie=1e8a39640f536588 in_seq=0 out_seq=0 2024-04-24T09:08:24.772 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.772+0000 7f6ad3e00640 10 client.28618 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:08:24.772 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.772+0000 7f6ad3e00640 1 -- 172.21.15.83:0/1475763353 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5564c58f5320 con 0x5564c599c550 2024-04-24T09:08:24.772 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.772+0000 7f6ad3e00640 10 client.28618 handle_command_reply: tid=0 2024-04-24T09:08:24.772 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.773+0000 7f6afc568740 1 client.28618 shutdown 2024-04-24T09:08:24.773 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.773+0000 7f6afc568740 1 -- 172.21.15.83:0/1475763353 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5564c598c390 msgr2=0x5564c5989df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:24.773 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.773+0000 7f6afc568740 1 --2- 172.21.15.83:0/1475763353 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5564c598c390 0x5564c5989df0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6aec004cf0 tx=0x7f6aec005000 comp rx=0 tx=0).stop 2024-04-24T09:08:24.773 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.773+0000 7f6afc568740 1 -- 172.21.15.83:0/1475763353 shutdown_connections 2024-04-24T09:08:24.774 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.773+0000 7f6afc568740 1 --2- 172.21.15.83:0/1475763353 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5564c599c550 0x5564c599e940 crc :-1 s=READY pgs=425 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:24.774 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.773+0000 7f6afc568740 1 --2- 172.21.15.83:0/1475763353 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5564c598c390 0x5564c5989df0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:24.774 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.773+0000 7f6afc568740 1 --2- 172.21.15.83:0/1475763353 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5564c5981ad0 0x5564c59898b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:24.774 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.773+0000 7f6ad3e00640 0 client.28618 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:08:24.774 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.773+0000 7f6afc568740 1 --2- 172.21.15.83:0/1475763353 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5564c59811b0 0x5564c5989370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:24.774 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.773+0000 7f6afc568740 1 -- 172.21.15.83:0/1475763353 >> 172.21.15.83:0/1475763353 conn(0x5564c597d6c0 msgr2=0x5564c5983440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:24.774 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.773+0000 7f6afc568740 1 -- 172.21.15.83:0/1475763353 shutdown_connections 2024-04-24T09:08:24.774 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.773+0000 7f6afc568740 1 -- 172.21.15.83:0/1475763353 wait complete. 2024-04-24T09:08:24.774 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.774+0000 7f6afc568740 20 client.28618 trim_cache size 0 max 16384 2024-04-24T09:08:24.799 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.799+0000 7f6afc568740 1 Processor -- start 2024-04-24T09:08:24.800 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.799+0000 7f6afc568740 1 -- start start 2024-04-24T09:08:24.800 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.799+0000 7f6afc568740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5564c59811b0 0x5564c5997ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:24.800 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.799+0000 7f6afc568740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5564c5981ad0 0x5564c5983b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:24.800 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.799+0000 7f6afc568740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5564c598c390 0x5564c59a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:24.800 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.799+0000 7f6afc568740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad4010110 con 0x5564c598c390 2024-04-24T09:08:24.800 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.800+0000 7f6afc568740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad4011cb0 con 0x5564c59811b0 2024-04-24T09:08:24.800 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.800+0000 7f6afc568740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad40107e0 con 0x5564c5981ad0 2024-04-24T09:08:24.800 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.800+0000 7f6af3400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5564c598c390 0x5564c59a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:24.800 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.800+0000 7f6af3400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5564c598c390 0x5564c59a3f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38814/0 (socket says 172.21.15.83:38814) 2024-04-24T09:08:24.800 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.800+0000 7f6af3400640 1 -- 172.21.15.83:0/2843821366 learned_addr learned my addr 172.21.15.83:0/2843821366 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:08:24.800 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.800+0000 7f6af2a00640 1 --2- 172.21.15.83:0/2843821366 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5564c59811b0 0x5564c5997ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:24.801 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.800+0000 7f6af2000640 1 --2- 172.21.15.83:0/2843821366 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5564c5981ad0 0x5564c5983b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:24.801 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.800+0000 7f6af3400640 1 -- 172.21.15.83:0/2843821366 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5564c5981ad0 msgr2=0x5564c5983b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:24.802 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.800+0000 7f6af3400640 1 --2- 172.21.15.83:0/2843821366 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5564c5981ad0 0x5564c5983b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:24.802 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.800+0000 7f6af3400640 1 -- 172.21.15.83:0/2843821366 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5564c59811b0 msgr2=0x5564c5997ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:24.802 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.800+0000 7f6af3400640 1 --2- 172.21.15.83:0/2843821366 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5564c59811b0 0x5564c5997ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:24.802 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.800+0000 7f6af3400640 1 -- 172.21.15.83:0/2843821366 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564c56c6620 con 0x5564c598c390 2024-04-24T09:08:24.802 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.801+0000 7f6af3400640 1 --2- 172.21.15.83:0/2843821366 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5564c598c390 0x5564c59a3f70 secure :-1 s=READY pgs=2309 cs=0 l=1 rev1=1 crypto rx=0x7f6aec009d30 tx=0x7f6aec009820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3f0699b898bdf36 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:24.802 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.801+0000 7f6adea00640 1 -- 172.21.15.83:0/2843821366 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aec039070 con 0x5564c598c390 2024-04-24T09:08:24.802 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.801+0000 7f6adea00640 1 -- 172.21.15.83:0/2843821366 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6aec005070 con 0x5564c598c390 2024-04-24T09:08:24.802 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.801+0000 7f6adea00640 1 -- 172.21.15.83:0/2843821366 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aec029040 con 0x5564c598c390 2024-04-24T09:08:24.803 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.802+0000 7f6afc568740 1 -- 172.21.15.83:0/2843821366 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5564c598c390 msgr2=0x5564c59a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:24.803 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.802+0000 7f6afc568740 1 --2- 172.21.15.83:0/2843821366 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5564c598c390 0x5564c59a3f70 secure :-1 s=READY pgs=2309 cs=0 l=1 rev1=1 crypto rx=0x7f6aec009d30 tx=0x7f6aec009820 comp rx=0 tx=0).stop 2024-04-24T09:08:24.803 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.802+0000 7f6afc568740 1 -- 172.21.15.83:0/2843821366 shutdown_connections 2024-04-24T09:08:24.803 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.802+0000 7f6afc568740 1 --2- 172.21.15.83:0/2843821366 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5564c598c390 0x5564c59a3f70 unknown :-1 s=CLOSED pgs=2309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:24.803 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.802+0000 7f6afc568740 1 --2- 172.21.15.83:0/2843821366 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5564c5981ad0 0x5564c5983b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:24.803 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.802+0000 7f6afc568740 1 --2- 172.21.15.83:0/2843821366 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5564c59811b0 0x5564c5997ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:24.803 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.802+0000 7f6afc568740 1 -- 172.21.15.83:0/2843821366 >> 172.21.15.83:0/2843821366 conn(0x5564c597d6c0 msgr2=0x5564c597c840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:24.803 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.803+0000 7f6afc568740 1 -- 172.21.15.83:0/2843821366 shutdown_connections 2024-04-24T09:08:24.803 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.803+0000 7f6afc568740 1 -- 172.21.15.83:0/2843821366 wait complete. 2024-04-24T09:08:24.804 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.803+0000 7f6afc568740 1 -- start start 2024-04-24T09:08:24.804 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.804+0000 7f6afc568740 1 Processor -- start 2024-04-24T09:08:24.804 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.804+0000 7f6afc568740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5564c59811b0 0x5564c59fe170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:24.805 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.804+0000 7f6afc568740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5564c5981ad0 0x5564c59943f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:24.805 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.804+0000 7f6afc568740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5564c598c390 0x5564c5994930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:24.805 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.804+0000 7f6afc568740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad4011580 con 0x5564c5981ad0 2024-04-24T09:08:24.805 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.804+0000 7f6afc568740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad4010eb0 con 0x5564c598c390 2024-04-24T09:08:24.805 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.804+0000 7f6afc568740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad400e110 con 0x5564c59811b0 2024-04-24T09:08:24.805 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.804+0000 7f6afc568740 10 client.0 _ll_register_callbacks cb 0x7f6aec009df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:08:24.806 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.805+0000 7f6af2000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5564c5981ad0 0x5564c59943f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:24.806 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.805+0000 7f6af3400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5564c598c390 0x5564c5994930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:24.806 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.805+0000 7f6af2000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5564c5981ad0 0x5564c59943f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38826/0 (socket says 172.21.15.83:38826) 2024-04-24T09:08:24.806 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.805+0000 7f6af2000640 1 -- 172.21.15.83:0/22202556 learned_addr learned my addr 172.21.15.83:0/22202556 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:08:24.806 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.805+0000 7f6af2a00640 1 --2- 172.21.15.83:0/22202556 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5564c59811b0 0x5564c59fe170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:24.806 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.805+0000 7f6af3400640 1 -- 172.21.15.83:0/22202556 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5564c59811b0 msgr2=0x5564c59fe170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:24.807 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.805+0000 7f6af3400640 1 --2- 172.21.15.83:0/22202556 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5564c59811b0 0x5564c59fe170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:24.807 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.805+0000 7f6af3400640 1 -- 172.21.15.83:0/22202556 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5564c5981ad0 msgr2=0x5564c59943f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:24.807 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.805+0000 7f6af3400640 1 --2- 172.21.15.83:0/22202556 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5564c5981ad0 0x5564c59943f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:24.807 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.806+0000 7f6af3400640 1 -- 172.21.15.83:0/22202556 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564c59865c0 con 0x5564c598c390 2024-04-24T09:08:24.807 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.806+0000 7f6af3400640 1 --2- 172.21.15.83:0/22202556 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5564c598c390 0x5564c5994930 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6aec01d010 tx=0x7f6aec01a3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71468e17d91fe99 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:24.807 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.806+0000 7f6ad3e00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:08:24.808 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.806+0000 7f6ad3e00640 1 -- 172.21.15.83:0/22202556 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aec041060 con 0x5564c598c390 2024-04-24T09:08:24.808 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.806+0000 7f6afc568740 1 -- 172.21.15.83:0/22202556 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6aec005070 con 0x5564c598c390 2024-04-24T09:08:24.808 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.806+0000 7f6ad3e00640 1 -- 172.21.15.83:0/22202556 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9583+0+0 (secure 0 0 0) 0x7f6aec024670 con 0x5564c598c390 2024-04-24T09:08:24.808 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.807+0000 7f6ad3e00640 1 -- 172.21.15.83:0/22202556 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f6aec039070 con 0x5564c598c390 2024-04-24T09:08:24.808 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.807+0000 7f6afc568740 10 client.38618 fetch_fsmap learned FSMap version 57 2024-04-24T09:08:24.808 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.807+0000 7f6afc568740 1 -- 172.21.15.83:0/22202556 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x5564c5a13d90 con 0x5564c598c390 2024-04-24T09:08:24.808 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.807+0000 7f6ad3e00640 1 -- 172.21.15.83:0/22202556 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6aec005070 con 0x5564c598c390 2024-04-24T09:08:24.809 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.807+0000 7f6ad3e00640 1 -- 172.21.15.83:0/22202556 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 57) ==== 8382+0+0 (secure 0 0 0) 0x7f6aec012450 con 0x5564c598c390 2024-04-24T09:08:24.809 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.808+0000 7f6afc568740 10 client.38618 fetch_fsmap finished waiting for FSMap version 57 2024-04-24T09:08:24.809 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.808+0000 7f6afc568740 10 client.38618 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:08:24.809 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.808+0000 7f6afc568740 20 client.38618 populate_metadata read hostname 'smithi083' 2024-04-24T09:08:24.809 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.808+0000 7f6afc568740 1 --2- 172.21.15.83:0/22202556 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5564c5a14420 0x5564c5a16840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:24.809 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.808+0000 7f6afc568740 4 client.38618 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T09:08:24.809 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.808+0000 7f6afc568740 1 -- 172.21.15.83:0/22202556 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5564c58ef190 con 0x5564c5a14420 2024-04-24T09:08:24.809 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.808+0000 7f6af2a00640 1 --2- 172.21.15.83:0/22202556 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5564c5a14420 0x5564c5a16840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:24.809 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.809+0000 7f6ad3e00640 10 client.38618 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:08:24.809 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.809+0000 7f6af2a00640 1 --2- 172.21.15.83:0/22202556 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5564c5a14420 0x5564c5a16840 crc :-1 s=READY pgs=426 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba27c8f92b5a90c3 server_cookie=80010080038df3d8 in_seq=0 out_seq=0 2024-04-24T09:08:24.960 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.959+0000 7f6ad3e00640 1 -- 172.21.15.83:0/22202556 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5564c58ef190 con 0x5564c5a14420 2024-04-24T09:08:24.960 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.959+0000 7f6ad3e00640 10 client.38618 handle_command_reply: tid=0 2024-04-24T09:08:24.960 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.959+0000 7f6afc568740 1 client.38618 shutdown 2024-04-24T09:08:24.960 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.959+0000 7f6afc568740 1 -- 172.21.15.83:0/22202556 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5564c598c390 msgr2=0x5564c5994930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:24.960 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.959+0000 7f6afc568740 1 --2- 172.21.15.83:0/22202556 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5564c598c390 0x5564c5994930 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6aec01d010 tx=0x7f6aec01a3f0 comp rx=0 tx=0).stop 2024-04-24T09:08:24.960 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.960+0000 7f6afc568740 1 -- 172.21.15.83:0/22202556 shutdown_connections 2024-04-24T09:08:24.961 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.960+0000 7f6afc568740 1 --2- 172.21.15.83:0/22202556 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5564c5a14420 0x5564c5a16840 crc :-1 s=READY pgs=426 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:24.961 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.960+0000 7f6afc568740 1 --2- 172.21.15.83:0/22202556 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5564c598c390 0x5564c5994930 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:24.961 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.960+0000 7f6afc568740 1 --2- 172.21.15.83:0/22202556 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5564c5981ad0 0x5564c59943f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:24.961 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.960+0000 7f6afc568740 1 --2- 172.21.15.83:0/22202556 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5564c59811b0 0x5564c59fe170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:24.961 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.960+0000 7f6afc568740 1 -- 172.21.15.83:0/22202556 >> 172.21.15.83:0/22202556 conn(0x5564c597d6c0 msgr2=0x5564c5989160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:24.961 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.960+0000 7f6ad3e00640 0 client.38618 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:08:24.961 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.960+0000 7f6afc568740 1 -- 172.21.15.83:0/22202556 shutdown_connections 2024-04-24T09:08:24.961 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.961+0000 7f6afc568740 1 -- 172.21.15.83:0/22202556 wait complete. 2024-04-24T09:08:24.961 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.961+0000 7f6afc568740 20 client.38618 trim_cache size 0 max 16384 2024-04-24T09:08:24.962 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T09:08:24.962 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T09:08:24.962 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T09:08:24.962 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T09:08:24.963 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.962+0000 7f6af9600640 1 -- 172.21.15.83:0/3426098797 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f6ad80a3e40 msgr2=0x7f6ad80a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:24.963 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.962+0000 7f6af9600640 1 --2- 172.21.15.83:0/3426098797 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f6ad80a3e40 0x7f6ad80a6300 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x5564c597eae0 tx=0x7f6ae00046c0 comp rx=0 tx=0).stop 2024-04-24T09:08:24.963 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.962+0000 7f6af9600640 1 -- 172.21.15.83:0/3426098797 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6af4104450 msgr2=0x7f6af4076510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:24.963 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.962+0000 7f6af9600640 1 --2- 172.21.15.83:0/3426098797 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6af4104450 0x7f6af4076510 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6ae402aa00 tx=0x7f6ae4003c80 comp rx=0 tx=0).stop 2024-04-24T09:08:24.963 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.962+0000 7f6af9600640 1 -- 172.21.15.83:0/3426098797 shutdown_connections 2024-04-24T09:08:24.963 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.962+0000 7f6af9600640 1 --2- 172.21.15.83:0/3426098797 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f6ad80a3e40 0x7f6ad80a6300 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:24.963 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.962+0000 7f6af9600640 1 --2- 172.21.15.83:0/3426098797 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6af4076f90 0x7f6af41aaff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:24.963 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.962+0000 7f6af9600640 1 --2- 172.21.15.83:0/3426098797 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6af4109520 0x7f6af4076a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:24.963 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.962+0000 7f6af9600640 1 --2- 172.21.15.83:0/3426098797 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6af4104450 0x7f6af4076510 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:24.963 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.962+0000 7f6af9600640 1 -- 172.21.15.83:0/3426098797 >> 172.21.15.83:0/3426098797 conn(0x7f6af40ffcf0 msgr2=0x7f6af4102000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:24.964 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.963+0000 7f6af9600640 1 -- 172.21.15.83:0/3426098797 shutdown_connections 2024-04-24T09:08:24.964 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:24.963+0000 7f6af9600640 1 -- 172.21.15.83:0/3426098797 wait complete. 2024-04-24T09:08:25.165 INFO:tasks.workunit.client.0.smithi083.stdout: 10 22747 7.12 MB/sec execute 56 sec latency 143.407 ms 2024-04-24T09:08:26.165 INFO:tasks.workunit.client.0.smithi083.stdout: 10 22849 7.06 MB/sec execute 57 sec latency 89.555 ms 2024-04-24T09:08:26.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:08:26 smithi138 ceph-mon[31441]: pgmap v529: 129 pgs: 129 active+clean; 488 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.9 MiB/s wr, 197 op/s 2024-04-24T09:08:26.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:08:26 smithi189 ceph-mon[31791]: pgmap v529: 129 pgs: 129 active+clean; 488 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.9 MiB/s wr, 197 op/s 2024-04-24T09:08:26.552 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:08:26 smithi083 ceph-mon[27907]: pgmap v529: 129 pgs: 129 active+clean; 488 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.9 MiB/s wr, 197 op/s 2024-04-24T09:08:27.165 INFO:tasks.workunit.client.0.smithi083.stdout: 10 22946 7.00 MB/sec execute 58 sec latency 70.918 ms 2024-04-24T09:08:28.165 INFO:tasks.workunit.client.0.smithi083.stdout: 10 23078 7.01 MB/sec execute 59 sec latency 71.415 ms 2024-04-24T09:08:28.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:08:28 smithi138 ceph-mon[31441]: pgmap v530: 129 pgs: 129 active+clean; 488 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.9 MiB/s wr, 197 op/s 2024-04-24T09:08:28.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:08:28 smithi138 ceph-mon[31441]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:08:28.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:08:28 smithi189 ceph-mon[31791]: pgmap v530: 129 pgs: 129 active+clean; 488 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.9 MiB/s wr, 197 op/s 2024-04-24T09:08:28.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:08:28 smithi189 ceph-mon[31791]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:08:28.552 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:08:28 smithi083 ceph-mon[27907]: pgmap v530: 129 pgs: 129 active+clean; 488 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.9 MiB/s wr, 197 op/s 2024-04-24T09:08:28.552 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:08:28 smithi083 ceph-mon[27907]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:08:29.166 INFO:tasks.workunit.client.0.smithi083.stdout: 10 23221 7.05 MB/sec execute 60 sec latency 98.948 ms 2024-04-24T09:08:30.165 INFO:tasks.workunit.client.0.smithi083.stdout: 10 23363 7.09 MB/sec execute 61 sec latency 67.861 ms 2024-04-24T09:08:30.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:08:30 smithi138 ceph-mon[31441]: pgmap v531: 129 pgs: 129 active+clean; 492 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.7 MiB/s wr, 330 op/s 2024-04-24T09:08:30.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:08:30 smithi189 ceph-mon[31791]: pgmap v531: 129 pgs: 129 active+clean; 492 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.7 MiB/s wr, 330 op/s 2024-04-24T09:08:30.552 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:08:30 smithi083 ceph-mon[27907]: pgmap v531: 129 pgs: 129 active+clean; 492 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.7 MiB/s wr, 330 op/s 2024-04-24T09:08:31.166 INFO:tasks.workunit.client.0.smithi083.stdout: 10 23475 7.02 MB/sec execute 62 sec latency 73.308 ms 2024-04-24T09:08:32.166 INFO:tasks.workunit.client.0.smithi083.stdout: 10 23582 6.96 MB/sec execute 63 sec latency 65.101 ms 2024-04-24T09:08:32.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:08:32 smithi138 ceph-mon[31441]: pgmap v532: 129 pgs: 129 active+clean; 492 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.6 MiB/s wr, 230 op/s 2024-04-24T09:08:32.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:08:32 smithi189 ceph-mon[31791]: pgmap v532: 129 pgs: 129 active+clean; 492 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.6 MiB/s wr, 230 op/s 2024-04-24T09:08:32.552 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:08:32 smithi083 ceph-mon[27907]: pgmap v532: 129 pgs: 129 active+clean; 492 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.6 MiB/s wr, 230 op/s 2024-04-24T09:08:33.166 INFO:tasks.workunit.client.0.smithi083.stdout: 10 23658 6.87 MB/sec execute 64 sec latency 427.143 ms 2024-04-24T09:08:34.166 INFO:tasks.workunit.client.0.smithi083.stdout: 10 23792 6.85 MB/sec execute 65 sec latency 444.003 ms 2024-04-24T09:08:34.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:08:34 smithi138 ceph-mon[31441]: pgmap v533: 129 pgs: 129 active+clean; 498 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.3 MiB/s wr, 326 op/s 2024-04-24T09:08:34.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:08:34 smithi189 ceph-mon[31791]: pgmap v533: 129 pgs: 129 active+clean; 498 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.3 MiB/s wr, 326 op/s 2024-04-24T09:08:34.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:08:34 smithi083 ceph-mon[27907]: pgmap v533: 129 pgs: 129 active+clean; 498 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.3 MiB/s wr, 326 op/s 2024-04-24T09:08:35.166 INFO:tasks.workunit.client.0.smithi083.stdout: 10 23944 6.87 MB/sec execute 66 sec latency 77.875 ms 2024-04-24T09:08:36.167 INFO:tasks.workunit.client.0.smithi083.stdout: 10 24096 6.89 MB/sec execute 67 sec latency 107.966 ms 2024-04-24T09:08:36.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:08:36 smithi138 ceph-mon[31441]: pgmap v534: 129 pgs: 129 active+clean; 498 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.6 MiB/s wr, 229 op/s 2024-04-24T09:08:36.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:08:36 smithi189 ceph-mon[31791]: pgmap v534: 129 pgs: 129 active+clean; 498 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.6 MiB/s wr, 229 op/s 2024-04-24T09:08:36.552 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:08:36 smithi083 ceph-mon[27907]: pgmap v534: 129 pgs: 129 active+clean; 498 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.6 MiB/s wr, 229 op/s 2024-04-24T09:08:37.166 INFO:tasks.workunit.client.0.smithi083.stdout: 10 24252 6.86 MB/sec execute 68 sec latency 74.077 ms 2024-04-24T09:08:38.139 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.139+0000 7f39c9600640 1 -- 172.21.15.83:0/772177906 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39c4104c60 msgr2=0x7f39c4108ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:38.139 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.139+0000 7f39c9600640 1 --2- 172.21.15.83:0/772177906 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39c4104c60 0x7f39c4108ca0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f39b0009900 tx=0x7f39b002a3f0 comp rx=0 tx=0).stop 2024-04-24T09:08:38.139 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.139+0000 7f39c9600640 1 -- 172.21.15.83:0/772177906 shutdown_connections 2024-04-24T09:08:38.139 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.139+0000 7f39c9600640 1 --2- 172.21.15.83:0/772177906 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f39c4109380 0x7f39c4110f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:38.139 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.139+0000 7f39c9600640 1 --2- 172.21.15.83:0/772177906 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39c4104c60 0x7f39c4108ca0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:38.139 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.139+0000 7f39c9600640 1 --2- 172.21.15.83:0/772177906 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39c41042b0 0x7f39c4104690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:38.139 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.139+0000 7f39c9600640 1 -- 172.21.15.83:0/772177906 >> 172.21.15.83:0/772177906 conn(0x7f39c40ffb70 msgr2=0x7f39c4101f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:38.139 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.139+0000 7f39c9600640 1 -- 172.21.15.83:0/772177906 shutdown_connections 2024-04-24T09:08:38.139 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.140+0000 7f39c9600640 1 -- 172.21.15.83:0/772177906 wait complete. 2024-04-24T09:08:38.140 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.141+0000 7f39c9600640 1 Processor -- start 2024-04-24T09:08:38.140 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.141+0000 7f39c9600640 1 -- start start 2024-04-24T09:08:38.140 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.141+0000 7f39c9600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39c41042b0 0x7f39c4076510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:38.141 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.141+0000 7f39c9600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39c4109380 0x7f39c4076a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:38.141 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.141+0000 7f39c9600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f39c4076f90 0x7f39c41aaed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:38.141 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.141+0000 7f39c9600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f39c41142c0 con 0x7f39c41042b0 2024-04-24T09:08:38.141 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.141+0000 7f39c9600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f39c4114140 con 0x7f39c4109380 2024-04-24T09:08:38.141 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.141+0000 7f39c9600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f39c4114440 con 0x7f39c4076f90 2024-04-24T09:08:38.142 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.142+0000 7f39c3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39c41042b0 0x7f39c4076510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:38.142 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.142+0000 7f39c3400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39c4109380 0x7f39c4076a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:38.142 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.142+0000 7f39c3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39c41042b0 0x7f39c4076510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58824/0 (socket says 172.21.15.83:58824) 2024-04-24T09:08:38.143 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.142+0000 7f39c3e00640 1 -- 172.21.15.83:0/1978657397 learned_addr learned my addr 172.21.15.83:0/1978657397 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:08:38.143 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.142+0000 7f39c3400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39c4109380 0x7f39c4076a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:38448/0 (socket says 172.21.15.83:38448) 2024-04-24T09:08:38.143 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.142+0000 7f39c8c00640 1 --2- 172.21.15.83:0/1978657397 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f39c4076f90 0x7f39c41aaed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:38.143 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.142+0000 7f39c3400640 1 -- 172.21.15.83:0/1978657397 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f39c4076f90 msgr2=0x7f39c41aaed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:38.143 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.142+0000 7f39c3400640 1 --2- 172.21.15.83:0/1978657397 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f39c4076f90 0x7f39c41aaed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:38.143 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.142+0000 7f39c3400640 1 -- 172.21.15.83:0/1978657397 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39c41042b0 msgr2=0x7f39c4076510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:38.143 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.142+0000 7f39c3400640 1 --2- 172.21.15.83:0/1978657397 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39c41042b0 0x7f39c4076510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:38.143 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.142+0000 7f39c3400640 1 -- 172.21.15.83:0/1978657397 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39c41ab410 con 0x7f39c4109380 2024-04-24T09:08:38.143 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.143+0000 7f39c3400640 1 --2- 172.21.15.83:0/1978657397 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39c4109380 0x7f39c4076a50 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f39b002a910 tx=0x7f39b0004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:38.143 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.143+0000 7f39c0c00640 1 -- 172.21.15.83:0/1978657397 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39b003f070 con 0x7f39c4109380 2024-04-24T09:08:38.143 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.143+0000 7f39c9600640 1 -- 172.21.15.83:0/1978657397 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39c41ab6a0 con 0x7f39c4109380 2024-04-24T09:08:38.143 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.143+0000 7f39c0c00640 1 -- 172.21.15.83:0/1978657397 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f39b0003ca0 con 0x7f39c4109380 2024-04-24T09:08:38.143 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.143+0000 7f39c0c00640 1 -- 172.21.15.83:0/1978657397 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39b0035070 con 0x7f39c4109380 2024-04-24T09:08:38.143 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.143+0000 7f39c9600640 1 -- 172.21.15.83:0/1978657397 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39c41abbb0 con 0x7f39c4109380 2024-04-24T09:08:38.147 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.148+0000 7f39c0c00640 1 -- 172.21.15.83:0/1978657397 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f39b003b560 con 0x7f39c4109380 2024-04-24T09:08:38.147 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.148+0000 7f39cd9a2740 1 Processor -- start 2024-04-24T09:08:38.148 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.149+0000 7f39cd9a2740 1 -- start start 2024-04-24T09:08:38.148 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.149+0000 7f39cd9a2740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5601c03733c0 0x5601c03737a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:38.149 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.149+0000 7f39cd9a2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5601c0373ce0 0x5601c037e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:38.149 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.149+0000 7f39cd9a2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5601c037e5a0 0x5601c0380990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:38.149 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.149+0000 7f39cd9a2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5601c02d9900 con 0x5601c037e5a0 2024-04-24T09:08:38.149 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.149+0000 7f39cd9a2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5601c027ff20 con 0x5601c0373ce0 2024-04-24T09:08:38.149 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.149+0000 7f39cd9a2740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5601c02d58c0 con 0x5601c03733c0 2024-04-24T09:08:38.149 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.149+0000 7f39c3400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5601c037e5a0 0x5601c0380990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:38.149 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.149+0000 7f39c3400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5601c037e5a0 0x5601c0380990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58830/0 (socket says 172.21.15.83:58830) 2024-04-24T09:08:38.149 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.149+0000 7f39c3400640 1 -- 172.21.15.83:0/2591941879 learned_addr learned my addr 172.21.15.83:0/2591941879 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:08:38.149 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.150+0000 7f39c8c00640 1 --2- 172.21.15.83:0/2591941879 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5601c03733c0 0x5601c03737a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:38.150 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.150+0000 7f39c3400640 1 -- 172.21.15.83:0/2591941879 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5601c03733c0 msgr2=0x5601c03737a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:38.150 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.150+0000 7f39c3400640 1 --2- 172.21.15.83:0/2591941879 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5601c03733c0 0x5601c03737a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:38.150 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.150+0000 7f39c3400640 1 -- 172.21.15.83:0/2591941879 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5601c0373ce0 msgr2=0x5601c037e060 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-24T09:08:38.150 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.150+0000 7f39c3400640 1 --2- 172.21.15.83:0/2591941879 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5601c0373ce0 0x5601c037e060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:38.150 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.150+0000 7f39c3400640 1 -- 172.21.15.83:0/2591941879 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601c02a4e10 con 0x5601c037e5a0 2024-04-24T09:08:38.150 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.151+0000 7f39c0c00640 1 --2- 172.21.15.83:0/1978657397 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f39a80a3d10 0x7f39a80a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:38.151 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.151+0000 7f39c0c00640 1 -- 172.21.15.83:0/1978657397 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 9583+0+0 (secure 0 0 0) 0x7f39b00f1150 con 0x7f39c4109380 2024-04-24T09:08:38.151 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.151+0000 7f39c3400640 1 --2- 172.21.15.83:0/2591941879 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5601c037e5a0 0x5601c0380990 secure :-1 s=READY pgs=2310 cs=0 l=1 rev1=1 crypto rx=0x7f39b00ee360 tx=0x7f39b00f95e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1e8c75a0b50784a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:38.155 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.151+0000 7f39c8c00640 1 --2- 172.21.15.83:0/1978657397 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f39a80a3d10 0x7f39a80a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:38.155 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.152+0000 7f39b9200640 1 -- 172.21.15.83:0/2591941879 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39b0048070 con 0x5601c037e5a0 2024-04-24T09:08:38.155 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.152+0000 7f39b9200640 1 -- 172.21.15.83:0/2591941879 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f39b00bd370 con 0x5601c037e5a0 2024-04-24T09:08:38.155 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.152+0000 7f39b9200640 1 -- 172.21.15.83:0/2591941879 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39b00bc7b0 con 0x5601c037e5a0 2024-04-24T09:08:38.156 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.153+0000 7f39cd9a2740 1 -- 172.21.15.83:0/2591941879 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5601c037e5a0 msgr2=0x5601c0380990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:38.156 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.153+0000 7f39cd9a2740 1 --2- 172.21.15.83:0/2591941879 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5601c037e5a0 0x5601c0380990 secure :-1 s=READY pgs=2310 cs=0 l=1 rev1=1 crypto rx=0x7f39b00ee360 tx=0x7f39b00f95e0 comp rx=0 tx=0).stop 2024-04-24T09:08:38.156 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.153+0000 7f39c8c00640 1 --2- 172.21.15.83:0/1978657397 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f39a80a3d10 0x7f39a80a61d0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f39bc0092f0 tx=0x7f39bc009280 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:38.156 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.153+0000 7f39cd9a2740 1 -- 172.21.15.83:0/2591941879 shutdown_connections 2024-04-24T09:08:38.156 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.153+0000 7f39cd9a2740 1 --2- 172.21.15.83:0/2591941879 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5601c037e5a0 0x5601c0380990 unknown :-1 s=CLOSED pgs=2310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:38.156 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.153+0000 7f39cd9a2740 1 --2- 172.21.15.83:0/2591941879 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5601c0373ce0 0x5601c037e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:38.156 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.153+0000 7f39cd9a2740 1 --2- 172.21.15.83:0/2591941879 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5601c03733c0 0x5601c03737a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:38.156 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.153+0000 7f39cd9a2740 1 -- 172.21.15.83:0/2591941879 >> 172.21.15.83:0/2591941879 conn(0x5601c036f8d0 msgr2=0x5601c036fcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:38.156 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.154+0000 7f39cd9a2740 1 -- 172.21.15.83:0/2591941879 shutdown_connections 2024-04-24T09:08:38.156 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.154+0000 7f39cd9a2740 1 -- 172.21.15.83:0/2591941879 wait complete. 2024-04-24T09:08:38.156 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.154+0000 7f39cd9a2740 1 -- start start 2024-04-24T09:08:38.156 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.155+0000 7f39cd9a2740 1 Processor -- start 2024-04-24T09:08:38.157 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.155+0000 7f39cd9a2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5601c03733c0 0x5601c037b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:38.157 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.155+0000 7f39cd9a2740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5601c0373ce0 0x5601c037bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:38.157 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.155+0000 7f39cd9a2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5601c037e5a0 0x5601c037c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:38.157 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.155+0000 7f39cd9a2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5601c0380f90 con 0x5601c037e5a0 2024-04-24T09:08:38.157 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.155+0000 7f39cd9a2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5601c02932c0 con 0x5601c03733c0 2024-04-24T09:08:38.157 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.155+0000 7f39cd9a2740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5601c0381110 con 0x5601c0373ce0 2024-04-24T09:08:38.157 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.156+0000 7f39cd9a2740 10 client.0 _ll_register_callbacks cb 0x5601bffeba00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:08:38.157 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.156+0000 7f39c3e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5601c03733c0 0x5601c037b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:38.157 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.156+0000 7f39c3e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5601c03733c0 0x5601c037b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:38460/0 (socket says 172.21.15.83:38460) 2024-04-24T09:08:38.157 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.156+0000 7f39c3e00640 1 -- 172.21.15.83:0/699604790 learned_addr learned my addr 172.21.15.83:0/699604790 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:08:38.158 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.156+0000 7f39c8c00640 1 --2- 172.21.15.83:0/699604790 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5601c037e5a0 0x5601c037c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:38.158 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.156+0000 7f39c3400640 1 --2- 172.21.15.83:0/699604790 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5601c0373ce0 0x5601c037bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:38.158 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.157+0000 7f39c8c00640 1 -- 172.21.15.83:0/699604790 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5601c0373ce0 msgr2=0x5601c037bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:38.158 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.157+0000 7f39c8c00640 1 --2- 172.21.15.83:0/699604790 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5601c0373ce0 0x5601c037bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:38.159 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.157+0000 7f39c8c00640 1 -- 172.21.15.83:0/699604790 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5601c03733c0 msgr2=0x5601c037b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:38.159 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.157+0000 7f39c8c00640 1 --2- 172.21.15.83:0/699604790 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5601c03733c0 0x5601c037b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:38.159 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.157+0000 7f39c8c00640 1 -- 172.21.15.83:0/699604790 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601c02a3fd0 con 0x5601c037e5a0 2024-04-24T09:08:38.159 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.157+0000 7f39c3400640 1 --2- 172.21.15.83:0/699604790 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5601c0373ce0 0x5601c037bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:08:38.159 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.157+0000 7f39c8c00640 1 --2- 172.21.15.83:0/699604790 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5601c037e5a0 0x5601c037c080 secure :-1 s=READY pgs=2311 cs=0 l=1 rev1=1 crypto rx=0x7f39bc005fd0 tx=0x7f39bc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9864d6e3a756d78 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:38.159 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.157+0000 7f39ad600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:08:38.159 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.158+0000 7f39ad600640 1 -- 172.21.15.83:0/699604790 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39bc004d20 con 0x5601c037e5a0 2024-04-24T09:08:38.159 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.159+0000 7f39cd9a2740 1 -- 172.21.15.83:0/699604790 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39b00bd370 con 0x5601c037e5a0 2024-04-24T09:08:38.159 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.159+0000 7f39ad600640 1 -- 172.21.15.83:0/699604790 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9583+0+0 (secure 0 0 0) 0x7f39bc00e7a0 con 0x5601c037e5a0 2024-04-24T09:08:38.159 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.159+0000 7f39ad600640 1 -- 172.21.15.83:0/699604790 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f39bc02f050 con 0x5601c037e5a0 2024-04-24T09:08:38.159 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.159+0000 7f39cd9a2740 10 client.21486 fetch_fsmap learned FSMap version 58 2024-04-24T09:08:38.160 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.159+0000 7f39cd9a2740 1 -- 172.21.15.83:0/699604790 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x5601bffec210 con 0x5601c037e5a0 2024-04-24T09:08:38.161 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.160+0000 7f39ad600640 1 -- 172.21.15.83:0/699604790 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f39b00bd370 con 0x5601c037e5a0 2024-04-24T09:08:38.161 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.160+0000 7f39ad600640 1 -- 172.21.15.83:0/699604790 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 58) ==== 8374+0+0 (secure 0 0 0) 0x7f39bc00f040 con 0x5601c037e5a0 2024-04-24T09:08:38.161 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.160+0000 7f39cd9a2740 10 client.21486 fetch_fsmap finished waiting for FSMap version 58 2024-04-24T09:08:38.161 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.160+0000 7f39cd9a2740 10 client.21486 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:08:38.161 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.161+0000 7f39cd9a2740 20 client.21486 populate_metadata read hostname 'smithi083' 2024-04-24T09:08:38.161 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.161+0000 7f39cd9a2740 1 --2- 172.21.15.83:0/699604790 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5601c038e550 0x5601c0390940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:38.161 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.161+0000 7f39cd9a2740 4 client.21486 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:08:38.161 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.161+0000 7f39cd9a2740 1 -- 172.21.15.83:0/699604790 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5601c02e7320 con 0x5601c038e550 2024-04-24T09:08:38.161 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.161+0000 7f39c3e00640 1 --2- 172.21.15.83:0/699604790 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5601c038e550 0x5601c0390940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:38.162 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.162+0000 7f39c3e00640 1 --2- 172.21.15.83:0/699604790 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5601c038e550 0x5601c0390940 crc :-1 s=READY pgs=427 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fda6bc4fa4933572 server_cookie=fbeab81c6e13a247 in_seq=0 out_seq=0 2024-04-24T09:08:38.162 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.163+0000 7f39ad600640 10 client.21486 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:08:38.163 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.164+0000 7f39ad600640 1 -- 172.21.15.83:0/699604790 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5601c02e7320 con 0x5601c038e550 2024-04-24T09:08:38.164 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.164+0000 7f39ad600640 10 client.21486 handle_command_reply: tid=0 2024-04-24T09:08:38.164 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.164+0000 7f39cd9a2740 1 client.21486 shutdown 2024-04-24T09:08:38.164 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.165+0000 7f39cd9a2740 1 -- 172.21.15.83:0/699604790 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5601c037e5a0 msgr2=0x5601c037c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:38.164 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.165+0000 7f39cd9a2740 1 --2- 172.21.15.83:0/699604790 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5601c037e5a0 0x5601c037c080 secure :-1 s=READY pgs=2311 cs=0 l=1 rev1=1 crypto rx=0x7f39bc005fd0 tx=0x7f39bc004a40 comp rx=0 tx=0).stop 2024-04-24T09:08:38.164 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.165+0000 7f39cd9a2740 1 -- 172.21.15.83:0/699604790 shutdown_connections 2024-04-24T09:08:38.165 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.165+0000 7f39cd9a2740 1 --2- 172.21.15.83:0/699604790 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5601c038e550 0x5601c0390940 crc :-1 s=READY pgs=427 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:38.165 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.165+0000 7f39ad600640 0 client.21486 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:08:38.165 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.165+0000 7f39cd9a2740 1 --2- 172.21.15.83:0/699604790 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5601c037e5a0 0x5601c037c080 unknown :-1 s=CLOSED pgs=2311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:38.165 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.165+0000 7f39cd9a2740 1 --2- 172.21.15.83:0/699604790 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5601c0373ce0 0x5601c037bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:38.165 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.165+0000 7f39cd9a2740 1 --2- 172.21.15.83:0/699604790 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5601c03733c0 0x5601c037b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:38.165 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.165+0000 7f39cd9a2740 1 -- 172.21.15.83:0/699604790 >> 172.21.15.83:0/699604790 conn(0x5601c036f8d0 msgr2=0x5601c0375600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:38.165 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.165+0000 7f39cd9a2740 1 -- 172.21.15.83:0/699604790 shutdown_connections 2024-04-24T09:08:38.166 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.166+0000 7f39cd9a2740 1 -- 172.21.15.83:0/699604790 wait complete. 2024-04-24T09:08:38.166 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.166+0000 7f39cd9a2740 20 client.21486 trim_cache size 0 max 16384 2024-04-24T09:08:38.166 INFO:tasks.workunit.client.0.smithi083.stdout: 10 24387 6.90 MB/sec execute 69 sec latency 92.644 ms 2024-04-24T09:08:38.221 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.222+0000 7f39cd9a2740 1 Processor -- start 2024-04-24T09:08:38.221 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.222+0000 7f39cd9a2740 1 -- start start 2024-04-24T09:08:38.222 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.222+0000 7f39cd9a2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5601c03733c0 0x5601c037bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:38.222 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.222+0000 7f39cd9a2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5601c0373ce0 0x5601c036e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:38.222 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.222+0000 7f39cd9a2740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5601c037e5a0 0x5601c0396250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:38.222 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.222+0000 7f39cd9a2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f39c41b5ba0 con 0x5601c0373ce0 2024-04-24T09:08:38.223 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.223+0000 7f39cd9a2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f39c41b76e0 con 0x5601c03733c0 2024-04-24T09:08:38.223 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.223+0000 7f39cd9a2740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f39c41b6270 con 0x5601c037e5a0 2024-04-24T09:08:38.223 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.223+0000 7f39c3e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5601c03733c0 0x5601c037bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:38.223 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.223+0000 7f39c3400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5601c0373ce0 0x5601c036e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:38.223 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.223+0000 7f39c3e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5601c03733c0 0x5601c037bea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:38464/0 (socket says 172.21.15.83:38464) 2024-04-24T09:08:38.223 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.223+0000 7f39c3e00640 1 -- 172.21.15.83:0/1696253250 learned_addr learned my addr 172.21.15.83:0/1696253250 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:08:38.223 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.223+0000 7f39c8c00640 1 --2- 172.21.15.83:0/1696253250 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5601c037e5a0 0x5601c0396250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:38.224 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.224+0000 7f39c3e00640 1 -- 172.21.15.83:0/1696253250 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5601c037e5a0 msgr2=0x5601c0396250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:38.224 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.224+0000 7f39c3e00640 1 --2- 172.21.15.83:0/1696253250 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5601c037e5a0 0x5601c0396250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:38.224 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.224+0000 7f39c3e00640 1 -- 172.21.15.83:0/1696253250 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5601c0373ce0 msgr2=0x5601c036e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:38.225 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.224+0000 7f39c3e00640 1 --2- 172.21.15.83:0/1696253250 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5601c0373ce0 0x5601c036e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:38.225 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.224+0000 7f39c3e00640 1 -- 172.21.15.83:0/1696253250 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601c00b8620 con 0x5601c03733c0 2024-04-24T09:08:38.225 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.224+0000 7f39c3400640 1 --2- 172.21.15.83:0/1696253250 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5601c0373ce0 0x5601c036e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:08:38.225 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.224+0000 7f39c3e00640 1 --2- 172.21.15.83:0/1696253250 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5601c03733c0 0x5601c037bea0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x5601c03911c0 tx=0x7f39b400beb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f87404f030145002 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:38.225 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.225+0000 7f39afe00640 1 -- 172.21.15.83:0/1696253250 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39b4003d00 con 0x5601c03733c0 2024-04-24T09:08:38.225 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.225+0000 7f39afe00640 1 -- 172.21.15.83:0/1696253250 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f39b4003ea0 con 0x5601c03733c0 2024-04-24T09:08:38.225 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.225+0000 7f39afe00640 1 -- 172.21.15.83:0/1696253250 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39b4011860 con 0x5601c03733c0 2024-04-24T09:08:38.226 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.226+0000 7f39cd9a2740 1 -- 172.21.15.83:0/1696253250 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5601c03733c0 msgr2=0x5601c037bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:38.226 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.226+0000 7f39cd9a2740 1 --2- 172.21.15.83:0/1696253250 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5601c03733c0 0x5601c037bea0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x5601c03911c0 tx=0x7f39b400beb0 comp rx=0 tx=0).stop 2024-04-24T09:08:38.226 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.226+0000 7f39cd9a2740 1 -- 172.21.15.83:0/1696253250 shutdown_connections 2024-04-24T09:08:38.226 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.226+0000 7f39cd9a2740 1 --2- 172.21.15.83:0/1696253250 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5601c037e5a0 0x5601c0396250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:38.227 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.226+0000 7f39cd9a2740 1 --2- 172.21.15.83:0/1696253250 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5601c0373ce0 0x5601c036e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:38.227 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.226+0000 7f39cd9a2740 1 --2- 172.21.15.83:0/1696253250 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5601c03733c0 0x5601c037bea0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:38.227 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.226+0000 7f39cd9a2740 1 -- 172.21.15.83:0/1696253250 >> 172.21.15.83:0/1696253250 conn(0x5601c036f8d0 msgr2=0x5601c036e200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:38.227 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.227+0000 7f39cd9a2740 1 -- 172.21.15.83:0/1696253250 shutdown_connections 2024-04-24T09:08:38.227 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.227+0000 7f39cd9a2740 1 -- 172.21.15.83:0/1696253250 wait complete. 2024-04-24T09:08:38.227 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.227+0000 7f39cd9a2740 1 -- start start 2024-04-24T09:08:38.227 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.228+0000 7f39cd9a2740 1 Processor -- start 2024-04-24T09:08:38.229 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.229+0000 7f39cd9a2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5601c03733c0 0x5601c03866e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:38.229 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.229+0000 7f39cd9a2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5601c0373ce0 0x5601c03832a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:38.229 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.229+0000 7f39cd9a2740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5601c037e5a0 0x5601c03837e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:38.229 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.229+0000 7f39cd9a2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f39c41b7010 con 0x5601c0373ce0 2024-04-24T09:08:38.229 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.229+0000 7f39cd9a2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f39c41b6940 con 0x5601c03733c0 2024-04-24T09:08:38.230 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.229+0000 7f39cd9a2740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f39c41b3ba0 con 0x5601c037e5a0 2024-04-24T09:08:38.230 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.229+0000 7f39cd9a2740 10 client.0 _ll_register_callbacks cb 0x7f39bc011ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:08:38.230 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.229+0000 7f39c3e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5601c03733c0 0x5601c03866e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:38.230 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.229+0000 7f39c3e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5601c03733c0 0x5601c03866e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:38468/0 (socket says 172.21.15.83:38468) 2024-04-24T09:08:38.230 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.229+0000 7f39c3e00640 1 -- 172.21.15.83:0/1355779476 learned_addr learned my addr 172.21.15.83:0/1355779476 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:08:38.230 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.229+0000 7f39c8c00640 1 --2- 172.21.15.83:0/1355779476 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5601c037e5a0 0x5601c03837e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:38.230 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.229+0000 7f39c3400640 1 --2- 172.21.15.83:0/1355779476 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5601c0373ce0 0x5601c03832a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:38.230 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.230+0000 7f39c8c00640 1 -- 172.21.15.83:0/1355779476 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5601c03733c0 msgr2=0x5601c03866e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:38.230 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.230+0000 7f39c8c00640 1 --2- 172.21.15.83:0/1355779476 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5601c03733c0 0x5601c03866e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:38.230 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.230+0000 7f39c8c00640 1 -- 172.21.15.83:0/1355779476 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5601c0373ce0 msgr2=0x5601c03832a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:38.230 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.230+0000 7f39c8c00640 1 --2- 172.21.15.83:0/1355779476 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5601c0373ce0 0x5601c03832a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:38.231 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.230+0000 7f39c8c00640 1 -- 172.21.15.83:0/1355779476 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601c0383fc0 con 0x5601c037e5a0 2024-04-24T09:08:38.231 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.230+0000 7f39c8c00640 1 --2- 172.21.15.83:0/1355779476 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5601c037e5a0 0x5601c03837e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f39bc0058d0 tx=0x7f39bc005b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b6a2e0f0a432df8c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:38.231 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.230+0000 7f39ad600640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-04-24T09:08:38.232 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.231+0000 7f39ad600640 1 -- 172.21.15.83:0/1355779476 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39bc00ed70 con 0x5601c037e5a0 2024-04-24T09:08:38.232 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.231+0000 7f39ad600640 1 -- 172.21.15.83:0/1355779476 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9583+0+0 (secure 0 0 0) 0x7f39bc03d460 con 0x5601c037e5a0 2024-04-24T09:08:38.232 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.231+0000 7f39cd9a2740 1 -- 172.21.15.83:0/1355779476 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39b4003ea0 con 0x5601c037e5a0 2024-04-24T09:08:38.233 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.231+0000 7f39ad600640 1 -- 172.21.15.83:0/1355779476 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f39bc046050 con 0x5601c037e5a0 2024-04-24T09:08:38.233 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.231+0000 7f39cd9a2740 10 client.28642 fetch_fsmap learned FSMap version 58 2024-04-24T09:08:38.233 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.231+0000 7f39cd9a2740 1 -- 172.21.15.83:0/1355779476 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x5601c03f5af0 con 0x5601c037e5a0 2024-04-24T09:08:38.233 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.232+0000 7f39ad600640 1 -- 172.21.15.83:0/1355779476 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f39b4003ea0 con 0x5601c037e5a0 2024-04-24T09:08:38.233 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.232+0000 7f39ad600640 1 -- 172.21.15.83:0/1355779476 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 58) ==== 8374+0+0 (secure 0 0 0) 0x7f39bc01f620 con 0x5601c037e5a0 2024-04-24T09:08:38.233 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.232+0000 7f39cd9a2740 10 client.28642 fetch_fsmap finished waiting for FSMap version 58 2024-04-24T09:08:38.233 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.232+0000 7f39cd9a2740 10 client.28642 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:08:38.233 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.232+0000 7f39cd9a2740 20 client.28642 populate_metadata read hostname 'smithi083' 2024-04-24T09:08:38.233 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.232+0000 7f39cd9a2740 1 --2- 172.21.15.83:0/1355779476 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5601c03f61b0 0x5601c03f85a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:38.233 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.232+0000 7f39cd9a2740 4 client.28642 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T09:08:38.233 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.232+0000 7f39cd9a2740 1 -- 172.21.15.83:0/1355779476 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5601c02e1190 con 0x5601c03f61b0 2024-04-24T09:08:38.233 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.232+0000 7f39c3e00640 1 --2- 172.21.15.83:0/1355779476 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5601c03f61b0 0x5601c03f85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:38.234 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.233+0000 7f39c3e00640 1 --2- 172.21.15.83:0/1355779476 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5601c03f61b0 0x5601c03f85a0 crc :-1 s=READY pgs=428 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3080b593e258dbd server_cookie=1c5474ade81f2302 in_seq=0 out_seq=0 2024-04-24T09:08:38.234 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.233+0000 7f39ad600640 10 client.28642 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:08:38.323 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.324+0000 7f39ad600640 1 -- 172.21.15.83:0/1355779476 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5601c02e1190 con 0x5601c03f61b0 2024-04-24T09:08:38.323 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.324+0000 7f39ad600640 10 client.28642 handle_command_reply: tid=0 2024-04-24T09:08:38.323 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.324+0000 7f39cd9a2740 1 client.28642 shutdown 2024-04-24T09:08:38.324 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.324+0000 7f39cd9a2740 1 -- 172.21.15.83:0/1355779476 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5601c037e5a0 msgr2=0x5601c03837e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:38.325 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.324+0000 7f39cd9a2740 1 --2- 172.21.15.83:0/1355779476 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5601c037e5a0 0x5601c03837e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f39bc0058d0 tx=0x7f39bc005b60 comp rx=0 tx=0).stop 2024-04-24T09:08:38.325 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.325+0000 7f39cd9a2740 1 -- 172.21.15.83:0/1355779476 shutdown_connections 2024-04-24T09:08:38.325 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.325+0000 7f39cd9a2740 1 --2- 172.21.15.83:0/1355779476 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5601c03f61b0 0x5601c03f85a0 crc :-1 s=READY pgs=428 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:38.325 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.325+0000 7f39cd9a2740 1 --2- 172.21.15.83:0/1355779476 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5601c037e5a0 0x5601c03837e0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:38.325 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.325+0000 7f39cd9a2740 1 --2- 172.21.15.83:0/1355779476 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5601c0373ce0 0x5601c03832a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:38.325 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.325+0000 7f39cd9a2740 1 --2- 172.21.15.83:0/1355779476 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5601c03733c0 0x5601c03866e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:38.325 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.325+0000 7f39cd9a2740 1 -- 172.21.15.83:0/1355779476 >> 172.21.15.83:0/1355779476 conn(0x5601c036f8d0 msgr2=0x5601c036ea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:38.325 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.325+0000 7f39ad600640 0 client.28642 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:08:38.325 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.325+0000 7f39cd9a2740 1 -- 172.21.15.83:0/1355779476 shutdown_connections 2024-04-24T09:08:38.325 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.325+0000 7f39cd9a2740 1 -- 172.21.15.83:0/1355779476 wait complete. 2024-04-24T09:08:38.326 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.325+0000 7f39cd9a2740 20 client.28642 trim_cache size 0 max 16384 2024-04-24T09:08:38.326 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T09:08:38.326 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T09:08:38.326 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T09:08:38.326 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T09:08:38.327 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.326+0000 7f39c9600640 1 -- 172.21.15.83:0/1978657397 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f39a80a3d10 msgr2=0x7f39a80a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:38.327 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.326+0000 7f39c9600640 1 --2- 172.21.15.83:0/1978657397 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f39a80a3d10 0x7f39a80a61d0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f39bc0092f0 tx=0x7f39bc009280 comp rx=0 tx=0).stop 2024-04-24T09:08:38.327 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.326+0000 7f39c9600640 1 -- 172.21.15.83:0/1978657397 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39c4109380 msgr2=0x7f39c4076a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:38.327 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.326+0000 7f39c9600640 1 --2- 172.21.15.83:0/1978657397 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39c4109380 0x7f39c4076a50 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f39b002a910 tx=0x7f39b0004270 comp rx=0 tx=0).stop 2024-04-24T09:08:38.327 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.326+0000 7f39c9600640 1 -- 172.21.15.83:0/1978657397 shutdown_connections 2024-04-24T09:08:38.327 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.326+0000 7f39c9600640 1 --2- 172.21.15.83:0/1978657397 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f39a80a3d10 0x7f39a80a61d0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:38.327 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.326+0000 7f39c9600640 1 --2- 172.21.15.83:0/1978657397 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f39c4076f90 0x7f39c41aaed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:38.327 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.326+0000 7f39c9600640 1 --2- 172.21.15.83:0/1978657397 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39c4109380 0x7f39c4076a50 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:38.327 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.326+0000 7f39c9600640 1 --2- 172.21.15.83:0/1978657397 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39c41042b0 0x7f39c4076510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:38.328 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.326+0000 7f39c9600640 1 -- 172.21.15.83:0/1978657397 >> 172.21.15.83:0/1978657397 conn(0x7f39c40ffb70 msgr2=0x7f39c4100f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:38.328 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.327+0000 7f39c9600640 1 -- 172.21.15.83:0/1978657397 shutdown_connections 2024-04-24T09:08:38.328 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:38.327+0000 7f39c9600640 1 -- 172.21.15.83:0/1978657397 wait complete. 2024-04-24T09:08:38.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:08:38 smithi189 ceph-mon[31791]: pgmap v535: 129 pgs: 129 active+clean; 498 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.6 MiB/s wr, 229 op/s 2024-04-24T09:08:38.552 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:08:38 smithi083 ceph-mon[27907]: pgmap v535: 129 pgs: 129 active+clean; 498 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.6 MiB/s wr, 229 op/s 2024-04-24T09:08:38.704 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:08:38 smithi138 ceph-mon[31441]: pgmap v535: 129 pgs: 129 active+clean; 498 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.6 MiB/s wr, 229 op/s 2024-04-24T09:08:39.167 INFO:tasks.workunit.client.0.smithi083.stdout: 10 24502 6.86 MB/sec execute 70 sec latency 112.826 ms 2024-04-24T09:08:39.590 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T09:08:39.591 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T09:08:39.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.843+0000 7f49dc400640 1 -- 172.21.15.83:0/2769834323 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49d40742d0 msgr2=0x7f49d40746b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:39.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.843+0000 7f49dc400640 1 --2- 172.21.15.83:0/2769834323 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49d40742d0 0x7f49d40746b0 secure :-1 s=READY pgs=2312 cs=0 l=1 rev1=1 crypto rx=0x7f49c8009900 tx=0x7f49c802a480 comp rx=0 tx=0).stop 2024-04-24T09:08:39.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.843+0000 7f49dc400640 1 -- 172.21.15.83:0/2769834323 shutdown_connections 2024-04-24T09:08:39.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.843+0000 7f49dc400640 1 --2- 172.21.15.83:0/2769834323 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f49d410e8e0 0x7f49d4110cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:39.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.843+0000 7f49dc400640 1 --2- 172.21.15.83:0/2769834323 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f49d4074bf0 0x7f49d410e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:39.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.843+0000 7f49dc400640 1 --2- 172.21.15.83:0/2769834323 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49d40742d0 0x7f49d40746b0 unknown :-1 s=CLOSED pgs=2312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:39.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.843+0000 7f49dc400640 1 -- 172.21.15.83:0/2769834323 >> 172.21.15.83:0/2769834323 conn(0x7f49d4079450 msgr2=0x7f49d4079860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:39.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.843+0000 7f49dc400640 1 -- 172.21.15.83:0/2769834323 shutdown_connections 2024-04-24T09:08:39.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.843+0000 7f49dc400640 1 -- 172.21.15.83:0/2769834323 wait complete. 2024-04-24T09:08:39.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.844+0000 7f49dc400640 1 Processor -- start 2024-04-24T09:08:39.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.845+0000 7f49dc400640 1 -- start start 2024-04-24T09:08:39.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.845+0000 7f49dc400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49d40742d0 0x7f49d4102000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:39.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.845+0000 7f49dc400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f49d4074bf0 0x7f49d4102540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:39.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.845+0000 7f49dc400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f49d410e8e0 0x7f49d4102a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:39.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.845+0000 7f49dc400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f49d41144f0 con 0x7f49d40742d0 2024-04-24T09:08:39.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.845+0000 7f49dc400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f49d4114370 con 0x7f49d4074bf0 2024-04-24T09:08:39.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.845+0000 7f49dc400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f49d4114670 con 0x7f49d410e8e0 2024-04-24T09:08:39.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.845+0000 7f49d9a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49d40742d0 0x7f49d4102000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:39.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.845+0000 7f49d9000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f49d4074bf0 0x7f49d4102540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:39.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.845+0000 7f49d9a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49d40742d0 0x7f49d4102000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54542/0 (socket says 172.21.15.83:54542) 2024-04-24T09:08:39.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.845+0000 7f49d9a00640 1 -- 172.21.15.83:0/3700537167 learned_addr learned my addr 172.21.15.83:0/3700537167 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:08:39.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.846+0000 7f49da400640 1 --2- 172.21.15.83:0/3700537167 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f49d410e8e0 0x7f49d4102a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:39.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.846+0000 7f49d9a00640 1 -- 172.21.15.83:0/3700537167 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f49d410e8e0 msgr2=0x7f49d4102a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:39.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.846+0000 7f49d9a00640 1 --2- 172.21.15.83:0/3700537167 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f49d410e8e0 0x7f49d4102a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:39.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.846+0000 7f49d9a00640 1 -- 172.21.15.83:0/3700537167 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f49d4074bf0 msgr2=0x7f49d4102540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:39.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.846+0000 7f49d9a00640 1 --2- 172.21.15.83:0/3700537167 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f49d4074bf0 0x7f49d4102540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:39.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.846+0000 7f49d9a00640 1 -- 172.21.15.83:0/3700537167 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49d41ab0e0 con 0x7f49d40742d0 2024-04-24T09:08:39.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.846+0000 7f49d9000640 1 --2- 172.21.15.83:0/3700537167 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f49d4074bf0 0x7f49d4102540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:08:39.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.846+0000 7f49d9a00640 1 --2- 172.21.15.83:0/3700537167 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49d40742d0 0x7f49d4102000 secure :-1 s=READY pgs=2313 cs=0 l=1 rev1=1 crypto rx=0x7f49c802aa70 tx=0x7f49c8003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:39.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.847+0000 7f49d2000640 1 -- 172.21.15.83:0/3700537167 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c803f070 con 0x7f49d40742d0 2024-04-24T09:08:39.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.847+0000 7f49dc400640 1 -- 172.21.15.83:0/3700537167 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49d41ab370 con 0x7f49d40742d0 2024-04-24T09:08:39.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.847+0000 7f49d2000640 1 -- 172.21.15.83:0/3700537167 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f49c8003ec0 con 0x7f49d40742d0 2024-04-24T09:08:39.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.847+0000 7f49d2000640 1 -- 172.21.15.83:0/3700537167 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c8035030 con 0x7f49d40742d0 2024-04-24T09:08:39.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.847+0000 7f49dc400640 1 -- 172.21.15.83:0/3700537167 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49d41ab8b0 con 0x7f49d40742d0 2024-04-24T09:08:39.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.851+0000 7f49df3ce740 1 Processor -- start 2024-04-24T09:08:39.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.852+0000 7f49df3ce740 1 -- start start 2024-04-24T09:08:39.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.852+0000 7f49d2000640 1 -- 172.21.15.83:0/3700537167 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f49c8042070 con 0x7f49d40742d0 2024-04-24T09:08:39.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.852+0000 7f49df3ce740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ddf387860 0x555ddf387c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:39.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.852+0000 7f49df3ce740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ddf388180 0x555ddf392310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:39.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.852+0000 7f49df3ce740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555ddf392850 0x555ddf394c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:39.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.852+0000 7f49df3ce740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x555ddf2ed1f0 con 0x555ddf388180 2024-04-24T09:08:39.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.852+0000 7f49df3ce740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555ddf293350 con 0x555ddf387860 2024-04-24T09:08:39.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.852+0000 7f49df3ce740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x555ddf2e91b0 con 0x555ddf392850 2024-04-24T09:08:39.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.853+0000 7f49da400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ddf387860 0x555ddf387c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:39.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.853+0000 7f49da400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ddf387860 0x555ddf387c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:53386/0 (socket says 172.21.15.83:53386) 2024-04-24T09:08:39.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.853+0000 7f49da400640 1 -- 172.21.15.83:0/1576245913 learned_addr learned my addr 172.21.15.83:0/1576245913 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:08:39.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.853+0000 7f49d9a00640 1 --2- 172.21.15.83:0/1576245913 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ddf388180 0x555ddf392310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:39.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.853+0000 7f49d9000640 1 --2- 172.21.15.83:0/1576245913 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555ddf392850 0x555ddf394c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:39.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.853+0000 7f49da400640 1 -- 172.21.15.83:0/1576245913 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555ddf392850 msgr2=0x555ddf394c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:39.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.853+0000 7f49da400640 1 --2- 172.21.15.83:0/1576245913 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555ddf392850 0x555ddf394c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:39.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.853+0000 7f49da400640 1 -- 172.21.15.83:0/1576245913 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ddf388180 msgr2=0x555ddf392310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:39.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.853+0000 7f49da400640 1 --2- 172.21.15.83:0/1576245913 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ddf388180 0x555ddf392310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:39.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.853+0000 7f49da400640 1 -- 172.21.15.83:0/1576245913 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ddf2e7f50 con 0x555ddf387860 2024-04-24T09:08:39.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.853+0000 7f49d9000640 1 --2- 172.21.15.83:0/1576245913 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555ddf392850 0x555ddf394c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:08:39.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.854+0000 7f49da400640 1 --2- 172.21.15.83:0/1576245913 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ddf387860 0x555ddf387c40 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f49cc009e30 tx=0x7f49cc010740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7627dff28f21950a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:39.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.854+0000 7f49c6a00640 1 -- 172.21.15.83:0/1576245913 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49cc004d20 con 0x555ddf387860 2024-04-24T09:08:39.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.854+0000 7f49c6a00640 1 -- 172.21.15.83:0/1576245913 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f49cc0054d0 con 0x555ddf387860 2024-04-24T09:08:39.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.854+0000 7f49c6a00640 1 -- 172.21.15.83:0/1576245913 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49cc02b8f0 con 0x555ddf387860 2024-04-24T09:08:39.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.855+0000 7f49df3ce740 1 -- 172.21.15.83:0/1576245913 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ddf387860 msgr2=0x555ddf387c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:39.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.855+0000 7f49df3ce740 1 --2- 172.21.15.83:0/1576245913 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ddf387860 0x555ddf387c40 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f49cc009e30 tx=0x7f49cc010740 comp rx=0 tx=0).stop 2024-04-24T09:08:39.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.856+0000 7f49df3ce740 1 -- 172.21.15.83:0/1576245913 shutdown_connections 2024-04-24T09:08:39.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.856+0000 7f49df3ce740 1 --2- 172.21.15.83:0/1576245913 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555ddf392850 0x555ddf394c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:39.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.856+0000 7f49df3ce740 1 --2- 172.21.15.83:0/1576245913 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ddf388180 0x555ddf392310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:39.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.856+0000 7f49df3ce740 1 --2- 172.21.15.83:0/1576245913 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ddf387860 0x555ddf387c40 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:39.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.856+0000 7f49df3ce740 1 -- 172.21.15.83:0/1576245913 >> 172.21.15.83:0/1576245913 conn(0x555ddf383b80 msgr2=0x555ddf383f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:39.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.856+0000 7f49df3ce740 1 -- 172.21.15.83:0/1576245913 shutdown_connections 2024-04-24T09:08:39.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.856+0000 7f49df3ce740 1 -- 172.21.15.83:0/1576245913 wait complete. 2024-04-24T09:08:39.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.856+0000 7f49d2000640 1 --2- 172.21.15.83:0/3700537167 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f49bc0a3c30 0x7f49bc0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:39.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.857+0000 7f49d2000640 1 -- 172.21.15.83:0/3700537167 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 9583+0+0 (secure 0 0 0) 0x7f49c80f1000 con 0x7f49d40742d0 2024-04-24T09:08:39.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.857+0000 7f49df3ce740 1 -- start start 2024-04-24T09:08:39.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.857+0000 7f49df3ce740 1 Processor -- start 2024-04-24T09:08:39.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.857+0000 7f49df3ce740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555ddf387860 0x555ddf38f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:39.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.858+0000 7f49df3ce740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ddf388180 0x555ddf38fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:39.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.858+0000 7f49df3ce740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ddf392850 0x555ddf390250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:39.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.858+0000 7f49df3ce740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x555ddf395640 con 0x555ddf388180 2024-04-24T09:08:39.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.858+0000 7f49df3ce740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555ddf2a6690 con 0x555ddf392850 2024-04-24T09:08:39.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.858+0000 7f49df3ce740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x555ddf3957c0 con 0x555ddf387860 2024-04-24T09:08:39.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.858+0000 7f49d9000640 1 --2- 172.21.15.83:0/3700537167 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f49bc0a3c30 0x7f49bc0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:39.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.858+0000 7f49df3ce740 10 client.0 _ll_register_callbacks cb 0x555ddf2e2650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:08:39.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.858+0000 7f49da400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ddf392850 0x555ddf390250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:39.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.858+0000 7f49da400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ddf392850 0x555ddf390250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:53392/0 (socket says 172.21.15.83:53392) 2024-04-24T09:08:39.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.858+0000 7f49da400640 1 -- 172.21.15.83:0/799926499 learned_addr learned my addr 172.21.15.83:0/799926499 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:08:39.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.858+0000 7f49d9000640 1 --2- 172.21.15.83:0/799926499 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ddf388180 0x555ddf38fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:39.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.858+0000 7f49d9a00640 1 --2- 172.21.15.83:0/799926499 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555ddf387860 0x555ddf38f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:39.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.859+0000 7f49d9000640 1 -- 172.21.15.83:0/799926499 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555ddf387860 msgr2=0x555ddf38f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:39.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.859+0000 7f49d9000640 1 --2- 172.21.15.83:0/799926499 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555ddf387860 0x555ddf38f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:39.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.859+0000 7f49d9000640 1 -- 172.21.15.83:0/799926499 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ddf392850 msgr2=0x555ddf390250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:39.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.859+0000 7f49d9000640 1 --2- 172.21.15.83:0/799926499 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ddf392850 0x555ddf390250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:39.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.859+0000 7f49d9000640 1 -- 172.21.15.83:0/799926499 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ddf2a5920 con 0x555ddf388180 2024-04-24T09:08:39.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.859+0000 7f49da400640 1 --2- 172.21.15.83:0/799926499 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ddf392850 0x555ddf390250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:08:39.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.859+0000 7f49d9000640 1 --2- 172.21.15.83:0/3700537167 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f49bc0a3c30 0x7f49bc0a60f0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f49c000bfd0 tx=0x7f49c000bf60 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:39.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.859+0000 7f49d9000640 1 --2- 172.21.15.83:0/799926499 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ddf388180 0x555ddf38fd10 secure :-1 s=READY pgs=2314 cs=0 l=1 rev1=1 crypto rx=0x7f49d41037c0 tx=0x7f49c0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ac3e7e49c4ba4b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:39.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.859+0000 7f49bb400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:08:39.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.860+0000 7f49bb400640 1 -- 172.21.15.83:0/799926499 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c0025070 con 0x555ddf388180 2024-04-24T09:08:39.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.860+0000 7f49bb400640 1 -- 172.21.15.83:0/799926499 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9583+0+0 (secure 0 0 0) 0x7f49c0020de0 con 0x555ddf388180 2024-04-24T09:08:39.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.860+0000 7f49df3ce740 1 -- 172.21.15.83:0/799926499 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49cc0054d0 con 0x555ddf388180 2024-04-24T09:08:39.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.860+0000 7f49bb400640 1 -- 172.21.15.83:0/799926499 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f49c0004020 con 0x555ddf388180 2024-04-24T09:08:39.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.860+0000 7f49df3ce740 10 client.21504 fetch_fsmap learned FSMap version 58 2024-04-24T09:08:39.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.860+0000 7f49df3ce740 1 -- 172.21.15.83:0/799926499 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x555ddf0c97d0 con 0x555ddf388180 2024-04-24T09:08:39.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.861+0000 7f49bb400640 1 -- 172.21.15.83:0/799926499 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f49cc0054d0 con 0x555ddf388180 2024-04-24T09:08:39.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.861+0000 7f49bb400640 1 -- 172.21.15.83:0/799926499 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 58) ==== 8374+0+0 (secure 0 0 0) 0x7f49c0020050 con 0x555ddf388180 2024-04-24T09:08:39.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.861+0000 7f49df3ce740 10 client.21504 fetch_fsmap finished waiting for FSMap version 58 2024-04-24T09:08:39.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.861+0000 7f49df3ce740 10 client.21504 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:08:39.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.861+0000 7f49df3ce740 20 client.21504 populate_metadata read hostname 'smithi083' 2024-04-24T09:08:39.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.861+0000 7f49df3ce740 1 --2- 172.21.15.83:0/799926499 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x555ddf3a2c50 0x555ddf3a5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:39.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.861+0000 7f49df3ce740 4 client.21504 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:08:39.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.861+0000 7f49df3ce740 1 -- 172.21.15.83:0/799926499 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555ddf2fac10 con 0x555ddf3a2c50 2024-04-24T09:08:39.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.861+0000 7f49d9a00640 1 --2- 172.21.15.83:0/799926499 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x555ddf3a2c50 0x555ddf3a5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:39.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.862+0000 7f49bb400640 10 client.21504 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:08:39.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.862+0000 7f49d9a00640 1 --2- 172.21.15.83:0/799926499 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x555ddf3a2c50 0x555ddf3a5060 crc :-1 s=READY pgs=429 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39f48b2763e62eaa server_cookie=3021a46ca0ddc188 in_seq=0 out_seq=0 2024-04-24T09:08:39.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.864+0000 7f49bb400640 1 -- 172.21.15.83:0/799926499 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555ddf2fac10 con 0x555ddf3a2c50 2024-04-24T09:08:39.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.864+0000 7f49bb400640 10 client.21504 handle_command_reply: tid=0 2024-04-24T09:08:39.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.864+0000 7f49df3ce740 1 client.21504 shutdown 2024-04-24T09:08:39.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.864+0000 7f49df3ce740 1 -- 172.21.15.83:0/799926499 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ddf388180 msgr2=0x555ddf38fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:39.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.864+0000 7f49df3ce740 1 --2- 172.21.15.83:0/799926499 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ddf388180 0x555ddf38fd10 secure :-1 s=READY pgs=2314 cs=0 l=1 rev1=1 crypto rx=0x7f49d41037c0 tx=0x7f49c0004730 comp rx=0 tx=0).stop 2024-04-24T09:08:39.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.864+0000 7f49df3ce740 1 -- 172.21.15.83:0/799926499 shutdown_connections 2024-04-24T09:08:39.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.864+0000 7f49df3ce740 1 --2- 172.21.15.83:0/799926499 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x555ddf3a2c50 0x555ddf3a5060 crc :-1 s=READY pgs=429 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:39.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.864+0000 7f49df3ce740 1 --2- 172.21.15.83:0/799926499 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ddf392850 0x555ddf390250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:39.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.864+0000 7f49df3ce740 1 --2- 172.21.15.83:0/799926499 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ddf388180 0x555ddf38fd10 secure :-1 s=CLOSED pgs=2314 cs=0 l=1 rev1=1 crypto rx=0x7f49d41037c0 tx=0x7f49c0004730 comp rx=0 tx=0).stop 2024-04-24T09:08:39.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.864+0000 7f49df3ce740 1 --2- 172.21.15.83:0/799926499 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555ddf387860 0x555ddf38f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:39.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.864+0000 7f49df3ce740 1 -- 172.21.15.83:0/799926499 >> 172.21.15.83:0/799926499 conn(0x555ddf383b80 msgr2=0x555ddf389880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:39.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.864+0000 7f49bb400640 0 client.21504 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:08:39.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.865+0000 7f49df3ce740 1 -- 172.21.15.83:0/799926499 shutdown_connections 2024-04-24T09:08:39.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.865+0000 7f49df3ce740 1 -- 172.21.15.83:0/799926499 wait complete. 2024-04-24T09:08:39.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.865+0000 7f49df3ce740 20 client.21504 trim_cache size 0 max 16384 2024-04-24T09:08:39.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.896+0000 7f49df3ce740 1 Processor -- start 2024-04-24T09:08:39.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.896+0000 7f49df3ce740 1 -- start start 2024-04-24T09:08:39.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.896+0000 7f49df3ce740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555ddf387860 0x555ddf39e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:39.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.896+0000 7f49df3ce740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ddf392850 0x555ddf38fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:39.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.896+0000 7f49df3ce740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ddf3903e0 0x555ddf3aa970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:39.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.896+0000 7f49df3ce740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f49b4010110 con 0x555ddf3903e0 2024-04-24T09:08:39.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.896+0000 7f49df3ce740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f49b4011c50 con 0x555ddf392850 2024-04-24T09:08:39.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.896+0000 7f49df3ce740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f49b40107e0 con 0x555ddf387860 2024-04-24T09:08:39.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.897+0000 7f49da400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ddf3903e0 0x555ddf3aa970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:39.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.897+0000 7f49d9000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ddf392850 0x555ddf38fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:39.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.897+0000 7f49da400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ddf3903e0 0x555ddf3aa970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54566/0 (socket says 172.21.15.83:54566) 2024-04-24T09:08:39.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.897+0000 7f49da400640 1 -- 172.21.15.83:0/3216993389 learned_addr learned my addr 172.21.15.83:0/3216993389 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:08:39.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.897+0000 7f49d9a00640 1 --2- 172.21.15.83:0/3216993389 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555ddf387860 0x555ddf39e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:39.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.898+0000 7f49da400640 1 -- 172.21.15.83:0/3216993389 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555ddf387860 msgr2=0x555ddf39e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:39.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.898+0000 7f49da400640 1 --2- 172.21.15.83:0/3216993389 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555ddf387860 0x555ddf39e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:39.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.898+0000 7f49da400640 1 -- 172.21.15.83:0/3216993389 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ddf392850 msgr2=0x555ddf38fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:39.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.898+0000 7f49da400640 1 --2- 172.21.15.83:0/3216993389 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ddf392850 0x555ddf38fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:39.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.898+0000 7f49da400640 1 -- 172.21.15.83:0/3216993389 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ddefff3f0 con 0x555ddf3903e0 2024-04-24T09:08:39.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.898+0000 7f49d9000640 1 --2- 172.21.15.83:0/3216993389 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ddf392850 0x555ddf38fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:08:39.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.898+0000 7f49da400640 1 --2- 172.21.15.83:0/3216993389 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ddf3903e0 0x555ddf3aa970 secure :-1 s=READY pgs=2315 cs=0 l=1 rev1=1 crypto rx=0x7f49cc009d50 tx=0x7f49cc005c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=725fdde89e45dc45 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:39.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.899+0000 7f49c6000640 1 -- 172.21.15.83:0/3216993389 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49cc03b070 con 0x555ddf3903e0 2024-04-24T09:08:39.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.899+0000 7f49c6000640 1 -- 172.21.15.83:0/3216993389 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f49cc02bbf0 con 0x555ddf3903e0 2024-04-24T09:08:39.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.899+0000 7f49c6000640 1 -- 172.21.15.83:0/3216993389 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49cc036df0 con 0x555ddf3903e0 2024-04-24T09:08:39.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.900+0000 7f49df3ce740 1 -- 172.21.15.83:0/3216993389 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ddf3903e0 msgr2=0x555ddf3aa970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:39.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.900+0000 7f49df3ce740 1 --2- 172.21.15.83:0/3216993389 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ddf3903e0 0x555ddf3aa970 secure :-1 s=READY pgs=2315 cs=0 l=1 rev1=1 crypto rx=0x7f49cc009d50 tx=0x7f49cc005c80 comp rx=0 tx=0).stop 2024-04-24T09:08:39.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.900+0000 7f49df3ce740 1 -- 172.21.15.83:0/3216993389 shutdown_connections 2024-04-24T09:08:39.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.900+0000 7f49df3ce740 1 --2- 172.21.15.83:0/3216993389 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ddf3903e0 0x555ddf3aa970 unknown :-1 s=CLOSED pgs=2315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:39.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.900+0000 7f49df3ce740 1 --2- 172.21.15.83:0/3216993389 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ddf392850 0x555ddf38fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:39.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.900+0000 7f49df3ce740 1 --2- 172.21.15.83:0/3216993389 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555ddf387860 0x555ddf39e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:39.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.900+0000 7f49df3ce740 1 -- 172.21.15.83:0/3216993389 >> 172.21.15.83:0/3216993389 conn(0x555ddf383b80 msgr2=0x555ddf38f790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:39.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.900+0000 7f49df3ce740 1 -- 172.21.15.83:0/3216993389 shutdown_connections 2024-04-24T09:08:39.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.900+0000 7f49df3ce740 1 -- 172.21.15.83:0/3216993389 wait complete. 2024-04-24T09:08:39.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.901+0000 7f49df3ce740 1 -- start start 2024-04-24T09:08:39.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.901+0000 7f49df3ce740 1 Processor -- start 2024-04-24T09:08:39.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.902+0000 7f49df3ce740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ddf387860 0x555ddf39ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:39.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.902+0000 7f49df3ce740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ddf3903e0 0x555ddf3961b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:39.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.902+0000 7f49df3ce740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555ddf392850 0x555ddf3966f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:39.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.902+0000 7f49df3ce740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f49b4011580 con 0x555ddf3903e0 2024-04-24T09:08:39.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.902+0000 7f49df3ce740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f49b4010eb0 con 0x555ddf387860 2024-04-24T09:08:39.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.902+0000 7f49df3ce740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f49b400e110 con 0x555ddf392850 2024-04-24T09:08:39.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.902+0000 7f49df3ce740 10 client.0 _ll_register_callbacks cb 0x7f49c0013840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:08:39.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.902+0000 7f49d9000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ddf3903e0 0x555ddf3961b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:39.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.902+0000 7f49d9000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ddf3903e0 0x555ddf3961b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54574/0 (socket says 172.21.15.83:54574) 2024-04-24T09:08:39.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.902+0000 7f49d9000640 1 -- 172.21.15.83:0/1944330852 learned_addr learned my addr 172.21.15.83:0/1944330852 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:08:39.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.902+0000 7f49d9a00640 1 --2- 172.21.15.83:0/1944330852 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ddf387860 0x555ddf39ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:39.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.903+0000 7f49da400640 1 --2- 172.21.15.83:0/1944330852 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555ddf392850 0x555ddf3966f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:39.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.903+0000 7f49d9000640 1 -- 172.21.15.83:0/1944330852 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555ddf392850 msgr2=0x555ddf3966f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:39.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.903+0000 7f49d9000640 1 --2- 172.21.15.83:0/1944330852 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555ddf392850 0x555ddf3966f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:39.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.903+0000 7f49d9000640 1 -- 172.21.15.83:0/1944330852 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ddf387860 msgr2=0x555ddf39ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:39.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.903+0000 7f49d9000640 1 --2- 172.21.15.83:0/1944330852 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ddf387860 0x555ddf39ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:39.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.903+0000 7f49d9000640 1 -- 172.21.15.83:0/1944330852 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ddefbf0f0 con 0x555ddf3903e0 2024-04-24T09:08:39.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.903+0000 7f49bb400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:08:39.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.903+0000 7f49d9000640 1 --2- 172.21.15.83:0/1944330852 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ddf3903e0 0x555ddf3961b0 secure :-1 s=READY pgs=2316 cs=0 l=1 rev1=1 crypto rx=0x7f49c0003ea0 tx=0x7f49c0011d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61a17fae5d069eaf server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:39.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.904+0000 7f49bb400640 1 -- 172.21.15.83:0/1944330852 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c003b060 con 0x555ddf3903e0 2024-04-24T09:08:39.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.904+0000 7f49bb400640 1 -- 172.21.15.83:0/1944330852 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9583+0+0 (secure 0 0 0) 0x7f49c0033070 con 0x555ddf3903e0 2024-04-24T09:08:39.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.904+0000 7f49df3ce740 1 -- 172.21.15.83:0/1944330852 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49cc02bbf0 con 0x555ddf3903e0 2024-04-24T09:08:39.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.904+0000 7f49bb400640 1 -- 172.21.15.83:0/1944330852 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f49c0042050 con 0x555ddf3903e0 2024-04-24T09:08:39.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.904+0000 7f49df3ce740 10 client.21510 fetch_fsmap learned FSMap version 58 2024-04-24T09:08:39.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.904+0000 7f49df3ce740 1 -- 172.21.15.83:0/1944330852 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x555ddf2b2810 con 0x555ddf3903e0 2024-04-24T09:08:39.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.905+0000 7f49bb400640 1 -- 172.21.15.83:0/1944330852 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f49cc02bbf0 con 0x555ddf3903e0 2024-04-24T09:08:39.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.905+0000 7f49bb400640 1 -- 172.21.15.83:0/1944330852 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 58) ==== 8374+0+0 (secure 0 0 0) 0x7f49c0029490 con 0x555ddf3903e0 2024-04-24T09:08:39.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.905+0000 7f49df3ce740 10 client.21510 fetch_fsmap finished waiting for FSMap version 58 2024-04-24T09:08:39.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.905+0000 7f49df3ce740 10 client.21510 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:08:39.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.905+0000 7f49df3ce740 20 client.21510 populate_metadata read hostname 'smithi083' 2024-04-24T09:08:39.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.905+0000 7f49df3ce740 1 --2- 172.21.15.83:0/1944330852 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x555ddf409500 0x555ddf40b920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:39.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.905+0000 7f49df3ce740 4 client.21510 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T09:08:39.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.905+0000 7f49df3ce740 1 -- 172.21.15.83:0/1944330852 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555ddf2f4a80 con 0x555ddf409500 2024-04-24T09:08:39.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.906+0000 7f49d9a00640 1 --2- 172.21.15.83:0/1944330852 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x555ddf409500 0x555ddf40b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:39.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.906+0000 7f49bb400640 10 client.21510 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:08:39.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.906+0000 7f49d9a00640 1 --2- 172.21.15.83:0/1944330852 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x555ddf409500 0x555ddf40b920 crc :-1 s=READY pgs=430 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=330cd9a65518170 server_cookie=25a2bf3aa8e6e050 in_seq=0 out_seq=0 2024-04-24T09:08:39.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.906+0000 7f49bb400640 1 -- 172.21.15.83:0/1944330852 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x555ddf2f4a80 con 0x555ddf409500 2024-04-24T09:08:39.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.906+0000 7f49bb400640 10 client.21510 handle_command_reply: tid=0 2024-04-24T09:08:39.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.907+0000 7f49df3ce740 1 client.21510 shutdown 2024-04-24T09:08:39.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.907+0000 7f49df3ce740 1 -- 172.21.15.83:0/1944330852 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ddf3903e0 msgr2=0x555ddf3961b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:39.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.907+0000 7f49df3ce740 1 --2- 172.21.15.83:0/1944330852 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ddf3903e0 0x555ddf3961b0 secure :-1 s=READY pgs=2316 cs=0 l=1 rev1=1 crypto rx=0x7f49c0003ea0 tx=0x7f49c0011d00 comp rx=0 tx=0).stop 2024-04-24T09:08:39.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.907+0000 7f49df3ce740 1 -- 172.21.15.83:0/1944330852 shutdown_connections 2024-04-24T09:08:39.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.907+0000 7f49df3ce740 1 --2- 172.21.15.83:0/1944330852 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x555ddf409500 0x555ddf40b920 crc :-1 s=READY pgs=430 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:39.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.907+0000 7f49df3ce740 1 --2- 172.21.15.83:0/1944330852 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555ddf392850 0x555ddf3966f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:39.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.907+0000 7f49df3ce740 1 --2- 172.21.15.83:0/1944330852 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ddf3903e0 0x555ddf3961b0 unknown :-1 s=CLOSED pgs=2316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:39.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.907+0000 7f49df3ce740 1 --2- 172.21.15.83:0/1944330852 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ddf387860 0x555ddf39ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:39.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.907+0000 7f49df3ce740 1 -- 172.21.15.83:0/1944330852 >> 172.21.15.83:0/1944330852 conn(0x555ddf383b80 msgr2=0x555ddf38e500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:39.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.907+0000 7f49bb400640 0 client.21510 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:08:39.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.907+0000 7f49df3ce740 1 -- 172.21.15.83:0/1944330852 shutdown_connections 2024-04-24T09:08:39.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.907+0000 7f49df3ce740 1 -- 172.21.15.83:0/1944330852 wait complete. 2024-04-24T09:08:39.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.907+0000 7f49df3ce740 20 client.21510 trim_cache size 0 max 16384 2024-04-24T09:08:39.909 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T09:08:39.909 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-04-24T09:08:39.909 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-04-24T09:08:39.909 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T09:08:39.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.908+0000 7f49dc400640 1 -- 172.21.15.83:0/3700537167 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f49bc0a3c30 msgr2=0x7f49bc0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:39.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.908+0000 7f49dc400640 1 --2- 172.21.15.83:0/3700537167 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f49bc0a3c30 0x7f49bc0a60f0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f49c000bfd0 tx=0x7f49c000bf60 comp rx=0 tx=0).stop 2024-04-24T09:08:39.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.908+0000 7f49dc400640 1 -- 172.21.15.83:0/3700537167 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49d40742d0 msgr2=0x7f49d4102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:39.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.908+0000 7f49dc400640 1 --2- 172.21.15.83:0/3700537167 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49d40742d0 0x7f49d4102000 secure :-1 s=READY pgs=2313 cs=0 l=1 rev1=1 crypto rx=0x7f49c802aa70 tx=0x7f49c8003cd0 comp rx=0 tx=0).stop 2024-04-24T09:08:39.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.908+0000 7f49dc400640 1 -- 172.21.15.83:0/3700537167 shutdown_connections 2024-04-24T09:08:39.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.908+0000 7f49dc400640 1 --2- 172.21.15.83:0/3700537167 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f49bc0a3c30 0x7f49bc0a60f0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:39.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.908+0000 7f49dc400640 1 --2- 172.21.15.83:0/3700537167 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f49d410e8e0 0x7f49d4102a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:39.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.908+0000 7f49dc400640 1 --2- 172.21.15.83:0/3700537167 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f49d4074bf0 0x7f49d4102540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:39.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.908+0000 7f49dc400640 1 --2- 172.21.15.83:0/3700537167 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49d40742d0 0x7f49d4102000 unknown :-1 s=CLOSED pgs=2313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:39.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.908+0000 7f49dc400640 1 -- 172.21.15.83:0/3700537167 >> 172.21.15.83:0/3700537167 conn(0x7f49d4079450 msgr2=0x7f49d40fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:39.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.909+0000 7f49dc400640 1 -- 172.21.15.83:0/3700537167 shutdown_connections 2024-04-24T09:08:39.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:39.909+0000 7f49dc400640 1 -- 172.21.15.83:0/3700537167 wait complete. 2024-04-24T09:08:39.930 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T09:08:39.930 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T09:08:40.167 INFO:tasks.workunit.client.0.smithi083.stdout: 10 24632 6.92 MB/sec execute 71 sec latency 77.081 ms 2024-04-24T09:08:40.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.187+0000 7f5ef1e00640 1 -- 172.21.15.83:0/2297954589 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5eec104440 msgr2=0x7f5eec104820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:40.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.188+0000 7f5ef1e00640 1 --2- 172.21.15.83:0/2297954589 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5eec104440 0x7f5eec104820 secure :-1 s=READY pgs=2317 cs=0 l=1 rev1=1 crypto rx=0x7f5edc009900 tx=0x7f5edc02a3f0 comp rx=0 tx=0).stop 2024-04-24T09:08:40.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.189+0000 7f5ef1e00640 1 -- 172.21.15.83:0/2297954589 shutdown_connections 2024-04-24T09:08:40.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.189+0000 7f5ef1e00640 1 --2- 172.21.15.83:0/2297954589 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5eec109510 0x7f5eec1110e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:40.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.189+0000 7f5ef1e00640 1 --2- 172.21.15.83:0/2297954589 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5eec104df0 0x7f5eec108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:40.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.189+0000 7f5ef1e00640 1 --2- 172.21.15.83:0/2297954589 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5eec104440 0x7f5eec104820 unknown :-1 s=CLOSED pgs=2317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:40.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.189+0000 7f5ef1e00640 1 -- 172.21.15.83:0/2297954589 >> 172.21.15.83:0/2297954589 conn(0x7f5eec0ffcc0 msgr2=0x7f5eec1020e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:40.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.190+0000 7f5ef1e00640 1 -- 172.21.15.83:0/2297954589 shutdown_connections 2024-04-24T09:08:40.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.191+0000 7f5ef1e00640 1 -- 172.21.15.83:0/2297954589 wait complete. 2024-04-24T09:08:40.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.192+0000 7f5ef1e00640 1 Processor -- start 2024-04-24T09:08:40.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.193+0000 7f5ef1e00640 1 -- start start 2024-04-24T09:08:40.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.195+0000 7f5ef1e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5eec104440 0x7f5eec110790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:40.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.195+0000 7f5ef1e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5eec109510 0x7f5eec110cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:40.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.195+0000 7f5ef1e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5eec1a38c0 0x7f5eec1a3d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:40.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.195+0000 7f5ef1e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5eec0780f0 con 0x7f5eec1a38c0 2024-04-24T09:08:40.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.195+0000 7f5ef1e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5eec077f70 con 0x7f5eec104440 2024-04-24T09:08:40.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.195+0000 7f5ef1e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5eec078270 con 0x7f5eec109510 2024-04-24T09:08:40.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.196+0000 7f5ee3e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5eec109510 0x7f5eec110cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:40.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.196+0000 7f5eeaa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5eec104440 0x7f5eec110790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:40.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.196+0000 7f5eeaa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5eec104440 0x7f5eec110790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:53434/0 (socket says 172.21.15.83:53434) 2024-04-24T09:08:40.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.196+0000 7f5eeaa00640 1 -- 172.21.15.83:0/2886398826 learned_addr learned my addr 172.21.15.83:0/2886398826 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:08:40.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.196+0000 7f5eeb400640 1 --2- 172.21.15.83:0/2886398826 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5eec1a38c0 0x7f5eec1a3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:40.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.197+0000 7f5ee3e00640 1 -- 172.21.15.83:0/2886398826 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5eec104440 msgr2=0x7f5eec110790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:40.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.197+0000 7f5ee3e00640 1 --2- 172.21.15.83:0/2886398826 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5eec104440 0x7f5eec110790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:40.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.197+0000 7f5ee3e00640 1 -- 172.21.15.83:0/2886398826 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5eec1a38c0 msgr2=0x7f5eec1a3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:40.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.197+0000 7f5ee3e00640 1 --2- 172.21.15.83:0/2886398826 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5eec1a38c0 0x7f5eec1a3d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:40.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.198+0000 7f5ee3e00640 1 -- 172.21.15.83:0/2886398826 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5eec1a43b0 con 0x7f5eec109510 2024-04-24T09:08:40.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.199+0000 7f5ee3e00640 1 --2- 172.21.15.83:0/2886398826 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5eec109510 0x7f5eec110cd0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5ed400b3c0 tx=0x7f5ed400b880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:40.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.201+0000 7f5ef0a00640 1 -- 172.21.15.83:0/2886398826 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ed4011830 con 0x7f5eec109510 2024-04-24T09:08:40.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.201+0000 7f5ef0a00640 1 -- 172.21.15.83:0/2886398826 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5ed4016070 con 0x7f5eec109510 2024-04-24T09:08:40.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.201+0000 7f5ef0a00640 1 -- 172.21.15.83:0/2886398826 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ed4010e10 con 0x7f5eec109510 2024-04-24T09:08:40.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.201+0000 7f5ef1e00640 1 -- 172.21.15.83:0/2886398826 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5eec1a4640 con 0x7f5eec109510 2024-04-24T09:08:40.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.201+0000 7f5ef1e00640 1 -- 172.21.15.83:0/2886398826 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5eec1abdf0 con 0x7f5eec109510 2024-04-24T09:08:40.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.204+0000 7f5ef1e00640 1 -- 172.21.15.83:0/2886398826 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5eec069270 con 0x7f5eec109510 2024-04-24T09:08:40.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.208+0000 7f5ef0a00640 1 -- 172.21.15.83:0/2886398826 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5ed40119d0 con 0x7f5eec109510 2024-04-24T09:08:40.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.210+0000 7f5ef0a00640 1 --2- 172.21.15.83:0/2886398826 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f5ed00a3ea0 0x7f5ed00a6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:40.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.210+0000 7f5ef0a00640 1 -- 172.21.15.83:0/2886398826 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 9583+0+0 (secure 0 0 0) 0x7f5ed40d01f0 con 0x7f5eec109510 2024-04-24T09:08:40.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.211+0000 7f5eeaa00640 1 --2- 172.21.15.83:0/2886398826 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f5ed00a3ea0 0x7f5ed00a6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:40.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.211+0000 7f5eeaa00640 1 --2- 172.21.15.83:0/2886398826 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f5ed00a3ea0 0x7f5ed00a6360 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f5edc0096c0 tx=0x7f5edc004040 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:40.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.213+0000 7f5ef0a00640 1 -- 172.21.15.83:0/2886398826 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5ed409c600 con 0x7f5eec109510 2024-04-24T09:08:40.506 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:08:40 smithi138 ceph-mon[31441]: pgmap v536: 129 pgs: 129 active+clean; 496 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.9 MiB/s wr, 338 op/s 2024-04-24T09:08:40.552 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:08:40 smithi083 ceph-mon[27907]: pgmap v536: 129 pgs: 129 active+clean; 496 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.9 MiB/s wr, 338 op/s 2024-04-24T09:08:40.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.629+0000 7f5ef1e00640 1 -- 172.21.15.83:0/2886398826 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5eec104df0 con 0x7f5eec109510 2024-04-24T09:08:40.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.632+0000 7f5ef0a00640 1 -- 172.21.15.83:0/2886398826 <== mon.1 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) ==== 94+0+11949 (secure 0 0 0) 0x7f5ed409bf00 con 0x7f5eec109510 2024-04-24T09:08:40.631 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T09:08:40.631 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":58,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/358872257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":358872257},{"type":"v1","addr":"172.21.15.83:6835","nonce":358872257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3157070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3157070851},{"type":"v1","addr":"172.21.15.83:6837","nonce":3157070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":58,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T08:48:29.422576+0000","modified":"2024-04-24T09:08:27.108181+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34520,"mds_1":34493,"mds_2":34451,"mds_3":34427,"mds_4":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14658":{"gid":14658,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6839/4085470844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":4085470844},{"type":"v1","addr":"172.21.15.83:6839","nonce":4085470844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14676":{"gid":14676,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6841/2634754210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":2634754210},{"type":"v1","addr":"172.21.15.83:6841","nonce":2634754210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24382":{"gid":24382,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/1859921310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1859921310},{"type":"v1","addr":"172.21.15.189:6833","nonce":1859921310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/2241923197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2241923197},{"type":"v1","addr":"172.21.15.189:6835","nonce":2241923197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.189:6839/2548965336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":2548965336},{"type":"v1","addr":"172.21.15.189:6839","nonce":2548965336}]},"join_fscid":-1,"export_targets":[0,1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26686":{"gid":26686,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6837/1678817161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":1678817161},{"type":"v1","addr":"172.21.15.189:6837","nonce":1678817161}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.138:6833/1157967658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1157967658},{"type":"v1","addr":"172.21.15.138:6833","nonce":1157967658}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.138:6835/1909219758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1909219758},{"type":"v1","addr":"172.21.15.138:6835","nonce":1909219758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34493":{"gid":34493,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.138:6837/900566754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":900566754},{"type":"v1","addr":"172.21.15.138:6837","nonce":900566754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34520":{"gid":34520,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.138:6839/2236932393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2236932393},{"type":"v1","addr":"172.21.15.138:6839","nonce":2236932393}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34520,"qdb_cluster":[34493,34451,34427,24454,34520]},"id":1}]} 2024-04-24T09:08:40.631 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 58 2024-04-24T09:08:40.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.635+0000 7f5ef1e00640 1 -- 172.21.15.83:0/2886398826 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f5ed00a3ea0 msgr2=0x7f5ed00a6360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:40.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.635+0000 7f5ef1e00640 1 --2- 172.21.15.83:0/2886398826 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f5ed00a3ea0 0x7f5ed00a6360 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f5edc0096c0 tx=0x7f5edc004040 comp rx=0 tx=0).stop 2024-04-24T09:08:40.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.635+0000 7f5ef1e00640 1 -- 172.21.15.83:0/2886398826 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5eec109510 msgr2=0x7f5eec110cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:40.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.635+0000 7f5ef1e00640 1 --2- 172.21.15.83:0/2886398826 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5eec109510 0x7f5eec110cd0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5ed400b3c0 tx=0x7f5ed400b880 comp rx=0 tx=0).stop 2024-04-24T09:08:40.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.635+0000 7f5ef1e00640 1 -- 172.21.15.83:0/2886398826 shutdown_connections 2024-04-24T09:08:40.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.635+0000 7f5ef1e00640 1 --2- 172.21.15.83:0/2886398826 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f5ed00a3ea0 0x7f5ed00a6360 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:40.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.635+0000 7f5ef1e00640 1 --2- 172.21.15.83:0/2886398826 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5eec1a38c0 0x7f5eec1a3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:40.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.635+0000 7f5ef1e00640 1 --2- 172.21.15.83:0/2886398826 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5eec109510 0x7f5eec110cd0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:40.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.635+0000 7f5ef1e00640 1 --2- 172.21.15.83:0/2886398826 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5eec104440 0x7f5eec110790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:40.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.635+0000 7f5ef1e00640 1 -- 172.21.15.83:0/2886398826 >> 172.21.15.83:0/2886398826 conn(0x7f5eec0ffcc0 msgr2=0x7f5eec0798a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:40.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.636+0000 7f5ef1e00640 1 -- 172.21.15.83:0/2886398826 shutdown_connections 2024-04-24T09:08:40.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.636+0000 7f5ef1e00640 1 -- 172.21.15.83:0/2886398826 wait complete. 2024-04-24T09:08:40.657 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-24T09:08:40.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:08:40 smithi189 ceph-mon[31791]: pgmap v536: 129 pgs: 129 active+clean; 496 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.9 MiB/s wr, 338 op/s 2024-04-24T09:08:40.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.912+0000 7f376b200640 1 -- 172.21.15.83:0/3061442281 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f376c076530 msgr2=0x7f376c076910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:40.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.912+0000 7f376b200640 1 --2- 172.21.15.83:0/3061442281 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f376c076530 0x7f376c076910 secure :-1 s=READY pgs=2318 cs=0 l=1 rev1=1 crypto rx=0x7f375c009900 tx=0x7f375c02a3f0 comp rx=0 tx=0).stop 2024-04-24T09:08:40.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.912+0000 7f376b200640 1 -- 172.21.15.83:0/3061442281 shutdown_connections 2024-04-24T09:08:40.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.912+0000 7f376b200640 1 --2- 172.21.15.83:0/3061442281 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f376c112d50 0x7f376c115140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:40.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.912+0000 7f376b200640 1 --2- 172.21.15.83:0/3061442281 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f376c076e50 0x7f376c112810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:40.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.912+0000 7f376b200640 1 --2- 172.21.15.83:0/3061442281 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f376c076530 0x7f376c076910 unknown :-1 s=CLOSED pgs=2318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:40.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.912+0000 7f376b200640 1 -- 172.21.15.83:0/3061442281 >> 172.21.15.83:0/3061442281 conn(0x7f376c0ffb40 msgr2=0x7f376c101f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:40.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.912+0000 7f376b200640 1 -- 172.21.15.83:0/3061442281 shutdown_connections 2024-04-24T09:08:40.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.912+0000 7f376b200640 1 -- 172.21.15.83:0/3061442281 wait complete. 2024-04-24T09:08:40.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.914+0000 7f376b200640 1 Processor -- start 2024-04-24T09:08:40.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.914+0000 7f376b200640 1 -- start start 2024-04-24T09:08:40.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.915+0000 7f376b200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f376c076e50 0x7f376c19eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:40.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.915+0000 7f376b200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f376c112d50 0x7f376c19f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:40.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.915+0000 7f376b200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f376c1a3720 0x7f376c1a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:40.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.915+0000 7f376b200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f376c1187a0 con 0x7f376c076e50 2024-04-24T09:08:40.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.915+0000 7f376b200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f376c118620 con 0x7f376c1a3720 2024-04-24T09:08:40.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.915+0000 7f376b200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f376c118920 con 0x7f376c112d50 2024-04-24T09:08:40.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.916+0000 7f3769e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f376c076e50 0x7f376c19eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:40.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.916+0000 7f376a800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f376c1a3720 0x7f376c1a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:40.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.916+0000 7f376a800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f376c1a3720 0x7f376c1a3bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:53450/0 (socket says 172.21.15.83:53450) 2024-04-24T09:08:40.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.916+0000 7f376a800640 1 -- 172.21.15.83:0/3302889864 learned_addr learned my addr 172.21.15.83:0/3302889864 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:08:40.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.916+0000 7f3769400640 1 --2- 172.21.15.83:0/3302889864 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f376c112d50 0x7f376c19f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:40.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.916+0000 7f376a800640 1 -- 172.21.15.83:0/3302889864 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f376c112d50 msgr2=0x7f376c19f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:40.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.916+0000 7f376a800640 1 --2- 172.21.15.83:0/3302889864 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f376c112d50 0x7f376c19f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:40.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.916+0000 7f376a800640 1 -- 172.21.15.83:0/3302889864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f376c076e50 msgr2=0x7f376c19eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:40.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.916+0000 7f376a800640 1 --2- 172.21.15.83:0/3302889864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f376c076e50 0x7f376c19eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:40.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.916+0000 7f376a800640 1 -- 172.21.15.83:0/3302889864 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f376c1a4210 con 0x7f376c1a3720 2024-04-24T09:08:40.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.917+0000 7f376a800640 1 --2- 172.21.15.83:0/3302889864 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f376c1a3720 0x7f376c1a3bd0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f376400ccb0 tx=0x7f376400a620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:40.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.917+0000 7f3762a00640 1 -- 172.21.15.83:0/3302889864 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f376400f450 con 0x7f376c1a3720 2024-04-24T09:08:40.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.918+0000 7f3762a00640 1 -- 172.21.15.83:0/3302889864 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f376400faa0 con 0x7f376c1a3720 2024-04-24T09:08:40.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.918+0000 7f3762a00640 1 -- 172.21.15.83:0/3302889864 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f376400f450 con 0x7f376c1a3720 2024-04-24T09:08:40.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.918+0000 7f376b200640 1 -- 172.21.15.83:0/3302889864 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f376c1a4500 con 0x7f376c1a3720 2024-04-24T09:08:40.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.918+0000 7f376b200640 1 -- 172.21.15.83:0/3302889864 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f376c1abc40 con 0x7f376c1a3720 2024-04-24T09:08:40.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.923+0000 7f3762a00640 1 -- 172.21.15.83:0/3302889864 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f376400fc50 con 0x7f376c1a3720 2024-04-24T09:08:40.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.923+0000 7f3773de7740 1 Processor -- start 2024-04-24T09:08:40.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.923+0000 7f3773de7740 1 -- start start 2024-04-24T09:08:40.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.924+0000 7f3773de7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563eb0377b40 0x563eb0377f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:40.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.924+0000 7f3773de7740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563eb0378460 0x563eb03825f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:40.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.924+0000 7f3769400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563eb0377b40 0x563eb0377f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:40.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.924+0000 7f3773de7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563eb0382b30 0x563eb0384f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:40.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.924+0000 7f3769400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563eb0377b40 0x563eb0377f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54612/0 (socket says 172.21.15.83:54612) 2024-04-24T09:08:40.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.924+0000 7f3769400640 1 -- 172.21.15.83:0/3957717479 learned_addr learned my addr 172.21.15.83:0/3957717479 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:08:40.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.924+0000 7f3773de7740 1 -- 172.21.15.83:0/3957717479 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x563eb02dd1d0 con 0x563eb0377b40 2024-04-24T09:08:40.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.924+0000 7f3773de7740 1 -- 172.21.15.83:0/3957717479 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563eb0283450 con 0x563eb0382b30 2024-04-24T09:08:40.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.924+0000 7f3773de7740 1 -- 172.21.15.83:0/3957717479 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x563eb02d9190 con 0x563eb0378460 2024-04-24T09:08:40.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.924+0000 7f3769e00640 1 --2- 172.21.15.83:0/3957717479 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563eb0378460 0x563eb03825f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:40.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.924+0000 7f3762a00640 1 --2- 172.21.15.83:0/3302889864 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f37500a3fa0 0x7f37500a6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:40.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.925+0000 7f3762a00640 1 -- 172.21.15.83:0/3302889864 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 9583+0+0 (secure 0 0 0) 0x7f37640d0b00 con 0x7f376c1a3720 2024-04-24T09:08:40.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.925+0000 7f3769400640 1 --2- 172.21.15.83:0/3957717479 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563eb0382b30 0x563eb0384f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:40.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.925+0000 7f376a800640 1 --2- 172.21.15.83:0/3302889864 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f37500a3fa0 0x7f37500a6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:40.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.925+0000 7f3769e00640 1 -- 172.21.15.83:0/3957717479 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563eb0382b30 msgr2=0x563eb0384f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:40.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.925+0000 7f3769e00640 1 --2- 172.21.15.83:0/3957717479 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563eb0382b30 0x563eb0384f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:40.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.925+0000 7f3769e00640 1 -- 172.21.15.83:0/3957717479 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563eb0377b40 msgr2=0x563eb0377f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:40.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.925+0000 7f3769e00640 1 --2- 172.21.15.83:0/3957717479 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563eb0377b40 0x563eb0377f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:40.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.925+0000 7f3769e00640 1 -- 172.21.15.83:0/3957717479 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563eb0340310 con 0x563eb0378460 2024-04-24T09:08:40.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.925+0000 7f3769400640 1 --2- 172.21.15.83:0/3957717479 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563eb0382b30 0x563eb0384f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:08:40.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.926+0000 7f376a800640 1 --2- 172.21.15.83:0/3302889864 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f37500a3fa0 0x7f37500a6460 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f3764099050 tx=0x7f376400e4c0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:40.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.926+0000 7f3769e00640 1 --2- 172.21.15.83:0/3957717479 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563eb0378460 0x563eb03825f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f375c008000 tx=0x7f375c004200 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dae6db3e68efcc98 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:40.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.927+0000 7f375b400640 1 -- 172.21.15.83:0/3957717479 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f375c040070 con 0x563eb0378460 2024-04-24T09:08:40.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.927+0000 7f375b400640 1 -- 172.21.15.83:0/3957717479 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f375c0054f0 con 0x563eb0378460 2024-04-24T09:08:40.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.927+0000 7f375b400640 1 -- 172.21.15.83:0/3957717479 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f375c035040 con 0x563eb0378460 2024-04-24T09:08:40.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.928+0000 7f3773de7740 1 -- 172.21.15.83:0/3957717479 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563eb0378460 msgr2=0x563eb03825f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:40.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.928+0000 7f3773de7740 1 --2- 172.21.15.83:0/3957717479 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563eb0378460 0x563eb03825f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f375c008000 tx=0x7f375c004200 comp rx=0 tx=0).stop 2024-04-24T09:08:40.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.928+0000 7f3773de7740 1 -- 172.21.15.83:0/3957717479 shutdown_connections 2024-04-24T09:08:40.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.928+0000 7f3773de7740 1 --2- 172.21.15.83:0/3957717479 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563eb0382b30 0x563eb0384f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:40.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.928+0000 7f3773de7740 1 --2- 172.21.15.83:0/3957717479 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563eb0378460 0x563eb03825f0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:40.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.928+0000 7f3773de7740 1 --2- 172.21.15.83:0/3957717479 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563eb0377b40 0x563eb0377f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:40.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.928+0000 7f3773de7740 1 -- 172.21.15.83:0/3957717479 >> 172.21.15.83:0/3957717479 conn(0x563eb0373e60 msgr2=0x563eb0374240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:40.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.929+0000 7f3773de7740 1 -- 172.21.15.83:0/3957717479 shutdown_connections 2024-04-24T09:08:40.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.929+0000 7f3773de7740 1 -- 172.21.15.83:0/3957717479 wait complete. 2024-04-24T09:08:40.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.929+0000 7f3773de7740 1 -- start start 2024-04-24T09:08:40.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.930+0000 7f3773de7740 1 Processor -- start 2024-04-24T09:08:40.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.930+0000 7f3773de7740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563eb0377b40 0x563eb03825e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:40.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.930+0000 7f3773de7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563eb0378460 0x563eb037f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:40.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.930+0000 7f3773de7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563eb0382b30 0x563eb0380010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:40.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.930+0000 7f3773de7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x563eb0385960 con 0x563eb0382b30 2024-04-24T09:08:40.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.930+0000 7f3773de7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563eb0296810 con 0x563eb0378460 2024-04-24T09:08:40.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.930+0000 7f3773de7740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x563eb0385ae0 con 0x563eb0377b40 2024-04-24T09:08:40.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.930+0000 7f3773de7740 10 client.0 _ll_register_callbacks cb 0x563eb0264140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:08:40.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.931+0000 7f3769400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563eb0382b30 0x563eb0380010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:40.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.931+0000 7f3769400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563eb0382b30 0x563eb0380010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54628/0 (socket says 172.21.15.83:54628) 2024-04-24T09:08:40.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.931+0000 7f3769e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563eb0378460 0x563eb037f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:40.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.932+0000 7f3769e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563eb0378460 0x563eb037f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:53470/0 (socket says 172.21.15.83:53470) 2024-04-24T09:08:40.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.932+0000 7f3769e00640 1 -- 172.21.15.83:0/1317730121 learned_addr learned my addr 172.21.15.83:0/1317730121 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:08:40.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.932+0000 7f3769400640 1 --2- 172.21.15.83:0/1317730121 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563eb0377b40 0x563eb03825e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:40.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.932+0000 7f3769400640 1 -- 172.21.15.83:0/1317730121 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563eb0377b40 msgr2=0x563eb03825e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:40.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.932+0000 7f3769400640 1 --2- 172.21.15.83:0/1317730121 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563eb0377b40 0x563eb03825e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:40.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.932+0000 7f3769400640 1 -- 172.21.15.83:0/1317730121 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563eb0378460 msgr2=0x563eb037f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:40.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.932+0000 7f3769400640 1 --2- 172.21.15.83:0/1317730121 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563eb0378460 0x563eb037f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:40.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.932+0000 7f3769400640 1 -- 172.21.15.83:0/1317730121 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563eb02a73f0 con 0x563eb0382b30 2024-04-24T09:08:40.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.932+0000 7f3769400640 1 --2- 172.21.15.83:0/1317730121 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563eb0382b30 0x563eb0380010 secure :-1 s=READY pgs=2319 cs=0 l=1 rev1=1 crypto rx=0x7f37540058c0 tx=0x7f3754015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f29d1752845bc058 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:40.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.932+0000 7f374fe00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:08:40.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.933+0000 7f374fe00640 1 -- 172.21.15.83:0/1317730121 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f375401c040 con 0x563eb0382b30 2024-04-24T09:08:40.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.933+0000 7f374fe00640 1 -- 172.21.15.83:0/1317730121 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9583+0+0 (secure 0 0 0) 0x7f3754004030 con 0x563eb0382b30 2024-04-24T09:08:40.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.933+0000 7f3773de7740 1 -- 172.21.15.83:0/1317730121 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f375c0054f0 con 0x563eb0382b30 2024-04-24T09:08:40.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.933+0000 7f374fe00640 1 -- 172.21.15.83:0/1317730121 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f3754004230 con 0x563eb0382b30 2024-04-24T09:08:40.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.933+0000 7f3773de7740 10 client.21528 fetch_fsmap learned FSMap version 58 2024-04-24T09:08:40.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.933+0000 7f3773de7740 1 -- 172.21.15.83:0/1317730121 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x563eaffef3f0 con 0x563eb0382b30 2024-04-24T09:08:40.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.934+0000 7f374fe00640 1 -- 172.21.15.83:0/1317730121 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3754029070 con 0x563eb0382b30 2024-04-24T09:08:40.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.934+0000 7f374fe00640 1 -- 172.21.15.83:0/1317730121 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 58) ==== 8374+0+0 (secure 0 0 0) 0x7f3754021440 con 0x563eb0382b30 2024-04-24T09:08:40.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.934+0000 7f3773de7740 10 client.21528 fetch_fsmap finished waiting for FSMap version 58 2024-04-24T09:08:40.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.934+0000 7f3773de7740 10 client.21528 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-24T09:08:40.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.934+0000 7f3773de7740 20 client.21528 populate_metadata read hostname 'smithi083' 2024-04-24T09:08:40.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.934+0000 7f3773de7740 1 --2- 172.21.15.83:0/1317730121 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x563eb0392e00 0x563eb0395240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:40.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.934+0000 7f3773de7740 4 client.21528 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:08:40.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.934+0000 7f3773de7740 1 -- 172.21.15.83:0/1317730121 --> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563eb02eabf0 con 0x563eb0392e00 2024-04-24T09:08:40.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.935+0000 7f3769e00640 1 --2- 172.21.15.83:0/1317730121 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x563eb0392e00 0x563eb0395240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:40.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.936+0000 7f3769e00640 1 --2- 172.21.15.83:0/1317730121 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x563eb0392e00 0x563eb0395240 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=109600ac0db48862 server_cookie=4acecc76f26952bb in_seq=0 out_seq=0 2024-04-24T09:08:40.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.936+0000 7f374fe00640 10 client.21528 ms_handle_connect on v2:172.21.15.189:6838/2548965336 2024-04-24T09:08:40.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.938+0000 7f374fe00640 1 -- 172.21.15.83:0/1317730121 <== mds.4 v2:172.21.15.189:6838/2548965336 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563eb02eabf0 con 0x563eb0392e00 2024-04-24T09:08:40.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.938+0000 7f374fe00640 10 client.21528 handle_command_reply: tid=0 2024-04-24T09:08:40.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.939+0000 7f3773de7740 1 client.21528 shutdown 2024-04-24T09:08:40.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.939+0000 7f3773de7740 1 -- 172.21.15.83:0/1317730121 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563eb0382b30 msgr2=0x563eb0380010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:40.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.939+0000 7f3773de7740 1 --2- 172.21.15.83:0/1317730121 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563eb0382b30 0x563eb0380010 secure :-1 s=READY pgs=2319 cs=0 l=1 rev1=1 crypto rx=0x7f37540058c0 tx=0x7f3754015040 comp rx=0 tx=0).stop 2024-04-24T09:08:40.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.939+0000 7f3773de7740 1 -- 172.21.15.83:0/1317730121 shutdown_connections 2024-04-24T09:08:40.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.939+0000 7f3773de7740 1 --2- 172.21.15.83:0/1317730121 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x563eb0392e00 0x563eb0395240 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:40.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.939+0000 7f374fe00640 0 client.21528 ms_handle_reset on v2:172.21.15.189:6838/2548965336 2024-04-24T09:08:40.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.939+0000 7f3773de7740 1 --2- 172.21.15.83:0/1317730121 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563eb0382b30 0x563eb0380010 unknown :-1 s=CLOSED pgs=2319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:40.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.939+0000 7f3773de7740 1 --2- 172.21.15.83:0/1317730121 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563eb0378460 0x563eb037f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:40.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.939+0000 7f3773de7740 1 --2- 172.21.15.83:0/1317730121 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563eb0377b40 0x563eb03825e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:40.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.939+0000 7f3773de7740 1 -- 172.21.15.83:0/1317730121 >> 172.21.15.83:0/1317730121 conn(0x563eb0373e60 msgr2=0x563eb0379c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:40.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.940+0000 7f3773de7740 1 -- 172.21.15.83:0/1317730121 shutdown_connections 2024-04-24T09:08:40.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.940+0000 7f3773de7740 1 -- 172.21.15.83:0/1317730121 wait complete. 2024-04-24T09:08:40.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.940+0000 7f3773de7740 20 client.21528 trim_cache size 0 max 16384 2024-04-24T09:08:40.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.964+0000 7f3773de7740 1 Processor -- start 2024-04-24T09:08:40.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.964+0000 7f3773de7740 1 -- start start 2024-04-24T09:08:40.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.965+0000 7f3773de7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563eb0377b40 0x563eb038e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:40.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.965+0000 7f3773de7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563eb0378460 0x563eb037e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:40.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.965+0000 7f3773de7740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563eb0382b30 0x563eb0382110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:40.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.965+0000 7f3773de7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f376c1b5f00 con 0x563eb0377b40 2024-04-24T09:08:40.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.965+0000 7f3773de7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f376c1b7a40 con 0x563eb0378460 2024-04-24T09:08:40.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.965+0000 7f3773de7740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f376c1b65d0 con 0x563eb0382b30 2024-04-24T09:08:40.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.966+0000 7f3769e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563eb0378460 0x563eb037e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:40.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.966+0000 7f3769e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563eb0378460 0x563eb037e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:53472/0 (socket says 172.21.15.83:53472) 2024-04-24T09:08:40.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.966+0000 7f3769e00640 1 -- 172.21.15.83:0/50186800 learned_addr learned my addr 172.21.15.83:0/50186800 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:08:40.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.966+0000 7f3769400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563eb0377b40 0x563eb038e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:40.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.966+0000 7f3769e00640 1 -- 172.21.15.83:0/50186800 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563eb0382b30 msgr2=0x563eb0382110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:40.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.966+0000 7f3769e00640 1 --2- 172.21.15.83:0/50186800 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563eb0382b30 0x563eb0382110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:40.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.966+0000 7f3769e00640 1 -- 172.21.15.83:0/50186800 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563eb0377b40 msgr2=0x563eb038e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:40.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.966+0000 7f3769e00640 1 --2- 172.21.15.83:0/50186800 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563eb0377b40 0x563eb038e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:40.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.967+0000 7f3769e00640 1 -- 172.21.15.83:0/50186800 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563eb00b97d0 con 0x563eb0378460 2024-04-24T09:08:40.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.967+0000 7f3769e00640 1 --2- 172.21.15.83:0/50186800 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563eb0378460 0x563eb037e8b0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f375c009900 tx=0x7f375c048040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=136bb1c2b91880f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:40.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.967+0000 7f375aa00640 1 -- 172.21.15.83:0/50186800 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f375c040040 con 0x563eb0378460 2024-04-24T09:08:40.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.968+0000 7f375aa00640 1 -- 172.21.15.83:0/50186800 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x563eb02eabf0 con 0x563eb0378460 2024-04-24T09:08:40.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.968+0000 7f375aa00640 1 -- 172.21.15.83:0/50186800 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f375c035040 con 0x563eb0378460 2024-04-24T09:08:40.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.968+0000 7f3773de7740 1 -- 172.21.15.83:0/50186800 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563eb0378460 msgr2=0x563eb037e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:40.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.968+0000 7f3773de7740 1 --2- 172.21.15.83:0/50186800 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563eb0378460 0x563eb037e8b0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f375c009900 tx=0x7f375c048040 comp rx=0 tx=0).stop 2024-04-24T09:08:40.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.969+0000 7f3773de7740 1 -- 172.21.15.83:0/50186800 shutdown_connections 2024-04-24T09:08:40.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.969+0000 7f3773de7740 1 --2- 172.21.15.83:0/50186800 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563eb0382b30 0x563eb0382110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:40.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.969+0000 7f3773de7740 1 --2- 172.21.15.83:0/50186800 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563eb0378460 0x563eb037e8b0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:40.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.969+0000 7f3773de7740 1 --2- 172.21.15.83:0/50186800 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563eb0377b40 0x563eb038e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:40.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.969+0000 7f3773de7740 1 -- 172.21.15.83:0/50186800 >> 172.21.15.83:0/50186800 conn(0x563eb0373e60 msgr2=0x563eb037e1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:40.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.969+0000 7f3773de7740 1 -- 172.21.15.83:0/50186800 shutdown_connections 2024-04-24T09:08:40.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.969+0000 7f3773de7740 1 -- 172.21.15.83:0/50186800 wait complete. 2024-04-24T09:08:40.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.969+0000 7f3773de7740 1 -- start start 2024-04-24T09:08:40.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.969+0000 7f3773de7740 1 Processor -- start 2024-04-24T09:08:40.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.970+0000 7f3773de7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563eb0377b40 0x563eb037d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:40.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.970+0000 7f3773de7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563eb0378460 0x563eb03f63b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:40.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.970+0000 7f3769400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563eb0377b40 0x563eb037d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:40.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.970+0000 7f3769400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563eb0377b40 0x563eb037d130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:53482/0 (socket says 172.21.15.83:53482) 2024-04-24T09:08:40.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.970+0000 7f3773de7740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563eb0382b30 0x563eb03f68f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:40.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.970+0000 7f3773de7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f376c1b7370 con 0x563eb0378460 2024-04-24T09:08:40.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.970+0000 7f3769400640 1 -- 172.21.15.83:0/3067320426 learned_addr learned my addr 172.21.15.83:0/3067320426 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:08:40.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.970+0000 7f3773de7740 1 -- 172.21.15.83:0/3067320426 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f376c1b6ca0 con 0x563eb0377b40 2024-04-24T09:08:40.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.970+0000 7f3773de7740 1 -- 172.21.15.83:0/3067320426 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f376c1b3f00 con 0x563eb0382b30 2024-04-24T09:08:40.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.970+0000 7f3773de7740 10 client.0 _ll_register_callbacks cb 0x7f3754009df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:08:40.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.971+0000 7f3769e00640 1 --2- 172.21.15.83:0/3067320426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563eb0378460 0x563eb03f63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:40.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.971+0000 7f3769400640 1 -- 172.21.15.83:0/3067320426 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563eb0382b30 msgr2=0x563eb03f68f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:40.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.971+0000 7f3769400640 1 --2- 172.21.15.83:0/3067320426 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563eb0382b30 0x563eb03f68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:40.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.971+0000 7f3769400640 1 -- 172.21.15.83:0/3067320426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563eb0378460 msgr2=0x563eb03f63b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:40.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.971+0000 7f3769400640 1 --2- 172.21.15.83:0/3067320426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563eb0378460 0x563eb03f63b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:40.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.971+0000 7f3769400640 1 -- 172.21.15.83:0/3067320426 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563eb03f6e30 con 0x563eb0377b40 2024-04-24T09:08:40.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.971+0000 7f3769e00640 1 --2- 172.21.15.83:0/3067320426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563eb0378460 0x563eb03f63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:08:40.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.971+0000 7f3769400640 1 --2- 172.21.15.83:0/3067320426 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563eb0377b40 0x563eb037d130 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3754009a90 tx=0x7f3754009270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73bcbf840a91a71e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:40.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.971+0000 7f374fe00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:08:40.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.972+0000 7f374fe00640 1 -- 172.21.15.83:0/3067320426 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f375400fe20 con 0x563eb0377b40 2024-04-24T09:08:40.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.972+0000 7f3773de7740 1 -- 172.21.15.83:0/3067320426 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563eb02eabf0 con 0x563eb0377b40 2024-04-24T09:08:40.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.972+0000 7f374fe00640 1 -- 172.21.15.83:0/3067320426 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9583+0+0 (secure 0 0 0) 0x7f37540213b0 con 0x563eb0377b40 2024-04-24T09:08:40.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.973+0000 7f374fe00640 1 -- 172.21.15.83:0/3067320426 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f37540215b0 con 0x563eb0377b40 2024-04-24T09:08:40.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.973+0000 7f3773de7740 10 client.38672 fetch_fsmap learned FSMap version 58 2024-04-24T09:08:40.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.973+0000 7f3773de7740 1 -- 172.21.15.83:0/3067320426 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x563eb037c850 con 0x563eb0377b40 2024-04-24T09:08:40.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.973+0000 7f374fe00640 1 -- 172.21.15.83:0/3067320426 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x563eb02eabf0 con 0x563eb0377b40 2024-04-24T09:08:40.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.973+0000 7f374fe00640 1 -- 172.21.15.83:0/3067320426 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 58) ==== 8374+0+0 (secure 0 0 0) 0x7f375402d430 con 0x563eb0377b40 2024-04-24T09:08:40.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.973+0000 7f3773de7740 10 client.38672 fetch_fsmap finished waiting for FSMap version 58 2024-04-24T09:08:40.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.973+0000 7f3773de7740 10 client.38672 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-24T09:08:40.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.973+0000 7f3773de7740 20 client.38672 populate_metadata read hostname 'smithi083' 2024-04-24T09:08:40.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.973+0000 7f3773de7740 1 --2- 172.21.15.83:0/3067320426 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x563eb038a090 0x563eb038a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:40.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.973+0000 7f3773de7740 4 client.38672 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:08:40.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.973+0000 7f3773de7740 1 -- 172.21.15.83:0/3067320426 --> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563eb02e4a60 con 0x563eb038a090 2024-04-24T09:08:40.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.974+0000 7f3769e00640 1 --2- 172.21.15.83:0/3067320426 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x563eb038a090 0x563eb038a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:40.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.976+0000 7f3769e00640 1 --2- 172.21.15.83:0/3067320426 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x563eb038a090 0x563eb038a4a0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7258f4d99303d0e8 server_cookie=5db2109633f735b6 in_seq=0 out_seq=0 2024-04-24T09:08:40.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.976+0000 7f374fe00640 10 client.38672 ms_handle_connect on v2:172.21.15.189:6838/2548965336 2024-04-24T09:08:40.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.978+0000 7f374fe00640 1 -- 172.21.15.83:0/3067320426 <== mds.4 v2:172.21.15.189:6838/2548965336 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563eb02e4a60 con 0x563eb038a090 2024-04-24T09:08:40.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.978+0000 7f374fe00640 10 client.38672 handle_command_reply: tid=0 2024-04-24T09:08:40.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.978+0000 7f3773de7740 1 client.38672 shutdown 2024-04-24T09:08:40.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.979+0000 7f3773de7740 1 -- 172.21.15.83:0/3067320426 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563eb0377b40 msgr2=0x563eb037d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:40.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.979+0000 7f3773de7740 1 --2- 172.21.15.83:0/3067320426 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563eb0377b40 0x563eb037d130 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3754009a90 tx=0x7f3754009270 comp rx=0 tx=0).stop 2024-04-24T09:08:40.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.979+0000 7f3773de7740 1 -- 172.21.15.83:0/3067320426 shutdown_connections 2024-04-24T09:08:40.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.979+0000 7f3773de7740 1 --2- 172.21.15.83:0/3067320426 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x563eb038a090 0x563eb038a4a0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:40.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.979+0000 7f3773de7740 1 --2- 172.21.15.83:0/3067320426 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563eb0382b30 0x563eb03f68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:40.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.979+0000 7f3773de7740 1 --2- 172.21.15.83:0/3067320426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563eb0378460 0x563eb03f63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:40.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.979+0000 7f374fe00640 0 client.38672 ms_handle_reset on v2:172.21.15.189:6838/2548965336 2024-04-24T09:08:40.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.979+0000 7f3773de7740 1 --2- 172.21.15.83:0/3067320426 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563eb0377b40 0x563eb037d130 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:40.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.979+0000 7f3773de7740 1 -- 172.21.15.83:0/3067320426 >> 172.21.15.83:0/3067320426 conn(0x563eb0373e60 msgr2=0x563eb0372f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:40.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.979+0000 7f3773de7740 1 -- 172.21.15.83:0/3067320426 shutdown_connections 2024-04-24T09:08:40.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.979+0000 7f3773de7740 1 -- 172.21.15.83:0/3067320426 wait complete. 2024-04-24T09:08:40.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.979+0000 7f3773de7740 20 client.38672 trim_cache size 0 max 16384 2024-04-24T09:08:40.980 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:08:40.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.980+0000 7f376b200640 1 -- 172.21.15.83:0/3302889864 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f37500a3fa0 msgr2=0x7f37500a6460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:40.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.980+0000 7f376b200640 1 --2- 172.21.15.83:0/3302889864 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f37500a3fa0 0x7f37500a6460 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f3764099050 tx=0x7f376400e4c0 comp rx=0 tx=0).stop 2024-04-24T09:08:40.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.980+0000 7f376b200640 1 -- 172.21.15.83:0/3302889864 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f376c1a3720 msgr2=0x7f376c1a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:40.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.980+0000 7f376b200640 1 --2- 172.21.15.83:0/3302889864 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f376c1a3720 0x7f376c1a3bd0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f376400ccb0 tx=0x7f376400a620 comp rx=0 tx=0).stop 2024-04-24T09:08:40.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.980+0000 7f376b200640 1 -- 172.21.15.83:0/3302889864 shutdown_connections 2024-04-24T09:08:40.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.980+0000 7f376b200640 1 --2- 172.21.15.83:0/3302889864 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f37500a3fa0 0x7f37500a6460 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:40.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.980+0000 7f376b200640 1 --2- 172.21.15.83:0/3302889864 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f376c1a3720 0x7f376c1a3bd0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:40.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.980+0000 7f376b200640 1 --2- 172.21.15.83:0/3302889864 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f376c112d50 0x7f376c19f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:40.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.980+0000 7f376b200640 1 --2- 172.21.15.83:0/3302889864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f376c076e50 0x7f376c19eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:40.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.980+0000 7f376b200640 1 -- 172.21.15.83:0/3302889864 >> 172.21.15.83:0/3302889864 conn(0x7f376c0ffb40 msgr2=0x7f376c101f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:40.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.980+0000 7f376b200640 1 -- 172.21.15.83:0/3302889864 shutdown_connections 2024-04-24T09:08:40.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:40.980+0000 7f376b200640 1 -- 172.21.15.83:0/3302889864 wait complete. 2024-04-24T09:08:41.002 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-24T09:08:41.167 INFO:tasks.workunit.client.0.smithi083.stdout: 10 24749 6.87 MB/sec execute 72 sec latency 91.480 ms 2024-04-24T09:08:41.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.271+0000 7fbb1a200640 1 -- 172.21.15.83:0/3814445754 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb14102240 msgr2=0x7fbb14102620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:41.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.271+0000 7fbb1a200640 1 --2- 172.21.15.83:0/3814445754 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb14102240 0x7fbb14102620 secure :-1 s=READY pgs=2320 cs=0 l=1 rev1=1 crypto rx=0x7fbb04009900 tx=0x7fbb0402a3f0 comp rx=0 tx=0).stop 2024-04-24T09:08:41.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.272+0000 7fbb1a200640 1 -- 172.21.15.83:0/3814445754 shutdown_connections 2024-04-24T09:08:41.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.272+0000 7fbb1a200640 1 --2- 172.21.15.83:0/3814445754 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb14107310 0x7fbb1410eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:41.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.272+0000 7fbb1a200640 1 --2- 172.21.15.83:0/3814445754 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbb14102bf0 0x7fbb14106c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:41.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.272+0000 7fbb1a200640 1 --2- 172.21.15.83:0/3814445754 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb14102240 0x7fbb14102620 unknown :-1 s=CLOSED pgs=2320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:41.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.272+0000 7fbb1a200640 1 -- 172.21.15.83:0/3814445754 >> 172.21.15.83:0/3814445754 conn(0x7fbb14079450 msgr2=0x7fbb14079860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:41.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.272+0000 7fbb1a200640 1 -- 172.21.15.83:0/3814445754 shutdown_connections 2024-04-24T09:08:41.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.272+0000 7fbb1a200640 1 -- 172.21.15.83:0/3814445754 wait complete. 2024-04-24T09:08:41.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.273+0000 7fbb1a200640 1 Processor -- start 2024-04-24T09:08:41.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.273+0000 7fbb1a200640 1 -- start start 2024-04-24T09:08:41.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.274+0000 7fbb1a200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb14102240 0x7fbb140738a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:41.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.274+0000 7fbb1a200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb14107310 0x7fbb1406c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:41.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.274+0000 7fbb1a200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbb1406ce60 0x7fbb1406d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:41.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.274+0000 7fbb1a200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbb14110120 con 0x7fbb14107310 2024-04-24T09:08:41.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.274+0000 7fbb1a200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbb1410ffa0 con 0x7fbb14102240 2024-04-24T09:08:41.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.274+0000 7fbb1a200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fbb141102a0 con 0x7fbb1406ce60 2024-04-24T09:08:41.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.274+0000 7fbb13e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbb1406ce60 0x7fbb1406d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:41.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.274+0000 7fbb13e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbb1406ce60 0x7fbb1406d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:44450/0 (socket says 172.21.15.83:44450) 2024-04-24T09:08:41.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.274+0000 7fbb13e00640 1 -- 172.21.15.83:0/4035943384 learned_addr learned my addr 172.21.15.83:0/4035943384 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:08:41.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.274+0000 7fbb13400640 1 --2- 172.21.15.83:0/4035943384 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb14102240 0x7fbb140738a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:41.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.275+0000 7fbb12a00640 1 --2- 172.21.15.83:0/4035943384 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb14107310 0x7fbb1406c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:41.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.275+0000 7fbb13400640 1 -- 172.21.15.83:0/4035943384 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbb1406ce60 msgr2=0x7fbb1406d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:41.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.275+0000 7fbb13400640 1 --2- 172.21.15.83:0/4035943384 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbb1406ce60 0x7fbb1406d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:41.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.275+0000 7fbb13400640 1 -- 172.21.15.83:0/4035943384 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb14107310 msgr2=0x7fbb1406c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:41.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.275+0000 7fbb13400640 1 --2- 172.21.15.83:0/4035943384 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb14107310 0x7fbb1406c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:41.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.275+0000 7fbb13400640 1 -- 172.21.15.83:0/4035943384 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb140713c0 con 0x7fbb14102240 2024-04-24T09:08:41.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.275+0000 7fbb12a00640 1 --2- 172.21.15.83:0/4035943384 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb14107310 0x7fbb1406c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:08:41.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.276+0000 7fbb13400640 1 --2- 172.21.15.83:0/4035943384 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb14102240 0x7fbb140738a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fbb0402a9e0 tx=0x7fbb04003cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:41.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.276+0000 7fbb0be00640 1 -- 172.21.15.83:0/4035943384 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb0403f070 con 0x7fbb14102240 2024-04-24T09:08:41.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.276+0000 7fbb1a200640 1 -- 172.21.15.83:0/4035943384 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb14071650 con 0x7fbb14102240 2024-04-24T09:08:41.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.276+0000 7fbb1a200640 1 -- 172.21.15.83:0/4035943384 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb14071b60 con 0x7fbb14102240 2024-04-24T09:08:41.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.276+0000 7fbb0be00640 1 -- 172.21.15.83:0/4035943384 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbb04031d60 con 0x7fbb14102240 2024-04-24T09:08:41.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.276+0000 7fbb0be00640 1 -- 172.21.15.83:0/4035943384 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb04035030 con 0x7fbb14102240 2024-04-24T09:08:41.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.280+0000 7fbb1cf6c740 1 Processor -- start 2024-04-24T09:08:41.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.281+0000 7fbb1cf6c740 1 -- start start 2024-04-24T09:08:41.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.282+0000 7fbb1cf6c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613d8a23950 0x5613d8a23d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:41.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.282+0000 7fbb1cf6c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5613d8a24270 0x5613d8a2e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:41.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.282+0000 7fbb1cf6c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5613d8a2e940 0x5613d8a30d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:41.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.282+0000 7fbb1cf6c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5613d89891d0 con 0x5613d8a2e940 2024-04-24T09:08:41.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.282+0000 7fbb0be00640 1 -- 172.21.15.83:0/4035943384 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fbb04042070 con 0x7fbb14102240 2024-04-24T09:08:41.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.282+0000 7fbb13e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613d8a23950 0x5613d8a23d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:41.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.282+0000 7fbb13e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613d8a23950 0x5613d8a23d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:44462/0 (socket says 172.21.15.83:44462) 2024-04-24T09:08:41.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.282+0000 7fbb13e00640 1 -- 172.21.15.83:0/2434210645 learned_addr learned my addr 172.21.15.83:0/2434210645 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:08:41.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.282+0000 7fbb1cf6c740 1 -- 172.21.15.83:0/2434210645 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5613d892f450 con 0x5613d8a24270 2024-04-24T09:08:41.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.282+0000 7fbb1cf6c740 1 -- 172.21.15.83:0/2434210645 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5613d8985190 con 0x5613d8a23950 2024-04-24T09:08:41.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.283+0000 7fbb12a00640 1 --2- 172.21.15.83:0/2434210645 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5613d8a2e940 0x5613d8a30d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:41.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.283+0000 7fbb13400640 1 --2- 172.21.15.83:0/2434210645 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5613d8a24270 0x5613d8a2e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:41.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.283+0000 7fbb13e00640 1 -- 172.21.15.83:0/2434210645 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5613d8a24270 msgr2=0x5613d8a2e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:41.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.283+0000 7fbb13e00640 1 --2- 172.21.15.83:0/2434210645 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5613d8a24270 0x5613d8a2e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:41.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.283+0000 7fbb13e00640 1 -- 172.21.15.83:0/2434210645 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5613d8a2e940 msgr2=0x5613d8a30d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:41.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.283+0000 7fbb13e00640 1 --2- 172.21.15.83:0/2434210645 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5613d8a2e940 0x5613d8a30d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:41.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.283+0000 7fbb13e00640 1 -- 172.21.15.83:0/2434210645 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613d89ec310 con 0x5613d8a23950 2024-04-24T09:08:41.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.283+0000 7fbb12a00640 1 --2- 172.21.15.83:0/2434210645 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5613d8a2e940 0x5613d8a30d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:08:41.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.284+0000 7fbb13e00640 1 --2- 172.21.15.83:0/2434210645 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613d8a23950 0x5613d8a23d30 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c0104d0 tx=0x7fbb0c010990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5a341396b9504cf server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:41.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.284+0000 7fbaffe00640 1 -- 172.21.15.83:0/2434210645 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb0c011760 con 0x5613d8a23950 2024-04-24T09:08:41.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.284+0000 7fbaffe00640 1 -- 172.21.15.83:0/2434210645 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbb0c011db0 con 0x5613d8a23950 2024-04-24T09:08:41.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.284+0000 7fbaffe00640 1 -- 172.21.15.83:0/2434210645 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb0c0193d0 con 0x5613d8a23950 2024-04-24T09:08:41.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.284+0000 7fbb0be00640 1 --2- 172.21.15.83:0/4035943384 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fbaf80a3d80 0x7fbaf80a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:41.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.285+0000 7fbb0be00640 1 -- 172.21.15.83:0/4035943384 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 9583+0+0 (secure 0 0 0) 0x7fbb040f1250 con 0x7fbb14102240 2024-04-24T09:08:41.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.285+0000 7fbb1cf6c740 1 -- 172.21.15.83:0/2434210645 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613d8a23950 msgr2=0x5613d8a23d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:41.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.285+0000 7fbb1cf6c740 1 --2- 172.21.15.83:0/2434210645 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613d8a23950 0x5613d8a23d30 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c0104d0 tx=0x7fbb0c010990 comp rx=0 tx=0).stop 2024-04-24T09:08:41.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.285+0000 7fbb13400640 1 --2- 172.21.15.83:0/4035943384 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fbaf80a3d80 0x7fbaf80a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:41.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.286+0000 7fbb1cf6c740 1 -- 172.21.15.83:0/2434210645 shutdown_connections 2024-04-24T09:08:41.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.286+0000 7fbb1cf6c740 1 --2- 172.21.15.83:0/2434210645 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5613d8a2e940 0x5613d8a30d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:41.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.286+0000 7fbb1cf6c740 1 --2- 172.21.15.83:0/2434210645 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5613d8a24270 0x5613d8a2e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:41.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.286+0000 7fbb1cf6c740 1 --2- 172.21.15.83:0/2434210645 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613d8a23950 0x5613d8a23d30 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:41.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.286+0000 7fbb13400640 1 --2- 172.21.15.83:0/4035943384 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fbaf80a3d80 0x7fbaf80a6240 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x5613d8a24f10 tx=0x7fbb04031400 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:41.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.286+0000 7fbb1cf6c740 1 -- 172.21.15.83:0/2434210645 >> 172.21.15.83:0/2434210645 conn(0x5613d8a1fc70 msgr2=0x5613d8a20050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:41.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.286+0000 7fbb1cf6c740 1 -- 172.21.15.83:0/2434210645 shutdown_connections 2024-04-24T09:08:41.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.286+0000 7fbb1cf6c740 1 -- 172.21.15.83:0/2434210645 wait complete. 2024-04-24T09:08:41.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.287+0000 7fbb1cf6c740 1 -- start start 2024-04-24T09:08:41.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.287+0000 7fbb1cf6c740 1 Processor -- start 2024-04-24T09:08:41.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.288+0000 7fbb1cf6c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5613d8a23950 0x5613d8a2e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:41.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.288+0000 7fbb1cf6c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5613d8a24270 0x5613d8a2b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:41.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.288+0000 7fbb1cf6c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613d8a2e940 0x5613d8a2bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:41.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.288+0000 7fbb1cf6c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5613d8a317d0 con 0x5613d8a23950 2024-04-24T09:08:41.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.288+0000 7fbb1cf6c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5613d8942810 con 0x5613d8a24270 2024-04-24T09:08:41.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.288+0000 7fbb1cf6c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5613d8a31950 con 0x5613d8a2e940 2024-04-24T09:08:41.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.288+0000 7fbb1cf6c740 10 client.0 _ll_register_callbacks cb 0x5613d8910140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:08:41.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.288+0000 7fbb13e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5613d8a23950 0x5613d8a2e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:41.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.288+0000 7fbb13e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5613d8a23950 0x5613d8a2e380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54686/0 (socket says 172.21.15.83:54686) 2024-04-24T09:08:41.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.288+0000 7fbb13e00640 1 -- 172.21.15.83:0/3090204664 learned_addr learned my addr 172.21.15.83:0/3090204664 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:08:41.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.288+0000 7fbb12a00640 1 --2- 172.21.15.83:0/3090204664 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5613d8a24270 0x5613d8a2b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:41.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.288+0000 7fbb13e00640 1 -- 172.21.15.83:0/3090204664 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613d8a2e940 msgr2=0x5613d8a2bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:41.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.288+0000 7fbb13e00640 1 --2- 172.21.15.83:0/3090204664 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613d8a2e940 0x5613d8a2bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:41.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.288+0000 7fbb13e00640 1 -- 172.21.15.83:0/3090204664 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5613d8a24270 msgr2=0x5613d8a2b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:41.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.288+0000 7fbb13e00640 1 --2- 172.21.15.83:0/3090204664 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5613d8a24270 0x5613d8a2b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:41.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.288+0000 7fbb13e00640 1 -- 172.21.15.83:0/3090204664 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613d89533f0 con 0x5613d8a23950 2024-04-24T09:08:41.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.289+0000 7fbb13e00640 1 --2- 172.21.15.83:0/3090204664 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5613d8a23950 0x5613d8a2e380 secure :-1 s=READY pgs=2321 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c009af0 tx=0x7fbb0c00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adca1a58a7eb722 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:41.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.289+0000 7fbafcc00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:08:41.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.289+0000 7fbafcc00640 1 -- 172.21.15.83:0/3090204664 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb0c027030 con 0x5613d8a23950 2024-04-24T09:08:41.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.289+0000 7fbafcc00640 1 -- 172.21.15.83:0/3090204664 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9583+0+0 (secure 0 0 0) 0x7fbb0c01f070 con 0x5613d8a23950 2024-04-24T09:08:41.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.290+0000 7fbb1cf6c740 1 -- 172.21.15.83:0/3090204664 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb0c011db0 con 0x5613d8a23950 2024-04-24T09:08:41.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.290+0000 7fbafcc00640 1 -- 172.21.15.83:0/3090204664 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7fbb0c004b00 con 0x5613d8a23950 2024-04-24T09:08:41.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.290+0000 7fbb1cf6c740 10 client.21543 fetch_fsmap learned FSMap version 58 2024-04-24T09:08:41.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.290+0000 7fbb1cf6c740 1 -- 172.21.15.83:0/3090204664 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x5613d869b3f0 con 0x5613d8a23950 2024-04-24T09:08:41.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.290+0000 7fbafcc00640 1 -- 172.21.15.83:0/3090204664 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbb0c011db0 con 0x5613d8a23950 2024-04-24T09:08:41.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.291+0000 7fbafcc00640 1 -- 172.21.15.83:0/3090204664 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 58) ==== 8374+0+0 (secure 0 0 0) 0x7fbb0c011890 con 0x5613d8a23950 2024-04-24T09:08:41.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.291+0000 7fbb1cf6c740 10 client.21543 fetch_fsmap finished waiting for FSMap version 58 2024-04-24T09:08:41.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.291+0000 7fbb1cf6c740 10 client.21543 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-24T09:08:41.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.291+0000 7fbb1cf6c740 20 client.21543 populate_metadata read hostname 'smithi083' 2024-04-24T09:08:41.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.291+0000 7fbb1cf6c740 1 --2- 172.21.15.83:0/3090204664 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x5613d8a3eb50 0x5613d8a40f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:41.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.291+0000 7fbb1cf6c740 4 client.21543 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:08:41.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.291+0000 7fbb1cf6c740 1 -- 172.21.15.83:0/3090204664 --> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5613d8996bf0 con 0x5613d8a3eb50 2024-04-24T09:08:41.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.291+0000 7fbb12a00640 1 --2- 172.21.15.83:0/3090204664 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x5613d8a3eb50 0x5613d8a40f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:41.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.292+0000 7fbb12a00640 1 --2- 172.21.15.83:0/3090204664 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x5613d8a3eb50 0x5613d8a40f40 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f4f90e20c23cc5f5 server_cookie=ba4cb7aa1c2c2313 in_seq=0 out_seq=0 2024-04-24T09:08:41.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.292+0000 7fbafcc00640 10 client.21543 ms_handle_connect on v2:172.21.15.138:6832/1157967658 2024-04-24T09:08:41.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.293+0000 7fbafcc00640 1 -- 172.21.15.83:0/3090204664 <== mds.3 v2:172.21.15.138:6832/1157967658 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5613d8996bf0 con 0x5613d8a3eb50 2024-04-24T09:08:41.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.293+0000 7fbafcc00640 10 client.21543 handle_command_reply: tid=0 2024-04-24T09:08:41.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.293+0000 7fbb1cf6c740 1 client.21543 shutdown 2024-04-24T09:08:41.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.294+0000 7fbb1cf6c740 1 -- 172.21.15.83:0/3090204664 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5613d8a23950 msgr2=0x5613d8a2e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:41.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.294+0000 7fbb1cf6c740 1 --2- 172.21.15.83:0/3090204664 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5613d8a23950 0x5613d8a2e380 secure :-1 s=READY pgs=2321 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c009af0 tx=0x7fbb0c00ed70 comp rx=0 tx=0).stop 2024-04-24T09:08:41.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.294+0000 7fbb1cf6c740 1 -- 172.21.15.83:0/3090204664 shutdown_connections 2024-04-24T09:08:41.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.294+0000 7fbb1cf6c740 1 --2- 172.21.15.83:0/3090204664 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x5613d8a3eb50 0x5613d8a40f40 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:41.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.294+0000 7fbb1cf6c740 1 --2- 172.21.15.83:0/3090204664 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613d8a2e940 0x5613d8a2bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:41.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.294+0000 7fbb1cf6c740 1 --2- 172.21.15.83:0/3090204664 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5613d8a24270 0x5613d8a2b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:41.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.294+0000 7fbb1cf6c740 1 --2- 172.21.15.83:0/3090204664 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5613d8a23950 0x5613d8a2e380 unknown :-1 s=CLOSED pgs=2321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:41.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.294+0000 7fbb1cf6c740 1 -- 172.21.15.83:0/3090204664 >> 172.21.15.83:0/3090204664 conn(0x5613d8a1fc70 msgr2=0x5613d8a25970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:41.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.294+0000 7fbafcc00640 0 client.21543 ms_handle_reset on v2:172.21.15.138:6832/1157967658 2024-04-24T09:08:41.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.294+0000 7fbb1cf6c740 1 -- 172.21.15.83:0/3090204664 shutdown_connections 2024-04-24T09:08:41.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.294+0000 7fbb1cf6c740 1 -- 172.21.15.83:0/3090204664 wait complete. 2024-04-24T09:08:41.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.294+0000 7fbb1cf6c740 20 client.21543 trim_cache size 0 max 16384 2024-04-24T09:08:41.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:08:41 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/2886398826' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:08:41.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.316+0000 7fbb1cf6c740 1 Processor -- start 2024-04-24T09:08:41.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.316+0000 7fbb1cf6c740 1 -- start start 2024-04-24T09:08:41.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.317+0000 7fbb1cf6c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613d8a23950 0x5613d8a3a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:41.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.317+0000 7fbb1cf6c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5613d8a24270 0x5613d8a2deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:41.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.317+0000 7fbb1cf6c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5613d8a2e940 0x5613d8a2e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:41.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.317+0000 7fbb1cf6c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf4010100 con 0x5613d8a24270 2024-04-24T09:08:41.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.317+0000 7fbb1cf6c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf4011c40 con 0x5613d8a2e940 2024-04-24T09:08:41.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.317+0000 7fbb1cf6c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf40107d0 con 0x5613d8a23950 2024-04-24T09:08:41.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.317+0000 7fbb12a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5613d8a24270 0x5613d8a2deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:41.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.317+0000 7fbb12a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5613d8a24270 0x5613d8a2deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54700/0 (socket says 172.21.15.83:54700) 2024-04-24T09:08:41.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.317+0000 7fbb12a00640 1 -- 172.21.15.83:0/1124204318 learned_addr learned my addr 172.21.15.83:0/1124204318 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:08:41.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.317+0000 7fbb13e00640 1 --2- 172.21.15.83:0/1124204318 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5613d8a2e940 0x5613d8a2e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:41.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.317+0000 7fbb13e00640 1 --2- 172.21.15.83:0/1124204318 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613d8a23950 0x5613d8a3a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:41.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.318+0000 7fbb12a00640 1 -- 172.21.15.83:0/1124204318 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613d8a23950 msgr2=0x5613d8a3a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:41.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.318+0000 7fbb12a00640 1 --2- 172.21.15.83:0/1124204318 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613d8a23950 0x5613d8a3a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:41.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.318+0000 7fbb12a00640 1 -- 172.21.15.83:0/1124204318 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5613d8a2e940 msgr2=0x5613d8a2e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:41.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.318+0000 7fbb12a00640 1 --2- 172.21.15.83:0/1124204318 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5613d8a2e940 0x5613d8a2e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:41.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.318+0000 7fbb12a00640 1 -- 172.21.15.83:0/1124204318 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613d87657d0 con 0x5613d8a24270 2024-04-24T09:08:41.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.318+0000 7fbb13e00640 1 --2- 172.21.15.83:0/1124204318 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613d8a23950 0x5613d8a3a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:08:41.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.318+0000 7fbb12a00640 1 --2- 172.21.15.83:0/1124204318 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5613d8a24270 0x5613d8a2deb0 secure :-1 s=READY pgs=2322 cs=0 l=1 rev1=1 crypto rx=0x7fbb1406b8d0 tx=0x7fbb000047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b92a708abdde744e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:41.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.319+0000 7fbaff400640 1 -- 172.21.15.83:0/1124204318 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb00020020 con 0x5613d8a24270 2024-04-24T09:08:41.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.319+0000 7fbaff400640 1 -- 172.21.15.83:0/1124204318 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbb00014df0 con 0x5613d8a24270 2024-04-24T09:08:41.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.319+0000 7fbaff400640 1 -- 172.21.15.83:0/1124204318 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb00013600 con 0x5613d8a24270 2024-04-24T09:08:41.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.320+0000 7fbb1cf6c740 1 -- 172.21.15.83:0/1124204318 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5613d8a24270 msgr2=0x5613d8a2deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:41.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.320+0000 7fbb1cf6c740 1 --2- 172.21.15.83:0/1124204318 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5613d8a24270 0x5613d8a2deb0 secure :-1 s=READY pgs=2322 cs=0 l=1 rev1=1 crypto rx=0x7fbb1406b8d0 tx=0x7fbb000047d0 comp rx=0 tx=0).stop 2024-04-24T09:08:41.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.320+0000 7fbb1cf6c740 1 -- 172.21.15.83:0/1124204318 shutdown_connections 2024-04-24T09:08:41.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.320+0000 7fbb1cf6c740 1 --2- 172.21.15.83:0/1124204318 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5613d8a2e940 0x5613d8a2e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:41.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.320+0000 7fbb1cf6c740 1 --2- 172.21.15.83:0/1124204318 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5613d8a24270 0x5613d8a2deb0 unknown :-1 s=CLOSED pgs=2322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:41.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.320+0000 7fbb1cf6c740 1 --2- 172.21.15.83:0/1124204318 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613d8a23950 0x5613d8a3a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:41.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.320+0000 7fbb1cf6c740 1 -- 172.21.15.83:0/1124204318 >> 172.21.15.83:0/1124204318 conn(0x5613d8a1fc70 msgr2=0x5613d8a26940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:41.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.320+0000 7fbb1cf6c740 1 -- 172.21.15.83:0/1124204318 shutdown_connections 2024-04-24T09:08:41.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.320+0000 7fbb1cf6c740 1 -- 172.21.15.83:0/1124204318 wait complete. 2024-04-24T09:08:41.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.321+0000 7fbb1cf6c740 1 -- start start 2024-04-24T09:08:41.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.321+0000 7fbb1cf6c740 1 Processor -- start 2024-04-24T09:08:41.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.321+0000 7fbb1cf6c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5613d8a23950 0x5613d8a36c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:41.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.321+0000 7fbb1cf6c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613d8a24270 0x5613d8a37170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:41.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.321+0000 7fbb1cf6c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5613d8a2e940 0x5613d8a288f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:41.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.322+0000 7fbb1cf6c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf4011570 con 0x5613d8a23950 2024-04-24T09:08:41.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.322+0000 7fbb1cf6c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf4010ea0 con 0x5613d8a2e940 2024-04-24T09:08:41.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.322+0000 7fbb1cf6c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf400e100 con 0x5613d8a24270 2024-04-24T09:08:41.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.322+0000 7fbb1cf6c740 10 client.0 _ll_register_callbacks cb 0x7fbb0000a150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:08:41.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.322+0000 7fbb13e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5613d8a23950 0x5613d8a36c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:41.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.322+0000 7fbb13e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5613d8a23950 0x5613d8a36c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54708/0 (socket says 172.21.15.83:54708) 2024-04-24T09:08:41.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.322+0000 7fbb13e00640 1 -- 172.21.15.83:0/2490720835 learned_addr learned my addr 172.21.15.83:0/2490720835 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:08:41.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.322+0000 7fbb12a00640 1 --2- 172.21.15.83:0/2490720835 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613d8a24270 0x5613d8a37170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:41.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.322+0000 7fbb13e00640 1 --2- 172.21.15.83:0/2490720835 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5613d8a2e940 0x5613d8a288f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:41.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.322+0000 7fbb13e00640 1 -- 172.21.15.83:0/2490720835 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613d8a24270 msgr2=0x5613d8a37170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:41.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.322+0000 7fbb13e00640 1 --2- 172.21.15.83:0/2490720835 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613d8a24270 0x5613d8a37170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:41.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.322+0000 7fbb13e00640 1 -- 172.21.15.83:0/2490720835 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5613d8a2e940 msgr2=0x5613d8a288f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:41.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.322+0000 7fbb13e00640 1 --2- 172.21.15.83:0/2490720835 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5613d8a2e940 0x5613d8a288f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:41.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.322+0000 7fbb13e00640 1 -- 172.21.15.83:0/2490720835 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613d8a29060 con 0x5613d8a23950 2024-04-24T09:08:41.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.323+0000 7fbb13e00640 1 --2- 172.21.15.83:0/2490720835 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5613d8a23950 0x5613d8a36c30 secure :-1 s=READY pgs=2323 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c011c60 tx=0x7fbb0c00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f25aae7e7584f6cc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:41.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.323+0000 7fbafcc00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:08:41.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.323+0000 7fbafcc00640 1 -- 172.21.15.83:0/2490720835 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb0c03d060 con 0x5613d8a23950 2024-04-24T09:08:41.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.323+0000 7fbafcc00640 1 -- 172.21.15.83:0/2490720835 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9583+0+0 (secure 0 0 0) 0x7fbb0c004ca0 con 0x5613d8a23950 2024-04-24T09:08:41.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.324+0000 7fbb1cf6c740 1 -- 172.21.15.83:0/2490720835 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb00014df0 con 0x5613d8a23950 2024-04-24T09:08:41.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.324+0000 7fbafcc00640 1 -- 172.21.15.83:0/2490720835 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7fbb0c027040 con 0x5613d8a23950 2024-04-24T09:08:41.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.324+0000 7fbb1cf6c740 10 client.21549 fetch_fsmap learned FSMap version 58 2024-04-24T09:08:41.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.324+0000 7fbb1cf6c740 1 -- 172.21.15.83:0/2490720835 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x5613d8a2f7a0 con 0x5613d8a23950 2024-04-24T09:08:41.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.325+0000 7fbafcc00640 1 -- 172.21.15.83:0/2490720835 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbb00014df0 con 0x5613d8a23950 2024-04-24T09:08:41.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.325+0000 7fbafcc00640 1 -- 172.21.15.83:0/2490720835 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 58) ==== 8374+0+0 (secure 0 0 0) 0x7fbb0c011560 con 0x5613d8a23950 2024-04-24T09:08:41.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.325+0000 7fbb1cf6c740 10 client.21549 fetch_fsmap finished waiting for FSMap version 58 2024-04-24T09:08:41.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.325+0000 7fbb1cf6c740 10 client.21549 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-24T09:08:41.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.325+0000 7fbb1cf6c740 20 client.21549 populate_metadata read hostname 'smithi083' 2024-04-24T09:08:41.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.325+0000 7fbb1cf6c740 1 --2- 172.21.15.83:0/2490720835 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x5613d8aa4eb0 0x5613d8aa72d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:41.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.325+0000 7fbb1cf6c740 4 client.21549 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:08:41.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.325+0000 7fbb1cf6c740 1 -- 172.21.15.83:0/2490720835 --> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5613d8990a60 con 0x5613d8aa4eb0 2024-04-24T09:08:41.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.325+0000 7fbb12a00640 1 --2- 172.21.15.83:0/2490720835 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x5613d8aa4eb0 0x5613d8aa72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:41.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.326+0000 7fbb12a00640 1 --2- 172.21.15.83:0/2490720835 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x5613d8aa4eb0 0x5613d8aa72d0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3ce6e8d0c3e3012f server_cookie=40f34fedc83cec5f in_seq=0 out_seq=0 2024-04-24T09:08:41.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.326+0000 7fbafcc00640 10 client.21549 ms_handle_connect on v2:172.21.15.138:6832/1157967658 2024-04-24T09:08:41.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.326+0000 7fbafcc00640 1 -- 172.21.15.83:0/2490720835 <== mds.3 v2:172.21.15.138:6832/1157967658 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5613d8990a60 con 0x5613d8aa4eb0 2024-04-24T09:08:41.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.326+0000 7fbafcc00640 10 client.21549 handle_command_reply: tid=0 2024-04-24T09:08:41.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.326+0000 7fbb1cf6c740 1 client.21549 shutdown 2024-04-24T09:08:41.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.327+0000 7fbb1cf6c740 1 -- 172.21.15.83:0/2490720835 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5613d8a23950 msgr2=0x5613d8a36c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:41.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.327+0000 7fbb1cf6c740 1 --2- 172.21.15.83:0/2490720835 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5613d8a23950 0x5613d8a36c30 secure :-1 s=READY pgs=2323 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c011c60 tx=0x7fbb0c00ed70 comp rx=0 tx=0).stop 2024-04-24T09:08:41.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.327+0000 7fbb1cf6c740 1 -- 172.21.15.83:0/2490720835 shutdown_connections 2024-04-24T09:08:41.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.327+0000 7fbb1cf6c740 1 --2- 172.21.15.83:0/2490720835 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5613d8a2e940 0x5613d8a288f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:41.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.327+0000 7fbb1cf6c740 1 --2- 172.21.15.83:0/2490720835 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x5613d8aa4eb0 0x5613d8aa72d0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:41.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.327+0000 7fbb1cf6c740 1 --2- 172.21.15.83:0/2490720835 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613d8a24270 0x5613d8a37170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:41.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.327+0000 7fbb1cf6c740 1 --2- 172.21.15.83:0/2490720835 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5613d8a23950 0x5613d8a36c30 unknown :-1 s=CLOSED pgs=2323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:41.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.327+0000 7fbb1cf6c740 1 -- 172.21.15.83:0/2490720835 >> 172.21.15.83:0/2490720835 conn(0x5613d8a1fc70 msgr2=0x5613d8a29f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:41.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.327+0000 7fbafcc00640 0 client.21549 ms_handle_reset on v2:172.21.15.138:6832/1157967658 2024-04-24T09:08:41.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.327+0000 7fbb1cf6c740 1 -- 172.21.15.83:0/2490720835 shutdown_connections 2024-04-24T09:08:41.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.327+0000 7fbb1cf6c740 1 -- 172.21.15.83:0/2490720835 wait complete. 2024-04-24T09:08:41.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.328+0000 7fbb1cf6c740 20 client.21549 trim_cache size 0 max 16384 2024-04-24T09:08:41.329 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:08:41.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.328+0000 7fbb1a200640 1 -- 172.21.15.83:0/4035943384 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fbaf80a3d80 msgr2=0x7fbaf80a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:41.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.328+0000 7fbb1a200640 1 --2- 172.21.15.83:0/4035943384 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fbaf80a3d80 0x7fbaf80a6240 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x5613d8a24f10 tx=0x7fbb04031400 comp rx=0 tx=0).stop 2024-04-24T09:08:41.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.328+0000 7fbb1a200640 1 -- 172.21.15.83:0/4035943384 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb14102240 msgr2=0x7fbb140738a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:41.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.328+0000 7fbb1a200640 1 --2- 172.21.15.83:0/4035943384 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb14102240 0x7fbb140738a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fbb0402a9e0 tx=0x7fbb04003cd0 comp rx=0 tx=0).stop 2024-04-24T09:08:41.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.328+0000 7fbb1a200640 1 -- 172.21.15.83:0/4035943384 shutdown_connections 2024-04-24T09:08:41.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.328+0000 7fbb1a200640 1 --2- 172.21.15.83:0/4035943384 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fbaf80a3d80 0x7fbaf80a6240 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:41.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.328+0000 7fbb1a200640 1 --2- 172.21.15.83:0/4035943384 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbb1406ce60 0x7fbb1406d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:41.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.328+0000 7fbb1a200640 1 --2- 172.21.15.83:0/4035943384 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb14107310 0x7fbb1406c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:41.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.328+0000 7fbb1a200640 1 --2- 172.21.15.83:0/4035943384 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb14102240 0x7fbb140738a0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:41.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.328+0000 7fbb1a200640 1 -- 172.21.15.83:0/4035943384 >> 172.21.15.83:0/4035943384 conn(0x7fbb14079450 msgr2=0x7fbb140fffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:41.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.329+0000 7fbb1a200640 1 -- 172.21.15.83:0/4035943384 shutdown_connections 2024-04-24T09:08:41.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.329+0000 7fbb1a200640 1 -- 172.21.15.83:0/4035943384 wait complete. 2024-04-24T09:08:41.354 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T09:08:41.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.613+0000 7ff468e00640 1 -- 172.21.15.83:0/1684388799 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff464109580 msgr2=0x7ff464111150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:41.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.613+0000 7ff468e00640 1 --2- 172.21.15.83:0/1684388799 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff464109580 0x7ff464111150 secure :-1 s=READY pgs=2324 cs=0 l=1 rev1=1 crypto rx=0x7ff45c009ca0 tx=0x7ff45c02a8c0 comp rx=0 tx=0).stop 2024-04-24T09:08:41.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.613+0000 7ff468e00640 1 -- 172.21.15.83:0/1684388799 shutdown_connections 2024-04-24T09:08:41.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.613+0000 7ff468e00640 1 --2- 172.21.15.83:0/1684388799 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff464109580 0x7ff464111150 unknown :-1 s=CLOSED pgs=2324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:41.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.613+0000 7ff468e00640 1 --2- 172.21.15.83:0/1684388799 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff464104e60 0x7ff464108ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:41.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.613+0000 7ff468e00640 1 --2- 172.21.15.83:0/1684388799 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff4641044b0 0x7ff464104890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:41.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.613+0000 7ff468e00640 1 -- 172.21.15.83:0/1684388799 >> 172.21.15.83:0/1684388799 conn(0x7ff4640ffd90 msgr2=0x7ff4641021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:41.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.613+0000 7ff468e00640 1 -- 172.21.15.83:0/1684388799 shutdown_connections 2024-04-24T09:08:41.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.613+0000 7ff468e00640 1 -- 172.21.15.83:0/1684388799 wait complete. 2024-04-24T09:08:41.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.614+0000 7ff468e00640 1 Processor -- start 2024-04-24T09:08:41.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.614+0000 7ff468e00640 1 -- start start 2024-04-24T09:08:41.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.615+0000 7ff468e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff4641044b0 0x7ff4641a0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:41.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.615+0000 7ff468e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff464109580 0x7ff4641a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:41.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.615+0000 7ff468e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff46419a8e0 0x7ff46419ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:41.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.615+0000 7ff468e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff464114520 con 0x7ff46419a8e0 2024-04-24T09:08:41.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.615+0000 7ff468e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff4641143a0 con 0x7ff4641044b0 2024-04-24T09:08:41.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.615+0000 7ff468e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff4641146a0 con 0x7ff464109580 2024-04-24T09:08:41.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.615+0000 7ff462800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff46419a8e0 0x7ff46419ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:41.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.615+0000 7ff461e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff4641044b0 0x7ff4641a0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:41.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.615+0000 7ff461e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff4641044b0 0x7ff4641a0730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:53560/0 (socket says 172.21.15.83:53560) 2024-04-24T09:08:41.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.615+0000 7ff461e00640 1 -- 172.21.15.83:0/3942823426 learned_addr learned my addr 172.21.15.83:0/3942823426 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:08:41.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.616+0000 7ff461400640 1 --2- 172.21.15.83:0/3942823426 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff464109580 0x7ff4641a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:41.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.616+0000 7ff461e00640 1 -- 172.21.15.83:0/3942823426 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff464109580 msgr2=0x7ff4641a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:41.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.616+0000 7ff461e00640 1 --2- 172.21.15.83:0/3942823426 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff464109580 0x7ff4641a0c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:41.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.616+0000 7ff461e00640 1 -- 172.21.15.83:0/3942823426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff46419a8e0 msgr2=0x7ff46419ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:41.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.616+0000 7ff461e00640 1 --2- 172.21.15.83:0/3942823426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff46419a8e0 0x7ff46419ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:41.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.616+0000 7ff461e00640 1 -- 172.21.15.83:0/3942823426 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff46419b400 con 0x7ff4641044b0 2024-04-24T09:08:41.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.617+0000 7ff461e00640 1 --2- 172.21.15.83:0/3942823426 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff4641044b0 0x7ff4641a0730 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff45400ea60 tx=0x7ff45400ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:41.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.617+0000 7ff45aa00640 1 -- 172.21.15.83:0/3942823426 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff45400cde0 con 0x7ff4641044b0 2024-04-24T09:08:41.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.617+0000 7ff45aa00640 1 -- 172.21.15.83:0/3942823426 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff4540035f0 con 0x7ff4641044b0 2024-04-24T09:08:41.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.617+0000 7ff468e00640 1 -- 172.21.15.83:0/3942823426 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff46419b6f0 con 0x7ff4641044b0 2024-04-24T09:08:41.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.617+0000 7ff468e00640 1 -- 172.21.15.83:0/3942823426 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4641a7940 con 0x7ff4641044b0 2024-04-24T09:08:41.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.618+0000 7ff45aa00640 1 -- 172.21.15.83:0/3942823426 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff454010690 con 0x7ff4641044b0 2024-04-24T09:08:41.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.622+0000 7ff46bbea740 1 Processor -- start 2024-04-24T09:08:41.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.622+0000 7ff46bbea740 1 -- start start 2024-04-24T09:08:41.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.623+0000 7ff46bbea740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fed27f56a0 0x55fed27f5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:41.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.623+0000 7ff46bbea740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fed27f6050 0x55fed2808c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:41.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.623+0000 7ff46bbea740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fed2809150 0x55fed280b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:41.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.623+0000 7ff46bbea740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55fed275b1d0 con 0x55fed2809150 2024-04-24T09:08:41.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.623+0000 7ff46bbea740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55fed2701450 con 0x55fed27f56a0 2024-04-24T09:08:41.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.623+0000 7ff46bbea740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55fed2757190 con 0x55fed27f6050 2024-04-24T09:08:41.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.623+0000 7ff45aa00640 1 -- 172.21.15.83:0/3942823426 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff454004000 con 0x7ff4641044b0 2024-04-24T09:08:41.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.623+0000 7ff461400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fed2809150 0x55fed280b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:41.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.623+0000 7ff461400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fed2809150 0x55fed280b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54730/0 (socket says 172.21.15.83:54730) 2024-04-24T09:08:41.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.623+0000 7ff461400640 1 -- 172.21.15.83:0/3983642609 learned_addr learned my addr 172.21.15.83:0/3983642609 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:08:41.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.624+0000 7ff462800640 1 --2- 172.21.15.83:0/3983642609 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fed27f56a0 0x55fed27f5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:41.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.624+0000 7ff461e00640 1 --2- 172.21.15.83:0/3983642609 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fed27f6050 0x55fed2808c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:41.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.624+0000 7ff462800640 1 -- 172.21.15.83:0/3983642609 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fed27f6050 msgr2=0x55fed2808c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:41.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.624+0000 7ff462800640 1 --2- 172.21.15.83:0/3983642609 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fed27f6050 0x55fed2808c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:41.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.624+0000 7ff462800640 1 -- 172.21.15.83:0/3983642609 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fed2809150 msgr2=0x55fed280b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:41.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.624+0000 7ff462800640 1 --2- 172.21.15.83:0/3983642609 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fed2809150 0x55fed280b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:41.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.624+0000 7ff462800640 1 -- 172.21.15.83:0/3983642609 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fed27be310 con 0x55fed27f56a0 2024-04-24T09:08:41.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.625+0000 7ff462800640 1 --2- 172.21.15.83:0/3983642609 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fed27f56a0 0x55fed27f5a80 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff45c031040 tx=0x7ff45c031a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6ea38693a9513d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:41.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.625+0000 7ff44f400640 1 -- 172.21.15.83:0/3983642609 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff45c031db0 con 0x55fed27f56a0 2024-04-24T09:08:41.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.625+0000 7ff44f400640 1 -- 172.21.15.83:0/3983642609 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff45c036030 con 0x55fed27f56a0 2024-04-24T09:08:41.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.625+0000 7ff44f400640 1 -- 172.21.15.83:0/3983642609 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff45c004370 con 0x55fed27f56a0 2024-04-24T09:08:41.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.626+0000 7ff45aa00640 1 --2- 172.21.15.83:0/3942823426 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff4480a3ef0 0x7ff4480a63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:41.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.626+0000 7ff45aa00640 1 -- 172.21.15.83:0/3942823426 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 9583+0+0 (secure 0 0 0) 0x7ff4540d0d60 con 0x7ff4641044b0 2024-04-24T09:08:41.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.626+0000 7ff461e00640 1 --2- 172.21.15.83:0/3942823426 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff4480a3ef0 0x7ff4480a63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:41.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.626+0000 7ff46bbea740 1 -- 172.21.15.83:0/3983642609 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fed27f56a0 msgr2=0x55fed27f5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:41.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.626+0000 7ff46bbea740 1 --2- 172.21.15.83:0/3983642609 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fed27f56a0 0x55fed27f5a80 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff45c031040 tx=0x7ff45c031a40 comp rx=0 tx=0).stop 2024-04-24T09:08:41.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.626+0000 7ff46bbea740 1 -- 172.21.15.83:0/3983642609 shutdown_connections 2024-04-24T09:08:41.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.626+0000 7ff46bbea740 1 --2- 172.21.15.83:0/3983642609 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fed2809150 0x55fed280b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:41.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.626+0000 7ff46bbea740 1 --2- 172.21.15.83:0/3983642609 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fed27f6050 0x55fed2808c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:41.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.626+0000 7ff46bbea740 1 --2- 172.21.15.83:0/3983642609 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fed27f56a0 0x55fed27f5a80 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:41.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.626+0000 7ff46bbea740 1 -- 172.21.15.83:0/3983642609 >> 172.21.15.83:0/3983642609 conn(0x55fed27f1c70 msgr2=0x55fed27f2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:41.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.627+0000 7ff461e00640 1 --2- 172.21.15.83:0/3942823426 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff4480a3ef0 0x7ff4480a63b0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7ff45409c810 tx=0x7ff45400ef90 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:41.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.627+0000 7ff46bbea740 1 -- 172.21.15.83:0/3983642609 shutdown_connections 2024-04-24T09:08:41.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.627+0000 7ff46bbea740 1 -- 172.21.15.83:0/3983642609 wait complete. 2024-04-24T09:08:41.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.628+0000 7ff46bbea740 1 -- start start 2024-04-24T09:08:41.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.628+0000 7ff46bbea740 1 Processor -- start 2024-04-24T09:08:41.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.628+0000 7ff46bbea740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fed27f56a0 0x55fed2808ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:41.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.628+0000 7ff46bbea740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fed27f6050 0x55fed2805e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:41.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.628+0000 7ff46bbea740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fed2809150 0x55fed28065d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:41.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.628+0000 7ff46bbea740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55fed280bdd0 con 0x55fed2809150 2024-04-24T09:08:41.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.628+0000 7ff46bbea740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55fed2714810 con 0x55fed27f6050 2024-04-24T09:08:41.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.628+0000 7ff46bbea740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55fed280bf50 con 0x55fed27f56a0 2024-04-24T09:08:41.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.628+0000 7ff46bbea740 10 client.0 _ll_register_callbacks cb 0x55fed26e2140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:08:41.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.629+0000 7ff461400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fed27f6050 0x55fed2805e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:41.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.629+0000 7ff461400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fed27f6050 0x55fed2805e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:53582/0 (socket says 172.21.15.83:53582) 2024-04-24T09:08:41.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.629+0000 7ff461400640 1 -- 172.21.15.83:0/4281271975 learned_addr learned my addr 172.21.15.83:0/4281271975 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:08:41.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.629+0000 7ff462800640 1 --2- 172.21.15.83:0/4281271975 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fed2809150 0x55fed28065d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:41.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.629+0000 7ff462800640 1 --2- 172.21.15.83:0/4281271975 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fed27f56a0 0x55fed2808ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:41.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.630+0000 7ff461400640 1 -- 172.21.15.83:0/4281271975 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fed27f56a0 msgr2=0x55fed2808ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:41.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.630+0000 7ff461400640 1 --2- 172.21.15.83:0/4281271975 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fed27f56a0 0x55fed2808ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:41.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.630+0000 7ff461400640 1 -- 172.21.15.83:0/4281271975 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fed2809150 msgr2=0x55fed28065d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:41.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.630+0000 7ff461400640 1 --2- 172.21.15.83:0/4281271975 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fed2809150 0x55fed28065d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:41.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.630+0000 7ff461400640 1 -- 172.21.15.83:0/4281271975 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fed27253f0 con 0x55fed27f6050 2024-04-24T09:08:41.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.630+0000 7ff462800640 1 --2- 172.21.15.83:0/4281271975 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fed27f56a0 0x55fed2808ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:08:41.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.630+0000 7ff461400640 1 --2- 172.21.15.83:0/4281271975 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fed27f6050 0x55fed2805e90 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff450009dc0 tx=0x7ff4500117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f0e424ca60da74d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:41.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.630+0000 7ff443e00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:08:41.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.630+0000 7ff443e00640 1 -- 172.21.15.83:0/4281271975 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff450005220 con 0x55fed27f6050 2024-04-24T09:08:41.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.630+0000 7ff443e00640 1 -- 172.21.15.83:0/4281271975 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9583+0+0 (secure 0 0 0) 0x7ff4500053c0 con 0x55fed27f6050 2024-04-24T09:08:41.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.631+0000 7ff46bbea740 1 -- 172.21.15.83:0/4281271975 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff45c036030 con 0x55fed27f6050 2024-04-24T09:08:41.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.631+0000 7ff443e00640 1 -- 172.21.15.83:0/4281271975 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7ff4500055c0 con 0x55fed27f6050 2024-04-24T09:08:41.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.631+0000 7ff46bbea740 10 client.38702 fetch_fsmap learned FSMap version 58 2024-04-24T09:08:41.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.631+0000 7ff46bbea740 1 -- 172.21.15.83:0/4281271975 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55fed246d3f0 con 0x55fed27f6050 2024-04-24T09:08:41.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.631+0000 7ff443e00640 1 -- 172.21.15.83:0/4281271975 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff45c036030 con 0x55fed27f6050 2024-04-24T09:08:41.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.632+0000 7ff443e00640 1 -- 172.21.15.83:0/4281271975 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 58) ==== 8374+0+0 (secure 0 0 0) 0x7ff45000f040 con 0x55fed27f6050 2024-04-24T09:08:41.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.632+0000 7ff46bbea740 10 client.38702 fetch_fsmap finished waiting for FSMap version 58 2024-04-24T09:08:41.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.632+0000 7ff46bbea740 10 client.38702 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-24T09:08:41.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.632+0000 7ff46bbea740 20 client.38702 populate_metadata read hostname 'smithi083' 2024-04-24T09:08:41.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.632+0000 7ff46bbea740 1 --2- 172.21.15.83:0/4281271975 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55fed2819250 0x55fed281b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:41.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.632+0000 7ff46bbea740 4 client.38702 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:08:41.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.632+0000 7ff46bbea740 1 -- 172.21.15.83:0/4281271975 --> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fed2768bf0 con 0x55fed2819250 2024-04-24T09:08:41.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.632+0000 7ff462800640 1 --2- 172.21.15.83:0/4281271975 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55fed2819250 0x55fed281b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:41.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.633+0000 7ff443e00640 10 client.38702 ms_handle_connect on v2:172.21.15.138:6834/1909219758 2024-04-24T09:08:41.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.634+0000 7ff462800640 1 --2- 172.21.15.83:0/4281271975 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55fed2819250 0x55fed281b640 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9af00afd1f055040 server_cookie=54a510bb6f48261f in_seq=0 out_seq=0 2024-04-24T09:08:41.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.636+0000 7ff443e00640 1 -- 172.21.15.83:0/4281271975 <== mds.2 v2:172.21.15.138:6834/1909219758 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fed2768bf0 con 0x55fed2819250 2024-04-24T09:08:41.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.636+0000 7ff443e00640 10 client.38702 handle_command_reply: tid=0 2024-04-24T09:08:41.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.636+0000 7ff46bbea740 1 client.38702 shutdown 2024-04-24T09:08:41.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.636+0000 7ff46bbea740 1 -- 172.21.15.83:0/4281271975 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fed27f6050 msgr2=0x55fed2805e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:41.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.636+0000 7ff46bbea740 1 --2- 172.21.15.83:0/4281271975 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fed27f6050 0x55fed2805e90 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff450009dc0 tx=0x7ff4500117b0 comp rx=0 tx=0).stop 2024-04-24T09:08:41.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.636+0000 7ff46bbea740 1 -- 172.21.15.83:0/4281271975 shutdown_connections 2024-04-24T09:08:41.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.636+0000 7ff46bbea740 1 --2- 172.21.15.83:0/4281271975 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55fed2819250 0x55fed281b640 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:41.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.636+0000 7ff46bbea740 1 --2- 172.21.15.83:0/4281271975 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fed2809150 0x55fed28065d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:41.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.636+0000 7ff46bbea740 1 --2- 172.21.15.83:0/4281271975 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fed27f6050 0x55fed2805e90 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:41.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.636+0000 7ff46bbea740 1 --2- 172.21.15.83:0/4281271975 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fed27f56a0 0x55fed2808ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:41.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.636+0000 7ff46bbea740 1 -- 172.21.15.83:0/4281271975 >> 172.21.15.83:0/4281271975 conn(0x55fed27f1c70 msgr2=0x55fed27f7ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:41.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.636+0000 7ff443e00640 0 client.38702 ms_handle_reset on v2:172.21.15.138:6834/1909219758 2024-04-24T09:08:41.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.637+0000 7ff46bbea740 1 -- 172.21.15.83:0/4281271975 shutdown_connections 2024-04-24T09:08:41.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.637+0000 7ff46bbea740 1 -- 172.21.15.83:0/4281271975 wait complete. 2024-04-24T09:08:41.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.637+0000 7ff46bbea740 20 client.38702 trim_cache size 0 max 16384 2024-04-24T09:08:41.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.660+0000 7ff46bbea740 1 Processor -- start 2024-04-24T09:08:41.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.660+0000 7ff46bbea740 1 -- start start 2024-04-24T09:08:41.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.660+0000 7ff46bbea740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fed27f56a0 0x55fed27f1570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:41.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.660+0000 7ff46bbea740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fed27f6050 0x55fed2804ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:41.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.660+0000 7ff46bbea740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fed2809150 0x55fed28086d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:41.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.660+0000 7ff46bbea740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff4440100b0 con 0x55fed2809150 2024-04-24T09:08:41.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.660+0000 7ff46bbea740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff444011bf0 con 0x55fed27f6050 2024-04-24T09:08:41.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.660+0000 7ff46bbea740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff444010780 con 0x55fed27f56a0 2024-04-24T09:08:41.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.660+0000 7ff462800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fed2809150 0x55fed28086d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:41.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.660+0000 7ff461400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fed27f6050 0x55fed2804ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:41.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.661+0000 7ff462800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fed2809150 0x55fed28086d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54746/0 (socket says 172.21.15.83:54746) 2024-04-24T09:08:41.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.661+0000 7ff462800640 1 -- 172.21.15.83:0/3833696373 learned_addr learned my addr 172.21.15.83:0/3833696373 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:08:41.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.661+0000 7ff461400640 1 --2- 172.21.15.83:0/3833696373 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fed27f6050 0x55fed2804ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:53590/0 (socket says 172.21.15.83:53590) 2024-04-24T09:08:41.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.661+0000 7ff462800640 1 --2- 172.21.15.83:0/3833696373 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fed27f56a0 0x55fed27f1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:41.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.661+0000 7ff462800640 1 -- 172.21.15.83:0/3833696373 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fed27f56a0 msgr2=0x55fed27f1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:41.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.661+0000 7ff462800640 1 --2- 172.21.15.83:0/3833696373 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fed27f56a0 0x55fed27f1570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:41.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.661+0000 7ff462800640 1 -- 172.21.15.83:0/3833696373 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fed27f6050 msgr2=0x55fed2804ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:41.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.661+0000 7ff462800640 1 --2- 172.21.15.83:0/3833696373 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fed27f6050 0x55fed2804ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:41.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.661+0000 7ff462800640 1 -- 172.21.15.83:0/3833696373 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fed281b9b0 con 0x55fed2809150 2024-04-24T09:08:41.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.661+0000 7ff462800640 1 --2- 172.21.15.83:0/3833696373 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fed2809150 0x55fed28086d0 secure :-1 s=READY pgs=2325 cs=0 l=1 rev1=1 crypto rx=0x7ff45c004ad0 tx=0x7ff45c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81a178eb198f8571 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:41.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.662+0000 7ff44ea00640 1 -- 172.21.15.83:0/3833696373 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff45c059070 con 0x55fed2809150 2024-04-24T09:08:41.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.662+0000 7ff44ea00640 1 -- 172.21.15.83:0/3833696373 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff45c036050 con 0x55fed2809150 2024-04-24T09:08:41.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.662+0000 7ff44ea00640 1 -- 172.21.15.83:0/3833696373 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff45c004310 con 0x55fed2809150 2024-04-24T09:08:41.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.663+0000 7ff46bbea740 1 -- 172.21.15.83:0/3833696373 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fed2809150 msgr2=0x55fed28086d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:41.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.663+0000 7ff46bbea740 1 --2- 172.21.15.83:0/3833696373 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fed2809150 0x55fed28086d0 secure :-1 s=READY pgs=2325 cs=0 l=1 rev1=1 crypto rx=0x7ff45c004ad0 tx=0x7ff45c004040 comp rx=0 tx=0).stop 2024-04-24T09:08:41.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.663+0000 7ff46bbea740 1 -- 172.21.15.83:0/3833696373 shutdown_connections 2024-04-24T09:08:41.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.663+0000 7ff46bbea740 1 --2- 172.21.15.83:0/3833696373 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fed2809150 0x55fed28086d0 unknown :-1 s=CLOSED pgs=2325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:41.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.663+0000 7ff46bbea740 1 --2- 172.21.15.83:0/3833696373 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fed27f6050 0x55fed2804ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:41.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.663+0000 7ff46bbea740 1 --2- 172.21.15.83:0/3833696373 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fed27f56a0 0x55fed27f1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:41.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.663+0000 7ff46bbea740 1 -- 172.21.15.83:0/3833696373 >> 172.21.15.83:0/3833696373 conn(0x55fed27f1c70 msgr2=0x55fed27f2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:41.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.663+0000 7ff46bbea740 1 -- 172.21.15.83:0/3833696373 shutdown_connections 2024-04-24T09:08:41.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.663+0000 7ff46bbea740 1 -- 172.21.15.83:0/3833696373 wait complete. 2024-04-24T09:08:41.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.664+0000 7ff46bbea740 1 -- start start 2024-04-24T09:08:41.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.664+0000 7ff46bbea740 1 Processor -- start 2024-04-24T09:08:41.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.665+0000 7ff46bbea740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fed27f56a0 0x55fed287cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:41.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.665+0000 7ff46bbea740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fed27f6050 0x55fed2811230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:41.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.665+0000 7ff46bbea740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fed2809150 0x55fed2811770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:41.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.665+0000 7ff46bbea740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff444011520 con 0x55fed27f56a0 2024-04-24T09:08:41.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.665+0000 7ff46bbea740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff444010e50 con 0x55fed27f6050 2024-04-24T09:08:41.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.665+0000 7ff46bbea740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff44400e0b0 con 0x55fed2809150 2024-04-24T09:08:41.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.665+0000 7ff46bbea740 10 client.0 _ll_register_callbacks cb 0x7ff4500096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:08:41.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.665+0000 7ff462800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fed27f56a0 0x55fed287cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:41.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.666+0000 7ff462800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fed27f56a0 0x55fed287cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54750/0 (socket says 172.21.15.83:54750) 2024-04-24T09:08:41.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.666+0000 7ff462800640 1 -- 172.21.15.83:0/3355533531 learned_addr learned my addr 172.21.15.83:0/3355533531 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:08:41.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.666+0000 7ff461400640 1 --2- 172.21.15.83:0/3355533531 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fed27f6050 0x55fed2811230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:41.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.666+0000 7ff462800640 1 -- 172.21.15.83:0/3355533531 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fed2809150 msgr2=0x55fed2811770 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-24T09:08:41.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.666+0000 7ff462800640 1 --2- 172.21.15.83:0/3355533531 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fed2809150 0x55fed2811770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:41.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.666+0000 7ff462800640 1 -- 172.21.15.83:0/3355533531 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fed27f6050 msgr2=0x55fed2811230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:41.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.666+0000 7ff462800640 1 --2- 172.21.15.83:0/3355533531 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fed27f6050 0x55fed2811230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:41.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.666+0000 7ff462800640 1 -- 172.21.15.83:0/3355533531 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fed25377d0 con 0x55fed27f56a0 2024-04-24T09:08:41.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.667+0000 7ff462800640 1 --2- 172.21.15.83:0/3355533531 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fed27f56a0 0x55fed287cda0 secure :-1 s=READY pgs=2326 cs=0 l=1 rev1=1 crypto rx=0x7ff45c03e040 tx=0x7ff45c003ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1ad6eb19bcb213 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:41.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.667+0000 7ff443e00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:08:41.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.667+0000 7ff443e00640 1 -- 172.21.15.83:0/3355533531 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff45c059070 con 0x55fed27f56a0 2024-04-24T09:08:41.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.667+0000 7ff443e00640 1 -- 172.21.15.83:0/3355533531 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9583+0+0 (secure 0 0 0) 0x7ff45c003b70 con 0x55fed27f56a0 2024-04-24T09:08:41.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.667+0000 7ff46bbea740 1 -- 172.21.15.83:0/3355533531 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff45c036050 con 0x55fed27f56a0 2024-04-24T09:08:41.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.668+0000 7ff443e00640 1 -- 172.21.15.83:0/3355533531 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7ff45c031450 con 0x55fed27f56a0 2024-04-24T09:08:41.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.668+0000 7ff46bbea740 10 client.21567 fetch_fsmap learned FSMap version 58 2024-04-24T09:08:41.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.668+0000 7ff46bbea740 1 -- 172.21.15.83:0/3355533531 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55fed27fa390 con 0x55fed27f56a0 2024-04-24T09:08:41.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.669+0000 7ff443e00640 1 -- 172.21.15.83:0/3355533531 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff45c036050 con 0x55fed27f56a0 2024-04-24T09:08:41.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.669+0000 7ff443e00640 1 -- 172.21.15.83:0/3355533531 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 58) ==== 8374+0+0 (secure 0 0 0) 0x7ff45c043070 con 0x55fed27f56a0 2024-04-24T09:08:41.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.669+0000 7ff46bbea740 10 client.21567 fetch_fsmap finished waiting for FSMap version 58 2024-04-24T09:08:41.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.669+0000 7ff46bbea740 10 client.21567 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-24T09:08:41.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.669+0000 7ff46bbea740 20 client.21567 populate_metadata read hostname 'smithi083' 2024-04-24T09:08:41.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.669+0000 7ff46bbea740 1 --2- 172.21.15.83:0/3355533531 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55fed27fa6d0 0x55fed27fcaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:41.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.669+0000 7ff46bbea740 4 client.21567 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:08:41.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.669+0000 7ff46bbea740 1 -- 172.21.15.83:0/3355533531 --> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fed2762a60 con 0x55fed27fa6d0 2024-04-24T09:08:41.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.669+0000 7ff461400640 1 --2- 172.21.15.83:0/3355533531 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55fed27fa6d0 0x55fed27fcaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:41.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.670+0000 7ff461400640 1 --2- 172.21.15.83:0/3355533531 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55fed27fa6d0 0x55fed27fcaf0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1cc1009f131c2bc1 server_cookie=32280828fbc00d99 in_seq=0 out_seq=0 2024-04-24T09:08:41.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.670+0000 7ff443e00640 10 client.21567 ms_handle_connect on v2:172.21.15.138:6834/1909219758 2024-04-24T09:08:41.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.672+0000 7ff443e00640 1 -- 172.21.15.83:0/3355533531 <== mds.2 v2:172.21.15.138:6834/1909219758 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fed2762a60 con 0x55fed27fa6d0 2024-04-24T09:08:41.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.672+0000 7ff443e00640 10 client.21567 handle_command_reply: tid=0 2024-04-24T09:08:41.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.672+0000 7ff46bbea740 1 client.21567 shutdown 2024-04-24T09:08:41.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.673+0000 7ff46bbea740 1 -- 172.21.15.83:0/3355533531 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fed27f56a0 msgr2=0x55fed287cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:41.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.673+0000 7ff46bbea740 1 --2- 172.21.15.83:0/3355533531 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fed27f56a0 0x55fed287cda0 secure :-1 s=READY pgs=2326 cs=0 l=1 rev1=1 crypto rx=0x7ff45c03e040 tx=0x7ff45c003ed0 comp rx=0 tx=0).stop 2024-04-24T09:08:41.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.673+0000 7ff46bbea740 1 -- 172.21.15.83:0/3355533531 shutdown_connections 2024-04-24T09:08:41.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.673+0000 7ff46bbea740 1 --2- 172.21.15.83:0/3355533531 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55fed27fa6d0 0x55fed27fcaf0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:41.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.673+0000 7ff46bbea740 1 --2- 172.21.15.83:0/3355533531 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fed2809150 0x55fed2811770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:41.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.673+0000 7ff46bbea740 1 --2- 172.21.15.83:0/3355533531 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fed27f6050 0x55fed2811230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:41.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.673+0000 7ff46bbea740 1 --2- 172.21.15.83:0/3355533531 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fed27f56a0 0x55fed287cda0 unknown :-1 s=CLOSED pgs=2326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:41.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.673+0000 7ff443e00640 0 client.21567 ms_handle_reset on v2:172.21.15.138:6834/1909219758 2024-04-24T09:08:41.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.673+0000 7ff46bbea740 1 -- 172.21.15.83:0/3355533531 >> 172.21.15.83:0/3355533531 conn(0x55fed27f1c70 msgr2=0x55fed27f71e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:41.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.673+0000 7ff46bbea740 1 -- 172.21.15.83:0/3355533531 shutdown_connections 2024-04-24T09:08:41.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.674+0000 7ff46bbea740 1 -- 172.21.15.83:0/3355533531 wait complete. 2024-04-24T09:08:41.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.674+0000 7ff46bbea740 20 client.21567 trim_cache size 0 max 16384 2024-04-24T09:08:41.674 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:08:41.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.675+0000 7ff468e00640 1 -- 172.21.15.83:0/3942823426 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff4480a3ef0 msgr2=0x7ff4480a63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:41.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.675+0000 7ff468e00640 1 --2- 172.21.15.83:0/3942823426 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff4480a3ef0 0x7ff4480a63b0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7ff45409c810 tx=0x7ff45400ef90 comp rx=0 tx=0).stop 2024-04-24T09:08:41.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.675+0000 7ff468e00640 1 -- 172.21.15.83:0/3942823426 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff4641044b0 msgr2=0x7ff4641a0730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:41.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.675+0000 7ff468e00640 1 --2- 172.21.15.83:0/3942823426 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff4641044b0 0x7ff4641a0730 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff45400ea60 tx=0x7ff45400ef20 comp rx=0 tx=0).stop 2024-04-24T09:08:41.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.675+0000 7ff468e00640 1 -- 172.21.15.83:0/3942823426 shutdown_connections 2024-04-24T09:08:41.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.675+0000 7ff468e00640 1 --2- 172.21.15.83:0/3942823426 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff4480a3ef0 0x7ff4480a63b0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:41.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.675+0000 7ff468e00640 1 --2- 172.21.15.83:0/3942823426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff46419a8e0 0x7ff46419ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:41.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.675+0000 7ff468e00640 1 --2- 172.21.15.83:0/3942823426 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff464109580 0x7ff4641a0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:41.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.675+0000 7ff468e00640 1 --2- 172.21.15.83:0/3942823426 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff4641044b0 0x7ff4641a0730 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:41.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.675+0000 7ff468e00640 1 -- 172.21.15.83:0/3942823426 >> 172.21.15.83:0/3942823426 conn(0x7ff4640ffd90 msgr2=0x7ff4641020a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:41.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.675+0000 7ff468e00640 1 -- 172.21.15.83:0/3942823426 shutdown_connections 2024-04-24T09:08:41.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.675+0000 7ff468e00640 1 -- 172.21.15.83:0/3942823426 wait complete. 2024-04-24T09:08:41.694 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T09:08:41.704 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:08:41 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/2886398826' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:08:41.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:08:41 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/2886398826' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:08:41.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.954+0000 7f79bfe00640 1 -- 172.21.15.83:0/2715074163 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79b8074c20 msgr2=0x7f79b810e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:41.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.954+0000 7f79bfe00640 1 --2- 172.21.15.83:0/2715074163 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79b8074c20 0x7f79b810e3a0 secure :-1 s=READY pgs=2327 cs=0 l=1 rev1=1 crypto rx=0x7f79a4009900 tx=0x7f79a402a480 comp rx=0 tx=0).stop 2024-04-24T09:08:41.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.955+0000 7f79bfe00640 1 -- 172.21.15.83:0/2715074163 shutdown_connections 2024-04-24T09:08:41.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.955+0000 7f79bfe00640 1 --2- 172.21.15.83:0/2715074163 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79b810e8e0 0x7f79b8110cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:41.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.955+0000 7f79bfe00640 1 --2- 172.21.15.83:0/2715074163 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79b8074c20 0x7f79b810e3a0 unknown :-1 s=CLOSED pgs=2327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:41.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.955+0000 7f79bfe00640 1 --2- 172.21.15.83:0/2715074163 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f79b8074300 0x7f79b80746e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:41.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.955+0000 7f79bfe00640 1 -- 172.21.15.83:0/2715074163 >> 172.21.15.83:0/2715074163 conn(0x7f79b8079490 msgr2=0x7f79b80798a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:41.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.956+0000 7f79bfe00640 1 -- 172.21.15.83:0/2715074163 shutdown_connections 2024-04-24T09:08:41.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.956+0000 7f79bfe00640 1 -- 172.21.15.83:0/2715074163 wait complete. 2024-04-24T09:08:41.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.957+0000 7f79bfe00640 1 Processor -- start 2024-04-24T09:08:41.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.957+0000 7f79bfe00640 1 -- start start 2024-04-24T09:08:41.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.958+0000 7f79bfe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79b8074300 0x7f79b81a06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:41.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.958+0000 7f79bfe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f79b8074c20 0x7f79b81a0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:41.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.958+0000 7f79bfe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79b810e8e0 0x7f79b819a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:41.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.958+0000 7f79bfe00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f79b8114540 con 0x7f79b8074300 2024-04-24T09:08:41.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.958+0000 7f79bfe00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f79b81143c0 con 0x7f79b810e8e0 2024-04-24T09:08:41.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.958+0000 7f79bfe00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f79b81146c0 con 0x7f79b8074c20 2024-04-24T09:08:41.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.958+0000 7f79bde00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79b810e8e0 0x7f79b819a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:41.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.958+0000 7f79bd400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79b8074300 0x7f79b81a06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:41.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.958+0000 7f79bde00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79b810e8e0 0x7f79b819a880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:53610/0 (socket says 172.21.15.83:53610) 2024-04-24T09:08:41.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.958+0000 7f79bde00640 1 -- 172.21.15.83:0/3018001834 learned_addr learned my addr 172.21.15.83:0/3018001834 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:08:41.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.958+0000 7f79bca00640 1 --2- 172.21.15.83:0/3018001834 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f79b8074c20 0x7f79b81a0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:41.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.959+0000 7f79bde00640 1 -- 172.21.15.83:0/3018001834 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f79b8074c20 msgr2=0x7f79b81a0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:41.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.959+0000 7f79bde00640 1 --2- 172.21.15.83:0/3018001834 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f79b8074c20 0x7f79b81a0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:41.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.959+0000 7f79bde00640 1 -- 172.21.15.83:0/3018001834 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79b8074300 msgr2=0x7f79b81a06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:41.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.959+0000 7f79bde00640 1 --2- 172.21.15.83:0/3018001834 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79b8074300 0x7f79b81a06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:41.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.959+0000 7f79bde00640 1 -- 172.21.15.83:0/3018001834 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79b819b050 con 0x7f79b810e8e0 2024-04-24T09:08:41.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.959+0000 7f79bd400640 1 --2- 172.21.15.83:0/3018001834 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79b8074300 0x7f79b81a06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:08:41.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.959+0000 7f79bca00640 1 --2- 172.21.15.83:0/3018001834 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f79b8074c20 0x7f79b81a0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:08:41.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.960+0000 7f79bde00640 1 --2- 172.21.15.83:0/3018001834 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79b810e8e0 0x7f79b819a880 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f79b000ea60 tx=0x7f79b000ee60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:41.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.960+0000 7f79b6000640 1 -- 172.21.15.83:0/3018001834 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b000cd80 con 0x7f79b810e8e0 2024-04-24T09:08:41.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.960+0000 7f79b6000640 1 -- 172.21.15.83:0/3018001834 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f79b0005020 con 0x7f79b810e8e0 2024-04-24T09:08:41.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.960+0000 7f79b6000640 1 -- 172.21.15.83:0/3018001834 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b0010690 con 0x7f79b810e8e0 2024-04-24T09:08:41.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.960+0000 7f79bfe00640 1 -- 172.21.15.83:0/3018001834 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79b819b340 con 0x7f79b810e8e0 2024-04-24T09:08:41.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.960+0000 7f79bfe00640 1 -- 172.21.15.83:0/3018001834 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79b81a7590 con 0x7f79b810e8e0 2024-04-24T09:08:41.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.965+0000 7f79c2c1e740 1 Processor -- start 2024-04-24T09:08:41.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.966+0000 7f79b6000640 1 -- 172.21.15.83:0/3018001834 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f79b00051d0 con 0x7f79b810e8e0 2024-04-24T09:08:41.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.966+0000 7f79c2c1e740 1 -- start start 2024-04-24T09:08:41.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.967+0000 7f79c2c1e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f34e458950 0x55f34e458d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:41.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.967+0000 7f79c2c1e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f34e459270 0x55f34e463400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:41.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.967+0000 7f79c2c1e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f34e463940 0x55f34e465d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:41.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.967+0000 7f79c2c1e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55f34e3be1d0 con 0x55f34e463940 2024-04-24T09:08:41.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.967+0000 7f79c2c1e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f34e364450 con 0x55f34e459270 2024-04-24T09:08:41.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.967+0000 7f79c2c1e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55f34e3ba190 con 0x55f34e458950 2024-04-24T09:08:41.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.967+0000 7f79bd400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f34e459270 0x55f34e463400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:41.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.967+0000 7f79bd400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f34e459270 0x55f34e463400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:53620/0 (socket says 172.21.15.83:53620) 2024-04-24T09:08:41.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.967+0000 7f79bca00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f34e463940 0x55f34e465d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:41.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.967+0000 7f79bd400640 1 -- 172.21.15.83:0/2522705962 learned_addr learned my addr 172.21.15.83:0/2522705962 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:08:41.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.968+0000 7f79bca00640 1 --2- 172.21.15.83:0/2522705962 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f34e458950 0x55f34e458d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:41.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.968+0000 7f79bd400640 1 -- 172.21.15.83:0/2522705962 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f34e458950 msgr2=0x55f34e458d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:41.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.968+0000 7f79bd400640 1 --2- 172.21.15.83:0/2522705962 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f34e458950 0x55f34e458d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:41.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.968+0000 7f79bd400640 1 -- 172.21.15.83:0/2522705962 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f34e463940 msgr2=0x55f34e465d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:41.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.968+0000 7f79bd400640 1 --2- 172.21.15.83:0/2522705962 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f34e463940 0x55f34e465d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:41.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.968+0000 7f79bd400640 1 -- 172.21.15.83:0/2522705962 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f34e421310 con 0x55f34e459270 2024-04-24T09:08:41.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.968+0000 7f79b6000640 1 --2- 172.21.15.83:0/3018001834 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f79a00a3de0 0x7f79a00a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:41.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.969+0000 7f79bd400640 1 --2- 172.21.15.83:0/2522705962 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f34e459270 0x55f34e463400 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f79ac009e00 tx=0x7f79ac00e5a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6ccee23151bfac server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:41.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.969+0000 7f79bca00640 1 --2- 172.21.15.83:0/3018001834 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f79a00a3de0 0x7f79a00a62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:41.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.969+0000 7f79aa600640 1 -- 172.21.15.83:0/2522705962 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79ac017070 con 0x55f34e459270 2024-04-24T09:08:41.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.969+0000 7f79b6000640 1 -- 172.21.15.83:0/3018001834 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 9583+0+0 (secure 0 0 0) 0x7f79b00d1190 con 0x7f79b810e8e0 2024-04-24T09:08:41.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.969+0000 7f79aa600640 1 -- 172.21.15.83:0/2522705962 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f79ac013920 con 0x55f34e459270 2024-04-24T09:08:41.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.969+0000 7f79aa600640 1 -- 172.21.15.83:0/2522705962 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79ac012920 con 0x55f34e459270 2024-04-24T09:08:41.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.970+0000 7f79c2c1e740 1 -- 172.21.15.83:0/2522705962 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f34e459270 msgr2=0x55f34e463400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:41.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.970+0000 7f79bca00640 1 --2- 172.21.15.83:0/3018001834 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f79a00a3de0 0x7f79a00a62a0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f79a4009850 tx=0x7f79a4031400 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:41.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.970+0000 7f79c2c1e740 1 --2- 172.21.15.83:0/2522705962 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f34e459270 0x55f34e463400 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f79ac009e00 tx=0x7f79ac00e5a0 comp rx=0 tx=0).stop 2024-04-24T09:08:41.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.970+0000 7f79c2c1e740 1 -- 172.21.15.83:0/2522705962 shutdown_connections 2024-04-24T09:08:41.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.970+0000 7f79c2c1e740 1 --2- 172.21.15.83:0/2522705962 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f34e463940 0x55f34e465d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:41.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.970+0000 7f79c2c1e740 1 --2- 172.21.15.83:0/2522705962 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f34e459270 0x55f34e463400 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:41.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.970+0000 7f79c2c1e740 1 --2- 172.21.15.83:0/2522705962 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f34e458950 0x55f34e458d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:41.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.970+0000 7f79c2c1e740 1 -- 172.21.15.83:0/2522705962 >> 172.21.15.83:0/2522705962 conn(0x55f34e454c70 msgr2=0x55f34e455050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:41.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.970+0000 7f79c2c1e740 1 -- 172.21.15.83:0/2522705962 shutdown_connections 2024-04-24T09:08:41.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.971+0000 7f79c2c1e740 1 -- 172.21.15.83:0/2522705962 wait complete. 2024-04-24T09:08:41.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.971+0000 7f79c2c1e740 1 -- start start 2024-04-24T09:08:41.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.971+0000 7f79c2c1e740 1 Processor -- start 2024-04-24T09:08:41.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.972+0000 7f79c2c1e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f34e458950 0x55f34e4632f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:41.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.972+0000 7f79c2c1e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f34e459270 0x55f34e4605e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:41.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.972+0000 7f79c2c1e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f34e463940 0x55f34e460b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:41.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.972+0000 7f79c2c1e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55f34e4667d0 con 0x55f34e458950 2024-04-24T09:08:41.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.972+0000 7f79c2c1e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f34e377810 con 0x55f34e459270 2024-04-24T09:08:41.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.972+0000 7f79c2c1e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55f34e466950 con 0x55f34e463940 2024-04-24T09:08:41.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.972+0000 7f79c2c1e740 10 client.0 _ll_register_callbacks cb 0x55f34e345140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:08:41.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.972+0000 7f79bd400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f34e458950 0x55f34e4632f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:41.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.973+0000 7f79bd400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f34e458950 0x55f34e4632f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54786/0 (socket says 172.21.15.83:54786) 2024-04-24T09:08:41.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.973+0000 7f79bd400640 1 -- 172.21.15.83:0/1378398339 learned_addr learned my addr 172.21.15.83:0/1378398339 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:08:41.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.973+0000 7f79bd400640 1 -- 172.21.15.83:0/1378398339 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f34e463940 msgr2=0x55f34e460b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:41.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.973+0000 7f79bd400640 1 --2- 172.21.15.83:0/1378398339 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f34e463940 0x55f34e460b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:41.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.973+0000 7f79bd400640 1 -- 172.21.15.83:0/1378398339 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f34e459270 msgr2=0x55f34e4605e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:41.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.973+0000 7f79bd400640 1 --2- 172.21.15.83:0/1378398339 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f34e459270 0x55f34e4605e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:41.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.973+0000 7f79bd400640 1 -- 172.21.15.83:0/1378398339 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f34e3883f0 con 0x55f34e458950 2024-04-24T09:08:41.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.973+0000 7f79bd400640 1 --2- 172.21.15.83:0/1378398339 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f34e458950 0x55f34e4632f0 secure :-1 s=READY pgs=2328 cs=0 l=1 rev1=1 crypto rx=0x7f79ac009330 tx=0x7f79ac0123d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aec5de1b45dd06ea server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:41.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.973+0000 7f799ea00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:08:41.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.974+0000 7f799ea00640 1 -- 172.21.15.83:0/1378398339 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79ac017040 con 0x55f34e458950 2024-04-24T09:08:41.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.974+0000 7f799ea00640 1 -- 172.21.15.83:0/1378398339 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9583+0+0 (secure 0 0 0) 0x7f79ac0126a0 con 0x55f34e458950 2024-04-24T09:08:41.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.974+0000 7f79c2c1e740 1 -- 172.21.15.83:0/1378398339 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79ac013920 con 0x55f34e458950 2024-04-24T09:08:41.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.975+0000 7f799ea00640 1 -- 172.21.15.83:0/1378398339 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f79ac0128a0 con 0x55f34e458950 2024-04-24T09:08:41.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.975+0000 7f79c2c1e740 10 client.21579 fetch_fsmap learned FSMap version 58 2024-04-24T09:08:41.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.975+0000 7f79c2c1e740 1 -- 172.21.15.83:0/1378398339 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55f34e0d03f0 con 0x55f34e458950 2024-04-24T09:08:41.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.975+0000 7f799ea00640 1 -- 172.21.15.83:0/1378398339 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f79ac013920 con 0x55f34e458950 2024-04-24T09:08:41.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.976+0000 7f799ea00640 1 -- 172.21.15.83:0/1378398339 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 58) ==== 8374+0+0 (secure 0 0 0) 0x7f79ac023070 con 0x55f34e458950 2024-04-24T09:08:41.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.976+0000 7f79c2c1e740 10 client.21579 fetch_fsmap finished waiting for FSMap version 58 2024-04-24T09:08:41.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.976+0000 7f79c2c1e740 10 client.21579 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-24T09:08:41.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.976+0000 7f79c2c1e740 20 client.21579 populate_metadata read hostname 'smithi083' 2024-04-24T09:08:41.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.976+0000 7f79c2c1e740 1 --2- 172.21.15.83:0/1378398339 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x55f34e47c140 0x55f34e47e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:41.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.977+0000 7f79c2c1e740 4 client.21579 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:08:41.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.977+0000 7f79c2c1e740 1 -- 172.21.15.83:0/1378398339 --> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f34e3cbbf0 con 0x55f34e47c140 2024-04-24T09:08:41.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.977+0000 7f79bca00640 1 --2- 172.21.15.83:0/1378398339 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x55f34e47c140 0x55f34e47e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:41.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.978+0000 7f79bca00640 1 --2- 172.21.15.83:0/1378398339 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x55f34e47c140 0x55f34e47e530 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d08968c21856f479 server_cookie=db836a5db36e4ad4 in_seq=0 out_seq=0 2024-04-24T09:08:41.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.978+0000 7f799ea00640 10 client.21579 ms_handle_connect on v2:172.21.15.138:6836/900566754 2024-04-24T09:08:41.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.979+0000 7f799ea00640 1 -- 172.21.15.83:0/1378398339 <== mds.1 v2:172.21.15.138:6836/900566754 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f34e3cbbf0 con 0x55f34e47c140 2024-04-24T09:08:41.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.979+0000 7f799ea00640 10 client.21579 handle_command_reply: tid=0 2024-04-24T09:08:41.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.979+0000 7f79c2c1e740 1 client.21579 shutdown 2024-04-24T09:08:41.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.979+0000 7f79c2c1e740 1 -- 172.21.15.83:0/1378398339 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f34e458950 msgr2=0x55f34e4632f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:41.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.979+0000 7f79c2c1e740 1 --2- 172.21.15.83:0/1378398339 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f34e458950 0x55f34e4632f0 secure :-1 s=READY pgs=2328 cs=0 l=1 rev1=1 crypto rx=0x7f79ac009330 tx=0x7f79ac0123d0 comp rx=0 tx=0).stop 2024-04-24T09:08:41.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.980+0000 7f79c2c1e740 1 -- 172.21.15.83:0/1378398339 shutdown_connections 2024-04-24T09:08:41.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.980+0000 7f79c2c1e740 1 --2- 172.21.15.83:0/1378398339 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x55f34e47c140 0x55f34e47e530 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:41.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.980+0000 7f79c2c1e740 1 --2- 172.21.15.83:0/1378398339 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f34e463940 0x55f34e460b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:41.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.980+0000 7f79c2c1e740 1 --2- 172.21.15.83:0/1378398339 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f34e459270 0x55f34e4605e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:41.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.980+0000 7f79c2c1e740 1 --2- 172.21.15.83:0/1378398339 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f34e458950 0x55f34e4632f0 unknown :-1 s=CLOSED pgs=2328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:41.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.980+0000 7f79c2c1e740 1 -- 172.21.15.83:0/1378398339 >> 172.21.15.83:0/1378398339 conn(0x55f34e454c70 msgr2=0x55f34e45aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:41.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.980+0000 7f799ea00640 0 client.21579 ms_handle_reset on v2:172.21.15.138:6836/900566754 2024-04-24T09:08:41.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.980+0000 7f79c2c1e740 1 -- 172.21.15.83:0/1378398339 shutdown_connections 2024-04-24T09:08:41.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.980+0000 7f79c2c1e740 1 -- 172.21.15.83:0/1378398339 wait complete. 2024-04-24T09:08:41.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:41.980+0000 7f79c2c1e740 20 client.21579 trim_cache size 0 max 16384 2024-04-24T09:08:42.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.005+0000 7f79c2c1e740 1 Processor -- start 2024-04-24T09:08:42.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.006+0000 7f79c2c1e740 1 -- start start 2024-04-24T09:08:42.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.006+0000 7f79c2c1e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f34e458950 0x55f34e46f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:42.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.006+0000 7f79c2c1e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f34e459270 0x55f34e462e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:42.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.006+0000 7f79c2c1e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f34e463940 0x55f34e483bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:42.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.006+0000 7f79c2c1e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f79b81ad5e0 con 0x55f34e463940 2024-04-24T09:08:42.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.006+0000 7f79c2c1e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f79b81af120 con 0x55f34e459270 2024-04-24T09:08:42.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.006+0000 7f79c2c1e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f79b81adcb0 con 0x55f34e458950 2024-04-24T09:08:42.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.006+0000 7f79bca00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f34e459270 0x55f34e462e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:42.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.007+0000 7f79bca00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f34e459270 0x55f34e462e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:53648/0 (socket says 172.21.15.83:53648) 2024-04-24T09:08:42.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.007+0000 7f79bca00640 1 -- 172.21.15.83:0/3691768783 learned_addr learned my addr 172.21.15.83:0/3691768783 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:08:42.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.007+0000 7f79bde00640 1 --2- 172.21.15.83:0/3691768783 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f34e463940 0x55f34e483bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:42.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.007+0000 7f79bd400640 1 --2- 172.21.15.83:0/3691768783 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f34e458950 0x55f34e46f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:42.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.007+0000 7f79bca00640 1 -- 172.21.15.83:0/3691768783 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f34e458950 msgr2=0x55f34e46f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:42.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.007+0000 7f79bca00640 1 --2- 172.21.15.83:0/3691768783 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f34e458950 0x55f34e46f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:42.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.007+0000 7f79bca00640 1 -- 172.21.15.83:0/3691768783 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f34e463940 msgr2=0x55f34e483bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:42.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.007+0000 7f79bca00640 1 --2- 172.21.15.83:0/3691768783 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f34e463940 0x55f34e483bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:42.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.007+0000 7f79bca00640 1 -- 172.21.15.83:0/3691768783 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f34e460ef0 con 0x55f34e459270 2024-04-24T09:08:42.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.007+0000 7f79bde00640 1 --2- 172.21.15.83:0/3691768783 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f34e463940 0x55f34e483bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:08:42.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.008+0000 7f79bca00640 1 --2- 172.21.15.83:0/3691768783 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f34e459270 0x55f34e462e20 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f79a4009900 tx=0x7f79a4005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de996ceedb1b3ee8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:42.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.008+0000 7f79a9c00640 1 -- 172.21.15.83:0/3691768783 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79a4035040 con 0x55f34e459270 2024-04-24T09:08:42.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.008+0000 7f79a9c00640 1 -- 172.21.15.83:0/3691768783 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f79a4057070 con 0x55f34e459270 2024-04-24T09:08:42.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.008+0000 7f79a9c00640 1 -- 172.21.15.83:0/3691768783 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79a4031e00 con 0x55f34e459270 2024-04-24T09:08:42.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.009+0000 7f79c2c1e740 1 -- 172.21.15.83:0/3691768783 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f34e459270 msgr2=0x55f34e462e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:42.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.009+0000 7f79c2c1e740 1 --2- 172.21.15.83:0/3691768783 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f34e459270 0x55f34e462e20 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f79a4009900 tx=0x7f79a4005fd0 comp rx=0 tx=0).stop 2024-04-24T09:08:42.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.009+0000 7f79c2c1e740 1 -- 172.21.15.83:0/3691768783 shutdown_connections 2024-04-24T09:08:42.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.009+0000 7f79c2c1e740 1 --2- 172.21.15.83:0/3691768783 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f34e463940 0x55f34e483bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:42.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.009+0000 7f79c2c1e740 1 --2- 172.21.15.83:0/3691768783 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f34e459270 0x55f34e462e20 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:42.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.009+0000 7f79c2c1e740 1 --2- 172.21.15.83:0/3691768783 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f34e458950 0x55f34e46f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:42.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.009+0000 7f79c2c1e740 1 -- 172.21.15.83:0/3691768783 >> 172.21.15.83:0/3691768783 conn(0x55f34e454c70 msgr2=0x55f34e45b8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:42.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.009+0000 7f79c2c1e740 1 -- 172.21.15.83:0/3691768783 shutdown_connections 2024-04-24T09:08:42.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.009+0000 7f79c2c1e740 1 -- 172.21.15.83:0/3691768783 wait complete. 2024-04-24T09:08:42.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.010+0000 7f79c2c1e740 1 -- start start 2024-04-24T09:08:42.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.010+0000 7f79c2c1e740 1 Processor -- start 2024-04-24T09:08:42.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.010+0000 7f79c2c1e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f34e458950 0x55f34e463f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:42.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.010+0000 7f79c2c1e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f34e459270 0x55f34e464460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:42.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.011+0000 7f79c2c1e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f34e463940 0x55f34e46abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:42.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.011+0000 7f79c2c1e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f79b81aea50 con 0x55f34e458950 2024-04-24T09:08:42.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.011+0000 7f79c2c1e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f79b81ae380 con 0x55f34e459270 2024-04-24T09:08:42.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.011+0000 7f79c2c1e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f79b81ab5e0 con 0x55f34e463940 2024-04-24T09:08:42.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.011+0000 7f79c2c1e740 10 client.0 _ll_register_callbacks cb 0x7f79b00093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:08:42.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.011+0000 7f79bca00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f34e459270 0x55f34e464460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:42.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.011+0000 7f79bca00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f34e459270 0x55f34e464460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:53664/0 (socket says 172.21.15.83:53664) 2024-04-24T09:08:42.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.011+0000 7f79bca00640 1 -- 172.21.15.83:0/985788317 learned_addr learned my addr 172.21.15.83:0/985788317 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:08:42.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.011+0000 7f79bd400640 1 --2- 172.21.15.83:0/985788317 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f34e458950 0x55f34e463f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:42.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.011+0000 7f79bde00640 1 --2- 172.21.15.83:0/985788317 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f34e463940 0x55f34e46abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:42.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.012+0000 7f79bca00640 1 -- 172.21.15.83:0/985788317 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f34e463940 msgr2=0x55f34e46abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:42.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.012+0000 7f79bca00640 1 --2- 172.21.15.83:0/985788317 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f34e463940 0x55f34e46abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:42.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.012+0000 7f79bca00640 1 -- 172.21.15.83:0/985788317 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f34e458950 msgr2=0x55f34e463f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:42.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.012+0000 7f79bca00640 1 --2- 172.21.15.83:0/985788317 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f34e458950 0x55f34e463f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:42.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.012+0000 7f79bca00640 1 -- 172.21.15.83:0/985788317 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f34e19a7d0 con 0x55f34e459270 2024-04-24T09:08:42.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.012+0000 7f79bca00640 1 --2- 172.21.15.83:0/985788317 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f34e459270 0x55f34e464460 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f79a40057c0 tx=0x7f79a402ac00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d1a621c29d6ba4f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:42.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.012+0000 7f799ea00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:08:42.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.012+0000 7f799ea00640 1 -- 172.21.15.83:0/985788317 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79a4060060 con 0x55f34e459270 2024-04-24T09:08:42.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.013+0000 7f799ea00640 1 -- 172.21.15.83:0/985788317 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9583+0+0 (secure 0 0 0) 0x7f79a40314a0 con 0x55f34e459270 2024-04-24T09:08:42.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.013+0000 7f79c2c1e740 1 -- 172.21.15.83:0/985788317 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79a4057070 con 0x55f34e459270 2024-04-24T09:08:42.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.013+0000 7f799ea00640 1 -- 172.21.15.83:0/985788317 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f79a4035030 con 0x55f34e459270 2024-04-24T09:08:42.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.013+0000 7f79c2c1e740 10 client.38723 fetch_fsmap learned FSMap version 58 2024-04-24T09:08:42.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.013+0000 7f79c2c1e740 1 -- 172.21.15.83:0/985788317 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55f34e46c0e0 con 0x55f34e459270 2024-04-24T09:08:42.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.014+0000 7f799ea00640 1 -- 172.21.15.83:0/985788317 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f79a4031b50 con 0x55f34e459270 2024-04-24T09:08:42.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.014+0000 7f799ea00640 1 -- 172.21.15.83:0/985788317 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 58) ==== 8374+0+0 (secure 0 0 0) 0x7f79a403a050 con 0x55f34e459270 2024-04-24T09:08:42.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.014+0000 7f79c2c1e740 10 client.38723 fetch_fsmap finished waiting for FSMap version 58 2024-04-24T09:08:42.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.014+0000 7f79c2c1e740 10 client.38723 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-24T09:08:42.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.014+0000 7f79c2c1e740 20 client.38723 populate_metadata read hostname 'smithi083' 2024-04-24T09:08:42.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.014+0000 7f79c2c1e740 1 --2- 172.21.15.83:0/985788317 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x55f34e4f3580 0x55f34e4f59a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:42.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.014+0000 7f79c2c1e740 4 client.38723 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:08:42.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.014+0000 7f79c2c1e740 1 -- 172.21.15.83:0/985788317 --> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f79a4057070 con 0x55f34e4f3580 2024-04-24T09:08:42.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.015+0000 7f79bd400640 1 --2- 172.21.15.83:0/985788317 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x55f34e4f3580 0x55f34e4f59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:42.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.015+0000 7f79bd400640 1 --2- 172.21.15.83:0/985788317 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x55f34e4f3580 0x55f34e4f59a0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b38a242c9673ffb7 server_cookie=f0823ffe2c694242 in_seq=0 out_seq=0 2024-04-24T09:08:42.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.015+0000 7f799ea00640 10 client.38723 ms_handle_connect on v2:172.21.15.138:6836/900566754 2024-04-24T09:08:42.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.016+0000 7f799ea00640 1 -- 172.21.15.83:0/985788317 <== mds.1 v2:172.21.15.138:6836/900566754 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f79a4057070 con 0x55f34e4f3580 2024-04-24T09:08:42.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.016+0000 7f799ea00640 10 client.38723 handle_command_reply: tid=0 2024-04-24T09:08:42.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.016+0000 7f79c2c1e740 1 client.38723 shutdown 2024-04-24T09:08:42.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.016+0000 7f79c2c1e740 1 -- 172.21.15.83:0/985788317 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f34e459270 msgr2=0x55f34e464460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:42.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.016+0000 7f79c2c1e740 1 --2- 172.21.15.83:0/985788317 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f34e459270 0x55f34e464460 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f79a40057c0 tx=0x7f79a402ac00 comp rx=0 tx=0).stop 2024-04-24T09:08:42.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.016+0000 7f79c2c1e740 1 -- 172.21.15.83:0/985788317 shutdown_connections 2024-04-24T09:08:42.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.016+0000 7f79c2c1e740 1 --2- 172.21.15.83:0/985788317 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x55f34e4f3580 0x55f34e4f59a0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:42.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.016+0000 7f79c2c1e740 1 --2- 172.21.15.83:0/985788317 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f34e463940 0x55f34e46abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:42.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.016+0000 7f79c2c1e740 1 --2- 172.21.15.83:0/985788317 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f34e459270 0x55f34e464460 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:42.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.016+0000 7f79c2c1e740 1 --2- 172.21.15.83:0/985788317 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f34e458950 0x55f34e463f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:42.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.016+0000 7f79c2c1e740 1 -- 172.21.15.83:0/985788317 >> 172.21.15.83:0/985788317 conn(0x55f34e454c70 msgr2=0x55f34e45edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:42.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.016+0000 7f799ea00640 0 client.38723 ms_handle_reset on v2:172.21.15.138:6836/900566754 2024-04-24T09:08:42.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.017+0000 7f79c2c1e740 1 -- 172.21.15.83:0/985788317 shutdown_connections 2024-04-24T09:08:42.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.017+0000 7f79c2c1e740 1 -- 172.21.15.83:0/985788317 wait complete. 2024-04-24T09:08:42.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.017+0000 7f79c2c1e740 20 client.38723 trim_cache size 0 max 16384 2024-04-24T09:08:42.017 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:08:42.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.018+0000 7f79bfe00640 1 -- 172.21.15.83:0/3018001834 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f79a00a3de0 msgr2=0x7f79a00a62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:42.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.018+0000 7f79bfe00640 1 --2- 172.21.15.83:0/3018001834 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f79a00a3de0 0x7f79a00a62a0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f79a4009850 tx=0x7f79a4031400 comp rx=0 tx=0).stop 2024-04-24T09:08:42.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.018+0000 7f79bfe00640 1 -- 172.21.15.83:0/3018001834 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79b810e8e0 msgr2=0x7f79b819a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:42.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.018+0000 7f79bfe00640 1 --2- 172.21.15.83:0/3018001834 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79b810e8e0 0x7f79b819a880 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f79b000ea60 tx=0x7f79b000ee60 comp rx=0 tx=0).stop 2024-04-24T09:08:42.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.019+0000 7f79bfe00640 1 -- 172.21.15.83:0/3018001834 shutdown_connections 2024-04-24T09:08:42.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.019+0000 7f79bfe00640 1 --2- 172.21.15.83:0/3018001834 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f79a00a3de0 0x7f79a00a62a0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:42.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.019+0000 7f79bfe00640 1 --2- 172.21.15.83:0/3018001834 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79b810e8e0 0x7f79b819a880 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:42.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.019+0000 7f79bfe00640 1 --2- 172.21.15.83:0/3018001834 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f79b8074c20 0x7f79b81a0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:42.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.019+0000 7f79bfe00640 1 --2- 172.21.15.83:0/3018001834 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79b8074300 0x7f79b81a06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:42.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.019+0000 7f79bfe00640 1 -- 172.21.15.83:0/3018001834 >> 172.21.15.83:0/3018001834 conn(0x7f79b8079490 msgr2=0x7f79b80fefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:42.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.019+0000 7f79bfe00640 1 -- 172.21.15.83:0/3018001834 shutdown_connections 2024-04-24T09:08:42.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.019+0000 7f79bfe00640 1 -- 172.21.15.83:0/3018001834 wait complete. 2024-04-24T09:08:42.039 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T09:08:42.167 INFO:tasks.workunit.client.0.smithi083.stdout: 10 24899 6.93 MB/sec execute 73 sec latency 101.772 ms 2024-04-24T09:08:42.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:08:42 smithi083 ceph-mon[27907]: pgmap v537: 129 pgs: 129 active+clean; 496 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.1 MiB/s wr, 205 op/s 2024-04-24T09:08:42.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.309+0000 7fe977200640 1 -- 172.21.15.83:0/1377618938 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe97810c5b0 msgr2=0x7fe978109000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:42.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.309+0000 7fe977200640 1 --2- 172.21.15.83:0/1377618938 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe97810c5b0 0x7fe978109000 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe968009900 tx=0x7fe96802a3f0 comp rx=0 tx=0).stop 2024-04-24T09:08:42.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.309+0000 7fe977200640 1 -- 172.21.15.83:0/1377618938 shutdown_connections 2024-04-24T09:08:42.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.309+0000 7fe977200640 1 --2- 172.21.15.83:0/1377618938 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe978117000 0x7fe9781193f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:42.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.309+0000 7fe977200640 1 --2- 172.21.15.83:0/1377618938 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe978109540 0x7fe9781099a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:42.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.309+0000 7fe977200640 1 --2- 172.21.15.83:0/1377618938 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe97810c5b0 0x7fe978109000 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:42.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.309+0000 7fe977200640 1 -- 172.21.15.83:0/1377618938 >> 172.21.15.83:0/1377618938 conn(0x7fe97806cf80 msgr2=0x7fe97806d390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:42.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.310+0000 7fe977200640 1 -- 172.21.15.83:0/1377618938 shutdown_connections 2024-04-24T09:08:42.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.310+0000 7fe977200640 1 -- 172.21.15.83:0/1377618938 wait complete. 2024-04-24T09:08:42.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.311+0000 7fe977200640 1 Processor -- start 2024-04-24T09:08:42.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.311+0000 7fe977200640 1 -- start start 2024-04-24T09:08:42.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.311+0000 7fe977200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe978109540 0x7fe978116a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:42.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.311+0000 7fe977200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe97810c5b0 0x7fe978113b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:42.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.311+0000 7fe977200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe978117000 0x7fe978114290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:42.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.311+0000 7fe977200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe97811cb90 con 0x7fe97810c5b0 2024-04-24T09:08:42.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.311+0000 7fe977200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe97811ca10 con 0x7fe978109540 2024-04-24T09:08:42.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.311+0000 7fe977200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe97811cd10 con 0x7fe978117000 2024-04-24T09:08:42.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.312+0000 7fe975400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe97810c5b0 0x7fe978113b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:42.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.312+0000 7fe975e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe978109540 0x7fe978116a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:42.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.312+0000 7fe975e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe978109540 0x7fe978116a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:53678/0 (socket says 172.21.15.83:53678) 2024-04-24T09:08:42.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.312+0000 7fe975e00640 1 -- 172.21.15.83:0/2524130106 learned_addr learned my addr 172.21.15.83:0/2524130106 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:08:42.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.312+0000 7fe976800640 1 --2- 172.21.15.83:0/2524130106 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe978117000 0x7fe978114290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:42.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.312+0000 7fe975e00640 1 -- 172.21.15.83:0/2524130106 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe978117000 msgr2=0x7fe978114290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:42.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.312+0000 7fe975e00640 1 --2- 172.21.15.83:0/2524130106 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe978117000 0x7fe978114290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:42.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.312+0000 7fe975e00640 1 -- 172.21.15.83:0/2524130106 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe97810c5b0 msgr2=0x7fe978113b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:42.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.312+0000 7fe975e00640 1 --2- 172.21.15.83:0/2524130106 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe97810c5b0 0x7fe978113b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:42.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.312+0000 7fe975e00640 1 -- 172.21.15.83:0/2524130106 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9781b38b0 con 0x7fe978109540 2024-04-24T09:08:42.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.312+0000 7fe975400640 1 --2- 172.21.15.83:0/2524130106 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe97810c5b0 0x7fe978113b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:08:42.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.313+0000 7fe975e00640 1 --2- 172.21.15.83:0/2524130106 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe978109540 0x7fe978116a30 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe968005ab0 tx=0x7fe968003c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:42.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.313+0000 7fe96ea00640 1 -- 172.21.15.83:0/2524130106 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe96803f070 con 0x7fe978109540 2024-04-24T09:08:42.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.313+0000 7fe977200640 1 -- 172.21.15.83:0/2524130106 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9781b3a80 con 0x7fe978109540 2024-04-24T09:08:42.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.313+0000 7fe96ea00640 1 -- 172.21.15.83:0/2524130106 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe968031b50 con 0x7fe978109540 2024-04-24T09:08:42.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.313+0000 7fe96ea00640 1 -- 172.21.15.83:0/2524130106 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe968035030 con 0x7fe978109540 2024-04-24T09:08:42.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.314+0000 7fe977200640 1 -- 172.21.15.83:0/2524130106 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9781b4020 con 0x7fe978109540 2024-04-24T09:08:42.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.317+0000 7fe97f9c4740 1 Processor -- start 2024-04-24T09:08:42.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.317+0000 7fe97f9c4740 1 -- start start 2024-04-24T09:08:42.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.318+0000 7fe97f9c4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564c4b11db40 0x564c4b11df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:42.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.318+0000 7fe97f9c4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564c4b11e460 0x564c4b1285f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:42.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.318+0000 7fe97f9c4740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564c4b128b30 0x564c4b12af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:42.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.318+0000 7fe97f9c4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x564c4b0831d0 con 0x564c4b11e460 2024-04-24T09:08:42.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.318+0000 7fe97f9c4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x564c4b029450 con 0x564c4b11db40 2024-04-24T09:08:42.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.318+0000 7fe97f9c4740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x564c4b07f190 con 0x564c4b128b30 2024-04-24T09:08:42.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.318+0000 7fe96ea00640 1 -- 172.21.15.83:0/2524130106 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe968042070 con 0x7fe978109540 2024-04-24T09:08:42.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.318+0000 7fe976800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564c4b11db40 0x564c4b11df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:42.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.318+0000 7fe976800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564c4b11db40 0x564c4b11df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:53692/0 (socket says 172.21.15.83:53692) 2024-04-24T09:08:42.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.318+0000 7fe976800640 1 -- 172.21.15.83:0/2491148963 learned_addr learned my addr 172.21.15.83:0/2491148963 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:08:42.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.319+0000 7fe975e00640 1 --2- 172.21.15.83:0/2491148963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564c4b11e460 0x564c4b1285f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:42.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.319+0000 7fe975400640 1 --2- 172.21.15.83:0/2491148963 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564c4b128b30 0x564c4b12af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:42.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.319+0000 7fe976800640 1 -- 172.21.15.83:0/2491148963 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564c4b128b30 msgr2=0x564c4b12af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:42.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.319+0000 7fe976800640 1 --2- 172.21.15.83:0/2491148963 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564c4b128b30 0x564c4b12af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:42.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.319+0000 7fe976800640 1 -- 172.21.15.83:0/2491148963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564c4b11e460 msgr2=0x564c4b1285f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:42.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.319+0000 7fe976800640 1 --2- 172.21.15.83:0/2491148963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564c4b11e460 0x564c4b1285f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:42.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.319+0000 7fe976800640 1 -- 172.21.15.83:0/2491148963 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c4b0e6310 con 0x564c4b11db40 2024-04-24T09:08:42.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.319+0000 7fe975e00640 1 --2- 172.21.15.83:0/2491148963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564c4b11e460 0x564c4b1285f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:08:42.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.320+0000 7fe976800640 1 --2- 172.21.15.83:0/2491148963 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564c4b11db40 0x564c4b11df20 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe970009e20 tx=0x7fe970010740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d7fdd04b5117d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:42.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.320+0000 7fe963000640 1 -- 172.21.15.83:0/2491148963 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe970016070 con 0x564c4b11db40 2024-04-24T09:08:42.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.320+0000 7fe963000640 1 -- 172.21.15.83:0/2491148963 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe970004d20 con 0x564c4b11db40 2024-04-24T09:08:42.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.320+0000 7fe963000640 1 -- 172.21.15.83:0/2491148963 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe970012780 con 0x564c4b11db40 2024-04-24T09:08:42.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.320+0000 7fe96ea00640 1 --2- 172.21.15.83:0/2524130106 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe95c0a3d60 0x7fe95c0a6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:42.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.320+0000 7fe96ea00640 1 -- 172.21.15.83:0/2524130106 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 9583+0+0 (secure 0 0 0) 0x7fe9680f1060 con 0x7fe978109540 2024-04-24T09:08:42.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.320+0000 7fe975e00640 1 --2- 172.21.15.83:0/2524130106 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe95c0a3d60 0x7fe95c0a6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:42.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.321+0000 7fe97f9c4740 1 -- 172.21.15.83:0/2491148963 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564c4b11db40 msgr2=0x564c4b11df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:42.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.321+0000 7fe97f9c4740 1 --2- 172.21.15.83:0/2491148963 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564c4b11db40 0x564c4b11df20 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe970009e20 tx=0x7fe970010740 comp rx=0 tx=0).stop 2024-04-24T09:08:42.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.321+0000 7fe975e00640 1 --2- 172.21.15.83:0/2524130106 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe95c0a3d60 0x7fe95c0a6220 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x564c4b11f100 tx=0x7fe96802a3f0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:42.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.321+0000 7fe97f9c4740 1 -- 172.21.15.83:0/2491148963 shutdown_connections 2024-04-24T09:08:42.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.321+0000 7fe97f9c4740 1 --2- 172.21.15.83:0/2491148963 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564c4b128b30 0x564c4b12af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:42.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.321+0000 7fe97f9c4740 1 --2- 172.21.15.83:0/2491148963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564c4b11e460 0x564c4b1285f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:42.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.321+0000 7fe97f9c4740 1 --2- 172.21.15.83:0/2491148963 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564c4b11db40 0x564c4b11df20 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:42.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.321+0000 7fe97f9c4740 1 -- 172.21.15.83:0/2491148963 >> 172.21.15.83:0/2491148963 conn(0x564c4b119e60 msgr2=0x564c4b11a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:42.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.321+0000 7fe97f9c4740 1 -- 172.21.15.83:0/2491148963 shutdown_connections 2024-04-24T09:08:42.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.321+0000 7fe97f9c4740 1 -- 172.21.15.83:0/2491148963 wait complete. 2024-04-24T09:08:42.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.322+0000 7fe97f9c4740 1 -- start start 2024-04-24T09:08:42.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.322+0000 7fe97f9c4740 1 Processor -- start 2024-04-24T09:08:42.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.323+0000 7fe97f9c4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564c4b11db40 0x564c4b125930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:42.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.323+0000 7fe97f9c4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564c4b11e460 0x564c4b125e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:42.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.323+0000 7fe97f9c4740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564c4b128b30 0x564c4b1263b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:42.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.323+0000 7fe97f9c4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x564c4b12b9c0 con 0x564c4b11e460 2024-04-24T09:08:42.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.323+0000 7fe97f9c4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x564c4b03c810 con 0x564c4b11db40 2024-04-24T09:08:42.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.323+0000 7fe97f9c4740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x564c4b12bb40 con 0x564c4b128b30 2024-04-24T09:08:42.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.323+0000 7fe97f9c4740 10 client.0 _ll_register_callbacks cb 0x564c4b00a140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:08:42.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.323+0000 7fe975400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564c4b11e460 0x564c4b125e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:42.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.323+0000 7fe975400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564c4b11e460 0x564c4b125e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54846/0 (socket says 172.21.15.83:54846) 2024-04-24T09:08:42.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.323+0000 7fe975400640 1 -- 172.21.15.83:0/574204242 learned_addr learned my addr 172.21.15.83:0/574204242 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:08:42.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.324+0000 7fe976800640 1 --2- 172.21.15.83:0/574204242 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564c4b11db40 0x564c4b125930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:42.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.324+0000 7fe976800640 1 --2- 172.21.15.83:0/574204242 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564c4b128b30 0x564c4b1263b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:42.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.324+0000 7fe975400640 1 -- 172.21.15.83:0/574204242 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564c4b128b30 msgr2=0x564c4b1263b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:42.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.324+0000 7fe975400640 1 --2- 172.21.15.83:0/574204242 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564c4b128b30 0x564c4b1263b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:42.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.324+0000 7fe975400640 1 -- 172.21.15.83:0/574204242 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564c4b11db40 msgr2=0x564c4b125930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:42.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.324+0000 7fe975400640 1 --2- 172.21.15.83:0/574204242 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564c4b11db40 0x564c4b125930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:42.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.324+0000 7fe975400640 1 -- 172.21.15.83:0/574204242 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c4b04d3f0 con 0x564c4b11e460 2024-04-24T09:08:42.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.324+0000 7fe976800640 1 --2- 172.21.15.83:0/574204242 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564c4b11db40 0x564c4b125930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:08:42.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.324+0000 7fe975400640 1 --2- 172.21.15.83:0/574204242 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564c4b11e460 0x564c4b125e70 secure :-1 s=READY pgs=2329 cs=0 l=1 rev1=1 crypto rx=0x7fe964012a10 tx=0x7fe964012ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa1a354714d97732 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:42.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.324+0000 7fe957400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:08:42.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.325+0000 7fe957400640 1 -- 172.21.15.83:0/574204242 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe96400f9e0 con 0x564c4b11e460 2024-04-24T09:08:42.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.325+0000 7fe957400640 1 -- 172.21.15.83:0/574204242 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9583+0+0 (secure 0 0 0) 0x7fe96400fb80 con 0x564c4b11e460 2024-04-24T09:08:42.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.325+0000 7fe97f9c4740 1 -- 172.21.15.83:0/574204242 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe970004d20 con 0x564c4b11e460 2024-04-24T09:08:42.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.325+0000 7fe957400640 1 -- 172.21.15.83:0/574204242 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7fe96400fd80 con 0x564c4b11e460 2024-04-24T09:08:42.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.325+0000 7fe97f9c4740 10 client.21597 fetch_fsmap learned FSMap version 58 2024-04-24T09:08:42.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.325+0000 7fe97f9c4740 1 -- 172.21.15.83:0/574204242 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x564c4ad953f0 con 0x564c4b11e460 2024-04-24T09:08:42.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.326+0000 7fe957400640 1 -- 172.21.15.83:0/574204242 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe970004d20 con 0x564c4b11e460 2024-04-24T09:08:42.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.326+0000 7fe957400640 1 -- 172.21.15.83:0/574204242 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 58) ==== 8374+0+0 (secure 0 0 0) 0x7fe964003c00 con 0x564c4b11e460 2024-04-24T09:08:42.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.326+0000 7fe97f9c4740 10 client.21597 fetch_fsmap finished waiting for FSMap version 58 2024-04-24T09:08:42.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.326+0000 7fe97f9c4740 10 client.21597 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:08:42.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.326+0000 7fe97f9c4740 20 client.21597 populate_metadata read hostname 'smithi083' 2024-04-24T09:08:42.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.326+0000 7fe97f9c4740 1 --2- 172.21.15.83:0/574204242 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x564c4b138e20 0x564c4b13b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:42.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.326+0000 7fe97f9c4740 4 client.21597 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:08:42.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.326+0000 7fe97f9c4740 1 -- 172.21.15.83:0/574204242 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564c4b090bf0 con 0x564c4b138e20 2024-04-24T09:08:42.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.327+0000 7fe976800640 1 --2- 172.21.15.83:0/574204242 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x564c4b138e20 0x564c4b13b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:42.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.328+0000 7fe976800640 1 --2- 172.21.15.83:0/574204242 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x564c4b138e20 0x564c4b13b250 crc :-1 s=READY pgs=431 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a96fb7c1255d4d4 server_cookie=9f68a457205ab6da in_seq=0 out_seq=0 2024-04-24T09:08:42.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.328+0000 7fe957400640 10 client.21597 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:08:42.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.329+0000 7fe957400640 1 -- 172.21.15.83:0/574204242 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564c4b090bf0 con 0x564c4b138e20 2024-04-24T09:08:42.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.329+0000 7fe957400640 10 client.21597 handle_command_reply: tid=0 2024-04-24T09:08:42.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.329+0000 7fe97f9c4740 1 client.21597 shutdown 2024-04-24T09:08:42.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.330+0000 7fe97f9c4740 1 -- 172.21.15.83:0/574204242 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564c4b11e460 msgr2=0x564c4b125e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:42.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.330+0000 7fe97f9c4740 1 --2- 172.21.15.83:0/574204242 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564c4b11e460 0x564c4b125e70 secure :-1 s=READY pgs=2329 cs=0 l=1 rev1=1 crypto rx=0x7fe964012a10 tx=0x7fe964012ed0 comp rx=0 tx=0).stop 2024-04-24T09:08:42.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.330+0000 7fe97f9c4740 1 -- 172.21.15.83:0/574204242 shutdown_connections 2024-04-24T09:08:42.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.330+0000 7fe97f9c4740 1 --2- 172.21.15.83:0/574204242 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x564c4b138e20 0x564c4b13b250 crc :-1 s=READY pgs=431 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:42.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.330+0000 7fe97f9c4740 1 --2- 172.21.15.83:0/574204242 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564c4b128b30 0x564c4b1263b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:42.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.330+0000 7fe97f9c4740 1 --2- 172.21.15.83:0/574204242 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564c4b11e460 0x564c4b125e70 unknown :-1 s=CLOSED pgs=2329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:42.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.330+0000 7fe97f9c4740 1 --2- 172.21.15.83:0/574204242 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564c4b11db40 0x564c4b125930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:42.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.330+0000 7fe97f9c4740 1 -- 172.21.15.83:0/574204242 >> 172.21.15.83:0/574204242 conn(0x564c4b119e60 msgr2=0x564c4b11fb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:42.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.330+0000 7fe957400640 0 client.21597 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:08:42.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.331+0000 7fe97f9c4740 1 -- 172.21.15.83:0/574204242 shutdown_connections 2024-04-24T09:08:42.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.331+0000 7fe97f9c4740 1 -- 172.21.15.83:0/574204242 wait complete. 2024-04-24T09:08:42.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.331+0000 7fe97f9c4740 20 client.21597 trim_cache size 0 max 16384 2024-04-24T09:08:42.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.350+0000 7fe97f9c4740 1 Processor -- start 2024-04-24T09:08:42.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.350+0000 7fe97f9c4740 1 -- start start 2024-04-24T09:08:42.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.350+0000 7fe97f9c4740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564c4b11db40 0x564c4b134880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:42.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.350+0000 7fe97f9c4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564c4b11e460 0x564c4b124910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:42.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.350+0000 7fe97f9c4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564c4b128b30 0x564c4b140b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:42.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.350+0000 7fe97f9c4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe9781be230 con 0x564c4b11e460 2024-04-24T09:08:42.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.350+0000 7fe97f9c4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe9781bfd70 con 0x564c4b128b30 2024-04-24T09:08:42.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.350+0000 7fe97f9c4740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe9781be900 con 0x564c4b11db40 2024-04-24T09:08:42.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.351+0000 7fe975400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564c4b11e460 0x564c4b124910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:42.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.351+0000 7fe976800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564c4b128b30 0x564c4b140b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:42.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.351+0000 7fe976800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564c4b128b30 0x564c4b140b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:53702/0 (socket says 172.21.15.83:53702) 2024-04-24T09:08:42.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.351+0000 7fe976800640 1 -- 172.21.15.83:0/1406932933 learned_addr learned my addr 172.21.15.83:0/1406932933 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:08:42.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.351+0000 7fe975400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564c4b11e460 0x564c4b124910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54862/0 (socket says 172.21.15.83:54862) 2024-04-24T09:08:42.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.351+0000 7fe976800640 1 --2- 172.21.15.83:0/1406932933 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564c4b11db40 0x564c4b134880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:42.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.351+0000 7fe975400640 1 -- 172.21.15.83:0/1406932933 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564c4b11db40 msgr2=0x564c4b134880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:42.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.351+0000 7fe975400640 1 --2- 172.21.15.83:0/1406932933 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564c4b11db40 0x564c4b134880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:42.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.351+0000 7fe975400640 1 -- 172.21.15.83:0/1406932933 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564c4b128b30 msgr2=0x564c4b140b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:42.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.351+0000 7fe975400640 1 --2- 172.21.15.83:0/1406932933 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564c4b128b30 0x564c4b140b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:42.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.351+0000 7fe975400640 1 -- 172.21.15.83:0/1406932933 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c4ae5f7d0 con 0x564c4b11e460 2024-04-24T09:08:42.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.352+0000 7fe975400640 1 --2- 172.21.15.83:0/1406932933 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564c4b11e460 0x564c4b124910 secure :-1 s=READY pgs=2330 cs=0 l=1 rev1=1 crypto rx=0x7fe964010640 tx=0x7fe96400fca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4edc58370cd6fc63 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:42.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.352+0000 7fe962600640 1 -- 172.21.15.83:0/1406932933 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9640050b0 con 0x564c4b11e460 2024-04-24T09:08:42.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.352+0000 7fe962600640 1 -- 172.21.15.83:0/1406932933 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe9640106c0 con 0x564c4b11e460 2024-04-24T09:08:42.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.352+0000 7fe962600640 1 -- 172.21.15.83:0/1406932933 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe964003830 con 0x564c4b11e460 2024-04-24T09:08:42.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.353+0000 7fe97f9c4740 1 -- 172.21.15.83:0/1406932933 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564c4b11e460 msgr2=0x564c4b124910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:42.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.353+0000 7fe97f9c4740 1 --2- 172.21.15.83:0/1406932933 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564c4b11e460 0x564c4b124910 secure :-1 s=READY pgs=2330 cs=0 l=1 rev1=1 crypto rx=0x7fe964010640 tx=0x7fe96400fca0 comp rx=0 tx=0).stop 2024-04-24T09:08:42.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.353+0000 7fe97f9c4740 1 -- 172.21.15.83:0/1406932933 shutdown_connections 2024-04-24T09:08:42.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.353+0000 7fe97f9c4740 1 --2- 172.21.15.83:0/1406932933 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564c4b128b30 0x564c4b140b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:42.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.353+0000 7fe97f9c4740 1 --2- 172.21.15.83:0/1406932933 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564c4b11e460 0x564c4b124910 unknown :-1 s=CLOSED pgs=2330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:42.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.353+0000 7fe97f9c4740 1 --2- 172.21.15.83:0/1406932933 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564c4b11db40 0x564c4b134880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:42.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.353+0000 7fe97f9c4740 1 -- 172.21.15.83:0/1406932933 >> 172.21.15.83:0/1406932933 conn(0x564c4b119e60 msgr2=0x564c4b124200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:42.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.354+0000 7fe97f9c4740 1 -- 172.21.15.83:0/1406932933 shutdown_connections 2024-04-24T09:08:42.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.354+0000 7fe97f9c4740 1 -- 172.21.15.83:0/1406932933 wait complete. 2024-04-24T09:08:42.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.354+0000 7fe97f9c4740 1 -- start start 2024-04-24T09:08:42.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.355+0000 7fe97f9c4740 1 Processor -- start 2024-04-24T09:08:42.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.355+0000 7fe97f9c4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564c4b11db40 0x564c4b130d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:42.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.355+0000 7fe97f9c4740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564c4b11e460 0x564c4b1312d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:42.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.355+0000 7fe97f9c4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564c4b128b30 0x564c4b122bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:42.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.355+0000 7fe97f9c4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe9781bf6a0 con 0x564c4b11db40 2024-04-24T09:08:42.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.355+0000 7fe97f9c4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe9781befd0 con 0x564c4b128b30 2024-04-24T09:08:42.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.355+0000 7fe97f9c4740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe9781bc230 con 0x564c4b11e460 2024-04-24T09:08:42.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.355+0000 7fe97f9c4740 10 client.0 _ll_register_callbacks cb 0x7fe964009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:08:42.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.356+0000 7fe976800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564c4b11db40 0x564c4b130d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:42.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.356+0000 7fe976800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564c4b11db40 0x564c4b130d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54866/0 (socket says 172.21.15.83:54866) 2024-04-24T09:08:42.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.356+0000 7fe976800640 1 -- 172.21.15.83:0/3891644673 learned_addr learned my addr 172.21.15.83:0/3891644673 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:08:42.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.356+0000 7fe975400640 1 --2- 172.21.15.83:0/3891644673 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564c4b11e460 0x564c4b1312d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:42.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.356+0000 7fe976800640 1 -- 172.21.15.83:0/3891644673 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564c4b11e460 msgr2=0x564c4b1312d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:42.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.356+0000 7fe976800640 1 --2- 172.21.15.83:0/3891644673 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564c4b11e460 0x564c4b1312d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:42.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.356+0000 7fe976800640 1 -- 172.21.15.83:0/3891644673 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564c4b128b30 msgr2=0x564c4b122bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:42.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.356+0000 7fe976800640 1 --2- 172.21.15.83:0/3891644673 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564c4b128b30 0x564c4b122bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:42.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.356+0000 7fe976800640 1 -- 172.21.15.83:0/3891644673 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c4b1230f0 con 0x564c4b11db40 2024-04-24T09:08:42.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.357+0000 7fe976800640 1 --2- 172.21.15.83:0/3891644673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564c4b11db40 0x564c4b130d90 secure :-1 s=READY pgs=2331 cs=0 l=1 rev1=1 crypto rx=0x7fe970009df0 tx=0x7fe970000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90adb36ab508c2e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:42.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.357+0000 7fe957400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:08:42.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.357+0000 7fe957400640 1 -- 172.21.15.83:0/3891644673 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe970016040 con 0x564c4b11db40 2024-04-24T09:08:42.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.357+0000 7fe957400640 1 -- 172.21.15.83:0/3891644673 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9583+0+0 (secure 0 0 0) 0x7fe970029870 con 0x564c4b11db40 2024-04-24T09:08:42.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.357+0000 7fe97f9c4740 1 -- 172.21.15.83:0/3891644673 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe9640106c0 con 0x564c4b11db40 2024-04-24T09:08:42.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.358+0000 7fe957400640 1 -- 172.21.15.83:0/3891644673 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7fe97003b050 con 0x564c4b11db40 2024-04-24T09:08:42.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.358+0000 7fe97f9c4740 10 client.21603 fetch_fsmap learned FSMap version 58 2024-04-24T09:08:42.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.358+0000 7fe97f9c4740 1 -- 172.21.15.83:0/3891644673 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x564c4b1297a0 con 0x564c4b11db40 2024-04-24T09:08:42.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.358+0000 7fe957400640 1 -- 172.21.15.83:0/3891644673 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe9640106c0 con 0x564c4b11db40 2024-04-24T09:08:42.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.358+0000 7fe957400640 1 -- 172.21.15.83:0/3891644673 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 58) ==== 8374+0+0 (secure 0 0 0) 0x7fe970030020 con 0x564c4b11db40 2024-04-24T09:08:42.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.359+0000 7fe97f9c4740 10 client.21603 fetch_fsmap finished waiting for FSMap version 58 2024-04-24T09:08:42.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.359+0000 7fe97f9c4740 10 client.21603 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:08:42.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.359+0000 7fe97f9c4740 20 client.21603 populate_metadata read hostname 'smithi083' 2024-04-24T09:08:42.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.359+0000 7fe97f9c4740 1 --2- 172.21.15.83:0/3891644673 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x564c4b1b03e0 0x564c4b1b27d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:42.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.359+0000 7fe97f9c4740 4 client.21603 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:08:42.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.359+0000 7fe97f9c4740 1 -- 172.21.15.83:0/3891644673 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564c4b08aa60 con 0x564c4b1b03e0 2024-04-24T09:08:42.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.359+0000 7fe975400640 1 --2- 172.21.15.83:0/3891644673 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x564c4b1b03e0 0x564c4b1b27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:42.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.360+0000 7fe975400640 1 --2- 172.21.15.83:0/3891644673 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x564c4b1b03e0 0x564c4b1b27d0 crc :-1 s=READY pgs=432 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10627fffa3f910d0 server_cookie=c3a0af3fc6238002 in_seq=0 out_seq=0 2024-04-24T09:08:42.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.360+0000 7fe957400640 10 client.21603 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:08:42.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.362+0000 7fe957400640 1 -- 172.21.15.83:0/3891644673 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564c4b08aa60 con 0x564c4b1b03e0 2024-04-24T09:08:42.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.362+0000 7fe957400640 10 client.21603 handle_command_reply: tid=0 2024-04-24T09:08:42.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.362+0000 7fe97f9c4740 1 client.21603 shutdown 2024-04-24T09:08:42.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.362+0000 7fe97f9c4740 1 -- 172.21.15.83:0/3891644673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564c4b11db40 msgr2=0x564c4b130d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:42.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.362+0000 7fe97f9c4740 1 --2- 172.21.15.83:0/3891644673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564c4b11db40 0x564c4b130d90 secure :-1 s=READY pgs=2331 cs=0 l=1 rev1=1 crypto rx=0x7fe970009df0 tx=0x7fe970000f40 comp rx=0 tx=0).stop 2024-04-24T09:08:42.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.362+0000 7fe97f9c4740 1 -- 172.21.15.83:0/3891644673 shutdown_connections 2024-04-24T09:08:42.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.362+0000 7fe97f9c4740 1 --2- 172.21.15.83:0/3891644673 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x564c4b1b03e0 0x564c4b1b27d0 crc :-1 s=READY pgs=432 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:42.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.362+0000 7fe97f9c4740 1 --2- 172.21.15.83:0/3891644673 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564c4b128b30 0x564c4b122bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:42.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.362+0000 7fe97f9c4740 1 --2- 172.21.15.83:0/3891644673 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564c4b11e460 0x564c4b1312d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:42.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.362+0000 7fe97f9c4740 1 --2- 172.21.15.83:0/3891644673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564c4b11db40 0x564c4b130d90 unknown :-1 s=CLOSED pgs=2331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:42.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.362+0000 7fe97f9c4740 1 -- 172.21.15.83:0/3891644673 >> 172.21.15.83:0/3891644673 conn(0x564c4b119e60 msgr2=0x564c4b125870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:42.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.362+0000 7fe957400640 0 client.21603 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:08:42.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.362+0000 7fe97f9c4740 1 -- 172.21.15.83:0/3891644673 shutdown_connections 2024-04-24T09:08:42.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.362+0000 7fe97f9c4740 1 -- 172.21.15.83:0/3891644673 wait complete. 2024-04-24T09:08:42.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.363+0000 7fe97f9c4740 20 client.21603 trim_cache size 0 max 16384 2024-04-24T09:08:42.363 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:08:42.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.364+0000 7fe977200640 1 -- 172.21.15.83:0/2524130106 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe95c0a3d60 msgr2=0x7fe95c0a6220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:42.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.364+0000 7fe977200640 1 --2- 172.21.15.83:0/2524130106 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe95c0a3d60 0x7fe95c0a6220 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x564c4b11f100 tx=0x7fe96802a3f0 comp rx=0 tx=0).stop 2024-04-24T09:08:42.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.364+0000 7fe977200640 1 -- 172.21.15.83:0/2524130106 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe978109540 msgr2=0x7fe978116a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:42.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.364+0000 7fe977200640 1 --2- 172.21.15.83:0/2524130106 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe978109540 0x7fe978116a30 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe968005ab0 tx=0x7fe968003c20 comp rx=0 tx=0).stop 2024-04-24T09:08:42.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.364+0000 7fe977200640 1 -- 172.21.15.83:0/2524130106 shutdown_connections 2024-04-24T09:08:42.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.364+0000 7fe977200640 1 --2- 172.21.15.83:0/2524130106 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe95c0a3d60 0x7fe95c0a6220 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:42.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.364+0000 7fe977200640 1 --2- 172.21.15.83:0/2524130106 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe978117000 0x7fe978114290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:42.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.364+0000 7fe977200640 1 --2- 172.21.15.83:0/2524130106 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe97810c5b0 0x7fe978113b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:42.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.364+0000 7fe977200640 1 --2- 172.21.15.83:0/2524130106 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe978109540 0x7fe978116a30 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:42.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.364+0000 7fe977200640 1 -- 172.21.15.83:0/2524130106 >> 172.21.15.83:0/2524130106 conn(0x7fe97806cf80 msgr2=0x7fe9781173e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:42.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.364+0000 7fe977200640 1 -- 172.21.15.83:0/2524130106 shutdown_connections 2024-04-24T09:08:42.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:42.364+0000 7fe977200640 1 -- 172.21.15.83:0/2524130106 wait complete. 2024-04-24T09:08:42.704 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:08:42 smithi138 ceph-mon[31441]: pgmap v537: 129 pgs: 129 active+clean; 496 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.1 MiB/s wr, 205 op/s 2024-04-24T09:08:42.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:08:42 smithi189 ceph-mon[31791]: pgmap v537: 129 pgs: 129 active+clean; 496 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.1 MiB/s wr, 205 op/s 2024-04-24T09:08:43.167 INFO:tasks.workunit.client.0.smithi083.stdout: 10 25047 6.94 MB/sec execute 74 sec latency 88.678 ms 2024-04-24T09:08:43.386 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T09:08:43.386 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T09:08:43.386 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T09:08:43.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.645+0000 7feb99200640 1 -- 172.21.15.83:0/885154810 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb94103980 msgr2=0x7feb9410be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:43.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.645+0000 7feb99200640 1 --2- 172.21.15.83:0/885154810 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb94103980 0x7feb9410be60 secure :-1 s=READY pgs=2332 cs=0 l=1 rev1=1 crypto rx=0x7feb80009900 tx=0x7feb8002a480 comp rx=0 tx=0).stop 2024-04-24T09:08:43.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.646+0000 7feb99200640 1 -- 172.21.15.83:0/885154810 shutdown_connections 2024-04-24T09:08:43.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.646+0000 7feb99200640 1 --2- 172.21.15.83:0/885154810 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feb9410c480 0x7feb9410e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:43.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.646+0000 7feb99200640 1 --2- 172.21.15.83:0/885154810 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb94103980 0x7feb9410be60 unknown :-1 s=CLOSED pgs=2332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:43.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.646+0000 7feb99200640 1 --2- 172.21.15.83:0/885154810 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb94103060 0x7feb94103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:43.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.646+0000 7feb99200640 1 -- 172.21.15.83:0/885154810 >> 172.21.15.83:0/885154810 conn(0x7feb940fc880 msgr2=0x7feb940feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:43.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.646+0000 7feb99200640 1 -- 172.21.15.83:0/885154810 shutdown_connections 2024-04-24T09:08:43.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.646+0000 7feb99200640 1 -- 172.21.15.83:0/885154810 wait complete. 2024-04-24T09:08:43.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.647+0000 7feb99200640 1 Processor -- start 2024-04-24T09:08:43.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.647+0000 7feb99200640 1 -- start start 2024-04-24T09:08:43.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.648+0000 7feb99200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feb94103060 0x7feb941986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:43.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.648+0000 7feb99200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb94103980 0x7feb94198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:43.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.648+0000 7feb99200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb9410c480 0x7feb9419cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:43.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.648+0000 7feb99200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7feb94112140 con 0x7feb9410c480 2024-04-24T09:08:43.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.648+0000 7feb99200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7feb94111fc0 con 0x7feb94103980 2024-04-24T09:08:43.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.648+0000 7feb99200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7feb941122c0 con 0x7feb94103060 2024-04-24T09:08:43.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.648+0000 7feb93e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb9410c480 0x7feb9419cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:43.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.648+0000 7feb93400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feb94103060 0x7feb941986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:43.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.648+0000 7feb93e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb9410c480 0x7feb9419cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54892/0 (socket says 172.21.15.83:54892) 2024-04-24T09:08:43.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.648+0000 7feb93e00640 1 -- 172.21.15.83:0/3536751163 learned_addr learned my addr 172.21.15.83:0/3536751163 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:08:43.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.648+0000 7feb92a00640 1 --2- 172.21.15.83:0/3536751163 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb94103980 0x7feb94198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:43.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.649+0000 7feb93e00640 1 -- 172.21.15.83:0/3536751163 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feb94103060 msgr2=0x7feb941986a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:43.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.649+0000 7feb93e00640 1 --2- 172.21.15.83:0/3536751163 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feb94103060 0x7feb941986a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:43.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.649+0000 7feb93e00640 1 -- 172.21.15.83:0/3536751163 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb94103980 msgr2=0x7feb94198be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:43.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.649+0000 7feb93e00640 1 --2- 172.21.15.83:0/3536751163 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb94103980 0x7feb94198be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:43.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.649+0000 7feb93e00640 1 -- 172.21.15.83:0/3536751163 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb9419d640 con 0x7feb9410c480 2024-04-24T09:08:43.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.649+0000 7feb93e00640 1 --2- 172.21.15.83:0/3536751163 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb9410c480 0x7feb9419cf10 secure :-1 s=READY pgs=2333 cs=0 l=1 rev1=1 crypto rx=0x7feb8c00cce0 tx=0x7feb8c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:43.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.650+0000 7feb8be00640 1 -- 172.21.15.83:0/3536751163 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb8c00f450 con 0x7feb9410c480 2024-04-24T09:08:43.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.650+0000 7feb8be00640 1 -- 172.21.15.83:0/3536751163 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feb8c00faa0 con 0x7feb9410c480 2024-04-24T09:08:43.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.650+0000 7feb99200640 1 -- 172.21.15.83:0/3536751163 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb9419d930 con 0x7feb9410c480 2024-04-24T09:08:43.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.650+0000 7feb99200640 1 -- 172.21.15.83:0/3536751163 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb94100be0 con 0x7feb9410c480 2024-04-24T09:08:43.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.650+0000 7feb8be00640 1 -- 172.21.15.83:0/3536751163 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb8c018b70 con 0x7feb9410c480 2024-04-24T09:08:43.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.654+0000 7feb8be00640 1 -- 172.21.15.83:0/3536751163 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7feb8c018400 con 0x7feb9410c480 2024-04-24T09:08:43.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.654+0000 7feb9d61d740 1 Processor -- start 2024-04-24T09:08:43.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.655+0000 7feb9d61d740 1 -- start start 2024-04-24T09:08:43.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.655+0000 7feb9d61d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562a954aa270 0x562a954aa650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:43.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.655+0000 7feb9d61d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a954aab90 0x562a954b4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:43.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.655+0000 7feb9d61d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562a954b5260 0x562a954b7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:43.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.655+0000 7feb9d61d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562a9540fe70 con 0x562a954b5260 2024-04-24T09:08:43.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.655+0000 7feb9d61d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562a953b6380 con 0x562a954aa270 2024-04-24T09:08:43.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.655+0000 7feb9d61d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x562a9540be30 con 0x562a954aab90 2024-04-24T09:08:43.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.656+0000 7feb92a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562a954b5260 0x562a954b7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:43.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.656+0000 7feb92a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562a954b5260 0x562a954b7650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54908/0 (socket says 172.21.15.83:54908) 2024-04-24T09:08:43.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.656+0000 7feb92a00640 1 -- 172.21.15.83:0/2250387212 learned_addr learned my addr 172.21.15.83:0/2250387212 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:08:43.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.656+0000 7feb92a00640 1 --2- 172.21.15.83:0/2250387212 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562a954aa270 0x562a954aa650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:43.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.656+0000 7feb8be00640 1 --2- 172.21.15.83:0/3536751163 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7feb780a3da0 0x7feb780a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:43.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.656+0000 7feb93400640 1 --2- 172.21.15.83:0/2250387212 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a954aab90 0x562a954b4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:43.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.656+0000 7feb8be00640 1 -- 172.21.15.83:0/3536751163 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 9583+0+0 (secure 0 0 0) 0x7feb8c0d1050 con 0x7feb9410c480 2024-04-24T09:08:43.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.656+0000 7feb93e00640 1 --2- 172.21.15.83:0/3536751163 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7feb780a3da0 0x7feb780a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:43.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.657+0000 7feb92a00640 1 -- 172.21.15.83:0/2250387212 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a954aab90 msgr2=0x562a954b4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:43.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.657+0000 7feb92a00640 1 --2- 172.21.15.83:0/2250387212 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a954aab90 0x562a954b4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:43.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.657+0000 7feb92a00640 1 -- 172.21.15.83:0/2250387212 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562a954aa270 msgr2=0x562a954aa650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:43.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.657+0000 7feb92a00640 1 --2- 172.21.15.83:0/2250387212 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562a954aa270 0x562a954aa650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:43.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.657+0000 7feb92a00640 1 -- 172.21.15.83:0/2250387212 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a951ea7e0 con 0x562a954b5260 2024-04-24T09:08:43.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.657+0000 7feb93e00640 1 --2- 172.21.15.83:0/3536751163 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7feb780a3da0 0x7feb780a6260 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7feb8c0995a0 tx=0x7feb8c005c10 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:43.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.658+0000 7feb92a00640 1 --2- 172.21.15.83:0/2250387212 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562a954b5260 0x562a954b7650 secure :-1 s=READY pgs=2334 cs=0 l=1 rev1=1 crypto rx=0x7feb80009230 tx=0x7feb80000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=796e0820e7f90157 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:43.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.659+0000 7feb88c00640 1 -- 172.21.15.83:0/2250387212 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb80035040 con 0x562a954b5260 2024-04-24T09:08:43.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.659+0000 7feb88c00640 1 -- 172.21.15.83:0/2250387212 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feb80005020 con 0x562a954b5260 2024-04-24T09:08:43.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.659+0000 7feb88c00640 1 -- 172.21.15.83:0/2250387212 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb80004bc0 con 0x562a954b5260 2024-04-24T09:08:43.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.660+0000 7feb9d61d740 1 -- 172.21.15.83:0/2250387212 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562a954b5260 msgr2=0x562a954b7650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:43.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.660+0000 7feb9d61d740 1 --2- 172.21.15.83:0/2250387212 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562a954b5260 0x562a954b7650 secure :-1 s=READY pgs=2334 cs=0 l=1 rev1=1 crypto rx=0x7feb80009230 tx=0x7feb80000bc0 comp rx=0 tx=0).stop 2024-04-24T09:08:43.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.660+0000 7feb9d61d740 1 -- 172.21.15.83:0/2250387212 shutdown_connections 2024-04-24T09:08:43.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.660+0000 7feb9d61d740 1 --2- 172.21.15.83:0/2250387212 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562a954b5260 0x562a954b7650 unknown :-1 s=CLOSED pgs=2334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:43.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.660+0000 7feb9d61d740 1 --2- 172.21.15.83:0/2250387212 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a954aab90 0x562a954b4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:43.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.660+0000 7feb9d61d740 1 --2- 172.21.15.83:0/2250387212 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562a954aa270 0x562a954aa650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:43.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.660+0000 7feb9d61d740 1 -- 172.21.15.83:0/2250387212 >> 172.21.15.83:0/2250387212 conn(0x562a954a6590 msgr2=0x562a954a6970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:43.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.660+0000 7feb9d61d740 1 -- 172.21.15.83:0/2250387212 shutdown_connections 2024-04-24T09:08:43.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.660+0000 7feb9d61d740 1 -- 172.21.15.83:0/2250387212 wait complete. 2024-04-24T09:08:43.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.661+0000 7feb9d61d740 1 -- start start 2024-04-24T09:08:43.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.661+0000 7feb9d61d740 1 Processor -- start 2024-04-24T09:08:43.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.662+0000 7feb9d61d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a954aa270 0x562a954b2150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:43.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.662+0000 7feb9d61d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562a954aab90 0x562a954b2690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:43.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.662+0000 7feb9d61d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562a954b5260 0x562a954b2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:43.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.662+0000 7feb9d61d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562a954b7ff0 con 0x562a954b5260 2024-04-24T09:08:43.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.662+0000 7feb9d61d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562a953c9660 con 0x562a954aab90 2024-04-24T09:08:43.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.662+0000 7feb9d61d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x562a954b8170 con 0x562a954aa270 2024-04-24T09:08:43.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.662+0000 7feb9d61d740 10 client.0 _ll_register_callbacks cb 0x562a953b7aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:08:43.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.663+0000 7feb93400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562a954aab90 0x562a954b2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:43.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.663+0000 7feb93400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562a954aab90 0x562a954b2690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:53748/0 (socket says 172.21.15.83:53748) 2024-04-24T09:08:43.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.663+0000 7feb93400640 1 -- 172.21.15.83:0/3396016864 learned_addr learned my addr 172.21.15.83:0/3396016864 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:08:43.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.663+0000 7feb92a00640 1 --2- 172.21.15.83:0/3396016864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562a954b5260 0x562a954b2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:43.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.663+0000 7feb92a00640 1 --2- 172.21.15.83:0/3396016864 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a954aa270 0x562a954b2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:43.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.664+0000 7feb93400640 1 -- 172.21.15.83:0/3396016864 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a954aa270 msgr2=0x562a954b2150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:43.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.664+0000 7feb93400640 1 --2- 172.21.15.83:0/3396016864 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a954aa270 0x562a954b2150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:43.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.664+0000 7feb93400640 1 -- 172.21.15.83:0/3396016864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562a954b5260 msgr2=0x562a954b2bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:43.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.664+0000 7feb93400640 1 --2- 172.21.15.83:0/3396016864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562a954b5260 0x562a954b2bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:43.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.664+0000 7feb93400640 1 -- 172.21.15.83:0/3396016864 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a9524af70 con 0x562a954aab90 2024-04-24T09:08:43.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.665+0000 7feb93400640 1 --2- 172.21.15.83:0/3396016864 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562a954aab90 0x562a954b2690 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7feb84007860 tx=0x7feb84007d20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c4a2b06217628a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:43.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.665+0000 7feb7d600640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:08:43.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.666+0000 7feb7d600640 1 -- 172.21.15.83:0/3396016864 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb8401f060 con 0x562a954aab90 2024-04-24T09:08:43.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.666+0000 7feb9d61d740 1 -- 172.21.15.83:0/3396016864 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb80005020 con 0x562a954aab90 2024-04-24T09:08:43.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.666+0000 7feb7d600640 1 -- 172.21.15.83:0/3396016864 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9583+0+0 (secure 0 0 0) 0x7feb84014ac0 con 0x562a954aab90 2024-04-24T09:08:43.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.666+0000 7feb7d600640 1 -- 172.21.15.83:0/3396016864 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7feb84026050 con 0x562a954aab90 2024-04-24T09:08:43.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.666+0000 7feb9d61d740 10 client.38747 fetch_fsmap learned FSMap version 58 2024-04-24T09:08:43.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.666+0000 7feb9d61d740 1 -- 172.21.15.83:0/3396016864 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x562a950e2170 con 0x562a954aab90 2024-04-24T09:08:43.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.667+0000 7feb7d600640 1 -- 172.21.15.83:0/3396016864 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feb80005020 con 0x562a954aab90 2024-04-24T09:08:43.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.667+0000 7feb7d600640 1 -- 172.21.15.83:0/3396016864 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 58) ==== 8374+0+0 (secure 0 0 0) 0x7feb84019350 con 0x562a954aab90 2024-04-24T09:08:43.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.667+0000 7feb9d61d740 10 client.38747 fetch_fsmap finished waiting for FSMap version 58 2024-04-24T09:08:43.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.667+0000 7feb9d61d740 10 client.38747 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:08:43.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.667+0000 7feb9d61d740 20 client.38747 populate_metadata read hostname 'smithi083' 2024-04-24T09:08:43.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.667+0000 7feb9d61d740 1 --2- 172.21.15.83:0/3396016864 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x562a954c5670 0x562a954c7a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:43.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.668+0000 7feb9d61d740 4 client.38747 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:08:43.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.668+0000 7feb9d61d740 1 -- 172.21.15.83:0/3396016864 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562a9541d890 con 0x562a954c5670 2024-04-24T09:08:43.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.668+0000 7feb92a00640 1 --2- 172.21.15.83:0/3396016864 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x562a954c5670 0x562a954c7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:43.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.669+0000 7feb92a00640 1 --2- 172.21.15.83:0/3396016864 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x562a954c5670 0x562a954c7a90 crc :-1 s=READY pgs=433 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e5029358844556c server_cookie=db38d158f1908015 in_seq=0 out_seq=0 2024-04-24T09:08:43.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.669+0000 7feb7d600640 10 client.38747 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:08:43.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.670+0000 7feb7d600640 1 -- 172.21.15.83:0/3396016864 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562a9541d890 con 0x562a954c5670 2024-04-24T09:08:43.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.670+0000 7feb7d600640 10 client.38747 handle_command_reply: tid=0 2024-04-24T09:08:43.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.670+0000 7feb9d61d740 1 client.38747 shutdown 2024-04-24T09:08:43.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.670+0000 7feb9d61d740 1 -- 172.21.15.83:0/3396016864 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562a954aab90 msgr2=0x562a954b2690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:43.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.670+0000 7feb9d61d740 1 --2- 172.21.15.83:0/3396016864 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562a954aab90 0x562a954b2690 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7feb84007860 tx=0x7feb84007d20 comp rx=0 tx=0).stop 2024-04-24T09:08:43.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.670+0000 7feb9d61d740 1 -- 172.21.15.83:0/3396016864 shutdown_connections 2024-04-24T09:08:43.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.670+0000 7feb9d61d740 1 --2- 172.21.15.83:0/3396016864 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x562a954c5670 0x562a954c7a90 crc :-1 s=READY pgs=433 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:43.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.670+0000 7feb9d61d740 1 --2- 172.21.15.83:0/3396016864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562a954b5260 0x562a954b2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:43.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.670+0000 7feb9d61d740 1 --2- 172.21.15.83:0/3396016864 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562a954aab90 0x562a954b2690 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:43.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.670+0000 7feb9d61d740 1 --2- 172.21.15.83:0/3396016864 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a954aa270 0x562a954b2150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:43.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.670+0000 7feb7d600640 0 client.38747 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:08:43.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.670+0000 7feb9d61d740 1 -- 172.21.15.83:0/3396016864 >> 172.21.15.83:0/3396016864 conn(0x562a954a6590 msgr2=0x562a954ac290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:43.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.671+0000 7feb9d61d740 1 -- 172.21.15.83:0/3396016864 shutdown_connections 2024-04-24T09:08:43.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.671+0000 7feb9d61d740 1 -- 172.21.15.83:0/3396016864 wait complete. 2024-04-24T09:08:43.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.671+0000 7feb9d61d740 20 client.38747 trim_cache size 0 max 16384 2024-04-24T09:08:43.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.699+0000 7feb9d61d740 1 Processor -- start 2024-04-24T09:08:43.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.699+0000 7feb9d61d740 1 -- start start 2024-04-24T09:08:43.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.699+0000 7feb9d61d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a954aa270 0x562a954c0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:43.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.699+0000 7feb9d61d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562a954aab90 0x562a954cd3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:43.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.699+0000 7feb9d61d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562a954b5260 0x562a954cd8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:43.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.699+0000 7feb9d61d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7feb740100c0 con 0x562a954aab90 2024-04-24T09:08:43.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.699+0000 7feb9d61d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7feb74011c00 con 0x562a954b5260 2024-04-24T09:08:43.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.699+0000 7feb9d61d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7feb74010790 con 0x562a954aa270 2024-04-24T09:08:43.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.700+0000 7feb93400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562a954aab90 0x562a954cd3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:43.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.700+0000 7feb92a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562a954b5260 0x562a954cd8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:43.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.700+0000 7feb92a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562a954b5260 0x562a954cd8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:53760/0 (socket says 172.21.15.83:53760) 2024-04-24T09:08:43.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.700+0000 7feb92a00640 1 -- 172.21.15.83:0/2938630595 learned_addr learned my addr 172.21.15.83:0/2938630595 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:08:43.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.700+0000 7feb92a00640 1 --2- 172.21.15.83:0/2938630595 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a954aa270 0x562a954c0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:43.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.701+0000 7feb92a00640 1 -- 172.21.15.83:0/2938630595 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a954aa270 msgr2=0x562a954c0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:43.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.701+0000 7feb92a00640 1 --2- 172.21.15.83:0/2938630595 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a954aa270 0x562a954c0f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:43.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.701+0000 7feb92a00640 1 -- 172.21.15.83:0/2938630595 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562a954aab90 msgr2=0x562a954cd3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:43.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.701+0000 7feb92a00640 1 --2- 172.21.15.83:0/2938630595 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562a954aab90 0x562a954cd3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:43.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.701+0000 7feb92a00640 1 -- 172.21.15.83:0/2938630595 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a954c2390 con 0x562a954b5260 2024-04-24T09:08:43.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.701+0000 7feb92a00640 1 --2- 172.21.15.83:0/2938630595 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562a954b5260 0x562a954cd8e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7feb80006010 tx=0x7feb8002a480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77fb7fca6119535a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:43.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.701+0000 7feb7fe00640 1 -- 172.21.15.83:0/2938630595 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb80035040 con 0x562a954b5260 2024-04-24T09:08:43.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.702+0000 7feb7fe00640 1 -- 172.21.15.83:0/2938630595 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feb80003d70 con 0x562a954b5260 2024-04-24T09:08:43.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.702+0000 7feb7fe00640 1 -- 172.21.15.83:0/2938630595 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb80004d40 con 0x562a954b5260 2024-04-24T09:08:43.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.702+0000 7feb9d61d740 1 -- 172.21.15.83:0/2938630595 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562a954b5260 msgr2=0x562a954cd8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:43.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.702+0000 7feb9d61d740 1 --2- 172.21.15.83:0/2938630595 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562a954b5260 0x562a954cd8e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7feb80006010 tx=0x7feb8002a480 comp rx=0 tx=0).stop 2024-04-24T09:08:43.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.703+0000 7feb9d61d740 1 -- 172.21.15.83:0/2938630595 shutdown_connections 2024-04-24T09:08:43.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.703+0000 7feb9d61d740 1 --2- 172.21.15.83:0/2938630595 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562a954b5260 0x562a954cd8e0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:43.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.703+0000 7feb9d61d740 1 --2- 172.21.15.83:0/2938630595 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562a954aab90 0x562a954cd3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:43.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.703+0000 7feb9d61d740 1 --2- 172.21.15.83:0/2938630595 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a954aa270 0x562a954c0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:43.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.703+0000 7feb9d61d740 1 -- 172.21.15.83:0/2938630595 >> 172.21.15.83:0/2938630595 conn(0x562a954a6590 msgr2=0x562a954a56f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:43.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.703+0000 7feb9d61d740 1 -- 172.21.15.83:0/2938630595 shutdown_connections 2024-04-24T09:08:43.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.703+0000 7feb9d61d740 1 -- 172.21.15.83:0/2938630595 wait complete. 2024-04-24T09:08:43.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.703+0000 7feb9d61d740 1 -- start start 2024-04-24T09:08:43.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.703+0000 7feb9d61d740 1 Processor -- start 2024-04-24T09:08:43.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.704+0000 7feb9d61d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562a954aa270 0x562a954ba420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:43.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.704+0000 7feb9d61d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a954aab90 0x562a954ba960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:43.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.704+0000 7feb9d61d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562a954b5260 0x562a954baea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:43.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.704+0000 7feb9d61d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7feb74011530 con 0x562a954aa270 2024-04-24T09:08:43.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.704+0000 7feb9d61d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7feb74010e60 con 0x562a954b5260 2024-04-24T09:08:43.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.704+0000 7feb9d61d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7feb7400e0c0 con 0x562a954aab90 2024-04-24T09:08:43.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.704+0000 7feb9d61d740 10 client.0 _ll_register_callbacks cb 0x7feb8400c6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:08:43.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.704+0000 7feb92a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562a954aa270 0x562a954ba420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:43.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.705+0000 7feb92a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562a954aa270 0x562a954ba420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54942/0 (socket says 172.21.15.83:54942) 2024-04-24T09:08:43.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.705+0000 7feb92a00640 1 -- 172.21.15.83:0/2176528944 learned_addr learned my addr 172.21.15.83:0/2176528944 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:08:43.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.705+0000 7feb93400640 1 --2- 172.21.15.83:0/2176528944 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a954aab90 0x562a954ba960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:43.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.705+0000 7feb92a00640 1 --2- 172.21.15.83:0/2176528944 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562a954b5260 0x562a954baea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:43.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.705+0000 7feb92a00640 1 -- 172.21.15.83:0/2176528944 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a954aab90 msgr2=0x562a954ba960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:43.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.705+0000 7feb92a00640 1 --2- 172.21.15.83:0/2176528944 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a954aab90 0x562a954ba960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:43.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.705+0000 7feb92a00640 1 -- 172.21.15.83:0/2176528944 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562a954b5260 msgr2=0x562a954baea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:43.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.705+0000 7feb92a00640 1 --2- 172.21.15.83:0/2176528944 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562a954b5260 0x562a954baea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:43.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.705+0000 7feb92a00640 1 -- 172.21.15.83:0/2176528944 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a9552be70 con 0x562a954aa270 2024-04-24T09:08:43.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.705+0000 7feb93400640 1 --2- 172.21.15.83:0/2176528944 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a954aab90 0x562a954ba960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:08:43.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.706+0000 7feb92a00640 1 --2- 172.21.15.83:0/2176528944 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562a954aa270 0x562a954ba420 secure :-1 s=READY pgs=2335 cs=0 l=1 rev1=1 crypto rx=0x7feb8002a9a0 tx=0x7feb80004920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9261034274b85a6a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:43.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.706+0000 7feb7d600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:08:43.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.706+0000 7feb7d600640 1 -- 172.21.15.83:0/2176528944 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb80035030 con 0x562a954aa270 2024-04-24T09:08:43.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.706+0000 7feb7d600640 1 -- 172.21.15.83:0/2176528944 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9583+0+0 (secure 0 0 0) 0x7feb80005290 con 0x562a954aa270 2024-04-24T09:08:43.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.706+0000 7feb9d61d740 1 -- 172.21.15.83:0/2176528944 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb80003d70 con 0x562a954aa270 2024-04-24T09:08:43.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.706+0000 7feb7d600640 1 -- 172.21.15.83:0/2176528944 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7feb80067050 con 0x562a954aa270 2024-04-24T09:08:43.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.707+0000 7feb9d61d740 10 client.21621 fetch_fsmap learned FSMap version 58 2024-04-24T09:08:43.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.707+0000 7feb9d61d740 1 -- 172.21.15.83:0/2176528944 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x562a9552d190 con 0x562a954aa270 2024-04-24T09:08:43.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.707+0000 7feb7d600640 1 -- 172.21.15.83:0/2176528944 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feb80004f70 con 0x562a954aa270 2024-04-24T09:08:43.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.707+0000 7feb7d600640 1 -- 172.21.15.83:0/2176528944 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 58) ==== 8374+0+0 (secure 0 0 0) 0x7feb8005b070 con 0x562a954aa270 2024-04-24T09:08:43.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.707+0000 7feb9d61d740 10 client.21621 fetch_fsmap finished waiting for FSMap version 58 2024-04-24T09:08:43.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.707+0000 7feb9d61d740 10 client.21621 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:08:43.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.707+0000 7feb9d61d740 20 client.21621 populate_metadata read hostname 'smithi083' 2024-04-24T09:08:43.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.707+0000 7feb9d61d740 1 --2- 172.21.15.83:0/2176528944 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x562a9552d8f0 0x562a9552fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:43.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.707+0000 7feb9d61d740 4 client.21621 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T09:08:43.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.707+0000 7feb9d61d740 1 -- 172.21.15.83:0/2176528944 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7feb80003d70 con 0x562a9552d8f0 2024-04-24T09:08:43.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.708+0000 7feb93400640 1 --2- 172.21.15.83:0/2176528944 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x562a9552d8f0 0x562a9552fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:43.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.709+0000 7feb93400640 1 --2- 172.21.15.83:0/2176528944 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x562a9552d8f0 0x562a9552fd10 crc :-1 s=READY pgs=434 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=455b77bc5ab71e15 server_cookie=fabd0f87399e30f in_seq=0 out_seq=0 2024-04-24T09:08:43.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.709+0000 7feb7d600640 10 client.21621 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:08:43.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.712+0000 7feb7d600640 1 -- 172.21.15.83:0/2176528944 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7feb80003d70 con 0x562a9552d8f0 2024-04-24T09:08:43.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.712+0000 7feb7d600640 10 client.21621 handle_command_reply: tid=0 2024-04-24T09:08:43.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.712+0000 7feb9d61d740 1 client.21621 shutdown 2024-04-24T09:08:43.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.712+0000 7feb9d61d740 1 -- 172.21.15.83:0/2176528944 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562a954aa270 msgr2=0x562a954ba420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:43.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.712+0000 7feb9d61d740 1 --2- 172.21.15.83:0/2176528944 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562a954aa270 0x562a954ba420 secure :-1 s=READY pgs=2335 cs=0 l=1 rev1=1 crypto rx=0x7feb8002a9a0 tx=0x7feb80004920 comp rx=0 tx=0).stop 2024-04-24T09:08:43.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.712+0000 7feb9d61d740 1 -- 172.21.15.83:0/2176528944 shutdown_connections 2024-04-24T09:08:43.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.712+0000 7feb9d61d740 1 --2- 172.21.15.83:0/2176528944 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x562a9552d8f0 0x562a9552fd10 crc :-1 s=READY pgs=434 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:43.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.712+0000 7feb9d61d740 1 --2- 172.21.15.83:0/2176528944 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562a954b5260 0x562a954baea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:43.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.712+0000 7feb9d61d740 1 --2- 172.21.15.83:0/2176528944 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a954aab90 0x562a954ba960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:43.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.712+0000 7feb9d61d740 1 --2- 172.21.15.83:0/2176528944 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562a954aa270 0x562a954ba420 unknown :-1 s=CLOSED pgs=2335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:43.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.712+0000 7feb9d61d740 1 -- 172.21.15.83:0/2176528944 >> 172.21.15.83:0/2176528944 conn(0x562a954a6590 msgr2=0x562a954a4e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:43.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.712+0000 7feb7d600640 0 client.21621 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:08:43.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.713+0000 7feb9d61d740 1 -- 172.21.15.83:0/2176528944 shutdown_connections 2024-04-24T09:08:43.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.713+0000 7feb9d61d740 1 -- 172.21.15.83:0/2176528944 wait complete. 2024-04-24T09:08:43.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.713+0000 7feb9d61d740 20 client.21621 trim_cache size 0 max 16384 2024-04-24T09:08:43.713 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T09:08:43.714 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-04-24T09:08:43.714 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "3e571ba8-ca02-4c33-8ce2-923fc7736bdc", 2024-04-24T09:08:43.714 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-04-24T09:08:43.714 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T09:08:43.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.714+0000 7feb99200640 1 -- 172.21.15.83:0/3536751163 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7feb780a3da0 msgr2=0x7feb780a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:43.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.714+0000 7feb99200640 1 --2- 172.21.15.83:0/3536751163 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7feb780a3da0 0x7feb780a6260 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7feb8c0995a0 tx=0x7feb8c005c10 comp rx=0 tx=0).stop 2024-04-24T09:08:43.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.714+0000 7feb99200640 1 -- 172.21.15.83:0/3536751163 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb9410c480 msgr2=0x7feb9419cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:43.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.714+0000 7feb99200640 1 --2- 172.21.15.83:0/3536751163 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb9410c480 0x7feb9419cf10 secure :-1 s=READY pgs=2333 cs=0 l=1 rev1=1 crypto rx=0x7feb8c00cce0 tx=0x7feb8c00a620 comp rx=0 tx=0).stop 2024-04-24T09:08:43.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.714+0000 7feb99200640 1 -- 172.21.15.83:0/3536751163 shutdown_connections 2024-04-24T09:08:43.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.714+0000 7feb99200640 1 --2- 172.21.15.83:0/3536751163 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7feb780a3da0 0x7feb780a6260 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:43.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.714+0000 7feb99200640 1 --2- 172.21.15.83:0/3536751163 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb9410c480 0x7feb9419cf10 unknown :-1 s=CLOSED pgs=2333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:43.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.714+0000 7feb99200640 1 --2- 172.21.15.83:0/3536751163 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb94103980 0x7feb94198be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:43.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.714+0000 7feb99200640 1 --2- 172.21.15.83:0/3536751163 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feb94103060 0x7feb941986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:43.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.714+0000 7feb99200640 1 -- 172.21.15.83:0/3536751163 >> 172.21.15.83:0/3536751163 conn(0x7feb940fc880 msgr2=0x7feb940fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:43.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.714+0000 7feb99200640 1 -- 172.21.15.83:0/3536751163 shutdown_connections 2024-04-24T09:08:43.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.714+0000 7feb99200640 1 -- 172.21.15.83:0/3536751163 wait complete. 2024-04-24T09:08:43.737 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T09:08:43.737 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T09:08:43.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.991+0000 7fae9f200640 1 -- 172.21.15.83:0/4035351424 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae98101770 msgr2=0x7fae98105c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:43.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.991+0000 7fae9f200640 1 --2- 172.21.15.83:0/4035351424 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae98101770 0x7fae98105c30 secure :-1 s=READY pgs=2336 cs=0 l=1 rev1=1 crypto rx=0x7fae88009900 tx=0x7fae8802a470 comp rx=0 tx=0).stop 2024-04-24T09:08:43.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.992+0000 7fae9f200640 1 -- 172.21.15.83:0/4035351424 shutdown_connections 2024-04-24T09:08:43.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.992+0000 7fae9f200640 1 --2- 172.21.15.83:0/4035351424 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fae98106250 0x7fae981066e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:43.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.992+0000 7fae9f200640 1 --2- 172.21.15.83:0/4035351424 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae98101770 0x7fae98105c30 unknown :-1 s=CLOSED pgs=2336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:43.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.992+0000 7fae9f200640 1 --2- 172.21.15.83:0/4035351424 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae98100e50 0x7fae98101230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:43.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.992+0000 7fae9f200640 1 -- 172.21.15.83:0/4035351424 >> 172.21.15.83:0/4035351424 conn(0x7fae980fc880 msgr2=0x7fae980feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:43.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.992+0000 7fae9f200640 1 -- 172.21.15.83:0/4035351424 shutdown_connections 2024-04-24T09:08:43.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.992+0000 7fae9f200640 1 -- 172.21.15.83:0/4035351424 wait complete. 2024-04-24T09:08:43.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.993+0000 7fae9f200640 1 Processor -- start 2024-04-24T09:08:43.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.993+0000 7fae9f200640 1 -- start start 2024-04-24T09:08:43.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.994+0000 7fae9f200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae98100e50 0x7fae981a48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:43.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.994+0000 7fae9f200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae98101770 0x7fae981a4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:43.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.994+0000 7fae9f200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fae98106250 0x7fae9819ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:43.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.994+0000 7fae9f200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fae98118750 con 0x7fae98101770 2024-04-24T09:08:43.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.994+0000 7fae9f200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fae981185d0 con 0x7fae98100e50 2024-04-24T09:08:43.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.994+0000 7fae9f200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fae981188d0 con 0x7fae98106250 2024-04-24T09:08:43.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.994+0000 7fae9d400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae98101770 0x7fae981a4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:43.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.994+0000 7fae9de00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae98100e50 0x7fae981a48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:43.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.994+0000 7fae9d400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae98101770 0x7fae981a4e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54968/0 (socket says 172.21.15.83:54968) 2024-04-24T09:08:43.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.994+0000 7fae9d400640 1 -- 172.21.15.83:0/3054521647 learned_addr learned my addr 172.21.15.83:0/3054521647 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:08:43.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.994+0000 7fae9de00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae98100e50 0x7fae981a48c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:53776/0 (socket says 172.21.15.83:53776) 2024-04-24T09:08:43.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.994+0000 7fae9e800640 1 --2- 172.21.15.83:0/3054521647 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fae98106250 0x7fae9819ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:43.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.995+0000 7fae9de00640 1 -- 172.21.15.83:0/3054521647 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fae98106250 msgr2=0x7fae9819ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:43.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.995+0000 7fae9de00640 1 --2- 172.21.15.83:0/3054521647 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fae98106250 0x7fae9819ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:43.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.995+0000 7fae9de00640 1 -- 172.21.15.83:0/3054521647 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae98101770 msgr2=0x7fae981a4e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:43.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.995+0000 7fae9de00640 1 --2- 172.21.15.83:0/3054521647 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae98101770 0x7fae981a4e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:43.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.995+0000 7fae9de00640 1 -- 172.21.15.83:0/3054521647 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae9819f1f0 con 0x7fae98100e50 2024-04-24T09:08:43.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.995+0000 7fae9d400640 1 --2- 172.21.15.83:0/3054521647 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae98101770 0x7fae981a4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:08:43.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.995+0000 7fae9de00640 1 --2- 172.21.15.83:0/3054521647 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae98100e50 0x7fae981a48c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fae8c00e9f0 tx=0x7fae8c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:43.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.995+0000 7fae92a00640 1 -- 172.21.15.83:0/3054521647 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae8c00ccc0 con 0x7fae98100e50 2024-04-24T09:08:43.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.995+0000 7fae9f200640 1 -- 172.21.15.83:0/3054521647 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae9819f4e0 con 0x7fae98100e50 2024-04-24T09:08:43.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.996+0000 7fae9f200640 1 -- 172.21.15.83:0/3054521647 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae981ab780 con 0x7fae98100e50 2024-04-24T09:08:43.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.996+0000 7fae92a00640 1 -- 172.21.15.83:0/3054521647 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fae8c0108e0 con 0x7fae98100e50 2024-04-24T09:08:43.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:43.996+0000 7fae92a00640 1 -- 172.21.15.83:0/3054521647 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae8c018880 con 0x7fae98100e50 2024-04-24T09:08:44.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.001+0000 7faea3583740 1 Processor -- start 2024-04-24T09:08:44.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.001+0000 7faea3583740 1 -- start start 2024-04-24T09:08:44.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.002+0000 7fae92a00640 1 -- 172.21.15.83:0/3054521647 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fae8c010430 con 0x7fae98100e50 2024-04-24T09:08:44.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.003+0000 7faea3583740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56099b326a30 0x56099b326e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:44.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.003+0000 7faea3583740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56099b327350 0x56099b3314e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:44.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.003+0000 7faea3583740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56099b331a20 0x56099b333e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:44.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.003+0000 7faea3583740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56099b28c1f0 con 0x56099b326a30 2024-04-24T09:08:44.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.003+0000 7faea3583740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56099b232350 con 0x56099b331a20 2024-04-24T09:08:44.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.003+0000 7faea3583740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56099b2881b0 con 0x56099b327350 2024-04-24T09:08:44.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.003+0000 7fae9d400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56099b331a20 0x56099b333e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:44.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.003+0000 7fae9e800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56099b326a30 0x56099b326e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:44.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.003+0000 7fae9d400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56099b331a20 0x56099b333e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:53790/0 (socket says 172.21.15.83:53790) 2024-04-24T09:08:44.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.003+0000 7fae9d400640 1 -- 172.21.15.83:0/1760064417 learned_addr learned my addr 172.21.15.83:0/1760064417 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:08:44.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.004+0000 7fae9d400640 1 -- 172.21.15.83:0/1760064417 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56099b327350 msgr2=0x56099b3314e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:44.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.004+0000 7fae9d400640 1 --2- 172.21.15.83:0/1760064417 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56099b327350 0x56099b3314e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:44.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.004+0000 7fae9d400640 1 -- 172.21.15.83:0/1760064417 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56099b326a30 msgr2=0x56099b326e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:44.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.004+0000 7fae9d400640 1 --2- 172.21.15.83:0/1760064417 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56099b326a30 0x56099b326e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:44.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.004+0000 7fae9d400640 1 -- 172.21.15.83:0/1760064417 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56099b286f50 con 0x56099b331a20 2024-04-24T09:08:44.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.004+0000 7fae9e800640 1 --2- 172.21.15.83:0/1760064417 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56099b326a30 0x56099b326e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:08:44.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.005+0000 7fae92a00640 1 --2- 172.21.15.83:0/3054521647 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fae800a3e20 0x7fae800a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:44.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.005+0000 7fae92a00640 1 -- 172.21.15.83:0/3054521647 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 9583+0+0 (secure 0 0 0) 0x7fae8c0d0ec0 con 0x7fae98100e50 2024-04-24T09:08:44.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.005+0000 7fae9d400640 1 --2- 172.21.15.83:0/1760064417 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56099b331a20 0x56099b333e10 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fae8802aa60 tx=0x7fae88009600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ce0fff01c027cb5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:44.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.006+0000 7fae9e800640 1 --2- 172.21.15.83:0/3054521647 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fae800a3e20 0x7fae800a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:44.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.006+0000 7fae87000640 1 -- 172.21.15.83:0/1760064417 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae88040070 con 0x56099b331a20 2024-04-24T09:08:44.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.006+0000 7fae87000640 1 -- 172.21.15.83:0/1760064417 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fae880047a0 con 0x56099b331a20 2024-04-24T09:08:44.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.006+0000 7fae87000640 1 -- 172.21.15.83:0/1760064417 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae88035040 con 0x56099b331a20 2024-04-24T09:08:44.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.007+0000 7faea3583740 1 -- 172.21.15.83:0/1760064417 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56099b331a20 msgr2=0x56099b333e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:44.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.007+0000 7faea3583740 1 --2- 172.21.15.83:0/1760064417 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56099b331a20 0x56099b333e10 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fae8802aa60 tx=0x7fae88009600 comp rx=0 tx=0).stop 2024-04-24T09:08:44.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.008+0000 7fae9e800640 1 --2- 172.21.15.83:0/3054521647 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fae800a3e20 0x7fae800a62e0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fae94007fd0 tx=0x7fae94007600 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:44.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.008+0000 7faea3583740 1 -- 172.21.15.83:0/1760064417 shutdown_connections 2024-04-24T09:08:44.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.008+0000 7faea3583740 1 --2- 172.21.15.83:0/1760064417 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56099b331a20 0x56099b333e10 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:44.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.008+0000 7faea3583740 1 --2- 172.21.15.83:0/1760064417 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56099b327350 0x56099b3314e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:44.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.008+0000 7faea3583740 1 --2- 172.21.15.83:0/1760064417 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56099b326a30 0x56099b326e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:44.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.008+0000 7faea3583740 1 -- 172.21.15.83:0/1760064417 >> 172.21.15.83:0/1760064417 conn(0x56099b322d50 msgr2=0x56099b323130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:44.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.008+0000 7faea3583740 1 -- 172.21.15.83:0/1760064417 shutdown_connections 2024-04-24T09:08:44.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.008+0000 7faea3583740 1 -- 172.21.15.83:0/1760064417 wait complete. 2024-04-24T09:08:44.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.009+0000 7faea3583740 1 -- start start 2024-04-24T09:08:44.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.009+0000 7faea3583740 1 Processor -- start 2024-04-24T09:08:44.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.010+0000 7faea3583740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56099b326a30 0x56099b32ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:44.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.010+0000 7faea3583740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56099b327350 0x56099b32ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:44.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.010+0000 7faea3583740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56099b331a20 0x56099b32f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:44.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.010+0000 7faea3583740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56099b3347b0 con 0x56099b326a30 2024-04-24T09:08:44.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.010+0000 7faea3583740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56099b245690 con 0x56099b327350 2024-04-24T09:08:44.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.010+0000 7faea3583740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56099b334930 con 0x56099b331a20 2024-04-24T09:08:44.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.010+0000 7faea3583740 10 client.0 _ll_register_callbacks cb 0x56099b281650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:08:44.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.011+0000 7fae9de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56099b326a30 0x56099b32ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:44.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.011+0000 7fae9de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56099b326a30 0x56099b32ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54978/0 (socket says 172.21.15.83:54978) 2024-04-24T09:08:44.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.011+0000 7fae9de00640 1 -- 172.21.15.83:0/2963715712 learned_addr learned my addr 172.21.15.83:0/2963715712 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:08:44.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.011+0000 7fae9d400640 1 --2- 172.21.15.83:0/2963715712 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56099b327350 0x56099b32ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:44.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.011+0000 7fae9d400640 1 -- 172.21.15.83:0/2963715712 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56099b331a20 msgr2=0x56099b32f490 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-24T09:08:44.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.011+0000 7fae9d400640 1 --2- 172.21.15.83:0/2963715712 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56099b331a20 0x56099b32f490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:44.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.011+0000 7fae9d400640 1 -- 172.21.15.83:0/2963715712 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56099b326a30 msgr2=0x56099b32ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:44.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.012+0000 7fae9d400640 1 --2- 172.21.15.83:0/2963715712 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56099b326a30 0x56099b32ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:44.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.012+0000 7fae9d400640 1 -- 172.21.15.83:0/2963715712 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56099b244920 con 0x56099b327350 2024-04-24T09:08:44.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.012+0000 7fae9d400640 1 --2- 172.21.15.83:0/2963715712 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56099b327350 0x56099b32ef50 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fae88005ec0 tx=0x7fae88004b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7749e41b4b17c50e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:44.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.012+0000 7fae7b400640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:08:44.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.013+0000 7fae7b400640 1 -- 172.21.15.83:0/2963715712 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae88035070 con 0x56099b327350 2024-04-24T09:08:44.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.013+0000 7fae7b400640 1 -- 172.21.15.83:0/2963715712 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9583+0+0 (secure 0 0 0) 0x7fae88003ab0 con 0x56099b327350 2024-04-24T09:08:44.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.013+0000 7faea3583740 1 -- 172.21.15.83:0/2963715712 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae880047a0 con 0x56099b327350 2024-04-24T09:08:44.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.014+0000 7fae7b400640 1 -- 172.21.15.83:0/2963715712 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7fae88040040 con 0x56099b327350 2024-04-24T09:08:44.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.014+0000 7faea3583740 10 client.38765 fetch_fsmap learned FSMap version 58 2024-04-24T09:08:44.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.014+0000 7faea3583740 1 -- 172.21.15.83:0/2963715712 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x56099b0687d0 con 0x56099b327350 2024-04-24T09:08:44.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.014+0000 7fae7b400640 1 -- 172.21.15.83:0/2963715712 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fae880047a0 con 0x56099b327350 2024-04-24T09:08:44.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.014+0000 7fae7b400640 1 -- 172.21.15.83:0/2963715712 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 58) ==== 8374+0+0 (secure 0 0 0) 0x7fae88031530 con 0x56099b327350 2024-04-24T09:08:44.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.015+0000 7faea3583740 10 client.38765 fetch_fsmap finished waiting for FSMap version 58 2024-04-24T09:08:44.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.015+0000 7faea3583740 10 client.38765 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:08:44.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.015+0000 7faea3583740 20 client.38765 populate_metadata read hostname 'smithi083' 2024-04-24T09:08:44.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.015+0000 7faea3583740 1 --2- 172.21.15.83:0/2963715712 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56099b341cc0 0x56099b3440b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:44.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.015+0000 7faea3583740 4 client.38765 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:08:44.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.015+0000 7faea3583740 1 -- 172.21.15.83:0/2963715712 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56099b299c10 con 0x56099b341cc0 2024-04-24T09:08:44.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.016+0000 7fae9de00640 1 --2- 172.21.15.83:0/2963715712 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56099b341cc0 0x56099b3440b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:44.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.016+0000 7fae9de00640 1 --2- 172.21.15.83:0/2963715712 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56099b341cc0 0x56099b3440b0 crc :-1 s=READY pgs=435 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e242c711932fbd2 server_cookie=c6e1c89386ead1a1 in_seq=0 out_seq=0 2024-04-24T09:08:44.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.016+0000 7fae7b400640 10 client.38765 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:08:44.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.017+0000 7fae7b400640 1 -- 172.21.15.83:0/2963715712 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56099b299c10 con 0x56099b341cc0 2024-04-24T09:08:44.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.017+0000 7fae7b400640 10 client.38765 handle_command_reply: tid=0 2024-04-24T09:08:44.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.018+0000 7faea3583740 1 client.38765 shutdown 2024-04-24T09:08:44.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.018+0000 7faea3583740 1 -- 172.21.15.83:0/2963715712 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56099b327350 msgr2=0x56099b32ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:44.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.018+0000 7faea3583740 1 --2- 172.21.15.83:0/2963715712 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56099b327350 0x56099b32ef50 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fae88005ec0 tx=0x7fae88004b60 comp rx=0 tx=0).stop 2024-04-24T09:08:44.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.018+0000 7faea3583740 1 -- 172.21.15.83:0/2963715712 shutdown_connections 2024-04-24T09:08:44.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.018+0000 7faea3583740 1 --2- 172.21.15.83:0/2963715712 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56099b341cc0 0x56099b3440b0 crc :-1 s=READY pgs=435 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:44.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.018+0000 7faea3583740 1 --2- 172.21.15.83:0/2963715712 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56099b331a20 0x56099b32f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:44.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.018+0000 7faea3583740 1 --2- 172.21.15.83:0/2963715712 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56099b327350 0x56099b32ef50 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:44.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.018+0000 7faea3583740 1 --2- 172.21.15.83:0/2963715712 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56099b326a30 0x56099b32ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:44.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.018+0000 7fae7b400640 0 client.38765 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:08:44.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.018+0000 7faea3583740 1 -- 172.21.15.83:0/2963715712 >> 172.21.15.83:0/2963715712 conn(0x56099b322d50 msgr2=0x56099b321fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:44.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.018+0000 7faea3583740 1 -- 172.21.15.83:0/2963715712 shutdown_connections 2024-04-24T09:08:44.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.018+0000 7faea3583740 1 -- 172.21.15.83:0/2963715712 wait complete. 2024-04-24T09:08:44.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.019+0000 7faea3583740 20 client.38765 trim_cache size 0 max 16384 2024-04-24T09:08:44.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.048+0000 7faea3583740 1 Processor -- start 2024-04-24T09:08:44.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.048+0000 7faea3583740 1 -- start start 2024-04-24T09:08:44.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.048+0000 7faea3583740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56099b326a30 0x56099b322750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:44.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.048+0000 7faea3583740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56099b327350 0x56099b3470d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:44.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.048+0000 7faea3583740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56099b331a20 0x56099b347610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:44.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.048+0000 7faea3583740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fae7c010140 con 0x56099b327350 2024-04-24T09:08:44.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.048+0000 7faea3583740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fae7c011c80 con 0x56099b331a20 2024-04-24T09:08:44.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.048+0000 7faea3583740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fae7c010810 con 0x56099b326a30 2024-04-24T09:08:44.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.049+0000 7fae9d400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56099b327350 0x56099b3470d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:44.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.049+0000 7fae9e800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56099b331a20 0x56099b347610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:44.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.049+0000 7fae9e800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56099b331a20 0x56099b347610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:53818/0 (socket says 172.21.15.83:53818) 2024-04-24T09:08:44.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.049+0000 7fae9e800640 1 -- 172.21.15.83:0/2562863558 learned_addr learned my addr 172.21.15.83:0/2562863558 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:08:44.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.049+0000 7fae9d400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56099b327350 0x56099b3470d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54994/0 (socket says 172.21.15.83:54994) 2024-04-24T09:08:44.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.049+0000 7fae9de00640 1 --2- 172.21.15.83:0/2562863558 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56099b326a30 0x56099b322750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:44.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.049+0000 7fae9d400640 1 -- 172.21.15.83:0/2562863558 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56099b326a30 msgr2=0x56099b322750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:44.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.049+0000 7fae9d400640 1 --2- 172.21.15.83:0/2562863558 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56099b326a30 0x56099b322750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:44.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.049+0000 7fae9d400640 1 -- 172.21.15.83:0/2562863558 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56099b331a20 msgr2=0x56099b347610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:44.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.049+0000 7fae9d400640 1 --2- 172.21.15.83:0/2562863558 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56099b331a20 0x56099b347610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:44.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.049+0000 7fae9d400640 1 -- 172.21.15.83:0/2562863558 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56099af9e3f0 con 0x56099b327350 2024-04-24T09:08:44.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.049+0000 7fae9e800640 1 --2- 172.21.15.83:0/2562863558 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56099b331a20 0x56099b347610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:08:44.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.050+0000 7fae9d400640 1 --2- 172.21.15.83:0/2562863558 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56099b327350 0x56099b3470d0 secure :-1 s=READY pgs=2337 cs=0 l=1 rev1=1 crypto rx=0x7fae88004650 tx=0x7fae88005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18ce50b0beee4143 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:44.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.050+0000 7fae86600640 1 -- 172.21.15.83:0/2562863558 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae8805b030 con 0x56099b327350 2024-04-24T09:08:44.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.050+0000 7fae86600640 1 -- 172.21.15.83:0/2562863558 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fae88031b20 con 0x56099b327350 2024-04-24T09:08:44.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.050+0000 7fae86600640 1 -- 172.21.15.83:0/2562863558 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae88035070 con 0x56099b327350 2024-04-24T09:08:44.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.051+0000 7faea3583740 1 -- 172.21.15.83:0/2562863558 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56099b327350 msgr2=0x56099b3470d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:44.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.051+0000 7faea3583740 1 --2- 172.21.15.83:0/2562863558 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56099b327350 0x56099b3470d0 secure :-1 s=READY pgs=2337 cs=0 l=1 rev1=1 crypto rx=0x7fae88004650 tx=0x7fae88005a20 comp rx=0 tx=0).stop 2024-04-24T09:08:44.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.051+0000 7faea3583740 1 -- 172.21.15.83:0/2562863558 shutdown_connections 2024-04-24T09:08:44.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.051+0000 7faea3583740 1 --2- 172.21.15.83:0/2562863558 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56099b331a20 0x56099b347610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:44.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.051+0000 7faea3583740 1 --2- 172.21.15.83:0/2562863558 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56099b327350 0x56099b3470d0 unknown :-1 s=CLOSED pgs=2337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:44.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.051+0000 7faea3583740 1 --2- 172.21.15.83:0/2562863558 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56099b326a30 0x56099b322750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:44.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.051+0000 7faea3583740 1 -- 172.21.15.83:0/2562863558 >> 172.21.15.83:0/2562863558 conn(0x56099b322d50 msgr2=0x56099b328a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:44.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.052+0000 7faea3583740 1 -- 172.21.15.83:0/2562863558 shutdown_connections 2024-04-24T09:08:44.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.052+0000 7faea3583740 1 -- 172.21.15.83:0/2562863558 wait complete. 2024-04-24T09:08:44.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.052+0000 7faea3583740 1 -- start start 2024-04-24T09:08:44.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.052+0000 7faea3583740 1 Processor -- start 2024-04-24T09:08:44.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.053+0000 7faea3583740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56099b326a30 0x56099b3452b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:44.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.053+0000 7faea3583740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56099b327350 0x56099b3457f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:44.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.053+0000 7faea3583740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56099b331a20 0x56099b3a6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:44.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.053+0000 7faea3583740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fae7c0115b0 con 0x56099b327350 2024-04-24T09:08:44.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.053+0000 7faea3583740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fae7c010ee0 con 0x56099b326a30 2024-04-24T09:08:44.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.053+0000 7faea3583740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fae7c00e140 con 0x56099b331a20 2024-04-24T09:08:44.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.053+0000 7faea3583740 10 client.0 _ll_register_callbacks cb 0x7fae940074a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:08:44.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.053+0000 7fae9d400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56099b327350 0x56099b3457f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:44.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.053+0000 7fae9de00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56099b326a30 0x56099b3452b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:44.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.054+0000 7fae9de00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56099b326a30 0x56099b3452b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:53830/0 (socket says 172.21.15.83:53830) 2024-04-24T09:08:44.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.054+0000 7fae9de00640 1 -- 172.21.15.83:0/323023193 learned_addr learned my addr 172.21.15.83:0/323023193 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:08:44.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.054+0000 7fae9e800640 1 --2- 172.21.15.83:0/323023193 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56099b331a20 0x56099b3a6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:44.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.054+0000 7fae9de00640 1 -- 172.21.15.83:0/323023193 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56099b331a20 msgr2=0x56099b3a6ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:44.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.054+0000 7fae9de00640 1 --2- 172.21.15.83:0/323023193 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56099b331a20 0x56099b3a6ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:44.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.054+0000 7fae9de00640 1 -- 172.21.15.83:0/323023193 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56099b327350 msgr2=0x56099b3457f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:44.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.054+0000 7fae9de00640 1 --2- 172.21.15.83:0/323023193 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56099b327350 0x56099b3457f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:44.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.054+0000 7fae9de00640 1 -- 172.21.15.83:0/323023193 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56099af5e0f0 con 0x56099b326a30 2024-04-24T09:08:44.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.054+0000 7fae9de00640 1 --2- 172.21.15.83:0/323023193 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56099b326a30 0x56099b3452b0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fae8c098e40 tx=0x7fae8c087040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=399531c9fd9b80c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:44.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.054+0000 7fae7b400640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:08:44.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.055+0000 7fae7b400640 1 -- 172.21.15.83:0/323023193 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae8c09c9e0 con 0x56099b326a30 2024-04-24T09:08:44.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.055+0000 7faea3583740 1 -- 172.21.15.83:0/323023193 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae88031b20 con 0x56099b326a30 2024-04-24T09:08:44.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.055+0000 7fae7b400640 1 -- 172.21.15.83:0/323023193 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9583+0+0 (secure 0 0 0) 0x7fae8c099050 con 0x56099b326a30 2024-04-24T09:08:44.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.055+0000 7fae7b400640 1 -- 172.21.15.83:0/323023193 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7fae8c0a0d90 con 0x56099b326a30 2024-04-24T09:08:44.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.055+0000 7faea3583740 10 client.38771 fetch_fsmap learned FSMap version 58 2024-04-24T09:08:44.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.055+0000 7faea3583740 1 -- 172.21.15.83:0/323023193 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x56099b251810 con 0x56099b326a30 2024-04-24T09:08:44.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.055+0000 7fae7b400640 1 -- 172.21.15.83:0/323023193 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fae88031b20 con 0x56099b326a30 2024-04-24T09:08:44.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.056+0000 7fae7b400640 1 -- 172.21.15.83:0/323023193 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 58) ==== 8374+0+0 (secure 0 0 0) 0x7fae8c0870b0 con 0x56099b326a30 2024-04-24T09:08:44.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.056+0000 7faea3583740 10 client.38771 fetch_fsmap finished waiting for FSMap version 58 2024-04-24T09:08:44.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.056+0000 7faea3583740 10 client.38771 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:08:44.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.056+0000 7faea3583740 20 client.38771 populate_metadata read hostname 'smithi083' 2024-04-24T09:08:44.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.056+0000 7faea3583740 1 --2- 172.21.15.83:0/323023193 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56099b3a8b00 0x56099b3aaf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:44.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.056+0000 7faea3583740 4 client.38771 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T09:08:44.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.056+0000 7faea3583740 1 -- 172.21.15.83:0/323023193 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56099b293a80 con 0x56099b3a8b00 2024-04-24T09:08:44.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.057+0000 7fae9d400640 1 --2- 172.21.15.83:0/323023193 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56099b3a8b00 0x56099b3aaf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:44.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.057+0000 7fae9d400640 1 --2- 172.21.15.83:0/323023193 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56099b3a8b00 0x56099b3aaf20 crc :-1 s=READY pgs=436 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be28cedad9a8ac8d server_cookie=f632930b0be0f0da in_seq=0 out_seq=0 2024-04-24T09:08:44.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.057+0000 7fae7b400640 10 client.38771 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:08:44.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.058+0000 7fae7b400640 1 -- 172.21.15.83:0/323023193 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56099b293a80 con 0x56099b3a8b00 2024-04-24T09:08:44.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.058+0000 7fae7b400640 10 client.38771 handle_command_reply: tid=0 2024-04-24T09:08:44.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.058+0000 7faea3583740 1 client.38771 shutdown 2024-04-24T09:08:44.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.058+0000 7faea3583740 1 -- 172.21.15.83:0/323023193 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56099b326a30 msgr2=0x56099b3452b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:44.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.058+0000 7faea3583740 1 --2- 172.21.15.83:0/323023193 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56099b326a30 0x56099b3452b0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fae8c098e40 tx=0x7fae8c087040 comp rx=0 tx=0).stop 2024-04-24T09:08:44.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.058+0000 7faea3583740 1 -- 172.21.15.83:0/323023193 shutdown_connections 2024-04-24T09:08:44.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.058+0000 7faea3583740 1 --2- 172.21.15.83:0/323023193 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56099b3a8b00 0x56099b3aaf20 crc :-1 s=READY pgs=436 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:44.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.058+0000 7faea3583740 1 --2- 172.21.15.83:0/323023193 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56099b331a20 0x56099b3a6ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:44.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.058+0000 7faea3583740 1 --2- 172.21.15.83:0/323023193 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56099b327350 0x56099b3457f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:44.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.058+0000 7faea3583740 1 --2- 172.21.15.83:0/323023193 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56099b326a30 0x56099b3452b0 secure :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fae8c098e40 tx=0x7fae8c087040 comp rx=0 tx=0).stop 2024-04-24T09:08:44.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.058+0000 7faea3583740 1 -- 172.21.15.83:0/323023193 >> 172.21.15.83:0/323023193 conn(0x56099b322d50 msgr2=0x56099b3281c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:44.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.058+0000 7fae7b400640 0 client.38771 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:08:44.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.058+0000 7faea3583740 1 -- 172.21.15.83:0/323023193 shutdown_connections 2024-04-24T09:08:44.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.059+0000 7faea3583740 1 -- 172.21.15.83:0/323023193 wait complete. 2024-04-24T09:08:44.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.059+0000 7faea3583740 20 client.38771 trim_cache size 0 max 16384 2024-04-24T09:08:44.059 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T09:08:44.060 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T09:08:44.060 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-04-24T09:08:44.060 INFO:teuthology.orchestra.run.smithi083.stdout: "3e571ba8-ca02-4c33-8ce2-923fc7736bdc": { 2024-04-24T09:08:44.060 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-04-24T09:08:44.060 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "3e571ba8-ca02-4c33-8ce2-923fc7736bdc", 2024-04-24T09:08:44.060 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-04-24T09:08:44.060 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:08:44.060 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:08:44.060 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T09:08:44.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.059+0000 7fae9f200640 1 -- 172.21.15.83:0/3054521647 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fae800a3e20 msgr2=0x7fae800a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:44.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.059+0000 7fae9f200640 1 --2- 172.21.15.83:0/3054521647 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fae800a3e20 0x7fae800a62e0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fae94007fd0 tx=0x7fae94007600 comp rx=0 tx=0).stop 2024-04-24T09:08:44.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.059+0000 7fae9f200640 1 -- 172.21.15.83:0/3054521647 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae98100e50 msgr2=0x7fae981a48c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:44.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.059+0000 7fae9f200640 1 --2- 172.21.15.83:0/3054521647 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae98100e50 0x7fae981a48c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fae8c00e9f0 tx=0x7fae8c00eeb0 comp rx=0 tx=0).stop 2024-04-24T09:08:44.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.060+0000 7fae9f200640 1 -- 172.21.15.83:0/3054521647 shutdown_connections 2024-04-24T09:08:44.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.060+0000 7fae9f200640 1 --2- 172.21.15.83:0/3054521647 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fae800a3e20 0x7fae800a62e0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:44.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.060+0000 7fae9f200640 1 --2- 172.21.15.83:0/3054521647 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fae98106250 0x7fae9819ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:44.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.060+0000 7fae9f200640 1 --2- 172.21.15.83:0/3054521647 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae98101770 0x7fae981a4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:44.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.060+0000 7fae9f200640 1 --2- 172.21.15.83:0/3054521647 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae98100e50 0x7fae981a48c0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:44.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.060+0000 7fae9f200640 1 -- 172.21.15.83:0/3054521647 >> 172.21.15.83:0/3054521647 conn(0x7fae980fc880 msgr2=0x7fae980fd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:44.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.060+0000 7fae9f200640 1 -- 172.21.15.83:0/3054521647 shutdown_connections 2024-04-24T09:08:44.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:08:44.060+0000 7fae9f200640 1 -- 172.21.15.83:0/3054521647 wait complete. 2024-04-24T09:08:44.080 INFO:tasks.cephfs.filesystem:scrub status for tag:3e571ba8-ca02-4c33-8ce2-923fc7736bdc - {'path': '/', 'tag': '3e571ba8-ca02-4c33-8ce2-923fc7736bdc', 'options': 'recursive,force'} 2024-04-24T09:08:44.167 INFO:tasks.workunit.client.0.smithi083.stdout: 10 25183 6.92 MB/sec execute 75 sec latency 106.342 ms 2024-04-24T09:08:44.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:08:44 smithi083 ceph-mon[27907]: pgmap v538: 129 pgs: 129 active+clean; 501 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.6 MiB/s wr, 321 op/s 2024-04-24T09:08:44.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:08:44 smithi083 ceph-mon[27907]: scrub queued for path: / 2024-04-24T09:08:44.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:08:44 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:08:44.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:08:44 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:08:44.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:08:44 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:08:44.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:08:44 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:08:44.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:08:44 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:08:44.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:08:44 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:08:44.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:08:44 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:08:44.555 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:08:44 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:08:44.555 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:08:44 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:08:44.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:08:44 smithi138 ceph-mon[31441]: pgmap v538: 129 pgs: 129 active+clean; 501 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.6 MiB/s wr, 321 op/s 2024-04-24T09:08:44.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:08:44 smithi138 ceph-mon[31441]: scrub queued for path: / 2024-04-24T09:08:44.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:08:44 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:08:44.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:08:44 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:08:44.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:08:44 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:08:44.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:08:44 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:08:44.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:08:44 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:08:44.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:08:44 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:08:44.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:08:44 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:08:44.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:08:44 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:08:44.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:08:44 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:08:44.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:08:44 smithi189 ceph-mon[31791]: pgmap v538: 129 pgs: 129 active+clean; 501 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.6 MiB/s wr, 321 op/s 2024-04-24T09:08:44.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:08:44 smithi189 ceph-mon[31791]: scrub queued for path: / 2024-04-24T09:08:44.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:08:44 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:08:44.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:08:44 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:08:44.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:08:44 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:08:44.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:08:44 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:08:44.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:08:44 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:08:44.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:08:44 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:08:44.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:08:44 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:08:44.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:08:44 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:08:44.709 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:08:44 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:08:45.167 INFO:tasks.workunit.client.0.smithi083.stdout: 10 25326 6.95 MB/sec execute 76 sec latency 63.493 ms 2024-04-24T09:08:45.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:08:45 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:08:45.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:08:45 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:08:45.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:08:45 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:08:45.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:08:45 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:08:45.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:08:45 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:08:45.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:08:45 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:08:45.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:08:45 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:08:45.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:08:45 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:08:45.704 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:08:45 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:08:45.704 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:08:45 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:08:45.704 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:08:45 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:08:45.704 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:08:45 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:08:45.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:08:45 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:08:45.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:08:45 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:08:45.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:08:45 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:08:45.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:08:45 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:08:45.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:08:45 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:08:45.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:08:45 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:08:45.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:08:45 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:08:45.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:08:45 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:08:45.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:08:45 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:08:45.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:08:45 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:08:45.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:08:45 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:08:45.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:08:45 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:08:46.167 INFO:tasks.workunit.client.0.smithi083.stdout: 10 25446 6.97 MB/sec execute 77 sec latency 65.720 ms 2024-04-24T09:08:46.552 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:08:46 smithi083 ceph-mon[27907]: pgmap v539: 129 pgs: 129 active+clean; 501 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.9 MiB/s wr, 225 op/s 2024-04-24T09:08:46.704 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:08:46 smithi138 ceph-mon[31441]: pgmap v539: 129 pgs: 129 active+clean; 501 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.9 MiB/s wr, 225 op/s 2024-04-24T09:08:46.706 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:08:46 smithi189 ceph-mon[31791]: pgmap v539: 129 pgs: 129 active+clean; 501 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.9 MiB/s wr, 225 op/s 2024-04-24T09:08:47.167 INFO:tasks.workunit.client.0.smithi083.stdout: 10 25563 6.97 MB/sec execute 78 sec latency 82.169 ms 2024-04-24T09:08:48.167 INFO:tasks.workunit.client.0.smithi083.stdout: 10 25713 7.06 MB/sec execute 79 sec latency 72.422 ms 2024-04-24T09:08:48.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:08:48 smithi083 ceph-mon[27907]: pgmap v540: 129 pgs: 129 active+clean; 501 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.9 MiB/s wr, 225 op/s 2024-04-24T09:08:48.704 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:08:48 smithi138 ceph-mon[31441]: pgmap v540: 129 pgs: 129 active+clean; 501 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.9 MiB/s wr, 225 op/s 2024-04-24T09:08:48.706 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:08:48 smithi189 ceph-mon[31791]: pgmap v540: 129 pgs: 129 active+clean; 501 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.9 MiB/s wr, 225 op/s 2024-04-24T09:08:49.167 INFO:tasks.workunit.client.0.smithi083.stdout: 10 25736 6.98 MB/sec execute 80 sec latency 512.228 ms 2024-04-24T09:08:50.167 INFO:tasks.workunit.client.0.smithi083.stdout: 10 25908 7.05 MB/sec execute 81 sec latency 107.164 ms 2024-04-24T09:08:50.552 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:08:50 smithi083 ceph-mon[27907]: pgmap v541: 129 pgs: 129 active+clean; 515 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.9 MiB/s wr, 370 op/s 2024-04-24T09:08:50.704 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:08:50 smithi138 ceph-mon[31441]: pgmap v541: 129 pgs: 129 active+clean; 515 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.9 MiB/s wr, 370 op/s 2024-04-24T09:08:50.706 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:08:50 smithi189 ceph-mon[31791]: pgmap v541: 129 pgs: 129 active+clean; 515 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.9 MiB/s wr, 370 op/s 2024-04-24T09:08:51.167 INFO:tasks.workunit.client.0.smithi083.stdout: 10 26043 7.06 MB/sec execute 82 sec latency 65.409 ms 2024-04-24T09:08:51.505 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.505+0000 7fa016400640 1 -- 172.21.15.83:0/2378918139 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa010112cd0 msgr2=0x7fa0101150c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:51.505 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.505+0000 7fa016400640 1 --2- 172.21.15.83:0/2378918139 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa010112cd0 0x7fa0101150c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9ffc009930 tx=0x7f9ffc02a420 comp rx=0 tx=0).stop 2024-04-24T09:08:51.506 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.505+0000 7fa016400640 1 -- 172.21.15.83:0/2378918139 shutdown_connections 2024-04-24T09:08:51.506 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.505+0000 7fa016400640 1 --2- 172.21.15.83:0/2378918139 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa010112cd0 0x7fa0101150c0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:51.506 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.505+0000 7fa016400640 1 --2- 172.21.15.83:0/2378918139 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa010076e50 0x7fa010112790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:51.506 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.505+0000 7fa016400640 1 --2- 172.21.15.83:0/2378918139 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa010076530 0x7fa010076910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:51.506 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.505+0000 7fa016400640 1 -- 172.21.15.83:0/2378918139 >> 172.21.15.83:0/2378918139 conn(0x7fa0100ffae0 msgr2=0x7fa010101f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:51.506 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.506+0000 7fa016400640 1 -- 172.21.15.83:0/2378918139 shutdown_connections 2024-04-24T09:08:51.506 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.506+0000 7fa016400640 1 -- 172.21.15.83:0/2378918139 wait complete. 2024-04-24T09:08:51.507 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.507+0000 7fa016400640 1 Processor -- start 2024-04-24T09:08:51.507 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.507+0000 7fa016400640 1 -- start start 2024-04-24T09:08:51.507 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.508+0000 7fa016400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa010076530 0x7fa0101a4920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:51.508 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.508+0000 7fa016400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa010076e50 0x7fa0101a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:51.508 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.508+0000 7fa016400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa010112cd0 0x7fa01019ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:51.508 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.508+0000 7fa016400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa0101186f0 con 0x7fa010076530 2024-04-24T09:08:51.508 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.508+0000 7fa016400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa010118570 con 0x7fa010076e50 2024-04-24T09:08:51.509 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.508+0000 7fa016400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa010118870 con 0x7fa010112cd0 2024-04-24T09:08:51.509 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.508+0000 7fa015000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa010076530 0x7fa0101a4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:51.509 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.508+0000 7fa015000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa010076530 0x7fa0101a4920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44518/0 (socket says 172.21.15.83:44518) 2024-04-24T09:08:51.509 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.508+0000 7fa015000640 1 -- 172.21.15.83:0/2826580970 learned_addr learned my addr 172.21.15.83:0/2826580970 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:08:51.509 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.508+0000 7fa015a00640 1 --2- 172.21.15.83:0/2826580970 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa010112cd0 0x7fa01019ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:51.509 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.508+0000 7fa00fe00640 1 --2- 172.21.15.83:0/2826580970 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa010076e50 0x7fa0101a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:51.510 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.509+0000 7fa015000640 1 -- 172.21.15.83:0/2826580970 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa010112cd0 msgr2=0x7fa01019ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:51.510 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.509+0000 7fa015000640 1 --2- 172.21.15.83:0/2826580970 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa010112cd0 0x7fa01019ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:51.510 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.509+0000 7fa015000640 1 -- 172.21.15.83:0/2826580970 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa010076e50 msgr2=0x7fa0101a4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:51.510 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.509+0000 7fa015000640 1 --2- 172.21.15.83:0/2826580970 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa010076e50 0x7fa0101a4e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:51.510 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.509+0000 7fa015000640 1 -- 172.21.15.83:0/2826580970 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa01019f250 con 0x7fa010076530 2024-04-24T09:08:51.510 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.509+0000 7fa015a00640 1 --2- 172.21.15.83:0/2826580970 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa010112cd0 0x7fa01019ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:08:51.510 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.509+0000 7fa015000640 1 --2- 172.21.15.83:0/2826580970 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa010076530 0x7fa0101a4920 secure :-1 s=READY pgs=2338 cs=0 l=1 rev1=1 crypto rx=0x7fa00800ebc0 tx=0x7fa00800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:51.511 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.510+0000 7fa00d600640 1 -- 172.21.15.83:0/2826580970 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa008010630 con 0x7fa010076530 2024-04-24T09:08:51.511 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.510+0000 7fa00d600640 1 -- 172.21.15.83:0/2826580970 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa008010c80 con 0x7fa010076530 2024-04-24T09:08:51.511 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.510+0000 7fa016400640 1 -- 172.21.15.83:0/2826580970 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa01019f540 con 0x7fa010076530 2024-04-24T09:08:51.511 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.510+0000 7fa016400640 1 -- 172.21.15.83:0/2826580970 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0101ab7e0 con 0x7fa010076530 2024-04-24T09:08:51.511 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.510+0000 7fa00d600640 1 -- 172.21.15.83:0/2826580970 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa008018b30 con 0x7fa010076530 2024-04-24T09:08:51.513 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.513+0000 7fa00d600640 1 -- 172.21.15.83:0/2826580970 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa008018cd0 con 0x7fa010076530 2024-04-24T09:08:51.513 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.514+0000 7fa01a822740 1 Processor -- start 2024-04-24T09:08:51.514 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.514+0000 7fa01a822740 1 -- start start 2024-04-24T09:08:51.514 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.514+0000 7fa01a822740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd50bce3c0 0x55cd50bce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:51.514 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.514+0000 7fa01a822740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd50bcece0 0x55cd50bd9060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:51.514 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.514+0000 7fa01a822740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd50bd95a0 0x55cd50bdb990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:51.514 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.514+0000 7fa01a822740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55cd50b34900 con 0x55cd50bd95a0 2024-04-24T09:08:51.515 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.514+0000 7fa01a822740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55cd50adaf20 con 0x55cd50bcece0 2024-04-24T09:08:51.515 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.514+0000 7fa01a822740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55cd50b308c0 con 0x55cd50bce3c0 2024-04-24T09:08:51.516 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.515+0000 7fa00fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd50bd95a0 0x55cd50bdb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:51.516 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.515+0000 7fa015000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd50bcece0 0x55cd50bd9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:51.516 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.515+0000 7fa00fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd50bd95a0 0x55cd50bdb990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44530/0 (socket says 172.21.15.83:44530) 2024-04-24T09:08:51.516 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.515+0000 7fa00fe00640 1 -- 172.21.15.83:0/3678733355 learned_addr learned my addr 172.21.15.83:0/3678733355 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:08:51.516 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.515+0000 7fa015a00640 1 --2- 172.21.15.83:0/3678733355 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd50bce3c0 0x55cd50bce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:51.516 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.515+0000 7fa00d600640 1 --2- 172.21.15.83:0/2826580970 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f9ff40a3d10 0x7f9ff40a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:51.516 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.515+0000 7fa00d600640 1 -- 172.21.15.83:0/2826580970 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 9583+0+0 (secure 0 0 0) 0x7fa0080d1260 con 0x7fa010076530 2024-04-24T09:08:51.516 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.515+0000 7fa015000640 1 -- 172.21.15.83:0/3678733355 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd50bce3c0 msgr2=0x55cd50bce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:51.516 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.515+0000 7fa015000640 1 --2- 172.21.15.83:0/3678733355 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd50bce3c0 0x55cd50bce7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:51.516 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.515+0000 7fa015000640 1 -- 172.21.15.83:0/3678733355 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd50bd95a0 msgr2=0x55cd50bdb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:51.516 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.515+0000 7fa015000640 1 --2- 172.21.15.83:0/3678733355 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd50bd95a0 0x55cd50bdb990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:51.516 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.515+0000 7fa015000640 1 -- 172.21.15.83:0/3678733355 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd50affe10 con 0x55cd50bcece0 2024-04-24T09:08:51.516 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.515+0000 7fa015a00640 1 --2- 172.21.15.83:0/2826580970 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f9ff40a3d10 0x7f9ff40a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:51.516 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.515+0000 7fa00fe00640 1 --2- 172.21.15.83:0/3678733355 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd50bd95a0 0x55cd50bdb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:08:51.517 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.516+0000 7fa015000640 1 --2- 172.21.15.83:0/3678733355 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd50bcece0 0x55cd50bd9060 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa0080ce530 tx=0x7fa008000f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6568b09b62a40e11 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:51.517 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.516+0000 7fa006000640 1 -- 172.21.15.83:0/3678733355 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa00809cb50 con 0x55cd50bcece0 2024-04-24T09:08:51.517 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.516+0000 7fa006000640 1 -- 172.21.15.83:0/3678733355 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa00809d480 con 0x55cd50bcece0 2024-04-24T09:08:51.517 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.516+0000 7fa006000640 1 -- 172.21.15.83:0/3678733355 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa00809c6e0 con 0x55cd50bcece0 2024-04-24T09:08:51.518 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.517+0000 7fa01a822740 1 -- 172.21.15.83:0/3678733355 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd50bcece0 msgr2=0x55cd50bd9060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:51.518 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.517+0000 7fa01a822740 1 --2- 172.21.15.83:0/3678733355 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd50bcece0 0x55cd50bd9060 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa0080ce530 tx=0x7fa008000f40 comp rx=0 tx=0).stop 2024-04-24T09:08:51.518 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.517+0000 7fa015a00640 1 --2- 172.21.15.83:0/2826580970 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f9ff40a3d10 0x7f9ff40a61d0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f9ffc009230 tx=0x7f9ffc009260 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:51.518 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.518+0000 7fa01a822740 1 -- 172.21.15.83:0/3678733355 shutdown_connections 2024-04-24T09:08:51.518 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.518+0000 7fa01a822740 1 --2- 172.21.15.83:0/3678733355 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd50bd95a0 0x55cd50bdb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:51.519 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.518+0000 7fa01a822740 1 --2- 172.21.15.83:0/3678733355 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd50bcece0 0x55cd50bd9060 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:51.519 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.518+0000 7fa01a822740 1 --2- 172.21.15.83:0/3678733355 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd50bce3c0 0x55cd50bce7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:51.519 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.518+0000 7fa01a822740 1 -- 172.21.15.83:0/3678733355 >> 172.21.15.83:0/3678733355 conn(0x55cd50bca8d0 msgr2=0x55cd50bcacb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:51.519 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.518+0000 7fa01a822740 1 -- 172.21.15.83:0/3678733355 shutdown_connections 2024-04-24T09:08:51.519 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.518+0000 7fa01a822740 1 -- 172.21.15.83:0/3678733355 wait complete. 2024-04-24T09:08:51.519 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.518+0000 7fa01a822740 1 -- start start 2024-04-24T09:08:51.519 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.519+0000 7fa01a822740 1 Processor -- start 2024-04-24T09:08:51.520 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.519+0000 7fa01a822740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd50bce3c0 0x55cd50bd65a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:51.520 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.520+0000 7fa01a822740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd50bcece0 0x55cd50bd6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:51.520 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.520+0000 7fa01a822740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd50bd95a0 0x55cd50bd7020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:51.520 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.520+0000 7fa01a822740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55cd50bdc0e0 con 0x55cd50bcece0 2024-04-24T09:08:51.520 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.520+0000 7fa01a822740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55cd50aee2c0 con 0x55cd50bce3c0 2024-04-24T09:08:51.520 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.520+0000 7fa01a822740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55cd50bdc260 con 0x55cd50bd95a0 2024-04-24T09:08:51.520 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.520+0000 7fa01a822740 10 client.0 _ll_register_callbacks cb 0x55cd50846a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:08:51.520 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.520+0000 7fa00fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd50bcece0 0x55cd50bd6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:51.520 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.520+0000 7fa015000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd50bce3c0 0x55cd50bd65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:51.521 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.520+0000 7fa00fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd50bcece0 0x55cd50bd6ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44534/0 (socket says 172.21.15.83:44534) 2024-04-24T09:08:51.521 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.520+0000 7fa00fe00640 1 -- 172.21.15.83:0/3094005575 learned_addr learned my addr 172.21.15.83:0/3094005575 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:08:51.521 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.520+0000 7fa015a00640 1 --2- 172.21.15.83:0/3094005575 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd50bd95a0 0x55cd50bd7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:51.521 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.521+0000 7fa00fe00640 1 -- 172.21.15.83:0/3094005575 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd50bd95a0 msgr2=0x55cd50bd7020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:51.521 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.521+0000 7fa00fe00640 1 --2- 172.21.15.83:0/3094005575 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd50bd95a0 0x55cd50bd7020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:51.521 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.521+0000 7fa00fe00640 1 -- 172.21.15.83:0/3094005575 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd50bce3c0 msgr2=0x55cd50bd65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:51.522 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.521+0000 7fa00fe00640 1 --2- 172.21.15.83:0/3094005575 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd50bce3c0 0x55cd50bd65a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:51.522 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.521+0000 7fa00fe00640 1 -- 172.21.15.83:0/3094005575 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd50afefd0 con 0x55cd50bcece0 2024-04-24T09:08:51.522 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.521+0000 7fa00fe00640 1 --2- 172.21.15.83:0/3094005575 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd50bcece0 0x55cd50bd6ae0 secure :-1 s=READY pgs=2339 cs=0 l=1 rev1=1 crypto rx=0x7fa00000b9f0 tx=0x7fa00000beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51cef5030d34b661 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:51.522 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.521+0000 7f9ffaa00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:08:51.522 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.522+0000 7f9ffaa00640 1 -- 172.21.15.83:0/3094005575 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa000003d60 con 0x55cd50bcece0 2024-04-24T09:08:51.523 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.522+0000 7f9ffaa00640 1 -- 172.21.15.83:0/3094005575 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9583+0+0 (secure 0 0 0) 0x7fa000005020 con 0x55cd50bcece0 2024-04-24T09:08:51.523 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.522+0000 7fa01a822740 1 -- 172.21.15.83:0/3094005575 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa00809d480 con 0x55cd50bcece0 2024-04-24T09:08:51.523 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.522+0000 7f9ffaa00640 1 -- 172.21.15.83:0/3094005575 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7fa000005220 con 0x55cd50bcece0 2024-04-24T09:08:51.523 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.522+0000 7fa01a822740 10 client.21651 fetch_fsmap learned FSMap version 58 2024-04-24T09:08:51.523 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.522+0000 7fa01a822740 1 -- 172.21.15.83:0/3094005575 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55cd50847210 con 0x55cd50bcece0 2024-04-24T09:08:51.524 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.523+0000 7f9ffaa00640 1 -- 172.21.15.83:0/3094005575 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa00809d480 con 0x55cd50bcece0 2024-04-24T09:08:51.524 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.523+0000 7f9ffaa00640 1 -- 172.21.15.83:0/3094005575 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 58) ==== 8374+0+0 (secure 0 0 0) 0x7fa00001b580 con 0x55cd50bcece0 2024-04-24T09:08:51.524 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.523+0000 7fa01a822740 10 client.21651 fetch_fsmap finished waiting for FSMap version 58 2024-04-24T09:08:51.524 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.523+0000 7fa01a822740 10 client.21651 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:08:51.524 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.523+0000 7fa01a822740 20 client.21651 populate_metadata read hostname 'smithi083' 2024-04-24T09:08:51.524 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.523+0000 7fa01a822740 1 --2- 172.21.15.83:0/3094005575 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55cd50be97b0 0x55cd50bebbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:51.524 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.523+0000 7fa01a822740 4 client.21651 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:08:51.524 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.523+0000 7fa01a822740 1 -- 172.21.15.83:0/3094005575 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cd50b42320 con 0x55cd50be97b0 2024-04-24T09:08:51.524 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.524+0000 7fa015000640 1 --2- 172.21.15.83:0/3094005575 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55cd50be97b0 0x55cd50bebbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:51.525 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.524+0000 7fa015000640 1 --2- 172.21.15.83:0/3094005575 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55cd50be97b0 0x55cd50bebbe0 crc :-1 s=READY pgs=437 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36d011a4512d0293 server_cookie=7680760a7d8b4055 in_seq=0 out_seq=0 2024-04-24T09:08:51.525 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.524+0000 7f9ffaa00640 10 client.21651 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:08:51.526 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.526+0000 7f9ffaa00640 1 -- 172.21.15.83:0/3094005575 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cd50b42320 con 0x55cd50be97b0 2024-04-24T09:08:51.527 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.526+0000 7f9ffaa00640 10 client.21651 handle_command_reply: tid=0 2024-04-24T09:08:51.527 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.526+0000 7fa01a822740 1 client.21651 shutdown 2024-04-24T09:08:51.527 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.527+0000 7fa01a822740 1 -- 172.21.15.83:0/3094005575 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd50bcece0 msgr2=0x55cd50bd6ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:51.527 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.527+0000 7fa01a822740 1 --2- 172.21.15.83:0/3094005575 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd50bcece0 0x55cd50bd6ae0 secure :-1 s=READY pgs=2339 cs=0 l=1 rev1=1 crypto rx=0x7fa00000b9f0 tx=0x7fa00000beb0 comp rx=0 tx=0).stop 2024-04-24T09:08:51.527 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.527+0000 7fa01a822740 1 -- 172.21.15.83:0/3094005575 shutdown_connections 2024-04-24T09:08:51.527 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.527+0000 7fa01a822740 1 --2- 172.21.15.83:0/3094005575 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55cd50be97b0 0x55cd50bebbe0 crc :-1 s=READY pgs=437 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:51.527 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.527+0000 7fa01a822740 1 --2- 172.21.15.83:0/3094005575 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd50bd95a0 0x55cd50bd7020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:51.527 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.527+0000 7fa01a822740 1 --2- 172.21.15.83:0/3094005575 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd50bcece0 0x55cd50bd6ae0 unknown :-1 s=CLOSED pgs=2339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:51.527 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.527+0000 7fa01a822740 1 --2- 172.21.15.83:0/3094005575 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd50bce3c0 0x55cd50bd65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:51.527 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.527+0000 7f9ffaa00640 0 client.21651 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:08:51.527 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.527+0000 7fa01a822740 1 -- 172.21.15.83:0/3094005575 >> 172.21.15.83:0/3094005575 conn(0x55cd50bca8d0 msgr2=0x55cd50bc9a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:51.528 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.527+0000 7fa01a822740 1 -- 172.21.15.83:0/3094005575 shutdown_connections 2024-04-24T09:08:51.528 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.527+0000 7fa01a822740 1 -- 172.21.15.83:0/3094005575 wait complete. 2024-04-24T09:08:51.528 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.527+0000 7fa01a822740 20 client.21651 trim_cache size 0 max 16384 2024-04-24T09:08:51.552 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.552+0000 7fa01a822740 1 Processor -- start 2024-04-24T09:08:51.552 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.552+0000 7fa01a822740 1 -- start start 2024-04-24T09:08:51.553 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.553+0000 7fa01a822740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd50bce3c0 0x55cd50bd8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:51.553 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.553+0000 7fa01a822740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd50bd95a0 0x55cd50be5110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:51.553 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.553+0000 7fa01a822740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd50bf14f0 0x55cd50bf1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:51.553 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.553+0000 7fa01a822740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff00101d0 con 0x55cd50bce3c0 2024-04-24T09:08:51.553 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.553+0000 7fa01a822740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff0011d10 con 0x55cd50bf14f0 2024-04-24T09:08:51.553 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.553+0000 7fa01a822740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff00108a0 con 0x55cd50bd95a0 2024-04-24T09:08:51.553 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.553+0000 7fa015000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd50bce3c0 0x55cd50bd8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:51.554 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.553+0000 7fa015a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd50bf14f0 0x55cd50bf1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:51.554 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.553+0000 7fa015a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd50bf14f0 0x55cd50bf1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:37550/0 (socket says 172.21.15.83:37550) 2024-04-24T09:08:51.554 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.553+0000 7fa015000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd50bce3c0 0x55cd50bd8cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44538/0 (socket says 172.21.15.83:44538) 2024-04-24T09:08:51.554 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.553+0000 7fa015a00640 1 -- 172.21.15.83:0/1750308311 learned_addr learned my addr 172.21.15.83:0/1750308311 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:08:51.555 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.554+0000 7fa00fe00640 1 --2- 172.21.15.83:0/1750308311 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd50bd95a0 0x55cd50be5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:51.555 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.554+0000 7fa015a00640 1 -- 172.21.15.83:0/1750308311 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd50bd95a0 msgr2=0x55cd50be5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:51.555 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.554+0000 7fa015a00640 1 --2- 172.21.15.83:0/1750308311 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd50bd95a0 0x55cd50be5110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:51.555 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.554+0000 7fa015a00640 1 -- 172.21.15.83:0/1750308311 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd50bce3c0 msgr2=0x55cd50bd8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:51.555 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.554+0000 7fa015a00640 1 --2- 172.21.15.83:0/1750308311 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd50bce3c0 0x55cd50bd8cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:51.555 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.554+0000 7fa015a00640 1 -- 172.21.15.83:0/1750308311 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd50bceb10 con 0x55cd50bf14f0 2024-04-24T09:08:51.555 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.555+0000 7fa015a00640 1 --2- 172.21.15.83:0/1750308311 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd50bf14f0 0x55cd50bf1950 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9ffc009930 tx=0x7f9ffc03e000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ce8bbbb3358ff12 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:51.555 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.555+0000 7fa005600640 1 -- 172.21.15.83:0/1750308311 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ffc035040 con 0x55cd50bf14f0 2024-04-24T09:08:51.556 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.556+0000 7fa005600640 1 -- 172.21.15.83:0/1750308311 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9ffc03db00 con 0x55cd50bf14f0 2024-04-24T09:08:51.556 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.556+0000 7fa005600640 1 -- 172.21.15.83:0/1750308311 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ffc04aa30 con 0x55cd50bf14f0 2024-04-24T09:08:51.557 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.557+0000 7fa01a822740 1 -- 172.21.15.83:0/1750308311 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd50bf14f0 msgr2=0x55cd50bf1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:51.557 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.557+0000 7fa01a822740 1 --2- 172.21.15.83:0/1750308311 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd50bf14f0 0x55cd50bf1950 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9ffc009930 tx=0x7f9ffc03e000 comp rx=0 tx=0).stop 2024-04-24T09:08:51.557 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.557+0000 7fa01a822740 1 -- 172.21.15.83:0/1750308311 shutdown_connections 2024-04-24T09:08:51.557 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.557+0000 7fa01a822740 1 --2- 172.21.15.83:0/1750308311 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd50bf14f0 0x55cd50bf1950 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:51.558 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.557+0000 7fa01a822740 1 --2- 172.21.15.83:0/1750308311 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd50bd95a0 0x55cd50be5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:51.558 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.557+0000 7fa01a822740 1 --2- 172.21.15.83:0/1750308311 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd50bce3c0 0x55cd50bd8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:51.558 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.557+0000 7fa01a822740 1 -- 172.21.15.83:0/1750308311 >> 172.21.15.83:0/1750308311 conn(0x55cd50bca8d0 msgr2=0x55cd50bcacb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:51.558 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.557+0000 7fa01a822740 1 -- 172.21.15.83:0/1750308311 shutdown_connections 2024-04-24T09:08:51.558 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.557+0000 7fa01a822740 1 -- 172.21.15.83:0/1750308311 wait complete. 2024-04-24T09:08:51.558 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.558+0000 7fa01a822740 1 -- start start 2024-04-24T09:08:51.558 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.558+0000 7fa01a822740 1 Processor -- start 2024-04-24T09:08:51.559 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.559+0000 7fa01a822740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd50bce3c0 0x55cd50be1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:51.559 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.559+0000 7fa01a822740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd50bd95a0 0x55cd50bddc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:51.559 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.559+0000 7fa01a822740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd50bf14f0 0x55cd50bde140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:51.559 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.559+0000 7fa01a822740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff0011640 con 0x55cd50bf14f0 2024-04-24T09:08:51.560 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.559+0000 7fa01a822740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff0010f70 con 0x55cd50bce3c0 2024-04-24T09:08:51.560 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.559+0000 7fa01a822740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff000e1d0 con 0x55cd50bd95a0 2024-04-24T09:08:51.560 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.559+0000 7fa01a822740 10 client.0 _ll_register_callbacks cb 0x7fa000004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:08:51.560 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.559+0000 7fa015000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd50bce3c0 0x55cd50be1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:51.560 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.559+0000 7fa015000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd50bce3c0 0x55cd50be1830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:37562/0 (socket says 172.21.15.83:37562) 2024-04-24T09:08:51.560 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.559+0000 7fa015000640 1 -- 172.21.15.83:0/14996867 learned_addr learned my addr 172.21.15.83:0/14996867 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:08:51.560 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.559+0000 7fa015a00640 1 --2- 172.21.15.83:0/14996867 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd50bf14f0 0x55cd50bde140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:51.560 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.559+0000 7fa00fe00640 1 --2- 172.21.15.83:0/14996867 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd50bd95a0 0x55cd50bddc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:51.561 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.560+0000 7fa015000640 1 -- 172.21.15.83:0/14996867 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd50bd95a0 msgr2=0x55cd50bddc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:51.561 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.560+0000 7fa015000640 1 --2- 172.21.15.83:0/14996867 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd50bd95a0 0x55cd50bddc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:51.561 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.560+0000 7fa015000640 1 -- 172.21.15.83:0/14996867 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd50bf14f0 msgr2=0x55cd50bde140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:08:51.561 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.560+0000 7fa015000640 1 --2- 172.21.15.83:0/14996867 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd50bf14f0 0x55cd50bde140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:51.561 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.560+0000 7fa015000640 1 -- 172.21.15.83:0/14996867 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd50913620 con 0x55cd50bce3c0 2024-04-24T09:08:51.561 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.560+0000 7fa015a00640 1 --2- 172.21.15.83:0/14996867 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd50bf14f0 0x55cd50bde140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:08:51.561 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.560+0000 7fa015000640 1 --2- 172.21.15.83:0/14996867 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd50bce3c0 0x55cd50be1830 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa0080a54c0 tx=0x7fa00809c6e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11762b988477f725 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:08:51.561 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.561+0000 7f9ffaa00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:08:51.561 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.561+0000 7f9ffaa00640 1 -- 172.21.15.83:0/14996867 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa00809c8c0 con 0x55cd50bce3c0 2024-04-24T09:08:51.561 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.561+0000 7f9ffaa00640 1 -- 172.21.15.83:0/14996867 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9583+0+0 (secure 0 0 0) 0x7fa00809b080 con 0x55cd50bce3c0 2024-04-24T09:08:51.561 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.561+0000 7fa01a822740 1 -- 172.21.15.83:0/14996867 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ffc03db00 con 0x55cd50bce3c0 2024-04-24T09:08:51.562 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.561+0000 7f9ffaa00640 1 -- 172.21.15.83:0/14996867 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7fa00809cd80 con 0x55cd50bce3c0 2024-04-24T09:08:51.562 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.561+0000 7fa01a822740 10 client.38789 fetch_fsmap learned FSMap version 58 2024-04-24T09:08:51.563 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.561+0000 7fa01a822740 1 -- 172.21.15.83:0/14996867 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55cd50c50a60 con 0x55cd50bce3c0 2024-04-24T09:08:51.563 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.561+0000 7f9ffaa00640 1 -- 172.21.15.83:0/14996867 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9ffc03db00 con 0x55cd50bce3c0 2024-04-24T09:08:51.563 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.562+0000 7f9ffaa00640 1 -- 172.21.15.83:0/14996867 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 58) ==== 8374+0+0 (secure 0 0 0) 0x7fa0080b6080 con 0x55cd50bce3c0 2024-04-24T09:08:51.563 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.562+0000 7fa01a822740 10 client.38789 fetch_fsmap finished waiting for FSMap version 58 2024-04-24T09:08:51.563 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.562+0000 7fa01a822740 10 client.38789 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:08:51.563 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.562+0000 7fa01a822740 20 client.38789 populate_metadata read hostname 'smithi083' 2024-04-24T09:08:51.563 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.562+0000 7fa01a822740 1 --2- 172.21.15.83:0/14996867 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55cd50c50d50 0x55cd50c53140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:08:51.563 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.562+0000 7fa01a822740 4 client.38789 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T09:08:51.563 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.562+0000 7fa01a822740 1 -- 172.21.15.83:0/14996867 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cd50b3c190 con 0x55cd50c50d50 2024-04-24T09:08:51.563 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.562+0000 7fa00fe00640 1 --2- 172.21.15.83:0/14996867 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55cd50c50d50 0x55cd50c53140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:08:51.563 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.563+0000 7fa00fe00640 1 --2- 172.21.15.83:0/14996867 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55cd50c50d50 0x55cd50c53140 crc :-1 s=READY pgs=438 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21778507d59ff04b server_cookie=4d3e835fd14d6f95 in_seq=0 out_seq=0 2024-04-24T09:08:51.564 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.563+0000 7f9ffaa00640 10 client.38789 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:08:51.783 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.783+0000 7f9ffaa00640 1 -- 172.21.15.83:0/14996867 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55cd50b3c190 con 0x55cd50c50d50 2024-04-24T09:08:51.783 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.783+0000 7f9ffaa00640 10 client.38789 handle_command_reply: tid=0 2024-04-24T09:08:51.784 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.783+0000 7fa01a822740 1 client.38789 shutdown 2024-04-24T09:08:51.784 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.783+0000 7fa01a822740 1 -- 172.21.15.83:0/14996867 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd50bce3c0 msgr2=0x55cd50be1830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:51.784 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.783+0000 7fa01a822740 1 --2- 172.21.15.83:0/14996867 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd50bce3c0 0x55cd50be1830 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa0080a54c0 tx=0x7fa00809c6e0 comp rx=0 tx=0).stop 2024-04-24T09:08:51.784 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.783+0000 7fa01a822740 1 -- 172.21.15.83:0/14996867 shutdown_connections 2024-04-24T09:08:51.784 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.783+0000 7fa01a822740 1 --2- 172.21.15.83:0/14996867 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55cd50c50d50 0x55cd50c53140 crc :-1 s=READY pgs=438 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:51.784 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.783+0000 7fa01a822740 1 --2- 172.21.15.83:0/14996867 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd50bf14f0 0x55cd50bde140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:51.784 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.783+0000 7fa01a822740 1 --2- 172.21.15.83:0/14996867 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cd50bd95a0 0x55cd50bddc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:51.784 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.783+0000 7fa01a822740 1 --2- 172.21.15.83:0/14996867 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd50bce3c0 0x55cd50be1830 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:51.784 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.783+0000 7f9ffaa00640 0 client.38789 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:08:51.784 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.783+0000 7fa01a822740 1 -- 172.21.15.83:0/14996867 >> 172.21.15.83:0/14996867 conn(0x55cd50bca8d0 msgr2=0x55cd50bc9200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:51.785 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.784+0000 7fa01a822740 1 -- 172.21.15.83:0/14996867 shutdown_connections 2024-04-24T09:08:51.785 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.784+0000 7fa01a822740 1 -- 172.21.15.83:0/14996867 wait complete. 2024-04-24T09:08:51.785 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.784+0000 7fa01a822740 20 client.38789 trim_cache size 0 max 16384 2024-04-24T09:08:51.785 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T09:08:51.785 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T09:08:51.785 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T09:08:51.785 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T09:08:51.786 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.785+0000 7fa016400640 1 -- 172.21.15.83:0/2826580970 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f9ff40a3d10 msgr2=0x7f9ff40a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:51.786 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.785+0000 7fa016400640 1 --2- 172.21.15.83:0/2826580970 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f9ff40a3d10 0x7f9ff40a61d0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f9ffc009230 tx=0x7f9ffc009260 comp rx=0 tx=0).stop 2024-04-24T09:08:51.786 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.785+0000 7fa016400640 1 -- 172.21.15.83:0/2826580970 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa010076530 msgr2=0x7fa0101a4920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:08:51.786 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.785+0000 7fa016400640 1 --2- 172.21.15.83:0/2826580970 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa010076530 0x7fa0101a4920 secure :-1 s=READY pgs=2338 cs=0 l=1 rev1=1 crypto rx=0x7fa00800ebc0 tx=0x7fa00800c6a0 comp rx=0 tx=0).stop 2024-04-24T09:08:51.786 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.785+0000 7fa016400640 1 -- 172.21.15.83:0/2826580970 shutdown_connections 2024-04-24T09:08:51.787 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.785+0000 7fa016400640 1 --2- 172.21.15.83:0/2826580970 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f9ff40a3d10 0x7f9ff40a61d0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:51.787 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.785+0000 7fa016400640 1 --2- 172.21.15.83:0/2826580970 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa010112cd0 0x7fa01019ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:51.787 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.785+0000 7fa016400640 1 --2- 172.21.15.83:0/2826580970 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa010076e50 0x7fa0101a4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:51.787 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.785+0000 7fa016400640 1 --2- 172.21.15.83:0/2826580970 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa010076530 0x7fa0101a4920 unknown :-1 s=CLOSED pgs=2338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:08:51.787 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.785+0000 7fa016400640 1 -- 172.21.15.83:0/2826580970 >> 172.21.15.83:0/2826580970 conn(0x7fa0100ffae0 msgr2=0x7fa010113940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:08:51.787 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.785+0000 7fa016400640 1 -- 172.21.15.83:0/2826580970 shutdown_connections 2024-04-24T09:08:51.787 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:08:51.786+0000 7fa016400640 1 -- 172.21.15.83:0/2826580970 wait complete. 2024-04-24T09:08:52.168 INFO:tasks.workunit.client.0.smithi083.stdout: 10 26129 7.03 MB/sec execute 83 sec latency 161.168 ms 2024-04-24T09:08:52.552 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:08:52 smithi083 ceph-mon[27907]: pgmap v542: 129 pgs: 129 active+clean; 515 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.5 MiB/s wr, 260 op/s 2024-04-24T09:08:52.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:08:52 smithi138 ceph-mon[31441]: pgmap v542: 129 pgs: 129 active+clean; 515 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.5 MiB/s wr, 260 op/s 2024-04-24T09:08:52.706 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:08:52 smithi189 ceph-mon[31791]: pgmap v542: 129 pgs: 129 active+clean; 515 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.5 MiB/s wr, 260 op/s 2024-04-24T09:08:53.168 INFO:tasks.workunit.client.0.smithi083.stdout: 10 26231 7.02 MB/sec execute 84 sec latency 93.230 ms 2024-04-24T09:08:54.168 INFO:tasks.workunit.client.0.smithi083.stdout: 10 26275 6.96 MB/sec execute 85 sec latency 565.115 ms 2024-04-24T09:08:54.552 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:08:54 smithi083 ceph-mon[27907]: pgmap v543: 129 pgs: 129 active+clean; 527 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.2 MiB/s wr, 372 op/s 2024-04-24T09:08:54.704 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:08:54 smithi138 ceph-mon[31441]: pgmap v543: 129 pgs: 129 active+clean; 527 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.2 MiB/s wr, 372 op/s 2024-04-24T09:08:54.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:08:54 smithi189 ceph-mon[31791]: pgmap v543: 129 pgs: 129 active+clean; 527 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.2 MiB/s wr, 372 op/s 2024-04-24T09:08:55.168 INFO:tasks.workunit.client.0.smithi083.stdout: 10 26405 6.98 MB/sec execute 86 sec latency 104.147 ms 2024-04-24T09:08:56.168 INFO:tasks.workunit.client.0.smithi083.stdout: 10 26519 6.95 MB/sec execute 87 sec latency 81.393 ms 2024-04-24T09:08:56.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:08:56 smithi083 ceph-mon[27907]: pgmap v544: 129 pgs: 129 active+clean; 527 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.7 MiB/s wr, 256 op/s 2024-04-24T09:08:56.704 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:08:56 smithi138 ceph-mon[31441]: pgmap v544: 129 pgs: 129 active+clean; 527 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.7 MiB/s wr, 256 op/s 2024-04-24T09:08:56.706 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:08:56 smithi189 ceph-mon[31791]: pgmap v544: 129 pgs: 129 active+clean; 527 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.7 MiB/s wr, 256 op/s 2024-04-24T09:08:57.168 INFO:tasks.workunit.client.0.smithi083.stdout: 10 26639 6.92 MB/sec execute 88 sec latency 68.224 ms 2024-04-24T09:08:57.552 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:08:57 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:08:57.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:08:57 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:08:57.704 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:08:57 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:08:57.704 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:08:57 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:08:57.706 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:08:57 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:08:57.706 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:08:57 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:08:58.168 INFO:tasks.workunit.client.0.smithi083.stdout: 10 26766 6.93 MB/sec execute 89 sec latency 85.666 ms 2024-04-24T09:08:58.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:08:58 smithi083 ceph-mon[27907]: pgmap v545: 129 pgs: 129 active+clean; 527 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.7 MiB/s wr, 256 op/s 2024-04-24T09:08:58.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:08:58 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:08:58.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:08:58 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:08:58.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:08:58 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:08:58.704 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:08:58 smithi138 ceph-mon[31441]: pgmap v545: 129 pgs: 129 active+clean; 527 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.7 MiB/s wr, 256 op/s 2024-04-24T09:08:58.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:08:58 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:08:58.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:08:58 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:08:58.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:08:58 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:08:58.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:08:58 smithi189 ceph-mon[31791]: pgmap v545: 129 pgs: 129 active+clean; 527 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.7 MiB/s wr, 256 op/s 2024-04-24T09:08:58.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:08:58 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:08:58.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:08:58 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:08:58.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:08:58 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:08:59.168 INFO:tasks.workunit.client.0.smithi083.stdout: 10 26884 6.90 MB/sec execute 90 sec latency 76.179 ms 2024-04-24T09:09:00.169 INFO:tasks.workunit.client.0.smithi083.stdout: 10 27012 6.88 MB/sec execute 91 sec latency 98.089 ms 2024-04-24T09:09:00.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:09:00 smithi083 ceph-mon[27907]: pgmap v546: 129 pgs: 129 active+clean; 527 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.4 MiB/s wr, 352 op/s 2024-04-24T09:09:00.706 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:09:00 smithi138 ceph-mon[31441]: pgmap v546: 129 pgs: 129 active+clean; 527 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.4 MiB/s wr, 352 op/s 2024-04-24T09:09:00.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:09:00 smithi189 ceph-mon[31791]: pgmap v546: 129 pgs: 129 active+clean; 527 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.4 MiB/s wr, 352 op/s 2024-04-24T09:09:01.168 INFO:tasks.workunit.client.0.smithi083.stdout: 10 27115 6.82 MB/sec execute 92 sec latency 179.058 ms 2024-04-24T09:09:02.168 INFO:tasks.workunit.client.0.smithi083.stdout: 10 27263 6.82 MB/sec execute 93 sec latency 78.082 ms 2024-04-24T09:09:02.609 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:09:02 smithi189 ceph-mon[31791]: pgmap v547: 129 pgs: 129 active+clean; 527 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.4 MiB/s wr, 208 op/s 2024-04-24T09:09:02.704 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:09:02 smithi138 ceph-mon[31441]: pgmap v547: 129 pgs: 129 active+clean; 527 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.4 MiB/s wr, 208 op/s 2024-04-24T09:09:02.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:09:02 smithi083 ceph-mon[27907]: pgmap v547: 129 pgs: 129 active+clean; 527 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.4 MiB/s wr, 208 op/s 2024-04-24T09:09:03.168 INFO:tasks.workunit.client.0.smithi083.stdout: 10 27378 6.77 MB/sec execute 94 sec latency 120.174 ms 2024-04-24T09:09:03.704 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:09:03 smithi138 ceph-mon[31441]: osdmap e117: 12 total, 12 up, 12 in 2024-04-24T09:09:03.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:09:03 smithi189 ceph-mon[31791]: osdmap e117: 12 total, 12 up, 12 in 2024-04-24T09:09:03.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:09:03 smithi083 ceph-mon[27907]: osdmap e117: 12 total, 12 up, 12 in 2024-04-24T09:09:04.168 INFO:tasks.workunit.client.0.smithi083.stdout: 10 27529 6.77 MB/sec execute 95 sec latency 64.020 ms 2024-04-24T09:09:04.704 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:09:04 smithi138 ceph-mon[31441]: pgmap v549: 129 pgs: 129 active+clean; 525 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.5 MiB/s wr, 272 op/s 2024-04-24T09:09:04.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:09:04 smithi189 ceph-mon[31791]: pgmap v549: 129 pgs: 129 active+clean; 525 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.5 MiB/s wr, 272 op/s 2024-04-24T09:09:04.802 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:09:04 smithi083 ceph-mon[27907]: pgmap v549: 129 pgs: 129 active+clean; 525 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.5 MiB/s wr, 272 op/s 2024-04-24T09:09:04.967 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.967+0000 7f8a87400640 1 -- 172.21.15.83:0/1181524692 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a8010e690 msgr2=0x7f8a8010ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:04.968 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.967+0000 7f8a87400640 1 --2- 172.21.15.83:0/1181524692 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a8010e690 0x7f8a8010ea70 secure :-1 s=READY pgs=2340 cs=0 l=1 rev1=1 crypto rx=0x7f8a74009890 tx=0x7f8a7402a380 comp rx=0 tx=0).stop 2024-04-24T09:09:04.968 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.968+0000 7f8a87400640 1 -- 172.21.15.83:0/1181524692 shutdown_connections 2024-04-24T09:09:04.968 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.968+0000 7f8a87400640 1 --2- 172.21.15.83:0/1181524692 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a80108fb0 0x7f8a801154c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:04.968 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.968+0000 7f8a87400640 1 --2- 172.21.15.83:0/1181524692 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8a801085f0 0x7f8a80108a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:04.968 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.968+0000 7f8a87400640 1 --2- 172.21.15.83:0/1181524692 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a8010e690 0x7f8a8010ea70 unknown :-1 s=CLOSED pgs=2340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:04.969 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.968+0000 7f8a87400640 1 -- 172.21.15.83:0/1181524692 >> 172.21.15.83:0/1181524692 conn(0x7f8a80076590 msgr2=0x7f8a800789b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:04.970 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.970+0000 7f8a87400640 1 -- 172.21.15.83:0/1181524692 shutdown_connections 2024-04-24T09:09:04.971 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.971+0000 7f8a87400640 1 -- 172.21.15.83:0/1181524692 wait complete. 2024-04-24T09:09:04.972 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.972+0000 7f8a87400640 1 Processor -- start 2024-04-24T09:09:04.973 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.973+0000 7f8a87400640 1 -- start start 2024-04-24T09:09:04.974 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.973+0000 7f8a87400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8a801085f0 0x7f8a8019eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:04.974 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.973+0000 7f8a87400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a80108fb0 0x7f8a8019f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:04.974 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.973+0000 7f8a87400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a8010e690 0x7f8a801a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:04.974 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.973+0000 7f8a87400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8a801188c0 con 0x7f8a80108fb0 2024-04-24T09:09:04.975 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.973+0000 7f8a87400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8a80118740 con 0x7f8a8010e690 2024-04-24T09:09:04.975 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.973+0000 7f8a87400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8a80118a40 con 0x7f8a801085f0 2024-04-24T09:09:04.975 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.974+0000 7f8a85400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a8010e690 0x7f8a801a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:04.975 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.974+0000 7f8a85400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a8010e690 0x7f8a801a3660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:42026/0 (socket says 172.21.15.83:42026) 2024-04-24T09:09:04.975 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.974+0000 7f8a85400640 1 -- 172.21.15.83:0/2771993676 learned_addr learned my addr 172.21.15.83:0/2771993676 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:09:04.975 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.974+0000 7f8a84a00640 1 --2- 172.21.15.83:0/2771993676 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8a801085f0 0x7f8a8019eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:04.975 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.974+0000 7f8a7fe00640 1 --2- 172.21.15.83:0/2771993676 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a80108fb0 0x7f8a8019f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:04.975 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.975+0000 7f8a85400640 1 -- 172.21.15.83:0/2771993676 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8a801085f0 msgr2=0x7f8a8019eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:04.976 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.975+0000 7f8a85400640 1 --2- 172.21.15.83:0/2771993676 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8a801085f0 0x7f8a8019eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:04.976 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.975+0000 7f8a85400640 1 -- 172.21.15.83:0/2771993676 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a80108fb0 msgr2=0x7f8a8019f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:04.976 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.976+0000 7f8a85400640 1 --2- 172.21.15.83:0/2771993676 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a80108fb0 0x7f8a8019f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:04.976 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.976+0000 7f8a85400640 1 -- 172.21.15.83:0/2771993676 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a801a3d90 con 0x7f8a8010e690 2024-04-24T09:09:04.976 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.976+0000 7f8a7fe00640 1 --2- 172.21.15.83:0/2771993676 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a80108fb0 0x7f8a8019f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:09:04.976 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.976+0000 7f8a85400640 1 --2- 172.21.15.83:0/2771993676 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a8010e690 0x7f8a801a3660 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8a78009fd0 tx=0x7f8a7800c740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:04.977 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.977+0000 7f8a7d600640 1 -- 172.21.15.83:0/2771993676 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a78004b20 con 0x7f8a8010e690 2024-04-24T09:09:04.978 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.977+0000 7f8a7d600640 1 -- 172.21.15.83:0/2771993676 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8a78003e20 con 0x7f8a8010e690 2024-04-24T09:09:04.978 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.977+0000 7f8a7d600640 1 -- 172.21.15.83:0/2771993676 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a78004820 con 0x7f8a8010e690 2024-04-24T09:09:04.978 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.977+0000 7f8a87400640 1 -- 172.21.15.83:0/2771993676 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a801a4080 con 0x7f8a8010e690 2024-04-24T09:09:04.978 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.977+0000 7f8a87400640 1 -- 172.21.15.83:0/2771993676 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a801ab910 con 0x7f8a8010e690 2024-04-24T09:09:04.981 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.981+0000 7f8a8a18e740 1 Processor -- start 2024-04-24T09:09:04.983 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.983+0000 7f8a7d600640 1 -- 172.21.15.83:0/2771993676 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8a7801d020 con 0x7f8a8010e690 2024-04-24T09:09:04.983 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.983+0000 7f8a8a18e740 1 -- start start 2024-04-24T09:09:04.983 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.983+0000 7f8a8a18e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559ded4601b0 0x559ded460590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:04.984 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.983+0000 7f8a8a18e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ded460ad0 0x559ded46ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:04.984 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.983+0000 7f8a8a18e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ded46b390 0x559ded46d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:04.984 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.983+0000 7f8a8a18e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559ded3c6900 con 0x559ded46b390 2024-04-24T09:09:04.984 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.983+0000 7f8a8a18e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559ded36cf20 con 0x559ded460ad0 2024-04-24T09:09:04.984 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.983+0000 7f8a8a18e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559ded3c28c0 con 0x559ded4601b0 2024-04-24T09:09:04.985 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.984+0000 7f8a84a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ded460ad0 0x559ded46ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:04.985 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.984+0000 7f8a7fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ded46b390 0x559ded46d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:04.985 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.984+0000 7f8a84a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ded460ad0 0x559ded46ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:42038/0 (socket says 172.21.15.83:42038) 2024-04-24T09:09:04.985 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.984+0000 7f8a84a00640 1 -- 172.21.15.83:0/3886938688 learned_addr learned my addr 172.21.15.83:0/3886938688 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:09:04.985 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.984+0000 7f8a7fe00640 1 --2- 172.21.15.83:0/3886938688 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559ded4601b0 0x559ded460590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:04.985 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.985+0000 7f8a7fe00640 1 -- 172.21.15.83:0/3886938688 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559ded4601b0 msgr2=0x559ded460590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:04.985 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.985+0000 7f8a7fe00640 1 --2- 172.21.15.83:0/3886938688 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559ded4601b0 0x559ded460590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:04.985 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.985+0000 7f8a7fe00640 1 -- 172.21.15.83:0/3886938688 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ded460ad0 msgr2=0x559ded46ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:04.985 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.985+0000 7f8a7fe00640 1 --2- 172.21.15.83:0/3886938688 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ded460ad0 0x559ded46ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:04.985 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.985+0000 7f8a7fe00640 1 -- 172.21.15.83:0/3886938688 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ded391e10 con 0x559ded46b390 2024-04-24T09:09:04.985 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.985+0000 7f8a7d600640 1 --2- 172.21.15.83:0/2771993676 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f8a680a3dc0 0x7f8a680a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:04.986 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.985+0000 7f8a7d600640 1 -- 172.21.15.83:0/2771993676 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 9583+0+0 (secure 0 0 0) 0x7f8a780d0ca0 con 0x7f8a8010e690 2024-04-24T09:09:04.986 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.985+0000 7f8a7fe00640 1 --2- 172.21.15.83:0/3886938688 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ded46b390 0x559ded46d780 secure :-1 s=READY pgs=2341 cs=0 l=1 rev1=1 crypto rx=0x7f8a6c0159c0 tx=0x7f8a6c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9bd9420339c3e9c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:04.986 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.986+0000 7f8a84a00640 1 --2- 172.21.15.83:0/2771993676 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f8a680a3dc0 0x7f8a680a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:04.986 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.986+0000 7f8a71c00640 1 -- 172.21.15.83:0/3886938688 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a6c00e7e0 con 0x559ded46b390 2024-04-24T09:09:04.986 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.986+0000 7f8a71c00640 1 -- 172.21.15.83:0/3886938688 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8a6c01d070 con 0x559ded46b390 2024-04-24T09:09:04.987 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.986+0000 7f8a71c00640 1 -- 172.21.15.83:0/3886938688 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a6c018b20 con 0x559ded46b390 2024-04-24T09:09:04.987 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.987+0000 7f8a8a18e740 1 -- 172.21.15.83:0/3886938688 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ded46b390 msgr2=0x559ded46d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:04.987 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.987+0000 7f8a8a18e740 1 --2- 172.21.15.83:0/3886938688 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ded46b390 0x559ded46d780 secure :-1 s=READY pgs=2341 cs=0 l=1 rev1=1 crypto rx=0x7f8a6c0159c0 tx=0x7f8a6c009840 comp rx=0 tx=0).stop 2024-04-24T09:09:04.987 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.987+0000 7f8a84a00640 1 --2- 172.21.15.83:0/2771993676 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f8a680a3dc0 0x7f8a680a6280 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f8a740095f0 tx=0x7f8a74005a90 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:04.987 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.987+0000 7f8a8a18e740 1 -- 172.21.15.83:0/3886938688 shutdown_connections 2024-04-24T09:09:04.988 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.987+0000 7f8a8a18e740 1 --2- 172.21.15.83:0/3886938688 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ded46b390 0x559ded46d780 unknown :-1 s=CLOSED pgs=2341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:04.988 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.987+0000 7f8a8a18e740 1 --2- 172.21.15.83:0/3886938688 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ded460ad0 0x559ded46ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:04.988 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.987+0000 7f8a8a18e740 1 --2- 172.21.15.83:0/3886938688 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559ded4601b0 0x559ded460590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:04.988 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.987+0000 7f8a8a18e740 1 -- 172.21.15.83:0/3886938688 >> 172.21.15.83:0/3886938688 conn(0x559ded45c6c0 msgr2=0x559ded45caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:04.988 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.987+0000 7f8a8a18e740 1 -- 172.21.15.83:0/3886938688 shutdown_connections 2024-04-24T09:09:04.988 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.987+0000 7f8a8a18e740 1 -- 172.21.15.83:0/3886938688 wait complete. 2024-04-24T09:09:04.989 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.988+0000 7f8a8a18e740 1 -- start start 2024-04-24T09:09:04.989 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.988+0000 7f8a8a18e740 1 Processor -- start 2024-04-24T09:09:04.989 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.989+0000 7f8a8a18e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ded4601b0 0x559ded468460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:04.989 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.989+0000 7f8a8a18e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559ded460ad0 0x559ded4689a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:04.989 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.989+0000 7f8a8a18e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ded46b390 0x559ded468ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:04.989 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.989+0000 7f8a8a18e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559ded46df60 con 0x559ded4601b0 2024-04-24T09:09:04.989 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.989+0000 7f8a8a18e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559ded3802c0 con 0x559ded46b390 2024-04-24T09:09:04.989 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.989+0000 7f8a8a18e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559ded46e0e0 con 0x559ded460ad0 2024-04-24T09:09:04.990 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.989+0000 7f8a8a18e740 10 client.0 _ll_register_callbacks cb 0x559ded0d8a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:09:04.990 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.989+0000 7f8a85400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ded46b390 0x559ded468ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:04.990 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.989+0000 7f8a85400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ded46b390 0x559ded468ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:42042/0 (socket says 172.21.15.83:42042) 2024-04-24T09:09:04.990 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.989+0000 7f8a85400640 1 -- 172.21.15.83:0/2930241234 learned_addr learned my addr 172.21.15.83:0/2930241234 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:09:04.990 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.989+0000 7f8a7fe00640 1 --2- 172.21.15.83:0/2930241234 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559ded460ad0 0x559ded4689a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:04.992 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.989+0000 7f8a85400640 1 -- 172.21.15.83:0/2930241234 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559ded460ad0 msgr2=0x559ded4689a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:04.992 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.989+0000 7f8a85400640 1 --2- 172.21.15.83:0/2930241234 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559ded460ad0 0x559ded4689a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:04.992 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.989+0000 7f8a85400640 1 -- 172.21.15.83:0/2930241234 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ded4601b0 msgr2=0x559ded468460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:04.992 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.989+0000 7f8a84a00640 1 --2- 172.21.15.83:0/2930241234 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ded4601b0 0x559ded468460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:04.992 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.990+0000 7f8a85400640 1 --2- 172.21.15.83:0/2930241234 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ded4601b0 0x559ded468460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:04.992 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.990+0000 7f8a85400640 1 -- 172.21.15.83:0/2930241234 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ded390fd0 con 0x559ded46b390 2024-04-24T09:09:04.992 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.990+0000 7f8a84a00640 1 --2- 172.21.15.83:0/2930241234 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ded4601b0 0x559ded468460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:09:04.992 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.990+0000 7f8a85400640 1 --2- 172.21.15.83:0/2930241234 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ded46b390 0x559ded468ee0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8a7800cfd0 tx=0x7f8a7809c3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e88b5f768a26fc6c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:04.993 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.990+0000 7f8a66000640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:09:04.993 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.991+0000 7f8a66000640 1 -- 172.21.15.83:0/2930241234 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a7801a040 con 0x559ded46b390 2024-04-24T09:09:04.993 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.991+0000 7f8a66000640 1 -- 172.21.15.83:0/2930241234 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9583+0+0 (secure 0 0 0) 0x7f8a780a4fa0 con 0x559ded46b390 2024-04-24T09:09:04.993 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.991+0000 7f8a8a18e740 1 -- 172.21.15.83:0/2930241234 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a6c01d070 con 0x559ded46b390 2024-04-24T09:09:04.993 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.991+0000 7f8a66000640 1 -- 172.21.15.83:0/2930241234 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f8a7809cec0 con 0x559ded46b390 2024-04-24T09:09:04.993 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.991+0000 7f8a8a18e740 10 client.38801 fetch_fsmap learned FSMap version 58 2024-04-24T09:09:04.993 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.991+0000 7f8a8a18e740 1 -- 172.21.15.83:0/2930241234 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x559ded0d9210 con 0x559ded46b390 2024-04-24T09:09:04.993 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.992+0000 7f8a66000640 1 -- 172.21.15.83:0/2930241234 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8a6c01d070 con 0x559ded46b390 2024-04-24T09:09:04.993 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.992+0000 7f8a66000640 1 -- 172.21.15.83:0/2930241234 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 58) ==== 8374+0+0 (secure 0 0 0) 0x7f8a780b3020 con 0x559ded46b390 2024-04-24T09:09:04.993 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.992+0000 7f8a8a18e740 10 client.38801 fetch_fsmap finished waiting for FSMap version 58 2024-04-24T09:09:04.993 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.992+0000 7f8a8a18e740 10 client.38801 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:09:04.993 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.992+0000 7f8a8a18e740 20 client.38801 populate_metadata read hostname 'smithi083' 2024-04-24T09:09:04.993 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.992+0000 7f8a8a18e740 1 --2- 172.21.15.83:0/2930241234 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x559ded47b600 0x559ded47d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:04.993 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.992+0000 7f8a8a18e740 4 client.38801 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:09:04.994 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.992+0000 7f8a8a18e740 1 -- 172.21.15.83:0/2930241234 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559ded3d4320 con 0x559ded47b600 2024-04-24T09:09:04.994 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.993+0000 7f8a84a00640 1 --2- 172.21.15.83:0/2930241234 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x559ded47b600 0x559ded47d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:04.994 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.994+0000 7f8a84a00640 1 --2- 172.21.15.83:0/2930241234 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x559ded47b600 0x559ded47d9f0 crc :-1 s=READY pgs=439 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=629cc23938b4fcdf server_cookie=9faef045ad87b593 in_seq=0 out_seq=0 2024-04-24T09:09:04.994 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.994+0000 7f8a66000640 10 client.38801 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:09:04.996 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.995+0000 7f8a66000640 1 -- 172.21.15.83:0/2930241234 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559ded3d4320 con 0x559ded47b600 2024-04-24T09:09:04.996 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.995+0000 7f8a66000640 10 client.38801 handle_command_reply: tid=0 2024-04-24T09:09:04.996 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.995+0000 7f8a8a18e740 1 client.38801 shutdown 2024-04-24T09:09:04.996 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.995+0000 7f8a8a18e740 1 -- 172.21.15.83:0/2930241234 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ded46b390 msgr2=0x559ded468ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:04.996 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.995+0000 7f8a8a18e740 1 --2- 172.21.15.83:0/2930241234 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ded46b390 0x559ded468ee0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8a7800cfd0 tx=0x7f8a7809c3c0 comp rx=0 tx=0).stop 2024-04-24T09:09:04.996 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.995+0000 7f8a8a18e740 1 -- 172.21.15.83:0/2930241234 shutdown_connections 2024-04-24T09:09:04.996 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.995+0000 7f8a8a18e740 1 --2- 172.21.15.83:0/2930241234 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x559ded47b600 0x559ded47d9f0 crc :-1 s=READY pgs=439 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:04.996 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.995+0000 7f8a8a18e740 1 --2- 172.21.15.83:0/2930241234 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ded46b390 0x559ded468ee0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:04.996 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.995+0000 7f8a8a18e740 1 --2- 172.21.15.83:0/2930241234 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559ded460ad0 0x559ded4689a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:04.996 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.995+0000 7f8a8a18e740 1 --2- 172.21.15.83:0/2930241234 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ded4601b0 0x559ded468460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:04.996 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.995+0000 7f8a8a18e740 1 -- 172.21.15.83:0/2930241234 >> 172.21.15.83:0/2930241234 conn(0x559ded45c6c0 msgr2=0x559ded462440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:04.996 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.995+0000 7f8a66000640 0 client.38801 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:09:04.996 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.996+0000 7f8a8a18e740 1 -- 172.21.15.83:0/2930241234 shutdown_connections 2024-04-24T09:09:04.997 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.996+0000 7f8a8a18e740 1 -- 172.21.15.83:0/2930241234 wait complete. 2024-04-24T09:09:04.997 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:04.996+0000 7f8a8a18e740 20 client.38801 trim_cache size 0 max 16384 2024-04-24T09:09:05.021 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:05.021+0000 7f8a8a18e740 1 Processor -- start 2024-04-24T09:09:05.021 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:05.021+0000 7f8a8a18e740 1 -- start start 2024-04-24T09:09:05.021 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:05.021+0000 7f8a8a18e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ded4601b0 0x559ded45bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:05.021 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:05.021+0000 7f8a8a18e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559ded460ad0 0x559ded462b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:05.021 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:05.021+0000 7f8a8a18e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ded46b390 0x559ded468b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:05.021 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:05.021+0000 7f8a8a18e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8a801b5c00 con 0x559ded4601b0 2024-04-24T09:09:05.021 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:05.021+0000 7f8a8a18e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8a801b7740 con 0x559ded46b390 2024-04-24T09:09:05.021 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:05.021+0000 7f8a8a18e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8a801b62d0 con 0x559ded460ad0 2024-04-24T09:09:05.022 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:05.021+0000 7f8a84a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ded4601b0 0x559ded45bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:05.022 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:05.021+0000 7f8a84a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ded4601b0 0x559ded45bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45634/0 (socket says 172.21.15.83:45634) 2024-04-24T09:09:05.023 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:05.021+0000 7f8a84a00640 1 -- 172.21.15.83:0/1569089813 learned_addr learned my addr 172.21.15.83:0/1569089813 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:09:05.023 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:05.022+0000 7f8a85400640 1 --2- 172.21.15.83:0/1569089813 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ded46b390 0x559ded468b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:05.023 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:05.022+0000 7f8a7fe00640 1 --2- 172.21.15.83:0/1569089813 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559ded460ad0 0x559ded462b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:05.023 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:05.022+0000 7f8a84a00640 1 -- 172.21.15.83:0/1569089813 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559ded460ad0 msgr2=0x559ded462b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:05.023 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:05.022+0000 7f8a84a00640 1 --2- 172.21.15.83:0/1569089813 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559ded460ad0 0x559ded462b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:05.023 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:05.022+0000 7f8a84a00640 1 -- 172.21.15.83:0/1569089813 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ded46b390 msgr2=0x559ded468b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:05.023 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:05.022+0000 7f8a84a00640 1 --2- 172.21.15.83:0/1569089813 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ded46b390 0x559ded468b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:05.023 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:05.022+0000 7f8a84a00640 1 -- 172.21.15.83:0/1569089813 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ded46daf0 con 0x559ded4601b0 2024-04-24T09:09:05.023 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:05.022+0000 7f8a85400640 1 --2- 172.21.15.83:0/1569089813 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ded46b390 0x559ded468b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:09:05.023 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:05.022+0000 7f8a84a00640 1 --2- 172.21.15.83:0/1569089813 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ded4601b0 0x559ded45bfc0 secure :-1 s=READY pgs=2342 cs=0 l=1 rev1=1 crypto rx=0x7f8a7402a970 tx=0x7f8a7403e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9988152b042d8875 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:05.024 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:05.023+0000 7f8a71200640 1 -- 172.21.15.83:0/1569089813 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a74037040 con 0x559ded4601b0 2024-04-24T09:09:05.024 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:05.023+0000 7f8a71200640 1 -- 172.21.15.83:0/1569089813 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8a74044070 con 0x559ded4601b0 2024-04-24T09:09:05.024 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:05.023+0000 7f8a71200640 1 -- 172.21.15.83:0/1569089813 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a74051880 con 0x559ded4601b0 2024-04-24T09:09:05.024 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:05.024+0000 7f8a8a18e740 1 -- 172.21.15.83:0/1569089813 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ded4601b0 msgr2=0x559ded45bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:05.024 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:05.024+0000 7f8a8a18e740 1 --2- 172.21.15.83:0/1569089813 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ded4601b0 0x559ded45bfc0 secure :-1 s=READY pgs=2342 cs=0 l=1 rev1=1 crypto rx=0x7f8a7402a970 tx=0x7f8a7403e000 comp rx=0 tx=0).stop 2024-04-24T09:09:05.024 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:05.024+0000 7f8a8a18e740 1 -- 172.21.15.83:0/1569089813 shutdown_connections 2024-04-24T09:09:05.025 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:05.024+0000 7f8a8a18e740 1 --2- 172.21.15.83:0/1569089813 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ded46b390 0x559ded468b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:05.025 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:05.024+0000 7f8a8a18e740 1 --2- 172.21.15.83:0/1569089813 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559ded460ad0 0x559ded462b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:05.025 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:05.024+0000 7f8a8a18e740 1 --2- 172.21.15.83:0/1569089813 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ded4601b0 0x559ded45bfc0 unknown :-1 s=CLOSED pgs=2342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:05.025 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:05.024+0000 7f8a8a18e740 1 -- 172.21.15.83:0/1569089813 >> 172.21.15.83:0/1569089813 conn(0x559ded45c6c0 msgr2=0x559ded468420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:05.025 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:05.024+0000 7f8a8a18e740 1 -- 172.21.15.83:0/1569089813 shutdown_connections 2024-04-24T09:09:05.025 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:05.024+0000 7f8a8a18e740 1 -- 172.21.15.83:0/1569089813 wait complete. 2024-04-24T09:09:05.026 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:05.025+0000 7f8a8a18e740 1 -- start start 2024-04-24T09:09:05.026 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:05.025+0000 7f8a8a18e740 1 Processor -- start 2024-04-24T09:09:05.026 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:05.026+0000 7f8a8a18e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ded4601b0 0x559ded473520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:05.026 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:05.026+0000 7f8a8a18e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559ded460ad0 0x559ded464d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:05.026 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:05.026+0000 7f8a8a18e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ded46b390 0x559ded465250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:05.026 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:05.026+0000 7f8a8a18e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8a801b7070 con 0x559ded46b390 2024-04-24T09:09:05.026 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:05.026+0000 7f8a8a18e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8a801b69a0 con 0x559ded4601b0 2024-04-24T09:09:05.026 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:05.026+0000 7f8a8a18e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8a801b3c00 con 0x559ded460ad0 2024-04-24T09:09:05.026 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:05.026+0000 7f8a8a18e740 10 client.0 _ll_register_callbacks cb 0x7f8a78004020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:09:05.027 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:05.026+0000 7f8a85400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ded46b390 0x559ded465250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:05.027 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:05.026+0000 7f8a85400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ded46b390 0x559ded465250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45640/0 (socket says 172.21.15.83:45640) 2024-04-24T09:09:05.027 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:05.026+0000 7f8a85400640 1 -- 172.21.15.83:0/2515084541 learned_addr learned my addr 172.21.15.83:0/2515084541 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:09:05.027 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:05.026+0000 7f8a84a00640 1 --2- 172.21.15.83:0/2515084541 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ded4601b0 0x559ded473520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:05.028 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:05.026+0000 7f8a7fe00640 1 --2- 172.21.15.83:0/2515084541 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559ded460ad0 0x559ded464d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:05.028 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:05.027+0000 7f8a85400640 1 -- 172.21.15.83:0/2515084541 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559ded460ad0 msgr2=0x559ded464d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:05.028 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:05.027+0000 7f8a85400640 1 --2- 172.21.15.83:0/2515084541 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559ded460ad0 0x559ded464d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:05.028 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:05.027+0000 7f8a85400640 1 -- 172.21.15.83:0/2515084541 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ded4601b0 msgr2=0x559ded473520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:05.028 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:05.027+0000 7f8a85400640 1 --2- 172.21.15.83:0/2515084541 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ded4601b0 0x559ded473520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:05.028 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:05.027+0000 7f8a85400640 1 -- 172.21.15.83:0/2515084541 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ded1a5620 con 0x559ded46b390 2024-04-24T09:09:05.028 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:05.027+0000 7f8a84a00640 1 --2- 172.21.15.83:0/2515084541 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ded4601b0 0x559ded473520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:09:05.028 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:05.027+0000 7f8a85400640 1 --2- 172.21.15.83:0/2515084541 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ded46b390 0x559ded465250 secure :-1 s=READY pgs=2343 cs=0 l=1 rev1=1 crypto rx=0x7f8a7809c750 tx=0x7f8a7800cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc0521bdb9c36d19 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:05.028 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:05.027+0000 7f8a66000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:09:05.028 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:05.028+0000 7f8a66000640 1 -- 172.21.15.83:0/2515084541 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a7801a070 con 0x559ded46b390 2024-04-24T09:09:05.028 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:05.028+0000 7f8a66000640 1 -- 172.21.15.83:0/2515084541 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9583+0+0 (secure 0 0 0) 0x7f8a780985e0 con 0x559ded46b390 2024-04-24T09:09:05.029 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:05.028+0000 7f8a8a18e740 1 -- 172.21.15.83:0/2515084541 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a74044070 con 0x559ded46b390 2024-04-24T09:09:05.029 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:05.028+0000 7f8a66000640 1 -- 172.21.15.83:0/2515084541 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f8a78098a60 con 0x559ded46b390 2024-04-24T09:09:05.029 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:05.028+0000 7f8a8a18e740 10 client.21672 fetch_fsmap learned FSMap version 58 2024-04-24T09:09:05.030 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:05.029+0000 7f8a8a18e740 1 -- 172.21.15.83:0/2515084541 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x559ded4f27d0 con 0x559ded46b390 2024-04-24T09:09:05.030 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:05.029+0000 7f8a66000640 1 -- 172.21.15.83:0/2515084541 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8a74044070 con 0x559ded46b390 2024-04-24T09:09:05.030 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:05.029+0000 7f8a66000640 1 -- 172.21.15.83:0/2515084541 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 58) ==== 8374+0+0 (secure 0 0 0) 0x7f8a780b3040 con 0x559ded46b390 2024-04-24T09:09:05.030 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:05.029+0000 7f8a8a18e740 10 client.21672 fetch_fsmap finished waiting for FSMap version 58 2024-04-24T09:09:05.030 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:05.029+0000 7f8a8a18e740 10 client.21672 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:09:05.030 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:05.029+0000 7f8a8a18e740 20 client.21672 populate_metadata read hostname 'smithi083' 2024-04-24T09:09:05.030 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:05.029+0000 7f8a8a18e740 1 --2- 172.21.15.83:0/2515084541 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x559ded4f2fa0 0x559ded4f53c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:05.030 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:05.029+0000 7f8a8a18e740 4 client.21672 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T09:09:05.030 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:05.029+0000 7f8a8a18e740 1 -- 172.21.15.83:0/2515084541 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559ded3ce190 con 0x559ded4f2fa0 2024-04-24T09:09:05.031 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:05.030+0000 7f8a84a00640 1 --2- 172.21.15.83:0/2515084541 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x559ded4f2fa0 0x559ded4f53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:05.031 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:05.031+0000 7f8a84a00640 1 --2- 172.21.15.83:0/2515084541 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x559ded4f2fa0 0x559ded4f53c0 crc :-1 s=READY pgs=440 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ad5122cc9e5abc7 server_cookie=31ede841ac9f5df2 in_seq=0 out_seq=0 2024-04-24T09:09:05.031 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:05.031+0000 7f8a66000640 10 client.21672 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:09:05.168 INFO:tasks.workunit.client.0.smithi083.stdout: 10 27664 6.77 MB/sec execute 96 sec latency 81.858 ms 2024-04-24T09:09:05.235 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:05.235+0000 7f8a66000640 1 -- 172.21.15.83:0/2515084541 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559ded3ce190 con 0x559ded4f2fa0 2024-04-24T09:09:05.236 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:05.235+0000 7f8a66000640 10 client.21672 handle_command_reply: tid=0 2024-04-24T09:09:05.236 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:05.235+0000 7f8a8a18e740 1 client.21672 shutdown 2024-04-24T09:09:05.236 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:05.235+0000 7f8a8a18e740 1 -- 172.21.15.83:0/2515084541 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ded46b390 msgr2=0x559ded465250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:05.236 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:05.235+0000 7f8a8a18e740 1 --2- 172.21.15.83:0/2515084541 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ded46b390 0x559ded465250 secure :-1 s=READY pgs=2343 cs=0 l=1 rev1=1 crypto rx=0x7f8a7809c750 tx=0x7f8a7800cd70 comp rx=0 tx=0).stop 2024-04-24T09:09:05.237 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:05.236+0000 7f8a8a18e740 1 -- 172.21.15.83:0/2515084541 shutdown_connections 2024-04-24T09:09:05.237 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:05.236+0000 7f8a8a18e740 1 --2- 172.21.15.83:0/2515084541 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x559ded4f2fa0 0x559ded4f53c0 crc :-1 s=READY pgs=440 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:05.237 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:05.236+0000 7f8a8a18e740 1 --2- 172.21.15.83:0/2515084541 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ded46b390 0x559ded465250 unknown :-1 s=CLOSED pgs=2343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:05.237 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:05.236+0000 7f8a8a18e740 1 --2- 172.21.15.83:0/2515084541 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559ded460ad0 0x559ded464d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:05.237 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:05.236+0000 7f8a8a18e740 1 --2- 172.21.15.83:0/2515084541 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559ded4601b0 0x559ded473520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:05.237 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:05.236+0000 7f8a8a18e740 1 -- 172.21.15.83:0/2515084541 >> 172.21.15.83:0/2515084541 conn(0x559ded45c6c0 msgr2=0x559ded468cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:05.237 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:05.236+0000 7f8a66000640 0 client.21672 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:09:05.237 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:05.236+0000 7f8a8a18e740 1 -- 172.21.15.83:0/2515084541 shutdown_connections 2024-04-24T09:09:05.237 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:05.236+0000 7f8a8a18e740 1 -- 172.21.15.83:0/2515084541 wait complete. 2024-04-24T09:09:05.237 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:05.236+0000 7f8a8a18e740 20 client.21672 trim_cache size 0 max 16384 2024-04-24T09:09:05.237 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T09:09:05.238 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T09:09:05.238 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T09:09:05.238 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T09:09:05.238 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:05.238+0000 7f8a87400640 1 -- 172.21.15.83:0/2771993676 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f8a680a3dc0 msgr2=0x7f8a680a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:05.238 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:05.238+0000 7f8a87400640 1 --2- 172.21.15.83:0/2771993676 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f8a680a3dc0 0x7f8a680a6280 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f8a740095f0 tx=0x7f8a74005a90 comp rx=0 tx=0).stop 2024-04-24T09:09:05.239 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:05.238+0000 7f8a87400640 1 -- 172.21.15.83:0/2771993676 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a8010e690 msgr2=0x7f8a801a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:05.239 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:05.238+0000 7f8a87400640 1 --2- 172.21.15.83:0/2771993676 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a8010e690 0x7f8a801a3660 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8a78009fd0 tx=0x7f8a7800c740 comp rx=0 tx=0).stop 2024-04-24T09:09:05.239 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:05.238+0000 7f8a87400640 1 -- 172.21.15.83:0/2771993676 shutdown_connections 2024-04-24T09:09:05.239 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:05.238+0000 7f8a87400640 1 --2- 172.21.15.83:0/2771993676 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f8a680a3dc0 0x7f8a680a6280 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:05.239 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:05.238+0000 7f8a87400640 1 --2- 172.21.15.83:0/2771993676 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a8010e690 0x7f8a801a3660 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:05.239 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:05.238+0000 7f8a87400640 1 --2- 172.21.15.83:0/2771993676 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a80108fb0 0x7f8a8019f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:05.239 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:05.238+0000 7f8a87400640 1 --2- 172.21.15.83:0/2771993676 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8a801085f0 0x7f8a8019eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:05.239 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:05.238+0000 7f8a87400640 1 -- 172.21.15.83:0/2771993676 >> 172.21.15.83:0/2771993676 conn(0x7f8a80076590 msgr2=0x7f8a80077d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:05.239 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:05.239+0000 7f8a87400640 1 -- 172.21.15.83:0/2771993676 shutdown_connections 2024-04-24T09:09:05.240 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:05.239+0000 7f8a87400640 1 -- 172.21.15.83:0/2771993676 wait complete. 2024-04-24T09:09:05.954 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:09:05 smithi138 ceph-mon[31441]: osdmap e118: 12 total, 12 up, 12 in 2024-04-24T09:09:05.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:09:05 smithi138 ceph-mon[31441]: pgmap v551: 129 pgs: 129 active+clean; 525 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.4 MiB/s wr, 341 op/s 2024-04-24T09:09:05.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:09:05 smithi189 ceph-mon[31791]: osdmap e118: 12 total, 12 up, 12 in 2024-04-24T09:09:05.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:09:05 smithi189 ceph-mon[31791]: pgmap v551: 129 pgs: 129 active+clean; 525 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.4 MiB/s wr, 341 op/s 2024-04-24T09:09:06.052 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:09:05 smithi083 ceph-mon[27907]: osdmap e118: 12 total, 12 up, 12 in 2024-04-24T09:09:06.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:09:05 smithi083 ceph-mon[27907]: pgmap v551: 129 pgs: 129 active+clean; 525 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.4 MiB/s wr, 341 op/s 2024-04-24T09:09:06.168 INFO:tasks.workunit.client.0.smithi083.stdout: 10 27802 6.77 MB/sec execute 97 sec latency 105.408 ms 2024-04-24T09:09:07.168 INFO:tasks.workunit.client.0.smithi083.stdout: 10 27927 6.76 MB/sec execute 98 sec latency 65.445 ms 2024-04-24T09:09:08.168 INFO:tasks.workunit.client.0.smithi083.stdout: 10 28054 6.74 MB/sec execute 99 sec latency 75.133 ms 2024-04-24T09:09:08.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:09:07 smithi138 ceph-mon[31441]: pgmap v552: 129 pgs: 129 active+clean; 525 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 195 op/s 2024-04-24T09:09:08.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:09:07 smithi189 ceph-mon[31791]: pgmap v552: 129 pgs: 129 active+clean; 525 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 195 op/s 2024-04-24T09:09:08.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:09:07 smithi083 ceph-mon[27907]: pgmap v552: 129 pgs: 129 active+clean; 525 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 195 op/s 2024-04-24T09:09:09.169 INFO:tasks.workunit.client.0.smithi083.stdout: 10 28195 6.75 MB/sec execute 100 sec latency 132.535 ms 2024-04-24T09:09:10.169 INFO:tasks.workunit.client.0.smithi083.stdout: 10 28370 6.80 MB/sec execute 101 sec latency 70.881 ms 2024-04-24T09:09:10.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:09:09 smithi138 ceph-mon[31441]: pgmap v553: 129 pgs: 129 active+clean; 529 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.0 MiB/s wr, 375 op/s 2024-04-24T09:09:10.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:09:09 smithi189 ceph-mon[31791]: pgmap v553: 129 pgs: 129 active+clean; 529 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.0 MiB/s wr, 375 op/s 2024-04-24T09:09:10.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:09:09 smithi083 ceph-mon[27907]: pgmap v553: 129 pgs: 129 active+clean; 529 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.0 MiB/s wr, 375 op/s 2024-04-24T09:09:11.169 INFO:tasks.workunit.client.0.smithi083.stdout: 10 28550 6.83 MB/sec execute 102 sec latency 81.857 ms 2024-04-24T09:09:12.169 INFO:tasks.workunit.client.0.smithi083.stdout: 10 28700 6.82 MB/sec execute 103 sec latency 104.462 ms 2024-04-24T09:09:12.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:09:11 smithi138 ceph-mon[31441]: pgmap v554: 129 pgs: 129 active+clean; 529 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.7 MiB/s wr, 350 op/s 2024-04-24T09:09:12.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:09:11 smithi189 ceph-mon[31791]: pgmap v554: 129 pgs: 129 active+clean; 529 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.7 MiB/s wr, 350 op/s 2024-04-24T09:09:12.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:09:11 smithi083 ceph-mon[27907]: pgmap v554: 129 pgs: 129 active+clean; 529 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.7 MiB/s wr, 350 op/s 2024-04-24T09:09:13.169 INFO:tasks.workunit.client.0.smithi083.stdout: 10 28819 6.86 MB/sec execute 104 sec latency 346.597 ms 2024-04-24T09:09:14.081 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T09:09:14.081 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T09:09:14.169 INFO:tasks.workunit.client.0.smithi083.stdout: 10 28949 6.92 MB/sec execute 105 sec latency 83.050 ms 2024-04-24T09:09:14.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:09:13 smithi138 ceph-mon[31441]: pgmap v555: 129 pgs: 129 active+clean; 546 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.8 MiB/s wr, 270 op/s 2024-04-24T09:09:14.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:09:13 smithi189 ceph-mon[31791]: pgmap v555: 129 pgs: 129 active+clean; 546 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.8 MiB/s wr, 270 op/s 2024-04-24T09:09:14.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:09:13 smithi083 ceph-mon[27907]: pgmap v555: 129 pgs: 129 active+clean; 546 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.8 MiB/s wr, 270 op/s 2024-04-24T09:09:14.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.326+0000 7f722ca00640 1 -- 172.21.15.83:0/721438914 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7228102110 msgr2=0x7f72281024f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:14.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.326+0000 7f722ca00640 1 --2- 172.21.15.83:0/721438914 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7228102110 0x7f72281024f0 secure :-1 s=READY pgs=2344 cs=0 l=1 rev1=1 crypto rx=0x7f7218009900 tx=0x7f721802a3f0 comp rx=0 tx=0).stop 2024-04-24T09:09:14.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.326+0000 7f722ca00640 1 -- 172.21.15.83:0/721438914 shutdown_connections 2024-04-24T09:09:14.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.326+0000 7f722ca00640 1 --2- 172.21.15.83:0/721438914 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f72281071e0 0x7f722810edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:14.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.326+0000 7f722ca00640 1 --2- 172.21.15.83:0/721438914 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7228102ac0 0x7f7228106b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:14.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.326+0000 7f722ca00640 1 --2- 172.21.15.83:0/721438914 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7228102110 0x7f72281024f0 unknown :-1 s=CLOSED pgs=2344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:14.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.326+0000 7f722ca00640 1 -- 172.21.15.83:0/721438914 >> 172.21.15.83:0/721438914 conn(0x7f72280fd9d0 msgr2=0x7f72280ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:14.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.326+0000 7f722ca00640 1 -- 172.21.15.83:0/721438914 shutdown_connections 2024-04-24T09:09:14.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.326+0000 7f722ca00640 1 -- 172.21.15.83:0/721438914 wait complete. 2024-04-24T09:09:14.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.328+0000 7f722ca00640 1 Processor -- start 2024-04-24T09:09:14.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.328+0000 7f722ca00640 1 -- start start 2024-04-24T09:09:14.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.328+0000 7f722ca00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7228102110 0x7f72281a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:14.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.328+0000 7f722ca00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f72281071e0 0x7f72281a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:14.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.328+0000 7f722ca00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f722819a6d0 0x7f722819ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:14.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.328+0000 7f722ca00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f72281143d0 con 0x7f7228102110 2024-04-24T09:09:14.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.328+0000 7f722ca00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7228114250 con 0x7f722819a6d0 2024-04-24T09:09:14.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.328+0000 7f722ca00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7228114550 con 0x7f72281071e0 2024-04-24T09:09:14.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.329+0000 7f7227400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7228102110 0x7f72281a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:14.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.329+0000 7f7227400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7228102110 0x7f72281a0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60142/0 (socket says 172.21.15.83:60142) 2024-04-24T09:09:14.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.329+0000 7f7227400640 1 -- 172.21.15.83:0/550951894 learned_addr learned my addr 172.21.15.83:0/550951894 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:09:14.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.329+0000 7f7227e00640 1 --2- 172.21.15.83:0/550951894 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f722819a6d0 0x7f722819ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:14.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.329+0000 7f7226a00640 1 --2- 172.21.15.83:0/550951894 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f72281071e0 0x7f72281a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:14.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.329+0000 7f7227e00640 1 -- 172.21.15.83:0/550951894 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f72281071e0 msgr2=0x7f72281a0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:14.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.329+0000 7f7227e00640 1 --2- 172.21.15.83:0/550951894 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f72281071e0 0x7f72281a0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:14.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.329+0000 7f7227e00640 1 -- 172.21.15.83:0/550951894 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7228102110 msgr2=0x7f72281a0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:14.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.329+0000 7f7227e00640 1 --2- 172.21.15.83:0/550951894 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7228102110 0x7f72281a0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:14.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.329+0000 7f7227e00640 1 -- 172.21.15.83:0/550951894 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f722819b2d0 con 0x7f722819a6d0 2024-04-24T09:09:14.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.330+0000 7f7227e00640 1 --2- 172.21.15.83:0/550951894 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f722819a6d0 0x7f722819ab50 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f7220009fd0 tx=0x7f722000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:14.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.330+0000 7f721fe00640 1 -- 172.21.15.83:0/550951894 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f722000ce20 con 0x7f722819a6d0 2024-04-24T09:09:14.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.330+0000 7f722ca00640 1 -- 172.21.15.83:0/550951894 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72281a73b0 con 0x7f722819a6d0 2024-04-24T09:09:14.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.330+0000 7f722ca00640 1 -- 172.21.15.83:0/550951894 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72281a77b0 con 0x7f722819a6d0 2024-04-24T09:09:14.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.330+0000 7f721fe00640 1 -- 172.21.15.83:0/550951894 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7220005020 con 0x7f722819a6d0 2024-04-24T09:09:14.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.330+0000 7f721fe00640 1 -- 172.21.15.83:0/550951894 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7220010690 con 0x7f722819a6d0 2024-04-24T09:09:14.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.334+0000 7f7230df0740 1 Processor -- start 2024-04-24T09:09:14.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.335+0000 7f721fe00640 1 -- 172.21.15.83:0/550951894 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7220004a40 con 0x7f722819a6d0 2024-04-24T09:09:14.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.336+0000 7f7230df0740 1 -- start start 2024-04-24T09:09:14.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.336+0000 7f7230df0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5566da3fca30 0x5566da3fce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:14.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.336+0000 7f7230df0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5566da3fd350 0x5566da4074e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:14.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.336+0000 7f7230df0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5566da407a20 0x5566da409e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:14.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.336+0000 7f7230df0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5566da3621f0 con 0x5566da407a20 2024-04-24T09:09:14.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.336+0000 7f7230df0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5566da308350 con 0x5566da3fca30 2024-04-24T09:09:14.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.336+0000 7f7230df0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5566da35e1b0 con 0x5566da3fd350 2024-04-24T09:09:14.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.337+0000 7f7226a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5566da3fca30 0x5566da3fce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:14.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.337+0000 7f7226a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5566da3fca30 0x5566da3fce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:52922/0 (socket says 172.21.15.83:52922) 2024-04-24T09:09:14.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.337+0000 7f7226a00640 1 -- 172.21.15.83:0/1598870989 learned_addr learned my addr 172.21.15.83:0/1598870989 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:09:14.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.337+0000 7f7226a00640 1 --2- 172.21.15.83:0/1598870989 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5566da407a20 0x5566da409e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:14.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.337+0000 7f7227400640 1 --2- 172.21.15.83:0/1598870989 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5566da3fd350 0x5566da4074e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:14.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.337+0000 7f7226a00640 1 -- 172.21.15.83:0/1598870989 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5566da3fd350 msgr2=0x5566da4074e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:14.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.337+0000 7f7226a00640 1 --2- 172.21.15.83:0/1598870989 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5566da3fd350 0x5566da4074e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:14.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.337+0000 7f7226a00640 1 -- 172.21.15.83:0/1598870989 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5566da407a20 msgr2=0x5566da409e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:14.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.337+0000 7f7226a00640 1 --2- 172.21.15.83:0/1598870989 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5566da407a20 0x5566da409e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:14.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.337+0000 7f7226a00640 1 -- 172.21.15.83:0/1598870989 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566da35cf50 con 0x5566da3fca30 2024-04-24T09:09:14.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.337+0000 7f721fe00640 1 --2- 172.21.15.83:0/550951894 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f720c0a3da0 0x7f720c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:14.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.338+0000 7f7226a00640 1 --2- 172.21.15.83:0/1598870989 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5566da3fca30 0x5566da3fce10 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f7214005880 tx=0x7f72140046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11ae70bd601de311 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:14.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.338+0000 7f721fe00640 1 -- 172.21.15.83:0/550951894 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 9583+0+0 (secure 0 0 0) 0x7f72200d1140 con 0x7f722819a6d0 2024-04-24T09:09:14.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.338+0000 7f7213e00640 1 -- 172.21.15.83:0/1598870989 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7214012ce0 con 0x5566da3fca30 2024-04-24T09:09:14.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.338+0000 7f7213e00640 1 -- 172.21.15.83:0/1598870989 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7214005020 con 0x5566da3fca30 2024-04-24T09:09:14.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.338+0000 7f7213e00640 1 -- 172.21.15.83:0/1598870989 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72140044e0 con 0x5566da3fca30 2024-04-24T09:09:14.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.339+0000 7f7227400640 1 --2- 172.21.15.83:0/550951894 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f720c0a3da0 0x7f720c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:14.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.339+0000 7f7230df0740 1 -- 172.21.15.83:0/1598870989 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5566da3fca30 msgr2=0x5566da3fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:14.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.339+0000 7f7230df0740 1 --2- 172.21.15.83:0/1598870989 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5566da3fca30 0x5566da3fce10 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f7214005880 tx=0x7f72140046c0 comp rx=0 tx=0).stop 2024-04-24T09:09:14.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.339+0000 7f7230df0740 1 -- 172.21.15.83:0/1598870989 shutdown_connections 2024-04-24T09:09:14.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.339+0000 7f7230df0740 1 --2- 172.21.15.83:0/1598870989 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5566da407a20 0x5566da409e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:14.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.339+0000 7f7230df0740 1 --2- 172.21.15.83:0/1598870989 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5566da3fd350 0x5566da4074e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:14.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.339+0000 7f7230df0740 1 --2- 172.21.15.83:0/1598870989 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5566da3fca30 0x5566da3fce10 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:14.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.339+0000 7f7230df0740 1 -- 172.21.15.83:0/1598870989 >> 172.21.15.83:0/1598870989 conn(0x5566da3f8d50 msgr2=0x5566da3f9130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:14.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.340+0000 7f7227400640 1 --2- 172.21.15.83:0/550951894 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f720c0a3da0 0x7f720c0a6260 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x5566da3fdff0 tx=0x7f7218005a20 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:14.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.340+0000 7f7230df0740 1 -- 172.21.15.83:0/1598870989 shutdown_connections 2024-04-24T09:09:14.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.340+0000 7f7230df0740 1 -- 172.21.15.83:0/1598870989 wait complete. 2024-04-24T09:09:14.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.340+0000 7f7230df0740 1 -- start start 2024-04-24T09:09:14.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.341+0000 7f7230df0740 1 Processor -- start 2024-04-24T09:09:14.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.341+0000 7f7230df0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5566da3fca30 0x5566da404930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:14.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.341+0000 7f7230df0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5566da3fd350 0x5566da404e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:14.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.341+0000 7f7230df0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5566da407a20 0x5566da4053b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:14.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.341+0000 7f7230df0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5566da41b820 con 0x5566da3fd350 2024-04-24T09:09:14.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.341+0000 7f7230df0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5566da31b690 con 0x5566da3fca30 2024-04-24T09:09:14.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.341+0000 7f7230df0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5566da41b9a0 con 0x5566da407a20 2024-04-24T09:09:14.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.341+0000 7f7230df0740 10 client.0 _ll_register_callbacks cb 0x5566da357650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:09:14.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.342+0000 7f7227400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5566da3fca30 0x5566da404930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:14.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.342+0000 7f7227400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5566da3fca30 0x5566da404930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:52926/0 (socket says 172.21.15.83:52926) 2024-04-24T09:09:14.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.342+0000 7f7227400640 1 -- 172.21.15.83:0/2695576377 learned_addr learned my addr 172.21.15.83:0/2695576377 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:09:14.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.342+0000 7f7226a00640 1 --2- 172.21.15.83:0/2695576377 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5566da3fd350 0x5566da404e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:14.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.342+0000 7f7227e00640 1 --2- 172.21.15.83:0/2695576377 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5566da407a20 0x5566da4053b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:14.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.342+0000 7f7226a00640 1 -- 172.21.15.83:0/2695576377 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5566da407a20 msgr2=0x5566da4053b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:14.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.342+0000 7f7226a00640 1 --2- 172.21.15.83:0/2695576377 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5566da407a20 0x5566da4053b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:14.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.342+0000 7f7226a00640 1 -- 172.21.15.83:0/2695576377 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5566da3fca30 msgr2=0x5566da404930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:14.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.342+0000 7f7226a00640 1 --2- 172.21.15.83:0/2695576377 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5566da3fca30 0x5566da404930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:14.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.343+0000 7f7226a00640 1 -- 172.21.15.83:0/2695576377 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566da31a920 con 0x5566da3fd350 2024-04-24T09:09:14.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.343+0000 7f7226a00640 1 --2- 172.21.15.83:0/2695576377 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5566da3fd350 0x5566da404e70 secure :-1 s=READY pgs=2345 cs=0 l=1 rev1=1 crypto rx=0x7f7214005850 tx=0x7f7214005400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9cfb7efa12fe4d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:14.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.343+0000 7f7210c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:09:14.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.343+0000 7f7210c00640 1 -- 172.21.15.83:0/2695576377 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f721402c060 con 0x5566da3fd350 2024-04-24T09:09:14.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.343+0000 7f7210c00640 1 -- 172.21.15.83:0/2695576377 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9583+0+0 (secure 0 0 0) 0x7f7214004450 con 0x5566da3fd350 2024-04-24T09:09:14.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.344+0000 7f7230df0740 1 -- 172.21.15.83:0/2695576377 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7214005020 con 0x5566da3fd350 2024-04-24T09:09:14.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.344+0000 7f7210c00640 1 -- 172.21.15.83:0/2695576377 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f7214033050 con 0x5566da3fd350 2024-04-24T09:09:14.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.344+0000 7f7230df0740 10 client.21684 fetch_fsmap learned FSMap version 58 2024-04-24T09:09:14.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.344+0000 7f7230df0740 1 -- 172.21.15.83:0/2695576377 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x5566da13e7d0 con 0x5566da3fd350 2024-04-24T09:09:14.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.344+0000 7f7210c00640 1 -- 172.21.15.83:0/2695576377 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7214005020 con 0x5566da3fd350 2024-04-24T09:09:14.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.345+0000 7f7210c00640 1 -- 172.21.15.83:0/2695576377 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 58) ==== 8374+0+0 (secure 0 0 0) 0x7f72140048d0 con 0x5566da3fd350 2024-04-24T09:09:14.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.345+0000 7f7230df0740 10 client.21684 fetch_fsmap finished waiting for FSMap version 58 2024-04-24T09:09:14.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.345+0000 7f7230df0740 10 client.21684 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:09:14.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.345+0000 7f7230df0740 20 client.21684 populate_metadata read hostname 'smithi083' 2024-04-24T09:09:14.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.345+0000 7f7230df0740 1 --2- 172.21.15.83:0/2695576377 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5566da428c50 0x5566da42b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:14.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.345+0000 7f7230df0740 4 client.21684 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:09:14.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.345+0000 7f7230df0740 1 -- 172.21.15.83:0/2695576377 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5566da36fc10 con 0x5566da428c50 2024-04-24T09:09:14.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.345+0000 7f7227400640 1 --2- 172.21.15.83:0/2695576377 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5566da428c50 0x5566da42b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:14.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.346+0000 7f7227400640 1 --2- 172.21.15.83:0/2695576377 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5566da428c50 0x5566da42b040 crc :-1 s=READY pgs=441 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=614e74eb4683b8fc server_cookie=b236acfcfe563c20 in_seq=0 out_seq=0 2024-04-24T09:09:14.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.346+0000 7f7210c00640 10 client.21684 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:09:14.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.347+0000 7f7210c00640 1 -- 172.21.15.83:0/2695576377 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5566da36fc10 con 0x5566da428c50 2024-04-24T09:09:14.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.347+0000 7f7210c00640 10 client.21684 handle_command_reply: tid=0 2024-04-24T09:09:14.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.347+0000 7f7230df0740 1 client.21684 shutdown 2024-04-24T09:09:14.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.347+0000 7f7230df0740 1 -- 172.21.15.83:0/2695576377 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5566da3fd350 msgr2=0x5566da404e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:14.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.347+0000 7f7230df0740 1 --2- 172.21.15.83:0/2695576377 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5566da3fd350 0x5566da404e70 secure :-1 s=READY pgs=2345 cs=0 l=1 rev1=1 crypto rx=0x7f7214005850 tx=0x7f7214005400 comp rx=0 tx=0).stop 2024-04-24T09:09:14.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.347+0000 7f7230df0740 1 -- 172.21.15.83:0/2695576377 shutdown_connections 2024-04-24T09:09:14.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.347+0000 7f7230df0740 1 --2- 172.21.15.83:0/2695576377 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5566da428c50 0x5566da42b040 crc :-1 s=READY pgs=441 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:14.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.347+0000 7f7230df0740 1 --2- 172.21.15.83:0/2695576377 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5566da407a20 0x5566da4053b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:14.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.347+0000 7f7230df0740 1 --2- 172.21.15.83:0/2695576377 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5566da3fd350 0x5566da404e70 unknown :-1 s=CLOSED pgs=2345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:14.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.347+0000 7f7210c00640 0 client.21684 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:09:14.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.347+0000 7f7230df0740 1 --2- 172.21.15.83:0/2695576377 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5566da3fca30 0x5566da404930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:14.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.347+0000 7f7230df0740 1 -- 172.21.15.83:0/2695576377 >> 172.21.15.83:0/2695576377 conn(0x5566da3f8d50 msgr2=0x5566da3f7fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:14.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.348+0000 7f7230df0740 1 -- 172.21.15.83:0/2695576377 shutdown_connections 2024-04-24T09:09:14.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.348+0000 7f7230df0740 1 -- 172.21.15.83:0/2695576377 wait complete. 2024-04-24T09:09:14.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.348+0000 7f7230df0740 20 client.21684 trim_cache size 0 max 16384 2024-04-24T09:09:14.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.382+0000 7f7230df0740 1 Processor -- start 2024-04-24T09:09:14.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.382+0000 7f7230df0740 1 -- start start 2024-04-24T09:09:14.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.383+0000 7f7230df0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5566da3fca30 0x5566da424770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:14.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.383+0000 7f7230df0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5566da3fd350 0x5566da4308b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:14.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.383+0000 7f7230df0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5566da407a20 0x5566da430df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:14.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.383+0000 7f7230df0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7208010110 con 0x5566da407a20 2024-04-24T09:09:14.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.383+0000 7f7230df0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7208011c50 con 0x5566da3fd350 2024-04-24T09:09:14.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.383+0000 7f7230df0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f72080107e0 con 0x5566da3fca30 2024-04-24T09:09:14.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.383+0000 7f7227e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5566da407a20 0x5566da430df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:14.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.383+0000 7f7227400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5566da3fca30 0x5566da424770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:14.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.383+0000 7f7227e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5566da407a20 0x5566da430df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60150/0 (socket says 172.21.15.83:60150) 2024-04-24T09:09:14.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.383+0000 7f7227e00640 1 -- 172.21.15.83:0/1240363773 learned_addr learned my addr 172.21.15.83:0/1240363773 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:09:14.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.383+0000 7f7226a00640 1 --2- 172.21.15.83:0/1240363773 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5566da3fd350 0x5566da4308b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:14.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.384+0000 7f7227e00640 1 -- 172.21.15.83:0/1240363773 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5566da3fca30 msgr2=0x5566da424770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:14.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.384+0000 7f7227e00640 1 --2- 172.21.15.83:0/1240363773 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5566da3fca30 0x5566da424770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:14.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.384+0000 7f7227e00640 1 -- 172.21.15.83:0/1240363773 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5566da3fd350 msgr2=0x5566da4308b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:14.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.384+0000 7f7227e00640 1 --2- 172.21.15.83:0/1240363773 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5566da3fd350 0x5566da4308b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:14.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.384+0000 7f7227e00640 1 -- 172.21.15.83:0/1240363773 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566da0743f0 con 0x5566da407a20 2024-04-24T09:09:14.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.384+0000 7f7226a00640 1 --2- 172.21.15.83:0/1240363773 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5566da3fd350 0x5566da4308b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:09:14.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.384+0000 7f7227e00640 1 --2- 172.21.15.83:0/1240363773 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5566da407a20 0x5566da430df0 secure :-1 s=READY pgs=2346 cs=0 l=1 rev1=1 crypto rx=0x7f722000c760 tx=0x7f722009cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cbf67b1e164c0a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:14.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.385+0000 7f7213400640 1 -- 172.21.15.83:0/1240363773 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f722009af60 con 0x5566da407a20 2024-04-24T09:09:14.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.385+0000 7f7213400640 1 -- 172.21.15.83:0/1240363773 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7220004d30 con 0x5566da407a20 2024-04-24T09:09:14.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.385+0000 7f7213400640 1 -- 172.21.15.83:0/1240363773 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72200bd070 con 0x5566da407a20 2024-04-24T09:09:14.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.385+0000 7f7230df0740 1 -- 172.21.15.83:0/1240363773 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5566da407a20 msgr2=0x5566da430df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:14.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.386+0000 7f7230df0740 1 --2- 172.21.15.83:0/1240363773 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5566da407a20 0x5566da430df0 secure :-1 s=READY pgs=2346 cs=0 l=1 rev1=1 crypto rx=0x7f722000c760 tx=0x7f722009cd20 comp rx=0 tx=0).stop 2024-04-24T09:09:14.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.386+0000 7f7230df0740 1 -- 172.21.15.83:0/1240363773 shutdown_connections 2024-04-24T09:09:14.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.386+0000 7f7230df0740 1 --2- 172.21.15.83:0/1240363773 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5566da407a20 0x5566da430df0 unknown :-1 s=CLOSED pgs=2346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:14.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.386+0000 7f7230df0740 1 --2- 172.21.15.83:0/1240363773 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5566da3fd350 0x5566da4308b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:14.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.386+0000 7f7230df0740 1 --2- 172.21.15.83:0/1240363773 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5566da3fca30 0x5566da424770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:14.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.386+0000 7f7230df0740 1 -- 172.21.15.83:0/1240363773 >> 172.21.15.83:0/1240363773 conn(0x5566da3f8d50 msgr2=0x5566da405180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:14.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.386+0000 7f7230df0740 1 -- 172.21.15.83:0/1240363773 shutdown_connections 2024-04-24T09:09:14.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.386+0000 7f7230df0740 1 -- 172.21.15.83:0/1240363773 wait complete. 2024-04-24T09:09:14.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.387+0000 7f7230df0740 1 -- start start 2024-04-24T09:09:14.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.387+0000 7f7230df0740 1 Processor -- start 2024-04-24T09:09:14.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.387+0000 7f7230df0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5566da3fca30 0x5566da421160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:14.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.387+0000 7f7230df0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5566da3fd350 0x5566da4013b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:14.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.388+0000 7f7230df0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5566da407a20 0x5566da401930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:14.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.388+0000 7f7230df0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7208011580 con 0x5566da3fca30 2024-04-24T09:09:14.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.388+0000 7f7230df0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7208010eb0 con 0x5566da3fd350 2024-04-24T09:09:14.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.388+0000 7f7230df0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f720800e110 con 0x5566da407a20 2024-04-24T09:09:14.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.388+0000 7f7230df0740 10 client.0 _ll_register_callbacks cb 0x7f721400be80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:09:14.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.388+0000 7f7227400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5566da3fca30 0x5566da421160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:14.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.388+0000 7f7227400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5566da3fca30 0x5566da421160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60166/0 (socket says 172.21.15.83:60166) 2024-04-24T09:09:14.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.388+0000 7f7227400640 1 -- 172.21.15.83:0/523417886 learned_addr learned my addr 172.21.15.83:0/523417886 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:09:14.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.388+0000 7f7226a00640 1 --2- 172.21.15.83:0/523417886 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5566da3fd350 0x5566da4013b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:14.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.389+0000 7f7226a00640 1 -- 172.21.15.83:0/523417886 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5566da407a20 msgr2=0x5566da401930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:14.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.389+0000 7f7226a00640 1 --2- 172.21.15.83:0/523417886 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5566da407a20 0x5566da401930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:14.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.389+0000 7f7226a00640 1 -- 172.21.15.83:0/523417886 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5566da3fca30 msgr2=0x5566da421160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:14.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.389+0000 7f7226a00640 1 --2- 172.21.15.83:0/523417886 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5566da3fca30 0x5566da421160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:14.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.389+0000 7f7226a00640 1 -- 172.21.15.83:0/523417886 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566da0340f0 con 0x5566da3fd350 2024-04-24T09:09:14.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.389+0000 7f7227400640 1 --2- 172.21.15.83:0/523417886 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5566da3fca30 0x5566da421160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:09:14.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.389+0000 7f7226a00640 1 --2- 172.21.15.83:0/523417886 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5566da3fd350 0x5566da4013b0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7214003610 tx=0x7f72140050d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6da2a807ed2d365 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:14.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.389+0000 7f7210c00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:09:14.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.389+0000 7f7210c00640 1 -- 172.21.15.83:0/523417886 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f721402c040 con 0x5566da3fd350 2024-04-24T09:09:14.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.389+0000 7f7210c00640 1 -- 172.21.15.83:0/523417886 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9583+0+0 (secure 0 0 0) 0x7f7214003a70 con 0x5566da3fd350 2024-04-24T09:09:14.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.389+0000 7f7230df0740 1 -- 172.21.15.83:0/523417886 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7220004d30 con 0x5566da3fd350 2024-04-24T09:09:14.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.390+0000 7f7210c00640 1 -- 172.21.15.83:0/523417886 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f72140056b0 con 0x5566da3fd350 2024-04-24T09:09:14.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.390+0000 7f7230df0740 10 client.38825 fetch_fsmap learned FSMap version 58 2024-04-24T09:09:14.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.390+0000 7f7230df0740 1 -- 172.21.15.83:0/523417886 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x5566da327810 con 0x5566da3fd350 2024-04-24T09:09:14.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.390+0000 7f7210c00640 1 -- 172.21.15.83:0/523417886 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7220004d30 con 0x5566da3fd350 2024-04-24T09:09:14.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.391+0000 7f7210c00640 1 -- 172.21.15.83:0/523417886 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 58) ==== 8374+0+0 (secure 0 0 0) 0x7f7214012ce0 con 0x5566da3fd350 2024-04-24T09:09:14.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.391+0000 7f7230df0740 10 client.38825 fetch_fsmap finished waiting for FSMap version 58 2024-04-24T09:09:14.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.391+0000 7f7230df0740 10 client.38825 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:09:14.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.391+0000 7f7230df0740 20 client.38825 populate_metadata read hostname 'smithi083' 2024-04-24T09:09:14.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.391+0000 7f7230df0740 1 --2- 172.21.15.83:0/523417886 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5566da48f550 0x5566da491970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:14.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.391+0000 7f7230df0740 4 client.38825 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T09:09:14.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.391+0000 7f7230df0740 1 -- 172.21.15.83:0/523417886 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5566da369a80 con 0x5566da48f550 2024-04-24T09:09:14.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.391+0000 7f7227400640 1 --2- 172.21.15.83:0/523417886 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5566da48f550 0x5566da491970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:14.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.392+0000 7f7210c00640 10 client.38825 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:09:14.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.393+0000 7f7227400640 1 --2- 172.21.15.83:0/523417886 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5566da48f550 0x5566da491970 crc :-1 s=READY pgs=442 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e78a4a1131e3db8d server_cookie=def50e66d93853f6 in_seq=0 out_seq=0 2024-04-24T09:09:14.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.393+0000 7f7210c00640 1 -- 172.21.15.83:0/523417886 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5566da369a80 con 0x5566da48f550 2024-04-24T09:09:14.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.393+0000 7f7210c00640 10 client.38825 handle_command_reply: tid=0 2024-04-24T09:09:14.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.393+0000 7f7230df0740 1 client.38825 shutdown 2024-04-24T09:09:14.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.393+0000 7f7230df0740 1 -- 172.21.15.83:0/523417886 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5566da3fd350 msgr2=0x5566da4013b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:14.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.393+0000 7f7230df0740 1 --2- 172.21.15.83:0/523417886 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5566da3fd350 0x5566da4013b0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7214003610 tx=0x7f72140050d0 comp rx=0 tx=0).stop 2024-04-24T09:09:14.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.393+0000 7f7230df0740 1 -- 172.21.15.83:0/523417886 shutdown_connections 2024-04-24T09:09:14.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.393+0000 7f7230df0740 1 --2- 172.21.15.83:0/523417886 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5566da48f550 0x5566da491970 crc :-1 s=READY pgs=442 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:14.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.393+0000 7f7230df0740 1 --2- 172.21.15.83:0/523417886 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5566da407a20 0x5566da401930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:14.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.393+0000 7f7230df0740 1 --2- 172.21.15.83:0/523417886 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5566da3fd350 0x5566da4013b0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:14.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.393+0000 7f7230df0740 1 --2- 172.21.15.83:0/523417886 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5566da3fca30 0x5566da421160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:14.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.393+0000 7f7210c00640 0 client.38825 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:09:14.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.393+0000 7f7230df0740 1 -- 172.21.15.83:0/523417886 >> 172.21.15.83:0/523417886 conn(0x5566da3f8d50 msgr2=0x5566da3fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:14.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.394+0000 7f7230df0740 1 -- 172.21.15.83:0/523417886 shutdown_connections 2024-04-24T09:09:14.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.394+0000 7f7230df0740 1 -- 172.21.15.83:0/523417886 wait complete. 2024-04-24T09:09:14.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.394+0000 7f7230df0740 20 client.38825 trim_cache size 0 max 16384 2024-04-24T09:09:14.395 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T09:09:14.396 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-04-24T09:09:14.396 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-04-24T09:09:14.396 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T09:09:14.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.396+0000 7f722ca00640 1 -- 172.21.15.83:0/550951894 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f720c0a3da0 msgr2=0x7f720c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:14.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.396+0000 7f722ca00640 1 --2- 172.21.15.83:0/550951894 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f720c0a3da0 0x7f720c0a6260 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x5566da3fdff0 tx=0x7f7218005a20 comp rx=0 tx=0).stop 2024-04-24T09:09:14.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.396+0000 7f722ca00640 1 -- 172.21.15.83:0/550951894 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f722819a6d0 msgr2=0x7f722819ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:14.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.396+0000 7f722ca00640 1 --2- 172.21.15.83:0/550951894 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f722819a6d0 0x7f722819ab50 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f7220009fd0 tx=0x7f722000ef10 comp rx=0 tx=0).stop 2024-04-24T09:09:14.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.397+0000 7f722ca00640 1 -- 172.21.15.83:0/550951894 shutdown_connections 2024-04-24T09:09:14.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.397+0000 7f722ca00640 1 --2- 172.21.15.83:0/550951894 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f720c0a3da0 0x7f720c0a6260 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:14.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.397+0000 7f722ca00640 1 --2- 172.21.15.83:0/550951894 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f722819a6d0 0x7f722819ab50 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:14.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.397+0000 7f722ca00640 1 --2- 172.21.15.83:0/550951894 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f72281071e0 0x7f72281a0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:14.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.397+0000 7f722ca00640 1 --2- 172.21.15.83:0/550951894 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7228102110 0x7f72281a0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:14.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.397+0000 7f722ca00640 1 -- 172.21.15.83:0/550951894 >> 172.21.15.83:0/550951894 conn(0x7f72280fd9d0 msgr2=0x7f72280fed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:14.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.397+0000 7f722ca00640 1 -- 172.21.15.83:0/550951894 shutdown_connections 2024-04-24T09:09:14.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.397+0000 7f722ca00640 1 -- 172.21.15.83:0/550951894 wait complete. 2024-04-24T09:09:14.418 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T09:09:14.419 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T09:09:14.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.661+0000 7f063cc00640 1 -- 172.21.15.83:0/4120847142 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0638074d30 msgr2=0x7f063810e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:14.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.661+0000 7f063cc00640 1 --2- 172.21.15.83:0/4120847142 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0638074d30 0x7f063810e2f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f0620009900 tx=0x7f062002a480 comp rx=0 tx=0).stop 2024-04-24T09:09:14.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.662+0000 7f063cc00640 1 -- 172.21.15.83:0/4120847142 shutdown_connections 2024-04-24T09:09:14.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.662+0000 7f063cc00640 1 --2- 172.21.15.83:0/4120847142 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f063810e830 0x7f0638110c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:14.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.662+0000 7f063cc00640 1 --2- 172.21.15.83:0/4120847142 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0638074d30 0x7f063810e2f0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:14.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.662+0000 7f063cc00640 1 --2- 172.21.15.83:0/4120847142 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0638074410 0x7f06380747f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:14.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.662+0000 7f063cc00640 1 -- 172.21.15.83:0/4120847142 >> 172.21.15.83:0/4120847142 conn(0x7f06380793c0 msgr2=0x7f06380797d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:14.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.662+0000 7f063cc00640 1 -- 172.21.15.83:0/4120847142 shutdown_connections 2024-04-24T09:09:14.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.662+0000 7f063cc00640 1 -- 172.21.15.83:0/4120847142 wait complete. 2024-04-24T09:09:14.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.663+0000 7f063cc00640 1 Processor -- start 2024-04-24T09:09:14.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.663+0000 7f063cc00640 1 -- start start 2024-04-24T09:09:14.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.664+0000 7f063cc00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0638074410 0x7f06381a06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:14.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.664+0000 7f063cc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0638074d30 0x7f06381a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:14.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.664+0000 7f063cc00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f063810e830 0x7f063819a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:14.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.664+0000 7f063cc00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f06381144b0 con 0x7f0638074d30 2024-04-24T09:09:14.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.664+0000 7f063cc00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0638114330 con 0x7f0638074410 2024-04-24T09:09:14.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.664+0000 7f063cc00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0638114630 con 0x7f063810e830 2024-04-24T09:09:14.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.664+0000 7f0635e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0638074410 0x7f06381a06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:14.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.664+0000 7f0635e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0638074410 0x7f06381a06e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:52972/0 (socket says 172.21.15.83:52972) 2024-04-24T09:09:14.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.664+0000 7f0635e00640 1 -- 172.21.15.83:0/3011951614 learned_addr learned my addr 172.21.15.83:0/3011951614 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:09:14.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.665+0000 7f062fe00640 1 --2- 172.21.15.83:0/3011951614 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0638074d30 0x7f06381a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:14.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.665+0000 7f0636800640 1 --2- 172.21.15.83:0/3011951614 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f063810e830 0x7f063819a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:14.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.665+0000 7f0635e00640 1 -- 172.21.15.83:0/3011951614 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f063810e830 msgr2=0x7f063819a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:14.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.665+0000 7f0635e00640 1 --2- 172.21.15.83:0/3011951614 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f063810e830 0x7f063819a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:14.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.665+0000 7f0635e00640 1 -- 172.21.15.83:0/3011951614 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0638074d30 msgr2=0x7f06381a0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:14.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.665+0000 7f0635e00640 1 --2- 172.21.15.83:0/3011951614 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0638074d30 0x7f06381a0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:14.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.665+0000 7f0635e00640 1 -- 172.21.15.83:0/3011951614 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f063819b010 con 0x7f0638074410 2024-04-24T09:09:14.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.666+0000 7f0635e00640 1 --2- 172.21.15.83:0/3011951614 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0638074410 0x7f06381a06e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f062800ea60 tx=0x7f062800ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:14.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.666+0000 7f062ea00640 1 -- 172.21.15.83:0/3011951614 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f062800cde0 con 0x7f0638074410 2024-04-24T09:09:14.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.666+0000 7f063cc00640 1 -- 172.21.15.83:0/3011951614 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f063819b300 con 0x7f0638074410 2024-04-24T09:09:14.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.666+0000 7f062ea00640 1 -- 172.21.15.83:0/3011951614 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f06280035f0 con 0x7f0638074410 2024-04-24T09:09:14.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.666+0000 7f062ea00640 1 -- 172.21.15.83:0/3011951614 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0628010690 con 0x7f0638074410 2024-04-24T09:09:14.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.666+0000 7f063cc00640 1 -- 172.21.15.83:0/3011951614 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06381a75a0 con 0x7f0638074410 2024-04-24T09:09:14.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.668+0000 7f063cc00640 1 -- 172.21.15.83:0/3011951614 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0638069270 con 0x7f0638074410 2024-04-24T09:09:14.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.672+0000 7f062ea00640 1 -- 172.21.15.83:0/3011951614 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f06280037a0 con 0x7f0638074410 2024-04-24T09:09:14.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.674+0000 7f062ea00640 1 --2- 172.21.15.83:0/3011951614 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f061c0a3de0 0x7f061c0a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:14.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.675+0000 7f062ea00640 1 -- 172.21.15.83:0/3011951614 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 9583+0+0 (secure 0 0 0) 0x7f06280d0ec0 con 0x7f0638074410 2024-04-24T09:09:14.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.675+0000 7f062fe00640 1 --2- 172.21.15.83:0/3011951614 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f061c0a3de0 0x7f061c0a62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:14.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.676+0000 7f062fe00640 1 --2- 172.21.15.83:0/3011951614 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f061c0a3de0 0x7f061c0a62a0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f0620005ab0 tx=0x7f0620000bc0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:14.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:14.679+0000 7f062ea00640 1 -- 172.21.15.83:0/3011951614 <== mon.2 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f062809d0e0 con 0x7f0638074410 2024-04-24T09:09:15.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.084+0000 7f063cc00640 1 -- 172.21.15.83:0/3011951614 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0638109a10 con 0x7f0638074410 2024-04-24T09:09:15.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.085+0000 7f062ea00640 1 -- 172.21.15.83:0/3011951614 <== mon.2 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) ==== 94+0+11949 (secure 0 0 0) 0x7f062809c9e0 con 0x7f0638074410 2024-04-24T09:09:15.085 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T09:09:15.085 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":58,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/358872257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":358872257},{"type":"v1","addr":"172.21.15.83:6835","nonce":358872257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3157070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3157070851},{"type":"v1","addr":"172.21.15.83:6837","nonce":3157070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":58,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T08:48:29.422576+0000","modified":"2024-04-24T09:08:27.108181+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34520,"mds_1":34493,"mds_2":34451,"mds_3":34427,"mds_4":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14658":{"gid":14658,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6839/4085470844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":4085470844},{"type":"v1","addr":"172.21.15.83:6839","nonce":4085470844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14676":{"gid":14676,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6841/2634754210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":2634754210},{"type":"v1","addr":"172.21.15.83:6841","nonce":2634754210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24382":{"gid":24382,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/1859921310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1859921310},{"type":"v1","addr":"172.21.15.189:6833","nonce":1859921310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/2241923197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2241923197},{"type":"v1","addr":"172.21.15.189:6835","nonce":2241923197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.189:6839/2548965336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":2548965336},{"type":"v1","addr":"172.21.15.189:6839","nonce":2548965336}]},"join_fscid":-1,"export_targets":[0,1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26686":{"gid":26686,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6837/1678817161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":1678817161},{"type":"v1","addr":"172.21.15.189:6837","nonce":1678817161}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.138:6833/1157967658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1157967658},{"type":"v1","addr":"172.21.15.138:6833","nonce":1157967658}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.138:6835/1909219758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1909219758},{"type":"v1","addr":"172.21.15.138:6835","nonce":1909219758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34493":{"gid":34493,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.138:6837/900566754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":900566754},{"type":"v1","addr":"172.21.15.138:6837","nonce":900566754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34520":{"gid":34520,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.138:6839/2236932393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2236932393},{"type":"v1","addr":"172.21.15.138:6839","nonce":2236932393}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34520,"qdb_cluster":[34493,34451,34427,24454,34520]},"id":1}]} 2024-04-24T09:09:15.085 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 58 2024-04-24T09:09:15.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.088+0000 7f063cc00640 1 -- 172.21.15.83:0/3011951614 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f061c0a3de0 msgr2=0x7f061c0a62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:15.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.088+0000 7f063cc00640 1 --2- 172.21.15.83:0/3011951614 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f061c0a3de0 0x7f061c0a62a0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f0620005ab0 tx=0x7f0620000bc0 comp rx=0 tx=0).stop 2024-04-24T09:09:15.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.088+0000 7f063cc00640 1 -- 172.21.15.83:0/3011951614 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0638074410 msgr2=0x7f06381a06e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:15.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.088+0000 7f063cc00640 1 --2- 172.21.15.83:0/3011951614 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0638074410 0x7f06381a06e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f062800ea60 tx=0x7f062800ef20 comp rx=0 tx=0).stop 2024-04-24T09:09:15.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.088+0000 7f063cc00640 1 -- 172.21.15.83:0/3011951614 shutdown_connections 2024-04-24T09:09:15.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.088+0000 7f063cc00640 1 --2- 172.21.15.83:0/3011951614 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f061c0a3de0 0x7f061c0a62a0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:15.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.088+0000 7f063cc00640 1 --2- 172.21.15.83:0/3011951614 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f063810e830 0x7f063819a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:15.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.088+0000 7f063cc00640 1 --2- 172.21.15.83:0/3011951614 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0638074d30 0x7f06381a0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:15.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.088+0000 7f063cc00640 1 --2- 172.21.15.83:0/3011951614 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0638074410 0x7f06381a06e0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:15.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.088+0000 7f063cc00640 1 -- 172.21.15.83:0/3011951614 >> 172.21.15.83:0/3011951614 conn(0x7f06380793c0 msgr2=0x7f06380fefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:15.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.088+0000 7f063cc00640 1 -- 172.21.15.83:0/3011951614 shutdown_connections 2024-04-24T09:09:15.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.088+0000 7f063cc00640 1 -- 172.21.15.83:0/3011951614 wait complete. 2024-04-24T09:09:15.109 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-24T09:09:15.169 INFO:tasks.workunit.client.0.smithi083.stdout: 10 29064 6.91 MB/sec execute 106 sec latency 378.049 ms 2024-04-24T09:09:15.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.369+0000 7f2c0fe00640 1 -- 172.21.15.83:0/2932162437 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c10101770 msgr2=0x7f2c10105c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:15.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.369+0000 7f2c0fe00640 1 --2- 172.21.15.83:0/2932162437 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c10101770 0x7f2c10105c30 secure :-1 s=READY pgs=2347 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc009900 tx=0x7f2bfc02a490 comp rx=0 tx=0).stop 2024-04-24T09:09:15.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.369+0000 7f2c0fe00640 1 -- 172.21.15.83:0/2932162437 shutdown_connections 2024-04-24T09:09:15.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.369+0000 7f2c0fe00640 1 --2- 172.21.15.83:0/2932162437 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2c10106250 0x7f2c101066e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:15.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.369+0000 7f2c0fe00640 1 --2- 172.21.15.83:0/2932162437 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c10101770 0x7f2c10105c30 unknown :-1 s=CLOSED pgs=2347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:15.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.369+0000 7f2c0fe00640 1 --2- 172.21.15.83:0/2932162437 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c10100e50 0x7f2c10101230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:15.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.369+0000 7f2c0fe00640 1 -- 172.21.15.83:0/2932162437 >> 172.21.15.83:0/2932162437 conn(0x7f2c100fc880 msgr2=0x7f2c100feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:15.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.369+0000 7f2c0fe00640 1 -- 172.21.15.83:0/2932162437 shutdown_connections 2024-04-24T09:09:15.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.369+0000 7f2c0fe00640 1 -- 172.21.15.83:0/2932162437 wait complete. 2024-04-24T09:09:15.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.370+0000 7f2c0fe00640 1 Processor -- start 2024-04-24T09:09:15.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.371+0000 7f2c0fe00640 1 -- start start 2024-04-24T09:09:15.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.371+0000 7f2c0fe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2c10100e50 0x7f2c101a4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:15.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.371+0000 7f2c0fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c10101770 0x7f2c101a4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:15.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.371+0000 7f2c0fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c10106250 0x7f2c1019eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:15.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.373+0000 7f2c0e000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c10101770 0x7f2c101a4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:15.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.373+0000 7f2c0e000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c10101770 0x7f2c101a4e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60218/0 (socket says 172.21.15.83:60218) 2024-04-24T09:09:15.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.373+0000 7f2c0e000640 1 -- 172.21.15.83:0/1344110609 learned_addr learned my addr 172.21.15.83:0/1344110609 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:09:15.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.373+0000 7f2c0fe00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2c10118750 con 0x7f2c10101770 2024-04-24T09:09:15.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.373+0000 7f2c0fe00640 1 -- 172.21.15.83:0/1344110609 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2c101185d0 con 0x7f2c10106250 2024-04-24T09:09:15.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.373+0000 7f2c0fe00640 1 -- 172.21.15.83:0/1344110609 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2c101188d0 con 0x7f2c10100e50 2024-04-24T09:09:15.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.373+0000 7f2c0ea00640 1 --2- 172.21.15.83:0/1344110609 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2c10100e50 0x7f2c101a4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:15.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.373+0000 7f2c0f400640 1 --2- 172.21.15.83:0/1344110609 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c10106250 0x7f2c1019eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:15.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.373+0000 7f2c0f400640 1 -- 172.21.15.83:0/1344110609 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2c10100e50 msgr2=0x7f2c101a4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:15.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.373+0000 7f2c0f400640 1 --2- 172.21.15.83:0/1344110609 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2c10100e50 0x7f2c101a4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:15.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.373+0000 7f2c0f400640 1 -- 172.21.15.83:0/1344110609 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c10101770 msgr2=0x7f2c101a4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:15.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.373+0000 7f2c0f400640 1 --2- 172.21.15.83:0/1344110609 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c10101770 0x7f2c101a4e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:15.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.374+0000 7f2c0f400640 1 -- 172.21.15.83:0/1344110609 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c1019f230 con 0x7f2c10106250 2024-04-24T09:09:15.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.374+0000 7f2c0e000640 1 --2- 172.21.15.83:0/1344110609 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c10101770 0x7f2c101a4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:09:15.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.374+0000 7f2c0f400640 1 --2- 172.21.15.83:0/1344110609 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c10106250 0x7f2c1019eab0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f2c0800e940 tx=0x7f2c0800ee00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:15.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.374+0000 7f2c07400640 1 -- 172.21.15.83:0/1344110609 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c08004d20 con 0x7f2c10106250 2024-04-24T09:09:15.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.374+0000 7f2c0fe00640 1 -- 172.21.15.83:0/1344110609 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c1019f520 con 0x7f2c10106250 2024-04-24T09:09:15.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.375+0000 7f2c0fe00640 1 -- 172.21.15.83:0/1344110609 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c101ab7c0 con 0x7f2c10106250 2024-04-24T09:09:15.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.375+0000 7f2c07400640 1 -- 172.21.15.83:0/1344110609 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2c08005020 con 0x7f2c10106250 2024-04-24T09:09:15.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.375+0000 7f2c07400640 1 -- 172.21.15.83:0/1344110609 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c080116d0 con 0x7f2c10106250 2024-04-24T09:09:15.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.380+0000 7f2c18a31740 1 Processor -- start 2024-04-24T09:09:15.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.380+0000 7f2c07400640 1 -- 172.21.15.83:0/1344110609 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2c0800cb20 con 0x7f2c10106250 2024-04-24T09:09:15.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.381+0000 7f2c18a31740 1 -- start start 2024-04-24T09:09:15.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.381+0000 7f2c18a31740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558abdc75b40 0x558abdc75f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:15.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.381+0000 7f2c18a31740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558abdc76460 0x558abdc805f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:15.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.381+0000 7f2c18a31740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558abdc80b30 0x558abdc82f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:15.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.381+0000 7f2c18a31740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x558abdbdb1d0 con 0x558abdc76460 2024-04-24T09:09:15.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.381+0000 7f2c18a31740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558abdb81450 con 0x558abdc80b30 2024-04-24T09:09:15.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.381+0000 7f2c18a31740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x558abdbd7190 con 0x558abdc75b40 2024-04-24T09:09:15.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.382+0000 7f2c0ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558abdc76460 0x558abdc805f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:15.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.382+0000 7f2c0e000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558abdc80b30 0x558abdc82f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:15.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.382+0000 7f2c0e000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558abdc75b40 0x558abdc75f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:15.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.382+0000 7f2c0ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558abdc76460 0x558abdc805f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60226/0 (socket says 172.21.15.83:60226) 2024-04-24T09:09:15.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.382+0000 7f2c0ea00640 1 -- 172.21.15.83:0/2082273279 learned_addr learned my addr 172.21.15.83:0/2082273279 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:09:15.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.382+0000 7f2c07400640 1 --2- 172.21.15.83:0/1344110609 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f2bf40a3e20 0x7f2bf40a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:15.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.382+0000 7f2c07400640 1 -- 172.21.15.83:0/1344110609 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 9583+0+0 (secure 0 0 0) 0x7f2c080d0b00 con 0x7f2c10106250 2024-04-24T09:09:15.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.382+0000 7f2c0ea00640 1 -- 172.21.15.83:0/2082273279 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558abdc75b40 msgr2=0x558abdc75f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:15.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.382+0000 7f2c0ea00640 1 --2- 172.21.15.83:0/2082273279 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558abdc75b40 0x558abdc75f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:15.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.382+0000 7f2c0ea00640 1 -- 172.21.15.83:0/2082273279 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558abdc80b30 msgr2=0x558abdc82f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:15.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.382+0000 7f2c0ea00640 1 --2- 172.21.15.83:0/2082273279 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558abdc80b30 0x558abdc82f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:15.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.382+0000 7f2c0ea00640 1 -- 172.21.15.83:0/2082273279 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558abdc3e310 con 0x558abdc76460 2024-04-24T09:09:15.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.383+0000 7f2c0f400640 1 --2- 172.21.15.83:0/1344110609 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f2bf40a3e20 0x7f2bf40a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:15.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.383+0000 7f2c0ea00640 1 --2- 172.21.15.83:0/2082273279 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558abdc76460 0x558abdc805f0 secure :-1 s=READY pgs=2348 cs=0 l=1 rev1=1 crypto rx=0x7f2c00009c90 tx=0x7f2c00011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f059b11f5fac354 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:15.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.384+0000 7f2bfbe00640 1 -- 172.21.15.83:0/2082273279 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c00005220 con 0x558abdc76460 2024-04-24T09:09:15.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.384+0000 7f2c0f400640 1 --2- 172.21.15.83:0/1344110609 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f2bf40a3e20 0x7f2bf40a62e0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f2c080a4d60 tx=0x7f2c0800ef30 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:15.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.384+0000 7f2bfbe00640 1 -- 172.21.15.83:0/2082273279 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2c00004c50 con 0x558abdc76460 2024-04-24T09:09:15.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.384+0000 7f2bfbe00640 1 -- 172.21.15.83:0/2082273279 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c00019670 con 0x558abdc76460 2024-04-24T09:09:15.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.385+0000 7f2c18a31740 1 -- 172.21.15.83:0/2082273279 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558abdc76460 msgr2=0x558abdc805f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:15.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.385+0000 7f2c18a31740 1 --2- 172.21.15.83:0/2082273279 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558abdc76460 0x558abdc805f0 secure :-1 s=READY pgs=2348 cs=0 l=1 rev1=1 crypto rx=0x7f2c00009c90 tx=0x7f2c00011810 comp rx=0 tx=0).stop 2024-04-24T09:09:15.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.385+0000 7f2c18a31740 1 -- 172.21.15.83:0/2082273279 shutdown_connections 2024-04-24T09:09:15.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.385+0000 7f2c18a31740 1 --2- 172.21.15.83:0/2082273279 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558abdc80b30 0x558abdc82f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:15.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.385+0000 7f2c18a31740 1 --2- 172.21.15.83:0/2082273279 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558abdc76460 0x558abdc805f0 unknown :-1 s=CLOSED pgs=2348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:15.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.385+0000 7f2c18a31740 1 --2- 172.21.15.83:0/2082273279 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558abdc75b40 0x558abdc75f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:15.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.385+0000 7f2c18a31740 1 -- 172.21.15.83:0/2082273279 >> 172.21.15.83:0/2082273279 conn(0x558abdc71e60 msgr2=0x558abdc72240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:15.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.385+0000 7f2c18a31740 1 -- 172.21.15.83:0/2082273279 shutdown_connections 2024-04-24T09:09:15.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.385+0000 7f2c18a31740 1 -- 172.21.15.83:0/2082273279 wait complete. 2024-04-24T09:09:15.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.386+0000 7f2c18a31740 1 -- start start 2024-04-24T09:09:15.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.386+0000 7f2c18a31740 1 Processor -- start 2024-04-24T09:09:15.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.387+0000 7f2c18a31740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558abdc75b40 0x558abdc80560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:15.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.387+0000 7f2c18a31740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558abdc76460 0x558abdc7d850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:15.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.387+0000 7f2c18a31740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558abdc80b30 0x558abdc7df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:15.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.387+0000 7f2c18a31740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x558abdc839c0 con 0x558abdc80b30 2024-04-24T09:09:15.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.387+0000 7f2c18a31740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558abdb94810 con 0x558abdc75b40 2024-04-24T09:09:15.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.387+0000 7f2c18a31740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x558abdc83b40 con 0x558abdc76460 2024-04-24T09:09:15.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.387+0000 7f2c18a31740 10 client.0 _ll_register_callbacks cb 0x558abdb62140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:09:15.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.387+0000 7f2c0e000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558abdc80b30 0x558abdc7df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:15.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.387+0000 7f2c0ea00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558abdc76460 0x558abdc7d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:15.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.387+0000 7f2c0e000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558abdc80b30 0x558abdc7df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60242/0 (socket says 172.21.15.83:60242) 2024-04-24T09:09:15.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.387+0000 7f2c0e000640 1 -- 172.21.15.83:0/2844287557 learned_addr learned my addr 172.21.15.83:0/2844287557 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:09:15.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.387+0000 7f2c0e000640 1 --2- 172.21.15.83:0/2844287557 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558abdc75b40 0x558abdc80560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:15.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.388+0000 7f2c0ea00640 1 -- 172.21.15.83:0/2844287557 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558abdc75b40 msgr2=0x558abdc80560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:15.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.388+0000 7f2c0ea00640 1 --2- 172.21.15.83:0/2844287557 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558abdc75b40 0x558abdc80560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:15.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.388+0000 7f2c0ea00640 1 -- 172.21.15.83:0/2844287557 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558abdc80b30 msgr2=0x558abdc7df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:15.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.388+0000 7f2c0ea00640 1 --2- 172.21.15.83:0/2844287557 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558abdc80b30 0x558abdc7df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:15.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.388+0000 7f2c0ea00640 1 -- 172.21.15.83:0/2844287557 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558abdba53f0 con 0x558abdc76460 2024-04-24T09:09:15.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.388+0000 7f2c0e000640 1 --2- 172.21.15.83:0/2844287557 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558abdc80b30 0x558abdc7df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:09:15.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.388+0000 7f2c0ea00640 1 --2- 172.21.15.83:0/2844287557 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558abdc76460 0x558abdc7d850 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2c00005890 tx=0x7f2c00005230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41b883b96abf087c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:15.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.388+0000 7f2bf8c00640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-04-24T09:09:15.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.389+0000 7f2bf8c00640 1 -- 172.21.15.83:0/2844287557 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c00023840 con 0x558abdc76460 2024-04-24T09:09:15.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.389+0000 7f2c18a31740 1 -- 172.21.15.83:0/2844287557 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c00004c50 con 0x558abdc76460 2024-04-24T09:09:15.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.389+0000 7f2bf8c00640 1 -- 172.21.15.83:0/2844287557 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9583+0+0 (secure 0 0 0) 0x7f2c000239e0 con 0x558abdc76460 2024-04-24T09:09:15.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.390+0000 7f2bf8c00640 1 -- 172.21.15.83:0/2844287557 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f2c00023c40 con 0x558abdc76460 2024-04-24T09:09:15.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.390+0000 7f2c18a31740 10 client.28795 fetch_fsmap learned FSMap version 58 2024-04-24T09:09:15.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.390+0000 7f2c18a31740 1 -- 172.21.15.83:0/2844287557 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x558abd8ed3f0 con 0x558abdc76460 2024-04-24T09:09:15.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.391+0000 7f2bf8c00640 1 -- 172.21.15.83:0/2844287557 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2c00004c50 con 0x558abdc76460 2024-04-24T09:09:15.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.391+0000 7f2bf8c00640 1 -- 172.21.15.83:0/2844287557 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 58) ==== 8374+0+0 (secure 0 0 0) 0x7f2c000198e0 con 0x558abdc76460 2024-04-24T09:09:15.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.391+0000 7f2c18a31740 10 client.28795 fetch_fsmap finished waiting for FSMap version 58 2024-04-24T09:09:15.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.391+0000 7f2c18a31740 10 client.28795 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-24T09:09:15.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.391+0000 7f2c18a31740 20 client.28795 populate_metadata read hostname 'smithi083' 2024-04-24T09:09:15.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.391+0000 7f2c18a31740 1 --2- 172.21.15.83:0/2844287557 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x558abdc90d50 0x558abdc93140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:15.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.391+0000 7f2c18a31740 4 client.28795 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:09:15.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.391+0000 7f2c18a31740 1 -- 172.21.15.83:0/2844287557 --> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558abdbe8bf0 con 0x558abdc90d50 2024-04-24T09:09:15.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.392+0000 7f2c0e000640 1 --2- 172.21.15.83:0/2844287557 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x558abdc90d50 0x558abdc93140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:15.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.393+0000 7f2c0e000640 1 --2- 172.21.15.83:0/2844287557 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x558abdc90d50 0x558abdc93140 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=75f65c20aa5d45c9 server_cookie=22e606d04bfb0f38 in_seq=0 out_seq=0 2024-04-24T09:09:15.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.393+0000 7f2bf8c00640 10 client.28795 ms_handle_connect on v2:172.21.15.189:6838/2548965336 2024-04-24T09:09:15.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.395+0000 7f2bf8c00640 1 -- 172.21.15.83:0/2844287557 <== mds.4 v2:172.21.15.189:6838/2548965336 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558abdbe8bf0 con 0x558abdc90d50 2024-04-24T09:09:15.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.395+0000 7f2bf8c00640 10 client.28795 handle_command_reply: tid=0 2024-04-24T09:09:15.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.396+0000 7f2c18a31740 1 client.28795 shutdown 2024-04-24T09:09:15.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.396+0000 7f2c18a31740 1 -- 172.21.15.83:0/2844287557 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558abdc76460 msgr2=0x558abdc7d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:15.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.396+0000 7f2c18a31740 1 --2- 172.21.15.83:0/2844287557 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558abdc76460 0x558abdc7d850 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2c00005890 tx=0x7f2c00005230 comp rx=0 tx=0).stop 2024-04-24T09:09:15.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.397+0000 7f2c18a31740 1 -- 172.21.15.83:0/2844287557 shutdown_connections 2024-04-24T09:09:15.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.397+0000 7f2c18a31740 1 --2- 172.21.15.83:0/2844287557 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x558abdc90d50 0x558abdc93140 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:15.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.397+0000 7f2c18a31740 1 --2- 172.21.15.83:0/2844287557 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558abdc80b30 0x558abdc7df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:15.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.397+0000 7f2c18a31740 1 --2- 172.21.15.83:0/2844287557 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558abdc76460 0x558abdc7d850 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:15.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.397+0000 7f2bf8c00640 0 client.28795 ms_handle_reset on v2:172.21.15.189:6838/2548965336 2024-04-24T09:09:15.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.397+0000 7f2c18a31740 1 --2- 172.21.15.83:0/2844287557 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558abdc75b40 0x558abdc80560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:15.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.397+0000 7f2c18a31740 1 -- 172.21.15.83:0/2844287557 >> 172.21.15.83:0/2844287557 conn(0x558abdc71e60 msgr2=0x558abdc77b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:15.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.397+0000 7f2c18a31740 1 -- 172.21.15.83:0/2844287557 shutdown_connections 2024-04-24T09:09:15.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.397+0000 7f2c18a31740 1 -- 172.21.15.83:0/2844287557 wait complete. 2024-04-24T09:09:15.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.397+0000 7f2c18a31740 20 client.28795 trim_cache size 0 max 16384 2024-04-24T09:09:15.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.418+0000 7f2c18a31740 1 Processor -- start 2024-04-24T09:09:15.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.418+0000 7f2c18a31740 1 -- start start 2024-04-24T09:09:15.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.418+0000 7f2c18a31740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558abdc75b40 0x558abdc8c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:15.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.418+0000 7f2c18a31740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558abdc76460 0x558abdc80090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:15.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.418+0000 7f2c18a31740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558abdc80b30 0x558abdc805d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:15.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.418+0000 7f2c18a31740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf00100b0 con 0x558abdc76460 2024-04-24T09:09:15.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.418+0000 7f2c18a31740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf0011bf0 con 0x558abdc80b30 2024-04-24T09:09:15.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.418+0000 7f2c18a31740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf0010780 con 0x558abdc75b40 2024-04-24T09:09:15.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.419+0000 7f2c0ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558abdc76460 0x558abdc80090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:15.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.419+0000 7f2c0ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558abdc76460 0x558abdc80090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60244/0 (socket says 172.21.15.83:60244) 2024-04-24T09:09:15.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.419+0000 7f2c0ea00640 1 -- 172.21.15.83:0/1359511087 learned_addr learned my addr 172.21.15.83:0/1359511087 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:09:15.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.419+0000 7f2c0e000640 1 --2- 172.21.15.83:0/1359511087 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558abdc80b30 0x558abdc805d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:15.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.419+0000 7f2c0e000640 1 --2- 172.21.15.83:0/1359511087 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558abdc75b40 0x558abdc8c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:15.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.419+0000 7f2c0ea00640 1 -- 172.21.15.83:0/1359511087 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558abdc75b40 msgr2=0x558abdc8c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:15.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.419+0000 7f2c0ea00640 1 --2- 172.21.15.83:0/1359511087 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558abdc75b40 0x558abdc8c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:15.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.419+0000 7f2c0ea00640 1 -- 172.21.15.83:0/1359511087 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558abdc80b30 msgr2=0x558abdc805d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:15.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.419+0000 7f2c0ea00640 1 --2- 172.21.15.83:0/1359511087 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558abdc80b30 0x558abdc805d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:15.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.419+0000 7f2c0ea00640 1 -- 172.21.15.83:0/1359511087 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558abdc7e300 con 0x558abdc76460 2024-04-24T09:09:15.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.420+0000 7f2c0ea00640 1 --2- 172.21.15.83:0/1359511087 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558abdc76460 0x558abdc80090 secure :-1 s=READY pgs=2349 cs=0 l=1 rev1=1 crypto rx=0x7f2c00019ef0 tx=0x7f2c00009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=111a09e344969975 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:15.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.420+0000 7f2bfb400640 1 -- 172.21.15.83:0/1359511087 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c000097a0 con 0x558abdc76460 2024-04-24T09:09:15.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.420+0000 7f2bfb400640 1 -- 172.21.15.83:0/1359511087 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2c000052a0 con 0x558abdc76460 2024-04-24T09:09:15.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.420+0000 7f2bfb400640 1 -- 172.21.15.83:0/1359511087 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c00004020 con 0x558abdc76460 2024-04-24T09:09:15.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.421+0000 7f2c18a31740 1 -- 172.21.15.83:0/1359511087 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558abdc76460 msgr2=0x558abdc80090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:15.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.421+0000 7f2c18a31740 1 --2- 172.21.15.83:0/1359511087 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558abdc76460 0x558abdc80090 secure :-1 s=READY pgs=2349 cs=0 l=1 rev1=1 crypto rx=0x7f2c00019ef0 tx=0x7f2c00009310 comp rx=0 tx=0).stop 2024-04-24T09:09:15.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.421+0000 7f2c18a31740 1 -- 172.21.15.83:0/1359511087 shutdown_connections 2024-04-24T09:09:15.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.421+0000 7f2c18a31740 1 --2- 172.21.15.83:0/1359511087 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558abdc80b30 0x558abdc805d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:15.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.421+0000 7f2c18a31740 1 --2- 172.21.15.83:0/1359511087 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558abdc76460 0x558abdc80090 unknown :-1 s=CLOSED pgs=2349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:15.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.421+0000 7f2c18a31740 1 --2- 172.21.15.83:0/1359511087 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558abdc75b40 0x558abdc8c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:15.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.421+0000 7f2c18a31740 1 -- 172.21.15.83:0/1359511087 >> 172.21.15.83:0/1359511087 conn(0x558abdc71e60 msgr2=0x558abdc72240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:15.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.421+0000 7f2c18a31740 1 -- 172.21.15.83:0/1359511087 shutdown_connections 2024-04-24T09:09:15.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.421+0000 7f2c18a31740 1 -- 172.21.15.83:0/1359511087 wait complete. 2024-04-24T09:09:15.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.422+0000 7f2c18a31740 1 -- start start 2024-04-24T09:09:15.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.422+0000 7f2c18a31740 1 Processor -- start 2024-04-24T09:09:15.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.423+0000 7f2c18a31740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558abdc75b40 0x558abdc82880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:15.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.423+0000 7f2c18a31740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558abdc76460 0x558abdc80f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:15.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.423+0000 7f2c18a31740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558abdc80b30 0x558abdc815b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:15.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.423+0000 7f2c18a31740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf0011520 con 0x558abdc80b30 2024-04-24T09:09:15.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.423+0000 7f2c18a31740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf0010e50 con 0x558abdc75b40 2024-04-24T09:09:15.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.423+0000 7f2c18a31740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf000e0b0 con 0x558abdc76460 2024-04-24T09:09:15.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.423+0000 7f2c18a31740 10 client.0 _ll_register_callbacks cb 0x7f2bfc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:09:15.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.423+0000 7f2c0e000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558abdc75b40 0x558abdc82880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:15.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.423+0000 7f2c0e000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558abdc75b40 0x558abdc82880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:53024/0 (socket says 172.21.15.83:53024) 2024-04-24T09:09:15.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.423+0000 7f2c0e000640 1 -- 172.21.15.83:0/1355314726 learned_addr learned my addr 172.21.15.83:0/1355314726 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:09:15.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.424+0000 7f2c0ea00640 1 --2- 172.21.15.83:0/1355314726 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558abdc76460 0x558abdc80f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:15.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.424+0000 7f2c0e000640 1 -- 172.21.15.83:0/1355314726 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558abdc76460 msgr2=0x558abdc80f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:15.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.424+0000 7f2c0e000640 1 --2- 172.21.15.83:0/1355314726 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558abdc76460 0x558abdc80f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:15.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.424+0000 7f2c0e000640 1 -- 172.21.15.83:0/1355314726 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558abdc80b30 msgr2=0x558abdc815b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:15.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.424+0000 7f2c0e000640 1 --2- 172.21.15.83:0/1355314726 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558abdc80b30 0x558abdc815b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:15.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.424+0000 7f2c0e000640 1 -- 172.21.15.83:0/1355314726 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558abd9b77d0 con 0x558abdc75b40 2024-04-24T09:09:15.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.426+0000 7f2c0e000640 1 --2- 172.21.15.83:0/1355314726 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558abdc75b40 0x558abdc82880 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc004460 tx=0x7f2bfc0043f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57976c0d4e1184d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:15.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.426+0000 7f2bf8c00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:09:15.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.427+0000 7f2bf8c00640 1 -- 172.21.15.83:0/1355314726 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bfc035040 con 0x558abdc75b40 2024-04-24T09:09:15.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.427+0000 7f2bf8c00640 1 -- 172.21.15.83:0/1355314726 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9583+0+0 (secure 0 0 0) 0x7f2bfc04d3b0 con 0x558abdc75b40 2024-04-24T09:09:15.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.427+0000 7f2c18a31740 1 -- 172.21.15.83:0/1355314726 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c000052a0 con 0x558abdc75b40 2024-04-24T09:09:15.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.427+0000 7f2bf8c00640 1 -- 172.21.15.83:0/1355314726 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f2bfc04d5b0 con 0x558abdc75b40 2024-04-24T09:09:15.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.428+0000 7f2c18a31740 10 client.38849 fetch_fsmap learned FSMap version 58 2024-04-24T09:09:15.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.428+0000 7f2c18a31740 1 -- 172.21.15.83:0/1355314726 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x558abdc88d60 con 0x558abdc75b40 2024-04-24T09:09:15.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.428+0000 7f2bf8c00640 1 -- 172.21.15.83:0/1355314726 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2c000052a0 con 0x558abdc75b40 2024-04-24T09:09:15.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.429+0000 7f2bf8c00640 1 -- 172.21.15.83:0/1355314726 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 58) ==== 8374+0+0 (secure 0 0 0) 0x7f2bfc031430 con 0x558abdc75b40 2024-04-24T09:09:15.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.429+0000 7f2c18a31740 10 client.38849 fetch_fsmap finished waiting for FSMap version 58 2024-04-24T09:09:15.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.429+0000 7f2c18a31740 10 client.38849 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-24T09:09:15.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.429+0000 7f2c18a31740 20 client.38849 populate_metadata read hostname 'smithi083' 2024-04-24T09:09:15.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.430+0000 7f2c18a31740 1 --2- 172.21.15.83:0/1355314726 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x558abdc894d0 0x558abdcf9250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:15.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.430+0000 7f2c18a31740 4 client.38849 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:09:15.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.430+0000 7f2c18a31740 1 -- 172.21.15.83:0/1355314726 --> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558abdbe2a60 con 0x558abdc894d0 2024-04-24T09:09:15.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.431+0000 7f2c0ea00640 1 --2- 172.21.15.83:0/1355314726 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x558abdc894d0 0x558abdcf9250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:15.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.433+0000 7f2bf8c00640 10 client.38849 ms_handle_connect on v2:172.21.15.189:6838/2548965336 2024-04-24T09:09:15.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.433+0000 7f2c0ea00640 1 --2- 172.21.15.83:0/1355314726 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x558abdc894d0 0x558abdcf9250 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1c9fa400771208a8 server_cookie=d396fa916be04c16 in_seq=0 out_seq=0 2024-04-24T09:09:15.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.434+0000 7f2bf8c00640 1 -- 172.21.15.83:0/1355314726 <== mds.4 v2:172.21.15.189:6838/2548965336 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558abdbe2a60 con 0x558abdc894d0 2024-04-24T09:09:15.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.434+0000 7f2bf8c00640 10 client.38849 handle_command_reply: tid=0 2024-04-24T09:09:15.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.434+0000 7f2c18a31740 1 client.38849 shutdown 2024-04-24T09:09:15.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.435+0000 7f2c18a31740 1 -- 172.21.15.83:0/1355314726 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558abdc75b40 msgr2=0x558abdc82880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:15.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.435+0000 7f2c18a31740 1 --2- 172.21.15.83:0/1355314726 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558abdc75b40 0x558abdc82880 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc004460 tx=0x7f2bfc0043f0 comp rx=0 tx=0).stop 2024-04-24T09:09:15.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.436+0000 7f2c18a31740 1 -- 172.21.15.83:0/1355314726 shutdown_connections 2024-04-24T09:09:15.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.436+0000 7f2c18a31740 1 --2- 172.21.15.83:0/1355314726 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x558abdc894d0 0x558abdcf9250 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:15.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.436+0000 7f2c18a31740 1 --2- 172.21.15.83:0/1355314726 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558abdc80b30 0x558abdc815b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:15.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.436+0000 7f2c18a31740 1 --2- 172.21.15.83:0/1355314726 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558abdc76460 0x558abdc80f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:15.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.436+0000 7f2c18a31740 1 --2- 172.21.15.83:0/1355314726 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558abdc75b40 0x558abdc82880 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:15.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.437+0000 7f2c18a31740 1 -- 172.21.15.83:0/1355314726 >> 172.21.15.83:0/1355314726 conn(0x558abdc71e60 msgr2=0x558abdc7c010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:15.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.437+0000 7f2bf8c00640 0 client.38849 ms_handle_reset on v2:172.21.15.189:6838/2548965336 2024-04-24T09:09:15.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.437+0000 7f2c18a31740 1 -- 172.21.15.83:0/1355314726 shutdown_connections 2024-04-24T09:09:15.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.437+0000 7f2c18a31740 1 -- 172.21.15.83:0/1355314726 wait complete. 2024-04-24T09:09:15.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.437+0000 7f2c18a31740 20 client.38849 trim_cache size 0 max 16384 2024-04-24T09:09:15.438 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:09:15.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.438+0000 7f2c0fe00640 1 -- 172.21.15.83:0/1344110609 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f2bf40a3e20 msgr2=0x7f2bf40a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:15.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.438+0000 7f2c0fe00640 1 --2- 172.21.15.83:0/1344110609 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f2bf40a3e20 0x7f2bf40a62e0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f2c080a4d60 tx=0x7f2c0800ef30 comp rx=0 tx=0).stop 2024-04-24T09:09:15.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.438+0000 7f2c0fe00640 1 -- 172.21.15.83:0/1344110609 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c10106250 msgr2=0x7f2c1019eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:15.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.438+0000 7f2c0fe00640 1 --2- 172.21.15.83:0/1344110609 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c10106250 0x7f2c1019eab0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f2c0800e940 tx=0x7f2c0800ee00 comp rx=0 tx=0).stop 2024-04-24T09:09:15.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.438+0000 7f2c0fe00640 1 -- 172.21.15.83:0/1344110609 shutdown_connections 2024-04-24T09:09:15.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.438+0000 7f2c0fe00640 1 --2- 172.21.15.83:0/1344110609 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f2bf40a3e20 0x7f2bf40a62e0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:15.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.438+0000 7f2c0fe00640 1 --2- 172.21.15.83:0/1344110609 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c10106250 0x7f2c1019eab0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:15.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.438+0000 7f2c0fe00640 1 --2- 172.21.15.83:0/1344110609 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c10101770 0x7f2c101a4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:15.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.438+0000 7f2c0fe00640 1 --2- 172.21.15.83:0/1344110609 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2c10100e50 0x7f2c101a4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:15.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.438+0000 7f2c0fe00640 1 -- 172.21.15.83:0/1344110609 >> 172.21.15.83:0/1344110609 conn(0x7f2c100fc880 msgr2=0x7f2c100fd940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:15.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.439+0000 7f2c0fe00640 1 -- 172.21.15.83:0/1344110609 shutdown_connections 2024-04-24T09:09:15.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.439+0000 7f2c0fe00640 1 -- 172.21.15.83:0/1344110609 wait complete. 2024-04-24T09:09:15.461 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-24T09:09:15.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.723+0000 7ff85ec00640 1 -- 172.21.15.83:0/3758891084 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff858103980 msgr2=0x7ff85810be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:15.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.723+0000 7ff85ec00640 1 --2- 172.21.15.83:0/3758891084 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff858103980 0x7ff85810be60 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff850009900 tx=0x7ff85002a480 comp rx=0 tx=0).stop 2024-04-24T09:09:15.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.723+0000 7ff85ec00640 1 -- 172.21.15.83:0/3758891084 shutdown_connections 2024-04-24T09:09:15.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.723+0000 7ff85ec00640 1 --2- 172.21.15.83:0/3758891084 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff85810c480 0x7ff85810e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:15.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.723+0000 7ff85ec00640 1 --2- 172.21.15.83:0/3758891084 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff858103980 0x7ff85810be60 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:15.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.723+0000 7ff85ec00640 1 --2- 172.21.15.83:0/3758891084 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff858103060 0x7ff858103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:15.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.723+0000 7ff85ec00640 1 -- 172.21.15.83:0/3758891084 >> 172.21.15.83:0/3758891084 conn(0x7ff8580fc880 msgr2=0x7ff8580feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:15.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.723+0000 7ff85ec00640 1 -- 172.21.15.83:0/3758891084 shutdown_connections 2024-04-24T09:09:15.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.724+0000 7ff85ec00640 1 -- 172.21.15.83:0/3758891084 wait complete. 2024-04-24T09:09:15.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.725+0000 7ff85ec00640 1 Processor -- start 2024-04-24T09:09:15.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.727+0000 7ff85ec00640 1 -- start start 2024-04-24T09:09:15.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.729+0000 7ff85ec00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff858103060 0x7ff858073850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:15.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.729+0000 7ff85ec00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff858103980 0x7ff85806c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:15.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.729+0000 7ff85ec00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff85810c480 0x7ff85806ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:15.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.729+0000 7ff85ec00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff858118780 con 0x7ff85810c480 2024-04-24T09:09:15.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.729+0000 7ff85ec00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff858118600 con 0x7ff858103060 2024-04-24T09:09:15.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.729+0000 7ff85ec00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff858118900 con 0x7ff858103980 2024-04-24T09:09:15.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.730+0000 7ff85e200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff85810c480 0x7ff85806ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:15.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.730+0000 7ff85d800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff858103060 0x7ff858073850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:15.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.730+0000 7ff85e200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff85810c480 0x7ff85806ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60252/0 (socket says 172.21.15.83:60252) 2024-04-24T09:09:15.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.730+0000 7ff85e200640 1 -- 172.21.15.83:0/3338464856 learned_addr learned my addr 172.21.15.83:0/3338464856 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:09:15.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.730+0000 7ff85ce00640 1 --2- 172.21.15.83:0/3338464856 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff858103980 0x7ff85806c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:15.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.731+0000 7ff85e200640 1 -- 172.21.15.83:0/3338464856 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff858103980 msgr2=0x7ff85806c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:15.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.731+0000 7ff85e200640 1 --2- 172.21.15.83:0/3338464856 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff858103980 0x7ff85806c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:15.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.731+0000 7ff85e200640 1 -- 172.21.15.83:0/3338464856 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff858103060 msgr2=0x7ff858073850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:15.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.731+0000 7ff85e200640 1 --2- 172.21.15.83:0/3338464856 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff858103060 0x7ff858073850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:15.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.732+0000 7ff85e200640 1 -- 172.21.15.83:0/3338464856 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff85806d450 con 0x7ff85810c480 2024-04-24T09:09:15.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.735+0000 7ff85e200640 1 --2- 172.21.15.83:0/3338464856 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff85810c480 0x7ff85806ce10 secure :-1 s=READY pgs=2350 cs=0 l=1 rev1=1 crypto rx=0x7ff84c00eb10 tx=0x7ff84c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:15.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.737+0000 7ff856000640 1 -- 172.21.15.83:0/3338464856 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff84c004b20 con 0x7ff85810c480 2024-04-24T09:09:15.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.737+0000 7ff856000640 1 -- 172.21.15.83:0/3338464856 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff84c004cc0 con 0x7ff85810c480 2024-04-24T09:09:15.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.737+0000 7ff856000640 1 -- 172.21.15.83:0/3338464856 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff84c010690 con 0x7ff85810c480 2024-04-24T09:09:15.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.737+0000 7ff85ec00640 1 -- 172.21.15.83:0/3338464856 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff858071370 con 0x7ff85810c480 2024-04-24T09:09:15.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.739+0000 7ff85ec00640 1 -- 172.21.15.83:0/3338464856 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff858071840 con 0x7ff85810c480 2024-04-24T09:09:15.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.752+0000 7ff86302c740 1 Processor -- start 2024-04-24T09:09:15.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.757+0000 7ff856000640 1 -- 172.21.15.83:0/3338464856 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff84c005000 con 0x7ff85810c480 2024-04-24T09:09:15.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.757+0000 7ff86302c740 1 -- start start 2024-04-24T09:09:15.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.758+0000 7ff86302c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bc068b8b40 0x55bc068b8f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:15.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.758+0000 7ff86302c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc068b9460 0x55bc068c35f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:15.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.758+0000 7ff86302c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bc068c3b30 0x55bc068c5f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:15.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.758+0000 7ff86302c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55bc0681e1d0 con 0x55bc068c3b30 2024-04-24T09:09:15.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.758+0000 7ff86302c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55bc067c4450 con 0x55bc068b9460 2024-04-24T09:09:15.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.758+0000 7ff86302c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55bc0681a190 con 0x55bc068b8b40 2024-04-24T09:09:15.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.759+0000 7ff856000640 1 --2- 172.21.15.83:0/3338464856 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff8400a3eb0 0x7ff8400a6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:15.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.759+0000 7ff85ce00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bc068b8b40 0x55bc068b8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:15.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.759+0000 7ff85ce00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bc068b8b40 0x55bc068b8f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:50856/0 (socket says 172.21.15.83:50856) 2024-04-24T09:09:15.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.759+0000 7ff85ce00640 1 -- 172.21.15.83:0/3634213771 learned_addr learned my addr 172.21.15.83:0/3634213771 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:09:15.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.760+0000 7ff85ce00640 1 --2- 172.21.15.83:0/3634213771 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bc068c3b30 0x55bc068c5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:15.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.760+0000 7ff856000640 1 -- 172.21.15.83:0/3338464856 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 9583+0+0 (secure 0 0 0) 0x7ff84c0d11e0 con 0x7ff85810c480 2024-04-24T09:09:15.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.760+0000 7ff85d800640 1 --2- 172.21.15.83:0/3634213771 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc068b9460 0x55bc068c35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:15.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.760+0000 7ff85e200640 1 --2- 172.21.15.83:0/3338464856 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff8400a3eb0 0x7ff8400a6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:15.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.761+0000 7ff85ce00640 1 -- 172.21.15.83:0/3634213771 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bc068b8b40 msgr2=0x55bc068b8f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:15.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.761+0000 7ff85ce00640 1 --2- 172.21.15.83:0/3634213771 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bc068b8b40 0x55bc068b8f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:15.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.761+0000 7ff85ce00640 1 -- 172.21.15.83:0/3634213771 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc068b9460 msgr2=0x55bc068c35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:15.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.761+0000 7ff85ce00640 1 --2- 172.21.15.83:0/3634213771 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc068b9460 0x55bc068c35f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:15.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.761+0000 7ff85ce00640 1 -- 172.21.15.83:0/3634213771 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc06881310 con 0x55bc068c3b30 2024-04-24T09:09:15.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.761+0000 7ff85ce00640 1 --2- 172.21.15.83:0/3634213771 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bc068c3b30 0x55bc068c5f20 secure :-1 s=READY pgs=2351 cs=0 l=1 rev1=1 crypto rx=0x7ff850005dc0 tx=0x7ff850031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20d5cb6a64e4af9e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:15.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.761+0000 7ff85e200640 1 --2- 172.21.15.83:0/3338464856 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff8400a3eb0 0x7ff8400a6370 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7ff84c0049d0 tx=0x7ff84c005b40 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:15.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.761+0000 7ff84aa00640 1 -- 172.21.15.83:0/3634213771 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff850035040 con 0x55bc068c3b30 2024-04-24T09:09:15.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.762+0000 7ff84aa00640 1 -- 172.21.15.83:0/3634213771 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff8500046c0 con 0x55bc068c3b30 2024-04-24T09:09:15.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.762+0000 7ff84aa00640 1 -- 172.21.15.83:0/3634213771 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff850003850 con 0x55bc068c3b30 2024-04-24T09:09:15.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.763+0000 7ff86302c740 1 -- 172.21.15.83:0/3634213771 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bc068c3b30 msgr2=0x55bc068c5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:15.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.763+0000 7ff86302c740 1 --2- 172.21.15.83:0/3634213771 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bc068c3b30 0x55bc068c5f20 secure :-1 s=READY pgs=2351 cs=0 l=1 rev1=1 crypto rx=0x7ff850005dc0 tx=0x7ff850031400 comp rx=0 tx=0).stop 2024-04-24T09:09:15.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.763+0000 7ff86302c740 1 -- 172.21.15.83:0/3634213771 shutdown_connections 2024-04-24T09:09:15.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.763+0000 7ff86302c740 1 --2- 172.21.15.83:0/3634213771 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bc068c3b30 0x55bc068c5f20 unknown :-1 s=CLOSED pgs=2351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:15.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.763+0000 7ff86302c740 1 --2- 172.21.15.83:0/3634213771 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc068b9460 0x55bc068c35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:15.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.763+0000 7ff86302c740 1 --2- 172.21.15.83:0/3634213771 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bc068b8b40 0x55bc068b8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:15.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.763+0000 7ff86302c740 1 -- 172.21.15.83:0/3634213771 >> 172.21.15.83:0/3634213771 conn(0x55bc068b4e60 msgr2=0x55bc068b5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:15.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.764+0000 7ff86302c740 1 -- 172.21.15.83:0/3634213771 shutdown_connections 2024-04-24T09:09:15.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.764+0000 7ff86302c740 1 -- 172.21.15.83:0/3634213771 wait complete. 2024-04-24T09:09:15.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.765+0000 7ff86302c740 1 -- start start 2024-04-24T09:09:15.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.765+0000 7ff86302c740 1 Processor -- start 2024-04-24T09:09:15.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.766+0000 7ff86302c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bc068b8b40 0x55bc068c35e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:15.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.766+0000 7ff86302c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc068b9460 0x55bc068c08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:15.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.766+0000 7ff86302c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bc068c3b30 0x55bc068c1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:15.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.766+0000 7ff86302c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55bc068c6960 con 0x55bc068c3b30 2024-04-24T09:09:15.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.766+0000 7ff86302c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55bc067d7810 con 0x55bc068b9460 2024-04-24T09:09:15.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.766+0000 7ff86302c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55bc068c6ae0 con 0x55bc068b8b40 2024-04-24T09:09:15.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.766+0000 7ff86302c740 10 client.0 _ll_register_callbacks cb 0x55bc067a5140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:09:15.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.767+0000 7ff85ce00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bc068b8b40 0x55bc068c35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:15.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.767+0000 7ff85d800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc068b9460 0x55bc068c08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:15.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.767+0000 7ff85d800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc068b9460 0x55bc068c08d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:53080/0 (socket says 172.21.15.83:53080) 2024-04-24T09:09:15.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.767+0000 7ff85ce00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bc068b8b40 0x55bc068c35e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:50870/0 (socket says 172.21.15.83:50870) 2024-04-24T09:09:15.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.767+0000 7ff85d800640 1 -- 172.21.15.83:0/2303915917 learned_addr learned my addr 172.21.15.83:0/2303915917 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:09:15.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.768+0000 7ff85ce00640 1 --2- 172.21.15.83:0/2303915917 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bc068c3b30 0x55bc068c1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:15.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.768+0000 7ff85d800640 1 -- 172.21.15.83:0/2303915917 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bc068b8b40 msgr2=0x55bc068c35e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:15.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.768+0000 7ff85d800640 1 --2- 172.21.15.83:0/2303915917 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bc068b8b40 0x55bc068c35e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:15.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.768+0000 7ff85d800640 1 -- 172.21.15.83:0/2303915917 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bc068c3b30 msgr2=0x55bc068c1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:15.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.768+0000 7ff85d800640 1 --2- 172.21.15.83:0/2303915917 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bc068c3b30 0x55bc068c1010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:15.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.768+0000 7ff85d800640 1 -- 172.21.15.83:0/2303915917 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc067e83f0 con 0x55bc068b9460 2024-04-24T09:09:15.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.769+0000 7ff85d800640 1 --2- 172.21.15.83:0/2303915917 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc068b9460 0x55bc068c08d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff8440106a0 tx=0x7ff844010b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b47b6e565526eed3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:15.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.769+0000 7ff83f400640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:09:15.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.770+0000 7ff83f400640 1 -- 172.21.15.83:0/2303915917 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff844010ea0 con 0x55bc068b9460 2024-04-24T09:09:15.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.770+0000 7ff83f400640 1 -- 172.21.15.83:0/2303915917 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9583+0+0 (secure 0 0 0) 0x7ff84400ead0 con 0x55bc068b9460 2024-04-24T09:09:15.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.770+0000 7ff86302c740 1 -- 172.21.15.83:0/2303915917 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8500046c0 con 0x55bc068b9460 2024-04-24T09:09:15.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.770+0000 7ff83f400640 1 -- 172.21.15.83:0/2303915917 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7ff844025050 con 0x55bc068b9460 2024-04-24T09:09:15.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.770+0000 7ff86302c740 10 client.38858 fetch_fsmap learned FSMap version 58 2024-04-24T09:09:15.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.770+0000 7ff86302c740 1 -- 172.21.15.83:0/2303915917 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55bc065303f0 con 0x55bc068b9460 2024-04-24T09:09:15.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.771+0000 7ff83f400640 1 -- 172.21.15.83:0/2303915917 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff8500046c0 con 0x55bc068b9460 2024-04-24T09:09:15.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.771+0000 7ff83f400640 1 -- 172.21.15.83:0/2303915917 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 58) ==== 8374+0+0 (secure 0 0 0) 0x7ff84400f040 con 0x55bc068b9460 2024-04-24T09:09:15.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.771+0000 7ff86302c740 10 client.38858 fetch_fsmap finished waiting for FSMap version 58 2024-04-24T09:09:15.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.771+0000 7ff86302c740 10 client.38858 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-24T09:09:15.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.771+0000 7ff86302c740 20 client.38858 populate_metadata read hostname 'smithi083' 2024-04-24T09:09:15.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.771+0000 7ff86302c740 1 --2- 172.21.15.83:0/2303915917 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55bc068d3e00 0x55bc068d6220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:15.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.771+0000 7ff86302c740 4 client.38858 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:09:15.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.771+0000 7ff86302c740 1 -- 172.21.15.83:0/2303915917 --> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bc0682bbf0 con 0x55bc068d3e00 2024-04-24T09:09:15.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.772+0000 7ff85ce00640 1 --2- 172.21.15.83:0/2303915917 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55bc068d3e00 0x55bc068d6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:15.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.772+0000 7ff85ce00640 1 --2- 172.21.15.83:0/2303915917 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55bc068d3e00 0x55bc068d6220 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5c31c8401fde4950 server_cookie=6d3bdb85b4a47d61 in_seq=0 out_seq=0 2024-04-24T09:09:15.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.772+0000 7ff83f400640 10 client.38858 ms_handle_connect on v2:172.21.15.138:6832/1157967658 2024-04-24T09:09:15.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.774+0000 7ff83f400640 1 -- 172.21.15.83:0/2303915917 <== mds.3 v2:172.21.15.138:6832/1157967658 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bc0682bbf0 con 0x55bc068d3e00 2024-04-24T09:09:15.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.774+0000 7ff83f400640 10 client.38858 handle_command_reply: tid=0 2024-04-24T09:09:15.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.774+0000 7ff86302c740 1 client.38858 shutdown 2024-04-24T09:09:15.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.775+0000 7ff86302c740 1 -- 172.21.15.83:0/2303915917 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc068b9460 msgr2=0x55bc068c08d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:15.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.775+0000 7ff86302c740 1 --2- 172.21.15.83:0/2303915917 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc068b9460 0x55bc068c08d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff8440106a0 tx=0x7ff844010b60 comp rx=0 tx=0).stop 2024-04-24T09:09:15.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.775+0000 7ff86302c740 1 -- 172.21.15.83:0/2303915917 shutdown_connections 2024-04-24T09:09:15.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.775+0000 7ff86302c740 1 --2- 172.21.15.83:0/2303915917 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bc068c3b30 0x55bc068c1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:15.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.775+0000 7ff86302c740 1 --2- 172.21.15.83:0/2303915917 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc068b9460 0x55bc068c08d0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:15.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.775+0000 7ff86302c740 1 --2- 172.21.15.83:0/2303915917 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55bc068d3e00 0x55bc068d6220 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:15.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.775+0000 7ff86302c740 1 --2- 172.21.15.83:0/2303915917 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bc068b8b40 0x55bc068c35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:15.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.775+0000 7ff86302c740 1 -- 172.21.15.83:0/2303915917 >> 172.21.15.83:0/2303915917 conn(0x55bc068b4e60 msgr2=0x55bc068bac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:15.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.775+0000 7ff83f400640 0 client.38858 ms_handle_reset on v2:172.21.15.138:6832/1157967658 2024-04-24T09:09:15.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.775+0000 7ff86302c740 1 -- 172.21.15.83:0/2303915917 shutdown_connections 2024-04-24T09:09:15.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.776+0000 7ff86302c740 1 -- 172.21.15.83:0/2303915917 wait complete. 2024-04-24T09:09:15.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.776+0000 7ff86302c740 20 client.38858 trim_cache size 0 max 16384 2024-04-24T09:09:15.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.806+0000 7ff86302c740 1 Processor -- start 2024-04-24T09:09:15.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.806+0000 7ff86302c740 1 -- start start 2024-04-24T09:09:15.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.806+0000 7ff86302c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bc068b8b40 0x55bc068cf820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:15.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.806+0000 7ff86302c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bc068b9460 0x55bc068c3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:15.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.806+0000 7ff86302c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc068c3b30 0x55bc068dbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:15.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.806+0000 7ff86302c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff8581be230 con 0x55bc068b8b40 2024-04-24T09:09:15.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.806+0000 7ff86302c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff8581bfd70 con 0x55bc068c3b30 2024-04-24T09:09:15.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.807+0000 7ff86302c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff8581be900 con 0x55bc068b9460 2024-04-24T09:09:15.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.807+0000 7ff85ce00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bc068b8b40 0x55bc068cf820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:15.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.807+0000 7ff85ce00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bc068b8b40 0x55bc068cf820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60282/0 (socket says 172.21.15.83:60282) 2024-04-24T09:09:15.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.807+0000 7ff85ce00640 1 -- 172.21.15.83:0/3667443729 learned_addr learned my addr 172.21.15.83:0/3667443729 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:09:15.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.807+0000 7ff85ce00640 1 --2- 172.21.15.83:0/3667443729 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc068c3b30 0x55bc068dbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:15.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.807+0000 7ff85d800640 1 --2- 172.21.15.83:0/3667443729 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bc068b9460 0x55bc068c3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:15.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.807+0000 7ff85ce00640 1 -- 172.21.15.83:0/3667443729 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bc068b9460 msgr2=0x55bc068c3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:15.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.807+0000 7ff85ce00640 1 --2- 172.21.15.83:0/3667443729 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bc068b9460 0x55bc068c3110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:15.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.807+0000 7ff85ce00640 1 -- 172.21.15.83:0/3667443729 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc068c3b30 msgr2=0x55bc068dbaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:15.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.807+0000 7ff85ce00640 1 --2- 172.21.15.83:0/3667443729 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc068c3b30 0x55bc068dbaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:15.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.807+0000 7ff85ce00640 1 -- 172.21.15.83:0/3667443729 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc065fa7d0 con 0x55bc068b8b40 2024-04-24T09:09:15.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.808+0000 7ff85ce00640 1 --2- 172.21.15.83:0/3667443729 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bc068b8b40 0x55bc068cf820 secure :-1 s=READY pgs=2352 cs=0 l=1 rev1=1 crypto rx=0x7ff850009230 tx=0x7ff850009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=772be8a96ac1bffb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:15.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.808+0000 7ff84a000640 1 -- 172.21.15.83:0/3667443729 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff850035070 con 0x55bc068b8b40 2024-04-24T09:09:15.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.808+0000 7ff84a000640 1 -- 172.21.15.83:0/3667443729 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff850004380 con 0x55bc068b8b40 2024-04-24T09:09:15.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.808+0000 7ff84a000640 1 -- 172.21.15.83:0/3667443729 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff850031820 con 0x55bc068b8b40 2024-04-24T09:09:15.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.809+0000 7ff86302c740 1 -- 172.21.15.83:0/3667443729 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bc068b8b40 msgr2=0x55bc068cf820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:15.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.809+0000 7ff86302c740 1 --2- 172.21.15.83:0/3667443729 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bc068b8b40 0x55bc068cf820 secure :-1 s=READY pgs=2352 cs=0 l=1 rev1=1 crypto rx=0x7ff850009230 tx=0x7ff850009580 comp rx=0 tx=0).stop 2024-04-24T09:09:15.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.809+0000 7ff86302c740 1 -- 172.21.15.83:0/3667443729 shutdown_connections 2024-04-24T09:09:15.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.809+0000 7ff86302c740 1 --2- 172.21.15.83:0/3667443729 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc068c3b30 0x55bc068dbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:15.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.809+0000 7ff86302c740 1 --2- 172.21.15.83:0/3667443729 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bc068b9460 0x55bc068c3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:15.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.809+0000 7ff86302c740 1 --2- 172.21.15.83:0/3667443729 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bc068b8b40 0x55bc068cf820 unknown :-1 s=CLOSED pgs=2352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:15.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.809+0000 7ff86302c740 1 -- 172.21.15.83:0/3667443729 >> 172.21.15.83:0/3667443729 conn(0x55bc068b4e60 msgr2=0x55bc068bbb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:15.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.810+0000 7ff86302c740 1 -- 172.21.15.83:0/3667443729 shutdown_connections 2024-04-24T09:09:15.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.810+0000 7ff86302c740 1 -- 172.21.15.83:0/3667443729 wait complete. 2024-04-24T09:09:15.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.810+0000 7ff86302c740 1 -- start start 2024-04-24T09:09:15.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.810+0000 7ff86302c740 1 Processor -- start 2024-04-24T09:09:15.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.811+0000 7ff86302c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc068b8b40 0x55bc068be2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:15.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.812+0000 7ff86302c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bc068b9460 0x55bc06937350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:15.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.812+0000 7ff86302c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bc068c3b30 0x55bc06937890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:15.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.812+0000 7ff86302c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff8581bf6a0 con 0x55bc068c3b30 2024-04-24T09:09:15.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.812+0000 7ff86302c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff8581befd0 con 0x55bc068b8b40 2024-04-24T09:09:15.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.812+0000 7ff85ce00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc068b8b40 0x55bc068be2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:15.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.812+0000 7ff85ce00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc068b8b40 0x55bc068be2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:53092/0 (socket says 172.21.15.83:53092) 2024-04-24T09:09:15.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.812+0000 7ff85ce00640 1 -- 172.21.15.83:0/1991030669 learned_addr learned my addr 172.21.15.83:0/1991030669 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:09:15.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.812+0000 7ff86302c740 1 -- 172.21.15.83:0/1991030669 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff8581bc230 con 0x55bc068b9460 2024-04-24T09:09:15.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.813+0000 7ff86302c740 10 client.0 _ll_register_callbacks cb 0x7ff84400b3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:09:15.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.812+0000 7ff85d800640 1 --2- 172.21.15.83:0/1991030669 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bc068b9460 0x55bc06937350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:15.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.813+0000 7ff85ce00640 1 --2- 172.21.15.83:0/1991030669 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bc068c3b30 0x55bc06937890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:15.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.813+0000 7ff85ce00640 1 -- 172.21.15.83:0/1991030669 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bc068b9460 msgr2=0x55bc06937350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:15.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.813+0000 7ff85ce00640 1 --2- 172.21.15.83:0/1991030669 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bc068b9460 0x55bc06937350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:15.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.813+0000 7ff85ce00640 1 -- 172.21.15.83:0/1991030669 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bc068c3b30 msgr2=0x55bc06937890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:15.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.813+0000 7ff85ce00640 1 --2- 172.21.15.83:0/1991030669 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bc068c3b30 0x55bc06937890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:15.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.813+0000 7ff85ce00640 1 -- 172.21.15.83:0/1991030669 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc06937f00 con 0x55bc068b8b40 2024-04-24T09:09:15.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.814+0000 7ff85ce00640 1 --2- 172.21.15.83:0/1991030669 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc068b8b40 0x55bc068be2b0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff850040040 tx=0x7ff85002a980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=346d62b32120d9d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:15.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.814+0000 7ff83f400640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:09:15.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.814+0000 7ff83f400640 1 -- 172.21.15.83:0/1991030669 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff850035070 con 0x55bc068b8b40 2024-04-24T09:09:15.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.814+0000 7ff83f400640 1 -- 172.21.15.83:0/1991030669 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9583+0+0 (secure 0 0 0) 0x7ff850003d30 con 0x55bc068b8b40 2024-04-24T09:09:15.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.814+0000 7ff86302c740 1 -- 172.21.15.83:0/1991030669 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff850004380 con 0x55bc068b8b40 2024-04-24T09:09:15.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.815+0000 7ff83f400640 1 -- 172.21.15.83:0/1991030669 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7ff8500314a0 con 0x55bc068b8b40 2024-04-24T09:09:15.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.815+0000 7ff86302c740 10 client.38864 fetch_fsmap learned FSMap version 58 2024-04-24T09:09:15.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.815+0000 7ff86302c740 1 -- 172.21.15.83:0/1991030669 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55bc068cad20 con 0x55bc068b8b40 2024-04-24T09:09:15.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.815+0000 7ff83f400640 1 -- 172.21.15.83:0/1991030669 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff850004380 con 0x55bc068b8b40 2024-04-24T09:09:15.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.815+0000 7ff83f400640 1 -- 172.21.15.83:0/1991030669 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 58) ==== 8374+0+0 (secure 0 0 0) 0x7ff85003a040 con 0x55bc068b8b40 2024-04-24T09:09:15.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.816+0000 7ff86302c740 10 client.38864 fetch_fsmap finished waiting for FSMap version 58 2024-04-24T09:09:15.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.816+0000 7ff86302c740 10 client.38864 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-24T09:09:15.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.816+0000 7ff86302c740 20 client.38864 populate_metadata read hostname 'smithi083' 2024-04-24T09:09:15.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.816+0000 7ff86302c740 1 --2- 172.21.15.83:0/1991030669 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55bc068cb010 0x55bc068cb3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:15.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.816+0000 7ff86302c740 4 client.38864 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:09:15.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.816+0000 7ff86302c740 1 -- 172.21.15.83:0/1991030669 --> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bc06825a60 con 0x55bc068cb010 2024-04-24T09:09:15.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.816+0000 7ff85d800640 1 --2- 172.21.15.83:0/1991030669 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55bc068cb010 0x55bc068cb3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:15.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.817+0000 7ff85d800640 1 --2- 172.21.15.83:0/1991030669 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55bc068cb010 0x55bc068cb3f0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c00af2171d0f1520 server_cookie=dd123740b98daf0b in_seq=0 out_seq=0 2024-04-24T09:09:15.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.817+0000 7ff83f400640 10 client.38864 ms_handle_connect on v2:172.21.15.138:6832/1157967658 2024-04-24T09:09:15.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.817+0000 7ff83f400640 1 -- 172.21.15.83:0/1991030669 <== mds.3 v2:172.21.15.138:6832/1157967658 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bc06825a60 con 0x55bc068cb010 2024-04-24T09:09:15.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.817+0000 7ff83f400640 10 client.38864 handle_command_reply: tid=0 2024-04-24T09:09:15.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.817+0000 7ff86302c740 1 client.38864 shutdown 2024-04-24T09:09:15.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.817+0000 7ff86302c740 1 -- 172.21.15.83:0/1991030669 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc068b8b40 msgr2=0x55bc068be2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:15.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.817+0000 7ff86302c740 1 --2- 172.21.15.83:0/1991030669 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc068b8b40 0x55bc068be2b0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff850040040 tx=0x7ff85002a980 comp rx=0 tx=0).stop 2024-04-24T09:09:15.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.818+0000 7ff86302c740 1 -- 172.21.15.83:0/1991030669 shutdown_connections 2024-04-24T09:09:15.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.818+0000 7ff86302c740 1 --2- 172.21.15.83:0/1991030669 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bc068c3b30 0x55bc06937890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:15.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.818+0000 7ff86302c740 1 --2- 172.21.15.83:0/1991030669 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55bc068cb010 0x55bc068cb3f0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:15.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.818+0000 7ff86302c740 1 --2- 172.21.15.83:0/1991030669 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bc068b9460 0x55bc06937350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:15.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.818+0000 7ff86302c740 1 --2- 172.21.15.83:0/1991030669 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc068b8b40 0x55bc068be2b0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:15.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.818+0000 7ff86302c740 1 -- 172.21.15.83:0/1991030669 >> 172.21.15.83:0/1991030669 conn(0x55bc068b4e60 msgr2=0x55bc068b36c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:15.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.818+0000 7ff83f400640 0 client.38864 ms_handle_reset on v2:172.21.15.138:6832/1157967658 2024-04-24T09:09:15.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.818+0000 7ff86302c740 1 -- 172.21.15.83:0/1991030669 shutdown_connections 2024-04-24T09:09:15.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.818+0000 7ff86302c740 1 -- 172.21.15.83:0/1991030669 wait complete. 2024-04-24T09:09:15.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.818+0000 7ff86302c740 20 client.38864 trim_cache size 0 max 16384 2024-04-24T09:09:15.819 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:09:15.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.819+0000 7ff85ec00640 1 -- 172.21.15.83:0/3338464856 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff8400a3eb0 msgr2=0x7ff8400a6370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:15.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.819+0000 7ff85ec00640 1 --2- 172.21.15.83:0/3338464856 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff8400a3eb0 0x7ff8400a6370 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7ff84c0049d0 tx=0x7ff84c005b40 comp rx=0 tx=0).stop 2024-04-24T09:09:15.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.819+0000 7ff85ec00640 1 -- 172.21.15.83:0/3338464856 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff85810c480 msgr2=0x7ff85806ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:15.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.819+0000 7ff85ec00640 1 --2- 172.21.15.83:0/3338464856 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff85810c480 0x7ff85806ce10 secure :-1 s=READY pgs=2350 cs=0 l=1 rev1=1 crypto rx=0x7ff84c00eb10 tx=0x7ff84c00c2d0 comp rx=0 tx=0).stop 2024-04-24T09:09:15.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.819+0000 7ff85ec00640 1 -- 172.21.15.83:0/3338464856 shutdown_connections 2024-04-24T09:09:15.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.819+0000 7ff85ec00640 1 --2- 172.21.15.83:0/3338464856 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff8400a3eb0 0x7ff8400a6370 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:15.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.819+0000 7ff85ec00640 1 --2- 172.21.15.83:0/3338464856 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff85810c480 0x7ff85806ce10 unknown :-1 s=CLOSED pgs=2350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:15.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.819+0000 7ff85ec00640 1 --2- 172.21.15.83:0/3338464856 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff858103980 0x7ff85806c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:15.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.819+0000 7ff85ec00640 1 --2- 172.21.15.83:0/3338464856 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff858103060 0x7ff858073850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:15.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.820+0000 7ff85ec00640 1 -- 172.21.15.83:0/3338464856 >> 172.21.15.83:0/3338464856 conn(0x7ff8580fc880 msgr2=0x7ff8580fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:15.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.820+0000 7ff85ec00640 1 -- 172.21.15.83:0/3338464856 shutdown_connections 2024-04-24T09:09:15.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:15.820+0000 7ff85ec00640 1 -- 172.21.15.83:0/3338464856 wait complete. 2024-04-24T09:09:15.842 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T09:09:16.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.101+0000 7efd1b200640 1 -- 172.21.15.83:0/1257890791 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd1c10c480 msgr2=0x7efd1c10e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:16.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.101+0000 7efd1b200640 1 --2- 172.21.15.83:0/1257890791 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd1c10c480 0x7efd1c10e920 secure :-1 s=READY pgs=2353 cs=0 l=1 rev1=1 crypto rx=0x7efd14009be0 tx=0x7efd1402a750 comp rx=0 tx=0).stop 2024-04-24T09:09:16.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.101+0000 7efd1b200640 1 -- 172.21.15.83:0/1257890791 shutdown_connections 2024-04-24T09:09:16.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.101+0000 7efd1b200640 1 --2- 172.21.15.83:0/1257890791 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd1c10c480 0x7efd1c10e920 unknown :-1 s=CLOSED pgs=2353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:16.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.101+0000 7efd1b200640 1 --2- 172.21.15.83:0/1257890791 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efd1c103980 0x7efd1c10be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:16.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.101+0000 7efd1b200640 1 --2- 172.21.15.83:0/1257890791 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd1c103060 0x7efd1c103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:16.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.101+0000 7efd1b200640 1 -- 172.21.15.83:0/1257890791 >> 172.21.15.83:0/1257890791 conn(0x7efd1c0fc880 msgr2=0x7efd1c0feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:16.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.102+0000 7efd1b200640 1 -- 172.21.15.83:0/1257890791 shutdown_connections 2024-04-24T09:09:16.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.102+0000 7efd1b200640 1 -- 172.21.15.83:0/1257890791 wait complete. 2024-04-24T09:09:16.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.103+0000 7efd1b200640 1 Processor -- start 2024-04-24T09:09:16.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.103+0000 7efd1b200640 1 -- start start 2024-04-24T09:09:16.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.104+0000 7efd1b200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd1c103060 0x7efd1c1a4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:16.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.104+0000 7efd1b200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd1c103980 0x7efd1c1a4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:16.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.104+0000 7efd1b200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efd1c10c480 0x7efd1c19eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:16.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.104+0000 7efd1b200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7efd1c10ff40 con 0x7efd1c103980 2024-04-24T09:09:16.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.104+0000 7efd1b200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efd1c10fdc0 con 0x7efd1c103060 2024-04-24T09:09:16.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.104+0000 7efd1b200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7efd1c1100c0 con 0x7efd1c10c480 2024-04-24T09:09:16.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.104+0000 7efd19e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd1c103060 0x7efd1c1a4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:16.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.104+0000 7efd19e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd1c103060 0x7efd1c1a4940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:53108/0 (socket says 172.21.15.83:53108) 2024-04-24T09:09:16.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.104+0000 7efd19e00640 1 -- 172.21.15.83:0/1494154224 learned_addr learned my addr 172.21.15.83:0/1494154224 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:09:16.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.104+0000 7efd19400640 1 --2- 172.21.15.83:0/1494154224 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd1c103980 0x7efd1c1a4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:16.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.104+0000 7efd1a800640 1 --2- 172.21.15.83:0/1494154224 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efd1c10c480 0x7efd1c19eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:16.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.105+0000 7efd19400640 1 -- 172.21.15.83:0/1494154224 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efd1c10c480 msgr2=0x7efd1c19eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:16.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.105+0000 7efd19400640 1 --2- 172.21.15.83:0/1494154224 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efd1c10c480 0x7efd1c19eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:16.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.105+0000 7efd19400640 1 -- 172.21.15.83:0/1494154224 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd1c103060 msgr2=0x7efd1c1a4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:16.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.105+0000 7efd19400640 1 --2- 172.21.15.83:0/1494154224 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd1c103060 0x7efd1c1a4940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:16.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.105+0000 7efd19400640 1 -- 172.21.15.83:0/1494154224 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd1c19f240 con 0x7efd1c103980 2024-04-24T09:09:16.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.105+0000 7efd19400640 1 --2- 172.21.15.83:0/1494154224 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd1c103980 0x7efd1c1a4e80 secure :-1 s=READY pgs=2354 cs=0 l=1 rev1=1 crypto rx=0x7efd0800b9c0 tx=0x7efd0800be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:16.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.105+0000 7efd12a00640 1 -- 172.21.15.83:0/1494154224 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd0801a070 con 0x7efd1c103980 2024-04-24T09:09:16.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.105+0000 7efd12a00640 1 -- 172.21.15.83:0/1494154224 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efd080054d0 con 0x7efd1c103980 2024-04-24T09:09:16.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.106+0000 7efd12a00640 1 -- 172.21.15.83:0/1494154224 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd08009c30 con 0x7efd1c103980 2024-04-24T09:09:16.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.106+0000 7efd1b200640 1 -- 172.21.15.83:0/1494154224 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd1c19f530 con 0x7efd1c103980 2024-04-24T09:09:16.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.106+0000 7efd1b200640 1 -- 172.21.15.83:0/1494154224 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd1c1953d0 con 0x7efd1c103980 2024-04-24T09:09:16.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.110+0000 7efd23fda740 1 Processor -- start 2024-04-24T09:09:16.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.110+0000 7efd23fda740 1 -- start start 2024-04-24T09:09:16.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.111+0000 7efd23fda740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d2e2f65b40 0x55d2e2f65f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:16.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.111+0000 7efd23fda740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d2e2f66460 0x55d2e2f705f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:16.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.111+0000 7efd23fda740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2e2f70b30 0x55d2e2f72f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:16.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.111+0000 7efd23fda740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d2e2ecb1d0 con 0x55d2e2f66460 2024-04-24T09:09:16.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.111+0000 7efd23fda740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d2e2e71450 con 0x55d2e2f70b30 2024-04-24T09:09:16.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.111+0000 7efd23fda740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d2e2ec7190 con 0x55d2e2f65b40 2024-04-24T09:09:16.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.111+0000 7efd1a800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d2e2f65b40 0x55d2e2f65f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:16.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.112+0000 7efd1a800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d2e2f65b40 0x55d2e2f65f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:50910/0 (socket says 172.21.15.83:50910) 2024-04-24T09:09:16.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.112+0000 7efd1a800640 1 -- 172.21.15.83:0/2704741030 learned_addr learned my addr 172.21.15.83:0/2704741030 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:09:16.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.112+0000 7efd12a00640 1 -- 172.21.15.83:0/1494154224 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7efd0801d070 con 0x7efd1c103980 2024-04-24T09:09:16.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.112+0000 7efd19e00640 1 --2- 172.21.15.83:0/2704741030 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d2e2f66460 0x55d2e2f705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:16.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.113+0000 7efd1a800640 1 -- 172.21.15.83:0/2704741030 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2e2f70b30 msgr2=0x55d2e2f72f20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-24T09:09:16.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.113+0000 7efd1a800640 1 --2- 172.21.15.83:0/2704741030 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2e2f70b30 0x55d2e2f72f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:16.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.113+0000 7efd1a800640 1 -- 172.21.15.83:0/2704741030 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d2e2f66460 msgr2=0x55d2e2f705f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:16.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.113+0000 7efd1a800640 1 --2- 172.21.15.83:0/2704741030 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d2e2f66460 0x55d2e2f705f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:16.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.113+0000 7efd1a800640 1 -- 172.21.15.83:0/2704741030 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2e2f2e310 con 0x55d2e2f65b40 2024-04-24T09:09:16.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.113+0000 7efd19e00640 1 --2- 172.21.15.83:0/2704741030 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d2e2f66460 0x55d2e2f705f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:09:16.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.114+0000 7efd1a800640 1 --2- 172.21.15.83:0/2704741030 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d2e2f65b40 0x55d2e2f65f20 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7efd14033000 tx=0x7efd140303a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6d91702118a0909 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:16.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.114+0000 7efd12a00640 1 --2- 172.21.15.83:0/1494154224 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7efd000a3d60 0x7efd000a6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:16.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.114+0000 7efd07400640 1 -- 172.21.15.83:0/2704741030 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd14040030 con 0x55d2e2f65b40 2024-04-24T09:09:16.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.114+0000 7efd12a00640 1 -- 172.21.15.83:0/1494154224 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 9583+0+0 (secure 0 0 0) 0x7efd080cffb0 con 0x7efd1c103980 2024-04-24T09:09:16.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.114+0000 7efd07400640 1 -- 172.21.15.83:0/2704741030 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efd14037040 con 0x55d2e2f65b40 2024-04-24T09:09:16.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.114+0000 7efd07400640 1 -- 172.21.15.83:0/2704741030 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd14036070 con 0x55d2e2f65b40 2024-04-24T09:09:16.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.114+0000 7efd19e00640 1 --2- 172.21.15.83:0/1494154224 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7efd000a3d60 0x7efd000a6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:16.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.115+0000 7efd23fda740 1 -- 172.21.15.83:0/2704741030 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d2e2f65b40 msgr2=0x55d2e2f65f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:16.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.115+0000 7efd23fda740 1 --2- 172.21.15.83:0/2704741030 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d2e2f65b40 0x55d2e2f65f20 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7efd14033000 tx=0x7efd140303a0 comp rx=0 tx=0).stop 2024-04-24T09:09:16.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.116+0000 7efd19e00640 1 --2- 172.21.15.83:0/1494154224 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7efd000a3d60 0x7efd000a6220 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x55d2e2f67100 tx=0x7efd0c009260 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:16.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.116+0000 7efd23fda740 1 -- 172.21.15.83:0/2704741030 shutdown_connections 2024-04-24T09:09:16.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.116+0000 7efd23fda740 1 --2- 172.21.15.83:0/2704741030 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2e2f70b30 0x55d2e2f72f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:16.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.116+0000 7efd23fda740 1 --2- 172.21.15.83:0/2704741030 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d2e2f66460 0x55d2e2f705f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:16.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.116+0000 7efd23fda740 1 --2- 172.21.15.83:0/2704741030 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d2e2f65b40 0x55d2e2f65f20 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:16.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.116+0000 7efd23fda740 1 -- 172.21.15.83:0/2704741030 >> 172.21.15.83:0/2704741030 conn(0x55d2e2f61e60 msgr2=0x55d2e2f62240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:16.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.116+0000 7efd23fda740 1 -- 172.21.15.83:0/2704741030 shutdown_connections 2024-04-24T09:09:16.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.116+0000 7efd23fda740 1 -- 172.21.15.83:0/2704741030 wait complete. 2024-04-24T09:09:16.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.116+0000 7efd23fda740 1 -- start start 2024-04-24T09:09:16.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.117+0000 7efd23fda740 1 Processor -- start 2024-04-24T09:09:16.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.117+0000 7efd23fda740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2e2f65b40 0x55d2e2f6d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:16.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.117+0000 7efd23fda740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d2e2f66460 0x55d2e2f6de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:16.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.117+0000 7efd23fda740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d2e2f70b30 0x55d2e2f6e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:16.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.117+0000 7efd23fda740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d2e2f739b0 con 0x55d2e2f66460 2024-04-24T09:09:16.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.117+0000 7efd23fda740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d2e2e84810 con 0x55d2e2f65b40 2024-04-24T09:09:16.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.117+0000 7efd23fda740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d2e2f73b30 con 0x55d2e2f70b30 2024-04-24T09:09:16.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.117+0000 7efd23fda740 10 client.0 _ll_register_callbacks cb 0x55d2e2e52140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:09:16.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.118+0000 7efd19400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d2e2f66460 0x55d2e2f6de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:16.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.118+0000 7efd19400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d2e2f66460 0x55d2e2f6de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60306/0 (socket says 172.21.15.83:60306) 2024-04-24T09:09:16.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.118+0000 7efd19400640 1 -- 172.21.15.83:0/4018420387 learned_addr learned my addr 172.21.15.83:0/4018420387 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:09:16.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.118+0000 7efd1a800640 1 --2- 172.21.15.83:0/4018420387 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d2e2f70b30 0x55d2e2f6e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:16.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.118+0000 7efd19e00640 1 --2- 172.21.15.83:0/4018420387 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2e2f65b40 0x55d2e2f6d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:16.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.119+0000 7efd1a800640 1 -- 172.21.15.83:0/4018420387 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2e2f65b40 msgr2=0x55d2e2f6d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:16.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.119+0000 7efd1a800640 1 --2- 172.21.15.83:0/4018420387 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2e2f65b40 0x55d2e2f6d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:16.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.119+0000 7efd1a800640 1 -- 172.21.15.83:0/4018420387 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d2e2f66460 msgr2=0x55d2e2f6de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:16.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.119+0000 7efd1a800640 1 --2- 172.21.15.83:0/4018420387 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d2e2f66460 0x55d2e2f6de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:16.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.119+0000 7efd1a800640 1 -- 172.21.15.83:0/4018420387 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2e2e953f0 con 0x55d2e2f70b30 2024-04-24T09:09:16.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.120+0000 7efd1a800640 1 --2- 172.21.15.83:0/4018420387 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d2e2f70b30 0x55d2e2f6e3d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7efd14009b30 tx=0x7efd1402ac50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70ea168bed35e489 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:16.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.120+0000 7efcfbe00640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-04-24T09:09:16.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.121+0000 7efcfbe00640 1 -- 172.21.15.83:0/4018420387 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd14049030 con 0x55d2e2f70b30 2024-04-24T09:09:16.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.121+0000 7efcfbe00640 1 -- 172.21.15.83:0/4018420387 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9583+0+0 (secure 0 0 0) 0x7efd14030c00 con 0x55d2e2f70b30 2024-04-24T09:09:16.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.122+0000 7efd23fda740 1 -- 172.21.15.83:0/4018420387 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd14037040 con 0x55d2e2f70b30 2024-04-24T09:09:16.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.122+0000 7efcfbe00640 1 -- 172.21.15.83:0/4018420387 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7efd14056050 con 0x55d2e2f70b30 2024-04-24T09:09:16.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.122+0000 7efd23fda740 10 client.28831 fetch_fsmap learned FSMap version 58 2024-04-24T09:09:16.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.122+0000 7efd23fda740 1 -- 172.21.15.83:0/4018420387 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55d2e2bdd3f0 con 0x55d2e2f70b30 2024-04-24T09:09:16.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.123+0000 7efcfbe00640 1 -- 172.21.15.83:0/4018420387 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efd14037040 con 0x55d2e2f70b30 2024-04-24T09:09:16.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.123+0000 7efcfbe00640 1 -- 172.21.15.83:0/4018420387 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 58) ==== 8374+0+0 (secure 0 0 0) 0x7efd14004460 con 0x55d2e2f70b30 2024-04-24T09:09:16.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.123+0000 7efd23fda740 10 client.28831 fetch_fsmap finished waiting for FSMap version 58 2024-04-24T09:09:16.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.123+0000 7efd23fda740 10 client.28831 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-24T09:09:16.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.124+0000 7efd23fda740 20 client.28831 populate_metadata read hostname 'smithi083' 2024-04-24T09:09:16.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.124+0000 7efd23fda740 1 --2- 172.21.15.83:0/4018420387 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55d2e2f80c90 0x55d2e2f83080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:16.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.124+0000 7efd23fda740 4 client.28831 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:09:16.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.124+0000 7efd23fda740 1 -- 172.21.15.83:0/4018420387 --> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d2e2ed8bf0 con 0x55d2e2f80c90 2024-04-24T09:09:16.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.124+0000 7efd19e00640 1 --2- 172.21.15.83:0/4018420387 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55d2e2f80c90 0x55d2e2f83080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:16.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.125+0000 7efd19e00640 1 --2- 172.21.15.83:0/4018420387 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55d2e2f80c90 0x55d2e2f83080 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8df75c741dd272b2 server_cookie=255722af61f03d2f in_seq=0 out_seq=0 2024-04-24T09:09:16.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.125+0000 7efcfbe00640 10 client.28831 ms_handle_connect on v2:172.21.15.138:6834/1909219758 2024-04-24T09:09:16.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.126+0000 7efcfbe00640 1 -- 172.21.15.83:0/4018420387 <== mds.2 v2:172.21.15.138:6834/1909219758 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d2e2ed8bf0 con 0x55d2e2f80c90 2024-04-24T09:09:16.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.126+0000 7efcfbe00640 10 client.28831 handle_command_reply: tid=0 2024-04-24T09:09:16.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.126+0000 7efd23fda740 1 client.28831 shutdown 2024-04-24T09:09:16.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.126+0000 7efd23fda740 1 -- 172.21.15.83:0/4018420387 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d2e2f70b30 msgr2=0x55d2e2f6e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:16.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.126+0000 7efd23fda740 1 --2- 172.21.15.83:0/4018420387 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d2e2f70b30 0x55d2e2f6e3d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7efd14009b30 tx=0x7efd1402ac50 comp rx=0 tx=0).stop 2024-04-24T09:09:16.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.127+0000 7efd23fda740 1 -- 172.21.15.83:0/4018420387 shutdown_connections 2024-04-24T09:09:16.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.127+0000 7efd23fda740 1 --2- 172.21.15.83:0/4018420387 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55d2e2f80c90 0x55d2e2f83080 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:16.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.127+0000 7efcfbe00640 0 client.28831 ms_handle_reset on v2:172.21.15.138:6834/1909219758 2024-04-24T09:09:16.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.127+0000 7efd23fda740 1 --2- 172.21.15.83:0/4018420387 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d2e2f70b30 0x55d2e2f6e3d0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:16.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.127+0000 7efd23fda740 1 --2- 172.21.15.83:0/4018420387 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d2e2f66460 0x55d2e2f6de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:16.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.127+0000 7efd23fda740 1 --2- 172.21.15.83:0/4018420387 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2e2f65b40 0x55d2e2f6d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:16.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.127+0000 7efd23fda740 1 -- 172.21.15.83:0/4018420387 >> 172.21.15.83:0/4018420387 conn(0x55d2e2f61e60 msgr2=0x55d2e2f67b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:16.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.127+0000 7efd23fda740 1 -- 172.21.15.83:0/4018420387 shutdown_connections 2024-04-24T09:09:16.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.127+0000 7efd23fda740 1 -- 172.21.15.83:0/4018420387 wait complete. 2024-04-24T09:09:16.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.127+0000 7efd23fda740 20 client.28831 trim_cache size 0 max 16384 2024-04-24T09:09:16.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.151+0000 7efd23fda740 1 Processor -- start 2024-04-24T09:09:16.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.152+0000 7efd23fda740 1 -- start start 2024-04-24T09:09:16.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.152+0000 7efd23fda740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d2e2f65b40 0x55d2e2f6e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:16.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.152+0000 7efd23fda740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d2e2f66460 0x55d2e2f6c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:16.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.152+0000 7efd23fda740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2e2f70b30 0x55d2e2f71f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:16.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.152+0000 7efd23fda740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7efd1c1be3b0 con 0x55d2e2f65b40 2024-04-24T09:09:16.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.152+0000 7efd23fda740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efd1c1bfef0 con 0x55d2e2f70b30 2024-04-24T09:09:16.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.152+0000 7efd23fda740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7efd1c1bea80 con 0x55d2e2f66460 2024-04-24T09:09:16.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.152+0000 7efd19e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d2e2f65b40 0x55d2e2f6e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:16.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.152+0000 7efd1a800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2e2f70b30 0x55d2e2f71f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:16.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.153+0000 7efd1a800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2e2f70b30 0x55d2e2f71f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:53132/0 (socket says 172.21.15.83:53132) 2024-04-24T09:09:16.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.153+0000 7efd1a800640 1 -- 172.21.15.83:0/2077495327 learned_addr learned my addr 172.21.15.83:0/2077495327 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:09:16.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.153+0000 7efd19e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d2e2f65b40 0x55d2e2f6e1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60314/0 (socket says 172.21.15.83:60314) 2024-04-24T09:09:16.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.153+0000 7efd19400640 1 --2- 172.21.15.83:0/2077495327 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d2e2f66460 0x55d2e2f6c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:16.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.153+0000 7efd1a800640 1 -- 172.21.15.83:0/2077495327 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d2e2f66460 msgr2=0x55d2e2f6c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:16.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.153+0000 7efd1a800640 1 --2- 172.21.15.83:0/2077495327 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d2e2f66460 0x55d2e2f6c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:16.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.153+0000 7efd1a800640 1 -- 172.21.15.83:0/2077495327 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d2e2f65b40 msgr2=0x55d2e2f6e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:16.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.153+0000 7efd1a800640 1 --2- 172.21.15.83:0/2077495327 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d2e2f65b40 0x55d2e2f6e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:16.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.153+0000 7efd1a800640 1 -- 172.21.15.83:0/2077495327 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2e2ca77d0 con 0x55d2e2f70b30 2024-04-24T09:09:16.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.153+0000 7efd19e00640 1 --2- 172.21.15.83:0/2077495327 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d2e2f65b40 0x55d2e2f6e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:09:16.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.153+0000 7efd1a800640 1 --2- 172.21.15.83:0/2077495327 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2e2f70b30 0x55d2e2f71f20 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7efd14005f00 tx=0x7efd14009300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4427393945a001fb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:16.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.154+0000 7efd06a00640 1 -- 172.21.15.83:0/2077495327 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd14049040 con 0x55d2e2f70b30 2024-04-24T09:09:16.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.154+0000 7efd06a00640 1 -- 172.21.15.83:0/2077495327 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efd14030990 con 0x55d2e2f70b30 2024-04-24T09:09:16.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.154+0000 7efd06a00640 1 -- 172.21.15.83:0/2077495327 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd14040040 con 0x55d2e2f70b30 2024-04-24T09:09:16.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.155+0000 7efd23fda740 1 -- 172.21.15.83:0/2077495327 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2e2f70b30 msgr2=0x55d2e2f71f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:16.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.155+0000 7efd23fda740 1 --2- 172.21.15.83:0/2077495327 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2e2f70b30 0x55d2e2f71f20 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7efd14005f00 tx=0x7efd14009300 comp rx=0 tx=0).stop 2024-04-24T09:09:16.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.155+0000 7efd23fda740 1 -- 172.21.15.83:0/2077495327 shutdown_connections 2024-04-24T09:09:16.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.155+0000 7efd23fda740 1 --2- 172.21.15.83:0/2077495327 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2e2f70b30 0x55d2e2f71f20 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:16.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.155+0000 7efd23fda740 1 --2- 172.21.15.83:0/2077495327 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d2e2f66460 0x55d2e2f6c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:16.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.155+0000 7efd23fda740 1 --2- 172.21.15.83:0/2077495327 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d2e2f65b40 0x55d2e2f6e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:16.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.155+0000 7efd23fda740 1 -- 172.21.15.83:0/2077495327 >> 172.21.15.83:0/2077495327 conn(0x55d2e2f61e60 msgr2=0x55d2e2f6c110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:16.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.155+0000 7efd23fda740 1 -- 172.21.15.83:0/2077495327 shutdown_connections 2024-04-24T09:09:16.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.156+0000 7efd23fda740 1 -- 172.21.15.83:0/2077495327 wait complete. 2024-04-24T09:09:16.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.157+0000 7efd23fda740 1 -- start start 2024-04-24T09:09:16.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.157+0000 7efd23fda740 1 Processor -- start 2024-04-24T09:09:16.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.158+0000 7efd23fda740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d2e2f65b40 0x55d2e2f81260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:16.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.158+0000 7efd23fda740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d2e2f66460 0x55d2e2f817a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:16.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.158+0000 7efd23fda740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2e2f70b30 0x55d2e2fe4e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:16.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.158+0000 7efd23fda740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7efd1c1bf820 con 0x55d2e2f66460 2024-04-24T09:09:16.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.158+0000 7efd23fda740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efd1c1bf150 con 0x55d2e2f70b30 2024-04-24T09:09:16.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.158+0000 7efd23fda740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7efd1c19d780 con 0x55d2e2f65b40 2024-04-24T09:09:16.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.158+0000 7efd23fda740 10 client.0 _ll_register_callbacks cb 0x7efd0c005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:09:16.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.158+0000 7efd1a800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2e2f70b30 0x55d2e2fe4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:16.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.158+0000 7efd19400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d2e2f66460 0x55d2e2f817a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:16.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.158+0000 7efd1a800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2e2f70b30 0x55d2e2fe4e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:53144/0 (socket says 172.21.15.83:53144) 2024-04-24T09:09:16.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.158+0000 7efd1a800640 1 -- 172.21.15.83:0/3446394220 learned_addr learned my addr 172.21.15.83:0/3446394220 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:09:16.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.159+0000 7efd19400640 1 -- 172.21.15.83:0/3446394220 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d2e2f65b40 msgr2=0x55d2e2f81260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:16.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.159+0000 7efd19e00640 1 --2- 172.21.15.83:0/3446394220 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d2e2f65b40 0x55d2e2f81260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:16.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.159+0000 7efd19400640 1 --2- 172.21.15.83:0/3446394220 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d2e2f65b40 0x55d2e2f81260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:16.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.159+0000 7efd19400640 1 -- 172.21.15.83:0/3446394220 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2e2f70b30 msgr2=0x55d2e2fe4e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:16.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.159+0000 7efd19400640 1 --2- 172.21.15.83:0/3446394220 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2e2f70b30 0x55d2e2fe4e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:16.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.159+0000 7efd19400640 1 -- 172.21.15.83:0/3446394220 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2e2fe5390 con 0x55d2e2f66460 2024-04-24T09:09:16.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.159+0000 7efd1a800640 1 --2- 172.21.15.83:0/3446394220 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2e2f70b30 0x55d2e2fe4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:09:16.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.159+0000 7efd19400640 1 --2- 172.21.15.83:0/3446394220 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d2e2f66460 0x55d2e2f817a0 secure :-1 s=READY pgs=2355 cs=0 l=1 rev1=1 crypto rx=0x7efd080cd8e0 tx=0x7efd0809b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82ba9b6ac77abdec server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:16.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.159+0000 7efcfbe00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:09:16.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.160+0000 7efcfbe00640 1 -- 172.21.15.83:0/3446394220 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd0809ce60 con 0x55d2e2f66460 2024-04-24T09:09:16.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.160+0000 7efcfbe00640 1 -- 172.21.15.83:0/3446394220 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9583+0+0 (secure 0 0 0) 0x7efd080cd560 con 0x55d2e2f66460 2024-04-24T09:09:16.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.160+0000 7efd23fda740 1 -- 172.21.15.83:0/3446394220 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd14030990 con 0x55d2e2f66460 2024-04-24T09:09:16.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.160+0000 7efcfbe00640 1 -- 172.21.15.83:0/3446394220 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7efd0809c230 con 0x55d2e2f66460 2024-04-24T09:09:16.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.161+0000 7efd23fda740 10 client.21747 fetch_fsmap learned FSMap version 58 2024-04-24T09:09:16.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.161+0000 7efd23fda740 1 -- 172.21.15.83:0/3446394220 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55d2e2fe6670 con 0x55d2e2f66460 2024-04-24T09:09:16.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.162+0000 7efcfbe00640 1 -- 172.21.15.83:0/3446394220 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efd14030990 con 0x55d2e2f66460 2024-04-24T09:09:16.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.162+0000 7efcfbe00640 1 -- 172.21.15.83:0/3446394220 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 58) ==== 8374+0+0 (secure 0 0 0) 0x7efd0809af80 con 0x55d2e2f66460 2024-04-24T09:09:16.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.162+0000 7efd23fda740 10 client.21747 fetch_fsmap finished waiting for FSMap version 58 2024-04-24T09:09:16.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.162+0000 7efd23fda740 10 client.21747 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-24T09:09:16.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.162+0000 7efd23fda740 20 client.21747 populate_metadata read hostname 'smithi083' 2024-04-24T09:09:16.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.162+0000 7efd23fda740 1 --2- 172.21.15.83:0/3446394220 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55d2e2fe6df0 0x55d2e2fe9210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:16.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.162+0000 7efd23fda740 4 client.21747 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:09:16.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.162+0000 7efd23fda740 1 -- 172.21.15.83:0/3446394220 --> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d2e2ed2a60 con 0x55d2e2fe6df0 2024-04-24T09:09:16.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.163+0000 7efd19e00640 1 --2- 172.21.15.83:0/3446394220 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55d2e2fe6df0 0x55d2e2fe9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:16.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.164+0000 7efd19e00640 1 --2- 172.21.15.83:0/3446394220 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55d2e2fe6df0 0x55d2e2fe9210 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e9e0b6ce9884453a server_cookie=da31962d723221f4 in_seq=0 out_seq=0 2024-04-24T09:09:16.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.164+0000 7efcfbe00640 10 client.21747 ms_handle_connect on v2:172.21.15.138:6834/1909219758 2024-04-24T09:09:16.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.164+0000 7efcfbe00640 1 -- 172.21.15.83:0/3446394220 <== mds.2 v2:172.21.15.138:6834/1909219758 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d2e2ed2a60 con 0x55d2e2fe6df0 2024-04-24T09:09:16.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.164+0000 7efcfbe00640 10 client.21747 handle_command_reply: tid=0 2024-04-24T09:09:16.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.164+0000 7efd23fda740 1 client.21747 shutdown 2024-04-24T09:09:16.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.165+0000 7efd23fda740 1 -- 172.21.15.83:0/3446394220 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d2e2f66460 msgr2=0x55d2e2f817a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:16.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.165+0000 7efd23fda740 1 --2- 172.21.15.83:0/3446394220 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d2e2f66460 0x55d2e2f817a0 secure :-1 s=READY pgs=2355 cs=0 l=1 rev1=1 crypto rx=0x7efd080cd8e0 tx=0x7efd0809b9f0 comp rx=0 tx=0).stop 2024-04-24T09:09:16.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.165+0000 7efd23fda740 1 -- 172.21.15.83:0/3446394220 shutdown_connections 2024-04-24T09:09:16.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.165+0000 7efd23fda740 1 --2- 172.21.15.83:0/3446394220 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55d2e2fe6df0 0x55d2e2fe9210 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:16.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.165+0000 7efd23fda740 1 --2- 172.21.15.83:0/3446394220 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2e2f70b30 0x55d2e2fe4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:16.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.165+0000 7efd23fda740 1 --2- 172.21.15.83:0/3446394220 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d2e2f66460 0x55d2e2f817a0 unknown :-1 s=CLOSED pgs=2355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:16.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.165+0000 7efd23fda740 1 --2- 172.21.15.83:0/3446394220 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d2e2f65b40 0x55d2e2f81260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:16.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.165+0000 7efd23fda740 1 -- 172.21.15.83:0/3446394220 >> 172.21.15.83:0/3446394220 conn(0x55d2e2f61e60 msgr2=0x55d2e2f6e120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:16.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.165+0000 7efcfbe00640 0 client.21747 ms_handle_reset on v2:172.21.15.138:6834/1909219758 2024-04-24T09:09:16.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.165+0000 7efd23fda740 1 -- 172.21.15.83:0/3446394220 shutdown_connections 2024-04-24T09:09:16.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.166+0000 7efd23fda740 1 -- 172.21.15.83:0/3446394220 wait complete. 2024-04-24T09:09:16.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.166+0000 7efd23fda740 20 client.21747 trim_cache size 0 max 16384 2024-04-24T09:09:16.166 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:09:16.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.168+0000 7efd1b200640 1 -- 172.21.15.83:0/1494154224 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7efd000a3d60 msgr2=0x7efd000a6220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:16.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.168+0000 7efd1b200640 1 --2- 172.21.15.83:0/1494154224 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7efd000a3d60 0x7efd000a6220 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x55d2e2f67100 tx=0x7efd0c009260 comp rx=0 tx=0).stop 2024-04-24T09:09:16.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.168+0000 7efd1b200640 1 -- 172.21.15.83:0/1494154224 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd1c103980 msgr2=0x7efd1c1a4e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:16.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.168+0000 7efd1b200640 1 --2- 172.21.15.83:0/1494154224 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd1c103980 0x7efd1c1a4e80 secure :-1 s=READY pgs=2354 cs=0 l=1 rev1=1 crypto rx=0x7efd0800b9c0 tx=0x7efd0800be80 comp rx=0 tx=0).stop 2024-04-24T09:09:16.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.169+0000 7efd1b200640 1 -- 172.21.15.83:0/1494154224 shutdown_connections 2024-04-24T09:09:16.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.169+0000 7efd1b200640 1 --2- 172.21.15.83:0/1494154224 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7efd000a3d60 0x7efd000a6220 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:16.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.169+0000 7efd1b200640 1 --2- 172.21.15.83:0/1494154224 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efd1c10c480 0x7efd1c19eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:16.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.169+0000 7efd1b200640 1 --2- 172.21.15.83:0/1494154224 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd1c103980 0x7efd1c1a4e80 unknown :-1 s=CLOSED pgs=2354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:16.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.169+0000 7efd1b200640 1 --2- 172.21.15.83:0/1494154224 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd1c103060 0x7efd1c1a4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:16.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.169+0000 7efd1b200640 1 -- 172.21.15.83:0/1494154224 >> 172.21.15.83:0/1494154224 conn(0x7efd1c0fc880 msgr2=0x7efd1c0fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:16.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.169+0000 7efd1b200640 1 -- 172.21.15.83:0/1494154224 shutdown_connections 2024-04-24T09:09:16.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.169+0000 7efd1b200640 1 -- 172.21.15.83:0/1494154224 wait complete. 2024-04-24T09:09:16.169 INFO:tasks.workunit.client.0.smithi083.stdout: 10 29238 7.03 MB/sec execute 107 sec latency 77.286 ms 2024-04-24T09:09:16.187 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:09:15 smithi083 ceph-mon[27907]: pgmap v556: 129 pgs: 129 active+clean; 546 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.7 MiB/s wr, 266 op/s 2024-04-24T09:09:16.187 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:09:15 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/3011951614' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:09:16.190 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T09:09:16.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:09:15 smithi138 ceph-mon[31441]: pgmap v556: 129 pgs: 129 active+clean; 546 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.7 MiB/s wr, 266 op/s 2024-04-24T09:09:16.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:09:15 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/3011951614' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:09:16.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:09:15 smithi189 ceph-mon[31791]: pgmap v556: 129 pgs: 129 active+clean; 546 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.7 MiB/s wr, 266 op/s 2024-04-24T09:09:16.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:09:15 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/3011951614' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:09:16.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.436+0000 7fe706c00640 1 -- 172.21.15.83:0/1463226231 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe70010e7f0 msgr2=0x7fe700110be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:16.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.436+0000 7fe706c00640 1 --2- 172.21.15.83:0/1463226231 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe70010e7f0 0x7fe700110be0 secure :-1 s=READY pgs=2356 cs=0 l=1 rev1=1 crypto rx=0x7fe6f8009ca0 tx=0x7fe6f802a8b0 comp rx=0 tx=0).stop 2024-04-24T09:09:16.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.436+0000 7fe706c00640 1 -- 172.21.15.83:0/1463226231 shutdown_connections 2024-04-24T09:09:16.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.436+0000 7fe706c00640 1 --2- 172.21.15.83:0/1463226231 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe70010e7f0 0x7fe700110be0 unknown :-1 s=CLOSED pgs=2356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:16.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.436+0000 7fe706c00640 1 --2- 172.21.15.83:0/1463226231 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe7001015f0 0x7fe70010e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:16.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.436+0000 7fe706c00640 1 --2- 172.21.15.83:0/1463226231 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe700100c40 0x7fe700101020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:16.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.436+0000 7fe706c00640 1 -- 172.21.15.83:0/1463226231 >> 172.21.15.83:0/1463226231 conn(0x7fe7000fc880 msgr2=0x7fe7000feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:16.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.436+0000 7fe706c00640 1 -- 172.21.15.83:0/1463226231 shutdown_connections 2024-04-24T09:09:16.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.437+0000 7fe706c00640 1 -- 172.21.15.83:0/1463226231 wait complete. 2024-04-24T09:09:16.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.438+0000 7fe706c00640 1 Processor -- start 2024-04-24T09:09:16.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.438+0000 7fe706c00640 1 -- start start 2024-04-24T09:09:16.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.438+0000 7fe706c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe700100c40 0x7fe700073770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:16.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.438+0000 7fe706c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7001015f0 0x7fe700073cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:16.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.438+0000 7fe706c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe70010e7f0 0x7fe70006ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:16.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.438+0000 7fe706c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe700114340 con 0x7fe700100c40 2024-04-24T09:09:16.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.438+0000 7fe706c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe7001141c0 con 0x7fe7001015f0 2024-04-24T09:09:16.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.438+0000 7fe706c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe7001144c0 con 0x7fe70010e7f0 2024-04-24T09:09:16.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.439+0000 7fe705800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe700100c40 0x7fe700073770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:16.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.439+0000 7fe705800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe700100c40 0x7fe700073770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60336/0 (socket says 172.21.15.83:60336) 2024-04-24T09:09:16.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.439+0000 7fe705800640 1 -- 172.21.15.83:0/398030053 learned_addr learned my addr 172.21.15.83:0/398030053 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:09:16.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.439+0000 7fe704e00640 1 --2- 172.21.15.83:0/398030053 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7001015f0 0x7fe700073cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:16.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.439+0000 7fe706200640 1 --2- 172.21.15.83:0/398030053 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe70010e7f0 0x7fe70006ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:16.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.439+0000 7fe705800640 1 -- 172.21.15.83:0/398030053 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe70010e7f0 msgr2=0x7fe70006ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:16.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.439+0000 7fe705800640 1 --2- 172.21.15.83:0/398030053 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe70010e7f0 0x7fe70006ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:16.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.439+0000 7fe705800640 1 -- 172.21.15.83:0/398030053 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7001015f0 msgr2=0x7fe700073cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:16.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.439+0000 7fe705800640 1 --2- 172.21.15.83:0/398030053 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7001015f0 0x7fe700073cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:16.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.439+0000 7fe705800640 1 -- 172.21.15.83:0/398030053 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe70006d2e0 con 0x7fe700100c40 2024-04-24T09:09:16.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.440+0000 7fe705800640 1 --2- 172.21.15.83:0/398030053 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe700100c40 0x7fe700073770 secure :-1 s=READY pgs=2357 cs=0 l=1 rev1=1 crypto rx=0x7fe6f400ea60 tx=0x7fe6f400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:16.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.440+0000 7fe6fe000640 1 -- 172.21.15.83:0/398030053 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f400cde0 con 0x7fe700100c40 2024-04-24T09:09:16.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.440+0000 7fe6fe000640 1 -- 172.21.15.83:0/398030053 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe6f40035f0 con 0x7fe700100c40 2024-04-24T09:09:16.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.440+0000 7fe6fe000640 1 -- 172.21.15.83:0/398030053 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f4010690 con 0x7fe700100c40 2024-04-24T09:09:16.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.440+0000 7fe706c00640 1 -- 172.21.15.83:0/398030053 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe70006d5d0 con 0x7fe700100c40 2024-04-24T09:09:16.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.440+0000 7fe706c00640 1 -- 172.21.15.83:0/398030053 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe700071530 con 0x7fe700100c40 2024-04-24T09:09:16.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.444+0000 7fe70afe0740 1 Processor -- start 2024-04-24T09:09:16.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.445+0000 7fe70afe0740 1 -- start start 2024-04-24T09:09:16.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.445+0000 7fe70afe0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b029a4b40 0x563b029a4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:16.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.445+0000 7fe70afe0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b029a5460 0x563b029af5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:16.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.445+0000 7fe70afe0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563b029afb30 0x563b029b1f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:16.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.445+0000 7fe70afe0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x563b0290a1d0 con 0x563b029a5460 2024-04-24T09:09:16.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.445+0000 7fe70afe0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563b028b0450 con 0x563b029afb30 2024-04-24T09:09:16.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.445+0000 7fe70afe0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x563b02906190 con 0x563b029a4b40 2024-04-24T09:09:16.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.446+0000 7fe704e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563b029afb30 0x563b029b1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:16.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.446+0000 7fe704e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563b029afb30 0x563b029b1f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:53170/0 (socket says 172.21.15.83:53170) 2024-04-24T09:09:16.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.446+0000 7fe704e00640 1 -- 172.21.15.83:0/1081203277 learned_addr learned my addr 172.21.15.83:0/1081203277 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:09:16.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.446+0000 7fe706200640 1 --2- 172.21.15.83:0/1081203277 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b029a4b40 0x563b029a4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:16.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.446+0000 7fe706200640 1 -- 172.21.15.83:0/1081203277 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563b029afb30 msgr2=0x563b029b1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:16.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.446+0000 7fe706200640 1 --2- 172.21.15.83:0/1081203277 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563b029afb30 0x563b029b1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:16.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.447+0000 7fe706200640 1 -- 172.21.15.83:0/1081203277 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b029a5460 msgr2=0x563b029af5f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T09:09:16.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.447+0000 7fe706200640 1 --2- 172.21.15.83:0/1081203277 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b029a5460 0x563b029af5f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:16.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.447+0000 7fe706200640 1 -- 172.21.15.83:0/1081203277 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b0296d310 con 0x563b029a4b40 2024-04-24T09:09:16.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.447+0000 7fe6fe000640 1 -- 172.21.15.83:0/398030053 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe6f40037a0 con 0x7fe700100c40 2024-04-24T09:09:16.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.448+0000 7fe706200640 1 --2- 172.21.15.83:0/1081203277 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b029a4b40 0x563b029a4f20 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe6f802add0 tx=0x7fe6f8009a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e581eba3d326021 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:16.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.449+0000 7fe6fe000640 1 --2- 172.21.15.83:0/398030053 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe6e80a3e20 0x7fe6e80a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:16.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.449+0000 7fe6f2a00640 1 -- 172.21.15.83:0/1081203277 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f80305a0 con 0x563b029a4b40 2024-04-24T09:09:16.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.449+0000 7fe6f2a00640 1 -- 172.21.15.83:0/1081203277 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe6f8036070 con 0x563b029a4b40 2024-04-24T09:09:16.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.449+0000 7fe6f2a00640 1 -- 172.21.15.83:0/1081203277 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f80042a0 con 0x563b029a4b40 2024-04-24T09:09:16.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.449+0000 7fe6fe000640 1 -- 172.21.15.83:0/398030053 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 9583+0+0 (secure 0 0 0) 0x7fe6f40d0fb0 con 0x7fe700100c40 2024-04-24T09:09:16.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.449+0000 7fe705800640 1 --2- 172.21.15.83:0/398030053 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe6e80a3e20 0x7fe6e80a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:16.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.450+0000 7fe70afe0740 1 -- 172.21.15.83:0/1081203277 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b029a4b40 msgr2=0x563b029a4f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:16.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.450+0000 7fe705800640 1 --2- 172.21.15.83:0/398030053 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe6e80a3e20 0x7fe6e80a62e0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x563b029a6100 tx=0x7fe6f409c8a0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:16.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.450+0000 7fe70afe0740 1 --2- 172.21.15.83:0/1081203277 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b029a4b40 0x563b029a4f20 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe6f802add0 tx=0x7fe6f8009a20 comp rx=0 tx=0).stop 2024-04-24T09:09:16.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.450+0000 7fe70afe0740 1 -- 172.21.15.83:0/1081203277 shutdown_connections 2024-04-24T09:09:16.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.450+0000 7fe70afe0740 1 --2- 172.21.15.83:0/1081203277 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563b029afb30 0x563b029b1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:16.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.450+0000 7fe70afe0740 1 --2- 172.21.15.83:0/1081203277 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b029a5460 0x563b029af5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:16.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.450+0000 7fe70afe0740 1 --2- 172.21.15.83:0/1081203277 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b029a4b40 0x563b029a4f20 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:16.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.450+0000 7fe70afe0740 1 -- 172.21.15.83:0/1081203277 >> 172.21.15.83:0/1081203277 conn(0x563b029a0e60 msgr2=0x563b029a1240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:16.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.450+0000 7fe70afe0740 1 -- 172.21.15.83:0/1081203277 shutdown_connections 2024-04-24T09:09:16.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.450+0000 7fe70afe0740 1 -- 172.21.15.83:0/1081203277 wait complete. 2024-04-24T09:09:16.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.451+0000 7fe70afe0740 1 -- start start 2024-04-24T09:09:16.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.451+0000 7fe70afe0740 1 Processor -- start 2024-04-24T09:09:16.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.452+0000 7fe70afe0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b029a4b40 0x563b029af5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:16.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.452+0000 7fe70afe0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b029a5460 0x563b029ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:16.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.452+0000 7fe70afe0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563b029afb30 0x563b029ad010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:16.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.452+0000 7fe70afe0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x563b029b2960 con 0x563b029a4b40 2024-04-24T09:09:16.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.452+0000 7fe70afe0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563b028c3810 con 0x563b029afb30 2024-04-24T09:09:16.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.452+0000 7fe70afe0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x563b029b2ae0 con 0x563b029a5460 2024-04-24T09:09:16.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.452+0000 7fe70afe0740 10 client.0 _ll_register_callbacks cb 0x563b02891140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:09:16.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.453+0000 7fe706200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b029a4b40 0x563b029af5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:16.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.453+0000 7fe704e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b029a5460 0x563b029ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:16.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.453+0000 7fe704e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b029a5460 0x563b029ac8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:50960/0 (socket says 172.21.15.83:50960) 2024-04-24T09:09:16.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.453+0000 7fe704e00640 1 -- 172.21.15.83:0/363494083 learned_addr learned my addr 172.21.15.83:0/363494083 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:09:16.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.453+0000 7fe706200640 1 -- 172.21.15.83:0/363494083 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b029a5460 msgr2=0x563b029ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:16.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.453+0000 7fe706200640 1 --2- 172.21.15.83:0/363494083 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b029a5460 0x563b029ac8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:16.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.453+0000 7fe706200640 1 -- 172.21.15.83:0/363494083 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563b029afb30 msgr2=0x563b029ad010 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T09:09:16.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.453+0000 7fe706200640 1 --2- 172.21.15.83:0/363494083 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563b029afb30 0x563b029ad010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:16.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.453+0000 7fe706200640 1 -- 172.21.15.83:0/363494083 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b028d43f0 con 0x563b029a4b40 2024-04-24T09:09:16.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.454+0000 7fe706200640 1 --2- 172.21.15.83:0/363494083 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b029a4b40 0x563b029af5e0 secure :-1 s=READY pgs=2358 cs=0 l=1 rev1=1 crypto rx=0x7fe6f8004af0 tx=0x7fe6f8030710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f5e760be7b6a54c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:16.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.454+0000 7fe6e7400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:09:16.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.454+0000 7fe6e7400640 1 -- 172.21.15.83:0/363494083 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f8003c70 con 0x563b029a4b40 2024-04-24T09:09:16.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.455+0000 7fe6e7400640 1 -- 172.21.15.83:0/363494083 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9583+0+0 (secure 0 0 0) 0x7fe6f8030860 con 0x563b029a4b40 2024-04-24T09:09:16.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.455+0000 7fe70afe0740 1 -- 172.21.15.83:0/363494083 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6f8036070 con 0x563b029a4b40 2024-04-24T09:09:16.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.455+0000 7fe6e7400640 1 -- 172.21.15.83:0/363494083 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7fe6f8030a60 con 0x563b029a4b40 2024-04-24T09:09:16.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.455+0000 7fe70afe0740 10 client.21756 fetch_fsmap learned FSMap version 58 2024-04-24T09:09:16.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.455+0000 7fe70afe0740 1 -- 172.21.15.83:0/363494083 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x563b0261c3f0 con 0x563b029a4b40 2024-04-24T09:09:16.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.456+0000 7fe6e7400640 1 -- 172.21.15.83:0/363494083 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe6f8036070 con 0x563b029a4b40 2024-04-24T09:09:16.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.456+0000 7fe6e7400640 1 -- 172.21.15.83:0/363494083 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 58) ==== 8374+0+0 (secure 0 0 0) 0x7fe6f804d070 con 0x563b029a4b40 2024-04-24T09:09:16.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.456+0000 7fe70afe0740 10 client.21756 fetch_fsmap finished waiting for FSMap version 58 2024-04-24T09:09:16.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.456+0000 7fe70afe0740 10 client.21756 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-24T09:09:16.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.456+0000 7fe70afe0740 20 client.21756 populate_metadata read hostname 'smithi083' 2024-04-24T09:09:16.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.456+0000 7fe70afe0740 1 --2- 172.21.15.83:0/363494083 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x563b029bfbc0 0x563b029c1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:16.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.456+0000 7fe70afe0740 4 client.21756 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:09:16.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.456+0000 7fe70afe0740 1 -- 172.21.15.83:0/363494083 --> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563b02917bf0 con 0x563b029bfbc0 2024-04-24T09:09:16.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.457+0000 7fe704e00640 1 --2- 172.21.15.83:0/363494083 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x563b029bfbc0 0x563b029c1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:16.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.458+0000 7fe704e00640 1 --2- 172.21.15.83:0/363494083 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x563b029bfbc0 0x563b029c1fb0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4a6fcbd0b99deeb8 server_cookie=29a5ebf871b70d5a in_seq=0 out_seq=0 2024-04-24T09:09:16.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.458+0000 7fe6e7400640 10 client.21756 ms_handle_connect on v2:172.21.15.138:6836/900566754 2024-04-24T09:09:16.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.459+0000 7fe6e7400640 1 -- 172.21.15.83:0/363494083 <== mds.1 v2:172.21.15.138:6836/900566754 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563b02917bf0 con 0x563b029bfbc0 2024-04-24T09:09:16.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.459+0000 7fe6e7400640 10 client.21756 handle_command_reply: tid=0 2024-04-24T09:09:16.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.459+0000 7fe70afe0740 1 client.21756 shutdown 2024-04-24T09:09:16.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.459+0000 7fe70afe0740 1 -- 172.21.15.83:0/363494083 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b029a4b40 msgr2=0x563b029af5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:16.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.459+0000 7fe70afe0740 1 --2- 172.21.15.83:0/363494083 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b029a4b40 0x563b029af5e0 secure :-1 s=READY pgs=2358 cs=0 l=1 rev1=1 crypto rx=0x7fe6f8004af0 tx=0x7fe6f8030710 comp rx=0 tx=0).stop 2024-04-24T09:09:16.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.459+0000 7fe70afe0740 1 -- 172.21.15.83:0/363494083 shutdown_connections 2024-04-24T09:09:16.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.459+0000 7fe70afe0740 1 --2- 172.21.15.83:0/363494083 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563b029afb30 0x563b029ad010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:16.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.459+0000 7fe70afe0740 1 --2- 172.21.15.83:0/363494083 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x563b029bfbc0 0x563b029c1fb0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:16.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.459+0000 7fe70afe0740 1 --2- 172.21.15.83:0/363494083 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b029a5460 0x563b029ac8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:16.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.459+0000 7fe70afe0740 1 --2- 172.21.15.83:0/363494083 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b029a4b40 0x563b029af5e0 unknown :-1 s=CLOSED pgs=2358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:16.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.459+0000 7fe70afe0740 1 -- 172.21.15.83:0/363494083 >> 172.21.15.83:0/363494083 conn(0x563b029a0e60 msgr2=0x563b029a6b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:16.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.459+0000 7fe6e7400640 0 client.21756 ms_handle_reset on v2:172.21.15.138:6836/900566754 2024-04-24T09:09:16.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.460+0000 7fe70afe0740 1 -- 172.21.15.83:0/363494083 shutdown_connections 2024-04-24T09:09:16.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.460+0000 7fe70afe0740 1 -- 172.21.15.83:0/363494083 wait complete. 2024-04-24T09:09:16.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.461+0000 7fe70afe0740 20 client.21756 trim_cache size 0 max 16384 2024-04-24T09:09:16.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.481+0000 7fe70afe0740 1 Processor -- start 2024-04-24T09:09:16.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.482+0000 7fe70afe0740 1 -- start start 2024-04-24T09:09:16.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.482+0000 7fe70afe0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b029a4b40 0x563b029af110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:16.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.482+0000 7fe70afe0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b029a5460 0x563b029c75b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:16.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.482+0000 7fe70afe0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563b029afb30 0x563b029c7af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:16.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.482+0000 7fe70afe0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe7001b9db0 con 0x563b029a5460 2024-04-24T09:09:16.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.482+0000 7fe70afe0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe7001bb8f0 con 0x563b029afb30 2024-04-24T09:09:16.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.482+0000 7fe70afe0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe7001ba480 con 0x563b029a4b40 2024-04-24T09:09:16.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.482+0000 7fe704e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b029a5460 0x563b029c75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:16.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.482+0000 7fe704e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b029a5460 0x563b029c75b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60350/0 (socket says 172.21.15.83:60350) 2024-04-24T09:09:16.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.482+0000 7fe704e00640 1 -- 172.21.15.83:0/2459190747 learned_addr learned my addr 172.21.15.83:0/2459190747 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:09:16.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.483+0000 7fe706200640 1 --2- 172.21.15.83:0/2459190747 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563b029afb30 0x563b029c7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:16.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.483+0000 7fe704e00640 1 -- 172.21.15.83:0/2459190747 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b029a4b40 msgr2=0x563b029af110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:16.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.483+0000 7fe704e00640 1 --2- 172.21.15.83:0/2459190747 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b029a4b40 0x563b029af110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:16.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.483+0000 7fe704e00640 1 -- 172.21.15.83:0/2459190747 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563b029afb30 msgr2=0x563b029c7af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:16.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.483+0000 7fe704e00640 1 --2- 172.21.15.83:0/2459190747 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563b029afb30 0x563b029c7af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:16.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.483+0000 7fe704e00640 1 -- 172.21.15.83:0/2459190747 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b029ad380 con 0x563b029a5460 2024-04-24T09:09:16.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.484+0000 7fe704e00640 1 --2- 172.21.15.83:0/2459190747 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b029a5460 0x563b029c75b0 secure :-1 s=READY pgs=2359 cs=0 l=1 rev1=1 crypto rx=0x563b029ac130 tx=0x7fe6ec00cd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94bce987aa1e3bc7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:16.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.484+0000 7fe6f2000640 1 -- 172.21.15.83:0/2459190747 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6ec022070 con 0x563b029a5460 2024-04-24T09:09:16.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.484+0000 7fe6f2000640 1 -- 172.21.15.83:0/2459190747 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe6ec01c8e0 con 0x563b029a5460 2024-04-24T09:09:16.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.484+0000 7fe6f2000640 1 -- 172.21.15.83:0/2459190747 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6ec010c90 con 0x563b029a5460 2024-04-24T09:09:16.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.485+0000 7fe70afe0740 1 -- 172.21.15.83:0/2459190747 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b029a5460 msgr2=0x563b029c75b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:16.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.485+0000 7fe70afe0740 1 --2- 172.21.15.83:0/2459190747 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b029a5460 0x563b029c75b0 secure :-1 s=READY pgs=2359 cs=0 l=1 rev1=1 crypto rx=0x563b029ac130 tx=0x7fe6ec00cd90 comp rx=0 tx=0).stop 2024-04-24T09:09:16.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.485+0000 7fe70afe0740 1 -- 172.21.15.83:0/2459190747 shutdown_connections 2024-04-24T09:09:16.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.485+0000 7fe70afe0740 1 --2- 172.21.15.83:0/2459190747 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563b029afb30 0x563b029c7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:16.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.485+0000 7fe70afe0740 1 --2- 172.21.15.83:0/2459190747 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b029a5460 0x563b029c75b0 unknown :-1 s=CLOSED pgs=2359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:16.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.485+0000 7fe70afe0740 1 --2- 172.21.15.83:0/2459190747 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b029a4b40 0x563b029af110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:16.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.485+0000 7fe70afe0740 1 -- 172.21.15.83:0/2459190747 >> 172.21.15.83:0/2459190747 conn(0x563b029a0e60 msgr2=0x563b029a6290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:16.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.486+0000 7fe70afe0740 1 -- 172.21.15.83:0/2459190747 shutdown_connections 2024-04-24T09:09:16.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.486+0000 7fe70afe0740 1 -- 172.21.15.83:0/2459190747 wait complete. 2024-04-24T09:09:16.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.486+0000 7fe70afe0740 1 -- start start 2024-04-24T09:09:16.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.487+0000 7fe70afe0740 1 Processor -- start 2024-04-24T09:09:16.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.490+0000 7fe70afe0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b029a4b40 0x563b029b7d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:16.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.490+0000 7fe70afe0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563b029a5460 0x563b029b82b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:16.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.490+0000 7fe70afe0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b029afb30 0x563b029b2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:16.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.490+0000 7fe70afe0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe7001bb220 con 0x563b029afb30 2024-04-24T09:09:16.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.490+0000 7fe70afe0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe7001bab50 con 0x563b029a5460 2024-04-24T09:09:16.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.490+0000 7fe70afe0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe7001b7db0 con 0x563b029a4b40 2024-04-24T09:09:16.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.490+0000 7fe70afe0740 10 client.0 _ll_register_callbacks cb 0x7fe6ec00c350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:09:16.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.490+0000 7fe706200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b029afb30 0x563b029b2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:16.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.490+0000 7fe706200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b029afb30 0x563b029b2e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60362/0 (socket says 172.21.15.83:60362) 2024-04-24T09:09:16.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.490+0000 7fe706200640 1 -- 172.21.15.83:0/3133529704 learned_addr learned my addr 172.21.15.83:0/3133529704 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:09:16.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.491+0000 7fe706200640 1 --2- 172.21.15.83:0/3133529704 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b029a4b40 0x563b029b7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:16.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.491+0000 7fe704e00640 1 --2- 172.21.15.83:0/3133529704 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563b029a5460 0x563b029b82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:16.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.491+0000 7fe706200640 1 -- 172.21.15.83:0/3133529704 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b029a4b40 msgr2=0x563b029b7d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:16.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.491+0000 7fe706200640 1 --2- 172.21.15.83:0/3133529704 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b029a4b40 0x563b029b7d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:16.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.491+0000 7fe706200640 1 -- 172.21.15.83:0/3133529704 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563b029a5460 msgr2=0x563b029b82b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:16.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.491+0000 7fe706200640 1 --2- 172.21.15.83:0/3133529704 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563b029a5460 0x563b029b82b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:16.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.491+0000 7fe706200640 1 -- 172.21.15.83:0/3133529704 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b026e67d0 con 0x563b029afb30 2024-04-24T09:09:16.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.492+0000 7fe706200640 1 --2- 172.21.15.83:0/3133529704 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b029afb30 0x563b029b2e10 secure :-1 s=READY pgs=2360 cs=0 l=1 rev1=1 crypto rx=0x7fe6f802add0 tx=0x7fe6f8046040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5614750559cd44b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:16.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.492+0000 7fe6e7400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:09:16.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.492+0000 7fe6e7400640 1 -- 172.21.15.83:0/3133529704 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f8060070 con 0x563b029afb30 2024-04-24T09:09:16.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.492+0000 7fe70afe0740 1 -- 172.21.15.83:0/3133529704 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6ec01c8e0 con 0x563b029afb30 2024-04-24T09:09:16.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.492+0000 7fe6e7400640 1 -- 172.21.15.83:0/3133529704 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9583+0+0 (secure 0 0 0) 0x7fe6f8058a30 con 0x563b029afb30 2024-04-24T09:09:16.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.492+0000 7fe6e7400640 1 -- 172.21.15.83:0/3133529704 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7fe6f80587f0 con 0x563b029afb30 2024-04-24T09:09:16.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.492+0000 7fe70afe0740 10 client.21762 fetch_fsmap learned FSMap version 58 2024-04-24T09:09:16.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.492+0000 7fe70afe0740 1 -- 172.21.15.83:0/3133529704 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x563b02a26800 con 0x563b029afb30 2024-04-24T09:09:16.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.493+0000 7fe6e7400640 1 -- 172.21.15.83:0/3133529704 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe6ec01c8e0 con 0x563b029afb30 2024-04-24T09:09:16.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.493+0000 7fe6e7400640 1 -- 172.21.15.83:0/3133529704 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 58) ==== 8374+0+0 (secure 0 0 0) 0x7fe6f8031a60 con 0x563b029afb30 2024-04-24T09:09:16.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.493+0000 7fe70afe0740 10 client.21762 fetch_fsmap finished waiting for FSMap version 58 2024-04-24T09:09:16.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.493+0000 7fe70afe0740 10 client.21762 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-24T09:09:16.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.493+0000 7fe70afe0740 20 client.21762 populate_metadata read hostname 'smithi083' 2024-04-24T09:09:16.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.493+0000 7fe70afe0740 1 --2- 172.21.15.83:0/3133529704 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x563b02a26f50 0x563b02a29370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:16.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.493+0000 7fe70afe0740 4 client.21762 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:09:16.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.493+0000 7fe70afe0740 1 -- 172.21.15.83:0/3133529704 --> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563b02911a60 con 0x563b02a26f50 2024-04-24T09:09:16.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.494+0000 7fe704e00640 1 --2- 172.21.15.83:0/3133529704 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x563b02a26f50 0x563b02a29370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:16.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.495+0000 7fe704e00640 1 --2- 172.21.15.83:0/3133529704 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x563b02a26f50 0x563b02a29370 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=43cb78e9b7b1bc22 server_cookie=e7035ce67f868d3a in_seq=0 out_seq=0 2024-04-24T09:09:16.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.495+0000 7fe6e7400640 10 client.21762 ms_handle_connect on v2:172.21.15.138:6836/900566754 2024-04-24T09:09:16.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.495+0000 7fe6e7400640 1 -- 172.21.15.83:0/3133529704 <== mds.1 v2:172.21.15.138:6836/900566754 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563b02911a60 con 0x563b02a26f50 2024-04-24T09:09:16.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.495+0000 7fe6e7400640 10 client.21762 handle_command_reply: tid=0 2024-04-24T09:09:16.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.495+0000 7fe70afe0740 1 client.21762 shutdown 2024-04-24T09:09:16.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.496+0000 7fe70afe0740 1 -- 172.21.15.83:0/3133529704 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b029afb30 msgr2=0x563b029b2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:16.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.496+0000 7fe70afe0740 1 --2- 172.21.15.83:0/3133529704 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b029afb30 0x563b029b2e10 secure :-1 s=READY pgs=2360 cs=0 l=1 rev1=1 crypto rx=0x7fe6f802add0 tx=0x7fe6f8046040 comp rx=0 tx=0).stop 2024-04-24T09:09:16.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.496+0000 7fe70afe0740 1 -- 172.21.15.83:0/3133529704 shutdown_connections 2024-04-24T09:09:16.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.496+0000 7fe70afe0740 1 --2- 172.21.15.83:0/3133529704 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b029afb30 0x563b029b2e10 unknown :-1 s=CLOSED pgs=2360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:16.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.496+0000 7fe70afe0740 1 --2- 172.21.15.83:0/3133529704 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563b029a5460 0x563b029b82b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:16.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.496+0000 7fe70afe0740 1 --2- 172.21.15.83:0/3133529704 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x563b02a26f50 0x563b02a29370 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:16.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.496+0000 7fe70afe0740 1 --2- 172.21.15.83:0/3133529704 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b029a4b40 0x563b029b7d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:16.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.496+0000 7fe70afe0740 1 -- 172.21.15.83:0/3133529704 >> 172.21.15.83:0/3133529704 conn(0x563b029a0e60 msgr2=0x563b0299ff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:16.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.496+0000 7fe6e7400640 0 client.21762 ms_handle_reset on v2:172.21.15.138:6836/900566754 2024-04-24T09:09:16.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.496+0000 7fe70afe0740 1 -- 172.21.15.83:0/3133529704 shutdown_connections 2024-04-24T09:09:16.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.496+0000 7fe70afe0740 1 -- 172.21.15.83:0/3133529704 wait complete. 2024-04-24T09:09:16.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.496+0000 7fe70afe0740 20 client.21762 trim_cache size 0 max 16384 2024-04-24T09:09:16.497 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:09:16.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.498+0000 7fe706c00640 1 -- 172.21.15.83:0/398030053 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe6e80a3e20 msgr2=0x7fe6e80a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:16.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.498+0000 7fe706c00640 1 --2- 172.21.15.83:0/398030053 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe6e80a3e20 0x7fe6e80a62e0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x563b029a6100 tx=0x7fe6f409c8a0 comp rx=0 tx=0).stop 2024-04-24T09:09:16.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.498+0000 7fe706c00640 1 -- 172.21.15.83:0/398030053 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe700100c40 msgr2=0x7fe700073770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:16.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.498+0000 7fe706c00640 1 --2- 172.21.15.83:0/398030053 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe700100c40 0x7fe700073770 secure :-1 s=READY pgs=2357 cs=0 l=1 rev1=1 crypto rx=0x7fe6f400ea60 tx=0x7fe6f400ef20 comp rx=0 tx=0).stop 2024-04-24T09:09:16.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.498+0000 7fe706c00640 1 -- 172.21.15.83:0/398030053 shutdown_connections 2024-04-24T09:09:16.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.498+0000 7fe706c00640 1 --2- 172.21.15.83:0/398030053 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe6e80a3e20 0x7fe6e80a62e0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:16.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.498+0000 7fe706c00640 1 --2- 172.21.15.83:0/398030053 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe70010e7f0 0x7fe70006ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:16.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.498+0000 7fe706c00640 1 --2- 172.21.15.83:0/398030053 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7001015f0 0x7fe700073cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:16.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.498+0000 7fe706c00640 1 --2- 172.21.15.83:0/398030053 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe700100c40 0x7fe700073770 unknown :-1 s=CLOSED pgs=2357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:16.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.498+0000 7fe706c00640 1 -- 172.21.15.83:0/398030053 >> 172.21.15.83:0/398030053 conn(0x7fe7000fc880 msgr2=0x7fe70010fa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:16.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.498+0000 7fe706c00640 1 -- 172.21.15.83:0/398030053 shutdown_connections 2024-04-24T09:09:16.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.498+0000 7fe706c00640 1 -- 172.21.15.83:0/398030053 wait complete. 2024-04-24T09:09:16.521 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T09:09:16.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.787+0000 7f4650600640 1 -- 172.21.15.83:0/2232212703 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f464810a800 msgr2=0x7f46481111d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:16.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.787+0000 7f4650600640 1 --2- 172.21.15.83:0/2232212703 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f464810a800 0x7f46481111d0 secure :-1 s=READY pgs=2361 cs=0 l=1 rev1=1 crypto rx=0x7f4640009ca0 tx=0x7f464002a8c0 comp rx=0 tx=0).stop 2024-04-24T09:09:16.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.788+0000 7f4650600640 1 -- 172.21.15.83:0/2232212703 shutdown_connections 2024-04-24T09:09:16.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.788+0000 7f4650600640 1 --2- 172.21.15.83:0/2232212703 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f464810a800 0x7f46481111d0 unknown :-1 s=CLOSED pgs=2361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:16.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.788+0000 7f4650600640 1 --2- 172.21.15.83:0/2232212703 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4648100d60 0x7f464810a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:16.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.788+0000 7f4650600640 1 --2- 172.21.15.83:0/2232212703 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f46481003b0 0x7f4648100790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:16.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.788+0000 7f4650600640 1 -- 172.21.15.83:0/2232212703 >> 172.21.15.83:0/2232212703 conn(0x7f4648077060 msgr2=0x7f46480fe450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:16.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.788+0000 7f4650600640 1 -- 172.21.15.83:0/2232212703 shutdown_connections 2024-04-24T09:09:16.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.788+0000 7f4650600640 1 -- 172.21.15.83:0/2232212703 wait complete. 2024-04-24T09:09:16.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.789+0000 7f4650600640 1 Processor -- start 2024-04-24T09:09:16.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.789+0000 7f4650600640 1 -- start start 2024-04-24T09:09:16.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.789+0000 7f4650600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46481003b0 0x7f46481a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:16.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.789+0000 7f4650600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f464810a800 0x7f46481a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:16.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.789+0000 7f4650600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f464819a810 0x7f464819acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:16.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.790+0000 7f464dc00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46481003b0 0x7f46481a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:16.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.790+0000 7f4650600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4648114510 con 0x7f464819a810 2024-04-24T09:09:16.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.790+0000 7f4650600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4648114390 con 0x7f46481003b0 2024-04-24T09:09:16.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.790+0000 7f4650600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4648114690 con 0x7f464810a800 2024-04-24T09:09:16.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.790+0000 7f464e600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f464819a810 0x7f464819acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:16.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.790+0000 7f464dc00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46481003b0 0x7f46481a06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:53210/0 (socket says 172.21.15.83:53210) 2024-04-24T09:09:16.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.790+0000 7f464dc00640 1 -- 172.21.15.83:0/171125463 learned_addr learned my addr 172.21.15.83:0/171125463 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:09:16.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.790+0000 7f464d200640 1 --2- 172.21.15.83:0/171125463 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f464810a800 0x7f46481a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:16.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.791+0000 7f464dc00640 1 -- 172.21.15.83:0/171125463 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f464810a800 msgr2=0x7f46481a0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:16.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.791+0000 7f464dc00640 1 --2- 172.21.15.83:0/171125463 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f464810a800 0x7f46481a0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:16.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.791+0000 7f464dc00640 1 -- 172.21.15.83:0/171125463 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f464819a810 msgr2=0x7f464819acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:16.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.791+0000 7f464dc00640 1 --2- 172.21.15.83:0/171125463 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f464819a810 0x7f464819acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:16.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.791+0000 7f464dc00640 1 -- 172.21.15.83:0/171125463 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f464819b3a0 con 0x7f46481003b0 2024-04-24T09:09:16.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.791+0000 7f464dc00640 1 --2- 172.21.15.83:0/171125463 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46481003b0 0x7f46481a06a0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f463c00d950 tx=0x7f463c00de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:16.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.791+0000 7f4646000640 1 -- 172.21.15.83:0/171125463 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f463c017070 con 0x7f46481003b0 2024-04-24T09:09:16.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.791+0000 7f4650600640 1 -- 172.21.15.83:0/171125463 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f464819b690 con 0x7f46481003b0 2024-04-24T09:09:16.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.791+0000 7f4650600640 1 -- 172.21.15.83:0/171125463 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46481a78c0 con 0x7f46481003b0 2024-04-24T09:09:16.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.792+0000 7f4646000640 1 -- 172.21.15.83:0/171125463 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f463c009c30 con 0x7f46481003b0 2024-04-24T09:09:16.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.792+0000 7f4646000640 1 -- 172.21.15.83:0/171125463 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f463c00f620 con 0x7f46481003b0 2024-04-24T09:09:16.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.795+0000 7f4646000640 1 -- 172.21.15.83:0/171125463 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f463c010460 con 0x7f46481003b0 2024-04-24T09:09:16.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.796+0000 7f4653433740 1 Processor -- start 2024-04-24T09:09:16.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.796+0000 7f4653433740 1 -- start start 2024-04-24T09:09:16.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.797+0000 7f4653433740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5603594516a0 0x560359451a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:16.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.797+0000 7f4653433740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560359452050 0x560359464c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:16.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.797+0000 7f4653433740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560359465150 0x560359467540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:16.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.797+0000 7f4653433740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5603593b71d0 con 0x560359465150 2024-04-24T09:09:16.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.797+0000 7f4653433740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56035935d450 con 0x5603594516a0 2024-04-24T09:09:16.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.797+0000 7f4653433740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5603593b3190 con 0x560359452050 2024-04-24T09:09:16.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.797+0000 7f4646000640 1 --2- 172.21.15.83:0/171125463 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f46300a3de0 0x7f46300a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:16.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.798+0000 7f4646000640 1 -- 172.21.15.83:0/171125463 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 9583+0+0 (secure 0 0 0) 0x7f463c0d07b0 con 0x7f46481003b0 2024-04-24T09:09:16.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.798+0000 7f464e600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5603594516a0 0x560359451a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:16.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.798+0000 7f464e600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5603594516a0 0x560359451a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:53226/0 (socket says 172.21.15.83:53226) 2024-04-24T09:09:16.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.798+0000 7f464e600640 1 -- 172.21.15.83:0/1801874768 learned_addr learned my addr 172.21.15.83:0/1801874768 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:09:16.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.798+0000 7f464d200640 1 --2- 172.21.15.83:0/1801874768 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560359465150 0x560359467540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:16.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.798+0000 7f464e600640 1 --2- 172.21.15.83:0/171125463 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f46300a3de0 0x7f46300a62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:16.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.798+0000 7f464d200640 1 -- 172.21.15.83:0/1801874768 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560359452050 msgr2=0x560359464c10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-24T09:09:16.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.798+0000 7f464d200640 1 --2- 172.21.15.83:0/1801874768 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560359452050 0x560359464c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:16.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.798+0000 7f464d200640 1 -- 172.21.15.83:0/1801874768 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5603594516a0 msgr2=0x560359451a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:16.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.798+0000 7f464d200640 1 --2- 172.21.15.83:0/1801874768 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5603594516a0 0x560359451a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:16.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.798+0000 7f464d200640 1 -- 172.21.15.83:0/1801874768 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56035941a310 con 0x560359465150 2024-04-24T09:09:16.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.799+0000 7f464e600640 1 --2- 172.21.15.83:0/171125463 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f46300a3de0 0x7f46300a62a0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f4640004af0 tx=0x7f4640031490 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:16.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.799+0000 7f464d200640 1 --2- 172.21.15.83:0/1801874768 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560359465150 0x560359467540 secure :-1 s=READY pgs=2362 cs=0 l=1 rev1=1 crypto rx=0x7f464819bb60 tx=0x7f463400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21838afa80250196 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:16.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.800+0000 7f463aa00640 1 -- 172.21.15.83:0/1801874768 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f463400dea0 con 0x560359465150 2024-04-24T09:09:16.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.800+0000 7f463aa00640 1 -- 172.21.15.83:0/1801874768 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f46340054d0 con 0x560359465150 2024-04-24T09:09:16.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.800+0000 7f463aa00640 1 -- 172.21.15.83:0/1801874768 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4634011440 con 0x560359465150 2024-04-24T09:09:16.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.801+0000 7f4653433740 1 -- 172.21.15.83:0/1801874768 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560359465150 msgr2=0x560359467540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:16.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.801+0000 7f4653433740 1 --2- 172.21.15.83:0/1801874768 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560359465150 0x560359467540 secure :-1 s=READY pgs=2362 cs=0 l=1 rev1=1 crypto rx=0x7f464819bb60 tx=0x7f463400f730 comp rx=0 tx=0).stop 2024-04-24T09:09:16.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.801+0000 7f4653433740 1 -- 172.21.15.83:0/1801874768 shutdown_connections 2024-04-24T09:09:16.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.801+0000 7f4653433740 1 --2- 172.21.15.83:0/1801874768 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560359465150 0x560359467540 unknown :-1 s=CLOSED pgs=2362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:16.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.801+0000 7f4653433740 1 --2- 172.21.15.83:0/1801874768 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560359452050 0x560359464c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:16.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.801+0000 7f4653433740 1 --2- 172.21.15.83:0/1801874768 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5603594516a0 0x560359451a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:16.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.802+0000 7f4653433740 1 -- 172.21.15.83:0/1801874768 >> 172.21.15.83:0/1801874768 conn(0x56035944dc70 msgr2=0x56035944e050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:16.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.802+0000 7f4653433740 1 -- 172.21.15.83:0/1801874768 shutdown_connections 2024-04-24T09:09:16.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.802+0000 7f4653433740 1 -- 172.21.15.83:0/1801874768 wait complete. 2024-04-24T09:09:16.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.802+0000 7f4653433740 1 -- start start 2024-04-24T09:09:16.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.803+0000 7f4653433740 1 Processor -- start 2024-04-24T09:09:16.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.804+0000 7f4653433740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5603594516a0 0x560359464be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:16.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.804+0000 7f4653433740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560359452050 0x560359461ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:16.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.804+0000 7f4653433740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560359465150 0x5603594625c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:16.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.804+0000 7f4653433740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x560359467fb0 con 0x560359452050 2024-04-24T09:09:16.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.804+0000 7f4653433740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560359370810 con 0x5603594516a0 2024-04-24T09:09:16.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.804+0000 7f4653433740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x560359468130 con 0x560359465150 2024-04-24T09:09:16.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.804+0000 7f4653433740 10 client.0 _ll_register_callbacks cb 0x56035933e140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:09:16.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.804+0000 7f464d200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560359452050 0x560359461ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:16.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.804+0000 7f464d200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560359452050 0x560359461ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60412/0 (socket says 172.21.15.83:60412) 2024-04-24T09:09:16.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.804+0000 7f464d200640 1 -- 172.21.15.83:0/2402025107 learned_addr learned my addr 172.21.15.83:0/2402025107 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:09:16.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.804+0000 7f464e600640 1 --2- 172.21.15.83:0/2402025107 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560359465150 0x5603594625c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:16.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.805+0000 7f464d200640 1 -- 172.21.15.83:0/2402025107 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560359465150 msgr2=0x5603594625c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:16.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.805+0000 7f464d200640 1 --2- 172.21.15.83:0/2402025107 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560359465150 0x5603594625c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:16.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.805+0000 7f464d200640 1 -- 172.21.15.83:0/2402025107 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5603594516a0 msgr2=0x560359464be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:16.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.805+0000 7f464d200640 1 --2- 172.21.15.83:0/2402025107 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5603594516a0 0x560359464be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:16.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.805+0000 7f464d200640 1 -- 172.21.15.83:0/2402025107 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603593813f0 con 0x560359452050 2024-04-24T09:09:16.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.805+0000 7f464d200640 1 --2- 172.21.15.83:0/2402025107 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560359452050 0x560359461ed0 secure :-1 s=READY pgs=2363 cs=0 l=1 rev1=1 crypto rx=0x7f4634019000 tx=0x7f463400ded0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e5a51543dba832f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:16.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.805+0000 7f462f400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:09:16.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.806+0000 7f462f400640 1 -- 172.21.15.83:0/2402025107 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f463402b060 con 0x560359452050 2024-04-24T09:09:16.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.806+0000 7f462f400640 1 -- 172.21.15.83:0/2402025107 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9583+0+0 (secure 0 0 0) 0x7f46340048d0 con 0x560359452050 2024-04-24T09:09:16.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.806+0000 7f4653433740 1 -- 172.21.15.83:0/2402025107 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46340054d0 con 0x560359452050 2024-04-24T09:09:16.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.807+0000 7f462f400640 1 -- 172.21.15.83:0/2402025107 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f4634004d60 con 0x560359452050 2024-04-24T09:09:16.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.807+0000 7f4653433740 10 client.21774 fetch_fsmap learned FSMap version 58 2024-04-24T09:09:16.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.807+0000 7f4653433740 1 -- 172.21.15.83:0/2402025107 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x5603590c93f0 con 0x560359452050 2024-04-24T09:09:16.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.808+0000 7f462f400640 1 -- 172.21.15.83:0/2402025107 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f46340054d0 con 0x560359452050 2024-04-24T09:09:16.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.808+0000 7f462f400640 1 -- 172.21.15.83:0/2402025107 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 58) ==== 8374+0+0 (secure 0 0 0) 0x7f463401ba90 con 0x560359452050 2024-04-24T09:09:16.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.808+0000 7f4653433740 10 client.21774 fetch_fsmap finished waiting for FSMap version 58 2024-04-24T09:09:16.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.808+0000 7f4653433740 10 client.21774 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:09:16.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.809+0000 7f4653433740 20 client.21774 populate_metadata read hostname 'smithi083' 2024-04-24T09:09:16.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.809+0000 7f4653433740 1 --2- 172.21.15.83:0/2402025107 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x560359475290 0x560359477680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:16.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.809+0000 7f4653433740 4 client.21774 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:09:16.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.809+0000 7f4653433740 1 -- 172.21.15.83:0/2402025107 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5603593c4bf0 con 0x560359475290 2024-04-24T09:09:16.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.810+0000 7f464dc00640 1 --2- 172.21.15.83:0/2402025107 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x560359475290 0x560359477680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:16.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.811+0000 7f464dc00640 1 --2- 172.21.15.83:0/2402025107 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x560359475290 0x560359477680 crc :-1 s=READY pgs=443 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24b4cf13223b3d71 server_cookie=5b1583c956a998c6 in_seq=0 out_seq=0 2024-04-24T09:09:16.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.811+0000 7f462f400640 10 client.21774 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:09:16.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.812+0000 7f462f400640 1 -- 172.21.15.83:0/2402025107 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5603593c4bf0 con 0x560359475290 2024-04-24T09:09:16.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.812+0000 7f462f400640 10 client.21774 handle_command_reply: tid=0 2024-04-24T09:09:16.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.812+0000 7f4653433740 1 client.21774 shutdown 2024-04-24T09:09:16.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.812+0000 7f4653433740 1 -- 172.21.15.83:0/2402025107 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560359452050 msgr2=0x560359461ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:16.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.812+0000 7f4653433740 1 --2- 172.21.15.83:0/2402025107 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560359452050 0x560359461ed0 secure :-1 s=READY pgs=2363 cs=0 l=1 rev1=1 crypto rx=0x7f4634019000 tx=0x7f463400ded0 comp rx=0 tx=0).stop 2024-04-24T09:09:16.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.813+0000 7f4653433740 1 -- 172.21.15.83:0/2402025107 shutdown_connections 2024-04-24T09:09:16.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.813+0000 7f4653433740 1 --2- 172.21.15.83:0/2402025107 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x560359475290 0x560359477680 crc :-1 s=READY pgs=443 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:16.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.813+0000 7f4653433740 1 --2- 172.21.15.83:0/2402025107 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560359465150 0x5603594625c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:16.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.813+0000 7f4653433740 1 --2- 172.21.15.83:0/2402025107 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560359452050 0x560359461ed0 unknown :-1 s=CLOSED pgs=2363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:16.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.813+0000 7f4653433740 1 --2- 172.21.15.83:0/2402025107 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5603594516a0 0x560359464be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:16.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.813+0000 7f462f400640 0 client.21774 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:09:16.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.813+0000 7f4653433740 1 -- 172.21.15.83:0/2402025107 >> 172.21.15.83:0/2402025107 conn(0x56035944dc70 msgr2=0x560359453ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:16.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.813+0000 7f4653433740 1 -- 172.21.15.83:0/2402025107 shutdown_connections 2024-04-24T09:09:16.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.813+0000 7f4653433740 1 -- 172.21.15.83:0/2402025107 wait complete. 2024-04-24T09:09:16.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.813+0000 7f4653433740 20 client.21774 trim_cache size 0 max 16384 2024-04-24T09:09:16.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.841+0000 7f4653433740 1 Processor -- start 2024-04-24T09:09:16.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.841+0000 7f4653433740 1 -- start start 2024-04-24T09:09:16.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.842+0000 7f4653433740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5603594516a0 0x56035944d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:16.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.842+0000 7f4653433740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560359452050 0x560359464710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:16.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.842+0000 7f4653433740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560359465150 0x56035947cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:16.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.842+0000 7f4653433740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4628010100 con 0x560359465150 2024-04-24T09:09:16.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.842+0000 7f4653433740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4628011c40 con 0x5603594516a0 2024-04-24T09:09:16.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.842+0000 7f4653433740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f46280107d0 con 0x560359452050 2024-04-24T09:09:16.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.842+0000 7f464e600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560359465150 0x56035947cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:16.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.842+0000 7f464e600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560359465150 0x56035947cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60422/0 (socket says 172.21.15.83:60422) 2024-04-24T09:09:16.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.842+0000 7f464e600640 1 -- 172.21.15.83:0/1349646662 learned_addr learned my addr 172.21.15.83:0/1349646662 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:09:16.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.842+0000 7f464dc00640 1 --2- 172.21.15.83:0/1349646662 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5603594516a0 0x56035944d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:16.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.843+0000 7f464d200640 1 --2- 172.21.15.83:0/1349646662 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560359452050 0x560359464710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:16.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.843+0000 7f464e600640 1 -- 172.21.15.83:0/1349646662 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560359452050 msgr2=0x560359464710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:16.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.843+0000 7f464e600640 1 --2- 172.21.15.83:0/1349646662 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560359452050 0x560359464710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:16.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.843+0000 7f464e600640 1 -- 172.21.15.83:0/1349646662 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5603594516a0 msgr2=0x56035944d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:16.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.843+0000 7f464e600640 1 --2- 172.21.15.83:0/1349646662 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5603594516a0 0x56035944d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:16.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.843+0000 7f464e600640 1 -- 172.21.15.83:0/1349646662 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603591937d0 con 0x560359465150 2024-04-24T09:09:16.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.843+0000 7f464dc00640 1 --2- 172.21.15.83:0/1349646662 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5603594516a0 0x56035944d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:09:16.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.843+0000 7f464e600640 1 --2- 172.21.15.83:0/1349646662 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560359465150 0x56035947cd10 secure :-1 s=READY pgs=2364 cs=0 l=1 rev1=1 crypto rx=0x7f4640009420 tx=0x7f464003f070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af11f15a49b3d46e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:16.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.844+0000 7f463a000640 1 -- 172.21.15.83:0/1349646662 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4640055070 con 0x560359465150 2024-04-24T09:09:16.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.844+0000 7f463a000640 1 -- 172.21.15.83:0/1349646662 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4640036070 con 0x560359465150 2024-04-24T09:09:16.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.844+0000 7f463a000640 1 -- 172.21.15.83:0/1349646662 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4640030b10 con 0x560359465150 2024-04-24T09:09:16.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.845+0000 7f4653433740 1 -- 172.21.15.83:0/1349646662 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560359465150 msgr2=0x56035947cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:16.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.845+0000 7f4653433740 1 --2- 172.21.15.83:0/1349646662 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560359465150 0x56035947cd10 secure :-1 s=READY pgs=2364 cs=0 l=1 rev1=1 crypto rx=0x7f4640009420 tx=0x7f464003f070 comp rx=0 tx=0).stop 2024-04-24T09:09:16.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.845+0000 7f4653433740 1 -- 172.21.15.83:0/1349646662 shutdown_connections 2024-04-24T09:09:16.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.845+0000 7f4653433740 1 --2- 172.21.15.83:0/1349646662 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560359465150 0x56035947cd10 unknown :-1 s=CLOSED pgs=2364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:16.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.845+0000 7f4653433740 1 --2- 172.21.15.83:0/1349646662 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560359452050 0x560359464710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:16.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.845+0000 7f4653433740 1 --2- 172.21.15.83:0/1349646662 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5603594516a0 0x56035944d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:16.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.845+0000 7f4653433740 1 -- 172.21.15.83:0/1349646662 >> 172.21.15.83:0/1349646662 conn(0x56035944dc70 msgr2=0x56035945d290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:16.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.845+0000 7f4653433740 1 -- 172.21.15.83:0/1349646662 shutdown_connections 2024-04-24T09:09:16.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.845+0000 7f4653433740 1 -- 172.21.15.83:0/1349646662 wait complete. 2024-04-24T09:09:16.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.846+0000 7f4653433740 1 -- start start 2024-04-24T09:09:16.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.846+0000 7f4653433740 1 Processor -- start 2024-04-24T09:09:16.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.847+0000 7f4653433740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5603594516a0 0x56035945ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:16.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.847+0000 7f4653433740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560359452050 0x56035945f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:16.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.847+0000 7f4653433740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560359465150 0x56035946b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:16.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.847+0000 7f4653433740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4628011570 con 0x560359465150 2024-04-24T09:09:16.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.847+0000 7f4653433740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4628010ea0 con 0x560359452050 2024-04-24T09:09:16.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.847+0000 7f4653433740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f462800e100 con 0x5603594516a0 2024-04-24T09:09:16.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.847+0000 7f4653433740 10 client.0 _ll_register_callbacks cb 0x7f4634015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:09:16.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.847+0000 7f464d200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560359452050 0x56035945f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:16.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.847+0000 7f464d200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560359452050 0x56035945f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:53262/0 (socket says 172.21.15.83:53262) 2024-04-24T09:09:16.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.847+0000 7f464e600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560359465150 0x56035946b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:16.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.847+0000 7f464d200640 1 -- 172.21.15.83:0/2816396035 learned_addr learned my addr 172.21.15.83:0/2816396035 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:09:16.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.847+0000 7f464dc00640 1 --2- 172.21.15.83:0/2816396035 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5603594516a0 0x56035945ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:16.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.848+0000 7f464e600640 1 -- 172.21.15.83:0/2816396035 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5603594516a0 msgr2=0x56035945ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:16.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.848+0000 7f464e600640 1 --2- 172.21.15.83:0/2816396035 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5603594516a0 0x56035945ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:16.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.848+0000 7f464e600640 1 -- 172.21.15.83:0/2816396035 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560359452050 msgr2=0x56035945f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:16.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.848+0000 7f464e600640 1 --2- 172.21.15.83:0/2816396035 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560359452050 0x56035945f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:16.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.848+0000 7f464e600640 1 -- 172.21.15.83:0/2816396035 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56035946bec0 con 0x560359465150 2024-04-24T09:09:16.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.848+0000 7f464e600640 1 --2- 172.21.15.83:0/2816396035 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560359465150 0x56035946b980 secure :-1 s=READY pgs=2365 cs=0 l=1 rev1=1 crypto rx=0x7f464002f3a0 tx=0x7f4640004a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b21b57aef9c8a978 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:16.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.848+0000 7f462f400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:09:16.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.849+0000 7f462f400640 1 -- 172.21.15.83:0/2816396035 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4640060060 con 0x560359465150 2024-04-24T09:09:16.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.849+0000 7f462f400640 1 -- 172.21.15.83:0/2816396035 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9583+0+0 (secure 0 0 0) 0x7f4640030aa0 con 0x560359465150 2024-04-24T09:09:16.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.849+0000 7f4653433740 1 -- 172.21.15.83:0/2816396035 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4640036070 con 0x560359465150 2024-04-24T09:09:16.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.849+0000 7f462f400640 1 -- 172.21.15.83:0/2816396035 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f4640055030 con 0x560359465150 2024-04-24T09:09:16.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.849+0000 7f4653433740 10 client.21780 fetch_fsmap learned FSMap version 58 2024-04-24T09:09:16.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.849+0000 7f4653433740 1 -- 172.21.15.83:0/2816396035 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x5603594dbbd0 con 0x560359465150 2024-04-24T09:09:16.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.850+0000 7f462f400640 1 -- 172.21.15.83:0/2816396035 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4640036070 con 0x560359465150 2024-04-24T09:09:16.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.850+0000 7f462f400640 1 -- 172.21.15.83:0/2816396035 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 58) ==== 8374+0+0 (secure 0 0 0) 0x7f464004c450 con 0x560359465150 2024-04-24T09:09:16.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.851+0000 7f4653433740 10 client.21780 fetch_fsmap finished waiting for FSMap version 58 2024-04-24T09:09:16.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.851+0000 7f4653433740 10 client.21780 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:09:16.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.851+0000 7f4653433740 20 client.21780 populate_metadata read hostname 'smithi083' 2024-04-24T09:09:16.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.851+0000 7f4653433740 1 --2- 172.21.15.83:0/2816396035 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5603594dc240 0x5603594de660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:16.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.851+0000 7f4653433740 4 client.21780 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:09:16.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.851+0000 7f4653433740 1 -- 172.21.15.83:0/2816396035 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5603593bea60 con 0x5603594dc240 2024-04-24T09:09:16.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.852+0000 7f464dc00640 1 --2- 172.21.15.83:0/2816396035 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5603594dc240 0x5603594de660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:16.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.853+0000 7f462f400640 10 client.21780 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:09:16.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.853+0000 7f464dc00640 1 --2- 172.21.15.83:0/2816396035 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5603594dc240 0x5603594de660 crc :-1 s=READY pgs=444 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e42162930731c31 server_cookie=7514c816fa675bfd in_seq=0 out_seq=0 2024-04-24T09:09:16.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.854+0000 7f462f400640 1 -- 172.21.15.83:0/2816396035 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5603593bea60 con 0x5603594dc240 2024-04-24T09:09:16.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.854+0000 7f462f400640 10 client.21780 handle_command_reply: tid=0 2024-04-24T09:09:16.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.854+0000 7f4653433740 1 client.21780 shutdown 2024-04-24T09:09:16.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.854+0000 7f4653433740 1 -- 172.21.15.83:0/2816396035 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560359465150 msgr2=0x56035946b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:16.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.854+0000 7f4653433740 1 --2- 172.21.15.83:0/2816396035 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560359465150 0x56035946b980 secure :-1 s=READY pgs=2365 cs=0 l=1 rev1=1 crypto rx=0x7f464002f3a0 tx=0x7f4640004a80 comp rx=0 tx=0).stop 2024-04-24T09:09:16.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.855+0000 7f4653433740 1 -- 172.21.15.83:0/2816396035 shutdown_connections 2024-04-24T09:09:16.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.855+0000 7f4653433740 1 --2- 172.21.15.83:0/2816396035 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5603594dc240 0x5603594de660 crc :-1 s=READY pgs=444 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:16.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.855+0000 7f4653433740 1 --2- 172.21.15.83:0/2816396035 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560359465150 0x56035946b980 unknown :-1 s=CLOSED pgs=2365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:16.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.855+0000 7f4653433740 1 --2- 172.21.15.83:0/2816396035 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560359452050 0x56035945f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:16.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.855+0000 7f4653433740 1 --2- 172.21.15.83:0/2816396035 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5603594516a0 0x56035945ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:16.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.855+0000 7f4653433740 1 -- 172.21.15.83:0/2816396035 >> 172.21.15.83:0/2816396035 conn(0x56035944dc70 msgr2=0x5603594531e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:16.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.855+0000 7f462f400640 0 client.21780 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:09:16.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.855+0000 7f4653433740 1 -- 172.21.15.83:0/2816396035 shutdown_connections 2024-04-24T09:09:16.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.855+0000 7f4653433740 1 -- 172.21.15.83:0/2816396035 wait complete. 2024-04-24T09:09:16.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.856+0000 7f4653433740 20 client.21780 trim_cache size 0 max 16384 2024-04-24T09:09:16.856 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:09:16.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.856+0000 7f4650600640 1 -- 172.21.15.83:0/171125463 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f46300a3de0 msgr2=0x7f46300a62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:16.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.856+0000 7f4650600640 1 --2- 172.21.15.83:0/171125463 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f46300a3de0 0x7f46300a62a0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f4640004af0 tx=0x7f4640031490 comp rx=0 tx=0).stop 2024-04-24T09:09:16.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.856+0000 7f4650600640 1 -- 172.21.15.83:0/171125463 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46481003b0 msgr2=0x7f46481a06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:16.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.856+0000 7f4650600640 1 --2- 172.21.15.83:0/171125463 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46481003b0 0x7f46481a06a0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f463c00d950 tx=0x7f463c00de10 comp rx=0 tx=0).stop 2024-04-24T09:09:16.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.857+0000 7f4650600640 1 -- 172.21.15.83:0/171125463 shutdown_connections 2024-04-24T09:09:16.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.857+0000 7f4650600640 1 --2- 172.21.15.83:0/171125463 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f46300a3de0 0x7f46300a62a0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:16.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.857+0000 7f4650600640 1 --2- 172.21.15.83:0/171125463 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f464819a810 0x7f464819acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:16.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.857+0000 7f4650600640 1 --2- 172.21.15.83:0/171125463 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f464810a800 0x7f46481a0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:16.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.857+0000 7f4650600640 1 --2- 172.21.15.83:0/171125463 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46481003b0 0x7f46481a06a0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:16.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.857+0000 7f4650600640 1 -- 172.21.15.83:0/171125463 >> 172.21.15.83:0/171125463 conn(0x7f4648077060 msgr2=0x7f46480fe450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:16.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.857+0000 7f4650600640 1 -- 172.21.15.83:0/171125463 shutdown_connections 2024-04-24T09:09:16.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:16.857+0000 7f4650600640 1 -- 172.21.15.83:0/171125463 wait complete. 2024-04-24T09:09:17.169 INFO:tasks.workunit.client.0.smithi083.stdout: 10 29368 7.03 MB/sec execute 108 sec latency 82.358 ms 2024-04-24T09:09:17.878 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T09:09:17.878 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T09:09:17.878 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T09:09:18.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.136+0000 7f9d13000640 1 -- 172.21.15.83:0/572885815 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d0c109580 msgr2=0x7f9d0c111150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:18.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.136+0000 7f9d13000640 1 --2- 172.21.15.83:0/572885815 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d0c109580 0x7f9d0c111150 secure :-1 s=READY pgs=2366 cs=0 l=1 rev1=1 crypto rx=0x7f9d04009be0 tx=0x7f9d0402a6c0 comp rx=0 tx=0).stop 2024-04-24T09:09:18.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.136+0000 7f9d13000640 1 -- 172.21.15.83:0/572885815 shutdown_connections 2024-04-24T09:09:18.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.136+0000 7f9d13000640 1 --2- 172.21.15.83:0/572885815 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d0c109580 0x7f9d0c111150 unknown :-1 s=CLOSED pgs=2366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:18.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.136+0000 7f9d13000640 1 --2- 172.21.15.83:0/572885815 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9d0c104e60 0x7f9d0c108ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:18.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.136+0000 7f9d13000640 1 --2- 172.21.15.83:0/572885815 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d0c1044b0 0x7f9d0c104890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:18.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.136+0000 7f9d13000640 1 -- 172.21.15.83:0/572885815 >> 172.21.15.83:0/572885815 conn(0x7f9d0c0ffd90 msgr2=0x7f9d0c1021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:18.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.136+0000 7f9d13000640 1 -- 172.21.15.83:0/572885815 shutdown_connections 2024-04-24T09:09:18.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.136+0000 7f9d13000640 1 -- 172.21.15.83:0/572885815 wait complete. 2024-04-24T09:09:18.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.138+0000 7f9d13000640 1 Processor -- start 2024-04-24T09:09:18.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.138+0000 7f9d13000640 1 -- start start 2024-04-24T09:09:18.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.138+0000 7f9d13000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d0c1044b0 0x7f9d0c076510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:18.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.138+0000 7f9d13000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d0c109580 0x7f9d0c076a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:18.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.139+0000 7f9d13000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9d0c076f90 0x7f9d0c1ab040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:18.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.139+0000 7f9d13000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9d0c1144f0 con 0x7f9d0c1044b0 2024-04-24T09:09:18.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.139+0000 7f9d13000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9d0c114370 con 0x7f9d0c109580 2024-04-24T09:09:18.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.139+0000 7f9d13000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9d0c114670 con 0x7f9d0c076f90 2024-04-24T09:09:18.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.139+0000 7f9d0be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d0c1044b0 0x7f9d0c076510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:18.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.139+0000 7f9d0be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d0c1044b0 0x7f9d0c076510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60446/0 (socket says 172.21.15.83:60446) 2024-04-24T09:09:18.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.139+0000 7f9d0be00640 1 -- 172.21.15.83:0/2294444290 learned_addr learned my addr 172.21.15.83:0/2294444290 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:09:18.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.139+0000 7f9d0b400640 1 --2- 172.21.15.83:0/2294444290 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d0c109580 0x7f9d0c076a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:18.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.140+0000 7f9d11000640 1 --2- 172.21.15.83:0/2294444290 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9d0c076f90 0x7f9d0c1ab040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:18.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.140+0000 7f9d0be00640 1 -- 172.21.15.83:0/2294444290 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9d0c076f90 msgr2=0x7f9d0c1ab040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:18.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.140+0000 7f9d0be00640 1 --2- 172.21.15.83:0/2294444290 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9d0c076f90 0x7f9d0c1ab040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:18.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.140+0000 7f9d0be00640 1 -- 172.21.15.83:0/2294444290 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d0c109580 msgr2=0x7f9d0c076a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:18.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.140+0000 7f9d0be00640 1 --2- 172.21.15.83:0/2294444290 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d0c109580 0x7f9d0c076a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:18.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.140+0000 7f9d0be00640 1 -- 172.21.15.83:0/2294444290 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d0c1ab580 con 0x7f9d0c1044b0 2024-04-24T09:09:18.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.140+0000 7f9d0be00640 1 --2- 172.21.15.83:0/2294444290 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d0c1044b0 0x7f9d0c076510 secure :-1 s=READY pgs=2367 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc00e9c0 tx=0x7f9cfc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:18.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.141+0000 7f9d08c00640 1 -- 172.21.15.83:0/2294444290 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cfc00ccc0 con 0x7f9d0c1044b0 2024-04-24T09:09:18.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.141+0000 7f9d08c00640 1 -- 172.21.15.83:0/2294444290 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9cfc0054d0 con 0x7f9d0c1044b0 2024-04-24T09:09:18.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.141+0000 7f9d13000640 1 -- 172.21.15.83:0/2294444290 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d0c1ab810 con 0x7f9d0c1044b0 2024-04-24T09:09:18.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.141+0000 7f9d08c00640 1 -- 172.21.15.83:0/2294444290 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cfc010430 con 0x7f9d0c1044b0 2024-04-24T09:09:18.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.141+0000 7f9d13000640 1 -- 172.21.15.83:0/2294444290 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d0c1abd80 con 0x7f9d0c1044b0 2024-04-24T09:09:18.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.145+0000 7f9d08c00640 1 -- 172.21.15.83:0/2294444290 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9cfc0046c0 con 0x7f9d0c1044b0 2024-04-24T09:09:18.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.146+0000 7f9d15e2c740 1 Processor -- start 2024-04-24T09:09:18.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.146+0000 7f9d15e2c740 1 -- start start 2024-04-24T09:09:18.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.147+0000 7f9d15e2c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55985e371090 0x55985e371470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:18.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.147+0000 7f9d15e2c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55985e3719b0 0x55985e37bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:18.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.147+0000 7f9d15e2c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55985e37c080 0x55985e37e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:18.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.147+0000 7f9d15e2c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55985e2d6e70 con 0x55985e371090 2024-04-24T09:09:18.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.147+0000 7f9d15e2c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55985e27d380 con 0x55985e37c080 2024-04-24T09:09:18.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.147+0000 7f9d15e2c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55985e2d2e30 con 0x55985e3719b0 2024-04-24T09:09:18.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.147+0000 7f9d11000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55985e371090 0x55985e371470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:18.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.147+0000 7f9d11000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55985e371090 0x55985e371470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60454/0 (socket says 172.21.15.83:60454) 2024-04-24T09:09:18.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.147+0000 7f9d11000640 1 -- 172.21.15.83:0/148333612 learned_addr learned my addr 172.21.15.83:0/148333612 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:09:18.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.147+0000 7f9d0be00640 1 --2- 172.21.15.83:0/148333612 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55985e3719b0 0x55985e37bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:18.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.148+0000 7f9d08c00640 1 --2- 172.21.15.83:0/2294444290 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f9cf00a3da0 0x7f9cf00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:18.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.148+0000 7f9d0b400640 1 --2- 172.21.15.83:0/148333612 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55985e37c080 0x55985e37e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:18.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.148+0000 7f9d08c00640 1 -- 172.21.15.83:0/2294444290 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 9583+0+0 (secure 0 0 0) 0x7f9cfc0d0f10 con 0x7f9d0c1044b0 2024-04-24T09:09:18.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.148+0000 7f9d11000640 1 -- 172.21.15.83:0/148333612 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55985e3719b0 msgr2=0x55985e37bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:18.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.148+0000 7f9d11000640 1 --2- 172.21.15.83:0/148333612 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55985e3719b0 0x55985e37bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:18.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.148+0000 7f9d11000640 1 -- 172.21.15.83:0/148333612 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55985e37c080 msgr2=0x55985e37e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:18.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.148+0000 7f9d11000640 1 --2- 172.21.15.83:0/148333612 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55985e37c080 0x55985e37e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:18.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.148+0000 7f9d11000640 1 -- 172.21.15.83:0/148333612 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55985e0b17e0 con 0x55985e371090 2024-04-24T09:09:18.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.149+0000 7f9d11000640 1 --2- 172.21.15.83:0/2294444290 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f9cf00a3da0 0x7f9cf00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:18.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.149+0000 7f9d11000640 1 --2- 172.21.15.83:0/148333612 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55985e371090 0x55985e371470 secure :-1 s=READY pgs=2368 cs=0 l=1 rev1=1 crypto rx=0x7f9d0402acb0 tx=0x7f9d040303a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eee06f5ce5e08f0c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:18.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.150+0000 7f9d01600640 1 -- 172.21.15.83:0/148333612 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d04040040 con 0x55985e371090 2024-04-24T09:09:18.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.150+0000 7f9d01600640 1 -- 172.21.15.83:0/148333612 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9d04037040 con 0x55985e371090 2024-04-24T09:09:18.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.150+0000 7f9d01600640 1 -- 172.21.15.83:0/148333612 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d04036070 con 0x55985e371090 2024-04-24T09:09:18.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.150+0000 7f9d11000640 1 --2- 172.21.15.83:0/2294444290 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f9cf00a3da0 0x7f9cf00a6260 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f9d04048000 tx=0x7f9d0404b000 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:18.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.151+0000 7f9d15e2c740 1 -- 172.21.15.83:0/148333612 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55985e371090 msgr2=0x55985e371470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:18.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.151+0000 7f9d15e2c740 1 --2- 172.21.15.83:0/148333612 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55985e371090 0x55985e371470 secure :-1 s=READY pgs=2368 cs=0 l=1 rev1=1 crypto rx=0x7f9d0402acb0 tx=0x7f9d040303a0 comp rx=0 tx=0).stop 2024-04-24T09:09:18.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.152+0000 7f9d15e2c740 1 -- 172.21.15.83:0/148333612 shutdown_connections 2024-04-24T09:09:18.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.152+0000 7f9d15e2c740 1 --2- 172.21.15.83:0/148333612 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55985e37c080 0x55985e37e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:18.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.152+0000 7f9d15e2c740 1 --2- 172.21.15.83:0/148333612 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55985e3719b0 0x55985e37bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:18.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.152+0000 7f9d15e2c740 1 --2- 172.21.15.83:0/148333612 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55985e371090 0x55985e371470 unknown :-1 s=CLOSED pgs=2368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:18.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.152+0000 7f9d15e2c740 1 -- 172.21.15.83:0/148333612 >> 172.21.15.83:0/148333612 conn(0x55985e36d3b0 msgr2=0x55985e36d790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:18.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.152+0000 7f9d15e2c740 1 -- 172.21.15.83:0/148333612 shutdown_connections 2024-04-24T09:09:18.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.153+0000 7f9d15e2c740 1 -- 172.21.15.83:0/148333612 wait complete. 2024-04-24T09:09:18.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.153+0000 7f9d15e2c740 1 -- start start 2024-04-24T09:09:18.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.154+0000 7f9d15e2c740 1 Processor -- start 2024-04-24T09:09:18.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.154+0000 7f9d15e2c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55985e371090 0x55985e378fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:18.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.154+0000 7f9d15e2c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55985e3719b0 0x55985e3794e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:18.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.154+0000 7f9d15e2c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55985e37c080 0x55985e379a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:18.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.154+0000 7f9d15e2c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55985e37ee70 con 0x55985e371090 2024-04-24T09:09:18.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.154+0000 7f9d15e2c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55985e290660 con 0x55985e3719b0 2024-04-24T09:09:18.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.154+0000 7f9d15e2c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55985e37eff0 con 0x55985e37c080 2024-04-24T09:09:18.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.154+0000 7f9d15e2c740 10 client.0 _ll_register_callbacks cb 0x55985e27eaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:09:18.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.155+0000 7f9d0be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55985e371090 0x55985e378fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:18.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.155+0000 7f9d0be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55985e371090 0x55985e378fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60468/0 (socket says 172.21.15.83:60468) 2024-04-24T09:09:18.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.155+0000 7f9d0be00640 1 -- 172.21.15.83:0/330671926 learned_addr learned my addr 172.21.15.83:0/330671926 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:09:18.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.155+0000 7f9d11000640 1 --2- 172.21.15.83:0/330671926 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55985e37c080 0x55985e379a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:18.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.155+0000 7f9d0be00640 1 -- 172.21.15.83:0/330671926 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55985e37c080 msgr2=0x55985e379a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:18.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.155+0000 7f9d0be00640 1 --2- 172.21.15.83:0/330671926 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55985e37c080 0x55985e379a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:18.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.155+0000 7f9d0be00640 1 -- 172.21.15.83:0/330671926 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55985e3719b0 msgr2=0x55985e3794e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:18.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.155+0000 7f9d0be00640 1 --2- 172.21.15.83:0/330671926 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55985e3719b0 0x55985e3794e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:18.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.155+0000 7f9d0be00640 1 -- 172.21.15.83:0/330671926 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55985e111f70 con 0x55985e371090 2024-04-24T09:09:18.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.155+0000 7f9d0be00640 1 --2- 172.21.15.83:0/330671926 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55985e371090 0x55985e378fa0 secure :-1 s=READY pgs=2369 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc09c9c0 tx=0x7f9cfc0094c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a441ef02385fd241 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:18.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.155+0000 7f9cf6000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:09:18.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.156+0000 7f9cf6000640 1 -- 172.21.15.83:0/330671926 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cfc0ce670 con 0x55985e371090 2024-04-24T09:09:18.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.156+0000 7f9cf6000640 1 -- 172.21.15.83:0/330671926 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9583+0+0 (secure 0 0 0) 0x7f9cfc0a5410 con 0x55985e371090 2024-04-24T09:09:18.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.156+0000 7f9d15e2c740 1 -- 172.21.15.83:0/330671926 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d04037040 con 0x55985e371090 2024-04-24T09:09:18.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.156+0000 7f9cf6000640 1 -- 172.21.15.83:0/330671926 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f9cfc09d130 con 0x55985e371090 2024-04-24T09:09:18.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.156+0000 7f9d15e2c740 10 client.21792 fetch_fsmap learned FSMap version 58 2024-04-24T09:09:18.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.156+0000 7f9d15e2c740 1 -- 172.21.15.83:0/330671926 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55985dfa9170 con 0x55985e371090 2024-04-24T09:09:18.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.157+0000 7f9cf6000640 1 -- 172.21.15.83:0/330671926 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9d04037040 con 0x55985e371090 2024-04-24T09:09:18.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.157+0000 7f9cf6000640 1 -- 172.21.15.83:0/330671926 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 58) ==== 8374+0+0 (secure 0 0 0) 0x7f9cfc09c010 con 0x55985e371090 2024-04-24T09:09:18.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.157+0000 7f9d15e2c740 10 client.21792 fetch_fsmap finished waiting for FSMap version 58 2024-04-24T09:09:18.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.157+0000 7f9d15e2c740 10 client.21792 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:09:18.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.157+0000 7f9d15e2c740 20 client.21792 populate_metadata read hostname 'smithi083' 2024-04-24T09:09:18.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.157+0000 7f9d15e2c740 1 --2- 172.21.15.83:0/330671926 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55985e38c2d0 0x55985e38e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:18.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.157+0000 7f9d15e2c740 4 client.21792 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:09:18.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.157+0000 7f9d15e2c740 1 -- 172.21.15.83:0/330671926 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55985e2e4890 con 0x55985e38c2d0 2024-04-24T09:09:18.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.157+0000 7f9d0b400640 1 --2- 172.21.15.83:0/330671926 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55985e38c2d0 0x55985e38e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:18.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.158+0000 7f9d0b400640 1 --2- 172.21.15.83:0/330671926 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55985e38c2d0 0x55985e38e760 crc :-1 s=READY pgs=445 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5b24e2da38f9f66 server_cookie=a7c0d77cac05b8c0 in_seq=0 out_seq=0 2024-04-24T09:09:18.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.158+0000 7f9cf6000640 10 client.21792 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:09:18.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.159+0000 7f9cf6000640 1 -- 172.21.15.83:0/330671926 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55985e2e4890 con 0x55985e38c2d0 2024-04-24T09:09:18.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.159+0000 7f9cf6000640 10 client.21792 handle_command_reply: tid=0 2024-04-24T09:09:18.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.160+0000 7f9d15e2c740 1 client.21792 shutdown 2024-04-24T09:09:18.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.160+0000 7f9d15e2c740 1 -- 172.21.15.83:0/330671926 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55985e371090 msgr2=0x55985e378fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:18.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.160+0000 7f9d15e2c740 1 --2- 172.21.15.83:0/330671926 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55985e371090 0x55985e378fa0 secure :-1 s=READY pgs=2369 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc09c9c0 tx=0x7f9cfc0094c0 comp rx=0 tx=0).stop 2024-04-24T09:09:18.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.160+0000 7f9d15e2c740 1 -- 172.21.15.83:0/330671926 shutdown_connections 2024-04-24T09:09:18.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.160+0000 7f9d15e2c740 1 --2- 172.21.15.83:0/330671926 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55985e38c2d0 0x55985e38e760 crc :-1 s=READY pgs=445 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:18.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.160+0000 7f9d15e2c740 1 --2- 172.21.15.83:0/330671926 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55985e37c080 0x55985e379a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:18.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.160+0000 7f9d15e2c740 1 --2- 172.21.15.83:0/330671926 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55985e3719b0 0x55985e3794e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:18.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.160+0000 7f9d15e2c740 1 --2- 172.21.15.83:0/330671926 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55985e371090 0x55985e378fa0 unknown :-1 s=CLOSED pgs=2369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:18.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.160+0000 7f9d15e2c740 1 -- 172.21.15.83:0/330671926 >> 172.21.15.83:0/330671926 conn(0x55985e36d3b0 msgr2=0x55985e3731b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:18.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.160+0000 7f9cf6000640 0 client.21792 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:09:18.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.161+0000 7f9d15e2c740 1 -- 172.21.15.83:0/330671926 shutdown_connections 2024-04-24T09:09:18.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.161+0000 7f9d15e2c740 1 -- 172.21.15.83:0/330671926 wait complete. 2024-04-24T09:09:18.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.161+0000 7f9d15e2c740 20 client.21792 trim_cache size 0 max 16384 2024-04-24T09:09:18.169 INFO:tasks.workunit.client.0.smithi083.stdout: 10 29481 7.04 MB/sec execute 109 sec latency 82.396 ms 2024-04-24T09:09:18.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.192+0000 7f9d15e2c740 1 Processor -- start 2024-04-24T09:09:18.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.192+0000 7f9d15e2c740 1 -- start start 2024-04-24T09:09:18.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.192+0000 7f9d15e2c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55985e371090 0x55985e387d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:18.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.192+0000 7f9d15e2c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55985e3719b0 0x55985e3788a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:18.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.192+0000 7f9d15e2c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55985e37c080 0x55985e372fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:18.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.192+0000 7f9d15e2c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9d0c1b5d30 con 0x55985e3719b0 2024-04-24T09:09:18.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.192+0000 7f9d15e2c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9d0c1b7870 con 0x55985e37c080 2024-04-24T09:09:18.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.192+0000 7f9d15e2c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9d0c1b6400 con 0x55985e371090 2024-04-24T09:09:18.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.193+0000 7f9d0b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55985e3719b0 0x55985e3788a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:18.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.193+0000 7f9d11000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55985e37c080 0x55985e372fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:18.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.193+0000 7f9d11000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55985e37c080 0x55985e372fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:53308/0 (socket says 172.21.15.83:53308) 2024-04-24T09:09:18.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.193+0000 7f9d11000640 1 -- 172.21.15.83:0/1449275603 learned_addr learned my addr 172.21.15.83:0/1449275603 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:09:18.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.193+0000 7f9d0be00640 1 --2- 172.21.15.83:0/1449275603 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55985e371090 0x55985e387d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:18.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.193+0000 7f9d0b400640 1 -- 172.21.15.83:0/1449275603 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55985e371090 msgr2=0x55985e387d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:18.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.193+0000 7f9d0b400640 1 --2- 172.21.15.83:0/1449275603 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55985e371090 0x55985e387d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:18.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.193+0000 7f9d0b400640 1 -- 172.21.15.83:0/1449275603 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55985e37c080 msgr2=0x55985e372fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:18.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.193+0000 7f9d0b400640 1 --2- 172.21.15.83:0/1449275603 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55985e37c080 0x55985e372fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:18.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.193+0000 7f9d0b400640 1 -- 172.21.15.83:0/1449275603 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55985e3706a0 con 0x55985e3719b0 2024-04-24T09:09:18.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.193+0000 7f9d11000640 1 --2- 172.21.15.83:0/1449275603 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55985e37c080 0x55985e372fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:09:18.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.194+0000 7f9d0b400640 1 --2- 172.21.15.83:0/1449275603 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55985e3719b0 0x55985e3788a0 secure :-1 s=READY pgs=2370 cs=0 l=1 rev1=1 crypto rx=0x55985e36e540 tx=0x7f9cf800b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6561d3e6ddb7e45d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:18.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.194+0000 7f9d00c00640 1 -- 172.21.15.83:0/1449275603 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cf801a5f0 con 0x55985e3719b0 2024-04-24T09:09:18.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.194+0000 7f9d00c00640 1 -- 172.21.15.83:0/1449275603 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9cf80054d0 con 0x55985e3719b0 2024-04-24T09:09:18.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.194+0000 7f9d00c00640 1 -- 172.21.15.83:0/1449275603 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cf8004d80 con 0x55985e3719b0 2024-04-24T09:09:18.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.195+0000 7f9d15e2c740 1 -- 172.21.15.83:0/1449275603 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55985e3719b0 msgr2=0x55985e3788a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:18.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.195+0000 7f9d15e2c740 1 --2- 172.21.15.83:0/1449275603 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55985e3719b0 0x55985e3788a0 secure :-1 s=READY pgs=2370 cs=0 l=1 rev1=1 crypto rx=0x55985e36e540 tx=0x7f9cf800b5b0 comp rx=0 tx=0).stop 2024-04-24T09:09:18.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.195+0000 7f9d15e2c740 1 -- 172.21.15.83:0/1449275603 shutdown_connections 2024-04-24T09:09:18.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.195+0000 7f9d15e2c740 1 --2- 172.21.15.83:0/1449275603 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55985e37c080 0x55985e372fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:18.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.195+0000 7f9d15e2c740 1 --2- 172.21.15.83:0/1449275603 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55985e3719b0 0x55985e3788a0 unknown :-1 s=CLOSED pgs=2370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:18.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.195+0000 7f9d15e2c740 1 --2- 172.21.15.83:0/1449275603 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55985e371090 0x55985e387d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:18.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.195+0000 7f9d15e2c740 1 -- 172.21.15.83:0/1449275603 >> 172.21.15.83:0/1449275603 conn(0x55985e36d3b0 msgr2=0x55985e3728d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:18.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.196+0000 7f9d15e2c740 1 -- 172.21.15.83:0/1449275603 shutdown_connections 2024-04-24T09:09:18.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.196+0000 7f9d15e2c740 1 -- 172.21.15.83:0/1449275603 wait complete. 2024-04-24T09:09:18.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.196+0000 7f9d15e2c740 1 -- start start 2024-04-24T09:09:18.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.196+0000 7f9d15e2c740 1 Processor -- start 2024-04-24T09:09:18.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.197+0000 7f9d15e2c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55985e371090 0x55985e376040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:18.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.198+0000 7f9d15e2c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55985e3719b0 0x55985e376580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:18.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.198+0000 7f9d15e2c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55985e37c080 0x55985e3f18f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:18.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.198+0000 7f9d0be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55985e371090 0x55985e376040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:18.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.198+0000 7f9d0be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55985e371090 0x55985e376040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60480/0 (socket says 172.21.15.83:60480) 2024-04-24T09:09:18.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.198+0000 7f9d0be00640 1 -- 172.21.15.83:0/2281835131 learned_addr learned my addr 172.21.15.83:0/2281835131 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:09:18.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.198+0000 7f9d15e2c740 1 -- 172.21.15.83:0/2281835131 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9d0c1b71a0 con 0x55985e371090 2024-04-24T09:09:18.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.198+0000 7f9d15e2c740 1 -- 172.21.15.83:0/2281835131 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9d0c1b6ad0 con 0x55985e37c080 2024-04-24T09:09:18.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.198+0000 7f9d15e2c740 1 -- 172.21.15.83:0/2281835131 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9d0c1b3d30 con 0x55985e3719b0 2024-04-24T09:09:18.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.198+0000 7f9d15e2c740 10 client.0 _ll_register_callbacks cb 0x7f9cfc0d46b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:09:18.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.198+0000 7f9d0b400640 1 --2- 172.21.15.83:0/2281835131 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55985e3719b0 0x55985e376580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:18.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.198+0000 7f9d11000640 1 --2- 172.21.15.83:0/2281835131 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55985e37c080 0x55985e3f18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:18.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.199+0000 7f9d0be00640 1 -- 172.21.15.83:0/2281835131 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55985e3719b0 msgr2=0x55985e376580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:18.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.199+0000 7f9d0be00640 1 --2- 172.21.15.83:0/2281835131 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55985e3719b0 0x55985e376580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:18.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.199+0000 7f9d0be00640 1 -- 172.21.15.83:0/2281835131 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55985e37c080 msgr2=0x55985e3f18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:18.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.199+0000 7f9d0be00640 1 --2- 172.21.15.83:0/2281835131 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55985e37c080 0x55985e3f18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:18.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.199+0000 7f9d0be00640 1 -- 172.21.15.83:0/2281835131 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55985e3f1e30 con 0x55985e371090 2024-04-24T09:09:18.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.199+0000 7f9d0be00640 1 --2- 172.21.15.83:0/2281835131 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55985e371090 0x55985e376040 secure :-1 s=READY pgs=2371 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc098e90 tx=0x7f9cfc010d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb29554a209b8a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:18.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.199+0000 7f9cf6000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:09:18.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.200+0000 7f9cf6000640 1 -- 172.21.15.83:0/2281835131 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cfc0ce670 con 0x55985e371090 2024-04-24T09:09:18.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.200+0000 7f9cf6000640 1 -- 172.21.15.83:0/2281835131 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9583+0+0 (secure 0 0 0) 0x7f9cfc09be80 con 0x55985e371090 2024-04-24T09:09:18.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.200+0000 7f9d15e2c740 1 -- 172.21.15.83:0/2281835131 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9cf80054d0 con 0x55985e371090 2024-04-24T09:09:18.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.200+0000 7f9cf6000640 1 -- 172.21.15.83:0/2281835131 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f9cfc0bd0b0 con 0x55985e371090 2024-04-24T09:09:18.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.200+0000 7f9d15e2c740 10 client.21798 fetch_fsmap learned FSMap version 58 2024-04-24T09:09:18.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.200+0000 7f9d15e2c740 1 -- 172.21.15.83:0/2281835131 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55985e3f2fb0 con 0x55985e371090 2024-04-24T09:09:18.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.201+0000 7f9cf6000640 1 -- 172.21.15.83:0/2281835131 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9cf80054d0 con 0x55985e371090 2024-04-24T09:09:18.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.201+0000 7f9cf6000640 1 -- 172.21.15.83:0/2281835131 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 58) ==== 8374+0+0 (secure 0 0 0) 0x7f9cfc0d6ce0 con 0x55985e371090 2024-04-24T09:09:18.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.201+0000 7f9d15e2c740 10 client.21798 fetch_fsmap finished waiting for FSMap version 58 2024-04-24T09:09:18.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.201+0000 7f9d15e2c740 10 client.21798 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:09:18.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.201+0000 7f9d15e2c740 20 client.21798 populate_metadata read hostname 'smithi083' 2024-04-24T09:09:18.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.201+0000 7f9d15e2c740 1 --2- 172.21.15.83:0/2281835131 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55985e3f36b0 0x55985e3f5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:18.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.201+0000 7f9d15e2c740 4 client.21798 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T09:09:18.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.201+0000 7f9d15e2c740 1 -- 172.21.15.83:0/2281835131 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55985e2de700 con 0x55985e3f36b0 2024-04-24T09:09:18.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.202+0000 7f9d0b400640 1 --2- 172.21.15.83:0/2281835131 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55985e3f36b0 0x55985e3f5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:18.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.203+0000 7f9d0b400640 1 --2- 172.21.15.83:0/2281835131 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55985e3f36b0 0x55985e3f5ad0 crc :-1 s=READY pgs=446 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9277062fbe94bba8 server_cookie=bedddbc3b399a3a1 in_seq=0 out_seq=0 2024-04-24T09:09:18.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.203+0000 7f9cf6000640 10 client.21798 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:09:18.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.205+0000 7f9cf6000640 1 -- 172.21.15.83:0/2281835131 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55985e2de700 con 0x55985e3f36b0 2024-04-24T09:09:18.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.205+0000 7f9cf6000640 10 client.21798 handle_command_reply: tid=0 2024-04-24T09:09:18.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.205+0000 7f9d15e2c740 1 client.21798 shutdown 2024-04-24T09:09:18.205 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:09:17 smithi083 ceph-mon[27907]: pgmap v557: 129 pgs: 129 active+clean; 546 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.2 MiB/s wr, 225 op/s 2024-04-24T09:09:18.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.206+0000 7f9d15e2c740 1 -- 172.21.15.83:0/2281835131 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55985e371090 msgr2=0x55985e376040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:18.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.206+0000 7f9d15e2c740 1 --2- 172.21.15.83:0/2281835131 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55985e371090 0x55985e376040 secure :-1 s=READY pgs=2371 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc098e90 tx=0x7f9cfc010d90 comp rx=0 tx=0).stop 2024-04-24T09:09:18.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.206+0000 7f9d15e2c740 1 -- 172.21.15.83:0/2281835131 shutdown_connections 2024-04-24T09:09:18.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.206+0000 7f9d15e2c740 1 --2- 172.21.15.83:0/2281835131 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55985e3f36b0 0x55985e3f5ad0 crc :-1 s=READY pgs=446 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:18.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.206+0000 7f9d15e2c740 1 --2- 172.21.15.83:0/2281835131 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55985e37c080 0x55985e3f18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:18.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.206+0000 7f9d15e2c740 1 --2- 172.21.15.83:0/2281835131 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55985e3719b0 0x55985e376580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:18.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.206+0000 7f9d15e2c740 1 --2- 172.21.15.83:0/2281835131 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55985e371090 0x55985e376040 unknown :-1 s=CLOSED pgs=2371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:18.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.206+0000 7f9d15e2c740 1 -- 172.21.15.83:0/2281835131 >> 172.21.15.83:0/2281835131 conn(0x55985e36d3b0 msgr2=0x55985e3797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:18.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.206+0000 7f9cf6000640 0 client.21798 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:09:18.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.206+0000 7f9d15e2c740 1 -- 172.21.15.83:0/2281835131 shutdown_connections 2024-04-24T09:09:18.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.206+0000 7f9d15e2c740 1 -- 172.21.15.83:0/2281835131 wait complete. 2024-04-24T09:09:18.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.207+0000 7f9d15e2c740 20 client.21798 trim_cache size 0 max 16384 2024-04-24T09:09:18.209 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T09:09:18.209 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-04-24T09:09:18.210 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "62024573-0364-4ade-abd8-e4f8620a7d40", 2024-04-24T09:09:18.210 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-04-24T09:09:18.210 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T09:09:18.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.208+0000 7f9d13000640 1 -- 172.21.15.83:0/2294444290 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f9cf00a3da0 msgr2=0x7f9cf00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:18.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.208+0000 7f9d13000640 1 --2- 172.21.15.83:0/2294444290 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f9cf00a3da0 0x7f9cf00a6260 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f9d04048000 tx=0x7f9d0404b000 comp rx=0 tx=0).stop 2024-04-24T09:09:18.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.208+0000 7f9d13000640 1 -- 172.21.15.83:0/2294444290 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d0c1044b0 msgr2=0x7f9d0c076510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:18.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.208+0000 7f9d13000640 1 --2- 172.21.15.83:0/2294444290 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d0c1044b0 0x7f9d0c076510 secure :-1 s=READY pgs=2367 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc00e9c0 tx=0x7f9cfc00ee80 comp rx=0 tx=0).stop 2024-04-24T09:09:18.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.208+0000 7f9d13000640 1 -- 172.21.15.83:0/2294444290 shutdown_connections 2024-04-24T09:09:18.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.208+0000 7f9d13000640 1 --2- 172.21.15.83:0/2294444290 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f9cf00a3da0 0x7f9cf00a6260 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:18.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.208+0000 7f9d13000640 1 --2- 172.21.15.83:0/2294444290 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9d0c076f90 0x7f9d0c1ab040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:18.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.208+0000 7f9d13000640 1 --2- 172.21.15.83:0/2294444290 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d0c109580 0x7f9d0c076a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:18.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.208+0000 7f9d13000640 1 --2- 172.21.15.83:0/2294444290 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d0c1044b0 0x7f9d0c076510 unknown :-1 s=CLOSED pgs=2367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:18.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.208+0000 7f9d13000640 1 -- 172.21.15.83:0/2294444290 >> 172.21.15.83:0/2294444290 conn(0x7f9d0c0ffd90 msgr2=0x7f9d0c101160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:18.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.209+0000 7f9d13000640 1 -- 172.21.15.83:0/2294444290 shutdown_connections 2024-04-24T09:09:18.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.209+0000 7f9d13000640 1 -- 172.21.15.83:0/2294444290 wait complete. 2024-04-24T09:09:18.213 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:09:17 smithi189 ceph-mon[31791]: pgmap v557: 129 pgs: 129 active+clean; 546 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.2 MiB/s wr, 225 op/s 2024-04-24T09:09:18.213 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:09:17 smithi138 ceph-mon[31441]: pgmap v557: 129 pgs: 129 active+clean; 546 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.2 MiB/s wr, 225 op/s 2024-04-24T09:09:18.233 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T09:09:18.234 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T09:09:18.430 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.431+0000 7f1e4ce00640 1 -- 172.21.15.83:0/2440220970 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e48109540 msgr2=0x7f1e48111110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:18.431 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.431+0000 7f1e4ce00640 1 --2- 172.21.15.83:0/2440220970 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e48109540 0x7f1e48111110 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f1e40009ca0 tx=0x7f1e4002a8c0 comp rx=0 tx=0).stop 2024-04-24T09:09:18.431 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.431+0000 7f1e4ce00640 1 -- 172.21.15.83:0/2440220970 shutdown_connections 2024-04-24T09:09:18.431 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.431+0000 7f1e4ce00640 1 --2- 172.21.15.83:0/2440220970 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e48109540 0x7f1e48111110 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:18.431 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.431+0000 7f1e4ce00640 1 --2- 172.21.15.83:0/2440220970 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1e48104e20 0x7f1e48108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:18.431 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.431+0000 7f1e4ce00640 1 --2- 172.21.15.83:0/2440220970 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e48104470 0x7f1e48104850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:18.431 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.431+0000 7f1e4ce00640 1 -- 172.21.15.83:0/2440220970 >> 172.21.15.83:0/2440220970 conn(0x7f1e480ffcd0 msgr2=0x7f1e481020f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:18.432 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.432+0000 7f1e4ce00640 1 -- 172.21.15.83:0/2440220970 shutdown_connections 2024-04-24T09:09:18.432 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.432+0000 7f1e4ce00640 1 -- 172.21.15.83:0/2440220970 wait complete. 2024-04-24T09:09:18.432 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.433+0000 7f1e4ce00640 1 Processor -- start 2024-04-24T09:09:18.433 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.433+0000 7f1e4ce00640 1 -- start start 2024-04-24T09:09:18.433 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.433+0000 7f1e4ce00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e48104470 0x7f1e4810fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:18.433 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.433+0000 7f1e4ce00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1e48109540 0x7f1e48110080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:18.433 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.434+0000 7f1e4ce00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e481a8980 0x7f1e481aad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:18.433 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.434+0000 7f1e4ce00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1e480774f0 con 0x7f1e48104470 2024-04-24T09:09:18.433 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.434+0000 7f1e4ce00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1e48077370 con 0x7f1e481a8980 2024-04-24T09:09:18.433 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.434+0000 7f1e4ce00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1e48077670 con 0x7f1e48109540 2024-04-24T09:09:18.434 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.434+0000 7f1e46800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e481a8980 0x7f1e481aad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:18.434 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.434+0000 7f1e45400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1e48109540 0x7f1e48110080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:18.435 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.434+0000 7f1e46800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e481a8980 0x7f1e481aad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:53340/0 (socket says 172.21.15.83:53340) 2024-04-24T09:09:18.435 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.434+0000 7f1e46800640 1 -- 172.21.15.83:0/2714080367 learned_addr learned my addr 172.21.15.83:0/2714080367 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:09:18.435 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.435+0000 7f1e46800640 1 -- 172.21.15.83:0/2714080367 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1e48109540 msgr2=0x7f1e48110080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:18.435 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.435+0000 7f1e46800640 1 --2- 172.21.15.83:0/2714080367 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1e48109540 0x7f1e48110080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:18.435 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.435+0000 7f1e46800640 1 -- 172.21.15.83:0/2714080367 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e48104470 msgr2=0x7f1e4810fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:18.435 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.435+0000 7f1e45e00640 1 --2- 172.21.15.83:0/2714080367 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e48104470 0x7f1e4810fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:18.435 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.435+0000 7f1e46800640 1 --2- 172.21.15.83:0/2714080367 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e48104470 0x7f1e4810fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:18.435 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.435+0000 7f1e46800640 1 -- 172.21.15.83:0/2714080367 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e4806bb50 con 0x7f1e481a8980 2024-04-24T09:09:18.435 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.435+0000 7f1e45e00640 1 --2- 172.21.15.83:0/2714080367 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e48104470 0x7f1e4810fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:09:18.436 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.436+0000 7f1e46800640 1 --2- 172.21.15.83:0/2714080367 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e481a8980 0x7f1e481aad70 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f1e40009dc0 tx=0x7f1e40032ff0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:18.436 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.436+0000 7f1e3ea00640 1 -- 172.21.15.83:0/2714080367 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e40003c30 con 0x7f1e481a8980 2024-04-24T09:09:18.436 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.436+0000 7f1e3ea00640 1 -- 172.21.15.83:0/2714080367 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1e40003dd0 con 0x7f1e481a8980 2024-04-24T09:09:18.436 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.436+0000 7f1e3ea00640 1 -- 172.21.15.83:0/2714080367 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e40030ac0 con 0x7f1e481a8980 2024-04-24T09:09:18.436 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.436+0000 7f1e4ce00640 1 -- 172.21.15.83:0/2714080367 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e4806bde0 con 0x7f1e481a8980 2024-04-24T09:09:18.436 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.436+0000 7f1e4ce00640 1 -- 172.21.15.83:0/2714080367 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e4806c2e0 con 0x7f1e481a8980 2024-04-24T09:09:18.440 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.441+0000 7f1e4fb87740 1 Processor -- start 2024-04-24T09:09:18.441 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.443+0000 7f1e3ea00640 1 -- 172.21.15.83:0/2714080367 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1e40004680 con 0x7f1e481a8980 2024-04-24T09:09:18.442 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.443+0000 7f1e4fb87740 1 -- start start 2024-04-24T09:09:18.443 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.444+0000 7f1e4fb87740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a89757e1b0 0x55a89757e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:18.443 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.444+0000 7f1e4fb87740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a89757ead0 0x55a897588e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:18.443 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.444+0000 7f1e4fb87740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a897589390 0x55a89758b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:18.443 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.444+0000 7f1e4fb87740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a8974e4900 con 0x55a89757e1b0 2024-04-24T09:09:18.444 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.444+0000 7f1e4fb87740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a89748af20 con 0x55a89757ead0 2024-04-24T09:09:18.444 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.444+0000 7f1e4fb87740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a8974e08c0 con 0x55a897589390 2024-04-24T09:09:18.444 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.444+0000 7f1e45e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a89757ead0 0x55a897588e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:18.444 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.444+0000 7f1e45e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a89757ead0 0x55a897588e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:53346/0 (socket says 172.21.15.83:53346) 2024-04-24T09:09:18.444 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.444+0000 7f1e45e00640 1 -- 172.21.15.83:0/357297147 learned_addr learned my addr 172.21.15.83:0/357297147 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:09:18.445 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.445+0000 7f1e45e00640 1 -- 172.21.15.83:0/357297147 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a897589390 msgr2=0x55a89758b780 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-24T09:09:18.446 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.445+0000 7f1e45e00640 1 --2- 172.21.15.83:0/357297147 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a897589390 0x55a89758b780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:18.446 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.445+0000 7f1e45e00640 1 -- 172.21.15.83:0/357297147 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a89757e1b0 msgr2=0x55a89757e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:18.446 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.445+0000 7f1e45e00640 1 --2- 172.21.15.83:0/357297147 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a89757e1b0 0x55a89757e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:18.446 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.445+0000 7f1e45e00640 1 -- 172.21.15.83:0/357297147 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8974afe10 con 0x55a89757ead0 2024-04-24T09:09:18.446 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.445+0000 7f1e3ea00640 1 --2- 172.21.15.83:0/2714080367 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f1e2c0a3e70 0x7f1e2c0a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:18.446 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.445+0000 7f1e45e00640 1 --2- 172.21.15.83:0/357297147 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a89757ead0 0x55a897588e50 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f1e38009c60 tx=0x7f1e38010740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39695cab82e14a43 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:18.446 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.445+0000 7f1e3ea00640 1 -- 172.21.15.83:0/2714080367 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 9583+0+0 (secure 0 0 0) 0x7f1e400f21e0 con 0x7f1e481a8980 2024-04-24T09:09:18.446 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.446+0000 7f1e33400640 1 -- 172.21.15.83:0/357297147 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e38005020 con 0x55a89757ead0 2024-04-24T09:09:18.447 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.446+0000 7f1e33400640 1 -- 172.21.15.83:0/357297147 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1e380047a0 con 0x55a89757ead0 2024-04-24T09:09:18.447 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.446+0000 7f1e33400640 1 -- 172.21.15.83:0/357297147 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e38012740 con 0x55a89757ead0 2024-04-24T09:09:18.447 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.446+0000 7f1e45400640 1 --2- 172.21.15.83:0/2714080367 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f1e2c0a3e70 0x7f1e2c0a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:18.447 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.447+0000 7f1e4fb87740 1 -- 172.21.15.83:0/357297147 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a89757ead0 msgr2=0x55a897588e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:18.447 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.447+0000 7f1e4fb87740 1 --2- 172.21.15.83:0/357297147 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a89757ead0 0x55a897588e50 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f1e38009c60 tx=0x7f1e38010740 comp rx=0 tx=0).stop 2024-04-24T09:09:18.448 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.447+0000 7f1e4fb87740 1 -- 172.21.15.83:0/357297147 shutdown_connections 2024-04-24T09:09:18.448 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.447+0000 7f1e4fb87740 1 --2- 172.21.15.83:0/357297147 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a897589390 0x55a89758b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:18.448 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.447+0000 7f1e4fb87740 1 --2- 172.21.15.83:0/357297147 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a89757ead0 0x55a897588e50 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:18.449 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.447+0000 7f1e4fb87740 1 --2- 172.21.15.83:0/357297147 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a89757e1b0 0x55a89757e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:18.449 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.447+0000 7f1e4fb87740 1 -- 172.21.15.83:0/357297147 >> 172.21.15.83:0/357297147 conn(0x55a89757a6c0 msgr2=0x55a89757aaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:18.449 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.448+0000 7f1e45400640 1 --2- 172.21.15.83:0/2714080367 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f1e2c0a3e70 0x7f1e2c0a6330 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x55a89757bae0 tx=0x7f1e340097e0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:18.449 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.448+0000 7f1e4fb87740 1 -- 172.21.15.83:0/357297147 shutdown_connections 2024-04-24T09:09:18.449 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.448+0000 7f1e4fb87740 1 -- 172.21.15.83:0/357297147 wait complete. 2024-04-24T09:09:18.449 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.448+0000 7f1e4fb87740 1 -- start start 2024-04-24T09:09:18.449 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.449+0000 7f1e4fb87740 1 Processor -- start 2024-04-24T09:09:18.451 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.450+0000 7f1e4fb87740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a89757e1b0 0x55a8975864b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:18.451 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.450+0000 7f1e4fb87740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a89757ead0 0x55a8975869f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:18.451 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.450+0000 7f1e4fb87740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a897589390 0x55a897586f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:18.451 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.450+0000 7f1e4fb87740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a89758bfc0 con 0x55a89757ead0 2024-04-24T09:09:18.451 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.451+0000 7f1e45400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a89757ead0 0x55a8975869f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:18.451 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.451+0000 7f1e45e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a89757e1b0 0x55a8975864b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:18.451 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.451+0000 7f1e46800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a897589390 0x55a897586f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:18.452 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.451+0000 7f1e46800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a897589390 0x55a897586f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:53356/0 (socket says 172.21.15.83:53356) 2024-04-24T09:09:18.452 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.451+0000 7f1e46800640 1 -- 172.21.15.83:0/3316190032 learned_addr learned my addr 172.21.15.83:0/3316190032 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:09:18.452 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.451+0000 7f1e4fb87740 1 -- 172.21.15.83:0/3316190032 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a89749e2c0 con 0x55a897589390 2024-04-24T09:09:18.452 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.451+0000 7f1e4fb87740 1 -- 172.21.15.83:0/3316190032 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a89758c140 con 0x55a89757e1b0 2024-04-24T09:09:18.452 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.451+0000 7f1e4fb87740 10 client.0 _ll_register_callbacks cb 0x55a8971f6a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:09:18.453 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.452+0000 7f1e45400640 1 -- 172.21.15.83:0/3316190032 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a89757e1b0 msgr2=0x55a8975864b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:18.453 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.452+0000 7f1e45400640 1 --2- 172.21.15.83:0/3316190032 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a89757e1b0 0x55a8975864b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:18.453 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.452+0000 7f1e45400640 1 -- 172.21.15.83:0/3316190032 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a897589390 msgr2=0x55a897586f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:18.453 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.452+0000 7f1e45400640 1 --2- 172.21.15.83:0/3316190032 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a897589390 0x55a897586f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:18.453 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.452+0000 7f1e45400640 1 -- 172.21.15.83:0/3316190032 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8974aefd0 con 0x55a89757ead0 2024-04-24T09:09:18.453 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.453+0000 7f1e45400640 1 --2- 172.21.15.83:0/3316190032 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a89757ead0 0x55a8975869f0 secure :-1 s=READY pgs=2372 cs=0 l=1 rev1=1 crypto rx=0x7f1e34005f60 tx=0x7f1e34005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dc399bc5236983b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:18.453 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.453+0000 7f1e27e00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:09:18.453 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.454+0000 7f1e27e00640 1 -- 172.21.15.83:0/3316190032 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e3401ce60 con 0x55a89757ead0 2024-04-24T09:09:18.455 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.454+0000 7f1e27e00640 1 -- 172.21.15.83:0/3316190032 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9583+0+0 (secure 0 0 0) 0x7f1e3401d820 con 0x55a89757ead0 2024-04-24T09:09:18.455 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.454+0000 7f1e4fb87740 1 -- 172.21.15.83:0/3316190032 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e380047a0 con 0x55a89757ead0 2024-04-24T09:09:18.456 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.455+0000 7f1e27e00640 1 -- 172.21.15.83:0/3316190032 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f1e3401da60 con 0x55a89757ead0 2024-04-24T09:09:18.456 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.455+0000 7f1e4fb87740 10 client.21804 fetch_fsmap learned FSMap version 58 2024-04-24T09:09:18.456 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.455+0000 7f1e4fb87740 1 -- 172.21.15.83:0/3316190032 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55a8971f7210 con 0x55a89757ead0 2024-04-24T09:09:18.456 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.455+0000 7f1e27e00640 1 -- 172.21.15.83:0/3316190032 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1e380047a0 con 0x55a89757ead0 2024-04-24T09:09:18.456 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.456+0000 7f1e27e00640 1 -- 172.21.15.83:0/3316190032 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 58) ==== 8374+0+0 (secure 0 0 0) 0x7f1e34032420 con 0x55a89757ead0 2024-04-24T09:09:18.456 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.456+0000 7f1e4fb87740 10 client.21804 fetch_fsmap finished waiting for FSMap version 58 2024-04-24T09:09:18.456 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.456+0000 7f1e4fb87740 10 client.21804 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:09:18.456 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.456+0000 7f1e4fb87740 20 client.21804 populate_metadata read hostname 'smithi083' 2024-04-24T09:09:18.456 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.456+0000 7f1e4fb87740 1 --2- 172.21.15.83:0/3316190032 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55a8975996d0 0x55a89759bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:18.456 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.456+0000 7f1e4fb87740 4 client.21804 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:09:18.457 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.456+0000 7f1e4fb87740 1 -- 172.21.15.83:0/3316190032 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a8974f2320 con 0x55a8975996d0 2024-04-24T09:09:18.457 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.457+0000 7f1e45e00640 1 --2- 172.21.15.83:0/3316190032 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55a8975996d0 0x55a89759bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:18.458 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.458+0000 7f1e45e00640 1 --2- 172.21.15.83:0/3316190032 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55a8975996d0 0x55a89759bac0 crc :-1 s=READY pgs=447 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf445bdce4c809fa server_cookie=5e9f44c6da40b55 in_seq=0 out_seq=0 2024-04-24T09:09:18.458 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.458+0000 7f1e27e00640 10 client.21804 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:09:18.458 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.459+0000 7f1e27e00640 1 -- 172.21.15.83:0/3316190032 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a8974f2320 con 0x55a8975996d0 2024-04-24T09:09:18.458 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.459+0000 7f1e27e00640 10 client.21804 handle_command_reply: tid=0 2024-04-24T09:09:18.458 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.459+0000 7f1e4fb87740 1 client.21804 shutdown 2024-04-24T09:09:18.459 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.460+0000 7f1e4fb87740 1 -- 172.21.15.83:0/3316190032 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a89757ead0 msgr2=0x55a8975869f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:18.459 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.460+0000 7f1e4fb87740 1 --2- 172.21.15.83:0/3316190032 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a89757ead0 0x55a8975869f0 secure :-1 s=READY pgs=2372 cs=0 l=1 rev1=1 crypto rx=0x7f1e34005f60 tx=0x7f1e34005f90 comp rx=0 tx=0).stop 2024-04-24T09:09:18.460 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.460+0000 7f1e4fb87740 1 -- 172.21.15.83:0/3316190032 shutdown_connections 2024-04-24T09:09:18.460 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.460+0000 7f1e4fb87740 1 --2- 172.21.15.83:0/3316190032 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55a8975996d0 0x55a89759bac0 crc :-1 s=READY pgs=447 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:18.460 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.461+0000 7f1e4fb87740 1 --2- 172.21.15.83:0/3316190032 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a897589390 0x55a897586f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:18.460 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.461+0000 7f1e4fb87740 1 --2- 172.21.15.83:0/3316190032 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a89757ead0 0x55a8975869f0 unknown :-1 s=CLOSED pgs=2372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:18.461 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.461+0000 7f1e4fb87740 1 --2- 172.21.15.83:0/3316190032 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a89757e1b0 0x55a8975864b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:18.461 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.461+0000 7f1e4fb87740 1 -- 172.21.15.83:0/3316190032 >> 172.21.15.83:0/3316190032 conn(0x55a89757a6c0 msgr2=0x55a897580440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:18.461 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.461+0000 7f1e27e00640 0 client.21804 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:09:18.461 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.461+0000 7f1e4fb87740 1 -- 172.21.15.83:0/3316190032 shutdown_connections 2024-04-24T09:09:18.461 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.462+0000 7f1e4fb87740 1 -- 172.21.15.83:0/3316190032 wait complete. 2024-04-24T09:09:18.461 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.462+0000 7f1e4fb87740 20 client.21804 trim_cache size 0 max 16384 2024-04-24T09:09:18.496 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.497+0000 7f1e4fb87740 1 Processor -- start 2024-04-24T09:09:18.496 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.497+0000 7f1e4fb87740 1 -- start start 2024-04-24T09:09:18.497 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.499+0000 7f1e4fb87740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a89757e1b0 0x55a8975a1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:18.498 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.499+0000 7f1e4fb87740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a897589390 0x55a8975a1890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:18.498 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.499+0000 7f1e4fb87740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a897581e60 0x55a897578fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:18.498 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.499+0000 7f1e4fb87740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1e481be3d0 con 0x55a897581e60 2024-04-24T09:09:18.498 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.499+0000 7f1e4fb87740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1e481bff10 con 0x55a897589390 2024-04-24T09:09:18.498 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.499+0000 7f1e4fb87740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1e481beaa0 con 0x55a89757e1b0 2024-04-24T09:09:18.499 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.499+0000 7f1e45e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a89757e1b0 0x55a8975a1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:18.499 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.500+0000 7f1e45400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a897589390 0x55a8975a1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:18.499 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.500+0000 7f1e45e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a89757e1b0 0x55a8975a1350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:51122/0 (socket says 172.21.15.83:51122) 2024-04-24T09:09:18.499 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.500+0000 7f1e45400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a897589390 0x55a8975a1890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:53370/0 (socket says 172.21.15.83:53370) 2024-04-24T09:09:18.499 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.500+0000 7f1e45400640 1 -- 172.21.15.83:0/2791901183 learned_addr learned my addr 172.21.15.83:0/2791901183 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:09:18.500 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.500+0000 7f1e45400640 1 -- 172.21.15.83:0/2791901183 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a89757e1b0 msgr2=0x55a8975a1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:18.500 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.500+0000 7f1e45400640 1 --2- 172.21.15.83:0/2791901183 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a89757e1b0 0x55a8975a1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:18.500 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.500+0000 7f1e45400640 1 -- 172.21.15.83:0/2791901183 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a897581e60 msgr2=0x55a897578fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:18.500 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.500+0000 7f1e45400640 1 --2- 172.21.15.83:0/2791901183 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a897581e60 0x55a897578fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:18.500 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.500+0000 7f1e45400640 1 -- 172.21.15.83:0/2791901183 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8972c3620 con 0x55a897589390 2024-04-24T09:09:18.501 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.501+0000 7f1e45400640 1 --2- 172.21.15.83:0/2791901183 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a897589390 0x55a8975a1890 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1e34009b00 tx=0x7f1e3401cca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f031b809df901968 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:18.501 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.501+0000 7f1e32a00640 1 -- 172.21.15.83:0/2791901183 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e34005b90 con 0x55a897589390 2024-04-24T09:09:18.501 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.501+0000 7f1e32a00640 1 -- 172.21.15.83:0/2791901183 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1e34032420 con 0x55a897589390 2024-04-24T09:09:18.501 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.501+0000 7f1e32a00640 1 -- 172.21.15.83:0/2791901183 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e3401d860 con 0x55a897589390 2024-04-24T09:09:18.502 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.502+0000 7f1e4fb87740 1 -- 172.21.15.83:0/2791901183 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a897589390 msgr2=0x55a8975a1890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:18.502 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.502+0000 7f1e4fb87740 1 --2- 172.21.15.83:0/2791901183 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a897589390 0x55a8975a1890 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1e34009b00 tx=0x7f1e3401cca0 comp rx=0 tx=0).stop 2024-04-24T09:09:18.502 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.502+0000 7f1e4fb87740 1 -- 172.21.15.83:0/2791901183 shutdown_connections 2024-04-24T09:09:18.502 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.502+0000 7f1e4fb87740 1 --2- 172.21.15.83:0/2791901183 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a897581e60 0x55a897578fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:18.502 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.502+0000 7f1e4fb87740 1 --2- 172.21.15.83:0/2791901183 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a897589390 0x55a8975a1890 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:18.502 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.502+0000 7f1e4fb87740 1 --2- 172.21.15.83:0/2791901183 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a89757e1b0 0x55a8975a1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:18.502 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.502+0000 7f1e4fb87740 1 -- 172.21.15.83:0/2791901183 >> 172.21.15.83:0/2791901183 conn(0x55a89757a6c0 msgr2=0x55a897581750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:18.502 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.503+0000 7f1e4fb87740 1 -- 172.21.15.83:0/2791901183 shutdown_connections 2024-04-24T09:09:18.502 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.503+0000 7f1e4fb87740 1 -- 172.21.15.83:0/2791901183 wait complete. 2024-04-24T09:09:18.503 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.503+0000 7f1e4fb87740 1 -- start start 2024-04-24T09:09:18.503 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.504+0000 7f1e4fb87740 1 Processor -- start 2024-04-24T09:09:18.504 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.504+0000 7f1e4fb87740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a89757e1b0 0x55a897583010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:18.504 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.504+0000 7f1e4fb87740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a897581e60 0x55a897583550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:18.504 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.504+0000 7f1e4fb87740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a897589390 0x55a897583a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:18.504 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.504+0000 7f1e4fb87740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1e481bf840 con 0x55a89757e1b0 2024-04-24T09:09:18.504 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.504+0000 7f1e4fb87740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1e481bf170 con 0x55a897589390 2024-04-24T09:09:18.504 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.504+0000 7f1e4fb87740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1e48110e40 con 0x55a897581e60 2024-04-24T09:09:18.504 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.505+0000 7f1e4fb87740 10 client.0 _ll_register_callbacks cb 0x7f1e34009970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:09:18.505 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.505+0000 7f1e45e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a89757e1b0 0x55a897583010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:18.505 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.505+0000 7f1e45e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a89757e1b0 0x55a897583010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60534/0 (socket says 172.21.15.83:60534) 2024-04-24T09:09:18.505 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.505+0000 7f1e45e00640 1 -- 172.21.15.83:0/3013281510 learned_addr learned my addr 172.21.15.83:0/3013281510 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:09:18.505 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.505+0000 7f1e45400640 1 --2- 172.21.15.83:0/3013281510 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a897581e60 0x55a897583550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:18.506 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.506+0000 7f1e46800640 1 --2- 172.21.15.83:0/3013281510 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a897589390 0x55a897583a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:18.506 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.506+0000 7f1e45400640 1 -- 172.21.15.83:0/3013281510 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a897589390 msgr2=0x55a897583a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:18.506 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.506+0000 7f1e45400640 1 --2- 172.21.15.83:0/3013281510 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a897589390 0x55a897583a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:18.506 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.506+0000 7f1e45400640 1 -- 172.21.15.83:0/3013281510 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a89757e1b0 msgr2=0x55a897583010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:18.506 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.506+0000 7f1e45400640 1 --2- 172.21.15.83:0/3013281510 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a89757e1b0 0x55a897583010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:18.506 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.506+0000 7f1e45400640 1 -- 172.21.15.83:0/3013281510 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8975febc0 con 0x55a897581e60 2024-04-24T09:09:18.506 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.506+0000 7f1e45e00640 1 --2- 172.21.15.83:0/3013281510 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a89757e1b0 0x55a897583010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:09:18.506 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.507+0000 7f1e45400640 1 --2- 172.21.15.83:0/3013281510 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a897581e60 0x55a897583550 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x55a897595da0 tx=0x7f1e34005b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b489bc3b2b25606e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:18.506 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.507+0000 7f1e27e00640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-04-24T09:09:18.507 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.508+0000 7f1e27e00640 1 -- 172.21.15.83:0/3013281510 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e34032d00 con 0x55a897581e60 2024-04-24T09:09:18.507 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.508+0000 7f1e27e00640 1 -- 172.21.15.83:0/3013281510 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9583+0+0 (secure 0 0 0) 0x7f1e34028480 con 0x55a897581e60 2024-04-24T09:09:18.507 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.508+0000 7f1e4fb87740 1 -- 172.21.15.83:0/3013281510 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e34032420 con 0x55a897581e60 2024-04-24T09:09:18.508 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.509+0000 7f1e27e00640 1 -- 172.21.15.83:0/3013281510 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f1e3404a050 con 0x55a897581e60 2024-04-24T09:09:18.508 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.509+0000 7f1e4fb87740 10 client.28891 fetch_fsmap learned FSMap version 58 2024-04-24T09:09:18.508 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.509+0000 7f1e4fb87740 1 -- 172.21.15.83:0/3013281510 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55a8975ffd70 con 0x55a897581e60 2024-04-24T09:09:18.508 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.510+0000 7f1e27e00640 1 -- 172.21.15.83:0/3013281510 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1e34032420 con 0x55a897581e60 2024-04-24T09:09:18.509 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.510+0000 7f1e27e00640 1 -- 172.21.15.83:0/3013281510 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 58) ==== 8374+0+0 (secure 0 0 0) 0x7f1e3403c760 con 0x55a897581e60 2024-04-24T09:09:18.509 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.510+0000 7f1e4fb87740 10 client.28891 fetch_fsmap finished waiting for FSMap version 58 2024-04-24T09:09:18.509 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.510+0000 7f1e4fb87740 10 client.28891 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:09:18.509 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.510+0000 7f1e4fb87740 20 client.28891 populate_metadata read hostname 'smithi083' 2024-04-24T09:09:18.509 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.510+0000 7f1e4fb87740 1 --2- 172.21.15.83:0/3013281510 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55a8976003c0 0x55a8976027b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:18.510 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.510+0000 7f1e4fb87740 4 client.28891 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T09:09:18.510 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.510+0000 7f1e4fb87740 1 -- 172.21.15.83:0/3013281510 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a8974ec190 con 0x55a8976003c0 2024-04-24T09:09:18.510 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.511+0000 7f1e45e00640 1 --2- 172.21.15.83:0/3013281510 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55a8976003c0 0x55a8976027b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:18.511 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.512+0000 7f1e45e00640 1 --2- 172.21.15.83:0/3013281510 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55a8976003c0 0x55a8976027b0 crc :-1 s=READY pgs=448 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e0d80311881c9e2 server_cookie=baf6c4f0d21f0da2 in_seq=0 out_seq=0 2024-04-24T09:09:18.511 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.512+0000 7f1e27e00640 10 client.28891 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:09:18.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.530+0000 7f7ce7200640 1 -- 172.21.15.83:0/700908495 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ce0104320 msgr2=0x7f7ce0104700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:18.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.530+0000 7f7ce7200640 1 --2- 172.21.15.83:0/700908495 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ce0104320 0x7f7ce0104700 secure :-1 s=READY pgs=2373 cs=0 l=1 rev1=1 crypto rx=0x7f7cd4009900 tx=0x7f7cd402a410 comp rx=0 tx=0).stop 2024-04-24T09:09:18.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.531+0000 7f7ce7200640 1 -- 172.21.15.83:0/700908495 shutdown_connections 2024-04-24T09:09:18.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.531+0000 7f7ce7200640 1 --2- 172.21.15.83:0/700908495 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7ce01093f0 0x7f7ce0110fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:18.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.531+0000 7f7ce7200640 1 --2- 172.21.15.83:0/700908495 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ce0104cd0 0x7f7ce0108d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:18.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.531+0000 7f7ce7200640 1 --2- 172.21.15.83:0/700908495 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ce0104320 0x7f7ce0104700 unknown :-1 s=CLOSED pgs=2373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:18.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.531+0000 7f7ce7200640 1 -- 172.21.15.83:0/700908495 >> 172.21.15.83:0/700908495 conn(0x7f7ce00ffb40 msgr2=0x7f7ce0101f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:18.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.531+0000 7f7ce7200640 1 -- 172.21.15.83:0/700908495 shutdown_connections 2024-04-24T09:09:18.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.532+0000 7f7ce7200640 1 -- 172.21.15.83:0/700908495 wait complete. 2024-04-24T09:09:18.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.533+0000 7f7ce7200640 1 Processor -- start 2024-04-24T09:09:18.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.533+0000 7f7ce7200640 1 -- start start 2024-04-24T09:09:18.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.533+0000 7f7ce7200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7ce0104320 0x7f7ce01a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:18.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.534+0000 7f7ce7200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ce01093f0 0x7f7ce01a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:18.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.534+0000 7f7ce7200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ce019a710 0x7f7ce019ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:18.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.534+0000 7f7ce7200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7ce01143b0 con 0x7f7ce019a710 2024-04-24T09:09:18.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.534+0000 7f7ce7200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7ce0114230 con 0x7f7ce01093f0 2024-04-24T09:09:18.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.534+0000 7f7ce7200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7ce0114530 con 0x7f7ce0104320 2024-04-24T09:09:18.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.534+0000 7f7ce6800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ce019a710 0x7f7ce019ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:18.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.534+0000 7f7ce6800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ce019a710 0x7f7ce019ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60548/0 (socket says 172.21.15.83:60548) 2024-04-24T09:09:18.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.534+0000 7f7ce6800640 1 -- 172.21.15.83:0/844525737 learned_addr learned my addr 172.21.15.83:0/844525737 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:09:18.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.534+0000 7f7ce5400640 1 --2- 172.21.15.83:0/844525737 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ce01093f0 0x7f7ce01a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:18.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.534+0000 7f7ce6800640 1 -- 172.21.15.83:0/844525737 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7ce0104320 msgr2=0x7f7ce01a0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:18.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.534+0000 7f7ce6800640 1 --2- 172.21.15.83:0/844525737 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7ce0104320 0x7f7ce01a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:18.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.534+0000 7f7ce6800640 1 -- 172.21.15.83:0/844525737 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ce01093f0 msgr2=0x7f7ce01a0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:18.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.534+0000 7f7ce6800640 1 --2- 172.21.15.83:0/844525737 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ce01093f0 0x7f7ce01a0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:18.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.535+0000 7f7ce6800640 1 -- 172.21.15.83:0/844525737 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ce019b310 con 0x7f7ce019a710 2024-04-24T09:09:18.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.539+0000 7f7ce6800640 1 --2- 172.21.15.83:0/844525737 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ce019a710 0x7f7ce019ab90 secure :-1 s=READY pgs=2374 cs=0 l=1 rev1=1 crypto rx=0x7f7cdc00eaa0 tx=0x7f7cdc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:18.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.541+0000 7f7cdaa00640 1 -- 172.21.15.83:0/844525737 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cdc00cdb0 con 0x7f7ce019a710 2024-04-24T09:09:18.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.541+0000 7f7cdaa00640 1 -- 172.21.15.83:0/844525737 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7cdc005020 con 0x7f7ce019a710 2024-04-24T09:09:18.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.541+0000 7f7cdaa00640 1 -- 172.21.15.83:0/844525737 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cdc0116d0 con 0x7f7ce019a710 2024-04-24T09:09:18.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.541+0000 7f7ce7200640 1 -- 172.21.15.83:0/844525737 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ce01a73f0 con 0x7f7ce019a710 2024-04-24T09:09:18.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.541+0000 7f7ce7200640 1 -- 172.21.15.83:0/844525737 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ce01a77a0 con 0x7f7ce019a710 2024-04-24T09:09:18.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.550+0000 7f7ceb5c1740 1 Processor -- start 2024-04-24T09:09:18.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.553+0000 7f7cdaa00640 1 -- 172.21.15.83:0/844525737 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7cdc0051d0 con 0x7f7ce019a710 2024-04-24T09:09:18.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.555+0000 7f7ceb5c1740 1 -- start start 2024-04-24T09:09:18.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.557+0000 7f7cdaa00640 1 --2- 172.21.15.83:0/844525737 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f7cc80a3ef0 0x7f7cc80a63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:18.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.558+0000 7f7cdaa00640 1 -- 172.21.15.83:0/844525737 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 9583+0+0 (secure 0 0 0) 0x7f7cdc0d0c90 con 0x7f7ce019a710 2024-04-24T09:09:18.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.560+0000 7f7ceb5c1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a455c9240 0x559a455c9620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:18.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.560+0000 7f7ce5400640 1 --2- 172.21.15.83:0/844525737 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f7cc80a3ef0 0x7f7cc80a63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:18.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.560+0000 7f7ceb5c1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a455c9b60 0x559a455d3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:18.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.560+0000 7f7ceb5c1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a455d4230 0x559a455d6620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:18.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.561+0000 7f7ceb5c1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559a455261f0 con 0x559a455d4230 2024-04-24T09:09:18.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.561+0000 7f7ceb5c1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559a454cc350 con 0x559a455c9240 2024-04-24T09:09:18.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.561+0000 7f7ceb5c1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559a455221b0 con 0x559a455c9b60 2024-04-24T09:09:18.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.561+0000 7f7ce5e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a455c9240 0x559a455c9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:18.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.562+0000 7f7ce6800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a455d4230 0x559a455d6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:18.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.562+0000 7f7ce6800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a455d4230 0x559a455d6620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60564/0 (socket says 172.21.15.83:60564) 2024-04-24T09:09:18.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.562+0000 7f7ce6800640 1 -- 172.21.15.83:0/3436282332 learned_addr learned my addr 172.21.15.83:0/3436282332 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:09:18.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.564+0000 7f7ce5e00640 1 -- 172.21.15.83:0/3436282332 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a455c9b60 msgr2=0x559a455d3cf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T09:09:18.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.564+0000 7f7ce5e00640 1 --2- 172.21.15.83:0/3436282332 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a455c9b60 0x559a455d3cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:18.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.564+0000 7f7ce5e00640 1 -- 172.21.15.83:0/3436282332 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a455d4230 msgr2=0x559a455d6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:18.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.564+0000 7f7ce5e00640 1 --2- 172.21.15.83:0/3436282332 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a455d4230 0x559a455d6620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:18.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.564+0000 7f7ce5e00640 1 -- 172.21.15.83:0/3436282332 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a45520f50 con 0x559a455c9240 2024-04-24T09:09:18.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.564+0000 7f7ce6800640 1 --2- 172.21.15.83:0/3436282332 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a455d4230 0x559a455d6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:09:18.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.565+0000 7f7ce5400640 1 --2- 172.21.15.83:0/844525737 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f7cc80a3ef0 0x7f7cc80a63b0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f7cd00095f0 tx=0x7f7cd0005f90 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:18.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.565+0000 7f7ce5e00640 1 --2- 172.21.15.83:0/3436282332 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a455c9240 0x559a455c9620 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7cd402a930 tx=0x7f7cd4009680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2af50e8a51478140 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:18.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.568+0000 7f7ccf000640 1 -- 172.21.15.83:0/3436282332 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cd403e070 con 0x559a455c9240 2024-04-24T09:09:18.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.568+0000 7f7ccf000640 1 -- 172.21.15.83:0/3436282332 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7cd40054e0 con 0x559a455c9240 2024-04-24T09:09:18.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.568+0000 7f7ccf000640 1 -- 172.21.15.83:0/3436282332 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cd4035040 con 0x559a455c9240 2024-04-24T09:09:18.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.569+0000 7f7ceb5c1740 1 -- 172.21.15.83:0/3436282332 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a455c9240 msgr2=0x559a455c9620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:18.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.569+0000 7f7ceb5c1740 1 --2- 172.21.15.83:0/3436282332 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a455c9240 0x559a455c9620 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7cd402a930 tx=0x7f7cd4009680 comp rx=0 tx=0).stop 2024-04-24T09:09:18.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.572+0000 7f7ceb5c1740 1 -- 172.21.15.83:0/3436282332 shutdown_connections 2024-04-24T09:09:18.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.572+0000 7f7ceb5c1740 1 --2- 172.21.15.83:0/3436282332 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a455d4230 0x559a455d6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:18.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.572+0000 7f7ceb5c1740 1 --2- 172.21.15.83:0/3436282332 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a455c9b60 0x559a455d3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:18.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.572+0000 7f7ceb5c1740 1 --2- 172.21.15.83:0/3436282332 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a455c9240 0x559a455c9620 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:18.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.572+0000 7f7ceb5c1740 1 -- 172.21.15.83:0/3436282332 >> 172.21.15.83:0/3436282332 conn(0x559a455c5560 msgr2=0x559a455c5940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:18.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.572+0000 7f7ceb5c1740 1 -- 172.21.15.83:0/3436282332 shutdown_connections 2024-04-24T09:09:18.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.573+0000 7f7ceb5c1740 1 -- 172.21.15.83:0/3436282332 wait complete. 2024-04-24T09:09:18.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.574+0000 7f7ceb5c1740 1 -- start start 2024-04-24T09:09:18.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.575+0000 7f7ceb5c1740 1 Processor -- start 2024-04-24T09:09:18.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.577+0000 7f7ceb5c1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a455c9240 0x559a455d1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:18.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.577+0000 7f7ceb5c1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a455c9b60 0x559a455d16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:18.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.577+0000 7f7ceb5c1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a455d4230 0x559a455d1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:18.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.578+0000 7f7ceb5c1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559a455d6fc0 con 0x559a455c9240 2024-04-24T09:09:18.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.578+0000 7f7ceb5c1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559a454df690 con 0x559a455d4230 2024-04-24T09:09:18.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.578+0000 7f7ceb5c1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559a455d7140 con 0x559a455c9b60 2024-04-24T09:09:18.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.578+0000 7f7ceb5c1740 10 client.0 _ll_register_callbacks cb 0x559a4551b650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:09:18.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.578+0000 7f7ce5400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a455c9b60 0x559a455d16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:18.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.578+0000 7f7ce5400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a455c9b60 0x559a455d16a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:51154/0 (socket says 172.21.15.83:51154) 2024-04-24T09:09:18.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.578+0000 7f7ce5400640 1 -- 172.21.15.83:0/2245163291 learned_addr learned my addr 172.21.15.83:0/2245163291 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:09:18.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.579+0000 7f7ce5e00640 1 --2- 172.21.15.83:0/2245163291 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a455c9240 0x559a455d1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:18.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.579+0000 7f7ce6800640 1 --2- 172.21.15.83:0/2245163291 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a455d4230 0x559a455d1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:18.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.579+0000 7f7ce5400640 1 -- 172.21.15.83:0/2245163291 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a455d4230 msgr2=0x559a455d1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:18.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.579+0000 7f7ce5400640 1 --2- 172.21.15.83:0/2245163291 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a455d4230 0x559a455d1be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:18.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.579+0000 7f7ce5400640 1 -- 172.21.15.83:0/2245163291 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a455c9240 msgr2=0x559a455d1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:18.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.579+0000 7f7ce5400640 1 --2- 172.21.15.83:0/2245163291 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a455c9240 0x559a455d1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:18.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.579+0000 7f7ce5400640 1 -- 172.21.15.83:0/2245163291 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a454de920 con 0x559a455c9b60 2024-04-24T09:09:18.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.580+0000 7f7ce6800640 1 --2- 172.21.15.83:0/2245163291 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a455d4230 0x559a455d1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:09:18.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.580+0000 7f7ce5400640 1 --2- 172.21.15.83:0/2245163291 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a455c9b60 0x559a455d16a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7cd00058a0 tx=0x7f7cd0004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9fa5d4538198cb96 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:18.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.581+0000 7f7cc3400640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-04-24T09:09:18.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.583+0000 7f7cc3400640 1 -- 172.21.15.83:0/2245163291 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cd001cd30 con 0x559a455c9b60 2024-04-24T09:09:18.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.583+0000 7f7cc3400640 1 -- 172.21.15.83:0/2245163291 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9583+0+0 (secure 0 0 0) 0x7f7cd0028480 con 0x559a455c9b60 2024-04-24T09:09:18.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.585+0000 7f7ceb5c1740 1 -- 172.21.15.83:0/2245163291 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7cd40054e0 con 0x559a455c9b60 2024-04-24T09:09:18.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.586+0000 7f7cc3400640 1 -- 172.21.15.83:0/2245163291 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f7cd000ee30 con 0x559a455c9b60 2024-04-24T09:09:18.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.586+0000 7f7ceb5c1740 10 client.28897 fetch_fsmap learned FSMap version 58 2024-04-24T09:09:18.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.586+0000 7f7ceb5c1740 1 -- 172.21.15.83:0/2245163291 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x559a453027d0 con 0x559a455c9b60 2024-04-24T09:09:18.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.588+0000 7f7cc3400640 1 -- 172.21.15.83:0/2245163291 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7cd40054e0 con 0x559a455c9b60 2024-04-24T09:09:18.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.588+0000 7f7cc3400640 1 -- 172.21.15.83:0/2245163291 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 58) ==== 8374+0+0 (secure 0 0 0) 0x7f7cd00327d0 con 0x559a455c9b60 2024-04-24T09:09:18.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.588+0000 7f7ceb5c1740 10 client.28897 fetch_fsmap finished waiting for FSMap version 58 2024-04-24T09:09:18.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.588+0000 7f7ceb5c1740 10 client.28897 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:09:18.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.588+0000 7f7ceb5c1740 20 client.28897 populate_metadata read hostname 'smithi083' 2024-04-24T09:09:18.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.588+0000 7f7ceb5c1740 1 --2- 172.21.15.83:0/2245163291 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x559a455e44f0 0x559a455e6910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:18.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.588+0000 7f7ceb5c1740 4 client.28897 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:09:18.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.588+0000 7f7ceb5c1740 1 -- 172.21.15.83:0/2245163291 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559a45533c10 con 0x559a455e44f0 2024-04-24T09:09:18.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.592+0000 7f7ce5e00640 1 --2- 172.21.15.83:0/2245163291 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x559a455e44f0 0x559a455e6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:18.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.597+0000 7f7ce5e00640 1 --2- 172.21.15.83:0/2245163291 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x559a455e44f0 0x559a455e6910 crc :-1 s=READY pgs=449 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75512714f076cba7 server_cookie=e6ce6414d4e32a4f in_seq=0 out_seq=0 2024-04-24T09:09:18.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.597+0000 7f7cc3400640 10 client.28897 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:09:18.617 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.618+0000 7f1e27e00640 1 -- 172.21.15.83:0/3013281510 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a8974ec190 con 0x55a8976003c0 2024-04-24T09:09:18.617 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.618+0000 7f1e27e00640 10 client.28891 handle_command_reply: tid=0 2024-04-24T09:09:18.617 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.618+0000 7f1e4fb87740 1 client.28891 shutdown 2024-04-24T09:09:18.617 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.618+0000 7f1e4fb87740 1 -- 172.21.15.83:0/3013281510 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a897581e60 msgr2=0x55a897583550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:18.617 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.618+0000 7f1e4fb87740 1 --2- 172.21.15.83:0/3013281510 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a897581e60 0x55a897583550 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x55a897595da0 tx=0x7f1e34005b90 comp rx=0 tx=0).stop 2024-04-24T09:09:18.618 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.618+0000 7f1e4fb87740 1 -- 172.21.15.83:0/3013281510 shutdown_connections 2024-04-24T09:09:18.618 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.618+0000 7f1e4fb87740 1 --2- 172.21.15.83:0/3013281510 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55a8976003c0 0x55a8976027b0 crc :-1 s=READY pgs=448 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:18.618 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.618+0000 7f1e27e00640 0 client.28891 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:09:18.618 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.618+0000 7f1e4fb87740 1 --2- 172.21.15.83:0/3013281510 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a897589390 0x55a897583a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:18.618 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.618+0000 7f1e4fb87740 1 --2- 172.21.15.83:0/3013281510 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a897581e60 0x55a897583550 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:18.619 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.618+0000 7f1e4fb87740 1 --2- 172.21.15.83:0/3013281510 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a89757e1b0 0x55a897583010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:18.619 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.618+0000 7f1e4fb87740 1 -- 172.21.15.83:0/3013281510 >> 172.21.15.83:0/3013281510 conn(0x55a89757a6c0 msgr2=0x55a89757fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:18.619 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.618+0000 7f1e4fb87740 1 -- 172.21.15.83:0/3013281510 shutdown_connections 2024-04-24T09:09:18.619 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.619+0000 7f1e4fb87740 1 -- 172.21.15.83:0/3013281510 wait complete. 2024-04-24T09:09:18.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.619+0000 7f7cc3400640 1 -- 172.21.15.83:0/2245163291 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559a45533c10 con 0x559a455e44f0 2024-04-24T09:09:18.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.619+0000 7f7cc3400640 10 client.28897 handle_command_reply: tid=0 2024-04-24T09:09:18.620 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.619+0000 7f1e4fb87740 20 client.28891 trim_cache size 0 max 16384 2024-04-24T09:09:18.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.619+0000 7f7ceb5c1740 1 client.28897 shutdown 2024-04-24T09:09:18.620 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T09:09:18.620 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T09:09:18.621 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T09:09:18.621 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T09:09:18.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.619+0000 7f7ceb5c1740 1 -- 172.21.15.83:0/2245163291 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a455c9b60 msgr2=0x559a455d16a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:18.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.619+0000 7f7ceb5c1740 1 --2- 172.21.15.83:0/2245163291 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a455c9b60 0x559a455d16a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7cd00058a0 tx=0x7f7cd0004730 comp rx=0 tx=0).stop 2024-04-24T09:09:18.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.619+0000 7f7ceb5c1740 1 -- 172.21.15.83:0/2245163291 shutdown_connections 2024-04-24T09:09:18.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.619+0000 7f7ceb5c1740 1 --2- 172.21.15.83:0/2245163291 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x559a455e44f0 0x559a455e6910 crc :-1 s=READY pgs=449 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:18.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.619+0000 7f7ceb5c1740 1 --2- 172.21.15.83:0/2245163291 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a455d4230 0x559a455d1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:18.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.619+0000 7f7ceb5c1740 1 --2- 172.21.15.83:0/2245163291 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a455c9b60 0x559a455d16a0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:18.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.619+0000 7f7ceb5c1740 1 --2- 172.21.15.83:0/2245163291 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a455c9240 0x559a455d1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:18.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.619+0000 7f7ceb5c1740 1 -- 172.21.15.83:0/2245163291 >> 172.21.15.83:0/2245163291 conn(0x559a455c5560 msgr2=0x559a455bbfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:18.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.619+0000 7f7cc3400640 0 client.28897 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:09:18.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.620+0000 7f7ceb5c1740 1 -- 172.21.15.83:0/2245163291 shutdown_connections 2024-04-24T09:09:18.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.620+0000 7f7ceb5c1740 1 -- 172.21.15.83:0/2245163291 wait complete. 2024-04-24T09:09:18.623 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.620+0000 7f1e4ce00640 1 -- 172.21.15.83:0/2714080367 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f1e2c0a3e70 msgr2=0x7f1e2c0a6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:18.623 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.620+0000 7f1e4ce00640 1 --2- 172.21.15.83:0/2714080367 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f1e2c0a3e70 0x7f1e2c0a6330 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x55a89757bae0 tx=0x7f1e340097e0 comp rx=0 tx=0).stop 2024-04-24T09:09:18.623 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.620+0000 7f1e4ce00640 1 -- 172.21.15.83:0/2714080367 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e481a8980 msgr2=0x7f1e481aad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:18.623 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.620+0000 7f1e4ce00640 1 --2- 172.21.15.83:0/2714080367 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e481a8980 0x7f1e481aad70 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f1e40009dc0 tx=0x7f1e40032ff0 comp rx=0 tx=0).stop 2024-04-24T09:09:18.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.620+0000 7f7ceb5c1740 20 client.28897 trim_cache size 0 max 16384 2024-04-24T09:09:18.624 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.620+0000 7f1e4ce00640 1 -- 172.21.15.83:0/2714080367 shutdown_connections 2024-04-24T09:09:18.624 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.620+0000 7f1e4ce00640 1 --2- 172.21.15.83:0/2714080367 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f1e2c0a3e70 0x7f1e2c0a6330 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:18.624 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.620+0000 7f1e4ce00640 1 --2- 172.21.15.83:0/2714080367 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e481a8980 0x7f1e481aad70 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:18.624 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.620+0000 7f1e4ce00640 1 --2- 172.21.15.83:0/2714080367 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1e48109540 0x7f1e48110080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:18.624 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.620+0000 7f1e4ce00640 1 --2- 172.21.15.83:0/2714080367 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e48104470 0x7f1e4810fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:18.624 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.620+0000 7f1e4ce00640 1 -- 172.21.15.83:0/2714080367 >> 172.21.15.83:0/2714080367 conn(0x7f1e480ffcd0 msgr2=0x7f1e48100f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:18.624 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.620+0000 7f1e4ce00640 1 -- 172.21.15.83:0/2714080367 shutdown_connections 2024-04-24T09:09:18.624 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:18.620+0000 7f1e4ce00640 1 -- 172.21.15.83:0/2714080367 wait complete. 2024-04-24T09:09:18.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.654+0000 7f7ceb5c1740 1 Processor -- start 2024-04-24T09:09:18.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.654+0000 7f7ceb5c1740 1 -- start start 2024-04-24T09:09:18.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.654+0000 7f7ceb5c1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a455c9240 0x559a455dfe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:18.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.654+0000 7f7ceb5c1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a455c9b60 0x559a455ec1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:18.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.654+0000 7f7ceb5c1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a455d4230 0x559a455ec700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:18.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.654+0000 7f7ceb5c1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7ce01ad350 con 0x559a455d4230 2024-04-24T09:09:18.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.654+0000 7f7ceb5c1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7ce01aee90 con 0x559a455c9b60 2024-04-24T09:09:18.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.654+0000 7f7ceb5c1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7ce01ada20 con 0x559a455c9240 2024-04-24T09:09:18.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.654+0000 7f7ce6800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a455d4230 0x559a455ec700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:18.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.654+0000 7f7ce5e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a455c9240 0x559a455dfe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:18.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.654+0000 7f7ce5e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a455c9240 0x559a455dfe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:51156/0 (socket says 172.21.15.83:51156) 2024-04-24T09:09:18.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.654+0000 7f7ce5e00640 1 -- 172.21.15.83:0/4055911021 learned_addr learned my addr 172.21.15.83:0/4055911021 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:09:18.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.655+0000 7f7ce5400640 1 --2- 172.21.15.83:0/4055911021 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a455c9b60 0x559a455ec1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:18.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.655+0000 7f7ce6800640 1 -- 172.21.15.83:0/4055911021 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a455c9240 msgr2=0x559a455dfe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:18.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.655+0000 7f7ce6800640 1 --2- 172.21.15.83:0/4055911021 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a455c9240 0x559a455dfe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:18.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.655+0000 7f7ce6800640 1 -- 172.21.15.83:0/4055911021 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a455c9b60 msgr2=0x559a455ec1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:18.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.655+0000 7f7ce6800640 1 --2- 172.21.15.83:0/4055911021 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a455c9b60 0x559a455ec1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:18.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.655+0000 7f7ce6800640 1 -- 172.21.15.83:0/4055911021 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a452383f0 con 0x559a455d4230 2024-04-24T09:09:18.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.655+0000 7f7ce5e00640 1 --2- 172.21.15.83:0/4055911021 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a455c9240 0x559a455dfe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:09:18.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.655+0000 7f7ce6800640 1 --2- 172.21.15.83:0/4055911021 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a455d4230 0x559a455ec700 secure :-1 s=READY pgs=2375 cs=0 l=1 rev1=1 crypto rx=0x7f7cdc099180 tx=0x7f7cdc00ea20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91e1c3f0b2891633 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:18.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.656+0000 7f7cce600640 1 -- 172.21.15.83:0/4055911021 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cdc01f040 con 0x559a455d4230 2024-04-24T09:09:18.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.656+0000 7f7cce600640 1 -- 172.21.15.83:0/4055911021 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7cdc005560 con 0x559a455d4230 2024-04-24T09:09:18.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.656+0000 7f7cce600640 1 -- 172.21.15.83:0/4055911021 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cdc09dd40 con 0x559a455d4230 2024-04-24T09:09:18.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.656+0000 7f7ceb5c1740 1 -- 172.21.15.83:0/4055911021 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a455d4230 msgr2=0x559a455ec700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:18.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.656+0000 7f7ceb5c1740 1 --2- 172.21.15.83:0/4055911021 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a455d4230 0x559a455ec700 secure :-1 s=READY pgs=2375 cs=0 l=1 rev1=1 crypto rx=0x7f7cdc099180 tx=0x7f7cdc00ea20 comp rx=0 tx=0).stop 2024-04-24T09:09:18.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.657+0000 7f7ceb5c1740 1 -- 172.21.15.83:0/4055911021 shutdown_connections 2024-04-24T09:09:18.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.657+0000 7f7ceb5c1740 1 --2- 172.21.15.83:0/4055911021 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a455d4230 0x559a455ec700 unknown :-1 s=CLOSED pgs=2375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:18.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.657+0000 7f7ceb5c1740 1 --2- 172.21.15.83:0/4055911021 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a455c9b60 0x559a455ec1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:18.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.657+0000 7f7ceb5c1740 1 --2- 172.21.15.83:0/4055911021 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a455c9240 0x559a455dfe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:18.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.657+0000 7f7ceb5c1740 1 -- 172.21.15.83:0/4055911021 >> 172.21.15.83:0/4055911021 conn(0x559a455c5560 msgr2=0x559a455cc470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:18.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.657+0000 7f7ceb5c1740 1 -- 172.21.15.83:0/4055911021 shutdown_connections 2024-04-24T09:09:18.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.657+0000 7f7ceb5c1740 1 -- 172.21.15.83:0/4055911021 wait complete. 2024-04-24T09:09:18.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.657+0000 7f7ceb5c1740 1 -- start start 2024-04-24T09:09:18.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.657+0000 7f7ceb5c1740 1 Processor -- start 2024-04-24T09:09:18.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.658+0000 7f7ceb5c1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a455c9240 0x559a455dc480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:18.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.658+0000 7f7ceb5c1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a455c9b60 0x559a455d78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:18.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.658+0000 7f7ceb5c1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a455d4230 0x559a455d7e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:18.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.658+0000 7f7ceb5c1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7ce01ae7c0 con 0x559a455c9b60 2024-04-24T09:09:18.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.658+0000 7f7ceb5c1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7ce01ae0f0 con 0x559a455c9240 2024-04-24T09:09:18.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.658+0000 7f7ceb5c1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7ce01ab350 con 0x559a455d4230 2024-04-24T09:09:18.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.658+0000 7f7ceb5c1740 10 client.0 _ll_register_callbacks cb 0x7f7cd0009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:09:18.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.658+0000 7f7ce5e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a455c9240 0x559a455dc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:18.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.658+0000 7f7ce5e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a455c9240 0x559a455dc480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:53432/0 (socket says 172.21.15.83:53432) 2024-04-24T09:09:18.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.658+0000 7f7ce5e00640 1 -- 172.21.15.83:0/3854024248 learned_addr learned my addr 172.21.15.83:0/3854024248 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:09:18.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.658+0000 7f7ce5400640 1 --2- 172.21.15.83:0/3854024248 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a455c9b60 0x559a455d78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:18.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.658+0000 7f7ce6800640 1 --2- 172.21.15.83:0/3854024248 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a455d4230 0x559a455d7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:18.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.659+0000 7f7ce6800640 1 -- 172.21.15.83:0/3854024248 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a455c9240 msgr2=0x559a455dc480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:18.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.659+0000 7f7ce6800640 1 --2- 172.21.15.83:0/3854024248 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a455c9240 0x559a455dc480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:18.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.659+0000 7f7ce6800640 1 -- 172.21.15.83:0/3854024248 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a455c9b60 msgr2=0x559a455d78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:18.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.659+0000 7f7ce6800640 1 --2- 172.21.15.83:0/3854024248 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a455c9b60 0x559a455d78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:18.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.659+0000 7f7ce6800640 1 -- 172.21.15.83:0/3854024248 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a451f80f0 con 0x559a455d4230 2024-04-24T09:09:18.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.659+0000 7f7ce5400640 1 --2- 172.21.15.83:0/3854024248 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a455c9b60 0x559a455d78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:09:18.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.659+0000 7f7ce6800640 1 --2- 172.21.15.83:0/3854024248 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a455d4230 0x559a455d7e30 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7cdc099180 tx=0x7f7cdc005780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60e8116843feebb5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:18.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.659+0000 7f7cc3400640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-04-24T09:09:18.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.659+0000 7f7cc3400640 1 -- 172.21.15.83:0/3854024248 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cdc0b9030 con 0x559a455d4230 2024-04-24T09:09:18.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.659+0000 7f7ceb5c1740 1 -- 172.21.15.83:0/3854024248 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7cdc005560 con 0x559a455d4230 2024-04-24T09:09:18.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.659+0000 7f7cc3400640 1 -- 172.21.15.83:0/3854024248 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9583+0+0 (secure 0 0 0) 0x7f7cdc09be10 con 0x559a455d4230 2024-04-24T09:09:18.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.659+0000 7f7cc3400640 1 -- 172.21.15.83:0/3854024248 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f7cdc01f040 con 0x559a455d4230 2024-04-24T09:09:18.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.659+0000 7f7ceb5c1740 10 client.28903 fetch_fsmap learned FSMap version 58 2024-04-24T09:09:18.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.659+0000 7f7ceb5c1740 1 -- 172.21.15.83:0/3854024248 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x559a454eb810 con 0x559a455d4230 2024-04-24T09:09:18.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.660+0000 7f7cc3400640 1 -- 172.21.15.83:0/3854024248 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7cdc005560 con 0x559a455d4230 2024-04-24T09:09:18.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.660+0000 7f7cc3400640 1 -- 172.21.15.83:0/3854024248 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 58) ==== 8374+0+0 (secure 0 0 0) 0x7f7cdc0b3020 con 0x559a455d4230 2024-04-24T09:09:18.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.660+0000 7f7ceb5c1740 10 client.28903 fetch_fsmap finished waiting for FSMap version 58 2024-04-24T09:09:18.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.660+0000 7f7ceb5c1740 10 client.28903 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:09:18.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.660+0000 7f7ceb5c1740 20 client.28903 populate_metadata read hostname 'smithi083' 2024-04-24T09:09:18.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.660+0000 7f7ceb5c1740 1 --2- 172.21.15.83:0/3854024248 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x559a4564aa30 0x559a4564ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:18.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.660+0000 7f7ceb5c1740 4 client.28903 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T09:09:18.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.660+0000 7f7ceb5c1740 1 -- 172.21.15.83:0/3854024248 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559a4552da80 con 0x559a4564aa30 2024-04-24T09:09:18.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.736+0000 7f7ce5e00640 1 --2- 172.21.15.83:0/3854024248 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x559a4564aa30 0x559a4564ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:18.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.738+0000 7f7ce5e00640 1 --2- 172.21.15.83:0/3854024248 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x559a4564aa30 0x559a4564ce20 crc :-1 s=READY pgs=450 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea6a75453ccbf788 server_cookie=429bde8c83f18509 in_seq=0 out_seq=0 2024-04-24T09:09:18.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.738+0000 7f7cc3400640 10 client.28903 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:09:18.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.745+0000 7f7cc3400640 1 -- 172.21.15.83:0/3854024248 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x559a4552da80 con 0x559a4564aa30 2024-04-24T09:09:18.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.745+0000 7f7cc3400640 10 client.28903 handle_command_reply: tid=0 2024-04-24T09:09:18.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.745+0000 7f7ceb5c1740 1 client.28903 shutdown 2024-04-24T09:09:18.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.745+0000 7f7ceb5c1740 1 -- 172.21.15.83:0/3854024248 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a455d4230 msgr2=0x559a455d7e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:18.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.745+0000 7f7ceb5c1740 1 --2- 172.21.15.83:0/3854024248 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a455d4230 0x559a455d7e30 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7cdc099180 tx=0x7f7cdc005780 comp rx=0 tx=0).stop 2024-04-24T09:09:18.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.746+0000 7f7ceb5c1740 1 -- 172.21.15.83:0/3854024248 shutdown_connections 2024-04-24T09:09:18.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.746+0000 7f7ceb5c1740 1 --2- 172.21.15.83:0/3854024248 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x559a4564aa30 0x559a4564ce20 crc :-1 s=READY pgs=450 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:18.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.746+0000 7f7ceb5c1740 1 --2- 172.21.15.83:0/3854024248 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a455d4230 0x559a455d7e30 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:18.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.746+0000 7f7ceb5c1740 1 --2- 172.21.15.83:0/3854024248 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a455c9b60 0x559a455d78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:18.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.746+0000 7f7cc3400640 0 client.28903 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:09:18.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.746+0000 7f7ceb5c1740 1 --2- 172.21.15.83:0/3854024248 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a455c9240 0x559a455dc480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:18.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.746+0000 7f7ceb5c1740 1 -- 172.21.15.83:0/3854024248 >> 172.21.15.83:0/3854024248 conn(0x559a455c5560 msgr2=0x559a455bb740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:18.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.746+0000 7f7ceb5c1740 1 -- 172.21.15.83:0/3854024248 shutdown_connections 2024-04-24T09:09:18.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.746+0000 7f7ceb5c1740 1 -- 172.21.15.83:0/3854024248 wait complete. 2024-04-24T09:09:18.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.747+0000 7f7ceb5c1740 20 client.28903 trim_cache size 0 max 16384 2024-04-24T09:09:18.747 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T09:09:18.747 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T09:09:18.747 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-04-24T09:09:18.748 INFO:teuthology.orchestra.run.smithi083.stdout: "62024573-0364-4ade-abd8-e4f8620a7d40": { 2024-04-24T09:09:18.748 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-04-24T09:09:18.748 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "62024573-0364-4ade-abd8-e4f8620a7d40", 2024-04-24T09:09:18.748 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-04-24T09:09:18.748 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:09:18.748 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:09:18.748 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T09:09:18.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.748+0000 7f7ce7200640 1 -- 172.21.15.83:0/844525737 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f7cc80a3ef0 msgr2=0x7f7cc80a63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:18.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.748+0000 7f7ce7200640 1 --2- 172.21.15.83:0/844525737 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f7cc80a3ef0 0x7f7cc80a63b0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f7cd00095f0 tx=0x7f7cd0005f90 comp rx=0 tx=0).stop 2024-04-24T09:09:18.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.748+0000 7f7ce7200640 1 -- 172.21.15.83:0/844525737 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ce019a710 msgr2=0x7f7ce019ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:18.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.748+0000 7f7ce7200640 1 --2- 172.21.15.83:0/844525737 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ce019a710 0x7f7ce019ab90 secure :-1 s=READY pgs=2374 cs=0 l=1 rev1=1 crypto rx=0x7f7cdc00eaa0 tx=0x7f7cdc00ef60 comp rx=0 tx=0).stop 2024-04-24T09:09:18.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.749+0000 7f7ce7200640 1 -- 172.21.15.83:0/844525737 shutdown_connections 2024-04-24T09:09:18.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.749+0000 7f7ce7200640 1 --2- 172.21.15.83:0/844525737 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f7cc80a3ef0 0x7f7cc80a63b0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:18.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.749+0000 7f7ce7200640 1 --2- 172.21.15.83:0/844525737 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ce019a710 0x7f7ce019ab90 unknown :-1 s=CLOSED pgs=2374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:18.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.749+0000 7f7ce7200640 1 --2- 172.21.15.83:0/844525737 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ce01093f0 0x7f7ce01a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:18.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.749+0000 7f7ce7200640 1 --2- 172.21.15.83:0/844525737 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7ce0104320 0x7f7ce01a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:18.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.749+0000 7f7ce7200640 1 -- 172.21.15.83:0/844525737 >> 172.21.15.83:0/844525737 conn(0x7f7ce00ffb40 msgr2=0x7f7ce0100f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:18.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.749+0000 7f7ce7200640 1 -- 172.21.15.83:0/844525737 shutdown_connections 2024-04-24T09:09:18.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:18.749+0000 7f7ce7200640 1 -- 172.21.15.83:0/844525737 wait complete. 2024-04-24T09:09:18.790 INFO:tasks.cephfs.filesystem:scrub status for tag:62024573-0364-4ade-abd8-e4f8620a7d40 - {'path': '/', 'tag': '62024573-0364-4ade-abd8-e4f8620a7d40', 'options': 'recursive,force'} 2024-04-24T09:09:19.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:09:18 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:09:19.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:09:18 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:09:19.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:09:18 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:09:19.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:09:18 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:09:19.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:09:18 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:09:19.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:09:18 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:09:19.169 INFO:tasks.workunit.client.0.smithi083.stdout: 10 29548 7.01 MB/sec execute 110 sec latency 197.262 ms 2024-04-24T09:09:19.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:09:18 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:09:19.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:09:18 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:09:19.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:09:18 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:09:19.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:09:18 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:09:19.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:09:18 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:09:19.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:09:18 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:09:19.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:09:18 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:09:19.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:09:18 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:09:19.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:09:18 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:09:19.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:09:18 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:09:19.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:09:18 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:09:19.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:09:18 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:09:20.169 INFO:tasks.workunit.client.0.smithi083.stdout: 10 29658 7.00 MB/sec execute 111 sec latency 92.692 ms 2024-04-24T09:09:20.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:09:19 smithi083 ceph-mon[27907]: scrub queued for path: / 2024-04-24T09:09:20.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:09:19 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:09:20.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:09:19 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:09:20.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:09:19 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:09:20.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:09:19 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:09:20.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:09:19 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:09:20.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:09:19 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:09:20.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:09:19 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:09:20.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:09:19 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:09:20.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:09:19 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:09:20.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:09:19 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:09:20.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:09:19 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:09:20.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:09:19 smithi083 ceph-mon[27907]: pgmap v558: 129 pgs: 129 active+clean; 562 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.4 MiB/s wr, 309 op/s 2024-04-24T09:09:20.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:09:19 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:09:20.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:09:19 smithi189 ceph-mon[31791]: scrub queued for path: / 2024-04-24T09:09:20.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:09:19 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:09:20.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:09:19 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:09:20.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:09:19 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:09:20.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:09:19 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:09:20.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:09:19 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:09:20.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:09:19 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:09:20.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:09:19 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:09:20.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:09:19 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:09:20.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:09:19 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:09:20.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:09:19 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:09:20.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:09:19 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:09:20.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:09:19 smithi189 ceph-mon[31791]: pgmap v558: 129 pgs: 129 active+clean; 562 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.4 MiB/s wr, 309 op/s 2024-04-24T09:09:20.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:09:19 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:09:20.468 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:09:19 smithi138 ceph-mon[31441]: scrub queued for path: / 2024-04-24T09:09:20.468 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:09:19 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:09:20.468 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:09:19 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:09:20.468 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:09:19 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:09:20.468 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:09:19 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:09:20.468 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:09:19 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:09:20.468 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:09:19 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:09:20.468 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:09:19 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:09:20.468 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:09:19 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:09:20.469 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:09:19 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:09:20.469 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:09:19 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:09:20.469 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:09:19 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:09:20.469 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:09:19 smithi138 ceph-mon[31441]: pgmap v558: 129 pgs: 129 active+clean; 562 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.4 MiB/s wr, 309 op/s 2024-04-24T09:09:20.469 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:09:19 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:09:21.171 INFO:tasks.workunit.client.0.smithi083.stdout: 10 29763 7.01 MB/sec execute 112 sec latency 86.519 ms 2024-04-24T09:09:22.170 INFO:tasks.workunit.client.0.smithi083.stdout: 10 29819 6.96 MB/sec execute 113 sec latency 453.312 ms 2024-04-24T09:09:22.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:09:21 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:09:22.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:09:21 smithi138 ceph-mon[31441]: pgmap v559: 129 pgs: 129 active+clean; 562 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.1 MiB/s wr, 190 op/s 2024-04-24T09:09:22.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:09:21 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:09:22.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:09:21 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:09:22.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:09:21 smithi189 ceph-mon[31791]: pgmap v559: 129 pgs: 129 active+clean; 562 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.1 MiB/s wr, 190 op/s 2024-04-24T09:09:22.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:09:21 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:09:22.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:09:21 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:09:22.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:09:21 smithi083 ceph-mon[27907]: pgmap v559: 129 pgs: 129 active+clean; 562 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.1 MiB/s wr, 190 op/s 2024-04-24T09:09:22.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:09:21 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:09:23.170 INFO:tasks.workunit.client.0.smithi083.stdout: 10 29873 6.92 MB/sec execute 114 sec latency 485.281 ms 2024-04-24T09:09:23.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:09:22 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:09:23.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:09:22 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:09:23.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:09:22 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:09:23.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:09:22 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:09:23.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:09:22 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:09:23.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:09:22 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:09:23.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:09:22 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:09:23.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:09:22 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:09:23.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:09:22 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:09:24.170 INFO:tasks.workunit.client.0.smithi083.stdout: 10 29971 6.88 MB/sec execute 115 sec latency 70.083 ms 2024-04-24T09:09:24.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:09:23 smithi189 ceph-mon[31791]: pgmap v560: 129 pgs: 129 active+clean; 566 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.6 MiB/s wr, 339 op/s 2024-04-24T09:09:24.213 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:09:23 smithi138 ceph-mon[31441]: pgmap v560: 129 pgs: 129 active+clean; 566 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.6 MiB/s wr, 339 op/s 2024-04-24T09:09:24.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:09:23 smithi083 ceph-mon[27907]: pgmap v560: 129 pgs: 129 active+clean; 566 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.6 MiB/s wr, 339 op/s 2024-04-24T09:09:25.170 INFO:tasks.workunit.client.0.smithi083.stdout: 10 30079 6.86 MB/sec execute 116 sec latency 101.278 ms 2024-04-24T09:09:26.170 INFO:tasks.workunit.client.0.smithi083.stdout: 10 30194 6.84 MB/sec execute 117 sec latency 83.845 ms 2024-04-24T09:09:26.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:09:25 smithi189 ceph-mon[31791]: pgmap v561: 129 pgs: 129 active+clean; 566 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.8 MiB/s wr, 233 op/s 2024-04-24T09:09:26.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:09:25 smithi189 ceph-mon[31791]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:09:26.218 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:09:25 smithi138 ceph-mon[31441]: pgmap v561: 129 pgs: 129 active+clean; 566 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.8 MiB/s wr, 233 op/s 2024-04-24T09:09:26.218 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:09:25 smithi138 ceph-mon[31441]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:09:26.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:09:25 smithi083 ceph-mon[27907]: pgmap v561: 129 pgs: 129 active+clean; 566 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.8 MiB/s wr, 233 op/s 2024-04-24T09:09:26.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:09:25 smithi083 ceph-mon[27907]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:09:27.170 INFO:tasks.workunit.client.0.smithi083.stdout: 10 30309 6.81 MB/sec execute 118 sec latency 100.421 ms 2024-04-24T09:09:28.170 INFO:tasks.workunit.client.0.smithi083.stdout: 10 30376 6.76 MB/sec execute 119 sec latency 394.784 ms 2024-04-24T09:09:28.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:09:27 smithi138 ceph-mon[31441]: pgmap v562: 129 pgs: 129 active+clean; 566 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.8 MiB/s wr, 233 op/s 2024-04-24T09:09:28.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:09:27 smithi189 ceph-mon[31791]: pgmap v562: 129 pgs: 129 active+clean; 566 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.8 MiB/s wr, 233 op/s 2024-04-24T09:09:28.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:09:27 smithi083 ceph-mon[27907]: pgmap v562: 129 pgs: 129 active+clean; 566 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.8 MiB/s wr, 233 op/s 2024-04-24T09:09:29.170 INFO:tasks.workunit.client.0.smithi083.stdout: 10 30509 6.78 MB/sec execute 120 sec latency 144.203 ms 2024-04-24T09:09:30.170 INFO:tasks.workunit.client.0.smithi083.stdout: 10 30629 6.75 MB/sec execute 121 sec latency 112.196 ms 2024-04-24T09:09:30.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:09:29 smithi138 ceph-mon[31441]: pgmap v563: 129 pgs: 129 active+clean; 566 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.3 MiB/s wr, 334 op/s 2024-04-24T09:09:30.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:09:29 smithi189 ceph-mon[31791]: pgmap v563: 129 pgs: 129 active+clean; 566 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.3 MiB/s wr, 334 op/s 2024-04-24T09:09:30.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:09:29 smithi083 ceph-mon[27907]: pgmap v563: 129 pgs: 129 active+clean; 566 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.3 MiB/s wr, 334 op/s 2024-04-24T09:09:31.170 INFO:tasks.workunit.client.0.smithi083.stdout: 10 30755 6.76 MB/sec execute 122 sec latency 75.487 ms 2024-04-24T09:09:31.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:09:30 smithi138 ceph-mon[31441]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:09:31.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:09:30 smithi189 ceph-mon[31791]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:09:31.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:09:30 smithi083 ceph-mon[27907]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:09:31.802 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.802+0000 7f8cd4200640 1 -- 172.21.15.83:0/3309267621 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ccc076530 msgr2=0x7f8ccc076910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:31.802 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.802+0000 7f8cd4200640 1 --2- 172.21.15.83:0/3309267621 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ccc076530 0x7f8ccc076910 secure :-1 s=READY pgs=2376 cs=0 l=1 rev1=1 crypto rx=0x7f8cc0009900 tx=0x7f8cc002a410 comp rx=0 tx=0).stop 2024-04-24T09:09:31.803 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.802+0000 7f8cd4200640 1 -- 172.21.15.83:0/3309267621 shutdown_connections 2024-04-24T09:09:31.803 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.802+0000 7f8cd4200640 1 --2- 172.21.15.83:0/3309267621 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ccc112e80 0x7f8ccc115270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:31.803 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.802+0000 7f8cd4200640 1 --2- 172.21.15.83:0/3309267621 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8ccc076e50 0x7f8ccc112940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:31.803 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.802+0000 7f8cd4200640 1 --2- 172.21.15.83:0/3309267621 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ccc076530 0x7f8ccc076910 unknown :-1 s=CLOSED pgs=2376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:31.803 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.802+0000 7f8cd4200640 1 -- 172.21.15.83:0/3309267621 >> 172.21.15.83:0/3309267621 conn(0x7f8ccc0ffcf0 msgr2=0x7f8ccc102110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:31.803 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.802+0000 7f8cd4200640 1 -- 172.21.15.83:0/3309267621 shutdown_connections 2024-04-24T09:09:31.804 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.802+0000 7f8cd4200640 1 -- 172.21.15.83:0/3309267621 wait complete. 2024-04-24T09:09:31.804 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.803+0000 7f8cd4200640 1 Processor -- start 2024-04-24T09:09:31.804 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.804+0000 7f8cd4200640 1 -- start start 2024-04-24T09:09:31.805 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.804+0000 7f8cd4200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ccc076530 0x7f8ccc19edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:31.805 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.804+0000 7f8cd4200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ccc076e50 0x7f8ccc19f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:31.805 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.804+0000 7f8cd4200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8ccc112e80 0x7f8ccc1a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:31.805 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.804+0000 7f8cd4200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8ccc1188e0 con 0x7f8ccc076e50 2024-04-24T09:09:31.805 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.804+0000 7f8cd4200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8ccc118760 con 0x7f8ccc076530 2024-04-24T09:09:31.805 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.804+0000 7f8cd4200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8ccc118a60 con 0x7f8ccc112e80 2024-04-24T09:09:31.805 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.804+0000 7f8cd1800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ccc076530 0x7f8ccc19edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:31.805 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.804+0000 7f8cd1800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ccc076530 0x7f8ccc19edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:50430/0 (socket says 172.21.15.83:50430) 2024-04-24T09:09:31.806 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.804+0000 7f8cd1800640 1 -- 172.21.15.83:0/1419951140 learned_addr learned my addr 172.21.15.83:0/1419951140 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:09:31.806 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.805+0000 7f8cd2200640 1 --2- 172.21.15.83:0/1419951140 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8ccc112e80 0x7f8ccc1a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:31.806 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.805+0000 7f8cd1800640 1 -- 172.21.15.83:0/1419951140 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8ccc112e80 msgr2=0x7f8ccc1a3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:31.806 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.805+0000 7f8cd1800640 1 --2- 172.21.15.83:0/1419951140 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8ccc112e80 0x7f8ccc1a3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:31.806 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.805+0000 7f8cd0e00640 1 --2- 172.21.15.83:0/1419951140 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ccc076e50 0x7f8ccc19f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:31.806 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.805+0000 7f8cd1800640 1 -- 172.21.15.83:0/1419951140 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ccc076e50 msgr2=0x7f8ccc19f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:31.806 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.805+0000 7f8cd1800640 1 --2- 172.21.15.83:0/1419951140 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ccc076e50 0x7f8ccc19f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:31.806 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.805+0000 7f8cd1800640 1 -- 172.21.15.83:0/1419951140 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ccc1a3db0 con 0x7f8ccc076530 2024-04-24T09:09:31.806 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.805+0000 7f8cd0e00640 1 --2- 172.21.15.83:0/1419951140 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ccc076e50 0x7f8ccc19f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:09:31.807 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.805+0000 7f8cd1800640 1 --2- 172.21.15.83:0/1419951140 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ccc076530 0x7f8ccc19edc0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f8cc0004770 tx=0x7f8cc0003ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:31.807 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.806+0000 7f8cca000640 1 -- 172.21.15.83:0/1419951140 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc003f070 con 0x7f8ccc076530 2024-04-24T09:09:31.807 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.806+0000 7f8cd4200640 1 -- 172.21.15.83:0/1419951140 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ccc1a4040 con 0x7f8ccc076530 2024-04-24T09:09:31.807 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.806+0000 7f8cca000640 1 -- 172.21.15.83:0/1419951140 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8cc003bd20 con 0x7f8ccc076530 2024-04-24T09:09:31.807 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.806+0000 7f8cca000640 1 -- 172.21.15.83:0/1419951140 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc0035030 con 0x7f8ccc076530 2024-04-24T09:09:31.807 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.806+0000 7f8cd4200640 1 -- 172.21.15.83:0/1419951140 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ccc1ab930 con 0x7f8ccc076530 2024-04-24T09:09:31.811 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.810+0000 7f8cca000640 1 -- 172.21.15.83:0/1419951140 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8cc0042070 con 0x7f8ccc076530 2024-04-24T09:09:31.811 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.810+0000 7f8cd71ac740 1 Processor -- start 2024-04-24T09:09:31.812 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.811+0000 7f8cca000640 1 --2- 172.21.15.83:0/1419951140 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f8cb40a3e40 0x7f8cb40a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:31.812 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.812+0000 7f8cca000640 1 -- 172.21.15.83:0/1419951140 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 9583+0+0 (secure 0 0 0) 0x7f8cc00f1470 con 0x7f8ccc076530 2024-04-24T09:09:31.812 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.812+0000 7f8cd71ac740 1 -- start start 2024-04-24T09:09:31.812 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.812+0000 7f8cd2200640 1 --2- 172.21.15.83:0/1419951140 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f8cb40a3e40 0x7f8cb40a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:31.812 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.812+0000 7f8cd71ac740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56150f4051b0 0x56150f405590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:31.813 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.812+0000 7f8cd71ac740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56150f405ad0 0x56150f40fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:31.813 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.812+0000 7f8cd71ac740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56150f410390 0x56150f412780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:31.813 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.812+0000 7f8cd71ac740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56150f36b900 con 0x56150f4051b0 2024-04-24T09:09:31.813 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.812+0000 7f8cd71ac740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56150f311f20 con 0x56150f410390 2024-04-24T09:09:31.813 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.812+0000 7f8cd71ac740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56150f3678c0 con 0x56150f405ad0 2024-04-24T09:09:31.813 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.813+0000 7f8cd0e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56150f405ad0 0x56150f40fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:31.814 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.813+0000 7f8cd2200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56150f410390 0x56150f412780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:31.814 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.813+0000 7f8cd0e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56150f405ad0 0x56150f40fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:56392/0 (socket says 172.21.15.83:56392) 2024-04-24T09:09:31.814 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.813+0000 7f8cd0e00640 1 -- 172.21.15.83:0/3098196970 learned_addr learned my addr 172.21.15.83:0/3098196970 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:09:31.814 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.813+0000 7f8cd1800640 1 --2- 172.21.15.83:0/3098196970 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56150f4051b0 0x56150f405590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:31.814 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.814+0000 7f8cd1800640 1 -- 172.21.15.83:0/3098196970 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56150f405ad0 msgr2=0x56150f40fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:31.814 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.814+0000 7f8cd1800640 1 --2- 172.21.15.83:0/3098196970 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56150f405ad0 0x56150f40fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:31.815 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.814+0000 7f8cd1800640 1 -- 172.21.15.83:0/3098196970 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56150f410390 msgr2=0x56150f412780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:31.815 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.814+0000 7f8cd1800640 1 --2- 172.21.15.83:0/3098196970 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56150f410390 0x56150f412780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:31.815 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.814+0000 7f8cd1800640 1 -- 172.21.15.83:0/3098196970 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56150f336e10 con 0x56150f4051b0 2024-04-24T09:09:31.815 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.814+0000 7f8cd2200640 1 --2- 172.21.15.83:0/3098196970 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56150f410390 0x56150f412780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:09:31.816 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.815+0000 7f8cd2200640 1 --2- 172.21.15.83:0/1419951140 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f8cb40a3e40 0x7f8cb40a6300 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f8cc40092f0 tx=0x7f8cc40147e0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:31.816 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.815+0000 7f8cd1800640 1 --2- 172.21.15.83:0/3098196970 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56150f4051b0 0x56150f405590 secure :-1 s=READY pgs=2377 cs=0 l=1 rev1=1 crypto rx=0x7f8cc00ee740 tx=0x7f8cc00bd690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fc067408275328c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:31.816 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.815+0000 7f8cbea00640 1 -- 172.21.15.83:0/3098196970 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc00d6070 con 0x56150f4051b0 2024-04-24T09:09:31.816 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.815+0000 7f8cbea00640 1 -- 172.21.15.83:0/3098196970 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8cc00c5c00 con 0x56150f4051b0 2024-04-24T09:09:31.816 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.815+0000 7f8cbea00640 1 -- 172.21.15.83:0/3098196970 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc0049040 con 0x56150f4051b0 2024-04-24T09:09:31.817 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.817+0000 7f8cd71ac740 1 -- 172.21.15.83:0/3098196970 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56150f4051b0 msgr2=0x56150f405590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:31.817 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.817+0000 7f8cd71ac740 1 --2- 172.21.15.83:0/3098196970 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56150f4051b0 0x56150f405590 secure :-1 s=READY pgs=2377 cs=0 l=1 rev1=1 crypto rx=0x7f8cc00ee740 tx=0x7f8cc00bd690 comp rx=0 tx=0).stop 2024-04-24T09:09:31.817 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.817+0000 7f8cd71ac740 1 -- 172.21.15.83:0/3098196970 shutdown_connections 2024-04-24T09:09:31.817 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.817+0000 7f8cd71ac740 1 --2- 172.21.15.83:0/3098196970 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56150f410390 0x56150f412780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:31.817 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.817+0000 7f8cd71ac740 1 --2- 172.21.15.83:0/3098196970 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56150f405ad0 0x56150f40fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:31.817 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.817+0000 7f8cd71ac740 1 --2- 172.21.15.83:0/3098196970 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56150f4051b0 0x56150f405590 unknown :-1 s=CLOSED pgs=2377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:31.818 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.817+0000 7f8cd71ac740 1 -- 172.21.15.83:0/3098196970 >> 172.21.15.83:0/3098196970 conn(0x56150f4016c0 msgr2=0x56150f401aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:31.818 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.817+0000 7f8cd71ac740 1 -- 172.21.15.83:0/3098196970 shutdown_connections 2024-04-24T09:09:31.818 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.817+0000 7f8cd71ac740 1 -- 172.21.15.83:0/3098196970 wait complete. 2024-04-24T09:09:31.818 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.818+0000 7f8cd71ac740 1 -- start start 2024-04-24T09:09:31.818 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.818+0000 7f8cd71ac740 1 Processor -- start 2024-04-24T09:09:31.819 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.818+0000 7f8cd71ac740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56150f4051b0 0x56150f40d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:31.820 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.818+0000 7f8cd71ac740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56150f405ad0 0x56150f40d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:31.820 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.818+0000 7f8cd71ac740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56150f410390 0x56150f40dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:31.820 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.819+0000 7f8cd71ac740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56150f412d80 con 0x56150f4051b0 2024-04-24T09:09:31.820 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.819+0000 7f8cd71ac740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56150f3252c0 con 0x56150f405ad0 2024-04-24T09:09:31.820 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.819+0000 7f8cd71ac740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56150f412f00 con 0x56150f410390 2024-04-24T09:09:31.820 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.819+0000 7f8cd71ac740 10 client.0 _ll_register_callbacks cb 0x56150f07da00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:09:31.820 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.819+0000 7f8cd1800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56150f4051b0 0x56150f40d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:31.820 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.819+0000 7f8cd1800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56150f4051b0 0x56150f40d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52848/0 (socket says 172.21.15.83:52848) 2024-04-24T09:09:31.821 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.819+0000 7f8cd1800640 1 -- 172.21.15.83:0/1493968118 learned_addr learned my addr 172.21.15.83:0/1493968118 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:09:31.821 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.819+0000 7f8cd0e00640 1 --2- 172.21.15.83:0/1493968118 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56150f405ad0 0x56150f40d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:31.821 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.819+0000 7f8cd1800640 1 -- 172.21.15.83:0/1493968118 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56150f410390 msgr2=0x56150f40dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:31.821 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.819+0000 7f8cd1800640 1 --2- 172.21.15.83:0/1493968118 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56150f410390 0x56150f40dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:31.821 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.819+0000 7f8cd1800640 1 -- 172.21.15.83:0/1493968118 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56150f405ad0 msgr2=0x56150f40d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:31.821 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.819+0000 7f8cd1800640 1 --2- 172.21.15.83:0/1493968118 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56150f405ad0 0x56150f40d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:31.821 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.819+0000 7f8cd1800640 1 -- 172.21.15.83:0/1493968118 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56150f335fd0 con 0x56150f4051b0 2024-04-24T09:09:31.821 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.820+0000 7f8cd0e00640 1 --2- 172.21.15.83:0/1493968118 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56150f405ad0 0x56150f40d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:09:31.821 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.820+0000 7f8cd1800640 1 --2- 172.21.15.83:0/1493968118 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56150f4051b0 0x56150f40d300 secure :-1 s=READY pgs=2378 cs=0 l=1 rev1=1 crypto rx=0x7f8cc00c56d0 tx=0x7f8cc0003e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a944a8da9043597a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:31.821 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.820+0000 7f8cb3400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:09:31.821 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.820+0000 7f8cb3400640 1 -- 172.21.15.83:0/1493968118 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc00da020 con 0x56150f4051b0 2024-04-24T09:09:31.821 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.820+0000 7f8cb3400640 1 -- 172.21.15.83:0/1493968118 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9583+0+0 (secure 0 0 0) 0x7f8cc00bd1f0 con 0x56150f4051b0 2024-04-24T09:09:31.822 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.820+0000 7f8cd71ac740 1 -- 172.21.15.83:0/1493968118 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8cc00c5c00 con 0x56150f4051b0 2024-04-24T09:09:31.822 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.821+0000 7f8cb3400640 1 -- 172.21.15.83:0/1493968118 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f8cc0049070 con 0x56150f4051b0 2024-04-24T09:09:31.822 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.821+0000 7f8cd71ac740 10 client.21834 fetch_fsmap learned FSMap version 60 2024-04-24T09:09:31.822 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.821+0000 7f8cd71ac740 1 -- 172.21.15.83:0/1493968118 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x56150f07e210 con 0x56150f4051b0 2024-04-24T09:09:31.822 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.821+0000 7f8cb3400640 1 -- 172.21.15.83:0/1493968118 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8cc00c5c00 con 0x56150f4051b0 2024-04-24T09:09:31.822 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.821+0000 7f8cb3400640 1 -- 172.21.15.83:0/1493968118 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 60) ==== 8366+0+0 (secure 0 0 0) 0x7f8cc0031430 con 0x56150f4051b0 2024-04-24T09:09:31.822 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.821+0000 7f8cd71ac740 10 client.21834 fetch_fsmap finished waiting for FSMap version 60 2024-04-24T09:09:31.822 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.821+0000 7f8cd71ac740 10 client.21834 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:09:31.823 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.821+0000 7f8cd71ac740 20 client.21834 populate_metadata read hostname 'smithi083' 2024-04-24T09:09:31.823 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.821+0000 7f8cd71ac740 1 --2- 172.21.15.83:0/1493968118 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56150f420070 0x56150f422460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:31.823 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.821+0000 7f8cd71ac740 4 client.21834 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:09:31.823 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.821+0000 7f8cd71ac740 1 -- 172.21.15.83:0/1493968118 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56150f379320 con 0x56150f420070 2024-04-24T09:09:31.823 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.822+0000 7f8cd0e00640 1 --2- 172.21.15.83:0/1493968118 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56150f420070 0x56150f422460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:31.823 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.823+0000 7f8cd0e00640 1 --2- 172.21.15.83:0/1493968118 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56150f420070 0x56150f422460 crc :-1 s=READY pgs=451 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1046148a90f9f58 server_cookie=29d9607f55378fef in_seq=0 out_seq=0 2024-04-24T09:09:31.823 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.823+0000 7f8cb3400640 10 client.21834 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:09:31.824 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.824+0000 7f8cb3400640 1 -- 172.21.15.83:0/1493968118 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56150f379320 con 0x56150f420070 2024-04-24T09:09:31.824 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.824+0000 7f8cb3400640 10 client.21834 handle_command_reply: tid=0 2024-04-24T09:09:31.824 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.824+0000 7f8cd71ac740 1 client.21834 shutdown 2024-04-24T09:09:31.825 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.824+0000 7f8cd71ac740 1 -- 172.21.15.83:0/1493968118 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56150f4051b0 msgr2=0x56150f40d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:31.825 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.824+0000 7f8cd71ac740 1 --2- 172.21.15.83:0/1493968118 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56150f4051b0 0x56150f40d300 secure :-1 s=READY pgs=2378 cs=0 l=1 rev1=1 crypto rx=0x7f8cc00c56d0 tx=0x7f8cc0003e90 comp rx=0 tx=0).stop 2024-04-24T09:09:31.825 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.825+0000 7f8cd71ac740 1 -- 172.21.15.83:0/1493968118 shutdown_connections 2024-04-24T09:09:31.825 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.825+0000 7f8cd71ac740 1 --2- 172.21.15.83:0/1493968118 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56150f420070 0x56150f422460 crc :-1 s=READY pgs=451 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:31.825 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.825+0000 7f8cd71ac740 1 --2- 172.21.15.83:0/1493968118 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56150f410390 0x56150f40dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:31.825 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.825+0000 7f8cd71ac740 1 --2- 172.21.15.83:0/1493968118 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56150f405ad0 0x56150f40d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:31.826 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.825+0000 7f8cd71ac740 1 --2- 172.21.15.83:0/1493968118 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56150f4051b0 0x56150f40d300 unknown :-1 s=CLOSED pgs=2378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:31.826 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.825+0000 7f8cd71ac740 1 -- 172.21.15.83:0/1493968118 >> 172.21.15.83:0/1493968118 conn(0x56150f4016c0 msgr2=0x56150f4073f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:31.826 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.825+0000 7f8cb3400640 0 client.21834 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:09:31.826 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.825+0000 7f8cd71ac740 1 -- 172.21.15.83:0/1493968118 shutdown_connections 2024-04-24T09:09:31.826 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.825+0000 7f8cd71ac740 1 -- 172.21.15.83:0/1493968118 wait complete. 2024-04-24T09:09:31.826 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.825+0000 7f8cd71ac740 20 client.21834 trim_cache size 0 max 16384 2024-04-24T09:09:31.860 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.860+0000 7f8cd71ac740 1 Processor -- start 2024-04-24T09:09:31.861 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.861+0000 7f8cd71ac740 1 -- start start 2024-04-24T09:09:31.861 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.861+0000 7f8cd71ac740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56150f4051b0 0x56150f41ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:31.861 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.861+0000 7f8cd71ac740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56150f405ad0 0x56150f40f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:31.861 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.861+0000 7f8cd71ac740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56150f410390 0x56150f427a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:31.861 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.861+0000 7f8cd71ac740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8ccc1b5ba0 con 0x56150f405ad0 2024-04-24T09:09:31.861 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.861+0000 7f8cd71ac740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8ccc1b76e0 con 0x56150f4051b0 2024-04-24T09:09:31.862 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.861+0000 7f8cd71ac740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8ccc1b6270 con 0x56150f410390 2024-04-24T09:09:31.862 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.862+0000 7f8cd1800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56150f4051b0 0x56150f41ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:31.862 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.862+0000 7f8cd1800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56150f4051b0 0x56150f41ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:50468/0 (socket says 172.21.15.83:50468) 2024-04-24T09:09:31.862 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.862+0000 7f8cd1800640 1 -- 172.21.15.83:0/4045175124 learned_addr learned my addr 172.21.15.83:0/4045175124 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:09:31.862 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.862+0000 7f8cd0e00640 1 --2- 172.21.15.83:0/4045175124 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56150f405ad0 0x56150f40f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:31.863 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.863+0000 7f8cd1800640 1 -- 172.21.15.83:0/4045175124 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56150f410390 msgr2=0x56150f427a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:31.863 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.863+0000 7f8cd1800640 1 --2- 172.21.15.83:0/4045175124 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56150f410390 0x56150f427a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:31.863 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.863+0000 7f8cd1800640 1 -- 172.21.15.83:0/4045175124 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56150f405ad0 msgr2=0x56150f40f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:31.863 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.863+0000 7f8cd1800640 1 --2- 172.21.15.83:0/4045175124 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56150f405ad0 0x56150f40f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:31.863 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.863+0000 7f8cd1800640 1 -- 172.21.15.83:0/4045175124 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56150f14a620 con 0x56150f4051b0 2024-04-24T09:09:31.864 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.864+0000 7f8cd1800640 1 --2- 172.21.15.83:0/4045175124 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56150f4051b0 0x56150f41ba40 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f8cc00b9960 tx=0x7f8cc00bb4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=65b87c4b4eefff43 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:31.865 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.865+0000 7f8cbe000640 1 -- 172.21.15.83:0/4045175124 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc00da070 con 0x56150f4051b0 2024-04-24T09:09:31.866 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.865+0000 7f8cbe000640 1 -- 172.21.15.83:0/4045175124 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8cc00c5e50 con 0x56150f4051b0 2024-04-24T09:09:31.866 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.865+0000 7f8cbe000640 1 -- 172.21.15.83:0/4045175124 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc00d6040 con 0x56150f4051b0 2024-04-24T09:09:31.866 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.866+0000 7f8cd71ac740 1 -- 172.21.15.83:0/4045175124 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56150f4051b0 msgr2=0x56150f41ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:31.866 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.866+0000 7f8cd71ac740 1 --2- 172.21.15.83:0/4045175124 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56150f4051b0 0x56150f41ba40 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f8cc00b9960 tx=0x7f8cc00bb4a0 comp rx=0 tx=0).stop 2024-04-24T09:09:31.867 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.867+0000 7f8cd71ac740 1 -- 172.21.15.83:0/4045175124 shutdown_connections 2024-04-24T09:09:31.867 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.867+0000 7f8cd71ac740 1 --2- 172.21.15.83:0/4045175124 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56150f410390 0x56150f427a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:31.867 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.867+0000 7f8cd71ac740 1 --2- 172.21.15.83:0/4045175124 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56150f405ad0 0x56150f40f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:31.867 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.867+0000 7f8cd71ac740 1 --2- 172.21.15.83:0/4045175124 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56150f4051b0 0x56150f41ba40 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:31.868 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.867+0000 7f8cd71ac740 1 -- 172.21.15.83:0/4045175124 >> 172.21.15.83:0/4045175124 conn(0x56150f4016c0 msgr2=0x56150f40eb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:31.868 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.868+0000 7f8cd71ac740 1 -- 172.21.15.83:0/4045175124 shutdown_connections 2024-04-24T09:09:31.869 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.869+0000 7f8cd71ac740 1 -- 172.21.15.83:0/4045175124 wait complete. 2024-04-24T09:09:31.870 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.870+0000 7f8cd71ac740 1 -- start start 2024-04-24T09:09:31.870 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.870+0000 7f8cd71ac740 1 Processor -- start 2024-04-24T09:09:31.872 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.871+0000 7f8cd71ac740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56150f4051b0 0x56150f481b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:31.872 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.871+0000 7f8cd71ac740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56150f405ad0 0x56150f417f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:31.872 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.871+0000 7f8cd71ac740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56150f410390 0x56150f418490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:31.872 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.871+0000 7f8cd71ac740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8ccc1b7010 con 0x56150f4051b0 2024-04-24T09:09:31.872 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.871+0000 7f8cd71ac740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8ccc1b6940 con 0x56150f410390 2024-04-24T09:09:31.872 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.871+0000 7f8cd71ac740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8ccc1b3ba0 con 0x56150f405ad0 2024-04-24T09:09:31.872 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.871+0000 7f8cd71ac740 10 client.0 _ll_register_callbacks cb 0x7f8cc00d4070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:09:31.873 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.872+0000 7f8cd0e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56150f405ad0 0x56150f417f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:31.873 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.872+0000 7f8cd0e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56150f405ad0 0x56150f417f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:56410/0 (socket says 172.21.15.83:56410) 2024-04-24T09:09:31.873 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.872+0000 7f8cd2200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56150f410390 0x56150f418490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:31.873 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.872+0000 7f8cd0e00640 1 -- 172.21.15.83:0/3995416769 learned_addr learned my addr 172.21.15.83:0/3995416769 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:09:31.873 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.872+0000 7f8cd1800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56150f4051b0 0x56150f481b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:31.874 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.873+0000 7f8cd1800640 1 -- 172.21.15.83:0/3995416769 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56150f405ad0 msgr2=0x56150f417f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:31.874 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.873+0000 7f8cd1800640 1 --2- 172.21.15.83:0/3995416769 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56150f405ad0 0x56150f417f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:31.874 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.873+0000 7f8cd1800640 1 -- 172.21.15.83:0/3995416769 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56150f410390 msgr2=0x56150f418490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:31.874 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.873+0000 7f8cd1800640 1 --2- 172.21.15.83:0/3995416769 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56150f410390 0x56150f418490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:31.874 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.873+0000 7f8cd1800640 1 -- 172.21.15.83:0/3995416769 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56150f40a440 con 0x56150f4051b0 2024-04-24T09:09:31.875 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.873+0000 7f8cd0e00640 1 --2- 172.21.15.83:0/3995416769 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56150f405ad0 0x56150f417f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:09:31.875 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.874+0000 7f8cd1800640 1 --2- 172.21.15.83:0/3995416769 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56150f4051b0 0x56150f481b20 secure :-1 s=READY pgs=2379 cs=0 l=1 rev1=1 crypto rx=0x7f8cc0005c20 tx=0x7f8cc00bd690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c11e93ba78938eb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:31.875 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.874+0000 7f8cb3400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:09:31.875 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.874+0000 7f8cb3400640 1 -- 172.21.15.83:0/3995416769 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc00da050 con 0x56150f4051b0 2024-04-24T09:09:31.875 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.875+0000 7f8cb3400640 1 -- 172.21.15.83:0/3995416769 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9583+0+0 (secure 0 0 0) 0x7f8cc00d61e0 con 0x56150f4051b0 2024-04-24T09:09:31.875 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.875+0000 7f8cd71ac740 1 -- 172.21.15.83:0/3995416769 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8cc00c5e50 con 0x56150f4051b0 2024-04-24T09:09:31.876 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.875+0000 7f8cb3400640 1 -- 172.21.15.83:0/3995416769 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f8cc00d6040 con 0x56150f4051b0 2024-04-24T09:09:31.876 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.875+0000 7f8cd71ac740 10 client.21840 fetch_fsmap learned FSMap version 60 2024-04-24T09:09:31.876 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.875+0000 7f8cd71ac740 1 -- 172.21.15.83:0/3995416769 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x56150f40a440 con 0x56150f4051b0 2024-04-24T09:09:31.876 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.876+0000 7f8cb3400640 1 -- 172.21.15.83:0/3995416769 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8cc00ac030 con 0x56150f4051b0 2024-04-24T09:09:31.877 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.877+0000 7f8cb3400640 1 -- 172.21.15.83:0/3995416769 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 60) ==== 8366+0+0 (secure 0 0 0) 0x7f8cc00bc3e0 con 0x56150f4051b0 2024-04-24T09:09:31.877 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.877+0000 7f8cd71ac740 10 client.21840 fetch_fsmap finished waiting for FSMap version 60 2024-04-24T09:09:31.877 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.877+0000 7f8cd71ac740 10 client.21840 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:09:31.877 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.877+0000 7f8cd71ac740 20 client.21840 populate_metadata read hostname 'smithi083' 2024-04-24T09:09:31.877 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.877+0000 7f8cd71ac740 1 --2- 172.21.15.83:0/3995416769 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56150f497a00 0x56150f499e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:31.877 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.877+0000 7f8cd71ac740 4 client.21840 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T09:09:31.877 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.877+0000 7f8cd71ac740 1 -- 172.21.15.83:0/3995416769 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56150f373190 con 0x56150f497a00 2024-04-24T09:09:31.878 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.877+0000 7f8cd0e00640 1 --2- 172.21.15.83:0/3995416769 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56150f497a00 0x56150f499e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:31.878 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.878+0000 7f8cd0e00640 1 --2- 172.21.15.83:0/3995416769 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56150f497a00 0x56150f499e20 crc :-1 s=READY pgs=452 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f58f3e9fc877c845 server_cookie=6137e22832e38387 in_seq=0 out_seq=0 2024-04-24T09:09:31.878 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.878+0000 7f8cb3400640 10 client.21840 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:09:31.994 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.994+0000 7f8cb3400640 1 -- 172.21.15.83:0/3995416769 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56150f373190 con 0x56150f497a00 2024-04-24T09:09:31.995 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.994+0000 7f8cb3400640 10 client.21840 handle_command_reply: tid=0 2024-04-24T09:09:31.995 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.994+0000 7f8cd71ac740 1 client.21840 shutdown 2024-04-24T09:09:31.996 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.995+0000 7f8cd71ac740 1 -- 172.21.15.83:0/3995416769 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56150f4051b0 msgr2=0x56150f481b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:31.996 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.995+0000 7f8cd71ac740 1 --2- 172.21.15.83:0/3995416769 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56150f4051b0 0x56150f481b20 secure :-1 s=READY pgs=2379 cs=0 l=1 rev1=1 crypto rx=0x7f8cc0005c20 tx=0x7f8cc00bd690 comp rx=0 tx=0).stop 2024-04-24T09:09:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.996+0000 7f8cd71ac740 1 -- 172.21.15.83:0/3995416769 shutdown_connections 2024-04-24T09:09:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.996+0000 7f8cd71ac740 1 --2- 172.21.15.83:0/3995416769 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56150f497a00 0x56150f499e20 crc :-1 s=READY pgs=452 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.996+0000 7f8cd71ac740 1 --2- 172.21.15.83:0/3995416769 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56150f410390 0x56150f418490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.996+0000 7f8cd71ac740 1 --2- 172.21.15.83:0/3995416769 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56150f405ad0 0x56150f417f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.996+0000 7f8cd71ac740 1 --2- 172.21.15.83:0/3995416769 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56150f4051b0 0x56150f481b20 unknown :-1 s=CLOSED pgs=2379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.996+0000 7f8cd71ac740 1 -- 172.21.15.83:0/3995416769 >> 172.21.15.83:0/3995416769 conn(0x56150f4016c0 msgr2=0x56150f40db50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.996+0000 7f8cb3400640 0 client.21840 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:09:31.998 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.997+0000 7f8cd71ac740 1 -- 172.21.15.83:0/3995416769 shutdown_connections 2024-04-24T09:09:31.998 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.997+0000 7f8cd71ac740 1 -- 172.21.15.83:0/3995416769 wait complete. 2024-04-24T09:09:31.998 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.997+0000 7f8cd71ac740 20 client.21840 trim_cache size 0 max 16384 2024-04-24T09:09:31.998 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T09:09:31.998 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T09:09:31.998 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T09:09:31.998 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T09:09:31.999 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.999+0000 7f8cd4200640 1 -- 172.21.15.83:0/1419951140 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f8cb40a3e40 msgr2=0x7f8cb40a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:31.999 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.999+0000 7f8cd4200640 1 --2- 172.21.15.83:0/1419951140 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f8cb40a3e40 0x7f8cb40a6300 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f8cc40092f0 tx=0x7f8cc40147e0 comp rx=0 tx=0).stop 2024-04-24T09:09:31.999 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.999+0000 7f8cd4200640 1 -- 172.21.15.83:0/1419951140 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ccc076530 msgr2=0x7f8ccc19edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:31.999 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.999+0000 7f8cd4200640 1 --2- 172.21.15.83:0/1419951140 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ccc076530 0x7f8ccc19edc0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f8cc0004770 tx=0x7f8cc0003ca0 comp rx=0 tx=0).stop 2024-04-24T09:09:32.000 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.999+0000 7f8cd4200640 1 -- 172.21.15.83:0/1419951140 shutdown_connections 2024-04-24T09:09:32.000 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.999+0000 7f8cd4200640 1 --2- 172.21.15.83:0/1419951140 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f8cb40a3e40 0x7f8cb40a6300 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:32.000 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.999+0000 7f8cd4200640 1 --2- 172.21.15.83:0/1419951140 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8ccc112e80 0x7f8ccc1a3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:32.000 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.999+0000 7f8cd4200640 1 --2- 172.21.15.83:0/1419951140 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ccc076e50 0x7f8ccc19f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:32.000 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.999+0000 7f8cd4200640 1 --2- 172.21.15.83:0/1419951140 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ccc076530 0x7f8ccc19edc0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:32.000 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:31.999+0000 7f8cd4200640 1 -- 172.21.15.83:0/1419951140 >> 172.21.15.83:0/1419951140 conn(0x7f8ccc0ffcf0 msgr2=0x7f8ccc113af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:32.000 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:32.000+0000 7f8cd4200640 1 -- 172.21.15.83:0/1419951140 shutdown_connections 2024-04-24T09:09:32.000 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:32.000+0000 7f8cd4200640 1 -- 172.21.15.83:0/1419951140 wait complete. 2024-04-24T09:09:32.170 INFO:tasks.workunit.client.0.smithi083.stdout: 10 30875 6.73 MB/sec execute 123 sec latency 110.676 ms 2024-04-24T09:09:32.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:09:31 smithi138 ceph-mon[31441]: pgmap v564: 129 pgs: 129 active+clean; 566 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.0 MiB/s wr, 249 op/s 2024-04-24T09:09:32.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:09:31 smithi189 ceph-mon[31791]: pgmap v564: 129 pgs: 129 active+clean; 566 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.0 MiB/s wr, 249 op/s 2024-04-24T09:09:32.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:09:31 smithi083 ceph-mon[27907]: pgmap v564: 129 pgs: 129 active+clean; 566 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.0 MiB/s wr, 249 op/s 2024-04-24T09:09:33.170 INFO:tasks.workunit.client.0.smithi083.stdout: 10 31014 6.74 MB/sec execute 124 sec latency 83.584 ms 2024-04-24T09:09:33.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:09:32 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:09:33.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:09:32 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:09:33.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:09:32 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:09:33.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:09:32 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:09:33.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:09:32 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:09:33.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:09:32 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:09:33.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:09:32 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:09:33.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:09:32 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:09:33.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:09:32 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:09:33.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:09:32 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:09:33.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:09:32 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:09:33.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:09:32 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:09:34.171 INFO:tasks.workunit.client.0.smithi083.stdout: 10 31104 6.74 MB/sec execute 125 sec latency 447.442 ms 2024-04-24T09:09:34.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:09:33 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:09:34.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:09:34 smithi138 ceph-mon[31441]: pgmap v565: 129 pgs: 129 active+clean; 558 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.4 MiB/s wr, 393 op/s 2024-04-24T09:09:34.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:09:33 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:09:34.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:09:33 smithi189 ceph-mon[31791]: pgmap v565: 129 pgs: 129 active+clean; 558 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.4 MiB/s wr, 393 op/s 2024-04-24T09:09:34.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:09:33 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:09:34.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:09:33 smithi083 ceph-mon[27907]: pgmap v565: 129 pgs: 129 active+clean; 558 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.4 MiB/s wr, 393 op/s 2024-04-24T09:09:35.171 INFO:tasks.workunit.client.0.smithi083.stdout: 10 31220 6.70 MB/sec execute 126 sec latency 490.982 ms 2024-04-24T09:09:36.171 INFO:tasks.workunit.client.0.smithi083.stdout: 10 31356 6.70 MB/sec execute 127 sec latency 95.060 ms 2024-04-24T09:09:36.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:09:35 smithi083 ceph-mon[27907]: pgmap v566: 129 pgs: 129 active+clean; 558 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.9 MiB/s wr, 244 op/s 2024-04-24T09:09:36.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:09:35 smithi083 ceph-mon[27907]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:09:36.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:09:35 smithi138 ceph-mon[31441]: pgmap v566: 129 pgs: 129 active+clean; 558 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.9 MiB/s wr, 244 op/s 2024-04-24T09:09:36.474 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:09:35 smithi138 ceph-mon[31441]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:09:36.475 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:09:35 smithi189 ceph-mon[31791]: pgmap v566: 129 pgs: 129 active+clean; 558 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.9 MiB/s wr, 244 op/s 2024-04-24T09:09:36.475 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:09:35 smithi189 ceph-mon[31791]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:09:37.171 INFO:tasks.workunit.client.0.smithi083.stdout: 10 31493 6.67 MB/sec execute 128 sec latency 73.307 ms 2024-04-24T09:09:38.171 INFO:tasks.workunit.client.0.smithi083.stdout: 10 31647 6.67 MB/sec execute 129 sec latency 75.949 ms 2024-04-24T09:09:38.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:09:37 smithi083 ceph-mon[27907]: pgmap v567: 129 pgs: 129 active+clean; 558 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.9 MiB/s wr, 244 op/s 2024-04-24T09:09:38.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:09:37 smithi138 ceph-mon[31441]: pgmap v567: 129 pgs: 129 active+clean; 558 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.9 MiB/s wr, 244 op/s 2024-04-24T09:09:38.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:09:37 smithi189 ceph-mon[31791]: pgmap v567: 129 pgs: 129 active+clean; 558 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.9 MiB/s wr, 244 op/s 2024-04-24T09:09:39.171 INFO:tasks.workunit.client.0.smithi083.stdout: 10 31785 6.69 MB/sec execute 130 sec latency 77.710 ms 2024-04-24T09:09:40.172 INFO:tasks.workunit.client.0.smithi083.stdout: 10 31924 6.69 MB/sec execute 131 sec latency 74.866 ms 2024-04-24T09:09:40.256 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:09:39 smithi138 ceph-mon[31441]: pgmap v568: 129 pgs: 129 active+clean; 564 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.0 MiB/s wr, 344 op/s 2024-04-24T09:09:40.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:09:39 smithi083 ceph-mon[27907]: pgmap v568: 129 pgs: 129 active+clean; 564 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.0 MiB/s wr, 344 op/s 2024-04-24T09:09:40.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:09:39 smithi189 ceph-mon[31791]: pgmap v568: 129 pgs: 129 active+clean; 564 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.0 MiB/s wr, 344 op/s 2024-04-24T09:09:41.172 INFO:tasks.workunit.client.0.smithi083.stdout: 10 32113 6.76 MB/sec execute 132 sec latency 81.450 ms 2024-04-24T09:09:42.172 INFO:tasks.workunit.client.0.smithi083.stdout: 10 32265 6.78 MB/sec execute 133 sec latency 72.940 ms 2024-04-24T09:09:42.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:09:41 smithi083 ceph-mon[27907]: pgmap v569: 129 pgs: 129 active+clean; 564 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.5 MiB/s wr, 242 op/s 2024-04-24T09:09:42.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:09:41 smithi138 ceph-mon[31441]: pgmap v569: 129 pgs: 129 active+clean; 564 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.5 MiB/s wr, 242 op/s 2024-04-24T09:09:42.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:09:41 smithi189 ceph-mon[31791]: pgmap v569: 129 pgs: 129 active+clean; 564 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.5 MiB/s wr, 242 op/s 2024-04-24T09:09:43.172 INFO:tasks.workunit.client.0.smithi083.stdout: 10 32389 6.80 MB/sec execute 134 sec latency 76.080 ms 2024-04-24T09:09:44.172 INFO:tasks.workunit.client.0.smithi083.stdout: 10 32524 6.83 MB/sec execute 135 sec latency 64.565 ms 2024-04-24T09:09:44.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:09:43 smithi083 ceph-mon[27907]: pgmap v570: 129 pgs: 129 active+clean; 579 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.7 MiB/s wr, 331 op/s 2024-04-24T09:09:44.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:09:43 smithi138 ceph-mon[31441]: pgmap v570: 129 pgs: 129 active+clean; 579 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.7 MiB/s wr, 331 op/s 2024-04-24T09:09:44.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:09:43 smithi189 ceph-mon[31791]: pgmap v570: 129 pgs: 129 active+clean; 579 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.7 MiB/s wr, 331 op/s 2024-04-24T09:09:45.172 INFO:tasks.workunit.client.0.smithi083.stdout: 10 32641 6.82 MB/sec execute 136 sec latency 335.333 ms 2024-04-24T09:09:45.180 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.179+0000 7f8a8be00640 1 -- 172.21.15.83:0/1072352534 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a8c1038e0 msgr2=0x7f8a8c10bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:45.180 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.179+0000 7f8a8be00640 1 --2- 172.21.15.83:0/1072352534 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a8c1038e0 0x7f8a8c10bdc0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8a74009860 tx=0x7f8a7402a3f0 comp rx=0 tx=0).stop 2024-04-24T09:09:45.180 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.179+0000 7f8a8be00640 1 -- 172.21.15.83:0/1072352534 shutdown_connections 2024-04-24T09:09:45.180 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.179+0000 7f8a8be00640 1 --2- 172.21.15.83:0/1072352534 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a8c10c3e0 0x7f8a8c10e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:45.181 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.179+0000 7f8a8be00640 1 --2- 172.21.15.83:0/1072352534 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a8c1038e0 0x7f8a8c10bdc0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:45.181 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.179+0000 7f8a8be00640 1 --2- 172.21.15.83:0/1072352534 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8a8c102fc0 0x7f8a8c1033a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:45.181 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.179+0000 7f8a8be00640 1 -- 172.21.15.83:0/1072352534 >> 172.21.15.83:0/1072352534 conn(0x7f8a8c0fc820 msgr2=0x7f8a8c0fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:45.181 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.180+0000 7f8a8be00640 1 -- 172.21.15.83:0/1072352534 shutdown_connections 2024-04-24T09:09:45.181 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.180+0000 7f8a8be00640 1 -- 172.21.15.83:0/1072352534 wait complete. 2024-04-24T09:09:45.182 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.181+0000 7f8a8be00640 1 Processor -- start 2024-04-24T09:09:45.182 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.182+0000 7f8a8be00640 1 -- start start 2024-04-24T09:09:45.182 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.182+0000 7f8a8be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a8c102fc0 0x7f8a8c10a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:45.182 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.182+0000 7f8a8be00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8a8c1038e0 0x7f8a8c10a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:45.183 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.182+0000 7f8a8be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a8c10c3e0 0x7f8a8c10ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:45.183 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.182+0000 7f8a8be00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8a8c10ff70 con 0x7f8a8c102fc0 2024-04-24T09:09:45.183 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.182+0000 7f8a8be00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8a8c10fdf0 con 0x7f8a8c10c3e0 2024-04-24T09:09:45.183 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.182+0000 7f8a8be00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8a8c1100f0 con 0x7f8a8c1038e0 2024-04-24T09:09:45.184 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.183+0000 7f8a8aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a8c102fc0 0x7f8a8c10a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:45.184 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.183+0000 7f8a8b400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a8c10c3e0 0x7f8a8c10ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:45.184 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.183+0000 7f8a8b400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a8c10c3e0 0x7f8a8c10ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:45096/0 (socket says 172.21.15.83:45096) 2024-04-24T09:09:45.184 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.183+0000 7f8a8b400640 1 -- 172.21.15.83:0/3110672334 learned_addr learned my addr 172.21.15.83:0/3110672334 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:09:45.184 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.183+0000 7f8a8aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a8c102fc0 0x7f8a8c10a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49262/0 (socket says 172.21.15.83:49262) 2024-04-24T09:09:45.184 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.183+0000 7f8a8a000640 1 --2- 172.21.15.83:0/3110672334 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8a8c1038e0 0x7f8a8c10a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:45.184 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.183+0000 7f8a8aa00640 1 -- 172.21.15.83:0/3110672334 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8a8c1038e0 msgr2=0x7f8a8c10a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:45.184 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.183+0000 7f8a8aa00640 1 --2- 172.21.15.83:0/3110672334 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8a8c1038e0 0x7f8a8c10a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:45.184 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.184+0000 7f8a8aa00640 1 -- 172.21.15.83:0/3110672334 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a8c10c3e0 msgr2=0x7f8a8c10ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:45.185 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.184+0000 7f8a8aa00640 1 --2- 172.21.15.83:0/3110672334 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a8c10c3e0 0x7f8a8c10ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:45.185 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.184+0000 7f8a8aa00640 1 -- 172.21.15.83:0/3110672334 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a8c1ab090 con 0x7f8a8c102fc0 2024-04-24T09:09:45.185 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.184+0000 7f8a8b400640 1 --2- 172.21.15.83:0/3110672334 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a8c10c3e0 0x7f8a8c10ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:09:45.185 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.184+0000 7f8a8aa00640 1 --2- 172.21.15.83:0/3110672334 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a8c102fc0 0x7f8a8c10a1d0 secure :-1 s=READY pgs=2380 cs=0 l=1 rev1=1 crypto rx=0x7f8a8400be20 tx=0x7f8a84009f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:45.185 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.184+0000 7f8a83400640 1 -- 172.21.15.83:0/3110672334 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a84019070 con 0x7f8a8c102fc0 2024-04-24T09:09:45.185 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.184+0000 7f8a83400640 1 -- 172.21.15.83:0/3110672334 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8a840148f0 con 0x7f8a8c102fc0 2024-04-24T09:09:45.185 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.184+0000 7f8a83400640 1 -- 172.21.15.83:0/3110672334 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a84012880 con 0x7f8a8c102fc0 2024-04-24T09:09:45.186 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.185+0000 7f8a8be00640 1 -- 172.21.15.83:0/3110672334 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a8c1ab380 con 0x7f8a8c102fc0 2024-04-24T09:09:45.186 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.185+0000 7f8a8be00640 1 -- 172.21.15.83:0/3110672334 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a8c1ab880 con 0x7f8a8c102fc0 2024-04-24T09:09:45.193 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.192+0000 7f8a945c5740 1 Processor -- start 2024-04-24T09:09:45.193 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.193+0000 7f8a945c5740 1 -- start start 2024-04-24T09:09:45.194 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.193+0000 7f8a945c5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5638ee1573c0 0x5638ee1577a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:45.194 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.193+0000 7f8a945c5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5638ee157ce0 0x5638ee162060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:45.194 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.193+0000 7f8a945c5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5638ee1625a0 0x5638ee164990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:45.194 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.193+0000 7f8a945c5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5638ee0bd900 con 0x5638ee157ce0 2024-04-24T09:09:45.194 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.193+0000 7f8a945c5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5638ee063f20 con 0x5638ee1573c0 2024-04-24T09:09:45.194 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.193+0000 7f8a945c5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5638ee0b98c0 con 0x5638ee1625a0 2024-04-24T09:09:45.194 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.194+0000 7f8a8a000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5638ee1625a0 0x5638ee164990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:45.195 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.194+0000 7f8a8a000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5638ee1625a0 0x5638ee164990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:44956/0 (socket says 172.21.15.83:44956) 2024-04-24T09:09:45.195 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.194+0000 7f8a8a000640 1 -- 172.21.15.83:0/2961786772 learned_addr learned my addr 172.21.15.83:0/2961786772 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:09:45.195 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.194+0000 7f8a8b400640 1 --2- 172.21.15.83:0/2961786772 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5638ee1573c0 0x5638ee1577a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:45.195 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.195+0000 7f8a8a000640 1 -- 172.21.15.83:0/2961786772 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5638ee1573c0 msgr2=0x5638ee1577a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:45.196 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.195+0000 7f8a8a000640 1 --2- 172.21.15.83:0/2961786772 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5638ee1573c0 0x5638ee1577a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:45.196 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.195+0000 7f8a8a000640 1 -- 172.21.15.83:0/2961786772 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5638ee157ce0 msgr2=0x5638ee162060 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T09:09:45.196 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.195+0000 7f8a8a000640 1 --2- 172.21.15.83:0/2961786772 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5638ee157ce0 0x5638ee162060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:45.196 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.195+0000 7f8a8a000640 1 -- 172.21.15.83:0/2961786772 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638ee088e10 con 0x5638ee1625a0 2024-04-24T09:09:45.196 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.195+0000 7f8a8b400640 1 --2- 172.21.15.83:0/2961786772 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5638ee1573c0 0x5638ee1577a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:09:45.196 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.195+0000 7f8a83400640 1 -- 172.21.15.83:0/3110672334 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8a8401c070 con 0x7f8a8c102fc0 2024-04-24T09:09:45.196 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.196+0000 7f8a8a000640 1 --2- 172.21.15.83:0/2961786772 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5638ee1625a0 0x5638ee164990 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f8a74009860 tx=0x7f8a74030640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9de9677d956fd1f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:45.197 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.197+0000 7f8a7b400640 1 -- 172.21.15.83:0/2961786772 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a74005c90 con 0x5638ee1625a0 2024-04-24T09:09:45.197 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.197+0000 7f8a7b400640 1 -- 172.21.15.83:0/2961786772 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8a74040030 con 0x5638ee1625a0 2024-04-24T09:09:45.197 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.197+0000 7f8a7b400640 1 -- 172.21.15.83:0/2961786772 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a740055b0 con 0x5638ee1625a0 2024-04-24T09:09:45.198 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.198+0000 7f8a83400640 1 --2- 172.21.15.83:0/3110672334 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f8a700a3e60 0x7f8a700a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:45.198 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.198+0000 7f8a945c5740 1 -- 172.21.15.83:0/2961786772 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5638ee1625a0 msgr2=0x5638ee164990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:45.198 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.198+0000 7f8a945c5740 1 --2- 172.21.15.83:0/2961786772 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5638ee1625a0 0x5638ee164990 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f8a74009860 tx=0x7f8a74030640 comp rx=0 tx=0).stop 2024-04-24T09:09:45.198 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.198+0000 7f8a83400640 1 -- 172.21.15.83:0/3110672334 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 9583+0+0 (secure 0 0 0) 0x7f8a840d0410 con 0x7f8a8c102fc0 2024-04-24T09:09:45.199 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.199+0000 7f8a8b400640 1 --2- 172.21.15.83:0/3110672334 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f8a700a3e60 0x7f8a700a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:45.199 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.199+0000 7f8a945c5740 1 -- 172.21.15.83:0/2961786772 shutdown_connections 2024-04-24T09:09:45.199 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.199+0000 7f8a945c5740 1 --2- 172.21.15.83:0/2961786772 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5638ee1625a0 0x5638ee164990 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:45.199 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.199+0000 7f8a945c5740 1 --2- 172.21.15.83:0/2961786772 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5638ee157ce0 0x5638ee162060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:45.199 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.199+0000 7f8a945c5740 1 --2- 172.21.15.83:0/2961786772 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5638ee1573c0 0x5638ee1577a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:45.199 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.199+0000 7f8a945c5740 1 -- 172.21.15.83:0/2961786772 >> 172.21.15.83:0/2961786772 conn(0x5638ee1538d0 msgr2=0x5638ee153cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:45.200 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.199+0000 7f8a945c5740 1 -- 172.21.15.83:0/2961786772 shutdown_connections 2024-04-24T09:09:45.200 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.200+0000 7f8a8b400640 1 --2- 172.21.15.83:0/3110672334 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f8a700a3e60 0x7f8a700a6320 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f8a7c009e40 tx=0x7f8a7c014040 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:45.200 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.200+0000 7f8a945c5740 1 -- 172.21.15.83:0/2961786772 wait complete. 2024-04-24T09:09:45.201 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.201+0000 7f8a945c5740 1 -- start start 2024-04-24T09:09:45.201 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.201+0000 7f8a945c5740 1 Processor -- start 2024-04-24T09:09:45.201 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.201+0000 7f8a945c5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5638ee1573c0 0x5638ee15f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:45.202 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.201+0000 7f8a945c5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5638ee157ce0 0x5638ee15faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:45.202 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.201+0000 7f8a945c5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5638ee1625a0 0x5638ee160030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:45.202 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.201+0000 7f8a945c5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5638ee1650e0 con 0x5638ee1573c0 2024-04-24T09:09:45.202 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.201+0000 7f8a945c5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5638ee0772c0 con 0x5638ee157ce0 2024-04-24T09:09:45.202 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.201+0000 7f8a945c5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5638ee165260 con 0x5638ee1625a0 2024-04-24T09:09:45.202 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.201+0000 7f8a945c5740 10 client.0 _ll_register_callbacks cb 0x5638eddcfa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:09:45.204 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.202+0000 7f8a8b400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5638ee1625a0 0x5638ee160030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:45.204 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.202+0000 7f8a8b400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5638ee1625a0 0x5638ee160030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:44958/0 (socket says 172.21.15.83:44958) 2024-04-24T09:09:45.204 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.202+0000 7f8a8b400640 1 -- 172.21.15.83:0/2378609010 learned_addr learned my addr 172.21.15.83:0/2378609010 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:09:45.204 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.202+0000 7f8a8aa00640 1 --2- 172.21.15.83:0/2378609010 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5638ee1573c0 0x5638ee15f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:45.204 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.202+0000 7f8a8a000640 1 --2- 172.21.15.83:0/2378609010 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5638ee157ce0 0x5638ee15faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:45.204 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.202+0000 7f8a8b400640 1 -- 172.21.15.83:0/2378609010 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5638ee157ce0 msgr2=0x5638ee15faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:45.204 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.202+0000 7f8a8b400640 1 --2- 172.21.15.83:0/2378609010 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5638ee157ce0 0x5638ee15faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:45.204 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.202+0000 7f8a8b400640 1 -- 172.21.15.83:0/2378609010 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5638ee1573c0 msgr2=0x5638ee15f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:45.204 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.202+0000 7f8a8b400640 1 --2- 172.21.15.83:0/2378609010 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5638ee1573c0 0x5638ee15f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:45.204 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.202+0000 7f8a8b400640 1 -- 172.21.15.83:0/2378609010 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638ee087fd0 con 0x5638ee1625a0 2024-04-24T09:09:45.204 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.202+0000 7f8a8aa00640 1 --2- 172.21.15.83:0/2378609010 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5638ee1573c0 0x5638ee15f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:09:45.204 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.203+0000 7f8a8b400640 1 --2- 172.21.15.83:0/2378609010 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5638ee1625a0 0x5638ee160030 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8a7c003970 tx=0x7f8a7c004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb75936cba70e685 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:45.205 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.203+0000 7f8a6fe00640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-04-24T09:09:45.205 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.203+0000 7f8a6fe00640 1 -- 172.21.15.83:0/2378609010 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a7c00f890 con 0x5638ee1625a0 2024-04-24T09:09:45.205 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.203+0000 7f8a945c5740 1 -- 172.21.15.83:0/2378609010 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a74040030 con 0x5638ee1625a0 2024-04-24T09:09:45.205 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.203+0000 7f8a6fe00640 1 -- 172.21.15.83:0/2378609010 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9583+0+0 (secure 0 0 0) 0x7f8a7c00fa30 con 0x5638ee1625a0 2024-04-24T09:09:45.205 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.204+0000 7f8a6fe00640 1 -- 172.21.15.83:0/2378609010 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f8a7c00fc30 con 0x5638ee1625a0 2024-04-24T09:09:45.205 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.204+0000 7f8a945c5740 10 client.28924 fetch_fsmap learned FSMap version 61 2024-04-24T09:09:45.205 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.204+0000 7f8a945c5740 1 -- 172.21.15.83:0/2378609010 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x5638eddd0210 con 0x5638ee1625a0 2024-04-24T09:09:45.206 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.204+0000 7f8a6fe00640 1 -- 172.21.15.83:0/2378609010 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8a74040030 con 0x5638ee1625a0 2024-04-24T09:09:45.206 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.204+0000 7f8a6fe00640 1 -- 172.21.15.83:0/2378609010 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 61) ==== 8362+0+0 (secure 0 0 0) 0x7f8a7c025070 con 0x5638ee1625a0 2024-04-24T09:09:45.206 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.205+0000 7f8a945c5740 10 client.28924 fetch_fsmap finished waiting for FSMap version 61 2024-04-24T09:09:45.206 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.205+0000 7f8a945c5740 10 client.28924 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:09:45.206 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.205+0000 7f8a945c5740 20 client.28924 populate_metadata read hostname 'smithi083' 2024-04-24T09:09:45.206 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.205+0000 7f8a945c5740 1 --2- 172.21.15.83:0/2378609010 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5638ee172810 0x5638ee174c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:45.206 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.205+0000 7f8a945c5740 4 client.28924 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:09:45.206 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.205+0000 7f8a945c5740 1 -- 172.21.15.83:0/2378609010 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5638ee0cb320 con 0x5638ee172810 2024-04-24T09:09:45.207 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.205+0000 7f8a8aa00640 1 --2- 172.21.15.83:0/2378609010 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5638ee172810 0x5638ee174c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:45.207 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.206+0000 7f8a6fe00640 10 client.28924 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:09:45.207 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.206+0000 7f8a8aa00640 1 --2- 172.21.15.83:0/2378609010 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5638ee172810 0x5638ee174c00 crc :-1 s=READY pgs=453 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b7547a37391cac4 server_cookie=da2a2ecd2628202d in_seq=0 out_seq=0 2024-04-24T09:09:45.208 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.207+0000 7f8a6fe00640 1 -- 172.21.15.83:0/2378609010 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5638ee0cb320 con 0x5638ee172810 2024-04-24T09:09:45.208 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.207+0000 7f8a6fe00640 10 client.28924 handle_command_reply: tid=0 2024-04-24T09:09:45.208 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.207+0000 7f8a945c5740 1 client.28924 shutdown 2024-04-24T09:09:45.208 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.207+0000 7f8a945c5740 1 -- 172.21.15.83:0/2378609010 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5638ee1625a0 msgr2=0x5638ee160030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:45.208 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.207+0000 7f8a945c5740 1 --2- 172.21.15.83:0/2378609010 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5638ee1625a0 0x5638ee160030 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8a7c003970 tx=0x7f8a7c004730 comp rx=0 tx=0).stop 2024-04-24T09:09:45.208 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.207+0000 7f8a945c5740 1 -- 172.21.15.83:0/2378609010 shutdown_connections 2024-04-24T09:09:45.209 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.207+0000 7f8a945c5740 1 --2- 172.21.15.83:0/2378609010 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5638ee172810 0x5638ee174c00 crc :-1 s=READY pgs=453 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:45.209 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.207+0000 7f8a945c5740 1 --2- 172.21.15.83:0/2378609010 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5638ee1625a0 0x5638ee160030 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:45.209 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.207+0000 7f8a945c5740 1 --2- 172.21.15.83:0/2378609010 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5638ee157ce0 0x5638ee15faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:45.209 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.207+0000 7f8a945c5740 1 --2- 172.21.15.83:0/2378609010 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5638ee1573c0 0x5638ee15f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:45.209 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.207+0000 7f8a945c5740 1 -- 172.21.15.83:0/2378609010 >> 172.21.15.83:0/2378609010 conn(0x5638ee1538d0 msgr2=0x5638ee159650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:45.209 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.207+0000 7f8a6fe00640 0 client.28924 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:09:45.209 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.208+0000 7f8a945c5740 1 -- 172.21.15.83:0/2378609010 shutdown_connections 2024-04-24T09:09:45.209 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.208+0000 7f8a945c5740 1 -- 172.21.15.83:0/2378609010 wait complete. 2024-04-24T09:09:45.209 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.208+0000 7f8a945c5740 20 client.28924 trim_cache size 0 max 16384 2024-04-24T09:09:45.233 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.233+0000 7f8a945c5740 1 Processor -- start 2024-04-24T09:09:45.233 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.233+0000 7f8a945c5740 1 -- start start 2024-04-24T09:09:45.233 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.233+0000 7f8a945c5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5638ee1573c0 0x5638ee161ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:45.234 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.233+0000 7f8a945c5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5638ee157ce0 0x5638ee159dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:45.234 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.233+0000 7f8a945c5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5638ee1625a0 0x5638ee17a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:45.234 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.233+0000 7f8a945c5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8a8c1b59a0 con 0x5638ee157ce0 2024-04-24T09:09:45.234 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.233+0000 7f8a945c5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8a8c1b74e0 con 0x5638ee1573c0 2024-04-24T09:09:45.234 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.233+0000 7f8a945c5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8a8c1b6070 con 0x5638ee1625a0 2024-04-24T09:09:45.235 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.234+0000 7f8a8aa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5638ee1573c0 0x5638ee161ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:45.235 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.234+0000 7f8a8a000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5638ee157ce0 0x5638ee159dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:45.235 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.234+0000 7f8a8a000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5638ee157ce0 0x5638ee159dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49270/0 (socket says 172.21.15.83:49270) 2024-04-24T09:09:45.235 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.234+0000 7f8a8a000640 1 -- 172.21.15.83:0/1714947158 learned_addr learned my addr 172.21.15.83:0/1714947158 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:09:45.235 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.234+0000 7f8a8b400640 1 --2- 172.21.15.83:0/1714947158 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5638ee1625a0 0x5638ee17a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:45.235 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.234+0000 7f8a8a000640 1 -- 172.21.15.83:0/1714947158 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5638ee1625a0 msgr2=0x5638ee17a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:45.235 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.234+0000 7f8a8a000640 1 --2- 172.21.15.83:0/1714947158 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5638ee1625a0 0x5638ee17a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:45.235 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.234+0000 7f8a8a000640 1 -- 172.21.15.83:0/1714947158 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5638ee1573c0 msgr2=0x5638ee161ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:45.235 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.234+0000 7f8a8a000640 1 --2- 172.21.15.83:0/1714947158 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5638ee1573c0 0x5638ee161ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:45.235 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.234+0000 7f8a8a000640 1 -- 172.21.15.83:0/1714947158 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638ee174f70 con 0x5638ee157ce0 2024-04-24T09:09:45.236 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.235+0000 7f8a8a000640 1 --2- 172.21.15.83:0/1714947158 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5638ee157ce0 0x5638ee159dd0 secure :-1 s=READY pgs=2381 cs=0 l=1 rev1=1 crypto rx=0x7f8a74006010 tx=0x7f8a740095b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=331e52d84b2973d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:45.236 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.236+0000 7f8a7aa00640 1 -- 172.21.15.83:0/1714947158 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a74005c90 con 0x5638ee157ce0 2024-04-24T09:09:45.236 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.236+0000 7f8a7aa00640 1 -- 172.21.15.83:0/1714947158 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8a74040070 con 0x5638ee157ce0 2024-04-24T09:09:45.237 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.236+0000 7f8a7aa00640 1 -- 172.21.15.83:0/1714947158 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a74005450 con 0x5638ee157ce0 2024-04-24T09:09:45.238 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.237+0000 7f8a945c5740 1 -- 172.21.15.83:0/1714947158 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5638ee157ce0 msgr2=0x5638ee159dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:45.238 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.237+0000 7f8a945c5740 1 --2- 172.21.15.83:0/1714947158 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5638ee157ce0 0x5638ee159dd0 secure :-1 s=READY pgs=2381 cs=0 l=1 rev1=1 crypto rx=0x7f8a74006010 tx=0x7f8a740095b0 comp rx=0 tx=0).stop 2024-04-24T09:09:45.238 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.237+0000 7f8a945c5740 1 -- 172.21.15.83:0/1714947158 shutdown_connections 2024-04-24T09:09:45.238 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.237+0000 7f8a945c5740 1 --2- 172.21.15.83:0/1714947158 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5638ee1625a0 0x5638ee17a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:45.238 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.237+0000 7f8a945c5740 1 --2- 172.21.15.83:0/1714947158 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5638ee157ce0 0x5638ee159dd0 unknown :-1 s=CLOSED pgs=2381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:45.238 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.237+0000 7f8a945c5740 1 --2- 172.21.15.83:0/1714947158 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5638ee1573c0 0x5638ee161ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:45.238 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.237+0000 7f8a945c5740 1 -- 172.21.15.83:0/1714947158 >> 172.21.15.83:0/1714947158 conn(0x5638ee1538d0 msgr2=0x5638ee152200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:45.238 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.237+0000 7f8a945c5740 1 -- 172.21.15.83:0/1714947158 shutdown_connections 2024-04-24T09:09:45.238 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.238+0000 7f8a945c5740 1 -- 172.21.15.83:0/1714947158 wait complete. 2024-04-24T09:09:45.239 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.238+0000 7f8a945c5740 1 -- start start 2024-04-24T09:09:45.239 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.238+0000 7f8a945c5740 1 Processor -- start 2024-04-24T09:09:45.239 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.239+0000 7f8a945c5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5638ee1573c0 0x5638ee166800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:45.240 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.239+0000 7f8a945c5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5638ee157ce0 0x5638ee166d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:45.240 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.239+0000 7f8a945c5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5638ee1625a0 0x5638ee1d1d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:45.240 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.239+0000 7f8a945c5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8a8c1b6e10 con 0x5638ee157ce0 2024-04-24T09:09:45.240 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.239+0000 7f8a945c5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8a8c1b6740 con 0x5638ee1573c0 2024-04-24T09:09:45.240 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.239+0000 7f8a945c5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8a8c1b39a0 con 0x5638ee1625a0 2024-04-24T09:09:45.240 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.239+0000 7f8a945c5740 10 client.0 _ll_register_callbacks cb 0x7f8a7c0035f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:09:45.240 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.239+0000 7f8a8aa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5638ee1573c0 0x5638ee166800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:45.240 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.239+0000 7f8a8aa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5638ee1573c0 0x5638ee166800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:45122/0 (socket says 172.21.15.83:45122) 2024-04-24T09:09:45.240 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.239+0000 7f8a8aa00640 1 -- 172.21.15.83:0/3457398007 learned_addr learned my addr 172.21.15.83:0/3457398007 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:09:45.240 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.240+0000 7f8a8a000640 1 --2- 172.21.15.83:0/3457398007 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5638ee157ce0 0x5638ee166d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:45.241 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.240+0000 7f8a8b400640 1 --2- 172.21.15.83:0/3457398007 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5638ee1625a0 0x5638ee1d1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:45.241 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.240+0000 7f8a8a000640 1 -- 172.21.15.83:0/3457398007 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5638ee1625a0 msgr2=0x5638ee1d1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:45.241 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.240+0000 7f8a8a000640 1 --2- 172.21.15.83:0/3457398007 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5638ee1625a0 0x5638ee1d1d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:45.241 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.240+0000 7f8a8a000640 1 -- 172.21.15.83:0/3457398007 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5638ee1573c0 msgr2=0x5638ee166800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:45.242 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.240+0000 7f8a8b400640 1 --2- 172.21.15.83:0/3457398007 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5638ee1625a0 0x5638ee1d1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:09:45.242 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.240+0000 7f8a8a000640 1 --2- 172.21.15.83:0/3457398007 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5638ee1573c0 0x5638ee166800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:45.242 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.240+0000 7f8a8a000640 1 -- 172.21.15.83:0/3457398007 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638ede9c620 con 0x5638ee157ce0 2024-04-24T09:09:45.242 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.241+0000 7f8a8aa00640 1 --2- 172.21.15.83:0/3457398007 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5638ee1573c0 0x5638ee166800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:09:45.242 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.241+0000 7f8a8a000640 1 --2- 172.21.15.83:0/3457398007 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5638ee157ce0 0x5638ee166d40 secure :-1 s=READY pgs=2382 cs=0 l=1 rev1=1 crypto rx=0x7f8a74000c00 tx=0x7f8a74005930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88884fd2b84221f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:45.242 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.241+0000 7f8a6fe00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:09:45.243 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.242+0000 7f8a6fe00640 1 -- 172.21.15.83:0/3457398007 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a74005c90 con 0x5638ee157ce0 2024-04-24T09:09:45.243 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.243+0000 7f8a6fe00640 1 -- 172.21.15.83:0/3457398007 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9583+0+0 (secure 0 0 0) 0x7f8a740053e0 con 0x5638ee157ce0 2024-04-24T09:09:45.243 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.243+0000 7f8a945c5740 1 -- 172.21.15.83:0/3457398007 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a74040070 con 0x5638ee157ce0 2024-04-24T09:09:45.244 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.243+0000 7f8a6fe00640 1 -- 172.21.15.83:0/3457398007 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f8a74005600 con 0x5638ee157ce0 2024-04-24T09:09:45.244 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.243+0000 7f8a945c5740 10 client.21855 fetch_fsmap learned FSMap version 61 2024-04-24T09:09:45.244 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.243+0000 7f8a945c5740 1 -- 172.21.15.83:0/3457398007 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x5638ee1d94f0 con 0x5638ee157ce0 2024-04-24T09:09:45.245 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.245+0000 7f8a6fe00640 1 -- 172.21.15.83:0/3457398007 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8a74040070 con 0x5638ee157ce0 2024-04-24T09:09:45.245 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.245+0000 7f8a6fe00640 1 -- 172.21.15.83:0/3457398007 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 61) ==== 8362+0+0 (secure 0 0 0) 0x7f8a74004c00 con 0x5638ee157ce0 2024-04-24T09:09:45.245 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.245+0000 7f8a945c5740 10 client.21855 fetch_fsmap finished waiting for FSMap version 61 2024-04-24T09:09:45.245 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.245+0000 7f8a945c5740 10 client.21855 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:09:45.245 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.245+0000 7f8a945c5740 20 client.21855 populate_metadata read hostname 'smithi083' 2024-04-24T09:09:45.245 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.245+0000 7f8a945c5740 1 --2- 172.21.15.83:0/3457398007 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5638ee1d9720 0x5638ee1dbb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:45.245 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.245+0000 7f8a945c5740 4 client.21855 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T09:09:45.245 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.245+0000 7f8a945c5740 1 -- 172.21.15.83:0/3457398007 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5638ee0c5190 con 0x5638ee1d9720 2024-04-24T09:09:45.248 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.248+0000 7f8a8aa00640 1 --2- 172.21.15.83:0/3457398007 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5638ee1d9720 0x5638ee1dbb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:45.249 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.249+0000 7f8a8aa00640 1 --2- 172.21.15.83:0/3457398007 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5638ee1d9720 0x5638ee1dbb10 crc :-1 s=READY pgs=454 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb260a0c30266c56 server_cookie=528950749fde3a6f in_seq=0 out_seq=0 2024-04-24T09:09:45.249 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.249+0000 7f8a6fe00640 10 client.21855 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:09:45.377 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.377+0000 7f8a6fe00640 1 -- 172.21.15.83:0/3457398007 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5638ee0c5190 con 0x5638ee1d9720 2024-04-24T09:09:45.378 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.377+0000 7f8a6fe00640 10 client.21855 handle_command_reply: tid=0 2024-04-24T09:09:45.378 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.377+0000 7f8a945c5740 1 client.21855 shutdown 2024-04-24T09:09:45.378 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.377+0000 7f8a945c5740 1 -- 172.21.15.83:0/3457398007 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5638ee157ce0 msgr2=0x5638ee166d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:45.378 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.377+0000 7f8a945c5740 1 --2- 172.21.15.83:0/3457398007 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5638ee157ce0 0x5638ee166d40 secure :-1 s=READY pgs=2382 cs=0 l=1 rev1=1 crypto rx=0x7f8a74000c00 tx=0x7f8a74005930 comp rx=0 tx=0).stop 2024-04-24T09:09:45.378 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.377+0000 7f8a945c5740 1 -- 172.21.15.83:0/3457398007 shutdown_connections 2024-04-24T09:09:45.378 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.377+0000 7f8a945c5740 1 --2- 172.21.15.83:0/3457398007 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5638ee1d9720 0x5638ee1dbb10 crc :-1 s=READY pgs=454 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:45.378 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.378+0000 7f8a945c5740 1 --2- 172.21.15.83:0/3457398007 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5638ee1625a0 0x5638ee1d1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:45.378 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.378+0000 7f8a945c5740 1 --2- 172.21.15.83:0/3457398007 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5638ee157ce0 0x5638ee166d40 unknown :-1 s=CLOSED pgs=2382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:45.378 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.378+0000 7f8a945c5740 1 --2- 172.21.15.83:0/3457398007 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5638ee1573c0 0x5638ee166800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:45.378 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.378+0000 7f8a6fe00640 0 client.21855 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:09:45.379 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.378+0000 7f8a945c5740 1 -- 172.21.15.83:0/3457398007 >> 172.21.15.83:0/3457398007 conn(0x5638ee1538d0 msgr2=0x5638ee152a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:45.379 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.378+0000 7f8a945c5740 1 -- 172.21.15.83:0/3457398007 shutdown_connections 2024-04-24T09:09:45.379 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.378+0000 7f8a945c5740 1 -- 172.21.15.83:0/3457398007 wait complete. 2024-04-24T09:09:45.379 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.378+0000 7f8a945c5740 20 client.21855 trim_cache size 0 max 16384 2024-04-24T09:09:45.380 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T09:09:45.380 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T09:09:45.380 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T09:09:45.380 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T09:09:45.381 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.379+0000 7f8a8be00640 1 -- 172.21.15.83:0/3110672334 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f8a700a3e60 msgr2=0x7f8a700a6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:45.381 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.379+0000 7f8a8be00640 1 --2- 172.21.15.83:0/3110672334 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f8a700a3e60 0x7f8a700a6320 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f8a7c009e40 tx=0x7f8a7c014040 comp rx=0 tx=0).stop 2024-04-24T09:09:45.381 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.379+0000 7f8a8be00640 1 -- 172.21.15.83:0/3110672334 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a8c102fc0 msgr2=0x7f8a8c10a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:45.381 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.379+0000 7f8a8be00640 1 --2- 172.21.15.83:0/3110672334 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a8c102fc0 0x7f8a8c10a1d0 secure :-1 s=READY pgs=2380 cs=0 l=1 rev1=1 crypto rx=0x7f8a8400be20 tx=0x7f8a84009f70 comp rx=0 tx=0).stop 2024-04-24T09:09:45.381 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.380+0000 7f8a8be00640 1 -- 172.21.15.83:0/3110672334 shutdown_connections 2024-04-24T09:09:45.381 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.380+0000 7f8a8be00640 1 --2- 172.21.15.83:0/3110672334 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f8a700a3e60 0x7f8a700a6320 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:45.381 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.380+0000 7f8a8be00640 1 --2- 172.21.15.83:0/3110672334 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a8c10c3e0 0x7f8a8c10ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:45.381 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.380+0000 7f8a8be00640 1 --2- 172.21.15.83:0/3110672334 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8a8c1038e0 0x7f8a8c10a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:45.381 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.380+0000 7f8a8be00640 1 --2- 172.21.15.83:0/3110672334 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a8c102fc0 0x7f8a8c10a1d0 unknown :-1 s=CLOSED pgs=2380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:45.382 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.380+0000 7f8a8be00640 1 -- 172.21.15.83:0/3110672334 >> 172.21.15.83:0/3110672334 conn(0x7f8a8c0fc820 msgr2=0x7f8a8c0fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:45.382 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.380+0000 7f8a8be00640 1 -- 172.21.15.83:0/3110672334 shutdown_connections 2024-04-24T09:09:45.382 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:45.380+0000 7f8a8be00640 1 -- 172.21.15.83:0/3110672334 wait complete. 2024-04-24T09:09:46.173 INFO:tasks.workunit.client.0.smithi083.stdout: 10 32768 6.87 MB/sec execute 137 sec latency 111.598 ms 2024-04-24T09:09:46.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:09:45 smithi083 ceph-mon[27907]: pgmap v571: 129 pgs: 129 active+clean; 579 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1017 KiB/s rd, 3.3 MiB/s wr, 188 op/s 2024-04-24T09:09:46.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:09:45 smithi138 ceph-mon[31441]: pgmap v571: 129 pgs: 129 active+clean; 579 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1017 KiB/s rd, 3.3 MiB/s wr, 188 op/s 2024-04-24T09:09:46.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:09:45 smithi189 ceph-mon[31791]: pgmap v571: 129 pgs: 129 active+clean; 579 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1017 KiB/s rd, 3.3 MiB/s wr, 188 op/s 2024-04-24T09:09:47.172 INFO:tasks.workunit.client.0.smithi083.stdout: 10 32889 6.86 MB/sec execute 138 sec latency 63.481 ms 2024-04-24T09:09:48.172 INFO:tasks.workunit.client.0.smithi083.stdout: 10 33031 6.88 MB/sec execute 139 sec latency 103.993 ms 2024-04-24T09:09:48.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:09:47 smithi083 ceph-mon[27907]: pgmap v572: 129 pgs: 129 active+clean; 579 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1017 KiB/s rd, 3.3 MiB/s wr, 188 op/s 2024-04-24T09:09:48.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:09:47 smithi138 ceph-mon[31441]: pgmap v572: 129 pgs: 129 active+clean; 579 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1017 KiB/s rd, 3.3 MiB/s wr, 188 op/s 2024-04-24T09:09:48.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:09:47 smithi189 ceph-mon[31791]: pgmap v572: 129 pgs: 129 active+clean; 579 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1017 KiB/s rd, 3.3 MiB/s wr, 188 op/s 2024-04-24T09:09:48.792 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T09:09:48.792 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T09:09:49.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.047+0000 7f55ac200640 1 -- 172.21.15.83:0/590512958 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f55a4102c40 msgr2=0x7f55a4106c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:49.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.047+0000 7f55ac200640 1 --2- 172.21.15.83:0/590512958 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f55a4102c40 0x7f55a4106c80 secure :-1 s=READY pgs=2383 cs=0 l=1 rev1=1 crypto rx=0x7f5590009900 tx=0x7f559002a400 comp rx=0 tx=0).stop 2024-04-24T09:09:49.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.047+0000 7f55ac200640 1 -- 172.21.15.83:0/590512958 shutdown_connections 2024-04-24T09:09:49.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.047+0000 7f55ac200640 1 --2- 172.21.15.83:0/590512958 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f55a4107360 0x7f55a410ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:49.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.047+0000 7f55ac200640 1 --2- 172.21.15.83:0/590512958 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f55a4102c40 0x7f55a4106c80 unknown :-1 s=CLOSED pgs=2383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:49.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.047+0000 7f55ac200640 1 --2- 172.21.15.83:0/590512958 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f55a4102290 0x7f55a4102670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:49.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.047+0000 7f55ac200640 1 -- 172.21.15.83:0/590512958 >> 172.21.15.83:0/590512958 conn(0x7f55a40fdb10 msgr2=0x7f55a40fff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:49.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.048+0000 7f55ac200640 1 -- 172.21.15.83:0/590512958 shutdown_connections 2024-04-24T09:09:49.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.048+0000 7f55ac200640 1 -- 172.21.15.83:0/590512958 wait complete. 2024-04-24T09:09:49.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.049+0000 7f55ac200640 1 Processor -- start 2024-04-24T09:09:49.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.049+0000 7f55ac200640 1 -- start start 2024-04-24T09:09:49.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.050+0000 7f55ac200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f55a4102290 0x7f55a419ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:49.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.050+0000 7f55ac200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f55a4107360 0x7f55a419f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:49.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.050+0000 7f55ac200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f55a41a36c0 0x7f55a41a3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:49.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.050+0000 7f55ac200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f55a4118910 con 0x7f55a4102290 2024-04-24T09:09:49.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.050+0000 7f55ac200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f55a4118790 con 0x7f55a41a36c0 2024-04-24T09:09:49.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.050+0000 7f55ac200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f55a4118a90 con 0x7f55a4107360 2024-04-24T09:09:49.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.050+0000 7f55a9800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f55a4102290 0x7f55a419ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:49.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.050+0000 7f55a9800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f55a4102290 0x7f55a419ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58140/0 (socket says 172.21.15.83:58140) 2024-04-24T09:09:49.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.050+0000 7f55a9800640 1 -- 172.21.15.83:0/716999496 learned_addr learned my addr 172.21.15.83:0/716999496 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:09:49.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.050+0000 7f55a8e00640 1 --2- 172.21.15.83:0/716999496 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f55a4107360 0x7f55a419f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:49.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.050+0000 7f55aa200640 1 --2- 172.21.15.83:0/716999496 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f55a41a36c0 0x7f55a41a3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:49.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.051+0000 7f55a9800640 1 -- 172.21.15.83:0/716999496 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f55a4107360 msgr2=0x7f55a419f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:49.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.051+0000 7f55a9800640 1 --2- 172.21.15.83:0/716999496 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f55a4107360 0x7f55a419f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:49.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.051+0000 7f55a9800640 1 -- 172.21.15.83:0/716999496 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f55a41a36c0 msgr2=0x7f55a41a3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:49.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.051+0000 7f55a9800640 1 --2- 172.21.15.83:0/716999496 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f55a41a36c0 0x7f55a41a3b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:49.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.051+0000 7f55a9800640 1 -- 172.21.15.83:0/716999496 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55a41a41b0 con 0x7f55a4102290 2024-04-24T09:09:49.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.051+0000 7f55a9800640 1 --2- 172.21.15.83:0/716999496 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f55a4102290 0x7f55a419ee50 secure :-1 s=READY pgs=2384 cs=0 l=1 rev1=1 crypto rx=0x7f559800c890 tx=0x7f559800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:49.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.051+0000 7f55a2000640 1 -- 172.21.15.83:0/716999496 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5598004220 con 0x7f55a4102290 2024-04-24T09:09:49.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.051+0000 7f55ac200640 1 -- 172.21.15.83:0/716999496 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55a41a44a0 con 0x7f55a4102290 2024-04-24T09:09:49.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.052+0000 7f55ac200640 1 -- 172.21.15.83:0/716999496 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55a41abcb0 con 0x7f55a4102290 2024-04-24T09:09:49.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.052+0000 7f55a2000640 1 -- 172.21.15.83:0/716999496 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f55980043c0 con 0x7f55a4102290 2024-04-24T09:09:49.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.052+0000 7f55a2000640 1 -- 172.21.15.83:0/716999496 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f559801d060 con 0x7f55a4102290 2024-04-24T09:09:49.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.056+0000 7f55af01e740 1 Processor -- start 2024-04-24T09:09:49.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.056+0000 7f55af01e740 1 -- start start 2024-04-24T09:09:49.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.057+0000 7f55af01e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ab5a307860 0x55ab5a307c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:49.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.057+0000 7f55af01e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ab5a308180 0x55ab5a312310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:49.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.057+0000 7f55af01e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ab5a312850 0x55ab5a314c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:49.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.057+0000 7f55af01e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ab5a26d1f0 con 0x55ab5a312850 2024-04-24T09:09:49.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.057+0000 7f55af01e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ab5a213350 con 0x55ab5a307860 2024-04-24T09:09:49.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.057+0000 7f55af01e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55ab5a2691b0 con 0x55ab5a308180 2024-04-24T09:09:49.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.057+0000 7f55a8e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ab5a312850 0x55ab5a314c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:49.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.057+0000 7f55a8e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ab5a312850 0x55ab5a314c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58156/0 (socket says 172.21.15.83:58156) 2024-04-24T09:09:49.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.057+0000 7f55a8e00640 1 -- 172.21.15.83:0/1744733330 learned_addr learned my addr 172.21.15.83:0/1744733330 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:09:49.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.057+0000 7f55aa200640 1 --2- 172.21.15.83:0/1744733330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ab5a307860 0x55ab5a307c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:49.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.058+0000 7f55a8e00640 1 -- 172.21.15.83:0/1744733330 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ab5a308180 msgr2=0x55ab5a312310 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T09:09:49.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.058+0000 7f55a8e00640 1 --2- 172.21.15.83:0/1744733330 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ab5a308180 0x55ab5a312310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:49.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.058+0000 7f55a8e00640 1 -- 172.21.15.83:0/1744733330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ab5a307860 msgr2=0x55ab5a307c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:49.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.058+0000 7f55a8e00640 1 --2- 172.21.15.83:0/1744733330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ab5a307860 0x55ab5a307c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:49.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.058+0000 7f55a8e00640 1 -- 172.21.15.83:0/1744733330 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab5a267f50 con 0x55ab5a312850 2024-04-24T09:09:49.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.058+0000 7f55a2000640 1 -- 172.21.15.83:0/716999496 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5598004a00 con 0x7f55a4102290 2024-04-24T09:09:49.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.058+0000 7f55a8e00640 1 --2- 172.21.15.83:0/1744733330 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ab5a312850 0x55ab5a314c40 secure :-1 s=READY pgs=2385 cs=0 l=1 rev1=1 crypto rx=0x7f55a406b6e0 tx=0x7f55900043d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13dd3598f05f1f41 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:49.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.059+0000 7f5596a00640 1 -- 172.21.15.83:0/1744733330 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5590040070 con 0x55ab5a312850 2024-04-24T09:09:49.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.059+0000 7f5596a00640 1 -- 172.21.15.83:0/1744733330 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f55900047a0 con 0x55ab5a312850 2024-04-24T09:09:49.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.059+0000 7f5596a00640 1 -- 172.21.15.83:0/1744733330 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5590035040 con 0x55ab5a312850 2024-04-24T09:09:49.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.060+0000 7f55a2000640 1 --2- 172.21.15.83:0/716999496 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f558c0a3d80 0x7f558c0a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:49.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.060+0000 7f55af01e740 1 -- 172.21.15.83:0/1744733330 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ab5a312850 msgr2=0x55ab5a314c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:49.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.060+0000 7f55af01e740 1 --2- 172.21.15.83:0/1744733330 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ab5a312850 0x55ab5a314c40 secure :-1 s=READY pgs=2385 cs=0 l=1 rev1=1 crypto rx=0x7f55a406b6e0 tx=0x7f55900043d0 comp rx=0 tx=0).stop 2024-04-24T09:09:49.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.060+0000 7f55a2000640 1 -- 172.21.15.83:0/716999496 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 9583+0+0 (secure 0 0 0) 0x7f55980d0710 con 0x7f55a4102290 2024-04-24T09:09:49.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.061+0000 7f55aa200640 1 --2- 172.21.15.83:0/716999496 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f558c0a3d80 0x7f558c0a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:49.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.062+0000 7f55aa200640 1 --2- 172.21.15.83:0/716999496 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f558c0a3d80 0x7f558c0a6240 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f559c007de0 tx=0x7f559c007520 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:49.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.062+0000 7f55af01e740 1 -- 172.21.15.83:0/1744733330 shutdown_connections 2024-04-24T09:09:49.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.062+0000 7f55af01e740 1 --2- 172.21.15.83:0/1744733330 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ab5a312850 0x55ab5a314c40 unknown :-1 s=CLOSED pgs=2385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:49.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.062+0000 7f55af01e740 1 --2- 172.21.15.83:0/1744733330 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ab5a308180 0x55ab5a312310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:49.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.062+0000 7f55af01e740 1 --2- 172.21.15.83:0/1744733330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ab5a307860 0x55ab5a307c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:49.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.062+0000 7f55af01e740 1 -- 172.21.15.83:0/1744733330 >> 172.21.15.83:0/1744733330 conn(0x55ab5a303b80 msgr2=0x55ab5a303f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:49.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.062+0000 7f55af01e740 1 -- 172.21.15.83:0/1744733330 shutdown_connections 2024-04-24T09:09:49.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.062+0000 7f55af01e740 1 -- 172.21.15.83:0/1744733330 wait complete. 2024-04-24T09:09:49.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.063+0000 7f55af01e740 1 -- start start 2024-04-24T09:09:49.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.063+0000 7f55af01e740 1 Processor -- start 2024-04-24T09:09:49.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.063+0000 7f55af01e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ab5a307860 0x55ab5a30f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:49.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.063+0000 7f55af01e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ab5a308180 0x55ab5a30fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:49.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.064+0000 7f55af01e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ab5a312850 0x55ab5a3102b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:49.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.064+0000 7f55af01e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ab5a3155e0 con 0x55ab5a308180 2024-04-24T09:09:49.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.064+0000 7f55af01e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ab5a226690 con 0x55ab5a307860 2024-04-24T09:09:49.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.064+0000 7f55af01e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55ab5a315760 con 0x55ab5a312850 2024-04-24T09:09:49.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.064+0000 7f55af01e740 10 client.0 _ll_register_callbacks cb 0x55ab5a262650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:09:49.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.064+0000 7f55a9800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ab5a307860 0x55ab5a30f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:49.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.064+0000 7f55a8e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ab5a308180 0x55ab5a30fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:49.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.064+0000 7f55a9800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ab5a307860 0x55ab5a30f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:50162/0 (socket says 172.21.15.83:50162) 2024-04-24T09:09:49.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.064+0000 7f55a9800640 1 -- 172.21.15.83:0/3061090419 learned_addr learned my addr 172.21.15.83:0/3061090419 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:09:49.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.064+0000 7f55aa200640 1 --2- 172.21.15.83:0/3061090419 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ab5a312850 0x55ab5a3102b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:49.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.064+0000 7f55a8e00640 1 -- 172.21.15.83:0/3061090419 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ab5a312850 msgr2=0x55ab5a3102b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:49.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.064+0000 7f55a8e00640 1 --2- 172.21.15.83:0/3061090419 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ab5a312850 0x55ab5a3102b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:49.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.064+0000 7f55a8e00640 1 -- 172.21.15.83:0/3061090419 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ab5a307860 msgr2=0x55ab5a30f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:49.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.064+0000 7f55a8e00640 1 --2- 172.21.15.83:0/3061090419 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ab5a307860 0x55ab5a30f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:49.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.064+0000 7f55a8e00640 1 -- 172.21.15.83:0/3061090419 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab5a225920 con 0x55ab5a308180 2024-04-24T09:09:49.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.064+0000 7f55aa200640 1 --2- 172.21.15.83:0/3061090419 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ab5a312850 0x55ab5a3102b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:09:49.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.065+0000 7f55a8e00640 1 --2- 172.21.15.83:0/3061090419 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ab5a308180 0x55ab5a30fd70 secure :-1 s=READY pgs=2386 cs=0 l=1 rev1=1 crypto rx=0x7f559002a920 tx=0x7f5590004bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d293f1a69392980 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:49.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.065+0000 7f558b400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:09:49.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.065+0000 7f558b400640 1 -- 172.21.15.83:0/3061090419 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5590035070 con 0x55ab5a308180 2024-04-24T09:09:49.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.065+0000 7f55af01e740 1 -- 172.21.15.83:0/3061090419 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55900047a0 con 0x55ab5a308180 2024-04-24T09:09:49.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.065+0000 7f558b400640 1 -- 172.21.15.83:0/3061090419 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9583+0+0 (secure 0 0 0) 0x7f5590005430 con 0x55ab5a308180 2024-04-24T09:09:49.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.066+0000 7f558b400640 1 -- 172.21.15.83:0/3061090419 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f5590040040 con 0x55ab5a308180 2024-04-24T09:09:49.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.066+0000 7f55af01e740 10 client.21867 fetch_fsmap learned FSMap version 61 2024-04-24T09:09:49.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.066+0000 7f55af01e740 1 -- 172.21.15.83:0/3061090419 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x55ab5a0497d0 con 0x55ab5a308180 2024-04-24T09:09:49.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.066+0000 7f558b400640 1 -- 172.21.15.83:0/3061090419 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f55900047a0 con 0x55ab5a308180 2024-04-24T09:09:49.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.067+0000 7f558b400640 1 -- 172.21.15.83:0/3061090419 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 61) ==== 8362+0+0 (secure 0 0 0) 0x7f5590031530 con 0x55ab5a308180 2024-04-24T09:09:49.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.067+0000 7f55af01e740 10 client.21867 fetch_fsmap finished waiting for FSMap version 61 2024-04-24T09:09:49.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.067+0000 7f55af01e740 10 client.21867 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:09:49.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.067+0000 7f55af01e740 20 client.21867 populate_metadata read hostname 'smithi083' 2024-04-24T09:09:49.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.067+0000 7f55af01e740 1 --2- 172.21.15.83:0/3061090419 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ab5a322ae0 0x55ab5a324ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:49.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.067+0000 7f55af01e740 4 client.21867 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:09:49.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.067+0000 7f55af01e740 1 -- 172.21.15.83:0/3061090419 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ab5a27ac10 con 0x55ab5a322ae0 2024-04-24T09:09:49.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.068+0000 7f55a9800640 1 --2- 172.21.15.83:0/3061090419 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ab5a322ae0 0x55ab5a324ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:49.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.069+0000 7f55a9800640 1 --2- 172.21.15.83:0/3061090419 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ab5a322ae0 0x55ab5a324ed0 crc :-1 s=READY pgs=455 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b083460e32ce2c4b server_cookie=2b87be900bc17593 in_seq=0 out_seq=0 2024-04-24T09:09:49.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.069+0000 7f558b400640 10 client.21867 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:09:49.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.071+0000 7f558b400640 1 -- 172.21.15.83:0/3061090419 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ab5a27ac10 con 0x55ab5a322ae0 2024-04-24T09:09:49.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.071+0000 7f558b400640 10 client.21867 handle_command_reply: tid=0 2024-04-24T09:09:49.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.071+0000 7f55af01e740 1 client.21867 shutdown 2024-04-24T09:09:49.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.071+0000 7f55af01e740 1 -- 172.21.15.83:0/3061090419 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ab5a308180 msgr2=0x55ab5a30fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:49.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.071+0000 7f55af01e740 1 --2- 172.21.15.83:0/3061090419 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ab5a308180 0x55ab5a30fd70 secure :-1 s=READY pgs=2386 cs=0 l=1 rev1=1 crypto rx=0x7f559002a920 tx=0x7f5590004bf0 comp rx=0 tx=0).stop 2024-04-24T09:09:49.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.072+0000 7f55af01e740 1 -- 172.21.15.83:0/3061090419 shutdown_connections 2024-04-24T09:09:49.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.072+0000 7f55af01e740 1 --2- 172.21.15.83:0/3061090419 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ab5a322ae0 0x55ab5a324ed0 crc :-1 s=READY pgs=455 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:49.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.072+0000 7f55af01e740 1 --2- 172.21.15.83:0/3061090419 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ab5a312850 0x55ab5a3102b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:49.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.072+0000 7f55af01e740 1 --2- 172.21.15.83:0/3061090419 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ab5a308180 0x55ab5a30fd70 unknown :-1 s=CLOSED pgs=2386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:49.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.072+0000 7f55af01e740 1 --2- 172.21.15.83:0/3061090419 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ab5a307860 0x55ab5a30f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:49.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.072+0000 7f55af01e740 1 -- 172.21.15.83:0/3061090419 >> 172.21.15.83:0/3061090419 conn(0x55ab5a303b80 msgr2=0x55ab5a309880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:49.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.072+0000 7f558b400640 0 client.21867 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:09:49.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.072+0000 7f55af01e740 1 -- 172.21.15.83:0/3061090419 shutdown_connections 2024-04-24T09:09:49.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.072+0000 7f55af01e740 1 -- 172.21.15.83:0/3061090419 wait complete. 2024-04-24T09:09:49.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.072+0000 7f55af01e740 20 client.21867 trim_cache size 0 max 16384 2024-04-24T09:09:49.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.101+0000 7f55af01e740 1 Processor -- start 2024-04-24T09:09:49.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.101+0000 7f55af01e740 1 -- start start 2024-04-24T09:09:49.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.101+0000 7f55af01e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ab5a307860 0x55ab5a31e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:49.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.101+0000 7f55af01e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ab5a308180 0x55ab5a32a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:49.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.101+0000 7f55af01e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ab5a312850 0x55ab5a32a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:49.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.101+0000 7f55af01e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5584010100 con 0x55ab5a308180 2024-04-24T09:09:49.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.101+0000 7f55af01e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5584011c40 con 0x55ab5a307860 2024-04-24T09:09:49.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.101+0000 7f55af01e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f55840107d0 con 0x55ab5a312850 2024-04-24T09:09:49.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.102+0000 7f55a8e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ab5a308180 0x55ab5a32a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:49.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.102+0000 7f55a9800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ab5a307860 0x55ab5a31e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:49.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.102+0000 7f55a8e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ab5a308180 0x55ab5a32a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58182/0 (socket says 172.21.15.83:58182) 2024-04-24T09:09:49.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.102+0000 7f55a8e00640 1 -- 172.21.15.83:0/4257481742 learned_addr learned my addr 172.21.15.83:0/4257481742 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:09:49.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.102+0000 7f55aa200640 1 --2- 172.21.15.83:0/4257481742 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ab5a312850 0x55ab5a32a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:49.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.102+0000 7f55a8e00640 1 -- 172.21.15.83:0/4257481742 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ab5a312850 msgr2=0x55ab5a32a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:49.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.102+0000 7f55a8e00640 1 --2- 172.21.15.83:0/4257481742 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ab5a312850 0x55ab5a32a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:49.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.102+0000 7f55a8e00640 1 -- 172.21.15.83:0/4257481742 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ab5a307860 msgr2=0x55ab5a31e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:49.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.102+0000 7f55a8e00640 1 --2- 172.21.15.83:0/4257481742 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ab5a307860 0x55ab5a31e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:49.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.102+0000 7f55a8e00640 1 -- 172.21.15.83:0/4257481742 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab59f7f3f0 con 0x55ab5a308180 2024-04-24T09:09:49.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.102+0000 7f55a9800640 1 --2- 172.21.15.83:0/4257481742 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ab5a307860 0x55ab5a31e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:09:49.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.103+0000 7f55a8e00640 1 --2- 172.21.15.83:0/4257481742 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ab5a308180 0x55ab5a32a4a0 secure :-1 s=READY pgs=2387 cs=0 l=1 rev1=1 crypto rx=0x7f5590031440 tx=0x7f55900058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99570743dbe2903 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:49.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.103+0000 7f5596000640 1 -- 172.21.15.83:0/4257481742 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5590035070 con 0x55ab5a308180 2024-04-24T09:09:49.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.103+0000 7f5596000640 1 -- 172.21.15.83:0/4257481742 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5590009520 con 0x55ab5a308180 2024-04-24T09:09:49.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.103+0000 7f5596000640 1 -- 172.21.15.83:0/4257481742 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55900035f0 con 0x55ab5a308180 2024-04-24T09:09:49.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.104+0000 7f55af01e740 1 -- 172.21.15.83:0/4257481742 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ab5a308180 msgr2=0x55ab5a32a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:49.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.104+0000 7f55af01e740 1 --2- 172.21.15.83:0/4257481742 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ab5a308180 0x55ab5a32a4a0 secure :-1 s=READY pgs=2387 cs=0 l=1 rev1=1 crypto rx=0x7f5590031440 tx=0x7f55900058d0 comp rx=0 tx=0).stop 2024-04-24T09:09:49.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.104+0000 7f55af01e740 1 -- 172.21.15.83:0/4257481742 shutdown_connections 2024-04-24T09:09:49.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.104+0000 7f55af01e740 1 --2- 172.21.15.83:0/4257481742 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ab5a312850 0x55ab5a32a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:49.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.104+0000 7f55af01e740 1 --2- 172.21.15.83:0/4257481742 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ab5a308180 0x55ab5a32a4a0 unknown :-1 s=CLOSED pgs=2387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:49.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.104+0000 7f55af01e740 1 --2- 172.21.15.83:0/4257481742 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ab5a307860 0x55ab5a31e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:49.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.104+0000 7f55af01e740 1 -- 172.21.15.83:0/4257481742 >> 172.21.15.83:0/4257481742 conn(0x55ab5a303b80 msgr2=0x55ab5a303f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:49.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.104+0000 7f55af01e740 1 -- 172.21.15.83:0/4257481742 shutdown_connections 2024-04-24T09:09:49.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.105+0000 7f55af01e740 1 -- 172.21.15.83:0/4257481742 wait complete. 2024-04-24T09:09:49.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.105+0000 7f55af01e740 1 -- start start 2024-04-24T09:09:49.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.105+0000 7f55af01e740 1 Processor -- start 2024-04-24T09:09:49.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.106+0000 7f55af01e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ab5a307860 0x55ab5a31aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:49.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.106+0000 7f55af01e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ab5a308180 0x55ab5a30cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:49.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.106+0000 7f55af01e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ab5a312850 0x55ab5a30d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:49.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.106+0000 7f55af01e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5584011570 con 0x55ab5a312850 2024-04-24T09:09:49.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.106+0000 7f55af01e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5584010ea0 con 0x55ab5a308180 2024-04-24T09:09:49.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.106+0000 7f55af01e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f558400e100 con 0x55ab5a307860 2024-04-24T09:09:49.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.106+0000 7f55af01e740 10 client.0 _ll_register_callbacks cb 0x7f55900042a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:09:49.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.106+0000 7f55aa200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ab5a312850 0x55ab5a30d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:49.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.106+0000 7f55aa200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ab5a312850 0x55ab5a30d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58190/0 (socket says 172.21.15.83:58190) 2024-04-24T09:09:49.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.106+0000 7f55aa200640 1 -- 172.21.15.83:0/293271886 learned_addr learned my addr 172.21.15.83:0/293271886 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:09:49.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.106+0000 7f55a9800640 1 --2- 172.21.15.83:0/293271886 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ab5a307860 0x55ab5a31aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:49.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.107+0000 7f55a8e00640 1 --2- 172.21.15.83:0/293271886 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ab5a308180 0x55ab5a30cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:49.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.107+0000 7f55a8e00640 1 -- 172.21.15.83:0/293271886 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ab5a307860 msgr2=0x55ab5a31aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:49.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.107+0000 7f55a8e00640 1 --2- 172.21.15.83:0/293271886 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ab5a307860 0x55ab5a31aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:49.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.107+0000 7f55a8e00640 1 -- 172.21.15.83:0/293271886 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ab5a312850 msgr2=0x55ab5a30d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:49.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.107+0000 7f55a8e00640 1 --2- 172.21.15.83:0/293271886 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ab5a312850 0x55ab5a30d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:49.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.107+0000 7f55a8e00640 1 -- 172.21.15.83:0/293271886 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab59f3f0f0 con 0x55ab5a308180 2024-04-24T09:09:49.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.107+0000 7f55aa200640 1 --2- 172.21.15.83:0/293271886 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ab5a312850 0x55ab5a30d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:09:49.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.107+0000 7f55a8e00640 1 --2- 172.21.15.83:0/293271886 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ab5a308180 0x55ab5a30cc00 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x55ab5a2ff560 tx=0x7f5590004080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e15743f94b05ca41 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:49.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.107+0000 7f558b400640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:09:49.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.108+0000 7f558b400640 1 -- 172.21.15.83:0/293271886 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5590035040 con 0x55ab5a308180 2024-04-24T09:09:49.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.108+0000 7f558b400640 1 -- 172.21.15.83:0/293271886 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9583+0+0 (secure 0 0 0) 0x7f5590005430 con 0x55ab5a308180 2024-04-24T09:09:49.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.108+0000 7f55af01e740 1 -- 172.21.15.83:0/293271886 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5590009520 con 0x55ab5a308180 2024-04-24T09:09:49.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.108+0000 7f558b400640 1 -- 172.21.15.83:0/293271886 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f5590004820 con 0x55ab5a308180 2024-04-24T09:09:49.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.108+0000 7f55af01e740 10 client.39005 fetch_fsmap learned FSMap version 61 2024-04-24T09:09:49.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.108+0000 7f55af01e740 1 -- 172.21.15.83:0/293271886 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x55ab5a232810 con 0x55ab5a308180 2024-04-24T09:09:49.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.109+0000 7f558b400640 1 -- 172.21.15.83:0/293271886 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5590009520 con 0x55ab5a308180 2024-04-24T09:09:49.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.109+0000 7f558b400640 1 -- 172.21.15.83:0/293271886 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 61) ==== 8362+0+0 (secure 0 0 0) 0x7f559005a070 con 0x55ab5a308180 2024-04-24T09:09:49.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.109+0000 7f55af01e740 10 client.39005 fetch_fsmap finished waiting for FSMap version 61 2024-04-24T09:09:49.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.109+0000 7f55af01e740 10 client.39005 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:09:49.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.109+0000 7f55af01e740 20 client.39005 populate_metadata read hostname 'smithi083' 2024-04-24T09:09:49.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.109+0000 7f55af01e740 1 --2- 172.21.15.83:0/293271886 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ab5a387f30 0x55ab5a38a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:49.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.109+0000 7f55af01e740 4 client.39005 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T09:09:49.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.109+0000 7f55af01e740 1 -- 172.21.15.83:0/293271886 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ab5a274a80 con 0x55ab5a387f30 2024-04-24T09:09:49.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.109+0000 7f55a9800640 1 --2- 172.21.15.83:0/293271886 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ab5a387f30 0x55ab5a38a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:49.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.110+0000 7f55a9800640 1 --2- 172.21.15.83:0/293271886 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ab5a387f30 0x55ab5a38a320 crc :-1 s=READY pgs=456 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ac9b47f35ac2af5 server_cookie=94a53aac04cb5dbc in_seq=0 out_seq=0 2024-04-24T09:09:49.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.110+0000 7f558b400640 10 client.39005 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:09:49.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.112+0000 7f558b400640 1 -- 172.21.15.83:0/293271886 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55ab5a274a80 con 0x55ab5a387f30 2024-04-24T09:09:49.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.112+0000 7f558b400640 10 client.39005 handle_command_reply: tid=0 2024-04-24T09:09:49.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.112+0000 7f55af01e740 1 client.39005 shutdown 2024-04-24T09:09:49.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.112+0000 7f55af01e740 1 -- 172.21.15.83:0/293271886 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ab5a308180 msgr2=0x55ab5a30cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:49.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.112+0000 7f55af01e740 1 --2- 172.21.15.83:0/293271886 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ab5a308180 0x55ab5a30cc00 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x55ab5a2ff560 tx=0x7f5590004080 comp rx=0 tx=0).stop 2024-04-24T09:09:49.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.112+0000 7f55af01e740 1 -- 172.21.15.83:0/293271886 shutdown_connections 2024-04-24T09:09:49.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.112+0000 7f55af01e740 1 --2- 172.21.15.83:0/293271886 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ab5a387f30 0x55ab5a38a320 crc :-1 s=READY pgs=456 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:49.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.112+0000 7f55af01e740 1 --2- 172.21.15.83:0/293271886 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ab5a312850 0x55ab5a30d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:49.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.112+0000 7f55af01e740 1 --2- 172.21.15.83:0/293271886 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ab5a308180 0x55ab5a30cc00 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:49.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.112+0000 7f55af01e740 1 --2- 172.21.15.83:0/293271886 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ab5a307860 0x55ab5a31aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:49.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.112+0000 7f55af01e740 1 -- 172.21.15.83:0/293271886 >> 172.21.15.83:0/293271886 conn(0x55ab5a303b80 msgr2=0x55ab5a30ab30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:49.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.112+0000 7f558b400640 0 client.39005 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:09:49.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.112+0000 7f55af01e740 1 -- 172.21.15.83:0/293271886 shutdown_connections 2024-04-24T09:09:49.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.112+0000 7f55af01e740 1 -- 172.21.15.83:0/293271886 wait complete. 2024-04-24T09:09:49.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.113+0000 7f55af01e740 20 client.39005 trim_cache size 0 max 16384 2024-04-24T09:09:49.114 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T09:09:49.114 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-04-24T09:09:49.114 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-04-24T09:09:49.114 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T09:09:49.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.113+0000 7f55ac200640 1 -- 172.21.15.83:0/716999496 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f558c0a3d80 msgr2=0x7f558c0a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:49.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.113+0000 7f55ac200640 1 --2- 172.21.15.83:0/716999496 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f558c0a3d80 0x7f558c0a6240 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f559c007de0 tx=0x7f559c007520 comp rx=0 tx=0).stop 2024-04-24T09:09:49.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.113+0000 7f55ac200640 1 -- 172.21.15.83:0/716999496 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f55a4102290 msgr2=0x7f55a419ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:49.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.113+0000 7f55ac200640 1 --2- 172.21.15.83:0/716999496 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f55a4102290 0x7f55a419ee50 secure :-1 s=READY pgs=2384 cs=0 l=1 rev1=1 crypto rx=0x7f559800c890 tx=0x7f559800cd50 comp rx=0 tx=0).stop 2024-04-24T09:09:49.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.114+0000 7f55ac200640 1 -- 172.21.15.83:0/716999496 shutdown_connections 2024-04-24T09:09:49.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.114+0000 7f55ac200640 1 --2- 172.21.15.83:0/716999496 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f558c0a3d80 0x7f558c0a6240 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:49.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.114+0000 7f55ac200640 1 --2- 172.21.15.83:0/716999496 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f55a41a36c0 0x7f55a41a3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:49.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.114+0000 7f55ac200640 1 --2- 172.21.15.83:0/716999496 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f55a4107360 0x7f55a419f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:49.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.114+0000 7f55ac200640 1 --2- 172.21.15.83:0/716999496 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f55a4102290 0x7f55a419ee50 unknown :-1 s=CLOSED pgs=2384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:49.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.114+0000 7f55ac200640 1 -- 172.21.15.83:0/716999496 >> 172.21.15.83:0/716999496 conn(0x7f55a40fdb10 msgr2=0x7f55a40ffe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:49.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.114+0000 7f55ac200640 1 -- 172.21.15.83:0/716999496 shutdown_connections 2024-04-24T09:09:49.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.114+0000 7f55ac200640 1 -- 172.21.15.83:0/716999496 wait complete. 2024-04-24T09:09:49.135 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T09:09:49.135 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T09:09:49.172 INFO:tasks.workunit.client.0.smithi083.stdout: 10 33157 6.86 MB/sec execute 140 sec latency 77.126 ms 2024-04-24T09:09:49.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.400+0000 7f491e000640 1 -- 172.21.15.83:0/3933352437 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4910031880 con 0x7f4920102090 2024-04-24T09:09:49.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.400+0000 7f491fe00640 1 -- 172.21.15.83:0/3933352437 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4920102090 msgr2=0x7f4920102470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:49.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.400+0000 7f491fe00640 1 --2- 172.21.15.83:0/3933352437 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4920102090 0x7f4920102470 secure :-1 s=READY pgs=2388 cs=0 l=1 rev1=1 crypto rx=0x7f4910009900 tx=0x7f491002a410 comp rx=0 tx=0).stop 2024-04-24T09:09:49.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.400+0000 7f491fe00640 1 -- 172.21.15.83:0/3933352437 shutdown_connections 2024-04-24T09:09:49.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.400+0000 7f491fe00640 1 --2- 172.21.15.83:0/3933352437 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4920107160 0x7f492010ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:49.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.400+0000 7f491fe00640 1 --2- 172.21.15.83:0/3933352437 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4920102a40 0x7f4920106a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:49.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.400+0000 7f491fe00640 1 --2- 172.21.15.83:0/3933352437 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4920102090 0x7f4920102470 unknown :-1 s=CLOSED pgs=2388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:49.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.400+0000 7f491fe00640 1 -- 172.21.15.83:0/3933352437 >> 172.21.15.83:0/3933352437 conn(0x7f49200fd930 msgr2=0x7f49200ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:49.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.401+0000 7f491fe00640 1 -- 172.21.15.83:0/3933352437 shutdown_connections 2024-04-24T09:09:49.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.401+0000 7f491fe00640 1 -- 172.21.15.83:0/3933352437 wait complete. 2024-04-24T09:09:49.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.402+0000 7f491fe00640 1 Processor -- start 2024-04-24T09:09:49.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.402+0000 7f491fe00640 1 -- start start 2024-04-24T09:09:49.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.403+0000 7f491fe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4920102090 0x7f49201a05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:49.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.403+0000 7f491fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4920107160 0x7f49201a0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:49.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.403+0000 7f491fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f492019a690 0x7f492019ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:49.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.403+0000 7f491fe00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4920112110 con 0x7f492019a690 2024-04-24T09:09:49.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.403+0000 7f491fe00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4920111f90 con 0x7f4920107160 2024-04-24T09:09:49.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.403+0000 7f491fe00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4920112290 con 0x7f4920102090 2024-04-24T09:09:49.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.403+0000 7f491ea00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4920102090 0x7f49201a05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:49.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.403+0000 7f491ea00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4920102090 0x7f49201a05d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:41696/0 (socket says 172.21.15.83:41696) 2024-04-24T09:09:49.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.403+0000 7f4916000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4920107160 0x7f49201a0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:49.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.403+0000 7f491ea00640 1 -- 172.21.15.83:0/2430788928 learned_addr learned my addr 172.21.15.83:0/2430788928 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:09:49.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.403+0000 7f491f400640 1 --2- 172.21.15.83:0/2430788928 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f492019a690 0x7f492019ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:49.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.404+0000 7f4916000640 1 -- 172.21.15.83:0/2430788928 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4920102090 msgr2=0x7f49201a05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:49.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.404+0000 7f4916000640 1 --2- 172.21.15.83:0/2430788928 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4920102090 0x7f49201a05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:49.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.404+0000 7f4916000640 1 -- 172.21.15.83:0/2430788928 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f492019a690 msgr2=0x7f492019ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:49.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.404+0000 7f4916000640 1 --2- 172.21.15.83:0/2430788928 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f492019a690 0x7f492019ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:49.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.404+0000 7f4916000640 1 -- 172.21.15.83:0/2430788928 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f492019b290 con 0x7f4920107160 2024-04-24T09:09:49.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.405+0000 7f4916000640 1 --2- 172.21.15.83:0/2430788928 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4920107160 0x7f49201a0b10 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f490800e9c0 tx=0x7f490800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:49.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.405+0000 7f4924a00640 1 -- 172.21.15.83:0/2430788928 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f490800ccc0 con 0x7f4920107160 2024-04-24T09:09:49.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.405+0000 7f4924a00640 1 -- 172.21.15.83:0/2430788928 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4908005020 con 0x7f4920107160 2024-04-24T09:09:49.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.405+0000 7f4924a00640 1 -- 172.21.15.83:0/2430788928 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4908004a00 con 0x7f4920107160 2024-04-24T09:09:49.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.406+0000 7f491fe00640 1 -- 172.21.15.83:0/2430788928 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49201a7370 con 0x7f4920107160 2024-04-24T09:09:49.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.406+0000 7f491fe00640 1 -- 172.21.15.83:0/2430788928 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49201a7720 con 0x7f4920107160 2024-04-24T09:09:49.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.407+0000 7f491fe00640 1 -- 172.21.15.83:0/2430788928 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4920103560 con 0x7f4920107160 2024-04-24T09:09:49.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.411+0000 7f4924a00640 1 -- 172.21.15.83:0/2430788928 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f49080051d0 con 0x7f4920107160 2024-04-24T09:09:49.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.413+0000 7f4924a00640 1 --2- 172.21.15.83:0/2430788928 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f49040a3da0 0x7f49040a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:49.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.413+0000 7f4924a00640 1 -- 172.21.15.83:0/2430788928 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 9583+0+0 (secure 0 0 0) 0x7f49080d0be0 con 0x7f4920107160 2024-04-24T09:09:49.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.413+0000 7f491ea00640 1 --2- 172.21.15.83:0/2430788928 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f49040a3da0 0x7f49040a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:49.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.414+0000 7f491ea00640 1 --2- 172.21.15.83:0/2430788928 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f49040a3da0 0x7f49040a6260 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f4910031440 tx=0x7f49100058d0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:49.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.419+0000 7f4924a00640 1 -- 172.21.15.83:0/2430788928 <== mon.2 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f490809ce90 con 0x7f4920107160 2024-04-24T09:09:49.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.867+0000 7f491fe00640 1 -- 172.21.15.83:0/2430788928 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4920068d10 con 0x7f4920107160 2024-04-24T09:09:49.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.868+0000 7f4924a00640 1 -- 172.21.15.83:0/2430788928 <== mon.2 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) ==== 94+0+11944 (secure 0 0 0) 0x7f490809c790 con 0x7f4920107160 2024-04-24T09:09:49.868 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T09:09:49.869 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":61,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/358872257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":358872257},{"type":"v1","addr":"172.21.15.83:6835","nonce":358872257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3157070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3157070851},{"type":"v1","addr":"172.21.15.83:6837","nonce":3157070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":61,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T08:48:29.422576+0000","modified":"2024-04-24T09:09:34.468831+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34520,"mds_1":34493,"mds_2":34451,"mds_3":34427,"mds_4":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14658":{"gid":14658,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6839/4085470844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":4085470844},{"type":"v1","addr":"172.21.15.83:6839","nonce":4085470844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14676":{"gid":14676,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6841/2634754210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":2634754210},{"type":"v1","addr":"172.21.15.83:6841","nonce":2634754210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24382":{"gid":24382,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/1859921310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1859921310},{"type":"v1","addr":"172.21.15.189:6833","nonce":1859921310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/2241923197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2241923197},{"type":"v1","addr":"172.21.15.189:6835","nonce":2241923197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.189:6839/2548965336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":2548965336},{"type":"v1","addr":"172.21.15.189:6839","nonce":2548965336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26686":{"gid":26686,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6837/1678817161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":1678817161},{"type":"v1","addr":"172.21.15.189:6837","nonce":1678817161}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.138:6833/1157967658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1157967658},{"type":"v1","addr":"172.21.15.138:6833","nonce":1157967658}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.138:6835/1909219758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1909219758},{"type":"v1","addr":"172.21.15.138:6835","nonce":1909219758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34493":{"gid":34493,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.138:6837/900566754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":900566754},{"type":"v1","addr":"172.21.15.138:6837","nonce":900566754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34520":{"gid":34520,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.138:6839/2236932393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2236932393},{"type":"v1","addr":"172.21.15.138:6839","nonce":2236932393}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34520,"qdb_cluster":[24454,34520,34427,34451,34493]},"id":1}]} 2024-04-24T09:09:49.869 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 61 2024-04-24T09:09:49.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.871+0000 7f491fe00640 1 -- 172.21.15.83:0/2430788928 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f49040a3da0 msgr2=0x7f49040a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:49.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.871+0000 7f491fe00640 1 --2- 172.21.15.83:0/2430788928 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f49040a3da0 0x7f49040a6260 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f4910031440 tx=0x7f49100058d0 comp rx=0 tx=0).stop 2024-04-24T09:09:49.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.871+0000 7f491fe00640 1 -- 172.21.15.83:0/2430788928 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4920107160 msgr2=0x7f49201a0b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:49.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.871+0000 7f491fe00640 1 --2- 172.21.15.83:0/2430788928 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4920107160 0x7f49201a0b10 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f490800e9c0 tx=0x7f490800ee80 comp rx=0 tx=0).stop 2024-04-24T09:09:49.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.871+0000 7f491fe00640 1 -- 172.21.15.83:0/2430788928 shutdown_connections 2024-04-24T09:09:49.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.871+0000 7f491fe00640 1 --2- 172.21.15.83:0/2430788928 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f49040a3da0 0x7f49040a6260 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:49.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.872+0000 7f491fe00640 1 --2- 172.21.15.83:0/2430788928 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f492019a690 0x7f492019ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:49.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.872+0000 7f491fe00640 1 --2- 172.21.15.83:0/2430788928 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4920107160 0x7f49201a0b10 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:49.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.872+0000 7f491fe00640 1 --2- 172.21.15.83:0/2430788928 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4920102090 0x7f49201a05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:49.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.872+0000 7f491fe00640 1 -- 172.21.15.83:0/2430788928 >> 172.21.15.83:0/2430788928 conn(0x7f49200fd930 msgr2=0x7f49200ffc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:49.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.872+0000 7f491fe00640 1 -- 172.21.15.83:0/2430788928 shutdown_connections 2024-04-24T09:09:49.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:49.872+0000 7f491fe00640 1 -- 172.21.15.83:0/2430788928 wait complete. 2024-04-24T09:09:49.895 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-24T09:09:50.173 INFO:tasks.workunit.client.0.smithi083.stdout: 10 33243 6.84 MB/sec execute 141 sec latency 105.089 ms 2024-04-24T09:09:50.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.195+0000 7f5052800640 1 -- 172.21.15.83:0/3532696632 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f504c0743f0 msgr2=0x7f504c0747d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:50.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.195+0000 7f5052800640 1 --2- 172.21.15.83:0/3532696632 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f504c0743f0 0x7f504c0747d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5040009900 tx=0x7f504002a480 comp rx=0 tx=0).stop 2024-04-24T09:09:50.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.195+0000 7f5052800640 1 -- 172.21.15.83:0/3532696632 shutdown_connections 2024-04-24T09:09:50.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.195+0000 7f5052800640 1 --2- 172.21.15.83:0/3532696632 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f504c10e720 0x7f504c110b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:50.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.195+0000 7f5052800640 1 --2- 172.21.15.83:0/3532696632 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f504c074d10 0x7f504c10e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:50.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.195+0000 7f5052800640 1 --2- 172.21.15.83:0/3532696632 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f504c0743f0 0x7f504c0747d0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:50.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.195+0000 7f5052800640 1 -- 172.21.15.83:0/3532696632 >> 172.21.15.83:0/3532696632 conn(0x7f504c0fd9d0 msgr2=0x7f504c0ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:50.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.195+0000 7f5052800640 1 -- 172.21.15.83:0/3532696632 shutdown_connections 2024-04-24T09:09:50.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.195+0000 7f5052800640 1 -- 172.21.15.83:0/3532696632 wait complete. 2024-04-24T09:09:50.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.197+0000 7f5052800640 1 Processor -- start 2024-04-24T09:09:50.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.197+0000 7f5052800640 1 -- start start 2024-04-24T09:09:50.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.197+0000 7f5052800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f504c0743f0 0x7f504c1078b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:50.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.197+0000 7f5052800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f504c074d10 0x7f504c107df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:50.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.197+0000 7f5052800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f504c10e720 0x7f504c1029c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:50.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.198+0000 7f5052800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f504c114390 con 0x7f504c074d10 2024-04-24T09:09:50.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.198+0000 7f5052800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f504c114210 con 0x7f504c0743f0 2024-04-24T09:09:50.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.198+0000 7f5052800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f504c114510 con 0x7f504c10e720 2024-04-24T09:09:50.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.198+0000 7f5051400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f504c0743f0 0x7f504c1078b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:50.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.198+0000 7f5051400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f504c0743f0 0x7f504c1078b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:50212/0 (socket says 172.21.15.83:50212) 2024-04-24T09:09:50.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.198+0000 7f5051400640 1 -- 172.21.15.83:0/1390738096 learned_addr learned my addr 172.21.15.83:0/1390738096 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:09:50.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.198+0000 7f5050a00640 1 --2- 172.21.15.83:0/1390738096 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f504c074d10 0x7f504c107df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:50.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.199+0000 7f5051e00640 1 --2- 172.21.15.83:0/1390738096 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f504c10e720 0x7f504c1029c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:50.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.199+0000 7f5051400640 1 -- 172.21.15.83:0/1390738096 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f504c10e720 msgr2=0x7f504c1029c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:50.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.199+0000 7f5051400640 1 --2- 172.21.15.83:0/1390738096 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f504c10e720 0x7f504c1029c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:50.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.199+0000 7f5051400640 1 -- 172.21.15.83:0/1390738096 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f504c074d10 msgr2=0x7f504c107df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:50.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.199+0000 7f5051400640 1 --2- 172.21.15.83:0/1390738096 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f504c074d10 0x7f504c107df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:50.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.199+0000 7f5051400640 1 -- 172.21.15.83:0/1390738096 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f504c103280 con 0x7f504c0743f0 2024-04-24T09:09:50.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.199+0000 7f5051400640 1 --2- 172.21.15.83:0/1390738096 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f504c0743f0 0x7f504c1078b0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f5040005ab0 tx=0x7f5040003c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:50.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.200+0000 7f504a000640 1 -- 172.21.15.83:0/1390738096 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f504003f070 con 0x7f504c0743f0 2024-04-24T09:09:50.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.200+0000 7f5052800640 1 -- 172.21.15.83:0/1390738096 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f504c103510 con 0x7f504c0743f0 2024-04-24T09:09:50.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.200+0000 7f5052800640 1 -- 172.21.15.83:0/1390738096 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f504c10d7a0 con 0x7f504c0743f0 2024-04-24T09:09:50.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.200+0000 7f504a000640 1 -- 172.21.15.83:0/1390738096 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5040031d60 con 0x7f504c0743f0 2024-04-24T09:09:50.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.200+0000 7f504a000640 1 -- 172.21.15.83:0/1390738096 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5040049060 con 0x7f504c0743f0 2024-04-24T09:09:50.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.204+0000 7f5056db7740 1 Processor -- start 2024-04-24T09:09:50.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.204+0000 7f5056db7740 1 -- start start 2024-04-24T09:09:50.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.205+0000 7f5056db7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5636e3261b40 0x5636e3261f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:50.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.205+0000 7f5056db7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636e3262460 0x5636e326c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:50.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.205+0000 7f5056db7740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5636e326cb30 0x5636e326ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:50.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.205+0000 7f5056db7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5636e31c71d0 con 0x5636e3262460 2024-04-24T09:09:50.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.205+0000 7f5056db7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5636e316d450 con 0x5636e3261b40 2024-04-24T09:09:50.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.205+0000 7f5056db7740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5636e31c3190 con 0x5636e326cb30 2024-04-24T09:09:50.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.205+0000 7f504a000640 1 -- 172.21.15.83:0/1390738096 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5040042070 con 0x7f504c0743f0 2024-04-24T09:09:50.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.205+0000 7f5051e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5636e3261b40 0x5636e3261f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:50.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.205+0000 7f5051e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5636e3261b40 0x5636e3261f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:50214/0 (socket says 172.21.15.83:50214) 2024-04-24T09:09:50.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.205+0000 7f5051e00640 1 -- 172.21.15.83:0/3865312627 learned_addr learned my addr 172.21.15.83:0/3865312627 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:09:50.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.205+0000 7f5050a00640 1 --2- 172.21.15.83:0/3865312627 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5636e326cb30 0x5636e326ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:50.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.205+0000 7f5051400640 1 --2- 172.21.15.83:0/3865312627 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636e3262460 0x5636e326c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:50.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.206+0000 7f5051e00640 1 -- 172.21.15.83:0/3865312627 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5636e326cb30 msgr2=0x5636e326ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:50.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.206+0000 7f5051e00640 1 --2- 172.21.15.83:0/3865312627 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5636e326cb30 0x5636e326ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:50.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.206+0000 7f5051e00640 1 -- 172.21.15.83:0/3865312627 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636e3262460 msgr2=0x5636e326c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:50.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.206+0000 7f5051e00640 1 --2- 172.21.15.83:0/3865312627 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636e3262460 0x5636e326c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:50.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.206+0000 7f5051e00640 1 -- 172.21.15.83:0/3865312627 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636e322a310 con 0x5636e3261b40 2024-04-24T09:09:50.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.206+0000 7f5051400640 1 --2- 172.21.15.83:0/3865312627 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636e3262460 0x5636e326c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:09:50.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.206+0000 7f5051e00640 1 --2- 172.21.15.83:0/3865312627 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5636e3261b40 0x5636e3261f20 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f5044009e00 tx=0x7f5044010740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b701915ab3e94c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:50.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.207+0000 7f503e600640 1 -- 172.21.15.83:0/3865312627 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5044016070 con 0x5636e3261b40 2024-04-24T09:09:50.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.207+0000 7f503e600640 1 -- 172.21.15.83:0/3865312627 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5044004d20 con 0x5636e3261b40 2024-04-24T09:09:50.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.207+0000 7f503e600640 1 -- 172.21.15.83:0/3865312627 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5044012650 con 0x5636e3261b40 2024-04-24T09:09:50.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.207+0000 7f504a000640 1 --2- 172.21.15.83:0/1390738096 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f50340a3da0 0x7f50340a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:50.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.208+0000 7f504a000640 1 -- 172.21.15.83:0/1390738096 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 9583+0+0 (secure 0 0 0) 0x7f50400f1160 con 0x7f504c0743f0 2024-04-24T09:09:50.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.208+0000 7f5056db7740 1 -- 172.21.15.83:0/3865312627 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5636e3261b40 msgr2=0x5636e3261f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:50.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.208+0000 7f5056db7740 1 --2- 172.21.15.83:0/3865312627 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5636e3261b40 0x5636e3261f20 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f5044009e00 tx=0x7f5044010740 comp rx=0 tx=0).stop 2024-04-24T09:09:50.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.208+0000 7f5056db7740 1 -- 172.21.15.83:0/3865312627 shutdown_connections 2024-04-24T09:09:50.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.208+0000 7f5056db7740 1 --2- 172.21.15.83:0/3865312627 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5636e326cb30 0x5636e326ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:50.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.208+0000 7f5056db7740 1 --2- 172.21.15.83:0/3865312627 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636e3262460 0x5636e326c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:50.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.208+0000 7f5056db7740 1 --2- 172.21.15.83:0/3865312627 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5636e3261b40 0x5636e3261f20 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:50.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.208+0000 7f5056db7740 1 -- 172.21.15.83:0/3865312627 >> 172.21.15.83:0/3865312627 conn(0x5636e325de60 msgr2=0x5636e325e240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:50.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.208+0000 7f5051400640 1 --2- 172.21.15.83:0/1390738096 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f50340a3da0 0x7f50340a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:50.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.208+0000 7f5056db7740 1 -- 172.21.15.83:0/3865312627 shutdown_connections 2024-04-24T09:09:50.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.209+0000 7f5056db7740 1 -- 172.21.15.83:0/3865312627 wait complete. 2024-04-24T09:09:50.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.209+0000 7f5051400640 1 --2- 172.21.15.83:0/1390738096 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f50340a3da0 0x7f50340a6260 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x5636e3263100 tx=0x7f5040031f10 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:50.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.209+0000 7f5056db7740 1 -- start start 2024-04-24T09:09:50.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.210+0000 7f5056db7740 1 Processor -- start 2024-04-24T09:09:50.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.210+0000 7f5056db7740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5636e3261b40 0x5636e326c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:50.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.210+0000 7f5056db7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636e3262460 0x5636e32698e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:50.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.210+0000 7f5056db7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5636e326cb30 0x5636e326a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:50.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.210+0000 7f5056db7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5636e326f9c0 con 0x5636e3262460 2024-04-24T09:09:50.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.210+0000 7f5056db7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5636e3180810 con 0x5636e326cb30 2024-04-24T09:09:50.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.210+0000 7f5056db7740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5636e326fb40 con 0x5636e3261b40 2024-04-24T09:09:50.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.210+0000 7f5056db7740 10 client.0 _ll_register_callbacks cb 0x5636e314e140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:09:50.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.211+0000 7f5050a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636e3262460 0x5636e32698e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:50.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.211+0000 7f5050a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636e3262460 0x5636e32698e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58244/0 (socket says 172.21.15.83:58244) 2024-04-24T09:09:50.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.211+0000 7f5050a00640 1 -- 172.21.15.83:0/709508797 learned_addr learned my addr 172.21.15.83:0/709508797 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:09:50.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.211+0000 7f5051e00640 1 --2- 172.21.15.83:0/709508797 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5636e326cb30 0x5636e326a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:50.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.211+0000 7f5051e00640 1 --2- 172.21.15.83:0/709508797 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5636e3261b40 0x5636e326c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:50.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.211+0000 7f5050a00640 1 -- 172.21.15.83:0/709508797 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5636e3261b40 msgr2=0x5636e326c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:50.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.211+0000 7f5050a00640 1 --2- 172.21.15.83:0/709508797 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5636e3261b40 0x5636e326c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:50.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.211+0000 7f5050a00640 1 -- 172.21.15.83:0/709508797 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5636e326cb30 msgr2=0x5636e326a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:50.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.211+0000 7f5050a00640 1 --2- 172.21.15.83:0/709508797 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5636e326cb30 0x5636e326a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:50.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.211+0000 7f5050a00640 1 -- 172.21.15.83:0/709508797 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636e31913f0 con 0x5636e3262460 2024-04-24T09:09:50.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.212+0000 7f5050a00640 1 --2- 172.21.15.83:0/709508797 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636e3262460 0x5636e32698e0 secure :-1 s=READY pgs=2389 cs=0 l=1 rev1=1 crypto rx=0x7f5038012700 tx=0x7f5038012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1621efc4c381764d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:50.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.212+0000 7f5032a00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:09:50.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.212+0000 7f5032a00640 1 -- 172.21.15.83:0/709508797 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f503800f920 con 0x5636e3262460 2024-04-24T09:09:50.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.212+0000 7f5056db7740 1 -- 172.21.15.83:0/709508797 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5044004d20 con 0x5636e3262460 2024-04-24T09:09:50.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.212+0000 7f5032a00640 1 -- 172.21.15.83:0/709508797 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9583+0+0 (secure 0 0 0) 0x7f503800fac0 con 0x5636e3262460 2024-04-24T09:09:50.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.213+0000 7f5032a00640 1 -- 172.21.15.83:0/709508797 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f503800fcc0 con 0x5636e3262460 2024-04-24T09:09:50.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.213+0000 7f5056db7740 10 client.21891 fetch_fsmap learned FSMap version 61 2024-04-24T09:09:50.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.213+0000 7f5056db7740 1 -- 172.21.15.83:0/709508797 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x5636e2ed93f0 con 0x5636e3262460 2024-04-24T09:09:50.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.213+0000 7f5032a00640 1 -- 172.21.15.83:0/709508797 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f50380108e0 con 0x5636e3262460 2024-04-24T09:09:50.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.213+0000 7f5032a00640 1 -- 172.21.15.83:0/709508797 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 61) ==== 8362+0+0 (secure 0 0 0) 0x7f503801c440 con 0x5636e3262460 2024-04-24T09:09:50.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.213+0000 7f5056db7740 10 client.21891 fetch_fsmap finished waiting for FSMap version 61 2024-04-24T09:09:50.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.213+0000 7f5056db7740 10 client.21891 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-24T09:09:50.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.213+0000 7f5056db7740 20 client.21891 populate_metadata read hostname 'smithi083' 2024-04-24T09:09:50.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.214+0000 7f5056db7740 1 --2- 172.21.15.83:0/709508797 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x5636e327cc50 0x5636e327f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:50.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.214+0000 7f5056db7740 4 client.21891 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:09:50.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.214+0000 7f5056db7740 1 -- 172.21.15.83:0/709508797 --> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f5044004d20 con 0x5636e327cc50 2024-04-24T09:09:50.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.214+0000 7f5051e00640 1 --2- 172.21.15.83:0/709508797 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x5636e327cc50 0x5636e327f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:50.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.215+0000 7f5051e00640 1 --2- 172.21.15.83:0/709508797 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x5636e327cc50 0x5636e327f070 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=99e032cbb4cf7c5a server_cookie=bccde312eeb92b59 in_seq=0 out_seq=0 2024-04-24T09:09:50.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.215+0000 7f5032a00640 10 client.21891 ms_handle_connect on v2:172.21.15.189:6838/2548965336 2024-04-24T09:09:50.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.217+0000 7f5032a00640 1 -- 172.21.15.83:0/709508797 <== mds.4 v2:172.21.15.189:6838/2548965336 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f5044004d20 con 0x5636e327cc50 2024-04-24T09:09:50.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.217+0000 7f5032a00640 10 client.21891 handle_command_reply: tid=0 2024-04-24T09:09:50.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.217+0000 7f5056db7740 1 client.21891 shutdown 2024-04-24T09:09:50.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.218+0000 7f5056db7740 1 -- 172.21.15.83:0/709508797 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636e3262460 msgr2=0x5636e32698e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:50.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.218+0000 7f5056db7740 1 --2- 172.21.15.83:0/709508797 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636e3262460 0x5636e32698e0 secure :-1 s=READY pgs=2389 cs=0 l=1 rev1=1 crypto rx=0x7f5038012700 tx=0x7f5038012bc0 comp rx=0 tx=0).stop 2024-04-24T09:09:50.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.218+0000 7f5056db7740 1 -- 172.21.15.83:0/709508797 shutdown_connections 2024-04-24T09:09:50.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.218+0000 7f5056db7740 1 --2- 172.21.15.83:0/709508797 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x5636e327cc50 0x5636e327f070 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:50.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.218+0000 7f5056db7740 1 --2- 172.21.15.83:0/709508797 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5636e326cb30 0x5636e326a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:50.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.218+0000 7f5056db7740 1 --2- 172.21.15.83:0/709508797 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636e3262460 0x5636e32698e0 unknown :-1 s=CLOSED pgs=2389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:50.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.218+0000 7f5056db7740 1 --2- 172.21.15.83:0/709508797 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5636e3261b40 0x5636e326c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:50.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.218+0000 7f5056db7740 1 -- 172.21.15.83:0/709508797 >> 172.21.15.83:0/709508797 conn(0x5636e325de60 msgr2=0x5636e3263b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:50.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.218+0000 7f5032a00640 0 client.21891 ms_handle_reset on v2:172.21.15.189:6838/2548965336 2024-04-24T09:09:50.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.218+0000 7f5056db7740 1 -- 172.21.15.83:0/709508797 shutdown_connections 2024-04-24T09:09:50.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.218+0000 7f5056db7740 1 -- 172.21.15.83:0/709508797 wait complete. 2024-04-24T09:09:50.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.218+0000 7f5056db7740 20 client.21891 trim_cache size 0 max 16384 2024-04-24T09:09:50.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.242+0000 7f5056db7740 1 Processor -- start 2024-04-24T09:09:50.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.242+0000 7f5056db7740 1 -- start start 2024-04-24T09:09:50.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.243+0000 7f5056db7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5636e3261b40 0x5636e3269880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:50.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.243+0000 7f5056db7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636e3262460 0x5636e32788e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:50.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.243+0000 7f5056db7740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5636e326cb30 0x5636e326c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:50.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.243+0000 7f5056db7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f504c1be240 con 0x5636e3262460 2024-04-24T09:09:50.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.243+0000 7f5056db7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f504c1bfd80 con 0x5636e3261b40 2024-04-24T09:09:50.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.243+0000 7f5056db7740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f504c1be910 con 0x5636e326cb30 2024-04-24T09:09:50.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.243+0000 7f5050a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636e3262460 0x5636e32788e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:50.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.243+0000 7f5050a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636e3262460 0x5636e32788e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58260/0 (socket says 172.21.15.83:58260) 2024-04-24T09:09:50.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.243+0000 7f5050a00640 1 -- 172.21.15.83:0/3504236125 learned_addr learned my addr 172.21.15.83:0/3504236125 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:09:50.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.243+0000 7f5051e00640 1 --2- 172.21.15.83:0/3504236125 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5636e3261b40 0x5636e3269880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:50.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.244+0000 7f5050a00640 1 -- 172.21.15.83:0/3504236125 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5636e326cb30 msgr2=0x5636e326c120 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T09:09:50.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.244+0000 7f5050a00640 1 --2- 172.21.15.83:0/3504236125 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5636e326cb30 0x5636e326c120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:50.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.244+0000 7f5050a00640 1 -- 172.21.15.83:0/3504236125 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5636e3261b40 msgr2=0x5636e3269880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:50.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.244+0000 7f5050a00640 1 --2- 172.21.15.83:0/3504236125 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5636e3261b40 0x5636e3269880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:50.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.244+0000 7f5050a00640 1 -- 172.21.15.83:0/3504236125 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636e2fa37d0 con 0x5636e3262460 2024-04-24T09:09:50.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.244+0000 7f5051e00640 1 --2- 172.21.15.83:0/3504236125 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5636e3261b40 0x5636e3269880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:09:50.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.244+0000 7f5050a00640 1 --2- 172.21.15.83:0/3504236125 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636e3262460 0x5636e32788e0 secure :-1 s=READY pgs=2390 cs=0 l=1 rev1=1 crypto rx=0x7f5038000c00 tx=0x7f50380096f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae2b1d2568635d2f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:50.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.244+0000 7f503dc00640 1 -- 172.21.15.83:0/3504236125 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f503800f920 con 0x5636e3262460 2024-04-24T09:09:50.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.244+0000 7f503dc00640 1 -- 172.21.15.83:0/3504236125 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f503800fac0 con 0x5636e3262460 2024-04-24T09:09:50.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.245+0000 7f503dc00640 1 -- 172.21.15.83:0/3504236125 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5038005560 con 0x5636e3262460 2024-04-24T09:09:50.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.245+0000 7f5056db7740 1 -- 172.21.15.83:0/3504236125 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636e3262460 msgr2=0x5636e32788e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:50.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.245+0000 7f5056db7740 1 --2- 172.21.15.83:0/3504236125 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636e3262460 0x5636e32788e0 secure :-1 s=READY pgs=2390 cs=0 l=1 rev1=1 crypto rx=0x7f5038000c00 tx=0x7f50380096f0 comp rx=0 tx=0).stop 2024-04-24T09:09:50.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.246+0000 7f5056db7740 1 -- 172.21.15.83:0/3504236125 shutdown_connections 2024-04-24T09:09:50.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.246+0000 7f5056db7740 1 --2- 172.21.15.83:0/3504236125 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5636e326cb30 0x5636e326c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:50.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.246+0000 7f5056db7740 1 --2- 172.21.15.83:0/3504236125 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636e3262460 0x5636e32788e0 unknown :-1 s=CLOSED pgs=2390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:50.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.246+0000 7f5056db7740 1 --2- 172.21.15.83:0/3504236125 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5636e3261b40 0x5636e3269880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:50.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.246+0000 7f5056db7740 1 -- 172.21.15.83:0/3504236125 >> 172.21.15.83:0/3504236125 conn(0x5636e325de60 msgr2=0x5636e3269170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:50.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.246+0000 7f5056db7740 1 -- 172.21.15.83:0/3504236125 shutdown_connections 2024-04-24T09:09:50.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.246+0000 7f5056db7740 1 -- 172.21.15.83:0/3504236125 wait complete. 2024-04-24T09:09:50.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.246+0000 7f5056db7740 1 -- start start 2024-04-24T09:09:50.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.247+0000 7f5056db7740 1 Processor -- start 2024-04-24T09:09:50.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.247+0000 7f5056db7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5636e3261b40 0x5636e326d850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:50.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.247+0000 7f5056db7740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5636e3262460 0x5636e326dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:50.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.247+0000 7f5056db7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636e326cb30 0x5636e326e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:50.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.247+0000 7f5056db7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f504c1bf6b0 con 0x5636e326cb30 2024-04-24T09:09:50.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.247+0000 7f5056db7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f504c1befe0 con 0x5636e3261b40 2024-04-24T09:09:50.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.247+0000 7f5056db7740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f504c1bc240 con 0x5636e3262460 2024-04-24T09:09:50.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.247+0000 7f5056db7740 10 client.0 _ll_register_callbacks cb 0x7f5038009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:09:50.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.248+0000 7f5051e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5636e3261b40 0x5636e326d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:50.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.248+0000 7f5051e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5636e3261b40 0x5636e326d850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:50224/0 (socket says 172.21.15.83:50224) 2024-04-24T09:09:50.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.248+0000 7f5051e00640 1 -- 172.21.15.83:0/2648399578 learned_addr learned my addr 172.21.15.83:0/2648399578 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:09:50.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.248+0000 7f5051e00640 1 --2- 172.21.15.83:0/2648399578 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636e326cb30 0x5636e326e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:50.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.248+0000 7f5050a00640 1 --2- 172.21.15.83:0/2648399578 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5636e3262460 0x5636e326dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:50.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.249+0000 7f5051e00640 1 -- 172.21.15.83:0/2648399578 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5636e3262460 msgr2=0x5636e326dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:50.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.249+0000 7f5051e00640 1 --2- 172.21.15.83:0/2648399578 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5636e3262460 0x5636e326dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:50.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.249+0000 7f5051e00640 1 -- 172.21.15.83:0/2648399578 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5636e3261b40 msgr2=0x5636e326d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:50.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.249+0000 7f5051e00640 1 --2- 172.21.15.83:0/2648399578 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5636e3261b40 0x5636e326d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:50.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.249+0000 7f5051e00640 1 -- 172.21.15.83:0/2648399578 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636e3272010 con 0x5636e326cb30 2024-04-24T09:09:50.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.249+0000 7f5051e00640 1 --2- 172.21.15.83:0/2648399578 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636e326cb30 0x5636e326e2d0 secure :-1 s=READY pgs=2391 cs=0 l=1 rev1=1 crypto rx=0x7f5044010cc0 tx=0x7f5044009f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=877edf5f96fcc6a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:50.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.249+0000 7f5032a00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:09:50.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.250+0000 7f5032a00640 1 -- 172.21.15.83:0/2648399578 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5044016040 con 0x5636e326cb30 2024-04-24T09:09:50.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.250+0000 7f5056db7740 1 -- 172.21.15.83:0/2648399578 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f503800fac0 con 0x5636e326cb30 2024-04-24T09:09:50.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.250+0000 7f5032a00640 1 -- 172.21.15.83:0/2648399578 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9583+0+0 (secure 0 0 0) 0x7f504402aae0 con 0x5636e326cb30 2024-04-24T09:09:50.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.250+0000 7f5032a00640 1 -- 172.21.15.83:0/2648399578 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f504403a050 con 0x5636e326cb30 2024-04-24T09:09:50.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.250+0000 7f5056db7740 10 client.21897 fetch_fsmap learned FSMap version 61 2024-04-24T09:09:50.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.250+0000 7f5056db7740 1 -- 172.21.15.83:0/2648399578 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x5636e32e3460 con 0x5636e326cb30 2024-04-24T09:09:50.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.251+0000 7f5032a00640 1 -- 172.21.15.83:0/2648399578 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f503800fac0 con 0x5636e326cb30 2024-04-24T09:09:50.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.251+0000 7f5032a00640 1 -- 172.21.15.83:0/2648399578 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 61) ==== 8362+0+0 (secure 0 0 0) 0x7f504400f040 con 0x5636e326cb30 2024-04-24T09:09:50.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.251+0000 7f5056db7740 10 client.21897 fetch_fsmap finished waiting for FSMap version 61 2024-04-24T09:09:50.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.251+0000 7f5056db7740 10 client.21897 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-24T09:09:50.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.251+0000 7f5056db7740 20 client.21897 populate_metadata read hostname 'smithi083' 2024-04-24T09:09:50.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.251+0000 7f5056db7740 1 --2- 172.21.15.83:0/2648399578 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x5636e32e3ad0 0x5636e32e5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:50.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.251+0000 7f5056db7740 4 client.21897 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:09:50.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.251+0000 7f5056db7740 1 -- 172.21.15.83:0/2648399578 --> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5636e31d4bf0 con 0x5636e32e3ad0 2024-04-24T09:09:50.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.252+0000 7f5050a00640 1 --2- 172.21.15.83:0/2648399578 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x5636e32e3ad0 0x5636e32e5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:50.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.253+0000 7f5050a00640 1 --2- 172.21.15.83:0/2648399578 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x5636e32e3ad0 0x5636e32e5ef0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6685ce241b2c78d server_cookie=23334a1febc23302 in_seq=0 out_seq=0 2024-04-24T09:09:50.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.253+0000 7f5032a00640 10 client.21897 ms_handle_connect on v2:172.21.15.189:6838/2548965336 2024-04-24T09:09:50.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.254+0000 7f5032a00640 1 -- 172.21.15.83:0/2648399578 <== mds.4 v2:172.21.15.189:6838/2548965336 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5636e31d4bf0 con 0x5636e32e3ad0 2024-04-24T09:09:50.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.254+0000 7f5032a00640 10 client.21897 handle_command_reply: tid=0 2024-04-24T09:09:50.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.254+0000 7f5056db7740 1 client.21897 shutdown 2024-04-24T09:09:50.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.254+0000 7f5056db7740 1 -- 172.21.15.83:0/2648399578 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636e326cb30 msgr2=0x5636e326e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:50.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.254+0000 7f5056db7740 1 --2- 172.21.15.83:0/2648399578 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636e326cb30 0x5636e326e2d0 secure :-1 s=READY pgs=2391 cs=0 l=1 rev1=1 crypto rx=0x7f5044010cc0 tx=0x7f5044009f50 comp rx=0 tx=0).stop 2024-04-24T09:09:50.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.254+0000 7f5056db7740 1 -- 172.21.15.83:0/2648399578 shutdown_connections 2024-04-24T09:09:50.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.254+0000 7f5056db7740 1 --2- 172.21.15.83:0/2648399578 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x5636e32e3ad0 0x5636e32e5ef0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:50.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.254+0000 7f5056db7740 1 --2- 172.21.15.83:0/2648399578 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636e326cb30 0x5636e326e2d0 unknown :-1 s=CLOSED pgs=2391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:50.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.254+0000 7f5056db7740 1 --2- 172.21.15.83:0/2648399578 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5636e3262460 0x5636e326dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:50.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.254+0000 7f5056db7740 1 --2- 172.21.15.83:0/2648399578 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5636e3261b40 0x5636e326d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:50.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.254+0000 7f5056db7740 1 -- 172.21.15.83:0/2648399578 >> 172.21.15.83:0/2648399578 conn(0x5636e325de60 msgr2=0x5636e325c6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:50.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.254+0000 7f5032a00640 0 client.21897 ms_handle_reset on v2:172.21.15.189:6838/2548965336 2024-04-24T09:09:50.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.255+0000 7f5056db7740 1 -- 172.21.15.83:0/2648399578 shutdown_connections 2024-04-24T09:09:50.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.255+0000 7f5056db7740 1 -- 172.21.15.83:0/2648399578 wait complete. 2024-04-24T09:09:50.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.255+0000 7f5056db7740 20 client.21897 trim_cache size 0 max 16384 2024-04-24T09:09:50.256 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:09:50.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.256+0000 7f5052800640 1 -- 172.21.15.83:0/1390738096 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f50340a3da0 msgr2=0x7f50340a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:50.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.256+0000 7f5052800640 1 --2- 172.21.15.83:0/1390738096 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f50340a3da0 0x7f50340a6260 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x5636e3263100 tx=0x7f5040031f10 comp rx=0 tx=0).stop 2024-04-24T09:09:50.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.256+0000 7f5052800640 1 -- 172.21.15.83:0/1390738096 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f504c0743f0 msgr2=0x7f504c1078b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:50.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.256+0000 7f5052800640 1 --2- 172.21.15.83:0/1390738096 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f504c0743f0 0x7f504c1078b0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f5040005ab0 tx=0x7f5040003c80 comp rx=0 tx=0).stop 2024-04-24T09:09:50.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.256+0000 7f5052800640 1 -- 172.21.15.83:0/1390738096 shutdown_connections 2024-04-24T09:09:50.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.256+0000 7f5052800640 1 --2- 172.21.15.83:0/1390738096 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f50340a3da0 0x7f50340a6260 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:50.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.256+0000 7f5052800640 1 --2- 172.21.15.83:0/1390738096 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f504c10e720 0x7f504c1029c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:50.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.256+0000 7f5052800640 1 --2- 172.21.15.83:0/1390738096 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f504c074d10 0x7f504c107df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:50.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.256+0000 7f5052800640 1 --2- 172.21.15.83:0/1390738096 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f504c0743f0 0x7f504c1078b0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:50.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.256+0000 7f5052800640 1 -- 172.21.15.83:0/1390738096 >> 172.21.15.83:0/1390738096 conn(0x7f504c0fd9d0 msgr2=0x7f504c0fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:50.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.256+0000 7f5052800640 1 -- 172.21.15.83:0/1390738096 shutdown_connections 2024-04-24T09:09:50.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.256+0000 7f5052800640 1 -- 172.21.15.83:0/1390738096 wait complete. 2024-04-24T09:09:50.275 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:09:49 smithi083 ceph-mon[27907]: pgmap v573: 129 pgs: 129 active+clean; 594 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.4 MiB/s wr, 306 op/s 2024-04-24T09:09:50.275 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:09:49 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/2430788928' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:09:50.278 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-24T09:09:50.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:09:49 smithi138 ceph-mon[31441]: pgmap v573: 129 pgs: 129 active+clean; 594 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.4 MiB/s wr, 306 op/s 2024-04-24T09:09:50.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:09:49 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/2430788928' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:09:50.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:09:49 smithi189 ceph-mon[31791]: pgmap v573: 129 pgs: 129 active+clean; 594 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.4 MiB/s wr, 306 op/s 2024-04-24T09:09:50.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:09:49 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/2430788928' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:09:50.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.530+0000 7fc65e800640 1 -- 172.21.15.83:0/2324526790 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc658102ac0 msgr2=0x7fc658106b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:50.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.530+0000 7fc65e800640 1 --2- 172.21.15.83:0/2324526790 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc658102ac0 0x7fc658106b00 secure :-1 s=READY pgs=2392 cs=0 l=1 rev1=1 crypto rx=0x7fc644009900 tx=0x7fc64402a410 comp rx=0 tx=0).stop 2024-04-24T09:09:50.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.530+0000 7fc65e800640 1 -- 172.21.15.83:0/2324526790 shutdown_connections 2024-04-24T09:09:50.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.530+0000 7fc65e800640 1 --2- 172.21.15.83:0/2324526790 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc6581071e0 0x7fc65810edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:50.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.530+0000 7fc65e800640 1 --2- 172.21.15.83:0/2324526790 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc658102ac0 0x7fc658106b00 unknown :-1 s=CLOSED pgs=2392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:50.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.530+0000 7fc65e800640 1 --2- 172.21.15.83:0/2324526790 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc658102110 0x7fc6581024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:50.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.530+0000 7fc65e800640 1 -- 172.21.15.83:0/2324526790 >> 172.21.15.83:0/2324526790 conn(0x7fc6580fd9d0 msgr2=0x7fc6580ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:50.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.530+0000 7fc65e800640 1 -- 172.21.15.83:0/2324526790 shutdown_connections 2024-04-24T09:09:50.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.531+0000 7fc65e800640 1 -- 172.21.15.83:0/2324526790 wait complete. 2024-04-24T09:09:50.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.532+0000 7fc65e800640 1 Processor -- start 2024-04-24T09:09:50.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.532+0000 7fc65e800640 1 -- start start 2024-04-24T09:09:50.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.533+0000 7fc65e800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc658102110 0x7fc6581a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:50.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.533+0000 7fc65e800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6581071e0 0x7fc6581a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:50.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.533+0000 7fc65e800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc65819a8c0 0x7fc65819ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:50.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.533+0000 7fc65e800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc658114470 con 0x7fc6581071e0 2024-04-24T09:09:50.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.533+0000 7fc65e800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc6581142f0 con 0x7fc658102110 2024-04-24T09:09:50.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.533+0000 7fc65e800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc6581145f0 con 0x7fc65819a8c0 2024-04-24T09:09:50.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.533+0000 7fc65d400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc658102110 0x7fc6581a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:50.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.533+0000 7fc65ca00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6581071e0 0x7fc6581a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:50.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.533+0000 7fc65ca00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6581071e0 0x7fc6581a0bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58296/0 (socket says 172.21.15.83:58296) 2024-04-24T09:09:50.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.533+0000 7fc65ca00640 1 -- 172.21.15.83:0/1289726962 learned_addr learned my addr 172.21.15.83:0/1289726962 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:09:50.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.533+0000 7fc65de00640 1 --2- 172.21.15.83:0/1289726962 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc65819a8c0 0x7fc65819ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:50.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.534+0000 7fc65ca00640 1 -- 172.21.15.83:0/1289726962 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc65819a8c0 msgr2=0x7fc65819ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:50.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.534+0000 7fc65ca00640 1 --2- 172.21.15.83:0/1289726962 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc65819a8c0 0x7fc65819ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:50.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.534+0000 7fc65ca00640 1 -- 172.21.15.83:0/1289726962 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc658102110 msgr2=0x7fc6581a06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:50.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.534+0000 7fc65ca00640 1 --2- 172.21.15.83:0/1289726962 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc658102110 0x7fc6581a06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:50.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.534+0000 7fc65ca00640 1 -- 172.21.15.83:0/1289726962 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc65819b450 con 0x7fc6581071e0 2024-04-24T09:09:50.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.534+0000 7fc65ca00640 1 --2- 172.21.15.83:0/1289726962 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6581071e0 0x7fc6581a0bf0 secure :-1 s=READY pgs=2393 cs=0 l=1 rev1=1 crypto rx=0x7fc644031440 tx=0x7fc644003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:50.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.535+0000 7fc656000640 1 -- 172.21.15.83:0/1289726962 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc64403f070 con 0x7fc6581071e0 2024-04-24T09:09:50.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.535+0000 7fc656000640 1 -- 172.21.15.83:0/1289726962 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc644031d60 con 0x7fc6581071e0 2024-04-24T09:09:50.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.535+0000 7fc65e800640 1 -- 172.21.15.83:0/1289726962 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc65819b6e0 con 0x7fc6581071e0 2024-04-24T09:09:50.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.535+0000 7fc65e800640 1 -- 172.21.15.83:0/1289726962 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6581a7820 con 0x7fc6581071e0 2024-04-24T09:09:50.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.535+0000 7fc656000640 1 -- 172.21.15.83:0/1289726962 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc644035030 con 0x7fc6581071e0 2024-04-24T09:09:50.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.540+0000 7fc662c4e740 1 Processor -- start 2024-04-24T09:09:50.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.540+0000 7fc662c4e740 1 -- start start 2024-04-24T09:09:50.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.540+0000 7fc656000640 1 -- 172.21.15.83:0/1289726962 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc644042070 con 0x7fc6581071e0 2024-04-24T09:09:50.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.540+0000 7fc662c4e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5566986f5350 0x5566986f5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:50.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.540+0000 7fc662c4e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5566986f5c70 0x5566986ffe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:50.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.540+0000 7fc662c4e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556698700340 0x556698702730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:50.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.540+0000 7fc662c4e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5566986521d0 con 0x5566986f5350 2024-04-24T09:09:50.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.540+0000 7fc662c4e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5566985f8450 con 0x556698700340 2024-04-24T09:09:50.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.540+0000 7fc662c4e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55669864e190 con 0x5566986f5c70 2024-04-24T09:09:50.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.541+0000 7fc65ca00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556698700340 0x556698702730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:50.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.541+0000 7fc65ca00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556698700340 0x556698702730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:50254/0 (socket says 172.21.15.83:50254) 2024-04-24T09:09:50.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.541+0000 7fc65ca00640 1 -- 172.21.15.83:0/3946423132 learned_addr learned my addr 172.21.15.83:0/3946423132 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:09:50.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.541+0000 7fc65de00640 1 --2- 172.21.15.83:0/3946423132 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5566986f5350 0x5566986f5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:50.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.541+0000 7fc65d400640 1 --2- 172.21.15.83:0/3946423132 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5566986f5c70 0x5566986ffe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:50.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.541+0000 7fc65ca00640 1 -- 172.21.15.83:0/3946423132 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5566986f5c70 msgr2=0x5566986ffe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:50.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.541+0000 7fc65ca00640 1 --2- 172.21.15.83:0/3946423132 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5566986f5c70 0x5566986ffe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:50.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.541+0000 7fc65ca00640 1 -- 172.21.15.83:0/3946423132 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5566986f5350 msgr2=0x5566986f5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:50.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.541+0000 7fc65ca00640 1 --2- 172.21.15.83:0/3946423132 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5566986f5350 0x5566986f5730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:50.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.541+0000 7fc65ca00640 1 -- 172.21.15.83:0/3946423132 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566986b5310 con 0x556698700340 2024-04-24T09:09:50.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.542+0000 7fc65ca00640 1 --2- 172.21.15.83:0/3946423132 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556698700340 0x556698702730 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc6440ee780 tx=0x7fc6440f7790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29313561168abe97 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:50.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.542+0000 7fc656000640 1 --2- 172.21.15.83:0/1289726962 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fc6400a3da0 0x7fc6400a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:50.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.542+0000 7fc64a600640 1 -- 172.21.15.83:0/3946423132 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6440b9300 con 0x556698700340 2024-04-24T09:09:50.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.542+0000 7fc64a600640 1 -- 172.21.15.83:0/3946423132 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc6440fe030 con 0x556698700340 2024-04-24T09:09:50.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.542+0000 7fc64a600640 1 -- 172.21.15.83:0/3946423132 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6440bc4b0 con 0x556698700340 2024-04-24T09:09:50.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.542+0000 7fc656000640 1 -- 172.21.15.83:0/1289726962 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 9583+0+0 (secure 0 0 0) 0x7fc6440f14e0 con 0x7fc6581071e0 2024-04-24T09:09:50.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.543+0000 7fc65de00640 1 --2- 172.21.15.83:0/1289726962 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fc6400a3da0 0x7fc6400a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:50.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.544+0000 7fc662c4e740 1 -- 172.21.15.83:0/3946423132 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556698700340 msgr2=0x556698702730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:50.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.544+0000 7fc662c4e740 1 --2- 172.21.15.83:0/3946423132 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556698700340 0x556698702730 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc6440ee780 tx=0x7fc6440f7790 comp rx=0 tx=0).stop 2024-04-24T09:09:50.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.544+0000 7fc65de00640 1 --2- 172.21.15.83:0/1289726962 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fc6400a3da0 0x7fc6400a6260 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fc650009fd0 tx=0x7fc65000e040 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:50.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.544+0000 7fc662c4e740 1 -- 172.21.15.83:0/3946423132 shutdown_connections 2024-04-24T09:09:50.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.544+0000 7fc662c4e740 1 --2- 172.21.15.83:0/3946423132 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556698700340 0x556698702730 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:50.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.544+0000 7fc662c4e740 1 --2- 172.21.15.83:0/3946423132 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5566986f5c70 0x5566986ffe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:50.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.544+0000 7fc662c4e740 1 --2- 172.21.15.83:0/3946423132 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5566986f5350 0x5566986f5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:50.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.544+0000 7fc662c4e740 1 -- 172.21.15.83:0/3946423132 >> 172.21.15.83:0/3946423132 conn(0x5566986f1670 msgr2=0x5566986f1a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:50.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.544+0000 7fc662c4e740 1 -- 172.21.15.83:0/3946423132 shutdown_connections 2024-04-24T09:09:50.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.544+0000 7fc662c4e740 1 -- 172.21.15.83:0/3946423132 wait complete. 2024-04-24T09:09:50.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.545+0000 7fc662c4e740 1 -- start start 2024-04-24T09:09:50.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.545+0000 7fc662c4e740 1 Processor -- start 2024-04-24T09:09:50.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.546+0000 7fc662c4e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5566986f5350 0x5566986ffd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:50.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.546+0000 7fc662c4e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5566986f5c70 0x5566986fd000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:50.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.546+0000 7fc662c4e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556698700340 0x5566986fd610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:50.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.546+0000 7fc662c4e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556698702ef0 con 0x5566986f5350 2024-04-24T09:09:50.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.546+0000 7fc662c4e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55669860b810 con 0x5566986f5c70 2024-04-24T09:09:50.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.546+0000 7fc662c4e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x556698703070 con 0x556698700340 2024-04-24T09:09:50.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.546+0000 7fc662c4e740 10 client.0 _ll_register_callbacks cb 0x5566985d9140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:09:50.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.547+0000 7fc65d400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5566986f5350 0x5566986ffd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:50.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.547+0000 7fc65d400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5566986f5350 0x5566986ffd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58312/0 (socket says 172.21.15.83:58312) 2024-04-24T09:09:50.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.547+0000 7fc65de00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556698700340 0x5566986fd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:50.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.547+0000 7fc65d400640 1 -- 172.21.15.83:0/1945569035 learned_addr learned my addr 172.21.15.83:0/1945569035 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:09:50.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.547+0000 7fc65ca00640 1 --2- 172.21.15.83:0/1945569035 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5566986f5c70 0x5566986fd000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:50.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.548+0000 7fc65d400640 1 -- 172.21.15.83:0/1945569035 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556698700340 msgr2=0x5566986fd610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:50.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.548+0000 7fc65d400640 1 --2- 172.21.15.83:0/1945569035 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556698700340 0x5566986fd610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:50.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.548+0000 7fc65d400640 1 -- 172.21.15.83:0/1945569035 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5566986f5c70 msgr2=0x5566986fd000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:50.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.548+0000 7fc65d400640 1 --2- 172.21.15.83:0/1945569035 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5566986f5c70 0x5566986fd000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:50.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.548+0000 7fc65d400640 1 -- 172.21.15.83:0/1945569035 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55669861c3f0 con 0x5566986f5350 2024-04-24T09:09:50.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.548+0000 7fc65de00640 1 --2- 172.21.15.83:0/1945569035 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556698700340 0x5566986fd610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:09:50.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.549+0000 7fc65d400640 1 --2- 172.21.15.83:0/1945569035 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5566986f5350 0x5566986ffd10 secure :-1 s=READY pgs=2394 cs=0 l=1 rev1=1 crypto rx=0x7fc64c00bfb0 tx=0x7fc64c00f670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9aa79ceff9c4978e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:50.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.549+0000 7fc63ea00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:09:50.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.549+0000 7fc63ea00640 1 -- 172.21.15.83:0/1945569035 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc64c018020 con 0x5566986f5350 2024-04-24T09:09:50.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.549+0000 7fc63ea00640 1 -- 172.21.15.83:0/1945569035 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9583+0+0 (secure 0 0 0) 0x7fc64c014870 con 0x5566986f5350 2024-04-24T09:09:50.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.549+0000 7fc662c4e740 1 -- 172.21.15.83:0/1945569035 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6440fe030 con 0x5566986f5350 2024-04-24T09:09:50.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.549+0000 7fc63ea00640 1 -- 172.21.15.83:0/1945569035 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7fc64c023050 con 0x5566986f5350 2024-04-24T09:09:50.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.550+0000 7fc662c4e740 10 client.21909 fetch_fsmap learned FSMap version 61 2024-04-24T09:09:50.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.550+0000 7fc662c4e740 1 -- 172.21.15.83:0/1945569035 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x5566983643f0 con 0x5566986f5350 2024-04-24T09:09:50.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.550+0000 7fc63ea00640 1 -- 172.21.15.83:0/1945569035 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc6440fe030 con 0x5566986f5350 2024-04-24T09:09:50.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.550+0000 7fc63ea00640 1 -- 172.21.15.83:0/1945569035 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 61) ==== 8362+0+0 (secure 0 0 0) 0x7fc64c013ad0 con 0x5566986f5350 2024-04-24T09:09:50.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.551+0000 7fc662c4e740 10 client.21909 fetch_fsmap finished waiting for FSMap version 61 2024-04-24T09:09:50.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.551+0000 7fc662c4e740 10 client.21909 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-24T09:09:50.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.551+0000 7fc662c4e740 20 client.21909 populate_metadata read hostname 'smithi083' 2024-04-24T09:09:50.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.551+0000 7fc662c4e740 1 --2- 172.21.15.83:0/1945569035 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x556698710260 0x5566987126d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:50.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.551+0000 7fc662c4e740 4 client.21909 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:09:50.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.551+0000 7fc662c4e740 1 -- 172.21.15.83:0/1945569035 --> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55669865fbf0 con 0x556698710260 2024-04-24T09:09:50.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.551+0000 7fc65ca00640 1 --2- 172.21.15.83:0/1945569035 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x556698710260 0x5566987126d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:50.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.552+0000 7fc63ea00640 10 client.21909 ms_handle_connect on v2:172.21.15.138:6832/1157967658 2024-04-24T09:09:50.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.552+0000 7fc65ca00640 1 --2- 172.21.15.83:0/1945569035 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x556698710260 0x5566987126d0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d73820ed6f167a67 server_cookie=cba8c657bc4b7d2f in_seq=0 out_seq=0 2024-04-24T09:09:50.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.553+0000 7fc63ea00640 1 -- 172.21.15.83:0/1945569035 <== mds.3 v2:172.21.15.138:6832/1157967658 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55669865fbf0 con 0x556698710260 2024-04-24T09:09:50.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.553+0000 7fc63ea00640 10 client.21909 handle_command_reply: tid=0 2024-04-24T09:09:50.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.553+0000 7fc662c4e740 1 client.21909 shutdown 2024-04-24T09:09:50.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.553+0000 7fc662c4e740 1 -- 172.21.15.83:0/1945569035 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5566986f5350 msgr2=0x5566986ffd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:50.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.553+0000 7fc662c4e740 1 --2- 172.21.15.83:0/1945569035 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5566986f5350 0x5566986ffd10 secure :-1 s=READY pgs=2394 cs=0 l=1 rev1=1 crypto rx=0x7fc64c00bfb0 tx=0x7fc64c00f670 comp rx=0 tx=0).stop 2024-04-24T09:09:50.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.554+0000 7fc662c4e740 1 -- 172.21.15.83:0/1945569035 shutdown_connections 2024-04-24T09:09:50.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.554+0000 7fc662c4e740 1 --2- 172.21.15.83:0/1945569035 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x556698710260 0x5566987126d0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:50.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.554+0000 7fc662c4e740 1 --2- 172.21.15.83:0/1945569035 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556698700340 0x5566986fd610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:50.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.554+0000 7fc662c4e740 1 --2- 172.21.15.83:0/1945569035 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5566986f5c70 0x5566986fd000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:50.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.554+0000 7fc63ea00640 0 client.21909 ms_handle_reset on v2:172.21.15.138:6832/1157967658 2024-04-24T09:09:50.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.554+0000 7fc662c4e740 1 --2- 172.21.15.83:0/1945569035 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5566986f5350 0x5566986ffd10 unknown :-1 s=CLOSED pgs=2394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:50.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.554+0000 7fc662c4e740 1 -- 172.21.15.83:0/1945569035 >> 172.21.15.83:0/1945569035 conn(0x5566986f1670 msgr2=0x5566986f7470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:50.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.554+0000 7fc662c4e740 1 -- 172.21.15.83:0/1945569035 shutdown_connections 2024-04-24T09:09:50.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.554+0000 7fc662c4e740 1 -- 172.21.15.83:0/1945569035 wait complete. 2024-04-24T09:09:50.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.554+0000 7fc662c4e740 20 client.21909 trim_cache size 0 max 16384 2024-04-24T09:09:50.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.579+0000 7fc662c4e740 1 Processor -- start 2024-04-24T09:09:50.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.579+0000 7fc662c4e740 1 -- start start 2024-04-24T09:09:50.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.579+0000 7fc662c4e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5566986f5350 0x55669870bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:50.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.579+0000 7fc662c4e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5566986f5c70 0x5566986fbfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:50.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.579+0000 7fc662c4e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556698700340 0x5566986ff840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:50.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.580+0000 7fc662c4e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc6581ad2d0 con 0x5566986f5c70 2024-04-24T09:09:50.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.580+0000 7fc65d400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5566986f5350 0x55669870bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:50.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.580+0000 7fc65ca00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5566986f5c70 0x5566986fbfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:50.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.580+0000 7fc65d400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5566986f5350 0x55669870bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:50268/0 (socket says 172.21.15.83:50268) 2024-04-24T09:09:50.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.580+0000 7fc65d400640 1 -- 172.21.15.83:0/498637835 learned_addr learned my addr 172.21.15.83:0/498637835 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:09:50.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.580+0000 7fc662c4e740 1 -- 172.21.15.83:0/498637835 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc6581aee10 con 0x5566986f5350 2024-04-24T09:09:50.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.580+0000 7fc662c4e740 1 -- 172.21.15.83:0/498637835 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc6581ad9a0 con 0x556698700340 2024-04-24T09:09:50.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.580+0000 7fc65de00640 1 --2- 172.21.15.83:0/498637835 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556698700340 0x5566986ff840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:50.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.581+0000 7fc65ca00640 1 -- 172.21.15.83:0/498637835 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556698700340 msgr2=0x5566986ff840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:50.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.581+0000 7fc65ca00640 1 --2- 172.21.15.83:0/498637835 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556698700340 0x5566986ff840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:50.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.581+0000 7fc65ca00640 1 -- 172.21.15.83:0/498637835 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5566986f5350 msgr2=0x55669870bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:50.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.581+0000 7fc65ca00640 1 --2- 172.21.15.83:0/498637835 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5566986f5350 0x55669870bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:50.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.581+0000 7fc65ca00640 1 -- 172.21.15.83:0/498637835 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55669842e7d0 con 0x5566986f5c70 2024-04-24T09:09:50.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.581+0000 7fc65d400640 1 --2- 172.21.15.83:0/498637835 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5566986f5350 0x55669870bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:09:50.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.582+0000 7fc65ca00640 1 --2- 172.21.15.83:0/498637835 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5566986f5c70 0x5566986fbfe0 secure :-1 s=READY pgs=2395 cs=0 l=1 rev1=1 crypto rx=0x7fc64408e3e0 tx=0x7fc64402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ab70c2384216278 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:50.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.582+0000 7fc649c00640 1 -- 172.21.15.83:0/498637835 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6440bc7e0 con 0x5566986f5c70 2024-04-24T09:09:50.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.582+0000 7fc649c00640 1 -- 172.21.15.83:0/498637835 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc6440fe040 con 0x5566986f5c70 2024-04-24T09:09:50.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.583+0000 7fc649c00640 1 -- 172.21.15.83:0/498637835 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6440bc9c0 con 0x5566986f5c70 2024-04-24T09:09:50.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.583+0000 7fc662c4e740 1 -- 172.21.15.83:0/498637835 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5566986f5c70 msgr2=0x5566986fbfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:50.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.584+0000 7fc662c4e740 1 --2- 172.21.15.83:0/498637835 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5566986f5c70 0x5566986fbfe0 secure :-1 s=READY pgs=2395 cs=0 l=1 rev1=1 crypto rx=0x7fc64408e3e0 tx=0x7fc64402a410 comp rx=0 tx=0).stop 2024-04-24T09:09:50.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.584+0000 7fc662c4e740 1 -- 172.21.15.83:0/498637835 shutdown_connections 2024-04-24T09:09:50.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.584+0000 7fc662c4e740 1 --2- 172.21.15.83:0/498637835 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556698700340 0x5566986ff840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:50.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.584+0000 7fc662c4e740 1 --2- 172.21.15.83:0/498637835 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5566986f5c70 0x5566986fbfe0 unknown :-1 s=CLOSED pgs=2395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:50.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.584+0000 7fc662c4e740 1 --2- 172.21.15.83:0/498637835 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5566986f5350 0x55669870bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:50.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.584+0000 7fc662c4e740 1 -- 172.21.15.83:0/498637835 >> 172.21.15.83:0/498637835 conn(0x5566986f1670 msgr2=0x5566986fb8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:50.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.584+0000 7fc662c4e740 1 -- 172.21.15.83:0/498637835 shutdown_connections 2024-04-24T09:09:50.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.584+0000 7fc662c4e740 1 -- 172.21.15.83:0/498637835 wait complete. 2024-04-24T09:09:50.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.584+0000 7fc662c4e740 1 -- start start 2024-04-24T09:09:50.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.585+0000 7fc662c4e740 1 Processor -- start 2024-04-24T09:09:50.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.585+0000 7fc662c4e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5566986f5350 0x5566987084b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:50.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.585+0000 7fc662c4e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5566986f5c70 0x5566986fa280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:50.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.585+0000 7fc662c4e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556698700340 0x5566986fa7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:50.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.585+0000 7fc662c4e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc6581ae740 con 0x556698700340 2024-04-24T09:09:50.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.585+0000 7fc662c4e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc6581ae070 con 0x5566986f5c70 2024-04-24T09:09:50.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.585+0000 7fc662c4e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc6581ab2d0 con 0x5566986f5350 2024-04-24T09:09:50.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.586+0000 7fc662c4e740 10 client.0 _ll_register_callbacks cb 0x7fc64c00b2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:09:50.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.586+0000 7fc65ca00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5566986f5c70 0x5566986fa280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:50.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.586+0000 7fc65ca00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5566986f5c70 0x5566986fa280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:50274/0 (socket says 172.21.15.83:50274) 2024-04-24T09:09:50.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.586+0000 7fc65ca00640 1 -- 172.21.15.83:0/3092756307 learned_addr learned my addr 172.21.15.83:0/3092756307 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:09:50.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.586+0000 7fc65de00640 1 --2- 172.21.15.83:0/3092756307 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556698700340 0x5566986fa7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:50.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.586+0000 7fc65d400640 1 --2- 172.21.15.83:0/3092756307 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5566986f5350 0x5566987084b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:50.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.587+0000 7fc65de00640 1 -- 172.21.15.83:0/3092756307 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5566986f5350 msgr2=0x5566987084b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:50.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.587+0000 7fc65de00640 1 --2- 172.21.15.83:0/3092756307 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5566986f5350 0x5566987084b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:50.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.587+0000 7fc65de00640 1 -- 172.21.15.83:0/3092756307 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5566986f5c70 msgr2=0x5566986fa280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:50.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.587+0000 7fc65de00640 1 --2- 172.21.15.83:0/3092756307 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5566986f5c70 0x5566986fa280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:50.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.587+0000 7fc65de00640 1 -- 172.21.15.83:0/3092756307 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566986fad30 con 0x556698700340 2024-04-24T09:09:50.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.587+0000 7fc65d400640 1 --2- 172.21.15.83:0/3092756307 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5566986f5350 0x5566987084b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:09:50.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.587+0000 7fc65de00640 1 --2- 172.21.15.83:0/3092756307 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556698700340 0x5566986fa7c0 secure :-1 s=READY pgs=2396 cs=0 l=1 rev1=1 crypto rx=0x7fc650009520 tx=0x7fc650004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82532620b88df81e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:50.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.587+0000 7fc63ea00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:09:50.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.588+0000 7fc63ea00640 1 -- 172.21.15.83:0/3092756307 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc650029020 con 0x556698700340 2024-04-24T09:09:50.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.588+0000 7fc63ea00640 1 -- 172.21.15.83:0/3092756307 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9583+0+0 (secure 0 0 0) 0x7fc650014c20 con 0x556698700340 2024-04-24T09:09:50.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.588+0000 7fc662c4e740 1 -- 172.21.15.83:0/3092756307 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6440fe040 con 0x556698700340 2024-04-24T09:09:50.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.588+0000 7fc63ea00640 1 -- 172.21.15.83:0/3092756307 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7fc650034050 con 0x556698700340 2024-04-24T09:09:50.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.588+0000 7fc662c4e740 10 client.21915 fetch_fsmap learned FSMap version 61 2024-04-24T09:09:50.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.588+0000 7fc662c4e740 1 -- 172.21.15.83:0/3092756307 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x556698701950 con 0x556698700340 2024-04-24T09:09:50.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.588+0000 7fc63ea00640 1 -- 172.21.15.83:0/3092756307 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc6440fe040 con 0x556698700340 2024-04-24T09:09:50.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.589+0000 7fc63ea00640 1 -- 172.21.15.83:0/3092756307 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 61) ==== 8362+0+0 (secure 0 0 0) 0x7fc650016070 con 0x556698700340 2024-04-24T09:09:50.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.589+0000 7fc662c4e740 10 client.21915 fetch_fsmap finished waiting for FSMap version 61 2024-04-24T09:09:50.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.589+0000 7fc662c4e740 10 client.21915 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-24T09:09:50.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.589+0000 7fc662c4e740 20 client.21915 populate_metadata read hostname 'smithi083' 2024-04-24T09:09:50.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.589+0000 7fc662c4e740 1 --2- 172.21.15.83:0/3092756307 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x556698776e10 0x556698779230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:50.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.589+0000 7fc662c4e740 4 client.21915 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:09:50.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.589+0000 7fc662c4e740 1 -- 172.21.15.83:0/3092756307 --> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556698659a60 con 0x556698776e10 2024-04-24T09:09:50.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.589+0000 7fc65d400640 1 --2- 172.21.15.83:0/3092756307 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x556698776e10 0x556698779230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:50.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.590+0000 7fc65d400640 1 --2- 172.21.15.83:0/3092756307 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x556698776e10 0x556698779230 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=802bebbf865772df server_cookie=dd589d77d4356039 in_seq=0 out_seq=0 2024-04-24T09:09:50.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.590+0000 7fc63ea00640 10 client.21915 ms_handle_connect on v2:172.21.15.138:6832/1157967658 2024-04-24T09:09:50.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.591+0000 7fc63ea00640 1 -- 172.21.15.83:0/3092756307 <== mds.3 v2:172.21.15.138:6832/1157967658 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556698659a60 con 0x556698776e10 2024-04-24T09:09:50.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.591+0000 7fc63ea00640 10 client.21915 handle_command_reply: tid=0 2024-04-24T09:09:50.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.591+0000 7fc662c4e740 1 client.21915 shutdown 2024-04-24T09:09:50.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.591+0000 7fc662c4e740 1 -- 172.21.15.83:0/3092756307 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556698700340 msgr2=0x5566986fa7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:50.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.591+0000 7fc662c4e740 1 --2- 172.21.15.83:0/3092756307 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556698700340 0x5566986fa7c0 secure :-1 s=READY pgs=2396 cs=0 l=1 rev1=1 crypto rx=0x7fc650009520 tx=0x7fc650004a40 comp rx=0 tx=0).stop 2024-04-24T09:09:50.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.591+0000 7fc662c4e740 1 -- 172.21.15.83:0/3092756307 shutdown_connections 2024-04-24T09:09:50.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.591+0000 7fc662c4e740 1 --2- 172.21.15.83:0/3092756307 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556698700340 0x5566986fa7c0 unknown :-1 s=CLOSED pgs=2396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:50.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.591+0000 7fc662c4e740 1 --2- 172.21.15.83:0/3092756307 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5566986f5c70 0x5566986fa280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:50.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.591+0000 7fc662c4e740 1 --2- 172.21.15.83:0/3092756307 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x556698776e10 0x556698779230 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:50.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.591+0000 7fc662c4e740 1 --2- 172.21.15.83:0/3092756307 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5566986f5350 0x5566987084b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:50.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.591+0000 7fc662c4e740 1 -- 172.21.15.83:0/3092756307 >> 172.21.15.83:0/3092756307 conn(0x5566986f1670 msgr2=0x5566986fc160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:50.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.591+0000 7fc63ea00640 0 client.21915 ms_handle_reset on v2:172.21.15.138:6832/1157967658 2024-04-24T09:09:50.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.591+0000 7fc662c4e740 1 -- 172.21.15.83:0/3092756307 shutdown_connections 2024-04-24T09:09:50.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.592+0000 7fc662c4e740 1 -- 172.21.15.83:0/3092756307 wait complete. 2024-04-24T09:09:50.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.592+0000 7fc662c4e740 20 client.21915 trim_cache size 0 max 16384 2024-04-24T09:09:50.593 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:09:50.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.593+0000 7fc65e800640 1 -- 172.21.15.83:0/1289726962 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fc6400a3da0 msgr2=0x7fc6400a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:50.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.593+0000 7fc65e800640 1 --2- 172.21.15.83:0/1289726962 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fc6400a3da0 0x7fc6400a6260 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fc650009fd0 tx=0x7fc65000e040 comp rx=0 tx=0).stop 2024-04-24T09:09:50.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.593+0000 7fc65e800640 1 -- 172.21.15.83:0/1289726962 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6581071e0 msgr2=0x7fc6581a0bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:50.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.593+0000 7fc65e800640 1 --2- 172.21.15.83:0/1289726962 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6581071e0 0x7fc6581a0bf0 secure :-1 s=READY pgs=2393 cs=0 l=1 rev1=1 crypto rx=0x7fc644031440 tx=0x7fc644003d50 comp rx=0 tx=0).stop 2024-04-24T09:09:50.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.593+0000 7fc65e800640 1 -- 172.21.15.83:0/1289726962 shutdown_connections 2024-04-24T09:09:50.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.593+0000 7fc65e800640 1 --2- 172.21.15.83:0/1289726962 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fc6400a3da0 0x7fc6400a6260 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:50.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.593+0000 7fc65e800640 1 --2- 172.21.15.83:0/1289726962 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc65819a8c0 0x7fc65819ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:50.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.593+0000 7fc65e800640 1 --2- 172.21.15.83:0/1289726962 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6581071e0 0x7fc6581a0bf0 unknown :-1 s=CLOSED pgs=2393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:50.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.593+0000 7fc65e800640 1 --2- 172.21.15.83:0/1289726962 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc658102110 0x7fc6581a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:50.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.593+0000 7fc65e800640 1 -- 172.21.15.83:0/1289726962 >> 172.21.15.83:0/1289726962 conn(0x7fc6580fd9d0 msgr2=0x7fc6580fede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:50.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.593+0000 7fc65e800640 1 -- 172.21.15.83:0/1289726962 shutdown_connections 2024-04-24T09:09:50.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.593+0000 7fc65e800640 1 -- 172.21.15.83:0/1289726962 wait complete. 2024-04-24T09:09:50.615 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T09:09:50.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.874+0000 7fd187200640 1 -- 172.21.15.83:0/2477251376 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd180103980 msgr2=0x7fd18010be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:50.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.874+0000 7fd187200640 1 --2- 172.21.15.83:0/2477251376 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd180103980 0x7fd18010be60 secure :-1 s=READY pgs=2397 cs=0 l=1 rev1=1 crypto rx=0x7fd170009900 tx=0x7fd17002a490 comp rx=0 tx=0).stop 2024-04-24T09:09:50.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.876+0000 7fd187200640 1 -- 172.21.15.83:0/2477251376 shutdown_connections 2024-04-24T09:09:50.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.876+0000 7fd187200640 1 --2- 172.21.15.83:0/2477251376 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd18010c480 0x7fd18010e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:50.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.876+0000 7fd187200640 1 --2- 172.21.15.83:0/2477251376 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd180103980 0x7fd18010be60 unknown :-1 s=CLOSED pgs=2397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:50.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.876+0000 7fd187200640 1 --2- 172.21.15.83:0/2477251376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd180103060 0x7fd180103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:50.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.876+0000 7fd187200640 1 -- 172.21.15.83:0/2477251376 >> 172.21.15.83:0/2477251376 conn(0x7fd1800fc880 msgr2=0x7fd1800feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:50.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.877+0000 7fd187200640 1 -- 172.21.15.83:0/2477251376 shutdown_connections 2024-04-24T09:09:50.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.877+0000 7fd187200640 1 -- 172.21.15.83:0/2477251376 wait complete. 2024-04-24T09:09:50.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.879+0000 7fd187200640 1 Processor -- start 2024-04-24T09:09:50.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.880+0000 7fd187200640 1 -- start start 2024-04-24T09:09:50.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.880+0000 7fd187200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd180103060 0x7fd18010b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:50.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.880+0000 7fd187200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd180103980 0x7fd180108ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:50.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.881+0000 7fd187200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd18010c480 0x7fd180108ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:50.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.881+0000 7fd187200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd1801022b0 con 0x7fd18010c480 2024-04-24T09:09:50.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.881+0000 7fd187200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd180102130 con 0x7fd180103980 2024-04-24T09:09:50.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.881+0000 7fd187200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd180102430 con 0x7fd180103060 2024-04-24T09:09:50.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.881+0000 7fd186800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd18010c480 0x7fd180108ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:50.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.881+0000 7fd185e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd180103060 0x7fd18010b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:50.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.881+0000 7fd185400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd180103980 0x7fd180108ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:50.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.881+0000 7fd185e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd180103060 0x7fd18010b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:41806/0 (socket says 172.21.15.83:41806) 2024-04-24T09:09:50.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.881+0000 7fd185e00640 1 -- 172.21.15.83:0/1164139926 learned_addr learned my addr 172.21.15.83:0/1164139926 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:09:50.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.882+0000 7fd186800640 1 -- 172.21.15.83:0/1164139926 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd180103060 msgr2=0x7fd18010b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:50.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.882+0000 7fd186800640 1 --2- 172.21.15.83:0/1164139926 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd180103060 0x7fd18010b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:50.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.882+0000 7fd186800640 1 -- 172.21.15.83:0/1164139926 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd180103980 msgr2=0x7fd180108ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:50.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.882+0000 7fd186800640 1 --2- 172.21.15.83:0/1164139926 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd180103980 0x7fd180108ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:50.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.883+0000 7fd186800640 1 -- 172.21.15.83:0/1164139926 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1801095e0 con 0x7fd18010c480 2024-04-24T09:09:50.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.883+0000 7fd185e00640 1 --2- 172.21.15.83:0/1164139926 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd180103060 0x7fd18010b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:09:50.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.883+0000 7fd185400640 1 --2- 172.21.15.83:0/1164139926 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd180103980 0x7fd180108ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:09:50.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.883+0000 7fd186800640 1 --2- 172.21.15.83:0/1164139926 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd18010c480 0x7fd180108ff0 secure :-1 s=READY pgs=2398 cs=0 l=1 rev1=1 crypto rx=0x7fd17c00e940 tx=0x7fd17c00ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:50.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.884+0000 7fd17aa00640 1 -- 172.21.15.83:0/1164139926 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd17c004d20 con 0x7fd18010c480 2024-04-24T09:09:50.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.884+0000 7fd17aa00640 1 -- 172.21.15.83:0/1164139926 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd17c005020 con 0x7fd18010c480 2024-04-24T09:09:50.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.884+0000 7fd17aa00640 1 -- 172.21.15.83:0/1164139926 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd17c0116d0 con 0x7fd18010c480 2024-04-24T09:09:50.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.884+0000 7fd187200640 1 -- 172.21.15.83:0/1164139926 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1801098d0 con 0x7fd18010c480 2024-04-24T09:09:50.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.885+0000 7fd187200640 1 -- 172.21.15.83:0/1164139926 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1801ab700 con 0x7fd18010c480 2024-04-24T09:09:50.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.893+0000 7fd17aa00640 1 -- 172.21.15.83:0/1164139926 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd17c00cb20 con 0x7fd18010c480 2024-04-24T09:09:50.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.893+0000 7fd18b5a1740 1 Processor -- start 2024-04-24T09:09:50.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.893+0000 7fd18b5a1740 1 -- start start 2024-04-24T09:09:50.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.894+0000 7fd18b5a1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564aab5aeb40 0x564aab5aef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:50.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.894+0000 7fd18b5a1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564aab5af460 0x564aab5b95f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:50.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.894+0000 7fd18b5a1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564aab5b9b30 0x564aab5bbf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:50.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.894+0000 7fd18b5a1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x564aab5141d0 con 0x564aab5af460 2024-04-24T09:09:50.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.894+0000 7fd18b5a1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x564aab4ba450 con 0x564aab5b9b30 2024-04-24T09:09:50.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.894+0000 7fd18b5a1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x564aab510190 con 0x564aab5aeb40 2024-04-24T09:09:50.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.894+0000 7fd185e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564aab5af460 0x564aab5b95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:50.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.894+0000 7fd185e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564aab5af460 0x564aab5b95f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58348/0 (socket says 172.21.15.83:58348) 2024-04-24T09:09:50.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.894+0000 7fd185e00640 1 -- 172.21.15.83:0/2922695529 learned_addr learned my addr 172.21.15.83:0/2922695529 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:09:50.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.894+0000 7fd185400640 1 --2- 172.21.15.83:0/2922695529 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564aab5aeb40 0x564aab5aef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:50.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.894+0000 7fd185400640 1 --2- 172.21.15.83:0/2922695529 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564aab5b9b30 0x564aab5bbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:50.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.895+0000 7fd185e00640 1 -- 172.21.15.83:0/2922695529 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564aab5aeb40 msgr2=0x564aab5aef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:50.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.895+0000 7fd185e00640 1 --2- 172.21.15.83:0/2922695529 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564aab5aeb40 0x564aab5aef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:50.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.895+0000 7fd185e00640 1 -- 172.21.15.83:0/2922695529 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564aab5b9b30 msgr2=0x564aab5bbf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:50.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.895+0000 7fd185e00640 1 --2- 172.21.15.83:0/2922695529 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564aab5b9b30 0x564aab5bbf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:50.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.895+0000 7fd185e00640 1 -- 172.21.15.83:0/2922695529 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564aab577310 con 0x564aab5af460 2024-04-24T09:09:50.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.895+0000 7fd185400640 1 --2- 172.21.15.83:0/2922695529 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564aab5aeb40 0x564aab5aef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:09:50.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.895+0000 7fd17aa00640 1 --2- 172.21.15.83:0/1164139926 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fd1680a3f20 0x7fd1680a63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:50.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.895+0000 7fd185e00640 1 --2- 172.21.15.83:0/2922695529 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564aab5af460 0x564aab5b95f0 secure :-1 s=READY pgs=2399 cs=0 l=1 rev1=1 crypto rx=0x7fd17400f4f0 tx=0x7fd17400f9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32e4f4dbdf7c1319 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:50.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.895+0000 7fd17aa00640 1 -- 172.21.15.83:0/1164139926 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 9583+0+0 (secure 0 0 0) 0x7fd17c0d0b60 con 0x7fd18010c480 2024-04-24T09:09:50.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.896+0000 7fd185400640 1 --2- 172.21.15.83:0/1164139926 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fd1680a3f20 0x7fd1680a63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:50.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.896+0000 7fd16f000640 1 -- 172.21.15.83:0/2922695529 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd174005220 con 0x564aab5af460 2024-04-24T09:09:50.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.896+0000 7fd16f000640 1 -- 172.21.15.83:0/2922695529 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd1740047a0 con 0x564aab5af460 2024-04-24T09:09:50.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.896+0000 7fd16f000640 1 -- 172.21.15.83:0/2922695529 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd174010c60 con 0x564aab5af460 2024-04-24T09:09:50.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.896+0000 7fd18b5a1740 1 -- 172.21.15.83:0/2922695529 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564aab5af460 msgr2=0x564aab5b95f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:50.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.896+0000 7fd185400640 1 --2- 172.21.15.83:0/1164139926 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fd1680a3f20 0x7fd1680a63e0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fd170005ab0 tx=0x7fd170000bc0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:50.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.896+0000 7fd18b5a1740 1 --2- 172.21.15.83:0/2922695529 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564aab5af460 0x564aab5b95f0 secure :-1 s=READY pgs=2399 cs=0 l=1 rev1=1 crypto rx=0x7fd17400f4f0 tx=0x7fd17400f9b0 comp rx=0 tx=0).stop 2024-04-24T09:09:50.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.897+0000 7fd18b5a1740 1 -- 172.21.15.83:0/2922695529 shutdown_connections 2024-04-24T09:09:50.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.897+0000 7fd18b5a1740 1 --2- 172.21.15.83:0/2922695529 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564aab5b9b30 0x564aab5bbf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:50.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.897+0000 7fd18b5a1740 1 --2- 172.21.15.83:0/2922695529 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564aab5af460 0x564aab5b95f0 unknown :-1 s=CLOSED pgs=2399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:50.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.897+0000 7fd18b5a1740 1 --2- 172.21.15.83:0/2922695529 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564aab5aeb40 0x564aab5aef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:50.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.897+0000 7fd18b5a1740 1 -- 172.21.15.83:0/2922695529 >> 172.21.15.83:0/2922695529 conn(0x564aab5aae60 msgr2=0x564aab5ab240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:50.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.897+0000 7fd18b5a1740 1 -- 172.21.15.83:0/2922695529 shutdown_connections 2024-04-24T09:09:50.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.897+0000 7fd18b5a1740 1 -- 172.21.15.83:0/2922695529 wait complete. 2024-04-24T09:09:50.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.897+0000 7fd18b5a1740 1 -- start start 2024-04-24T09:09:50.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.898+0000 7fd18b5a1740 1 Processor -- start 2024-04-24T09:09:50.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.898+0000 7fd18b5a1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564aab5aeb40 0x564aab5b6920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:50.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.898+0000 7fd18b5a1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564aab5af460 0x564aab5b6e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:50.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.898+0000 7fd18b5a1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564aab5b9b30 0x564aab5b73a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:50.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.898+0000 7fd18b5a1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x564aab5bc9c0 con 0x564aab5af460 2024-04-24T09:09:50.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.898+0000 7fd18b5a1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x564aab4cd810 con 0x564aab5b9b30 2024-04-24T09:09:50.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.898+0000 7fd18b5a1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x564aab5bcb40 con 0x564aab5aeb40 2024-04-24T09:09:50.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.898+0000 7fd18b5a1740 10 client.0 _ll_register_callbacks cb 0x564aab49b140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:09:50.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.898+0000 7fd185400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564aab5af460 0x564aab5b6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:50.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.899+0000 7fd185400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564aab5af460 0x564aab5b6e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58350/0 (socket says 172.21.15.83:58350) 2024-04-24T09:09:50.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.899+0000 7fd185400640 1 -- 172.21.15.83:0/766259612 learned_addr learned my addr 172.21.15.83:0/766259612 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:09:50.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.899+0000 7fd186800640 1 --2- 172.21.15.83:0/766259612 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564aab5b9b30 0x564aab5b73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:50.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.899+0000 7fd185e00640 1 --2- 172.21.15.83:0/766259612 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564aab5aeb40 0x564aab5b6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:50.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.899+0000 7fd185400640 1 -- 172.21.15.83:0/766259612 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564aab5aeb40 msgr2=0x564aab5b6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:50.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.899+0000 7fd185400640 1 --2- 172.21.15.83:0/766259612 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564aab5aeb40 0x564aab5b6920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:50.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.899+0000 7fd185400640 1 -- 172.21.15.83:0/766259612 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564aab5b9b30 msgr2=0x564aab5b73a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:50.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.899+0000 7fd185400640 1 --2- 172.21.15.83:0/766259612 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564aab5b9b30 0x564aab5b73a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:50.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.899+0000 7fd185400640 1 -- 172.21.15.83:0/766259612 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564aab4de3f0 con 0x564aab5af460 2024-04-24T09:09:50.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.899+0000 7fd185e00640 1 --2- 172.21.15.83:0/766259612 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564aab5aeb40 0x564aab5b6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:09:50.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.899+0000 7fd186800640 1 --2- 172.21.15.83:0/766259612 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564aab5b9b30 0x564aab5b73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:09:50.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.900+0000 7fd185400640 1 --2- 172.21.15.83:0/766259612 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564aab5af460 0x564aab5b6e60 secure :-1 s=READY pgs=2400 cs=0 l=1 rev1=1 crypto rx=0x7fd170033000 tx=0x7fd170005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=157d6afae3ffb540 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:50.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.900+0000 7fd163400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:09:50.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.900+0000 7fd163400640 1 -- 172.21.15.83:0/766259612 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd170051470 con 0x564aab5af460 2024-04-24T09:09:50.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.900+0000 7fd163400640 1 -- 172.21.15.83:0/766259612 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9583+0+0 (secure 0 0 0) 0x7fd170051610 con 0x564aab5af460 2024-04-24T09:09:50.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.900+0000 7fd18b5a1740 1 -- 172.21.15.83:0/766259612 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1740047a0 con 0x564aab5af460 2024-04-24T09:09:50.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.901+0000 7fd163400640 1 -- 172.21.15.83:0/766259612 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7fd17005a050 con 0x564aab5af460 2024-04-24T09:09:50.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.901+0000 7fd18b5a1740 10 client.21927 fetch_fsmap learned FSMap version 61 2024-04-24T09:09:50.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.901+0000 7fd18b5a1740 1 -- 172.21.15.83:0/766259612 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x564aab2263f0 con 0x564aab5af460 2024-04-24T09:09:50.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.902+0000 7fd163400640 1 -- 172.21.15.83:0/766259612 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd1740047a0 con 0x564aab5af460 2024-04-24T09:09:50.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.902+0000 7fd163400640 1 -- 172.21.15.83:0/766259612 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 61) ==== 8362+0+0 (secure 0 0 0) 0x7fd170044030 con 0x564aab5af460 2024-04-24T09:09:50.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.902+0000 7fd18b5a1740 10 client.21927 fetch_fsmap finished waiting for FSMap version 61 2024-04-24T09:09:50.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.902+0000 7fd18b5a1740 10 client.21927 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-24T09:09:50.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.902+0000 7fd18b5a1740 20 client.21927 populate_metadata read hostname 'smithi083' 2024-04-24T09:09:50.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.902+0000 7fd18b5a1740 1 --2- 172.21.15.83:0/766259612 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x564aab5c9db0 0x564aab5cc200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:50.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.902+0000 7fd18b5a1740 4 client.21927 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:09:50.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.902+0000 7fd18b5a1740 1 -- 172.21.15.83:0/766259612 --> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564aab521bf0 con 0x564aab5c9db0 2024-04-24T09:09:50.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.902+0000 7fd185e00640 1 --2- 172.21.15.83:0/766259612 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x564aab5c9db0 0x564aab5cc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:50.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.903+0000 7fd185e00640 1 --2- 172.21.15.83:0/766259612 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x564aab5c9db0 0x564aab5cc200 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4907cb0899275b77 server_cookie=9d75f6b7e15be63f in_seq=0 out_seq=0 2024-04-24T09:09:50.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.903+0000 7fd163400640 10 client.21927 ms_handle_connect on v2:172.21.15.138:6834/1909219758 2024-04-24T09:09:50.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.904+0000 7fd163400640 1 -- 172.21.15.83:0/766259612 <== mds.2 v2:172.21.15.138:6834/1909219758 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564aab521bf0 con 0x564aab5c9db0 2024-04-24T09:09:50.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.904+0000 7fd163400640 10 client.21927 handle_command_reply: tid=0 2024-04-24T09:09:50.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.904+0000 7fd18b5a1740 1 client.21927 shutdown 2024-04-24T09:09:50.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.904+0000 7fd18b5a1740 1 -- 172.21.15.83:0/766259612 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564aab5af460 msgr2=0x564aab5b6e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:50.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.904+0000 7fd18b5a1740 1 --2- 172.21.15.83:0/766259612 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564aab5af460 0x564aab5b6e60 secure :-1 s=READY pgs=2400 cs=0 l=1 rev1=1 crypto rx=0x7fd170033000 tx=0x7fd170005fd0 comp rx=0 tx=0).stop 2024-04-24T09:09:50.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.905+0000 7fd18b5a1740 1 -- 172.21.15.83:0/766259612 shutdown_connections 2024-04-24T09:09:50.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.905+0000 7fd18b5a1740 1 --2- 172.21.15.83:0/766259612 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x564aab5c9db0 0x564aab5cc200 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:50.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.905+0000 7fd18b5a1740 1 --2- 172.21.15.83:0/766259612 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564aab5b9b30 0x564aab5b73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:50.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.905+0000 7fd18b5a1740 1 --2- 172.21.15.83:0/766259612 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564aab5af460 0x564aab5b6e60 unknown :-1 s=CLOSED pgs=2400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:50.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.905+0000 7fd18b5a1740 1 --2- 172.21.15.83:0/766259612 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564aab5aeb40 0x564aab5b6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:50.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.905+0000 7fd163400640 0 client.21927 ms_handle_reset on v2:172.21.15.138:6834/1909219758 2024-04-24T09:09:50.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.905+0000 7fd18b5a1740 1 -- 172.21.15.83:0/766259612 >> 172.21.15.83:0/766259612 conn(0x564aab5aae60 msgr2=0x564aab5b0c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:50.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.905+0000 7fd18b5a1740 1 -- 172.21.15.83:0/766259612 shutdown_connections 2024-04-24T09:09:50.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.905+0000 7fd18b5a1740 1 -- 172.21.15.83:0/766259612 wait complete. 2024-04-24T09:09:50.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.905+0000 7fd18b5a1740 20 client.21927 trim_cache size 0 max 16384 2024-04-24T09:09:50.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.925+0000 7fd18b5a1740 1 Processor -- start 2024-04-24T09:09:50.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.925+0000 7fd18b5a1740 1 -- start start 2024-04-24T09:09:50.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.925+0000 7fd18b5a1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564aab5aeb40 0x564aab5b12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:50.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.925+0000 7fd18b5a1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564aab5af460 0x564aab5c5880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:50.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.925+0000 7fd18b5a1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564aab5b9b30 0x564aab5b5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:50.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.925+0000 7fd18b5a1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd1801b59b0 con 0x564aab5b9b30 2024-04-24T09:09:50.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.925+0000 7fd18b5a1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd1801b74f0 con 0x564aab5af460 2024-04-24T09:09:50.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.925+0000 7fd18b5a1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd1801b6080 con 0x564aab5aeb40 2024-04-24T09:09:50.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.926+0000 7fd185e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564aab5aeb40 0x564aab5b12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:50.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.926+0000 7fd185400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564aab5af460 0x564aab5c5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:50.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.926+0000 7fd185400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564aab5af460 0x564aab5c5880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:50314/0 (socket says 172.21.15.83:50314) 2024-04-24T09:09:50.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.926+0000 7fd185400640 1 -- 172.21.15.83:0/3171123005 learned_addr learned my addr 172.21.15.83:0/3171123005 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:09:50.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.926+0000 7fd186800640 1 --2- 172.21.15.83:0/3171123005 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564aab5b9b30 0x564aab5b5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:50.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.926+0000 7fd185400640 1 -- 172.21.15.83:0/3171123005 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564aab5aeb40 msgr2=0x564aab5b12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:50.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.926+0000 7fd185400640 1 --2- 172.21.15.83:0/3171123005 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564aab5aeb40 0x564aab5b12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:50.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.926+0000 7fd185400640 1 -- 172.21.15.83:0/3171123005 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564aab5b9b30 msgr2=0x564aab5b5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:50.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.926+0000 7fd185400640 1 --2- 172.21.15.83:0/3171123005 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564aab5b9b30 0x564aab5b5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:50.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.926+0000 7fd185400640 1 -- 172.21.15.83:0/3171123005 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564aab2f07d0 con 0x564aab5af460 2024-04-24T09:09:50.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.927+0000 7fd185400640 1 --2- 172.21.15.83:0/3171123005 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564aab5af460 0x564aab5c5880 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd17003d910 tx=0x7fd170009230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9faf557a616ed440 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:50.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.927+0000 7fd16e600640 1 -- 172.21.15.83:0/3171123005 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd17003d450 con 0x564aab5af460 2024-04-24T09:09:50.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.927+0000 7fd16e600640 1 -- 172.21.15.83:0/3171123005 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd17004c070 con 0x564aab5af460 2024-04-24T09:09:50.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.927+0000 7fd16e600640 1 -- 172.21.15.83:0/3171123005 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd170051d10 con 0x564aab5af460 2024-04-24T09:09:50.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.928+0000 7fd18b5a1740 1 -- 172.21.15.83:0/3171123005 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564aab5af460 msgr2=0x564aab5c5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:50.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.928+0000 7fd18b5a1740 1 --2- 172.21.15.83:0/3171123005 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564aab5af460 0x564aab5c5880 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd17003d910 tx=0x7fd170009230 comp rx=0 tx=0).stop 2024-04-24T09:09:50.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.928+0000 7fd18b5a1740 1 -- 172.21.15.83:0/3171123005 shutdown_connections 2024-04-24T09:09:50.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.928+0000 7fd18b5a1740 1 --2- 172.21.15.83:0/3171123005 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564aab5b9b30 0x564aab5b5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:50.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.928+0000 7fd18b5a1740 1 --2- 172.21.15.83:0/3171123005 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564aab5af460 0x564aab5c5880 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:50.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.928+0000 7fd18b5a1740 1 --2- 172.21.15.83:0/3171123005 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564aab5aeb40 0x564aab5b12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:50.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.928+0000 7fd18b5a1740 1 -- 172.21.15.83:0/3171123005 >> 172.21.15.83:0/3171123005 conn(0x564aab5aae60 msgr2=0x564aab5b68e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:50.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.928+0000 7fd18b5a1740 1 -- 172.21.15.83:0/3171123005 shutdown_connections 2024-04-24T09:09:50.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.929+0000 7fd18b5a1740 1 -- 172.21.15.83:0/3171123005 wait complete. 2024-04-24T09:09:50.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.929+0000 7fd18b5a1740 1 -- start start 2024-04-24T09:09:50.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.929+0000 7fd18b5a1740 1 Processor -- start 2024-04-24T09:09:50.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.930+0000 7fd18b5a1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564aab5aeb40 0x564aab5ba920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:50.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.930+0000 7fd18b5a1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564aab5af460 0x564aab5bae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:50.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.930+0000 7fd18b5a1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564aab5b9b30 0x564aab5bb3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:50.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.930+0000 7fd18b5a1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd1801b6e20 con 0x564aab5af460 2024-04-24T09:09:50.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.930+0000 7fd185e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564aab5aeb40 0x564aab5ba920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:50.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.930+0000 7fd185e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564aab5aeb40 0x564aab5ba920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:50322/0 (socket says 172.21.15.83:50322) 2024-04-24T09:09:50.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.930+0000 7fd185e00640 1 -- 172.21.15.83:0/374988393 learned_addr learned my addr 172.21.15.83:0/374988393 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:09:50.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.930+0000 7fd18b5a1740 1 -- 172.21.15.83:0/374988393 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd1801b6750 con 0x564aab5aeb40 2024-04-24T09:09:50.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.930+0000 7fd18b5a1740 1 -- 172.21.15.83:0/374988393 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd1801b39b0 con 0x564aab5b9b30 2024-04-24T09:09:50.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.930+0000 7fd18b5a1740 10 client.0 _ll_register_callbacks cb 0x7fd170009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:09:50.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.930+0000 7fd185400640 1 --2- 172.21.15.83:0/374988393 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564aab5af460 0x564aab5bae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:50.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.931+0000 7fd186800640 1 --2- 172.21.15.83:0/374988393 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564aab5b9b30 0x564aab5bb3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:50.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.931+0000 7fd185e00640 1 -- 172.21.15.83:0/374988393 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564aab5b9b30 msgr2=0x564aab5bb3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:50.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.931+0000 7fd185e00640 1 --2- 172.21.15.83:0/374988393 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564aab5b9b30 0x564aab5bb3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:50.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.931+0000 7fd185e00640 1 -- 172.21.15.83:0/374988393 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564aab5af460 msgr2=0x564aab5bae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:50.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.931+0000 7fd185e00640 1 --2- 172.21.15.83:0/374988393 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564aab5af460 0x564aab5bae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:50.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.931+0000 7fd185e00640 1 -- 172.21.15.83:0/374988393 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564aab5bffe0 con 0x564aab5aeb40 2024-04-24T09:09:50.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.931+0000 7fd185e00640 1 --2- 172.21.15.83:0/374988393 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564aab5aeb40 0x564aab5ba920 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fd17400feb0 tx=0x7fd1740109b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e97aa5a5404e4e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:50.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.931+0000 7fd163400640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:09:50.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.932+0000 7fd163400640 1 -- 172.21.15.83:0/374988393 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd174010cf0 con 0x564aab5aeb40 2024-04-24T09:09:50.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.932+0000 7fd18b5a1740 1 -- 172.21.15.83:0/374988393 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd17004c070 con 0x564aab5aeb40 2024-04-24T09:09:50.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.932+0000 7fd163400640 1 -- 172.21.15.83:0/374988393 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9583+0+0 (secure 0 0 0) 0x7fd17402e030 con 0x564aab5aeb40 2024-04-24T09:09:50.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.932+0000 7fd163400640 1 -- 172.21.15.83:0/374988393 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7fd174038050 con 0x564aab5aeb40 2024-04-24T09:09:50.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.932+0000 7fd18b5a1740 10 client.39056 fetch_fsmap learned FSMap version 61 2024-04-24T09:09:50.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.933+0000 7fd18b5a1740 1 -- 172.21.15.83:0/374988393 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x564aab630670 con 0x564aab5aeb40 2024-04-24T09:09:50.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.933+0000 7fd163400640 1 -- 172.21.15.83:0/374988393 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd17004c070 con 0x564aab5aeb40 2024-04-24T09:09:50.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.933+0000 7fd163400640 1 -- 172.21.15.83:0/374988393 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 61) ==== 8362+0+0 (secure 0 0 0) 0x7fd174004960 con 0x564aab5aeb40 2024-04-24T09:09:50.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.934+0000 7fd18b5a1740 10 client.39056 fetch_fsmap finished waiting for FSMap version 61 2024-04-24T09:09:50.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.934+0000 7fd18b5a1740 10 client.39056 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-24T09:09:50.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.934+0000 7fd18b5a1740 20 client.39056 populate_metadata read hostname 'smithi083' 2024-04-24T09:09:50.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.934+0000 7fd18b5a1740 1 --2- 172.21.15.83:0/374988393 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x564aab630ab0 0x564aab632ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:50.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.934+0000 7fd18b5a1740 4 client.39056 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:09:50.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.934+0000 7fd18b5a1740 1 -- 172.21.15.83:0/374988393 --> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564aab51ba60 con 0x564aab630ab0 2024-04-24T09:09:50.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.934+0000 7fd185400640 1 --2- 172.21.15.83:0/374988393 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x564aab630ab0 0x564aab632ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:50.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.935+0000 7fd185400640 1 --2- 172.21.15.83:0/374988393 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x564aab630ab0 0x564aab632ed0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9415ca96f4fa3817 server_cookie=21c70b441e4db39a in_seq=0 out_seq=0 2024-04-24T09:09:50.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.935+0000 7fd163400640 10 client.39056 ms_handle_connect on v2:172.21.15.138:6834/1909219758 2024-04-24T09:09:50.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.936+0000 7fd163400640 1 -- 172.21.15.83:0/374988393 <== mds.2 v2:172.21.15.138:6834/1909219758 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564aab51ba60 con 0x564aab630ab0 2024-04-24T09:09:50.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.936+0000 7fd163400640 10 client.39056 handle_command_reply: tid=0 2024-04-24T09:09:50.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.936+0000 7fd18b5a1740 1 client.39056 shutdown 2024-04-24T09:09:50.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.936+0000 7fd18b5a1740 1 -- 172.21.15.83:0/374988393 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564aab5aeb40 msgr2=0x564aab5ba920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:50.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.936+0000 7fd18b5a1740 1 --2- 172.21.15.83:0/374988393 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564aab5aeb40 0x564aab5ba920 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fd17400feb0 tx=0x7fd1740109b0 comp rx=0 tx=0).stop 2024-04-24T09:09:50.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.936+0000 7fd18b5a1740 1 -- 172.21.15.83:0/374988393 shutdown_connections 2024-04-24T09:09:50.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.936+0000 7fd18b5a1740 1 --2- 172.21.15.83:0/374988393 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x564aab630ab0 0x564aab632ed0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:50.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.936+0000 7fd18b5a1740 1 --2- 172.21.15.83:0/374988393 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564aab5b9b30 0x564aab5bb3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:50.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.936+0000 7fd18b5a1740 1 --2- 172.21.15.83:0/374988393 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564aab5af460 0x564aab5bae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:50.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.936+0000 7fd18b5a1740 1 --2- 172.21.15.83:0/374988393 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564aab5aeb40 0x564aab5ba920 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:50.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.936+0000 7fd18b5a1740 1 -- 172.21.15.83:0/374988393 >> 172.21.15.83:0/374988393 conn(0x564aab5aae60 msgr2=0x564aab5a9f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:50.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.936+0000 7fd163400640 0 client.39056 ms_handle_reset on v2:172.21.15.138:6834/1909219758 2024-04-24T09:09:50.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.937+0000 7fd18b5a1740 1 -- 172.21.15.83:0/374988393 shutdown_connections 2024-04-24T09:09:50.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.937+0000 7fd18b5a1740 1 -- 172.21.15.83:0/374988393 wait complete. 2024-04-24T09:09:50.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.937+0000 7fd18b5a1740 20 client.39056 trim_cache size 0 max 16384 2024-04-24T09:09:50.938 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:09:50.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.939+0000 7fd187200640 1 -- 172.21.15.83:0/1164139926 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fd1680a3f20 msgr2=0x7fd1680a63e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:50.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.939+0000 7fd187200640 1 --2- 172.21.15.83:0/1164139926 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fd1680a3f20 0x7fd1680a63e0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fd170005ab0 tx=0x7fd170000bc0 comp rx=0 tx=0).stop 2024-04-24T09:09:50.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.939+0000 7fd187200640 1 -- 172.21.15.83:0/1164139926 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd18010c480 msgr2=0x7fd180108ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:50.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.939+0000 7fd187200640 1 --2- 172.21.15.83:0/1164139926 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd18010c480 0x7fd180108ff0 secure :-1 s=READY pgs=2398 cs=0 l=1 rev1=1 crypto rx=0x7fd17c00e940 tx=0x7fd17c00ee00 comp rx=0 tx=0).stop 2024-04-24T09:09:50.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.941+0000 7fd187200640 1 -- 172.21.15.83:0/1164139926 shutdown_connections 2024-04-24T09:09:50.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.941+0000 7fd187200640 1 --2- 172.21.15.83:0/1164139926 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fd1680a3f20 0x7fd1680a63e0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:50.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.941+0000 7fd187200640 1 --2- 172.21.15.83:0/1164139926 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd18010c480 0x7fd180108ff0 unknown :-1 s=CLOSED pgs=2398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:50.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.941+0000 7fd187200640 1 --2- 172.21.15.83:0/1164139926 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd180103980 0x7fd180108ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:50.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.941+0000 7fd187200640 1 --2- 172.21.15.83:0/1164139926 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd180103060 0x7fd18010b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:50.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.941+0000 7fd187200640 1 -- 172.21.15.83:0/1164139926 >> 172.21.15.83:0/1164139926 conn(0x7fd1800fc880 msgr2=0x7fd18010dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:50.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.941+0000 7fd187200640 1 -- 172.21.15.83:0/1164139926 shutdown_connections 2024-04-24T09:09:50.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:50.941+0000 7fd187200640 1 -- 172.21.15.83:0/1164139926 wait complete. 2024-04-24T09:09:50.963 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T09:09:51.172 INFO:tasks.workunit.client.0.smithi083.stdout: 10 33332 6.80 MB/sec execute 142 sec latency 75.425 ms 2024-04-24T09:09:51.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.209+0000 7f6cae800640 1 -- 172.21.15.83:0/4142425137 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ca8109580 msgr2=0x7f6ca8111150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:51.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.209+0000 7f6cae800640 1 --2- 172.21.15.83:0/4142425137 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ca8109580 0x7f6ca8111150 secure :-1 s=READY pgs=2401 cs=0 l=1 rev1=1 crypto rx=0x7f6ca0009ca0 tx=0x7f6ca002a8e0 comp rx=0 tx=0).stop 2024-04-24T09:09:51.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.209+0000 7f6cae800640 1 -- 172.21.15.83:0/4142425137 shutdown_connections 2024-04-24T09:09:51.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.209+0000 7f6cae800640 1 --2- 172.21.15.83:0/4142425137 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ca8109580 0x7f6ca8111150 unknown :-1 s=CLOSED pgs=2401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:51.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.209+0000 7f6cae800640 1 --2- 172.21.15.83:0/4142425137 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6ca8104e60 0x7f6ca8108ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:51.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.209+0000 7f6cae800640 1 --2- 172.21.15.83:0/4142425137 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ca81044b0 0x7f6ca8104890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:51.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.209+0000 7f6cae800640 1 -- 172.21.15.83:0/4142425137 >> 172.21.15.83:0/4142425137 conn(0x7f6ca80ffd90 msgr2=0x7f6ca81021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:51.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.210+0000 7f6cae800640 1 -- 172.21.15.83:0/4142425137 shutdown_connections 2024-04-24T09:09:51.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.210+0000 7f6cae800640 1 -- 172.21.15.83:0/4142425137 wait complete. 2024-04-24T09:09:51.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.214+0000 7f6cae800640 1 Processor -- start 2024-04-24T09:09:51.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.215+0000 7f6cae800640 1 -- start start 2024-04-24T09:09:51.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.215+0000 7f6cae800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ca81044b0 0x7f6ca810fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:51.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.215+0000 7f6cae800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6ca8109580 0x7f6ca81101f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:51.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.215+0000 7f6cae800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ca81a4970 0x7f6ca81a4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:51.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.215+0000 7f6cae800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca8077650 con 0x7f6ca81a4970 2024-04-24T09:09:51.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.215+0000 7f6cae800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca80774d0 con 0x7f6ca81044b0 2024-04-24T09:09:51.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.215+0000 7f6cae800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca80777d0 con 0x7f6ca8109580 2024-04-24T09:09:51.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.216+0000 7f6ca7400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ca81044b0 0x7f6ca810fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:51.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.216+0000 7f6ca6a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6ca8109580 0x7f6ca81101f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:51.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.216+0000 7f6ca7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ca81a4970 0x7f6ca81a4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:51.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.216+0000 7f6ca7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ca81a4970 0x7f6ca81a4e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58380/0 (socket says 172.21.15.83:58380) 2024-04-24T09:09:51.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.216+0000 7f6ca7e00640 1 -- 172.21.15.83:0/524900398 learned_addr learned my addr 172.21.15.83:0/524900398 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:09:51.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.217+0000 7f6ca7e00640 1 -- 172.21.15.83:0/524900398 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6ca8109580 msgr2=0x7f6ca81101f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:51.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.218+0000 7f6ca7e00640 1 --2- 172.21.15.83:0/524900398 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6ca8109580 0x7f6ca81101f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:51.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.218+0000 7f6ca7e00640 1 -- 172.21.15.83:0/524900398 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ca81044b0 msgr2=0x7f6ca810fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:51.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.218+0000 7f6ca7e00640 1 --2- 172.21.15.83:0/524900398 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ca81044b0 0x7f6ca810fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:51.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.218+0000 7f6ca7e00640 1 -- 172.21.15.83:0/524900398 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ca81a5490 con 0x7f6ca81a4970 2024-04-24T09:09:51.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.218+0000 7f6ca7400640 1 --2- 172.21.15.83:0/524900398 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ca81044b0 0x7f6ca810fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:09:51.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.219+0000 7f6ca7e00640 1 --2- 172.21.15.83:0/524900398 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ca81a4970 0x7f6ca81a4e20 secure :-1 s=READY pgs=2402 cs=0 l=1 rev1=1 crypto rx=0x7f6ca002ae50 tx=0x7f6ca0003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:51.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.219+0000 7f6c9fe00640 1 -- 172.21.15.83:0/524900398 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca0031d70 con 0x7f6ca81a4970 2024-04-24T09:09:51.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.219+0000 7f6c9fe00640 1 -- 172.21.15.83:0/524900398 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6ca0031d70 con 0x7f6ca81a4970 2024-04-24T09:09:51.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.219+0000 7f6c9fe00640 1 -- 172.21.15.83:0/524900398 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca0030950 con 0x7f6ca81a4970 2024-04-24T09:09:51.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.220+0000 7f6cae800640 1 -- 172.21.15.83:0/524900398 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ca81a5720 con 0x7f6ca81a4970 2024-04-24T09:09:51.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.220+0000 7f6cae800640 1 -- 172.21.15.83:0/524900398 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ca81abcc0 con 0x7f6ca81a4970 2024-04-24T09:09:51.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.226+0000 7f6c9fe00640 1 -- 172.21.15.83:0/524900398 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6ca0030af0 con 0x7f6ca81a4970 2024-04-24T09:09:51.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.227+0000 7f6cb178b740 1 Processor -- start 2024-04-24T09:09:51.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.227+0000 7f6cb178b740 1 -- start start 2024-04-24T09:09:51.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.228+0000 7f6cb178b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556989b4f950 0x556989b4fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:51.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.228+0000 7f6cb178b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556989b50270 0x556989b5a400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:51.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.228+0000 7f6cb178b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556989b5a940 0x556989b5cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:51.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.228+0000 7f6cb178b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556989ab51d0 con 0x556989b4f950 2024-04-24T09:09:51.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.228+0000 7f6cb178b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556989a5b450 con 0x556989b5a940 2024-04-24T09:09:51.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.228+0000 7f6cb178b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x556989ab1190 con 0x556989b50270 2024-04-24T09:09:51.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.229+0000 7f6ca7400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556989b50270 0x556989b5a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:51.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.229+0000 7f6ca7400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556989b50270 0x556989b5a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:41848/0 (socket says 172.21.15.83:41848) 2024-04-24T09:09:51.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.229+0000 7f6ca7400640 1 -- 172.21.15.83:0/324111534 learned_addr learned my addr 172.21.15.83:0/324111534 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:09:51.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.230+0000 7f6ca6a00640 1 --2- 172.21.15.83:0/324111534 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556989b4f950 0x556989b4fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:51.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.230+0000 7f6ca7400640 1 -- 172.21.15.83:0/324111534 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556989b5a940 msgr2=0x556989b5cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:51.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.230+0000 7f6ca7400640 1 --2- 172.21.15.83:0/324111534 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556989b5a940 0x556989b5cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:51.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.230+0000 7f6ca7400640 1 -- 172.21.15.83:0/324111534 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556989b4f950 msgr2=0x556989b4fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:51.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.230+0000 7f6c9fe00640 1 --2- 172.21.15.83:0/524900398 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f6c8c0a41e0 0x7f6c8c0a66a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:51.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.231+0000 7f6ca7400640 1 --2- 172.21.15.83:0/324111534 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556989b4f950 0x556989b4fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:51.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.231+0000 7f6ca7400640 1 -- 172.21.15.83:0/324111534 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556989b18310 con 0x556989b50270 2024-04-24T09:09:51.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.231+0000 7f6c9fe00640 1 -- 172.21.15.83:0/524900398 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 9583+0+0 (secure 0 0 0) 0x7f6ca0003d10 con 0x7f6ca81a4970 2024-04-24T09:09:51.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.231+0000 7f6ca6a00640 1 --2- 172.21.15.83:0/324111534 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556989b4f950 0x556989b4fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:09:51.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.233+0000 7f6ca7400640 1 --2- 172.21.15.83:0/324111534 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556989b50270 0x556989b5a400 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6c9800bd40 tx=0x7f6c9800f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b3386c17adbbf0d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:51.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.233+0000 7f6ca6a00640 1 --2- 172.21.15.83:0/524900398 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f6c8c0a41e0 0x7f6c8c0a66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:51.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.234+0000 7f6c9cc00640 1 -- 172.21.15.83:0/324111534 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c98015070 con 0x556989b50270 2024-04-24T09:09:51.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.234+0000 7f6c9cc00640 1 -- 172.21.15.83:0/324111534 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6c9800be30 con 0x556989b50270 2024-04-24T09:09:51.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.234+0000 7f6c9cc00640 1 -- 172.21.15.83:0/324111534 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c98011990 con 0x556989b50270 2024-04-24T09:09:51.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.236+0000 7f6cb178b740 1 -- 172.21.15.83:0/324111534 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556989b50270 msgr2=0x556989b5a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:51.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.236+0000 7f6cb178b740 1 --2- 172.21.15.83:0/324111534 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556989b50270 0x556989b5a400 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6c9800bd40 tx=0x7f6c9800f730 comp rx=0 tx=0).stop 2024-04-24T09:09:51.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.236+0000 7f6cb178b740 1 -- 172.21.15.83:0/324111534 shutdown_connections 2024-04-24T09:09:51.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.236+0000 7f6cb178b740 1 --2- 172.21.15.83:0/324111534 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556989b5a940 0x556989b5cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:51.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.236+0000 7f6cb178b740 1 --2- 172.21.15.83:0/324111534 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556989b50270 0x556989b5a400 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:51.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.237+0000 7f6cb178b740 1 --2- 172.21.15.83:0/324111534 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556989b4f950 0x556989b4fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:51.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.237+0000 7f6cb178b740 1 -- 172.21.15.83:0/324111534 >> 172.21.15.83:0/324111534 conn(0x556989b4bc70 msgr2=0x556989b4c050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:51.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.237+0000 7f6ca6a00640 1 --2- 172.21.15.83:0/524900398 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f6c8c0a41e0 0x7f6c8c0a66a0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f6c94006e10 tx=0x7f6c940047d0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:51.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.238+0000 7f6cb178b740 1 -- 172.21.15.83:0/324111534 shutdown_connections 2024-04-24T09:09:51.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.239+0000 7f6cb178b740 1 -- 172.21.15.83:0/324111534 wait complete. 2024-04-24T09:09:51.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.240+0000 7f6cb178b740 1 -- start start 2024-04-24T09:09:51.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.240+0000 7f6cb178b740 1 Processor -- start 2024-04-24T09:09:51.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.241+0000 7f6cb178b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556989b4f950 0x556989b5a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:51.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.241+0000 7f6cb178b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556989b50270 0x556989b576c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:51.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.241+0000 7f6cb178b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556989b5a940 0x556989b57db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:51.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.242+0000 7f6ca7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556989b5a940 0x556989b57db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:51.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.242+0000 7f6ca7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556989b5a940 0x556989b57db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58406/0 (socket says 172.21.15.83:58406) 2024-04-24T09:09:51.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.242+0000 7f6ca7e00640 1 -- 172.21.15.83:0/1163000982 learned_addr learned my addr 172.21.15.83:0/1163000982 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:09:51.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.242+0000 7f6cb178b740 1 -- 172.21.15.83:0/1163000982 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556989b5d7d0 con 0x556989b5a940 2024-04-24T09:09:51.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.242+0000 7f6cb178b740 1 -- 172.21.15.83:0/1163000982 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556989a6e810 con 0x556989b50270 2024-04-24T09:09:51.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.242+0000 7f6cb178b740 1 -- 172.21.15.83:0/1163000982 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x556989b5d950 con 0x556989b4f950 2024-04-24T09:09:51.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.242+0000 7f6ca7400640 1 --2- 172.21.15.83:0/1163000982 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556989b4f950 0x556989b5a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:51.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.243+0000 7f6ca6a00640 1 --2- 172.21.15.83:0/1163000982 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556989b50270 0x556989b576c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:51.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.243+0000 7f6ca7e00640 1 -- 172.21.15.83:0/1163000982 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556989b4f950 msgr2=0x556989b5a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:51.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.243+0000 7f6ca7e00640 1 --2- 172.21.15.83:0/1163000982 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556989b4f950 0x556989b5a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:51.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.243+0000 7f6ca7e00640 1 -- 172.21.15.83:0/1163000982 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556989b50270 msgr2=0x556989b576c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:51.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.243+0000 7f6ca7e00640 1 --2- 172.21.15.83:0/1163000982 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556989b50270 0x556989b576c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:51.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.243+0000 7f6cb178b740 10 client.0 _ll_register_callbacks cb 0x556989a3c140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:09:51.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.243+0000 7f6ca7e00640 1 -- 172.21.15.83:0/1163000982 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556989a7f3f0 con 0x556989b5a940 2024-04-24T09:09:51.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.243+0000 7f6ca6a00640 1 --2- 172.21.15.83:0/1163000982 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556989b50270 0x556989b576c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:09:51.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.243+0000 7f6cb178b740 1 -- 172.21.15.83:0/1163000982 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c9800be30 con 0x556989b5a940 2024-04-24T09:09:51.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.243+0000 7f6ca7e00640 1 --2- 172.21.15.83:0/1163000982 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556989b5a940 0x556989b57db0 secure :-1 s=READY pgs=2403 cs=0 l=1 rev1=1 crypto rx=0x7f6ca00c6e30 tx=0x7f6ca00baac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdb3f715ed787e23 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:51.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.244+0000 7f6c91600640 10 client.21942 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:09:51.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.244+0000 7f6c91600640 1 -- 172.21.15.83:0/1163000982 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca00be6f0 con 0x556989b5a940 2024-04-24T09:09:51.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.245+0000 7f6c91600640 1 -- 172.21.15.83:0/1163000982 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9583+0+0 (secure 0 0 0) 0x7f6ca0030de0 con 0x556989b5a940 2024-04-24T09:09:51.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.245+0000 7f6c91600640 1 -- 172.21.15.83:0/1163000982 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f6ca00be6f0 con 0x556989b5a940 2024-04-24T09:09:51.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.245+0000 7f6cb178b740 10 client.21942 fetch_fsmap learned FSMap version 61 2024-04-24T09:09:51.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.245+0000 7f6cb178b740 1 -- 172.21.15.83:0/1163000982 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({fsmap=61}) -- 0x5569897c73f0 con 0x556989b5a940 2024-04-24T09:09:51.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.245+0000 7f6c91600640 1 -- 172.21.15.83:0/1163000982 <== mon.0 v2:172.21.15.83:3300/0 4 ==== fsmap(e 61) ==== 8362+0+0 (secure 0 0 0) 0x7f6ca00bad60 con 0x556989b5a940 2024-04-24T09:09:51.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.246+0000 7f6cb178b740 10 client.21942 fetch_fsmap finished waiting for FSMap version 61 2024-04-24T09:09:51.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.246+0000 7f6cb178b740 10 client.21942 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-24T09:09:51.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.246+0000 7f6cb178b740 20 client.21942 populate_metadata read hostname 'smithi083' 2024-04-24T09:09:51.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.246+0000 7f6cb178b740 1 --2- 172.21.15.83:0/1163000982 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x556989b7b850 0x556989b7dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:51.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.246+0000 7f6cb178b740 4 client.21942 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:09:51.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.246+0000 7f6cb178b740 1 -- 172.21.15.83:0/1163000982 --> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556989ac2bf0 con 0x556989b7b850 2024-04-24T09:09:51.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.247+0000 7f6ca7400640 1 --2- 172.21.15.83:0/1163000982 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x556989b7b850 0x556989b7dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:51.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.250+0000 7f6ca7400640 1 --2- 172.21.15.83:0/1163000982 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x556989b7b850 0x556989b7dd10 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=14960f98bd5c23ff server_cookie=2f8a58800947c910 in_seq=0 out_seq=0 2024-04-24T09:09:51.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.250+0000 7f6c91600640 10 client.21942 ms_handle_connect on v2:172.21.15.138:6836/900566754 2024-04-24T09:09:51.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.252+0000 7f6c91600640 1 -- 172.21.15.83:0/1163000982 <== mds.1 v2:172.21.15.138:6836/900566754 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556989ac2bf0 con 0x556989b7b850 2024-04-24T09:09:51.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.252+0000 7f6c91600640 10 client.21942 handle_command_reply: tid=0 2024-04-24T09:09:51.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.253+0000 7f6cb178b740 1 client.21942 shutdown 2024-04-24T09:09:51.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.254+0000 7f6cb178b740 1 -- 172.21.15.83:0/1163000982 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556989b5a940 msgr2=0x556989b57db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:51.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.254+0000 7f6cb178b740 1 --2- 172.21.15.83:0/1163000982 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556989b5a940 0x556989b57db0 secure :-1 s=READY pgs=2403 cs=0 l=1 rev1=1 crypto rx=0x7f6ca00c6e30 tx=0x7f6ca00baac0 comp rx=0 tx=0).stop 2024-04-24T09:09:51.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.254+0000 7f6cb178b740 1 -- 172.21.15.83:0/1163000982 shutdown_connections 2024-04-24T09:09:51.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.254+0000 7f6cb178b740 1 --2- 172.21.15.83:0/1163000982 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556989b5a940 0x556989b57db0 unknown :-1 s=CLOSED pgs=2403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:51.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.254+0000 7f6cb178b740 1 --2- 172.21.15.83:0/1163000982 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556989b50270 0x556989b576c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:51.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.254+0000 7f6cb178b740 1 --2- 172.21.15.83:0/1163000982 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x556989b7b850 0x556989b7dd10 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:51.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.254+0000 7f6cb178b740 1 --2- 172.21.15.83:0/1163000982 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556989b4f950 0x556989b5a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:51.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.254+0000 7f6cb178b740 1 -- 172.21.15.83:0/1163000982 >> 172.21.15.83:0/1163000982 conn(0x556989b4bc70 msgr2=0x556989b51a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:51.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.255+0000 7f6c91600640 0 client.21942 ms_handle_reset on v2:172.21.15.138:6836/900566754 2024-04-24T09:09:51.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.256+0000 7f6cb178b740 1 -- 172.21.15.83:0/1163000982 shutdown_connections 2024-04-24T09:09:51.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.256+0000 7f6cb178b740 1 -- 172.21.15.83:0/1163000982 wait complete. 2024-04-24T09:09:51.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.256+0000 7f6cb178b740 20 client.21942 trim_cache size 0 max 16384 2024-04-24T09:09:51.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.285+0000 7f6cb178b740 1 Processor -- start 2024-04-24T09:09:51.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.285+0000 7f6cb178b740 1 -- start start 2024-04-24T09:09:51.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.285+0000 7f6cb178b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556989b4f950 0x556989b520b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:51.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.285+0000 7f6cb178b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556989b50270 0x556989b59f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:51.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.285+0000 7f6cb178b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556989b5a940 0x556989b83620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:51.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.285+0000 7f6cb178b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca81ad360 con 0x556989b5a940 2024-04-24T09:09:51.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.285+0000 7f6cb178b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca81aeea0 con 0x556989b4f950 2024-04-24T09:09:51.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.285+0000 7f6cb178b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca81ada30 con 0x556989b50270 2024-04-24T09:09:51.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.286+0000 7f6ca6a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556989b50270 0x556989b59f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:51.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.286+0000 7f6ca6a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556989b50270 0x556989b59f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:41858/0 (socket says 172.21.15.83:41858) 2024-04-24T09:09:51.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.286+0000 7f6ca6a00640 1 -- 172.21.15.83:0/3114783457 learned_addr learned my addr 172.21.15.83:0/3114783457 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:09:51.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.286+0000 7f6ca7400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556989b4f950 0x556989b520b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:51.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.286+0000 7f6ca7e00640 1 --2- 172.21.15.83:0/3114783457 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556989b5a940 0x556989b83620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:51.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.287+0000 7f6ca6a00640 1 -- 172.21.15.83:0/3114783457 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556989b4f950 msgr2=0x556989b520b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:51.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.287+0000 7f6ca6a00640 1 --2- 172.21.15.83:0/3114783457 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556989b4f950 0x556989b520b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:51.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.287+0000 7f6ca6a00640 1 -- 172.21.15.83:0/3114783457 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556989b5a940 msgr2=0x556989b83620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:51.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.287+0000 7f6ca6a00640 1 --2- 172.21.15.83:0/3114783457 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556989b5a940 0x556989b83620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:51.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.287+0000 7f6ca6a00640 1 -- 172.21.15.83:0/3114783457 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569898917d0 con 0x556989b50270 2024-04-24T09:09:51.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.287+0000 7f6ca7e00640 1 --2- 172.21.15.83:0/3114783457 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556989b5a940 0x556989b83620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:09:51.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.287+0000 7f6ca6a00640 1 --2- 172.21.15.83:0/3114783457 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556989b50270 0x556989b59f00 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6ca806b750 tx=0x7f6c94011000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c071318b0d15c911 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:51.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.288+0000 7f6c93e00640 1 -- 172.21.15.83:0/3114783457 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c94013ca0 con 0x556989b50270 2024-04-24T09:09:51.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.288+0000 7f6c93e00640 1 -- 172.21.15.83:0/3114783457 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6c94024070 con 0x556989b50270 2024-04-24T09:09:51.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.288+0000 7f6c93e00640 1 -- 172.21.15.83:0/3114783457 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c94012610 con 0x556989b50270 2024-04-24T09:09:51.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.289+0000 7f6cb178b740 1 -- 172.21.15.83:0/3114783457 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556989b50270 msgr2=0x556989b59f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:51.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.289+0000 7f6cb178b740 1 --2- 172.21.15.83:0/3114783457 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556989b50270 0x556989b59f00 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6ca806b750 tx=0x7f6c94011000 comp rx=0 tx=0).stop 2024-04-24T09:09:51.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.289+0000 7f6cb178b740 1 -- 172.21.15.83:0/3114783457 shutdown_connections 2024-04-24T09:09:51.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.289+0000 7f6cb178b740 1 --2- 172.21.15.83:0/3114783457 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556989b5a940 0x556989b83620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:51.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.289+0000 7f6cb178b740 1 --2- 172.21.15.83:0/3114783457 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556989b50270 0x556989b59f00 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:51.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.289+0000 7f6cb178b740 1 --2- 172.21.15.83:0/3114783457 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556989b4f950 0x556989b520b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:51.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.289+0000 7f6cb178b740 1 -- 172.21.15.83:0/3114783457 >> 172.21.15.83:0/3114783457 conn(0x556989b4bc70 msgr2=0x556989b52980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:51.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.289+0000 7f6cb178b740 1 -- 172.21.15.83:0/3114783457 shutdown_connections 2024-04-24T09:09:51.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.289+0000 7f6cb178b740 1 -- 172.21.15.83:0/3114783457 wait complete. 2024-04-24T09:09:51.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.289+0000 7f6cb178b740 1 -- start start 2024-04-24T09:09:51.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.290+0000 7f6cb178b740 1 Processor -- start 2024-04-24T09:09:51.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.290+0000 7f6cb178b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556989b4f950 0x556989b5c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:51.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.291+0000 7f6cb178b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556989b50270 0x556989b5ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:51.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.291+0000 7f6cb178b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556989b5a940 0x556989b5b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:51.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.291+0000 7f6cb178b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca81ae7d0 con 0x556989b4f950 2024-04-24T09:09:51.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.291+0000 7f6cb178b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca81ae100 con 0x556989b50270 2024-04-24T09:09:51.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.291+0000 7f6cb178b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca8069270 con 0x556989b5a940 2024-04-24T09:09:51.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.291+0000 7f6cb178b740 10 client.0 _ll_register_callbacks cb 0x7f6ca0009900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:09:51.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.291+0000 7f6ca7400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556989b4f950 0x556989b5c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:51.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.291+0000 7f6ca7400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556989b4f950 0x556989b5c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58424/0 (socket says 172.21.15.83:58424) 2024-04-24T09:09:51.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.291+0000 7f6ca7400640 1 -- 172.21.15.83:0/2277647993 learned_addr learned my addr 172.21.15.83:0/2277647993 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:09:51.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.291+0000 7f6ca7e00640 1 --2- 172.21.15.83:0/2277647993 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556989b5a940 0x556989b5b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:51.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.291+0000 7f6ca6a00640 1 --2- 172.21.15.83:0/2277647993 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556989b50270 0x556989b5ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:51.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.292+0000 7f6ca7400640 1 -- 172.21.15.83:0/2277647993 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556989b5a940 msgr2=0x556989b5b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:51.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.292+0000 7f6ca7400640 1 --2- 172.21.15.83:0/2277647993 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556989b5a940 0x556989b5b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:51.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.292+0000 7f6ca7400640 1 -- 172.21.15.83:0/2277647993 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556989b50270 msgr2=0x556989b5ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:51.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.292+0000 7f6ca7400640 1 --2- 172.21.15.83:0/2277647993 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556989b50270 0x556989b5ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:51.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.292+0000 7f6ca7400640 1 -- 172.21.15.83:0/2277647993 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556989b72c30 con 0x556989b4f950 2024-04-24T09:09:51.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.292+0000 7f6ca7400640 1 --2- 172.21.15.83:0/2277647993 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556989b4f950 0x556989b5c790 secure :-1 s=READY pgs=2404 cs=0 l=1 rev1=1 crypto rx=0x7f6c9800fc30 tx=0x7f6c9800b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dd3b2a2739b2ab8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:51.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.292+0000 7f6c91600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:09:51.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.293+0000 7f6c91600640 1 -- 172.21.15.83:0/2277647993 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c98015040 con 0x556989b4f950 2024-04-24T09:09:51.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.293+0000 7f6c91600640 1 -- 172.21.15.83:0/2277647993 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9583+0+0 (secure 0 0 0) 0x7f6c980055d0 con 0x556989b4f950 2024-04-24T09:09:51.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.293+0000 7f6cb178b740 1 -- 172.21.15.83:0/2277647993 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c94024070 con 0x556989b4f950 2024-04-24T09:09:51.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.293+0000 7f6c91600640 1 -- 172.21.15.83:0/2277647993 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f6c98004020 con 0x556989b4f950 2024-04-24T09:09:51.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.293+0000 7f6cb178b740 10 client.21948 fetch_fsmap learned FSMap version 61 2024-04-24T09:09:51.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.293+0000 7f6cb178b740 1 -- 172.21.15.83:0/2277647993 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x556989b73f30 con 0x556989b4f950 2024-04-24T09:09:51.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.293+0000 7f6c91600640 1 -- 172.21.15.83:0/2277647993 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6c94024070 con 0x556989b4f950 2024-04-24T09:09:51.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.294+0000 7f6c91600640 1 -- 172.21.15.83:0/2277647993 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 61) ==== 8362+0+0 (secure 0 0 0) 0x7f6c98029130 con 0x556989b4f950 2024-04-24T09:09:51.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.294+0000 7f6cb178b740 10 client.21948 fetch_fsmap finished waiting for FSMap version 61 2024-04-24T09:09:51.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.294+0000 7f6cb178b740 10 client.21948 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-24T09:09:51.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.294+0000 7f6cb178b740 20 client.21948 populate_metadata read hostname 'smithi083' 2024-04-24T09:09:51.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.294+0000 7f6cb178b740 1 --2- 172.21.15.83:0/2277647993 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x556989be1e90 0x556989be4280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:51.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.294+0000 7f6cb178b740 4 client.21948 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:09:51.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.294+0000 7f6cb178b740 1 -- 172.21.15.83:0/2277647993 --> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556989abca60 con 0x556989be1e90 2024-04-24T09:09:51.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.294+0000 7f6ca6a00640 1 --2- 172.21.15.83:0/2277647993 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x556989be1e90 0x556989be4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:51.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.296+0000 7f6ca6a00640 1 --2- 172.21.15.83:0/2277647993 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x556989be1e90 0x556989be4280 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6c98cded158d8525 server_cookie=c7deabb0b83c5006 in_seq=0 out_seq=0 2024-04-24T09:09:51.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.296+0000 7f6c91600640 10 client.21948 ms_handle_connect on v2:172.21.15.138:6836/900566754 2024-04-24T09:09:51.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.296+0000 7f6c91600640 1 -- 172.21.15.83:0/2277647993 <== mds.1 v2:172.21.15.138:6836/900566754 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556989abca60 con 0x556989be1e90 2024-04-24T09:09:51.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.296+0000 7f6c91600640 10 client.21948 handle_command_reply: tid=0 2024-04-24T09:09:51.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.296+0000 7f6cb178b740 1 client.21948 shutdown 2024-04-24T09:09:51.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.296+0000 7f6cb178b740 1 -- 172.21.15.83:0/2277647993 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556989b4f950 msgr2=0x556989b5c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:51.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.296+0000 7f6cb178b740 1 --2- 172.21.15.83:0/2277647993 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556989b4f950 0x556989b5c790 secure :-1 s=READY pgs=2404 cs=0 l=1 rev1=1 crypto rx=0x7f6c9800fc30 tx=0x7f6c9800b9a0 comp rx=0 tx=0).stop 2024-04-24T09:09:51.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.297+0000 7f6cb178b740 1 -- 172.21.15.83:0/2277647993 shutdown_connections 2024-04-24T09:09:51.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.297+0000 7f6cb178b740 1 --2- 172.21.15.83:0/2277647993 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x556989be1e90 0x556989be4280 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:51.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.297+0000 7f6cb178b740 1 --2- 172.21.15.83:0/2277647993 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556989b5a940 0x556989b5b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:51.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.297+0000 7f6cb178b740 1 --2- 172.21.15.83:0/2277647993 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556989b50270 0x556989b5ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:51.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.297+0000 7f6cb178b740 1 --2- 172.21.15.83:0/2277647993 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556989b4f950 0x556989b5c790 unknown :-1 s=CLOSED pgs=2404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:51.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.297+0000 7f6cb178b740 1 -- 172.21.15.83:0/2277647993 >> 172.21.15.83:0/2277647993 conn(0x556989b4bc70 msgr2=0x556989b55f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:51.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.297+0000 7f6c91600640 0 client.21948 ms_handle_reset on v2:172.21.15.138:6836/900566754 2024-04-24T09:09:51.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.297+0000 7f6cb178b740 1 -- 172.21.15.83:0/2277647993 shutdown_connections 2024-04-24T09:09:51.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.297+0000 7f6cb178b740 1 -- 172.21.15.83:0/2277647993 wait complete. 2024-04-24T09:09:51.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.297+0000 7f6cb178b740 20 client.21948 trim_cache size 0 max 16384 2024-04-24T09:09:51.298 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:09:51.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.298+0000 7f6cae800640 1 -- 172.21.15.83:0/524900398 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f6c8c0a41e0 msgr2=0x7f6c8c0a66a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:51.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.298+0000 7f6cae800640 1 --2- 172.21.15.83:0/524900398 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f6c8c0a41e0 0x7f6c8c0a66a0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f6c94006e10 tx=0x7f6c940047d0 comp rx=0 tx=0).stop 2024-04-24T09:09:51.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.298+0000 7f6cae800640 1 -- 172.21.15.83:0/524900398 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ca81a4970 msgr2=0x7f6ca81a4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:51.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.298+0000 7f6cae800640 1 --2- 172.21.15.83:0/524900398 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ca81a4970 0x7f6ca81a4e20 secure :-1 s=READY pgs=2402 cs=0 l=1 rev1=1 crypto rx=0x7f6ca002ae50 tx=0x7f6ca0003ba0 comp rx=0 tx=0).stop 2024-04-24T09:09:51.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.298+0000 7f6cae800640 1 -- 172.21.15.83:0/524900398 shutdown_connections 2024-04-24T09:09:51.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.298+0000 7f6cae800640 1 --2- 172.21.15.83:0/524900398 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f6c8c0a41e0 0x7f6c8c0a66a0 secure :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f6c94006e10 tx=0x7f6c940047d0 comp rx=0 tx=0).stop 2024-04-24T09:09:51.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.298+0000 7f6cae800640 1 --2- 172.21.15.83:0/524900398 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ca81a4970 0x7f6ca81a4e20 unknown :-1 s=CLOSED pgs=2402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:51.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.298+0000 7f6cae800640 1 --2- 172.21.15.83:0/524900398 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6ca8109580 0x7f6ca81101f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:51.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.298+0000 7f6cae800640 1 --2- 172.21.15.83:0/524900398 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ca81044b0 0x7f6ca810fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:51.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.298+0000 7f6cae800640 1 -- 172.21.15.83:0/524900398 >> 172.21.15.83:0/524900398 conn(0x7f6ca80ffd90 msgr2=0x7f6ca81011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:51.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.298+0000 7f6cae800640 1 -- 172.21.15.83:0/524900398 shutdown_connections 2024-04-24T09:09:51.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.298+0000 7f6cae800640 1 -- 172.21.15.83:0/524900398 wait complete. 2024-04-24T09:09:51.320 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T09:09:51.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.586+0000 7fb4a7e00640 1 -- 172.21.15.83:0/4048896428 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4a8113f50 msgr2=0x7fb4a8116340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:51.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.586+0000 7fb4a7e00640 1 --2- 172.21.15.83:0/4048896428 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4a8113f50 0x7fb4a8116340 secure :-1 s=READY pgs=2405 cs=0 l=1 rev1=1 crypto rx=0x7fb4a0009ca0 tx=0x7fb4a002a940 comp rx=0 tx=0).stop 2024-04-24T09:09:51.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.586+0000 7fb4a7e00640 1 -- 172.21.15.83:0/4048896428 shutdown_connections 2024-04-24T09:09:51.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.586+0000 7fb4a7e00640 1 --2- 172.21.15.83:0/4048896428 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4a8113f50 0x7fb4a8116340 unknown :-1 s=CLOSED pgs=2405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:51.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.586+0000 7fb4a7e00640 1 --2- 172.21.15.83:0/4048896428 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb4a810b2c0 0x7fb4a8113a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:51.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.586+0000 7fb4a7e00640 1 --2- 172.21.15.83:0/4048896428 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4a810a9a0 0x7fb4a810ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:51.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.586+0000 7fb4a7e00640 1 -- 172.21.15.83:0/4048896428 >> 172.21.15.83:0/4048896428 conn(0x7fb4a806cf80 msgr2=0x7fb4a806d390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:51.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.586+0000 7fb4a7e00640 1 -- 172.21.15.83:0/4048896428 shutdown_connections 2024-04-24T09:09:51.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.586+0000 7fb4a7e00640 1 -- 172.21.15.83:0/4048896428 wait complete. 2024-04-24T09:09:51.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.587+0000 7fb4a7e00640 1 Processor -- start 2024-04-24T09:09:51.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.588+0000 7fb4a7e00640 1 -- start start 2024-04-24T09:09:51.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.588+0000 7fb4a7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4a810a9a0 0x7fb4a81a48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:51.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.588+0000 7fb4a7e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb4a810b2c0 0x7fb4a81a4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:51.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.588+0000 7fb4a7e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4a8113f50 0x7fb4a819ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:51.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.588+0000 7fb4a7e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a8118750 con 0x7fb4a810a9a0 2024-04-24T09:09:51.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.588+0000 7fb4a7e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a81185d0 con 0x7fb4a8113f50 2024-04-24T09:09:51.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.588+0000 7fb4a7e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a81188d0 con 0x7fb4a810b2c0 2024-04-24T09:09:51.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.589+0000 7fb4a6a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4a810a9a0 0x7fb4a81a48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:51.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.589+0000 7fb4a6a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4a810a9a0 0x7fb4a81a48c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58446/0 (socket says 172.21.15.83:58446) 2024-04-24T09:09:51.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.589+0000 7fb4a7400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4a8113f50 0x7fb4a819ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:51.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.589+0000 7fb4a6a00640 1 -- 172.21.15.83:0/1602974543 learned_addr learned my addr 172.21.15.83:0/1602974543 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:09:51.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.589+0000 7fb4a6000640 1 --2- 172.21.15.83:0/1602974543 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb4a810b2c0 0x7fb4a81a4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:51.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.589+0000 7fb4a7400640 1 -- 172.21.15.83:0/1602974543 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb4a810b2c0 msgr2=0x7fb4a81a4e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:51.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.589+0000 7fb4a7400640 1 --2- 172.21.15.83:0/1602974543 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb4a810b2c0 0x7fb4a81a4e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:51.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.589+0000 7fb4a7400640 1 -- 172.21.15.83:0/1602974543 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4a810a9a0 msgr2=0x7fb4a81a48c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:51.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.589+0000 7fb4a7400640 1 --2- 172.21.15.83:0/1602974543 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4a810a9a0 0x7fb4a81a48c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:51.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.589+0000 7fb4a7400640 1 -- 172.21.15.83:0/1602974543 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4a819f1f0 con 0x7fb4a8113f50 2024-04-24T09:09:51.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.590+0000 7fb4a7400640 1 --2- 172.21.15.83:0/1602974543 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4a8113f50 0x7fb4a819ea70 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb4a002af30 tx=0x7fb4a002f700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:51.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.590+0000 7fb49f400640 1 -- 172.21.15.83:0/1602974543 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a0034040 con 0x7fb4a8113f50 2024-04-24T09:09:51.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.590+0000 7fb4a7e00640 1 -- 172.21.15.83:0/1602974543 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4a819f480 con 0x7fb4a8113f50 2024-04-24T09:09:51.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.590+0000 7fb4a7e00640 1 -- 172.21.15.83:0/1602974543 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4a81ab780 con 0x7fb4a8113f50 2024-04-24T09:09:51.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.590+0000 7fb49f400640 1 -- 172.21.15.83:0/1602974543 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb4a002fd60 con 0x7fb4a8113f50 2024-04-24T09:09:51.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.590+0000 7fb49f400640 1 -- 172.21.15.83:0/1602974543 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a00042a0 con 0x7fb4a8113f50 2024-04-24T09:09:51.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.599+0000 7fb4b09ec740 1 Processor -- start 2024-04-24T09:09:51.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.599+0000 7fb49f400640 1 -- 172.21.15.83:0/1602974543 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb4a0037020 con 0x7fb4a8113f50 2024-04-24T09:09:51.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.600+0000 7fb4b09ec740 1 -- start start 2024-04-24T09:09:51.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.600+0000 7fb4b09ec740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558956d17350 0x558956d17730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:51.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.600+0000 7fb4b09ec740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558956d17c70 0x558956d21e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:51.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.600+0000 7fb4b09ec740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558956d22340 0x558956d24730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:51.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.600+0000 7fb4b09ec740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x558956c741d0 con 0x558956d17350 2024-04-24T09:09:51.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.600+0000 7fb4b09ec740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558956c1a450 con 0x558956d17c70 2024-04-24T09:09:51.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.600+0000 7fb4b09ec740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x558956c70190 con 0x558956d22340 2024-04-24T09:09:51.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.601+0000 7fb4a6a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558956d17c70 0x558956d21e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:51.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.601+0000 7fb4a6a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558956d17c70 0x558956d21e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:50388/0 (socket says 172.21.15.83:50388) 2024-04-24T09:09:51.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.601+0000 7fb4a6000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558956d17350 0x558956d17730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:51.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.601+0000 7fb4a6000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558956d17350 0x558956d17730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58452/0 (socket says 172.21.15.83:58452) 2024-04-24T09:09:51.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.601+0000 7fb4a6000640 1 -- 172.21.15.83:0/3786377472 learned_addr learned my addr 172.21.15.83:0/3786377472 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:09:51.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.602+0000 7fb49f400640 1 --2- 172.21.15.83:0/1602974543 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb48c0a3de0 0x7fb48c0a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:51.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.602+0000 7fb49f400640 1 -- 172.21.15.83:0/1602974543 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 9583+0+0 (secure 0 0 0) 0x7fb4a00f1960 con 0x7fb4a8113f50 2024-04-24T09:09:51.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.602+0000 7fb4a6000640 1 -- 172.21.15.83:0/3786377472 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558956d22340 msgr2=0x558956d24730 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T09:09:51.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.602+0000 7fb4a6000640 1 --2- 172.21.15.83:0/3786377472 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558956d22340 0x558956d24730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:51.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.602+0000 7fb4a6000640 1 -- 172.21.15.83:0/3786377472 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558956d17c70 msgr2=0x558956d21e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:51.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.602+0000 7fb4a6000640 1 --2- 172.21.15.83:0/3786377472 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558956d17c70 0x558956d21e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:51.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.602+0000 7fb4a6000640 1 -- 172.21.15.83:0/3786377472 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558956cd7310 con 0x558956d17350 2024-04-24T09:09:51.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.602+0000 7fb4a6a00640 1 --2- 172.21.15.83:0/3786377472 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558956d17c70 0x558956d21e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:09:51.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.603+0000 7fb4a7400640 1 --2- 172.21.15.83:0/1602974543 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb48c0a3de0 0x7fb48c0a62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:51.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.603+0000 7fb4a6000640 1 --2- 172.21.15.83:0/3786377472 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558956d17350 0x558956d17730 secure :-1 s=READY pgs=2406 cs=0 l=1 rev1=1 crypto rx=0x7fb494009b40 tx=0x7fb4940113e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=611ca562baeae1d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:51.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.603+0000 7fb493e00640 1 -- 172.21.15.83:0/3786377472 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb494011e10 con 0x558956d17350 2024-04-24T09:09:51.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.603+0000 7fb493e00640 1 -- 172.21.15.83:0/3786377472 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb494012900 con 0x558956d17350 2024-04-24T09:09:51.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.603+0000 7fb493e00640 1 -- 172.21.15.83:0/3786377472 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb49400fa80 con 0x558956d17350 2024-04-24T09:09:51.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.604+0000 7fb4b09ec740 1 -- 172.21.15.83:0/3786377472 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558956d17350 msgr2=0x558956d17730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:51.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.604+0000 7fb4a7400640 1 --2- 172.21.15.83:0/1602974543 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb48c0a3de0 0x7fb48c0a62a0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fb4a00c5d30 tx=0x7fb4a00095c0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:51.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.604+0000 7fb4b09ec740 1 --2- 172.21.15.83:0/3786377472 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558956d17350 0x558956d17730 secure :-1 s=READY pgs=2406 cs=0 l=1 rev1=1 crypto rx=0x7fb494009b40 tx=0x7fb4940113e0 comp rx=0 tx=0).stop 2024-04-24T09:09:51.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.605+0000 7fb4b09ec740 1 -- 172.21.15.83:0/3786377472 shutdown_connections 2024-04-24T09:09:51.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.605+0000 7fb4b09ec740 1 --2- 172.21.15.83:0/3786377472 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558956d22340 0x558956d24730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:51.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.605+0000 7fb4b09ec740 1 --2- 172.21.15.83:0/3786377472 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558956d17c70 0x558956d21e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:51.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.605+0000 7fb4b09ec740 1 --2- 172.21.15.83:0/3786377472 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558956d17350 0x558956d17730 unknown :-1 s=CLOSED pgs=2406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:51.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.605+0000 7fb4b09ec740 1 -- 172.21.15.83:0/3786377472 >> 172.21.15.83:0/3786377472 conn(0x558956d13670 msgr2=0x558956d13a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:51.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.605+0000 7fb4b09ec740 1 -- 172.21.15.83:0/3786377472 shutdown_connections 2024-04-24T09:09:51.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.605+0000 7fb4b09ec740 1 -- 172.21.15.83:0/3786377472 wait complete. 2024-04-24T09:09:51.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.605+0000 7fb4b09ec740 1 -- start start 2024-04-24T09:09:51.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.606+0000 7fb4b09ec740 1 Processor -- start 2024-04-24T09:09:51.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.606+0000 7fb4b09ec740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558956d17350 0x558956d21d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:51.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.606+0000 7fb4b09ec740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558956d17c70 0x558956d1f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:51.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.606+0000 7fb4b09ec740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558956d22340 0x558956d1f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:51.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.606+0000 7fb4b09ec740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x558956d251d0 con 0x558956d17c70 2024-04-24T09:09:51.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.606+0000 7fb4b09ec740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558956c2d810 con 0x558956d22340 2024-04-24T09:09:51.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.606+0000 7fb4b09ec740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x558956d25350 con 0x558956d17350 2024-04-24T09:09:51.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.606+0000 7fb4b09ec740 10 client.0 _ll_register_callbacks cb 0x558956bfb140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:09:51.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.606+0000 7fb4a6a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558956d17c70 0x558956d1f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:51.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.606+0000 7fb4a6a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558956d17c70 0x558956d1f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58464/0 (socket says 172.21.15.83:58464) 2024-04-24T09:09:51.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.606+0000 7fb4a6a00640 1 -- 172.21.15.83:0/3420705759 learned_addr learned my addr 172.21.15.83:0/3420705759 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:09:51.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.607+0000 7fb4a6000640 1 --2- 172.21.15.83:0/3420705759 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558956d17350 0x558956d21d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:51.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.607+0000 7fb4a6000640 1 --2- 172.21.15.83:0/3420705759 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558956d22340 0x558956d1f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:51.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.607+0000 7fb4a6a00640 1 -- 172.21.15.83:0/3420705759 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558956d17350 msgr2=0x558956d21d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:51.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.607+0000 7fb4a6a00640 1 --2- 172.21.15.83:0/3420705759 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558956d17350 0x558956d21d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:51.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.607+0000 7fb4a6a00640 1 -- 172.21.15.83:0/3420705759 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558956d22340 msgr2=0x558956d1f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:51.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.607+0000 7fb4a6a00640 1 --2- 172.21.15.83:0/3420705759 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558956d22340 0x558956d1f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:51.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.607+0000 7fb4a6a00640 1 -- 172.21.15.83:0/3420705759 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558956c3e3f0 con 0x558956d17c70 2024-04-24T09:09:51.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.607+0000 7fb4a6000640 1 --2- 172.21.15.83:0/3420705759 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558956d22340 0x558956d1f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:09:51.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.607+0000 7fb4a6a00640 1 --2- 172.21.15.83:0/3420705759 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558956d17c70 0x558956d1f0b0 secure :-1 s=READY pgs=2407 cs=0 l=1 rev1=1 crypto rx=0x7fb49800bb10 tx=0x7fb49800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=991e24702f5e82a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:51.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.607+0000 7fb490c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:09:51.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.607+0000 7fb490c00640 1 -- 172.21.15.83:0/3420705759 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb49801b640 con 0x558956d17c70 2024-04-24T09:09:51.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.607+0000 7fb490c00640 1 -- 172.21.15.83:0/3420705759 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9583+0+0 (secure 0 0 0) 0x7fb49801b7e0 con 0x558956d17c70 2024-04-24T09:09:51.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.608+0000 7fb4b09ec740 1 -- 172.21.15.83:0/3420705759 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb494012900 con 0x558956d17c70 2024-04-24T09:09:51.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.608+0000 7fb490c00640 1 -- 172.21.15.83:0/3420705759 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7fb498024050 con 0x558956d17c70 2024-04-24T09:09:51.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.608+0000 7fb4b09ec740 10 client.21960 fetch_fsmap learned FSMap version 61 2024-04-24T09:09:51.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.608+0000 7fb4b09ec740 1 -- 172.21.15.83:0/3420705759 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x5589569863f0 con 0x558956d17c70 2024-04-24T09:09:51.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.608+0000 7fb490c00640 1 -- 172.21.15.83:0/3420705759 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb494012900 con 0x558956d17c70 2024-04-24T09:09:51.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.608+0000 7fb490c00640 1 -- 172.21.15.83:0/3420705759 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 61) ==== 8362+0+0 (secure 0 0 0) 0x7fb498019020 con 0x558956d17c70 2024-04-24T09:09:51.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.609+0000 7fb4b09ec740 10 client.21960 fetch_fsmap finished waiting for FSMap version 61 2024-04-24T09:09:51.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.609+0000 7fb4b09ec740 10 client.21960 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:09:51.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.609+0000 7fb4b09ec740 20 client.21960 populate_metadata read hostname 'smithi083' 2024-04-24T09:09:51.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.609+0000 7fb4b09ec740 1 --2- 172.21.15.83:0/3420705759 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x558956d32760 0x558956d34b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:51.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.609+0000 7fb4b09ec740 4 client.21960 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:09:51.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.609+0000 7fb4b09ec740 1 -- 172.21.15.83:0/3420705759 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558956c81bf0 con 0x558956d32760 2024-04-24T09:09:51.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.609+0000 7fb4a6000640 1 --2- 172.21.15.83:0/3420705759 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x558956d32760 0x558956d34b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:51.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.610+0000 7fb4a6000640 1 --2- 172.21.15.83:0/3420705759 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x558956d32760 0x558956d34b70 crc :-1 s=READY pgs=457 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff82ab23395c7afd server_cookie=c1a4452208dfb8f2 in_seq=0 out_seq=0 2024-04-24T09:09:51.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.610+0000 7fb490c00640 10 client.21960 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:09:51.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.610+0000 7fb490c00640 1 -- 172.21.15.83:0/3420705759 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558956c81bf0 con 0x558956d32760 2024-04-24T09:09:51.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.611+0000 7fb490c00640 10 client.21960 handle_command_reply: tid=0 2024-04-24T09:09:51.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.611+0000 7fb4b09ec740 1 client.21960 shutdown 2024-04-24T09:09:51.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.611+0000 7fb4b09ec740 1 -- 172.21.15.83:0/3420705759 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558956d17c70 msgr2=0x558956d1f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:51.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.611+0000 7fb4b09ec740 1 --2- 172.21.15.83:0/3420705759 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558956d17c70 0x558956d1f0b0 secure :-1 s=READY pgs=2407 cs=0 l=1 rev1=1 crypto rx=0x7fb49800bb10 tx=0x7fb49800bf10 comp rx=0 tx=0).stop 2024-04-24T09:09:51.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.611+0000 7fb4b09ec740 1 -- 172.21.15.83:0/3420705759 shutdown_connections 2024-04-24T09:09:51.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.611+0000 7fb4b09ec740 1 --2- 172.21.15.83:0/3420705759 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x558956d32760 0x558956d34b70 crc :-1 s=READY pgs=457 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:51.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.611+0000 7fb4b09ec740 1 --2- 172.21.15.83:0/3420705759 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558956d22340 0x558956d1f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:51.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.611+0000 7fb4b09ec740 1 --2- 172.21.15.83:0/3420705759 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558956d17c70 0x558956d1f0b0 unknown :-1 s=CLOSED pgs=2407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:51.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.611+0000 7fb4b09ec740 1 --2- 172.21.15.83:0/3420705759 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558956d17350 0x558956d21d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:51.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.611+0000 7fb4b09ec740 1 -- 172.21.15.83:0/3420705759 >> 172.21.15.83:0/3420705759 conn(0x558956d13670 msgr2=0x558956d09f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:51.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.611+0000 7fb490c00640 0 client.21960 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:09:51.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.611+0000 7fb4b09ec740 1 -- 172.21.15.83:0/3420705759 shutdown_connections 2024-04-24T09:09:51.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.611+0000 7fb4b09ec740 1 -- 172.21.15.83:0/3420705759 wait complete. 2024-04-24T09:09:51.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.612+0000 7fb4b09ec740 20 client.21960 trim_cache size 0 max 16384 2024-04-24T09:09:51.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.632+0000 7fb4b09ec740 1 Processor -- start 2024-04-24T09:09:51.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.632+0000 7fb4b09ec740 1 -- start start 2024-04-24T09:09:51.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.633+0000 7fb4b09ec740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558956d17350 0x558956d2e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:51.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.633+0000 7fb4b09ec740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558956d17c70 0x558956d191b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:51.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.633+0000 7fb4b09ec740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558956d22340 0x558956d3a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:51.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.633+0000 7fb4b09ec740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb488010100 con 0x558956d17c70 2024-04-24T09:09:51.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.633+0000 7fb4b09ec740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb488011c40 con 0x558956d22340 2024-04-24T09:09:51.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.633+0000 7fb4b09ec740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb4880107d0 con 0x558956d17350 2024-04-24T09:09:51.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.633+0000 7fb4a6000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558956d22340 0x558956d3a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:51.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.633+0000 7fb4a6a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558956d17c70 0x558956d191b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:51.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.634+0000 7fb4a6000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558956d22340 0x558956d3a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:50412/0 (socket says 172.21.15.83:50412) 2024-04-24T09:09:51.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.634+0000 7fb4a6000640 1 -- 172.21.15.83:0/1925830521 learned_addr learned my addr 172.21.15.83:0/1925830521 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:09:51.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.634+0000 7fb4a6000640 1 --2- 172.21.15.83:0/1925830521 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558956d17350 0x558956d2e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:51.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.634+0000 7fb4a6a00640 1 -- 172.21.15.83:0/1925830521 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558956d17350 msgr2=0x558956d2e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:51.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.634+0000 7fb4a6a00640 1 --2- 172.21.15.83:0/1925830521 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558956d17350 0x558956d2e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:51.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.634+0000 7fb4a6a00640 1 -- 172.21.15.83:0/1925830521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558956d22340 msgr2=0x558956d3a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:51.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.634+0000 7fb4a6a00640 1 --2- 172.21.15.83:0/1925830521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558956d22340 0x558956d3a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:51.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.634+0000 7fb4a6a00640 1 -- 172.21.15.83:0/1925830521 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558956a507d0 con 0x558956d17c70 2024-04-24T09:09:51.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.634+0000 7fb4a6000640 1 --2- 172.21.15.83:0/1925830521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558956d22340 0x558956d3a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:09:51.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.635+0000 7fb4a6a00640 1 --2- 172.21.15.83:0/1925830521 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558956d17c70 0x558956d191b0 secure :-1 s=READY pgs=2408 cs=0 l=1 rev1=1 crypto rx=0x7fb498011b90 tx=0x7fb498005c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6d60db262e61c6d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:51.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.635+0000 7fb493400640 1 -- 172.21.15.83:0/1925830521 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb49802c020 con 0x558956d17c70 2024-04-24T09:09:51.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.635+0000 7fb493400640 1 -- 172.21.15.83:0/1925830521 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb4980118f0 con 0x558956d17c70 2024-04-24T09:09:51.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.636+0000 7fb493400640 1 -- 172.21.15.83:0/1925830521 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb498005470 con 0x558956d17c70 2024-04-24T09:09:51.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.636+0000 7fb4b09ec740 1 -- 172.21.15.83:0/1925830521 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558956d17c70 msgr2=0x558956d191b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:51.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.636+0000 7fb4b09ec740 1 --2- 172.21.15.83:0/1925830521 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558956d17c70 0x558956d191b0 secure :-1 s=READY pgs=2408 cs=0 l=1 rev1=1 crypto rx=0x7fb498011b90 tx=0x7fb498005c30 comp rx=0 tx=0).stop 2024-04-24T09:09:51.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.636+0000 7fb4b09ec740 1 -- 172.21.15.83:0/1925830521 shutdown_connections 2024-04-24T09:09:51.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.636+0000 7fb4b09ec740 1 --2- 172.21.15.83:0/1925830521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558956d22340 0x558956d3a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:51.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.636+0000 7fb4b09ec740 1 --2- 172.21.15.83:0/1925830521 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558956d17c70 0x558956d191b0 unknown :-1 s=CLOSED pgs=2408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:51.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.636+0000 7fb4b09ec740 1 --2- 172.21.15.83:0/1925830521 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558956d17350 0x558956d2e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:51.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.636+0000 7fb4b09ec740 1 -- 172.21.15.83:0/1925830521 >> 172.21.15.83:0/1925830521 conn(0x558956d13670 msgr2=0x558956d18aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:51.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.636+0000 7fb4b09ec740 1 -- 172.21.15.83:0/1925830521 shutdown_connections 2024-04-24T09:09:51.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.636+0000 7fb4b09ec740 1 -- 172.21.15.83:0/1925830521 wait complete. 2024-04-24T09:09:51.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.637+0000 7fb4b09ec740 1 -- start start 2024-04-24T09:09:51.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.637+0000 7fb4b09ec740 1 Processor -- start 2024-04-24T09:09:51.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.638+0000 7fb4b09ec740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558956d17350 0x558956d2a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:51.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.638+0000 7fb4b09ec740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558956d17c70 0x558956d22d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:51.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.638+0000 7fb4b09ec740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558956d22340 0x558956d23280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:51.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.638+0000 7fb4b09ec740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb488011570 con 0x558956d17350 2024-04-24T09:09:51.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.638+0000 7fb4b09ec740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb488010ea0 con 0x558956d17c70 2024-04-24T09:09:51.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.638+0000 7fb4b09ec740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb48800e100 con 0x558956d22340 2024-04-24T09:09:51.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.638+0000 7fb4b09ec740 10 client.0 _ll_register_callbacks cb 0x7fb498004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:09:51.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.638+0000 7fb4a6000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558956d17350 0x558956d2a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:51.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.638+0000 7fb4a6000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558956d17350 0x558956d2a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58486/0 (socket says 172.21.15.83:58486) 2024-04-24T09:09:51.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.638+0000 7fb4a6000640 1 -- 172.21.15.83:0/3382642 learned_addr learned my addr 172.21.15.83:0/3382642 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:09:51.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.638+0000 7fb4a6a00640 1 --2- 172.21.15.83:0/3382642 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558956d17c70 0x558956d22d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:51.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.638+0000 7fb4a6000640 1 --2- 172.21.15.83:0/3382642 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558956d22340 0x558956d23280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:51.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.639+0000 7fb4a6a00640 1 -- 172.21.15.83:0/3382642 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558956d22340 msgr2=0x558956d23280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:51.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.639+0000 7fb4a6a00640 1 --2- 172.21.15.83:0/3382642 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558956d22340 0x558956d23280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:51.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.639+0000 7fb4a6a00640 1 -- 172.21.15.83:0/3382642 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558956d17350 msgr2=0x558956d2a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:51.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.639+0000 7fb4a6a00640 1 --2- 172.21.15.83:0/3382642 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558956d17350 0x558956d2a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:51.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.639+0000 7fb4a6a00640 1 -- 172.21.15.83:0/3382642 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558956d23970 con 0x558956d17c70 2024-04-24T09:09:51.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.639+0000 7fb4a6000640 1 --2- 172.21.15.83:0/3382642 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558956d17350 0x558956d2a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:09:51.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.639+0000 7fb4a6a00640 1 --2- 172.21.15.83:0/3382642 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558956d17c70 0x558956d22d40 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb498011c30 tx=0x7fb4980116f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac0693fd17a3fbc7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:51.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.639+0000 7fb490c00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:09:51.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.639+0000 7fb490c00640 1 -- 172.21.15.83:0/3382642 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4980053c0 con 0x558956d17c70 2024-04-24T09:09:51.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.639+0000 7fb4b09ec740 1 -- 172.21.15.83:0/3382642 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4980118f0 con 0x558956d17c70 2024-04-24T09:09:51.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.640+0000 7fb490c00640 1 -- 172.21.15.83:0/3382642 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9583+0+0 (secure 0 0 0) 0x7fb498005560 con 0x558956d17c70 2024-04-24T09:09:51.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.640+0000 7fb490c00640 1 -- 172.21.15.83:0/3382642 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7fb498040050 con 0x558956d17c70 2024-04-24T09:09:51.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.640+0000 7fb4b09ec740 10 client.39086 fetch_fsmap learned FSMap version 61 2024-04-24T09:09:51.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.640+0000 7fb4b09ec740 1 -- 172.21.15.83:0/3382642 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x558956d98cb0 con 0x558956d17c70 2024-04-24T09:09:51.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.640+0000 7fb490c00640 1 -- 172.21.15.83:0/3382642 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb4980118f0 con 0x558956d17c70 2024-04-24T09:09:51.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.641+0000 7fb490c00640 1 -- 172.21.15.83:0/3382642 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 61) ==== 8362+0+0 (secure 0 0 0) 0x7fb498019040 con 0x558956d17c70 2024-04-24T09:09:51.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.641+0000 7fb4b09ec740 10 client.39086 fetch_fsmap finished waiting for FSMap version 61 2024-04-24T09:09:51.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.641+0000 7fb4b09ec740 10 client.39086 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:09:51.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.641+0000 7fb4b09ec740 20 client.39086 populate_metadata read hostname 'smithi083' 2024-04-24T09:09:51.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.641+0000 7fb4b09ec740 1 --2- 172.21.15.83:0/3382642 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x558956d992b0 0x558956d9b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:51.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.641+0000 7fb4b09ec740 4 client.39086 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:09:51.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.641+0000 7fb4b09ec740 1 -- 172.21.15.83:0/3382642 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558956c7ba60 con 0x558956d992b0 2024-04-24T09:09:51.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.641+0000 7fb4a6000640 1 --2- 172.21.15.83:0/3382642 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x558956d992b0 0x558956d9b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:51.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.643+0000 7fb490c00640 10 client.39086 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:09:51.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.643+0000 7fb4a6000640 1 --2- 172.21.15.83:0/3382642 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x558956d992b0 0x558956d9b6d0 crc :-1 s=READY pgs=458 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fbe88fb0d2c4f0b2 server_cookie=a3e6e440078d6fd0 in_seq=0 out_seq=0 2024-04-24T09:09:51.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.643+0000 7fb490c00640 1 -- 172.21.15.83:0/3382642 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558956c7ba60 con 0x558956d992b0 2024-04-24T09:09:51.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.643+0000 7fb490c00640 10 client.39086 handle_command_reply: tid=0 2024-04-24T09:09:51.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.643+0000 7fb4b09ec740 1 client.39086 shutdown 2024-04-24T09:09:51.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.644+0000 7fb4b09ec740 1 -- 172.21.15.83:0/3382642 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558956d17c70 msgr2=0x558956d22d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:51.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.644+0000 7fb4b09ec740 1 --2- 172.21.15.83:0/3382642 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558956d17c70 0x558956d22d40 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb498011c30 tx=0x7fb4980116f0 comp rx=0 tx=0).stop 2024-04-24T09:09:51.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.644+0000 7fb4b09ec740 1 -- 172.21.15.83:0/3382642 shutdown_connections 2024-04-24T09:09:51.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.644+0000 7fb4b09ec740 1 --2- 172.21.15.83:0/3382642 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x558956d992b0 0x558956d9b6d0 crc :-1 s=READY pgs=458 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:51.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.644+0000 7fb4b09ec740 1 --2- 172.21.15.83:0/3382642 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558956d22340 0x558956d23280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:51.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.644+0000 7fb4b09ec740 1 --2- 172.21.15.83:0/3382642 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558956d17c70 0x558956d22d40 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:51.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.644+0000 7fb4b09ec740 1 --2- 172.21.15.83:0/3382642 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558956d17350 0x558956d2a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:51.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.644+0000 7fb490c00640 0 client.39086 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:09:51.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.644+0000 7fb4b09ec740 1 -- 172.21.15.83:0/3382642 >> 172.21.15.83:0/3382642 conn(0x558956d13670 msgr2=0x558956d21170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:51.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.644+0000 7fb4b09ec740 1 -- 172.21.15.83:0/3382642 shutdown_connections 2024-04-24T09:09:51.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.644+0000 7fb4b09ec740 1 -- 172.21.15.83:0/3382642 wait complete. 2024-04-24T09:09:51.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.644+0000 7fb4b09ec740 20 client.39086 trim_cache size 0 max 16384 2024-04-24T09:09:51.645 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:09:51.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.645+0000 7fb4a7e00640 1 -- 172.21.15.83:0/1602974543 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb48c0a3de0 msgr2=0x7fb48c0a62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:51.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.645+0000 7fb4a7e00640 1 --2- 172.21.15.83:0/1602974543 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb48c0a3de0 0x7fb48c0a62a0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fb4a00c5d30 tx=0x7fb4a00095c0 comp rx=0 tx=0).stop 2024-04-24T09:09:51.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.645+0000 7fb4a7e00640 1 -- 172.21.15.83:0/1602974543 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4a8113f50 msgr2=0x7fb4a819ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:51.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.645+0000 7fb4a7e00640 1 --2- 172.21.15.83:0/1602974543 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4a8113f50 0x7fb4a819ea70 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb4a002af30 tx=0x7fb4a002f700 comp rx=0 tx=0).stop 2024-04-24T09:09:51.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.645+0000 7fb4a7e00640 1 -- 172.21.15.83:0/1602974543 shutdown_connections 2024-04-24T09:09:51.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.645+0000 7fb4a7e00640 1 --2- 172.21.15.83:0/1602974543 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb48c0a3de0 0x7fb48c0a62a0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:51.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.645+0000 7fb4a7e00640 1 --2- 172.21.15.83:0/1602974543 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4a8113f50 0x7fb4a819ea70 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:51.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.645+0000 7fb4a7e00640 1 --2- 172.21.15.83:0/1602974543 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb4a810b2c0 0x7fb4a81a4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:51.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.645+0000 7fb4a7e00640 1 --2- 172.21.15.83:0/1602974543 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4a810a9a0 0x7fb4a81a48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:51.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.645+0000 7fb4a7e00640 1 -- 172.21.15.83:0/1602974543 >> 172.21.15.83:0/1602974543 conn(0x7fb4a806cf80 msgr2=0x7fb4a8070c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:51.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.645+0000 7fb4a7e00640 1 -- 172.21.15.83:0/1602974543 shutdown_connections 2024-04-24T09:09:51.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:51.645+0000 7fb4a7e00640 1 -- 172.21.15.83:0/1602974543 wait complete. 2024-04-24T09:09:51.995 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:09:51 smithi083 ceph-mon[27907]: pgmap v574: 129 pgs: 129 active+clean; 594 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.3 MiB/s wr, 206 op/s 2024-04-24T09:09:52.172 INFO:tasks.workunit.client.0.smithi083.stdout: 10 33441 6.79 MB/sec execute 143 sec latency 89.938 ms 2024-04-24T09:09:52.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:09:51 smithi138 ceph-mon[31441]: pgmap v574: 129 pgs: 129 active+clean; 594 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.3 MiB/s wr, 206 op/s 2024-04-24T09:09:52.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:09:51 smithi189 ceph-mon[31791]: pgmap v574: 129 pgs: 129 active+clean; 594 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.3 MiB/s wr, 206 op/s 2024-04-24T09:09:52.666 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T09:09:52.666 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T09:09:52.666 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T09:09:52.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.924+0000 7f0942a00640 1 -- 172.21.15.83:0/1562714997 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f093c104cd0 msgr2=0x7f093c108d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:52.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.924+0000 7f0942a00640 1 --2- 172.21.15.83:0/1562714997 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f093c104cd0 0x7f093c108d10 secure :-1 s=READY pgs=2409 cs=0 l=1 rev1=1 crypto rx=0x7f0928009900 tx=0x7f092802a3f0 comp rx=0 tx=0).stop 2024-04-24T09:09:52.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.924+0000 7f0942a00640 1 -- 172.21.15.83:0/1562714997 shutdown_connections 2024-04-24T09:09:52.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.924+0000 7f0942a00640 1 --2- 172.21.15.83:0/1562714997 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f093c1093f0 0x7f093c110fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:52.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.924+0000 7f0942a00640 1 --2- 172.21.15.83:0/1562714997 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f093c104cd0 0x7f093c108d10 unknown :-1 s=CLOSED pgs=2409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:52.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.924+0000 7f0942a00640 1 --2- 172.21.15.83:0/1562714997 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f093c104320 0x7f093c104700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:52.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.924+0000 7f0942a00640 1 -- 172.21.15.83:0/1562714997 >> 172.21.15.83:0/1562714997 conn(0x7f093c0ffb40 msgr2=0x7f093c101f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:52.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.925+0000 7f0942a00640 1 -- 172.21.15.83:0/1562714997 shutdown_connections 2024-04-24T09:09:52.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.925+0000 7f0942a00640 1 -- 172.21.15.83:0/1562714997 wait complete. 2024-04-24T09:09:52.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.926+0000 7f0942a00640 1 Processor -- start 2024-04-24T09:09:52.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.926+0000 7f0942a00640 1 -- start start 2024-04-24T09:09:52.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.926+0000 7f0942a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f093c104320 0x7f093c10f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:52.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.926+0000 7f0942a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f093c1093f0 0x7f093c10f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:52.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.927+0000 7f0942a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f093c10fbb0 0x7f093c1aaff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:52.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.927+0000 7f0942a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f093c076d40 con 0x7f093c1093f0 2024-04-24T09:09:52.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.927+0000 7f0942a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f093c076bc0 con 0x7f093c104320 2024-04-24T09:09:52.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.927+0000 7f0942a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f093c076ec0 con 0x7f093c10fbb0 2024-04-24T09:09:52.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.927+0000 7f0940c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f093c1093f0 0x7f093c10f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:52.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.927+0000 7f0941600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f093c104320 0x7f093c10f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:52.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.927+0000 7f0941600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f093c104320 0x7f093c10f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:50434/0 (socket says 172.21.15.83:50434) 2024-04-24T09:09:52.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.927+0000 7f0941600640 1 -- 172.21.15.83:0/2783106040 learned_addr learned my addr 172.21.15.83:0/2783106040 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:09:52.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.927+0000 7f0942000640 1 --2- 172.21.15.83:0/2783106040 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f093c10fbb0 0x7f093c1aaff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:52.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.927+0000 7f0941600640 1 -- 172.21.15.83:0/2783106040 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f093c10fbb0 msgr2=0x7f093c1aaff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:52.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.927+0000 7f0941600640 1 --2- 172.21.15.83:0/2783106040 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f093c10fbb0 0x7f093c1aaff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:52.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.927+0000 7f0941600640 1 -- 172.21.15.83:0/2783106040 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f093c1093f0 msgr2=0x7f093c10f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:52.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.928+0000 7f0941600640 1 --2- 172.21.15.83:0/2783106040 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f093c1093f0 0x7f093c10f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:52.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.928+0000 7f0941600640 1 -- 172.21.15.83:0/2783106040 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f093c1ab530 con 0x7f093c104320 2024-04-24T09:09:52.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.928+0000 7f0940c00640 1 --2- 172.21.15.83:0/2783106040 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f093c1093f0 0x7f093c10f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:09:52.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.928+0000 7f0941600640 1 --2- 172.21.15.83:0/2783106040 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f093c104320 0x7f093c10f130 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f093000d950 tx=0x7f093000de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:52.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.928+0000 7f093a000640 1 -- 172.21.15.83:0/2783106040 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0930017070 con 0x7f093c104320 2024-04-24T09:09:52.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.928+0000 7f093a000640 1 -- 172.21.15.83:0/2783106040 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0930009c30 con 0x7f093c104320 2024-04-24T09:09:52.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.928+0000 7f093a000640 1 -- 172.21.15.83:0/2783106040 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f093000f620 con 0x7f093c104320 2024-04-24T09:09:52.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.928+0000 7f0942a00640 1 -- 172.21.15.83:0/2783106040 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f093c1ab760 con 0x7f093c104320 2024-04-24T09:09:52.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.929+0000 7f0942a00640 1 -- 172.21.15.83:0/2783106040 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f093c1abc30 con 0x7f093c104320 2024-04-24T09:09:52.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.932+0000 7f0946fdc740 1 Processor -- start 2024-04-24T09:09:52.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.933+0000 7f0946fdc740 1 -- start start 2024-04-24T09:09:52.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.933+0000 7f093a000640 1 -- 172.21.15.83:0/2783106040 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f09300109a0 con 0x7f093c104320 2024-04-24T09:09:52.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.933+0000 7f0946fdc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569bede0270 0x5569bede0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:52.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.933+0000 7f0946fdc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569bede0b90 0x5569bedead20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:52.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.933+0000 7f0946fdc740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5569bedeb260 0x5569beded650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:52.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.933+0000 7f0946fdc740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5569bed45e70 con 0x5569bede0b90 2024-04-24T09:09:52.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.933+0000 7f0946fdc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5569becec380 con 0x5569bede0270 2024-04-24T09:09:52.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.933+0000 7f0946fdc740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5569bed41e30 con 0x5569bedeb260 2024-04-24T09:09:52.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.934+0000 7f0941600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569bede0b90 0x5569bedead20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:52.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.934+0000 7f0942000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569bede0270 0x5569bede0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:52.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.934+0000 7f0942000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569bede0270 0x5569bede0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:50438/0 (socket says 172.21.15.83:50438) 2024-04-24T09:09:52.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.934+0000 7f0942000640 1 -- 172.21.15.83:0/3125481505 learned_addr learned my addr 172.21.15.83:0/3125481505 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:09:52.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.934+0000 7f0941600640 1 --2- 172.21.15.83:0/3125481505 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569bede0b90 0x5569bedead20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58520/0 (socket says 172.21.15.83:58520) 2024-04-24T09:09:52.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.934+0000 7f0940c00640 1 --2- 172.21.15.83:0/3125481505 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5569bedeb260 0x5569beded650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:52.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.934+0000 7f0942000640 1 -- 172.21.15.83:0/3125481505 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5569bedeb260 msgr2=0x5569beded650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:52.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.934+0000 7f0942000640 1 --2- 172.21.15.83:0/3125481505 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5569bedeb260 0x5569beded650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:52.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.934+0000 7f0942000640 1 -- 172.21.15.83:0/3125481505 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569bede0b90 msgr2=0x5569bedead20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:52.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.934+0000 7f0942000640 1 --2- 172.21.15.83:0/3125481505 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569bede0b90 0x5569bedead20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:52.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.934+0000 7f0942000640 1 -- 172.21.15.83:0/3125481505 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569beb207e0 con 0x5569bede0270 2024-04-24T09:09:52.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.934+0000 7f0941600640 1 --2- 172.21.15.83:0/3125481505 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569bede0b90 0x5569bedead20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:09:52.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.935+0000 7f093a000640 1 --2- 172.21.15.83:0/2783106040 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f09240a3e20 0x7f09240a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:52.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.935+0000 7f0942000640 1 --2- 172.21.15.83:0/3125481505 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569bede0270 0x5569bede0650 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f0934009f50 tx=0x7f09340118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5dee9361bc632747 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:52.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.935+0000 7f092e600640 1 -- 172.21.15.83:0/3125481505 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0934004d20 con 0x5569bede0270 2024-04-24T09:09:52.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.935+0000 7f093a000640 1 -- 172.21.15.83:0/2783106040 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 9583+0+0 (secure 0 0 0) 0x7f09300d0770 con 0x7f093c104320 2024-04-24T09:09:52.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.935+0000 7f092e600640 1 -- 172.21.15.83:0/3125481505 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0934004050 con 0x5569bede0270 2024-04-24T09:09:52.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.935+0000 7f092e600640 1 -- 172.21.15.83:0/3125481505 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0934005280 con 0x5569bede0270 2024-04-24T09:09:52.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.935+0000 7f0941600640 1 --2- 172.21.15.83:0/2783106040 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f09240a3e20 0x7f09240a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:52.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.936+0000 7f0946fdc740 1 -- 172.21.15.83:0/3125481505 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569bede0270 msgr2=0x5569bede0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:52.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.936+0000 7f0946fdc740 1 --2- 172.21.15.83:0/3125481505 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569bede0270 0x5569bede0650 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f0934009f50 tx=0x7f09340118d0 comp rx=0 tx=0).stop 2024-04-24T09:09:52.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.936+0000 7f0946fdc740 1 -- 172.21.15.83:0/3125481505 shutdown_connections 2024-04-24T09:09:52.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.936+0000 7f0946fdc740 1 --2- 172.21.15.83:0/3125481505 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5569bedeb260 0x5569beded650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:52.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.936+0000 7f0946fdc740 1 --2- 172.21.15.83:0/3125481505 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569bede0b90 0x5569bedead20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:52.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.936+0000 7f0946fdc740 1 --2- 172.21.15.83:0/3125481505 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569bede0270 0x5569bede0650 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:52.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.936+0000 7f0946fdc740 1 -- 172.21.15.83:0/3125481505 >> 172.21.15.83:0/3125481505 conn(0x5569beddc590 msgr2=0x5569beddc970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:52.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.936+0000 7f0946fdc740 1 -- 172.21.15.83:0/3125481505 shutdown_connections 2024-04-24T09:09:52.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.936+0000 7f0941600640 1 --2- 172.21.15.83:0/2783106040 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f09240a3e20 0x7f09240a62e0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x5569bede1830 tx=0x7f09300d6530 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:52.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.937+0000 7f0946fdc740 1 -- 172.21.15.83:0/3125481505 wait complete. 2024-04-24T09:09:52.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.937+0000 7f0946fdc740 1 -- start start 2024-04-24T09:09:52.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.937+0000 7f0946fdc740 1 Processor -- start 2024-04-24T09:09:52.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.938+0000 7f0946fdc740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5569bede0270 0x5569bede8220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:52.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.938+0000 7f0946fdc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569bede0b90 0x5569bede8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:52.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.938+0000 7f0946fdc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569bedeb260 0x5569bede8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:52.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.938+0000 7f0946fdc740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5569bedee050 con 0x5569bede0b90 2024-04-24T09:09:52.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.938+0000 7f0946fdc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5569becff660 con 0x5569bedeb260 2024-04-24T09:09:52.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.938+0000 7f0946fdc740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5569bedee1d0 con 0x5569bede0270 2024-04-24T09:09:52.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.938+0000 7f0946fdc740 10 client.0 _ll_register_callbacks cb 0x5569becedaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:09:52.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.939+0000 7f0940c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569bede0b90 0x5569bede8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:52.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.939+0000 7f0942000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569bedeb260 0x5569bede8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:52.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.939+0000 7f0940c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569bede0b90 0x5569bede8760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58526/0 (socket says 172.21.15.83:58526) 2024-04-24T09:09:52.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.939+0000 7f0940c00640 1 -- 172.21.15.83:0/190979408 learned_addr learned my addr 172.21.15.83:0/190979408 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:09:52.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.939+0000 7f0942000640 1 --2- 172.21.15.83:0/190979408 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5569bede0270 0x5569bede8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:52.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.939+0000 7f0940c00640 1 -- 172.21.15.83:0/190979408 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5569bede0270 msgr2=0x5569bede8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:52.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.939+0000 7f0940c00640 1 --2- 172.21.15.83:0/190979408 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5569bede0270 0x5569bede8220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:52.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.939+0000 7f0940c00640 1 -- 172.21.15.83:0/190979408 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569bedeb260 msgr2=0x5569bede8ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:52.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.939+0000 7f0940c00640 1 --2- 172.21.15.83:0/190979408 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569bedeb260 0x5569bede8ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:52.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.939+0000 7f0940c00640 1 -- 172.21.15.83:0/190979408 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569beb80f70 con 0x5569bede0b90 2024-04-24T09:09:52.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.939+0000 7f0942000640 1 --2- 172.21.15.83:0/190979408 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569bedeb260 0x5569bede8ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:09:52.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.940+0000 7f0940c00640 1 --2- 172.21.15.83:0/190979408 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569bede0b90 0x5569bede8760 secure :-1 s=READY pgs=2410 cs=0 l=1 rev1=1 crypto rx=0x7f0928005ab0 tx=0x7f0928005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce62fc93cfe022f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:52.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.940+0000 7f0922a00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:09:52.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.940+0000 7f0922a00640 1 -- 172.21.15.83:0/190979408 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0928035040 con 0x5569bede0b90 2024-04-24T09:09:52.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.940+0000 7f0922a00640 1 -- 172.21.15.83:0/190979408 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9583+0+0 (secure 0 0 0) 0x7f0928004970 con 0x5569bede0b90 2024-04-24T09:09:52.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.940+0000 7f0946fdc740 1 -- 172.21.15.83:0/190979408 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0934004050 con 0x5569bede0b90 2024-04-24T09:09:52.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.941+0000 7f0922a00640 1 -- 172.21.15.83:0/190979408 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f0928004b70 con 0x5569bede0b90 2024-04-24T09:09:52.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.941+0000 7f0946fdc740 10 client.21978 fetch_fsmap learned FSMap version 61 2024-04-24T09:09:52.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.941+0000 7f0946fdc740 1 -- 172.21.15.83:0/190979408 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x5569bea18170 con 0x5569bede0b90 2024-04-24T09:09:52.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.942+0000 7f0922a00640 1 -- 172.21.15.83:0/190979408 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0934004050 con 0x5569bede0b90 2024-04-24T09:09:52.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.942+0000 7f0922a00640 1 -- 172.21.15.83:0/190979408 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 61) ==== 8362+0+0 (secure 0 0 0) 0x7f09280314a0 con 0x5569bede0b90 2024-04-24T09:09:52.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.942+0000 7f0946fdc740 10 client.21978 fetch_fsmap finished waiting for FSMap version 61 2024-04-24T09:09:52.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.942+0000 7f0946fdc740 10 client.21978 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:09:52.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.942+0000 7f0946fdc740 20 client.21978 populate_metadata read hostname 'smithi083' 2024-04-24T09:09:52.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.942+0000 7f0946fdc740 1 --2- 172.21.15.83:0/190979408 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5569bedfb610 0x5569bedfda30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:52.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.942+0000 7f0946fdc740 4 client.21978 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:09:52.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.942+0000 7f0946fdc740 1 -- 172.21.15.83:0/190979408 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5569bed53890 con 0x5569bedfb610 2024-04-24T09:09:52.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.942+0000 7f0942000640 1 --2- 172.21.15.83:0/190979408 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5569bedfb610 0x5569bedfda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:52.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.943+0000 7f0942000640 1 --2- 172.21.15.83:0/190979408 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5569bedfb610 0x5569bedfda30 crc :-1 s=READY pgs=459 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25e170a5cfcfea7e server_cookie=38898a44b984043d in_seq=0 out_seq=0 2024-04-24T09:09:52.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.943+0000 7f0922a00640 10 client.21978 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:09:52.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.944+0000 7f0922a00640 1 -- 172.21.15.83:0/190979408 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5569bed53890 con 0x5569bedfb610 2024-04-24T09:09:52.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.944+0000 7f0922a00640 10 client.21978 handle_command_reply: tid=0 2024-04-24T09:09:52.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.944+0000 7f0946fdc740 1 client.21978 shutdown 2024-04-24T09:09:52.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.945+0000 7f0946fdc740 1 -- 172.21.15.83:0/190979408 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569bede0b90 msgr2=0x5569bede8760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:52.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.945+0000 7f0946fdc740 1 --2- 172.21.15.83:0/190979408 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569bede0b90 0x5569bede8760 secure :-1 s=READY pgs=2410 cs=0 l=1 rev1=1 crypto rx=0x7f0928005ab0 tx=0x7f0928005270 comp rx=0 tx=0).stop 2024-04-24T09:09:52.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.945+0000 7f0946fdc740 1 -- 172.21.15.83:0/190979408 shutdown_connections 2024-04-24T09:09:52.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.945+0000 7f0946fdc740 1 --2- 172.21.15.83:0/190979408 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5569bedfb610 0x5569bedfda30 crc :-1 s=READY pgs=459 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:52.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.945+0000 7f0946fdc740 1 --2- 172.21.15.83:0/190979408 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569bedeb260 0x5569bede8ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:52.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.945+0000 7f0946fdc740 1 --2- 172.21.15.83:0/190979408 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569bede0b90 0x5569bede8760 unknown :-1 s=CLOSED pgs=2410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:52.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.945+0000 7f0946fdc740 1 --2- 172.21.15.83:0/190979408 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5569bede0270 0x5569bede8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:52.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.945+0000 7f0946fdc740 1 -- 172.21.15.83:0/190979408 >> 172.21.15.83:0/190979408 conn(0x5569beddc590 msgr2=0x5569bede2390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:52.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.945+0000 7f0922a00640 0 client.21978 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:09:52.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.945+0000 7f0946fdc740 1 -- 172.21.15.83:0/190979408 shutdown_connections 2024-04-24T09:09:52.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.945+0000 7f0946fdc740 1 -- 172.21.15.83:0/190979408 wait complete. 2024-04-24T09:09:52.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.945+0000 7f0946fdc740 20 client.21978 trim_cache size 0 max 16384 2024-04-24T09:09:52.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.979+0000 7f0946fdc740 1 Processor -- start 2024-04-24T09:09:52.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.979+0000 7f0946fdc740 1 -- start start 2024-04-24T09:09:52.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.980+0000 7f0946fdc740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5569bede0270 0x5569bedf6f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:52.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.980+0000 7f0946fdc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569bedeb260 0x5569bede8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:52.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.980+0000 7f0946fdc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569bede7390 0x5569bee03300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:52.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.980+0000 7f0946fdc740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f093c1b5b00 con 0x5569bedeb260 2024-04-24T09:09:52.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.980+0000 7f0946fdc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f093c1b7640 con 0x5569bede7390 2024-04-24T09:09:52.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.980+0000 7f0946fdc740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f093c1b61d0 con 0x5569bede0270 2024-04-24T09:09:52.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.980+0000 7f0940c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569bedeb260 0x5569bede8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:52.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.980+0000 7f0942000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569bede7390 0x5569bee03300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:52.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.980+0000 7f0942000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569bede7390 0x5569bee03300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:50458/0 (socket says 172.21.15.83:50458) 2024-04-24T09:09:52.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.980+0000 7f0942000640 1 -- 172.21.15.83:0/2681263310 learned_addr learned my addr 172.21.15.83:0/2681263310 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:09:52.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.980+0000 7f0942000640 1 --2- 172.21.15.83:0/2681263310 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5569bede0270 0x5569bedf6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:52.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.981+0000 7f0942000640 1 -- 172.21.15.83:0/2681263310 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5569bede0270 msgr2=0x5569bedf6f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:52.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.981+0000 7f0942000640 1 --2- 172.21.15.83:0/2681263310 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5569bede0270 0x5569bedf6f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:52.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.981+0000 7f0942000640 1 -- 172.21.15.83:0/2681263310 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569bedeb260 msgr2=0x5569bede8ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:52.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.981+0000 7f0942000640 1 --2- 172.21.15.83:0/2681263310 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569bedeb260 0x5569bede8ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:52.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.981+0000 7f0942000640 1 -- 172.21.15.83:0/2681263310 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569bedd7e80 con 0x5569bede7390 2024-04-24T09:09:52.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.982+0000 7f0942000640 1 --2- 172.21.15.83:0/2681263310 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569bede7390 0x5569bee03300 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f0934000f80 tx=0x7f09340047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4475fcb9aa6f46f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:52.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.982+0000 7f092dc00640 1 -- 172.21.15.83:0/2681263310 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0934003e30 con 0x5569bede7390 2024-04-24T09:09:52.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.982+0000 7f092dc00640 1 -- 172.21.15.83:0/2681263310 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f09340044b0 con 0x5569bede7390 2024-04-24T09:09:52.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.982+0000 7f092dc00640 1 -- 172.21.15.83:0/2681263310 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0934028a50 con 0x5569bede7390 2024-04-24T09:09:52.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.983+0000 7f0946fdc740 1 -- 172.21.15.83:0/2681263310 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569bede7390 msgr2=0x5569bee03300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:52.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.983+0000 7f0946fdc740 1 --2- 172.21.15.83:0/2681263310 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569bede7390 0x5569bee03300 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f0934000f80 tx=0x7f09340047b0 comp rx=0 tx=0).stop 2024-04-24T09:09:52.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.983+0000 7f0946fdc740 1 -- 172.21.15.83:0/2681263310 shutdown_connections 2024-04-24T09:09:52.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.983+0000 7f0946fdc740 1 --2- 172.21.15.83:0/2681263310 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569bede7390 0x5569bee03300 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:52.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.983+0000 7f0946fdc740 1 --2- 172.21.15.83:0/2681263310 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569bedeb260 0x5569bede8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:52.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.983+0000 7f0946fdc740 1 --2- 172.21.15.83:0/2681263310 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5569bede0270 0x5569bedf6f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:52.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.983+0000 7f0946fdc740 1 -- 172.21.15.83:0/2681263310 >> 172.21.15.83:0/2681263310 conn(0x5569beddc590 msgr2=0x5569bede2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:52.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.984+0000 7f0946fdc740 1 -- 172.21.15.83:0/2681263310 shutdown_connections 2024-04-24T09:09:52.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.984+0000 7f0946fdc740 1 -- 172.21.15.83:0/2681263310 wait complete. 2024-04-24T09:09:52.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.984+0000 7f0946fdc740 1 -- start start 2024-04-24T09:09:52.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.984+0000 7f0946fdc740 1 Processor -- start 2024-04-24T09:09:52.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.985+0000 7f0946fdc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569bede0270 0x5569bedef510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:52.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.985+0000 7f0946fdc740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5569bede7390 0x5569bedefa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:52.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.985+0000 7f0946fdc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569bedeb260 0x5569bee60b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:52.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.985+0000 7f0946fdc740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f093c1b6f70 con 0x5569bede0270 2024-04-24T09:09:52.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.985+0000 7f0946fdc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f093c1b68a0 con 0x5569bedeb260 2024-04-24T09:09:52.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.985+0000 7f0946fdc740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f093c1b3b00 con 0x5569bede7390 2024-04-24T09:09:52.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.985+0000 7f0946fdc740 10 client.0 _ll_register_callbacks cb 0x7f0928009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:09:52.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.986+0000 7f0942000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569bede0270 0x5569bedef510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:52.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.986+0000 7f0940c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5569bede7390 0x5569bedefa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:52.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.986+0000 7f0940c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5569bede7390 0x5569bedefa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:41962/0 (socket says 172.21.15.83:41962) 2024-04-24T09:09:52.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.986+0000 7f0940c00640 1 -- 172.21.15.83:0/4097809885 learned_addr learned my addr 172.21.15.83:0/4097809885 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:09:52.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.986+0000 7f0942000640 1 --2- 172.21.15.83:0/4097809885 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569bedeb260 0x5569bee60b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:52.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.986+0000 7f0942000640 1 -- 172.21.15.83:0/4097809885 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5569bede7390 msgr2=0x5569bedefa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:52.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.986+0000 7f0942000640 1 --2- 172.21.15.83:0/4097809885 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5569bede7390 0x5569bedefa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:52.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.986+0000 7f0942000640 1 -- 172.21.15.83:0/4097809885 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569bedeb260 msgr2=0x5569bee60b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:52.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.986+0000 7f0942000640 1 --2- 172.21.15.83:0/4097809885 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569bedeb260 0x5569bee60b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:52.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.987+0000 7f0942000640 1 -- 172.21.15.83:0/4097809885 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569bee610b0 con 0x5569bede0270 2024-04-24T09:09:52.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.987+0000 7f0942000640 1 --2- 172.21.15.83:0/4097809885 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569bede0270 0x5569bedef510 secure :-1 s=READY pgs=2411 cs=0 l=1 rev1=1 crypto rx=0x7f093401b040 tx=0x7f09340094e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52269fc302ec523b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:52.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.987+0000 7f0922a00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:09:52.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.987+0000 7f0922a00640 1 -- 172.21.15.83:0/4097809885 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0934004000 con 0x5569bede0270 2024-04-24T09:09:52.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.987+0000 7f0922a00640 1 -- 172.21.15.83:0/4097809885 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9583+0+0 (secure 0 0 0) 0x7f09340041a0 con 0x5569bede0270 2024-04-24T09:09:52.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.987+0000 7f0946fdc740 1 -- 172.21.15.83:0/4097809885 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09340044b0 con 0x5569bede0270 2024-04-24T09:09:52.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.988+0000 7f0922a00640 1 -- 172.21.15.83:0/4097809885 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f0934028450 con 0x5569bede0270 2024-04-24T09:09:52.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.988+0000 7f0946fdc740 10 client.21984 fetch_fsmap learned FSMap version 61 2024-04-24T09:09:52.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.988+0000 7f0946fdc740 1 -- 172.21.15.83:0/4097809885 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x5569bee622f0 con 0x5569bede0270 2024-04-24T09:09:52.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.989+0000 7f0922a00640 1 -- 172.21.15.83:0/4097809885 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f09340044b0 con 0x5569bede0270 2024-04-24T09:09:52.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.989+0000 7f0922a00640 1 -- 172.21.15.83:0/4097809885 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 61) ==== 8362+0+0 (secure 0 0 0) 0x7f0934034020 con 0x5569bede0270 2024-04-24T09:09:52.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.989+0000 7f0946fdc740 10 client.21984 fetch_fsmap finished waiting for FSMap version 61 2024-04-24T09:09:52.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.989+0000 7f0946fdc740 10 client.21984 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:09:52.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.989+0000 7f0946fdc740 20 client.21984 populate_metadata read hostname 'smithi083' 2024-04-24T09:09:52.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.989+0000 7f0946fdc740 1 --2- 172.21.15.83:0/4097809885 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5569bee62680 0x5569bee64a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:52.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.989+0000 7f0946fdc740 4 client.21984 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T09:09:52.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.989+0000 7f0946fdc740 1 -- 172.21.15.83:0/4097809885 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5569bed4d700 con 0x5569bee62680 2024-04-24T09:09:52.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.990+0000 7f0940c00640 1 --2- 172.21.15.83:0/4097809885 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5569bee62680 0x5569bee64a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:52.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.991+0000 7f0940c00640 1 --2- 172.21.15.83:0/4097809885 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5569bee62680 0x5569bee64a70 crc :-1 s=READY pgs=460 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e0d26966a8d9213 server_cookie=983f860283f26bef in_seq=0 out_seq=0 2024-04-24T09:09:52.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.991+0000 7f0922a00640 10 client.21984 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:09:52.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.992+0000 7f0922a00640 1 -- 172.21.15.83:0/4097809885 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5569bed4d700 con 0x5569bee62680 2024-04-24T09:09:52.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.992+0000 7f0922a00640 10 client.21984 handle_command_reply: tid=0 2024-04-24T09:09:52.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.993+0000 7f0946fdc740 1 client.21984 shutdown 2024-04-24T09:09:52.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.995+0000 7f0946fdc740 1 -- 172.21.15.83:0/4097809885 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569bede0270 msgr2=0x5569bedef510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:52.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.995+0000 7f0946fdc740 1 --2- 172.21.15.83:0/4097809885 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569bede0270 0x5569bedef510 secure :-1 s=READY pgs=2411 cs=0 l=1 rev1=1 crypto rx=0x7f093401b040 tx=0x7f09340094e0 comp rx=0 tx=0).stop 2024-04-24T09:09:52.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.995+0000 7f0946fdc740 1 -- 172.21.15.83:0/4097809885 shutdown_connections 2024-04-24T09:09:52.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.995+0000 7f0946fdc740 1 --2- 172.21.15.83:0/4097809885 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5569bee62680 0x5569bee64a70 crc :-1 s=READY pgs=460 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:52.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.995+0000 7f0946fdc740 1 --2- 172.21.15.83:0/4097809885 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569bedeb260 0x5569bee60b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:52.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.995+0000 7f0946fdc740 1 --2- 172.21.15.83:0/4097809885 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5569bede7390 0x5569bedefa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:52.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.995+0000 7f0922a00640 0 client.21984 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:09:52.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.995+0000 7f0946fdc740 1 --2- 172.21.15.83:0/4097809885 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569bede0270 0x5569bedef510 unknown :-1 s=CLOSED pgs=2411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:52.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.996+0000 7f0946fdc740 1 -- 172.21.15.83:0/4097809885 >> 172.21.15.83:0/4097809885 conn(0x5569beddc590 msgr2=0x5569beddae60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:52.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.996+0000 7f0946fdc740 1 -- 172.21.15.83:0/4097809885 shutdown_connections 2024-04-24T09:09:52.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.996+0000 7f0946fdc740 1 -- 172.21.15.83:0/4097809885 wait complete. 2024-04-24T09:09:52.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.997+0000 7f0946fdc740 20 client.21984 trim_cache size 0 max 16384 2024-04-24T09:09:52.997 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T09:09:52.997 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-04-24T09:09:52.998 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "21fb61f3-d7e4-4786-b9ef-249f4d91bd5f", 2024-04-24T09:09:52.998 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-04-24T09:09:52.998 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T09:09:52.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.997+0000 7f0942a00640 1 -- 172.21.15.83:0/2783106040 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f09240a3e20 msgr2=0x7f09240a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:52.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.997+0000 7f0942a00640 1 --2- 172.21.15.83:0/2783106040 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f09240a3e20 0x7f09240a62e0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x5569bede1830 tx=0x7f09300d6530 comp rx=0 tx=0).stop 2024-04-24T09:09:52.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.997+0000 7f0942a00640 1 -- 172.21.15.83:0/2783106040 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f093c104320 msgr2=0x7f093c10f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:52.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.997+0000 7f0942a00640 1 --2- 172.21.15.83:0/2783106040 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f093c104320 0x7f093c10f130 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f093000d950 tx=0x7f093000de10 comp rx=0 tx=0).stop 2024-04-24T09:09:52.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.998+0000 7f0942a00640 1 -- 172.21.15.83:0/2783106040 shutdown_connections 2024-04-24T09:09:52.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.998+0000 7f0942a00640 1 --2- 172.21.15.83:0/2783106040 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f09240a3e20 0x7f09240a62e0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:52.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.998+0000 7f0942a00640 1 --2- 172.21.15.83:0/2783106040 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f093c10fbb0 0x7f093c1aaff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:52.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.998+0000 7f0942a00640 1 --2- 172.21.15.83:0/2783106040 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f093c1093f0 0x7f093c10f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:52.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.998+0000 7f0942a00640 1 --2- 172.21.15.83:0/2783106040 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f093c104320 0x7f093c10f130 secure :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f093000d950 tx=0x7f093000de10 comp rx=0 tx=0).stop 2024-04-24T09:09:52.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.998+0000 7f0942a00640 1 -- 172.21.15.83:0/2783106040 >> 172.21.15.83:0/2783106040 conn(0x7f093c0ffb40 msgr2=0x7f093c100ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:52.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.998+0000 7f0942a00640 1 -- 172.21.15.83:0/2783106040 shutdown_connections 2024-04-24T09:09:52.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:52.999+0000 7f0942a00640 1 -- 172.21.15.83:0/2783106040 wait complete. 2024-04-24T09:09:53.019 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T09:09:53.019 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T09:09:53.173 INFO:tasks.workunit.client.0.smithi083.stdout: 10 33572 6.82 MB/sec execute 144 sec latency 102.379 ms 2024-04-24T09:09:53.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.280+0000 7ff4adc00640 1 -- 172.21.15.83:0/2810714938 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff4a8100e60 msgr2=0x7ff4a8101240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:53.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.280+0000 7ff4adc00640 1 --2- 172.21.15.83:0/2810714938 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff4a8100e60 0x7ff4a8101240 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff498009900 tx=0x7ff49802a480 comp rx=0 tx=0).stop 2024-04-24T09:09:53.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.280+0000 7ff4adc00640 1 -- 172.21.15.83:0/2810714938 shutdown_connections 2024-04-24T09:09:53.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.280+0000 7ff4adc00640 1 --2- 172.21.15.83:0/2810714938 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4a8106260 0x7ff4a81066f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:53.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.280+0000 7ff4adc00640 1 --2- 172.21.15.83:0/2810714938 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff4a8101780 0x7ff4a8105c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:53.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.280+0000 7ff4adc00640 1 --2- 172.21.15.83:0/2810714938 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff4a8100e60 0x7ff4a8101240 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:53.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.280+0000 7ff4adc00640 1 -- 172.21.15.83:0/2810714938 >> 172.21.15.83:0/2810714938 conn(0x7ff4a80fc850 msgr2=0x7ff4a80fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:53.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.280+0000 7ff4adc00640 1 -- 172.21.15.83:0/2810714938 shutdown_connections 2024-04-24T09:09:53.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.280+0000 7ff4adc00640 1 -- 172.21.15.83:0/2810714938 wait complete. 2024-04-24T09:09:53.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.281+0000 7ff4adc00640 1 Processor -- start 2024-04-24T09:09:53.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.282+0000 7ff4adc00640 1 -- start start 2024-04-24T09:09:53.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.282+0000 7ff4adc00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff4a8100e60 0x7ff4a819ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:53.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.282+0000 7ff4adc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4a8101780 0x7ff4a819f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:53.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.282+0000 7ff4adc00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff4a8106260 0x7ff4a81a3620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:53.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.282+0000 7ff4adc00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff4a8118700 con 0x7ff4a8101780 2024-04-24T09:09:53.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.282+0000 7ff4adc00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff4a8118580 con 0x7ff4a8106260 2024-04-24T09:09:53.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.282+0000 7ff4adc00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff4a8118880 con 0x7ff4a8100e60 2024-04-24T09:09:53.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.283+0000 7ff4ad200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff4a8106260 0x7ff4a81a3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:53.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.283+0000 7ff4a7400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4a8101780 0x7ff4a819f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:53.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.283+0000 7ff4ad200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff4a8106260 0x7ff4a81a3620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:50488/0 (socket says 172.21.15.83:50488) 2024-04-24T09:09:53.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.283+0000 7ff4ad200640 1 -- 172.21.15.83:0/1749495226 learned_addr learned my addr 172.21.15.83:0/1749495226 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:09:53.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.283+0000 7ff4a7e00640 1 --2- 172.21.15.83:0/1749495226 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff4a8100e60 0x7ff4a819ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:53.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.283+0000 7ff4ad200640 1 -- 172.21.15.83:0/1749495226 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff4a8100e60 msgr2=0x7ff4a819ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:53.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.283+0000 7ff4ad200640 1 --2- 172.21.15.83:0/1749495226 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff4a8100e60 0x7ff4a819ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:53.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.283+0000 7ff4ad200640 1 -- 172.21.15.83:0/1749495226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4a8101780 msgr2=0x7ff4a819f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:53.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.283+0000 7ff4ad200640 1 --2- 172.21.15.83:0/1749495226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4a8101780 0x7ff4a819f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:53.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.283+0000 7ff4ad200640 1 -- 172.21.15.83:0/1749495226 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4a81a3d50 con 0x7ff4a8106260 2024-04-24T09:09:53.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.284+0000 7ff4ad200640 1 --2- 172.21.15.83:0/1749495226 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff4a8106260 0x7ff4a81a3620 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff4a0009fd0 tx=0x7ff4a000b260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:53.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.284+0000 7ff4a4c00640 1 -- 172.21.15.83:0/1749495226 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a0017070 con 0x7ff4a8106260 2024-04-24T09:09:53.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.284+0000 7ff4a4c00640 1 -- 172.21.15.83:0/1749495226 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff4a0003e20 con 0x7ff4a8106260 2024-04-24T09:09:53.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.284+0000 7ff4a4c00640 1 -- 172.21.15.83:0/1749495226 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a0010460 con 0x7ff4a8106260 2024-04-24T09:09:53.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.284+0000 7ff4adc00640 1 -- 172.21.15.83:0/1749495226 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4a81a4040 con 0x7ff4a8106260 2024-04-24T09:09:53.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.284+0000 7ff4adc00640 1 -- 172.21.15.83:0/1749495226 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4a81ab8d0 con 0x7ff4a8106260 2024-04-24T09:09:53.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.288+0000 7ff4b202a740 1 Processor -- start 2024-04-24T09:09:53.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.289+0000 7ff4a4c00640 1 -- 172.21.15.83:0/1749495226 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff4a0003970 con 0x7ff4a8106260 2024-04-24T09:09:53.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.290+0000 7ff4b202a740 1 -- start start 2024-04-24T09:09:53.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.290+0000 7ff4b202a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5577968b3a30 0x5577968b3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:53.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.290+0000 7ff4b202a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577968b4350 0x5577968be4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:53.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.290+0000 7ff4b202a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5577968bea20 0x5577968c0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:53.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.290+0000 7ff4b202a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5577968191f0 con 0x5577968b4350 2024-04-24T09:09:53.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.290+0000 7ff4b202a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5577967bf350 con 0x5577968bea20 2024-04-24T09:09:53.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.290+0000 7ff4b202a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5577968151b0 con 0x5577968b3a30 2024-04-24T09:09:53.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.290+0000 7ff4a7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577968b4350 0x5577968be4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:53.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.290+0000 7ff4a7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577968b4350 0x5577968be4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58590/0 (socket says 172.21.15.83:58590) 2024-04-24T09:09:53.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.290+0000 7ff4a7e00640 1 -- 172.21.15.83:0/3047628770 learned_addr learned my addr 172.21.15.83:0/3047628770 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:09:53.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.291+0000 7ff4a7400640 1 --2- 172.21.15.83:0/3047628770 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5577968b3a30 0x5577968b3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:53.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.291+0000 7ff4a7e00640 1 -- 172.21.15.83:0/3047628770 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5577968b3a30 msgr2=0x5577968b3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:53.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.291+0000 7ff4a7e00640 1 --2- 172.21.15.83:0/3047628770 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5577968b3a30 0x5577968b3e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:53.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.291+0000 7ff4a7e00640 1 -- 172.21.15.83:0/3047628770 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5577968bea20 msgr2=0x5577968c0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:53.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.291+0000 7ff4a7400640 1 --2- 172.21.15.83:0/3047628770 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5577968bea20 0x5577968c0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:53.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.291+0000 7ff4a7e00640 1 --2- 172.21.15.83:0/3047628770 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5577968bea20 0x5577968c0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:53.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.291+0000 7ff4a7e00640 1 -- 172.21.15.83:0/3047628770 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557796813f50 con 0x5577968b4350 2024-04-24T09:09:53.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.291+0000 7ff4a7400640 1 --2- 172.21.15.83:0/3047628770 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5577968bea20 0x5577968c0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:09:53.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.291+0000 7ff4a4c00640 1 --2- 172.21.15.83:0/1749495226 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff48c0a3e70 0x7ff48c0a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:53.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.292+0000 7ff4a7e00640 1 --2- 172.21.15.83:0/3047628770 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577968b4350 0x5577968be4e0 secure :-1 s=READY pgs=2412 cs=0 l=1 rev1=1 crypto rx=0x7ff498009900 tx=0x7ff498005e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5c544d59d428191 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:53.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.292+0000 7ff4a4c00640 1 -- 172.21.15.83:0/1749495226 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 9583+0+0 (secure 0 0 0) 0x7ff4a00d07f0 con 0x7ff4a8106260 2024-04-24T09:09:53.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.292+0000 7ff49d600640 1 -- 172.21.15.83:0/3047628770 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff498040070 con 0x5577968b4350 2024-04-24T09:09:53.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.292+0000 7ff49d600640 1 -- 172.21.15.83:0/3047628770 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff4980054d0 con 0x5577968b4350 2024-04-24T09:09:53.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.292+0000 7ff49d600640 1 -- 172.21.15.83:0/3047628770 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff498035040 con 0x5577968b4350 2024-04-24T09:09:53.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.292+0000 7ff4a7400640 1 --2- 172.21.15.83:0/1749495226 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff48c0a3e70 0x7ff48c0a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:53.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.293+0000 7ff4b202a740 1 -- 172.21.15.83:0/3047628770 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577968b4350 msgr2=0x5577968be4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:53.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.293+0000 7ff4b202a740 1 --2- 172.21.15.83:0/3047628770 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577968b4350 0x5577968be4e0 secure :-1 s=READY pgs=2412 cs=0 l=1 rev1=1 crypto rx=0x7ff498009900 tx=0x7ff498005e00 comp rx=0 tx=0).stop 2024-04-24T09:09:53.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.293+0000 7ff4b202a740 1 -- 172.21.15.83:0/3047628770 shutdown_connections 2024-04-24T09:09:53.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.293+0000 7ff4b202a740 1 --2- 172.21.15.83:0/3047628770 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5577968bea20 0x5577968c0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:53.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.293+0000 7ff4b202a740 1 --2- 172.21.15.83:0/3047628770 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577968b4350 0x5577968be4e0 unknown :-1 s=CLOSED pgs=2412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:53.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.293+0000 7ff4b202a740 1 --2- 172.21.15.83:0/3047628770 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5577968b3a30 0x5577968b3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:53.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.293+0000 7ff4b202a740 1 -- 172.21.15.83:0/3047628770 >> 172.21.15.83:0/3047628770 conn(0x5577968afd50 msgr2=0x5577968b0130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:53.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.293+0000 7ff4a7400640 1 --2- 172.21.15.83:0/1749495226 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff48c0a3e70 0x7ff48c0a6330 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x5577968b0ee0 tx=0x7ff494004730 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:53.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.293+0000 7ff4b202a740 1 -- 172.21.15.83:0/3047628770 shutdown_connections 2024-04-24T09:09:53.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.294+0000 7ff4b202a740 1 -- 172.21.15.83:0/3047628770 wait complete. 2024-04-24T09:09:53.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.294+0000 7ff4b202a740 1 -- start start 2024-04-24T09:09:53.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.294+0000 7ff4b202a740 1 Processor -- start 2024-04-24T09:09:53.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.295+0000 7ff4b202a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577968b3a30 0x5577968bb920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:53.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.295+0000 7ff4b202a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5577968b4350 0x5577968bbe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:53.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.295+0000 7ff4b202a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5577968bea20 0x5577968bc3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:53.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.295+0000 7ff4b202a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5577968d27c0 con 0x5577968b3a30 2024-04-24T09:09:53.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.295+0000 7ff4b202a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5577967d2690 con 0x5577968b4350 2024-04-24T09:09:53.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.295+0000 7ff4b202a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5577968d2940 con 0x5577968bea20 2024-04-24T09:09:53.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.295+0000 7ff4b202a740 10 client.0 _ll_register_callbacks cb 0x55779680e650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:09:53.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.295+0000 7ff4a7400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5577968b4350 0x5577968bbe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:53.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.295+0000 7ff4a7400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5577968b4350 0x5577968bbe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:50502/0 (socket says 172.21.15.83:50502) 2024-04-24T09:09:53.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.295+0000 7ff4a7400640 1 -- 172.21.15.83:0/1330235024 learned_addr learned my addr 172.21.15.83:0/1330235024 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:09:53.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.295+0000 7ff4a7e00640 1 --2- 172.21.15.83:0/1330235024 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577968b3a30 0x5577968bb920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:53.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.296+0000 7ff4ad200640 1 --2- 172.21.15.83:0/1330235024 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5577968bea20 0x5577968bc3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:53.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.296+0000 7ff4a7e00640 1 -- 172.21.15.83:0/1330235024 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5577968bea20 msgr2=0x5577968bc3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:53.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.296+0000 7ff4a7e00640 1 --2- 172.21.15.83:0/1330235024 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5577968bea20 0x5577968bc3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:53.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.296+0000 7ff4a7e00640 1 -- 172.21.15.83:0/1330235024 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5577968b4350 msgr2=0x5577968bbe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:53.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.296+0000 7ff4a7e00640 1 --2- 172.21.15.83:0/1330235024 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5577968b4350 0x5577968bbe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:53.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.296+0000 7ff4a7e00640 1 -- 172.21.15.83:0/1330235024 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577967d1920 con 0x5577968b3a30 2024-04-24T09:09:53.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.296+0000 7ff4a7e00640 1 --2- 172.21.15.83:0/1330235024 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577968b3a30 0x5577968bb920 secure :-1 s=READY pgs=2413 cs=0 l=1 rev1=1 crypto rx=0x7ff49802a4c0 tx=0x7ff498004e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d2c9a08e1b8d9d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:53.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.296+0000 7ff492000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:09:53.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.297+0000 7ff492000640 1 -- 172.21.15.83:0/1330235024 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff498040030 con 0x5577968b3a30 2024-04-24T09:09:53.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.297+0000 7ff492000640 1 -- 172.21.15.83:0/1330235024 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9583+0+0 (secure 0 0 0) 0x7ff498005230 con 0x5577968b3a30 2024-04-24T09:09:53.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.297+0000 7ff4b202a740 1 -- 172.21.15.83:0/1330235024 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4980054d0 con 0x5577968b3a30 2024-04-24T09:09:53.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.297+0000 7ff492000640 1 -- 172.21.15.83:0/1330235024 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7ff498054050 con 0x5577968b3a30 2024-04-24T09:09:53.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.297+0000 7ff4b202a740 10 client.21996 fetch_fsmap learned FSMap version 61 2024-04-24T09:09:53.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.297+0000 7ff4b202a740 1 -- 172.21.15.83:0/1330235024 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x5577965f57d0 con 0x5577968b3a30 2024-04-24T09:09:53.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.298+0000 7ff492000640 1 -- 172.21.15.83:0/1330235024 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff4980054d0 con 0x5577968b3a30 2024-04-24T09:09:53.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.298+0000 7ff492000640 1 -- 172.21.15.83:0/1330235024 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 61) ==== 8362+0+0 (secure 0 0 0) 0x7ff498031c10 con 0x5577968b3a30 2024-04-24T09:09:53.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.298+0000 7ff4b202a740 10 client.21996 fetch_fsmap finished waiting for FSMap version 61 2024-04-24T09:09:53.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.298+0000 7ff4b202a740 10 client.21996 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:09:53.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.298+0000 7ff4b202a740 20 client.21996 populate_metadata read hostname 'smithi083' 2024-04-24T09:09:53.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.298+0000 7ff4b202a740 1 --2- 172.21.15.83:0/1330235024 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5577968dfae0 0x5577968e1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:53.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.298+0000 7ff4b202a740 4 client.21996 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:09:53.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.298+0000 7ff4b202a740 1 -- 172.21.15.83:0/1330235024 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557796826c10 con 0x5577968dfae0 2024-04-24T09:09:53.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.299+0000 7ff4a7400640 1 --2- 172.21.15.83:0/1330235024 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5577968dfae0 0x5577968e1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:53.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.300+0000 7ff4a7400640 1 --2- 172.21.15.83:0/1330235024 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5577968dfae0 0x5577968e1ed0 crc :-1 s=READY pgs=461 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5288b0a64de99ee3 server_cookie=49533ad4492acf17 in_seq=0 out_seq=0 2024-04-24T09:09:53.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.300+0000 7ff492000640 10 client.21996 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:09:53.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.301+0000 7ff492000640 1 -- 172.21.15.83:0/1330235024 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557796826c10 con 0x5577968dfae0 2024-04-24T09:09:53.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.301+0000 7ff492000640 10 client.21996 handle_command_reply: tid=0 2024-04-24T09:09:53.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.301+0000 7ff4b202a740 1 client.21996 shutdown 2024-04-24T09:09:53.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.301+0000 7ff4b202a740 1 -- 172.21.15.83:0/1330235024 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577968b3a30 msgr2=0x5577968bb920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:53.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.301+0000 7ff4b202a740 1 --2- 172.21.15.83:0/1330235024 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577968b3a30 0x5577968bb920 secure :-1 s=READY pgs=2413 cs=0 l=1 rev1=1 crypto rx=0x7ff49802a4c0 tx=0x7ff498004e60 comp rx=0 tx=0).stop 2024-04-24T09:09:53.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.301+0000 7ff4b202a740 1 -- 172.21.15.83:0/1330235024 shutdown_connections 2024-04-24T09:09:53.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.301+0000 7ff4b202a740 1 --2- 172.21.15.83:0/1330235024 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5577968dfae0 0x5577968e1ed0 crc :-1 s=READY pgs=461 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:53.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.302+0000 7ff4b202a740 1 --2- 172.21.15.83:0/1330235024 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5577968bea20 0x5577968bc3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:53.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.302+0000 7ff4b202a740 1 --2- 172.21.15.83:0/1330235024 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5577968b4350 0x5577968bbe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:53.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.302+0000 7ff4b202a740 1 --2- 172.21.15.83:0/1330235024 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577968b3a30 0x5577968bb920 unknown :-1 s=CLOSED pgs=2413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:53.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.302+0000 7ff4b202a740 1 -- 172.21.15.83:0/1330235024 >> 172.21.15.83:0/1330235024 conn(0x5577968afd50 msgr2=0x5577968aefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:53.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.302+0000 7ff492000640 0 client.21996 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:09:53.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.302+0000 7ff4b202a740 1 -- 172.21.15.83:0/1330235024 shutdown_connections 2024-04-24T09:09:53.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.302+0000 7ff4b202a740 1 -- 172.21.15.83:0/1330235024 wait complete. 2024-04-24T09:09:53.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.302+0000 7ff4b202a740 20 client.21996 trim_cache size 0 max 16384 2024-04-24T09:09:53.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.332+0000 7ff4b202a740 1 Processor -- start 2024-04-24T09:09:53.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.332+0000 7ff4b202a740 1 -- start start 2024-04-24T09:09:53.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.333+0000 7ff4b202a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5577968b3a30 0x5577968b61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:53.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.333+0000 7ff4b202a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577968b4350 0x5577968bc1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:53.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.333+0000 7ff4b202a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5577968bea20 0x5577968e7500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:53.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.333+0000 7ff4b202a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff4a81ad060 con 0x5577968b4350 2024-04-24T09:09:53.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.333+0000 7ff4b202a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff4a81aeba0 con 0x5577968b3a30 2024-04-24T09:09:53.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.333+0000 7ff4b202a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff4a81ad730 con 0x5577968bea20 2024-04-24T09:09:53.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.333+0000 7ff4a7e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5577968b3a30 0x5577968b61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:53.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.333+0000 7ff4a7e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5577968b3a30 0x5577968b61d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:50512/0 (socket says 172.21.15.83:50512) 2024-04-24T09:09:53.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.333+0000 7ff4a7e00640 1 -- 172.21.15.83:0/175596678 learned_addr learned my addr 172.21.15.83:0/175596678 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:09:53.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.333+0000 7ff4a7400640 1 --2- 172.21.15.83:0/175596678 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577968b4350 0x5577968bc1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:53.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.333+0000 7ff4ad200640 1 --2- 172.21.15.83:0/175596678 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5577968bea20 0x5577968e7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:53.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.334+0000 7ff4a7e00640 1 -- 172.21.15.83:0/175596678 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5577968bea20 msgr2=0x5577968e7500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:53.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.334+0000 7ff4a7e00640 1 --2- 172.21.15.83:0/175596678 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5577968bea20 0x5577968e7500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:53.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.334+0000 7ff4a7e00640 1 -- 172.21.15.83:0/175596678 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577968b4350 msgr2=0x5577968bc1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:53.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.334+0000 7ff4a7e00640 1 --2- 172.21.15.83:0/175596678 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577968b4350 0x5577968bc1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:53.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.334+0000 7ff4a7e00640 1 -- 172.21.15.83:0/175596678 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55779652b3f0 con 0x5577968b3a30 2024-04-24T09:09:53.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.334+0000 7ff4a7e00640 1 --2- 172.21.15.83:0/175596678 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5577968b3a30 0x5577968b61d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff498004aa0 tx=0x7ff498005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c32be5cde166586 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:53.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.334+0000 7ff49cc00640 1 -- 172.21.15.83:0/175596678 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff49805d070 con 0x5577968b3a30 2024-04-24T09:09:53.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.334+0000 7ff49cc00640 1 -- 172.21.15.83:0/175596678 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff4980047d0 con 0x5577968b3a30 2024-04-24T09:09:53.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.335+0000 7ff49cc00640 1 -- 172.21.15.83:0/175596678 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff498040040 con 0x5577968b3a30 2024-04-24T09:09:53.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.335+0000 7ff4b202a740 1 -- 172.21.15.83:0/175596678 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5577968b3a30 msgr2=0x5577968b61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:53.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.335+0000 7ff4b202a740 1 --2- 172.21.15.83:0/175596678 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5577968b3a30 0x5577968b61d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff498004aa0 tx=0x7ff498005740 comp rx=0 tx=0).stop 2024-04-24T09:09:53.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.336+0000 7ff4b202a740 1 -- 172.21.15.83:0/175596678 shutdown_connections 2024-04-24T09:09:53.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.336+0000 7ff4b202a740 1 --2- 172.21.15.83:0/175596678 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5577968bea20 0x5577968e7500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:53.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.336+0000 7ff4b202a740 1 --2- 172.21.15.83:0/175596678 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577968b4350 0x5577968bc1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:53.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.336+0000 7ff4b202a740 1 --2- 172.21.15.83:0/175596678 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5577968b3a30 0x5577968b61d0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:53.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.336+0000 7ff4b202a740 1 -- 172.21.15.83:0/175596678 >> 172.21.15.83:0/175596678 conn(0x5577968afd50 msgr2=0x5577968aefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:53.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.336+0000 7ff4b202a740 1 -- 172.21.15.83:0/175596678 shutdown_connections 2024-04-24T09:09:53.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.336+0000 7ff4b202a740 1 -- 172.21.15.83:0/175596678 wait complete. 2024-04-24T09:09:53.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.336+0000 7ff4b202a740 1 -- start start 2024-04-24T09:09:53.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.336+0000 7ff4b202a740 1 Processor -- start 2024-04-24T09:09:53.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.337+0000 7ff4b202a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577968b3a30 0x5577968d37b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:53.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.337+0000 7ff4b202a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5577968b4350 0x5577968d3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:53.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.337+0000 7ff4b202a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5577968bea20 0x5577968d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:53.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.337+0000 7ff4b202a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff4a81ae4d0 con 0x5577968b3a30 2024-04-24T09:09:53.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.337+0000 7ff4b202a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff4a81ade00 con 0x5577968bea20 2024-04-24T09:09:53.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.337+0000 7ff4b202a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff4a80684b0 con 0x5577968b4350 2024-04-24T09:09:53.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.337+0000 7ff4b202a740 10 client.0 _ll_register_callbacks cb 0x7ff498005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:09:53.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.337+0000 7ff4a7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577968b3a30 0x5577968d37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:53.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.337+0000 7ff4ad200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5577968bea20 0x5577968d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:53.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.337+0000 7ff4ad200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5577968bea20 0x5577968d4230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:50522/0 (socket says 172.21.15.83:50522) 2024-04-24T09:09:53.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.338+0000 7ff4ad200640 1 -- 172.21.15.83:0/2385511053 learned_addr learned my addr 172.21.15.83:0/2385511053 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:09:53.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.338+0000 7ff4a7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577968b3a30 0x5577968d37b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58616/0 (socket says 172.21.15.83:58616) 2024-04-24T09:09:53.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.338+0000 7ff4a7400640 1 --2- 172.21.15.83:0/2385511053 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5577968b4350 0x5577968d3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:53.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.338+0000 7ff4ad200640 1 -- 172.21.15.83:0/2385511053 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5577968b4350 msgr2=0x5577968d3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:53.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.338+0000 7ff4ad200640 1 --2- 172.21.15.83:0/2385511053 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5577968b4350 0x5577968d3cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:53.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.338+0000 7ff4ad200640 1 -- 172.21.15.83:0/2385511053 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577968b3a30 msgr2=0x5577968d37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:53.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.338+0000 7ff4ad200640 1 --2- 172.21.15.83:0/2385511053 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577968b3a30 0x5577968d37b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:53.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.338+0000 7ff4ad200640 1 -- 172.21.15.83:0/2385511053 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577964eb0f0 con 0x5577968bea20 2024-04-24T09:09:53.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.338+0000 7ff4a7e00640 1 --2- 172.21.15.83:0/2385511053 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577968b3a30 0x5577968d37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:09:53.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.338+0000 7ff4ad200640 1 --2- 172.21.15.83:0/2385511053 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5577968bea20 0x5577968d4230 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff4a0098df0 tx=0x7ff4a0003d20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f66f66383da2fd6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:53.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.338+0000 7ff492000640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:09:53.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.338+0000 7ff492000640 1 -- 172.21.15.83:0/2385511053 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a00b4020 con 0x5577968bea20 2024-04-24T09:09:53.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.339+0000 7ff492000640 1 -- 172.21.15.83:0/2385511053 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9583+0+0 (secure 0 0 0) 0x7ff4a0098a90 con 0x5577968bea20 2024-04-24T09:09:53.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.339+0000 7ff4b202a740 1 -- 172.21.15.83:0/2385511053 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4980047d0 con 0x5577968bea20 2024-04-24T09:09:53.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.339+0000 7ff492000640 1 -- 172.21.15.83:0/2385511053 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7ff4a0020040 con 0x5577968bea20 2024-04-24T09:09:53.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.339+0000 7ff4b202a740 10 client.39119 fetch_fsmap learned FSMap version 61 2024-04-24T09:09:53.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.339+0000 7ff4b202a740 1 -- 172.21.15.83:0/2385511053 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x5577967de810 con 0x5577968bea20 2024-04-24T09:09:53.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.339+0000 7ff492000640 1 -- 172.21.15.83:0/2385511053 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff4980047d0 con 0x5577968bea20 2024-04-24T09:09:53.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.339+0000 7ff492000640 1 -- 172.21.15.83:0/2385511053 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 61) ==== 8362+0+0 (secure 0 0 0) 0x7ff4a0098170 con 0x5577968bea20 2024-04-24T09:09:53.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.339+0000 7ff4b202a740 10 client.39119 fetch_fsmap finished waiting for FSMap version 61 2024-04-24T09:09:53.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.339+0000 7ff4b202a740 10 client.39119 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:09:53.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.339+0000 7ff4b202a740 20 client.39119 populate_metadata read hostname 'smithi083' 2024-04-24T09:09:53.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.339+0000 7ff4b202a740 1 --2- 172.21.15.83:0/2385511053 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x557796946670 0x557796948a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:53.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.339+0000 7ff4b202a740 4 client.39119 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T09:09:53.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.339+0000 7ff4b202a740 1 -- 172.21.15.83:0/2385511053 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557796820a80 con 0x557796946670 2024-04-24T09:09:53.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.340+0000 7ff4a7e00640 1 --2- 172.21.15.83:0/2385511053 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x557796946670 0x557796948a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:53.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.341+0000 7ff4a7e00640 1 --2- 172.21.15.83:0/2385511053 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x557796946670 0x557796948a90 crc :-1 s=READY pgs=462 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63084ffbe1567f06 server_cookie=bd598ce40efe76f8 in_seq=0 out_seq=0 2024-04-24T09:09:53.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.341+0000 7ff492000640 10 client.39119 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:09:53.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.341+0000 7ff492000640 1 -- 172.21.15.83:0/2385511053 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x557796820a80 con 0x557796946670 2024-04-24T09:09:53.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.341+0000 7ff492000640 10 client.39119 handle_command_reply: tid=0 2024-04-24T09:09:53.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.341+0000 7ff4b202a740 1 client.39119 shutdown 2024-04-24T09:09:53.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.341+0000 7ff4b202a740 1 -- 172.21.15.83:0/2385511053 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5577968bea20 msgr2=0x5577968d4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:53.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.341+0000 7ff4b202a740 1 --2- 172.21.15.83:0/2385511053 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5577968bea20 0x5577968d4230 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff4a0098df0 tx=0x7ff4a0003d20 comp rx=0 tx=0).stop 2024-04-24T09:09:53.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.342+0000 7ff4b202a740 1 -- 172.21.15.83:0/2385511053 shutdown_connections 2024-04-24T09:09:53.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.342+0000 7ff4b202a740 1 --2- 172.21.15.83:0/2385511053 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x557796946670 0x557796948a90 crc :-1 s=READY pgs=462 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:53.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.342+0000 7ff4b202a740 1 --2- 172.21.15.83:0/2385511053 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5577968bea20 0x5577968d4230 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:53.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.342+0000 7ff492000640 0 client.39119 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:09:53.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.342+0000 7ff4b202a740 1 --2- 172.21.15.83:0/2385511053 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5577968b4350 0x5577968d3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:53.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.342+0000 7ff4b202a740 1 --2- 172.21.15.83:0/2385511053 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577968b3a30 0x5577968d37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:53.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.342+0000 7ff4b202a740 1 -- 172.21.15.83:0/2385511053 >> 172.21.15.83:0/2385511053 conn(0x5577968afd50 msgr2=0x5577968bd150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:53.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.342+0000 7ff4b202a740 1 -- 172.21.15.83:0/2385511053 shutdown_connections 2024-04-24T09:09:53.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.342+0000 7ff4b202a740 1 -- 172.21.15.83:0/2385511053 wait complete. 2024-04-24T09:09:53.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.342+0000 7ff4b202a740 20 client.39119 trim_cache size 0 max 16384 2024-04-24T09:09:53.345 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T09:09:53.345 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T09:09:53.345 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-04-24T09:09:53.345 INFO:teuthology.orchestra.run.smithi083.stdout: "21fb61f3-d7e4-4786-b9ef-249f4d91bd5f": { 2024-04-24T09:09:53.345 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-04-24T09:09:53.345 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "21fb61f3-d7e4-4786-b9ef-249f4d91bd5f", 2024-04-24T09:09:53.345 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-04-24T09:09:53.345 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:09:53.345 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:09:53.345 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T09:09:53.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.343+0000 7ff4adc00640 1 -- 172.21.15.83:0/1749495226 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff48c0a3e70 msgr2=0x7ff48c0a6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:53.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.343+0000 7ff4adc00640 1 --2- 172.21.15.83:0/1749495226 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff48c0a3e70 0x7ff48c0a6330 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x5577968b0ee0 tx=0x7ff494004730 comp rx=0 tx=0).stop 2024-04-24T09:09:53.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.343+0000 7ff4adc00640 1 -- 172.21.15.83:0/1749495226 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff4a8106260 msgr2=0x7ff4a81a3620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:53.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.343+0000 7ff4adc00640 1 --2- 172.21.15.83:0/1749495226 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff4a8106260 0x7ff4a81a3620 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff4a0009fd0 tx=0x7ff4a000b260 comp rx=0 tx=0).stop 2024-04-24T09:09:53.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.343+0000 7ff4adc00640 1 -- 172.21.15.83:0/1749495226 shutdown_connections 2024-04-24T09:09:53.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.343+0000 7ff4adc00640 1 --2- 172.21.15.83:0/1749495226 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff48c0a3e70 0x7ff48c0a6330 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:53.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.343+0000 7ff4adc00640 1 --2- 172.21.15.83:0/1749495226 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff4a8106260 0x7ff4a81a3620 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:53.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.343+0000 7ff4adc00640 1 --2- 172.21.15.83:0/1749495226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4a8101780 0x7ff4a819f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:53.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.343+0000 7ff4adc00640 1 --2- 172.21.15.83:0/1749495226 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff4a8100e60 0x7ff4a819ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:53.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.343+0000 7ff4adc00640 1 -- 172.21.15.83:0/1749495226 >> 172.21.15.83:0/1749495226 conn(0x7ff4a80fc850 msgr2=0x7ff4a80fd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:53.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.343+0000 7ff4adc00640 1 -- 172.21.15.83:0/1749495226 shutdown_connections 2024-04-24T09:09:53.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:09:53.343+0000 7ff4adc00640 1 -- 172.21.15.83:0/1749495226 wait complete. 2024-04-24T09:09:53.366 INFO:tasks.cephfs.filesystem:scrub status for tag:21fb61f3-d7e4-4786-b9ef-249f4d91bd5f - {'path': '/', 'tag': '21fb61f3-d7e4-4786-b9ef-249f4d91bd5f', 'options': 'recursive,force'} 2024-04-24T09:09:54.173 INFO:tasks.workunit.client.0.smithi083.stdout: 10 33696 6.80 MB/sec execute 145 sec latency 92.364 ms 2024-04-24T09:09:54.306 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:09:54 smithi083 ceph-mon[27907]: pgmap v575: 129 pgs: 129 active+clean; 598 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.8 MiB/s wr, 294 op/s 2024-04-24T09:09:54.306 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:09:54 smithi083 ceph-mon[27907]: scrub queued for path: / 2024-04-24T09:09:54.306 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:09:54 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:09:54.306 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:09:54 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:09:54.306 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:09:54 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:09:54.306 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:09:54 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:09:54.306 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:09:54 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:09:54.306 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:09:54 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:09:54.306 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:09:54 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:09:54.307 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:09:54 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:09:54.307 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:09:54 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:09:54.307 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:09:54 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:09:54.307 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:09:54 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:09:54.307 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:09:54 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:09:54.307 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:09:54 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:09:54.307 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:09:54 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:09:54.307 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:09:54 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:09:54.307 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:09:54 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:09:54.307 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:09:54 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:09:54.307 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:09:54 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:09:54.307 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:09:54 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:09:54.307 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:09:54 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:09:54.307 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:09:54 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:09:54.307 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:09:54 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:09:54.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:09:54 smithi138 ceph-mon[31441]: pgmap v575: 129 pgs: 129 active+clean; 598 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.8 MiB/s wr, 294 op/s 2024-04-24T09:09:54.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:09:54 smithi138 ceph-mon[31441]: scrub queued for path: / 2024-04-24T09:09:54.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:09:54 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:09:54.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:09:54 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:09:54.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:09:54 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:09:54.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:09:54 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:09:54.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:09:54 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:09:54.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:09:54 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:09:54.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:09:54 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:09:54.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:09:54 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:09:54.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:09:54 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:09:54.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:09:54 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:09:54.457 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:09:54 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:09:54.457 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:09:54 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:09:54.457 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:09:54 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:09:54.457 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:09:54 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:09:54.457 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:09:54 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:09:54.457 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:09:54 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:09:54.457 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:09:54 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:09:54.457 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:09:54 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:09:54.457 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:09:54 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:09:54.457 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:09:54 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:09:54.457 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:09:54 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:09:54.457 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:09:54 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:09:54.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:09:54 smithi189 ceph-mon[31791]: pgmap v575: 129 pgs: 129 active+clean; 598 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.8 MiB/s wr, 294 op/s 2024-04-24T09:09:54.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:09:54 smithi189 ceph-mon[31791]: scrub queued for path: / 2024-04-24T09:09:54.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:09:54 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:09:54.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:09:54 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:09:54.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:09:54 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:09:54.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:09:54 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:09:54.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:09:54 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:09:54.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:09:54 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:09:54.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:09:54 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:09:54.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:09:54 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:09:54.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:09:54 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:09:54.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:09:54 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:09:54.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:09:54 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:09:54.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:09:54 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:09:54.459 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:09:54 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:09:54.459 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:09:54 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:09:54.459 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:09:54 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:09:54.459 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:09:54 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:09:54.459 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:09:54 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:09:54.459 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:09:54 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:09:54.459 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:09:54 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:09:54.459 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:09:54 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:09:54.459 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:09:54 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:09:54.459 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:09:54 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:09:55.173 INFO:tasks.workunit.client.0.smithi083.stdout: 10 33787 6.80 MB/sec execute 146 sec latency 445.789 ms 2024-04-24T09:09:55.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:09:55 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:09:55.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:09:55 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:09:55.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:09:55 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:09:55.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:09:55 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:09:55.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:09:55 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:09:55.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:09:55 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:09:55.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:09:55 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:09:55.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:09:55 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:09:55.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:09:55 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:09:55.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:09:55 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:09:55.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:09:55 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:09:55.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:09:55 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:09:55.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:09:55 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:09:55.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:09:55 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:09:55.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:09:55 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:09:56.173 INFO:tasks.workunit.client.0.smithi083.stdout: 10 33879 6.77 MB/sec execute 147 sec latency 395.808 ms 2024-04-24T09:09:56.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:09:56 smithi083 ceph-mon[27907]: pgmap v576: 129 pgs: 129 active+clean; 598 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.6 MiB/s wr, 205 op/s 2024-04-24T09:09:56.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:09:56 smithi138 ceph-mon[31441]: pgmap v576: 129 pgs: 129 active+clean; 598 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.6 MiB/s wr, 205 op/s 2024-04-24T09:09:56.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:09:56 smithi189 ceph-mon[31791]: pgmap v576: 129 pgs: 129 active+clean; 598 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.6 MiB/s wr, 205 op/s 2024-04-24T09:09:57.173 INFO:tasks.workunit.client.0.smithi083.stdout: 10 33996 6.76 MB/sec execute 148 sec latency 80.999 ms 2024-04-24T09:09:58.173 INFO:tasks.workunit.client.0.smithi083.stdout: 10 34114 6.78 MB/sec execute 149 sec latency 76.767 ms 2024-04-24T09:09:58.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:09:58 smithi083 ceph-mon[27907]: pgmap v577: 129 pgs: 129 active+clean; 598 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.6 MiB/s wr, 205 op/s 2024-04-24T09:09:58.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:09:58 smithi138 ceph-mon[31441]: pgmap v577: 129 pgs: 129 active+clean; 598 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.6 MiB/s wr, 205 op/s 2024-04-24T09:09:58.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:09:58 smithi189 ceph-mon[31791]: pgmap v577: 129 pgs: 129 active+clean; 598 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.6 MiB/s wr, 205 op/s 2024-04-24T09:09:58.576 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.577+0000 7face1800640 1 -- 172.21.15.83:0/3473773221 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7facdc074300 msgr2=0x7facdc0746e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:58.576 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.577+0000 7face1800640 1 --2- 172.21.15.83:0/3473773221 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7facdc074300 0x7facdc0746e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7faccc009900 tx=0x7faccc02a480 comp rx=0 tx=0).stop 2024-04-24T09:09:58.577 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.577+0000 7face1800640 1 -- 172.21.15.83:0/3473773221 shutdown_connections 2024-04-24T09:09:58.577 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.577+0000 7face1800640 1 --2- 172.21.15.83:0/3473773221 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7facdc10e8f0 0x7facdc110ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:58.577 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.577+0000 7face1800640 1 --2- 172.21.15.83:0/3473773221 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7facdc074c20 0x7facdc10e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:58.577 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.577+0000 7face1800640 1 --2- 172.21.15.83:0/3473773221 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7facdc074300 0x7facdc0746e0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:58.577 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.577+0000 7face1800640 1 -- 172.21.15.83:0/3473773221 >> 172.21.15.83:0/3473773221 conn(0x7facdc079490 msgr2=0x7facdc0798a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:58.577 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.578+0000 7face1800640 1 -- 172.21.15.83:0/3473773221 shutdown_connections 2024-04-24T09:09:58.577 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.578+0000 7face1800640 1 -- 172.21.15.83:0/3473773221 wait complete. 2024-04-24T09:09:58.578 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.579+0000 7face1800640 1 Processor -- start 2024-04-24T09:09:58.578 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.579+0000 7face1800640 1 -- start start 2024-04-24T09:09:58.579 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.579+0000 7face1800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7facdc074300 0x7facdc1a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:58.579 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.579+0000 7face1800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7facdc074c20 0x7facdc1a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:58.579 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.579+0000 7face1800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7facdc10e8f0 0x7facdc19a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:58.579 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.580+0000 7face1800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7facdc1144e0 con 0x7facdc074c20 2024-04-24T09:09:58.579 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.580+0000 7face1800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7facdc114360 con 0x7facdc10e8f0 2024-04-24T09:09:58.579 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.580+0000 7face1800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7facdc114660 con 0x7facdc074300 2024-04-24T09:09:58.580 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.580+0000 7facdaa00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7facdc074300 0x7facdc1a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:58.580 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.580+0000 7facdaa00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7facdc074300 0x7facdc1a0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:42048/0 (socket says 172.21.15.83:42048) 2024-04-24T09:09:58.580 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.580+0000 7facdaa00640 1 -- 172.21.15.83:0/4002398585 learned_addr learned my addr 172.21.15.83:0/4002398585 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:09:58.580 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.580+0000 7facda000640 1 --2- 172.21.15.83:0/4002398585 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7facdc074c20 0x7facdc1a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:58.580 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.580+0000 7facdb400640 1 --2- 172.21.15.83:0/4002398585 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7facdc10e8f0 0x7facdc19a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:58.580 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.580+0000 7facdaa00640 1 -- 172.21.15.83:0/4002398585 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7facdc10e8f0 msgr2=0x7facdc19a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:58.580 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.580+0000 7facdaa00640 1 --2- 172.21.15.83:0/4002398585 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7facdc10e8f0 0x7facdc19a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:58.580 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.580+0000 7facdaa00640 1 -- 172.21.15.83:0/4002398585 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7facdc074c20 msgr2=0x7facdc1a0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:58.580 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.580+0000 7facdaa00640 1 --2- 172.21.15.83:0/4002398585 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7facdc074c20 0x7facdc1a0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:58.580 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.580+0000 7facdaa00640 1 -- 172.21.15.83:0/4002398585 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facdc19afe0 con 0x7facdc074300 2024-04-24T09:09:58.581 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.581+0000 7facdaa00640 1 --2- 172.21.15.83:0/4002398585 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7facdc074300 0x7facdc1a0610 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7faccc005ab0 tx=0x7faccc003c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:58.581 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.581+0000 7facd3400640 1 -- 172.21.15.83:0/4002398585 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faccc03f070 con 0x7facdc074300 2024-04-24T09:09:58.581 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.581+0000 7facd3400640 1 -- 172.21.15.83:0/4002398585 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faccc031d60 con 0x7facdc074300 2024-04-24T09:09:58.581 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.581+0000 7face1800640 1 -- 172.21.15.83:0/4002398585 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facdc19b270 con 0x7facdc074300 2024-04-24T09:09:58.581 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.581+0000 7facd3400640 1 -- 172.21.15.83:0/4002398585 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faccc035030 con 0x7facdc074300 2024-04-24T09:09:58.582 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.582+0000 7face1800640 1 -- 172.21.15.83:0/4002398585 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facdc1a73b0 con 0x7facdc074300 2024-04-24T09:09:58.586 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.586+0000 7face4655740 1 Processor -- start 2024-04-24T09:09:58.586 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.586+0000 7face4655740 1 -- start start 2024-04-24T09:09:58.586 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.587+0000 7face4655740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5609f22619c0 0x5609f2261da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:58.587 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.588+0000 7face4655740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5609f22622e0 0x5609f226c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:58.587 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.588+0000 7face4655740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5609f226cba0 0x5609f226ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:58.587 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.588+0000 7face4655740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5609f21bf900 con 0x5609f22622e0 2024-04-24T09:09:58.587 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.588+0000 7facdb400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5609f22619c0 0x5609f2261da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:58.587 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.588+0000 7face4655740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5609f2165f20 con 0x5609f226cba0 2024-04-24T09:09:58.587 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.588+0000 7face4655740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5609f21bb8c0 con 0x5609f22619c0 2024-04-24T09:09:58.587 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.588+0000 7facdb400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5609f22619c0 0x5609f2261da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:42052/0 (socket says 172.21.15.83:42052) 2024-04-24T09:09:58.587 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.588+0000 7facdb400640 1 -- 172.21.15.83:0/3974660931 learned_addr learned my addr 172.21.15.83:0/3974660931 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:09:58.589 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.588+0000 7facd3400640 1 -- 172.21.15.83:0/4002398585 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7faccc042070 con 0x7facdc074300 2024-04-24T09:09:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.588+0000 7facdaa00640 1 --2- 172.21.15.83:0/3974660931 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5609f22622e0 0x5609f226c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.589+0000 7facdb400640 1 -- 172.21.15.83:0/3974660931 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5609f226cba0 msgr2=0x5609f226ef90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T09:09:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.589+0000 7facda000640 1 --2- 172.21.15.83:0/3974660931 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5609f226cba0 0x5609f226ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.589+0000 7facdb400640 1 --2- 172.21.15.83:0/3974660931 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5609f226cba0 0x5609f226ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.589+0000 7facdb400640 1 -- 172.21.15.83:0/3974660931 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5609f22622e0 msgr2=0x5609f226c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.589+0000 7facdb400640 1 --2- 172.21.15.83:0/3974660931 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5609f22622e0 0x5609f226c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.589+0000 7facdb400640 1 -- 172.21.15.83:0/3974660931 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609f218ae10 con 0x5609f22619c0 2024-04-24T09:09:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.589+0000 7facdaa00640 1 --2- 172.21.15.83:0/3974660931 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5609f22622e0 0x5609f226c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:09:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.590+0000 7facd3400640 1 --2- 172.21.15.83:0/4002398585 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7facc00a3ec0 0x7facc00a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.590+0000 7facd3400640 1 -- 172.21.15.83:0/4002398585 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 9583+0+0 (secure 0 0 0) 0x7faccc0f0f70 con 0x7facdc074300 2024-04-24T09:09:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.590+0000 7facdb400640 1 --2- 172.21.15.83:0/3974660931 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5609f22619c0 0x5609f2261da0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7facd4009f00 tx=0x7facd4010740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ae718d56884c83d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.591+0000 7facdaa00640 1 --2- 172.21.15.83:0/4002398585 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7facc00a3ec0 0x7facc00a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.592+0000 7facc7e00640 1 -- 172.21.15.83:0/3974660931 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd4016070 con 0x5609f22619c0 2024-04-24T09:09:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.592+0000 7facc7e00640 1 -- 172.21.15.83:0/3974660931 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7facd401bb10 con 0x5609f22619c0 2024-04-24T09:09:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.592+0000 7facc7e00640 1 -- 172.21.15.83:0/3974660931 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd4012c30 con 0x5609f22619c0 2024-04-24T09:09:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.593+0000 7face4655740 1 -- 172.21.15.83:0/3974660931 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5609f22619c0 msgr2=0x5609f2261da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.593+0000 7facdaa00640 1 --2- 172.21.15.83:0/4002398585 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7facc00a3ec0 0x7facc00a6380 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x5609f2262f60 tx=0x7faccc031400 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.593+0000 7face4655740 1 --2- 172.21.15.83:0/3974660931 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5609f22619c0 0x5609f2261da0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7facd4009f00 tx=0x7facd4010740 comp rx=0 tx=0).stop 2024-04-24T09:09:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.593+0000 7face4655740 1 -- 172.21.15.83:0/3974660931 shutdown_connections 2024-04-24T09:09:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.593+0000 7face4655740 1 --2- 172.21.15.83:0/3974660931 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5609f226cba0 0x5609f226ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.593+0000 7face4655740 1 --2- 172.21.15.83:0/3974660931 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5609f22622e0 0x5609f226c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.593+0000 7face4655740 1 --2- 172.21.15.83:0/3974660931 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5609f22619c0 0x5609f2261da0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.593+0000 7face4655740 1 -- 172.21.15.83:0/3974660931 >> 172.21.15.83:0/3974660931 conn(0x5609f225ded0 msgr2=0x5609f225e2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:58.593 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.593+0000 7face4655740 1 -- 172.21.15.83:0/3974660931 shutdown_connections 2024-04-24T09:09:58.593 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.593+0000 7face4655740 1 -- 172.21.15.83:0/3974660931 wait complete. 2024-04-24T09:09:58.594 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.593+0000 7face4655740 1 -- start start 2024-04-24T09:09:58.594 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.594+0000 7face4655740 1 Processor -- start 2024-04-24T09:09:58.594 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.594+0000 7face4655740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5609f22619c0 0x5609f2269ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:58.594 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.594+0000 7face4655740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5609f22622e0 0x5609f226a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:58.594 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.594+0000 7face4655740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5609f226cba0 0x5609f226a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:58.594 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.594+0000 7face4655740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5609f226f7d0 con 0x5609f22619c0 2024-04-24T09:09:58.594 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.594+0000 7face4655740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5609f21792c0 con 0x5609f226cba0 2024-04-24T09:09:58.594 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.594+0000 7face4655740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5609f226f950 con 0x5609f22622e0 2024-04-24T09:09:58.594 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.594+0000 7face4655740 10 client.0 _ll_register_callbacks cb 0x5609f1ed1a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:09:58.594 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.595+0000 7facdb400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5609f22619c0 0x5609f2269ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:58.594 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.595+0000 7facdb400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5609f22619c0 0x5609f2269ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58662/0 (socket says 172.21.15.83:58662) 2024-04-24T09:09:58.594 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.595+0000 7facdb400640 1 -- 172.21.15.83:0/632606970 learned_addr learned my addr 172.21.15.83:0/632606970 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:09:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.595+0000 7facda000640 1 --2- 172.21.15.83:0/632606970 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5609f22622e0 0x5609f226a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.595+0000 7facdb400640 1 -- 172.21.15.83:0/632606970 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5609f22622e0 msgr2=0x5609f226a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.595+0000 7facdb400640 1 --2- 172.21.15.83:0/632606970 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5609f22622e0 0x5609f226a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.595+0000 7facdb400640 1 -- 172.21.15.83:0/632606970 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5609f226cba0 msgr2=0x5609f226a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:58.596 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.595+0000 7facdb400640 1 --2- 172.21.15.83:0/632606970 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5609f226cba0 0x5609f226a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:58.596 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.595+0000 7facdb400640 1 -- 172.21.15.83:0/632606970 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609f2189fd0 con 0x5609f22619c0 2024-04-24T09:09:58.596 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.596+0000 7facdb400640 1 --2- 172.21.15.83:0/632606970 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5609f22619c0 0x5609f2269ba0 secure :-1 s=READY pgs=2414 cs=0 l=1 rev1=1 crypto rx=0x7facd401cee0 tx=0x7facd401cf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff34312ffa459fc9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:58.596 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.596+0000 7facc4c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:09:58.596 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.596+0000 7facc4c00640 1 -- 172.21.15.83:0/632606970 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd4028020 con 0x5609f22619c0 2024-04-24T09:09:58.596 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.596+0000 7facc4c00640 1 -- 172.21.15.83:0/632606970 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9583+0+0 (secure 0 0 0) 0x7facd4003970 con 0x5609f22619c0 2024-04-24T09:09:58.596 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.596+0000 7face4655740 1 -- 172.21.15.83:0/632606970 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7facd401bb10 con 0x5609f22619c0 2024-04-24T09:09:58.597 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.597+0000 7facc4c00640 1 -- 172.21.15.83:0/632606970 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7facd4030050 con 0x5609f22619c0 2024-04-24T09:09:58.597 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.597+0000 7face4655740 10 client.22014 fetch_fsmap learned FSMap version 61 2024-04-24T09:09:58.597 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.597+0000 7face4655740 1 -- 172.21.15.83:0/632606970 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x5609f1ed2210 con 0x5609f22619c0 2024-04-24T09:09:58.597 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.597+0000 7facc4c00640 1 -- 172.21.15.83:0/632606970 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7facd401bb10 con 0x5609f22619c0 2024-04-24T09:09:58.597 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.597+0000 7facc4c00640 1 -- 172.21.15.83:0/632606970 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 61) ==== 8362+0+0 (secure 0 0 0) 0x7facd4023070 con 0x5609f22619c0 2024-04-24T09:09:58.597 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.597+0000 7face4655740 10 client.22014 fetch_fsmap finished waiting for FSMap version 61 2024-04-24T09:09:58.597 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.597+0000 7face4655740 10 client.22014 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:09:58.597 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.597+0000 7face4655740 20 client.22014 populate_metadata read hostname 'smithi083' 2024-04-24T09:09:58.597 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.598+0000 7face4655740 1 --2- 172.21.15.83:0/632606970 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5609f227ccb0 0x5609f227f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:58.597 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.598+0000 7face4655740 4 client.22014 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:09:58.597 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.598+0000 7face4655740 1 -- 172.21.15.83:0/632606970 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5609f21cd320 con 0x5609f227ccb0 2024-04-24T09:09:58.599 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.600+0000 7facda000640 1 --2- 172.21.15.83:0/632606970 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5609f227ccb0 0x5609f227f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:58.600 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.601+0000 7facda000640 1 --2- 172.21.15.83:0/632606970 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5609f227ccb0 0x5609f227f0a0 crc :-1 s=READY pgs=463 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=531e461884a2faa server_cookie=ca956024ff6419fc in_seq=0 out_seq=0 2024-04-24T09:09:58.600 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.601+0000 7facc4c00640 10 client.22014 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:09:58.602 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.602+0000 7facc4c00640 1 -- 172.21.15.83:0/632606970 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5609f21cd320 con 0x5609f227ccb0 2024-04-24T09:09:58.602 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.602+0000 7facc4c00640 10 client.22014 handle_command_reply: tid=0 2024-04-24T09:09:58.602 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.603+0000 7face4655740 1 client.22014 shutdown 2024-04-24T09:09:58.603 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.603+0000 7face4655740 1 -- 172.21.15.83:0/632606970 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5609f22619c0 msgr2=0x5609f2269ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:58.603 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.603+0000 7face4655740 1 --2- 172.21.15.83:0/632606970 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5609f22619c0 0x5609f2269ba0 secure :-1 s=READY pgs=2414 cs=0 l=1 rev1=1 crypto rx=0x7facd401cee0 tx=0x7facd401cf10 comp rx=0 tx=0).stop 2024-04-24T09:09:58.603 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.603+0000 7face4655740 1 -- 172.21.15.83:0/632606970 shutdown_connections 2024-04-24T09:09:58.603 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.603+0000 7face4655740 1 --2- 172.21.15.83:0/632606970 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5609f227ccb0 0x5609f227f0a0 crc :-1 s=READY pgs=463 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:58.603 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.603+0000 7face4655740 1 --2- 172.21.15.83:0/632606970 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5609f226cba0 0x5609f226a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:58.603 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.603+0000 7face4655740 1 --2- 172.21.15.83:0/632606970 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5609f22622e0 0x5609f226a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:58.603 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.603+0000 7face4655740 1 --2- 172.21.15.83:0/632606970 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5609f22619c0 0x5609f2269ba0 unknown :-1 s=CLOSED pgs=2414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:58.603 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.603+0000 7face4655740 1 -- 172.21.15.83:0/632606970 >> 172.21.15.83:0/632606970 conn(0x5609f225ded0 msgr2=0x5609f2254840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:58.603 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.603+0000 7facc4c00640 0 client.22014 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:09:58.603 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.604+0000 7face4655740 1 -- 172.21.15.83:0/632606970 shutdown_connections 2024-04-24T09:09:58.603 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.604+0000 7face4655740 1 -- 172.21.15.83:0/632606970 wait complete. 2024-04-24T09:09:58.604 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.604+0000 7face4655740 20 client.22014 trim_cache size 0 max 16384 2024-04-24T09:09:58.629 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.630+0000 7face4655740 1 Processor -- start 2024-04-24T09:09:58.631 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.630+0000 7face4655740 1 -- start start 2024-04-24T09:09:58.631 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.631+0000 7face4655740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5609f22619c0 0x5609f2254a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:58.631 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.631+0000 7face4655740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5609f22622e0 0x5609f2278690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:58.631 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.631+0000 7face4655740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5609f226cba0 0x5609f2269350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:58.631 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.631+0000 7face4655740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7facdc1ad3c0 con 0x5609f226cba0 2024-04-24T09:09:58.631 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.631+0000 7face4655740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7facdc1aef00 con 0x5609f22622e0 2024-04-24T09:09:58.631 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.631+0000 7face4655740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7facdc1ada90 con 0x5609f22619c0 2024-04-24T09:09:58.631 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.631+0000 7facda000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5609f22622e0 0x5609f2278690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:58.632 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.631+0000 7facda000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5609f22622e0 0x5609f2278690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:50582/0 (socket says 172.21.15.83:50582) 2024-04-24T09:09:58.632 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.631+0000 7facdb400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5609f226cba0 0x5609f2269350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:58.633 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.631+0000 7facda000640 1 -- 172.21.15.83:0/1886485093 learned_addr learned my addr 172.21.15.83:0/1886485093 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:09:58.633 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.632+0000 7facdb400640 1 --2- 172.21.15.83:0/1886485093 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5609f22619c0 0x5609f2254a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:58.634 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.632+0000 7facdb400640 1 -- 172.21.15.83:0/1886485093 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5609f22619c0 msgr2=0x5609f2254a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:58.634 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.632+0000 7facdb400640 1 --2- 172.21.15.83:0/1886485093 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5609f22619c0 0x5609f2254a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:58.634 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.632+0000 7facdb400640 1 -- 172.21.15.83:0/1886485093 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5609f22622e0 msgr2=0x5609f2278690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:58.634 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.632+0000 7facdb400640 1 --2- 172.21.15.83:0/1886485093 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5609f22622e0 0x5609f2278690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:58.634 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.632+0000 7facdb400640 1 -- 172.21.15.83:0/1886485093 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609f1f9e620 con 0x5609f226cba0 2024-04-24T09:09:58.634 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.632+0000 7facda000640 1 --2- 172.21.15.83:0/1886485093 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5609f22622e0 0x5609f2278690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:09:58.634 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.632+0000 7facdb400640 1 --2- 172.21.15.83:0/1886485093 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5609f226cba0 0x5609f2269350 secure :-1 s=READY pgs=2415 cs=0 l=1 rev1=1 crypto rx=0x7facd4009e50 tx=0x7facd4010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de2d03c9c1f6eac7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:58.635 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.633+0000 7facc7400640 1 -- 172.21.15.83:0/1886485093 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd4028070 con 0x5609f226cba0 2024-04-24T09:09:58.635 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.633+0000 7facc7400640 1 -- 172.21.15.83:0/1886485093 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7facd4003e20 con 0x5609f226cba0 2024-04-24T09:09:58.635 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.633+0000 7facc7400640 1 -- 172.21.15.83:0/1886485093 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd4016040 con 0x5609f226cba0 2024-04-24T09:09:58.635 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.634+0000 7face4655740 1 -- 172.21.15.83:0/1886485093 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5609f226cba0 msgr2=0x5609f2269350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:58.635 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.634+0000 7face4655740 1 --2- 172.21.15.83:0/1886485093 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5609f226cba0 0x5609f2269350 secure :-1 s=READY pgs=2415 cs=0 l=1 rev1=1 crypto rx=0x7facd4009e50 tx=0x7facd4010740 comp rx=0 tx=0).stop 2024-04-24T09:09:58.635 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.634+0000 7face4655740 1 -- 172.21.15.83:0/1886485093 shutdown_connections 2024-04-24T09:09:58.635 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.634+0000 7face4655740 1 --2- 172.21.15.83:0/1886485093 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5609f226cba0 0x5609f2269350 unknown :-1 s=CLOSED pgs=2415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:58.635 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.634+0000 7face4655740 1 --2- 172.21.15.83:0/1886485093 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5609f22622e0 0x5609f2278690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:58.635 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.634+0000 7face4655740 1 --2- 172.21.15.83:0/1886485093 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5609f22619c0 0x5609f2254a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:58.635 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.634+0000 7face4655740 1 -- 172.21.15.83:0/1886485093 >> 172.21.15.83:0/1886485093 conn(0x5609f225ded0 msgr2=0x5609f2263c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:58.635 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.634+0000 7face4655740 1 -- 172.21.15.83:0/1886485093 shutdown_connections 2024-04-24T09:09:58.635 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.634+0000 7face4655740 1 -- 172.21.15.83:0/1886485093 wait complete. 2024-04-24T09:09:58.635 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.635+0000 7face4655740 1 -- start start 2024-04-24T09:09:58.635 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.635+0000 7face4655740 1 Processor -- start 2024-04-24T09:09:58.636 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.636+0000 7face4655740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5609f22619c0 0x5609f22db690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:58.637 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.636+0000 7face4655740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5609f22622e0 0x5609f22dbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:58.637 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.636+0000 7face4655740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5609f226cba0 0x5609f2273b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:58.637 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.636+0000 7face4655740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7facdc1ae830 con 0x5609f226cba0 2024-04-24T09:09:58.637 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.636+0000 7face4655740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7facdc1ae160 con 0x5609f22619c0 2024-04-24T09:09:58.637 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.636+0000 7face4655740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7facdc1ab3c0 con 0x5609f22622e0 2024-04-24T09:09:58.637 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.636+0000 7face4655740 10 client.0 _ll_register_callbacks cb 0x7facd4009d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:09:58.637 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.636+0000 7facdb400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5609f22619c0 0x5609f22db690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:58.637 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.636+0000 7facdb400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5609f22619c0 0x5609f22db690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:50590/0 (socket says 172.21.15.83:50590) 2024-04-24T09:09:58.637 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.636+0000 7facdb400640 1 -- 172.21.15.83:0/2101871725 learned_addr learned my addr 172.21.15.83:0/2101871725 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:09:58.637 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.636+0000 7facdb400640 1 --2- 172.21.15.83:0/2101871725 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5609f226cba0 0x5609f2273b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:58.637 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.636+0000 7facda000640 1 --2- 172.21.15.83:0/2101871725 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5609f22622e0 0x5609f22dbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:58.637 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.637+0000 7facdb400640 1 -- 172.21.15.83:0/2101871725 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5609f22622e0 msgr2=0x5609f22dbbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:58.637 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.637+0000 7facdb400640 1 --2- 172.21.15.83:0/2101871725 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5609f22622e0 0x5609f22dbbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:58.637 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.637+0000 7facdb400640 1 -- 172.21.15.83:0/2101871725 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5609f226cba0 msgr2=0x5609f2273b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:09:58.638 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.637+0000 7facdb400640 1 --2- 172.21.15.83:0/2101871725 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5609f226cba0 0x5609f2273b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:58.638 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.637+0000 7facdb400640 1 -- 172.21.15.83:0/2101871725 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609f2274150 con 0x5609f22619c0 2024-04-24T09:09:58.638 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.637+0000 7facdb400640 1 --2- 172.21.15.83:0/2101871725 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5609f22619c0 0x5609f22db690 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7facd4009cc0 tx=0x7facd4004f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67e45a37e22e6f00 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:09:58.638 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.637+0000 7facc4c00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:09:58.638 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.638+0000 7facc4c00640 1 -- 172.21.15.83:0/2101871725 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd4016070 con 0x5609f22619c0 2024-04-24T09:09:58.639 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.638+0000 7facc4c00640 1 -- 172.21.15.83:0/2101871725 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9583+0+0 (secure 0 0 0) 0x7facd4012760 con 0x5609f22619c0 2024-04-24T09:09:58.639 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.638+0000 7face4655740 1 -- 172.21.15.83:0/2101871725 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7facd4003e20 con 0x5609f22619c0 2024-04-24T09:09:58.639 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.638+0000 7facc4c00640 1 -- 172.21.15.83:0/2101871725 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7facd4028030 con 0x5609f22619c0 2024-04-24T09:09:58.639 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.638+0000 7face4655740 10 client.39131 fetch_fsmap learned FSMap version 61 2024-04-24T09:09:58.639 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.638+0000 7face4655740 1 -- 172.21.15.83:0/2101871725 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x5609f2275420 con 0x5609f22619c0 2024-04-24T09:09:58.639 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.639+0000 7facc4c00640 1 -- 172.21.15.83:0/2101871725 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7facd4003e20 con 0x5609f22619c0 2024-04-24T09:09:58.639 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.639+0000 7facc4c00640 1 -- 172.21.15.83:0/2101871725 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 61) ==== 8362+0+0 (secure 0 0 0) 0x7facd4023040 con 0x5609f22619c0 2024-04-24T09:09:58.639 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.639+0000 7face4655740 10 client.39131 fetch_fsmap finished waiting for FSMap version 61 2024-04-24T09:09:58.639 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.639+0000 7face4655740 10 client.39131 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:09:58.640 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.639+0000 7face4655740 20 client.39131 populate_metadata read hostname 'smithi083' 2024-04-24T09:09:58.640 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.639+0000 7face4655740 1 --2- 172.21.15.83:0/2101871725 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5609f22f4210 0x5609f22f6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:09:58.640 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.639+0000 7face4655740 4 client.39131 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T09:09:58.640 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.639+0000 7face4655740 1 -- 172.21.15.83:0/2101871725 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5609f21c7190 con 0x5609f22f4210 2024-04-24T09:09:58.640 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.640+0000 7facda000640 1 --2- 172.21.15.83:0/2101871725 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5609f22f4210 0x5609f22f6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:09:58.640 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.641+0000 7facda000640 1 --2- 172.21.15.83:0/2101871725 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5609f22f4210 0x5609f22f6630 crc :-1 s=READY pgs=464 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f1e8d9f814fecf4 server_cookie=ec5376d305f2168a in_seq=0 out_seq=0 2024-04-24T09:09:58.640 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.641+0000 7facc4c00640 10 client.39131 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:09:58.773 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.774+0000 7facc4c00640 1 -- 172.21.15.83:0/2101871725 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5609f21c7190 con 0x5609f22f4210 2024-04-24T09:09:58.774 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.774+0000 7facc4c00640 10 client.39131 handle_command_reply: tid=0 2024-04-24T09:09:58.774 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.774+0000 7face4655740 1 client.39131 shutdown 2024-04-24T09:09:58.774 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.774+0000 7face4655740 1 -- 172.21.15.83:0/2101871725 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5609f22619c0 msgr2=0x5609f22db690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:58.774 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.774+0000 7face4655740 1 --2- 172.21.15.83:0/2101871725 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5609f22619c0 0x5609f22db690 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7facd4009cc0 tx=0x7facd4004f60 comp rx=0 tx=0).stop 2024-04-24T09:09:58.774 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.774+0000 7face4655740 1 -- 172.21.15.83:0/2101871725 shutdown_connections 2024-04-24T09:09:58.774 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.774+0000 7face4655740 1 --2- 172.21.15.83:0/2101871725 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5609f22f4210 0x5609f22f6630 crc :-1 s=READY pgs=464 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:58.774 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.774+0000 7face4655740 1 --2- 172.21.15.83:0/2101871725 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5609f226cba0 0x5609f2273b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:58.774 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.774+0000 7face4655740 1 --2- 172.21.15.83:0/2101871725 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5609f22622e0 0x5609f22dbbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:58.774 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.774+0000 7facc4c00640 0 client.39131 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:09:58.774 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.774+0000 7face4655740 1 --2- 172.21.15.83:0/2101871725 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5609f22619c0 0x5609f22db690 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:58.774 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.774+0000 7face4655740 1 -- 172.21.15.83:0/2101871725 >> 172.21.15.83:0/2101871725 conn(0x5609f225ded0 msgr2=0x5609f2269b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:58.774 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.774+0000 7face4655740 1 -- 172.21.15.83:0/2101871725 shutdown_connections 2024-04-24T09:09:58.775 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.774+0000 7face4655740 1 -- 172.21.15.83:0/2101871725 wait complete. 2024-04-24T09:09:58.775 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.775+0000 7face4655740 20 client.39131 trim_cache size 0 max 16384 2024-04-24T09:09:58.775 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T09:09:58.775 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T09:09:58.775 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T09:09:58.775 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T09:09:58.776 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.776+0000 7face1800640 1 -- 172.21.15.83:0/4002398585 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7facc00a3ec0 msgr2=0x7facc00a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:58.776 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.776+0000 7face1800640 1 --2- 172.21.15.83:0/4002398585 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7facc00a3ec0 0x7facc00a6380 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x5609f2262f60 tx=0x7faccc031400 comp rx=0 tx=0).stop 2024-04-24T09:09:58.776 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.776+0000 7face1800640 1 -- 172.21.15.83:0/4002398585 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7facdc074300 msgr2=0x7facdc1a0610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:09:58.776 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.776+0000 7face1800640 1 --2- 172.21.15.83:0/4002398585 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7facdc074300 0x7facdc1a0610 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7faccc005ab0 tx=0x7faccc003c80 comp rx=0 tx=0).stop 2024-04-24T09:09:58.776 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.777+0000 7face1800640 1 -- 172.21.15.83:0/4002398585 shutdown_connections 2024-04-24T09:09:58.776 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.777+0000 7face1800640 1 --2- 172.21.15.83:0/4002398585 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7facc00a3ec0 0x7facc00a6380 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:58.776 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.777+0000 7face1800640 1 --2- 172.21.15.83:0/4002398585 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7facdc10e8f0 0x7facdc19a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:58.777 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.777+0000 7face1800640 1 --2- 172.21.15.83:0/4002398585 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7facdc074c20 0x7facdc1a0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:58.777 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.777+0000 7face1800640 1 --2- 172.21.15.83:0/4002398585 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7facdc074300 0x7facdc1a0610 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:09:58.777 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.777+0000 7face1800640 1 -- 172.21.15.83:0/4002398585 >> 172.21.15.83:0/4002398585 conn(0x7facdc079490 msgr2=0x7facdc0feec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:09:58.779 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.780+0000 7face1800640 1 -- 172.21.15.83:0/4002398585 shutdown_connections 2024-04-24T09:09:58.779 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:09:58.781+0000 7face1800640 1 -- 172.21.15.83:0/4002398585 wait complete. 2024-04-24T09:09:59.173 INFO:tasks.workunit.client.0.smithi083.stdout: 10 34212 6.74 MB/sec execute 150 sec latency 143.726 ms 2024-04-24T09:10:00.173 INFO:tasks.workunit.client.0.smithi083.stdout: 10 34348 6.76 MB/sec execute 151 sec latency 80.276 ms 2024-04-24T09:10:00.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:10:00 smithi083 ceph-mon[27907]: pgmap v578: 129 pgs: 129 active+clean; 596 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.0 MiB/s wr, 353 op/s 2024-04-24T09:10:00.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:10:00 smithi083 ceph-mon[27907]: overall HEALTH_OK 2024-04-24T09:10:00.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:10:00 smithi189 ceph-mon[31791]: pgmap v578: 129 pgs: 129 active+clean; 596 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.0 MiB/s wr, 353 op/s 2024-04-24T09:10:00.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:10:00 smithi189 ceph-mon[31791]: overall HEALTH_OK 2024-04-24T09:10:00.458 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:10:00 smithi138 ceph-mon[31441]: pgmap v578: 129 pgs: 129 active+clean; 596 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.0 MiB/s wr, 353 op/s 2024-04-24T09:10:00.458 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:10:00 smithi138 ceph-mon[31441]: overall HEALTH_OK 2024-04-24T09:10:01.173 INFO:tasks.workunit.client.0.smithi083.stdout: 10 34460 6.73 MB/sec execute 152 sec latency 117.791 ms 2024-04-24T09:10:02.173 INFO:tasks.workunit.client.0.smithi083.stdout: 10 34582 6.70 MB/sec execute 153 sec latency 64.680 ms 2024-04-24T09:10:02.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:10:02 smithi083 ceph-mon[27907]: pgmap v579: 129 pgs: 129 active+clean; 596 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.0 MiB/s wr, 235 op/s 2024-04-24T09:10:02.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:10:02 smithi138 ceph-mon[31441]: pgmap v579: 129 pgs: 129 active+clean; 596 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.0 MiB/s wr, 235 op/s 2024-04-24T09:10:02.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:10:02 smithi189 ceph-mon[31791]: pgmap v579: 129 pgs: 129 active+clean; 596 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.0 MiB/s wr, 235 op/s 2024-04-24T09:10:03.173 INFO:tasks.workunit.client.0.smithi083.stdout: 10 34717 6.70 MB/sec execute 154 sec latency 119.409 ms 2024-04-24T09:10:04.174 INFO:tasks.workunit.client.0.smithi083.stdout: 10 34854 6.68 MB/sec execute 155 sec latency 123.891 ms 2024-04-24T09:10:04.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:10:04 smithi083 ceph-mon[27907]: pgmap v580: 129 pgs: 129 active+clean; 601 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.3 MiB/s wr, 381 op/s 2024-04-24T09:10:04.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:10:04 smithi083 ceph-mon[27907]: osdmap e119: 12 total, 12 up, 12 in 2024-04-24T09:10:04.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:10:04 smithi138 ceph-mon[31441]: pgmap v580: 129 pgs: 129 active+clean; 601 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.3 MiB/s wr, 381 op/s 2024-04-24T09:10:04.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:10:04 smithi138 ceph-mon[31441]: osdmap e119: 12 total, 12 up, 12 in 2024-04-24T09:10:04.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:10:04 smithi189 ceph-mon[31791]: pgmap v580: 129 pgs: 129 active+clean; 601 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.3 MiB/s wr, 381 op/s 2024-04-24T09:10:04.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:10:04 smithi189 ceph-mon[31791]: osdmap e119: 12 total, 12 up, 12 in 2024-04-24T09:10:05.174 INFO:tasks.workunit.client.0.smithi083.stdout: 10 34983 6.69 MB/sec execute 156 sec latency 85.479 ms 2024-04-24T09:10:06.174 INFO:tasks.workunit.client.0.smithi083.stdout: 10 35100 6.69 MB/sec execute 157 sec latency 80.643 ms 2024-04-24T09:10:06.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:10:06 smithi083 ceph-mon[27907]: pgmap v582: 129 pgs: 129 active+clean; 601 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.3 MiB/s wr, 352 op/s 2024-04-24T09:10:06.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:10:06 smithi138 ceph-mon[31441]: pgmap v582: 129 pgs: 129 active+clean; 601 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.3 MiB/s wr, 352 op/s 2024-04-24T09:10:06.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:10:06 smithi189 ceph-mon[31791]: pgmap v582: 129 pgs: 129 active+clean; 601 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.3 MiB/s wr, 352 op/s 2024-04-24T09:10:07.174 INFO:tasks.workunit.client.0.smithi083.stdout: 10 35263 6.73 MB/sec execute 158 sec latency 71.965 ms 2024-04-24T09:10:08.174 INFO:tasks.workunit.client.0.smithi083.stdout: 10 35419 6.75 MB/sec execute 159 sec latency 79.848 ms 2024-04-24T09:10:08.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:10:08 smithi138 ceph-mon[31441]: pgmap v583: 129 pgs: 129 active+clean; 601 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.3 MiB/s wr, 352 op/s 2024-04-24T09:10:08.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:10:08 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:10:08.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:10:08 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:10:08.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:10:08 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:10:08.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:10:08 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:10:08.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:10:08 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:10:08.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:10:08 smithi189 ceph-mon[31791]: pgmap v583: 129 pgs: 129 active+clean; 601 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.3 MiB/s wr, 352 op/s 2024-04-24T09:10:08.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:10:08 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:10:08.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:10:08 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:10:08.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:10:08 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:10:08.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:10:08 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:10:08.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:10:08 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:10:08.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:10:08 smithi083 ceph-mon[27907]: pgmap v583: 129 pgs: 129 active+clean; 601 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.3 MiB/s wr, 352 op/s 2024-04-24T09:10:08.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:10:08 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:10:08.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:10:08 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:10:08.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:10:08 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:10:08.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:10:08 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:10:08.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:10:08 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:10:09.174 INFO:tasks.workunit.client.0.smithi083.stdout: 10 35574 6.78 MB/sec execute 160 sec latency 81.111 ms 2024-04-24T09:10:10.174 INFO:tasks.workunit.client.0.smithi083.stdout: 10 35711 6.83 MB/sec execute 161 sec latency 200.553 ms 2024-04-24T09:10:10.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:10:10 smithi138 ceph-mon[31441]: pgmap v584: 129 pgs: 129 active+clean; 620 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.0 MiB/s wr, 303 op/s 2024-04-24T09:10:10.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:10:10 smithi138 ceph-mon[31441]: osdmap e120: 12 total, 12 up, 12 in 2024-04-24T09:10:10.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:10:10 smithi189 ceph-mon[31791]: pgmap v584: 129 pgs: 129 active+clean; 620 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.0 MiB/s wr, 303 op/s 2024-04-24T09:10:10.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:10:10 smithi189 ceph-mon[31791]: osdmap e120: 12 total, 12 up, 12 in 2024-04-24T09:10:10.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:10:10 smithi083 ceph-mon[27907]: pgmap v584: 129 pgs: 129 active+clean; 620 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.0 MiB/s wr, 303 op/s 2024-04-24T09:10:10.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:10:10 smithi083 ceph-mon[27907]: osdmap e120: 12 total, 12 up, 12 in 2024-04-24T09:10:11.174 INFO:tasks.workunit.client.0.smithi083.stdout: 10 35849 6.83 MB/sec execute 162 sec latency 92.164 ms 2024-04-24T09:10:11.957 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.958+0000 7f1d05600640 1 -- 172.21.15.83:0/2226816046 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d00100180 msgr2=0x7f1d00100560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:11.957 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.958+0000 7f1d05600640 1 --2- 172.21.15.83:0/2226816046 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d00100180 0x7f1d00100560 secure :-1 s=READY pgs=2416 cs=0 l=1 rev1=1 crypto rx=0x7f1cf0009930 tx=0x7f1cf002a420 comp rx=0 tx=0).stop 2024-04-24T09:10:11.958 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.958+0000 7f1d05600640 1 -- 172.21.15.83:0/2226816046 shutdown_connections 2024-04-24T09:10:11.958 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.958+0000 7f1d05600640 1 --2- 172.21.15.83:0/2226816046 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1d0010a690 0x7f1d00111010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:11.958 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.958+0000 7f1d05600640 1 --2- 172.21.15.83:0/2226816046 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d00100b30 0x7f1d00109f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:11.958 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.958+0000 7f1d05600640 1 --2- 172.21.15.83:0/2226816046 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d00100180 0x7f1d00100560 unknown :-1 s=CLOSED pgs=2416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:11.958 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.958+0000 7f1d05600640 1 -- 172.21.15.83:0/2226816046 >> 172.21.15.83:0/2226816046 conn(0x7f1d00076e60 msgr2=0x7f1d000fe220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:11.958 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.958+0000 7f1d05600640 1 -- 172.21.15.83:0/2226816046 shutdown_connections 2024-04-24T09:10:11.958 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.958+0000 7f1d05600640 1 -- 172.21.15.83:0/2226816046 wait complete. 2024-04-24T09:10:11.959 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.959+0000 7f1d05600640 1 Processor -- start 2024-04-24T09:10:11.959 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.960+0000 7f1d05600640 1 -- start start 2024-04-24T09:10:11.959 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.960+0000 7f1d05600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1d0010a690 0x7f1d001a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:11.959 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.960+0000 7f1d05600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d001a0bf0 0x7f1d0019a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:11.959 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.960+0000 7f1d05600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d0019ada0 0x7f1d0019b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:11.960 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.960+0000 7f1d05600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1d00114330 con 0x7f1d0019ada0 2024-04-24T09:10:11.960 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.960+0000 7f1d05600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1d001141b0 con 0x7f1d001a0bf0 2024-04-24T09:10:11.960 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.960+0000 7f1d05600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1d001144b0 con 0x7f1d0010a690 2024-04-24T09:10:11.961 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.961+0000 7f1d04c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d0019ada0 0x7f1d0019b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:11.961 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.961+0000 7f1d04c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d0019ada0 0x7f1d0019b220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42222/0 (socket says 172.21.15.83:42222) 2024-04-24T09:10:11.961 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.961+0000 7f1d04c00640 1 -- 172.21.15.83:0/3473904150 learned_addr learned my addr 172.21.15.83:0/3473904150 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:10:11.961 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.961+0000 7f1cf7e00640 1 --2- 172.21.15.83:0/3473904150 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d001a0bf0 0x7f1d0019a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:11.961 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.961+0000 7f1cffe00640 1 --2- 172.21.15.83:0/3473904150 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1d0010a690 0x7f1d001a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:11.961 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.961+0000 7f1d04c00640 1 -- 172.21.15.83:0/3473904150 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1d0010a690 msgr2=0x7f1d001a06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:11.961 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.961+0000 7f1d04c00640 1 --2- 172.21.15.83:0/3473904150 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1d0010a690 0x7f1d001a06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:11.961 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.961+0000 7f1d04c00640 1 -- 172.21.15.83:0/3473904150 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d001a0bf0 msgr2=0x7f1d0019a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:11.961 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.961+0000 7f1d04c00640 1 --2- 172.21.15.83:0/3473904150 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d001a0bf0 0x7f1d0019a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:11.961 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.961+0000 7f1d04c00640 1 -- 172.21.15.83:0/3473904150 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d001a7570 con 0x7f1d0019ada0 2024-04-24T09:10:11.961 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.962+0000 7f1d04c00640 1 --2- 172.21.15.83:0/3473904150 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d0019ada0 0x7f1d0019b220 secure :-1 s=READY pgs=2417 cs=0 l=1 rev1=1 crypto rx=0x7f1cf800eaa0 tx=0x7f1cf800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:11.962 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.962+0000 7f1cfd600640 1 -- 172.21.15.83:0/3473904150 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cf800cdb0 con 0x7f1d0019ada0 2024-04-24T09:10:11.962 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.962+0000 7f1cfd600640 1 -- 172.21.15.83:0/3473904150 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1cf8005020 con 0x7f1d0019ada0 2024-04-24T09:10:11.962 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.962+0000 7f1d05600640 1 -- 172.21.15.83:0/3473904150 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d001a77a0 con 0x7f1d0019ada0 2024-04-24T09:10:11.963 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.962+0000 7f1d05600640 1 -- 172.21.15.83:0/3473904150 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d001a7c70 con 0x7f1d0019ada0 2024-04-24T09:10:11.964 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.963+0000 7f1cfd600640 1 -- 172.21.15.83:0/3473904150 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cf80116d0 con 0x7f1d0019ada0 2024-04-24T09:10:11.966 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.967+0000 7f1cfd600640 1 -- 172.21.15.83:0/3473904150 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1cf8004a40 con 0x7f1d0019ada0 2024-04-24T09:10:11.967 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.968+0000 7f1d099c3740 1 Processor -- start 2024-04-24T09:10:11.967 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.968+0000 7f1d099c3740 1 -- start start 2024-04-24T09:10:11.968 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.969+0000 7f1d099c3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cbfca8e3c0 0x55cbfca8e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:11.968 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.969+0000 7f1d099c3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cbfca8ece0 0x55cbfca99060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:11.968 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.969+0000 7f1d099c3740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbfca995a0 0x55cbfca9b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:11.968 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.969+0000 7f1d099c3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55cbfc9f4900 con 0x55cbfca8e3c0 2024-04-24T09:10:11.968 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.969+0000 7f1d099c3740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55cbfc99af20 con 0x55cbfca995a0 2024-04-24T09:10:11.968 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.969+0000 7f1d099c3740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55cbfc9f08c0 con 0x55cbfca8ece0 2024-04-24T09:10:11.969 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.969+0000 7f1cffe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cbfca8ece0 0x55cbfca99060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:11.969 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.970+0000 7f1cffe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cbfca8ece0 0x55cbfca99060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:56980/0 (socket says 172.21.15.83:56980) 2024-04-24T09:10:11.969 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.970+0000 7f1cffe00640 1 -- 172.21.15.83:0/3693249965 learned_addr learned my addr 172.21.15.83:0/3693249965 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:10:11.970 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.970+0000 7f1cf7e00640 1 --2- 172.21.15.83:0/3693249965 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbfca995a0 0x55cbfca9b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:11.970 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.970+0000 7f1cf7e00640 1 --2- 172.21.15.83:0/3693249965 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cbfca8e3c0 0x55cbfca8e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:11.970 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.970+0000 7f1cfd600640 1 --2- 172.21.15.83:0/3473904150 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f1ce40a3d10 0x7f1ce40a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:11.970 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.970+0000 7f1cffe00640 1 -- 172.21.15.83:0/3693249965 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbfca995a0 msgr2=0x55cbfca9b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:11.970 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.970+0000 7f1d04c00640 1 --2- 172.21.15.83:0/3473904150 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f1ce40a3d10 0x7f1ce40a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:11.970 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.970+0000 7f1cffe00640 1 --2- 172.21.15.83:0/3693249965 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbfca995a0 0x55cbfca9b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:11.970 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.970+0000 7f1cffe00640 1 -- 172.21.15.83:0/3693249965 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cbfca8e3c0 msgr2=0x55cbfca8e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:11.971 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.970+0000 7f1cfd600640 1 -- 172.21.15.83:0/3473904150 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 9583+0+0 (secure 0 0 0) 0x7f1cf80d0d80 con 0x7f1d0019ada0 2024-04-24T09:10:11.971 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.971+0000 7f1cffe00640 1 --2- 172.21.15.83:0/3693249965 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cbfca8e3c0 0x55cbfca8e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:11.971 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.971+0000 7f1cffe00640 1 -- 172.21.15.83:0/3693249965 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbfc9bfe10 con 0x55cbfca8ece0 2024-04-24T09:10:11.971 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.971+0000 7f1cf7e00640 1 --2- 172.21.15.83:0/3693249965 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cbfca8e3c0 0x55cbfca8e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:10:11.971 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.971+0000 7f1d04c00640 1 --2- 172.21.15.83:0/3473904150 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f1ce40a3d10 0x7f1ce40a61d0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f1cf809c600 tx=0x7f1cf8098c90 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:11.971 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.972+0000 7f1cffe00640 1 --2- 172.21.15.83:0/3693249965 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cbfca8ece0 0x55cbfca99060 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f1cf0000c00 tx=0x7f1cf00040b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b46dea7266c122dd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:11.972 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.972+0000 7f1cf5200640 1 -- 172.21.15.83:0/3693249965 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cf003f070 con 0x55cbfca8ece0 2024-04-24T09:10:11.972 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.972+0000 7f1cf5200640 1 -- 172.21.15.83:0/3693249965 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1cf0004420 con 0x55cbfca8ece0 2024-04-24T09:10:11.972 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.972+0000 7f1cf5200640 1 -- 172.21.15.83:0/3693249965 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cf0035070 con 0x55cbfca8ece0 2024-04-24T09:10:11.973 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.973+0000 7f1d099c3740 1 -- 172.21.15.83:0/3693249965 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cbfca8ece0 msgr2=0x55cbfca99060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:11.973 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.973+0000 7f1d099c3740 1 --2- 172.21.15.83:0/3693249965 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cbfca8ece0 0x55cbfca99060 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f1cf0000c00 tx=0x7f1cf00040b0 comp rx=0 tx=0).stop 2024-04-24T09:10:11.973 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.973+0000 7f1d099c3740 1 -- 172.21.15.83:0/3693249965 shutdown_connections 2024-04-24T09:10:11.973 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.973+0000 7f1d099c3740 1 --2- 172.21.15.83:0/3693249965 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbfca995a0 0x55cbfca9b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:11.973 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.973+0000 7f1d099c3740 1 --2- 172.21.15.83:0/3693249965 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cbfca8ece0 0x55cbfca99060 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:11.973 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.973+0000 7f1d099c3740 1 --2- 172.21.15.83:0/3693249965 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cbfca8e3c0 0x55cbfca8e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:11.974 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.973+0000 7f1d099c3740 1 -- 172.21.15.83:0/3693249965 >> 172.21.15.83:0/3693249965 conn(0x55cbfca8a8d0 msgr2=0x55cbfca8acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:11.974 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.974+0000 7f1d099c3740 1 -- 172.21.15.83:0/3693249965 shutdown_connections 2024-04-24T09:10:11.974 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.974+0000 7f1d099c3740 1 -- 172.21.15.83:0/3693249965 wait complete. 2024-04-24T09:10:11.974 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.974+0000 7f1d099c3740 1 -- start start 2024-04-24T09:10:11.975 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.974+0000 7f1d099c3740 1 Processor -- start 2024-04-24T09:10:11.975 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.975+0000 7f1d099c3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cbfca8e3c0 0x55cbfca96460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:11.975 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.975+0000 7f1d099c3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cbfca8ece0 0x55cbfca969a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:11.976 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.975+0000 7f1d099c3740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbfca995a0 0x55cbfca96ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:11.976 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.975+0000 7f1d099c3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55cbfca9c130 con 0x55cbfca8ece0 2024-04-24T09:10:11.976 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.975+0000 7f1d099c3740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55cbfc9ae2c0 con 0x55cbfca995a0 2024-04-24T09:10:11.976 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.975+0000 7f1d099c3740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55cbfca9c2b0 con 0x55cbfca8e3c0 2024-04-24T09:10:11.976 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.975+0000 7f1d099c3740 10 client.0 _ll_register_callbacks cb 0x55cbfc706a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:10:11.976 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.975+0000 7f1cf7e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbfca995a0 0x55cbfca96ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:11.976 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.976+0000 7f1cf7e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbfca995a0 0x55cbfca96ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:48106/0 (socket says 172.21.15.83:48106) 2024-04-24T09:10:11.976 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.976+0000 7f1cf7e00640 1 -- 172.21.15.83:0/3186445311 learned_addr learned my addr 172.21.15.83:0/3186445311 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:10:11.976 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.976+0000 7f1cffe00640 1 --2- 172.21.15.83:0/3186445311 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cbfca8ece0 0x55cbfca969a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:11.976 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.976+0000 7f1cf7e00640 1 --2- 172.21.15.83:0/3186445311 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cbfca8e3c0 0x55cbfca96460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:11.976 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.976+0000 7f1cf7e00640 1 -- 172.21.15.83:0/3186445311 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cbfca8e3c0 msgr2=0x55cbfca96460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:11.976 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.976+0000 7f1cf7e00640 1 --2- 172.21.15.83:0/3186445311 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cbfca8e3c0 0x55cbfca96460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:11.977 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.976+0000 7f1cf7e00640 1 -- 172.21.15.83:0/3186445311 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cbfca8ece0 msgr2=0x55cbfca969a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:11.977 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.976+0000 7f1cf7e00640 1 --2- 172.21.15.83:0/3186445311 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cbfca8ece0 0x55cbfca969a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:11.978 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.976+0000 7f1cf7e00640 1 -- 172.21.15.83:0/3186445311 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cbfc9befd0 con 0x55cbfca995a0 2024-04-24T09:10:11.978 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.977+0000 7f1cffe00640 1 --2- 172.21.15.83:0/3186445311 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cbfca8ece0 0x55cbfca969a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:10:11.978 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.977+0000 7f1cf7e00640 1 --2- 172.21.15.83:0/3186445311 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbfca995a0 0x55cbfca96ee0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f1ce8012420 tx=0x7f1ce8014040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56c4dc44d1a05a8c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:11.978 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.978+0000 7f1ced600640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:10:11.978 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.978+0000 7f1ced600640 1 -- 172.21.15.83:0/3186445311 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce8010e00 con 0x55cbfca995a0 2024-04-24T09:10:11.978 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.979+0000 7f1d099c3740 1 -- 172.21.15.83:0/3186445311 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1cf0004420 con 0x55cbfca995a0 2024-04-24T09:10:11.978 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.979+0000 7f1ced600640 1 -- 172.21.15.83:0/3186445311 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(120..120 src has 1..120) ==== 9583+0+0 (secure 0 0 0) 0x7f1ce8004020 con 0x55cbfca995a0 2024-04-24T09:10:11.980 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.979+0000 7f1ced600640 1 -- 172.21.15.83:0/3186445311 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f1ce8028050 con 0x55cbfca995a0 2024-04-24T09:10:11.980 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.979+0000 7f1d099c3740 10 client.39143 fetch_fsmap learned FSMap version 61 2024-04-24T09:10:11.980 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.979+0000 7f1d099c3740 1 -- 172.21.15.83:0/3186445311 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x55cbfc707210 con 0x55cbfca995a0 2024-04-24T09:10:11.980 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.979+0000 7f1ced600640 1 -- 172.21.15.83:0/3186445311 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1cf0004420 con 0x55cbfca995a0 2024-04-24T09:10:11.980 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.980+0000 7f1ced600640 1 -- 172.21.15.83:0/3186445311 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 61) ==== 8362+0+0 (secure 0 0 0) 0x7f1ce801d400 con 0x55cbfca995a0 2024-04-24T09:10:11.980 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.980+0000 7f1d099c3740 10 client.39143 fetch_fsmap finished waiting for FSMap version 61 2024-04-24T09:10:11.980 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.980+0000 7f1d099c3740 10 client.39143 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:10:11.980 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.980+0000 7f1d099c3740 20 client.39143 populate_metadata read hostname 'smithi083' 2024-04-24T09:10:11.980 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.980+0000 7f1d099c3740 1 --2- 172.21.15.83:0/3186445311 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55cbfcab1e70 0x55cbfcab4280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:11.980 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.980+0000 7f1d099c3740 4 client.39143 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:10:11.980 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.980+0000 7f1d099c3740 1 -- 172.21.15.83:0/3186445311 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cbfca02320 con 0x55cbfcab1e70 2024-04-24T09:10:11.981 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.980+0000 7f1cffe00640 1 --2- 172.21.15.83:0/3186445311 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55cbfcab1e70 0x55cbfcab4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:11.981 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.982+0000 7f1cffe00640 1 --2- 172.21.15.83:0/3186445311 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55cbfcab1e70 0x55cbfcab4280 crc :-1 s=READY pgs=465 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ebd0ca6301eeeda4 server_cookie=bdee6b1767bacb4e in_seq=0 out_seq=0 2024-04-24T09:10:11.981 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.982+0000 7f1ced600640 10 client.39143 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:10:11.982 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.983+0000 7f1ced600640 1 -- 172.21.15.83:0/3186445311 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cbfca02320 con 0x55cbfcab1e70 2024-04-24T09:10:11.982 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.983+0000 7f1ced600640 10 client.39143 handle_command_reply: tid=0 2024-04-24T09:10:11.982 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.983+0000 7f1d099c3740 1 client.39143 shutdown 2024-04-24T09:10:11.984 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.984+0000 7f1d099c3740 1 -- 172.21.15.83:0/3186445311 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbfca995a0 msgr2=0x55cbfca96ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:11.984 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.984+0000 7f1d099c3740 1 --2- 172.21.15.83:0/3186445311 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbfca995a0 0x55cbfca96ee0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f1ce8012420 tx=0x7f1ce8014040 comp rx=0 tx=0).stop 2024-04-24T09:10:11.984 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.984+0000 7f1d099c3740 1 -- 172.21.15.83:0/3186445311 shutdown_connections 2024-04-24T09:10:11.984 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.984+0000 7f1d099c3740 1 --2- 172.21.15.83:0/3186445311 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55cbfcab1e70 0x55cbfcab4280 crc :-1 s=READY pgs=465 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:11.984 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.984+0000 7f1d099c3740 1 --2- 172.21.15.83:0/3186445311 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbfca995a0 0x55cbfca96ee0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:11.984 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.984+0000 7f1ced600640 0 client.39143 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:10:11.984 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.984+0000 7f1d099c3740 1 --2- 172.21.15.83:0/3186445311 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cbfca8ece0 0x55cbfca969a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:11.984 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.984+0000 7f1d099c3740 1 --2- 172.21.15.83:0/3186445311 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cbfca8e3c0 0x55cbfca96460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:11.984 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.984+0000 7f1d099c3740 1 -- 172.21.15.83:0/3186445311 >> 172.21.15.83:0/3186445311 conn(0x55cbfca8a8d0 msgr2=0x55cbfca90600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:11.985 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.985+0000 7f1d099c3740 1 -- 172.21.15.83:0/3186445311 shutdown_connections 2024-04-24T09:10:11.985 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.985+0000 7f1d099c3740 1 -- 172.21.15.83:0/3186445311 wait complete. 2024-04-24T09:10:11.985 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:11.985+0000 7f1d099c3740 20 client.39143 trim_cache size 0 max 16384 2024-04-24T09:10:12.009 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:12.010+0000 7f1d099c3740 1 Processor -- start 2024-04-24T09:10:12.009 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:12.010+0000 7f1d099c3740 1 -- start start 2024-04-24T09:10:12.010 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:12.010+0000 7f1d099c3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cbfca8e3c0 0x55cbfca98b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:12.010 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:12.010+0000 7f1d099c3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cbfca8ece0 0x55cbfca96d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:12.010 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:12.010+0000 7f1d099c3740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbfca995a0 0x55cbfcab9be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:12.010 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:12.010+0000 7f1d099c3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1cdc010110 con 0x55cbfca8ece0 2024-04-24T09:10:12.010 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:12.010+0000 7f1d099c3740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1cdc011c50 con 0x55cbfca995a0 2024-04-24T09:10:12.010 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:12.010+0000 7f1d099c3740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1cdc0107e0 con 0x55cbfca8e3c0 2024-04-24T09:10:12.011 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:12.011+0000 7f1cffe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cbfca8ece0 0x55cbfca96d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:12.011 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:12.011+0000 7f1cffe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cbfca8ece0 0x55cbfca96d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42266/0 (socket says 172.21.15.83:42266) 2024-04-24T09:10:12.012 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:12.011+0000 7f1cffe00640 1 -- 172.21.15.83:0/3405416432 learned_addr learned my addr 172.21.15.83:0/3405416432 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:10:12.012 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:12.011+0000 7f1cf7e00640 1 --2- 172.21.15.83:0/3405416432 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbfca995a0 0x55cbfcab9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:12.012 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:12.012+0000 7f1cffe00640 1 -- 172.21.15.83:0/3405416432 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cbfca8e3c0 msgr2=0x55cbfca98b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:12.012 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:12.012+0000 7f1cf7e00640 1 --2- 172.21.15.83:0/3405416432 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cbfca8e3c0 0x55cbfca98b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:12.012 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:12.012+0000 7f1cffe00640 1 --2- 172.21.15.83:0/3405416432 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cbfca8e3c0 0x55cbfca98b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:12.012 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:12.012+0000 7f1cffe00640 1 -- 172.21.15.83:0/3405416432 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbfca995a0 msgr2=0x55cbfcab9be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:12.012 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:12.012+0000 7f1cffe00640 1 --2- 172.21.15.83:0/3405416432 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbfca995a0 0x55cbfcab9be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:12.012 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:12.012+0000 7f1cffe00640 1 -- 172.21.15.83:0/3405416432 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbfca8eb10 con 0x55cbfca8ece0 2024-04-24T09:10:12.012 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:12.012+0000 7f1cf7e00640 1 --2- 172.21.15.83:0/3405416432 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cbfca8e3c0 0x55cbfca98b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:10:12.012 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:12.012+0000 7f1cffe00640 1 --2- 172.21.15.83:0/3405416432 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cbfca8ece0 0x55cbfca96d00 secure :-1 s=READY pgs=2418 cs=0 l=1 rev1=1 crypto rx=0x7f1cf0009930 tx=0x7f1cf002a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbd2cb06c8d664f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:12.014 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:12.013+0000 7f1cefe00640 1 -- 172.21.15.83:0/3405416432 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cf0055070 con 0x55cbfca8ece0 2024-04-24T09:10:12.014 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:12.013+0000 7f1cefe00640 1 -- 172.21.15.83:0/3405416432 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1cf0050670 con 0x55cbfca8ece0 2024-04-24T09:10:12.014 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:12.013+0000 7f1cefe00640 1 -- 172.21.15.83:0/3405416432 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cf003f040 con 0x55cbfca8ece0 2024-04-24T09:10:12.014 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:12.014+0000 7f1d099c3740 1 -- 172.21.15.83:0/3405416432 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cbfca8ece0 msgr2=0x55cbfca96d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:12.014 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:12.014+0000 7f1d099c3740 1 --2- 172.21.15.83:0/3405416432 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cbfca8ece0 0x55cbfca96d00 secure :-1 s=READY pgs=2418 cs=0 l=1 rev1=1 crypto rx=0x7f1cf0009930 tx=0x7f1cf002a420 comp rx=0 tx=0).stop 2024-04-24T09:10:12.014 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:12.014+0000 7f1d099c3740 1 -- 172.21.15.83:0/3405416432 shutdown_connections 2024-04-24T09:10:12.014 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:12.014+0000 7f1d099c3740 1 --2- 172.21.15.83:0/3405416432 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbfca995a0 0x55cbfcab9be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:12.014 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:12.014+0000 7f1d099c3740 1 --2- 172.21.15.83:0/3405416432 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cbfca8ece0 0x55cbfca96d00 unknown :-1 s=CLOSED pgs=2418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:12.014 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:12.014+0000 7f1d099c3740 1 --2- 172.21.15.83:0/3405416432 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cbfca8e3c0 0x55cbfca98b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:12.014 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:12.014+0000 7f1d099c3740 1 -- 172.21.15.83:0/3405416432 >> 172.21.15.83:0/3405416432 conn(0x55cbfca8a8d0 msgr2=0x55cbfca8acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:12.015 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:12.014+0000 7f1d099c3740 1 -- 172.21.15.83:0/3405416432 shutdown_connections 2024-04-24T09:10:12.015 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:12.014+0000 7f1d099c3740 1 -- 172.21.15.83:0/3405416432 wait complete. 2024-04-24T09:10:12.015 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:12.015+0000 7f1d099c3740 1 -- start start 2024-04-24T09:10:12.015 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:12.015+0000 7f1d099c3740 1 Processor -- start 2024-04-24T09:10:12.016 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:12.015+0000 7f1d099c3740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbfca8e3c0 0x55cbfcb15450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:12.016 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:12.015+0000 7f1d099c3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cbfca8ece0 0x55cbfcb15990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:12.016 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:12.015+0000 7f1d099c3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cbfca995a0 0x55cbfcaa1610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:12.016 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:12.015+0000 7f1d099c3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1cdc011580 con 0x55cbfca995a0 2024-04-24T09:10:12.016 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:12.015+0000 7f1d099c3740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1cdc010eb0 con 0x55cbfca8e3c0 2024-04-24T09:10:12.016 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:12.015+0000 7f1d099c3740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1cdc00e110 con 0x55cbfca8ece0 2024-04-24T09:10:12.016 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:12.015+0000 7f1d099c3740 10 client.0 _ll_register_callbacks cb 0x7f1ce80125a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:10:12.016 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:12.016+0000 7f1cf7e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbfca8e3c0 0x55cbfcb15450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:12.017 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:12.016+0000 7f1cf7e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbfca8e3c0 0x55cbfcb15450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:48128/0 (socket says 172.21.15.83:48128) 2024-04-24T09:10:12.017 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:12.016+0000 7f1cf7e00640 1 -- 172.21.15.83:0/91921964 learned_addr learned my addr 172.21.15.83:0/91921964 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:10:12.017 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:12.016+0000 7f1cf7e00640 1 --2- 172.21.15.83:0/91921964 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cbfca995a0 0x55cbfcaa1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:12.017 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:12.016+0000 7f1cffe00640 1 --2- 172.21.15.83:0/91921964 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cbfca8ece0 0x55cbfcb15990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:12.017 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:12.017+0000 7f1cf7e00640 1 -- 172.21.15.83:0/91921964 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cbfca8ece0 msgr2=0x55cbfcb15990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:12.017 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:12.017+0000 7f1cf7e00640 1 --2- 172.21.15.83:0/91921964 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cbfca8ece0 0x55cbfcb15990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:12.017 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:12.017+0000 7f1cf7e00640 1 -- 172.21.15.83:0/91921964 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbfca8e3c0 msgr2=0x55cbfcb15450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:12.017 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:12.017+0000 7f1cf7e00640 1 --2- 172.21.15.83:0/91921964 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbfca8e3c0 0x55cbfcb15450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:12.017 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:12.017+0000 7f1cf7e00640 1 -- 172.21.15.83:0/91921964 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cbfc7d3620 con 0x55cbfca995a0 2024-04-24T09:10:12.018 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:12.018+0000 7f1cf7e00640 1 --2- 172.21.15.83:0/91921964 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cbfca995a0 0x55cbfcaa1610 secure :-1 s=READY pgs=2419 cs=0 l=1 rev1=1 crypto rx=0x7f1ce8012fd0 tx=0x7f1ce80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a37b9464feab4914 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:12.018 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:12.018+0000 7f1ced600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:10:12.018 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:12.018+0000 7f1ced600640 1 -- 172.21.15.83:0/91921964 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce801dc40 con 0x55cbfca995a0 2024-04-24T09:10:12.019 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:12.018+0000 7f1ced600640 1 -- 172.21.15.83:0/91921964 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(120..120 src has 1..120) ==== 9583+0+0 (secure 0 0 0) 0x7f1ce8005020 con 0x55cbfca995a0 2024-04-24T09:10:12.019 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:12.019+0000 7f1d099c3740 1 -- 172.21.15.83:0/91921964 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1cf0050670 con 0x55cbfca995a0 2024-04-24T09:10:12.019 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:12.019+0000 7f1ced600640 1 -- 172.21.15.83:0/91921964 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f1ce8005220 con 0x55cbfca995a0 2024-04-24T09:10:12.019 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:12.019+0000 7f1d099c3740 10 client.22038 fetch_fsmap learned FSMap version 61 2024-04-24T09:10:12.019 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:12.019+0000 7f1d099c3740 1 -- 172.21.15.83:0/91921964 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x55cbfca9a700 con 0x55cbfca995a0 2024-04-24T09:10:12.020 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:12.020+0000 7f1ced600640 1 -- 172.21.15.83:0/91921964 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1cf0050670 con 0x55cbfca995a0 2024-04-24T09:10:12.020 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:12.020+0000 7f1ced600640 1 -- 172.21.15.83:0/91921964 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 61) ==== 8362+0+0 (secure 0 0 0) 0x7f1ce8037060 con 0x55cbfca995a0 2024-04-24T09:10:12.020 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:12.020+0000 7f1d099c3740 10 client.22038 fetch_fsmap finished waiting for FSMap version 61 2024-04-24T09:10:12.020 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:12.020+0000 7f1d099c3740 10 client.22038 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:10:12.020 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:12.020+0000 7f1d099c3740 20 client.22038 populate_metadata read hostname 'smithi083' 2024-04-24T09:10:12.020 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:12.020+0000 7f1d099c3740 1 --2- 172.21.15.83:0/91921964 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55cbfcb29570 0x55cbfcb2b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:12.020 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:12.020+0000 7f1d099c3740 4 client.22038 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T09:10:12.020 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:12.020+0000 7f1d099c3740 1 -- 172.21.15.83:0/91921964 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cbfc9fc190 con 0x55cbfcb29570 2024-04-24T09:10:12.020 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:12.020+0000 7f1cffe00640 1 --2- 172.21.15.83:0/91921964 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55cbfcb29570 0x55cbfcb2b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:12.021 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:12.021+0000 7f1cffe00640 1 --2- 172.21.15.83:0/91921964 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55cbfcb29570 0x55cbfcb2b960 crc :-1 s=READY pgs=466 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f554e99cde1aac2b server_cookie=df3b1ff5c8759494 in_seq=0 out_seq=0 2024-04-24T09:10:12.021 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:12.021+0000 7f1ced600640 10 client.22038 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:10:12.174 INFO:tasks.workunit.client.0.smithi083.stdout: 10 35953 6.82 MB/sec execute 163 sec latency 120.473 ms 2024-04-24T09:10:12.200 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:12.201+0000 7f1ced600640 1 -- 172.21.15.83:0/91921964 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55cbfc9fc190 con 0x55cbfcb29570 2024-04-24T09:10:12.200 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:12.201+0000 7f1ced600640 10 client.22038 handle_command_reply: tid=0 2024-04-24T09:10:12.201 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:12.201+0000 7f1d099c3740 1 client.22038 shutdown 2024-04-24T09:10:12.201 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:12.202+0000 7f1d099c3740 1 -- 172.21.15.83:0/91921964 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cbfca995a0 msgr2=0x55cbfcaa1610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:12.201 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:12.202+0000 7f1d099c3740 1 --2- 172.21.15.83:0/91921964 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cbfca995a0 0x55cbfcaa1610 secure :-1 s=READY pgs=2419 cs=0 l=1 rev1=1 crypto rx=0x7f1ce8012fd0 tx=0x7f1ce80046c0 comp rx=0 tx=0).stop 2024-04-24T09:10:12.202 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:12.203+0000 7f1d099c3740 1 -- 172.21.15.83:0/91921964 shutdown_connections 2024-04-24T09:10:12.202 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:12.203+0000 7f1d099c3740 1 --2- 172.21.15.83:0/91921964 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55cbfcb29570 0x55cbfcb2b960 crc :-1 s=READY pgs=466 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:12.202 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:12.203+0000 7f1d099c3740 1 --2- 172.21.15.83:0/91921964 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cbfca995a0 0x55cbfcaa1610 unknown :-1 s=CLOSED pgs=2419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:12.202 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:12.203+0000 7f1d099c3740 1 --2- 172.21.15.83:0/91921964 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cbfca8ece0 0x55cbfcb15990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:12.202 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:12.203+0000 7f1d099c3740 1 --2- 172.21.15.83:0/91921964 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbfca8e3c0 0x55cbfcb15450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:12.202 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:12.203+0000 7f1ced600640 0 client.22038 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:10:12.202 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:12.203+0000 7f1d099c3740 1 -- 172.21.15.83:0/91921964 >> 172.21.15.83:0/91921964 conn(0x55cbfca8a8d0 msgr2=0x55cbfca89200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:12.202 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:12.203+0000 7f1d099c3740 1 -- 172.21.15.83:0/91921964 shutdown_connections 2024-04-24T09:10:12.203 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:12.203+0000 7f1d099c3740 1 -- 172.21.15.83:0/91921964 wait complete. 2024-04-24T09:10:12.203 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:12.204+0000 7f1d099c3740 20 client.22038 trim_cache size 0 max 16384 2024-04-24T09:10:12.203 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T09:10:12.203 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T09:10:12.203 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T09:10:12.203 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T09:10:12.204 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:12.204+0000 7f1d05600640 1 -- 172.21.15.83:0/3473904150 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f1ce40a3d10 msgr2=0x7f1ce40a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:12.204 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:12.205+0000 7f1d05600640 1 --2- 172.21.15.83:0/3473904150 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f1ce40a3d10 0x7f1ce40a61d0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f1cf809c600 tx=0x7f1cf8098c90 comp rx=0 tx=0).stop 2024-04-24T09:10:12.204 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:12.205+0000 7f1d05600640 1 -- 172.21.15.83:0/3473904150 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d0019ada0 msgr2=0x7f1d0019b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:12.204 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:12.205+0000 7f1d05600640 1 --2- 172.21.15.83:0/3473904150 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d0019ada0 0x7f1d0019b220 secure :-1 s=READY pgs=2417 cs=0 l=1 rev1=1 crypto rx=0x7f1cf800eaa0 tx=0x7f1cf800ef60 comp rx=0 tx=0).stop 2024-04-24T09:10:12.205 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:12.205+0000 7f1d05600640 1 -- 172.21.15.83:0/3473904150 shutdown_connections 2024-04-24T09:10:12.205 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:12.205+0000 7f1d05600640 1 --2- 172.21.15.83:0/3473904150 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f1ce40a3d10 0x7f1ce40a61d0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:12.205 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:12.205+0000 7f1d05600640 1 --2- 172.21.15.83:0/3473904150 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d0019ada0 0x7f1d0019b220 unknown :-1 s=CLOSED pgs=2417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:12.205 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:12.205+0000 7f1d05600640 1 --2- 172.21.15.83:0/3473904150 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d001a0bf0 0x7f1d0019a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:12.205 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:12.205+0000 7f1d05600640 1 --2- 172.21.15.83:0/3473904150 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1d0010a690 0x7f1d001a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:12.205 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:12.205+0000 7f1d05600640 1 -- 172.21.15.83:0/3473904150 >> 172.21.15.83:0/3473904150 conn(0x7f1d00076e60 msgr2=0x7f1d000fe220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:12.205 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:12.205+0000 7f1d05600640 1 -- 172.21.15.83:0/3473904150 shutdown_connections 2024-04-24T09:10:12.205 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:12.205+0000 7f1d05600640 1 -- 172.21.15.83:0/3473904150 wait complete. 2024-04-24T09:10:12.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:10:12 smithi138 ceph-mon[31441]: pgmap v586: 129 pgs: 129 active+clean; 620 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 978 KiB/s rd, 3.1 MiB/s wr, 160 op/s 2024-04-24T09:10:12.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:10:12 smithi189 ceph-mon[31791]: pgmap v586: 129 pgs: 129 active+clean; 620 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 978 KiB/s rd, 3.1 MiB/s wr, 160 op/s 2024-04-24T09:10:12.552 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:10:12 smithi083 ceph-mon[27907]: pgmap v586: 129 pgs: 129 active+clean; 620 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 978 KiB/s rd, 3.1 MiB/s wr, 160 op/s 2024-04-24T09:10:13.174 INFO:tasks.workunit.client.0.smithi083.stdout: 10 36070 6.82 MB/sec execute 164 sec latency 135.574 ms 2024-04-24T09:10:14.174 INFO:tasks.workunit.client.0.smithi083.stdout: 10 36150 6.79 MB/sec execute 165 sec latency 381.823 ms 2024-04-24T09:10:14.326 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:10:14 smithi138 ceph-mon[31441]: pgmap v587: 129 pgs: 129 active+clean; 632 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.9 MiB/s wr, 281 op/s 2024-04-24T09:10:14.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:10:14 smithi189 ceph-mon[31791]: pgmap v587: 129 pgs: 129 active+clean; 632 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.9 MiB/s wr, 281 op/s 2024-04-24T09:10:14.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:10:14 smithi083 ceph-mon[27907]: pgmap v587: 129 pgs: 129 active+clean; 632 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.9 MiB/s wr, 281 op/s 2024-04-24T09:10:15.174 INFO:tasks.workunit.client.0.smithi083.stdout: 10 36289 6.80 MB/sec execute 166 sec latency 75.390 ms 2024-04-24T09:10:16.174 INFO:tasks.workunit.client.0.smithi083.stdout: 10 36416 6.80 MB/sec execute 167 sec latency 67.104 ms 2024-04-24T09:10:16.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:10:16 smithi138 ceph-mon[31441]: pgmap v588: 129 pgs: 129 active+clean; 632 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.8 MiB/s wr, 276 op/s 2024-04-24T09:10:16.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:10:16 smithi189 ceph-mon[31791]: pgmap v588: 129 pgs: 129 active+clean; 632 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.8 MiB/s wr, 276 op/s 2024-04-24T09:10:16.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:10:16 smithi083 ceph-mon[27907]: pgmap v588: 129 pgs: 129 active+clean; 632 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.8 MiB/s wr, 276 op/s 2024-04-24T09:10:17.174 INFO:tasks.workunit.client.0.smithi083.stdout: 10 36529 6.77 MB/sec execute 168 sec latency 83.422 ms 2024-04-24T09:10:18.174 INFO:tasks.workunit.client.0.smithi083.stdout: 10 36648 6.77 MB/sec execute 169 sec latency 89.182 ms 2024-04-24T09:10:18.354 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:10:18 smithi083 ceph-mon[27907]: pgmap v589: 129 pgs: 129 active+clean; 632 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.8 MiB/s wr, 276 op/s 2024-04-24T09:10:18.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:10:18 smithi138 ceph-mon[31441]: pgmap v589: 129 pgs: 129 active+clean; 632 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.8 MiB/s wr, 276 op/s 2024-04-24T09:10:18.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:10:18 smithi189 ceph-mon[31791]: pgmap v589: 129 pgs: 129 active+clean; 632 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.8 MiB/s wr, 276 op/s 2024-04-24T09:10:19.174 INFO:tasks.workunit.client.0.smithi083.stdout: 10 36756 6.74 MB/sec execute 170 sec latency 85.897 ms 2024-04-24T09:10:20.174 INFO:tasks.workunit.client.0.smithi083.stdout: 10 36878 6.72 MB/sec execute 171 sec latency 87.099 ms 2024-04-24T09:10:20.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:10:20 smithi138 ceph-mon[31441]: pgmap v590: 129 pgs: 129 active+clean; 640 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.5 MiB/s wr, 259 op/s 2024-04-24T09:10:20.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:10:20 smithi189 ceph-mon[31791]: pgmap v590: 129 pgs: 129 active+clean; 640 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.5 MiB/s wr, 259 op/s 2024-04-24T09:10:20.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:10:20 smithi083 ceph-mon[27907]: pgmap v590: 129 pgs: 129 active+clean; 640 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.5 MiB/s wr, 259 op/s 2024-04-24T09:10:21.174 INFO:tasks.workunit.client.0.smithi083.stdout: 10 37022 6.74 MB/sec execute 172 sec latency 68.443 ms 2024-04-24T09:10:22.175 INFO:tasks.workunit.client.0.smithi083.stdout: 10 37180 6.75 MB/sec execute 173 sec latency 75.079 ms 2024-04-24T09:10:22.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:10:22 smithi138 ceph-mon[31441]: pgmap v591: 129 pgs: 129 active+clean; 640 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.0 MiB/s wr, 231 op/s 2024-04-24T09:10:22.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:10:22 smithi189 ceph-mon[31791]: pgmap v591: 129 pgs: 129 active+clean; 640 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.0 MiB/s wr, 231 op/s 2024-04-24T09:10:22.552 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:10:22 smithi083 ceph-mon[27907]: pgmap v591: 129 pgs: 129 active+clean; 640 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.0 MiB/s wr, 231 op/s 2024-04-24T09:10:23.175 INFO:tasks.workunit.client.0.smithi083.stdout: 10 37322 6.76 MB/sec execute 174 sec latency 97.766 ms 2024-04-24T09:10:23.367 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T09:10:23.368 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T09:10:23.443 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:10:23 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:10:23.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:10:23 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:10:23.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:10:23 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:10:23.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.626+0000 7f2147c00640 1 -- 172.21.15.83:0/1928207735 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f21401044b0 msgr2=0x7f2140104890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:23.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.626+0000 7f2147c00640 1 --2- 172.21.15.83:0/1928207735 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f21401044b0 0x7f2140104890 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f2134009900 tx=0x7f213402a3f0 comp rx=0 tx=0).stop 2024-04-24T09:10:23.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.627+0000 7f2147c00640 1 -- 172.21.15.83:0/1928207735 shutdown_connections 2024-04-24T09:10:23.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.627+0000 7f2147c00640 1 --2- 172.21.15.83:0/1928207735 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2140109580 0x7f2140111150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:23.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.627+0000 7f2147c00640 1 --2- 172.21.15.83:0/1928207735 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2140104e60 0x7f2140108ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:23.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.627+0000 7f2147c00640 1 --2- 172.21.15.83:0/1928207735 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f21401044b0 0x7f2140104890 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:23.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.627+0000 7f2147c00640 1 -- 172.21.15.83:0/1928207735 >> 172.21.15.83:0/1928207735 conn(0x7f21400ffd90 msgr2=0x7f21401021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:23.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.627+0000 7f2147c00640 1 -- 172.21.15.83:0/1928207735 shutdown_connections 2024-04-24T09:10:23.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.627+0000 7f2147c00640 1 -- 172.21.15.83:0/1928207735 wait complete. 2024-04-24T09:10:23.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.628+0000 7f2147c00640 1 Processor -- start 2024-04-24T09:10:23.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.628+0000 7f2147c00640 1 -- start start 2024-04-24T09:10:23.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.629+0000 7f2147c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f21401044b0 0x7f21401110d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:23.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.629+0000 7f2147c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2140109580 0x7f214010e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:23.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.629+0000 7f2147c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f214010e810 0x7f21401ab250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:23.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.629+0000 7f2145200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f21401044b0 0x7f21401110d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:23.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.630+0000 7f2147c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2140114510 con 0x7f21401044b0 2024-04-24T09:10:23.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.630+0000 7f2147c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2140114390 con 0x7f214010e810 2024-04-24T09:10:23.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.630+0000 7f2147c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2140114690 con 0x7f2140109580 2024-04-24T09:10:23.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.630+0000 7f2145200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f21401044b0 0x7f21401110d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34518/0 (socket says 172.21.15.83:34518) 2024-04-24T09:10:23.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.630+0000 7f2145200640 1 -- 172.21.15.83:0/3380983210 learned_addr learned my addr 172.21.15.83:0/3380983210 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:10:23.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.630+0000 7f213fe00640 1 --2- 172.21.15.83:0/3380983210 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2140109580 0x7f214010e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:23.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.630+0000 7f2145c00640 1 --2- 172.21.15.83:0/3380983210 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f214010e810 0x7f21401ab250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:23.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.630+0000 7f2145200640 1 -- 172.21.15.83:0/3380983210 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2140109580 msgr2=0x7f214010e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:23.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.630+0000 7f2145200640 1 --2- 172.21.15.83:0/3380983210 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2140109580 0x7f214010e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:23.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.630+0000 7f2145200640 1 -- 172.21.15.83:0/3380983210 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f214010e810 msgr2=0x7f21401ab250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:23.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.630+0000 7f2145200640 1 --2- 172.21.15.83:0/3380983210 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f214010e810 0x7f21401ab250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:23.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.630+0000 7f2145200640 1 -- 172.21.15.83:0/3380983210 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21401ab790 con 0x7f21401044b0 2024-04-24T09:10:23.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.631+0000 7f2145200640 1 --2- 172.21.15.83:0/3380983210 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f21401044b0 0x7f21401110d0 secure :-1 s=READY pgs=2420 cs=0 l=1 rev1=1 crypto rx=0x7f2134005ab0 tx=0x7f2134003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:23.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.631+0000 7f213d600640 1 -- 172.21.15.83:0/3380983210 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f213403f070 con 0x7f21401044b0 2024-04-24T09:10:23.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.631+0000 7f213d600640 1 -- 172.21.15.83:0/3380983210 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2134031b50 con 0x7f21401044b0 2024-04-24T09:10:23.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.631+0000 7f213d600640 1 -- 172.21.15.83:0/3380983210 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2134035030 con 0x7f21401044b0 2024-04-24T09:10:23.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.632+0000 7f2147c00640 1 -- 172.21.15.83:0/3380983210 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21401ab960 con 0x7f21401044b0 2024-04-24T09:10:23.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.632+0000 7f2147c00640 1 -- 172.21.15.83:0/3380983210 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21401abd70 con 0x7f21401044b0 2024-04-24T09:10:23.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.638+0000 7f214a9bc740 1 Processor -- start 2024-04-24T09:10:23.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.638+0000 7f214a9bc740 1 -- start start 2024-04-24T09:10:23.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.639+0000 7f214a9bc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5585b014c860 0x5585b014cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:23.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.639+0000 7f214a9bc740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5585b014d180 0x5585b0157310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:23.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.639+0000 7f214a9bc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5585b0157850 0x5585b0159c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:23.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.639+0000 7f214a9bc740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5585b00b21f0 con 0x5585b0157850 2024-04-24T09:10:23.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.639+0000 7f214a9bc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5585b0058350 con 0x5585b014c860 2024-04-24T09:10:23.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.639+0000 7f214a9bc740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5585b00ae1b0 con 0x5585b014d180 2024-04-24T09:10:23.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.640+0000 7f213d600640 1 -- 172.21.15.83:0/3380983210 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2134042070 con 0x7f21401044b0 2024-04-24T09:10:23.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.640+0000 7f2145200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5585b014d180 0x5585b0157310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:23.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.640+0000 7f2145200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5585b014d180 0x5585b0157310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:51096/0 (socket says 172.21.15.83:51096) 2024-04-24T09:10:23.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.640+0000 7f2145200640 1 -- 172.21.15.83:0/3770567039 learned_addr learned my addr 172.21.15.83:0/3770567039 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:10:23.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.641+0000 7f2145c00640 1 --2- 172.21.15.83:0/3770567039 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5585b014c860 0x5585b014cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:23.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.641+0000 7f213fe00640 1 --2- 172.21.15.83:0/3770567039 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5585b0157850 0x5585b0159c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:23.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.641+0000 7f2145200640 1 -- 172.21.15.83:0/3770567039 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5585b014c860 msgr2=0x5585b014cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:23.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.641+0000 7f2145200640 1 --2- 172.21.15.83:0/3770567039 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5585b014c860 0x5585b014cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:23.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.641+0000 7f2145200640 1 -- 172.21.15.83:0/3770567039 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5585b0157850 msgr2=0x5585b0159c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:23.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.641+0000 7f2145200640 1 --2- 172.21.15.83:0/3770567039 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5585b0157850 0x5585b0159c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:23.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.641+0000 7f2145200640 1 -- 172.21.15.83:0/3770567039 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585b00acf50 con 0x5585b014d180 2024-04-24T09:10:23.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.641+0000 7f213fe00640 1 --2- 172.21.15.83:0/3770567039 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5585b0157850 0x5585b0159c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:10:23.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.641+0000 7f2145200640 1 --2- 172.21.15.83:0/3770567039 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5585b014d180 0x5585b0157310 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f21340ee580 tx=0x7f21340f9580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb5914ea9d77ced4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:23.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.642+0000 7f2132000640 1 -- 172.21.15.83:0/3770567039 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21340f3310 con 0x5585b014d180 2024-04-24T09:10:23.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.642+0000 7f2132000640 1 -- 172.21.15.83:0/3770567039 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f21340f3db0 con 0x5585b014d180 2024-04-24T09:10:23.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.643+0000 7f214a9bc740 1 -- 172.21.15.83:0/3770567039 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5585b014d180 msgr2=0x5585b0157310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:23.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.644+0000 7f2132000640 1 -- 172.21.15.83:0/3770567039 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21340f3310 con 0x5585b014d180 2024-04-24T09:10:23.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.644+0000 7f213d600640 1 --2- 172.21.15.83:0/3380983210 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f21280a3f20 0x7f21280a63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:23.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.644+0000 7f214a9bc740 1 --2- 172.21.15.83:0/3770567039 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5585b014d180 0x5585b0157310 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f21340ee580 tx=0x7f21340f9580 comp rx=0 tx=0).stop 2024-04-24T09:10:23.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.644+0000 7f213d600640 1 -- 172.21.15.83:0/3380983210 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 9583+0+0 (secure 0 0 0) 0x7f21340f1370 con 0x7f21401044b0 2024-04-24T09:10:23.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.645+0000 7f2145c00640 1 --2- 172.21.15.83:0/3380983210 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f21280a3f20 0x7f21280a63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:23.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.645+0000 7f214a9bc740 1 -- 172.21.15.83:0/3770567039 shutdown_connections 2024-04-24T09:10:23.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.645+0000 7f214a9bc740 1 --2- 172.21.15.83:0/3770567039 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5585b0157850 0x5585b0159c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:23.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.645+0000 7f214a9bc740 1 --2- 172.21.15.83:0/3770567039 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5585b014d180 0x5585b0157310 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:23.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.645+0000 7f214a9bc740 1 --2- 172.21.15.83:0/3770567039 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5585b014c860 0x5585b014cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:23.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.645+0000 7f214a9bc740 1 -- 172.21.15.83:0/3770567039 >> 172.21.15.83:0/3770567039 conn(0x5585b0148b80 msgr2=0x5585b0148f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:23.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.646+0000 7f214a9bc740 1 -- 172.21.15.83:0/3770567039 shutdown_connections 2024-04-24T09:10:23.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.646+0000 7f214a9bc740 1 -- 172.21.15.83:0/3770567039 wait complete. 2024-04-24T09:10:23.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.646+0000 7f2145c00640 1 --2- 172.21.15.83:0/3380983210 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f21280a3f20 0x7f21280a63e0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f21380092f0 tx=0x7f2138009280 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:23.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.647+0000 7f214a9bc740 1 -- start start 2024-04-24T09:10:23.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.647+0000 7f214a9bc740 1 Processor -- start 2024-04-24T09:10:23.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.648+0000 7f214a9bc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5585b014c860 0x5585b01546e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:23.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.648+0000 7f214a9bc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5585b014d180 0x5585b0154c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:23.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.649+0000 7f214a9bc740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5585b0157850 0x5585b0155160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:23.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.649+0000 7f214a9bc740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5585b015a400 con 0x5585b014d180 2024-04-24T09:10:23.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.649+0000 7f214a9bc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5585b006b690 con 0x5585b014c860 2024-04-24T09:10:23.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.649+0000 7f213fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5585b014d180 0x5585b0154c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:23.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.649+0000 7f2145200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5585b014c860 0x5585b01546e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:23.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.649+0000 7f2145200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5585b014c860 0x5585b01546e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:41700/0 (socket says 172.21.15.83:41700) 2024-04-24T09:10:23.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.649+0000 7f2145200640 1 -- 172.21.15.83:0/2206512797 learned_addr learned my addr 172.21.15.83:0/2206512797 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:10:23.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.649+0000 7f214a9bc740 1 -- 172.21.15.83:0/2206512797 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5585b015a580 con 0x5585b0157850 2024-04-24T09:10:23.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.649+0000 7f214a9bc740 10 client.0 _ll_register_callbacks cb 0x5585b00a7650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:10:23.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.650+0000 7f2145c00640 1 --2- 172.21.15.83:0/2206512797 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5585b0157850 0x5585b0155160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:23.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.650+0000 7f213fe00640 1 -- 172.21.15.83:0/2206512797 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5585b0157850 msgr2=0x5585b0155160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:23.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.650+0000 7f213fe00640 1 --2- 172.21.15.83:0/2206512797 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5585b0157850 0x5585b0155160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:23.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.650+0000 7f213fe00640 1 -- 172.21.15.83:0/2206512797 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5585b014c860 msgr2=0x5585b01546e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:23.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.650+0000 7f213fe00640 1 --2- 172.21.15.83:0/2206512797 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5585b014c860 0x5585b01546e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:23.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.650+0000 7f213fe00640 1 -- 172.21.15.83:0/2206512797 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585b006a920 con 0x5585b014d180 2024-04-24T09:10:23.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.650+0000 7f2145200640 1 --2- 172.21.15.83:0/2206512797 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5585b014c860 0x5585b01546e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:10:23.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.651+0000 7f213fe00640 1 --2- 172.21.15.83:0/2206512797 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5585b014d180 0x5585b0154c20 secure :-1 s=READY pgs=2421 cs=0 l=1 rev1=1 crypto rx=0x7f214010f930 tx=0x7f212c012b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48e02cc7c7bbaf0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:23.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.651+0000 7f2126a00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:10:23.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.652+0000 7f2126a00640 1 -- 172.21.15.83:0/2206512797 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f212c004460 con 0x5585b014d180 2024-04-24T09:10:23.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.652+0000 7f2126a00640 1 -- 172.21.15.83:0/2206512797 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(120..120 src has 1..120) ==== 9583+0+0 (secure 0 0 0) 0x7f212c010980 con 0x5585b014d180 2024-04-24T09:10:23.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.653+0000 7f214a9bc740 1 -- 172.21.15.83:0/2206512797 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21340f3db0 con 0x5585b014d180 2024-04-24T09:10:23.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.654+0000 7f2126a00640 1 -- 172.21.15.83:0/2206512797 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f212c010b80 con 0x5585b014d180 2024-04-24T09:10:23.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.655+0000 7f214a9bc740 10 client.22047 fetch_fsmap learned FSMap version 61 2024-04-24T09:10:23.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.655+0000 7f214a9bc740 1 -- 172.21.15.83:0/2206512797 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x5585afe8e7d0 con 0x5585b014d180 2024-04-24T09:10:23.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.655+0000 7f2126a00640 1 -- 172.21.15.83:0/2206512797 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f21340f3db0 con 0x5585b014d180 2024-04-24T09:10:23.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.656+0000 7f2126a00640 1 -- 172.21.15.83:0/2206512797 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 61) ==== 8362+0+0 (secure 0 0 0) 0x7f212c019070 con 0x5585b014d180 2024-04-24T09:10:23.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.656+0000 7f214a9bc740 10 client.22047 fetch_fsmap finished waiting for FSMap version 61 2024-04-24T09:10:23.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.656+0000 7f214a9bc740 10 client.22047 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:10:23.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.656+0000 7f214a9bc740 20 client.22047 populate_metadata read hostname 'smithi083' 2024-04-24T09:10:23.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.656+0000 7f214a9bc740 1 --2- 172.21.15.83:0/2206512797 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5585b01675f0 0x5585b0169a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:23.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.656+0000 7f214a9bc740 4 client.22047 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:10:23.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.656+0000 7f214a9bc740 1 -- 172.21.15.83:0/2206512797 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5585b00bfc10 con 0x5585b01675f0 2024-04-24T09:10:23.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.657+0000 7f2145200640 1 --2- 172.21.15.83:0/2206512797 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5585b01675f0 0x5585b0169a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:23.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.658+0000 7f2145200640 1 --2- 172.21.15.83:0/2206512797 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5585b01675f0 0x5585b0169a50 crc :-1 s=READY pgs=467 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81f8864bbe91c52f server_cookie=57b46bb72d476a1b in_seq=0 out_seq=0 2024-04-24T09:10:23.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.658+0000 7f2126a00640 10 client.22047 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:10:23.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.661+0000 7f2126a00640 1 -- 172.21.15.83:0/2206512797 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5585b00bfc10 con 0x5585b01675f0 2024-04-24T09:10:23.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.661+0000 7f2126a00640 10 client.22047 handle_command_reply: tid=0 2024-04-24T09:10:23.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.661+0000 7f214a9bc740 1 client.22047 shutdown 2024-04-24T09:10:23.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.665+0000 7f214a9bc740 1 -- 172.21.15.83:0/2206512797 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5585b014d180 msgr2=0x5585b0154c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:23.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.665+0000 7f214a9bc740 1 --2- 172.21.15.83:0/2206512797 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5585b014d180 0x5585b0154c20 secure :-1 s=READY pgs=2421 cs=0 l=1 rev1=1 crypto rx=0x7f214010f930 tx=0x7f212c012b60 comp rx=0 tx=0).stop 2024-04-24T09:10:23.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.665+0000 7f214a9bc740 1 -- 172.21.15.83:0/2206512797 shutdown_connections 2024-04-24T09:10:23.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.665+0000 7f214a9bc740 1 --2- 172.21.15.83:0/2206512797 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5585b01675f0 0x5585b0169a50 crc :-1 s=READY pgs=467 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:23.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.665+0000 7f214a9bc740 1 --2- 172.21.15.83:0/2206512797 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5585b0157850 0x5585b0155160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:23.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.665+0000 7f214a9bc740 1 --2- 172.21.15.83:0/2206512797 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5585b014d180 0x5585b0154c20 unknown :-1 s=CLOSED pgs=2421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:23.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.665+0000 7f214a9bc740 1 --2- 172.21.15.83:0/2206512797 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5585b014c860 0x5585b01546e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:23.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.665+0000 7f214a9bc740 1 -- 172.21.15.83:0/2206512797 >> 172.21.15.83:0/2206512797 conn(0x5585b0148b80 msgr2=0x5585b014e980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:23.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.665+0000 7f2126a00640 0 client.22047 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:10:23.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.665+0000 7f214a9bc740 1 -- 172.21.15.83:0/2206512797 shutdown_connections 2024-04-24T09:10:23.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.666+0000 7f214a9bc740 1 -- 172.21.15.83:0/2206512797 wait complete. 2024-04-24T09:10:23.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.666+0000 7f214a9bc740 20 client.22047 trim_cache size 0 max 16384 2024-04-24T09:10:23.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.704+0000 7f214a9bc740 1 Processor -- start 2024-04-24T09:10:23.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.704+0000 7f214a9bc740 1 -- start start 2024-04-24T09:10:23.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.705+0000 7f214a9bc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5585b014c860 0x5585b01632c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:23.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.706+0000 7f214a9bc740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5585b014d180 0x5585b014f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:23.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.706+0000 7f214a9bc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5585b0157850 0x5585b016f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:23.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.706+0000 7f214a9bc740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f21401b5d90 con 0x5585b014c860 2024-04-24T09:10:23.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.706+0000 7f214a9bc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f21401b78d0 con 0x5585b0157850 2024-04-24T09:10:23.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.706+0000 7f214a9bc740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f21401b6460 con 0x5585b014d180 2024-04-24T09:10:23.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.706+0000 7f2145200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5585b014c860 0x5585b01632c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:23.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.706+0000 7f2145200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5585b014c860 0x5585b01632c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34550/0 (socket says 172.21.15.83:34550) 2024-04-24T09:10:23.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.706+0000 7f2145200640 1 -- 172.21.15.83:0/1225287850 learned_addr learned my addr 172.21.15.83:0/1225287850 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:10:23.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.706+0000 7f213fe00640 1 --2- 172.21.15.83:0/1225287850 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5585b014d180 0x5585b014f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:23.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.707+0000 7f2145200640 1 -- 172.21.15.83:0/1225287850 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5585b014d180 msgr2=0x5585b014f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:23.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.707+0000 7f2145c00640 1 --2- 172.21.15.83:0/1225287850 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5585b0157850 0x5585b016f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:23.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.707+0000 7f2145200640 1 --2- 172.21.15.83:0/1225287850 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5585b014d180 0x5585b014f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:23.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.707+0000 7f2145200640 1 -- 172.21.15.83:0/1225287850 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5585b0157850 msgr2=0x5585b016f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:23.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.707+0000 7f2145200640 1 --2- 172.21.15.83:0/1225287850 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5585b0157850 0x5585b016f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:23.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.707+0000 7f2145200640 1 -- 172.21.15.83:0/1225287850 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585afdc43f0 con 0x5585b014c860 2024-04-24T09:10:23.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.707+0000 7f2145c00640 1 --2- 172.21.15.83:0/1225287850 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5585b0157850 0x5585b016f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:10:23.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.707+0000 7f2145200640 1 --2- 172.21.15.83:0/1225287850 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5585b014c860 0x5585b01632c0 secure :-1 s=READY pgs=2422 cs=0 l=1 rev1=1 crypto rx=0x7f21340f3fd0 tx=0x7f213402a8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92d786e13c54c3da server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:23.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.708+0000 7f2131600640 1 -- 172.21.15.83:0/1225287850 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21340bc9d0 con 0x5585b014c860 2024-04-24T09:10:23.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.708+0000 7f2131600640 1 -- 172.21.15.83:0/1225287850 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f21340bd590 con 0x5585b014c860 2024-04-24T09:10:23.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.708+0000 7f2131600640 1 -- 172.21.15.83:0/1225287850 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21340f1170 con 0x5585b014c860 2024-04-24T09:10:23.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.709+0000 7f214a9bc740 1 -- 172.21.15.83:0/1225287850 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5585b014c860 msgr2=0x5585b01632c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:23.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.709+0000 7f214a9bc740 1 --2- 172.21.15.83:0/1225287850 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5585b014c860 0x5585b01632c0 secure :-1 s=READY pgs=2422 cs=0 l=1 rev1=1 crypto rx=0x7f21340f3fd0 tx=0x7f213402a8f0 comp rx=0 tx=0).stop 2024-04-24T09:10:23.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.709+0000 7f214a9bc740 1 -- 172.21.15.83:0/1225287850 shutdown_connections 2024-04-24T09:10:23.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.709+0000 7f214a9bc740 1 --2- 172.21.15.83:0/1225287850 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5585b0157850 0x5585b016f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:23.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.709+0000 7f214a9bc740 1 --2- 172.21.15.83:0/1225287850 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5585b014d180 0x5585b014f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:23.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.709+0000 7f214a9bc740 1 --2- 172.21.15.83:0/1225287850 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5585b014c860 0x5585b01632c0 unknown :-1 s=CLOSED pgs=2422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:23.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.709+0000 7f214a9bc740 1 -- 172.21.15.83:0/1225287850 >> 172.21.15.83:0/1225287850 conn(0x5585b0148b80 msgr2=0x5585b014fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:23.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.709+0000 7f214a9bc740 1 -- 172.21.15.83:0/1225287850 shutdown_connections 2024-04-24T09:10:23.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.709+0000 7f214a9bc740 1 -- 172.21.15.83:0/1225287850 wait complete. 2024-04-24T09:10:23.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.710+0000 7f214a9bc740 1 -- start start 2024-04-24T09:10:23.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.710+0000 7f214a9bc740 1 Processor -- start 2024-04-24T09:10:23.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.711+0000 7f214a9bc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5585b014c860 0x5585b015fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:23.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.711+0000 7f214a9bc740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5585b014d180 0x5585b0151e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:23.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.711+0000 7f214a9bc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5585b0157850 0x5585b0150990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:23.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.711+0000 7f214a9bc740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f21401b7200 con 0x5585b0157850 2024-04-24T09:10:23.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.711+0000 7f214a9bc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f21401b6b30 con 0x5585b014c860 2024-04-24T09:10:23.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.711+0000 7f214a9bc740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f21401b3d90 con 0x5585b014d180 2024-04-24T09:10:23.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.711+0000 7f214a9bc740 10 client.0 _ll_register_callbacks cb 0x7f212c009550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:10:23.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.711+0000 7f213fe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5585b014d180 0x5585b0151e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:23.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.711+0000 7f213fe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5585b014d180 0x5585b0151e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:51118/0 (socket says 172.21.15.83:51118) 2024-04-24T09:10:23.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.711+0000 7f213fe00640 1 -- 172.21.15.83:0/2940444133 learned_addr learned my addr 172.21.15.83:0/2940444133 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:10:23.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.711+0000 7f2145c00640 1 --2- 172.21.15.83:0/2940444133 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5585b0157850 0x5585b0150990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:23.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.711+0000 7f2145200640 1 --2- 172.21.15.83:0/2940444133 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5585b014c860 0x5585b015fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:23.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.713+0000 7f213fe00640 1 -- 172.21.15.83:0/2940444133 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5585b014c860 msgr2=0x5585b015fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:23.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.713+0000 7f213fe00640 1 --2- 172.21.15.83:0/2940444133 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5585b014c860 0x5585b015fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:23.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.713+0000 7f213fe00640 1 -- 172.21.15.83:0/2940444133 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5585b0157850 msgr2=0x5585b0150990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:23.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.713+0000 7f213fe00640 1 --2- 172.21.15.83:0/2940444133 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5585b0157850 0x5585b0150990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:23.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.713+0000 7f213fe00640 1 -- 172.21.15.83:0/2940444133 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585afd840f0 con 0x5585b014d180 2024-04-24T09:10:23.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.714+0000 7f213fe00640 1 --2- 172.21.15.83:0/2940444133 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5585b014d180 0x5585b0151e40 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x5585b0155cb0 tx=0x7f212c009740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d5d6fcc2d3a1184 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:23.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.714+0000 7f2126a00640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-04-24T09:10:23.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.715+0000 7f2126a00640 1 -- 172.21.15.83:0/2940444133 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f212c0054c0 con 0x5585b014d180 2024-04-24T09:10:23.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.715+0000 7f2126a00640 1 -- 172.21.15.83:0/2940444133 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(120..120 src has 1..120) ==== 9583+0+0 (secure 0 0 0) 0x7f212c025a90 con 0x5585b014d180 2024-04-24T09:10:23.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.716+0000 7f214a9bc740 1 -- 172.21.15.83:0/2940444133 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21340bd590 con 0x5585b014d180 2024-04-24T09:10:23.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.716+0000 7f2126a00640 1 -- 172.21.15.83:0/2940444133 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f212c034050 con 0x5585b014d180 2024-04-24T09:10:23.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.717+0000 7f214a9bc740 10 client.29101 fetch_fsmap learned FSMap version 61 2024-04-24T09:10:23.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.717+0000 7f214a9bc740 1 -- 172.21.15.83:0/2940444133 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x5585b0077810 con 0x5585b014d180 2024-04-24T09:10:23.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.717+0000 7f2126a00640 1 -- 172.21.15.83:0/2940444133 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f21340bd590 con 0x5585b014d180 2024-04-24T09:10:23.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.718+0000 7f2126a00640 1 -- 172.21.15.83:0/2940444133 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 61) ==== 8362+0+0 (secure 0 0 0) 0x7f212c019040 con 0x5585b014d180 2024-04-24T09:10:23.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.718+0000 7f214a9bc740 10 client.29101 fetch_fsmap finished waiting for FSMap version 61 2024-04-24T09:10:23.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.718+0000 7f214a9bc740 10 client.29101 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:10:23.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.718+0000 7f214a9bc740 20 client.29101 populate_metadata read hostname 'smithi083' 2024-04-24T09:10:23.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.718+0000 7f214a9bc740 1 --2- 172.21.15.83:0/2940444133 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5585b01de570 0x5585b01e0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:23.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.718+0000 7f214a9bc740 4 client.29101 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T09:10:23.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.718+0000 7f214a9bc740 1 -- 172.21.15.83:0/2940444133 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5585b00b9a80 con 0x5585b01de570 2024-04-24T09:10:23.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.718+0000 7f2145200640 1 --2- 172.21.15.83:0/2940444133 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5585b01de570 0x5585b01e0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:23.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.719+0000 7f2145200640 1 --2- 172.21.15.83:0/2940444133 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5585b01de570 0x5585b01e0990 crc :-1 s=READY pgs=468 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=afe3af595a8e3dfb server_cookie=a42f9d655350e3cc in_seq=0 out_seq=0 2024-04-24T09:10:23.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.720+0000 7f2126a00640 10 client.29101 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:10:23.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.720+0000 7f2126a00640 1 -- 172.21.15.83:0/2940444133 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5585b00b9a80 con 0x5585b01de570 2024-04-24T09:10:23.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.720+0000 7f2126a00640 10 client.29101 handle_command_reply: tid=0 2024-04-24T09:10:23.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.720+0000 7f214a9bc740 1 client.29101 shutdown 2024-04-24T09:10:23.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.720+0000 7f214a9bc740 1 -- 172.21.15.83:0/2940444133 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5585b014d180 msgr2=0x5585b0151e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:23.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.720+0000 7f214a9bc740 1 --2- 172.21.15.83:0/2940444133 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5585b014d180 0x5585b0151e40 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x5585b0155cb0 tx=0x7f212c009740 comp rx=0 tx=0).stop 2024-04-24T09:10:23.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.720+0000 7f214a9bc740 1 -- 172.21.15.83:0/2940444133 shutdown_connections 2024-04-24T09:10:23.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.720+0000 7f214a9bc740 1 --2- 172.21.15.83:0/2940444133 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5585b01de570 0x5585b01e0990 crc :-1 s=READY pgs=468 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:23.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.720+0000 7f214a9bc740 1 --2- 172.21.15.83:0/2940444133 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5585b0157850 0x5585b0150990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:23.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.720+0000 7f214a9bc740 1 --2- 172.21.15.83:0/2940444133 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5585b014d180 0x5585b0151e40 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:23.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.720+0000 7f214a9bc740 1 --2- 172.21.15.83:0/2940444133 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5585b014c860 0x5585b015fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:23.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.720+0000 7f2126a00640 0 client.29101 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:10:23.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.720+0000 7f214a9bc740 1 -- 172.21.15.83:0/2940444133 >> 172.21.15.83:0/2940444133 conn(0x5585b0148b80 msgr2=0x5585b0152ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:23.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.721+0000 7f214a9bc740 1 -- 172.21.15.83:0/2940444133 shutdown_connections 2024-04-24T09:10:23.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.721+0000 7f214a9bc740 1 -- 172.21.15.83:0/2940444133 wait complete. 2024-04-24T09:10:23.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.721+0000 7f214a9bc740 20 client.29101 trim_cache size 0 max 16384 2024-04-24T09:10:23.722 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T09:10:23.722 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-04-24T09:10:23.722 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-04-24T09:10:23.722 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T09:10:23.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.722+0000 7f2147c00640 1 -- 172.21.15.83:0/3380983210 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f21280a3f20 msgr2=0x7f21280a63e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:23.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.722+0000 7f2147c00640 1 --2- 172.21.15.83:0/3380983210 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f21280a3f20 0x7f21280a63e0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f21380092f0 tx=0x7f2138009280 comp rx=0 tx=0).stop 2024-04-24T09:10:23.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.722+0000 7f2147c00640 1 -- 172.21.15.83:0/3380983210 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f21401044b0 msgr2=0x7f21401110d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:23.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.722+0000 7f2147c00640 1 --2- 172.21.15.83:0/3380983210 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f21401044b0 0x7f21401110d0 secure :-1 s=READY pgs=2420 cs=0 l=1 rev1=1 crypto rx=0x7f2134005ab0 tx=0x7f2134003c20 comp rx=0 tx=0).stop 2024-04-24T09:10:23.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.723+0000 7f2147c00640 1 -- 172.21.15.83:0/3380983210 shutdown_connections 2024-04-24T09:10:23.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.723+0000 7f2147c00640 1 --2- 172.21.15.83:0/3380983210 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f21280a3f20 0x7f21280a63e0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:23.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.723+0000 7f2147c00640 1 --2- 172.21.15.83:0/3380983210 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f214010e810 0x7f21401ab250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:23.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.723+0000 7f2147c00640 1 --2- 172.21.15.83:0/3380983210 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2140109580 0x7f214010e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:23.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.723+0000 7f2147c00640 1 --2- 172.21.15.83:0/3380983210 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f21401044b0 0x7f21401110d0 unknown :-1 s=CLOSED pgs=2420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:23.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.723+0000 7f2147c00640 1 -- 172.21.15.83:0/3380983210 >> 172.21.15.83:0/3380983210 conn(0x7f21400ffd90 msgr2=0x7f2140102110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:23.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.723+0000 7f2147c00640 1 -- 172.21.15.83:0/3380983210 shutdown_connections 2024-04-24T09:10:23.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:23.723+0000 7f2147c00640 1 -- 172.21.15.83:0/3380983210 wait complete. 2024-04-24T09:10:23.748 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T09:10:23.748 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T09:10:24.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.011+0000 7f90af800640 1 -- 172.21.15.83:0/1969206356 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f90a8108fd0 msgr2=0x7f90a81154e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:24.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.011+0000 7f90af800640 1 --2- 172.21.15.83:0/1969206356 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f90a8108fd0 0x7f90a81154e0 secure :-1 s=READY pgs=2423 cs=0 l=1 rev1=1 crypto rx=0x7f90a0009cd0 tx=0x7f90a002a8f0 comp rx=0 tx=0).stop 2024-04-24T09:10:24.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.011+0000 7f90af800640 1 -- 172.21.15.83:0/1969206356 shutdown_connections 2024-04-24T09:10:24.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.011+0000 7f90af800640 1 --2- 172.21.15.83:0/1969206356 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f90a8108fd0 0x7f90a81154e0 unknown :-1 s=CLOSED pgs=2423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:24.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.011+0000 7f90af800640 1 --2- 172.21.15.83:0/1969206356 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90a8108610 0x7f90a8108a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:24.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.011+0000 7f90af800640 1 --2- 172.21.15.83:0/1969206356 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f90a810e6b0 0x7f90a810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:24.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.011+0000 7f90af800640 1 -- 172.21.15.83:0/1969206356 >> 172.21.15.83:0/1969206356 conn(0x7f90a8075800 msgr2=0x7f90a8106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:24.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.011+0000 7f90af800640 1 -- 172.21.15.83:0/1969206356 shutdown_connections 2024-04-24T09:10:24.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.011+0000 7f90af800640 1 -- 172.21.15.83:0/1969206356 wait complete. 2024-04-24T09:10:24.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.012+0000 7f90af800640 1 Processor -- start 2024-04-24T09:10:24.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.013+0000 7f90af800640 1 -- start start 2024-04-24T09:10:24.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.013+0000 7f90af800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f90a8108610 0x7f90a819ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:24.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.013+0000 7f90af800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f90a8108fd0 0x7f90a819f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:24.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.013+0000 7f90af800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90a810e6b0 0x7f90a81a35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:24.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.013+0000 7f90af800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f90a81188f0 con 0x7f90a8108610 2024-04-24T09:10:24.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.013+0000 7f90af800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f90a8118770 con 0x7f90a810e6b0 2024-04-24T09:10:24.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.013+0000 7f90af800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f90a8118a70 con 0x7f90a8108fd0 2024-04-24T09:10:24.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.013+0000 7f90ace00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f90a8108610 0x7f90a819ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:24.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.013+0000 7f90ad800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90a810e6b0 0x7f90a81a35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:24.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.013+0000 7f90ace00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f90a8108610 0x7f90a819ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34594/0 (socket says 172.21.15.83:34594) 2024-04-24T09:10:24.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.013+0000 7f90ace00640 1 -- 172.21.15.83:0/3697673284 learned_addr learned my addr 172.21.15.83:0/3697673284 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:10:24.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.013+0000 7f90a7e00640 1 --2- 172.21.15.83:0/3697673284 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f90a8108fd0 0x7f90a819f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:24.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.014+0000 7f90ad800640 1 -- 172.21.15.83:0/3697673284 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f90a8108fd0 msgr2=0x7f90a819f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:24.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.014+0000 7f90ad800640 1 --2- 172.21.15.83:0/3697673284 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f90a8108fd0 0x7f90a819f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:24.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.014+0000 7f90ad800640 1 -- 172.21.15.83:0/3697673284 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f90a8108610 msgr2=0x7f90a819ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:24.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.014+0000 7f90ad800640 1 --2- 172.21.15.83:0/3697673284 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f90a8108610 0x7f90a819ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:24.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.014+0000 7f90ad800640 1 -- 172.21.15.83:0/3697673284 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90a81a3d20 con 0x7f90a810e6b0 2024-04-24T09:10:24.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.014+0000 7f90ace00640 1 --2- 172.21.15.83:0/3697673284 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f90a8108610 0x7f90a819ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:10:24.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.014+0000 7f90ad800640 1 --2- 172.21.15.83:0/3697673284 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90a810e6b0 0x7f90a81a35f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f90a002ae10 tx=0x7f90a0003ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:24.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.015+0000 7f90a5600640 1 -- 172.21.15.83:0/3697673284 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a0004240 con 0x7f90a810e6b0 2024-04-24T09:10:24.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.015+0000 7f90af800640 1 -- 172.21.15.83:0/3697673284 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90a81a3fb0 con 0x7f90a810e6b0 2024-04-24T09:10:24.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.015+0000 7f90af800640 1 -- 172.21.15.83:0/3697673284 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90a81ab8a0 con 0x7f90a810e6b0 2024-04-24T09:10:24.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.015+0000 7f90a5600640 1 -- 172.21.15.83:0/3697673284 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f90a0040070 con 0x7f90a810e6b0 2024-04-24T09:10:24.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.015+0000 7f90a5600640 1 -- 172.21.15.83:0/3697673284 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a0031b50 con 0x7f90a810e6b0 2024-04-24T09:10:24.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.017+0000 7f90af800640 1 -- 172.21.15.83:0/3697673284 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90a8069270 con 0x7f90a810e6b0 2024-04-24T09:10:24.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.020+0000 7f90a5600640 1 -- 172.21.15.83:0/3697673284 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f90a0031cf0 con 0x7f90a810e6b0 2024-04-24T09:10:24.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.022+0000 7f90a5600640 1 --2- 172.21.15.83:0/3697673284 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f90900a3da0 0x7f90900a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:24.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.022+0000 7f90a5600640 1 -- 172.21.15.83:0/3697673284 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 9583+0+0 (secure 0 0 0) 0x7f90a00f1ef0 con 0x7f90a810e6b0 2024-04-24T09:10:24.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.023+0000 7f90ace00640 1 --2- 172.21.15.83:0/3697673284 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f90900a3da0 0x7f90900a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:24.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.023+0000 7f90ace00640 1 --2- 172.21.15.83:0/3697673284 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f90900a3da0 0x7f90900a6260 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f909c005fd0 tx=0x7f909c00b4a0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:24.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.026+0000 7f90a5600640 1 -- 172.21.15.83:0/3697673284 <== mon.2 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f90a00be220 con 0x7f90a810e6b0 2024-04-24T09:10:24.175 INFO:tasks.workunit.client.0.smithi083.stdout: 10 37444 6.74 MB/sec execute 175 sec latency 104.026 ms 2024-04-24T09:10:24.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.429+0000 7f90af800640 1 -- 172.21.15.83:0/3697673284 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f90a8110b80 con 0x7f90a810e6b0 2024-04-24T09:10:24.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.430+0000 7f90a5600640 1 -- 172.21.15.83:0/3697673284 <== mon.2 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) ==== 94+0+11944 (secure 0 0 0) 0x7f90a00bdb20 con 0x7f90a810e6b0 2024-04-24T09:10:24.433 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T09:10:24.433 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":61,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/358872257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":358872257},{"type":"v1","addr":"172.21.15.83:6835","nonce":358872257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3157070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3157070851},{"type":"v1","addr":"172.21.15.83:6837","nonce":3157070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":61,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T08:48:29.422576+0000","modified":"2024-04-24T09:09:34.468831+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34520,"mds_1":34493,"mds_2":34451,"mds_3":34427,"mds_4":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14658":{"gid":14658,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6839/4085470844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":4085470844},{"type":"v1","addr":"172.21.15.83:6839","nonce":4085470844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14676":{"gid":14676,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6841/2634754210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":2634754210},{"type":"v1","addr":"172.21.15.83:6841","nonce":2634754210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24382":{"gid":24382,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/1859921310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1859921310},{"type":"v1","addr":"172.21.15.189:6833","nonce":1859921310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/2241923197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2241923197},{"type":"v1","addr":"172.21.15.189:6835","nonce":2241923197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.189:6839/2548965336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":2548965336},{"type":"v1","addr":"172.21.15.189:6839","nonce":2548965336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26686":{"gid":26686,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6837/1678817161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":1678817161},{"type":"v1","addr":"172.21.15.189:6837","nonce":1678817161}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.138:6833/1157967658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1157967658},{"type":"v1","addr":"172.21.15.138:6833","nonce":1157967658}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.138:6835/1909219758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1909219758},{"type":"v1","addr":"172.21.15.138:6835","nonce":1909219758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34493":{"gid":34493,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.138:6837/900566754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":900566754},{"type":"v1","addr":"172.21.15.138:6837","nonce":900566754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34520":{"gid":34520,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.138:6839/2236932393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2236932393},{"type":"v1","addr":"172.21.15.138:6839","nonce":2236932393}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34520,"qdb_cluster":[24454,34520,34427,34451,34493]},"id":1}]} 2024-04-24T09:10:24.433 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 61 2024-04-24T09:10:24.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.434+0000 7f90af800640 1 -- 172.21.15.83:0/3697673284 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f90900a3da0 msgr2=0x7f90900a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:24.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.434+0000 7f90af800640 1 --2- 172.21.15.83:0/3697673284 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f90900a3da0 0x7f90900a6260 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f909c005fd0 tx=0x7f909c00b4a0 comp rx=0 tx=0).stop 2024-04-24T09:10:24.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.434+0000 7f90af800640 1 -- 172.21.15.83:0/3697673284 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90a810e6b0 msgr2=0x7f90a81a35f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:24.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.434+0000 7f90af800640 1 --2- 172.21.15.83:0/3697673284 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90a810e6b0 0x7f90a81a35f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f90a002ae10 tx=0x7f90a0003ba0 comp rx=0 tx=0).stop 2024-04-24T09:10:24.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.435+0000 7f90af800640 1 -- 172.21.15.83:0/3697673284 shutdown_connections 2024-04-24T09:10:24.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.435+0000 7f90af800640 1 --2- 172.21.15.83:0/3697673284 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f90900a3da0 0x7f90900a6260 secure :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f909c005fd0 tx=0x7f909c00b4a0 comp rx=0 tx=0).stop 2024-04-24T09:10:24.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.435+0000 7f90af800640 1 --2- 172.21.15.83:0/3697673284 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90a810e6b0 0x7f90a81a35f0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:24.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.435+0000 7f90af800640 1 --2- 172.21.15.83:0/3697673284 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f90a8108fd0 0x7f90a819f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:24.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.435+0000 7f90af800640 1 --2- 172.21.15.83:0/3697673284 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f90a8108610 0x7f90a819ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:24.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.435+0000 7f90af800640 1 -- 172.21.15.83:0/3697673284 >> 172.21.15.83:0/3697673284 conn(0x7f90a8075800 msgr2=0x7f90a8076770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:24.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.435+0000 7f90af800640 1 -- 172.21.15.83:0/3697673284 shutdown_connections 2024-04-24T09:10:24.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.435+0000 7f90af800640 1 -- 172.21.15.83:0/3697673284 wait complete. 2024-04-24T09:10:24.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:10:24 smithi138 ceph-mon[31441]: pgmap v592: 129 pgs: 129 active+clean; 643 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.4 MiB/s wr, 329 op/s 2024-04-24T09:10:24.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:10:24 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:10:24.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:10:24 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:10:24.455 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:10:24 smithi083 ceph-mon[27907]: pgmap v592: 129 pgs: 129 active+clean; 643 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.4 MiB/s wr, 329 op/s 2024-04-24T09:10:24.455 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:10:24 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:10:24.455 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:10:24 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:10:24.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:10:24 smithi189 ceph-mon[31791]: pgmap v592: 129 pgs: 129 active+clean; 643 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.4 MiB/s wr, 329 op/s 2024-04-24T09:10:24.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:10:24 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:10:24.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:10:24 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:10:24.458 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-24T09:10:24.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.734+0000 7fdb01800640 1 -- 172.21.15.83:0/3489393123 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdafc1044b0 msgr2=0x7fdafc104890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:24.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.734+0000 7fdb01800640 1 --2- 172.21.15.83:0/3489393123 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdafc1044b0 0x7fdafc104890 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fdaec009900 tx=0x7fdaec02a3f0 comp rx=0 tx=0).stop 2024-04-24T09:10:24.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.734+0000 7fdb01800640 1 -- 172.21.15.83:0/3489393123 shutdown_connections 2024-04-24T09:10:24.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.734+0000 7fdb01800640 1 --2- 172.21.15.83:0/3489393123 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdafc109580 0x7fdafc111150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:24.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.734+0000 7fdb01800640 1 --2- 172.21.15.83:0/3489393123 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdafc104e60 0x7fdafc108ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:24.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.734+0000 7fdb01800640 1 --2- 172.21.15.83:0/3489393123 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdafc1044b0 0x7fdafc104890 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:24.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.734+0000 7fdb01800640 1 -- 172.21.15.83:0/3489393123 >> 172.21.15.83:0/3489393123 conn(0x7fdafc0ffd90 msgr2=0x7fdafc1021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:24.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.734+0000 7fdb01800640 1 -- 172.21.15.83:0/3489393123 shutdown_connections 2024-04-24T09:10:24.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.735+0000 7fdb01800640 1 -- 172.21.15.83:0/3489393123 wait complete. 2024-04-24T09:10:24.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.736+0000 7fdb01800640 1 Processor -- start 2024-04-24T09:10:24.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.736+0000 7fdb01800640 1 -- start start 2024-04-24T09:10:24.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.736+0000 7fdb01800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdafc1044b0 0x7fdafc076510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:24.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.736+0000 7fdb01800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdafc109580 0x7fdafc076a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:24.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.736+0000 7fdb01800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdafc076f90 0x7fdafc1ab0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:24.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.736+0000 7fdb01800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdafc114510 con 0x7fdafc076f90 2024-04-24T09:10:24.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.736+0000 7fdb01800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdafc114390 con 0x7fdafc1044b0 2024-04-24T09:10:24.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.736+0000 7fdb01800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdafc114690 con 0x7fdafc109580 2024-04-24T09:10:24.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.737+0000 7fdafb400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdafc076f90 0x7fdafc1ab0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:24.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.737+0000 7fdafb400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdafc076f90 0x7fdafc1ab0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34618/0 (socket says 172.21.15.83:34618) 2024-04-24T09:10:24.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.737+0000 7fdafb400640 1 -- 172.21.15.83:0/3560290956 learned_addr learned my addr 172.21.15.83:0/3560290956 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:10:24.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.737+0000 7fdafaa00640 1 --2- 172.21.15.83:0/3560290956 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdafc1044b0 0x7fdafc076510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:24.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.737+0000 7fdafa000640 1 --2- 172.21.15.83:0/3560290956 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdafc109580 0x7fdafc076a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:24.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.737+0000 7fdafb400640 1 -- 172.21.15.83:0/3560290956 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdafc109580 msgr2=0x7fdafc076a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:24.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.737+0000 7fdafb400640 1 --2- 172.21.15.83:0/3560290956 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdafc109580 0x7fdafc076a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:24.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.737+0000 7fdafb400640 1 -- 172.21.15.83:0/3560290956 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdafc1044b0 msgr2=0x7fdafc076510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:24.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.737+0000 7fdafb400640 1 --2- 172.21.15.83:0/3560290956 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdafc1044b0 0x7fdafc076510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:24.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.737+0000 7fdafb400640 1 -- 172.21.15.83:0/3560290956 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdafc1ab620 con 0x7fdafc076f90 2024-04-24T09:10:24.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.737+0000 7fdafaa00640 1 --2- 172.21.15.83:0/3560290956 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdafc1044b0 0x7fdafc076510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:10:24.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.738+0000 7fdafb400640 1 --2- 172.21.15.83:0/3560290956 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdafc076f90 0x7fdafc1ab0e0 secure :-1 s=READY pgs=2424 cs=0 l=1 rev1=1 crypto rx=0x7fdaf400eaa0 tx=0x7fdaf400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:24.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.738+0000 7fdaf3400640 1 -- 172.21.15.83:0/3560290956 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf400cdb0 con 0x7fdafc076f90 2024-04-24T09:10:24.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.738+0000 7fdaf3400640 1 -- 172.21.15.83:0/3560290956 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdaf4005020 con 0x7fdafc076f90 2024-04-24T09:10:24.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.738+0000 7fdaf3400640 1 -- 172.21.15.83:0/3560290956 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf40116d0 con 0x7fdafc076f90 2024-04-24T09:10:24.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.738+0000 7fdb01800640 1 -- 172.21.15.83:0/3560290956 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdafc1ab910 con 0x7fdafc076f90 2024-04-24T09:10:24.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.738+0000 7fdb01800640 1 -- 172.21.15.83:0/3560290956 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdafc1abde0 con 0x7fdafc076f90 2024-04-24T09:10:24.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.743+0000 7fdb0460e740 1 Processor -- start 2024-04-24T09:10:24.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.743+0000 7fdaf3400640 1 -- 172.21.15.83:0/3560290956 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fdaf40051d0 con 0x7fdafc076f90 2024-04-24T09:10:24.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.745+0000 7fdb0460e740 1 -- start start 2024-04-24T09:10:24.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.745+0000 7fdaf3400640 1 --2- 172.21.15.83:0/3560290956 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fdae00a3ea0 0x7fdae00a6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:24.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.746+0000 7fdb0460e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563065fec950 0x563065fecd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:24.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.746+0000 7fdaf3400640 1 -- 172.21.15.83:0/3560290956 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 9583+0+0 (secure 0 0 0) 0x7fdaf40d0c30 con 0x7fdafc076f90 2024-04-24T09:10:24.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.746+0000 7fdb0460e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563065fed270 0x563065ff7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:24.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.746+0000 7fdb0460e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563065ff7940 0x563065ff9d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:24.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.746+0000 7fdb0460e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x563065f521d0 con 0x563065fed270 2024-04-24T09:10:24.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.746+0000 7fdb0460e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563065ef8450 con 0x563065ff7940 2024-04-24T09:10:24.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.746+0000 7fdb0460e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x563065f4e190 con 0x563065fec950 2024-04-24T09:10:24.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.746+0000 7fdafa000640 1 --2- 172.21.15.83:0/3560290956 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fdae00a3ea0 0x7fdae00a6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:24.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.746+0000 7fdafaa00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563065fec950 0x563065fecd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:24.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.746+0000 7fdafaa00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563065fec950 0x563065fecd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:51166/0 (socket says 172.21.15.83:51166) 2024-04-24T09:10:24.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.746+0000 7fdafaa00640 1 -- 172.21.15.83:0/2498991002 learned_addr learned my addr 172.21.15.83:0/2498991002 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:10:24.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.747+0000 7fdafa000640 1 --2- 172.21.15.83:0/2498991002 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563065fed270 0x563065ff7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:24.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.747+0000 7fdafb400640 1 --2- 172.21.15.83:0/2498991002 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563065ff7940 0x563065ff9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:24.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.747+0000 7fdafaa00640 1 -- 172.21.15.83:0/2498991002 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563065ff7940 msgr2=0x563065ff9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:24.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.747+0000 7fdafaa00640 1 --2- 172.21.15.83:0/2498991002 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563065ff7940 0x563065ff9d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:24.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.747+0000 7fdafaa00640 1 -- 172.21.15.83:0/2498991002 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563065fed270 msgr2=0x563065ff7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:24.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.747+0000 7fdafaa00640 1 --2- 172.21.15.83:0/2498991002 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563065fed270 0x563065ff7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:24.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.747+0000 7fdafaa00640 1 -- 172.21.15.83:0/2498991002 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563065fb5310 con 0x563065fec950 2024-04-24T09:10:24.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.747+0000 7fdafaa00640 1 --2- 172.21.15.83:0/2498991002 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563065fec950 0x563065fecd30 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fdaec031440 tx=0x7fdaec004210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9cc09679dcdbbd6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:24.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.748+0000 7fdafa000640 1 --2- 172.21.15.83:0/3560290956 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fdae00a3ea0 0x7fdae00a6360 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x563065fedf10 tx=0x7fdae800b7c0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:24.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.748+0000 7fdae7e00640 1 -- 172.21.15.83:0/2498991002 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaec035030 con 0x563065fec950 2024-04-24T09:10:24.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.748+0000 7fdae7e00640 1 -- 172.21.15.83:0/2498991002 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdaec0054f0 con 0x563065fec950 2024-04-24T09:10:24.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.749+0000 7fdae7e00640 1 -- 172.21.15.83:0/2498991002 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaec0035f0 con 0x563065fec950 2024-04-24T09:10:24.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.749+0000 7fdb0460e740 1 -- 172.21.15.83:0/2498991002 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563065fec950 msgr2=0x563065fecd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:24.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.749+0000 7fdb0460e740 1 --2- 172.21.15.83:0/2498991002 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563065fec950 0x563065fecd30 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fdaec031440 tx=0x7fdaec004210 comp rx=0 tx=0).stop 2024-04-24T09:10:24.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.749+0000 7fdb0460e740 1 -- 172.21.15.83:0/2498991002 shutdown_connections 2024-04-24T09:10:24.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.749+0000 7fdb0460e740 1 --2- 172.21.15.83:0/2498991002 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563065ff7940 0x563065ff9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:24.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.749+0000 7fdb0460e740 1 --2- 172.21.15.83:0/2498991002 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563065fed270 0x563065ff7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:24.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.749+0000 7fdb0460e740 1 --2- 172.21.15.83:0/2498991002 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563065fec950 0x563065fecd30 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:24.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.749+0000 7fdb0460e740 1 -- 172.21.15.83:0/2498991002 >> 172.21.15.83:0/2498991002 conn(0x563065fe8c70 msgr2=0x563065fe9050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:24.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.749+0000 7fdb0460e740 1 -- 172.21.15.83:0/2498991002 shutdown_connections 2024-04-24T09:10:24.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.749+0000 7fdb0460e740 1 -- 172.21.15.83:0/2498991002 wait complete. 2024-04-24T09:10:24.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.750+0000 7fdb0460e740 1 -- start start 2024-04-24T09:10:24.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.750+0000 7fdb0460e740 1 Processor -- start 2024-04-24T09:10:24.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.751+0000 7fdb0460e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563065fec950 0x563065ff7260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:24.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.751+0000 7fdb0460e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563065fed270 0x563065ff4550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:24.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.751+0000 7fdb0460e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563065ff7940 0x563065ff4a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:24.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.751+0000 7fdb0460e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x563065ffa7c0 con 0x563065ff7940 2024-04-24T09:10:24.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.751+0000 7fdb0460e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563065f0b810 con 0x563065fec950 2024-04-24T09:10:24.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.751+0000 7fdb0460e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x563065ffa940 con 0x563065fed270 2024-04-24T09:10:24.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.751+0000 7fdb0460e740 10 client.0 _ll_register_callbacks cb 0x563065ed9140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:10:24.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.752+0000 7fdafb400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563065ff7940 0x563065ff4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:24.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.752+0000 7fdafb400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563065ff7940 0x563065ff4a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34642/0 (socket says 172.21.15.83:34642) 2024-04-24T09:10:24.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.752+0000 7fdafb400640 1 -- 172.21.15.83:0/394828503 learned_addr learned my addr 172.21.15.83:0/394828503 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:10:24.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.752+0000 7fdafaa00640 1 --2- 172.21.15.83:0/394828503 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563065fec950 0x563065ff7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:24.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.752+0000 7fdafa000640 1 --2- 172.21.15.83:0/394828503 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563065fed270 0x563065ff4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:24.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.752+0000 7fdafb400640 1 -- 172.21.15.83:0/394828503 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563065fed270 msgr2=0x563065ff4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:24.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.752+0000 7fdafb400640 1 --2- 172.21.15.83:0/394828503 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563065fed270 0x563065ff4550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:24.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.752+0000 7fdafb400640 1 -- 172.21.15.83:0/394828503 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563065fec950 msgr2=0x563065ff7260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:24.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.752+0000 7fdafb400640 1 --2- 172.21.15.83:0/394828503 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563065fec950 0x563065ff7260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:24.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.752+0000 7fdafb400640 1 -- 172.21.15.83:0/394828503 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563065f1c3f0 con 0x563065ff7940 2024-04-24T09:10:24.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.753+0000 7fdafb400640 1 --2- 172.21.15.83:0/394828503 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563065ff7940 0x563065ff4a90 secure :-1 s=READY pgs=2425 cs=0 l=1 rev1=1 crypto rx=0x7fdaf40ce560 tx=0x7fdaf409ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80e28c1aaf305ee8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:24.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.753+0000 7fdae4c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:10:24.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.753+0000 7fdae4c00640 1 -- 172.21.15.83:0/394828503 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf401f070 con 0x563065ff7940 2024-04-24T09:10:24.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.753+0000 7fdb0460e740 1 -- 172.21.15.83:0/394828503 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdaec0054f0 con 0x563065ff7940 2024-04-24T09:10:24.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.753+0000 7fdae4c00640 1 -- 172.21.15.83:0/394828503 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(120..120 src has 1..120) ==== 9583+0+0 (secure 0 0 0) 0x7fdaf409bc00 con 0x563065ff7940 2024-04-24T09:10:24.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.753+0000 7fdae4c00640 1 -- 172.21.15.83:0/394828503 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7fdaf40a0b60 con 0x563065ff7940 2024-04-24T09:10:24.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.753+0000 7fdb0460e740 10 client.22071 fetch_fsmap learned FSMap version 61 2024-04-24T09:10:24.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.753+0000 7fdb0460e740 1 -- 172.21.15.83:0/394828503 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x563065c643f0 con 0x563065ff7940 2024-04-24T09:10:24.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.754+0000 7fdae4c00640 1 -- 172.21.15.83:0/394828503 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdaec0054f0 con 0x563065ff7940 2024-04-24T09:10:24.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.754+0000 7fdae4c00640 1 -- 172.21.15.83:0/394828503 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 61) ==== 8362+0+0 (secure 0 0 0) 0x7fdaf40b3020 con 0x563065ff7940 2024-04-24T09:10:24.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.754+0000 7fdb0460e740 10 client.22071 fetch_fsmap finished waiting for FSMap version 61 2024-04-24T09:10:24.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.754+0000 7fdb0460e740 10 client.22071 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-24T09:10:24.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.754+0000 7fdb0460e740 20 client.22071 populate_metadata read hostname 'smithi083' 2024-04-24T09:10:24.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.754+0000 7fdb0460e740 1 --2- 172.21.15.83:0/394828503 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x5630660079c0 0x563066009e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:24.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.754+0000 7fdb0460e740 4 client.22071 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:10:24.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.754+0000 7fdb0460e740 1 -- 172.21.15.83:0/394828503 --> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563065f5fbf0 con 0x5630660079c0 2024-04-24T09:10:24.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.755+0000 7fdafaa00640 1 --2- 172.21.15.83:0/394828503 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x5630660079c0 0x563066009e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:24.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.755+0000 7fdae4c00640 10 client.22071 ms_handle_connect on v2:172.21.15.189:6838/2548965336 2024-04-24T09:10:24.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.755+0000 7fdafaa00640 1 --2- 172.21.15.83:0/394828503 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x5630660079c0 0x563066009e10 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=58a3e98f7fe75380 server_cookie=141c2600b795d592 in_seq=0 out_seq=0 2024-04-24T09:10:24.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.757+0000 7fdae4c00640 1 -- 172.21.15.83:0/394828503 <== mds.4 v2:172.21.15.189:6838/2548965336 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563065f5fbf0 con 0x5630660079c0 2024-04-24T09:10:24.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.757+0000 7fdae4c00640 10 client.22071 handle_command_reply: tid=0 2024-04-24T09:10:24.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.757+0000 7fdb0460e740 1 client.22071 shutdown 2024-04-24T09:10:24.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.758+0000 7fdb0460e740 1 -- 172.21.15.83:0/394828503 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563065ff7940 msgr2=0x563065ff4a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:24.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.758+0000 7fdb0460e740 1 --2- 172.21.15.83:0/394828503 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563065ff7940 0x563065ff4a90 secure :-1 s=READY pgs=2425 cs=0 l=1 rev1=1 crypto rx=0x7fdaf40ce560 tx=0x7fdaf409ceb0 comp rx=0 tx=0).stop 2024-04-24T09:10:24.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.758+0000 7fdb0460e740 1 -- 172.21.15.83:0/394828503 shutdown_connections 2024-04-24T09:10:24.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.758+0000 7fdb0460e740 1 --2- 172.21.15.83:0/394828503 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x5630660079c0 0x563066009e10 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:24.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.758+0000 7fdb0460e740 1 --2- 172.21.15.83:0/394828503 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563065ff7940 0x563065ff4a90 unknown :-1 s=CLOSED pgs=2425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:24.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.758+0000 7fdb0460e740 1 --2- 172.21.15.83:0/394828503 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563065fed270 0x563065ff4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:24.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.758+0000 7fdae4c00640 0 client.22071 ms_handle_reset on v2:172.21.15.189:6838/2548965336 2024-04-24T09:10:24.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.758+0000 7fdb0460e740 1 --2- 172.21.15.83:0/394828503 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563065fec950 0x563065ff7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:24.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.758+0000 7fdb0460e740 1 -- 172.21.15.83:0/394828503 >> 172.21.15.83:0/394828503 conn(0x563065fe8c70 msgr2=0x563065feea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:24.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.758+0000 7fdb0460e740 1 -- 172.21.15.83:0/394828503 shutdown_connections 2024-04-24T09:10:24.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.758+0000 7fdb0460e740 1 -- 172.21.15.83:0/394828503 wait complete. 2024-04-24T09:10:24.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.758+0000 7fdb0460e740 20 client.22071 trim_cache size 0 max 16384 2024-04-24T09:10:24.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.782+0000 7fdb0460e740 1 Processor -- start 2024-04-24T09:10:24.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.782+0000 7fdb0460e740 1 -- start start 2024-04-24T09:10:24.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.782+0000 7fdb0460e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563065fec950 0x5630660036d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:24.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.782+0000 7fdb0460e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563065fed270 0x563065ff6d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:24.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.782+0000 7fdb0460e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563065ff7940 0x56306600f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:24.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.782+0000 7fdb0460e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdafc1b5c50 con 0x563065fed270 2024-04-24T09:10:24.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.782+0000 7fdb0460e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdafc1b7790 con 0x563065fec950 2024-04-24T09:10:24.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.782+0000 7fdb0460e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdafc1b6320 con 0x563065ff7940 2024-04-24T09:10:24.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.783+0000 7fdafaa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563065fec950 0x5630660036d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:24.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.783+0000 7fdafaa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563065fec950 0x5630660036d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:41768/0 (socket says 172.21.15.83:41768) 2024-04-24T09:10:24.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.783+0000 7fdafaa00640 1 -- 172.21.15.83:0/3623805028 learned_addr learned my addr 172.21.15.83:0/3623805028 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:10:24.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.783+0000 7fdafa000640 1 --2- 172.21.15.83:0/3623805028 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563065fed270 0x563065ff6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:24.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.783+0000 7fdafb400640 1 --2- 172.21.15.83:0/3623805028 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563065ff7940 0x56306600f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:24.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.783+0000 7fdafaa00640 1 -- 172.21.15.83:0/3623805028 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563065ff7940 msgr2=0x56306600f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:24.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.783+0000 7fdafaa00640 1 --2- 172.21.15.83:0/3623805028 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563065ff7940 0x56306600f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:24.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.783+0000 7fdafaa00640 1 -- 172.21.15.83:0/3623805028 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563065fed270 msgr2=0x563065ff6d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:24.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.783+0000 7fdafaa00640 1 --2- 172.21.15.83:0/3623805028 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563065fed270 0x563065ff6d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:24.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.784+0000 7fdafaa00640 1 -- 172.21.15.83:0/3623805028 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563065d2e7d0 con 0x563065fec950 2024-04-24T09:10:24.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.784+0000 7fdafb400640 1 --2- 172.21.15.83:0/3623805028 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563065ff7940 0x56306600f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:10:24.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.784+0000 7fdafa000640 1 --2- 172.21.15.83:0/3623805028 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563065fed270 0x563065ff6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:10:24.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.784+0000 7fdafaa00640 1 --2- 172.21.15.83:0/3623805028 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563065fec950 0x5630660036d0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fdaec005dc0 tx=0x7fdaec003780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f837ee245f2f3ba server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:24.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.784+0000 7fdae7400640 1 -- 172.21.15.83:0/3623805028 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaec035040 con 0x563065fec950 2024-04-24T09:10:24.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.785+0000 7fdae7400640 1 -- 172.21.15.83:0/3623805028 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdaec005050 con 0x563065fec950 2024-04-24T09:10:24.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.785+0000 7fdae7400640 1 -- 172.21.15.83:0/3623805028 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaec004980 con 0x563065fec950 2024-04-24T09:10:24.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.786+0000 7fdb0460e740 1 -- 172.21.15.83:0/3623805028 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563065fec950 msgr2=0x5630660036d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:24.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.786+0000 7fdb0460e740 1 --2- 172.21.15.83:0/3623805028 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563065fec950 0x5630660036d0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fdaec005dc0 tx=0x7fdaec003780 comp rx=0 tx=0).stop 2024-04-24T09:10:24.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.786+0000 7fdb0460e740 1 -- 172.21.15.83:0/3623805028 shutdown_connections 2024-04-24T09:10:24.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.786+0000 7fdb0460e740 1 --2- 172.21.15.83:0/3623805028 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563065ff7940 0x56306600f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:24.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.786+0000 7fdb0460e740 1 --2- 172.21.15.83:0/3623805028 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563065fed270 0x563065ff6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:24.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.786+0000 7fdb0460e740 1 --2- 172.21.15.83:0/3623805028 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563065fec950 0x5630660036d0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:24.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.786+0000 7fdb0460e740 1 -- 172.21.15.83:0/3623805028 >> 172.21.15.83:0/3623805028 conn(0x563065fe8c70 msgr2=0x563065ff4510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:24.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.786+0000 7fdb0460e740 1 -- 172.21.15.83:0/3623805028 shutdown_connections 2024-04-24T09:10:24.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.786+0000 7fdb0460e740 1 -- 172.21.15.83:0/3623805028 wait complete. 2024-04-24T09:10:24.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.787+0000 7fdb0460e740 1 -- start start 2024-04-24T09:10:24.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.787+0000 7fdb0460e740 1 Processor -- start 2024-04-24T09:10:24.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.787+0000 7fdb0460e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563065fec950 0x563065fffbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:24.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.788+0000 7fdb0460e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563065fed270 0x563066000120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:24.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.788+0000 7fdb0460e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563065ff7940 0x563065ff0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:24.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.788+0000 7fdb0460e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdafc1b70c0 con 0x563065ff7940 2024-04-24T09:10:24.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.788+0000 7fdb0460e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdafc1b69f0 con 0x563065fed270 2024-04-24T09:10:24.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.788+0000 7fdb0460e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdafc1b3c50 con 0x563065fec950 2024-04-24T09:10:24.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.788+0000 7fdb0460e740 10 client.0 _ll_register_callbacks cb 0x7fdaf4009270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:10:24.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.788+0000 7fdafb400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563065ff7940 0x563065ff0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:24.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.788+0000 7fdafaa00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563065fec950 0x563065fffbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:24.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.788+0000 7fdafb400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563065ff7940 0x563065ff0990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34654/0 (socket says 172.21.15.83:34654) 2024-04-24T09:10:24.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.788+0000 7fdafaa00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563065fec950 0x563065fffbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:51192/0 (socket says 172.21.15.83:51192) 2024-04-24T09:10:24.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.788+0000 7fdafb400640 1 -- 172.21.15.83:0/949752272 learned_addr learned my addr 172.21.15.83:0/949752272 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:10:24.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.788+0000 7fdafa000640 1 --2- 172.21.15.83:0/949752272 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563065fed270 0x563066000120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:24.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.789+0000 7fdafaa00640 1 -- 172.21.15.83:0/949752272 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563065fed270 msgr2=0x563066000120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:24.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.789+0000 7fdafaa00640 1 --2- 172.21.15.83:0/949752272 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563065fed270 0x563066000120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:24.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.789+0000 7fdafaa00640 1 -- 172.21.15.83:0/949752272 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563065ff7940 msgr2=0x563065ff0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:24.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.789+0000 7fdafaa00640 1 --2- 172.21.15.83:0/949752272 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563065ff7940 0x563065ff0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:24.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.789+0000 7fdafaa00640 1 -- 172.21.15.83:0/949752272 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563065ff11c0 con 0x563065fec950 2024-04-24T09:10:24.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.789+0000 7fdafb400640 1 --2- 172.21.15.83:0/949752272 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563065ff7940 0x563065ff0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:10:24.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.789+0000 7fdafaa00640 1 --2- 172.21.15.83:0/949752272 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563065fec950 0x563065fffbe0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fdaec02a9e0 tx=0x7fdaec003680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=599b72e5f2bc4aff server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:24.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.789+0000 7fdae4c00640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-04-24T09:10:24.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.789+0000 7fdae4c00640 1 -- 172.21.15.83:0/949752272 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaec035040 con 0x563065fec950 2024-04-24T09:10:24.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.789+0000 7fdb0460e740 1 -- 172.21.15.83:0/949752272 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdaec005050 con 0x563065fec950 2024-04-24T09:10:24.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.789+0000 7fdae4c00640 1 -- 172.21.15.83:0/949752272 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(120..120 src has 1..120) ==== 9583+0+0 (secure 0 0 0) 0x7fdaec0038f0 con 0x563065fec950 2024-04-24T09:10:24.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.790+0000 7fdae4c00640 1 -- 172.21.15.83:0/949752272 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7fdaec005610 con 0x563065fec950 2024-04-24T09:10:24.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.790+0000 7fdb0460e740 10 client.29125 fetch_fsmap learned FSMap version 61 2024-04-24T09:10:24.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.790+0000 7fdb0460e740 1 -- 172.21.15.83:0/949752272 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x563065ff24c0 con 0x563065fec950 2024-04-24T09:10:24.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.790+0000 7fdae4c00640 1 -- 172.21.15.83:0/949752272 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdaec005050 con 0x563065fec950 2024-04-24T09:10:24.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.790+0000 7fdae4c00640 1 -- 172.21.15.83:0/949752272 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 61) ==== 8362+0+0 (secure 0 0 0) 0x7fdaec004d10 con 0x563065fec950 2024-04-24T09:10:24.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.790+0000 7fdb0460e740 10 client.29125 fetch_fsmap finished waiting for FSMap version 61 2024-04-24T09:10:24.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.790+0000 7fdb0460e740 10 client.29125 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-24T09:10:24.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.790+0000 7fdb0460e740 20 client.29125 populate_metadata read hostname 'smithi083' 2024-04-24T09:10:24.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.790+0000 7fdb0460e740 1 --2- 172.21.15.83:0/949752272 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x56306606ef60 0x563066071350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:24.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.790+0000 7fdb0460e740 4 client.29125 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:10:24.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.791+0000 7fdb0460e740 1 -- 172.21.15.83:0/949752272 --> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563065f59a60 con 0x56306606ef60 2024-04-24T09:10:24.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.791+0000 7fdafa000640 1 --2- 172.21.15.83:0/949752272 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x56306606ef60 0x563066071350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:24.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.792+0000 7fdafa000640 1 --2- 172.21.15.83:0/949752272 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x56306606ef60 0x563066071350 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=176bcf77705478b8 server_cookie=ffa9b1253cb73db9 in_seq=0 out_seq=0 2024-04-24T09:10:24.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.792+0000 7fdae4c00640 10 client.29125 ms_handle_connect on v2:172.21.15.189:6838/2548965336 2024-04-24T09:10:24.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.792+0000 7fdae4c00640 1 -- 172.21.15.83:0/949752272 <== mds.4 v2:172.21.15.189:6838/2548965336 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563065f59a60 con 0x56306606ef60 2024-04-24T09:10:24.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.792+0000 7fdae4c00640 10 client.29125 handle_command_reply: tid=0 2024-04-24T09:10:24.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.792+0000 7fdb0460e740 1 client.29125 shutdown 2024-04-24T09:10:24.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.792+0000 7fdb0460e740 1 -- 172.21.15.83:0/949752272 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563065fec950 msgr2=0x563065fffbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:24.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.792+0000 7fdb0460e740 1 --2- 172.21.15.83:0/949752272 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563065fec950 0x563065fffbe0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fdaec02a9e0 tx=0x7fdaec003680 comp rx=0 tx=0).stop 2024-04-24T09:10:24.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.792+0000 7fdb0460e740 1 -- 172.21.15.83:0/949752272 shutdown_connections 2024-04-24T09:10:24.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.792+0000 7fdb0460e740 1 --2- 172.21.15.83:0/949752272 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x56306606ef60 0x563066071350 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:24.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.792+0000 7fdb0460e740 1 --2- 172.21.15.83:0/949752272 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563065ff7940 0x563065ff0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:24.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.792+0000 7fdb0460e740 1 --2- 172.21.15.83:0/949752272 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563065fed270 0x563066000120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:24.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.792+0000 7fdb0460e740 1 --2- 172.21.15.83:0/949752272 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563065fec950 0x563065fffbe0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:24.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.792+0000 7fdae4c00640 0 client.29125 ms_handle_reset on v2:172.21.15.189:6838/2548965336 2024-04-24T09:10:24.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.792+0000 7fdb0460e740 1 -- 172.21.15.83:0/949752272 >> 172.21.15.83:0/949752272 conn(0x563065fe8c70 msgr2=0x563065fe7560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:24.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.793+0000 7fdb0460e740 1 -- 172.21.15.83:0/949752272 shutdown_connections 2024-04-24T09:10:24.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.793+0000 7fdb0460e740 1 -- 172.21.15.83:0/949752272 wait complete. 2024-04-24T09:10:24.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.793+0000 7fdb0460e740 20 client.29125 trim_cache size 0 max 16384 2024-04-24T09:10:24.795 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:10:24.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.794+0000 7fdb01800640 1 -- 172.21.15.83:0/3560290956 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fdae00a3ea0 msgr2=0x7fdae00a6360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:24.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.794+0000 7fdb01800640 1 --2- 172.21.15.83:0/3560290956 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fdae00a3ea0 0x7fdae00a6360 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x563065fedf10 tx=0x7fdae800b7c0 comp rx=0 tx=0).stop 2024-04-24T09:10:24.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.794+0000 7fdb01800640 1 -- 172.21.15.83:0/3560290956 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdafc076f90 msgr2=0x7fdafc1ab0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:24.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.794+0000 7fdb01800640 1 --2- 172.21.15.83:0/3560290956 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdafc076f90 0x7fdafc1ab0e0 secure :-1 s=READY pgs=2424 cs=0 l=1 rev1=1 crypto rx=0x7fdaf400eaa0 tx=0x7fdaf400ef60 comp rx=0 tx=0).stop 2024-04-24T09:10:24.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.794+0000 7fdb01800640 1 -- 172.21.15.83:0/3560290956 shutdown_connections 2024-04-24T09:10:24.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.794+0000 7fdb01800640 1 --2- 172.21.15.83:0/3560290956 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fdae00a3ea0 0x7fdae00a6360 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:24.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.794+0000 7fdb01800640 1 --2- 172.21.15.83:0/3560290956 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdafc076f90 0x7fdafc1ab0e0 unknown :-1 s=CLOSED pgs=2424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:24.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.794+0000 7fdb01800640 1 --2- 172.21.15.83:0/3560290956 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdafc109580 0x7fdafc076a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:24.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.794+0000 7fdb01800640 1 --2- 172.21.15.83:0/3560290956 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdafc1044b0 0x7fdafc076510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:24.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.794+0000 7fdb01800640 1 -- 172.21.15.83:0/3560290956 >> 172.21.15.83:0/3560290956 conn(0x7fdafc0ffd90 msgr2=0x7fdafc1011c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:24.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.795+0000 7fdb01800640 1 -- 172.21.15.83:0/3560290956 shutdown_connections 2024-04-24T09:10:24.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:24.795+0000 7fdb01800640 1 -- 172.21.15.83:0/3560290956 wait complete. 2024-04-24T09:10:24.818 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-24T09:10:25.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.058+0000 7f2f1b200640 1 -- 172.21.15.83:0/914873138 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f14103060 msgr2=0x7f2f14103440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:25.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.058+0000 7f2f1b200640 1 --2- 172.21.15.83:0/914873138 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f14103060 0x7f2f14103440 secure :-1 s=READY pgs=2426 cs=0 l=1 rev1=1 crypto rx=0x7f2f08009900 tx=0x7f2f0802a480 comp rx=0 tx=0).stop 2024-04-24T09:10:25.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.058+0000 7f2f1b200640 1 -- 172.21.15.83:0/914873138 shutdown_connections 2024-04-24T09:10:25.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.058+0000 7f2f1b200640 1 --2- 172.21.15.83:0/914873138 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f1410c480 0x7f2f1410e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.058+0000 7f2f1b200640 1 --2- 172.21.15.83:0/914873138 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2f14103980 0x7f2f1410be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.058+0000 7f2f1b200640 1 --2- 172.21.15.83:0/914873138 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f14103060 0x7f2f14103440 unknown :-1 s=CLOSED pgs=2426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.058+0000 7f2f1b200640 1 -- 172.21.15.83:0/914873138 >> 172.21.15.83:0/914873138 conn(0x7f2f140fc880 msgr2=0x7f2f140feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:25.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.058+0000 7f2f1b200640 1 -- 172.21.15.83:0/914873138 shutdown_connections 2024-04-24T09:10:25.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.058+0000 7f2f1b200640 1 -- 172.21.15.83:0/914873138 wait complete. 2024-04-24T09:10:25.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.059+0000 7f2f1b200640 1 Processor -- start 2024-04-24T09:10:25.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.059+0000 7f2f1b200640 1 -- start start 2024-04-24T09:10:25.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.059+0000 7f2f1b200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2f14103060 0x7f2f141a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:25.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.059+0000 7f2f1b200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f14103980 0x7f2f141a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:25.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.059+0000 7f2f1b200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f1410c480 0x7f2f1419a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:25.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.059+0000 7f2f1b200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2f14112140 con 0x7f2f1410c480 2024-04-24T09:10:25.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.059+0000 7f2f1b200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2f14111fc0 con 0x7f2f14103980 2024-04-24T09:10:25.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.059+0000 7f2f1b200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2f141122c0 con 0x7f2f14103060 2024-04-24T09:10:25.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.060+0000 7f2f1a800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f1410c480 0x7f2f1419a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:25.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.060+0000 7f2f1a800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f1410c480 0x7f2f1419a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34678/0 (socket says 172.21.15.83:34678) 2024-04-24T09:10:25.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.060+0000 7f2f1a800640 1 -- 172.21.15.83:0/3066702186 learned_addr learned my addr 172.21.15.83:0/3066702186 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:10:25.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.060+0000 7f2f19e00640 1 --2- 172.21.15.83:0/3066702186 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2f14103060 0x7f2f141a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:25.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.060+0000 7f2f19400640 1 --2- 172.21.15.83:0/3066702186 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f14103980 0x7f2f141a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:25.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.060+0000 7f2f1a800640 1 -- 172.21.15.83:0/3066702186 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2f14103060 msgr2=0x7f2f141a0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:25.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.060+0000 7f2f1a800640 1 --2- 172.21.15.83:0/3066702186 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2f14103060 0x7f2f141a0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.060+0000 7f2f1a800640 1 -- 172.21.15.83:0/3066702186 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f14103980 msgr2=0x7f2f141a0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:25.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.060+0000 7f2f1a800640 1 --2- 172.21.15.83:0/3066702186 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f14103980 0x7f2f141a0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.060+0000 7f2f1a800640 1 -- 172.21.15.83:0/3066702186 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f1419afe0 con 0x7f2f1410c480 2024-04-24T09:10:25.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.060+0000 7f2f19400640 1 --2- 172.21.15.83:0/3066702186 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f14103980 0x7f2f141a0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:10:25.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.061+0000 7f2f1a800640 1 --2- 172.21.15.83:0/3066702186 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f1410c480 0x7f2f1419a720 secure :-1 s=READY pgs=2427 cs=0 l=1 rev1=1 crypto rx=0x7f2f1000eaa0 tx=0x7f2f1000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:25.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.061+0000 7f2f0ea00640 1 -- 172.21.15.83:0/3066702186 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f1000cdb0 con 0x7f2f1410c480 2024-04-24T09:10:25.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.061+0000 7f2f0ea00640 1 -- 172.21.15.83:0/3066702186 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2f10005020 con 0x7f2f1410c480 2024-04-24T09:10:25.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.061+0000 7f2f1b200640 1 -- 172.21.15.83:0/3066702186 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f1419b2d0 con 0x7f2f1410c480 2024-04-24T09:10:25.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.061+0000 7f2f0ea00640 1 -- 172.21.15.83:0/3066702186 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f100116d0 con 0x7f2f1410c480 2024-04-24T09:10:25.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.061+0000 7f2f1b200640 1 -- 172.21.15.83:0/3066702186 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f141a7590 con 0x7f2f1410c480 2024-04-24T09:10:25.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.064+0000 7f2f1f613740 1 Processor -- start 2024-04-24T09:10:25.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.064+0000 7f2f0ea00640 1 -- 172.21.15.83:0/3066702186 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2f100051d0 con 0x7f2f1410c480 2024-04-24T09:10:25.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.064+0000 7f2f1f613740 1 -- start start 2024-04-24T09:10:25.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.065+0000 7f2f1f613740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c190230b40 0x55c190230f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:25.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.065+0000 7f2f1f613740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c190231460 0x55c19023b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:25.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.065+0000 7f2f1f613740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c19023bb30 0x55c19023df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:25.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.065+0000 7f2f1f613740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c1901961d0 con 0x55c190230b40 2024-04-24T09:10:25.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.065+0000 7f2f1f613740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c19013c450 con 0x55c19023bb30 2024-04-24T09:10:25.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.065+0000 7f2f1f613740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55c190192190 con 0x55c190231460 2024-04-24T09:10:25.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.065+0000 7f2f19400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c190230b40 0x55c190230f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:25.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.065+0000 7f2f19400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c190230b40 0x55c190230f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34690/0 (socket says 172.21.15.83:34690) 2024-04-24T09:10:25.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.065+0000 7f2f19400640 1 -- 172.21.15.83:0/2068401332 learned_addr learned my addr 172.21.15.83:0/2068401332 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:10:25.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.065+0000 7f2f19e00640 1 --2- 172.21.15.83:0/2068401332 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c190231460 0x55c19023b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:25.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.065+0000 7f2f19400640 1 --2- 172.21.15.83:0/2068401332 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c19023bb30 0x55c19023df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:25.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.066+0000 7f2f19400640 1 -- 172.21.15.83:0/2068401332 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c190231460 msgr2=0x55c19023b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:25.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.066+0000 7f2f19400640 1 --2- 172.21.15.83:0/2068401332 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c190231460 0x55c19023b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.066+0000 7f2f19400640 1 -- 172.21.15.83:0/2068401332 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c19023bb30 msgr2=0x55c19023df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:25.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.066+0000 7f2f19400640 1 --2- 172.21.15.83:0/2068401332 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c19023bb30 0x55c19023df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.066+0000 7f2f0ea00640 1 --2- 172.21.15.83:0/3066702186 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f2efc0a3d20 0x7f2efc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:25.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.066+0000 7f2f19400640 1 -- 172.21.15.83:0/2068401332 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1901f9310 con 0x55c190230b40 2024-04-24T09:10:25.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.066+0000 7f2f19e00640 1 --2- 172.21.15.83:0/2068401332 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c190231460 0x55c19023b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:10:25.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.066+0000 7f2f0ea00640 1 -- 172.21.15.83:0/3066702186 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 9583+0+0 (secure 0 0 0) 0x7f2f100d0eb0 con 0x7f2f1410c480 2024-04-24T09:10:25.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.066+0000 7f2f1a800640 1 --2- 172.21.15.83:0/3066702186 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f2efc0a3d20 0x7f2efc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:25.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.066+0000 7f2f19400640 1 --2- 172.21.15.83:0/2068401332 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c190230b40 0x55c190230f20 secure :-1 s=READY pgs=2428 cs=0 l=1 rev1=1 crypto rx=0x7f2f04015650 tx=0x7f2f04009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9180b1f6ea7b45e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:25.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.066+0000 7f2f03000640 1 -- 172.21.15.83:0/2068401332 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f040150f0 con 0x55c190230b40 2024-04-24T09:10:25.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.066+0000 7f2f03000640 1 -- 172.21.15.83:0/2068401332 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2f04005020 con 0x55c190230b40 2024-04-24T09:10:25.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.066+0000 7f2f03000640 1 -- 172.21.15.83:0/2068401332 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f04004a00 con 0x55c190230b40 2024-04-24T09:10:25.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.067+0000 7f2f1f613740 1 -- 172.21.15.83:0/2068401332 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c190230b40 msgr2=0x55c190230f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:25.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.067+0000 7f2f1f613740 1 --2- 172.21.15.83:0/2068401332 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c190230b40 0x55c190230f20 secure :-1 s=READY pgs=2428 cs=0 l=1 rev1=1 crypto rx=0x7f2f04015650 tx=0x7f2f04009680 comp rx=0 tx=0).stop 2024-04-24T09:10:25.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.067+0000 7f2f1a800640 1 --2- 172.21.15.83:0/3066702186 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f2efc0a3d20 0x7f2efc0a61e0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f2f100993a0 tx=0x7f2f10005830 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:25.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.067+0000 7f2f1f613740 1 -- 172.21.15.83:0/2068401332 shutdown_connections 2024-04-24T09:10:25.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.067+0000 7f2f1f613740 1 --2- 172.21.15.83:0/2068401332 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c19023bb30 0x55c19023df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.067+0000 7f2f1f613740 1 --2- 172.21.15.83:0/2068401332 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c190231460 0x55c19023b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.067+0000 7f2f1f613740 1 --2- 172.21.15.83:0/2068401332 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c190230b40 0x55c190230f20 unknown :-1 s=CLOSED pgs=2428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.067+0000 7f2f1f613740 1 -- 172.21.15.83:0/2068401332 >> 172.21.15.83:0/2068401332 conn(0x55c19022ce60 msgr2=0x55c19022d240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:25.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.067+0000 7f2f1f613740 1 -- 172.21.15.83:0/2068401332 shutdown_connections 2024-04-24T09:10:25.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.067+0000 7f2f1f613740 1 -- 172.21.15.83:0/2068401332 wait complete. 2024-04-24T09:10:25.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.068+0000 7f2f1f613740 1 -- start start 2024-04-24T09:10:25.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.068+0000 7f2f1f613740 1 Processor -- start 2024-04-24T09:10:25.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.068+0000 7f2f1f613740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c190230b40 0x55c190238950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:25.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.068+0000 7f2f1f613740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c190231460 0x55c190238e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:25.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.068+0000 7f2f1f613740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c19023bb30 0x55c1902393d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:25.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.068+0000 7f2f1f613740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c19023e9c0 con 0x55c190231460 2024-04-24T09:10:25.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.068+0000 7f2f1f613740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c19014f810 con 0x55c19023bb30 2024-04-24T09:10:25.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.068+0000 7f2f1f613740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55c19023eb40 con 0x55c190230b40 2024-04-24T09:10:25.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.068+0000 7f2f1f613740 10 client.0 _ll_register_callbacks cb 0x55c19011d140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:10:25.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.069+0000 7f2f19e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c190231460 0x55c190238e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:25.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.069+0000 7f2f19e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c190231460 0x55c190238e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34702/0 (socket says 172.21.15.83:34702) 2024-04-24T09:10:25.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.069+0000 7f2f19e00640 1 -- 172.21.15.83:0/1203484334 learned_addr learned my addr 172.21.15.83:0/1203484334 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:10:25.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.069+0000 7f2f19400640 1 --2- 172.21.15.83:0/1203484334 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c190230b40 0x55c190238950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:25.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.069+0000 7f2f19400640 1 --2- 172.21.15.83:0/1203484334 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c19023bb30 0x55c1902393d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:25.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.069+0000 7f2f19e00640 1 -- 172.21.15.83:0/1203484334 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c190230b40 msgr2=0x55c190238950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:25.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.069+0000 7f2f19e00640 1 --2- 172.21.15.83:0/1203484334 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c190230b40 0x55c190238950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.069+0000 7f2f19e00640 1 -- 172.21.15.83:0/1203484334 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c19023bb30 msgr2=0x55c1902393d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:25.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.069+0000 7f2f19e00640 1 --2- 172.21.15.83:0/1203484334 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c19023bb30 0x55c1902393d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.069+0000 7f2f19e00640 1 -- 172.21.15.83:0/1203484334 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1901603f0 con 0x55c190231460 2024-04-24T09:10:25.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.069+0000 7f2f19400640 1 --2- 172.21.15.83:0/1203484334 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c19023bb30 0x55c1902393d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:10:25.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.069+0000 7f2f19e00640 1 --2- 172.21.15.83:0/1203484334 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c190231460 0x55c190238e90 secure :-1 s=READY pgs=2429 cs=0 l=1 rev1=1 crypto rx=0x7f2f08009230 tx=0x7f2f080052f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2955d9f6e192783 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:25.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.069+0000 7f2ef7400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:10:25.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.070+0000 7f2ef7400640 1 -- 172.21.15.83:0/1203484334 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f08035030 con 0x55c190231460 2024-04-24T09:10:25.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.070+0000 7f2ef7400640 1 -- 172.21.15.83:0/1203484334 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(120..120 src has 1..120) ==== 9583+0+0 (secure 0 0 0) 0x7f2f080048c0 con 0x55c190231460 2024-04-24T09:10:25.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.070+0000 7f2f1f613740 1 -- 172.21.15.83:0/1203484334 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f04005020 con 0x55c190231460 2024-04-24T09:10:25.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.070+0000 7f2ef7400640 1 -- 172.21.15.83:0/1203484334 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f2f08005d50 con 0x55c190231460 2024-04-24T09:10:25.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.070+0000 7f2f1f613740 10 client.22089 fetch_fsmap learned FSMap version 61 2024-04-24T09:10:25.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.070+0000 7f2f1f613740 1 -- 172.21.15.83:0/1203484334 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x55c18fea83f0 con 0x55c190231460 2024-04-24T09:10:25.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.070+0000 7f2ef7400640 1 -- 172.21.15.83:0/1203484334 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2f04005020 con 0x55c190231460 2024-04-24T09:10:25.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.070+0000 7f2ef7400640 1 -- 172.21.15.83:0/1203484334 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 61) ==== 8362+0+0 (secure 0 0 0) 0x7f2f080314a0 con 0x55c190231460 2024-04-24T09:10:25.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.070+0000 7f2f1f613740 10 client.22089 fetch_fsmap finished waiting for FSMap version 61 2024-04-24T09:10:25.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.070+0000 7f2f1f613740 10 client.22089 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-24T09:10:25.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.070+0000 7f2f1f613740 20 client.22089 populate_metadata read hostname 'smithi083' 2024-04-24T09:10:25.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.070+0000 7f2f1f613740 1 --2- 172.21.15.83:0/1203484334 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55c19024bde0 0x55c19024e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:25.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.071+0000 7f2f1f613740 4 client.22089 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:10:25.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.071+0000 7f2f1f613740 1 -- 172.21.15.83:0/1203484334 --> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c1901a3bf0 con 0x55c19024bde0 2024-04-24T09:10:25.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.071+0000 7f2f19400640 1 --2- 172.21.15.83:0/1203484334 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55c19024bde0 0x55c19024e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:25.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.071+0000 7f2f19400640 1 --2- 172.21.15.83:0/1203484334 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55c19024bde0 0x55c19024e1d0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8d4c21714d211bca server_cookie=58890aafb8199c2 in_seq=0 out_seq=0 2024-04-24T09:10:25.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.071+0000 7f2ef7400640 10 client.22089 ms_handle_connect on v2:172.21.15.138:6832/1157967658 2024-04-24T09:10:25.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.073+0000 7f2ef7400640 1 -- 172.21.15.83:0/1203484334 <== mds.3 v2:172.21.15.138:6832/1157967658 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c1901a3bf0 con 0x55c19024bde0 2024-04-24T09:10:25.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.073+0000 7f2ef7400640 10 client.22089 handle_command_reply: tid=0 2024-04-24T09:10:25.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.073+0000 7f2f1f613740 1 client.22089 shutdown 2024-04-24T09:10:25.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.073+0000 7f2f1f613740 1 -- 172.21.15.83:0/1203484334 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c190231460 msgr2=0x55c190238e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:25.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.073+0000 7f2f1f613740 1 --2- 172.21.15.83:0/1203484334 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c190231460 0x55c190238e90 secure :-1 s=READY pgs=2429 cs=0 l=1 rev1=1 crypto rx=0x7f2f08009230 tx=0x7f2f080052f0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.073+0000 7f2f1f613740 1 -- 172.21.15.83:0/1203484334 shutdown_connections 2024-04-24T09:10:25.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.073+0000 7f2f1f613740 1 --2- 172.21.15.83:0/1203484334 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c19023bb30 0x55c1902393d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.073+0000 7f2f1f613740 1 --2- 172.21.15.83:0/1203484334 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c190231460 0x55c190238e90 unknown :-1 s=CLOSED pgs=2429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.073+0000 7f2f1f613740 1 --2- 172.21.15.83:0/1203484334 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55c19024bde0 0x55c19024e1d0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.073+0000 7f2f1f613740 1 --2- 172.21.15.83:0/1203484334 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c190230b40 0x55c190238950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.073+0000 7f2f1f613740 1 -- 172.21.15.83:0/1203484334 >> 172.21.15.83:0/1203484334 conn(0x55c19022ce60 msgr2=0x55c190232c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:25.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.073+0000 7f2ef7400640 0 client.22089 ms_handle_reset on v2:172.21.15.138:6832/1157967658 2024-04-24T09:10:25.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.073+0000 7f2f1f613740 1 -- 172.21.15.83:0/1203484334 shutdown_connections 2024-04-24T09:10:25.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.073+0000 7f2f1f613740 1 -- 172.21.15.83:0/1203484334 wait complete. 2024-04-24T09:10:25.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.073+0000 7f2f1f613740 20 client.22089 trim_cache size 0 max 16384 2024-04-24T09:10:25.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.087+0000 7f2f1f613740 1 Processor -- start 2024-04-24T09:10:25.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.087+0000 7f2f1f613740 1 -- start start 2024-04-24T09:10:25.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.087+0000 7f2f1f613740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c190230b40 0x55c190247880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:25.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.087+0000 7f2f1f613740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c190231460 0x55c1902332a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:25.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.087+0000 7f2f1f613740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c19023bb30 0x55c19022bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:25.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.087+0000 7f2f1f613740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2f141b15a0 con 0x55c190231460 2024-04-24T09:10:25.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.087+0000 7f2f1f613740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2f141b30e0 con 0x55c190230b40 2024-04-24T09:10:25.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.087+0000 7f2f1f613740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2f141b1c70 con 0x55c19023bb30 2024-04-24T09:10:25.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.088+0000 7f2f19e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c190231460 0x55c1902332a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:25.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.088+0000 7f2f19e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c190231460 0x55c1902332a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34712/0 (socket says 172.21.15.83:34712) 2024-04-24T09:10:25.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.088+0000 7f2f19e00640 1 -- 172.21.15.83:0/795149795 learned_addr learned my addr 172.21.15.83:0/795149795 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:10:25.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.088+0000 7f2f19400640 1 --2- 172.21.15.83:0/795149795 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c190230b40 0x55c190247880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:25.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.088+0000 7f2f19400640 1 --2- 172.21.15.83:0/795149795 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c19023bb30 0x55c19022bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:25.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.088+0000 7f2f19e00640 1 -- 172.21.15.83:0/795149795 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c19023bb30 msgr2=0x55c19022bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:25.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.088+0000 7f2f19e00640 1 --2- 172.21.15.83:0/795149795 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c19023bb30 0x55c19022bdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.088+0000 7f2f19e00640 1 -- 172.21.15.83:0/795149795 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c190230b40 msgr2=0x55c190247880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:25.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.088+0000 7f2f19e00640 1 --2- 172.21.15.83:0/795149795 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c190230b40 0x55c190247880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.088+0000 7f2f19e00640 1 -- 172.21.15.83:0/795149795 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c18ff727d0 con 0x55c190231460 2024-04-24T09:10:25.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.088+0000 7f2f19e00640 1 --2- 172.21.15.83:0/795149795 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c190231460 0x55c1902332a0 secure :-1 s=READY pgs=2430 cs=0 l=1 rev1=1 crypto rx=0x7f2f08031440 tx=0x7f2f08005250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7e70c42241300ad server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:25.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.088+0000 7f2f02600640 1 -- 172.21.15.83:0/795149795 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f08055020 con 0x55c190231460 2024-04-24T09:10:25.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.089+0000 7f2f02600640 1 -- 172.21.15.83:0/795149795 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2f0803f070 con 0x55c190231460 2024-04-24T09:10:25.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.089+0000 7f2f02600640 1 -- 172.21.15.83:0/795149795 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f08035070 con 0x55c190231460 2024-04-24T09:10:25.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.089+0000 7f2f1f613740 1 -- 172.21.15.83:0/795149795 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c190231460 msgr2=0x55c1902332a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:25.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.089+0000 7f2f1f613740 1 --2- 172.21.15.83:0/795149795 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c190231460 0x55c1902332a0 secure :-1 s=READY pgs=2430 cs=0 l=1 rev1=1 crypto rx=0x7f2f08031440 tx=0x7f2f08005250 comp rx=0 tx=0).stop 2024-04-24T09:10:25.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.089+0000 7f2f1f613740 1 -- 172.21.15.83:0/795149795 shutdown_connections 2024-04-24T09:10:25.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.089+0000 7f2f1f613740 1 --2- 172.21.15.83:0/795149795 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c19023bb30 0x55c19022bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.089+0000 7f2f1f613740 1 --2- 172.21.15.83:0/795149795 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c190231460 0x55c1902332a0 unknown :-1 s=CLOSED pgs=2430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.089+0000 7f2f1f613740 1 --2- 172.21.15.83:0/795149795 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c190230b40 0x55c190247880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.089+0000 7f2f1f613740 1 -- 172.21.15.83:0/795149795 >> 172.21.15.83:0/795149795 conn(0x55c19022ce60 msgr2=0x55c19022b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:25.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.089+0000 7f2f1f613740 1 -- 172.21.15.83:0/795149795 shutdown_connections 2024-04-24T09:10:25.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.089+0000 7f2f1f613740 1 -- 172.21.15.83:0/795149795 wait complete. 2024-04-24T09:10:25.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.090+0000 7f2f1f613740 1 -- start start 2024-04-24T09:10:25.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.090+0000 7f2f1f613740 1 Processor -- start 2024-04-24T09:10:25.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.090+0000 7f2f1f613740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c190230b40 0x55c19023c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:25.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.090+0000 7f2f1f613740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c190231460 0x55c19023cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:25.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.090+0000 7f2f1f613740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c19023bb30 0x55c190242d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:25.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.090+0000 7f2f1f613740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2f141b2a10 con 0x55c190230b40 2024-04-24T09:10:25.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.090+0000 7f2f1f613740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2f141b2340 con 0x55c190231460 2024-04-24T09:10:25.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.090+0000 7f2f1f613740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2f141af5a0 con 0x55c19023bb30 2024-04-24T09:10:25.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.090+0000 7f2f1f613740 10 client.0 _ll_register_callbacks cb 0x7f2f08009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:10:25.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.090+0000 7f2f19400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c190230b40 0x55c19023c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:25.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.090+0000 7f2f19400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c190230b40 0x55c19023c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34714/0 (socket says 172.21.15.83:34714) 2024-04-24T09:10:25.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.090+0000 7f2f19400640 1 -- 172.21.15.83:0/1000330333 learned_addr learned my addr 172.21.15.83:0/1000330333 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:10:25.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.091+0000 7f2f19e00640 1 --2- 172.21.15.83:0/1000330333 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c190231460 0x55c19023cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:25.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.091+0000 7f2f19400640 1 -- 172.21.15.83:0/1000330333 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c19023bb30 msgr2=0x55c190242d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:25.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.091+0000 7f2f19400640 1 --2- 172.21.15.83:0/1000330333 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c19023bb30 0x55c190242d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.091+0000 7f2f19400640 1 -- 172.21.15.83:0/1000330333 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c190231460 msgr2=0x55c19023cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:25.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.091+0000 7f2f19400640 1 --2- 172.21.15.83:0/1000330333 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c190231460 0x55c19023cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.091+0000 7f2f19400640 1 -- 172.21.15.83:0/1000330333 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c190243380 con 0x55c190230b40 2024-04-24T09:10:25.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.091+0000 7f2f19400640 1 --2- 172.21.15.83:0/1000330333 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c190230b40 0x55c19023c700 secure :-1 s=READY pgs=2431 cs=0 l=1 rev1=1 crypto rx=0x7f2f040045e0 tx=0x7f2f04015050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba4b4ac7610a8073 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:25.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.091+0000 7f2ef7400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:10:25.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.091+0000 7f2ef7400640 1 -- 172.21.15.83:0/1000330333 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f04015270 con 0x55c190230b40 2024-04-24T09:10:25.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.091+0000 7f2ef7400640 1 -- 172.21.15.83:0/1000330333 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(120..120 src has 1..120) ==== 9583+0+0 (secure 0 0 0) 0x7f2f040104d0 con 0x55c190230b40 2024-04-24T09:10:25.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.091+0000 7f2f1f613740 1 -- 172.21.15.83:0/1000330333 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f0803f070 con 0x55c190230b40 2024-04-24T09:10:25.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.092+0000 7f2ef7400640 1 -- 172.21.15.83:0/1000330333 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f2f040106d0 con 0x55c190230b40 2024-04-24T09:10:25.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.092+0000 7f2f1f613740 10 client.22095 fetch_fsmap learned FSMap version 61 2024-04-24T09:10:25.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.092+0000 7f2f1f613740 1 -- 172.21.15.83:0/1000330333 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x55c1902b2330 con 0x55c190230b40 2024-04-24T09:10:25.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.092+0000 7f2ef7400640 1 -- 172.21.15.83:0/1000330333 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2f0803f070 con 0x55c190230b40 2024-04-24T09:10:25.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.092+0000 7f2ef7400640 1 -- 172.21.15.83:0/1000330333 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 61) ==== 8362+0+0 (secure 0 0 0) 0x7f2f04021040 con 0x55c190230b40 2024-04-24T09:10:25.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.092+0000 7f2f1f613740 10 client.22095 fetch_fsmap finished waiting for FSMap version 61 2024-04-24T09:10:25.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.092+0000 7f2f1f613740 10 client.22095 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-24T09:10:25.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.092+0000 7f2f1f613740 20 client.22095 populate_metadata read hostname 'smithi083' 2024-04-24T09:10:25.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.092+0000 7f2f1f613740 1 --2- 172.21.15.83:0/1000330333 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55c1902b2800 0x55c1902b4c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:25.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.092+0000 7f2f1f613740 4 client.22095 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:10:25.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.092+0000 7f2f1f613740 1 -- 172.21.15.83:0/1000330333 --> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c19019da60 con 0x55c1902b2800 2024-04-24T09:10:25.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.093+0000 7f2f19e00640 1 --2- 172.21.15.83:0/1000330333 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55c1902b2800 0x55c1902b4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:25.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.093+0000 7f2f19e00640 1 --2- 172.21.15.83:0/1000330333 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55c1902b2800 0x55c1902b4c20 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f9590baaa4b9b256 server_cookie=70bcb894a903144f in_seq=0 out_seq=0 2024-04-24T09:10:25.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.093+0000 7f2ef7400640 10 client.22095 ms_handle_connect on v2:172.21.15.138:6832/1157967658 2024-04-24T09:10:25.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.093+0000 7f2ef7400640 1 -- 172.21.15.83:0/1000330333 <== mds.3 v2:172.21.15.138:6832/1157967658 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c19019da60 con 0x55c1902b2800 2024-04-24T09:10:25.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.094+0000 7f2ef7400640 10 client.22095 handle_command_reply: tid=0 2024-04-24T09:10:25.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.094+0000 7f2f1f613740 1 client.22095 shutdown 2024-04-24T09:10:25.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.094+0000 7f2f1f613740 1 -- 172.21.15.83:0/1000330333 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c190230b40 msgr2=0x55c19023c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:25.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.094+0000 7f2f1f613740 1 --2- 172.21.15.83:0/1000330333 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c190230b40 0x55c19023c700 secure :-1 s=READY pgs=2431 cs=0 l=1 rev1=1 crypto rx=0x7f2f040045e0 tx=0x7f2f04015050 comp rx=0 tx=0).stop 2024-04-24T09:10:25.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.094+0000 7f2f1f613740 1 -- 172.21.15.83:0/1000330333 shutdown_connections 2024-04-24T09:10:25.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.094+0000 7f2f1f613740 1 --2- 172.21.15.83:0/1000330333 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55c1902b2800 0x55c1902b4c20 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.094+0000 7f2f1f613740 1 --2- 172.21.15.83:0/1000330333 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c19023bb30 0x55c190242d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.094+0000 7f2f1f613740 1 --2- 172.21.15.83:0/1000330333 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c190231460 0x55c19023cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.094+0000 7f2f1f613740 1 --2- 172.21.15.83:0/1000330333 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c190230b40 0x55c19023c700 unknown :-1 s=CLOSED pgs=2431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.094+0000 7f2f1f613740 1 -- 172.21.15.83:0/1000330333 >> 172.21.15.83:0/1000330333 conn(0x55c19022ce60 msgr2=0x55c19022bf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:25.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.094+0000 7f2ef7400640 0 client.22095 ms_handle_reset on v2:172.21.15.138:6832/1157967658 2024-04-24T09:10:25.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.094+0000 7f2f1f613740 1 -- 172.21.15.83:0/1000330333 shutdown_connections 2024-04-24T09:10:25.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.094+0000 7f2f1f613740 1 -- 172.21.15.83:0/1000330333 wait complete. 2024-04-24T09:10:25.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.094+0000 7f2f1f613740 20 client.22095 trim_cache size 0 max 16384 2024-04-24T09:10:25.099 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:10:25.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.095+0000 7f2f1b200640 1 -- 172.21.15.83:0/3066702186 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f2efc0a3d20 msgr2=0x7f2efc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:25.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.095+0000 7f2f1b200640 1 --2- 172.21.15.83:0/3066702186 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f2efc0a3d20 0x7f2efc0a61e0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f2f100993a0 tx=0x7f2f10005830 comp rx=0 tx=0).stop 2024-04-24T09:10:25.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.095+0000 7f2f1b200640 1 -- 172.21.15.83:0/3066702186 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f1410c480 msgr2=0x7f2f1419a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:25.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.095+0000 7f2f1b200640 1 --2- 172.21.15.83:0/3066702186 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f1410c480 0x7f2f1419a720 secure :-1 s=READY pgs=2427 cs=0 l=1 rev1=1 crypto rx=0x7f2f1000eaa0 tx=0x7f2f1000ef60 comp rx=0 tx=0).stop 2024-04-24T09:10:25.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.095+0000 7f2f1b200640 1 -- 172.21.15.83:0/3066702186 shutdown_connections 2024-04-24T09:10:25.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.095+0000 7f2f1b200640 1 --2- 172.21.15.83:0/3066702186 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f2efc0a3d20 0x7f2efc0a61e0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.095+0000 7f2f1b200640 1 --2- 172.21.15.83:0/3066702186 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f1410c480 0x7f2f1419a720 unknown :-1 s=CLOSED pgs=2427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.095+0000 7f2f1b200640 1 --2- 172.21.15.83:0/3066702186 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f14103980 0x7f2f141a0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.095+0000 7f2f1b200640 1 --2- 172.21.15.83:0/3066702186 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2f14103060 0x7f2f141a0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.095+0000 7f2f1b200640 1 -- 172.21.15.83:0/3066702186 >> 172.21.15.83:0/3066702186 conn(0x7f2f140fc880 msgr2=0x7f2f14100c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:25.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.095+0000 7f2f1b200640 1 -- 172.21.15.83:0/3066702186 shutdown_connections 2024-04-24T09:10:25.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.095+0000 7f2f1b200640 1 -- 172.21.15.83:0/3066702186 wait complete. 2024-04-24T09:10:25.112 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T09:10:25.175 INFO:tasks.workunit.client.0.smithi083.stdout: 10 37532 6.72 MB/sec execute 176 sec latency 230.806 ms 2024-04-24T09:10:25.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.290+0000 7f7ec2800640 1 -- 172.21.15.83:0/407673844 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ebc103100 msgr2=0x7f7ebc1034e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:25.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.290+0000 7f7ec2800640 1 --2- 172.21.15.83:0/407673844 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ebc103100 0x7f7ebc1034e0 secure :-1 s=READY pgs=2432 cs=0 l=1 rev1=1 crypto rx=0x7f7eb0009900 tx=0x7f7eb002a480 comp rx=0 tx=0).stop 2024-04-24T09:10:25.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.290+0000 7f7ec2800640 1 -- 172.21.15.83:0/407673844 shutdown_connections 2024-04-24T09:10:25.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.290+0000 7f7ec2800640 1 --2- 172.21.15.83:0/407673844 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ebc10c4d0 0x7f7ebc10e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.290+0000 7f7ec2800640 1 --2- 172.21.15.83:0/407673844 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7ebc103a20 0x7f7ebc10bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.290+0000 7f7ec2800640 1 --2- 172.21.15.83:0/407673844 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ebc103100 0x7f7ebc1034e0 unknown :-1 s=CLOSED pgs=2432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.290+0000 7f7ec2800640 1 -- 172.21.15.83:0/407673844 >> 172.21.15.83:0/407673844 conn(0x7f7ebc0fc880 msgr2=0x7f7ebc0feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:25.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.291+0000 7f7ec2800640 1 -- 172.21.15.83:0/407673844 shutdown_connections 2024-04-24T09:10:25.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.291+0000 7f7ec2800640 1 -- 172.21.15.83:0/407673844 wait complete. 2024-04-24T09:10:25.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.292+0000 7f7ec2800640 1 Processor -- start 2024-04-24T09:10:25.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.292+0000 7f7ec2800640 1 -- start start 2024-04-24T09:10:25.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.293+0000 7f7ec2800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ebc103100 0x7f7ebc100d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:25.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.293+0000 7f7ec2800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ebc103a20 0x7f7ebc101290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:25.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.293+0000 7f7ec2800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7ebc10c4d0 0x7f7ebc1017d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:25.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.293+0000 7f7ec2800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7ebc112230 con 0x7f7ebc103a20 2024-04-24T09:10:25.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.293+0000 7f7ec2800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7ebc1120b0 con 0x7f7ebc103100 2024-04-24T09:10:25.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.293+0000 7f7ec2800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7ebc1123b0 con 0x7f7ebc10c4d0 2024-04-24T09:10:25.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.293+0000 7f7ec0a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ebc103a20 0x7f7ebc101290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:25.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.293+0000 7f7ec0a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ebc103a20 0x7f7ebc101290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34738/0 (socket says 172.21.15.83:34738) 2024-04-24T09:10:25.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.293+0000 7f7ec0a00640 1 -- 172.21.15.83:0/1578179448 learned_addr learned my addr 172.21.15.83:0/1578179448 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:10:25.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.293+0000 7f7ec1400640 1 --2- 172.21.15.83:0/1578179448 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ebc103100 0x7f7ebc100d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:25.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.293+0000 7f7ec0a00640 1 -- 172.21.15.83:0/1578179448 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7ebc10c4d0 msgr2=0x7f7ebc1017d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:25.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.293+0000 7f7ec0a00640 1 --2- 172.21.15.83:0/1578179448 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7ebc10c4d0 0x7f7ebc1017d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.293+0000 7f7ec0a00640 1 -- 172.21.15.83:0/1578179448 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ebc103100 msgr2=0x7f7ebc100d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:25.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.293+0000 7f7ec0a00640 1 --2- 172.21.15.83:0/1578179448 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ebc103100 0x7f7ebc100d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.293+0000 7f7ec0a00640 1 -- 172.21.15.83:0/1578179448 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ebc1a6cd0 con 0x7f7ebc103a20 2024-04-24T09:10:25.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.294+0000 7f7ec0a00640 1 --2- 172.21.15.83:0/1578179448 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ebc103a20 0x7f7ebc101290 secure :-1 s=READY pgs=2433 cs=0 l=1 rev1=1 crypto rx=0x7f7ea800b4a0 tx=0x7f7ea800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:25.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.294+0000 7f7eba000640 1 -- 172.21.15.83:0/1578179448 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea8014070 con 0x7f7ebc103a20 2024-04-24T09:10:25.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.294+0000 7f7eba000640 1 -- 172.21.15.83:0/1578179448 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7ea8010ac0 con 0x7f7ebc103a20 2024-04-24T09:10:25.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.294+0000 7f7ec2800640 1 -- 172.21.15.83:0/1578179448 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ebc1a6fc0 con 0x7f7ebc103a20 2024-04-24T09:10:25.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.294+0000 7f7eba000640 1 -- 172.21.15.83:0/1578179448 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea8017c30 con 0x7f7ebc103a20 2024-04-24T09:10:25.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.294+0000 7f7ec2800640 1 -- 172.21.15.83:0/1578179448 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ebc1a74d0 con 0x7f7ebc103a20 2024-04-24T09:10:25.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.297+0000 7f7ec6c27740 1 Processor -- start 2024-04-24T09:10:25.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.297+0000 7f7ec6c27740 1 -- start start 2024-04-24T09:10:25.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.297+0000 7f7eba000640 1 -- 172.21.15.83:0/1578179448 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7ea8017070 con 0x7f7ebc103a20 2024-04-24T09:10:25.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.298+0000 7f7ec6c27740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e215303b40 0x55e215303f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:25.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.298+0000 7f7ec6c27740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e215304460 0x55e21530e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:25.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.298+0000 7f7ec6c27740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e21530eb30 0x55e215310f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:25.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.298+0000 7f7ec6c27740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e2152691d0 con 0x55e215304460 2024-04-24T09:10:25.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.298+0000 7f7ec6c27740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e21520f450 con 0x55e21530eb30 2024-04-24T09:10:25.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.298+0000 7f7ec6c27740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55e215265190 con 0x55e215303b40 2024-04-24T09:10:25.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.298+0000 7f7ec1400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e215304460 0x55e21530e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:25.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.298+0000 7f7ec1400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e215304460 0x55e21530e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34748/0 (socket says 172.21.15.83:34748) 2024-04-24T09:10:25.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.298+0000 7f7ec1400640 1 -- 172.21.15.83:0/2300680512 learned_addr learned my addr 172.21.15.83:0/2300680512 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:10:25.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.298+0000 7f7ec1e00640 1 --2- 172.21.15.83:0/2300680512 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e215303b40 0x55e215303f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:25.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.298+0000 7f7ec1400640 1 -- 172.21.15.83:0/2300680512 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e215303b40 msgr2=0x55e215303f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:25.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.298+0000 7f7ec1400640 1 --2- 172.21.15.83:0/2300680512 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e215303b40 0x55e215303f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.298+0000 7f7ec1400640 1 -- 172.21.15.83:0/2300680512 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e21530eb30 msgr2=0x55e215310f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:25.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.298+0000 7f7ec1400640 1 --2- 172.21.15.83:0/2300680512 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e21530eb30 0x55e215310f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.298+0000 7f7ec1400640 1 -- 172.21.15.83:0/2300680512 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2152cc310 con 0x55e215304460 2024-04-24T09:10:25.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.298+0000 7f7ec1e00640 1 --2- 172.21.15.83:0/2300680512 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e215303b40 0x55e215303f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:10:25.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.299+0000 7f7ec1400640 1 --2- 172.21.15.83:0/2300680512 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e215304460 0x55e21530e5f0 secure :-1 s=READY pgs=2434 cs=0 l=1 rev1=1 crypto rx=0x7f7eb0031440 tx=0x7f7eb0004390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e9c79ef8ba2876f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:25.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.299+0000 7f7eae600640 1 -- 172.21.15.83:0/2300680512 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eb003f070 con 0x55e215304460 2024-04-24T09:10:25.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.299+0000 7f7eae600640 1 -- 172.21.15.83:0/2300680512 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7eb00047a0 con 0x55e215304460 2024-04-24T09:10:25.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.299+0000 7f7eae600640 1 -- 172.21.15.83:0/2300680512 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eb0035070 con 0x55e215304460 2024-04-24T09:10:25.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.300+0000 7f7eba000640 1 --2- 172.21.15.83:0/1578179448 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f7ea40a3da0 0x7f7ea40a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:25.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.300+0000 7f7ec6c27740 1 -- 172.21.15.83:0/2300680512 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e215304460 msgr2=0x55e21530e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:25.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.300+0000 7f7ec6c27740 1 --2- 172.21.15.83:0/2300680512 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e215304460 0x55e21530e5f0 secure :-1 s=READY pgs=2434 cs=0 l=1 rev1=1 crypto rx=0x7f7eb0031440 tx=0x7f7eb0004390 comp rx=0 tx=0).stop 2024-04-24T09:10:25.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.300+0000 7f7eba000640 1 -- 172.21.15.83:0/1578179448 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 9583+0+0 (secure 0 0 0) 0x7f7ea80d0250 con 0x7f7ebc103a20 2024-04-24T09:10:25.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.300+0000 7f7ec1e00640 1 --2- 172.21.15.83:0/1578179448 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f7ea40a3da0 0x7f7ea40a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:25.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.300+0000 7f7ec6c27740 1 -- 172.21.15.83:0/2300680512 shutdown_connections 2024-04-24T09:10:25.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.300+0000 7f7ec6c27740 1 --2- 172.21.15.83:0/2300680512 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e21530eb30 0x55e215310f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.300+0000 7f7ec6c27740 1 --2- 172.21.15.83:0/2300680512 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e215304460 0x55e21530e5f0 unknown :-1 s=CLOSED pgs=2434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.300+0000 7f7ec6c27740 1 --2- 172.21.15.83:0/2300680512 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e215303b40 0x55e215303f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.300+0000 7f7ec6c27740 1 -- 172.21.15.83:0/2300680512 >> 172.21.15.83:0/2300680512 conn(0x55e2152ffe60 msgr2=0x55e215300240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:25.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.300+0000 7f7ec6c27740 1 -- 172.21.15.83:0/2300680512 shutdown_connections 2024-04-24T09:10:25.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.300+0000 7f7ec6c27740 1 -- 172.21.15.83:0/2300680512 wait complete. 2024-04-24T09:10:25.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.301+0000 7f7ec1e00640 1 --2- 172.21.15.83:0/1578179448 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f7ea40a3da0 0x7f7ea40a6260 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f7eb4007d10 tx=0x7f7eb4007340 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:25.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.301+0000 7f7ec6c27740 1 -- start start 2024-04-24T09:10:25.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.301+0000 7f7ec6c27740 1 Processor -- start 2024-04-24T09:10:25.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.301+0000 7f7ec6c27740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e215303b40 0x55e21530e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:25.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.301+0000 7f7ec6c27740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e215304460 0x55e21530b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:25.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.301+0000 7f7ec6c27740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e21530eb30 0x55e21530c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:25.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.301+0000 7f7ec6c27740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e215311960 con 0x55e215303b40 2024-04-24T09:10:25.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.301+0000 7f7ec6c27740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e215222810 con 0x55e21530eb30 2024-04-24T09:10:25.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.301+0000 7f7ec6c27740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55e215311ae0 con 0x55e215304460 2024-04-24T09:10:25.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.301+0000 7f7ec6c27740 10 client.0 _ll_register_callbacks cb 0x55e2151f0140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:10:25.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.302+0000 7f7ec1400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e215303b40 0x55e21530e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:25.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.302+0000 7f7ec1400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e215303b40 0x55e21530e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34762/0 (socket says 172.21.15.83:34762) 2024-04-24T09:10:25.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.302+0000 7f7ec1400640 1 -- 172.21.15.83:0/4207177100 learned_addr learned my addr 172.21.15.83:0/4207177100 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:10:25.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.302+0000 7f7ec1400640 1 -- 172.21.15.83:0/4207177100 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e215304460 msgr2=0x55e21530b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:25.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.302+0000 7f7ec1400640 1 --2- 172.21.15.83:0/4207177100 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e215304460 0x55e21530b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.302+0000 7f7ec1e00640 1 --2- 172.21.15.83:0/4207177100 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e21530eb30 0x55e21530c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:25.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.302+0000 7f7ec1400640 1 -- 172.21.15.83:0/4207177100 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e21530eb30 msgr2=0x55e21530c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:25.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.302+0000 7f7ec1400640 1 --2- 172.21.15.83:0/4207177100 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e21530eb30 0x55e21530c020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.302+0000 7f7ec1400640 1 -- 172.21.15.83:0/4207177100 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2152333f0 con 0x55e215303b40 2024-04-24T09:10:25.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.302+0000 7f7ec1400640 1 --2- 172.21.15.83:0/4207177100 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e215303b40 0x55e21530e5f0 secure :-1 s=READY pgs=2435 cs=0 l=1 rev1=1 crypto rx=0x7f7eb002aa70 tx=0x7f7eb00052d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b63ab5bab805cd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:25.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.302+0000 7f7ea2a00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:10:25.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.303+0000 7f7ea2a00640 1 -- 172.21.15.83:0/4207177100 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eb004c060 con 0x55e215303b40 2024-04-24T09:10:25.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.303+0000 7f7ea2a00640 1 -- 172.21.15.83:0/4207177100 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(120..120 src has 1..120) ==== 9583+0+0 (secure 0 0 0) 0x7f7eb00316a0 con 0x55e215303b40 2024-04-24T09:10:25.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.303+0000 7f7ec6c27740 1 -- 172.21.15.83:0/4207177100 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7eb00047a0 con 0x55e215303b40 2024-04-24T09:10:25.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.303+0000 7f7ea2a00640 1 -- 172.21.15.83:0/4207177100 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f7eb0053050 con 0x55e215303b40 2024-04-24T09:10:25.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.303+0000 7f7ec6c27740 10 client.22107 fetch_fsmap learned FSMap version 61 2024-04-24T09:10:25.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.303+0000 7f7ec6c27740 1 -- 172.21.15.83:0/4207177100 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x55e214f7b3f0 con 0x55e215303b40 2024-04-24T09:10:25.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.303+0000 7f7ea2a00640 1 -- 172.21.15.83:0/4207177100 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7eb00047a0 con 0x55e215303b40 2024-04-24T09:10:25.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.303+0000 7f7ea2a00640 1 -- 172.21.15.83:0/4207177100 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 61) ==== 8362+0+0 (secure 0 0 0) 0x7f7eb003b930 con 0x55e215303b40 2024-04-24T09:10:25.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.303+0000 7f7ec6c27740 10 client.22107 fetch_fsmap finished waiting for FSMap version 61 2024-04-24T09:10:25.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.303+0000 7f7ec6c27740 10 client.22107 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-24T09:10:25.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.303+0000 7f7ec6c27740 20 client.22107 populate_metadata read hostname 'smithi083' 2024-04-24T09:10:25.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.304+0000 7f7ec6c27740 1 --2- 172.21.15.83:0/4207177100 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55e21531eb30 0x55e215320f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:25.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.304+0000 7f7ec6c27740 4 client.22107 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:10:25.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.304+0000 7f7ec6c27740 1 -- 172.21.15.83:0/4207177100 --> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e215276bf0 con 0x55e21531eb30 2024-04-24T09:10:25.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.304+0000 7f7ec0a00640 1 --2- 172.21.15.83:0/4207177100 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55e21531eb30 0x55e215320f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:25.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.305+0000 7f7ec0a00640 1 --2- 172.21.15.83:0/4207177100 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55e21531eb30 0x55e215320f20 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=360d7ce086d10aa server_cookie=a523aa1a7c5d71be in_seq=0 out_seq=0 2024-04-24T09:10:25.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.305+0000 7f7ea2a00640 10 client.22107 ms_handle_connect on v2:172.21.15.138:6834/1909219758 2024-04-24T09:10:25.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.307+0000 7f7ea2a00640 1 -- 172.21.15.83:0/4207177100 <== mds.2 v2:172.21.15.138:6834/1909219758 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e215276bf0 con 0x55e21531eb30 2024-04-24T09:10:25.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.307+0000 7f7ea2a00640 10 client.22107 handle_command_reply: tid=0 2024-04-24T09:10:25.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.307+0000 7f7ec6c27740 1 client.22107 shutdown 2024-04-24T09:10:25.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.307+0000 7f7ec6c27740 1 -- 172.21.15.83:0/4207177100 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e215303b40 msgr2=0x55e21530e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:25.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.307+0000 7f7ec6c27740 1 --2- 172.21.15.83:0/4207177100 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e215303b40 0x55e21530e5f0 secure :-1 s=READY pgs=2435 cs=0 l=1 rev1=1 crypto rx=0x7f7eb002aa70 tx=0x7f7eb00052d0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.307+0000 7f7ec6c27740 1 -- 172.21.15.83:0/4207177100 shutdown_connections 2024-04-24T09:10:25.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.307+0000 7f7ec6c27740 1 --2- 172.21.15.83:0/4207177100 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55e21531eb30 0x55e215320f20 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.307+0000 7f7ec6c27740 1 --2- 172.21.15.83:0/4207177100 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e21530eb30 0x55e21530c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.307+0000 7f7ec6c27740 1 --2- 172.21.15.83:0/4207177100 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e215304460 0x55e21530b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.307+0000 7f7ea2a00640 0 client.22107 ms_handle_reset on v2:172.21.15.138:6834/1909219758 2024-04-24T09:10:25.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.307+0000 7f7ec6c27740 1 --2- 172.21.15.83:0/4207177100 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e215303b40 0x55e21530e5f0 unknown :-1 s=CLOSED pgs=2435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.307+0000 7f7ec6c27740 1 -- 172.21.15.83:0/4207177100 >> 172.21.15.83:0/4207177100 conn(0x55e2152ffe60 msgr2=0x55e215305b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:25.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.307+0000 7f7ec6c27740 1 -- 172.21.15.83:0/4207177100 shutdown_connections 2024-04-24T09:10:25.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.307+0000 7f7ec6c27740 1 -- 172.21.15.83:0/4207177100 wait complete. 2024-04-24T09:10:25.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.307+0000 7f7ec6c27740 20 client.22107 trim_cache size 0 max 16384 2024-04-24T09:10:25.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.321+0000 7f7ec6c27740 1 Processor -- start 2024-04-24T09:10:25.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.321+0000 7f7ec6c27740 1 -- start start 2024-04-24T09:10:25.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.322+0000 7f7ec6c27740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e215303b40 0x55e2153062c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:25.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.322+0000 7f7ec6c27740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e215304460 0x55e21530e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:25.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.322+0000 7f7ec6c27740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e21530eb30 0x55e215308b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:25.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.322+0000 7f7ec6c27740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7ebc1b1730 con 0x55e215304460 2024-04-24T09:10:25.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.322+0000 7f7ec6c27740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7ebc1b3270 con 0x55e21530eb30 2024-04-24T09:10:25.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.322+0000 7f7ec6c27740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7ebc1b1e00 con 0x55e215303b40 2024-04-24T09:10:25.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.322+0000 7f7ec0a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e215304460 0x55e21530e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:25.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.322+0000 7f7ec0a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e215304460 0x55e21530e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34764/0 (socket says 172.21.15.83:34764) 2024-04-24T09:10:25.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.322+0000 7f7ec0a00640 1 -- 172.21.15.83:0/3957781643 learned_addr learned my addr 172.21.15.83:0/3957781643 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:10:25.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.322+0000 7f7ec1400640 1 --2- 172.21.15.83:0/3957781643 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e215303b40 0x55e2153062c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:25.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.322+0000 7f7ec1e00640 1 --2- 172.21.15.83:0/3957781643 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e21530eb30 0x55e215308b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:25.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.322+0000 7f7ec0a00640 1 -- 172.21.15.83:0/3957781643 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e215303b40 msgr2=0x55e2153062c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:25.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.322+0000 7f7ec0a00640 1 --2- 172.21.15.83:0/3957781643 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e215303b40 0x55e2153062c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.322+0000 7f7ec0a00640 1 -- 172.21.15.83:0/3957781643 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e21530eb30 msgr2=0x55e215308b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:25.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.322+0000 7f7ec0a00640 1 --2- 172.21.15.83:0/3957781643 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e21530eb30 0x55e215308b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.322+0000 7f7ec0a00640 1 -- 172.21.15.83:0/3957781643 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e215304290 con 0x55e215304460 2024-04-24T09:10:25.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.323+0000 7f7ec0a00640 1 --2- 172.21.15.83:0/3957781643 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e215304460 0x55e21530e120 secure :-1 s=READY pgs=2436 cs=0 l=1 rev1=1 crypto rx=0x7f7ea809be10 tx=0x7f7ea80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8102cfd53b2c6272 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:25.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.323+0000 7f7eadc00640 1 -- 172.21.15.83:0/3957781643 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea80b5070 con 0x55e215304460 2024-04-24T09:10:25.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.323+0000 7f7eadc00640 1 -- 172.21.15.83:0/3957781643 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7ea80a4af0 con 0x55e215304460 2024-04-24T09:10:25.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.323+0000 7f7eadc00640 1 -- 172.21.15.83:0/3957781643 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea801e040 con 0x55e215304460 2024-04-24T09:10:25.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.324+0000 7f7ec6c27740 1 -- 172.21.15.83:0/3957781643 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e215304460 msgr2=0x55e21530e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:25.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.324+0000 7f7ec6c27740 1 --2- 172.21.15.83:0/3957781643 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e215304460 0x55e21530e120 secure :-1 s=READY pgs=2436 cs=0 l=1 rev1=1 crypto rx=0x7f7ea809be10 tx=0x7f7ea80046c0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.324+0000 7f7ec6c27740 1 -- 172.21.15.83:0/3957781643 shutdown_connections 2024-04-24T09:10:25.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.324+0000 7f7ec6c27740 1 --2- 172.21.15.83:0/3957781643 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e21530eb30 0x55e215308b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.324+0000 7f7ec6c27740 1 --2- 172.21.15.83:0/3957781643 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e215304460 0x55e21530e120 unknown :-1 s=CLOSED pgs=2436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.324+0000 7f7ec6c27740 1 --2- 172.21.15.83:0/3957781643 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e215303b40 0x55e2153062c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.324+0000 7f7ec6c27740 1 -- 172.21.15.83:0/3957781643 >> 172.21.15.83:0/3957781643 conn(0x55e2152ffe60 msgr2=0x55e215300240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:25.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.324+0000 7f7ec6c27740 1 -- 172.21.15.83:0/3957781643 shutdown_connections 2024-04-24T09:10:25.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.324+0000 7f7ec6c27740 1 -- 172.21.15.83:0/3957781643 wait complete. 2024-04-24T09:10:25.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.324+0000 7f7ec6c27740 1 -- start start 2024-04-24T09:10:25.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.324+0000 7f7ec6c27740 1 Processor -- start 2024-04-24T09:10:25.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.325+0000 7f7ec6c27740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e215303b40 0x55e2153086e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:25.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.325+0000 7f7ec6c27740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e215304460 0x55e215308c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:25.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.325+0000 7f7ec6c27740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e21530eb30 0x55e215309160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:25.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.325+0000 7f7ec6c27740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7ebc1b2ba0 con 0x55e215304460 2024-04-24T09:10:25.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.325+0000 7f7ec6c27740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7ebc1b24d0 con 0x55e215303b40 2024-04-24T09:10:25.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.325+0000 7f7ec6c27740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7ebc1af730 con 0x55e21530eb30 2024-04-24T09:10:25.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.325+0000 7f7ec6c27740 10 client.0 _ll_register_callbacks cb 0x7f7eb0004260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:10:25.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.325+0000 7f7ec0a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e215304460 0x55e215308c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:25.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.325+0000 7f7ec0a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e215304460 0x55e215308c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34772/0 (socket says 172.21.15.83:34772) 2024-04-24T09:10:25.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.325+0000 7f7ec0a00640 1 -- 172.21.15.83:0/579363194 learned_addr learned my addr 172.21.15.83:0/579363194 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:10:25.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.325+0000 7f7ec1400640 1 --2- 172.21.15.83:0/579363194 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e215303b40 0x55e2153086e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:25.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.325+0000 7f7ec0a00640 1 -- 172.21.15.83:0/579363194 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e21530eb30 msgr2=0x55e215309160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:25.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.325+0000 7f7ec1e00640 1 --2- 172.21.15.83:0/579363194 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e21530eb30 0x55e215309160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:25.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.326+0000 7f7ec0a00640 1 --2- 172.21.15.83:0/579363194 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e21530eb30 0x55e215309160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.326+0000 7f7ec0a00640 1 -- 172.21.15.83:0/579363194 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e215303b40 msgr2=0x55e2153086e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:25.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.326+0000 7f7ec0a00640 1 --2- 172.21.15.83:0/579363194 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e215303b40 0x55e2153086e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.326+0000 7f7ec0a00640 1 -- 172.21.15.83:0/579363194 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2150457d0 con 0x55e215304460 2024-04-24T09:10:25.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.326+0000 7f7ec1400640 1 --2- 172.21.15.83:0/579363194 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e215303b40 0x55e2153086e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:10:25.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.326+0000 7f7ec0a00640 1 --2- 172.21.15.83:0/579363194 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e215304460 0x55e215308c20 secure :-1 s=READY pgs=2437 cs=0 l=1 rev1=1 crypto rx=0x7f7ea8017590 tx=0x7f7ea80d0150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5883443cc1de5206 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:25.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.326+0000 7f7ea2a00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:10:25.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.326+0000 7f7ea2a00640 1 -- 172.21.15.83:0/579363194 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea80ba070 con 0x55e215304460 2024-04-24T09:10:25.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.326+0000 7f7ea2a00640 1 -- 172.21.15.83:0/579363194 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(120..120 src has 1..120) ==== 9583+0+0 (secure 0 0 0) 0x7f7ea8010c70 con 0x55e215304460 2024-04-24T09:10:25.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.326+0000 7f7ec6c27740 1 -- 172.21.15.83:0/579363194 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ea80a4af0 con 0x55e215304460 2024-04-24T09:10:25.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.326+0000 7f7ea2a00640 1 -- 172.21.15.83:0/579363194 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f7ea80b5040 con 0x55e215304460 2024-04-24T09:10:25.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.326+0000 7f7ec6c27740 10 client.22113 fetch_fsmap learned FSMap version 61 2024-04-24T09:10:25.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.326+0000 7f7ec6c27740 1 -- 172.21.15.83:0/579363194 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x55e215384f70 con 0x55e215304460 2024-04-24T09:10:25.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.327+0000 7f7ea2a00640 1 -- 172.21.15.83:0/579363194 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7ea80a4af0 con 0x55e215304460 2024-04-24T09:10:25.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.327+0000 7f7ea2a00640 1 -- 172.21.15.83:0/579363194 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 61) ==== 8362+0+0 (secure 0 0 0) 0x7f7ea80a4660 con 0x55e215304460 2024-04-24T09:10:25.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.327+0000 7f7ec6c27740 10 client.22113 fetch_fsmap finished waiting for FSMap version 61 2024-04-24T09:10:25.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.327+0000 7f7ec6c27740 10 client.22113 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-24T09:10:25.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.327+0000 7f7ec6c27740 20 client.22113 populate_metadata read hostname 'smithi083' 2024-04-24T09:10:25.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.327+0000 7f7ec6c27740 1 --2- 172.21.15.83:0/579363194 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55e215385660 0x55e215387a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:25.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.327+0000 7f7ec6c27740 4 client.22113 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:10:25.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.327+0000 7f7ec6c27740 1 -- 172.21.15.83:0/579363194 --> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e215270a60 con 0x55e215385660 2024-04-24T09:10:25.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.327+0000 7f7ec1400640 1 --2- 172.21.15.83:0/579363194 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55e215385660 0x55e215387a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:25.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.328+0000 7f7ec1400640 1 --2- 172.21.15.83:0/579363194 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55e215385660 0x55e215387a80 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5af9b70994da6acf server_cookie=1c89c7c4c10decdf in_seq=0 out_seq=0 2024-04-24T09:10:25.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.328+0000 7f7ea2a00640 10 client.22113 ms_handle_connect on v2:172.21.15.138:6834/1909219758 2024-04-24T09:10:25.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.329+0000 7f7ea2a00640 1 -- 172.21.15.83:0/579363194 <== mds.2 v2:172.21.15.138:6834/1909219758 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e215270a60 con 0x55e215385660 2024-04-24T09:10:25.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.329+0000 7f7ea2a00640 10 client.22113 handle_command_reply: tid=0 2024-04-24T09:10:25.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.329+0000 7f7ec6c27740 1 client.22113 shutdown 2024-04-24T09:10:25.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.329+0000 7f7ec6c27740 1 -- 172.21.15.83:0/579363194 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e215304460 msgr2=0x55e215308c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:25.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.329+0000 7f7ec6c27740 1 --2- 172.21.15.83:0/579363194 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e215304460 0x55e215308c20 secure :-1 s=READY pgs=2437 cs=0 l=1 rev1=1 crypto rx=0x7f7ea8017590 tx=0x7f7ea80d0150 comp rx=0 tx=0).stop 2024-04-24T09:10:25.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.329+0000 7f7ec6c27740 1 -- 172.21.15.83:0/579363194 shutdown_connections 2024-04-24T09:10:25.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.329+0000 7f7ec6c27740 1 --2- 172.21.15.83:0/579363194 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55e215385660 0x55e215387a80 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.329+0000 7f7ec6c27740 1 --2- 172.21.15.83:0/579363194 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e21530eb30 0x55e215309160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.329+0000 7f7ec6c27740 1 --2- 172.21.15.83:0/579363194 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e215304460 0x55e215308c20 unknown :-1 s=CLOSED pgs=2437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.329+0000 7f7ea2a00640 0 client.22113 ms_handle_reset on v2:172.21.15.138:6834/1909219758 2024-04-24T09:10:25.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.329+0000 7f7ec6c27740 1 --2- 172.21.15.83:0/579363194 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e215303b40 0x55e2153086e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.329+0000 7f7ec6c27740 1 -- 172.21.15.83:0/579363194 >> 172.21.15.83:0/579363194 conn(0x55e2152ffe60 msgr2=0x55e215319d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:25.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.329+0000 7f7ec6c27740 1 -- 172.21.15.83:0/579363194 shutdown_connections 2024-04-24T09:10:25.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.329+0000 7f7ec6c27740 1 -- 172.21.15.83:0/579363194 wait complete. 2024-04-24T09:10:25.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.329+0000 7f7ec6c27740 20 client.22113 trim_cache size 0 max 16384 2024-04-24T09:10:25.333 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:10:25.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.330+0000 7f7ec2800640 1 -- 172.21.15.83:0/1578179448 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f7ea40a3da0 msgr2=0x7f7ea40a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:25.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.330+0000 7f7ec2800640 1 --2- 172.21.15.83:0/1578179448 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f7ea40a3da0 0x7f7ea40a6260 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f7eb4007d10 tx=0x7f7eb4007340 comp rx=0 tx=0).stop 2024-04-24T09:10:25.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.330+0000 7f7ec2800640 1 -- 172.21.15.83:0/1578179448 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ebc103a20 msgr2=0x7f7ebc101290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:25.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.330+0000 7f7ec2800640 1 --2- 172.21.15.83:0/1578179448 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ebc103a20 0x7f7ebc101290 secure :-1 s=READY pgs=2433 cs=0 l=1 rev1=1 crypto rx=0x7f7ea800b4a0 tx=0x7f7ea800b960 comp rx=0 tx=0).stop 2024-04-24T09:10:25.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.330+0000 7f7ec2800640 1 -- 172.21.15.83:0/1578179448 shutdown_connections 2024-04-24T09:10:25.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.330+0000 7f7ec2800640 1 --2- 172.21.15.83:0/1578179448 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f7ea40a3da0 0x7f7ea40a6260 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.330+0000 7f7ec2800640 1 --2- 172.21.15.83:0/1578179448 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7ebc10c4d0 0x7f7ebc1017d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.330+0000 7f7ec2800640 1 --2- 172.21.15.83:0/1578179448 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ebc103a20 0x7f7ebc101290 unknown :-1 s=CLOSED pgs=2433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.330+0000 7f7ec2800640 1 --2- 172.21.15.83:0/1578179448 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ebc103100 0x7f7ebc100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.330+0000 7f7ec2800640 1 -- 172.21.15.83:0/1578179448 >> 172.21.15.83:0/1578179448 conn(0x7f7ebc0fc880 msgr2=0x7f7ebc0fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:25.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.330+0000 7f7ec2800640 1 -- 172.21.15.83:0/1578179448 shutdown_connections 2024-04-24T09:10:25.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.330+0000 7f7ec2800640 1 -- 172.21.15.83:0/1578179448 wait complete. 2024-04-24T09:10:25.340 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.340+0000 7faa22200640 1 -- 172.21.15.83:0/4209902245 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa1c0743a0 msgr2=0x7faa1c074780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:25.340 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.340+0000 7faa22200640 1 --2- 172.21.15.83:0/4209902245 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa1c0743a0 0x7faa1c074780 secure :-1 s=READY pgs=2438 cs=0 l=1 rev1=1 crypto rx=0x7faa10009900 tx=0x7faa1002a480 comp rx=0 tx=0).stop 2024-04-24T09:10:25.340 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.340+0000 7faa22200640 1 -- 172.21.15.83:0/4209902245 shutdown_connections 2024-04-24T09:10:25.340 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.340+0000 7faa22200640 1 --2- 172.21.15.83:0/4209902245 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa1c112e30 0x7faa1c115220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.340 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.340+0000 7faa22200640 1 --2- 172.21.15.83:0/4209902245 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faa1c074cc0 0x7faa1c1128f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.340 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.340+0000 7faa22200640 1 --2- 172.21.15.83:0/4209902245 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa1c0743a0 0x7faa1c074780 unknown :-1 s=CLOSED pgs=2438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.340 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.340+0000 7faa22200640 1 -- 172.21.15.83:0/4209902245 >> 172.21.15.83:0/4209902245 conn(0x7faa1c0fd950 msgr2=0x7faa1c0ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:25.341 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.340+0000 7faa22200640 1 -- 172.21.15.83:0/4209902245 shutdown_connections 2024-04-24T09:10:25.341 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.340+0000 7faa22200640 1 -- 172.21.15.83:0/4209902245 wait complete. 2024-04-24T09:10:25.343 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.341+0000 7faa22200640 1 Processor -- start 2024-04-24T09:10:25.343 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.341+0000 7faa22200640 1 -- start start 2024-04-24T09:10:25.343 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.341+0000 7faa22200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faa1c0743a0 0x7faa1c1a4930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:25.343 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.341+0000 7faa22200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa1c074cc0 0x7faa1c1a4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:25.343 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.341+0000 7faa22200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa1c112e30 0x7faa1c19eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:25.343 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.341+0000 7faa22200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7faa1c1186d0 con 0x7faa1c112e30 2024-04-24T09:10:25.343 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.341+0000 7faa22200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7faa1c118550 con 0x7faa1c074cc0 2024-04-24T09:10:25.343 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.341+0000 7faa22200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7faa1c118850 con 0x7faa1c0743a0 2024-04-24T09:10:25.343 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.342+0000 7faa21800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa1c112e30 0x7faa1c19eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:25.343 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.342+0000 7faa21800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa1c112e30 0x7faa1c19eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34788/0 (socket says 172.21.15.83:34788) 2024-04-24T09:10:25.343 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.342+0000 7faa21800640 1 -- 172.21.15.83:0/328851809 learned_addr learned my addr 172.21.15.83:0/328851809 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:10:25.343 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.342+0000 7faa1be00640 1 --2- 172.21.15.83:0/328851809 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa1c074cc0 0x7faa1c1a4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:25.344 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.342+0000 7faa20e00640 1 --2- 172.21.15.83:0/328851809 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faa1c0743a0 0x7faa1c1a4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:25.344 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.342+0000 7faa21800640 1 -- 172.21.15.83:0/328851809 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faa1c0743a0 msgr2=0x7faa1c1a4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:25.344 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.342+0000 7faa21800640 1 --2- 172.21.15.83:0/328851809 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faa1c0743a0 0x7faa1c1a4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.344 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.342+0000 7faa21800640 1 -- 172.21.15.83:0/328851809 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa1c074cc0 msgr2=0x7faa1c1a4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:25.344 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.342+0000 7faa21800640 1 --2- 172.21.15.83:0/328851809 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa1c074cc0 0x7faa1c1a4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.344 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.342+0000 7faa21800640 1 -- 172.21.15.83:0/328851809 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa1c19f2b0 con 0x7faa1c112e30 2024-04-24T09:10:25.344 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.342+0000 7faa21800640 1 --2- 172.21.15.83:0/328851809 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa1c112e30 0x7faa1c19eb30 secure :-1 s=READY pgs=2439 cs=0 l=1 rev1=1 crypto rx=0x7faa1400e9d0 tx=0x7faa1400ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:25.344 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.343+0000 7faa19600640 1 -- 172.21.15.83:0/328851809 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa1400cd80 con 0x7faa1c112e30 2024-04-24T09:10:25.344 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.343+0000 7faa22200640 1 -- 172.21.15.83:0/328851809 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa1c19f5a0 con 0x7faa1c112e30 2024-04-24T09:10:25.344 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.343+0000 7faa19600640 1 -- 172.21.15.83:0/328851809 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faa14005020 con 0x7faa1c112e30 2024-04-24T09:10:25.344 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.343+0000 7faa19600640 1 -- 172.21.15.83:0/328851809 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa140116d0 con 0x7faa1c112e30 2024-04-24T09:10:25.344 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.343+0000 7faa22200640 1 -- 172.21.15.83:0/328851809 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa1c1ab7f0 con 0x7faa1c112e30 2024-04-24T09:10:25.346 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T09:10:25.347 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.347+0000 7faa267ef740 1 Processor -- start 2024-04-24T09:10:25.348 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.347+0000 7faa19600640 1 -- 172.21.15.83:0/328851809 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7faa140051d0 con 0x7faa1c112e30 2024-04-24T09:10:25.348 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.347+0000 7faa267ef740 1 -- start start 2024-04-24T09:10:25.348 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.347+0000 7faa267ef740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557df3b2d3c0 0x557df3b2d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:25.348 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.347+0000 7faa267ef740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557df3b2dce0 0x557df3b38060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:25.348 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.347+0000 7faa267ef740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557df3b385a0 0x557df3b3a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:25.348 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.348+0000 7faa267ef740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x557df3a93900 con 0x557df3b2dce0 2024-04-24T09:10:25.348 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.348+0000 7faa267ef740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x557df3a39f20 con 0x557df3b385a0 2024-04-24T09:10:25.348 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.348+0000 7faa267ef740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x557df3a8f8c0 con 0x557df3b2d3c0 2024-04-24T09:10:25.353 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.348+0000 7faa20e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557df3b2dce0 0x557df3b38060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:25.354 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.348+0000 7faa20e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557df3b2dce0 0x557df3b38060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34790/0 (socket says 172.21.15.83:34790) 2024-04-24T09:10:25.354 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.348+0000 7faa20e00640 1 -- 172.21.15.83:0/349453884 learned_addr learned my addr 172.21.15.83:0/349453884 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:10:25.354 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.348+0000 7faa1be00640 1 --2- 172.21.15.83:0/349453884 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557df3b385a0 0x557df3b3a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:25.354 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.348+0000 7faa1be00640 1 --2- 172.21.15.83:0/349453884 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557df3b2d3c0 0x557df3b2d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:25.354 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.348+0000 7faa20e00640 1 -- 172.21.15.83:0/349453884 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557df3b2d3c0 msgr2=0x557df3b2d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:25.354 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.348+0000 7faa20e00640 1 --2- 172.21.15.83:0/349453884 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557df3b2d3c0 0x557df3b2d7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.354 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.348+0000 7faa20e00640 1 -- 172.21.15.83:0/349453884 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557df3b385a0 msgr2=0x557df3b3a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:25.354 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.348+0000 7faa20e00640 1 --2- 172.21.15.83:0/349453884 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557df3b385a0 0x557df3b3a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.354 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.348+0000 7faa20e00640 1 -- 172.21.15.83:0/349453884 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557df3a5ee10 con 0x557df3b2dce0 2024-04-24T09:10:25.354 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.348+0000 7faa1be00640 1 --2- 172.21.15.83:0/349453884 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557df3b385a0 0x557df3b3a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:10:25.354 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.349+0000 7faa19600640 1 --2- 172.21.15.83:0/328851809 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7faa040a3d90 0x7faa040a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:25.354 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.349+0000 7faa20e00640 1 --2- 172.21.15.83:0/349453884 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557df3b2dce0 0x557df3b38060 secure :-1 s=READY pgs=2440 cs=0 l=1 rev1=1 crypto rx=0x7faa10009900 tx=0x7faa10032c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b081ecb781b315c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:25.354 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.349+0000 7faa19600640 1 -- 172.21.15.83:0/328851809 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 9583+0+0 (secure 0 0 0) 0x7faa140d0d90 con 0x7faa1c112e30 2024-04-24T09:10:25.355 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.349+0000 7faa1be00640 1 --2- 172.21.15.83:0/328851809 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7faa040a3d90 0x7faa040a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:25.355 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.349+0000 7faa0e000640 1 -- 172.21.15.83:0/349453884 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa1003f070 con 0x557df3b2dce0 2024-04-24T09:10:25.355 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.349+0000 7faa0e000640 1 -- 172.21.15.83:0/349453884 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faa100316a0 con 0x557df3b2dce0 2024-04-24T09:10:25.355 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.349+0000 7faa0e000640 1 -- 172.21.15.83:0/349453884 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa10035070 con 0x557df3b2dce0 2024-04-24T09:10:25.355 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.350+0000 7faa267ef740 1 -- 172.21.15.83:0/349453884 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557df3b2dce0 msgr2=0x557df3b38060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:25.355 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.350+0000 7faa1be00640 1 --2- 172.21.15.83:0/328851809 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7faa040a3d90 0x7faa040a6250 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x557df3b2acf0 tx=0x7faa080046c0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:25.355 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.350+0000 7faa267ef740 1 --2- 172.21.15.83:0/349453884 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557df3b2dce0 0x557df3b38060 secure :-1 s=READY pgs=2440 cs=0 l=1 rev1=1 crypto rx=0x7faa10009900 tx=0x7faa10032c20 comp rx=0 tx=0).stop 2024-04-24T09:10:25.355 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.350+0000 7faa267ef740 1 -- 172.21.15.83:0/349453884 shutdown_connections 2024-04-24T09:10:25.355 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.350+0000 7faa267ef740 1 --2- 172.21.15.83:0/349453884 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557df3b385a0 0x557df3b3a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.355 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.350+0000 7faa267ef740 1 --2- 172.21.15.83:0/349453884 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557df3b2dce0 0x557df3b38060 unknown :-1 s=CLOSED pgs=2440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.355 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.350+0000 7faa267ef740 1 --2- 172.21.15.83:0/349453884 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557df3b2d3c0 0x557df3b2d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.355 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.350+0000 7faa267ef740 1 -- 172.21.15.83:0/349453884 >> 172.21.15.83:0/349453884 conn(0x557df3b298d0 msgr2=0x557df3b29cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:25.355 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.350+0000 7faa267ef740 1 -- 172.21.15.83:0/349453884 shutdown_connections 2024-04-24T09:10:25.355 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.350+0000 7faa267ef740 1 -- 172.21.15.83:0/349453884 wait complete. 2024-04-24T09:10:25.355 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.350+0000 7faa267ef740 1 -- start start 2024-04-24T09:10:25.356 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.351+0000 7faa267ef740 1 Processor -- start 2024-04-24T09:10:25.356 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.351+0000 7faa267ef740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557df3b2d3c0 0x557df3b35590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:25.356 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.351+0000 7faa267ef740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557df3b2dce0 0x557df3b35ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:25.356 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.351+0000 7faa267ef740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557df3b385a0 0x557df3b36010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:25.356 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.351+0000 7faa267ef740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x557df3b3b0e0 con 0x557df3b2dce0 2024-04-24T09:10:25.356 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.351+0000 7faa267ef740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x557df3a4d2c0 con 0x557df3b2d3c0 2024-04-24T09:10:25.356 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.351+0000 7faa267ef740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x557df3b3b260 con 0x557df3b385a0 2024-04-24T09:10:25.356 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.351+0000 7faa267ef740 10 client.0 _ll_register_callbacks cb 0x557df37a5a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:10:25.356 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.351+0000 7faa1be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557df3b2dce0 0x557df3b35ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:25.356 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.351+0000 7faa1be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557df3b2dce0 0x557df3b35ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34792/0 (socket says 172.21.15.83:34792) 2024-04-24T09:10:25.356 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.351+0000 7faa1be00640 1 -- 172.21.15.83:0/120414883 learned_addr learned my addr 172.21.15.83:0/120414883 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:10:25.356 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.352+0000 7faa20e00640 1 --2- 172.21.15.83:0/120414883 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557df3b2d3c0 0x557df3b35590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:25.356 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.352+0000 7faa21800640 1 --2- 172.21.15.83:0/120414883 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557df3b385a0 0x557df3b36010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:25.356 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.352+0000 7faa1be00640 1 -- 172.21.15.83:0/120414883 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557df3b385a0 msgr2=0x557df3b36010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:25.357 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.352+0000 7faa1be00640 1 --2- 172.21.15.83:0/120414883 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557df3b385a0 0x557df3b36010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.357 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.352+0000 7faa1be00640 1 -- 172.21.15.83:0/120414883 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557df3b2d3c0 msgr2=0x557df3b35590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:25.357 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.352+0000 7faa1be00640 1 --2- 172.21.15.83:0/120414883 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557df3b2d3c0 0x557df3b35590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.357 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.352+0000 7faa1be00640 1 -- 172.21.15.83:0/120414883 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557df3a5dfd0 con 0x557df3b2dce0 2024-04-24T09:10:25.357 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.352+0000 7faa20e00640 1 --2- 172.21.15.83:0/120414883 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557df3b2d3c0 0x557df3b35590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:10:25.357 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.352+0000 7faa1be00640 1 --2- 172.21.15.83:0/120414883 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557df3b2dce0 0x557df3b35ad0 secure :-1 s=READY pgs=2441 cs=0 l=1 rev1=1 crypto rx=0x7faa0800f670 tx=0x7faa08004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56cc306f93d06e5c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:25.357 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.352+0000 7faa02a00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:10:25.357 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.352+0000 7faa02a00640 1 -- 172.21.15.83:0/120414883 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa08027430 con 0x557df3b2dce0 2024-04-24T09:10:25.357 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.352+0000 7faa02a00640 1 -- 172.21.15.83:0/120414883 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(120..120 src has 1..120) ==== 9583+0+0 (secure 0 0 0) 0x7faa0801d410 con 0x557df3b2dce0 2024-04-24T09:10:25.357 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.352+0000 7faa267ef740 1 -- 172.21.15.83:0/120414883 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa100316a0 con 0x557df3b2dce0 2024-04-24T09:10:25.357 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.353+0000 7faa02a00640 1 -- 172.21.15.83:0/120414883 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7faa0801d610 con 0x557df3b2dce0 2024-04-24T09:10:25.357 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.353+0000 7faa267ef740 10 client.22125 fetch_fsmap learned FSMap version 61 2024-04-24T09:10:25.357 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.353+0000 7faa267ef740 1 -- 172.21.15.83:0/120414883 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x557df37a6210 con 0x557df3b2dce0 2024-04-24T09:10:25.357 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.353+0000 7faa02a00640 1 -- 172.21.15.83:0/120414883 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faa100316a0 con 0x557df3b2dce0 2024-04-24T09:10:25.357 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.353+0000 7faa02a00640 1 -- 172.21.15.83:0/120414883 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 61) ==== 8362+0+0 (secure 0 0 0) 0x7faa080278e0 con 0x557df3b2dce0 2024-04-24T09:10:25.358 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.353+0000 7faa267ef740 10 client.22125 fetch_fsmap finished waiting for FSMap version 61 2024-04-24T09:10:25.358 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.353+0000 7faa267ef740 10 client.22125 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:10:25.358 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.353+0000 7faa267ef740 20 client.22125 populate_metadata read hostname 'smithi083' 2024-04-24T09:10:25.358 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.353+0000 7faa267ef740 1 --2- 172.21.15.83:0/120414883 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x557df3b48700 0x557df3b4ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:25.358 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.353+0000 7faa267ef740 4 client.22125 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:10:25.358 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.353+0000 7faa267ef740 1 -- 172.21.15.83:0/120414883 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557df3aa1320 con 0x557df3b48700 2024-04-24T09:10:25.396 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:10:25 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/3697673284' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:10:25.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:10:25 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/3697673284' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:10:25.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:10:25 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/3697673284' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:10:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.495+0000 7faa20e00640 1 --2- 172.21.15.83:0/120414883 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x557df3b48700 0x557df3b4ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:25.497 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.497+0000 7faa20e00640 1 --2- 172.21.15.83:0/120414883 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x557df3b48700 0x557df3b4ab90 crc :-1 s=READY pgs=469 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7896de940f83d6b8 server_cookie=4c3d49d76681f398 in_seq=0 out_seq=0 2024-04-24T09:10:25.497 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.497+0000 7faa02a00640 10 client.22125 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:10:25.504 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.503+0000 7faa02a00640 1 -- 172.21.15.83:0/120414883 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557df3aa1320 con 0x557df3b48700 2024-04-24T09:10:25.504 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.503+0000 7faa02a00640 10 client.22125 handle_command_reply: tid=0 2024-04-24T09:10:25.504 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.503+0000 7faa267ef740 1 client.22125 shutdown 2024-04-24T09:10:25.504 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.504+0000 7faa267ef740 1 -- 172.21.15.83:0/120414883 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557df3b2dce0 msgr2=0x557df3b35ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:25.504 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.504+0000 7faa267ef740 1 --2- 172.21.15.83:0/120414883 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557df3b2dce0 0x557df3b35ad0 secure :-1 s=READY pgs=2441 cs=0 l=1 rev1=1 crypto rx=0x7faa0800f670 tx=0x7faa08004730 comp rx=0 tx=0).stop 2024-04-24T09:10:25.504 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.504+0000 7faa267ef740 1 -- 172.21.15.83:0/120414883 shutdown_connections 2024-04-24T09:10:25.504 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.504+0000 7faa267ef740 1 --2- 172.21.15.83:0/120414883 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x557df3b48700 0x557df3b4ab90 crc :-1 s=READY pgs=469 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.505 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.504+0000 7faa267ef740 1 --2- 172.21.15.83:0/120414883 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557df3b385a0 0x557df3b36010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.505 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.504+0000 7faa267ef740 1 --2- 172.21.15.83:0/120414883 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557df3b2dce0 0x557df3b35ad0 unknown :-1 s=CLOSED pgs=2441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.505 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.504+0000 7faa267ef740 1 --2- 172.21.15.83:0/120414883 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557df3b2d3c0 0x557df3b35590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.505 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.504+0000 7faa267ef740 1 -- 172.21.15.83:0/120414883 >> 172.21.15.83:0/120414883 conn(0x557df3b298d0 msgr2=0x557df3b28a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:25.506 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.505+0000 7faa02a00640 0 client.22125 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:10:25.506 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.505+0000 7faa267ef740 1 -- 172.21.15.83:0/120414883 shutdown_connections 2024-04-24T09:10:25.506 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.505+0000 7faa267ef740 1 -- 172.21.15.83:0/120414883 wait complete. 2024-04-24T09:10:25.506 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.505+0000 7faa267ef740 20 client.22125 trim_cache size 0 max 16384 2024-04-24T09:10:25.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.517+0000 7f68d1a00640 1 -- 172.21.15.83:0/1216115218 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68cc100c70 msgr2=0x7f68cc101050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:25.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.517+0000 7f68d1a00640 1 --2- 172.21.15.83:0/1216115218 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68cc100c70 0x7f68cc101050 secure :-1 s=READY pgs=2442 cs=0 l=1 rev1=1 crypto rx=0x7f68bc009900 tx=0x7f68bc02a410 comp rx=0 tx=0).stop 2024-04-24T09:10:25.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.517+0000 7f68d1a00640 1 -- 172.21.15.83:0/1216115218 shutdown_connections 2024-04-24T09:10:25.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.517+0000 7f68d1a00640 1 --2- 172.21.15.83:0/1216115218 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68cc10e820 0x7f68cc110c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.517+0000 7f68d1a00640 1 --2- 172.21.15.83:0/1216115218 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f68cc101620 0x7f68cc10e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.517+0000 7f68d1a00640 1 --2- 172.21.15.83:0/1216115218 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68cc100c70 0x7f68cc101050 unknown :-1 s=CLOSED pgs=2442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.518+0000 7f68d1a00640 1 -- 172.21.15.83:0/1216115218 >> 172.21.15.83:0/1216115218 conn(0x7f68cc0fc880 msgr2=0x7f68cc0feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:25.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.518+0000 7f68d1a00640 1 -- 172.21.15.83:0/1216115218 shutdown_connections 2024-04-24T09:10:25.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.518+0000 7f68d1a00640 1 -- 172.21.15.83:0/1216115218 wait complete. 2024-04-24T09:10:25.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.519+0000 7f68d1a00640 1 Processor -- start 2024-04-24T09:10:25.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.519+0000 7f68d1a00640 1 -- start start 2024-04-24T09:10:25.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.520+0000 7f68d1a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f68cc100c70 0x7f68cc102ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:25.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.520+0000 7f68d1a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68cc101620 0x7f68cc103400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:25.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.520+0000 7f68d1a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68cc10e820 0x7f68cc103940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:25.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.520+0000 7f68d1a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f68cc114410 con 0x7f68cc10e820 2024-04-24T09:10:25.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.520+0000 7f68d1a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f68cc114290 con 0x7f68cc101620 2024-04-24T09:10:25.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.520+0000 7f68d1a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f68cc114590 con 0x7f68cc100c70 2024-04-24T09:10:25.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.520+0000 7f68d1000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68cc10e820 0x7f68cc103940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:25.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.520+0000 7f68d1000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68cc10e820 0x7f68cc103940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34802/0 (socket says 172.21.15.83:34802) 2024-04-24T09:10:25.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.520+0000 7f68d1000640 1 -- 172.21.15.83:0/2936829389 learned_addr learned my addr 172.21.15.83:0/2936829389 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:10:25.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.520+0000 7f68cbe00640 1 --2- 172.21.15.83:0/2936829389 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f68cc100c70 0x7f68cc102ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:25.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.520+0000 7f68cb400640 1 --2- 172.21.15.83:0/2936829389 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68cc101620 0x7f68cc103400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:25.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.521+0000 7f68d1000640 1 -- 172.21.15.83:0/2936829389 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f68cc100c70 msgr2=0x7f68cc102ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:25.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.521+0000 7f68d1000640 1 --2- 172.21.15.83:0/2936829389 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f68cc100c70 0x7f68cc102ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.521+0000 7f68d1000640 1 -- 172.21.15.83:0/2936829389 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68cc101620 msgr2=0x7f68cc103400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:25.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.521+0000 7f68d1000640 1 --2- 172.21.15.83:0/2936829389 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68cc101620 0x7f68cc103400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.521+0000 7f68d1000640 1 -- 172.21.15.83:0/2936829389 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68cc1aafe0 con 0x7f68cc10e820 2024-04-24T09:10:25.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.521+0000 7f68d1000640 1 --2- 172.21.15.83:0/2936829389 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68cc10e820 0x7f68cc103940 secure :-1 s=READY pgs=2443 cs=0 l=1 rev1=1 crypto rx=0x7f68c400ba10 tx=0x7f68c400bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:25.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.522+0000 7f68c8c00640 1 -- 172.21.15.83:0/2936829389 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68c4019070 con 0x7f68cc10e820 2024-04-24T09:10:25.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.522+0000 7f68c8c00640 1 -- 172.21.15.83:0/2936829389 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f68c4014440 con 0x7f68cc10e820 2024-04-24T09:10:25.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.522+0000 7f68d1a00640 1 -- 172.21.15.83:0/2936829389 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68cc1ab2d0 con 0x7f68cc10e820 2024-04-24T09:10:25.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.522+0000 7f68c8c00640 1 -- 172.21.15.83:0/2936829389 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68c40125a0 con 0x7f68cc10e820 2024-04-24T09:10:25.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.522+0000 7f68d1a00640 1 -- 172.21.15.83:0/2936829389 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68cc1ab820 con 0x7f68cc10e820 2024-04-24T09:10:25.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.526+0000 7f68d5e26740 1 Processor -- start 2024-04-24T09:10:25.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.527+0000 7f68c8c00640 1 -- 172.21.15.83:0/2936829389 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f68c401c070 con 0x7f68cc10e820 2024-04-24T09:10:25.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.527+0000 7f68d5e26740 1 -- start start 2024-04-24T09:10:25.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.528+0000 7f68d5e26740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56478930cb40 0x56478930cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:25.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.528+0000 7f68d5e26740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56478930d460 0x5647893175f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:25.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.528+0000 7f68d5e26740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564789317b30 0x564789319f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:25.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.528+0000 7f68d5e26740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5647892721d0 con 0x56478930cb40 2024-04-24T09:10:25.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.528+0000 7f68d5e26740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x564789218450 con 0x564789317b30 2024-04-24T09:10:25.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.528+0000 7f68d5e26740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56478926e190 con 0x56478930d460 2024-04-24T09:10:25.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.528+0000 7f68cb400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56478930cb40 0x56478930cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:25.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.528+0000 7f68cb400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56478930cb40 0x56478930cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34804/0 (socket says 172.21.15.83:34804) 2024-04-24T09:10:25.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.528+0000 7f68cb400640 1 -- 172.21.15.83:0/3691992969 learned_addr learned my addr 172.21.15.83:0/3691992969 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:10:25.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.529+0000 7f68cbe00640 1 --2- 172.21.15.83:0/3691992969 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56478930d460 0x5647893175f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:25.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.529+0000 7f68cb400640 1 --2- 172.21.15.83:0/3691992969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564789317b30 0x564789319f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:25.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.529+0000 7f68cb400640 1 -- 172.21.15.83:0/3691992969 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56478930d460 msgr2=0x5647893175f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:25.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.529+0000 7f68cb400640 1 --2- 172.21.15.83:0/3691992969 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56478930d460 0x5647893175f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.529+0000 7f68cb400640 1 -- 172.21.15.83:0/3691992969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564789317b30 msgr2=0x564789319f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:25.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.529+0000 7f68cb400640 1 --2- 172.21.15.83:0/3691992969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564789317b30 0x564789319f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.529+0000 7f68cb400640 1 -- 172.21.15.83:0/3691992969 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647892d5310 con 0x56478930cb40 2024-04-24T09:10:25.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.529+0000 7f68c8c00640 1 --2- 172.21.15.83:0/2936829389 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f68b00a3da0 0x7f68b00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:25.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.529+0000 7f68c8c00640 1 -- 172.21.15.83:0/2936829389 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 9583+0+0 (secure 0 0 0) 0x7f68c40cffe0 con 0x7f68cc10e820 2024-04-24T09:10:25.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.530+0000 7f68cb400640 1 --2- 172.21.15.83:0/3691992969 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56478930cb40 0x56478930cf20 secure :-1 s=READY pgs=2444 cs=0 l=1 rev1=1 crypto rx=0x7f68b80138d0 tx=0x7f68b800a720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0c3fdf3e39f0da2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:25.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.530+0000 7f68cbe00640 1 --2- 172.21.15.83:0/2936829389 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f68b00a3da0 0x7f68b00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:25.531 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.530+0000 7faa267ef740 1 Processor -- start 2024-04-24T09:10:25.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.530+0000 7f68c1600640 1 -- 172.21.15.83:0/3691992969 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b8013250 con 0x56478930cb40 2024-04-24T09:10:25.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.530+0000 7f68c1600640 1 -- 172.21.15.83:0/3691992969 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f68b80054d0 con 0x56478930cb40 2024-04-24T09:10:25.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.530+0000 7f68c1600640 1 -- 172.21.15.83:0/3691992969 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b8004d40 con 0x56478930cb40 2024-04-24T09:10:25.532 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.531+0000 7faa267ef740 1 -- start start 2024-04-24T09:10:25.532 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.531+0000 7faa267ef740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557df3b2d3c0 0x557df3b37cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:25.532 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.531+0000 7faa267ef740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557df3b2dce0 0x557df3b440e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:25.532 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.531+0000 7faa267ef740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557df3b385a0 0x557df3b504a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:25.533 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.531+0000 7faa267ef740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7faa1c1b59f0 con 0x557df3b2dce0 2024-04-24T09:10:25.533 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.531+0000 7faa267ef740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7faa1c1b7530 con 0x557df3b2d3c0 2024-04-24T09:10:25.533 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.531+0000 7faa267ef740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7faa1c1b60c0 con 0x557df3b385a0 2024-04-24T09:10:25.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.531+0000 7f68d5e26740 1 -- 172.21.15.83:0/3691992969 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56478930cb40 msgr2=0x56478930cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:25.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.531+0000 7f68d5e26740 1 --2- 172.21.15.83:0/3691992969 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56478930cb40 0x56478930cf20 secure :-1 s=READY pgs=2444 cs=0 l=1 rev1=1 crypto rx=0x7f68b80138d0 tx=0x7f68b800a720 comp rx=0 tx=0).stop 2024-04-24T09:10:25.534 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.531+0000 7faa1be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557df3b2dce0 0x557df3b440e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:25.534 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.531+0000 7faa1be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557df3b2dce0 0x557df3b440e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34812/0 (socket says 172.21.15.83:34812) 2024-04-24T09:10:25.534 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.531+0000 7faa1be00640 1 -- 172.21.15.83:0/4102207037 learned_addr learned my addr 172.21.15.83:0/4102207037 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:10:25.534 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.531+0000 7faa20e00640 1 --2- 172.21.15.83:0/4102207037 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557df3b2d3c0 0x557df3b37cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:25.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.531+0000 7f68cbe00640 1 --2- 172.21.15.83:0/2936829389 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f68b00a3da0 0x7f68b00a6260 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f68bc000c00 tx=0x7f68bc005a20 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:25.534 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.532+0000 7faa21800640 1 --2- 172.21.15.83:0/4102207037 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557df3b385a0 0x557df3b504a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:25.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.532+0000 7f68d5e26740 1 -- 172.21.15.83:0/3691992969 shutdown_connections 2024-04-24T09:10:25.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.532+0000 7f68d5e26740 1 --2- 172.21.15.83:0/3691992969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564789317b30 0x564789319f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.532+0000 7f68d5e26740 1 --2- 172.21.15.83:0/3691992969 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56478930d460 0x5647893175f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.532+0000 7f68d5e26740 1 --2- 172.21.15.83:0/3691992969 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56478930cb40 0x56478930cf20 unknown :-1 s=CLOSED pgs=2444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.532+0000 7f68d5e26740 1 -- 172.21.15.83:0/3691992969 >> 172.21.15.83:0/3691992969 conn(0x564789308e60 msgr2=0x564789309240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:25.535 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.532+0000 7faa1be00640 1 -- 172.21.15.83:0/4102207037 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557df3b385a0 msgr2=0x557df3b504a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:25.535 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.532+0000 7faa1be00640 1 --2- 172.21.15.83:0/4102207037 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557df3b385a0 0x557df3b504a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.536 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.532+0000 7faa1be00640 1 -- 172.21.15.83:0/4102207037 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557df3b2d3c0 msgr2=0x557df3b37cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:25.536 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.532+0000 7faa1be00640 1 --2- 172.21.15.83:0/4102207037 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557df3b2d3c0 0x557df3b37cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.532+0000 7f68d5e26740 1 -- 172.21.15.83:0/3691992969 shutdown_connections 2024-04-24T09:10:25.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.532+0000 7f68d5e26740 1 -- 172.21.15.83:0/3691992969 wait complete. 2024-04-24T09:10:25.536 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.532+0000 7faa1be00640 1 -- 172.21.15.83:0/4102207037 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557df3872620 con 0x557df3b2dce0 2024-04-24T09:10:25.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.533+0000 7f68d5e26740 1 -- start start 2024-04-24T09:10:25.537 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.533+0000 7faa1be00640 1 --2- 172.21.15.83:0/4102207037 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557df3b2dce0 0x557df3b440e0 secure :-1 s=READY pgs=2445 cs=0 l=1 rev1=1 crypto rx=0x7faa08005b60 tx=0x7faa0800bb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9070d02dfb261700 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:25.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.533+0000 7f68d5e26740 1 Processor -- start 2024-04-24T09:10:25.538 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.533+0000 7faa0d600640 1 -- 172.21.15.83:0/4102207037 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa08039070 con 0x557df3b2dce0 2024-04-24T09:10:25.538 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.533+0000 7faa0d600640 1 -- 172.21.15.83:0/4102207037 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faa0801e410 con 0x557df3b2dce0 2024-04-24T09:10:25.538 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.533+0000 7faa0d600640 1 -- 172.21.15.83:0/4102207037 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa0801ea10 con 0x557df3b2dce0 2024-04-24T09:10:25.538 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.534+0000 7faa267ef740 1 -- 172.21.15.83:0/4102207037 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557df3b2dce0 msgr2=0x557df3b440e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:25.538 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.534+0000 7faa267ef740 1 --2- 172.21.15.83:0/4102207037 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557df3b2dce0 0x557df3b440e0 secure :-1 s=READY pgs=2445 cs=0 l=1 rev1=1 crypto rx=0x7faa08005b60 tx=0x7faa0800bb00 comp rx=0 tx=0).stop 2024-04-24T09:10:25.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.534+0000 7f68d5e26740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56478930cb40 0x5647893175e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:25.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.534+0000 7f68d5e26740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56478930d460 0x5647893148d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:25.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.534+0000 7f68d5e26740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564789317b30 0x564789315010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:25.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.534+0000 7f68d5e26740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56478931a960 con 0x56478930d460 2024-04-24T09:10:25.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.534+0000 7f68d5e26740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56478922b810 con 0x564789317b30 2024-04-24T09:10:25.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.534+0000 7f68d5e26740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56478931aae0 con 0x56478930cb40 2024-04-24T09:10:25.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.534+0000 7f68d5e26740 10 client.0 _ll_register_callbacks cb 0x5647891f9140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:10:25.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.534+0000 7f68cbe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56478930cb40 0x5647893175e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:25.539 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.535+0000 7faa267ef740 1 -- 172.21.15.83:0/4102207037 shutdown_connections 2024-04-24T09:10:25.540 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.535+0000 7faa267ef740 1 --2- 172.21.15.83:0/4102207037 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557df3b385a0 0x557df3b504a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.540 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.535+0000 7faa267ef740 1 --2- 172.21.15.83:0/4102207037 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557df3b2dce0 0x557df3b440e0 unknown :-1 s=CLOSED pgs=2445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.540 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.535+0000 7faa267ef740 1 --2- 172.21.15.83:0/4102207037 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557df3b2d3c0 0x557df3b37cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.540 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.535+0000 7faa267ef740 1 -- 172.21.15.83:0/4102207037 >> 172.21.15.83:0/4102207037 conn(0x557df3b298d0 msgr2=0x557df3b30880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:25.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.535+0000 7f68cbe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56478930cb40 0x5647893175e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:51384/0 (socket says 172.21.15.83:51384) 2024-04-24T09:10:25.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.535+0000 7f68cbe00640 1 -- 172.21.15.83:0/1227037328 learned_addr learned my addr 172.21.15.83:0/1227037328 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:10:25.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.535+0000 7f68d1000640 1 --2- 172.21.15.83:0/1227037328 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564789317b30 0x564789315010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:25.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.535+0000 7f68cb400640 1 --2- 172.21.15.83:0/1227037328 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56478930d460 0x5647893148d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:25.541 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.535+0000 7faa267ef740 1 -- 172.21.15.83:0/4102207037 shutdown_connections 2024-04-24T09:10:25.541 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.535+0000 7faa267ef740 1 -- 172.21.15.83:0/4102207037 wait complete. 2024-04-24T09:10:25.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.535+0000 7f68cbe00640 1 -- 172.21.15.83:0/1227037328 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564789317b30 msgr2=0x564789315010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:25.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.535+0000 7f68cbe00640 1 --2- 172.21.15.83:0/1227037328 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564789317b30 0x564789315010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.541 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.535+0000 7faa267ef740 1 -- start start 2024-04-24T09:10:25.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.535+0000 7f68cbe00640 1 -- 172.21.15.83:0/1227037328 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56478930d460 msgr2=0x5647893148d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:25.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.535+0000 7f68cbe00640 1 --2- 172.21.15.83:0/1227037328 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56478930d460 0x5647893148d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.535+0000 7f68cbe00640 1 -- 172.21.15.83:0/1227037328 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56478923c3f0 con 0x56478930cb40 2024-04-24T09:10:25.542 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.536+0000 7faa267ef740 1 Processor -- start 2024-04-24T09:10:25.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.536+0000 7f68cb400640 1 --2- 172.21.15.83:0/1227037328 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56478930d460 0x5647893148d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:10:25.543 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.536+0000 7faa267ef740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557df3b2d3c0 0x557df3b406c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:25.543 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.536+0000 7faa267ef740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557df3b2dce0 0x557df3b3bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:25.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.536+0000 7f68cbe00640 1 --2- 172.21.15.83:0/1227037328 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56478930cb40 0x5647893175e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f68bc009900 tx=0x7f68bc009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc237356864768ad server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:25.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.536+0000 7f68b6000640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-04-24T09:10:25.543 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.537+0000 7faa1be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557df3b2dce0 0x557df3b3bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:25.544 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.537+0000 7faa267ef740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557df3b385a0 0x557df3b3c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:25.544 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.537+0000 7faa267ef740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7faa1c1b6e60 con 0x557df3b2dce0 2024-04-24T09:10:25.544 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.537+0000 7faa267ef740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7faa1c1b6790 con 0x557df3b385a0 2024-04-24T09:10:25.544 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.537+0000 7faa267ef740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7faa1c1b39f0 con 0x557df3b2d3c0 2024-04-24T09:10:25.544 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.537+0000 7faa1be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557df3b2dce0 0x557df3b3bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34840/0 (socket says 172.21.15.83:34840) 2024-04-24T09:10:25.544 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.537+0000 7faa1be00640 1 -- 172.21.15.83:0/3465781632 learned_addr learned my addr 172.21.15.83:0/3465781632 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:10:25.544 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.537+0000 7faa267ef740 10 client.0 _ll_register_callbacks cb 0x7faa08005a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:10:25.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.537+0000 7f68b6000640 1 -- 172.21.15.83:0/1227037328 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68bc04d070 con 0x56478930cb40 2024-04-24T09:10:25.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.537+0000 7f68d5e26740 1 -- 172.21.15.83:0/1227037328 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68b80054d0 con 0x56478930cb40 2024-04-24T09:10:25.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.537+0000 7f68b6000640 1 -- 172.21.15.83:0/1227037328 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(120..120 src has 1..120) ==== 9583+0+0 (secure 0 0 0) 0x7f68bc0318b0 con 0x56478930cb40 2024-04-24T09:10:25.545 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.537+0000 7faa20e00640 1 --2- 172.21.15.83:0/3465781632 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557df3b2d3c0 0x557df3b406c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:25.545 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.537+0000 7faa1be00640 1 -- 172.21.15.83:0/3465781632 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557df3b2d3c0 msgr2=0x557df3b406c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:25.545 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.537+0000 7faa1be00640 1 --2- 172.21.15.83:0/3465781632 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557df3b2d3c0 0x557df3b406c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.545 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.537+0000 7faa1be00640 1 -- 172.21.15.83:0/3465781632 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557df3b385a0 msgr2=0x557df3b3c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:25.545 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.537+0000 7faa1be00640 1 --2- 172.21.15.83:0/3465781632 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557df3b385a0 0x557df3b3c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.545 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.537+0000 7faa1be00640 1 -- 172.21.15.83:0/3465781632 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557df3b3c630 con 0x557df3b2dce0 2024-04-24T09:10:25.545 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.538+0000 7faa1be00640 1 --2- 172.21.15.83:0/3465781632 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557df3b2dce0 0x557df3b3bbb0 secure :-1 s=READY pgs=2446 cs=0 l=1 rev1=1 crypto rx=0x7faa08027e70 tx=0x7faa08004250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e675bd2addff205 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:25.545 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.538+0000 7faa02a00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:10:25.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.537+0000 7f68b6000640 1 -- 172.21.15.83:0/1227037328 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f68bc059050 con 0x56478930cb40 2024-04-24T09:10:25.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.538+0000 7f68d5e26740 10 client.29158 fetch_fsmap learned FSMap version 61 2024-04-24T09:10:25.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.538+0000 7f68d5e26740 1 -- 172.21.15.83:0/1227037328 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x564788f843f0 con 0x56478930cb40 2024-04-24T09:10:25.546 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.538+0000 7faa02a00640 1 -- 172.21.15.83:0/3465781632 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa08042070 con 0x557df3b2dce0 2024-04-24T09:10:25.546 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.538+0000 7faa02a00640 1 -- 172.21.15.83:0/3465781632 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(120..120 src has 1..120) ==== 9583+0+0 (secure 0 0 0) 0x7faa0801d7b0 con 0x557df3b2dce0 2024-04-24T09:10:25.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.538+0000 7f68b6000640 1 -- 172.21.15.83:0/1227037328 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f68b80054d0 con 0x56478930cb40 2024-04-24T09:10:25.547 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.538+0000 7faa267ef740 1 -- 172.21.15.83:0/3465781632 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa0801e410 con 0x557df3b2dce0 2024-04-24T09:10:25.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.539+0000 7f68b6000640 1 -- 172.21.15.83:0/1227037328 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 61) ==== 8362+0+0 (secure 0 0 0) 0x7f68bc049b80 con 0x56478930cb40 2024-04-24T09:10:25.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.539+0000 7f68d5e26740 10 client.29158 fetch_fsmap finished waiting for FSMap version 61 2024-04-24T09:10:25.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.539+0000 7f68d5e26740 10 client.29158 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-24T09:10:25.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.539+0000 7f68d5e26740 20 client.29158 populate_metadata read hostname 'smithi083' 2024-04-24T09:10:25.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.539+0000 7f68d5e26740 1 --2- 172.21.15.83:0/1227037328 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x564789327ca0 0x56478932a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:25.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.539+0000 7f68d5e26740 4 client.29158 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:10:25.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.539+0000 7f68d5e26740 1 -- 172.21.15.83:0/1227037328 --> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56478927fbf0 con 0x564789327ca0 2024-04-24T09:10:25.548 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.539+0000 7faa02a00640 1 -- 172.21.15.83:0/3465781632 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7faa08039070 con 0x557df3b2dce0 2024-04-24T09:10:25.548 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.539+0000 7faa267ef740 10 client.22143 fetch_fsmap learned FSMap version 61 2024-04-24T09:10:25.549 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.539+0000 7faa267ef740 1 -- 172.21.15.83:0/3465781632 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x557df3baea30 con 0x557df3b2dce0 2024-04-24T09:10:25.549 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.539+0000 7faa02a00640 1 -- 172.21.15.83:0/3465781632 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faa0801e410 con 0x557df3b2dce0 2024-04-24T09:10:25.549 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.539+0000 7faa02a00640 1 -- 172.21.15.83:0/3465781632 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 61) ==== 8362+0+0 (secure 0 0 0) 0x7faa08027940 con 0x557df3b2dce0 2024-04-24T09:10:25.549 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.539+0000 7faa267ef740 10 client.22143 fetch_fsmap finished waiting for FSMap version 61 2024-04-24T09:10:25.549 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.539+0000 7faa267ef740 10 client.22143 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:10:25.549 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.539+0000 7faa267ef740 20 client.22143 populate_metadata read hostname 'smithi083' 2024-04-24T09:10:25.549 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.539+0000 7faa267ef740 1 --2- 172.21.15.83:0/3465781632 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x557df3baef30 0x557df3bb1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:25.549 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.539+0000 7faa267ef740 4 client.22143 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T09:10:25.549 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.539+0000 7faa267ef740 1 -- 172.21.15.83:0/3465781632 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557df3a9b190 con 0x557df3baef30 2024-04-24T09:10:25.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.539+0000 7f68cb400640 1 --2- 172.21.15.83:0/1227037328 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x564789327ca0 0x56478932a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:25.550 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.540+0000 7faa20e00640 1 --2- 172.21.15.83:0/3465781632 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x557df3baef30 0x557df3bb1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:25.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.540+0000 7f68cb400640 1 --2- 172.21.15.83:0/1227037328 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x564789327ca0 0x56478932a0e0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=31f5a62fa50d0ec6 server_cookie=6ce1cddf05a05764 in_seq=0 out_seq=0 2024-04-24T09:10:25.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.540+0000 7f68b6000640 10 client.29158 ms_handle_connect on v2:172.21.15.138:6836/900566754 2024-04-24T09:10:25.550 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.541+0000 7faa20e00640 1 --2- 172.21.15.83:0/3465781632 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x557df3baef30 0x557df3bb1350 crc :-1 s=READY pgs=470 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7101c8b1943893f8 server_cookie=21a79952fde870d0 in_seq=0 out_seq=0 2024-04-24T09:10:25.551 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.541+0000 7faa02a00640 10 client.22143 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:10:25.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.542+0000 7f68b6000640 1 -- 172.21.15.83:0/1227037328 <== mds.1 v2:172.21.15.138:6836/900566754 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56478927fbf0 con 0x564789327ca0 2024-04-24T09:10:25.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.542+0000 7f68b6000640 10 client.29158 handle_command_reply: tid=0 2024-04-24T09:10:25.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.542+0000 7f68d5e26740 1 client.29158 shutdown 2024-04-24T09:10:25.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.542+0000 7f68d5e26740 1 -- 172.21.15.83:0/1227037328 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56478930cb40 msgr2=0x5647893175e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:25.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.542+0000 7f68d5e26740 1 --2- 172.21.15.83:0/1227037328 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56478930cb40 0x5647893175e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f68bc009900 tx=0x7f68bc009260 comp rx=0 tx=0).stop 2024-04-24T09:10:25.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.543+0000 7f68d5e26740 1 -- 172.21.15.83:0/1227037328 shutdown_connections 2024-04-24T09:10:25.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.543+0000 7f68d5e26740 1 --2- 172.21.15.83:0/1227037328 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564789317b30 0x564789315010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.543+0000 7f68d5e26740 1 --2- 172.21.15.83:0/1227037328 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56478930d460 0x5647893148d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.543+0000 7f68d5e26740 1 --2- 172.21.15.83:0/1227037328 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x564789327ca0 0x56478932a0e0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.543+0000 7f68d5e26740 1 --2- 172.21.15.83:0/1227037328 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56478930cb40 0x5647893175e0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.543+0000 7f68d5e26740 1 -- 172.21.15.83:0/1227037328 >> 172.21.15.83:0/1227037328 conn(0x564789308e60 msgr2=0x56478930ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:25.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.543+0000 7f68b6000640 0 client.29158 ms_handle_reset on v2:172.21.15.138:6836/900566754 2024-04-24T09:10:25.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.543+0000 7f68d5e26740 1 -- 172.21.15.83:0/1227037328 shutdown_connections 2024-04-24T09:10:25.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.543+0000 7f68d5e26740 1 -- 172.21.15.83:0/1227037328 wait complete. 2024-04-24T09:10:25.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.544+0000 7f68d5e26740 20 client.29158 trim_cache size 0 max 16384 2024-04-24T09:10:25.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.570+0000 7f68d5e26740 1 Processor -- start 2024-04-24T09:10:25.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.571+0000 7f68d5e26740 1 -- start start 2024-04-24T09:10:25.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.572+0000 7f68d5e26740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56478930cb40 0x564789323820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:25.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.572+0000 7f68d5e26740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56478930d460 0x564789314520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:25.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.572+0000 7f68d5e26740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564789317b30 0x564789317110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:25.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.572+0000 7f68d5e26740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f68cc1b5ab0 con 0x564789317b30 2024-04-24T09:10:25.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.572+0000 7f68d5e26740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f68cc1b75f0 con 0x56478930d460 2024-04-24T09:10:25.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.572+0000 7f68d5e26740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f68cc1b6180 con 0x56478930cb40 2024-04-24T09:10:25.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.572+0000 7f68cb400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56478930d460 0x564789314520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:25.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.572+0000 7f68cb400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56478930d460 0x564789314520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:41980/0 (socket says 172.21.15.83:41980) 2024-04-24T09:10:25.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.572+0000 7f68cb400640 1 -- 172.21.15.83:0/2174798459 learned_addr learned my addr 172.21.15.83:0/2174798459 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:10:25.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.572+0000 7f68d1000640 1 --2- 172.21.15.83:0/2174798459 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564789317b30 0x564789317110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:25.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.573+0000 7f68cb400640 1 -- 172.21.15.83:0/2174798459 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56478930cb40 msgr2=0x564789323820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:25.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.573+0000 7f68cb400640 1 --2- 172.21.15.83:0/2174798459 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56478930cb40 0x564789323820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.573+0000 7f68cb400640 1 -- 172.21.15.83:0/2174798459 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564789317b30 msgr2=0x564789317110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:25.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.573+0000 7f68cb400640 1 --2- 172.21.15.83:0/2174798459 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564789317b30 0x564789317110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.573+0000 7f68cb400640 1 -- 172.21.15.83:0/2174798459 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56478904e7d0 con 0x56478930d460 2024-04-24T09:10:25.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.574+0000 7f68d1000640 1 --2- 172.21.15.83:0/2174798459 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564789317b30 0x564789317110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:10:25.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.574+0000 7f68cb400640 1 --2- 172.21.15.83:0/2174798459 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56478930d460 0x564789314520 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f68b8004990 tx=0x7f68b80049c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35828c2545746d87 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:25.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.575+0000 7f68c0c00640 1 -- 172.21.15.83:0/2174798459 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b8004660 con 0x56478930d460 2024-04-24T09:10:25.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.575+0000 7f68c0c00640 1 -- 172.21.15.83:0/2174798459 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f68b8005330 con 0x56478930d460 2024-04-24T09:10:25.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.575+0000 7f68c0c00640 1 -- 172.21.15.83:0/2174798459 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b8004280 con 0x56478930d460 2024-04-24T09:10:25.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.577+0000 7f68d5e26740 1 -- 172.21.15.83:0/2174798459 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56478930d460 msgr2=0x564789314520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:25.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.577+0000 7f68d5e26740 1 --2- 172.21.15.83:0/2174798459 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56478930d460 0x564789314520 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f68b8004990 tx=0x7f68b80049c0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.577+0000 7f68d5e26740 1 -- 172.21.15.83:0/2174798459 shutdown_connections 2024-04-24T09:10:25.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.577+0000 7f68d5e26740 1 --2- 172.21.15.83:0/2174798459 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564789317b30 0x564789317110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.577+0000 7f68d5e26740 1 --2- 172.21.15.83:0/2174798459 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56478930d460 0x564789314520 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.577+0000 7f68d5e26740 1 --2- 172.21.15.83:0/2174798459 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56478930cb40 0x564789323820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.577+0000 7f68d5e26740 1 -- 172.21.15.83:0/2174798459 >> 172.21.15.83:0/2174798459 conn(0x564789308e60 msgr2=0x564789313e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:25.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.577+0000 7f68d5e26740 1 -- 172.21.15.83:0/2174798459 shutdown_connections 2024-04-24T09:10:25.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.577+0000 7f68d5e26740 1 -- 172.21.15.83:0/2174798459 wait complete. 2024-04-24T09:10:25.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.578+0000 7f68d5e26740 1 -- start start 2024-04-24T09:10:25.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.578+0000 7f68d5e26740 1 Processor -- start 2024-04-24T09:10:25.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.579+0000 7f68d5e26740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56478930cb40 0x564789320180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:25.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.579+0000 7f68d5e26740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56478930d460 0x564789311e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:25.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.579+0000 7f68d5e26740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564789317b30 0x5647893123c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:25.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.579+0000 7f68d5e26740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f68cc1b6f20 con 0x56478930d460 2024-04-24T09:10:25.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.579+0000 7f68d5e26740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f68cc1b6850 con 0x564789317b30 2024-04-24T09:10:25.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.579+0000 7f68d5e26740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f68cc1b3ab0 con 0x56478930cb40 2024-04-24T09:10:25.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.579+0000 7f68d5e26740 10 client.0 _ll_register_callbacks cb 0x7f68bc0045b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:10:25.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.579+0000 7f68cb400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56478930d460 0x564789311e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:25.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.579+0000 7f68d1000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564789317b30 0x5647893123c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:25.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.579+0000 7f68cb400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56478930d460 0x564789311e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34868/0 (socket says 172.21.15.83:34868) 2024-04-24T09:10:25.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.579+0000 7f68cb400640 1 -- 172.21.15.83:0/1596615974 learned_addr learned my addr 172.21.15.83:0/1596615974 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:10:25.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.580+0000 7f68cb400640 1 -- 172.21.15.83:0/1596615974 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56478930cb40 msgr2=0x564789320180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:25.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.580+0000 7f68cb400640 1 --2- 172.21.15.83:0/1596615974 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56478930cb40 0x564789320180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.580+0000 7f68cb400640 1 -- 172.21.15.83:0/1596615974 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564789317b30 msgr2=0x5647893123c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:25.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.580+0000 7f68cb400640 1 --2- 172.21.15.83:0/1596615974 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564789317b30 0x5647893123c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.580+0000 7f68cb400640 1 -- 172.21.15.83:0/1596615974 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564789318a10 con 0x56478930d460 2024-04-24T09:10:25.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.581+0000 7f68cb400640 1 --2- 172.21.15.83:0/1596615974 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56478930d460 0x564789311e20 secure :-1 s=READY pgs=2447 cs=0 l=1 rev1=1 crypto rx=0x7f68b80057c0 tx=0x7f68b800ae80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3f4b34adc7e4c86 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:25.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.581+0000 7f68b6000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:10:25.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.581+0000 7f68b6000640 1 -- 172.21.15.83:0/1596615974 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b8013470 con 0x56478930d460 2024-04-24T09:10:25.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.581+0000 7f68b6000640 1 -- 172.21.15.83:0/1596615974 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(120..120 src has 1..120) ==== 9583+0+0 (secure 0 0 0) 0x7f68b8004a30 con 0x56478930d460 2024-04-24T09:10:25.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.581+0000 7f68d5e26740 1 -- 172.21.15.83:0/1596615974 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68b8005330 con 0x56478930d460 2024-04-24T09:10:25.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.582+0000 7f68b6000640 1 -- 172.21.15.83:0/1596615974 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f68b803f050 con 0x56478930d460 2024-04-24T09:10:25.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.582+0000 7f68d5e26740 10 client.22149 fetch_fsmap learned FSMap version 61 2024-04-24T09:10:25.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.582+0000 7f68d5e26740 1 -- 172.21.15.83:0/1596615974 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x56478938e5c0 con 0x56478930d460 2024-04-24T09:10:25.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.582+0000 7f68b6000640 1 -- 172.21.15.83:0/1596615974 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f68b8005330 con 0x56478930d460 2024-04-24T09:10:25.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.583+0000 7f68b6000640 1 -- 172.21.15.83:0/1596615974 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 61) ==== 8362+0+0 (secure 0 0 0) 0x7f68b801d020 con 0x56478930d460 2024-04-24T09:10:25.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.583+0000 7f68d5e26740 10 client.22149 fetch_fsmap finished waiting for FSMap version 61 2024-04-24T09:10:25.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.583+0000 7f68d5e26740 10 client.22149 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-24T09:10:25.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.583+0000 7f68d5e26740 20 client.22149 populate_metadata read hostname 'smithi083' 2024-04-24T09:10:25.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.583+0000 7f68d5e26740 1 --2- 172.21.15.83:0/1596615974 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x56478938eac0 0x564789390ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:25.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.583+0000 7f68d5e26740 4 client.22149 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:10:25.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.583+0000 7f68d5e26740 1 -- 172.21.15.83:0/1596615974 --> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564789279a60 con 0x56478938eac0 2024-04-24T09:10:25.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.583+0000 7f68cbe00640 1 --2- 172.21.15.83:0/1596615974 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x56478938eac0 0x564789390ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:25.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.586+0000 7f68cbe00640 1 --2- 172.21.15.83:0/1596615974 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x56478938eac0 0x564789390ee0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7f67a4f55d611a65 server_cookie=d0331e64d757de3f in_seq=0 out_seq=0 2024-04-24T09:10:25.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.586+0000 7f68b6000640 10 client.22149 ms_handle_connect on v2:172.21.15.138:6836/900566754 2024-04-24T09:10:25.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.587+0000 7f68b6000640 1 -- 172.21.15.83:0/1596615974 <== mds.1 v2:172.21.15.138:6836/900566754 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564789279a60 con 0x56478938eac0 2024-04-24T09:10:25.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.587+0000 7f68b6000640 10 client.22149 handle_command_reply: tid=0 2024-04-24T09:10:25.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.587+0000 7f68d5e26740 1 client.22149 shutdown 2024-04-24T09:10:25.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.588+0000 7f68d5e26740 1 -- 172.21.15.83:0/1596615974 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56478930d460 msgr2=0x564789311e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:25.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.588+0000 7f68d5e26740 1 --2- 172.21.15.83:0/1596615974 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56478930d460 0x564789311e20 secure :-1 s=READY pgs=2447 cs=0 l=1 rev1=1 crypto rx=0x7f68b80057c0 tx=0x7f68b800ae80 comp rx=0 tx=0).stop 2024-04-24T09:10:25.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.588+0000 7f68d5e26740 1 -- 172.21.15.83:0/1596615974 shutdown_connections 2024-04-24T09:10:25.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.588+0000 7f68d5e26740 1 --2- 172.21.15.83:0/1596615974 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564789317b30 0x5647893123c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.588+0000 7f68d5e26740 1 --2- 172.21.15.83:0/1596615974 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56478930d460 0x564789311e20 unknown :-1 s=CLOSED pgs=2447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.588+0000 7f68d5e26740 1 --2- 172.21.15.83:0/1596615974 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x56478938eac0 0x564789390ee0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.588+0000 7f68d5e26740 1 --2- 172.21.15.83:0/1596615974 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56478930cb40 0x564789320180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.588+0000 7f68d5e26740 1 -- 172.21.15.83:0/1596615974 >> 172.21.15.83:0/1596615974 conn(0x564789308e60 msgr2=0x5647893146a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:25.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.588+0000 7f68b6000640 0 client.22149 ms_handle_reset on v2:172.21.15.138:6836/900566754 2024-04-24T09:10:25.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.588+0000 7f68d5e26740 1 -- 172.21.15.83:0/1596615974 shutdown_connections 2024-04-24T09:10:25.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.589+0000 7f68d5e26740 1 -- 172.21.15.83:0/1596615974 wait complete. 2024-04-24T09:10:25.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.589+0000 7f68d5e26740 20 client.22149 trim_cache size 0 max 16384 2024-04-24T09:10:25.590 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:10:25.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.590+0000 7f68d1a00640 1 -- 172.21.15.83:0/2936829389 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f68b00a3da0 msgr2=0x7f68b00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:25.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.590+0000 7f68d1a00640 1 --2- 172.21.15.83:0/2936829389 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f68b00a3da0 0x7f68b00a6260 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f68bc000c00 tx=0x7f68bc005a20 comp rx=0 tx=0).stop 2024-04-24T09:10:25.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.590+0000 7f68d1a00640 1 -- 172.21.15.83:0/2936829389 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68cc10e820 msgr2=0x7f68cc103940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:25.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.590+0000 7f68d1a00640 1 --2- 172.21.15.83:0/2936829389 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68cc10e820 0x7f68cc103940 secure :-1 s=READY pgs=2443 cs=0 l=1 rev1=1 crypto rx=0x7f68c400ba10 tx=0x7f68c400bed0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.590+0000 7f68d1a00640 1 -- 172.21.15.83:0/2936829389 shutdown_connections 2024-04-24T09:10:25.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.590+0000 7f68d1a00640 1 --2- 172.21.15.83:0/2936829389 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f68b00a3da0 0x7f68b00a6260 secure :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f68bc000c00 tx=0x7f68bc005a20 comp rx=0 tx=0).stop 2024-04-24T09:10:25.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.590+0000 7f68d1a00640 1 --2- 172.21.15.83:0/2936829389 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68cc10e820 0x7f68cc103940 unknown :-1 s=CLOSED pgs=2443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.590+0000 7f68d1a00640 1 --2- 172.21.15.83:0/2936829389 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68cc101620 0x7f68cc103400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.590+0000 7f68d1a00640 1 --2- 172.21.15.83:0/2936829389 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f68cc100c70 0x7f68cc102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.590+0000 7f68d1a00640 1 -- 172.21.15.83:0/2936829389 >> 172.21.15.83:0/2936829389 conn(0x7f68cc0fc880 msgr2=0x7f68cc0fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:25.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.590+0000 7f68d1a00640 1 -- 172.21.15.83:0/2936829389 shutdown_connections 2024-04-24T09:10:25.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.591+0000 7f68d1a00640 1 -- 172.21.15.83:0/2936829389 wait complete. 2024-04-24T09:10:25.614 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T09:10:25.735 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.735+0000 7faa02a00640 1 -- 172.21.15.83:0/3465781632 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557df3a9b190 con 0x557df3baef30 2024-04-24T09:10:25.735 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.735+0000 7faa02a00640 10 client.22143 handle_command_reply: tid=0 2024-04-24T09:10:25.735 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.735+0000 7faa267ef740 1 client.22143 shutdown 2024-04-24T09:10:25.735 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.735+0000 7faa267ef740 1 -- 172.21.15.83:0/3465781632 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557df3b2dce0 msgr2=0x557df3b3bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:25.735 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.735+0000 7faa267ef740 1 --2- 172.21.15.83:0/3465781632 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557df3b2dce0 0x557df3b3bbb0 secure :-1 s=READY pgs=2446 cs=0 l=1 rev1=1 crypto rx=0x7faa08027e70 tx=0x7faa08004250 comp rx=0 tx=0).stop 2024-04-24T09:10:25.736 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.735+0000 7faa267ef740 1 -- 172.21.15.83:0/3465781632 shutdown_connections 2024-04-24T09:10:25.736 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.735+0000 7faa267ef740 1 --2- 172.21.15.83:0/3465781632 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x557df3baef30 0x557df3bb1350 crc :-1 s=READY pgs=470 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.736 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.735+0000 7faa267ef740 1 --2- 172.21.15.83:0/3465781632 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557df3b385a0 0x557df3b3c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.736 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.735+0000 7faa267ef740 1 --2- 172.21.15.83:0/3465781632 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557df3b2dce0 0x557df3b3bbb0 unknown :-1 s=CLOSED pgs=2446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.736 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.735+0000 7faa267ef740 1 --2- 172.21.15.83:0/3465781632 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557df3b2d3c0 0x557df3b406c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.735+0000 7faa267ef740 1 -- 172.21.15.83:0/3465781632 >> 172.21.15.83:0/3465781632 conn(0x557df3b298d0 msgr2=0x557df3b28200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.735+0000 7faa02a00640 0 client.22143 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:10:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.736+0000 7faa267ef740 1 -- 172.21.15.83:0/3465781632 shutdown_connections 2024-04-24T09:10:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.736+0000 7faa267ef740 1 -- 172.21.15.83:0/3465781632 wait complete. 2024-04-24T09:10:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.736+0000 7faa267ef740 20 client.22143 trim_cache size 0 max 16384 2024-04-24T09:10:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T09:10:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T09:10:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T09:10:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T09:10:25.738 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.737+0000 7faa22200640 1 -- 172.21.15.83:0/328851809 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7faa040a3d90 msgr2=0x7faa040a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:25.738 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.737+0000 7faa22200640 1 --2- 172.21.15.83:0/328851809 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7faa040a3d90 0x7faa040a6250 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x557df3b2acf0 tx=0x7faa080046c0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.738 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.737+0000 7faa22200640 1 -- 172.21.15.83:0/328851809 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa1c112e30 msgr2=0x7faa1c19eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:25.738 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.737+0000 7faa22200640 1 --2- 172.21.15.83:0/328851809 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa1c112e30 0x7faa1c19eb30 secure :-1 s=READY pgs=2439 cs=0 l=1 rev1=1 crypto rx=0x7faa1400e9d0 tx=0x7faa1400ee90 comp rx=0 tx=0).stop 2024-04-24T09:10:25.738 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.737+0000 7faa22200640 1 -- 172.21.15.83:0/328851809 shutdown_connections 2024-04-24T09:10:25.738 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.737+0000 7faa22200640 1 --2- 172.21.15.83:0/328851809 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7faa040a3d90 0x7faa040a6250 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.738 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.737+0000 7faa22200640 1 --2- 172.21.15.83:0/328851809 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa1c112e30 0x7faa1c19eb30 unknown :-1 s=CLOSED pgs=2439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.738 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.737+0000 7faa22200640 1 --2- 172.21.15.83:0/328851809 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa1c074cc0 0x7faa1c1a4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.738 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.737+0000 7faa22200640 1 --2- 172.21.15.83:0/328851809 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faa1c0743a0 0x7faa1c1a4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.738 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.737+0000 7faa22200640 1 -- 172.21.15.83:0/328851809 >> 172.21.15.83:0/328851809 conn(0x7faa1c0fd950 msgr2=0x7faa1c0fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:25.739 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.738+0000 7faa22200640 1 -- 172.21.15.83:0/328851809 shutdown_connections 2024-04-24T09:10:25.739 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:25.738+0000 7faa22200640 1 -- 172.21.15.83:0/328851809 wait complete. 2024-04-24T09:10:25.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.936+0000 7f4bc1e00640 1 -- 172.21.15.83:0/538363789 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4bbc076530 msgr2=0x7f4bbc076910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:25.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.936+0000 7f4bc1e00640 1 --2- 172.21.15.83:0/538363789 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4bbc076530 0x7f4bbc076910 secure :-1 s=READY pgs=2448 cs=0 l=1 rev1=1 crypto rx=0x7f4bb4009be0 tx=0x7f4bb402a6c0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.936+0000 7f4bc1e00640 1 -- 172.21.15.83:0/538363789 shutdown_connections 2024-04-24T09:10:25.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.936+0000 7f4bc1e00640 1 --2- 172.21.15.83:0/538363789 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4bbc112e80 0x7f4bbc115270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.936+0000 7f4bc1e00640 1 --2- 172.21.15.83:0/538363789 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4bbc076e50 0x7f4bbc112940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.936+0000 7f4bc1e00640 1 --2- 172.21.15.83:0/538363789 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4bbc076530 0x7f4bbc076910 unknown :-1 s=CLOSED pgs=2448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.936+0000 7f4bc1e00640 1 -- 172.21.15.83:0/538363789 >> 172.21.15.83:0/538363789 conn(0x7f4bbc0ffc70 msgr2=0x7f4bbc102090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:25.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.937+0000 7f4bc1e00640 1 -- 172.21.15.83:0/538363789 shutdown_connections 2024-04-24T09:10:25.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.937+0000 7f4bc1e00640 1 -- 172.21.15.83:0/538363789 wait complete. 2024-04-24T09:10:25.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.938+0000 7f4bc1e00640 1 Processor -- start 2024-04-24T09:10:25.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.938+0000 7f4bc1e00640 1 -- start start 2024-04-24T09:10:25.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.938+0000 7f4bc1e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4bbc076530 0x7f4bbc112500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:25.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.938+0000 7f4bc1e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4bbc076e50 0x7f4bbc10b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:25.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.938+0000 7f4bc1e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4bbc112e80 0x7f4bbc10bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:25.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.938+0000 7f4bc1e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4bbc118890 con 0x7f4bbc076530 2024-04-24T09:10:25.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.938+0000 7f4bc1e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4bbc118710 con 0x7f4bbc076e50 2024-04-24T09:10:25.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.938+0000 7f4bc1e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4bbc118a10 con 0x7f4bbc112e80 2024-04-24T09:10:25.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.939+0000 7f4bbaa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4bbc076530 0x7f4bbc112500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:25.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.939+0000 7f4bbb400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4bbc112e80 0x7f4bbc10bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:25.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.939+0000 7f4bbb400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4bbc112e80 0x7f4bbc10bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:51416/0 (socket says 172.21.15.83:51416) 2024-04-24T09:10:25.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.939+0000 7f4bbb400640 1 -- 172.21.15.83:0/4201007663 learned_addr learned my addr 172.21.15.83:0/4201007663 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:10:25.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.939+0000 7f4bba000640 1 --2- 172.21.15.83:0/4201007663 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4bbc076e50 0x7f4bbc10b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:25.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.940+0000 7f4bbaa00640 1 -- 172.21.15.83:0/4201007663 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4bbc112e80 msgr2=0x7f4bbc10bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:25.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.940+0000 7f4bbaa00640 1 --2- 172.21.15.83:0/4201007663 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4bbc112e80 0x7f4bbc10bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.940+0000 7f4bbaa00640 1 -- 172.21.15.83:0/4201007663 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4bbc076e50 msgr2=0x7f4bbc10b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:25.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.940+0000 7f4bbaa00640 1 --2- 172.21.15.83:0/4201007663 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4bbc076e50 0x7f4bbc10b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.940+0000 7f4bbaa00640 1 -- 172.21.15.83:0/4201007663 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bbc10c1d0 con 0x7f4bbc076530 2024-04-24T09:10:25.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.940+0000 7f4bbb400640 1 --2- 172.21.15.83:0/4201007663 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4bbc112e80 0x7f4bbc10bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:10:25.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.940+0000 7f4bbaa00640 1 --2- 172.21.15.83:0/4201007663 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4bbc076530 0x7f4bbc112500 secure :-1 s=READY pgs=2449 cs=0 l=1 rev1=1 crypto rx=0x7f4bb402abe0 tx=0x7f4bb4003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:25.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.940+0000 7f4bc0a00640 1 -- 172.21.15.83:0/4201007663 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb4041020 con 0x7f4bbc076530 2024-04-24T09:10:25.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.940+0000 7f4bc1e00640 1 -- 172.21.15.83:0/4201007663 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bbc10c460 con 0x7f4bbc076530 2024-04-24T09:10:25.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.940+0000 7f4bc0a00640 1 -- 172.21.15.83:0/4201007663 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4bb4037040 con 0x7f4bbc076530 2024-04-24T09:10:25.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.941+0000 7f4bc1e00640 1 -- 172.21.15.83:0/4201007663 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bbc110360 con 0x7f4bbc076530 2024-04-24T09:10:25.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.941+0000 7f4bc0a00640 1 -- 172.21.15.83:0/4201007663 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb4036070 con 0x7f4bbc076530 2024-04-24T09:10:25.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.944+0000 7f4bc4d72740 1 Processor -- start 2024-04-24T09:10:25.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.944+0000 7f4bc4d72740 1 -- start start 2024-04-24T09:10:25.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.944+0000 7f4bc4d72740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555fd9032950 0x555fd9032d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:25.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.944+0000 7f4bc4d72740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555fd9033270 0x555fd903d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:25.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.944+0000 7f4bc4d72740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555fd903d940 0x555fd903fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:25.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.944+0000 7f4bc4d72740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x555fd8f981d0 con 0x555fd9033270 2024-04-24T09:10:25.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.944+0000 7f4bc4d72740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555fd8f3e450 con 0x555fd9032950 2024-04-24T09:10:25.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.944+0000 7f4bc4d72740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x555fd8f94190 con 0x555fd903d940 2024-04-24T09:10:25.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.945+0000 7f4bbb400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555fd9032950 0x555fd9032d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:25.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.945+0000 7f4bbb400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555fd9032950 0x555fd9032d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:42022/0 (socket says 172.21.15.83:42022) 2024-04-24T09:10:25.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.945+0000 7f4bbb400640 1 -- 172.21.15.83:0/3443482247 learned_addr learned my addr 172.21.15.83:0/3443482247 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:10:25.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.945+0000 7f4bba000640 1 --2- 172.21.15.83:0/3443482247 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555fd903d940 0x555fd903fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:25.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.945+0000 7f4bbb400640 1 -- 172.21.15.83:0/3443482247 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555fd903d940 msgr2=0x555fd903fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:25.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.945+0000 7f4bbb400640 1 --2- 172.21.15.83:0/3443482247 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555fd903d940 0x555fd903fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.945+0000 7f4bbb400640 1 -- 172.21.15.83:0/3443482247 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555fd9033270 msgr2=0x555fd903d400 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T09:10:25.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.945+0000 7f4bbb400640 1 --2- 172.21.15.83:0/3443482247 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555fd9033270 0x555fd903d400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.945+0000 7f4bbb400640 1 -- 172.21.15.83:0/3443482247 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555fd8ffb310 con 0x555fd9032950 2024-04-24T09:10:25.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.945+0000 7f4bba000640 1 --2- 172.21.15.83:0/3443482247 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555fd903d940 0x555fd903fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:10:25.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.946+0000 7f4bbb400640 1 --2- 172.21.15.83:0/3443482247 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555fd9032950 0x555fd9032d30 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f4bac010480 tx=0x7f4bac00bf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=518a96c9fd3a4ea1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:25.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.947+0000 7f4ba7e00640 1 -- 172.21.15.83:0/3443482247 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bac011820 con 0x555fd9032950 2024-04-24T09:10:25.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.947+0000 7f4ba7e00640 1 -- 172.21.15.83:0/3443482247 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4bac01a380 con 0x555fd9032950 2024-04-24T09:10:25.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.947+0000 7f4ba7e00640 1 -- 172.21.15.83:0/3443482247 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bac01b640 con 0x555fd9032950 2024-04-24T09:10:25.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.947+0000 7f4bc0a00640 1 -- 172.21.15.83:0/4201007663 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4bb4044020 con 0x7f4bbc076530 2024-04-24T09:10:25.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.948+0000 7f4bc4d72740 1 -- 172.21.15.83:0/3443482247 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555fd9032950 msgr2=0x555fd9032d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:25.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.948+0000 7f4bc4d72740 1 --2- 172.21.15.83:0/3443482247 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555fd9032950 0x555fd9032d30 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f4bac010480 tx=0x7f4bac00bf90 comp rx=0 tx=0).stop 2024-04-24T09:10:25.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.948+0000 7f4bc4d72740 1 -- 172.21.15.83:0/3443482247 shutdown_connections 2024-04-24T09:10:25.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.948+0000 7f4bc4d72740 1 --2- 172.21.15.83:0/3443482247 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555fd903d940 0x555fd903fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.948+0000 7f4bc4d72740 1 --2- 172.21.15.83:0/3443482247 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555fd9033270 0x555fd903d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.948+0000 7f4bc4d72740 1 --2- 172.21.15.83:0/3443482247 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555fd9032950 0x555fd9032d30 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.948+0000 7f4bc4d72740 1 -- 172.21.15.83:0/3443482247 >> 172.21.15.83:0/3443482247 conn(0x555fd902ec70 msgr2=0x555fd902f050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:25.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.948+0000 7f4bc4d72740 1 -- 172.21.15.83:0/3443482247 shutdown_connections 2024-04-24T09:10:25.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.948+0000 7f4bc4d72740 1 -- 172.21.15.83:0/3443482247 wait complete. 2024-04-24T09:10:25.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.949+0000 7f4bc4d72740 1 -- start start 2024-04-24T09:10:25.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.950+0000 7f4bc4d72740 1 Processor -- start 2024-04-24T09:10:25.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.950+0000 7f4bc0a00640 1 --2- 172.21.15.83:0/4201007663 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f4ba00a3c70 0x7f4ba00a6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:25.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.951+0000 7f4bc0a00640 1 -- 172.21.15.83:0/4201007663 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 9583+0+0 (secure 0 0 0) 0x7f4bb40f0fc0 con 0x7f4bbc076530 2024-04-24T09:10:25.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.951+0000 7f4bc4d72740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555fd9032950 0x555fd903a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:25.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.951+0000 7f4bbb400640 1 --2- 172.21.15.83:0/4201007663 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f4ba00a3c70 0x7f4ba00a6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:25.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.951+0000 7f4bc4d72740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555fd9033270 0x555fd903ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:25.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.951+0000 7f4bc4d72740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555fd903d940 0x555fd903b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:25.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.951+0000 7f4bbaa00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555fd9032950 0x555fd903a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:25.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.952+0000 7f4bbaa00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555fd9032950 0x555fd903a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:51432/0 (socket says 172.21.15.83:51432) 2024-04-24T09:10:25.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.952+0000 7f4bbaa00640 1 -- 172.21.15.83:0/2606480849 learned_addr learned my addr 172.21.15.83:0/2606480849 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:10:25.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.952+0000 7f4bc4d72740 1 -- 172.21.15.83:0/2606480849 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x555fd90407d0 con 0x555fd9033270 2024-04-24T09:10:25.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.952+0000 7f4bc4d72740 1 -- 172.21.15.83:0/2606480849 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555fd8f51810 con 0x555fd903d940 2024-04-24T09:10:25.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.952+0000 7f4bc4d72740 1 -- 172.21.15.83:0/2606480849 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x555fd9040950 con 0x555fd9032950 2024-04-24T09:10:25.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.952+0000 7f4bc4d72740 10 client.0 _ll_register_callbacks cb 0x555fd8f1f140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:10:25.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.952+0000 7f4bba000640 1 --2- 172.21.15.83:0/2606480849 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555fd9033270 0x555fd903ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:25.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.952+0000 7f4bbaa00640 1 -- 172.21.15.83:0/2606480849 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555fd903d940 msgr2=0x555fd903b180 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T09:10:25.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.952+0000 7f4bbaa00640 1 --2- 172.21.15.83:0/2606480849 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555fd903d940 0x555fd903b180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.952+0000 7f4bbaa00640 1 -- 172.21.15.83:0/2606480849 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555fd9033270 msgr2=0x555fd903ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:25.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.952+0000 7f4bbaa00640 1 --2- 172.21.15.83:0/2606480849 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555fd9033270 0x555fd903ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.952+0000 7f4bbaa00640 1 -- 172.21.15.83:0/2606480849 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555fd8f623f0 con 0x555fd9032950 2024-04-24T09:10:25.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.952+0000 7f4bba000640 1 --2- 172.21.15.83:0/2606480849 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555fd9033270 0x555fd903ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:10:25.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.953+0000 7f4bbb400640 1 --2- 172.21.15.83:0/4201007663 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f4ba00a3c70 0x7f4ba00a6130 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f4bac010450 tx=0x7f4bac013040 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:25.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.953+0000 7f4bbaa00640 1 --2- 172.21.15.83:0/2606480849 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555fd9032950 0x555fd903a700 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4bb4004af0 tx=0x7f4bb40bc740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9dd9031fad9067a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:25.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.953+0000 7f4ba4c00640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-04-24T09:10:25.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.953+0000 7f4ba4c00640 1 -- 172.21.15.83:0/2606480849 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb40bcb40 con 0x555fd9032950 2024-04-24T09:10:25.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.953+0000 7f4ba4c00640 1 -- 172.21.15.83:0/2606480849 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(120..120 src has 1..120) ==== 9583+0+0 (secure 0 0 0) 0x7f4bb40b91b0 con 0x555fd9032950 2024-04-24T09:10:25.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.953+0000 7f4bc4d72740 1 -- 172.21.15.83:0/2606480849 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4bac01a380 con 0x555fd9032950 2024-04-24T09:10:25.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.954+0000 7f4ba4c00640 1 -- 172.21.15.83:0/2606480849 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f4bb40c0ef0 con 0x555fd9032950 2024-04-24T09:10:25.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.954+0000 7f4bc4d72740 10 client.29170 fetch_fsmap learned FSMap version 61 2024-04-24T09:10:25.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.954+0000 7f4bc4d72740 1 -- 172.21.15.83:0/2606480849 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x555fd8caa3f0 con 0x555fd9032950 2024-04-24T09:10:25.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.954+0000 7f4ba4c00640 1 -- 172.21.15.83:0/2606480849 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4bac01a380 con 0x555fd9032950 2024-04-24T09:10:25.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.955+0000 7f4ba4c00640 1 -- 172.21.15.83:0/2606480849 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 61) ==== 8362+0+0 (secure 0 0 0) 0x7f4bb40c53b0 con 0x555fd9032950 2024-04-24T09:10:25.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.955+0000 7f4bc4d72740 10 client.29170 fetch_fsmap finished waiting for FSMap version 61 2024-04-24T09:10:25.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.955+0000 7f4bc4d72740 10 client.29170 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:10:25.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.955+0000 7f4bc4d72740 20 client.29170 populate_metadata read hostname 'smithi083' 2024-04-24T09:10:25.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.955+0000 7f4bc4d72740 1 --2- 172.21.15.83:0/2606480849 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x555fd904db90 0x555fd904ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:25.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.955+0000 7f4bc4d72740 4 client.29170 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:10:25.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.955+0000 7f4bc4d72740 1 -- 172.21.15.83:0/2606480849 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555fd8fa5bf0 con 0x555fd904db90 2024-04-24T09:10:25.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.955+0000 7f4bba000640 1 --2- 172.21.15.83:0/2606480849 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x555fd904db90 0x555fd904ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:25.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.957+0000 7f4bba000640 1 --2- 172.21.15.83:0/2606480849 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x555fd904db90 0x555fd904ffe0 crc :-1 s=READY pgs=471 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9e20de0887543f7 server_cookie=568f663fe6c7ade3 in_seq=0 out_seq=0 2024-04-24T09:10:25.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.957+0000 7f4ba4c00640 10 client.29170 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:10:25.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.958+0000 7f4ba4c00640 1 -- 172.21.15.83:0/2606480849 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555fd8fa5bf0 con 0x555fd904db90 2024-04-24T09:10:25.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.958+0000 7f4ba4c00640 10 client.29170 handle_command_reply: tid=0 2024-04-24T09:10:25.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.958+0000 7f4bc4d72740 1 client.29170 shutdown 2024-04-24T09:10:25.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.958+0000 7f4bc4d72740 1 -- 172.21.15.83:0/2606480849 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555fd9032950 msgr2=0x555fd903a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:25.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.958+0000 7f4bc4d72740 1 --2- 172.21.15.83:0/2606480849 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555fd9032950 0x555fd903a700 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4bb4004af0 tx=0x7f4bb40bc740 comp rx=0 tx=0).stop 2024-04-24T09:10:25.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.958+0000 7f4bc4d72740 1 -- 172.21.15.83:0/2606480849 shutdown_connections 2024-04-24T09:10:25.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.958+0000 7f4bc4d72740 1 --2- 172.21.15.83:0/2606480849 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x555fd904db90 0x555fd904ffe0 crc :-1 s=READY pgs=471 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.958+0000 7f4bc4d72740 1 --2- 172.21.15.83:0/2606480849 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555fd903d940 0x555fd903b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.958+0000 7f4bc4d72740 1 --2- 172.21.15.83:0/2606480849 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555fd9033270 0x555fd903ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.958+0000 7f4bc4d72740 1 --2- 172.21.15.83:0/2606480849 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555fd9032950 0x555fd903a700 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.958+0000 7f4bc4d72740 1 -- 172.21.15.83:0/2606480849 >> 172.21.15.83:0/2606480849 conn(0x555fd902ec70 msgr2=0x555fd9034970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:25.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.958+0000 7f4ba4c00640 0 client.29170 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:10:25.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.959+0000 7f4bc4d72740 1 -- 172.21.15.83:0/2606480849 shutdown_connections 2024-04-24T09:10:25.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.959+0000 7f4bc4d72740 1 -- 172.21.15.83:0/2606480849 wait complete. 2024-04-24T09:10:25.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.959+0000 7f4bc4d72740 20 client.29170 trim_cache size 0 max 16384 2024-04-24T09:10:25.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.978+0000 7f4bc4d72740 1 Processor -- start 2024-04-24T09:10:25.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.978+0000 7f4bc4d72740 1 -- start start 2024-04-24T09:10:25.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.978+0000 7f4bc4d72740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555fd9032950 0x555fd9049710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:25.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.978+0000 7f4bc4d72740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555fd9033270 0x555fd90350b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:25.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.979+0000 7f4bc4d72740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555fd903d940 0x555fd902dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:25.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.979+0000 7f4bc4d72740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4bbc1be3b0 con 0x555fd9032950 2024-04-24T09:10:25.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.979+0000 7f4bc4d72740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4bbc1bfef0 con 0x555fd9033270 2024-04-24T09:10:25.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.979+0000 7f4bc4d72740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4bbc1bea80 con 0x555fd903d940 2024-04-24T09:10:25.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.979+0000 7f4bba000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555fd9033270 0x555fd90350b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:25.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.979+0000 7f4bba000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555fd9033270 0x555fd90350b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:42046/0 (socket says 172.21.15.83:42046) 2024-04-24T09:10:25.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.979+0000 7f4bba000640 1 -- 172.21.15.83:0/3466346980 learned_addr learned my addr 172.21.15.83:0/3466346980 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:10:25.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.979+0000 7f4bbaa00640 1 --2- 172.21.15.83:0/3466346980 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555fd9032950 0x555fd9049710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:25.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.980+0000 7f4bbb400640 1 --2- 172.21.15.83:0/3466346980 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555fd903d940 0x555fd902dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:25.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.980+0000 7f4bba000640 1 -- 172.21.15.83:0/3466346980 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555fd903d940 msgr2=0x555fd902dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:25.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.980+0000 7f4bba000640 1 --2- 172.21.15.83:0/3466346980 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555fd903d940 0x555fd902dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.980+0000 7f4bba000640 1 -- 172.21.15.83:0/3466346980 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555fd9032950 msgr2=0x555fd9049710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:25.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.980+0000 7f4bba000640 1 --2- 172.21.15.83:0/3466346980 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555fd9032950 0x555fd9049710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.980+0000 7f4bba000640 1 -- 172.21.15.83:0/3466346980 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555fd8d747d0 con 0x555fd9033270 2024-04-24T09:10:25.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.980+0000 7f4bba000640 1 --2- 172.21.15.83:0/3466346980 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555fd9033270 0x555fd90350b0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f4ba80096d0 tx=0x7f4ba8009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2058cf90cc192e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:25.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.981+0000 7f4ba7400640 1 -- 172.21.15.83:0/3466346980 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ba80054d0 con 0x555fd9033270 2024-04-24T09:10:25.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.981+0000 7f4ba7400640 1 -- 172.21.15.83:0/3466346980 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4ba8004c60 con 0x555fd9033270 2024-04-24T09:10:25.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.981+0000 7f4ba7400640 1 -- 172.21.15.83:0/3466346980 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ba80035f0 con 0x555fd9033270 2024-04-24T09:10:25.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.982+0000 7f4bc4d72740 1 -- 172.21.15.83:0/3466346980 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555fd9033270 msgr2=0x555fd90350b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:25.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.982+0000 7f4bc4d72740 1 --2- 172.21.15.83:0/3466346980 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555fd9033270 0x555fd90350b0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f4ba80096d0 tx=0x7f4ba8009260 comp rx=0 tx=0).stop 2024-04-24T09:10:25.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.982+0000 7f4bc4d72740 1 -- 172.21.15.83:0/3466346980 shutdown_connections 2024-04-24T09:10:25.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.982+0000 7f4bc4d72740 1 --2- 172.21.15.83:0/3466346980 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555fd903d940 0x555fd902dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.982+0000 7f4bc4d72740 1 --2- 172.21.15.83:0/3466346980 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555fd9033270 0x555fd90350b0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.982+0000 7f4bc4d72740 1 --2- 172.21.15.83:0/3466346980 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555fd9032950 0x555fd9049710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.982+0000 7f4bc4d72740 1 -- 172.21.15.83:0/3466346980 >> 172.21.15.83:0/3466346980 conn(0x555fd902ec70 msgr2=0x555fd902d560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:25.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.982+0000 7f4bc4d72740 1 -- 172.21.15.83:0/3466346980 shutdown_connections 2024-04-24T09:10:25.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.982+0000 7f4bc4d72740 1 -- 172.21.15.83:0/3466346980 wait complete. 2024-04-24T09:10:25.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.983+0000 7f4bc4d72740 1 -- start start 2024-04-24T09:10:25.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.983+0000 7f4bc4d72740 1 Processor -- start 2024-04-24T09:10:25.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.984+0000 7f4bc4d72740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555fd9032950 0x555fd9037100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:25.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.984+0000 7f4bc4d72740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555fd9033270 0x555fd9037640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:25.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.984+0000 7f4bc4d72740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555fd903d940 0x555fd9044c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:25.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.984+0000 7f4bc4d72740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4bbc1bf820 con 0x555fd903d940 2024-04-24T09:10:25.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.984+0000 7f4bc4d72740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4bbc1bf150 con 0x555fd9032950 2024-04-24T09:10:25.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.984+0000 7f4bc4d72740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4bbc1bc3b0 con 0x555fd9033270 2024-04-24T09:10:25.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.984+0000 7f4bc4d72740 10 client.0 _ll_register_callbacks cb 0x7f4bb40b8e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:10:25.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.984+0000 7f4bbaa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555fd9032950 0x555fd9037100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:25.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.984+0000 7f4bbb400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555fd903d940 0x555fd9044c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:25.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.984+0000 7f4bbaa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555fd9032950 0x555fd9037100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:42058/0 (socket says 172.21.15.83:42058) 2024-04-24T09:10:25.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.984+0000 7f4bba000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555fd9033270 0x555fd9037640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:25.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.984+0000 7f4bbaa00640 1 -- 172.21.15.83:0/2949446693 learned_addr learned my addr 172.21.15.83:0/2949446693 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:10:25.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.985+0000 7f4bbaa00640 1 -- 172.21.15.83:0/2949446693 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555fd9033270 msgr2=0x555fd9037640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:25.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.985+0000 7f4bbaa00640 1 --2- 172.21.15.83:0/2949446693 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555fd9033270 0x555fd9037640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.985+0000 7f4bbaa00640 1 -- 172.21.15.83:0/2949446693 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555fd903d940 msgr2=0x555fd9044c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:25.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.985+0000 7f4bbaa00640 1 --2- 172.21.15.83:0/2949446693 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555fd903d940 0x555fd9044c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.985+0000 7f4bbaa00640 1 -- 172.21.15.83:0/2949446693 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555fd9045190 con 0x555fd9032950 2024-04-24T09:10:25.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.985+0000 7f4bbaa00640 1 --2- 172.21.15.83:0/2949446693 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555fd9032950 0x555fd9037100 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f4bb40c5720 tx=0x7f4bb402a6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=994fe01830c5b963 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:25.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.985+0000 7f4ba4c00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:10:25.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.985+0000 7f4ba4c00640 1 -- 172.21.15.83:0/2949446693 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb40bc910 con 0x555fd9032950 2024-04-24T09:10:25.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.985+0000 7f4ba4c00640 1 -- 172.21.15.83:0/2949446693 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(120..120 src has 1..120) ==== 9583+0+0 (secure 0 0 0) 0x7f4bb4004b40 con 0x555fd9032950 2024-04-24T09:10:25.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.985+0000 7f4bc4d72740 1 -- 172.21.15.83:0/2949446693 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ba8004c60 con 0x555fd9032950 2024-04-24T09:10:25.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.986+0000 7f4ba4c00640 1 -- 172.21.15.83:0/2949446693 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f4bb40c5530 con 0x555fd9032950 2024-04-24T09:10:25.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.986+0000 7f4bc4d72740 10 client.39242 fetch_fsmap learned FSMap version 61 2024-04-24T09:10:25.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.986+0000 7f4bc4d72740 1 -- 172.21.15.83:0/2949446693 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x555fd9046490 con 0x555fd9032950 2024-04-24T09:10:25.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.986+0000 7f4ba4c00640 1 -- 172.21.15.83:0/2949446693 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4ba8004c60 con 0x555fd9032950 2024-04-24T09:10:25.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.986+0000 7f4ba4c00640 1 -- 172.21.15.83:0/2949446693 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 61) ==== 8362+0+0 (secure 0 0 0) 0x7f4bb40c2080 con 0x555fd9032950 2024-04-24T09:10:25.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.986+0000 7f4bc4d72740 10 client.39242 fetch_fsmap finished waiting for FSMap version 61 2024-04-24T09:10:25.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.986+0000 7f4bc4d72740 10 client.39242 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:10:25.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.986+0000 7f4bc4d72740 20 client.39242 populate_metadata read hostname 'smithi083' 2024-04-24T09:10:25.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.986+0000 7f4bc4d72740 1 --2- 172.21.15.83:0/2949446693 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x555fd90b4480 0x555fd90b6870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:25.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.987+0000 7f4bc4d72740 4 client.39242 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:10:25.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.987+0000 7f4bc4d72740 1 -- 172.21.15.83:0/2949446693 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555fd8f9fa60 con 0x555fd90b4480 2024-04-24T09:10:25.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.987+0000 7f4bba000640 1 --2- 172.21.15.83:0/2949446693 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x555fd90b4480 0x555fd90b6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:25.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.988+0000 7f4bba000640 1 --2- 172.21.15.83:0/2949446693 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x555fd90b4480 0x555fd90b6870 crc :-1 s=READY pgs=472 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1256e883461ad4fa server_cookie=92d7c886ddff6e31 in_seq=0 out_seq=0 2024-04-24T09:10:25.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.988+0000 7f4ba4c00640 10 client.39242 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:10:25.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.988+0000 7f4ba4c00640 1 -- 172.21.15.83:0/2949446693 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555fd8f9fa60 con 0x555fd90b4480 2024-04-24T09:10:25.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.988+0000 7f4ba4c00640 10 client.39242 handle_command_reply: tid=0 2024-04-24T09:10:25.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.988+0000 7f4bc4d72740 1 client.39242 shutdown 2024-04-24T09:10:25.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.989+0000 7f4bc4d72740 1 -- 172.21.15.83:0/2949446693 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555fd9032950 msgr2=0x555fd9037100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:25.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.989+0000 7f4bc4d72740 1 --2- 172.21.15.83:0/2949446693 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555fd9032950 0x555fd9037100 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f4bb40c5720 tx=0x7f4bb402a6c0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.989+0000 7f4bc4d72740 1 -- 172.21.15.83:0/2949446693 shutdown_connections 2024-04-24T09:10:25.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.989+0000 7f4bc4d72740 1 --2- 172.21.15.83:0/2949446693 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x555fd90b4480 0x555fd90b6870 crc :-1 s=READY pgs=472 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.989+0000 7f4bc4d72740 1 --2- 172.21.15.83:0/2949446693 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555fd903d940 0x555fd9044c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.989+0000 7f4ba4c00640 0 client.39242 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:10:25.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.989+0000 7f4bc4d72740 1 --2- 172.21.15.83:0/2949446693 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555fd9033270 0x555fd9037640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.989+0000 7f4bc4d72740 1 --2- 172.21.15.83:0/2949446693 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555fd9032950 0x555fd9037100 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.989+0000 7f4bc4d72740 1 -- 172.21.15.83:0/2949446693 >> 172.21.15.83:0/2949446693 conn(0x555fd902ec70 msgr2=0x555fd902ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:25.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.989+0000 7f4bc4d72740 1 -- 172.21.15.83:0/2949446693 shutdown_connections 2024-04-24T09:10:25.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.989+0000 7f4bc4d72740 1 -- 172.21.15.83:0/2949446693 wait complete. 2024-04-24T09:10:25.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.989+0000 7f4bc4d72740 20 client.39242 trim_cache size 0 max 16384 2024-04-24T09:10:25.993 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:10:25.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.990+0000 7f4bc1e00640 1 -- 172.21.15.83:0/4201007663 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f4ba00a3c70 msgr2=0x7f4ba00a6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:25.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.990+0000 7f4bc1e00640 1 --2- 172.21.15.83:0/4201007663 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f4ba00a3c70 0x7f4ba00a6130 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f4bac010450 tx=0x7f4bac013040 comp rx=0 tx=0).stop 2024-04-24T09:10:25.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.990+0000 7f4bc1e00640 1 -- 172.21.15.83:0/4201007663 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4bbc076530 msgr2=0x7f4bbc112500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:25.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.990+0000 7f4bc1e00640 1 --2- 172.21.15.83:0/4201007663 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4bbc076530 0x7f4bbc112500 secure :-1 s=READY pgs=2449 cs=0 l=1 rev1=1 crypto rx=0x7f4bb402abe0 tx=0x7f4bb4003ce0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.991+0000 7f4bc1e00640 1 -- 172.21.15.83:0/4201007663 shutdown_connections 2024-04-24T09:10:25.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.991+0000 7f4bc1e00640 1 --2- 172.21.15.83:0/4201007663 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f4ba00a3c70 0x7f4ba00a6130 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.991+0000 7f4bc1e00640 1 --2- 172.21.15.83:0/4201007663 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4bbc112e80 0x7f4bbc10bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.991+0000 7f4bc1e00640 1 --2- 172.21.15.83:0/4201007663 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4bbc076e50 0x7f4bbc10b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.991+0000 7f4bc1e00640 1 --2- 172.21.15.83:0/4201007663 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4bbc076530 0x7f4bbc112500 unknown :-1 s=CLOSED pgs=2449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:25.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.991+0000 7f4bc1e00640 1 -- 172.21.15.83:0/4201007663 >> 172.21.15.83:0/4201007663 conn(0x7f4bbc0ffc70 msgr2=0x7f4bbc102060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:25.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.991+0000 7f4bc1e00640 1 -- 172.21.15.83:0/4201007663 shutdown_connections 2024-04-24T09:10:25.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:25.991+0000 7f4bc1e00640 1 -- 172.21.15.83:0/4201007663 wait complete. 2024-04-24T09:10:26.175 INFO:tasks.workunit.client.0.smithi083.stdout: 10 37615 6.74 MB/sec execute 177 sec latency 587.181 ms 2024-04-24T09:10:26.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:10:26 smithi083 ceph-mon[27907]: pgmap v593: 129 pgs: 129 active+clean; 643 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 206 op/s 2024-04-24T09:10:26.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:10:26 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:10:26.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:10:26 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:10:26.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:10:26 smithi138 ceph-mon[31441]: pgmap v593: 129 pgs: 129 active+clean; 643 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 206 op/s 2024-04-24T09:10:26.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:10:26 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:10:26.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:10:26 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:10:26.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:10:26 smithi189 ceph-mon[31791]: pgmap v593: 129 pgs: 129 active+clean; 643 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 206 op/s 2024-04-24T09:10:26.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:10:26 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:10:26.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:10:26 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:10:27.014 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T09:10:27.014 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T09:10:27.014 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T09:10:27.175 INFO:tasks.workunit.client.0.smithi083.stdout: 10 37714 6.71 MB/sec execute 178 sec latency 79.723 ms 2024-04-24T09:10:27.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.273+0000 7fb3af200640 1 -- 172.21.15.83:0/3212215877 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb3b0076530 msgr2=0x7fb3b0076910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:27.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.273+0000 7fb3af200640 1 --2- 172.21.15.83:0/3212215877 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb3b0076530 0x7fb3b0076910 secure :-1 s=READY pgs=2450 cs=0 l=1 rev1=1 crypto rx=0x7fb3a0009900 tx=0x7fb3a002a3f0 comp rx=0 tx=0).stop 2024-04-24T09:10:27.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.274+0000 7fb3af200640 1 -- 172.21.15.83:0/3212215877 shutdown_connections 2024-04-24T09:10:27.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.274+0000 7fb3af200640 1 --2- 172.21.15.83:0/3212215877 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3b0112d50 0x7fb3b0115140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:27.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.274+0000 7fb3af200640 1 --2- 172.21.15.83:0/3212215877 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb3b0076e50 0x7fb3b0112810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:27.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.274+0000 7fb3af200640 1 --2- 172.21.15.83:0/3212215877 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb3b0076530 0x7fb3b0076910 unknown :-1 s=CLOSED pgs=2450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:27.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.274+0000 7fb3af200640 1 -- 172.21.15.83:0/3212215877 >> 172.21.15.83:0/3212215877 conn(0x7fb3b00ffb40 msgr2=0x7fb3b0101f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:27.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.274+0000 7fb3af200640 1 -- 172.21.15.83:0/3212215877 shutdown_connections 2024-04-24T09:10:27.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.274+0000 7fb3af200640 1 -- 172.21.15.83:0/3212215877 wait complete. 2024-04-24T09:10:27.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.275+0000 7fb3af200640 1 Processor -- start 2024-04-24T09:10:27.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.276+0000 7fb3af200640 1 -- start start 2024-04-24T09:10:27.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.276+0000 7fb3af200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb3b0076530 0x7fb3b01a4950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:27.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.276+0000 7fb3af200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb3b0076e50 0x7fb3b01a4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:27.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.276+0000 7fb3af200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3b0112d50 0x7fb3b019eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:27.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.276+0000 7fb3af200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b01187a0 con 0x7fb3b0076e50 2024-04-24T09:10:27.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.276+0000 7fb3af200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b0118620 con 0x7fb3b0112d50 2024-04-24T09:10:27.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.276+0000 7fb3af200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b0118920 con 0x7fb3b0076530 2024-04-24T09:10:27.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.276+0000 7fb3ae800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3b0112d50 0x7fb3b019eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:27.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.276+0000 7fb3ae800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3b0112d50 0x7fb3b019eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:42076/0 (socket says 172.21.15.83:42076) 2024-04-24T09:10:27.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.276+0000 7fb3ae800640 1 -- 172.21.15.83:0/67490477 learned_addr learned my addr 172.21.15.83:0/67490477 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:10:27.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.277+0000 7fb3ad400640 1 --2- 172.21.15.83:0/67490477 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb3b0076e50 0x7fb3b01a4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:27.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.277+0000 7fb3ade00640 1 --2- 172.21.15.83:0/67490477 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb3b0076530 0x7fb3b01a4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:27.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.277+0000 7fb3ae800640 1 -- 172.21.15.83:0/67490477 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb3b0076530 msgr2=0x7fb3b01a4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:27.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.277+0000 7fb3ae800640 1 --2- 172.21.15.83:0/67490477 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb3b0076530 0x7fb3b01a4950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:27.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.277+0000 7fb3ae800640 1 -- 172.21.15.83:0/67490477 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb3b0076e50 msgr2=0x7fb3b01a4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:27.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.277+0000 7fb3ae800640 1 --2- 172.21.15.83:0/67490477 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb3b0076e50 0x7fb3b01a4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:27.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.277+0000 7fb3ae800640 1 -- 172.21.15.83:0/67490477 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3b019f280 con 0x7fb3b0112d50 2024-04-24T09:10:27.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.277+0000 7fb3ae800640 1 --2- 172.21.15.83:0/67490477 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3b0112d50 0x7fb3b019eb00 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb3a800eb40 tx=0x7fb3a800c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:27.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.278+0000 7fb3a6a00640 1 -- 172.21.15.83:0/67490477 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a8004b20 con 0x7fb3b0112d50 2024-04-24T09:10:27.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.278+0000 7fb3a6a00640 1 -- 172.21.15.83:0/67490477 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb3a8004cc0 con 0x7fb3b0112d50 2024-04-24T09:10:27.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.278+0000 7fb3af200640 1 -- 172.21.15.83:0/67490477 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3b019f570 con 0x7fb3b0112d50 2024-04-24T09:10:27.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.278+0000 7fb3a6a00640 1 -- 172.21.15.83:0/67490477 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a80107b0 con 0x7fb3b0112d50 2024-04-24T09:10:27.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.278+0000 7fb3af200640 1 -- 172.21.15.83:0/67490477 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3b01ab810 con 0x7fb3b0112d50 2024-04-24T09:10:27.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.282+0000 7fb3a6a00640 1 -- 172.21.15.83:0/67490477 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb3a8010950 con 0x7fb3b0112d50 2024-04-24T09:10:27.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.282+0000 7fb3b7988740 1 Processor -- start 2024-04-24T09:10:27.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.282+0000 7fb3b7988740 1 -- start start 2024-04-24T09:10:27.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.283+0000 7fb3b7988740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef9d0a0270 0x55ef9d0a0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:27.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.283+0000 7fb3b7988740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef9d0a0b90 0x55ef9d0aad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:27.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.283+0000 7fb3b7988740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef9d0ab260 0x55ef9d0ad650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:27.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.283+0000 7fb3b7988740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ef9d005e70 con 0x55ef9d0a0b90 2024-04-24T09:10:27.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.283+0000 7fb3b7988740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ef9cfac380 con 0x55ef9d0ab260 2024-04-24T09:10:27.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.283+0000 7fb3b7988740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55ef9d001e30 con 0x55ef9d0a0270 2024-04-24T09:10:27.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.283+0000 7fb3ade00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef9d0a0b90 0x55ef9d0aad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:27.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.283+0000 7fb3ad400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef9d0ab260 0x55ef9d0ad650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:27.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.283+0000 7fb3ad400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef9d0ab260 0x55ef9d0ad650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:42090/0 (socket says 172.21.15.83:42090) 2024-04-24T09:10:27.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.283+0000 7fb3ad400640 1 -- 172.21.15.83:0/2195993048 learned_addr learned my addr 172.21.15.83:0/2195993048 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:10:27.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.283+0000 7fb3ad400640 1 --2- 172.21.15.83:0/2195993048 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef9d0a0270 0x55ef9d0a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:27.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.284+0000 7fb3ad400640 1 -- 172.21.15.83:0/2195993048 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef9d0a0270 msgr2=0x55ef9d0a0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:27.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.284+0000 7fb3ad400640 1 --2- 172.21.15.83:0/2195993048 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef9d0a0270 0x55ef9d0a0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:27.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.284+0000 7fb3ad400640 1 -- 172.21.15.83:0/2195993048 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef9d0a0b90 msgr2=0x55ef9d0aad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:27.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.284+0000 7fb3ad400640 1 --2- 172.21.15.83:0/2195993048 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef9d0a0b90 0x55ef9d0aad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:27.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.284+0000 7fb3ad400640 1 -- 172.21.15.83:0/2195993048 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef9cde07e0 con 0x55ef9d0ab260 2024-04-24T09:10:27.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.284+0000 7fb3a6a00640 1 --2- 172.21.15.83:0/67490477 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb3940a3e70 0x7fb3940a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:27.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.284+0000 7fb3a6a00640 1 -- 172.21.15.83:0/67490477 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 9583+0+0 (secure 0 0 0) 0x7fb3a80d1300 con 0x7fb3b0112d50 2024-04-24T09:10:27.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.284+0000 7fb3ad400640 1 --2- 172.21.15.83:0/2195993048 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef9d0ab260 0x55ef9d0ad650 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb39c012770 tx=0x7fb39c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b472bb80df1ffc93 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:27.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.285+0000 7fb39b000640 1 -- 172.21.15.83:0/2195993048 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb39c012cf0 con 0x55ef9d0ab260 2024-04-24T09:10:27.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.285+0000 7fb3ade00640 1 --2- 172.21.15.83:0/67490477 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb3940a3e70 0x7fb3940a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:27.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.285+0000 7fb39b000640 1 -- 172.21.15.83:0/2195993048 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb39c010770 con 0x55ef9d0ab260 2024-04-24T09:10:27.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.285+0000 7fb39b000640 1 -- 172.21.15.83:0/2195993048 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb39c012cf0 con 0x55ef9d0ab260 2024-04-24T09:10:27.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.286+0000 7fb3b7988740 1 -- 172.21.15.83:0/2195993048 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef9d0ab260 msgr2=0x55ef9d0ad650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:27.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.286+0000 7fb3b7988740 1 --2- 172.21.15.83:0/2195993048 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef9d0ab260 0x55ef9d0ad650 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb39c012770 tx=0x7fb39c0046c0 comp rx=0 tx=0).stop 2024-04-24T09:10:27.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.287+0000 7fb3ade00640 1 --2- 172.21.15.83:0/67490477 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb3940a3e70 0x7fb3940a6330 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fb3a0000c00 tx=0x7fb3a0005b50 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:27.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.287+0000 7fb3b7988740 1 -- 172.21.15.83:0/2195993048 shutdown_connections 2024-04-24T09:10:27.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.287+0000 7fb3b7988740 1 --2- 172.21.15.83:0/2195993048 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef9d0ab260 0x55ef9d0ad650 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:27.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.287+0000 7fb3b7988740 1 --2- 172.21.15.83:0/2195993048 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef9d0a0b90 0x55ef9d0aad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:27.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.287+0000 7fb3b7988740 1 --2- 172.21.15.83:0/2195993048 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef9d0a0270 0x55ef9d0a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:27.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.287+0000 7fb3b7988740 1 -- 172.21.15.83:0/2195993048 >> 172.21.15.83:0/2195993048 conn(0x55ef9d09c590 msgr2=0x55ef9d09c970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:27.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.288+0000 7fb3b7988740 1 -- 172.21.15.83:0/2195993048 shutdown_connections 2024-04-24T09:10:27.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.288+0000 7fb3b7988740 1 -- 172.21.15.83:0/2195993048 wait complete. 2024-04-24T09:10:27.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.288+0000 7fb3b7988740 1 -- start start 2024-04-24T09:10:27.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.289+0000 7fb3b7988740 1 Processor -- start 2024-04-24T09:10:27.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.289+0000 7fb3b7988740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef9d0a0270 0x55ef9d0a81f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:27.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.289+0000 7fb3b7988740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef9d0a0b90 0x55ef9d0a8730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:27.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.290+0000 7fb3ade00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef9d0a0270 0x55ef9d0a81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:27.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.290+0000 7fb3ade00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef9d0a0270 0x55ef9d0a81f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:42100/0 (socket says 172.21.15.83:42100) 2024-04-24T09:10:27.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.290+0000 7fb3ade00640 1 -- 172.21.15.83:0/2754911885 learned_addr learned my addr 172.21.15.83:0/2754911885 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:10:27.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.290+0000 7fb3b7988740 1 --2- 172.21.15.83:0/2754911885 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef9d0ab260 0x55ef9d0a8c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:27.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.290+0000 7fb3ad400640 1 --2- 172.21.15.83:0/2754911885 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef9d0a0b90 0x55ef9d0a8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:27.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.290+0000 7fb3b7988740 1 -- 172.21.15.83:0/2754911885 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ef9d0ae050 con 0x55ef9d0a0b90 2024-04-24T09:10:27.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.291+0000 7fb3b7988740 1 -- 172.21.15.83:0/2754911885 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ef9cfbf660 con 0x55ef9d0a0270 2024-04-24T09:10:27.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.291+0000 7fb3b7988740 1 -- 172.21.15.83:0/2754911885 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55ef9d0ae1d0 con 0x55ef9d0ab260 2024-04-24T09:10:27.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.291+0000 7fb3b7988740 10 client.0 _ll_register_callbacks cb 0x55ef9cfadaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:10:27.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.291+0000 7fb3ae800640 1 --2- 172.21.15.83:0/2754911885 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef9d0ab260 0x55ef9d0a8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:27.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.291+0000 7fb3ade00640 1 -- 172.21.15.83:0/2754911885 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef9d0ab260 msgr2=0x55ef9d0a8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:27.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.291+0000 7fb3ade00640 1 --2- 172.21.15.83:0/2754911885 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef9d0ab260 0x55ef9d0a8c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:27.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.291+0000 7fb3ade00640 1 -- 172.21.15.83:0/2754911885 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef9d0a0b90 msgr2=0x55ef9d0a8730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:27.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.291+0000 7fb3ade00640 1 --2- 172.21.15.83:0/2754911885 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef9d0a0b90 0x55ef9d0a8730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:27.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.292+0000 7fb3ade00640 1 -- 172.21.15.83:0/2754911885 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef9ce40f70 con 0x55ef9d0a0270 2024-04-24T09:10:27.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.292+0000 7fb3ad400640 1 --2- 172.21.15.83:0/2754911885 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef9d0a0b90 0x55ef9d0a8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:10:27.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.292+0000 7fb3ade00640 1 --2- 172.21.15.83:0/2754911885 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef9d0a0270 0x55ef9d0a81f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb3a0033000 tx=0x7fb3a0009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=65b2e75572194fc7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:27.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.292+0000 7fb38f400640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:10:27.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.292+0000 7fb38f400640 1 -- 172.21.15.83:0/2754911885 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a004d070 con 0x55ef9d0a0270 2024-04-24T09:10:27.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.293+0000 7fb38f400640 1 -- 172.21.15.83:0/2754911885 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(120..120 src has 1..120) ==== 9583+0+0 (secure 0 0 0) 0x7fb3a00035f0 con 0x55ef9d0a0270 2024-04-24T09:10:27.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.293+0000 7fb3b7988740 1 -- 172.21.15.83:0/2754911885 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb39c010770 con 0x55ef9d0a0270 2024-04-24T09:10:27.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.293+0000 7fb38f400640 1 -- 172.21.15.83:0/2754911885 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7fb3a0057050 con 0x55ef9d0a0270 2024-04-24T09:10:27.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.293+0000 7fb3b7988740 10 client.39254 fetch_fsmap learned FSMap version 61 2024-04-24T09:10:27.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.293+0000 7fb3b7988740 1 -- 172.21.15.83:0/2754911885 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x55ef9ccd8170 con 0x55ef9d0a0270 2024-04-24T09:10:27.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.294+0000 7fb38f400640 1 -- 172.21.15.83:0/2754911885 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb39c010770 con 0x55ef9d0a0270 2024-04-24T09:10:27.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.294+0000 7fb38f400640 1 -- 172.21.15.83:0/2754911885 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 61) ==== 8362+0+0 (secure 0 0 0) 0x7fb3a00499b0 con 0x55ef9d0a0270 2024-04-24T09:10:27.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.294+0000 7fb3b7988740 10 client.39254 fetch_fsmap finished waiting for FSMap version 61 2024-04-24T09:10:27.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.294+0000 7fb3b7988740 10 client.39254 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:10:27.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.295+0000 7fb3b7988740 20 client.39254 populate_metadata read hostname 'smithi083' 2024-04-24T09:10:27.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.295+0000 7fb3b7988740 1 --2- 172.21.15.83:0/2754911885 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ef9d0bb690 0x55ef9d0bdb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:27.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.295+0000 7fb3b7988740 4 client.39254 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:10:27.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.295+0000 7fb3b7988740 1 -- 172.21.15.83:0/2754911885 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ef9d013890 con 0x55ef9d0bb690 2024-04-24T09:10:27.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.295+0000 7fb3ad400640 1 --2- 172.21.15.83:0/2754911885 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ef9d0bb690 0x55ef9d0bdb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:27.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.296+0000 7fb3ad400640 1 --2- 172.21.15.83:0/2754911885 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ef9d0bb690 0x55ef9d0bdb30 crc :-1 s=READY pgs=473 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee4e5e60fcf6b19e server_cookie=8cad037b32cd4e65 in_seq=0 out_seq=0 2024-04-24T09:10:27.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.296+0000 7fb38f400640 10 client.39254 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:10:27.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.298+0000 7fb38f400640 1 -- 172.21.15.83:0/2754911885 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ef9d013890 con 0x55ef9d0bb690 2024-04-24T09:10:27.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.298+0000 7fb38f400640 10 client.39254 handle_command_reply: tid=0 2024-04-24T09:10:27.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.298+0000 7fb3b7988740 1 client.39254 shutdown 2024-04-24T09:10:27.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.298+0000 7fb3b7988740 1 -- 172.21.15.83:0/2754911885 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef9d0a0270 msgr2=0x55ef9d0a81f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:27.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.298+0000 7fb3b7988740 1 --2- 172.21.15.83:0/2754911885 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef9d0a0270 0x55ef9d0a81f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb3a0033000 tx=0x7fb3a0009260 comp rx=0 tx=0).stop 2024-04-24T09:10:27.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.298+0000 7fb3b7988740 1 -- 172.21.15.83:0/2754911885 shutdown_connections 2024-04-24T09:10:27.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.298+0000 7fb3b7988740 1 --2- 172.21.15.83:0/2754911885 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ef9d0bb690 0x55ef9d0bdb30 crc :-1 s=READY pgs=473 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:27.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.298+0000 7fb3b7988740 1 --2- 172.21.15.83:0/2754911885 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef9d0ab260 0x55ef9d0a8c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:27.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.298+0000 7fb3b7988740 1 --2- 172.21.15.83:0/2754911885 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef9d0a0b90 0x55ef9d0a8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:27.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.298+0000 7fb3b7988740 1 --2- 172.21.15.83:0/2754911885 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef9d0a0270 0x55ef9d0a81f0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:27.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.298+0000 7fb3b7988740 1 -- 172.21.15.83:0/2754911885 >> 172.21.15.83:0/2754911885 conn(0x55ef9d09c590 msgr2=0x55ef9d09b6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:27.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.298+0000 7fb38f400640 0 client.39254 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:10:27.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.298+0000 7fb3b7988740 1 -- 172.21.15.83:0/2754911885 shutdown_connections 2024-04-24T09:10:27.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.299+0000 7fb3b7988740 1 -- 172.21.15.83:0/2754911885 wait complete. 2024-04-24T09:10:27.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.299+0000 7fb3b7988740 20 client.39254 trim_cache size 0 max 16384 2024-04-24T09:10:27.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.332+0000 7fb3b7988740 1 Processor -- start 2024-04-24T09:10:27.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.332+0000 7fb3b7988740 1 -- start start 2024-04-24T09:10:27.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.333+0000 7fb3b7988740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef9d0a0270 0x55ef9d0b6f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:27.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.333+0000 7fb3b7988740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef9d0a0b90 0x55ef9d0a29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:27.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.333+0000 7fb3b7988740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef9d0ab260 0x55ef9d09b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:27.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.333+0000 7fb3b7988740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b01b5a90 con 0x55ef9d0a0b90 2024-04-24T09:10:27.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.333+0000 7fb3b7988740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b01b75d0 con 0x55ef9d0a0270 2024-04-24T09:10:27.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.333+0000 7fb3b7988740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b01b6160 con 0x55ef9d0ab260 2024-04-24T09:10:27.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.333+0000 7fb3ade00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef9d0a0270 0x55ef9d0b6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:27.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.333+0000 7fb3ad400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef9d0a0b90 0x55ef9d0a29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:27.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.333+0000 7fb3ade00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef9d0a0270 0x55ef9d0b6f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:42114/0 (socket says 172.21.15.83:42114) 2024-04-24T09:10:27.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.333+0000 7fb3ade00640 1 -- 172.21.15.83:0/3091321498 learned_addr learned my addr 172.21.15.83:0/3091321498 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:10:27.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.333+0000 7fb3ae800640 1 --2- 172.21.15.83:0/3091321498 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef9d0ab260 0x55ef9d09b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:27.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.334+0000 7fb3ad400640 1 -- 172.21.15.83:0/3091321498 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef9d0ab260 msgr2=0x55ef9d09b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:27.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.334+0000 7fb3ad400640 1 --2- 172.21.15.83:0/3091321498 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef9d0ab260 0x55ef9d09b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:27.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.334+0000 7fb3ad400640 1 -- 172.21.15.83:0/3091321498 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef9d0a0270 msgr2=0x55ef9d0b6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:27.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.334+0000 7fb3ad400640 1 --2- 172.21.15.83:0/3091321498 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef9d0a0270 0x55ef9d0b6f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:27.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.334+0000 7fb3ad400640 1 -- 172.21.15.83:0/3091321498 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef9d0a6740 con 0x55ef9d0a0b90 2024-04-24T09:10:27.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.334+0000 7fb3ade00640 1 --2- 172.21.15.83:0/3091321498 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef9d0a0270 0x55ef9d0b6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:10:27.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.334+0000 7fb3ad400640 1 --2- 172.21.15.83:0/3091321498 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef9d0a0b90 0x55ef9d0a29d0 secure :-1 s=READY pgs=2451 cs=0 l=1 rev1=1 crypto rx=0x7fb39c00b3d0 tx=0x7fb39c0051d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a31f0b6809e4af5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:27.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.335+0000 7fb39a600640 1 -- 172.21.15.83:0/3091321498 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb39c005540 con 0x55ef9d0a0b90 2024-04-24T09:10:27.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.335+0000 7fb39a600640 1 -- 172.21.15.83:0/3091321498 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb39c004730 con 0x55ef9d0a0b90 2024-04-24T09:10:27.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.335+0000 7fb39a600640 1 -- 172.21.15.83:0/3091321498 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb39c004520 con 0x55ef9d0a0b90 2024-04-24T09:10:27.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.336+0000 7fb3b7988740 1 -- 172.21.15.83:0/3091321498 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef9d0a0b90 msgr2=0x55ef9d0a29d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:27.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.336+0000 7fb3b7988740 1 --2- 172.21.15.83:0/3091321498 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef9d0a0b90 0x55ef9d0a29d0 secure :-1 s=READY pgs=2451 cs=0 l=1 rev1=1 crypto rx=0x7fb39c00b3d0 tx=0x7fb39c0051d0 comp rx=0 tx=0).stop 2024-04-24T09:10:27.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.336+0000 7fb3b7988740 1 -- 172.21.15.83:0/3091321498 shutdown_connections 2024-04-24T09:10:27.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.336+0000 7fb3b7988740 1 --2- 172.21.15.83:0/3091321498 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef9d0ab260 0x55ef9d09b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:27.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.336+0000 7fb3b7988740 1 --2- 172.21.15.83:0/3091321498 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef9d0a0b90 0x55ef9d0a29d0 unknown :-1 s=CLOSED pgs=2451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:27.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.336+0000 7fb3b7988740 1 --2- 172.21.15.83:0/3091321498 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef9d0a0270 0x55ef9d0b6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:27.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.336+0000 7fb3b7988740 1 -- 172.21.15.83:0/3091321498 >> 172.21.15.83:0/3091321498 conn(0x55ef9d09c590 msgr2=0x55ef9d09ae60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:27.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.336+0000 7fb3b7988740 1 -- 172.21.15.83:0/3091321498 shutdown_connections 2024-04-24T09:10:27.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.336+0000 7fb3b7988740 1 -- 172.21.15.83:0/3091321498 wait complete. 2024-04-24T09:10:27.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.337+0000 7fb3b7988740 1 -- start start 2024-04-24T09:10:27.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.337+0000 7fb3b7988740 1 Processor -- start 2024-04-24T09:10:27.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.337+0000 7fb3b7988740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef9d0a0270 0x55ef9d0b3600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:27.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.337+0000 7fb3b7988740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef9d0a0b90 0x55ef9d0aea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:27.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.337+0000 7fb3b7988740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef9d0ab260 0x55ef9d0aef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:27.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.337+0000 7fb3b7988740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b01b6f00 con 0x55ef9d0ab260 2024-04-24T09:10:27.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.337+0000 7fb3b7988740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b01b6830 con 0x55ef9d0a0270 2024-04-24T09:10:27.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.337+0000 7fb3b7988740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b01b3a90 con 0x55ef9d0a0b90 2024-04-24T09:10:27.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.337+0000 7fb3b7988740 10 client.0 _ll_register_callbacks cb 0x7fb3a0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:10:27.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.338+0000 7fb3ae800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef9d0ab260 0x55ef9d0aef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:27.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.338+0000 7fb3ade00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef9d0a0270 0x55ef9d0b3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:27.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.338+0000 7fb3ae800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef9d0ab260 0x55ef9d0aef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34980/0 (socket says 172.21.15.83:34980) 2024-04-24T09:10:27.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.338+0000 7fb3ae800640 1 -- 172.21.15.83:0/4281034692 learned_addr learned my addr 172.21.15.83:0/4281034692 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:10:27.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.338+0000 7fb3ade00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef9d0a0270 0x55ef9d0b3600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:42126/0 (socket says 172.21.15.83:42126) 2024-04-24T09:10:27.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.338+0000 7fb3ad400640 1 --2- 172.21.15.83:0/4281034692 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef9d0a0b90 0x55ef9d0aea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:27.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.338+0000 7fb3ade00640 1 -- 172.21.15.83:0/4281034692 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef9d0a0b90 msgr2=0x55ef9d0aea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:27.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.338+0000 7fb3ade00640 1 --2- 172.21.15.83:0/4281034692 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef9d0a0b90 0x55ef9d0aea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:27.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.338+0000 7fb3ade00640 1 -- 172.21.15.83:0/4281034692 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef9d0ab260 msgr2=0x55ef9d0aef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:27.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.338+0000 7fb3ade00640 1 --2- 172.21.15.83:0/4281034692 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef9d0ab260 0x55ef9d0aef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:27.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.338+0000 7fb3ade00640 1 -- 172.21.15.83:0/4281034692 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef9d0af5b0 con 0x55ef9d0a0270 2024-04-24T09:10:27.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.338+0000 7fb3ae800640 1 --2- 172.21.15.83:0/4281034692 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef9d0ab260 0x55ef9d0aef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:10:27.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.339+0000 7fb3ade00640 1 --2- 172.21.15.83:0/4281034692 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef9d0a0270 0x55ef9d0b3600 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fb3a0004650 tx=0x7fb3a0004450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc260cc75b8c965a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:27.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.339+0000 7fb38f400640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:10:27.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.339+0000 7fb38f400640 1 -- 172.21.15.83:0/4281034692 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a004d040 con 0x55ef9d0a0270 2024-04-24T09:10:27.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.339+0000 7fb3b7988740 1 -- 172.21.15.83:0/4281034692 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb39c004730 con 0x55ef9d0a0270 2024-04-24T09:10:27.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.339+0000 7fb38f400640 1 -- 172.21.15.83:0/4281034692 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(120..120 src has 1..120) ==== 9583+0+0 (secure 0 0 0) 0x7fb3a003c9a0 con 0x55ef9d0a0270 2024-04-24T09:10:27.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.340+0000 7fb38f400640 1 -- 172.21.15.83:0/4281034692 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7fb3a0035040 con 0x55ef9d0a0270 2024-04-24T09:10:27.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.341+0000 7fb3b7988740 10 client.39260 fetch_fsmap learned FSMap version 61 2024-04-24T09:10:27.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.341+0000 7fb3b7988740 1 -- 172.21.15.83:0/4281034692 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x55ef9d121ac0 con 0x55ef9d0a0270 2024-04-24T09:10:27.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.341+0000 7fb38f400640 1 -- 172.21.15.83:0/4281034692 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb39c004730 con 0x55ef9d0a0270 2024-04-24T09:10:27.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.342+0000 7fb38f400640 1 -- 172.21.15.83:0/4281034692 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 61) ==== 8362+0+0 (secure 0 0 0) 0x7fb3a00035f0 con 0x55ef9d0a0270 2024-04-24T09:10:27.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.342+0000 7fb3b7988740 10 client.39260 fetch_fsmap finished waiting for FSMap version 61 2024-04-24T09:10:27.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.342+0000 7fb3b7988740 10 client.39260 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:10:27.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.342+0000 7fb3b7988740 20 client.39260 populate_metadata read hostname 'smithi083' 2024-04-24T09:10:27.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.342+0000 7fb3b7988740 1 --2- 172.21.15.83:0/4281034692 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ef9d122180 0x55ef9d1245a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:27.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.342+0000 7fb3b7988740 4 client.39260 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T09:10:27.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.342+0000 7fb3b7988740 1 -- 172.21.15.83:0/4281034692 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ef9d00d700 con 0x55ef9d122180 2024-04-24T09:10:27.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.342+0000 7fb3ad400640 1 --2- 172.21.15.83:0/4281034692 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ef9d122180 0x55ef9d1245a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:27.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.344+0000 7fb38f400640 10 client.39260 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:10:27.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.344+0000 7fb3ad400640 1 --2- 172.21.15.83:0/4281034692 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ef9d122180 0x55ef9d1245a0 crc :-1 s=READY pgs=474 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d47fa8b4c063bb3 server_cookie=16ba142b4a1c700a in_seq=0 out_seq=0 2024-04-24T09:10:27.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.345+0000 7fb38f400640 1 -- 172.21.15.83:0/4281034692 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ef9d00d700 con 0x55ef9d122180 2024-04-24T09:10:27.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.345+0000 7fb38f400640 10 client.39260 handle_command_reply: tid=0 2024-04-24T09:10:27.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.345+0000 7fb3b7988740 1 client.39260 shutdown 2024-04-24T09:10:27.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.346+0000 7fb3b7988740 1 -- 172.21.15.83:0/4281034692 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef9d0a0270 msgr2=0x55ef9d0b3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:27.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.346+0000 7fb3b7988740 1 --2- 172.21.15.83:0/4281034692 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef9d0a0270 0x55ef9d0b3600 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fb3a0004650 tx=0x7fb3a0004450 comp rx=0 tx=0).stop 2024-04-24T09:10:27.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.346+0000 7fb3b7988740 1 -- 172.21.15.83:0/4281034692 shutdown_connections 2024-04-24T09:10:27.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.346+0000 7fb3b7988740 1 --2- 172.21.15.83:0/4281034692 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ef9d122180 0x55ef9d1245a0 crc :-1 s=READY pgs=474 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:27.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.346+0000 7fb3b7988740 1 --2- 172.21.15.83:0/4281034692 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef9d0ab260 0x55ef9d0aef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:27.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.346+0000 7fb3b7988740 1 --2- 172.21.15.83:0/4281034692 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef9d0a0b90 0x55ef9d0aea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:27.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.346+0000 7fb3b7988740 1 --2- 172.21.15.83:0/4281034692 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef9d0a0270 0x55ef9d0b3600 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:27.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.346+0000 7fb3b7988740 1 -- 172.21.15.83:0/4281034692 >> 172.21.15.83:0/4281034692 conn(0x55ef9d09c590 msgr2=0x55ef9d09b6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:27.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.346+0000 7fb38f400640 0 client.39260 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:10:27.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.346+0000 7fb3b7988740 1 -- 172.21.15.83:0/4281034692 shutdown_connections 2024-04-24T09:10:27.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.347+0000 7fb3b7988740 1 -- 172.21.15.83:0/4281034692 wait complete. 2024-04-24T09:10:27.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.347+0000 7fb3b7988740 20 client.39260 trim_cache size 0 max 16384 2024-04-24T09:10:27.348 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T09:10:27.348 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-04-24T09:10:27.348 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "0dc76183-1de2-4c38-b2e1-7c98f33693f8", 2024-04-24T09:10:27.348 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-04-24T09:10:27.348 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T09:10:27.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.348+0000 7fb3af200640 1 -- 172.21.15.83:0/67490477 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb3940a3e70 msgr2=0x7fb3940a6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:27.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.348+0000 7fb3af200640 1 --2- 172.21.15.83:0/67490477 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb3940a3e70 0x7fb3940a6330 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fb3a0000c00 tx=0x7fb3a0005b50 comp rx=0 tx=0).stop 2024-04-24T09:10:27.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.348+0000 7fb3af200640 1 -- 172.21.15.83:0/67490477 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3b0112d50 msgr2=0x7fb3b019eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:27.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.348+0000 7fb3af200640 1 --2- 172.21.15.83:0/67490477 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3b0112d50 0x7fb3b019eb00 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb3a800eb40 tx=0x7fb3a800c2d0 comp rx=0 tx=0).stop 2024-04-24T09:10:27.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.348+0000 7fb3af200640 1 -- 172.21.15.83:0/67490477 shutdown_connections 2024-04-24T09:10:27.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.348+0000 7fb3af200640 1 --2- 172.21.15.83:0/67490477 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb3940a3e70 0x7fb3940a6330 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:27.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.348+0000 7fb3af200640 1 --2- 172.21.15.83:0/67490477 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3b0112d50 0x7fb3b019eb00 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:27.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.348+0000 7fb3af200640 1 --2- 172.21.15.83:0/67490477 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb3b0076e50 0x7fb3b01a4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:27.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.348+0000 7fb3af200640 1 --2- 172.21.15.83:0/67490477 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb3b0076530 0x7fb3b01a4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:27.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.348+0000 7fb3af200640 1 -- 172.21.15.83:0/67490477 >> 172.21.15.83:0/67490477 conn(0x7fb3b00ffb40 msgr2=0x7fb3b0101f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:27.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.348+0000 7fb3af200640 1 -- 172.21.15.83:0/67490477 shutdown_connections 2024-04-24T09:10:27.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.348+0000 7fb3af200640 1 -- 172.21.15.83:0/67490477 wait complete. 2024-04-24T09:10:27.371 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T09:10:27.371 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T09:10:27.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.633+0000 7f26b4c00640 1 -- 172.21.15.83:0/3162912521 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f26b0100570 msgr2=0x7f26b01009f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:27.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.633+0000 7f26b4c00640 1 --2- 172.21.15.83:0/3162912521 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f26b0100570 0x7f26b01009f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f269c009930 tx=0x7f269c02a440 comp rx=0 tx=0).stop 2024-04-24T09:10:27.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.633+0000 7f26b4c00640 1 -- 172.21.15.83:0/3162912521 shutdown_connections 2024-04-24T09:10:27.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.633+0000 7f26b4c00640 1 --2- 172.21.15.83:0/3162912521 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26b0100f30 0x7f26b010ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:27.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.633+0000 7f26b4c00640 1 --2- 172.21.15.83:0/3162912521 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f26b0100570 0x7f26b01009f0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:27.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.633+0000 7f26b4c00640 1 --2- 172.21.15.83:0/3162912521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26b0107000 0x7f26b01073e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:27.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.633+0000 7f26b4c00640 1 -- 172.21.15.83:0/3162912521 >> 172.21.15.83:0/3162912521 conn(0x7f26b0077500 msgr2=0x7f26b00fe880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:27.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.633+0000 7f26b4c00640 1 -- 172.21.15.83:0/3162912521 shutdown_connections 2024-04-24T09:10:27.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.633+0000 7f26b4c00640 1 -- 172.21.15.83:0/3162912521 wait complete. 2024-04-24T09:10:27.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.634+0000 7f26b4c00640 1 Processor -- start 2024-04-24T09:10:27.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.635+0000 7f26b4c00640 1 -- start start 2024-04-24T09:10:27.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.635+0000 7f26b4c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26b0100570 0x7f26b01a0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:27.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.635+0000 7f26b4c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f26b0100f30 0x7f26b01a0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:27.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.635+0000 7f26b4c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26b0107000 0x7f26b019a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:27.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.635+0000 7f26b4c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f26b01123e0 con 0x7f26b0107000 2024-04-24T09:10:27.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.635+0000 7f26b4c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f26b0112260 con 0x7f26b0100570 2024-04-24T09:10:27.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.635+0000 7f26b4c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f26b0112560 con 0x7f26b0100f30 2024-04-24T09:10:27.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.635+0000 7f26ade00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26b0100570 0x7f26b01a0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:27.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.635+0000 7f26ade00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26b0100570 0x7f26b01a0790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:42144/0 (socket says 172.21.15.83:42144) 2024-04-24T09:10:27.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.635+0000 7f26ade00640 1 -- 172.21.15.83:0/3428077160 learned_addr learned my addr 172.21.15.83:0/3428077160 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:10:27.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.635+0000 7f26ae800640 1 --2- 172.21.15.83:0/3428077160 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26b0107000 0x7f26b019a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:27.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.636+0000 7f26ad400640 1 --2- 172.21.15.83:0/3428077160 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f26b0100f30 0x7f26b01a0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:27.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.636+0000 7f26ade00640 1 -- 172.21.15.83:0/3428077160 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f26b0100f30 msgr2=0x7f26b01a0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:27.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.636+0000 7f26ade00640 1 --2- 172.21.15.83:0/3428077160 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f26b0100f30 0x7f26b01a0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:27.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.636+0000 7f26ade00640 1 -- 172.21.15.83:0/3428077160 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26b0107000 msgr2=0x7f26b019a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:27.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.636+0000 7f26ade00640 1 --2- 172.21.15.83:0/3428077160 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26b0107000 0x7f26b019a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:27.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.636+0000 7f26ade00640 1 -- 172.21.15.83:0/3428077160 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26b019b0c0 con 0x7f26b0100570 2024-04-24T09:10:27.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.636+0000 7f26ade00640 1 --2- 172.21.15.83:0/3428077160 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26b0100570 0x7f26b01a0790 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f26a000d950 tx=0x7f26a000de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:27.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.637+0000 7f26a6a00640 1 -- 172.21.15.83:0/3428077160 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a0017070 con 0x7f26b0100570 2024-04-24T09:10:27.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.637+0000 7f26b4c00640 1 -- 172.21.15.83:0/3428077160 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26b019b3b0 con 0x7f26b0100570 2024-04-24T09:10:27.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.637+0000 7f26a6a00640 1 -- 172.21.15.83:0/3428077160 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f26a0009c30 con 0x7f26b0100570 2024-04-24T09:10:27.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.637+0000 7f26a6a00640 1 -- 172.21.15.83:0/3428077160 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a000f620 con 0x7f26b0100570 2024-04-24T09:10:27.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.637+0000 7f26b4c00640 1 -- 172.21.15.83:0/3428077160 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26b01a7650 con 0x7f26b0100570 2024-04-24T09:10:27.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.641+0000 7f26a6a00640 1 -- 172.21.15.83:0/3428077160 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f26a00109a0 con 0x7f26b0100570 2024-04-24T09:10:27.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.641+0000 7f26b799c740 1 Processor -- start 2024-04-24T09:10:27.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.642+0000 7f26b799c740 1 -- start start 2024-04-24T09:10:27.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.642+0000 7f26b799c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557229bdf860 0x557229bdfc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:27.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.642+0000 7f26b799c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557229be0180 0x557229bea310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:27.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.642+0000 7f26b799c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557229bea850 0x557229becc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:27.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.642+0000 7f26b799c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x557229b451f0 con 0x557229be0180 2024-04-24T09:10:27.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.642+0000 7f26b799c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x557229aeb350 con 0x557229bea850 2024-04-24T09:10:27.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.642+0000 7f26b799c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x557229b411b0 con 0x557229bdf860 2024-04-24T09:10:27.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.643+0000 7f26ad400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557229bea850 0x557229becc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:27.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.643+0000 7f26ae800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557229bdf860 0x557229bdfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:27.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.643+0000 7f26ad400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557229bea850 0x557229becc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:42156/0 (socket says 172.21.15.83:42156) 2024-04-24T09:10:27.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.643+0000 7f26ad400640 1 -- 172.21.15.83:0/1289026218 learned_addr learned my addr 172.21.15.83:0/1289026218 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:10:27.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.643+0000 7f26ade00640 1 --2- 172.21.15.83:0/1289026218 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557229be0180 0x557229bea310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:27.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.643+0000 7f26ae800640 1 -- 172.21.15.83:0/1289026218 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557229bea850 msgr2=0x557229becc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:27.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.643+0000 7f26ae800640 1 --2- 172.21.15.83:0/1289026218 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557229bea850 0x557229becc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:27.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.643+0000 7f26ae800640 1 -- 172.21.15.83:0/1289026218 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557229be0180 msgr2=0x557229bea310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:27.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.643+0000 7f26ae800640 1 --2- 172.21.15.83:0/1289026218 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557229be0180 0x557229bea310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:27.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.643+0000 7f26ae800640 1 -- 172.21.15.83:0/1289026218 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557229b3ff50 con 0x557229bdf860 2024-04-24T09:10:27.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.643+0000 7f26ade00640 1 --2- 172.21.15.83:0/1289026218 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557229be0180 0x557229bea310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:10:27.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.643+0000 7f26ad400640 1 --2- 172.21.15.83:0/1289026218 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557229bea850 0x557229becc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:10:27.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.644+0000 7f26a6a00640 1 --2- 172.21.15.83:0/3428077160 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f26940a3d20 0x7f26940a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:27.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.644+0000 7f26a6a00640 1 -- 172.21.15.83:0/3428077160 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 9583+0+0 (secure 0 0 0) 0x7f26a00d07b0 con 0x7f26b0100570 2024-04-24T09:10:27.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.644+0000 7f26ae800640 1 --2- 172.21.15.83:0/1289026218 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557229bdf860 0x557229bdfc40 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f26a8009f70 tx=0x7f26a8009650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8af64b1a33e446 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:27.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.645+0000 7f269b000640 1 -- 172.21.15.83:0/1289026218 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a8013850 con 0x557229bdf860 2024-04-24T09:10:27.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.645+0000 7f26ade00640 1 --2- 172.21.15.83:0/3428077160 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f26940a3d20 0x7f26940a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:27.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.645+0000 7f269b000640 1 -- 172.21.15.83:0/1289026218 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f26a8019070 con 0x557229bdf860 2024-04-24T09:10:27.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.645+0000 7f269b000640 1 -- 172.21.15.83:0/1289026218 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a8013850 con 0x557229bdf860 2024-04-24T09:10:27.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.646+0000 7f26b799c740 1 -- 172.21.15.83:0/1289026218 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557229bdf860 msgr2=0x557229bdfc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:27.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.646+0000 7f26b799c740 1 --2- 172.21.15.83:0/1289026218 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557229bdf860 0x557229bdfc40 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f26a8009f70 tx=0x7f26a8009650 comp rx=0 tx=0).stop 2024-04-24T09:10:27.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.646+0000 7f26ade00640 1 --2- 172.21.15.83:0/3428077160 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f26940a3d20 0x7f26940a61e0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f26a00cdb90 tx=0x7f26a0098040 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:27.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.646+0000 7f26b799c740 1 -- 172.21.15.83:0/1289026218 shutdown_connections 2024-04-24T09:10:27.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.646+0000 7f26b799c740 1 --2- 172.21.15.83:0/1289026218 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557229bea850 0x557229becc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:27.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.646+0000 7f26b799c740 1 --2- 172.21.15.83:0/1289026218 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557229be0180 0x557229bea310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:27.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.646+0000 7f26b799c740 1 --2- 172.21.15.83:0/1289026218 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557229bdf860 0x557229bdfc40 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:27.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.646+0000 7f26b799c740 1 -- 172.21.15.83:0/1289026218 >> 172.21.15.83:0/1289026218 conn(0x557229bdbb80 msgr2=0x557229bdbf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:27.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.646+0000 7f26b799c740 1 -- 172.21.15.83:0/1289026218 shutdown_connections 2024-04-24T09:10:27.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.646+0000 7f26b799c740 1 -- 172.21.15.83:0/1289026218 wait complete. 2024-04-24T09:10:27.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.647+0000 7f26b799c740 1 -- start start 2024-04-24T09:10:27.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.647+0000 7f26b799c740 1 Processor -- start 2024-04-24T09:10:27.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.648+0000 7f26b799c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557229bdf860 0x557229be77f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:27.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.648+0000 7f26b799c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557229be0180 0x557229be7d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:27.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.648+0000 7f26b799c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557229bea850 0x557229be8270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:27.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.648+0000 7f26b799c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x557229bed640 con 0x557229be0180 2024-04-24T09:10:27.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.648+0000 7f26b799c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x557229afe690 con 0x557229bdf860 2024-04-24T09:10:27.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.648+0000 7f26b799c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x557229bed7c0 con 0x557229bea850 2024-04-24T09:10:27.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.648+0000 7f26b799c740 10 client.0 _ll_register_callbacks cb 0x557229b3a650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:10:27.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.649+0000 7f26ad400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557229be0180 0x557229be7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:27.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.649+0000 7f26ae800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557229bdf860 0x557229be77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:27.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.649+0000 7f26ad400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557229be0180 0x557229be7d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35040/0 (socket says 172.21.15.83:35040) 2024-04-24T09:10:27.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.649+0000 7f26ad400640 1 -- 172.21.15.83:0/1031861351 learned_addr learned my addr 172.21.15.83:0/1031861351 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:10:27.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.649+0000 7f26ae800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557229bdf860 0x557229be77f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:42168/0 (socket says 172.21.15.83:42168) 2024-04-24T09:10:27.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.649+0000 7f26ae800640 1 --2- 172.21.15.83:0/1031861351 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557229bea850 0x557229be8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:27.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.649+0000 7f26ad400640 1 -- 172.21.15.83:0/1031861351 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557229bea850 msgr2=0x557229be8270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:27.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.649+0000 7f26ad400640 1 --2- 172.21.15.83:0/1031861351 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557229bea850 0x557229be8270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:27.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.649+0000 7f26ad400640 1 -- 172.21.15.83:0/1031861351 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557229bdf860 msgr2=0x557229be77f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:27.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.649+0000 7f26ad400640 1 --2- 172.21.15.83:0/1031861351 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557229bdf860 0x557229be77f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:27.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.649+0000 7f26ad400640 1 -- 172.21.15.83:0/1031861351 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557229afd920 con 0x557229be0180 2024-04-24T09:10:27.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.650+0000 7f26ae800640 1 --2- 172.21.15.83:0/1031861351 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557229bdf860 0x557229be77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:10:27.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.650+0000 7f26ad400640 1 --2- 172.21.15.83:0/1031861351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557229be0180 0x557229be7d30 secure :-1 s=READY pgs=2452 cs=0 l=1 rev1=1 crypto rx=0x7f269c004590 tx=0x7f269c0045c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abdc0db9c8a4a198 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:27.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.650+0000 7f268f400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:10:27.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.651+0000 7f268f400640 1 -- 172.21.15.83:0/1031861351 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f269c035040 con 0x557229be0180 2024-04-24T09:10:27.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.651+0000 7f268f400640 1 -- 172.21.15.83:0/1031861351 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(120..120 src has 1..120) ==== 9583+0+0 (secure 0 0 0) 0x7f269c005230 con 0x557229be0180 2024-04-24T09:10:27.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.652+0000 7f26b799c740 1 -- 172.21.15.83:0/1031861351 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26a8019070 con 0x557229be0180 2024-04-24T09:10:27.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.654+0000 7f268f400640 1 -- 172.21.15.83:0/1031861351 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f269c04b050 con 0x557229be0180 2024-04-24T09:10:27.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.654+0000 7f26b799c740 10 client.22185 fetch_fsmap learned FSMap version 61 2024-04-24T09:10:27.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.654+0000 7f26b799c740 1 -- 172.21.15.83:0/1031861351 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x5572299217d0 con 0x557229be0180 2024-04-24T09:10:27.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.656+0000 7f268f400640 1 -- 172.21.15.83:0/1031861351 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f26a8019070 con 0x557229be0180 2024-04-24T09:10:27.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.657+0000 7f268f400640 1 -- 172.21.15.83:0/1031861351 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 61) ==== 8362+0+0 (secure 0 0 0) 0x7f269c004880 con 0x557229be0180 2024-04-24T09:10:27.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.657+0000 7f26b799c740 10 client.22185 fetch_fsmap finished waiting for FSMap version 61 2024-04-24T09:10:27.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.657+0000 7f26b799c740 10 client.22185 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:10:27.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.657+0000 7f26b799c740 20 client.22185 populate_metadata read hostname 'smithi083' 2024-04-24T09:10:27.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.657+0000 7f26b799c740 1 --2- 172.21.15.83:0/1031861351 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x557229c0bb70 0x557229c0df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:27.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.657+0000 7f26b799c740 4 client.22185 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:10:27.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.657+0000 7f26b799c740 1 -- 172.21.15.83:0/1031861351 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557229b52c10 con 0x557229c0bb70 2024-04-24T09:10:27.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.659+0000 7f26ae800640 1 --2- 172.21.15.83:0/1031861351 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x557229c0bb70 0x557229c0df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:27.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.660+0000 7f26ae800640 1 --2- 172.21.15.83:0/1031861351 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x557229c0bb70 0x557229c0df60 crc :-1 s=READY pgs=475 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20b201858a3c65ed server_cookie=c5e8d5205a6056be in_seq=0 out_seq=0 2024-04-24T09:10:27.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.660+0000 7f268f400640 10 client.22185 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:10:27.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.662+0000 7f268f400640 1 -- 172.21.15.83:0/1031861351 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557229b52c10 con 0x557229c0bb70 2024-04-24T09:10:27.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.662+0000 7f268f400640 10 client.22185 handle_command_reply: tid=0 2024-04-24T09:10:27.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.663+0000 7f26b799c740 1 client.22185 shutdown 2024-04-24T09:10:27.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.663+0000 7f26b799c740 1 -- 172.21.15.83:0/1031861351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557229be0180 msgr2=0x557229be7d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:27.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.663+0000 7f26b799c740 1 --2- 172.21.15.83:0/1031861351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557229be0180 0x557229be7d30 secure :-1 s=READY pgs=2452 cs=0 l=1 rev1=1 crypto rx=0x7f269c004590 tx=0x7f269c0045c0 comp rx=0 tx=0).stop 2024-04-24T09:10:27.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.665+0000 7f26b799c740 1 -- 172.21.15.83:0/1031861351 shutdown_connections 2024-04-24T09:10:27.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.665+0000 7f26b799c740 1 --2- 172.21.15.83:0/1031861351 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x557229c0bb70 0x557229c0df60 crc :-1 s=READY pgs=475 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:27.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.665+0000 7f26b799c740 1 --2- 172.21.15.83:0/1031861351 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557229bea850 0x557229be8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:27.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.665+0000 7f26b799c740 1 --2- 172.21.15.83:0/1031861351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557229be0180 0x557229be7d30 unknown :-1 s=CLOSED pgs=2452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:27.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.665+0000 7f26b799c740 1 --2- 172.21.15.83:0/1031861351 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557229bdf860 0x557229be77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:27.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.665+0000 7f26b799c740 1 -- 172.21.15.83:0/1031861351 >> 172.21.15.83:0/1031861351 conn(0x557229bdbb80 msgr2=0x557229be1980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:27.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.666+0000 7f268f400640 0 client.22185 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:10:27.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.668+0000 7f26b799c740 1 -- 172.21.15.83:0/1031861351 shutdown_connections 2024-04-24T09:10:27.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.668+0000 7f26b799c740 1 -- 172.21.15.83:0/1031861351 wait complete. 2024-04-24T09:10:27.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.668+0000 7f26b799c740 20 client.22185 trim_cache size 0 max 16384 2024-04-24T09:10:27.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.705+0000 7f26b799c740 1 Processor -- start 2024-04-24T09:10:27.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.706+0000 7f26b799c740 1 -- start start 2024-04-24T09:10:27.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.706+0000 7f26b799c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557229bdf860 0x557229bf6500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:27.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.706+0000 7f26b799c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557229be0180 0x557229be20d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:27.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.706+0000 7f26b799c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557229bea850 0x557229be9730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:27.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.706+0000 7f26b799c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f26b01a9290 con 0x557229bea850 2024-04-24T09:10:27.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.706+0000 7f26b799c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f26b01aadd0 con 0x557229be0180 2024-04-24T09:10:27.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.706+0000 7f26b799c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f26b01a9960 con 0x557229bdf860 2024-04-24T09:10:27.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.706+0000 7f26ad400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557229be0180 0x557229be20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:27.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.707+0000 7f26ae800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557229bdf860 0x557229bf6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:27.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.707+0000 7f26ae800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557229bdf860 0x557229bf6500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:51564/0 (socket says 172.21.15.83:51564) 2024-04-24T09:10:27.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.707+0000 7f26ad400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557229be0180 0x557229be20d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:42184/0 (socket says 172.21.15.83:42184) 2024-04-24T09:10:27.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.707+0000 7f26ae800640 1 -- 172.21.15.83:0/678908296 learned_addr learned my addr 172.21.15.83:0/678908296 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:10:27.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.707+0000 7f26ae800640 1 --2- 172.21.15.83:0/678908296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557229bea850 0x557229be9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:27.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.707+0000 7f26ad400640 1 -- 172.21.15.83:0/678908296 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557229bdf860 msgr2=0x557229bf6500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:27.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.707+0000 7f26ad400640 1 --2- 172.21.15.83:0/678908296 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557229bdf860 0x557229bf6500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:27.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.708+0000 7f26ad400640 1 -- 172.21.15.83:0/678908296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557229bea850 msgr2=0x557229be9730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:27.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.708+0000 7f26ad400640 1 --2- 172.21.15.83:0/678908296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557229bea850 0x557229be9730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:27.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.708+0000 7f26ad400640 1 -- 172.21.15.83:0/678908296 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572298573f0 con 0x557229be0180 2024-04-24T09:10:27.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.708+0000 7f26ae800640 1 --2- 172.21.15.83:0/678908296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557229bea850 0x557229be9730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:10:27.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.709+0000 7f26ad400640 1 --2- 172.21.15.83:0/678908296 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557229be0180 0x557229be20d0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f269c009930 tx=0x7f269c001e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9b07216b7662f6f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:27.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.709+0000 7f269a600640 1 -- 172.21.15.83:0/678908296 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f269c035070 con 0x557229be0180 2024-04-24T09:10:27.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.709+0000 7f269a600640 1 -- 172.21.15.83:0/678908296 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f269c005000 con 0x557229be0180 2024-04-24T09:10:27.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.709+0000 7f269a600640 1 -- 172.21.15.83:0/678908296 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f269c046bd0 con 0x557229be0180 2024-04-24T09:10:27.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.710+0000 7f26b799c740 1 -- 172.21.15.83:0/678908296 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557229be0180 msgr2=0x557229be20d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:27.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.710+0000 7f26b799c740 1 --2- 172.21.15.83:0/678908296 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557229be0180 0x557229be20d0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f269c009930 tx=0x7f269c001e70 comp rx=0 tx=0).stop 2024-04-24T09:10:27.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.711+0000 7f26b799c740 1 -- 172.21.15.83:0/678908296 shutdown_connections 2024-04-24T09:10:27.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.711+0000 7f26b799c740 1 --2- 172.21.15.83:0/678908296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557229bea850 0x557229be9730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:27.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.711+0000 7f26b799c740 1 --2- 172.21.15.83:0/678908296 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557229be0180 0x557229be20d0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:27.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.711+0000 7f26b799c740 1 --2- 172.21.15.83:0/678908296 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557229bdf860 0x557229bf6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:27.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.711+0000 7f26b799c740 1 -- 172.21.15.83:0/678908296 >> 172.21.15.83:0/678908296 conn(0x557229bdbb80 msgr2=0x557229be9020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:27.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.711+0000 7f26b799c740 1 -- 172.21.15.83:0/678908296 shutdown_connections 2024-04-24T09:10:27.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.711+0000 7f26b799c740 1 -- 172.21.15.83:0/678908296 wait complete. 2024-04-24T09:10:27.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.711+0000 7f26b799c740 1 -- start start 2024-04-24T09:10:27.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.712+0000 7f26b799c740 1 Processor -- start 2024-04-24T09:10:27.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.713+0000 7f26b799c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557229bdf860 0x557229bf2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:27.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.713+0000 7f26b799c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557229be0180 0x557229bf2f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:27.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.713+0000 7f26b799c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557229bea850 0x557229be3e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:27.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.713+0000 7f26b799c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f26b01aa700 con 0x557229be0180 2024-04-24T09:10:27.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.713+0000 7f26b799c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f26b01aa030 con 0x557229bea850 2024-04-24T09:10:27.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.713+0000 7f26b799c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f26b019be70 con 0x557229bdf860 2024-04-24T09:10:27.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.713+0000 7f26b799c740 10 client.0 _ll_register_callbacks cb 0x7f269c009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:10:27.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.714+0000 7f26ad400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557229be0180 0x557229bf2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:27.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.714+0000 7f26ad400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557229be0180 0x557229bf2f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35054/0 (socket says 172.21.15.83:35054) 2024-04-24T09:10:27.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.714+0000 7f26ad400640 1 -- 172.21.15.83:0/1227343009 learned_addr learned my addr 172.21.15.83:0/1227343009 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:10:27.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.714+0000 7f26ae800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557229bdf860 0x557229bf2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:27.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.714+0000 7f26ae800640 1 --2- 172.21.15.83:0/1227343009 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557229bea850 0x557229be3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:27.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.714+0000 7f26ad400640 1 -- 172.21.15.83:0/1227343009 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557229bdf860 msgr2=0x557229bf2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:27.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.714+0000 7f26ad400640 1 --2- 172.21.15.83:0/1227343009 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557229bdf860 0x557229bf2a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:27.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.714+0000 7f26ad400640 1 -- 172.21.15.83:0/1227343009 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557229bea850 msgr2=0x557229be3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:27.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.714+0000 7f26ad400640 1 --2- 172.21.15.83:0/1227343009 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557229bea850 0x557229be3e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:27.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.714+0000 7f26ad400640 1 -- 172.21.15.83:0/1227343009 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572298170f0 con 0x557229be0180 2024-04-24T09:10:27.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.714+0000 7f26ae800640 1 --2- 172.21.15.83:0/1227343009 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557229bea850 0x557229be3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:10:27.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.715+0000 7f26ad400640 1 --2- 172.21.15.83:0/1227343009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557229be0180 0x557229bf2f50 secure :-1 s=READY pgs=2453 cs=0 l=1 rev1=1 crypto rx=0x7f269c009850 tx=0x7f269c005a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff573f28f48e14e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:27.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.715+0000 7f268f400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:10:27.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.715+0000 7f268f400640 1 -- 172.21.15.83:0/1227343009 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f269c035070 con 0x557229be0180 2024-04-24T09:10:27.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.715+0000 7f268f400640 1 -- 172.21.15.83:0/1227343009 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(120..120 src has 1..120) ==== 9583+0+0 (secure 0 0 0) 0x7f269c046d30 con 0x557229be0180 2024-04-24T09:10:27.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.716+0000 7f26b799c740 1 -- 172.21.15.83:0/1227343009 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f269c005000 con 0x557229be0180 2024-04-24T09:10:27.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.716+0000 7f268f400640 1 -- 172.21.15.83:0/1227343009 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f269c046460 con 0x557229be0180 2024-04-24T09:10:27.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.716+0000 7f26b799c740 10 client.22191 fetch_fsmap learned FSMap version 61 2024-04-24T09:10:27.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.717+0000 7f26b799c740 1 -- 172.21.15.83:0/1227343009 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x557229b0a810 con 0x557229be0180 2024-04-24T09:10:27.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.717+0000 7f268f400640 1 -- 172.21.15.83:0/1227343009 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f269c005000 con 0x557229be0180 2024-04-24T09:10:27.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.718+0000 7f268f400640 1 -- 172.21.15.83:0/1227343009 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 61) ==== 8362+0+0 (secure 0 0 0) 0x7f269c0038c0 con 0x557229be0180 2024-04-24T09:10:27.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.718+0000 7f26b799c740 10 client.22191 fetch_fsmap finished waiting for FSMap version 61 2024-04-24T09:10:27.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.718+0000 7f26b799c740 10 client.22191 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:10:27.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.718+0000 7f26b799c740 20 client.22191 populate_metadata read hostname 'smithi083' 2024-04-24T09:10:27.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.718+0000 7f26b799c740 1 --2- 172.21.15.83:0/1227343009 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x557229c71e70 0x557229c74290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:27.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.718+0000 7f26b799c740 4 client.22191 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T09:10:27.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.718+0000 7f26b799c740 1 -- 172.21.15.83:0/1227343009 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557229b4ca80 con 0x557229c71e70 2024-04-24T09:10:27.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.719+0000 7f26ae800640 1 --2- 172.21.15.83:0/1227343009 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x557229c71e70 0x557229c74290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:27.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.720+0000 7f26ae800640 1 --2- 172.21.15.83:0/1227343009 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x557229c71e70 0x557229c74290 crc :-1 s=READY pgs=476 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74bf4eb82496a92f server_cookie=c48f26549201ac09 in_seq=0 out_seq=0 2024-04-24T09:10:27.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.720+0000 7f268f400640 10 client.22191 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:10:27.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.721+0000 7f268f400640 1 -- 172.21.15.83:0/1227343009 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x557229b4ca80 con 0x557229c71e70 2024-04-24T09:10:27.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.721+0000 7f268f400640 10 client.22191 handle_command_reply: tid=0 2024-04-24T09:10:27.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.721+0000 7f26b799c740 1 client.22191 shutdown 2024-04-24T09:10:27.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.721+0000 7f26b799c740 1 -- 172.21.15.83:0/1227343009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557229be0180 msgr2=0x557229bf2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:27.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.721+0000 7f26b799c740 1 --2- 172.21.15.83:0/1227343009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557229be0180 0x557229bf2f50 secure :-1 s=READY pgs=2453 cs=0 l=1 rev1=1 crypto rx=0x7f269c009850 tx=0x7f269c005a60 comp rx=0 tx=0).stop 2024-04-24T09:10:27.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.722+0000 7f26b799c740 1 -- 172.21.15.83:0/1227343009 shutdown_connections 2024-04-24T09:10:27.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.722+0000 7f26b799c740 1 --2- 172.21.15.83:0/1227343009 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x557229c71e70 0x557229c74290 crc :-1 s=READY pgs=476 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:27.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.722+0000 7f26b799c740 1 --2- 172.21.15.83:0/1227343009 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557229bea850 0x557229be3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:27.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.722+0000 7f26b799c740 1 --2- 172.21.15.83:0/1227343009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557229be0180 0x557229bf2f50 unknown :-1 s=CLOSED pgs=2453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:27.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.722+0000 7f26b799c740 1 --2- 172.21.15.83:0/1227343009 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557229bdf860 0x557229bf2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:27.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.722+0000 7f26b799c740 1 -- 172.21.15.83:0/1227343009 >> 172.21.15.83:0/1227343009 conn(0x557229bdbb80 msgr2=0x557229be5cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:27.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.722+0000 7f268f400640 0 client.22191 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:10:27.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.722+0000 7f26b799c740 1 -- 172.21.15.83:0/1227343009 shutdown_connections 2024-04-24T09:10:27.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.722+0000 7f26b799c740 1 -- 172.21.15.83:0/1227343009 wait complete. 2024-04-24T09:10:27.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.722+0000 7f26b799c740 20 client.22191 trim_cache size 0 max 16384 2024-04-24T09:10:27.724 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T09:10:27.724 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T09:10:27.724 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-04-24T09:10:27.724 INFO:teuthology.orchestra.run.smithi083.stdout: "0dc76183-1de2-4c38-b2e1-7c98f33693f8": { 2024-04-24T09:10:27.724 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-04-24T09:10:27.724 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "0dc76183-1de2-4c38-b2e1-7c98f33693f8", 2024-04-24T09:10:27.724 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-04-24T09:10:27.724 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:10:27.724 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:10:27.724 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T09:10:27.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.723+0000 7f26b4c00640 1 -- 172.21.15.83:0/3428077160 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f26940a3d20 msgr2=0x7f26940a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:27.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.723+0000 7f26b4c00640 1 --2- 172.21.15.83:0/3428077160 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f26940a3d20 0x7f26940a61e0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f26a00cdb90 tx=0x7f26a0098040 comp rx=0 tx=0).stop 2024-04-24T09:10:27.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.723+0000 7f26b4c00640 1 -- 172.21.15.83:0/3428077160 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26b0100570 msgr2=0x7f26b01a0790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:27.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.723+0000 7f26b4c00640 1 --2- 172.21.15.83:0/3428077160 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26b0100570 0x7f26b01a0790 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f26a000d950 tx=0x7f26a000de10 comp rx=0 tx=0).stop 2024-04-24T09:10:27.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.723+0000 7f26b4c00640 1 -- 172.21.15.83:0/3428077160 shutdown_connections 2024-04-24T09:10:27.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.723+0000 7f26b4c00640 1 --2- 172.21.15.83:0/3428077160 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f26940a3d20 0x7f26940a61e0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:27.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.723+0000 7f26b4c00640 1 --2- 172.21.15.83:0/3428077160 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26b0107000 0x7f26b019a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:27.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.723+0000 7f26b4c00640 1 --2- 172.21.15.83:0/3428077160 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f26b0100f30 0x7f26b01a0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:27.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.723+0000 7f26b4c00640 1 --2- 172.21.15.83:0/3428077160 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26b0100570 0x7f26b01a0790 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:27.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.723+0000 7f26b4c00640 1 -- 172.21.15.83:0/3428077160 >> 172.21.15.83:0/3428077160 conn(0x7f26b0077500 msgr2=0x7f26b0105320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:27.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.724+0000 7f26b4c00640 1 -- 172.21.15.83:0/3428077160 shutdown_connections 2024-04-24T09:10:27.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:27.724+0000 7f26b4c00640 1 -- 172.21.15.83:0/3428077160 wait complete. 2024-04-24T09:10:27.748 INFO:tasks.cephfs.filesystem:scrub status for tag:0dc76183-1de2-4c38-b2e1-7c98f33693f8 - {'path': '/', 'tag': '0dc76183-1de2-4c38-b2e1-7c98f33693f8', 'options': 'recursive,force'} 2024-04-24T09:10:28.175 INFO:tasks.workunit.client.0.smithi083.stdout: 10 37829 6.70 MB/sec execute 179 sec latency 129.052 ms 2024-04-24T09:10:28.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:10:28 smithi138 ceph-mon[31441]: pgmap v594: 129 pgs: 129 active+clean; 643 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 206 op/s 2024-04-24T09:10:28.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:10:28 smithi138 ceph-mon[31441]: scrub queued for path: / 2024-04-24T09:10:28.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:10:28 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:10:28.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:10:28 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:10:28.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:10:28 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:10:28.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:10:28 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:10:28.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:10:28 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:10:28.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:10:28 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:10:28.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:10:28 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:10:28.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:10:28 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:10:28.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:10:28 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:10:28.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:10:28 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:10:28.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:10:28 smithi189 ceph-mon[31791]: pgmap v594: 129 pgs: 129 active+clean; 643 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 206 op/s 2024-04-24T09:10:28.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:10:28 smithi189 ceph-mon[31791]: scrub queued for path: / 2024-04-24T09:10:28.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:10:28 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:10:28.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:10:28 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:10:28.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:10:28 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:10:28.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:10:28 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:10:28.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:10:28 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:10:28.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:10:28 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:10:28.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:10:28 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:10:28.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:10:28 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:10:28.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:10:28 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:10:28.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:10:28 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:10:28.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:10:28 smithi083 ceph-mon[27907]: pgmap v594: 129 pgs: 129 active+clean; 643 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 206 op/s 2024-04-24T09:10:28.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:10:28 smithi083 ceph-mon[27907]: scrub queued for path: / 2024-04-24T09:10:28.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:10:28 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:10:28.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:10:28 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:10:28.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:10:28 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:10:28.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:10:28 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:10:28.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:10:28 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:10:28.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:10:28 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:10:28.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:10:28 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:10:28.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:10:28 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:10:28.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:10:28 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:10:28.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:10:28 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:10:29.175 INFO:tasks.workunit.client.0.smithi083.stdout: 10 37984 6.73 MB/sec execute 180 sec latency 66.643 ms 2024-04-24T09:10:29.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:10:29 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:10:29.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:10:29 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:10:29.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:10:29 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:10:29.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:10:29 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:10:29.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:10:29 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:10:29.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:10:29 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:10:29.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:10:29 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:10:29.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:10:29 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:10:29.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:10:29 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:10:29.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:10:29 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:10:29.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:10:29 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:10:29.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:10:29 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:10:29.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:10:29 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:10:29.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:10:29 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:10:29.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:10:29 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:10:29.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:10:29 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:10:29.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:10:29 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:10:29.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:10:29 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:10:29.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:10:29 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:10:29.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:10:29 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:10:29.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:10:29 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:10:29.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:10:29 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:10:29.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:10:29 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:10:29.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:10:29 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:10:30.175 INFO:tasks.workunit.client.0.smithi083.stdout: 10 38137 6.73 MB/sec execute 181 sec latency 66.431 ms 2024-04-24T09:10:30.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:10:30 smithi138 ceph-mon[31441]: pgmap v595: 129 pgs: 129 active+clean; 641 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.6 MiB/s wr, 357 op/s 2024-04-24T09:10:30.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:10:30 smithi189 ceph-mon[31791]: pgmap v595: 129 pgs: 129 active+clean; 641 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.6 MiB/s wr, 357 op/s 2024-04-24T09:10:30.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:10:30 smithi083 ceph-mon[27907]: pgmap v595: 129 pgs: 129 active+clean; 641 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.6 MiB/s wr, 357 op/s 2024-04-24T09:10:31.175 INFO:tasks.workunit.client.0.smithi083.stdout: 10 38223 6.71 MB/sec execute 182 sec latency 395.951 ms 2024-04-24T09:10:32.175 INFO:tasks.workunit.client.0.smithi083.stdout: 10 38362 6.72 MB/sec execute 183 sec latency 69.450 ms 2024-04-24T09:10:32.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:10:32 smithi138 ceph-mon[31441]: pgmap v596: 129 pgs: 129 active+clean; 641 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.9 MiB/s wr, 265 op/s 2024-04-24T09:10:32.706 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:10:32 smithi189 ceph-mon[31791]: pgmap v596: 129 pgs: 129 active+clean; 641 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.9 MiB/s wr, 265 op/s 2024-04-24T09:10:32.802 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:10:32 smithi083 ceph-mon[27907]: pgmap v596: 129 pgs: 129 active+clean; 641 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.9 MiB/s wr, 265 op/s 2024-04-24T09:10:33.175 INFO:tasks.workunit.client.0.smithi083.stdout: 10 38498 6.72 MB/sec execute 184 sec latency 68.563 ms 2024-04-24T09:10:34.175 INFO:tasks.workunit.client.0.smithi083.stdout: 10 38591 6.74 MB/sec execute 185 sec latency 418.464 ms 2024-04-24T09:10:34.704 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:10:34 smithi138 ceph-mon[31441]: pgmap v597: 129 pgs: 129 active+clean; 654 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.5 MiB/s wr, 355 op/s 2024-04-24T09:10:34.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:10:34 smithi189 ceph-mon[31791]: pgmap v597: 129 pgs: 129 active+clean; 654 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.5 MiB/s wr, 355 op/s 2024-04-24T09:10:34.802 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:10:34 smithi083 ceph-mon[27907]: pgmap v597: 129 pgs: 129 active+clean; 654 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.5 MiB/s wr, 355 op/s 2024-04-24T09:10:35.175 INFO:tasks.workunit.client.0.smithi083.stdout: 10 38715 6.74 MB/sec execute 186 sec latency 65.461 ms 2024-04-24T09:10:36.052 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:10:35 smithi083 ceph-mon[27907]: pgmap v598: 129 pgs: 129 active+clean; 654 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.9 MiB/s wr, 241 op/s 2024-04-24T09:10:36.175 INFO:tasks.workunit.client.0.smithi083.stdout: 10 38850 6.76 MB/sec execute 187 sec latency 62.530 ms 2024-04-24T09:10:36.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:10:35 smithi138 ceph-mon[31441]: pgmap v598: 129 pgs: 129 active+clean; 654 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.9 MiB/s wr, 241 op/s 2024-04-24T09:10:36.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:10:35 smithi189 ceph-mon[31791]: pgmap v598: 129 pgs: 129 active+clean; 654 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.9 MiB/s wr, 241 op/s 2024-04-24T09:10:37.176 INFO:tasks.workunit.client.0.smithi083.stdout: 10 39005 6.80 MB/sec execute 188 sec latency 73.919 ms 2024-04-24T09:10:38.177 INFO:tasks.workunit.client.0.smithi083.stdout: 10 39171 6.81 MB/sec execute 189 sec latency 71.234 ms 2024-04-24T09:10:38.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:10:37 smithi138 ceph-mon[31441]: pgmap v599: 129 pgs: 129 active+clean; 654 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.9 MiB/s wr, 241 op/s 2024-04-24T09:10:38.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:10:37 smithi189 ceph-mon[31791]: pgmap v599: 129 pgs: 129 active+clean; 654 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.9 MiB/s wr, 241 op/s 2024-04-24T09:10:38.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:10:37 smithi083 ceph-mon[27907]: pgmap v599: 129 pgs: 129 active+clean; 654 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.9 MiB/s wr, 241 op/s 2024-04-24T09:10:38.912 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.911+0000 7f18fb200640 1 -- 172.21.15.83:0/2137949612 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18fc108e40 msgr2=0x7f18fc1092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:38.912 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.911+0000 7f18fb200640 1 --2- 172.21.15.83:0/2137949612 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18fc108e40 0x7f18fc1092c0 secure :-1 s=READY pgs=2454 cs=0 l=1 rev1=1 crypto rx=0x7f18ec009900 tx=0x7f18ec02a410 comp rx=0 tx=0).stop 2024-04-24T09:10:38.912 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.911+0000 7f18fb200640 1 -- 172.21.15.83:0/2137949612 shutdown_connections 2024-04-24T09:10:38.912 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.911+0000 7f18fb200640 1 --2- 172.21.15.83:0/2137949612 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f18fc109800 0x7f18fc115300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:38.912 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.911+0000 7f18fb200640 1 --2- 172.21.15.83:0/2137949612 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18fc108e40 0x7f18fc1092c0 unknown :-1 s=CLOSED pgs=2454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:38.912 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.911+0000 7f18fb200640 1 --2- 172.21.15.83:0/2137949612 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18fc10e4b0 0x7f18fc10e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:38.912 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.911+0000 7f18fb200640 1 -- 172.21.15.83:0/2137949612 >> 172.21.15.83:0/2137949612 conn(0x7f18fc077a50 msgr2=0x7f18fc079e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:38.912 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.911+0000 7f18fb200640 1 -- 172.21.15.83:0/2137949612 shutdown_connections 2024-04-24T09:10:38.913 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.912+0000 7f18fb200640 1 -- 172.21.15.83:0/2137949612 wait complete. 2024-04-24T09:10:38.914 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.913+0000 7f18fb200640 1 Processor -- start 2024-04-24T09:10:38.915 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.913+0000 7f18fb200640 1 -- start start 2024-04-24T09:10:38.915 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.913+0000 7f18fb200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f18fc108e40 0x7f18fc1a4860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:38.915 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.913+0000 7f18fb200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18fc109800 0x7f18fc1a4da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:38.915 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.913+0000 7f18fb200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18fc10e4b0 0x7f18fc19ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:38.915 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.913+0000 7f18fb200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f18fc1186b0 con 0x7f18fc10e4b0 2024-04-24T09:10:38.915 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.913+0000 7f18fb200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f18fc118530 con 0x7f18fc109800 2024-04-24T09:10:38.915 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.913+0000 7f18fb200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f18fc118830 con 0x7f18fc108e40 2024-04-24T09:10:38.915 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.914+0000 7f18f9400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18fc109800 0x7f18fc1a4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:38.916 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.914+0000 7f18f9400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18fc109800 0x7f18fc1a4da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:44312/0 (socket says 172.21.15.83:44312) 2024-04-24T09:10:38.916 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.914+0000 7f18fa800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18fc10e4b0 0x7f18fc19ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:38.916 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.914+0000 7f18f9400640 1 -- 172.21.15.83:0/2459277466 learned_addr learned my addr 172.21.15.83:0/2459277466 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:10:38.916 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.914+0000 7f18f9e00640 1 --2- 172.21.15.83:0/2459277466 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f18fc108e40 0x7f18fc1a4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:38.916 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.914+0000 7f18f9400640 1 -- 172.21.15.83:0/2459277466 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f18fc108e40 msgr2=0x7f18fc1a4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:38.916 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.914+0000 7f18f9400640 1 --2- 172.21.15.83:0/2459277466 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f18fc108e40 0x7f18fc1a4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:38.916 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.914+0000 7f18f9400640 1 -- 172.21.15.83:0/2459277466 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18fc10e4b0 msgr2=0x7f18fc19ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:38.916 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.914+0000 7f18f9400640 1 --2- 172.21.15.83:0/2459277466 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18fc10e4b0 0x7f18fc19ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:38.917 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.914+0000 7f18f9400640 1 -- 172.21.15.83:0/2459277466 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18fc19f190 con 0x7f18fc109800 2024-04-24T09:10:38.918 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.915+0000 7f18f9400640 1 --2- 172.21.15.83:0/2459277466 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18fc109800 0x7f18fc1a4da0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f18ec006010 tx=0x7f18ec004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:38.918 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.915+0000 7f18f2a00640 1 -- 172.21.15.83:0/2459277466 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18ec03f030 con 0x7f18fc109800 2024-04-24T09:10:38.918 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.915+0000 7f18f2a00640 1 -- 172.21.15.83:0/2459277466 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f18ec003aa0 con 0x7f18fc109800 2024-04-24T09:10:38.918 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.915+0000 7f18f2a00640 1 -- 172.21.15.83:0/2459277466 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18ec035070 con 0x7f18fc109800 2024-04-24T09:10:38.918 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.915+0000 7f18fb200640 1 -- 172.21.15.83:0/2459277466 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18fc19f420 con 0x7f18fc109800 2024-04-24T09:10:38.919 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.915+0000 7f18fb200640 1 -- 172.21.15.83:0/2459277466 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18fc1ab720 con 0x7f18fc109800 2024-04-24T09:10:38.920 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.920+0000 7f18f2a00640 1 -- 172.21.15.83:0/2459277466 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f18ec042030 con 0x7f18fc109800 2024-04-24T09:10:38.921 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.920+0000 7f1903c07740 1 Processor -- start 2024-04-24T09:10:38.921 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.920+0000 7f1903c07740 1 -- start start 2024-04-24T09:10:38.922 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.921+0000 7f1903c07740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556fbd5753c0 0x556fbd5757a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:38.922 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.921+0000 7f1903c07740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556fbd575ce0 0x556fbd580060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:38.922 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.921+0000 7f1903c07740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556fbd5805a0 0x556fbd582990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:38.923 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.921+0000 7f1903c07740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556fbd4db900 con 0x556fbd5805a0 2024-04-24T09:10:38.923 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.921+0000 7f1903c07740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556fbd481f20 con 0x556fbd5753c0 2024-04-24T09:10:38.923 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.921+0000 7f1903c07740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x556fbd4d78c0 con 0x556fbd575ce0 2024-04-24T09:10:38.923 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.921+0000 7f18fa800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556fbd5753c0 0x556fbd5757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:38.923 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.921+0000 7f18fa800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556fbd5753c0 0x556fbd5757a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:44316/0 (socket says 172.21.15.83:44316) 2024-04-24T09:10:38.923 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.921+0000 7f18fa800640 1 -- 172.21.15.83:0/4213401267 learned_addr learned my addr 172.21.15.83:0/4213401267 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:10:38.923 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.921+0000 7f18f9e00640 1 --2- 172.21.15.83:0/4213401267 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556fbd575ce0 0x556fbd580060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:38.923 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.922+0000 7f18fa800640 1 -- 172.21.15.83:0/4213401267 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556fbd575ce0 msgr2=0x556fbd580060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:38.923 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.922+0000 7f18fa800640 1 --2- 172.21.15.83:0/4213401267 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556fbd575ce0 0x556fbd580060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:38.924 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.922+0000 7f18fa800640 1 -- 172.21.15.83:0/4213401267 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556fbd5805a0 msgr2=0x556fbd582990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:38.924 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.922+0000 7f18f9400640 1 --2- 172.21.15.83:0/4213401267 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556fbd5805a0 0x556fbd582990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:38.924 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.922+0000 7f18fa800640 1 --2- 172.21.15.83:0/4213401267 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556fbd5805a0 0x556fbd582990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:38.924 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.922+0000 7f18fa800640 1 -- 172.21.15.83:0/4213401267 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556fbd4a6e10 con 0x556fbd5753c0 2024-04-24T09:10:38.924 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.922+0000 7f18f9400640 1 --2- 172.21.15.83:0/4213401267 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556fbd5805a0 0x556fbd582990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:10:38.924 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.922+0000 7f18fa800640 1 --2- 172.21.15.83:0/4213401267 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556fbd5753c0 0x556fbd5757a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f18f400ae90 tx=0x7f18f400f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fcb8a7230aeb0e7b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:38.925 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.922+0000 7f18f2a00640 1 --2- 172.21.15.83:0/2459277466 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f18e00a3e10 0x7f18e00a62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:38.925 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.922+0000 7f18eb400640 1 -- 172.21.15.83:0/4213401267 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18f4003b70 con 0x556fbd5753c0 2024-04-24T09:10:38.926 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.922+0000 7f18eb400640 1 -- 172.21.15.83:0/4213401267 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f18f4003d10 con 0x556fbd5753c0 2024-04-24T09:10:38.926 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.922+0000 7f18eb400640 1 -- 172.21.15.83:0/4213401267 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18f4010b90 con 0x556fbd5753c0 2024-04-24T09:10:38.926 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.923+0000 7f18f2a00640 1 -- 172.21.15.83:0/2459277466 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 9583+0+0 (secure 0 0 0) 0x7f18ec0f1210 con 0x7f18fc109800 2024-04-24T09:10:38.926 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.923+0000 7f18f9e00640 1 --2- 172.21.15.83:0/2459277466 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f18e00a3e10 0x7f18e00a62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:38.926 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.923+0000 7f1903c07740 1 -- 172.21.15.83:0/4213401267 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556fbd5753c0 msgr2=0x556fbd5757a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:38.926 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.923+0000 7f1903c07740 1 --2- 172.21.15.83:0/4213401267 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556fbd5753c0 0x556fbd5757a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f18f400ae90 tx=0x7f18f400f790 comp rx=0 tx=0).stop 2024-04-24T09:10:38.926 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.924+0000 7f18f9e00640 1 --2- 172.21.15.83:0/2459277466 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f18e00a3e10 0x7f18e00a62d0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x556fbd576960 tx=0x7f18e4005f90 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:38.926 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.924+0000 7f1903c07740 1 -- 172.21.15.83:0/4213401267 shutdown_connections 2024-04-24T09:10:38.927 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.924+0000 7f1903c07740 1 --2- 172.21.15.83:0/4213401267 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556fbd5805a0 0x556fbd582990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:38.927 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.924+0000 7f1903c07740 1 --2- 172.21.15.83:0/4213401267 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556fbd575ce0 0x556fbd580060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:38.927 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.924+0000 7f1903c07740 1 --2- 172.21.15.83:0/4213401267 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556fbd5753c0 0x556fbd5757a0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:38.927 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.924+0000 7f1903c07740 1 -- 172.21.15.83:0/4213401267 >> 172.21.15.83:0/4213401267 conn(0x556fbd5718d0 msgr2=0x556fbd571cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:38.927 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.924+0000 7f1903c07740 1 -- 172.21.15.83:0/4213401267 shutdown_connections 2024-04-24T09:10:38.927 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.924+0000 7f1903c07740 1 -- 172.21.15.83:0/4213401267 wait complete. 2024-04-24T09:10:38.927 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.924+0000 7f1903c07740 1 -- start start 2024-04-24T09:10:38.927 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.925+0000 7f1903c07740 1 Processor -- start 2024-04-24T09:10:38.931 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.925+0000 7f1903c07740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556fbd5753c0 0x556fbd57d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:38.931 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.925+0000 7f1903c07740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556fbd575ce0 0x556fbd57daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:38.931 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.925+0000 7f1903c07740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556fbd5805a0 0x556fbd57dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:38.931 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.925+0000 7f1903c07740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556fbd583170 con 0x556fbd5805a0 2024-04-24T09:10:38.931 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.925+0000 7f1903c07740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556fbd4952c0 con 0x556fbd575ce0 2024-04-24T09:10:38.932 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.925+0000 7f1903c07740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x556fbd5832f0 con 0x556fbd5753c0 2024-04-24T09:10:38.932 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.925+0000 7f1903c07740 10 client.0 _ll_register_callbacks cb 0x556fbd1eda00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:10:38.932 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.926+0000 7f18f9400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556fbd575ce0 0x556fbd57daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:38.932 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.926+0000 7f18f9400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556fbd575ce0 0x556fbd57daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:44326/0 (socket says 172.21.15.83:44326) 2024-04-24T09:10:38.932 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.926+0000 7f18f9400640 1 -- 172.21.15.83:0/3406245551 learned_addr learned my addr 172.21.15.83:0/3406245551 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:10:38.932 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.926+0000 7f18f9e00640 1 --2- 172.21.15.83:0/3406245551 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556fbd5753c0 0x556fbd57d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:38.932 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.926+0000 7f18fa800640 1 --2- 172.21.15.83:0/3406245551 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556fbd5805a0 0x556fbd57dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:38.932 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.926+0000 7f18f9400640 1 -- 172.21.15.83:0/3406245551 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556fbd5753c0 msgr2=0x556fbd57d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:38.932 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.926+0000 7f18f9400640 1 --2- 172.21.15.83:0/3406245551 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556fbd5753c0 0x556fbd57d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:38.932 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.926+0000 7f18f9400640 1 -- 172.21.15.83:0/3406245551 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556fbd5805a0 msgr2=0x556fbd57dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:38.932 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.926+0000 7f18f9400640 1 --2- 172.21.15.83:0/3406245551 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556fbd5805a0 0x556fbd57dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:38.932 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.926+0000 7f18f9400640 1 -- 172.21.15.83:0/3406245551 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556fbd4a5fd0 con 0x556fbd575ce0 2024-04-24T09:10:38.932 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.927+0000 7f18f9400640 1 --2- 172.21.15.83:0/3406245551 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556fbd575ce0 0x556fbd57daa0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f18ec0bccc0 tx=0x7f18ec0bc930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f68297ff958b7d59 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:38.932 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.927+0000 7f18dfe00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:10:38.933 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.928+0000 7f18dfe00640 1 -- 172.21.15.83:0/3406245551 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18ec049040 con 0x556fbd575ce0 2024-04-24T09:10:38.933 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.928+0000 7f18dfe00640 1 -- 172.21.15.83:0/3406245551 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(120..120 src has 1..120) ==== 9583+0+0 (secure 0 0 0) 0x7f18ec003c50 con 0x556fbd575ce0 2024-04-24T09:10:38.933 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.928+0000 7f1903c07740 1 -- 172.21.15.83:0/3406245551 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18f4003d10 con 0x556fbd575ce0 2024-04-24T09:10:38.933 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.928+0000 7f18dfe00640 1 -- 172.21.15.83:0/3406245551 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f18ec0c10e0 con 0x556fbd575ce0 2024-04-24T09:10:38.933 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.928+0000 7f1903c07740 10 client.39290 fetch_fsmap learned FSMap version 61 2024-04-24T09:10:38.933 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.928+0000 7f1903c07740 1 -- 172.21.15.83:0/3406245551 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x556fbd1ee210 con 0x556fbd575ce0 2024-04-24T09:10:38.933 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.929+0000 7f18dfe00640 1 -- 172.21.15.83:0/3406245551 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f18f4003d10 con 0x556fbd575ce0 2024-04-24T09:10:38.933 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.929+0000 7f18dfe00640 1 -- 172.21.15.83:0/3406245551 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 61) ==== 8362+0+0 (secure 0 0 0) 0x7f18ec0bcf90 con 0x556fbd575ce0 2024-04-24T09:10:38.933 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.929+0000 7f1903c07740 10 client.39290 fetch_fsmap finished waiting for FSMap version 61 2024-04-24T09:10:38.933 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.929+0000 7f1903c07740 10 client.39290 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:10:38.933 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.929+0000 7f1903c07740 20 client.39290 populate_metadata read hostname 'smithi083' 2024-04-24T09:10:38.933 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.929+0000 7f1903c07740 1 --2- 172.21.15.83:0/3406245551 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556fbd590590 0x556fbd5929c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:38.933 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.929+0000 7f1903c07740 4 client.39290 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:10:38.933 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.929+0000 7f1903c07740 1 -- 172.21.15.83:0/3406245551 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556fbd4e9320 con 0x556fbd590590 2024-04-24T09:10:38.933 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.930+0000 7f18f9e00640 1 --2- 172.21.15.83:0/3406245551 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556fbd590590 0x556fbd5929c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:38.934 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.930+0000 7f18f9e00640 1 --2- 172.21.15.83:0/3406245551 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556fbd590590 0x556fbd5929c0 crc :-1 s=READY pgs=477 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65c508c63e738bde server_cookie=620cbf3f5d8e3207 in_seq=0 out_seq=0 2024-04-24T09:10:38.934 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.930+0000 7f18dfe00640 10 client.39290 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:10:38.934 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.931+0000 7f18dfe00640 1 -- 172.21.15.83:0/3406245551 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556fbd4e9320 con 0x556fbd590590 2024-04-24T09:10:38.934 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.931+0000 7f18dfe00640 10 client.39290 handle_command_reply: tid=0 2024-04-24T09:10:38.934 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.931+0000 7f1903c07740 1 client.39290 shutdown 2024-04-24T09:10:38.935 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.932+0000 7f1903c07740 1 -- 172.21.15.83:0/3406245551 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556fbd575ce0 msgr2=0x556fbd57daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:38.935 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.932+0000 7f1903c07740 1 --2- 172.21.15.83:0/3406245551 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556fbd575ce0 0x556fbd57daa0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f18ec0bccc0 tx=0x7f18ec0bc930 comp rx=0 tx=0).stop 2024-04-24T09:10:38.935 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.932+0000 7f1903c07740 1 -- 172.21.15.83:0/3406245551 shutdown_connections 2024-04-24T09:10:38.935 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.932+0000 7f1903c07740 1 --2- 172.21.15.83:0/3406245551 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556fbd590590 0x556fbd5929c0 crc :-1 s=READY pgs=477 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:38.935 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.932+0000 7f1903c07740 1 --2- 172.21.15.83:0/3406245551 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556fbd5805a0 0x556fbd57dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:38.935 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.932+0000 7f1903c07740 1 --2- 172.21.15.83:0/3406245551 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556fbd575ce0 0x556fbd57daa0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:38.935 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.932+0000 7f1903c07740 1 --2- 172.21.15.83:0/3406245551 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556fbd5753c0 0x556fbd57d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:38.935 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.932+0000 7f1903c07740 1 -- 172.21.15.83:0/3406245551 >> 172.21.15.83:0/3406245551 conn(0x556fbd5718d0 msgr2=0x556fbd577600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:38.935 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.932+0000 7f18dfe00640 0 client.39290 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:10:38.935 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.932+0000 7f1903c07740 1 -- 172.21.15.83:0/3406245551 shutdown_connections 2024-04-24T09:10:38.936 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.932+0000 7f1903c07740 1 -- 172.21.15.83:0/3406245551 wait complete. 2024-04-24T09:10:38.936 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.932+0000 7f1903c07740 20 client.39290 trim_cache size 0 max 16384 2024-04-24T09:10:38.961 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.961+0000 7f1903c07740 1 Processor -- start 2024-04-24T09:10:38.962 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.961+0000 7f1903c07740 1 -- start start 2024-04-24T09:10:38.962 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.961+0000 7f1903c07740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556fbd5753c0 0x556fbd58c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:38.962 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.962+0000 7f1903c07740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556fbd575ce0 0x556fbd577d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:38.962 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.962+0000 7f1903c07740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556fbd5805a0 0x556fbd57dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:38.962 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.962+0000 7f1903c07740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f18fc1b5b20 con 0x556fbd5753c0 2024-04-24T09:10:38.962 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.962+0000 7f1903c07740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f18fc1b7660 con 0x556fbd5805a0 2024-04-24T09:10:38.962 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.962+0000 7f1903c07740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f18fc1b61f0 con 0x556fbd575ce0 2024-04-24T09:10:38.963 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.962+0000 7f18f9e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556fbd5753c0 0x556fbd58c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:38.963 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.962+0000 7f18f9e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556fbd5753c0 0x556fbd58c030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47270/0 (socket says 172.21.15.83:47270) 2024-04-24T09:10:38.963 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.962+0000 7f18f9e00640 1 -- 172.21.15.83:0/35073136 learned_addr learned my addr 172.21.15.83:0/35073136 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:10:38.964 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.962+0000 7f18f9400640 1 --2- 172.21.15.83:0/35073136 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556fbd575ce0 0x556fbd577d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:38.964 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.962+0000 7f18fa800640 1 --2- 172.21.15.83:0/35073136 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556fbd5805a0 0x556fbd57dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:38.964 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.963+0000 7f18f9e00640 1 -- 172.21.15.83:0/35073136 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556fbd575ce0 msgr2=0x556fbd577d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:38.964 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.963+0000 7f18f9e00640 1 --2- 172.21.15.83:0/35073136 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556fbd575ce0 0x556fbd577d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:38.964 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.963+0000 7f18f9e00640 1 -- 172.21.15.83:0/35073136 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556fbd5805a0 msgr2=0x556fbd57dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:38.964 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.963+0000 7f18f9e00640 1 --2- 172.21.15.83:0/35073136 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556fbd5805a0 0x556fbd57dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:38.964 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.963+0000 7f18f9e00640 1 -- 172.21.15.83:0/35073136 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556fbd582d00 con 0x556fbd5753c0 2024-04-24T09:10:38.964 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.963+0000 7f18f9e00640 1 --2- 172.21.15.83:0/35073136 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556fbd5753c0 0x556fbd58c030 secure :-1 s=READY pgs=2455 cs=0 l=1 rev1=1 crypto rx=0x7f18e4009e00 tx=0x7f18e4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c46526b88810f4c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:38.965 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.964+0000 7f18eaa00640 1 -- 172.21.15.83:0/35073136 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18e402f070 con 0x556fbd5753c0 2024-04-24T09:10:38.965 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.964+0000 7f18eaa00640 1 -- 172.21.15.83:0/35073136 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f18e4004030 con 0x556fbd5753c0 2024-04-24T09:10:38.965 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.964+0000 7f18eaa00640 1 -- 172.21.15.83:0/35073136 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18e4029930 con 0x556fbd5753c0 2024-04-24T09:10:38.965 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.965+0000 7f1903c07740 1 -- 172.21.15.83:0/35073136 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556fbd5753c0 msgr2=0x556fbd58c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:38.965 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.965+0000 7f1903c07740 1 --2- 172.21.15.83:0/35073136 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556fbd5753c0 0x556fbd58c030 secure :-1 s=READY pgs=2455 cs=0 l=1 rev1=1 crypto rx=0x7f18e4009e00 tx=0x7f18e4016000 comp rx=0 tx=0).stop 2024-04-24T09:10:38.966 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.965+0000 7f1903c07740 1 -- 172.21.15.83:0/35073136 shutdown_connections 2024-04-24T09:10:38.966 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.965+0000 7f1903c07740 1 --2- 172.21.15.83:0/35073136 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556fbd5805a0 0x556fbd57dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:38.966 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.965+0000 7f1903c07740 1 --2- 172.21.15.83:0/35073136 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556fbd575ce0 0x556fbd577d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:38.966 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.965+0000 7f1903c07740 1 --2- 172.21.15.83:0/35073136 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556fbd5753c0 0x556fbd58c030 unknown :-1 s=CLOSED pgs=2455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:38.966 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.965+0000 7f1903c07740 1 -- 172.21.15.83:0/35073136 >> 172.21.15.83:0/35073136 conn(0x556fbd5718d0 msgr2=0x556fbd57d520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:38.966 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.965+0000 7f1903c07740 1 -- 172.21.15.83:0/35073136 shutdown_connections 2024-04-24T09:10:38.966 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.965+0000 7f1903c07740 1 -- 172.21.15.83:0/35073136 wait complete. 2024-04-24T09:10:38.967 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.966+0000 7f1903c07740 1 -- start start 2024-04-24T09:10:38.967 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.966+0000 7f1903c07740 1 Processor -- start 2024-04-24T09:10:38.967 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.967+0000 7f1903c07740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556fbd5753c0 0x556fbd588ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:38.967 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.967+0000 7f1903c07740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556fbd575ce0 0x556fbd57a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:38.967 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.967+0000 7f1903c07740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556fbd5805a0 0x556fbd57a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:38.967 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.967+0000 7f1903c07740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f18fc1b6f90 con 0x556fbd575ce0 2024-04-24T09:10:38.967 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.967+0000 7f1903c07740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f18fc1b68c0 con 0x556fbd5805a0 2024-04-24T09:10:38.967 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.967+0000 7f1903c07740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f18fc1b3b20 con 0x556fbd5753c0 2024-04-24T09:10:38.968 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.967+0000 7f1903c07740 10 client.0 _ll_register_callbacks cb 0x7f18ec0bcb00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:10:38.969 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.967+0000 7f18f9400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556fbd575ce0 0x556fbd57a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:38.969 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.967+0000 7f18f9400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556fbd575ce0 0x556fbd57a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47278/0 (socket says 172.21.15.83:47278) 2024-04-24T09:10:38.969 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.967+0000 7f18f9400640 1 -- 172.21.15.83:0/3459549916 learned_addr learned my addr 172.21.15.83:0/3459549916 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:10:38.969 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.967+0000 7f18f9e00640 1 --2- 172.21.15.83:0/3459549916 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556fbd5753c0 0x556fbd588ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:38.969 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.968+0000 7f18fa800640 1 --2- 172.21.15.83:0/3459549916 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556fbd5805a0 0x556fbd57a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:38.969 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.968+0000 7f18f9400640 1 -- 172.21.15.83:0/3459549916 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556fbd5753c0 msgr2=0x556fbd588ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:38.970 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.968+0000 7f18f9400640 1 --2- 172.21.15.83:0/3459549916 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556fbd5753c0 0x556fbd588ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:38.970 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.968+0000 7f18f9400640 1 -- 172.21.15.83:0/3459549916 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556fbd5805a0 msgr2=0x556fbd57a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:38.970 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.968+0000 7f18f9400640 1 --2- 172.21.15.83:0/3459549916 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556fbd5805a0 0x556fbd57a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:38.970 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.968+0000 7f18f9400640 1 -- 172.21.15.83:0/3459549916 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556fbd2ba620 con 0x556fbd575ce0 2024-04-24T09:10:38.970 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.968+0000 7f18f9e00640 1 --2- 172.21.15.83:0/3459549916 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556fbd5753c0 0x556fbd588ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:10:38.970 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.968+0000 7f18f9400640 1 --2- 172.21.15.83:0/3459549916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556fbd575ce0 0x556fbd57a0c0 secure :-1 s=READY pgs=2456 cs=0 l=1 rev1=1 crypto rx=0x7f18ec0b9190 tx=0x7f18ec009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc4bd7c13d58791 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:38.970 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.968+0000 7f18dfe00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:10:38.970 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.969+0000 7f18dfe00640 1 -- 172.21.15.83:0/3459549916 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18ec09f060 con 0x556fbd575ce0 2024-04-24T09:10:38.970 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.969+0000 7f18dfe00640 1 -- 172.21.15.83:0/3459549916 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(120..120 src has 1..120) ==== 9583+0+0 (secure 0 0 0) 0x7f18ec0dd0d0 con 0x556fbd575ce0 2024-04-24T09:10:38.970 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.969+0000 7f1903c07740 1 -- 172.21.15.83:0/3459549916 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18e4004030 con 0x556fbd575ce0 2024-04-24T09:10:38.970 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.969+0000 7f18dfe00640 1 -- 172.21.15.83:0/3459549916 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f18ec049040 con 0x556fbd575ce0 2024-04-24T09:10:38.970 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.969+0000 7f1903c07740 10 client.22206 fetch_fsmap learned FSMap version 61 2024-04-24T09:10:38.970 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.969+0000 7f1903c07740 1 -- 172.21.15.83:0/3459549916 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x556fbd5f6ac0 con 0x556fbd575ce0 2024-04-24T09:10:38.971 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.970+0000 7f18dfe00640 1 -- 172.21.15.83:0/3459549916 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f18e4004030 con 0x556fbd575ce0 2024-04-24T09:10:38.971 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.970+0000 7f18dfe00640 1 -- 172.21.15.83:0/3459549916 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 61) ==== 8362+0+0 (secure 0 0 0) 0x7f18ec0035f0 con 0x556fbd575ce0 2024-04-24T09:10:38.971 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.970+0000 7f1903c07740 10 client.22206 fetch_fsmap finished waiting for FSMap version 61 2024-04-24T09:10:38.971 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.970+0000 7f1903c07740 10 client.22206 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:10:38.971 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.970+0000 7f1903c07740 20 client.22206 populate_metadata read hostname 'smithi083' 2024-04-24T09:10:38.971 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.970+0000 7f1903c07740 1 --2- 172.21.15.83:0/3459549916 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556fbd5f7030 0x556fbd5f9450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:38.972 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.970+0000 7f1903c07740 4 client.22206 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T09:10:38.972 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.970+0000 7f1903c07740 1 -- 172.21.15.83:0/3459549916 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556fbd4e3190 con 0x556fbd5f7030 2024-04-24T09:10:38.972 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.971+0000 7f18f9e00640 1 --2- 172.21.15.83:0/3459549916 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556fbd5f7030 0x556fbd5f9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:38.972 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.971+0000 7f18f9e00640 1 --2- 172.21.15.83:0/3459549916 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556fbd5f7030 0x556fbd5f9450 crc :-1 s=READY pgs=478 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85a1577eb06b5aae server_cookie=dcf74cca6bcb452e in_seq=0 out_seq=0 2024-04-24T09:10:38.972 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:38.972+0000 7f18dfe00640 10 client.22206 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:10:39.176 INFO:tasks.workunit.client.0.smithi083.stdout: 10 39262 6.80 MB/sec execute 190 sec latency 230.549 ms 2024-04-24T09:10:39.503 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:39.503+0000 7f18dfe00640 1 -- 172.21.15.83:0/3459549916 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556fbd4e3190 con 0x556fbd5f7030 2024-04-24T09:10:39.504 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:39.503+0000 7f18dfe00640 10 client.22206 handle_command_reply: tid=0 2024-04-24T09:10:39.504 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:39.503+0000 7f1903c07740 1 client.22206 shutdown 2024-04-24T09:10:39.504 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:39.504+0000 7f1903c07740 1 -- 172.21.15.83:0/3459549916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556fbd575ce0 msgr2=0x556fbd57a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:39.504 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:39.504+0000 7f1903c07740 1 --2- 172.21.15.83:0/3459549916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556fbd575ce0 0x556fbd57a0c0 secure :-1 s=READY pgs=2456 cs=0 l=1 rev1=1 crypto rx=0x7f18ec0b9190 tx=0x7f18ec009680 comp rx=0 tx=0).stop 2024-04-24T09:10:39.505 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:39.504+0000 7f1903c07740 1 -- 172.21.15.83:0/3459549916 shutdown_connections 2024-04-24T09:10:39.505 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:39.504+0000 7f1903c07740 1 --2- 172.21.15.83:0/3459549916 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556fbd5f7030 0x556fbd5f9450 crc :-1 s=READY pgs=478 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:39.505 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:39.504+0000 7f1903c07740 1 --2- 172.21.15.83:0/3459549916 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556fbd5805a0 0x556fbd57a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:39.505 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:39.504+0000 7f1903c07740 1 --2- 172.21.15.83:0/3459549916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556fbd575ce0 0x556fbd57a0c0 unknown :-1 s=CLOSED pgs=2456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:39.506 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:39.504+0000 7f1903c07740 1 --2- 172.21.15.83:0/3459549916 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556fbd5753c0 0x556fbd588ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:39.506 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:39.504+0000 7f1903c07740 1 -- 172.21.15.83:0/3459549916 >> 172.21.15.83:0/3459549916 conn(0x556fbd5718d0 msgr2=0x556fbd570a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:39.506 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:39.505+0000 7f18dfe00640 0 client.22206 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:10:39.506 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:39.505+0000 7f1903c07740 1 -- 172.21.15.83:0/3459549916 shutdown_connections 2024-04-24T09:10:39.506 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:39.505+0000 7f1903c07740 1 -- 172.21.15.83:0/3459549916 wait complete. 2024-04-24T09:10:39.506 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:39.505+0000 7f1903c07740 20 client.22206 trim_cache size 0 max 16384 2024-04-24T09:10:39.507 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T09:10:39.507 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T09:10:39.507 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T09:10:39.507 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T09:10:39.508 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:39.506+0000 7f18fb200640 1 -- 172.21.15.83:0/2459277466 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f18e00a3e10 msgr2=0x7f18e00a62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:39.508 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:39.506+0000 7f18fb200640 1 --2- 172.21.15.83:0/2459277466 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f18e00a3e10 0x7f18e00a62d0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x556fbd576960 tx=0x7f18e4005f90 comp rx=0 tx=0).stop 2024-04-24T09:10:39.508 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:39.506+0000 7f18fb200640 1 -- 172.21.15.83:0/2459277466 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18fc109800 msgr2=0x7f18fc1a4da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:39.508 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:39.506+0000 7f18fb200640 1 --2- 172.21.15.83:0/2459277466 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18fc109800 0x7f18fc1a4da0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f18ec006010 tx=0x7f18ec004270 comp rx=0 tx=0).stop 2024-04-24T09:10:39.508 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:39.507+0000 7f18fb200640 1 -- 172.21.15.83:0/2459277466 shutdown_connections 2024-04-24T09:10:39.508 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:39.507+0000 7f18fb200640 1 --2- 172.21.15.83:0/2459277466 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f18e00a3e10 0x7f18e00a62d0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:39.508 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:39.507+0000 7f18fb200640 1 --2- 172.21.15.83:0/2459277466 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18fc10e4b0 0x7f18fc19ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:39.508 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:39.507+0000 7f18fb200640 1 --2- 172.21.15.83:0/2459277466 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18fc109800 0x7f18fc1a4da0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:39.508 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:39.507+0000 7f18fb200640 1 --2- 172.21.15.83:0/2459277466 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f18fc108e40 0x7f18fc1a4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:39.508 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:39.507+0000 7f18fb200640 1 -- 172.21.15.83:0/2459277466 >> 172.21.15.83:0/2459277466 conn(0x7f18fc077a50 msgr2=0x7f18fc07a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:39.508 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:39.507+0000 7f18fb200640 1 -- 172.21.15.83:0/2459277466 shutdown_connections 2024-04-24T09:10:39.508 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:39.507+0000 7f18fb200640 1 -- 172.21.15.83:0/2459277466 wait complete. 2024-04-24T09:10:40.176 INFO:tasks.workunit.client.0.smithi083.stdout: 10 39332 6.78 MB/sec execute 191 sec latency 515.712 ms 2024-04-24T09:10:40.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:10:39 smithi138 ceph-mon[31441]: pgmap v600: 129 pgs: 129 active+clean; 672 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.2 MiB/s wr, 331 op/s 2024-04-24T09:10:40.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:10:39 smithi189 ceph-mon[31791]: pgmap v600: 129 pgs: 129 active+clean; 672 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.2 MiB/s wr, 331 op/s 2024-04-24T09:10:40.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:10:39 smithi083 ceph-mon[27907]: pgmap v600: 129 pgs: 129 active+clean; 672 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.2 MiB/s wr, 331 op/s 2024-04-24T09:10:41.176 INFO:tasks.workunit.client.0.smithi083.stdout: 10 39424 6.76 MB/sec execute 192 sec latency 92.671 ms 2024-04-24T09:10:42.176 INFO:tasks.workunit.client.0.smithi083.stdout: 10 39565 6.79 MB/sec execute 193 sec latency 70.430 ms 2024-04-24T09:10:42.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:10:41 smithi189 ceph-mon[31791]: pgmap v601: 129 pgs: 129 active+clean; 672 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.0 MiB/s wr, 179 op/s 2024-04-24T09:10:42.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:10:41 smithi083 ceph-mon[27907]: pgmap v601: 129 pgs: 129 active+clean; 672 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.0 MiB/s wr, 179 op/s 2024-04-24T09:10:42.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:10:41 smithi138 ceph-mon[31441]: pgmap v601: 129 pgs: 129 active+clean; 672 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.0 MiB/s wr, 179 op/s 2024-04-24T09:10:43.176 INFO:tasks.workunit.client.0.smithi083.stdout: 10 39680 6.77 MB/sec execute 194 sec latency 67.684 ms 2024-04-24T09:10:43.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:10:42 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:10:43.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:10:42 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:10:43.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:10:42 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:10:43.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:10:42 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:10:43.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:10:42 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:10:43.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:10:42 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:10:43.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:10:42 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:10:43.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:10:42 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:10:43.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:10:42 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:10:43.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:10:42 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:10:43.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:10:42 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:10:43.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:10:42 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:10:44.176 INFO:tasks.workunit.client.0.smithi083.stdout: 10 39786 6.75 MB/sec execute 195 sec latency 79.427 ms 2024-04-24T09:10:44.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:10:43 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:10:44.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:10:43 smithi083 ceph-mon[27907]: pgmap v602: 129 pgs: 129 active+clean; 675 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.8 MiB/s wr, 323 op/s 2024-04-24T09:10:44.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:10:43 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:10:44.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:10:43 smithi138 ceph-mon[31441]: pgmap v602: 129 pgs: 129 active+clean; 675 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.8 MiB/s wr, 323 op/s 2024-04-24T09:10:44.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:10:43 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:10:44.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:10:43 smithi189 ceph-mon[31791]: pgmap v602: 129 pgs: 129 active+clean; 675 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.8 MiB/s wr, 323 op/s 2024-04-24T09:10:45.176 INFO:tasks.workunit.client.0.smithi083.stdout: 10 39878 6.77 MB/sec execute 196 sec latency 415.093 ms 2024-04-24T09:10:46.176 INFO:tasks.workunit.client.0.smithi083.stdout: 10 39986 6.74 MB/sec execute 197 sec latency 96.757 ms 2024-04-24T09:10:46.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:10:45 smithi083 ceph-mon[27907]: pgmap v603: 129 pgs: 129 active+clean; 675 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.1 MiB/s wr, 234 op/s 2024-04-24T09:10:46.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:10:45 smithi138 ceph-mon[31441]: pgmap v603: 129 pgs: 129 active+clean; 675 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.1 MiB/s wr, 234 op/s 2024-04-24T09:10:46.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:10:45 smithi189 ceph-mon[31791]: pgmap v603: 129 pgs: 129 active+clean; 675 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.1 MiB/s wr, 234 op/s 2024-04-24T09:10:47.176 INFO:tasks.workunit.client.0.smithi083.stdout: 10 40099 6.72 MB/sec execute 198 sec latency 95.271 ms 2024-04-24T09:10:48.177 INFO:tasks.workunit.client.0.smithi083.stdout: 10 40222 6.71 MB/sec execute 199 sec latency 110.457 ms 2024-04-24T09:10:48.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:10:47 smithi083 ceph-mon[27907]: pgmap v604: 129 pgs: 129 active+clean; 675 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.1 MiB/s wr, 234 op/s 2024-04-24T09:10:48.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:10:47 smithi138 ceph-mon[31441]: pgmap v604: 129 pgs: 129 active+clean; 675 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.1 MiB/s wr, 234 op/s 2024-04-24T09:10:48.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:10:47 smithi189 ceph-mon[31791]: pgmap v604: 129 pgs: 129 active+clean; 675 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.1 MiB/s wr, 234 op/s 2024-04-24T09:10:49.177 INFO:tasks.workunit.client.0.smithi083.stdout: 10 40355 6.69 MB/sec execute 200 sec latency 84.733 ms 2024-04-24T09:10:50.177 INFO:tasks.workunit.client.0.smithi083.stdout: 10 40483 6.69 MB/sec execute 201 sec latency 95.519 ms 2024-04-24T09:10:50.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:10:49 smithi083 ceph-mon[27907]: pgmap v605: 129 pgs: 129 active+clean; 674 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.6 MiB/s wr, 342 op/s 2024-04-24T09:10:50.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:10:49 smithi138 ceph-mon[31441]: pgmap v605: 129 pgs: 129 active+clean; 674 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.6 MiB/s wr, 342 op/s 2024-04-24T09:10:50.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:10:49 smithi189 ceph-mon[31791]: pgmap v605: 129 pgs: 129 active+clean; 674 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.6 MiB/s wr, 342 op/s 2024-04-24T09:10:51.177 INFO:tasks.workunit.client.0.smithi083.stdout: 10 40616 6.68 MB/sec execute 202 sec latency 105.096 ms 2024-04-24T09:10:52.177 INFO:tasks.workunit.client.0.smithi083.stdout: 10 40751 6.68 MB/sec execute 203 sec latency 80.275 ms 2024-04-24T09:10:52.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:10:51 smithi083 ceph-mon[27907]: pgmap v606: 129 pgs: 129 active+clean; 674 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.2 MiB/s wr, 252 op/s 2024-04-24T09:10:52.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:10:51 smithi138 ceph-mon[31441]: pgmap v606: 129 pgs: 129 active+clean; 674 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.2 MiB/s wr, 252 op/s 2024-04-24T09:10:52.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:10:51 smithi189 ceph-mon[31791]: pgmap v606: 129 pgs: 129 active+clean; 674 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.2 MiB/s wr, 252 op/s 2024-04-24T09:10:52.698 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.699+0000 7f11df200640 1 -- 172.21.15.83:0/2440180390 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11e0104c50 msgr2=0x7f11e0108c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:52.699 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.699+0000 7f11df200640 1 --2- 172.21.15.83:0/2440180390 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11e0104c50 0x7f11e0108c90 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f11c8009ca0 tx=0x7f11c802a8b0 comp rx=0 tx=0).stop 2024-04-24T09:10:52.699 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.700+0000 7f11df200640 1 -- 172.21.15.83:0/2440180390 shutdown_connections 2024-04-24T09:10:52.699 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.700+0000 7f11df200640 1 --2- 172.21.15.83:0/2440180390 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11e0109370 0x7f11e0110f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:52.700 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.700+0000 7f11df200640 1 --2- 172.21.15.83:0/2440180390 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11e0104c50 0x7f11e0108c90 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:52.700 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.700+0000 7f11df200640 1 --2- 172.21.15.83:0/2440180390 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f11e01042a0 0x7f11e0104680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:52.700 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.700+0000 7f11df200640 1 -- 172.21.15.83:0/2440180390 >> 172.21.15.83:0/2440180390 conn(0x7f11e00ffae0 msgr2=0x7f11e0101f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:52.700 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.700+0000 7f11df200640 1 -- 172.21.15.83:0/2440180390 shutdown_connections 2024-04-24T09:10:52.700 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.700+0000 7f11df200640 1 -- 172.21.15.83:0/2440180390 wait complete. 2024-04-24T09:10:52.700 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.701+0000 7f11df200640 1 Processor -- start 2024-04-24T09:10:52.701 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.702+0000 7f11df200640 1 -- start start 2024-04-24T09:10:52.703 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.703+0000 7f11df200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11e01042a0 0x7f11e019a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:52.703 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.703+0000 7f11df200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f11e0109370 0x7f11e019ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:52.703 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.703+0000 7f11df200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11e019ef80 0x7f11e019f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:52.704 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.703+0000 7f11df200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f11e01142e0 con 0x7f11e019ef80 2024-04-24T09:10:52.704 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.703+0000 7f11df200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f11e0114160 con 0x7f11e01042a0 2024-04-24T09:10:52.704 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.703+0000 7f11df200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f11e0114460 con 0x7f11e0109370 2024-04-24T09:10:52.704 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.704+0000 7f11dd400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f11e0109370 0x7f11e019ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:52.704 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.704+0000 7f11dd400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f11e0109370 0x7f11e019ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:47068/0 (socket says 172.21.15.83:47068) 2024-04-24T09:10:52.704 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.704+0000 7f11dd400640 1 -- 172.21.15.83:0/620066989 learned_addr learned my addr 172.21.15.83:0/620066989 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:10:52.705 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.704+0000 7f11dde00640 1 --2- 172.21.15.83:0/620066989 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11e01042a0 0x7f11e019a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:52.705 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.705+0000 7f11de800640 1 --2- 172.21.15.83:0/620066989 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11e019ef80 0x7f11e019f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:52.705 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.705+0000 7f11de800640 1 -- 172.21.15.83:0/620066989 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f11e0109370 msgr2=0x7f11e019ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:52.705 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.705+0000 7f11de800640 1 --2- 172.21.15.83:0/620066989 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f11e0109370 0x7f11e019ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:52.705 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.705+0000 7f11de800640 1 -- 172.21.15.83:0/620066989 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11e01042a0 msgr2=0x7f11e019a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:52.706 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.705+0000 7f11de800640 1 --2- 172.21.15.83:0/620066989 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11e01042a0 0x7f11e019a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:52.706 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.706+0000 7f11de800640 1 -- 172.21.15.83:0/620066989 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11e019fa70 con 0x7f11e019ef80 2024-04-24T09:10:52.706 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.706+0000 7f11dde00640 1 --2- 172.21.15.83:0/620066989 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11e01042a0 0x7f11e019a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:10:52.706 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.706+0000 7f11de800640 1 --2- 172.21.15.83:0/620066989 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11e019ef80 0x7f11e019f430 secure :-1 s=READY pgs=2457 cs=0 l=1 rev1=1 crypto rx=0x7f11d000d6b0 tx=0x7f11d000db70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:52.707 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.707+0000 7f11d6a00640 1 -- 172.21.15.83:0/620066989 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d0017070 con 0x7f11e019ef80 2024-04-24T09:10:52.707 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.707+0000 7f11d6a00640 1 -- 172.21.15.83:0/620066989 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f11d0009cd0 con 0x7f11e019ef80 2024-04-24T09:10:52.707 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.707+0000 7f11d6a00640 1 -- 172.21.15.83:0/620066989 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d0003cc0 con 0x7f11e019ef80 2024-04-24T09:10:52.707 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.707+0000 7f11df200640 1 -- 172.21.15.83:0/620066989 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11e019fd60 con 0x7f11e019ef80 2024-04-24T09:10:52.707 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.707+0000 7f11df200640 1 -- 172.21.15.83:0/620066989 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11e01a7500 con 0x7f11e019ef80 2024-04-24T09:10:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.714+0000 7f11d6a00640 1 -- 172.21.15.83:0/620066989 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f11d0009820 con 0x7f11e019ef80 2024-04-24T09:10:52.719 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.719+0000 7f11e7f75740 1 Processor -- start 2024-04-24T09:10:52.719 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.719+0000 7f11e7f75740 1 -- start start 2024-04-24T09:10:52.719 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.720+0000 7f11d6a00640 1 --2- 172.21.15.83:0/620066989 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f11c00a3d90 0x7f11c00a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:52.720 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.721+0000 7f11dd400640 1 --2- 172.21.15.83:0/620066989 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f11c00a3d90 0x7f11c00a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:52.720 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.721+0000 7f11d6a00640 1 -- 172.21.15.83:0/620066989 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 9583+0+0 (secure 0 0 0) 0x7f11d00d0380 con 0x7f11e019ef80 2024-04-24T09:10:52.720 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.721+0000 7f11e7f75740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d91d0e93c0 0x55d91d0e97a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:52.720 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.721+0000 7f11e7f75740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d91d0e9ce0 0x55d91d0f4060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:52.721 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.721+0000 7f11e7f75740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d91d0f45a0 0x55d91d0f6990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:52.721 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.721+0000 7f11e7f75740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d91d04f900 con 0x55d91d0e9ce0 2024-04-24T09:10:52.721 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.721+0000 7f11e7f75740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d91cff5f20 con 0x55d91d0f45a0 2024-04-24T09:10:52.721 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.721+0000 7f11e7f75740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d91d04b8c0 con 0x55d91d0e93c0 2024-04-24T09:10:52.721 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.722+0000 7f11dd400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d91d0e9ce0 0x55d91d0f4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:52.722 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.722+0000 7f11de800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d91d0f45a0 0x55d91d0f6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:52.722 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.722+0000 7f11dde00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d91d0e93c0 0x55d91d0e97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:52.722 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.722+0000 7f11dde00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d91d0e93c0 0x55d91d0e97a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:47084/0 (socket says 172.21.15.83:47084) 2024-04-24T09:10:52.722 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.722+0000 7f11dde00640 1 -- 172.21.15.83:0/1375004819 learned_addr learned my addr 172.21.15.83:0/1375004819 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:10:52.723 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.723+0000 7f11dde00640 1 -- 172.21.15.83:0/1375004819 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d91d0f45a0 msgr2=0x55d91d0f6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:52.723 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.723+0000 7f11dde00640 1 --2- 172.21.15.83:0/1375004819 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d91d0f45a0 0x55d91d0f6990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:52.723 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.723+0000 7f11dde00640 1 -- 172.21.15.83:0/1375004819 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d91d0e9ce0 msgr2=0x55d91d0f4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:52.724 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.723+0000 7f11dde00640 1 --2- 172.21.15.83:0/1375004819 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d91d0e9ce0 0x55d91d0f4060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:52.724 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.723+0000 7f11dde00640 1 -- 172.21.15.83:0/1375004819 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d91d01ae10 con 0x55d91d0e93c0 2024-04-24T09:10:52.724 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.724+0000 7f11dde00640 1 --2- 172.21.15.83:0/1375004819 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d91d0e93c0 0x55d91d0e97a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f11d8010480 tx=0x7f11d8010940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31f74e4ffe93e0d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:52.724 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.725+0000 7f11dd400640 1 --2- 172.21.15.83:0/620066989 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f11c00a3d90 0x7f11c00a6250 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f11c8009620 tx=0x7f11c80304d0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:52.724 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.725+0000 7f11cf400640 1 -- 172.21.15.83:0/1375004819 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d8011760 con 0x55d91d0e93c0 2024-04-24T09:10:52.724 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.725+0000 7f11cf400640 1 -- 172.21.15.83:0/1375004819 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f11d8011db0 con 0x55d91d0e93c0 2024-04-24T09:10:52.724 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.725+0000 7f11cf400640 1 -- 172.21.15.83:0/1375004819 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d800ee70 con 0x55d91d0e93c0 2024-04-24T09:10:52.726 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.727+0000 7f11e7f75740 1 -- 172.21.15.83:0/1375004819 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d91d0e93c0 msgr2=0x55d91d0e97a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:52.726 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.727+0000 7f11e7f75740 1 --2- 172.21.15.83:0/1375004819 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d91d0e93c0 0x55d91d0e97a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f11d8010480 tx=0x7f11d8010940 comp rx=0 tx=0).stop 2024-04-24T09:10:52.727 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.727+0000 7f11e7f75740 1 -- 172.21.15.83:0/1375004819 shutdown_connections 2024-04-24T09:10:52.727 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.727+0000 7f11e7f75740 1 --2- 172.21.15.83:0/1375004819 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d91d0f45a0 0x55d91d0f6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:52.727 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.727+0000 7f11e7f75740 1 --2- 172.21.15.83:0/1375004819 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d91d0e9ce0 0x55d91d0f4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:52.727 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.727+0000 7f11e7f75740 1 --2- 172.21.15.83:0/1375004819 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d91d0e93c0 0x55d91d0e97a0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:52.727 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.727+0000 7f11e7f75740 1 -- 172.21.15.83:0/1375004819 >> 172.21.15.83:0/1375004819 conn(0x55d91d0e58d0 msgr2=0x55d91d0e5cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:52.727 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.728+0000 7f11e7f75740 1 -- 172.21.15.83:0/1375004819 shutdown_connections 2024-04-24T09:10:52.727 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.728+0000 7f11e7f75740 1 -- 172.21.15.83:0/1375004819 wait complete. 2024-04-24T09:10:52.728 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.728+0000 7f11e7f75740 1 -- start start 2024-04-24T09:10:52.728 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.729+0000 7f11e7f75740 1 Processor -- start 2024-04-24T09:10:52.729 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.729+0000 7f11e7f75740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d91d0e93c0 0x55d91d0f15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:52.729 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.729+0000 7f11e7f75740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d91d0e9ce0 0x55d91d0f1b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:52.729 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.729+0000 7f11e7f75740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d91d0f45a0 0x55d91d0f2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:52.729 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.729+0000 7f11e7f75740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d91d0ff9e0 con 0x55d91d0e9ce0 2024-04-24T09:10:52.729 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.729+0000 7f11e7f75740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d91d0092c0 con 0x55d91d0f45a0 2024-04-24T09:10:52.729 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.729+0000 7f11e7f75740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d91d0ffb60 con 0x55d91d0e93c0 2024-04-24T09:10:52.729 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.729+0000 7f11e7f75740 10 client.0 _ll_register_callbacks cb 0x55d91cd61a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:10:52.729 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.730+0000 7f11dde00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d91d0e93c0 0x55d91d0f15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:52.729 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.730+0000 7f11dde00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d91d0e93c0 0x55d91d0f15c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:47086/0 (socket says 172.21.15.83:47086) 2024-04-24T09:10:52.729 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.730+0000 7f11dde00640 1 -- 172.21.15.83:0/2822400244 learned_addr learned my addr 172.21.15.83:0/2822400244 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:10:52.730 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.730+0000 7f11de800640 1 --2- 172.21.15.83:0/2822400244 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d91d0f45a0 0x55d91d0f2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:52.730 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.730+0000 7f11dde00640 1 -- 172.21.15.83:0/2822400244 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d91d0f45a0 msgr2=0x55d91d0f2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:52.730 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.730+0000 7f11dde00640 1 --2- 172.21.15.83:0/2822400244 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d91d0f45a0 0x55d91d0f2040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:52.730 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.730+0000 7f11dd400640 1 --2- 172.21.15.83:0/2822400244 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d91d0e9ce0 0x55d91d0f1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:52.731 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.730+0000 7f11dde00640 1 -- 172.21.15.83:0/2822400244 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d91d0e9ce0 msgr2=0x55d91d0f1b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:52.731 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.730+0000 7f11dde00640 1 --2- 172.21.15.83:0/2822400244 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d91d0e9ce0 0x55d91d0f1b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:52.731 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.730+0000 7f11dde00640 1 -- 172.21.15.83:0/2822400244 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d91d019fd0 con 0x55d91d0e93c0 2024-04-24T09:10:52.731 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.730+0000 7f11de800640 1 --2- 172.21.15.83:0/2822400244 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d91d0f45a0 0x55d91d0f2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:10:52.731 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.730+0000 7f11dd400640 1 --2- 172.21.15.83:0/2822400244 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d91d0e9ce0 0x55d91d0f1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:10:52.731 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.731+0000 7f11dde00640 1 --2- 172.21.15.83:0/2822400244 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d91d0e93c0 0x55d91d0f15c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f11d800ef70 tx=0x7f11d800efa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d22ecea7b54adb00 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:52.731 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.731+0000 7f11c7e00640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-04-24T09:10:52.731 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.731+0000 7f11c7e00640 1 -- 172.21.15.83:0/2822400244 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d802a060 con 0x55d91d0e93c0 2024-04-24T09:10:52.732 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.731+0000 7f11c7e00640 1 -- 172.21.15.83:0/2822400244 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(120..120 src has 1..120) ==== 9583+0+0 (secure 0 0 0) 0x7f11d8005040 con 0x55d91d0e93c0 2024-04-24T09:10:52.732 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.731+0000 7f11e7f75740 1 -- 172.21.15.83:0/2822400244 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11d8011db0 con 0x55d91d0e93c0 2024-04-24T09:10:52.732 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.732+0000 7f11c7e00640 1 -- 172.21.15.83:0/2822400244 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f11d8031050 con 0x55d91d0e93c0 2024-04-24T09:10:52.732 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.732+0000 7f11e7f75740 10 client.29239 fetch_fsmap learned FSMap version 61 2024-04-24T09:10:52.732 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.732+0000 7f11e7f75740 1 -- 172.21.15.83:0/2822400244 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x55d91cd62210 con 0x55d91d0e93c0 2024-04-24T09:10:52.732 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.732+0000 7f11c7e00640 1 -- 172.21.15.83:0/2822400244 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f11d8011db0 con 0x55d91d0e93c0 2024-04-24T09:10:52.732 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.732+0000 7f11c7e00640 1 -- 172.21.15.83:0/2822400244 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 61) ==== 8362+0+0 (secure 0 0 0) 0x7f11d80046c0 con 0x55d91d0e93c0 2024-04-24T09:10:52.732 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.733+0000 7f11e7f75740 10 client.29239 fetch_fsmap finished waiting for FSMap version 61 2024-04-24T09:10:52.732 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.733+0000 7f11e7f75740 10 client.29239 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:10:52.733 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.733+0000 7f11e7f75740 20 client.29239 populate_metadata read hostname 'smithi083' 2024-04-24T09:10:52.733 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.733+0000 7f11e7f75740 1 --2- 172.21.15.83:0/2822400244 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d91d10cda0 0x55d91d10f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:52.733 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.733+0000 7f11e7f75740 4 client.29239 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:10:52.733 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.733+0000 7f11e7f75740 1 -- 172.21.15.83:0/2822400244 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d91d05d320 con 0x55d91d10cda0 2024-04-24T09:10:52.733 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.734+0000 7f11dd400640 1 --2- 172.21.15.83:0/2822400244 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d91d10cda0 0x55d91d10f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:52.734 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.735+0000 7f11dd400640 1 --2- 172.21.15.83:0/2822400244 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d91d10cda0 0x55d91d10f190 crc :-1 s=READY pgs=479 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c92c94c67707f3b9 server_cookie=548693704c5a8d52 in_seq=0 out_seq=0 2024-04-24T09:10:52.734 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.735+0000 7f11c7e00640 10 client.29239 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:10:52.736 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.737+0000 7f11c7e00640 1 -- 172.21.15.83:0/2822400244 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d91d05d320 con 0x55d91d10cda0 2024-04-24T09:10:52.736 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.737+0000 7f11c7e00640 10 client.29239 handle_command_reply: tid=0 2024-04-24T09:10:52.736 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.737+0000 7f11e7f75740 1 client.29239 shutdown 2024-04-24T09:10:52.737 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.738+0000 7f11e7f75740 1 -- 172.21.15.83:0/2822400244 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d91d0e93c0 msgr2=0x55d91d0f15c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:52.737 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.738+0000 7f11e7f75740 1 --2- 172.21.15.83:0/2822400244 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d91d0e93c0 0x55d91d0f15c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f11d800ef70 tx=0x7f11d800efa0 comp rx=0 tx=0).stop 2024-04-24T09:10:52.737 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.738+0000 7f11e7f75740 1 -- 172.21.15.83:0/2822400244 shutdown_connections 2024-04-24T09:10:52.737 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.738+0000 7f11e7f75740 1 --2- 172.21.15.83:0/2822400244 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d91d10cda0 0x55d91d10f190 crc :-1 s=READY pgs=479 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:52.738 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.738+0000 7f11e7f75740 1 --2- 172.21.15.83:0/2822400244 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d91d0f45a0 0x55d91d0f2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:52.738 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.738+0000 7f11e7f75740 1 --2- 172.21.15.83:0/2822400244 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d91d0e9ce0 0x55d91d0f1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:52.738 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.738+0000 7f11e7f75740 1 --2- 172.21.15.83:0/2822400244 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d91d0e93c0 0x55d91d0f15c0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:52.738 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.738+0000 7f11e7f75740 1 -- 172.21.15.83:0/2822400244 >> 172.21.15.83:0/2822400244 conn(0x55d91d0e58d0 msgr2=0x55d91d0eb650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:52.738 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.738+0000 7f11c7e00640 0 client.29239 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:10:52.738 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.738+0000 7f11e7f75740 1 -- 172.21.15.83:0/2822400244 shutdown_connections 2024-04-24T09:10:52.738 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.738+0000 7f11e7f75740 1 -- 172.21.15.83:0/2822400244 wait complete. 2024-04-24T09:10:52.738 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.738+0000 7f11e7f75740 20 client.29239 trim_cache size 0 max 16384 2024-04-24T09:10:52.763 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.760+0000 7f11e7f75740 1 Processor -- start 2024-04-24T09:10:52.763 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.760+0000 7f11e7f75740 1 -- start start 2024-04-24T09:10:52.763 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.761+0000 7f11e7f75740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d91d0e93c0 0x55d91d0ec000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:52.764 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.761+0000 7f11e7f75740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d91d0e9ce0 0x55d91d0e4910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:52.764 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.761+0000 7f11e7f75740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d91d0f45a0 0x55d91d111dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:52.764 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.761+0000 7f11e7f75740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f11e01ad0c0 con 0x55d91d0e9ce0 2024-04-24T09:10:52.764 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.761+0000 7f11e7f75740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f11e01aec00 con 0x55d91d0e93c0 2024-04-24T09:10:52.764 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.761+0000 7f11e7f75740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f11e01ad790 con 0x55d91d0f45a0 2024-04-24T09:10:52.764 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.761+0000 7f11dde00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d91d0e93c0 0x55d91d0ec000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:52.764 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.761+0000 7f11dde00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d91d0e93c0 0x55d91d0ec000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:33818/0 (socket says 172.21.15.83:33818) 2024-04-24T09:10:52.764 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.761+0000 7f11dde00640 1 -- 172.21.15.83:0/3144226231 learned_addr learned my addr 172.21.15.83:0/3144226231 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:10:52.764 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.761+0000 7f11dd400640 1 --2- 172.21.15.83:0/3144226231 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d91d0e9ce0 0x55d91d0e4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:52.764 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.761+0000 7f11de800640 1 --2- 172.21.15.83:0/3144226231 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d91d0f45a0 0x55d91d111dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:52.764 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.761+0000 7f11dde00640 1 -- 172.21.15.83:0/3144226231 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d91d0f45a0 msgr2=0x55d91d111dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:52.764 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.761+0000 7f11dde00640 1 --2- 172.21.15.83:0/3144226231 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d91d0f45a0 0x55d91d111dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:52.764 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.761+0000 7f11dde00640 1 -- 172.21.15.83:0/3144226231 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d91d0e9ce0 msgr2=0x55d91d0e4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:52.765 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.761+0000 7f11dde00640 1 --2- 172.21.15.83:0/3144226231 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d91d0e9ce0 0x55d91d0e4910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:52.765 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.762+0000 7f11dde00640 1 -- 172.21.15.83:0/3144226231 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d91d0f0570 con 0x55d91d0e93c0 2024-04-24T09:10:52.765 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.762+0000 7f11dd400640 1 --2- 172.21.15.83:0/3144226231 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d91d0e9ce0 0x55d91d0e4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:10:52.765 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.762+0000 7f11dde00640 1 --2- 172.21.15.83:0/3144226231 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d91d0e93c0 0x55d91d0ec000 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f11d8000c00 tx=0x7f11d800b630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34a0da6a86445b91 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:52.765 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.762+0000 7f11cea00640 1 -- 172.21.15.83:0/3144226231 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d803a070 con 0x55d91d0e93c0 2024-04-24T09:10:52.765 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.762+0000 7f11cea00640 1 -- 172.21.15.83:0/3144226231 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f11d800bc90 con 0x55d91d0e93c0 2024-04-24T09:10:52.765 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.762+0000 7f11cea00640 1 -- 172.21.15.83:0/3144226231 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d802a040 con 0x55d91d0e93c0 2024-04-24T09:10:52.765 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.763+0000 7f11e7f75740 1 -- 172.21.15.83:0/3144226231 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d91d0e93c0 msgr2=0x55d91d0ec000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:52.765 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.763+0000 7f11e7f75740 1 --2- 172.21.15.83:0/3144226231 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d91d0e93c0 0x55d91d0ec000 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f11d8000c00 tx=0x7f11d800b630 comp rx=0 tx=0).stop 2024-04-24T09:10:52.765 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.764+0000 7f11e7f75740 1 -- 172.21.15.83:0/3144226231 shutdown_connections 2024-04-24T09:10:52.765 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.764+0000 7f11e7f75740 1 --2- 172.21.15.83:0/3144226231 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d91d0f45a0 0x55d91d111dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:52.765 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.764+0000 7f11e7f75740 1 --2- 172.21.15.83:0/3144226231 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d91d0e9ce0 0x55d91d0e4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:52.765 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.764+0000 7f11e7f75740 1 --2- 172.21.15.83:0/3144226231 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d91d0e93c0 0x55d91d0ec000 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:52.765 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.764+0000 7f11e7f75740 1 -- 172.21.15.83:0/3144226231 >> 172.21.15.83:0/3144226231 conn(0x55d91d0e58d0 msgr2=0x55d91d0e4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:52.765 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.764+0000 7f11e7f75740 1 -- 172.21.15.83:0/3144226231 shutdown_connections 2024-04-24T09:10:52.766 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.764+0000 7f11e7f75740 1 -- 172.21.15.83:0/3144226231 wait complete. 2024-04-24T09:10:52.766 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.764+0000 7f11e7f75740 1 -- start start 2024-04-24T09:10:52.767 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.765+0000 7f11e7f75740 1 Processor -- start 2024-04-24T09:10:52.767 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.765+0000 7f11e7f75740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d91d0e93c0 0x55d91d16bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:52.768 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.765+0000 7f11e7f75740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d91d0e9ce0 0x55d91d16c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:52.768 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.765+0000 7f11e7f75740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d91d0f45a0 0x55d91d16c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:52.768 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.765+0000 7f11e7f75740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f11e01ae530 con 0x55d91d0f45a0 2024-04-24T09:10:52.768 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.765+0000 7f11e7f75740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f11e01ade60 con 0x55d91d0e9ce0 2024-04-24T09:10:52.768 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.765+0000 7f11e7f75740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f11e01ab0e0 con 0x55d91d0e93c0 2024-04-24T09:10:52.768 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.765+0000 7f11e7f75740 10 client.0 _ll_register_callbacks cb 0x7f11c80303a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:10:52.768 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.766+0000 7f11dde00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d91d0e93c0 0x55d91d16bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:52.768 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.766+0000 7f11dde00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d91d0e93c0 0x55d91d16bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:47100/0 (socket says 172.21.15.83:47100) 2024-04-24T09:10:52.768 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.766+0000 7f11dde00640 1 -- 172.21.15.83:0/561708946 learned_addr learned my addr 172.21.15.83:0/561708946 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:10:52.768 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.766+0000 7f11dd400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d91d0e9ce0 0x55d91d16c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:52.768 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.766+0000 7f11de800640 1 --2- 172.21.15.83:0/561708946 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d91d0f45a0 0x55d91d16c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:52.768 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.766+0000 7f11dde00640 1 -- 172.21.15.83:0/561708946 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d91d0e9ce0 msgr2=0x55d91d16c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:52.768 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.766+0000 7f11dde00640 1 --2- 172.21.15.83:0/561708946 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d91d0e9ce0 0x55d91d16c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:52.768 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.766+0000 7f11dde00640 1 -- 172.21.15.83:0/561708946 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d91d0f45a0 msgr2=0x55d91d16c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.766+0000 7f11dde00640 1 --2- 172.21.15.83:0/561708946 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d91d0f45a0 0x55d91d16c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.766+0000 7f11dde00640 1 -- 172.21.15.83:0/561708946 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d91ce2e620 con 0x55d91d0e93c0 2024-04-24T09:10:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.767+0000 7f11dde00640 1 --2- 172.21.15.83:0/561708946 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d91d0e93c0 0x55d91d16bbe0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f11d8004ac0 tx=0x7f11d8004f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3b5303bca11bcef server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.767+0000 7f11c7e00640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-04-24T09:10:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.767+0000 7f11c7e00640 1 -- 172.21.15.83:0/561708946 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d803a070 con 0x55d91d0e93c0 2024-04-24T09:10:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.767+0000 7f11c7e00640 1 -- 172.21.15.83:0/561708946 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(120..120 src has 1..120) ==== 9583+0+0 (secure 0 0 0) 0x7f11d8011bd0 con 0x55d91d0e93c0 2024-04-24T09:10:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.768+0000 7f11e7f75740 1 -- 172.21.15.83:0/561708946 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11d800bc90 con 0x55d91d0e93c0 2024-04-24T09:10:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.768+0000 7f11c7e00640 1 -- 172.21.15.83:0/561708946 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f11d802a040 con 0x55d91d0e93c0 2024-04-24T09:10:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.768+0000 7f11e7f75740 10 client.29245 fetch_fsmap learned FSMap version 61 2024-04-24T09:10:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.768+0000 7f11e7f75740 1 -- 172.21.15.83:0/561708946 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x55d91d0fd290 con 0x55d91d0e93c0 2024-04-24T09:10:52.770 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.769+0000 7f11c7e00640 1 -- 172.21.15.83:0/561708946 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f11d800bc90 con 0x55d91d0e93c0 2024-04-24T09:10:52.770 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.769+0000 7f11c7e00640 1 -- 172.21.15.83:0/561708946 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 61) ==== 8362+0+0 (secure 0 0 0) 0x7f11d8005240 con 0x55d91d0e93c0 2024-04-24T09:10:52.770 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.769+0000 7f11e7f75740 10 client.29245 fetch_fsmap finished waiting for FSMap version 61 2024-04-24T09:10:52.770 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.769+0000 7f11e7f75740 10 client.29245 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:10:52.771 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.769+0000 7f11e7f75740 20 client.29245 populate_metadata read hostname 'smithi083' 2024-04-24T09:10:52.771 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.769+0000 7f11e7f75740 1 --2- 172.21.15.83:0/561708946 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d91d0fd960 0x55d91d0fdd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:52.771 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.769+0000 7f11e7f75740 4 client.29245 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T09:10:52.771 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.769+0000 7f11e7f75740 1 -- 172.21.15.83:0/561708946 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d91d057190 con 0x55d91d0fd960 2024-04-24T09:10:52.771 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.770+0000 7f11dd400640 1 --2- 172.21.15.83:0/561708946 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d91d0fd960 0x55d91d0fdd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:52.771 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.770+0000 7f11c7e00640 10 client.29245 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:10:52.771 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.770+0000 7f11dd400640 1 --2- 172.21.15.83:0/561708946 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d91d0fd960 0x55d91d0fdd70 crc :-1 s=READY pgs=480 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb1236d2455511f6 server_cookie=27dba32e69fea320 in_seq=0 out_seq=0 2024-04-24T09:10:52.873 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.873+0000 7f11c7e00640 1 -- 172.21.15.83:0/561708946 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d91d057190 con 0x55d91d0fd960 2024-04-24T09:10:52.873 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.873+0000 7f11c7e00640 10 client.29245 handle_command_reply: tid=0 2024-04-24T09:10:52.873 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.873+0000 7f11e7f75740 1 client.29245 shutdown 2024-04-24T09:10:52.873 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.874+0000 7f11e7f75740 1 -- 172.21.15.83:0/561708946 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d91d0e93c0 msgr2=0x55d91d16bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:52.873 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.874+0000 7f11e7f75740 1 --2- 172.21.15.83:0/561708946 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d91d0e93c0 0x55d91d16bbe0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f11d8004ac0 tx=0x7f11d8004f60 comp rx=0 tx=0).stop 2024-04-24T09:10:52.873 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.874+0000 7f11e7f75740 1 -- 172.21.15.83:0/561708946 shutdown_connections 2024-04-24T09:10:52.873 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.874+0000 7f11e7f75740 1 --2- 172.21.15.83:0/561708946 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d91d0fd960 0x55d91d0fdd70 crc :-1 s=READY pgs=480 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:52.874 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.874+0000 7f11e7f75740 1 --2- 172.21.15.83:0/561708946 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d91d0f45a0 0x55d91d16c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:52.874 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.874+0000 7f11e7f75740 1 --2- 172.21.15.83:0/561708946 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d91d0e9ce0 0x55d91d16c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:52.874 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.874+0000 7f11e7f75740 1 --2- 172.21.15.83:0/561708946 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d91d0e93c0 0x55d91d16bbe0 secure :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f11d8004ac0 tx=0x7f11d8004f60 comp rx=0 tx=0).stop 2024-04-24T09:10:52.874 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.874+0000 7f11e7f75740 1 -- 172.21.15.83:0/561708946 >> 172.21.15.83:0/561708946 conn(0x55d91d0e58d0 msgr2=0x55d91d0e4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:52.874 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.874+0000 7f11c7e00640 0 client.29245 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:10:52.875 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.874+0000 7f11e7f75740 1 -- 172.21.15.83:0/561708946 shutdown_connections 2024-04-24T09:10:52.875 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.874+0000 7f11e7f75740 1 -- 172.21.15.83:0/561708946 wait complete. 2024-04-24T09:10:52.875 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.875+0000 7f11e7f75740 20 client.29245 trim_cache size 0 max 16384 2024-04-24T09:10:52.876 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T09:10:52.876 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T09:10:52.876 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T09:10:52.876 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T09:10:52.877 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.876+0000 7f11df200640 1 -- 172.21.15.83:0/620066989 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f11c00a3d90 msgr2=0x7f11c00a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:52.877 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.876+0000 7f11df200640 1 --2- 172.21.15.83:0/620066989 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f11c00a3d90 0x7f11c00a6250 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f11c8009620 tx=0x7f11c80304d0 comp rx=0 tx=0).stop 2024-04-24T09:10:52.877 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.876+0000 7f11df200640 1 -- 172.21.15.83:0/620066989 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11e019ef80 msgr2=0x7f11e019f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:52.877 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.876+0000 7f11df200640 1 --2- 172.21.15.83:0/620066989 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11e019ef80 0x7f11e019f430 secure :-1 s=READY pgs=2457 cs=0 l=1 rev1=1 crypto rx=0x7f11d000d6b0 tx=0x7f11d000db70 comp rx=0 tx=0).stop 2024-04-24T09:10:52.877 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.876+0000 7f11df200640 1 -- 172.21.15.83:0/620066989 shutdown_connections 2024-04-24T09:10:52.877 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.876+0000 7f11df200640 1 --2- 172.21.15.83:0/620066989 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f11c00a3d90 0x7f11c00a6250 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:52.877 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.876+0000 7f11df200640 1 --2- 172.21.15.83:0/620066989 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11e019ef80 0x7f11e019f430 unknown :-1 s=CLOSED pgs=2457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:52.877 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.876+0000 7f11df200640 1 --2- 172.21.15.83:0/620066989 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f11e0109370 0x7f11e019ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:52.877 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.876+0000 7f11df200640 1 --2- 172.21.15.83:0/620066989 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11e01042a0 0x7f11e019a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:52.877 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.876+0000 7f11df200640 1 -- 172.21.15.83:0/620066989 >> 172.21.15.83:0/620066989 conn(0x7f11e00ffae0 msgr2=0x7f11e0100560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:52.877 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.876+0000 7f11df200640 1 -- 172.21.15.83:0/620066989 shutdown_connections 2024-04-24T09:10:52.877 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:10:52.877+0000 7f11df200640 1 -- 172.21.15.83:0/620066989 wait complete. 2024-04-24T09:10:53.177 INFO:tasks.workunit.client.0.smithi083.stdout: 10 40888 6.69 MB/sec execute 204 sec latency 117.857 ms 2024-04-24T09:10:54.177 INFO:tasks.workunit.client.0.smithi083.stdout: 10 41004 6.67 MB/sec execute 205 sec latency 83.909 ms 2024-04-24T09:10:54.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:10:53 smithi083 ceph-mon[27907]: pgmap v607: 129 pgs: 129 active+clean; 674 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.6 MiB/s wr, 356 op/s 2024-04-24T09:10:54.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:10:53 smithi138 ceph-mon[31441]: pgmap v607: 129 pgs: 129 active+clean; 674 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.6 MiB/s wr, 356 op/s 2024-04-24T09:10:54.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:10:53 smithi189 ceph-mon[31791]: pgmap v607: 129 pgs: 129 active+clean; 674 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.6 MiB/s wr, 356 op/s 2024-04-24T09:10:55.177 INFO:tasks.workunit.client.0.smithi083.stdout: 10 41082 6.67 MB/sec execute 206 sec latency 489.392 ms 2024-04-24T09:10:56.177 INFO:tasks.workunit.client.0.smithi083.stdout: 10 41223 6.66 MB/sec execute 207 sec latency 487.606 ms 2024-04-24T09:10:56.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:10:55 smithi083 ceph-mon[27907]: pgmap v608: 129 pgs: 129 active+clean; 674 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.8 MiB/s wr, 212 op/s 2024-04-24T09:10:56.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:10:55 smithi138 ceph-mon[31441]: pgmap v608: 129 pgs: 129 active+clean; 674 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.8 MiB/s wr, 212 op/s 2024-04-24T09:10:56.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:10:55 smithi189 ceph-mon[31791]: pgmap v608: 129 pgs: 129 active+clean; 674 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.8 MiB/s wr, 212 op/s 2024-04-24T09:10:57.178 INFO:tasks.workunit.client.0.smithi083.stdout: 10 41379 6.67 MB/sec execute 208 sec latency 73.278 ms 2024-04-24T09:10:57.749 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T09:10:57.749 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T09:10:57.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:57.999+0000 7f2b96c00640 1 -- 172.21.15.83:0/2581347682 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2b9010e820 msgr2=0x7f2b90110c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:57.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:57.999+0000 7f2b96c00640 1 --2- 172.21.15.83:0/2581347682 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2b9010e820 0x7f2b90110c10 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f2b88009be0 tx=0x7f2b8802a6c0 comp rx=0 tx=0).stop 2024-04-24T09:10:57.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:57.999+0000 7f2b96c00640 1 -- 172.21.15.83:0/2581347682 shutdown_connections 2024-04-24T09:10:58.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:57.999+0000 7f2b96c00640 1 --2- 172.21.15.83:0/2581347682 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2b9010e820 0x7f2b90110c10 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:58.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:57.999+0000 7f2b96c00640 1 --2- 172.21.15.83:0/2581347682 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b90101620 0x7f2b9010e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:58.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:57.999+0000 7f2b96c00640 1 --2- 172.21.15.83:0/2581347682 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2b90100c70 0x7f2b90101050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:58.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:57.999+0000 7f2b96c00640 1 -- 172.21.15.83:0/2581347682 >> 172.21.15.83:0/2581347682 conn(0x7f2b900fc880 msgr2=0x7f2b900feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:58.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:57.999+0000 7f2b96c00640 1 -- 172.21.15.83:0/2581347682 shutdown_connections 2024-04-24T09:10:58.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:57.999+0000 7f2b96c00640 1 -- 172.21.15.83:0/2581347682 wait complete. 2024-04-24T09:10:58.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.000+0000 7f2b96c00640 1 Processor -- start 2024-04-24T09:10:58.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.001+0000 7f2b96c00640 1 -- start start 2024-04-24T09:10:58.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.001+0000 7f2b96c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b90100c70 0x7f2b9010e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:58.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.001+0000 7f2b96c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2b90101620 0x7f2b9010b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:58.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.001+0000 7f2b96c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2b9010e820 0x7f2b9010b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:58.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.001+0000 7f2b96c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2b90114320 con 0x7f2b90100c70 2024-04-24T09:10:58.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.001+0000 7f2b96c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2b901141a0 con 0x7f2b9010e820 2024-04-24T09:10:58.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.001+0000 7f2b96c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2b901144a0 con 0x7f2b90101620 2024-04-24T09:10:58.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.001+0000 7f2b95800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b90100c70 0x7f2b9010e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:58.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.001+0000 7f2b95800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b90100c70 0x7f2b9010e180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39038/0 (socket says 172.21.15.83:39038) 2024-04-24T09:10:58.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.001+0000 7f2b95800640 1 -- 172.21.15.83:0/808608487 learned_addr learned my addr 172.21.15.83:0/808608487 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:10:58.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.002+0000 7f2b94e00640 1 --2- 172.21.15.83:0/808608487 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2b90101620 0x7f2b9010b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:58.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.002+0000 7f2b96200640 1 --2- 172.21.15.83:0/808608487 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2b9010e820 0x7f2b9010b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:58.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.002+0000 7f2b95800640 1 -- 172.21.15.83:0/808608487 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2b90101620 msgr2=0x7f2b9010b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:58.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.002+0000 7f2b95800640 1 --2- 172.21.15.83:0/808608487 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2b90101620 0x7f2b9010b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:58.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.002+0000 7f2b95800640 1 -- 172.21.15.83:0/808608487 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2b9010e820 msgr2=0x7f2b9010b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:58.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.002+0000 7f2b95800640 1 --2- 172.21.15.83:0/808608487 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2b9010e820 0x7f2b9010b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:58.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.002+0000 7f2b95800640 1 -- 172.21.15.83:0/808608487 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b901ab060 con 0x7f2b90100c70 2024-04-24T09:10:58.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.002+0000 7f2b95800640 1 --2- 172.21.15.83:0/808608487 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b90100c70 0x7f2b9010e180 secure :-1 s=READY pgs=2458 cs=0 l=1 rev1=1 crypto rx=0x7f2b8400d770 tx=0x7f2b8400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:58.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.005+0000 7f2b8e000640 1 -- 172.21.15.83:0/808608487 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b84017070 con 0x7f2b90100c70 2024-04-24T09:10:58.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.005+0000 7f2b96c00640 1 -- 172.21.15.83:0/808608487 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b901ab230 con 0x7f2b90100c70 2024-04-24T09:10:58.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.005+0000 7f2b96c00640 1 -- 172.21.15.83:0/808608487 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b901ab720 con 0x7f2b90100c70 2024-04-24T09:10:58.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.005+0000 7f2b8e000640 1 -- 172.21.15.83:0/808608487 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2b840054d0 con 0x7f2b90100c70 2024-04-24T09:10:58.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.005+0000 7f2b8e000640 1 -- 172.21.15.83:0/808608487 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b84004cf0 con 0x7f2b90100c70 2024-04-24T09:10:58.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.009+0000 7f2b9aff7740 1 Processor -- start 2024-04-24T09:10:58.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.009+0000 7f2b9aff7740 1 -- start start 2024-04-24T09:10:58.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.009+0000 7f2b9aff7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c1b886a30 0x563c1b886e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:58.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.009+0000 7f2b9aff7740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c1b887350 0x563c1b8914e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:58.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.009+0000 7f2b9aff7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c1b891a20 0x563c1b893e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:58.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.009+0000 7f2b9aff7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x563c1b7ec1f0 con 0x563c1b891a20 2024-04-24T09:10:58.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.009+0000 7f2b9aff7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563c1b792350 con 0x563c1b886a30 2024-04-24T09:10:58.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.009+0000 7f2b9aff7740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x563c1b7e81b0 con 0x563c1b887350 2024-04-24T09:10:58.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.010+0000 7f2b96200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c1b886a30 0x563c1b886e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:58.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.010+0000 7f2b96200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c1b886a30 0x563c1b886e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:33852/0 (socket says 172.21.15.83:33852) 2024-04-24T09:10:58.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.010+0000 7f2b96200640 1 -- 172.21.15.83:0/1387868397 learned_addr learned my addr 172.21.15.83:0/1387868397 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:10:58.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.010+0000 7f2b8e000640 1 -- 172.21.15.83:0/808608487 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2b84010460 con 0x7f2b90100c70 2024-04-24T09:10:58.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.010+0000 7f2b94e00640 1 --2- 172.21.15.83:0/1387868397 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c1b891a20 0x563c1b893e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:58.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.010+0000 7f2b96200640 1 -- 172.21.15.83:0/1387868397 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c1b887350 msgr2=0x563c1b8914e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:58.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.010+0000 7f2b96200640 1 --2- 172.21.15.83:0/1387868397 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c1b887350 0x563c1b8914e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:58.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.010+0000 7f2b96200640 1 -- 172.21.15.83:0/1387868397 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c1b891a20 msgr2=0x563c1b893e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:58.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.010+0000 7f2b96200640 1 --2- 172.21.15.83:0/1387868397 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c1b891a20 0x563c1b893e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:58.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.010+0000 7f2b96200640 1 -- 172.21.15.83:0/1387868397 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c1b7e6f50 con 0x563c1b886a30 2024-04-24T09:10:58.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.010+0000 7f2b94e00640 1 --2- 172.21.15.83:0/1387868397 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c1b891a20 0x563c1b893e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:10:58.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.011+0000 7f2b96200640 1 --2- 172.21.15.83:0/1387868397 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c1b886a30 0x563c1b886e10 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f2b88006010 tx=0x7f2b88030430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed5f9d5a0b2c4fa3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:58.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.011+0000 7f2b82a00640 1 -- 172.21.15.83:0/1387868397 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b88040030 con 0x563c1b886a30 2024-04-24T09:10:58.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.011+0000 7f2b82a00640 1 -- 172.21.15.83:0/1387868397 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2b88037040 con 0x563c1b886a30 2024-04-24T09:10:58.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.011+0000 7f2b82a00640 1 -- 172.21.15.83:0/1387868397 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b88036070 con 0x563c1b886a30 2024-04-24T09:10:58.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.012+0000 7f2b9aff7740 1 -- 172.21.15.83:0/1387868397 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c1b886a30 msgr2=0x563c1b886e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:58.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.012+0000 7f2b9aff7740 1 --2- 172.21.15.83:0/1387868397 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c1b886a30 0x563c1b886e10 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f2b88006010 tx=0x7f2b88030430 comp rx=0 tx=0).stop 2024-04-24T09:10:58.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.012+0000 7f2b9aff7740 1 -- 172.21.15.83:0/1387868397 shutdown_connections 2024-04-24T09:10:58.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.012+0000 7f2b9aff7740 1 --2- 172.21.15.83:0/1387868397 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c1b891a20 0x563c1b893e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:58.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.012+0000 7f2b9aff7740 1 --2- 172.21.15.83:0/1387868397 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c1b887350 0x563c1b8914e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:58.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.012+0000 7f2b9aff7740 1 --2- 172.21.15.83:0/1387868397 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c1b886a30 0x563c1b886e10 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:58.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.012+0000 7f2b9aff7740 1 -- 172.21.15.83:0/1387868397 >> 172.21.15.83:0/1387868397 conn(0x563c1b882d50 msgr2=0x563c1b883130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:58.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.012+0000 7f2b8e000640 1 --2- 172.21.15.83:0/808608487 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f2b780a3da0 0x7f2b780a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:58.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.013+0000 7f2b9aff7740 1 -- 172.21.15.83:0/1387868397 shutdown_connections 2024-04-24T09:10:58.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.013+0000 7f2b8e000640 1 -- 172.21.15.83:0/808608487 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 9583+0+0 (secure 0 0 0) 0x7f2b840d0610 con 0x7f2b90100c70 2024-04-24T09:10:58.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.013+0000 7f2b9aff7740 1 -- 172.21.15.83:0/1387868397 wait complete. 2024-04-24T09:10:58.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.013+0000 7f2b94e00640 1 --2- 172.21.15.83:0/808608487 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f2b780a3da0 0x7f2b780a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:58.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.013+0000 7f2b9aff7740 1 -- start start 2024-04-24T09:10:58.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.014+0000 7f2b9aff7740 1 Processor -- start 2024-04-24T09:10:58.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.014+0000 7f2b94e00640 1 --2- 172.21.15.83:0/808608487 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f2b780a3da0 0x7f2b780a6260 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x563c1b883ee0 tx=0x7f2b7c009260 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:58.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.014+0000 7f2b9aff7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c1b886a30 0x563c1b88e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:58.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.014+0000 7f2b9aff7740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c1b887350 0x563c1b88ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:58.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.014+0000 7f2b9aff7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c1b891a20 0x563c1b88f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:58.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.014+0000 7f2b9aff7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x563c1b8947b0 con 0x563c1b886a30 2024-04-24T09:10:58.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.014+0000 7f2b9aff7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563c1b7a5690 con 0x563c1b891a20 2024-04-24T09:10:58.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.014+0000 7f2b9aff7740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x563c1b894930 con 0x563c1b887350 2024-04-24T09:10:58.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.014+0000 7f2b9aff7740 10 client.0 _ll_register_callbacks cb 0x563c1b7e1650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:10:58.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.015+0000 7f2b96200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c1b891a20 0x563c1b88f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:58.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.015+0000 7f2b95800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c1b886a30 0x563c1b88e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:58.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.015+0000 7f2b96200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c1b891a20 0x563c1b88f3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:33856/0 (socket says 172.21.15.83:33856) 2024-04-24T09:10:58.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.015+0000 7f2b96200640 1 -- 172.21.15.83:0/3701050769 learned_addr learned my addr 172.21.15.83:0/3701050769 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:10:58.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.015+0000 7f2b94e00640 1 --2- 172.21.15.83:0/3701050769 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c1b887350 0x563c1b88ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:58.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.015+0000 7f2b96200640 1 -- 172.21.15.83:0/3701050769 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c1b887350 msgr2=0x563c1b88ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:58.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.015+0000 7f2b96200640 1 --2- 172.21.15.83:0/3701050769 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c1b887350 0x563c1b88ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:58.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.015+0000 7f2b96200640 1 -- 172.21.15.83:0/3701050769 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c1b886a30 msgr2=0x563c1b88e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:58.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.015+0000 7f2b96200640 1 --2- 172.21.15.83:0/3701050769 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c1b886a30 0x563c1b88e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:58.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.015+0000 7f2b96200640 1 -- 172.21.15.83:0/3701050769 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c1b7a4920 con 0x563c1b891a20 2024-04-24T09:10:58.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.015+0000 7f2b95800640 1 --2- 172.21.15.83:0/3701050769 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c1b886a30 0x563c1b88e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:10:58.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.015+0000 7f2b96200640 1 --2- 172.21.15.83:0/3701050769 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c1b891a20 0x563c1b88f3d0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f2b880092f0 tx=0x7f2b88030660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4ece639064c7e40 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:58.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.015+0000 7f2b77400640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:10:58.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.016+0000 7f2b77400640 1 -- 172.21.15.83:0/3701050769 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b88036040 con 0x563c1b891a20 2024-04-24T09:10:58.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.016+0000 7f2b77400640 1 -- 172.21.15.83:0/3701050769 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(120..120 src has 1..120) ==== 9583+0+0 (secure 0 0 0) 0x7f2b88003b70 con 0x563c1b891a20 2024-04-24T09:10:58.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.016+0000 7f2b9aff7740 1 -- 172.21.15.83:0/3701050769 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b88037040 con 0x563c1b891a20 2024-04-24T09:10:58.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.016+0000 7f2b77400640 1 -- 172.21.15.83:0/3701050769 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f2b88056050 con 0x563c1b891a20 2024-04-24T09:10:58.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.016+0000 7f2b9aff7740 10 client.39323 fetch_fsmap learned FSMap version 61 2024-04-24T09:10:58.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.016+0000 7f2b9aff7740 1 -- 172.21.15.83:0/3701050769 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x563c1b5c87d0 con 0x563c1b891a20 2024-04-24T09:10:58.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.017+0000 7f2b77400640 1 -- 172.21.15.83:0/3701050769 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2b88037040 con 0x563c1b891a20 2024-04-24T09:10:58.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.017+0000 7f2b77400640 1 -- 172.21.15.83:0/3701050769 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 61) ==== 8362+0+0 (secure 0 0 0) 0x7f2b88031440 con 0x563c1b891a20 2024-04-24T09:10:58.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.017+0000 7f2b9aff7740 10 client.39323 fetch_fsmap finished waiting for FSMap version 61 2024-04-24T09:10:58.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.017+0000 7f2b9aff7740 10 client.39323 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:10:58.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.017+0000 7f2b9aff7740 20 client.39323 populate_metadata read hostname 'smithi083' 2024-04-24T09:10:58.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.017+0000 7f2b9aff7740 1 --2- 172.21.15.83:0/3701050769 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x563c1b8a1b70 0x563c1b8a3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:58.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.017+0000 7f2b9aff7740 4 client.39323 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:10:58.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.017+0000 7f2b9aff7740 1 -- 172.21.15.83:0/3701050769 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563c1b7f9c10 con 0x563c1b8a1b70 2024-04-24T09:10:58.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.018+0000 7f2b95800640 1 --2- 172.21.15.83:0/3701050769 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x563c1b8a1b70 0x563c1b8a3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:58.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.018+0000 7f2b95800640 1 --2- 172.21.15.83:0/3701050769 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x563c1b8a1b70 0x563c1b8a3f60 crc :-1 s=READY pgs=481 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2b1347235137b5 server_cookie=efe3a975d78a2650 in_seq=0 out_seq=0 2024-04-24T09:10:58.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.018+0000 7f2b77400640 10 client.39323 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:10:58.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.019+0000 7f2b77400640 1 -- 172.21.15.83:0/3701050769 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563c1b7f9c10 con 0x563c1b8a1b70 2024-04-24T09:10:58.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.019+0000 7f2b77400640 10 client.39323 handle_command_reply: tid=0 2024-04-24T09:10:58.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.019+0000 7f2b9aff7740 1 client.39323 shutdown 2024-04-24T09:10:58.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.019+0000 7f2b9aff7740 1 -- 172.21.15.83:0/3701050769 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c1b891a20 msgr2=0x563c1b88f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:58.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.019+0000 7f2b9aff7740 1 --2- 172.21.15.83:0/3701050769 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c1b891a20 0x563c1b88f3d0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f2b880092f0 tx=0x7f2b88030660 comp rx=0 tx=0).stop 2024-04-24T09:10:58.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.020+0000 7f2b9aff7740 1 -- 172.21.15.83:0/3701050769 shutdown_connections 2024-04-24T09:10:58.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.020+0000 7f2b9aff7740 1 --2- 172.21.15.83:0/3701050769 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x563c1b8a1b70 0x563c1b8a3f60 crc :-1 s=READY pgs=481 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:58.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.020+0000 7f2b9aff7740 1 --2- 172.21.15.83:0/3701050769 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c1b891a20 0x563c1b88f3d0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:58.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.020+0000 7f2b9aff7740 1 --2- 172.21.15.83:0/3701050769 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c1b887350 0x563c1b88ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:58.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.020+0000 7f2b9aff7740 1 --2- 172.21.15.83:0/3701050769 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c1b886a30 0x563c1b88e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:58.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.020+0000 7f2b9aff7740 1 -- 172.21.15.83:0/3701050769 >> 172.21.15.83:0/3701050769 conn(0x563c1b882d50 msgr2=0x563c1b881fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:58.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.020+0000 7f2b77400640 0 client.39323 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:10:58.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.020+0000 7f2b9aff7740 1 -- 172.21.15.83:0/3701050769 shutdown_connections 2024-04-24T09:10:58.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.020+0000 7f2b9aff7740 1 -- 172.21.15.83:0/3701050769 wait complete. 2024-04-24T09:10:58.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.020+0000 7f2b9aff7740 20 client.39323 trim_cache size 0 max 16384 2024-04-24T09:10:58.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.049+0000 7f2b9aff7740 1 Processor -- start 2024-04-24T09:10:58.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.049+0000 7f2b9aff7740 1 -- start start 2024-04-24T09:10:58.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.050+0000 7f2b9aff7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c1b886a30 0x563c1b889190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:58.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.050+0000 7f2b9aff7740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c1b887350 0x563c1b88f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:58.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.050+0000 7f2b9aff7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c1b891a20 0x563c1b8a9500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:58.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.050+0000 7f2b9aff7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2b70010110 con 0x563c1b891a20 2024-04-24T09:10:58.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.050+0000 7f2b9aff7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2b70011c50 con 0x563c1b886a30 2024-04-24T09:10:58.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.050+0000 7f2b9aff7740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2b700107e0 con 0x563c1b887350 2024-04-24T09:10:58.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.050+0000 7f2b96200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c1b891a20 0x563c1b8a9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:58.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.050+0000 7f2b96200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c1b891a20 0x563c1b8a9500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39068/0 (socket says 172.21.15.83:39068) 2024-04-24T09:10:58.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.050+0000 7f2b95800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c1b886a30 0x563c1b889190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:58.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.050+0000 7f2b96200640 1 -- 172.21.15.83:0/1269690083 learned_addr learned my addr 172.21.15.83:0/1269690083 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:10:58.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.051+0000 7f2b94e00640 1 --2- 172.21.15.83:0/1269690083 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c1b887350 0x563c1b88f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:58.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.051+0000 7f2b95800640 1 -- 172.21.15.83:0/1269690083 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c1b887350 msgr2=0x563c1b88f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:58.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.051+0000 7f2b95800640 1 --2- 172.21.15.83:0/1269690083 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c1b887350 0x563c1b88f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:58.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.051+0000 7f2b95800640 1 -- 172.21.15.83:0/1269690083 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c1b891a20 msgr2=0x563c1b8a9500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:58.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.051+0000 7f2b95800640 1 --2- 172.21.15.83:0/1269690083 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c1b891a20 0x563c1b8a9500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:58.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.051+0000 7f2b95800640 1 -- 172.21.15.83:0/1269690083 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c1b4fe3f0 con 0x563c1b886a30 2024-04-24T09:10:58.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.051+0000 7f2b94e00640 1 --2- 172.21.15.83:0/1269690083 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c1b887350 0x563c1b88f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:10:58.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.051+0000 7f2b96200640 1 --2- 172.21.15.83:0/1269690083 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c1b891a20 0x563c1b8a9500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:10:58.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.052+0000 7f2b95800640 1 --2- 172.21.15.83:0/1269690083 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c1b886a30 0x563c1b889190 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f2b840cd910 tx=0x7f2b840d2040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8d92149aacf9a41 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:58.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.052+0000 7f2b82000640 1 -- 172.21.15.83:0/1269690083 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b840b4030 con 0x563c1b886a30 2024-04-24T09:10:58.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.052+0000 7f2b82000640 1 -- 172.21.15.83:0/1269690083 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2b84015020 con 0x563c1b886a30 2024-04-24T09:10:58.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.052+0000 7f2b82000640 1 -- 172.21.15.83:0/1269690083 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b84020040 con 0x563c1b886a30 2024-04-24T09:10:58.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.053+0000 7f2b9aff7740 1 -- 172.21.15.83:0/1269690083 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c1b886a30 msgr2=0x563c1b889190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:58.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.053+0000 7f2b9aff7740 1 --2- 172.21.15.83:0/1269690083 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c1b886a30 0x563c1b889190 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f2b840cd910 tx=0x7f2b840d2040 comp rx=0 tx=0).stop 2024-04-24T09:10:58.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.054+0000 7f2b9aff7740 1 -- 172.21.15.83:0/1269690083 shutdown_connections 2024-04-24T09:10:58.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.054+0000 7f2b9aff7740 1 --2- 172.21.15.83:0/1269690083 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c1b891a20 0x563c1b8a9500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:58.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.054+0000 7f2b9aff7740 1 --2- 172.21.15.83:0/1269690083 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c1b887350 0x563c1b88f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:58.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.054+0000 7f2b9aff7740 1 --2- 172.21.15.83:0/1269690083 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c1b886a30 0x563c1b889190 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:58.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.054+0000 7f2b9aff7740 1 -- 172.21.15.83:0/1269690083 >> 172.21.15.83:0/1269690083 conn(0x563c1b882d50 msgr2=0x563c1b881fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:58.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.054+0000 7f2b9aff7740 1 -- 172.21.15.83:0/1269690083 shutdown_connections 2024-04-24T09:10:58.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.054+0000 7f2b9aff7740 1 -- 172.21.15.83:0/1269690083 wait complete. 2024-04-24T09:10:58.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.055+0000 7f2b9aff7740 1 -- start start 2024-04-24T09:10:58.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.055+0000 7f2b9aff7740 1 Processor -- start 2024-04-24T09:10:58.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.055+0000 7f2b9aff7740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c1b886a30 0x563c1b899c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:58.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.056+0000 7f2b9aff7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c1b887350 0x563c1b89a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:58.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.056+0000 7f2b9aff7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c1b891a20 0x563c1b88ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:58.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.056+0000 7f2b9aff7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2b70011580 con 0x563c1b887350 2024-04-24T09:10:58.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.056+0000 7f2b9aff7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2b70010eb0 con 0x563c1b891a20 2024-04-24T09:10:58.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.056+0000 7f2b9aff7740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2b7000e110 con 0x563c1b886a30 2024-04-24T09:10:58.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.056+0000 7f2b9aff7740 10 client.0 _ll_register_callbacks cb 0x7f2b7c005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:10:58.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.056+0000 7f2b95800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c1b886a30 0x563c1b899c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:58.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.056+0000 7f2b94e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c1b887350 0x563c1b89a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:58.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.056+0000 7f2b94e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c1b887350 0x563c1b89a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39080/0 (socket says 172.21.15.83:39080) 2024-04-24T09:10:58.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.056+0000 7f2b94e00640 1 -- 172.21.15.83:0/3644280318 learned_addr learned my addr 172.21.15.83:0/3644280318 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:10:58.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.057+0000 7f2b96200640 1 --2- 172.21.15.83:0/3644280318 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c1b891a20 0x563c1b88ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:58.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.057+0000 7f2b94e00640 1 -- 172.21.15.83:0/3644280318 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c1b886a30 msgr2=0x563c1b899c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:58.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.057+0000 7f2b94e00640 1 --2- 172.21.15.83:0/3644280318 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c1b886a30 0x563c1b899c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:58.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.057+0000 7f2b94e00640 1 -- 172.21.15.83:0/3644280318 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c1b891a20 msgr2=0x563c1b88ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:58.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.057+0000 7f2b94e00640 1 --2- 172.21.15.83:0/3644280318 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c1b891a20 0x563c1b88ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:58.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.057+0000 7f2b94e00640 1 -- 172.21.15.83:0/3644280318 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c1b4be0f0 con 0x563c1b887350 2024-04-24T09:10:58.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.057+0000 7f2b96200640 1 --2- 172.21.15.83:0/3644280318 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c1b891a20 0x563c1b88ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:10:58.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.058+0000 7f2b94e00640 1 --2- 172.21.15.83:0/3644280318 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c1b887350 0x563c1b89a150 secure :-1 s=READY pgs=2459 cs=0 l=1 rev1=1 crypto rx=0x7f2b9010e760 tx=0x7f2b7c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea8d509d1d1d671c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:58.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.058+0000 7f2b77400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:10:58.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.058+0000 7f2b77400640 1 -- 172.21.15.83:0/3644280318 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b7c021a30 con 0x563c1b887350 2024-04-24T09:10:58.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.058+0000 7f2b77400640 1 -- 172.21.15.83:0/3644280318 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(120..120 src has 1..120) ==== 9583+0+0 (secure 0 0 0) 0x7f2b7c021bd0 con 0x563c1b887350 2024-04-24T09:10:58.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.058+0000 7f2b9aff7740 1 -- 172.21.15.83:0/3644280318 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b84015020 con 0x563c1b887350 2024-04-24T09:10:58.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.059+0000 7f2b77400640 1 -- 172.21.15.83:0/3644280318 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f2b7c021dd0 con 0x563c1b887350 2024-04-24T09:10:58.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.059+0000 7f2b9aff7740 10 client.22236 fetch_fsmap learned FSMap version 61 2024-04-24T09:10:58.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.059+0000 7f2b9aff7740 1 -- 172.21.15.83:0/3644280318 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x563c1b7b1810 con 0x563c1b887350 2024-04-24T09:10:58.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.059+0000 7f2b77400640 1 -- 172.21.15.83:0/3644280318 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2b84015020 con 0x563c1b887350 2024-04-24T09:10:58.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.060+0000 7f2b77400640 1 -- 172.21.15.83:0/3644280318 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 61) ==== 8362+0+0 (secure 0 0 0) 0x7f2b7c028070 con 0x563c1b887350 2024-04-24T09:10:58.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.060+0000 7f2b9aff7740 10 client.22236 fetch_fsmap finished waiting for FSMap version 61 2024-04-24T09:10:58.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.060+0000 7f2b9aff7740 10 client.22236 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:10:58.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.060+0000 7f2b9aff7740 20 client.22236 populate_metadata read hostname 'smithi083' 2024-04-24T09:10:58.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.060+0000 7f2b9aff7740 1 --2- 172.21.15.83:0/3644280318 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x563c1b907c90 0x563c1b90a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:58.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.060+0000 7f2b9aff7740 4 client.22236 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T09:10:58.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.060+0000 7f2b9aff7740 1 -- 172.21.15.83:0/3644280318 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563c1b7f3a80 con 0x563c1b907c90 2024-04-24T09:10:58.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.060+0000 7f2b95800640 1 --2- 172.21.15.83:0/3644280318 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x563c1b907c90 0x563c1b90a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:58.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.061+0000 7f2b95800640 1 --2- 172.21.15.83:0/3644280318 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x563c1b907c90 0x563c1b90a0b0 crc :-1 s=READY pgs=482 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99d01f11f6119a85 server_cookie=fa651e329fb2a6b9 in_seq=0 out_seq=0 2024-04-24T09:10:58.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.061+0000 7f2b77400640 10 client.22236 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:10:58.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.062+0000 7f2b77400640 1 -- 172.21.15.83:0/3644280318 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x563c1b7f3a80 con 0x563c1b907c90 2024-04-24T09:10:58.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.062+0000 7f2b77400640 10 client.22236 handle_command_reply: tid=0 2024-04-24T09:10:58.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.062+0000 7f2b9aff7740 1 client.22236 shutdown 2024-04-24T09:10:58.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.062+0000 7f2b9aff7740 1 -- 172.21.15.83:0/3644280318 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c1b887350 msgr2=0x563c1b89a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:58.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.062+0000 7f2b9aff7740 1 --2- 172.21.15.83:0/3644280318 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c1b887350 0x563c1b89a150 secure :-1 s=READY pgs=2459 cs=0 l=1 rev1=1 crypto rx=0x7f2b9010e760 tx=0x7f2b7c0046c0 comp rx=0 tx=0).stop 2024-04-24T09:10:58.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.062+0000 7f2b9aff7740 1 -- 172.21.15.83:0/3644280318 shutdown_connections 2024-04-24T09:10:58.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.062+0000 7f2b9aff7740 1 --2- 172.21.15.83:0/3644280318 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x563c1b907c90 0x563c1b90a0b0 crc :-1 s=READY pgs=482 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:58.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.062+0000 7f2b9aff7740 1 --2- 172.21.15.83:0/3644280318 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c1b891a20 0x563c1b88ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:58.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.062+0000 7f2b77400640 0 client.22236 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:10:58.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.062+0000 7f2b9aff7740 1 --2- 172.21.15.83:0/3644280318 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c1b887350 0x563c1b89a150 unknown :-1 s=CLOSED pgs=2459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:58.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.062+0000 7f2b9aff7740 1 --2- 172.21.15.83:0/3644280318 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c1b886a30 0x563c1b899c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:58.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.062+0000 7f2b9aff7740 1 -- 172.21.15.83:0/3644280318 >> 172.21.15.83:0/3644280318 conn(0x563c1b882d50 msgr2=0x563c1b890180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:58.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.063+0000 7f2b9aff7740 1 -- 172.21.15.83:0/3644280318 shutdown_connections 2024-04-24T09:10:58.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.063+0000 7f2b9aff7740 1 -- 172.21.15.83:0/3644280318 wait complete. 2024-04-24T09:10:58.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.063+0000 7f2b9aff7740 20 client.22236 trim_cache size 0 max 16384 2024-04-24T09:10:58.064 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T09:10:58.065 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-04-24T09:10:58.065 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-04-24T09:10:58.065 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T09:10:58.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.064+0000 7f2b96c00640 1 -- 172.21.15.83:0/808608487 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f2b780a3da0 msgr2=0x7f2b780a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:58.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.064+0000 7f2b96c00640 1 --2- 172.21.15.83:0/808608487 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f2b780a3da0 0x7f2b780a6260 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x563c1b883ee0 tx=0x7f2b7c009260 comp rx=0 tx=0).stop 2024-04-24T09:10:58.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.064+0000 7f2b96c00640 1 -- 172.21.15.83:0/808608487 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b90100c70 msgr2=0x7f2b9010e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:58.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.064+0000 7f2b96c00640 1 --2- 172.21.15.83:0/808608487 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b90100c70 0x7f2b9010e180 secure :-1 s=READY pgs=2458 cs=0 l=1 rev1=1 crypto rx=0x7f2b8400d770 tx=0x7f2b8400dc30 comp rx=0 tx=0).stop 2024-04-24T09:10:58.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.064+0000 7f2b96c00640 1 -- 172.21.15.83:0/808608487 shutdown_connections 2024-04-24T09:10:58.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.064+0000 7f2b96c00640 1 --2- 172.21.15.83:0/808608487 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f2b780a3da0 0x7f2b780a6260 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:58.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.064+0000 7f2b96c00640 1 --2- 172.21.15.83:0/808608487 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2b9010e820 0x7f2b9010b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:58.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.064+0000 7f2b96c00640 1 --2- 172.21.15.83:0/808608487 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2b90101620 0x7f2b9010b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:58.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.064+0000 7f2b96c00640 1 --2- 172.21.15.83:0/808608487 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b90100c70 0x7f2b9010e180 unknown :-1 s=CLOSED pgs=2458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:58.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.064+0000 7f2b96c00640 1 -- 172.21.15.83:0/808608487 >> 172.21.15.83:0/808608487 conn(0x7f2b900fc880 msgr2=0x7f2b900fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:58.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.065+0000 7f2b96c00640 1 -- 172.21.15.83:0/808608487 shutdown_connections 2024-04-24T09:10:58.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.065+0000 7f2b96c00640 1 -- 172.21.15.83:0/808608487 wait complete. 2024-04-24T09:10:58.086 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:10:58 smithi083 ceph-mon[27907]: pgmap v609: 129 pgs: 129 active+clean; 674 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.8 MiB/s wr, 212 op/s 2024-04-24T09:10:58.088 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T09:10:58.088 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T09:10:58.177 INFO:tasks.workunit.client.0.smithi083.stdout: 10 41537 6.68 MB/sec execute 209 sec latency 89.233 ms 2024-04-24T09:10:58.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.362+0000 7f59fcc00640 1 -- 172.21.15.83:0/3738445074 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59f8104c40 msgr2=0x7f59f8108c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:58.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.362+0000 7f59fcc00640 1 --2- 172.21.15.83:0/3738445074 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59f8104c40 0x7f59f8108c80 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f59e0009900 tx=0x7f59e002a400 comp rx=0 tx=0).stop 2024-04-24T09:10:58.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.362+0000 7f59fcc00640 1 -- 172.21.15.83:0/3738445074 shutdown_connections 2024-04-24T09:10:58.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.362+0000 7f59fcc00640 1 --2- 172.21.15.83:0/3738445074 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f59f8109360 0x7f59f8110f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:58.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.362+0000 7f59fcc00640 1 --2- 172.21.15.83:0/3738445074 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59f8104c40 0x7f59f8108c80 secure :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f59e0009900 tx=0x7f59e002a400 comp rx=0 tx=0).stop 2024-04-24T09:10:58.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.362+0000 7f59fcc00640 1 --2- 172.21.15.83:0/3738445074 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59f8104290 0x7f59f8104670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:58.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.362+0000 7f59fcc00640 1 -- 172.21.15.83:0/3738445074 >> 172.21.15.83:0/3738445074 conn(0x7f59f80ffad0 msgr2=0x7f59f8101ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:58.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.362+0000 7f59fcc00640 1 -- 172.21.15.83:0/3738445074 shutdown_connections 2024-04-24T09:10:58.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.362+0000 7f59fcc00640 1 -- 172.21.15.83:0/3738445074 wait complete. 2024-04-24T09:10:58.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.364+0000 7f59fcc00640 1 Processor -- start 2024-04-24T09:10:58.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.364+0000 7f59fcc00640 1 -- start start 2024-04-24T09:10:58.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.364+0000 7f59fcc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59f8104290 0x7f59f81a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:58.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.364+0000 7f59fcc00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59f8109360 0x7f59f81a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:58.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.364+0000 7f59fcc00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f59f819a860 0x7f59f819ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:58.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.364+0000 7f59fcc00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f59f81142d0 con 0x7f59f8104290 2024-04-24T09:10:58.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.364+0000 7f59fcc00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f59f8114150 con 0x7f59f8109360 2024-04-24T09:10:58.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.364+0000 7f59fcc00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f59f8114450 con 0x7f59f819a860 2024-04-24T09:10:58.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.365+0000 7f59f6a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59f8109360 0x7f59f81a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:58.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.365+0000 7f59f7400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59f8104290 0x7f59f81a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:58.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.365+0000 7f59f6a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59f8109360 0x7f59f81a0bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:33886/0 (socket says 172.21.15.83:33886) 2024-04-24T09:10:58.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.365+0000 7f59f6a00640 1 -- 172.21.15.83:0/2822061963 learned_addr learned my addr 172.21.15.83:0/2822061963 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:10:58.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.365+0000 7f59f7e00640 1 --2- 172.21.15.83:0/2822061963 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f59f819a860 0x7f59f819ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:58.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.365+0000 7f59f6a00640 1 -- 172.21.15.83:0/2822061963 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f59f819a860 msgr2=0x7f59f819ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:58.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.365+0000 7f59f6a00640 1 --2- 172.21.15.83:0/2822061963 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f59f819a860 0x7f59f819ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:58.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.365+0000 7f59f6a00640 1 -- 172.21.15.83:0/2822061963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59f8104290 msgr2=0x7f59f81a06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:58.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.365+0000 7f59f6a00640 1 --2- 172.21.15.83:0/2822061963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59f8104290 0x7f59f81a06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:58.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.366+0000 7f59f6a00640 1 -- 172.21.15.83:0/2822061963 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59f819b380 con 0x7f59f8109360 2024-04-24T09:10:58.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.366+0000 7f59f7400640 1 --2- 172.21.15.83:0/2822061963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59f8104290 0x7f59f81a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:10:58.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.366+0000 7f59f6a00640 1 --2- 172.21.15.83:0/2822061963 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59f8109360 0x7f59f81a0bf0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f59e002a9f0 tx=0x7f59e0003c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:58.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.367+0000 7f59efe00640 1 -- 172.21.15.83:0/2822061963 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59e003f070 con 0x7f59f8109360 2024-04-24T09:10:58.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.367+0000 7f59efe00640 1 -- 172.21.15.83:0/2822061963 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f59e0031d60 con 0x7f59f8109360 2024-04-24T09:10:58.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.367+0000 7f59efe00640 1 -- 172.21.15.83:0/2822061963 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59e0035070 con 0x7f59f8109360 2024-04-24T09:10:58.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.367+0000 7f59fcc00640 1 -- 172.21.15.83:0/2822061963 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59f819b610 con 0x7f59f8109360 2024-04-24T09:10:58.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.367+0000 7f59fcc00640 1 -- 172.21.15.83:0/2822061963 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59f81a77b0 con 0x7f59f8109360 2024-04-24T09:10:58.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.369+0000 7f59fcc00640 1 -- 172.21.15.83:0/2822061963 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59f8068d40 con 0x7f59f8109360 2024-04-24T09:10:58.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.372+0000 7f59efe00640 1 -- 172.21.15.83:0/2822061963 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f59e003b4f0 con 0x7f59f8109360 2024-04-24T09:10:58.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.373+0000 7f59efe00640 1 --2- 172.21.15.83:0/2822061963 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f59dc0a3ef0 0x7f59dc0a63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:58.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.373+0000 7f59efe00640 1 -- 172.21.15.83:0/2822061963 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 9583+0+0 (secure 0 0 0) 0x7f59e00f1080 con 0x7f59f8109360 2024-04-24T09:10:58.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.374+0000 7f59f7400640 1 --2- 172.21.15.83:0/2822061963 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f59dc0a3ef0 0x7f59dc0a63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:58.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.374+0000 7f59f7400640 1 --2- 172.21.15.83:0/2822061963 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f59dc0a3ef0 0x7f59dc0a63b0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f59f819b9a0 tx=0x7f59e8005f90 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:58.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.378+0000 7f59efe00640 1 -- 172.21.15.83:0/2822061963 <== mon.2 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f59e0048020 con 0x7f59f8109360 2024-04-24T09:10:58.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:10:58 smithi138 ceph-mon[31441]: pgmap v609: 129 pgs: 129 active+clean; 674 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.8 MiB/s wr, 212 op/s 2024-04-24T09:10:58.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:10:58 smithi189 ceph-mon[31791]: pgmap v609: 129 pgs: 129 active+clean; 674 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.8 MiB/s wr, 212 op/s 2024-04-24T09:10:58.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.814+0000 7f59fcc00640 1 -- 172.21.15.83:0/2822061963 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f59f802d620 con 0x7f59f8109360 2024-04-24T09:10:58.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.815+0000 7f59efe00640 1 -- 172.21.15.83:0/2822061963 <== mon.2 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) ==== 94+0+11944 (secure 0 0 0) 0x7f59e00bd320 con 0x7f59f8109360 2024-04-24T09:10:58.818 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T09:10:58.818 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":61,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/358872257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":358872257},{"type":"v1","addr":"172.21.15.83:6835","nonce":358872257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3157070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3157070851},{"type":"v1","addr":"172.21.15.83:6837","nonce":3157070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":61,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T08:48:29.422576+0000","modified":"2024-04-24T09:09:34.468831+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34520,"mds_1":34493,"mds_2":34451,"mds_3":34427,"mds_4":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14658":{"gid":14658,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6839/4085470844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":4085470844},{"type":"v1","addr":"172.21.15.83:6839","nonce":4085470844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14676":{"gid":14676,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6841/2634754210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":2634754210},{"type":"v1","addr":"172.21.15.83:6841","nonce":2634754210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24382":{"gid":24382,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/1859921310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1859921310},{"type":"v1","addr":"172.21.15.189:6833","nonce":1859921310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/2241923197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2241923197},{"type":"v1","addr":"172.21.15.189:6835","nonce":2241923197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.189:6839/2548965336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":2548965336},{"type":"v1","addr":"172.21.15.189:6839","nonce":2548965336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26686":{"gid":26686,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6837/1678817161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":1678817161},{"type":"v1","addr":"172.21.15.189:6837","nonce":1678817161}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.138:6833/1157967658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1157967658},{"type":"v1","addr":"172.21.15.138:6833","nonce":1157967658}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.138:6835/1909219758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1909219758},{"type":"v1","addr":"172.21.15.138:6835","nonce":1909219758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34493":{"gid":34493,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.138:6837/900566754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":900566754},{"type":"v1","addr":"172.21.15.138:6837","nonce":900566754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34520":{"gid":34520,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.138:6839/2236932393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2236932393},{"type":"v1","addr":"172.21.15.138:6839","nonce":2236932393}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34520,"qdb_cluster":[24454,34520,34427,34451,34493]},"id":1}]} 2024-04-24T09:10:58.818 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 61 2024-04-24T09:10:58.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.818+0000 7f59fcc00640 1 -- 172.21.15.83:0/2822061963 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f59dc0a3ef0 msgr2=0x7f59dc0a63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:58.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.818+0000 7f59fcc00640 1 --2- 172.21.15.83:0/2822061963 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f59dc0a3ef0 0x7f59dc0a63b0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f59f819b9a0 tx=0x7f59e8005f90 comp rx=0 tx=0).stop 2024-04-24T09:10:58.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.818+0000 7f59fcc00640 1 -- 172.21.15.83:0/2822061963 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59f8109360 msgr2=0x7f59f81a0bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:58.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.818+0000 7f59fcc00640 1 --2- 172.21.15.83:0/2822061963 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59f8109360 0x7f59f81a0bf0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f59e002a9f0 tx=0x7f59e0003c80 comp rx=0 tx=0).stop 2024-04-24T09:10:58.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.819+0000 7f59fcc00640 1 -- 172.21.15.83:0/2822061963 shutdown_connections 2024-04-24T09:10:58.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.819+0000 7f59fcc00640 1 --2- 172.21.15.83:0/2822061963 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f59dc0a3ef0 0x7f59dc0a63b0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:58.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.819+0000 7f59fcc00640 1 --2- 172.21.15.83:0/2822061963 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f59f819a860 0x7f59f819ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:58.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.819+0000 7f59fcc00640 1 --2- 172.21.15.83:0/2822061963 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59f8109360 0x7f59f81a0bf0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:58.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.819+0000 7f59fcc00640 1 --2- 172.21.15.83:0/2822061963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59f8104290 0x7f59f81a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:58.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.819+0000 7f59fcc00640 1 -- 172.21.15.83:0/2822061963 >> 172.21.15.83:0/2822061963 conn(0x7f59f80ffad0 msgr2=0x7f59f81003e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:58.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.819+0000 7f59fcc00640 1 -- 172.21.15.83:0/2822061963 shutdown_connections 2024-04-24T09:10:58.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:58.820+0000 7f59fcc00640 1 -- 172.21.15.83:0/2822061963 wait complete. 2024-04-24T09:10:58.846 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-24T09:10:59.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.094+0000 7fee92e00640 1 -- 172.21.15.83:0/2563128894 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee8c108610 msgr2=0x7fee8c108a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:59.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.094+0000 7fee92e00640 1 --2- 172.21.15.83:0/2563128894 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee8c108610 0x7fee8c108a90 secure :-1 s=READY pgs=2460 cs=0 l=1 rev1=1 crypto rx=0x7fee78009930 tx=0x7fee7802a420 comp rx=0 tx=0).stop 2024-04-24T09:10:59.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.094+0000 7fee92e00640 1 -- 172.21.15.83:0/2563128894 shutdown_connections 2024-04-24T09:10:59.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.094+0000 7fee92e00640 1 --2- 172.21.15.83:0/2563128894 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fee8c108fd0 0x7fee8c115490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:59.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.094+0000 7fee92e00640 1 --2- 172.21.15.83:0/2563128894 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee8c108610 0x7fee8c108a90 unknown :-1 s=CLOSED pgs=2460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:59.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.094+0000 7fee92e00640 1 --2- 172.21.15.83:0/2563128894 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee8c10e660 0x7fee8c10ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:59.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.094+0000 7fee92e00640 1 -- 172.21.15.83:0/2563128894 >> 172.21.15.83:0/2563128894 conn(0x7fee8c076540 msgr2=0x7fee8c078960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:59.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.095+0000 7fee92e00640 1 -- 172.21.15.83:0/2563128894 shutdown_connections 2024-04-24T09:10:59.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.095+0000 7fee92e00640 1 -- 172.21.15.83:0/2563128894 wait complete. 2024-04-24T09:10:59.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.096+0000 7fee92e00640 1 Processor -- start 2024-04-24T09:10:59.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.096+0000 7fee92e00640 1 -- start start 2024-04-24T09:10:59.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.097+0000 7fee92e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee8c108610 0x7fee8c19ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:59.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.097+0000 7fee92e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fee8c108fd0 0x7fee8c19f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:59.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.097+0000 7fee92e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee8c10e660 0x7fee8c1a36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:59.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.097+0000 7fee92e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fee8c1188a0 con 0x7fee8c108610 2024-04-24T09:10:59.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.097+0000 7fee92e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fee8c118720 con 0x7fee8c10e660 2024-04-24T09:10:59.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.097+0000 7fee92e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fee8c118a20 con 0x7fee8c108fd0 2024-04-24T09:10:59.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.097+0000 7fee8be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee8c108610 0x7fee8c19ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:59.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.097+0000 7fee8be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee8c108610 0x7fee8c19ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58316/0 (socket says 172.21.15.83:58316) 2024-04-24T09:10:59.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.097+0000 7fee8be00640 1 -- 172.21.15.83:0/1002038685 learned_addr learned my addr 172.21.15.83:0/1002038685 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:10:59.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.097+0000 7fee8b400640 1 --2- 172.21.15.83:0/1002038685 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fee8c108fd0 0x7fee8c19f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:59.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.098+0000 7fee8be00640 1 -- 172.21.15.83:0/1002038685 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fee8c108fd0 msgr2=0x7fee8c19f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:59.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.098+0000 7fee8be00640 1 --2- 172.21.15.83:0/1002038685 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fee8c108fd0 0x7fee8c19f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:59.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.098+0000 7fee8be00640 1 -- 172.21.15.83:0/1002038685 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee8c10e660 msgr2=0x7fee8c1a36b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T09:10:59.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.098+0000 7fee8be00640 1 --2- 172.21.15.83:0/1002038685 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee8c10e660 0x7fee8c1a36b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:59.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.098+0000 7fee8be00640 1 -- 172.21.15.83:0/1002038685 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee8c1a3de0 con 0x7fee8c108610 2024-04-24T09:10:59.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.098+0000 7fee8be00640 1 --2- 172.21.15.83:0/1002038685 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee8c108610 0x7fee8c19ee40 secure :-1 s=READY pgs=2461 cs=0 l=1 rev1=1 crypto rx=0x7fee7c00e9c0 tx=0x7fee7c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:59.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.099+0000 7fee88c00640 1 -- 172.21.15.83:0/1002038685 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee7c00cd80 con 0x7fee8c108610 2024-04-24T09:10:59.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.099+0000 7fee88c00640 1 -- 172.21.15.83:0/1002038685 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fee7c0035f0 con 0x7fee8c108610 2024-04-24T09:10:59.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.099+0000 7fee88c00640 1 -- 172.21.15.83:0/1002038685 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee7c010690 con 0x7fee8c108610 2024-04-24T09:10:59.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.099+0000 7fee92e00640 1 -- 172.21.15.83:0/1002038685 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee8c1a40d0 con 0x7fee8c108610 2024-04-24T09:10:59.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.099+0000 7fee92e00640 1 -- 172.21.15.83:0/1002038685 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee8c1ab960 con 0x7fee8c108610 2024-04-24T09:10:59.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.104+0000 7fee95c07740 1 Processor -- start 2024-04-24T09:10:59.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.104+0000 7fee88c00640 1 -- 172.21.15.83:0/1002038685 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fee7c0037a0 con 0x7fee8c108610 2024-04-24T09:10:59.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.105+0000 7fee95c07740 1 -- start start 2024-04-24T09:10:59.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.105+0000 7fee95c07740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ec07a1c950 0x55ec07a1cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:59.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.105+0000 7fee95c07740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec07a1d270 0x55ec07a27400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:59.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.105+0000 7fee95c07740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec07a27940 0x55ec07a29d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:59.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.105+0000 7fee95c07740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ec079821d0 con 0x55ec07a27940 2024-04-24T09:10:59.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.105+0000 7fee95c07740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ec07928450 con 0x55ec07a1d270 2024-04-24T09:10:59.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.105+0000 7fee95c07740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55ec0797e190 con 0x55ec07a1c950 2024-04-24T09:10:59.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.106+0000 7fee8b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec07a27940 0x55ec07a29d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:59.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.106+0000 7fee8b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec07a27940 0x55ec07a29d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58332/0 (socket says 172.21.15.83:58332) 2024-04-24T09:10:59.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.106+0000 7fee8b400640 1 -- 172.21.15.83:0/647379455 learned_addr learned my addr 172.21.15.83:0/647379455 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:10:59.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.106+0000 7fee90e00640 1 --2- 172.21.15.83:0/647379455 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ec07a1c950 0x55ec07a1cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:59.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.106+0000 7fee8be00640 1 --2- 172.21.15.83:0/647379455 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec07a1d270 0x55ec07a27400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:59.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.108+0000 7fee8b400640 1 -- 172.21.15.83:0/647379455 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ec07a1c950 msgr2=0x55ec07a1cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:59.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.108+0000 7fee8b400640 1 --2- 172.21.15.83:0/647379455 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ec07a1c950 0x55ec07a1cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:59.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.108+0000 7fee8b400640 1 -- 172.21.15.83:0/647379455 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec07a1d270 msgr2=0x55ec07a27400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:59.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.108+0000 7fee8b400640 1 --2- 172.21.15.83:0/647379455 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec07a1d270 0x55ec07a27400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:59.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.108+0000 7fee8b400640 1 -- 172.21.15.83:0/647379455 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec079e5310 con 0x55ec07a27940 2024-04-24T09:10:59.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.109+0000 7fee8be00640 1 --2- 172.21.15.83:0/647379455 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec07a1d270 0x55ec07a27400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:10:59.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.110+0000 7fee8b400640 1 --2- 172.21.15.83:0/647379455 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec07a27940 0x55ec07a29d30 secure :-1 s=READY pgs=2462 cs=0 l=1 rev1=1 crypto rx=0x7fee7802aa10 tx=0x7fee78005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3f1136b60a26557 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:59.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.111+0000 7fee81600640 1 -- 172.21.15.83:0/647379455 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee78035040 con 0x55ec07a27940 2024-04-24T09:10:59.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.111+0000 7fee88c00640 1 --2- 172.21.15.83:0/1002038685 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fee700a3d60 0x7fee700a6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:59.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.111+0000 7fee81600640 1 -- 172.21.15.83:0/647379455 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fee780043d0 con 0x55ec07a27940 2024-04-24T09:10:59.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.112+0000 7fee81600640 1 -- 172.21.15.83:0/647379455 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee78035040 con 0x55ec07a27940 2024-04-24T09:10:59.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.113+0000 7fee95c07740 1 -- 172.21.15.83:0/647379455 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec07a27940 msgr2=0x55ec07a29d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:59.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.113+0000 7fee95c07740 1 --2- 172.21.15.83:0/647379455 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec07a27940 0x55ec07a29d30 secure :-1 s=READY pgs=2462 cs=0 l=1 rev1=1 crypto rx=0x7fee7802aa10 tx=0x7fee78005fd0 comp rx=0 tx=0).stop 2024-04-24T09:10:59.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.114+0000 7fee95c07740 1 -- 172.21.15.83:0/647379455 shutdown_connections 2024-04-24T09:10:59.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.114+0000 7fee95c07740 1 --2- 172.21.15.83:0/647379455 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec07a27940 0x55ec07a29d30 unknown :-1 s=CLOSED pgs=2462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:59.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.114+0000 7fee95c07740 1 --2- 172.21.15.83:0/647379455 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec07a1d270 0x55ec07a27400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:59.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.114+0000 7fee95c07740 1 --2- 172.21.15.83:0/647379455 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ec07a1c950 0x55ec07a1cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:59.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.114+0000 7fee95c07740 1 -- 172.21.15.83:0/647379455 >> 172.21.15.83:0/647379455 conn(0x55ec07a18c70 msgr2=0x55ec07a19050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:59.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.114+0000 7fee88c00640 1 -- 172.21.15.83:0/1002038685 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 9583+0+0 (secure 0 0 0) 0x7fee7c0d10a0 con 0x7fee8c108610 2024-04-24T09:10:59.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.114+0000 7fee95c07740 1 -- 172.21.15.83:0/647379455 shutdown_connections 2024-04-24T09:10:59.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.114+0000 7fee90e00640 1 --2- 172.21.15.83:0/1002038685 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fee700a3d60 0x7fee700a6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:59.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.115+0000 7fee95c07740 1 -- 172.21.15.83:0/647379455 wait complete. 2024-04-24T09:10:59.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.116+0000 7fee95c07740 1 -- start start 2024-04-24T09:10:59.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.117+0000 7fee90e00640 1 --2- 172.21.15.83:0/1002038685 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fee700a3d60 0x7fee700a6220 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fee84005fd0 tx=0x7fee84009500 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:59.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.117+0000 7fee95c07740 1 Processor -- start 2024-04-24T09:10:59.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.118+0000 7fee95c07740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec07a1c950 0x55ec07a27340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:59.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.118+0000 7fee95c07740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ec07a1d270 0x55ec07a24630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:59.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.118+0000 7fee95c07740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec07a27940 0x55ec07a24c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:59.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.118+0000 7fee95c07740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ec07a2a7c0 con 0x55ec07a27940 2024-04-24T09:10:59.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.118+0000 7fee95c07740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ec0793b810 con 0x55ec07a1c950 2024-04-24T09:10:59.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.118+0000 7fee95c07740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55ec07a2a940 con 0x55ec07a1d270 2024-04-24T09:10:59.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.118+0000 7fee95c07740 10 client.0 _ll_register_callbacks cb 0x55ec07909140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:10:59.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.118+0000 7fee8b400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ec07a1d270 0x55ec07a24630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:59.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.119+0000 7fee8b400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ec07a1d270 0x55ec07a24630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:39556/0 (socket says 172.21.15.83:39556) 2024-04-24T09:10:59.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.119+0000 7fee8b400640 1 -- 172.21.15.83:0/146834321 learned_addr learned my addr 172.21.15.83:0/146834321 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:10:59.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.119+0000 7fee90e00640 1 --2- 172.21.15.83:0/146834321 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec07a27940 0x55ec07a24c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:59.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.119+0000 7fee8be00640 1 --2- 172.21.15.83:0/146834321 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec07a1c950 0x55ec07a27340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:59.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.119+0000 7fee90e00640 1 -- 172.21.15.83:0/146834321 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ec07a1d270 msgr2=0x55ec07a24630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:59.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.119+0000 7fee90e00640 1 --2- 172.21.15.83:0/146834321 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ec07a1d270 0x55ec07a24630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:59.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.119+0000 7fee90e00640 1 -- 172.21.15.83:0/146834321 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec07a1c950 msgr2=0x55ec07a27340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:59.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.119+0000 7fee90e00640 1 --2- 172.21.15.83:0/146834321 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec07a1c950 0x55ec07a27340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:59.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.119+0000 7fee90e00640 1 -- 172.21.15.83:0/146834321 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec0794c3f0 con 0x55ec07a27940 2024-04-24T09:10:59.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.120+0000 7fee90e00640 1 --2- 172.21.15.83:0/146834321 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec07a27940 0x55ec07a24c40 secure :-1 s=READY pgs=2463 cs=0 l=1 rev1=1 crypto rx=0x7fee84011ca0 tx=0x7fee84004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6462d6bf98b238f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:59.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.120+0000 7fee76000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:10:59.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.121+0000 7fee76000640 1 -- 172.21.15.83:0/146834321 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee84022070 con 0x55ec07a27940 2024-04-24T09:10:59.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.121+0000 7fee76000640 1 -- 172.21.15.83:0/146834321 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(120..120 src has 1..120) ==== 9583+0+0 (secure 0 0 0) 0x7fee8400fd90 con 0x55ec07a27940 2024-04-24T09:10:59.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.121+0000 7fee95c07740 1 -- 172.21.15.83:0/146834321 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee780043d0 con 0x55ec07a27940 2024-04-24T09:10:59.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.121+0000 7fee76000640 1 -- 172.21.15.83:0/146834321 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7fee8402c050 con 0x55ec07a27940 2024-04-24T09:10:59.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.122+0000 7fee95c07740 10 client.22254 fetch_fsmap learned FSMap version 61 2024-04-24T09:10:59.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.122+0000 7fee95c07740 1 -- 172.21.15.83:0/146834321 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x55ec076943f0 con 0x55ec07a27940 2024-04-24T09:10:59.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.122+0000 7fee76000640 1 -- 172.21.15.83:0/146834321 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fee780043d0 con 0x55ec07a27940 2024-04-24T09:10:59.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.122+0000 7fee76000640 1 -- 172.21.15.83:0/146834321 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 61) ==== 8362+0+0 (secure 0 0 0) 0x7fee84027440 con 0x55ec07a27940 2024-04-24T09:10:59.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.123+0000 7fee95c07740 10 client.22254 fetch_fsmap finished waiting for FSMap version 61 2024-04-24T09:10:59.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.123+0000 7fee95c07740 10 client.22254 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-24T09:10:59.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.123+0000 7fee95c07740 20 client.22254 populate_metadata read hostname 'smithi083' 2024-04-24T09:10:59.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.123+0000 7fee95c07740 1 --2- 172.21.15.83:0/146834321 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55ec07a48980 0x55ec07a4ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:59.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.123+0000 7fee95c07740 4 client.22254 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:10:59.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.123+0000 7fee95c07740 1 -- 172.21.15.83:0/146834321 --> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ec0798fbf0 con 0x55ec07a48980 2024-04-24T09:10:59.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.125+0000 7fee8be00640 1 --2- 172.21.15.83:0/146834321 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55ec07a48980 0x55ec07a4ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:59.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.127+0000 7fee8be00640 1 --2- 172.21.15.83:0/146834321 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55ec07a48980 0x55ec07a4ad90 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d818123e484eafb7 server_cookie=cb342d36705e79 in_seq=0 out_seq=0 2024-04-24T09:10:59.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.127+0000 7fee76000640 10 client.22254 ms_handle_connect on v2:172.21.15.189:6838/2548965336 2024-04-24T09:10:59.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.130+0000 7fee76000640 1 -- 172.21.15.83:0/146834321 <== mds.4 v2:172.21.15.189:6838/2548965336 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ec0798fbf0 con 0x55ec07a48980 2024-04-24T09:10:59.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.130+0000 7fee76000640 10 client.22254 handle_command_reply: tid=0 2024-04-24T09:10:59.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.130+0000 7fee95c07740 1 client.22254 shutdown 2024-04-24T09:10:59.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.131+0000 7fee95c07740 1 -- 172.21.15.83:0/146834321 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec07a27940 msgr2=0x55ec07a24c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:59.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.131+0000 7fee95c07740 1 --2- 172.21.15.83:0/146834321 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec07a27940 0x55ec07a24c40 secure :-1 s=READY pgs=2463 cs=0 l=1 rev1=1 crypto rx=0x7fee84011ca0 tx=0x7fee84004ab0 comp rx=0 tx=0).stop 2024-04-24T09:10:59.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.132+0000 7fee95c07740 1 -- 172.21.15.83:0/146834321 shutdown_connections 2024-04-24T09:10:59.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.132+0000 7fee95c07740 1 --2- 172.21.15.83:0/146834321 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55ec07a48980 0x55ec07a4ad90 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:59.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.132+0000 7fee95c07740 1 --2- 172.21.15.83:0/146834321 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec07a27940 0x55ec07a24c40 unknown :-1 s=CLOSED pgs=2463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:59.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.132+0000 7fee95c07740 1 --2- 172.21.15.83:0/146834321 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ec07a1d270 0x55ec07a24630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:59.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.132+0000 7fee95c07740 1 --2- 172.21.15.83:0/146834321 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec07a1c950 0x55ec07a27340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:59.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.132+0000 7fee95c07740 1 -- 172.21.15.83:0/146834321 >> 172.21.15.83:0/146834321 conn(0x55ec07a18c70 msgr2=0x55ec07a1ea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:59.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.132+0000 7fee76000640 0 client.22254 ms_handle_reset on v2:172.21.15.189:6838/2548965336 2024-04-24T09:10:59.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.133+0000 7fee95c07740 1 -- 172.21.15.83:0/146834321 shutdown_connections 2024-04-24T09:10:59.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.133+0000 7fee95c07740 1 -- 172.21.15.83:0/146834321 wait complete. 2024-04-24T09:10:59.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.134+0000 7fee95c07740 20 client.22254 trim_cache size 0 max 16384 2024-04-24T09:10:59.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.163+0000 7fee95c07740 1 Processor -- start 2024-04-24T09:10:59.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.163+0000 7fee95c07740 1 -- start start 2024-04-24T09:10:59.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.164+0000 7fee95c07740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec07a1c950 0x55ec07a26e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:59.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.164+0000 7fee95c07740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec07a1d270 0x55ec07a506a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:59.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.164+0000 7fee95c07740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ec07a27940 0x55ec07a50be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:59.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.164+0000 7fee95c07740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fee8c1ad560 con 0x55ec07a1d270 2024-04-24T09:10:59.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.164+0000 7fee95c07740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fee8c1af0a0 con 0x55ec07a1c950 2024-04-24T09:10:59.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.164+0000 7fee95c07740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fee8c1adc30 con 0x55ec07a27940 2024-04-24T09:10:59.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.164+0000 7fee8b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec07a1d270 0x55ec07a506a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:59.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.164+0000 7fee8b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec07a1d270 0x55ec07a506a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58358/0 (socket says 172.21.15.83:58358) 2024-04-24T09:10:59.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.164+0000 7fee8b400640 1 -- 172.21.15.83:0/3592500948 learned_addr learned my addr 172.21.15.83:0/3592500948 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:10:59.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.165+0000 7fee90e00640 1 --2- 172.21.15.83:0/3592500948 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ec07a27940 0x55ec07a50be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:59.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.165+0000 7fee8b400640 1 -- 172.21.15.83:0/3592500948 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ec07a27940 msgr2=0x55ec07a50be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:59.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.165+0000 7fee8b400640 1 --2- 172.21.15.83:0/3592500948 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ec07a27940 0x55ec07a50be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:59.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.165+0000 7fee8b400640 1 -- 172.21.15.83:0/3592500948 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec07a1c950 msgr2=0x55ec07a26e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:59.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.165+0000 7fee8b400640 1 --2- 172.21.15.83:0/3592500948 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec07a1c950 0x55ec07a26e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:59.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.165+0000 7fee8b400640 1 -- 172.21.15.83:0/3592500948 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec0775e7d0 con 0x55ec07a1d270 2024-04-24T09:10:59.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.165+0000 7fee8b400640 1 --2- 172.21.15.83:0/3592500948 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec07a1d270 0x55ec07a506a0 secure :-1 s=READY pgs=2464 cs=0 l=1 rev1=1 crypto rx=0x7fee78031440 tx=0x7fee7802ab90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cbbe5d634eefbb2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:59.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.166+0000 7fee80c00640 1 -- 172.21.15.83:0/3592500948 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee78052070 con 0x55ec07a1d270 2024-04-24T09:10:59.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.166+0000 7fee80c00640 1 -- 172.21.15.83:0/3592500948 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fee780043d0 con 0x55ec07a1d270 2024-04-24T09:10:59.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.166+0000 7fee80c00640 1 -- 172.21.15.83:0/3592500948 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee78035040 con 0x55ec07a1d270 2024-04-24T09:10:59.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.167+0000 7fee95c07740 1 -- 172.21.15.83:0/3592500948 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec07a1d270 msgr2=0x55ec07a506a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:59.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.167+0000 7fee95c07740 1 --2- 172.21.15.83:0/3592500948 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec07a1d270 0x55ec07a506a0 secure :-1 s=READY pgs=2464 cs=0 l=1 rev1=1 crypto rx=0x7fee78031440 tx=0x7fee7802ab90 comp rx=0 tx=0).stop 2024-04-24T09:10:59.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.167+0000 7fee95c07740 1 -- 172.21.15.83:0/3592500948 shutdown_connections 2024-04-24T09:10:59.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.167+0000 7fee95c07740 1 --2- 172.21.15.83:0/3592500948 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ec07a27940 0x55ec07a50be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:59.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.167+0000 7fee95c07740 1 --2- 172.21.15.83:0/3592500948 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec07a1d270 0x55ec07a506a0 unknown :-1 s=CLOSED pgs=2464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:59.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.167+0000 7fee95c07740 1 --2- 172.21.15.83:0/3592500948 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec07a1c950 0x55ec07a26e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:59.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.167+0000 7fee95c07740 1 -- 172.21.15.83:0/3592500948 >> 172.21.15.83:0/3592500948 conn(0x55ec07a18c70 msgr2=0x55ec07a245f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:59.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.168+0000 7fee95c07740 1 -- 172.21.15.83:0/3592500948 shutdown_connections 2024-04-24T09:10:59.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.168+0000 7fee95c07740 1 -- 172.21.15.83:0/3592500948 wait complete. 2024-04-24T09:10:59.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.168+0000 7fee95c07740 1 -- start start 2024-04-24T09:10:59.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.169+0000 7fee95c07740 1 Processor -- start 2024-04-24T09:10:59.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.170+0000 7fee95c07740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec07a1c950 0x55ec07a41050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:59.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.170+0000 7fee95c07740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec07a1d270 0x55ec07a283a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:59.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.170+0000 7fee95c07740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ec07a27940 0x55ec07a288e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:59.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.170+0000 7fee95c07740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fee8c1ae9d0 con 0x55ec07a1c950 2024-04-24T09:10:59.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.170+0000 7fee95c07740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fee8c1ae300 con 0x55ec07a1d270 2024-04-24T09:10:59.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.170+0000 7fee95c07740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fee8c082a80 con 0x55ec07a27940 2024-04-24T09:10:59.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.170+0000 7fee95c07740 10 client.0 _ll_register_callbacks cb 0x7fee84009bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:10:59.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.170+0000 7fee8be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec07a1c950 0x55ec07a41050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:59.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.170+0000 7fee8b400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec07a1d270 0x55ec07a283a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:59.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.170+0000 7fee8b400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec07a1d270 0x55ec07a283a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:41694/0 (socket says 172.21.15.83:41694) 2024-04-24T09:10:59.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.170+0000 7fee8b400640 1 -- 172.21.15.83:0/498728273 learned_addr learned my addr 172.21.15.83:0/498728273 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:10:59.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.170+0000 7fee90e00640 1 --2- 172.21.15.83:0/498728273 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ec07a27940 0x55ec07a288e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:59.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.171+0000 7fee8be00640 1 -- 172.21.15.83:0/498728273 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ec07a27940 msgr2=0x55ec07a288e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:59.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.171+0000 7fee8be00640 1 --2- 172.21.15.83:0/498728273 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ec07a27940 0x55ec07a288e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:59.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.171+0000 7fee8be00640 1 -- 172.21.15.83:0/498728273 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec07a1d270 msgr2=0x55ec07a283a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:59.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.171+0000 7fee8be00640 1 --2- 172.21.15.83:0/498728273 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec07a1d270 0x55ec07a283a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:59.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.171+0000 7fee8be00640 1 -- 172.21.15.83:0/498728273 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec07a290c0 con 0x55ec07a1c950 2024-04-24T09:10:59.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.171+0000 7fee8be00640 1 --2- 172.21.15.83:0/498728273 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec07a1c950 0x55ec07a41050 secure :-1 s=READY pgs=2465 cs=0 l=1 rev1=1 crypto rx=0x7fee7c0ce370 tx=0x7fee7c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d69ad6a7c49d712f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:59.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.171+0000 7fee76000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:10:59.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.172+0000 7fee76000640 1 -- 172.21.15.83:0/498728273 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee7c003b30 con 0x55ec07a1c950 2024-04-24T09:10:59.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.172+0000 7fee76000640 1 -- 172.21.15.83:0/498728273 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(120..120 src has 1..120) ==== 9583+0+0 (secure 0 0 0) 0x7fee7c099230 con 0x55ec07a1c950 2024-04-24T09:10:59.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.172+0000 7fee95c07740 1 -- 172.21.15.83:0/498728273 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee780043d0 con 0x55ec07a1c950 2024-04-24T09:10:59.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.172+0000 7fee76000640 1 -- 172.21.15.83:0/498728273 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7fee7c09d2c0 con 0x55ec07a1c950 2024-04-24T09:10:59.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.172+0000 7fee95c07740 10 client.22260 fetch_fsmap learned FSMap version 61 2024-04-24T09:10:59.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.172+0000 7fee95c07740 1 -- 172.21.15.83:0/498728273 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x55ec07aaef70 con 0x55ec07a1c950 2024-04-24T09:10:59.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.172+0000 7fee76000640 1 -- 172.21.15.83:0/498728273 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fee780043d0 con 0x55ec07a1c950 2024-04-24T09:10:59.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.172+0000 7fee76000640 1 -- 172.21.15.83:0/498728273 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 61) ==== 8362+0+0 (secure 0 0 0) 0x7fee7c0ce5f0 con 0x55ec07a1c950 2024-04-24T09:10:59.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.173+0000 7fee95c07740 10 client.22260 fetch_fsmap finished waiting for FSMap version 61 2024-04-24T09:10:59.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.173+0000 7fee95c07740 10 client.22260 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-24T09:10:59.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.173+0000 7fee95c07740 20 client.22260 populate_metadata read hostname 'smithi083' 2024-04-24T09:10:59.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.173+0000 7fee95c07740 1 --2- 172.21.15.83:0/498728273 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55ec07aaf5f0 0x55ec07ab1a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:59.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.173+0000 7fee95c07740 4 client.22260 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:10:59.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.173+0000 7fee95c07740 1 -- 172.21.15.83:0/498728273 --> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ec07989a60 con 0x55ec07aaf5f0 2024-04-24T09:10:59.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.173+0000 7fee8b400640 1 --2- 172.21.15.83:0/498728273 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55ec07aaf5f0 0x55ec07ab1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:59.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.174+0000 7fee76000640 10 client.22260 ms_handle_connect on v2:172.21.15.189:6838/2548965336 2024-04-24T09:10:59.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.174+0000 7fee8b400640 1 --2- 172.21.15.83:0/498728273 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55ec07aaf5f0 0x55ec07ab1a90 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6eec23cde6967d92 server_cookie=bc9ec83ac65ee0b9 in_seq=0 out_seq=0 2024-04-24T09:10:59.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.175+0000 7fee76000640 1 -- 172.21.15.83:0/498728273 <== mds.4 v2:172.21.15.189:6838/2548965336 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ec07989a60 con 0x55ec07aaf5f0 2024-04-24T09:10:59.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.175+0000 7fee76000640 10 client.22260 handle_command_reply: tid=0 2024-04-24T09:10:59.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.175+0000 7fee95c07740 1 client.22260 shutdown 2024-04-24T09:10:59.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.175+0000 7fee95c07740 1 -- 172.21.15.83:0/498728273 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec07a1c950 msgr2=0x55ec07a41050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:59.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.175+0000 7fee95c07740 1 --2- 172.21.15.83:0/498728273 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec07a1c950 0x55ec07a41050 secure :-1 s=READY pgs=2465 cs=0 l=1 rev1=1 crypto rx=0x7fee7c0ce370 tx=0x7fee7c000bc0 comp rx=0 tx=0).stop 2024-04-24T09:10:59.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.176+0000 7fee95c07740 1 -- 172.21.15.83:0/498728273 shutdown_connections 2024-04-24T09:10:59.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.176+0000 7fee95c07740 1 --2- 172.21.15.83:0/498728273 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55ec07aaf5f0 0x55ec07ab1a90 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:59.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.176+0000 7fee95c07740 1 --2- 172.21.15.83:0/498728273 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ec07a27940 0x55ec07a288e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:59.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.176+0000 7fee95c07740 1 --2- 172.21.15.83:0/498728273 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec07a1d270 0x55ec07a283a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:59.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.176+0000 7fee95c07740 1 --2- 172.21.15.83:0/498728273 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ec07a1c950 0x55ec07a41050 unknown :-1 s=CLOSED pgs=2465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:59.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.176+0000 7fee95c07740 1 -- 172.21.15.83:0/498728273 >> 172.21.15.83:0/498728273 conn(0x55ec07a18c70 msgr2=0x55ec07a22e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:59.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.176+0000 7fee76000640 0 client.22260 ms_handle_reset on v2:172.21.15.189:6838/2548965336 2024-04-24T09:10:59.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.176+0000 7fee95c07740 1 -- 172.21.15.83:0/498728273 shutdown_connections 2024-04-24T09:10:59.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.176+0000 7fee95c07740 1 -- 172.21.15.83:0/498728273 wait complete. 2024-04-24T09:10:59.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.176+0000 7fee95c07740 20 client.22260 trim_cache size 0 max 16384 2024-04-24T09:10:59.177 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:10:59.178 INFO:tasks.workunit.client.0.smithi083.stdout: 10 41660 6.67 MB/sec execute 210 sec latency 95.551 ms 2024-04-24T09:10:59.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.177+0000 7fee92e00640 1 -- 172.21.15.83:0/1002038685 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fee700a3d60 msgr2=0x7fee700a6220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:59.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.177+0000 7fee92e00640 1 --2- 172.21.15.83:0/1002038685 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fee700a3d60 0x7fee700a6220 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fee84005fd0 tx=0x7fee84009500 comp rx=0 tx=0).stop 2024-04-24T09:10:59.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.178+0000 7fee92e00640 1 -- 172.21.15.83:0/1002038685 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee8c108610 msgr2=0x7fee8c19ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:59.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.178+0000 7fee92e00640 1 --2- 172.21.15.83:0/1002038685 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee8c108610 0x7fee8c19ee40 secure :-1 s=READY pgs=2461 cs=0 l=1 rev1=1 crypto rx=0x7fee7c00e9c0 tx=0x7fee7c00ee80 comp rx=0 tx=0).stop 2024-04-24T09:10:59.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.178+0000 7fee92e00640 1 -- 172.21.15.83:0/1002038685 shutdown_connections 2024-04-24T09:10:59.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.178+0000 7fee92e00640 1 --2- 172.21.15.83:0/1002038685 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fee700a3d60 0x7fee700a6220 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:59.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.178+0000 7fee92e00640 1 --2- 172.21.15.83:0/1002038685 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee8c10e660 0x7fee8c1a36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:59.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.178+0000 7fee92e00640 1 --2- 172.21.15.83:0/1002038685 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fee8c108fd0 0x7fee8c19f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:59.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.178+0000 7fee92e00640 1 --2- 172.21.15.83:0/1002038685 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee8c108610 0x7fee8c19ee40 unknown :-1 s=CLOSED pgs=2461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:59.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.179+0000 7fee92e00640 1 -- 172.21.15.83:0/1002038685 >> 172.21.15.83:0/1002038685 conn(0x7fee8c076540 msgr2=0x7fee8c077d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:59.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.179+0000 7fee92e00640 1 -- 172.21.15.83:0/1002038685 shutdown_connections 2024-04-24T09:10:59.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.179+0000 7fee92e00640 1 -- 172.21.15.83:0/1002038685 wait complete. 2024-04-24T09:10:59.199 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:10:59 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/2822061963' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:10:59.202 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-24T09:10:59.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:10:59 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/2822061963' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:10:59.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:10:59 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/2822061963' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:10:59.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.461+0000 7f1647600640 1 -- 172.21.15.83:0/1898387459 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1640103b80 msgr2=0x7f164010c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:59.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.461+0000 7f1647600640 1 --2- 172.21.15.83:0/1898387459 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1640103b80 0x7f164010c060 secure :-1 s=READY pgs=2466 cs=0 l=1 rev1=1 crypto rx=0x7f162c009900 tx=0x7f162c02a480 comp rx=0 tx=0).stop 2024-04-24T09:10:59.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.461+0000 7f1647600640 1 -- 172.21.15.83:0/1898387459 shutdown_connections 2024-04-24T09:10:59.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.461+0000 7f1647600640 1 --2- 172.21.15.83:0/1898387459 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f164010c630 0x7f164010ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:59.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.461+0000 7f1647600640 1 --2- 172.21.15.83:0/1898387459 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1640103b80 0x7f164010c060 unknown :-1 s=CLOSED pgs=2466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:59.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.461+0000 7f1647600640 1 --2- 172.21.15.83:0/1898387459 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1640103260 0x7f1640103640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:59.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.461+0000 7f1647600640 1 -- 172.21.15.83:0/1898387459 >> 172.21.15.83:0/1898387459 conn(0x7f1640077280 msgr2=0x7f1640077690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:59.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.461+0000 7f1647600640 1 -- 172.21.15.83:0/1898387459 shutdown_connections 2024-04-24T09:10:59.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.461+0000 7f1647600640 1 -- 172.21.15.83:0/1898387459 wait complete. 2024-04-24T09:10:59.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.462+0000 7f1647600640 1 Processor -- start 2024-04-24T09:10:59.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.463+0000 7f1647600640 1 -- start start 2024-04-24T09:10:59.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.463+0000 7f1647600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1640103260 0x7f16401a0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:59.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.463+0000 7f1647600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1640103b80 0x7f16401a0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:59.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.463+0000 7f1647600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f164010c630 0x7f164019a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:59.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.463+0000 7f1647600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f16401122c0 con 0x7f164010c630 2024-04-24T09:10:59.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.463+0000 7f1647600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1640112140 con 0x7f1640103b80 2024-04-24T09:10:59.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.463+0000 7f1647600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1640112440 con 0x7f1640103260 2024-04-24T09:10:59.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.463+0000 7f163fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1640103b80 0x7f16401a0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:59.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.463+0000 7f1645600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f164010c630 0x7f164019a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:59.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.463+0000 7f163fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1640103b80 0x7f16401a0c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:41706/0 (socket says 172.21.15.83:41706) 2024-04-24T09:10:59.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.463+0000 7f163fe00640 1 -- 172.21.15.83:0/2737237643 learned_addr learned my addr 172.21.15.83:0/2737237643 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:10:59.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.464+0000 7f1644c00640 1 --2- 172.21.15.83:0/2737237643 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1640103260 0x7f16401a0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:59.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.464+0000 7f1645600640 1 -- 172.21.15.83:0/2737237643 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1640103260 msgr2=0x7f16401a0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:59.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.464+0000 7f1645600640 1 --2- 172.21.15.83:0/2737237643 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1640103260 0x7f16401a0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:59.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.464+0000 7f1645600640 1 -- 172.21.15.83:0/2737237643 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1640103b80 msgr2=0x7f16401a0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:59.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.464+0000 7f1645600640 1 --2- 172.21.15.83:0/2737237643 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1640103b80 0x7f16401a0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:59.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.464+0000 7f1645600640 1 -- 172.21.15.83:0/2737237643 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f164019b080 con 0x7f164010c630 2024-04-24T09:10:59.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.464+0000 7f163fe00640 1 --2- 172.21.15.83:0/2737237643 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1640103b80 0x7f16401a0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:10:59.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.464+0000 7f1644c00640 1 --2- 172.21.15.83:0/2737237643 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1640103260 0x7f16401a0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:10:59.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.464+0000 7f1645600640 1 --2- 172.21.15.83:0/2737237643 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f164010c630 0x7f164019a900 secure :-1 s=READY pgs=2467 cs=0 l=1 rev1=1 crypto rx=0x7f1638005fd0 tx=0x7f163800ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:59.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.465+0000 7f163d600640 1 -- 172.21.15.83:0/2737237643 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f163800cdf0 con 0x7f164010c630 2024-04-24T09:10:59.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.465+0000 7f163d600640 1 -- 172.21.15.83:0/2737237643 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1638005020 con 0x7f164010c630 2024-04-24T09:10:59.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.465+0000 7f163d600640 1 -- 172.21.15.83:0/2737237643 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1638010690 con 0x7f164010c630 2024-04-24T09:10:59.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.465+0000 7f1647600640 1 -- 172.21.15.83:0/2737237643 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f164019b370 con 0x7f164010c630 2024-04-24T09:10:59.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.466+0000 7f1647600640 1 -- 172.21.15.83:0/2737237643 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16401a7610 con 0x7f164010c630 2024-04-24T09:10:59.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.470+0000 7f164a430740 1 Processor -- start 2024-04-24T09:10:59.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.471+0000 7f163d600640 1 -- 172.21.15.83:0/2737237643 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1638004a40 con 0x7f164010c630 2024-04-24T09:10:59.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.471+0000 7f164a430740 1 -- start start 2024-04-24T09:10:59.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.472+0000 7f164a430740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55705646f6a0 0x55705646fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:59.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.472+0000 7f164a430740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557056470050 0x557056482c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:59.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.472+0000 7f164a430740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557056483150 0x557056485540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:59.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.472+0000 7f164a430740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5570563d51d0 con 0x557056470050 2024-04-24T09:10:59.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.472+0000 7f164a430740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55705637b450 con 0x55705646f6a0 2024-04-24T09:10:59.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.472+0000 7f164a430740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5570563d1190 con 0x557056483150 2024-04-24T09:10:59.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.472+0000 7f163fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55705646f6a0 0x55705646fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:59.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.472+0000 7f1644c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557056470050 0x557056482c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:59.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.472+0000 7f163fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55705646f6a0 0x55705646fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:41710/0 (socket says 172.21.15.83:41710) 2024-04-24T09:10:59.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.472+0000 7f163fe00640 1 -- 172.21.15.83:0/2294150288 learned_addr learned my addr 172.21.15.83:0/2294150288 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:10:59.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.473+0000 7f163fe00640 1 --2- 172.21.15.83:0/2294150288 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557056483150 0x557056485540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:59.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.473+0000 7f1644c00640 1 -- 172.21.15.83:0/2294150288 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557056483150 msgr2=0x557056485540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:59.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.473+0000 7f1644c00640 1 --2- 172.21.15.83:0/2294150288 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557056483150 0x557056485540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:59.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.473+0000 7f1644c00640 1 -- 172.21.15.83:0/2294150288 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55705646f6a0 msgr2=0x55705646fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:59.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.473+0000 7f1644c00640 1 --2- 172.21.15.83:0/2294150288 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55705646f6a0 0x55705646fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:59.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.473+0000 7f1644c00640 1 -- 172.21.15.83:0/2294150288 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557056438310 con 0x557056470050 2024-04-24T09:10:59.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.473+0000 7f163fe00640 1 --2- 172.21.15.83:0/2294150288 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55705646f6a0 0x55705646fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:10:59.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.473+0000 7f1644c00640 1 --2- 172.21.15.83:0/2294150288 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557056470050 0x557056482c10 secure :-1 s=READY pgs=2468 cs=0 l=1 rev1=1 crypto rx=0x7f1634009c00 tx=0x7f16340117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf1d5ef346171fd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:59.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.474+0000 7f1631c00640 1 -- 172.21.15.83:0/2294150288 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1634005020 con 0x557056470050 2024-04-24T09:10:59.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.474+0000 7f1631c00640 1 -- 172.21.15.83:0/2294150288 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f16340047a0 con 0x557056470050 2024-04-24T09:10:59.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.474+0000 7f1631c00640 1 -- 172.21.15.83:0/2294150288 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1634012670 con 0x557056470050 2024-04-24T09:10:59.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.475+0000 7f163d600640 1 --2- 172.21.15.83:0/2737237643 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f16280a3da0 0x7f16280a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:59.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.475+0000 7f163d600640 1 -- 172.21.15.83:0/2737237643 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 9583+0+0 (secure 0 0 0) 0x7f16380d16c0 con 0x7f164010c630 2024-04-24T09:10:59.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.475+0000 7f163fe00640 1 --2- 172.21.15.83:0/2737237643 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f16280a3da0 0x7f16280a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:59.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.475+0000 7f164a430740 1 -- 172.21.15.83:0/2294150288 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557056470050 msgr2=0x557056482c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:59.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.475+0000 7f164a430740 1 --2- 172.21.15.83:0/2294150288 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557056470050 0x557056482c10 secure :-1 s=READY pgs=2468 cs=0 l=1 rev1=1 crypto rx=0x7f1634009c00 tx=0x7f16340117b0 comp rx=0 tx=0).stop 2024-04-24T09:10:59.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.475+0000 7f164a430740 1 -- 172.21.15.83:0/2294150288 shutdown_connections 2024-04-24T09:10:59.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.475+0000 7f164a430740 1 --2- 172.21.15.83:0/2294150288 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557056483150 0x557056485540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:59.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.475+0000 7f164a430740 1 --2- 172.21.15.83:0/2294150288 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557056470050 0x557056482c10 unknown :-1 s=CLOSED pgs=2468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:59.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.475+0000 7f164a430740 1 --2- 172.21.15.83:0/2294150288 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55705646f6a0 0x55705646fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:59.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.475+0000 7f164a430740 1 -- 172.21.15.83:0/2294150288 >> 172.21.15.83:0/2294150288 conn(0x55705646bc70 msgr2=0x55705646c050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:59.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.476+0000 7f164a430740 1 -- 172.21.15.83:0/2294150288 shutdown_connections 2024-04-24T09:10:59.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.476+0000 7f164a430740 1 -- 172.21.15.83:0/2294150288 wait complete. 2024-04-24T09:10:59.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.476+0000 7f163fe00640 1 --2- 172.21.15.83:0/2737237643 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f16280a3da0 0x7f16280a6260 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f162c004650 tx=0x7f162c009560 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:59.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.477+0000 7f164a430740 1 -- start start 2024-04-24T09:10:59.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.477+0000 7f164a430740 1 Processor -- start 2024-04-24T09:10:59.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.478+0000 7f164a430740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55705646f6a0 0x557056482ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:59.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.478+0000 7f164a430740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557056470050 0x55705647fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:59.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.478+0000 7f164a430740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557056483150 0x5570564805d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:59.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.478+0000 7f164a430740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x557056485fb0 con 0x557056470050 2024-04-24T09:10:59.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.478+0000 7f164a430740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55705638e810 con 0x55705646f6a0 2024-04-24T09:10:59.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.478+0000 7f164a430740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x557056486130 con 0x557056483150 2024-04-24T09:10:59.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.478+0000 7f164a430740 10 client.0 _ll_register_callbacks cb 0x55705635c140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:10:59.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.478+0000 7f1644c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55705646f6a0 0x557056482ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:59.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.478+0000 7f163fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557056470050 0x55705647fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:59.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.478+0000 7f1644c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55705646f6a0 0x557056482ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:41722/0 (socket says 172.21.15.83:41722) 2024-04-24T09:10:59.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.478+0000 7f1644c00640 1 -- 172.21.15.83:0/36201482 learned_addr learned my addr 172.21.15.83:0/36201482 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:10:59.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.479+0000 7f1644c00640 1 -- 172.21.15.83:0/36201482 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557056483150 msgr2=0x5570564805d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:59.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.479+0000 7f1644c00640 1 --2- 172.21.15.83:0/36201482 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557056483150 0x5570564805d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:59.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.479+0000 7f1644c00640 1 -- 172.21.15.83:0/36201482 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557056470050 msgr2=0x55705647fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:59.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.479+0000 7f1644c00640 1 --2- 172.21.15.83:0/36201482 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557056470050 0x55705647fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:59.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.479+0000 7f1644c00640 1 -- 172.21.15.83:0/36201482 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55705639f3f0 con 0x55705646f6a0 2024-04-24T09:10:59.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.479+0000 7f1644c00640 1 --2- 172.21.15.83:0/36201482 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55705646f6a0 0x557056482ba0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f1634004e90 tx=0x7f1634009910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d57a0a7128ae924 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:59.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.480+0000 7f1626000640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:10:59.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.480+0000 7f1626000640 1 -- 172.21.15.83:0/36201482 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1634028030 con 0x55705646f6a0 2024-04-24T09:10:59.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.480+0000 7f1626000640 1 -- 172.21.15.83:0/36201482 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(120..120 src has 1..120) ==== 9583+0+0 (secure 0 0 0) 0x7f163401f050 con 0x55705646f6a0 2024-04-24T09:10:59.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.480+0000 7f164a430740 1 -- 172.21.15.83:0/36201482 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16340047a0 con 0x55705646f6a0 2024-04-24T09:10:59.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.481+0000 7f1626000640 1 -- 172.21.15.83:0/36201482 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f1634032050 con 0x55705646f6a0 2024-04-24T09:10:59.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.481+0000 7f164a430740 10 client.39353 fetch_fsmap learned FSMap version 61 2024-04-24T09:10:59.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.481+0000 7f164a430740 1 -- 172.21.15.83:0/36201482 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x5570560e73f0 con 0x55705646f6a0 2024-04-24T09:10:59.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.481+0000 7f1626000640 1 -- 172.21.15.83:0/36201482 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f16340047a0 con 0x55705646f6a0 2024-04-24T09:10:59.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.481+0000 7f1626000640 1 -- 172.21.15.83:0/36201482 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 61) ==== 8362+0+0 (secure 0 0 0) 0x7f1634024b90 con 0x55705646f6a0 2024-04-24T09:10:59.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.482+0000 7f164a430740 10 client.39353 fetch_fsmap finished waiting for FSMap version 61 2024-04-24T09:10:59.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.482+0000 7f164a430740 10 client.39353 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-24T09:10:59.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.482+0000 7f164a430740 20 client.39353 populate_metadata read hostname 'smithi083' 2024-04-24T09:10:59.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.482+0000 7f164a430740 1 --2- 172.21.15.83:0/36201482 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x557056493300 0x5570564956f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:59.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.482+0000 7f164a430740 4 client.39353 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:10:59.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.482+0000 7f164a430740 1 -- 172.21.15.83:0/36201482 --> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5570563e2bf0 con 0x557056493300 2024-04-24T09:10:59.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.482+0000 7f163fe00640 1 --2- 172.21.15.83:0/36201482 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x557056493300 0x5570564956f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:59.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.483+0000 7f163fe00640 1 --2- 172.21.15.83:0/36201482 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x557056493300 0x5570564956f0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=959aa9f2e3aede3e server_cookie=c51e3c76f3b1ba15 in_seq=0 out_seq=0 2024-04-24T09:10:59.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.483+0000 7f1626000640 10 client.39353 ms_handle_connect on v2:172.21.15.138:6832/1157967658 2024-04-24T09:10:59.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.485+0000 7f1626000640 1 -- 172.21.15.83:0/36201482 <== mds.3 v2:172.21.15.138:6832/1157967658 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5570563e2bf0 con 0x557056493300 2024-04-24T09:10:59.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.485+0000 7f1626000640 10 client.39353 handle_command_reply: tid=0 2024-04-24T09:10:59.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.485+0000 7f164a430740 1 client.39353 shutdown 2024-04-24T09:10:59.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.485+0000 7f164a430740 1 -- 172.21.15.83:0/36201482 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55705646f6a0 msgr2=0x557056482ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:59.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.485+0000 7f164a430740 1 --2- 172.21.15.83:0/36201482 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55705646f6a0 0x557056482ba0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f1634004e90 tx=0x7f1634009910 comp rx=0 tx=0).stop 2024-04-24T09:10:59.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.486+0000 7f164a430740 1 -- 172.21.15.83:0/36201482 shutdown_connections 2024-04-24T09:10:59.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.486+0000 7f164a430740 1 --2- 172.21.15.83:0/36201482 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x557056493300 0x5570564956f0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:59.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.486+0000 7f164a430740 1 --2- 172.21.15.83:0/36201482 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557056483150 0x5570564805d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:59.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.486+0000 7f164a430740 1 --2- 172.21.15.83:0/36201482 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557056470050 0x55705647fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:59.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.486+0000 7f1626000640 0 client.39353 ms_handle_reset on v2:172.21.15.138:6832/1157967658 2024-04-24T09:10:59.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.486+0000 7f164a430740 1 --2- 172.21.15.83:0/36201482 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55705646f6a0 0x557056482ba0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:59.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.486+0000 7f164a430740 1 -- 172.21.15.83:0/36201482 >> 172.21.15.83:0/36201482 conn(0x55705646bc70 msgr2=0x557056471a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:59.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.486+0000 7f164a430740 1 -- 172.21.15.83:0/36201482 shutdown_connections 2024-04-24T09:10:59.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.486+0000 7f164a430740 1 -- 172.21.15.83:0/36201482 wait complete. 2024-04-24T09:10:59.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.486+0000 7f164a430740 20 client.39353 trim_cache size 0 max 16384 2024-04-24T09:10:59.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.509+0000 7f164a430740 1 Processor -- start 2024-04-24T09:10:59.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.509+0000 7f164a430740 1 -- start start 2024-04-24T09:10:59.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.510+0000 7f164a430740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55705646f6a0 0x55705648eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:59.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.510+0000 7f164a430740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557056470050 0x55705647ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:59.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.510+0000 7f164a430740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557056483150 0x5570564826d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:59.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.510+0000 7f164a430740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f16401ad440 con 0x557056470050 2024-04-24T09:10:59.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.510+0000 7f164a430740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f16401aef80 con 0x55705646f6a0 2024-04-24T09:10:59.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.510+0000 7f164a430740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f16401adb10 con 0x557056483150 2024-04-24T09:10:59.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.510+0000 7f1644c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55705646f6a0 0x55705648eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:59.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.510+0000 7f163fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557056470050 0x55705647ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:59.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.510+0000 7f1644c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55705646f6a0 0x55705648eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:41734/0 (socket says 172.21.15.83:41734) 2024-04-24T09:10:59.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.510+0000 7f1644c00640 1 -- 172.21.15.83:0/4159494934 learned_addr learned my addr 172.21.15.83:0/4159494934 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:10:59.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.511+0000 7f1645600640 1 --2- 172.21.15.83:0/4159494934 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557056483150 0x5570564826d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:59.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.511+0000 7f1644c00640 1 -- 172.21.15.83:0/4159494934 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557056483150 msgr2=0x5570564826d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:59.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.511+0000 7f1644c00640 1 --2- 172.21.15.83:0/4159494934 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557056483150 0x5570564826d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:59.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.511+0000 7f1644c00640 1 -- 172.21.15.83:0/4159494934 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557056470050 msgr2=0x55705647ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:59.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.511+0000 7f1644c00640 1 --2- 172.21.15.83:0/4159494934 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557056470050 0x55705647ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:59.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.511+0000 7f1644c00640 1 -- 172.21.15.83:0/4159494934 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570561b17d0 con 0x55705646f6a0 2024-04-24T09:10:59.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.512+0000 7f1644c00640 1 --2- 172.21.15.83:0/4159494934 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55705646f6a0 0x55705648eee0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f1634004e60 tx=0x7f16340117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=caa6065a89418604 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:59.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.512+0000 7f1631200640 1 -- 172.21.15.83:0/4159494934 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1634028070 con 0x55705646f6a0 2024-04-24T09:10:59.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.512+0000 7f1631200640 1 -- 172.21.15.83:0/4159494934 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1634004780 con 0x55705646f6a0 2024-04-24T09:10:59.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.512+0000 7f1631200640 1 -- 172.21.15.83:0/4159494934 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16340051b0 con 0x55705646f6a0 2024-04-24T09:10:59.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.513+0000 7f164a430740 1 -- 172.21.15.83:0/4159494934 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55705646f6a0 msgr2=0x55705648eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:59.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.513+0000 7f164a430740 1 --2- 172.21.15.83:0/4159494934 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55705646f6a0 0x55705648eee0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f1634004e60 tx=0x7f16340117b0 comp rx=0 tx=0).stop 2024-04-24T09:10:59.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.513+0000 7f164a430740 1 -- 172.21.15.83:0/4159494934 shutdown_connections 2024-04-24T09:10:59.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.513+0000 7f164a430740 1 --2- 172.21.15.83:0/4159494934 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557056483150 0x5570564826d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:59.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.513+0000 7f164a430740 1 --2- 172.21.15.83:0/4159494934 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557056470050 0x55705647ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:59.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.513+0000 7f164a430740 1 --2- 172.21.15.83:0/4159494934 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55705646f6a0 0x55705648eee0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:59.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.513+0000 7f164a430740 1 -- 172.21.15.83:0/4159494934 >> 172.21.15.83:0/4159494934 conn(0x55705646bc70 msgr2=0x55705647e760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:59.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.513+0000 7f164a430740 1 -- 172.21.15.83:0/4159494934 shutdown_connections 2024-04-24T09:10:59.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.513+0000 7f164a430740 1 -- 172.21.15.83:0/4159494934 wait complete. 2024-04-24T09:10:59.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.514+0000 7f164a430740 1 -- start start 2024-04-24T09:10:59.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.514+0000 7f164a430740 1 Processor -- start 2024-04-24T09:10:59.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.514+0000 7f164a430740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55705646f6a0 0x557056483b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:59.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.515+0000 7f164a430740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557056470050 0x557056484090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:59.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.515+0000 7f164a430740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557056483150 0x55705648a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:59.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.515+0000 7f164a430740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f16401ae8b0 con 0x557056470050 2024-04-24T09:10:59.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.515+0000 7f164a430740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f16401ae1e0 con 0x557056483150 2024-04-24T09:10:59.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.515+0000 7f164a430740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f16401ab4a0 con 0x55705646f6a0 2024-04-24T09:10:59.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.515+0000 7f163fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557056470050 0x557056484090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:59.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.515+0000 7f164a430740 10 client.0 _ll_register_callbacks cb 0x7f1634005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:10:59.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.515+0000 7f163fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557056470050 0x557056484090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58406/0 (socket says 172.21.15.83:58406) 2024-04-24T09:10:59.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.515+0000 7f163fe00640 1 -- 172.21.15.83:0/3712646815 learned_addr learned my addr 172.21.15.83:0/3712646815 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:10:59.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.515+0000 7f1645600640 1 --2- 172.21.15.83:0/3712646815 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557056483150 0x55705648a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:59.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.515+0000 7f1644c00640 1 --2- 172.21.15.83:0/3712646815 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55705646f6a0 0x557056483b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:59.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.516+0000 7f1645600640 1 -- 172.21.15.83:0/3712646815 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55705646f6a0 msgr2=0x557056483b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:59.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.516+0000 7f1645600640 1 --2- 172.21.15.83:0/3712646815 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55705646f6a0 0x557056483b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:59.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.516+0000 7f1645600640 1 -- 172.21.15.83:0/3712646815 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557056470050 msgr2=0x557056484090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:59.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.516+0000 7f1645600640 1 --2- 172.21.15.83:0/3712646815 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557056470050 0x557056484090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:59.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.516+0000 7f1645600640 1 -- 172.21.15.83:0/3712646815 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55705648a930 con 0x557056483150 2024-04-24T09:10:59.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.516+0000 7f163fe00640 1 --2- 172.21.15.83:0/3712646815 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557056470050 0x557056484090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:10:59.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.516+0000 7f1645600640 1 --2- 172.21.15.83:0/3712646815 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557056483150 0x55705648a3f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f16380ce990 tx=0x7f1638005c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b62045b3a9020a0a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:59.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.516+0000 7f1626000640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:10:59.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.517+0000 7f1626000640 1 -- 172.21.15.83:0/3712646815 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16380bb060 con 0x557056483150 2024-04-24T09:10:59.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.517+0000 7f164a430740 1 -- 172.21.15.83:0/3712646815 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1634004780 con 0x557056483150 2024-04-24T09:10:59.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.517+0000 7f1626000640 1 -- 172.21.15.83:0/3712646815 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(120..120 src has 1..120) ==== 9583+0+0 (secure 0 0 0) 0x7f16380051d0 con 0x557056483150 2024-04-24T09:10:59.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.517+0000 7f1626000640 1 -- 172.21.15.83:0/3712646815 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f16380b6070 con 0x557056483150 2024-04-24T09:10:59.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.517+0000 7f164a430740 10 client.39359 fetch_fsmap learned FSMap version 61 2024-04-24T09:10:59.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.517+0000 7f164a430740 1 -- 172.21.15.83:0/3712646815 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x55705648bc90 con 0x557056483150 2024-04-24T09:10:59.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.518+0000 7f1626000640 1 -- 172.21.15.83:0/3712646815 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1634004780 con 0x557056483150 2024-04-24T09:10:59.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.518+0000 7f1626000640 1 -- 172.21.15.83:0/3712646815 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 61) ==== 8362+0+0 (secure 0 0 0) 0x7f1638020070 con 0x557056483150 2024-04-24T09:10:59.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.518+0000 7f164a430740 10 client.39359 fetch_fsmap finished waiting for FSMap version 61 2024-04-24T09:10:59.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.518+0000 7f164a430740 10 client.39359 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-24T09:10:59.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.519+0000 7f164a430740 20 client.39359 populate_metadata read hostname 'smithi083' 2024-04-24T09:10:59.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.519+0000 7f164a430740 1 --2- 172.21.15.83:0/3712646815 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x5570564f99b0 0x5570564fbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:59.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.519+0000 7f164a430740 4 client.39359 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:10:59.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.519+0000 7f164a430740 1 -- 172.21.15.83:0/3712646815 --> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5570563dca60 con 0x5570564f99b0 2024-04-24T09:10:59.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.519+0000 7f1644c00640 1 --2- 172.21.15.83:0/3712646815 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x5570564f99b0 0x5570564fbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:59.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.520+0000 7f1644c00640 1 --2- 172.21.15.83:0/3712646815 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x5570564f99b0 0x5570564fbdd0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a6760bf6864422e server_cookie=78da1de00c5806f5 in_seq=0 out_seq=0 2024-04-24T09:10:59.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.520+0000 7f1626000640 10 client.39359 ms_handle_connect on v2:172.21.15.138:6832/1157967658 2024-04-24T09:10:59.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.520+0000 7f1626000640 1 -- 172.21.15.83:0/3712646815 <== mds.3 v2:172.21.15.138:6832/1157967658 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5570563dca60 con 0x5570564f99b0 2024-04-24T09:10:59.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.521+0000 7f1626000640 10 client.39359 handle_command_reply: tid=0 2024-04-24T09:10:59.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.521+0000 7f164a430740 1 client.39359 shutdown 2024-04-24T09:10:59.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.521+0000 7f164a430740 1 -- 172.21.15.83:0/3712646815 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557056483150 msgr2=0x55705648a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:59.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.521+0000 7f164a430740 1 --2- 172.21.15.83:0/3712646815 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557056483150 0x55705648a3f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f16380ce990 tx=0x7f1638005c80 comp rx=0 tx=0).stop 2024-04-24T09:10:59.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.521+0000 7f164a430740 1 -- 172.21.15.83:0/3712646815 shutdown_connections 2024-04-24T09:10:59.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.521+0000 7f164a430740 1 --2- 172.21.15.83:0/3712646815 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557056483150 0x55705648a3f0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:59.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.521+0000 7f164a430740 1 --2- 172.21.15.83:0/3712646815 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557056470050 0x557056484090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:59.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.521+0000 7f164a430740 1 --2- 172.21.15.83:0/3712646815 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x5570564f99b0 0x5570564fbdd0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:59.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.521+0000 7f164a430740 1 --2- 172.21.15.83:0/3712646815 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55705646f6a0 0x557056483b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:59.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.521+0000 7f164a430740 1 -- 172.21.15.83:0/3712646815 >> 172.21.15.83:0/3712646815 conn(0x55705646bc70 msgr2=0x557056471a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:59.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.521+0000 7f1626000640 0 client.39359 ms_handle_reset on v2:172.21.15.138:6832/1157967658 2024-04-24T09:10:59.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.522+0000 7f164a430740 1 -- 172.21.15.83:0/3712646815 shutdown_connections 2024-04-24T09:10:59.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.522+0000 7f164a430740 1 -- 172.21.15.83:0/3712646815 wait complete. 2024-04-24T09:10:59.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.522+0000 7f164a430740 20 client.39359 trim_cache size 0 max 16384 2024-04-24T09:10:59.524 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:10:59.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.523+0000 7f1647600640 1 -- 172.21.15.83:0/2737237643 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f16280a3da0 msgr2=0x7f16280a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:59.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.523+0000 7f1647600640 1 --2- 172.21.15.83:0/2737237643 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f16280a3da0 0x7f16280a6260 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f162c004650 tx=0x7f162c009560 comp rx=0 tx=0).stop 2024-04-24T09:10:59.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.523+0000 7f1647600640 1 -- 172.21.15.83:0/2737237643 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f164010c630 msgr2=0x7f164019a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:59.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.523+0000 7f1647600640 1 --2- 172.21.15.83:0/2737237643 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f164010c630 0x7f164019a900 secure :-1 s=READY pgs=2467 cs=0 l=1 rev1=1 crypto rx=0x7f1638005fd0 tx=0x7f163800ef10 comp rx=0 tx=0).stop 2024-04-24T09:10:59.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.523+0000 7f1647600640 1 -- 172.21.15.83:0/2737237643 shutdown_connections 2024-04-24T09:10:59.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.523+0000 7f1647600640 1 --2- 172.21.15.83:0/2737237643 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f16280a3da0 0x7f16280a6260 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:59.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.524+0000 7f1647600640 1 --2- 172.21.15.83:0/2737237643 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f164010c630 0x7f164019a900 unknown :-1 s=CLOSED pgs=2467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:59.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.524+0000 7f1647600640 1 --2- 172.21.15.83:0/2737237643 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1640103b80 0x7f16401a0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:59.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.524+0000 7f1647600640 1 --2- 172.21.15.83:0/2737237643 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1640103260 0x7f16401a0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:59.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.524+0000 7f1647600640 1 -- 172.21.15.83:0/2737237643 >> 172.21.15.83:0/2737237643 conn(0x7f1640077280 msgr2=0x7f1640100ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:59.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.524+0000 7f1647600640 1 -- 172.21.15.83:0/2737237643 shutdown_connections 2024-04-24T09:10:59.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.524+0000 7f1647600640 1 -- 172.21.15.83:0/2737237643 wait complete. 2024-04-24T09:10:59.547 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T09:10:59.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.805+0000 7f16be400640 1 -- 172.21.15.83:0/3199661217 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f16b81073d0 msgr2=0x7f16b806ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:59.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.805+0000 7f16be400640 1 --2- 172.21.15.83:0/3199661217 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f16b81073d0 0x7f16b806ab40 secure :-1 s=READY pgs=2469 cs=0 l=1 rev1=1 crypto rx=0x7f16a0009860 tx=0x7f16a002a360 comp rx=0 tx=0).stop 2024-04-24T09:10:59.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.806+0000 7f16be400640 1 -- 172.21.15.83:0/3199661217 shutdown_connections 2024-04-24T09:10:59.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.806+0000 7f16be400640 1 --2- 172.21.15.83:0/3199661217 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16b806b220 0x7f16b8104a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:59.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.806+0000 7f16be400640 1 --2- 172.21.15.83:0/3199661217 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f16b81073d0 0x7f16b806ab40 unknown :-1 s=CLOSED pgs=2469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:59.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.806+0000 7f16be400640 1 --2- 172.21.15.83:0/3199661217 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f16b8106ff0 0x7f16b806a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:59.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.806+0000 7f16be400640 1 -- 172.21.15.83:0/3199661217 >> 172.21.15.83:0/3199661217 conn(0x7f16b8076d00 msgr2=0x7f16b8077110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:59.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.806+0000 7f16be400640 1 -- 172.21.15.83:0/3199661217 shutdown_connections 2024-04-24T09:10:59.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.806+0000 7f16be400640 1 -- 172.21.15.83:0/3199661217 wait complete. 2024-04-24T09:10:59.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.808+0000 7f16be400640 1 Processor -- start 2024-04-24T09:10:59.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.808+0000 7f16be400640 1 -- start start 2024-04-24T09:10:59.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.808+0000 7f16be400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f16b8106ff0 0x7f16b819ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:59.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.808+0000 7f16be400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f16b81073d0 0x7f16b819f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:59.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.808+0000 7f16be400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16b81a3820 0x7f16b81a3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:59.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.808+0000 7f16be400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f16b81188e0 con 0x7f16b8106ff0 2024-04-24T09:10:59.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.808+0000 7f16be400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f16b8118760 con 0x7f16b81a3820 2024-04-24T09:10:59.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.808+0000 7f16be400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f16b8118a60 con 0x7f16b81073d0 2024-04-24T09:10:59.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.809+0000 7f16b7400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f16b8106ff0 0x7f16b819ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:59.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.809+0000 7f16b7400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f16b8106ff0 0x7f16b819ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58430/0 (socket says 172.21.15.83:58430) 2024-04-24T09:10:59.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.809+0000 7f16b7400640 1 -- 172.21.15.83:0/1115448459 learned_addr learned my addr 172.21.15.83:0/1115448459 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:10:59.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.809+0000 7f16b7e00640 1 --2- 172.21.15.83:0/1115448459 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16b81a3820 0x7f16b81a3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:59.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.809+0000 7f16b6a00640 1 --2- 172.21.15.83:0/1115448459 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f16b81073d0 0x7f16b819f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:59.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.809+0000 7f16b7400640 1 -- 172.21.15.83:0/1115448459 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f16b81073d0 msgr2=0x7f16b819f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:59.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.809+0000 7f16b7400640 1 --2- 172.21.15.83:0/1115448459 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f16b81073d0 0x7f16b819f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:59.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.809+0000 7f16b7400640 1 -- 172.21.15.83:0/1115448459 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16b81a3820 msgr2=0x7f16b81a3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:59.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.809+0000 7f16b7400640 1 --2- 172.21.15.83:0/1115448459 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16b81a3820 0x7f16b81a3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:59.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.809+0000 7f16b7400640 1 -- 172.21.15.83:0/1115448459 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16b81a4310 con 0x7f16b8106ff0 2024-04-24T09:10:59.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.809+0000 7f16b7e00640 1 --2- 172.21.15.83:0/1115448459 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16b81a3820 0x7f16b81a3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:10:59.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.810+0000 7f16b7400640 1 --2- 172.21.15.83:0/1115448459 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f16b8106ff0 0x7f16b819ef60 secure :-1 s=READY pgs=2470 cs=0 l=1 rev1=1 crypto rx=0x7f16a800e9f0 tx=0x7f16a800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:59.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.810+0000 7f16afe00640 1 -- 172.21.15.83:0/1115448459 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a800cd80 con 0x7f16b8106ff0 2024-04-24T09:10:59.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.810+0000 7f16afe00640 1 -- 172.21.15.83:0/1115448459 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f16a80054d0 con 0x7f16b8106ff0 2024-04-24T09:10:59.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.810+0000 7f16be400640 1 -- 172.21.15.83:0/1115448459 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16b81a4600 con 0x7f16b8106ff0 2024-04-24T09:10:59.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.810+0000 7f16afe00640 1 -- 172.21.15.83:0/1115448459 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a8010490 con 0x7f16b8106ff0 2024-04-24T09:10:59.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.810+0000 7f16be400640 1 -- 172.21.15.83:0/1115448459 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16b81abdc0 con 0x7f16b8106ff0 2024-04-24T09:10:59.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.814+0000 7f16c13e3740 1 Processor -- start 2024-04-24T09:10:59.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.814+0000 7f16c13e3740 1 -- start start 2024-04-24T09:10:59.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.815+0000 7f16c13e3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bfe8c7950 0x562bfe8c7d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:59.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.815+0000 7f16c13e3740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562bfe8c8270 0x562bfe8d2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:59.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.815+0000 7f16c13e3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bfe8d2940 0x562bfe8d4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:59.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.815+0000 7f16c13e3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562bfe82d1d0 con 0x562bfe8c7950 2024-04-24T09:10:59.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.815+0000 7f16c13e3740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562bfe7d3450 con 0x562bfe8c8270 2024-04-24T09:10:59.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.815+0000 7f16c13e3740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x562bfe829190 con 0x562bfe8d2940 2024-04-24T09:10:59.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.815+0000 7f16b7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bfe8c7950 0x562bfe8c7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:59.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.815+0000 7f16b7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bfe8c7950 0x562bfe8c7d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58446/0 (socket says 172.21.15.83:58446) 2024-04-24T09:10:59.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.815+0000 7f16b7e00640 1 -- 172.21.15.83:0/846551662 learned_addr learned my addr 172.21.15.83:0/846551662 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:10:59.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.816+0000 7f16afe00640 1 -- 172.21.15.83:0/1115448459 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f16a8005020 con 0x7f16b8106ff0 2024-04-24T09:10:59.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.816+0000 7f16b6a00640 1 --2- 172.21.15.83:0/846551662 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bfe8d2940 0x562bfe8d4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:59.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.816+0000 7f16b7e00640 1 -- 172.21.15.83:0/846551662 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bfe8d2940 msgr2=0x562bfe8d4d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:59.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.816+0000 7f16b7400640 1 --2- 172.21.15.83:0/846551662 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562bfe8c8270 0x562bfe8d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:59.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.816+0000 7f16b7e00640 1 --2- 172.21.15.83:0/846551662 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bfe8d2940 0x562bfe8d4d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:59.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.816+0000 7f16b7e00640 1 -- 172.21.15.83:0/846551662 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562bfe8c8270 msgr2=0x562bfe8d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:59.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.816+0000 7f16b7e00640 1 --2- 172.21.15.83:0/846551662 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562bfe8c8270 0x562bfe8d2400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:59.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.816+0000 7f16b7e00640 1 -- 172.21.15.83:0/846551662 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562bfe890310 con 0x562bfe8c7950 2024-04-24T09:10:59.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.816+0000 7f16b6a00640 1 --2- 172.21.15.83:0/846551662 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bfe8d2940 0x562bfe8d4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:10:59.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.817+0000 7f16b7e00640 1 --2- 172.21.15.83:0/846551662 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bfe8c7950 0x562bfe8c7d30 secure :-1 s=READY pgs=2471 cs=0 l=1 rev1=1 crypto rx=0x7f16b0009f80 tx=0x7f16b000f890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51a5591e36677704 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:59.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.817+0000 7f16acc00640 1 -- 172.21.15.83:0/846551662 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16b001d070 con 0x562bfe8c7950 2024-04-24T09:10:59.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.817+0000 7f16acc00640 1 -- 172.21.15.83:0/846551662 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f16b000caf0 con 0x562bfe8c7950 2024-04-24T09:10:59.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.818+0000 7f16acc00640 1 -- 172.21.15.83:0/846551662 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16b0018c60 con 0x562bfe8c7950 2024-04-24T09:10:59.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.818+0000 7f16afe00640 1 --2- 172.21.15.83:0/1115448459 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f169c0a3e60 0x7f169c0a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:59.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.818+0000 7f16afe00640 1 -- 172.21.15.83:0/1115448459 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 9583+0+0 (secure 0 0 0) 0x7f16a80d1080 con 0x7f16b8106ff0 2024-04-24T09:10:59.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.819+0000 7f16c13e3740 1 -- 172.21.15.83:0/846551662 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bfe8c7950 msgr2=0x562bfe8c7d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:59.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.819+0000 7f16c13e3740 1 --2- 172.21.15.83:0/846551662 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bfe8c7950 0x562bfe8c7d30 secure :-1 s=READY pgs=2471 cs=0 l=1 rev1=1 crypto rx=0x7f16b0009f80 tx=0x7f16b000f890 comp rx=0 tx=0).stop 2024-04-24T09:10:59.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.819+0000 7f16c13e3740 1 -- 172.21.15.83:0/846551662 shutdown_connections 2024-04-24T09:10:59.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.819+0000 7f16c13e3740 1 --2- 172.21.15.83:0/846551662 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bfe8d2940 0x562bfe8d4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:59.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.819+0000 7f16c13e3740 1 --2- 172.21.15.83:0/846551662 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562bfe8c8270 0x562bfe8d2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:59.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.819+0000 7f16c13e3740 1 --2- 172.21.15.83:0/846551662 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bfe8c7950 0x562bfe8c7d30 unknown :-1 s=CLOSED pgs=2471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:59.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.819+0000 7f16c13e3740 1 -- 172.21.15.83:0/846551662 >> 172.21.15.83:0/846551662 conn(0x562bfe8c3c70 msgr2=0x562bfe8c4050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:59.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.819+0000 7f16b7400640 1 --2- 172.21.15.83:0/1115448459 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f169c0a3e60 0x7f169c0a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:59.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.819+0000 7f16c13e3740 1 -- 172.21.15.83:0/846551662 shutdown_connections 2024-04-24T09:10:59.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.820+0000 7f16c13e3740 1 -- 172.21.15.83:0/846551662 wait complete. 2024-04-24T09:10:59.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.820+0000 7f16b7400640 1 --2- 172.21.15.83:0/1115448459 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f169c0a3e60 0x7f169c0a6320 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x562bfe8c8f10 tx=0x7f16a809ca80 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:59.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.820+0000 7f16c13e3740 1 -- start start 2024-04-24T09:10:59.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.821+0000 7f16c13e3740 1 Processor -- start 2024-04-24T09:10:59.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.821+0000 7f16c13e3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bfe8c7950 0x562bfe8d23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:59.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.821+0000 7f16c13e3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bfe8c8270 0x562bfe8cf6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:59.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.821+0000 7f16c13e3740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562bfe8d2940 0x562bfe8cfdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:59.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.821+0000 7f16c13e3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562bfe8d57d0 con 0x562bfe8c8270 2024-04-24T09:10:59.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.821+0000 7f16c13e3740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562bfe7e6810 con 0x562bfe8d2940 2024-04-24T09:10:59.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.821+0000 7f16c13e3740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x562bfe8d5950 con 0x562bfe8c7950 2024-04-24T09:10:59.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.822+0000 7f16c13e3740 10 client.0 _ll_register_callbacks cb 0x562bfe7b4140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:10:59.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.822+0000 7f16b6a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bfe8c8270 0x562bfe8cf6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:59.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.822+0000 7f16b6a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bfe8c8270 0x562bfe8cf6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58460/0 (socket says 172.21.15.83:58460) 2024-04-24T09:10:59.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.822+0000 7f16b6a00640 1 -- 172.21.15.83:0/114750584 learned_addr learned my addr 172.21.15.83:0/114750584 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:10:59.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.822+0000 7f16b7e00640 1 --2- 172.21.15.83:0/114750584 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562bfe8d2940 0x562bfe8cfdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:59.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.822+0000 7f16b7e00640 1 --2- 172.21.15.83:0/114750584 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bfe8c7950 0x562bfe8d23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:59.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.822+0000 7f16b6a00640 1 -- 172.21.15.83:0/114750584 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bfe8c7950 msgr2=0x562bfe8d23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:59.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.823+0000 7f16b6a00640 1 --2- 172.21.15.83:0/114750584 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bfe8c7950 0x562bfe8d23c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:59.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.823+0000 7f16b6a00640 1 -- 172.21.15.83:0/114750584 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562bfe8d2940 msgr2=0x562bfe8cfdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:59.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.823+0000 7f16b6a00640 1 --2- 172.21.15.83:0/114750584 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562bfe8d2940 0x562bfe8cfdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:59.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.823+0000 7f16b6a00640 1 -- 172.21.15.83:0/114750584 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562bfe7f73f0 con 0x562bfe8c8270 2024-04-24T09:10:59.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.823+0000 7f16b7e00640 1 --2- 172.21.15.83:0/114750584 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562bfe8d2940 0x562bfe8cfdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:10:59.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.823+0000 7f16b6a00640 1 --2- 172.21.15.83:0/114750584 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bfe8c8270 0x562bfe8cf6b0 secure :-1 s=READY pgs=2472 cs=0 l=1 rev1=1 crypto rx=0x7f16a0005d00 tx=0x7f16a0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f12b9597044beac server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:59.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.823+0000 7f16a5600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:10:59.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.823+0000 7f16a5600640 1 -- 172.21.15.83:0/114750584 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a0037040 con 0x562bfe8c8270 2024-04-24T09:10:59.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.823+0000 7f16c13e3740 1 -- 172.21.15.83:0/114750584 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16b000caf0 con 0x562bfe8c8270 2024-04-24T09:10:59.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.823+0000 7f16a5600640 1 -- 172.21.15.83:0/114750584 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(120..120 src has 1..120) ==== 9583+0+0 (secure 0 0 0) 0x7f16a0030450 con 0x562bfe8c8270 2024-04-24T09:10:59.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.824+0000 7f16a5600640 1 -- 172.21.15.83:0/114750584 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f16a004b050 con 0x562bfe8c8270 2024-04-24T09:10:59.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.824+0000 7f16c13e3740 10 client.22290 fetch_fsmap learned FSMap version 61 2024-04-24T09:10:59.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.824+0000 7f16c13e3740 1 -- 172.21.15.83:0/114750584 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x562bfe53f3f0 con 0x562bfe8c8270 2024-04-24T09:10:59.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.824+0000 7f16a5600640 1 -- 172.21.15.83:0/114750584 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f16b000caf0 con 0x562bfe8c8270 2024-04-24T09:10:59.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.824+0000 7f16a5600640 1 -- 172.21.15.83:0/114750584 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 61) ==== 8362+0+0 (secure 0 0 0) 0x7f16a00046c0 con 0x562bfe8c8270 2024-04-24T09:10:59.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.824+0000 7f16c13e3740 10 client.22290 fetch_fsmap finished waiting for FSMap version 61 2024-04-24T09:10:59.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.824+0000 7f16c13e3740 10 client.22290 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-24T09:10:59.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.824+0000 7f16c13e3740 20 client.22290 populate_metadata read hostname 'smithi083' 2024-04-24T09:10:59.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.824+0000 7f16c13e3740 1 --2- 172.21.15.83:0/114750584 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x562bfe8e2b60 0x562bfe8e4f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:59.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.824+0000 7f16c13e3740 4 client.22290 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:10:59.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.824+0000 7f16c13e3740 1 -- 172.21.15.83:0/114750584 --> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562bfe83abf0 con 0x562bfe8e2b60 2024-04-24T09:10:59.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.825+0000 7f16b7e00640 1 --2- 172.21.15.83:0/114750584 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x562bfe8e2b60 0x562bfe8e4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:59.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.826+0000 7f16b7e00640 1 --2- 172.21.15.83:0/114750584 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x562bfe8e2b60 0x562bfe8e4f80 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=809b379a1a542ffa server_cookie=f6c1d1bb825d61f2 in_seq=0 out_seq=0 2024-04-24T09:10:59.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.826+0000 7f16a5600640 10 client.22290 ms_handle_connect on v2:172.21.15.138:6834/1909219758 2024-04-24T09:10:59.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.827+0000 7f16a5600640 1 -- 172.21.15.83:0/114750584 <== mds.2 v2:172.21.15.138:6834/1909219758 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562bfe83abf0 con 0x562bfe8e2b60 2024-04-24T09:10:59.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.827+0000 7f16a5600640 10 client.22290 handle_command_reply: tid=0 2024-04-24T09:10:59.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.827+0000 7f16c13e3740 1 client.22290 shutdown 2024-04-24T09:10:59.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.827+0000 7f16c13e3740 1 -- 172.21.15.83:0/114750584 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bfe8c8270 msgr2=0x562bfe8cf6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:59.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.827+0000 7f16c13e3740 1 --2- 172.21.15.83:0/114750584 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bfe8c8270 0x562bfe8cf6b0 secure :-1 s=READY pgs=2472 cs=0 l=1 rev1=1 crypto rx=0x7f16a0005d00 tx=0x7f16a0004270 comp rx=0 tx=0).stop 2024-04-24T09:10:59.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.827+0000 7f16c13e3740 1 -- 172.21.15.83:0/114750584 shutdown_connections 2024-04-24T09:10:59.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.827+0000 7f16c13e3740 1 --2- 172.21.15.83:0/114750584 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x562bfe8e2b60 0x562bfe8e4f80 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:59.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.827+0000 7f16c13e3740 1 --2- 172.21.15.83:0/114750584 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562bfe8d2940 0x562bfe8cfdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:59.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.827+0000 7f16c13e3740 1 --2- 172.21.15.83:0/114750584 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bfe8c8270 0x562bfe8cf6b0 unknown :-1 s=CLOSED pgs=2472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:59.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.827+0000 7f16c13e3740 1 --2- 172.21.15.83:0/114750584 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bfe8c7950 0x562bfe8d23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:59.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.827+0000 7f16c13e3740 1 -- 172.21.15.83:0/114750584 >> 172.21.15.83:0/114750584 conn(0x562bfe8c3c70 msgr2=0x562bfe8c9970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:59.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.827+0000 7f16a5600640 0 client.22290 ms_handle_reset on v2:172.21.15.138:6834/1909219758 2024-04-24T09:10:59.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.828+0000 7f16c13e3740 1 -- 172.21.15.83:0/114750584 shutdown_connections 2024-04-24T09:10:59.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.828+0000 7f16c13e3740 1 -- 172.21.15.83:0/114750584 wait complete. 2024-04-24T09:10:59.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.828+0000 7f16c13e3740 20 client.22290 trim_cache size 0 max 16384 2024-04-24T09:10:59.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.851+0000 7f16c13e3740 1 Processor -- start 2024-04-24T09:10:59.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.851+0000 7f16c13e3740 1 -- start start 2024-04-24T09:10:59.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.852+0000 7f16c13e3740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562bfe8c7950 0x562bfe8de690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:59.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.852+0000 7f16c13e3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bfe8c8270 0x562bfe8ce690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:59.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.852+0000 7f16c13e3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bfe8d2940 0x562bfe8d1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:59.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.852+0000 7f16c13e3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f16b81b5c00 con 0x562bfe8d2940 2024-04-24T09:10:59.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.852+0000 7f16c13e3740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f16b81b7740 con 0x562bfe8c7950 2024-04-24T09:10:59.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.852+0000 7f16c13e3740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f16b81b62d0 con 0x562bfe8c8270 2024-04-24T09:10:59.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.852+0000 7f16b7e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562bfe8c7950 0x562bfe8de690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:59.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.852+0000 7f16b7e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562bfe8c7950 0x562bfe8de690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:41804/0 (socket says 172.21.15.83:41804) 2024-04-24T09:10:59.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.852+0000 7f16b7e00640 1 -- 172.21.15.83:0/943464749 learned_addr learned my addr 172.21.15.83:0/943464749 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:10:59.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.852+0000 7f16b7e00640 1 --2- 172.21.15.83:0/943464749 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bfe8d2940 0x562bfe8d1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:59.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.853+0000 7f16b6a00640 1 --2- 172.21.15.83:0/943464749 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bfe8c8270 0x562bfe8ce690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:59.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.853+0000 7f16b7e00640 1 -- 172.21.15.83:0/943464749 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bfe8c8270 msgr2=0x562bfe8ce690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:59.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.853+0000 7f16b7e00640 1 --2- 172.21.15.83:0/943464749 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bfe8c8270 0x562bfe8ce690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:59.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.853+0000 7f16b7e00640 1 -- 172.21.15.83:0/943464749 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562bfe8c7950 msgr2=0x562bfe8de690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:59.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.853+0000 7f16b7e00640 1 --2- 172.21.15.83:0/943464749 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562bfe8c7950 0x562bfe8de690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:59.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.853+0000 7f16b7e00640 1 -- 172.21.15.83:0/943464749 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562bfe6097d0 con 0x562bfe8d2940 2024-04-24T09:10:59.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.853+0000 7f16b7e00640 1 --2- 172.21.15.83:0/943464749 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bfe8d2940 0x562bfe8d1ef0 secure :-1 s=READY pgs=2473 cs=0 l=1 rev1=1 crypto rx=0x7f16b0018670 tx=0x7f16b000f890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72b2983b8b883198 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:59.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.854+0000 7f16a7e00640 1 -- 172.21.15.83:0/943464749 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16b001d040 con 0x562bfe8d2940 2024-04-24T09:10:59.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.854+0000 7f16a7e00640 1 -- 172.21.15.83:0/943464749 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f16b0031070 con 0x562bfe8d2940 2024-04-24T09:10:59.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.854+0000 7f16a7e00640 1 -- 172.21.15.83:0/943464749 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16b0018970 con 0x562bfe8d2940 2024-04-24T09:10:59.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.855+0000 7f16c13e3740 1 -- 172.21.15.83:0/943464749 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bfe8d2940 msgr2=0x562bfe8d1ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:59.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.855+0000 7f16c13e3740 1 --2- 172.21.15.83:0/943464749 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bfe8d2940 0x562bfe8d1ef0 secure :-1 s=READY pgs=2473 cs=0 l=1 rev1=1 crypto rx=0x7f16b0018670 tx=0x7f16b000f890 comp rx=0 tx=0).stop 2024-04-24T09:10:59.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.855+0000 7f16c13e3740 1 -- 172.21.15.83:0/943464749 shutdown_connections 2024-04-24T09:10:59.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.855+0000 7f16c13e3740 1 --2- 172.21.15.83:0/943464749 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bfe8d2940 0x562bfe8d1ef0 unknown :-1 s=CLOSED pgs=2473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:59.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.855+0000 7f16c13e3740 1 --2- 172.21.15.83:0/943464749 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bfe8c8270 0x562bfe8ce690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:59.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.855+0000 7f16c13e3740 1 --2- 172.21.15.83:0/943464749 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562bfe8c7950 0x562bfe8de690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:59.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.855+0000 7f16c13e3740 1 -- 172.21.15.83:0/943464749 >> 172.21.15.83:0/943464749 conn(0x562bfe8c3c70 msgr2=0x562bfe8cdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:59.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.855+0000 7f16c13e3740 1 -- 172.21.15.83:0/943464749 shutdown_connections 2024-04-24T09:10:59.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.855+0000 7f16c13e3740 1 -- 172.21.15.83:0/943464749 wait complete. 2024-04-24T09:10:59.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.856+0000 7f16c13e3740 1 -- start start 2024-04-24T09:10:59.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.856+0000 7f16c13e3740 1 Processor -- start 2024-04-24T09:10:59.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.856+0000 7f16c13e3740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562bfe8c7950 0x562bfe8daea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:59.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.856+0000 7f16c13e3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bfe8c8270 0x562bfe8d32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:59.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.856+0000 7f16c13e3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bfe8d2940 0x562bfe8d3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:59.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.856+0000 7f16c13e3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f16b81b7070 con 0x562bfe8c8270 2024-04-24T09:10:59.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.856+0000 7f16c13e3740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f16b81b69a0 con 0x562bfe8c7950 2024-04-24T09:10:59.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.856+0000 7f16c13e3740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f16b81b3c00 con 0x562bfe8d2940 2024-04-24T09:10:59.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.857+0000 7f16c13e3740 10 client.0 _ll_register_callbacks cb 0x7f16a0035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:10:59.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.857+0000 7f16b6a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bfe8c8270 0x562bfe8d32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:59.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.857+0000 7f16b6a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bfe8c8270 0x562bfe8d32c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58474/0 (socket says 172.21.15.83:58474) 2024-04-24T09:10:59.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.857+0000 7f16b6a00640 1 -- 172.21.15.83:0/673711413 learned_addr learned my addr 172.21.15.83:0/673711413 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:10:59.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.857+0000 7f16b7e00640 1 --2- 172.21.15.83:0/673711413 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562bfe8c7950 0x562bfe8daea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:59.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.857+0000 7f16b6a00640 1 -- 172.21.15.83:0/673711413 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bfe8d2940 msgr2=0x562bfe8d3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:59.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.857+0000 7f16b6a00640 1 --2- 172.21.15.83:0/673711413 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bfe8d2940 0x562bfe8d3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:59.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.857+0000 7f16b6a00640 1 -- 172.21.15.83:0/673711413 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562bfe8c7950 msgr2=0x562bfe8daea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:10:59.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.857+0000 7f16b6a00640 1 --2- 172.21.15.83:0/673711413 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562bfe8c7950 0x562bfe8daea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:59.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.857+0000 7f16b6a00640 1 -- 172.21.15.83:0/673711413 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562bfe8d3ef0 con 0x562bfe8c8270 2024-04-24T09:10:59.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.857+0000 7f16b7e00640 1 --2- 172.21.15.83:0/673711413 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562bfe8c7950 0x562bfe8daea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:10:59.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.858+0000 7f16b6a00640 1 --2- 172.21.15.83:0/673711413 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bfe8c8270 0x562bfe8d32c0 secure :-1 s=READY pgs=2474 cs=0 l=1 rev1=1 crypto rx=0x7f16a0004c80 tx=0x7f16a00048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3465fd405613ef31 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:10:59.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.858+0000 7f16a5600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:10:59.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.858+0000 7f16a5600640 1 -- 172.21.15.83:0/673711413 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a0037070 con 0x562bfe8c8270 2024-04-24T09:10:59.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.858+0000 7f16a5600640 1 -- 172.21.15.83:0/673711413 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(120..120 src has 1..120) ==== 9583+0+0 (secure 0 0 0) 0x7f16a0030450 con 0x562bfe8c8270 2024-04-24T09:10:59.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.858+0000 7f16c13e3740 1 -- 172.21.15.83:0/673711413 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16b0031070 con 0x562bfe8c8270 2024-04-24T09:10:59.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.859+0000 7f16a5600640 1 -- 172.21.15.83:0/673711413 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f16a0030670 con 0x562bfe8c8270 2024-04-24T09:10:59.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.859+0000 7f16c13e3740 10 client.22296 fetch_fsmap learned FSMap version 61 2024-04-24T09:10:59.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.859+0000 7f16c13e3740 1 -- 172.21.15.83:0/673711413 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x562bfe9490c0 con 0x562bfe8c8270 2024-04-24T09:10:59.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.860+0000 7f16a5600640 1 -- 172.21.15.83:0/673711413 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f16b0031070 con 0x562bfe8c8270 2024-04-24T09:10:59.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.860+0000 7f16a5600640 1 -- 172.21.15.83:0/673711413 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 61) ==== 8362+0+0 (secure 0 0 0) 0x7f16a0004d00 con 0x562bfe8c8270 2024-04-24T09:10:59.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.860+0000 7f16c13e3740 10 client.22296 fetch_fsmap finished waiting for FSMap version 61 2024-04-24T09:10:59.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.860+0000 7f16c13e3740 10 client.22296 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-24T09:10:59.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.860+0000 7f16c13e3740 20 client.22296 populate_metadata read hostname 'smithi083' 2024-04-24T09:10:59.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.860+0000 7f16c13e3740 1 --2- 172.21.15.83:0/673711413 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x562bfe9497e0 0x562bfe94bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:10:59.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.860+0000 7f16c13e3740 4 client.22296 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:10:59.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.860+0000 7f16c13e3740 1 -- 172.21.15.83:0/673711413 --> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562bfe834a60 con 0x562bfe9497e0 2024-04-24T09:10:59.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.860+0000 7f16b7e00640 1 --2- 172.21.15.83:0/673711413 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x562bfe9497e0 0x562bfe94bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:10:59.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.861+0000 7f16b7e00640 1 --2- 172.21.15.83:0/673711413 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x562bfe9497e0 0x562bfe94bc00 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a2a93cf01f1d216a server_cookie=8c54a66fe1a816c in_seq=0 out_seq=0 2024-04-24T09:10:59.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.861+0000 7f16a5600640 10 client.22296 ms_handle_connect on v2:172.21.15.138:6834/1909219758 2024-04-24T09:10:59.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.862+0000 7f16a5600640 1 -- 172.21.15.83:0/673711413 <== mds.2 v2:172.21.15.138:6834/1909219758 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562bfe834a60 con 0x562bfe9497e0 2024-04-24T09:10:59.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.862+0000 7f16a5600640 10 client.22296 handle_command_reply: tid=0 2024-04-24T09:10:59.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.862+0000 7f16c13e3740 1 client.22296 shutdown 2024-04-24T09:10:59.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.863+0000 7f16c13e3740 1 -- 172.21.15.83:0/673711413 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bfe8c8270 msgr2=0x562bfe8d32c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:59.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.863+0000 7f16c13e3740 1 --2- 172.21.15.83:0/673711413 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bfe8c8270 0x562bfe8d32c0 secure :-1 s=READY pgs=2474 cs=0 l=1 rev1=1 crypto rx=0x7f16a0004c80 tx=0x7f16a00048f0 comp rx=0 tx=0).stop 2024-04-24T09:10:59.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.863+0000 7f16c13e3740 1 -- 172.21.15.83:0/673711413 shutdown_connections 2024-04-24T09:10:59.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.863+0000 7f16c13e3740 1 --2- 172.21.15.83:0/673711413 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x562bfe9497e0 0x562bfe94bc00 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:59.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.863+0000 7f16c13e3740 1 --2- 172.21.15.83:0/673711413 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bfe8d2940 0x562bfe8d3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:59.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.863+0000 7f16c13e3740 1 --2- 172.21.15.83:0/673711413 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bfe8c8270 0x562bfe8d32c0 unknown :-1 s=CLOSED pgs=2474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:59.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.863+0000 7f16c13e3740 1 --2- 172.21.15.83:0/673711413 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562bfe8c7950 0x562bfe8daea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:59.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.863+0000 7f16c13e3740 1 -- 172.21.15.83:0/673711413 >> 172.21.15.83:0/673711413 conn(0x562bfe8c3c70 msgr2=0x562bfe8c2560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:59.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.863+0000 7f16a5600640 0 client.22296 ms_handle_reset on v2:172.21.15.138:6834/1909219758 2024-04-24T09:10:59.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.863+0000 7f16c13e3740 1 -- 172.21.15.83:0/673711413 shutdown_connections 2024-04-24T09:10:59.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.863+0000 7f16c13e3740 1 -- 172.21.15.83:0/673711413 wait complete. 2024-04-24T09:10:59.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.864+0000 7f16c13e3740 20 client.22296 trim_cache size 0 max 16384 2024-04-24T09:10:59.864 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:10:59.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.864+0000 7f16be400640 1 -- 172.21.15.83:0/1115448459 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f169c0a3e60 msgr2=0x7f169c0a6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:59.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.864+0000 7f16be400640 1 --2- 172.21.15.83:0/1115448459 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f169c0a3e60 0x7f169c0a6320 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x562bfe8c8f10 tx=0x7f16a809ca80 comp rx=0 tx=0).stop 2024-04-24T09:10:59.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.865+0000 7f16be400640 1 -- 172.21.15.83:0/1115448459 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f16b8106ff0 msgr2=0x7f16b819ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:10:59.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.865+0000 7f16be400640 1 --2- 172.21.15.83:0/1115448459 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f16b8106ff0 0x7f16b819ef60 secure :-1 s=READY pgs=2470 cs=0 l=1 rev1=1 crypto rx=0x7f16a800e9f0 tx=0x7f16a800edf0 comp rx=0 tx=0).stop 2024-04-24T09:10:59.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.865+0000 7f16be400640 1 -- 172.21.15.83:0/1115448459 shutdown_connections 2024-04-24T09:10:59.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.865+0000 7f16be400640 1 --2- 172.21.15.83:0/1115448459 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f169c0a3e60 0x7f169c0a6320 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:59.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.865+0000 7f16be400640 1 --2- 172.21.15.83:0/1115448459 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16b81a3820 0x7f16b81a3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:59.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.865+0000 7f16be400640 1 --2- 172.21.15.83:0/1115448459 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f16b81073d0 0x7f16b819f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:59.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.865+0000 7f16be400640 1 --2- 172.21.15.83:0/1115448459 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f16b8106ff0 0x7f16b819ef60 unknown :-1 s=CLOSED pgs=2470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:10:59.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.865+0000 7f16be400640 1 -- 172.21.15.83:0/1115448459 >> 172.21.15.83:0/1115448459 conn(0x7f16b8076d00 msgr2=0x7f16b80fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:10:59.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.865+0000 7f16be400640 1 -- 172.21.15.83:0/1115448459 shutdown_connections 2024-04-24T09:10:59.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:10:59.865+0000 7f16be400640 1 -- 172.21.15.83:0/1115448459 wait complete. 2024-04-24T09:10:59.887 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T09:11:00.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.165+0000 7ff087200640 1 -- 172.21.15.83:0/2946316857 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff088102120 msgr2=0x7ff088102500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:00.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.165+0000 7ff087200640 1 --2- 172.21.15.83:0/2946316857 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff088102120 0x7ff088102500 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff080009be0 tx=0x7ff08002a6e0 comp rx=0 tx=0).stop 2024-04-24T09:11:00.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.165+0000 7ff087200640 1 -- 172.21.15.83:0/2946316857 shutdown_connections 2024-04-24T09:11:00.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.165+0000 7ff087200640 1 --2- 172.21.15.83:0/2946316857 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff0881071f0 0x7ff08810edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:00.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.165+0000 7ff087200640 1 --2- 172.21.15.83:0/2946316857 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff088102ad0 0x7ff088106b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:00.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.165+0000 7ff087200640 1 --2- 172.21.15.83:0/2946316857 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff088102120 0x7ff088102500 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:00.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.165+0000 7ff087200640 1 -- 172.21.15.83:0/2946316857 >> 172.21.15.83:0/2946316857 conn(0x7ff0880fd9a0 msgr2=0x7ff0880ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:00.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.166+0000 7ff087200640 1 -- 172.21.15.83:0/2946316857 shutdown_connections 2024-04-24T09:11:00.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.167+0000 7ff087200640 1 -- 172.21.15.83:0/2946316857 wait complete. 2024-04-24T09:11:00.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.169+0000 7ff087200640 1 Processor -- start 2024-04-24T09:11:00.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.169+0000 7ff087200640 1 -- start start 2024-04-24T09:11:00.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.170+0000 7ff087200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff088102120 0x7ff08819e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:00.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.170+0000 7ff087200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff0881071f0 0x7ff08819ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:00.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.170+0000 7ff087200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff0881986e0 0x7ff088198b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:00.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.171+0000 7ff085400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff0881071f0 0x7ff08819ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:00.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.171+0000 7ff086800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff0881986e0 0x7ff088198b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:00.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.171+0000 7ff085400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff0881071f0 0x7ff08819ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:41830/0 (socket says 172.21.15.83:41830) 2024-04-24T09:11:00.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.171+0000 7ff086800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff0881986e0 0x7ff088198b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:39670/0 (socket says 172.21.15.83:39670) 2024-04-24T09:11:00.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.172+0000 7ff087200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff088112200 con 0x7ff088102120 2024-04-24T09:11:00.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.172+0000 7ff087200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff088112080 con 0x7ff0881071f0 2024-04-24T09:11:00.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.172+0000 7ff087200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff088112380 con 0x7ff0881986e0 2024-04-24T09:11:00.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.173+0000 7ff085e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff088102120 0x7ff08819e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:00.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.173+0000 7ff085400640 1 -- 172.21.15.83:0/3148403063 learned_addr learned my addr 172.21.15.83:0/3148403063 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:11:00.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.174+0000 7ff085400640 1 -- 172.21.15.83:0/3148403063 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff0881986e0 msgr2=0x7ff088198b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:00.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.174+0000 7ff085400640 1 --2- 172.21.15.83:0/3148403063 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff0881986e0 0x7ff088198b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:00.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.174+0000 7ff085400640 1 -- 172.21.15.83:0/3148403063 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff088102120 msgr2=0x7ff08819e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:00.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.174+0000 7ff085400640 1 --2- 172.21.15.83:0/3148403063 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff088102120 0x7ff08819e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:00.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.174+0000 7ff085400640 1 -- 172.21.15.83:0/3148403063 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0881992c0 con 0x7ff0881071f0 2024-04-24T09:11:00.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.174+0000 7ff086800640 1 --2- 172.21.15.83:0/3148403063 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff0881986e0 0x7ff088198b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:11:00.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.175+0000 7ff085400640 1 --2- 172.21.15.83:0/3148403063 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff0881071f0 0x7ff08819ead0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff07400b450 tx=0x7ff07400b850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:00.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.176+0000 7ff07ea00640 1 -- 172.21.15.83:0/3148403063 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff074015020 con 0x7ff0881071f0 2024-04-24T09:11:00.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.176+0000 7ff07ea00640 1 -- 172.21.15.83:0/3148403063 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff074011ce0 con 0x7ff0881071f0 2024-04-24T09:11:00.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.176+0000 7ff07ea00640 1 -- 172.21.15.83:0/3148403063 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff074010dc0 con 0x7ff0881071f0 2024-04-24T09:11:00.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.176+0000 7ff087200640 1 -- 172.21.15.83:0/3148403063 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff08806ba20 con 0x7ff0881071f0 2024-04-24T09:11:00.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.177+0000 7ff087200640 1 -- 172.21.15.83:0/3148403063 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff08806bf20 con 0x7ff0881071f0 2024-04-24T09:11:00.178 INFO:tasks.workunit.client.0.smithi083.stdout: 10 41824 6.70 MB/sec execute 211 sec latency 100.392 ms 2024-04-24T09:11:00.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.181+0000 7ff08ff7a740 1 Processor -- start 2024-04-24T09:11:00.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.181+0000 7ff08ff7a740 1 -- start start 2024-04-24T09:11:00.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.181+0000 7ff08ff7a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f2b737f50 0x555f2b738330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:00.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.182+0000 7ff08ff7a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555f2b738870 0x555f2b742a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:00.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.182+0000 7ff08ff7a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f2b742f40 0x555f2b745330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:00.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.182+0000 7ff08ff7a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x555f2b6991d0 con 0x555f2b742f40 2024-04-24T09:11:00.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.182+0000 7ff08ff7a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555f2b63f450 con 0x555f2b737f50 2024-04-24T09:11:00.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.182+0000 7ff08ff7a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x555f2b695190 con 0x555f2b738870 2024-04-24T09:11:00.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.182+0000 7ff085e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555f2b738870 0x555f2b742a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:00.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.182+0000 7ff085e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555f2b738870 0x555f2b742a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:39674/0 (socket says 172.21.15.83:39674) 2024-04-24T09:11:00.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.182+0000 7ff085e00640 1 -- 172.21.15.83:0/2936192681 learned_addr learned my addr 172.21.15.83:0/2936192681 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:11:00.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.182+0000 7ff086800640 1 --2- 172.21.15.83:0/2936192681 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f2b737f50 0x555f2b738330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:00.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.183+0000 7ff085e00640 1 -- 172.21.15.83:0/2936192681 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f2b737f50 msgr2=0x555f2b738330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:00.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.183+0000 7ff085e00640 1 --2- 172.21.15.83:0/2936192681 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f2b737f50 0x555f2b738330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:00.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.183+0000 7ff085e00640 1 -- 172.21.15.83:0/2936192681 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f2b742f40 msgr2=0x555f2b745330 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T09:11:00.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.183+0000 7ff085e00640 1 --2- 172.21.15.83:0/2936192681 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f2b742f40 0x555f2b745330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:00.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.183+0000 7ff085e00640 1 -- 172.21.15.83:0/2936192681 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f2b6fc310 con 0x555f2b738870 2024-04-24T09:11:00.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.183+0000 7ff085e00640 1 --2- 172.21.15.83:0/2936192681 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555f2b738870 0x555f2b742a00 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff080009b30 tx=0x7ff0800305c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3d3cf8ebe729fed server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:00.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.183+0000 7ff073400640 1 -- 172.21.15.83:0/2936192681 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff080036040 con 0x555f2b738870 2024-04-24T09:11:00.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.183+0000 7ff073400640 1 -- 172.21.15.83:0/2936192681 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff080030730 con 0x555f2b738870 2024-04-24T09:11:00.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.184+0000 7ff073400640 1 -- 172.21.15.83:0/2936192681 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0800319d0 con 0x555f2b738870 2024-04-24T09:11:00.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.184+0000 7ff07ea00640 1 -- 172.21.15.83:0/3148403063 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff074018020 con 0x7ff0881071f0 2024-04-24T09:11:00.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.184+0000 7ff08ff7a740 1 -- 172.21.15.83:0/2936192681 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555f2b738870 msgr2=0x555f2b742a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:00.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.184+0000 7ff08ff7a740 1 --2- 172.21.15.83:0/2936192681 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555f2b738870 0x555f2b742a00 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff080009b30 tx=0x7ff0800305c0 comp rx=0 tx=0).stop 2024-04-24T09:11:00.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.185+0000 7ff08ff7a740 1 -- 172.21.15.83:0/2936192681 shutdown_connections 2024-04-24T09:11:00.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.185+0000 7ff08ff7a740 1 --2- 172.21.15.83:0/2936192681 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f2b742f40 0x555f2b745330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:00.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.185+0000 7ff08ff7a740 1 --2- 172.21.15.83:0/2936192681 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555f2b738870 0x555f2b742a00 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:00.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.185+0000 7ff08ff7a740 1 --2- 172.21.15.83:0/2936192681 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f2b737f50 0x555f2b738330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:00.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.185+0000 7ff08ff7a740 1 -- 172.21.15.83:0/2936192681 >> 172.21.15.83:0/2936192681 conn(0x555f2b734270 msgr2=0x555f2b734650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:00.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.185+0000 7ff08ff7a740 1 -- 172.21.15.83:0/2936192681 shutdown_connections 2024-04-24T09:11:00.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.185+0000 7ff08ff7a740 1 -- 172.21.15.83:0/2936192681 wait complete. 2024-04-24T09:11:00.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.185+0000 7ff08ff7a740 1 -- start start 2024-04-24T09:11:00.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.186+0000 7ff08ff7a740 1 Processor -- start 2024-04-24T09:11:00.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.186+0000 7ff08ff7a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555f2b737f50 0x555f2b742930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:00.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.186+0000 7ff08ff7a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f2b738870 0x555f2b73fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:00.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.187+0000 7ff08ff7a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f2b742f40 0x555f2b740230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:00.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.187+0000 7ff08ff7a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x555f2b745dc0 con 0x555f2b742f40 2024-04-24T09:11:00.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.187+0000 7ff08ff7a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555f2b652810 con 0x555f2b738870 2024-04-24T09:11:00.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.187+0000 7ff08ff7a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x555f2b745f40 con 0x555f2b737f50 2024-04-24T09:11:00.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.187+0000 7ff08ff7a740 10 client.0 _ll_register_callbacks cb 0x555f2b620140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:11:00.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.187+0000 7ff086800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555f2b737f50 0x555f2b742930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:00.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.187+0000 7ff07ea00640 1 --2- 172.21.15.83:0/3148403063 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff06c0b4dc0 0x7ff06c0b7280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:00.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.187+0000 7ff085e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f2b738870 0x555f2b73fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:00.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.187+0000 7ff086800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555f2b737f50 0x555f2b742930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:39690/0 (socket says 172.21.15.83:39690) 2024-04-24T09:11:00.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.187+0000 7ff086800640 1 -- 172.21.15.83:0/472599922 learned_addr learned my addr 172.21.15.83:0/472599922 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:11:00.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.187+0000 7ff07ea00640 1 -- 172.21.15.83:0/3148403063 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 9583+0+0 (secure 0 0 0) 0x7ff0740d0340 con 0x7ff0881071f0 2024-04-24T09:11:00.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.187+0000 7ff085400640 1 --2- 172.21.15.83:0/472599922 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f2b742f40 0x555f2b740230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:00.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.188+0000 7ff086800640 1 --2- 172.21.15.83:0/3148403063 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff06c0b4dc0 0x7ff06c0b7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:00.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.188+0000 7ff085400640 1 -- 172.21.15.83:0/472599922 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555f2b737f50 msgr2=0x555f2b742930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:00.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.188+0000 7ff085400640 1 --2- 172.21.15.83:0/472599922 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555f2b737f50 0x555f2b742930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:00.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.188+0000 7ff085400640 1 -- 172.21.15.83:0/472599922 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f2b738870 msgr2=0x555f2b73fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:00.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.188+0000 7ff085400640 1 --2- 172.21.15.83:0/472599922 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f2b738870 0x555f2b73fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:00.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.188+0000 7ff085400640 1 -- 172.21.15.83:0/472599922 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f2b6633f0 con 0x555f2b742f40 2024-04-24T09:11:00.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.188+0000 7ff085400640 1 --2- 172.21.15.83:0/472599922 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f2b742f40 0x555f2b740230 secure :-1 s=READY pgs=2475 cs=0 l=1 rev1=1 crypto rx=0x7ff0740cd720 tx=0x7ff074009280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c0f1984766cde4b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:00.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.188+0000 7ff067e00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:11:00.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.189+0000 7ff067e00640 1 -- 172.21.15.83:0/472599922 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0740cda90 con 0x555f2b742f40 2024-04-24T09:11:00.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.189+0000 7ff067e00640 1 -- 172.21.15.83:0/472599922 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(120..120 src has 1..120) ==== 9583+0+0 (secure 0 0 0) 0x7ff074011830 con 0x555f2b742f40 2024-04-24T09:11:00.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.189+0000 7ff08ff7a740 1 -- 172.21.15.83:0/472599922 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff080030730 con 0x555f2b742f40 2024-04-24T09:11:00.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.189+0000 7ff086800640 1 --2- 172.21.15.83:0/3148403063 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff06c0b4dc0 0x7ff06c0b7280 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7ff0780058c0 tx=0x7ff07801e8c0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:00.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.189+0000 7ff067e00640 1 -- 172.21.15.83:0/472599922 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7ff07409bf70 con 0x555f2b742f40 2024-04-24T09:11:00.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.189+0000 7ff08ff7a740 10 client.22305 fetch_fsmap learned FSMap version 61 2024-04-24T09:11:00.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.189+0000 7ff08ff7a740 1 -- 172.21.15.83:0/472599922 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x555f2b3ab3f0 con 0x555f2b742f40 2024-04-24T09:11:00.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.190+0000 7ff067e00640 1 -- 172.21.15.83:0/472599922 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff080030730 con 0x555f2b742f40 2024-04-24T09:11:00.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.190+0000 7ff067e00640 1 -- 172.21.15.83:0/472599922 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 61) ==== 8362+0+0 (secure 0 0 0) 0x7ff0740b5200 con 0x555f2b742f40 2024-04-24T09:11:00.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.191+0000 7ff08ff7a740 10 client.22305 fetch_fsmap finished waiting for FSMap version 61 2024-04-24T09:11:00.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.191+0000 7ff08ff7a740 10 client.22305 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-24T09:11:00.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.191+0000 7ff08ff7a740 20 client.22305 populate_metadata read hostname 'smithi083' 2024-04-24T09:11:00.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.191+0000 7ff08ff7a740 1 --2- 172.21.15.83:0/472599922 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x555f2b730b30 0x555f2b732f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:00.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.191+0000 7ff08ff7a740 4 client.22305 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:11:00.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.191+0000 7ff08ff7a740 1 -- 172.21.15.83:0/472599922 --> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555f2b6a6bf0 con 0x555f2b730b30 2024-04-24T09:11:00.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.191+0000 7ff085e00640 1 --2- 172.21.15.83:0/472599922 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x555f2b730b30 0x555f2b732f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:00.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.193+0000 7ff067e00640 10 client.22305 ms_handle_connect on v2:172.21.15.138:6836/900566754 2024-04-24T09:11:00.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.193+0000 7ff085e00640 1 --2- 172.21.15.83:0/472599922 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x555f2b730b30 0x555f2b732f50 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c26e90e8c82034a2 server_cookie=ede93e04202bbf54 in_seq=0 out_seq=0 2024-04-24T09:11:00.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.194+0000 7ff067e00640 1 -- 172.21.15.83:0/472599922 <== mds.1 v2:172.21.15.138:6836/900566754 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555f2b6a6bf0 con 0x555f2b730b30 2024-04-24T09:11:00.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.194+0000 7ff067e00640 10 client.22305 handle_command_reply: tid=0 2024-04-24T09:11:00.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.194+0000 7ff08ff7a740 1 client.22305 shutdown 2024-04-24T09:11:00.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.195+0000 7ff08ff7a740 1 -- 172.21.15.83:0/472599922 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f2b742f40 msgr2=0x555f2b740230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:00.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.195+0000 7ff08ff7a740 1 --2- 172.21.15.83:0/472599922 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f2b742f40 0x555f2b740230 secure :-1 s=READY pgs=2475 cs=0 l=1 rev1=1 crypto rx=0x7ff0740cd720 tx=0x7ff074009280 comp rx=0 tx=0).stop 2024-04-24T09:11:00.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.195+0000 7ff08ff7a740 1 -- 172.21.15.83:0/472599922 shutdown_connections 2024-04-24T09:11:00.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.195+0000 7ff08ff7a740 1 --2- 172.21.15.83:0/472599922 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f2b742f40 0x555f2b740230 unknown :-1 s=CLOSED pgs=2475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:00.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.195+0000 7ff08ff7a740 1 --2- 172.21.15.83:0/472599922 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f2b738870 0x555f2b73fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:00.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.195+0000 7ff08ff7a740 1 --2- 172.21.15.83:0/472599922 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x555f2b730b30 0x555f2b732f50 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:00.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.195+0000 7ff08ff7a740 1 --2- 172.21.15.83:0/472599922 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555f2b737f50 0x555f2b742930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:00.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.195+0000 7ff08ff7a740 1 -- 172.21.15.83:0/472599922 >> 172.21.15.83:0/472599922 conn(0x555f2b734270 msgr2=0x555f2b73a070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:00.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.195+0000 7ff067e00640 0 client.22305 ms_handle_reset on v2:172.21.15.138:6836/900566754 2024-04-24T09:11:00.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.196+0000 7ff08ff7a740 1 -- 172.21.15.83:0/472599922 shutdown_connections 2024-04-24T09:11:00.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.196+0000 7ff08ff7a740 1 -- 172.21.15.83:0/472599922 wait complete. 2024-04-24T09:11:00.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.196+0000 7ff08ff7a740 20 client.22305 trim_cache size 0 max 16384 2024-04-24T09:11:00.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.218+0000 7ff08ff7a740 1 Processor -- start 2024-04-24T09:11:00.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.218+0000 7ff08ff7a740 1 -- start start 2024-04-24T09:11:00.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.219+0000 7ff08ff7a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f2b737f50 0x555f2b73fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:00.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.219+0000 7ff08ff7a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f2b738870 0x555f2b742460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:00.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.219+0000 7ff08ff7a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555f2b742f40 0x555f2b744330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:00.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.219+0000 7ff08ff7a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff088075b60 con 0x555f2b737f50 2024-04-24T09:11:00.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.219+0000 7ff08ff7a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff0881a6610 con 0x555f2b738870 2024-04-24T09:11:00.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.219+0000 7ff08ff7a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff088076230 con 0x555f2b742f40 2024-04-24T09:11:00.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.219+0000 7ff085400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f2b738870 0x555f2b742460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:00.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.219+0000 7ff085e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f2b737f50 0x555f2b73fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:00.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.219+0000 7ff085400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f2b738870 0x555f2b742460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:41848/0 (socket says 172.21.15.83:41848) 2024-04-24T09:11:00.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.219+0000 7ff085400640 1 -- 172.21.15.83:0/2114642280 learned_addr learned my addr 172.21.15.83:0/2114642280 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:11:00.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.220+0000 7ff086800640 1 --2- 172.21.15.83:0/2114642280 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555f2b742f40 0x555f2b744330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:00.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.220+0000 7ff085400640 1 -- 172.21.15.83:0/2114642280 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555f2b742f40 msgr2=0x555f2b744330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:00.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.220+0000 7ff085400640 1 --2- 172.21.15.83:0/2114642280 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555f2b742f40 0x555f2b744330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:00.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.220+0000 7ff085400640 1 -- 172.21.15.83:0/2114642280 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f2b737f50 msgr2=0x555f2b73fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:00.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.220+0000 7ff085400640 1 --2- 172.21.15.83:0/2114642280 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f2b737f50 0x555f2b73fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:00.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.220+0000 7ff085400640 1 -- 172.21.15.83:0/2114642280 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f2b4757d0 con 0x555f2b738870 2024-04-24T09:11:00.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.220+0000 7ff085e00640 1 --2- 172.21.15.83:0/2114642280 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f2b737f50 0x555f2b73fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:11:00.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.221+0000 7ff085400640 1 --2- 172.21.15.83:0/2114642280 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f2b738870 0x555f2b742460 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ff07400bd50 tx=0x7ff07409bd40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=844e48e0cdee45a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:00.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.221+0000 7ff072a00640 1 -- 172.21.15.83:0/2114642280 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0740a8070 con 0x555f2b738870 2024-04-24T09:11:00.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.221+0000 7ff072a00640 1 -- 172.21.15.83:0/2114642280 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff0740a0320 con 0x555f2b738870 2024-04-24T09:11:00.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.222+0000 7ff08ff7a740 1 -- 172.21.15.83:0/2114642280 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f2b738870 msgr2=0x555f2b742460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:00.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.222+0000 7ff08ff7a740 1 --2- 172.21.15.83:0/2114642280 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f2b738870 0x555f2b742460 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ff07400bd50 tx=0x7ff07409bd40 comp rx=0 tx=0).stop 2024-04-24T09:11:00.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.222+0000 7ff08ff7a740 1 -- 172.21.15.83:0/2114642280 shutdown_connections 2024-04-24T09:11:00.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.222+0000 7ff08ff7a740 1 --2- 172.21.15.83:0/2114642280 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555f2b742f40 0x555f2b744330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:00.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.222+0000 7ff08ff7a740 1 --2- 172.21.15.83:0/2114642280 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f2b738870 0x555f2b742460 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:00.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.222+0000 7ff08ff7a740 1 --2- 172.21.15.83:0/2114642280 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f2b737f50 0x555f2b73fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:00.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.222+0000 7ff08ff7a740 1 -- 172.21.15.83:0/2114642280 >> 172.21.15.83:0/2114642280 conn(0x555f2b734270 msgr2=0x555f2b73f4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:00.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.222+0000 7ff08ff7a740 1 -- 172.21.15.83:0/2114642280 shutdown_connections 2024-04-24T09:11:00.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.222+0000 7ff08ff7a740 1 -- 172.21.15.83:0/2114642280 wait complete. 2024-04-24T09:11:00.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.223+0000 7ff08ff7a740 1 -- start start 2024-04-24T09:11:00.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.223+0000 7ff08ff7a740 1 Processor -- start 2024-04-24T09:11:00.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.224+0000 7ff08ff7a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f2b737f50 0x555f2b752810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:00.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.224+0000 7ff08ff7a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555f2b742f40 0x555f2b752d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:00.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.224+0000 7ff08ff7a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f2b753290 0x555f2b7b4470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:00.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.224+0000 7ff08ff7a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff0881a5f40 con 0x555f2b753290 2024-04-24T09:11:00.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.224+0000 7ff08ff7a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff0881a5870 con 0x555f2b737f50 2024-04-24T09:11:00.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.224+0000 7ff08ff7a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff088073b60 con 0x555f2b742f40 2024-04-24T09:11:00.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.224+0000 7ff08ff7a740 10 client.0 _ll_register_callbacks cb 0x7ff074010380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:11:00.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.224+0000 7ff085e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f2b737f50 0x555f2b752810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:00.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.224+0000 7ff085400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555f2b742f40 0x555f2b752d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:00.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.224+0000 7ff085e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f2b737f50 0x555f2b752810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:41854/0 (socket says 172.21.15.83:41854) 2024-04-24T09:11:00.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.224+0000 7ff085e00640 1 -- 172.21.15.83:0/1660021477 learned_addr learned my addr 172.21.15.83:0/1660021477 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:11:00.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.224+0000 7ff086800640 1 --2- 172.21.15.83:0/1660021477 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f2b753290 0x555f2b7b4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:00.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.225+0000 7ff085400640 1 -- 172.21.15.83:0/1660021477 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f2b737f50 msgr2=0x555f2b752810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:00.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.225+0000 7ff085400640 1 --2- 172.21.15.83:0/1660021477 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f2b737f50 0x555f2b752810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:00.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.225+0000 7ff085400640 1 -- 172.21.15.83:0/1660021477 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f2b753290 msgr2=0x555f2b7b4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:00.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.225+0000 7ff085400640 1 --2- 172.21.15.83:0/1660021477 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f2b753290 0x555f2b7b4470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:00.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.225+0000 7ff085400640 1 -- 172.21.15.83:0/1660021477 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f2b7b4b60 con 0x555f2b742f40 2024-04-24T09:11:00.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.225+0000 7ff085e00640 1 --2- 172.21.15.83:0/1660021477 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f2b737f50 0x555f2b752810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:11:00.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.225+0000 7ff085400640 1 --2- 172.21.15.83:0/1660021477 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555f2b742f40 0x555f2b752d50 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x555f2b73a8c0 tx=0x7ff074009280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c98b4fdb0c258587 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:00.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.225+0000 7ff067e00640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-04-24T09:11:00.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.225+0000 7ff067e00640 1 -- 172.21.15.83:0/1660021477 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0740a8070 con 0x555f2b742f40 2024-04-24T09:11:00.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.225+0000 7ff08ff7a740 1 -- 172.21.15.83:0/1660021477 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0740a0320 con 0x555f2b742f40 2024-04-24T09:11:00.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.226+0000 7ff067e00640 1 -- 172.21.15.83:0/1660021477 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(120..120 src has 1..120) ==== 9583+0+0 (secure 0 0 0) 0x7ff074097ed0 con 0x555f2b742f40 2024-04-24T09:11:00.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.226+0000 7ff067e00640 1 -- 172.21.15.83:0/1660021477 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7ff07409bab0 con 0x555f2b742f40 2024-04-24T09:11:00.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.226+0000 7ff08ff7a740 10 client.29323 fetch_fsmap learned FSMap version 61 2024-04-24T09:11:00.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.226+0000 7ff08ff7a740 1 -- 172.21.15.83:0/1660021477 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x555f2b7b4b60 con 0x555f2b742f40 2024-04-24T09:11:00.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.226+0000 7ff067e00640 1 -- 172.21.15.83:0/1660021477 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff0740a0320 con 0x555f2b742f40 2024-04-24T09:11:00.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.227+0000 7ff067e00640 1 -- 172.21.15.83:0/1660021477 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 61) ==== 8362+0+0 (secure 0 0 0) 0x7ff0740a14b0 con 0x555f2b742f40 2024-04-24T09:11:00.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.227+0000 7ff08ff7a740 10 client.29323 fetch_fsmap finished waiting for FSMap version 61 2024-04-24T09:11:00.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.227+0000 7ff08ff7a740 10 client.29323 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-24T09:11:00.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.227+0000 7ff08ff7a740 20 client.29323 populate_metadata read hostname 'smithi083' 2024-04-24T09:11:00.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.227+0000 7ff08ff7a740 1 --2- 172.21.15.83:0/1660021477 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x555f2b7b6090 0x555f2b7b8480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:00.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.227+0000 7ff08ff7a740 4 client.29323 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:11:00.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.227+0000 7ff08ff7a740 1 -- 172.21.15.83:0/1660021477 --> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555f2b6a0a60 con 0x555f2b7b6090 2024-04-24T09:11:00.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.227+0000 7ff085e00640 1 --2- 172.21.15.83:0/1660021477 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x555f2b7b6090 0x555f2b7b8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:00.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.228+0000 7ff085e00640 1 --2- 172.21.15.83:0/1660021477 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x555f2b7b6090 0x555f2b7b8480 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cc2cb743e6b82794 server_cookie=bcf2cb4bdc3976f9 in_seq=0 out_seq=0 2024-04-24T09:11:00.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.228+0000 7ff067e00640 10 client.29323 ms_handle_connect on v2:172.21.15.138:6836/900566754 2024-04-24T09:11:00.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.228+0000 7ff067e00640 1 -- 172.21.15.83:0/1660021477 <== mds.1 v2:172.21.15.138:6836/900566754 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555f2b6a0a60 con 0x555f2b7b6090 2024-04-24T09:11:00.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.228+0000 7ff067e00640 10 client.29323 handle_command_reply: tid=0 2024-04-24T09:11:00.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.228+0000 7ff08ff7a740 1 client.29323 shutdown 2024-04-24T09:11:00.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.229+0000 7ff08ff7a740 1 -- 172.21.15.83:0/1660021477 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555f2b742f40 msgr2=0x555f2b752d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:00.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.229+0000 7ff08ff7a740 1 --2- 172.21.15.83:0/1660021477 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555f2b742f40 0x555f2b752d50 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x555f2b73a8c0 tx=0x7ff074009280 comp rx=0 tx=0).stop 2024-04-24T09:11:00.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.229+0000 7ff08ff7a740 1 -- 172.21.15.83:0/1660021477 shutdown_connections 2024-04-24T09:11:00.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.229+0000 7ff08ff7a740 1 --2- 172.21.15.83:0/1660021477 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555f2b753290 0x555f2b7b4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:00.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.229+0000 7ff08ff7a740 1 --2- 172.21.15.83:0/1660021477 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x555f2b7b6090 0x555f2b7b8480 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:00.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.229+0000 7ff08ff7a740 1 --2- 172.21.15.83:0/1660021477 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555f2b742f40 0x555f2b752d50 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:00.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.229+0000 7ff08ff7a740 1 --2- 172.21.15.83:0/1660021477 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f2b737f50 0x555f2b752810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:00.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.229+0000 7ff08ff7a740 1 -- 172.21.15.83:0/1660021477 >> 172.21.15.83:0/1660021477 conn(0x555f2b734270 msgr2=0x555f2b7396e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:00.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.229+0000 7ff067e00640 0 client.29323 ms_handle_reset on v2:172.21.15.138:6836/900566754 2024-04-24T09:11:00.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.229+0000 7ff08ff7a740 1 -- 172.21.15.83:0/1660021477 shutdown_connections 2024-04-24T09:11:00.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.229+0000 7ff08ff7a740 1 -- 172.21.15.83:0/1660021477 wait complete. 2024-04-24T09:11:00.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.229+0000 7ff08ff7a740 20 client.29323 trim_cache size 0 max 16384 2024-04-24T09:11:00.231 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:11:00.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.230+0000 7ff087200640 1 -- 172.21.15.83:0/3148403063 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff06c0b4dc0 msgr2=0x7ff06c0b7280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:00.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.230+0000 7ff087200640 1 --2- 172.21.15.83:0/3148403063 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff06c0b4dc0 0x7ff06c0b7280 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7ff0780058c0 tx=0x7ff07801e8c0 comp rx=0 tx=0).stop 2024-04-24T09:11:00.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.230+0000 7ff087200640 1 -- 172.21.15.83:0/3148403063 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff0881071f0 msgr2=0x7ff08819ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:00.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.230+0000 7ff087200640 1 --2- 172.21.15.83:0/3148403063 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff0881071f0 0x7ff08819ead0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff07400b450 tx=0x7ff07400b850 comp rx=0 tx=0).stop 2024-04-24T09:11:00.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.231+0000 7ff087200640 1 -- 172.21.15.83:0/3148403063 shutdown_connections 2024-04-24T09:11:00.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.231+0000 7ff087200640 1 --2- 172.21.15.83:0/3148403063 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff06c0b4dc0 0x7ff06c0b7280 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:00.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.231+0000 7ff087200640 1 --2- 172.21.15.83:0/3148403063 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff0881986e0 0x7ff088198b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:00.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.231+0000 7ff087200640 1 --2- 172.21.15.83:0/3148403063 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff0881071f0 0x7ff08819ead0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:00.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.231+0000 7ff087200640 1 --2- 172.21.15.83:0/3148403063 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff088102120 0x7ff08819e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:00.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.231+0000 7ff087200640 1 -- 172.21.15.83:0/3148403063 >> 172.21.15.83:0/3148403063 conn(0x7ff0880fd9a0 msgr2=0x7ff0880fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:00.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.231+0000 7ff087200640 1 -- 172.21.15.83:0/3148403063 shutdown_connections 2024-04-24T09:11:00.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.231+0000 7ff087200640 1 -- 172.21.15.83:0/3148403063 wait complete. 2024-04-24T09:11:00.251 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:11:00 smithi083 ceph-mon[27907]: pgmap v610: 129 pgs: 129 active+clean; 681 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.3 MiB/s wr, 331 op/s 2024-04-24T09:11:00.254 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T09:11:00.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:11:00 smithi138 ceph-mon[31441]: pgmap v610: 129 pgs: 129 active+clean; 681 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.3 MiB/s wr, 331 op/s 2024-04-24T09:11:00.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:11:00 smithi189 ceph-mon[31791]: pgmap v610: 129 pgs: 129 active+clean; 681 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.3 MiB/s wr, 331 op/s 2024-04-24T09:11:00.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.516+0000 7f5adb800640 1 -- 172.21.15.83:0/2150774222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ad4100140 msgr2=0x7f5ad4100520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:00.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.516+0000 7f5adb800640 1 --2- 172.21.15.83:0/2150774222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ad4100140 0x7f5ad4100520 secure :-1 s=READY pgs=2476 cs=0 l=1 rev1=1 crypto rx=0x7f5abc009900 tx=0x7f5abc02a480 comp rx=0 tx=0).stop 2024-04-24T09:11:00.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.516+0000 7f5adb800640 1 -- 172.21.15.83:0/2150774222 shutdown_connections 2024-04-24T09:11:00.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.516+0000 7f5adb800640 1 --2- 172.21.15.83:0/2150774222 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5ad4112e60 0x7f5ad4115250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:00.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.516+0000 7f5adb800640 1 --2- 172.21.15.83:0/2150774222 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ad4100a60 0x7f5ad4112920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:00.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.516+0000 7f5adb800640 1 --2- 172.21.15.83:0/2150774222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ad4100140 0x7f5ad4100520 unknown :-1 s=CLOSED pgs=2476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:00.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.516+0000 7f5adb800640 1 -- 172.21.15.83:0/2150774222 >> 172.21.15.83:0/2150774222 conn(0x7f5ad4077060 msgr2=0x7f5ad40fe450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:00.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.516+0000 7f5adb800640 1 -- 172.21.15.83:0/2150774222 shutdown_connections 2024-04-24T09:11:00.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.516+0000 7f5adb800640 1 -- 172.21.15.83:0/2150774222 wait complete. 2024-04-24T09:11:00.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.517+0000 7f5adb800640 1 Processor -- start 2024-04-24T09:11:00.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.517+0000 7f5adb800640 1 -- start start 2024-04-24T09:11:00.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.518+0000 7f5adb800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5ad4100140 0x7f5ad419ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:00.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.518+0000 7f5adb800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ad4100a60 0x7f5ad419f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:00.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.518+0000 7f5adb800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ad4112e60 0x7f5ad41a36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:00.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.518+0000 7f5adb800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5ad4118940 con 0x7f5ad4112e60 2024-04-24T09:11:00.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.518+0000 7f5adb800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5ad41187c0 con 0x7f5ad4100a60 2024-04-24T09:11:00.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.518+0000 7f5adb800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5ad4118ac0 con 0x7f5ad4100140 2024-04-24T09:11:00.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.518+0000 7f5acbe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ad4100a60 0x7f5ad419f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:00.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.518+0000 7f5ad8e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5ad4100140 0x7f5ad419ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:00.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.518+0000 7f5acbe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ad4100a60 0x7f5ad419f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:41872/0 (socket says 172.21.15.83:41872) 2024-04-24T09:11:00.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.518+0000 7f5acbe00640 1 -- 172.21.15.83:0/3541205029 learned_addr learned my addr 172.21.15.83:0/3541205029 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:11:00.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.518+0000 7f5ad9800640 1 --2- 172.21.15.83:0/3541205029 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ad4112e60 0x7f5ad41a36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:00.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.519+0000 7f5ad9800640 1 -- 172.21.15.83:0/3541205029 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5ad4100140 msgr2=0x7f5ad419ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:00.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.519+0000 7f5ad9800640 1 --2- 172.21.15.83:0/3541205029 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5ad4100140 0x7f5ad419ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:00.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.519+0000 7f5ad9800640 1 -- 172.21.15.83:0/3541205029 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ad4100a60 msgr2=0x7f5ad419f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:00.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.519+0000 7f5ad9800640 1 --2- 172.21.15.83:0/3541205029 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ad4100a60 0x7f5ad419f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:00.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.519+0000 7f5ad9800640 1 -- 172.21.15.83:0/3541205029 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ad41a3de0 con 0x7f5ad4112e60 2024-04-24T09:11:00.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.519+0000 7f5acbe00640 1 --2- 172.21.15.83:0/3541205029 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ad4100a60 0x7f5ad419f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:11:00.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.519+0000 7f5ad9800640 1 --2- 172.21.15.83:0/3541205029 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ad4112e60 0x7f5ad41a36b0 secure :-1 s=READY pgs=2477 cs=0 l=1 rev1=1 crypto rx=0x7f5acc00c9d0 tx=0x7f5acc00ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:00.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.520+0000 7f5ad2000640 1 -- 172.21.15.83:0/3541205029 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5acc00ad60 con 0x7f5ad4112e60 2024-04-24T09:11:00.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.520+0000 7f5adb800640 1 -- 172.21.15.83:0/3541205029 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ad41a40d0 con 0x7f5ad4112e60 2024-04-24T09:11:00.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.520+0000 7f5ad2000640 1 -- 172.21.15.83:0/3541205029 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5acc00f900 con 0x7f5ad4112e60 2024-04-24T09:11:00.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.520+0000 7f5ad2000640 1 -- 172.21.15.83:0/3541205029 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5acc018960 con 0x7f5ad4112e60 2024-04-24T09:11:00.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.520+0000 7f5adb800640 1 -- 172.21.15.83:0/3541205029 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ad41ab960 con 0x7f5ad4112e60 2024-04-24T09:11:00.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.525+0000 7f5ad2000640 1 -- 172.21.15.83:0/3541205029 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5acc00fab0 con 0x7f5ad4112e60 2024-04-24T09:11:00.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.525+0000 7f5ade684740 1 Processor -- start 2024-04-24T09:11:00.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.525+0000 7f5ade684740 1 -- start start 2024-04-24T09:11:00.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.525+0000 7f5ade684740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fb87e78950 0x55fb87e78d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:00.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.526+0000 7f5ade684740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fb87e79270 0x55fb87e83400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:00.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.526+0000 7f5ade684740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb87e83940 0x55fb87e85d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:00.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.526+0000 7f5ade684740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55fb87dde1d0 con 0x55fb87e78950 2024-04-24T09:11:00.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.526+0000 7f5ade684740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55fb87d84450 con 0x55fb87e83940 2024-04-24T09:11:00.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.526+0000 7f5ade684740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55fb87dda190 con 0x55fb87e79270 2024-04-24T09:11:00.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.526+0000 7f5acbe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fb87e78950 0x55fb87e78d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:00.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.526+0000 7f5acbe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fb87e78950 0x55fb87e78d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58524/0 (socket says 172.21.15.83:58524) 2024-04-24T09:11:00.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.526+0000 7f5acbe00640 1 -- 172.21.15.83:0/1550288941 learned_addr learned my addr 172.21.15.83:0/1550288941 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:11:00.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.526+0000 7f5ad8e00640 1 --2- 172.21.15.83:0/1550288941 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fb87e79270 0x55fb87e83400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:00.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.527+0000 7f5acbe00640 1 --2- 172.21.15.83:0/1550288941 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb87e83940 0x55fb87e85d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:00.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.527+0000 7f5acbe00640 1 -- 172.21.15.83:0/1550288941 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fb87e79270 msgr2=0x55fb87e83400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:00.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.527+0000 7f5acbe00640 1 --2- 172.21.15.83:0/1550288941 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fb87e79270 0x55fb87e83400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:00.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.527+0000 7f5acbe00640 1 -- 172.21.15.83:0/1550288941 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb87e83940 msgr2=0x55fb87e85d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:00.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.527+0000 7f5acbe00640 1 --2- 172.21.15.83:0/1550288941 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb87e83940 0x55fb87e85d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:00.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.527+0000 7f5acbe00640 1 -- 172.21.15.83:0/1550288941 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb87e41310 con 0x55fb87e78950 2024-04-24T09:11:00.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.527+0000 7f5acbe00640 1 --2- 172.21.15.83:0/1550288941 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fb87e78950 0x55fb87e78d30 secure :-1 s=READY pgs=2478 cs=0 l=1 rev1=1 crypto rx=0x7f5ac4011af0 tx=0x7f5ac4009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d8852994baeda01 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:00.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.527+0000 7f5ad2000640 1 --2- 172.21.15.83:0/3541205029 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f5ab40a3da0 0x7f5ab40a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:00.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.528+0000 7f5aca000640 1 -- 172.21.15.83:0/1550288941 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac401a070 con 0x55fb87e78950 2024-04-24T09:11:00.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.528+0000 7f5aca000640 1 -- 172.21.15.83:0/1550288941 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5ac400fc40 con 0x55fb87e78950 2024-04-24T09:11:00.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.528+0000 7f5aca000640 1 -- 172.21.15.83:0/1550288941 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac40047a0 con 0x55fb87e78950 2024-04-24T09:11:00.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.528+0000 7f5ad8e00640 1 --2- 172.21.15.83:0/3541205029 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f5ab40a3da0 0x7f5ab40a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:00.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.528+0000 7f5ad2000640 1 -- 172.21.15.83:0/3541205029 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 9583+0+0 (secure 0 0 0) 0x7f5acc0d0a60 con 0x7f5ad4112e60 2024-04-24T09:11:00.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.529+0000 7f5ade684740 1 -- 172.21.15.83:0/1550288941 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fb87e78950 msgr2=0x55fb87e78d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:00.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.529+0000 7f5ade684740 1 --2- 172.21.15.83:0/1550288941 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fb87e78950 0x55fb87e78d30 secure :-1 s=READY pgs=2478 cs=0 l=1 rev1=1 crypto rx=0x7f5ac4011af0 tx=0x7f5ac4009da0 comp rx=0 tx=0).stop 2024-04-24T09:11:00.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.529+0000 7f5ade684740 1 -- 172.21.15.83:0/1550288941 shutdown_connections 2024-04-24T09:11:00.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.529+0000 7f5ade684740 1 --2- 172.21.15.83:0/1550288941 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb87e83940 0x55fb87e85d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:00.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.529+0000 7f5ade684740 1 --2- 172.21.15.83:0/1550288941 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fb87e79270 0x55fb87e83400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:00.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.529+0000 7f5ade684740 1 --2- 172.21.15.83:0/1550288941 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fb87e78950 0x55fb87e78d30 unknown :-1 s=CLOSED pgs=2478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:00.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.529+0000 7f5ade684740 1 -- 172.21.15.83:0/1550288941 >> 172.21.15.83:0/1550288941 conn(0x55fb87e74c70 msgr2=0x55fb87e75050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:00.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.529+0000 7f5ad8e00640 1 --2- 172.21.15.83:0/3541205029 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f5ab40a3da0 0x7f5ab40a6260 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f5abc005ab0 tx=0x7f5abc005a20 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:00.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.529+0000 7f5ade684740 1 -- 172.21.15.83:0/1550288941 shutdown_connections 2024-04-24T09:11:00.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.530+0000 7f5ade684740 1 -- 172.21.15.83:0/1550288941 wait complete. 2024-04-24T09:11:00.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.530+0000 7f5ade684740 1 -- start start 2024-04-24T09:11:00.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.531+0000 7f5ade684740 1 Processor -- start 2024-04-24T09:11:00.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.531+0000 7f5ade684740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb87e78950 0x55fb87e83390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:00.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.531+0000 7f5ade684740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fb87e79270 0x55fb87e80680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:00.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.532+0000 7f5ade684740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fb87e83940 0x55fb87e80dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:00.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.532+0000 7f5ade684740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55fb87e867d0 con 0x55fb87e79270 2024-04-24T09:11:00.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.532+0000 7f5ade684740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55fb87d97810 con 0x55fb87e78950 2024-04-24T09:11:00.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.532+0000 7f5ade684740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55fb87e86950 con 0x55fb87e83940 2024-04-24T09:11:00.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.532+0000 7f5acbe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fb87e79270 0x55fb87e80680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:00.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.532+0000 7f5acbe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fb87e79270 0x55fb87e80680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58540/0 (socket says 172.21.15.83:58540) 2024-04-24T09:11:00.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.532+0000 7f5acbe00640 1 -- 172.21.15.83:0/1620147916 learned_addr learned my addr 172.21.15.83:0/1620147916 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:11:00.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.532+0000 7f5ade684740 10 client.0 _ll_register_callbacks cb 0x55fb87d65140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:11:00.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.532+0000 7f5ad8e00640 1 --2- 172.21.15.83:0/1620147916 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb87e78950 0x55fb87e83390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:00.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.532+0000 7f5acbe00640 1 -- 172.21.15.83:0/1620147916 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fb87e83940 msgr2=0x55fb87e80dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:00.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.532+0000 7f5acbe00640 1 --2- 172.21.15.83:0/1620147916 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fb87e83940 0x55fb87e80dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:00.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.532+0000 7f5acbe00640 1 -- 172.21.15.83:0/1620147916 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb87e78950 msgr2=0x55fb87e83390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:00.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.532+0000 7f5acbe00640 1 --2- 172.21.15.83:0/1620147916 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb87e78950 0x55fb87e83390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:00.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.532+0000 7f5acbe00640 1 -- 172.21.15.83:0/1620147916 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb87da83f0 con 0x55fb87e79270 2024-04-24T09:11:00.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.533+0000 7f5acbe00640 1 --2- 172.21.15.83:0/1620147916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fb87e79270 0x55fb87e80680 secure :-1 s=READY pgs=2479 cs=0 l=1 rev1=1 crypto rx=0x7f5ac40117e0 tx=0x7f5ac4005da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27067e5f9a8c116f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:00.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.533+0000 7f5ac2a00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:11:00.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.533+0000 7f5ac2a00640 1 -- 172.21.15.83:0/1620147916 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac401a040 con 0x55fb87e79270 2024-04-24T09:11:00.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.533+0000 7f5ac2a00640 1 -- 172.21.15.83:0/1620147916 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(120..120 src has 1..120) ==== 9583+0+0 (secure 0 0 0) 0x7f5ac40046c0 con 0x55fb87e79270 2024-04-24T09:11:00.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.534+0000 7f5ade684740 1 -- 172.21.15.83:0/1620147916 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ac400fc40 con 0x55fb87e79270 2024-04-24T09:11:00.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.534+0000 7f5ac2a00640 1 -- 172.21.15.83:0/1620147916 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f5ac40048c0 con 0x55fb87e79270 2024-04-24T09:11:00.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.534+0000 7f5ade684740 10 client.22323 fetch_fsmap learned FSMap version 61 2024-04-24T09:11:00.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.534+0000 7f5ade684740 1 -- 172.21.15.83:0/1620147916 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x55fb87af03f0 con 0x55fb87e79270 2024-04-24T09:11:00.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.535+0000 7f5ac2a00640 1 -- 172.21.15.83:0/1620147916 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5ac400fc40 con 0x55fb87e79270 2024-04-24T09:11:00.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.535+0000 7f5ac2a00640 1 -- 172.21.15.83:0/1620147916 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 61) ==== 8362+0+0 (secure 0 0 0) 0x7f5ac4032be0 con 0x55fb87e79270 2024-04-24T09:11:00.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.535+0000 7f5ade684740 10 client.22323 fetch_fsmap finished waiting for FSMap version 61 2024-04-24T09:11:00.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.535+0000 7f5ade684740 10 client.22323 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:11:00.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.535+0000 7f5ade684740 20 client.22323 populate_metadata read hostname 'smithi083' 2024-04-24T09:11:00.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.535+0000 7f5ade684740 1 --2- 172.21.15.83:0/1620147916 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55fb87e939e0 0x55fb87e95dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:00.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.535+0000 7f5ade684740 4 client.22323 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:11:00.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.535+0000 7f5ade684740 1 -- 172.21.15.83:0/1620147916 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fb87debbf0 con 0x55fb87e939e0 2024-04-24T09:11:00.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.536+0000 7f5ad8e00640 1 --2- 172.21.15.83:0/1620147916 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55fb87e939e0 0x55fb87e95dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:00.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.536+0000 7f5ad8e00640 1 --2- 172.21.15.83:0/1620147916 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55fb87e939e0 0x55fb87e95dd0 crc :-1 s=READY pgs=483 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc854c9eb4c05187 server_cookie=779b393e812b4a7f in_seq=0 out_seq=0 2024-04-24T09:11:00.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.536+0000 7f5ac2a00640 10 client.22323 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:11:00.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.538+0000 7f5ac2a00640 1 -- 172.21.15.83:0/1620147916 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fb87debbf0 con 0x55fb87e939e0 2024-04-24T09:11:00.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.538+0000 7f5ac2a00640 10 client.22323 handle_command_reply: tid=0 2024-04-24T09:11:00.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.538+0000 7f5ade684740 1 client.22323 shutdown 2024-04-24T09:11:00.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.538+0000 7f5ade684740 1 -- 172.21.15.83:0/1620147916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fb87e79270 msgr2=0x55fb87e80680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:00.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.538+0000 7f5ade684740 1 --2- 172.21.15.83:0/1620147916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fb87e79270 0x55fb87e80680 secure :-1 s=READY pgs=2479 cs=0 l=1 rev1=1 crypto rx=0x7f5ac40117e0 tx=0x7f5ac4005da0 comp rx=0 tx=0).stop 2024-04-24T09:11:00.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.538+0000 7f5ade684740 1 -- 172.21.15.83:0/1620147916 shutdown_connections 2024-04-24T09:11:00.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.538+0000 7f5ade684740 1 --2- 172.21.15.83:0/1620147916 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55fb87e939e0 0x55fb87e95dd0 crc :-1 s=READY pgs=483 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:00.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.538+0000 7f5ade684740 1 --2- 172.21.15.83:0/1620147916 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fb87e83940 0x55fb87e80dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:00.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.538+0000 7f5ade684740 1 --2- 172.21.15.83:0/1620147916 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fb87e79270 0x55fb87e80680 unknown :-1 s=CLOSED pgs=2479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:00.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.538+0000 7f5ade684740 1 --2- 172.21.15.83:0/1620147916 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb87e78950 0x55fb87e83390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:00.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.538+0000 7f5ade684740 1 -- 172.21.15.83:0/1620147916 >> 172.21.15.83:0/1620147916 conn(0x55fb87e74c70 msgr2=0x55fb87e7aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:00.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.538+0000 7f5ac2a00640 0 client.22323 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:11:00.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.538+0000 7f5ade684740 1 -- 172.21.15.83:0/1620147916 shutdown_connections 2024-04-24T09:11:00.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.538+0000 7f5ade684740 1 -- 172.21.15.83:0/1620147916 wait complete. 2024-04-24T09:11:00.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.539+0000 7f5ade684740 20 client.22323 trim_cache size 0 max 16384 2024-04-24T09:11:00.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.561+0000 7f5ade684740 1 Processor -- start 2024-04-24T09:11:00.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.561+0000 7f5ade684740 1 -- start start 2024-04-24T09:11:00.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.561+0000 7f5ade684740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fb87e78950 0x55fb87e8f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:00.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.561+0000 7f5ade684740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb87e79270 0x55fb87e82ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:00.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.561+0000 7f5ade684740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fb87e83940 0x55fb87e83400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:00.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.561+0000 7f5ade684740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5ad41b5b40 con 0x55fb87e78950 2024-04-24T09:11:00.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.561+0000 7f5ade684740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5ad41b7680 con 0x55fb87e79270 2024-04-24T09:11:00.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.561+0000 7f5ade684740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5ad41b6210 con 0x55fb87e83940 2024-04-24T09:11:00.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.562+0000 7f5ad8e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fb87e78950 0x55fb87e8f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:00.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.562+0000 7f5ad8e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fb87e78950 0x55fb87e8f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58550/0 (socket says 172.21.15.83:58550) 2024-04-24T09:11:00.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.562+0000 7f5ad8e00640 1 -- 172.21.15.83:0/2828636068 learned_addr learned my addr 172.21.15.83:0/2828636068 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:11:00.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.562+0000 7f5acbe00640 1 --2- 172.21.15.83:0/2828636068 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb87e79270 0x55fb87e82ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:00.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.562+0000 7f5ad8e00640 1 -- 172.21.15.83:0/2828636068 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fb87e83940 msgr2=0x55fb87e83400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:00.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.562+0000 7f5ad8e00640 1 --2- 172.21.15.83:0/2828636068 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fb87e83940 0x55fb87e83400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:00.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.562+0000 7f5ad8e00640 1 -- 172.21.15.83:0/2828636068 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb87e79270 msgr2=0x55fb87e82ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:00.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.562+0000 7f5ad8e00640 1 --2- 172.21.15.83:0/2828636068 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb87e79270 0x55fb87e82ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:00.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.562+0000 7f5ad8e00640 1 -- 172.21.15.83:0/2828636068 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb87bba7d0 con 0x55fb87e78950 2024-04-24T09:11:00.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.563+0000 7f5ad8e00640 1 --2- 172.21.15.83:0/2828636068 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fb87e78950 0x55fb87e8f690 secure :-1 s=READY pgs=2480 cs=0 l=1 rev1=1 crypto rx=0x7f5abc004250 tx=0x7f5abc040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=947fee9064114b23 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:00.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.563+0000 7f5ac9600640 1 -- 172.21.15.83:0/2828636068 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5abc057070 con 0x55fb87e78950 2024-04-24T09:11:00.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.563+0000 7f5ac9600640 1 -- 172.21.15.83:0/2828636068 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5abc0512d0 con 0x55fb87e78950 2024-04-24T09:11:00.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.564+0000 7f5ac9600640 1 -- 172.21.15.83:0/2828636068 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5abc035040 con 0x55fb87e78950 2024-04-24T09:11:00.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.564+0000 7f5ade684740 1 -- 172.21.15.83:0/2828636068 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fb87e78950 msgr2=0x55fb87e8f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:00.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.564+0000 7f5ade684740 1 --2- 172.21.15.83:0/2828636068 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fb87e78950 0x55fb87e8f690 secure :-1 s=READY pgs=2480 cs=0 l=1 rev1=1 crypto rx=0x7f5abc004250 tx=0x7f5abc040000 comp rx=0 tx=0).stop 2024-04-24T09:11:00.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.565+0000 7f5ade684740 1 -- 172.21.15.83:0/2828636068 shutdown_connections 2024-04-24T09:11:00.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.565+0000 7f5ade684740 1 --2- 172.21.15.83:0/2828636068 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fb87e83940 0x55fb87e83400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:00.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.565+0000 7f5ade684740 1 --2- 172.21.15.83:0/2828636068 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb87e79270 0x55fb87e82ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:00.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.565+0000 7f5ade684740 1 --2- 172.21.15.83:0/2828636068 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fb87e78950 0x55fb87e8f690 unknown :-1 s=CLOSED pgs=2480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:00.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.565+0000 7f5ade684740 1 -- 172.21.15.83:0/2828636068 >> 172.21.15.83:0/2828636068 conn(0x55fb87e74c70 msgr2=0x55fb87e7a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:00.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.565+0000 7f5ade684740 1 -- 172.21.15.83:0/2828636068 shutdown_connections 2024-04-24T09:11:00.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.565+0000 7f5ade684740 1 -- 172.21.15.83:0/2828636068 wait complete. 2024-04-24T09:11:00.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.565+0000 7f5ade684740 1 -- start start 2024-04-24T09:11:00.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.566+0000 7f5ade684740 1 Processor -- start 2024-04-24T09:11:00.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.566+0000 7f5ade684740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb87e78950 0x55fb87e8c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:00.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.566+0000 7f5ade684740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fb87e79270 0x55fb87e7de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:00.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.566+0000 7f5ade684740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fb87e83940 0x55fb87e849d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:00.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.566+0000 7f5ade684740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5ad41b6fb0 con 0x55fb87e79270 2024-04-24T09:11:00.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.566+0000 7f5ade684740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5ad41b68e0 con 0x55fb87e78950 2024-04-24T09:11:00.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.566+0000 7f5ade684740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5ad41b3b40 con 0x55fb87e83940 2024-04-24T09:11:00.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.566+0000 7f5ade684740 10 client.0 _ll_register_callbacks cb 0x7f5ac4011950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:11:00.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.567+0000 7f5ad8e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb87e78950 0x55fb87e8c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:00.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.567+0000 7f5ad8e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb87e78950 0x55fb87e8c0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:41916/0 (socket says 172.21.15.83:41916) 2024-04-24T09:11:00.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.567+0000 7f5ad8e00640 1 -- 172.21.15.83:0/2576018495 learned_addr learned my addr 172.21.15.83:0/2576018495 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:11:00.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.567+0000 7f5acbe00640 1 --2- 172.21.15.83:0/2576018495 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fb87e79270 0x55fb87e7de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:00.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.567+0000 7f5ad9800640 1 --2- 172.21.15.83:0/2576018495 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fb87e83940 0x55fb87e849d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:00.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.567+0000 7f5ad8e00640 1 -- 172.21.15.83:0/2576018495 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fb87e83940 msgr2=0x55fb87e849d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:00.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.567+0000 7f5ad8e00640 1 --2- 172.21.15.83:0/2576018495 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fb87e83940 0x55fb87e849d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:00.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.567+0000 7f5ad8e00640 1 -- 172.21.15.83:0/2576018495 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fb87e79270 msgr2=0x55fb87e7de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:00.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.567+0000 7f5ad8e00640 1 --2- 172.21.15.83:0/2576018495 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fb87e79270 0x55fb87e7de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:00.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.567+0000 7f5ad8e00640 1 -- 172.21.15.83:0/2576018495 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb87e84f10 con 0x55fb87e78950 2024-04-24T09:11:00.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.568+0000 7f5ad8e00640 1 --2- 172.21.15.83:0/2576018495 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb87e78950 0x55fb87e8c0c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f5abc02a9a0 tx=0x7f5abc0044d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d185b9cdfee3f11 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:00.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.568+0000 7f5ac2a00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:11:00.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.568+0000 7f5ac2a00640 1 -- 172.21.15.83:0/2576018495 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5abc060060 con 0x55fb87e78950 2024-04-24T09:11:00.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.568+0000 7f5ac2a00640 1 -- 172.21.15.83:0/2576018495 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(120..120 src has 1..120) ==== 9583+0+0 (secure 0 0 0) 0x7f5abc050050 con 0x55fb87e78950 2024-04-24T09:11:00.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.568+0000 7f5ade684740 1 -- 172.21.15.83:0/2576018495 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5abc0512d0 con 0x55fb87e78950 2024-04-24T09:11:00.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.569+0000 7f5ac2a00640 1 -- 172.21.15.83:0/2576018495 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f5abc067050 con 0x55fb87e78950 2024-04-24T09:11:00.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.569+0000 7f5ade684740 10 client.39407 fetch_fsmap learned FSMap version 61 2024-04-24T09:11:00.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.569+0000 7f5ade684740 1 -- 172.21.15.83:0/2576018495 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x55fb87f0b3b0 con 0x55fb87e78950 2024-04-24T09:11:00.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.569+0000 7f5ac2a00640 1 -- 172.21.15.83:0/2576018495 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5abc0512d0 con 0x55fb87e78950 2024-04-24T09:11:00.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.569+0000 7f5ac2a00640 1 -- 172.21.15.83:0/2576018495 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 61) ==== 8362+0+0 (secure 0 0 0) 0x7f5abc004b20 con 0x55fb87e78950 2024-04-24T09:11:00.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.569+0000 7f5ade684740 10 client.39407 fetch_fsmap finished waiting for FSMap version 61 2024-04-24T09:11:00.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.569+0000 7f5ade684740 10 client.39407 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:11:00.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.569+0000 7f5ade684740 20 client.39407 populate_metadata read hostname 'smithi083' 2024-04-24T09:11:00.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.569+0000 7f5ade684740 1 --2- 172.21.15.83:0/2576018495 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55fb87f0ba30 0x55fb87f0de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:00.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.569+0000 7f5ade684740 4 client.39407 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:11:00.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.570+0000 7f5ade684740 1 -- 172.21.15.83:0/2576018495 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fb87de5a60 con 0x55fb87f0ba30 2024-04-24T09:11:00.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.570+0000 7f5acbe00640 1 --2- 172.21.15.83:0/2576018495 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55fb87f0ba30 0x55fb87f0de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:00.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.570+0000 7f5ac2a00640 10 client.39407 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:11:00.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.570+0000 7f5acbe00640 1 --2- 172.21.15.83:0/2576018495 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55fb87f0ba30 0x55fb87f0de50 crc :-1 s=READY pgs=484 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5e2548a24b279c5 server_cookie=14e24078d064e50d in_seq=0 out_seq=0 2024-04-24T09:11:00.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.571+0000 7f5ac2a00640 1 -- 172.21.15.83:0/2576018495 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fb87de5a60 con 0x55fb87f0ba30 2024-04-24T09:11:00.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.571+0000 7f5ac2a00640 10 client.39407 handle_command_reply: tid=0 2024-04-24T09:11:00.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.571+0000 7f5ade684740 1 client.39407 shutdown 2024-04-24T09:11:00.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.571+0000 7f5ade684740 1 -- 172.21.15.83:0/2576018495 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb87e78950 msgr2=0x55fb87e8c0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:00.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.571+0000 7f5ade684740 1 --2- 172.21.15.83:0/2576018495 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb87e78950 0x55fb87e8c0c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f5abc02a9a0 tx=0x7f5abc0044d0 comp rx=0 tx=0).stop 2024-04-24T09:11:00.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.572+0000 7f5ade684740 1 -- 172.21.15.83:0/2576018495 shutdown_connections 2024-04-24T09:11:00.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.572+0000 7f5ade684740 1 --2- 172.21.15.83:0/2576018495 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55fb87f0ba30 0x55fb87f0de50 crc :-1 s=READY pgs=484 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:00.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.572+0000 7f5ade684740 1 --2- 172.21.15.83:0/2576018495 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fb87e83940 0x55fb87e849d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:00.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.572+0000 7f5ade684740 1 --2- 172.21.15.83:0/2576018495 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fb87e79270 0x55fb87e7de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:00.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.572+0000 7f5ac2a00640 0 client.39407 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:11:00.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.572+0000 7f5ade684740 1 --2- 172.21.15.83:0/2576018495 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb87e78950 0x55fb87e8c0c0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:00.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.572+0000 7f5ade684740 1 -- 172.21.15.83:0/2576018495 >> 172.21.15.83:0/2576018495 conn(0x55fb87e74c70 msgr2=0x55fb87e7a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:00.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.572+0000 7f5ade684740 1 -- 172.21.15.83:0/2576018495 shutdown_connections 2024-04-24T09:11:00.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.572+0000 7f5ade684740 1 -- 172.21.15.83:0/2576018495 wait complete. 2024-04-24T09:11:00.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.572+0000 7f5ade684740 20 client.39407 trim_cache size 0 max 16384 2024-04-24T09:11:00.573 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:11:00.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.573+0000 7f5adb800640 1 -- 172.21.15.83:0/3541205029 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f5ab40a3da0 msgr2=0x7f5ab40a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:00.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.573+0000 7f5adb800640 1 --2- 172.21.15.83:0/3541205029 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f5ab40a3da0 0x7f5ab40a6260 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f5abc005ab0 tx=0x7f5abc005a20 comp rx=0 tx=0).stop 2024-04-24T09:11:00.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.573+0000 7f5adb800640 1 -- 172.21.15.83:0/3541205029 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ad4112e60 msgr2=0x7f5ad41a36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:00.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.573+0000 7f5adb800640 1 --2- 172.21.15.83:0/3541205029 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ad4112e60 0x7f5ad41a36b0 secure :-1 s=READY pgs=2477 cs=0 l=1 rev1=1 crypto rx=0x7f5acc00c9d0 tx=0x7f5acc00ce90 comp rx=0 tx=0).stop 2024-04-24T09:11:00.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.573+0000 7f5adb800640 1 -- 172.21.15.83:0/3541205029 shutdown_connections 2024-04-24T09:11:00.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.573+0000 7f5adb800640 1 --2- 172.21.15.83:0/3541205029 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f5ab40a3da0 0x7f5ab40a6260 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:00.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.573+0000 7f5adb800640 1 --2- 172.21.15.83:0/3541205029 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ad4112e60 0x7f5ad41a36b0 unknown :-1 s=CLOSED pgs=2477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:00.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.573+0000 7f5adb800640 1 --2- 172.21.15.83:0/3541205029 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ad4100a60 0x7f5ad419f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:00.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.573+0000 7f5adb800640 1 --2- 172.21.15.83:0/3541205029 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5ad4100140 0x7f5ad419ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:00.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.573+0000 7f5adb800640 1 -- 172.21.15.83:0/3541205029 >> 172.21.15.83:0/3541205029 conn(0x7f5ad4077060 msgr2=0x7f5ad4111820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:00.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.573+0000 7f5adb800640 1 -- 172.21.15.83:0/3541205029 shutdown_connections 2024-04-24T09:11:00.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:00.573+0000 7f5adb800640 1 -- 172.21.15.83:0/3541205029 wait complete. 2024-04-24T09:11:01.178 INFO:tasks.workunit.client.0.smithi083.stdout: 10 41944 6.71 MB/sec execute 212 sec latency 113.864 ms 2024-04-24T09:11:01.595 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T09:11:01.595 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T09:11:01.595 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T09:11:01.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.853+0000 7f43ca400640 1 -- 172.21.15.83:0/439679215 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43c4108e40 msgr2=0x7f43c41092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:01.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.853+0000 7f43ca400640 1 --2- 172.21.15.83:0/439679215 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43c4108e40 0x7f43c41092c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f43b0009890 tx=0x7f43b002a380 comp rx=0 tx=0).stop 2024-04-24T09:11:01.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.854+0000 7f43ca400640 1 -- 172.21.15.83:0/439679215 shutdown_connections 2024-04-24T09:11:01.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.854+0000 7f43ca400640 1 --2- 172.21.15.83:0/439679215 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f43c4109800 0x7f43c4115360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:01.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.854+0000 7f43ca400640 1 --2- 172.21.15.83:0/439679215 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43c4108e40 0x7f43c41092c0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:01.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.854+0000 7f43ca400640 1 --2- 172.21.15.83:0/439679215 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43c410e5a0 0x7f43c410e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:01.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.854+0000 7f43ca400640 1 -- 172.21.15.83:0/439679215 >> 172.21.15.83:0/439679215 conn(0x7f43c4077a20 msgr2=0x7f43c4079e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:01.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.854+0000 7f43ca400640 1 -- 172.21.15.83:0/439679215 shutdown_connections 2024-04-24T09:11:01.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.854+0000 7f43ca400640 1 -- 172.21.15.83:0/439679215 wait complete. 2024-04-24T09:11:01.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.855+0000 7f43ca400640 1 Processor -- start 2024-04-24T09:11:01.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.856+0000 7f43ca400640 1 -- start start 2024-04-24T09:11:01.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.856+0000 7f43ca400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f43c4108e40 0x7f43c4073850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:01.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.856+0000 7f43ca400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43c4109800 0x7f43c406c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:01.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.856+0000 7f43ca400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43c410e5a0 0x7f43c406cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:01.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.856+0000 7f43ca400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f43c4118730 con 0x7f43c410e5a0 2024-04-24T09:11:01.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.856+0000 7f43ca400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f43c41185b0 con 0x7f43c4109800 2024-04-24T09:11:01.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.856+0000 7f43ca400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f43c41188b0 con 0x7f43c4108e40 2024-04-24T09:11:01.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.856+0000 7f43c3e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43c4109800 0x7f43c406c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:01.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.856+0000 7f43c9a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43c410e5a0 0x7f43c406cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:01.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.856+0000 7f43c9a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43c410e5a0 0x7f43c406cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58582/0 (socket says 172.21.15.83:58582) 2024-04-24T09:11:01.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.856+0000 7f43c9a00640 1 -- 172.21.15.83:0/1711817431 learned_addr learned my addr 172.21.15.83:0/1711817431 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:11:01.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.857+0000 7f43c9000640 1 --2- 172.21.15.83:0/1711817431 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f43c4108e40 0x7f43c4073850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:01.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.857+0000 7f43c3e00640 1 -- 172.21.15.83:0/1711817431 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f43c4108e40 msgr2=0x7f43c4073850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:01.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.857+0000 7f43c3e00640 1 --2- 172.21.15.83:0/1711817431 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f43c4108e40 0x7f43c4073850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:01.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.857+0000 7f43c3e00640 1 -- 172.21.15.83:0/1711817431 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43c410e5a0 msgr2=0x7f43c406cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:01.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.857+0000 7f43c3e00640 1 --2- 172.21.15.83:0/1711817431 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43c410e5a0 0x7f43c406cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:01.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.857+0000 7f43c3e00640 1 -- 172.21.15.83:0/1711817431 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43c406d540 con 0x7f43c4109800 2024-04-24T09:11:01.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.857+0000 7f43c9a00640 1 --2- 172.21.15.83:0/1711817431 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43c410e5a0 0x7f43c406cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:11:01.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.858+0000 7f43c9000640 1 --2- 172.21.15.83:0/1711817431 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f43c4108e40 0x7f43c4073850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:11:01.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.858+0000 7f43c3e00640 1 --2- 172.21.15.83:0/1711817431 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43c4109800 0x7f43c406c9c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f43b0009780 tx=0x7f43b00042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:01.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.858+0000 7f43c1600640 1 -- 172.21.15.83:0/1711817431 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b0037050 con 0x7f43c4109800 2024-04-24T09:11:01.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.858+0000 7f43c1600640 1 -- 172.21.15.83:0/1711817431 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f43b0038040 con 0x7f43c4109800 2024-04-24T09:11:01.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.858+0000 7f43c1600640 1 -- 172.21.15.83:0/1711817431 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b003eac0 con 0x7f43c4109800 2024-04-24T09:11:01.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.858+0000 7f43ca400640 1 -- 172.21.15.83:0/1711817431 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43c406d7d0 con 0x7f43c4109800 2024-04-24T09:11:01.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.858+0000 7f43ca400640 1 -- 172.21.15.83:0/1711817431 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43c4071610 con 0x7f43c4109800 2024-04-24T09:11:01.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.863+0000 7f43ce7f2740 1 Processor -- start 2024-04-24T09:11:01.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.864+0000 7f43ce7f2740 1 -- start start 2024-04-24T09:11:01.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.866+0000 7f43c1600640 1 -- 172.21.15.83:0/1711817431 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f43b0045030 con 0x7f43c4109800 2024-04-24T09:11:01.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.866+0000 7f43ce7f2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ee2f0c9270 0x55ee2f0c9650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:01.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.866+0000 7f43ce7f2740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee2f0c9b90 0x55ee2f0d3d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:01.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.866+0000 7f43ce7f2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee2f0d4260 0x55ee2f0d6650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:01.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.866+0000 7f43ce7f2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ee2f02ee70 con 0x55ee2f0d4260 2024-04-24T09:11:01.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.866+0000 7f43ce7f2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ee2efd5380 con 0x55ee2f0c9270 2024-04-24T09:11:01.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.867+0000 7f43ce7f2740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55ee2f02ae30 con 0x55ee2f0c9b90 2024-04-24T09:11:01.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.867+0000 7f43c3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee2f0d4260 0x55ee2f0d6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:01.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.867+0000 7f43c9a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ee2f0c9270 0x55ee2f0c9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:01.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.867+0000 7f43c9a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ee2f0c9270 0x55ee2f0c9650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:41930/0 (socket says 172.21.15.83:41930) 2024-04-24T09:11:01.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.867+0000 7f43c9a00640 1 -- 172.21.15.83:0/2241096299 learned_addr learned my addr 172.21.15.83:0/2241096299 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:11:01.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.868+0000 7f43c9000640 1 --2- 172.21.15.83:0/2241096299 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee2f0c9b90 0x55ee2f0d3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:01.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.868+0000 7f43c1600640 1 --2- 172.21.15.83:0/1711817431 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f43ac0a3e20 0x7f43ac0a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:01.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.868+0000 7f43c9a00640 1 -- 172.21.15.83:0/2241096299 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee2f0c9b90 msgr2=0x55ee2f0d3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:01.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.868+0000 7f43c9a00640 1 --2- 172.21.15.83:0/2241096299 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee2f0c9b90 0x55ee2f0d3d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:01.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.868+0000 7f43c9a00640 1 -- 172.21.15.83:0/2241096299 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee2f0d4260 msgr2=0x55ee2f0d6650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:01.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.869+0000 7f43c9a00640 1 --2- 172.21.15.83:0/2241096299 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee2f0d4260 0x55ee2f0d6650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:01.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.869+0000 7f43c9a00640 1 -- 172.21.15.83:0/2241096299 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee2ee097e0 con 0x55ee2f0c9270 2024-04-24T09:11:01.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.869+0000 7f43c9000640 1 --2- 172.21.15.83:0/2241096299 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee2f0c9b90 0x55ee2f0d3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:11:01.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.869+0000 7f43c3e00640 1 --2- 172.21.15.83:0/2241096299 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee2f0d4260 0x55ee2f0d6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:11:01.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.868+0000 7f43c1600640 1 -- 172.21.15.83:0/1711817431 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 9583+0+0 (secure 0 0 0) 0x7f43b00f0860 con 0x7f43c4109800 2024-04-24T09:11:01.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.870+0000 7f43c9a00640 1 --2- 172.21.15.83:0/1711817431 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f43ac0a3e20 0x7f43ac0a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:01.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.871+0000 7f43c9a00640 1 --2- 172.21.15.83:0/2241096299 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ee2f0c9270 0x55ee2f0c9650 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f43bc0117e0 tx=0x7f43bc011ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b64c7427d18cd3bf server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:01.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.872+0000 7f43b6000640 1 -- 172.21.15.83:0/2241096299 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43bc023070 con 0x55ee2f0c9270 2024-04-24T09:11:01.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.872+0000 7f43b6000640 1 -- 172.21.15.83:0/2241096299 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f43bc00fde0 con 0x55ee2f0c9270 2024-04-24T09:11:01.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.872+0000 7f43b6000640 1 -- 172.21.15.83:0/2241096299 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43bc010630 con 0x55ee2f0c9270 2024-04-24T09:11:01.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.872+0000 7f43c9a00640 1 --2- 172.21.15.83:0/1711817431 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f43ac0a3e20 0x7f43ac0a62e0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f43bc020000 tx=0x7f43bc021000 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:01.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.872+0000 7f43ce7f2740 1 -- 172.21.15.83:0/2241096299 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ee2f0c9270 msgr2=0x55ee2f0c9650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:01.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.872+0000 7f43ce7f2740 1 --2- 172.21.15.83:0/2241096299 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ee2f0c9270 0x55ee2f0c9650 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f43bc0117e0 tx=0x7f43bc011ca0 comp rx=0 tx=0).stop 2024-04-24T09:11:01.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.873+0000 7f43ce7f2740 1 -- 172.21.15.83:0/2241096299 shutdown_connections 2024-04-24T09:11:01.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.873+0000 7f43ce7f2740 1 --2- 172.21.15.83:0/2241096299 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee2f0d4260 0x55ee2f0d6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:01.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.873+0000 7f43ce7f2740 1 --2- 172.21.15.83:0/2241096299 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee2f0c9b90 0x55ee2f0d3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:01.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.873+0000 7f43ce7f2740 1 --2- 172.21.15.83:0/2241096299 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ee2f0c9270 0x55ee2f0c9650 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:01.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.873+0000 7f43ce7f2740 1 -- 172.21.15.83:0/2241096299 >> 172.21.15.83:0/2241096299 conn(0x55ee2f0c5590 msgr2=0x55ee2f0c5970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:01.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.873+0000 7f43ce7f2740 1 -- 172.21.15.83:0/2241096299 shutdown_connections 2024-04-24T09:11:01.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.873+0000 7f43ce7f2740 1 -- 172.21.15.83:0/2241096299 wait complete. 2024-04-24T09:11:01.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.874+0000 7f43ce7f2740 1 -- start start 2024-04-24T09:11:01.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.874+0000 7f43ce7f2740 1 Processor -- start 2024-04-24T09:11:01.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.875+0000 7f43ce7f2740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee2f0c9270 0x55ee2f0d1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:01.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.875+0000 7f43ce7f2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ee2f0c9b90 0x55ee2f0d17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:01.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.875+0000 7f43ce7f2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee2f0d4260 0x55ee2f0d1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:01.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.875+0000 7f43ce7f2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ee2f0d7050 con 0x55ee2f0d4260 2024-04-24T09:11:01.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.875+0000 7f43ce7f2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ee2efe8660 con 0x55ee2f0c9b90 2024-04-24T09:11:01.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.875+0000 7f43ce7f2740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55ee2f0d71d0 con 0x55ee2f0c9270 2024-04-24T09:11:01.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.875+0000 7f43ce7f2740 10 client.0 _ll_register_callbacks cb 0x55ee2efd6aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:11:01.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.875+0000 7f43c9000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee2f0c9270 0x55ee2f0d1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:01.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.875+0000 7f43c9000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee2f0c9270 0x55ee2f0d1260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:39794/0 (socket says 172.21.15.83:39794) 2024-04-24T09:11:01.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.875+0000 7f43c9000640 1 -- 172.21.15.83:0/1164708362 learned_addr learned my addr 172.21.15.83:0/1164708362 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:11:01.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.875+0000 7f43c3e00640 1 --2- 172.21.15.83:0/1164708362 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ee2f0c9b90 0x55ee2f0d17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:01.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.876+0000 7f43c9a00640 1 --2- 172.21.15.83:0/1164708362 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee2f0d4260 0x55ee2f0d1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:01.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.876+0000 7f43c9000640 1 -- 172.21.15.83:0/1164708362 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ee2f0c9b90 msgr2=0x55ee2f0d17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:01.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.876+0000 7f43c9000640 1 --2- 172.21.15.83:0/1164708362 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ee2f0c9b90 0x55ee2f0d17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:01.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.876+0000 7f43c9000640 1 -- 172.21.15.83:0/1164708362 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee2f0d4260 msgr2=0x55ee2f0d1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:01.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.876+0000 7f43c9000640 1 --2- 172.21.15.83:0/1164708362 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee2f0d4260 0x55ee2f0d1ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:01.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.876+0000 7f43c9000640 1 -- 172.21.15.83:0/1164708362 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee2ee69f70 con 0x55ee2f0c9270 2024-04-24T09:11:01.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.876+0000 7f43c9a00640 1 --2- 172.21.15.83:0/1164708362 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee2f0d4260 0x55ee2f0d1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:11:01.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.877+0000 7f43c9000640 1 --2- 172.21.15.83:0/1164708362 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee2f0c9270 0x55ee2f0d1260 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f43b8012780 tx=0x7f43b8012b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb5a2471a8b860db server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:01.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.877+0000 7f43aaa00640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-04-24T09:11:01.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.877+0000 7f43aaa00640 1 -- 172.21.15.83:0/1164708362 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b8010dc0 con 0x55ee2f0c9270 2024-04-24T09:11:01.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.877+0000 7f43ce7f2740 1 -- 172.21.15.83:0/1164708362 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43bc00fde0 con 0x55ee2f0c9270 2024-04-24T09:11:01.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.877+0000 7f43aaa00640 1 -- 172.21.15.83:0/1164708362 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(120..120 src has 1..120) ==== 9583+0+0 (secure 0 0 0) 0x7f43b801c3f0 con 0x55ee2f0c9270 2024-04-24T09:11:01.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.878+0000 7f43aaa00640 1 -- 172.21.15.83:0/1164708362 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f43b8027050 con 0x55ee2f0c9270 2024-04-24T09:11:01.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.878+0000 7f43ce7f2740 10 client.29338 fetch_fsmap learned FSMap version 61 2024-04-24T09:11:01.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.878+0000 7f43ce7f2740 1 -- 172.21.15.83:0/1164708362 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x55ee2ed01170 con 0x55ee2f0c9270 2024-04-24T09:11:01.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.879+0000 7f43aaa00640 1 -- 172.21.15.83:0/1164708362 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f43bc00fde0 con 0x55ee2f0c9270 2024-04-24T09:11:01.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.879+0000 7f43aaa00640 1 -- 172.21.15.83:0/1164708362 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 61) ==== 8362+0+0 (secure 0 0 0) 0x7f43b800e6f0 con 0x55ee2f0c9270 2024-04-24T09:11:01.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.880+0000 7f43ce7f2740 10 client.29338 fetch_fsmap finished waiting for FSMap version 61 2024-04-24T09:11:01.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.880+0000 7f43ce7f2740 10 client.29338 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:11:01.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.880+0000 7f43ce7f2740 20 client.29338 populate_metadata read hostname 'smithi083' 2024-04-24T09:11:01.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.880+0000 7f43ce7f2740 1 --2- 172.21.15.83:0/1164708362 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ee2f0e4650 0x55ee2f0e6a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:01.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.880+0000 7f43ce7f2740 4 client.29338 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:11:01.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.880+0000 7f43ce7f2740 1 -- 172.21.15.83:0/1164708362 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ee2f03c890 con 0x55ee2f0e4650 2024-04-24T09:11:01.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.880+0000 7f43c3e00640 1 --2- 172.21.15.83:0/1164708362 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ee2f0e4650 0x55ee2f0e6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:01.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.881+0000 7f43c3e00640 1 --2- 172.21.15.83:0/1164708362 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ee2f0e4650 0x55ee2f0e6a80 crc :-1 s=READY pgs=485 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3609a5648e6cf747 server_cookie=edbc70fd264de620 in_seq=0 out_seq=0 2024-04-24T09:11:01.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.881+0000 7f43aaa00640 10 client.29338 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:11:01.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.882+0000 7f43aaa00640 1 -- 172.21.15.83:0/1164708362 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ee2f03c890 con 0x55ee2f0e4650 2024-04-24T09:11:01.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.882+0000 7f43aaa00640 10 client.29338 handle_command_reply: tid=0 2024-04-24T09:11:01.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.882+0000 7f43ce7f2740 1 client.29338 shutdown 2024-04-24T09:11:01.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.882+0000 7f43ce7f2740 1 -- 172.21.15.83:0/1164708362 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee2f0c9270 msgr2=0x55ee2f0d1260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:01.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.882+0000 7f43ce7f2740 1 --2- 172.21.15.83:0/1164708362 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee2f0c9270 0x55ee2f0d1260 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f43b8012780 tx=0x7f43b8012b80 comp rx=0 tx=0).stop 2024-04-24T09:11:01.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.882+0000 7f43ce7f2740 1 -- 172.21.15.83:0/1164708362 shutdown_connections 2024-04-24T09:11:01.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.882+0000 7f43ce7f2740 1 --2- 172.21.15.83:0/1164708362 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ee2f0e4650 0x55ee2f0e6a80 crc :-1 s=READY pgs=485 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:01.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.882+0000 7f43ce7f2740 1 --2- 172.21.15.83:0/1164708362 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee2f0d4260 0x55ee2f0d1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:01.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.882+0000 7f43aaa00640 0 client.29338 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:11:01.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.882+0000 7f43ce7f2740 1 --2- 172.21.15.83:0/1164708362 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ee2f0c9b90 0x55ee2f0d17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:01.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.882+0000 7f43ce7f2740 1 --2- 172.21.15.83:0/1164708362 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee2f0c9270 0x55ee2f0d1260 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:01.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.882+0000 7f43ce7f2740 1 -- 172.21.15.83:0/1164708362 >> 172.21.15.83:0/1164708362 conn(0x55ee2f0c5590 msgr2=0x55ee2f0cb390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:01.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.883+0000 7f43ce7f2740 1 -- 172.21.15.83:0/1164708362 shutdown_connections 2024-04-24T09:11:01.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.883+0000 7f43ce7f2740 1 -- 172.21.15.83:0/1164708362 wait complete. 2024-04-24T09:11:01.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.883+0000 7f43ce7f2740 20 client.29338 trim_cache size 0 max 16384 2024-04-24T09:11:01.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.914+0000 7f43ce7f2740 1 Processor -- start 2024-04-24T09:11:01.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.914+0000 7f43ce7f2740 1 -- start start 2024-04-24T09:11:01.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.914+0000 7f43ce7f2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ee2f0c9270 0x55ee2f0dff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:01.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.914+0000 7f43ce7f2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee2f0c9b90 0x55ee2f0c4570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:01.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.914+0000 7f43ce7f2740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee2f0d4260 0x55ee2f0ec350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:01.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.914+0000 7f43ce7f2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f43c41be350 con 0x55ee2f0c9b90 2024-04-24T09:11:01.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.914+0000 7f43ce7f2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f43c41bfe90 con 0x55ee2f0c9270 2024-04-24T09:11:01.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.914+0000 7f43ce7f2740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f43c41bea20 con 0x55ee2f0d4260 2024-04-24T09:11:01.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.915+0000 7f43c3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee2f0c9b90 0x55ee2f0c4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:01.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.915+0000 7f43c3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee2f0c9b90 0x55ee2f0c4570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58600/0 (socket says 172.21.15.83:58600) 2024-04-24T09:11:01.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.915+0000 7f43c3e00640 1 -- 172.21.15.83:0/3861561217 learned_addr learned my addr 172.21.15.83:0/3861561217 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:11:01.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.915+0000 7f43c9000640 1 --2- 172.21.15.83:0/3861561217 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ee2f0c9270 0x55ee2f0dff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:01.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.915+0000 7f43c9a00640 1 --2- 172.21.15.83:0/3861561217 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee2f0d4260 0x55ee2f0ec350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:01.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.915+0000 7f43c3e00640 1 -- 172.21.15.83:0/3861561217 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee2f0d4260 msgr2=0x55ee2f0ec350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:01.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.915+0000 7f43c3e00640 1 --2- 172.21.15.83:0/3861561217 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee2f0d4260 0x55ee2f0ec350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:01.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.915+0000 7f43c3e00640 1 -- 172.21.15.83:0/3861561217 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ee2f0c9270 msgr2=0x55ee2f0dff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:01.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.915+0000 7f43c3e00640 1 --2- 172.21.15.83:0/3861561217 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ee2f0c9270 0x55ee2f0dff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:01.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.915+0000 7f43c3e00640 1 -- 172.21.15.83:0/3861561217 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee2f0c99c0 con 0x55ee2f0c9b90 2024-04-24T09:11:01.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.916+0000 7f43c3e00640 1 --2- 172.21.15.83:0/3861561217 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee2f0c9b90 0x55ee2f0c4570 secure :-1 s=READY pgs=2481 cs=0 l=1 rev1=1 crypto rx=0x7f43b00bc4b0 tx=0x7f43b00f2040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7497a7474e71b102 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:01.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.916+0000 7f43b5600640 1 -- 172.21.15.83:0/3861561217 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b00cbe60 con 0x55ee2f0c9b90 2024-04-24T09:11:01.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.916+0000 7f43b5600640 1 -- 172.21.15.83:0/3861561217 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f43b003e450 con 0x55ee2f0c9b90 2024-04-24T09:11:01.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.916+0000 7f43b5600640 1 -- 172.21.15.83:0/3861561217 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b00d4070 con 0x55ee2f0c9b90 2024-04-24T09:11:01.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.917+0000 7f43ce7f2740 1 -- 172.21.15.83:0/3861561217 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee2f0c9b90 msgr2=0x55ee2f0c4570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:01.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.917+0000 7f43ce7f2740 1 --2- 172.21.15.83:0/3861561217 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee2f0c9b90 0x55ee2f0c4570 secure :-1 s=READY pgs=2481 cs=0 l=1 rev1=1 crypto rx=0x7f43b00bc4b0 tx=0x7f43b00f2040 comp rx=0 tx=0).stop 2024-04-24T09:11:01.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.917+0000 7f43ce7f2740 1 -- 172.21.15.83:0/3861561217 shutdown_connections 2024-04-24T09:11:01.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.917+0000 7f43ce7f2740 1 --2- 172.21.15.83:0/3861561217 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee2f0d4260 0x55ee2f0ec350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:01.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.917+0000 7f43ce7f2740 1 --2- 172.21.15.83:0/3861561217 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee2f0c9b90 0x55ee2f0c4570 unknown :-1 s=CLOSED pgs=2481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:01.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.917+0000 7f43ce7f2740 1 --2- 172.21.15.83:0/3861561217 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ee2f0c9270 0x55ee2f0dff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:01.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.917+0000 7f43ce7f2740 1 -- 172.21.15.83:0/3861561217 >> 172.21.15.83:0/3861561217 conn(0x55ee2f0c5590 msgr2=0x55ee2f0c3e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:01.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.917+0000 7f43ce7f2740 1 -- 172.21.15.83:0/3861561217 shutdown_connections 2024-04-24T09:11:01.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.917+0000 7f43ce7f2740 1 -- 172.21.15.83:0/3861561217 wait complete. 2024-04-24T09:11:01.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.918+0000 7f43ce7f2740 1 -- start start 2024-04-24T09:11:01.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.918+0000 7f43ce7f2740 1 Processor -- start 2024-04-24T09:11:01.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.919+0000 7f43ce7f2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ee2f0c9270 0x55ee2f0d8590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:01.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.919+0000 7f43ce7f2740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee2f0d4260 0x55ee2f0d8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:01.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.919+0000 7f43ce7f2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee2f0e4650 0x55ee2f149bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:01.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.919+0000 7f43ce7f2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f43c41bf7c0 con 0x55ee2f0e4650 2024-04-24T09:11:01.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.919+0000 7f43ce7f2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f43c41bf0f0 con 0x55ee2f0c9270 2024-04-24T09:11:01.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.920+0000 7f43ce7f2740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f43c41bc350 con 0x55ee2f0d4260 2024-04-24T09:11:01.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.920+0000 7f43ce7f2740 10 client.0 _ll_register_callbacks cb 0x7f43b8009660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:11:01.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.920+0000 7f43c9000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ee2f0c9270 0x55ee2f0d8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:01.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.920+0000 7f43c9000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ee2f0c9270 0x55ee2f0d8590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:41944/0 (socket says 172.21.15.83:41944) 2024-04-24T09:11:01.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.920+0000 7f43c9000640 1 -- 172.21.15.83:0/2620623229 learned_addr learned my addr 172.21.15.83:0/2620623229 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:11:01.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.920+0000 7f43c9a00640 1 --2- 172.21.15.83:0/2620623229 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee2f0e4650 0x55ee2f149bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:01.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.920+0000 7f43c3e00640 1 --2- 172.21.15.83:0/2620623229 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee2f0d4260 0x55ee2f0d8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:01.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.921+0000 7f43c9000640 1 -- 172.21.15.83:0/2620623229 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee2f0d4260 msgr2=0x55ee2f0d8ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:01.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.921+0000 7f43c9000640 1 --2- 172.21.15.83:0/2620623229 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee2f0d4260 0x55ee2f0d8ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:01.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.921+0000 7f43c9000640 1 -- 172.21.15.83:0/2620623229 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee2f0e4650 msgr2=0x55ee2f149bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:01.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.921+0000 7f43c9000640 1 --2- 172.21.15.83:0/2620623229 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee2f0e4650 0x55ee2f149bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:01.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.921+0000 7f43c9000640 1 -- 172.21.15.83:0/2620623229 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee2f14a180 con 0x55ee2f0c9270 2024-04-24T09:11:01.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.922+0000 7f43c9000640 1 --2- 172.21.15.83:0/2620623229 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ee2f0c9270 0x55ee2f0d8590 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f43b800eb90 tx=0x7f43b8009c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33cc256ff02bdace server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:01.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.922+0000 7f43aaa00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:11:01.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.922+0000 7f43aaa00640 1 -- 172.21.15.83:0/2620623229 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b8030020 con 0x55ee2f0c9270 2024-04-24T09:11:01.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.922+0000 7f43aaa00640 1 -- 172.21.15.83:0/2620623229 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(120..120 src has 1..120) ==== 9583+0+0 (secure 0 0 0) 0x7f43b8010dd0 con 0x55ee2f0c9270 2024-04-24T09:11:01.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.922+0000 7f43ce7f2740 1 -- 172.21.15.83:0/2620623229 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ee2f036700 con 0x55ee2f0c9270 2024-04-24T09:11:01.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.923+0000 7f43aaa00640 1 -- 172.21.15.83:0/2620623229 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f43b803b050 con 0x55ee2f0c9270 2024-04-24T09:11:01.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.923+0000 7f43ce7f2740 10 client.39425 fetch_fsmap learned FSMap version 61 2024-04-24T09:11:01.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.923+0000 7f43ce7f2740 1 -- 172.21.15.83:0/2620623229 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x55ee2f14b4a0 con 0x55ee2f0c9270 2024-04-24T09:11:01.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.924+0000 7f43aaa00640 1 -- 172.21.15.83:0/2620623229 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55ee2f036700 con 0x55ee2f0c9270 2024-04-24T09:11:01.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.924+0000 7f43aaa00640 1 -- 172.21.15.83:0/2620623229 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 61) ==== 8362+0+0 (secure 0 0 0) 0x7f43b80035f0 con 0x55ee2f0c9270 2024-04-24T09:11:01.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.924+0000 7f43ce7f2740 10 client.39425 fetch_fsmap finished waiting for FSMap version 61 2024-04-24T09:11:01.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.924+0000 7f43ce7f2740 10 client.39425 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:11:01.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.924+0000 7f43ce7f2740 20 client.39425 populate_metadata read hostname 'smithi083' 2024-04-24T09:11:01.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.924+0000 7f43ce7f2740 1 --2- 172.21.15.83:0/2620623229 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ee2f14bbb0 0x55ee2f14dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:01.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.924+0000 7f43ce7f2740 4 client.39425 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T09:11:01.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.924+0000 7f43ce7f2740 1 -- 172.21.15.83:0/2620623229 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ee2efb2980 con 0x55ee2f14bbb0 2024-04-24T09:11:01.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.925+0000 7f43c3e00640 1 --2- 172.21.15.83:0/2620623229 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ee2f14bbb0 0x55ee2f14dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:01.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.925+0000 7f43c3e00640 1 --2- 172.21.15.83:0/2620623229 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ee2f14bbb0 0x55ee2f14dfa0 crc :-1 s=READY pgs=486 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=183320cba8903972 server_cookie=6cea650f6d79dcf2 in_seq=0 out_seq=0 2024-04-24T09:11:01.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.925+0000 7f43aaa00640 10 client.39425 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:11:01.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.929+0000 7f43aaa00640 1 -- 172.21.15.83:0/2620623229 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ee2efb2980 con 0x55ee2f14bbb0 2024-04-24T09:11:01.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.929+0000 7f43aaa00640 10 client.39425 handle_command_reply: tid=0 2024-04-24T09:11:01.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.930+0000 7f43ce7f2740 1 client.39425 shutdown 2024-04-24T09:11:01.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.930+0000 7f43ce7f2740 1 -- 172.21.15.83:0/2620623229 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ee2f0c9270 msgr2=0x55ee2f0d8590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:01.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.930+0000 7f43ce7f2740 1 --2- 172.21.15.83:0/2620623229 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ee2f0c9270 0x55ee2f0d8590 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f43b800eb90 tx=0x7f43b8009c90 comp rx=0 tx=0).stop 2024-04-24T09:11:01.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.930+0000 7f43ce7f2740 1 -- 172.21.15.83:0/2620623229 shutdown_connections 2024-04-24T09:11:01.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.930+0000 7f43ce7f2740 1 --2- 172.21.15.83:0/2620623229 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ee2f14bbb0 0x55ee2f14dfa0 crc :-1 s=READY pgs=486 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:01.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.930+0000 7f43ce7f2740 1 --2- 172.21.15.83:0/2620623229 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee2f0e4650 0x55ee2f149bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:01.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.930+0000 7f43aaa00640 0 client.39425 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:11:01.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.930+0000 7f43ce7f2740 1 --2- 172.21.15.83:0/2620623229 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee2f0d4260 0x55ee2f0d8ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:01.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.930+0000 7f43ce7f2740 1 --2- 172.21.15.83:0/2620623229 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ee2f0c9270 0x55ee2f0d8590 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:01.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.930+0000 7f43ce7f2740 1 -- 172.21.15.83:0/2620623229 >> 172.21.15.83:0/2620623229 conn(0x55ee2f0c5590 msgr2=0x55ee2f0c46f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:01.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.930+0000 7f43ce7f2740 1 -- 172.21.15.83:0/2620623229 shutdown_connections 2024-04-24T09:11:01.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.930+0000 7f43ce7f2740 1 -- 172.21.15.83:0/2620623229 wait complete. 2024-04-24T09:11:01.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.930+0000 7f43ce7f2740 20 client.39425 trim_cache size 0 max 16384 2024-04-24T09:11:01.932 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T09:11:01.932 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-04-24T09:11:01.932 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "b7b5b5bd-8add-404d-a716-4fe458597c16", 2024-04-24T09:11:01.932 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-04-24T09:11:01.932 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T09:11:01.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.931+0000 7f43ca400640 1 -- 172.21.15.83:0/1711817431 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f43ac0a3e20 msgr2=0x7f43ac0a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:01.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.931+0000 7f43ca400640 1 --2- 172.21.15.83:0/1711817431 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f43ac0a3e20 0x7f43ac0a62e0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f43bc020000 tx=0x7f43bc021000 comp rx=0 tx=0).stop 2024-04-24T09:11:01.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.931+0000 7f43ca400640 1 -- 172.21.15.83:0/1711817431 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43c4109800 msgr2=0x7f43c406c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:01.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.931+0000 7f43ca400640 1 --2- 172.21.15.83:0/1711817431 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43c4109800 0x7f43c406c9c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f43b0009780 tx=0x7f43b00042f0 comp rx=0 tx=0).stop 2024-04-24T09:11:01.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.932+0000 7f43ca400640 1 -- 172.21.15.83:0/1711817431 shutdown_connections 2024-04-24T09:11:01.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.932+0000 7f43ca400640 1 --2- 172.21.15.83:0/1711817431 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f43ac0a3e20 0x7f43ac0a62e0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:01.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.932+0000 7f43ca400640 1 --2- 172.21.15.83:0/1711817431 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43c410e5a0 0x7f43c406cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:01.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.932+0000 7f43ca400640 1 --2- 172.21.15.83:0/1711817431 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43c4109800 0x7f43c406c9c0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:01.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.932+0000 7f43ca400640 1 --2- 172.21.15.83:0/1711817431 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f43c4108e40 0x7f43c4073850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:01.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.932+0000 7f43ca400640 1 -- 172.21.15.83:0/1711817431 >> 172.21.15.83:0/1711817431 conn(0x7f43c4077a20 msgr2=0x7f43c407a530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:01.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.932+0000 7f43ca400640 1 -- 172.21.15.83:0/1711817431 shutdown_connections 2024-04-24T09:11:01.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:01.932+0000 7f43ca400640 1 -- 172.21.15.83:0/1711817431 wait complete. 2024-04-24T09:11:01.955 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T09:11:01.956 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T09:11:02.177 INFO:tasks.workunit.client.0.smithi083.stdout: 10 42064 6.71 MB/sec execute 213 sec latency 91.253 ms 2024-04-24T09:11:02.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.228+0000 7f0cf7200640 1 -- 172.21.15.83:0/2807804642 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0cf8076e50 msgr2=0x7f0cf8112810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:02.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.228+0000 7f0cf7200640 1 --2- 172.21.15.83:0/2807804642 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0cf8076e50 0x7f0cf8112810 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0ce4009930 tx=0x7f0ce402a420 comp rx=0 tx=0).stop 2024-04-24T09:11:02.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.228+0000 7f0cf7200640 1 -- 172.21.15.83:0/2807804642 shutdown_connections 2024-04-24T09:11:02.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.228+0000 7f0cf7200640 1 --2- 172.21.15.83:0/2807804642 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0cf8112d50 0x7f0cf8115140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:02.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.228+0000 7f0cf7200640 1 --2- 172.21.15.83:0/2807804642 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0cf8076e50 0x7f0cf8112810 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:02.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.228+0000 7f0cf7200640 1 --2- 172.21.15.83:0/2807804642 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0cf8076530 0x7f0cf8076910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:02.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.228+0000 7f0cf7200640 1 -- 172.21.15.83:0/2807804642 >> 172.21.15.83:0/2807804642 conn(0x7f0cf80ffb40 msgr2=0x7f0cf8101f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:02.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.228+0000 7f0cf7200640 1 -- 172.21.15.83:0/2807804642 shutdown_connections 2024-04-24T09:11:02.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.228+0000 7f0cf7200640 1 -- 172.21.15.83:0/2807804642 wait complete. 2024-04-24T09:11:02.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.229+0000 7f0cf7200640 1 Processor -- start 2024-04-24T09:11:02.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.230+0000 7f0cf7200640 1 -- start start 2024-04-24T09:11:02.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.230+0000 7f0cf7200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0cf8076530 0x7f0cf819ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:02.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.230+0000 7f0cf7200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0cf8076e50 0x7f0cf819f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:02.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.230+0000 7f0cf7200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0cf8112d50 0x7f0cf81a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:02.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.230+0000 7f0cf7200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf81187c0 con 0x7f0cf8076530 2024-04-24T09:11:02.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.230+0000 7f0cf7200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf8118640 con 0x7f0cf8076e50 2024-04-24T09:11:02.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.230+0000 7f0cf7200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf8118940 con 0x7f0cf8112d50 2024-04-24T09:11:02.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.231+0000 7f0cf5e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0cf8076530 0x7f0cf819ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:02.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.231+0000 7f0cf6800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0cf8112d50 0x7f0cf81a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:02.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.231+0000 7f0cf5e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0cf8076530 0x7f0cf819ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58628/0 (socket says 172.21.15.83:58628) 2024-04-24T09:11:02.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.231+0000 7f0cf5e00640 1 -- 172.21.15.83:0/1862859135 learned_addr learned my addr 172.21.15.83:0/1862859135 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:11:02.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.231+0000 7f0cf5400640 1 --2- 172.21.15.83:0/1862859135 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0cf8076e50 0x7f0cf819f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:02.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.231+0000 7f0cf5e00640 1 -- 172.21.15.83:0/1862859135 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0cf8112d50 msgr2=0x7f0cf81a3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:02.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.231+0000 7f0cf5e00640 1 --2- 172.21.15.83:0/1862859135 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0cf8112d50 0x7f0cf81a3600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:02.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.231+0000 7f0cf5e00640 1 -- 172.21.15.83:0/1862859135 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0cf8076e50 msgr2=0x7f0cf819f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:02.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.231+0000 7f0cf5e00640 1 --2- 172.21.15.83:0/1862859135 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0cf8076e50 0x7f0cf819f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:02.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.231+0000 7f0cf5e00640 1 -- 172.21.15.83:0/1862859135 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cf81a3d30 con 0x7f0cf8076530 2024-04-24T09:11:02.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.231+0000 7f0cf5e00640 1 --2- 172.21.15.83:0/1862859135 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0cf8076530 0x7f0cf819ed40 secure :-1 s=READY pgs=2482 cs=0 l=1 rev1=1 crypto rx=0x7f0cf000eb90 tx=0x7f0cf000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:02.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.232+0000 7f0ceea00640 1 -- 172.21.15.83:0/1862859135 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf0004d20 con 0x7f0cf8076530 2024-04-24T09:11:02.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.232+0000 7f0cf7200640 1 -- 172.21.15.83:0/1862859135 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cf81a4020 con 0x7f0cf8076530 2024-04-24T09:11:02.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.232+0000 7f0ceea00640 1 -- 172.21.15.83:0/1862859135 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0cf000ee90 con 0x7f0cf8076530 2024-04-24T09:11:02.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.232+0000 7f0ceea00640 1 -- 172.21.15.83:0/1862859135 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf0010670 con 0x7f0cf8076530 2024-04-24T09:11:02.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.232+0000 7f0cf7200640 1 -- 172.21.15.83:0/1862859135 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cf81ab8b0 con 0x7f0cf8076530 2024-04-24T09:11:02.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.235+0000 7f0cff770740 1 Processor -- start 2024-04-24T09:11:02.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.236+0000 7f0cff770740 1 -- start start 2024-04-24T09:11:02.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.236+0000 7f0cff770740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631b444fa30 0x5631b444fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:02.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.236+0000 7f0cff770740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5631b4450350 0x5631b445a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:02.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.236+0000 7f0cff770740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631b445aa20 0x5631b445ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:02.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.236+0000 7f0cff770740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5631b43b51f0 con 0x5631b445aa20 2024-04-24T09:11:02.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.236+0000 7f0cff770740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5631b435b350 con 0x5631b4450350 2024-04-24T09:11:02.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.236+0000 7f0cff770740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5631b43b11b0 con 0x5631b444fa30 2024-04-24T09:11:02.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.236+0000 7f0ceea00640 1 -- 172.21.15.83:0/1862859135 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0cf0005020 con 0x7f0cf8076530 2024-04-24T09:11:02.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.237+0000 7f0cf5e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5631b4450350 0x5631b445a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:02.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.237+0000 7f0cf5e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5631b4450350 0x5631b445a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:41966/0 (socket says 172.21.15.83:41966) 2024-04-24T09:11:02.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.237+0000 7f0cf5e00640 1 -- 172.21.15.83:0/2763140371 learned_addr learned my addr 172.21.15.83:0/2763140371 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:11:02.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.237+0000 7f0cf5400640 1 --2- 172.21.15.83:0/2763140371 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631b445aa20 0x5631b445ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:02.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.237+0000 7f0cf6800640 1 --2- 172.21.15.83:0/2763140371 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631b444fa30 0x5631b444fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:02.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.237+0000 7f0cf5e00640 1 -- 172.21.15.83:0/2763140371 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631b444fa30 msgr2=0x5631b444fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:02.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.237+0000 7f0cf5e00640 1 --2- 172.21.15.83:0/2763140371 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631b444fa30 0x5631b444fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:02.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.237+0000 7f0cf5e00640 1 -- 172.21.15.83:0/2763140371 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631b445aa20 msgr2=0x5631b445ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:02.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.237+0000 7f0cf5e00640 1 --2- 172.21.15.83:0/2763140371 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631b445aa20 0x5631b445ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:02.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.237+0000 7f0cf5e00640 1 -- 172.21.15.83:0/2763140371 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631b43aff50 con 0x5631b4450350 2024-04-24T09:11:02.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.238+0000 7f0cf5e00640 1 --2- 172.21.15.83:0/2763140371 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5631b4450350 0x5631b445a4e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f0cf00ce4c0 tx=0x7f0cf00d9330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25ed4126deec32d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:02.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.239+0000 7f0ceea00640 1 --2- 172.21.15.83:0/1862859135 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f0cdc0a3e60 0x7f0cdc0a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:02.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.239+0000 7f0ce3000640 1 -- 172.21.15.83:0/2763140371 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf00e4070 con 0x5631b4450350 2024-04-24T09:11:02.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.239+0000 7f0ce3000640 1 -- 172.21.15.83:0/2763140371 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0cf009d4d0 con 0x5631b4450350 2024-04-24T09:11:02.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.239+0000 7f0ce3000640 1 -- 172.21.15.83:0/2763140371 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf009c910 con 0x5631b4450350 2024-04-24T09:11:02.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.239+0000 7f0ceea00640 1 -- 172.21.15.83:0/1862859135 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 9583+0+0 (secure 0 0 0) 0x7f0cf00d12b0 con 0x7f0cf8076530 2024-04-24T09:11:02.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.240+0000 7f0cff770740 1 -- 172.21.15.83:0/2763140371 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5631b4450350 msgr2=0x5631b445a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:02.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.240+0000 7f0cff770740 1 --2- 172.21.15.83:0/2763140371 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5631b4450350 0x5631b445a4e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f0cf00ce4c0 tx=0x7f0cf00d9330 comp rx=0 tx=0).stop 2024-04-24T09:11:02.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.241+0000 7f0cff770740 1 -- 172.21.15.83:0/2763140371 shutdown_connections 2024-04-24T09:11:02.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.241+0000 7f0cff770740 1 --2- 172.21.15.83:0/2763140371 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631b445aa20 0x5631b445ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:02.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.241+0000 7f0cff770740 1 --2- 172.21.15.83:0/2763140371 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5631b4450350 0x5631b445a4e0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:02.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.241+0000 7f0cff770740 1 --2- 172.21.15.83:0/2763140371 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631b444fa30 0x5631b444fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:02.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.241+0000 7f0cff770740 1 -- 172.21.15.83:0/2763140371 >> 172.21.15.83:0/2763140371 conn(0x5631b444bd50 msgr2=0x5631b444c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:02.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.242+0000 7f0cf6800640 1 --2- 172.21.15.83:0/1862859135 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f0cdc0a3e60 0x7f0cdc0a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:02.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.242+0000 7f0cff770740 1 -- 172.21.15.83:0/2763140371 shutdown_connections 2024-04-24T09:11:02.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.243+0000 7f0cff770740 1 -- 172.21.15.83:0/2763140371 wait complete. 2024-04-24T09:11:02.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.243+0000 7f0cf6800640 1 --2- 172.21.15.83:0/1862859135 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f0cdc0a3e60 0x7f0cdc0a6320 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f0ce8005fd0 tx=0x7f0ce8005cc0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:02.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.243+0000 7f0cff770740 1 -- start start 2024-04-24T09:11:02.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.244+0000 7f0cff770740 1 Processor -- start 2024-04-24T09:11:02.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.244+0000 7f0cff770740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631b444fa30 0x5631b4457980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:02.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.244+0000 7f0cff770740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5631b4450350 0x5631b4457ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:02.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.244+0000 7f0cff770740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631b445aa20 0x5631b4458400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:02.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.245+0000 7f0cff770740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5631b445d810 con 0x5631b445aa20 2024-04-24T09:11:02.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.245+0000 7f0cff770740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5631b436e690 con 0x5631b4450350 2024-04-24T09:11:02.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.245+0000 7f0cff770740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5631b445d990 con 0x5631b444fa30 2024-04-24T09:11:02.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.245+0000 7f0cf5400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5631b4450350 0x5631b4457ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:02.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.245+0000 7f0cf5400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5631b4450350 0x5631b4457ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:41974/0 (socket says 172.21.15.83:41974) 2024-04-24T09:11:02.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.245+0000 7f0cf5400640 1 -- 172.21.15.83:0/1596665754 learned_addr learned my addr 172.21.15.83:0/1596665754 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:11:02.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.245+0000 7f0cff770740 10 client.0 _ll_register_callbacks cb 0x5631b43aa650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:11:02.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.245+0000 7f0cf5e00640 1 --2- 172.21.15.83:0/1596665754 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631b444fa30 0x5631b4457980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:02.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.245+0000 7f0cf6800640 1 --2- 172.21.15.83:0/1596665754 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631b445aa20 0x5631b4458400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:02.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.245+0000 7f0cf5400640 1 -- 172.21.15.83:0/1596665754 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631b444fa30 msgr2=0x5631b4457980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:02.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.245+0000 7f0cf5400640 1 --2- 172.21.15.83:0/1596665754 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631b444fa30 0x5631b4457980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:02.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.245+0000 7f0cf5400640 1 -- 172.21.15.83:0/1596665754 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631b445aa20 msgr2=0x5631b4458400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:02.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.245+0000 7f0cf5400640 1 --2- 172.21.15.83:0/1596665754 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631b445aa20 0x5631b4458400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:02.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.245+0000 7f0cf5400640 1 -- 172.21.15.83:0/1596665754 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631b436d920 con 0x5631b4450350 2024-04-24T09:11:02.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.245+0000 7f0cf6800640 1 --2- 172.21.15.83:0/1596665754 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631b445aa20 0x5631b4458400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:11:02.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.245+0000 7f0cf5e00640 1 --2- 172.21.15.83:0/1596665754 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631b444fa30 0x5631b4457980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:11:02.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.245+0000 7f0cf5400640 1 --2- 172.21.15.83:0/1596665754 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5631b4450350 0x5631b4457ec0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f0ce402a990 tx=0x7f0ce4005270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e9093dc3175a8c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:02.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.246+0000 7f0cd7400640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:11:02.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.246+0000 7f0cd7400640 1 -- 172.21.15.83:0/1596665754 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce4035040 con 0x5631b4450350 2024-04-24T09:11:02.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.246+0000 7f0cff770740 1 -- 172.21.15.83:0/1596665754 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0cf009d4d0 con 0x5631b4450350 2024-04-24T09:11:02.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.246+0000 7f0cd7400640 1 -- 172.21.15.83:0/1596665754 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(120..120 src has 1..120) ==== 9583+0+0 (secure 0 0 0) 0x7f0ce40048c0 con 0x5631b4450350 2024-04-24T09:11:02.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.246+0000 7f0cd7400640 1 -- 172.21.15.83:0/1596665754 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f0ce404e050 con 0x5631b4450350 2024-04-24T09:11:02.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.246+0000 7f0cff770740 10 client.39437 fetch_fsmap learned FSMap version 61 2024-04-24T09:11:02.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.246+0000 7f0cff770740 1 -- 172.21.15.83:0/1596665754 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x5631b41917d0 con 0x5631b4450350 2024-04-24T09:11:02.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.247+0000 7f0cd7400640 1 -- 172.21.15.83:0/1596665754 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0cf009d4d0 con 0x5631b4450350 2024-04-24T09:11:02.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.247+0000 7f0cd7400640 1 -- 172.21.15.83:0/1596665754 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 61) ==== 8362+0+0 (secure 0 0 0) 0x7f0ce4003b60 con 0x5631b4450350 2024-04-24T09:11:02.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.247+0000 7f0cff770740 10 client.39437 fetch_fsmap finished waiting for FSMap version 61 2024-04-24T09:11:02.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.247+0000 7f0cff770740 10 client.39437 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:11:02.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.247+0000 7f0cff770740 20 client.39437 populate_metadata read hostname 'smithi083' 2024-04-24T09:11:02.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.247+0000 7f0cff770740 1 --2- 172.21.15.83:0/1596665754 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5631b446ad40 0x5631b446d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:02.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.247+0000 7f0cff770740 4 client.39437 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:11:02.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.247+0000 7f0cff770740 1 -- 172.21.15.83:0/1596665754 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5631b43c2c10 con 0x5631b446ad40 2024-04-24T09:11:02.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.248+0000 7f0cf5e00640 1 --2- 172.21.15.83:0/1596665754 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5631b446ad40 0x5631b446d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:02.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.248+0000 7f0cf5e00640 1 --2- 172.21.15.83:0/1596665754 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5631b446ad40 0x5631b446d1e0 crc :-1 s=READY pgs=487 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cdc7a1bd237bdb33 server_cookie=60c1da3d2ab7f820 in_seq=0 out_seq=0 2024-04-24T09:11:02.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.248+0000 7f0cd7400640 10 client.39437 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:11:02.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.249+0000 7f0cd7400640 1 -- 172.21.15.83:0/1596665754 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5631b43c2c10 con 0x5631b446ad40 2024-04-24T09:11:02.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.249+0000 7f0cd7400640 10 client.39437 handle_command_reply: tid=0 2024-04-24T09:11:02.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.249+0000 7f0cff770740 1 client.39437 shutdown 2024-04-24T09:11:02.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.250+0000 7f0cff770740 1 -- 172.21.15.83:0/1596665754 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5631b4450350 msgr2=0x5631b4457ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:02.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.250+0000 7f0cff770740 1 --2- 172.21.15.83:0/1596665754 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5631b4450350 0x5631b4457ec0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f0ce402a990 tx=0x7f0ce4005270 comp rx=0 tx=0).stop 2024-04-24T09:11:02.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.250+0000 7f0cff770740 1 -- 172.21.15.83:0/1596665754 shutdown_connections 2024-04-24T09:11:02.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.250+0000 7f0cff770740 1 --2- 172.21.15.83:0/1596665754 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5631b446ad40 0x5631b446d1e0 crc :-1 s=READY pgs=487 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:02.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.250+0000 7f0cff770740 1 --2- 172.21.15.83:0/1596665754 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631b445aa20 0x5631b4458400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:02.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.250+0000 7f0cff770740 1 --2- 172.21.15.83:0/1596665754 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5631b4450350 0x5631b4457ec0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:02.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.250+0000 7f0cff770740 1 --2- 172.21.15.83:0/1596665754 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631b444fa30 0x5631b4457980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:02.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.250+0000 7f0cd7400640 0 client.39437 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:11:02.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.250+0000 7f0cff770740 1 -- 172.21.15.83:0/1596665754 >> 172.21.15.83:0/1596665754 conn(0x5631b444bd50 msgr2=0x5631b444afd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:02.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.250+0000 7f0cff770740 1 -- 172.21.15.83:0/1596665754 shutdown_connections 2024-04-24T09:11:02.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.250+0000 7f0cff770740 1 -- 172.21.15.83:0/1596665754 wait complete. 2024-04-24T09:11:02.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.251+0000 7f0cff770740 20 client.39437 trim_cache size 0 max 16384 2024-04-24T09:11:02.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.285+0000 7f0cff770740 1 Processor -- start 2024-04-24T09:11:02.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.285+0000 7f0cff770740 1 -- start start 2024-04-24T09:11:02.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.285+0000 7f0cff770740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631b444fa30 0x5631b44666d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:02.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.285+0000 7f0cff770740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5631b4450350 0x5631b4452190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:02.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.285+0000 7f0cff770740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631b445aa20 0x5631b444ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:02.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.285+0000 7f0cff770740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf81b5ae0 con 0x5631b444fa30 2024-04-24T09:11:02.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.285+0000 7f0cff770740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf81b7620 con 0x5631b4450350 2024-04-24T09:11:02.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.285+0000 7f0cff770740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf81b61b0 con 0x5631b445aa20 2024-04-24T09:11:02.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.286+0000 7f0cf5e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631b444fa30 0x5631b44666d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:02.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.286+0000 7f0cf5400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5631b4450350 0x5631b4452190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:02.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.286+0000 7f0cf5e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631b444fa30 0x5631b44666d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58658/0 (socket says 172.21.15.83:58658) 2024-04-24T09:11:02.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.286+0000 7f0cf5e00640 1 -- 172.21.15.83:0/2480055495 learned_addr learned my addr 172.21.15.83:0/2480055495 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:11:02.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.286+0000 7f0cf6800640 1 --2- 172.21.15.83:0/2480055495 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631b445aa20 0x5631b444ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:02.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.286+0000 7f0cf5e00640 1 -- 172.21.15.83:0/2480055495 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631b445aa20 msgr2=0x5631b444ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:02.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.286+0000 7f0cf5e00640 1 --2- 172.21.15.83:0/2480055495 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631b445aa20 0x5631b444ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:02.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.286+0000 7f0cf5e00640 1 -- 172.21.15.83:0/2480055495 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5631b4450350 msgr2=0x5631b4452190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:02.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.286+0000 7f0cf5e00640 1 --2- 172.21.15.83:0/2480055495 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5631b4450350 0x5631b4452190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:02.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.287+0000 7f0cf5e00640 1 -- 172.21.15.83:0/2480055495 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631b40c73f0 con 0x5631b444fa30 2024-04-24T09:11:02.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.287+0000 7f0cf6800640 1 --2- 172.21.15.83:0/2480055495 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631b445aa20 0x5631b444ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:11:02.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.287+0000 7f0cf5e00640 1 --2- 172.21.15.83:0/2480055495 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631b444fa30 0x5631b44666d0 secure :-1 s=READY pgs=2483 cs=0 l=1 rev1=1 crypto rx=0x7f0cf009cad0 tx=0x7f0cf00bd040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b35044026803b558 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:02.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.287+0000 7f0ce2600640 1 -- 172.21.15.83:0/2480055495 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf00044d0 con 0x5631b444fa30 2024-04-24T09:11:02.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.287+0000 7f0ce2600640 1 -- 172.21.15.83:0/2480055495 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0cf009d4d0 con 0x5631b444fa30 2024-04-24T09:11:02.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.287+0000 7f0ce2600640 1 -- 172.21.15.83:0/2480055495 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf00046d0 con 0x5631b444fa30 2024-04-24T09:11:02.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.288+0000 7f0cff770740 1 -- 172.21.15.83:0/2480055495 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631b444fa30 msgr2=0x5631b44666d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:02.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.288+0000 7f0cff770740 1 --2- 172.21.15.83:0/2480055495 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631b444fa30 0x5631b44666d0 secure :-1 s=READY pgs=2483 cs=0 l=1 rev1=1 crypto rx=0x7f0cf009cad0 tx=0x7f0cf00bd040 comp rx=0 tx=0).stop 2024-04-24T09:11:02.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.289+0000 7f0cff770740 1 -- 172.21.15.83:0/2480055495 shutdown_connections 2024-04-24T09:11:02.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.289+0000 7f0cff770740 1 --2- 172.21.15.83:0/2480055495 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631b445aa20 0x5631b444ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:02.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.289+0000 7f0cff770740 1 --2- 172.21.15.83:0/2480055495 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5631b4450350 0x5631b4452190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:02.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.289+0000 7f0cff770740 1 --2- 172.21.15.83:0/2480055495 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631b444fa30 0x5631b44666d0 unknown :-1 s=CLOSED pgs=2483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:02.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.289+0000 7f0cff770740 1 -- 172.21.15.83:0/2480055495 >> 172.21.15.83:0/2480055495 conn(0x5631b444bd50 msgr2=0x5631b444a740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:02.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.289+0000 7f0cff770740 1 -- 172.21.15.83:0/2480055495 shutdown_connections 2024-04-24T09:11:02.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.289+0000 7f0cff770740 1 -- 172.21.15.83:0/2480055495 wait complete. 2024-04-24T09:11:02.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.290+0000 7f0cff770740 1 -- start start 2024-04-24T09:11:02.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.290+0000 7f0cff770740 1 Processor -- start 2024-04-24T09:11:02.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.290+0000 7f0cff770740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631b444fa30 0x5631b4462fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:02.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.290+0000 7f0cff770740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5631b4450350 0x5631b445e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:02.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.290+0000 7f0cff770740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631b445aa20 0x5631b445e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:02.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.291+0000 7f0cff770740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf81b6f50 con 0x5631b444fa30 2024-04-24T09:11:02.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.291+0000 7f0cff770740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf81b6880 con 0x5631b4450350 2024-04-24T09:11:02.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.291+0000 7f0cff770740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf81b3ae0 con 0x5631b445aa20 2024-04-24T09:11:02.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.291+0000 7f0cff770740 10 client.0 _ll_register_callbacks cb 0x7f0ce4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:11:02.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.291+0000 7f0cf5400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5631b4450350 0x5631b445e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:02.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.291+0000 7f0cf5400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5631b4450350 0x5631b445e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:41994/0 (socket says 172.21.15.83:41994) 2024-04-24T09:11:02.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.291+0000 7f0cf5400640 1 -- 172.21.15.83:0/2253524409 learned_addr learned my addr 172.21.15.83:0/2253524409 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:11:02.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.291+0000 7f0cf5e00640 1 --2- 172.21.15.83:0/2253524409 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631b444fa30 0x5631b4462fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:02.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.291+0000 7f0cf6800640 1 --2- 172.21.15.83:0/2253524409 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631b445aa20 0x5631b445e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:02.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.291+0000 7f0cf5400640 1 -- 172.21.15.83:0/2253524409 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631b445aa20 msgr2=0x5631b445e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:02.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.291+0000 7f0cf5400640 1 --2- 172.21.15.83:0/2253524409 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631b445aa20 0x5631b445e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:02.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.291+0000 7f0cf5400640 1 -- 172.21.15.83:0/2253524409 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631b444fa30 msgr2=0x5631b4462fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:02.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.292+0000 7f0cf5400640 1 --2- 172.21.15.83:0/2253524409 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631b444fa30 0x5631b4462fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:02.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.292+0000 7f0cf5400640 1 -- 172.21.15.83:0/2253524409 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631b40870f0 con 0x5631b4450350 2024-04-24T09:11:02.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.292+0000 7f0cf5e00640 1 --2- 172.21.15.83:0/2253524409 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631b444fa30 0x5631b4462fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:11:02.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.292+0000 7f0cf5400640 1 --2- 172.21.15.83:0/2253524409 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5631b4450350 0x5631b445e310 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f0ce4006010 tx=0x7f0ce4004610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57913c6521e4c643 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:02.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.292+0000 7f0cd7400640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:11:02.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.292+0000 7f0cd7400640 1 -- 172.21.15.83:0/2253524409 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce4035040 con 0x5631b4450350 2024-04-24T09:11:02.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.292+0000 7f0cd7400640 1 -- 172.21.15.83:0/2253524409 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(120..120 src has 1..120) ==== 9583+0+0 (secure 0 0 0) 0x7f0ce4031a60 con 0x5631b4450350 2024-04-24T09:11:02.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.293+0000 7f0cff770740 1 -- 172.21.15.83:0/2253524409 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0cf009d4d0 con 0x5631b4450350 2024-04-24T09:11:02.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.293+0000 7f0cd7400640 1 -- 172.21.15.83:0/2253524409 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f0ce405f050 con 0x5631b4450350 2024-04-24T09:11:02.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.293+0000 7f0cff770740 10 client.39443 fetch_fsmap learned FSMap version 61 2024-04-24T09:11:02.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.293+0000 7f0cff770740 1 -- 172.21.15.83:0/2253524409 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x5631b437a810 con 0x5631b4450350 2024-04-24T09:11:02.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.294+0000 7f0cd7400640 1 -- 172.21.15.83:0/2253524409 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0cf009d4d0 con 0x5631b4450350 2024-04-24T09:11:02.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.294+0000 7f0cd7400640 1 -- 172.21.15.83:0/2253524409 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 61) ==== 8362+0+0 (secure 0 0 0) 0x7f0ce405a030 con 0x5631b4450350 2024-04-24T09:11:02.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.294+0000 7f0cff770740 10 client.39443 fetch_fsmap finished waiting for FSMap version 61 2024-04-24T09:11:02.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.294+0000 7f0cff770740 10 client.39443 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:11:02.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.294+0000 7f0cff770740 20 client.39443 populate_metadata read hostname 'smithi083' 2024-04-24T09:11:02.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.295+0000 7f0cff770740 1 --2- 172.21.15.83:0/2253524409 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5631b44d1750 0x5631b44d3b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:02.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.295+0000 7f0cff770740 4 client.39443 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T09:11:02.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.295+0000 7f0cff770740 1 -- 172.21.15.83:0/2253524409 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5631b43bca80 con 0x5631b44d1750 2024-04-24T09:11:02.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.296+0000 7f0cf5e00640 1 --2- 172.21.15.83:0/2253524409 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5631b44d1750 0x5631b44d3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:02.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.297+0000 7f0cd7400640 10 client.39443 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:11:02.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.297+0000 7f0cf5e00640 1 --2- 172.21.15.83:0/2253524409 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5631b44d1750 0x5631b44d3b70 crc :-1 s=READY pgs=488 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84e1096910dfc732 server_cookie=4d28499b6b512f31 in_seq=0 out_seq=0 2024-04-24T09:11:02.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.298+0000 7f0cd7400640 1 -- 172.21.15.83:0/2253524409 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5631b43bca80 con 0x5631b44d1750 2024-04-24T09:11:02.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.298+0000 7f0cd7400640 10 client.39443 handle_command_reply: tid=0 2024-04-24T09:11:02.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.298+0000 7f0cff770740 1 client.39443 shutdown 2024-04-24T09:11:02.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.298+0000 7f0cff770740 1 -- 172.21.15.83:0/2253524409 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5631b4450350 msgr2=0x5631b445e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:02.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.298+0000 7f0cff770740 1 --2- 172.21.15.83:0/2253524409 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5631b4450350 0x5631b445e310 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f0ce4006010 tx=0x7f0ce4004610 comp rx=0 tx=0).stop 2024-04-24T09:11:02.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.298+0000 7f0cff770740 1 -- 172.21.15.83:0/2253524409 shutdown_connections 2024-04-24T09:11:02.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.298+0000 7f0cff770740 1 --2- 172.21.15.83:0/2253524409 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5631b44d1750 0x5631b44d3b70 crc :-1 s=READY pgs=488 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:02.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.299+0000 7f0cff770740 1 --2- 172.21.15.83:0/2253524409 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5631b445aa20 0x5631b445e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:02.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.299+0000 7f0cff770740 1 --2- 172.21.15.83:0/2253524409 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5631b4450350 0x5631b445e310 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:02.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.299+0000 7f0cff770740 1 --2- 172.21.15.83:0/2253524409 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631b444fa30 0x5631b4462fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:02.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.299+0000 7f0cff770740 1 -- 172.21.15.83:0/2253524409 >> 172.21.15.83:0/2253524409 conn(0x5631b444bd50 msgr2=0x5631b44581b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:02.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.299+0000 7f0cd7400640 0 client.39443 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:11:02.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.299+0000 7f0cff770740 1 -- 172.21.15.83:0/2253524409 shutdown_connections 2024-04-24T09:11:02.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.299+0000 7f0cff770740 1 -- 172.21.15.83:0/2253524409 wait complete. 2024-04-24T09:11:02.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.299+0000 7f0cff770740 20 client.39443 trim_cache size 0 max 16384 2024-04-24T09:11:02.301 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T09:11:02.301 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T09:11:02.301 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-04-24T09:11:02.301 INFO:teuthology.orchestra.run.smithi083.stdout: "b7b5b5bd-8add-404d-a716-4fe458597c16": { 2024-04-24T09:11:02.301 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-04-24T09:11:02.301 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "b7b5b5bd-8add-404d-a716-4fe458597c16", 2024-04-24T09:11:02.301 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-04-24T09:11:02.301 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:11:02.301 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:11:02.301 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T09:11:02.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.300+0000 7f0cf7200640 1 -- 172.21.15.83:0/1862859135 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f0cdc0a3e60 msgr2=0x7f0cdc0a6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:02.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.300+0000 7f0cf7200640 1 --2- 172.21.15.83:0/1862859135 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f0cdc0a3e60 0x7f0cdc0a6320 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f0ce8005fd0 tx=0x7f0ce8005cc0 comp rx=0 tx=0).stop 2024-04-24T09:11:02.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.300+0000 7f0cf7200640 1 -- 172.21.15.83:0/1862859135 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0cf8076530 msgr2=0x7f0cf819ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:02.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.300+0000 7f0cf7200640 1 --2- 172.21.15.83:0/1862859135 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0cf8076530 0x7f0cf819ed40 secure :-1 s=READY pgs=2482 cs=0 l=1 rev1=1 crypto rx=0x7f0cf000eb90 tx=0x7f0cf000c6a0 comp rx=0 tx=0).stop 2024-04-24T09:11:02.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.300+0000 7f0cf7200640 1 -- 172.21.15.83:0/1862859135 shutdown_connections 2024-04-24T09:11:02.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.300+0000 7f0cf7200640 1 --2- 172.21.15.83:0/1862859135 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f0cdc0a3e60 0x7f0cdc0a6320 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:02.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.300+0000 7f0cf7200640 1 --2- 172.21.15.83:0/1862859135 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0cf8112d50 0x7f0cf81a3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:02.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.300+0000 7f0cf7200640 1 --2- 172.21.15.83:0/1862859135 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0cf8076e50 0x7f0cf819f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:02.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.300+0000 7f0cf7200640 1 --2- 172.21.15.83:0/1862859135 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0cf8076530 0x7f0cf819ed40 unknown :-1 s=CLOSED pgs=2482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:02.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.300+0000 7f0cf7200640 1 -- 172.21.15.83:0/1862859135 >> 172.21.15.83:0/1862859135 conn(0x7f0cf80ffb40 msgr2=0x7f0cf8101f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:02.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.301+0000 7f0cf7200640 1 -- 172.21.15.83:0/1862859135 shutdown_connections 2024-04-24T09:11:02.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:02.301+0000 7f0cf7200640 1 -- 172.21.15.83:0/1862859135 wait complete. 2024-04-24T09:11:02.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:11:02 smithi083 ceph-mon[27907]: pgmap v611: 129 pgs: 129 active+clean; 681 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 222 op/s 2024-04-24T09:11:02.322 INFO:tasks.cephfs.filesystem:scrub status for tag:b7b5b5bd-8add-404d-a716-4fe458597c16 - {'path': '/', 'tag': 'b7b5b5bd-8add-404d-a716-4fe458597c16', 'options': 'recursive,force'} 2024-04-24T09:11:02.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:11:02 smithi138 ceph-mon[31441]: pgmap v611: 129 pgs: 129 active+clean; 681 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 222 op/s 2024-04-24T09:11:02.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:11:02 smithi189 ceph-mon[31791]: pgmap v611: 129 pgs: 129 active+clean; 681 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 222 op/s 2024-04-24T09:11:03.178 INFO:tasks.workunit.client.0.smithi083.stdout: 10 42170 6.73 MB/sec execute 214 sec latency 93.770 ms 2024-04-24T09:11:03.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:11:03 smithi083 ceph-mon[27907]: scrub queued for path: / 2024-04-24T09:11:03.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:11:03 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:11:03.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:11:03 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:11:03.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:11:03 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:11:03.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:11:03 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:11:03.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:11:03 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:11:03.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:11:03 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:11:03.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:11:03 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:11:03.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:11:03 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:11:03.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:11:03 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:11:03.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:11:03 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:11:03.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:11:03 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:11:03.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:11:03 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:11:03.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:11:03 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:11:03.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:11:03 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:11:03.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:11:03 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:11:03.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:11:03 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:11:03.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:11:03 smithi138 ceph-mon[31441]: scrub queued for path: / 2024-04-24T09:11:03.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:11:03 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:11:03.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:11:03 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:11:03.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:11:03 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:11:03.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:11:03 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:11:03.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:11:03 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:11:03.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:11:03 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:11:03.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:11:03 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:11:03.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:11:03 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:11:03.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:11:03 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:11:03.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:11:03 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:11:03.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:11:03 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:11:03.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:11:03 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:11:03.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:11:03 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:11:03.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:11:03 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:11:03.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:11:03 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:11:03.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:11:03 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:11:03.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:11:03 smithi189 ceph-mon[31791]: scrub queued for path: / 2024-04-24T09:11:03.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:11:03 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:11:03.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:11:03 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:11:03.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:11:03 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:11:03.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:11:03 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:11:03.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:11:03 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:11:03.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:11:03 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:11:03.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:11:03 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:11:03.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:11:03 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:11:03.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:11:03 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:11:03.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:11:03 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:11:03.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:11:03 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:11:03.459 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:11:03 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:11:03.459 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:11:03 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:11:03.459 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:11:03 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:11:03.459 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:11:03 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:11:03.459 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:11:03 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:11:04.178 INFO:tasks.workunit.client.0.smithi083.stdout: 10 42275 6.74 MB/sec execute 215 sec latency 232.259 ms 2024-04-24T09:11:04.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:11:04 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:11:04.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:11:04 smithi083 ceph-mon[27907]: pgmap v612: 129 pgs: 129 active+clean; 699 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.1 MiB/s wr, 362 op/s 2024-04-24T09:11:04.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:11:04 smithi083 ceph-mon[27907]: osdmap e121: 12 total, 12 up, 12 in 2024-04-24T09:11:04.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:11:04 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:11:04.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:11:04 smithi138 ceph-mon[31441]: pgmap v612: 129 pgs: 129 active+clean; 699 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.1 MiB/s wr, 362 op/s 2024-04-24T09:11:04.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:11:04 smithi138 ceph-mon[31441]: osdmap e121: 12 total, 12 up, 12 in 2024-04-24T09:11:04.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:11:04 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:11:04.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:11:04 smithi189 ceph-mon[31791]: pgmap v612: 129 pgs: 129 active+clean; 699 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.1 MiB/s wr, 362 op/s 2024-04-24T09:11:04.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:11:04 smithi189 ceph-mon[31791]: osdmap e121: 12 total, 12 up, 12 in 2024-04-24T09:11:05.178 INFO:tasks.workunit.client.0.smithi083.stdout: 10 42371 6.73 MB/sec execute 216 sec latency 363.142 ms 2024-04-24T09:11:06.047 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.047+0000 7fa454c00640 1 -- 172.21.15.83:0/2726970795 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa450109370 msgr2=0x7fa450110f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:06.048 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.047+0000 7fa454c00640 1 --2- 172.21.15.83:0/2726970795 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa450109370 0x7fa450110f40 secure :-1 s=READY pgs=2484 cs=0 l=1 rev1=1 crypto rx=0x7fa448009ca0 tx=0x7fa44802a8c0 comp rx=0 tx=0).stop 2024-04-24T09:11:06.048 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.047+0000 7fa454c00640 1 -- 172.21.15.83:0/2726970795 shutdown_connections 2024-04-24T09:11:06.048 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.047+0000 7fa454c00640 1 --2- 172.21.15.83:0/2726970795 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa450109370 0x7fa450110f40 unknown :-1 s=CLOSED pgs=2484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:06.048 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.047+0000 7fa454c00640 1 --2- 172.21.15.83:0/2726970795 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa450104c50 0x7fa450108c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:06.048 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.047+0000 7fa454c00640 1 --2- 172.21.15.83:0/2726970795 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4501042a0 0x7fa450104680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:06.048 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.047+0000 7fa454c00640 1 -- 172.21.15.83:0/2726970795 >> 172.21.15.83:0/2726970795 conn(0x7fa4500ffae0 msgr2=0x7fa450101f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:06.048 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.047+0000 7fa454c00640 1 -- 172.21.15.83:0/2726970795 shutdown_connections 2024-04-24T09:11:06.048 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.047+0000 7fa454c00640 1 -- 172.21.15.83:0/2726970795 wait complete. 2024-04-24T09:11:06.051 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.048+0000 7fa454c00640 1 Processor -- start 2024-04-24T09:11:06.051 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.048+0000 7fa454c00640 1 -- start start 2024-04-24T09:11:06.051 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.048+0000 7fa454c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4501042a0 0x7fa4501a05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:06.051 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.048+0000 7fa454c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa450109370 0x7fa4501a0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:06.051 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.048+0000 7fa454c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa45019a6e0 0x7fa45019ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:06.051 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.048+0000 7fa454c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa450114380 con 0x7fa45019a6e0 2024-04-24T09:11:06.051 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.048+0000 7fa454c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa450114200 con 0x7fa4501042a0 2024-04-24T09:11:06.052 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.048+0000 7fa454c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa450114500 con 0x7fa450109370 2024-04-24T09:11:06.052 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.049+0000 7fa44fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa45019a6e0 0x7fa45019ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:06.052 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.049+0000 7fa44fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa45019a6e0 0x7fa45019ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58688/0 (socket says 172.21.15.83:58688) 2024-04-24T09:11:06.052 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.049+0000 7fa44fe00640 1 -- 172.21.15.83:0/1848473100 learned_addr learned my addr 172.21.15.83:0/1848473100 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:11:06.052 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.049+0000 7fa44f400640 1 --2- 172.21.15.83:0/1848473100 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4501042a0 0x7fa4501a05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:06.052 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.049+0000 7fa44ea00640 1 --2- 172.21.15.83:0/1848473100 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa450109370 0x7fa4501a0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:06.052 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.049+0000 7fa44fe00640 1 -- 172.21.15.83:0/1848473100 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa450109370 msgr2=0x7fa4501a0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:06.052 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.049+0000 7fa44fe00640 1 --2- 172.21.15.83:0/1848473100 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa450109370 0x7fa4501a0b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:06.052 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.049+0000 7fa44fe00640 1 -- 172.21.15.83:0/1848473100 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4501042a0 msgr2=0x7fa4501a05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:06.052 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.049+0000 7fa44fe00640 1 --2- 172.21.15.83:0/1848473100 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4501042a0 0x7fa4501a05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:06.052 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.049+0000 7fa44fe00640 1 -- 172.21.15.83:0/1848473100 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa45019b2e0 con 0x7fa45019a6e0 2024-04-24T09:11:06.052 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.049+0000 7fa44fe00640 1 --2- 172.21.15.83:0/1848473100 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa45019a6e0 0x7fa45019ab60 secure :-1 s=READY pgs=2485 cs=0 l=1 rev1=1 crypto rx=0x7fa448031040 tx=0x7fa448003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:06.053 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.050+0000 7fa447e00640 1 -- 172.21.15.83:0/1848473100 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa448003ce0 con 0x7fa45019a6e0 2024-04-24T09:11:06.053 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.050+0000 7fa447e00640 1 -- 172.21.15.83:0/1848473100 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa4480044f0 con 0x7fa45019a6e0 2024-04-24T09:11:06.053 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.050+0000 7fa447e00640 1 -- 172.21.15.83:0/1848473100 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa448030c90 con 0x7fa45019a6e0 2024-04-24T09:11:06.053 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.050+0000 7fa454c00640 1 -- 172.21.15.83:0/1848473100 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa45019b570 con 0x7fa45019a6e0 2024-04-24T09:11:06.053 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.050+0000 7fa454c00640 1 -- 172.21.15.83:0/1848473100 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4501a76c0 con 0x7fa45019a6e0 2024-04-24T09:11:06.056 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.053+0000 7fa45900f740 1 Processor -- start 2024-04-24T09:11:06.057 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.053+0000 7fa447e00640 1 -- 172.21.15.83:0/1848473100 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa4480046a0 con 0x7fa45019a6e0 2024-04-24T09:11:06.057 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.053+0000 7fa45900f740 1 -- start start 2024-04-24T09:11:06.057 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.053+0000 7fa45900f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56112e9a53c0 0x56112e9a57a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:06.057 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.053+0000 7fa45900f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56112e9a5ce0 0x56112e9b0060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:06.057 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.053+0000 7fa45900f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56112e9b05a0 0x56112e9b2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:06.057 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.053+0000 7fa45900f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56112e90b900 con 0x56112e9a5ce0 2024-04-24T09:11:06.057 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.053+0000 7fa45900f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56112e8b1f20 con 0x56112e9a53c0 2024-04-24T09:11:06.057 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.053+0000 7fa45900f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56112e9078c0 con 0x56112e9b05a0 2024-04-24T09:11:06.057 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.054+0000 7fa44f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56112e9a5ce0 0x56112e9b0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:06.057 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.054+0000 7fa44f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56112e9a5ce0 0x56112e9b0060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58700/0 (socket says 172.21.15.83:58700) 2024-04-24T09:11:06.057 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.054+0000 7fa44f400640 1 -- 172.21.15.83:0/2697593098 learned_addr learned my addr 172.21.15.83:0/2697593098 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:11:06.057 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.054+0000 7fa44ea00640 1 --2- 172.21.15.83:0/2697593098 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56112e9a53c0 0x56112e9a57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:06.058 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.054+0000 7fa44ea00640 1 --2- 172.21.15.83:0/2697593098 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56112e9b05a0 0x56112e9b2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:06.058 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.054+0000 7fa44f400640 1 -- 172.21.15.83:0/2697593098 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56112e9b05a0 msgr2=0x56112e9b2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:06.058 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.054+0000 7fa44f400640 1 --2- 172.21.15.83:0/2697593098 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56112e9b05a0 0x56112e9b2990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:06.058 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.054+0000 7fa44f400640 1 -- 172.21.15.83:0/2697593098 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56112e9a53c0 msgr2=0x56112e9a57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:06.058 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.054+0000 7fa44f400640 1 --2- 172.21.15.83:0/2697593098 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56112e9a53c0 0x56112e9a57a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:06.058 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.054+0000 7fa44f400640 1 -- 172.21.15.83:0/2697593098 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56112e8d6e10 con 0x56112e9a5ce0 2024-04-24T09:11:06.058 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.054+0000 7fa44f400640 1 --2- 172.21.15.83:0/2697593098 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56112e9a5ce0 0x56112e9b0060 secure :-1 s=READY pgs=2486 cs=0 l=1 rev1=1 crypto rx=0x7fa44000bd10 tx=0x7fa4400107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16a752dfabc6eda4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:06.058 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.055+0000 7fa447e00640 1 --2- 172.21.15.83:0/1848473100 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa4340a3d10 0x7fa4340a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:06.058 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.055+0000 7fa43be00640 1 -- 172.21.15.83:0/2697593098 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa440004030 con 0x56112e9a5ce0 2024-04-24T09:11:06.058 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.055+0000 7fa43be00640 1 -- 172.21.15.83:0/2697593098 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa4400035f0 con 0x56112e9a5ce0 2024-04-24T09:11:06.058 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.055+0000 7fa43be00640 1 -- 172.21.15.83:0/2697593098 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4400117b0 con 0x56112e9a5ce0 2024-04-24T09:11:06.058 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.055+0000 7fa447e00640 1 -- 172.21.15.83:0/1848473100 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 9671+0+0 (secure 0 0 0) 0x7fa4480f2240 con 0x7fa45019a6e0 2024-04-24T09:11:06.058 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.055+0000 7fa44ea00640 1 --2- 172.21.15.83:0/1848473100 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa4340a3d10 0x7fa4340a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:06.059 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.056+0000 7fa45900f740 1 -- 172.21.15.83:0/2697593098 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56112e9a5ce0 msgr2=0x56112e9b0060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:06.059 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.056+0000 7fa45900f740 1 --2- 172.21.15.83:0/2697593098 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56112e9a5ce0 0x56112e9b0060 secure :-1 s=READY pgs=2486 cs=0 l=1 rev1=1 crypto rx=0x7fa44000bd10 tx=0x7fa4400107a0 comp rx=0 tx=0).stop 2024-04-24T09:11:06.059 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.056+0000 7fa44ea00640 1 --2- 172.21.15.83:0/1848473100 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa4340a3d10 0x7fa4340a61d0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x56112e9a2cf0 tx=0x7fa43c008040 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:06.059 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.056+0000 7fa45900f740 1 -- 172.21.15.83:0/2697593098 shutdown_connections 2024-04-24T09:11:06.059 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.056+0000 7fa45900f740 1 --2- 172.21.15.83:0/2697593098 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56112e9b05a0 0x56112e9b2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:06.059 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.056+0000 7fa45900f740 1 --2- 172.21.15.83:0/2697593098 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56112e9a5ce0 0x56112e9b0060 unknown :-1 s=CLOSED pgs=2486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:06.059 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.056+0000 7fa45900f740 1 --2- 172.21.15.83:0/2697593098 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56112e9a53c0 0x56112e9a57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:06.059 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.056+0000 7fa45900f740 1 -- 172.21.15.83:0/2697593098 >> 172.21.15.83:0/2697593098 conn(0x56112e9a18d0 msgr2=0x56112e9a1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:06.059 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.056+0000 7fa45900f740 1 -- 172.21.15.83:0/2697593098 shutdown_connections 2024-04-24T09:11:06.059 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.056+0000 7fa45900f740 1 -- 172.21.15.83:0/2697593098 wait complete. 2024-04-24T09:11:06.059 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.056+0000 7fa45900f740 1 -- start start 2024-04-24T09:11:06.059 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.056+0000 7fa45900f740 1 Processor -- start 2024-04-24T09:11:06.063 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.057+0000 7fa45900f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56112e9a53c0 0x56112e9ad650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:06.063 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.057+0000 7fa45900f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56112e9a5ce0 0x56112e9adb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:06.064 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.057+0000 7fa45900f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56112e9b05a0 0x56112e9ae0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:06.064 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.057+0000 7fa45900f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56112e9b3140 con 0x56112e9a5ce0 2024-04-24T09:11:06.064 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.057+0000 7fa45900f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56112e8c52c0 con 0x56112e9a53c0 2024-04-24T09:11:06.064 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.057+0000 7fa45900f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56112e9b32c0 con 0x56112e9b05a0 2024-04-24T09:11:06.064 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.057+0000 7fa45900f740 10 client.0 _ll_register_callbacks cb 0x56112e61da00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:11:06.064 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.057+0000 7fa44ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56112e9a5ce0 0x56112e9adb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:06.064 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.057+0000 7fa44ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56112e9a5ce0 0x56112e9adb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58714/0 (socket says 172.21.15.83:58714) 2024-04-24T09:11:06.064 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.057+0000 7fa44ea00640 1 -- 172.21.15.83:0/975338109 learned_addr learned my addr 172.21.15.83:0/975338109 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:11:06.064 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.057+0000 7fa44f400640 1 --2- 172.21.15.83:0/975338109 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56112e9a53c0 0x56112e9ad650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:06.064 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.058+0000 7fa44fe00640 1 --2- 172.21.15.83:0/975338109 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56112e9b05a0 0x56112e9ae0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:06.065 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.058+0000 7fa44ea00640 1 -- 172.21.15.83:0/975338109 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56112e9b05a0 msgr2=0x56112e9ae0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:06.065 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.058+0000 7fa44ea00640 1 --2- 172.21.15.83:0/975338109 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56112e9b05a0 0x56112e9ae0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:06.065 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.058+0000 7fa44ea00640 1 -- 172.21.15.83:0/975338109 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56112e9a53c0 msgr2=0x56112e9ad650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:06.065 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.058+0000 7fa44ea00640 1 --2- 172.21.15.83:0/975338109 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56112e9a53c0 0x56112e9ad650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:06.065 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.058+0000 7fa44ea00640 1 -- 172.21.15.83:0/975338109 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56112e8d5fd0 con 0x56112e9a5ce0 2024-04-24T09:11:06.065 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.058+0000 7fa44ea00640 1 --2- 172.21.15.83:0/975338109 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56112e9a5ce0 0x56112e9adb90 secure :-1 s=READY pgs=2487 cs=0 l=1 rev1=1 crypto rx=0x7fa43c010f80 tx=0x7fa43c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=532cc7baf527c8f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:06.065 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.058+0000 7fa438c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:11:06.065 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.058+0000 7fa438c00640 1 -- 172.21.15.83:0/975338109 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa43c024070 con 0x56112e9a5ce0 2024-04-24T09:11:06.065 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.058+0000 7fa438c00640 1 -- 172.21.15.83:0/975338109 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9671+0+0 (secure 0 0 0) 0x7fa43c016810 con 0x56112e9a5ce0 2024-04-24T09:11:06.065 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.058+0000 7fa45900f740 1 -- 172.21.15.83:0/975338109 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4400035f0 con 0x56112e9a5ce0 2024-04-24T09:11:06.066 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.058+0000 7fa438c00640 1 -- 172.21.15.83:0/975338109 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7fa43c030050 con 0x56112e9a5ce0 2024-04-24T09:11:06.066 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.058+0000 7fa45900f740 10 client.22371 fetch_fsmap learned FSMap version 61 2024-04-24T09:11:06.066 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.058+0000 7fa45900f740 1 -- 172.21.15.83:0/975338109 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x56112e61e210 con 0x56112e9a5ce0 2024-04-24T09:11:06.066 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.059+0000 7fa438c00640 1 -- 172.21.15.83:0/975338109 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa4400035f0 con 0x56112e9a5ce0 2024-04-24T09:11:06.066 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.059+0000 7fa438c00640 1 -- 172.21.15.83:0/975338109 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 61) ==== 8362+0+0 (secure 0 0 0) 0x7fa43c01f070 con 0x56112e9a5ce0 2024-04-24T09:11:06.066 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.059+0000 7fa45900f740 10 client.22371 fetch_fsmap finished waiting for FSMap version 61 2024-04-24T09:11:06.066 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.059+0000 7fa45900f740 10 client.22371 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:11:06.066 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.059+0000 7fa45900f740 20 client.22371 populate_metadata read hostname 'smithi083' 2024-04-24T09:11:06.066 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.059+0000 7fa45900f740 1 --2- 172.21.15.83:0/975338109 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56112e9c0840 0x56112e9c2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:06.066 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.059+0000 7fa45900f740 4 client.22371 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:11:06.066 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.059+0000 7fa45900f740 1 -- 172.21.15.83:0/975338109 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56112e919320 con 0x56112e9c0840 2024-04-24T09:11:06.067 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.060+0000 7fa44f400640 1 --2- 172.21.15.83:0/975338109 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56112e9c0840 0x56112e9c2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:06.067 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.060+0000 7fa44f400640 1 --2- 172.21.15.83:0/975338109 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56112e9c0840 0x56112e9c2cb0 crc :-1 s=READY pgs=489 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f1d9641a2729d3e server_cookie=e27fa77b4b9e1505 in_seq=0 out_seq=0 2024-04-24T09:11:06.067 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.060+0000 7fa438c00640 10 client.22371 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:11:06.067 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.062+0000 7fa438c00640 1 -- 172.21.15.83:0/975338109 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56112e919320 con 0x56112e9c0840 2024-04-24T09:11:06.067 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.062+0000 7fa438c00640 10 client.22371 handle_command_reply: tid=0 2024-04-24T09:11:06.067 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.062+0000 7fa45900f740 1 client.22371 shutdown 2024-04-24T09:11:06.067 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.063+0000 7fa45900f740 1 -- 172.21.15.83:0/975338109 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56112e9a5ce0 msgr2=0x56112e9adb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:06.067 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.063+0000 7fa45900f740 1 --2- 172.21.15.83:0/975338109 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56112e9a5ce0 0x56112e9adb90 secure :-1 s=READY pgs=2487 cs=0 l=1 rev1=1 crypto rx=0x7fa43c010f80 tx=0x7fa43c0046c0 comp rx=0 tx=0).stop 2024-04-24T09:11:06.067 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.063+0000 7fa45900f740 1 -- 172.21.15.83:0/975338109 shutdown_connections 2024-04-24T09:11:06.067 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.063+0000 7fa45900f740 1 --2- 172.21.15.83:0/975338109 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56112e9c0840 0x56112e9c2cb0 crc :-1 s=READY pgs=489 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:06.067 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.063+0000 7fa45900f740 1 --2- 172.21.15.83:0/975338109 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56112e9b05a0 0x56112e9ae0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:06.067 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.063+0000 7fa45900f740 1 --2- 172.21.15.83:0/975338109 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56112e9a5ce0 0x56112e9adb90 unknown :-1 s=CLOSED pgs=2487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:06.068 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.063+0000 7fa45900f740 1 --2- 172.21.15.83:0/975338109 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56112e9a53c0 0x56112e9ad650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:06.068 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.063+0000 7fa438c00640 0 client.22371 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:11:06.068 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.063+0000 7fa45900f740 1 -- 172.21.15.83:0/975338109 >> 172.21.15.83:0/975338109 conn(0x56112e9a18d0 msgr2=0x56112e9a7650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:06.068 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.063+0000 7fa45900f740 1 -- 172.21.15.83:0/975338109 shutdown_connections 2024-04-24T09:11:06.068 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.063+0000 7fa45900f740 1 -- 172.21.15.83:0/975338109 wait complete. 2024-04-24T09:11:06.068 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.063+0000 7fa45900f740 20 client.22371 trim_cache size 0 max 16384 2024-04-24T09:11:06.081 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.081+0000 7fa45900f740 1 Processor -- start 2024-04-24T09:11:06.082 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.081+0000 7fa45900f740 1 -- start start 2024-04-24T09:11:06.084 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.082+0000 7fa45900f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56112e9a53c0 0x56112e9afd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:06.084 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.082+0000 7fa45900f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56112e9a5ce0 0x56112e9bc110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:06.084 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.082+0000 7fa45900f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56112e9b05a0 0x56112e9c85c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:06.084 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.082+0000 7fa45900f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa4501b1810 con 0x56112e9a5ce0 2024-04-24T09:11:06.084 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.082+0000 7fa45900f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa4501b3350 con 0x56112e9a53c0 2024-04-24T09:11:06.084 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.082+0000 7fa45900f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa4501b1ee0 con 0x56112e9b05a0 2024-04-24T09:11:06.084 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.082+0000 7fa44ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56112e9a5ce0 0x56112e9bc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:06.084 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.082+0000 7fa44ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56112e9a5ce0 0x56112e9bc110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58726/0 (socket says 172.21.15.83:58726) 2024-04-24T09:11:06.084 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.082+0000 7fa44ea00640 1 -- 172.21.15.83:0/739666636 learned_addr learned my addr 172.21.15.83:0/739666636 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:11:06.084 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.082+0000 7fa44f400640 1 --2- 172.21.15.83:0/739666636 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56112e9a53c0 0x56112e9afd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:06.084 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.083+0000 7fa44fe00640 1 --2- 172.21.15.83:0/739666636 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56112e9b05a0 0x56112e9c85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:06.085 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.083+0000 7fa44ea00640 1 -- 172.21.15.83:0/739666636 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56112e9b05a0 msgr2=0x56112e9c85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:06.085 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.083+0000 7fa44ea00640 1 --2- 172.21.15.83:0/739666636 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56112e9b05a0 0x56112e9c85c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:06.085 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.083+0000 7fa44ea00640 1 -- 172.21.15.83:0/739666636 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56112e9a53c0 msgr2=0x56112e9afd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:06.085 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.083+0000 7fa44ea00640 1 --2- 172.21.15.83:0/739666636 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56112e9a53c0 0x56112e9afd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:06.085 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.083+0000 7fa44ea00640 1 -- 172.21.15.83:0/739666636 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56112e9b2d00 con 0x56112e9a5ce0 2024-04-24T09:11:06.085 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.083+0000 7fa44f400640 1 --2- 172.21.15.83:0/739666636 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56112e9a53c0 0x56112e9afd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:11:06.085 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.083+0000 7fa44ea00640 1 --2- 172.21.15.83:0/739666636 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56112e9a5ce0 0x56112e9bc110 secure :-1 s=READY pgs=2488 cs=0 l=1 rev1=1 crypto rx=0x7fa43c00d010 tx=0x7fa43c00c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89b73d98937f3063 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:06.085 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.084+0000 7fa43b400640 1 -- 172.21.15.83:0/739666636 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa43c024070 con 0x56112e9a5ce0 2024-04-24T09:11:06.085 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.084+0000 7fa43b400640 1 -- 172.21.15.83:0/739666636 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa43c01a070 con 0x56112e9a5ce0 2024-04-24T09:11:06.085 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.084+0000 7fa43b400640 1 -- 172.21.15.83:0/739666636 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa43c00f040 con 0x56112e9a5ce0 2024-04-24T09:11:06.086 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.085+0000 7fa45900f740 1 -- 172.21.15.83:0/739666636 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56112e9a5ce0 msgr2=0x56112e9bc110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:06.087 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.085+0000 7fa45900f740 1 --2- 172.21.15.83:0/739666636 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56112e9a5ce0 0x56112e9bc110 secure :-1 s=READY pgs=2488 cs=0 l=1 rev1=1 crypto rx=0x7fa43c00d010 tx=0x7fa43c00c820 comp rx=0 tx=0).stop 2024-04-24T09:11:06.087 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.085+0000 7fa45900f740 1 -- 172.21.15.83:0/739666636 shutdown_connections 2024-04-24T09:11:06.087 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.085+0000 7fa45900f740 1 --2- 172.21.15.83:0/739666636 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56112e9b05a0 0x56112e9c85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:06.087 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.085+0000 7fa45900f740 1 --2- 172.21.15.83:0/739666636 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56112e9a5ce0 0x56112e9bc110 unknown :-1 s=CLOSED pgs=2488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:06.087 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.085+0000 7fa45900f740 1 --2- 172.21.15.83:0/739666636 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56112e9a53c0 0x56112e9afd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:06.087 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.085+0000 7fa45900f740 1 -- 172.21.15.83:0/739666636 >> 172.21.15.83:0/739666636 conn(0x56112e9a18d0 msgr2=0x56112e9a1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:06.087 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.085+0000 7fa45900f740 1 -- 172.21.15.83:0/739666636 shutdown_connections 2024-04-24T09:11:06.087 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.085+0000 7fa45900f740 1 -- 172.21.15.83:0/739666636 wait complete. 2024-04-24T09:11:06.087 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.085+0000 7fa45900f740 1 -- start start 2024-04-24T09:11:06.087 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.086+0000 7fa45900f740 1 Processor -- start 2024-04-24T09:11:06.087 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.086+0000 7fa45900f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56112e9a53c0 0x56112e9b8620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:06.088 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.086+0000 7fa45900f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56112e9b05a0 0x56112e9b8b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:06.088 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.086+0000 7fa45900f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56112e9b55f0 0x56112e9b5a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:06.088 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.086+0000 7fa45900f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa4501b2c80 con 0x56112e9b05a0 2024-04-24T09:11:06.088 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.086+0000 7fa45900f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa4501b25b0 con 0x56112e9b55f0 2024-04-24T09:11:06.088 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.086+0000 7fa45900f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa4501af810 con 0x56112e9a53c0 2024-04-24T09:11:06.088 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.086+0000 7fa45900f740 10 client.0 _ll_register_callbacks cb 0x7fa43c010c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:11:06.089 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.087+0000 7fa44fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56112e9b55f0 0x56112e9b5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:06.089 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.087+0000 7fa44fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56112e9b55f0 0x56112e9b5a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:42040/0 (socket says 172.21.15.83:42040) 2024-04-24T09:11:06.089 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.087+0000 7fa44fe00640 1 -- 172.21.15.83:0/1121972400 learned_addr learned my addr 172.21.15.83:0/1121972400 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:11:06.089 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.087+0000 7fa44f400640 1 --2- 172.21.15.83:0/1121972400 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56112e9a53c0 0x56112e9b8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:06.089 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.088+0000 7fa44ea00640 1 --2- 172.21.15.83:0/1121972400 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56112e9b05a0 0x56112e9b8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:06.089 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.088+0000 7fa44ea00640 1 -- 172.21.15.83:0/1121972400 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56112e9a53c0 msgr2=0x56112e9b8620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:06.090 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.088+0000 7fa44ea00640 1 --2- 172.21.15.83:0/1121972400 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56112e9a53c0 0x56112e9b8620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:06.090 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.088+0000 7fa44ea00640 1 -- 172.21.15.83:0/1121972400 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56112e9b55f0 msgr2=0x56112e9b5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:06.090 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.089+0000 7fa44ea00640 1 --2- 172.21.15.83:0/1121972400 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56112e9b55f0 0x56112e9b5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:06.090 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.089+0000 7fa44ea00640 1 -- 172.21.15.83:0/1121972400 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56112e6ea620 con 0x56112e9b05a0 2024-04-24T09:11:06.090 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.089+0000 7fa44fe00640 1 --2- 172.21.15.83:0/1121972400 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56112e9b55f0 0x56112e9b5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:11:06.092 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.089+0000 7fa44ea00640 1 --2- 172.21.15.83:0/1121972400 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56112e9b05a0 0x56112e9b8b60 secure :-1 s=READY pgs=2489 cs=0 l=1 rev1=1 crypto rx=0x56112e9abe50 tx=0x7fa43c020620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5bcca9544402963 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:06.092 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.089+0000 7fa438c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:11:06.092 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.090+0000 7fa438c00640 1 -- 172.21.15.83:0/1121972400 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa43c041020 con 0x56112e9b05a0 2024-04-24T09:11:06.092 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.090+0000 7fa438c00640 1 -- 172.21.15.83:0/1121972400 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9671+0+0 (secure 0 0 0) 0x7fa43c005bc0 con 0x56112e9b05a0 2024-04-24T09:11:06.092 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.090+0000 7fa45900f740 1 -- 172.21.15.83:0/1121972400 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa43c01a070 con 0x56112e9b05a0 2024-04-24T09:11:06.092 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.090+0000 7fa438c00640 1 -- 172.21.15.83:0/1121972400 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7fa43c00f050 con 0x56112e9b05a0 2024-04-24T09:11:06.092 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.090+0000 7fa45900f740 10 client.22377 fetch_fsmap learned FSMap version 61 2024-04-24T09:11:06.092 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.090+0000 7fa45900f740 1 -- 172.21.15.83:0/1121972400 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x56112ea279f0 con 0x56112e9b05a0 2024-04-24T09:11:06.092 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.091+0000 7fa438c00640 1 -- 172.21.15.83:0/1121972400 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa43c01a070 con 0x56112e9b05a0 2024-04-24T09:11:06.092 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.091+0000 7fa438c00640 1 -- 172.21.15.83:0/1121972400 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 61) ==== 8362+0+0 (secure 0 0 0) 0x7fa43c01f040 con 0x56112e9b05a0 2024-04-24T09:11:06.092 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.091+0000 7fa45900f740 10 client.22377 fetch_fsmap finished waiting for FSMap version 61 2024-04-24T09:11:06.093 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.091+0000 7fa45900f740 10 client.22377 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:11:06.093 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.091+0000 7fa45900f740 20 client.22377 populate_metadata read hostname 'smithi083' 2024-04-24T09:11:06.093 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.091+0000 7fa45900f740 1 --2- 172.21.15.83:0/1121972400 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56112ea27ff0 0x56112ea2a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:06.093 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.091+0000 7fa45900f740 4 client.22377 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T09:11:06.093 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.091+0000 7fa45900f740 1 -- 172.21.15.83:0/1121972400 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56112e913190 con 0x56112ea27ff0 2024-04-24T09:11:06.093 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.092+0000 7fa44f400640 1 --2- 172.21.15.83:0/1121972400 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56112ea27ff0 0x56112ea2a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:06.093 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.092+0000 7fa44f400640 1 --2- 172.21.15.83:0/1121972400 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56112ea27ff0 0x56112ea2a3e0 crc :-1 s=READY pgs=490 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ab6a5400f5cd7bc server_cookie=5abf93e844121e64 in_seq=0 out_seq=0 2024-04-24T09:11:06.093 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.092+0000 7fa438c00640 10 client.22377 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:11:06.178 INFO:tasks.workunit.client.0.smithi083.stdout: 10 42526 6.77 MB/sec execute 217 sec latency 139.393 ms 2024-04-24T09:11:06.237 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.237+0000 7fa438c00640 1 -- 172.21.15.83:0/1121972400 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56112e913190 con 0x56112ea27ff0 2024-04-24T09:11:06.237 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.237+0000 7fa438c00640 10 client.22377 handle_command_reply: tid=0 2024-04-24T09:11:06.238 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.237+0000 7fa45900f740 1 client.22377 shutdown 2024-04-24T09:11:06.238 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.238+0000 7fa45900f740 1 -- 172.21.15.83:0/1121972400 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56112e9b05a0 msgr2=0x56112e9b8b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:06.238 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.238+0000 7fa45900f740 1 --2- 172.21.15.83:0/1121972400 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56112e9b05a0 0x56112e9b8b60 secure :-1 s=READY pgs=2489 cs=0 l=1 rev1=1 crypto rx=0x56112e9abe50 tx=0x7fa43c020620 comp rx=0 tx=0).stop 2024-04-24T09:11:06.239 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.238+0000 7fa45900f740 1 -- 172.21.15.83:0/1121972400 shutdown_connections 2024-04-24T09:11:06.239 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.238+0000 7fa45900f740 1 --2- 172.21.15.83:0/1121972400 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56112ea27ff0 0x56112ea2a3e0 crc :-1 s=READY pgs=490 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:06.239 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.238+0000 7fa45900f740 1 --2- 172.21.15.83:0/1121972400 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56112e9b55f0 0x56112e9b5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:06.239 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.238+0000 7fa45900f740 1 --2- 172.21.15.83:0/1121972400 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56112e9b05a0 0x56112e9b8b60 unknown :-1 s=CLOSED pgs=2489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:06.239 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.238+0000 7fa45900f740 1 --2- 172.21.15.83:0/1121972400 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56112e9a53c0 0x56112e9b8620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:06.239 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.238+0000 7fa45900f740 1 -- 172.21.15.83:0/1121972400 >> 172.21.15.83:0/1121972400 conn(0x56112e9a18d0 msgr2=0x56112e9ad5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:06.239 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.238+0000 7fa438c00640 0 client.22377 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:11:06.240 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.238+0000 7fa45900f740 1 -- 172.21.15.83:0/1121972400 shutdown_connections 2024-04-24T09:11:06.240 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.239+0000 7fa45900f740 1 -- 172.21.15.83:0/1121972400 wait complete. 2024-04-24T09:11:06.240 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.239+0000 7fa45900f740 20 client.22377 trim_cache size 0 max 16384 2024-04-24T09:11:06.240 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T09:11:06.240 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T09:11:06.240 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T09:11:06.240 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T09:11:06.241 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.241+0000 7fa454c00640 1 -- 172.21.15.83:0/1848473100 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa4340a3d10 msgr2=0x7fa4340a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:06.241 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.241+0000 7fa454c00640 1 --2- 172.21.15.83:0/1848473100 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa4340a3d10 0x7fa4340a61d0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x56112e9a2cf0 tx=0x7fa43c008040 comp rx=0 tx=0).stop 2024-04-24T09:11:06.241 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.241+0000 7fa454c00640 1 -- 172.21.15.83:0/1848473100 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa45019a6e0 msgr2=0x7fa45019ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:06.241 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.241+0000 7fa454c00640 1 --2- 172.21.15.83:0/1848473100 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa45019a6e0 0x7fa45019ab60 secure :-1 s=READY pgs=2485 cs=0 l=1 rev1=1 crypto rx=0x7fa448031040 tx=0x7fa448003970 comp rx=0 tx=0).stop 2024-04-24T09:11:06.241 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.241+0000 7fa454c00640 1 -- 172.21.15.83:0/1848473100 shutdown_connections 2024-04-24T09:11:06.241 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.241+0000 7fa454c00640 1 --2- 172.21.15.83:0/1848473100 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa4340a3d10 0x7fa4340a61d0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:06.241 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.241+0000 7fa454c00640 1 --2- 172.21.15.83:0/1848473100 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa45019a6e0 0x7fa45019ab60 unknown :-1 s=CLOSED pgs=2485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:06.242 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.241+0000 7fa454c00640 1 --2- 172.21.15.83:0/1848473100 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa450109370 0x7fa4501a0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:06.242 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.241+0000 7fa454c00640 1 --2- 172.21.15.83:0/1848473100 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4501042a0 0x7fa4501a05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:06.242 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.241+0000 7fa454c00640 1 -- 172.21.15.83:0/1848473100 >> 172.21.15.83:0/1848473100 conn(0x7fa4500ffae0 msgr2=0x7fa450101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:06.242 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.242+0000 7fa454c00640 1 -- 172.21.15.83:0/1848473100 shutdown_connections 2024-04-24T09:11:06.242 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:06.242+0000 7fa454c00640 1 -- 172.21.15.83:0/1848473100 wait complete. 2024-04-24T09:11:06.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:11:06 smithi083 ceph-mon[27907]: pgmap v614: 129 pgs: 129 active+clean; 699 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.5 MiB/s wr, 310 op/s 2024-04-24T09:11:06.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:11:06 smithi138 ceph-mon[31441]: pgmap v614: 129 pgs: 129 active+clean; 699 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.5 MiB/s wr, 310 op/s 2024-04-24T09:11:06.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:11:06 smithi189 ceph-mon[31791]: pgmap v614: 129 pgs: 129 active+clean; 699 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.5 MiB/s wr, 310 op/s 2024-04-24T09:11:07.178 INFO:tasks.workunit.client.0.smithi083.stdout: 10 42634 6.77 MB/sec execute 218 sec latency 123.154 ms 2024-04-24T09:11:08.178 INFO:tasks.workunit.client.0.smithi083.stdout: 10 42758 6.76 MB/sec execute 219 sec latency 70.822 ms 2024-04-24T09:11:08.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:11:08 smithi083 ceph-mon[27907]: pgmap v615: 129 pgs: 129 active+clean; 699 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.5 MiB/s wr, 310 op/s 2024-04-24T09:11:08.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:11:08 smithi138 ceph-mon[31441]: pgmap v615: 129 pgs: 129 active+clean; 699 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.5 MiB/s wr, 310 op/s 2024-04-24T09:11:08.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:11:08 smithi189 ceph-mon[31791]: pgmap v615: 129 pgs: 129 active+clean; 699 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.5 MiB/s wr, 310 op/s 2024-04-24T09:11:09.179 INFO:tasks.workunit.client.0.smithi083.stdout: 10 42874 6.77 MB/sec execute 220 sec latency 84.639 ms 2024-04-24T09:11:10.179 INFO:tasks.workunit.client.0.smithi083.stdout: 10 42969 6.76 MB/sec execute 221 sec latency 105.412 ms 2024-04-24T09:11:10.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:11:10 smithi083 ceph-mon[27907]: pgmap v616: 129 pgs: 129 active+clean; 714 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.1 MiB/s wr, 301 op/s 2024-04-24T09:11:10.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:11:10 smithi083 ceph-mon[27907]: osdmap e122: 12 total, 12 up, 12 in 2024-04-24T09:11:10.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:11:10 smithi138 ceph-mon[31441]: pgmap v616: 129 pgs: 129 active+clean; 714 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.1 MiB/s wr, 301 op/s 2024-04-24T09:11:10.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:11:10 smithi138 ceph-mon[31441]: osdmap e122: 12 total, 12 up, 12 in 2024-04-24T09:11:10.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:11:10 smithi189 ceph-mon[31791]: pgmap v616: 129 pgs: 129 active+clean; 714 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.1 MiB/s wr, 301 op/s 2024-04-24T09:11:10.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:11:10 smithi189 ceph-mon[31791]: osdmap e122: 12 total, 12 up, 12 in 2024-04-24T09:11:11.179 INFO:tasks.workunit.client.0.smithi083.stdout: 10 43074 6.74 MB/sec execute 222 sec latency 73.137 ms 2024-04-24T09:11:12.180 INFO:tasks.workunit.client.0.smithi083.stdout: 10 43192 6.75 MB/sec execute 223 sec latency 92.464 ms 2024-04-24T09:11:12.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:11:12 smithi138 ceph-mon[31441]: pgmap v618: 129 pgs: 129 active+clean; 714 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.0 MiB/s wr, 166 op/s 2024-04-24T09:11:12.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:11:12 smithi189 ceph-mon[31791]: pgmap v618: 129 pgs: 129 active+clean; 714 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.0 MiB/s wr, 166 op/s 2024-04-24T09:11:12.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:11:12 smithi083 ceph-mon[27907]: pgmap v618: 129 pgs: 129 active+clean; 714 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.0 MiB/s wr, 166 op/s 2024-04-24T09:11:13.180 INFO:tasks.workunit.client.0.smithi083.stdout: 10 43304 6.73 MB/sec execute 224 sec latency 93.789 ms 2024-04-24T09:11:13.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:11:13 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:11:13.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:11:13 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:11:13.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:11:13 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:11:13.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:11:13 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:11:13.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:11:13 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:11:13.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:11:13 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:11:13.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:11:13 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:11:13.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:11:13 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:11:13.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:11:13 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:11:13.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:11:13 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:11:13.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:11:13 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:11:13.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:11:13 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:11:14.180 INFO:tasks.workunit.client.0.smithi083.stdout: 10 43423 6.73 MB/sec execute 225 sec latency 92.276 ms 2024-04-24T09:11:14.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:11:14 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:11:14.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:11:14 smithi138 ceph-mon[31441]: pgmap v619: 129 pgs: 129 active+clean; 730 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.7 MiB/s wr, 264 op/s 2024-04-24T09:11:14.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:11:14 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:11:14.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:11:14 smithi189 ceph-mon[31791]: pgmap v619: 129 pgs: 129 active+clean; 730 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.7 MiB/s wr, 264 op/s 2024-04-24T09:11:14.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:11:14 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:11:14.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:11:14 smithi083 ceph-mon[27907]: pgmap v619: 129 pgs: 129 active+clean; 730 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.7 MiB/s wr, 264 op/s 2024-04-24T09:11:15.180 INFO:tasks.workunit.client.0.smithi083.stdout: 10 43542 6.71 MB/sec execute 226 sec latency 98.796 ms 2024-04-24T09:11:16.052 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:11:15 smithi083 ceph-mon[27907]: pgmap v620: 129 pgs: 129 active+clean; 730 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.6 MiB/s wr, 260 op/s 2024-04-24T09:11:16.180 INFO:tasks.workunit.client.0.smithi083.stdout: 10 43668 6.70 MB/sec execute 227 sec latency 105.196 ms 2024-04-24T09:11:16.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:11:15 smithi138 ceph-mon[31441]: pgmap v620: 129 pgs: 129 active+clean; 730 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.6 MiB/s wr, 260 op/s 2024-04-24T09:11:16.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:11:15 smithi189 ceph-mon[31791]: pgmap v620: 129 pgs: 129 active+clean; 730 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.6 MiB/s wr, 260 op/s 2024-04-24T09:11:17.180 INFO:tasks.workunit.client.0.smithi083.stdout: 10 43795 6.70 MB/sec execute 228 sec latency 111.751 ms 2024-04-24T09:11:18.180 INFO:tasks.workunit.client.0.smithi083.stdout: 10 43910 6.69 MB/sec execute 229 sec latency 120.275 ms 2024-04-24T09:11:18.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:11:17 smithi083 ceph-mon[27907]: pgmap v621: 129 pgs: 129 active+clean; 730 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.6 MiB/s wr, 260 op/s 2024-04-24T09:11:18.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:11:17 smithi138 ceph-mon[31441]: pgmap v621: 129 pgs: 129 active+clean; 730 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.6 MiB/s wr, 260 op/s 2024-04-24T09:11:18.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:11:17 smithi189 ceph-mon[31791]: pgmap v621: 129 pgs: 129 active+clean; 730 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.6 MiB/s wr, 260 op/s 2024-04-24T09:11:19.180 INFO:tasks.workunit.client.0.smithi083.stdout: 10 44002 6.66 MB/sec execute 230 sec latency 418.280 ms 2024-04-24T09:11:19.423 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.422+0000 7f8bb6a00640 1 -- 172.21.15.83:0/1551413944 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8bb010a850 msgr2=0x7f8bb01111d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:19.423 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.422+0000 7f8bb6a00640 1 --2- 172.21.15.83:0/1551413944 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8bb010a850 0x7f8bb01111d0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f8ba8009cd0 tx=0x7f8ba802a8e0 comp rx=0 tx=0).stop 2024-04-24T09:11:19.423 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.423+0000 7f8bb6a00640 1 -- 172.21.15.83:0/1551413944 shutdown_connections 2024-04-24T09:11:19.423 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.423+0000 7f8bb6a00640 1 --2- 172.21.15.83:0/1551413944 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8bb010a850 0x7f8bb01111d0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:19.424 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.423+0000 7f8bb6a00640 1 --2- 172.21.15.83:0/1551413944 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8bb0100cf0 0x7f8bb010a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:19.424 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.423+0000 7f8bb6a00640 1 --2- 172.21.15.83:0/1551413944 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bb0100340 0x7f8bb0100720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:19.424 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.423+0000 7f8bb6a00640 1 -- 172.21.15.83:0/1551413944 >> 172.21.15.83:0/1551413944 conn(0x7f8bb0077000 msgr2=0x7f8bb00fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:19.424 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.423+0000 7f8bb6a00640 1 -- 172.21.15.83:0/1551413944 shutdown_connections 2024-04-24T09:11:19.424 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.423+0000 7f8bb6a00640 1 -- 172.21.15.83:0/1551413944 wait complete. 2024-04-24T09:11:19.425 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.424+0000 7f8bb6a00640 1 Processor -- start 2024-04-24T09:11:19.426 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.424+0000 7f8bb6a00640 1 -- start start 2024-04-24T09:11:19.426 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.424+0000 7f8bb6a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bb0100340 0x7f8bb0110fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:19.426 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.425+0000 7f8bb6a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8bb010a850 0x7f8bb010e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:19.426 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.425+0000 7f8bb6a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8bb010e590 0x7f8bb010ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:19.426 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.425+0000 7f8bb6a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb0114530 con 0x7f8bb0100340 2024-04-24T09:11:19.426 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.425+0000 7f8bb6a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb01143b0 con 0x7f8bb010a850 2024-04-24T09:11:19.426 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.425+0000 7f8bb6a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb01146b0 con 0x7f8bb010e590 2024-04-24T09:11:19.426 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.425+0000 7f8bafe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bb0100340 0x7f8bb0110fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:19.426 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.425+0000 7f8bafe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bb0100340 0x7f8bb0110fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39236/0 (socket says 172.21.15.83:39236) 2024-04-24T09:11:19.426 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.425+0000 7f8bafe00640 1 -- 172.21.15.83:0/1508239383 learned_addr learned my addr 172.21.15.83:0/1508239383 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:11:19.426 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.425+0000 7f8baf400640 1 --2- 172.21.15.83:0/1508239383 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8bb010a850 0x7f8bb010e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:19.426 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.425+0000 7f8bb4a00640 1 --2- 172.21.15.83:0/1508239383 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8bb010e590 0x7f8bb010ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:19.427 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.425+0000 7f8baf400640 1 -- 172.21.15.83:0/1508239383 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8bb010e590 msgr2=0x7f8bb010ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:19.427 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.425+0000 7f8baf400640 1 --2- 172.21.15.83:0/1508239383 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8bb010e590 0x7f8bb010ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:19.427 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.426+0000 7f8baf400640 1 -- 172.21.15.83:0/1508239383 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bb0100340 msgr2=0x7f8bb0110fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:19.427 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.426+0000 7f8baf400640 1 --2- 172.21.15.83:0/1508239383 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bb0100340 0x7f8bb0110fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:19.427 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.426+0000 7f8baf400640 1 -- 172.21.15.83:0/1508239383 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bb01ab380 con 0x7f8bb010a850 2024-04-24T09:11:19.427 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.426+0000 7f8bb4a00640 1 --2- 172.21.15.83:0/1508239383 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8bb010e590 0x7f8bb010ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:11:19.427 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.426+0000 7f8baf400640 1 --2- 172.21.15.83:0/1508239383 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8bb010a850 0x7f8bb010e050 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8bb0068dc0 tx=0x7f8b9800dbb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:19.428 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.426+0000 7f8bacc00640 1 -- 172.21.15.83:0/1508239383 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b98017070 con 0x7f8bb010a850 2024-04-24T09:11:19.428 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.426+0000 7f8bb6a00640 1 -- 172.21.15.83:0/1508239383 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bb01ab670 con 0x7f8bb010a850 2024-04-24T09:11:19.428 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.426+0000 7f8bb6a00640 1 -- 172.21.15.83:0/1508239383 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bb01abbc0 con 0x7f8bb010a850 2024-04-24T09:11:19.428 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.426+0000 7f8bacc00640 1 -- 172.21.15.83:0/1508239383 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8b980099a0 con 0x7f8bb010a850 2024-04-24T09:11:19.428 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.427+0000 7f8bacc00640 1 -- 172.21.15.83:0/1508239383 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b98004df0 con 0x7f8bb010a850 2024-04-24T09:11:19.431 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.431+0000 7f8bacc00640 1 -- 172.21.15.83:0/1508239383 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8b98009b50 con 0x7f8bb010a850 2024-04-24T09:11:19.432 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.432+0000 7f8bb9804740 1 Processor -- start 2024-04-24T09:11:19.432 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.432+0000 7f8bb9804740 1 -- start start 2024-04-24T09:11:19.433 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.432+0000 7f8bb9804740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e254861b0 0x561e25486590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:19.433 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.432+0000 7f8bb9804740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561e25486ad0 0x561e25490e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:19.433 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.433+0000 7f8bb9804740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561e25491390 0x561e25493780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:19.433 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.433+0000 7f8bb9804740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561e253ec900 con 0x561e25486ad0 2024-04-24T09:11:19.433 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.433+0000 7f8bb9804740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x561e25392f20 con 0x561e254861b0 2024-04-24T09:11:19.433 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.433+0000 7f8bb9804740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x561e253e88c0 con 0x561e25491390 2024-04-24T09:11:19.434 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.433+0000 7f8bb4a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e254861b0 0x561e25486590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:19.434 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.433+0000 7f8bb4a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e254861b0 0x561e25486590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:52334/0 (socket says 172.21.15.83:52334) 2024-04-24T09:11:19.434 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.433+0000 7f8bb4a00640 1 -- 172.21.15.83:0/3656524222 learned_addr learned my addr 172.21.15.83:0/3656524222 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:11:19.434 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.433+0000 7f8baf400640 1 --2- 172.21.15.83:0/3656524222 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561e25491390 0x561e25493780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:19.435 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.434+0000 7f8bb4a00640 1 -- 172.21.15.83:0/3656524222 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561e25491390 msgr2=0x561e25493780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:19.435 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.434+0000 7f8bb4a00640 1 --2- 172.21.15.83:0/3656524222 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561e25491390 0x561e25493780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:19.435 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.434+0000 7f8bb4a00640 1 -- 172.21.15.83:0/3656524222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561e25486ad0 msgr2=0x561e25490e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:19.435 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.434+0000 7f8bafe00640 1 --2- 172.21.15.83:0/3656524222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561e25486ad0 0x561e25490e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:19.435 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.434+0000 7f8bb4a00640 1 --2- 172.21.15.83:0/3656524222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561e25486ad0 0x561e25490e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:19.435 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.434+0000 7f8bb4a00640 1 -- 172.21.15.83:0/3656524222 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e253b7e10 con 0x561e254861b0 2024-04-24T09:11:19.435 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.434+0000 7f8bafe00640 1 --2- 172.21.15.83:0/3656524222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561e25486ad0 0x561e25490e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:11:19.435 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.434+0000 7f8bacc00640 1 --2- 172.21.15.83:0/1508239383 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f8b940a3df0 0x7f8b940a62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:19.435 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.435+0000 7f8bacc00640 1 -- 172.21.15.83:0/1508239383 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 9583+0+0 (secure 0 0 0) 0x7f8b980d0790 con 0x7f8bb010a850 2024-04-24T09:11:19.435 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.435+0000 7f8bafe00640 1 --2- 172.21.15.83:0/1508239383 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f8b940a3df0 0x7f8b940a62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:19.436 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.435+0000 7f8bb4a00640 1 --2- 172.21.15.83:0/3656524222 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e254861b0 0x561e25486590 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8ba8009cd0 tx=0x7f8ba8009a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54a91f6083f2d0d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:19.436 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.436+0000 7f8bafe00640 1 --2- 172.21.15.83:0/1508239383 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f8b940a3df0 0x7f8b940a62b0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x561e25487750 tx=0x7f8ba0005f90 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:19.436 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.436+0000 7f8ba5200640 1 -- 172.21.15.83:0/3656524222 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ba8031850 con 0x561e254861b0 2024-04-24T09:11:19.436 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.436+0000 7f8ba5200640 1 -- 172.21.15.83:0/3656524222 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8ba8036040 con 0x561e254861b0 2024-04-24T09:11:19.436 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.436+0000 7f8ba5200640 1 -- 172.21.15.83:0/3656524222 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ba80044d0 con 0x561e254861b0 2024-04-24T09:11:19.437 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.437+0000 7f8bb9804740 1 -- 172.21.15.83:0/3656524222 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e254861b0 msgr2=0x561e25486590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:19.437 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.437+0000 7f8bb9804740 1 --2- 172.21.15.83:0/3656524222 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e254861b0 0x561e25486590 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8ba8009cd0 tx=0x7f8ba8009a20 comp rx=0 tx=0).stop 2024-04-24T09:11:19.438 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.437+0000 7f8bb9804740 1 -- 172.21.15.83:0/3656524222 shutdown_connections 2024-04-24T09:11:19.438 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.437+0000 7f8bb9804740 1 --2- 172.21.15.83:0/3656524222 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561e25491390 0x561e25493780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:19.438 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.437+0000 7f8bb9804740 1 --2- 172.21.15.83:0/3656524222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561e25486ad0 0x561e25490e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:19.438 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.437+0000 7f8bb9804740 1 --2- 172.21.15.83:0/3656524222 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e254861b0 0x561e25486590 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:19.438 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.437+0000 7f8bb9804740 1 -- 172.21.15.83:0/3656524222 >> 172.21.15.83:0/3656524222 conn(0x561e254826c0 msgr2=0x561e25482aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:19.438 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.438+0000 7f8bb9804740 1 -- 172.21.15.83:0/3656524222 shutdown_connections 2024-04-24T09:11:19.438 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.438+0000 7f8bb9804740 1 -- 172.21.15.83:0/3656524222 wait complete. 2024-04-24T09:11:19.439 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.438+0000 7f8bb9804740 1 -- start start 2024-04-24T09:11:19.439 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.439+0000 7f8bb9804740 1 Processor -- start 2024-04-24T09:11:19.439 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.439+0000 7f8bb9804740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561e254861b0 0x561e2548e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:19.440 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.439+0000 7f8bb9804740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561e25486ad0 0x561e2548e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:19.440 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.439+0000 7f8bb9804740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e25491390 0x561e2548edc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:19.440 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.439+0000 7f8bb9804740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561e25493f60 con 0x561e25486ad0 2024-04-24T09:11:19.440 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.439+0000 7f8bb9804740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x561e253a62c0 con 0x561e25491390 2024-04-24T09:11:19.440 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.439+0000 7f8bb9804740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x561e254940e0 con 0x561e254861b0 2024-04-24T09:11:19.440 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.439+0000 7f8bb9804740 10 client.0 _ll_register_callbacks cb 0x561e250fea00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:11:19.441 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.440+0000 7f8baf400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561e25486ad0 0x561e2548e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:19.441 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.440+0000 7f8bb4a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e25491390 0x561e2548edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:19.441 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.440+0000 7f8baf400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561e25486ad0 0x561e2548e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39252/0 (socket says 172.21.15.83:39252) 2024-04-24T09:11:19.441 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.440+0000 7f8baf400640 1 -- 172.21.15.83:0/3460535121 learned_addr learned my addr 172.21.15.83:0/3460535121 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:11:19.441 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.440+0000 7f8bb4a00640 1 --2- 172.21.15.83:0/3460535121 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e25491390 0x561e2548edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:52348/0 (socket says 172.21.15.83:52348) 2024-04-24T09:11:19.441 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.440+0000 7f8bafe00640 1 --2- 172.21.15.83:0/3460535121 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561e254861b0 0x561e2548e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:19.441 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.441+0000 7f8baf400640 1 -- 172.21.15.83:0/3460535121 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561e254861b0 msgr2=0x561e2548e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:19.442 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.441+0000 7f8baf400640 1 --2- 172.21.15.83:0/3460535121 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561e254861b0 0x561e2548e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:19.442 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.441+0000 7f8baf400640 1 -- 172.21.15.83:0/3460535121 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e25491390 msgr2=0x561e2548edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:19.442 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.441+0000 7f8baf400640 1 --2- 172.21.15.83:0/3460535121 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e25491390 0x561e2548edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:19.442 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.441+0000 7f8baf400640 1 -- 172.21.15.83:0/3460535121 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e253b6fd0 con 0x561e25486ad0 2024-04-24T09:11:19.442 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.441+0000 7f8bb4a00640 1 --2- 172.21.15.83:0/3460535121 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e25491390 0x561e2548edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:11:19.442 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.442+0000 7f8baf400640 1 --2- 172.21.15.83:0/3460535121 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561e25486ad0 0x561e2548e880 secure :-1 s=READY pgs=2490 cs=0 l=1 rev1=1 crypto rx=0x7f8b980cdb20 tx=0x7f8b98009e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d877b994d696529a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:19.442 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.442+0000 7f8b9d600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:11:19.443 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.442+0000 7f8b9d600640 1 -- 172.21.15.83:0/3460535121 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b980a06c0 con 0x561e25486ad0 2024-04-24T09:11:19.443 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.442+0000 7f8b9d600640 1 -- 172.21.15.83:0/3460535121 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 9583+0+0 (secure 0 0 0) 0x7f8b980cdd40 con 0x561e25486ad0 2024-04-24T09:11:19.443 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.442+0000 7f8bb9804740 1 -- 172.21.15.83:0/3460535121 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ba8036040 con 0x561e25486ad0 2024-04-24T09:11:19.444 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.444+0000 7f8b9d600640 1 -- 172.21.15.83:0/3460535121 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f8b980a4f80 con 0x561e25486ad0 2024-04-24T09:11:19.444 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.444+0000 7f8bb9804740 10 client.22386 fetch_fsmap learned FSMap version 61 2024-04-24T09:11:19.444 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.444+0000 7f8bb9804740 1 -- 172.21.15.83:0/3460535121 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x561e250ff210 con 0x561e25486ad0 2024-04-24T09:11:19.445 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.444+0000 7f8b9d600640 1 -- 172.21.15.83:0/3460535121 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8ba8036040 con 0x561e25486ad0 2024-04-24T09:11:19.446 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.445+0000 7f8b9d600640 1 -- 172.21.15.83:0/3460535121 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 61) ==== 8362+0+0 (secure 0 0 0) 0x7f8b98098060 con 0x561e25486ad0 2024-04-24T09:11:19.446 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.446+0000 7f8bb9804740 10 client.22386 fetch_fsmap finished waiting for FSMap version 61 2024-04-24T09:11:19.446 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.446+0000 7f8bb9804740 10 client.22386 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:11:19.446 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.446+0000 7f8bb9804740 20 client.22386 populate_metadata read hostname 'smithi083' 2024-04-24T09:11:19.446 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.446+0000 7f8bb9804740 1 --2- 172.21.15.83:0/3460535121 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x561e254a11a0 0x561e254a3590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:19.446 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.446+0000 7f8bb9804740 4 client.22386 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:11:19.447 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.446+0000 7f8bb9804740 1 -- 172.21.15.83:0/3460535121 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561e253fa320 con 0x561e254a11a0 2024-04-24T09:11:19.447 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.447+0000 7f8bafe00640 1 --2- 172.21.15.83:0/3460535121 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x561e254a11a0 0x561e254a3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:19.448 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.447+0000 7f8bafe00640 1 --2- 172.21.15.83:0/3460535121 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x561e254a11a0 0x561e254a3590 crc :-1 s=READY pgs=491 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79210058e6b96938 server_cookie=d4f9572127ac7d7c in_seq=0 out_seq=0 2024-04-24T09:11:19.448 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.448+0000 7f8b9d600640 10 client.22386 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:11:19.450 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.449+0000 7f8b9d600640 1 -- 172.21.15.83:0/3460535121 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561e253fa320 con 0x561e254a11a0 2024-04-24T09:11:19.450 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.449+0000 7f8b9d600640 10 client.22386 handle_command_reply: tid=0 2024-04-24T09:11:19.450 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.450+0000 7f8bb9804740 1 client.22386 shutdown 2024-04-24T09:11:19.450 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.450+0000 7f8bb9804740 1 -- 172.21.15.83:0/3460535121 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561e25486ad0 msgr2=0x561e2548e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:19.450 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.450+0000 7f8bb9804740 1 --2- 172.21.15.83:0/3460535121 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561e25486ad0 0x561e2548e880 secure :-1 s=READY pgs=2490 cs=0 l=1 rev1=1 crypto rx=0x7f8b980cdb20 tx=0x7f8b98009e40 comp rx=0 tx=0).stop 2024-04-24T09:11:19.451 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.450+0000 7f8bb9804740 1 -- 172.21.15.83:0/3460535121 shutdown_connections 2024-04-24T09:11:19.451 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.450+0000 7f8bb9804740 1 --2- 172.21.15.83:0/3460535121 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x561e254a11a0 0x561e254a3590 crc :-1 s=READY pgs=491 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:19.451 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.450+0000 7f8bb9804740 1 --2- 172.21.15.83:0/3460535121 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e25491390 0x561e2548edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:19.451 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.450+0000 7f8bb9804740 1 --2- 172.21.15.83:0/3460535121 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561e25486ad0 0x561e2548e880 unknown :-1 s=CLOSED pgs=2490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:19.451 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.450+0000 7f8bb9804740 1 --2- 172.21.15.83:0/3460535121 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561e254861b0 0x561e2548e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:19.451 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.450+0000 7f8bb9804740 1 -- 172.21.15.83:0/3460535121 >> 172.21.15.83:0/3460535121 conn(0x561e254826c0 msgr2=0x561e25488440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:19.451 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.450+0000 7f8b9d600640 0 client.22386 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:11:19.451 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.450+0000 7f8bb9804740 1 -- 172.21.15.83:0/3460535121 shutdown_connections 2024-04-24T09:11:19.452 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.451+0000 7f8bb9804740 1 -- 172.21.15.83:0/3460535121 wait complete. 2024-04-24T09:11:19.452 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.451+0000 7f8bb9804740 20 client.22386 trim_cache size 0 max 16384 2024-04-24T09:11:19.479 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.478+0000 7f8bb9804740 1 Processor -- start 2024-04-24T09:11:19.479 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.479+0000 7f8bb9804740 1 -- start start 2024-04-24T09:11:19.479 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.479+0000 7f8bb9804740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e254861b0 0x561e254a8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:19.479 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.479+0000 7f8bb9804740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561e25486ad0 0x561e254a93e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:19.480 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.479+0000 7f8bb9804740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561e25491390 0x561e2548b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:19.480 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.479+0000 7f8bb9804740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8b8c0101e0 con 0x561e25486ad0 2024-04-24T09:11:19.480 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.479+0000 7f8bb9804740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8b8c011d20 con 0x561e254861b0 2024-04-24T09:11:19.480 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.479+0000 7f8bb9804740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8b8c0108b0 con 0x561e25491390 2024-04-24T09:11:19.481 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.479+0000 7f8bafe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e254861b0 0x561e254a8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:19.481 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.480+0000 7f8bafe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e254861b0 0x561e254a8ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:52364/0 (socket says 172.21.15.83:52364) 2024-04-24T09:11:19.481 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.480+0000 7f8bafe00640 1 -- 172.21.15.83:0/2866639408 learned_addr learned my addr 172.21.15.83:0/2866639408 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:11:19.481 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.480+0000 7f8baf400640 1 --2- 172.21.15.83:0/2866639408 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561e25486ad0 0x561e254a93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:19.481 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.480+0000 7f8bb4a00640 1 --2- 172.21.15.83:0/2866639408 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561e25491390 0x561e2548b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:19.481 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.480+0000 7f8baf400640 1 -- 172.21.15.83:0/2866639408 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561e25491390 msgr2=0x561e2548b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:19.481 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.480+0000 7f8baf400640 1 --2- 172.21.15.83:0/2866639408 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561e25491390 0x561e2548b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:19.481 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.480+0000 7f8baf400640 1 -- 172.21.15.83:0/2866639408 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e254861b0 msgr2=0x561e254a8ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:19.482 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.480+0000 7f8baf400640 1 --2- 172.21.15.83:0/2866639408 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e254861b0 0x561e254a8ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:19.482 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.480+0000 7f8baf400640 1 -- 172.21.15.83:0/2866639408 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e25489160 con 0x561e25486ad0 2024-04-24T09:11:19.482 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.481+0000 7f8baf400640 1 --2- 172.21.15.83:0/2866639408 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561e25486ad0 0x561e254a93e0 secure :-1 s=READY pgs=2491 cs=0 l=1 rev1=1 crypto rx=0x7f8b9809c010 tx=0x7f8b980d6280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b297ec7f5df25fbd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:19.482 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.481+0000 7f8b9fe00640 1 -- 172.21.15.83:0/2866639408 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b980cdd40 con 0x561e25486ad0 2024-04-24T09:11:19.482 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.481+0000 7f8b9fe00640 1 -- 172.21.15.83:0/2866639408 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8b980d0310 con 0x561e25486ad0 2024-04-24T09:11:19.483 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.481+0000 7f8b9fe00640 1 -- 172.21.15.83:0/2866639408 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b98098060 con 0x561e25486ad0 2024-04-24T09:11:19.483 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.482+0000 7f8bb9804740 1 -- 172.21.15.83:0/2866639408 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561e25486ad0 msgr2=0x561e254a93e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:19.483 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.482+0000 7f8bb9804740 1 --2- 172.21.15.83:0/2866639408 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561e25486ad0 0x561e254a93e0 secure :-1 s=READY pgs=2491 cs=0 l=1 rev1=1 crypto rx=0x7f8b9809c010 tx=0x7f8b980d6280 comp rx=0 tx=0).stop 2024-04-24T09:11:19.483 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.482+0000 7f8bb9804740 1 -- 172.21.15.83:0/2866639408 shutdown_connections 2024-04-24T09:11:19.483 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.482+0000 7f8bb9804740 1 --2- 172.21.15.83:0/2866639408 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561e25491390 0x561e2548b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:19.483 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.482+0000 7f8bb9804740 1 --2- 172.21.15.83:0/2866639408 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561e25486ad0 0x561e254a93e0 unknown :-1 s=CLOSED pgs=2491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:19.483 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.482+0000 7f8bb9804740 1 --2- 172.21.15.83:0/2866639408 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e254861b0 0x561e254a8ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:19.483 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.482+0000 7f8bb9804740 1 -- 172.21.15.83:0/2866639408 >> 172.21.15.83:0/2866639408 conn(0x561e254826c0 msgr2=0x561e25488440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:19.483 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.482+0000 7f8bb9804740 1 -- 172.21.15.83:0/2866639408 shutdown_connections 2024-04-24T09:11:19.483 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.482+0000 7f8bb9804740 1 -- 172.21.15.83:0/2866639408 wait complete. 2024-04-24T09:11:19.483 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.483+0000 7f8bb9804740 1 -- start start 2024-04-24T09:11:19.483 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.483+0000 7f8bb9804740 1 Processor -- start 2024-04-24T09:11:19.484 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.483+0000 7f8bb9804740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561e254861b0 0x561e2548b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:19.485 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.483+0000 7f8bb9804740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e25486ad0 0x561e2548b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:19.485 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.483+0000 7f8bb9804740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561e25491390 0x561e2548bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:19.485 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.483+0000 7f8bb9804740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8b8c011650 con 0x561e254861b0 2024-04-24T09:11:19.485 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.483+0000 7f8bb9804740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8b8c010f80 con 0x561e25486ad0 2024-04-24T09:11:19.485 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.483+0000 7f8bb9804740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8b8c00e1e0 con 0x561e25491390 2024-04-24T09:11:19.485 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.483+0000 7f8bb9804740 10 client.0 _ll_register_callbacks cb 0x7f8b98005bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:11:19.485 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.484+0000 7f8bafe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561e254861b0 0x561e2548b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:19.485 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.484+0000 7f8baf400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e25486ad0 0x561e2548b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:19.485 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.484+0000 7f8bafe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561e254861b0 0x561e2548b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39268/0 (socket says 172.21.15.83:39268) 2024-04-24T09:11:19.485 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.484+0000 7f8baf400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e25486ad0 0x561e2548b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:52374/0 (socket says 172.21.15.83:52374) 2024-04-24T09:11:19.485 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.484+0000 7f8bafe00640 1 -- 172.21.15.83:0/2597557687 learned_addr learned my addr 172.21.15.83:0/2597557687 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:11:19.485 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.484+0000 7f8bafe00640 1 -- 172.21.15.83:0/2597557687 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561e25491390 msgr2=0x561e2548bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:19.485 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.484+0000 7f8bb4a00640 1 --2- 172.21.15.83:0/2597557687 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561e25491390 0x561e2548bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:19.485 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.484+0000 7f8bafe00640 1 --2- 172.21.15.83:0/2597557687 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561e25491390 0x561e2548bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:19.486 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.484+0000 7f8bafe00640 1 -- 172.21.15.83:0/2597557687 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e25486ad0 msgr2=0x561e2548b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:19.486 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.484+0000 7f8bafe00640 1 --2- 172.21.15.83:0/2597557687 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e25486ad0 0x561e2548b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:19.486 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.484+0000 7f8bafe00640 1 -- 172.21.15.83:0/2597557687 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e25486900 con 0x561e254861b0 2024-04-24T09:11:19.486 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.485+0000 7f8baf400640 1 --2- 172.21.15.83:0/2597557687 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e25486ad0 0x561e2548b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:11:19.487 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.485+0000 7f8bb4a00640 1 --2- 172.21.15.83:0/2597557687 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561e25491390 0x561e2548bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:11:19.487 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.485+0000 7f8bafe00640 1 --2- 172.21.15.83:0/2597557687 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561e254861b0 0x561e2548b210 secure :-1 s=READY pgs=2492 cs=0 l=1 rev1=1 crypto rx=0x7f8ba002b5e0 tx=0x7f8ba00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd9f8e82f6101128 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:19.487 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.486+0000 7f8b9d600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:11:19.487 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.486+0000 7f8b9d600640 1 -- 172.21.15.83:0/2597557687 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ba0005660 con 0x561e254861b0 2024-04-24T09:11:19.487 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.486+0000 7f8b9d600640 1 -- 172.21.15.83:0/2597557687 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 9583+0+0 (secure 0 0 0) 0x7f8ba00047a0 con 0x561e254861b0 2024-04-24T09:11:19.487 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.486+0000 7f8bb9804740 1 -- 172.21.15.83:0/2597557687 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b980d0310 con 0x561e254861b0 2024-04-24T09:11:19.488 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.487+0000 7f8b9d600640 1 -- 172.21.15.83:0/2597557687 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f8ba00049a0 con 0x561e254861b0 2024-04-24T09:11:19.488 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.487+0000 7f8bb9804740 10 client.22392 fetch_fsmap learned FSMap version 61 2024-04-24T09:11:19.488 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.487+0000 7f8bb9804740 1 -- 172.21.15.83:0/2597557687 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x561e251cb620 con 0x561e254861b0 2024-04-24T09:11:19.488 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.487+0000 7f8b9d600640 1 -- 172.21.15.83:0/2597557687 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8b980d0310 con 0x561e254861b0 2024-04-24T09:11:19.489 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.488+0000 7f8b9d600640 1 -- 172.21.15.83:0/2597557687 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 61) ==== 8362+0+0 (secure 0 0 0) 0x7f8ba003e420 con 0x561e254861b0 2024-04-24T09:11:19.489 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.488+0000 7f8bb9804740 10 client.22392 fetch_fsmap finished waiting for FSMap version 61 2024-04-24T09:11:19.489 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.488+0000 7f8bb9804740 10 client.22392 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:11:19.489 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.488+0000 7f8bb9804740 20 client.22392 populate_metadata read hostname 'smithi083' 2024-04-24T09:11:19.489 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.488+0000 7f8bb9804740 1 --2- 172.21.15.83:0/2597557687 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x561e25507fd0 0x561e2550a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:19.489 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.488+0000 7f8bb9804740 4 client.22392 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T09:11:19.489 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.488+0000 7f8bb9804740 1 -- 172.21.15.83:0/2597557687 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561e253f4190 con 0x561e25507fd0 2024-04-24T09:11:19.489 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.489+0000 7f8baf400640 1 --2- 172.21.15.83:0/2597557687 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x561e25507fd0 0x561e2550a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:19.490 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.489+0000 7f8baf400640 1 --2- 172.21.15.83:0/2597557687 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x561e25507fd0 0x561e2550a3f0 crc :-1 s=READY pgs=492 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba488dd4798f8932 server_cookie=7aff04a30d188686 in_seq=0 out_seq=0 2024-04-24T09:11:19.490 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.489+0000 7f8b9d600640 10 client.22392 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:11:19.635 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.635+0000 7f8b9d600640 1 -- 172.21.15.83:0/2597557687 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561e253f4190 con 0x561e25507fd0 2024-04-24T09:11:19.635 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.635+0000 7f8b9d600640 10 client.22392 handle_command_reply: tid=0 2024-04-24T09:11:19.635 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.635+0000 7f8bb9804740 1 client.22392 shutdown 2024-04-24T09:11:19.636 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.636+0000 7f8bb9804740 1 -- 172.21.15.83:0/2597557687 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561e254861b0 msgr2=0x561e2548b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:19.636 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.636+0000 7f8bb9804740 1 --2- 172.21.15.83:0/2597557687 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561e254861b0 0x561e2548b210 secure :-1 s=READY pgs=2492 cs=0 l=1 rev1=1 crypto rx=0x7f8ba002b5e0 tx=0x7f8ba00046c0 comp rx=0 tx=0).stop 2024-04-24T09:11:19.637 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.636+0000 7f8bb9804740 1 -- 172.21.15.83:0/2597557687 shutdown_connections 2024-04-24T09:11:19.637 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.636+0000 7f8bb9804740 1 --2- 172.21.15.83:0/2597557687 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x561e25507fd0 0x561e2550a3f0 crc :-1 s=READY pgs=492 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:19.637 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.636+0000 7f8bb9804740 1 --2- 172.21.15.83:0/2597557687 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561e25491390 0x561e2548bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:19.637 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.636+0000 7f8bb9804740 1 --2- 172.21.15.83:0/2597557687 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e25486ad0 0x561e2548b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:19.637 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.636+0000 7f8bb9804740 1 --2- 172.21.15.83:0/2597557687 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561e254861b0 0x561e2548b210 unknown :-1 s=CLOSED pgs=2492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:19.637 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.636+0000 7f8bb9804740 1 -- 172.21.15.83:0/2597557687 >> 172.21.15.83:0/2597557687 conn(0x561e254826c0 msgr2=0x561e2548e300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:19.637 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.636+0000 7f8b9d600640 0 client.22392 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:11:19.637 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.637+0000 7f8bb9804740 1 -- 172.21.15.83:0/2597557687 shutdown_connections 2024-04-24T09:11:19.637 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.637+0000 7f8bb9804740 1 -- 172.21.15.83:0/2597557687 wait complete. 2024-04-24T09:11:19.638 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.637+0000 7f8bb9804740 20 client.22392 trim_cache size 0 max 16384 2024-04-24T09:11:19.638 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T09:11:19.638 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T09:11:19.638 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T09:11:19.638 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T09:11:19.639 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.639+0000 7f8bb6a00640 1 -- 172.21.15.83:0/1508239383 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f8b940a3df0 msgr2=0x7f8b940a62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:19.639 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.639+0000 7f8bb6a00640 1 --2- 172.21.15.83:0/1508239383 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f8b940a3df0 0x7f8b940a62b0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x561e25487750 tx=0x7f8ba0005f90 comp rx=0 tx=0).stop 2024-04-24T09:11:19.639 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.639+0000 7f8bb6a00640 1 -- 172.21.15.83:0/1508239383 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8bb010a850 msgr2=0x7f8bb010e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:19.639 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.639+0000 7f8bb6a00640 1 --2- 172.21.15.83:0/1508239383 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8bb010a850 0x7f8bb010e050 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8bb0068dc0 tx=0x7f8b9800dbb0 comp rx=0 tx=0).stop 2024-04-24T09:11:19.640 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.639+0000 7f8bb6a00640 1 -- 172.21.15.83:0/1508239383 shutdown_connections 2024-04-24T09:11:19.640 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.639+0000 7f8bb6a00640 1 --2- 172.21.15.83:0/1508239383 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f8b940a3df0 0x7f8b940a62b0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:19.640 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.639+0000 7f8bb6a00640 1 --2- 172.21.15.83:0/1508239383 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8bb010e590 0x7f8bb010ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:19.640 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.639+0000 7f8bb6a00640 1 --2- 172.21.15.83:0/1508239383 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8bb010a850 0x7f8bb010e050 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:19.640 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.639+0000 7f8bb6a00640 1 --2- 172.21.15.83:0/1508239383 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bb0100340 0x7f8bb0110fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:19.640 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.639+0000 7f8bb6a00640 1 -- 172.21.15.83:0/1508239383 >> 172.21.15.83:0/1508239383 conn(0x7f8bb0077000 msgr2=0x7f8bb0108f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:19.640 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.640+0000 7f8bb6a00640 1 -- 172.21.15.83:0/1508239383 shutdown_connections 2024-04-24T09:11:19.640 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:19.640+0000 7f8bb6a00640 1 -- 172.21.15.83:0/1508239383 wait complete. 2024-04-24T09:11:20.181 INFO:tasks.workunit.client.0.smithi083.stdout: 10 44119 6.67 MB/sec execute 231 sec latency 123.207 ms 2024-04-24T09:11:20.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:11:19 smithi083 ceph-mon[27907]: pgmap v622: 129 pgs: 129 active+clean; 734 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.0 MiB/s wr, 272 op/s 2024-04-24T09:11:20.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:11:19 smithi138 ceph-mon[31441]: pgmap v622: 129 pgs: 129 active+clean; 734 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.0 MiB/s wr, 272 op/s 2024-04-24T09:11:20.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:11:19 smithi189 ceph-mon[31791]: pgmap v622: 129 pgs: 129 active+clean; 734 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.0 MiB/s wr, 272 op/s 2024-04-24T09:11:21.181 INFO:tasks.workunit.client.0.smithi083.stdout: 10 44218 6.68 MB/sec execute 232 sec latency 492.154 ms 2024-04-24T09:11:22.181 INFO:tasks.workunit.client.0.smithi083.stdout: 10 44342 6.67 MB/sec execute 233 sec latency 92.733 ms 2024-04-24T09:11:22.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:11:21 smithi083 ceph-mon[27907]: pgmap v623: 129 pgs: 129 active+clean; 734 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.6 MiB/s wr, 244 op/s 2024-04-24T09:11:22.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:11:21 smithi138 ceph-mon[31441]: pgmap v623: 129 pgs: 129 active+clean; 734 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.6 MiB/s wr, 244 op/s 2024-04-24T09:11:22.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:11:21 smithi189 ceph-mon[31791]: pgmap v623: 129 pgs: 129 active+clean; 734 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.6 MiB/s wr, 244 op/s 2024-04-24T09:11:23.181 INFO:tasks.workunit.client.0.smithi083.stdout: 10 44475 6.65 MB/sec execute 234 sec latency 74.550 ms 2024-04-24T09:11:24.181 INFO:tasks.workunit.client.0.smithi083.stdout: 10 44634 6.67 MB/sec execute 235 sec latency 78.655 ms 2024-04-24T09:11:24.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:11:23 smithi083 ceph-mon[27907]: pgmap v624: 129 pgs: 129 active+clean; 734 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.8 MiB/s wr, 360 op/s 2024-04-24T09:11:24.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:11:23 smithi138 ceph-mon[31441]: pgmap v624: 129 pgs: 129 active+clean; 734 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.8 MiB/s wr, 360 op/s 2024-04-24T09:11:24.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:11:23 smithi189 ceph-mon[31791]: pgmap v624: 129 pgs: 129 active+clean; 734 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.8 MiB/s wr, 360 op/s 2024-04-24T09:11:25.181 INFO:tasks.workunit.client.0.smithi083.stdout: 10 44766 6.66 MB/sec execute 236 sec latency 83.065 ms 2024-04-24T09:11:26.181 INFO:tasks.workunit.client.0.smithi083.stdout: 10 44910 6.66 MB/sec execute 237 sec latency 81.983 ms 2024-04-24T09:11:26.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:11:25 smithi083 ceph-mon[27907]: pgmap v625: 129 pgs: 129 active+clean; 734 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.0 MiB/s wr, 253 op/s 2024-04-24T09:11:26.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:11:25 smithi138 ceph-mon[31441]: pgmap v625: 129 pgs: 129 active+clean; 734 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.0 MiB/s wr, 253 op/s 2024-04-24T09:11:26.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:11:25 smithi189 ceph-mon[31791]: pgmap v625: 129 pgs: 129 active+clean; 734 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.0 MiB/s wr, 253 op/s 2024-04-24T09:11:27.181 INFO:tasks.workunit.client.0.smithi083.stdout: 10 45077 6.68 MB/sec execute 238 sec latency 72.668 ms 2024-04-24T09:11:28.181 INFO:tasks.workunit.client.0.smithi083.stdout: 10 45233 6.69 MB/sec execute 239 sec latency 70.407 ms 2024-04-24T09:11:28.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:11:27 smithi083 ceph-mon[27907]: pgmap v626: 129 pgs: 129 active+clean; 734 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.0 MiB/s wr, 253 op/s 2024-04-24T09:11:28.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:11:27 smithi138 ceph-mon[31441]: pgmap v626: 129 pgs: 129 active+clean; 734 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.0 MiB/s wr, 253 op/s 2024-04-24T09:11:28.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:11:27 smithi189 ceph-mon[31791]: pgmap v626: 129 pgs: 129 active+clean; 734 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.0 MiB/s wr, 253 op/s 2024-04-24T09:11:29.181 INFO:tasks.workunit.client.0.smithi083.stdout: 10 45361 6.69 MB/sec execute 240 sec latency 153.097 ms 2024-04-24T09:11:29.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:11:28 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:11:29.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:11:28 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:11:29.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:11:28 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:11:30.181 INFO:tasks.workunit.client.0.smithi083.stdout: 10 45467 6.70 MB/sec execute 241 sec latency 103.665 ms 2024-04-24T09:11:30.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:11:29 smithi083 ceph-mon[27907]: pgmap v627: 129 pgs: 129 active+clean; 747 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.7 MiB/s wr, 345 op/s 2024-04-24T09:11:30.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:11:29 smithi138 ceph-mon[31441]: pgmap v627: 129 pgs: 129 active+clean; 747 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.7 MiB/s wr, 345 op/s 2024-04-24T09:11:30.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:11:29 smithi189 ceph-mon[31791]: pgmap v627: 129 pgs: 129 active+clean; 747 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.7 MiB/s wr, 345 op/s 2024-04-24T09:11:31.181 INFO:tasks.workunit.client.0.smithi083.stdout: 10 45548 6.69 MB/sec execute 242 sec latency 453.122 ms 2024-04-24T09:11:31.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:11:30 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:11:31.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:11:30 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:11:31.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:11:30 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:11:31.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:11:30 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:11:31.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:11:30 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:11:31.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:11:30 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:11:31.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:11:30 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:11:31.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:11:30 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:11:31.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:11:30 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:11:32.181 INFO:tasks.workunit.client.0.smithi083.stdout: 10 45727 6.72 MB/sec execute 243 sec latency 69.550 ms 2024-04-24T09:11:32.324 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T09:11:32.324 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T09:11:32.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:11:32 smithi083 ceph-mon[27907]: pgmap v628: 129 pgs: 129 active+clean; 747 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.2 MiB/s wr, 224 op/s 2024-04-24T09:11:32.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.579+0000 7f310a800640 1 -- 172.21.15.83:0/337483190 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3104074bf0 msgr2=0x7f310410e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:32.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.579+0000 7f310a800640 1 --2- 172.21.15.83:0/337483190 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3104074bf0 0x7f310410e3a0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f30ec009860 tx=0x7f30ec02a3d0 comp rx=0 tx=0).stop 2024-04-24T09:11:32.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.580+0000 7f310a800640 1 -- 172.21.15.83:0/337483190 shutdown_connections 2024-04-24T09:11:32.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.580+0000 7f310a800640 1 --2- 172.21.15.83:0/337483190 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f310410e8e0 0x7f3104110cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:32.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.580+0000 7f310a800640 1 --2- 172.21.15.83:0/337483190 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3104074bf0 0x7f310410e3a0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:32.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.580+0000 7f310a800640 1 --2- 172.21.15.83:0/337483190 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f31040742d0 0x7f31040746b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:32.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.580+0000 7f310a800640 1 -- 172.21.15.83:0/337483190 >> 172.21.15.83:0/337483190 conn(0x7f3104079450 msgr2=0x7f3104079860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:32.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.580+0000 7f310a800640 1 -- 172.21.15.83:0/337483190 shutdown_connections 2024-04-24T09:11:32.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.580+0000 7f310a800640 1 -- 172.21.15.83:0/337483190 wait complete. 2024-04-24T09:11:32.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.581+0000 7f310a800640 1 Processor -- start 2024-04-24T09:11:32.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.581+0000 7f310a800640 1 -- start start 2024-04-24T09:11:32.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.582+0000 7f310a800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f31040742d0 0x7f3104102000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:32.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.582+0000 7f310a800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3104074bf0 0x7f3104102540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:32.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.582+0000 7f310a800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f310410e8e0 0x7f3104102a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:32.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.582+0000 7f310a800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f31041144d0 con 0x7f31040742d0 2024-04-24T09:11:32.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.582+0000 7f310a800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3104114350 con 0x7f3104074bf0 2024-04-24T09:11:32.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.582+0000 7f310a800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3104114650 con 0x7f310410e8e0 2024-04-24T09:11:32.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.582+0000 7f30fbe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f31040742d0 0x7f3104102000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:32.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.582+0000 7f30fbe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f31040742d0 0x7f3104102000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55164/0 (socket says 172.21.15.83:55164) 2024-04-24T09:11:32.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.582+0000 7f30fbe00640 1 -- 172.21.15.83:0/8905559 learned_addr learned my addr 172.21.15.83:0/8905559 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:11:32.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.582+0000 7f3103400640 1 --2- 172.21.15.83:0/8905559 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3104074bf0 0x7f3104102540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:32.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.583+0000 7f3103e00640 1 --2- 172.21.15.83:0/8905559 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f310410e8e0 0x7f3104102a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:32.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.583+0000 7f30fbe00640 1 -- 172.21.15.83:0/8905559 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f310410e8e0 msgr2=0x7f3104102a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:32.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.583+0000 7f30fbe00640 1 --2- 172.21.15.83:0/8905559 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f310410e8e0 0x7f3104102a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:32.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.583+0000 7f30fbe00640 1 -- 172.21.15.83:0/8905559 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3104074bf0 msgr2=0x7f3104102540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:32.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.583+0000 7f30fbe00640 1 --2- 172.21.15.83:0/8905559 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3104074bf0 0x7f3104102540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:32.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.583+0000 7f30fbe00640 1 -- 172.21.15.83:0/8905559 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31041ab0e0 con 0x7f31040742d0 2024-04-24T09:11:32.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.583+0000 7f3103400640 1 --2- 172.21.15.83:0/8905559 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3104074bf0 0x7f3104102540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:11:32.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.583+0000 7f30fbe00640 1 --2- 172.21.15.83:0/8905559 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f31040742d0 0x7f3104102000 secure :-1 s=READY pgs=2493 cs=0 l=1 rev1=1 crypto rx=0x7f30f400ea20 tx=0x7f30f400eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:32.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.584+0000 7f3100c00640 1 -- 172.21.15.83:0/8905559 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f400cd80 con 0x7f31040742d0 2024-04-24T09:11:32.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.584+0000 7f310a800640 1 -- 172.21.15.83:0/8905559 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31041ab3d0 con 0x7f31040742d0 2024-04-24T09:11:32.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.584+0000 7f310a800640 1 -- 172.21.15.83:0/8905559 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31041ab940 con 0x7f31040742d0 2024-04-24T09:11:32.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.584+0000 7f3100c00640 1 -- 172.21.15.83:0/8905559 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f30f40108e0 con 0x7f31040742d0 2024-04-24T09:11:32.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.584+0000 7f3100c00640 1 -- 172.21.15.83:0/8905559 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f4018840 con 0x7f31040742d0 2024-04-24T09:11:32.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.587+0000 7f310d5c5740 1 Processor -- start 2024-04-24T09:11:32.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.588+0000 7f310d5c5740 1 -- start start 2024-04-24T09:11:32.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.588+0000 7f310d5c5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb4cb92860 0x55fb4cb92c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:32.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.588+0000 7f310d5c5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fb4cb93180 0x55fb4cb9d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:32.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.588+0000 7f310d5c5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fb4cb9d850 0x55fb4cb9fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:32.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.588+0000 7f310d5c5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55fb4caf81f0 con 0x55fb4cb9d850 2024-04-24T09:11:32.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.588+0000 7f310d5c5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55fb4ca9e350 con 0x55fb4cb92860 2024-04-24T09:11:32.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.588+0000 7f310d5c5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55fb4caf41b0 con 0x55fb4cb93180 2024-04-24T09:11:32.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.589+0000 7f3103e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb4cb92860 0x55fb4cb92c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:32.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.589+0000 7f3103e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb4cb92860 0x55fb4cb92c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:57522/0 (socket says 172.21.15.83:57522) 2024-04-24T09:11:32.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.589+0000 7f3103e00640 1 -- 172.21.15.83:0/3115680189 learned_addr learned my addr 172.21.15.83:0/3115680189 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:11:32.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.589+0000 7f3103400640 1 --2- 172.21.15.83:0/3115680189 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fb4cb9d850 0x55fb4cb9fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:32.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.589+0000 7f3100c00640 1 -- 172.21.15.83:0/8905559 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f30f4010430 con 0x7f31040742d0 2024-04-24T09:11:32.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.589+0000 7f3103e00640 1 -- 172.21.15.83:0/3115680189 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fb4cb93180 msgr2=0x55fb4cb9d310 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-24T09:11:32.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.589+0000 7f3103e00640 1 --2- 172.21.15.83:0/3115680189 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fb4cb93180 0x55fb4cb9d310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:32.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.589+0000 7f3103e00640 1 -- 172.21.15.83:0/3115680189 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fb4cb9d850 msgr2=0x55fb4cb9fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:32.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.589+0000 7f3103e00640 1 --2- 172.21.15.83:0/3115680189 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fb4cb9d850 0x55fb4cb9fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:32.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.590+0000 7f3103e00640 1 -- 172.21.15.83:0/3115680189 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb4caf2f50 con 0x55fb4cb92860 2024-04-24T09:11:32.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.590+0000 7f3103400640 1 --2- 172.21.15.83:0/3115680189 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fb4cb9d850 0x55fb4cb9fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:11:32.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.590+0000 7f3103e00640 1 --2- 172.21.15.83:0/3115680189 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb4cb92860 0x55fb4cb92c40 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f30fc009fa0 tx=0x7f30fc00f890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dcd8c938c5a5c880 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:32.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.590+0000 7f30f8c00640 1 -- 172.21.15.83:0/3115680189 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30fc004b20 con 0x55fb4cb92860 2024-04-24T09:11:32.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.591+0000 7f30f8c00640 1 -- 172.21.15.83:0/3115680189 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f30fc004cc0 con 0x55fb4cb92860 2024-04-24T09:11:32.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.591+0000 7f30f8c00640 1 -- 172.21.15.83:0/3115680189 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30fc0054b0 con 0x55fb4cb92860 2024-04-24T09:11:32.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.591+0000 7f3100c00640 1 --2- 172.21.15.83:0/8905559 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f30e40a3d60 0x7f30e40a6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:32.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.592+0000 7f3100c00640 1 -- 172.21.15.83:0/8905559 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 9583+0+0 (secure 0 0 0) 0x7f30f40d0fb0 con 0x7f31040742d0 2024-04-24T09:11:32.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.592+0000 7f310d5c5740 1 -- 172.21.15.83:0/3115680189 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb4cb92860 msgr2=0x55fb4cb92c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:32.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.592+0000 7f310d5c5740 1 --2- 172.21.15.83:0/3115680189 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb4cb92860 0x55fb4cb92c40 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f30fc009fa0 tx=0x7f30fc00f890 comp rx=0 tx=0).stop 2024-04-24T09:11:32.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.592+0000 7f30fbe00640 1 --2- 172.21.15.83:0/8905559 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f30e40a3d60 0x7f30e40a6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:32.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.592+0000 7f310d5c5740 1 -- 172.21.15.83:0/3115680189 shutdown_connections 2024-04-24T09:11:32.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.592+0000 7f310d5c5740 1 --2- 172.21.15.83:0/3115680189 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fb4cb9d850 0x55fb4cb9fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:32.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.592+0000 7f310d5c5740 1 --2- 172.21.15.83:0/3115680189 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fb4cb93180 0x55fb4cb9d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:32.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.592+0000 7f310d5c5740 1 --2- 172.21.15.83:0/3115680189 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb4cb92860 0x55fb4cb92c40 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:32.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.592+0000 7f310d5c5740 1 -- 172.21.15.83:0/3115680189 >> 172.21.15.83:0/3115680189 conn(0x55fb4cb8eb80 msgr2=0x55fb4cb8ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:32.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.593+0000 7f310d5c5740 1 -- 172.21.15.83:0/3115680189 shutdown_connections 2024-04-24T09:11:32.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.593+0000 7f310d5c5740 1 -- 172.21.15.83:0/3115680189 wait complete. 2024-04-24T09:11:32.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.593+0000 7f30fbe00640 1 --2- 172.21.15.83:0/8905559 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f30e40a3d60 0x7f30e40a6220 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x55fb4cb93e20 tx=0x7f30f400cf90 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:32.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.593+0000 7f310d5c5740 1 -- start start 2024-04-24T09:11:32.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.594+0000 7f310d5c5740 1 Processor -- start 2024-04-24T09:11:32.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.594+0000 7f310d5c5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fb4cb92860 0x55fb4cb9a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:32.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.594+0000 7f310d5c5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fb4cb93180 0x55fb4cb9ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:32.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.594+0000 7f310d5c5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb4cb9d850 0x55fb4cb9b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:32.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.594+0000 7f310d5c5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55fb4cba0640 con 0x55fb4cb93180 2024-04-24T09:11:32.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.594+0000 7f310d5c5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55fb4cab1690 con 0x55fb4cb9d850 2024-04-24T09:11:32.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.594+0000 7f310d5c5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55fb4cba07c0 con 0x55fb4cb92860 2024-04-24T09:11:32.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.594+0000 7f310d5c5740 10 client.0 _ll_register_callbacks cb 0x55fb4caed650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:11:32.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.594+0000 7f3103400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fb4cb93180 0x55fb4cb9ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:32.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.594+0000 7f3103400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fb4cb93180 0x55fb4cb9ad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55170/0 (socket says 172.21.15.83:55170) 2024-04-24T09:11:32.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.595+0000 7f3103400640 1 -- 172.21.15.83:0/3719328266 learned_addr learned my addr 172.21.15.83:0/3719328266 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:11:32.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.595+0000 7f3103e00640 1 --2- 172.21.15.83:0/3719328266 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb4cb9d850 0x55fb4cb9b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:32.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.595+0000 7f3103e00640 1 --2- 172.21.15.83:0/3719328266 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fb4cb92860 0x55fb4cb9a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:32.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.595+0000 7f3103400640 1 -- 172.21.15.83:0/3719328266 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fb4cb92860 msgr2=0x55fb4cb9a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:32.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.595+0000 7f3103400640 1 --2- 172.21.15.83:0/3719328266 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fb4cb92860 0x55fb4cb9a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:32.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.595+0000 7f3103400640 1 -- 172.21.15.83:0/3719328266 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb4cb9d850 msgr2=0x55fb4cb9b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:32.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.595+0000 7f3103400640 1 --2- 172.21.15.83:0/3719328266 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb4cb9d850 0x55fb4cb9b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:32.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.595+0000 7f3103400640 1 -- 172.21.15.83:0/3719328266 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb4cab0920 con 0x55fb4cb93180 2024-04-24T09:11:32.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.595+0000 7f3103e00640 1 --2- 172.21.15.83:0/3719328266 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb4cb9d850 0x55fb4cb9b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:11:32.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.595+0000 7f3103400640 1 --2- 172.21.15.83:0/3719328266 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fb4cb93180 0x55fb4cb9ad60 secure :-1 s=READY pgs=2494 cs=0 l=1 rev1=1 crypto rx=0x7f30ec009860 tx=0x7f30ec030ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b42ca7d8eef2b94 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:32.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.595+0000 7f30f1600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:11:32.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.596+0000 7f30f1600640 1 -- 172.21.15.83:0/3719328266 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30ec004050 con 0x55fb4cb93180 2024-04-24T09:11:32.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.596+0000 7f30f1600640 1 -- 172.21.15.83:0/3719328266 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 9583+0+0 (secure 0 0 0) 0x7f30ec0041f0 con 0x55fb4cb93180 2024-04-24T09:11:32.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.596+0000 7f310d5c5740 1 -- 172.21.15.83:0/3719328266 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30fc004cc0 con 0x55fb4cb93180 2024-04-24T09:11:32.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.596+0000 7f30f1600640 1 -- 172.21.15.83:0/3719328266 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f30ec0043f0 con 0x55fb4cb93180 2024-04-24T09:11:32.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.596+0000 7f310d5c5740 10 client.22404 fetch_fsmap learned FSMap version 61 2024-04-24T09:11:32.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.596+0000 7f310d5c5740 1 -- 172.21.15.83:0/3719328266 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x55fb4c8d47d0 con 0x55fb4cb93180 2024-04-24T09:11:32.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.597+0000 7f30f1600640 1 -- 172.21.15.83:0/3719328266 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f30fc004cc0 con 0x55fb4cb93180 2024-04-24T09:11:32.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.597+0000 7f30f1600640 1 -- 172.21.15.83:0/3719328266 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 61) ==== 8362+0+0 (secure 0 0 0) 0x7f30ec004ad0 con 0x55fb4cb93180 2024-04-24T09:11:32.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.597+0000 7f310d5c5740 10 client.22404 fetch_fsmap finished waiting for FSMap version 61 2024-04-24T09:11:32.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.597+0000 7f310d5c5740 10 client.22404 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:11:32.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.597+0000 7f310d5c5740 20 client.22404 populate_metadata read hostname 'smithi083' 2024-04-24T09:11:32.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.597+0000 7f310d5c5740 1 --2- 172.21.15.83:0/3719328266 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55fb4cbadd50 0x55fb4cbb0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:32.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.597+0000 7f310d5c5740 4 client.22404 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:11:32.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.597+0000 7f310d5c5740 1 -- 172.21.15.83:0/3719328266 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fb4cb05c10 con 0x55fb4cbadd50 2024-04-24T09:11:32.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.597+0000 7f3103e00640 1 --2- 172.21.15.83:0/3719328266 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55fb4cbadd50 0x55fb4cbb0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:32.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.598+0000 7f3103e00640 1 --2- 172.21.15.83:0/3719328266 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55fb4cbadd50 0x55fb4cbb0170 crc :-1 s=READY pgs=493 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=963a465fbedddd64 server_cookie=9888647162d58550 in_seq=0 out_seq=0 2024-04-24T09:11:32.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.598+0000 7f30f1600640 10 client.22404 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:11:32.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.599+0000 7f30f1600640 1 -- 172.21.15.83:0/3719328266 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fb4cb05c10 con 0x55fb4cbadd50 2024-04-24T09:11:32.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.599+0000 7f30f1600640 10 client.22404 handle_command_reply: tid=0 2024-04-24T09:11:32.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.599+0000 7f310d5c5740 1 client.22404 shutdown 2024-04-24T09:11:32.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.599+0000 7f310d5c5740 1 -- 172.21.15.83:0/3719328266 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fb4cb93180 msgr2=0x55fb4cb9ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:32.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.599+0000 7f310d5c5740 1 --2- 172.21.15.83:0/3719328266 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fb4cb93180 0x55fb4cb9ad60 secure :-1 s=READY pgs=2494 cs=0 l=1 rev1=1 crypto rx=0x7f30ec009860 tx=0x7f30ec030ba0 comp rx=0 tx=0).stop 2024-04-24T09:11:32.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.600+0000 7f310d5c5740 1 -- 172.21.15.83:0/3719328266 shutdown_connections 2024-04-24T09:11:32.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.600+0000 7f310d5c5740 1 --2- 172.21.15.83:0/3719328266 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55fb4cbadd50 0x55fb4cbb0170 crc :-1 s=READY pgs=493 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:32.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.600+0000 7f310d5c5740 1 --2- 172.21.15.83:0/3719328266 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb4cb9d850 0x55fb4cb9b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:32.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.600+0000 7f310d5c5740 1 --2- 172.21.15.83:0/3719328266 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fb4cb93180 0x55fb4cb9ad60 unknown :-1 s=CLOSED pgs=2494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:32.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.600+0000 7f310d5c5740 1 --2- 172.21.15.83:0/3719328266 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fb4cb92860 0x55fb4cb9a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:32.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.600+0000 7f310d5c5740 1 -- 172.21.15.83:0/3719328266 >> 172.21.15.83:0/3719328266 conn(0x55fb4cb8eb80 msgr2=0x55fb4cb94880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:32.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.600+0000 7f30f1600640 0 client.22404 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:11:32.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.600+0000 7f310d5c5740 1 -- 172.21.15.83:0/3719328266 shutdown_connections 2024-04-24T09:11:32.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.600+0000 7f310d5c5740 1 -- 172.21.15.83:0/3719328266 wait complete. 2024-04-24T09:11:32.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.600+0000 7f310d5c5740 20 client.22404 trim_cache size 0 max 16384 2024-04-24T09:11:32.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.625+0000 7f310d5c5740 1 Processor -- start 2024-04-24T09:11:32.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.625+0000 7f310d5c5740 1 -- start start 2024-04-24T09:11:32.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.626+0000 7f310d5c5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fb4cb92860 0x55fb4cb8ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:32.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.626+0000 7f310d5c5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fb4cb93180 0x55fb4cb8e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:32.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.626+0000 7f310d5c5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb4cb9d850 0x55fb4cb9b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:32.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.626+0000 7f310d5c5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f30e0010110 con 0x55fb4cb93180 2024-04-24T09:11:32.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.626+0000 7f310d5c5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f30e0011c50 con 0x55fb4cb9d850 2024-04-24T09:11:32.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.626+0000 7f310d5c5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f30e00107e0 con 0x55fb4cb92860 2024-04-24T09:11:32.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.626+0000 7f3103400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fb4cb93180 0x55fb4cb8e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:32.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.626+0000 7f3103400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fb4cb93180 0x55fb4cb8e3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55180/0 (socket says 172.21.15.83:55180) 2024-04-24T09:11:32.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.626+0000 7f3103400640 1 -- 172.21.15.83:0/3774423617 learned_addr learned my addr 172.21.15.83:0/3774423617 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:11:32.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.626+0000 7f3103400640 1 -- 172.21.15.83:0/3774423617 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fb4cb92860 msgr2=0x55fb4cb8ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:32.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.626+0000 7f3103400640 1 --2- 172.21.15.83:0/3774423617 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fb4cb92860 0x55fb4cb8ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:32.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.626+0000 7f3103400640 1 -- 172.21.15.83:0/3774423617 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb4cb9d850 msgr2=0x55fb4cb9b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:32.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.626+0000 7f3103400640 1 --2- 172.21.15.83:0/3774423617 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb4cb9d850 0x55fb4cb9b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:32.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.626+0000 7f3103400640 1 -- 172.21.15.83:0/3774423617 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb4c80a3f0 con 0x55fb4cb93180 2024-04-24T09:11:32.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.627+0000 7f3103400640 1 --2- 172.21.15.83:0/3774423617 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fb4cb93180 0x55fb4cb8e3a0 secure :-1 s=READY pgs=2495 cs=0 l=1 rev1=1 crypto rx=0x7f30ec003730 tx=0x7f30ec004590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a2f26a8126a7741 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:32.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.627+0000 7f30f3e00640 1 -- 172.21.15.83:0/3774423617 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30ec0037b0 con 0x55fb4cb93180 2024-04-24T09:11:32.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.627+0000 7f30f3e00640 1 -- 172.21.15.83:0/3774423617 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f30ec003950 con 0x55fb4cb93180 2024-04-24T09:11:32.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.627+0000 7f30f3e00640 1 -- 172.21.15.83:0/3774423617 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30ec004920 con 0x55fb4cb93180 2024-04-24T09:11:32.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.628+0000 7f310d5c5740 1 -- 172.21.15.83:0/3774423617 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fb4cb93180 msgr2=0x55fb4cb8e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:32.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.628+0000 7f310d5c5740 1 --2- 172.21.15.83:0/3774423617 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fb4cb93180 0x55fb4cb8e3a0 secure :-1 s=READY pgs=2495 cs=0 l=1 rev1=1 crypto rx=0x7f30ec003730 tx=0x7f30ec004590 comp rx=0 tx=0).stop 2024-04-24T09:11:32.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.628+0000 7f310d5c5740 1 -- 172.21.15.83:0/3774423617 shutdown_connections 2024-04-24T09:11:32.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.628+0000 7f310d5c5740 1 --2- 172.21.15.83:0/3774423617 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb4cb9d850 0x55fb4cb9b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:32.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.628+0000 7f310d5c5740 1 --2- 172.21.15.83:0/3774423617 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fb4cb93180 0x55fb4cb8e3a0 unknown :-1 s=CLOSED pgs=2495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:32.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.628+0000 7f310d5c5740 1 --2- 172.21.15.83:0/3774423617 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fb4cb92860 0x55fb4cb8ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:32.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.628+0000 7f310d5c5740 1 -- 172.21.15.83:0/3774423617 >> 172.21.15.83:0/3774423617 conn(0x55fb4cb8eb80 msgr2=0x55fb4cb93ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:32.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.629+0000 7f310d5c5740 1 -- 172.21.15.83:0/3774423617 shutdown_connections 2024-04-24T09:11:32.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.629+0000 7f310d5c5740 1 -- 172.21.15.83:0/3774423617 wait complete. 2024-04-24T09:11:32.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.629+0000 7f310d5c5740 1 -- start start 2024-04-24T09:11:32.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.629+0000 7f310d5c5740 1 Processor -- start 2024-04-24T09:11:32.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.630+0000 7f310d5c5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fb4cb92860 0x55fb4cba5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:32.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.630+0000 7f310d5c5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb4cb93180 0x55fb4cba5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:32.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.630+0000 7f310d5c5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fb4cb9d850 0x55fb4cba0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:32.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.630+0000 7f310d5c5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f30e0011580 con 0x55fb4cb92860 2024-04-24T09:11:32.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.630+0000 7f310d5c5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f30e0010eb0 con 0x55fb4cb93180 2024-04-24T09:11:32.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.630+0000 7f310d5c5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f30e000e110 con 0x55fb4cb9d850 2024-04-24T09:11:32.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.630+0000 7f310d5c5740 10 client.0 _ll_register_callbacks cb 0x7f30f4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:11:32.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.630+0000 7f3103400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb4cb93180 0x55fb4cba5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:32.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.630+0000 7f3103400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb4cb93180 0x55fb4cba5c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:57540/0 (socket says 172.21.15.83:57540) 2024-04-24T09:11:32.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.630+0000 7f3103400640 1 -- 172.21.15.83:0/4154501338 learned_addr learned my addr 172.21.15.83:0/4154501338 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:11:32.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.630+0000 7f3103e00640 1 --2- 172.21.15.83:0/4154501338 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fb4cb92860 0x55fb4cba5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:32.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.631+0000 7f3103400640 1 -- 172.21.15.83:0/4154501338 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fb4cb9d850 msgr2=0x55fb4cba0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:32.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.631+0000 7f3103400640 1 --2- 172.21.15.83:0/4154501338 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fb4cb9d850 0x55fb4cba0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:32.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.631+0000 7f3103400640 1 -- 172.21.15.83:0/4154501338 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fb4cb92860 msgr2=0x55fb4cba5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:32.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.631+0000 7f3103400640 1 --2- 172.21.15.83:0/4154501338 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fb4cb92860 0x55fb4cba5730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:32.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.631+0000 7f3103400640 1 -- 172.21.15.83:0/4154501338 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb4c7ca0f0 con 0x55fb4cb93180 2024-04-24T09:11:32.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.631+0000 7f3103400640 1 --2- 172.21.15.83:0/4154501338 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb4cb93180 0x55fb4cba5c70 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f30ec005b60 tx=0x7f30ec003db0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6bfd4243d01faaac server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:32.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.632+0000 7f30f1600640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:11:32.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.632+0000 7f30f1600640 1 -- 172.21.15.83:0/4154501338 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30ec0037b0 con 0x55fb4cb93180 2024-04-24T09:11:32.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.632+0000 7f310d5c5740 1 -- 172.21.15.83:0/4154501338 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30ec003950 con 0x55fb4cb93180 2024-04-24T09:11:32.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.632+0000 7f30f1600640 1 -- 172.21.15.83:0/4154501338 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 9583+0+0 (secure 0 0 0) 0x7f30ec030740 con 0x55fb4cb93180 2024-04-24T09:11:32.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.632+0000 7f30f1600640 1 -- 172.21.15.83:0/4154501338 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f30ec069050 con 0x55fb4cb93180 2024-04-24T09:11:32.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.632+0000 7f310d5c5740 10 client.39494 fetch_fsmap learned FSMap version 61 2024-04-24T09:11:32.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.632+0000 7f310d5c5740 1 -- 172.21.15.83:0/4154501338 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x55fb4cabd810 con 0x55fb4cb93180 2024-04-24T09:11:32.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.633+0000 7f30f1600640 1 -- 172.21.15.83:0/4154501338 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f30ec003950 con 0x55fb4cb93180 2024-04-24T09:11:32.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.633+0000 7f30f1600640 1 -- 172.21.15.83:0/4154501338 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 61) ==== 8362+0+0 (secure 0 0 0) 0x7f30ec005050 con 0x55fb4cb93180 2024-04-24T09:11:32.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.633+0000 7f310d5c5740 10 client.39494 fetch_fsmap finished waiting for FSMap version 61 2024-04-24T09:11:32.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.633+0000 7f310d5c5740 10 client.39494 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:11:32.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.633+0000 7f310d5c5740 20 client.39494 populate_metadata read hostname 'smithi083' 2024-04-24T09:11:32.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.633+0000 7f310d5c5740 1 --2- 172.21.15.83:0/4154501338 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55fb4cc14090 0x55fb4cc164b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:32.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.633+0000 7f310d5c5740 4 client.39494 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T09:11:32.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.634+0000 7f310d5c5740 1 -- 172.21.15.83:0/4154501338 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55fb4caffa80 con 0x55fb4cc14090 2024-04-24T09:11:32.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.634+0000 7f3103e00640 1 --2- 172.21.15.83:0/4154501338 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55fb4cc14090 0x55fb4cc164b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:32.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.635+0000 7f3103e00640 1 --2- 172.21.15.83:0/4154501338 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55fb4cc14090 0x55fb4cc164b0 crc :-1 s=READY pgs=494 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6b1839c05ac9ddd server_cookie=91ffef7aab894a36 in_seq=0 out_seq=0 2024-04-24T09:11:32.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.635+0000 7f30f1600640 10 client.39494 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:11:32.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.635+0000 7f30f1600640 1 -- 172.21.15.83:0/4154501338 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55fb4caffa80 con 0x55fb4cc14090 2024-04-24T09:11:32.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.635+0000 7f30f1600640 10 client.39494 handle_command_reply: tid=0 2024-04-24T09:11:32.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.635+0000 7f310d5c5740 1 client.39494 shutdown 2024-04-24T09:11:32.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.635+0000 7f310d5c5740 1 -- 172.21.15.83:0/4154501338 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb4cb93180 msgr2=0x55fb4cba5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:32.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.635+0000 7f310d5c5740 1 --2- 172.21.15.83:0/4154501338 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb4cb93180 0x55fb4cba5c70 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f30ec005b60 tx=0x7f30ec003db0 comp rx=0 tx=0).stop 2024-04-24T09:11:32.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.635+0000 7f310d5c5740 1 -- 172.21.15.83:0/4154501338 shutdown_connections 2024-04-24T09:11:32.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.635+0000 7f310d5c5740 1 --2- 172.21.15.83:0/4154501338 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55fb4cc14090 0x55fb4cc164b0 crc :-1 s=READY pgs=494 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:32.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.635+0000 7f310d5c5740 1 --2- 172.21.15.83:0/4154501338 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fb4cb9d850 0x55fb4cba0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:32.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.635+0000 7f310d5c5740 1 --2- 172.21.15.83:0/4154501338 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb4cb93180 0x55fb4cba5c70 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:32.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.635+0000 7f310d5c5740 1 --2- 172.21.15.83:0/4154501338 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fb4cb92860 0x55fb4cba5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:32.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.635+0000 7f310d5c5740 1 -- 172.21.15.83:0/4154501338 >> 172.21.15.83:0/4154501338 conn(0x55fb4cb8eb80 msgr2=0x55fb4cb9c050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:32.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.635+0000 7f30f1600640 0 client.39494 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:11:32.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.636+0000 7f310d5c5740 1 -- 172.21.15.83:0/4154501338 shutdown_connections 2024-04-24T09:11:32.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.636+0000 7f310d5c5740 1 -- 172.21.15.83:0/4154501338 wait complete. 2024-04-24T09:11:32.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.636+0000 7f310d5c5740 20 client.39494 trim_cache size 0 max 16384 2024-04-24T09:11:32.636 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T09:11:32.637 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-04-24T09:11:32.637 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-04-24T09:11:32.637 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T09:11:32.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.637+0000 7f310a800640 1 -- 172.21.15.83:0/8905559 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f30e40a3d60 msgr2=0x7f30e40a6220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:32.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.637+0000 7f310a800640 1 --2- 172.21.15.83:0/8905559 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f30e40a3d60 0x7f30e40a6220 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x55fb4cb93e20 tx=0x7f30f400cf90 comp rx=0 tx=0).stop 2024-04-24T09:11:32.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.637+0000 7f310a800640 1 -- 172.21.15.83:0/8905559 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f31040742d0 msgr2=0x7f3104102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:32.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.637+0000 7f310a800640 1 --2- 172.21.15.83:0/8905559 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f31040742d0 0x7f3104102000 secure :-1 s=READY pgs=2493 cs=0 l=1 rev1=1 crypto rx=0x7f30f400ea20 tx=0x7f30f400eee0 comp rx=0 tx=0).stop 2024-04-24T09:11:32.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.637+0000 7f310a800640 1 -- 172.21.15.83:0/8905559 shutdown_connections 2024-04-24T09:11:32.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.637+0000 7f310a800640 1 --2- 172.21.15.83:0/8905559 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f30e40a3d60 0x7f30e40a6220 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:32.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.637+0000 7f310a800640 1 --2- 172.21.15.83:0/8905559 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f310410e8e0 0x7f3104102a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:32.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.637+0000 7f310a800640 1 --2- 172.21.15.83:0/8905559 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3104074bf0 0x7f3104102540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:32.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.637+0000 7f310a800640 1 --2- 172.21.15.83:0/8905559 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f31040742d0 0x7f3104102000 unknown :-1 s=CLOSED pgs=2493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:32.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.637+0000 7f310a800640 1 -- 172.21.15.83:0/8905559 >> 172.21.15.83:0/8905559 conn(0x7f3104079450 msgr2=0x7f31040ff010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:32.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.637+0000 7f310a800640 1 -- 172.21.15.83:0/8905559 shutdown_connections 2024-04-24T09:11:32.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.637+0000 7f310a800640 1 -- 172.21.15.83:0/8905559 wait complete. 2024-04-24T09:11:32.659 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T09:11:32.659 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T09:11:32.704 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:11:32 smithi138 ceph-mon[31441]: pgmap v628: 129 pgs: 129 active+clean; 747 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.2 MiB/s wr, 224 op/s 2024-04-24T09:11:32.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:11:32 smithi189 ceph-mon[31791]: pgmap v628: 129 pgs: 129 active+clean; 747 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.2 MiB/s wr, 224 op/s 2024-04-24T09:11:32.828 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.828+0000 7fec0f000640 1 -- 172.21.15.83:0/1983897458 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec08102280 msgr2=0x7fec08102660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:32.828 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.828+0000 7fec0f000640 1 --2- 172.21.15.83:0/1983897458 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec08102280 0x7fec08102660 secure :-1 s=READY pgs=2496 cs=0 l=1 rev1=1 crypto rx=0x7fec00009ca0 tx=0x7fec0002a8c0 comp rx=0 tx=0).stop 2024-04-24T09:11:32.828 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.829+0000 7fec0f000640 1 -- 172.21.15.83:0/1983897458 shutdown_connections 2024-04-24T09:11:32.829 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.829+0000 7fec0f000640 1 --2- 172.21.15.83:0/1983897458 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec08107350 0x7fec0810ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:32.829 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.829+0000 7fec0f000640 1 --2- 172.21.15.83:0/1983897458 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fec08102c30 0x7fec08106c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:32.829 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.829+0000 7fec0f000640 1 --2- 172.21.15.83:0/1983897458 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec08102280 0x7fec08102660 unknown :-1 s=CLOSED pgs=2496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:32.829 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.829+0000 7fec0f000640 1 -- 172.21.15.83:0/1983897458 >> 172.21.15.83:0/1983897458 conn(0x7fec08079490 msgr2=0x7fec080798a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:32.829 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.829+0000 7fec0f000640 1 -- 172.21.15.83:0/1983897458 shutdown_connections 2024-04-24T09:11:32.829 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.829+0000 7fec0f000640 1 -- 172.21.15.83:0/1983897458 wait complete. 2024-04-24T09:11:32.830 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.831+0000 7fec0f000640 1 Processor -- start 2024-04-24T09:11:32.832 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.832+0000 7fec0f000640 1 -- start start 2024-04-24T09:11:32.832 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.832+0000 7fec0f000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec08107350 0x7fec0819e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:32.832 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.832+0000 7fec0f000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec0819ebe0 0x7fec08198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:32.832 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.832+0000 7fec0f000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fec08198ca0 0x7fec08199150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:32.832 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.832+0000 7fec0f000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fec081122e0 con 0x7fec08107350 2024-04-24T09:11:32.832 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.832+0000 7fec0f000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fec08112160 con 0x7fec0819ebe0 2024-04-24T09:11:32.832 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.832+0000 7fec0f000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fec08112460 con 0x7fec08198ca0 2024-04-24T09:11:32.833 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.833+0000 7fec07e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec08107350 0x7fec0819e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:32.833 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.833+0000 7fec07e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec08107350 0x7fec0819e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55198/0 (socket says 172.21.15.83:55198) 2024-04-24T09:11:32.833 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.833+0000 7fec07e00640 1 -- 172.21.15.83:0/879143958 learned_addr learned my addr 172.21.15.83:0/879143958 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:11:32.833 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.833+0000 7fec0d000640 1 --2- 172.21.15.83:0/879143958 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fec08198ca0 0x7fec08199150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:32.833 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.834+0000 7fec07e00640 1 -- 172.21.15.83:0/879143958 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fec08198ca0 msgr2=0x7fec08199150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:32.833 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.834+0000 7fec07e00640 1 --2- 172.21.15.83:0/879143958 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fec08198ca0 0x7fec08199150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:32.833 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.834+0000 7fec07e00640 1 -- 172.21.15.83:0/879143958 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec0819ebe0 msgr2=0x7fec08198760 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-24T09:11:32.833 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.834+0000 7fec07e00640 1 --2- 172.21.15.83:0/879143958 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec0819ebe0 0x7fec08198760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:32.833 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.834+0000 7fec07e00640 1 -- 172.21.15.83:0/879143958 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec0806ba20 con 0x7fec08107350 2024-04-24T09:11:32.833 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.834+0000 7fec07e00640 1 --2- 172.21.15.83:0/879143958 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec08107350 0x7fec0819e6a0 secure :-1 s=READY pgs=2497 cs=0 l=1 rev1=1 crypto rx=0x7fec00035000 tx=0x7fec00031700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:32.834 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.835+0000 7fec04c00640 1 -- 172.21.15.83:0/879143958 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec00031e50 con 0x7fec08107350 2024-04-24T09:11:32.834 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.835+0000 7fec04c00640 1 -- 172.21.15.83:0/879143958 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fec00040030 con 0x7fec08107350 2024-04-24T09:11:32.834 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.835+0000 7fec04c00640 1 -- 172.21.15.83:0/879143958 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec00004710 con 0x7fec08107350 2024-04-24T09:11:32.835 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.835+0000 7fec0f000640 1 -- 172.21.15.83:0/879143958 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec0806bcb0 con 0x7fec08107350 2024-04-24T09:11:32.835 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.836+0000 7fec0f000640 1 -- 172.21.15.83:0/879143958 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec0806c1b0 con 0x7fec08107350 2024-04-24T09:11:32.839 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.840+0000 7fec11dd1740 1 Processor -- start 2024-04-24T09:11:32.839 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.840+0000 7fec11dd1740 1 -- start start 2024-04-24T09:11:32.840 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.841+0000 7fec11dd1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aacae376f0 0x55aacae36070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:32.840 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.841+0000 7fec11dd1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aacae365b0 0x55aacae36a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:32.840 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.841+0000 7fec11dd1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aacae49a30 0x55aacae4be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:32.840 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.841+0000 7fec11dd1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55aacad9c900 con 0x55aacae49a30 2024-04-24T09:11:32.840 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.841+0000 7fec11dd1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55aacad42f20 con 0x55aacae376f0 2024-04-24T09:11:32.840 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.841+0000 7fec11dd1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55aacad988c0 con 0x55aacae365b0 2024-04-24T09:11:32.840 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.841+0000 7fec0d000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aacae376f0 0x55aacae36070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:32.840 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.841+0000 7fec0d000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aacae376f0 0x55aacae36070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:57572/0 (socket says 172.21.15.83:57572) 2024-04-24T09:11:32.841 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.841+0000 7fec0d000640 1 -- 172.21.15.83:0/2276244858 learned_addr learned my addr 172.21.15.83:0/2276244858 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:11:32.841 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.841+0000 7fec07400640 1 --2- 172.21.15.83:0/2276244858 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aacae49a30 0x55aacae4be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:32.842 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.842+0000 7fec0d000640 1 -- 172.21.15.83:0/2276244858 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aacae365b0 msgr2=0x55aacae36a30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T09:11:32.842 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.842+0000 7fec0d000640 1 --2- 172.21.15.83:0/2276244858 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aacae365b0 0x55aacae36a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:32.842 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.842+0000 7fec0d000640 1 -- 172.21.15.83:0/2276244858 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aacae49a30 msgr2=0x55aacae4be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:32.842 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.842+0000 7fec0d000640 1 --2- 172.21.15.83:0/2276244858 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aacae49a30 0x55aacae4be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:32.842 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.842+0000 7fec0d000640 1 -- 172.21.15.83:0/2276244858 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aacad67e10 con 0x55aacae376f0 2024-04-24T09:11:32.842 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.842+0000 7fec07400640 1 --2- 172.21.15.83:0/2276244858 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aacae49a30 0x55aacae4be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:11:32.843 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.842+0000 7fec04c00640 1 -- 172.21.15.83:0/879143958 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fec00003970 con 0x7fec08107350 2024-04-24T09:11:32.845 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.843+0000 7fec0d000640 1 --2- 172.21.15.83:0/2276244858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aacae376f0 0x55aacae36070 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7febf8009c10 tx=0x7febf8011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=36ff654e96ab2fe7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:32.845 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.843+0000 7febfd600640 1 -- 172.21.15.83:0/2276244858 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf801e070 con 0x55aacae376f0 2024-04-24T09:11:32.845 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.843+0000 7febfd600640 1 -- 172.21.15.83:0/2276244858 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7febf80198e0 con 0x55aacae376f0 2024-04-24T09:11:32.845 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.843+0000 7febfd600640 1 -- 172.21.15.83:0/2276244858 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf800e7e0 con 0x55aacae376f0 2024-04-24T09:11:32.845 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.844+0000 7fec11dd1740 1 -- 172.21.15.83:0/2276244858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aacae376f0 msgr2=0x55aacae36070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:32.845 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.844+0000 7fec11dd1740 1 --2- 172.21.15.83:0/2276244858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aacae376f0 0x55aacae36070 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7febf8009c10 tx=0x7febf8011810 comp rx=0 tx=0).stop 2024-04-24T09:11:32.845 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.844+0000 7fec11dd1740 1 -- 172.21.15.83:0/2276244858 shutdown_connections 2024-04-24T09:11:32.845 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.844+0000 7fec11dd1740 1 --2- 172.21.15.83:0/2276244858 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aacae49a30 0x55aacae4be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:32.845 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.844+0000 7fec11dd1740 1 --2- 172.21.15.83:0/2276244858 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aacae365b0 0x55aacae36a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:32.845 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.844+0000 7fec11dd1740 1 --2- 172.21.15.83:0/2276244858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aacae376f0 0x55aacae36070 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:32.846 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.844+0000 7fec11dd1740 1 -- 172.21.15.83:0/2276244858 >> 172.21.15.83:0/2276244858 conn(0x55aacae326c0 msgr2=0x55aacae32aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:32.846 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.845+0000 7fec11dd1740 1 -- 172.21.15.83:0/2276244858 shutdown_connections 2024-04-24T09:11:32.846 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.845+0000 7fec11dd1740 1 -- 172.21.15.83:0/2276244858 wait complete. 2024-04-24T09:11:32.846 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.845+0000 7fec04c00640 1 --2- 172.21.15.83:0/879143958 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7febec0a3f70 0x7febec0a6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:32.846 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.845+0000 7fec04c00640 1 -- 172.21.15.83:0/879143958 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 9583+0+0 (secure 0 0 0) 0x7fec000f1a90 con 0x7fec08107350 2024-04-24T09:11:32.846 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.845+0000 7fec11dd1740 1 -- start start 2024-04-24T09:11:32.846 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.845+0000 7fec07400640 1 --2- 172.21.15.83:0/879143958 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7febec0a3f70 0x7febec0a6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:32.846 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.846+0000 7fec11dd1740 1 Processor -- start 2024-04-24T09:11:32.847 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.846+0000 7fec11dd1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aacae365b0 0x55aacae39e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:32.847 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.846+0000 7fec11dd1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aacae376f0 0x55aacae3a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:32.847 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.846+0000 7fec11dd1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aacae49a30 0x55aacae3a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:32.847 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.846+0000 7fec11dd1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55aacae4c790 con 0x55aacae49a30 2024-04-24T09:11:32.848 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.846+0000 7fec11dd1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55aacad562c0 con 0x55aacae376f0 2024-04-24T09:11:32.848 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.846+0000 7fec11dd1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55aacae4c910 con 0x55aacae365b0 2024-04-24T09:11:32.848 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.846+0000 7fec11dd1740 10 client.0 _ll_register_callbacks cb 0x55aacaaaea00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:11:32.848 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.847+0000 7fec0d000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aacae49a30 0x55aacae3a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:32.848 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.847+0000 7fec07e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aacae365b0 0x55aacae39e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:32.848 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.847+0000 7fec07e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aacae365b0 0x55aacae39e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:38294/0 (socket says 172.21.15.83:38294) 2024-04-24T09:11:32.848 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.847+0000 7fec07e00640 1 -- 172.21.15.83:0/1648890432 learned_addr learned my addr 172.21.15.83:0/1648890432 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:11:32.848 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.847+0000 7fec07400640 1 --2- 172.21.15.83:0/879143958 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7febec0a3f70 0x7febec0a6430 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x55aacae33ae0 tx=0x7febf000b2e0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:32.848 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.847+0000 7fec07400640 1 --2- 172.21.15.83:0/1648890432 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aacae376f0 0x55aacae3a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:32.848 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.848+0000 7fec07400640 1 -- 172.21.15.83:0/1648890432 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aacae365b0 msgr2=0x55aacae39e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:32.848 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.848+0000 7fec07400640 1 --2- 172.21.15.83:0/1648890432 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aacae365b0 0x55aacae39e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:32.848 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.848+0000 7fec07400640 1 -- 172.21.15.83:0/1648890432 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aacae49a30 msgr2=0x55aacae3a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:32.848 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.848+0000 7fec07400640 1 --2- 172.21.15.83:0/1648890432 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aacae49a30 0x55aacae3a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:32.848 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.848+0000 7fec07400640 1 -- 172.21.15.83:0/1648890432 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aacad66fd0 con 0x55aacae376f0 2024-04-24T09:11:32.848 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.848+0000 7fec07e00640 1 --2- 172.21.15.83:0/1648890432 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aacae365b0 0x55aacae39e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:11:32.849 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.848+0000 7fec07400640 1 --2- 172.21.15.83:0/1648890432 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aacae376f0 0x55aacae3a3c0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fec0806b6e0 tx=0x7febf0004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=122599fffcacd844 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:32.849 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.848+0000 7febf6000640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:11:32.850 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.848+0000 7febf6000640 1 -- 172.21.15.83:0/1648890432 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf0004230 con 0x55aacae376f0 2024-04-24T09:11:32.850 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.849+0000 7febf6000640 1 -- 172.21.15.83:0/1648890432 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 9583+0+0 (secure 0 0 0) 0x7febf00043d0 con 0x55aacae376f0 2024-04-24T09:11:32.850 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.849+0000 7fec11dd1740 1 -- 172.21.15.83:0/1648890432 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7febf80198e0 con 0x55aacae376f0 2024-04-24T09:11:32.850 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.849+0000 7febf6000640 1 -- 172.21.15.83:0/1648890432 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7febf00035f0 con 0x55aacae376f0 2024-04-24T09:11:32.850 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.849+0000 7fec11dd1740 10 client.39503 fetch_fsmap learned FSMap version 61 2024-04-24T09:11:32.850 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.849+0000 7fec11dd1740 1 -- 172.21.15.83:0/1648890432 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x55aacaaaf210 con 0x55aacae376f0 2024-04-24T09:11:32.850 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.850+0000 7febf6000640 1 -- 172.21.15.83:0/1648890432 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7febf80198e0 con 0x55aacae376f0 2024-04-24T09:11:32.851 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.850+0000 7febf6000640 1 -- 172.21.15.83:0/1648890432 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 61) ==== 8362+0+0 (secure 0 0 0) 0x7febf001e910 con 0x55aacae376f0 2024-04-24T09:11:32.851 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.850+0000 7fec11dd1740 10 client.39503 fetch_fsmap finished waiting for FSMap version 61 2024-04-24T09:11:32.851 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.850+0000 7fec11dd1740 10 client.39503 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:11:32.851 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.850+0000 7fec11dd1740 20 client.39503 populate_metadata read hostname 'smithi083' 2024-04-24T09:11:32.851 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.850+0000 7fec11dd1740 1 --2- 172.21.15.83:0/1648890432 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55aacae59dd0 0x55aacae5c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:32.851 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.850+0000 7fec11dd1740 4 client.39503 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:11:32.851 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.850+0000 7fec11dd1740 1 -- 172.21.15.83:0/1648890432 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aacadaa320 con 0x55aacae59dd0 2024-04-24T09:11:32.851 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.851+0000 7fec07e00640 1 --2- 172.21.15.83:0/1648890432 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55aacae59dd0 0x55aacae5c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:32.851 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.851+0000 7fec07e00640 1 --2- 172.21.15.83:0/1648890432 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55aacae59dd0 0x55aacae5c1c0 crc :-1 s=READY pgs=495 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c7e749f72400f9 server_cookie=583e1e7a4b8ee006 in_seq=0 out_seq=0 2024-04-24T09:11:32.851 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.851+0000 7febf6000640 10 client.39503 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:11:32.852 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.853+0000 7febf6000640 1 -- 172.21.15.83:0/1648890432 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55aacadaa320 con 0x55aacae59dd0 2024-04-24T09:11:32.852 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.853+0000 7febf6000640 10 client.39503 handle_command_reply: tid=0 2024-04-24T09:11:32.852 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.853+0000 7fec11dd1740 1 client.39503 shutdown 2024-04-24T09:11:32.853 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.853+0000 7fec11dd1740 1 -- 172.21.15.83:0/1648890432 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aacae376f0 msgr2=0x55aacae3a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:32.853 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.853+0000 7fec11dd1740 1 --2- 172.21.15.83:0/1648890432 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aacae376f0 0x55aacae3a3c0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fec0806b6e0 tx=0x7febf0004730 comp rx=0 tx=0).stop 2024-04-24T09:11:32.853 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.853+0000 7fec11dd1740 1 -- 172.21.15.83:0/1648890432 shutdown_connections 2024-04-24T09:11:32.853 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.853+0000 7fec11dd1740 1 --2- 172.21.15.83:0/1648890432 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55aacae59dd0 0x55aacae5c1c0 crc :-1 s=READY pgs=495 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:32.853 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.853+0000 7fec11dd1740 1 --2- 172.21.15.83:0/1648890432 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aacae49a30 0x55aacae3a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:32.853 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.853+0000 7fec11dd1740 1 --2- 172.21.15.83:0/1648890432 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aacae376f0 0x55aacae3a3c0 secure :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fec0806b6e0 tx=0x7febf0004730 comp rx=0 tx=0).stop 2024-04-24T09:11:32.853 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.853+0000 7fec11dd1740 1 --2- 172.21.15.83:0/1648890432 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aacae365b0 0x55aacae39e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:32.853 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.853+0000 7fec11dd1740 1 -- 172.21.15.83:0/1648890432 >> 172.21.15.83:0/1648890432 conn(0x55aacae326c0 msgr2=0x55aacae38510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:32.853 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.853+0000 7febf6000640 0 client.39503 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:11:32.854 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.854+0000 7fec11dd1740 1 -- 172.21.15.83:0/1648890432 shutdown_connections 2024-04-24T09:11:32.854 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.854+0000 7fec11dd1740 1 -- 172.21.15.83:0/1648890432 wait complete. 2024-04-24T09:11:32.854 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.854+0000 7fec11dd1740 20 client.39503 trim_cache size 0 max 16384 2024-04-24T09:11:32.881 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.882+0000 7fec11dd1740 1 Processor -- start 2024-04-24T09:11:32.881 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.882+0000 7fec11dd1740 1 -- start start 2024-04-24T09:11:32.881 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.882+0000 7fec11dd1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aacae365b0 0x55aacae3c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:32.882 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.882+0000 7fec11dd1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aacae49a30 0x55aacae556c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:32.882 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.882+0000 7fec11dd1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aacae3a780 0x55aacae38c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:32.882 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.882+0000 7fec11dd1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fec080761c0 con 0x55aacae365b0 2024-04-24T09:11:32.882 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.882+0000 7fec11dd1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fec081b7d20 con 0x55aacae49a30 2024-04-24T09:11:32.882 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.882+0000 7fec11dd1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fec081b68b0 con 0x55aacae3a780 2024-04-24T09:11:32.884 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.883+0000 7fec07400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aacae49a30 0x55aacae556c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:32.884 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.883+0000 7fec07400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aacae49a30 0x55aacae556c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:57580/0 (socket says 172.21.15.83:57580) 2024-04-24T09:11:32.884 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.883+0000 7fec07400640 1 -- 172.21.15.83:0/3868880113 learned_addr learned my addr 172.21.15.83:0/3868880113 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:11:32.884 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.883+0000 7fec07e00640 1 --2- 172.21.15.83:0/3868880113 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aacae365b0 0x55aacae3c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:32.884 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.884+0000 7fec0d000640 1 --2- 172.21.15.83:0/3868880113 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aacae3a780 0x55aacae38c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:32.884 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.884+0000 7fec07e00640 1 -- 172.21.15.83:0/3868880113 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aacae3a780 msgr2=0x55aacae38c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:32.884 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.884+0000 7fec07e00640 1 --2- 172.21.15.83:0/3868880113 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aacae3a780 0x55aacae38c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:32.884 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.884+0000 7fec07e00640 1 -- 172.21.15.83:0/3868880113 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aacae49a30 msgr2=0x55aacae556c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:32.885 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.884+0000 7fec07e00640 1 --2- 172.21.15.83:0/3868880113 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aacae49a30 0x55aacae556c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:32.885 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.884+0000 7fec07e00640 1 -- 172.21.15.83:0/3868880113 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aacab7b620 con 0x55aacae365b0 2024-04-24T09:11:32.885 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.885+0000 7fec07e00640 1 --2- 172.21.15.83:0/3868880113 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aacae365b0 0x55aacae3c650 secure :-1 s=READY pgs=2498 cs=0 l=1 rev1=1 crypto rx=0x7fec000bd3c0 tx=0x7fec0003e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f5efcbe4ccc450d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:32.885 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.885+0000 7febfcc00640 1 -- 172.21.15.83:0/3868880113 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec000bd200 con 0x55aacae365b0 2024-04-24T09:11:32.885 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.886+0000 7febfcc00640 1 -- 172.21.15.83:0/3868880113 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fec00036020 con 0x55aacae365b0 2024-04-24T09:11:32.885 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.886+0000 7febfcc00640 1 -- 172.21.15.83:0/3868880113 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec000bd020 con 0x55aacae365b0 2024-04-24T09:11:32.886 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.887+0000 7fec11dd1740 1 -- 172.21.15.83:0/3868880113 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aacae365b0 msgr2=0x55aacae3c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:32.886 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.887+0000 7fec11dd1740 1 --2- 172.21.15.83:0/3868880113 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aacae365b0 0x55aacae3c650 secure :-1 s=READY pgs=2498 cs=0 l=1 rev1=1 crypto rx=0x7fec000bd3c0 tx=0x7fec0003e040 comp rx=0 tx=0).stop 2024-04-24T09:11:32.886 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.887+0000 7fec11dd1740 1 -- 172.21.15.83:0/3868880113 shutdown_connections 2024-04-24T09:11:32.886 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.887+0000 7fec11dd1740 1 --2- 172.21.15.83:0/3868880113 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aacae3a780 0x55aacae38c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:32.886 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.887+0000 7fec11dd1740 1 --2- 172.21.15.83:0/3868880113 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aacae49a30 0x55aacae556c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:32.886 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.887+0000 7fec11dd1740 1 --2- 172.21.15.83:0/3868880113 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aacae365b0 0x55aacae3c650 secure :-1 s=CLOSED pgs=2498 cs=0 l=1 rev1=1 crypto rx=0x7fec000bd3c0 tx=0x7fec0003e040 comp rx=0 tx=0).stop 2024-04-24T09:11:32.887 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.887+0000 7fec11dd1740 1 -- 172.21.15.83:0/3868880113 >> 172.21.15.83:0/3868880113 conn(0x55aacae326c0 msgr2=0x55aacae3bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:32.887 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.888+0000 7fec11dd1740 1 -- 172.21.15.83:0/3868880113 shutdown_connections 2024-04-24T09:11:32.887 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.888+0000 7fec11dd1740 1 -- 172.21.15.83:0/3868880113 wait complete. 2024-04-24T09:11:32.888 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.889+0000 7fec11dd1740 1 -- start start 2024-04-24T09:11:32.888 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.889+0000 7fec11dd1740 1 Processor -- start 2024-04-24T09:11:32.890 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.891+0000 7fec11dd1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aacae3a780 0x55aacae51f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:32.890 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.891+0000 7fec11dd1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aacae49a30 0x55aacae46db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:32.890 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.891+0000 7fec11dd1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aacae472f0 0x55aacae47770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:32.891 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.892+0000 7fec11dd1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fec081b7650 con 0x55aacae472f0 2024-04-24T09:11:32.891 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.892+0000 7fec11dd1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fec081b6f80 con 0x55aacae3a780 2024-04-24T09:11:32.891 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.892+0000 7fec11dd1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fec080741c0 con 0x55aacae49a30 2024-04-24T09:11:32.891 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.892+0000 7fec11dd1740 10 client.0 _ll_register_callbacks cb 0x7febf000b740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:11:32.891 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.892+0000 7fec07400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aacae49a30 0x55aacae46db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:32.892 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.892+0000 7fec07400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aacae49a30 0x55aacae46db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:38314/0 (socket says 172.21.15.83:38314) 2024-04-24T09:11:32.892 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.892+0000 7fec07400640 1 -- 172.21.15.83:0/108459945 learned_addr learned my addr 172.21.15.83:0/108459945 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:11:32.892 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.892+0000 7fec07e00640 1 --2- 172.21.15.83:0/108459945 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aacae3a780 0x55aacae51f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:32.892 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.893+0000 7fec0d000640 1 --2- 172.21.15.83:0/108459945 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aacae472f0 0x55aacae47770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:32.893 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.893+0000 7fec0d000640 1 -- 172.21.15.83:0/108459945 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aacae49a30 msgr2=0x55aacae46db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:32.894 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.893+0000 7fec0d000640 1 --2- 172.21.15.83:0/108459945 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aacae49a30 0x55aacae46db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:32.894 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.893+0000 7fec0d000640 1 -- 172.21.15.83:0/108459945 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aacae3a780 msgr2=0x55aacae51f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:32.894 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.893+0000 7fec0d000640 1 --2- 172.21.15.83:0/108459945 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aacae3a780 0x55aacae51f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:32.894 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.894+0000 7fec0d000640 1 -- 172.21.15.83:0/108459945 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aacae47fa0 con 0x55aacae472f0 2024-04-24T09:11:32.895 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.896+0000 7fec0d000640 1 --2- 172.21.15.83:0/108459945 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aacae472f0 0x55aacae47770 secure :-1 s=READY pgs=2499 cs=0 l=1 rev1=1 crypto rx=0x7febf8005890 tx=0x7febf8005630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d43b188796a7a37 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:32.896 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.896+0000 7febf6000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:11:32.896 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.897+0000 7febf6000640 1 -- 172.21.15.83:0/108459945 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf801e040 con 0x55aacae472f0 2024-04-24T09:11:32.896 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.897+0000 7fec11dd1740 1 -- 172.21.15.83:0/108459945 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec00036020 con 0x55aacae472f0 2024-04-24T09:11:32.896 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.897+0000 7febf6000640 1 -- 172.21.15.83:0/108459945 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 9583+0+0 (secure 0 0 0) 0x7febf80037f0 con 0x55aacae472f0 2024-04-24T09:11:32.899 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.900+0000 7febf6000640 1 -- 172.21.15.83:0/108459945 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7febf8009860 con 0x55aacae472f0 2024-04-24T09:11:32.899 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.900+0000 7fec11dd1740 10 client.22428 fetch_fsmap learned FSMap version 61 2024-04-24T09:11:32.899 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.900+0000 7fec11dd1740 1 -- 172.21.15.83:0/108459945 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x55aacaec16d0 con 0x55aacae472f0 2024-04-24T09:11:32.900 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.901+0000 7febf6000640 1 -- 172.21.15.83:0/108459945 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fec00036020 con 0x55aacae472f0 2024-04-24T09:11:32.900 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.901+0000 7febf6000640 1 -- 172.21.15.83:0/108459945 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 61) ==== 8362+0+0 (secure 0 0 0) 0x7febf802a030 con 0x55aacae472f0 2024-04-24T09:11:32.901 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.902+0000 7fec11dd1740 10 client.22428 fetch_fsmap finished waiting for FSMap version 61 2024-04-24T09:11:32.901 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.902+0000 7fec11dd1740 10 client.22428 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:11:32.901 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.902+0000 7fec11dd1740 20 client.22428 populate_metadata read hostname 'smithi083' 2024-04-24T09:11:32.901 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.902+0000 7fec11dd1740 1 --2- 172.21.15.83:0/108459945 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55aacaec1c10 0x55aacaec4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:32.901 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.902+0000 7fec11dd1740 4 client.22428 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T09:11:32.901 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.902+0000 7fec11dd1740 1 -- 172.21.15.83:0/108459945 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55aacada4190 con 0x55aacaec1c10 2024-04-24T09:11:32.902 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.903+0000 7fec07e00640 1 --2- 172.21.15.83:0/108459945 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55aacaec1c10 0x55aacaec4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:32.904 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.904+0000 7fec07e00640 1 --2- 172.21.15.83:0/108459945 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55aacaec1c10 0x55aacaec4000 crc :-1 s=READY pgs=496 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b73f95b482044c3 server_cookie=4e2e413344bd4888 in_seq=0 out_seq=0 2024-04-24T09:11:32.904 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:32.904+0000 7febf6000640 10 client.22428 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:11:32.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.921+0000 7f719c200640 1 -- 172.21.15.83:0/1082218224 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7194102bf0 msgr2=0x7f7194106c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:32.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.921+0000 7f719c200640 1 --2- 172.21.15.83:0/1082218224 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7194102bf0 0x7f7194106c30 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f7180009900 tx=0x7f718002a400 comp rx=0 tx=0).stop 2024-04-24T09:11:32.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.921+0000 7f719c200640 1 -- 172.21.15.83:0/1082218224 shutdown_connections 2024-04-24T09:11:32.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.921+0000 7f719c200640 1 --2- 172.21.15.83:0/1082218224 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7194107310 0x7f719410eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:32.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.921+0000 7f719c200640 1 --2- 172.21.15.83:0/1082218224 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7194102bf0 0x7f7194106c30 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:32.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.921+0000 7f719c200640 1 --2- 172.21.15.83:0/1082218224 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7194102240 0x7f7194102620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:32.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.921+0000 7f719c200640 1 -- 172.21.15.83:0/1082218224 >> 172.21.15.83:0/1082218224 conn(0x7f71940793c0 msgr2=0x7f71940797d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:32.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.921+0000 7f719c200640 1 -- 172.21.15.83:0/1082218224 shutdown_connections 2024-04-24T09:11:32.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.921+0000 7f719c200640 1 -- 172.21.15.83:0/1082218224 wait complete. 2024-04-24T09:11:32.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.923+0000 7f719c200640 1 Processor -- start 2024-04-24T09:11:32.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.923+0000 7f719c200640 1 -- start start 2024-04-24T09:11:32.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.923+0000 7f719c200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7194102240 0x7f719419e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:32.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.923+0000 7f719c200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7194107310 0x7f719419eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:32.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.923+0000 7f719c200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7194198640 0x7f7194198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:32.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.923+0000 7f719c200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7194112280 con 0x7f7194198640 2024-04-24T09:11:32.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.923+0000 7f719c200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7194112100 con 0x7f7194102240 2024-04-24T09:11:32.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.923+0000 7f719c200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7194112400 con 0x7f7194107310 2024-04-24T09:11:32.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.924+0000 7f719a200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7194198640 0x7f7194198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:32.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.924+0000 7f7199800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7194102240 0x7f719419e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:32.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.924+0000 7f719a200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7194198640 0x7f7194198ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55258/0 (socket says 172.21.15.83:55258) 2024-04-24T09:11:32.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.924+0000 7f719a200640 1 -- 172.21.15.83:0/2164636584 learned_addr learned my addr 172.21.15.83:0/2164636584 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:11:32.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.924+0000 7f7199800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7194102240 0x7f719419e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:57606/0 (socket says 172.21.15.83:57606) 2024-04-24T09:11:32.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.924+0000 7f7199800640 1 -- 172.21.15.83:0/2164636584 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7194107310 msgr2=0x7f719419eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:32.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.924+0000 7f7199800640 1 --2- 172.21.15.83:0/2164636584 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7194107310 0x7f719419eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:32.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.924+0000 7f7199800640 1 -- 172.21.15.83:0/2164636584 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7194198640 msgr2=0x7f7194198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:32.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.924+0000 7f7199800640 1 --2- 172.21.15.83:0/2164636584 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7194198640 0x7f7194198ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:32.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.924+0000 7f7199800640 1 -- 172.21.15.83:0/2164636584 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7194199240 con 0x7f7194102240 2024-04-24T09:11:32.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.925+0000 7f7199800640 1 --2- 172.21.15.83:0/2164636584 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7194102240 0x7f719419e580 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f718800e9f0 tx=0x7f718800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:32.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.926+0000 7f7192000640 1 -- 172.21.15.83:0/2164636584 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f718800ccc0 con 0x7f7194102240 2024-04-24T09:11:32.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.926+0000 7f7192000640 1 -- 172.21.15.83:0/2164636584 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f71880108e0 con 0x7f7194102240 2024-04-24T09:11:32.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.926+0000 7f7192000640 1 -- 172.21.15.83:0/2164636584 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7188018880 con 0x7f7194102240 2024-04-24T09:11:32.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.926+0000 7f719c200640 1 -- 172.21.15.83:0/2164636584 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7194074230 con 0x7f7194102240 2024-04-24T09:11:32.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.926+0000 7f719c200640 1 -- 172.21.15.83:0/2164636584 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7194074730 con 0x7f7194102240 2024-04-24T09:11:32.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.927+0000 7f719c200640 1 -- 172.21.15.83:0/2164636584 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7194103710 con 0x7f7194102240 2024-04-24T09:11:32.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.931+0000 7f7192000640 1 -- 172.21.15.83:0/2164636584 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7188010a90 con 0x7f7194102240 2024-04-24T09:11:32.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.934+0000 7f7192000640 1 --2- 172.21.15.83:0/2164636584 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f717c0a3ef0 0x7f717c0a63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:32.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.934+0000 7f7192000640 1 -- 172.21.15.83:0/2164636584 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 9583+0+0 (secure 0 0 0) 0x7f71880d13d0 con 0x7f7194102240 2024-04-24T09:11:32.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.935+0000 7f7198e00640 1 --2- 172.21.15.83:0/2164636584 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f717c0a3ef0 0x7f717c0a63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:32.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.937+0000 7f7198e00640 1 --2- 172.21.15.83:0/2164636584 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f717c0a3ef0 0x7f717c0a63b0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f718002a920 tx=0x7f7180031400 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:32.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:32.939+0000 7f7192000640 1 -- 172.21.15.83:0/2164636584 <== mon.2 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f718809d670 con 0x7f7194102240 2024-04-24T09:11:33.033 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:33.032+0000 7febf6000640 1 -- 172.21.15.83:0/108459945 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55aacada4190 con 0x55aacaec1c10 2024-04-24T09:11:33.033 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:33.032+0000 7febf6000640 10 client.22428 handle_command_reply: tid=0 2024-04-24T09:11:33.033 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:33.034+0000 7fec11dd1740 1 client.22428 shutdown 2024-04-24T09:11:33.033 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:33.034+0000 7fec11dd1740 1 -- 172.21.15.83:0/108459945 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aacae472f0 msgr2=0x55aacae47770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:33.033 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:33.034+0000 7fec11dd1740 1 --2- 172.21.15.83:0/108459945 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aacae472f0 0x55aacae47770 secure :-1 s=READY pgs=2499 cs=0 l=1 rev1=1 crypto rx=0x7febf8005890 tx=0x7febf8005630 comp rx=0 tx=0).stop 2024-04-24T09:11:33.034 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:33.035+0000 7fec11dd1740 1 -- 172.21.15.83:0/108459945 shutdown_connections 2024-04-24T09:11:33.034 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:33.035+0000 7fec11dd1740 1 --2- 172.21.15.83:0/108459945 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55aacaec1c10 0x55aacaec4000 crc :-1 s=READY pgs=496 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:33.034 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:33.035+0000 7fec11dd1740 1 --2- 172.21.15.83:0/108459945 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aacae472f0 0x55aacae47770 unknown :-1 s=CLOSED pgs=2499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:33.034 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:33.035+0000 7fec11dd1740 1 --2- 172.21.15.83:0/108459945 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aacae49a30 0x55aacae46db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:33.034 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:33.035+0000 7fec11dd1740 1 --2- 172.21.15.83:0/108459945 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aacae3a780 0x55aacae51f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:33.034 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:33.035+0000 7fec11dd1740 1 -- 172.21.15.83:0/108459945 >> 172.21.15.83:0/108459945 conn(0x55aacae326c0 msgr2=0x55aacae3b6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:33.035 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:33.035+0000 7febf6000640 0 client.22428 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:11:33.035 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:33.036+0000 7fec11dd1740 1 -- 172.21.15.83:0/108459945 shutdown_connections 2024-04-24T09:11:33.035 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:33.036+0000 7fec11dd1740 1 -- 172.21.15.83:0/108459945 wait complete. 2024-04-24T09:11:33.035 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:33.036+0000 7fec11dd1740 20 client.22428 trim_cache size 0 max 16384 2024-04-24T09:11:33.036 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T09:11:33.036 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T09:11:33.036 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T09:11:33.036 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T09:11:33.038 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:33.038+0000 7fec0f000640 1 -- 172.21.15.83:0/879143958 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7febec0a3f70 msgr2=0x7febec0a6430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:33.038 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:33.038+0000 7fec0f000640 1 --2- 172.21.15.83:0/879143958 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7febec0a3f70 0x7febec0a6430 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x55aacae33ae0 tx=0x7febf000b2e0 comp rx=0 tx=0).stop 2024-04-24T09:11:33.038 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:33.038+0000 7fec0f000640 1 -- 172.21.15.83:0/879143958 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec08107350 msgr2=0x7fec0819e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:33.038 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:33.038+0000 7fec0f000640 1 --2- 172.21.15.83:0/879143958 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec08107350 0x7fec0819e6a0 secure :-1 s=READY pgs=2497 cs=0 l=1 rev1=1 crypto rx=0x7fec00035000 tx=0x7fec00031700 comp rx=0 tx=0).stop 2024-04-24T09:11:33.039 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:33.039+0000 7fec0f000640 1 -- 172.21.15.83:0/879143958 shutdown_connections 2024-04-24T09:11:33.039 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:33.039+0000 7fec0f000640 1 --2- 172.21.15.83:0/879143958 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7febec0a3f70 0x7febec0a6430 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:33.039 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:33.039+0000 7fec0f000640 1 --2- 172.21.15.83:0/879143958 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fec08198ca0 0x7fec08199150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:33.039 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:33.039+0000 7fec0f000640 1 --2- 172.21.15.83:0/879143958 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec0819ebe0 0x7fec08198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:33.040 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:33.039+0000 7fec0f000640 1 --2- 172.21.15.83:0/879143958 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec08107350 0x7fec0819e6a0 unknown :-1 s=CLOSED pgs=2497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:33.040 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:33.039+0000 7fec0f000640 1 -- 172.21.15.83:0/879143958 >> 172.21.15.83:0/879143958 conn(0x7fec08079490 msgr2=0x7fec080fff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:33.040 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:33.039+0000 7fec0f000640 1 -- 172.21.15.83:0/879143958 shutdown_connections 2024-04-24T09:11:33.040 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:33.040+0000 7fec0f000640 1 -- 172.21.15.83:0/879143958 wait complete. 2024-04-24T09:11:33.181 INFO:tasks.workunit.client.0.smithi083.stdout: 10 45810 6.70 MB/sec execute 244 sec latency 132.142 ms 2024-04-24T09:11:33.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.357+0000 7f719c200640 1 -- 172.21.15.83:0/2164636584 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7194069270 con 0x7f7194102240 2024-04-24T09:11:33.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.358+0000 7f7192000640 1 -- 172.21.15.83:0/2164636584 <== mon.2 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) ==== 94+0+11944 (secure 0 0 0) 0x7f718809cf70 con 0x7f7194102240 2024-04-24T09:11:33.358 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T09:11:33.358 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":61,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/358872257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":358872257},{"type":"v1","addr":"172.21.15.83:6835","nonce":358872257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3157070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3157070851},{"type":"v1","addr":"172.21.15.83:6837","nonce":3157070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":61,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T08:48:29.422576+0000","modified":"2024-04-24T09:09:34.468831+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34520,"mds_1":34493,"mds_2":34451,"mds_3":34427,"mds_4":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14658":{"gid":14658,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6839/4085470844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":4085470844},{"type":"v1","addr":"172.21.15.83:6839","nonce":4085470844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14676":{"gid":14676,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6841/2634754210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":2634754210},{"type":"v1","addr":"172.21.15.83:6841","nonce":2634754210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24382":{"gid":24382,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/1859921310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1859921310},{"type":"v1","addr":"172.21.15.189:6833","nonce":1859921310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/2241923197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2241923197},{"type":"v1","addr":"172.21.15.189:6835","nonce":2241923197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.189:6839/2548965336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":2548965336},{"type":"v1","addr":"172.21.15.189:6839","nonce":2548965336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26686":{"gid":26686,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6837/1678817161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":1678817161},{"type":"v1","addr":"172.21.15.189:6837","nonce":1678817161}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.138:6833/1157967658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1157967658},{"type":"v1","addr":"172.21.15.138:6833","nonce":1157967658}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.138:6835/1909219758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1909219758},{"type":"v1","addr":"172.21.15.138:6835","nonce":1909219758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34493":{"gid":34493,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.138:6837/900566754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":900566754},{"type":"v1","addr":"172.21.15.138:6837","nonce":900566754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34520":{"gid":34520,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.138:6839/2236932393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2236932393},{"type":"v1","addr":"172.21.15.138:6839","nonce":2236932393}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34520,"qdb_cluster":[24454,34520,34427,34451,34493]},"id":1}]} 2024-04-24T09:11:33.358 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 61 2024-04-24T09:11:33.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.362+0000 7f719c200640 1 -- 172.21.15.83:0/2164636584 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f717c0a3ef0 msgr2=0x7f717c0a63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:33.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.362+0000 7f719c200640 1 --2- 172.21.15.83:0/2164636584 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f717c0a3ef0 0x7f717c0a63b0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f718002a920 tx=0x7f7180031400 comp rx=0 tx=0).stop 2024-04-24T09:11:33.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.362+0000 7f719c200640 1 -- 172.21.15.83:0/2164636584 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7194102240 msgr2=0x7f719419e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:33.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.362+0000 7f719c200640 1 --2- 172.21.15.83:0/2164636584 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7194102240 0x7f719419e580 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f718800e9f0 tx=0x7f718800eeb0 comp rx=0 tx=0).stop 2024-04-24T09:11:33.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.362+0000 7f719c200640 1 -- 172.21.15.83:0/2164636584 shutdown_connections 2024-04-24T09:11:33.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.362+0000 7f719c200640 1 --2- 172.21.15.83:0/2164636584 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f717c0a3ef0 0x7f717c0a63b0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:33.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.362+0000 7f719c200640 1 --2- 172.21.15.83:0/2164636584 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7194198640 0x7f7194198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:33.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.362+0000 7f719c200640 1 --2- 172.21.15.83:0/2164636584 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7194107310 0x7f719419eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:33.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.362+0000 7f719c200640 1 --2- 172.21.15.83:0/2164636584 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7194102240 0x7f719419e580 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:33.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.362+0000 7f719c200640 1 -- 172.21.15.83:0/2164636584 >> 172.21.15.83:0/2164636584 conn(0x7f71940793c0 msgr2=0x7f7194100310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:33.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.362+0000 7f719c200640 1 -- 172.21.15.83:0/2164636584 shutdown_connections 2024-04-24T09:11:33.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.363+0000 7f719c200640 1 -- 172.21.15.83:0/2164636584 wait complete. 2024-04-24T09:11:33.387 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-24T09:11:33.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.651+0000 7f1b2ae00640 1 -- 172.21.15.83:0/3718378556 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1b241044b0 msgr2=0x7f1b24104890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:33.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.651+0000 7f1b2ae00640 1 --2- 172.21.15.83:0/3718378556 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1b241044b0 0x7f1b24104890 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1b14009900 tx=0x7f1b1402a400 comp rx=0 tx=0).stop 2024-04-24T09:11:33.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.651+0000 7f1b2ae00640 1 -- 172.21.15.83:0/3718378556 shutdown_connections 2024-04-24T09:11:33.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.651+0000 7f1b2ae00640 1 --2- 172.21.15.83:0/3718378556 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1b24109580 0x7f1b24111150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:33.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.651+0000 7f1b2ae00640 1 --2- 172.21.15.83:0/3718378556 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b24104e60 0x7f1b24108ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:33.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.651+0000 7f1b2ae00640 1 --2- 172.21.15.83:0/3718378556 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1b241044b0 0x7f1b24104890 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:33.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.651+0000 7f1b2ae00640 1 -- 172.21.15.83:0/3718378556 >> 172.21.15.83:0/3718378556 conn(0x7f1b240ffd90 msgr2=0x7f1b241021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:33.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.651+0000 7f1b2ae00640 1 -- 172.21.15.83:0/3718378556 shutdown_connections 2024-04-24T09:11:33.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.651+0000 7f1b2ae00640 1 -- 172.21.15.83:0/3718378556 wait complete. 2024-04-24T09:11:33.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.652+0000 7f1b2ae00640 1 Processor -- start 2024-04-24T09:11:33.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.653+0000 7f1b2ae00640 1 -- start start 2024-04-24T09:11:33.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.653+0000 7f1b2ae00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1b241044b0 0x7f1b24111150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:33.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.653+0000 7f1b2ae00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1b24109580 0x7f1b2410e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:33.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.653+0000 7f1b2ae00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b2410e8e0 0x7f1b241ab200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:33.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.653+0000 7f1b2ae00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1b241144c0 con 0x7f1b2410e8e0 2024-04-24T09:11:33.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.653+0000 7f1b2ae00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1b24114340 con 0x7f1b241044b0 2024-04-24T09:11:33.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.653+0000 7f1b2ae00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1b24114640 con 0x7f1b24109580 2024-04-24T09:11:33.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.654+0000 7f1b1be00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1b24109580 0x7f1b2410e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:33.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.654+0000 7f1b1be00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1b24109580 0x7f1b2410e240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:38352/0 (socket says 172.21.15.83:38352) 2024-04-24T09:11:33.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.654+0000 7f1b1be00640 1 -- 172.21.15.83:0/3165454560 learned_addr learned my addr 172.21.15.83:0/3165454560 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:11:33.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.654+0000 7f1b28e00640 1 --2- 172.21.15.83:0/3165454560 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b2410e8e0 0x7f1b241ab200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:33.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.654+0000 7f1b23e00640 1 --2- 172.21.15.83:0/3165454560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1b241044b0 0x7f1b24111150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:33.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.655+0000 7f1b28e00640 1 -- 172.21.15.83:0/3165454560 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1b24109580 msgr2=0x7f1b2410e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:33.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.655+0000 7f1b28e00640 1 --2- 172.21.15.83:0/3165454560 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1b24109580 0x7f1b2410e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:33.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.655+0000 7f1b28e00640 1 -- 172.21.15.83:0/3165454560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1b241044b0 msgr2=0x7f1b24111150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:33.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.655+0000 7f1b28e00640 1 --2- 172.21.15.83:0/3165454560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1b241044b0 0x7f1b24111150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:33.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.655+0000 7f1b28e00640 1 -- 172.21.15.83:0/3165454560 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b241ab740 con 0x7f1b2410e8e0 2024-04-24T09:11:33.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.656+0000 7f1b28e00640 1 --2- 172.21.15.83:0/3165454560 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b2410e8e0 0x7f1b241ab200 secure :-1 s=READY pgs=2500 cs=0 l=1 rev1=1 crypto rx=0x7f1b1c00ec50 tx=0x7f1b1c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:33.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.656+0000 7f1b21600640 1 -- 172.21.15.83:0/3165454560 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b1c003b70 con 0x7f1b2410e8e0 2024-04-24T09:11:33.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.656+0000 7f1b21600640 1 -- 172.21.15.83:0/3165454560 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1b1c003d10 con 0x7f1b2410e8e0 2024-04-24T09:11:33.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.656+0000 7f1b21600640 1 -- 172.21.15.83:0/3165454560 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b1c010760 con 0x7f1b2410e8e0 2024-04-24T09:11:33.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.657+0000 7f1b2ae00640 1 -- 172.21.15.83:0/3165454560 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b241ab910 con 0x7f1b2410e8e0 2024-04-24T09:11:33.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.657+0000 7f1b2ae00640 1 -- 172.21.15.83:0/3165454560 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b241abd80 con 0x7f1b2410e8e0 2024-04-24T09:11:33.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.661+0000 7f1b2dbb2740 1 Processor -- start 2024-04-24T09:11:33.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.662+0000 7f1b21600640 1 -- 172.21.15.83:0/3165454560 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1b1c010900 con 0x7f1b2410e8e0 2024-04-24T09:11:33.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.662+0000 7f1b2dbb2740 1 -- start start 2024-04-24T09:11:33.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.663+0000 7f1b2dbb2740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd54662950 0x55fd54662d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:33.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.663+0000 7f1b2dbb2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd54663270 0x55fd5466d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:33.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.663+0000 7f1b2dbb2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fd5466d940 0x55fd5466fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:33.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.663+0000 7f1b2dbb2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55fd545c81d0 con 0x55fd54663270 2024-04-24T09:11:33.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.663+0000 7f1b2dbb2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55fd5456e450 con 0x55fd5466d940 2024-04-24T09:11:33.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.663+0000 7f1b2dbb2740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55fd545c4190 con 0x55fd54662950 2024-04-24T09:11:33.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.664+0000 7f1b1be00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd54662950 0x55fd54662d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:33.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.664+0000 7f1b23e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd54663270 0x55fd5466d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:33.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.664+0000 7f1b1be00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd54662950 0x55fd54662d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:38362/0 (socket says 172.21.15.83:38362) 2024-04-24T09:11:33.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.664+0000 7f1b1be00640 1 -- 172.21.15.83:0/99294893 learned_addr learned my addr 172.21.15.83:0/99294893 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:11:33.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.664+0000 7f1b1be00640 1 --2- 172.21.15.83:0/99294893 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fd5466d940 0x55fd5466fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:33.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.664+0000 7f1b21600640 1 --2- 172.21.15.83:0/3165454560 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f1b080a3e70 0x7f1b080a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:33.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.665+0000 7f1b21600640 1 -- 172.21.15.83:0/3165454560 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 9583+0+0 (secure 0 0 0) 0x7f1b1c0d0f40 con 0x7f1b2410e8e0 2024-04-24T09:11:33.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.665+0000 7f1b1be00640 1 -- 172.21.15.83:0/99294893 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fd5466d940 msgr2=0x55fd5466fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:33.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.665+0000 7f1b1be00640 1 --2- 172.21.15.83:0/99294893 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fd5466d940 0x55fd5466fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:33.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.666+0000 7f1b28e00640 1 --2- 172.21.15.83:0/3165454560 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f1b080a3e70 0x7f1b080a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:33.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.666+0000 7f1b1be00640 1 -- 172.21.15.83:0/99294893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd54663270 msgr2=0x55fd5466d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:33.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.666+0000 7f1b1be00640 1 --2- 172.21.15.83:0/99294893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd54663270 0x55fd5466d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:33.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.666+0000 7f1b1be00640 1 -- 172.21.15.83:0/99294893 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd5462b310 con 0x55fd54662950 2024-04-24T09:11:33.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.666+0000 7f1b23e00640 1 --2- 172.21.15.83:0/99294893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd54663270 0x55fd5466d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:11:33.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.666+0000 7f1b28e00640 1 --2- 172.21.15.83:0/3165454560 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f1b080a3e70 0x7f1b080a6330 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f1b1c015040 tx=0x7f1b1c009b50 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:33.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.667+0000 7f1b1be00640 1 --2- 172.21.15.83:0/99294893 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd54662950 0x55fd54662d30 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c012810 tx=0x7f1b0c004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f925ae284d1d740 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:33.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.667+0000 7f1b19600640 1 -- 172.21.15.83:0/99294893 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b0c012d90 con 0x55fd54662950 2024-04-24T09:11:33.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.668+0000 7f1b19600640 1 -- 172.21.15.83:0/99294893 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1b0c010770 con 0x55fd54662950 2024-04-24T09:11:33.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.668+0000 7f1b19600640 1 -- 172.21.15.83:0/99294893 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b0c0044e0 con 0x55fd54662950 2024-04-24T09:11:33.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.669+0000 7f1b2dbb2740 1 -- 172.21.15.83:0/99294893 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd54662950 msgr2=0x55fd54662d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:33.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.669+0000 7f1b2dbb2740 1 --2- 172.21.15.83:0/99294893 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd54662950 0x55fd54662d30 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c012810 tx=0x7f1b0c004730 comp rx=0 tx=0).stop 2024-04-24T09:11:33.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.669+0000 7f1b2dbb2740 1 -- 172.21.15.83:0/99294893 shutdown_connections 2024-04-24T09:11:33.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.669+0000 7f1b2dbb2740 1 --2- 172.21.15.83:0/99294893 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fd5466d940 0x55fd5466fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:33.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.669+0000 7f1b2dbb2740 1 --2- 172.21.15.83:0/99294893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd54663270 0x55fd5466d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:33.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.669+0000 7f1b2dbb2740 1 --2- 172.21.15.83:0/99294893 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd54662950 0x55fd54662d30 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:33.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.669+0000 7f1b2dbb2740 1 -- 172.21.15.83:0/99294893 >> 172.21.15.83:0/99294893 conn(0x55fd5465ec70 msgr2=0x55fd5465f050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:33.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.670+0000 7f1b2dbb2740 1 -- 172.21.15.83:0/99294893 shutdown_connections 2024-04-24T09:11:33.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.670+0000 7f1b2dbb2740 1 -- 172.21.15.83:0/99294893 wait complete. 2024-04-24T09:11:33.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.672+0000 7f1b2dbb2740 1 -- start start 2024-04-24T09:11:33.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.672+0000 7f1b2dbb2740 1 Processor -- start 2024-04-24T09:11:33.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.672+0000 7f1b2dbb2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fd54662950 0x55fd5466d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:33.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.672+0000 7f1b2dbb2740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd54663270 0x55fd5466a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:33.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.672+0000 7f1b2dbb2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd5466d940 0x55fd5466ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:33.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.672+0000 7f1b2dbb2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55fd546817e0 con 0x55fd5466d940 2024-04-24T09:11:33.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.673+0000 7f1b2dbb2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55fd54581810 con 0x55fd54662950 2024-04-24T09:11:33.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.673+0000 7f1b2dbb2740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55fd54681960 con 0x55fd54663270 2024-04-24T09:11:33.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.673+0000 7f1b2dbb2740 10 client.0 _ll_register_callbacks cb 0x55fd5454f140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:11:33.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.673+0000 7f1b1be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd5466d940 0x55fd5466ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:33.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.673+0000 7f1b1be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd5466d940 0x55fd5466ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55286/0 (socket says 172.21.15.83:55286) 2024-04-24T09:11:33.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.673+0000 7f1b1be00640 1 -- 172.21.15.83:0/2327774397 learned_addr learned my addr 172.21.15.83:0/2327774397 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:11:33.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.673+0000 7f1b1be00640 1 --2- 172.21.15.83:0/2327774397 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fd54662950 0x55fd5466d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:33.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.673+0000 7f1b23e00640 1 --2- 172.21.15.83:0/2327774397 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd54663270 0x55fd5466a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:33.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.674+0000 7f1b1be00640 1 -- 172.21.15.83:0/2327774397 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd54663270 msgr2=0x55fd5466a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:33.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.674+0000 7f1b1be00640 1 --2- 172.21.15.83:0/2327774397 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd54663270 0x55fd5466a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:33.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.674+0000 7f1b1be00640 1 -- 172.21.15.83:0/2327774397 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd5466d940 msgr2=0x55fd5466ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:33.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.674+0000 7f1b1be00640 1 --2- 172.21.15.83:0/2327774397 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd5466d940 0x55fd5466ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:33.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.674+0000 7f1b1be00640 1 -- 172.21.15.83:0/2327774397 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd545923f0 con 0x55fd54662950 2024-04-24T09:11:33.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.674+0000 7f1b23e00640 1 --2- 172.21.15.83:0/2327774397 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd54663270 0x55fd5466a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:11:33.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.675+0000 7f1b1be00640 1 --2- 172.21.15.83:0/2327774397 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fd54662950 0x55fd5466d2c0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c012fc0 tx=0x7f1b0c00b410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=350980afb199fc94 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:33.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.675+0000 7f1b12000640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:11:33.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.675+0000 7f1b12000640 1 -- 172.21.15.83:0/2327774397 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b0c004030 con 0x55fd54662950 2024-04-24T09:11:33.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.675+0000 7f1b12000640 1 -- 172.21.15.83:0/2327774397 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 9583+0+0 (secure 0 0 0) 0x7f1b0c027040 con 0x55fd54662950 2024-04-24T09:11:33.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.676+0000 7f1b2dbb2740 1 -- 172.21.15.83:0/2327774397 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b0c010770 con 0x55fd54662950 2024-04-24T09:11:33.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.676+0000 7f1b12000640 1 -- 172.21.15.83:0/2327774397 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f1b0c0041d0 con 0x55fd54662950 2024-04-24T09:11:33.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.676+0000 7f1b2dbb2740 10 client.39524 fetch_fsmap learned FSMap version 61 2024-04-24T09:11:33.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.676+0000 7f1b2dbb2740 1 -- 172.21.15.83:0/2327774397 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x55fd542da3f0 con 0x55fd54662950 2024-04-24T09:11:33.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.676+0000 7f1b12000640 1 -- 172.21.15.83:0/2327774397 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1b0c010770 con 0x55fd54662950 2024-04-24T09:11:33.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.676+0000 7f1b12000640 1 -- 172.21.15.83:0/2327774397 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 61) ==== 8362+0+0 (secure 0 0 0) 0x7f1b0c01f460 con 0x55fd54662950 2024-04-24T09:11:33.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.677+0000 7f1b2dbb2740 10 client.39524 fetch_fsmap finished waiting for FSMap version 61 2024-04-24T09:11:33.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.677+0000 7f1b2dbb2740 10 client.39524 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-24T09:11:33.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.677+0000 7f1b2dbb2740 20 client.39524 populate_metadata read hostname 'smithi083' 2024-04-24T09:11:33.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.677+0000 7f1b2dbb2740 1 --2- 172.21.15.83:0/2327774397 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55fd5468e9a0 0x55fd54690d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:33.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.677+0000 7f1b2dbb2740 4 client.39524 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:11:33.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.677+0000 7f1b2dbb2740 1 -- 172.21.15.83:0/2327774397 --> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fd545d5bf0 con 0x55fd5468e9a0 2024-04-24T09:11:33.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.677+0000 7f1b23e00640 1 --2- 172.21.15.83:0/2327774397 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55fd5468e9a0 0x55fd54690d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:33.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.678+0000 7f1b23e00640 1 --2- 172.21.15.83:0/2327774397 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55fd5468e9a0 0x55fd54690d90 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ef0defa93bae8932 server_cookie=7e53568fa9647f73 in_seq=0 out_seq=0 2024-04-24T09:11:33.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.678+0000 7f1b12000640 10 client.39524 ms_handle_connect on v2:172.21.15.189:6838/2548965336 2024-04-24T09:11:33.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.680+0000 7f1b12000640 1 -- 172.21.15.83:0/2327774397 <== mds.4 v2:172.21.15.189:6838/2548965336 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fd545d5bf0 con 0x55fd5468e9a0 2024-04-24T09:11:33.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.680+0000 7f1b12000640 10 client.39524 handle_command_reply: tid=0 2024-04-24T09:11:33.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.680+0000 7f1b2dbb2740 1 client.39524 shutdown 2024-04-24T09:11:33.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.680+0000 7f1b2dbb2740 1 -- 172.21.15.83:0/2327774397 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fd54662950 msgr2=0x55fd5466d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:33.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.680+0000 7f1b2dbb2740 1 --2- 172.21.15.83:0/2327774397 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fd54662950 0x55fd5466d2c0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c012fc0 tx=0x7f1b0c00b410 comp rx=0 tx=0).stop 2024-04-24T09:11:33.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.680+0000 7f1b2dbb2740 1 -- 172.21.15.83:0/2327774397 shutdown_connections 2024-04-24T09:11:33.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.680+0000 7f1b2dbb2740 1 --2- 172.21.15.83:0/2327774397 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55fd5468e9a0 0x55fd54690d90 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:33.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.680+0000 7f1b2dbb2740 1 --2- 172.21.15.83:0/2327774397 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd5466d940 0x55fd5466ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:33.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.680+0000 7f1b2dbb2740 1 --2- 172.21.15.83:0/2327774397 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd54663270 0x55fd5466a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:33.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.680+0000 7f1b2dbb2740 1 --2- 172.21.15.83:0/2327774397 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fd54662950 0x55fd5466d2c0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:33.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.680+0000 7f1b2dbb2740 1 -- 172.21.15.83:0/2327774397 >> 172.21.15.83:0/2327774397 conn(0x55fd5465ec70 msgr2=0x55fd54664930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:33.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.681+0000 7f1b12000640 0 client.39524 ms_handle_reset on v2:172.21.15.189:6838/2548965336 2024-04-24T09:11:33.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.681+0000 7f1b2dbb2740 1 -- 172.21.15.83:0/2327774397 shutdown_connections 2024-04-24T09:11:33.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.681+0000 7f1b2dbb2740 1 -- 172.21.15.83:0/2327774397 wait complete. 2024-04-24T09:11:33.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.681+0000 7f1b2dbb2740 20 client.39524 trim_cache size 0 max 16384 2024-04-24T09:11:33.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.703+0000 7f1b2dbb2740 1 Processor -- start 2024-04-24T09:11:33.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.703+0000 7f1b2dbb2740 1 -- start start 2024-04-24T09:11:33.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.703+0000 7f1b2dbb2740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd54662950 0x55fd5468a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:33.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.703+0000 7f1b2dbb2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fd54663270 0x55fd5466ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:33.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.703+0000 7f1b2dbb2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd5466d940 0x55fd546964b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:33.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.703+0000 7f1b2dbb2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1b00010100 con 0x55fd5466d940 2024-04-24T09:11:33.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.703+0000 7f1b2dbb2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1b00011c40 con 0x55fd54663270 2024-04-24T09:11:33.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.703+0000 7f1b2dbb2740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1b000107d0 con 0x55fd54662950 2024-04-24T09:11:33.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.704+0000 7f1b1be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd5466d940 0x55fd546964b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:33.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.704+0000 7f1b1be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd5466d940 0x55fd546964b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55288/0 (socket says 172.21.15.83:55288) 2024-04-24T09:11:33.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.704+0000 7f1b1be00640 1 -- 172.21.15.83:0/3573540946 learned_addr learned my addr 172.21.15.83:0/3573540946 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:11:33.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.704+0000 7f1b1be00640 1 --2- 172.21.15.83:0/3573540946 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd54662950 0x55fd5468a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:33.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.704+0000 7f1b23e00640 1 --2- 172.21.15.83:0/3573540946 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fd54663270 0x55fd5466ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:33.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.704+0000 7f1b1be00640 1 -- 172.21.15.83:0/3573540946 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd54662950 msgr2=0x55fd5468a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:33.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.704+0000 7f1b1be00640 1 --2- 172.21.15.83:0/3573540946 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd54662950 0x55fd5468a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:33.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.704+0000 7f1b1be00640 1 -- 172.21.15.83:0/3573540946 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fd54663270 msgr2=0x55fd5466ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:33.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.704+0000 7f1b1be00640 1 --2- 172.21.15.83:0/3573540946 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fd54663270 0x55fd5466ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:33.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.704+0000 7f1b1be00640 1 -- 172.21.15.83:0/3573540946 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd543a47d0 con 0x55fd5466d940 2024-04-24T09:11:33.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.704+0000 7f1b23e00640 1 --2- 172.21.15.83:0/3573540946 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fd54663270 0x55fd5466ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:11:33.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.704+0000 7f1b1be00640 1 --2- 172.21.15.83:0/3573540946 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd5466d940 0x55fd546964b0 secure :-1 s=READY pgs=2501 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c001eb0 tx=0x7f1b0c005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed4a32320f25ddfd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:33.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.705+0000 7f1b18c00640 1 -- 172.21.15.83:0/3573540946 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b0c00b520 con 0x55fd5466d940 2024-04-24T09:11:33.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.705+0000 7f1b18c00640 1 -- 172.21.15.83:0/3573540946 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1b0c00b860 con 0x55fd5466d940 2024-04-24T09:11:33.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.705+0000 7f1b18c00640 1 -- 172.21.15.83:0/3573540946 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b0c01f460 con 0x55fd5466d940 2024-04-24T09:11:33.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.706+0000 7f1b2dbb2740 1 -- 172.21.15.83:0/3573540946 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd5466d940 msgr2=0x55fd546964b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:33.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.706+0000 7f1b2dbb2740 1 --2- 172.21.15.83:0/3573540946 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd5466d940 0x55fd546964b0 secure :-1 s=READY pgs=2501 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c001eb0 tx=0x7f1b0c005740 comp rx=0 tx=0).stop 2024-04-24T09:11:33.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.706+0000 7f1b2dbb2740 1 -- 172.21.15.83:0/3573540946 shutdown_connections 2024-04-24T09:11:33.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.706+0000 7f1b2dbb2740 1 --2- 172.21.15.83:0/3573540946 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd5466d940 0x55fd546964b0 unknown :-1 s=CLOSED pgs=2501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:33.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.706+0000 7f1b2dbb2740 1 --2- 172.21.15.83:0/3573540946 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fd54663270 0x55fd5466ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:33.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.706+0000 7f1b2dbb2740 1 --2- 172.21.15.83:0/3573540946 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd54662950 0x55fd5468a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:33.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.706+0000 7f1b2dbb2740 1 -- 172.21.15.83:0/3573540946 >> 172.21.15.83:0/3573540946 conn(0x55fd5465ec70 msgr2=0x55fd546640a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:33.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.706+0000 7f1b2dbb2740 1 -- 172.21.15.83:0/3573540946 shutdown_connections 2024-04-24T09:11:33.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.706+0000 7f1b2dbb2740 1 -- 172.21.15.83:0/3573540946 wait complete. 2024-04-24T09:11:33.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.707+0000 7f1b2dbb2740 1 -- start start 2024-04-24T09:11:33.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.707+0000 7f1b2dbb2740 1 Processor -- start 2024-04-24T09:11:33.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.708+0000 7f1b2dbb2740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd54662950 0x55fd54686ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:33.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.708+0000 7f1b2dbb2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd54663270 0x55fd546678c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:33.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.708+0000 7f1b2dbb2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fd5466d940 0x55fd54667e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:33.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.708+0000 7f1b2dbb2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1b00011570 con 0x55fd54663270 2024-04-24T09:11:33.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.708+0000 7f1b2dbb2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1b00010ea0 con 0x55fd5466d940 2024-04-24T09:11:33.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.708+0000 7f1b2dbb2740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1b0000e100 con 0x55fd54662950 2024-04-24T09:11:33.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.708+0000 7f1b2dbb2740 10 client.0 _ll_register_callbacks cb 0x7f1b14009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:11:33.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.708+0000 7f1b23e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd54663270 0x55fd546678c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:33.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.708+0000 7f1b23e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd54663270 0x55fd546678c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55302/0 (socket says 172.21.15.83:55302) 2024-04-24T09:11:33.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.708+0000 7f1b23e00640 1 -- 172.21.15.83:0/1840506331 learned_addr learned my addr 172.21.15.83:0/1840506331 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:11:33.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.708+0000 7f1b1be00640 1 --2- 172.21.15.83:0/1840506331 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fd5466d940 0x55fd54667e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:33.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.708+0000 7f1b1be00640 1 --2- 172.21.15.83:0/1840506331 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd54662950 0x55fd54686ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:33.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.709+0000 7f1b23e00640 1 -- 172.21.15.83:0/1840506331 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd54662950 msgr2=0x55fd54686ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:33.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.709+0000 7f1b23e00640 1 --2- 172.21.15.83:0/1840506331 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd54662950 0x55fd54686ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:33.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.709+0000 7f1b23e00640 1 -- 172.21.15.83:0/1840506331 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fd5466d940 msgr2=0x55fd54667e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:33.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.709+0000 7f1b23e00640 1 --2- 172.21.15.83:0/1840506331 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fd5466d940 0x55fd54667e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:33.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.709+0000 7f1b23e00640 1 -- 172.21.15.83:0/1840506331 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd54668540 con 0x55fd54663270 2024-04-24T09:11:33.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.709+0000 7f1b1be00640 1 --2- 172.21.15.83:0/1840506331 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fd5466d940 0x55fd54667e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:11:33.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.709+0000 7f1b23e00640 1 --2- 172.21.15.83:0/1840506331 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd54663270 0x55fd546678c0 secure :-1 s=READY pgs=2502 cs=0 l=1 rev1=1 crypto rx=0x7f1b1402a440 tx=0x7f1b14004090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=496cb300d906518e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:33.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.709+0000 7f1b12000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:11:33.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.709+0000 7f1b12000640 1 -- 172.21.15.83:0/1840506331 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b14035040 con 0x55fd54663270 2024-04-24T09:11:33.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.709+0000 7f1b12000640 1 -- 172.21.15.83:0/1840506331 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 9583+0+0 (secure 0 0 0) 0x7f1b14004400 con 0x55fd54663270 2024-04-24T09:11:33.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.710+0000 7f1b2dbb2740 1 -- 172.21.15.83:0/1840506331 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b0c00b860 con 0x55fd54663270 2024-04-24T09:11:33.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.710+0000 7f1b12000640 1 -- 172.21.15.83:0/1840506331 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f1b14053050 con 0x55fd54663270 2024-04-24T09:11:33.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.710+0000 7f1b2dbb2740 10 client.22452 fetch_fsmap learned FSMap version 61 2024-04-24T09:11:33.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.710+0000 7f1b2dbb2740 1 -- 172.21.15.83:0/1840506331 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x55fd546f4d20 con 0x55fd54663270 2024-04-24T09:11:33.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.710+0000 7f1b12000640 1 -- 172.21.15.83:0/1840506331 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1b0c00b860 con 0x55fd54663270 2024-04-24T09:11:33.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.711+0000 7f1b12000640 1 -- 172.21.15.83:0/1840506331 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 61) ==== 8362+0+0 (secure 0 0 0) 0x7f1b140054a0 con 0x55fd54663270 2024-04-24T09:11:33.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.711+0000 7f1b2dbb2740 10 client.22452 fetch_fsmap finished waiting for FSMap version 61 2024-04-24T09:11:33.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.711+0000 7f1b2dbb2740 10 client.22452 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-24T09:11:33.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.711+0000 7f1b2dbb2740 20 client.22452 populate_metadata read hostname 'smithi083' 2024-04-24T09:11:33.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.711+0000 7f1b2dbb2740 1 --2- 172.21.15.83:0/1840506331 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55fd546f5430 0x55fd546f78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:33.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.711+0000 7f1b2dbb2740 4 client.22452 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:11:33.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.711+0000 7f1b2dbb2740 1 -- 172.21.15.83:0/1840506331 --> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fd545cfa60 con 0x55fd546f5430 2024-04-24T09:11:33.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.712+0000 7f1b1be00640 1 --2- 172.21.15.83:0/1840506331 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55fd546f5430 0x55fd546f78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:33.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.713+0000 7f1b1be00640 1 --2- 172.21.15.83:0/1840506331 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55fd546f5430 0x55fd546f78f0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=269a813738c929cb server_cookie=84fc16d4708faae1 in_seq=0 out_seq=0 2024-04-24T09:11:33.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.713+0000 7f1b12000640 10 client.22452 ms_handle_connect on v2:172.21.15.189:6838/2548965336 2024-04-24T09:11:33.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.713+0000 7f1b12000640 1 -- 172.21.15.83:0/1840506331 <== mds.4 v2:172.21.15.189:6838/2548965336 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fd545cfa60 con 0x55fd546f5430 2024-04-24T09:11:33.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.713+0000 7f1b12000640 10 client.22452 handle_command_reply: tid=0 2024-04-24T09:11:33.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.713+0000 7f1b2dbb2740 1 client.22452 shutdown 2024-04-24T09:11:33.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.714+0000 7f1b2dbb2740 1 -- 172.21.15.83:0/1840506331 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd54663270 msgr2=0x55fd546678c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:33.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.714+0000 7f1b2dbb2740 1 --2- 172.21.15.83:0/1840506331 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd54663270 0x55fd546678c0 secure :-1 s=READY pgs=2502 cs=0 l=1 rev1=1 crypto rx=0x7f1b1402a440 tx=0x7f1b14004090 comp rx=0 tx=0).stop 2024-04-24T09:11:33.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.714+0000 7f1b2dbb2740 1 -- 172.21.15.83:0/1840506331 shutdown_connections 2024-04-24T09:11:33.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.714+0000 7f1b2dbb2740 1 --2- 172.21.15.83:0/1840506331 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55fd546f5430 0x55fd546f78f0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:33.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.714+0000 7f1b2dbb2740 1 --2- 172.21.15.83:0/1840506331 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fd5466d940 0x55fd54667e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:33.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.714+0000 7f1b2dbb2740 1 --2- 172.21.15.83:0/1840506331 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd54663270 0x55fd546678c0 unknown :-1 s=CLOSED pgs=2502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:33.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.714+0000 7f1b2dbb2740 1 --2- 172.21.15.83:0/1840506331 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd54662950 0x55fd54686ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:33.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.714+0000 7f1b2dbb2740 1 -- 172.21.15.83:0/1840506331 >> 172.21.15.83:0/1840506331 conn(0x55fd5465ec70 msgr2=0x55fd54664930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:33.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.714+0000 7f1b12000640 0 client.22452 ms_handle_reset on v2:172.21.15.189:6838/2548965336 2024-04-24T09:11:33.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.714+0000 7f1b2dbb2740 1 -- 172.21.15.83:0/1840506331 shutdown_connections 2024-04-24T09:11:33.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.715+0000 7f1b2dbb2740 1 -- 172.21.15.83:0/1840506331 wait complete. 2024-04-24T09:11:33.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.715+0000 7f1b2dbb2740 20 client.22452 trim_cache size 0 max 16384 2024-04-24T09:11:33.715 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:11:33.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.716+0000 7f1b2ae00640 1 -- 172.21.15.83:0/3165454560 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f1b080a3e70 msgr2=0x7f1b080a6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:33.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.716+0000 7f1b2ae00640 1 --2- 172.21.15.83:0/3165454560 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f1b080a3e70 0x7f1b080a6330 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f1b1c015040 tx=0x7f1b1c009b50 comp rx=0 tx=0).stop 2024-04-24T09:11:33.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.716+0000 7f1b2ae00640 1 -- 172.21.15.83:0/3165454560 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b2410e8e0 msgr2=0x7f1b241ab200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:33.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.716+0000 7f1b2ae00640 1 --2- 172.21.15.83:0/3165454560 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b2410e8e0 0x7f1b241ab200 secure :-1 s=READY pgs=2500 cs=0 l=1 rev1=1 crypto rx=0x7f1b1c00ec50 tx=0x7f1b1c00c6a0 comp rx=0 tx=0).stop 2024-04-24T09:11:33.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.716+0000 7f1b2ae00640 1 -- 172.21.15.83:0/3165454560 shutdown_connections 2024-04-24T09:11:33.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.716+0000 7f1b2ae00640 1 --2- 172.21.15.83:0/3165454560 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f1b080a3e70 0x7f1b080a6330 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:33.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.716+0000 7f1b2ae00640 1 --2- 172.21.15.83:0/3165454560 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b2410e8e0 0x7f1b241ab200 unknown :-1 s=CLOSED pgs=2500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:33.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.716+0000 7f1b2ae00640 1 --2- 172.21.15.83:0/3165454560 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1b24109580 0x7f1b2410e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:33.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.716+0000 7f1b2ae00640 1 --2- 172.21.15.83:0/3165454560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1b241044b0 0x7f1b24111150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:33.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.716+0000 7f1b2ae00640 1 -- 172.21.15.83:0/3165454560 >> 172.21.15.83:0/3165454560 conn(0x7f1b240ffd90 msgr2=0x7f1b24101180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:33.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.716+0000 7f1b2ae00640 1 -- 172.21.15.83:0/3165454560 shutdown_connections 2024-04-24T09:11:33.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:33.716+0000 7f1b2ae00640 1 -- 172.21.15.83:0/3165454560 wait complete. 2024-04-24T09:11:33.736 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-24T09:11:34.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.015+0000 7f394e600640 1 -- 172.21.15.83:0/1533149752 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3948107310 msgr2=0x7f394810eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:34.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.015+0000 7f394e600640 1 --2- 172.21.15.83:0/1533149752 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3948107310 0x7f394810eee0 secure :-1 s=READY pgs=2503 cs=0 l=1 rev1=1 crypto rx=0x7f3938009900 tx=0x7f393802a410 comp rx=0 tx=0).stop 2024-04-24T09:11:34.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.015+0000 7f394e600640 1 -- 172.21.15.83:0/1533149752 shutdown_connections 2024-04-24T09:11:34.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.015+0000 7f394e600640 1 --2- 172.21.15.83:0/1533149752 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3948107310 0x7f394810eee0 unknown :-1 s=CLOSED pgs=2503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:34.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.015+0000 7f394e600640 1 --2- 172.21.15.83:0/1533149752 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3948102bf0 0x7f3948106c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:34.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.015+0000 7f394e600640 1 --2- 172.21.15.83:0/1533149752 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3948102240 0x7f3948102620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:34.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.015+0000 7f394e600640 1 -- 172.21.15.83:0/1533149752 >> 172.21.15.83:0/1533149752 conn(0x7f3948079450 msgr2=0x7f3948079860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:34.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.016+0000 7f394e600640 1 -- 172.21.15.83:0/1533149752 shutdown_connections 2024-04-24T09:11:34.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.016+0000 7f394e600640 1 -- 172.21.15.83:0/1533149752 wait complete. 2024-04-24T09:11:34.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.017+0000 7f394e600640 1 Processor -- start 2024-04-24T09:11:34.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.017+0000 7f394e600640 1 -- start start 2024-04-24T09:11:34.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.018+0000 7f394e600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3948102240 0x7f394810c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:34.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.018+0000 7f394e600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3948107310 0x7f394810c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:34.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.018+0000 7f394e600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f394810cae0 0x7f39481ab040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:34.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.018+0000 7f394e600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3948075de0 con 0x7f3948107310 2024-04-24T09:11:34.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.018+0000 7f394e600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3948075c60 con 0x7f394810cae0 2024-04-24T09:11:34.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.018+0000 7f394e600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3948075f60 con 0x7f3948102240 2024-04-24T09:11:34.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.018+0000 7f3946a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3948107310 0x7f394810c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:34.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.018+0000 7f3946a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3948107310 0x7f394810c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55318/0 (socket says 172.21.15.83:55318) 2024-04-24T09:11:34.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.018+0000 7f3946a00640 1 -- 172.21.15.83:0/1717049240 learned_addr learned my addr 172.21.15.83:0/1717049240 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:11:34.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.018+0000 7f3947e00640 1 --2- 172.21.15.83:0/1717049240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f394810cae0 0x7f39481ab040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:34.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.018+0000 7f3947400640 1 --2- 172.21.15.83:0/1717049240 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3948102240 0x7f394810c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:34.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.019+0000 7f3946a00640 1 -- 172.21.15.83:0/1717049240 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3948102240 msgr2=0x7f394810c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:34.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.019+0000 7f3946a00640 1 --2- 172.21.15.83:0/1717049240 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3948102240 0x7f394810c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:34.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.019+0000 7f3946a00640 1 -- 172.21.15.83:0/1717049240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f394810cae0 msgr2=0x7f39481ab040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:34.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.019+0000 7f3946a00640 1 --2- 172.21.15.83:0/1717049240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f394810cae0 0x7f39481ab040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:34.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.019+0000 7f3946a00640 1 -- 172.21.15.83:0/1717049240 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39481ab580 con 0x7f3948107310 2024-04-24T09:11:34.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.019+0000 7f3947e00640 1 --2- 172.21.15.83:0/1717049240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f394810cae0 0x7f39481ab040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:11:34.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.019+0000 7f3946a00640 1 --2- 172.21.15.83:0/1717049240 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3948107310 0x7f394810c5a0 secure :-1 s=READY pgs=2504 cs=0 l=1 rev1=1 crypto rx=0x7f394000e9c0 tx=0x7f394000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:34.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.020+0000 7f393fe00640 1 -- 172.21.15.83:0/1717049240 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f394000cd80 con 0x7f3948107310 2024-04-24T09:11:34.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.020+0000 7f393fe00640 1 -- 172.21.15.83:0/1717049240 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f39400054d0 con 0x7f3948107310 2024-04-24T09:11:34.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.020+0000 7f394e600640 1 -- 172.21.15.83:0/1717049240 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39481ab7b0 con 0x7f3948107310 2024-04-24T09:11:34.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.020+0000 7f393fe00640 1 -- 172.21.15.83:0/1717049240 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3940010430 con 0x7f3948107310 2024-04-24T09:11:34.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.020+0000 7f394e600640 1 -- 172.21.15.83:0/1717049240 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39481abc80 con 0x7f3948107310 2024-04-24T09:11:34.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.024+0000 7f39515d1740 1 Processor -- start 2024-04-24T09:11:34.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.025+0000 7f39515d1740 1 -- start start 2024-04-24T09:11:34.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.025+0000 7f393fe00640 1 -- 172.21.15.83:0/1717049240 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f39400105d0 con 0x7f3948107310 2024-04-24T09:11:34.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.025+0000 7f39515d1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558baa5ce950 0x558baa5ced30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:34.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.025+0000 7f39515d1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558baa5cf270 0x558baa5d9400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:34.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.025+0000 7f39515d1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558baa5d9940 0x558baa5dbd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:34.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.025+0000 7f39515d1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x558baa5341d0 con 0x558baa5d9940 2024-04-24T09:11:34.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.026+0000 7f3946a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558baa5d9940 0x558baa5dbd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:34.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.026+0000 7f3946a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558baa5d9940 0x558baa5dbd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55324/0 (socket says 172.21.15.83:55324) 2024-04-24T09:11:34.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.026+0000 7f3946a00640 1 -- 172.21.15.83:0/2825200110 learned_addr learned my addr 172.21.15.83:0/2825200110 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:11:34.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.025+0000 7f39515d1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558baa4da450 con 0x558baa5cf270 2024-04-24T09:11:34.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.026+0000 7f39515d1740 1 -- 172.21.15.83:0/2825200110 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x558baa530190 con 0x558baa5ce950 2024-04-24T09:11:34.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.026+0000 7f3947400640 1 --2- 172.21.15.83:0/2825200110 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558baa5cf270 0x558baa5d9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:34.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.026+0000 7f3947e00640 1 --2- 172.21.15.83:0/2825200110 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558baa5ce950 0x558baa5ced30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:34.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.027+0000 7f3947400640 1 -- 172.21.15.83:0/2825200110 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558baa5ce950 msgr2=0x558baa5ced30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:34.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.027+0000 7f3947400640 1 --2- 172.21.15.83:0/2825200110 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558baa5ce950 0x558baa5ced30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:34.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.027+0000 7f3947400640 1 -- 172.21.15.83:0/2825200110 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558baa5d9940 msgr2=0x558baa5dbd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:34.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.027+0000 7f3947400640 1 --2- 172.21.15.83:0/2825200110 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558baa5d9940 0x558baa5dbd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:34.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.027+0000 7f3947400640 1 -- 172.21.15.83:0/2825200110 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558baa597310 con 0x558baa5cf270 2024-04-24T09:11:34.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.027+0000 7f3946a00640 1 --2- 172.21.15.83:0/2825200110 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558baa5d9940 0x558baa5dbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:11:34.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.027+0000 7f393fe00640 1 --2- 172.21.15.83:0/1717049240 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f39280a3d60 0x7f39280a6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:34.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.027+0000 7f3947400640 1 --2- 172.21.15.83:0/2825200110 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558baa5cf270 0x558baa5d9400 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f3930009fb0 tx=0x7f3930003e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=355ce5cc7672cca6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:34.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.028+0000 7f393fe00640 1 -- 172.21.15.83:0/1717049240 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 9583+0+0 (secure 0 0 0) 0x7f39400d1100 con 0x7f3948107310 2024-04-24T09:11:34.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.028+0000 7f393cc00640 1 -- 172.21.15.83:0/2825200110 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3930004b20 con 0x558baa5cf270 2024-04-24T09:11:34.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.028+0000 7f393cc00640 1 -- 172.21.15.83:0/2825200110 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3930004cc0 con 0x558baa5cf270 2024-04-24T09:11:34.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.028+0000 7f393cc00640 1 -- 172.21.15.83:0/2825200110 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39300126b0 con 0x558baa5cf270 2024-04-24T09:11:34.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.028+0000 7f3947e00640 1 --2- 172.21.15.83:0/1717049240 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f39280a3d60 0x7f39280a6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:34.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.029+0000 7f39515d1740 1 -- 172.21.15.83:0/2825200110 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558baa5cf270 msgr2=0x558baa5d9400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:34.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.029+0000 7f39515d1740 1 --2- 172.21.15.83:0/2825200110 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558baa5cf270 0x558baa5d9400 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f3930009fb0 tx=0x7f3930003e00 comp rx=0 tx=0).stop 2024-04-24T09:11:34.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.030+0000 7f39515d1740 1 -- 172.21.15.83:0/2825200110 shutdown_connections 2024-04-24T09:11:34.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.030+0000 7f39515d1740 1 --2- 172.21.15.83:0/2825200110 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558baa5d9940 0x558baa5dbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:34.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.030+0000 7f39515d1740 1 --2- 172.21.15.83:0/2825200110 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558baa5cf270 0x558baa5d9400 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:34.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.030+0000 7f39515d1740 1 --2- 172.21.15.83:0/2825200110 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558baa5ce950 0x558baa5ced30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:34.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.030+0000 7f39515d1740 1 -- 172.21.15.83:0/2825200110 >> 172.21.15.83:0/2825200110 conn(0x558baa5cac70 msgr2=0x558baa5cb050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:34.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.030+0000 7f39515d1740 1 -- 172.21.15.83:0/2825200110 shutdown_connections 2024-04-24T09:11:34.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.030+0000 7f3947e00640 1 --2- 172.21.15.83:0/1717049240 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f39280a3d60 0x7f39280a6220 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f3938031440 tx=0x7f3938005d50 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:34.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.030+0000 7f39515d1740 1 -- 172.21.15.83:0/2825200110 wait complete. 2024-04-24T09:11:34.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.031+0000 7f39515d1740 1 -- start start 2024-04-24T09:11:34.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.031+0000 7f39515d1740 1 Processor -- start 2024-04-24T09:11:34.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.031+0000 7f39515d1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558baa5ce950 0x558baa5d9380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:34.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.031+0000 7f39515d1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558baa5cf270 0x558baa5d6670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:34.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.032+0000 7f39515d1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558baa5d9940 0x558baa5d6db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:34.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.032+0000 7f39515d1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x558baa5dc7d0 con 0x558baa5cf270 2024-04-24T09:11:34.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.032+0000 7f39515d1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558baa4ed810 con 0x558baa5d9940 2024-04-24T09:11:34.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.032+0000 7f39515d1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x558baa5dc950 con 0x558baa5ce950 2024-04-24T09:11:34.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.032+0000 7f39515d1740 10 client.0 _ll_register_callbacks cb 0x558baa4bb140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:11:34.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.032+0000 7f3947e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558baa5d9940 0x558baa5d6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:34.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.032+0000 7f3947e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558baa5d9940 0x558baa5d6db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:57750/0 (socket says 172.21.15.83:57750) 2024-04-24T09:11:34.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.032+0000 7f3947e00640 1 -- 172.21.15.83:0/247341481 learned_addr learned my addr 172.21.15.83:0/247341481 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:11:34.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.032+0000 7f3947400640 1 --2- 172.21.15.83:0/247341481 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558baa5ce950 0x558baa5d9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:34.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.032+0000 7f3946a00640 1 --2- 172.21.15.83:0/247341481 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558baa5cf270 0x558baa5d6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:34.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.032+0000 7f3947e00640 1 -- 172.21.15.83:0/247341481 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558baa5ce950 msgr2=0x558baa5d9380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:34.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.032+0000 7f3947e00640 1 --2- 172.21.15.83:0/247341481 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558baa5ce950 0x558baa5d9380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:34.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.033+0000 7f3947e00640 1 -- 172.21.15.83:0/247341481 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558baa5cf270 msgr2=0x558baa5d6670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:34.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.033+0000 7f3947e00640 1 --2- 172.21.15.83:0/247341481 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558baa5cf270 0x558baa5d6670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:34.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.033+0000 7f3947e00640 1 -- 172.21.15.83:0/247341481 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558baa4fe3f0 con 0x558baa5d9940 2024-04-24T09:11:34.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.033+0000 7f3947e00640 1 --2- 172.21.15.83:0/247341481 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558baa5d9940 0x558baa5d6db0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f3938033000 tx=0x7f3938009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a95b08ec89e454c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:34.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.033+0000 7f3935600640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:11:34.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.033+0000 7f3935600640 1 -- 172.21.15.83:0/247341481 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f393804c070 con 0x558baa5d9940 2024-04-24T09:11:34.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.034+0000 7f3935600640 1 -- 172.21.15.83:0/247341481 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 9583+0+0 (secure 0 0 0) 0x7f3938004ad0 con 0x558baa5d9940 2024-04-24T09:11:34.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.034+0000 7f39515d1740 1 -- 172.21.15.83:0/247341481 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3930004cc0 con 0x558baa5d9940 2024-04-24T09:11:34.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.034+0000 7f3935600640 1 -- 172.21.15.83:0/247341481 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f3938056050 con 0x558baa5d9940 2024-04-24T09:11:34.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.034+0000 7f39515d1740 10 client.39542 fetch_fsmap learned FSMap version 61 2024-04-24T09:11:34.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.034+0000 7f39515d1740 1 -- 172.21.15.83:0/247341481 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x558baa2463f0 con 0x558baa5d9940 2024-04-24T09:11:34.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.035+0000 7f3935600640 1 -- 172.21.15.83:0/247341481 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3930004cc0 con 0x558baa5d9940 2024-04-24T09:11:34.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.035+0000 7f3935600640 1 -- 172.21.15.83:0/247341481 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 61) ==== 8362+0+0 (secure 0 0 0) 0x7f39380324b0 con 0x558baa5d9940 2024-04-24T09:11:34.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.035+0000 7f39515d1740 10 client.39542 fetch_fsmap finished waiting for FSMap version 61 2024-04-24T09:11:34.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.035+0000 7f39515d1740 10 client.39542 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-24T09:11:34.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.035+0000 7f39515d1740 20 client.39542 populate_metadata read hostname 'smithi083' 2024-04-24T09:11:34.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.035+0000 7f39515d1740 1 --2- 172.21.15.83:0/247341481 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x558baa5e9b70 0x558baa5ebfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:34.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.035+0000 7f39515d1740 4 client.39542 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:11:34.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.035+0000 7f39515d1740 1 -- 172.21.15.83:0/247341481 --> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558baa541bf0 con 0x558baa5e9b70 2024-04-24T09:11:34.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.036+0000 7f3947400640 1 --2- 172.21.15.83:0/247341481 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x558baa5e9b70 0x558baa5ebfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:34.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.036+0000 7f3947400640 1 --2- 172.21.15.83:0/247341481 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x558baa5e9b70 0x558baa5ebfa0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f4c4b559e9ad4739 server_cookie=a532046cd8bb8380 in_seq=0 out_seq=0 2024-04-24T09:11:34.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.036+0000 7f3935600640 10 client.39542 ms_handle_connect on v2:172.21.15.138:6832/1157967658 2024-04-24T09:11:34.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.037+0000 7f3935600640 1 -- 172.21.15.83:0/247341481 <== mds.3 v2:172.21.15.138:6832/1157967658 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558baa541bf0 con 0x558baa5e9b70 2024-04-24T09:11:34.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.037+0000 7f3935600640 10 client.39542 handle_command_reply: tid=0 2024-04-24T09:11:34.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.037+0000 7f39515d1740 1 client.39542 shutdown 2024-04-24T09:11:34.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.038+0000 7f39515d1740 1 -- 172.21.15.83:0/247341481 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558baa5d9940 msgr2=0x558baa5d6db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:34.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.038+0000 7f39515d1740 1 --2- 172.21.15.83:0/247341481 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558baa5d9940 0x558baa5d6db0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f3938033000 tx=0x7f3938009260 comp rx=0 tx=0).stop 2024-04-24T09:11:34.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.038+0000 7f39515d1740 1 -- 172.21.15.83:0/247341481 shutdown_connections 2024-04-24T09:11:34.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.038+0000 7f39515d1740 1 --2- 172.21.15.83:0/247341481 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558baa5d9940 0x558baa5d6db0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:34.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.038+0000 7f39515d1740 1 --2- 172.21.15.83:0/247341481 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558baa5cf270 0x558baa5d6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:34.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.038+0000 7f39515d1740 1 --2- 172.21.15.83:0/247341481 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x558baa5e9b70 0x558baa5ebfa0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:34.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.038+0000 7f39515d1740 1 --2- 172.21.15.83:0/247341481 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558baa5ce950 0x558baa5d9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:34.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.038+0000 7f3935600640 0 client.39542 ms_handle_reset on v2:172.21.15.138:6832/1157967658 2024-04-24T09:11:34.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.038+0000 7f39515d1740 1 -- 172.21.15.83:0/247341481 >> 172.21.15.83:0/247341481 conn(0x558baa5cac70 msgr2=0x558baa5d0a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:34.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.038+0000 7f39515d1740 1 -- 172.21.15.83:0/247341481 shutdown_connections 2024-04-24T09:11:34.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.038+0000 7f39515d1740 1 -- 172.21.15.83:0/247341481 wait complete. 2024-04-24T09:11:34.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.038+0000 7f39515d1740 20 client.39542 trim_cache size 0 max 16384 2024-04-24T09:11:34.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.061+0000 7f39515d1740 1 Processor -- start 2024-04-24T09:11:34.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.061+0000 7f39515d1740 1 -- start start 2024-04-24T09:11:34.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.061+0000 7f39515d1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558baa5ce950 0x558baa5d10b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:34.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.061+0000 7f39515d1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558baa5cf270 0x558baa5d8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:34.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.061+0000 7f39515d1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558baa5d9940 0x558baa5d93f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:34.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.061+0000 7f39515d1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f39481b5ca0 con 0x558baa5ce950 2024-04-24T09:11:34.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.061+0000 7f39515d1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f39481b77e0 con 0x558baa5d9940 2024-04-24T09:11:34.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.061+0000 7f39515d1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f39481b6370 con 0x558baa5cf270 2024-04-24T09:11:34.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.062+0000 7f3947400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558baa5ce950 0x558baa5d10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:34.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.062+0000 7f3947400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558baa5ce950 0x558baa5d10b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55346/0 (socket says 172.21.15.83:55346) 2024-04-24T09:11:34.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.062+0000 7f3947400640 1 -- 172.21.15.83:0/3725668110 learned_addr learned my addr 172.21.15.83:0/3725668110 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:11:34.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.062+0000 7f3947e00640 1 --2- 172.21.15.83:0/3725668110 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558baa5d9940 0x558baa5d93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:34.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.062+0000 7f3946a00640 1 --2- 172.21.15.83:0/3725668110 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558baa5cf270 0x558baa5d8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:34.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.062+0000 7f3947e00640 1 -- 172.21.15.83:0/3725668110 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558baa5cf270 msgr2=0x558baa5d8eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:34.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.062+0000 7f3947e00640 1 --2- 172.21.15.83:0/3725668110 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558baa5cf270 0x558baa5d8eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:34.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.062+0000 7f3947e00640 1 -- 172.21.15.83:0/3725668110 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558baa5ce950 msgr2=0x558baa5d10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:34.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.062+0000 7f3947e00640 1 --2- 172.21.15.83:0/3725668110 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558baa5ce950 0x558baa5d10b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:34.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.062+0000 7f3947e00640 1 -- 172.21.15.83:0/3725668110 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558baa5cf0a0 con 0x558baa5d9940 2024-04-24T09:11:34.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.063+0000 7f3947e00640 1 --2- 172.21.15.83:0/3725668110 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558baa5d9940 0x558baa5d93f0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f39380057c0 tx=0x7f3938032740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d73c493810f0f78 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:34.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.063+0000 7f3937e00640 1 -- 172.21.15.83:0/3725668110 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f393804c070 con 0x558baa5d9940 2024-04-24T09:11:34.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.063+0000 7f3937e00640 1 -- 172.21.15.83:0/3725668110 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3938004d80 con 0x558baa5d9940 2024-04-24T09:11:34.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.063+0000 7f3937e00640 1 -- 172.21.15.83:0/3725668110 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3938035050 con 0x558baa5d9940 2024-04-24T09:11:34.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.064+0000 7f39515d1740 1 -- 172.21.15.83:0/3725668110 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558baa5d9940 msgr2=0x558baa5d93f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:34.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.064+0000 7f39515d1740 1 --2- 172.21.15.83:0/3725668110 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558baa5d9940 0x558baa5d93f0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f39380057c0 tx=0x7f3938032740 comp rx=0 tx=0).stop 2024-04-24T09:11:34.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.065+0000 7f39515d1740 1 -- 172.21.15.83:0/3725668110 shutdown_connections 2024-04-24T09:11:34.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.065+0000 7f39515d1740 1 --2- 172.21.15.83:0/3725668110 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558baa5d9940 0x558baa5d93f0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:34.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.065+0000 7f39515d1740 1 --2- 172.21.15.83:0/3725668110 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558baa5cf270 0x558baa5d8eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:34.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.065+0000 7f39515d1740 1 --2- 172.21.15.83:0/3725668110 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558baa5ce950 0x558baa5d10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:34.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.065+0000 7f39515d1740 1 -- 172.21.15.83:0/3725668110 >> 172.21.15.83:0/3725668110 conn(0x558baa5cac70 msgr2=0x558baa5d19e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:34.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.065+0000 7f39515d1740 1 -- 172.21.15.83:0/3725668110 shutdown_connections 2024-04-24T09:11:34.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.065+0000 7f39515d1740 1 -- 172.21.15.83:0/3725668110 wait complete. 2024-04-24T09:11:34.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.066+0000 7f39515d1740 1 -- start start 2024-04-24T09:11:34.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.066+0000 7f39515d1740 1 Processor -- start 2024-04-24T09:11:34.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.066+0000 7f39515d1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558baa5ce950 0x558baa5d42f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:34.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.066+0000 7f39515d1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558baa5cf270 0x558baa5e1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:34.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.066+0000 7f39515d1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558baa5d9940 0x558baa5daa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:34.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.066+0000 7f39515d1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f39481b7110 con 0x558baa5d9940 2024-04-24T09:11:34.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.066+0000 7f39515d1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f39481b6a40 con 0x558baa5cf270 2024-04-24T09:11:34.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.066+0000 7f39515d1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f39481b3ca0 con 0x558baa5ce950 2024-04-24T09:11:34.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.066+0000 7f39515d1740 10 client.0 _ll_register_callbacks cb 0x7f3940005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:11:34.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.067+0000 7f3946a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558baa5cf270 0x558baa5e1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:34.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.067+0000 7f3946a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558baa5cf270 0x558baa5e1bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:57776/0 (socket says 172.21.15.83:57776) 2024-04-24T09:11:34.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.067+0000 7f3946a00640 1 -- 172.21.15.83:0/2918587561 learned_addr learned my addr 172.21.15.83:0/2918587561 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:11:34.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.067+0000 7f3947e00640 1 --2- 172.21.15.83:0/2918587561 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558baa5d9940 0x558baa5daa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:34.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.067+0000 7f3947400640 1 --2- 172.21.15.83:0/2918587561 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558baa5ce950 0x558baa5d42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:34.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.067+0000 7f3946a00640 1 -- 172.21.15.83:0/2918587561 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558baa5ce950 msgr2=0x558baa5d42f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:34.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.067+0000 7f3946a00640 1 --2- 172.21.15.83:0/2918587561 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558baa5ce950 0x558baa5d42f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:34.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.067+0000 7f3946a00640 1 -- 172.21.15.83:0/2918587561 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558baa5d9940 msgr2=0x558baa5daa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:34.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.067+0000 7f3946a00640 1 --2- 172.21.15.83:0/2918587561 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558baa5d9940 0x558baa5daa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:34.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.067+0000 7f3946a00640 1 -- 172.21.15.83:0/2918587561 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558baa3107d0 con 0x558baa5cf270 2024-04-24T09:11:34.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.068+0000 7f3946a00640 1 --2- 172.21.15.83:0/2918587561 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558baa5cf270 0x558baa5e1bd0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f394000c6a0 tx=0x7f39400d4810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2fe837c82dac2c40 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:34.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.068+0000 7f3935600640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:11:34.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.068+0000 7f3935600640 1 -- 172.21.15.83:0/2918587561 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f394009c5e0 con 0x558baa5cf270 2024-04-24T09:11:34.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.068+0000 7f3935600640 1 -- 172.21.15.83:0/2918587561 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 9583+0+0 (secure 0 0 0) 0x7f394009cee0 con 0x558baa5cf270 2024-04-24T09:11:34.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.068+0000 7f39515d1740 1 -- 172.21.15.83:0/2918587561 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3938004d80 con 0x558baa5cf270 2024-04-24T09:11:34.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.068+0000 7f3935600640 1 -- 172.21.15.83:0/2918587561 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f39400ce8b0 con 0x558baa5cf270 2024-04-24T09:11:34.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.068+0000 7f39515d1740 10 client.39548 fetch_fsmap learned FSMap version 61 2024-04-24T09:11:34.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.069+0000 7f39515d1740 1 -- 172.21.15.83:0/2918587561 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x558baa650580 con 0x558baa5cf270 2024-04-24T09:11:34.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.069+0000 7f3935600640 1 -- 172.21.15.83:0/2918587561 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3938004d80 con 0x558baa5cf270 2024-04-24T09:11:34.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.069+0000 7f3935600640 1 -- 172.21.15.83:0/2918587561 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 61) ==== 8362+0+0 (secure 0 0 0) 0x7f39400a5460 con 0x558baa5cf270 2024-04-24T09:11:34.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.069+0000 7f39515d1740 10 client.39548 fetch_fsmap finished waiting for FSMap version 61 2024-04-24T09:11:34.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.069+0000 7f39515d1740 10 client.39548 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-24T09:11:34.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.069+0000 7f39515d1740 20 client.39548 populate_metadata read hostname 'smithi083' 2024-04-24T09:11:34.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.069+0000 7f39515d1740 1 --2- 172.21.15.83:0/2918587561 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x558baa650ce0 0x558baa653100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:34.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.069+0000 7f39515d1740 4 client.39548 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:11:34.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.069+0000 7f39515d1740 1 -- 172.21.15.83:0/2918587561 --> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558baa53ba60 con 0x558baa650ce0 2024-04-24T09:11:34.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.070+0000 7f3947400640 1 --2- 172.21.15.83:0/2918587561 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x558baa650ce0 0x558baa653100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:34.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.071+0000 7f3935600640 10 client.39548 ms_handle_connect on v2:172.21.15.138:6832/1157967658 2024-04-24T09:11:34.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.071+0000 7f3947400640 1 --2- 172.21.15.83:0/2918587561 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x558baa650ce0 0x558baa653100 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6c76909886458d15 server_cookie=fb9f693fcef400bd in_seq=0 out_seq=0 2024-04-24T09:11:34.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.071+0000 7f3935600640 1 -- 172.21.15.83:0/2918587561 <== mds.3 v2:172.21.15.138:6832/1157967658 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558baa53ba60 con 0x558baa650ce0 2024-04-24T09:11:34.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.071+0000 7f3935600640 10 client.39548 handle_command_reply: tid=0 2024-04-24T09:11:34.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.071+0000 7f39515d1740 1 client.39548 shutdown 2024-04-24T09:11:34.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.071+0000 7f39515d1740 1 -- 172.21.15.83:0/2918587561 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558baa5cf270 msgr2=0x558baa5e1bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:34.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.071+0000 7f39515d1740 1 --2- 172.21.15.83:0/2918587561 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558baa5cf270 0x558baa5e1bd0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f394000c6a0 tx=0x7f39400d4810 comp rx=0 tx=0).stop 2024-04-24T09:11:34.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.072+0000 7f39515d1740 1 -- 172.21.15.83:0/2918587561 shutdown_connections 2024-04-24T09:11:34.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.072+0000 7f39515d1740 1 --2- 172.21.15.83:0/2918587561 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558baa5d9940 0x558baa5daa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:34.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.072+0000 7f39515d1740 1 --2- 172.21.15.83:0/2918587561 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558baa5cf270 0x558baa5e1bd0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:34.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.072+0000 7f39515d1740 1 --2- 172.21.15.83:0/2918587561 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x558baa650ce0 0x558baa653100 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:34.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.072+0000 7f39515d1740 1 --2- 172.21.15.83:0/2918587561 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558baa5ce950 0x558baa5d42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:34.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.072+0000 7f3935600640 0 client.39548 ms_handle_reset on v2:172.21.15.138:6832/1157967658 2024-04-24T09:11:34.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.072+0000 7f39515d1740 1 -- 172.21.15.83:0/2918587561 >> 172.21.15.83:0/2918587561 conn(0x558baa5cac70 msgr2=0x558baa5c9560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:34.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.072+0000 7f39515d1740 1 -- 172.21.15.83:0/2918587561 shutdown_connections 2024-04-24T09:11:34.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.072+0000 7f39515d1740 1 -- 172.21.15.83:0/2918587561 wait complete. 2024-04-24T09:11:34.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.072+0000 7f39515d1740 20 client.39548 trim_cache size 0 max 16384 2024-04-24T09:11:34.072 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:11:34.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.073+0000 7f394e600640 1 -- 172.21.15.83:0/1717049240 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f39280a3d60 msgr2=0x7f39280a6220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:34.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.073+0000 7f394e600640 1 --2- 172.21.15.83:0/1717049240 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f39280a3d60 0x7f39280a6220 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f3938031440 tx=0x7f3938005d50 comp rx=0 tx=0).stop 2024-04-24T09:11:34.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.073+0000 7f394e600640 1 -- 172.21.15.83:0/1717049240 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3948107310 msgr2=0x7f394810c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:34.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.073+0000 7f394e600640 1 --2- 172.21.15.83:0/1717049240 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3948107310 0x7f394810c5a0 secure :-1 s=READY pgs=2504 cs=0 l=1 rev1=1 crypto rx=0x7f394000e9c0 tx=0x7f394000ee80 comp rx=0 tx=0).stop 2024-04-24T09:11:34.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.074+0000 7f394e600640 1 -- 172.21.15.83:0/1717049240 shutdown_connections 2024-04-24T09:11:34.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.074+0000 7f394e600640 1 --2- 172.21.15.83:0/1717049240 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f39280a3d60 0x7f39280a6220 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:34.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.074+0000 7f394e600640 1 --2- 172.21.15.83:0/1717049240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f394810cae0 0x7f39481ab040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:34.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.074+0000 7f394e600640 1 --2- 172.21.15.83:0/1717049240 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3948107310 0x7f394810c5a0 unknown :-1 s=CLOSED pgs=2504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:34.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.074+0000 7f394e600640 1 --2- 172.21.15.83:0/1717049240 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3948102240 0x7f394810c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:34.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.074+0000 7f394e600640 1 -- 172.21.15.83:0/1717049240 >> 172.21.15.83:0/1717049240 conn(0x7f3948079450 msgr2=0x7f39480fff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:34.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.074+0000 7f394e600640 1 -- 172.21.15.83:0/1717049240 shutdown_connections 2024-04-24T09:11:34.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.074+0000 7f394e600640 1 -- 172.21.15.83:0/1717049240 wait complete. 2024-04-24T09:11:34.095 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T09:11:34.181 INFO:tasks.workunit.client.0.smithi083.stdout: 10 45942 6.72 MB/sec execute 245 sec latency 105.860 ms 2024-04-24T09:11:34.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.355+0000 7f8ce3e00640 1 -- 172.21.15.83:0/3800903303 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ce410c480 msgr2=0x7f8ce410e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:34.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.355+0000 7f8ce3e00640 1 --2- 172.21.15.83:0/3800903303 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ce410c480 0x7f8ce410e920 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f8cdc009900 tx=0x7f8cdc02a490 comp rx=0 tx=0).stop 2024-04-24T09:11:34.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.355+0000 7f8ce3e00640 1 -- 172.21.15.83:0/3800903303 shutdown_connections 2024-04-24T09:11:34.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.355+0000 7f8ce3e00640 1 --2- 172.21.15.83:0/3800903303 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ce410c480 0x7f8ce410e920 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:34.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.355+0000 7f8ce3e00640 1 --2- 172.21.15.83:0/3800903303 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ce4103980 0x7f8ce410be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:34.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.355+0000 7f8ce3e00640 1 --2- 172.21.15.83:0/3800903303 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8ce4103060 0x7f8ce4103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:34.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.355+0000 7f8ce3e00640 1 -- 172.21.15.83:0/3800903303 >> 172.21.15.83:0/3800903303 conn(0x7f8ce40fc880 msgr2=0x7f8ce40feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:34.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.356+0000 7f8ce3e00640 1 -- 172.21.15.83:0/3800903303 shutdown_connections 2024-04-24T09:11:34.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.356+0000 7f8ce3e00640 1 -- 172.21.15.83:0/3800903303 wait complete. 2024-04-24T09:11:34.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.357+0000 7f8ce3e00640 1 Processor -- start 2024-04-24T09:11:34.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.357+0000 7f8ce3e00640 1 -- start start 2024-04-24T09:11:34.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.358+0000 7f8ce3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ce4103060 0x7f8ce41a05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:34.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.358+0000 7f8ce3e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ce4103980 0x7f8ce41a0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:34.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.358+0000 7f8ce3e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8ce410c480 0x7f8ce419a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:34.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.358+0000 7f8ce3e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8ce4112160 con 0x7f8ce4103060 2024-04-24T09:11:34.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.358+0000 7f8ce3e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8ce4111fe0 con 0x7f8ce4103980 2024-04-24T09:11:34.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.358+0000 7f8ce3e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8ce41122e0 con 0x7f8ce410c480 2024-04-24T09:11:34.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.358+0000 7f8ce2a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ce4103060 0x7f8ce41a05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:34.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.358+0000 7f8ce2a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ce4103060 0x7f8ce41a05b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55376/0 (socket says 172.21.15.83:55376) 2024-04-24T09:11:34.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.358+0000 7f8ce2a00640 1 -- 172.21.15.83:0/898916474 learned_addr learned my addr 172.21.15.83:0/898916474 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:11:34.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.358+0000 7f8ce2000640 1 --2- 172.21.15.83:0/898916474 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ce4103980 0x7f8ce41a0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:34.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.359+0000 7f8ce2000640 1 -- 172.21.15.83:0/898916474 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8ce410c480 msgr2=0x7f8ce419a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:34.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.359+0000 7f8ce2000640 1 --2- 172.21.15.83:0/898916474 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8ce410c480 0x7f8ce419a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:34.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.359+0000 7f8ce2000640 1 -- 172.21.15.83:0/898916474 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ce4103060 msgr2=0x7f8ce41a05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:34.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.359+0000 7f8ce2000640 1 --2- 172.21.15.83:0/898916474 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ce4103060 0x7f8ce41a05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:34.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.359+0000 7f8ce2000640 1 -- 172.21.15.83:0/898916474 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ce419b020 con 0x7f8ce4103980 2024-04-24T09:11:34.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.359+0000 7f8ce2a00640 1 --2- 172.21.15.83:0/898916474 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ce4103060 0x7f8ce41a05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:11:34.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.359+0000 7f8ce2000640 1 --2- 172.21.15.83:0/898916474 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ce4103980 0x7f8ce41a0af0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f8cd000e9f0 tx=0x7f8cd000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:34.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.360+0000 7f8ce8a00640 1 -- 172.21.15.83:0/898916474 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cd000ccc0 con 0x7f8ce4103980 2024-04-24T09:11:34.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.360+0000 7f8ce3e00640 1 -- 172.21.15.83:0/898916474 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ce419b310 con 0x7f8ce4103980 2024-04-24T09:11:34.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.360+0000 7f8ce8a00640 1 -- 172.21.15.83:0/898916474 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8cd00035f0 con 0x7f8ce4103980 2024-04-24T09:11:34.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.360+0000 7f8ce8a00640 1 -- 172.21.15.83:0/898916474 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cd000ce60 con 0x7f8ce4103980 2024-04-24T09:11:34.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.360+0000 7f8ce3e00640 1 -- 172.21.15.83:0/898916474 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ce41a7530 con 0x7f8ce4103980 2024-04-24T09:11:34.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.365+0000 7f8cecd66740 1 Processor -- start 2024-04-24T09:11:34.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.365+0000 7f8ce8a00640 1 -- 172.21.15.83:0/898916474 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8cd00037a0 con 0x7f8ce4103980 2024-04-24T09:11:34.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.366+0000 7f8cecd66740 1 -- start start 2024-04-24T09:11:34.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.366+0000 7f8cecd66740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5642cf5a1b40 0x5642cf5a1f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:34.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.366+0000 7f8cecd66740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642cf5a2460 0x5642cf5ac5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:34.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.366+0000 7f8cecd66740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642cf5acb30 0x5642cf5aef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:34.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.366+0000 7f8cecd66740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5642cf5071d0 con 0x5642cf5acb30 2024-04-24T09:11:34.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.366+0000 7f8cecd66740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5642cf4ad450 con 0x5642cf5a2460 2024-04-24T09:11:34.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.366+0000 7f8cecd66740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5642cf503190 con 0x5642cf5a1b40 2024-04-24T09:11:34.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.367+0000 7f8ce2000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642cf5acb30 0x5642cf5aef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:34.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.367+0000 7f8ce3400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5642cf5a1b40 0x5642cf5a1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:34.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.367+0000 7f8ce3400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5642cf5a1b40 0x5642cf5a1f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:38482/0 (socket says 172.21.15.83:38482) 2024-04-24T09:11:34.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.367+0000 7f8ce3400640 1 -- 172.21.15.83:0/3497445002 learned_addr learned my addr 172.21.15.83:0/3497445002 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:11:34.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.367+0000 7f8ce2a00640 1 --2- 172.21.15.83:0/3497445002 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642cf5a2460 0x5642cf5ac5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:34.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.368+0000 7f8ce3400640 1 -- 172.21.15.83:0/3497445002 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642cf5a2460 msgr2=0x5642cf5ac5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:34.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.368+0000 7f8ce3400640 1 --2- 172.21.15.83:0/3497445002 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642cf5a2460 0x5642cf5ac5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:34.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.368+0000 7f8ce3400640 1 -- 172.21.15.83:0/3497445002 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642cf5acb30 msgr2=0x5642cf5aef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:34.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.368+0000 7f8ce3400640 1 --2- 172.21.15.83:0/3497445002 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642cf5acb30 0x5642cf5aef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:34.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.368+0000 7f8ce3400640 1 -- 172.21.15.83:0/3497445002 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642cf56a310 con 0x5642cf5a1b40 2024-04-24T09:11:34.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.368+0000 7f8ce2000640 1 --2- 172.21.15.83:0/3497445002 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642cf5acb30 0x5642cf5aef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:11:34.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.369+0000 7f8ce8a00640 1 --2- 172.21.15.83:0/898916474 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f8cc80a3da0 0x7f8cc80a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:34.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.369+0000 7f8ce3400640 1 --2- 172.21.15.83:0/3497445002 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5642cf5a1b40 0x5642cf5a1f20 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f8cdc009630 tx=0x7f8cdc009660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e95cc8f5efabba9e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:34.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.369+0000 7f8ce8a00640 1 -- 172.21.15.83:0/898916474 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 9583+0+0 (secure 0 0 0) 0x7f8cd00d0e20 con 0x7f8ce4103980 2024-04-24T09:11:34.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.370+0000 7f8ccfe00640 1 -- 172.21.15.83:0/3497445002 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cdc03e030 con 0x5642cf5a1b40 2024-04-24T09:11:34.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.370+0000 7f8ccfe00640 1 -- 172.21.15.83:0/3497445002 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8cdc004230 con 0x5642cf5a1b40 2024-04-24T09:11:34.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.370+0000 7f8ccfe00640 1 -- 172.21.15.83:0/3497445002 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cdc035040 con 0x5642cf5a1b40 2024-04-24T09:11:34.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.370+0000 7f8ce2a00640 1 --2- 172.21.15.83:0/898916474 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f8cc80a3da0 0x7f8cc80a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:34.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.371+0000 7f8cecd66740 1 -- 172.21.15.83:0/3497445002 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5642cf5a1b40 msgr2=0x5642cf5a1f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:34.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.371+0000 7f8cecd66740 1 --2- 172.21.15.83:0/3497445002 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5642cf5a1b40 0x5642cf5a1f20 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f8cdc009630 tx=0x7f8cdc009660 comp rx=0 tx=0).stop 2024-04-24T09:11:34.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.371+0000 7f8cecd66740 1 -- 172.21.15.83:0/3497445002 shutdown_connections 2024-04-24T09:11:34.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.371+0000 7f8cecd66740 1 --2- 172.21.15.83:0/3497445002 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642cf5acb30 0x5642cf5aef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:34.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.371+0000 7f8cecd66740 1 --2- 172.21.15.83:0/3497445002 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642cf5a2460 0x5642cf5ac5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:34.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.371+0000 7f8cecd66740 1 --2- 172.21.15.83:0/3497445002 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5642cf5a1b40 0x5642cf5a1f20 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:34.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.371+0000 7f8cecd66740 1 -- 172.21.15.83:0/3497445002 >> 172.21.15.83:0/3497445002 conn(0x5642cf59de60 msgr2=0x5642cf59e240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:34.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.372+0000 7f8cecd66740 1 -- 172.21.15.83:0/3497445002 shutdown_connections 2024-04-24T09:11:34.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.372+0000 7f8cecd66740 1 -- 172.21.15.83:0/3497445002 wait complete. 2024-04-24T09:11:34.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.372+0000 7f8ce2a00640 1 --2- 172.21.15.83:0/898916474 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f8cc80a3da0 0x7f8cc80a6260 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x5642cf5a3100 tx=0x7f8cd4007600 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:34.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.373+0000 7f8cecd66740 1 -- start start 2024-04-24T09:11:34.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.374+0000 7f8cecd66740 1 Processor -- start 2024-04-24T09:11:34.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.374+0000 7f8cecd66740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5642cf5a1b40 0x5642cf5ac5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:34.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.374+0000 7f8cecd66740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642cf5a2460 0x5642cf5a98b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:34.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.374+0000 7f8cecd66740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642cf5acb30 0x5642cf5a9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:34.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.374+0000 7f8cecd66740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5642cf5af960 con 0x5642cf5a2460 2024-04-24T09:11:34.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.374+0000 7f8cecd66740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5642cf4c0810 con 0x5642cf5acb30 2024-04-24T09:11:34.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.374+0000 7f8cecd66740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5642cf5afae0 con 0x5642cf5a1b40 2024-04-24T09:11:34.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.374+0000 7f8cecd66740 10 client.0 _ll_register_callbacks cb 0x5642cf48e140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:11:34.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.375+0000 7f8ce3400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642cf5acb30 0x5642cf5a9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:34.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.375+0000 7f8ce3400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642cf5acb30 0x5642cf5a9fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:57822/0 (socket says 172.21.15.83:57822) 2024-04-24T09:11:34.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.375+0000 7f8ce3400640 1 -- 172.21.15.83:0/1086483873 learned_addr learned my addr 172.21.15.83:0/1086483873 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:11:34.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.375+0000 7f8ce2a00640 1 --2- 172.21.15.83:0/1086483873 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5642cf5a1b40 0x5642cf5ac5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:34.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.375+0000 7f8ce2000640 1 --2- 172.21.15.83:0/1086483873 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642cf5a2460 0x5642cf5a98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:34.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.375+0000 7f8ce3400640 1 -- 172.21.15.83:0/1086483873 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5642cf5a1b40 msgr2=0x5642cf5ac5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:34.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.376+0000 7f8ce3400640 1 --2- 172.21.15.83:0/1086483873 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5642cf5a1b40 0x5642cf5ac5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:34.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.376+0000 7f8ce3400640 1 -- 172.21.15.83:0/1086483873 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642cf5a2460 msgr2=0x5642cf5a98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:34.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.376+0000 7f8ce3400640 1 --2- 172.21.15.83:0/1086483873 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642cf5a2460 0x5642cf5a98b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:34.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.376+0000 7f8ce3400640 1 -- 172.21.15.83:0/1086483873 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642cf4d13f0 con 0x5642cf5acb30 2024-04-24T09:11:34.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.376+0000 7f8ce2000640 1 --2- 172.21.15.83:0/1086483873 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642cf5a2460 0x5642cf5a98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:11:34.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.376+0000 7f8ce3400640 1 --2- 172.21.15.83:0/1086483873 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642cf5acb30 0x5642cf5a9fa0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f8cdc02aa80 tx=0x7f8cdc004610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ea0faa7face36b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:34.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.376+0000 7f8cccc00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:11:34.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.377+0000 7f8cccc00640 1 -- 172.21.15.83:0/1086483873 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cdc035070 con 0x5642cf5acb30 2024-04-24T09:11:34.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.377+0000 7f8cccc00640 1 -- 172.21.15.83:0/1086483873 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 9583+0+0 (secure 0 0 0) 0x7f8cdc0037f0 con 0x5642cf5acb30 2024-04-24T09:11:34.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.377+0000 7f8cecd66740 1 -- 172.21.15.83:0/1086483873 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8cdc004230 con 0x5642cf5acb30 2024-04-24T09:11:34.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.378+0000 7f8cccc00640 1 -- 172.21.15.83:0/1086483873 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f8cdc03e040 con 0x5642cf5acb30 2024-04-24T09:11:34.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.378+0000 7f8cecd66740 10 client.39560 fetch_fsmap learned FSMap version 61 2024-04-24T09:11:34.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.378+0000 7f8cecd66740 1 -- 172.21.15.83:0/1086483873 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x5642cf2193f0 con 0x5642cf5acb30 2024-04-24T09:11:34.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.378+0000 7f8cccc00640 1 -- 172.21.15.83:0/1086483873 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8cdc004230 con 0x5642cf5acb30 2024-04-24T09:11:34.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.378+0000 7f8cccc00640 1 -- 172.21.15.83:0/1086483873 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 61) ==== 8362+0+0 (secure 0 0 0) 0x7f8cdc055420 con 0x5642cf5acb30 2024-04-24T09:11:34.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.379+0000 7f8cecd66740 10 client.39560 fetch_fsmap finished waiting for FSMap version 61 2024-04-24T09:11:34.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.379+0000 7f8cecd66740 10 client.39560 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-24T09:11:34.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.379+0000 7f8cecd66740 20 client.39560 populate_metadata read hostname 'smithi083' 2024-04-24T09:11:34.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.379+0000 7f8cecd66740 1 --2- 172.21.15.83:0/1086483873 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x5642cf5cdbd0 0x5642cf5cffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:34.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.379+0000 7f8cecd66740 4 client.39560 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:11:34.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.379+0000 7f8cecd66740 1 -- 172.21.15.83:0/1086483873 --> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5642cf514bf0 con 0x5642cf5cdbd0 2024-04-24T09:11:34.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.379+0000 7f8ce2a00640 1 --2- 172.21.15.83:0/1086483873 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x5642cf5cdbd0 0x5642cf5cffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:34.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.381+0000 7f8ce2a00640 1 --2- 172.21.15.83:0/1086483873 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x5642cf5cdbd0 0x5642cf5cffc0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9365d61bb8029ad5 server_cookie=83d6a96cf4c21349 in_seq=0 out_seq=0 2024-04-24T09:11:34.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.381+0000 7f8cccc00640 10 client.39560 ms_handle_connect on v2:172.21.15.138:6834/1909219758 2024-04-24T09:11:34.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.382+0000 7f8cccc00640 1 -- 172.21.15.83:0/1086483873 <== mds.2 v2:172.21.15.138:6834/1909219758 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5642cf514bf0 con 0x5642cf5cdbd0 2024-04-24T09:11:34.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.382+0000 7f8cccc00640 10 client.39560 handle_command_reply: tid=0 2024-04-24T09:11:34.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.382+0000 7f8cecd66740 1 client.39560 shutdown 2024-04-24T09:11:34.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.382+0000 7f8cecd66740 1 -- 172.21.15.83:0/1086483873 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642cf5acb30 msgr2=0x5642cf5a9fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:34.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.382+0000 7f8cecd66740 1 --2- 172.21.15.83:0/1086483873 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642cf5acb30 0x5642cf5a9fa0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f8cdc02aa80 tx=0x7f8cdc004610 comp rx=0 tx=0).stop 2024-04-24T09:11:34.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.382+0000 7f8cecd66740 1 -- 172.21.15.83:0/1086483873 shutdown_connections 2024-04-24T09:11:34.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.382+0000 7f8cecd66740 1 --2- 172.21.15.83:0/1086483873 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x5642cf5cdbd0 0x5642cf5cffc0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:34.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.382+0000 7f8cecd66740 1 --2- 172.21.15.83:0/1086483873 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642cf5acb30 0x5642cf5a9fa0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:34.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.382+0000 7f8cecd66740 1 --2- 172.21.15.83:0/1086483873 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642cf5a2460 0x5642cf5a98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:34.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.382+0000 7f8cecd66740 1 --2- 172.21.15.83:0/1086483873 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5642cf5a1b40 0x5642cf5ac5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:34.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.382+0000 7f8cecd66740 1 -- 172.21.15.83:0/1086483873 >> 172.21.15.83:0/1086483873 conn(0x5642cf59de60 msgr2=0x5642cf5a3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:34.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.382+0000 7f8cccc00640 0 client.39560 ms_handle_reset on v2:172.21.15.138:6834/1909219758 2024-04-24T09:11:34.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.383+0000 7f8cecd66740 1 -- 172.21.15.83:0/1086483873 shutdown_connections 2024-04-24T09:11:34.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.383+0000 7f8cecd66740 1 -- 172.21.15.83:0/1086483873 wait complete. 2024-04-24T09:11:34.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.383+0000 7f8cecd66740 20 client.39560 trim_cache size 0 max 16384 2024-04-24T09:11:34.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.403+0000 7f8cecd66740 1 Processor -- start 2024-04-24T09:11:34.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.403+0000 7f8cecd66740 1 -- start start 2024-04-24T09:11:34.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.404+0000 7f8cecd66740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642cf5a1b40 0x5642cf5ac0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:34.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.404+0000 7f8cecd66740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5642cf5a2460 0x5642cf5a6b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:34.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.404+0000 7f8cecd66740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642cf5acb30 0x5642cf5a7070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:34.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.404+0000 7f8cecd66740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8ce41a8d60 con 0x5642cf5acb30 2024-04-24T09:11:34.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.404+0000 7f8cecd66740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8ce41aa8a0 con 0x5642cf5a1b40 2024-04-24T09:11:34.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.404+0000 7f8cecd66740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8ce41a9430 con 0x5642cf5a2460 2024-04-24T09:11:34.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.404+0000 7f8ce3400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642cf5acb30 0x5642cf5a7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:34.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.404+0000 7f8ce2a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642cf5a1b40 0x5642cf5ac0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:34.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.404+0000 7f8ce2a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642cf5a1b40 0x5642cf5ac0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:57824/0 (socket says 172.21.15.83:57824) 2024-04-24T09:11:34.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.404+0000 7f8ce2a00640 1 -- 172.21.15.83:0/2541357365 learned_addr learned my addr 172.21.15.83:0/2541357365 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:11:34.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.405+0000 7f8ce2000640 1 --2- 172.21.15.83:0/2541357365 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5642cf5a2460 0x5642cf5a6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:34.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.405+0000 7f8ce2a00640 1 -- 172.21.15.83:0/2541357365 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5642cf5a2460 msgr2=0x5642cf5a6b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:34.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.405+0000 7f8ce2a00640 1 --2- 172.21.15.83:0/2541357365 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5642cf5a2460 0x5642cf5a6b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:34.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.405+0000 7f8ce2a00640 1 -- 172.21.15.83:0/2541357365 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642cf5acb30 msgr2=0x5642cf5a7070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:34.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.405+0000 7f8ce2a00640 1 --2- 172.21.15.83:0/2541357365 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642cf5acb30 0x5642cf5a7070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:34.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.405+0000 7f8ce2a00640 1 -- 172.21.15.83:0/2541357365 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642cf2e37d0 con 0x5642cf5a1b40 2024-04-24T09:11:34.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.405+0000 7f8ce3400640 1 --2- 172.21.15.83:0/2541357365 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642cf5acb30 0x5642cf5a7070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:11:34.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.406+0000 7f8ce2a00640 1 --2- 172.21.15.83:0/2541357365 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642cf5a1b40 0x5642cf5ac0f0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f8cd400fb80 tx=0x7f8cd4018000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76e947253ed8838f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:34.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.406+0000 7f8ccf400640 1 -- 172.21.15.83:0/2541357365 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cd4029470 con 0x5642cf5a1b40 2024-04-24T09:11:34.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.406+0000 7f8ccf400640 1 -- 172.21.15.83:0/2541357365 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8cd402e070 con 0x5642cf5a1b40 2024-04-24T09:11:34.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.406+0000 7f8ccf400640 1 -- 172.21.15.83:0/2541357365 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cd401c670 con 0x5642cf5a1b40 2024-04-24T09:11:34.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.407+0000 7f8cecd66740 1 -- 172.21.15.83:0/2541357365 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642cf5a1b40 msgr2=0x5642cf5ac0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:34.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.407+0000 7f8cecd66740 1 --2- 172.21.15.83:0/2541357365 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642cf5a1b40 0x5642cf5ac0f0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f8cd400fb80 tx=0x7f8cd4018000 comp rx=0 tx=0).stop 2024-04-24T09:11:34.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.407+0000 7f8cecd66740 1 -- 172.21.15.83:0/2541357365 shutdown_connections 2024-04-24T09:11:34.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.407+0000 7f8cecd66740 1 --2- 172.21.15.83:0/2541357365 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642cf5acb30 0x5642cf5a7070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:34.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.407+0000 7f8cecd66740 1 --2- 172.21.15.83:0/2541357365 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5642cf5a2460 0x5642cf5a6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:34.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.407+0000 7f8cecd66740 1 --2- 172.21.15.83:0/2541357365 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642cf5a1b40 0x5642cf5ac0f0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:34.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.407+0000 7f8cecd66740 1 -- 172.21.15.83:0/2541357365 >> 172.21.15.83:0/2541357365 conn(0x5642cf59de60 msgr2=0x5642cf5a4b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:34.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.407+0000 7f8cecd66740 1 -- 172.21.15.83:0/2541357365 shutdown_connections 2024-04-24T09:11:34.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.407+0000 7f8cecd66740 1 -- 172.21.15.83:0/2541357365 wait complete. 2024-04-24T09:11:34.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.408+0000 7f8cecd66740 1 -- start start 2024-04-24T09:11:34.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.408+0000 7f8cecd66740 1 Processor -- start 2024-04-24T09:11:34.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.409+0000 7f8cecd66740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642cf5a1b40 0x5642cf5d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:34.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.409+0000 7f8cecd66740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5642cf5a2460 0x5642cf5d3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:34.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.409+0000 7f8cecd66740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642cf5acb30 0x5642cf5a5c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:34.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.409+0000 7f8cecd66740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8ce41aa1d0 con 0x5642cf5acb30 2024-04-24T09:11:34.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.409+0000 7f8cecd66740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8ce41a9b00 con 0x5642cf5a1b40 2024-04-24T09:11:34.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.409+0000 7f8cecd66740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8ce4069a70 con 0x5642cf5a2460 2024-04-24T09:11:34.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.409+0000 7f8cecd66740 10 client.0 _ll_register_callbacks cb 0x7f8cd00d4840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:11:34.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.410+0000 7f8ce2000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5642cf5a2460 0x5642cf5d3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:34.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.410+0000 7f8ce3400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642cf5acb30 0x5642cf5a5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:34.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.410+0000 7f8ce2000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5642cf5a2460 0x5642cf5d3150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:38492/0 (socket says 172.21.15.83:38492) 2024-04-24T09:11:34.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.410+0000 7f8ce2000640 1 -- 172.21.15.83:0/2227998043 learned_addr learned my addr 172.21.15.83:0/2227998043 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:11:34.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.410+0000 7f8ce2a00640 1 --2- 172.21.15.83:0/2227998043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642cf5a1b40 0x5642cf5d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:34.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.410+0000 7f8ce2000640 1 -- 172.21.15.83:0/2227998043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642cf5a1b40 msgr2=0x5642cf5d2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:34.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.410+0000 7f8ce2000640 1 --2- 172.21.15.83:0/2227998043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642cf5a1b40 0x5642cf5d2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:34.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.410+0000 7f8ce2000640 1 -- 172.21.15.83:0/2227998043 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642cf5acb30 msgr2=0x5642cf5a5c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:34.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.410+0000 7f8ce2000640 1 --2- 172.21.15.83:0/2227998043 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642cf5acb30 0x5642cf5a5c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:34.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.410+0000 7f8ce2000640 1 -- 172.21.15.83:0/2227998043 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642cf5a6270 con 0x5642cf5a2460 2024-04-24T09:11:34.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.411+0000 7f8ce2000640 1 --2- 172.21.15.83:0/2227998043 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5642cf5a2460 0x5642cf5d3150 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8cd000c6a0 tx=0x7f8cd009c830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3493088270053fe server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:34.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.411+0000 7f8cccc00640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-04-24T09:11:34.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.411+0000 7f8cccc00640 1 -- 172.21.15.83:0/2227998043 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cd0009460 con 0x5642cf5a2460 2024-04-24T09:11:34.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.411+0000 7f8cecd66740 1 -- 172.21.15.83:0/2227998043 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8cd402e070 con 0x5642cf5a2460 2024-04-24T09:11:34.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.411+0000 7f8cccc00640 1 -- 172.21.15.83:0/2227998043 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 9583+0+0 (secure 0 0 0) 0x7f8cd0099010 con 0x5642cf5a2460 2024-04-24T09:11:34.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.412+0000 7f8cccc00640 1 -- 172.21.15.83:0/2227998043 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f8cd009d0a0 con 0x5642cf5a2460 2024-04-24T09:11:34.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.412+0000 7f8cecd66740 10 client.29452 fetch_fsmap learned FSMap version 61 2024-04-24T09:11:34.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.412+0000 7f8cecd66740 1 -- 172.21.15.83:0/2227998043 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x5642cf63bf60 con 0x5642cf5a2460 2024-04-24T09:11:34.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.412+0000 7f8cccc00640 1 -- 172.21.15.83:0/2227998043 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8cd402e070 con 0x5642cf5a2460 2024-04-24T09:11:34.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.413+0000 7f8cccc00640 1 -- 172.21.15.83:0/2227998043 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 61) ==== 8362+0+0 (secure 0 0 0) 0x7f8cd00870b0 con 0x5642cf5a2460 2024-04-24T09:11:34.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.413+0000 7f8cecd66740 10 client.29452 fetch_fsmap finished waiting for FSMap version 61 2024-04-24T09:11:34.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.413+0000 7f8cecd66740 10 client.29452 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-24T09:11:34.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.413+0000 7f8cecd66740 20 client.29452 populate_metadata read hostname 'smithi083' 2024-04-24T09:11:34.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.413+0000 7f8cecd66740 1 --2- 172.21.15.83:0/2227998043 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x5642cf63c690 0x5642cf63eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:34.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.413+0000 7f8cecd66740 4 client.29452 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:11:34.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.413+0000 7f8cecd66740 1 -- 172.21.15.83:0/2227998043 --> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5642cf50ea60 con 0x5642cf63c690 2024-04-24T09:11:34.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.413+0000 7f8ce2a00640 1 --2- 172.21.15.83:0/2227998043 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x5642cf63c690 0x5642cf63eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:34.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.414+0000 7f8ce2a00640 1 --2- 172.21.15.83:0/2227998043 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x5642cf63c690 0x5642cf63eab0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b1d10cf9e01471da server_cookie=c781a75a0420cb26 in_seq=0 out_seq=0 2024-04-24T09:11:34.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.414+0000 7f8cccc00640 10 client.29452 ms_handle_connect on v2:172.21.15.138:6834/1909219758 2024-04-24T09:11:34.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.415+0000 7f8cccc00640 1 -- 172.21.15.83:0/2227998043 <== mds.2 v2:172.21.15.138:6834/1909219758 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5642cf50ea60 con 0x5642cf63c690 2024-04-24T09:11:34.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.415+0000 7f8cccc00640 10 client.29452 handle_command_reply: tid=0 2024-04-24T09:11:34.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.415+0000 7f8cecd66740 1 client.29452 shutdown 2024-04-24T09:11:34.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.415+0000 7f8cecd66740 1 -- 172.21.15.83:0/2227998043 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5642cf5a2460 msgr2=0x5642cf5d3150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:34.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.415+0000 7f8cecd66740 1 --2- 172.21.15.83:0/2227998043 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5642cf5a2460 0x5642cf5d3150 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8cd000c6a0 tx=0x7f8cd009c830 comp rx=0 tx=0).stop 2024-04-24T09:11:34.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.415+0000 7f8cecd66740 1 -- 172.21.15.83:0/2227998043 shutdown_connections 2024-04-24T09:11:34.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.415+0000 7f8cecd66740 1 --2- 172.21.15.83:0/2227998043 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x5642cf63c690 0x5642cf63eab0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:34.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.415+0000 7f8cecd66740 1 --2- 172.21.15.83:0/2227998043 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642cf5acb30 0x5642cf5a5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:34.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.415+0000 7f8cecd66740 1 --2- 172.21.15.83:0/2227998043 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5642cf5a2460 0x5642cf5d3150 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:34.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.415+0000 7f8cecd66740 1 --2- 172.21.15.83:0/2227998043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642cf5a1b40 0x5642cf5d2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:34.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.415+0000 7f8cecd66740 1 -- 172.21.15.83:0/2227998043 >> 172.21.15.83:0/2227998043 conn(0x5642cf59de60 msgr2=0x5642cf5a8180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:34.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.415+0000 7f8cccc00640 0 client.29452 ms_handle_reset on v2:172.21.15.138:6834/1909219758 2024-04-24T09:11:34.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.415+0000 7f8cecd66740 1 -- 172.21.15.83:0/2227998043 shutdown_connections 2024-04-24T09:11:34.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.416+0000 7f8cecd66740 1 -- 172.21.15.83:0/2227998043 wait complete. 2024-04-24T09:11:34.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.416+0000 7f8cecd66740 20 client.29452 trim_cache size 0 max 16384 2024-04-24T09:11:34.416 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:11:34.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.417+0000 7f8ce3e00640 1 -- 172.21.15.83:0/898916474 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f8cc80a3da0 msgr2=0x7f8cc80a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:34.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.417+0000 7f8ce3e00640 1 --2- 172.21.15.83:0/898916474 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f8cc80a3da0 0x7f8cc80a6260 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x5642cf5a3100 tx=0x7f8cd4007600 comp rx=0 tx=0).stop 2024-04-24T09:11:34.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.417+0000 7f8ce3e00640 1 -- 172.21.15.83:0/898916474 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ce4103980 msgr2=0x7f8ce41a0af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:34.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.417+0000 7f8ce3e00640 1 --2- 172.21.15.83:0/898916474 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ce4103980 0x7f8ce41a0af0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f8cd000e9f0 tx=0x7f8cd000eeb0 comp rx=0 tx=0).stop 2024-04-24T09:11:34.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.417+0000 7f8ce3e00640 1 -- 172.21.15.83:0/898916474 shutdown_connections 2024-04-24T09:11:34.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.417+0000 7f8ce3e00640 1 --2- 172.21.15.83:0/898916474 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f8cc80a3da0 0x7f8cc80a6260 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:34.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.417+0000 7f8ce3e00640 1 --2- 172.21.15.83:0/898916474 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8ce410c480 0x7f8ce419a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:34.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.417+0000 7f8ce3e00640 1 --2- 172.21.15.83:0/898916474 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ce4103980 0x7f8ce41a0af0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:34.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.417+0000 7f8ce3e00640 1 --2- 172.21.15.83:0/898916474 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ce4103060 0x7f8ce41a05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:34.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.417+0000 7f8ce3e00640 1 -- 172.21.15.83:0/898916474 >> 172.21.15.83:0/898916474 conn(0x7f8ce40fc880 msgr2=0x7f8ce4101e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:34.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.417+0000 7f8ce3e00640 1 -- 172.21.15.83:0/898916474 shutdown_connections 2024-04-24T09:11:34.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.417+0000 7f8ce3e00640 1 -- 172.21.15.83:0/898916474 wait complete. 2024-04-24T09:11:34.437 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:11:34 smithi083 ceph-mon[27907]: pgmap v629: 129 pgs: 129 active+clean; 763 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.2 MiB/s wr, 305 op/s 2024-04-24T09:11:34.437 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:11:34 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/2164636584' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:11:34.439 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T09:11:34.704 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:11:34 smithi138 ceph-mon[31441]: pgmap v629: 129 pgs: 129 active+clean; 763 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.2 MiB/s wr, 305 op/s 2024-04-24T09:11:34.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:11:34 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/2164636584' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:11:34.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:11:34 smithi189 ceph-mon[31791]: pgmap v629: 129 pgs: 129 active+clean; 763 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.2 MiB/s wr, 305 op/s 2024-04-24T09:11:34.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:11:34 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/2164636584' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:11:34.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.711+0000 7efc51400640 1 -- 172.21.15.83:0/2799134799 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc4c076e50 msgr2=0x7efc4c1129a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:34.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.711+0000 7efc51400640 1 --2- 172.21.15.83:0/2799134799 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc4c076e50 0x7efc4c1129a0 secure :-1 s=READY pgs=2505 cs=0 l=1 rev1=1 crypto rx=0x7efc38009900 tx=0x7efc3802a410 comp rx=0 tx=0).stop 2024-04-24T09:11:34.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.711+0000 7efc51400640 1 -- 172.21.15.83:0/2799134799 shutdown_connections 2024-04-24T09:11:34.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.711+0000 7efc51400640 1 --2- 172.21.15.83:0/2799134799 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc4c112ee0 0x7efc4c1152d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:34.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.711+0000 7efc51400640 1 --2- 172.21.15.83:0/2799134799 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc4c076e50 0x7efc4c1129a0 unknown :-1 s=CLOSED pgs=2505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:34.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.711+0000 7efc51400640 1 --2- 172.21.15.83:0/2799134799 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efc4c076530 0x7efc4c076910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:34.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.711+0000 7efc51400640 1 -- 172.21.15.83:0/2799134799 >> 172.21.15.83:0/2799134799 conn(0x7efc4c0ffd90 msgr2=0x7efc4c1021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:34.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.712+0000 7efc51400640 1 -- 172.21.15.83:0/2799134799 shutdown_connections 2024-04-24T09:11:34.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.712+0000 7efc51400640 1 -- 172.21.15.83:0/2799134799 wait complete. 2024-04-24T09:11:34.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.713+0000 7efc51400640 1 Processor -- start 2024-04-24T09:11:34.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.713+0000 7efc51400640 1 -- start start 2024-04-24T09:11:34.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.714+0000 7efc51400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc4c076530 0x7efc4c19eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:34.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.714+0000 7efc51400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc4c076e50 0x7efc4c19f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:34.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.714+0000 7efc51400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efc4c112ee0 0x7efc4c1a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:34.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.714+0000 7efc51400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7efc4c1189a0 con 0x7efc4c076e50 2024-04-24T09:11:34.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.714+0000 7efc51400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efc4c118820 con 0x7efc4c076530 2024-04-24T09:11:34.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.714+0000 7efc51400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7efc4c118b20 con 0x7efc4c112ee0 2024-04-24T09:11:34.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.714+0000 7efc4a800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc4c076530 0x7efc4c19eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:34.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.714+0000 7efc49e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc4c076e50 0x7efc4c19f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:34.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.714+0000 7efc4a800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc4c076530 0x7efc4c19eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:57850/0 (socket says 172.21.15.83:57850) 2024-04-24T09:11:34.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.714+0000 7efc4a800640 1 -- 172.21.15.83:0/2334851735 learned_addr learned my addr 172.21.15.83:0/2334851735 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:11:34.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.714+0000 7efc4b200640 1 --2- 172.21.15.83:0/2334851735 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efc4c112ee0 0x7efc4c1a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:34.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.715+0000 7efc4a800640 1 -- 172.21.15.83:0/2334851735 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efc4c112ee0 msgr2=0x7efc4c1a3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:34.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.715+0000 7efc4a800640 1 --2- 172.21.15.83:0/2334851735 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efc4c112ee0 0x7efc4c1a3750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:34.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.715+0000 7efc4a800640 1 -- 172.21.15.83:0/2334851735 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc4c076e50 msgr2=0x7efc4c19f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:34.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.715+0000 7efc4a800640 1 --2- 172.21.15.83:0/2334851735 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc4c076e50 0x7efc4c19f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:34.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.715+0000 7efc4a800640 1 -- 172.21.15.83:0/2334851735 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc4c1a3e80 con 0x7efc4c076530 2024-04-24T09:11:34.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.715+0000 7efc4a800640 1 --2- 172.21.15.83:0/2334851735 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc4c076530 0x7efc4c19eee0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7efc3c00b4a0 tx=0x7efc3c00b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:34.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.716+0000 7efc43400640 1 -- 172.21.15.83:0/2334851735 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc3c014070 con 0x7efc4c076530 2024-04-24T09:11:34.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.716+0000 7efc51400640 1 -- 172.21.15.83:0/2334851735 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc4c1a4170 con 0x7efc4c076530 2024-04-24T09:11:34.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.716+0000 7efc43400640 1 -- 172.21.15.83:0/2334851735 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efc3c010ac0 con 0x7efc4c076530 2024-04-24T09:11:34.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.716+0000 7efc51400640 1 -- 172.21.15.83:0/2334851735 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc4c1aba00 con 0x7efc4c076530 2024-04-24T09:11:34.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.717+0000 7efc43400640 1 -- 172.21.15.83:0/2334851735 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc3c017c30 con 0x7efc4c076530 2024-04-24T09:11:34.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.720+0000 7efc541ed740 1 Processor -- start 2024-04-24T09:11:34.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.720+0000 7efc541ed740 1 -- start start 2024-04-24T09:11:34.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.721+0000 7efc541ed740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556631e4e950 0x556631e4ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:34.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.721+0000 7efc541ed740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556631e4f270 0x556631e59400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:34.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.721+0000 7efc541ed740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556631e59940 0x556631e5bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:34.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.721+0000 7efc541ed740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556631db41d0 con 0x556631e4e950 2024-04-24T09:11:34.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.721+0000 7efc541ed740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556631d5a450 con 0x556631e4f270 2024-04-24T09:11:34.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.721+0000 7efc541ed740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x556631db0190 con 0x556631e59940 2024-04-24T09:11:34.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.721+0000 7efc4b200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556631e4e950 0x556631e4ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:34.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.721+0000 7efc4b200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556631e4e950 0x556631e4ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55430/0 (socket says 172.21.15.83:55430) 2024-04-24T09:11:34.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.721+0000 7efc4b200640 1 -- 172.21.15.83:0/3564198897 learned_addr learned my addr 172.21.15.83:0/3564198897 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:11:34.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.722+0000 7efc49e00640 1 --2- 172.21.15.83:0/3564198897 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556631e59940 0x556631e5bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:34.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.722+0000 7efc4b200640 1 -- 172.21.15.83:0/3564198897 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556631e59940 msgr2=0x556631e5bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:34.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.722+0000 7efc4b200640 1 --2- 172.21.15.83:0/3564198897 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556631e59940 0x556631e5bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:34.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.722+0000 7efc4b200640 1 -- 172.21.15.83:0/3564198897 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556631e4f270 msgr2=0x556631e59400 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T09:11:34.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.722+0000 7efc43400640 1 -- 172.21.15.83:0/2334851735 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7efc3c017070 con 0x7efc4c076530 2024-04-24T09:11:34.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.722+0000 7efc4b200640 1 --2- 172.21.15.83:0/3564198897 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556631e4f270 0x556631e59400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:34.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.722+0000 7efc4b200640 1 -- 172.21.15.83:0/3564198897 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556631e17310 con 0x556631e4e950 2024-04-24T09:11:34.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.723+0000 7efc4b200640 1 --2- 172.21.15.83:0/3564198897 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556631e4e950 0x556631e4ed30 secure :-1 s=READY pgs=2506 cs=0 l=1 rev1=1 crypto rx=0x7efc44007ee0 tx=0x7efc4400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8c57405d37869e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:34.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.723+0000 7efc37400640 1 -- 172.21.15.83:0/3564198897 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc44010840 con 0x556631e4e950 2024-04-24T09:11:34.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.723+0000 7efc37400640 1 -- 172.21.15.83:0/3564198897 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efc440183f0 con 0x556631e4e950 2024-04-24T09:11:34.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.723+0000 7efc37400640 1 -- 172.21.15.83:0/3564198897 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc44017600 con 0x556631e4e950 2024-04-24T09:11:34.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.724+0000 7efc541ed740 1 -- 172.21.15.83:0/3564198897 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556631e4e950 msgr2=0x556631e4ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:34.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.724+0000 7efc541ed740 1 --2- 172.21.15.83:0/3564198897 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556631e4e950 0x556631e4ed30 secure :-1 s=READY pgs=2506 cs=0 l=1 rev1=1 crypto rx=0x7efc44007ee0 tx=0x7efc4400f790 comp rx=0 tx=0).stop 2024-04-24T09:11:34.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.724+0000 7efc43400640 1 --2- 172.21.15.83:0/2334851735 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7efc300a3d20 0x7efc300a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:34.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.725+0000 7efc43400640 1 -- 172.21.15.83:0/2334851735 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 9583+0+0 (secure 0 0 0) 0x7efc3c0cfea0 con 0x7efc4c076530 2024-04-24T09:11:34.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.725+0000 7efc541ed740 1 -- 172.21.15.83:0/3564198897 shutdown_connections 2024-04-24T09:11:34.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.725+0000 7efc541ed740 1 --2- 172.21.15.83:0/3564198897 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556631e59940 0x556631e5bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:34.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.725+0000 7efc541ed740 1 --2- 172.21.15.83:0/3564198897 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556631e4f270 0x556631e59400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:34.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.725+0000 7efc541ed740 1 --2- 172.21.15.83:0/3564198897 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556631e4e950 0x556631e4ed30 unknown :-1 s=CLOSED pgs=2506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:34.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.725+0000 7efc541ed740 1 -- 172.21.15.83:0/3564198897 >> 172.21.15.83:0/3564198897 conn(0x556631e4ac70 msgr2=0x556631e4b050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:34.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.725+0000 7efc4a800640 1 --2- 172.21.15.83:0/2334851735 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7efc300a3d20 0x7efc300a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:34.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.726+0000 7efc541ed740 1 -- 172.21.15.83:0/3564198897 shutdown_connections 2024-04-24T09:11:34.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.726+0000 7efc541ed740 1 -- 172.21.15.83:0/3564198897 wait complete. 2024-04-24T09:11:34.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.726+0000 7efc4a800640 1 --2- 172.21.15.83:0/2334851735 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7efc300a3d20 0x7efc300a61e0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x556631e4ff10 tx=0x7efc3c0047d0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:34.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.726+0000 7efc541ed740 1 -- start start 2024-04-24T09:11:34.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.727+0000 7efc541ed740 1 Processor -- start 2024-04-24T09:11:34.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.727+0000 7efc541ed740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556631e4e950 0x556631e56720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:34.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.727+0000 7efc541ed740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556631e4f270 0x556631e56c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:34.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.727+0000 7efc541ed740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556631e59940 0x556631e571a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:34.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.728+0000 7efc541ed740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556631e5c7d0 con 0x556631e4f270 2024-04-24T09:11:34.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.728+0000 7efc541ed740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556631d6d810 con 0x556631e59940 2024-04-24T09:11:34.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.728+0000 7efc541ed740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x556631e5c950 con 0x556631e4e950 2024-04-24T09:11:34.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.728+0000 7efc541ed740 10 client.0 _ll_register_callbacks cb 0x556631d3b140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:11:34.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.728+0000 7efc49e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556631e4f270 0x556631e56c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:34.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.728+0000 7efc49e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556631e4f270 0x556631e56c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55436/0 (socket says 172.21.15.83:55436) 2024-04-24T09:11:34.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.728+0000 7efc49e00640 1 -- 172.21.15.83:0/3962497950 learned_addr learned my addr 172.21.15.83:0/3962497950 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:11:34.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.728+0000 7efc4b200640 1 --2- 172.21.15.83:0/3962497950 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556631e59940 0x556631e571a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:34.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.728+0000 7efc4b200640 1 --2- 172.21.15.83:0/3962497950 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556631e4e950 0x556631e56720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:34.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.729+0000 7efc4b200640 1 -- 172.21.15.83:0/3962497950 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556631e4e950 msgr2=0x556631e56720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:34.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.729+0000 7efc4b200640 1 --2- 172.21.15.83:0/3962497950 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556631e4e950 0x556631e56720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:34.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.729+0000 7efc4b200640 1 -- 172.21.15.83:0/3962497950 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556631e4f270 msgr2=0x556631e56c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:34.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.729+0000 7efc4b200640 1 --2- 172.21.15.83:0/3962497950 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556631e4f270 0x556631e56c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:34.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.729+0000 7efc4b200640 1 -- 172.21.15.83:0/3962497950 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556631d7e3f0 con 0x556631e59940 2024-04-24T09:11:34.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.729+0000 7efc49e00640 1 --2- 172.21.15.83:0/3962497950 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556631e4f270 0x556631e56c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:11:34.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.729+0000 7efc4b200640 1 --2- 172.21.15.83:0/3962497950 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556631e59940 0x556631e571a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7efc44007c70 tx=0x7efc44012040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e68a33c4dc7442e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:34.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.729+0000 7efc2be00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:11:34.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.729+0000 7efc2be00640 1 -- 172.21.15.83:0/3962497950 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc44010c70 con 0x556631e59940 2024-04-24T09:11:34.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.730+0000 7efc541ed740 1 -- 172.21.15.83:0/3962497950 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc440183f0 con 0x556631e59940 2024-04-24T09:11:34.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.730+0000 7efc2be00640 1 -- 172.21.15.83:0/3962497950 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 9583+0+0 (secure 0 0 0) 0x7efc44025400 con 0x556631e59940 2024-04-24T09:11:34.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.730+0000 7efc2be00640 1 -- 172.21.15.83:0/3962497950 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7efc44010e10 con 0x556631e59940 2024-04-24T09:11:34.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.730+0000 7efc541ed740 10 client.39572 fetch_fsmap learned FSMap version 61 2024-04-24T09:11:34.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.730+0000 7efc541ed740 1 -- 172.21.15.83:0/3962497950 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x556631ac63f0 con 0x556631e59940 2024-04-24T09:11:34.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.730+0000 7efc2be00640 1 -- 172.21.15.83:0/3962497950 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efc440183f0 con 0x556631e59940 2024-04-24T09:11:34.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.731+0000 7efc2be00640 1 -- 172.21.15.83:0/3962497950 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 61) ==== 8362+0+0 (secure 0 0 0) 0x7efc44024040 con 0x556631e59940 2024-04-24T09:11:34.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.731+0000 7efc541ed740 10 client.39572 fetch_fsmap finished waiting for FSMap version 61 2024-04-24T09:11:34.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.731+0000 7efc541ed740 10 client.39572 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-24T09:11:34.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.731+0000 7efc541ed740 20 client.39572 populate_metadata read hostname 'smithi083' 2024-04-24T09:11:34.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.731+0000 7efc541ed740 1 --2- 172.21.15.83:0/3962497950 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x556631e69aa0 0x556631e6be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:34.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.731+0000 7efc541ed740 4 client.39572 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:11:34.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.731+0000 7efc541ed740 1 -- 172.21.15.83:0/3962497950 --> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556631dc1bf0 con 0x556631e69aa0 2024-04-24T09:11:34.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.732+0000 7efc49e00640 1 --2- 172.21.15.83:0/3962497950 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x556631e69aa0 0x556631e6be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:34.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.732+0000 7efc49e00640 1 --2- 172.21.15.83:0/3962497950 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x556631e69aa0 0x556631e6be90 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ad0753e6c40ff2cd server_cookie=ef5d518af1b77ad8 in_seq=0 out_seq=0 2024-04-24T09:11:34.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.732+0000 7efc2be00640 10 client.39572 ms_handle_connect on v2:172.21.15.138:6836/900566754 2024-04-24T09:11:34.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.733+0000 7efc2be00640 1 -- 172.21.15.83:0/3962497950 <== mds.1 v2:172.21.15.138:6836/900566754 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556631dc1bf0 con 0x556631e69aa0 2024-04-24T09:11:34.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.733+0000 7efc2be00640 10 client.39572 handle_command_reply: tid=0 2024-04-24T09:11:34.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.733+0000 7efc541ed740 1 client.39572 shutdown 2024-04-24T09:11:34.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.733+0000 7efc541ed740 1 -- 172.21.15.83:0/3962497950 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556631e59940 msgr2=0x556631e571a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:34.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.733+0000 7efc541ed740 1 --2- 172.21.15.83:0/3962497950 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556631e59940 0x556631e571a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7efc44007c70 tx=0x7efc44012040 comp rx=0 tx=0).stop 2024-04-24T09:11:34.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.733+0000 7efc541ed740 1 -- 172.21.15.83:0/3962497950 shutdown_connections 2024-04-24T09:11:34.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.733+0000 7efc541ed740 1 --2- 172.21.15.83:0/3962497950 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556631e59940 0x556631e571a0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:34.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.733+0000 7efc541ed740 1 --2- 172.21.15.83:0/3962497950 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556631e4f270 0x556631e56c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:34.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.733+0000 7efc541ed740 1 --2- 172.21.15.83:0/3962497950 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x556631e69aa0 0x556631e6be90 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:34.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.734+0000 7efc541ed740 1 --2- 172.21.15.83:0/3962497950 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556631e4e950 0x556631e56720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:34.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.734+0000 7efc2be00640 0 client.39572 ms_handle_reset on v2:172.21.15.138:6836/900566754 2024-04-24T09:11:34.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.734+0000 7efc541ed740 1 -- 172.21.15.83:0/3962497950 >> 172.21.15.83:0/3962497950 conn(0x556631e4ac70 msgr2=0x556631e50a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:34.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.734+0000 7efc541ed740 1 -- 172.21.15.83:0/3962497950 shutdown_connections 2024-04-24T09:11:34.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.734+0000 7efc541ed740 1 -- 172.21.15.83:0/3962497950 wait complete. 2024-04-24T09:11:34.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.734+0000 7efc541ed740 20 client.39572 trim_cache size 0 max 16384 2024-04-24T09:11:34.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.755+0000 7efc541ed740 1 Processor -- start 2024-04-24T09:11:34.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.755+0000 7efc541ed740 1 -- start start 2024-04-24T09:11:34.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.755+0000 7efc541ed740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556631e4e950 0x556631e65690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:34.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.755+0000 7efc541ed740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556631e4f270 0x556631e56fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:34.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.755+0000 7efc541ed740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556631e59940 0x556631e71580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:34.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.755+0000 7efc541ed740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7efc4c1b5c90 con 0x556631e4f270 2024-04-24T09:11:34.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.755+0000 7efc541ed740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efc4c1b77d0 con 0x556631e4e950 2024-04-24T09:11:34.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.755+0000 7efc541ed740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7efc4c1b6360 con 0x556631e59940 2024-04-24T09:11:34.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.756+0000 7efc49e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556631e4f270 0x556631e56fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:34.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.756+0000 7efc49e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556631e4f270 0x556631e56fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55442/0 (socket says 172.21.15.83:55442) 2024-04-24T09:11:34.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.756+0000 7efc49e00640 1 -- 172.21.15.83:0/1922534997 learned_addr learned my addr 172.21.15.83:0/1922534997 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:11:34.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.757+0000 7efc49e00640 1 -- 172.21.15.83:0/1922534997 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556631e59940 msgr2=0x556631e71580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:34.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.757+0000 7efc49e00640 1 --2- 172.21.15.83:0/1922534997 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556631e59940 0x556631e71580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:34.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.757+0000 7efc49e00640 1 -- 172.21.15.83:0/1922534997 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556631e4e950 msgr2=0x556631e65690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:34.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.757+0000 7efc49e00640 1 --2- 172.21.15.83:0/1922534997 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556631e4e950 0x556631e65690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:34.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.757+0000 7efc49e00640 1 -- 172.21.15.83:0/1922534997 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556631b907d0 con 0x556631e4f270 2024-04-24T09:11:34.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.757+0000 7efc49e00640 1 --2- 172.21.15.83:0/1922534997 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556631e4f270 0x556631e56fc0 secure :-1 s=READY pgs=2507 cs=0 l=1 rev1=1 crypto rx=0x7efc3802a930 tx=0x7efc38004320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e911ea338a1fdc6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:34.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.757+0000 7efc36a00640 1 -- 172.21.15.83:0/1922534997 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc3804a020 con 0x556631e4f270 2024-04-24T09:11:34.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.758+0000 7efc36a00640 1 -- 172.21.15.83:0/1922534997 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efc380467f0 con 0x556631e4f270 2024-04-24T09:11:34.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.758+0000 7efc36a00640 1 -- 172.21.15.83:0/1922534997 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc38035040 con 0x556631e4f270 2024-04-24T09:11:34.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.759+0000 7efc541ed740 1 -- 172.21.15.83:0/1922534997 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556631e4f270 msgr2=0x556631e56fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:34.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.759+0000 7efc541ed740 1 --2- 172.21.15.83:0/1922534997 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556631e4f270 0x556631e56fc0 secure :-1 s=READY pgs=2507 cs=0 l=1 rev1=1 crypto rx=0x7efc3802a930 tx=0x7efc38004320 comp rx=0 tx=0).stop 2024-04-24T09:11:34.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.760+0000 7efc541ed740 1 -- 172.21.15.83:0/1922534997 shutdown_connections 2024-04-24T09:11:34.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.760+0000 7efc541ed740 1 --2- 172.21.15.83:0/1922534997 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556631e59940 0x556631e71580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:34.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.760+0000 7efc541ed740 1 --2- 172.21.15.83:0/1922534997 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556631e4f270 0x556631e56fc0 unknown :-1 s=CLOSED pgs=2507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:34.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.760+0000 7efc541ed740 1 --2- 172.21.15.83:0/1922534997 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556631e4e950 0x556631e65690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:34.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.760+0000 7efc541ed740 1 -- 172.21.15.83:0/1922534997 >> 172.21.15.83:0/1922534997 conn(0x556631e4ac70 msgr2=0x556631e55750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:34.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.760+0000 7efc541ed740 1 -- 172.21.15.83:0/1922534997 shutdown_connections 2024-04-24T09:11:34.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.760+0000 7efc541ed740 1 -- 172.21.15.83:0/1922534997 wait complete. 2024-04-24T09:11:34.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.762+0000 7efc541ed740 1 -- start start 2024-04-24T09:11:34.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.762+0000 7efc541ed740 1 Processor -- start 2024-04-24T09:11:34.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.763+0000 7efc541ed740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556631e4e950 0x556631e5b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:34.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.763+0000 7efc541ed740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556631e4f270 0x556631e59d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:34.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.763+0000 7efc541ed740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556631e59940 0x556631e5a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:34.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.763+0000 7efc4b200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556631e4e950 0x556631e5b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:34.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.764+0000 7efc4b200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556631e4e950 0x556631e5b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55458/0 (socket says 172.21.15.83:55458) 2024-04-24T09:11:34.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.764+0000 7efc541ed740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7efc4c1b7100 con 0x556631e4e950 2024-04-24T09:11:34.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.764+0000 7efc541ed740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efc4c1b6a30 con 0x556631e59940 2024-04-24T09:11:34.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.764+0000 7efc541ed740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7efc4c1b3c90 con 0x556631e4f270 2024-04-24T09:11:34.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.764+0000 7efc541ed740 10 client.0 _ll_register_callbacks cb 0x7efc44007d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:11:34.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.764+0000 7efc4b200640 1 -- 172.21.15.83:0/3328975442 learned_addr learned my addr 172.21.15.83:0/3328975442 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:11:34.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.765+0000 7efc4b200640 1 --2- 172.21.15.83:0/3328975442 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556631e59940 0x556631e5a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:34.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.765+0000 7efc4b200640 1 -- 172.21.15.83:0/3328975442 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556631e4f270 msgr2=0x556631e59d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:34.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.765+0000 7efc4b200640 1 --2- 172.21.15.83:0/3328975442 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556631e4f270 0x556631e59d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:34.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.765+0000 7efc4b200640 1 -- 172.21.15.83:0/3328975442 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556631e59940 msgr2=0x556631e5a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:34.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.765+0000 7efc4b200640 1 --2- 172.21.15.83:0/3328975442 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556631e59940 0x556631e5a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:34.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.765+0000 7efc4b200640 1 -- 172.21.15.83:0/3328975442 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556631e60ba0 con 0x556631e4e950 2024-04-24T09:11:34.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.765+0000 7efc2be00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:11:34.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.765+0000 7efc4b200640 1 --2- 172.21.15.83:0/3328975442 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556631e4e950 0x556631e5b790 secure :-1 s=READY pgs=2508 cs=0 l=1 rev1=1 crypto rx=0x7efc44002230 tx=0x7efc44027a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd490da4391c4077 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:34.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.766+0000 7efc2be00640 1 -- 172.21.15.83:0/3328975442 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc44018940 con 0x556631e4e950 2024-04-24T09:11:34.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.766+0000 7efc2be00640 1 -- 172.21.15.83:0/3328975442 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 9583+0+0 (secure 0 0 0) 0x7efc44027da0 con 0x556631e4e950 2024-04-24T09:11:34.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.766+0000 7efc541ed740 1 -- 172.21.15.83:0/3328975442 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc380467f0 con 0x556631e4e950 2024-04-24T09:11:34.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.766+0000 7efc2be00640 1 -- 172.21.15.83:0/3328975442 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7efc4403f050 con 0x556631e4e950 2024-04-24T09:11:34.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.766+0000 7efc541ed740 10 client.22506 fetch_fsmap learned FSMap version 61 2024-04-24T09:11:34.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.766+0000 7efc541ed740 1 -- 172.21.15.83:0/3328975442 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x556631e61ea0 con 0x556631e4e950 2024-04-24T09:11:34.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.767+0000 7efc2be00640 1 -- 172.21.15.83:0/3328975442 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efc380467f0 con 0x556631e4e950 2024-04-24T09:11:34.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.767+0000 7efc2be00640 1 -- 172.21.15.83:0/3328975442 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 61) ==== 8362+0+0 (secure 0 0 0) 0x7efc44005120 con 0x556631e4e950 2024-04-24T09:11:34.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.767+0000 7efc541ed740 10 client.22506 fetch_fsmap finished waiting for FSMap version 61 2024-04-24T09:11:34.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.767+0000 7efc541ed740 10 client.22506 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-24T09:11:34.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.767+0000 7efc541ed740 20 client.22506 populate_metadata read hostname 'smithi083' 2024-04-24T09:11:34.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.767+0000 7efc541ed740 1 --2- 172.21.15.83:0/3328975442 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x556631ecfdf0 0x556631ed21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:34.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.767+0000 7efc541ed740 4 client.22506 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:11:34.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.767+0000 7efc541ed740 1 -- 172.21.15.83:0/3328975442 --> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556631dbba60 con 0x556631ecfdf0 2024-04-24T09:11:34.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.768+0000 7efc49e00640 1 --2- 172.21.15.83:0/3328975442 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x556631ecfdf0 0x556631ed21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:34.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.769+0000 7efc2be00640 10 client.22506 ms_handle_connect on v2:172.21.15.138:6836/900566754 2024-04-24T09:11:34.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.769+0000 7efc49e00640 1 --2- 172.21.15.83:0/3328975442 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x556631ecfdf0 0x556631ed21e0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=712f68e17a617207 server_cookie=4c10314dc5ea3d85 in_seq=0 out_seq=0 2024-04-24T09:11:34.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.769+0000 7efc2be00640 1 -- 172.21.15.83:0/3328975442 <== mds.1 v2:172.21.15.138:6836/900566754 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556631dbba60 con 0x556631ecfdf0 2024-04-24T09:11:34.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.769+0000 7efc2be00640 10 client.22506 handle_command_reply: tid=0 2024-04-24T09:11:34.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.769+0000 7efc541ed740 1 client.22506 shutdown 2024-04-24T09:11:34.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.770+0000 7efc541ed740 1 -- 172.21.15.83:0/3328975442 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556631e4e950 msgr2=0x556631e5b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:34.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.770+0000 7efc541ed740 1 --2- 172.21.15.83:0/3328975442 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556631e4e950 0x556631e5b790 secure :-1 s=READY pgs=2508 cs=0 l=1 rev1=1 crypto rx=0x7efc44002230 tx=0x7efc44027a20 comp rx=0 tx=0).stop 2024-04-24T09:11:34.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.771+0000 7efc541ed740 1 -- 172.21.15.83:0/3328975442 shutdown_connections 2024-04-24T09:11:34.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.771+0000 7efc541ed740 1 --2- 172.21.15.83:0/3328975442 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556631e59940 0x556631e5a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:34.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.771+0000 7efc541ed740 1 --2- 172.21.15.83:0/3328975442 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x556631ecfdf0 0x556631ed21e0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:34.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.771+0000 7efc541ed740 1 --2- 172.21.15.83:0/3328975442 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556631e4f270 0x556631e59d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:34.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.771+0000 7efc2be00640 0 client.22506 ms_handle_reset on v2:172.21.15.138:6836/900566754 2024-04-24T09:11:34.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.771+0000 7efc541ed740 1 --2- 172.21.15.83:0/3328975442 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556631e4e950 0x556631e5b790 unknown :-1 s=CLOSED pgs=2508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:34.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.771+0000 7efc541ed740 1 -- 172.21.15.83:0/3328975442 >> 172.21.15.83:0/3328975442 conn(0x556631e4ac70 msgr2=0x556631e49560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:34.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.771+0000 7efc541ed740 1 -- 172.21.15.83:0/3328975442 shutdown_connections 2024-04-24T09:11:34.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.771+0000 7efc541ed740 1 -- 172.21.15.83:0/3328975442 wait complete. 2024-04-24T09:11:34.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.771+0000 7efc541ed740 20 client.22506 trim_cache size 0 max 16384 2024-04-24T09:11:34.772 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:11:34.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.773+0000 7efc51400640 1 -- 172.21.15.83:0/2334851735 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7efc300a3d20 msgr2=0x7efc300a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:34.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.773+0000 7efc51400640 1 --2- 172.21.15.83:0/2334851735 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7efc300a3d20 0x7efc300a61e0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x556631e4ff10 tx=0x7efc3c0047d0 comp rx=0 tx=0).stop 2024-04-24T09:11:34.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.773+0000 7efc51400640 1 -- 172.21.15.83:0/2334851735 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc4c076530 msgr2=0x7efc4c19eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:34.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.773+0000 7efc51400640 1 --2- 172.21.15.83:0/2334851735 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc4c076530 0x7efc4c19eee0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7efc3c00b4a0 tx=0x7efc3c00b960 comp rx=0 tx=0).stop 2024-04-24T09:11:34.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.773+0000 7efc51400640 1 -- 172.21.15.83:0/2334851735 shutdown_connections 2024-04-24T09:11:34.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.773+0000 7efc51400640 1 --2- 172.21.15.83:0/2334851735 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7efc300a3d20 0x7efc300a61e0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:34.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.773+0000 7efc51400640 1 --2- 172.21.15.83:0/2334851735 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efc4c112ee0 0x7efc4c1a3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:34.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.773+0000 7efc51400640 1 --2- 172.21.15.83:0/2334851735 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc4c076e50 0x7efc4c19f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:34.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.773+0000 7efc51400640 1 --2- 172.21.15.83:0/2334851735 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc4c076530 0x7efc4c19eee0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:34.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.773+0000 7efc51400640 1 -- 172.21.15.83:0/2334851735 >> 172.21.15.83:0/2334851735 conn(0x7efc4c0ffd90 msgr2=0x7efc4c102180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:34.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.774+0000 7efc51400640 1 -- 172.21.15.83:0/2334851735 shutdown_connections 2024-04-24T09:11:34.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.774+0000 7efc51400640 1 -- 172.21.15.83:0/2334851735 wait complete. 2024-04-24T09:11:34.795 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T09:11:34.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.974+0000 7f88e1400640 1 -- 172.21.15.83:0/1282959747 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88dc1071f0 msgr2=0x7f88dc10edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:34.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.974+0000 7f88e1400640 1 --2- 172.21.15.83:0/1282959747 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88dc1071f0 0x7f88dc10edc0 secure :-1 s=READY pgs=2509 cs=0 l=1 rev1=1 crypto rx=0x7f88d4009ca0 tx=0x7f88d402a8c0 comp rx=0 tx=0).stop 2024-04-24T09:11:34.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.974+0000 7f88e1400640 1 -- 172.21.15.83:0/1282959747 shutdown_connections 2024-04-24T09:11:34.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.974+0000 7f88e1400640 1 --2- 172.21.15.83:0/1282959747 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88dc1071f0 0x7f88dc10edc0 unknown :-1 s=CLOSED pgs=2509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:34.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.974+0000 7f88e1400640 1 --2- 172.21.15.83:0/1282959747 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f88dc102ad0 0x7f88dc106b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:34.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.974+0000 7f88e1400640 1 --2- 172.21.15.83:0/1282959747 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88dc102120 0x7f88dc102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:34.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.974+0000 7f88e1400640 1 -- 172.21.15.83:0/1282959747 >> 172.21.15.83:0/1282959747 conn(0x7f88dc0fd9a0 msgr2=0x7f88dc0ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:34.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.974+0000 7f88e1400640 1 -- 172.21.15.83:0/1282959747 shutdown_connections 2024-04-24T09:11:34.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.974+0000 7f88e1400640 1 -- 172.21.15.83:0/1282959747 wait complete. 2024-04-24T09:11:34.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.975+0000 7f88e1400640 1 Processor -- start 2024-04-24T09:11:34.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.975+0000 7f88e1400640 1 -- start start 2024-04-24T09:11:34.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.975+0000 7f88e1400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f88dc102120 0x7f88dc1986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:34.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.975+0000 7f88e1400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88dc1071f0 0x7f88dc198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:34.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.975+0000 7f88e1400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88dc19cf70 0x7f88dc19d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:34.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.975+0000 7f88e1400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f88dc1121b0 con 0x7f88dc19cf70 2024-04-24T09:11:34.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.975+0000 7f88e1400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f88dc112030 con 0x7f88dc1071f0 2024-04-24T09:11:34.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.975+0000 7f88e1400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f88dc112330 con 0x7f88dc102120 2024-04-24T09:11:34.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.976+0000 7f88e0a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88dc19cf70 0x7f88dc19d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:34.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.976+0000 7f88e0a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88dc19cf70 0x7f88dc19d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55486/0 (socket says 172.21.15.83:55486) 2024-04-24T09:11:34.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.976+0000 7f88e0a00640 1 -- 172.21.15.83:0/943046180 learned_addr learned my addr 172.21.15.83:0/943046180 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:11:34.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.976+0000 7f88dbe00640 1 --2- 172.21.15.83:0/943046180 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f88dc102120 0x7f88dc1986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:34.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.976+0000 7f88db400640 1 --2- 172.21.15.83:0/943046180 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88dc1071f0 0x7f88dc198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:34.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.976+0000 7f88e0a00640 1 -- 172.21.15.83:0/943046180 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f88dc102120 msgr2=0x7f88dc1986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:34.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.976+0000 7f88e0a00640 1 --2- 172.21.15.83:0/943046180 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f88dc102120 0x7f88dc1986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:34.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.976+0000 7f88e0a00640 1 -- 172.21.15.83:0/943046180 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88dc1071f0 msgr2=0x7f88dc198bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:34.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.976+0000 7f88e0a00640 1 --2- 172.21.15.83:0/943046180 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88dc1071f0 0x7f88dc198bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:34.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.976+0000 7f88e0a00640 1 -- 172.21.15.83:0/943046180 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88dc19da60 con 0x7f88dc19cf70 2024-04-24T09:11:34.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.977+0000 7f88e0a00640 1 --2- 172.21.15.83:0/943046180 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88dc19cf70 0x7f88dc19d420 secure :-1 s=READY pgs=2510 cs=0 l=1 rev1=1 crypto rx=0x7f88d4031040 tx=0x7f88d4003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:34.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.977+0000 7f88d8c00640 1 -- 172.21.15.83:0/943046180 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88d4003ce0 con 0x7f88dc19cf70 2024-04-24T09:11:34.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.977+0000 7f88d8c00640 1 -- 172.21.15.83:0/943046180 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f88d40044f0 con 0x7f88dc19cf70 2024-04-24T09:11:34.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.977+0000 7f88e1400640 1 -- 172.21.15.83:0/943046180 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88dc19dcf0 con 0x7f88dc19cf70 2024-04-24T09:11:34.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.977+0000 7f88d8c00640 1 -- 172.21.15.83:0/943046180 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88d4030c90 con 0x7f88dc19cf70 2024-04-24T09:11:34.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.977+0000 7f88e1400640 1 -- 172.21.15.83:0/943046180 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88dc1a5510 con 0x7f88dc19cf70 2024-04-24T09:11:34.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.980+0000 7f88e583b740 1 Processor -- start 2024-04-24T09:11:34.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.980+0000 7f88d8c00640 1 -- 172.21.15.83:0/943046180 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f88d40046a0 con 0x7f88dc19cf70 2024-04-24T09:11:34.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.980+0000 7f88e583b740 1 -- start start 2024-04-24T09:11:34.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.980+0000 7f88e583b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e327a2870 0x560e327a2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:34.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.980+0000 7f88e583b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e327a3220 0x560e327b1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:34.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.981+0000 7f88e583b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e327b1f40 0x560e327b4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:34.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.981+0000 7f88e583b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x560e327081d0 con 0x560e327a3220 2024-04-24T09:11:34.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.981+0000 7f88e583b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560e326ae450 con 0x560e327b1f40 2024-04-24T09:11:34.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.981+0000 7f88e583b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x560e32704190 con 0x560e327a2870 2024-04-24T09:11:34.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.981+0000 7f88dbe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e327a3220 0x560e327b1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:34.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.981+0000 7f88dbe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e327a3220 0x560e327b1a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55502/0 (socket says 172.21.15.83:55502) 2024-04-24T09:11:34.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.981+0000 7f88dbe00640 1 -- 172.21.15.83:0/673220683 learned_addr learned my addr 172.21.15.83:0/673220683 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:11:34.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.981+0000 7f88db400640 1 --2- 172.21.15.83:0/673220683 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e327a2870 0x560e327a2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:34.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.981+0000 7f88dbe00640 1 -- 172.21.15.83:0/673220683 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e327a2870 msgr2=0x560e327a2c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:34.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.981+0000 7f88dbe00640 1 --2- 172.21.15.83:0/673220683 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e327a2870 0x560e327a2c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:34.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.981+0000 7f88db400640 1 --2- 172.21.15.83:0/673220683 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e327b1f40 0x560e327b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:34.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.981+0000 7f88dbe00640 1 -- 172.21.15.83:0/673220683 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e327b1f40 msgr2=0x560e327b4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:34.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.981+0000 7f88dbe00640 1 --2- 172.21.15.83:0/673220683 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e327b1f40 0x560e327b4330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:34.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.981+0000 7f88dbe00640 1 -- 172.21.15.83:0/673220683 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e3276b310 con 0x560e327a3220 2024-04-24T09:11:34.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.982+0000 7f88d8c00640 1 --2- 172.21.15.83:0/943046180 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f88c00a3d20 0x7f88c00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:34.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.982+0000 7f88dbe00640 1 --2- 172.21.15.83:0/673220683 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e327a3220 0x560e327b1a00 secure :-1 s=READY pgs=2511 cs=0 l=1 rev1=1 crypto rx=0x7f88cc00cbf0 tx=0x7f88cc00e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=719ea7e007c5e38a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:34.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.982+0000 7f88d8c00640 1 -- 172.21.15.83:0/943046180 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 9583+0+0 (secure 0 0 0) 0x7f88d40f2250 con 0x7f88dc19cf70 2024-04-24T09:11:34.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.982+0000 7f88db400640 1 --2- 172.21.15.83:0/943046180 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f88c00a3d20 0x7f88c00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:34.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.982+0000 7f88d1200640 1 -- 172.21.15.83:0/673220683 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88cc010630 con 0x560e327a3220 2024-04-24T09:11:34.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.982+0000 7f88d1200640 1 -- 172.21.15.83:0/673220683 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f88cc010c80 con 0x560e327a3220 2024-04-24T09:11:34.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.982+0000 7f88d1200640 1 -- 172.21.15.83:0/673220683 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88cc019b50 con 0x560e327a3220 2024-04-24T09:11:34.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.983+0000 7f88e583b740 1 -- 172.21.15.83:0/673220683 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e327a3220 msgr2=0x560e327b1a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:34.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.983+0000 7f88db400640 1 --2- 172.21.15.83:0/943046180 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f88c00a3d20 0x7f88c00a61e0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x560e3279fff0 tx=0x7f88c40046c0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:34.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.983+0000 7f88e583b740 1 --2- 172.21.15.83:0/673220683 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e327a3220 0x560e327b1a00 secure :-1 s=READY pgs=2511 cs=0 l=1 rev1=1 crypto rx=0x7f88cc00cbf0 tx=0x7f88cc00e720 comp rx=0 tx=0).stop 2024-04-24T09:11:34.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.983+0000 7f88e583b740 1 -- 172.21.15.83:0/673220683 shutdown_connections 2024-04-24T09:11:34.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.983+0000 7f88e583b740 1 --2- 172.21.15.83:0/673220683 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e327b1f40 0x560e327b4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:34.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.983+0000 7f88e583b740 1 --2- 172.21.15.83:0/673220683 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e327a3220 0x560e327b1a00 unknown :-1 s=CLOSED pgs=2511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:34.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.983+0000 7f88e583b740 1 --2- 172.21.15.83:0/673220683 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e327a2870 0x560e327a2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:34.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.983+0000 7f88e583b740 1 -- 172.21.15.83:0/673220683 >> 172.21.15.83:0/673220683 conn(0x560e3279ee60 msgr2=0x560e3279f240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:34.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.983+0000 7f88e583b740 1 -- 172.21.15.83:0/673220683 shutdown_connections 2024-04-24T09:11:34.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.983+0000 7f88e583b740 1 -- 172.21.15.83:0/673220683 wait complete. 2024-04-24T09:11:34.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.983+0000 7f88e583b740 1 -- start start 2024-04-24T09:11:34.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.984+0000 7f88e583b740 1 Processor -- start 2024-04-24T09:11:34.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.984+0000 7f88e583b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e327a2870 0x560e327b1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:34.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.984+0000 7f88e583b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e327a3220 0x560e327aec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:34.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.984+0000 7f88e583b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e327b1f40 0x560e327af260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:34.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.984+0000 7f88e583b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x560e327b4da0 con 0x560e327a3220 2024-04-24T09:11:34.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.984+0000 7f88e583b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560e326c1810 con 0x560e327a2870 2024-04-24T09:11:34.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.984+0000 7f88e583b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x560e327b4f20 con 0x560e327b1f40 2024-04-24T09:11:34.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.984+0000 7f88e583b740 10 client.0 _ll_register_callbacks cb 0x560e3268f140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:11:34.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.984+0000 7f88db400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e327a3220 0x560e327aec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:34.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.984+0000 7f88db400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e327a3220 0x560e327aec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55518/0 (socket says 172.21.15.83:55518) 2024-04-24T09:11:34.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.984+0000 7f88db400640 1 -- 172.21.15.83:0/1139287562 learned_addr learned my addr 172.21.15.83:0/1139287562 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:11:34.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.985+0000 7f88e0a00640 1 --2- 172.21.15.83:0/1139287562 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e327b1f40 0x560e327af260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:34.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.985+0000 7f88dbe00640 1 --2- 172.21.15.83:0/1139287562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e327a2870 0x560e327b1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:34.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.985+0000 7f88db400640 1 -- 172.21.15.83:0/1139287562 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e327b1f40 msgr2=0x560e327af260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:34.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.985+0000 7f88db400640 1 --2- 172.21.15.83:0/1139287562 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e327b1f40 0x560e327af260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:34.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.985+0000 7f88db400640 1 -- 172.21.15.83:0/1139287562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e327a2870 msgr2=0x560e327b1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:34.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.985+0000 7f88db400640 1 --2- 172.21.15.83:0/1139287562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e327a2870 0x560e327b1960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:34.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.985+0000 7f88db400640 1 -- 172.21.15.83:0/1139287562 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e326d23f0 con 0x560e327a3220 2024-04-24T09:11:34.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.985+0000 7f88db400640 1 --2- 172.21.15.83:0/1139287562 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e327a3220 0x560e327aec50 secure :-1 s=READY pgs=2512 cs=0 l=1 rev1=1 crypto rx=0x7f88c401cb50 tx=0x7f88c4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b98e0b0752b6d231 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:34.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.985+0000 7f88c9600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:11:34.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.985+0000 7f88c9600640 1 -- 172.21.15.83:0/1139287562 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c4029060 con 0x560e327a3220 2024-04-24T09:11:34.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.985+0000 7f88e583b740 1 -- 172.21.15.83:0/1139287562 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88cc010c80 con 0x560e327a3220 2024-04-24T09:11:34.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.985+0000 7f88c9600640 1 -- 172.21.15.83:0/1139287562 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 9583+0+0 (secure 0 0 0) 0x7f88c401ab10 con 0x560e327a3220 2024-04-24T09:11:34.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.986+0000 7f88c9600640 1 -- 172.21.15.83:0/1139287562 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f88c4030050 con 0x560e327a3220 2024-04-24T09:11:34.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.986+0000 7f88e583b740 10 client.22518 fetch_fsmap learned FSMap version 61 2024-04-24T09:11:34.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.986+0000 7f88e583b740 1 -- 172.21.15.83:0/1139287562 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x560e3241a3f0 con 0x560e327a3220 2024-04-24T09:11:34.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.986+0000 7f88c9600640 1 -- 172.21.15.83:0/1139287562 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f88cc010c80 con 0x560e327a3220 2024-04-24T09:11:34.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.986+0000 7f88c9600640 1 -- 172.21.15.83:0/1139287562 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 61) ==== 8362+0+0 (secure 0 0 0) 0x7f88c4018040 con 0x560e327a3220 2024-04-24T09:11:34.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.986+0000 7f88e583b740 10 client.22518 fetch_fsmap finished waiting for FSMap version 61 2024-04-24T09:11:34.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.986+0000 7f88e583b740 10 client.22518 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:11:34.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.986+0000 7f88e583b740 20 client.22518 populate_metadata read hostname 'smithi083' 2024-04-24T09:11:34.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.986+0000 7f88e583b740 1 --2- 172.21.15.83:0/1139287562 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x560e327c21e0 0x560e327c4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:34.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.986+0000 7f88e583b740 4 client.22518 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:11:34.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.986+0000 7f88e583b740 1 -- 172.21.15.83:0/1139287562 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560e32715bf0 con 0x560e327c21e0 2024-04-24T09:11:34.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.987+0000 7f88dbe00640 1 --2- 172.21.15.83:0/1139287562 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x560e327c21e0 0x560e327c4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:34.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.988+0000 7f88dbe00640 1 --2- 172.21.15.83:0/1139287562 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x560e327c21e0 0x560e327c4650 crc :-1 s=READY pgs=497 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=257715554457534c server_cookie=d0fac282ba086812 in_seq=0 out_seq=0 2024-04-24T09:11:34.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.988+0000 7f88c9600640 10 client.22518 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:11:34.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.989+0000 7f88c9600640 1 -- 172.21.15.83:0/1139287562 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560e32715bf0 con 0x560e327c21e0 2024-04-24T09:11:34.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.989+0000 7f88c9600640 10 client.22518 handle_command_reply: tid=0 2024-04-24T09:11:34.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.990+0000 7f88e583b740 1 client.22518 shutdown 2024-04-24T09:11:34.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.990+0000 7f88e583b740 1 -- 172.21.15.83:0/1139287562 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e327a3220 msgr2=0x560e327aec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:34.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.990+0000 7f88e583b740 1 --2- 172.21.15.83:0/1139287562 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e327a3220 0x560e327aec50 secure :-1 s=READY pgs=2512 cs=0 l=1 rev1=1 crypto rx=0x7f88c401cb50 tx=0x7f88c4004730 comp rx=0 tx=0).stop 2024-04-24T09:11:34.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.990+0000 7f88e583b740 1 -- 172.21.15.83:0/1139287562 shutdown_connections 2024-04-24T09:11:34.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.990+0000 7f88e583b740 1 --2- 172.21.15.83:0/1139287562 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x560e327c21e0 0x560e327c4650 crc :-1 s=READY pgs=497 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:34.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.990+0000 7f88e583b740 1 --2- 172.21.15.83:0/1139287562 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e327b1f40 0x560e327af260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:34.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.990+0000 7f88e583b740 1 --2- 172.21.15.83:0/1139287562 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e327a3220 0x560e327aec50 unknown :-1 s=CLOSED pgs=2512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:34.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.990+0000 7f88c9600640 0 client.22518 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:11:34.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.990+0000 7f88e583b740 1 --2- 172.21.15.83:0/1139287562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e327a2870 0x560e327b1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:34.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.990+0000 7f88e583b740 1 -- 172.21.15.83:0/1139287562 >> 172.21.15.83:0/1139287562 conn(0x560e3279ee60 msgr2=0x560e327a4cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:34.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.990+0000 7f88e583b740 1 -- 172.21.15.83:0/1139287562 shutdown_connections 2024-04-24T09:11:34.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.990+0000 7f88e583b740 1 -- 172.21.15.83:0/1139287562 wait complete. 2024-04-24T09:11:34.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:34.990+0000 7f88e583b740 20 client.22518 trim_cache size 0 max 16384 2024-04-24T09:11:35.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:35.004+0000 7f88e583b740 1 Processor -- start 2024-04-24T09:11:35.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:35.004+0000 7f88e583b740 1 -- start start 2024-04-24T09:11:35.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:35.004+0000 7f88e583b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e327a2870 0x560e327bdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:35.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:35.004+0000 7f88e583b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e327a3220 0x560e327b1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:35.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:35.004+0000 7f88e583b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e327b1f40 0x560e327c9f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:35.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:35.004+0000 7f88e583b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f88dc1ad290 con 0x560e327a2870 2024-04-24T09:11:35.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:35.004+0000 7f88e583b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f88dc1aedd0 con 0x560e327a3220 2024-04-24T09:11:35.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:35.004+0000 7f88e583b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f88dc1ad960 con 0x560e327b1f40 2024-04-24T09:11:35.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:35.005+0000 7f88dbe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e327a2870 0x560e327bdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:35.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:35.005+0000 7f88dbe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e327a2870 0x560e327bdcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55532/0 (socket says 172.21.15.83:55532) 2024-04-24T09:11:35.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:35.005+0000 7f88dbe00640 1 -- 172.21.15.83:0/3821080765 learned_addr learned my addr 172.21.15.83:0/3821080765 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:11:35.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:35.005+0000 7f88dbe00640 1 -- 172.21.15.83:0/3821080765 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e327b1f40 msgr2=0x560e327c9f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:35.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:35.005+0000 7f88e0a00640 1 --2- 172.21.15.83:0/3821080765 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e327b1f40 0x560e327c9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:35.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:35.005+0000 7f88dbe00640 1 --2- 172.21.15.83:0/3821080765 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e327b1f40 0x560e327c9f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:35.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:35.005+0000 7f88dbe00640 1 -- 172.21.15.83:0/3821080765 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e327a3220 msgr2=0x560e327b1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:35.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:35.005+0000 7f88dbe00640 1 --2- 172.21.15.83:0/3821080765 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e327a3220 0x560e327b1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:35.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:35.005+0000 7f88dbe00640 1 -- 172.21.15.83:0/3821080765 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e324e47d0 con 0x560e327a2870 2024-04-24T09:11:35.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:35.005+0000 7f88dbe00640 1 --2- 172.21.15.83:0/3821080765 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e327a2870 0x560e327bdcd0 secure :-1 s=READY pgs=2513 cs=0 l=1 rev1=1 crypto rx=0x7f88cc019430 tx=0x7f88cc00c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af443d9d954428ff server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:35.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:35.006+0000 7f88cbe00640 1 -- 172.21.15.83:0/3821080765 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88cc004020 con 0x560e327a2870 2024-04-24T09:11:35.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:35.006+0000 7f88cbe00640 1 -- 172.21.15.83:0/3821080765 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f88cc026930 con 0x560e327a2870 2024-04-24T09:11:35.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:35.006+0000 7f88cbe00640 1 -- 172.21.15.83:0/3821080765 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88cc005590 con 0x560e327a2870 2024-04-24T09:11:35.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:35.006+0000 7f88e583b740 1 -- 172.21.15.83:0/3821080765 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e327a2870 msgr2=0x560e327bdcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:35.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:35.006+0000 7f88e583b740 1 --2- 172.21.15.83:0/3821080765 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e327a2870 0x560e327bdcd0 secure :-1 s=READY pgs=2513 cs=0 l=1 rev1=1 crypto rx=0x7f88cc019430 tx=0x7f88cc00c4f0 comp rx=0 tx=0).stop 2024-04-24T09:11:35.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:35.006+0000 7f88e583b740 1 -- 172.21.15.83:0/3821080765 shutdown_connections 2024-04-24T09:11:35.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:35.006+0000 7f88e583b740 1 --2- 172.21.15.83:0/3821080765 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e327b1f40 0x560e327c9f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:35.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:35.006+0000 7f88e583b740 1 --2- 172.21.15.83:0/3821080765 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e327a3220 0x560e327b1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:35.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:35.006+0000 7f88e583b740 1 --2- 172.21.15.83:0/3821080765 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e327a2870 0x560e327bdcd0 unknown :-1 s=CLOSED pgs=2513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:35.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:35.006+0000 7f88e583b740 1 -- 172.21.15.83:0/3821080765 >> 172.21.15.83:0/3821080765 conn(0x560e3279ee60 msgr2=0x560e327a9f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:35.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:35.006+0000 7f88e583b740 1 -- 172.21.15.83:0/3821080765 shutdown_connections 2024-04-24T09:11:35.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:35.006+0000 7f88e583b740 1 -- 172.21.15.83:0/3821080765 wait complete. 2024-04-24T09:11:35.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:35.007+0000 7f88e583b740 1 -- start start 2024-04-24T09:11:35.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:35.007+0000 7f88e583b740 1 Processor -- start 2024-04-24T09:11:35.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:35.007+0000 7f88e583b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e327a2870 0x560e327ba750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:35.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:35.007+0000 7f88e583b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e327a3220 0x560e327ac100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:35.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:35.007+0000 7f88e583b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e327b1f40 0x560e327ac7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:35.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:35.007+0000 7f88e583b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f88dc1ae700 con 0x560e327a3220 2024-04-24T09:11:35.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:35.007+0000 7f88e583b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f88dc1ae030 con 0x560e327a2870 2024-04-24T09:11:35.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:35.007+0000 7f88e583b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f88dc1ab290 con 0x560e327b1f40 2024-04-24T09:11:35.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:35.007+0000 7f88e583b740 10 client.0 _ll_register_callbacks cb 0x7f88c400e8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:11:35.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:35.007+0000 7f88db400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e327a3220 0x560e327ac100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:35.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:35.008+0000 7f88db400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e327a3220 0x560e327ac100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55538/0 (socket says 172.21.15.83:55538) 2024-04-24T09:11:35.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:35.008+0000 7f88db400640 1 -- 172.21.15.83:0/3229397041 learned_addr learned my addr 172.21.15.83:0/3229397041 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:11:35.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:35.008+0000 7f88db400640 1 -- 172.21.15.83:0/3229397041 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e327b1f40 msgr2=0x560e327ac7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:35.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:35.008+0000 7f88db400640 1 --2- 172.21.15.83:0/3229397041 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e327b1f40 0x560e327ac7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:35.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:35.008+0000 7f88db400640 1 -- 172.21.15.83:0/3229397041 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e327a2870 msgr2=0x560e327ba750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:35.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:35.008+0000 7f88db400640 1 --2- 172.21.15.83:0/3229397041 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e327a2870 0x560e327ba750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:35.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:35.008+0000 7f88db400640 1 -- 172.21.15.83:0/3229397041 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e327b2df0 con 0x560e327a3220 2024-04-24T09:11:35.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:35.008+0000 7f88db400640 1 --2- 172.21.15.83:0/3229397041 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e327a3220 0x560e327ac100 secure :-1 s=READY pgs=2514 cs=0 l=1 rev1=1 crypto rx=0x7f88c4024e70 tx=0x7f88c401adc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59466e3458c21ad9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:35.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:35.008+0000 7f88c9600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:11:35.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:35.008+0000 7f88c9600640 1 -- 172.21.15.83:0/3229397041 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c4005450 con 0x560e327a3220 2024-04-24T09:11:35.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:35.009+0000 7f88e583b740 1 -- 172.21.15.83:0/3229397041 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88cc026930 con 0x560e327a3220 2024-04-24T09:11:35.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:35.009+0000 7f88c9600640 1 -- 172.21.15.83:0/3229397041 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 9583+0+0 (secure 0 0 0) 0x7f88c4004730 con 0x560e327a3220 2024-04-24T09:11:35.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:35.009+0000 7f88c9600640 1 -- 172.21.15.83:0/3229397041 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f88c4029020 con 0x560e327a3220 2024-04-24T09:11:35.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:35.009+0000 7f88e583b740 10 client.22524 fetch_fsmap learned FSMap version 61 2024-04-24T09:11:35.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:35.009+0000 7f88e583b740 1 -- 172.21.15.83:0/3229397041 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x560e32828b50 con 0x560e327a3220 2024-04-24T09:11:35.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:35.009+0000 7f88c9600640 1 -- 172.21.15.83:0/3229397041 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f88cc026930 con 0x560e327a3220 2024-04-24T09:11:35.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:35.009+0000 7f88c9600640 1 -- 172.21.15.83:0/3229397041 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 61) ==== 8362+0+0 (secure 0 0 0) 0x7f88c403c030 con 0x560e327a3220 2024-04-24T09:11:35.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:35.009+0000 7f88e583b740 10 client.22524 fetch_fsmap finished waiting for FSMap version 61 2024-04-24T09:11:35.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:35.009+0000 7f88e583b740 10 client.22524 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:11:35.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:35.009+0000 7f88e583b740 20 client.22524 populate_metadata read hostname 'smithi083' 2024-04-24T09:11:35.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:35.009+0000 7f88e583b740 1 --2- 172.21.15.83:0/3229397041 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x560e32829320 0x560e3282b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:35.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:35.009+0000 7f88e583b740 4 client.22524 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:11:35.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:35.009+0000 7f88e583b740 1 -- 172.21.15.83:0/3229397041 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560e3270fa60 con 0x560e32829320 2024-04-24T09:11:35.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:35.010+0000 7f88dbe00640 1 --2- 172.21.15.83:0/3229397041 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x560e32829320 0x560e3282b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:35.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:35.011+0000 7f88dbe00640 1 --2- 172.21.15.83:0/3229397041 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x560e32829320 0x560e3282b740 crc :-1 s=READY pgs=498 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd21ee5628c849c3 server_cookie=c9ab883a924eafc3 in_seq=0 out_seq=0 2024-04-24T09:11:35.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:35.011+0000 7f88c9600640 10 client.22524 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:11:35.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:35.011+0000 7f88c9600640 1 -- 172.21.15.83:0/3229397041 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560e3270fa60 con 0x560e32829320 2024-04-24T09:11:35.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:35.011+0000 7f88c9600640 10 client.22524 handle_command_reply: tid=0 2024-04-24T09:11:35.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:35.011+0000 7f88e583b740 1 client.22524 shutdown 2024-04-24T09:11:35.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:35.012+0000 7f88e583b740 1 -- 172.21.15.83:0/3229397041 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e327a3220 msgr2=0x560e327ac100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:35.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:35.012+0000 7f88e583b740 1 --2- 172.21.15.83:0/3229397041 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e327a3220 0x560e327ac100 secure :-1 s=READY pgs=2514 cs=0 l=1 rev1=1 crypto rx=0x7f88c4024e70 tx=0x7f88c401adc0 comp rx=0 tx=0).stop 2024-04-24T09:11:35.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:35.012+0000 7f88e583b740 1 -- 172.21.15.83:0/3229397041 shutdown_connections 2024-04-24T09:11:35.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:35.012+0000 7f88e583b740 1 --2- 172.21.15.83:0/3229397041 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x560e32829320 0x560e3282b740 crc :-1 s=READY pgs=498 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:35.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:35.012+0000 7f88e583b740 1 --2- 172.21.15.83:0/3229397041 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e327b1f40 0x560e327ac7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:35.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:35.012+0000 7f88e583b740 1 --2- 172.21.15.83:0/3229397041 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e327a3220 0x560e327ac100 unknown :-1 s=CLOSED pgs=2514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:35.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:35.012+0000 7f88c9600640 0 client.22524 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:11:35.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:35.012+0000 7f88e583b740 1 --2- 172.21.15.83:0/3229397041 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e327a2870 0x560e327ba750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:35.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:35.012+0000 7f88e583b740 1 -- 172.21.15.83:0/3229397041 >> 172.21.15.83:0/3229397041 conn(0x560e3279ee60 msgr2=0x560e327a4420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:35.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:35.012+0000 7f88e583b740 1 -- 172.21.15.83:0/3229397041 shutdown_connections 2024-04-24T09:11:35.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:35.012+0000 7f88e583b740 1 -- 172.21.15.83:0/3229397041 wait complete. 2024-04-24T09:11:35.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:35.012+0000 7f88e583b740 20 client.22524 trim_cache size 0 max 16384 2024-04-24T09:11:35.014 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:11:35.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:35.012+0000 7f88e1400640 1 -- 172.21.15.83:0/943046180 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f88c00a3d20 msgr2=0x7f88c00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:35.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:35.012+0000 7f88e1400640 1 --2- 172.21.15.83:0/943046180 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f88c00a3d20 0x7f88c00a61e0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x560e3279fff0 tx=0x7f88c40046c0 comp rx=0 tx=0).stop 2024-04-24T09:11:35.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:35.012+0000 7f88e1400640 1 -- 172.21.15.83:0/943046180 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88dc19cf70 msgr2=0x7f88dc19d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:35.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:35.012+0000 7f88e1400640 1 --2- 172.21.15.83:0/943046180 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88dc19cf70 0x7f88dc19d420 secure :-1 s=READY pgs=2510 cs=0 l=1 rev1=1 crypto rx=0x7f88d4031040 tx=0x7f88d4003970 comp rx=0 tx=0).stop 2024-04-24T09:11:35.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:35.013+0000 7f88e1400640 1 -- 172.21.15.83:0/943046180 shutdown_connections 2024-04-24T09:11:35.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:35.013+0000 7f88e1400640 1 --2- 172.21.15.83:0/943046180 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f88c00a3d20 0x7f88c00a61e0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:35.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:35.013+0000 7f88e1400640 1 --2- 172.21.15.83:0/943046180 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88dc19cf70 0x7f88dc19d420 unknown :-1 s=CLOSED pgs=2510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:35.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:35.013+0000 7f88e1400640 1 --2- 172.21.15.83:0/943046180 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88dc1071f0 0x7f88dc198bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:35.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:35.013+0000 7f88e1400640 1 --2- 172.21.15.83:0/943046180 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f88dc102120 0x7f88dc1986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:35.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:35.013+0000 7f88e1400640 1 -- 172.21.15.83:0/943046180 >> 172.21.15.83:0/943046180 conn(0x7f88dc0fd9a0 msgr2=0x7f88dc0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:35.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:35.013+0000 7f88e1400640 1 -- 172.21.15.83:0/943046180 shutdown_connections 2024-04-24T09:11:35.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:35.013+0000 7f88e1400640 1 -- 172.21.15.83:0/943046180 wait complete. 2024-04-24T09:11:35.182 INFO:tasks.workunit.client.0.smithi083.stdout: 10 46004 6.71 MB/sec execute 246 sec latency 392.436 ms 2024-04-24T09:11:36.028 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T09:11:36.028 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T09:11:36.028 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T09:11:36.182 INFO:tasks.workunit.client.0.smithi083.stdout: 10 46097 6.71 MB/sec execute 247 sec latency 472.730 ms 2024-04-24T09:11:36.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.296+0000 7f8425400640 1 -- 172.21.15.83:0/4278360303 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8420104d00 msgr2=0x7f8420108d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:36.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.296+0000 7f8425400640 1 --2- 172.21.15.83:0/4278360303 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8420104d00 0x7f8420108d40 secure :-1 s=READY pgs=2515 cs=0 l=1 rev1=1 crypto rx=0x7f840c009900 tx=0x7f840c02a3f0 comp rx=0 tx=0).stop 2024-04-24T09:11:36.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.296+0000 7f8425400640 1 -- 172.21.15.83:0/4278360303 shutdown_connections 2024-04-24T09:11:36.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.296+0000 7f8425400640 1 --2- 172.21.15.83:0/4278360303 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8420109420 0x7f8420110ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:36.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.296+0000 7f8425400640 1 --2- 172.21.15.83:0/4278360303 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8420104d00 0x7f8420108d40 unknown :-1 s=CLOSED pgs=2515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:36.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.296+0000 7f8425400640 1 --2- 172.21.15.83:0/4278360303 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8420104350 0x7f8420104730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:36.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.296+0000 7f8425400640 1 -- 172.21.15.83:0/4278360303 >> 172.21.15.83:0/4278360303 conn(0x7f84200ffb70 msgr2=0x7f8420101f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:36.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.296+0000 7f8425400640 1 -- 172.21.15.83:0/4278360303 shutdown_connections 2024-04-24T09:11:36.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.296+0000 7f8425400640 1 -- 172.21.15.83:0/4278360303 wait complete. 2024-04-24T09:11:36.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.298+0000 7f8425400640 1 Processor -- start 2024-04-24T09:11:36.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.298+0000 7f8425400640 1 -- start start 2024-04-24T09:11:36.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.299+0000 7f8425400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8420104350 0x7f84201a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:36.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.299+0000 7f8425400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8420109420 0x7f84201a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:36.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.299+0000 7f8425400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f842019a860 0x7f842019ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:36.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.299+0000 7f8425400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8420114360 con 0x7f8420109420 2024-04-24T09:11:36.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.299+0000 7f8425400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f84201141e0 con 0x7f8420104350 2024-04-24T09:11:36.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.299+0000 7f8425400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f84201144e0 con 0x7f842019a860 2024-04-24T09:11:36.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.299+0000 7f841de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8420109420 0x7f84201a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:36.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.300+0000 7f841de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8420109420 0x7f84201a0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55560/0 (socket says 172.21.15.83:55560) 2024-04-24T09:11:36.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.300+0000 7f841de00640 1 -- 172.21.15.83:0/2416069072 learned_addr learned my addr 172.21.15.83:0/2416069072 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:11:36.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.300+0000 7f841e800640 1 --2- 172.21.15.83:0/2416069072 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8420104350 0x7f84201a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:36.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.300+0000 7f841de00640 1 -- 172.21.15.83:0/2416069072 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f842019a860 msgr2=0x7f842019ad10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-24T09:11:36.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.300+0000 7f841de00640 1 --2- 172.21.15.83:0/2416069072 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f842019a860 0x7f842019ad10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:36.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.300+0000 7f841de00640 1 -- 172.21.15.83:0/2416069072 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8420104350 msgr2=0x7f84201a06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:36.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.300+0000 7f841de00640 1 --2- 172.21.15.83:0/2416069072 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8420104350 0x7f84201a06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:36.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.300+0000 7f841de00640 1 -- 172.21.15.83:0/2416069072 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f842019b3f0 con 0x7f8420109420 2024-04-24T09:11:36.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.301+0000 7f841de00640 1 --2- 172.21.15.83:0/2416069072 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8420109420 0x7f84201a0be0 secure :-1 s=READY pgs=2516 cs=0 l=1 rev1=1 crypto rx=0x7f840c004270 tx=0x7f840c0042a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:36.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.301+0000 7f8417400640 1 -- 172.21.15.83:0/2416069072 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f840c03f030 con 0x7f8420109420 2024-04-24T09:11:36.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.301+0000 7f8425400640 1 -- 172.21.15.83:0/2416069072 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f842019b680 con 0x7f8420109420 2024-04-24T09:11:36.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.301+0000 7f8417400640 1 -- 172.21.15.83:0/2416069072 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f840c0044c0 con 0x7f8420109420 2024-04-24T09:11:36.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.301+0000 7f8417400640 1 -- 172.21.15.83:0/2416069072 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f840c035070 con 0x7f8420109420 2024-04-24T09:11:36.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.301+0000 7f8425400640 1 -- 172.21.15.83:0/2416069072 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84201a7810 con 0x7f8420109420 2024-04-24T09:11:36.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.306+0000 7f84283ad740 1 Processor -- start 2024-04-24T09:11:36.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.306+0000 7f84283ad740 1 -- start start 2024-04-24T09:11:36.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.307+0000 7f84283ad740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a1be9ddfc0 0x55a1be9de3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:36.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.307+0000 7f84283ad740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1be9de970 0x55a1be9f1530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:36.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.307+0000 7f84283ad740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1be9f1a70 0x55a1be9f3e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:36.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.307+0000 7f84283ad740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a1be943e70 con 0x55a1be9de970 2024-04-24T09:11:36.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.307+0000 7f84283ad740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a1be8ea380 con 0x55a1be9f1a70 2024-04-24T09:11:36.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.307+0000 7f84283ad740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a1be93fe30 con 0x55a1be9ddfc0 2024-04-24T09:11:36.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.307+0000 7f8417400640 1 -- 172.21.15.83:0/2416069072 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f840c042030 con 0x7f8420109420 2024-04-24T09:11:36.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.308+0000 7f841e800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1be9de970 0x55a1be9f1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:36.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.308+0000 7f841e800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1be9de970 0x55a1be9f1530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55572/0 (socket says 172.21.15.83:55572) 2024-04-24T09:11:36.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.308+0000 7f841e800640 1 -- 172.21.15.83:0/3349741812 learned_addr learned my addr 172.21.15.83:0/3349741812 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:11:36.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.308+0000 7f841de00640 1 --2- 172.21.15.83:0/3349741812 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1be9f1a70 0x55a1be9f3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:36.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.308+0000 7f841e800640 1 -- 172.21.15.83:0/3349741812 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a1be9ddfc0 msgr2=0x55a1be9de3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:36.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.308+0000 7f841e800640 1 --2- 172.21.15.83:0/3349741812 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a1be9ddfc0 0x55a1be9de3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:36.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.308+0000 7f841e800640 1 -- 172.21.15.83:0/3349741812 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1be9f1a70 msgr2=0x55a1be9f3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:36.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.308+0000 7f841e800640 1 --2- 172.21.15.83:0/3349741812 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1be9f1a70 0x55a1be9f3e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:36.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.308+0000 7f841e800640 1 -- 172.21.15.83:0/3349741812 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1be71e7e0 con 0x55a1be9de970 2024-04-24T09:11:36.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.309+0000 7f841e800640 1 --2- 172.21.15.83:0/3349741812 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1be9de970 0x55a1be9f1530 secure :-1 s=READY pgs=2517 cs=0 l=1 rev1=1 crypto rx=0x7f841000b760 tx=0x7f841000bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dc57836679a6c82 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:36.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.309+0000 7f840b400640 1 -- 172.21.15.83:0/3349741812 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8410003dd0 con 0x55a1be9de970 2024-04-24T09:11:36.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.309+0000 7f840b400640 1 -- 172.21.15.83:0/3349741812 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8410015070 con 0x55a1be9de970 2024-04-24T09:11:36.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.309+0000 7f840b400640 1 -- 172.21.15.83:0/3349741812 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8410004d90 con 0x55a1be9de970 2024-04-24T09:11:36.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.309+0000 7f8417400640 1 --2- 172.21.15.83:0/2416069072 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f84040a3da0 0x7f84040a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:36.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.309+0000 7f8417400640 1 -- 172.21.15.83:0/2416069072 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 9583+0+0 (secure 0 0 0) 0x7f840c0f0d80 con 0x7f8420109420 2024-04-24T09:11:36.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.310+0000 7f84283ad740 1 -- 172.21.15.83:0/3349741812 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1be9de970 msgr2=0x55a1be9f1530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:36.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.310+0000 7f84283ad740 1 --2- 172.21.15.83:0/3349741812 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1be9de970 0x55a1be9f1530 secure :-1 s=READY pgs=2517 cs=0 l=1 rev1=1 crypto rx=0x7f841000b760 tx=0x7f841000bc20 comp rx=0 tx=0).stop 2024-04-24T09:11:36.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.311+0000 7f84283ad740 1 -- 172.21.15.83:0/3349741812 shutdown_connections 2024-04-24T09:11:36.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.311+0000 7f84283ad740 1 --2- 172.21.15.83:0/3349741812 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1be9f1a70 0x55a1be9f3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:36.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.311+0000 7f84283ad740 1 --2- 172.21.15.83:0/3349741812 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1be9de970 0x55a1be9f1530 unknown :-1 s=CLOSED pgs=2517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:36.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.311+0000 7f84283ad740 1 --2- 172.21.15.83:0/3349741812 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a1be9ddfc0 0x55a1be9de3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:36.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.311+0000 7f84283ad740 1 -- 172.21.15.83:0/3349741812 >> 172.21.15.83:0/3349741812 conn(0x55a1be9da590 msgr2=0x55a1be9da970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:36.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.311+0000 7f841f200640 1 --2- 172.21.15.83:0/2416069072 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f84040a3da0 0x7f84040a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:36.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.312+0000 7f84283ad740 1 -- 172.21.15.83:0/3349741812 shutdown_connections 2024-04-24T09:11:36.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.313+0000 7f841f200640 1 --2- 172.21.15.83:0/2416069072 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f84040a3da0 0x7f84040a6260 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f8418005fd0 tx=0x7f8418009630 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:36.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.313+0000 7f84283ad740 1 -- 172.21.15.83:0/3349741812 wait complete. 2024-04-24T09:11:36.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.314+0000 7f84283ad740 1 -- start start 2024-04-24T09:11:36.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.314+0000 7f84283ad740 1 Processor -- start 2024-04-24T09:11:36.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.315+0000 7f84283ad740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1be9ddfc0 0x55a1be9eea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:36.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.316+0000 7f84283ad740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a1be9de970 0x55a1be9eefa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:36.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.316+0000 7f84283ad740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1be9f1a70 0x55a1be9ef4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:36.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.316+0000 7f84283ad740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a1be9f4830 con 0x55a1be9ddfc0 2024-04-24T09:11:36.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.316+0000 7f84283ad740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a1be8fd660 con 0x55a1be9f1a70 2024-04-24T09:11:36.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.316+0000 7f84283ad740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a1be9f49b0 con 0x55a1be9de970 2024-04-24T09:11:36.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.316+0000 7f84283ad740 10 client.0 _ll_register_callbacks cb 0x55a1be8ebaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:11:36.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.316+0000 7f841e800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1be9ddfc0 0x55a1be9eea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:36.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.316+0000 7f841e800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1be9ddfc0 0x55a1be9eea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55582/0 (socket says 172.21.15.83:55582) 2024-04-24T09:11:36.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.316+0000 7f841e800640 1 -- 172.21.15.83:0/3648570677 learned_addr learned my addr 172.21.15.83:0/3648570677 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:11:36.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.316+0000 7f841de00640 1 --2- 172.21.15.83:0/3648570677 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a1be9de970 0x55a1be9eefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:36.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.317+0000 7f841e800640 1 -- 172.21.15.83:0/3648570677 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a1be9de970 msgr2=0x55a1be9eefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:36.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.317+0000 7f841e800640 1 --2- 172.21.15.83:0/3648570677 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a1be9de970 0x55a1be9eefa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:36.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.317+0000 7f841e800640 1 -- 172.21.15.83:0/3648570677 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1be9f1a70 msgr2=0x55a1be9ef4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:36.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.317+0000 7f841f200640 1 --2- 172.21.15.83:0/3648570677 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1be9f1a70 0x55a1be9ef4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:36.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.317+0000 7f841e800640 1 --2- 172.21.15.83:0/3648570677 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1be9f1a70 0x55a1be9ef4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:36.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.317+0000 7f841e800640 1 -- 172.21.15.83:0/3648570677 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1be77ef70 con 0x55a1be9ddfc0 2024-04-24T09:11:36.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.318+0000 7f841e800640 1 --2- 172.21.15.83:0/3648570677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1be9ddfc0 0x55a1be9eea60 secure :-1 s=READY pgs=2518 cs=0 l=1 rev1=1 crypto rx=0x7f84100039c0 tx=0x7f8410003e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d38665086a8ff76b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:36.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.318+0000 7f83ffe00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:11:36.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.318+0000 7f841f200640 1 --2- 172.21.15.83:0/3648570677 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1be9f1a70 0x55a1be9ef4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:11:36.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.318+0000 7f83ffe00640 1 -- 172.21.15.83:0/3648570677 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f841000be70 con 0x55a1be9ddfc0 2024-04-24T09:11:36.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.319+0000 7f83ffe00640 1 -- 172.21.15.83:0/3648570677 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 9583+0+0 (secure 0 0 0) 0x7f8410011650 con 0x55a1be9ddfc0 2024-04-24T09:11:36.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.319+0000 7f84283ad740 1 -- 172.21.15.83:0/3648570677 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8410015070 con 0x55a1be9ddfc0 2024-04-24T09:11:36.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.319+0000 7f83ffe00640 1 -- 172.21.15.83:0/3648570677 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f8410030050 con 0x55a1be9ddfc0 2024-04-24T09:11:36.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.319+0000 7f84283ad740 10 client.22536 fetch_fsmap learned FSMap version 61 2024-04-24T09:11:36.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.319+0000 7f84283ad740 1 -- 172.21.15.83:0/3648570677 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x55a1be616170 con 0x55a1be9ddfc0 2024-04-24T09:11:36.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.320+0000 7f83ffe00640 1 -- 172.21.15.83:0/3648570677 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8410015070 con 0x55a1be9ddfc0 2024-04-24T09:11:36.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.320+0000 7f83ffe00640 1 -- 172.21.15.83:0/3648570677 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 61) ==== 8362+0+0 (secure 0 0 0) 0x7f8410025930 con 0x55a1be9ddfc0 2024-04-24T09:11:36.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.320+0000 7f84283ad740 10 client.22536 fetch_fsmap finished waiting for FSMap version 61 2024-04-24T09:11:36.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.320+0000 7f84283ad740 10 client.22536 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:11:36.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.320+0000 7f84283ad740 20 client.22536 populate_metadata read hostname 'smithi083' 2024-04-24T09:11:36.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.320+0000 7f84283ad740 1 --2- 172.21.15.83:0/3648570677 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55a1bea01d90 0x55a1bea04180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:36.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.320+0000 7f84283ad740 4 client.22536 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:11:36.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.320+0000 7f84283ad740 1 -- 172.21.15.83:0/3648570677 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a1be951890 con 0x55a1bea01d90 2024-04-24T09:11:36.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.321+0000 7f841de00640 1 --2- 172.21.15.83:0/3648570677 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55a1bea01d90 0x55a1bea04180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:36.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.322+0000 7f841de00640 1 --2- 172.21.15.83:0/3648570677 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55a1bea01d90 0x55a1bea04180 crc :-1 s=READY pgs=499 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b911ec8dab47117 server_cookie=ffb7b18deaf8d57c in_seq=0 out_seq=0 2024-04-24T09:11:36.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.322+0000 7f83ffe00640 10 client.22536 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:11:36.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.323+0000 7f83ffe00640 1 -- 172.21.15.83:0/3648570677 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a1be951890 con 0x55a1bea01d90 2024-04-24T09:11:36.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.323+0000 7f83ffe00640 10 client.22536 handle_command_reply: tid=0 2024-04-24T09:11:36.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.323+0000 7f84283ad740 1 client.22536 shutdown 2024-04-24T09:11:36.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.323+0000 7f84283ad740 1 -- 172.21.15.83:0/3648570677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1be9ddfc0 msgr2=0x55a1be9eea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:36.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.323+0000 7f84283ad740 1 --2- 172.21.15.83:0/3648570677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1be9ddfc0 0x55a1be9eea60 secure :-1 s=READY pgs=2518 cs=0 l=1 rev1=1 crypto rx=0x7f84100039c0 tx=0x7f8410003e00 comp rx=0 tx=0).stop 2024-04-24T09:11:36.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.323+0000 7f84283ad740 1 -- 172.21.15.83:0/3648570677 shutdown_connections 2024-04-24T09:11:36.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.323+0000 7f84283ad740 1 --2- 172.21.15.83:0/3648570677 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55a1bea01d90 0x55a1bea04180 crc :-1 s=READY pgs=499 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:36.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.323+0000 7f84283ad740 1 --2- 172.21.15.83:0/3648570677 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1be9f1a70 0x55a1be9ef4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:36.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.323+0000 7f84283ad740 1 --2- 172.21.15.83:0/3648570677 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a1be9de970 0x55a1be9eefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:36.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.323+0000 7f84283ad740 1 --2- 172.21.15.83:0/3648570677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1be9ddfc0 0x55a1be9eea60 unknown :-1 s=CLOSED pgs=2518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:36.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.323+0000 7f84283ad740 1 -- 172.21.15.83:0/3648570677 >> 172.21.15.83:0/3648570677 conn(0x55a1be9da590 msgr2=0x55a1be9e0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:36.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.323+0000 7f83ffe00640 0 client.22536 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:11:36.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.324+0000 7f84283ad740 1 -- 172.21.15.83:0/3648570677 shutdown_connections 2024-04-24T09:11:36.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.324+0000 7f84283ad740 1 -- 172.21.15.83:0/3648570677 wait complete. 2024-04-24T09:11:36.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.324+0000 7f84283ad740 20 client.22536 trim_cache size 0 max 16384 2024-04-24T09:11:36.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.360+0000 7f84283ad740 1 Processor -- start 2024-04-24T09:11:36.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.360+0000 7f84283ad740 1 -- start start 2024-04-24T09:11:36.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.361+0000 7f84283ad740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1be9ddfc0 0x55a1be9fd730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:36.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.361+0000 7f84283ad740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1be9de970 0x55a1be9ee340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:36.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.361+0000 7f84283ad740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a1be9f1a70 0x55a1bea098f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:36.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.361+0000 7f84283ad740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f84200711e0 con 0x55a1be9de970 2024-04-24T09:11:36.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.361+0000 7f84283ad740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8420072d20 con 0x55a1be9ddfc0 2024-04-24T09:11:36.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.361+0000 7f84283ad740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f84200718b0 con 0x55a1be9f1a70 2024-04-24T09:11:36.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.361+0000 7f841de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1be9de970 0x55a1be9ee340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:36.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.361+0000 7f841e800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1be9ddfc0 0x55a1be9fd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:36.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.361+0000 7f841de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1be9de970 0x55a1be9ee340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55598/0 (socket says 172.21.15.83:55598) 2024-04-24T09:11:36.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.361+0000 7f841de00640 1 -- 172.21.15.83:0/4130715333 learned_addr learned my addr 172.21.15.83:0/4130715333 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:11:36.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.362+0000 7f841f200640 1 --2- 172.21.15.83:0/4130715333 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a1be9f1a70 0x55a1bea098f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:36.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.362+0000 7f841e800640 1 -- 172.21.15.83:0/4130715333 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a1be9f1a70 msgr2=0x55a1bea098f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:36.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.362+0000 7f841e800640 1 --2- 172.21.15.83:0/4130715333 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a1be9f1a70 0x55a1bea098f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:36.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.362+0000 7f841e800640 1 -- 172.21.15.83:0/4130715333 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1be9de970 msgr2=0x55a1be9ee340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:36.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.362+0000 7f841e800640 1 --2- 172.21.15.83:0/4130715333 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1be9de970 0x55a1be9ee340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:36.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.362+0000 7f841e800640 1 -- 172.21.15.83:0/4130715333 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1be9db780 con 0x55a1be9ddfc0 2024-04-24T09:11:36.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.363+0000 7f841e800640 1 --2- 172.21.15.83:0/4130715333 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1be9ddfc0 0x55a1be9fd730 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f8410025ec0 tx=0x7f8410004120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9ae2b79cc9acaeb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:36.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.363+0000 7f840aa00640 1 -- 172.21.15.83:0/4130715333 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8410030050 con 0x55a1be9ddfc0 2024-04-24T09:11:36.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.363+0000 7f840aa00640 1 -- 172.21.15.83:0/4130715333 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8410015070 con 0x55a1be9ddfc0 2024-04-24T09:11:36.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.363+0000 7f840aa00640 1 -- 172.21.15.83:0/4130715333 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8410034430 con 0x55a1be9ddfc0 2024-04-24T09:11:36.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.364+0000 7f84283ad740 1 -- 172.21.15.83:0/4130715333 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1be9ddfc0 msgr2=0x55a1be9fd730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:36.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.364+0000 7f84283ad740 1 --2- 172.21.15.83:0/4130715333 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1be9ddfc0 0x55a1be9fd730 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f8410025ec0 tx=0x7f8410004120 comp rx=0 tx=0).stop 2024-04-24T09:11:36.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.364+0000 7f84283ad740 1 -- 172.21.15.83:0/4130715333 shutdown_connections 2024-04-24T09:11:36.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.364+0000 7f84283ad740 1 --2- 172.21.15.83:0/4130715333 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a1be9f1a70 0x55a1bea098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:36.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.364+0000 7f84283ad740 1 --2- 172.21.15.83:0/4130715333 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1be9de970 0x55a1be9ee340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:36.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.364+0000 7f84283ad740 1 --2- 172.21.15.83:0/4130715333 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1be9ddfc0 0x55a1be9fd730 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:36.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.364+0000 7f84283ad740 1 -- 172.21.15.83:0/4130715333 >> 172.21.15.83:0/4130715333 conn(0x55a1be9da590 msgr2=0x55a1be9d96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:36.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.365+0000 7f84283ad740 1 -- 172.21.15.83:0/4130715333 shutdown_connections 2024-04-24T09:11:36.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.365+0000 7f84283ad740 1 -- 172.21.15.83:0/4130715333 wait complete. 2024-04-24T09:11:36.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.365+0000 7f84283ad740 1 -- start start 2024-04-24T09:11:36.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.366+0000 7f84283ad740 1 Processor -- start 2024-04-24T09:11:36.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.366+0000 7f84283ad740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1be9ddfc0 0x55a1be9f9fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:36.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.366+0000 7f84283ad740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1be9de970 0x55a1be9f52e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:36.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.366+0000 7f84283ad740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a1be9f1a70 0x55a1be9f5820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:36.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.367+0000 7f84283ad740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8420072650 con 0x55a1be9de970 2024-04-24T09:11:36.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.367+0000 7f84283ad740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8420071f80 con 0x55a1be9ddfc0 2024-04-24T09:11:36.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.367+0000 7f84283ad740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f842006f1c0 con 0x55a1be9f1a70 2024-04-24T09:11:36.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.367+0000 7f84283ad740 10 client.0 _ll_register_callbacks cb 0x7f8410004590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:11:36.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.367+0000 7f841e800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1be9ddfc0 0x55a1be9f9fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:36.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.367+0000 7f841e800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1be9ddfc0 0x55a1be9f9fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:57974/0 (socket says 172.21.15.83:57974) 2024-04-24T09:11:36.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.367+0000 7f841e800640 1 -- 172.21.15.83:0/1762253003 learned_addr learned my addr 172.21.15.83:0/1762253003 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:11:36.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.367+0000 7f841de00640 1 --2- 172.21.15.83:0/1762253003 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1be9de970 0x55a1be9f52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:36.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.367+0000 7f841e800640 1 -- 172.21.15.83:0/1762253003 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a1be9f1a70 msgr2=0x55a1be9f5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:36.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.367+0000 7f841e800640 1 --2- 172.21.15.83:0/1762253003 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a1be9f1a70 0x55a1be9f5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:36.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.367+0000 7f841e800640 1 -- 172.21.15.83:0/1762253003 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1be9de970 msgr2=0x55a1be9f52e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:36.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.368+0000 7f841e800640 1 --2- 172.21.15.83:0/1762253003 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1be9de970 0x55a1be9f52e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:36.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.368+0000 7f841e800640 1 -- 172.21.15.83:0/1762253003 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1be9f3e60 con 0x55a1be9ddfc0 2024-04-24T09:11:36.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.368+0000 7f841de00640 1 --2- 172.21.15.83:0/1762253003 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1be9de970 0x55a1be9f52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:11:36.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.368+0000 7f841e800640 1 --2- 172.21.15.83:0/1762253003 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1be9ddfc0 0x55a1be9f9fc0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f841000c010 tx=0x7f84100058f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9419da518a333384 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:36.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.368+0000 7f83ffe00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:11:36.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.368+0000 7f83ffe00640 1 -- 172.21.15.83:0/1762253003 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8410041060 con 0x55a1be9ddfc0 2024-04-24T09:11:36.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.369+0000 7f83ffe00640 1 -- 172.21.15.83:0/1762253003 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 9583+0+0 (secure 0 0 0) 0x7f8410025430 con 0x55a1be9ddfc0 2024-04-24T09:11:36.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.369+0000 7f84283ad740 1 -- 172.21.15.83:0/1762253003 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8410015070 con 0x55a1be9ddfc0 2024-04-24T09:11:36.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.369+0000 7f83ffe00640 1 -- 172.21.15.83:0/1762253003 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f8410048050 con 0x55a1be9ddfc0 2024-04-24T09:11:36.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.369+0000 7f84283ad740 10 client.39593 fetch_fsmap learned FSMap version 61 2024-04-24T09:11:36.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.369+0000 7f84283ad740 1 -- 172.21.15.83:0/1762253003 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x55a1bea67f70 con 0x55a1be9ddfc0 2024-04-24T09:11:36.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.370+0000 7f83ffe00640 1 -- 172.21.15.83:0/1762253003 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8410015070 con 0x55a1be9ddfc0 2024-04-24T09:11:36.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.370+0000 7f83ffe00640 1 -- 172.21.15.83:0/1762253003 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 61) ==== 8362+0+0 (secure 0 0 0) 0x7f841003c440 con 0x55a1be9ddfc0 2024-04-24T09:11:36.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.370+0000 7f84283ad740 10 client.39593 fetch_fsmap finished waiting for FSMap version 61 2024-04-24T09:11:36.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.370+0000 7f84283ad740 10 client.39593 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:11:36.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.370+0000 7f84283ad740 20 client.39593 populate_metadata read hostname 'smithi083' 2024-04-24T09:11:36.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.370+0000 7f84283ad740 1 --2- 172.21.15.83:0/1762253003 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55a1bea68570 0x55a1bea6a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:36.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.370+0000 7f84283ad740 4 client.39593 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T09:11:36.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.370+0000 7f84283ad740 1 -- 172.21.15.83:0/1762253003 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a1be94b700 con 0x55a1bea68570 2024-04-24T09:11:36.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.371+0000 7f841de00640 1 --2- 172.21.15.83:0/1762253003 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55a1bea68570 0x55a1bea6a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:36.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.372+0000 7f841de00640 1 --2- 172.21.15.83:0/1762253003 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55a1bea68570 0x55a1bea6a990 crc :-1 s=READY pgs=500 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ef86b20d2a2a506 server_cookie=afadd722bd5998dd in_seq=0 out_seq=0 2024-04-24T09:11:36.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.372+0000 7f83ffe00640 10 client.39593 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:11:36.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.374+0000 7f83ffe00640 1 -- 172.21.15.83:0/1762253003 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a1be94b700 con 0x55a1bea68570 2024-04-24T09:11:36.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.374+0000 7f83ffe00640 10 client.39593 handle_command_reply: tid=0 2024-04-24T09:11:36.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.374+0000 7f84283ad740 1 client.39593 shutdown 2024-04-24T09:11:36.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.374+0000 7f84283ad740 1 -- 172.21.15.83:0/1762253003 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1be9ddfc0 msgr2=0x55a1be9f9fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:36.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.374+0000 7f84283ad740 1 --2- 172.21.15.83:0/1762253003 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1be9ddfc0 0x55a1be9f9fc0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f841000c010 tx=0x7f84100058f0 comp rx=0 tx=0).stop 2024-04-24T09:11:36.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.374+0000 7f84283ad740 1 -- 172.21.15.83:0/1762253003 shutdown_connections 2024-04-24T09:11:36.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.374+0000 7f84283ad740 1 --2- 172.21.15.83:0/1762253003 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55a1bea68570 0x55a1bea6a990 crc :-1 s=READY pgs=500 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:36.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.374+0000 7f84283ad740 1 --2- 172.21.15.83:0/1762253003 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a1be9f1a70 0x55a1be9f5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:36.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.374+0000 7f84283ad740 1 --2- 172.21.15.83:0/1762253003 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1be9de970 0x55a1be9f52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:36.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.374+0000 7f83ffe00640 0 client.39593 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:11:36.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.374+0000 7f84283ad740 1 --2- 172.21.15.83:0/1762253003 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1be9ddfc0 0x55a1be9f9fc0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:36.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.374+0000 7f84283ad740 1 -- 172.21.15.83:0/1762253003 >> 172.21.15.83:0/1762253003 conn(0x55a1be9da590 msgr2=0x55a1be9dfb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:36.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.374+0000 7f84283ad740 1 -- 172.21.15.83:0/1762253003 shutdown_connections 2024-04-24T09:11:36.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.375+0000 7f84283ad740 1 -- 172.21.15.83:0/1762253003 wait complete. 2024-04-24T09:11:36.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.375+0000 7f84283ad740 20 client.39593 trim_cache size 0 max 16384 2024-04-24T09:11:36.375 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T09:11:36.375 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-04-24T09:11:36.375 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "80751bc0-4da9-406c-97c1-3cf18e45c6df", 2024-04-24T09:11:36.375 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-04-24T09:11:36.375 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T09:11:36.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.376+0000 7f8425400640 1 -- 172.21.15.83:0/2416069072 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f84040a3da0 msgr2=0x7f84040a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:36.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.376+0000 7f8425400640 1 --2- 172.21.15.83:0/2416069072 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f84040a3da0 0x7f84040a6260 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f8418005fd0 tx=0x7f8418009630 comp rx=0 tx=0).stop 2024-04-24T09:11:36.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.376+0000 7f8425400640 1 -- 172.21.15.83:0/2416069072 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8420109420 msgr2=0x7f84201a0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:36.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.376+0000 7f8425400640 1 --2- 172.21.15.83:0/2416069072 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8420109420 0x7f84201a0be0 secure :-1 s=READY pgs=2516 cs=0 l=1 rev1=1 crypto rx=0x7f840c004270 tx=0x7f840c0042a0 comp rx=0 tx=0).stop 2024-04-24T09:11:36.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.376+0000 7f8425400640 1 -- 172.21.15.83:0/2416069072 shutdown_connections 2024-04-24T09:11:36.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.376+0000 7f8425400640 1 --2- 172.21.15.83:0/2416069072 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f84040a3da0 0x7f84040a6260 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:36.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.376+0000 7f8425400640 1 --2- 172.21.15.83:0/2416069072 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f842019a860 0x7f842019ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:36.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.376+0000 7f8425400640 1 --2- 172.21.15.83:0/2416069072 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8420109420 0x7f84201a0be0 unknown :-1 s=CLOSED pgs=2516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:36.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.376+0000 7f8425400640 1 --2- 172.21.15.83:0/2416069072 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8420104350 0x7f84201a06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:36.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.376+0000 7f8425400640 1 -- 172.21.15.83:0/2416069072 >> 172.21.15.83:0/2416069072 conn(0x7f84200ffb70 msgr2=0x7f8420101000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:36.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.376+0000 7f8425400640 1 -- 172.21.15.83:0/2416069072 shutdown_connections 2024-04-24T09:11:36.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.376+0000 7f8425400640 1 -- 172.21.15.83:0/2416069072 wait complete. 2024-04-24T09:11:36.394 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:11:36 smithi083 ceph-mon[27907]: pgmap v630: 129 pgs: 129 active+clean; 763 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.7 MiB/s wr, 173 op/s 2024-04-24T09:11:36.397 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T09:11:36.397 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T09:11:36.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.662+0000 7f8b51200640 1 -- 172.21.15.83:0/3874325087 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b4c109580 msgr2=0x7f8b4c111150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:36.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.662+0000 7f8b51200640 1 --2- 172.21.15.83:0/3874325087 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b4c109580 0x7f8b4c111150 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f8b44009be0 tx=0x7f8b4402a6c0 comp rx=0 tx=0).stop 2024-04-24T09:11:36.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.662+0000 7f8b51200640 1 -- 172.21.15.83:0/3874325087 shutdown_connections 2024-04-24T09:11:36.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.662+0000 7f8b51200640 1 --2- 172.21.15.83:0/3874325087 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b4c109580 0x7f8b4c111150 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:36.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.662+0000 7f8b51200640 1 --2- 172.21.15.83:0/3874325087 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b4c104e60 0x7f8b4c108ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:36.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.662+0000 7f8b51200640 1 --2- 172.21.15.83:0/3874325087 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8b4c1044b0 0x7f8b4c104890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:36.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.662+0000 7f8b51200640 1 -- 172.21.15.83:0/3874325087 >> 172.21.15.83:0/3874325087 conn(0x7f8b4c0ffd90 msgr2=0x7f8b4c1021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:36.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.662+0000 7f8b51200640 1 -- 172.21.15.83:0/3874325087 shutdown_connections 2024-04-24T09:11:36.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.663+0000 7f8b51200640 1 -- 172.21.15.83:0/3874325087 wait complete. 2024-04-24T09:11:36.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.664+0000 7f8b51200640 1 Processor -- start 2024-04-24T09:11:36.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.664+0000 7f8b51200640 1 -- start start 2024-04-24T09:11:36.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.665+0000 7f8b51200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8b4c1044b0 0x7f8b4c111130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:36.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.665+0000 7f8b51200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b4c109580 0x7f8b4c10e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:36.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.665+0000 7f8b51200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b4c10e910 0x7f8b4c1ab210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:36.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.665+0000 7f8b51200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8b4c1144d0 con 0x7f8b4c10e910 2024-04-24T09:11:36.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.665+0000 7f8b51200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8b4c114350 con 0x7f8b4c109580 2024-04-24T09:11:36.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.665+0000 7f8b51200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8b4c114650 con 0x7f8b4c1044b0 2024-04-24T09:11:36.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.665+0000 7f8b4a800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b4c10e910 0x7f8b4c1ab210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:36.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.665+0000 7f8b49e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8b4c1044b0 0x7f8b4c111130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:36.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.665+0000 7f8b43e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b4c109580 0x7f8b4c10e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:36.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.665+0000 7f8b49e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8b4c1044b0 0x7f8b4c111130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:38660/0 (socket says 172.21.15.83:38660) 2024-04-24T09:11:36.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.665+0000 7f8b49e00640 1 -- 172.21.15.83:0/728590953 learned_addr learned my addr 172.21.15.83:0/728590953 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:11:36.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.665+0000 7f8b4a800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b4c10e910 0x7f8b4c1ab210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55632/0 (socket says 172.21.15.83:55632) 2024-04-24T09:11:36.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.666+0000 7f8b4a800640 1 -- 172.21.15.83:0/728590953 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8b4c1044b0 msgr2=0x7f8b4c111130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:36.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.666+0000 7f8b4a800640 1 --2- 172.21.15.83:0/728590953 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8b4c1044b0 0x7f8b4c111130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:36.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.666+0000 7f8b4a800640 1 -- 172.21.15.83:0/728590953 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b4c109580 msgr2=0x7f8b4c10e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:36.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.666+0000 7f8b4a800640 1 --2- 172.21.15.83:0/728590953 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b4c109580 0x7f8b4c10e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:36.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.666+0000 7f8b4a800640 1 -- 172.21.15.83:0/728590953 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b4c1ab750 con 0x7f8b4c10e910 2024-04-24T09:11:36.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.666+0000 7f8b4a800640 1 --2- 172.21.15.83:0/728590953 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b4c10e910 0x7f8b4c1ab210 secure :-1 s=READY pgs=2519 cs=0 l=1 rev1=1 crypto rx=0x7f8b44009b30 tx=0x7f8b44003e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:36.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.667+0000 7f8b42a00640 1 -- 172.21.15.83:0/728590953 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b44041020 con 0x7f8b4c10e910 2024-04-24T09:11:36.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.667+0000 7f8b42a00640 1 -- 172.21.15.83:0/728590953 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8b44037040 con 0x7f8b4c10e910 2024-04-24T09:11:36.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.667+0000 7f8b42a00640 1 -- 172.21.15.83:0/728590953 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b44036070 con 0x7f8b4c10e910 2024-04-24T09:11:36.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.668+0000 7f8b51200640 1 -- 172.21.15.83:0/728590953 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b4c1ab920 con 0x7f8b4c10e910 2024-04-24T09:11:36.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.668+0000 7f8b51200640 1 -- 172.21.15.83:0/728590953 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b4c1abd30 con 0x7f8b4c10e910 2024-04-24T09:11:36.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.674+0000 7f8b42a00640 1 -- 172.21.15.83:0/728590953 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8b44044020 con 0x7f8b4c10e910 2024-04-24T09:11:36.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.674+0000 7f8b5404e740 1 Processor -- start 2024-04-24T09:11:36.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.675+0000 7f8b5404e740 1 -- start start 2024-04-24T09:11:36.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.675+0000 7f8b5404e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560527f5b860 0x560527f5bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:36.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.675+0000 7f8b5404e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560527f5c180 0x560527f66310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:36.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.675+0000 7f8b5404e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560527f66850 0x560527f68c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:36.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.675+0000 7f8b5404e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x560527ec11f0 con 0x560527f5b860 2024-04-24T09:11:36.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.675+0000 7f8b5404e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560527e67350 con 0x560527f5c180 2024-04-24T09:11:36.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.675+0000 7f8b5404e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x560527ebd1b0 con 0x560527f66850 2024-04-24T09:11:36.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.676+0000 7f8b43e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560527f5b860 0x560527f5bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:36.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.676+0000 7f8b43e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560527f5b860 0x560527f5bc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55634/0 (socket says 172.21.15.83:55634) 2024-04-24T09:11:36.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.676+0000 7f8b43e00640 1 -- 172.21.15.83:0/3180229903 learned_addr learned my addr 172.21.15.83:0/3180229903 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:11:36.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.676+0000 7f8b42a00640 1 --2- 172.21.15.83:0/728590953 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f8b300a3da0 0x7f8b300a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:36.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.676+0000 7f8b43e00640 1 --2- 172.21.15.83:0/3180229903 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560527f66850 0x560527f68c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:36.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.676+0000 7f8b42a00640 1 -- 172.21.15.83:0/728590953 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 9583+0+0 (secure 0 0 0) 0x7f8b440f1320 con 0x7f8b4c10e910 2024-04-24T09:11:36.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.676+0000 7f8b49e00640 1 --2- 172.21.15.83:0/3180229903 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560527f5c180 0x560527f66310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:36.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.677+0000 7f8b4a800640 1 --2- 172.21.15.83:0/728590953 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f8b300a3da0 0x7f8b300a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:36.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.677+0000 7f8b49e00640 1 -- 172.21.15.83:0/3180229903 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560527f66850 msgr2=0x560527f68c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:36.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.677+0000 7f8b49e00640 1 --2- 172.21.15.83:0/3180229903 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560527f66850 0x560527f68c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:36.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.677+0000 7f8b49e00640 1 -- 172.21.15.83:0/3180229903 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560527f5b860 msgr2=0x560527f5bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:36.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.677+0000 7f8b49e00640 1 --2- 172.21.15.83:0/3180229903 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560527f5b860 0x560527f5bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:36.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.677+0000 7f8b49e00640 1 -- 172.21.15.83:0/3180229903 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560527ebbf50 con 0x560527f5c180 2024-04-24T09:11:36.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.677+0000 7f8b43e00640 1 --2- 172.21.15.83:0/3180229903 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560527f5b860 0x560527f5bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:11:36.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.678+0000 7f8b4a800640 1 --2- 172.21.15.83:0/728590953 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f8b300a3da0 0x7f8b300a6260 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f8b440bcba0 tx=0x7f8b44005e50 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:36.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.678+0000 7f8b49e00640 1 --2- 172.21.15.83:0/3180229903 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560527f5c180 0x560527f66310 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c009c80 tx=0x7f8b3c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3bc289cf52c1c514 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:36.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.678+0000 7f8b3b400640 1 -- 172.21.15.83:0/3180229903 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b3c004230 con 0x560527f5c180 2024-04-24T09:11:36.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.678+0000 7f8b3b400640 1 -- 172.21.15.83:0/3180229903 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8b3c0043d0 con 0x560527f5c180 2024-04-24T09:11:36.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.678+0000 7f8b3b400640 1 -- 172.21.15.83:0/3180229903 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b3c019710 con 0x560527f5c180 2024-04-24T09:11:36.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.679+0000 7f8b5404e740 1 -- 172.21.15.83:0/3180229903 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560527f5c180 msgr2=0x560527f66310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:36.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.679+0000 7f8b5404e740 1 --2- 172.21.15.83:0/3180229903 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560527f5c180 0x560527f66310 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c009c80 tx=0x7f8b3c011810 comp rx=0 tx=0).stop 2024-04-24T09:11:36.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.679+0000 7f8b5404e740 1 -- 172.21.15.83:0/3180229903 shutdown_connections 2024-04-24T09:11:36.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.679+0000 7f8b5404e740 1 --2- 172.21.15.83:0/3180229903 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560527f66850 0x560527f68c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:36.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.679+0000 7f8b5404e740 1 --2- 172.21.15.83:0/3180229903 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560527f5c180 0x560527f66310 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:36.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.679+0000 7f8b5404e740 1 --2- 172.21.15.83:0/3180229903 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560527f5b860 0x560527f5bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:36.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.679+0000 7f8b5404e740 1 -- 172.21.15.83:0/3180229903 >> 172.21.15.83:0/3180229903 conn(0x560527f57b80 msgr2=0x560527f57f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:36.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.680+0000 7f8b5404e740 1 -- 172.21.15.83:0/3180229903 shutdown_connections 2024-04-24T09:11:36.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.680+0000 7f8b5404e740 1 -- 172.21.15.83:0/3180229903 wait complete. 2024-04-24T09:11:36.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.680+0000 7f8b5404e740 1 -- start start 2024-04-24T09:11:36.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.681+0000 7f8b5404e740 1 Processor -- start 2024-04-24T09:11:36.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.681+0000 7f8b5404e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560527f5b860 0x560527f63740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:36.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.681+0000 7f8b5404e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560527f5c180 0x560527f63c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:36.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.681+0000 7f8b5404e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560527f66850 0x560527f641c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:36.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.681+0000 7f8b5404e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x560527f69640 con 0x560527f5b860 2024-04-24T09:11:36.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.681+0000 7f8b5404e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560527e7a690 con 0x560527f66850 2024-04-24T09:11:36.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.681+0000 7f8b5404e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x560527f697c0 con 0x560527f5c180 2024-04-24T09:11:36.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.681+0000 7f8b5404e740 10 client.0 _ll_register_callbacks cb 0x560527eb6650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:11:36.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.682+0000 7f8b43e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560527f5b860 0x560527f63740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:36.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.682+0000 7f8b43e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560527f5b860 0x560527f63740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55642/0 (socket says 172.21.15.83:55642) 2024-04-24T09:11:36.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.682+0000 7f8b43e00640 1 -- 172.21.15.83:0/2098010643 learned_addr learned my addr 172.21.15.83:0/2098010643 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:11:36.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.682+0000 7f8b49e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560527f5c180 0x560527f63c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:36.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.683+0000 7f8b43e00640 1 -- 172.21.15.83:0/2098010643 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560527f5c180 msgr2=0x560527f63c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:36.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.683+0000 7f8b43e00640 1 --2- 172.21.15.83:0/2098010643 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560527f5c180 0x560527f63c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:36.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.683+0000 7f8b43e00640 1 -- 172.21.15.83:0/2098010643 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560527f66850 msgr2=0x560527f641c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T09:11:36.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.683+0000 7f8b43e00640 1 --2- 172.21.15.83:0/2098010643 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560527f66850 0x560527f641c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:36.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.683+0000 7f8b43e00640 1 -- 172.21.15.83:0/2098010643 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560527e79920 con 0x560527f5b860 2024-04-24T09:11:36.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.683+0000 7f8b43e00640 1 --2- 172.21.15.83:0/2098010643 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560527f5b860 0x560527f63740 secure :-1 s=READY pgs=2520 cs=0 l=1 rev1=1 crypto rx=0x7f8b34013f60 tx=0x7f8b34013f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab4f739b2d2278b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:36.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.683+0000 7f8b2fe00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:11:36.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.683+0000 7f8b2fe00640 1 -- 172.21.15.83:0/2098010643 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b3400fe90 con 0x560527f5b860 2024-04-24T09:11:36.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.684+0000 7f8b2fe00640 1 -- 172.21.15.83:0/2098010643 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 9583+0+0 (secure 0 0 0) 0x7f8b34013160 con 0x560527f5b860 2024-04-24T09:11:36.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.684+0000 7f8b5404e740 1 -- 172.21.15.83:0/2098010643 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b3c0043d0 con 0x560527f5b860 2024-04-24T09:11:36.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.684+0000 7f8b2fe00640 1 -- 172.21.15.83:0/2098010643 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f8b34004020 con 0x560527f5b860 2024-04-24T09:11:36.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.684+0000 7f8b5404e740 10 client.22554 fetch_fsmap learned FSMap version 61 2024-04-24T09:11:36.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.684+0000 7f8b5404e740 1 -- 172.21.15.83:0/2098010643 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x560527c9d7d0 con 0x560527f5b860 2024-04-24T09:11:36.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.685+0000 7f8b2fe00640 1 -- 172.21.15.83:0/2098010643 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8b3c0043d0 con 0x560527f5b860 2024-04-24T09:11:36.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.685+0000 7f8b2fe00640 1 -- 172.21.15.83:0/2098010643 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 61) ==== 8362+0+0 (secure 0 0 0) 0x7f8b34010430 con 0x560527f5b860 2024-04-24T09:11:36.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.685+0000 7f8b5404e740 10 client.22554 fetch_fsmap finished waiting for FSMap version 61 2024-04-24T09:11:36.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.685+0000 7f8b5404e740 10 client.22554 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:11:36.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.685+0000 7f8b5404e740 20 client.22554 populate_metadata read hostname 'smithi083' 2024-04-24T09:11:36.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.685+0000 7f8b5404e740 1 --2- 172.21.15.83:0/2098010643 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x560527f76b90 0x560527f78fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:36.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.685+0000 7f8b5404e740 4 client.22554 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:11:36.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.685+0000 7f8b5404e740 1 -- 172.21.15.83:0/2098010643 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560527ecec10 con 0x560527f76b90 2024-04-24T09:11:36.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.686+0000 7f8b49e00640 1 --2- 172.21.15.83:0/2098010643 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x560527f76b90 0x560527f78fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:36.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.687+0000 7f8b49e00640 1 --2- 172.21.15.83:0/2098010643 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x560527f76b90 0x560527f78fc0 crc :-1 s=READY pgs=501 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f4ff94b99b06627 server_cookie=b82d36bfe018dd9c in_seq=0 out_seq=0 2024-04-24T09:11:36.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.687+0000 7f8b2fe00640 10 client.22554 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:11:36.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.688+0000 7f8b2fe00640 1 -- 172.21.15.83:0/2098010643 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560527ecec10 con 0x560527f76b90 2024-04-24T09:11:36.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.688+0000 7f8b2fe00640 10 client.22554 handle_command_reply: tid=0 2024-04-24T09:11:36.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.688+0000 7f8b5404e740 1 client.22554 shutdown 2024-04-24T09:11:36.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.688+0000 7f8b5404e740 1 -- 172.21.15.83:0/2098010643 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560527f5b860 msgr2=0x560527f63740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:36.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.688+0000 7f8b5404e740 1 --2- 172.21.15.83:0/2098010643 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560527f5b860 0x560527f63740 secure :-1 s=READY pgs=2520 cs=0 l=1 rev1=1 crypto rx=0x7f8b34013f60 tx=0x7f8b34013f90 comp rx=0 tx=0).stop 2024-04-24T09:11:36.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.688+0000 7f8b5404e740 1 -- 172.21.15.83:0/2098010643 shutdown_connections 2024-04-24T09:11:36.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.688+0000 7f8b5404e740 1 --2- 172.21.15.83:0/2098010643 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x560527f76b90 0x560527f78fc0 crc :-1 s=READY pgs=501 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:36.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.688+0000 7f8b5404e740 1 --2- 172.21.15.83:0/2098010643 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560527f66850 0x560527f641c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:36.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.688+0000 7f8b5404e740 1 --2- 172.21.15.83:0/2098010643 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560527f5c180 0x560527f63c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:36.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.688+0000 7f8b2fe00640 0 client.22554 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:11:36.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.688+0000 7f8b5404e740 1 --2- 172.21.15.83:0/2098010643 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560527f5b860 0x560527f63740 unknown :-1 s=CLOSED pgs=2520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:36.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.688+0000 7f8b5404e740 1 -- 172.21.15.83:0/2098010643 >> 172.21.15.83:0/2098010643 conn(0x560527f57b80 msgr2=0x560527f5d880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:36.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.688+0000 7f8b5404e740 1 -- 172.21.15.83:0/2098010643 shutdown_connections 2024-04-24T09:11:36.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.689+0000 7f8b5404e740 1 -- 172.21.15.83:0/2098010643 wait complete. 2024-04-24T09:11:36.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.689+0000 7f8b5404e740 20 client.22554 trim_cache size 0 max 16384 2024-04-24T09:11:36.704 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:11:36 smithi138 ceph-mon[31441]: pgmap v630: 129 pgs: 129 active+clean; 763 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.7 MiB/s wr, 173 op/s 2024-04-24T09:11:36.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:11:36 smithi189 ceph-mon[31791]: pgmap v630: 129 pgs: 129 active+clean; 763 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.7 MiB/s wr, 173 op/s 2024-04-24T09:11:36.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.721+0000 7f8b5404e740 1 Processor -- start 2024-04-24T09:11:36.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.721+0000 7f8b5404e740 1 -- start start 2024-04-24T09:11:36.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.722+0000 7f8b5404e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560527f5b860 0x560527f72560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:36.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.722+0000 7f8b5404e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560527f5c180 0x560527f63e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:36.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.722+0000 7f8b5404e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560527f66850 0x560527f7e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:36.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.722+0000 7f8b5404e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8b4c1b5be0 con 0x560527f5b860 2024-04-24T09:11:36.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.722+0000 7f8b5404e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8b4c1b7720 con 0x560527f66850 2024-04-24T09:11:36.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.722+0000 7f8b5404e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8b4c1b62b0 con 0x560527f5c180 2024-04-24T09:11:36.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.722+0000 7f8b43e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560527f5b860 0x560527f72560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:36.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.722+0000 7f8b43e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560527f5b860 0x560527f72560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55648/0 (socket says 172.21.15.83:55648) 2024-04-24T09:11:36.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.722+0000 7f8b43e00640 1 -- 172.21.15.83:0/3400296792 learned_addr learned my addr 172.21.15.83:0/3400296792 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:11:36.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.722+0000 7f8b49e00640 1 --2- 172.21.15.83:0/3400296792 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560527f5c180 0x560527f63e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:36.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.722+0000 7f8b43e00640 1 --2- 172.21.15.83:0/3400296792 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560527f66850 0x560527f7e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:36.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.723+0000 7f8b43e00640 1 -- 172.21.15.83:0/3400296792 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560527f5c180 msgr2=0x560527f63e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:36.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.723+0000 7f8b43e00640 1 --2- 172.21.15.83:0/3400296792 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560527f5c180 0x560527f63e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:36.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.723+0000 7f8b43e00640 1 -- 172.21.15.83:0/3400296792 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560527f66850 msgr2=0x560527f7e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:36.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.723+0000 7f8b43e00640 1 --2- 172.21.15.83:0/3400296792 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560527f66850 0x560527f7e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:36.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.723+0000 7f8b43e00640 1 -- 172.21.15.83:0/3400296792 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560527bd33f0 con 0x560527f5b860 2024-04-24T09:11:36.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.723+0000 7f8b49e00640 1 --2- 172.21.15.83:0/3400296792 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560527f5c180 0x560527f63e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:11:36.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.723+0000 7f8b43e00640 1 --2- 172.21.15.83:0/3400296792 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560527f5b860 0x560527f72560 secure :-1 s=READY pgs=2521 cs=0 l=1 rev1=1 crypto rx=0x7f8b340059f0 tx=0x7f8b34000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=842fb688a034f017 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:36.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.724+0000 7f8b3aa00640 1 -- 172.21.15.83:0/3400296792 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b34010430 con 0x560527f5b860 2024-04-24T09:11:36.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.724+0000 7f8b3aa00640 1 -- 172.21.15.83:0/3400296792 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8b3401e030 con 0x560527f5b860 2024-04-24T09:11:36.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.724+0000 7f8b3aa00640 1 -- 172.21.15.83:0/3400296792 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b34003aa0 con 0x560527f5b860 2024-04-24T09:11:36.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.725+0000 7f8b5404e740 1 -- 172.21.15.83:0/3400296792 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560527f5b860 msgr2=0x560527f72560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:36.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.725+0000 7f8b5404e740 1 --2- 172.21.15.83:0/3400296792 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560527f5b860 0x560527f72560 secure :-1 s=READY pgs=2521 cs=0 l=1 rev1=1 crypto rx=0x7f8b340059f0 tx=0x7f8b34000bc0 comp rx=0 tx=0).stop 2024-04-24T09:11:36.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.725+0000 7f8b5404e740 1 -- 172.21.15.83:0/3400296792 shutdown_connections 2024-04-24T09:11:36.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.725+0000 7f8b5404e740 1 --2- 172.21.15.83:0/3400296792 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560527f66850 0x560527f7e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:36.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.725+0000 7f8b5404e740 1 --2- 172.21.15.83:0/3400296792 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560527f5c180 0x560527f63e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:36.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.725+0000 7f8b5404e740 1 --2- 172.21.15.83:0/3400296792 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560527f5b860 0x560527f72560 unknown :-1 s=CLOSED pgs=2521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:36.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.725+0000 7f8b5404e740 1 -- 172.21.15.83:0/3400296792 >> 172.21.15.83:0/3400296792 conn(0x560527f57b80 msgr2=0x560527f63700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:36.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.725+0000 7f8b5404e740 1 -- 172.21.15.83:0/3400296792 shutdown_connections 2024-04-24T09:11:36.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.725+0000 7f8b5404e740 1 -- 172.21.15.83:0/3400296792 wait complete. 2024-04-24T09:11:36.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.726+0000 7f8b5404e740 1 -- start start 2024-04-24T09:11:36.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.726+0000 7f8b5404e740 1 Processor -- start 2024-04-24T09:11:36.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.727+0000 7f8b5404e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560527f5b860 0x560527f6ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:36.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.727+0000 7f8b5404e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560527f5c180 0x560527f6a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:36.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.727+0000 7f8b5404e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560527f66850 0x560527f6a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:36.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.727+0000 7f8b5404e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8b4c1b7050 con 0x560527f5b860 2024-04-24T09:11:36.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.727+0000 7f8b5404e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8b4c1b6980 con 0x560527f66850 2024-04-24T09:11:36.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.727+0000 7f8b5404e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8b4c1b3be0 con 0x560527f5c180 2024-04-24T09:11:36.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.727+0000 7f8b5404e740 10 client.0 _ll_register_callbacks cb 0x7f8b340134f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:11:36.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.727+0000 7f8b43e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560527f5b860 0x560527f6ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:36.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.727+0000 7f8b43e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560527f5b860 0x560527f6ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55656/0 (socket says 172.21.15.83:55656) 2024-04-24T09:11:36.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.727+0000 7f8b43e00640 1 -- 172.21.15.83:0/1645486459 learned_addr learned my addr 172.21.15.83:0/1645486459 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:11:36.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.727+0000 7f8b49e00640 1 --2- 172.21.15.83:0/1645486459 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560527f5c180 0x560527f6a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:36.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.728+0000 7f8b43e00640 1 -- 172.21.15.83:0/1645486459 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560527f5c180 msgr2=0x560527f6a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:36.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.728+0000 7f8b43e00640 1 --2- 172.21.15.83:0/1645486459 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560527f5c180 0x560527f6a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:36.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.728+0000 7f8b43e00640 1 -- 172.21.15.83:0/1645486459 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560527f66850 msgr2=0x560527f6a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:36.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.728+0000 7f8b43e00640 1 --2- 172.21.15.83:0/1645486459 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560527f66850 0x560527f6a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:36.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.728+0000 7f8b43e00640 1 -- 172.21.15.83:0/1645486459 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560527b930f0 con 0x560527f5b860 2024-04-24T09:11:36.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.728+0000 7f8b43e00640 1 --2- 172.21.15.83:0/1645486459 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560527f5b860 0x560527f6ee30 secure :-1 s=READY pgs=2522 cs=0 l=1 rev1=1 crypto rx=0x7f8b34010970 tx=0x7f8b34005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9af359765121ea3d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:36.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.728+0000 7f8b2fe00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:11:36.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.729+0000 7f8b2fe00640 1 -- 172.21.15.83:0/1645486459 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b34010ac0 con 0x560527f5b860 2024-04-24T09:11:36.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.729+0000 7f8b2fe00640 1 -- 172.21.15.83:0/1645486459 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 9583+0+0 (secure 0 0 0) 0x7f8b34004230 con 0x560527f5b860 2024-04-24T09:11:36.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.729+0000 7f8b5404e740 1 -- 172.21.15.83:0/1645486459 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b3401e030 con 0x560527f5b860 2024-04-24T09:11:36.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.729+0000 7f8b2fe00640 1 -- 172.21.15.83:0/1645486459 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f8b34004430 con 0x560527f5b860 2024-04-24T09:11:36.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.729+0000 7f8b5404e740 10 client.22560 fetch_fsmap learned FSMap version 61 2024-04-24T09:11:36.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.730+0000 7f8b5404e740 1 -- 172.21.15.83:0/1645486459 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x560527e86810 con 0x560527f5b860 2024-04-24T09:11:36.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.730+0000 7f8b2fe00640 1 -- 172.21.15.83:0/1645486459 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8b3401e030 con 0x560527f5b860 2024-04-24T09:11:36.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.730+0000 7f8b2fe00640 1 -- 172.21.15.83:0/1645486459 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 61) ==== 8362+0+0 (secure 0 0 0) 0x7f8b340049a0 con 0x560527f5b860 2024-04-24T09:11:36.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.730+0000 7f8b5404e740 10 client.22560 fetch_fsmap finished waiting for FSMap version 61 2024-04-24T09:11:36.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.730+0000 7f8b5404e740 10 client.22560 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:11:36.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.730+0000 7f8b5404e740 20 client.22560 populate_metadata read hostname 'smithi083' 2024-04-24T09:11:36.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.730+0000 7f8b5404e740 1 --2- 172.21.15.83:0/1645486459 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x560527f76b90 0x560527fdf410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:36.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.730+0000 7f8b5404e740 4 client.22560 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T09:11:36.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.730+0000 7f8b5404e740 1 -- 172.21.15.83:0/1645486459 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560527ec8a80 con 0x560527f76b90 2024-04-24T09:11:36.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.731+0000 7f8b49e00640 1 --2- 172.21.15.83:0/1645486459 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x560527f76b90 0x560527fdf410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:36.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.731+0000 7f8b49e00640 1 --2- 172.21.15.83:0/1645486459 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x560527f76b90 0x560527fdf410 crc :-1 s=READY pgs=502 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4774694e398aaa70 server_cookie=30654e8fd47d1a91 in_seq=0 out_seq=0 2024-04-24T09:11:36.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.731+0000 7f8b2fe00640 10 client.22560 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:11:36.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.732+0000 7f8b2fe00640 1 -- 172.21.15.83:0/1645486459 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x560527ec8a80 con 0x560527f76b90 2024-04-24T09:11:36.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.733+0000 7f8b2fe00640 10 client.22560 handle_command_reply: tid=0 2024-04-24T09:11:36.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.733+0000 7f8b5404e740 1 client.22560 shutdown 2024-04-24T09:11:36.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.733+0000 7f8b5404e740 1 -- 172.21.15.83:0/1645486459 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560527f5b860 msgr2=0x560527f6ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:36.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.733+0000 7f8b5404e740 1 --2- 172.21.15.83:0/1645486459 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560527f5b860 0x560527f6ee30 secure :-1 s=READY pgs=2522 cs=0 l=1 rev1=1 crypto rx=0x7f8b34010970 tx=0x7f8b34005a20 comp rx=0 tx=0).stop 2024-04-24T09:11:36.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.733+0000 7f8b5404e740 1 -- 172.21.15.83:0/1645486459 shutdown_connections 2024-04-24T09:11:36.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.733+0000 7f8b5404e740 1 --2- 172.21.15.83:0/1645486459 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x560527f76b90 0x560527fdf410 crc :-1 s=READY pgs=502 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:36.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.733+0000 7f8b5404e740 1 --2- 172.21.15.83:0/1645486459 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560527f66850 0x560527f6a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:36.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.733+0000 7f8b2fe00640 0 client.22560 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:11:36.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.733+0000 7f8b5404e740 1 --2- 172.21.15.83:0/1645486459 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560527f5c180 0x560527f6a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:36.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.733+0000 7f8b5404e740 1 --2- 172.21.15.83:0/1645486459 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560527f5b860 0x560527f6ee30 unknown :-1 s=CLOSED pgs=2522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:36.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.733+0000 7f8b5404e740 1 -- 172.21.15.83:0/1645486459 >> 172.21.15.83:0/1645486459 conn(0x560527f57b80 msgr2=0x560527f56c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:36.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.733+0000 7f8b5404e740 1 -- 172.21.15.83:0/1645486459 shutdown_connections 2024-04-24T09:11:36.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.733+0000 7f8b5404e740 1 -- 172.21.15.83:0/1645486459 wait complete. 2024-04-24T09:11:36.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.734+0000 7f8b5404e740 20 client.22560 trim_cache size 0 max 16384 2024-04-24T09:11:36.734 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T09:11:36.734 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T09:11:36.734 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-04-24T09:11:36.734 INFO:teuthology.orchestra.run.smithi083.stdout: "80751bc0-4da9-406c-97c1-3cf18e45c6df": { 2024-04-24T09:11:36.734 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-04-24T09:11:36.735 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "80751bc0-4da9-406c-97c1-3cf18e45c6df", 2024-04-24T09:11:36.735 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-04-24T09:11:36.735 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:11:36.735 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:11:36.735 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T09:11:36.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.734+0000 7f8b51200640 1 -- 172.21.15.83:0/728590953 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f8b300a3da0 msgr2=0x7f8b300a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:36.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.734+0000 7f8b51200640 1 --2- 172.21.15.83:0/728590953 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f8b300a3da0 0x7f8b300a6260 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f8b440bcba0 tx=0x7f8b44005e50 comp rx=0 tx=0).stop 2024-04-24T09:11:36.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.734+0000 7f8b51200640 1 -- 172.21.15.83:0/728590953 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b4c10e910 msgr2=0x7f8b4c1ab210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:36.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.734+0000 7f8b51200640 1 --2- 172.21.15.83:0/728590953 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b4c10e910 0x7f8b4c1ab210 secure :-1 s=READY pgs=2519 cs=0 l=1 rev1=1 crypto rx=0x7f8b44009b30 tx=0x7f8b44003e60 comp rx=0 tx=0).stop 2024-04-24T09:11:36.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.735+0000 7f8b51200640 1 -- 172.21.15.83:0/728590953 shutdown_connections 2024-04-24T09:11:36.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.735+0000 7f8b51200640 1 --2- 172.21.15.83:0/728590953 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f8b300a3da0 0x7f8b300a6260 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:36.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.735+0000 7f8b51200640 1 --2- 172.21.15.83:0/728590953 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b4c10e910 0x7f8b4c1ab210 unknown :-1 s=CLOSED pgs=2519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:36.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.735+0000 7f8b51200640 1 --2- 172.21.15.83:0/728590953 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b4c109580 0x7f8b4c10e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:36.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.735+0000 7f8b51200640 1 --2- 172.21.15.83:0/728590953 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8b4c1044b0 0x7f8b4c111130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:36.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.735+0000 7f8b51200640 1 -- 172.21.15.83:0/728590953 >> 172.21.15.83:0/728590953 conn(0x7f8b4c0ffd90 msgr2=0x7f8b4c1010a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:36.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.735+0000 7f8b51200640 1 -- 172.21.15.83:0/728590953 shutdown_connections 2024-04-24T09:11:36.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:11:36.735+0000 7f8b51200640 1 -- 172.21.15.83:0/728590953 wait complete. 2024-04-24T09:11:36.757 INFO:tasks.cephfs.filesystem:scrub status for tag:80751bc0-4da9-406c-97c1-3cf18e45c6df - {'path': '/', 'tag': '80751bc0-4da9-406c-97c1-3cf18e45c6df', 'options': 'recursive,force'} 2024-04-24T09:11:37.182 INFO:tasks.workunit.client.0.smithi083.stdout: 10 46234 6.71 MB/sec execute 248 sec latency 102.757 ms 2024-04-24T09:11:37.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:11:37 smithi083 ceph-mon[27907]: scrub queued for path: / 2024-04-24T09:11:37.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:11:37 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:11:37.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:11:37 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:11:37.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:11:37 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:11:37.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:11:37 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:11:37.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:11:37 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:11:37.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:11:37 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:11:37.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:11:37 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:11:37.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:11:37 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:11:37.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:11:37 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:11:37.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:11:37 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:11:37.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:11:37 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:11:37.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:11:37 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:11:37.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:11:37 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:11:37.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:11:37 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:11:37.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:11:37 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:11:37.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:11:37 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:11:37.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:11:37 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:11:37.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:11:37 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:11:37.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:11:37 smithi138 ceph-mon[31441]: scrub queued for path: / 2024-04-24T09:11:37.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:11:37 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:11:37.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:11:37 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:11:37.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:11:37 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:11:37.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:11:37 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:11:37.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:11:37 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:11:37.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:11:37 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:11:37.706 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:11:37 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:11:37.706 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:11:37 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:11:37.706 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:11:37 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:11:37.706 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:11:37 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:11:37.706 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:11:37 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:11:37.706 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:11:37 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:11:37.706 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:11:37 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:11:37.706 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:11:37 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:11:37.706 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:11:37 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:11:37.706 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:11:37 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:11:37.706 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:11:37 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:11:37.706 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:11:37 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:11:37.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:11:37 smithi189 ceph-mon[31791]: scrub queued for path: / 2024-04-24T09:11:37.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:11:37 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:11:37.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:11:37 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:11:37.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:11:37 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:11:37.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:11:37 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:11:37.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:11:37 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:11:37.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:11:37 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:11:37.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:11:37 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:11:37.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:11:37 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:11:37.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:11:37 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:11:37.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:11:37 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:11:37.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:11:37 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:11:37.709 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:11:37 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:11:37.709 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:11:37 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:11:37.709 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:11:37 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:11:37.709 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:11:37 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:11:37.709 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:11:37 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:11:37.709 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:11:37 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:11:37.709 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:11:37 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:11:38.182 INFO:tasks.workunit.client.0.smithi083.stdout: 10 46341 6.70 MB/sec execute 249 sec latency 70.789 ms 2024-04-24T09:11:38.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:11:38 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:11:38.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:11:38 smithi083 ceph-mon[27907]: pgmap v631: 129 pgs: 129 active+clean; 763 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.7 MiB/s wr, 173 op/s 2024-04-24T09:11:38.704 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:11:38 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:11:38.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:11:38 smithi138 ceph-mon[31441]: pgmap v631: 129 pgs: 129 active+clean; 763 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.7 MiB/s wr, 173 op/s 2024-04-24T09:11:38.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:11:38 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:11:38.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:11:38 smithi189 ceph-mon[31791]: pgmap v631: 129 pgs: 129 active+clean; 763 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.7 MiB/s wr, 173 op/s 2024-04-24T09:11:39.182 INFO:tasks.workunit.client.0.smithi083.stdout: 10 46434 6.69 MB/sec execute 250 sec latency 94.339 ms 2024-04-24T09:11:40.182 INFO:tasks.workunit.client.0.smithi083.stdout: 10 46560 6.69 MB/sec execute 251 sec latency 57.235 ms 2024-04-24T09:11:40.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:11:40 smithi083 ceph-mon[27907]: pgmap v632: 129 pgs: 129 active+clean; 766 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.4 MiB/s wr, 328 op/s 2024-04-24T09:11:40.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:11:40 smithi138 ceph-mon[31441]: pgmap v632: 129 pgs: 129 active+clean; 766 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.4 MiB/s wr, 328 op/s 2024-04-24T09:11:40.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:11:40 smithi189 ceph-mon[31791]: pgmap v632: 129 pgs: 129 active+clean; 766 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.4 MiB/s wr, 328 op/s 2024-04-24T09:11:41.183 INFO:tasks.workunit.client.0.smithi083.stdout: 10 46707 6.72 MB/sec execute 252 sec latency 68.237 ms 2024-04-24T09:11:42.182 INFO:tasks.workunit.client.0.smithi083.stdout: 10 46840 6.71 MB/sec execute 253 sec latency 61.762 ms 2024-04-24T09:11:42.704 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:11:42 smithi138 ceph-mon[31441]: pgmap v633: 129 pgs: 129 active+clean; 766 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.7 MiB/s wr, 237 op/s 2024-04-24T09:11:42.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:11:42 smithi189 ceph-mon[31791]: pgmap v633: 129 pgs: 129 active+clean; 766 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.7 MiB/s wr, 237 op/s 2024-04-24T09:11:42.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:11:42 smithi083 ceph-mon[27907]: pgmap v633: 129 pgs: 129 active+clean; 766 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.7 MiB/s wr, 237 op/s 2024-04-24T09:11:43.182 INFO:tasks.workunit.client.0.smithi083.stdout: 10 46995 6.72 MB/sec execute 254 sec latency 71.831 ms 2024-04-24T09:11:44.182 INFO:tasks.workunit.client.0.smithi083.stdout: 10 47105 6.70 MB/sec execute 255 sec latency 97.152 ms 2024-04-24T09:11:44.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:11:44 smithi138 ceph-mon[31441]: pgmap v634: 129 pgs: 129 active+clean; 773 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.6 MiB/s wr, 342 op/s 2024-04-24T09:11:44.725 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:11:44 smithi189 ceph-mon[31791]: pgmap v634: 129 pgs: 129 active+clean; 773 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.6 MiB/s wr, 342 op/s 2024-04-24T09:11:44.802 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:11:44 smithi083 ceph-mon[27907]: pgmap v634: 129 pgs: 129 active+clean; 773 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.6 MiB/s wr, 342 op/s 2024-04-24T09:11:45.182 INFO:tasks.workunit.client.0.smithi083.stdout: 10 47218 6.70 MB/sec execute 256 sec latency 81.009 ms 2024-04-24T09:11:46.183 INFO:tasks.workunit.client.0.smithi083.stdout: 10 47319 6.67 MB/sec execute 257 sec latency 235.770 ms 2024-04-24T09:11:46.222 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.222+0000 7f4fe9800640 1 -- 172.21.15.83:0/2734667339 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4fe41107d0 msgr2=0x7f4fe4110bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:46.223 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.222+0000 7f4fe9800640 1 --2- 172.21.15.83:0/2734667339 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4fe41107d0 0x7f4fe4110bb0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f4fd4009900 tx=0x7f4fd402a410 comp rx=0 tx=0).stop 2024-04-24T09:11:46.223 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.222+0000 7f4fe9800640 1 -- 172.21.15.83:0/2734667339 shutdown_connections 2024-04-24T09:11:46.223 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.222+0000 7f4fe9800640 1 --2- 172.21.15.83:0/2734667339 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4fe410b1d0 0x7f4fe4117690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:46.223 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.222+0000 7f4fe9800640 1 --2- 172.21.15.83:0/2734667339 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4fe410a810 0x7f4fe410ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:46.223 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.222+0000 7f4fe9800640 1 --2- 172.21.15.83:0/2734667339 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4fe41107d0 0x7f4fe4110bb0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:46.223 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.222+0000 7f4fe9800640 1 -- 172.21.15.83:0/2734667339 >> 172.21.15.83:0/2734667339 conn(0x7f4fe406f140 msgr2=0x7f4fe4109000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:46.223 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.222+0000 7f4fe9800640 1 -- 172.21.15.83:0/2734667339 shutdown_connections 2024-04-24T09:11:46.223 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.222+0000 7f4fe9800640 1 -- 172.21.15.83:0/2734667339 wait complete. 2024-04-24T09:11:46.224 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.223+0000 7f4fe9800640 1 Processor -- start 2024-04-24T09:11:46.224 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.223+0000 7f4fe9800640 1 -- start start 2024-04-24T09:11:46.224 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.224+0000 7f4fe9800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4fe410a810 0x7f4fe41a0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:46.224 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.224+0000 7f4fe9800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4fe410b1d0 0x7f4fe41a1420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:46.225 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.224+0000 7f4fe9800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4fe41107d0 0x7f4fe41a57a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:46.225 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.224+0000 7f4fe9800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe411aae0 con 0x7f4fe410a810 2024-04-24T09:11:46.225 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.224+0000 7f4fe9800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe411a960 con 0x7f4fe41107d0 2024-04-24T09:11:46.225 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.224+0000 7f4fe9800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe411ac60 con 0x7f4fe410b1d0 2024-04-24T09:11:46.225 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.224+0000 7f4fe2a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4fe410a810 0x7f4fe41a0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:46.225 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.224+0000 7f4fe2a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4fe410a810 0x7f4fe41a0ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35966/0 (socket says 172.21.15.83:35966) 2024-04-24T09:11:46.225 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.224+0000 7f4fe2a00640 1 -- 172.21.15.83:0/3392763122 learned_addr learned my addr 172.21.15.83:0/3392763122 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:11:46.225 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.224+0000 7f4fe2000640 1 --2- 172.21.15.83:0/3392763122 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4fe410b1d0 0x7f4fe41a1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:46.225 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.224+0000 7f4fe3400640 1 --2- 172.21.15.83:0/3392763122 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4fe41107d0 0x7f4fe41a57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:46.226 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.225+0000 7f4fe2a00640 1 -- 172.21.15.83:0/3392763122 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4fe410b1d0 msgr2=0x7f4fe41a1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:46.226 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.225+0000 7f4fe2a00640 1 --2- 172.21.15.83:0/3392763122 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4fe410b1d0 0x7f4fe41a1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:46.226 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.225+0000 7f4fe2a00640 1 -- 172.21.15.83:0/3392763122 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4fe41107d0 msgr2=0x7f4fe41a57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:46.226 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.225+0000 7f4fe2a00640 1 --2- 172.21.15.83:0/3392763122 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4fe41107d0 0x7f4fe41a57a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:46.226 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.225+0000 7f4fe2a00640 1 -- 172.21.15.83:0/3392763122 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fe41a5ed0 con 0x7f4fe410a810 2024-04-24T09:11:46.226 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.225+0000 7f4fe3400640 1 --2- 172.21.15.83:0/3392763122 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4fe41107d0 0x7f4fe41a57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:11:46.226 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.225+0000 7f4fe2a00640 1 --2- 172.21.15.83:0/3392763122 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4fe410a810 0x7f4fe41a0ee0 secure :-1 s=READY pgs=2523 cs=0 l=1 rev1=1 crypto rx=0x7f4fd402aa00 tx=0x7f4fd4003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:46.227 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.226+0000 7f4fdb400640 1 -- 172.21.15.83:0/3392763122 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd403f070 con 0x7f4fe410a810 2024-04-24T09:11:46.227 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.226+0000 7f4fdb400640 1 -- 172.21.15.83:0/3392763122 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4fd403bb20 con 0x7f4fe410a810 2024-04-24T09:11:46.227 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.226+0000 7f4fdb400640 1 -- 172.21.15.83:0/3392763122 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd4035030 con 0x7f4fe410a810 2024-04-24T09:11:46.227 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.226+0000 7f4fe9800640 1 -- 172.21.15.83:0/3392763122 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fe41a6160 con 0x7f4fe410a810 2024-04-24T09:11:46.227 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.226+0000 7f4fe9800640 1 -- 172.21.15.83:0/3392763122 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fe41ada50 con 0x7f4fe410a810 2024-04-24T09:11:46.230 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.230+0000 7f4fec649740 1 Processor -- start 2024-04-24T09:11:46.230 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.230+0000 7f4fec649740 1 -- start start 2024-04-24T09:11:46.231 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.231+0000 7f4fec649740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561c9b906f0 0x5561c9b8f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:46.231 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.231+0000 7f4fec649740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5561c9b8f5b0 0x5561c9b8fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:46.231 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.231+0000 7f4fec649740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5561c9ba2a30 0x5561c9ba4e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:46.231 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.231+0000 7f4fec649740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5561c9af5900 con 0x5561c9b8f5b0 2024-04-24T09:11:46.231 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.231+0000 7f4fec649740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5561c9a9bf20 con 0x5561c9ba2a30 2024-04-24T09:11:46.231 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.231+0000 7f4fec649740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5561c9af18c0 con 0x5561c9b906f0 2024-04-24T09:11:46.231 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.231+0000 7f4fdb400640 1 -- 172.21.15.83:0/3392763122 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4fd4042070 con 0x7f4fe410a810 2024-04-24T09:11:46.232 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.231+0000 7f4fe2a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5561c9b8f5b0 0x5561c9b8fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:46.232 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.231+0000 7f4fe3400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561c9b906f0 0x5561c9b8f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:46.232 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.231+0000 7f4fe3400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561c9b906f0 0x5561c9b8f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:58896/0 (socket says 172.21.15.83:58896) 2024-04-24T09:11:46.232 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.231+0000 7f4fe3400640 1 -- 172.21.15.83:0/3912895744 learned_addr learned my addr 172.21.15.83:0/3912895744 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:11:46.232 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.231+0000 7f4fe2000640 1 --2- 172.21.15.83:0/3912895744 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5561c9ba2a30 0x5561c9ba4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:46.233 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.232+0000 7f4fe2a00640 1 -- 172.21.15.83:0/3912895744 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561c9b906f0 msgr2=0x5561c9b8f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:46.233 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.232+0000 7f4fe2a00640 1 --2- 172.21.15.83:0/3912895744 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561c9b906f0 0x5561c9b8f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:46.233 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.232+0000 7f4fe2a00640 1 -- 172.21.15.83:0/3912895744 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5561c9ba2a30 msgr2=0x5561c9ba4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:46.233 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.232+0000 7f4fe2a00640 1 --2- 172.21.15.83:0/3912895744 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5561c9ba2a30 0x5561c9ba4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:46.233 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.232+0000 7f4fe2a00640 1 -- 172.21.15.83:0/3912895744 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561c9ac0e10 con 0x5561c9b8f5b0 2024-04-24T09:11:46.233 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.232+0000 7f4fe2a00640 1 --2- 172.21.15.83:0/3912895744 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5561c9b8f5b0 0x5561c9b8fa30 secure :-1 s=READY pgs=2524 cs=0 l=1 rev1=1 crypto rx=0x7f4fd40ee630 tx=0x7f4fd40f9620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e5461a5b25cef38 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:46.234 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.233+0000 7f4fdb400640 1 --2- 172.21.15.83:0/3392763122 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f4fc80a3e40 0x7f4fc80a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:46.234 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.233+0000 7f4fdb400640 1 -- 172.21.15.83:0/3392763122 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 9583+0+0 (secure 0 0 0) 0x7f4fd40f1300 con 0x7f4fe410a810 2024-04-24T09:11:46.234 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.233+0000 7f4fe3400640 1 --2- 172.21.15.83:0/3392763122 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f4fc80a3e40 0x7f4fc80a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:46.234 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.233+0000 7f4fcfe00640 1 -- 172.21.15.83:0/3912895744 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd40b9120 con 0x5561c9b8f5b0 2024-04-24T09:11:46.234 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.233+0000 7f4fcfe00640 1 -- 172.21.15.83:0/3912895744 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4fd40433b0 con 0x5561c9b8f5b0 2024-04-24T09:11:46.234 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.233+0000 7f4fcfe00640 1 -- 172.21.15.83:0/3912895744 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd40bc7e0 con 0x5561c9b8f5b0 2024-04-24T09:11:46.235 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.234+0000 7f4fec649740 1 -- 172.21.15.83:0/3912895744 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5561c9b8f5b0 msgr2=0x5561c9b8fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:46.235 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.234+0000 7f4fe3400640 1 --2- 172.21.15.83:0/3392763122 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f4fc80a3e40 0x7f4fc80a6300 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc013570 tx=0x7f4fdc009820 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:46.235 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.234+0000 7f4fec649740 1 --2- 172.21.15.83:0/3912895744 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5561c9b8f5b0 0x5561c9b8fa30 secure :-1 s=READY pgs=2524 cs=0 l=1 rev1=1 crypto rx=0x7f4fd40ee630 tx=0x7f4fd40f9620 comp rx=0 tx=0).stop 2024-04-24T09:11:46.235 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.235+0000 7f4fec649740 1 -- 172.21.15.83:0/3912895744 shutdown_connections 2024-04-24T09:11:46.235 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.235+0000 7f4fec649740 1 --2- 172.21.15.83:0/3912895744 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5561c9ba2a30 0x5561c9ba4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:46.235 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.235+0000 7f4fec649740 1 --2- 172.21.15.83:0/3912895744 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5561c9b8f5b0 0x5561c9b8fa30 unknown :-1 s=CLOSED pgs=2524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:46.235 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.235+0000 7f4fec649740 1 --2- 172.21.15.83:0/3912895744 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561c9b906f0 0x5561c9b8f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:46.235 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.235+0000 7f4fec649740 1 -- 172.21.15.83:0/3912895744 >> 172.21.15.83:0/3912895744 conn(0x5561c9b8b6c0 msgr2=0x5561c9b8baa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:46.236 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.235+0000 7f4fec649740 1 -- 172.21.15.83:0/3912895744 shutdown_connections 2024-04-24T09:11:46.236 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.235+0000 7f4fec649740 1 -- 172.21.15.83:0/3912895744 wait complete. 2024-04-24T09:11:46.236 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.235+0000 7f4fec649740 1 -- start start 2024-04-24T09:11:46.236 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.236+0000 7f4fec649740 1 Processor -- start 2024-04-24T09:11:46.237 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.237+0000 7f4fec649740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5561c9b8f5b0 0x5561c9b92e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:46.237 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.237+0000 7f4fec649740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561c9b906f0 0x5561c9b933b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:46.237 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.237+0000 7f4fec649740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5561c9ba2a30 0x5561c9b938f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:46.237 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.237+0000 7f4fec649740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5561c9ba5550 con 0x5561c9b8f5b0 2024-04-24T09:11:46.237 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.237+0000 7f4fec649740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5561c9aaf2c0 con 0x5561c9ba2a30 2024-04-24T09:11:46.237 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.237+0000 7f4fec649740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5561c9ba56d0 con 0x5561c9b906f0 2024-04-24T09:11:46.237 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.237+0000 7f4fec649740 10 client.0 _ll_register_callbacks cb 0x5561c9807a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:11:46.239 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.237+0000 7f4fe2a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5561c9b8f5b0 0x5561c9b92e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:46.239 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.237+0000 7f4fe2a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5561c9b8f5b0 0x5561c9b92e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35994/0 (socket says 172.21.15.83:35994) 2024-04-24T09:11:46.239 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.237+0000 7f4fe2a00640 1 -- 172.21.15.83:0/2252787043 learned_addr learned my addr 172.21.15.83:0/2252787043 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:11:46.239 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.237+0000 7f4fe2000640 1 --2- 172.21.15.83:0/2252787043 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561c9b906f0 0x5561c9b933b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:46.239 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.238+0000 7f4fe3400640 1 --2- 172.21.15.83:0/2252787043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5561c9ba2a30 0x5561c9b938f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:46.239 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.238+0000 7f4fe2a00640 1 -- 172.21.15.83:0/2252787043 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561c9b906f0 msgr2=0x5561c9b933b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:46.240 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.238+0000 7f4fe2a00640 1 --2- 172.21.15.83:0/2252787043 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561c9b906f0 0x5561c9b933b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:46.240 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.238+0000 7f4fe2a00640 1 -- 172.21.15.83:0/2252787043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5561c9ba2a30 msgr2=0x5561c9b938f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:46.240 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.238+0000 7f4fe2a00640 1 --2- 172.21.15.83:0/2252787043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5561c9ba2a30 0x5561c9b938f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:46.240 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.238+0000 7f4fe2a00640 1 -- 172.21.15.83:0/2252787043 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561c9abffd0 con 0x5561c9b8f5b0 2024-04-24T09:11:46.240 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.238+0000 7f4fe2a00640 1 --2- 172.21.15.83:0/2252787043 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5561c9b8f5b0 0x5561c9b92e70 secure :-1 s=READY pgs=2525 cs=0 l=1 rev1=1 crypto rx=0x7f4fd40ee6e0 tx=0x7f4fd4005e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8a44c4b528c2f72 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:46.240 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.238+0000 7f4fccc00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:11:46.240 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.239+0000 7f4fccc00640 1 -- 172.21.15.83:0/2252787043 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd40bc9c0 con 0x5561c9b8f5b0 2024-04-24T09:11:46.240 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.239+0000 7f4fccc00640 1 -- 172.21.15.83:0/2252787043 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 9583+0+0 (secure 0 0 0) 0x7f4fd40f9150 con 0x5561c9b8f5b0 2024-04-24T09:11:46.240 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.239+0000 7f4fec649740 1 -- 172.21.15.83:0/2252787043 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4fd40433b0 con 0x5561c9b8f5b0 2024-04-24T09:11:46.240 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.239+0000 7f4fccc00640 1 -- 172.21.15.83:0/2252787043 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f4fd40bcc50 con 0x5561c9b8f5b0 2024-04-24T09:11:46.240 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.239+0000 7f4fec649740 10 client.22572 fetch_fsmap learned FSMap version 61 2024-04-24T09:11:46.240 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.239+0000 7f4fec649740 1 -- 172.21.15.83:0/2252787043 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x5561c9808210 con 0x5561c9b8f5b0 2024-04-24T09:11:46.241 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.240+0000 7f4fccc00640 1 -- 172.21.15.83:0/2252787043 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4fd40433b0 con 0x5561c9b8f5b0 2024-04-24T09:11:46.241 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.240+0000 7f4fccc00640 1 -- 172.21.15.83:0/2252787043 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 61) ==== 8362+0+0 (secure 0 0 0) 0x7f4fd40ee8b0 con 0x5561c9b8f5b0 2024-04-24T09:11:46.241 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.240+0000 7f4fec649740 10 client.22572 fetch_fsmap finished waiting for FSMap version 61 2024-04-24T09:11:46.241 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.240+0000 7f4fec649740 10 client.22572 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:11:46.241 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.240+0000 7f4fec649740 20 client.22572 populate_metadata read hostname 'smithi083' 2024-04-24T09:11:46.241 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.240+0000 7f4fec649740 1 --2- 172.21.15.83:0/2252787043 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5561c9bb2940 0x5561c9bb4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:46.241 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.240+0000 7f4fec649740 4 client.22572 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:11:46.241 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.240+0000 7f4fec649740 1 -- 172.21.15.83:0/2252787043 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5561c9b03320 con 0x5561c9bb2940 2024-04-24T09:11:46.241 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.241+0000 7f4fe2000640 1 --2- 172.21.15.83:0/2252787043 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5561c9bb2940 0x5561c9bb4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:46.242 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.241+0000 7f4fe2000640 1 --2- 172.21.15.83:0/2252787043 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5561c9bb2940 0x5561c9bb4d30 crc :-1 s=READY pgs=503 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=adad33e755a336c2 server_cookie=4e1720646844e64 in_seq=0 out_seq=0 2024-04-24T09:11:46.242 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.241+0000 7f4fccc00640 10 client.22572 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:11:46.243 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.242+0000 7f4fccc00640 1 -- 172.21.15.83:0/2252787043 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5561c9b03320 con 0x5561c9bb2940 2024-04-24T09:11:46.243 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.242+0000 7f4fccc00640 10 client.22572 handle_command_reply: tid=0 2024-04-24T09:11:46.243 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.242+0000 7f4fec649740 1 client.22572 shutdown 2024-04-24T09:11:46.243 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.243+0000 7f4fec649740 1 -- 172.21.15.83:0/2252787043 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5561c9b8f5b0 msgr2=0x5561c9b92e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:46.243 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.243+0000 7f4fec649740 1 --2- 172.21.15.83:0/2252787043 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5561c9b8f5b0 0x5561c9b92e70 secure :-1 s=READY pgs=2525 cs=0 l=1 rev1=1 crypto rx=0x7f4fd40ee6e0 tx=0x7f4fd4005e00 comp rx=0 tx=0).stop 2024-04-24T09:11:46.243 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.243+0000 7f4fec649740 1 -- 172.21.15.83:0/2252787043 shutdown_connections 2024-04-24T09:11:46.243 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.243+0000 7f4fec649740 1 --2- 172.21.15.83:0/2252787043 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5561c9bb2940 0x5561c9bb4d30 crc :-1 s=READY pgs=503 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:46.244 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.243+0000 7f4fec649740 1 --2- 172.21.15.83:0/2252787043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5561c9ba2a30 0x5561c9b938f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:46.244 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.243+0000 7f4fec649740 1 --2- 172.21.15.83:0/2252787043 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561c9b906f0 0x5561c9b933b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:46.244 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.243+0000 7f4fec649740 1 --2- 172.21.15.83:0/2252787043 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5561c9b8f5b0 0x5561c9b92e70 unknown :-1 s=CLOSED pgs=2525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:46.244 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.243+0000 7f4fccc00640 0 client.22572 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:11:46.244 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.243+0000 7f4fec649740 1 -- 172.21.15.83:0/2252787043 >> 172.21.15.83:0/2252787043 conn(0x5561c9b8b6c0 msgr2=0x5561c9b91510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:46.244 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.243+0000 7f4fec649740 1 -- 172.21.15.83:0/2252787043 shutdown_connections 2024-04-24T09:11:46.244 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.243+0000 7f4fec649740 1 -- 172.21.15.83:0/2252787043 wait complete. 2024-04-24T09:11:46.244 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.244+0000 7f4fec649740 20 client.22572 trim_cache size 0 max 16384 2024-04-24T09:11:46.271 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.271+0000 7f4fec649740 1 Processor -- start 2024-04-24T09:11:46.271 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.271+0000 7f4fec649740 1 -- start start 2024-04-24T09:11:46.272 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.272+0000 7f4fec649740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5561c9b906f0 0x5561c9bae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:46.272 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.272+0000 7f4fec649740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5561c9ba2a30 0x5561c9b91c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:46.272 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.272+0000 7f4fec649740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561c9b8a6c0 0x5561c9bba3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:46.272 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.272+0000 7f4fec649740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe41b5ba0 con 0x5561c9ba2a30 2024-04-24T09:11:46.272 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.272+0000 7f4fec649740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe41b76e0 con 0x5561c9b906f0 2024-04-24T09:11:46.272 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.272+0000 7f4fec649740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe41b6270 con 0x5561c9b8a6c0 2024-04-24T09:11:46.272 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.272+0000 7f4fe2a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5561c9b906f0 0x5561c9bae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:46.273 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.272+0000 7f4fe2a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5561c9b906f0 0x5561c9bae410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:47898/0 (socket says 172.21.15.83:47898) 2024-04-24T09:11:46.273 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.272+0000 7f4fe2a00640 1 -- 172.21.15.83:0/1899098527 learned_addr learned my addr 172.21.15.83:0/1899098527 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:11:46.273 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.272+0000 7f4fe2000640 1 --2- 172.21.15.83:0/1899098527 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5561c9ba2a30 0x5561c9b91c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:46.273 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.273+0000 7f4fe3400640 1 --2- 172.21.15.83:0/1899098527 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561c9b8a6c0 0x5561c9bba3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:46.274 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.273+0000 7f4fe2a00640 1 -- 172.21.15.83:0/1899098527 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561c9b8a6c0 msgr2=0x5561c9bba3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:46.274 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.273+0000 7f4fe2a00640 1 --2- 172.21.15.83:0/1899098527 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561c9b8a6c0 0x5561c9bba3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:46.274 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.273+0000 7f4fe2a00640 1 -- 172.21.15.83:0/1899098527 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5561c9ba2a30 msgr2=0x5561c9b91c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:46.274 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.273+0000 7f4fe2a00640 1 --2- 172.21.15.83:0/1899098527 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5561c9ba2a30 0x5561c9b91c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:46.274 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.273+0000 7f4fe2a00640 1 -- 172.21.15.83:0/1899098527 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561c98d4620 con 0x5561c9b906f0 2024-04-24T09:11:46.274 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.273+0000 7f4fe2000640 1 --2- 172.21.15.83:0/1899098527 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5561c9ba2a30 0x5561c9b91c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:11:46.275 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.274+0000 7f4fe2a00640 1 --2- 172.21.15.83:0/1899098527 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5561c9b906f0 0x5561c9bae410 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f4fd40eec30 tx=0x7f4fd40f9620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=742b91e960446967 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:46.275 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.274+0000 7f4fcf400640 1 -- 172.21.15.83:0/1899098527 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd40bc9c0 con 0x5561c9b906f0 2024-04-24T09:11:46.275 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.274+0000 7f4fcf400640 1 -- 172.21.15.83:0/1899098527 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4fd40433b0 con 0x5561c9b906f0 2024-04-24T09:11:46.275 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.274+0000 7f4fcf400640 1 -- 172.21.15.83:0/1899098527 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd40b9120 con 0x5561c9b906f0 2024-04-24T09:11:46.276 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.276+0000 7f4fec649740 1 -- 172.21.15.83:0/1899098527 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5561c9b906f0 msgr2=0x5561c9bae410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:46.276 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.276+0000 7f4fec649740 1 --2- 172.21.15.83:0/1899098527 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5561c9b906f0 0x5561c9bae410 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f4fd40eec30 tx=0x7f4fd40f9620 comp rx=0 tx=0).stop 2024-04-24T09:11:46.277 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.277+0000 7f4fec649740 1 -- 172.21.15.83:0/1899098527 shutdown_connections 2024-04-24T09:11:46.277 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.277+0000 7f4fec649740 1 --2- 172.21.15.83:0/1899098527 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561c9b8a6c0 0x5561c9bba3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:46.278 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.277+0000 7f4fec649740 1 --2- 172.21.15.83:0/1899098527 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5561c9ba2a30 0x5561c9b91c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:46.278 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.277+0000 7f4fec649740 1 --2- 172.21.15.83:0/1899098527 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5561c9b906f0 0x5561c9bae410 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:46.278 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.277+0000 7f4fec649740 1 -- 172.21.15.83:0/1899098527 >> 172.21.15.83:0/1899098527 conn(0x5561c9b8b6c0 msgr2=0x5561c9b89fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:46.278 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.277+0000 7f4fec649740 1 -- 172.21.15.83:0/1899098527 shutdown_connections 2024-04-24T09:11:46.278 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.277+0000 7f4fec649740 1 -- 172.21.15.83:0/1899098527 wait complete. 2024-04-24T09:11:46.278 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.278+0000 7f4fec649740 1 -- start start 2024-04-24T09:11:46.278 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.278+0000 7f4fec649740 1 Processor -- start 2024-04-24T09:11:46.279 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.279+0000 7f4fec649740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5561c9b8a6c0 0x5561c9ba6370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:46.279 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.279+0000 7f4fec649740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5561c9b906f0 0x5561c9ba68b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:46.279 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.279+0000 7f4fec649740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561c9ba2a30 0x5561c9ba6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:46.279 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.279+0000 7f4fec649740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe41b7010 con 0x5561c9b8a6c0 2024-04-24T09:11:46.279 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.279+0000 7f4fec649740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe41b6940 con 0x5561c9b906f0 2024-04-24T09:11:46.279 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.279+0000 7f4fec649740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe41b3ba0 con 0x5561c9ba2a30 2024-04-24T09:11:46.280 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.279+0000 7f4fec649740 10 client.0 _ll_register_callbacks cb 0x7f4fd0009590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:11:46.280 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.280+0000 7f4fe2000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5561c9b906f0 0x5561c9ba68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:46.280 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.280+0000 7f4fe2a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5561c9b8a6c0 0x5561c9ba6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:46.281 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.280+0000 7f4fe2000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5561c9b906f0 0x5561c9ba68b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:47914/0 (socket says 172.21.15.83:47914) 2024-04-24T09:11:46.281 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.280+0000 7f4fe2000640 1 -- 172.21.15.83:0/3952048692 learned_addr learned my addr 172.21.15.83:0/3952048692 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:11:46.281 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.280+0000 7f4fe2a00640 1 -- 172.21.15.83:0/3952048692 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561c9ba2a30 msgr2=0x5561c9ba6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:46.281 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.280+0000 7f4fe2a00640 1 --2- 172.21.15.83:0/3952048692 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561c9ba2a30 0x5561c9ba6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:46.281 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.280+0000 7f4fe2a00640 1 -- 172.21.15.83:0/3952048692 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5561c9b906f0 msgr2=0x5561c9ba68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:11:46.281 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.280+0000 7f4fe2a00640 1 --2- 172.21.15.83:0/3952048692 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5561c9b906f0 0x5561c9ba68b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:46.281 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.280+0000 7f4fe2a00640 1 -- 172.21.15.83:0/3952048692 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561c9c17c60 con 0x5561c9b8a6c0 2024-04-24T09:11:46.281 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.280+0000 7f4fe2000640 1 --2- 172.21.15.83:0/3952048692 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5561c9b906f0 0x5561c9ba68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:11:46.282 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.281+0000 7f4fe2a00640 1 --2- 172.21.15.83:0/3952048692 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5561c9b8a6c0 0x5561c9ba6370 secure :-1 s=READY pgs=2526 cs=0 l=1 rev1=1 crypto rx=0x7f4fd40bce10 tx=0x7f4fd40f9750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a55393458ea1bfe server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:11:46.282 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.281+0000 7f4fccc00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:11:46.282 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.281+0000 7f4fccc00640 1 -- 172.21.15.83:0/3952048692 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd40da070 con 0x5561c9b8a6c0 2024-04-24T09:11:46.282 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.281+0000 7f4fccc00640 1 -- 172.21.15.83:0/3952048692 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 9583+0+0 (secure 0 0 0) 0x7f4fd40f4960 con 0x5561c9b8a6c0 2024-04-24T09:11:46.282 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.281+0000 7f4fec649740 1 -- 172.21.15.83:0/3952048692 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4fd40433b0 con 0x5561c9b8a6c0 2024-04-24T09:11:46.282 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.281+0000 7f4fccc00640 1 -- 172.21.15.83:0/3952048692 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f4fd40a1050 con 0x5561c9b8a6c0 2024-04-24T09:11:46.282 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.282+0000 7f4fec649740 10 client.22578 fetch_fsmap learned FSMap version 61 2024-04-24T09:11:46.282 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.282+0000 7f4fec649740 1 -- 172.21.15.83:0/3952048692 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x5561c9c17c60 con 0x5561c9b8a6c0 2024-04-24T09:11:46.283 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.282+0000 7f4fccc00640 1 -- 172.21.15.83:0/3952048692 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4fd40433b0 con 0x5561c9b8a6c0 2024-04-24T09:11:46.283 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.282+0000 7f4fccc00640 1 -- 172.21.15.83:0/3952048692 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 61) ==== 8362+0+0 (secure 0 0 0) 0x7f4fd40d6240 con 0x5561c9b8a6c0 2024-04-24T09:11:46.283 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.282+0000 7f4fec649740 10 client.22578 fetch_fsmap finished waiting for FSMap version 61 2024-04-24T09:11:46.283 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.282+0000 7f4fec649740 10 client.22578 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:11:46.283 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.282+0000 7f4fec649740 20 client.22578 populate_metadata read hostname 'smithi083' 2024-04-24T09:11:46.283 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.282+0000 7f4fec649740 1 --2- 172.21.15.83:0/3952048692 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5561c9c19250 0x5561c9c1b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:11:46.283 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.282+0000 7f4fec649740 4 client.22578 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T09:11:46.283 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.282+0000 7f4fec649740 1 -- 172.21.15.83:0/3952048692 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5561c9afd190 con 0x5561c9c19250 2024-04-24T09:11:46.283 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.283+0000 7f4fe2000640 1 --2- 172.21.15.83:0/3952048692 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5561c9c19250 0x5561c9c1b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:11:46.284 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.283+0000 7f4fe2000640 1 --2- 172.21.15.83:0/3952048692 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5561c9c19250 0x5561c9c1b640 crc :-1 s=READY pgs=504 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=882bcb9c45dc0650 server_cookie=920b9beb2826b447 in_seq=0 out_seq=0 2024-04-24T09:11:46.284 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.283+0000 7f4fccc00640 10 client.22578 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:11:46.409 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.408+0000 7f4fccc00640 1 -- 172.21.15.83:0/3952048692 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5561c9afd190 con 0x5561c9c19250 2024-04-24T09:11:46.409 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.408+0000 7f4fccc00640 10 client.22578 handle_command_reply: tid=0 2024-04-24T09:11:46.409 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.409+0000 7f4fec649740 1 client.22578 shutdown 2024-04-24T09:11:46.409 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.409+0000 7f4fec649740 1 -- 172.21.15.83:0/3952048692 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5561c9b8a6c0 msgr2=0x5561c9ba6370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:46.410 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.409+0000 7f4fec649740 1 --2- 172.21.15.83:0/3952048692 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5561c9b8a6c0 0x5561c9ba6370 secure :-1 s=READY pgs=2526 cs=0 l=1 rev1=1 crypto rx=0x7f4fd40bce10 tx=0x7f4fd40f9750 comp rx=0 tx=0).stop 2024-04-24T09:11:46.410 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.409+0000 7f4fec649740 1 -- 172.21.15.83:0/3952048692 shutdown_connections 2024-04-24T09:11:46.410 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.409+0000 7f4fec649740 1 --2- 172.21.15.83:0/3952048692 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5561c9c19250 0x5561c9c1b640 crc :-1 s=READY pgs=504 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:46.410 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.409+0000 7f4fec649740 1 --2- 172.21.15.83:0/3952048692 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561c9ba2a30 0x5561c9ba6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:46.410 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.409+0000 7f4fec649740 1 --2- 172.21.15.83:0/3952048692 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5561c9b906f0 0x5561c9ba68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:46.410 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.409+0000 7f4fec649740 1 --2- 172.21.15.83:0/3952048692 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5561c9b8a6c0 0x5561c9ba6370 secure :-1 s=CLOSED pgs=2526 cs=0 l=1 rev1=1 crypto rx=0x7f4fd40bce10 tx=0x7f4fd40f9750 comp rx=0 tx=0).stop 2024-04-24T09:11:46.410 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.409+0000 7f4fec649740 1 -- 172.21.15.83:0/3952048692 >> 172.21.15.83:0/3952048692 conn(0x5561c9b8b6c0 msgr2=0x5561c9b92e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:46.410 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.409+0000 7f4fccc00640 0 client.22578 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:11:46.410 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.410+0000 7f4fec649740 1 -- 172.21.15.83:0/3952048692 shutdown_connections 2024-04-24T09:11:46.411 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.410+0000 7f4fec649740 1 -- 172.21.15.83:0/3952048692 wait complete. 2024-04-24T09:11:46.411 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.410+0000 7f4fec649740 20 client.22578 trim_cache size 0 max 16384 2024-04-24T09:11:46.411 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T09:11:46.411 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T09:11:46.411 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T09:11:46.412 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T09:11:46.412 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.411+0000 7f4fe9800640 1 -- 172.21.15.83:0/3392763122 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f4fc80a3e40 msgr2=0x7f4fc80a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:46.412 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.411+0000 7f4fe9800640 1 --2- 172.21.15.83:0/3392763122 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f4fc80a3e40 0x7f4fc80a6300 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc013570 tx=0x7f4fdc009820 comp rx=0 tx=0).stop 2024-04-24T09:11:46.412 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.411+0000 7f4fe9800640 1 -- 172.21.15.83:0/3392763122 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4fe410a810 msgr2=0x7f4fe41a0ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:11:46.412 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.411+0000 7f4fe9800640 1 --2- 172.21.15.83:0/3392763122 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4fe410a810 0x7f4fe41a0ee0 secure :-1 s=READY pgs=2523 cs=0 l=1 rev1=1 crypto rx=0x7f4fd402aa00 tx=0x7f4fd4003c10 comp rx=0 tx=0).stop 2024-04-24T09:11:46.412 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.412+0000 7f4fe9800640 1 -- 172.21.15.83:0/3392763122 shutdown_connections 2024-04-24T09:11:46.413 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.412+0000 7f4fe9800640 1 --2- 172.21.15.83:0/3392763122 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f4fc80a3e40 0x7f4fc80a6300 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:46.413 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.412+0000 7f4fe9800640 1 --2- 172.21.15.83:0/3392763122 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4fe41107d0 0x7f4fe41a57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:46.413 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.412+0000 7f4fe9800640 1 --2- 172.21.15.83:0/3392763122 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4fe410b1d0 0x7f4fe41a1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:46.413 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.412+0000 7f4fe9800640 1 --2- 172.21.15.83:0/3392763122 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4fe410a810 0x7f4fe41a0ee0 unknown :-1 s=CLOSED pgs=2523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:11:46.413 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.412+0000 7f4fe9800640 1 -- 172.21.15.83:0/3392763122 >> 172.21.15.83:0/3392763122 conn(0x7f4fe406f140 msgr2=0x7f4fe406f660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:11:46.413 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.412+0000 7f4fe9800640 1 -- 172.21.15.83:0/3392763122 shutdown_connections 2024-04-24T09:11:46.413 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:11:46.412+0000 7f4fe9800640 1 -- 172.21.15.83:0/3392763122 wait complete. 2024-04-24T09:11:46.704 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:11:46 smithi138 ceph-mon[31441]: pgmap v635: 129 pgs: 129 active+clean; 773 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.5 MiB/s wr, 260 op/s 2024-04-24T09:11:46.709 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:11:46 smithi189 ceph-mon[31791]: pgmap v635: 129 pgs: 129 active+clean; 773 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.5 MiB/s wr, 260 op/s 2024-04-24T09:11:46.802 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:11:46 smithi083 ceph-mon[27907]: pgmap v635: 129 pgs: 129 active+clean; 773 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.5 MiB/s wr, 260 op/s 2024-04-24T09:11:47.183 INFO:tasks.workunit.client.0.smithi083.stdout: 10 47444 6.68 MB/sec execute 258 sec latency 121.580 ms 2024-04-24T09:11:48.183 INFO:tasks.workunit.client.0.smithi083.stdout: 10 47569 6.68 MB/sec execute 259 sec latency 94.957 ms 2024-04-24T09:11:48.604 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:11:48 smithi083 ceph-mon[27907]: pgmap v636: 129 pgs: 129 active+clean; 773 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.5 MiB/s wr, 260 op/s 2024-04-24T09:11:48.604 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:11:48 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:11:48.604 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:11:48 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:11:48.605 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:11:48 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:11:48.605 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:11:48 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:11:48.605 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:11:48 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:11:48.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:11:48 smithi138 ceph-mon[31441]: pgmap v636: 129 pgs: 129 active+clean; 773 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.5 MiB/s wr, 260 op/s 2024-04-24T09:11:48.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:11:48 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:11:48.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:11:48 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:11:48.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:11:48 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:11:48.706 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:11:48 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:11:48.706 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:11:48 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:11:48.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:11:48 smithi189 ceph-mon[31791]: pgmap v636: 129 pgs: 129 active+clean; 773 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.5 MiB/s wr, 260 op/s 2024-04-24T09:11:48.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:11:48 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:11:48.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:11:48 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:11:48.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:11:48 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:11:48.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:11:48 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:11:48.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:11:48 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:11:49.183 INFO:tasks.workunit.client.0.smithi083.stdout: 10 47694 6.67 MB/sec execute 260 sec latency 68.516 ms 2024-04-24T09:11:50.187 INFO:tasks.workunit.client.0.smithi083.stdout: 10 47848 6.68 MB/sec execute 261 sec latency 74.458 ms 2024-04-24T09:11:50.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:11:50 smithi138 ceph-mon[31441]: pgmap v637: 129 pgs: 129 active+clean; 774 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.2 MiB/s wr, 400 op/s 2024-04-24T09:11:50.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:11:50 smithi189 ceph-mon[31791]: pgmap v637: 129 pgs: 129 active+clean; 774 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.2 MiB/s wr, 400 op/s 2024-04-24T09:11:50.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:11:50 smithi083 ceph-mon[27907]: pgmap v637: 129 pgs: 129 active+clean; 774 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.2 MiB/s wr, 400 op/s 2024-04-24T09:11:51.183 INFO:tasks.workunit.client.0.smithi083.stdout: 10 47935 6.67 MB/sec execute 262 sec latency 106.232 ms 2024-04-24T09:11:52.183 INFO:tasks.workunit.client.0.smithi083.stdout: 10 48031 6.67 MB/sec execute 263 sec latency 368.205 ms 2024-04-24T09:11:52.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:11:52 smithi138 ceph-mon[31441]: pgmap v638: 129 pgs: 129 active+clean; 774 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.5 MiB/s wr, 244 op/s 2024-04-24T09:11:52.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:11:52 smithi189 ceph-mon[31791]: pgmap v638: 129 pgs: 129 active+clean; 774 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.5 MiB/s wr, 244 op/s 2024-04-24T09:11:52.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:11:52 smithi083 ceph-mon[27907]: pgmap v638: 129 pgs: 129 active+clean; 774 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.5 MiB/s wr, 244 op/s 2024-04-24T09:11:53.183 INFO:tasks.workunit.client.0.smithi083.stdout: 10 48140 6.66 MB/sec execute 264 sec latency 70.397 ms 2024-04-24T09:11:54.183 INFO:tasks.workunit.client.0.smithi083.stdout: 10 48284 6.67 MB/sec execute 265 sec latency 76.688 ms 2024-04-24T09:11:54.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:11:54 smithi138 ceph-mon[31441]: pgmap v639: 129 pgs: 129 active+clean; 779 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.0 MiB/s wr, 346 op/s 2024-04-24T09:11:54.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:11:54 smithi189 ceph-mon[31791]: pgmap v639: 129 pgs: 129 active+clean; 779 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.0 MiB/s wr, 346 op/s 2024-04-24T09:11:54.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:11:54 smithi083 ceph-mon[27907]: pgmap v639: 129 pgs: 129 active+clean; 779 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.0 MiB/s wr, 346 op/s 2024-04-24T09:11:55.183 INFO:tasks.workunit.client.0.smithi083.stdout: 10 48444 6.69 MB/sec execute 266 sec latency 88.661 ms 2024-04-24T09:11:56.183 INFO:tasks.workunit.client.0.smithi083.stdout: 10 48545 6.68 MB/sec execute 267 sec latency 77.541 ms 2024-04-24T09:11:56.704 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:11:56 smithi138 ceph-mon[31441]: pgmap v640: 129 pgs: 129 active+clean; 779 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.1 MiB/s wr, 241 op/s 2024-04-24T09:11:56.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:11:56 smithi189 ceph-mon[31791]: pgmap v640: 129 pgs: 129 active+clean; 779 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.1 MiB/s wr, 241 op/s 2024-04-24T09:11:56.802 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:11:56 smithi083 ceph-mon[27907]: pgmap v640: 129 pgs: 129 active+clean; 779 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.1 MiB/s wr, 241 op/s 2024-04-24T09:11:57.183 INFO:tasks.workunit.client.0.smithi083.stdout: 10 48656 6.68 MB/sec execute 268 sec latency 75.325 ms 2024-04-24T09:11:58.183 INFO:tasks.workunit.client.0.smithi083.stdout: 10 48799 6.70 MB/sec execute 269 sec latency 74.598 ms 2024-04-24T09:11:58.704 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:11:58 smithi138 ceph-mon[31441]: pgmap v641: 129 pgs: 129 active+clean; 779 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.1 MiB/s wr, 241 op/s 2024-04-24T09:11:58.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:11:58 smithi189 ceph-mon[31791]: pgmap v641: 129 pgs: 129 active+clean; 779 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.1 MiB/s wr, 241 op/s 2024-04-24T09:11:58.802 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:11:58 smithi083 ceph-mon[27907]: pgmap v641: 129 pgs: 129 active+clean; 779 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.1 MiB/s wr, 241 op/s 2024-04-24T09:11:59.183 INFO:tasks.workunit.client.0.smithi083.stdout: 10 48959 6.70 MB/sec execute 270 sec latency 86.466 ms 2024-04-24T09:12:00.183 INFO:tasks.workunit.client.0.smithi083.stdout: 10 49070 6.70 MB/sec execute 271 sec latency 95.654 ms 2024-04-24T09:12:00.704 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:12:00 smithi138 ceph-mon[31441]: pgmap v642: 129 pgs: 129 active+clean; 792 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.1 MiB/s wr, 332 op/s 2024-04-24T09:12:00.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:12:00 smithi189 ceph-mon[31791]: pgmap v642: 129 pgs: 129 active+clean; 792 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.1 MiB/s wr, 332 op/s 2024-04-24T09:12:00.784 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.784+0000 7fb11de00640 1 -- 172.21.15.83:0/3288413623 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb118104470 msgr2=0x7fb118104850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:00.785 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.784+0000 7fb11de00640 1 --2- 172.21.15.83:0/3288413623 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb118104470 0x7fb118104850 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fb108009900 tx=0x7fb10802a3f0 comp rx=0 tx=0).stop 2024-04-24T09:12:00.785 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.784+0000 7fb11de00640 1 -- 172.21.15.83:0/3288413623 shutdown_connections 2024-04-24T09:12:00.785 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.784+0000 7fb11de00640 1 --2- 172.21.15.83:0/3288413623 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb118109540 0x7fb118111110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:00.785 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.784+0000 7fb11de00640 1 --2- 172.21.15.83:0/3288413623 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb118104e20 0x7fb118108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:00.785 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.784+0000 7fb11de00640 1 --2- 172.21.15.83:0/3288413623 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb118104470 0x7fb118104850 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:00.785 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.784+0000 7fb11de00640 1 -- 172.21.15.83:0/3288413623 >> 172.21.15.83:0/3288413623 conn(0x7fb1180ffcd0 msgr2=0x7fb1181020f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:00.785 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.784+0000 7fb11de00640 1 -- 172.21.15.83:0/3288413623 shutdown_connections 2024-04-24T09:12:00.785 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.784+0000 7fb11de00640 1 -- 172.21.15.83:0/3288413623 wait complete. 2024-04-24T09:12:00.786 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.785+0000 7fb11de00640 1 Processor -- start 2024-04-24T09:12:00.786 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.785+0000 7fb11de00640 1 -- start start 2024-04-24T09:12:00.786 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.786+0000 7fb11de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb118104470 0x7fb1181a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:00.786 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.786+0000 7fb11de00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb118109540 0x7fb1181a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:00.787 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.786+0000 7fb11de00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb11819a7a0 0x7fb11819ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:00.787 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.786+0000 7fb11de00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb118114500 con 0x7fb118104470 2024-04-24T09:12:00.787 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.786+0000 7fb11de00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb118114380 con 0x7fb11819a7a0 2024-04-24T09:12:00.787 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.786+0000 7fb11de00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb118114680 con 0x7fb118109540 2024-04-24T09:12:00.787 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.786+0000 7fb117400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb11819a7a0 0x7fb11819ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:00.787 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.786+0000 7fb116a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb118104470 0x7fb1181a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:00.787 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.786+0000 7fb117400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb11819a7a0 0x7fb11819ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:42414/0 (socket says 172.21.15.83:42414) 2024-04-24T09:12:00.787 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.786+0000 7fb117400640 1 -- 172.21.15.83:0/2255260834 learned_addr learned my addr 172.21.15.83:0/2255260834 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:12:00.787 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.786+0000 7fb116000640 1 --2- 172.21.15.83:0/2255260834 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb118109540 0x7fb1181a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:00.788 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.787+0000 7fb116a00640 1 -- 172.21.15.83:0/2255260834 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb118109540 msgr2=0x7fb1181a0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:00.788 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.787+0000 7fb116a00640 1 --2- 172.21.15.83:0/2255260834 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb118109540 0x7fb1181a0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:00.789 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.787+0000 7fb116a00640 1 -- 172.21.15.83:0/2255260834 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb11819a7a0 msgr2=0x7fb11819ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:00.789 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.787+0000 7fb116a00640 1 --2- 172.21.15.83:0/2255260834 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb11819a7a0 0x7fb11819ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:00.789 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.787+0000 7fb116a00640 1 -- 172.21.15.83:0/2255260834 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb11819b330 con 0x7fb118104470 2024-04-24T09:12:00.789 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.787+0000 7fb117400640 1 --2- 172.21.15.83:0/2255260834 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb11819a7a0 0x7fb11819ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:12:00.789 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.788+0000 7fb116a00640 1 --2- 172.21.15.83:0/2255260834 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb118104470 0x7fb1181a0630 secure :-1 s=READY pgs=2527 cs=0 l=1 rev1=1 crypto rx=0x7fb10802a9e0 tx=0x7fb108003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:00.789 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.788+0000 7fb11ca00640 1 -- 172.21.15.83:0/2255260834 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb10803f070 con 0x7fb118104470 2024-04-24T09:12:00.789 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.788+0000 7fb11de00640 1 -- 172.21.15.83:0/2255260834 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb11819b5c0 con 0x7fb118104470 2024-04-24T09:12:00.789 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.788+0000 7fb11ca00640 1 -- 172.21.15.83:0/2255260834 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb108031b50 con 0x7fb118104470 2024-04-24T09:12:00.789 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.788+0000 7fb11ca00640 1 -- 172.21.15.83:0/2255260834 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb108035030 con 0x7fb118104470 2024-04-24T09:12:00.789 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.788+0000 7fb11de00640 1 -- 172.21.15.83:0/2255260834 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1181a7770 con 0x7fb118104470 2024-04-24T09:12:00.793 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.793+0000 7fb11ca00640 1 -- 172.21.15.83:0/2255260834 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb108042070 con 0x7fb118104470 2024-04-24T09:12:00.795 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.795+0000 7fb11ca00640 1 --2- 172.21.15.83:0/2255260834 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb0fc0a3dc0 0x7fb0fc0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:00.796 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.795+0000 7fb11ca00640 1 -- 172.21.15.83:0/2255260834 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 9583+0+0 (secure 0 0 0) 0x7fb1080f0f00 con 0x7fb118104470 2024-04-24T09:12:00.796 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.796+0000 7fb116000640 1 --2- 172.21.15.83:0/2255260834 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb0fc0a3dc0 0x7fb0fc0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:00.797 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.797+0000 7fb116000640 1 --2- 172.21.15.83:0/2255260834 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb0fc0a3dc0 0x7fb0fc0a6280 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fb1040095f0 tx=0x7fb104005f90 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:00.802 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:12:00 smithi083 ceph-mon[27907]: pgmap v642: 129 pgs: 129 active+clean; 792 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.1 MiB/s wr, 332 op/s 2024-04-24T09:12:00.814 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.814+0000 7fb120c19740 1 Processor -- start 2024-04-24T09:12:00.815 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.815+0000 7fb120c19740 1 -- start start 2024-04-24T09:12:00.815 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.815+0000 7fb120c19740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622decc70f0 0x5622decc74d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:00.815 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.815+0000 7fb120c19740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622decc7a10 0x5622decda660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:00.815 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.815+0000 7fb120c19740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622decdaba0 0x5622decdcf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:00.815 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.815+0000 7fb120c19740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5622dec2d900 con 0x5622decdaba0 2024-04-24T09:12:00.816 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.815+0000 7fb120c19740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5622debd3f20 con 0x5622decc7a10 2024-04-24T09:12:00.816 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.815+0000 7fb120c19740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5622dec298c0 con 0x5622decc70f0 2024-04-24T09:12:00.816 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.816+0000 7fb117400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622decdaba0 0x5622decdcf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:00.816 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.816+0000 7fb116000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622decc7a10 0x5622decda660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:00.816 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.816+0000 7fb117400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622decdaba0 0x5622decdcf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50338/0 (socket says 172.21.15.83:50338) 2024-04-24T09:12:00.817 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.816+0000 7fb117400640 1 -- 172.21.15.83:0/689650624 learned_addr learned my addr 172.21.15.83:0/689650624 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:12:00.817 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.816+0000 7fb116a00640 1 --2- 172.21.15.83:0/689650624 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622decc70f0 0x5622decc74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:00.817 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.816+0000 7fb117400640 1 -- 172.21.15.83:0/689650624 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622decc70f0 msgr2=0x5622decc74d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:00.817 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.816+0000 7fb117400640 1 --2- 172.21.15.83:0/689650624 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622decc70f0 0x5622decc74d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:00.817 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.816+0000 7fb117400640 1 -- 172.21.15.83:0/689650624 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622decc7a10 msgr2=0x5622decda660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:00.817 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.816+0000 7fb117400640 1 --2- 172.21.15.83:0/689650624 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622decc7a10 0x5622decda660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:00.817 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.816+0000 7fb117400640 1 -- 172.21.15.83:0/689650624 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622debf8e10 con 0x5622decdaba0 2024-04-24T09:12:00.817 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.817+0000 7fb117400640 1 --2- 172.21.15.83:0/689650624 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622decdaba0 0x5622decdcf90 secure :-1 s=READY pgs=2528 cs=0 l=1 rev1=1 crypto rx=0x7fb110011780 tx=0x7fb110011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1192a4c51eb4c39f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:00.818 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.817+0000 7fb103e00640 1 -- 172.21.15.83:0/689650624 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb11000fb40 con 0x5622decdaba0 2024-04-24T09:12:00.818 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.817+0000 7fb103e00640 1 -- 172.21.15.83:0/689650624 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb110011e90 con 0x5622decdaba0 2024-04-24T09:12:00.818 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.817+0000 7fb103e00640 1 -- 172.21.15.83:0/689650624 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb110012e60 con 0x5622decdaba0 2024-04-24T09:12:00.818 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.818+0000 7fb120c19740 1 -- 172.21.15.83:0/689650624 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622decdaba0 msgr2=0x5622decdcf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:00.818 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.818+0000 7fb120c19740 1 --2- 172.21.15.83:0/689650624 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622decdaba0 0x5622decdcf90 secure :-1 s=READY pgs=2528 cs=0 l=1 rev1=1 crypto rx=0x7fb110011780 tx=0x7fb110011c40 comp rx=0 tx=0).stop 2024-04-24T09:12:00.819 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.818+0000 7fb120c19740 1 -- 172.21.15.83:0/689650624 shutdown_connections 2024-04-24T09:12:00.819 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.818+0000 7fb120c19740 1 --2- 172.21.15.83:0/689650624 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622decdaba0 0x5622decdcf90 unknown :-1 s=CLOSED pgs=2528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:00.819 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.818+0000 7fb120c19740 1 --2- 172.21.15.83:0/689650624 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622decc7a10 0x5622decda660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:00.819 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.818+0000 7fb120c19740 1 --2- 172.21.15.83:0/689650624 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622decc70f0 0x5622decc74d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:00.819 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.818+0000 7fb120c19740 1 -- 172.21.15.83:0/689650624 >> 172.21.15.83:0/689650624 conn(0x5622decc36c0 msgr2=0x5622decc3aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:00.820 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.819+0000 7fb120c19740 1 -- 172.21.15.83:0/689650624 shutdown_connections 2024-04-24T09:12:00.820 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.819+0000 7fb120c19740 1 -- 172.21.15.83:0/689650624 wait complete. 2024-04-24T09:12:00.820 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.819+0000 7fb120c19740 1 -- start start 2024-04-24T09:12:00.821 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.819+0000 7fb120c19740 1 Processor -- start 2024-04-24T09:12:00.821 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.820+0000 7fb120c19740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622decc70f0 0x5622decd7b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:00.821 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.820+0000 7fb120c19740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622decc7a10 0x5622decd8070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:00.822 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.820+0000 7fb120c19740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622decdaba0 0x5622decd85b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:00.822 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.820+0000 7fb120c19740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5622decdd770 con 0x5622decdaba0 2024-04-24T09:12:00.822 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.820+0000 7fb120c19740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5622debe72c0 con 0x5622decc70f0 2024-04-24T09:12:00.822 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.820+0000 7fb120c19740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5622decdd8f0 con 0x5622decc7a10 2024-04-24T09:12:00.822 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.820+0000 7fb120c19740 10 client.0 _ll_register_callbacks cb 0x5622de93fa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:12:00.822 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.821+0000 7fb116a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622decc70f0 0x5622decd7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:00.822 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.821+0000 7fb116a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622decc70f0 0x5622decd7b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:42444/0 (socket says 172.21.15.83:42444) 2024-04-24T09:12:00.822 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.821+0000 7fb116a00640 1 -- 172.21.15.83:0/194129082 learned_addr learned my addr 172.21.15.83:0/194129082 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:12:00.822 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.821+0000 7fb117400640 1 --2- 172.21.15.83:0/194129082 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622decdaba0 0x5622decd85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:00.822 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.821+0000 7fb116000640 1 --2- 172.21.15.83:0/194129082 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622decc7a10 0x5622decd8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:00.822 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.821+0000 7fb116a00640 1 -- 172.21.15.83:0/194129082 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622decc7a10 msgr2=0x5622decd8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:00.822 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.821+0000 7fb116a00640 1 --2- 172.21.15.83:0/194129082 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622decc7a10 0x5622decd8070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:00.823 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.821+0000 7fb116a00640 1 -- 172.21.15.83:0/194129082 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622decdaba0 msgr2=0x5622decd85b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:00.823 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.821+0000 7fb116a00640 1 --2- 172.21.15.83:0/194129082 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622decdaba0 0x5622decd85b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:00.823 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.821+0000 7fb116a00640 1 -- 172.21.15.83:0/194129082 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622debf7fd0 con 0x5622decc70f0 2024-04-24T09:12:00.824 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.821+0000 7fb116a00640 1 --2- 172.21.15.83:0/194129082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622decc70f0 0x5622decd7b30 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fb1080bc7d0 tx=0x7fb1080bc430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3be0ec736804349b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:00.824 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.821+0000 7fb100c00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:12:00.824 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.822+0000 7fb100c00640 1 -- 172.21.15.83:0/194129082 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb108049040 con 0x5622decc70f0 2024-04-24T09:12:00.824 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.822+0000 7fb120c19740 1 -- 172.21.15.83:0/194129082 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb110011e90 con 0x5622decc70f0 2024-04-24T09:12:00.824 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.822+0000 7fb100c00640 1 -- 172.21.15.83:0/194129082 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 9583+0+0 (secure 0 0 0) 0x7fb1080b9140 con 0x5622decc70f0 2024-04-24T09:12:00.825 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.823+0000 7fb100c00640 1 -- 172.21.15.83:0/194129082 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7fb1080c5690 con 0x5622decc70f0 2024-04-24T09:12:00.825 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.823+0000 7fb120c19740 10 client.39635 fetch_fsmap learned FSMap version 61 2024-04-24T09:12:00.825 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.823+0000 7fb120c19740 1 -- 172.21.15.83:0/194129082 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x5622de940210 con 0x5622decc70f0 2024-04-24T09:12:00.825 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.823+0000 7fb100c00640 1 -- 172.21.15.83:0/194129082 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb110011e90 con 0x5622decc70f0 2024-04-24T09:12:00.825 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.823+0000 7fb100c00640 1 -- 172.21.15.83:0/194129082 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 61) ==== 8362+0+0 (secure 0 0 0) 0x7fb10804c060 con 0x5622decc70f0 2024-04-24T09:12:00.825 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.824+0000 7fb120c19740 10 client.39635 fetch_fsmap finished waiting for FSMap version 61 2024-04-24T09:12:00.825 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.824+0000 7fb120c19740 10 client.39635 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:12:00.825 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.824+0000 7fb120c19740 20 client.39635 populate_metadata read hostname 'smithi083' 2024-04-24T09:12:00.825 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.824+0000 7fb120c19740 1 --2- 172.21.15.83:0/194129082 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5622decead80 0x5622deced1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:00.825 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.824+0000 7fb120c19740 4 client.39635 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:12:00.825 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.824+0000 7fb120c19740 1 -- 172.21.15.83:0/194129082 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5622dec3b320 con 0x5622decead80 2024-04-24T09:12:00.826 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.824+0000 7fb116000640 1 --2- 172.21.15.83:0/194129082 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5622decead80 0x5622deced1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:00.827 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.825+0000 7fb100c00640 10 client.39635 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:12:00.827 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.825+0000 7fb116000640 1 --2- 172.21.15.83:0/194129082 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5622decead80 0x5622deced1e0 crc :-1 s=READY pgs=505 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=786b142c9a485e20 server_cookie=ee7331e6dfd96880 in_seq=0 out_seq=0 2024-04-24T09:12:00.827 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.826+0000 7fb100c00640 1 -- 172.21.15.83:0/194129082 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5622dec3b320 con 0x5622decead80 2024-04-24T09:12:00.827 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.826+0000 7fb100c00640 10 client.39635 handle_command_reply: tid=0 2024-04-24T09:12:00.827 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.826+0000 7fb120c19740 1 client.39635 shutdown 2024-04-24T09:12:00.827 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.826+0000 7fb120c19740 1 -- 172.21.15.83:0/194129082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622decc70f0 msgr2=0x5622decd7b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:00.827 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.826+0000 7fb120c19740 1 --2- 172.21.15.83:0/194129082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622decc70f0 0x5622decd7b30 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fb1080bc7d0 tx=0x7fb1080bc430 comp rx=0 tx=0).stop 2024-04-24T09:12:00.827 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.826+0000 7fb120c19740 1 -- 172.21.15.83:0/194129082 shutdown_connections 2024-04-24T09:12:00.827 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.826+0000 7fb120c19740 1 --2- 172.21.15.83:0/194129082 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5622decead80 0x5622deced1e0 crc :-1 s=READY pgs=505 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:00.827 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.826+0000 7fb120c19740 1 --2- 172.21.15.83:0/194129082 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622decdaba0 0x5622decd85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:00.828 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.826+0000 7fb120c19740 1 --2- 172.21.15.83:0/194129082 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622decc7a10 0x5622decd8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:00.828 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.826+0000 7fb120c19740 1 --2- 172.21.15.83:0/194129082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622decc70f0 0x5622decd7b30 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:00.828 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.826+0000 7fb120c19740 1 -- 172.21.15.83:0/194129082 >> 172.21.15.83:0/194129082 conn(0x5622decc36c0 msgr2=0x5622decc9440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:00.828 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.826+0000 7fb100c00640 0 client.39635 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:12:00.828 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.826+0000 7fb120c19740 1 -- 172.21.15.83:0/194129082 shutdown_connections 2024-04-24T09:12:00.828 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.826+0000 7fb120c19740 1 -- 172.21.15.83:0/194129082 wait complete. 2024-04-24T09:12:00.828 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.827+0000 7fb120c19740 20 client.39635 trim_cache size 0 max 16384 2024-04-24T09:12:00.855 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.855+0000 7fb120c19740 1 Processor -- start 2024-04-24T09:12:00.856 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.855+0000 7fb120c19740 1 -- start start 2024-04-24T09:12:00.857 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.856+0000 7fb120c19740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622decc70f0 0x5622dece6670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:00.857 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.856+0000 7fb120c19740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622decc7a10 0x5622decf2af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:00.857 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.856+0000 7fb120c19740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622decdaba0 0x5622decf3030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:00.857 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.856+0000 7fb120c19740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb1181ad290 con 0x5622decc70f0 2024-04-24T09:12:00.857 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.856+0000 7fb120c19740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb1181aedd0 con 0x5622decc7a10 2024-04-24T09:12:00.858 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.856+0000 7fb120c19740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb1181ad960 con 0x5622decdaba0 2024-04-24T09:12:00.858 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.856+0000 7fb116a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622decc70f0 0x5622dece6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:00.858 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.856+0000 7fb116a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622decc70f0 0x5622dece6670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50370/0 (socket says 172.21.15.83:50370) 2024-04-24T09:12:00.858 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.856+0000 7fb116a00640 1 -- 172.21.15.83:0/2138856794 learned_addr learned my addr 172.21.15.83:0/2138856794 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:12:00.858 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.856+0000 7fb116000640 1 --2- 172.21.15.83:0/2138856794 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622decc7a10 0x5622decf2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:00.858 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.857+0000 7fb117400640 1 --2- 172.21.15.83:0/2138856794 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622decdaba0 0x5622decf3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:00.858 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.857+0000 7fb116a00640 1 -- 172.21.15.83:0/2138856794 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622decdaba0 msgr2=0x5622decf3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:00.858 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.857+0000 7fb116a00640 1 --2- 172.21.15.83:0/2138856794 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622decdaba0 0x5622decf3030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:00.858 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.857+0000 7fb116a00640 1 -- 172.21.15.83:0/2138856794 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622decc7a10 msgr2=0x5622decf2af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:00.858 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.857+0000 7fb116a00640 1 --2- 172.21.15.83:0/2138856794 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622decc7a10 0x5622decf2af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:00.859 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.857+0000 7fb116a00640 1 -- 172.21.15.83:0/2138856794 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622decd2970 con 0x5622decc70f0 2024-04-24T09:12:00.859 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.857+0000 7fb116a00640 1 --2- 172.21.15.83:0/2138856794 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622decc70f0 0x5622dece6670 secure :-1 s=READY pgs=2529 cs=0 l=1 rev1=1 crypto rx=0x7fb10802a9b0 tx=0x7fb1080c0f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26eb58a02e622978 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:00.859 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.857+0000 7fb103400640 1 -- 172.21.15.83:0/2138856794 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb108049050 con 0x5622decc70f0 2024-04-24T09:12:00.860 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.857+0000 7fb103400640 1 -- 172.21.15.83:0/2138856794 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb1080bcad0 con 0x5622decc70f0 2024-04-24T09:12:00.860 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.857+0000 7fb103400640 1 -- 172.21.15.83:0/2138856794 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1080bbf20 con 0x5622decc70f0 2024-04-24T09:12:00.860 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.858+0000 7fb120c19740 1 -- 172.21.15.83:0/2138856794 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622decc70f0 msgr2=0x5622dece6670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:00.860 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.858+0000 7fb120c19740 1 --2- 172.21.15.83:0/2138856794 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622decc70f0 0x5622dece6670 secure :-1 s=READY pgs=2529 cs=0 l=1 rev1=1 crypto rx=0x7fb10802a9b0 tx=0x7fb1080c0f50 comp rx=0 tx=0).stop 2024-04-24T09:12:00.860 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.859+0000 7fb120c19740 1 -- 172.21.15.83:0/2138856794 shutdown_connections 2024-04-24T09:12:00.860 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.859+0000 7fb120c19740 1 --2- 172.21.15.83:0/2138856794 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622decdaba0 0x5622decf3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:00.860 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.859+0000 7fb120c19740 1 --2- 172.21.15.83:0/2138856794 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622decc7a10 0x5622decf2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:00.860 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.859+0000 7fb120c19740 1 --2- 172.21.15.83:0/2138856794 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622decc70f0 0x5622dece6670 unknown :-1 s=CLOSED pgs=2529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:00.860 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.859+0000 7fb120c19740 1 -- 172.21.15.83:0/2138856794 >> 172.21.15.83:0/2138856794 conn(0x5622decc36c0 msgr2=0x5622decc2840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:00.860 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.859+0000 7fb120c19740 1 -- 172.21.15.83:0/2138856794 shutdown_connections 2024-04-24T09:12:00.860 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.859+0000 7fb120c19740 1 -- 172.21.15.83:0/2138856794 wait complete. 2024-04-24T09:12:00.860 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.859+0000 7fb120c19740 1 -- start start 2024-04-24T09:12:00.860 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.859+0000 7fb120c19740 1 Processor -- start 2024-04-24T09:12:00.862 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.860+0000 7fb120c19740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622decc70f0 0x5622decd4a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:00.862 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.860+0000 7fb120c19740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622decc7a10 0x5622decd4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:00.862 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.860+0000 7fb120c19740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622decdaba0 0x5622decd5510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:00.862 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.860+0000 7fb120c19740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb1181ae700 con 0x5622decc7a10 2024-04-24T09:12:00.862 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.860+0000 7fb120c19740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb1181ae030 con 0x5622decdaba0 2024-04-24T09:12:00.862 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.860+0000 7fb120c19740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb1181ab290 con 0x5622decc70f0 2024-04-24T09:12:00.863 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.860+0000 7fb120c19740 10 client.0 _ll_register_callbacks cb 0x7fb108005b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:12:00.863 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.860+0000 7fb116000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622decc7a10 0x5622decd4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:00.863 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.861+0000 7fb116000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622decc7a10 0x5622decd4fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50374/0 (socket says 172.21.15.83:50374) 2024-04-24T09:12:00.863 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.861+0000 7fb117400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622decdaba0 0x5622decd5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:00.863 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.861+0000 7fb116000640 1 -- 172.21.15.83:0/1072001303 learned_addr learned my addr 172.21.15.83:0/1072001303 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:12:00.863 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.861+0000 7fb116a00640 1 --2- 172.21.15.83:0/1072001303 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622decc70f0 0x5622decd4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:00.863 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.861+0000 7fb116000640 1 -- 172.21.15.83:0/1072001303 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622decc70f0 msgr2=0x5622decd4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:00.863 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.861+0000 7fb116000640 1 --2- 172.21.15.83:0/1072001303 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622decc70f0 0x5622decd4a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:00.863 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.861+0000 7fb116000640 1 -- 172.21.15.83:0/1072001303 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622decdaba0 msgr2=0x5622decd5510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:00.863 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.861+0000 7fb116000640 1 --2- 172.21.15.83:0/1072001303 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622decdaba0 0x5622decd5510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:00.863 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.861+0000 7fb116000640 1 -- 172.21.15.83:0/1072001303 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622dea0c620 con 0x5622decc7a10 2024-04-24T09:12:00.863 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.861+0000 7fb117400640 1 --2- 172.21.15.83:0/1072001303 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622decdaba0 0x5622decd5510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:12:00.863 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.861+0000 7fb116a00640 1 --2- 172.21.15.83:0/1072001303 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622decc70f0 0x5622decd4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:12:00.863 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.862+0000 7fb116000640 1 --2- 172.21.15.83:0/1072001303 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622decc7a10 0x5622decd4fd0 secure :-1 s=READY pgs=2530 cs=0 l=1 rev1=1 crypto rx=0x7fb11810aae0 tx=0x7fb104004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcc823927c110eae server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:00.864 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.862+0000 7fb100c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:12:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.862+0000 7fb100c00640 1 -- 172.21.15.83:0/1072001303 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1040044c0 con 0x5622decc7a10 2024-04-24T09:12:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.862+0000 7fb100c00640 1 -- 172.21.15.83:0/1072001303 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 9583+0+0 (secure 0 0 0) 0x7fb10400e870 con 0x5622decc7a10 2024-04-24T09:12:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.862+0000 7fb120c19740 1 -- 172.21.15.83:0/1072001303 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1080bcad0 con 0x5622decc7a10 2024-04-24T09:12:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.863+0000 7fb100c00640 1 -- 172.21.15.83:0/1072001303 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7fb10400eab0 con 0x5622decc7a10 2024-04-24T09:12:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.863+0000 7fb120c19740 10 client.22596 fetch_fsmap learned FSMap version 61 2024-04-24T09:12:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.863+0000 7fb120c19740 1 -- 172.21.15.83:0/1072001303 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x5622ded51780 con 0x5622decc7a10 2024-04-24T09:12:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.863+0000 7fb100c00640 1 -- 172.21.15.83:0/1072001303 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb1080bcad0 con 0x5622decc7a10 2024-04-24T09:12:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.863+0000 7fb100c00640 1 -- 172.21.15.83:0/1072001303 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 61) ==== 8362+0+0 (secure 0 0 0) 0x7fb10401d410 con 0x5622decc7a10 2024-04-24T09:12:00.866 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.863+0000 7fb120c19740 10 client.22596 fetch_fsmap finished waiting for FSMap version 61 2024-04-24T09:12:00.866 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.863+0000 7fb120c19740 10 client.22596 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:12:00.866 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.863+0000 7fb120c19740 20 client.22596 populate_metadata read hostname 'smithi083' 2024-04-24T09:12:00.866 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.863+0000 7fb120c19740 1 --2- 172.21.15.83:0/1072001303 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5622ded51f50 0x5622ded54370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:00.866 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.863+0000 7fb120c19740 4 client.22596 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T09:12:00.866 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.863+0000 7fb120c19740 1 -- 172.21.15.83:0/1072001303 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5622dec35190 con 0x5622ded51f50 2024-04-24T09:12:00.866 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.864+0000 7fb116a00640 1 --2- 172.21.15.83:0/1072001303 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5622ded51f50 0x5622ded54370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:00.866 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.864+0000 7fb116a00640 1 --2- 172.21.15.83:0/1072001303 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5622ded51f50 0x5622ded54370 crc :-1 s=READY pgs=506 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b589a7f36abeee4a server_cookie=4e7dba11068fd306 in_seq=0 out_seq=0 2024-04-24T09:12:00.866 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:00.865+0000 7fb100c00640 10 client.22596 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:12:01.037 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:01.037+0000 7fb100c00640 1 -- 172.21.15.83:0/1072001303 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5622dec35190 con 0x5622ded51f50 2024-04-24T09:12:01.038 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:01.037+0000 7fb100c00640 10 client.22596 handle_command_reply: tid=0 2024-04-24T09:12:01.038 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:01.037+0000 7fb120c19740 1 client.22596 shutdown 2024-04-24T09:12:01.038 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:01.037+0000 7fb120c19740 1 -- 172.21.15.83:0/1072001303 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622decc7a10 msgr2=0x5622decd4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:01.038 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:01.037+0000 7fb120c19740 1 --2- 172.21.15.83:0/1072001303 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622decc7a10 0x5622decd4fd0 secure :-1 s=READY pgs=2530 cs=0 l=1 rev1=1 crypto rx=0x7fb11810aae0 tx=0x7fb104004730 comp rx=0 tx=0).stop 2024-04-24T09:12:01.039 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:01.038+0000 7fb120c19740 1 -- 172.21.15.83:0/1072001303 shutdown_connections 2024-04-24T09:12:01.039 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:01.038+0000 7fb120c19740 1 --2- 172.21.15.83:0/1072001303 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5622ded51f50 0x5622ded54370 crc :-1 s=READY pgs=506 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:01.040 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:01.038+0000 7fb120c19740 1 --2- 172.21.15.83:0/1072001303 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5622decdaba0 0x5622decd5510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:01.040 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:01.038+0000 7fb120c19740 1 --2- 172.21.15.83:0/1072001303 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5622decc7a10 0x5622decd4fd0 unknown :-1 s=CLOSED pgs=2530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:01.040 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:01.038+0000 7fb120c19740 1 --2- 172.21.15.83:0/1072001303 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622decc70f0 0x5622decd4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:01.040 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:01.038+0000 7fb120c19740 1 -- 172.21.15.83:0/1072001303 >> 172.21.15.83:0/1072001303 conn(0x5622decc36c0 msgr2=0x5622decd8360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:01.040 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:01.038+0000 7fb100c00640 0 client.22596 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:12:01.040 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:01.038+0000 7fb120c19740 1 -- 172.21.15.83:0/1072001303 shutdown_connections 2024-04-24T09:12:01.040 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:01.038+0000 7fb120c19740 1 -- 172.21.15.83:0/1072001303 wait complete. 2024-04-24T09:12:01.040 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:01.038+0000 7fb120c19740 20 client.22596 trim_cache size 0 max 16384 2024-04-24T09:12:01.041 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T09:12:01.041 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T09:12:01.041 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T09:12:01.041 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T09:12:01.042 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:01.039+0000 7fb11de00640 1 -- 172.21.15.83:0/2255260834 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb0fc0a3dc0 msgr2=0x7fb0fc0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:01.042 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:01.039+0000 7fb11de00640 1 --2- 172.21.15.83:0/2255260834 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb0fc0a3dc0 0x7fb0fc0a6280 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fb1040095f0 tx=0x7fb104005f90 comp rx=0 tx=0).stop 2024-04-24T09:12:01.042 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:01.039+0000 7fb11de00640 1 -- 172.21.15.83:0/2255260834 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb118104470 msgr2=0x7fb1181a0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:01.042 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:01.039+0000 7fb11de00640 1 --2- 172.21.15.83:0/2255260834 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb118104470 0x7fb1181a0630 secure :-1 s=READY pgs=2527 cs=0 l=1 rev1=1 crypto rx=0x7fb10802a9e0 tx=0x7fb108003b90 comp rx=0 tx=0).stop 2024-04-24T09:12:01.042 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:01.039+0000 7fb11de00640 1 -- 172.21.15.83:0/2255260834 shutdown_connections 2024-04-24T09:12:01.042 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:01.039+0000 7fb11de00640 1 --2- 172.21.15.83:0/2255260834 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb0fc0a3dc0 0x7fb0fc0a6280 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:01.042 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:01.039+0000 7fb11de00640 1 --2- 172.21.15.83:0/2255260834 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb11819a7a0 0x7fb11819ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:01.042 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:01.039+0000 7fb11de00640 1 --2- 172.21.15.83:0/2255260834 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb118109540 0x7fb1181a0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:01.042 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:01.039+0000 7fb11de00640 1 --2- 172.21.15.83:0/2255260834 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb118104470 0x7fb1181a0630 unknown :-1 s=CLOSED pgs=2527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:01.042 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:01.039+0000 7fb11de00640 1 -- 172.21.15.83:0/2255260834 >> 172.21.15.83:0/2255260834 conn(0x7fb1180ffcd0 msgr2=0x7fb118100850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:01.042 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:01.040+0000 7fb11de00640 1 -- 172.21.15.83:0/2255260834 shutdown_connections 2024-04-24T09:12:01.042 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:01.040+0000 7fb11de00640 1 -- 172.21.15.83:0/2255260834 wait complete. 2024-04-24T09:12:01.184 INFO:tasks.workunit.client.0.smithi083.stdout: 10 49160 6.68 MB/sec execute 272 sec latency 113.442 ms 2024-04-24T09:12:02.184 INFO:tasks.workunit.client.0.smithi083.stdout: 10 49218 6.66 MB/sec execute 273 sec latency 444.652 ms 2024-04-24T09:12:02.704 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:12:02 smithi138 ceph-mon[31441]: pgmap v643: 129 pgs: 129 active+clean; 792 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.4 MiB/s wr, 192 op/s 2024-04-24T09:12:02.706 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:12:02 smithi189 ceph-mon[31791]: pgmap v643: 129 pgs: 129 active+clean; 792 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.4 MiB/s wr, 192 op/s 2024-04-24T09:12:02.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:12:02 smithi083 ceph-mon[27907]: pgmap v643: 129 pgs: 129 active+clean; 792 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.4 MiB/s wr, 192 op/s 2024-04-24T09:12:03.184 INFO:tasks.workunit.client.0.smithi083.stdout: 10 49319 6.66 MB/sec execute 274 sec latency 110.590 ms 2024-04-24T09:12:03.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:12:03 smithi138 ceph-mon[31441]: osdmap e123: 12 total, 12 up, 12 in 2024-04-24T09:12:03.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:12:03 smithi189 ceph-mon[31791]: osdmap e123: 12 total, 12 up, 12 in 2024-04-24T09:12:03.802 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:12:03 smithi083 ceph-mon[27907]: osdmap e123: 12 total, 12 up, 12 in 2024-04-24T09:12:04.184 INFO:tasks.workunit.client.0.smithi083.stdout: 10 49474 6.68 MB/sec execute 275 sec latency 73.276 ms 2024-04-24T09:12:04.704 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:12:04 smithi138 ceph-mon[31441]: pgmap v645: 129 pgs: 15 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 76 active+clean; 780 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.6 MiB/s wr, 281 op/s 2024-04-24T09:12:04.706 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:12:04 smithi189 ceph-mon[31791]: pgmap v645: 129 pgs: 15 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 76 active+clean; 780 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.6 MiB/s wr, 281 op/s 2024-04-24T09:12:04.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:12:04 smithi083 ceph-mon[27907]: pgmap v645: 129 pgs: 15 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 76 active+clean; 780 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.6 MiB/s wr, 281 op/s 2024-04-24T09:12:05.184 INFO:tasks.workunit.client.0.smithi083.stdout: 10 49621 6.68 MB/sec execute 276 sec latency 83.680 ms 2024-04-24T09:12:06.184 INFO:tasks.workunit.client.0.smithi083.stdout: 10 49730 6.67 MB/sec execute 277 sec latency 82.070 ms 2024-04-24T09:12:06.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:12:06 smithi138 ceph-mon[31441]: pgmap v646: 129 pgs: 15 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 76 active+clean; 780 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.6 MiB/s wr, 281 op/s 2024-04-24T09:12:06.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:12:06 smithi189 ceph-mon[31791]: pgmap v646: 129 pgs: 15 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 76 active+clean; 780 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.6 MiB/s wr, 281 op/s 2024-04-24T09:12:06.758 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T09:12:06.759 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T09:12:07.184 INFO:tasks.workunit.client.0.smithi083.stdout: 10 49801 6.66 MB/sec execute 278 sec latency 447.714 ms 2024-04-24T09:12:07.287 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:12:06 smithi083 ceph-mon[27907]: pgmap v646: 129 pgs: 15 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 76 active+clean; 780 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.6 MiB/s wr, 281 op/s 2024-04-24T09:12:07.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.892+0000 7f0301600640 1 -- 172.21.15.83:0/743466345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02fc109210 msgr2=0x7f02fc1095f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:07.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.892+0000 7f0301600640 1 --2- 172.21.15.83:0/743466345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02fc109210 0x7f02fc1095f0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f02ec009900 tx=0x7f02ec02a3f0 comp rx=0 tx=0).stop 2024-04-24T09:12:07.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.893+0000 7f0301600640 1 -- 172.21.15.83:0/743466345 shutdown_connections 2024-04-24T09:12:07.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.893+0000 7f0301600640 1 --2- 172.21.15.83:0/743466345 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f02fc115150 0x7f02fc117540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:07.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.893+0000 7f0301600640 1 --2- 172.21.15.83:0/743466345 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f02fc109b30 0x7f02fc114c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:07.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.893+0000 7f0301600640 1 --2- 172.21.15.83:0/743466345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02fc109210 0x7f02fc1095f0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:07.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.893+0000 7f0301600640 1 -- 172.21.15.83:0/743466345 >> 172.21.15.83:0/743466345 conn(0x7f02fc06d110 msgr2=0x7f02fc06d520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:07.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.893+0000 7f0301600640 1 -- 172.21.15.83:0/743466345 shutdown_connections 2024-04-24T09:12:07.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.893+0000 7f0301600640 1 -- 172.21.15.83:0/743466345 wait complete. 2024-04-24T09:12:07.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.894+0000 7f0301600640 1 Processor -- start 2024-04-24T09:12:07.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.894+0000 7f0301600640 1 -- start start 2024-04-24T09:12:07.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.895+0000 7f0301600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f02fc109210 0x7f02fc1a3320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:07.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.895+0000 7f0301600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02fc109b30 0x7f02fc1a3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:07.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.895+0000 7f0301600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f02fc115150 0x7f02fc1a7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:07.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.895+0000 7f0301600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f02fc10c700 con 0x7f02fc115150 2024-04-24T09:12:07.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.895+0000 7f0301600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f02fc10c580 con 0x7f02fc109b30 2024-04-24T09:12:07.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.895+0000 7f0301600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f02fc10c880 con 0x7f02fc109210 2024-04-24T09:12:07.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.895+0000 7f02faa00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f02fc109210 0x7f02fc1a3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:07.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.895+0000 7f02faa00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f02fc109210 0x7f02fc1a3320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:56306/0 (socket says 172.21.15.83:56306) 2024-04-24T09:12:07.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.895+0000 7f02faa00640 1 -- 172.21.15.83:0/3481622467 learned_addr learned my addr 172.21.15.83:0/3481622467 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:12:07.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.895+0000 7f02fb400640 1 --2- 172.21.15.83:0/3481622467 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f02fc115150 0x7f02fc1a7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:07.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.895+0000 7f02fa000640 1 --2- 172.21.15.83:0/3481622467 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02fc109b30 0x7f02fc1a3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:07.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.896+0000 7f02faa00640 1 -- 172.21.15.83:0/3481622467 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02fc109b30 msgr2=0x7f02fc1a3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:07.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.896+0000 7f02faa00640 1 --2- 172.21.15.83:0/3481622467 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02fc109b30 0x7f02fc1a3860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:07.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.896+0000 7f02faa00640 1 -- 172.21.15.83:0/3481622467 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f02fc115150 msgr2=0x7f02fc1a7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:07.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.896+0000 7f02faa00640 1 --2- 172.21.15.83:0/3481622467 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f02fc115150 0x7f02fc1a7be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:07.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.896+0000 7f02faa00640 1 -- 172.21.15.83:0/3481622467 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02fc1a8310 con 0x7f02fc109210 2024-04-24T09:12:07.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.897+0000 7f02faa00640 1 --2- 172.21.15.83:0/3481622467 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f02fc109210 0x7f02fc1a3320 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f02ec005ab0 tx=0x7f02ec003cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:07.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.897+0000 7f02f3400640 1 -- 172.21.15.83:0/3481622467 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02ec03f070 con 0x7f02fc109210 2024-04-24T09:12:07.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.897+0000 7f0301600640 1 -- 172.21.15.83:0/3481622467 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02fc1a85a0 con 0x7f02fc109210 2024-04-24T09:12:07.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.897+0000 7f02f3400640 1 -- 172.21.15.83:0/3481622467 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f02ec031d60 con 0x7f02fc109210 2024-04-24T09:12:07.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.897+0000 7f02f3400640 1 -- 172.21.15.83:0/3481622467 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02ec035030 con 0x7f02fc109210 2024-04-24T09:12:07.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.898+0000 7f0301600640 1 -- 172.21.15.83:0/3481622467 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02fc118de0 con 0x7f02fc109210 2024-04-24T09:12:07.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.904+0000 7f02f3400640 1 -- 172.21.15.83:0/3481622467 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f02ec042070 con 0x7f02fc109210 2024-04-24T09:12:07.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.904+0000 7f03043ab740 1 Processor -- start 2024-04-24T09:12:07.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.904+0000 7f03043ab740 1 -- start start 2024-04-24T09:12:07.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.905+0000 7f03043ab740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627987d2860 0x5627987d2c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:07.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.905+0000 7f03043ab740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5627987d3180 0x5627987dd310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:07.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.905+0000 7f03043ab740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5627987dd850 0x5627987dfc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:07.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.905+0000 7f03043ab740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5627987381f0 con 0x5627987dd850 2024-04-24T09:12:07.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.905+0000 7f03043ab740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5627986de350 con 0x5627987d2860 2024-04-24T09:12:07.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.905+0000 7f03043ab740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5627987341b0 con 0x5627987d3180 2024-04-24T09:12:07.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.906+0000 7f02fb400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627987d2860 0x5627987d2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:07.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.906+0000 7f02fb400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627987d2860 0x5627987d2c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:42484/0 (socket says 172.21.15.83:42484) 2024-04-24T09:12:07.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.906+0000 7f02fb400640 1 -- 172.21.15.83:0/2565483473 learned_addr learned my addr 172.21.15.83:0/2565483473 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:12:07.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.906+0000 7f02fa000640 1 --2- 172.21.15.83:0/2565483473 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5627987dd850 0x5627987dfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:07.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.907+0000 7f02fb400640 1 -- 172.21.15.83:0/2565483473 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5627987d3180 msgr2=0x5627987dd310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:07.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.907+0000 7f02fb400640 1 --2- 172.21.15.83:0/2565483473 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5627987d3180 0x5627987dd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:07.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.907+0000 7f02fb400640 1 -- 172.21.15.83:0/2565483473 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5627987dd850 msgr2=0x5627987dfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:07.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.907+0000 7f02fb400640 1 --2- 172.21.15.83:0/2565483473 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5627987dd850 0x5627987dfc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:07.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.907+0000 7f02fb400640 1 -- 172.21.15.83:0/2565483473 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562798732f50 con 0x5627987d2860 2024-04-24T09:12:07.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.907+0000 7f02fa000640 1 --2- 172.21.15.83:0/2565483473 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5627987dd850 0x5627987dfc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:12:07.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.907+0000 7f02fb400640 1 --2- 172.21.15.83:0/2565483473 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627987d2860 0x5627987d2c40 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f02f40039f0 tx=0x7f02f4003eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e703228761b0325 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:07.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.908+0000 7f02e7400640 1 -- 172.21.15.83:0/2565483473 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f401a470 con 0x5627987d2860 2024-04-24T09:12:07.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.909+0000 7f02e7400640 1 -- 172.21.15.83:0/2565483473 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f02f4015070 con 0x5627987d2860 2024-04-24T09:12:07.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.909+0000 7f02f3400640 1 --2- 172.21.15.83:0/3481622467 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f02e00a3d20 0x7f02e00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:07.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.909+0000 7f02e7400640 1 -- 172.21.15.83:0/2565483473 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f40107e0 con 0x5627987d2860 2024-04-24T09:12:07.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.909+0000 7f02f3400640 1 -- 172.21.15.83:0/3481622467 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 9671+0+0 (secure 0 0 0) 0x7f02ec0f0fb0 con 0x7f02fc109210 2024-04-24T09:12:07.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.909+0000 7f03043ab740 1 -- 172.21.15.83:0/2565483473 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627987d2860 msgr2=0x5627987d2c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:07.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.909+0000 7f03043ab740 1 --2- 172.21.15.83:0/2565483473 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627987d2860 0x5627987d2c40 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f02f40039f0 tx=0x7f02f4003eb0 comp rx=0 tx=0).stop 2024-04-24T09:12:07.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.910+0000 7f03043ab740 1 -- 172.21.15.83:0/2565483473 shutdown_connections 2024-04-24T09:12:07.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.910+0000 7f03043ab740 1 --2- 172.21.15.83:0/2565483473 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5627987dd850 0x5627987dfc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:07.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.910+0000 7f03043ab740 1 --2- 172.21.15.83:0/2565483473 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5627987d3180 0x5627987dd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:07.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.910+0000 7f03043ab740 1 --2- 172.21.15.83:0/2565483473 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627987d2860 0x5627987d2c40 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:07.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.910+0000 7f03043ab740 1 -- 172.21.15.83:0/2565483473 >> 172.21.15.83:0/2565483473 conn(0x5627987ceb80 msgr2=0x5627987cef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:07.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.912+0000 7f02faa00640 1 --2- 172.21.15.83:0/3481622467 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f02e00a3d20 0x7f02e00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:07.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.913+0000 7f02faa00640 1 --2- 172.21.15.83:0/3481622467 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f02e00a3d20 0x7f02e00a61e0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x5627987d3e20 tx=0x7f02ec02a3f0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:07.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.913+0000 7f03043ab740 1 -- 172.21.15.83:0/2565483473 shutdown_connections 2024-04-24T09:12:07.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.914+0000 7f03043ab740 1 -- 172.21.15.83:0/2565483473 wait complete. 2024-04-24T09:12:07.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.914+0000 7f03043ab740 1 -- start start 2024-04-24T09:12:07.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.914+0000 7f03043ab740 1 Processor -- start 2024-04-24T09:12:07.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.915+0000 7f03043ab740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5627987d2860 0x5627987da7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:07.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.916+0000 7f02fb400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5627987d2860 0x5627987da7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:07.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.916+0000 7f03043ab740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627987d3180 0x5627987dad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:07.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.916+0000 7f03043ab740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5627987dd850 0x5627987db240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:07.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.916+0000 7f03043ab740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5627987e0640 con 0x5627987d2860 2024-04-24T09:12:07.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.916+0000 7f03043ab740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5627986f1690 con 0x5627987d3180 2024-04-24T09:12:07.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.916+0000 7f03043ab740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5627987e07c0 con 0x5627987dd850 2024-04-24T09:12:07.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.916+0000 7f03043ab740 10 client.0 _ll_register_callbacks cb 0x56279872d650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:12:07.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.916+0000 7f02fb400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5627987d2860 0x5627987da7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50422/0 (socket says 172.21.15.83:50422) 2024-04-24T09:12:07.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.916+0000 7f02fb400640 1 -- 172.21.15.83:0/1864269209 learned_addr learned my addr 172.21.15.83:0/1864269209 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:12:07.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.917+0000 7f02fa000640 1 --2- 172.21.15.83:0/1864269209 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627987d3180 0x5627987dad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:07.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.917+0000 7f02fb400640 1 -- 172.21.15.83:0/1864269209 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5627987dd850 msgr2=0x5627987db240 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T09:12:07.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.917+0000 7f02fb400640 1 --2- 172.21.15.83:0/1864269209 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5627987dd850 0x5627987db240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:07.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.917+0000 7f02fb400640 1 -- 172.21.15.83:0/1864269209 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627987d3180 msgr2=0x5627987dad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:07.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.917+0000 7f02fb400640 1 --2- 172.21.15.83:0/1864269209 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627987d3180 0x5627987dad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:07.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.918+0000 7f02fb400640 1 -- 172.21.15.83:0/1864269209 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627986f0920 con 0x5627987d2860 2024-04-24T09:12:07.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.918+0000 7f02fa000640 1 --2- 172.21.15.83:0/1864269209 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627987d3180 0x5627987dad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:12:07.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.918+0000 7f02fb400640 1 --2- 172.21.15.83:0/1864269209 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5627987d2860 0x5627987da7c0 secure :-1 s=READY pgs=2531 cs=0 l=1 rev1=1 crypto rx=0x7f02f4006010 tx=0x7f02f4004970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf111fea1e4b5c87 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:07.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.918+0000 7f02dbe00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:12:07.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.919+0000 7f02dbe00640 1 -- 172.21.15.83:0/1864269209 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f4011850 con 0x5627987d2860 2024-04-24T09:12:07.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.919+0000 7f03043ab740 1 -- 172.21.15.83:0/1864269209 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02f4015070 con 0x5627987d2860 2024-04-24T09:12:07.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.919+0000 7f02dbe00640 1 -- 172.21.15.83:0/1864269209 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9671+0+0 (secure 0 0 0) 0x7f02f40119f0 con 0x5627987d2860 2024-04-24T09:12:07.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.919+0000 7f02dbe00640 1 -- 172.21.15.83:0/1864269209 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f02f4031050 con 0x5627987d2860 2024-04-24T09:12:07.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.919+0000 7f03043ab740 10 client.22608 fetch_fsmap learned FSMap version 61 2024-04-24T09:12:07.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.919+0000 7f03043ab740 1 -- 172.21.15.83:0/1864269209 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x5627985147d0 con 0x5627987d2860 2024-04-24T09:12:07.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.920+0000 7f02dbe00640 1 -- 172.21.15.83:0/1864269209 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f02f4015070 con 0x5627987d2860 2024-04-24T09:12:07.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.920+0000 7f02dbe00640 1 -- 172.21.15.83:0/1864269209 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 61) ==== 8362+0+0 (secure 0 0 0) 0x7f02f4023610 con 0x5627987d2860 2024-04-24T09:12:07.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.921+0000 7f03043ab740 10 client.22608 fetch_fsmap finished waiting for FSMap version 61 2024-04-24T09:12:07.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.921+0000 7f03043ab740 10 client.22608 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:12:07.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.921+0000 7f03043ab740 20 client.22608 populate_metadata read hostname 'smithi083' 2024-04-24T09:12:07.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.921+0000 7f03043ab740 1 --2- 172.21.15.83:0/1864269209 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5627987edb60 0x5627987eff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:07.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.921+0000 7f03043ab740 4 client.22608 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:12:07.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.921+0000 7f03043ab740 1 -- 172.21.15.83:0/1864269209 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562798745c10 con 0x5627987edb60 2024-04-24T09:12:07.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.921+0000 7f02fa000640 1 --2- 172.21.15.83:0/1864269209 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5627987edb60 0x5627987eff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:07.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.922+0000 7f02fa000640 1 --2- 172.21.15.83:0/1864269209 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5627987edb60 0x5627987eff50 crc :-1 s=READY pgs=507 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98d42424bbd3ca01 server_cookie=e07b6b608ba323cb in_seq=0 out_seq=0 2024-04-24T09:12:07.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.922+0000 7f02dbe00640 10 client.22608 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:12:07.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.924+0000 7f02dbe00640 1 -- 172.21.15.83:0/1864269209 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562798745c10 con 0x5627987edb60 2024-04-24T09:12:07.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.924+0000 7f02dbe00640 10 client.22608 handle_command_reply: tid=0 2024-04-24T09:12:07.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.924+0000 7f03043ab740 1 client.22608 shutdown 2024-04-24T09:12:07.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.925+0000 7f03043ab740 1 -- 172.21.15.83:0/1864269209 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5627987d2860 msgr2=0x5627987da7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:07.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.925+0000 7f03043ab740 1 --2- 172.21.15.83:0/1864269209 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5627987d2860 0x5627987da7c0 secure :-1 s=READY pgs=2531 cs=0 l=1 rev1=1 crypto rx=0x7f02f4006010 tx=0x7f02f4004970 comp rx=0 tx=0).stop 2024-04-24T09:12:07.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.925+0000 7f03043ab740 1 -- 172.21.15.83:0/1864269209 shutdown_connections 2024-04-24T09:12:07.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.925+0000 7f03043ab740 1 --2- 172.21.15.83:0/1864269209 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5627987edb60 0x5627987eff50 crc :-1 s=READY pgs=507 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:07.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.925+0000 7f02dbe00640 0 client.22608 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:12:07.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.925+0000 7f03043ab740 1 --2- 172.21.15.83:0/1864269209 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5627987dd850 0x5627987db240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:07.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.925+0000 7f03043ab740 1 --2- 172.21.15.83:0/1864269209 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627987d3180 0x5627987dad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:07.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.925+0000 7f03043ab740 1 --2- 172.21.15.83:0/1864269209 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5627987d2860 0x5627987da7c0 unknown :-1 s=CLOSED pgs=2531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:07.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.925+0000 7f03043ab740 1 -- 172.21.15.83:0/1864269209 >> 172.21.15.83:0/1864269209 conn(0x5627987ceb80 msgr2=0x5627987d4880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:07.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.925+0000 7f03043ab740 1 -- 172.21.15.83:0/1864269209 shutdown_connections 2024-04-24T09:12:07.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.925+0000 7f03043ab740 1 -- 172.21.15.83:0/1864269209 wait complete. 2024-04-24T09:12:07.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.926+0000 7f03043ab740 20 client.22608 trim_cache size 0 max 16384 2024-04-24T09:12:07.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.963+0000 7f03043ab740 1 Processor -- start 2024-04-24T09:12:07.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.963+0000 7f03043ab740 1 -- start start 2024-04-24T09:12:07.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.964+0000 7f03043ab740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5627987d2860 0x5627987db010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:07.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.964+0000 7f03043ab740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5627987d3180 0x5627987d5000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:07.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.964+0000 7f03043ab740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627987dd850 0x5627987f55e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:07.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.964+0000 7f03043ab740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f02dc010230 con 0x5627987d3180 2024-04-24T09:12:07.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.964+0000 7f03043ab740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f02dc011d70 con 0x5627987dd850 2024-04-24T09:12:07.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.964+0000 7f03043ab740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f02dc010900 con 0x5627987d2860 2024-04-24T09:12:07.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.965+0000 7f02fb400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5627987d2860 0x5627987db010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:07.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.965+0000 7f02fb400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5627987d2860 0x5627987db010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:56326/0 (socket says 172.21.15.83:56326) 2024-04-24T09:12:07.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.965+0000 7f02fb400640 1 -- 172.21.15.83:0/3225210240 learned_addr learned my addr 172.21.15.83:0/3225210240 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:12:07.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.965+0000 7f02fb400640 1 --2- 172.21.15.83:0/3225210240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627987dd850 0x5627987f55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:07.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.965+0000 7f02fa000640 1 --2- 172.21.15.83:0/3225210240 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5627987d3180 0x5627987d5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:07.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.966+0000 7f02fa000640 1 -- 172.21.15.83:0/3225210240 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5627987d2860 msgr2=0x5627987db010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:07.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.966+0000 7f02fa000640 1 --2- 172.21.15.83:0/3225210240 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5627987d2860 0x5627987db010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:07.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.966+0000 7f02fa000640 1 -- 172.21.15.83:0/3225210240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627987dd850 msgr2=0x5627987f55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:07.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.966+0000 7f02fa000640 1 --2- 172.21.15.83:0/3225210240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627987dd850 0x5627987f55e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:07.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.966+0000 7f02fa000640 1 -- 172.21.15.83:0/3225210240 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56279844a3f0 con 0x5627987d3180 2024-04-24T09:12:07.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.967+0000 7f02fa000640 1 --2- 172.21.15.83:0/3225210240 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5627987d3180 0x5627987d5000 secure :-1 s=READY pgs=2532 cs=0 l=1 rev1=1 crypto rx=0x5627987f0550 tx=0x7f02e8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34f053b5484ecc91 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:07.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.967+0000 7f02e6a00640 1 -- 172.21.15.83:0/3225210240 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e8016e30 con 0x5627987d3180 2024-04-24T09:12:07.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.967+0000 7f02e6a00640 1 -- 172.21.15.83:0/3225210240 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f02e801d250 con 0x5627987d3180 2024-04-24T09:12:07.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.967+0000 7f02e6a00640 1 -- 172.21.15.83:0/3225210240 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e80156b0 con 0x5627987d3180 2024-04-24T09:12:07.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.969+0000 7f03043ab740 1 -- 172.21.15.83:0/3225210240 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5627987d3180 msgr2=0x5627987d5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:07.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.969+0000 7f03043ab740 1 --2- 172.21.15.83:0/3225210240 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5627987d3180 0x5627987d5000 secure :-1 s=READY pgs=2532 cs=0 l=1 rev1=1 crypto rx=0x5627987f0550 tx=0x7f02e8008040 comp rx=0 tx=0).stop 2024-04-24T09:12:07.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.969+0000 7f03043ab740 1 -- 172.21.15.83:0/3225210240 shutdown_connections 2024-04-24T09:12:07.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.970+0000 7f03043ab740 1 --2- 172.21.15.83:0/3225210240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627987dd850 0x5627987f55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:07.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.970+0000 7f03043ab740 1 --2- 172.21.15.83:0/3225210240 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5627987d3180 0x5627987d5000 unknown :-1 s=CLOSED pgs=2532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:07.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.970+0000 7f03043ab740 1 --2- 172.21.15.83:0/3225210240 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5627987d2860 0x5627987db010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:07.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.970+0000 7f03043ab740 1 -- 172.21.15.83:0/3225210240 >> 172.21.15.83:0/3225210240 conn(0x5627987ceb80 msgr2=0x5627987cd390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:07.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.971+0000 7f03043ab740 1 -- 172.21.15.83:0/3225210240 shutdown_connections 2024-04-24T09:12:07.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.971+0000 7f03043ab740 1 -- 172.21.15.83:0/3225210240 wait complete. 2024-04-24T09:12:07.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.973+0000 7f03043ab740 1 -- start start 2024-04-24T09:12:07.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.973+0000 7f03043ab740 1 Processor -- start 2024-04-24T09:12:07.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.975+0000 7f03043ab740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627987d2860 0x5627987e1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:07.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.975+0000 7f03043ab740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5627987d3180 0x5627987e2180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:07.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.975+0000 7f03043ab740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5627987dd850 0x562798852e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:07.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.975+0000 7f03043ab740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f02dc0116a0 con 0x5627987d3180 2024-04-24T09:12:07.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.975+0000 7f03043ab740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f02dc010fd0 con 0x5627987d2860 2024-04-24T09:12:07.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.976+0000 7f02fb400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5627987dd850 0x562798852e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:07.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.975+0000 7f03043ab740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f02dc00e230 con 0x5627987dd850 2024-04-24T09:12:07.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.976+0000 7f03043ab740 10 client.0 _ll_register_callbacks cb 0x7f02e80093c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:12:07.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.976+0000 7f02fb400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627987d2860 0x5627987e1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:07.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.976+0000 7f02fb400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627987d2860 0x5627987e1c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:42516/0 (socket says 172.21.15.83:42516) 2024-04-24T09:12:07.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.976+0000 7f02fb400640 1 -- 172.21.15.83:0/1951780489 learned_addr learned my addr 172.21.15.83:0/1951780489 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:12:07.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.977+0000 7f02fa000640 1 --2- 172.21.15.83:0/1951780489 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5627987d3180 0x5627987e2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:07.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.977+0000 7f02fb400640 1 -- 172.21.15.83:0/1951780489 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627987d2860 msgr2=0x5627987e1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:07.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.977+0000 7f02fb400640 1 --2- 172.21.15.83:0/1951780489 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627987d2860 0x5627987e1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:07.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.977+0000 7f02fb400640 1 -- 172.21.15.83:0/1951780489 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5627987d3180 msgr2=0x5627987e2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:07.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.977+0000 7f02fb400640 1 --2- 172.21.15.83:0/1951780489 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5627987d3180 0x5627987e2180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:07.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.977+0000 7f02fb400640 1 -- 172.21.15.83:0/1951780489 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56279840a0f0 con 0x5627987dd850 2024-04-24T09:12:07.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.978+0000 7f02fb400640 1 --2- 172.21.15.83:0/1951780489 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5627987dd850 0x562798852e50 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f02f4014040 tx=0x7f02f4004970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c137200d0f594170 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:07.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.978+0000 7f02dbe00640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-04-24T09:12:07.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.978+0000 7f02dbe00640 1 -- 172.21.15.83:0/1951780489 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f4011710 con 0x5627987dd850 2024-04-24T09:12:07.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.978+0000 7f02dbe00640 1 -- 172.21.15.83:0/1951780489 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9671+0+0 (secure 0 0 0) 0x7f02f40118b0 con 0x5627987dd850 2024-04-24T09:12:07.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.978+0000 7f03043ab740 1 -- 172.21.15.83:0/1951780489 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02e801d250 con 0x5627987dd850 2024-04-24T09:12:07.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.979+0000 7f02dbe00640 1 -- 172.21.15.83:0/1951780489 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f02f4049050 con 0x5627987dd850 2024-04-24T09:12:07.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.979+0000 7f03043ab740 10 client.29536 fetch_fsmap learned FSMap version 61 2024-04-24T09:12:07.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.979+0000 7f03043ab740 1 -- 172.21.15.83:0/1951780489 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x5627986fd810 con 0x5627987dd850 2024-04-24T09:12:07.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.979+0000 7f02dbe00640 1 -- 172.21.15.83:0/1951780489 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f02e801d250 con 0x5627987dd850 2024-04-24T09:12:07.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.980+0000 7f02dbe00640 1 -- 172.21.15.83:0/1951780489 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 61) ==== 8362+0+0 (secure 0 0 0) 0x7f02f401ac30 con 0x5627987dd850 2024-04-24T09:12:07.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.980+0000 7f03043ab740 10 client.29536 fetch_fsmap finished waiting for FSMap version 61 2024-04-24T09:12:07.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.980+0000 7f03043ab740 10 client.29536 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:12:07.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.980+0000 7f03043ab740 20 client.29536 populate_metadata read hostname 'smithi083' 2024-04-24T09:12:07.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.980+0000 7f03043ab740 1 --2- 172.21.15.83:0/1951780489 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x562798854950 0x562798856d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:07.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.980+0000 7f03043ab740 4 client.29536 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T09:12:07.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.980+0000 7f03043ab740 1 -- 172.21.15.83:0/1951780489 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56279873fa80 con 0x562798854950 2024-04-24T09:12:07.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.981+0000 7f02fa000640 1 --2- 172.21.15.83:0/1951780489 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x562798854950 0x562798856d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:07.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.982+0000 7f02dbe00640 10 client.29536 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:12:07.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.982+0000 7f02fa000640 1 --2- 172.21.15.83:0/1951780489 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x562798854950 0x562798856d70 crc :-1 s=READY pgs=508 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ae937a25ee2fc33 server_cookie=644f41968d7aa824 in_seq=0 out_seq=0 2024-04-24T09:12:07.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.982+0000 7f02dbe00640 1 -- 172.21.15.83:0/1951780489 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56279873fa80 con 0x562798854950 2024-04-24T09:12:07.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.982+0000 7f02dbe00640 10 client.29536 handle_command_reply: tid=0 2024-04-24T09:12:07.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.982+0000 7f03043ab740 1 client.29536 shutdown 2024-04-24T09:12:07.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.983+0000 7f03043ab740 1 -- 172.21.15.83:0/1951780489 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5627987dd850 msgr2=0x562798852e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:07.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.983+0000 7f03043ab740 1 --2- 172.21.15.83:0/1951780489 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5627987dd850 0x562798852e50 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f02f4014040 tx=0x7f02f4004970 comp rx=0 tx=0).stop 2024-04-24T09:12:07.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.983+0000 7f03043ab740 1 -- 172.21.15.83:0/1951780489 shutdown_connections 2024-04-24T09:12:07.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.983+0000 7f03043ab740 1 --2- 172.21.15.83:0/1951780489 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x562798854950 0x562798856d70 crc :-1 s=READY pgs=508 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:07.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.983+0000 7f03043ab740 1 --2- 172.21.15.83:0/1951780489 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5627987dd850 0x562798852e50 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:07.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.983+0000 7f03043ab740 1 --2- 172.21.15.83:0/1951780489 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5627987d3180 0x5627987e2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:07.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.983+0000 7f03043ab740 1 --2- 172.21.15.83:0/1951780489 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627987d2860 0x5627987e1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:07.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.983+0000 7f02dbe00640 0 client.29536 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:12:07.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.983+0000 7f03043ab740 1 -- 172.21.15.83:0/1951780489 >> 172.21.15.83:0/1951780489 conn(0x5627987ceb80 msgr2=0x5627987cdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:07.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.983+0000 7f03043ab740 1 -- 172.21.15.83:0/1951780489 shutdown_connections 2024-04-24T09:12:07.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.983+0000 7f03043ab740 1 -- 172.21.15.83:0/1951780489 wait complete. 2024-04-24T09:12:07.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.983+0000 7f03043ab740 20 client.29536 trim_cache size 0 max 16384 2024-04-24T09:12:07.984 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T09:12:07.984 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-04-24T09:12:07.984 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-04-24T09:12:07.984 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T09:12:07.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.984+0000 7f0301600640 1 -- 172.21.15.83:0/3481622467 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f02e00a3d20 msgr2=0x7f02e00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:07.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.984+0000 7f0301600640 1 --2- 172.21.15.83:0/3481622467 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f02e00a3d20 0x7f02e00a61e0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x5627987d3e20 tx=0x7f02ec02a3f0 comp rx=0 tx=0).stop 2024-04-24T09:12:07.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.984+0000 7f0301600640 1 -- 172.21.15.83:0/3481622467 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f02fc109210 msgr2=0x7f02fc1a3320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:07.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.984+0000 7f0301600640 1 --2- 172.21.15.83:0/3481622467 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f02fc109210 0x7f02fc1a3320 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f02ec005ab0 tx=0x7f02ec003cd0 comp rx=0 tx=0).stop 2024-04-24T09:12:07.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.984+0000 7f0301600640 1 -- 172.21.15.83:0/3481622467 shutdown_connections 2024-04-24T09:12:07.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.984+0000 7f0301600640 1 --2- 172.21.15.83:0/3481622467 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f02e00a3d20 0x7f02e00a61e0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:07.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.984+0000 7f0301600640 1 --2- 172.21.15.83:0/3481622467 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f02fc115150 0x7f02fc1a7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:07.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.984+0000 7f0301600640 1 --2- 172.21.15.83:0/3481622467 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02fc109b30 0x7f02fc1a3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:07.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.984+0000 7f0301600640 1 --2- 172.21.15.83:0/3481622467 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f02fc109210 0x7f02fc1a3320 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:07.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.984+0000 7f0301600640 1 -- 172.21.15.83:0/3481622467 >> 172.21.15.83:0/3481622467 conn(0x7f02fc06d110 msgr2=0x7f02fc070e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:07.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.984+0000 7f0301600640 1 -- 172.21.15.83:0/3481622467 shutdown_connections 2024-04-24T09:12:07.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:07.984+0000 7f0301600640 1 -- 172.21.15.83:0/3481622467 wait complete. 2024-04-24T09:12:08.011 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T09:12:08.011 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T09:12:08.184 INFO:tasks.workunit.client.0.smithi083.stdout: 10 49895 6.65 MB/sec execute 279 sec latency 109.944 ms 2024-04-24T09:12:08.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:08.269+0000 7f820be00640 1 -- 172.21.15.83:0/2348683360 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8204109510 msgr2=0x7f82041110e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:08.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:08.269+0000 7f820be00640 1 --2- 172.21.15.83:0/2348683360 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8204109510 0x7f82041110e0 secure :-1 s=READY pgs=2533 cs=0 l=1 rev1=1 crypto rx=0x7f81fc009ca0 tx=0x7f81fc02a8c0 comp rx=0 tx=0).stop 2024-04-24T09:12:08.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:08.270+0000 7f820be00640 1 -- 172.21.15.83:0/2348683360 shutdown_connections 2024-04-24T09:12:08.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:08.270+0000 7f820be00640 1 --2- 172.21.15.83:0/2348683360 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8204109510 0x7f82041110e0 unknown :-1 s=CLOSED pgs=2533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:08.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:08.270+0000 7f820be00640 1 --2- 172.21.15.83:0/2348683360 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8204104df0 0x7f8204108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:08.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:08.270+0000 7f820be00640 1 --2- 172.21.15.83:0/2348683360 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8204104440 0x7f8204104820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:08.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:08.270+0000 7f820be00640 1 -- 172.21.15.83:0/2348683360 >> 172.21.15.83:0/2348683360 conn(0x7f82040ffcc0 msgr2=0x7f82041020e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:08.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:08.270+0000 7f820be00640 1 -- 172.21.15.83:0/2348683360 shutdown_connections 2024-04-24T09:12:08.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:08.270+0000 7f820be00640 1 -- 172.21.15.83:0/2348683360 wait complete. 2024-04-24T09:12:08.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:08.272+0000 7f820be00640 1 Processor -- start 2024-04-24T09:12:08.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:08.273+0000 7f820be00640 1 -- start start 2024-04-24T09:12:08.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:08.273+0000 7f820be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8204104440 0x7f82041a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:08.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:08.273+0000 7f820be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8204109510 0x7f82041a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:08.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:08.273+0000 7f820be00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f820419a830 0x7f820419ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:08.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:08.273+0000 7f820be00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f82041144b0 con 0x7f8204109510 2024-04-24T09:12:08.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:08.273+0000 7f820be00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8204114330 con 0x7f8204104440 2024-04-24T09:12:08.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:08.273+0000 7f820be00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8204114630 con 0x7f820419a830 2024-04-24T09:12:08.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:08.273+0000 7f8209400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8204104440 0x7f82041a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:08.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:08.274+0000 7f8209400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8204104440 0x7f82041a0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:42524/0 (socket says 172.21.15.83:42524) 2024-04-24T09:12:08.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:08.274+0000 7f8209400640 1 -- 172.21.15.83:0/1455298047 learned_addr learned my addr 172.21.15.83:0/1455298047 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:12:08.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:08.274+0000 7f8208a00640 1 --2- 172.21.15.83:0/1455298047 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8204109510 0x7f82041a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:08.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:08.274+0000 7f8209e00640 1 --2- 172.21.15.83:0/1455298047 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f820419a830 0x7f820419ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:08.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:08.274+0000 7f8209400640 1 -- 172.21.15.83:0/1455298047 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f820419a830 msgr2=0x7f820419ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:08.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:08.274+0000 7f8209400640 1 --2- 172.21.15.83:0/1455298047 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f820419a830 0x7f820419ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:08.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:08.274+0000 7f8209400640 1 -- 172.21.15.83:0/1455298047 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8204109510 msgr2=0x7f82041a0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:08.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:08.274+0000 7f8209400640 1 --2- 172.21.15.83:0/1455298047 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8204109510 0x7f82041a0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:08.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:08.274+0000 7f8209400640 1 -- 172.21.15.83:0/1455298047 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f820419b3c0 con 0x7f8204104440 2024-04-24T09:12:08.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:08.274+0000 7f8209e00640 1 --2- 172.21.15.83:0/1455298047 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f820419a830 0x7f820419ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:12:08.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:08.275+0000 7f8209400640 1 --2- 172.21.15.83:0/1455298047 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8204104440 0x7f82041a0670 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f81f800c890 tx=0x7f81f800cd50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:08.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:08.275+0000 7f8202000640 1 -- 172.21.15.83:0/1455298047 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81f8004220 con 0x7f8204104440 2024-04-24T09:12:08.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:08.275+0000 7f8202000640 1 -- 172.21.15.83:0/1455298047 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f81f80043c0 con 0x7f8204104440 2024-04-24T09:12:08.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:08.275+0000 7f820be00640 1 -- 172.21.15.83:0/1455298047 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f820419b6b0 con 0x7f8204104440 2024-04-24T09:12:08.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:08.275+0000 7f8202000640 1 -- 172.21.15.83:0/1455298047 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81f8004a00 con 0x7f8204104440 2024-04-24T09:12:08.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:08.275+0000 7f820be00640 1 -- 172.21.15.83:0/1455298047 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82041a7890 con 0x7f8204104440 2024-04-24T09:12:08.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:08.277+0000 7f820be00640 1 -- 172.21.15.83:0/1455298047 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8204105910 con 0x7f8204104440 2024-04-24T09:12:08.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:08.280+0000 7f8202000640 1 -- 172.21.15.83:0/1455298047 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f81f8004ba0 con 0x7f8204104440 2024-04-24T09:12:08.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:08.282+0000 7f8202000640 1 --2- 172.21.15.83:0/1455298047 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f81ec0a3da0 0x7f81ec0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:08.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:08.283+0000 7f8202000640 1 -- 172.21.15.83:0/1455298047 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 9671+0+0 (secure 0 0 0) 0x7f81f80d0d70 con 0x7f8204104440 2024-04-24T09:12:08.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:08.283+0000 7f8208a00640 1 --2- 172.21.15.83:0/1455298047 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f81ec0a3da0 0x7f81ec0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:08.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:08.284+0000 7f8208a00640 1 --2- 172.21.15.83:0/1455298047 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f81ec0a3da0 0x7f81ec0a6260 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f81f00095f0 tx=0x7f81f0005f90 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:08.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:08.285+0000 7f8202000640 1 -- 172.21.15.83:0/1455298047 <== mon.2 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f820419b6b0 con 0x7f8204104440 2024-04-24T09:12:08.704 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:12:08 smithi138 ceph-mon[31441]: pgmap v647: 129 pgs: 15 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 76 active+clean; 780 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.6 MiB/s wr, 281 op/s 2024-04-24T09:12:08.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:12:08 smithi189 ceph-mon[31791]: pgmap v647: 129 pgs: 15 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 76 active+clean; 780 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.6 MiB/s wr, 281 op/s 2024-04-24T09:12:08.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:08.719+0000 7f820be00640 1 -- 172.21.15.83:0/1455298047 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f820419bd10 con 0x7f8204104440 2024-04-24T09:12:08.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:08.720+0000 7f8202000640 1 -- 172.21.15.83:0/1455298047 <== mon.2 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) ==== 94+0+11944 (secure 0 0 0) 0x7f820419bd10 con 0x7f8204104440 2024-04-24T09:12:08.719 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T09:12:08.719 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":61,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/358872257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":358872257},{"type":"v1","addr":"172.21.15.83:6835","nonce":358872257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3157070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3157070851},{"type":"v1","addr":"172.21.15.83:6837","nonce":3157070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":61,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T08:48:29.422576+0000","modified":"2024-04-24T09:09:34.468831+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34520,"mds_1":34493,"mds_2":34451,"mds_3":34427,"mds_4":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14658":{"gid":14658,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6839/4085470844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":4085470844},{"type":"v1","addr":"172.21.15.83:6839","nonce":4085470844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14676":{"gid":14676,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6841/2634754210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":2634754210},{"type":"v1","addr":"172.21.15.83:6841","nonce":2634754210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24382":{"gid":24382,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/1859921310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1859921310},{"type":"v1","addr":"172.21.15.189:6833","nonce":1859921310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/2241923197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2241923197},{"type":"v1","addr":"172.21.15.189:6835","nonce":2241923197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.189:6839/2548965336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":2548965336},{"type":"v1","addr":"172.21.15.189:6839","nonce":2548965336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26686":{"gid":26686,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6837/1678817161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":1678817161},{"type":"v1","addr":"172.21.15.189:6837","nonce":1678817161}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.138:6833/1157967658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1157967658},{"type":"v1","addr":"172.21.15.138:6833","nonce":1157967658}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.138:6835/1909219758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1909219758},{"type":"v1","addr":"172.21.15.138:6835","nonce":1909219758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34493":{"gid":34493,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.138:6837/900566754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":900566754},{"type":"v1","addr":"172.21.15.138:6837","nonce":900566754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34520":{"gid":34520,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.138:6839/2236932393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2236932393},{"type":"v1","addr":"172.21.15.138:6839","nonce":2236932393}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34520,"qdb_cluster":[24454,34520,34427,34451,34493]},"id":1}]} 2024-04-24T09:12:08.719 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 61 2024-04-24T09:12:08.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:08.723+0000 7f820be00640 1 -- 172.21.15.83:0/1455298047 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f81ec0a3da0 msgr2=0x7f81ec0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:08.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:08.723+0000 7f820be00640 1 --2- 172.21.15.83:0/1455298047 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f81ec0a3da0 0x7f81ec0a6260 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f81f00095f0 tx=0x7f81f0005f90 comp rx=0 tx=0).stop 2024-04-24T09:12:08.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:08.723+0000 7f820be00640 1 -- 172.21.15.83:0/1455298047 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8204104440 msgr2=0x7f82041a0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:08.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:08.723+0000 7f820be00640 1 --2- 172.21.15.83:0/1455298047 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8204104440 0x7f82041a0670 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f81f800c890 tx=0x7f81f800cd50 comp rx=0 tx=0).stop 2024-04-24T09:12:08.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:08.723+0000 7f820be00640 1 -- 172.21.15.83:0/1455298047 shutdown_connections 2024-04-24T09:12:08.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:08.723+0000 7f820be00640 1 --2- 172.21.15.83:0/1455298047 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f81ec0a3da0 0x7f81ec0a6260 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:08.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:08.723+0000 7f820be00640 1 --2- 172.21.15.83:0/1455298047 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f820419a830 0x7f820419ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:08.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:08.723+0000 7f820be00640 1 --2- 172.21.15.83:0/1455298047 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8204109510 0x7f82041a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:08.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:08.723+0000 7f820be00640 1 --2- 172.21.15.83:0/1455298047 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8204104440 0x7f82041a0670 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:08.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:08.723+0000 7f820be00640 1 -- 172.21.15.83:0/1455298047 >> 172.21.15.83:0/1455298047 conn(0x7f82040ffcc0 msgr2=0x7f8204101fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:08.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:08.723+0000 7f820be00640 1 -- 172.21.15.83:0/1455298047 shutdown_connections 2024-04-24T09:12:08.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:08.724+0000 7f820be00640 1 -- 172.21.15.83:0/1455298047 wait complete. 2024-04-24T09:12:08.745 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:12:08 smithi083 ceph-mon[27907]: pgmap v647: 129 pgs: 15 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 76 active+clean; 780 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.6 MiB/s wr, 281 op/s 2024-04-24T09:12:08.750 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-24T09:12:09.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.013+0000 7f05c6600640 1 -- 172.21.15.83:0/4158453096 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05c0104450 msgr2=0x7f05c0104830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:09.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.013+0000 7f05c6600640 1 --2- 172.21.15.83:0/4158453096 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05c0104450 0x7f05c0104830 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f05b0009860 tx=0x7f05b002a340 comp rx=0 tx=0).stop 2024-04-24T09:12:09.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.014+0000 7f05c6600640 1 -- 172.21.15.83:0/4158453096 shutdown_connections 2024-04-24T09:12:09.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.014+0000 7f05c6600640 1 --2- 172.21.15.83:0/4158453096 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f05c0109520 0x7f05c01110f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:09.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.014+0000 7f05c6600640 1 --2- 172.21.15.83:0/4158453096 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05c0104e00 0x7f05c0108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:09.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.014+0000 7f05c6600640 1 --2- 172.21.15.83:0/4158453096 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05c0104450 0x7f05c0104830 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:09.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.014+0000 7f05c6600640 1 -- 172.21.15.83:0/4158453096 >> 172.21.15.83:0/4158453096 conn(0x7f05c00ffc70 msgr2=0x7f05c0102090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:09.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.014+0000 7f05c6600640 1 -- 172.21.15.83:0/4158453096 shutdown_connections 2024-04-24T09:12:09.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.014+0000 7f05c6600640 1 -- 172.21.15.83:0/4158453096 wait complete. 2024-04-24T09:12:09.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.015+0000 7f05c6600640 1 Processor -- start 2024-04-24T09:12:09.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.015+0000 7f05c6600640 1 -- start start 2024-04-24T09:12:09.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.016+0000 7f05c6600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05c0104450 0x7f05c01a0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:09.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.016+0000 7f05c6600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05c0109520 0x7f05c01a0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:09.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.016+0000 7f05c6600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f05c019a930 0x7f05c019ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:09.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.016+0000 7f05c6600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f05c0114530 con 0x7f05c0104450 2024-04-24T09:12:09.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.016+0000 7f05c6600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f05c01143b0 con 0x7f05c0109520 2024-04-24T09:12:09.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.016+0000 7f05c6600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f05c01146b0 con 0x7f05c019a930 2024-04-24T09:12:09.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.016+0000 7f05bf400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05c0104450 0x7f05c01a0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:09.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.017+0000 7f05bf400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05c0104450 0x7f05c01a0780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49822/0 (socket says 172.21.15.83:49822) 2024-04-24T09:12:09.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.017+0000 7f05bf400640 1 -- 172.21.15.83:0/2877040386 learned_addr learned my addr 172.21.15.83:0/2877040386 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:12:09.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.017+0000 7f05bea00640 1 --2- 172.21.15.83:0/2877040386 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05c0109520 0x7f05c01a0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:09.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.017+0000 7f05bfe00640 1 --2- 172.21.15.83:0/2877040386 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f05c019a930 0x7f05c019ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:09.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.017+0000 7f05bf400640 1 -- 172.21.15.83:0/2877040386 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f05c019a930 msgr2=0x7f05c019ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:09.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.017+0000 7f05bf400640 1 --2- 172.21.15.83:0/2877040386 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f05c019a930 0x7f05c019ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:09.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.017+0000 7f05bf400640 1 -- 172.21.15.83:0/2877040386 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05c0109520 msgr2=0x7f05c01a0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:09.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.017+0000 7f05bf400640 1 --2- 172.21.15.83:0/2877040386 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05c0109520 0x7f05c01a0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:09.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.017+0000 7f05bf400640 1 -- 172.21.15.83:0/2877040386 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05c019b450 con 0x7f05c0104450 2024-04-24T09:12:09.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.017+0000 7f05bea00640 1 --2- 172.21.15.83:0/2877040386 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05c0109520 0x7f05c01a0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:12:09.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.018+0000 7f05bf400640 1 --2- 172.21.15.83:0/2877040386 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05c0104450 0x7f05c01a0780 secure :-1 s=READY pgs=2534 cs=0 l=1 rev1=1 crypto rx=0x7f05b002a8b0 tx=0x7f05b0030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:09.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.018+0000 7f05b7e00640 1 -- 172.21.15.83:0/2877040386 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b0037030 con 0x7f05c0104450 2024-04-24T09:12:09.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.018+0000 7f05b7e00640 1 -- 172.21.15.83:0/2877040386 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f05b0041070 con 0x7f05c0104450 2024-04-24T09:12:09.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.018+0000 7f05b7e00640 1 -- 172.21.15.83:0/2877040386 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b00046c0 con 0x7f05c0104450 2024-04-24T09:12:09.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.018+0000 7f05c6600640 1 -- 172.21.15.83:0/2877040386 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05c019b6e0 con 0x7f05c0104450 2024-04-24T09:12:09.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.018+0000 7f05c6600640 1 -- 172.21.15.83:0/2877040386 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05c01a7930 con 0x7f05c0104450 2024-04-24T09:12:09.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.023+0000 7f05c9456740 1 Processor -- start 2024-04-24T09:12:09.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.023+0000 7f05c9456740 1 -- start start 2024-04-24T09:12:09.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.023+0000 7f05c9456740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564f9b0b5950 0x564f9b0b5d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:09.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.023+0000 7f05c9456740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f9b0b6270 0x564f9b0c0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:09.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.023+0000 7f05c9456740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564f9b0c0940 0x564f9b0c2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:09.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.023+0000 7f05c9456740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x564f9b01b1d0 con 0x564f9b0c0940 2024-04-24T09:12:09.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.023+0000 7f05c9456740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x564f9afc1450 con 0x564f9b0b6270 2024-04-24T09:12:09.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.023+0000 7f05c9456740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x564f9b017190 con 0x564f9b0b5950 2024-04-24T09:12:09.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.024+0000 7f05b7e00640 1 -- 172.21.15.83:0/2877040386 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f05b0030450 con 0x7f05c0104450 2024-04-24T09:12:09.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.024+0000 7f05bea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564f9b0c0940 0x564f9b0c2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:09.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.024+0000 7f05bfe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564f9b0b5950 0x564f9b0b5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:09.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.024+0000 7f05bea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564f9b0c0940 0x564f9b0c2d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49834/0 (socket says 172.21.15.83:49834) 2024-04-24T09:12:09.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.024+0000 7f05bea00640 1 -- 172.21.15.83:0/3084682531 learned_addr learned my addr 172.21.15.83:0/3084682531 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:12:09.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.024+0000 7f05bf400640 1 --2- 172.21.15.83:0/3084682531 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f9b0b6270 0x564f9b0c0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:09.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.024+0000 7f05bea00640 1 -- 172.21.15.83:0/3084682531 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564f9b0b5950 msgr2=0x564f9b0b5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:09.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.024+0000 7f05bea00640 1 --2- 172.21.15.83:0/3084682531 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564f9b0b5950 0x564f9b0b5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:09.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.024+0000 7f05bea00640 1 -- 172.21.15.83:0/3084682531 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f9b0b6270 msgr2=0x564f9b0c0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:09.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.024+0000 7f05bea00640 1 --2- 172.21.15.83:0/3084682531 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f9b0b6270 0x564f9b0c0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:09.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.024+0000 7f05bea00640 1 -- 172.21.15.83:0/3084682531 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f9b07e310 con 0x564f9b0c0940 2024-04-24T09:12:09.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.025+0000 7f05bea00640 1 --2- 172.21.15.83:0/3084682531 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564f9b0c0940 0x564f9b0c2d30 secure :-1 s=READY pgs=2535 cs=0 l=1 rev1=1 crypto rx=0x7f05ac011470 tx=0x7f05ac011930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=872ed5cb0af303e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:09.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.025+0000 7f05b4c00640 1 -- 172.21.15.83:0/3084682531 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05ac01e070 con 0x564f9b0c0940 2024-04-24T09:12:09.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.025+0000 7f05b4c00640 1 -- 172.21.15.83:0/3084682531 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f05ac0035f0 con 0x564f9b0c0940 2024-04-24T09:12:09.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.025+0000 7f05b4c00640 1 -- 172.21.15.83:0/3084682531 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05ac00f840 con 0x564f9b0c0940 2024-04-24T09:12:09.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.026+0000 7f05b7e00640 1 --2- 172.21.15.83:0/2877040386 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f05a40a3de0 0x7f05a40a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:09.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.026+0000 7f05b7e00640 1 -- 172.21.15.83:0/2877040386 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 9671+0+0 (secure 0 0 0) 0x7f05b00f0800 con 0x7f05c0104450 2024-04-24T09:12:09.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.026+0000 7f05bfe00640 1 --2- 172.21.15.83:0/2877040386 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f05a40a3de0 0x7f05a40a62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:09.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.026+0000 7f05c9456740 1 -- 172.21.15.83:0/3084682531 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564f9b0c0940 msgr2=0x564f9b0c2d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:09.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.026+0000 7f05c9456740 1 --2- 172.21.15.83:0/3084682531 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564f9b0c0940 0x564f9b0c2d30 secure :-1 s=READY pgs=2535 cs=0 l=1 rev1=1 crypto rx=0x7f05ac011470 tx=0x7f05ac011930 comp rx=0 tx=0).stop 2024-04-24T09:12:09.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.027+0000 7f05c9456740 1 -- 172.21.15.83:0/3084682531 shutdown_connections 2024-04-24T09:12:09.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.027+0000 7f05c9456740 1 --2- 172.21.15.83:0/3084682531 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564f9b0c0940 0x564f9b0c2d30 unknown :-1 s=CLOSED pgs=2535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:09.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.027+0000 7f05c9456740 1 --2- 172.21.15.83:0/3084682531 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f9b0b6270 0x564f9b0c0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:09.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.027+0000 7f05c9456740 1 --2- 172.21.15.83:0/3084682531 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564f9b0b5950 0x564f9b0b5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:09.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.027+0000 7f05c9456740 1 -- 172.21.15.83:0/3084682531 >> 172.21.15.83:0/3084682531 conn(0x564f9b0b1c70 msgr2=0x564f9b0b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:09.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.027+0000 7f05c9456740 1 -- 172.21.15.83:0/3084682531 shutdown_connections 2024-04-24T09:12:09.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.027+0000 7f05c9456740 1 -- 172.21.15.83:0/3084682531 wait complete. 2024-04-24T09:12:09.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.027+0000 7f05bfe00640 1 --2- 172.21.15.83:0/2877040386 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f05a40a3de0 0x7f05a40a62a0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f05b800b4d0 tx=0x7f05b800e040 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:09.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.027+0000 7f05c9456740 1 -- start start 2024-04-24T09:12:09.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.028+0000 7f05c9456740 1 Processor -- start 2024-04-24T09:12:09.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.028+0000 7f05c9456740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564f9b0b5950 0x564f9b0bd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:09.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.028+0000 7f05c9456740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564f9b0b6270 0x564f9b0bddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:09.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.028+0000 7f05c9456740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f9b0be330 0x564f9b0d5480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:09.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.028+0000 7f05c9456740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x564f9b0c37d0 con 0x564f9b0b6270 2024-04-24T09:12:09.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.028+0000 7f05c9456740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x564f9afd4810 con 0x564f9b0be330 2024-04-24T09:12:09.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.028+0000 7f05c9456740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x564f9b0c3950 con 0x564f9b0b5950 2024-04-24T09:12:09.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.028+0000 7f05c9456740 10 client.0 _ll_register_callbacks cb 0x564f9afa2140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:12:09.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.029+0000 7f05bea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564f9b0b6270 0x564f9b0bddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:09.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.029+0000 7f05bea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564f9b0b6270 0x564f9b0bddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49838/0 (socket says 172.21.15.83:49838) 2024-04-24T09:12:09.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.029+0000 7f05bea00640 1 -- 172.21.15.83:0/3518397929 learned_addr learned my addr 172.21.15.83:0/3518397929 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:12:09.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.029+0000 7f05bfe00640 1 --2- 172.21.15.83:0/3518397929 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f9b0be330 0x564f9b0d5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:09.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.029+0000 7f05bf400640 1 --2- 172.21.15.83:0/3518397929 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564f9b0b5950 0x564f9b0bd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:09.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.029+0000 7f05bea00640 1 -- 172.21.15.83:0/3518397929 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564f9b0b5950 msgr2=0x564f9b0bd8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:09.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.029+0000 7f05bea00640 1 --2- 172.21.15.83:0/3518397929 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564f9b0b5950 0x564f9b0bd8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:09.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.029+0000 7f05bea00640 1 -- 172.21.15.83:0/3518397929 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f9b0be330 msgr2=0x564f9b0d5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:09.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.029+0000 7f05bea00640 1 --2- 172.21.15.83:0/3518397929 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f9b0be330 0x564f9b0d5480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:09.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.029+0000 7f05bea00640 1 -- 172.21.15.83:0/3518397929 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f9afe53f0 con 0x564f9b0b6270 2024-04-24T09:12:09.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.029+0000 7f05bfe00640 1 --2- 172.21.15.83:0/3518397929 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f9b0be330 0x564f9b0d5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:12:09.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.030+0000 7f05bea00640 1 --2- 172.21.15.83:0/3518397929 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564f9b0b6270 0x564f9b0bddf0 secure :-1 s=READY pgs=2536 cs=0 l=1 rev1=1 crypto rx=0x7f05ac011e30 tx=0x7f05ac004230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5195e52d49d8a8a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:09.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.030+0000 7f05a9600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:12:09.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.030+0000 7f05a9600640 1 -- 172.21.15.83:0/3518397929 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05ac01e070 con 0x564f9b0b6270 2024-04-24T09:12:09.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.030+0000 7f05c9456740 1 -- 172.21.15.83:0/3518397929 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05ac0035f0 con 0x564f9b0b6270 2024-04-24T09:12:09.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.030+0000 7f05a9600640 1 -- 172.21.15.83:0/3518397929 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9671+0+0 (secure 0 0 0) 0x7f05ac005040 con 0x564f9b0b6270 2024-04-24T09:12:09.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.031+0000 7f05a9600640 1 -- 172.21.15.83:0/3518397929 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f05ac003bd0 con 0x564f9b0b6270 2024-04-24T09:12:09.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.031+0000 7f05c9456740 10 client.22632 fetch_fsmap learned FSMap version 61 2024-04-24T09:12:09.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.031+0000 7f05c9456740 1 -- 172.21.15.83:0/3518397929 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x564f9ad2d3f0 con 0x564f9b0b6270 2024-04-24T09:12:09.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.031+0000 7f05a9600640 1 -- 172.21.15.83:0/3518397929 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f05ac0035f0 con 0x564f9b0b6270 2024-04-24T09:12:09.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.032+0000 7f05a9600640 1 -- 172.21.15.83:0/3518397929 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 61) ==== 8362+0+0 (secure 0 0 0) 0x7f05ac004b80 con 0x564f9b0b6270 2024-04-24T09:12:09.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.032+0000 7f05c9456740 10 client.22632 fetch_fsmap finished waiting for FSMap version 61 2024-04-24T09:12:09.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.032+0000 7f05c9456740 10 client.22632 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-24T09:12:09.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.032+0000 7f05c9456740 20 client.22632 populate_metadata read hostname 'smithi083' 2024-04-24T09:12:09.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.032+0000 7f05c9456740 1 --2- 172.21.15.83:0/3518397929 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x564f9b0d9840 0x564f9b0dbc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:09.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.032+0000 7f05c9456740 4 client.22632 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:12:09.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.032+0000 7f05c9456740 1 -- 172.21.15.83:0/3518397929 --> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564f9b028bf0 con 0x564f9b0d9840 2024-04-24T09:12:09.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.033+0000 7f05bf400640 1 --2- 172.21.15.83:0/3518397929 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x564f9b0d9840 0x564f9b0dbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:09.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.034+0000 7f05bf400640 1 --2- 172.21.15.83:0/3518397929 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x564f9b0d9840 0x564f9b0dbc30 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b4820ff73bc70a64 server_cookie=51d8ad90897477d2 in_seq=0 out_seq=0 2024-04-24T09:12:09.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.034+0000 7f05a9600640 10 client.22632 ms_handle_connect on v2:172.21.15.189:6838/2548965336 2024-04-24T09:12:09.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.037+0000 7f05a9600640 1 -- 172.21.15.83:0/3518397929 <== mds.4 v2:172.21.15.189:6838/2548965336 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564f9b028bf0 con 0x564f9b0d9840 2024-04-24T09:12:09.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.037+0000 7f05a9600640 10 client.22632 handle_command_reply: tid=0 2024-04-24T09:12:09.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.037+0000 7f05c9456740 1 client.22632 shutdown 2024-04-24T09:12:09.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.037+0000 7f05c9456740 1 -- 172.21.15.83:0/3518397929 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564f9b0b6270 msgr2=0x564f9b0bddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:09.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.037+0000 7f05c9456740 1 --2- 172.21.15.83:0/3518397929 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564f9b0b6270 0x564f9b0bddf0 secure :-1 s=READY pgs=2536 cs=0 l=1 rev1=1 crypto rx=0x7f05ac011e30 tx=0x7f05ac004230 comp rx=0 tx=0).stop 2024-04-24T09:12:09.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.037+0000 7f05c9456740 1 -- 172.21.15.83:0/3518397929 shutdown_connections 2024-04-24T09:12:09.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.037+0000 7f05c9456740 1 --2- 172.21.15.83:0/3518397929 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x564f9b0d9840 0x564f9b0dbc30 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:09.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.037+0000 7f05c9456740 1 --2- 172.21.15.83:0/3518397929 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f9b0be330 0x564f9b0d5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:09.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.037+0000 7f05c9456740 1 --2- 172.21.15.83:0/3518397929 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564f9b0b6270 0x564f9b0bddf0 unknown :-1 s=CLOSED pgs=2536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:09.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.037+0000 7f05a9600640 0 client.22632 ms_handle_reset on v2:172.21.15.189:6838/2548965336 2024-04-24T09:12:09.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.037+0000 7f05c9456740 1 --2- 172.21.15.83:0/3518397929 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564f9b0b5950 0x564f9b0bd8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:09.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.037+0000 7f05c9456740 1 -- 172.21.15.83:0/3518397929 >> 172.21.15.83:0/3518397929 conn(0x564f9b0b1c70 msgr2=0x564f9b0b7a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:09.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.038+0000 7f05c9456740 1 -- 172.21.15.83:0/3518397929 shutdown_connections 2024-04-24T09:12:09.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.038+0000 7f05c9456740 1 -- 172.21.15.83:0/3518397929 wait complete. 2024-04-24T09:12:09.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.038+0000 7f05c9456740 20 client.22632 trim_cache size 0 max 16384 2024-04-24T09:12:09.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.060+0000 7f05c9456740 1 Processor -- start 2024-04-24T09:12:09.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.061+0000 7f05c9456740 1 -- start start 2024-04-24T09:12:09.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.061+0000 7f05c9456740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f9b0b5950 0x564f9b0b80f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:09.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.061+0000 7f05c9456740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564f9b0b6270 0x564f9b0d4f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:09.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.061+0000 7f05c9456740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564f9b0be330 0x564f9b0d5490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:09.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.061+0000 7f05c9456740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f05c01ad5d0 con 0x564f9b0b6270 2024-04-24T09:12:09.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.061+0000 7f05c9456740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f05c01af110 con 0x564f9b0b5950 2024-04-24T09:12:09.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.061+0000 7f05c9456740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f05c01adca0 con 0x564f9b0be330 2024-04-24T09:12:09.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.061+0000 7f05bea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564f9b0b6270 0x564f9b0d4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:09.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.061+0000 7f05bf400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f9b0b5950 0x564f9b0b80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:09.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.061+0000 7f05bea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564f9b0b6270 0x564f9b0d4f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49854/0 (socket says 172.21.15.83:49854) 2024-04-24T09:12:09.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.061+0000 7f05bf400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f9b0b5950 0x564f9b0b80f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:59186/0 (socket says 172.21.15.83:59186) 2024-04-24T09:12:09.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.062+0000 7f05bea00640 1 -- 172.21.15.83:0/3093931449 learned_addr learned my addr 172.21.15.83:0/3093931449 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:12:09.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.062+0000 7f05bfe00640 1 --2- 172.21.15.83:0/3093931449 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564f9b0be330 0x564f9b0d5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:09.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.062+0000 7f05bea00640 1 -- 172.21.15.83:0/3093931449 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564f9b0be330 msgr2=0x564f9b0d5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:09.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.062+0000 7f05bea00640 1 --2- 172.21.15.83:0/3093931449 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564f9b0be330 0x564f9b0d5490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:09.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.062+0000 7f05bea00640 1 -- 172.21.15.83:0/3093931449 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f9b0b5950 msgr2=0x564f9b0b80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:09.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.062+0000 7f05bea00640 1 --2- 172.21.15.83:0/3093931449 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f9b0b5950 0x564f9b0b80f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:09.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.062+0000 7f05bea00640 1 -- 172.21.15.83:0/3093931449 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f9adf77d0 con 0x564f9b0b6270 2024-04-24T09:12:09.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.063+0000 7f05bea00640 1 --2- 172.21.15.83:0/3093931449 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564f9b0b6270 0x564f9b0d4f50 secure :-1 s=READY pgs=2537 cs=0 l=1 rev1=1 crypto rx=0x7f05ac004ef0 tx=0x7f05ac000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efe6bedeec6311ad server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:09.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.064+0000 7f05abe00640 1 -- 172.21.15.83:0/3093931449 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05ac00f8c0 con 0x564f9b0b6270 2024-04-24T09:12:09.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.064+0000 7f05abe00640 1 -- 172.21.15.83:0/3093931449 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f05ac028070 con 0x564f9b0b6270 2024-04-24T09:12:09.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.064+0000 7f05abe00640 1 -- 172.21.15.83:0/3093931449 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05ac00fe20 con 0x564f9b0b6270 2024-04-24T09:12:09.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.065+0000 7f05c9456740 1 -- 172.21.15.83:0/3093931449 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564f9b0b6270 msgr2=0x564f9b0d4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:09.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.065+0000 7f05c9456740 1 --2- 172.21.15.83:0/3093931449 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564f9b0b6270 0x564f9b0d4f50 secure :-1 s=READY pgs=2537 cs=0 l=1 rev1=1 crypto rx=0x7f05ac004ef0 tx=0x7f05ac000bc0 comp rx=0 tx=0).stop 2024-04-24T09:12:09.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.065+0000 7f05c9456740 1 -- 172.21.15.83:0/3093931449 shutdown_connections 2024-04-24T09:12:09.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.065+0000 7f05c9456740 1 --2- 172.21.15.83:0/3093931449 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564f9b0be330 0x564f9b0d5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:09.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.065+0000 7f05c9456740 1 --2- 172.21.15.83:0/3093931449 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564f9b0b6270 0x564f9b0d4f50 unknown :-1 s=CLOSED pgs=2537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:09.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.065+0000 7f05c9456740 1 --2- 172.21.15.83:0/3093931449 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f9b0b5950 0x564f9b0b80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:09.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.065+0000 7f05c9456740 1 -- 172.21.15.83:0/3093931449 >> 172.21.15.83:0/3093931449 conn(0x564f9b0b1c70 msgr2=0x564f9b0bd870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:09.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.065+0000 7f05c9456740 1 -- 172.21.15.83:0/3093931449 shutdown_connections 2024-04-24T09:12:09.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.065+0000 7f05c9456740 1 -- 172.21.15.83:0/3093931449 wait complete. 2024-04-24T09:12:09.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.066+0000 7f05c9456740 1 -- start start 2024-04-24T09:12:09.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.066+0000 7f05c9456740 1 Processor -- start 2024-04-24T09:12:09.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.067+0000 7f05c9456740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564f9b0b5950 0x564f9b0d19a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:09.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.067+0000 7f05c9456740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564f9b0b6270 0x564f9b0bafc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:09.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.067+0000 7f05c9456740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f9b0be330 0x564f9b0b9b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:09.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.067+0000 7f05c9456740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f05c01aea40 con 0x564f9b0b5950 2024-04-24T09:12:09.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.067+0000 7f05c9456740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f05c01ae370 con 0x564f9b0be330 2024-04-24T09:12:09.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.067+0000 7f05c9456740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f05c01ab5d0 con 0x564f9b0b6270 2024-04-24T09:12:09.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.067+0000 7f05c9456740 10 client.0 _ll_register_callbacks cb 0x7f05ac01c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:12:09.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.067+0000 7f05bf400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564f9b0b5950 0x564f9b0d19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:09.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.067+0000 7f05bfe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f9b0be330 0x564f9b0b9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:09.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.067+0000 7f05bfe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f9b0be330 0x564f9b0b9b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:59202/0 (socket says 172.21.15.83:59202) 2024-04-24T09:12:09.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.067+0000 7f05bfe00640 1 -- 172.21.15.83:0/930188735 learned_addr learned my addr 172.21.15.83:0/930188735 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:12:09.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.067+0000 7f05bf400640 1 --2- 172.21.15.83:0/930188735 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564f9b0b5950 0x564f9b0d19a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49862/0 (socket says 172.21.15.83:49862) 2024-04-24T09:12:09.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.067+0000 7f05bea00640 1 --2- 172.21.15.83:0/930188735 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564f9b0b6270 0x564f9b0bafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:09.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.068+0000 7f05bfe00640 1 -- 172.21.15.83:0/930188735 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564f9b0b6270 msgr2=0x564f9b0bafc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:09.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.068+0000 7f05bfe00640 1 --2- 172.21.15.83:0/930188735 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564f9b0b6270 0x564f9b0bafc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:09.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.068+0000 7f05bfe00640 1 -- 172.21.15.83:0/930188735 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564f9b0b5950 msgr2=0x564f9b0d19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:09.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.068+0000 7f05bfe00640 1 --2- 172.21.15.83:0/930188735 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564f9b0b5950 0x564f9b0d19a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:09.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.068+0000 7f05bfe00640 1 -- 172.21.15.83:0/930188735 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f9b0ba060 con 0x564f9b0be330 2024-04-24T09:12:09.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.068+0000 7f05bf400640 1 --2- 172.21.15.83:0/930188735 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564f9b0b5950 0x564f9b0d19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:12:09.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.068+0000 7f05bfe00640 1 --2- 172.21.15.83:0/930188735 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f9b0be330 0x564f9b0b9b20 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f05b8018f50 tx=0x7f05b8004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62df8bb6f91eb590 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:09.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.068+0000 7f05a9600640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:12:09.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.069+0000 7f05a9600640 1 -- 172.21.15.83:0/930188735 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b8017cf0 con 0x564f9b0be330 2024-04-24T09:12:09.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.069+0000 7f05c9456740 1 -- 172.21.15.83:0/930188735 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05ac028070 con 0x564f9b0be330 2024-04-24T09:12:09.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.069+0000 7f05a9600640 1 -- 172.21.15.83:0/930188735 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9671+0+0 (secure 0 0 0) 0x7f05b8025470 con 0x564f9b0be330 2024-04-24T09:12:09.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.069+0000 7f05a9600640 1 -- 172.21.15.83:0/930188735 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f05b8033050 con 0x564f9b0be330 2024-04-24T09:12:09.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.069+0000 7f05c9456740 10 client.39683 fetch_fsmap learned FSMap version 61 2024-04-24T09:12:09.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.069+0000 7f05c9456740 1 -- 172.21.15.83:0/930188735 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x564f9b13f360 con 0x564f9b0be330 2024-04-24T09:12:09.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.070+0000 7f05a9600640 1 -- 172.21.15.83:0/930188735 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f05ac028070 con 0x564f9b0be330 2024-04-24T09:12:09.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.070+0000 7f05a9600640 1 -- 172.21.15.83:0/930188735 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 61) ==== 8362+0+0 (secure 0 0 0) 0x7f05b802e5f0 con 0x564f9b0be330 2024-04-24T09:12:09.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.070+0000 7f05c9456740 10 client.39683 fetch_fsmap finished waiting for FSMap version 61 2024-04-24T09:12:09.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.070+0000 7f05c9456740 10 client.39683 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-24T09:12:09.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.070+0000 7f05c9456740 20 client.39683 populate_metadata read hostname 'smithi083' 2024-04-24T09:12:09.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.070+0000 7f05c9456740 1 --2- 172.21.15.83:0/930188735 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x564f9b13fae0 0x564f9b141f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:09.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.070+0000 7f05c9456740 4 client.39683 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:12:09.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.070+0000 7f05c9456740 1 -- 172.21.15.83:0/930188735 --> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564f9b022a60 con 0x564f9b13fae0 2024-04-24T09:12:09.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.071+0000 7f05bf400640 1 --2- 172.21.15.83:0/930188735 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x564f9b13fae0 0x564f9b141f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:09.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.072+0000 7f05bf400640 1 --2- 172.21.15.83:0/930188735 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x564f9b13fae0 0x564f9b141f00 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c2522f9aba2129f9 server_cookie=f6db03a36e2f871c in_seq=0 out_seq=0 2024-04-24T09:12:09.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.072+0000 7f05a9600640 10 client.39683 ms_handle_connect on v2:172.21.15.189:6838/2548965336 2024-04-24T09:12:09.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.074+0000 7f05a9600640 1 -- 172.21.15.83:0/930188735 <== mds.4 v2:172.21.15.189:6838/2548965336 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564f9b022a60 con 0x564f9b13fae0 2024-04-24T09:12:09.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.074+0000 7f05a9600640 10 client.39683 handle_command_reply: tid=0 2024-04-24T09:12:09.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.074+0000 7f05c9456740 1 client.39683 shutdown 2024-04-24T09:12:09.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.075+0000 7f05c9456740 1 -- 172.21.15.83:0/930188735 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f9b0be330 msgr2=0x564f9b0b9b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:09.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.075+0000 7f05c9456740 1 --2- 172.21.15.83:0/930188735 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f9b0be330 0x564f9b0b9b20 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f05b8018f50 tx=0x7f05b8004a40 comp rx=0 tx=0).stop 2024-04-24T09:12:09.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.075+0000 7f05c9456740 1 -- 172.21.15.83:0/930188735 shutdown_connections 2024-04-24T09:12:09.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.075+0000 7f05c9456740 1 --2- 172.21.15.83:0/930188735 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x564f9b13fae0 0x564f9b141f00 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:09.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.075+0000 7f05c9456740 1 --2- 172.21.15.83:0/930188735 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564f9b0be330 0x564f9b0b9b20 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:09.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.075+0000 7f05c9456740 1 --2- 172.21.15.83:0/930188735 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564f9b0b6270 0x564f9b0bafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:09.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.075+0000 7f05c9456740 1 --2- 172.21.15.83:0/930188735 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564f9b0b5950 0x564f9b0d19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:09.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.075+0000 7f05c9456740 1 -- 172.21.15.83:0/930188735 >> 172.21.15.83:0/930188735 conn(0x564f9b0b1c70 msgr2=0x564f9b0bc090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:09.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.075+0000 7f05a9600640 0 client.39683 ms_handle_reset on v2:172.21.15.189:6838/2548965336 2024-04-24T09:12:09.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.076+0000 7f05c9456740 1 -- 172.21.15.83:0/930188735 shutdown_connections 2024-04-24T09:12:09.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.076+0000 7f05c9456740 1 -- 172.21.15.83:0/930188735 wait complete. 2024-04-24T09:12:09.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.076+0000 7f05c9456740 20 client.39683 trim_cache size 0 max 16384 2024-04-24T09:12:09.076 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:12:09.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.078+0000 7f05c6600640 1 -- 172.21.15.83:0/2877040386 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f05a40a3de0 msgr2=0x7f05a40a62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:09.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.078+0000 7f05c6600640 1 --2- 172.21.15.83:0/2877040386 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f05a40a3de0 0x7f05a40a62a0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f05b800b4d0 tx=0x7f05b800e040 comp rx=0 tx=0).stop 2024-04-24T09:12:09.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.078+0000 7f05c6600640 1 -- 172.21.15.83:0/2877040386 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05c0104450 msgr2=0x7f05c01a0780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:09.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.078+0000 7f05c6600640 1 --2- 172.21.15.83:0/2877040386 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05c0104450 0x7f05c01a0780 secure :-1 s=READY pgs=2534 cs=0 l=1 rev1=1 crypto rx=0x7f05b002a8b0 tx=0x7f05b0030d80 comp rx=0 tx=0).stop 2024-04-24T09:12:09.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.078+0000 7f05c6600640 1 -- 172.21.15.83:0/2877040386 shutdown_connections 2024-04-24T09:12:09.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.078+0000 7f05c6600640 1 --2- 172.21.15.83:0/2877040386 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f05a40a3de0 0x7f05a40a62a0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:09.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.078+0000 7f05c6600640 1 --2- 172.21.15.83:0/2877040386 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f05c019a930 0x7f05c019ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:09.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.078+0000 7f05c6600640 1 --2- 172.21.15.83:0/2877040386 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05c0109520 0x7f05c01a0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:09.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.078+0000 7f05c6600640 1 --2- 172.21.15.83:0/2877040386 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05c0104450 0x7f05c01a0780 unknown :-1 s=CLOSED pgs=2534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:09.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.078+0000 7f05c6600640 1 -- 172.21.15.83:0/2877040386 >> 172.21.15.83:0/2877040386 conn(0x7f05c00ffc70 msgr2=0x7f05c0101010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:09.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.079+0000 7f05c6600640 1 -- 172.21.15.83:0/2877040386 shutdown_connections 2024-04-24T09:12:09.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.079+0000 7f05c6600640 1 -- 172.21.15.83:0/2877040386 wait complete. 2024-04-24T09:12:09.099 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-24T09:12:09.184 INFO:tasks.workunit.client.0.smithi083.stdout: 10 50015 6.64 MB/sec execute 280 sec latency 93.949 ms 2024-04-24T09:12:09.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.354+0000 7f71b3e00640 1 -- 172.21.15.83:0/3719857181 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71b41030c0 msgr2=0x7f71b41034a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:09.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.354+0000 7f71b3e00640 1 --2- 172.21.15.83:0/3719857181 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71b41030c0 0x7f71b41034a0 secure :-1 s=READY pgs=2538 cs=0 l=1 rev1=1 crypto rx=0x7f71a4009860 tx=0x7f71a402a3d0 comp rx=0 tx=0).stop 2024-04-24T09:12:09.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.354+0000 7f71b3e00640 1 -- 172.21.15.83:0/3719857181 shutdown_connections 2024-04-24T09:12:09.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.354+0000 7f71b3e00640 1 --2- 172.21.15.83:0/3719857181 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f71b410c490 0x7f71b410e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:09.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.354+0000 7f71b3e00640 1 --2- 172.21.15.83:0/3719857181 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f71b41039e0 0x7f71b410bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:09.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.354+0000 7f71b3e00640 1 --2- 172.21.15.83:0/3719857181 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71b41030c0 0x7f71b41034a0 unknown :-1 s=CLOSED pgs=2538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:09.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.354+0000 7f71b3e00640 1 -- 172.21.15.83:0/3719857181 >> 172.21.15.83:0/3719857181 conn(0x7f71b40fc880 msgr2=0x7f71b40feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:09.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.354+0000 7f71b3e00640 1 -- 172.21.15.83:0/3719857181 shutdown_connections 2024-04-24T09:12:09.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.354+0000 7f71b3e00640 1 -- 172.21.15.83:0/3719857181 wait complete. 2024-04-24T09:12:09.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.355+0000 7f71b3e00640 1 Processor -- start 2024-04-24T09:12:09.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.356+0000 7f71b3e00640 1 -- start start 2024-04-24T09:12:09.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.356+0000 7f71b3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71b41030c0 0x7f71b41a06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:09.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.356+0000 7f71b3e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f71b41039e0 0x7f71b41a0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:09.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.356+0000 7f71b3e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f71b410c490 0x7f71b419a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:09.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.356+0000 7f71b3e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f71b4112140 con 0x7f71b41030c0 2024-04-24T09:12:09.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.356+0000 7f71b3e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f71b4111fc0 con 0x7f71b410c490 2024-04-24T09:12:09.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.356+0000 7f71b3e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f71b41122c0 con 0x7f71b41039e0 2024-04-24T09:12:09.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.356+0000 7f71b2a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71b41030c0 0x7f71b41a06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:09.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.357+0000 7f71b2a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71b41030c0 0x7f71b41a06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49878/0 (socket says 172.21.15.83:49878) 2024-04-24T09:12:09.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.357+0000 7f71b2a00640 1 -- 172.21.15.83:0/1768785772 learned_addr learned my addr 172.21.15.83:0/1768785772 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:12:09.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.357+0000 7f71b3400640 1 --2- 172.21.15.83:0/1768785772 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f71b410c490 0x7f71b419a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:09.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.357+0000 7f71b2000640 1 --2- 172.21.15.83:0/1768785772 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f71b41039e0 0x7f71b41a0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:09.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.357+0000 7f71b2a00640 1 -- 172.21.15.83:0/1768785772 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f71b41039e0 msgr2=0x7f71b41a0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:09.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.357+0000 7f71b2a00640 1 --2- 172.21.15.83:0/1768785772 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f71b41039e0 0x7f71b41a0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:09.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.357+0000 7f71b2a00640 1 -- 172.21.15.83:0/1768785772 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f71b410c490 msgr2=0x7f71b419a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:09.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.357+0000 7f71b2a00640 1 --2- 172.21.15.83:0/1768785772 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f71b410c490 0x7f71b419a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:09.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.357+0000 7f71b2a00640 1 -- 172.21.15.83:0/1768785772 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71b419b100 con 0x7f71b41030c0 2024-04-24T09:12:09.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.358+0000 7f71b2a00640 1 --2- 172.21.15.83:0/1768785772 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71b41030c0 0x7f71b41a06f0 secure :-1 s=READY pgs=2539 cs=0 l=1 rev1=1 crypto rx=0x7f71a4000c00 tx=0x7f71a4005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:09.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.358+0000 7f71ab400640 1 -- 172.21.15.83:0/1768785772 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71a4005640 con 0x7f71b41030c0 2024-04-24T09:12:09.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.358+0000 7f71ab400640 1 -- 172.21.15.83:0/1768785772 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f71a4031400 con 0x7f71b41030c0 2024-04-24T09:12:09.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.358+0000 7f71b3e00640 1 -- 172.21.15.83:0/1768785772 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71b419b390 con 0x7f71b41030c0 2024-04-24T09:12:09.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.358+0000 7f71ab400640 1 -- 172.21.15.83:0/1768785772 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71a40049f0 con 0x7f71b41030c0 2024-04-24T09:12:09.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.359+0000 7f71b3e00640 1 -- 172.21.15.83:0/1768785772 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71b41a76e0 con 0x7f71b41030c0 2024-04-24T09:12:09.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.363+0000 7f71bc9a0740 1 Processor -- start 2024-04-24T09:12:09.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.363+0000 7f71bc9a0740 1 -- start start 2024-04-24T09:12:09.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.364+0000 7f71bc9a0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606e1b4bb40 0x5606e1b4bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:09.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.364+0000 7f71bc9a0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5606e1b4c460 0x5606e1b565f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:09.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.364+0000 7f71bc9a0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606e1b56b30 0x5606e1b58f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:09.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.364+0000 7f71bc9a0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5606e1ab11d0 con 0x5606e1b4bb40 2024-04-24T09:12:09.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.364+0000 7f71bc9a0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5606e1a57450 con 0x5606e1b56b30 2024-04-24T09:12:09.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.364+0000 7f71bc9a0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5606e1aad190 con 0x5606e1b4c460 2024-04-24T09:12:09.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.365+0000 7f71b3400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606e1b4bb40 0x5606e1b4bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:09.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.365+0000 7f71b3400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606e1b4bb40 0x5606e1b4bf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49894/0 (socket says 172.21.15.83:49894) 2024-04-24T09:12:09.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.365+0000 7f71b3400640 1 -- 172.21.15.83:0/1372369862 learned_addr learned my addr 172.21.15.83:0/1372369862 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:12:09.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.365+0000 7f71b2000640 1 --2- 172.21.15.83:0/1372369862 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606e1b56b30 0x5606e1b58f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:09.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.365+0000 7f71b3400640 1 -- 172.21.15.83:0/1372369862 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5606e1b4c460 msgr2=0x5606e1b565f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T09:12:09.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.365+0000 7f71b3400640 1 --2- 172.21.15.83:0/1372369862 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5606e1b4c460 0x5606e1b565f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:09.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.365+0000 7f71b3400640 1 -- 172.21.15.83:0/1372369862 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606e1b56b30 msgr2=0x5606e1b58f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:09.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.365+0000 7f71b3400640 1 --2- 172.21.15.83:0/1372369862 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606e1b56b30 0x5606e1b58f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:09.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.365+0000 7f71b3400640 1 -- 172.21.15.83:0/1372369862 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606e1b14310 con 0x5606e1b4bb40 2024-04-24T09:12:09.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.366+0000 7f71b3400640 1 --2- 172.21.15.83:0/1372369862 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606e1b4bb40 0x5606e1b4bf20 secure :-1 s=READY pgs=2540 cs=0 l=1 rev1=1 crypto rx=0x7f71ac009f70 tx=0x7f71ac010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f911678567050eb5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:09.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.366+0000 7f71ab400640 1 -- 172.21.15.83:0/1768785772 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f71a4039040 con 0x7f71b41030c0 2024-04-24T09:12:09.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.366+0000 7f719fe00640 1 -- 172.21.15.83:0/1372369862 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71ac016070 con 0x5606e1b4bb40 2024-04-24T09:12:09.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.366+0000 7f719fe00640 1 -- 172.21.15.83:0/1372369862 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f71ac004d20 con 0x5606e1b4bb40 2024-04-24T09:12:09.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.366+0000 7f719fe00640 1 -- 172.21.15.83:0/1372369862 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71ac0127d0 con 0x5606e1b4bb40 2024-04-24T09:12:09.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.368+0000 7f71bc9a0740 1 -- 172.21.15.83:0/1372369862 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606e1b4bb40 msgr2=0x5606e1b4bf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:09.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.368+0000 7f71bc9a0740 1 --2- 172.21.15.83:0/1372369862 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606e1b4bb40 0x5606e1b4bf20 secure :-1 s=READY pgs=2540 cs=0 l=1 rev1=1 crypto rx=0x7f71ac009f70 tx=0x7f71ac010800 comp rx=0 tx=0).stop 2024-04-24T09:12:09.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.368+0000 7f71bc9a0740 1 -- 172.21.15.83:0/1372369862 shutdown_connections 2024-04-24T09:12:09.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.368+0000 7f71bc9a0740 1 --2- 172.21.15.83:0/1372369862 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606e1b56b30 0x5606e1b58f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:09.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.368+0000 7f71bc9a0740 1 --2- 172.21.15.83:0/1372369862 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5606e1b4c460 0x5606e1b565f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:09.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.368+0000 7f71bc9a0740 1 --2- 172.21.15.83:0/1372369862 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606e1b4bb40 0x5606e1b4bf20 unknown :-1 s=CLOSED pgs=2540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:09.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.368+0000 7f71bc9a0740 1 -- 172.21.15.83:0/1372369862 >> 172.21.15.83:0/1372369862 conn(0x5606e1b47e60 msgr2=0x5606e1b48240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:09.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.369+0000 7f71bc9a0740 1 -- 172.21.15.83:0/1372369862 shutdown_connections 2024-04-24T09:12:09.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.369+0000 7f71bc9a0740 1 -- 172.21.15.83:0/1372369862 wait complete. 2024-04-24T09:12:09.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.369+0000 7f71bc9a0740 1 -- start start 2024-04-24T09:12:09.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.370+0000 7f71ab400640 1 --2- 172.21.15.83:0/1768785772 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f71980a3fd0 0x7f71980a6490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:09.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.370+0000 7f71bc9a0740 1 Processor -- start 2024-04-24T09:12:09.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.370+0000 7f71ab400640 1 -- 172.21.15.83:0/1768785772 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 9671+0+0 (secure 0 0 0) 0x7f71a40f1b80 con 0x7f71b41030c0 2024-04-24T09:12:09.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.370+0000 7f71b3400640 1 --2- 172.21.15.83:0/1768785772 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f71980a3fd0 0x7f71980a6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:09.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.371+0000 7f71bc9a0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606e1b4bb40 0x5606e1b53950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:09.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.371+0000 7f71bc9a0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5606e1b4c460 0x5606e1b53e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:09.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.371+0000 7f71bc9a0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606e1b56b30 0x5606e1b543d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:09.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.371+0000 7f71bc9a0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5606e1b599c0 con 0x5606e1b4bb40 2024-04-24T09:12:09.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.371+0000 7f71bc9a0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5606e1a6a810 con 0x5606e1b56b30 2024-04-24T09:12:09.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.371+0000 7f71bc9a0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5606e1b59b40 con 0x5606e1b4c460 2024-04-24T09:12:09.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.371+0000 7f71bc9a0740 10 client.0 _ll_register_callbacks cb 0x5606e1a38140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:12:09.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.372+0000 7f71b3400640 1 --2- 172.21.15.83:0/1768785772 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f71980a3fd0 0x7f71980a6490 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f71ac005a80 tx=0x7f71ac005fd0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:09.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.372+0000 7f71b3400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606e1b56b30 0x5606e1b543d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:09.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.372+0000 7f71b2a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606e1b4bb40 0x5606e1b53950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:09.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.372+0000 7f71b3400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606e1b56b30 0x5606e1b543d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:59238/0 (socket says 172.21.15.83:59238) 2024-04-24T09:12:09.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.372+0000 7f71b3400640 1 -- 172.21.15.83:0/42789548 learned_addr learned my addr 172.21.15.83:0/42789548 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:12:09.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.372+0000 7f71b3400640 1 -- 172.21.15.83:0/42789548 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5606e1b4c460 msgr2=0x5606e1b53e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:09.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.372+0000 7f71b3400640 1 --2- 172.21.15.83:0/42789548 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5606e1b4c460 0x5606e1b53e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:09.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.372+0000 7f71b3400640 1 -- 172.21.15.83:0/42789548 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606e1b4bb40 msgr2=0x5606e1b53950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:09.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.372+0000 7f71b3400640 1 --2- 172.21.15.83:0/42789548 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606e1b4bb40 0x5606e1b53950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:09.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.372+0000 7f71b3400640 1 -- 172.21.15.83:0/42789548 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606e1a7b3f0 con 0x5606e1b56b30 2024-04-24T09:12:09.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.372+0000 7f71b2a00640 1 --2- 172.21.15.83:0/42789548 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606e1b4bb40 0x5606e1b53950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:12:09.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.373+0000 7f71b3400640 1 --2- 172.21.15.83:0/42789548 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606e1b56b30 0x5606e1b543d0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f71ac005260 tx=0x7f71ac010fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff9067bec4d8927e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:09.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.373+0000 7f719cc00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:12:09.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.373+0000 7f719cc00640 1 -- 172.21.15.83:0/42789548 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71ac016050 con 0x5606e1b56b30 2024-04-24T09:12:09.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.373+0000 7f71bc9a0740 1 -- 172.21.15.83:0/42789548 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71ac004d20 con 0x5606e1b56b30 2024-04-24T09:12:09.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.373+0000 7f719cc00640 1 -- 172.21.15.83:0/42789548 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9671+0+0 (secure 0 0 0) 0x7f71ac025410 con 0x5606e1b56b30 2024-04-24T09:12:09.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.374+0000 7f719cc00640 1 -- 172.21.15.83:0/42789548 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f71ac03d050 con 0x5606e1b56b30 2024-04-24T09:12:09.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.374+0000 7f71bc9a0740 10 client.39695 fetch_fsmap learned FSMap version 61 2024-04-24T09:12:09.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.374+0000 7f71bc9a0740 1 -- 172.21.15.83:0/42789548 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x5606e17c33f0 con 0x5606e1b56b30 2024-04-24T09:12:09.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.374+0000 7f719cc00640 1 -- 172.21.15.83:0/42789548 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f71ac004d20 con 0x5606e1b56b30 2024-04-24T09:12:09.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.375+0000 7f719cc00640 1 -- 172.21.15.83:0/42789548 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 61) ==== 8362+0+0 (secure 0 0 0) 0x7f71ac00f040 con 0x5606e1b56b30 2024-04-24T09:12:09.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.375+0000 7f71bc9a0740 10 client.39695 fetch_fsmap finished waiting for FSMap version 61 2024-04-24T09:12:09.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.375+0000 7f71bc9a0740 10 client.39695 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-24T09:12:09.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.375+0000 7f71bc9a0740 20 client.39695 populate_metadata read hostname 'smithi083' 2024-04-24T09:12:09.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.375+0000 7f71bc9a0740 1 --2- 172.21.15.83:0/42789548 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x5606e1b6f5f0 0x5606e1b719e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:09.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.375+0000 7f71bc9a0740 4 client.39695 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:12:09.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.375+0000 7f71bc9a0740 1 -- 172.21.15.83:0/42789548 --> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5606e1abebf0 con 0x5606e1b6f5f0 2024-04-24T09:12:09.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.375+0000 7f71b2a00640 1 --2- 172.21.15.83:0/42789548 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x5606e1b6f5f0 0x5606e1b719e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:09.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.376+0000 7f71b2a00640 1 --2- 172.21.15.83:0/42789548 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x5606e1b6f5f0 0x5606e1b719e0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a7eeb77a32e730b6 server_cookie=9d900f77c3914dda in_seq=0 out_seq=0 2024-04-24T09:12:09.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.376+0000 7f719cc00640 10 client.39695 ms_handle_connect on v2:172.21.15.138:6832/1157967658 2024-04-24T09:12:09.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.377+0000 7f719cc00640 1 -- 172.21.15.83:0/42789548 <== mds.3 v2:172.21.15.138:6832/1157967658 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5606e1abebf0 con 0x5606e1b6f5f0 2024-04-24T09:12:09.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.377+0000 7f719cc00640 10 client.39695 handle_command_reply: tid=0 2024-04-24T09:12:09.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.378+0000 7f71bc9a0740 1 client.39695 shutdown 2024-04-24T09:12:09.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.378+0000 7f71bc9a0740 1 -- 172.21.15.83:0/42789548 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606e1b56b30 msgr2=0x5606e1b543d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:09.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.378+0000 7f71bc9a0740 1 --2- 172.21.15.83:0/42789548 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606e1b56b30 0x5606e1b543d0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f71ac005260 tx=0x7f71ac010fd0 comp rx=0 tx=0).stop 2024-04-24T09:12:09.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.379+0000 7f71bc9a0740 1 -- 172.21.15.83:0/42789548 shutdown_connections 2024-04-24T09:12:09.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.379+0000 7f71bc9a0740 1 --2- 172.21.15.83:0/42789548 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606e1b56b30 0x5606e1b543d0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:09.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.379+0000 7f71bc9a0740 1 --2- 172.21.15.83:0/42789548 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x5606e1b6f5f0 0x5606e1b719e0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:09.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.379+0000 7f71bc9a0740 1 --2- 172.21.15.83:0/42789548 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5606e1b4c460 0x5606e1b53e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:09.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.379+0000 7f719cc00640 0 client.39695 ms_handle_reset on v2:172.21.15.138:6832/1157967658 2024-04-24T09:12:09.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.379+0000 7f71bc9a0740 1 --2- 172.21.15.83:0/42789548 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606e1b4bb40 0x5606e1b53950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:09.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.379+0000 7f71bc9a0740 1 -- 172.21.15.83:0/42789548 >> 172.21.15.83:0/42789548 conn(0x5606e1b47e60 msgr2=0x5606e1b4db40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:09.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.379+0000 7f71bc9a0740 1 -- 172.21.15.83:0/42789548 shutdown_connections 2024-04-24T09:12:09.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.379+0000 7f71bc9a0740 1 -- 172.21.15.83:0/42789548 wait complete. 2024-04-24T09:12:09.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.379+0000 7f71bc9a0740 20 client.39695 trim_cache size 0 max 16384 2024-04-24T09:12:09.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.401+0000 7f71bc9a0740 1 Processor -- start 2024-04-24T09:12:09.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.401+0000 7f71bc9a0740 1 -- start start 2024-04-24T09:12:09.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.401+0000 7f71bc9a0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5606e1b4bb40 0x5606e1b4e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:09.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.401+0000 7f71bc9a0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606e1b4c460 0x5606e1b52ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:09.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.401+0000 7f71bc9a0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606e1b56b30 0x5606e1b54020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:09.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.402+0000 7f71bc9a0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f71b41ad280 con 0x5606e1b56b30 2024-04-24T09:12:09.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.402+0000 7f71bc9a0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f71b41aedc0 con 0x5606e1b4c460 2024-04-24T09:12:09.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.402+0000 7f71bc9a0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f71b41ad950 con 0x5606e1b4bb40 2024-04-24T09:12:09.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.402+0000 7f71b3400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606e1b56b30 0x5606e1b54020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:09.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.402+0000 7f71b2000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606e1b4c460 0x5606e1b52ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:09.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.402+0000 7f71b3400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606e1b56b30 0x5606e1b54020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49922/0 (socket says 172.21.15.83:49922) 2024-04-24T09:12:09.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.402+0000 7f71b3400640 1 -- 172.21.15.83:0/961030118 learned_addr learned my addr 172.21.15.83:0/961030118 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:12:09.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.402+0000 7f71b2a00640 1 --2- 172.21.15.83:0/961030118 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5606e1b4bb40 0x5606e1b4e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:09.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.402+0000 7f71b2000640 1 -- 172.21.15.83:0/961030118 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5606e1b4bb40 msgr2=0x5606e1b4e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:09.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.402+0000 7f71b2000640 1 --2- 172.21.15.83:0/961030118 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5606e1b4bb40 0x5606e1b4e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:09.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.402+0000 7f71b2000640 1 -- 172.21.15.83:0/961030118 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606e1b56b30 msgr2=0x5606e1b54020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:09.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.403+0000 7f71b2000640 1 --2- 172.21.15.83:0/961030118 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606e1b56b30 0x5606e1b54020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:09.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.403+0000 7f71b2000640 1 -- 172.21.15.83:0/961030118 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606e188d7d0 con 0x5606e1b4c460 2024-04-24T09:12:09.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.403+0000 7f71b3400640 1 --2- 172.21.15.83:0/961030118 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606e1b56b30 0x5606e1b54020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:12:09.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.403+0000 7f71b2000640 1 --2- 172.21.15.83:0/961030118 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606e1b4c460 0x5606e1b52ad0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x5606e1b531d0 tx=0x7f71a000ff90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66d3597adfe33ea0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:09.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.403+0000 7f719f400640 1 -- 172.21.15.83:0/961030118 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71a0011d10 con 0x5606e1b4c460 2024-04-24T09:12:09.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.403+0000 7f719f400640 1 -- 172.21.15.83:0/961030118 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f71a0016070 con 0x5606e1b4c460 2024-04-24T09:12:09.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.403+0000 7f719f400640 1 -- 172.21.15.83:0/961030118 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71a001c760 con 0x5606e1b4c460 2024-04-24T09:12:09.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.404+0000 7f71bc9a0740 1 -- 172.21.15.83:0/961030118 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606e1b4c460 msgr2=0x5606e1b52ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:09.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.404+0000 7f71bc9a0740 1 --2- 172.21.15.83:0/961030118 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606e1b4c460 0x5606e1b52ad0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x5606e1b531d0 tx=0x7f71a000ff90 comp rx=0 tx=0).stop 2024-04-24T09:12:09.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.404+0000 7f71bc9a0740 1 -- 172.21.15.83:0/961030118 shutdown_connections 2024-04-24T09:12:09.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.404+0000 7f71bc9a0740 1 --2- 172.21.15.83:0/961030118 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606e1b56b30 0x5606e1b54020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:09.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.404+0000 7f71bc9a0740 1 --2- 172.21.15.83:0/961030118 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606e1b4c460 0x5606e1b52ad0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:09.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.404+0000 7f71bc9a0740 1 --2- 172.21.15.83:0/961030118 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5606e1b4bb40 0x5606e1b4e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:09.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.404+0000 7f71bc9a0740 1 -- 172.21.15.83:0/961030118 >> 172.21.15.83:0/961030118 conn(0x5606e1b47e60 msgr2=0x5606e1b53910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:09.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.404+0000 7f71bc9a0740 1 -- 172.21.15.83:0/961030118 shutdown_connections 2024-04-24T09:12:09.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.405+0000 7f71bc9a0740 1 -- 172.21.15.83:0/961030118 wait complete. 2024-04-24T09:12:09.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.405+0000 7f71bc9a0740 1 -- start start 2024-04-24T09:12:09.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.405+0000 7f71bc9a0740 1 Processor -- start 2024-04-24T09:12:09.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.406+0000 7f71bc9a0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606e1b4bb40 0x5606e1b4fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:09.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.406+0000 7f71bc9a0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606e1b4c460 0x5606e1b50260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:09.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.406+0000 7f71bc9a0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5606e1b56b30 0x5606e1b507a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:09.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.406+0000 7f71bc9a0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f71b41ae6f0 con 0x5606e1b4c460 2024-04-24T09:12:09.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.406+0000 7f71bc9a0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f71b41ae020 con 0x5606e1b4bb40 2024-04-24T09:12:09.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.406+0000 7f71bc9a0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f71b41ab2c0 con 0x5606e1b56b30 2024-04-24T09:12:09.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.406+0000 7f71bc9a0740 10 client.0 _ll_register_callbacks cb 0x7f71ac009df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:12:09.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.406+0000 7f71b2a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606e1b4bb40 0x5606e1b4fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:09.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.406+0000 7f71b2a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606e1b4bb40 0x5606e1b4fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:59258/0 (socket says 172.21.15.83:59258) 2024-04-24T09:12:09.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.406+0000 7f71b2a00640 1 -- 172.21.15.83:0/2994596337 learned_addr learned my addr 172.21.15.83:0/2994596337 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:12:09.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.406+0000 7f71b2000640 1 --2- 172.21.15.83:0/2994596337 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606e1b4c460 0x5606e1b50260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:09.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.406+0000 7f71b3400640 1 --2- 172.21.15.83:0/2994596337 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5606e1b56b30 0x5606e1b507a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:09.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.407+0000 7f71b2a00640 1 -- 172.21.15.83:0/2994596337 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5606e1b56b30 msgr2=0x5606e1b507a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:09.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.407+0000 7f71b2a00640 1 --2- 172.21.15.83:0/2994596337 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5606e1b56b30 0x5606e1b507a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:09.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.407+0000 7f71b2a00640 1 -- 172.21.15.83:0/2994596337 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606e1b4c460 msgr2=0x5606e1b50260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:09.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.407+0000 7f71b2a00640 1 --2- 172.21.15.83:0/2994596337 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606e1b4c460 0x5606e1b50260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:09.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.407+0000 7f71b2a00640 1 -- 172.21.15.83:0/2994596337 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606e1b668c0 con 0x5606e1b4bb40 2024-04-24T09:12:09.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.408+0000 7f71b3400640 1 --2- 172.21.15.83:0/2994596337 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5606e1b56b30 0x5606e1b507a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:12:09.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.408+0000 7f71b2000640 1 --2- 172.21.15.83:0/2994596337 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606e1b4c460 0x5606e1b50260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:12:09.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.408+0000 7f71b2a00640 1 --2- 172.21.15.83:0/2994596337 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606e1b4bb40 0x5606e1b4fd20 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f71a40eee50 tx=0x7f71a40b92a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b479dcc80736f020 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:09.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.408+0000 7f719cc00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:12:09.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.408+0000 7f719cc00640 1 -- 172.21.15.83:0/2994596337 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71a40c62b0 con 0x5606e1b4bb40 2024-04-24T09:12:09.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.409+0000 7f719cc00640 1 -- 172.21.15.83:0/2994596337 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9671+0+0 (secure 0 0 0) 0x7f71a40ef070 con 0x5606e1b4bb40 2024-04-24T09:12:09.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.409+0000 7f71bc9a0740 1 -- 172.21.15.83:0/2994596337 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71a0016070 con 0x5606e1b4bb40 2024-04-24T09:12:09.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.415+0000 7f719cc00640 1 -- 172.21.15.83:0/2994596337 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f71a40ef270 con 0x5606e1b4bb40 2024-04-24T09:12:09.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.415+0000 7f71bc9a0740 10 client.39701 fetch_fsmap learned FSMap version 61 2024-04-24T09:12:09.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.415+0000 7f71bc9a0740 1 -- 172.21.15.83:0/2994596337 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x5606e1b67bc0 con 0x5606e1b4bb40 2024-04-24T09:12:09.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.415+0000 7f719cc00640 1 -- 172.21.15.83:0/2994596337 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f71a0016070 con 0x5606e1b4bb40 2024-04-24T09:12:09.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.415+0000 7f719cc00640 1 -- 172.21.15.83:0/2994596337 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 61) ==== 8362+0+0 (secure 0 0 0) 0x7f71a404c030 con 0x5606e1b4bb40 2024-04-24T09:12:09.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.415+0000 7f71bc9a0740 10 client.39701 fetch_fsmap finished waiting for FSMap version 61 2024-04-24T09:12:09.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.416+0000 7f71bc9a0740 10 client.39701 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-24T09:12:09.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.416+0000 7f71bc9a0740 20 client.39701 populate_metadata read hostname 'smithi083' 2024-04-24T09:12:09.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.416+0000 7f71bc9a0740 1 --2- 172.21.15.83:0/2994596337 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x5606e1bd5b40 0x5606e1bd7f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:09.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.416+0000 7f71bc9a0740 4 client.39701 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:12:09.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.416+0000 7f71bc9a0740 1 -- 172.21.15.83:0/2994596337 --> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5606e1ab8a60 con 0x5606e1bd5b40 2024-04-24T09:12:09.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.416+0000 7f71b2000640 1 --2- 172.21.15.83:0/2994596337 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x5606e1bd5b40 0x5606e1bd7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:09.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.417+0000 7f71b2000640 1 --2- 172.21.15.83:0/2994596337 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x5606e1bd5b40 0x5606e1bd7f30 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6ac418e1356c09cc server_cookie=56ce60783306ebf9 in_seq=0 out_seq=0 2024-04-24T09:12:09.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.417+0000 7f719cc00640 10 client.39701 ms_handle_connect on v2:172.21.15.138:6832/1157967658 2024-04-24T09:12:09.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.418+0000 7f719cc00640 1 -- 172.21.15.83:0/2994596337 <== mds.3 v2:172.21.15.138:6832/1157967658 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5606e1ab8a60 con 0x5606e1bd5b40 2024-04-24T09:12:09.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.418+0000 7f719cc00640 10 client.39701 handle_command_reply: tid=0 2024-04-24T09:12:09.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.418+0000 7f71bc9a0740 1 client.39701 shutdown 2024-04-24T09:12:09.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.418+0000 7f71bc9a0740 1 -- 172.21.15.83:0/2994596337 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606e1b4bb40 msgr2=0x5606e1b4fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:09.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.418+0000 7f71bc9a0740 1 --2- 172.21.15.83:0/2994596337 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606e1b4bb40 0x5606e1b4fd20 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f71a40eee50 tx=0x7f71a40b92a0 comp rx=0 tx=0).stop 2024-04-24T09:12:09.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.418+0000 7f71bc9a0740 1 -- 172.21.15.83:0/2994596337 shutdown_connections 2024-04-24T09:12:09.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.418+0000 7f71bc9a0740 1 --2- 172.21.15.83:0/2994596337 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x5606e1bd5b40 0x5606e1bd7f30 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:09.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.418+0000 7f71bc9a0740 1 --2- 172.21.15.83:0/2994596337 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5606e1b56b30 0x5606e1b507a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:09.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.418+0000 7f71bc9a0740 1 --2- 172.21.15.83:0/2994596337 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5606e1b4c460 0x5606e1b50260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:09.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.418+0000 7f71bc9a0740 1 --2- 172.21.15.83:0/2994596337 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606e1b4bb40 0x5606e1b4fd20 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:09.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.418+0000 7f719cc00640 0 client.39701 ms_handle_reset on v2:172.21.15.138:6832/1157967658 2024-04-24T09:12:09.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.418+0000 7f71bc9a0740 1 -- 172.21.15.83:0/2994596337 >> 172.21.15.83:0/2994596337 conn(0x5606e1b47e60 msgr2=0x5606e1b52950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:09.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.418+0000 7f71bc9a0740 1 -- 172.21.15.83:0/2994596337 shutdown_connections 2024-04-24T09:12:09.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.419+0000 7f71bc9a0740 1 -- 172.21.15.83:0/2994596337 wait complete. 2024-04-24T09:12:09.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.419+0000 7f71bc9a0740 20 client.39701 trim_cache size 0 max 16384 2024-04-24T09:12:09.419 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:12:09.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.420+0000 7f71b3e00640 1 -- 172.21.15.83:0/1768785772 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f71980a3fd0 msgr2=0x7f71980a6490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:09.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.420+0000 7f71b3e00640 1 --2- 172.21.15.83:0/1768785772 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f71980a3fd0 0x7f71980a6490 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f71ac005a80 tx=0x7f71ac005fd0 comp rx=0 tx=0).stop 2024-04-24T09:12:09.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.420+0000 7f71b3e00640 1 -- 172.21.15.83:0/1768785772 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71b41030c0 msgr2=0x7f71b41a06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:09.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.420+0000 7f71b3e00640 1 --2- 172.21.15.83:0/1768785772 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71b41030c0 0x7f71b41a06f0 secure :-1 s=READY pgs=2539 cs=0 l=1 rev1=1 crypto rx=0x7f71a4000c00 tx=0x7f71a4005040 comp rx=0 tx=0).stop 2024-04-24T09:12:09.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.420+0000 7f71b3e00640 1 -- 172.21.15.83:0/1768785772 shutdown_connections 2024-04-24T09:12:09.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.420+0000 7f71b3e00640 1 --2- 172.21.15.83:0/1768785772 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f71980a3fd0 0x7f71980a6490 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:09.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.420+0000 7f71b3e00640 1 --2- 172.21.15.83:0/1768785772 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f71b410c490 0x7f71b419a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:09.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.420+0000 7f71b3e00640 1 --2- 172.21.15.83:0/1768785772 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f71b41039e0 0x7f71b41a0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:09.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.420+0000 7f71b3e00640 1 --2- 172.21.15.83:0/1768785772 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71b41030c0 0x7f71b41a06f0 unknown :-1 s=CLOSED pgs=2539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:09.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.420+0000 7f71b3e00640 1 -- 172.21.15.83:0/1768785772 >> 172.21.15.83:0/1768785772 conn(0x7f71b40fc880 msgr2=0x7f71b4101f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:09.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.420+0000 7f71b3e00640 1 -- 172.21.15.83:0/1768785772 shutdown_connections 2024-04-24T09:12:09.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.420+0000 7f71b3e00640 1 -- 172.21.15.83:0/1768785772 wait complete. 2024-04-24T09:12:09.440 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T09:12:09.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.698+0000 7f24cb600640 1 -- 172.21.15.83:0/844408259 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24c4109580 msgr2=0x7f24c4111150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:09.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.698+0000 7f24cb600640 1 --2- 172.21.15.83:0/844408259 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24c4109580 0x7f24c4111150 secure :-1 s=READY pgs=2541 cs=0 l=1 rev1=1 crypto rx=0x7f24bc009ca0 tx=0x7f24bc02a8c0 comp rx=0 tx=0).stop 2024-04-24T09:12:09.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.698+0000 7f24cb600640 1 -- 172.21.15.83:0/844408259 shutdown_connections 2024-04-24T09:12:09.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.698+0000 7f24cb600640 1 --2- 172.21.15.83:0/844408259 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24c4109580 0x7f24c4111150 unknown :-1 s=CLOSED pgs=2541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:09.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.698+0000 7f24cb600640 1 --2- 172.21.15.83:0/844408259 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f24c4104e60 0x7f24c4108ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:09.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.698+0000 7f24cb600640 1 --2- 172.21.15.83:0/844408259 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f24c41044b0 0x7f24c4104890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:09.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.698+0000 7f24cb600640 1 -- 172.21.15.83:0/844408259 >> 172.21.15.83:0/844408259 conn(0x7f24c40ffd90 msgr2=0x7f24c41021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:09.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.699+0000 7f24cb600640 1 -- 172.21.15.83:0/844408259 shutdown_connections 2024-04-24T09:12:09.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.699+0000 7f24cb600640 1 -- 172.21.15.83:0/844408259 wait complete. 2024-04-24T09:12:09.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.700+0000 7f24cb600640 1 Processor -- start 2024-04-24T09:12:09.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.700+0000 7f24cb600640 1 -- start start 2024-04-24T09:12:09.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.700+0000 7f24cb600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f24c41044b0 0x7f24c410fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:09.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.700+0000 7f24cb600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24c4109580 0x7f24c4110170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:09.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.700+0000 7f24cb600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f24c41a8a10 0x7f24c41aae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:09.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.700+0000 7f24cb600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f24c40774d0 con 0x7f24c4109580 2024-04-24T09:12:09.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.700+0000 7f24cb600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f24c4077350 con 0x7f24c41a8a10 2024-04-24T09:12:09.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.700+0000 7f24cb600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f24c4077650 con 0x7f24c41044b0 2024-04-24T09:12:09.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.701+0000 7f24c3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24c4109580 0x7f24c4110170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:09.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.701+0000 7f24c9600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f24c41a8a10 0x7f24c41aae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:09.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.701+0000 7f24c9600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f24c41a8a10 0x7f24c41aae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:59282/0 (socket says 172.21.15.83:59282) 2024-04-24T09:12:09.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.701+0000 7f24c9600640 1 -- 172.21.15.83:0/728053006 learned_addr learned my addr 172.21.15.83:0/728053006 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:12:09.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.701+0000 7f24c8c00640 1 --2- 172.21.15.83:0/728053006 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f24c41044b0 0x7f24c410fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:09.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.701+0000 7f24c9600640 1 -- 172.21.15.83:0/728053006 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f24c41044b0 msgr2=0x7f24c410fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:09.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.701+0000 7f24c9600640 1 --2- 172.21.15.83:0/728053006 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f24c41044b0 0x7f24c410fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:09.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.701+0000 7f24c9600640 1 -- 172.21.15.83:0/728053006 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24c4109580 msgr2=0x7f24c4110170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:09.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.701+0000 7f24c9600640 1 --2- 172.21.15.83:0/728053006 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24c4109580 0x7f24c4110170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:09.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.701+0000 7f24c9600640 1 -- 172.21.15.83:0/728053006 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24c41ab500 con 0x7f24c41a8a10 2024-04-24T09:12:09.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.702+0000 7f24c9600640 1 --2- 172.21.15.83:0/728053006 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f24c41a8a10 0x7f24c41aae00 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f24bc031040 tx=0x7f24bc032f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:09.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.702+0000 7f24c1600640 1 -- 172.21.15.83:0/728053006 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24bc003b70 con 0x7f24c41a8a10 2024-04-24T09:12:09.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.702+0000 7f24cb600640 1 -- 172.21.15.83:0/728053006 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24c41ab790 con 0x7f24c41a8a10 2024-04-24T09:12:09.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.702+0000 7f24cb600640 1 -- 172.21.15.83:0/728053006 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24c41abd30 con 0x7f24c41a8a10 2024-04-24T09:12:09.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.702+0000 7f24c1600640 1 -- 172.21.15.83:0/728053006 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f24bc003d10 con 0x7f24c41a8a10 2024-04-24T09:12:09.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.702+0000 7f24c1600640 1 -- 172.21.15.83:0/728053006 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24bc0308e0 con 0x7f24c41a8a10 2024-04-24T09:12:09.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.705+0000 7f24ce47e740 1 Processor -- start 2024-04-24T09:12:09.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:12:09 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/1455298047' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:12:09.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.707+0000 7f24c1600640 1 -- 172.21.15.83:0/728053006 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f24bc030a80 con 0x7f24c41a8a10 2024-04-24T09:12:09.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.707+0000 7f24ce47e740 1 -- start start 2024-04-24T09:12:09.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:12:09 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/1455298047' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:12:09.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.708+0000 7f24ce47e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a20be03950 0x55a20be03d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:09.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.708+0000 7f24ce47e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a20be04270 0x55a20be0e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:09.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.708+0000 7f24ce47e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a20be0e940 0x55a20be10d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:09.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.708+0000 7f24ce47e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a20bd691d0 con 0x55a20be03950 2024-04-24T09:12:09.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.708+0000 7f24ce47e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a20bd0f450 con 0x55a20be0e940 2024-04-24T09:12:09.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.708+0000 7f24ce47e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a20bd65190 con 0x55a20be04270 2024-04-24T09:12:09.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.708+0000 7f24c3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a20be03950 0x55a20be03d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:09.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.709+0000 7f24c3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a20be03950 0x55a20be03d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49960/0 (socket says 172.21.15.83:49960) 2024-04-24T09:12:09.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.709+0000 7f24c3e00640 1 -- 172.21.15.83:0/102568286 learned_addr learned my addr 172.21.15.83:0/102568286 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:12:09.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.709+0000 7f24c8c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a20be04270 0x55a20be0e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:09.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.709+0000 7f24c3e00640 1 --2- 172.21.15.83:0/102568286 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a20be0e940 0x55a20be10d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:09.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.710+0000 7f24c1600640 1 --2- 172.21.15.83:0/728053006 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f24ac0a3f50 0x7f24ac0a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:09.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.710+0000 7f24c8c00640 1 -- 172.21.15.83:0/102568286 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a20be0e940 msgr2=0x55a20be10d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:09.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.710+0000 7f24c8c00640 1 --2- 172.21.15.83:0/102568286 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a20be0e940 0x55a20be10d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:09.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.710+0000 7f24c8c00640 1 -- 172.21.15.83:0/102568286 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a20be03950 msgr2=0x55a20be03d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:09.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.710+0000 7f24c8c00640 1 --2- 172.21.15.83:0/102568286 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a20be03950 0x55a20be03d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:09.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.710+0000 7f24c8c00640 1 -- 172.21.15.83:0/102568286 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a20bdcc310 con 0x55a20be04270 2024-04-24T09:12:09.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.710+0000 7f24c1600640 1 -- 172.21.15.83:0/728053006 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 9671+0+0 (secure 0 0 0) 0x7f24bc0306a0 con 0x7f24c41a8a10 2024-04-24T09:12:09.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.710+0000 7f24c3e00640 1 --2- 172.21.15.83:0/102568286 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a20be03950 0x55a20be03d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:12:09.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.711+0000 7f24c9600640 1 --2- 172.21.15.83:0/728053006 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f24ac0a3f50 0x7f24ac0a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:09.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.711+0000 7f24c8c00640 1 --2- 172.21.15.83:0/102568286 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a20be04270 0x55a20be0e400 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f24b8009b70 tx=0x7f24b80117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4edc40c7810de7c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:09.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.712+0000 7f24b5c00640 1 -- 172.21.15.83:0/102568286 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b8009e60 con 0x55a20be04270 2024-04-24T09:12:09.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.712+0000 7f24b5c00640 1 -- 172.21.15.83:0/102568286 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f24b8012900 con 0x55a20be04270 2024-04-24T09:12:09.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.712+0000 7f24b5c00640 1 -- 172.21.15.83:0/102568286 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b800f9a0 con 0x55a20be04270 2024-04-24T09:12:09.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.712+0000 7f24c9600640 1 --2- 172.21.15.83:0/728053006 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f24ac0a3f50 0x7f24ac0a6410 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f24bc0bd900 tx=0x7f24bc0095c0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:09.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.713+0000 7f24ce47e740 1 -- 172.21.15.83:0/102568286 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a20be04270 msgr2=0x55a20be0e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:09.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.713+0000 7f24ce47e740 1 --2- 172.21.15.83:0/102568286 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a20be04270 0x55a20be0e400 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f24b8009b70 tx=0x7f24b80117b0 comp rx=0 tx=0).stop 2024-04-24T09:12:09.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.713+0000 7f24ce47e740 1 -- 172.21.15.83:0/102568286 shutdown_connections 2024-04-24T09:12:09.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.714+0000 7f24ce47e740 1 --2- 172.21.15.83:0/102568286 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a20be0e940 0x55a20be10d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:09.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.714+0000 7f24ce47e740 1 --2- 172.21.15.83:0/102568286 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a20be04270 0x55a20be0e400 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:09.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.714+0000 7f24ce47e740 1 --2- 172.21.15.83:0/102568286 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a20be03950 0x55a20be03d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:09.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.714+0000 7f24ce47e740 1 -- 172.21.15.83:0/102568286 >> 172.21.15.83:0/102568286 conn(0x55a20bdffc70 msgr2=0x55a20be00050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:09.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.714+0000 7f24ce47e740 1 -- 172.21.15.83:0/102568286 shutdown_connections 2024-04-24T09:12:09.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.714+0000 7f24ce47e740 1 -- 172.21.15.83:0/102568286 wait complete. 2024-04-24T09:12:09.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.714+0000 7f24ce47e740 1 -- start start 2024-04-24T09:12:09.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.715+0000 7f24ce47e740 1 Processor -- start 2024-04-24T09:12:09.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.715+0000 7f24ce47e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a20be03950 0x55a20be0e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:09.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.715+0000 7f24ce47e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a20be04270 0x55a20be0b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:09.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.715+0000 7f24ce47e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a20be0e940 0x55a20be0bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:09.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.716+0000 7f24ce47e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a20be117d0 con 0x55a20be04270 2024-04-24T09:12:09.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.716+0000 7f24c3e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a20be03950 0x55a20be0e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:09.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.716+0000 7f24ce47e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a20bd22810 con 0x55a20be03950 2024-04-24T09:12:09.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.716+0000 7f24ce47e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a20be11950 con 0x55a20be0e940 2024-04-24T09:12:09.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.716+0000 7f24ce47e740 10 client.0 _ll_register_callbacks cb 0x55a20bcf0140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:12:09.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.716+0000 7f24c3e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a20be03950 0x55a20be0e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:59306/0 (socket says 172.21.15.83:59306) 2024-04-24T09:12:09.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.716+0000 7f24c3e00640 1 -- 172.21.15.83:0/2221393580 learned_addr learned my addr 172.21.15.83:0/2221393580 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:12:09.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.716+0000 7f24c8c00640 1 --2- 172.21.15.83:0/2221393580 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a20be04270 0x55a20be0b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:09.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.717+0000 7f24c3e00640 1 --2- 172.21.15.83:0/2221393580 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a20be0e940 0x55a20be0bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:09.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.717+0000 7f24c3e00640 1 -- 172.21.15.83:0/2221393580 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a20be0e940 msgr2=0x55a20be0bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:09.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.717+0000 7f24c3e00640 1 --2- 172.21.15.83:0/2221393580 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a20be0e940 0x55a20be0bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:09.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.717+0000 7f24c3e00640 1 -- 172.21.15.83:0/2221393580 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a20be04270 msgr2=0x55a20be0b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:09.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.717+0000 7f24c3e00640 1 --2- 172.21.15.83:0/2221393580 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a20be04270 0x55a20be0b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:09.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.717+0000 7f24c3e00640 1 -- 172.21.15.83:0/2221393580 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a20bd333f0 con 0x55a20be03950 2024-04-24T09:12:09.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.717+0000 7f24c8c00640 1 --2- 172.21.15.83:0/2221393580 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a20be04270 0x55a20be0b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:12:09.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.717+0000 7f24c3e00640 1 --2- 172.21.15.83:0/2221393580 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a20be03950 0x55a20be0e390 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f24b0004650 tx=0x7f24b0009770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8de5520d5d487a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:09.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.717+0000 7f24aa000640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:12:09.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.718+0000 7f24aa000640 1 -- 172.21.15.83:0/2221393580 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b0005020 con 0x55a20be03950 2024-04-24T09:12:09.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.718+0000 7f24aa000640 1 -- 172.21.15.83:0/2221393580 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9671+0+0 (secure 0 0 0) 0x7f24b00051c0 con 0x55a20be03950 2024-04-24T09:12:09.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.718+0000 7f24ce47e740 1 -- 172.21.15.83:0/2221393580 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24b8012900 con 0x55a20be03950 2024-04-24T09:12:09.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.719+0000 7f24aa000640 1 -- 172.21.15.83:0/2221393580 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f24b0028050 con 0x55a20be03950 2024-04-24T09:12:09.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.719+0000 7f24ce47e740 10 client.39713 fetch_fsmap learned FSMap version 61 2024-04-24T09:12:09.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.719+0000 7f24ce47e740 1 -- 172.21.15.83:0/2221393580 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x55a20ba7b3f0 con 0x55a20be03950 2024-04-24T09:12:09.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.720+0000 7f24aa000640 1 -- 172.21.15.83:0/2221393580 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f24b8012900 con 0x55a20be03950 2024-04-24T09:12:09.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.720+0000 7f24aa000640 1 -- 172.21.15.83:0/2221393580 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 61) ==== 8362+0+0 (secure 0 0 0) 0x7f24b000e470 con 0x55a20be03950 2024-04-24T09:12:09.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.720+0000 7f24ce47e740 10 client.39713 fetch_fsmap finished waiting for FSMap version 61 2024-04-24T09:12:09.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.720+0000 7f24ce47e740 10 client.39713 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-24T09:12:09.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.720+0000 7f24ce47e740 20 client.39713 populate_metadata read hostname 'smithi083' 2024-04-24T09:12:09.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.720+0000 7f24ce47e740 1 --2- 172.21.15.83:0/2221393580 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55a20be1eca0 0x55a20be21120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:09.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.720+0000 7f24ce47e740 4 client.39713 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:12:09.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.720+0000 7f24ce47e740 1 -- 172.21.15.83:0/2221393580 --> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a20bd76bf0 con 0x55a20be1eca0 2024-04-24T09:12:09.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.721+0000 7f24c8c00640 1 --2- 172.21.15.83:0/2221393580 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55a20be1eca0 0x55a20be21120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:09.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.721+0000 7f24c8c00640 1 --2- 172.21.15.83:0/2221393580 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55a20be1eca0 0x55a20be21120 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d8c5a2b64b67db19 server_cookie=950339d03986cc2a in_seq=0 out_seq=0 2024-04-24T09:12:09.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.721+0000 7f24aa000640 10 client.39713 ms_handle_connect on v2:172.21.15.138:6834/1909219758 2024-04-24T09:12:09.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.722+0000 7f24aa000640 1 -- 172.21.15.83:0/2221393580 <== mds.2 v2:172.21.15.138:6834/1909219758 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a20bd76bf0 con 0x55a20be1eca0 2024-04-24T09:12:09.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.722+0000 7f24aa000640 10 client.39713 handle_command_reply: tid=0 2024-04-24T09:12:09.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.722+0000 7f24ce47e740 1 client.39713 shutdown 2024-04-24T09:12:09.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.723+0000 7f24ce47e740 1 -- 172.21.15.83:0/2221393580 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a20be03950 msgr2=0x55a20be0e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:09.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.723+0000 7f24ce47e740 1 --2- 172.21.15.83:0/2221393580 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a20be03950 0x55a20be0e390 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f24b0004650 tx=0x7f24b0009770 comp rx=0 tx=0).stop 2024-04-24T09:12:09.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.723+0000 7f24ce47e740 1 -- 172.21.15.83:0/2221393580 shutdown_connections 2024-04-24T09:12:09.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.723+0000 7f24ce47e740 1 --2- 172.21.15.83:0/2221393580 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55a20be1eca0 0x55a20be21120 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:09.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.723+0000 7f24ce47e740 1 --2- 172.21.15.83:0/2221393580 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a20be0e940 0x55a20be0bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:09.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.723+0000 7f24aa000640 0 client.39713 ms_handle_reset on v2:172.21.15.138:6834/1909219758 2024-04-24T09:12:09.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.723+0000 7f24ce47e740 1 --2- 172.21.15.83:0/2221393580 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a20be04270 0x55a20be0b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:09.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.723+0000 7f24ce47e740 1 --2- 172.21.15.83:0/2221393580 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a20be03950 0x55a20be0e390 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:09.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.723+0000 7f24ce47e740 1 -- 172.21.15.83:0/2221393580 >> 172.21.15.83:0/2221393580 conn(0x55a20bdffc70 msgr2=0x55a20be05970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:09.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.723+0000 7f24ce47e740 1 -- 172.21.15.83:0/2221393580 shutdown_connections 2024-04-24T09:12:09.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.723+0000 7f24ce47e740 1 -- 172.21.15.83:0/2221393580 wait complete. 2024-04-24T09:12:09.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.723+0000 7f24ce47e740 20 client.39713 trim_cache size 0 max 16384 2024-04-24T09:12:09.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.752+0000 7f24ce47e740 1 Processor -- start 2024-04-24T09:12:09.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.752+0000 7f24ce47e740 1 -- start start 2024-04-24T09:12:09.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.752+0000 7f24ce47e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a20be03950 0x55a20bdff570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:09.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.752+0000 7f24ce47e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a20be04270 0x55a20be0dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:09.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.752+0000 7f24ce47e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a20be0e940 0x55a20be0e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:09.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.753+0000 7f24ce47e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f24a4010220 con 0x55a20be03950 2024-04-24T09:12:09.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.753+0000 7f24ce47e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f24a4011d60 con 0x55a20be0e940 2024-04-24T09:12:09.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.753+0000 7f24ce47e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f24a40108f0 con 0x55a20be04270 2024-04-24T09:12:09.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.754+0000 7f24c8c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a20be04270 0x55a20be0dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:09.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.754+0000 7f24c3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a20be03950 0x55a20bdff570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:09.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.754+0000 7f24c8c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a20be04270 0x55a20be0dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:59824/0 (socket says 172.21.15.83:59824) 2024-04-24T09:12:09.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.754+0000 7f24c8c00640 1 -- 172.21.15.83:0/486765130 learned_addr learned my addr 172.21.15.83:0/486765130 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:12:09.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.754+0000 7f24c8c00640 1 -- 172.21.15.83:0/486765130 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a20be0e940 msgr2=0x55a20be0e400 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-24T09:12:09.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.754+0000 7f24c8c00640 1 --2- 172.21.15.83:0/486765130 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a20be0e940 0x55a20be0e400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:09.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.754+0000 7f24c8c00640 1 -- 172.21.15.83:0/486765130 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a20be03950 msgr2=0x55a20bdff570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:09.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.754+0000 7f24c8c00640 1 --2- 172.21.15.83:0/486765130 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a20be03950 0x55a20bdff570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:09.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.754+0000 7f24c8c00640 1 -- 172.21.15.83:0/486765130 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a20bb457d0 con 0x55a20be04270 2024-04-24T09:12:09.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.755+0000 7f24c8c00640 1 --2- 172.21.15.83:0/486765130 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a20be04270 0x55a20be0dec0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f24b8009900 tx=0x7f24b800f4f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8b2395bf6e162d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:09.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.757+0000 7f24b5200640 1 -- 172.21.15.83:0/486765130 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b8005240 con 0x55a20be04270 2024-04-24T09:12:09.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.757+0000 7f24b5200640 1 -- 172.21.15.83:0/486765130 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f24b80053e0 con 0x55a20be04270 2024-04-24T09:12:09.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.757+0000 7f24b5200640 1 -- 172.21.15.83:0/486765130 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b8012810 con 0x55a20be04270 2024-04-24T09:12:09.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.759+0000 7f24ce47e740 1 -- 172.21.15.83:0/486765130 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a20be04270 msgr2=0x55a20be0dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:09.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.759+0000 7f24ce47e740 1 --2- 172.21.15.83:0/486765130 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a20be04270 0x55a20be0dec0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f24b8009900 tx=0x7f24b800f4f0 comp rx=0 tx=0).stop 2024-04-24T09:12:09.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.759+0000 7f24ce47e740 1 -- 172.21.15.83:0/486765130 shutdown_connections 2024-04-24T09:12:09.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.759+0000 7f24ce47e740 1 --2- 172.21.15.83:0/486765130 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a20be0e940 0x55a20be0e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:09.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.759+0000 7f24ce47e740 1 --2- 172.21.15.83:0/486765130 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a20be04270 0x55a20be0dec0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:09.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.759+0000 7f24ce47e740 1 --2- 172.21.15.83:0/486765130 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a20be03950 0x55a20bdff570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:09.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.759+0000 7f24ce47e740 1 -- 172.21.15.83:0/486765130 >> 172.21.15.83:0/486765130 conn(0x55a20bdffc70 msgr2=0x55a20be068e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:09.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.760+0000 7f24ce47e740 1 -- 172.21.15.83:0/486765130 shutdown_connections 2024-04-24T09:12:09.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.760+0000 7f24ce47e740 1 -- 172.21.15.83:0/486765130 wait complete. 2024-04-24T09:12:09.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.760+0000 7f24ce47e740 1 -- start start 2024-04-24T09:12:09.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.761+0000 7f24ce47e740 1 Processor -- start 2024-04-24T09:12:09.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.761+0000 7f24ce47e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a20be03950 0x55a20be17030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:09.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.761+0000 7f24ce47e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a20be04270 0x55a20be0f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:09.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.761+0000 7f24ce47e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a20be0e940 0x55a20be0f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:09.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.761+0000 7f24ce47e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f24a4011690 con 0x55a20be04270 2024-04-24T09:12:09.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.761+0000 7f24ce47e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f24a4010fc0 con 0x55a20be0e940 2024-04-24T09:12:09.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.761+0000 7f24ce47e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f24a400e220 con 0x55a20be03950 2024-04-24T09:12:09.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.762+0000 7f24ce47e740 10 client.0 _ll_register_callbacks cb 0x7f24b0009e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:12:09.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.762+0000 7f24c3e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a20be03950 0x55a20be17030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:09.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.762+0000 7f24c3e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a20be03950 0x55a20be17030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:59840/0 (socket says 172.21.15.83:59840) 2024-04-24T09:12:09.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.762+0000 7f24c3e00640 1 -- 172.21.15.83:0/820493352 learned_addr learned my addr 172.21.15.83:0/820493352 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:12:09.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.762+0000 7f24c8c00640 1 --2- 172.21.15.83:0/820493352 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a20be04270 0x55a20be0f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:09.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.763+0000 7f24c3e00640 1 --2- 172.21.15.83:0/820493352 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a20be0e940 0x55a20be0f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:09.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.764+0000 7f24c8c00640 1 -- 172.21.15.83:0/820493352 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a20be03950 msgr2=0x55a20be17030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:09.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.764+0000 7f24c8c00640 1 --2- 172.21.15.83:0/820493352 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a20be03950 0x55a20be17030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:09.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.764+0000 7f24c8c00640 1 -- 172.21.15.83:0/820493352 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a20be0e940 msgr2=0x55a20be0f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:09.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.764+0000 7f24c8c00640 1 --2- 172.21.15.83:0/820493352 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a20be0e940 0x55a20be0f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:09.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.764+0000 7f24c8c00640 1 -- 172.21.15.83:0/820493352 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a20be10170 con 0x55a20be04270 2024-04-24T09:12:09.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.764+0000 7f24c3e00640 1 --2- 172.21.15.83:0/820493352 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a20be03950 0x55a20be17030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:12:09.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.765+0000 7f24c8c00640 1 --2- 172.21.15.83:0/820493352 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a20be04270 0x55a20be0f400 secure :-1 s=READY pgs=2542 cs=0 l=1 rev1=1 crypto rx=0x55a20be0b980 tx=0x7f24b80056f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a3ee7cfe6d0275a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:09.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.765+0000 7f24aa000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:12:09.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.765+0000 7f24aa000640 1 -- 172.21.15.83:0/820493352 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b80035f0 con 0x55a20be04270 2024-04-24T09:12:09.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.766+0000 7f24aa000640 1 -- 172.21.15.83:0/820493352 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9583+0+0 (secure 0 0 0) 0x7f24b8003790 con 0x55a20be04270 2024-04-24T09:12:09.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.766+0000 7f24ce47e740 1 -- 172.21.15.83:0/820493352 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24b80053e0 con 0x55a20be04270 2024-04-24T09:12:09.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.766+0000 7f24aa000640 1 -- 172.21.15.83:0/820493352 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f24b803d050 con 0x55a20be04270 2024-04-24T09:12:09.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.766+0000 7f24ce47e740 10 client.22674 fetch_fsmap learned FSMap version 61 2024-04-24T09:12:09.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.766+0000 7f24ce47e740 1 -- 172.21.15.83:0/820493352 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x55a20be85300 con 0x55a20be04270 2024-04-24T09:12:09.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.767+0000 7f24aa000640 1 -- 172.21.15.83:0/820493352 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f24b80053e0 con 0x55a20be04270 2024-04-24T09:12:09.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.768+0000 7f24aa000640 1 -- 172.21.15.83:0/820493352 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 61) ==== 8362+0+0 (secure 0 0 0) 0x7f24b8038440 con 0x55a20be04270 2024-04-24T09:12:09.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.768+0000 7f24ce47e740 10 client.22674 fetch_fsmap finished waiting for FSMap version 61 2024-04-24T09:12:09.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.768+0000 7f24ce47e740 10 client.22674 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-24T09:12:09.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.768+0000 7f24ce47e740 20 client.22674 populate_metadata read hostname 'smithi083' 2024-04-24T09:12:09.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.768+0000 7f24ce47e740 1 --2- 172.21.15.83:0/820493352 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55a20be857e0 0x55a20be87bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:09.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.768+0000 7f24ce47e740 4 client.22674 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:12:09.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.768+0000 7f24ce47e740 1 -- 172.21.15.83:0/820493352 --> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a20bd70a60 con 0x55a20be857e0 2024-04-24T09:12:09.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.769+0000 7f24c3e00640 1 --2- 172.21.15.83:0/820493352 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55a20be857e0 0x55a20be87bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:09.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.770+0000 7f24c3e00640 1 --2- 172.21.15.83:0/820493352 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55a20be857e0 0x55a20be87bd0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4ca22f38a50ce509 server_cookie=c1bab3e30a087fd3 in_seq=0 out_seq=0 2024-04-24T09:12:09.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.770+0000 7f24aa000640 10 client.22674 ms_handle_connect on v2:172.21.15.138:6834/1909219758 2024-04-24T09:12:09.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.770+0000 7f24aa000640 1 -- 172.21.15.83:0/820493352 <== mds.2 v2:172.21.15.138:6834/1909219758 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a20bd70a60 con 0x55a20be857e0 2024-04-24T09:12:09.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.770+0000 7f24aa000640 10 client.22674 handle_command_reply: tid=0 2024-04-24T09:12:09.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.770+0000 7f24ce47e740 1 client.22674 shutdown 2024-04-24T09:12:09.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.770+0000 7f24ce47e740 1 -- 172.21.15.83:0/820493352 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a20be04270 msgr2=0x55a20be0f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:09.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.770+0000 7f24ce47e740 1 --2- 172.21.15.83:0/820493352 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a20be04270 0x55a20be0f400 secure :-1 s=READY pgs=2542 cs=0 l=1 rev1=1 crypto rx=0x55a20be0b980 tx=0x7f24b80056f0 comp rx=0 tx=0).stop 2024-04-24T09:12:09.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.770+0000 7f24ce47e740 1 -- 172.21.15.83:0/820493352 shutdown_connections 2024-04-24T09:12:09.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.770+0000 7f24ce47e740 1 --2- 172.21.15.83:0/820493352 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55a20be857e0 0x55a20be87bd0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:09.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.770+0000 7f24ce47e740 1 --2- 172.21.15.83:0/820493352 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a20be0e940 0x55a20be0f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:09.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.770+0000 7f24ce47e740 1 --2- 172.21.15.83:0/820493352 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a20be04270 0x55a20be0f400 unknown :-1 s=CLOSED pgs=2542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:09.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.770+0000 7f24ce47e740 1 --2- 172.21.15.83:0/820493352 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a20be03950 0x55a20be17030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:09.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.770+0000 7f24ce47e740 1 -- 172.21.15.83:0/820493352 >> 172.21.15.83:0/820493352 conn(0x55a20bdffc70 msgr2=0x55a20be050a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:09.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.770+0000 7f24aa000640 0 client.22674 ms_handle_reset on v2:172.21.15.138:6834/1909219758 2024-04-24T09:12:09.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.771+0000 7f24ce47e740 1 -- 172.21.15.83:0/820493352 shutdown_connections 2024-04-24T09:12:09.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.771+0000 7f24ce47e740 1 -- 172.21.15.83:0/820493352 wait complete. 2024-04-24T09:12:09.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.771+0000 7f24ce47e740 20 client.22674 trim_cache size 0 max 16384 2024-04-24T09:12:09.771 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:12:09.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.772+0000 7f24cb600640 1 -- 172.21.15.83:0/728053006 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f24ac0a3f50 msgr2=0x7f24ac0a6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:09.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.772+0000 7f24cb600640 1 --2- 172.21.15.83:0/728053006 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f24ac0a3f50 0x7f24ac0a6410 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f24bc0bd900 tx=0x7f24bc0095c0 comp rx=0 tx=0).stop 2024-04-24T09:12:09.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.772+0000 7f24cb600640 1 -- 172.21.15.83:0/728053006 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f24c41a8a10 msgr2=0x7f24c41aae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:09.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.772+0000 7f24cb600640 1 --2- 172.21.15.83:0/728053006 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f24c41a8a10 0x7f24c41aae00 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f24bc031040 tx=0x7f24bc032f80 comp rx=0 tx=0).stop 2024-04-24T09:12:09.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.772+0000 7f24cb600640 1 -- 172.21.15.83:0/728053006 shutdown_connections 2024-04-24T09:12:09.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.772+0000 7f24cb600640 1 --2- 172.21.15.83:0/728053006 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f24ac0a3f50 0x7f24ac0a6410 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:09.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.772+0000 7f24cb600640 1 --2- 172.21.15.83:0/728053006 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f24c41a8a10 0x7f24c41aae00 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:09.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.772+0000 7f24cb600640 1 --2- 172.21.15.83:0/728053006 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f24c4109580 0x7f24c4110170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:09.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.772+0000 7f24cb600640 1 --2- 172.21.15.83:0/728053006 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f24c41044b0 0x7f24c410fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:09.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.772+0000 7f24cb600640 1 -- 172.21.15.83:0/728053006 >> 172.21.15.83:0/728053006 conn(0x7f24c40ffd90 msgr2=0x7f24c4101040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:09.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.772+0000 7f24cb600640 1 -- 172.21.15.83:0/728053006 shutdown_connections 2024-04-24T09:12:09.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:09.772+0000 7f24cb600640 1 -- 172.21.15.83:0/728053006 wait complete. 2024-04-24T09:12:09.791 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:12:09 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/1455298047' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:12:09.793 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T09:12:10.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.075+0000 7eff91200640 1 -- 172.21.15.83:0/2008531200 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff8c1071e0 msgr2=0x7eff8c10edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:10.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.075+0000 7eff91200640 1 --2- 172.21.15.83:0/2008531200 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff8c1071e0 0x7eff8c10edb0 secure :-1 s=READY pgs=2543 cs=0 l=1 rev1=1 crypto rx=0x7eff84009ca0 tx=0x7eff8402a8c0 comp rx=0 tx=0).stop 2024-04-24T09:12:10.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.075+0000 7eff91200640 1 -- 172.21.15.83:0/2008531200 shutdown_connections 2024-04-24T09:12:10.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.075+0000 7eff91200640 1 --2- 172.21.15.83:0/2008531200 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff8c1071e0 0x7eff8c10edb0 unknown :-1 s=CLOSED pgs=2543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:10.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.075+0000 7eff91200640 1 --2- 172.21.15.83:0/2008531200 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7eff8c102ac0 0x7eff8c106b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:10.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.075+0000 7eff91200640 1 --2- 172.21.15.83:0/2008531200 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff8c102110 0x7eff8c1024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:10.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.075+0000 7eff91200640 1 -- 172.21.15.83:0/2008531200 >> 172.21.15.83:0/2008531200 conn(0x7eff8c0fd9d0 msgr2=0x7eff8c0ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:10.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.075+0000 7eff91200640 1 -- 172.21.15.83:0/2008531200 shutdown_connections 2024-04-24T09:12:10.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.076+0000 7eff91200640 1 -- 172.21.15.83:0/2008531200 wait complete. 2024-04-24T09:12:10.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.077+0000 7eff91200640 1 Processor -- start 2024-04-24T09:12:10.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.078+0000 7eff91200640 1 -- start start 2024-04-24T09:12:10.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.078+0000 7eff91200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff8c102110 0x7eff8c19d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:10.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.078+0000 7eff91200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff8c1071e0 0x7eff8c1962e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:10.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.078+0000 7eff91200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7eff8c19d860 0x7eff8c1968d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:10.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.078+0000 7eff8b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff8c102110 0x7eff8c19d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:10.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.079+0000 7eff8b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff8c102110 0x7eff8c19d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50026/0 (socket says 172.21.15.83:50026) 2024-04-24T09:12:10.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.079+0000 7eff91200640 1 -- 172.21.15.83:0/2631825973 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7eff8c10ff80 con 0x7eff8c102110 2024-04-24T09:12:10.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.079+0000 7eff91200640 1 -- 172.21.15.83:0/2631825973 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7eff8c10fe00 con 0x7eff8c1071e0 2024-04-24T09:12:10.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.079+0000 7eff8aa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff8c1071e0 0x7eff8c1962e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:10.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.079+0000 7eff91200640 1 -- 172.21.15.83:0/2631825973 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7eff8c110100 con 0x7eff8c19d860 2024-04-24T09:12:10.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.079+0000 7eff8b400640 1 -- 172.21.15.83:0/2631825973 learned_addr learned my addr 172.21.15.83:0/2631825973 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:12:10.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.079+0000 7eff8be00640 1 --2- 172.21.15.83:0/2631825973 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7eff8c19d860 0x7eff8c1968d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:10.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.080+0000 7eff8aa00640 1 -- 172.21.15.83:0/2631825973 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7eff8c19d860 msgr2=0x7eff8c1968d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:10.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.080+0000 7eff8aa00640 1 --2- 172.21.15.83:0/2631825973 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7eff8c19d860 0x7eff8c1968d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:10.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.080+0000 7eff8aa00640 1 -- 172.21.15.83:0/2631825973 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff8c102110 msgr2=0x7eff8c19d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:10.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.080+0000 7eff8aa00640 1 --2- 172.21.15.83:0/2631825973 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff8c102110 0x7eff8c19d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:10.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.080+0000 7eff8aa00640 1 -- 172.21.15.83:0/2631825973 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff8c19ad30 con 0x7eff8c1071e0 2024-04-24T09:12:10.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.080+0000 7eff8b400640 1 --2- 172.21.15.83:0/2631825973 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff8c102110 0x7eff8c19d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:12:10.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.080+0000 7eff8aa00640 1 --2- 172.21.15.83:0/2631825973 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff8c1071e0 0x7eff8c1962e0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7eff7800e9c0 tx=0x7eff7800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:10.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.080+0000 7eff83e00640 1 -- 172.21.15.83:0/2631825973 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff7800ccc0 con 0x7eff8c1071e0 2024-04-24T09:12:10.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.081+0000 7eff91200640 1 -- 172.21.15.83:0/2631825973 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff8c19b020 con 0x7eff8c1071e0 2024-04-24T09:12:10.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.081+0000 7eff83e00640 1 -- 172.21.15.83:0/2631825973 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7eff78005020 con 0x7eff8c1071e0 2024-04-24T09:12:10.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.081+0000 7eff83e00640 1 -- 172.21.15.83:0/2631825973 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff78004a00 con 0x7eff8c1071e0 2024-04-24T09:12:10.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.081+0000 7eff91200640 1 -- 172.21.15.83:0/2631825973 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff8c19b570 con 0x7eff8c1071e0 2024-04-24T09:12:10.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.085+0000 7eff957ba740 1 Processor -- start 2024-04-24T09:12:10.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.086+0000 7eff957ba740 1 -- start start 2024-04-24T09:12:10.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.086+0000 7eff957ba740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a556ddab40 0x55a556ddaf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:10.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.086+0000 7eff957ba740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a556ddb460 0x55a556de55f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:10.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.086+0000 7eff957ba740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a556de5b30 0x55a556de7f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:10.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.086+0000 7eff957ba740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a556d401d0 con 0x55a556ddb460 2024-04-24T09:12:10.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.086+0000 7eff957ba740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a556ce6450 con 0x55a556ddab40 2024-04-24T09:12:10.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.086+0000 7eff957ba740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a556d3c190 con 0x55a556de5b30 2024-04-24T09:12:10.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.087+0000 7eff8b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a556ddb460 0x55a556de55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:10.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.087+0000 7eff8be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a556ddab40 0x55a556ddaf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:10.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.087+0000 7eff8be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a556ddab40 0x55a556ddaf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:59340/0 (socket says 172.21.15.83:59340) 2024-04-24T09:12:10.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.087+0000 7eff8be00640 1 -- 172.21.15.83:0/1835431094 learned_addr learned my addr 172.21.15.83:0/1835431094 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:12:10.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.087+0000 7eff83e00640 1 -- 172.21.15.83:0/2631825973 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7eff780051d0 con 0x7eff8c1071e0 2024-04-24T09:12:10.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.087+0000 7eff8be00640 1 -- 172.21.15.83:0/1835431094 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a556de5b30 msgr2=0x55a556de7f20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-24T09:12:10.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.087+0000 7eff8be00640 1 --2- 172.21.15.83:0/1835431094 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a556de5b30 0x55a556de7f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:10.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.087+0000 7eff8be00640 1 -- 172.21.15.83:0/1835431094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a556ddb460 msgr2=0x55a556de55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:10.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.087+0000 7eff8be00640 1 --2- 172.21.15.83:0/1835431094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a556ddb460 0x55a556de55f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:10.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.087+0000 7eff8be00640 1 -- 172.21.15.83:0/1835431094 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a556da3310 con 0x55a556ddab40 2024-04-24T09:12:10.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.087+0000 7eff8be00640 1 --2- 172.21.15.83:0/1835431094 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a556ddab40 0x55a556ddaf20 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7eff8402aeb0 tx=0x7eff84005f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7308bbe3b8adba26 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:10.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.088+0000 7eff80c00640 1 -- 172.21.15.83:0/1835431094 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff84031d40 con 0x55a556ddab40 2024-04-24T09:12:10.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.088+0000 7eff80c00640 1 -- 172.21.15.83:0/1835431094 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7eff84036030 con 0x55a556ddab40 2024-04-24T09:12:10.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.088+0000 7eff80c00640 1 -- 172.21.15.83:0/1835431094 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff84004310 con 0x55a556ddab40 2024-04-24T09:12:10.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.089+0000 7eff83e00640 1 --2- 172.21.15.83:0/2631825973 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7eff700a3e00 0x7eff700a62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:10.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.089+0000 7eff83e00640 1 -- 172.21.15.83:0/2631825973 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 9583+0+0 (secure 0 0 0) 0x7eff780d0b20 con 0x7eff8c1071e0 2024-04-24T09:12:10.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.089+0000 7eff957ba740 1 -- 172.21.15.83:0/1835431094 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a556ddab40 msgr2=0x55a556ddaf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:10.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.089+0000 7eff957ba740 1 --2- 172.21.15.83:0/1835431094 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a556ddab40 0x55a556ddaf20 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7eff8402aeb0 tx=0x7eff84005f30 comp rx=0 tx=0).stop 2024-04-24T09:12:10.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.090+0000 7eff957ba740 1 -- 172.21.15.83:0/1835431094 shutdown_connections 2024-04-24T09:12:10.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.090+0000 7eff957ba740 1 --2- 172.21.15.83:0/1835431094 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a556de5b30 0x55a556de7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:10.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.090+0000 7eff957ba740 1 --2- 172.21.15.83:0/1835431094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a556ddb460 0x55a556de55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:10.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.090+0000 7eff957ba740 1 --2- 172.21.15.83:0/1835431094 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a556ddab40 0x55a556ddaf20 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:10.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.090+0000 7eff957ba740 1 -- 172.21.15.83:0/1835431094 >> 172.21.15.83:0/1835431094 conn(0x55a556dd6e60 msgr2=0x55a556dd7240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:10.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.090+0000 7eff8b400640 1 --2- 172.21.15.83:0/2631825973 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7eff700a3e00 0x7eff700a62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:10.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.091+0000 7eff8b400640 1 --2- 172.21.15.83:0/2631825973 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7eff700a3e00 0x7eff700a62c0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x55a556ddc100 tx=0x7eff7c005f90 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:10.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.091+0000 7eff957ba740 1 -- 172.21.15.83:0/1835431094 shutdown_connections 2024-04-24T09:12:10.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.091+0000 7eff957ba740 1 -- 172.21.15.83:0/1835431094 wait complete. 2024-04-24T09:12:10.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.092+0000 7eff957ba740 1 -- start start 2024-04-24T09:12:10.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.092+0000 7eff957ba740 1 Processor -- start 2024-04-24T09:12:10.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.093+0000 7eff957ba740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a556ddab40 0x55a556de2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:10.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.093+0000 7eff957ba740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a556ddb460 0x55a556de2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:10.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.093+0000 7eff957ba740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a556de5b30 0x55a556de33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:10.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.093+0000 7eff957ba740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a556de87e0 con 0x55a556ddb460 2024-04-24T09:12:10.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.093+0000 7eff957ba740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a556cf9810 con 0x55a556ddab40 2024-04-24T09:12:10.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.093+0000 7eff957ba740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a556de8960 con 0x55a556de5b30 2024-04-24T09:12:10.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.093+0000 7eff957ba740 10 client.0 _ll_register_callbacks cb 0x55a556cc7140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:12:10.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.093+0000 7eff8b400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a556ddab40 0x55a556de2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:10.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.093+0000 7eff8b400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a556ddab40 0x55a556de2950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:59346/0 (socket says 172.21.15.83:59346) 2024-04-24T09:12:10.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.094+0000 7eff8b400640 1 -- 172.21.15.83:0/1181837286 learned_addr learned my addr 172.21.15.83:0/1181837286 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:12:10.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.094+0000 7eff8aa00640 1 --2- 172.21.15.83:0/1181837286 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a556ddb460 0x55a556de2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:10.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.094+0000 7eff8be00640 1 --2- 172.21.15.83:0/1181837286 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a556de5b30 0x55a556de33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:10.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.094+0000 7eff8aa00640 1 -- 172.21.15.83:0/1181837286 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a556de5b30 msgr2=0x55a556de33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:10.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.094+0000 7eff8aa00640 1 --2- 172.21.15.83:0/1181837286 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a556de5b30 0x55a556de33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:10.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.094+0000 7eff8aa00640 1 -- 172.21.15.83:0/1181837286 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a556ddab40 msgr2=0x55a556de2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:10.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.094+0000 7eff8aa00640 1 --2- 172.21.15.83:0/1181837286 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a556ddab40 0x55a556de2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:10.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.094+0000 7eff8aa00640 1 -- 172.21.15.83:0/1181837286 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a556d0a3f0 con 0x55a556ddb460 2024-04-24T09:12:10.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.095+0000 7eff8aa00640 1 --2- 172.21.15.83:0/1181837286 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a556ddb460 0x55a556de2e90 secure :-1 s=READY pgs=2544 cs=0 l=1 rev1=1 crypto rx=0x7eff780cddf0 tx=0x7eff7809bfa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41fd9db25b02f5bd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:10.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.095+0000 7eff75600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:12:10.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.095+0000 7eff75600640 1 -- 172.21.15.83:0/1181837286 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff7801c040 con 0x55a556ddb460 2024-04-24T09:12:10.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.095+0000 7eff75600640 1 -- 172.21.15.83:0/1181837286 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9583+0+0 (secure 0 0 0) 0x7eff780a4e20 con 0x55a556ddb460 2024-04-24T09:12:10.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.095+0000 7eff957ba740 1 -- 172.21.15.83:0/1181837286 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff84036030 con 0x55a556ddb460 2024-04-24T09:12:10.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.096+0000 7eff75600640 1 -- 172.21.15.83:0/1181837286 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7eff780ce280 con 0x55a556ddb460 2024-04-24T09:12:10.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.096+0000 7eff957ba740 10 client.22686 fetch_fsmap learned FSMap version 61 2024-04-24T09:12:10.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.096+0000 7eff957ba740 1 -- 172.21.15.83:0/1181837286 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x55a556a523f0 con 0x55a556ddb460 2024-04-24T09:12:10.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.096+0000 7eff75600640 1 -- 172.21.15.83:0/1181837286 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7eff84036030 con 0x55a556ddb460 2024-04-24T09:12:10.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.097+0000 7eff75600640 1 -- 172.21.15.83:0/1181837286 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 61) ==== 8362+0+0 (secure 0 0 0) 0x7eff78004ba0 con 0x55a556ddb460 2024-04-24T09:12:10.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.097+0000 7eff957ba740 10 client.22686 fetch_fsmap finished waiting for FSMap version 61 2024-04-24T09:12:10.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.097+0000 7eff957ba740 10 client.22686 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-24T09:12:10.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.097+0000 7eff957ba740 20 client.22686 populate_metadata read hostname 'smithi083' 2024-04-24T09:12:10.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.097+0000 7eff957ba740 1 --2- 172.21.15.83:0/1181837286 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x55a556df5ba0 0x55a556df7f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:10.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.097+0000 7eff957ba740 4 client.22686 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:12:10.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.097+0000 7eff957ba740 1 -- 172.21.15.83:0/1181837286 --> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a556d4dbf0 con 0x55a556df5ba0 2024-04-24T09:12:10.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.097+0000 7eff8b400640 1 --2- 172.21.15.83:0/1181837286 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x55a556df5ba0 0x55a556df7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:10.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.098+0000 7eff8b400640 1 --2- 172.21.15.83:0/1181837286 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x55a556df5ba0 0x55a556df7f90 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f3a0991c1ac96221 server_cookie=2429013b6f7c7617 in_seq=0 out_seq=0 2024-04-24T09:12:10.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.098+0000 7eff75600640 10 client.22686 ms_handle_connect on v2:172.21.15.138:6836/900566754 2024-04-24T09:12:10.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.099+0000 7eff75600640 1 -- 172.21.15.83:0/1181837286 <== mds.1 v2:172.21.15.138:6836/900566754 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a556d4dbf0 con 0x55a556df5ba0 2024-04-24T09:12:10.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.099+0000 7eff75600640 10 client.22686 handle_command_reply: tid=0 2024-04-24T09:12:10.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.099+0000 7eff957ba740 1 client.22686 shutdown 2024-04-24T09:12:10.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.100+0000 7eff957ba740 1 -- 172.21.15.83:0/1181837286 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a556ddb460 msgr2=0x55a556de2e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:10.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.100+0000 7eff957ba740 1 --2- 172.21.15.83:0/1181837286 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a556ddb460 0x55a556de2e90 secure :-1 s=READY pgs=2544 cs=0 l=1 rev1=1 crypto rx=0x7eff780cddf0 tx=0x7eff7809bfa0 comp rx=0 tx=0).stop 2024-04-24T09:12:10.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.100+0000 7eff957ba740 1 -- 172.21.15.83:0/1181837286 shutdown_connections 2024-04-24T09:12:10.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.100+0000 7eff957ba740 1 --2- 172.21.15.83:0/1181837286 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x55a556df5ba0 0x55a556df7f90 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:10.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.100+0000 7eff957ba740 1 --2- 172.21.15.83:0/1181837286 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a556de5b30 0x55a556de33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:10.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.100+0000 7eff957ba740 1 --2- 172.21.15.83:0/1181837286 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a556ddb460 0x55a556de2e90 unknown :-1 s=CLOSED pgs=2544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:10.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.100+0000 7eff957ba740 1 --2- 172.21.15.83:0/1181837286 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a556ddab40 0x55a556de2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:10.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.100+0000 7eff75600640 0 client.22686 ms_handle_reset on v2:172.21.15.138:6836/900566754 2024-04-24T09:12:10.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.100+0000 7eff957ba740 1 -- 172.21.15.83:0/1181837286 >> 172.21.15.83:0/1181837286 conn(0x55a556dd6e60 msgr2=0x55a556dd5f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:10.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.100+0000 7eff957ba740 1 -- 172.21.15.83:0/1181837286 shutdown_connections 2024-04-24T09:12:10.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.100+0000 7eff957ba740 1 -- 172.21.15.83:0/1181837286 wait complete. 2024-04-24T09:12:10.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.100+0000 7eff957ba740 20 client.22686 trim_cache size 0 max 16384 2024-04-24T09:12:10.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.120+0000 7eff957ba740 1 Processor -- start 2024-04-24T09:12:10.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.120+0000 7eff957ba740 1 -- start start 2024-04-24T09:12:10.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.120+0000 7eff957ba740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a556ddab40 0x55a556ddd2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:10.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.120+0000 7eff957ba740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a556ddb460 0x55a556df16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:10.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.120+0000 7eff957ba740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a556de5b30 0x55a556dfd810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:10.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.120+0000 7eff957ba740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7eff8c1be0d0 con 0x55a556de5b30 2024-04-24T09:12:10.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.120+0000 7eff957ba740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7eff8c1bfc10 con 0x55a556ddb460 2024-04-24T09:12:10.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.120+0000 7eff957ba740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7eff8c1be7a0 con 0x55a556ddab40 2024-04-24T09:12:10.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.121+0000 7eff8aa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a556ddb460 0x55a556df16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:10.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.121+0000 7eff8aa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a556ddb460 0x55a556df16a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:59360/0 (socket says 172.21.15.83:59360) 2024-04-24T09:12:10.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.121+0000 7eff8aa00640 1 -- 172.21.15.83:0/577843930 learned_addr learned my addr 172.21.15.83:0/577843930 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:12:10.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.121+0000 7eff8be00640 1 --2- 172.21.15.83:0/577843930 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a556de5b30 0x55a556dfd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:10.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.121+0000 7eff8b400640 1 --2- 172.21.15.83:0/577843930 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a556ddab40 0x55a556ddd2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:10.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.121+0000 7eff8be00640 1 -- 172.21.15.83:0/577843930 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a556ddab40 msgr2=0x55a556ddd2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:10.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.121+0000 7eff8be00640 1 --2- 172.21.15.83:0/577843930 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a556ddab40 0x55a556ddd2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:10.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.121+0000 7eff8be00640 1 -- 172.21.15.83:0/577843930 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a556ddb460 msgr2=0x55a556df16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:10.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.121+0000 7eff8be00640 1 --2- 172.21.15.83:0/577843930 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a556ddb460 0x55a556df16a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:10.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.121+0000 7eff8be00640 1 -- 172.21.15.83:0/577843930 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a556b1c7d0 con 0x55a556de5b30 2024-04-24T09:12:10.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.121+0000 7eff8aa00640 1 --2- 172.21.15.83:0/577843930 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a556ddb460 0x55a556df16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:12:10.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.121+0000 7eff8b400640 1 --2- 172.21.15.83:0/577843930 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a556ddab40 0x55a556ddd2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:12:10.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.122+0000 7eff8be00640 1 --2- 172.21.15.83:0/577843930 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a556de5b30 0x55a556dfd810 secure :-1 s=READY pgs=2545 cs=0 l=1 rev1=1 crypto rx=0x7eff84009a60 tx=0x7eff84000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=456c2733e953fd31 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:10.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.123+0000 7eff77e00640 1 -- 172.21.15.83:0/577843930 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff84031b40 con 0x55a556de5b30 2024-04-24T09:12:10.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.123+0000 7eff77e00640 1 -- 172.21.15.83:0/577843930 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7eff84036040 con 0x55a556de5b30 2024-04-24T09:12:10.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.123+0000 7eff77e00640 1 -- 172.21.15.83:0/577843930 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff84004040 con 0x55a556de5b30 2024-04-24T09:12:10.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.124+0000 7eff957ba740 1 -- 172.21.15.83:0/577843930 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a556de5b30 msgr2=0x55a556dfd810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:10.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.124+0000 7eff957ba740 1 --2- 172.21.15.83:0/577843930 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a556de5b30 0x55a556dfd810 secure :-1 s=READY pgs=2545 cs=0 l=1 rev1=1 crypto rx=0x7eff84009a60 tx=0x7eff84000f40 comp rx=0 tx=0).stop 2024-04-24T09:12:10.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.124+0000 7eff957ba740 1 -- 172.21.15.83:0/577843930 shutdown_connections 2024-04-24T09:12:10.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.124+0000 7eff957ba740 1 --2- 172.21.15.83:0/577843930 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a556de5b30 0x55a556dfd810 unknown :-1 s=CLOSED pgs=2545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:10.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.124+0000 7eff957ba740 1 --2- 172.21.15.83:0/577843930 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a556ddb460 0x55a556df16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:10.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.124+0000 7eff957ba740 1 --2- 172.21.15.83:0/577843930 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a556ddab40 0x55a556ddd2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:10.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.124+0000 7eff957ba740 1 -- 172.21.15.83:0/577843930 >> 172.21.15.83:0/577843930 conn(0x55a556dd6e60 msgr2=0x55a556de31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:10.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.124+0000 7eff957ba740 1 -- 172.21.15.83:0/577843930 shutdown_connections 2024-04-24T09:12:10.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.125+0000 7eff957ba740 1 -- 172.21.15.83:0/577843930 wait complete. 2024-04-24T09:12:10.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.125+0000 7eff957ba740 1 -- start start 2024-04-24T09:12:10.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.125+0000 7eff957ba740 1 Processor -- start 2024-04-24T09:12:10.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.126+0000 7eff957ba740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a556ddab40 0x55a556dedfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:10.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.126+0000 7eff957ba740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a556ddb460 0x55a556de66d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:10.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.126+0000 7eff957ba740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a556de5b30 0x55a556de6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:10.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.126+0000 7eff957ba740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7eff8c1bf540 con 0x55a556de5b30 2024-04-24T09:12:10.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.126+0000 7eff957ba740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7eff8c1bee70 con 0x55a556ddb460 2024-04-24T09:12:10.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.126+0000 7eff957ba740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7eff8c1bc0d0 con 0x55a556ddab40 2024-04-24T09:12:10.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.126+0000 7eff957ba740 10 client.0 _ll_register_callbacks cb 0x7eff7809c410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:12:10.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.126+0000 7eff8be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a556de5b30 0x55a556de6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:10.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.126+0000 7eff8aa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a556ddb460 0x55a556de66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:10.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.126+0000 7eff8aa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a556ddb460 0x55a556de66d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:59372/0 (socket says 172.21.15.83:59372) 2024-04-24T09:12:10.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.126+0000 7eff8aa00640 1 -- 172.21.15.83:0/2470656114 learned_addr learned my addr 172.21.15.83:0/2470656114 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:12:10.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.127+0000 7eff8b400640 1 --2- 172.21.15.83:0/2470656114 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a556ddab40 0x55a556dedfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:10.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.127+0000 7eff8be00640 1 -- 172.21.15.83:0/2470656114 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a556ddab40 msgr2=0x55a556dedfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:10.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.127+0000 7eff8be00640 1 --2- 172.21.15.83:0/2470656114 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a556ddab40 0x55a556dedfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:10.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.127+0000 7eff8be00640 1 -- 172.21.15.83:0/2470656114 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a556ddb460 msgr2=0x55a556de66d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:10.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.127+0000 7eff8be00640 1 --2- 172.21.15.83:0/2470656114 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a556ddb460 0x55a556de66d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:10.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.127+0000 7eff8be00640 1 -- 172.21.15.83:0/2470656114 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a556de74c0 con 0x55a556de5b30 2024-04-24T09:12:10.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.127+0000 7eff8aa00640 1 --2- 172.21.15.83:0/2470656114 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a556ddb460 0x55a556de66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:12:10.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.127+0000 7eff8b400640 1 --2- 172.21.15.83:0/2470656114 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a556ddab40 0x55a556dedfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:12:10.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.128+0000 7eff8be00640 1 --2- 172.21.15.83:0/2470656114 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a556de5b30 0x55a556de6c90 secure :-1 s=READY pgs=2546 cs=0 l=1 rev1=1 crypto rx=0x7eff84009bf0 tx=0x7eff84031d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea56911309466b00 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:10.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.128+0000 7eff75600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:12:10.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.128+0000 7eff75600640 1 -- 172.21.15.83:0/2470656114 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff84059070 con 0x55a556de5b30 2024-04-24T09:12:10.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.128+0000 7eff957ba740 1 -- 172.21.15.83:0/2470656114 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff84036040 con 0x55a556de5b30 2024-04-24T09:12:10.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.128+0000 7eff75600640 1 -- 172.21.15.83:0/2470656114 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9583+0+0 (secure 0 0 0) 0x7eff84003970 con 0x55a556de5b30 2024-04-24T09:12:10.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.129+0000 7eff75600640 1 -- 172.21.15.83:0/2470656114 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7eff84065050 con 0x55a556de5b30 2024-04-24T09:12:10.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.129+0000 7eff957ba740 10 client.22692 fetch_fsmap learned FSMap version 61 2024-04-24T09:12:10.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.129+0000 7eff957ba740 1 -- 172.21.15.83:0/2470656114 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x55a556e5c170 con 0x55a556de5b30 2024-04-24T09:12:10.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.129+0000 7eff75600640 1 -- 172.21.15.83:0/2470656114 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7eff84036040 con 0x55a556de5b30 2024-04-24T09:12:10.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.129+0000 7eff75600640 1 -- 172.21.15.83:0/2470656114 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 61) ==== 8362+0+0 (secure 0 0 0) 0x7eff84043070 con 0x55a556de5b30 2024-04-24T09:12:10.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.129+0000 7eff957ba740 10 client.22692 fetch_fsmap finished waiting for FSMap version 61 2024-04-24T09:12:10.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.129+0000 7eff957ba740 10 client.22692 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-24T09:12:10.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.129+0000 7eff957ba740 20 client.22692 populate_metadata read hostname 'smithi083' 2024-04-24T09:12:10.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.129+0000 7eff957ba740 1 --2- 172.21.15.83:0/2470656114 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x55a556e5c580 0x55a556e5e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:10.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.129+0000 7eff957ba740 4 client.22692 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:12:10.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.129+0000 7eff957ba740 1 -- 172.21.15.83:0/2470656114 --> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a556d47a60 con 0x55a556e5c580 2024-04-24T09:12:10.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.130+0000 7eff8b400640 1 --2- 172.21.15.83:0/2470656114 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x55a556e5c580 0x55a556e5e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:10.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.131+0000 7eff8b400640 1 --2- 172.21.15.83:0/2470656114 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x55a556e5c580 0x55a556e5e9a0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3c3a279dab160ec0 server_cookie=e128d6a2fba72d36 in_seq=0 out_seq=0 2024-04-24T09:12:10.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.131+0000 7eff75600640 10 client.22692 ms_handle_connect on v2:172.21.15.138:6836/900566754 2024-04-24T09:12:10.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.131+0000 7eff75600640 1 -- 172.21.15.83:0/2470656114 <== mds.1 v2:172.21.15.138:6836/900566754 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a556d47a60 con 0x55a556e5c580 2024-04-24T09:12:10.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.131+0000 7eff75600640 10 client.22692 handle_command_reply: tid=0 2024-04-24T09:12:10.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.131+0000 7eff957ba740 1 client.22692 shutdown 2024-04-24T09:12:10.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.131+0000 7eff957ba740 1 -- 172.21.15.83:0/2470656114 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a556de5b30 msgr2=0x55a556de6c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:10.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.131+0000 7eff957ba740 1 --2- 172.21.15.83:0/2470656114 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a556de5b30 0x55a556de6c90 secure :-1 s=READY pgs=2546 cs=0 l=1 rev1=1 crypto rx=0x7eff84009bf0 tx=0x7eff84031d50 comp rx=0 tx=0).stop 2024-04-24T09:12:10.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.131+0000 7eff957ba740 1 -- 172.21.15.83:0/2470656114 shutdown_connections 2024-04-24T09:12:10.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.131+0000 7eff957ba740 1 --2- 172.21.15.83:0/2470656114 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a556de5b30 0x55a556de6c90 unknown :-1 s=CLOSED pgs=2546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:10.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.131+0000 7eff957ba740 1 --2- 172.21.15.83:0/2470656114 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a556ddb460 0x55a556de66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:10.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.131+0000 7eff957ba740 1 --2- 172.21.15.83:0/2470656114 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x55a556e5c580 0x55a556e5e9a0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:10.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.131+0000 7eff957ba740 1 --2- 172.21.15.83:0/2470656114 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a556ddab40 0x55a556dedfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:10.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.131+0000 7eff957ba740 1 -- 172.21.15.83:0/2470656114 >> 172.21.15.83:0/2470656114 conn(0x55a556dd6e60 msgr2=0x55a556de2880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:10.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.131+0000 7eff75600640 0 client.22692 ms_handle_reset on v2:172.21.15.138:6836/900566754 2024-04-24T09:12:10.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.132+0000 7eff957ba740 1 -- 172.21.15.83:0/2470656114 shutdown_connections 2024-04-24T09:12:10.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.132+0000 7eff957ba740 1 -- 172.21.15.83:0/2470656114 wait complete. 2024-04-24T09:12:10.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.132+0000 7eff957ba740 20 client.22692 trim_cache size 0 max 16384 2024-04-24T09:12:10.133 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:12:10.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.133+0000 7eff91200640 1 -- 172.21.15.83:0/2631825973 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7eff700a3e00 msgr2=0x7eff700a62c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:10.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.133+0000 7eff91200640 1 --2- 172.21.15.83:0/2631825973 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7eff700a3e00 0x7eff700a62c0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x55a556ddc100 tx=0x7eff7c005f90 comp rx=0 tx=0).stop 2024-04-24T09:12:10.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.133+0000 7eff91200640 1 -- 172.21.15.83:0/2631825973 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff8c1071e0 msgr2=0x7eff8c1962e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:10.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.133+0000 7eff91200640 1 --2- 172.21.15.83:0/2631825973 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff8c1071e0 0x7eff8c1962e0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7eff7800e9c0 tx=0x7eff7800ee80 comp rx=0 tx=0).stop 2024-04-24T09:12:10.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.133+0000 7eff91200640 1 -- 172.21.15.83:0/2631825973 shutdown_connections 2024-04-24T09:12:10.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.133+0000 7eff91200640 1 --2- 172.21.15.83:0/2631825973 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7eff700a3e00 0x7eff700a62c0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:10.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.133+0000 7eff91200640 1 --2- 172.21.15.83:0/2631825973 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7eff8c19d860 0x7eff8c1968d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:10.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.133+0000 7eff91200640 1 --2- 172.21.15.83:0/2631825973 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff8c1071e0 0x7eff8c1962e0 secure :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7eff7800e9c0 tx=0x7eff7800ee80 comp rx=0 tx=0).stop 2024-04-24T09:12:10.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.133+0000 7eff91200640 1 --2- 172.21.15.83:0/2631825973 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff8c102110 0x7eff8c19d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:10.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.133+0000 7eff91200640 1 -- 172.21.15.83:0/2631825973 >> 172.21.15.83:0/2631825973 conn(0x7eff8c0fd9d0 msgr2=0x7eff8c0ffd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:10.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.134+0000 7eff91200640 1 -- 172.21.15.83:0/2631825973 shutdown_connections 2024-04-24T09:12:10.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.134+0000 7eff91200640 1 -- 172.21.15.83:0/2631825973 wait complete. 2024-04-24T09:12:10.155 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T09:12:10.184 INFO:tasks.workunit.client.0.smithi083.stdout: 10 50111 6.62 MB/sec execute 281 sec latency 112.586 ms 2024-04-24T09:12:10.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.434+0000 7f17f5c00640 1 -- 172.21.15.83:0/3721295922 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f17f0102220 msgr2=0x7f17f0102600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:10.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.434+0000 7f17f5c00640 1 --2- 172.21.15.83:0/3721295922 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f17f0102220 0x7f17f0102600 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f17e0009900 tx=0x7f17e002a3f0 comp rx=0 tx=0).stop 2024-04-24T09:12:10.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.434+0000 7f17f5c00640 1 -- 172.21.15.83:0/3721295922 shutdown_connections 2024-04-24T09:12:10.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.434+0000 7f17f5c00640 1 --2- 172.21.15.83:0/3721295922 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17f01072f0 0x7f17f010eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:10.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.434+0000 7f17f5c00640 1 --2- 172.21.15.83:0/3721295922 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f17f0102bd0 0x7f17f0106c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:10.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.434+0000 7f17f5c00640 1 --2- 172.21.15.83:0/3721295922 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f17f0102220 0x7f17f0102600 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:10.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.434+0000 7f17f5c00640 1 -- 172.21.15.83:0/3721295922 >> 172.21.15.83:0/3721295922 conn(0x7f17f00fda80 msgr2=0x7f17f00ffea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:10.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.434+0000 7f17f5c00640 1 -- 172.21.15.83:0/3721295922 shutdown_connections 2024-04-24T09:12:10.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.434+0000 7f17f5c00640 1 -- 172.21.15.83:0/3721295922 wait complete. 2024-04-24T09:12:10.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.436+0000 7f17f5c00640 1 Processor -- start 2024-04-24T09:12:10.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.436+0000 7f17f5c00640 1 -- start start 2024-04-24T09:12:10.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.436+0000 7f17f5c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f17f0102220 0x7f17f019e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:10.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.436+0000 7f17f5c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17f01072f0 0x7f17f019eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:10.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.436+0000 7f17f5c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f17f01986f0 0x7f17f0198b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:10.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.436+0000 7f17f5c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f17f01122a0 con 0x7f17f01072f0 2024-04-24T09:12:10.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.436+0000 7f17f5c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f17f0112120 con 0x7f17f01986f0 2024-04-24T09:12:10.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.437+0000 7f17f5c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f17f0112420 con 0x7f17f0102220 2024-04-24T09:12:10.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.437+0000 7f17ef400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f17f01986f0 0x7f17f0198b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:10.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.437+0000 7f17ef400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f17f01986f0 0x7f17f0198b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:59384/0 (socket says 172.21.15.83:59384) 2024-04-24T09:12:10.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.437+0000 7f17ef400640 1 -- 172.21.15.83:0/2321495669 learned_addr learned my addr 172.21.15.83:0/2321495669 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:12:10.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.437+0000 7f17ee000640 1 --2- 172.21.15.83:0/2321495669 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17f01072f0 0x7f17f019eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:10.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.437+0000 7f17eea00640 1 --2- 172.21.15.83:0/2321495669 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f17f0102220 0x7f17f019e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:10.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.437+0000 7f17ef400640 1 -- 172.21.15.83:0/2321495669 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f17f0102220 msgr2=0x7f17f019e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:10.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.437+0000 7f17ef400640 1 --2- 172.21.15.83:0/2321495669 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f17f0102220 0x7f17f019e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:10.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.437+0000 7f17ef400640 1 -- 172.21.15.83:0/2321495669 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17f01072f0 msgr2=0x7f17f019eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:10.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.437+0000 7f17ef400640 1 --2- 172.21.15.83:0/2321495669 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17f01072f0 0x7f17f019eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:10.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.437+0000 7f17ef400640 1 -- 172.21.15.83:0/2321495669 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17f01992f0 con 0x7f17f01986f0 2024-04-24T09:12:10.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.438+0000 7f17ef400640 1 --2- 172.21.15.83:0/2321495669 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f17f01986f0 0x7f17f0198b70 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f17e800eaa0 tx=0x7f17e800ef60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:10.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.439+0000 7f17e7400640 1 -- 172.21.15.83:0/2321495669 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e800cdb0 con 0x7f17f01986f0 2024-04-24T09:12:10.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.439+0000 7f17e7400640 1 -- 172.21.15.83:0/2321495669 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f17e8005020 con 0x7f17f01986f0 2024-04-24T09:12:10.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.439+0000 7f17e7400640 1 -- 172.21.15.83:0/2321495669 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e80116d0 con 0x7f17f01986f0 2024-04-24T09:12:10.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.439+0000 7f17f5c00640 1 -- 172.21.15.83:0/2321495669 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17f0074230 con 0x7f17f01986f0 2024-04-24T09:12:10.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.439+0000 7f17f5c00640 1 -- 172.21.15.83:0/2321495669 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17f0074730 con 0x7f17f01986f0 2024-04-24T09:12:10.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.447+0000 7f17f8b92740 1 Processor -- start 2024-04-24T09:12:10.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.447+0000 7f17e7400640 1 -- 172.21.15.83:0/2321495669 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f17e80051d0 con 0x7f17f01986f0 2024-04-24T09:12:10.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.449+0000 7f17e7400640 1 --2- 172.21.15.83:0/2321495669 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f17d40a3e20 0x7f17d40a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:10.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.450+0000 7f17e7400640 1 -- 172.21.15.83:0/2321495669 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 9583+0+0 (secure 0 0 0) 0x7f17e80d0e10 con 0x7f17f01986f0 2024-04-24T09:12:10.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.450+0000 7f17ee000640 1 --2- 172.21.15.83:0/2321495669 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f17d40a3e20 0x7f17d40a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:10.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.450+0000 7f17f8b92740 1 -- start start 2024-04-24T09:12:10.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.451+0000 7f17f8b92740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e1cd934950 0x55e1cd934d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:10.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.451+0000 7f17f8b92740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e1cd935270 0x55e1cd93f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:10.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.451+0000 7f17f8b92740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e1cd93f940 0x55e1cd941d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:10.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.451+0000 7f17f8b92740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e1cd89a1d0 con 0x55e1cd93f940 2024-04-24T09:12:10.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.451+0000 7f17f8b92740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e1cd840450 con 0x55e1cd934950 2024-04-24T09:12:10.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.451+0000 7f17f8b92740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55e1cd896190 con 0x55e1cd935270 2024-04-24T09:12:10.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.451+0000 7f17ef400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e1cd93f940 0x55e1cd941d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:10.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.451+0000 7f17ef400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e1cd93f940 0x55e1cd941d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50106/0 (socket says 172.21.15.83:50106) 2024-04-24T09:12:10.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.451+0000 7f17ef400640 1 -- 172.21.15.83:0/2465063071 learned_addr learned my addr 172.21.15.83:0/2465063071 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:12:10.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.452+0000 7f17eea00640 1 --2- 172.21.15.83:0/2465063071 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e1cd934950 0x55e1cd934d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:10.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.452+0000 7f17ee000640 1 --2- 172.21.15.83:0/2321495669 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f17d40a3e20 0x7f17d40a62e0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f17f0199ef0 tx=0x7f17d8005f90 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:10.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.452+0000 7f17ef400640 1 -- 172.21.15.83:0/2465063071 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e1cd935270 msgr2=0x55e1cd93f400 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T09:12:10.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.452+0000 7f17ef400640 1 --2- 172.21.15.83:0/2465063071 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e1cd935270 0x55e1cd93f400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:10.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.452+0000 7f17ef400640 1 -- 172.21.15.83:0/2465063071 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e1cd934950 msgr2=0x55e1cd934d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:10.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.452+0000 7f17ef400640 1 --2- 172.21.15.83:0/2465063071 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e1cd934950 0x55e1cd934d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:10.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.452+0000 7f17ef400640 1 -- 172.21.15.83:0/2465063071 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1cd8fd310 con 0x55e1cd93f940 2024-04-24T09:12:10.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.452+0000 7f17eea00640 1 --2- 172.21.15.83:0/2465063071 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e1cd934950 0x55e1cd934d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:12:10.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.453+0000 7f17ef400640 1 --2- 172.21.15.83:0/2465063071 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e1cd93f940 0x55e1cd941d30 secure :-1 s=READY pgs=2547 cs=0 l=1 rev1=1 crypto rx=0x7f17e80a5070 tx=0x7f17e809d0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7970381b87280f42 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:10.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.454+0000 7f17dfe00640 1 -- 172.21.15.83:0/2465063071 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e801f040 con 0x55e1cd93f940 2024-04-24T09:12:10.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.454+0000 7f17dfe00640 1 -- 172.21.15.83:0/2465063071 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f17e80a55a0 con 0x55e1cd93f940 2024-04-24T09:12:10.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.454+0000 7f17dfe00640 1 -- 172.21.15.83:0/2465063071 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e801f040 con 0x55e1cd93f940 2024-04-24T09:12:10.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.455+0000 7f17f8b92740 1 -- 172.21.15.83:0/2465063071 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e1cd93f940 msgr2=0x55e1cd941d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:10.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.455+0000 7f17f8b92740 1 --2- 172.21.15.83:0/2465063071 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e1cd93f940 0x55e1cd941d30 secure :-1 s=READY pgs=2547 cs=0 l=1 rev1=1 crypto rx=0x7f17e80a5070 tx=0x7f17e809d0f0 comp rx=0 tx=0).stop 2024-04-24T09:12:10.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.455+0000 7f17f8b92740 1 -- 172.21.15.83:0/2465063071 shutdown_connections 2024-04-24T09:12:10.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.455+0000 7f17f8b92740 1 --2- 172.21.15.83:0/2465063071 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e1cd93f940 0x55e1cd941d30 unknown :-1 s=CLOSED pgs=2547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:10.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.455+0000 7f17f8b92740 1 --2- 172.21.15.83:0/2465063071 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e1cd935270 0x55e1cd93f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:10.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.455+0000 7f17f8b92740 1 --2- 172.21.15.83:0/2465063071 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e1cd934950 0x55e1cd934d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:10.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.455+0000 7f17f8b92740 1 -- 172.21.15.83:0/2465063071 >> 172.21.15.83:0/2465063071 conn(0x55e1cd930c70 msgr2=0x55e1cd931050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:10.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.455+0000 7f17f8b92740 1 -- 172.21.15.83:0/2465063071 shutdown_connections 2024-04-24T09:12:10.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.456+0000 7f17f8b92740 1 -- 172.21.15.83:0/2465063071 wait complete. 2024-04-24T09:12:10.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.456+0000 7f17f8b92740 1 -- start start 2024-04-24T09:12:10.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.457+0000 7f17f8b92740 1 Processor -- start 2024-04-24T09:12:10.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.458+0000 7f17f8b92740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e1cd934950 0x55e1cd93c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:10.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.458+0000 7f17f8b92740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e1cd935270 0x55e1cd93ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:10.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.458+0000 7f17f8b92740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e1cd93f940 0x55e1cd93d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:10.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.458+0000 7f17f8b92740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e1cd942710 con 0x55e1cd935270 2024-04-24T09:12:10.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.458+0000 7f17f8b92740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e1cd853810 con 0x55e1cd934950 2024-04-24T09:12:10.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.458+0000 7f17f8b92740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55e1cd942890 con 0x55e1cd93f940 2024-04-24T09:12:10.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.459+0000 7f17eea00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e1cd934950 0x55e1cd93c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:10.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.459+0000 7f17eea00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e1cd934950 0x55e1cd93c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:59398/0 (socket says 172.21.15.83:59398) 2024-04-24T09:12:10.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.459+0000 7f17eea00640 1 -- 172.21.15.83:0/1465388146 learned_addr learned my addr 172.21.15.83:0/1465388146 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:12:10.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.459+0000 7f17ef400640 1 --2- 172.21.15.83:0/1465388146 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e1cd93f940 0x55e1cd93d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:10.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.459+0000 7f17f8b92740 10 client.0 _ll_register_callbacks cb 0x55e1cd821140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:12:10.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.460+0000 7f17ee000640 1 --2- 172.21.15.83:0/1465388146 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e1cd935270 0x55e1cd93ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:10.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.460+0000 7f17ef400640 1 -- 172.21.15.83:0/1465388146 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e1cd934950 msgr2=0x55e1cd93c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:10.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.460+0000 7f17ef400640 1 --2- 172.21.15.83:0/1465388146 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e1cd934950 0x55e1cd93c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:10.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.460+0000 7f17ef400640 1 -- 172.21.15.83:0/1465388146 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e1cd935270 msgr2=0x55e1cd93ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:10.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.460+0000 7f17ef400640 1 --2- 172.21.15.83:0/1465388146 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e1cd935270 0x55e1cd93ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:10.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.460+0000 7f17ef400640 1 -- 172.21.15.83:0/1465388146 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1cd8643f0 con 0x55e1cd93f940 2024-04-24T09:12:10.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.460+0000 7f17ee000640 1 --2- 172.21.15.83:0/1465388146 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e1cd935270 0x55e1cd93ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:12:10.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.460+0000 7f17ef400640 1 --2- 172.21.15.83:0/1465388146 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e1cd93f940 0x55e1cd93d200 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f17e80a5070 tx=0x7f17e80984b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62c0708705374990 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:10.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.460+0000 7f17dcc00640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-04-24T09:12:10.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.461+0000 7f17dcc00640 1 -- 172.21.15.83:0/1465388146 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e801f050 con 0x55e1cd93f940 2024-04-24T09:12:10.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.461+0000 7f17dcc00640 1 -- 172.21.15.83:0/1465388146 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9583+0+0 (secure 0 0 0) 0x7f17e8087250 con 0x55e1cd93f940 2024-04-24T09:12:10.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.461+0000 7f17f8b92740 1 -- 172.21.15.83:0/1465388146 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17e80a55a0 con 0x55e1cd93f940 2024-04-24T09:12:10.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.462+0000 7f17dcc00640 1 -- 172.21.15.83:0/1465388146 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f17e8080050 con 0x55e1cd93f940 2024-04-24T09:12:10.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.462+0000 7f17f8b92740 10 client.29599 fetch_fsmap learned FSMap version 61 2024-04-24T09:12:10.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.462+0000 7f17f8b92740 1 -- 172.21.15.83:0/1465388146 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x55e1cd5ac3f0 con 0x55e1cd93f940 2024-04-24T09:12:10.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.462+0000 7f17dcc00640 1 -- 172.21.15.83:0/1465388146 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f17e80a55a0 con 0x55e1cd93f940 2024-04-24T09:12:10.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.463+0000 7f17dcc00640 1 -- 172.21.15.83:0/1465388146 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 61) ==== 8362+0+0 (secure 0 0 0) 0x7f17e800f040 con 0x55e1cd93f940 2024-04-24T09:12:10.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.464+0000 7f17f8b92740 10 client.29599 fetch_fsmap finished waiting for FSMap version 61 2024-04-24T09:12:10.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.464+0000 7f17f8b92740 10 client.29599 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:12:10.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.464+0000 7f17f8b92740 20 client.29599 populate_metadata read hostname 'smithi083' 2024-04-24T09:12:10.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.464+0000 7f17f8b92740 1 --2- 172.21.15.83:0/1465388146 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55e1cd94f910 0x55e1cd951d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:10.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.464+0000 7f17f8b92740 4 client.29599 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:12:10.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.464+0000 7f17f8b92740 1 -- 172.21.15.83:0/1465388146 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e1cd8a7bf0 con 0x55e1cd94f910 2024-04-24T09:12:10.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.465+0000 7f17eea00640 1 --2- 172.21.15.83:0/1465388146 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55e1cd94f910 0x55e1cd951d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:10.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.466+0000 7f17eea00640 1 --2- 172.21.15.83:0/1465388146 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55e1cd94f910 0x55e1cd951d00 crc :-1 s=READY pgs=509 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a08651bda2aa4b20 server_cookie=576bc8a5ecd65241 in_seq=0 out_seq=0 2024-04-24T09:12:10.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.466+0000 7f17dcc00640 10 client.29599 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:12:10.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.468+0000 7f17dcc00640 1 -- 172.21.15.83:0/1465388146 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e1cd8a7bf0 con 0x55e1cd94f910 2024-04-24T09:12:10.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.468+0000 7f17dcc00640 10 client.29599 handle_command_reply: tid=0 2024-04-24T09:12:10.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.468+0000 7f17f8b92740 1 client.29599 shutdown 2024-04-24T09:12:10.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.468+0000 7f17f8b92740 1 -- 172.21.15.83:0/1465388146 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e1cd93f940 msgr2=0x55e1cd93d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:10.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.469+0000 7f17f8b92740 1 --2- 172.21.15.83:0/1465388146 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e1cd93f940 0x55e1cd93d200 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f17e80a5070 tx=0x7f17e80984b0 comp rx=0 tx=0).stop 2024-04-24T09:12:10.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.469+0000 7f17f8b92740 1 -- 172.21.15.83:0/1465388146 shutdown_connections 2024-04-24T09:12:10.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.469+0000 7f17f8b92740 1 --2- 172.21.15.83:0/1465388146 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55e1cd94f910 0x55e1cd951d00 crc :-1 s=READY pgs=509 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:10.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.469+0000 7f17f8b92740 1 --2- 172.21.15.83:0/1465388146 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e1cd93f940 0x55e1cd93d200 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:10.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.469+0000 7f17f8b92740 1 --2- 172.21.15.83:0/1465388146 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e1cd935270 0x55e1cd93ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:10.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.469+0000 7f17f8b92740 1 --2- 172.21.15.83:0/1465388146 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e1cd934950 0x55e1cd93c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:10.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.469+0000 7f17dcc00640 0 client.29599 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:12:10.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.469+0000 7f17f8b92740 1 -- 172.21.15.83:0/1465388146 >> 172.21.15.83:0/1465388146 conn(0x55e1cd930c70 msgr2=0x55e1cd936a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:10.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.469+0000 7f17f8b92740 1 -- 172.21.15.83:0/1465388146 shutdown_connections 2024-04-24T09:12:10.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.469+0000 7f17f8b92740 1 -- 172.21.15.83:0/1465388146 wait complete. 2024-04-24T09:12:10.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.470+0000 7f17f8b92740 20 client.29599 trim_cache size 0 max 16384 2024-04-24T09:12:10.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.494+0000 7f17f8b92740 1 Processor -- start 2024-04-24T09:12:10.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.494+0000 7f17f8b92740 1 -- start start 2024-04-24T09:12:10.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.495+0000 7f17f8b92740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e1cd934950 0x55e1cd94b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:10.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.495+0000 7f17f8b92740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e1cd935270 0x55e1cd9370f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:10.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.495+0000 7f17f8b92740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e1cd93f940 0x55e1cd9572a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:10.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.495+0000 7f17f8b92740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f17f01ad4e0 con 0x55e1cd935270 2024-04-24T09:12:10.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.495+0000 7f17f8b92740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f17f01af020 con 0x55e1cd934950 2024-04-24T09:12:10.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.495+0000 7f17f8b92740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f17f01adbb0 con 0x55e1cd93f940 2024-04-24T09:12:10.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.495+0000 7f17ee000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e1cd935270 0x55e1cd9370f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:10.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.495+0000 7f17ee000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e1cd935270 0x55e1cd9370f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50122/0 (socket says 172.21.15.83:50122) 2024-04-24T09:12:10.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.495+0000 7f17ee000640 1 -- 172.21.15.83:0/4005126924 learned_addr learned my addr 172.21.15.83:0/4005126924 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:12:10.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.495+0000 7f17eea00640 1 --2- 172.21.15.83:0/4005126924 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e1cd934950 0x55e1cd94b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:10.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.496+0000 7f17ef400640 1 --2- 172.21.15.83:0/4005126924 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e1cd93f940 0x55e1cd9572a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:10.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.496+0000 7f17ee000640 1 -- 172.21.15.83:0/4005126924 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e1cd93f940 msgr2=0x55e1cd9572a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:10.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.496+0000 7f17ee000640 1 --2- 172.21.15.83:0/4005126924 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e1cd93f940 0x55e1cd9572a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:10.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.496+0000 7f17ee000640 1 -- 172.21.15.83:0/4005126924 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e1cd934950 msgr2=0x55e1cd94b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:10.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.496+0000 7f17ee000640 1 --2- 172.21.15.83:0/4005126924 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e1cd934950 0x55e1cd94b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:10.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.496+0000 7f17ee000640 1 -- 172.21.15.83:0/4005126924 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1cd952070 con 0x55e1cd935270 2024-04-24T09:12:10.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.496+0000 7f17ee000640 1 --2- 172.21.15.83:0/4005126924 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e1cd935270 0x55e1cd9370f0 secure :-1 s=READY pgs=2548 cs=0 l=1 rev1=1 crypto rx=0x7f17d8004230 tx=0x7f17d8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a549fb04a2d89389 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:10.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.497+0000 7f17df400640 1 -- 172.21.15.83:0/4005126924 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d8005020 con 0x55e1cd935270 2024-04-24T09:12:10.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.497+0000 7f17df400640 1 -- 172.21.15.83:0/4005126924 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f17d8022070 con 0x55e1cd935270 2024-04-24T09:12:10.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.497+0000 7f17df400640 1 -- 172.21.15.83:0/4005126924 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d8026680 con 0x55e1cd935270 2024-04-24T09:12:10.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.498+0000 7f17f8b92740 1 -- 172.21.15.83:0/4005126924 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e1cd935270 msgr2=0x55e1cd9370f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:10.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.498+0000 7f17f8b92740 1 --2- 172.21.15.83:0/4005126924 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e1cd935270 0x55e1cd9370f0 secure :-1 s=READY pgs=2548 cs=0 l=1 rev1=1 crypto rx=0x7f17d8004230 tx=0x7f17d8014000 comp rx=0 tx=0).stop 2024-04-24T09:12:10.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.498+0000 7f17f8b92740 1 -- 172.21.15.83:0/4005126924 shutdown_connections 2024-04-24T09:12:10.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.498+0000 7f17f8b92740 1 --2- 172.21.15.83:0/4005126924 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e1cd93f940 0x55e1cd9572a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:10.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.498+0000 7f17f8b92740 1 --2- 172.21.15.83:0/4005126924 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e1cd935270 0x55e1cd9370f0 secure :-1 s=CLOSED pgs=2548 cs=0 l=1 rev1=1 crypto rx=0x7f17d8004230 tx=0x7f17d8014000 comp rx=0 tx=0).stop 2024-04-24T09:12:10.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.498+0000 7f17f8b92740 1 --2- 172.21.15.83:0/4005126924 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e1cd934950 0x55e1cd94b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:10.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.498+0000 7f17f8b92740 1 -- 172.21.15.83:0/4005126924 >> 172.21.15.83:0/4005126924 conn(0x55e1cd930c70 msgr2=0x55e1cd92fdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:10.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.498+0000 7f17f8b92740 1 -- 172.21.15.83:0/4005126924 shutdown_connections 2024-04-24T09:12:10.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.499+0000 7f17f8b92740 1 -- 172.21.15.83:0/4005126924 wait complete. 2024-04-24T09:12:10.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.499+0000 7f17f8b92740 1 -- start start 2024-04-24T09:12:10.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.499+0000 7f17f8b92740 1 Processor -- start 2024-04-24T09:12:10.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.500+0000 7f17f8b92740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e1cd934950 0x55e1cd939700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:10.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.500+0000 7f17f8b92740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e1cd93f940 0x55e1cd939c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:10.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.500+0000 7f17f8b92740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e1cd93a180 0x55e1cd945f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:10.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.500+0000 7f17f8b92740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f17f01ae950 con 0x55e1cd934950 2024-04-24T09:12:10.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.500+0000 7f17f8b92740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f17f01ae280 con 0x55e1cd93a180 2024-04-24T09:12:10.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.500+0000 7f17f8b92740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f17f01ab4e0 con 0x55e1cd93f940 2024-04-24T09:12:10.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.500+0000 7f17f8b92740 10 client.0 _ll_register_callbacks cb 0x7f17e809c290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:12:10.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.500+0000 7f17eea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e1cd934950 0x55e1cd939700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:10.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.500+0000 7f17ef400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e1cd93a180 0x55e1cd945f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:10.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.500+0000 7f17eea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e1cd934950 0x55e1cd939700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50126/0 (socket says 172.21.15.83:50126) 2024-04-24T09:12:10.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.500+0000 7f17eea00640 1 -- 172.21.15.83:0/3905416066 learned_addr learned my addr 172.21.15.83:0/3905416066 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:12:10.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.500+0000 7f17ee000640 1 --2- 172.21.15.83:0/3905416066 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e1cd93f940 0x55e1cd939c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:10.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.501+0000 7f17eea00640 1 -- 172.21.15.83:0/3905416066 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e1cd93f940 msgr2=0x55e1cd939c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:10.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.501+0000 7f17eea00640 1 --2- 172.21.15.83:0/3905416066 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e1cd93f940 0x55e1cd939c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:10.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.501+0000 7f17eea00640 1 -- 172.21.15.83:0/3905416066 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e1cd93a180 msgr2=0x55e1cd945f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:10.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.501+0000 7f17eea00640 1 --2- 172.21.15.83:0/3905416066 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e1cd93a180 0x55e1cd945f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:10.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.501+0000 7f17eea00640 1 -- 172.21.15.83:0/3905416066 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1cd6767d0 con 0x55e1cd934950 2024-04-24T09:12:10.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.501+0000 7f17ef400640 1 --2- 172.21.15.83:0/3905416066 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e1cd93a180 0x55e1cd945f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:12:10.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.501+0000 7f17eea00640 1 --2- 172.21.15.83:0/3905416066 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e1cd934950 0x55e1cd939700 secure :-1 s=READY pgs=2549 cs=0 l=1 rev1=1 crypto rx=0x7f17e0009900 tx=0x7f17e00044f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74e1a55440d5c810 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:10.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.501+0000 7f17dcc00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:12:10.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.502+0000 7f17dcc00640 1 -- 172.21.15.83:0/3905416066 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e0035040 con 0x55e1cd934950 2024-04-24T09:12:10.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.502+0000 7f17f8b92740 1 -- 172.21.15.83:0/3905416066 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17d8022070 con 0x55e1cd934950 2024-04-24T09:12:10.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.502+0000 7f17dcc00640 1 -- 172.21.15.83:0/3905416066 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9583+0+0 (secure 0 0 0) 0x7f17e0005510 con 0x55e1cd934950 2024-04-24T09:12:10.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.502+0000 7f17dcc00640 1 -- 172.21.15.83:0/3905416066 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f17e0054050 con 0x55e1cd934950 2024-04-24T09:12:10.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.502+0000 7f17f8b92740 10 client.22707 fetch_fsmap learned FSMap version 61 2024-04-24T09:12:10.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.502+0000 7f17f8b92740 1 -- 172.21.15.83:0/3905416066 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x55e1cd9b62b0 con 0x55e1cd934950 2024-04-24T09:12:10.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.503+0000 7f17dcc00640 1 -- 172.21.15.83:0/3905416066 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f17d8022070 con 0x55e1cd934950 2024-04-24T09:12:10.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.503+0000 7f17dcc00640 1 -- 172.21.15.83:0/3905416066 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 61) ==== 8362+0+0 (secure 0 0 0) 0x7f17e00039c0 con 0x55e1cd934950 2024-04-24T09:12:10.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.503+0000 7f17f8b92740 10 client.22707 fetch_fsmap finished waiting for FSMap version 61 2024-04-24T09:12:10.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.503+0000 7f17f8b92740 10 client.22707 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:12:10.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.503+0000 7f17f8b92740 20 client.22707 populate_metadata read hostname 'smithi083' 2024-04-24T09:12:10.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.503+0000 7f17f8b92740 1 --2- 172.21.15.83:0/3905416066 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55e1cd9b6a60 0x55e1cd9b8e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:10.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.503+0000 7f17f8b92740 4 client.22707 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:12:10.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.503+0000 7f17f8b92740 1 -- 172.21.15.83:0/3905416066 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e1cd8a1a60 con 0x55e1cd9b6a60 2024-04-24T09:12:10.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.503+0000 7f17ee000640 1 --2- 172.21.15.83:0/3905416066 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55e1cd9b6a60 0x55e1cd9b8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:10.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.505+0000 7f17dcc00640 10 client.22707 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:12:10.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.505+0000 7f17ee000640 1 --2- 172.21.15.83:0/3905416066 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55e1cd9b6a60 0x55e1cd9b8e80 crc :-1 s=READY pgs=510 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b30a56d68b6fa258 server_cookie=cde6dda9282cf18b in_seq=0 out_seq=0 2024-04-24T09:12:10.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.507+0000 7f17dcc00640 1 -- 172.21.15.83:0/3905416066 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e1cd8a1a60 con 0x55e1cd9b6a60 2024-04-24T09:12:10.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.507+0000 7f17dcc00640 10 client.22707 handle_command_reply: tid=0 2024-04-24T09:12:10.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.507+0000 7f17f8b92740 1 client.22707 shutdown 2024-04-24T09:12:10.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.508+0000 7f17f8b92740 1 -- 172.21.15.83:0/3905416066 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e1cd934950 msgr2=0x55e1cd939700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:10.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.508+0000 7f17f8b92740 1 --2- 172.21.15.83:0/3905416066 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e1cd934950 0x55e1cd939700 secure :-1 s=READY pgs=2549 cs=0 l=1 rev1=1 crypto rx=0x7f17e0009900 tx=0x7f17e00044f0 comp rx=0 tx=0).stop 2024-04-24T09:12:10.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.508+0000 7f17f8b92740 1 -- 172.21.15.83:0/3905416066 shutdown_connections 2024-04-24T09:12:10.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.508+0000 7f17f8b92740 1 --2- 172.21.15.83:0/3905416066 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55e1cd9b6a60 0x55e1cd9b8e80 crc :-1 s=READY pgs=510 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:10.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.508+0000 7f17f8b92740 1 --2- 172.21.15.83:0/3905416066 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e1cd93a180 0x55e1cd945f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:10.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.508+0000 7f17f8b92740 1 --2- 172.21.15.83:0/3905416066 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e1cd93f940 0x55e1cd939c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:10.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.508+0000 7f17f8b92740 1 --2- 172.21.15.83:0/3905416066 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e1cd934950 0x55e1cd939700 secure :-1 s=CLOSED pgs=2549 cs=0 l=1 rev1=1 crypto rx=0x7f17e0009900 tx=0x7f17e00044f0 comp rx=0 tx=0).stop 2024-04-24T09:12:10.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.508+0000 7f17f8b92740 1 -- 172.21.15.83:0/3905416066 >> 172.21.15.83:0/3905416066 conn(0x55e1cd930c70 msgr2=0x55e1cd92f560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:10.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.508+0000 7f17dcc00640 0 client.22707 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:12:10.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.509+0000 7f17f8b92740 1 -- 172.21.15.83:0/3905416066 shutdown_connections 2024-04-24T09:12:10.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.510+0000 7f17f8b92740 1 -- 172.21.15.83:0/3905416066 wait complete. 2024-04-24T09:12:10.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.510+0000 7f17f8b92740 20 client.22707 trim_cache size 0 max 16384 2024-04-24T09:12:10.509 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:12:10.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.511+0000 7f17f5c00640 1 -- 172.21.15.83:0/2321495669 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f17d40a3e20 msgr2=0x7f17d40a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:10.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.511+0000 7f17f5c00640 1 --2- 172.21.15.83:0/2321495669 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f17d40a3e20 0x7f17d40a62e0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f17f0199ef0 tx=0x7f17d8005f90 comp rx=0 tx=0).stop 2024-04-24T09:12:10.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.511+0000 7f17f5c00640 1 -- 172.21.15.83:0/2321495669 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f17f01986f0 msgr2=0x7f17f0198b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:10.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.511+0000 7f17f5c00640 1 --2- 172.21.15.83:0/2321495669 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f17f01986f0 0x7f17f0198b70 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f17e800eaa0 tx=0x7f17e800ef60 comp rx=0 tx=0).stop 2024-04-24T09:12:10.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.512+0000 7f17f5c00640 1 -- 172.21.15.83:0/2321495669 shutdown_connections 2024-04-24T09:12:10.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.512+0000 7f17f5c00640 1 --2- 172.21.15.83:0/2321495669 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f17d40a3e20 0x7f17d40a62e0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:10.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.512+0000 7f17f5c00640 1 --2- 172.21.15.83:0/2321495669 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f17f01986f0 0x7f17f0198b70 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:10.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.512+0000 7f17f5c00640 1 --2- 172.21.15.83:0/2321495669 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17f01072f0 0x7f17f019eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:10.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.512+0000 7f17f5c00640 1 --2- 172.21.15.83:0/2321495669 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f17f0102220 0x7f17f019e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:10.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.512+0000 7f17f5c00640 1 -- 172.21.15.83:0/2321495669 >> 172.21.15.83:0/2321495669 conn(0x7f17f00fda80 msgr2=0x7f17f00feee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:10.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.512+0000 7f17f5c00640 1 -- 172.21.15.83:0/2321495669 shutdown_connections 2024-04-24T09:12:10.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:10.513+0000 7f17f5c00640 1 -- 172.21.15.83:0/2321495669 wait complete. 2024-04-24T09:12:10.706 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:12:10 smithi138 ceph-mon[31441]: pgmap v648: 129 pgs: 129 active+clean; 789 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.3 MiB/s wr, 286 op/s 2024-04-24T09:12:10.706 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:12:10 smithi138 ceph-mon[31441]: osdmap e124: 12 total, 12 up, 12 in 2024-04-24T09:12:10.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:12:10 smithi189 ceph-mon[31791]: pgmap v648: 129 pgs: 129 active+clean; 789 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.3 MiB/s wr, 286 op/s 2024-04-24T09:12:10.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:12:10 smithi189 ceph-mon[31791]: osdmap e124: 12 total, 12 up, 12 in 2024-04-24T09:12:10.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:12:10 smithi083 ceph-mon[27907]: pgmap v648: 129 pgs: 129 active+clean; 789 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.3 MiB/s wr, 286 op/s 2024-04-24T09:12:10.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:12:10 smithi083 ceph-mon[27907]: osdmap e124: 12 total, 12 up, 12 in 2024-04-24T09:12:11.184 INFO:tasks.workunit.client.0.smithi083.stdout: 10 50266 6.65 MB/sec execute 282 sec latency 93.281 ms 2024-04-24T09:12:11.537 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T09:12:11.537 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T09:12:11.537 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T09:12:11.704 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:12:11 smithi138 ceph-mon[31441]: pgmap v650: 129 pgs: 129 active+clean; 789 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.0 MiB/s wr, 334 op/s 2024-04-24T09:12:11.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:12:11 smithi189 ceph-mon[31791]: pgmap v650: 129 pgs: 129 active+clean; 789 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.0 MiB/s wr, 334 op/s 2024-04-24T09:12:11.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:12:11 smithi083 ceph-mon[27907]: pgmap v650: 129 pgs: 129 active+clean; 789 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.0 MiB/s wr, 334 op/s 2024-04-24T09:12:11.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.808+0000 7f515d600640 1 -- 172.21.15.83:0/1393854770 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f51581093f0 msgr2=0x7f5158110fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:11.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.808+0000 7f515d600640 1 --2- 172.21.15.83:0/1393854770 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f51581093f0 0x7f5158110fc0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f5150009ca0 tx=0x7f515002a8c0 comp rx=0 tx=0).stop 2024-04-24T09:12:11.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.808+0000 7f515d600640 1 -- 172.21.15.83:0/1393854770 shutdown_connections 2024-04-24T09:12:11.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.808+0000 7f515d600640 1 --2- 172.21.15.83:0/1393854770 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f51581093f0 0x7f5158110fc0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:11.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.808+0000 7f515d600640 1 --2- 172.21.15.83:0/1393854770 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5158104cd0 0x7f5158108d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:11.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.808+0000 7f515d600640 1 --2- 172.21.15.83:0/1393854770 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5158104320 0x7f5158104700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:11.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.808+0000 7f515d600640 1 -- 172.21.15.83:0/1393854770 >> 172.21.15.83:0/1393854770 conn(0x7f51580ffb40 msgr2=0x7f5158101f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:11.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.809+0000 7f515d600640 1 -- 172.21.15.83:0/1393854770 shutdown_connections 2024-04-24T09:12:11.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.809+0000 7f515d600640 1 -- 172.21.15.83:0/1393854770 wait complete. 2024-04-24T09:12:11.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.810+0000 7f515d600640 1 Processor -- start 2024-04-24T09:12:11.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.811+0000 7f515d600640 1 -- start start 2024-04-24T09:12:11.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.811+0000 7f515d600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5158104320 0x7f5158110fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:11.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.811+0000 7f515d600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51581093f0 0x7f515810e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:11.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.811+0000 7f515d600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f515810e820 0x7f51581ab0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:11.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.811+0000 7f515d600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f51581143b0 con 0x7f51581093f0 2024-04-24T09:12:11.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.811+0000 7f515d600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5158114230 con 0x7f5158104320 2024-04-24T09:12:11.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.811+0000 7f515d600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5158114530 con 0x7f515810e820 2024-04-24T09:12:11.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.811+0000 7f5157e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5158104320 0x7f5158110fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:11.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.811+0000 7f5157e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5158104320 0x7f5158110fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:59414/0 (socket says 172.21.15.83:59414) 2024-04-24T09:12:11.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.812+0000 7f5157e00640 1 -- 172.21.15.83:0/145434671 learned_addr learned my addr 172.21.15.83:0/145434671 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:12:11.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.812+0000 7f5157400640 1 --2- 172.21.15.83:0/145434671 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51581093f0 0x7f515810e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:11.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.812+0000 7f515cc00640 1 --2- 172.21.15.83:0/145434671 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f515810e820 0x7f51581ab0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:11.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.812+0000 7f5157e00640 1 -- 172.21.15.83:0/145434671 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f515810e820 msgr2=0x7f51581ab0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:11.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.812+0000 7f5157e00640 1 --2- 172.21.15.83:0/145434671 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f515810e820 0x7f51581ab0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:11.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.812+0000 7f5157e00640 1 -- 172.21.15.83:0/145434671 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51581093f0 msgr2=0x7f515810e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:11.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.812+0000 7f5157e00640 1 --2- 172.21.15.83:0/145434671 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51581093f0 0x7f515810e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:11.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.812+0000 7f5157e00640 1 -- 172.21.15.83:0/145434671 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51581ab630 con 0x7f5158104320 2024-04-24T09:12:11.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.813+0000 7f5157400640 1 --2- 172.21.15.83:0/145434671 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51581093f0 0x7f515810e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:12:11.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.813+0000 7f5157e00640 1 --2- 172.21.15.83:0/145434671 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5158104320 0x7f5158110fc0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f514800e9f0 tx=0x7f514800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:11.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.813+0000 7f5154c00640 1 -- 172.21.15.83:0/145434671 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f514800cd80 con 0x7f5158104320 2024-04-24T09:12:11.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.814+0000 7f515d600640 1 -- 172.21.15.83:0/145434671 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51581ab800 con 0x7f5158104320 2024-04-24T09:12:11.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.814+0000 7f515d600640 1 -- 172.21.15.83:0/145434671 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51581abc70 con 0x7f5158104320 2024-04-24T09:12:11.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.814+0000 7f5154c00640 1 -- 172.21.15.83:0/145434671 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f51480054d0 con 0x7f5158104320 2024-04-24T09:12:11.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.814+0000 7f5154c00640 1 -- 172.21.15.83:0/145434671 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5148010490 con 0x7f5158104320 2024-04-24T09:12:11.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.817+0000 7f5161a70740 1 Processor -- start 2024-04-24T09:12:11.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.818+0000 7f5161a70740 1 -- start start 2024-04-24T09:12:11.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.818+0000 7f5161a70740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557b4e5a4270 0x557b4e5a4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:11.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.818+0000 7f5161a70740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557b4e5a4b90 0x557b4e5aed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:11.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.818+0000 7f5161a70740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557b4e5af260 0x557b4e5b1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:11.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.818+0000 7f5161a70740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x557b4e509e70 con 0x557b4e5af260 2024-04-24T09:12:11.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.818+0000 7f5161a70740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x557b4e4b0380 con 0x557b4e5a4270 2024-04-24T09:12:11.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.818+0000 7f5161a70740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x557b4e505e30 con 0x557b4e5a4b90 2024-04-24T09:12:11.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.819+0000 7f515cc00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557b4e5a4270 0x557b4e5a4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:11.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.819+0000 7f515cc00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557b4e5a4270 0x557b4e5a4650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:59430/0 (socket says 172.21.15.83:59430) 2024-04-24T09:12:11.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.819+0000 7f515cc00640 1 -- 172.21.15.83:0/3508752529 learned_addr learned my addr 172.21.15.83:0/3508752529 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:12:11.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.819+0000 7f515cc00640 1 -- 172.21.15.83:0/3508752529 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557b4e5a4b90 msgr2=0x557b4e5aed20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T09:12:11.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.819+0000 7f515cc00640 1 --2- 172.21.15.83:0/3508752529 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557b4e5a4b90 0x557b4e5aed20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:11.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.819+0000 7f515cc00640 1 -- 172.21.15.83:0/3508752529 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557b4e5af260 msgr2=0x557b4e5b1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:11.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.819+0000 7f515cc00640 1 --2- 172.21.15.83:0/3508752529 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557b4e5af260 0x557b4e5b1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:11.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.819+0000 7f515cc00640 1 -- 172.21.15.83:0/3508752529 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b4e2e47e0 con 0x557b4e5a4270 2024-04-24T09:12:11.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.820+0000 7f515cc00640 1 --2- 172.21.15.83:0/3508752529 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557b4e5a4270 0x557b4e5a4650 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f515002aeb0 tx=0x7f5150005f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31cc9287141e2667 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:11.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.820+0000 7f5154c00640 1 -- 172.21.15.83:0/145434671 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5148005020 con 0x7f5158104320 2024-04-24T09:12:11.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.821+0000 7f514d200640 1 -- 172.21.15.83:0/3508752529 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5150031d40 con 0x557b4e5a4270 2024-04-24T09:12:11.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.821+0000 7f514d200640 1 -- 172.21.15.83:0/3508752529 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5150036030 con 0x557b4e5a4270 2024-04-24T09:12:11.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.821+0000 7f514d200640 1 -- 172.21.15.83:0/3508752529 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5150031d40 con 0x557b4e5a4270 2024-04-24T09:12:11.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.823+0000 7f5161a70740 1 -- 172.21.15.83:0/3508752529 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557b4e5a4270 msgr2=0x557b4e5a4650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:11.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.823+0000 7f5161a70740 1 --2- 172.21.15.83:0/3508752529 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557b4e5a4270 0x557b4e5a4650 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f515002aeb0 tx=0x7f5150005f30 comp rx=0 tx=0).stop 2024-04-24T09:12:11.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.824+0000 7f5161a70740 1 -- 172.21.15.83:0/3508752529 shutdown_connections 2024-04-24T09:12:11.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.824+0000 7f5161a70740 1 --2- 172.21.15.83:0/3508752529 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557b4e5af260 0x557b4e5b1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:11.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.824+0000 7f5161a70740 1 --2- 172.21.15.83:0/3508752529 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557b4e5a4b90 0x557b4e5aed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:11.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.824+0000 7f5161a70740 1 --2- 172.21.15.83:0/3508752529 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557b4e5a4270 0x557b4e5a4650 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:11.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.824+0000 7f5161a70740 1 -- 172.21.15.83:0/3508752529 >> 172.21.15.83:0/3508752529 conn(0x557b4e5a0590 msgr2=0x557b4e5a0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:11.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.824+0000 7f5154c00640 1 --2- 172.21.15.83:0/145434671 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f513c0a3da0 0x7f513c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:11.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.824+0000 7f5154c00640 1 -- 172.21.15.83:0/145434671 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 9583+0+0 (secure 0 0 0) 0x7f51480d0db0 con 0x7f5158104320 2024-04-24T09:12:11.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.824+0000 7f5157400640 1 --2- 172.21.15.83:0/145434671 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f513c0a3da0 0x7f513c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:11.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.825+0000 7f5161a70740 1 -- 172.21.15.83:0/3508752529 shutdown_connections 2024-04-24T09:12:11.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.826+0000 7f5161a70740 1 -- 172.21.15.83:0/3508752529 wait complete. 2024-04-24T09:12:11.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.826+0000 7f5161a70740 1 -- start start 2024-04-24T09:12:11.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.826+0000 7f5161a70740 1 Processor -- start 2024-04-24T09:12:11.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.827+0000 7f5157400640 1 --2- 172.21.15.83:0/145434671 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f513c0a3da0 0x7f513c0a6260 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x557b4e5a1720 tx=0x7f51440046c0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:11.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.827+0000 7f5161a70740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557b4e5a4270 0x557b4e5ac170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:11.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.827+0000 7f5161a70740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557b4e5a4b90 0x557b4e5ac6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:11.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.827+0000 7f5161a70740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557b4e5af260 0x557b4e5acbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:11.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.827+0000 7f5161a70740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x557b4e5c2e80 con 0x557b4e5af260 2024-04-24T09:12:11.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.827+0000 7f5161a70740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x557b4e4c3660 con 0x557b4e5a4b90 2024-04-24T09:12:11.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.827+0000 7f5161a70740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x557b4e5c3000 con 0x557b4e5a4270 2024-04-24T09:12:11.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.828+0000 7f5161a70740 10 client.0 _ll_register_callbacks cb 0x557b4e4b1aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:12:11.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.828+0000 7f515cc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557b4e5af260 0x557b4e5acbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:11.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.828+0000 7f5157e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557b4e5a4270 0x557b4e5ac170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:11.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.828+0000 7f5157400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557b4e5a4b90 0x557b4e5ac6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:11.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.828+0000 7f5157e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557b4e5a4270 0x557b4e5ac170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:59984/0 (socket says 172.21.15.83:59984) 2024-04-24T09:12:11.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.828+0000 7f5157e00640 1 -- 172.21.15.83:0/4055405648 learned_addr learned my addr 172.21.15.83:0/4055405648 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:12:11.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.829+0000 7f5157400640 1 -- 172.21.15.83:0/4055405648 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557b4e5a4270 msgr2=0x557b4e5ac170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:11.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.829+0000 7f5157400640 1 --2- 172.21.15.83:0/4055405648 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557b4e5a4270 0x557b4e5ac170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:11.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.829+0000 7f5157400640 1 -- 172.21.15.83:0/4055405648 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557b4e5af260 msgr2=0x557b4e5acbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:11.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.829+0000 7f5157400640 1 --2- 172.21.15.83:0/4055405648 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557b4e5af260 0x557b4e5acbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:11.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.829+0000 7f5157400640 1 -- 172.21.15.83:0/4055405648 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b4e344f70 con 0x557b4e5a4b90 2024-04-24T09:12:11.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.829+0000 7f515cc00640 1 --2- 172.21.15.83:0/4055405648 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557b4e5af260 0x557b4e5acbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:12:11.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.829+0000 7f5157400640 1 --2- 172.21.15.83:0/4055405648 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557b4e5a4b90 0x557b4e5ac6b0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f5144003850 tx=0x7f5144005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44daca4adcc3e642 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:11.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.829+0000 7f5141600640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:12:11.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.830+0000 7f5141600640 1 -- 172.21.15.83:0/4055405648 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f514401cb80 con 0x557b4e5a4b90 2024-04-24T09:12:11.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.830+0000 7f5141600640 1 -- 172.21.15.83:0/4055405648 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9583+0+0 (secure 0 0 0) 0x7f514401cd20 con 0x557b4e5a4b90 2024-04-24T09:12:11.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.830+0000 7f5161a70740 1 -- 172.21.15.83:0/4055405648 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5150036030 con 0x557b4e5a4b90 2024-04-24T09:12:11.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.830+0000 7f5141600640 1 -- 172.21.15.83:0/4055405648 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f514401a460 con 0x557b4e5a4b90 2024-04-24T09:12:11.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.831+0000 7f5161a70740 10 client.39761 fetch_fsmap learned FSMap version 61 2024-04-24T09:12:11.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.831+0000 7f5161a70740 1 -- 172.21.15.83:0/4055405648 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x557b4e1dc170 con 0x557b4e5a4b90 2024-04-24T09:12:11.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.831+0000 7f5141600640 1 -- 172.21.15.83:0/4055405648 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5150036030 con 0x557b4e5a4b90 2024-04-24T09:12:11.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.831+0000 7f5141600640 1 -- 172.21.15.83:0/4055405648 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 61) ==== 8362+0+0 (secure 0 0 0) 0x7f5144023070 con 0x557b4e5a4b90 2024-04-24T09:12:11.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.831+0000 7f5161a70740 10 client.39761 fetch_fsmap finished waiting for FSMap version 61 2024-04-24T09:12:11.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.831+0000 7f5161a70740 10 client.39761 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:12:11.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.831+0000 7f5161a70740 20 client.39761 populate_metadata read hostname 'smithi083' 2024-04-24T09:12:11.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.831+0000 7f5161a70740 1 --2- 172.21.15.83:0/4055405648 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x557b4e5d0510 0x557b4e5d2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:11.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.832+0000 7f5161a70740 4 client.39761 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:12:11.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.832+0000 7f5161a70740 1 -- 172.21.15.83:0/4055405648 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557b4e517890 con 0x557b4e5d0510 2024-04-24T09:12:11.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.832+0000 7f5157e00640 1 --2- 172.21.15.83:0/4055405648 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x557b4e5d0510 0x557b4e5d2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:11.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.833+0000 7f5157e00640 1 --2- 172.21.15.83:0/4055405648 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x557b4e5d0510 0x557b4e5d2930 crc :-1 s=READY pgs=511 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aafe06886ac8798b server_cookie=f8996cd65f0f0aa4 in_seq=0 out_seq=0 2024-04-24T09:12:11.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.833+0000 7f5141600640 10 client.39761 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:12:11.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.834+0000 7f5141600640 1 -- 172.21.15.83:0/4055405648 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557b4e517890 con 0x557b4e5d0510 2024-04-24T09:12:11.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.834+0000 7f5141600640 10 client.39761 handle_command_reply: tid=0 2024-04-24T09:12:11.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.834+0000 7f5161a70740 1 client.39761 shutdown 2024-04-24T09:12:11.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.835+0000 7f5161a70740 1 -- 172.21.15.83:0/4055405648 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557b4e5a4b90 msgr2=0x557b4e5ac6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:11.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.835+0000 7f5161a70740 1 --2- 172.21.15.83:0/4055405648 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557b4e5a4b90 0x557b4e5ac6b0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f5144003850 tx=0x7f5144005f90 comp rx=0 tx=0).stop 2024-04-24T09:12:11.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.835+0000 7f5161a70740 1 -- 172.21.15.83:0/4055405648 shutdown_connections 2024-04-24T09:12:11.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.835+0000 7f5161a70740 1 --2- 172.21.15.83:0/4055405648 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x557b4e5d0510 0x557b4e5d2930 crc :-1 s=READY pgs=511 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:11.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.835+0000 7f5161a70740 1 --2- 172.21.15.83:0/4055405648 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557b4e5af260 0x557b4e5acbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:11.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.835+0000 7f5161a70740 1 --2- 172.21.15.83:0/4055405648 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557b4e5a4b90 0x557b4e5ac6b0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:11.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.835+0000 7f5161a70740 1 --2- 172.21.15.83:0/4055405648 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557b4e5a4270 0x557b4e5ac170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:11.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.835+0000 7f5161a70740 1 -- 172.21.15.83:0/4055405648 >> 172.21.15.83:0/4055405648 conn(0x557b4e5a0590 msgr2=0x557b4e5a6250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:11.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.835+0000 7f5141600640 0 client.39761 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:12:11.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.836+0000 7f5161a70740 1 -- 172.21.15.83:0/4055405648 shutdown_connections 2024-04-24T09:12:11.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.836+0000 7f5161a70740 1 -- 172.21.15.83:0/4055405648 wait complete. 2024-04-24T09:12:11.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.836+0000 7f5161a70740 20 client.39761 trim_cache size 0 max 16384 2024-04-24T09:12:11.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.872+0000 7f5161a70740 1 Processor -- start 2024-04-24T09:12:11.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.872+0000 7f5161a70740 1 -- start start 2024-04-24T09:12:11.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.873+0000 7f5161a70740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557b4e5a4270 0x557b4e5cbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:11.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.873+0000 7f5161a70740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557b4e5a4b90 0x557b4e5d8200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:11.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.873+0000 7f5161a70740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557b4e5af260 0x557b4e5d8740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:11.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.873+0000 7f5157e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557b4e5a4270 0x557b4e5cbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:11.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.873+0000 7f5157e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557b4e5a4270 0x557b4e5cbdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:59452/0 (socket says 172.21.15.83:59452) 2024-04-24T09:12:11.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.874+0000 7f515cc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557b4e5af260 0x557b4e5d8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:11.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.874+0000 7f5157400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557b4e5a4b90 0x557b4e5d8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:11.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.874+0000 7f5157400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557b4e5a4b90 0x557b4e5d8200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:60000/0 (socket says 172.21.15.83:60000) 2024-04-24T09:12:11.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.874+0000 7f515cc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557b4e5af260 0x557b4e5d8740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50168/0 (socket says 172.21.15.83:50168) 2024-04-24T09:12:11.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.874+0000 7f5157e00640 1 -- 172.21.15.83:0/523459731 learned_addr learned my addr 172.21.15.83:0/523459731 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:12:11.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.874+0000 7f5161a70740 1 -- 172.21.15.83:0/523459731 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5138010110 con 0x557b4e5af260 2024-04-24T09:12:11.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.874+0000 7f5161a70740 1 -- 172.21.15.83:0/523459731 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5138011c50 con 0x557b4e5a4270 2024-04-24T09:12:11.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.874+0000 7f5161a70740 1 -- 172.21.15.83:0/523459731 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f51380107e0 con 0x557b4e5a4b90 2024-04-24T09:12:11.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.874+0000 7f5157e00640 1 -- 172.21.15.83:0/523459731 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557b4e5a4b90 msgr2=0x557b4e5d8200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:11.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.874+0000 7f5157e00640 1 --2- 172.21.15.83:0/523459731 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557b4e5a4b90 0x557b4e5d8200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:11.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.874+0000 7f5157e00640 1 -- 172.21.15.83:0/523459731 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557b4e5af260 msgr2=0x557b4e5d8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:11.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.874+0000 7f5157e00640 1 --2- 172.21.15.83:0/523459731 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557b4e5af260 0x557b4e5d8740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:11.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.874+0000 7f5157e00640 1 -- 172.21.15.83:0/523459731 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b4e5a50e0 con 0x557b4e5a4270 2024-04-24T09:12:11.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.875+0000 7f5157e00640 1 --2- 172.21.15.83:0/523459731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557b4e5a4270 0x557b4e5cbdf0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f5148009810 tx=0x7f5148087040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92c47c5a7870a3c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:11.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.875+0000 7f5143e00640 1 -- 172.21.15.83:0/523459731 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f514809cfd0 con 0x557b4e5a4270 2024-04-24T09:12:11.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.875+0000 7f5143e00640 1 -- 172.21.15.83:0/523459731 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f514809c8d0 con 0x557b4e5a4270 2024-04-24T09:12:11.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.875+0000 7f5143e00640 1 -- 172.21.15.83:0/523459731 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51480870b0 con 0x557b4e5a4270 2024-04-24T09:12:11.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.876+0000 7f515cc00640 1 --2- 172.21.15.83:0/523459731 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557b4e5af260 0x557b4e5d8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-04-24T09:12:11.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.876+0000 7f5161a70740 1 -- 172.21.15.83:0/523459731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557b4e5a4270 msgr2=0x557b4e5cbdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:11.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.876+0000 7f5161a70740 1 --2- 172.21.15.83:0/523459731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557b4e5a4270 0x557b4e5cbdf0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f5148009810 tx=0x7f5148087040 comp rx=0 tx=0).stop 2024-04-24T09:12:11.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.876+0000 7f5157400640 1 --2- 172.21.15.83:0/523459731 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557b4e5a4b90 0x557b4e5d8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-04-24T09:12:11.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.876+0000 7f5161a70740 1 -- 172.21.15.83:0/523459731 shutdown_connections 2024-04-24T09:12:11.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.876+0000 7f5161a70740 1 --2- 172.21.15.83:0/523459731 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557b4e5af260 0x557b4e5d8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:11.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.876+0000 7f5161a70740 1 --2- 172.21.15.83:0/523459731 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557b4e5a4b90 0x557b4e5d8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:11.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.876+0000 7f5161a70740 1 --2- 172.21.15.83:0/523459731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557b4e5a4270 0x557b4e5cbdf0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:11.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.876+0000 7f5161a70740 1 -- 172.21.15.83:0/523459731 >> 172.21.15.83:0/523459731 conn(0x557b4e5a0590 msgr2=0x557b4e5a59c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:11.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.877+0000 7f5161a70740 1 -- 172.21.15.83:0/523459731 shutdown_connections 2024-04-24T09:12:11.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.877+0000 7f5161a70740 1 -- 172.21.15.83:0/523459731 wait complete. 2024-04-24T09:12:11.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.877+0000 7f5161a70740 1 -- start start 2024-04-24T09:12:11.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.878+0000 7f5161a70740 1 Processor -- start 2024-04-24T09:12:11.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.881+0000 7f5161a70740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557b4e5a4270 0x557b4e5c8740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:11.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.881+0000 7f5161a70740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557b4e5a4b90 0x557b4e5c3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:11.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.881+0000 7f5161a70740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557b4e5af260 0x557b4e5c4010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:11.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.882+0000 7f5161a70740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5138011580 con 0x557b4e5a4b90 2024-04-24T09:12:11.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.882+0000 7f5161a70740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5138010eb0 con 0x557b4e5a4270 2024-04-24T09:12:11.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.882+0000 7f5161a70740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f513800e110 con 0x557b4e5af260 2024-04-24T09:12:11.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.882+0000 7f5157e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557b4e5a4270 0x557b4e5c8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:11.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.882+0000 7f5161a70740 10 client.0 _ll_register_callbacks cb 0x7f5144004050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:12:11.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.882+0000 7f5157e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557b4e5a4270 0x557b4e5c8740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:59454/0 (socket says 172.21.15.83:59454) 2024-04-24T09:12:11.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.882+0000 7f5157e00640 1 -- 172.21.15.83:0/2501278344 learned_addr learned my addr 172.21.15.83:0/2501278344 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:12:11.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.882+0000 7f5157400640 1 --2- 172.21.15.83:0/2501278344 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557b4e5a4b90 0x557b4e5c3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:11.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.882+0000 7f515cc00640 1 --2- 172.21.15.83:0/2501278344 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557b4e5af260 0x557b4e5c4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:11.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.883+0000 7f5157400640 1 -- 172.21.15.83:0/2501278344 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557b4e5af260 msgr2=0x557b4e5c4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:11.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.883+0000 7f5157400640 1 --2- 172.21.15.83:0/2501278344 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557b4e5af260 0x557b4e5c4010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:11.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.883+0000 7f5157400640 1 -- 172.21.15.83:0/2501278344 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557b4e5a4270 msgr2=0x557b4e5c8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:11.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.883+0000 7f5157400640 1 --2- 172.21.15.83:0/2501278344 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557b4e5a4270 0x557b4e5c8740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:11.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.883+0000 7f5157400640 1 -- 172.21.15.83:0/2501278344 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b4e5c4840 con 0x557b4e5a4b90 2024-04-24T09:12:11.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.883+0000 7f5157e00640 1 --2- 172.21.15.83:0/2501278344 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557b4e5a4270 0x557b4e5c8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:12:11.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.883+0000 7f5157400640 1 --2- 172.21.15.83:0/2501278344 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557b4e5a4b90 0x557b4e5c3ab0 secure :-1 s=READY pgs=2550 cs=0 l=1 rev1=1 crypto rx=0x557b4e5ad3d0 tx=0x7f514402fcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4611bcf454d2c900 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:11.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.883+0000 7f5141600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:12:11.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.884+0000 7f5141600640 1 -- 172.21.15.83:0/2501278344 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5144005250 con 0x557b4e5a4b90 2024-04-24T09:12:11.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.884+0000 7f5161a70740 1 -- 172.21.15.83:0/2501278344 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f514809c8d0 con 0x557b4e5a4b90 2024-04-24T09:12:11.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.884+0000 7f5141600640 1 -- 172.21.15.83:0/2501278344 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9583+0+0 (secure 0 0 0) 0x7f51440053f0 con 0x557b4e5a4b90 2024-04-24T09:12:11.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.884+0000 7f5141600640 1 -- 172.21.15.83:0/2501278344 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f51440055f0 con 0x557b4e5a4b90 2024-04-24T09:12:11.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.884+0000 7f5161a70740 10 client.22719 fetch_fsmap learned FSMap version 61 2024-04-24T09:12:11.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.884+0000 7f5161a70740 1 -- 172.21.15.83:0/2501278344 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x557b4e647ac0 con 0x557b4e5a4b90 2024-04-24T09:12:11.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.885+0000 7f5141600640 1 -- 172.21.15.83:0/2501278344 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f514809c8d0 con 0x557b4e5a4b90 2024-04-24T09:12:11.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.885+0000 7f5141600640 1 -- 172.21.15.83:0/2501278344 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 61) ==== 8362+0+0 (secure 0 0 0) 0x7f5144023040 con 0x557b4e5a4b90 2024-04-24T09:12:11.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.885+0000 7f5161a70740 10 client.22719 fetch_fsmap finished waiting for FSMap version 61 2024-04-24T09:12:11.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.885+0000 7f5161a70740 10 client.22719 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:12:11.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.885+0000 7f5161a70740 20 client.22719 populate_metadata read hostname 'smithi083' 2024-04-24T09:12:11.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.885+0000 7f5161a70740 1 --2- 172.21.15.83:0/2501278344 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x557b4e648230 0x557b4e64a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:11.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.885+0000 7f5161a70740 4 client.22719 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T09:12:11.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.885+0000 7f5161a70740 1 -- 172.21.15.83:0/2501278344 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x557b4e511700 con 0x557b4e648230 2024-04-24T09:12:11.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.885+0000 7f5157e00640 1 --2- 172.21.15.83:0/2501278344 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x557b4e648230 0x557b4e64a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:11.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.886+0000 7f5157e00640 1 --2- 172.21.15.83:0/2501278344 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x557b4e648230 0x557b4e64a620 crc :-1 s=READY pgs=512 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4dc00cb440051e61 server_cookie=dc4ce329067b1750 in_seq=0 out_seq=0 2024-04-24T09:12:11.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.886+0000 7f5141600640 10 client.22719 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:12:11.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.887+0000 7f5141600640 1 -- 172.21.15.83:0/2501278344 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x557b4e511700 con 0x557b4e648230 2024-04-24T09:12:11.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.887+0000 7f5141600640 10 client.22719 handle_command_reply: tid=0 2024-04-24T09:12:11.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.888+0000 7f5161a70740 1 client.22719 shutdown 2024-04-24T09:12:11.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.888+0000 7f5161a70740 1 -- 172.21.15.83:0/2501278344 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557b4e5a4b90 msgr2=0x557b4e5c3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:11.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.888+0000 7f5161a70740 1 --2- 172.21.15.83:0/2501278344 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557b4e5a4b90 0x557b4e5c3ab0 secure :-1 s=READY pgs=2550 cs=0 l=1 rev1=1 crypto rx=0x557b4e5ad3d0 tx=0x7f514402fcf0 comp rx=0 tx=0).stop 2024-04-24T09:12:11.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.888+0000 7f5161a70740 1 -- 172.21.15.83:0/2501278344 shutdown_connections 2024-04-24T09:12:11.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.888+0000 7f5161a70740 1 --2- 172.21.15.83:0/2501278344 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x557b4e648230 0x557b4e64a620 crc :-1 s=READY pgs=512 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:11.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.888+0000 7f5161a70740 1 --2- 172.21.15.83:0/2501278344 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557b4e5af260 0x557b4e5c4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:11.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.888+0000 7f5161a70740 1 --2- 172.21.15.83:0/2501278344 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557b4e5a4b90 0x557b4e5c3ab0 unknown :-1 s=CLOSED pgs=2550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:11.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.888+0000 7f5161a70740 1 --2- 172.21.15.83:0/2501278344 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557b4e5a4270 0x557b4e5c8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:11.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.888+0000 7f5141600640 0 client.22719 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:12:11.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.888+0000 7f5161a70740 1 -- 172.21.15.83:0/2501278344 >> 172.21.15.83:0/2501278344 conn(0x557b4e5a0590 msgr2=0x557b4e5a6250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:11.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.888+0000 7f5161a70740 1 -- 172.21.15.83:0/2501278344 shutdown_connections 2024-04-24T09:12:11.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.889+0000 7f5161a70740 1 -- 172.21.15.83:0/2501278344 wait complete. 2024-04-24T09:12:11.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.889+0000 7f5161a70740 20 client.22719 trim_cache size 0 max 16384 2024-04-24T09:12:11.889 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T09:12:11.889 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-04-24T09:12:11.889 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "4e0d793b-f743-4eb2-8497-0f9673df238d", 2024-04-24T09:12:11.889 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-04-24T09:12:11.889 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T09:12:11.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.889+0000 7f515d600640 1 -- 172.21.15.83:0/145434671 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f513c0a3da0 msgr2=0x7f513c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:11.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.889+0000 7f515d600640 1 --2- 172.21.15.83:0/145434671 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f513c0a3da0 0x7f513c0a6260 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x557b4e5a1720 tx=0x7f51440046c0 comp rx=0 tx=0).stop 2024-04-24T09:12:11.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.890+0000 7f515d600640 1 -- 172.21.15.83:0/145434671 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5158104320 msgr2=0x7f5158110fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:11.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.890+0000 7f515d600640 1 --2- 172.21.15.83:0/145434671 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5158104320 0x7f5158110fc0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f514800e9f0 tx=0x7f514800edf0 comp rx=0 tx=0).stop 2024-04-24T09:12:11.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.890+0000 7f515d600640 1 -- 172.21.15.83:0/145434671 shutdown_connections 2024-04-24T09:12:11.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.890+0000 7f515d600640 1 --2- 172.21.15.83:0/145434671 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f513c0a3da0 0x7f513c0a6260 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:11.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.890+0000 7f515d600640 1 --2- 172.21.15.83:0/145434671 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f515810e820 0x7f51581ab0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:11.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.890+0000 7f515d600640 1 --2- 172.21.15.83:0/145434671 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51581093f0 0x7f515810e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:11.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.890+0000 7f515d600640 1 --2- 172.21.15.83:0/145434671 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5158104320 0x7f5158110fc0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:11.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.890+0000 7f515d600640 1 -- 172.21.15.83:0/145434671 >> 172.21.15.83:0/145434671 conn(0x7f51580ffb40 msgr2=0x7f5158101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:11.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.890+0000 7f515d600640 1 -- 172.21.15.83:0/145434671 shutdown_connections 2024-04-24T09:12:11.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:11.890+0000 7f515d600640 1 -- 172.21.15.83:0/145434671 wait complete. 2024-04-24T09:12:11.911 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T09:12:11.911 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T09:12:12.184 INFO:tasks.workunit.client.0.smithi083.stdout: 10 50376 6.64 MB/sec execute 283 sec latency 109.421 ms 2024-04-24T09:12:12.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.200+0000 7f411d600640 1 -- 172.21.15.83:0/3651966964 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4118100e00 msgr2=0x7f41181011e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:12.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.200+0000 7f411d600640 1 --2- 172.21.15.83:0/3651966964 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4118100e00 0x7f41181011e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f4108009860 tx=0x7f410802a340 comp rx=0 tx=0).stop 2024-04-24T09:12:12.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.200+0000 7f411d600640 1 -- 172.21.15.83:0/3651966964 shutdown_connections 2024-04-24T09:12:12.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.200+0000 7f411d600640 1 --2- 172.21.15.83:0/3651966964 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f411810e9b0 0x7f4118110da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:12.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.200+0000 7f411d600640 1 --2- 172.21.15.83:0/3651966964 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4118101720 0x7f411810e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:12.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.200+0000 7f411d600640 1 --2- 172.21.15.83:0/3651966964 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4118100e00 0x7f41181011e0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:12.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.200+0000 7f411d600640 1 -- 172.21.15.83:0/3651966964 >> 172.21.15.83:0/3651966964 conn(0x7f4118078000 msgr2=0x7f4118078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:12.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.200+0000 7f411d600640 1 -- 172.21.15.83:0/3651966964 shutdown_connections 2024-04-24T09:12:12.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.200+0000 7f411d600640 1 -- 172.21.15.83:0/3651966964 wait complete. 2024-04-24T09:12:12.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.201+0000 7f411d600640 1 Processor -- start 2024-04-24T09:12:12.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.202+0000 7f411d600640 1 -- start start 2024-04-24T09:12:12.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.202+0000 7f411d600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4118100e00 0x7f411810e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:12.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.202+0000 7f411d600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4118101720 0x7f411810b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:12.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.202+0000 7f411d600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f411810e9b0 0x7f411810ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:12.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.202+0000 7f411d600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4118114470 con 0x7f4118100e00 2024-04-24T09:12:12.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.202+0000 7f411d600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f41181142f0 con 0x7f411810e9b0 2024-04-24T09:12:12.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.202+0000 7f411d600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f41181145f0 con 0x7f4118101720 2024-04-24T09:12:12.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.202+0000 7f4116a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4118100e00 0x7f411810e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:12.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.203+0000 7f4116a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4118100e00 0x7f411810e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50204/0 (socket says 172.21.15.83:50204) 2024-04-24T09:12:12.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.203+0000 7f4116a00640 1 -- 172.21.15.83:0/2983181551 learned_addr learned my addr 172.21.15.83:0/2983181551 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:12:12.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.203+0000 7f4117400640 1 --2- 172.21.15.83:0/2983181551 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f411810e9b0 0x7f411810ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:12.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.203+0000 7f4116a00640 1 -- 172.21.15.83:0/2983181551 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4118101720 msgr2=0x7f411810b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:12.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.203+0000 7f4116000640 1 --2- 172.21.15.83:0/2983181551 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4118101720 0x7f411810b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:12.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.203+0000 7f4116a00640 1 --2- 172.21.15.83:0/2983181551 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4118101720 0x7f411810b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:12.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.203+0000 7f4116a00640 1 -- 172.21.15.83:0/2983181551 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f411810e9b0 msgr2=0x7f411810ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:12.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.203+0000 7f4116a00640 1 --2- 172.21.15.83:0/2983181551 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f411810e9b0 0x7f411810ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:12.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.204+0000 7f4116a00640 1 -- 172.21.15.83:0/2983181551 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41181ab1b0 con 0x7f4118100e00 2024-04-24T09:12:12.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.204+0000 7f4116000640 1 --2- 172.21.15.83:0/2983181551 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4118101720 0x7f411810b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:12:12.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.204+0000 7f4117400640 1 --2- 172.21.15.83:0/2983181551 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f411810e9b0 0x7f411810ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:12:12.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.204+0000 7f4116a00640 1 --2- 172.21.15.83:0/2983181551 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4118100e00 0x7f411810e3d0 secure :-1 s=READY pgs=2551 cs=0 l=1 rev1=1 crypto rx=0x7f4108035040 tx=0x7f4108030bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:12.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.205+0000 7f410f400640 1 -- 172.21.15.83:0/2983181551 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4108040020 con 0x7f4118100e00 2024-04-24T09:12:12.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.205+0000 7f410f400640 1 -- 172.21.15.83:0/2983181551 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4108038040 con 0x7f4118100e00 2024-04-24T09:12:12.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.205+0000 7f410f400640 1 -- 172.21.15.83:0/2983181551 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4108037070 con 0x7f4118100e00 2024-04-24T09:12:12.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.205+0000 7f411d600640 1 -- 172.21.15.83:0/2983181551 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41181ab380 con 0x7f4118100e00 2024-04-24T09:12:12.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.205+0000 7f411d600640 1 -- 172.21.15.83:0/2983181551 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41181ab790 con 0x7f4118100e00 2024-04-24T09:12:12.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.210+0000 7f41203b5740 1 Processor -- start 2024-04-24T09:12:12.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.210+0000 7f41203b5740 1 -- start start 2024-04-24T09:12:12.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.210+0000 7f410f400640 1 -- 172.21.15.83:0/2983181551 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4108043020 con 0x7f4118100e00 2024-04-24T09:12:12.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.211+0000 7f41203b5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc721dd860 0x55dc721ddc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:12.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.211+0000 7f41203b5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc721de180 0x55dc721e8310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:12.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.211+0000 7f41203b5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dc721e8850 0x55dc721eac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:12.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.211+0000 7f41203b5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55dc721431f0 con 0x55dc721e8850 2024-04-24T09:12:12.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.211+0000 7f41203b5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55dc720e9350 con 0x55dc721de180 2024-04-24T09:12:12.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.211+0000 7f41203b5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55dc7213f1b0 con 0x55dc721dd860 2024-04-24T09:12:12.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.211+0000 7f4117400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc721dd860 0x55dc721ddc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:12.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.211+0000 7f4116a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc721de180 0x55dc721e8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:12.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.212+0000 7f4117400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc721dd860 0x55dc721ddc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:60014/0 (socket says 172.21.15.83:60014) 2024-04-24T09:12:12.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.212+0000 7f4117400640 1 -- 172.21.15.83:0/245398258 learned_addr learned my addr 172.21.15.83:0/245398258 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:12:12.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.212+0000 7f4116000640 1 --2- 172.21.15.83:0/245398258 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dc721e8850 0x55dc721eac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:12.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.212+0000 7f4116a00640 1 -- 172.21.15.83:0/245398258 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc721dd860 msgr2=0x55dc721ddc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:12.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.212+0000 7f4116a00640 1 --2- 172.21.15.83:0/245398258 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc721dd860 0x55dc721ddc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:12.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.212+0000 7f4116a00640 1 -- 172.21.15.83:0/245398258 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dc721e8850 msgr2=0x55dc721eac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:12.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.212+0000 7f4116a00640 1 --2- 172.21.15.83:0/245398258 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dc721e8850 0x55dc721eac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:12.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.212+0000 7f4116a00640 1 -- 172.21.15.83:0/245398258 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc7213df50 con 0x55dc721de180 2024-04-24T09:12:12.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.212+0000 7f410f400640 1 --2- 172.21.15.83:0/2983181551 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f40fc0a3da0 0x7f40fc0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:12.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.213+0000 7f410f400640 1 -- 172.21.15.83:0/2983181551 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 9583+0+0 (secure 0 0 0) 0x7f41080f0500 con 0x7f4118100e00 2024-04-24T09:12:12.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.213+0000 7f4116a00640 1 --2- 172.21.15.83:0/245398258 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc721de180 0x55dc721e8310 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f41080ed940 tx=0x7f41080f6620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee34af875110da25 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:12.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.213+0000 7f4117400640 1 --2- 172.21.15.83:0/2983181551 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f40fc0a3da0 0x7f40fc0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:12.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.213+0000 7f4103400640 1 -- 172.21.15.83:0/245398258 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41080b8430 con 0x55dc721de180 2024-04-24T09:12:12.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.213+0000 7f4103400640 1 -- 172.21.15.83:0/245398258 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f41080f6e20 con 0x55dc721de180 2024-04-24T09:12:12.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.213+0000 7f4103400640 1 -- 172.21.15.83:0/245398258 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41080bbaf0 con 0x55dc721de180 2024-04-24T09:12:12.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.214+0000 7f41203b5740 1 -- 172.21.15.83:0/245398258 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc721de180 msgr2=0x55dc721e8310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:12.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.214+0000 7f41203b5740 1 --2- 172.21.15.83:0/245398258 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc721de180 0x55dc721e8310 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f41080ed940 tx=0x7f41080f6620 comp rx=0 tx=0).stop 2024-04-24T09:12:12.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.214+0000 7f41203b5740 1 -- 172.21.15.83:0/245398258 shutdown_connections 2024-04-24T09:12:12.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.214+0000 7f41203b5740 1 --2- 172.21.15.83:0/245398258 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dc721e8850 0x55dc721eac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:12.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.214+0000 7f41203b5740 1 --2- 172.21.15.83:0/245398258 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc721de180 0x55dc721e8310 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:12.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.214+0000 7f41203b5740 1 --2- 172.21.15.83:0/245398258 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc721dd860 0x55dc721ddc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:12.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.214+0000 7f41203b5740 1 -- 172.21.15.83:0/245398258 >> 172.21.15.83:0/245398258 conn(0x55dc721d9b80 msgr2=0x55dc721d9f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:12.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.214+0000 7f41203b5740 1 -- 172.21.15.83:0/245398258 shutdown_connections 2024-04-24T09:12:12.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.214+0000 7f4117400640 1 --2- 172.21.15.83:0/2983181551 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f40fc0a3da0 0x7f40fc0a6260 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f41100099a0 tx=0x7f4110009500 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:12.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.214+0000 7f41203b5740 1 -- 172.21.15.83:0/245398258 wait complete. 2024-04-24T09:12:12.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.215+0000 7f41203b5740 1 -- start start 2024-04-24T09:12:12.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.215+0000 7f41203b5740 1 Processor -- start 2024-04-24T09:12:12.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.216+0000 7f41203b5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc721dd860 0x55dc721e56f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:12.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.216+0000 7f41203b5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dc721de180 0x55dc721e5c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:12.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.216+0000 7f41203b5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc721e8850 0x55dc721e6170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:12.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.216+0000 7f41203b5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55dc721eb400 con 0x55dc721de180 2024-04-24T09:12:12.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.216+0000 7f41203b5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55dc720fc690 con 0x55dc721e8850 2024-04-24T09:12:12.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.216+0000 7f41203b5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55dc721eb580 con 0x55dc721dd860 2024-04-24T09:12:12.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.216+0000 7f41203b5740 10 client.0 _ll_register_callbacks cb 0x55dc72138650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:12:12.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.217+0000 7f4116000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dc721de180 0x55dc721e5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:12.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.217+0000 7f4117400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc721e8850 0x55dc721e6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:12.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.217+0000 7f4116a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc721dd860 0x55dc721e56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:12.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.217+0000 7f4117400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc721e8850 0x55dc721e6170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:59494/0 (socket says 172.21.15.83:59494) 2024-04-24T09:12:12.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.217+0000 7f4117400640 1 -- 172.21.15.83:0/2151224546 learned_addr learned my addr 172.21.15.83:0/2151224546 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:12:12.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.217+0000 7f4117400640 1 -- 172.21.15.83:0/2151224546 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc721dd860 msgr2=0x55dc721e56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:12.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.217+0000 7f4117400640 1 --2- 172.21.15.83:0/2151224546 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc721dd860 0x55dc721e56f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:12.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.217+0000 7f4117400640 1 -- 172.21.15.83:0/2151224546 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dc721de180 msgr2=0x55dc721e5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:12.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.217+0000 7f4117400640 1 --2- 172.21.15.83:0/2151224546 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dc721de180 0x55dc721e5c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:12.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.217+0000 7f4117400640 1 -- 172.21.15.83:0/2151224546 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc720fb920 con 0x55dc721e8850 2024-04-24T09:12:12.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.218+0000 7f4116000640 1 --2- 172.21.15.83:0/2151224546 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dc721de180 0x55dc721e5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:12:12.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.218+0000 7f4116a00640 1 --2- 172.21.15.83:0/2151224546 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc721dd860 0x55dc721e56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:12:12.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.218+0000 7f4117400640 1 --2- 172.21.15.83:0/2151224546 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc721e8850 0x55dc721e6170 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f4110010b20 tx=0x7f4110004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3abbc9af12aed5b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:12.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.218+0000 7f40f7e00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:12:12.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.219+0000 7f40f7e00640 1 -- 172.21.15.83:0/2151224546 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f411002c060 con 0x55dc721e8850 2024-04-24T09:12:12.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.219+0000 7f41203b5740 1 -- 172.21.15.83:0/2151224546 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41080f6e20 con 0x55dc721e8850 2024-04-24T09:12:12.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.219+0000 7f40f7e00640 1 -- 172.21.15.83:0/2151224546 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9583+0+0 (secure 0 0 0) 0x7f4110017030 con 0x55dc721e8850 2024-04-24T09:12:12.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.219+0000 7f40f7e00640 1 -- 172.21.15.83:0/2151224546 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f411001fbc0 con 0x55dc721e8850 2024-04-24T09:12:12.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.219+0000 7f41203b5740 10 client.39779 fetch_fsmap learned FSMap version 61 2024-04-24T09:12:12.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.219+0000 7f41203b5740 1 -- 172.21.15.83:0/2151224546 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x55dc71f1f7d0 con 0x55dc721e8850 2024-04-24T09:12:12.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.220+0000 7f40f7e00640 1 -- 172.21.15.83:0/2151224546 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f41080f6e20 con 0x55dc721e8850 2024-04-24T09:12:12.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.220+0000 7f40f7e00640 1 -- 172.21.15.83:0/2151224546 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 61) ==== 8362+0+0 (secure 0 0 0) 0x7f41100043c0 con 0x55dc721e8850 2024-04-24T09:12:12.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.220+0000 7f41203b5740 10 client.39779 fetch_fsmap finished waiting for FSMap version 61 2024-04-24T09:12:12.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.220+0000 7f41203b5740 10 client.39779 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:12:12.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.220+0000 7f41203b5740 20 client.39779 populate_metadata read hostname 'smithi083' 2024-04-24T09:12:12.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.220+0000 7f41203b5740 1 --2- 172.21.15.83:0/2151224546 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55dc721f88c0 0x55dc721face0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:12.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.220+0000 7f41203b5740 4 client.39779 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:12:12.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.220+0000 7f41203b5740 1 -- 172.21.15.83:0/2151224546 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dc72150c10 con 0x55dc721f88c0 2024-04-24T09:12:12.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.221+0000 7f4116a00640 1 --2- 172.21.15.83:0/2151224546 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55dc721f88c0 0x55dc721face0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:12.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.222+0000 7f4116a00640 1 --2- 172.21.15.83:0/2151224546 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55dc721f88c0 0x55dc721face0 crc :-1 s=READY pgs=513 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a263df288c12eef8 server_cookie=fe9da94a4737e658 in_seq=0 out_seq=0 2024-04-24T09:12:12.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.222+0000 7f40f7e00640 10 client.39779 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:12:12.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.223+0000 7f40f7e00640 1 -- 172.21.15.83:0/2151224546 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55dc72150c10 con 0x55dc721f88c0 2024-04-24T09:12:12.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.223+0000 7f40f7e00640 10 client.39779 handle_command_reply: tid=0 2024-04-24T09:12:12.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.223+0000 7f41203b5740 1 client.39779 shutdown 2024-04-24T09:12:12.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.224+0000 7f41203b5740 1 -- 172.21.15.83:0/2151224546 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc721e8850 msgr2=0x55dc721e6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:12.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.224+0000 7f41203b5740 1 --2- 172.21.15.83:0/2151224546 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc721e8850 0x55dc721e6170 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f4110010b20 tx=0x7f4110004ab0 comp rx=0 tx=0).stop 2024-04-24T09:12:12.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.224+0000 7f41203b5740 1 -- 172.21.15.83:0/2151224546 shutdown_connections 2024-04-24T09:12:12.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.224+0000 7f41203b5740 1 --2- 172.21.15.83:0/2151224546 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55dc721f88c0 0x55dc721face0 crc :-1 s=READY pgs=513 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:12.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.224+0000 7f41203b5740 1 --2- 172.21.15.83:0/2151224546 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc721e8850 0x55dc721e6170 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:12.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.224+0000 7f41203b5740 1 --2- 172.21.15.83:0/2151224546 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dc721de180 0x55dc721e5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:12.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.224+0000 7f41203b5740 1 --2- 172.21.15.83:0/2151224546 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc721dd860 0x55dc721e56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:12.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.224+0000 7f40f7e00640 0 client.39779 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:12:12.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.224+0000 7f41203b5740 1 -- 172.21.15.83:0/2151224546 >> 172.21.15.83:0/2151224546 conn(0x55dc721d9b80 msgr2=0x55dc721df980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:12.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.224+0000 7f41203b5740 1 -- 172.21.15.83:0/2151224546 shutdown_connections 2024-04-24T09:12:12.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.224+0000 7f41203b5740 1 -- 172.21.15.83:0/2151224546 wait complete. 2024-04-24T09:12:12.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.224+0000 7f41203b5740 20 client.39779 trim_cache size 0 max 16384 2024-04-24T09:12:12.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.258+0000 7f41203b5740 1 Processor -- start 2024-04-24T09:12:12.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.258+0000 7f41203b5740 1 -- start start 2024-04-24T09:12:12.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.258+0000 7f41203b5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc721dd860 0x55dc721f4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:12.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.258+0000 7f41203b5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dc721de180 0x55dc721e5f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:12.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.258+0000 7f41203b5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc721e8850 0x55dc721d8c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:12.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.259+0000 7f41203b5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f41181b5cc0 con 0x55dc721de180 2024-04-24T09:12:12.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.259+0000 7f41203b5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f41181b7800 con 0x55dc721dd860 2024-04-24T09:12:12.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.259+0000 7f41203b5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f41181b6390 con 0x55dc721e8850 2024-04-24T09:12:12.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.259+0000 7f4116000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dc721de180 0x55dc721e5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:12.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.259+0000 7f4116000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dc721de180 0x55dc721e5f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50218/0 (socket says 172.21.15.83:50218) 2024-04-24T09:12:12.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.259+0000 7f4116a00640 1 --2- 172.21.15.83:0/1991529331 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc721dd860 0x55dc721f4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:12.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.259+0000 7f4116000640 1 -- 172.21.15.83:0/1991529331 learned_addr learned my addr 172.21.15.83:0/1991529331 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:12:12.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.259+0000 7f4117400640 1 --2- 172.21.15.83:0/1991529331 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc721e8850 0x55dc721d8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:12.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.259+0000 7f4116a00640 1 -- 172.21.15.83:0/1991529331 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc721e8850 msgr2=0x55dc721d8c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:12.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.260+0000 7f4116a00640 1 --2- 172.21.15.83:0/1991529331 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc721e8850 0x55dc721d8c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:12.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.260+0000 7f4116a00640 1 -- 172.21.15.83:0/1991529331 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dc721de180 msgr2=0x55dc721e5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:12.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.260+0000 7f4116a00640 1 --2- 172.21.15.83:0/1991529331 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dc721de180 0x55dc721e5f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:12.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.260+0000 7f4116a00640 1 -- 172.21.15.83:0/1991529331 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc71e553f0 con 0x55dc721dd860 2024-04-24T09:12:12.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.260+0000 7f4117400640 1 --2- 172.21.15.83:0/1991529331 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc721e8850 0x55dc721d8c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:12:12.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.260+0000 7f4116000640 1 --2- 172.21.15.83:0/1991529331 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dc721de180 0x55dc721e5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:12:12.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.260+0000 7f4116a00640 1 --2- 172.21.15.83:0/1991529331 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc721dd860 0x55dc721f4150 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f410802a930 tx=0x7f41080035f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=65fc2bf10562b785 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:12.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.260+0000 7f4102a00640 1 -- 172.21.15.83:0/1991529331 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41080bbcd0 con 0x55dc721dd860 2024-04-24T09:12:12.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.260+0000 7f4102a00640 1 -- 172.21.15.83:0/1991529331 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f41080f6e20 con 0x55dc721dd860 2024-04-24T09:12:12.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.260+0000 7f4102a00640 1 -- 172.21.15.83:0/1991529331 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41080bbf60 con 0x55dc721dd860 2024-04-24T09:12:12.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.261+0000 7f41203b5740 1 -- 172.21.15.83:0/1991529331 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc721dd860 msgr2=0x55dc721f4150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:12.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.261+0000 7f41203b5740 1 --2- 172.21.15.83:0/1991529331 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc721dd860 0x55dc721f4150 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f410802a930 tx=0x7f41080035f0 comp rx=0 tx=0).stop 2024-04-24T09:12:12.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.261+0000 7f41203b5740 1 -- 172.21.15.83:0/1991529331 shutdown_connections 2024-04-24T09:12:12.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.261+0000 7f41203b5740 1 --2- 172.21.15.83:0/1991529331 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc721e8850 0x55dc721d8c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:12.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.261+0000 7f41203b5740 1 --2- 172.21.15.83:0/1991529331 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dc721de180 0x55dc721e5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:12.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.261+0000 7f41203b5740 1 --2- 172.21.15.83:0/1991529331 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc721dd860 0x55dc721f4150 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:12.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.261+0000 7f41203b5740 1 -- 172.21.15.83:0/1991529331 >> 172.21.15.83:0/1991529331 conn(0x55dc721d9b80 msgr2=0x55dc721e4630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:12.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.262+0000 7f41203b5740 1 -- 172.21.15.83:0/1991529331 shutdown_connections 2024-04-24T09:12:12.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.262+0000 7f41203b5740 1 -- 172.21.15.83:0/1991529331 wait complete. 2024-04-24T09:12:12.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.262+0000 7f41203b5740 1 -- start start 2024-04-24T09:12:12.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.262+0000 7f41203b5740 1 Processor -- start 2024-04-24T09:12:12.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.263+0000 7f41203b5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc721dd860 0x55dc721f0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:12.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.263+0000 7f41203b5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc721de180 0x55dc721f0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:12.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.263+0000 7f41203b5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dc721e8850 0x55dc721e1f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:12.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.263+0000 7f41203b5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f41181b7130 con 0x55dc721e8850 2024-04-24T09:12:12.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.263+0000 7f41203b5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f41181b6a60 con 0x55dc721de180 2024-04-24T09:12:12.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.263+0000 7f41203b5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f41181b3cc0 con 0x55dc721dd860 2024-04-24T09:12:12.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.263+0000 7f41203b5740 10 client.0 _ll_register_callbacks cb 0x7f41100109e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:12:12.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.263+0000 7f4117400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dc721e8850 0x55dc721e1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:12.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.263+0000 7f4116000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc721de180 0x55dc721f0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:12.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.263+0000 7f4117400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dc721e8850 0x55dc721e1f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50220/0 (socket says 172.21.15.83:50220) 2024-04-24T09:12:12.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.263+0000 7f4117400640 1 -- 172.21.15.83:0/3316354732 learned_addr learned my addr 172.21.15.83:0/3316354732 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:12:12.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.264+0000 7f4116a00640 1 --2- 172.21.15.83:0/3316354732 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc721dd860 0x55dc721f0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:12.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.264+0000 7f4117400640 1 -- 172.21.15.83:0/3316354732 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc721dd860 msgr2=0x55dc721f0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:12.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.264+0000 7f4117400640 1 --2- 172.21.15.83:0/3316354732 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc721dd860 0x55dc721f0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:12.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.264+0000 7f4117400640 1 -- 172.21.15.83:0/3316354732 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc721de180 msgr2=0x55dc721f0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:12.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.264+0000 7f4117400640 1 --2- 172.21.15.83:0/3316354732 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc721de180 0x55dc721f0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:12.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.264+0000 7f4117400640 1 -- 172.21.15.83:0/3316354732 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc71e150f0 con 0x55dc721e8850 2024-04-24T09:12:12.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.265+0000 7f40f7e00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:12:12.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.265+0000 7f4117400640 1 --2- 172.21.15.83:0/3316354732 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dc721e8850 0x55dc721e1f20 secure :-1 s=READY pgs=2552 cs=0 l=1 rev1=1 crypto rx=0x7f4110006010 tx=0x7f411001c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25603b5d0c326985 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:12.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.265+0000 7f40f7e00640 1 -- 172.21.15.83:0/3316354732 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4110004d20 con 0x55dc721e8850 2024-04-24T09:12:12.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.266+0000 7f40f7e00640 1 -- 172.21.15.83:0/3316354732 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9583+0+0 (secure 0 0 0) 0x7f4110027b10 con 0x55dc721e8850 2024-04-24T09:12:12.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.266+0000 7f41203b5740 1 -- 172.21.15.83:0/3316354732 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41080f6e20 con 0x55dc721e8850 2024-04-24T09:12:12.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.266+0000 7f40f7e00640 1 -- 172.21.15.83:0/3316354732 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f411001f800 con 0x55dc721e8850 2024-04-24T09:12:12.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.266+0000 7f41203b5740 10 client.22737 fetch_fsmap learned FSMap version 61 2024-04-24T09:12:12.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.266+0000 7f41203b5740 1 -- 172.21.15.83:0/3316354732 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x55dc72108810 con 0x55dc721e8850 2024-04-24T09:12:12.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.267+0000 7f40f7e00640 1 -- 172.21.15.83:0/3316354732 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f41080f6e20 con 0x55dc721e8850 2024-04-24T09:12:12.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.267+0000 7f40f7e00640 1 -- 172.21.15.83:0/3316354732 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 61) ==== 8362+0+0 (secure 0 0 0) 0x7f4110005020 con 0x55dc721e8850 2024-04-24T09:12:12.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.269+0000 7f41203b5740 10 client.22737 fetch_fsmap finished waiting for FSMap version 61 2024-04-24T09:12:12.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.269+0000 7f41203b5740 10 client.22737 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:12:12.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.269+0000 7f41203b5740 20 client.22737 populate_metadata read hostname 'smithi083' 2024-04-24T09:12:12.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.269+0000 7f41203b5740 1 --2- 172.21.15.83:0/3316354732 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55dc7226fe00 0x55dc72272220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:12.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.269+0000 7f41203b5740 4 client.22737 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T09:12:12.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.269+0000 7f41203b5740 1 -- 172.21.15.83:0/3316354732 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55dc7214aa80 con 0x55dc7226fe00 2024-04-24T09:12:12.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.270+0000 7f4116a00640 1 --2- 172.21.15.83:0/3316354732 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55dc7226fe00 0x55dc72272220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:12.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.271+0000 7f40f7e00640 10 client.22737 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:12:12.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.271+0000 7f4116a00640 1 --2- 172.21.15.83:0/3316354732 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55dc7226fe00 0x55dc72272220 crc :-1 s=READY pgs=514 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e868d4965319f5e3 server_cookie=49f9e106c062b1db in_seq=0 out_seq=0 2024-04-24T09:12:12.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.272+0000 7f40f7e00640 1 -- 172.21.15.83:0/3316354732 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55dc7214aa80 con 0x55dc7226fe00 2024-04-24T09:12:12.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.272+0000 7f40f7e00640 10 client.22737 handle_command_reply: tid=0 2024-04-24T09:12:12.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.272+0000 7f41203b5740 1 client.22737 shutdown 2024-04-24T09:12:12.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.272+0000 7f41203b5740 1 -- 172.21.15.83:0/3316354732 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dc721e8850 msgr2=0x55dc721e1f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:12.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.272+0000 7f41203b5740 1 --2- 172.21.15.83:0/3316354732 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dc721e8850 0x55dc721e1f20 secure :-1 s=READY pgs=2552 cs=0 l=1 rev1=1 crypto rx=0x7f4110006010 tx=0x7f411001c6a0 comp rx=0 tx=0).stop 2024-04-24T09:12:12.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.273+0000 7f41203b5740 1 -- 172.21.15.83:0/3316354732 shutdown_connections 2024-04-24T09:12:12.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.273+0000 7f41203b5740 1 --2- 172.21.15.83:0/3316354732 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55dc7226fe00 0x55dc72272220 crc :-1 s=READY pgs=514 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:12.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.273+0000 7f41203b5740 1 --2- 172.21.15.83:0/3316354732 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dc721e8850 0x55dc721e1f20 unknown :-1 s=CLOSED pgs=2552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:12.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.273+0000 7f41203b5740 1 --2- 172.21.15.83:0/3316354732 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc721de180 0x55dc721f0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:12.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.273+0000 7f41203b5740 1 --2- 172.21.15.83:0/3316354732 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc721dd860 0x55dc721f0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:12.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.273+0000 7f41203b5740 1 -- 172.21.15.83:0/3316354732 >> 172.21.15.83:0/3316354732 conn(0x55dc721d9b80 msgr2=0x55dc721d8c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:12.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.273+0000 7f40f7e00640 0 client.22737 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:12:12.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.273+0000 7f41203b5740 1 -- 172.21.15.83:0/3316354732 shutdown_connections 2024-04-24T09:12:12.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.273+0000 7f41203b5740 1 -- 172.21.15.83:0/3316354732 wait complete. 2024-04-24T09:12:12.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.274+0000 7f41203b5740 20 client.22737 trim_cache size 0 max 16384 2024-04-24T09:12:12.278 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T09:12:12.278 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T09:12:12.278 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-04-24T09:12:12.278 INFO:teuthology.orchestra.run.smithi083.stdout: "4e0d793b-f743-4eb2-8497-0f9673df238d": { 2024-04-24T09:12:12.278 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-04-24T09:12:12.278 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "4e0d793b-f743-4eb2-8497-0f9673df238d", 2024-04-24T09:12:12.279 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-04-24T09:12:12.279 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:12:12.279 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:12:12.279 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T09:12:12.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.275+0000 7f411d600640 1 -- 172.21.15.83:0/2983181551 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f40fc0a3da0 msgr2=0x7f40fc0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:12.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.275+0000 7f411d600640 1 --2- 172.21.15.83:0/2983181551 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f40fc0a3da0 0x7f40fc0a6260 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f41100099a0 tx=0x7f4110009500 comp rx=0 tx=0).stop 2024-04-24T09:12:12.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.275+0000 7f411d600640 1 -- 172.21.15.83:0/2983181551 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4118100e00 msgr2=0x7f411810e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:12.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.275+0000 7f411d600640 1 --2- 172.21.15.83:0/2983181551 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4118100e00 0x7f411810e3d0 secure :-1 s=READY pgs=2551 cs=0 l=1 rev1=1 crypto rx=0x7f4108035040 tx=0x7f4108030bd0 comp rx=0 tx=0).stop 2024-04-24T09:12:12.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.275+0000 7f411d600640 1 -- 172.21.15.83:0/2983181551 shutdown_connections 2024-04-24T09:12:12.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.275+0000 7f411d600640 1 --2- 172.21.15.83:0/2983181551 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f40fc0a3da0 0x7f40fc0a6260 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:12.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.275+0000 7f411d600640 1 --2- 172.21.15.83:0/2983181551 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f411810e9b0 0x7f411810ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:12.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.275+0000 7f411d600640 1 --2- 172.21.15.83:0/2983181551 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4118101720 0x7f411810b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:12.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.275+0000 7f411d600640 1 --2- 172.21.15.83:0/2983181551 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4118100e00 0x7f411810e3d0 unknown :-1 s=CLOSED pgs=2551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:12.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.275+0000 7f411d600640 1 -- 172.21.15.83:0/2983181551 >> 172.21.15.83:0/2983181551 conn(0x7f4118078000 msgr2=0x7f4118110140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:12.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.275+0000 7f411d600640 1 -- 172.21.15.83:0/2983181551 shutdown_connections 2024-04-24T09:12:12.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:12.275+0000 7f411d600640 1 -- 172.21.15.83:0/2983181551 wait complete. 2024-04-24T09:12:12.296 INFO:tasks.cephfs.filesystem:scrub status for tag:4e0d793b-f743-4eb2-8497-0f9673df238d - {'path': '/', 'tag': '4e0d793b-f743-4eb2-8497-0f9673df238d', 'options': 'recursive,force'} 2024-04-24T09:12:12.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:12:12 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:12:12.704 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:12:12 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:12:12.706 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:12:12 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:12:13.185 INFO:tasks.workunit.client.0.smithi083.stdout: 10 50517 6.64 MB/sec execute 284 sec latency 102.526 ms 2024-04-24T09:12:13.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:12:13 smithi138 ceph-mon[31441]: scrub queued for path: / 2024-04-24T09:12:13.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:12:13 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:12:13.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:12:13 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:12:13.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:12:13 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:12:13.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:12:13 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:12:13.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:12:13 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:12:13.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:12:13 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:12:13.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:12:13 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:12:13.706 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:12:13 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:12:13.706 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:12:13 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:12:13.706 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:12:13 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:12:13.706 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:12:13 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:12:13.706 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:12:13 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:12:13.706 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:12:13 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:12:13.706 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:12:13 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:12:13.706 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:12:13 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:12:13.706 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:12:13 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:12:13.706 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:12:13 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:12:13.706 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:12:13 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:12:13.706 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:12:13 smithi138 ceph-mon[31441]: pgmap v651: 129 pgs: 129 active+clean; 804 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.8 MiB/s wr, 270 op/s 2024-04-24T09:12:13.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:12:13 smithi189 ceph-mon[31791]: scrub queued for path: / 2024-04-24T09:12:13.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:12:13 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:12:13.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:12:13 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:12:13.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:12:13 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:12:13.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:12:13 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:12:13.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:12:13 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:12:13.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:12:13 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:12:13.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:12:13 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:12:13.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:12:13 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:12:13.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:12:13 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:12:13.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:12:13 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:12:13.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:12:13 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:12:13.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:12:13 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:12:13.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:12:13 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:12:13.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:12:13 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:12:13.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:12:13 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:12:13.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:12:13 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:12:13.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:12:13 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:12:13.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:12:13 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:12:13.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:12:13 smithi189 ceph-mon[31791]: pgmap v651: 129 pgs: 129 active+clean; 804 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.8 MiB/s wr, 270 op/s 2024-04-24T09:12:13.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:12:13 smithi083 ceph-mon[27907]: scrub queued for path: / 2024-04-24T09:12:13.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:12:13 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:12:13.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:12:13 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:12:13.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:12:13 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:12:13.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:12:13 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:12:13.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:12:13 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:12:13.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:12:13 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:12:13.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:12:13 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:12:13.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:12:13 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:12:13.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:12:13 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:12:13.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:12:13 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:12:13.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:12:13 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:12:13.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:12:13 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:12:13.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:12:13 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:12:13.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:12:13 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:12:13.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:12:13 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:12:13.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:12:13 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:12:13.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:12:13 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:12:13.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:12:13 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:12:13.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:12:13 smithi083 ceph-mon[27907]: pgmap v651: 129 pgs: 129 active+clean; 804 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.8 MiB/s wr, 270 op/s 2024-04-24T09:12:14.184 INFO:tasks.workunit.client.0.smithi083.stdout: 10 50633 6.63 MB/sec execute 285 sec latency 83.523 ms 2024-04-24T09:12:14.242 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.242+0000 7f3aa3c00640 1 -- 172.21.15.83:0/3752452525 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a9c109520 msgr2=0x7f3a9c1110f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:14.243 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.242+0000 7f3aa3c00640 1 --2- 172.21.15.83:0/3752452525 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a9c109520 0x7f3a9c1110f0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f3a94009ca0 tx=0x7f3a9402a8c0 comp rx=0 tx=0).stop 2024-04-24T09:12:14.243 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.242+0000 7f3aa3c00640 1 -- 172.21.15.83:0/3752452525 shutdown_connections 2024-04-24T09:12:14.243 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.242+0000 7f3aa3c00640 1 --2- 172.21.15.83:0/3752452525 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a9c109520 0x7f3a9c1110f0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:14.243 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.242+0000 7f3aa3c00640 1 --2- 172.21.15.83:0/3752452525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a9c104e00 0x7f3a9c108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:14.243 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.242+0000 7f3aa3c00640 1 --2- 172.21.15.83:0/3752452525 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3a9c104450 0x7f3a9c104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:14.244 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.242+0000 7f3aa3c00640 1 -- 172.21.15.83:0/3752452525 >> 172.21.15.83:0/3752452525 conn(0x7f3a9c0ffcf0 msgr2=0x7f3a9c102110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:14.244 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.242+0000 7f3aa3c00640 1 -- 172.21.15.83:0/3752452525 shutdown_connections 2024-04-24T09:12:14.244 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.242+0000 7f3aa3c00640 1 -- 172.21.15.83:0/3752452525 wait complete. 2024-04-24T09:12:14.247 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.244+0000 7f3aa3c00640 1 Processor -- start 2024-04-24T09:12:14.247 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.244+0000 7f3aa3c00640 1 -- start start 2024-04-24T09:12:14.247 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.244+0000 7f3aa3c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a9c104450 0x7f3a9c110fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:14.247 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.244+0000 7f3aa3c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3a9c109520 0x7f3a9c10e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:14.248 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.244+0000 7f3aa3c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a9c10e690 0x7f3a9c10ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:14.248 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.244+0000 7f3aa3c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3a9c1144e0 con 0x7f3a9c10e690 2024-04-24T09:12:14.248 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.244+0000 7f3aa3c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3a9c114360 con 0x7f3a9c104450 2024-04-24T09:12:14.248 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.244+0000 7f3aa3c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3a9c114660 con 0x7f3a9c109520 2024-04-24T09:12:14.248 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.245+0000 7f3aa1c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a9c10e690 0x7f3a9c10ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:14.248 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.245+0000 7f3aa1c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a9c10e690 0x7f3a9c10ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50234/0 (socket says 172.21.15.83:50234) 2024-04-24T09:12:14.248 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.245+0000 7f3aa1c00640 1 -- 172.21.15.83:0/2328958530 learned_addr learned my addr 172.21.15.83:0/2328958530 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:12:14.249 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.245+0000 7f3aa1200640 1 --2- 172.21.15.83:0/2328958530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a9c104450 0x7f3a9c110fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:14.249 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.245+0000 7f3a9be00640 1 --2- 172.21.15.83:0/2328958530 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3a9c109520 0x7f3a9c10e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:14.249 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.245+0000 7f3aa1c00640 1 -- 172.21.15.83:0/2328958530 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3a9c109520 msgr2=0x7f3a9c10e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:14.249 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.245+0000 7f3aa1c00640 1 --2- 172.21.15.83:0/2328958530 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3a9c109520 0x7f3a9c10e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:14.249 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.245+0000 7f3aa1c00640 1 -- 172.21.15.83:0/2328958530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a9c104450 msgr2=0x7f3a9c110fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:14.249 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.245+0000 7f3aa1c00640 1 --2- 172.21.15.83:0/2328958530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a9c104450 0x7f3a9c110fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:14.249 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.245+0000 7f3aa1c00640 1 -- 172.21.15.83:0/2328958530 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a9c1ab460 con 0x7f3a9c10e690 2024-04-24T09:12:14.250 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.245+0000 7f3aa1c00640 1 --2- 172.21.15.83:0/2328958530 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a9c10e690 0x7f3a9c10ead0 secure :-1 s=READY pgs=2553 cs=0 l=1 rev1=1 crypto rx=0x7f3a94005e30 tx=0x7f3a940093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:14.250 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.246+0000 7f3a99600640 1 -- 172.21.15.83:0/2328958530 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a94003c70 con 0x7f3a9c10e690 2024-04-24T09:12:14.250 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.246+0000 7f3a99600640 1 -- 172.21.15.83:0/2328958530 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3a94003e10 con 0x7f3a9c10e690 2024-04-24T09:12:14.250 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.246+0000 7f3aa3c00640 1 -- 172.21.15.83:0/2328958530 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a9c1ab6f0 con 0x7f3a9c10e690 2024-04-24T09:12:14.250 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.246+0000 7f3a99600640 1 -- 172.21.15.83:0/2328958530 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a94030c50 con 0x7f3a9c10e690 2024-04-24T09:12:14.250 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.246+0000 7f3aa3c00640 1 -- 172.21.15.83:0/2328958530 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a9c1abc90 con 0x7f3a9c10e690 2024-04-24T09:12:14.251 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.250+0000 7f3aa6b95740 1 Processor -- start 2024-04-24T09:12:14.251 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.251+0000 7f3a99600640 1 -- 172.21.15.83:0/2328958530 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3a94004040 con 0x7f3a9c10e690 2024-04-24T09:12:14.251 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.252+0000 7f3aa6b95740 1 -- start start 2024-04-24T09:12:14.251 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.252+0000 7f3aa6b95740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602ab12c1b0 0x5602ab12c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:14.251 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.252+0000 7f3aa6b95740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5602ab12cad0 0x5602ab136e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:14.252 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.252+0000 7f3aa6b95740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602ab137390 0x5602ab139780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:14.252 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.252+0000 7f3aa6b95740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5602ab092900 con 0x5602ab12c1b0 2024-04-24T09:12:14.252 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.252+0000 7f3aa6b95740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5602ab038f20 con 0x5602ab12cad0 2024-04-24T09:12:14.252 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.252+0000 7f3aa6b95740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5602ab08e8c0 con 0x5602ab137390 2024-04-24T09:12:14.253 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.253+0000 7f3a99600640 1 --2- 172.21.15.83:0/2328958530 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f3a840a3ec0 0x7f3a840a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:14.253 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.253+0000 7f3a99600640 1 -- 172.21.15.83:0/2328958530 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 9583+0+0 (secure 0 0 0) 0x7f3a940f2150 con 0x7f3a9c10e690 2024-04-24T09:12:14.255 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.254+0000 7f3aa1200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5602ab12cad0 0x5602ab136e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:14.255 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.254+0000 7f3aa1200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5602ab12cad0 0x5602ab136e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:59508/0 (socket says 172.21.15.83:59508) 2024-04-24T09:12:14.255 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.254+0000 7f3aa1200640 1 -- 172.21.15.83:0/3699658310 learned_addr learned my addr 172.21.15.83:0/3699658310 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:12:14.255 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.254+0000 7f3a9be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602ab12c1b0 0x5602ab12c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:14.255 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.254+0000 7f3aa1c00640 1 --2- 172.21.15.83:0/2328958530 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f3a840a3ec0 0x7f3a840a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:14.255 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.255+0000 7f3a9be00640 1 -- 172.21.15.83:0/3699658310 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602ab137390 msgr2=0x5602ab139780 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-24T09:12:14.255 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.255+0000 7f3a9be00640 1 --2- 172.21.15.83:0/3699658310 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602ab137390 0x5602ab139780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:14.255 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.255+0000 7f3a9be00640 1 -- 172.21.15.83:0/3699658310 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5602ab12cad0 msgr2=0x5602ab136e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:14.256 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.255+0000 7f3a9be00640 1 --2- 172.21.15.83:0/3699658310 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5602ab12cad0 0x5602ab136e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:14.256 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.255+0000 7f3a9be00640 1 -- 172.21.15.83:0/3699658310 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602ab05de10 con 0x5602ab12c1b0 2024-04-24T09:12:14.256 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.255+0000 7f3aa1c00640 1 --2- 172.21.15.83:0/2328958530 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f3a840a3ec0 0x7f3a840a6380 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f3a940ba1e0 tx=0x7f3a940306d0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:14.256 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.256+0000 7f3a9be00640 1 --2- 172.21.15.83:0/3699658310 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602ab12c1b0 0x5602ab12c590 secure :-1 s=READY pgs=2554 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c100220 tx=0x7f3a8c00bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42850616cc9290d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:14.256 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.256+0000 7f3a8a000640 1 -- 172.21.15.83:0/3699658310 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a8c015070 con 0x5602ab12c1b0 2024-04-24T09:12:14.256 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.256+0000 7f3a8a000640 1 -- 172.21.15.83:0/3699658310 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3a8c00fcf0 con 0x5602ab12c1b0 2024-04-24T09:12:14.257 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.256+0000 7f3a8a000640 1 -- 172.21.15.83:0/3699658310 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a8c0117b0 con 0x5602ab12c1b0 2024-04-24T09:12:14.257 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.257+0000 7f3aa6b95740 1 -- 172.21.15.83:0/3699658310 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602ab12c1b0 msgr2=0x5602ab12c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:14.257 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.257+0000 7f3aa6b95740 1 --2- 172.21.15.83:0/3699658310 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602ab12c1b0 0x5602ab12c590 secure :-1 s=READY pgs=2554 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c100220 tx=0x7f3a8c00bf90 comp rx=0 tx=0).stop 2024-04-24T09:12:14.257 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.258+0000 7f3aa6b95740 1 -- 172.21.15.83:0/3699658310 shutdown_connections 2024-04-24T09:12:14.257 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.258+0000 7f3aa6b95740 1 --2- 172.21.15.83:0/3699658310 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602ab137390 0x5602ab139780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:14.258 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.258+0000 7f3aa6b95740 1 --2- 172.21.15.83:0/3699658310 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5602ab12cad0 0x5602ab136e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:14.258 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.258+0000 7f3aa6b95740 1 --2- 172.21.15.83:0/3699658310 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602ab12c1b0 0x5602ab12c590 unknown :-1 s=CLOSED pgs=2554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:14.258 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.258+0000 7f3aa6b95740 1 -- 172.21.15.83:0/3699658310 >> 172.21.15.83:0/3699658310 conn(0x5602ab1286c0 msgr2=0x5602ab128aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:14.258 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.258+0000 7f3aa6b95740 1 -- 172.21.15.83:0/3699658310 shutdown_connections 2024-04-24T09:12:14.258 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.258+0000 7f3aa6b95740 1 -- 172.21.15.83:0/3699658310 wait complete. 2024-04-24T09:12:14.258 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.258+0000 7f3aa6b95740 1 -- start start 2024-04-24T09:12:14.258 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.259+0000 7f3aa6b95740 1 Processor -- start 2024-04-24T09:12:14.259 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.259+0000 7f3aa6b95740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5602ab12c1b0 0x5602ab1343a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:14.259 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.259+0000 7f3aa6b95740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602ab12cad0 0x5602ab1348e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:14.259 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.259+0000 7f3aa6b95740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602ab137390 0x5602ab134e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:14.259 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.259+0000 7f3aa6b95740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5602ab139fc0 con 0x5602ab12cad0 2024-04-24T09:12:14.259 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.259+0000 7f3aa6b95740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5602ab04c2c0 con 0x5602ab12c1b0 2024-04-24T09:12:14.259 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.259+0000 7f3aa6b95740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5602ab13a140 con 0x5602ab137390 2024-04-24T09:12:14.259 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.259+0000 7f3aa6b95740 10 client.0 _ll_register_callbacks cb 0x5602aada4a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:12:14.260 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.260+0000 7f3aa1200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602ab12cad0 0x5602ab1348e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:14.260 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.260+0000 7f3a9be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5602ab12c1b0 0x5602ab1343a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:14.260 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.260+0000 7f3a9be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5602ab12c1b0 0x5602ab1343a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:59518/0 (socket says 172.21.15.83:59518) 2024-04-24T09:12:14.260 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.260+0000 7f3a9be00640 1 -- 172.21.15.83:0/2850215774 learned_addr learned my addr 172.21.15.83:0/2850215774 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:12:14.260 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.260+0000 7f3a9be00640 1 --2- 172.21.15.83:0/2850215774 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602ab137390 0x5602ab134e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:14.260 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.260+0000 7f3aa1200640 1 -- 172.21.15.83:0/2850215774 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602ab137390 msgr2=0x5602ab134e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:14.260 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.260+0000 7f3aa1200640 1 --2- 172.21.15.83:0/2850215774 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602ab137390 0x5602ab134e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:14.260 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.260+0000 7f3aa1200640 1 -- 172.21.15.83:0/2850215774 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5602ab12c1b0 msgr2=0x5602ab1343a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:14.260 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.260+0000 7f3aa1200640 1 --2- 172.21.15.83:0/2850215774 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5602ab12c1b0 0x5602ab1343a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:14.260 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.260+0000 7f3aa1200640 1 -- 172.21.15.83:0/2850215774 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602ab05cfd0 con 0x5602ab12cad0 2024-04-24T09:12:14.260 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.260+0000 7f3a9be00640 1 --2- 172.21.15.83:0/2850215774 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5602ab12c1b0 0x5602ab1343a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:12:14.261 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.261+0000 7f3aa1200640 1 --2- 172.21.15.83:0/2850215774 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602ab12cad0 0x5602ab1348e0 secure :-1 s=READY pgs=2555 cs=0 l=1 rev1=1 crypto rx=0x7f3a900126d0 tx=0x7f3a90012b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb652746ee5a8387 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:14.261 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.261+0000 7f3a7ea00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:12:14.261 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.261+0000 7f3a7ea00640 1 -- 172.21.15.83:0/2850215774 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a90004260 con 0x5602ab12cad0 2024-04-24T09:12:14.261 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.261+0000 7f3a7ea00640 1 -- 172.21.15.83:0/2850215774 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9583+0+0 (secure 0 0 0) 0x7f3a90004400 con 0x5602ab12cad0 2024-04-24T09:12:14.261 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.261+0000 7f3aa6b95740 1 -- 172.21.15.83:0/2850215774 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a8c00fcf0 con 0x5602ab12cad0 2024-04-24T09:12:14.262 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.262+0000 7f3a7ea00640 1 -- 172.21.15.83:0/2850215774 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f3a900119a0 con 0x5602ab12cad0 2024-04-24T09:12:14.262 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.262+0000 7f3aa6b95740 10 client.22749 fetch_fsmap learned FSMap version 61 2024-04-24T09:12:14.262 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.262+0000 7f3aa6b95740 1 -- 172.21.15.83:0/2850215774 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x5602aada5210 con 0x5602ab12cad0 2024-04-24T09:12:14.263 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.262+0000 7f3a7ea00640 1 -- 172.21.15.83:0/2850215774 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3a8c00fcf0 con 0x5602ab12cad0 2024-04-24T09:12:14.263 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.263+0000 7f3a7ea00640 1 -- 172.21.15.83:0/2850215774 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 61) ==== 8362+0+0 (secure 0 0 0) 0x7f3a90026420 con 0x5602ab12cad0 2024-04-24T09:12:14.263 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.263+0000 7f3aa6b95740 10 client.22749 fetch_fsmap finished waiting for FSMap version 61 2024-04-24T09:12:14.263 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.263+0000 7f3aa6b95740 10 client.22749 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:12:14.263 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.263+0000 7f3aa6b95740 20 client.22749 populate_metadata read hostname 'smithi083' 2024-04-24T09:12:14.263 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.263+0000 7f3aa6b95740 1 --2- 172.21.15.83:0/2850215774 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5602ab1475c0 0x5602ab1499b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:14.263 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.263+0000 7f3aa6b95740 4 client.22749 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:12:14.263 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.263+0000 7f3aa6b95740 1 -- 172.21.15.83:0/2850215774 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5602ab0a0320 con 0x5602ab1475c0 2024-04-24T09:12:14.263 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.264+0000 7f3a9be00640 1 --2- 172.21.15.83:0/2850215774 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5602ab1475c0 0x5602ab1499b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:14.264 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.265+0000 7f3a9be00640 1 --2- 172.21.15.83:0/2850215774 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5602ab1475c0 0x5602ab1499b0 crc :-1 s=READY pgs=515 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=660591b2572daff server_cookie=dcfa8619130d4308 in_seq=0 out_seq=0 2024-04-24T09:12:14.264 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.265+0000 7f3a7ea00640 10 client.22749 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:12:14.265 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.266+0000 7f3a7ea00640 1 -- 172.21.15.83:0/2850215774 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5602ab0a0320 con 0x5602ab1475c0 2024-04-24T09:12:14.265 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.266+0000 7f3a7ea00640 10 client.22749 handle_command_reply: tid=0 2024-04-24T09:12:14.266 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.266+0000 7f3aa6b95740 1 client.22749 shutdown 2024-04-24T09:12:14.266 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.266+0000 7f3aa6b95740 1 -- 172.21.15.83:0/2850215774 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602ab12cad0 msgr2=0x5602ab1348e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:14.266 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.266+0000 7f3aa6b95740 1 --2- 172.21.15.83:0/2850215774 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602ab12cad0 0x5602ab1348e0 secure :-1 s=READY pgs=2555 cs=0 l=1 rev1=1 crypto rx=0x7f3a900126d0 tx=0x7f3a90012b90 comp rx=0 tx=0).stop 2024-04-24T09:12:14.266 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.267+0000 7f3aa6b95740 1 -- 172.21.15.83:0/2850215774 shutdown_connections 2024-04-24T09:12:14.266 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.267+0000 7f3aa6b95740 1 --2- 172.21.15.83:0/2850215774 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5602ab1475c0 0x5602ab1499b0 crc :-1 s=READY pgs=515 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:14.267 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.267+0000 7f3aa6b95740 1 --2- 172.21.15.83:0/2850215774 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602ab137390 0x5602ab134e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:14.267 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.267+0000 7f3aa6b95740 1 --2- 172.21.15.83:0/2850215774 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602ab12cad0 0x5602ab1348e0 unknown :-1 s=CLOSED pgs=2555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:14.267 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.267+0000 7f3aa6b95740 1 --2- 172.21.15.83:0/2850215774 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5602ab12c1b0 0x5602ab1343a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:14.267 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.267+0000 7f3a7ea00640 0 client.22749 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:12:14.267 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.267+0000 7f3aa6b95740 1 -- 172.21.15.83:0/2850215774 >> 172.21.15.83:0/2850215774 conn(0x5602ab1286c0 msgr2=0x5602ab12e440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:14.267 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.267+0000 7f3aa6b95740 1 -- 172.21.15.83:0/2850215774 shutdown_connections 2024-04-24T09:12:14.267 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.267+0000 7f3aa6b95740 1 -- 172.21.15.83:0/2850215774 wait complete. 2024-04-24T09:12:14.267 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.267+0000 7f3aa6b95740 20 client.22749 trim_cache size 0 max 16384 2024-04-24T09:12:14.291 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.292+0000 7f3aa6b95740 1 Processor -- start 2024-04-24T09:12:14.292 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.292+0000 7f3aa6b95740 1 -- start start 2024-04-24T09:12:14.292 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.292+0000 7f3aa6b95740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5602ab12c1b0 0x5602ab14f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:14.292 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.292+0000 7f3aa6b95740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602ab12cad0 0x5602ab14f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:14.292 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.292+0000 7f3aa6b95740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602ab137390 0x5602ab131660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:14.292 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.292+0000 7f3aa6b95740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3a9c1b5cf0 con 0x5602ab12cad0 2024-04-24T09:12:14.292 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.292+0000 7f3aa6b95740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3a9c1b7830 con 0x5602ab12c1b0 2024-04-24T09:12:14.292 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.292+0000 7f3aa6b95740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3a9c1b63c0 con 0x5602ab137390 2024-04-24T09:12:14.293 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.293+0000 7f3aa1200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602ab12cad0 0x5602ab14f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:14.293 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.293+0000 7f3aa1200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602ab12cad0 0x5602ab14f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50264/0 (socket says 172.21.15.83:50264) 2024-04-24T09:12:14.293 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.293+0000 7f3aa1200640 1 -- 172.21.15.83:0/3565169675 learned_addr learned my addr 172.21.15.83:0/3565169675 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:12:14.293 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.293+0000 7f3a9be00640 1 --2- 172.21.15.83:0/3565169675 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5602ab12c1b0 0x5602ab14f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:14.294 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.293+0000 7f3a9be00640 1 --2- 172.21.15.83:0/3565169675 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602ab137390 0x5602ab131660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:14.294 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.293+0000 7f3aa1200640 1 -- 172.21.15.83:0/3565169675 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602ab137390 msgr2=0x5602ab131660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:14.294 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.293+0000 7f3aa1200640 1 --2- 172.21.15.83:0/3565169675 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602ab137390 0x5602ab131660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:14.294 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.293+0000 7f3aa1200640 1 -- 172.21.15.83:0/3565169675 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5602ab12c1b0 msgr2=0x5602ab14f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:14.294 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.293+0000 7f3aa1200640 1 --2- 172.21.15.83:0/3565169675 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5602ab12c1b0 0x5602ab14f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:14.294 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.293+0000 7f3aa1200640 1 -- 172.21.15.83:0/3565169675 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602aae71620 con 0x5602ab12cad0 2024-04-24T09:12:14.294 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.294+0000 7f3aa1200640 1 --2- 172.21.15.83:0/3565169675 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602ab12cad0 0x5602ab14f770 secure :-1 s=READY pgs=2556 cs=0 l=1 rev1=1 crypto rx=0x7f3a900269e0 tx=0x7f3a90009600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b546fc97b9dc09d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:14.294 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.294+0000 7f3a89600640 1 -- 172.21.15.83:0/3565169675 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a9002b040 con 0x5602ab12cad0 2024-04-24T09:12:14.294 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.294+0000 7f3a89600640 1 -- 172.21.15.83:0/3565169675 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3a900264e0 con 0x5602ab12cad0 2024-04-24T09:12:14.294 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.294+0000 7f3a89600640 1 -- 172.21.15.83:0/3565169675 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a90003850 con 0x5602ab12cad0 2024-04-24T09:12:14.295 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.295+0000 7f3aa6b95740 1 -- 172.21.15.83:0/3565169675 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602ab12cad0 msgr2=0x5602ab14f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:14.295 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.295+0000 7f3aa6b95740 1 --2- 172.21.15.83:0/3565169675 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602ab12cad0 0x5602ab14f770 secure :-1 s=READY pgs=2556 cs=0 l=1 rev1=1 crypto rx=0x7f3a900269e0 tx=0x7f3a90009600 comp rx=0 tx=0).stop 2024-04-24T09:12:14.296 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.295+0000 7f3aa6b95740 1 -- 172.21.15.83:0/3565169675 shutdown_connections 2024-04-24T09:12:14.296 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.295+0000 7f3aa6b95740 1 --2- 172.21.15.83:0/3565169675 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602ab137390 0x5602ab131660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:14.296 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.295+0000 7f3aa6b95740 1 --2- 172.21.15.83:0/3565169675 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602ab12cad0 0x5602ab14f770 unknown :-1 s=CLOSED pgs=2556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:14.296 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.295+0000 7f3aa6b95740 1 --2- 172.21.15.83:0/3565169675 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5602ab12c1b0 0x5602ab14f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:14.296 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.295+0000 7f3aa6b95740 1 -- 172.21.15.83:0/3565169675 >> 172.21.15.83:0/3565169675 conn(0x5602ab1286c0 msgr2=0x5602ab134bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:14.296 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.296+0000 7f3aa6b95740 1 -- 172.21.15.83:0/3565169675 shutdown_connections 2024-04-24T09:12:14.296 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.296+0000 7f3aa6b95740 1 -- 172.21.15.83:0/3565169675 wait complete. 2024-04-24T09:12:14.296 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.296+0000 7f3aa6b95740 1 -- start start 2024-04-24T09:12:14.296 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.296+0000 7f3aa6b95740 1 Processor -- start 2024-04-24T09:12:14.298 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.297+0000 7f3aa6b95740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602ab12c1b0 0x5602ab131350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:14.298 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.297+0000 7f3aa6b95740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5602ab12cad0 0x5602ab131890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:14.298 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.297+0000 7f3aa6b95740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602ab137390 0x5602ab131dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:14.298 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.297+0000 7f3aa6b95740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3a9c1b7160 con 0x5602ab12c1b0 2024-04-24T09:12:14.298 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.297+0000 7f3aa6b95740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3a9c1b6a90 con 0x5602ab12cad0 2024-04-24T09:12:14.298 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.297+0000 7f3aa6b95740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3a9c1b3cf0 con 0x5602ab137390 2024-04-24T09:12:14.298 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.297+0000 7f3aa6b95740 10 client.0 _ll_register_callbacks cb 0x7f3a90009550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:12:14.298 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.297+0000 7f3a9be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602ab12c1b0 0x5602ab131350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:14.298 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.297+0000 7f3a9be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602ab12c1b0 0x5602ab131350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50270/0 (socket says 172.21.15.83:50270) 2024-04-24T09:12:14.298 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.297+0000 7f3a9be00640 1 -- 172.21.15.83:0/3584364194 learned_addr learned my addr 172.21.15.83:0/3584364194 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:12:14.298 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.297+0000 7f3aa1200640 1 --2- 172.21.15.83:0/3584364194 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5602ab12cad0 0x5602ab131890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:14.299 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.297+0000 7f3a9be00640 1 --2- 172.21.15.83:0/3584364194 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602ab137390 0x5602ab131dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:14.299 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.298+0000 7f3a9be00640 1 -- 172.21.15.83:0/3584364194 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602ab137390 msgr2=0x5602ab131dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:14.299 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.298+0000 7f3a9be00640 1 --2- 172.21.15.83:0/3584364194 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602ab137390 0x5602ab131dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:14.299 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.298+0000 7f3a9be00640 1 -- 172.21.15.83:0/3584364194 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5602ab12cad0 msgr2=0x5602ab131890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:14.299 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.298+0000 7f3a9be00640 1 --2- 172.21.15.83:0/3584364194 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5602ab12cad0 0x5602ab131890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:14.299 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.298+0000 7f3a9be00640 1 -- 172.21.15.83:0/3584364194 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602ab1acd40 con 0x5602ab12c1b0 2024-04-24T09:12:14.299 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.298+0000 7f3aa1200640 1 --2- 172.21.15.83:0/3584364194 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5602ab12cad0 0x5602ab131890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:12:14.299 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.298+0000 7f3a9be00640 1 --2- 172.21.15.83:0/3584364194 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602ab12c1b0 0x5602ab131350 secure :-1 s=READY pgs=2557 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c00bad0 tx=0x7f3a8c00bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a98d8d227242a59 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:14.299 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.298+0000 7f3a7ea00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:12:14.299 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.298+0000 7f3a7ea00640 1 -- 172.21.15.83:0/3584364194 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a8c015040 con 0x5602ab12c1b0 2024-04-24T09:12:14.299 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.299+0000 7f3a7ea00640 1 -- 172.21.15.83:0/3584364194 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9583+0+0 (secure 0 0 0) 0x7f3a8c025040 con 0x5602ab12c1b0 2024-04-24T09:12:14.299 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.299+0000 7f3aa6b95740 1 -- 172.21.15.83:0/3584364194 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a900264e0 con 0x5602ab12c1b0 2024-04-24T09:12:14.300 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.299+0000 7f3a7ea00640 1 -- 172.21.15.83:0/3584364194 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f3a8c03f050 con 0x5602ab12c1b0 2024-04-24T09:12:14.300 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.299+0000 7f3aa6b95740 10 client.22755 fetch_fsmap learned FSMap version 61 2024-04-24T09:12:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.299+0000 7f3aa6b95740 1 -- 172.21.15.83:0/3584364194 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x5602ab1ae050 con 0x5602ab12c1b0 2024-04-24T09:12:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.300+0000 7f3a7ea00640 1 -- 172.21.15.83:0/3584364194 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3a900264e0 con 0x5602ab12c1b0 2024-04-24T09:12:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.300+0000 7f3a7ea00640 1 -- 172.21.15.83:0/3584364194 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 61) ==== 8362+0+0 (secure 0 0 0) 0x7f3a8c0051e0 con 0x5602ab12c1b0 2024-04-24T09:12:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.300+0000 7f3aa6b95740 10 client.22755 fetch_fsmap finished waiting for FSMap version 61 2024-04-24T09:12:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.300+0000 7f3aa6b95740 10 client.22755 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:12:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.300+0000 7f3aa6b95740 20 client.22755 populate_metadata read hostname 'smithi083' 2024-04-24T09:12:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.300+0000 7f3aa6b95740 1 --2- 172.21.15.83:0/3584364194 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5602ab1ae580 0x5602ab1b0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.300+0000 7f3aa6b95740 4 client.22755 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T09:12:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.300+0000 7f3aa6b95740 1 -- 172.21.15.83:0/3584364194 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5602ab09a190 con 0x5602ab1ae580 2024-04-24T09:12:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.301+0000 7f3aa1200640 1 --2- 172.21.15.83:0/3584364194 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5602ab1ae580 0x5602ab1b0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.301+0000 7f3aa1200640 1 --2- 172.21.15.83:0/3584364194 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5602ab1ae580 0x5602ab1b0970 crc :-1 s=READY pgs=516 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91369c90ebeba10 server_cookie=d93eaafedda15a0a in_seq=0 out_seq=0 2024-04-24T09:12:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.301+0000 7f3a7ea00640 10 client.22755 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:12:14.411 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.412+0000 7f3a7ea00640 1 -- 172.21.15.83:0/3584364194 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5602ab09a190 con 0x5602ab1ae580 2024-04-24T09:12:14.411 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.412+0000 7f3a7ea00640 10 client.22755 handle_command_reply: tid=0 2024-04-24T09:12:14.411 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.412+0000 7f3aa6b95740 1 client.22755 shutdown 2024-04-24T09:12:14.412 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.413+0000 7f3aa6b95740 1 -- 172.21.15.83:0/3584364194 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602ab12c1b0 msgr2=0x5602ab131350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:14.412 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.413+0000 7f3aa6b95740 1 --2- 172.21.15.83:0/3584364194 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602ab12c1b0 0x5602ab131350 secure :-1 s=READY pgs=2557 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c00bad0 tx=0x7f3a8c00bf90 comp rx=0 tx=0).stop 2024-04-24T09:12:14.413 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.413+0000 7f3aa6b95740 1 -- 172.21.15.83:0/3584364194 shutdown_connections 2024-04-24T09:12:14.413 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.413+0000 7f3aa6b95740 1 --2- 172.21.15.83:0/3584364194 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5602ab1ae580 0x5602ab1b0970 crc :-1 s=READY pgs=516 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:14.413 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.413+0000 7f3aa6b95740 1 --2- 172.21.15.83:0/3584364194 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5602ab137390 0x5602ab131dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:14.413 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.413+0000 7f3aa6b95740 1 --2- 172.21.15.83:0/3584364194 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5602ab12cad0 0x5602ab131890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:14.413 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.413+0000 7f3aa6b95740 1 --2- 172.21.15.83:0/3584364194 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602ab12c1b0 0x5602ab131350 unknown :-1 s=CLOSED pgs=2557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:14.413 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.413+0000 7f3aa6b95740 1 -- 172.21.15.83:0/3584364194 >> 172.21.15.83:0/3584364194 conn(0x5602ab1286c0 msgr2=0x5602ab126fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:14.413 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.413+0000 7f3a7ea00640 0 client.22755 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:12:14.414 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.414+0000 7f3aa6b95740 1 -- 172.21.15.83:0/3584364194 shutdown_connections 2024-04-24T09:12:14.414 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.414+0000 7f3aa6b95740 1 -- 172.21.15.83:0/3584364194 wait complete. 2024-04-24T09:12:14.414 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.415+0000 7f3aa6b95740 20 client.22755 trim_cache size 0 max 16384 2024-04-24T09:12:14.414 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T09:12:14.415 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T09:12:14.415 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T09:12:14.415 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T09:12:14.415 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.416+0000 7f3aa3c00640 1 -- 172.21.15.83:0/2328958530 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f3a840a3ec0 msgr2=0x7f3a840a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:14.415 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.416+0000 7f3aa3c00640 1 --2- 172.21.15.83:0/2328958530 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f3a840a3ec0 0x7f3a840a6380 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f3a940ba1e0 tx=0x7f3a940306d0 comp rx=0 tx=0).stop 2024-04-24T09:12:14.415 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.416+0000 7f3aa3c00640 1 -- 172.21.15.83:0/2328958530 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a9c10e690 msgr2=0x7f3a9c10ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:14.416 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.416+0000 7f3aa3c00640 1 --2- 172.21.15.83:0/2328958530 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a9c10e690 0x7f3a9c10ead0 secure :-1 s=READY pgs=2553 cs=0 l=1 rev1=1 crypto rx=0x7f3a94005e30 tx=0x7f3a940093d0 comp rx=0 tx=0).stop 2024-04-24T09:12:14.416 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.417+0000 7f3aa3c00640 1 -- 172.21.15.83:0/2328958530 shutdown_connections 2024-04-24T09:12:14.416 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.417+0000 7f3aa3c00640 1 --2- 172.21.15.83:0/2328958530 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f3a840a3ec0 0x7f3a840a6380 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:14.416 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.417+0000 7f3aa3c00640 1 --2- 172.21.15.83:0/2328958530 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a9c10e690 0x7f3a9c10ead0 unknown :-1 s=CLOSED pgs=2553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:14.416 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.417+0000 7f3aa3c00640 1 --2- 172.21.15.83:0/2328958530 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3a9c109520 0x7f3a9c10e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:14.416 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.417+0000 7f3aa3c00640 1 --2- 172.21.15.83:0/2328958530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a9c104450 0x7f3a9c110fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:14.416 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.417+0000 7f3aa3c00640 1 -- 172.21.15.83:0/2328958530 >> 172.21.15.83:0/2328958530 conn(0x7f3a9c0ffcf0 msgr2=0x7f3a9c100810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:14.417 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.417+0000 7f3aa3c00640 1 -- 172.21.15.83:0/2328958530 shutdown_connections 2024-04-24T09:12:14.417 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:14.417+0000 7f3aa3c00640 1 -- 172.21.15.83:0/2328958530 wait complete. 2024-04-24T09:12:15.185 INFO:tasks.workunit.client.0.smithi083.stdout: 10 50746 6.64 MB/sec execute 286 sec latency 114.354 ms 2024-04-24T09:12:16.185 INFO:tasks.workunit.client.0.smithi083.stdout: 10 50871 6.63 MB/sec execute 287 sec latency 74.906 ms 2024-04-24T09:12:16.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:12:15 smithi083 ceph-mon[27907]: pgmap v652: 129 pgs: 129 active+clean; 804 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.8 MiB/s wr, 270 op/s 2024-04-24T09:12:16.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:12:15 smithi138 ceph-mon[31441]: pgmap v652: 129 pgs: 129 active+clean; 804 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.8 MiB/s wr, 270 op/s 2024-04-24T09:12:16.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:12:15 smithi189 ceph-mon[31791]: pgmap v652: 129 pgs: 129 active+clean; 804 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.8 MiB/s wr, 270 op/s 2024-04-24T09:12:17.185 INFO:tasks.workunit.client.0.smithi083.stdout: 10 50996 6.64 MB/sec execute 288 sec latency 93.692 ms 2024-04-24T09:12:18.185 INFO:tasks.workunit.client.0.smithi083.stdout: 10 51169 6.66 MB/sec execute 289 sec latency 81.823 ms 2024-04-24T09:12:18.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:12:17 smithi083 ceph-mon[27907]: pgmap v653: 129 pgs: 129 active+clean; 804 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.8 MiB/s wr, 270 op/s 2024-04-24T09:12:18.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:12:17 smithi138 ceph-mon[31441]: pgmap v653: 129 pgs: 129 active+clean; 804 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.8 MiB/s wr, 270 op/s 2024-04-24T09:12:18.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:12:17 smithi189 ceph-mon[31791]: pgmap v653: 129 pgs: 129 active+clean; 804 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.8 MiB/s wr, 270 op/s 2024-04-24T09:12:19.185 INFO:tasks.workunit.client.0.smithi083.stdout: 10 51297 6.66 MB/sec execute 290 sec latency 72.540 ms 2024-04-24T09:12:20.185 INFO:tasks.workunit.client.0.smithi083.stdout: 10 51422 6.65 MB/sec execute 291 sec latency 70.133 ms 2024-04-24T09:12:20.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:12:19 smithi083 ceph-mon[27907]: pgmap v654: 129 pgs: 129 active+clean; 813 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.8 MiB/s wr, 295 op/s 2024-04-24T09:12:20.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:12:19 smithi138 ceph-mon[31441]: pgmap v654: 129 pgs: 129 active+clean; 813 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.8 MiB/s wr, 295 op/s 2024-04-24T09:12:20.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:12:19 smithi189 ceph-mon[31791]: pgmap v654: 129 pgs: 129 active+clean; 813 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.8 MiB/s wr, 295 op/s 2024-04-24T09:12:21.185 INFO:tasks.workunit.client.0.smithi083.stdout: 10 51573 6.68 MB/sec execute 292 sec latency 86.082 ms 2024-04-24T09:12:22.185 INFO:tasks.workunit.client.0.smithi083.stdout: 10 51718 6.68 MB/sec execute 293 sec latency 69.723 ms 2024-04-24T09:12:22.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:12:22 smithi083 ceph-mon[27907]: pgmap v655: 129 pgs: 129 active+clean; 813 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.4 MiB/s wr, 264 op/s 2024-04-24T09:12:22.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:12:22 smithi138 ceph-mon[31441]: pgmap v655: 129 pgs: 129 active+clean; 813 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.4 MiB/s wr, 264 op/s 2024-04-24T09:12:22.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:12:22 smithi189 ceph-mon[31791]: pgmap v655: 129 pgs: 129 active+clean; 813 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.4 MiB/s wr, 264 op/s 2024-04-24T09:12:23.185 INFO:tasks.workunit.client.0.smithi083.stdout: 10 51822 6.68 MB/sec execute 294 sec latency 80.210 ms 2024-04-24T09:12:23.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:12:23 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:12:23.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:12:23 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:12:23.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:12:23 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:12:23.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:12:23 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:12:23.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:12:23 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:12:23.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:12:23 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:12:23.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:12:23 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:12:23.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:12:23 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:12:23.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:12:23 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:12:23.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:12:23 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:12:23.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:12:23 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:12:23.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:12:23 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:12:24.185 INFO:tasks.workunit.client.0.smithi083.stdout: 10 51925 6.68 MB/sec execute 295 sec latency 72.163 ms 2024-04-24T09:12:24.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:12:24 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:12:24.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:12:24 smithi083 ceph-mon[27907]: pgmap v656: 129 pgs: 129 active+clean; 830 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.5 MiB/s wr, 343 op/s 2024-04-24T09:12:24.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:12:24 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:12:24.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:12:24 smithi138 ceph-mon[31441]: pgmap v656: 129 pgs: 129 active+clean; 830 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.5 MiB/s wr, 343 op/s 2024-04-24T09:12:24.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:12:24 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:12:24.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:12:24 smithi189 ceph-mon[31791]: pgmap v656: 129 pgs: 129 active+clean; 830 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.5 MiB/s wr, 343 op/s 2024-04-24T09:12:25.185 INFO:tasks.workunit.client.0.smithi083.stdout: 10 52042 6.69 MB/sec execute 296 sec latency 103.075 ms 2024-04-24T09:12:26.185 INFO:tasks.workunit.client.0.smithi083.stdout: 10 52190 6.69 MB/sec execute 297 sec latency 81.899 ms 2024-04-24T09:12:26.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:12:26 smithi083 ceph-mon[27907]: pgmap v657: 129 pgs: 129 active+clean; 830 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.0 MiB/s wr, 212 op/s 2024-04-24T09:12:26.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:12:26 smithi138 ceph-mon[31441]: pgmap v657: 129 pgs: 129 active+clean; 830 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.0 MiB/s wr, 212 op/s 2024-04-24T09:12:26.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:12:26 smithi189 ceph-mon[31791]: pgmap v657: 129 pgs: 129 active+clean; 830 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.0 MiB/s wr, 212 op/s 2024-04-24T09:12:27.185 INFO:tasks.workunit.client.0.smithi083.stdout: 10 52333 6.69 MB/sec execute 298 sec latency 75.471 ms 2024-04-24T09:12:27.625 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.624+0000 7f780ea00640 1 -- 172.21.15.83:0/2665573144 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7808106030 msgr2=0x7f78080696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:27.625 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.624+0000 7f780ea00640 1 --2- 172.21.15.83:0/2665573144 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7808106030 0x7f78080696b0 secure :-1 s=READY pgs=2558 cs=0 l=1 rev1=1 crypto rx=0x7f7800009ca0 tx=0x7f780002a8e0 comp rx=0 tx=0).stop 2024-04-24T09:12:27.625 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.625+0000 7f780ea00640 1 -- 172.21.15.83:0/2665573144 shutdown_connections 2024-04-24T09:12:27.625 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.625+0000 7f780ea00640 1 --2- 172.21.15.83:0/2665573144 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7808106030 0x7f78080696b0 unknown :-1 s=CLOSED pgs=2558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:27.625 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.625+0000 7f780ea00640 1 --2- 172.21.15.83:0/2665573144 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7808101910 0x7f7808105950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:27.626 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.625+0000 7f780ea00640 1 --2- 172.21.15.83:0/2665573144 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7808100f60 0x7f7808101340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:27.626 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.625+0000 7f780ea00640 1 -- 172.21.15.83:0/2665573144 >> 172.21.15.83:0/2665573144 conn(0x7f78080fc820 msgr2=0x7f78080fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:27.626 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.625+0000 7f780ea00640 1 -- 172.21.15.83:0/2665573144 shutdown_connections 2024-04-24T09:12:27.627 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.626+0000 7f780ea00640 1 -- 172.21.15.83:0/2665573144 wait complete. 2024-04-24T09:12:27.628 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.628+0000 7f780ea00640 1 Processor -- start 2024-04-24T09:12:27.628 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.628+0000 7f780ea00640 1 -- start start 2024-04-24T09:12:27.630 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.629+0000 7f780ea00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7808100f60 0x7f78081a0590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:27.630 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.629+0000 7f780ea00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7808106030 0x7f78081a0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:27.630 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.629+0000 7f780ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f780819a650 0x7f780819aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:27.630 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.629+0000 7f780ea00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f78081143a0 con 0x7f780819a650 2024-04-24T09:12:27.630 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.629+0000 7f780ea00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7808114220 con 0x7f7808100f60 2024-04-24T09:12:27.630 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.629+0000 7f780ea00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7808114520 con 0x7f7808106030 2024-04-24T09:12:27.631 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.630+0000 7f780d600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7808100f60 0x7f78081a0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:27.631 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.630+0000 7f780d600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7808100f60 0x7f78081a0590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:48066/0 (socket says 172.21.15.83:48066) 2024-04-24T09:12:27.631 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.630+0000 7f780d600640 1 -- 172.21.15.83:0/3759158282 learned_addr learned my addr 172.21.15.83:0/3759158282 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:12:27.631 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.630+0000 7f780cc00640 1 --2- 172.21.15.83:0/3759158282 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7808106030 0x7f78081a0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:27.631 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.630+0000 7f780e000640 1 --2- 172.21.15.83:0/3759158282 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f780819a650 0x7f780819aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:27.631 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.631+0000 7f780d600640 1 -- 172.21.15.83:0/3759158282 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7808106030 msgr2=0x7f78081a0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:27.631 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.631+0000 7f780d600640 1 --2- 172.21.15.83:0/3759158282 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7808106030 0x7f78081a0ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:27.631 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.631+0000 7f780d600640 1 -- 172.21.15.83:0/3759158282 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f780819a650 msgr2=0x7f780819aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:27.633 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.631+0000 7f780d600640 1 --2- 172.21.15.83:0/3759158282 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f780819a650 0x7f780819aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:27.633 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.631+0000 7f780d600640 1 -- 172.21.15.83:0/3759158282 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f780819b250 con 0x7f7808100f60 2024-04-24T09:12:27.633 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.633+0000 7f780d600640 1 --2- 172.21.15.83:0/3759158282 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7808100f60 0x7f78081a0590 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f77fc00c890 tx=0x7f77fc00cd50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:27.634 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.634+0000 7f7806000640 1 -- 172.21.15.83:0/3759158282 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77fc004220 con 0x7f7808100f60 2024-04-24T09:12:27.635 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.634+0000 7f7806000640 1 -- 172.21.15.83:0/3759158282 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f77fc0043c0 con 0x7f7808100f60 2024-04-24T09:12:27.635 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.634+0000 7f7806000640 1 -- 172.21.15.83:0/3759158282 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77fc004a00 con 0x7f7808100f60 2024-04-24T09:12:27.635 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.634+0000 7f780ea00640 1 -- 172.21.15.83:0/3759158282 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78081a7330 con 0x7f7808100f60 2024-04-24T09:12:27.635 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.634+0000 7f780ea00640 1 -- 172.21.15.83:0/3759158282 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78081a76e0 con 0x7f7808100f60 2024-04-24T09:12:27.639 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.639+0000 7f7812d73740 1 Processor -- start 2024-04-24T09:12:27.640 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.640+0000 7f7812d73740 1 -- start start 2024-04-24T09:12:27.640 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.640+0000 7f7812d73740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5600d95683c0 0x5600d95687a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:27.640 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.640+0000 7f7812d73740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5600d9568ce0 0x5600d9573060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:27.640 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.640+0000 7f7812d73740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5600d95735a0 0x5600d9575990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:27.640 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.640+0000 7f7812d73740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5600d94ce900 con 0x5600d95683c0 2024-04-24T09:12:27.641 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.640+0000 7f7812d73740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5600d9474f20 con 0x5600d9568ce0 2024-04-24T09:12:27.641 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.640+0000 7f7812d73740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5600d94ca8c0 con 0x5600d95735a0 2024-04-24T09:12:27.642 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.641+0000 7f780e000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5600d95683c0 0x5600d95687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:27.642 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.641+0000 7f780e000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5600d95683c0 0x5600d95687a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33072/0 (socket says 172.21.15.83:33072) 2024-04-24T09:12:27.642 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.641+0000 7f780e000640 1 -- 172.21.15.83:0/169170326 learned_addr learned my addr 172.21.15.83:0/169170326 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:12:27.642 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.642+0000 7f780e000640 1 -- 172.21.15.83:0/169170326 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5600d95735a0 msgr2=0x5600d9575990 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-24T09:12:27.642 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.642+0000 7f780e000640 1 --2- 172.21.15.83:0/169170326 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5600d95735a0 0x5600d9575990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:27.642 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.642+0000 7f780e000640 1 -- 172.21.15.83:0/169170326 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5600d9568ce0 msgr2=0x5600d9573060 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-24T09:12:27.643 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.642+0000 7f780e000640 1 --2- 172.21.15.83:0/169170326 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5600d9568ce0 0x5600d9573060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:27.643 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.642+0000 7f780e000640 1 -- 172.21.15.83:0/169170326 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600d9499e10 con 0x5600d95683c0 2024-04-24T09:12:27.643 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.643+0000 7f780e000640 1 --2- 172.21.15.83:0/169170326 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5600d95683c0 0x5600d95687a0 secure :-1 s=READY pgs=2559 cs=0 l=1 rev1=1 crypto rx=0x7f78000099c0 tx=0x7f78000099f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca9ac69ec463e5a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:27.643 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.643+0000 7f7806000640 1 -- 172.21.15.83:0/3759158282 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f77fc004ba0 con 0x7f7808100f60 2024-04-24T09:12:27.644 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.643+0000 7f77fa600640 1 -- 172.21.15.83:0/169170326 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7800031890 con 0x5600d95683c0 2024-04-24T09:12:27.644 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.643+0000 7f77fa600640 1 -- 172.21.15.83:0/169170326 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7800036040 con 0x5600d95683c0 2024-04-24T09:12:27.644 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.644+0000 7f77fa600640 1 -- 172.21.15.83:0/169170326 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f780004d690 con 0x5600d95683c0 2024-04-24T09:12:27.648 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.647+0000 7f7806000640 1 --2- 172.21.15.83:0/3759158282 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f77f00a3e10 0x7f77f00a62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:27.648 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.647+0000 7f7806000640 1 -- 172.21.15.83:0/3759158282 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 9583+0+0 (secure 0 0 0) 0x7f77fc0d0900 con 0x7f7808100f60 2024-04-24T09:12:27.649 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.648+0000 7f7812d73740 1 -- 172.21.15.83:0/169170326 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5600d95683c0 msgr2=0x5600d95687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:27.649 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.648+0000 7f7812d73740 1 --2- 172.21.15.83:0/169170326 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5600d95683c0 0x5600d95687a0 secure :-1 s=READY pgs=2559 cs=0 l=1 rev1=1 crypto rx=0x7f78000099c0 tx=0x7f78000099f0 comp rx=0 tx=0).stop 2024-04-24T09:12:27.649 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.648+0000 7f780d600640 1 --2- 172.21.15.83:0/3759158282 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f77f00a3e10 0x7f77f00a62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:27.649 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.649+0000 7f7812d73740 1 -- 172.21.15.83:0/169170326 shutdown_connections 2024-04-24T09:12:27.649 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.649+0000 7f7812d73740 1 --2- 172.21.15.83:0/169170326 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5600d95735a0 0x5600d9575990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:27.650 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.649+0000 7f7812d73740 1 --2- 172.21.15.83:0/169170326 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5600d9568ce0 0x5600d9573060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:27.650 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.649+0000 7f7812d73740 1 --2- 172.21.15.83:0/169170326 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5600d95683c0 0x5600d95687a0 unknown :-1 s=CLOSED pgs=2559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:27.650 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.649+0000 7f7812d73740 1 -- 172.21.15.83:0/169170326 >> 172.21.15.83:0/169170326 conn(0x5600d95648d0 msgr2=0x5600d9564cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:27.651 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.650+0000 7f7812d73740 1 -- 172.21.15.83:0/169170326 shutdown_connections 2024-04-24T09:12:27.651 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.650+0000 7f780d600640 1 --2- 172.21.15.83:0/3759158282 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f77f00a3e10 0x7f77f00a62d0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x5600d9569960 tx=0x7f77fc096f90 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:27.652 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.651+0000 7f7812d73740 1 -- 172.21.15.83:0/169170326 wait complete. 2024-04-24T09:12:27.652 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.651+0000 7f7812d73740 1 -- start start 2024-04-24T09:12:27.652 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.651+0000 7f7812d73740 1 Processor -- start 2024-04-24T09:12:27.652 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.652+0000 7f7812d73740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5600d95683c0 0x5600d9570630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:27.653 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.653+0000 7f7812d73740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5600d9568ce0 0x5600d9570b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:27.653 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.653+0000 7f7812d73740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5600d95735a0 0x5600d95710b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:27.653 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.653+0000 7f780e000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5600d95683c0 0x5600d9570630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:27.653 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.653+0000 7f780e000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5600d95683c0 0x5600d9570630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33080/0 (socket says 172.21.15.83:33080) 2024-04-24T09:12:27.653 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.653+0000 7f780e000640 1 -- 172.21.15.83:0/1667295013 learned_addr learned my addr 172.21.15.83:0/1667295013 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:12:27.653 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.653+0000 7f7812d73740 1 -- 172.21.15.83:0/1667295013 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5600d95760e0 con 0x5600d95683c0 2024-04-24T09:12:27.653 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.653+0000 7f7812d73740 1 -- 172.21.15.83:0/1667295013 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5600d94882c0 con 0x5600d9568ce0 2024-04-24T09:12:27.654 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.653+0000 7f7812d73740 1 -- 172.21.15.83:0/1667295013 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5600d9576260 con 0x5600d95735a0 2024-04-24T09:12:27.654 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.653+0000 7f7812d73740 10 client.0 _ll_register_callbacks cb 0x5600d91e0a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:12:27.654 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.653+0000 7f780cc00640 1 --2- 172.21.15.83:0/1667295013 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5600d9568ce0 0x5600d9570b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:27.654 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.654+0000 7f780e000640 1 -- 172.21.15.83:0/1667295013 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5600d95735a0 msgr2=0x5600d95710b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:27.654 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.654+0000 7f780e000640 1 --2- 172.21.15.83:0/1667295013 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5600d95735a0 0x5600d95710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:27.654 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.654+0000 7f780e000640 1 -- 172.21.15.83:0/1667295013 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5600d9568ce0 msgr2=0x5600d9570b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:27.654 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.654+0000 7f780e000640 1 --2- 172.21.15.83:0/1667295013 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5600d9568ce0 0x5600d9570b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:27.655 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.654+0000 7f780e000640 1 -- 172.21.15.83:0/1667295013 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600d9498fd0 con 0x5600d95683c0 2024-04-24T09:12:27.655 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.654+0000 7f780e000640 1 --2- 172.21.15.83:0/1667295013 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5600d95683c0 0x5600d9570630 secure :-1 s=READY pgs=2560 cs=0 l=1 rev1=1 crypto rx=0x7f7800030dc0 tx=0x7f7800005c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c441fb8e7ef31e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:27.655 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.654+0000 7f77eea00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:12:27.656 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.656+0000 7f77eea00640 1 -- 172.21.15.83:0/1667295013 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7800031ca0 con 0x5600d95683c0 2024-04-24T09:12:27.657 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.657+0000 7f77eea00640 1 -- 172.21.15.83:0/1667295013 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9583+0+0 (secure 0 0 0) 0x7f780004d430 con 0x5600d95683c0 2024-04-24T09:12:27.657 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.657+0000 7f7812d73740 1 -- 172.21.15.83:0/1667295013 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7800036040 con 0x5600d95683c0 2024-04-24T09:12:27.658 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.658+0000 7f77eea00640 1 -- 172.21.15.83:0/1667295013 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f7800030ea0 con 0x5600d95683c0 2024-04-24T09:12:27.658 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.658+0000 7f7812d73740 10 client.22767 fetch_fsmap learned FSMap version 61 2024-04-24T09:12:27.659 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.658+0000 7f7812d73740 1 -- 172.21.15.83:0/1667295013 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x5600d91e1210 con 0x5600d95683c0 2024-04-24T09:12:27.660 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.660+0000 7f77eea00640 1 -- 172.21.15.83:0/1667295013 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7800036040 con 0x5600d95683c0 2024-04-24T09:12:27.661 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.661+0000 7f77eea00640 1 -- 172.21.15.83:0/1667295013 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 61) ==== 8362+0+0 (secure 0 0 0) 0x7f7800058460 con 0x5600d95683c0 2024-04-24T09:12:27.661 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.661+0000 7f7812d73740 10 client.22767 fetch_fsmap finished waiting for FSMap version 61 2024-04-24T09:12:27.661 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.661+0000 7f7812d73740 10 client.22767 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:12:27.661 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.661+0000 7f7812d73740 20 client.22767 populate_metadata read hostname 'smithi083' 2024-04-24T09:12:27.662 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.661+0000 7f7812d73740 1 --2- 172.21.15.83:0/1667295013 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5600d9583630 0x5600d9585a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:27.662 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.661+0000 7f7812d73740 4 client.22767 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:12:27.662 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.661+0000 7f7812d73740 1 -- 172.21.15.83:0/1667295013 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5600d94dc320 con 0x5600d9583630 2024-04-24T09:12:27.662 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.662+0000 7f780cc00640 1 --2- 172.21.15.83:0/1667295013 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5600d9583630 0x5600d9585a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:27.664 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.663+0000 7f780cc00640 1 --2- 172.21.15.83:0/1667295013 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5600d9583630 0x5600d9585a20 crc :-1 s=READY pgs=517 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b362088404b4c759 server_cookie=2508639bff664861 in_seq=0 out_seq=0 2024-04-24T09:12:27.664 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.664+0000 7f77eea00640 10 client.22767 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:12:27.665 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.665+0000 7f77eea00640 1 -- 172.21.15.83:0/1667295013 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5600d94dc320 con 0x5600d9583630 2024-04-24T09:12:27.666 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.665+0000 7f77eea00640 10 client.22767 handle_command_reply: tid=0 2024-04-24T09:12:27.666 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.665+0000 7f7812d73740 1 client.22767 shutdown 2024-04-24T09:12:27.666 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.666+0000 7f7812d73740 1 -- 172.21.15.83:0/1667295013 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5600d95683c0 msgr2=0x5600d9570630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:27.666 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.666+0000 7f7812d73740 1 --2- 172.21.15.83:0/1667295013 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5600d95683c0 0x5600d9570630 secure :-1 s=READY pgs=2560 cs=0 l=1 rev1=1 crypto rx=0x7f7800030dc0 tx=0x7f7800005c60 comp rx=0 tx=0).stop 2024-04-24T09:12:27.667 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.667+0000 7f7812d73740 1 -- 172.21.15.83:0/1667295013 shutdown_connections 2024-04-24T09:12:27.667 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.667+0000 7f7812d73740 1 --2- 172.21.15.83:0/1667295013 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5600d9583630 0x5600d9585a20 crc :-1 s=READY pgs=517 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:27.667 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.667+0000 7f7812d73740 1 --2- 172.21.15.83:0/1667295013 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5600d95735a0 0x5600d95710b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:27.667 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.667+0000 7f7812d73740 1 --2- 172.21.15.83:0/1667295013 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5600d9568ce0 0x5600d9570b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:27.667 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.667+0000 7f7812d73740 1 --2- 172.21.15.83:0/1667295013 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5600d95683c0 0x5600d9570630 unknown :-1 s=CLOSED pgs=2560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:27.667 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.667+0000 7f7812d73740 1 -- 172.21.15.83:0/1667295013 >> 172.21.15.83:0/1667295013 conn(0x5600d95648d0 msgr2=0x5600d956a650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:27.667 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.667+0000 7f77eea00640 0 client.22767 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:12:27.668 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.667+0000 7f7812d73740 1 -- 172.21.15.83:0/1667295013 shutdown_connections 2024-04-24T09:12:27.668 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.668+0000 7f7812d73740 1 -- 172.21.15.83:0/1667295013 wait complete. 2024-04-24T09:12:27.668 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.668+0000 7f7812d73740 20 client.22767 trim_cache size 0 max 16384 2024-04-24T09:12:27.692 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.691+0000 7f7812d73740 1 Processor -- start 2024-04-24T09:12:27.692 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.691+0000 7f7812d73740 1 -- start start 2024-04-24T09:12:27.692 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.692+0000 7f7812d73740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5600d95683c0 0x5600d9572d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:27.692 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.692+0000 7f7812d73740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5600d9568ce0 0x5600d957f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:27.692 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.692+0000 7f7812d73740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5600d95735a0 0x5600d958afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:27.692 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.692+0000 7f7812d73740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f78081ad2b0 con 0x5600d95735a0 2024-04-24T09:12:27.693 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.692+0000 7f7812d73740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f78081aedf0 con 0x5600d95683c0 2024-04-24T09:12:27.693 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.692+0000 7f7812d73740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f78081ad980 con 0x5600d9568ce0 2024-04-24T09:12:27.693 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.692+0000 7f780e000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5600d95735a0 0x5600d958afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:27.693 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.692+0000 7f780cc00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5600d9568ce0 0x5600d957f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:27.693 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.692+0000 7f780e000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5600d95683c0 0x5600d9572d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:27.693 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.692+0000 7f780cc00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5600d9568ce0 0x5600d957f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:36908/0 (socket says 172.21.15.83:36908) 2024-04-24T09:12:27.693 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.692+0000 7f780cc00640 1 -- 172.21.15.83:0/2058159466 learned_addr learned my addr 172.21.15.83:0/2058159466 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:12:27.693 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.692+0000 7f780e000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5600d95683c0 0x5600d9572d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:48098/0 (socket says 172.21.15.83:48098) 2024-04-24T09:12:27.694 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.693+0000 7f780cc00640 1 -- 172.21.15.83:0/2058159466 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5600d95683c0 msgr2=0x5600d9572d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:27.694 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.693+0000 7f780cc00640 1 --2- 172.21.15.83:0/2058159466 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5600d95683c0 0x5600d9572d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:27.694 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.693+0000 7f780cc00640 1 -- 172.21.15.83:0/2058159466 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5600d95735a0 msgr2=0x5600d958afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:27.694 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.693+0000 7f780cc00640 1 --2- 172.21.15.83:0/2058159466 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5600d95735a0 0x5600d958afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:27.694 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.693+0000 7f780cc00640 1 -- 172.21.15.83:0/2058159466 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600d956f5e0 con 0x5600d9568ce0 2024-04-24T09:12:27.694 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.693+0000 7f780cc00640 1 --2- 172.21.15.83:0/2058159466 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5600d9568ce0 0x5600d957f0b0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x5600d9565a60 tx=0x7f77f4010800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a51874ed3cfed14 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:27.694 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.694+0000 7f77f9c00640 1 -- 172.21.15.83:0/2058159466 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f4004050 con 0x5600d9568ce0 2024-04-24T09:12:27.695 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.694+0000 7f77f9c00640 1 -- 172.21.15.83:0/2058159466 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f77f40041f0 con 0x5600d9568ce0 2024-04-24T09:12:27.695 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.695+0000 7f77f9c00640 1 -- 172.21.15.83:0/2058159466 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f401d280 con 0x5600d9568ce0 2024-04-24T09:12:27.695 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.695+0000 7f7812d73740 1 -- 172.21.15.83:0/2058159466 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5600d9568ce0 msgr2=0x5600d957f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:27.695 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.695+0000 7f7812d73740 1 --2- 172.21.15.83:0/2058159466 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5600d9568ce0 0x5600d957f0b0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x5600d9565a60 tx=0x7f77f4010800 comp rx=0 tx=0).stop 2024-04-24T09:12:27.696 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.695+0000 7f7812d73740 1 -- 172.21.15.83:0/2058159466 shutdown_connections 2024-04-24T09:12:27.696 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.695+0000 7f7812d73740 1 --2- 172.21.15.83:0/2058159466 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5600d95735a0 0x5600d958afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:27.696 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.695+0000 7f7812d73740 1 --2- 172.21.15.83:0/2058159466 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5600d9568ce0 0x5600d957f0b0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:27.696 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.695+0000 7f7812d73740 1 --2- 172.21.15.83:0/2058159466 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5600d95683c0 0x5600d9572d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:27.696 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.695+0000 7f7812d73740 1 -- 172.21.15.83:0/2058159466 >> 172.21.15.83:0/2058159466 conn(0x5600d95648d0 msgr2=0x5600d9564cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:27.696 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.696+0000 7f7812d73740 1 -- 172.21.15.83:0/2058159466 shutdown_connections 2024-04-24T09:12:27.696 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.696+0000 7f7812d73740 1 -- 172.21.15.83:0/2058159466 wait complete. 2024-04-24T09:12:27.697 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.696+0000 7f7812d73740 1 -- start start 2024-04-24T09:12:27.697 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.696+0000 7f7812d73740 1 Processor -- start 2024-04-24T09:12:27.698 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.697+0000 7f7812d73740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5600d95683c0 0x5600d9577020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:27.698 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.698+0000 7f7812d73740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5600d9568ce0 0x5600d9577560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:27.699 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.698+0000 7f7812d73740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5600d95735a0 0x5600d9577aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:27.699 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.698+0000 7f7812d73740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f78081ae720 con 0x5600d9568ce0 2024-04-24T09:12:27.699 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.698+0000 7f7812d73740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f78081ae050 con 0x5600d95735a0 2024-04-24T09:12:27.699 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.698+0000 7f7812d73740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f78081ab2d0 con 0x5600d95683c0 2024-04-24T09:12:27.699 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.698+0000 7f780e000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5600d95683c0 0x5600d9577020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:27.700 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.698+0000 7f7812d73740 10 client.0 _ll_register_callbacks cb 0x7f7800009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:12:27.700 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.698+0000 7f780e000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5600d95683c0 0x5600d9577020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:36924/0 (socket says 172.21.15.83:36924) 2024-04-24T09:12:27.700 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.698+0000 7f780e000640 1 -- 172.21.15.83:0/218200879 learned_addr learned my addr 172.21.15.83:0/218200879 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:12:27.700 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.698+0000 7f780cc00640 1 --2- 172.21.15.83:0/218200879 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5600d9568ce0 0x5600d9577560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:27.700 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.698+0000 7f780e000640 1 --2- 172.21.15.83:0/218200879 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5600d95735a0 0x5600d9577aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:27.700 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.699+0000 7f780e000640 1 -- 172.21.15.83:0/218200879 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5600d95735a0 msgr2=0x5600d9577aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:27.700 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.699+0000 7f780e000640 1 --2- 172.21.15.83:0/218200879 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5600d95735a0 0x5600d9577aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:27.700 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.699+0000 7f780e000640 1 -- 172.21.15.83:0/218200879 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5600d9568ce0 msgr2=0x5600d9577560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:27.700 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.699+0000 7f780e000640 1 --2- 172.21.15.83:0/218200879 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5600d9568ce0 0x5600d9577560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:27.700 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.699+0000 7f780e000640 1 -- 172.21.15.83:0/218200879 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600d9568b10 con 0x5600d95683c0 2024-04-24T09:12:27.700 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.699+0000 7f780cc00640 1 --2- 172.21.15.83:0/218200879 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5600d9568ce0 0x5600d9577560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:12:27.700 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.699+0000 7f780e000640 1 --2- 172.21.15.83:0/218200879 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5600d95683c0 0x5600d9577020 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7800031040 tx=0x7f7800030930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9ddac41493c6d57 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:27.700 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.699+0000 7f77eea00640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-04-24T09:12:27.701 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.700+0000 7f77eea00640 1 -- 172.21.15.83:0/218200879 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7800072060 con 0x5600d95683c0 2024-04-24T09:12:27.702 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.700+0000 7f7812d73740 1 -- 172.21.15.83:0/218200879 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77f40041f0 con 0x5600d95683c0 2024-04-24T09:12:27.702 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.700+0000 7f77eea00640 1 -- 172.21.15.83:0/218200879 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9583+0+0 (secure 0 0 0) 0x7f7800031aa0 con 0x5600d95683c0 2024-04-24T09:12:27.702 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.700+0000 7f77eea00640 1 -- 172.21.15.83:0/218200879 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f7800079050 con 0x5600d95683c0 2024-04-24T09:12:27.702 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.700+0000 7f7812d73740 10 client.29656 fetch_fsmap learned FSMap version 61 2024-04-24T09:12:27.702 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.700+0000 7f7812d73740 1 -- 172.21.15.83:0/218200879 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x5600d92ad620 con 0x5600d95683c0 2024-04-24T09:12:27.702 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.701+0000 7f77eea00640 1 -- 172.21.15.83:0/218200879 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f77f40041f0 con 0x5600d95683c0 2024-04-24T09:12:27.702 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.701+0000 7f77eea00640 1 -- 172.21.15.83:0/218200879 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 61) ==== 8362+0+0 (secure 0 0 0) 0x7f780006d420 con 0x5600d95683c0 2024-04-24T09:12:27.702 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.701+0000 7f7812d73740 10 client.29656 fetch_fsmap finished waiting for FSMap version 61 2024-04-24T09:12:27.702 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.701+0000 7f7812d73740 10 client.29656 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:12:27.702 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.701+0000 7f7812d73740 20 client.29656 populate_metadata read hostname 'smithi083' 2024-04-24T09:12:27.702 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.701+0000 7f7812d73740 1 --2- 172.21.15.83:0/218200879 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5600d95e9c40 0x5600d95ec030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:27.702 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.701+0000 7f7812d73740 4 client.29656 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T09:12:27.702 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.701+0000 7f7812d73740 1 -- 172.21.15.83:0/218200879 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5600d94d6190 con 0x5600d95e9c40 2024-04-24T09:12:27.702 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.701+0000 7f780cc00640 1 --2- 172.21.15.83:0/218200879 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5600d95e9c40 0x5600d95ec030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:27.703 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.703+0000 7f77eea00640 10 client.29656 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:12:27.703 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.703+0000 7f780cc00640 1 --2- 172.21.15.83:0/218200879 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5600d95e9c40 0x5600d95ec030 crc :-1 s=READY pgs=518 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7fc102b3b22eaf52 server_cookie=b8405ac5b110596c in_seq=0 out_seq=0 2024-04-24T09:12:27.904 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.904+0000 7f77eea00640 1 -- 172.21.15.83:0/218200879 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5600d94d6190 con 0x5600d95e9c40 2024-04-24T09:12:27.905 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.904+0000 7f77eea00640 10 client.29656 handle_command_reply: tid=0 2024-04-24T09:12:27.905 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.904+0000 7f7812d73740 1 client.29656 shutdown 2024-04-24T09:12:27.905 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.905+0000 7f7812d73740 1 -- 172.21.15.83:0/218200879 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5600d95683c0 msgr2=0x5600d9577020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:27.906 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.905+0000 7f7812d73740 1 --2- 172.21.15.83:0/218200879 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5600d95683c0 0x5600d9577020 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7800031040 tx=0x7f7800030930 comp rx=0 tx=0).stop 2024-04-24T09:12:27.906 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.905+0000 7f7812d73740 1 -- 172.21.15.83:0/218200879 shutdown_connections 2024-04-24T09:12:27.906 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.905+0000 7f7812d73740 1 --2- 172.21.15.83:0/218200879 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5600d95e9c40 0x5600d95ec030 crc :-1 s=READY pgs=518 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:27.906 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.905+0000 7f7812d73740 1 --2- 172.21.15.83:0/218200879 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5600d95735a0 0x5600d9577aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:27.906 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.905+0000 7f7812d73740 1 --2- 172.21.15.83:0/218200879 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5600d9568ce0 0x5600d9577560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:27.906 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.905+0000 7f7812d73740 1 --2- 172.21.15.83:0/218200879 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5600d95683c0 0x5600d9577020 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:27.906 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.905+0000 7f7812d73740 1 -- 172.21.15.83:0/218200879 >> 172.21.15.83:0/218200879 conn(0x5600d95648d0 msgr2=0x5600d9563200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:27.906 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.905+0000 7f77eea00640 0 client.29656 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:12:27.906 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.906+0000 7f7812d73740 1 -- 172.21.15.83:0/218200879 shutdown_connections 2024-04-24T09:12:27.907 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.906+0000 7f7812d73740 1 -- 172.21.15.83:0/218200879 wait complete. 2024-04-24T09:12:27.907 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.906+0000 7f7812d73740 20 client.29656 trim_cache size 0 max 16384 2024-04-24T09:12:27.907 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T09:12:27.907 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T09:12:27.907 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T09:12:27.907 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T09:12:27.908 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.908+0000 7f780ea00640 1 -- 172.21.15.83:0/3759158282 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f77f00a3e10 msgr2=0x7f77f00a62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:27.908 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.908+0000 7f780ea00640 1 --2- 172.21.15.83:0/3759158282 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f77f00a3e10 0x7f77f00a62d0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x5600d9569960 tx=0x7f77fc096f90 comp rx=0 tx=0).stop 2024-04-24T09:12:27.908 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.908+0000 7f780ea00640 1 -- 172.21.15.83:0/3759158282 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7808100f60 msgr2=0x7f78081a0590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:27.908 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.908+0000 7f780ea00640 1 --2- 172.21.15.83:0/3759158282 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7808100f60 0x7f78081a0590 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f77fc00c890 tx=0x7f77fc00cd50 comp rx=0 tx=0).stop 2024-04-24T09:12:27.908 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.908+0000 7f780ea00640 1 -- 172.21.15.83:0/3759158282 shutdown_connections 2024-04-24T09:12:27.909 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.908+0000 7f780ea00640 1 --2- 172.21.15.83:0/3759158282 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f77f00a3e10 0x7f77f00a62d0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:27.909 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.908+0000 7f780ea00640 1 --2- 172.21.15.83:0/3759158282 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f780819a650 0x7f780819aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:27.909 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.908+0000 7f780ea00640 1 --2- 172.21.15.83:0/3759158282 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7808106030 0x7f78081a0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:27.909 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.908+0000 7f780ea00640 1 --2- 172.21.15.83:0/3759158282 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7808100f60 0x7f78081a0590 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:27.909 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.908+0000 7f780ea00640 1 -- 172.21.15.83:0/3759158282 >> 172.21.15.83:0/3759158282 conn(0x7f78080fc820 msgr2=0x7f780810dcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:27.910 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.909+0000 7f780ea00640 1 -- 172.21.15.83:0/3759158282 shutdown_connections 2024-04-24T09:12:27.910 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:27.909+0000 7f780ea00640 1 -- 172.21.15.83:0/3759158282 wait complete. 2024-04-24T09:12:28.185 INFO:tasks.workunit.client.0.smithi083.stdout: 10 52415 6.68 MB/sec execute 299 sec latency 137.201 ms 2024-04-24T09:12:28.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:12:28 smithi083 ceph-mon[27907]: pgmap v658: 129 pgs: 129 active+clean; 830 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.0 MiB/s wr, 212 op/s 2024-04-24T09:12:28.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:12:28 smithi138 ceph-mon[31441]: pgmap v658: 129 pgs: 129 active+clean; 830 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.0 MiB/s wr, 212 op/s 2024-04-24T09:12:28.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:12:28 smithi189 ceph-mon[31791]: pgmap v658: 129 pgs: 129 active+clean; 830 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.0 MiB/s wr, 212 op/s 2024-04-24T09:12:29.186 INFO:tasks.workunit.client.0.smithi083.stdout: 10 52508 6.66 MB/sec execute 300 sec latency 96.590 ms 2024-04-24T09:12:29.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:12:29 smithi083 ceph-mon[27907]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:12:29.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:12:29 smithi138 ceph-mon[31441]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:12:29.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:12:29 smithi189 ceph-mon[31791]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:12:30.186 INFO:tasks.workunit.client.0.smithi083.stdout: 10 52575 6.65 MB/sec execute 301 sec latency 382.250 ms 2024-04-24T09:12:30.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:12:30 smithi083 ceph-mon[27907]: pgmap v659: 129 pgs: 129 active+clean; 843 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.1 MiB/s wr, 313 op/s 2024-04-24T09:12:30.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:12:30 smithi083 ceph-mon[27907]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:12:30.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:12:30 smithi138 ceph-mon[31441]: pgmap v659: 129 pgs: 129 active+clean; 843 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.1 MiB/s wr, 313 op/s 2024-04-24T09:12:30.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:12:30 smithi138 ceph-mon[31441]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:12:30.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:12:30 smithi189 ceph-mon[31791]: pgmap v659: 129 pgs: 129 active+clean; 843 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.1 MiB/s wr, 313 op/s 2024-04-24T09:12:30.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:12:30 smithi189 ceph-mon[31791]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:12:31.186 INFO:tasks.workunit.client.0.smithi083.stdout: 10 52698 6.66 MB/sec execute 302 sec latency 211.322 ms 2024-04-24T09:12:31.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:12:31 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:12:31.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:12:31 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:12:31.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:12:31 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:12:32.186 INFO:tasks.workunit.client.0.smithi083.stdout: 10 52838 6.67 MB/sec execute 303 sec latency 83.348 ms 2024-04-24T09:12:32.329 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:12:32 smithi138 ceph-mon[31441]: pgmap v660: 129 pgs: 129 active+clean; 843 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.3 MiB/s wr, 198 op/s 2024-04-24T09:12:32.359 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:12:32 smithi189 ceph-mon[31791]: pgmap v660: 129 pgs: 129 active+clean; 843 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.3 MiB/s wr, 198 op/s 2024-04-24T09:12:32.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:12:32 smithi083 ceph-mon[27907]: pgmap v660: 129 pgs: 129 active+clean; 843 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.3 MiB/s wr, 198 op/s 2024-04-24T09:12:33.186 INFO:tasks.workunit.client.0.smithi083.stdout: 10 52948 6.66 MB/sec execute 304 sec latency 89.946 ms 2024-04-24T09:12:34.186 INFO:tasks.workunit.client.0.smithi083.stdout: 10 53062 6.67 MB/sec execute 305 sec latency 244.098 ms 2024-04-24T09:12:34.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:12:34 smithi138 ceph-mon[31441]: pgmap v661: 129 pgs: 129 active+clean; 844 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.3 MiB/s wr, 346 op/s 2024-04-24T09:12:34.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:12:34 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:12:34.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:12:34 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:12:34.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:12:34 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:12:34.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:12:34 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:12:34.459 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:12:34 smithi189 ceph-mon[31791]: pgmap v661: 129 pgs: 129 active+clean; 844 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.3 MiB/s wr, 346 op/s 2024-04-24T09:12:34.459 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:12:34 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:12:34.459 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:12:34 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:12:34.459 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:12:34 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:12:34.459 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:12:34 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:12:34.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:12:34 smithi083 ceph-mon[27907]: pgmap v661: 129 pgs: 129 active+clean; 844 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.3 MiB/s wr, 346 op/s 2024-04-24T09:12:34.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:12:34 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:12:34.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:12:34 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:12:34.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:12:34 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:12:34.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:12:34 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:12:35.186 INFO:tasks.workunit.client.0.smithi083.stdout: 10 53158 6.66 MB/sec execute 306 sec latency 92.587 ms 2024-04-24T09:12:35.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:12:35 smithi138 ceph-mon[31441]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:12:35.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:12:35 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:12:35.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:12:35 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:12:35.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:12:35 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:12:35.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:12:35 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:12:35.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:12:35 smithi189 ceph-mon[31791]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:12:35.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:12:35 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:12:35.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:12:35 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:12:35.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:12:35 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:12:35.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:12:35 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:12:35.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:12:35 smithi083 ceph-mon[27907]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:12:35.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:12:35 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:12:35.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:12:35 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:12:35.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:12:35 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:12:35.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:12:35 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:12:36.186 INFO:tasks.workunit.client.0.smithi083.stdout: 10 53255 6.64 MB/sec execute 307 sec latency 82.019 ms 2024-04-24T09:12:36.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:12:36 smithi138 ceph-mon[31441]: Adjusting osd_memory_target on smithi189 to 176.4M 2024-04-24T09:12:36.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:12:36 smithi138 ceph-mon[31441]: Unable to set osd_memory_target on smithi189 to 185064857: error parsing value: Value '185064857' is below minimum 939524096 2024-04-24T09:12:36.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:12:36 smithi138 ceph-mon[31441]: pgmap v662: 129 pgs: 129 active+clean; 844 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.0 MiB/s wr, 249 op/s 2024-04-24T09:12:36.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:12:36 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:12:36.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:12:36 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:12:36.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:12:36 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:12:36.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:12:36 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:12:36.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:12:36 smithi138 ceph-mon[31441]: Adjusting osd_memory_target on smithi138 to 176.4M 2024-04-24T09:12:36.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:12:36 smithi138 ceph-mon[31441]: Unable to set osd_memory_target on smithi138 to 185047654: error parsing value: Value '185047654' is below minimum 939524096 2024-04-24T09:12:36.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:12:36 smithi189 ceph-mon[31791]: Adjusting osd_memory_target on smithi189 to 176.4M 2024-04-24T09:12:36.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:12:36 smithi189 ceph-mon[31791]: Unable to set osd_memory_target on smithi189 to 185064857: error parsing value: Value '185064857' is below minimum 939524096 2024-04-24T09:12:36.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:12:36 smithi189 ceph-mon[31791]: pgmap v662: 129 pgs: 129 active+clean; 844 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.0 MiB/s wr, 249 op/s 2024-04-24T09:12:36.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:12:36 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:12:36.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:12:36 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:12:36.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:12:36 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:12:36.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:12:36 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:12:36.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:12:36 smithi189 ceph-mon[31791]: Adjusting osd_memory_target on smithi138 to 176.4M 2024-04-24T09:12:36.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:12:36 smithi189 ceph-mon[31791]: Unable to set osd_memory_target on smithi138 to 185047654: error parsing value: Value '185047654' is below minimum 939524096 2024-04-24T09:12:36.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:12:36 smithi083 ceph-mon[27907]: Adjusting osd_memory_target on smithi189 to 176.4M 2024-04-24T09:12:36.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:12:36 smithi083 ceph-mon[27907]: Unable to set osd_memory_target on smithi189 to 185064857: error parsing value: Value '185064857' is below minimum 939524096 2024-04-24T09:12:36.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:12:36 smithi083 ceph-mon[27907]: pgmap v662: 129 pgs: 129 active+clean; 844 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.0 MiB/s wr, 249 op/s 2024-04-24T09:12:36.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:12:36 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:12:36.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:12:36 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:12:36.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:12:36 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:12:36.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:12:36 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:12:36.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:12:36 smithi083 ceph-mon[27907]: Adjusting osd_memory_target on smithi138 to 176.4M 2024-04-24T09:12:36.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:12:36 smithi083 ceph-mon[27907]: Unable to set osd_memory_target on smithi138 to 185047654: error parsing value: Value '185047654' is below minimum 939524096 2024-04-24T09:12:37.187 INFO:tasks.workunit.client.0.smithi083.stdout: 10 53379 6.64 MB/sec execute 308 sec latency 134.483 ms 2024-04-24T09:12:38.189 INFO:tasks.workunit.client.0.smithi083.stdout: 10 53496 6.63 MB/sec execute 309 sec latency 82.430 ms 2024-04-24T09:12:38.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:12:38 smithi138 ceph-mon[31441]: pgmap v663: 129 pgs: 129 active+clean; 844 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.0 MiB/s wr, 249 op/s 2024-04-24T09:12:38.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:12:38 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:12:38.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:12:38 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:12:38.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:12:38 smithi189 ceph-mon[31791]: pgmap v663: 129 pgs: 129 active+clean; 844 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.0 MiB/s wr, 249 op/s 2024-04-24T09:12:38.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:12:38 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:12:38.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:12:38 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:12:38.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:12:38 smithi083 ceph-mon[27907]: pgmap v663: 129 pgs: 129 active+clean; 844 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.0 MiB/s wr, 249 op/s 2024-04-24T09:12:38.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:12:38 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:12:38.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:12:38 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:12:39.189 INFO:tasks.workunit.client.0.smithi083.stdout: 10 53608 6.62 MB/sec execute 310 sec latency 78.926 ms 2024-04-24T09:12:39.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:12:39 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:12:39.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:12:39 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:12:39.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:12:39 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:12:39.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:12:39 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:12:39.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:12:39 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:12:39.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:12:39 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:12:39.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:12:39 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:12:39.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:12:39 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:12:39.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:12:39 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:12:39.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:12:39 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:12:39.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:12:39 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:12:39.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:12:39 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:12:39.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:12:39 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:12:39.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:12:39 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:12:39.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:12:39 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:12:39.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:12:39 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:12:39.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:12:39 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:12:39.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:12:39 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-24T09:12:39.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:12:39 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:12:39.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:12:39 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:12:39.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:12:39 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:12:40.189 INFO:tasks.workunit.client.0.smithi083.stdout: 10 53733 6.61 MB/sec execute 311 sec latency 91.796 ms 2024-04-24T09:12:40.356 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:12:40 smithi138 ceph-mon[31441]: Adjusting osd_memory_target on smithi083 to 176.4M 2024-04-24T09:12:40.357 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:12:40 smithi138 ceph-mon[31441]: Unable to set osd_memory_target on smithi083 to 185061990: error parsing value: Value '185061990' is below minimum 939524096 2024-04-24T09:12:40.357 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:12:40 smithi138 ceph-mon[31441]: pgmap v664: 129 pgs: 129 active+clean; 845 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.2 MiB/s wr, 351 op/s 2024-04-24T09:12:40.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:12:40 smithi189 ceph-mon[31791]: Adjusting osd_memory_target on smithi083 to 176.4M 2024-04-24T09:12:40.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:12:40 smithi189 ceph-mon[31791]: Unable to set osd_memory_target on smithi083 to 185061990: error parsing value: Value '185061990' is below minimum 939524096 2024-04-24T09:12:40.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:12:40 smithi189 ceph-mon[31791]: pgmap v664: 129 pgs: 129 active+clean; 845 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.2 MiB/s wr, 351 op/s 2024-04-24T09:12:40.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:12:40 smithi083 ceph-mon[27907]: Adjusting osd_memory_target on smithi083 to 176.4M 2024-04-24T09:12:40.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:12:40 smithi083 ceph-mon[27907]: Unable to set osd_memory_target on smithi083 to 185061990: error parsing value: Value '185061990' is below minimum 939524096 2024-04-24T09:12:40.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:12:40 smithi083 ceph-mon[27907]: pgmap v664: 129 pgs: 129 active+clean; 845 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.2 MiB/s wr, 351 op/s 2024-04-24T09:12:41.189 INFO:tasks.workunit.client.0.smithi083.stdout: 10 53858 6.61 MB/sec execute 312 sec latency 78.665 ms 2024-04-24T09:12:41.313 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.313+0000 7f6a57e00640 1 -- 172.21.15.83:0/3503731601 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a58108e40 msgr2=0x7f6a581092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:41.314 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.313+0000 7f6a57e00640 1 --2- 172.21.15.83:0/3503731601 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a58108e40 0x7f6a581092c0 secure :-1 s=READY pgs=2561 cs=0 l=1 rev1=1 crypto rx=0x7f6a44009900 tx=0x7f6a4402a3f0 comp rx=0 tx=0).stop 2024-04-24T09:12:41.314 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.313+0000 7f6a57e00640 1 -- 172.21.15.83:0/3503731601 shutdown_connections 2024-04-24T09:12:41.314 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.313+0000 7f6a57e00640 1 --2- 172.21.15.83:0/3503731601 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6a58109800 0x7f6a58115300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:41.314 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.313+0000 7f6a57e00640 1 --2- 172.21.15.83:0/3503731601 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a58108e40 0x7f6a581092c0 unknown :-1 s=CLOSED pgs=2561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:41.314 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.313+0000 7f6a57e00640 1 --2- 172.21.15.83:0/3503731601 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6a5810e4b0 0x7f6a5810e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:41.314 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.313+0000 7f6a57e00640 1 -- 172.21.15.83:0/3503731601 >> 172.21.15.83:0/3503731601 conn(0x7f6a58077a50 msgr2=0x7f6a58079e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:41.314 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.313+0000 7f6a57e00640 1 -- 172.21.15.83:0/3503731601 shutdown_connections 2024-04-24T09:12:41.314 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.313+0000 7f6a57e00640 1 -- 172.21.15.83:0/3503731601 wait complete. 2024-04-24T09:12:41.315 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.314+0000 7f6a57e00640 1 Processor -- start 2024-04-24T09:12:41.315 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.314+0000 7f6a57e00640 1 -- start start 2024-04-24T09:12:41.315 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.314+0000 7f6a57e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6a58108e40 0x7f6a581a47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:41.316 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.314+0000 7f6a57e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6a58109800 0x7f6a581a4d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:41.316 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.314+0000 7f6a57e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a5810e4b0 0x7f6a5819e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:41.316 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.314+0000 7f6a57e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6a58118700 con 0x7f6a5810e4b0 2024-04-24T09:12:41.316 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.314+0000 7f6a57e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6a58118580 con 0x7f6a58108e40 2024-04-24T09:12:41.316 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.314+0000 7f6a57e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6a58118880 con 0x7f6a58109800 2024-04-24T09:12:41.316 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.315+0000 7f6a57400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a5810e4b0 0x7f6a5819e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:41.316 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.315+0000 7f6a57400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a5810e4b0 0x7f6a5819e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43904/0 (socket says 172.21.15.83:43904) 2024-04-24T09:12:41.316 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.315+0000 7f6a57400640 1 -- 172.21.15.83:0/1985230041 learned_addr learned my addr 172.21.15.83:0/1985230041 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:12:41.316 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.315+0000 7f6a56a00640 1 --2- 172.21.15.83:0/1985230041 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6a58108e40 0x7f6a581a47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:41.316 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.315+0000 7f6a56000640 1 --2- 172.21.15.83:0/1985230041 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6a58109800 0x7f6a581a4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:41.317 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.315+0000 7f6a57400640 1 -- 172.21.15.83:0/1985230041 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6a58109800 msgr2=0x7f6a581a4d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:41.317 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.315+0000 7f6a57400640 1 --2- 172.21.15.83:0/1985230041 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6a58109800 0x7f6a581a4d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:41.317 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.315+0000 7f6a57400640 1 -- 172.21.15.83:0/1985230041 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6a58108e40 msgr2=0x7f6a581a47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:41.317 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.315+0000 7f6a57400640 1 --2- 172.21.15.83:0/1985230041 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6a58108e40 0x7f6a581a47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:41.317 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.315+0000 7f6a57400640 1 -- 172.21.15.83:0/1985230041 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a5819f100 con 0x7f6a5810e4b0 2024-04-24T09:12:41.317 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.316+0000 7f6a57400640 1 --2- 172.21.15.83:0/1985230041 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a5810e4b0 0x7f6a5819e980 secure :-1 s=READY pgs=2562 cs=0 l=1 rev1=1 crypto rx=0x7f6a50009fd0 tx=0x7f6a5000def0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:41.319 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.318+0000 7f6a4f400640 1 -- 172.21.15.83:0/1985230041 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a50017070 con 0x7f6a5810e4b0 2024-04-24T09:12:41.319 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.318+0000 7f6a4f400640 1 -- 172.21.15.83:0/1985230041 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6a50003e20 con 0x7f6a5810e4b0 2024-04-24T09:12:41.319 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.318+0000 7f6a4f400640 1 -- 172.21.15.83:0/1985230041 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a50010460 con 0x7f6a5810e4b0 2024-04-24T09:12:41.319 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.318+0000 7f6a57e00640 1 -- 172.21.15.83:0/1985230041 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a5819f3f0 con 0x7f6a5810e4b0 2024-04-24T09:12:41.319 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.318+0000 7f6a57e00640 1 -- 172.21.15.83:0/1985230041 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a581ab690 con 0x7f6a5810e4b0 2024-04-24T09:12:41.323 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.322+0000 7f6a60601740 1 Processor -- start 2024-04-24T09:12:41.323 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.323+0000 7f6a4f400640 1 -- 172.21.15.83:0/1985230041 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6a50003970 con 0x7f6a5810e4b0 2024-04-24T09:12:41.324 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.324+0000 7f6a60601740 1 -- start start 2024-04-24T09:12:41.324 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.324+0000 7f6a60601740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d765ca93c0 0x55d765ca97a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:41.324 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.324+0000 7f6a60601740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d765ca9ce0 0x55d765cb4060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:41.324 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.324+0000 7f6a60601740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d765cb45a0 0x55d765cb6990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:41.324 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.324+0000 7f6a60601740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d765c0f900 con 0x55d765ca9ce0 2024-04-24T09:12:41.324 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.324+0000 7f6a60601740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d765bb5f20 con 0x55d765cb45a0 2024-04-24T09:12:41.324 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.324+0000 7f6a60601740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d765c0b8c0 con 0x55d765ca93c0 2024-04-24T09:12:41.325 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.324+0000 7f6a56000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d765cb45a0 0x55d765cb6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:41.325 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.325+0000 7f6a56a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d765ca9ce0 0x55d765cb4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:41.325 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.325+0000 7f6a56a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d765ca9ce0 0x55d765cb4060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43920/0 (socket says 172.21.15.83:43920) 2024-04-24T09:12:41.325 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.325+0000 7f6a56a00640 1 -- 172.21.15.83:0/29979026 learned_addr learned my addr 172.21.15.83:0/29979026 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:12:41.325 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.325+0000 7f6a56000640 1 --2- 172.21.15.83:0/29979026 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d765ca93c0 0x55d765ca97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:41.326 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.325+0000 7f6a56a00640 1 -- 172.21.15.83:0/29979026 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d765ca93c0 msgr2=0x55d765ca97a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:41.326 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.325+0000 7f6a56a00640 1 --2- 172.21.15.83:0/29979026 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d765ca93c0 0x55d765ca97a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:41.326 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.325+0000 7f6a56a00640 1 -- 172.21.15.83:0/29979026 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d765cb45a0 msgr2=0x55d765cb6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:41.326 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.325+0000 7f6a56a00640 1 --2- 172.21.15.83:0/29979026 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d765cb45a0 0x55d765cb6990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:41.326 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.325+0000 7f6a56a00640 1 -- 172.21.15.83:0/29979026 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d765bdae10 con 0x55d765ca9ce0 2024-04-24T09:12:41.326 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.325+0000 7f6a56000640 1 --2- 172.21.15.83:0/29979026 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d765ca93c0 0x55d765ca97a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:12:41.327 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.325+0000 7f6a56a00640 1 --2- 172.21.15.83:0/29979026 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d765ca9ce0 0x55d765cb4060 secure :-1 s=READY pgs=2563 cs=0 l=1 rev1=1 crypto rx=0x7f6a4800ac00 tx=0x7f6a4800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2dc5c6df702cd0c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:41.327 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.326+0000 7f6a43e00640 1 -- 172.21.15.83:0/29979026 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a4801d070 con 0x55d765ca9ce0 2024-04-24T09:12:41.327 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.326+0000 7f6a4f400640 1 --2- 172.21.15.83:0/1985230041 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f6a3c0a3ea0 0x7f6a3c0a6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:41.327 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.326+0000 7f6a43e00640 1 -- 172.21.15.83:0/29979026 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6a48018af0 con 0x55d765ca9ce0 2024-04-24T09:12:41.327 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.326+0000 7f6a43e00640 1 -- 172.21.15.83:0/29979026 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a48010a90 con 0x55d765ca9ce0 2024-04-24T09:12:41.327 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.326+0000 7f6a4f400640 1 -- 172.21.15.83:0/1985230041 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 9583+0+0 (secure 0 0 0) 0x7f6a500d0890 con 0x7f6a5810e4b0 2024-04-24T09:12:41.327 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.326+0000 7f6a56000640 1 --2- 172.21.15.83:0/1985230041 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f6a3c0a3ea0 0x7f6a3c0a6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:41.327 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.327+0000 7f6a60601740 1 -- 172.21.15.83:0/29979026 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d765ca9ce0 msgr2=0x55d765cb4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:41.327 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.327+0000 7f6a60601740 1 --2- 172.21.15.83:0/29979026 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d765ca9ce0 0x55d765cb4060 secure :-1 s=READY pgs=2563 cs=0 l=1 rev1=1 crypto rx=0x7f6a4800ac00 tx=0x7f6a4800f790 comp rx=0 tx=0).stop 2024-04-24T09:12:41.328 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.327+0000 7f6a60601740 1 -- 172.21.15.83:0/29979026 shutdown_connections 2024-04-24T09:12:41.328 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.327+0000 7f6a60601740 1 --2- 172.21.15.83:0/29979026 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d765cb45a0 0x55d765cb6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:41.328 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.327+0000 7f6a60601740 1 --2- 172.21.15.83:0/29979026 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d765ca9ce0 0x55d765cb4060 unknown :-1 s=CLOSED pgs=2563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:41.328 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.327+0000 7f6a60601740 1 --2- 172.21.15.83:0/29979026 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d765ca93c0 0x55d765ca97a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:41.329 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.327+0000 7f6a60601740 1 -- 172.21.15.83:0/29979026 >> 172.21.15.83:0/29979026 conn(0x55d765ca58d0 msgr2=0x55d765ca5cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:41.329 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.327+0000 7f6a60601740 1 -- 172.21.15.83:0/29979026 shutdown_connections 2024-04-24T09:12:41.329 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.327+0000 7f6a60601740 1 -- 172.21.15.83:0/29979026 wait complete. 2024-04-24T09:12:41.329 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.328+0000 7f6a56000640 1 --2- 172.21.15.83:0/1985230041 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f6a3c0a3ea0 0x7f6a3c0a6360 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f6a44000c00 tx=0x7f6a44009260 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:41.329 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.328+0000 7f6a60601740 1 -- start start 2024-04-24T09:12:41.329 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.328+0000 7f6a60601740 1 Processor -- start 2024-04-24T09:12:41.330 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.329+0000 7f6a60601740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d765ca93c0 0x55d765cb1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:41.330 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.329+0000 7f6a60601740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d765cb45a0 0x55d765cb1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:41.330 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.329+0000 7f6a60601740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d765cb2210 0x55d765cc06c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:41.330 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.329+0000 7f6a60601740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d765cb7140 con 0x55d765ca93c0 2024-04-24T09:12:41.330 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.329+0000 7f6a60601740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d765bc92c0 con 0x55d765cb2210 2024-04-24T09:12:41.330 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.329+0000 7f6a60601740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d765cb72c0 con 0x55d765cb45a0 2024-04-24T09:12:41.330 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.329+0000 7f6a60601740 10 client.0 _ll_register_callbacks cb 0x55d765921a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:12:41.330 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.329+0000 7f6a56a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d765ca93c0 0x55d765cb1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:41.330 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.329+0000 7f6a56a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d765ca93c0 0x55d765cb1790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43922/0 (socket says 172.21.15.83:43922) 2024-04-24T09:12:41.331 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.329+0000 7f6a56a00640 1 -- 172.21.15.83:0/3815979864 learned_addr learned my addr 172.21.15.83:0/3815979864 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:12:41.331 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.330+0000 7f6a56a00640 1 -- 172.21.15.83:0/3815979864 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d765cb45a0 msgr2=0x55d765cb1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:41.331 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.330+0000 7f6a56a00640 1 --2- 172.21.15.83:0/3815979864 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d765cb45a0 0x55d765cb1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:41.331 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.330+0000 7f6a56a00640 1 -- 172.21.15.83:0/3815979864 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d765cb2210 msgr2=0x55d765cc06c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-24T09:12:41.331 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.330+0000 7f6a57400640 1 --2- 172.21.15.83:0/3815979864 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d765cb2210 0x55d765cc06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:41.331 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.330+0000 7f6a56a00640 1 --2- 172.21.15.83:0/3815979864 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d765cb2210 0x55d765cc06c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:41.331 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.330+0000 7f6a56a00640 1 -- 172.21.15.83:0/3815979864 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d765bd9fd0 con 0x55d765ca93c0 2024-04-24T09:12:41.332 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.330+0000 7f6a56a00640 1 --2- 172.21.15.83:0/3815979864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d765ca93c0 0x55d765cb1790 secure :-1 s=READY pgs=2564 cs=0 l=1 rev1=1 crypto rx=0x7f6a4800a5c0 tx=0x7f6a48018670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfda7a6441d7af server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:41.332 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.330+0000 7f6a40c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:12:41.332 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.331+0000 7f6a40c00640 1 -- 172.21.15.83:0/3815979864 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a48027020 con 0x55d765ca93c0 2024-04-24T09:12:41.332 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.331+0000 7f6a40c00640 1 -- 172.21.15.83:0/3815979864 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9583+0+0 (secure 0 0 0) 0x7f6a48005240 con 0x55d765ca93c0 2024-04-24T09:12:41.332 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.331+0000 7f6a60601740 1 -- 172.21.15.83:0/3815979864 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a48018af0 con 0x55d765ca93c0 2024-04-24T09:12:41.333 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.331+0000 7f6a40c00640 1 -- 172.21.15.83:0/3815979864 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f6a4801d070 con 0x55d765ca93c0 2024-04-24T09:12:41.333 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.331+0000 7f6a60601740 10 client.22785 fetch_fsmap learned FSMap version 64 2024-04-24T09:12:41.333 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.331+0000 7f6a60601740 1 -- 172.21.15.83:0/3815979864 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55d765922210 con 0x55d765ca93c0 2024-04-24T09:12:41.333 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.332+0000 7f6a40c00640 1 -- 172.21.15.83:0/3815979864 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6a48018af0 con 0x55d765ca93c0 2024-04-24T09:12:41.333 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.332+0000 7f6a40c00640 1 -- 172.21.15.83:0/3815979864 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 64) ==== 8374+0+0 (secure 0 0 0) 0x7f6a48010430 con 0x55d765ca93c0 2024-04-24T09:12:41.333 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.332+0000 7f6a60601740 10 client.22785 fetch_fsmap finished waiting for FSMap version 64 2024-04-24T09:12:41.333 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.332+0000 7f6a60601740 10 client.22785 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:12:41.333 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.332+0000 7f6a60601740 20 client.22785 populate_metadata read hostname 'smithi083' 2024-04-24T09:12:41.333 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.332+0000 7f6a60601740 1 --2- 172.21.15.83:0/3815979864 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d765cc4be0 0x55d765cc6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:41.333 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.332+0000 7f6a60601740 4 client.22785 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:12:41.333 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.332+0000 7f6a60601740 1 -- 172.21.15.83:0/3815979864 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d765c1d320 con 0x55d765cc4be0 2024-04-24T09:12:41.334 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.332+0000 7f6a56000640 1 --2- 172.21.15.83:0/3815979864 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d765cc4be0 0x55d765cc6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:41.334 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.333+0000 7f6a56000640 1 --2- 172.21.15.83:0/3815979864 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d765cc4be0 0x55d765cc6fd0 crc :-1 s=READY pgs=519 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9a8d634506e386e server_cookie=9d2b561e1b9aed14 in_seq=0 out_seq=0 2024-04-24T09:12:41.334 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.333+0000 7f6a40c00640 10 client.22785 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:12:41.335 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.335+0000 7f6a40c00640 1 -- 172.21.15.83:0/3815979864 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d765c1d320 con 0x55d765cc4be0 2024-04-24T09:12:41.335 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.335+0000 7f6a40c00640 10 client.22785 handle_command_reply: tid=0 2024-04-24T09:12:41.335 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.335+0000 7f6a60601740 1 client.22785 shutdown 2024-04-24T09:12:41.336 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.335+0000 7f6a60601740 1 -- 172.21.15.83:0/3815979864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d765ca93c0 msgr2=0x55d765cb1790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:41.336 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.335+0000 7f6a60601740 1 --2- 172.21.15.83:0/3815979864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d765ca93c0 0x55d765cb1790 secure :-1 s=READY pgs=2564 cs=0 l=1 rev1=1 crypto rx=0x7f6a4800a5c0 tx=0x7f6a48018670 comp rx=0 tx=0).stop 2024-04-24T09:12:41.336 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.335+0000 7f6a60601740 1 -- 172.21.15.83:0/3815979864 shutdown_connections 2024-04-24T09:12:41.336 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.335+0000 7f6a60601740 1 --2- 172.21.15.83:0/3815979864 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d765cc4be0 0x55d765cc6fd0 crc :-1 s=READY pgs=519 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:41.336 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.335+0000 7f6a60601740 1 --2- 172.21.15.83:0/3815979864 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d765cb2210 0x55d765cc06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:41.336 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.335+0000 7f6a60601740 1 --2- 172.21.15.83:0/3815979864 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d765cb45a0 0x55d765cb1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:41.336 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.335+0000 7f6a60601740 1 --2- 172.21.15.83:0/3815979864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d765ca93c0 0x55d765cb1790 unknown :-1 s=CLOSED pgs=2564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:41.336 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.335+0000 7f6a60601740 1 -- 172.21.15.83:0/3815979864 >> 172.21.15.83:0/3815979864 conn(0x55d765ca58d0 msgr2=0x55d765cab600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:41.336 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.335+0000 7f6a40c00640 0 client.22785 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:12:41.336 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.335+0000 7f6a60601740 1 -- 172.21.15.83:0/3815979864 shutdown_connections 2024-04-24T09:12:41.337 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.336+0000 7f6a60601740 1 -- 172.21.15.83:0/3815979864 wait complete. 2024-04-24T09:12:41.337 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.336+0000 7f6a60601740 20 client.22785 trim_cache size 0 max 16384 2024-04-24T09:12:41.363 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.362+0000 7f6a60601740 1 Processor -- start 2024-04-24T09:12:41.363 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.362+0000 7f6a60601740 1 -- start start 2024-04-24T09:12:41.363 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.362+0000 7f6a60601740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d765ca93c0 0x55d765cabd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:41.363 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.362+0000 7f6a60601740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d765cb2210 0x55d765ccc7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:41.363 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.362+0000 7f6a60601740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d765cb45a0 0x55d765cccd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:41.363 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.362+0000 7f6a60601740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6a581b59c0 con 0x55d765cb45a0 2024-04-24T09:12:41.363 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.362+0000 7f6a60601740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6a581b7590 con 0x55d765cb2210 2024-04-24T09:12:41.363 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.362+0000 7f6a60601740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6a581b6090 con 0x55d765ca93c0 2024-04-24T09:12:41.364 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.363+0000 7f6a57400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d765cb45a0 0x55d765cccd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:41.364 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.363+0000 7f6a57400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d765cb45a0 0x55d765cccd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43934/0 (socket says 172.21.15.83:43934) 2024-04-24T09:12:41.364 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.363+0000 7f6a57400640 1 -- 172.21.15.83:0/2777553738 learned_addr learned my addr 172.21.15.83:0/2777553738 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:12:41.364 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.363+0000 7f6a56000640 1 --2- 172.21.15.83:0/2777553738 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d765cb2210 0x55d765ccc7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:41.364 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.363+0000 7f6a56a00640 1 --2- 172.21.15.83:0/2777553738 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d765ca93c0 0x55d765cabd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:41.364 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.363+0000 7f6a57400640 1 -- 172.21.15.83:0/2777553738 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d765ca93c0 msgr2=0x55d765cabd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:41.365 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.363+0000 7f6a57400640 1 --2- 172.21.15.83:0/2777553738 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d765ca93c0 0x55d765cabd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:41.365 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.363+0000 7f6a57400640 1 -- 172.21.15.83:0/2777553738 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d765cb2210 msgr2=0x55d765ccc7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:41.365 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.363+0000 7f6a57400640 1 --2- 172.21.15.83:0/2777553738 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d765cb2210 0x55d765ccc7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:41.365 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.363+0000 7f6a57400640 1 -- 172.21.15.83:0/2777553738 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7659ee620 con 0x55d765cb45a0 2024-04-24T09:12:41.365 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.364+0000 7f6a57400640 1 --2- 172.21.15.83:0/2777553738 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d765cb45a0 0x55d765cccd30 secure :-1 s=READY pgs=2565 cs=0 l=1 rev1=1 crypto rx=0x7f6a5009c3a0 tx=0x7f6a500987c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15f62a64d0da4860 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:41.366 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.364+0000 7f6a43400640 1 -- 172.21.15.83:0/2777553738 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a50020040 con 0x55d765cb45a0 2024-04-24T09:12:41.366 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.364+0000 7f6a43400640 1 -- 172.21.15.83:0/2777553738 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6a5009c410 con 0x55d765cb45a0 2024-04-24T09:12:41.366 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.364+0000 7f6a43400640 1 -- 172.21.15.83:0/2777553738 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a50098a80 con 0x55d765cb45a0 2024-04-24T09:12:41.366 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.366+0000 7f6a60601740 1 -- 172.21.15.83:0/2777553738 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d765cb45a0 msgr2=0x55d765cccd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:41.366 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.366+0000 7f6a60601740 1 --2- 172.21.15.83:0/2777553738 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d765cb45a0 0x55d765cccd30 secure :-1 s=READY pgs=2565 cs=0 l=1 rev1=1 crypto rx=0x7f6a5009c3a0 tx=0x7f6a500987c0 comp rx=0 tx=0).stop 2024-04-24T09:12:41.366 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.366+0000 7f6a60601740 1 -- 172.21.15.83:0/2777553738 shutdown_connections 2024-04-24T09:12:41.366 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.366+0000 7f6a60601740 1 --2- 172.21.15.83:0/2777553738 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d765cb45a0 0x55d765cccd30 unknown :-1 s=CLOSED pgs=2565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:41.366 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.366+0000 7f6a60601740 1 --2- 172.21.15.83:0/2777553738 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d765cb2210 0x55d765ccc7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:41.366 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.366+0000 7f6a60601740 1 --2- 172.21.15.83:0/2777553738 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d765ca93c0 0x55d765cabd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:41.366 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.366+0000 7f6a60601740 1 -- 172.21.15.83:0/2777553738 >> 172.21.15.83:0/2777553738 conn(0x55d765ca58d0 msgr2=0x55d765cb0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:41.367 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.366+0000 7f6a60601740 1 -- 172.21.15.83:0/2777553738 shutdown_connections 2024-04-24T09:12:41.367 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.366+0000 7f6a60601740 1 -- 172.21.15.83:0/2777553738 wait complete. 2024-04-24T09:12:41.367 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.366+0000 7f6a60601740 1 -- start start 2024-04-24T09:12:41.367 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.367+0000 7f6a60601740 1 Processor -- start 2024-04-24T09:12:41.368 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.367+0000 7f6a60601740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d765ca93c0 0x55d765cbc620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:41.368 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.367+0000 7f6a60601740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d765cb2210 0x55d765cbcb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:41.368 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.367+0000 7f6a60601740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d765cb45a0 0x55d765cada60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:41.368 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.367+0000 7f6a60601740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6a581b6e90 con 0x55d765cb2210 2024-04-24T09:12:41.368 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.367+0000 7f6a60601740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6a581b67c0 con 0x55d765ca93c0 2024-04-24T09:12:41.369 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.367+0000 7f6a60601740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6a581b39c0 con 0x55d765cb45a0 2024-04-24T09:12:41.369 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.368+0000 7f6a56000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d765cb2210 0x55d765cbcb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:41.369 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.367+0000 7f6a60601740 10 client.0 _ll_register_callbacks cb 0x7f6a4801b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:12:41.369 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.368+0000 7f6a56000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d765cb2210 0x55d765cbcb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43950/0 (socket says 172.21.15.83:43950) 2024-04-24T09:12:41.369 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.368+0000 7f6a56000640 1 -- 172.21.15.83:0/1678106160 learned_addr learned my addr 172.21.15.83:0/1678106160 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:12:41.369 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.368+0000 7f6a56a00640 1 --2- 172.21.15.83:0/1678106160 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d765ca93c0 0x55d765cbc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:41.369 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.368+0000 7f6a57400640 1 --2- 172.21.15.83:0/1678106160 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d765cb45a0 0x55d765cada60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:41.370 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.368+0000 7f6a56000640 1 -- 172.21.15.83:0/1678106160 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d765cb45a0 msgr2=0x55d765cada60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:41.370 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.368+0000 7f6a56000640 1 --2- 172.21.15.83:0/1678106160 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d765cb45a0 0x55d765cada60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:41.370 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.368+0000 7f6a56000640 1 -- 172.21.15.83:0/1678106160 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d765ca93c0 msgr2=0x55d765cbc620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:41.370 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.369+0000 7f6a56000640 1 --2- 172.21.15.83:0/1678106160 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d765ca93c0 0x55d765cbc620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:41.370 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.369+0000 7f6a56000640 1 -- 172.21.15.83:0/1678106160 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d765cae0d0 con 0x55d765cb2210 2024-04-24T09:12:41.370 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.369+0000 7f6a56a00640 1 --2- 172.21.15.83:0/1678106160 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d765ca93c0 0x55d765cbc620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:12:41.370 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.370+0000 7f6a56000640 1 --2- 172.21.15.83:0/1678106160 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d765cb2210 0x55d765cbcb60 secure :-1 s=READY pgs=2566 cs=0 l=1 rev1=1 crypto rx=0x7f6a44009230 tx=0x7f6a44005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab3efb84baec3ea5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:41.370 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.370+0000 7f6a40c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:12:41.371 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.370+0000 7f6a40c00640 1 -- 172.21.15.83:0/1678106160 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a44035040 con 0x55d765cb2210 2024-04-24T09:12:41.371 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.370+0000 7f6a60601740 1 -- 172.21.15.83:0/1678106160 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a5009c410 con 0x55d765cb2210 2024-04-24T09:12:41.371 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.370+0000 7f6a40c00640 1 -- 172.21.15.83:0/1678106160 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9583+0+0 (secure 0 0 0) 0x7f6a44003d90 con 0x55d765cb2210 2024-04-24T09:12:41.371 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.370+0000 7f6a40c00640 1 -- 172.21.15.83:0/1678106160 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f6a44061050 con 0x55d765cb2210 2024-04-24T09:12:41.371 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.370+0000 7f6a60601740 10 client.22791 fetch_fsmap learned FSMap version 64 2024-04-24T09:12:41.371 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.370+0000 7f6a60601740 1 -- 172.21.15.83:0/1678106160 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55d765d3ba30 con 0x55d765cb2210 2024-04-24T09:12:41.371 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.371+0000 7f6a40c00640 1 -- 172.21.15.83:0/1678106160 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6a5009c410 con 0x55d765cb2210 2024-04-24T09:12:41.372 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.372+0000 7f6a40c00640 1 -- 172.21.15.83:0/1678106160 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 64) ==== 8374+0+0 (secure 0 0 0) 0x7f6a4403a040 con 0x55d765cb2210 2024-04-24T09:12:41.372 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.372+0000 7f6a60601740 10 client.22791 fetch_fsmap finished waiting for FSMap version 64 2024-04-24T09:12:41.372 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.372+0000 7f6a60601740 10 client.22791 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:12:41.372 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.372+0000 7f6a60601740 20 client.22791 populate_metadata read hostname 'smithi083' 2024-04-24T09:12:41.372 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.372+0000 7f6a60601740 1 --2- 172.21.15.83:0/1678106160 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d765d3c1e0 0x55d765d3e600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:41.372 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.372+0000 7f6a60601740 4 client.22791 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T09:12:41.373 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.372+0000 7f6a60601740 1 -- 172.21.15.83:0/1678106160 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d765c17190 con 0x55d765d3c1e0 2024-04-24T09:12:41.373 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.372+0000 7f6a56a00640 1 --2- 172.21.15.83:0/1678106160 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d765d3c1e0 0x55d765d3e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:41.373 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.373+0000 7f6a56a00640 1 --2- 172.21.15.83:0/1678106160 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d765d3c1e0 0x55d765d3e600 crc :-1 s=READY pgs=520 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d41fd21a99b9b53c server_cookie=686663cd12ac7397 in_seq=0 out_seq=0 2024-04-24T09:12:41.373 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.373+0000 7f6a40c00640 10 client.22791 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:12:41.518 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.518+0000 7f6a40c00640 1 -- 172.21.15.83:0/1678106160 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d765c17190 con 0x55d765d3c1e0 2024-04-24T09:12:41.519 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.518+0000 7f6a40c00640 10 client.22791 handle_command_reply: tid=0 2024-04-24T09:12:41.519 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.518+0000 7f6a60601740 1 client.22791 shutdown 2024-04-24T09:12:41.520 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.518+0000 7f6a60601740 1 -- 172.21.15.83:0/1678106160 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d765cb2210 msgr2=0x55d765cbcb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:41.521 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.518+0000 7f6a60601740 1 --2- 172.21.15.83:0/1678106160 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d765cb2210 0x55d765cbcb60 secure :-1 s=READY pgs=2566 cs=0 l=1 rev1=1 crypto rx=0x7f6a44009230 tx=0x7f6a44005a20 comp rx=0 tx=0).stop 2024-04-24T09:12:41.521 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.518+0000 7f6a60601740 1 -- 172.21.15.83:0/1678106160 shutdown_connections 2024-04-24T09:12:41.521 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.518+0000 7f6a60601740 1 --2- 172.21.15.83:0/1678106160 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d765d3c1e0 0x55d765d3e600 crc :-1 s=READY pgs=520 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:41.521 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.518+0000 7f6a60601740 1 --2- 172.21.15.83:0/1678106160 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d765cb45a0 0x55d765cada60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:41.521 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.518+0000 7f6a60601740 1 --2- 172.21.15.83:0/1678106160 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d765cb2210 0x55d765cbcb60 unknown :-1 s=CLOSED pgs=2566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:41.521 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.518+0000 7f6a60601740 1 --2- 172.21.15.83:0/1678106160 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d765ca93c0 0x55d765cbc620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:41.522 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.518+0000 7f6a60601740 1 -- 172.21.15.83:0/1678106160 >> 172.21.15.83:0/1678106160 conn(0x55d765ca58d0 msgr2=0x55d765ca4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:41.522 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.518+0000 7f6a40c00640 0 client.22791 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:12:41.522 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.520+0000 7f6a60601740 1 -- 172.21.15.83:0/1678106160 shutdown_connections 2024-04-24T09:12:41.522 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.520+0000 7f6a60601740 1 -- 172.21.15.83:0/1678106160 wait complete. 2024-04-24T09:12:41.523 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.520+0000 7f6a60601740 20 client.22791 trim_cache size 0 max 16384 2024-04-24T09:12:41.523 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T09:12:41.524 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T09:12:41.524 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T09:12:41.524 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T09:12:41.525 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.523+0000 7f6a57e00640 1 -- 172.21.15.83:0/1985230041 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f6a3c0a3ea0 msgr2=0x7f6a3c0a6360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:41.525 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.523+0000 7f6a57e00640 1 --2- 172.21.15.83:0/1985230041 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f6a3c0a3ea0 0x7f6a3c0a6360 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f6a44000c00 tx=0x7f6a44009260 comp rx=0 tx=0).stop 2024-04-24T09:12:41.526 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.523+0000 7f6a57e00640 1 -- 172.21.15.83:0/1985230041 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a5810e4b0 msgr2=0x7f6a5819e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:41.526 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.523+0000 7f6a57e00640 1 --2- 172.21.15.83:0/1985230041 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a5810e4b0 0x7f6a5819e980 secure :-1 s=READY pgs=2562 cs=0 l=1 rev1=1 crypto rx=0x7f6a50009fd0 tx=0x7f6a5000def0 comp rx=0 tx=0).stop 2024-04-24T09:12:41.526 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.524+0000 7f6a57e00640 1 -- 172.21.15.83:0/1985230041 shutdown_connections 2024-04-24T09:12:41.526 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.524+0000 7f6a57e00640 1 --2- 172.21.15.83:0/1985230041 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f6a3c0a3ea0 0x7f6a3c0a6360 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:41.526 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.524+0000 7f6a57e00640 1 --2- 172.21.15.83:0/1985230041 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a5810e4b0 0x7f6a5819e980 unknown :-1 s=CLOSED pgs=2562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:41.526 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.524+0000 7f6a57e00640 1 --2- 172.21.15.83:0/1985230041 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6a58109800 0x7f6a581a4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:41.527 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.524+0000 7f6a57e00640 1 --2- 172.21.15.83:0/1985230041 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6a58108e40 0x7f6a581a47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:41.527 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.524+0000 7f6a57e00640 1 -- 172.21.15.83:0/1985230041 >> 172.21.15.83:0/1985230041 conn(0x7f6a58077a50 msgr2=0x7f6a5807a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:41.527 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.524+0000 7f6a57e00640 1 -- 172.21.15.83:0/1985230041 shutdown_connections 2024-04-24T09:12:41.527 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:41.524+0000 7f6a57e00640 1 -- 172.21.15.83:0/1985230041 wait complete. 2024-04-24T09:12:42.189 INFO:tasks.workunit.client.0.smithi083.stdout: 10 53976 6.62 MB/sec execute 313 sec latency 192.030 ms 2024-04-24T09:12:42.298 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T09:12:42.298 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T09:12:42.377 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:12:42 smithi083 ceph-mon[27907]: pgmap v665: 129 pgs: 129 active+clean; 845 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.2 MiB/s wr, 249 op/s 2024-04-24T09:12:42.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:12:42 smithi138 ceph-mon[31441]: pgmap v665: 129 pgs: 129 active+clean; 845 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.2 MiB/s wr, 249 op/s 2024-04-24T09:12:42.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:12:42 smithi189 ceph-mon[31791]: pgmap v665: 129 pgs: 129 active+clean; 845 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.2 MiB/s wr, 249 op/s 2024-04-24T09:12:42.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.726+0000 7f68c3e00640 1 -- 172.21.15.83:0/1928875897 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68c4103060 msgr2=0x7f68c4103440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:42.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.726+0000 7f68c3e00640 1 --2- 172.21.15.83:0/1928875897 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68c4103060 0x7f68c4103440 secure :-1 s=READY pgs=2567 cs=0 l=1 rev1=1 crypto rx=0x7f68ac009900 tx=0x7f68ac02a490 comp rx=0 tx=0).stop 2024-04-24T09:12:42.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.726+0000 7f68c3e00640 1 -- 172.21.15.83:0/1928875897 shutdown_connections 2024-04-24T09:12:42.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.726+0000 7f68c3e00640 1 --2- 172.21.15.83:0/1928875897 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f68c410c480 0x7f68c410e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:42.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.726+0000 7f68c3e00640 1 --2- 172.21.15.83:0/1928875897 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68c4103980 0x7f68c410be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:42.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.726+0000 7f68c3e00640 1 --2- 172.21.15.83:0/1928875897 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68c4103060 0x7f68c4103440 unknown :-1 s=CLOSED pgs=2567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:42.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.726+0000 7f68c3e00640 1 -- 172.21.15.83:0/1928875897 >> 172.21.15.83:0/1928875897 conn(0x7f68c40fc880 msgr2=0x7f68c40feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:42.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.727+0000 7f68c3e00640 1 -- 172.21.15.83:0/1928875897 shutdown_connections 2024-04-24T09:12:42.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.727+0000 7f68c3e00640 1 -- 172.21.15.83:0/1928875897 wait complete. 2024-04-24T09:12:42.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.729+0000 7f68c3e00640 1 Processor -- start 2024-04-24T09:12:42.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.729+0000 7f68c3e00640 1 -- start start 2024-04-24T09:12:42.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.730+0000 7f68c3e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68c4103060 0x7f68c4100cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:42.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.730+0000 7f68c3e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f68c4103980 0x7f68c41011f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:42.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.730+0000 7f68c3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68c410c480 0x7f68c4101730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:42.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.730+0000 7f68c3e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f68c4112140 con 0x7f68c410c480 2024-04-24T09:12:42.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.730+0000 7f68c3e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f68c4111fc0 con 0x7f68c4103060 2024-04-24T09:12:42.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.730+0000 7f68c3e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f68c41122c0 con 0x7f68c4103980 2024-04-24T09:12:42.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.730+0000 7f68c2a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68c4103060 0x7f68c4100cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:42.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.730+0000 7f68c2a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68c4103060 0x7f68c4100cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:49748/0 (socket says 172.21.15.83:49748) 2024-04-24T09:12:42.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.730+0000 7f68c2a00640 1 -- 172.21.15.83:0/628866746 learned_addr learned my addr 172.21.15.83:0/628866746 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:12:42.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.730+0000 7f68ba000640 1 --2- 172.21.15.83:0/628866746 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f68c4103980 0x7f68c41011f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:42.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.730+0000 7f68c3400640 1 --2- 172.21.15.83:0/628866746 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68c410c480 0x7f68c4101730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:42.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.731+0000 7f68c2a00640 1 -- 172.21.15.83:0/628866746 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f68c4103980 msgr2=0x7f68c41011f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:42.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.731+0000 7f68c2a00640 1 --2- 172.21.15.83:0/628866746 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f68c4103980 0x7f68c41011f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:42.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.731+0000 7f68c2a00640 1 -- 172.21.15.83:0/628866746 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68c410c480 msgr2=0x7f68c4101730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:42.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.731+0000 7f68c2a00640 1 --2- 172.21.15.83:0/628866746 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68c410c480 0x7f68c4101730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:42.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.731+0000 7f68c2a00640 1 -- 172.21.15.83:0/628866746 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68c41a6b20 con 0x7f68c4103060 2024-04-24T09:12:42.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.731+0000 7f68c2a00640 1 --2- 172.21.15.83:0/628866746 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68c4103060 0x7f68c4100cb0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f68ac005ab0 tx=0x7f68ac003cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:42.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.731+0000 7f68c8a00640 1 -- 172.21.15.83:0/628866746 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68ac03f070 con 0x7f68c4103060 2024-04-24T09:12:42.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.732+0000 7f68c8a00640 1 -- 172.21.15.83:0/628866746 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f68ac031d60 con 0x7f68c4103060 2024-04-24T09:12:42.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.732+0000 7f68c8a00640 1 -- 172.21.15.83:0/628866746 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68ac035030 con 0x7f68c4103060 2024-04-24T09:12:42.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.732+0000 7f68c3e00640 1 -- 172.21.15.83:0/628866746 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68c41a6db0 con 0x7f68c4103060 2024-04-24T09:12:42.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.732+0000 7f68c3e00640 1 -- 172.21.15.83:0/628866746 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68c41a71f0 con 0x7f68c4103060 2024-04-24T09:12:42.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.736+0000 7f68c8a00640 1 -- 172.21.15.83:0/628866746 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f68ac042070 con 0x7f68c4103060 2024-04-24T09:12:42.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.738+0000 7f68ccbf0740 1 Processor -- start 2024-04-24T09:12:42.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.738+0000 7f68ccbf0740 1 -- start start 2024-04-24T09:12:42.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.739+0000 7f68ccbf0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562578db2a30 0x562578db2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:42.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.739+0000 7f68ccbf0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562578db3350 0x562578dbd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:42.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.739+0000 7f68ccbf0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562578dbda20 0x562578dbfe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:42.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.739+0000 7f68ccbf0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562578d181f0 con 0x562578db3350 2024-04-24T09:12:42.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.739+0000 7f68ccbf0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562578cbe350 con 0x562578dbda20 2024-04-24T09:12:42.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.739+0000 7f68ccbf0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x562578d141b0 con 0x562578db2a30 2024-04-24T09:12:42.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.739+0000 7f68c8a00640 1 --2- 172.21.15.83:0/628866746 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f68a80a3dd0 0x7f68a80a6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:42.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.739+0000 7f68ba000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562578dbda20 0x562578dbfe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:42.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.739+0000 7f68ba000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562578dbda20 0x562578dbfe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:49762/0 (socket says 172.21.15.83:49762) 2024-04-24T09:12:42.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.739+0000 7f68ba000640 1 -- 172.21.15.83:0/3088928903 learned_addr learned my addr 172.21.15.83:0/3088928903 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:12:42.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.739+0000 7f68c8a00640 1 -- 172.21.15.83:0/628866746 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 9583+0+0 (secure 0 0 0) 0x7f68ac0f1470 con 0x7f68c4103060 2024-04-24T09:12:42.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.740+0000 7f68ba000640 1 -- 172.21.15.83:0/3088928903 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562578db2a30 msgr2=0x562578db2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:42.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.740+0000 7f68ba000640 1 --2- 172.21.15.83:0/3088928903 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562578db2a30 0x562578db2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:42.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.740+0000 7f68ba000640 1 -- 172.21.15.83:0/3088928903 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562578db3350 msgr2=0x562578dbd4e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T09:12:42.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.740+0000 7f68c2a00640 1 --2- 172.21.15.83:0/3088928903 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562578db3350 0x562578dbd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:42.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.740+0000 7f68ba000640 1 --2- 172.21.15.83:0/3088928903 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562578db3350 0x562578dbd4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:42.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.740+0000 7f68c3400640 1 --2- 172.21.15.83:0/628866746 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f68a80a3dd0 0x7f68a80a6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:42.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.740+0000 7f68ba000640 1 -- 172.21.15.83:0/3088928903 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562578d12f50 con 0x562578dbda20 2024-04-24T09:12:42.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.740+0000 7f68ba000640 1 --2- 172.21.15.83:0/3088928903 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562578dbda20 0x562578dbfe10 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f68b400bc80 tx=0x7f68b40107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4469f35f7e2b84e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:42.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.740+0000 7f68b3e00640 1 -- 172.21.15.83:0/3088928903 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b4005220 con 0x562578dbda20 2024-04-24T09:12:42.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.741+0000 7f68b3e00640 1 -- 172.21.15.83:0/3088928903 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f68b400be30 con 0x562578dbda20 2024-04-24T09:12:42.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.741+0000 7f68b3e00640 1 -- 172.21.15.83:0/3088928903 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b40116a0 con 0x562578dbda20 2024-04-24T09:12:42.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.742+0000 7f68ccbf0740 1 -- 172.21.15.83:0/3088928903 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562578dbda20 msgr2=0x562578dbfe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:42.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.742+0000 7f68ccbf0740 1 --2- 172.21.15.83:0/3088928903 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562578dbda20 0x562578dbfe10 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f68b400bc80 tx=0x7f68b40107a0 comp rx=0 tx=0).stop 2024-04-24T09:12:42.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.742+0000 7f68c3400640 1 --2- 172.21.15.83:0/628866746 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f68a80a3dd0 0x7f68a80a6290 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f68bc007d90 tx=0x7f68bc007390 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:42.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.742+0000 7f68ccbf0740 1 -- 172.21.15.83:0/3088928903 shutdown_connections 2024-04-24T09:12:42.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.742+0000 7f68ccbf0740 1 --2- 172.21.15.83:0/3088928903 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562578dbda20 0x562578dbfe10 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:42.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.742+0000 7f68ccbf0740 1 --2- 172.21.15.83:0/3088928903 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562578db3350 0x562578dbd4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:42.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.742+0000 7f68ccbf0740 1 --2- 172.21.15.83:0/3088928903 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562578db2a30 0x562578db2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:42.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.742+0000 7f68ccbf0740 1 -- 172.21.15.83:0/3088928903 >> 172.21.15.83:0/3088928903 conn(0x562578daed50 msgr2=0x562578daf130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:42.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.742+0000 7f68ccbf0740 1 -- 172.21.15.83:0/3088928903 shutdown_connections 2024-04-24T09:12:42.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.742+0000 7f68ccbf0740 1 -- 172.21.15.83:0/3088928903 wait complete. 2024-04-24T09:12:42.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.743+0000 7f68ccbf0740 1 -- start start 2024-04-24T09:12:42.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.743+0000 7f68ccbf0740 1 Processor -- start 2024-04-24T09:12:42.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.744+0000 7f68ccbf0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562578db2a30 0x562578dba960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:42.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.744+0000 7f68ccbf0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562578db3350 0x562578dbaea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:42.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.744+0000 7f68ccbf0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562578dbda20 0x562578dbb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:42.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.744+0000 7f68ccbf0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562578dc0810 con 0x562578dbda20 2024-04-24T09:12:42.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.744+0000 7f68ccbf0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562578cd1690 con 0x562578db2a30 2024-04-24T09:12:42.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.744+0000 7f68ccbf0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x562578dc0990 con 0x562578db3350 2024-04-24T09:12:42.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.744+0000 7f68ccbf0740 10 client.0 _ll_register_callbacks cb 0x562578d0d650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:12:42.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.744+0000 7f68c2a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562578db2a30 0x562578dba960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:42.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.745+0000 7f68c2a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562578db2a30 0x562578dba960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:49774/0 (socket says 172.21.15.83:49774) 2024-04-24T09:12:42.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.745+0000 7f68c2a00640 1 -- 172.21.15.83:0/977606729 learned_addr learned my addr 172.21.15.83:0/977606729 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:12:42.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.745+0000 7f68c3400640 1 --2- 172.21.15.83:0/977606729 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562578dbda20 0x562578dbb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:42.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.745+0000 7f68ba000640 1 --2- 172.21.15.83:0/977606729 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562578db3350 0x562578dbaea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:42.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.745+0000 7f68c3400640 1 -- 172.21.15.83:0/977606729 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562578db3350 msgr2=0x562578dbaea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:42.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.745+0000 7f68c3400640 1 --2- 172.21.15.83:0/977606729 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562578db3350 0x562578dbaea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:42.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.745+0000 7f68c3400640 1 -- 172.21.15.83:0/977606729 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562578db2a30 msgr2=0x562578dba960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:42.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.745+0000 7f68c3400640 1 --2- 172.21.15.83:0/977606729 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562578db2a30 0x562578dba960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:42.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.745+0000 7f68c3400640 1 -- 172.21.15.83:0/977606729 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562578cd0920 con 0x562578dbda20 2024-04-24T09:12:42.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.746+0000 7f68c3400640 1 --2- 172.21.15.83:0/977606729 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562578dbda20 0x562578dbb3e0 secure :-1 s=READY pgs=2568 cs=0 l=1 rev1=1 crypto rx=0x7f68bc010e30 tx=0x7f68bc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3baac1c3a0bbf69 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:42.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.746+0000 7f68b0c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:12:42.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.746+0000 7f68b0c00640 1 -- 172.21.15.83:0/977606729 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68bc020070 con 0x562578dbda20 2024-04-24T09:12:42.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.746+0000 7f68b0c00640 1 -- 172.21.15.83:0/977606729 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9583+0+0 (secure 0 0 0) 0x7f68bc01c420 con 0x562578dbda20 2024-04-24T09:12:42.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.747+0000 7f68ccbf0740 1 -- 172.21.15.83:0/977606729 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68b400be30 con 0x562578dbda20 2024-04-24T09:12:42.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.747+0000 7f68b0c00640 1 -- 172.21.15.83:0/977606729 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f68bc02a050 con 0x562578dbda20 2024-04-24T09:12:42.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.747+0000 7f68ccbf0740 10 client.22800 fetch_fsmap learned FSMap version 64 2024-04-24T09:12:42.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.747+0000 7f68ccbf0740 1 -- 172.21.15.83:0/977606729 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x562578af47d0 con 0x562578dbda20 2024-04-24T09:12:42.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.748+0000 7f68b0c00640 1 -- 172.21.15.83:0/977606729 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f68b400be30 con 0x562578dbda20 2024-04-24T09:12:42.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.748+0000 7f68b0c00640 1 -- 172.21.15.83:0/977606729 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 64) ==== 8374+0+0 (secure 0 0 0) 0x7f68bc0257b0 con 0x562578dbda20 2024-04-24T09:12:42.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.748+0000 7f68ccbf0740 10 client.22800 fetch_fsmap finished waiting for FSMap version 64 2024-04-24T09:12:42.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.748+0000 7f68ccbf0740 10 client.22800 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:12:42.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.748+0000 7f68ccbf0740 20 client.22800 populate_metadata read hostname 'smithi083' 2024-04-24T09:12:42.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.748+0000 7f68ccbf0740 1 --2- 172.21.15.83:0/977606729 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x562578dcdde0 0x562578dd0230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:42.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.748+0000 7f68ccbf0740 4 client.22800 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:12:42.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.748+0000 7f68ccbf0740 1 -- 172.21.15.83:0/977606729 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562578d25c10 con 0x562578dcdde0 2024-04-24T09:12:42.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.748+0000 7f68c2a00640 1 --2- 172.21.15.83:0/977606729 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x562578dcdde0 0x562578dd0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:42.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.749+0000 7f68c2a00640 1 --2- 172.21.15.83:0/977606729 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x562578dcdde0 0x562578dd0230 crc :-1 s=READY pgs=521 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a9af5093c307741 server_cookie=c2111f16dbded78e in_seq=0 out_seq=0 2024-04-24T09:12:42.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.749+0000 7f68b0c00640 10 client.22800 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:12:42.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.750+0000 7f68b0c00640 1 -- 172.21.15.83:0/977606729 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562578d25c10 con 0x562578dcdde0 2024-04-24T09:12:42.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.750+0000 7f68b0c00640 10 client.22800 handle_command_reply: tid=0 2024-04-24T09:12:42.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.751+0000 7f68ccbf0740 1 client.22800 shutdown 2024-04-24T09:12:42.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.751+0000 7f68ccbf0740 1 -- 172.21.15.83:0/977606729 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562578dbda20 msgr2=0x562578dbb3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:42.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.751+0000 7f68ccbf0740 1 --2- 172.21.15.83:0/977606729 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562578dbda20 0x562578dbb3e0 secure :-1 s=READY pgs=2568 cs=0 l=1 rev1=1 crypto rx=0x7f68bc010e30 tx=0x7f68bc004a40 comp rx=0 tx=0).stop 2024-04-24T09:12:42.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.751+0000 7f68ccbf0740 1 -- 172.21.15.83:0/977606729 shutdown_connections 2024-04-24T09:12:42.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.751+0000 7f68ccbf0740 1 --2- 172.21.15.83:0/977606729 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x562578dcdde0 0x562578dd0230 crc :-1 s=READY pgs=521 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:42.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.751+0000 7f68ccbf0740 1 --2- 172.21.15.83:0/977606729 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562578dbda20 0x562578dbb3e0 unknown :-1 s=CLOSED pgs=2568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:42.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.751+0000 7f68ccbf0740 1 --2- 172.21.15.83:0/977606729 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562578db3350 0x562578dbaea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:42.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.751+0000 7f68ccbf0740 1 --2- 172.21.15.83:0/977606729 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562578db2a30 0x562578dba960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:42.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.751+0000 7f68ccbf0740 1 -- 172.21.15.83:0/977606729 >> 172.21.15.83:0/977606729 conn(0x562578daed50 msgr2=0x562578dadfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:42.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.751+0000 7f68b0c00640 0 client.22800 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:12:42.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.751+0000 7f68ccbf0740 1 -- 172.21.15.83:0/977606729 shutdown_connections 2024-04-24T09:12:42.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.751+0000 7f68ccbf0740 1 -- 172.21.15.83:0/977606729 wait complete. 2024-04-24T09:12:42.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.752+0000 7f68ccbf0740 20 client.22800 trim_cache size 0 max 16384 2024-04-24T09:12:42.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.785+0000 7f68ccbf0740 1 Processor -- start 2024-04-24T09:12:42.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.785+0000 7f68ccbf0740 1 -- start start 2024-04-24T09:12:42.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.785+0000 7f68ccbf0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562578db2a30 0x562578dc9700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:42.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.786+0000 7f68ccbf0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562578db3350 0x562578db5c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:42.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.786+0000 7f68ccbf0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562578dbda20 0x562578dd5b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:42.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.786+0000 7f68ccbf0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f68c41b16c0 con 0x562578db2a30 2024-04-24T09:12:42.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.786+0000 7f68ccbf0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f68c41b3290 con 0x562578db3350 2024-04-24T09:12:42.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.786+0000 7f68ccbf0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f68c41b1d90 con 0x562578dbda20 2024-04-24T09:12:42.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.786+0000 7f68c2a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562578db2a30 0x562578dc9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:42.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.786+0000 7f68c2a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562578db2a30 0x562578dc9700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43988/0 (socket says 172.21.15.83:43988) 2024-04-24T09:12:42.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.786+0000 7f68c2a00640 1 -- 172.21.15.83:0/1959462016 learned_addr learned my addr 172.21.15.83:0/1959462016 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:12:42.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.786+0000 7f68ba000640 1 --2- 172.21.15.83:0/1959462016 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562578db3350 0x562578db5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:42.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.786+0000 7f68c3400640 1 --2- 172.21.15.83:0/1959462016 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562578dbda20 0x562578dd5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:42.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.787+0000 7f68c2a00640 1 -- 172.21.15.83:0/1959462016 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562578dbda20 msgr2=0x562578dd5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:42.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.787+0000 7f68c2a00640 1 --2- 172.21.15.83:0/1959462016 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562578dbda20 0x562578dd5b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:42.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.787+0000 7f68c2a00640 1 -- 172.21.15.83:0/1959462016 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562578db3350 msgr2=0x562578db5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:42.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.787+0000 7f68c2a00640 1 --2- 172.21.15.83:0/1959462016 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562578db3350 0x562578db5c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:42.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.787+0000 7f68c2a00640 1 -- 172.21.15.83:0/1959462016 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562578a2a3f0 con 0x562578db2a30 2024-04-24T09:12:42.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.787+0000 7f68c2a00640 1 --2- 172.21.15.83:0/1959462016 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562578db2a30 0x562578dc9700 secure :-1 s=READY pgs=2569 cs=0 l=1 rev1=1 crypto rx=0x7f68ac005ec0 tx=0x7f68ac009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6dc1fc8dfe781cd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:42.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.788+0000 7f68b3400640 1 -- 172.21.15.83:0/1959462016 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68ac0d5070 con 0x562578db2a30 2024-04-24T09:12:42.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.788+0000 7f68b3400640 1 -- 172.21.15.83:0/1959462016 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f68ac03be50 con 0x562578db2a30 2024-04-24T09:12:42.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.788+0000 7f68b3400640 1 -- 172.21.15.83:0/1959462016 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68ac049040 con 0x562578db2a30 2024-04-24T09:12:42.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.789+0000 7f68ccbf0740 1 -- 172.21.15.83:0/1959462016 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562578db2a30 msgr2=0x562578dc9700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:42.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.789+0000 7f68ccbf0740 1 --2- 172.21.15.83:0/1959462016 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562578db2a30 0x562578dc9700 secure :-1 s=READY pgs=2569 cs=0 l=1 rev1=1 crypto rx=0x7f68ac005ec0 tx=0x7f68ac009260 comp rx=0 tx=0).stop 2024-04-24T09:12:42.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.789+0000 7f68ccbf0740 1 -- 172.21.15.83:0/1959462016 shutdown_connections 2024-04-24T09:12:42.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.789+0000 7f68ccbf0740 1 --2- 172.21.15.83:0/1959462016 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562578dbda20 0x562578dd5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:42.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.789+0000 7f68ccbf0740 1 --2- 172.21.15.83:0/1959462016 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562578db3350 0x562578db5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:42.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.789+0000 7f68ccbf0740 1 --2- 172.21.15.83:0/1959462016 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562578db2a30 0x562578dc9700 unknown :-1 s=CLOSED pgs=2569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:42.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.789+0000 7f68ccbf0740 1 -- 172.21.15.83:0/1959462016 >> 172.21.15.83:0/1959462016 conn(0x562578daed50 msgr2=0x562578db4a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:42.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.790+0000 7f68ccbf0740 1 -- 172.21.15.83:0/1959462016 shutdown_connections 2024-04-24T09:12:42.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.790+0000 7f68ccbf0740 1 -- 172.21.15.83:0/1959462016 wait complete. 2024-04-24T09:12:42.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.790+0000 7f68ccbf0740 1 -- start start 2024-04-24T09:12:42.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.790+0000 7f68ccbf0740 1 Processor -- start 2024-04-24T09:12:42.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.791+0000 7f68ccbf0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562578db2a30 0x562578dc6120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:42.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.791+0000 7f68ccbf0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562578db3350 0x562578db7430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:42.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.791+0000 7f68ccbf0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562578dbda20 0x562578db7b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:42.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.791+0000 7f68ccbf0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f68c41b2b90 con 0x562578dbda20 2024-04-24T09:12:42.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.791+0000 7f68ccbf0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f68c41b24c0 con 0x562578db3350 2024-04-24T09:12:42.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.791+0000 7f68ccbf0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f68c41af6c0 con 0x562578db2a30 2024-04-24T09:12:42.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.791+0000 7f68ccbf0740 10 client.0 _ll_register_callbacks cb 0x7f68bc005ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:12:42.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.792+0000 7f68c3400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562578dbda20 0x562578db7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:42.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.792+0000 7f68ba000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562578db3350 0x562578db7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:42.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.792+0000 7f68ba000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562578db3350 0x562578db7430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:49790/0 (socket says 172.21.15.83:49790) 2024-04-24T09:12:42.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.792+0000 7f68ba000640 1 -- 172.21.15.83:0/1324891786 learned_addr learned my addr 172.21.15.83:0/1324891786 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:12:42.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.792+0000 7f68c2a00640 1 --2- 172.21.15.83:0/1324891786 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562578db2a30 0x562578dc6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:42.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.792+0000 7f68ba000640 1 -- 172.21.15.83:0/1324891786 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562578db2a30 msgr2=0x562578dc6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:42.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.792+0000 7f68ba000640 1 --2- 172.21.15.83:0/1324891786 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562578db2a30 0x562578dc6120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:42.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.792+0000 7f68ba000640 1 -- 172.21.15.83:0/1324891786 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562578dbda20 msgr2=0x562578db7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:42.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.792+0000 7f68ba000640 1 --2- 172.21.15.83:0/1324891786 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562578dbda20 0x562578db7b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:42.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.792+0000 7f68ba000640 1 -- 172.21.15.83:0/1324891786 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625789ea0f0 con 0x562578db3350 2024-04-24T09:12:42.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.793+0000 7f68ba000640 1 --2- 172.21.15.83:0/1324891786 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562578db3350 0x562578db7430 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f68b40058c0 tx=0x7f68b4005270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9d25567d884ba74 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:42.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.793+0000 7f68b0c00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:12:42.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.793+0000 7f68b0c00640 1 -- 172.21.15.83:0/1324891786 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b4004020 con 0x562578db3350 2024-04-24T09:12:42.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.793+0000 7f68ccbf0740 1 -- 172.21.15.83:0/1324891786 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68ac03be50 con 0x562578db3350 2024-04-24T09:12:42.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.794+0000 7f68b0c00640 1 -- 172.21.15.83:0/1324891786 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9583+0+0 (secure 0 0 0) 0x7f68b402c020 con 0x562578db3350 2024-04-24T09:12:42.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.794+0000 7f68b0c00640 1 -- 172.21.15.83:0/1324891786 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f68b4034050 con 0x562578db3350 2024-04-24T09:12:42.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.794+0000 7f68ccbf0740 10 client.39845 fetch_fsmap learned FSMap version 64 2024-04-24T09:12:42.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.794+0000 7f68ccbf0740 1 -- 172.21.15.83:0/1324891786 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x562578cdd810 con 0x562578db3350 2024-04-24T09:12:42.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.795+0000 7f68b0c00640 1 -- 172.21.15.83:0/1324891786 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f68ac03be50 con 0x562578db3350 2024-04-24T09:12:42.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.795+0000 7f68b0c00640 1 -- 172.21.15.83:0/1324891786 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 64) ==== 8374+0+0 (secure 0 0 0) 0x7f68b40035f0 con 0x562578db3350 2024-04-24T09:12:42.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.795+0000 7f68ccbf0740 10 client.39845 fetch_fsmap finished waiting for FSMap version 64 2024-04-24T09:12:42.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.795+0000 7f68ccbf0740 10 client.39845 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:12:42.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.795+0000 7f68ccbf0740 20 client.39845 populate_metadata read hostname 'smithi083' 2024-04-24T09:12:42.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.795+0000 7f68ccbf0740 1 --2- 172.21.15.83:0/1324891786 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x562578e3cee0 0x562578e3f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:42.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.795+0000 7f68ccbf0740 4 client.39845 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T09:12:42.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.795+0000 7f68ccbf0740 1 -- 172.21.15.83:0/1324891786 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562578d1fa80 con 0x562578e3cee0 2024-04-24T09:12:42.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.796+0000 7f68c2a00640 1 --2- 172.21.15.83:0/1324891786 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x562578e3cee0 0x562578e3f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:42.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.797+0000 7f68b0c00640 10 client.39845 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:12:42.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.797+0000 7f68c2a00640 1 --2- 172.21.15.83:0/1324891786 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x562578e3cee0 0x562578e3f300 crc :-1 s=READY pgs=522 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5061133b68e85abf server_cookie=a0b5aa2aea99a1d2 in_seq=0 out_seq=0 2024-04-24T09:12:42.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.797+0000 7f68b0c00640 1 -- 172.21.15.83:0/1324891786 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x562578d1fa80 con 0x562578e3cee0 2024-04-24T09:12:42.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.797+0000 7f68b0c00640 10 client.39845 handle_command_reply: tid=0 2024-04-24T09:12:42.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.797+0000 7f68ccbf0740 1 client.39845 shutdown 2024-04-24T09:12:42.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.798+0000 7f68ccbf0740 1 -- 172.21.15.83:0/1324891786 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562578db3350 msgr2=0x562578db7430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:42.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.798+0000 7f68ccbf0740 1 --2- 172.21.15.83:0/1324891786 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562578db3350 0x562578db7430 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f68b40058c0 tx=0x7f68b4005270 comp rx=0 tx=0).stop 2024-04-24T09:12:42.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.798+0000 7f68ccbf0740 1 -- 172.21.15.83:0/1324891786 shutdown_connections 2024-04-24T09:12:42.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.798+0000 7f68ccbf0740 1 --2- 172.21.15.83:0/1324891786 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x562578e3cee0 0x562578e3f300 crc :-1 s=READY pgs=522 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:42.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.798+0000 7f68ccbf0740 1 --2- 172.21.15.83:0/1324891786 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562578dbda20 0x562578db7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:42.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.798+0000 7f68ccbf0740 1 --2- 172.21.15.83:0/1324891786 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562578db3350 0x562578db7430 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:42.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.798+0000 7f68ccbf0740 1 --2- 172.21.15.83:0/1324891786 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562578db2a30 0x562578dc6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:42.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.798+0000 7f68ccbf0740 1 -- 172.21.15.83:0/1324891786 >> 172.21.15.83:0/1324891786 conn(0x562578daed50 msgr2=0x562578db4180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:42.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.798+0000 7f68b0c00640 0 client.39845 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:12:42.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.798+0000 7f68ccbf0740 1 -- 172.21.15.83:0/1324891786 shutdown_connections 2024-04-24T09:12:42.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.798+0000 7f68ccbf0740 1 -- 172.21.15.83:0/1324891786 wait complete. 2024-04-24T09:12:42.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.798+0000 7f68ccbf0740 20 client.39845 trim_cache size 0 max 16384 2024-04-24T09:12:42.800 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T09:12:42.801 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-04-24T09:12:42.801 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-04-24T09:12:42.801 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T09:12:42.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.799+0000 7f68c3e00640 1 -- 172.21.15.83:0/628866746 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f68a80a3dd0 msgr2=0x7f68a80a6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:42.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.799+0000 7f68c3e00640 1 --2- 172.21.15.83:0/628866746 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f68a80a3dd0 0x7f68a80a6290 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f68bc007d90 tx=0x7f68bc007390 comp rx=0 tx=0).stop 2024-04-24T09:12:42.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.799+0000 7f68c3e00640 1 -- 172.21.15.83:0/628866746 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68c4103060 msgr2=0x7f68c4100cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:42.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.799+0000 7f68c3e00640 1 --2- 172.21.15.83:0/628866746 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68c4103060 0x7f68c4100cb0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f68ac005ab0 tx=0x7f68ac003cd0 comp rx=0 tx=0).stop 2024-04-24T09:12:42.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.799+0000 7f68c3e00640 1 -- 172.21.15.83:0/628866746 shutdown_connections 2024-04-24T09:12:42.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.799+0000 7f68c3e00640 1 --2- 172.21.15.83:0/628866746 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f68a80a3dd0 0x7f68a80a6290 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:42.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.799+0000 7f68c3e00640 1 --2- 172.21.15.83:0/628866746 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68c410c480 0x7f68c4101730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:42.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.799+0000 7f68c3e00640 1 --2- 172.21.15.83:0/628866746 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f68c4103980 0x7f68c41011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:42.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.799+0000 7f68c3e00640 1 --2- 172.21.15.83:0/628866746 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68c4103060 0x7f68c4100cb0 secure :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f68ac005ab0 tx=0x7f68ac003cd0 comp rx=0 tx=0).stop 2024-04-24T09:12:42.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.799+0000 7f68c3e00640 1 -- 172.21.15.83:0/628866746 >> 172.21.15.83:0/628866746 conn(0x7f68c40fc880 msgr2=0x7f68c40fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:42.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.800+0000 7f68c3e00640 1 -- 172.21.15.83:0/628866746 shutdown_connections 2024-04-24T09:12:42.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:42.800+0000 7f68c3e00640 1 -- 172.21.15.83:0/628866746 wait complete. 2024-04-24T09:12:42.825 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T09:12:42.825 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T09:12:43.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.108+0000 7f0f05800640 1 -- 172.21.15.83:0/245152531 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f00109520 msgr2=0x7f0f001110f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:43.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.108+0000 7f0f05800640 1 --2- 172.21.15.83:0/245152531 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f00109520 0x7f0f001110f0 secure :-1 s=READY pgs=2570 cs=0 l=1 rev1=1 crypto rx=0x7f0ef8009ca0 tx=0x7f0ef802a8c0 comp rx=0 tx=0).stop 2024-04-24T09:12:43.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.109+0000 7f0f05800640 1 -- 172.21.15.83:0/245152531 shutdown_connections 2024-04-24T09:12:43.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.109+0000 7f0f05800640 1 --2- 172.21.15.83:0/245152531 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f00109520 0x7f0f001110f0 unknown :-1 s=CLOSED pgs=2570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:43.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.109+0000 7f0f05800640 1 --2- 172.21.15.83:0/245152531 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0f00104e00 0x7f0f00108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:43.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.109+0000 7f0f05800640 1 --2- 172.21.15.83:0/245152531 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f00104450 0x7f0f00104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:43.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.109+0000 7f0f05800640 1 -- 172.21.15.83:0/245152531 >> 172.21.15.83:0/245152531 conn(0x7f0f000ffcf0 msgr2=0x7f0f00102110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:43.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.109+0000 7f0f05800640 1 -- 172.21.15.83:0/245152531 shutdown_connections 2024-04-24T09:12:43.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.109+0000 7f0f05800640 1 -- 172.21.15.83:0/245152531 wait complete. 2024-04-24T09:12:43.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.110+0000 7f0f05800640 1 Processor -- start 2024-04-24T09:12:43.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.111+0000 7f0f05800640 1 -- start start 2024-04-24T09:12:43.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.112+0000 7f0f05800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f00104450 0x7f0f00111030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:43.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.112+0000 7f0f05800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f00109520 0x7f0f0010e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:43.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.112+0000 7f0f05800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0f0010e610 0x7f0f0010ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:43.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.112+0000 7f0f05800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0f00114500 con 0x7f0f00104450 2024-04-24T09:12:43.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.112+0000 7f0f05800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0f00114380 con 0x7f0f00109520 2024-04-24T09:12:43.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.112+0000 7f0f05800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0f00114680 con 0x7f0f0010e610 2024-04-24T09:12:43.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.113+0000 7f0efe000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f00109520 0x7f0f0010e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:43.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.113+0000 7f0eff400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0f0010e610 0x7f0f0010ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:43.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.113+0000 7f0efe000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f00109520 0x7f0f0010e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:49810/0 (socket says 172.21.15.83:49810) 2024-04-24T09:12:43.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.113+0000 7f0efe000640 1 -- 172.21.15.83:0/800442258 learned_addr learned my addr 172.21.15.83:0/800442258 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:12:43.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.113+0000 7f0efea00640 1 --2- 172.21.15.83:0/800442258 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f00104450 0x7f0f00111030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:43.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.113+0000 7f0efe000640 1 -- 172.21.15.83:0/800442258 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0f0010e610 msgr2=0x7f0f0010ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:43.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.113+0000 7f0efe000640 1 --2- 172.21.15.83:0/800442258 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0f0010e610 0x7f0f0010ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:43.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.113+0000 7f0efe000640 1 -- 172.21.15.83:0/800442258 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f00104450 msgr2=0x7f0f00111030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:43.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.113+0000 7f0efe000640 1 --2- 172.21.15.83:0/800442258 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f00104450 0x7f0f00111030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:43.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.113+0000 7f0efe000640 1 -- 172.21.15.83:0/800442258 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f001ab4a0 con 0x7f0f00109520 2024-04-24T09:12:43.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.114+0000 7f0efe000640 1 --2- 172.21.15.83:0/800442258 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f00109520 0x7f0f0010e0d0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f0eec00e9f0 tx=0x7f0eec00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:43.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.114+0000 7f0ef7400640 1 -- 172.21.15.83:0/800442258 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eec00ccc0 con 0x7f0f00109520 2024-04-24T09:12:43.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.114+0000 7f0ef7400640 1 -- 172.21.15.83:0/800442258 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0eec0054d0 con 0x7f0f00109520 2024-04-24T09:12:43.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.114+0000 7f0ef7400640 1 -- 172.21.15.83:0/800442258 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eec0186d0 con 0x7f0f00109520 2024-04-24T09:12:43.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.114+0000 7f0f05800640 1 -- 172.21.15.83:0/800442258 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f001ab790 con 0x7f0f00109520 2024-04-24T09:12:43.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.114+0000 7f0f05800640 1 -- 172.21.15.83:0/800442258 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f001abc90 con 0x7f0f00109520 2024-04-24T09:12:43.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.116+0000 7f0f05800640 1 -- 172.21.15.83:0/800442258 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f00105920 con 0x7f0f00109520 2024-04-24T09:12:43.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.121+0000 7f0ef7400640 1 -- 172.21.15.83:0/800442258 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0eec004e10 con 0x7f0f00109520 2024-04-24T09:12:43.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.122+0000 7f0ef7400640 1 --2- 172.21.15.83:0/800442258 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f0ee40a3de0 0x7f0ee40a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:43.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.123+0000 7f0ef7400640 1 -- 172.21.15.83:0/800442258 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 9583+0+0 (secure 0 0 0) 0x7f0eec0d1160 con 0x7f0f00109520 2024-04-24T09:12:43.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.123+0000 7f0efea00640 1 --2- 172.21.15.83:0/800442258 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f0ee40a3de0 0x7f0ee40a62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:43.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.125+0000 7f0efea00640 1 --2- 172.21.15.83:0/800442258 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f0ee40a3de0 0x7f0ee40a62a0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f0ef0009990 tx=0x7f0ef0009110 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:43.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.125+0000 7f0ef7400640 1 -- 172.21.15.83:0/800442258 <== mon.2 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0eec09d3e0 con 0x7f0f00109520 2024-04-24T09:12:43.189 INFO:tasks.workunit.client.0.smithi083.stdout: 10 54090 6.61 MB/sec execute 314 sec latency 82.629 ms 2024-04-24T09:12:43.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.556+0000 7f0f05800640 1 -- 172.21.15.83:0/800442258 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0f00104e00 con 0x7f0f00109520 2024-04-24T09:12:43.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.557+0000 7f0ef7400640 1 -- 172.21.15.83:0/800442258 <== mon.2 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) ==== 94+0+11948 (secure 0 0 0) 0x7f0eec09cce0 con 0x7f0f00109520 2024-04-24T09:12:43.558 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T09:12:43.558 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":64,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/358872257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":358872257},{"type":"v1","addr":"172.21.15.83:6835","nonce":358872257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3157070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3157070851},{"type":"v1","addr":"172.21.15.83:6837","nonce":3157070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":64,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T08:48:29.422576+0000","modified":"2024-04-24T09:12:33.175729+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34520,"mds_1":34493,"mds_2":34451,"mds_3":34427,"mds_4":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14658":{"gid":14658,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6839/4085470844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":4085470844},{"type":"v1","addr":"172.21.15.83:6839","nonce":4085470844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14676":{"gid":14676,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6841/2634754210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":2634754210},{"type":"v1","addr":"172.21.15.83:6841","nonce":2634754210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24382":{"gid":24382,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/1859921310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1859921310},{"type":"v1","addr":"172.21.15.189:6833","nonce":1859921310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/2241923197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2241923197},{"type":"v1","addr":"172.21.15.189:6835","nonce":2241923197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.189:6839/2548965336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":2548965336},{"type":"v1","addr":"172.21.15.189:6839","nonce":2548965336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26686":{"gid":26686,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6837/1678817161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":1678817161},{"type":"v1","addr":"172.21.15.189:6837","nonce":1678817161}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.138:6833/1157967658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1157967658},{"type":"v1","addr":"172.21.15.138:6833","nonce":1157967658}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.138:6835/1909219758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1909219758},{"type":"v1","addr":"172.21.15.138:6835","nonce":1909219758}]},"join_fscid":-1,"export_targets":[0,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34493":{"gid":34493,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.138:6837/900566754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":900566754},{"type":"v1","addr":"172.21.15.138:6837","nonce":900566754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34520":{"gid":34520,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.138:6839/2236932393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2236932393},{"type":"v1","addr":"172.21.15.138:6839","nonce":2236932393}]},"join_fscid":-1,"export_targets":[4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34520,"qdb_cluster":[34451,34493,34427,34520,24454]},"id":1}]} 2024-04-24T09:12:43.558 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 64 2024-04-24T09:12:43.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.560+0000 7f0f05800640 1 -- 172.21.15.83:0/800442258 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f0ee40a3de0 msgr2=0x7f0ee40a62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:43.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.560+0000 7f0f05800640 1 --2- 172.21.15.83:0/800442258 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f0ee40a3de0 0x7f0ee40a62a0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f0ef0009990 tx=0x7f0ef0009110 comp rx=0 tx=0).stop 2024-04-24T09:12:43.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.560+0000 7f0f05800640 1 -- 172.21.15.83:0/800442258 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f00109520 msgr2=0x7f0f0010e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:43.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.560+0000 7f0f05800640 1 --2- 172.21.15.83:0/800442258 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f00109520 0x7f0f0010e0d0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f0eec00e9f0 tx=0x7f0eec00edf0 comp rx=0 tx=0).stop 2024-04-24T09:12:43.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.560+0000 7f0f05800640 1 -- 172.21.15.83:0/800442258 shutdown_connections 2024-04-24T09:12:43.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.560+0000 7f0f05800640 1 --2- 172.21.15.83:0/800442258 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f0ee40a3de0 0x7f0ee40a62a0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:43.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.560+0000 7f0f05800640 1 --2- 172.21.15.83:0/800442258 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0f0010e610 0x7f0f0010ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:43.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.560+0000 7f0f05800640 1 --2- 172.21.15.83:0/800442258 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f00109520 0x7f0f0010e0d0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:43.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.560+0000 7f0f05800640 1 --2- 172.21.15.83:0/800442258 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f00104450 0x7f0f00111030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:43.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.561+0000 7f0f05800640 1 -- 172.21.15.83:0/800442258 >> 172.21.15.83:0/800442258 conn(0x7f0f000ffcf0 msgr2=0x7f0f00101000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:43.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.561+0000 7f0f05800640 1 -- 172.21.15.83:0/800442258 shutdown_connections 2024-04-24T09:12:43.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.561+0000 7f0f05800640 1 -- 172.21.15.83:0/800442258 wait complete. 2024-04-24T09:12:43.584 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-24T09:12:43.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.847+0000 7fe227200640 1 -- 172.21.15.83:0/3674505643 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe22810e560 msgr2=0x7fe22810e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:43.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.847+0000 7fe227200640 1 --2- 172.21.15.83:0/3674505643 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe22810e560 0x7fe22810e940 secure :-1 s=READY pgs=2571 cs=0 l=1 rev1=1 crypto rx=0x7fe218009930 tx=0x7fe21802a420 comp rx=0 tx=0).stop 2024-04-24T09:12:43.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.847+0000 7fe227200640 1 -- 172.21.15.83:0/3674505643 shutdown_connections 2024-04-24T09:12:43.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.847+0000 7fe227200640 1 --2- 172.21.15.83:0/3674505643 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe228108ee0 0x7fe228115390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:43.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.847+0000 7fe227200640 1 --2- 172.21.15.83:0/3674505643 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe228108540 0x7fe2281089a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:43.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.847+0000 7fe227200640 1 --2- 172.21.15.83:0/3674505643 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe22810e560 0x7fe22810e940 unknown :-1 s=CLOSED pgs=2571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:43.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.847+0000 7fe227200640 1 -- 172.21.15.83:0/3674505643 >> 172.21.15.83:0/3674505643 conn(0x7fe228077e20 msgr2=0x7fe228106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:43.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.848+0000 7fe227200640 1 -- 172.21.15.83:0/3674505643 shutdown_connections 2024-04-24T09:12:43.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.849+0000 7fe227200640 1 -- 172.21.15.83:0/3674505643 wait complete. 2024-04-24T09:12:43.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.850+0000 7fe227200640 1 Processor -- start 2024-04-24T09:12:43.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.850+0000 7fe227200640 1 -- start start 2024-04-24T09:12:43.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.850+0000 7fe227200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe228108540 0x7fe2280737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:43.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.851+0000 7fe227200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe228108ee0 0x7fe22806d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:43.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.851+0000 7fe227200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe22806dd90 0x7fe22806e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:43.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.851+0000 7fe227200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe22806c030 con 0x7fe228108ee0 2024-04-24T09:12:43.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.851+0000 7fe227200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe22806beb0 con 0x7fe228108540 2024-04-24T09:12:43.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.851+0000 7fe227200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe22806c1b0 con 0x7fe22806dd90 2024-04-24T09:12:43.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.851+0000 7fe225400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe228108ee0 0x7fe22806d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:43.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.851+0000 7fe225e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe228108540 0x7fe2280737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:43.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.851+0000 7fe225400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe228108ee0 0x7fe22806d850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44020/0 (socket says 172.21.15.83:44020) 2024-04-24T09:12:43.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.851+0000 7fe225400640 1 -- 172.21.15.83:0/42234079 learned_addr learned my addr 172.21.15.83:0/42234079 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:12:43.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.851+0000 7fe226800640 1 --2- 172.21.15.83:0/42234079 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe22806dd90 0x7fe22806e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:43.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.852+0000 7fe225e00640 1 -- 172.21.15.83:0/42234079 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe22806dd90 msgr2=0x7fe22806e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:43.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.852+0000 7fe225e00640 1 --2- 172.21.15.83:0/42234079 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe22806dd90 0x7fe22806e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:43.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.852+0000 7fe225e00640 1 -- 172.21.15.83:0/42234079 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe228108ee0 msgr2=0x7fe22806d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:43.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.852+0000 7fe225e00640 1 --2- 172.21.15.83:0/42234079 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe228108ee0 0x7fe22806d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:43.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.852+0000 7fe225e00640 1 -- 172.21.15.83:0/42234079 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2281b3e60 con 0x7fe228108540 2024-04-24T09:12:43.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.852+0000 7fe226800640 1 --2- 172.21.15.83:0/42234079 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe22806dd90 0x7fe22806e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:12:43.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.852+0000 7fe225400640 1 --2- 172.21.15.83:0/42234079 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe228108ee0 0x7fe22806d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:12:43.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.852+0000 7fe225e00640 1 --2- 172.21.15.83:0/42234079 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe228108540 0x7fe2280737e0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fe218008000 tx=0x7fe2180042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:43.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.852+0000 7fe21ea00640 1 -- 172.21.15.83:0/42234079 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe21803f070 con 0x7fe228108540 2024-04-24T09:12:43.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.853+0000 7fe227200640 1 -- 172.21.15.83:0/42234079 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2281b4030 con 0x7fe228108540 2024-04-24T09:12:43.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.853+0000 7fe21ea00640 1 -- 172.21.15.83:0/42234079 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe21803b460 con 0x7fe228108540 2024-04-24T09:12:43.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.853+0000 7fe21ea00640 1 -- 172.21.15.83:0/42234079 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe218035070 con 0x7fe228108540 2024-04-24T09:12:43.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.853+0000 7fe227200640 1 -- 172.21.15.83:0/42234079 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2281b4440 con 0x7fe228108540 2024-04-24T09:12:43.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.857+0000 7fe22f830740 1 Processor -- start 2024-04-24T09:12:43.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.857+0000 7fe22f830740 1 -- start start 2024-04-24T09:12:43.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.857+0000 7fe21ea00640 1 -- 172.21.15.83:0/42234079 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe21803b9c0 con 0x7fe228108540 2024-04-24T09:12:43.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.858+0000 7fe22f830740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a77d9fb40 0x559a77d9ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:43.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.858+0000 7fe22f830740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a77da0460 0x559a77daa5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:43.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.858+0000 7fe22f830740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a77daab30 0x559a77dacf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:43.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.858+0000 7fe22f830740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559a77d051d0 con 0x559a77daab30 2024-04-24T09:12:43.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.858+0000 7fe22f830740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559a77cab450 con 0x559a77da0460 2024-04-24T09:12:43.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.858+0000 7fe22f830740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559a77d01190 con 0x559a77d9fb40 2024-04-24T09:12:43.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.859+0000 7fe225400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a77daab30 0x559a77dacf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:43.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.859+0000 7fe225400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a77daab30 0x559a77dacf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44034/0 (socket says 172.21.15.83:44034) 2024-04-24T09:12:43.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.859+0000 7fe225400640 1 -- 172.21.15.83:0/86764813 learned_addr learned my addr 172.21.15.83:0/86764813 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:12:43.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.859+0000 7fe225e00640 1 --2- 172.21.15.83:0/86764813 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a77da0460 0x559a77daa5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:43.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.859+0000 7fe225400640 1 -- 172.21.15.83:0/86764813 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a77d9fb40 msgr2=0x559a77d9ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:43.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.859+0000 7fe225400640 1 --2- 172.21.15.83:0/86764813 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a77d9fb40 0x559a77d9ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:43.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.859+0000 7fe225400640 1 -- 172.21.15.83:0/86764813 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a77da0460 msgr2=0x559a77daa5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:43.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.860+0000 7fe225400640 1 --2- 172.21.15.83:0/86764813 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a77da0460 0x559a77daa5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:43.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.860+0000 7fe225400640 1 -- 172.21.15.83:0/86764813 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a77d68310 con 0x559a77daab30 2024-04-24T09:12:43.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.860+0000 7fe21ea00640 1 --2- 172.21.15.83:0/42234079 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe20c0a3da0 0x7fe20c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:43.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.860+0000 7fe21ea00640 1 -- 172.21.15.83:0/42234079 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 9583+0+0 (secure 0 0 0) 0x7fe2180f0ec0 con 0x7fe228108540 2024-04-24T09:12:43.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.860+0000 7fe226800640 1 --2- 172.21.15.83:0/42234079 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe20c0a3da0 0x7fe20c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:43.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.860+0000 7fe225400640 1 --2- 172.21.15.83:0/86764813 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a77daab30 0x559a77dacf20 secure :-1 s=READY pgs=2572 cs=0 l=1 rev1=1 crypto rx=0x7fe214010580 tx=0x7fe214010a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43791b6cfa0c0051 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:43.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.861+0000 7fe213000640 1 -- 172.21.15.83:0/86764813 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe214011820 con 0x559a77daab30 2024-04-24T09:12:43.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.861+0000 7fe213000640 1 -- 172.21.15.83:0/86764813 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe21400e430 con 0x559a77daab30 2024-04-24T09:12:43.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.861+0000 7fe213000640 1 -- 172.21.15.83:0/86764813 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe21400d5a0 con 0x559a77daab30 2024-04-24T09:12:43.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.861+0000 7fe226800640 1 --2- 172.21.15.83:0/42234079 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe20c0a3da0 0x7fe20c0a6260 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fe220005fd0 tx=0x7fe220009540 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:43.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.862+0000 7fe22f830740 1 -- 172.21.15.83:0/86764813 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a77daab30 msgr2=0x559a77dacf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:43.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.862+0000 7fe22f830740 1 --2- 172.21.15.83:0/86764813 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a77daab30 0x559a77dacf20 secure :-1 s=READY pgs=2572 cs=0 l=1 rev1=1 crypto rx=0x7fe214010580 tx=0x7fe214010a40 comp rx=0 tx=0).stop 2024-04-24T09:12:43.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.862+0000 7fe22f830740 1 -- 172.21.15.83:0/86764813 shutdown_connections 2024-04-24T09:12:43.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.862+0000 7fe22f830740 1 --2- 172.21.15.83:0/86764813 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a77daab30 0x559a77dacf20 unknown :-1 s=CLOSED pgs=2572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:43.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.862+0000 7fe22f830740 1 --2- 172.21.15.83:0/86764813 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a77da0460 0x559a77daa5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:43.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.862+0000 7fe22f830740 1 --2- 172.21.15.83:0/86764813 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a77d9fb40 0x559a77d9ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:43.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.862+0000 7fe22f830740 1 -- 172.21.15.83:0/86764813 >> 172.21.15.83:0/86764813 conn(0x559a77d9be60 msgr2=0x559a77d9c240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:43.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.862+0000 7fe22f830740 1 -- 172.21.15.83:0/86764813 shutdown_connections 2024-04-24T09:12:43.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.862+0000 7fe22f830740 1 -- 172.21.15.83:0/86764813 wait complete. 2024-04-24T09:12:43.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.863+0000 7fe22f830740 1 -- start start 2024-04-24T09:12:43.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.863+0000 7fe22f830740 1 Processor -- start 2024-04-24T09:12:43.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.864+0000 7fe22f830740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a77d9fb40 0x559a77daa5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:43.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.864+0000 7fe22f830740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a77da0460 0x559a77da78c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:43.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.864+0000 7fe22f830740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a77daab30 0x559a77da7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:43.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.864+0000 7fe22f830740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559a77dad9c0 con 0x559a77daab30 2024-04-24T09:12:43.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.864+0000 7fe22f830740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559a77cbe810 con 0x559a77d9fb40 2024-04-24T09:12:43.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.864+0000 7fe22f830740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559a77dadb40 con 0x559a77da0460 2024-04-24T09:12:43.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.864+0000 7fe22f830740 10 client.0 _ll_register_callbacks cb 0x559a77c8c140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:12:43.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.864+0000 7fe226800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a77daab30 0x559a77da7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:43.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.864+0000 7fe226800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a77daab30 0x559a77da7fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44048/0 (socket says 172.21.15.83:44048) 2024-04-24T09:12:43.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.864+0000 7fe226800640 1 -- 172.21.15.83:0/1940428217 learned_addr learned my addr 172.21.15.83:0/1940428217 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:12:43.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.864+0000 7fe225e00640 1 --2- 172.21.15.83:0/1940428217 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a77d9fb40 0x559a77daa5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:43.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.865+0000 7fe225400640 1 --2- 172.21.15.83:0/1940428217 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a77da0460 0x559a77da78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:43.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.865+0000 7fe226800640 1 -- 172.21.15.83:0/1940428217 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a77da0460 msgr2=0x559a77da78c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:43.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.865+0000 7fe226800640 1 --2- 172.21.15.83:0/1940428217 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a77da0460 0x559a77da78c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:43.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.865+0000 7fe226800640 1 -- 172.21.15.83:0/1940428217 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a77d9fb40 msgr2=0x559a77daa5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:43.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.865+0000 7fe226800640 1 --2- 172.21.15.83:0/1940428217 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a77d9fb40 0x559a77daa5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:43.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.865+0000 7fe226800640 1 -- 172.21.15.83:0/1940428217 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a77ccf3f0 con 0x559a77daab30 2024-04-24T09:12:43.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.865+0000 7fe226800640 1 --2- 172.21.15.83:0/1940428217 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a77daab30 0x559a77da7fb0 secure :-1 s=READY pgs=2573 cs=0 l=1 rev1=1 crypto rx=0x7fe220004a80 tx=0x7fe220004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7de07d724340287 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:43.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.865+0000 7fe207400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:12:43.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.866+0000 7fe207400640 1 -- 172.21.15.83:0/1940428217 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe220023070 con 0x559a77daab30 2024-04-24T09:12:43.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.866+0000 7fe207400640 1 -- 172.21.15.83:0/1940428217 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9583+0+0 (secure 0 0 0) 0x7fe220011930 con 0x559a77daab30 2024-04-24T09:12:43.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.866+0000 7fe22f830740 1 -- 172.21.15.83:0/1940428217 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe21400e430 con 0x559a77daab30 2024-04-24T09:12:43.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.866+0000 7fe207400640 1 -- 172.21.15.83:0/1940428217 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7fe22002f050 con 0x559a77daab30 2024-04-24T09:12:43.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.866+0000 7fe22f830740 10 client.22824 fetch_fsmap learned FSMap version 64 2024-04-24T09:12:43.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.866+0000 7fe22f830740 1 -- 172.21.15.83:0/1940428217 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x559a77a173f0 con 0x559a77daab30 2024-04-24T09:12:43.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.867+0000 7fe207400640 1 -- 172.21.15.83:0/1940428217 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe21400e430 con 0x559a77daab30 2024-04-24T09:12:43.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.867+0000 7fe207400640 1 -- 172.21.15.83:0/1940428217 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 64) ==== 8374+0+0 (secure 0 0 0) 0x7fe22000f040 con 0x559a77daab30 2024-04-24T09:12:43.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.867+0000 7fe22f830740 10 client.22824 fetch_fsmap finished waiting for FSMap version 64 2024-04-24T09:12:43.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.867+0000 7fe22f830740 10 client.22824 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-24T09:12:43.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.867+0000 7fe22f830740 20 client.22824 populate_metadata read hostname 'smithi083' 2024-04-24T09:12:43.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.867+0000 7fe22f830740 1 --2- 172.21.15.83:0/1940428217 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x559a77dbad90 0x559a77dbd1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:43.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.867+0000 7fe22f830740 4 client.22824 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:12:43.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.867+0000 7fe22f830740 1 -- 172.21.15.83:0/1940428217 --> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559a77d12bf0 con 0x559a77dbad90 2024-04-24T09:12:43.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.867+0000 7fe225e00640 1 --2- 172.21.15.83:0/1940428217 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x559a77dbad90 0x559a77dbd1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:43.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.868+0000 7fe225e00640 1 --2- 172.21.15.83:0/1940428217 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x559a77dbad90 0x559a77dbd1d0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ab542e4d9a0d1d5 server_cookie=1e71237831db4b36 in_seq=0 out_seq=0 2024-04-24T09:12:43.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.868+0000 7fe207400640 10 client.22824 ms_handle_connect on v2:172.21.15.189:6838/2548965336 2024-04-24T09:12:43.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.869+0000 7fe207400640 1 -- 172.21.15.83:0/1940428217 <== mds.4 v2:172.21.15.189:6838/2548965336 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559a77d12bf0 con 0x559a77dbad90 2024-04-24T09:12:43.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.869+0000 7fe207400640 10 client.22824 handle_command_reply: tid=0 2024-04-24T09:12:43.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.869+0000 7fe22f830740 1 client.22824 shutdown 2024-04-24T09:12:43.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.869+0000 7fe22f830740 1 -- 172.21.15.83:0/1940428217 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a77daab30 msgr2=0x559a77da7fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:43.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.869+0000 7fe22f830740 1 --2- 172.21.15.83:0/1940428217 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a77daab30 0x559a77da7fb0 secure :-1 s=READY pgs=2573 cs=0 l=1 rev1=1 crypto rx=0x7fe220004a80 tx=0x7fe220004ab0 comp rx=0 tx=0).stop 2024-04-24T09:12:43.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.870+0000 7fe22f830740 1 -- 172.21.15.83:0/1940428217 shutdown_connections 2024-04-24T09:12:43.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.870+0000 7fe22f830740 1 --2- 172.21.15.83:0/1940428217 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x559a77dbad90 0x559a77dbd1d0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:43.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.870+0000 7fe22f830740 1 --2- 172.21.15.83:0/1940428217 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a77daab30 0x559a77da7fb0 unknown :-1 s=CLOSED pgs=2573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:43.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.870+0000 7fe22f830740 1 --2- 172.21.15.83:0/1940428217 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a77da0460 0x559a77da78c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:43.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.870+0000 7fe207400640 0 client.22824 ms_handle_reset on v2:172.21.15.189:6838/2548965336 2024-04-24T09:12:43.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.870+0000 7fe22f830740 1 --2- 172.21.15.83:0/1940428217 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a77d9fb40 0x559a77daa5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:43.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.870+0000 7fe22f830740 1 -- 172.21.15.83:0/1940428217 >> 172.21.15.83:0/1940428217 conn(0x559a77d9be60 msgr2=0x559a77da1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:43.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.870+0000 7fe22f830740 1 -- 172.21.15.83:0/1940428217 shutdown_connections 2024-04-24T09:12:43.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.870+0000 7fe22f830740 1 -- 172.21.15.83:0/1940428217 wait complete. 2024-04-24T09:12:43.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.870+0000 7fe22f830740 20 client.22824 trim_cache size 0 max 16384 2024-04-24T09:12:43.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.894+0000 7fe22f830740 1 Processor -- start 2024-04-24T09:12:43.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.894+0000 7fe22f830740 1 -- start start 2024-04-24T09:12:43.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.895+0000 7fe22f830740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a77d9fb40 0x559a77db6880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:43.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.895+0000 7fe22f830740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a77da0460 0x559a77daa100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:43.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.895+0000 7fe22f830740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a77daab30 0x559a77dc2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:43.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.895+0000 7fe22f830740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe2281be790 con 0x559a77d9fb40 2024-04-24T09:12:43.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.895+0000 7fe22f830740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe2281c0360 con 0x559a77da0460 2024-04-24T09:12:43.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.895+0000 7fe22f830740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe2281bee60 con 0x559a77daab30 2024-04-24T09:12:43.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.895+0000 7fe225e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a77d9fb40 0x559a77db6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:43.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.895+0000 7fe225e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a77d9fb40 0x559a77db6880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44052/0 (socket says 172.21.15.83:44052) 2024-04-24T09:12:43.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.895+0000 7fe225e00640 1 -- 172.21.15.83:0/1698574340 learned_addr learned my addr 172.21.15.83:0/1698574340 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:12:43.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.895+0000 7fe226800640 1 --2- 172.21.15.83:0/1698574340 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a77daab30 0x559a77dc2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:43.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.895+0000 7fe225400640 1 --2- 172.21.15.83:0/1698574340 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a77da0460 0x559a77daa100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:43.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.896+0000 7fe225e00640 1 -- 172.21.15.83:0/1698574340 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a77daab30 msgr2=0x559a77dc2aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:43.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.896+0000 7fe225e00640 1 --2- 172.21.15.83:0/1698574340 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a77daab30 0x559a77dc2aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:43.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.896+0000 7fe225e00640 1 -- 172.21.15.83:0/1698574340 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a77da0460 msgr2=0x559a77daa100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:43.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.896+0000 7fe225e00640 1 --2- 172.21.15.83:0/1698574340 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a77da0460 0x559a77daa100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:43.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.896+0000 7fe225e00640 1 -- 172.21.15.83:0/1698574340 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a77ae17d0 con 0x559a77d9fb40 2024-04-24T09:12:43.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.896+0000 7fe225e00640 1 --2- 172.21.15.83:0/1698574340 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a77d9fb40 0x559a77db6880 secure :-1 s=READY pgs=2574 cs=0 l=1 rev1=1 crypto rx=0x7fe2180b93b0 tx=0x7fe2180f3290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=264cebb96edcbab5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:43.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.897+0000 7fe212600640 1 -- 172.21.15.83:0/1698574340 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2180044e0 con 0x559a77d9fb40 2024-04-24T09:12:43.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.897+0000 7fe212600640 1 -- 172.21.15.83:0/1698574340 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe2180bd0e0 con 0x559a77d9fb40 2024-04-24T09:12:43.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.897+0000 7fe212600640 1 -- 172.21.15.83:0/1698574340 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2180bad70 con 0x559a77d9fb40 2024-04-24T09:12:43.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.897+0000 7fe22f830740 1 -- 172.21.15.83:0/1698574340 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a77d9fb40 msgr2=0x559a77db6880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:43.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.897+0000 7fe22f830740 1 --2- 172.21.15.83:0/1698574340 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a77d9fb40 0x559a77db6880 secure :-1 s=READY pgs=2574 cs=0 l=1 rev1=1 crypto rx=0x7fe2180b93b0 tx=0x7fe2180f3290 comp rx=0 tx=0).stop 2024-04-24T09:12:43.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.898+0000 7fe22f830740 1 -- 172.21.15.83:0/1698574340 shutdown_connections 2024-04-24T09:12:43.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.898+0000 7fe22f830740 1 --2- 172.21.15.83:0/1698574340 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a77daab30 0x559a77dc2aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:43.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.898+0000 7fe22f830740 1 --2- 172.21.15.83:0/1698574340 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a77da0460 0x559a77daa100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:43.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.898+0000 7fe22f830740 1 --2- 172.21.15.83:0/1698574340 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a77d9fb40 0x559a77db6880 unknown :-1 s=CLOSED pgs=2574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:43.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.898+0000 7fe22f830740 1 -- 172.21.15.83:0/1698574340 >> 172.21.15.83:0/1698574340 conn(0x559a77d9be60 msgr2=0x559a77da2bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:43.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.899+0000 7fe22f830740 1 -- 172.21.15.83:0/1698574340 shutdown_connections 2024-04-24T09:12:43.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.899+0000 7fe22f830740 1 -- 172.21.15.83:0/1698574340 wait complete. 2024-04-24T09:12:43.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.899+0000 7fe22f830740 1 -- start start 2024-04-24T09:12:43.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.900+0000 7fe22f830740 1 Processor -- start 2024-04-24T09:12:43.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.901+0000 7fe22f830740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a77d9fb40 0x559a77dabbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:43.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.901+0000 7fe22f830740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a77da0460 0x559a77dac100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:43.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.901+0000 7fe22f830740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a77daab30 0x559a77dac640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:43.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.901+0000 7fe22f830740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe2281bfc60 con 0x559a77daab30 2024-04-24T09:12:43.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.901+0000 7fe22f830740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe2281bf590 con 0x559a77da0460 2024-04-24T09:12:43.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.901+0000 7fe22f830740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe2281bc790 con 0x559a77d9fb40 2024-04-24T09:12:43.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.902+0000 7fe22f830740 10 client.0 _ll_register_callbacks cb 0x7fe220016070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:12:43.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.902+0000 7fe225400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a77da0460 0x559a77dac100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:43.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.902+0000 7fe225e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a77d9fb40 0x559a77dabbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:43.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.902+0000 7fe225e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a77d9fb40 0x559a77dabbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:57230/0 (socket says 172.21.15.83:57230) 2024-04-24T09:12:43.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.902+0000 7fe225e00640 1 -- 172.21.15.83:0/2391944786 learned_addr learned my addr 172.21.15.83:0/2391944786 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:12:43.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.902+0000 7fe226800640 1 --2- 172.21.15.83:0/2391944786 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a77daab30 0x559a77dac640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:43.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.903+0000 7fe225e00640 1 -- 172.21.15.83:0/2391944786 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a77da0460 msgr2=0x559a77dac100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:43.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.903+0000 7fe225e00640 1 --2- 172.21.15.83:0/2391944786 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a77da0460 0x559a77dac100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:43.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.903+0000 7fe225e00640 1 -- 172.21.15.83:0/2391944786 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a77daab30 msgr2=0x559a77dac640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:43.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.903+0000 7fe225e00640 1 --2- 172.21.15.83:0/2391944786 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a77daab30 0x559a77dac640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:43.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.903+0000 7fe225e00640 1 -- 172.21.15.83:0/2391944786 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a77db1420 con 0x559a77d9fb40 2024-04-24T09:12:43.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.904+0000 7fe225e00640 1 --2- 172.21.15.83:0/2391944786 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a77d9fb40 0x559a77dabbc0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe2180f11f0 tx=0x7fe2180ee180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d198681113522f65 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:43.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.904+0000 7fe207400640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-04-24T09:12:43.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.905+0000 7fe207400640 1 -- 172.21.15.83:0/2391944786 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2180c5650 con 0x559a77d9fb40 2024-04-24T09:12:43.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.905+0000 7fe207400640 1 -- 172.21.15.83:0/2391944786 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9583+0+0 (secure 0 0 0) 0x7fe2180b9050 con 0x559a77d9fb40 2024-04-24T09:12:43.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.905+0000 7fe22f830740 1 -- 172.21.15.83:0/2391944786 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2180bd0e0 con 0x559a77d9fb40 2024-04-24T09:12:43.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.907+0000 7fe207400640 1 -- 172.21.15.83:0/2391944786 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7fe2180bc9e0 con 0x559a77d9fb40 2024-04-24T09:12:43.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.907+0000 7fe22f830740 10 client.29701 fetch_fsmap learned FSMap version 64 2024-04-24T09:12:43.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.907+0000 7fe22f830740 1 -- 172.21.15.83:0/2391944786 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x559a77e21860 con 0x559a77d9fb40 2024-04-24T09:12:43.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.908+0000 7fe207400640 1 -- 172.21.15.83:0/2391944786 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe2180bd0e0 con 0x559a77d9fb40 2024-04-24T09:12:43.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.908+0000 7fe207400640 1 -- 172.21.15.83:0/2391944786 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 64) ==== 8374+0+0 (secure 0 0 0) 0x7fe2180dbba0 con 0x559a77d9fb40 2024-04-24T09:12:43.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.908+0000 7fe22f830740 10 client.29701 fetch_fsmap finished waiting for FSMap version 64 2024-04-24T09:12:43.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.908+0000 7fe22f830740 10 client.29701 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-24T09:12:43.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.908+0000 7fe22f830740 20 client.29701 populate_metadata read hostname 'smithi083' 2024-04-24T09:12:43.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.909+0000 7fe22f830740 1 --2- 172.21.15.83:0/2391944786 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x559a77e21ed0 0x559a77e242c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:43.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.909+0000 7fe22f830740 4 client.29701 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:12:43.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.909+0000 7fe22f830740 1 -- 172.21.15.83:0/2391944786 --> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559a77d0ca60 con 0x559a77e21ed0 2024-04-24T09:12:43.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.912+0000 7fe225400640 1 --2- 172.21.15.83:0/2391944786 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x559a77e21ed0 0x559a77e242c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:43.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.913+0000 7fe225400640 1 --2- 172.21.15.83:0/2391944786 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x559a77e21ed0 0x559a77e242c0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ac9c77a3f6763fa7 server_cookie=243be707208a057d in_seq=0 out_seq=0 2024-04-24T09:12:43.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.913+0000 7fe207400640 10 client.29701 ms_handle_connect on v2:172.21.15.189:6838/2548965336 2024-04-24T09:12:43.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.914+0000 7fe207400640 1 -- 172.21.15.83:0/2391944786 <== mds.4 v2:172.21.15.189:6838/2548965336 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559a77d0ca60 con 0x559a77e21ed0 2024-04-24T09:12:43.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.914+0000 7fe207400640 10 client.29701 handle_command_reply: tid=0 2024-04-24T09:12:43.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.914+0000 7fe22f830740 1 client.29701 shutdown 2024-04-24T09:12:43.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.914+0000 7fe22f830740 1 -- 172.21.15.83:0/2391944786 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a77d9fb40 msgr2=0x559a77dabbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:43.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.914+0000 7fe22f830740 1 --2- 172.21.15.83:0/2391944786 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a77d9fb40 0x559a77dabbc0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe2180f11f0 tx=0x7fe2180ee180 comp rx=0 tx=0).stop 2024-04-24T09:12:43.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.914+0000 7fe22f830740 1 -- 172.21.15.83:0/2391944786 shutdown_connections 2024-04-24T09:12:43.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.914+0000 7fe22f830740 1 --2- 172.21.15.83:0/2391944786 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x559a77e21ed0 0x559a77e242c0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:43.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.914+0000 7fe22f830740 1 --2- 172.21.15.83:0/2391944786 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559a77daab30 0x559a77dac640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:43.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.914+0000 7fe22f830740 1 --2- 172.21.15.83:0/2391944786 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559a77da0460 0x559a77dac100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:43.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.914+0000 7fe22f830740 1 --2- 172.21.15.83:0/2391944786 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a77d9fb40 0x559a77dabbc0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:43.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.914+0000 7fe22f830740 1 -- 172.21.15.83:0/2391944786 >> 172.21.15.83:0/2391944786 conn(0x559a77d9be60 msgr2=0x559a77d9a6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:43.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.914+0000 7fe207400640 0 client.29701 ms_handle_reset on v2:172.21.15.189:6838/2548965336 2024-04-24T09:12:43.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.915+0000 7fe22f830740 1 -- 172.21.15.83:0/2391944786 shutdown_connections 2024-04-24T09:12:43.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.915+0000 7fe22f830740 1 -- 172.21.15.83:0/2391944786 wait complete. 2024-04-24T09:12:43.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.916+0000 7fe22f830740 20 client.29701 trim_cache size 0 max 16384 2024-04-24T09:12:43.916 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:12:43.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.917+0000 7fe227200640 1 -- 172.21.15.83:0/42234079 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe20c0a3da0 msgr2=0x7fe20c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:43.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.917+0000 7fe227200640 1 --2- 172.21.15.83:0/42234079 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe20c0a3da0 0x7fe20c0a6260 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fe220005fd0 tx=0x7fe220009540 comp rx=0 tx=0).stop 2024-04-24T09:12:43.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.917+0000 7fe227200640 1 -- 172.21.15.83:0/42234079 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe228108540 msgr2=0x7fe2280737e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:43.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.917+0000 7fe227200640 1 --2- 172.21.15.83:0/42234079 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe228108540 0x7fe2280737e0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fe218008000 tx=0x7fe2180042f0 comp rx=0 tx=0).stop 2024-04-24T09:12:43.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.917+0000 7fe227200640 1 -- 172.21.15.83:0/42234079 shutdown_connections 2024-04-24T09:12:43.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.917+0000 7fe227200640 1 --2- 172.21.15.83:0/42234079 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe20c0a3da0 0x7fe20c0a6260 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:43.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.917+0000 7fe227200640 1 --2- 172.21.15.83:0/42234079 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe22806dd90 0x7fe22806e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:43.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.917+0000 7fe227200640 1 --2- 172.21.15.83:0/42234079 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe228108ee0 0x7fe22806d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:43.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.917+0000 7fe227200640 1 --2- 172.21.15.83:0/42234079 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe228108540 0x7fe2280737e0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:43.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.917+0000 7fe227200640 1 -- 172.21.15.83:0/42234079 >> 172.21.15.83:0/42234079 conn(0x7fe228077e20 msgr2=0x7fe228079150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:43.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.918+0000 7fe227200640 1 -- 172.21.15.83:0/42234079 shutdown_connections 2024-04-24T09:12:43.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:43.918+0000 7fe227200640 1 -- 172.21.15.83:0/42234079 wait complete. 2024-04-24T09:12:43.948 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-24T09:12:44.189 INFO:tasks.workunit.client.0.smithi083.stdout: 10 54207 6.60 MB/sec execute 315 sec latency 85.823 ms 2024-04-24T09:12:44.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.231+0000 7feb5be00640 1 -- 172.21.15.83:0/1208842320 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb5c074cc0 msgr2=0x7feb5c112970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:44.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.231+0000 7feb5be00640 1 --2- 172.21.15.83:0/1208842320 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb5c074cc0 0x7feb5c112970 secure :-1 s=READY pgs=2575 cs=0 l=1 rev1=1 crypto rx=0x7feb48009900 tx=0x7feb4802a480 comp rx=0 tx=0).stop 2024-04-24T09:12:44.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.231+0000 7feb5be00640 1 -- 172.21.15.83:0/1208842320 shutdown_connections 2024-04-24T09:12:44.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.231+0000 7feb5be00640 1 --2- 172.21.15.83:0/1208842320 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feb5c112eb0 0x7feb5c1152a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:44.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.231+0000 7feb5be00640 1 --2- 172.21.15.83:0/1208842320 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb5c074cc0 0x7feb5c112970 unknown :-1 s=CLOSED pgs=2575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:44.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.231+0000 7feb5be00640 1 --2- 172.21.15.83:0/1208842320 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb5c0743a0 0x7feb5c074780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:44.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.231+0000 7feb5be00640 1 -- 172.21.15.83:0/1208842320 >> 172.21.15.83:0/1208842320 conn(0x7feb5c0fd9d0 msgr2=0x7feb5c0ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:44.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.231+0000 7feb5be00640 1 -- 172.21.15.83:0/1208842320 shutdown_connections 2024-04-24T09:12:44.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.232+0000 7feb5be00640 1 -- 172.21.15.83:0/1208842320 wait complete. 2024-04-24T09:12:44.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.233+0000 7feb5be00640 1 Processor -- start 2024-04-24T09:12:44.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.233+0000 7feb5be00640 1 -- start start 2024-04-24T09:12:44.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.233+0000 7feb5be00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feb5c0743a0 0x7feb5c073860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:44.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.233+0000 7feb5be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb5c074cc0 0x7feb5c06c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:44.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.233+0000 7feb5be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb5c112eb0 0x7feb5c06ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:44.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.233+0000 7feb5be00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7feb5c118780 con 0x7feb5c112eb0 2024-04-24T09:12:44.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.233+0000 7feb5be00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7feb5c118600 con 0x7feb5c074cc0 2024-04-24T09:12:44.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.233+0000 7feb5be00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7feb5c118900 con 0x7feb5c0743a0 2024-04-24T09:12:44.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.234+0000 7feb5b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb5c112eb0 0x7feb5c06ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:44.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.234+0000 7feb5a000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb5c074cc0 0x7feb5c06c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:44.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.234+0000 7feb5b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb5c112eb0 0x7feb5c06ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44094/0 (socket says 172.21.15.83:44094) 2024-04-24T09:12:44.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.234+0000 7feb5a000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb5c074cc0 0x7feb5c06c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:49870/0 (socket says 172.21.15.83:49870) 2024-04-24T09:12:44.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.234+0000 7feb5b400640 1 -- 172.21.15.83:0/3000795308 learned_addr learned my addr 172.21.15.83:0/3000795308 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:12:44.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.234+0000 7feb5aa00640 1 --2- 172.21.15.83:0/3000795308 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feb5c0743a0 0x7feb5c073860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:44.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.234+0000 7feb5a000640 1 -- 172.21.15.83:0/3000795308 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feb5c0743a0 msgr2=0x7feb5c073860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:44.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.234+0000 7feb5a000640 1 --2- 172.21.15.83:0/3000795308 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feb5c0743a0 0x7feb5c073860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:44.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.234+0000 7feb5a000640 1 -- 172.21.15.83:0/3000795308 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb5c112eb0 msgr2=0x7feb5c06ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:44.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.234+0000 7feb5a000640 1 --2- 172.21.15.83:0/3000795308 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb5c112eb0 0x7feb5c06ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:44.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.234+0000 7feb5a000640 1 -- 172.21.15.83:0/3000795308 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb5c06d480 con 0x7feb5c074cc0 2024-04-24T09:12:44.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.235+0000 7feb5a000640 1 --2- 172.21.15.83:0/3000795308 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb5c074cc0 0x7feb5c06c930 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7feb4802aa70 tx=0x7feb48003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:44.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.235+0000 7feb53400640 1 -- 172.21.15.83:0/3000795308 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb4803f070 con 0x7feb5c074cc0 2024-04-24T09:12:44.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.235+0000 7feb53400640 1 -- 172.21.15.83:0/3000795308 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feb48031b50 con 0x7feb5c074cc0 2024-04-24T09:12:44.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.235+0000 7feb53400640 1 -- 172.21.15.83:0/3000795308 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb48035030 con 0x7feb5c074cc0 2024-04-24T09:12:44.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.235+0000 7feb5be00640 1 -- 172.21.15.83:0/3000795308 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb5c06d710 con 0x7feb5c074cc0 2024-04-24T09:12:44.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.235+0000 7feb5be00640 1 -- 172.21.15.83:0/3000795308 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb5c0715b0 con 0x7feb5c074cc0 2024-04-24T09:12:44.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.240+0000 7feb53400640 1 -- 172.21.15.83:0/3000795308 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7feb48042070 con 0x7feb5c074cc0 2024-04-24T09:12:44.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.241+0000 7feb645ac740 1 Processor -- start 2024-04-24T09:12:44.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.242+0000 7feb645ac740 1 -- start start 2024-04-24T09:12:44.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.242+0000 7feb645ac740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b592035b40 0x55b592035f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:44.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.242+0000 7feb645ac740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b592036460 0x55b5920405f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:44.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.242+0000 7feb645ac740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b592040b30 0x55b592042f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:44.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.242+0000 7feb645ac740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55b591f9b1d0 con 0x55b592040b30 2024-04-24T09:12:44.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.242+0000 7feb645ac740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b591f41450 con 0x55b592035b40 2024-04-24T09:12:44.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.242+0000 7feb645ac740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55b591f97190 con 0x55b592036460 2024-04-24T09:12:44.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.242+0000 7feb5b400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b592035b40 0x55b592035f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:44.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.243+0000 7feb5aa00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b592036460 0x55b5920405f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:44.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.243+0000 7feb5aa00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b592036460 0x55b5920405f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:57248/0 (socket says 172.21.15.83:57248) 2024-04-24T09:12:44.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.243+0000 7feb5aa00640 1 -- 172.21.15.83:0/877949842 learned_addr learned my addr 172.21.15.83:0/877949842 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:12:44.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.243+0000 7feb5b400640 1 -- 172.21.15.83:0/877949842 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b592036460 msgr2=0x55b5920405f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:44.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.243+0000 7feb5b400640 1 --2- 172.21.15.83:0/877949842 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b592036460 0x55b5920405f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:44.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.243+0000 7feb5b400640 1 -- 172.21.15.83:0/877949842 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b592040b30 msgr2=0x55b592042f20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T09:12:44.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.243+0000 7feb5b400640 1 --2- 172.21.15.83:0/877949842 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b592040b30 0x55b592042f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:44.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.243+0000 7feb53400640 1 --2- 172.21.15.83:0/3000795308 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7feb400a3e20 0x7feb400a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:44.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.243+0000 7feb5b400640 1 -- 172.21.15.83:0/877949842 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b591ffe310 con 0x55b592035b40 2024-04-24T09:12:44.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.243+0000 7feb53400640 1 -- 172.21.15.83:0/3000795308 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 9583+0+0 (secure 0 0 0) 0x7feb480f0e80 con 0x7feb5c074cc0 2024-04-24T09:12:44.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.244+0000 7feb5b400640 1 --2- 172.21.15.83:0/877949842 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b592035b40 0x55b592035f20 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7feb54009f40 tx=0x7feb54010800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de48b228f431ef59 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:44.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.244+0000 7feb5b400640 1 --2- 172.21.15.83:0/3000795308 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7feb400a3e20 0x7feb400a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:44.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.245+0000 7feb47400640 1 -- 172.21.15.83:0/877949842 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb54022070 con 0x55b592035b40 2024-04-24T09:12:44.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.245+0000 7feb47400640 1 -- 172.21.15.83:0/877949842 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feb540113e0 con 0x55b592035b40 2024-04-24T09:12:44.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.245+0000 7feb47400640 1 -- 172.21.15.83:0/877949842 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb54016070 con 0x55b592035b40 2024-04-24T09:12:44.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.246+0000 7feb645ac740 1 -- 172.21.15.83:0/877949842 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b592035b40 msgr2=0x55b592035f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:44.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.246+0000 7feb5b400640 1 --2- 172.21.15.83:0/3000795308 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7feb400a3e20 0x7feb400a62e0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7feb5401f000 tx=0x7feb54022000 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:44.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.246+0000 7feb645ac740 1 --2- 172.21.15.83:0/877949842 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b592035b40 0x55b592035f20 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7feb54009f40 tx=0x7feb54010800 comp rx=0 tx=0).stop 2024-04-24T09:12:44.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.246+0000 7feb645ac740 1 -- 172.21.15.83:0/877949842 shutdown_connections 2024-04-24T09:12:44.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.246+0000 7feb645ac740 1 --2- 172.21.15.83:0/877949842 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b592040b30 0x55b592042f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:44.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.246+0000 7feb645ac740 1 --2- 172.21.15.83:0/877949842 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b592036460 0x55b5920405f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:44.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.246+0000 7feb645ac740 1 --2- 172.21.15.83:0/877949842 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b592035b40 0x55b592035f20 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:44.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.246+0000 7feb645ac740 1 -- 172.21.15.83:0/877949842 >> 172.21.15.83:0/877949842 conn(0x55b592031e60 msgr2=0x55b592032240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:44.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.246+0000 7feb645ac740 1 -- 172.21.15.83:0/877949842 shutdown_connections 2024-04-24T09:12:44.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.247+0000 7feb645ac740 1 -- 172.21.15.83:0/877949842 wait complete. 2024-04-24T09:12:44.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.247+0000 7feb645ac740 1 -- start start 2024-04-24T09:12:44.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.247+0000 7feb645ac740 1 Processor -- start 2024-04-24T09:12:44.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.248+0000 7feb645ac740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b592035b40 0x55b59203d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:44.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.248+0000 7feb645ac740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b592036460 0x55b59203de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:44.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.248+0000 7feb645ac740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b592040b30 0x55b59203e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:44.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.248+0000 7feb645ac740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55b5920439c0 con 0x55b592035b40 2024-04-24T09:12:44.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.248+0000 7feb645ac740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b591f54810 con 0x55b592040b30 2024-04-24T09:12:44.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.248+0000 7feb645ac740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55b592043b40 con 0x55b592036460 2024-04-24T09:12:44.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.248+0000 7feb645ac740 10 client.0 _ll_register_callbacks cb 0x55b591f22140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:12:44.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.248+0000 7feb5aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b592035b40 0x55b59203d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:44.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.248+0000 7feb5aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b592035b40 0x55b59203d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44098/0 (socket says 172.21.15.83:44098) 2024-04-24T09:12:44.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.248+0000 7feb5aa00640 1 -- 172.21.15.83:0/1459076182 learned_addr learned my addr 172.21.15.83:0/1459076182 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:12:44.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.248+0000 7feb5b400640 1 --2- 172.21.15.83:0/1459076182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b592040b30 0x55b59203e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:44.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.249+0000 7feb5a000640 1 --2- 172.21.15.83:0/1459076182 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b592036460 0x55b59203de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:44.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.249+0000 7feb5aa00640 1 -- 172.21.15.83:0/1459076182 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b592036460 msgr2=0x55b59203de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:44.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.249+0000 7feb5aa00640 1 --2- 172.21.15.83:0/1459076182 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b592036460 0x55b59203de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:44.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.249+0000 7feb5aa00640 1 -- 172.21.15.83:0/1459076182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b592040b30 msgr2=0x55b59203e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:44.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.249+0000 7feb5aa00640 1 --2- 172.21.15.83:0/1459076182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b592040b30 0x55b59203e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:44.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.249+0000 7feb5aa00640 1 -- 172.21.15.83:0/1459076182 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b591f653f0 con 0x55b592035b40 2024-04-24T09:12:44.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.249+0000 7feb5aa00640 1 --2- 172.21.15.83:0/1459076182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b592035b40 0x55b59203d920 secure :-1 s=READY pgs=2576 cs=0 l=1 rev1=1 crypto rx=0x7feb4c012700 tx=0x7feb4c012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4157c7604e670b5f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:44.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.249+0000 7feb3be00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:12:44.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.249+0000 7feb3be00640 1 -- 172.21.15.83:0/1459076182 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb4c00f920 con 0x55b592035b40 2024-04-24T09:12:44.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.249+0000 7feb3be00640 1 -- 172.21.15.83:0/1459076182 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9583+0+0 (secure 0 0 0) 0x7feb4c00fac0 con 0x55b592035b40 2024-04-24T09:12:44.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.249+0000 7feb645ac740 1 -- 172.21.15.83:0/1459076182 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb540113e0 con 0x55b592035b40 2024-04-24T09:12:44.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.250+0000 7feb3be00640 1 -- 172.21.15.83:0/1459076182 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7feb4c00fcc0 con 0x55b592035b40 2024-04-24T09:12:44.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.250+0000 7feb645ac740 10 client.22839 fetch_fsmap learned FSMap version 64 2024-04-24T09:12:44.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.250+0000 7feb645ac740 1 -- 172.21.15.83:0/1459076182 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55b591cad3f0 con 0x55b592035b40 2024-04-24T09:12:44.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.250+0000 7feb3be00640 1 -- 172.21.15.83:0/1459076182 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feb540113e0 con 0x55b592035b40 2024-04-24T09:12:44.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.250+0000 7feb3be00640 1 -- 172.21.15.83:0/1459076182 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 64) ==== 8374+0+0 (secure 0 0 0) 0x7feb4c01c420 con 0x55b592035b40 2024-04-24T09:12:44.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.251+0000 7feb645ac740 10 client.22839 fetch_fsmap finished waiting for FSMap version 64 2024-04-24T09:12:44.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.251+0000 7feb645ac740 10 client.22839 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-24T09:12:44.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.251+0000 7feb645ac740 20 client.22839 populate_metadata read hostname 'smithi083' 2024-04-24T09:12:44.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.251+0000 7feb645ac740 1 --2- 172.21.15.83:0/1459076182 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55b592050f70 0x55b592053400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:44.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.251+0000 7feb645ac740 4 client.22839 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:12:44.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.251+0000 7feb645ac740 1 -- 172.21.15.83:0/1459076182 --> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b591fa8bf0 con 0x55b592050f70 2024-04-24T09:12:44.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.251+0000 7feb5a000640 1 --2- 172.21.15.83:0/1459076182 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55b592050f70 0x55b592053400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:44.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.252+0000 7feb5a000640 1 --2- 172.21.15.83:0/1459076182 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55b592050f70 0x55b592053400 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=816833a6ba8f71e9 server_cookie=66eaee75f7f101ec in_seq=0 out_seq=0 2024-04-24T09:12:44.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.252+0000 7feb3be00640 10 client.22839 ms_handle_connect on v2:172.21.15.138:6832/1157967658 2024-04-24T09:12:44.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.253+0000 7feb3be00640 1 -- 172.21.15.83:0/1459076182 <== mds.3 v2:172.21.15.138:6832/1157967658 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b591fa8bf0 con 0x55b592050f70 2024-04-24T09:12:44.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.253+0000 7feb3be00640 10 client.22839 handle_command_reply: tid=0 2024-04-24T09:12:44.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.254+0000 7feb645ac740 1 client.22839 shutdown 2024-04-24T09:12:44.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.254+0000 7feb645ac740 1 -- 172.21.15.83:0/1459076182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b592035b40 msgr2=0x55b59203d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:44.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.254+0000 7feb645ac740 1 --2- 172.21.15.83:0/1459076182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b592035b40 0x55b59203d920 secure :-1 s=READY pgs=2576 cs=0 l=1 rev1=1 crypto rx=0x7feb4c012700 tx=0x7feb4c012bc0 comp rx=0 tx=0).stop 2024-04-24T09:12:44.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.254+0000 7feb645ac740 1 -- 172.21.15.83:0/1459076182 shutdown_connections 2024-04-24T09:12:44.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.254+0000 7feb645ac740 1 --2- 172.21.15.83:0/1459076182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b592040b30 0x55b59203e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:44.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.254+0000 7feb645ac740 1 --2- 172.21.15.83:0/1459076182 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55b592050f70 0x55b592053400 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:44.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.254+0000 7feb645ac740 1 --2- 172.21.15.83:0/1459076182 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b592036460 0x55b59203de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:44.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.254+0000 7feb645ac740 1 --2- 172.21.15.83:0/1459076182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b592035b40 0x55b59203d920 unknown :-1 s=CLOSED pgs=2576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:44.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.254+0000 7feb3be00640 0 client.22839 ms_handle_reset on v2:172.21.15.138:6832/1157967658 2024-04-24T09:12:44.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.254+0000 7feb645ac740 1 -- 172.21.15.83:0/1459076182 >> 172.21.15.83:0/1459076182 conn(0x55b592031e60 msgr2=0x55b592037c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:44.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.254+0000 7feb645ac740 1 -- 172.21.15.83:0/1459076182 shutdown_connections 2024-04-24T09:12:44.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.254+0000 7feb645ac740 1 -- 172.21.15.83:0/1459076182 wait complete. 2024-04-24T09:12:44.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.254+0000 7feb645ac740 20 client.22839 trim_cache size 0 max 16384 2024-04-24T09:12:44.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.279+0000 7feb645ac740 1 Processor -- start 2024-04-24T09:12:44.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.279+0000 7feb645ac740 1 -- start start 2024-04-24T09:12:44.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.279+0000 7feb645ac740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b592035b40 0x55b59204d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:44.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.279+0000 7feb645ac740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b592036460 0x55b59204c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:44.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.279+0000 7feb645ac740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b592040b30 0x55b59203dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:44.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.279+0000 7feb645ac740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7feb5c1be330 con 0x55b592040b30 2024-04-24T09:12:44.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.279+0000 7feb645ac740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7feb5c1bff00 con 0x55b592036460 2024-04-24T09:12:44.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.279+0000 7feb645ac740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7feb5c1bea00 con 0x55b592035b40 2024-04-24T09:12:44.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.280+0000 7feb5a000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b592036460 0x55b59204c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:44.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.280+0000 7feb5a000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b592036460 0x55b59204c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:49904/0 (socket says 172.21.15.83:49904) 2024-04-24T09:12:44.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.280+0000 7feb5a000640 1 -- 172.21.15.83:0/269011399 learned_addr learned my addr 172.21.15.83:0/269011399 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:12:44.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.280+0000 7feb5b400640 1 --2- 172.21.15.83:0/269011399 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b592040b30 0x55b59203dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:44.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.280+0000 7feb5aa00640 1 --2- 172.21.15.83:0/269011399 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b592035b40 0x55b59204d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:44.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.281+0000 7feb5b400640 1 -- 172.21.15.83:0/269011399 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b592035b40 msgr2=0x55b59204d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:44.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.281+0000 7feb5b400640 1 --2- 172.21.15.83:0/269011399 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b592035b40 0x55b59204d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:44.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.281+0000 7feb5b400640 1 -- 172.21.15.83:0/269011399 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b592036460 msgr2=0x55b59204c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:44.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.281+0000 7feb5b400640 1 --2- 172.21.15.83:0/269011399 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b592036460 0x55b59204c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:44.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.281+0000 7feb5b400640 1 -- 172.21.15.83:0/269011399 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b591d777d0 con 0x55b592040b30 2024-04-24T09:12:44.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.281+0000 7feb5a000640 1 --2- 172.21.15.83:0/269011399 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b592036460 0x55b59204c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:12:44.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.282+0000 7feb5b400640 1 --2- 172.21.15.83:0/269011399 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b592040b30 0x55b59203dff0 secure :-1 s=READY pgs=2577 cs=0 l=1 rev1=1 crypto rx=0x7feb54011dd0 tx=0x7feb5401e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32663bd5e22c84db server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:44.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.282+0000 7feb46a00640 1 -- 172.21.15.83:0/269011399 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb54037070 con 0x55b592040b30 2024-04-24T09:12:44.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.283+0000 7feb46a00640 1 -- 172.21.15.83:0/269011399 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feb540127b0 con 0x55b592040b30 2024-04-24T09:12:44.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.283+0000 7feb46a00640 1 -- 172.21.15.83:0/269011399 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb54016040 con 0x55b592040b30 2024-04-24T09:12:44.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.284+0000 7feb645ac740 1 -- 172.21.15.83:0/269011399 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b592040b30 msgr2=0x55b59203dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:44.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.284+0000 7feb645ac740 1 --2- 172.21.15.83:0/269011399 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b592040b30 0x55b59203dff0 secure :-1 s=READY pgs=2577 cs=0 l=1 rev1=1 crypto rx=0x7feb54011dd0 tx=0x7feb5401e040 comp rx=0 tx=0).stop 2024-04-24T09:12:44.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.284+0000 7feb645ac740 1 -- 172.21.15.83:0/269011399 shutdown_connections 2024-04-24T09:12:44.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.284+0000 7feb645ac740 1 --2- 172.21.15.83:0/269011399 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b592040b30 0x55b59203dff0 unknown :-1 s=CLOSED pgs=2577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:44.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.284+0000 7feb645ac740 1 --2- 172.21.15.83:0/269011399 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b592036460 0x55b59204c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:44.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.284+0000 7feb645ac740 1 --2- 172.21.15.83:0/269011399 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b592035b40 0x55b59204d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:44.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.284+0000 7feb645ac740 1 -- 172.21.15.83:0/269011399 >> 172.21.15.83:0/269011399 conn(0x55b592031e60 msgr2=0x55b59203d8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:44.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.284+0000 7feb645ac740 1 -- 172.21.15.83:0/269011399 shutdown_connections 2024-04-24T09:12:44.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.285+0000 7feb645ac740 1 -- 172.21.15.83:0/269011399 wait complete. 2024-04-24T09:12:44.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.285+0000 7feb645ac740 1 -- start start 2024-04-24T09:12:44.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.285+0000 7feb645ac740 1 Processor -- start 2024-04-24T09:12:44.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.286+0000 7feb645ac740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b592035b40 0x55b592048e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:44.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.286+0000 7feb645ac740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b592036460 0x55b592049350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:44.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.286+0000 7feb645ac740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b592040b30 0x55b592044390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:44.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.286+0000 7feb645ac740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7feb5c1bf800 con 0x55b592040b30 2024-04-24T09:12:44.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.286+0000 7feb645ac740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7feb5c1bf130 con 0x55b592035b40 2024-04-24T09:12:44.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.286+0000 7feb645ac740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7feb5c1bc330 con 0x55b592036460 2024-04-24T09:12:44.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.286+0000 7feb645ac740 10 client.0 _ll_register_callbacks cb 0x7feb4c009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:12:44.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.286+0000 7feb5b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b592040b30 0x55b592044390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:44.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.286+0000 7feb5aa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b592035b40 0x55b592048e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:44.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.286+0000 7feb5aa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b592035b40 0x55b592048e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:49916/0 (socket says 172.21.15.83:49916) 2024-04-24T09:12:44.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.286+0000 7feb5b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b592040b30 0x55b592044390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44118/0 (socket says 172.21.15.83:44118) 2024-04-24T09:12:44.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.286+0000 7feb5aa00640 1 -- 172.21.15.83:0/3229542886 learned_addr learned my addr 172.21.15.83:0/3229542886 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:12:44.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.287+0000 7feb5a000640 1 --2- 172.21.15.83:0/3229542886 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b592036460 0x55b592049350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:44.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.287+0000 7feb5aa00640 1 -- 172.21.15.83:0/3229542886 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b592036460 msgr2=0x55b592049350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:44.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.287+0000 7feb5aa00640 1 --2- 172.21.15.83:0/3229542886 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b592036460 0x55b592049350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:44.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.287+0000 7feb5aa00640 1 -- 172.21.15.83:0/3229542886 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b592040b30 msgr2=0x55b592044390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:44.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.287+0000 7feb5aa00640 1 --2- 172.21.15.83:0/3229542886 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b592040b30 0x55b592044390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:44.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.287+0000 7feb5aa00640 1 -- 172.21.15.83:0/3229542886 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b592044b90 con 0x55b592035b40 2024-04-24T09:12:44.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.287+0000 7feb5b400640 1 --2- 172.21.15.83:0/3229542886 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b592040b30 0x55b592044390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:12:44.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.287+0000 7feb5aa00640 1 --2- 172.21.15.83:0/3229542886 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b592035b40 0x55b592048e10 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7feb4c012700 tx=0x7feb4c01c630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27fbb57edca2de60 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:44.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.287+0000 7feb3be00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:12:44.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.288+0000 7feb3be00640 1 -- 172.21.15.83:0/3229542886 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb4c02f060 con 0x55b592035b40 2024-04-24T09:12:44.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.288+0000 7feb645ac740 1 -- 172.21.15.83:0/3229542886 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb540127b0 con 0x55b592035b40 2024-04-24T09:12:44.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.288+0000 7feb3be00640 1 -- 172.21.15.83:0/3229542886 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9583+0+0 (secure 0 0 0) 0x7feb4c01ccc0 con 0x55b592035b40 2024-04-24T09:12:44.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.288+0000 7feb3be00640 1 -- 172.21.15.83:0/3229542886 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7feb4c00fe60 con 0x55b592035b40 2024-04-24T09:12:44.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.288+0000 7feb645ac740 10 client.39884 fetch_fsmap learned FSMap version 64 2024-04-24T09:12:44.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.288+0000 7feb645ac740 1 -- 172.21.15.83:0/3229542886 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55b5920b7200 con 0x55b592035b40 2024-04-24T09:12:44.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.289+0000 7feb3be00640 1 -- 172.21.15.83:0/3229542886 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feb4c005560 con 0x55b592035b40 2024-04-24T09:12:44.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.289+0000 7feb3be00640 1 -- 172.21.15.83:0/3229542886 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 64) ==== 8374+0+0 (secure 0 0 0) 0x7feb4c003930 con 0x55b592035b40 2024-04-24T09:12:44.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.289+0000 7feb645ac740 10 client.39884 fetch_fsmap finished waiting for FSMap version 64 2024-04-24T09:12:44.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.289+0000 7feb645ac740 10 client.39884 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-24T09:12:44.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.289+0000 7feb645ac740 20 client.39884 populate_metadata read hostname 'smithi083' 2024-04-24T09:12:44.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.289+0000 7feb645ac740 1 --2- 172.21.15.83:0/3229542886 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55b5920b79b0 0x55b5920b9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:44.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.289+0000 7feb645ac740 4 client.39884 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:12:44.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.289+0000 7feb645ac740 1 -- 172.21.15.83:0/3229542886 --> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7feb540127b0 con 0x55b5920b79b0 2024-04-24T09:12:44.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.289+0000 7feb5a000640 1 --2- 172.21.15.83:0/3229542886 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55b5920b79b0 0x55b5920b9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:44.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.290+0000 7feb5a000640 1 --2- 172.21.15.83:0/3229542886 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55b5920b79b0 0x55b5920b9dd0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3fb8da2d83155ff6 server_cookie=b447bda4d2ca839 in_seq=0 out_seq=0 2024-04-24T09:12:44.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.290+0000 7feb3be00640 10 client.39884 ms_handle_connect on v2:172.21.15.138:6832/1157967658 2024-04-24T09:12:44.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.290+0000 7feb3be00640 1 -- 172.21.15.83:0/3229542886 <== mds.3 v2:172.21.15.138:6832/1157967658 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7feb540127b0 con 0x55b5920b79b0 2024-04-24T09:12:44.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.290+0000 7feb3be00640 10 client.39884 handle_command_reply: tid=0 2024-04-24T09:12:44.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.290+0000 7feb645ac740 1 client.39884 shutdown 2024-04-24T09:12:44.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.291+0000 7feb645ac740 1 -- 172.21.15.83:0/3229542886 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b592035b40 msgr2=0x55b592048e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:44.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.291+0000 7feb645ac740 1 --2- 172.21.15.83:0/3229542886 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b592035b40 0x55b592048e10 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7feb4c012700 tx=0x7feb4c01c630 comp rx=0 tx=0).stop 2024-04-24T09:12:44.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.291+0000 7feb645ac740 1 -- 172.21.15.83:0/3229542886 shutdown_connections 2024-04-24T09:12:44.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.291+0000 7feb645ac740 1 --2- 172.21.15.83:0/3229542886 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b592040b30 0x55b592044390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:44.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.291+0000 7feb645ac740 1 --2- 172.21.15.83:0/3229542886 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55b5920b79b0 0x55b5920b9dd0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:44.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.291+0000 7feb645ac740 1 --2- 172.21.15.83:0/3229542886 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b592036460 0x55b592049350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:44.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.291+0000 7feb645ac740 1 --2- 172.21.15.83:0/3229542886 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b592035b40 0x55b592048e10 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:44.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.291+0000 7feb3be00640 0 client.39884 ms_handle_reset on v2:172.21.15.138:6832/1157967658 2024-04-24T09:12:44.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.291+0000 7feb645ac740 1 -- 172.21.15.83:0/3229542886 >> 172.21.15.83:0/3229542886 conn(0x55b592031e60 msgr2=0x55b59203e150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:44.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.291+0000 7feb645ac740 1 -- 172.21.15.83:0/3229542886 shutdown_connections 2024-04-24T09:12:44.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.291+0000 7feb645ac740 1 -- 172.21.15.83:0/3229542886 wait complete. 2024-04-24T09:12:44.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.291+0000 7feb645ac740 20 client.39884 trim_cache size 0 max 16384 2024-04-24T09:12:44.293 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:12:44.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.292+0000 7feb5be00640 1 -- 172.21.15.83:0/3000795308 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7feb400a3e20 msgr2=0x7feb400a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:44.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.292+0000 7feb5be00640 1 --2- 172.21.15.83:0/3000795308 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7feb400a3e20 0x7feb400a62e0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7feb5401f000 tx=0x7feb54022000 comp rx=0 tx=0).stop 2024-04-24T09:12:44.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.292+0000 7feb5be00640 1 -- 172.21.15.83:0/3000795308 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb5c074cc0 msgr2=0x7feb5c06c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:44.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.292+0000 7feb5be00640 1 --2- 172.21.15.83:0/3000795308 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb5c074cc0 0x7feb5c06c930 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7feb4802aa70 tx=0x7feb48003b90 comp rx=0 tx=0).stop 2024-04-24T09:12:44.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.292+0000 7feb5be00640 1 -- 172.21.15.83:0/3000795308 shutdown_connections 2024-04-24T09:12:44.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.292+0000 7feb5be00640 1 --2- 172.21.15.83:0/3000795308 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7feb400a3e20 0x7feb400a62e0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:44.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.292+0000 7feb5be00640 1 --2- 172.21.15.83:0/3000795308 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb5c112eb0 0x7feb5c06ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:44.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.292+0000 7feb5be00640 1 --2- 172.21.15.83:0/3000795308 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb5c074cc0 0x7feb5c06c930 secure :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7feb4802aa70 tx=0x7feb48003b90 comp rx=0 tx=0).stop 2024-04-24T09:12:44.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.292+0000 7feb5be00640 1 --2- 172.21.15.83:0/3000795308 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feb5c0743a0 0x7feb5c073860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:44.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.292+0000 7feb5be00640 1 -- 172.21.15.83:0/3000795308 >> 172.21.15.83:0/3000795308 conn(0x7feb5c0fd9d0 msgr2=0x7feb5c0ff2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:44.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.292+0000 7feb5be00640 1 -- 172.21.15.83:0/3000795308 shutdown_connections 2024-04-24T09:12:44.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.292+0000 7feb5be00640 1 -- 172.21.15.83:0/3000795308 wait complete. 2024-04-24T09:12:44.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:12:44 smithi083 ceph-mon[27907]: pgmap v666: 129 pgs: 129 active+clean; 845 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.6 MiB/s wr, 373 op/s 2024-04-24T09:12:44.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:12:44 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/800442258' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:12:44.317 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T09:12:44.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:12:44 smithi189 ceph-mon[31791]: pgmap v666: 129 pgs: 129 active+clean; 845 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.6 MiB/s wr, 373 op/s 2024-04-24T09:12:44.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:12:44 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/800442258' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:12:44.457 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:12:44 smithi138 ceph-mon[31441]: pgmap v666: 129 pgs: 129 active+clean; 845 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.6 MiB/s wr, 373 op/s 2024-04-24T09:12:44.458 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:12:44 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/800442258' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:12:44.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.559+0000 7fa091800640 1 -- 172.21.15.83:0/3652056644 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa08c1044b0 msgr2=0x7fa08c104890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:44.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.559+0000 7fa091800640 1 --2- 172.21.15.83:0/3652056644 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa08c1044b0 0x7fa08c104890 secure :-1 s=READY pgs=2578 cs=0 l=1 rev1=1 crypto rx=0x7fa07c009900 tx=0x7fa07c02a410 comp rx=0 tx=0).stop 2024-04-24T09:12:44.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.559+0000 7fa091800640 1 -- 172.21.15.83:0/3652056644 shutdown_connections 2024-04-24T09:12:44.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.559+0000 7fa091800640 1 --2- 172.21.15.83:0/3652056644 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa08c109580 0x7fa08c111150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:44.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.559+0000 7fa091800640 1 --2- 172.21.15.83:0/3652056644 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa08c104e60 0x7fa08c108ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:44.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.559+0000 7fa091800640 1 --2- 172.21.15.83:0/3652056644 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa08c1044b0 0x7fa08c104890 unknown :-1 s=CLOSED pgs=2578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:44.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.559+0000 7fa091800640 1 -- 172.21.15.83:0/3652056644 >> 172.21.15.83:0/3652056644 conn(0x7fa08c0ffd90 msgr2=0x7fa08c1021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:44.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.560+0000 7fa091800640 1 -- 172.21.15.83:0/3652056644 shutdown_connections 2024-04-24T09:12:44.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.560+0000 7fa091800640 1 -- 172.21.15.83:0/3652056644 wait complete. 2024-04-24T09:12:44.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.561+0000 7fa091800640 1 Processor -- start 2024-04-24T09:12:44.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.561+0000 7fa091800640 1 -- start start 2024-04-24T09:12:44.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.562+0000 7fa091800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa08c1044b0 0x7fa08c1a07b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:44.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.562+0000 7fa091800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa08c109580 0x7fa08c1a0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:44.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.562+0000 7fa091800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa08c19a9b0 0x7fa08c19ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:44.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.562+0000 7fa091800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa08c1144f0 con 0x7fa08c1044b0 2024-04-24T09:12:44.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.562+0000 7fa091800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa08c114370 con 0x7fa08c19a9b0 2024-04-24T09:12:44.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.562+0000 7fa091800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa08c114670 con 0x7fa08c109580 2024-04-24T09:12:44.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.562+0000 7fa08aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa08c1044b0 0x7fa08c1a07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:44.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.562+0000 7fa08b400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa08c19a9b0 0x7fa08c19ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:44.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.562+0000 7fa08aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa08c1044b0 0x7fa08c1a07b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44126/0 (socket says 172.21.15.83:44126) 2024-04-24T09:12:44.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.562+0000 7fa08aa00640 1 -- 172.21.15.83:0/4074454658 learned_addr learned my addr 172.21.15.83:0/4074454658 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:12:44.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.562+0000 7fa08b400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa08c19a9b0 0x7fa08c19ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:49928/0 (socket says 172.21.15.83:49928) 2024-04-24T09:12:44.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.562+0000 7fa08a000640 1 --2- 172.21.15.83:0/4074454658 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa08c109580 0x7fa08c1a0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:44.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.563+0000 7fa08aa00640 1 -- 172.21.15.83:0/4074454658 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa08c109580 msgr2=0x7fa08c1a0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:44.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.563+0000 7fa08aa00640 1 --2- 172.21.15.83:0/4074454658 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa08c109580 0x7fa08c1a0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:44.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.563+0000 7fa08aa00640 1 -- 172.21.15.83:0/4074454658 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa08c19a9b0 msgr2=0x7fa08c19ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:44.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.563+0000 7fa08aa00640 1 --2- 172.21.15.83:0/4074454658 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa08c19a9b0 0x7fa08c19ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:44.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.563+0000 7fa08aa00640 1 -- 172.21.15.83:0/4074454658 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa08c19b4d0 con 0x7fa08c1044b0 2024-04-24T09:12:44.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.563+0000 7fa08aa00640 1 --2- 172.21.15.83:0/4074454658 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa08c1044b0 0x7fa08c1a07b0 secure :-1 s=READY pgs=2579 cs=0 l=1 rev1=1 crypto rx=0x7fa07c005940 tx=0x7fa07c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:44.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.563+0000 7fa083400640 1 -- 172.21.15.83:0/4074454658 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa07c03f030 con 0x7fa08c1044b0 2024-04-24T09:12:44.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.564+0000 7fa091800640 1 -- 172.21.15.83:0/4074454658 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa08c19b760 con 0x7fa08c1044b0 2024-04-24T09:12:44.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.564+0000 7fa083400640 1 -- 172.21.15.83:0/4074454658 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa07c003cb0 con 0x7fa08c1044b0 2024-04-24T09:12:44.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.564+0000 7fa083400640 1 -- 172.21.15.83:0/4074454658 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa07c035070 con 0x7fa08c1044b0 2024-04-24T09:12:44.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.564+0000 7fa091800640 1 -- 172.21.15.83:0/4074454658 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa08c1a7960 con 0x7fa08c1044b0 2024-04-24T09:12:44.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.568+0000 7fa083400640 1 -- 172.21.15.83:0/4074454658 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa07c042030 con 0x7fa08c1044b0 2024-04-24T09:12:44.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.568+0000 7fa094653740 1 Processor -- start 2024-04-24T09:12:44.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.569+0000 7fa094653740 1 -- start start 2024-04-24T09:12:44.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.569+0000 7fa094653740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55623cabb950 0x55623cabbd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:44.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.569+0000 7fa094653740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55623cabc270 0x55623cac6400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:44.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.569+0000 7fa094653740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55623cac6940 0x55623cac8d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:44.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.569+0000 7fa094653740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55623ca211d0 con 0x55623cabc270 2024-04-24T09:12:44.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.569+0000 7fa094653740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55623c9c7450 con 0x55623cabb950 2024-04-24T09:12:44.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.569+0000 7fa094653740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55623ca1d190 con 0x55623cac6940 2024-04-24T09:12:44.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.570+0000 7fa08b400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55623cabb950 0x55623cabbd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:44.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.570+0000 7fa08b400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55623cabb950 0x55623cabbd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:49942/0 (socket says 172.21.15.83:49942) 2024-04-24T09:12:44.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.570+0000 7fa08b400640 1 -- 172.21.15.83:0/919287572 learned_addr learned my addr 172.21.15.83:0/919287572 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:12:44.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.570+0000 7fa08aa00640 1 --2- 172.21.15.83:0/919287572 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55623cabc270 0x55623cac6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:44.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.570+0000 7fa083400640 1 --2- 172.21.15.83:0/4074454658 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa0700a3d20 0x7fa0700a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:44.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.570+0000 7fa08b400640 1 -- 172.21.15.83:0/919287572 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55623cac6940 msgr2=0x55623cac8d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:44.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.570+0000 7fa08b400640 1 --2- 172.21.15.83:0/919287572 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55623cac6940 0x55623cac8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:44.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.570+0000 7fa08b400640 1 -- 172.21.15.83:0/919287572 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55623cabc270 msgr2=0x55623cac6400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:44.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.570+0000 7fa08b400640 1 --2- 172.21.15.83:0/919287572 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55623cabc270 0x55623cac6400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:44.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.570+0000 7fa08b400640 1 -- 172.21.15.83:0/919287572 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55623ca84310 con 0x55623cabb950 2024-04-24T09:12:44.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.570+0000 7fa083400640 1 -- 172.21.15.83:0/4074454658 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 9583+0+0 (secure 0 0 0) 0x7fa07c0f0fa0 con 0x7fa08c1044b0 2024-04-24T09:12:44.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.571+0000 7fa08b400640 1 --2- 172.21.15.83:0/4074454658 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa0700a3d20 0x7fa0700a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:44.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.571+0000 7fa08b400640 1 --2- 172.21.15.83:0/919287572 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55623cabb950 0x55623cabbd30 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fa084009f70 tx=0x7fa084010800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e4a54f16af4d517 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:44.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.572+0000 7fa08b400640 1 --2- 172.21.15.83:0/4074454658 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa0700a3d20 0x7fa0700a61e0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fa08401f000 tx=0x7fa084022000 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:44.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.572+0000 7fa077e00640 1 -- 172.21.15.83:0/919287572 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa084022070 con 0x55623cabb950 2024-04-24T09:12:44.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.572+0000 7fa077e00640 1 -- 172.21.15.83:0/919287572 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa0840113e0 con 0x55623cabb950 2024-04-24T09:12:44.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.572+0000 7fa077e00640 1 -- 172.21.15.83:0/919287572 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa084016070 con 0x55623cabb950 2024-04-24T09:12:44.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.573+0000 7fa094653740 1 -- 172.21.15.83:0/919287572 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55623cabb950 msgr2=0x55623cabbd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:44.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.573+0000 7fa094653740 1 --2- 172.21.15.83:0/919287572 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55623cabb950 0x55623cabbd30 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fa084009f70 tx=0x7fa084010800 comp rx=0 tx=0).stop 2024-04-24T09:12:44.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.573+0000 7fa094653740 1 -- 172.21.15.83:0/919287572 shutdown_connections 2024-04-24T09:12:44.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.573+0000 7fa094653740 1 --2- 172.21.15.83:0/919287572 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55623cac6940 0x55623cac8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:44.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.573+0000 7fa094653740 1 --2- 172.21.15.83:0/919287572 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55623cabc270 0x55623cac6400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:44.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.573+0000 7fa094653740 1 --2- 172.21.15.83:0/919287572 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55623cabb950 0x55623cabbd30 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:44.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.573+0000 7fa094653740 1 -- 172.21.15.83:0/919287572 >> 172.21.15.83:0/919287572 conn(0x55623cab7c70 msgr2=0x55623cab8050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:44.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.573+0000 7fa094653740 1 -- 172.21.15.83:0/919287572 shutdown_connections 2024-04-24T09:12:44.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.573+0000 7fa094653740 1 -- 172.21.15.83:0/919287572 wait complete. 2024-04-24T09:12:44.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.574+0000 7fa094653740 1 -- start start 2024-04-24T09:12:44.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.575+0000 7fa094653740 1 Processor -- start 2024-04-24T09:12:44.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.575+0000 7fa094653740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55623cabb950 0x55623cac62d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:44.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.575+0000 7fa094653740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55623cabc270 0x55623cac35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:44.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.575+0000 7fa094653740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55623cac6940 0x55623cac3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:44.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.575+0000 7fa094653740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55623cac97d0 con 0x55623cabb950 2024-04-24T09:12:44.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.576+0000 7fa094653740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55623c9da810 con 0x55623cabc270 2024-04-24T09:12:44.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.576+0000 7fa094653740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55623cac9950 con 0x55623cac6940 2024-04-24T09:12:44.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.576+0000 7fa094653740 10 client.0 _ll_register_callbacks cb 0x55623c9a8140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:12:44.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.576+0000 7fa08aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55623cabb950 0x55623cac62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:44.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.576+0000 7fa08aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55623cabb950 0x55623cac62d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44138/0 (socket says 172.21.15.83:44138) 2024-04-24T09:12:44.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.576+0000 7fa08aa00640 1 -- 172.21.15.83:0/2594333373 learned_addr learned my addr 172.21.15.83:0/2594333373 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:12:44.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.576+0000 7fa08a000640 1 --2- 172.21.15.83:0/2594333373 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55623cabc270 0x55623cac35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:44.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.576+0000 7fa08aa00640 1 -- 172.21.15.83:0/2594333373 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55623cac6940 msgr2=0x55623cac3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:44.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.576+0000 7fa08aa00640 1 --2- 172.21.15.83:0/2594333373 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55623cac6940 0x55623cac3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:44.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.576+0000 7fa08aa00640 1 -- 172.21.15.83:0/2594333373 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55623cabc270 msgr2=0x55623cac35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:44.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.576+0000 7fa08aa00640 1 --2- 172.21.15.83:0/2594333373 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55623cabc270 0x55623cac35c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:44.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.576+0000 7fa08aa00640 1 -- 172.21.15.83:0/2594333373 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55623c9eb3f0 con 0x55623cabb950 2024-04-24T09:12:44.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.577+0000 7fa08aa00640 1 --2- 172.21.15.83:0/2594333373 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55623cabb950 0x55623cac62d0 secure :-1 s=READY pgs=2580 cs=0 l=1 rev1=1 crypto rx=0x7fa07c0ee8d0 tx=0x7fa07c0b8760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4cf3a7caed85a53 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:44.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.577+0000 7fa074c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:12:44.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.577+0000 7fa074c00640 1 -- 172.21.15.83:0/2594333373 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa07c049070 con 0x55623cabb950 2024-04-24T09:12:44.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.577+0000 7fa074c00640 1 -- 172.21.15.83:0/2594333373 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9583+0+0 (secure 0 0 0) 0x7fa07c0ee550 con 0x55623cabb950 2024-04-24T09:12:44.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.577+0000 7fa094653740 1 -- 172.21.15.83:0/2594333373 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0840113e0 con 0x55623cabb950 2024-04-24T09:12:44.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.578+0000 7fa074c00640 1 -- 172.21.15.83:0/2594333373 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7fa07c005d50 con 0x55623cabb950 2024-04-24T09:12:44.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.578+0000 7fa094653740 10 client.22857 fetch_fsmap learned FSMap version 64 2024-04-24T09:12:44.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.578+0000 7fa094653740 1 -- 172.21.15.83:0/2594333373 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55623c7333f0 con 0x55623cabb950 2024-04-24T09:12:44.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.579+0000 7fa074c00640 1 -- 172.21.15.83:0/2594333373 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa0840113e0 con 0x55623cabb950 2024-04-24T09:12:44.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.579+0000 7fa074c00640 1 -- 172.21.15.83:0/2594333373 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 64) ==== 8374+0+0 (secure 0 0 0) 0x7fa07c0dbce0 con 0x55623cabb950 2024-04-24T09:12:44.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.579+0000 7fa094653740 10 client.22857 fetch_fsmap finished waiting for FSMap version 64 2024-04-24T09:12:44.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.579+0000 7fa094653740 10 client.22857 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-24T09:12:44.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.579+0000 7fa094653740 20 client.22857 populate_metadata read hostname 'smithi083' 2024-04-24T09:12:44.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.579+0000 7fa094653740 1 --2- 172.21.15.83:0/2594333373 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55623cadf240 0x55623cae16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:44.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.579+0000 7fa094653740 4 client.22857 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:12:44.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.579+0000 7fa094653740 1 -- 172.21.15.83:0/2594333373 --> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55623ca2ebf0 con 0x55623cadf240 2024-04-24T09:12:44.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.579+0000 7fa08a000640 1 --2- 172.21.15.83:0/2594333373 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55623cadf240 0x55623cae16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:44.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.580+0000 7fa08a000640 1 --2- 172.21.15.83:0/2594333373 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55623cadf240 0x55623cae16a0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=355301c863e8d774 server_cookie=d55c6518f4746f92 in_seq=0 out_seq=0 2024-04-24T09:12:44.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.580+0000 7fa074c00640 10 client.22857 ms_handle_connect on v2:172.21.15.138:6834/1909219758 2024-04-24T09:12:44.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.583+0000 7fa074c00640 1 -- 172.21.15.83:0/2594333373 <== mds.2 v2:172.21.15.138:6834/1909219758 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55623ca2ebf0 con 0x55623cadf240 2024-04-24T09:12:44.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.583+0000 7fa074c00640 10 client.22857 handle_command_reply: tid=0 2024-04-24T09:12:44.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.583+0000 7fa094653740 1 client.22857 shutdown 2024-04-24T09:12:44.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.583+0000 7fa094653740 1 -- 172.21.15.83:0/2594333373 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55623cabb950 msgr2=0x55623cac62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:44.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.583+0000 7fa094653740 1 --2- 172.21.15.83:0/2594333373 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55623cabb950 0x55623cac62d0 secure :-1 s=READY pgs=2580 cs=0 l=1 rev1=1 crypto rx=0x7fa07c0ee8d0 tx=0x7fa07c0b8760 comp rx=0 tx=0).stop 2024-04-24T09:12:44.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.583+0000 7fa094653740 1 -- 172.21.15.83:0/2594333373 shutdown_connections 2024-04-24T09:12:44.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.583+0000 7fa094653740 1 --2- 172.21.15.83:0/2594333373 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55623cadf240 0x55623cae16a0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:44.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.583+0000 7fa094653740 1 --2- 172.21.15.83:0/2594333373 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55623cac6940 0x55623cac3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:44.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.583+0000 7fa074c00640 0 client.22857 ms_handle_reset on v2:172.21.15.138:6834/1909219758 2024-04-24T09:12:44.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.583+0000 7fa094653740 1 --2- 172.21.15.83:0/2594333373 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55623cabc270 0x55623cac35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:44.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.583+0000 7fa094653740 1 --2- 172.21.15.83:0/2594333373 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55623cabb950 0x55623cac62d0 unknown :-1 s=CLOSED pgs=2580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:44.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.583+0000 7fa094653740 1 -- 172.21.15.83:0/2594333373 >> 172.21.15.83:0/2594333373 conn(0x55623cab7c70 msgr2=0x55623cabd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:44.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.584+0000 7fa094653740 1 -- 172.21.15.83:0/2594333373 shutdown_connections 2024-04-24T09:12:44.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.584+0000 7fa094653740 1 -- 172.21.15.83:0/2594333373 wait complete. 2024-04-24T09:12:44.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.584+0000 7fa094653740 20 client.22857 trim_cache size 0 max 16384 2024-04-24T09:12:44.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.608+0000 7fa094653740 1 Processor -- start 2024-04-24T09:12:44.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.609+0000 7fa094653740 1 -- start start 2024-04-24T09:12:44.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.609+0000 7fa094653740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55623cabb950 0x55623cab6c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:44.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.610+0000 7fa094653740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55623cabc270 0x55623cac5e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:44.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.610+0000 7fa094653740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55623cac6940 0x55623cae6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:44.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.610+0000 7fa094653740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa06c010150 con 0x55623cac6940 2024-04-24T09:12:44.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.610+0000 7fa094653740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa06c011d20 con 0x55623cabb950 2024-04-24T09:12:44.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.610+0000 7fa094653740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa06c010820 con 0x55623cabc270 2024-04-24T09:12:44.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.610+0000 7fa08aa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55623cabb950 0x55623cab6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:44.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.610+0000 7fa08b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55623cac6940 0x55623cae6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:44.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.610+0000 7fa08b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55623cac6940 0x55623cae6fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44142/0 (socket says 172.21.15.83:44142) 2024-04-24T09:12:44.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.610+0000 7fa08b400640 1 -- 172.21.15.83:0/1976063673 learned_addr learned my addr 172.21.15.83:0/1976063673 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:12:44.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.610+0000 7fa08a000640 1 --2- 172.21.15.83:0/1976063673 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55623cabc270 0x55623cac5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:44.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.610+0000 7fa08b400640 1 -- 172.21.15.83:0/1976063673 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55623cabc270 msgr2=0x55623cac5e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:44.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.611+0000 7fa08b400640 1 --2- 172.21.15.83:0/1976063673 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55623cabc270 0x55623cac5e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:44.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.611+0000 7fa08b400640 1 -- 172.21.15.83:0/1976063673 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55623cabb950 msgr2=0x55623cab6c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:44.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.611+0000 7fa08b400640 1 --2- 172.21.15.83:0/1976063673 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55623cabb950 0x55623cab6c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:44.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.611+0000 7fa08b400640 1 -- 172.21.15.83:0/1976063673 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55623cac3eb0 con 0x55623cac6940 2024-04-24T09:12:44.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.611+0000 7fa08a000640 1 --2- 172.21.15.83:0/1976063673 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55623cabc270 0x55623cac5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:12:44.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.611+0000 7fa08b400640 1 --2- 172.21.15.83:0/1976063673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55623cac6940 0x55623cae6fb0 secure :-1 s=READY pgs=2581 cs=0 l=1 rev1=1 crypto rx=0x7fa084005f90 tx=0x7fa084005fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c43a9b9db695edcc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:44.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.612+0000 7fa077400640 1 -- 172.21.15.83:0/1976063673 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa084016030 con 0x55623cac6940 2024-04-24T09:12:44.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.612+0000 7fa077400640 1 -- 172.21.15.83:0/1976063673 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa084003e20 con 0x55623cac6940 2024-04-24T09:12:44.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.612+0000 7fa077400640 1 -- 172.21.15.83:0/1976063673 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa08401b750 con 0x55623cac6940 2024-04-24T09:12:44.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.613+0000 7fa094653740 1 -- 172.21.15.83:0/1976063673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55623cac6940 msgr2=0x55623cae6fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:44.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.613+0000 7fa094653740 1 --2- 172.21.15.83:0/1976063673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55623cac6940 0x55623cae6fb0 secure :-1 s=READY pgs=2581 cs=0 l=1 rev1=1 crypto rx=0x7fa084005f90 tx=0x7fa084005fc0 comp rx=0 tx=0).stop 2024-04-24T09:12:44.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.613+0000 7fa094653740 1 -- 172.21.15.83:0/1976063673 shutdown_connections 2024-04-24T09:12:44.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.613+0000 7fa094653740 1 --2- 172.21.15.83:0/1976063673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55623cac6940 0x55623cae6fb0 unknown :-1 s=CLOSED pgs=2581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:44.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.613+0000 7fa094653740 1 --2- 172.21.15.83:0/1976063673 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55623cabc270 0x55623cac5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:44.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.613+0000 7fa094653740 1 --2- 172.21.15.83:0/1976063673 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55623cabb950 0x55623cab6c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:44.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.613+0000 7fa094653740 1 -- 172.21.15.83:0/1976063673 >> 172.21.15.83:0/1976063673 conn(0x55623cab7c70 msgr2=0x55623cab6560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:44.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.613+0000 7fa094653740 1 -- 172.21.15.83:0/1976063673 shutdown_connections 2024-04-24T09:12:44.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.613+0000 7fa094653740 1 -- 172.21.15.83:0/1976063673 wait complete. 2024-04-24T09:12:44.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.613+0000 7fa094653740 1 -- start start 2024-04-24T09:12:44.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.614+0000 7fa094653740 1 Processor -- start 2024-04-24T09:12:44.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.614+0000 7fa094653740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55623cabb950 0x55623cad7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:44.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.614+0000 7fa094653740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55623cabc270 0x55623cabf850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:44.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.614+0000 7fa094653740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55623cac6940 0x55623cabfd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:44.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.614+0000 7fa094653740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa06c011620 con 0x55623cabc270 2024-04-24T09:12:44.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.614+0000 7fa094653740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa06c010f50 con 0x55623cac6940 2024-04-24T09:12:44.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.614+0000 7fa094653740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa06c00e150 con 0x55623cabb950 2024-04-24T09:12:44.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.614+0000 7fa094653740 10 client.0 _ll_register_callbacks cb 0x7fa07c0b8e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:12:44.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.615+0000 7fa08a000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55623cabc270 0x55623cabf850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:44.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.615+0000 7fa08a000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55623cabc270 0x55623cabf850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44158/0 (socket says 172.21.15.83:44158) 2024-04-24T09:12:44.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.615+0000 7fa08a000640 1 -- 172.21.15.83:0/3777450717 learned_addr learned my addr 172.21.15.83:0/3777450717 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:12:44.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.615+0000 7fa08aa00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55623cabb950 0x55623cad7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:44.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.615+0000 7fa08b400640 1 --2- 172.21.15.83:0/3777450717 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55623cac6940 0x55623cabfd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:44.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.615+0000 7fa08a000640 1 -- 172.21.15.83:0/3777450717 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55623cabb950 msgr2=0x55623cad7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:44.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.615+0000 7fa08a000640 1 --2- 172.21.15.83:0/3777450717 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55623cabb950 0x55623cad7780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:44.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.615+0000 7fa08a000640 1 -- 172.21.15.83:0/3777450717 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55623cac6940 msgr2=0x55623cabfd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:44.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.615+0000 7fa08a000640 1 --2- 172.21.15.83:0/3777450717 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55623cac6940 0x55623cabfd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:44.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.615+0000 7fa08a000640 1 -- 172.21.15.83:0/3777450717 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55623c7fd7d0 con 0x55623cabc270 2024-04-24T09:12:44.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.616+0000 7fa08a000640 1 --2- 172.21.15.83:0/3777450717 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55623cabc270 0x55623cabf850 secure :-1 s=READY pgs=2582 cs=0 l=1 rev1=1 crypto rx=0x7fa08c1002f0 tx=0x7fa078012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8c4ffd337ddf85b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:44.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.616+0000 7fa074c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:12:44.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.616+0000 7fa074c00640 1 -- 172.21.15.83:0/3777450717 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa078012e60 con 0x55623cabc270 2024-04-24T09:12:44.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.616+0000 7fa074c00640 1 -- 172.21.15.83:0/3777450717 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9583+0+0 (secure 0 0 0) 0x7fa07801e800 con 0x55623cabc270 2024-04-24T09:12:44.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.616+0000 7fa094653740 1 -- 172.21.15.83:0/3777450717 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa084003e20 con 0x55623cabc270 2024-04-24T09:12:44.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.617+0000 7fa074c00640 1 -- 172.21.15.83:0/3777450717 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7fa078027470 con 0x55623cabc270 2024-04-24T09:12:44.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.617+0000 7fa094653740 10 client.22863 fetch_fsmap learned FSMap version 64 2024-04-24T09:12:44.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.617+0000 7fa094653740 1 -- 172.21.15.83:0/3777450717 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55623cac1610 con 0x55623cabc270 2024-04-24T09:12:44.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.617+0000 7fa074c00640 1 -- 172.21.15.83:0/3777450717 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa084003e20 con 0x55623cabc270 2024-04-24T09:12:44.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.617+0000 7fa074c00640 1 -- 172.21.15.83:0/3777450717 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 64) ==== 8374+0+0 (secure 0 0 0) 0x7fa078031710 con 0x55623cabc270 2024-04-24T09:12:44.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.617+0000 7fa094653740 10 client.22863 fetch_fsmap finished waiting for FSMap version 64 2024-04-24T09:12:44.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.617+0000 7fa094653740 10 client.22863 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-24T09:12:44.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.617+0000 7fa094653740 20 client.22863 populate_metadata read hostname 'smithi083' 2024-04-24T09:12:44.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.617+0000 7fa094653740 1 --2- 172.21.15.83:0/3777450717 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55623cb46d50 0x55623cb49170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:44.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.617+0000 7fa094653740 4 client.22863 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:12:44.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.617+0000 7fa094653740 1 -- 172.21.15.83:0/3777450717 --> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55623ca28a60 con 0x55623cb46d50 2024-04-24T09:12:44.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.618+0000 7fa08aa00640 1 --2- 172.21.15.83:0/3777450717 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55623cb46d50 0x55623cb49170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:44.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.618+0000 7fa08aa00640 1 --2- 172.21.15.83:0/3777450717 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55623cb46d50 0x55623cb49170 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2ddff29edbe4316 server_cookie=58eb54792041f96e in_seq=0 out_seq=0 2024-04-24T09:12:44.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.618+0000 7fa074c00640 10 client.22863 ms_handle_connect on v2:172.21.15.138:6834/1909219758 2024-04-24T09:12:44.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.618+0000 7fa074c00640 1 -- 172.21.15.83:0/3777450717 <== mds.2 v2:172.21.15.138:6834/1909219758 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55623ca28a60 con 0x55623cb46d50 2024-04-24T09:12:44.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.618+0000 7fa074c00640 10 client.22863 handle_command_reply: tid=0 2024-04-24T09:12:44.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.618+0000 7fa094653740 1 client.22863 shutdown 2024-04-24T09:12:44.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.619+0000 7fa094653740 1 -- 172.21.15.83:0/3777450717 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55623cabc270 msgr2=0x55623cabf850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:44.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.619+0000 7fa094653740 1 --2- 172.21.15.83:0/3777450717 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55623cabc270 0x55623cabf850 secure :-1 s=READY pgs=2582 cs=0 l=1 rev1=1 crypto rx=0x7fa08c1002f0 tx=0x7fa078012820 comp rx=0 tx=0).stop 2024-04-24T09:12:44.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.619+0000 7fa094653740 1 -- 172.21.15.83:0/3777450717 shutdown_connections 2024-04-24T09:12:44.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.619+0000 7fa094653740 1 --2- 172.21.15.83:0/3777450717 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55623cb46d50 0x55623cb49170 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:44.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.619+0000 7fa094653740 1 --2- 172.21.15.83:0/3777450717 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55623cac6940 0x55623cabfd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:44.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.619+0000 7fa094653740 1 --2- 172.21.15.83:0/3777450717 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55623cabc270 0x55623cabf850 unknown :-1 s=CLOSED pgs=2582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:44.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.619+0000 7fa094653740 1 --2- 172.21.15.83:0/3777450717 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55623cabb950 0x55623cad7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:44.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.619+0000 7fa094653740 1 -- 172.21.15.83:0/3777450717 >> 172.21.15.83:0/3777450717 conn(0x55623cab7c70 msgr2=0x55623cab6df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:44.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.619+0000 7fa074c00640 0 client.22863 ms_handle_reset on v2:172.21.15.138:6834/1909219758 2024-04-24T09:12:44.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.619+0000 7fa094653740 1 -- 172.21.15.83:0/3777450717 shutdown_connections 2024-04-24T09:12:44.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.619+0000 7fa094653740 1 -- 172.21.15.83:0/3777450717 wait complete. 2024-04-24T09:12:44.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.619+0000 7fa094653740 20 client.22863 trim_cache size 0 max 16384 2024-04-24T09:12:44.623 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:12:44.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.620+0000 7fa091800640 1 -- 172.21.15.83:0/4074454658 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa0700a3d20 msgr2=0x7fa0700a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:44.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.620+0000 7fa091800640 1 --2- 172.21.15.83:0/4074454658 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa0700a3d20 0x7fa0700a61e0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fa08401f000 tx=0x7fa084022000 comp rx=0 tx=0).stop 2024-04-24T09:12:44.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.620+0000 7fa091800640 1 -- 172.21.15.83:0/4074454658 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa08c1044b0 msgr2=0x7fa08c1a07b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:44.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.620+0000 7fa091800640 1 --2- 172.21.15.83:0/4074454658 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa08c1044b0 0x7fa08c1a07b0 secure :-1 s=READY pgs=2579 cs=0 l=1 rev1=1 crypto rx=0x7fa07c005940 tx=0x7fa07c0042f0 comp rx=0 tx=0).stop 2024-04-24T09:12:44.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.620+0000 7fa091800640 1 -- 172.21.15.83:0/4074454658 shutdown_connections 2024-04-24T09:12:44.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.620+0000 7fa091800640 1 --2- 172.21.15.83:0/4074454658 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa0700a3d20 0x7fa0700a61e0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:44.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.620+0000 7fa091800640 1 --2- 172.21.15.83:0/4074454658 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa08c19a9b0 0x7fa08c19ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:44.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.620+0000 7fa091800640 1 --2- 172.21.15.83:0/4074454658 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa08c109580 0x7fa08c1a0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:44.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.620+0000 7fa091800640 1 --2- 172.21.15.83:0/4074454658 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa08c1044b0 0x7fa08c1a07b0 unknown :-1 s=CLOSED pgs=2579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:44.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.620+0000 7fa091800640 1 -- 172.21.15.83:0/4074454658 >> 172.21.15.83:0/4074454658 conn(0x7fa08c0ffd90 msgr2=0x7fa08c1011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:44.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.621+0000 7fa091800640 1 -- 172.21.15.83:0/4074454658 shutdown_connections 2024-04-24T09:12:44.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:44.621+0000 7fa091800640 1 -- 172.21.15.83:0/4074454658 wait complete. 2024-04-24T09:12:44.641 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T09:12:45.189 INFO:tasks.workunit.client.0.smithi083.stdout: 10 54352 6.61 MB/sec execute 316 sec latency 90.143 ms 2024-04-24T09:12:45.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.263+0000 7ff075600640 1 -- 172.21.15.83:0/1427694608 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff070102bf0 msgr2=0x7ff070106c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:45.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.263+0000 7ff075600640 1 --2- 172.21.15.83:0/1427694608 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff070102bf0 0x7ff070106c30 secure :-1 s=READY pgs=2583 cs=0 l=1 rev1=1 crypto rx=0x7ff05c009900 tx=0x7ff05c02a400 comp rx=0 tx=0).stop 2024-04-24T09:12:45.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.263+0000 7ff075600640 1 -- 172.21.15.83:0/1427694608 shutdown_connections 2024-04-24T09:12:45.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.263+0000 7ff075600640 1 --2- 172.21.15.83:0/1427694608 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff070107310 0x7ff07010eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:45.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.263+0000 7ff075600640 1 --2- 172.21.15.83:0/1427694608 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff070102bf0 0x7ff070106c30 unknown :-1 s=CLOSED pgs=2583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:45.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.263+0000 7ff075600640 1 --2- 172.21.15.83:0/1427694608 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff070102240 0x7ff070102620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:45.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.263+0000 7ff075600640 1 -- 172.21.15.83:0/1427694608 >> 172.21.15.83:0/1427694608 conn(0x7ff070079450 msgr2=0x7ff070079860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:45.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.263+0000 7ff075600640 1 -- 172.21.15.83:0/1427694608 shutdown_connections 2024-04-24T09:12:45.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.263+0000 7ff075600640 1 -- 172.21.15.83:0/1427694608 wait complete. 2024-04-24T09:12:45.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.264+0000 7ff075600640 1 Processor -- start 2024-04-24T09:12:45.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.264+0000 7ff075600640 1 -- start start 2024-04-24T09:12:45.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.265+0000 7ff075600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff070102240 0x7ff07010f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:45.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.265+0000 7ff075600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff070107310 0x7ff07010fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:45.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.265+0000 7ff075600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff0701100f0 0x7ff0701aaf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:45.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.265+0000 7ff075600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff070114500 con 0x7ff070107310 2024-04-24T09:12:45.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.265+0000 7ff075600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff070114380 con 0x7ff070102240 2024-04-24T09:12:45.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.265+0000 7ff075600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff070114680 con 0x7ff0701100f0 2024-04-24T09:12:45.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.265+0000 7ff06e000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff070107310 0x7ff07010fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:45.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.265+0000 7ff06e000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff070107310 0x7ff07010fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44182/0 (socket says 172.21.15.83:44182) 2024-04-24T09:12:45.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.265+0000 7ff06e000640 1 -- 172.21.15.83:0/3641009461 learned_addr learned my addr 172.21.15.83:0/3641009461 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:12:45.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.265+0000 7ff06ea00640 1 --2- 172.21.15.83:0/3641009461 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff070102240 0x7ff07010f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:45.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.266+0000 7ff06f400640 1 --2- 172.21.15.83:0/3641009461 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff0701100f0 0x7ff0701aaf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:45.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.266+0000 7ff06e000640 1 -- 172.21.15.83:0/3641009461 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff0701100f0 msgr2=0x7ff0701aaf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:45.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.266+0000 7ff06e000640 1 --2- 172.21.15.83:0/3641009461 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff0701100f0 0x7ff0701aaf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:45.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.266+0000 7ff06e000640 1 -- 172.21.15.83:0/3641009461 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff070102240 msgr2=0x7ff07010f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:45.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.266+0000 7ff06e000640 1 --2- 172.21.15.83:0/3641009461 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff070102240 0x7ff07010f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:45.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.266+0000 7ff06e000640 1 -- 172.21.15.83:0/3641009461 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0701ab4b0 con 0x7ff070107310 2024-04-24T09:12:45.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.266+0000 7ff06e000640 1 --2- 172.21.15.83:0/3641009461 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff070107310 0x7ff07010fbb0 secure :-1 s=READY pgs=2584 cs=0 l=1 rev1=1 crypto rx=0x7ff05c031440 tx=0x7ff05c003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:45.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.267+0000 7ff067400640 1 -- 172.21.15.83:0/3641009461 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff05c03f070 con 0x7ff070107310 2024-04-24T09:12:45.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.267+0000 7ff067400640 1 -- 172.21.15.83:0/3641009461 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff05c031d60 con 0x7ff070107310 2024-04-24T09:12:45.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.267+0000 7ff075600640 1 -- 172.21.15.83:0/3641009461 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0701ab740 con 0x7ff070107310 2024-04-24T09:12:45.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.267+0000 7ff067400640 1 -- 172.21.15.83:0/3641009461 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff05c035030 con 0x7ff070107310 2024-04-24T09:12:45.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.267+0000 7ff075600640 1 -- 172.21.15.83:0/3641009461 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0701abc00 con 0x7ff070107310 2024-04-24T09:12:45.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.272+0000 7ff067400640 1 -- 172.21.15.83:0/3641009461 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff05c042070 con 0x7ff070107310 2024-04-24T09:12:45.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.273+0000 7ff078588740 1 Processor -- start 2024-04-24T09:12:45.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.274+0000 7ff078588740 1 -- start start 2024-04-24T09:12:45.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.274+0000 7ff078588740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5625ec22e950 0x5625ec22ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:45.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.274+0000 7ff078588740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5625ec22f270 0x5625ec239400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:45.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.274+0000 7ff067400640 1 --2- 172.21.15.83:0/3641009461 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff0540a3ef0 0x7ff0540a63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:45.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.274+0000 7ff078588740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5625ec239940 0x5625ec23bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:45.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.274+0000 7ff078588740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5625ec1941d0 con 0x5625ec22e950 2024-04-24T09:12:45.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.274+0000 7ff078588740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5625ec13a450 con 0x5625ec239940 2024-04-24T09:12:45.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.274+0000 7ff078588740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5625ec190190 con 0x5625ec22f270 2024-04-24T09:12:45.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.274+0000 7ff067400640 1 -- 172.21.15.83:0/3641009461 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 9583+0+0 (secure 0 0 0) 0x7ff05c0f1490 con 0x7ff070107310 2024-04-24T09:12:45.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.275+0000 7ff06e000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5625ec239940 0x5625ec23bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:45.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.275+0000 7ff06e000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5625ec239940 0x5625ec23bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:50004/0 (socket says 172.21.15.83:50004) 2024-04-24T09:12:45.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.275+0000 7ff06e000640 1 -- 172.21.15.83:0/3836814252 learned_addr learned my addr 172.21.15.83:0/3836814252 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:12:45.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.275+0000 7ff06ea00640 1 --2- 172.21.15.83:0/3836814252 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5625ec22f270 0x5625ec239400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:45.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.275+0000 7ff06f400640 1 --2- 172.21.15.83:0/3641009461 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff0540a3ef0 0x7ff0540a63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:45.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.275+0000 7ff06e000640 1 -- 172.21.15.83:0/3836814252 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5625ec22f270 msgr2=0x5625ec239400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:45.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.275+0000 7ff06e000640 1 --2- 172.21.15.83:0/3836814252 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5625ec22f270 0x5625ec239400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:45.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.275+0000 7ff06e000640 1 -- 172.21.15.83:0/3836814252 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5625ec22e950 msgr2=0x5625ec22ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:45.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.275+0000 7ff06e000640 1 --2- 172.21.15.83:0/3836814252 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5625ec22e950 0x5625ec22ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:45.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.275+0000 7ff06e000640 1 -- 172.21.15.83:0/3836814252 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625ec1f7310 con 0x5625ec239940 2024-04-24T09:12:45.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.276+0000 7ff06e000640 1 --2- 172.21.15.83:0/3836814252 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5625ec239940 0x5625ec23bd30 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7ff05c0c56f0 tx=0x7ff05c0bcc20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6fd23e9ce909d11 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:45.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.276+0000 7ff06f400640 1 --2- 172.21.15.83:0/3641009461 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff0540a3ef0 0x7ff0540a63b0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7ff068005cc0 tx=0x7ff06800b3e0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:45.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.276+0000 7ff05b400640 1 -- 172.21.15.83:0/3836814252 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff05c0dd070 con 0x5625ec239940 2024-04-24T09:12:45.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.276+0000 7ff05b400640 1 -- 172.21.15.83:0/3836814252 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff05c0bd6b0 con 0x5625ec239940 2024-04-24T09:12:45.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.276+0000 7ff05b400640 1 -- 172.21.15.83:0/3836814252 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff05c049040 con 0x5625ec239940 2024-04-24T09:12:45.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.277+0000 7ff078588740 1 -- 172.21.15.83:0/3836814252 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5625ec239940 msgr2=0x5625ec23bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:45.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.277+0000 7ff078588740 1 --2- 172.21.15.83:0/3836814252 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5625ec239940 0x5625ec23bd30 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7ff05c0c56f0 tx=0x7ff05c0bcc20 comp rx=0 tx=0).stop 2024-04-24T09:12:45.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.278+0000 7ff078588740 1 -- 172.21.15.83:0/3836814252 shutdown_connections 2024-04-24T09:12:45.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.278+0000 7ff078588740 1 --2- 172.21.15.83:0/3836814252 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5625ec239940 0x5625ec23bd30 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:45.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.278+0000 7ff078588740 1 --2- 172.21.15.83:0/3836814252 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5625ec22f270 0x5625ec239400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:45.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.278+0000 7ff078588740 1 --2- 172.21.15.83:0/3836814252 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5625ec22e950 0x5625ec22ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:45.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.278+0000 7ff078588740 1 -- 172.21.15.83:0/3836814252 >> 172.21.15.83:0/3836814252 conn(0x5625ec22ac70 msgr2=0x5625ec22b050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:45.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.278+0000 7ff078588740 1 -- 172.21.15.83:0/3836814252 shutdown_connections 2024-04-24T09:12:45.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.278+0000 7ff078588740 1 -- 172.21.15.83:0/3836814252 wait complete. 2024-04-24T09:12:45.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.278+0000 7ff078588740 1 -- start start 2024-04-24T09:12:45.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.279+0000 7ff078588740 1 Processor -- start 2024-04-24T09:12:45.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.279+0000 7ff078588740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5625ec22e950 0x5625ec2393e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:45.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.279+0000 7ff078588740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5625ec22f270 0x5625ec2366d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:45.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.279+0000 7ff078588740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5625ec239940 0x5625ec236dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:45.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.279+0000 7ff078588740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5625ec23c4f0 con 0x5625ec22e950 2024-04-24T09:12:45.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.279+0000 7ff078588740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5625ec14d810 con 0x5625ec239940 2024-04-24T09:12:45.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.279+0000 7ff078588740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5625ec23c670 con 0x5625ec22f270 2024-04-24T09:12:45.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.279+0000 7ff078588740 10 client.0 _ll_register_callbacks cb 0x5625ec11b140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:12:45.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.280+0000 7ff06f400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5625ec239940 0x5625ec236dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:45.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.280+0000 7ff06f400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5625ec239940 0x5625ec236dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:50014/0 (socket says 172.21.15.83:50014) 2024-04-24T09:12:45.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.280+0000 7ff06f400640 1 -- 172.21.15.83:0/1660642375 learned_addr learned my addr 172.21.15.83:0/1660642375 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:12:45.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.280+0000 7ff06ea00640 1 --2- 172.21.15.83:0/1660642375 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5625ec22e950 0x5625ec2393e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:45.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.280+0000 7ff06e000640 1 --2- 172.21.15.83:0/1660642375 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5625ec22f270 0x5625ec2366d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:45.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.280+0000 7ff06ea00640 1 -- 172.21.15.83:0/1660642375 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5625ec22f270 msgr2=0x5625ec2366d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:45.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.280+0000 7ff06ea00640 1 --2- 172.21.15.83:0/1660642375 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5625ec22f270 0x5625ec2366d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:45.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.280+0000 7ff06ea00640 1 -- 172.21.15.83:0/1660642375 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5625ec239940 msgr2=0x5625ec236dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:45.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.280+0000 7ff06ea00640 1 --2- 172.21.15.83:0/1660642375 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5625ec239940 0x5625ec236dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:45.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.281+0000 7ff06ea00640 1 -- 172.21.15.83:0/1660642375 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625ec15e3f0 con 0x5625ec22e950 2024-04-24T09:12:45.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.281+0000 7ff06f400640 1 --2- 172.21.15.83:0/1660642375 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5625ec239940 0x5625ec236dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:12:45.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.281+0000 7ff06ea00640 1 --2- 172.21.15.83:0/1660642375 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5625ec22e950 0x5625ec2393e0 secure :-1 s=READY pgs=2585 cs=0 l=1 rev1=1 crypto rx=0x7ff0600058c0 tx=0x7ff060010bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5236830828d0533d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:45.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.281+0000 7ff04fe00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:12:45.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.282+0000 7ff04fe00640 1 -- 172.21.15.83:0/1660642375 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff06000fc20 con 0x5625ec22e950 2024-04-24T09:12:45.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.282+0000 7ff04fe00640 1 -- 172.21.15.83:0/1660642375 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9583+0+0 (secure 0 0 0) 0x7ff060010e00 con 0x5625ec22e950 2024-04-24T09:12:45.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.282+0000 7ff078588740 1 -- 172.21.15.83:0/1660642375 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff05c0bd6b0 con 0x5625ec22e950 2024-04-24T09:12:45.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.282+0000 7ff04fe00640 1 -- 172.21.15.83:0/1660642375 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7ff060024050 con 0x5625ec22e950 2024-04-24T09:12:45.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.282+0000 7ff078588740 10 client.22872 fetch_fsmap learned FSMap version 64 2024-04-24T09:12:45.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.282+0000 7ff078588740 1 -- 172.21.15.83:0/1660642375 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x5625ebea63f0 con 0x5625ec22e950 2024-04-24T09:12:45.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.283+0000 7ff04fe00640 1 -- 172.21.15.83:0/1660642375 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff05c0bd6b0 con 0x5625ec22e950 2024-04-24T09:12:45.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.283+0000 7ff04fe00640 1 -- 172.21.15.83:0/1660642375 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 64) ==== 8374+0+0 (secure 0 0 0) 0x7ff06000e3b0 con 0x5625ec22e950 2024-04-24T09:12:45.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.283+0000 7ff078588740 10 client.22872 fetch_fsmap finished waiting for FSMap version 64 2024-04-24T09:12:45.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.283+0000 7ff078588740 10 client.22872 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-24T09:12:45.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.283+0000 7ff078588740 20 client.22872 populate_metadata read hostname 'smithi083' 2024-04-24T09:12:45.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.283+0000 7ff078588740 1 --2- 172.21.15.83:0/1660642375 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x5625ec249920 0x5625ec24bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:45.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.283+0000 7ff078588740 4 client.22872 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:12:45.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.283+0000 7ff078588740 1 -- 172.21.15.83:0/1660642375 --> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5625ec1a1bf0 con 0x5625ec249920 2024-04-24T09:12:45.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.283+0000 7ff06e000640 1 --2- 172.21.15.83:0/1660642375 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x5625ec249920 0x5625ec24bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:45.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.284+0000 7ff06e000640 1 --2- 172.21.15.83:0/1660642375 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x5625ec249920 0x5625ec24bd30 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=df1a5a59f9dda15a server_cookie=586c644106a39f16 in_seq=0 out_seq=0 2024-04-24T09:12:45.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.284+0000 7ff04fe00640 10 client.22872 ms_handle_connect on v2:172.21.15.138:6836/900566754 2024-04-24T09:12:45.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.285+0000 7ff04fe00640 1 -- 172.21.15.83:0/1660642375 <== mds.1 v2:172.21.15.138:6836/900566754 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5625ec1a1bf0 con 0x5625ec249920 2024-04-24T09:12:45.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.285+0000 7ff04fe00640 10 client.22872 handle_command_reply: tid=0 2024-04-24T09:12:45.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.285+0000 7ff078588740 1 client.22872 shutdown 2024-04-24T09:12:45.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.286+0000 7ff078588740 1 -- 172.21.15.83:0/1660642375 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5625ec22e950 msgr2=0x5625ec2393e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:45.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.286+0000 7ff078588740 1 --2- 172.21.15.83:0/1660642375 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5625ec22e950 0x5625ec2393e0 secure :-1 s=READY pgs=2585 cs=0 l=1 rev1=1 crypto rx=0x7ff0600058c0 tx=0x7ff060010bb0 comp rx=0 tx=0).stop 2024-04-24T09:12:45.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.286+0000 7ff078588740 1 -- 172.21.15.83:0/1660642375 shutdown_connections 2024-04-24T09:12:45.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.286+0000 7ff078588740 1 --2- 172.21.15.83:0/1660642375 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5625ec239940 0x5625ec236dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:45.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.286+0000 7ff078588740 1 --2- 172.21.15.83:0/1660642375 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x5625ec249920 0x5625ec24bd30 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:45.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.286+0000 7ff078588740 1 --2- 172.21.15.83:0/1660642375 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5625ec22f270 0x5625ec2366d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:45.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.286+0000 7ff078588740 1 --2- 172.21.15.83:0/1660642375 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5625ec22e950 0x5625ec2393e0 secure :-1 s=CLOSED pgs=2585 cs=0 l=1 rev1=1 crypto rx=0x7ff0600058c0 tx=0x7ff060010bb0 comp rx=0 tx=0).stop 2024-04-24T09:12:45.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.286+0000 7ff04fe00640 0 client.22872 ms_handle_reset on v2:172.21.15.138:6836/900566754 2024-04-24T09:12:45.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.286+0000 7ff078588740 1 -- 172.21.15.83:0/1660642375 >> 172.21.15.83:0/1660642375 conn(0x5625ec22ac70 msgr2=0x5625ec230a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:45.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.286+0000 7ff078588740 1 -- 172.21.15.83:0/1660642375 shutdown_connections 2024-04-24T09:12:45.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.287+0000 7ff078588740 1 -- 172.21.15.83:0/1660642375 wait complete. 2024-04-24T09:12:45.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.287+0000 7ff078588740 20 client.22872 trim_cache size 0 max 16384 2024-04-24T09:12:45.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.311+0000 7ff078588740 1 Processor -- start 2024-04-24T09:12:45.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.311+0000 7ff078588740 1 -- start start 2024-04-24T09:12:45.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.312+0000 7ff078588740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5625ec22f270 0x5625ec2453b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:45.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.312+0000 7ff078588740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5625ec239940 0x5625ec238f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:45.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.312+0000 7ff078588740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5625ec239450 0x5625ec251600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:45.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.312+0000 7ff078588740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff0701b5b90 con 0x5625ec239940 2024-04-24T09:12:45.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.312+0000 7ff078588740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff0701b7760 con 0x5625ec239450 2024-04-24T09:12:45.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.312+0000 7ff078588740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff0701b6260 con 0x5625ec22f270 2024-04-24T09:12:45.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.312+0000 7ff06e000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5625ec239940 0x5625ec238f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:45.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.312+0000 7ff06e000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5625ec239940 0x5625ec238f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44220/0 (socket says 172.21.15.83:44220) 2024-04-24T09:12:45.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.312+0000 7ff06e000640 1 -- 172.21.15.83:0/2991013240 learned_addr learned my addr 172.21.15.83:0/2991013240 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:12:45.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.313+0000 7ff06f400640 1 --2- 172.21.15.83:0/2991013240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5625ec239450 0x5625ec251600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:45.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.313+0000 7ff06ea00640 1 --2- 172.21.15.83:0/2991013240 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5625ec22f270 0x5625ec2453b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:45.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.313+0000 7ff06e000640 1 -- 172.21.15.83:0/2991013240 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5625ec22f270 msgr2=0x5625ec2453b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:45.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.313+0000 7ff06e000640 1 --2- 172.21.15.83:0/2991013240 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5625ec22f270 0x5625ec2453b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:45.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.313+0000 7ff06e000640 1 -- 172.21.15.83:0/2991013240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5625ec239450 msgr2=0x5625ec251600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:45.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.313+0000 7ff06e000640 1 --2- 172.21.15.83:0/2991013240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5625ec239450 0x5625ec251600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:45.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.313+0000 7ff06e000640 1 -- 172.21.15.83:0/2991013240 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625ec237130 con 0x5625ec239940 2024-04-24T09:12:45.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.313+0000 7ff06f400640 1 --2- 172.21.15.83:0/2991013240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5625ec239450 0x5625ec251600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:12:45.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.313+0000 7ff06e000640 1 --2- 172.21.15.83:0/2991013240 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5625ec239940 0x5625ec238f10 secure :-1 s=READY pgs=2586 cs=0 l=1 rev1=1 crypto rx=0x7ff05c0bccb0 tx=0x7ff05c085e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b09f368986f0e29b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:45.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.314+0000 7ff05aa00640 1 -- 172.21.15.83:0/2991013240 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff05c0dd040 con 0x5625ec239940 2024-04-24T09:12:45.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.314+0000 7ff05aa00640 1 -- 172.21.15.83:0/2991013240 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff05c0c5c20 con 0x5625ec239940 2024-04-24T09:12:45.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.314+0000 7ff05aa00640 1 -- 172.21.15.83:0/2991013240 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff05c049040 con 0x5625ec239940 2024-04-24T09:12:45.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.315+0000 7ff078588740 1 -- 172.21.15.83:0/2991013240 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5625ec239940 msgr2=0x5625ec238f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:45.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.315+0000 7ff078588740 1 --2- 172.21.15.83:0/2991013240 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5625ec239940 0x5625ec238f10 secure :-1 s=READY pgs=2586 cs=0 l=1 rev1=1 crypto rx=0x7ff05c0bccb0 tx=0x7ff05c085e20 comp rx=0 tx=0).stop 2024-04-24T09:12:45.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.315+0000 7ff078588740 1 -- 172.21.15.83:0/2991013240 shutdown_connections 2024-04-24T09:12:45.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.315+0000 7ff078588740 1 --2- 172.21.15.83:0/2991013240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5625ec239450 0x5625ec251600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:45.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.315+0000 7ff078588740 1 --2- 172.21.15.83:0/2991013240 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5625ec239940 0x5625ec238f10 unknown :-1 s=CLOSED pgs=2586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:45.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.315+0000 7ff078588740 1 --2- 172.21.15.83:0/2991013240 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5625ec22f270 0x5625ec2453b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:45.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.315+0000 7ff078588740 1 -- 172.21.15.83:0/2991013240 >> 172.21.15.83:0/2991013240 conn(0x5625ec22ac70 msgr2=0x5625ec22b050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:45.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.315+0000 7ff078588740 1 -- 172.21.15.83:0/2991013240 shutdown_connections 2024-04-24T09:12:45.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.315+0000 7ff078588740 1 -- 172.21.15.83:0/2991013240 wait complete. 2024-04-24T09:12:45.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.316+0000 7ff078588740 1 -- start start 2024-04-24T09:12:45.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.316+0000 7ff078588740 1 Processor -- start 2024-04-24T09:12:45.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.316+0000 7ff078588740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5625ec22f270 0x5625ec23aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:45.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.316+0000 7ff078588740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5625ec239450 0x5625ec23afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:45.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.316+0000 7ff078588740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5625ec23b600 0x5625ec23fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:45.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.316+0000 7ff078588740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff0701b7060 con 0x5625ec239450 2024-04-24T09:12:45.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.316+0000 7ff078588740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff0701b6990 con 0x5625ec23b600 2024-04-24T09:12:45.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.316+0000 7ff078588740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff0701b3b90 con 0x5625ec22f270 2024-04-24T09:12:45.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.317+0000 7ff078588740 10 client.0 _ll_register_callbacks cb 0x7ff06000a6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:12:45.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.317+0000 7ff06f400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5625ec23b600 0x5625ec23fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:45.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.317+0000 7ff06f400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5625ec23b600 0x5625ec23fe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:50024/0 (socket says 172.21.15.83:50024) 2024-04-24T09:12:45.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.317+0000 7ff06f400640 1 -- 172.21.15.83:0/2933845593 learned_addr learned my addr 172.21.15.83:0/2933845593 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:12:45.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.317+0000 7ff06e000640 1 --2- 172.21.15.83:0/2933845593 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5625ec239450 0x5625ec23afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:45.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.317+0000 7ff06ea00640 1 --2- 172.21.15.83:0/2933845593 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5625ec22f270 0x5625ec23aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:45.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.318+0000 7ff06f400640 1 -- 172.21.15.83:0/2933845593 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5625ec22f270 msgr2=0x5625ec23aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:45.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.318+0000 7ff06f400640 1 --2- 172.21.15.83:0/2933845593 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5625ec22f270 0x5625ec23aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:45.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.318+0000 7ff06f400640 1 -- 172.21.15.83:0/2933845593 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5625ec239450 msgr2=0x5625ec23afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:45.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.318+0000 7ff06f400640 1 --2- 172.21.15.83:0/2933845593 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5625ec239450 0x5625ec23afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:45.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.318+0000 7ff06f400640 1 -- 172.21.15.83:0/2933845593 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625ebf707d0 con 0x5625ec23b600 2024-04-24T09:12:45.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.318+0000 7ff06e000640 1 --2- 172.21.15.83:0/2933845593 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5625ec239450 0x5625ec23afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:12:45.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.318+0000 7ff06f400640 1 --2- 172.21.15.83:0/2933845593 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5625ec23b600 0x5625ec23fe20 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7ff06801d680 tx=0x7ff068004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3ec772c8358fe1c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:45.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.318+0000 7ff04fe00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:12:45.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.319+0000 7ff04fe00640 1 -- 172.21.15.83:0/2933845593 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff06802b060 con 0x5625ec23b600 2024-04-24T09:12:45.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.319+0000 7ff04fe00640 1 -- 172.21.15.83:0/2933845593 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9583+0+0 (secure 0 0 0) 0x7ff06800e690 con 0x5625ec23b600 2024-04-24T09:12:45.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.319+0000 7ff078588740 1 -- 172.21.15.83:0/2933845593 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff05c0c5c20 con 0x5625ec23b600 2024-04-24T09:12:45.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.319+0000 7ff04fe00640 1 -- 172.21.15.83:0/2933845593 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7ff068032050 con 0x5625ec23b600 2024-04-24T09:12:45.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.319+0000 7ff078588740 10 client.39917 fetch_fsmap learned FSMap version 64 2024-04-24T09:12:45.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.319+0000 7ff078588740 1 -- 172.21.15.83:0/2933845593 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x5625ec2b07d0 con 0x5625ec23b600 2024-04-24T09:12:45.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.319+0000 7ff04fe00640 1 -- 172.21.15.83:0/2933845593 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff05c0c5c20 con 0x5625ec23b600 2024-04-24T09:12:45.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.320+0000 7ff04fe00640 1 -- 172.21.15.83:0/2933845593 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 64) ==== 8374+0+0 (secure 0 0 0) 0x7ff068026810 con 0x5625ec23b600 2024-04-24T09:12:45.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.320+0000 7ff078588740 10 client.39917 fetch_fsmap finished waiting for FSMap version 64 2024-04-24T09:12:45.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.320+0000 7ff078588740 10 client.39917 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-24T09:12:45.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.320+0000 7ff078588740 20 client.39917 populate_metadata read hostname 'smithi083' 2024-04-24T09:12:45.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.320+0000 7ff078588740 1 --2- 172.21.15.83:0/2933845593 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x5625ec2b0f20 0x5625ec2b3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:45.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.320+0000 7ff078588740 4 client.39917 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:12:45.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.320+0000 7ff078588740 1 -- 172.21.15.83:0/2933845593 --> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5625ec19ba60 con 0x5625ec2b0f20 2024-04-24T09:12:45.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.320+0000 7ff06ea00640 1 --2- 172.21.15.83:0/2933845593 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x5625ec2b0f20 0x5625ec2b3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:45.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.321+0000 7ff06ea00640 1 --2- 172.21.15.83:0/2933845593 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x5625ec2b0f20 0x5625ec2b3340 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c87319e257877f6d server_cookie=55f32e59c904d2c7 in_seq=0 out_seq=0 2024-04-24T09:12:45.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.321+0000 7ff04fe00640 10 client.39917 ms_handle_connect on v2:172.21.15.138:6836/900566754 2024-04-24T09:12:45.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.321+0000 7ff04fe00640 1 -- 172.21.15.83:0/2933845593 <== mds.1 v2:172.21.15.138:6836/900566754 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5625ec19ba60 con 0x5625ec2b0f20 2024-04-24T09:12:45.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.321+0000 7ff04fe00640 10 client.39917 handle_command_reply: tid=0 2024-04-24T09:12:45.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.321+0000 7ff078588740 1 client.39917 shutdown 2024-04-24T09:12:45.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.321+0000 7ff078588740 1 -- 172.21.15.83:0/2933845593 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5625ec23b600 msgr2=0x5625ec23fe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:45.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.321+0000 7ff078588740 1 --2- 172.21.15.83:0/2933845593 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5625ec23b600 0x5625ec23fe20 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7ff06801d680 tx=0x7ff068004ab0 comp rx=0 tx=0).stop 2024-04-24T09:12:45.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.321+0000 7ff078588740 1 -- 172.21.15.83:0/2933845593 shutdown_connections 2024-04-24T09:12:45.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.321+0000 7ff078588740 1 --2- 172.21.15.83:0/2933845593 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5625ec23b600 0x5625ec23fe20 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:45.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.321+0000 7ff078588740 1 --2- 172.21.15.83:0/2933845593 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5625ec239450 0x5625ec23afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:45.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.321+0000 7ff078588740 1 --2- 172.21.15.83:0/2933845593 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x5625ec2b0f20 0x5625ec2b3340 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:45.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.321+0000 7ff078588740 1 --2- 172.21.15.83:0/2933845593 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5625ec22f270 0x5625ec23aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:45.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.321+0000 7ff078588740 1 -- 172.21.15.83:0/2933845593 >> 172.21.15.83:0/2933845593 conn(0x5625ec22ac70 msgr2=0x5625ec235f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:45.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.321+0000 7ff04fe00640 0 client.39917 ms_handle_reset on v2:172.21.15.138:6836/900566754 2024-04-24T09:12:45.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.322+0000 7ff078588740 1 -- 172.21.15.83:0/2933845593 shutdown_connections 2024-04-24T09:12:45.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.322+0000 7ff078588740 1 -- 172.21.15.83:0/2933845593 wait complete. 2024-04-24T09:12:45.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.322+0000 7ff078588740 20 client.39917 trim_cache size 0 max 16384 2024-04-24T09:12:45.325 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:12:45.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.323+0000 7ff075600640 1 -- 172.21.15.83:0/3641009461 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff0540a3ef0 msgr2=0x7ff0540a63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:45.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.323+0000 7ff075600640 1 --2- 172.21.15.83:0/3641009461 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff0540a3ef0 0x7ff0540a63b0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7ff068005cc0 tx=0x7ff06800b3e0 comp rx=0 tx=0).stop 2024-04-24T09:12:45.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.323+0000 7ff075600640 1 -- 172.21.15.83:0/3641009461 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff070107310 msgr2=0x7ff07010fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:45.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.323+0000 7ff075600640 1 --2- 172.21.15.83:0/3641009461 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff070107310 0x7ff07010fbb0 secure :-1 s=READY pgs=2584 cs=0 l=1 rev1=1 crypto rx=0x7ff05c031440 tx=0x7ff05c003d50 comp rx=0 tx=0).stop 2024-04-24T09:12:45.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.323+0000 7ff075600640 1 -- 172.21.15.83:0/3641009461 shutdown_connections 2024-04-24T09:12:45.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.323+0000 7ff075600640 1 --2- 172.21.15.83:0/3641009461 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff0540a3ef0 0x7ff0540a63b0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:45.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.323+0000 7ff075600640 1 --2- 172.21.15.83:0/3641009461 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff0701100f0 0x7ff0701aaf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:45.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.323+0000 7ff075600640 1 --2- 172.21.15.83:0/3641009461 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff070107310 0x7ff07010fbb0 unknown :-1 s=CLOSED pgs=2584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:45.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.323+0000 7ff075600640 1 --2- 172.21.15.83:0/3641009461 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff070102240 0x7ff07010f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:45.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.323+0000 7ff075600640 1 -- 172.21.15.83:0/3641009461 >> 172.21.15.83:0/3641009461 conn(0x7ff070079450 msgr2=0x7ff0700ffea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:45.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.323+0000 7ff075600640 1 -- 172.21.15.83:0/3641009461 shutdown_connections 2024-04-24T09:12:45.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.323+0000 7ff075600640 1 -- 172.21.15.83:0/3641009461 wait complete. 2024-04-24T09:12:45.347 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T09:12:45.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.597+0000 7fdc8fe00640 1 -- 172.21.15.83:0/610010024 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc90104cd0 msgr2=0x7fdc90108d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:45.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.597+0000 7fdc8fe00640 1 --2- 172.21.15.83:0/610010024 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc90104cd0 0x7fdc90108d10 secure :-1 s=READY pgs=2587 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c009860 tx=0x7fdc7c02a340 comp rx=0 tx=0).stop 2024-04-24T09:12:45.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.597+0000 7fdc8fe00640 1 -- 172.21.15.83:0/610010024 shutdown_connections 2024-04-24T09:12:45.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.597+0000 7fdc8fe00640 1 --2- 172.21.15.83:0/610010024 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc901093f0 0x7fdc90110fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:45.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.597+0000 7fdc8fe00640 1 --2- 172.21.15.83:0/610010024 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc90104cd0 0x7fdc90108d10 unknown :-1 s=CLOSED pgs=2587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:45.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.597+0000 7fdc8fe00640 1 --2- 172.21.15.83:0/610010024 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdc90104320 0x7fdc90104700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:45.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.597+0000 7fdc8fe00640 1 -- 172.21.15.83:0/610010024 >> 172.21.15.83:0/610010024 conn(0x7fdc900ffb40 msgr2=0x7fdc90101f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:45.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.597+0000 7fdc8fe00640 1 -- 172.21.15.83:0/610010024 shutdown_connections 2024-04-24T09:12:45.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.597+0000 7fdc8fe00640 1 -- 172.21.15.83:0/610010024 wait complete. 2024-04-24T09:12:45.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.598+0000 7fdc8fe00640 1 Processor -- start 2024-04-24T09:12:45.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.599+0000 7fdc8fe00640 1 -- start start 2024-04-24T09:12:45.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.599+0000 7fdc8fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc90104320 0x7fdc901a07a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:45.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.599+0000 7fdc8fe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdc901093f0 0x7fdc901a0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:45.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.599+0000 7fdc8fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc9019a950 0x7fdc9019ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:45.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.599+0000 7fdc8fe00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdc90114330 con 0x7fdc90104320 2024-04-24T09:12:45.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.599+0000 7fdc8fe00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdc901141b0 con 0x7fdc9019a950 2024-04-24T09:12:45.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.599+0000 7fdc8fe00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdc901144b0 con 0x7fdc901093f0 2024-04-24T09:12:45.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.600+0000 7fdc8ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc90104320 0x7fdc901a07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:45.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.600+0000 7fdc8ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc90104320 0x7fdc901a07a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44236/0 (socket says 172.21.15.83:44236) 2024-04-24T09:12:45.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.600+0000 7fdc8ea00640 1 -- 172.21.15.83:0/1764092797 learned_addr learned my addr 172.21.15.83:0/1764092797 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:12:45.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.600+0000 7fdc8f400640 1 --2- 172.21.15.83:0/1764092797 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc9019a950 0x7fdc9019ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:45.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.600+0000 7fdc8ea00640 1 -- 172.21.15.83:0/1764092797 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdc901093f0 msgr2=0x7fdc901a0ce0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T09:12:45.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.600+0000 7fdc8ea00640 1 --2- 172.21.15.83:0/1764092797 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdc901093f0 0x7fdc901a0ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:45.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.601+0000 7fdc8ea00640 1 -- 172.21.15.83:0/1764092797 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc9019a950 msgr2=0x7fdc9019ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:45.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.601+0000 7fdc8ea00640 1 --2- 172.21.15.83:0/1764092797 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc9019a950 0x7fdc9019ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:45.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.601+0000 7fdc8ea00640 1 -- 172.21.15.83:0/1764092797 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc9019b470 con 0x7fdc90104320 2024-04-24T09:12:45.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.601+0000 7fdc8f400640 1 --2- 172.21.15.83:0/1764092797 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc9019a950 0x7fdc9019ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:12:45.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.601+0000 7fdc8ea00640 1 --2- 172.21.15.83:0/1764092797 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc90104320 0x7fdc901a07a0 secure :-1 s=READY pgs=2588 cs=0 l=1 rev1=1 crypto rx=0x7fdc8000d770 tx=0x7fdc8000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:45.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.601+0000 7fdc87400640 1 -- 172.21.15.83:0/1764092797 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc80017070 con 0x7fdc90104320 2024-04-24T09:12:45.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.601+0000 7fdc8fe00640 1 -- 172.21.15.83:0/1764092797 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc9019b760 con 0x7fdc90104320 2024-04-24T09:12:45.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.602+0000 7fdc87400640 1 -- 172.21.15.83:0/1764092797 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdc800054d0 con 0x7fdc90104320 2024-04-24T09:12:45.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.602+0000 7fdc87400640 1 -- 172.21.15.83:0/1764092797 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc80004cf0 con 0x7fdc90104320 2024-04-24T09:12:45.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.602+0000 7fdc8fe00640 1 -- 172.21.15.83:0/1764092797 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc901a79b0 con 0x7fdc90104320 2024-04-24T09:12:45.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.606+0000 7fdc987dd740 1 Processor -- start 2024-04-24T09:12:45.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.606+0000 7fdc987dd740 1 -- start start 2024-04-24T09:12:45.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.607+0000 7fdc987dd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eb04fa5b40 0x55eb04fa5f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:45.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.607+0000 7fdc987dd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb04fa6460 0x55eb04fb05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:45.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.607+0000 7fdc987dd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb04fb0b30 0x55eb04fb2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:45.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.607+0000 7fdc987dd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55eb04f0b1d0 con 0x55eb04fa6460 2024-04-24T09:12:45.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.607+0000 7fdc987dd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55eb04eb1450 con 0x55eb04fa5b40 2024-04-24T09:12:45.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.607+0000 7fdc987dd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55eb04f07190 con 0x55eb04fb0b30 2024-04-24T09:12:45.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.607+0000 7fdc87400640 1 -- 172.21.15.83:0/1764092797 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fdc80004e90 con 0x7fdc90104320 2024-04-24T09:12:45.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.607+0000 7fdc8f400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eb04fa5b40 0x55eb04fa5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:45.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.607+0000 7fdc8f400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eb04fa5b40 0x55eb04fa5f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:50048/0 (socket says 172.21.15.83:50048) 2024-04-24T09:12:45.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.607+0000 7fdc8f400640 1 -- 172.21.15.83:0/28406687 learned_addr learned my addr 172.21.15.83:0/28406687 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:12:45.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.608+0000 7fdc8e000640 1 --2- 172.21.15.83:0/28406687 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb04fb0b30 0x55eb04fb2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:45.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.608+0000 7fdc8f400640 1 -- 172.21.15.83:0/28406687 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb04fb0b30 msgr2=0x55eb04fb2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:45.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.608+0000 7fdc8ea00640 1 --2- 172.21.15.83:0/28406687 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb04fa6460 0x55eb04fb05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:45.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.608+0000 7fdc8f400640 1 --2- 172.21.15.83:0/28406687 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb04fb0b30 0x55eb04fb2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:45.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.608+0000 7fdc8f400640 1 -- 172.21.15.83:0/28406687 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb04fa6460 msgr2=0x55eb04fb05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:45.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.608+0000 7fdc8f400640 1 --2- 172.21.15.83:0/28406687 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb04fa6460 0x55eb04fb05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:45.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.608+0000 7fdc8f400640 1 -- 172.21.15.83:0/28406687 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb04f6e310 con 0x55eb04fa5b40 2024-04-24T09:12:45.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.608+0000 7fdc8ea00640 1 --2- 172.21.15.83:0/28406687 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb04fa6460 0x55eb04fb05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:12:45.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.608+0000 7fdc8e000640 1 --2- 172.21.15.83:0/28406687 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb04fb0b30 0x55eb04fb2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:12:45.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.609+0000 7fdc8f400640 1 --2- 172.21.15.83:0/28406687 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eb04fa5b40 0x55eb04fa5f20 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fdc88009fc0 tx=0x7fdc8800ec30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fcb24318441308 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:45.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.609+0000 7fdc7be00640 1 -- 172.21.15.83:0/28406687 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc8801d070 con 0x55eb04fa5b40 2024-04-24T09:12:45.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.609+0000 7fdc7be00640 1 -- 172.21.15.83:0/28406687 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdc88003e20 con 0x55eb04fa5b40 2024-04-24T09:12:45.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.609+0000 7fdc7be00640 1 -- 172.21.15.83:0/28406687 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc88004670 con 0x55eb04fa5b40 2024-04-24T09:12:45.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.610+0000 7fdc87400640 1 --2- 172.21.15.83:0/1764092797 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fdc740a3da0 0x7fdc740a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:45.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.610+0000 7fdc87400640 1 -- 172.21.15.83:0/1764092797 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 9583+0+0 (secure 0 0 0) 0x7fdc800d0620 con 0x7fdc90104320 2024-04-24T09:12:45.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.610+0000 7fdc987dd740 1 -- 172.21.15.83:0/28406687 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eb04fa5b40 msgr2=0x55eb04fa5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:45.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.610+0000 7fdc987dd740 1 --2- 172.21.15.83:0/28406687 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eb04fa5b40 0x55eb04fa5f20 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fdc88009fc0 tx=0x7fdc8800ec30 comp rx=0 tx=0).stop 2024-04-24T09:12:45.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.610+0000 7fdc987dd740 1 -- 172.21.15.83:0/28406687 shutdown_connections 2024-04-24T09:12:45.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.611+0000 7fdc8ea00640 1 --2- 172.21.15.83:0/1764092797 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fdc740a3da0 0x7fdc740a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:45.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.611+0000 7fdc987dd740 1 --2- 172.21.15.83:0/28406687 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb04fb0b30 0x55eb04fb2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:45.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.611+0000 7fdc987dd740 1 --2- 172.21.15.83:0/28406687 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb04fa6460 0x55eb04fb05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:45.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.611+0000 7fdc987dd740 1 --2- 172.21.15.83:0/28406687 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eb04fa5b40 0x55eb04fa5f20 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:45.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.611+0000 7fdc987dd740 1 -- 172.21.15.83:0/28406687 >> 172.21.15.83:0/28406687 conn(0x55eb04fa1e60 msgr2=0x55eb04fa2240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:45.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.611+0000 7fdc987dd740 1 -- 172.21.15.83:0/28406687 shutdown_connections 2024-04-24T09:12:45.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.611+0000 7fdc987dd740 1 -- 172.21.15.83:0/28406687 wait complete. 2024-04-24T09:12:45.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.611+0000 7fdc8ea00640 1 --2- 172.21.15.83:0/1764092797 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fdc740a3da0 0x7fdc740a6260 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fdc800cda00 tx=0x7fdc800057f0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:45.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.612+0000 7fdc987dd740 1 -- start start 2024-04-24T09:12:45.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.612+0000 7fdc987dd740 1 Processor -- start 2024-04-24T09:12:45.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.613+0000 7fdc987dd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb04fa5b40 0x55eb04fb04f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:45.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.613+0000 7fdc987dd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb04fa6460 0x55eb04fad7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:45.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.613+0000 7fdc987dd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eb04fb0b30 0x55eb04faddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:45.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.613+0000 7fdc987dd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55eb04fb3a60 con 0x55eb04fa6460 2024-04-24T09:12:45.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.613+0000 7fdc987dd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55eb04ec4810 con 0x55eb04fb0b30 2024-04-24T09:12:45.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.613+0000 7fdc987dd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55eb04fb3be0 con 0x55eb04fa5b40 2024-04-24T09:12:45.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.613+0000 7fdc987dd740 10 client.0 _ll_register_callbacks cb 0x55eb04e92140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:12:45.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.614+0000 7fdc8f400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eb04fb0b30 0x55eb04faddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:45.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.614+0000 7fdc8e000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb04fa6460 0x55eb04fad7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:45.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.614+0000 7fdc8f400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eb04fb0b30 0x55eb04faddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:50050/0 (socket says 172.21.15.83:50050) 2024-04-24T09:12:45.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.614+0000 7fdc8f400640 1 -- 172.21.15.83:0/3647030384 learned_addr learned my addr 172.21.15.83:0/3647030384 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:12:45.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.614+0000 7fdc8f400640 1 --2- 172.21.15.83:0/3647030384 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb04fa5b40 0x55eb04fb04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:45.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.614+0000 7fdc8f400640 1 -- 172.21.15.83:0/3647030384 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb04fa5b40 msgr2=0x55eb04fb04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:45.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.614+0000 7fdc8f400640 1 --2- 172.21.15.83:0/3647030384 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb04fa5b40 0x55eb04fb04f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:45.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.614+0000 7fdc8f400640 1 -- 172.21.15.83:0/3647030384 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb04fa6460 msgr2=0x55eb04fad7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:45.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.614+0000 7fdc8f400640 1 --2- 172.21.15.83:0/3647030384 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb04fa6460 0x55eb04fad7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:45.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.614+0000 7fdc8f400640 1 -- 172.21.15.83:0/3647030384 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb04ed53f0 con 0x55eb04fb0b30 2024-04-24T09:12:45.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.615+0000 7fdc78c00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:12:45.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.615+0000 7fdc8f400640 1 --2- 172.21.15.83:0/3647030384 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eb04fb0b30 0x55eb04faddf0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fdc8800efa0 tx=0x7fdc88010040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9b3406fc7ceb8e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:45.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.615+0000 7fdc78c00640 1 -- 172.21.15.83:0/3647030384 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc8802c060 con 0x55eb04fb0b30 2024-04-24T09:12:45.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.616+0000 7fdc78c00640 1 -- 172.21.15.83:0/3647030384 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9583+0+0 (secure 0 0 0) 0x7fdc88004c40 con 0x55eb04fb0b30 2024-04-24T09:12:45.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.616+0000 7fdc987dd740 1 -- 172.21.15.83:0/3647030384 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc88003e20 con 0x55eb04fb0b30 2024-04-24T09:12:45.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.616+0000 7fdc78c00640 1 -- 172.21.15.83:0/3647030384 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7fdc8801d030 con 0x55eb04fb0b30 2024-04-24T09:12:45.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.616+0000 7fdc987dd740 10 client.39929 fetch_fsmap learned FSMap version 64 2024-04-24T09:12:45.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.616+0000 7fdc987dd740 1 -- 172.21.15.83:0/3647030384 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55eb04c1d3f0 con 0x55eb04fb0b30 2024-04-24T09:12:45.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.617+0000 7fdc78c00640 1 -- 172.21.15.83:0/3647030384 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdc88003e20 con 0x55eb04fb0b30 2024-04-24T09:12:45.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.617+0000 7fdc78c00640 1 -- 172.21.15.83:0/3647030384 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 64) ==== 8374+0+0 (secure 0 0 0) 0x7fdc8800b040 con 0x55eb04fb0b30 2024-04-24T09:12:45.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.617+0000 7fdc987dd740 10 client.39929 fetch_fsmap finished waiting for FSMap version 64 2024-04-24T09:12:45.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.617+0000 7fdc987dd740 10 client.39929 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:12:45.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.617+0000 7fdc987dd740 20 client.39929 populate_metadata read hostname 'smithi083' 2024-04-24T09:12:45.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.617+0000 7fdc987dd740 1 --2- 172.21.15.83:0/3647030384 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55eb04fc9440 0x55eb04fcb830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:45.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.617+0000 7fdc987dd740 4 client.39929 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:12:45.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.617+0000 7fdc987dd740 1 -- 172.21.15.83:0/3647030384 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eb04f18bf0 con 0x55eb04fc9440 2024-04-24T09:12:45.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.618+0000 7fdc8e000640 1 --2- 172.21.15.83:0/3647030384 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55eb04fc9440 0x55eb04fcb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:45.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.618+0000 7fdc78c00640 10 client.39929 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:12:45.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.619+0000 7fdc8e000640 1 --2- 172.21.15.83:0/3647030384 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55eb04fc9440 0x55eb04fcb830 crc :-1 s=READY pgs=523 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b200f621f23d4f9b server_cookie=3b6d1777008b6da7 in_seq=0 out_seq=0 2024-04-24T09:12:45.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.620+0000 7fdc78c00640 1 -- 172.21.15.83:0/3647030384 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55eb04f18bf0 con 0x55eb04fc9440 2024-04-24T09:12:45.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.620+0000 7fdc78c00640 10 client.39929 handle_command_reply: tid=0 2024-04-24T09:12:45.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.620+0000 7fdc987dd740 1 client.39929 shutdown 2024-04-24T09:12:45.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.621+0000 7fdc987dd740 1 -- 172.21.15.83:0/3647030384 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eb04fb0b30 msgr2=0x55eb04faddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:45.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.621+0000 7fdc987dd740 1 --2- 172.21.15.83:0/3647030384 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eb04fb0b30 0x55eb04faddf0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fdc8800efa0 tx=0x7fdc88010040 comp rx=0 tx=0).stop 2024-04-24T09:12:45.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.621+0000 7fdc987dd740 1 -- 172.21.15.83:0/3647030384 shutdown_connections 2024-04-24T09:12:45.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.621+0000 7fdc987dd740 1 --2- 172.21.15.83:0/3647030384 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55eb04fc9440 0x55eb04fcb830 crc :-1 s=READY pgs=523 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:45.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.621+0000 7fdc987dd740 1 --2- 172.21.15.83:0/3647030384 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eb04fb0b30 0x55eb04faddf0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:45.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.621+0000 7fdc987dd740 1 --2- 172.21.15.83:0/3647030384 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb04fa6460 0x55eb04fad7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:45.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.621+0000 7fdc987dd740 1 --2- 172.21.15.83:0/3647030384 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb04fa5b40 0x55eb04fb04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:45.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.621+0000 7fdc987dd740 1 -- 172.21.15.83:0/3647030384 >> 172.21.15.83:0/3647030384 conn(0x55eb04fa1e60 msgr2=0x55eb04fa7b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:45.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.621+0000 7fdc78c00640 0 client.39929 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:12:45.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.622+0000 7fdc987dd740 1 -- 172.21.15.83:0/3647030384 shutdown_connections 2024-04-24T09:12:45.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.622+0000 7fdc987dd740 1 -- 172.21.15.83:0/3647030384 wait complete. 2024-04-24T09:12:45.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.622+0000 7fdc987dd740 20 client.39929 trim_cache size 0 max 16384 2024-04-24T09:12:45.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.648+0000 7fdc987dd740 1 Processor -- start 2024-04-24T09:12:45.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.649+0000 7fdc987dd740 1 -- start start 2024-04-24T09:12:45.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.649+0000 7fdc987dd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb04fa5b40 0x55eb04fb0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:45.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.649+0000 7fdc987dd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eb04fa6460 0x55eb04fb1f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:45.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.649+0000 7fdc987dd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb04fb0b30 0x55eb04fb2460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:45.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.649+0000 7fdc987dd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdc901ad890 con 0x55eb04fb0b30 2024-04-24T09:12:45.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.649+0000 7fdc987dd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdc901af460 con 0x55eb04fa6460 2024-04-24T09:12:45.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.649+0000 7fdc987dd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdc901adf60 con 0x55eb04fa5b40 2024-04-24T09:12:45.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.650+0000 7fdc8e000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eb04fa6460 0x55eb04fb1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:45.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.650+0000 7fdc8e000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eb04fa6460 0x55eb04fb1f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:50062/0 (socket says 172.21.15.83:50062) 2024-04-24T09:12:45.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.650+0000 7fdc8e000640 1 -- 172.21.15.83:0/2197554219 learned_addr learned my addr 172.21.15.83:0/2197554219 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:12:45.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.650+0000 7fdc8f400640 1 --2- 172.21.15.83:0/2197554219 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb04fa5b40 0x55eb04fb0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:45.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.650+0000 7fdc8f400640 1 --2- 172.21.15.83:0/2197554219 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb04fb0b30 0x55eb04fb2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:45.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.650+0000 7fdc8e000640 1 -- 172.21.15.83:0/2197554219 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb04fa5b40 msgr2=0x55eb04fb0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:45.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.650+0000 7fdc8e000640 1 --2- 172.21.15.83:0/2197554219 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb04fa5b40 0x55eb04fb0020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:45.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.650+0000 7fdc8e000640 1 -- 172.21.15.83:0/2197554219 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb04fb0b30 msgr2=0x55eb04fb2460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:45.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.650+0000 7fdc8e000640 1 --2- 172.21.15.83:0/2197554219 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb04fb0b30 0x55eb04fb2460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:45.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.650+0000 7fdc8e000640 1 -- 172.21.15.83:0/2197554219 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb04ce77d0 con 0x55eb04fa6460 2024-04-24T09:12:45.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.651+0000 7fdc8e000640 1 --2- 172.21.15.83:0/2197554219 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eb04fa6460 0x55eb04fb1f20 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c035040 tx=0x7fdc7c009530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f63a67cc90037551 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:45.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.651+0000 7fdc7b400640 1 -- 172.21.15.83:0/2197554219 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc7c047070 con 0x55eb04fa6460 2024-04-24T09:12:45.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.651+0000 7fdc7b400640 1 -- 172.21.15.83:0/2197554219 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdc7c041e50 con 0x55eb04fa6460 2024-04-24T09:12:45.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.651+0000 7fdc7b400640 1 -- 172.21.15.83:0/2197554219 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc7c041070 con 0x55eb04fa6460 2024-04-24T09:12:45.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.652+0000 7fdc987dd740 1 -- 172.21.15.83:0/2197554219 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eb04fa6460 msgr2=0x55eb04fb1f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:45.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.652+0000 7fdc987dd740 1 --2- 172.21.15.83:0/2197554219 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eb04fa6460 0x55eb04fb1f20 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c035040 tx=0x7fdc7c009530 comp rx=0 tx=0).stop 2024-04-24T09:12:45.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.653+0000 7fdc987dd740 1 -- 172.21.15.83:0/2197554219 shutdown_connections 2024-04-24T09:12:45.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.653+0000 7fdc987dd740 1 --2- 172.21.15.83:0/2197554219 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb04fb0b30 0x55eb04fb2460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:45.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.653+0000 7fdc987dd740 1 --2- 172.21.15.83:0/2197554219 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eb04fa6460 0x55eb04fb1f20 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:45.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.653+0000 7fdc987dd740 1 --2- 172.21.15.83:0/2197554219 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb04fa5b40 0x55eb04fb0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:45.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.653+0000 7fdc987dd740 1 -- 172.21.15.83:0/2197554219 >> 172.21.15.83:0/2197554219 conn(0x55eb04fa1e60 msgr2=0x55eb04fbc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:45.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.653+0000 7fdc987dd740 1 -- 172.21.15.83:0/2197554219 shutdown_connections 2024-04-24T09:12:45.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.653+0000 7fdc987dd740 1 -- 172.21.15.83:0/2197554219 wait complete. 2024-04-24T09:12:45.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.653+0000 7fdc987dd740 1 -- start start 2024-04-24T09:12:45.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.654+0000 7fdc987dd740 1 Processor -- start 2024-04-24T09:12:45.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.655+0000 7fdc987dd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb04fa5b40 0x55eb04fc9440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:45.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.655+0000 7fdc8f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb04fa5b40 0x55eb04fc9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:45.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.655+0000 7fdc8f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb04fa5b40 0x55eb04fc9440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44264/0 (socket says 172.21.15.83:44264) 2024-04-24T09:12:45.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.655+0000 7fdc8f400640 1 -- 172.21.15.83:0/2383868136 learned_addr learned my addr 172.21.15.83:0/2383868136 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:12:45.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.656+0000 7fdc987dd740 1 --2- 172.21.15.83:0/2383868136 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eb04fa6460 0x55eb04fc9980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:45.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.656+0000 7fdc987dd740 1 --2- 172.21.15.83:0/2383868136 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb04fb0b30 0x55eb0502d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:45.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.656+0000 7fdc987dd740 1 -- 172.21.15.83:0/2383868136 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdc901aed60 con 0x55eb04fa5b40 2024-04-24T09:12:45.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.656+0000 7fdc987dd740 1 -- 172.21.15.83:0/2383868136 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdc901ae690 con 0x55eb04fa6460 2024-04-24T09:12:45.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.656+0000 7fdc987dd740 1 -- 172.21.15.83:0/2383868136 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdc901ab850 con 0x55eb04fb0b30 2024-04-24T09:12:45.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.656+0000 7fdc987dd740 10 client.0 _ll_register_callbacks cb 0x7fdc8801b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:12:45.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.656+0000 7fdc8e000640 1 --2- 172.21.15.83:0/2383868136 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eb04fa6460 0x55eb04fc9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:45.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.657+0000 7fdc8f400640 1 -- 172.21.15.83:0/2383868136 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb04fb0b30 msgr2=0x55eb0502d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:45.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.657+0000 7fdc8f400640 1 --2- 172.21.15.83:0/2383868136 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb04fb0b30 0x55eb0502d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:45.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.657+0000 7fdc8f400640 1 -- 172.21.15.83:0/2383868136 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eb04fa6460 msgr2=0x55eb04fc9980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:45.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.657+0000 7fdc8f400640 1 --2- 172.21.15.83:0/2383868136 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eb04fa6460 0x55eb04fc9980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:45.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.657+0000 7fdc8f400640 1 -- 172.21.15.83:0/2383868136 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb0502db40 con 0x55eb04fa5b40 2024-04-24T09:12:45.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.657+0000 7fdc8f400640 1 --2- 172.21.15.83:0/2383868136 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb04fa5b40 0x55eb04fc9440 secure :-1 s=READY pgs=2589 cs=0 l=1 rev1=1 crypto rx=0x7fdc8802fd60 tx=0x7fdc8800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dde5aa6caac0657 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:45.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.657+0000 7fdc78c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:12:45.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.657+0000 7fdc78c00640 1 -- 172.21.15.83:0/2383868136 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc8801d030 con 0x55eb04fa5b40 2024-04-24T09:12:45.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.658+0000 7fdc78c00640 1 -- 172.21.15.83:0/2383868136 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9583+0+0 (secure 0 0 0) 0x7fdc88036040 con 0x55eb04fa5b40 2024-04-24T09:12:45.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.658+0000 7fdc987dd740 1 -- 172.21.15.83:0/2383868136 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc7c041e50 con 0x55eb04fa5b40 2024-04-24T09:12:45.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.658+0000 7fdc78c00640 1 -- 172.21.15.83:0/2383868136 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7fdc88009540 con 0x55eb04fa5b40 2024-04-24T09:12:45.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.658+0000 7fdc987dd740 10 client.22893 fetch_fsmap learned FSMap version 64 2024-04-24T09:12:45.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.658+0000 7fdc987dd740 1 -- 172.21.15.83:0/2383868136 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55eb0502ed30 con 0x55eb04fa5b40 2024-04-24T09:12:45.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.659+0000 7fdc78c00640 1 -- 172.21.15.83:0/2383868136 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdc7c041e50 con 0x55eb04fa5b40 2024-04-24T09:12:45.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.660+0000 7fdc78c00640 1 -- 172.21.15.83:0/2383868136 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 64) ==== 8374+0+0 (secure 0 0 0) 0x7fdc8800b040 con 0x55eb04fa5b40 2024-04-24T09:12:45.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.661+0000 7fdc987dd740 10 client.22893 fetch_fsmap finished waiting for FSMap version 64 2024-04-24T09:12:45.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.661+0000 7fdc987dd740 10 client.22893 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:12:45.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.661+0000 7fdc987dd740 20 client.22893 populate_metadata read hostname 'smithi083' 2024-04-24T09:12:45.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.662+0000 7fdc987dd740 1 --2- 172.21.15.83:0/2383868136 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55eb0502f2b0 0x55eb050316a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:45.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.662+0000 7fdc987dd740 4 client.22893 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:12:45.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.662+0000 7fdc987dd740 1 -- 172.21.15.83:0/2383868136 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55eb04fa1cb0 con 0x55eb0502f2b0 2024-04-24T09:12:45.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.662+0000 7fdc8e000640 1 --2- 172.21.15.83:0/2383868136 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55eb0502f2b0 0x55eb050316a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:45.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.663+0000 7fdc8e000640 1 --2- 172.21.15.83:0/2383868136 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55eb0502f2b0 0x55eb050316a0 crc :-1 s=READY pgs=524 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=666e3ade03fe4f7c server_cookie=5eec9c86047f7a59 in_seq=0 out_seq=0 2024-04-24T09:12:45.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.663+0000 7fdc78c00640 10 client.22893 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:12:45.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.664+0000 7fdc78c00640 1 -- 172.21.15.83:0/2383868136 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55eb04fa1cb0 con 0x55eb0502f2b0 2024-04-24T09:12:45.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.664+0000 7fdc78c00640 10 client.22893 handle_command_reply: tid=0 2024-04-24T09:12:45.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.665+0000 7fdc987dd740 1 client.22893 shutdown 2024-04-24T09:12:45.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.666+0000 7fdc987dd740 1 -- 172.21.15.83:0/2383868136 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb04fa5b40 msgr2=0x55eb04fc9440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:45.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.666+0000 7fdc987dd740 1 --2- 172.21.15.83:0/2383868136 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb04fa5b40 0x55eb04fc9440 secure :-1 s=READY pgs=2589 cs=0 l=1 rev1=1 crypto rx=0x7fdc8802fd60 tx=0x7fdc8800cbf0 comp rx=0 tx=0).stop 2024-04-24T09:12:45.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.666+0000 7fdc987dd740 1 -- 172.21.15.83:0/2383868136 shutdown_connections 2024-04-24T09:12:45.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.666+0000 7fdc987dd740 1 --2- 172.21.15.83:0/2383868136 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55eb0502f2b0 0x55eb050316a0 crc :-1 s=READY pgs=524 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:45.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.666+0000 7fdc987dd740 1 --2- 172.21.15.83:0/2383868136 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb04fb0b30 0x55eb0502d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:45.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.666+0000 7fdc987dd740 1 --2- 172.21.15.83:0/2383868136 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eb04fa6460 0x55eb04fc9980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:45.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.667+0000 7fdc78c00640 0 client.22893 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:12:45.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.667+0000 7fdc987dd740 1 --2- 172.21.15.83:0/2383868136 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb04fa5b40 0x55eb04fc9440 unknown :-1 s=CLOSED pgs=2589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:45.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.667+0000 7fdc987dd740 1 -- 172.21.15.83:0/2383868136 >> 172.21.15.83:0/2383868136 conn(0x55eb04fa1e60 msgr2=0x55eb04fbbe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:45.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.667+0000 7fdc987dd740 1 -- 172.21.15.83:0/2383868136 shutdown_connections 2024-04-24T09:12:45.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.667+0000 7fdc987dd740 1 -- 172.21.15.83:0/2383868136 wait complete. 2024-04-24T09:12:45.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.667+0000 7fdc987dd740 20 client.22893 trim_cache size 0 max 16384 2024-04-24T09:12:45.673 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:12:45.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.669+0000 7fdc8fe00640 1 -- 172.21.15.83:0/1764092797 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fdc740a3da0 msgr2=0x7fdc740a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:45.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.669+0000 7fdc8fe00640 1 --2- 172.21.15.83:0/1764092797 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fdc740a3da0 0x7fdc740a6260 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fdc800cda00 tx=0x7fdc800057f0 comp rx=0 tx=0).stop 2024-04-24T09:12:45.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.669+0000 7fdc8fe00640 1 -- 172.21.15.83:0/1764092797 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc90104320 msgr2=0x7fdc901a07a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:45.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.669+0000 7fdc8fe00640 1 --2- 172.21.15.83:0/1764092797 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc90104320 0x7fdc901a07a0 secure :-1 s=READY pgs=2588 cs=0 l=1 rev1=1 crypto rx=0x7fdc8000d770 tx=0x7fdc8000dc30 comp rx=0 tx=0).stop 2024-04-24T09:12:45.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.671+0000 7fdc8fe00640 1 -- 172.21.15.83:0/1764092797 shutdown_connections 2024-04-24T09:12:45.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.671+0000 7fdc8fe00640 1 --2- 172.21.15.83:0/1764092797 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fdc740a3da0 0x7fdc740a6260 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:45.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.671+0000 7fdc8fe00640 1 --2- 172.21.15.83:0/1764092797 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc9019a950 0x7fdc9019ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:45.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.671+0000 7fdc8fe00640 1 --2- 172.21.15.83:0/1764092797 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdc901093f0 0x7fdc901a0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:45.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.671+0000 7fdc8fe00640 1 --2- 172.21.15.83:0/1764092797 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc90104320 0x7fdc901a07a0 unknown :-1 s=CLOSED pgs=2588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:45.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.671+0000 7fdc8fe00640 1 -- 172.21.15.83:0/1764092797 >> 172.21.15.83:0/1764092797 conn(0x7fdc900ffb40 msgr2=0x7fdc901005d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:45.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.671+0000 7fdc8fe00640 1 -- 172.21.15.83:0/1764092797 shutdown_connections 2024-04-24T09:12:45.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:45.672+0000 7fdc8fe00640 1 -- 172.21.15.83:0/1764092797 wait complete. 2024-04-24T09:12:46.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:12:45 smithi083 ceph-mon[27907]: pgmap v667: 129 pgs: 129 active+clean; 845 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.7 MiB/s wr, 225 op/s 2024-04-24T09:12:46.189 INFO:tasks.workunit.client.0.smithi083.stdout: 10 54456 6.60 MB/sec execute 317 sec latency 86.909 ms 2024-04-24T09:12:46.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:12:45 smithi138 ceph-mon[31441]: pgmap v667: 129 pgs: 129 active+clean; 845 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.7 MiB/s wr, 225 op/s 2024-04-24T09:12:46.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:12:45 smithi189 ceph-mon[31791]: pgmap v667: 129 pgs: 129 active+clean; 845 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.7 MiB/s wr, 225 op/s 2024-04-24T09:12:46.700 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T09:12:46.708 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T09:12:46.708 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T09:12:46.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:46.972+0000 7f89f5c00640 1 -- 172.21.15.83:0/3358182981 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89f01015f0 msgr2=0x7f89f010e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:46.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:46.972+0000 7f89f5c00640 1 --2- 172.21.15.83:0/3358182981 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89f01015f0 0x7f89f010e2b0 secure :-1 s=READY pgs=2590 cs=0 l=1 rev1=1 crypto rx=0x7f89dc009900 tx=0x7f89dc02a410 comp rx=0 tx=0).stop 2024-04-24T09:12:46.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:46.972+0000 7f89f5c00640 1 -- 172.21.15.83:0/3358182981 shutdown_connections 2024-04-24T09:12:46.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:46.972+0000 7f89f5c00640 1 --2- 172.21.15.83:0/3358182981 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f89f010e7f0 0x7f89f0110be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:46.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:46.972+0000 7f89f5c00640 1 --2- 172.21.15.83:0/3358182981 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89f01015f0 0x7f89f010e2b0 secure :-1 s=CLOSED pgs=2590 cs=0 l=1 rev1=1 crypto rx=0x7f89dc009900 tx=0x7f89dc02a410 comp rx=0 tx=0).stop 2024-04-24T09:12:46.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:46.972+0000 7f89f5c00640 1 --2- 172.21.15.83:0/3358182981 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f89f0100c40 0x7f89f0101020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:46.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:46.972+0000 7f89f5c00640 1 -- 172.21.15.83:0/3358182981 >> 172.21.15.83:0/3358182981 conn(0x7f89f00fc880 msgr2=0x7f89f00feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:46.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:46.973+0000 7f89f5c00640 1 -- 172.21.15.83:0/3358182981 shutdown_connections 2024-04-24T09:12:46.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:46.973+0000 7f89f5c00640 1 -- 172.21.15.83:0/3358182981 wait complete. 2024-04-24T09:12:46.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:46.974+0000 7f89f5c00640 1 Processor -- start 2024-04-24T09:12:46.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:46.974+0000 7f89f5c00640 1 -- start start 2024-04-24T09:12:46.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:46.974+0000 7f89f5c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89f0100c40 0x7f89f01a0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:46.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:46.974+0000 7f89f5c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f89f010e7f0 0x7f89f01a0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:46.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:46.974+0000 7f89f5c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f89f019a930 0x7f89f019ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:46.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:46.974+0000 7f89f5c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f89f0114390 con 0x7f89f0100c40 2024-04-24T09:12:46.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:46.974+0000 7f89f5c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f89f0114210 con 0x7f89f019a930 2024-04-24T09:12:46.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:46.974+0000 7f89f5c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f89f0114510 con 0x7f89f010e7f0 2024-04-24T09:12:46.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:46.974+0000 7f89efe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89f0100c40 0x7f89f01a0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:46.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:46.974+0000 7f89efe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89f0100c40 0x7f89f01a0780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44286/0 (socket says 172.21.15.83:44286) 2024-04-24T09:12:46.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:46.974+0000 7f89efe00640 1 -- 172.21.15.83:0/3546555397 learned_addr learned my addr 172.21.15.83:0/3546555397 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:12:46.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:46.975+0000 7f89f5200640 1 --2- 172.21.15.83:0/3546555397 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f89f019a930 0x7f89f019ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:46.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:46.975+0000 7f89ef400640 1 --2- 172.21.15.83:0/3546555397 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f89f010e7f0 0x7f89f01a0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:46.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:46.975+0000 7f89efe00640 1 -- 172.21.15.83:0/3546555397 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f89f010e7f0 msgr2=0x7f89f01a0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:46.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:46.975+0000 7f89efe00640 1 --2- 172.21.15.83:0/3546555397 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f89f010e7f0 0x7f89f01a0cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:46.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:46.975+0000 7f89efe00640 1 -- 172.21.15.83:0/3546555397 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f89f019a930 msgr2=0x7f89f019ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:46.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:46.975+0000 7f89efe00640 1 --2- 172.21.15.83:0/3546555397 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f89f019a930 0x7f89f019ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:46.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:46.975+0000 7f89efe00640 1 -- 172.21.15.83:0/3546555397 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89f019b450 con 0x7f89f0100c40 2024-04-24T09:12:46.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:46.976+0000 7f89efe00640 1 --2- 172.21.15.83:0/3546555397 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89f0100c40 0x7f89f01a0780 secure :-1 s=READY pgs=2591 cs=0 l=1 rev1=1 crypto rx=0x7f89e000d770 tx=0x7f89e000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:46.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:46.976+0000 7f89ecc00640 1 -- 172.21.15.83:0/3546555397 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89e0017070 con 0x7f89f0100c40 2024-04-24T09:12:46.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:46.976+0000 7f89f5c00640 1 -- 172.21.15.83:0/3546555397 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89f019b740 con 0x7f89f0100c40 2024-04-24T09:12:46.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:46.976+0000 7f89ecc00640 1 -- 172.21.15.83:0/3546555397 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f89e00054d0 con 0x7f89f0100c40 2024-04-24T09:12:46.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:46.976+0000 7f89ecc00640 1 -- 172.21.15.83:0/3546555397 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89e0004cf0 con 0x7f89f0100c40 2024-04-24T09:12:46.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:46.976+0000 7f89f5c00640 1 -- 172.21.15.83:0/3546555397 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89f01031a0 con 0x7f89f0100c40 2024-04-24T09:12:46.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:46.981+0000 7f89ecc00640 1 -- 172.21.15.83:0/3546555397 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f89e0004e90 con 0x7f89f0100c40 2024-04-24T09:12:46.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:46.981+0000 7f89fa005740 1 Processor -- start 2024-04-24T09:12:46.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:46.982+0000 7f89fa005740 1 -- start start 2024-04-24T09:12:46.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:46.984+0000 7f89fa005740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd02c76270 0x55fd02c76650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:46.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:46.984+0000 7f89fa005740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd02c76b90 0x55fd02c80d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:46.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:46.984+0000 7f89fa005740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fd02c81260 0x55fd02c83650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:46.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:46.984+0000 7f89fa005740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55fd02bdbe70 con 0x55fd02c76b90 2024-04-24T09:12:46.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:46.984+0000 7f89fa005740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55fd02b82380 con 0x55fd02c81260 2024-04-24T09:12:46.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:46.984+0000 7f89fa005740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55fd02bd7e30 con 0x55fd02c76270 2024-04-24T09:12:46.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:46.984+0000 7f89f5200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd02c76270 0x55fd02c76650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:46.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:46.985+0000 7f89ef400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fd02c81260 0x55fd02c83650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:46.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:46.985+0000 7f89f5200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd02c76270 0x55fd02c76650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:57428/0 (socket says 172.21.15.83:57428) 2024-04-24T09:12:46.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:46.985+0000 7f89f5200640 1 -- 172.21.15.83:0/3919975008 learned_addr learned my addr 172.21.15.83:0/3919975008 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:12:46.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:46.985+0000 7f89efe00640 1 --2- 172.21.15.83:0/3919975008 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd02c76b90 0x55fd02c80d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:46.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:46.987+0000 7f89f5200640 1 -- 172.21.15.83:0/3919975008 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fd02c81260 msgr2=0x55fd02c83650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:46.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:46.987+0000 7f89f5200640 1 --2- 172.21.15.83:0/3919975008 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fd02c81260 0x55fd02c83650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:46.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:46.987+0000 7f89f5200640 1 -- 172.21.15.83:0/3919975008 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd02c76b90 msgr2=0x55fd02c80d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:46.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:46.987+0000 7f89f5200640 1 --2- 172.21.15.83:0/3919975008 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd02c76b90 0x55fd02c80d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:46.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:46.987+0000 7f89f5200640 1 -- 172.21.15.83:0/3919975008 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd029b67e0 con 0x55fd02c76270 2024-04-24T09:12:46.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:46.987+0000 7f89efe00640 1 --2- 172.21.15.83:0/3919975008 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd02c76b90 0x55fd02c80d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:12:46.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:46.987+0000 7f89ecc00640 1 --2- 172.21.15.83:0/3546555397 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f89d40a3da0 0x7f89d40a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:46.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:46.988+0000 7f89f5200640 1 --2- 172.21.15.83:0/3919975008 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd02c76270 0x55fd02c76650 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f89e8009f30 tx=0x7f89e80118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee90ec361ec7b074 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:46.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:46.988+0000 7f89efe00640 1 --2- 172.21.15.83:0/3546555397 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f89d40a3da0 0x7f89d40a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:46.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:46.988+0000 7f89ecc00640 1 -- 172.21.15.83:0/3546555397 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 9583+0+0 (secure 0 0 0) 0x7f89e00d0700 con 0x7f89f0100c40 2024-04-24T09:12:46.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:46.989+0000 7f89e5600640 1 -- 172.21.15.83:0/3919975008 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89e8004d20 con 0x55fd02c76270 2024-04-24T09:12:46.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:46.989+0000 7f89e5600640 1 -- 172.21.15.83:0/3919975008 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f89e8004050 con 0x55fd02c76270 2024-04-24T09:12:46.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:46.989+0000 7f89e5600640 1 -- 172.21.15.83:0/3919975008 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89e8005260 con 0x55fd02c76270 2024-04-24T09:12:46.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:46.989+0000 7f89efe00640 1 --2- 172.21.15.83:0/3546555397 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f89d40a3da0 0x7f89d40a6260 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x55fd02c77830 tx=0x7f89e00d2040 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:46.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:46.991+0000 7f89fa005740 1 -- 172.21.15.83:0/3919975008 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd02c76270 msgr2=0x55fd02c76650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:46.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:46.991+0000 7f89fa005740 1 --2- 172.21.15.83:0/3919975008 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd02c76270 0x55fd02c76650 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f89e8009f30 tx=0x7f89e80118d0 comp rx=0 tx=0).stop 2024-04-24T09:12:46.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:46.991+0000 7f89fa005740 1 -- 172.21.15.83:0/3919975008 shutdown_connections 2024-04-24T09:12:46.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:46.991+0000 7f89fa005740 1 --2- 172.21.15.83:0/3919975008 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fd02c81260 0x55fd02c83650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:46.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:46.991+0000 7f89fa005740 1 --2- 172.21.15.83:0/3919975008 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd02c76b90 0x55fd02c80d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:46.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:46.991+0000 7f89fa005740 1 --2- 172.21.15.83:0/3919975008 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd02c76270 0x55fd02c76650 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:46.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:46.991+0000 7f89fa005740 1 -- 172.21.15.83:0/3919975008 >> 172.21.15.83:0/3919975008 conn(0x55fd02c72590 msgr2=0x55fd02c72970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:46.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:46.992+0000 7f89fa005740 1 -- 172.21.15.83:0/3919975008 shutdown_connections 2024-04-24T09:12:46.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:46.992+0000 7f89fa005740 1 -- 172.21.15.83:0/3919975008 wait complete. 2024-04-24T09:12:46.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:46.992+0000 7f89fa005740 1 -- start start 2024-04-24T09:12:46.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:46.993+0000 7f89fa005740 1 Processor -- start 2024-04-24T09:12:46.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:46.994+0000 7f89fa005740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fd02c76270 0x55fd02c7e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:46.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:46.994+0000 7f89fa005740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd02c76b90 0x55fd02c7e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:46.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:46.994+0000 7f89fa005740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd02c81260 0x55fd02c7ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:46.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:46.994+0000 7f89fa005740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55fd02c84050 con 0x55fd02c81260 2024-04-24T09:12:46.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:46.994+0000 7f89fa005740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55fd02b95660 con 0x55fd02c76270 2024-04-24T09:12:46.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:46.994+0000 7f89fa005740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55fd02c841d0 con 0x55fd02c76b90 2024-04-24T09:12:46.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:46.994+0000 7f89fa005740 10 client.0 _ll_register_callbacks cb 0x55fd02b83aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:12:46.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:46.994+0000 7f89f5200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd02c81260 0x55fd02c7ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:46.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:46.994+0000 7f89f5200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd02c81260 0x55fd02c7ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44306/0 (socket says 172.21.15.83:44306) 2024-04-24T09:12:46.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:46.994+0000 7f89f5200640 1 -- 172.21.15.83:0/1298496194 learned_addr learned my addr 172.21.15.83:0/1298496194 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:12:46.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:46.995+0000 7f89ef400640 1 --2- 172.21.15.83:0/1298496194 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd02c76b90 0x55fd02c7e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:46.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:46.995+0000 7f89f5200640 1 -- 172.21.15.83:0/1298496194 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd02c76b90 msgr2=0x55fd02c7e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:46.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:46.995+0000 7f89f5200640 1 --2- 172.21.15.83:0/1298496194 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd02c76b90 0x55fd02c7e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:46.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:46.995+0000 7f89f5200640 1 -- 172.21.15.83:0/1298496194 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fd02c76270 msgr2=0x55fd02c7e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:46.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:46.995+0000 7f89f5200640 1 --2- 172.21.15.83:0/1298496194 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fd02c76270 0x55fd02c7e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:46.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:46.995+0000 7f89f5200640 1 -- 172.21.15.83:0/1298496194 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd02a16f70 con 0x55fd02c81260 2024-04-24T09:12:46.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:46.995+0000 7f89ef400640 1 --2- 172.21.15.83:0/1298496194 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd02c76b90 0x55fd02c7e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:12:46.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:46.995+0000 7f89f5200640 1 --2- 172.21.15.83:0/1298496194 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd02c81260 0x55fd02c7ece0 secure :-1 s=READY pgs=2592 cs=0 l=1 rev1=1 crypto rx=0x7f89e8009250 tx=0x7f89e8009c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2e26932714ab950 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:46.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:46.996+0000 7f89da000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:12:46.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:46.996+0000 7f89da000640 1 -- 172.21.15.83:0/1298496194 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89e8004410 con 0x55fd02c81260 2024-04-24T09:12:46.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:46.996+0000 7f89da000640 1 -- 172.21.15.83:0/1298496194 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9583+0+0 (secure 0 0 0) 0x7f89e80045b0 con 0x55fd02c81260 2024-04-24T09:12:46.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:46.996+0000 7f89fa005740 1 -- 172.21.15.83:0/1298496194 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89e8004050 con 0x55fd02c81260 2024-04-24T09:12:46.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:46.997+0000 7f89da000640 1 -- 172.21.15.83:0/1298496194 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f89e8024470 con 0x55fd02c81260 2024-04-24T09:12:46.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:46.997+0000 7f89fa005740 10 client.22905 fetch_fsmap learned FSMap version 64 2024-04-24T09:12:46.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:46.997+0000 7f89fa005740 1 -- 172.21.15.83:0/1298496194 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55fd028ae170 con 0x55fd02c81260 2024-04-24T09:12:46.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:46.997+0000 7f89da000640 1 -- 172.21.15.83:0/1298496194 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f89e8004050 con 0x55fd02c81260 2024-04-24T09:12:46.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:46.999+0000 7f89da000640 1 -- 172.21.15.83:0/1298496194 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 64) ==== 8374+0+0 (secure 0 0 0) 0x7f89e802e820 con 0x55fd02c81260 2024-04-24T09:12:46.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:46.999+0000 7f89fa005740 10 client.22905 fetch_fsmap finished waiting for FSMap version 64 2024-04-24T09:12:46.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:46.999+0000 7f89fa005740 10 client.22905 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:12:46.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:46.999+0000 7f89fa005740 20 client.22905 populate_metadata read hostname 'smithi083' 2024-04-24T09:12:46.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:46.999+0000 7f89fa005740 1 --2- 172.21.15.83:0/1298496194 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55fd02c915e0 0x55fd02c939d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:46.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:46.999+0000 7f89fa005740 4 client.22905 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:12:47.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:46.999+0000 7f89fa005740 1 -- 172.21.15.83:0/1298496194 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fd02be9890 con 0x55fd02c915e0 2024-04-24T09:12:47.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.000+0000 7f89ef400640 1 --2- 172.21.15.83:0/1298496194 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55fd02c915e0 0x55fd02c939d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:47.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.001+0000 7f89ef400640 1 --2- 172.21.15.83:0/1298496194 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55fd02c915e0 0x55fd02c939d0 crc :-1 s=READY pgs=525 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ced9631d889223b1 server_cookie=cd09918baa41d7a1 in_seq=0 out_seq=0 2024-04-24T09:12:47.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.001+0000 7f89da000640 10 client.22905 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:12:47.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.003+0000 7f89da000640 1 -- 172.21.15.83:0/1298496194 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fd02be9890 con 0x55fd02c915e0 2024-04-24T09:12:47.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.003+0000 7f89da000640 10 client.22905 handle_command_reply: tid=0 2024-04-24T09:12:47.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.004+0000 7f89fa005740 1 client.22905 shutdown 2024-04-24T09:12:47.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.004+0000 7f89fa005740 1 -- 172.21.15.83:0/1298496194 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd02c81260 msgr2=0x55fd02c7ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:47.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.004+0000 7f89fa005740 1 --2- 172.21.15.83:0/1298496194 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd02c81260 0x55fd02c7ece0 secure :-1 s=READY pgs=2592 cs=0 l=1 rev1=1 crypto rx=0x7f89e8009250 tx=0x7f89e8009c80 comp rx=0 tx=0).stop 2024-04-24T09:12:47.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.004+0000 7f89fa005740 1 -- 172.21.15.83:0/1298496194 shutdown_connections 2024-04-24T09:12:47.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.004+0000 7f89fa005740 1 --2- 172.21.15.83:0/1298496194 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55fd02c915e0 0x55fd02c939d0 crc :-1 s=READY pgs=525 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:47.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.004+0000 7f89fa005740 1 --2- 172.21.15.83:0/1298496194 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd02c81260 0x55fd02c7ece0 unknown :-1 s=CLOSED pgs=2592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:47.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.004+0000 7f89fa005740 1 --2- 172.21.15.83:0/1298496194 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd02c76b90 0x55fd02c7e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:47.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.004+0000 7f89fa005740 1 --2- 172.21.15.83:0/1298496194 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fd02c76270 0x55fd02c7e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:47.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.004+0000 7f89fa005740 1 -- 172.21.15.83:0/1298496194 >> 172.21.15.83:0/1298496194 conn(0x55fd02c72590 msgr2=0x55fd02c78390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:47.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.004+0000 7f89da000640 0 client.22905 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:12:47.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.005+0000 7f89fa005740 1 -- 172.21.15.83:0/1298496194 shutdown_connections 2024-04-24T09:12:47.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.005+0000 7f89fa005740 1 -- 172.21.15.83:0/1298496194 wait complete. 2024-04-24T09:12:47.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.005+0000 7f89fa005740 20 client.22905 trim_cache size 0 max 16384 2024-04-24T09:12:47.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.035+0000 7f89fa005740 1 Processor -- start 2024-04-24T09:12:47.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.036+0000 7f89fa005740 1 -- start start 2024-04-24T09:12:47.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.036+0000 7f89fa005740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fd02c76270 0x55fd02c7db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:47.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.036+0000 7f89fa005740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd02c76b90 0x55fd02c789d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:47.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.036+0000 7f89fa005740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd02c81260 0x55fd02c71570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:47.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.036+0000 7f89fa005740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f89f01b61c0 con 0x55fd02c81260 2024-04-24T09:12:47.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.036+0000 7f89fa005740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f89f01b7d90 con 0x55fd02c76270 2024-04-24T09:12:47.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.036+0000 7f89fa005740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f89f01b6890 con 0x55fd02c76b90 2024-04-24T09:12:47.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.036+0000 7f89f5200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd02c81260 0x55fd02c71570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:47.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.036+0000 7f89f5200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd02c81260 0x55fd02c71570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44322/0 (socket says 172.21.15.83:44322) 2024-04-24T09:12:47.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.036+0000 7f89f5200640 1 -- 172.21.15.83:0/3800140701 learned_addr learned my addr 172.21.15.83:0/3800140701 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:12:47.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.036+0000 7f89ef400640 1 --2- 172.21.15.83:0/3800140701 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd02c76b90 0x55fd02c789d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:47.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.036+0000 7f89f5200640 1 --2- 172.21.15.83:0/3800140701 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fd02c76270 0x55fd02c7db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:47.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.037+0000 7f89f5200640 1 -- 172.21.15.83:0/3800140701 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd02c76b90 msgr2=0x55fd02c789d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:47.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.037+0000 7f89f5200640 1 --2- 172.21.15.83:0/3800140701 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd02c76b90 0x55fd02c789d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:47.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.037+0000 7f89f5200640 1 -- 172.21.15.83:0/3800140701 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fd02c76270 msgr2=0x55fd02c7db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:47.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.037+0000 7f89f5200640 1 --2- 172.21.15.83:0/3800140701 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fd02c76270 0x55fd02c7db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:47.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.037+0000 7f89f5200640 1 -- 172.21.15.83:0/3800140701 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd02c83f30 con 0x55fd02c81260 2024-04-24T09:12:47.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.037+0000 7f89f5200640 1 --2- 172.21.15.83:0/3800140701 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd02c81260 0x55fd02c71570 secure :-1 s=READY pgs=2593 cs=0 l=1 rev1=1 crypto rx=0x7f89e8009420 tx=0x7f89e80021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d160664bda39dd3a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:47.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.037+0000 7f89e4c00640 1 -- 172.21.15.83:0/3800140701 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89e8036070 con 0x55fd02c81260 2024-04-24T09:12:47.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.038+0000 7f89e4c00640 1 -- 172.21.15.83:0/3800140701 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f89e8037040 con 0x55fd02c81260 2024-04-24T09:12:47.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.038+0000 7f89e4c00640 1 -- 172.21.15.83:0/3800140701 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89e8024610 con 0x55fd02c81260 2024-04-24T09:12:47.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.039+0000 7f89fa005740 1 -- 172.21.15.83:0/3800140701 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd02c81260 msgr2=0x55fd02c71570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:47.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.039+0000 7f89fa005740 1 --2- 172.21.15.83:0/3800140701 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd02c81260 0x55fd02c71570 secure :-1 s=READY pgs=2593 cs=0 l=1 rev1=1 crypto rx=0x7f89e8009420 tx=0x7f89e80021d0 comp rx=0 tx=0).stop 2024-04-24T09:12:47.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.039+0000 7f89fa005740 1 -- 172.21.15.83:0/3800140701 shutdown_connections 2024-04-24T09:12:47.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.039+0000 7f89fa005740 1 --2- 172.21.15.83:0/3800140701 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd02c81260 0x55fd02c71570 unknown :-1 s=CLOSED pgs=2593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:47.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.039+0000 7f89fa005740 1 --2- 172.21.15.83:0/3800140701 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd02c76b90 0x55fd02c789d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:47.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.039+0000 7f89fa005740 1 --2- 172.21.15.83:0/3800140701 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fd02c76270 0x55fd02c7db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:47.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.039+0000 7f89fa005740 1 -- 172.21.15.83:0/3800140701 >> 172.21.15.83:0/3800140701 conn(0x55fd02c72590 msgr2=0x55fd02c70e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:47.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.039+0000 7f89fa005740 1 -- 172.21.15.83:0/3800140701 shutdown_connections 2024-04-24T09:12:47.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.039+0000 7f89fa005740 1 -- 172.21.15.83:0/3800140701 wait complete. 2024-04-24T09:12:47.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.039+0000 7f89fa005740 1 -- start start 2024-04-24T09:12:47.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.040+0000 7f89fa005740 1 Processor -- start 2024-04-24T09:12:47.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.040+0000 7f89fa005740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd02c76270 0x55fd02c89430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:47.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.040+0000 7f89fa005740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fd02c76b90 0x55fd02c89970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:47.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.040+0000 7f89fa005740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd02c81260 0x55fd02c849f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:47.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.040+0000 7f89fa005740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f89f01b7690 con 0x55fd02c76270 2024-04-24T09:12:47.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.040+0000 7f89fa005740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f89f01b6fc0 con 0x55fd02c76b90 2024-04-24T09:12:47.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.040+0000 7f89fa005740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f89f01b41c0 con 0x55fd02c81260 2024-04-24T09:12:47.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.040+0000 7f89fa005740 10 client.0 _ll_register_callbacks cb 0x7f89e00d6400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:12:47.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.041+0000 7f89ef400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fd02c76b90 0x55fd02c89970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:47.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.041+0000 7f89ef400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fd02c76b90 0x55fd02c89970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:50118/0 (socket says 172.21.15.83:50118) 2024-04-24T09:12:47.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.041+0000 7f89ef400640 1 -- 172.21.15.83:0/2435083322 learned_addr learned my addr 172.21.15.83:0/2435083322 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:12:47.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.041+0000 7f89f5200640 1 --2- 172.21.15.83:0/2435083322 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd02c76270 0x55fd02c89430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:47.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.041+0000 7f89ef400640 1 -- 172.21.15.83:0/2435083322 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd02c81260 msgr2=0x55fd02c849f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:47.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.041+0000 7f89ef400640 1 --2- 172.21.15.83:0/2435083322 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd02c81260 0x55fd02c849f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:47.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.041+0000 7f89ef400640 1 -- 172.21.15.83:0/2435083322 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd02c76270 msgr2=0x55fd02c89430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:47.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.041+0000 7f89ef400640 1 --2- 172.21.15.83:0/2435083322 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd02c76270 0x55fd02c89430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:47.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.041+0000 7f89ef400640 1 -- 172.21.15.83:0/2435083322 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd02c851f0 con 0x55fd02c76b90 2024-04-24T09:12:47.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.042+0000 7f89ef400640 1 --2- 172.21.15.83:0/2435083322 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fd02c76b90 0x55fd02c89970 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f89f0069390 tx=0x7f89dc004550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c95ad32fae55b942 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:47.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.042+0000 7f89da000640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:12:47.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.042+0000 7f89da000640 1 -- 172.21.15.83:0/2435083322 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89dc035040 con 0x55fd02c76b90 2024-04-24T09:12:47.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.042+0000 7f89da000640 1 -- 172.21.15.83:0/2435083322 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9583+0+0 (secure 0 0 0) 0x7f89dc005040 con 0x55fd02c76b90 2024-04-24T09:12:47.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.042+0000 7f89fa005740 1 -- 172.21.15.83:0/2435083322 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89e8037040 con 0x55fd02c76b90 2024-04-24T09:12:47.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.042+0000 7f89da000640 1 -- 172.21.15.83:0/2435083322 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f89dc053050 con 0x55fd02c76b90 2024-04-24T09:12:47.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.043+0000 7f89fa005740 10 client.39950 fetch_fsmap learned FSMap version 64 2024-04-24T09:12:47.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.043+0000 7f89fa005740 1 -- 172.21.15.83:0/2435083322 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55fd02d086b0 con 0x55fd02c76b90 2024-04-24T09:12:47.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.043+0000 7f89da000640 1 -- 172.21.15.83:0/2435083322 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f89e8037040 con 0x55fd02c76b90 2024-04-24T09:12:47.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.043+0000 7f89da000640 1 -- 172.21.15.83:0/2435083322 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 64) ==== 8374+0+0 (secure 0 0 0) 0x7f89dc043130 con 0x55fd02c76b90 2024-04-24T09:12:47.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.043+0000 7f89fa005740 10 client.39950 fetch_fsmap finished waiting for FSMap version 64 2024-04-24T09:12:47.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.043+0000 7f89fa005740 10 client.39950 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:12:47.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.043+0000 7f89fa005740 20 client.39950 populate_metadata read hostname 'smithi083' 2024-04-24T09:12:47.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.043+0000 7f89fa005740 1 --2- 172.21.15.83:0/2435083322 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55fd02d08e90 0x55fd02d0b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:47.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.043+0000 7f89fa005740 4 client.39950 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T09:12:47.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.043+0000 7f89fa005740 1 -- 172.21.15.83:0/2435083322 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55fd02c93f00 con 0x55fd02d08e90 2024-04-24T09:12:47.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.044+0000 7f89f5200640 1 --2- 172.21.15.83:0/2435083322 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55fd02d08e90 0x55fd02d0b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:47.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.044+0000 7f89da000640 10 client.39950 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:12:47.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.045+0000 7f89f5200640 1 --2- 172.21.15.83:0/2435083322 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55fd02d08e90 0x55fd02d0b2b0 crc :-1 s=READY pgs=526 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a190e5f0d98f3bef server_cookie=14225f37e8a9847e in_seq=0 out_seq=0 2024-04-24T09:12:47.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.046+0000 7f89da000640 1 -- 172.21.15.83:0/2435083322 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55fd02c93f00 con 0x55fd02d08e90 2024-04-24T09:12:47.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.046+0000 7f89da000640 10 client.39950 handle_command_reply: tid=0 2024-04-24T09:12:47.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.046+0000 7f89fa005740 1 client.39950 shutdown 2024-04-24T09:12:47.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.047+0000 7f89fa005740 1 -- 172.21.15.83:0/2435083322 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fd02c76b90 msgr2=0x55fd02c89970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:47.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.047+0000 7f89fa005740 1 --2- 172.21.15.83:0/2435083322 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fd02c76b90 0x55fd02c89970 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f89f0069390 tx=0x7f89dc004550 comp rx=0 tx=0).stop 2024-04-24T09:12:47.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.047+0000 7f89fa005740 1 -- 172.21.15.83:0/2435083322 shutdown_connections 2024-04-24T09:12:47.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.047+0000 7f89fa005740 1 --2- 172.21.15.83:0/2435083322 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55fd02d08e90 0x55fd02d0b2b0 crc :-1 s=READY pgs=526 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:47.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.047+0000 7f89fa005740 1 --2- 172.21.15.83:0/2435083322 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd02c81260 0x55fd02c849f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:47.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.047+0000 7f89fa005740 1 --2- 172.21.15.83:0/2435083322 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fd02c76b90 0x55fd02c89970 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:47.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.047+0000 7f89da000640 0 client.39950 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:12:47.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.047+0000 7f89fa005740 1 --2- 172.21.15.83:0/2435083322 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd02c76270 0x55fd02c89430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:47.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.047+0000 7f89fa005740 1 -- 172.21.15.83:0/2435083322 >> 172.21.15.83:0/2435083322 conn(0x55fd02c72590 msgr2=0x55fd02c716f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:47.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.048+0000 7f89fa005740 1 -- 172.21.15.83:0/2435083322 shutdown_connections 2024-04-24T09:12:47.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.048+0000 7f89fa005740 1 -- 172.21.15.83:0/2435083322 wait complete. 2024-04-24T09:12:47.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.048+0000 7f89fa005740 20 client.39950 trim_cache size 0 max 16384 2024-04-24T09:12:47.049 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T09:12:47.049 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-04-24T09:12:47.049 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "db37c041-9077-4929-89aa-f445895a6916", 2024-04-24T09:12:47.049 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-04-24T09:12:47.049 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T09:12:47.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.049+0000 7f89f5c00640 1 -- 172.21.15.83:0/3546555397 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f89d40a3da0 msgr2=0x7f89d40a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:47.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.049+0000 7f89f5c00640 1 --2- 172.21.15.83:0/3546555397 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f89d40a3da0 0x7f89d40a6260 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x55fd02c77830 tx=0x7f89e00d2040 comp rx=0 tx=0).stop 2024-04-24T09:12:47.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.049+0000 7f89f5c00640 1 -- 172.21.15.83:0/3546555397 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89f0100c40 msgr2=0x7f89f01a0780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:47.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.049+0000 7f89f5c00640 1 --2- 172.21.15.83:0/3546555397 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89f0100c40 0x7f89f01a0780 secure :-1 s=READY pgs=2591 cs=0 l=1 rev1=1 crypto rx=0x7f89e000d770 tx=0x7f89e000dc30 comp rx=0 tx=0).stop 2024-04-24T09:12:47.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.049+0000 7f89f5c00640 1 -- 172.21.15.83:0/3546555397 shutdown_connections 2024-04-24T09:12:47.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.049+0000 7f89f5c00640 1 --2- 172.21.15.83:0/3546555397 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f89d40a3da0 0x7f89d40a6260 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:47.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.049+0000 7f89f5c00640 1 --2- 172.21.15.83:0/3546555397 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f89f019a930 0x7f89f019ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:47.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.049+0000 7f89f5c00640 1 --2- 172.21.15.83:0/3546555397 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f89f010e7f0 0x7f89f01a0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:47.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.049+0000 7f89f5c00640 1 --2- 172.21.15.83:0/3546555397 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f89f0100c40 0x7f89f01a0780 unknown :-1 s=CLOSED pgs=2591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:47.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.049+0000 7f89f5c00640 1 -- 172.21.15.83:0/3546555397 >> 172.21.15.83:0/3546555397 conn(0x7f89f00fc880 msgr2=0x7f89f00fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:47.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.049+0000 7f89f5c00640 1 -- 172.21.15.83:0/3546555397 shutdown_connections 2024-04-24T09:12:47.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.049+0000 7f89f5c00640 1 -- 172.21.15.83:0/3546555397 wait complete. 2024-04-24T09:12:47.074 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T09:12:47.074 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T09:12:47.189 INFO:tasks.workunit.client.0.smithi083.stdout: 10 54618 6.61 MB/sec execute 318 sec latency 98.754 ms 2024-04-24T09:12:47.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.330+0000 7fa514800640 1 -- 172.21.15.83:0/3050808000 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa50c076530 msgr2=0x7fa50c076910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:47.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.330+0000 7fa514800640 1 --2- 172.21.15.83:0/3050808000 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa50c076530 0x7fa50c076910 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc009930 tx=0x7fa4fc02a420 comp rx=0 tx=0).stop 2024-04-24T09:12:47.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.331+0000 7fa514800640 1 -- 172.21.15.83:0/3050808000 shutdown_connections 2024-04-24T09:12:47.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.331+0000 7fa514800640 1 --2- 172.21.15.83:0/3050808000 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa50c112e80 0x7fa50c115270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:47.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.331+0000 7fa514800640 1 --2- 172.21.15.83:0/3050808000 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa50c076e50 0x7fa50c112940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:47.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.331+0000 7fa514800640 1 --2- 172.21.15.83:0/3050808000 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa50c076530 0x7fa50c076910 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:47.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.331+0000 7fa514800640 1 -- 172.21.15.83:0/3050808000 >> 172.21.15.83:0/3050808000 conn(0x7fa50c0ffc70 msgr2=0x7fa50c102090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:47.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.331+0000 7fa514800640 1 -- 172.21.15.83:0/3050808000 shutdown_connections 2024-04-24T09:12:47.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.331+0000 7fa514800640 1 -- 172.21.15.83:0/3050808000 wait complete. 2024-04-24T09:12:47.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.332+0000 7fa514800640 1 Processor -- start 2024-04-24T09:12:47.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.333+0000 7fa514800640 1 -- start start 2024-04-24T09:12:47.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.333+0000 7fa514800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa50c076530 0x7fa50c0738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:47.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.333+0000 7fa514800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa50c076e50 0x7fa50c06c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:47.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.333+0000 7fa514800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa50c112e80 0x7fa50c06cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:47.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.333+0000 7fa514800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa50c1188e0 con 0x7fa50c076530 2024-04-24T09:12:47.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.333+0000 7fa514800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa50c118760 con 0x7fa50c112e80 2024-04-24T09:12:47.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.333+0000 7fa514800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa50c118a60 con 0x7fa50c076e50 2024-04-24T09:12:47.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.333+0000 7fa511e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa50c076530 0x7fa50c0738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:47.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.334+0000 7fa511e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa50c076530 0x7fa50c0738d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44352/0 (socket says 172.21.15.83:44352) 2024-04-24T09:12:47.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.334+0000 7fa511e00640 1 -- 172.21.15.83:0/937684663 learned_addr learned my addr 172.21.15.83:0/937684663 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:12:47.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.334+0000 7fa511400640 1 --2- 172.21.15.83:0/937684663 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa50c076e50 0x7fa50c06c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:47.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.334+0000 7fa512800640 1 --2- 172.21.15.83:0/937684663 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa50c112e80 0x7fa50c06cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:47.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.334+0000 7fa511400640 1 -- 172.21.15.83:0/937684663 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa50c112e80 msgr2=0x7fa50c06cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:47.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.334+0000 7fa511400640 1 --2- 172.21.15.83:0/937684663 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa50c112e80 0x7fa50c06cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:47.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.334+0000 7fa511400640 1 -- 172.21.15.83:0/937684663 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa50c076530 msgr2=0x7fa50c0738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:47.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.334+0000 7fa511400640 1 --2- 172.21.15.83:0/937684663 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa50c076530 0x7fa50c0738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:47.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.334+0000 7fa511400640 1 -- 172.21.15.83:0/937684663 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa50c06d630 con 0x7fa50c076e50 2024-04-24T09:12:47.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.334+0000 7fa511e00640 1 --2- 172.21.15.83:0/937684663 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa50c076530 0x7fa50c0738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:12:47.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.335+0000 7fa511400640 1 --2- 172.21.15.83:0/937684663 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa50c076e50 0x7fa50c06c9c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa50800c970 tx=0x7fa50800ce30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:47.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.335+0000 7fa502a00640 1 -- 172.21.15.83:0/937684663 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa508007bb0 con 0x7fa50c076e50 2024-04-24T09:12:47.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.335+0000 7fa514800640 1 -- 172.21.15.83:0/937684663 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa50c071460 con 0x7fa50c076e50 2024-04-24T09:12:47.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.335+0000 7fa502a00640 1 -- 172.21.15.83:0/937684663 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa508007d50 con 0x7fa50c076e50 2024-04-24T09:12:47.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.335+0000 7fa502a00640 1 -- 172.21.15.83:0/937684663 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa508018780 con 0x7fa50c076e50 2024-04-24T09:12:47.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.335+0000 7fa514800640 1 -- 172.21.15.83:0/937684663 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa50c071a00 con 0x7fa50c076e50 2024-04-24T09:12:47.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.340+0000 7fa517655740 1 Processor -- start 2024-04-24T09:12:47.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.340+0000 7fa517655740 1 -- start start 2024-04-24T09:12:47.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.340+0000 7fa502a00640 1 -- 172.21.15.83:0/937684663 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa50800f450 con 0x7fa50c076e50 2024-04-24T09:12:47.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.340+0000 7fa517655740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e7aaa73860 0x55e7aaa73c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:47.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.340+0000 7fa517655740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e7aaa74180 0x55e7aaa7e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:47.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.340+0000 7fa517655740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7aaa7e850 0x55e7aaa80c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:47.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.340+0000 7fa517655740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e7aa9d91f0 con 0x55e7aaa7e850 2024-04-24T09:12:47.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.340+0000 7fa517655740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e7aa97f350 con 0x55e7aaa73860 2024-04-24T09:12:47.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.340+0000 7fa517655740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55e7aa9d51b0 con 0x55e7aaa74180 2024-04-24T09:12:47.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.341+0000 7fa511400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7aaa7e850 0x55e7aaa80c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:47.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.341+0000 7fa511400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7aaa7e850 0x55e7aaa80c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44368/0 (socket says 172.21.15.83:44368) 2024-04-24T09:12:47.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.341+0000 7fa511400640 1 -- 172.21.15.83:0/4142111269 learned_addr learned my addr 172.21.15.83:0/4142111269 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:12:47.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.341+0000 7fa511e00640 1 --2- 172.21.15.83:0/4142111269 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e7aaa74180 0x55e7aaa7e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:47.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.341+0000 7fa512800640 1 --2- 172.21.15.83:0/4142111269 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e7aaa73860 0x55e7aaa73c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:47.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.341+0000 7fa511e00640 1 -- 172.21.15.83:0/4142111269 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e7aaa73860 msgr2=0x55e7aaa73c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:47.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.341+0000 7fa511e00640 1 --2- 172.21.15.83:0/4142111269 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e7aaa73860 0x55e7aaa73c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:47.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.341+0000 7fa511e00640 1 -- 172.21.15.83:0/4142111269 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7aaa7e850 msgr2=0x55e7aaa80c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:47.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.341+0000 7fa511e00640 1 --2- 172.21.15.83:0/4142111269 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7aaa7e850 0x55e7aaa80c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:47.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.341+0000 7fa511e00640 1 -- 172.21.15.83:0/4142111269 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7aa9d3f50 con 0x55e7aaa74180 2024-04-24T09:12:47.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.341+0000 7fa511400640 1 --2- 172.21.15.83:0/4142111269 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7aaa7e850 0x55e7aaa80c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:12:47.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.342+0000 7fa511e00640 1 --2- 172.21.15.83:0/4142111269 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e7aaa74180 0x55e7aaa7e310 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc004770 tx=0x7fa4fc004610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e08f1d3d2558631c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:47.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.342+0000 7fa4f7000640 1 -- 172.21.15.83:0/4142111269 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4fc040070 con 0x55e7aaa74180 2024-04-24T09:12:47.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.342+0000 7fa502a00640 1 --2- 172.21.15.83:0/937684663 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa4f80a3d20 0x7fa4f80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:47.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.342+0000 7fa4f7000640 1 -- 172.21.15.83:0/4142111269 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa4fc005020 con 0x55e7aaa74180 2024-04-24T09:12:47.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.342+0000 7fa4f7000640 1 -- 172.21.15.83:0/4142111269 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4fc035040 con 0x55e7aaa74180 2024-04-24T09:12:47.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.342+0000 7fa502a00640 1 -- 172.21.15.83:0/937684663 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 9583+0+0 (secure 0 0 0) 0x7fa5080d0c80 con 0x7fa50c076e50 2024-04-24T09:12:47.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.343+0000 7fa512800640 1 --2- 172.21.15.83:0/937684663 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa4f80a3d20 0x7fa4f80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:47.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.343+0000 7fa517655740 1 -- 172.21.15.83:0/4142111269 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e7aaa74180 msgr2=0x55e7aaa7e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:47.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.343+0000 7fa517655740 1 --2- 172.21.15.83:0/4142111269 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e7aaa74180 0x55e7aaa7e310 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc004770 tx=0x7fa4fc004610 comp rx=0 tx=0).stop 2024-04-24T09:12:47.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.343+0000 7fa512800640 1 --2- 172.21.15.83:0/937684663 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa4f80a3d20 0x7fa4f80a61e0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fa50c06b3f0 tx=0x7fa504009600 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:47.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.344+0000 7fa517655740 1 -- 172.21.15.83:0/4142111269 shutdown_connections 2024-04-24T09:12:47.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.344+0000 7fa517655740 1 --2- 172.21.15.83:0/4142111269 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7aaa7e850 0x55e7aaa80c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:47.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.344+0000 7fa517655740 1 --2- 172.21.15.83:0/4142111269 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e7aaa74180 0x55e7aaa7e310 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:47.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.344+0000 7fa517655740 1 --2- 172.21.15.83:0/4142111269 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e7aaa73860 0x55e7aaa73c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:47.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.344+0000 7fa517655740 1 -- 172.21.15.83:0/4142111269 >> 172.21.15.83:0/4142111269 conn(0x55e7aaa6fb80 msgr2=0x55e7aaa6ff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:47.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.344+0000 7fa517655740 1 -- 172.21.15.83:0/4142111269 shutdown_connections 2024-04-24T09:12:47.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.344+0000 7fa517655740 1 -- 172.21.15.83:0/4142111269 wait complete. 2024-04-24T09:12:47.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.345+0000 7fa517655740 1 -- start start 2024-04-24T09:12:47.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.345+0000 7fa517655740 1 Processor -- start 2024-04-24T09:12:47.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.346+0000 7fa517655740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e7aaa73860 0x55e7aaa7b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:47.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.346+0000 7fa517655740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7aaa74180 0x55e7aaa7bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:47.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.346+0000 7fa517655740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e7aaa7e850 0x55e7aaa7c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:47.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.346+0000 7fa517655740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e7aaa815e0 con 0x55e7aaa74180 2024-04-24T09:12:47.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.346+0000 7fa517655740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e7aa992690 con 0x55e7aaa7e850 2024-04-24T09:12:47.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.346+0000 7fa517655740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55e7aaa81760 con 0x55e7aaa73860 2024-04-24T09:12:47.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.346+0000 7fa517655740 10 client.0 _ll_register_callbacks cb 0x55e7aa9ce650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:12:47.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.346+0000 7fa511e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e7aaa73860 0x55e7aaa7b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:47.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.346+0000 7fa511e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e7aaa73860 0x55e7aaa7b7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:57486/0 (socket says 172.21.15.83:57486) 2024-04-24T09:12:47.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.346+0000 7fa511e00640 1 -- 172.21.15.83:0/2057292904 learned_addr learned my addr 172.21.15.83:0/2057292904 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:12:47.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.347+0000 7fa511400640 1 --2- 172.21.15.83:0/2057292904 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7aaa74180 0x55e7aaa7bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:47.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.347+0000 7fa511e00640 1 -- 172.21.15.83:0/2057292904 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e7aaa7e850 msgr2=0x55e7aaa7c260 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-24T09:12:47.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.347+0000 7fa511e00640 1 --2- 172.21.15.83:0/2057292904 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e7aaa7e850 0x55e7aaa7c260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:47.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.347+0000 7fa511e00640 1 -- 172.21.15.83:0/2057292904 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7aaa74180 msgr2=0x55e7aaa7bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:47.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.347+0000 7fa511e00640 1 --2- 172.21.15.83:0/2057292904 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7aaa74180 0x55e7aaa7bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:47.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.347+0000 7fa511e00640 1 -- 172.21.15.83:0/2057292904 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7aa991920 con 0x55e7aaa73860 2024-04-24T09:12:47.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.347+0000 7fa511400640 1 --2- 172.21.15.83:0/2057292904 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7aaa74180 0x55e7aaa7bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:12:47.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.347+0000 7fa511e00640 1 --2- 172.21.15.83:0/2057292904 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e7aaa73860 0x55e7aaa7b7e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc005440 tx=0x7fa4fc005470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7573d1df37e1c9d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:47.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.347+0000 7fa4eb400640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-04-24T09:12:47.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.348+0000 7fa517655740 1 -- 172.21.15.83:0/2057292904 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4fc005020 con 0x55e7aaa73860 2024-04-24T09:12:47.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.348+0000 7fa4eb400640 1 -- 172.21.15.83:0/2057292904 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4fc035070 con 0x55e7aaa73860 2024-04-24T09:12:47.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.348+0000 7fa4eb400640 1 -- 172.21.15.83:0/2057292904 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9583+0+0 (secure 0 0 0) 0x7fa4fc003cc0 con 0x55e7aaa73860 2024-04-24T09:12:47.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.348+0000 7fa4eb400640 1 -- 172.21.15.83:0/2057292904 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7fa4fc040040 con 0x55e7aaa73860 2024-04-24T09:12:47.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.348+0000 7fa517655740 10 client.29779 fetch_fsmap learned FSMap version 64 2024-04-24T09:12:47.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.348+0000 7fa517655740 1 -- 172.21.15.83:0/2057292904 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({fsmap=64}) -- 0x55e7aa7b57d0 con 0x55e7aaa73860 2024-04-24T09:12:47.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.349+0000 7fa4eb400640 1 -- 172.21.15.83:0/2057292904 <== mon.1 v2:172.21.15.189:3300/0 4 ==== fsmap(e 64) ==== 8374+0+0 (secure 0 0 0) 0x7fa4fc004810 con 0x55e7aaa73860 2024-04-24T09:12:47.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.349+0000 7fa517655740 10 client.29779 fetch_fsmap finished waiting for FSMap version 64 2024-04-24T09:12:47.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.349+0000 7fa517655740 10 client.29779 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:12:47.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.349+0000 7fa517655740 20 client.29779 populate_metadata read hostname 'smithi083' 2024-04-24T09:12:47.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.349+0000 7fa517655740 1 --2- 172.21.15.83:0/2057292904 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55e7aaa8e8c0 0x55e7aaa90cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:47.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.349+0000 7fa517655740 4 client.29779 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:12:47.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.349+0000 7fa517655740 1 -- 172.21.15.83:0/2057292904 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e7aa9e6c10 con 0x55e7aaa8e8c0 2024-04-24T09:12:47.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.349+0000 7fa511400640 1 --2- 172.21.15.83:0/2057292904 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55e7aaa8e8c0 0x55e7aaa90cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:47.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.350+0000 7fa4eb400640 10 client.29779 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:12:47.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.350+0000 7fa511400640 1 --2- 172.21.15.83:0/2057292904 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55e7aaa8e8c0 0x55e7aaa90cb0 crc :-1 s=READY pgs=527 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4642004e15696ac server_cookie=b9b6d89dbfb6435c in_seq=0 out_seq=0 2024-04-24T09:12:47.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.351+0000 7fa4eb400640 1 -- 172.21.15.83:0/2057292904 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e7aa9e6c10 con 0x55e7aaa8e8c0 2024-04-24T09:12:47.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.351+0000 7fa4eb400640 10 client.29779 handle_command_reply: tid=0 2024-04-24T09:12:47.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.351+0000 7fa517655740 1 client.29779 shutdown 2024-04-24T09:12:47.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.351+0000 7fa517655740 1 -- 172.21.15.83:0/2057292904 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e7aaa73860 msgr2=0x55e7aaa7b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:47.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.351+0000 7fa517655740 1 --2- 172.21.15.83:0/2057292904 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e7aaa73860 0x55e7aaa7b7e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc005440 tx=0x7fa4fc005470 comp rx=0 tx=0).stop 2024-04-24T09:12:47.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.351+0000 7fa517655740 1 -- 172.21.15.83:0/2057292904 shutdown_connections 2024-04-24T09:12:47.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.351+0000 7fa517655740 1 --2- 172.21.15.83:0/2057292904 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55e7aaa8e8c0 0x55e7aaa90cb0 crc :-1 s=READY pgs=527 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:47.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.351+0000 7fa517655740 1 --2- 172.21.15.83:0/2057292904 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e7aaa7e850 0x55e7aaa7c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:47.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.351+0000 7fa517655740 1 --2- 172.21.15.83:0/2057292904 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7aaa74180 0x55e7aaa7bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:47.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.351+0000 7fa517655740 1 --2- 172.21.15.83:0/2057292904 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e7aaa73860 0x55e7aaa7b7e0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:47.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.351+0000 7fa517655740 1 -- 172.21.15.83:0/2057292904 >> 172.21.15.83:0/2057292904 conn(0x55e7aaa6fb80 msgr2=0x55e7aaa75980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:47.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.352+0000 7fa4eb400640 0 client.29779 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:12:47.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.352+0000 7fa517655740 1 -- 172.21.15.83:0/2057292904 shutdown_connections 2024-04-24T09:12:47.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.352+0000 7fa517655740 1 -- 172.21.15.83:0/2057292904 wait complete. 2024-04-24T09:12:47.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.352+0000 7fa517655740 20 client.29779 trim_cache size 0 max 16384 2024-04-24T09:12:47.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.384+0000 7fa517655740 1 Processor -- start 2024-04-24T09:12:47.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.385+0000 7fa517655740 1 -- start start 2024-04-24T09:12:47.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.385+0000 7fa517655740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7aaa73860 0x55e7aaa8a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:47.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.385+0000 7fa517655740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e7aaa74180 0x55e7aaa7a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:47.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.385+0000 7fa517655740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e7aaa7e850 0x55e7aaa96250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:47.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.385+0000 7fa517655740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa50c1be470 con 0x55e7aaa73860 2024-04-24T09:12:47.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.385+0000 7fa517655740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa50c1c0040 con 0x55e7aaa74180 2024-04-24T09:12:47.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.385+0000 7fa517655740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa50c1beb40 con 0x55e7aaa7e850 2024-04-24T09:12:47.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.386+0000 7fa512800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e7aaa7e850 0x55e7aaa96250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:47.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.386+0000 7fa512800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e7aaa7e850 0x55e7aaa96250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:57492/0 (socket says 172.21.15.83:57492) 2024-04-24T09:12:47.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.386+0000 7fa512800640 1 -- 172.21.15.83:0/2472396751 learned_addr learned my addr 172.21.15.83:0/2472396751 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:12:47.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.386+0000 7fa511400640 1 --2- 172.21.15.83:0/2472396751 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e7aaa74180 0x55e7aaa7a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:47.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.386+0000 7fa512800640 1 -- 172.21.15.83:0/2472396751 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e7aaa74180 msgr2=0x55e7aaa7a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:47.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.386+0000 7fa512800640 1 --2- 172.21.15.83:0/2472396751 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e7aaa74180 0x55e7aaa7a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:47.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.386+0000 7fa512800640 1 -- 172.21.15.83:0/2472396751 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7aaa73860 msgr2=0x55e7aaa8a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:47.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.386+0000 7fa512800640 1 --2- 172.21.15.83:0/2472396751 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7aaa73860 0x55e7aaa8a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:47.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.386+0000 7fa512800640 1 -- 172.21.15.83:0/2472396751 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7aaa73fb0 con 0x55e7aaa7e850 2024-04-24T09:12:47.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.387+0000 7fa512800640 1 --2- 172.21.15.83:0/2472396751 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e7aaa7e850 0x55e7aaa96250 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa504009a80 tx=0x7fa504016000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=318af8016f152b0d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:47.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.388+0000 7fa4f6600640 1 -- 172.21.15.83:0/2472396751 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5040206e0 con 0x55e7aaa7e850 2024-04-24T09:12:47.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.388+0000 7fa4f6600640 1 -- 172.21.15.83:0/2472396751 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa504024070 con 0x55e7aaa7e850 2024-04-24T09:12:47.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.388+0000 7fa4f6600640 1 -- 172.21.15.83:0/2472396751 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa504029cd0 con 0x55e7aaa7e850 2024-04-24T09:12:47.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.389+0000 7fa517655740 1 -- 172.21.15.83:0/2472396751 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e7aaa7e850 msgr2=0x55e7aaa96250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:47.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.389+0000 7fa517655740 1 --2- 172.21.15.83:0/2472396751 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e7aaa7e850 0x55e7aaa96250 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa504009a80 tx=0x7fa504016000 comp rx=0 tx=0).stop 2024-04-24T09:12:47.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.389+0000 7fa517655740 1 -- 172.21.15.83:0/2472396751 shutdown_connections 2024-04-24T09:12:47.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.389+0000 7fa517655740 1 --2- 172.21.15.83:0/2472396751 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e7aaa7e850 0x55e7aaa96250 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:47.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.389+0000 7fa517655740 1 --2- 172.21.15.83:0/2472396751 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e7aaa74180 0x55e7aaa7a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:47.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.389+0000 7fa517655740 1 --2- 172.21.15.83:0/2472396751 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7aaa73860 0x55e7aaa8a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:47.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.389+0000 7fa517655740 1 -- 172.21.15.83:0/2472396751 >> 172.21.15.83:0/2472396751 conn(0x55e7aaa6fb80 msgr2=0x55e7aaa6ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:47.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.390+0000 7fa517655740 1 -- 172.21.15.83:0/2472396751 shutdown_connections 2024-04-24T09:12:47.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.390+0000 7fa517655740 1 -- 172.21.15.83:0/2472396751 wait complete. 2024-04-24T09:12:47.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.390+0000 7fa517655740 1 -- start start 2024-04-24T09:12:47.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.390+0000 7fa517655740 1 Processor -- start 2024-04-24T09:12:47.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.391+0000 7fa517655740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e7aaa73860 0x55e7aaa86680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:47.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.391+0000 7fa517655740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e7aaa74180 0x55e7aaa86bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:47.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.391+0000 7fa517655740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7aaa7e850 0x55e7aaa82b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:47.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.391+0000 7fa517655740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa50c1bf940 con 0x55e7aaa7e850 2024-04-24T09:12:47.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.391+0000 7fa517655740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa50c1bf270 con 0x55e7aaa74180 2024-04-24T09:12:47.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.391+0000 7fa517655740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa50c1bc470 con 0x55e7aaa73860 2024-04-24T09:12:47.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.391+0000 7fa517655740 10 client.0 _ll_register_callbacks cb 0x7fa5080d4600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:12:47.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.391+0000 7fa512800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7aaa7e850 0x55e7aaa82b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:47.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.392+0000 7fa511400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e7aaa74180 0x55e7aaa86bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:47.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.392+0000 7fa512800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7aaa7e850 0x55e7aaa82b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44400/0 (socket says 172.21.15.83:44400) 2024-04-24T09:12:47.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.392+0000 7fa512800640 1 -- 172.21.15.83:0/3786543333 learned_addr learned my addr 172.21.15.83:0/3786543333 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:12:47.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.392+0000 7fa511400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e7aaa74180 0x55e7aaa86bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:50152/0 (socket says 172.21.15.83:50152) 2024-04-24T09:12:47.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.392+0000 7fa511e00640 1 --2- 172.21.15.83:0/3786543333 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e7aaa73860 0x55e7aaa86680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:47.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.393+0000 7fa512800640 1 -- 172.21.15.83:0/3786543333 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e7aaa73860 msgr2=0x55e7aaa86680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:47.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.393+0000 7fa512800640 1 --2- 172.21.15.83:0/3786543333 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e7aaa73860 0x55e7aaa86680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:47.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.393+0000 7fa512800640 1 -- 172.21.15.83:0/3786543333 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e7aaa74180 msgr2=0x55e7aaa86bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:47.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.393+0000 7fa512800640 1 --2- 172.21.15.83:0/3786543333 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e7aaa74180 0x55e7aaa86bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:47.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.393+0000 7fa512800640 1 -- 172.21.15.83:0/3786543333 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7aa6eb3f0 con 0x55e7aaa7e850 2024-04-24T09:12:47.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.393+0000 7fa512800640 1 --2- 172.21.15.83:0/3786543333 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7aaa7e850 0x55e7aaa82b50 secure :-1 s=READY pgs=2594 cs=0 l=1 rev1=1 crypto rx=0x7fa504029f10 tx=0x7fa504029f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d78ae19c5fa0166 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:47.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.393+0000 7fa4eb400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:12:47.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.394+0000 7fa4eb400640 1 -- 172.21.15.83:0/3786543333 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa504003d80 con 0x55e7aaa7e850 2024-04-24T09:12:47.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.394+0000 7fa4eb400640 1 -- 172.21.15.83:0/3786543333 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9583+0+0 (secure 0 0 0) 0x7fa504004030 con 0x55e7aaa7e850 2024-04-24T09:12:47.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.394+0000 7fa517655740 1 -- 172.21.15.83:0/3786543333 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa504024070 con 0x55e7aaa7e850 2024-04-24T09:12:47.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.395+0000 7fa4eb400640 1 -- 172.21.15.83:0/3786543333 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7fa504009df0 con 0x55e7aaa7e850 2024-04-24T09:12:47.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.395+0000 7fa517655740 10 client.22920 fetch_fsmap learned FSMap version 64 2024-04-24T09:12:47.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.395+0000 7fa517655740 1 -- 172.21.15.83:0/3786543333 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55e7aa6ab0f0 con 0x55e7aaa7e850 2024-04-24T09:12:47.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.395+0000 7fa4eb400640 1 -- 172.21.15.83:0/3786543333 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa504024070 con 0x55e7aaa7e850 2024-04-24T09:12:47.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.396+0000 7fa4eb400640 1 -- 172.21.15.83:0/3786543333 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 64) ==== 8374+0+0 (secure 0 0 0) 0x7fa50400f040 con 0x55e7aaa7e850 2024-04-24T09:12:47.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.396+0000 7fa517655740 10 client.22920 fetch_fsmap finished waiting for FSMap version 64 2024-04-24T09:12:47.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.396+0000 7fa517655740 10 client.22920 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:12:47.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.396+0000 7fa517655740 20 client.22920 populate_metadata read hostname 'smithi083' 2024-04-24T09:12:47.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.396+0000 7fa517655740 1 --2- 172.21.15.83:0/3786543333 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55e7aaaf5760 0x55e7aaaf7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:47.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.396+0000 7fa517655740 4 client.22920 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T09:12:47.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.396+0000 7fa517655740 1 -- 172.21.15.83:0/3786543333 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e7aa9e0a80 con 0x55e7aaaf5760 2024-04-24T09:12:47.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.397+0000 7fa511e00640 1 --2- 172.21.15.83:0/3786543333 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55e7aaaf5760 0x55e7aaaf7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:47.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.398+0000 7fa511e00640 1 --2- 172.21.15.83:0/3786543333 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55e7aaaf5760 0x55e7aaaf7b80 crc :-1 s=READY pgs=528 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c17e28267d06a598 server_cookie=6dac19fbc50cedf0 in_seq=0 out_seq=0 2024-04-24T09:12:47.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.398+0000 7fa4eb400640 10 client.22920 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:12:47.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.398+0000 7fa4eb400640 1 -- 172.21.15.83:0/3786543333 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55e7aa9e0a80 con 0x55e7aaaf5760 2024-04-24T09:12:47.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.398+0000 7fa4eb400640 10 client.22920 handle_command_reply: tid=0 2024-04-24T09:12:47.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.398+0000 7fa517655740 1 client.22920 shutdown 2024-04-24T09:12:47.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.399+0000 7fa517655740 1 -- 172.21.15.83:0/3786543333 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7aaa7e850 msgr2=0x55e7aaa82b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:47.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.399+0000 7fa517655740 1 --2- 172.21.15.83:0/3786543333 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7aaa7e850 0x55e7aaa82b50 secure :-1 s=READY pgs=2594 cs=0 l=1 rev1=1 crypto rx=0x7fa504029f10 tx=0x7fa504029f40 comp rx=0 tx=0).stop 2024-04-24T09:12:47.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.399+0000 7fa517655740 1 -- 172.21.15.83:0/3786543333 shutdown_connections 2024-04-24T09:12:47.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.399+0000 7fa517655740 1 --2- 172.21.15.83:0/3786543333 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55e7aaaf5760 0x55e7aaaf7b80 crc :-1 s=READY pgs=528 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:47.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.399+0000 7fa517655740 1 --2- 172.21.15.83:0/3786543333 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7aaa7e850 0x55e7aaa82b50 unknown :-1 s=CLOSED pgs=2594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:47.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.399+0000 7fa4eb400640 0 client.22920 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:12:47.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.399+0000 7fa517655740 1 --2- 172.21.15.83:0/3786543333 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e7aaa74180 0x55e7aaa86bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:47.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.399+0000 7fa517655740 1 --2- 172.21.15.83:0/3786543333 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e7aaa73860 0x55e7aaa86680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:47.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.399+0000 7fa517655740 1 -- 172.21.15.83:0/3786543333 >> 172.21.15.83:0/3786543333 conn(0x55e7aaa6fb80 msgr2=0x55e7aaa74fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:47.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.399+0000 7fa517655740 1 -- 172.21.15.83:0/3786543333 shutdown_connections 2024-04-24T09:12:47.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.400+0000 7fa517655740 1 -- 172.21.15.83:0/3786543333 wait complete. 2024-04-24T09:12:47.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.400+0000 7fa517655740 20 client.22920 trim_cache size 0 max 16384 2024-04-24T09:12:47.402 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T09:12:47.403 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T09:12:47.403 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-04-24T09:12:47.403 INFO:teuthology.orchestra.run.smithi083.stdout: "db37c041-9077-4929-89aa-f445895a6916": { 2024-04-24T09:12:47.403 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-04-24T09:12:47.403 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "db37c041-9077-4929-89aa-f445895a6916", 2024-04-24T09:12:47.403 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-04-24T09:12:47.403 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:12:47.403 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:12:47.403 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T09:12:47.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.400+0000 7fa514800640 1 -- 172.21.15.83:0/937684663 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa4f80a3d20 msgr2=0x7fa4f80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:47.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.401+0000 7fa514800640 1 --2- 172.21.15.83:0/937684663 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa4f80a3d20 0x7fa4f80a61e0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fa50c06b3f0 tx=0x7fa504009600 comp rx=0 tx=0).stop 2024-04-24T09:12:47.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.401+0000 7fa514800640 1 -- 172.21.15.83:0/937684663 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa50c076e50 msgr2=0x7fa50c06c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:47.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.401+0000 7fa514800640 1 --2- 172.21.15.83:0/937684663 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa50c076e50 0x7fa50c06c9c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa50800c970 tx=0x7fa50800ce30 comp rx=0 tx=0).stop 2024-04-24T09:12:47.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.401+0000 7fa514800640 1 -- 172.21.15.83:0/937684663 shutdown_connections 2024-04-24T09:12:47.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.401+0000 7fa514800640 1 --2- 172.21.15.83:0/937684663 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa4f80a3d20 0x7fa4f80a61e0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:47.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.401+0000 7fa514800640 1 --2- 172.21.15.83:0/937684663 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa50c112e80 0x7fa50c06cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:47.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.401+0000 7fa514800640 1 --2- 172.21.15.83:0/937684663 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa50c076e50 0x7fa50c06c9c0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:47.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.401+0000 7fa514800640 1 --2- 172.21.15.83:0/937684663 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa50c076530 0x7fa50c0738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:47.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.401+0000 7fa514800640 1 -- 172.21.15.83:0/937684663 >> 172.21.15.83:0/937684663 conn(0x7fa50c0ffc70 msgr2=0x7fa50c102060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:47.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.401+0000 7fa514800640 1 -- 172.21.15.83:0/937684663 shutdown_connections 2024-04-24T09:12:47.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:12:47.401+0000 7fa514800640 1 -- 172.21.15.83:0/937684663 wait complete. 2024-04-24T09:12:47.425 INFO:tasks.cephfs.filesystem:scrub status for tag:db37c041-9077-4929-89aa-f445895a6916 - {'path': '/', 'tag': 'db37c041-9077-4929-89aa-f445895a6916', 'options': 'recursive,force'} 2024-04-24T09:12:48.189 INFO:tasks.workunit.client.0.smithi083.stdout: 10 54774 6.63 MB/sec execute 319 sec latency 110.281 ms 2024-04-24T09:12:48.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:12:47 smithi083 ceph-mon[27907]: pgmap v668: 129 pgs: 129 active+clean; 845 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.7 MiB/s wr, 225 op/s 2024-04-24T09:12:48.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:12:47 smithi083 ceph-mon[27907]: scrub queued for path: / 2024-04-24T09:12:48.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:12:47 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:12:48.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:12:47 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:12:48.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:12:47 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:12:48.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:12:47 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:12:48.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:12:47 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:12:48.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:12:47 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:12:48.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:12:47 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:12:48.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:12:47 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:12:48.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:12:47 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:12:48.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:12:47 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:12:48.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:12:47 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:12:48.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:12:47 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:12:48.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:12:47 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:12:48.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:12:47 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:12:48.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:12:47 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:12:48.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:12:47 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:12:48.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:12:47 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:12:48.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:12:47 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:12:48.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:12:47 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:12:48.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:12:47 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:12:48.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:12:47 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:12:48.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:12:47 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:12:48.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:12:47 smithi138 ceph-mon[31441]: pgmap v668: 129 pgs: 129 active+clean; 845 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.7 MiB/s wr, 225 op/s 2024-04-24T09:12:48.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:12:47 smithi138 ceph-mon[31441]: scrub queued for path: / 2024-04-24T09:12:48.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:12:47 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:12:48.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:12:47 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:12:48.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:12:47 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:12:48.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:12:48 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:12:48.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:12:48 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:12:48.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:12:48 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:12:48.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:12:48 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:12:48.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:12:48 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:12:48.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:12:48 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:12:48.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:12:48 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:12:48.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:12:48 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:12:48.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:12:48 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:12:48.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:12:48 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:12:48.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:12:48 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:12:48.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:12:48 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:12:48.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:12:48 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:12:48.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:12:48 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:12:48.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:12:48 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:12:48.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:12:48 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:12:48.457 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:12:48 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:12:48.457 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:12:48 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:12:48.457 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:12:48 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:12:48.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:12:47 smithi189 ceph-mon[31791]: pgmap v668: 129 pgs: 129 active+clean; 845 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.7 MiB/s wr, 225 op/s 2024-04-24T09:12:48.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:12:47 smithi189 ceph-mon[31791]: scrub queued for path: / 2024-04-24T09:12:48.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:12:47 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:12:48.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:12:47 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:12:48.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:12:47 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:12:48.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:12:47 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:12:48.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:12:47 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:12:48.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:12:47 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:12:48.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:12:47 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:12:48.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:12:47 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:12:48.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:12:47 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:12:48.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:12:48 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:12:48.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:12:48 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:12:48.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:12:48 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:12:48.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:12:48 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:12:48.459 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:12:48 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:12:48.459 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:12:48 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:12:48.459 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:12:48 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:12:48.459 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:12:48 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:12:48.459 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:12:48 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:12:48.459 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:12:48 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:12:48.459 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:12:48 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:12:48.459 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:12:48 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:12:48.459 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:12:48 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:12:49.189 INFO:tasks.workunit.client.0.smithi083.stdout: 10 54967 6.64 MB/sec execute 320 sec latency 77.503 ms 2024-04-24T09:12:49.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:12:49 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:12:49.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:12:49 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:12:49.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:12:49 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:12:50.190 INFO:tasks.workunit.client.0.smithi083.stdout: 10 55080 6.65 MB/sec execute 321 sec latency 93.419 ms 2024-04-24T09:12:50.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:12:50 smithi083 ceph-mon[27907]: pgmap v669: 129 pgs: 129 active+clean; 860 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.6 MiB/s wr, 361 op/s 2024-04-24T09:12:50.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:12:50 smithi138 ceph-mon[31441]: pgmap v669: 129 pgs: 129 active+clean; 860 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.6 MiB/s wr, 361 op/s 2024-04-24T09:12:50.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:12:50 smithi189 ceph-mon[31791]: pgmap v669: 129 pgs: 129 active+clean; 860 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.6 MiB/s wr, 361 op/s 2024-04-24T09:12:51.189 INFO:tasks.workunit.client.0.smithi083.stdout: 10 55176 6.65 MB/sec execute 322 sec latency 76.611 ms 2024-04-24T09:12:52.190 INFO:tasks.workunit.client.0.smithi083.stdout: 10 55301 6.67 MB/sec execute 323 sec latency 91.122 ms 2024-04-24T09:12:52.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:12:52 smithi083 ceph-mon[27907]: pgmap v670: 129 pgs: 129 active+clean; 860 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.4 MiB/s wr, 259 op/s 2024-04-24T09:12:52.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:12:52 smithi138 ceph-mon[31441]: pgmap v670: 129 pgs: 129 active+clean; 860 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.4 MiB/s wr, 259 op/s 2024-04-24T09:12:52.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:12:52 smithi189 ceph-mon[31791]: pgmap v670: 129 pgs: 129 active+clean; 860 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.4 MiB/s wr, 259 op/s 2024-04-24T09:12:53.190 INFO:tasks.workunit.client.0.smithi083.stdout: 10 55456 6.69 MB/sec execute 324 sec latency 66.455 ms 2024-04-24T09:12:54.190 INFO:tasks.workunit.client.0.smithi083.stdout: 10 55613 6.69 MB/sec execute 325 sec latency 151.067 ms 2024-04-24T09:12:54.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:12:54 smithi083 ceph-mon[27907]: pgmap v671: 129 pgs: 129 active+clean; 876 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.7 MiB/s wr, 346 op/s 2024-04-24T09:12:54.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:12:54 smithi138 ceph-mon[31441]: pgmap v671: 129 pgs: 129 active+clean; 876 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.7 MiB/s wr, 346 op/s 2024-04-24T09:12:54.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:12:54 smithi189 ceph-mon[31791]: pgmap v671: 129 pgs: 129 active+clean; 876 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.7 MiB/s wr, 346 op/s 2024-04-24T09:12:54.778 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.778+0000 7f43c9800640 1 -- 172.21.15.83:0/1499220831 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f43c4102070 msgr2=0x7f43c4102450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:54.778 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.778+0000 7f43c9800640 1 --2- 172.21.15.83:0/1499220831 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f43c4102070 0x7f43c4102450 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f43b4009900 tx=0x7f43b402a3f0 comp rx=0 tx=0).stop 2024-04-24T09:12:54.778 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.778+0000 7f43c9800640 1 -- 172.21.15.83:0/1499220831 shutdown_connections 2024-04-24T09:12:54.778 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.778+0000 7f43c9800640 1 --2- 172.21.15.83:0/1499220831 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43c4107140 0x7f43c410ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:54.778 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.778+0000 7f43c9800640 1 --2- 172.21.15.83:0/1499220831 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43c4102a20 0x7f43c4106a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:54.778 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.778+0000 7f43c9800640 1 --2- 172.21.15.83:0/1499220831 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f43c4102070 0x7f43c4102450 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:54.778 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.778+0000 7f43c9800640 1 -- 172.21.15.83:0/1499220831 >> 172.21.15.83:0/1499220831 conn(0x7f43c40fd950 msgr2=0x7f43c40ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:54.778 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.778+0000 7f43c9800640 1 -- 172.21.15.83:0/1499220831 shutdown_connections 2024-04-24T09:12:54.778 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.778+0000 7f43c9800640 1 -- 172.21.15.83:0/1499220831 wait complete. 2024-04-24T09:12:54.779 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.780+0000 7f43c9800640 1 Processor -- start 2024-04-24T09:12:54.779 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.780+0000 7f43c9800640 1 -- start start 2024-04-24T09:12:54.779 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.780+0000 7f43c9800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43c4102070 0x7f43c41964d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:54.780 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.780+0000 7f43c9800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43c4107140 0x7f43c4196a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:54.781 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.780+0000 7f43c9800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f43c419ad90 0x7f43c419b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:54.781 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.780+0000 7f43c9800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f43c410ff50 con 0x7f43c4107140 2024-04-24T09:12:54.781 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.780+0000 7f43c9800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f43c410fdd0 con 0x7f43c4102070 2024-04-24T09:12:54.781 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.780+0000 7f43c9800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f43c41100d0 con 0x7f43c419ad90 2024-04-24T09:12:54.781 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.781+0000 7f43c3e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43c4102070 0x7f43c41964d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:54.781 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.781+0000 7f43c3e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43c4102070 0x7f43c41964d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:33794/0 (socket says 172.21.15.83:33794) 2024-04-24T09:12:54.781 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.781+0000 7f43c3e00640 1 -- 172.21.15.83:0/3158488779 learned_addr learned my addr 172.21.15.83:0/3158488779 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:12:54.781 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.781+0000 7f43c3400640 1 --2- 172.21.15.83:0/3158488779 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43c4107140 0x7f43c4196a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:54.781 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.781+0000 7f43c8e00640 1 --2- 172.21.15.83:0/3158488779 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f43c419ad90 0x7f43c419b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:54.781 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.781+0000 7f43c3e00640 1 -- 172.21.15.83:0/3158488779 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f43c419ad90 msgr2=0x7f43c419b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:54.781 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.781+0000 7f43c3e00640 1 --2- 172.21.15.83:0/3158488779 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f43c419ad90 0x7f43c419b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:54.781 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.781+0000 7f43c3e00640 1 -- 172.21.15.83:0/3158488779 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43c4107140 msgr2=0x7f43c4196a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:54.781 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.781+0000 7f43c3e00640 1 --2- 172.21.15.83:0/3158488779 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43c4107140 0x7f43c4196a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:54.782 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.781+0000 7f43c3e00640 1 -- 172.21.15.83:0/3158488779 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43c419b880 con 0x7f43c4102070 2024-04-24T09:12:54.782 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.781+0000 7f43c3400640 1 --2- 172.21.15.83:0/3158488779 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43c4107140 0x7f43c4196a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:12:54.782 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.782+0000 7f43c3e00640 1 --2- 172.21.15.83:0/3158488779 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43c4102070 0x7f43c41964d0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f43b4031440 tx=0x7f43b4003cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:54.782 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.782+0000 7f43c0c00640 1 -- 172.21.15.83:0/3158488779 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b403f070 con 0x7f43c4102070 2024-04-24T09:12:54.782 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.782+0000 7f43c9800640 1 -- 172.21.15.83:0/3158488779 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43c419bb10 con 0x7f43c4102070 2024-04-24T09:12:54.783 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.782+0000 7f43c0c00640 1 -- 172.21.15.83:0/3158488779 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f43b4003e40 con 0x7f43c4102070 2024-04-24T09:12:54.783 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.782+0000 7f43c0c00640 1 -- 172.21.15.83:0/3158488779 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b4035030 con 0x7f43c4102070 2024-04-24T09:12:54.783 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.782+0000 7f43c9800640 1 -- 172.21.15.83:0/3158488779 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43c41a3330 con 0x7f43c4102070 2024-04-24T09:12:54.790 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.786+0000 7f43c0c00640 1 -- 172.21.15.83:0/3158488779 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f43b4042070 con 0x7f43c4102070 2024-04-24T09:12:54.790 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.786+0000 7f43cdb99740 1 Processor -- start 2024-04-24T09:12:54.790 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.786+0000 7f43cdb99740 1 -- start start 2024-04-24T09:12:54.791 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.787+0000 7f43cdb99740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560f9dbb4bd0 0x560f9dbb4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:54.791 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.787+0000 7f43cdb99740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f9dbb54f0 0x560f9dbbf870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:54.791 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.787+0000 7f43cdb99740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f9dbbfdb0 0x560f9dbc21a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:54.791 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.787+0000 7f43cdb99740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x560f9db12900 con 0x560f9dbbfdb0 2024-04-24T09:12:54.791 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.787+0000 7f43cdb99740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560f9dab8f20 con 0x560f9dbb54f0 2024-04-24T09:12:54.791 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.787+0000 7f43cdb99740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x560f9db0e8c0 con 0x560f9dbb4bd0 2024-04-24T09:12:54.791 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.787+0000 7f43c3e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f9dbb54f0 0x560f9dbbf870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:54.791 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.787+0000 7f43c0c00640 1 --2- 172.21.15.83:0/3158488779 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f43a80a3d60 0x7f43a80a6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:54.791 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.787+0000 7f43c3e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f9dbb54f0 0x560f9dbbf870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:33806/0 (socket says 172.21.15.83:33806) 2024-04-24T09:12:54.791 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.787+0000 7f43c3e00640 1 -- 172.21.15.83:0/2730570052 learned_addr learned my addr 172.21.15.83:0/2730570052 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:12:54.791 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.788+0000 7f43c0c00640 1 -- 172.21.15.83:0/3158488779 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 9583+0+0 (secure 0 0 0) 0x7f43b40f1570 con 0x7f43c4102070 2024-04-24T09:12:54.791 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.788+0000 7f43c3400640 1 --2- 172.21.15.83:0/2730570052 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f9dbbfdb0 0x560f9dbc21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:54.791 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.788+0000 7f43c3e00640 1 -- 172.21.15.83:0/2730570052 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560f9dbb4bd0 msgr2=0x560f9dbb4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:54.792 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.788+0000 7f43c3e00640 1 --2- 172.21.15.83:0/2730570052 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560f9dbb4bd0 0x560f9dbb4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:54.792 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.788+0000 7f43c3e00640 1 -- 172.21.15.83:0/2730570052 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f9dbbfdb0 msgr2=0x560f9dbc21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:54.792 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.788+0000 7f43c3e00640 1 --2- 172.21.15.83:0/2730570052 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f9dbbfdb0 0x560f9dbc21a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:54.792 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.788+0000 7f43c3e00640 1 -- 172.21.15.83:0/2730570052 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f9dadde10 con 0x560f9dbb54f0 2024-04-24T09:12:54.792 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.788+0000 7f43c3400640 1 --2- 172.21.15.83:0/2730570052 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f9dbbfdb0 0x560f9dbc21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:12:54.792 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.788+0000 7f43c8e00640 1 --2- 172.21.15.83:0/3158488779 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f43a80a3d60 0x7f43a80a6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:54.792 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.788+0000 7f43c3e00640 1 --2- 172.21.15.83:0/2730570052 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f9dbb54f0 0x560f9dbbf870 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f43b40ee780 tx=0x7f43b4047000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=efe0303d8f813277 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:54.792 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.788+0000 7f43b9600640 1 -- 172.21.15.83:0/2730570052 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b4049070 con 0x560f9dbb54f0 2024-04-24T09:12:54.792 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.788+0000 7f43b9600640 1 -- 172.21.15.83:0/2730570052 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f43b40c1440 con 0x560f9dbb54f0 2024-04-24T09:12:54.792 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.788+0000 7f43b9600640 1 -- 172.21.15.83:0/2730570052 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b40eeac0 con 0x560f9dbb54f0 2024-04-24T09:12:54.792 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.789+0000 7f43cdb99740 1 -- 172.21.15.83:0/2730570052 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f9dbb54f0 msgr2=0x560f9dbbf870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:54.792 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.789+0000 7f43cdb99740 1 --2- 172.21.15.83:0/2730570052 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f9dbb54f0 0x560f9dbbf870 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f43b40ee780 tx=0x7f43b4047000 comp rx=0 tx=0).stop 2024-04-24T09:12:54.792 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.789+0000 7f43c8e00640 1 --2- 172.21.15.83:0/3158488779 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f43a80a3d60 0x7f43a80a6220 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f43bc009eb0 tx=0x7f43bc009540 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:54.792 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.790+0000 7f43cdb99740 1 -- 172.21.15.83:0/2730570052 shutdown_connections 2024-04-24T09:12:54.793 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.790+0000 7f43cdb99740 1 --2- 172.21.15.83:0/2730570052 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f9dbbfdb0 0x560f9dbc21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:54.793 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.790+0000 7f43cdb99740 1 --2- 172.21.15.83:0/2730570052 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f9dbb54f0 0x560f9dbbf870 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:54.793 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.790+0000 7f43cdb99740 1 --2- 172.21.15.83:0/2730570052 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560f9dbb4bd0 0x560f9dbb4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:54.793 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.790+0000 7f43cdb99740 1 -- 172.21.15.83:0/2730570052 >> 172.21.15.83:0/2730570052 conn(0x560f9dbb10e0 msgr2=0x560f9dbb14c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:54.793 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.790+0000 7f43cdb99740 1 -- 172.21.15.83:0/2730570052 shutdown_connections 2024-04-24T09:12:54.793 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.790+0000 7f43cdb99740 1 -- 172.21.15.83:0/2730570052 wait complete. 2024-04-24T09:12:54.793 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.790+0000 7f43cdb99740 1 -- start start 2024-04-24T09:12:54.793 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.790+0000 7f43cdb99740 1 Processor -- start 2024-04-24T09:12:54.794 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.791+0000 7f43cdb99740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560f9dbb4bd0 0x560f9dbbcd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:54.794 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.791+0000 7f43cdb99740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f9dbb54f0 0x560f9dbbd260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:54.794 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.792+0000 7f43cdb99740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f9dbbfdb0 0x560f9dbbd7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:54.794 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.792+0000 7f43c3e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560f9dbb4bd0 0x560f9dbbcd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:54.794 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.792+0000 7f43c3e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560f9dbb4bd0 0x560f9dbbcd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:54390/0 (socket says 172.21.15.83:54390) 2024-04-24T09:12:54.794 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.792+0000 7f43c3400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f9dbb54f0 0x560f9dbbd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:54.794 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.792+0000 7f43c3400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f9dbb54f0 0x560f9dbbd260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:33808/0 (socket says 172.21.15.83:33808) 2024-04-24T09:12:54.794 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.792+0000 7f43c8e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f9dbbfdb0 0x560f9dbbd7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:54.794 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.792+0000 7f43c8e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f9dbbfdb0 0x560f9dbbd7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34322/0 (socket says 172.21.15.83:34322) 2024-04-24T09:12:54.794 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.792+0000 7f43cdb99740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x560f9dbc27a0 con 0x560f9dbbfdb0 2024-04-24T09:12:54.795 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.792+0000 7f43c3e00640 1 -- 172.21.15.83:0/1215779710 learned_addr learned my addr 172.21.15.83:0/1215779710 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:12:54.795 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.792+0000 7f43cdb99740 1 -- 172.21.15.83:0/1215779710 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560f9dacc2c0 con 0x560f9dbb54f0 2024-04-24T09:12:54.795 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.792+0000 7f43cdb99740 1 -- 172.21.15.83:0/1215779710 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x560f9dbc2920 con 0x560f9dbb4bd0 2024-04-24T09:12:54.795 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.792+0000 7f43cdb99740 10 client.0 _ll_register_callbacks cb 0x560f9d824a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:12:54.795 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.793+0000 7f43c3e00640 1 -- 172.21.15.83:0/1215779710 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f9dbb54f0 msgr2=0x560f9dbbd260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:54.795 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.793+0000 7f43c3e00640 1 --2- 172.21.15.83:0/1215779710 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f9dbb54f0 0x560f9dbbd260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:54.795 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.793+0000 7f43c3e00640 1 -- 172.21.15.83:0/1215779710 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f9dbbfdb0 msgr2=0x560f9dbbd7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:54.795 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.793+0000 7f43c3e00640 1 --2- 172.21.15.83:0/1215779710 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f9dbbfdb0 0x560f9dbbd7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:54.795 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.793+0000 7f43c3e00640 1 -- 172.21.15.83:0/1215779710 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f9dadcfd0 con 0x560f9dbb4bd0 2024-04-24T09:12:54.795 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.793+0000 7f43cdb99740 1 -- 172.21.15.83:0/1215779710 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43b40c1440 con 0x560f9dbb4bd0 2024-04-24T09:12:54.795 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.793+0000 7f43c3e00640 1 --2- 172.21.15.83:0/1215779710 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560f9dbb4bd0 0x560f9dbbcd20 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f43b40bd020 tx=0x7f43b403b460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba56a56433b108e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:54.795 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.793+0000 7f43ae000640 10 client.29794 ms_handle_connect on v2:172.21.15.189:3300/0 2024-04-24T09:12:54.795 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.793+0000 7f43ae000640 1 -- 172.21.15.83:0/1215779710 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b40d6060 con 0x560f9dbb4bd0 2024-04-24T09:12:54.795 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.793+0000 7f43ae000640 1 -- 172.21.15.83:0/1215779710 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9583+0+0 (secure 0 0 0) 0x7f43b40f1170 con 0x560f9dbb4bd0 2024-04-24T09:12:54.796 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.793+0000 7f43ae000640 1 -- 172.21.15.83:0/1215779710 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f43b4049070 con 0x560f9dbb4bd0 2024-04-24T09:12:54.796 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.793+0000 7f43cdb99740 10 client.29794 fetch_fsmap learned FSMap version 64 2024-04-24T09:12:54.796 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.794+0000 7f43cdb99740 1 -- 172.21.15.83:0/1215779710 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({fsmap=64}) -- 0x560f9d825210 con 0x560f9dbb4bd0 2024-04-24T09:12:54.796 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.794+0000 7f43ae000640 1 -- 172.21.15.83:0/1215779710 <== mon.1 v2:172.21.15.189:3300/0 4 ==== fsmap(e 64) ==== 8374+0+0 (secure 0 0 0) 0x7f43b40eeac0 con 0x560f9dbb4bd0 2024-04-24T09:12:54.796 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.794+0000 7f43cdb99740 10 client.29794 fetch_fsmap finished waiting for FSMap version 64 2024-04-24T09:12:54.796 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.794+0000 7f43cdb99740 10 client.29794 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:12:54.796 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.794+0000 7f43cdb99740 20 client.29794 populate_metadata read hostname 'smithi083' 2024-04-24T09:12:54.796 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.794+0000 7f43cdb99740 1 --2- 172.21.15.83:0/1215779710 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x560f9dbcf940 0x560f9dbd1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:54.796 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.794+0000 7f43cdb99740 4 client.29794 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:12:54.796 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.794+0000 7f43cdb99740 1 -- 172.21.15.83:0/1215779710 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560f9db20320 con 0x560f9dbcf940 2024-04-24T09:12:54.796 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.795+0000 7f43c3400640 1 --2- 172.21.15.83:0/1215779710 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x560f9dbcf940 0x560f9dbd1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:54.797 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.795+0000 7f43c3400640 1 --2- 172.21.15.83:0/1215779710 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x560f9dbcf940 0x560f9dbd1d30 crc :-1 s=READY pgs=529 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63c24ae6427f070e server_cookie=bfc923eddd89feb6 in_seq=0 out_seq=0 2024-04-24T09:12:54.797 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.795+0000 7f43ae000640 10 client.29794 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:12:54.797 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.796+0000 7f43ae000640 1 -- 172.21.15.83:0/1215779710 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560f9db20320 con 0x560f9dbcf940 2024-04-24T09:12:54.797 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.796+0000 7f43ae000640 10 client.29794 handle_command_reply: tid=0 2024-04-24T09:12:54.798 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.796+0000 7f43cdb99740 1 client.29794 shutdown 2024-04-24T09:12:54.798 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.796+0000 7f43cdb99740 1 -- 172.21.15.83:0/1215779710 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560f9dbb4bd0 msgr2=0x560f9dbbcd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:54.798 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.796+0000 7f43cdb99740 1 --2- 172.21.15.83:0/1215779710 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560f9dbb4bd0 0x560f9dbbcd20 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f43b40bd020 tx=0x7f43b403b460 comp rx=0 tx=0).stop 2024-04-24T09:12:54.798 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.797+0000 7f43cdb99740 1 -- 172.21.15.83:0/1215779710 shutdown_connections 2024-04-24T09:12:54.798 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.797+0000 7f43cdb99740 1 --2- 172.21.15.83:0/1215779710 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x560f9dbcf940 0x560f9dbd1d30 crc :-1 s=READY pgs=529 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:54.798 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.797+0000 7f43cdb99740 1 --2- 172.21.15.83:0/1215779710 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f9dbbfdb0 0x560f9dbbd7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:54.798 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.797+0000 7f43cdb99740 1 --2- 172.21.15.83:0/1215779710 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f9dbb54f0 0x560f9dbbd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:54.798 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.797+0000 7f43cdb99740 1 --2- 172.21.15.83:0/1215779710 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560f9dbb4bd0 0x560f9dbbcd20 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:54.798 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.797+0000 7f43cdb99740 1 -- 172.21.15.83:0/1215779710 >> 172.21.15.83:0/1215779710 conn(0x560f9dbb10e0 msgr2=0x560f9dba7a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:54.798 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.797+0000 7f43ae000640 0 client.29794 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:12:54.798 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.797+0000 7f43cdb99740 1 -- 172.21.15.83:0/1215779710 shutdown_connections 2024-04-24T09:12:54.798 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.797+0000 7f43cdb99740 1 -- 172.21.15.83:0/1215779710 wait complete. 2024-04-24T09:12:54.798 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.797+0000 7f43cdb99740 20 client.29794 trim_cache size 0 max 16384 2024-04-24T09:12:54.821 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.821+0000 7f43cdb99740 1 Processor -- start 2024-04-24T09:12:54.821 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.822+0000 7f43cdb99740 1 -- start start 2024-04-24T09:12:54.821 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.822+0000 7f43cdb99740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f9dbb4bd0 0x560f9dbcb550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:54.821 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.822+0000 7f43cdb99740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560f9dbb54f0 0x560f9dbd7530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:54.821 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.822+0000 7f43cdb99740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f9dbbfdb0 0x560f9dbd7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:54.821 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.822+0000 7f43cdb99740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f43c41ad2d0 con 0x560f9dbb4bd0 2024-04-24T09:12:54.821 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.822+0000 7f43cdb99740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f43c41aeea0 con 0x560f9dbbfdb0 2024-04-24T09:12:54.822 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.822+0000 7f43cdb99740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f43c41ad9a0 con 0x560f9dbb54f0 2024-04-24T09:12:54.822 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.823+0000 7f43c3400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560f9dbb54f0 0x560f9dbd7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:54.822 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.823+0000 7f43c8e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f9dbbfdb0 0x560f9dbd7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:54.822 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.823+0000 7f43c3400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560f9dbb54f0 0x560f9dbd7530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:54406/0 (socket says 172.21.15.83:54406) 2024-04-24T09:12:54.822 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.823+0000 7f43c3400640 1 -- 172.21.15.83:0/2584489723 learned_addr learned my addr 172.21.15.83:0/2584489723 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:12:54.822 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.823+0000 7f43c3e00640 1 --2- 172.21.15.83:0/2584489723 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f9dbb4bd0 0x560f9dbcb550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:54.823 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.823+0000 7f43c3e00640 1 -- 172.21.15.83:0/2584489723 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560f9dbb54f0 msgr2=0x560f9dbd7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:54.823 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.823+0000 7f43c3e00640 1 --2- 172.21.15.83:0/2584489723 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560f9dbb54f0 0x560f9dbd7530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:54.823 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.823+0000 7f43c3e00640 1 -- 172.21.15.83:0/2584489723 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f9dbbfdb0 msgr2=0x560f9dbd7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:54.823 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.823+0000 7f43c3e00640 1 --2- 172.21.15.83:0/2584489723 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f9dbbfdb0 0x560f9dbd7a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:54.824 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.823+0000 7f43c3e00640 1 -- 172.21.15.83:0/2584489723 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f9d8f1620 con 0x560f9dbb4bd0 2024-04-24T09:12:54.824 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.823+0000 7f43c8e00640 1 --2- 172.21.15.83:0/2584489723 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f9dbbfdb0 0x560f9dbd7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:12:54.824 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.824+0000 7f43c3e00640 1 --2- 172.21.15.83:0/2584489723 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f9dbb4bd0 0x560f9dbcb550 secure :-1 s=READY pgs=2595 cs=0 l=1 rev1=1 crypto rx=0x7f43b4005ab0 tx=0x7f43b4045000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93cc255d3023c707 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:54.824 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.824+0000 7f43b8c00640 1 -- 172.21.15.83:0/2584489723 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b40eeac0 con 0x560f9dbb4bd0 2024-04-24T09:12:54.824 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.824+0000 7f43b8c00640 1 -- 172.21.15.83:0/2584489723 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f43b40c1440 con 0x560f9dbb4bd0 2024-04-24T09:12:54.824 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.824+0000 7f43b8c00640 1 -- 172.21.15.83:0/2584489723 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b40c5870 con 0x560f9dbb4bd0 2024-04-24T09:12:54.824 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.825+0000 7f43cdb99740 1 -- 172.21.15.83:0/2584489723 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f9dbb4bd0 msgr2=0x560f9dbcb550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:54.824 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.825+0000 7f43cdb99740 1 --2- 172.21.15.83:0/2584489723 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f9dbb4bd0 0x560f9dbcb550 secure :-1 s=READY pgs=2595 cs=0 l=1 rev1=1 crypto rx=0x7f43b4005ab0 tx=0x7f43b4045000 comp rx=0 tx=0).stop 2024-04-24T09:12:54.825 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.826+0000 7f43cdb99740 1 -- 172.21.15.83:0/2584489723 shutdown_connections 2024-04-24T09:12:54.825 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.826+0000 7f43cdb99740 1 --2- 172.21.15.83:0/2584489723 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f9dbbfdb0 0x560f9dbd7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:54.826 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.826+0000 7f43cdb99740 1 --2- 172.21.15.83:0/2584489723 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560f9dbb54f0 0x560f9dbd7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:54.826 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.826+0000 7f43cdb99740 1 --2- 172.21.15.83:0/2584489723 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f9dbb4bd0 0x560f9dbcb550 unknown :-1 s=CLOSED pgs=2595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:54.826 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.826+0000 7f43cdb99740 1 -- 172.21.15.83:0/2584489723 >> 172.21.15.83:0/2584489723 conn(0x560f9dbb10e0 msgr2=0x560f9dba7a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:54.826 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.826+0000 7f43cdb99740 1 -- 172.21.15.83:0/2584489723 shutdown_connections 2024-04-24T09:12:54.826 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.826+0000 7f43cdb99740 1 -- 172.21.15.83:0/2584489723 wait complete. 2024-04-24T09:12:54.826 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.827+0000 7f43cdb99740 1 -- start start 2024-04-24T09:12:54.826 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.827+0000 7f43cdb99740 1 Processor -- start 2024-04-24T09:12:54.828 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.828+0000 7f43cdb99740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560f9dbb4bd0 0x560f9dbb9910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:54.828 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.828+0000 7f43cdb99740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f9dbb54f0 0x560f9dbb9e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:54.828 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.828+0000 7f43cdb99740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f9dbbfdb0 0x560f9dbba390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:54.828 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.828+0000 7f43cdb99740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f43c41ae7a0 con 0x560f9dbbfdb0 2024-04-24T09:12:54.828 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.828+0000 7f43cdb99740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f43c41ae0d0 con 0x560f9dbb54f0 2024-04-24T09:12:54.828 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.828+0000 7f43cdb99740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f43c41ab2d0 con 0x560f9dbb4bd0 2024-04-24T09:12:54.828 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.828+0000 7f43cdb99740 10 client.0 _ll_register_callbacks cb 0x7f43b40bcbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:12:54.829 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.828+0000 7f43c3400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f9dbb54f0 0x560f9dbb9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:54.829 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.828+0000 7f43c3400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f9dbb54f0 0x560f9dbb9e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:33834/0 (socket says 172.21.15.83:33834) 2024-04-24T09:12:54.829 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.828+0000 7f43c3400640 1 -- 172.21.15.83:0/1123321528 learned_addr learned my addr 172.21.15.83:0/1123321528 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:12:54.829 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.829+0000 7f43c3e00640 1 --2- 172.21.15.83:0/1123321528 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560f9dbb4bd0 0x560f9dbb9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:54.829 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.829+0000 7f43c3400640 1 -- 172.21.15.83:0/1123321528 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560f9dbb4bd0 msgr2=0x560f9dbb9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:12:54.829 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.829+0000 7f43c3400640 1 --2- 172.21.15.83:0/1123321528 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560f9dbb4bd0 0x560f9dbb9910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:54.829 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.829+0000 7f43c3400640 1 -- 172.21.15.83:0/1123321528 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f9dbbfdb0 msgr2=0x560f9dbba390 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T09:12:54.829 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.829+0000 7f43c8e00640 1 --2- 172.21.15.83:0/1123321528 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f9dbbfdb0 0x560f9dbba390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:54.830 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.829+0000 7f43c3400640 1 --2- 172.21.15.83:0/1123321528 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f9dbbfdb0 0x560f9dbba390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:54.830 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.830+0000 7f43c3400640 1 -- 172.21.15.83:0/1123321528 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f9dc34da0 con 0x560f9dbb54f0 2024-04-24T09:12:54.830 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.830+0000 7f43c8e00640 1 --2- 172.21.15.83:0/1123321528 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f9dbbfdb0 0x560f9dbba390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:12:54.830 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.830+0000 7f43c3400640 1 --2- 172.21.15.83:0/1123321528 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f9dbb54f0 0x560f9dbb9e50 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x560f9dbb2270 tx=0x7f43b001e490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c10fdc57ec59ce50 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:12:54.830 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.830+0000 7f43ae000640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:12:54.830 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.831+0000 7f43ae000640 1 -- 172.21.15.83:0/1123321528 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b001b990 con 0x560f9dbb54f0 2024-04-24T09:12:54.831 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.831+0000 7f43cdb99740 1 -- 172.21.15.83:0/1123321528 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43b40c1440 con 0x560f9dbb54f0 2024-04-24T09:12:54.831 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.831+0000 7f43ae000640 1 -- 172.21.15.83:0/1123321528 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9583+0+0 (secure 0 0 0) 0x7f43b001bb30 con 0x560f9dbb54f0 2024-04-24T09:12:54.831 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.832+0000 7f43ae000640 1 -- 172.21.15.83:0/1123321528 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f43b001bd30 con 0x560f9dbb54f0 2024-04-24T09:12:54.832 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.832+0000 7f43cdb99740 10 client.39983 fetch_fsmap learned FSMap version 64 2024-04-24T09:12:54.832 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.832+0000 7f43cdb99740 1 -- 172.21.15.83:0/1123321528 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x560f9dc360a0 con 0x560f9dbb54f0 2024-04-24T09:12:54.832 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.832+0000 7f43ae000640 1 -- 172.21.15.83:0/1123321528 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f43b40c1440 con 0x560f9dbb54f0 2024-04-24T09:12:54.832 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.832+0000 7f43ae000640 1 -- 172.21.15.83:0/1123321528 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 64) ==== 8374+0+0 (secure 0 0 0) 0x7f43b001a480 con 0x560f9dbb54f0 2024-04-24T09:12:54.832 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.832+0000 7f43cdb99740 10 client.39983 fetch_fsmap finished waiting for FSMap version 64 2024-04-24T09:12:54.832 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.832+0000 7f43cdb99740 10 client.39983 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:12:54.832 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.832+0000 7f43cdb99740 20 client.39983 populate_metadata read hostname 'smithi083' 2024-04-24T09:12:54.832 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.832+0000 7f43cdb99740 1 --2- 172.21.15.83:0/1123321528 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x560f9dc367d0 0x560f9dc38bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:12:54.832 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.832+0000 7f43cdb99740 4 client.39983 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T09:12:54.832 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.832+0000 7f43cdb99740 1 -- 172.21.15.83:0/1123321528 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560f9db1a190 con 0x560f9dc367d0 2024-04-24T09:12:54.833 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.833+0000 7f43c3e00640 1 --2- 172.21.15.83:0/1123321528 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x560f9dc367d0 0x560f9dc38bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:12:54.833 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.833+0000 7f43c3e00640 1 --2- 172.21.15.83:0/1123321528 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x560f9dc367d0 0x560f9dc38bf0 crc :-1 s=READY pgs=530 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94995262697c9bf1 server_cookie=b88e1fa01346cebf in_seq=0 out_seq=0 2024-04-24T09:12:54.833 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:54.833+0000 7f43ae000640 10 client.39983 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:12:55.025 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:55.026+0000 7f43ae000640 1 -- 172.21.15.83:0/1123321528 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560f9db1a190 con 0x560f9dc367d0 2024-04-24T09:12:55.026 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:55.026+0000 7f43ae000640 10 client.39983 handle_command_reply: tid=0 2024-04-24T09:12:55.026 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:55.026+0000 7f43cdb99740 1 client.39983 shutdown 2024-04-24T09:12:55.026 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:55.027+0000 7f43cdb99740 1 -- 172.21.15.83:0/1123321528 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f9dbb54f0 msgr2=0x560f9dbb9e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:55.026 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:55.027+0000 7f43cdb99740 1 --2- 172.21.15.83:0/1123321528 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f9dbb54f0 0x560f9dbb9e50 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x560f9dbb2270 tx=0x7f43b001e490 comp rx=0 tx=0).stop 2024-04-24T09:12:55.027 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:55.028+0000 7f43cdb99740 1 -- 172.21.15.83:0/1123321528 shutdown_connections 2024-04-24T09:12:55.027 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:55.028+0000 7f43cdb99740 1 --2- 172.21.15.83:0/1123321528 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x560f9dc367d0 0x560f9dc38bf0 crc :-1 s=READY pgs=530 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:55.027 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:55.028+0000 7f43cdb99740 1 --2- 172.21.15.83:0/1123321528 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f9dbbfdb0 0x560f9dbba390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:55.027 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:55.028+0000 7f43cdb99740 1 --2- 172.21.15.83:0/1123321528 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f9dbb54f0 0x560f9dbb9e50 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:55.027 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:55.028+0000 7f43cdb99740 1 --2- 172.21.15.83:0/1123321528 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560f9dbb4bd0 0x560f9dbb9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:55.027 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:55.028+0000 7f43cdb99740 1 -- 172.21.15.83:0/1123321528 >> 172.21.15.83:0/1123321528 conn(0x560f9dbb10e0 msgr2=0x560f9dbb6580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:55.028 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:55.028+0000 7f43ae000640 0 client.39983 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:12:55.028 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:55.029+0000 7f43cdb99740 1 -- 172.21.15.83:0/1123321528 shutdown_connections 2024-04-24T09:12:55.030 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:55.031+0000 7f43cdb99740 1 -- 172.21.15.83:0/1123321528 wait complete. 2024-04-24T09:12:55.030 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:55.031+0000 7f43cdb99740 20 client.39983 trim_cache size 0 max 16384 2024-04-24T09:12:55.030 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T09:12:55.030 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T09:12:55.030 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T09:12:55.031 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T09:12:55.032 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:55.032+0000 7f43c9800640 1 -- 172.21.15.83:0/3158488779 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f43a80a3d60 msgr2=0x7f43a80a6220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:55.032 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:55.032+0000 7f43c9800640 1 --2- 172.21.15.83:0/3158488779 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f43a80a3d60 0x7f43a80a6220 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f43bc009eb0 tx=0x7f43bc009540 comp rx=0 tx=0).stop 2024-04-24T09:12:55.032 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:55.032+0000 7f43c9800640 1 -- 172.21.15.83:0/3158488779 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43c4102070 msgr2=0x7f43c41964d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:12:55.032 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:55.032+0000 7f43c9800640 1 --2- 172.21.15.83:0/3158488779 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43c4102070 0x7f43c41964d0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f43b4031440 tx=0x7f43b4003cd0 comp rx=0 tx=0).stop 2024-04-24T09:12:55.032 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:55.032+0000 7f43c9800640 1 -- 172.21.15.83:0/3158488779 shutdown_connections 2024-04-24T09:12:55.032 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:55.032+0000 7f43c9800640 1 --2- 172.21.15.83:0/3158488779 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f43a80a3d60 0x7f43a80a6220 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:55.032 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:55.032+0000 7f43c9800640 1 --2- 172.21.15.83:0/3158488779 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f43c419ad90 0x7f43c419b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:55.032 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:55.032+0000 7f43c9800640 1 --2- 172.21.15.83:0/3158488779 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43c4107140 0x7f43c4196a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:55.032 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:55.032+0000 7f43c9800640 1 --2- 172.21.15.83:0/3158488779 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43c4102070 0x7f43c41964d0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:12:55.032 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:55.032+0000 7f43c9800640 1 -- 172.21.15.83:0/3158488779 >> 172.21.15.83:0/3158488779 conn(0x7f43c40fd950 msgr2=0x7f43c40fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:12:55.032 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:55.032+0000 7f43c9800640 1 -- 172.21.15.83:0/3158488779 shutdown_connections 2024-04-24T09:12:55.033 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:12:55.032+0000 7f43c9800640 1 -- 172.21.15.83:0/3158488779 wait complete. 2024-04-24T09:12:55.190 INFO:tasks.workunit.client.0.smithi083.stdout: 10 55708 6.68 MB/sec execute 326 sec latency 201.961 ms 2024-04-24T09:12:56.191 INFO:tasks.workunit.client.0.smithi083.stdout: 10 55807 6.67 MB/sec execute 327 sec latency 69.056 ms 2024-04-24T09:12:56.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:12:56 smithi138 ceph-mon[31441]: pgmap v672: 129 pgs: 129 active+clean; 876 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.2 MiB/s wr, 221 op/s 2024-04-24T09:12:56.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:12:56 smithi083 ceph-mon[27907]: pgmap v672: 129 pgs: 129 active+clean; 876 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.2 MiB/s wr, 221 op/s 2024-04-24T09:12:56.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:12:56 smithi189 ceph-mon[31791]: pgmap v672: 129 pgs: 129 active+clean; 876 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.2 MiB/s wr, 221 op/s 2024-04-24T09:12:57.190 INFO:tasks.workunit.client.0.smithi083.stdout: 10 55928 6.68 MB/sec execute 328 sec latency 143.392 ms 2024-04-24T09:12:58.190 INFO:tasks.workunit.client.0.smithi083.stdout: 10 56044 6.67 MB/sec execute 329 sec latency 69.352 ms 2024-04-24T09:12:58.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:12:58 smithi083 ceph-mon[27907]: pgmap v673: 129 pgs: 129 active+clean; 876 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.2 MiB/s wr, 221 op/s 2024-04-24T09:12:58.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:12:58 smithi138 ceph-mon[31441]: pgmap v673: 129 pgs: 129 active+clean; 876 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.2 MiB/s wr, 221 op/s 2024-04-24T09:12:58.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:12:58 smithi189 ceph-mon[31791]: pgmap v673: 129 pgs: 129 active+clean; 876 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.2 MiB/s wr, 221 op/s 2024-04-24T09:12:59.190 INFO:tasks.workunit.client.0.smithi083.stdout: 10 56163 6.67 MB/sec execute 330 sec latency 68.111 ms 2024-04-24T09:13:00.190 INFO:tasks.workunit.client.0.smithi083.stdout: 10 56280 6.67 MB/sec execute 331 sec latency 74.301 ms 2024-04-24T09:13:00.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:13:00 smithi083 ceph-mon[27907]: pgmap v674: 129 pgs: 129 active+clean; 880 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.2 MiB/s wr, 359 op/s 2024-04-24T09:13:00.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:13:00 smithi138 ceph-mon[31441]: pgmap v674: 129 pgs: 129 active+clean; 880 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.2 MiB/s wr, 359 op/s 2024-04-24T09:13:00.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:13:00 smithi189 ceph-mon[31791]: pgmap v674: 129 pgs: 129 active+clean; 880 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.2 MiB/s wr, 359 op/s 2024-04-24T09:13:01.190 INFO:tasks.workunit.client.0.smithi083.stdout: 10 56391 6.67 MB/sec execute 332 sec latency 87.474 ms 2024-04-24T09:13:02.191 INFO:tasks.workunit.client.0.smithi083.stdout: 10 56515 6.66 MB/sec execute 333 sec latency 74.268 ms 2024-04-24T09:13:02.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:13:02 smithi083 ceph-mon[27907]: pgmap v675: 129 pgs: 129 active+clean; 880 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.3 MiB/s wr, 223 op/s 2024-04-24T09:13:02.464 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:13:02 smithi138 ceph-mon[31441]: pgmap v675: 129 pgs: 129 active+clean; 880 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.3 MiB/s wr, 223 op/s 2024-04-24T09:13:02.464 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:13:02 smithi189 ceph-mon[31791]: pgmap v675: 129 pgs: 129 active+clean; 880 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.3 MiB/s wr, 223 op/s 2024-04-24T09:13:03.191 INFO:tasks.workunit.client.0.smithi083.stdout: 10 56622 6.64 MB/sec execute 334 sec latency 96.928 ms 2024-04-24T09:13:03.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:13:03 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:13:03.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:13:03 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:13:03.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:13:03 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:13:03.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:13:03 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:13:03.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:13:03 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:13:03.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:13:03 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:13:03.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:13:03 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:13:03.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:13:03 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:13:03.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:13:03 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:13:03.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:13:03 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:13:03.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:13:03 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:13:03.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:13:03 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:13:04.191 INFO:tasks.workunit.client.0.smithi083.stdout: 10 56730 6.65 MB/sec execute 335 sec latency 152.688 ms 2024-04-24T09:13:04.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:13:04 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:13:04.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:13:04 smithi138 ceph-mon[31441]: pgmap v676: 129 pgs: 129 active+clean; 894 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.3 MiB/s wr, 348 op/s 2024-04-24T09:13:04.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:13:04 smithi138 ceph-mon[31441]: osdmap e125: 12 total, 12 up, 12 in 2024-04-24T09:13:04.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:13:04 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:13:04.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:13:04 smithi189 ceph-mon[31791]: pgmap v676: 129 pgs: 129 active+clean; 894 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.3 MiB/s wr, 348 op/s 2024-04-24T09:13:04.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:13:04 smithi189 ceph-mon[31791]: osdmap e125: 12 total, 12 up, 12 in 2024-04-24T09:13:04.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:13:04 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:13:04.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:13:04 smithi083 ceph-mon[27907]: pgmap v676: 129 pgs: 129 active+clean; 894 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.3 MiB/s wr, 348 op/s 2024-04-24T09:13:04.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:13:04 smithi083 ceph-mon[27907]: osdmap e125: 12 total, 12 up, 12 in 2024-04-24T09:13:05.192 INFO:tasks.workunit.client.0.smithi083.stdout: 10 56831 6.63 MB/sec execute 336 sec latency 87.561 ms 2024-04-24T09:13:06.192 INFO:tasks.workunit.client.0.smithi083.stdout: 10 56942 6.63 MB/sec execute 337 sec latency 152.133 ms 2024-04-24T09:13:06.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:13:06 smithi138 ceph-mon[31441]: pgmap v678: 129 pgs: 129 active+clean; 894 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.8 MiB/s wr, 314 op/s 2024-04-24T09:13:06.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:13:06 smithi189 ceph-mon[31791]: pgmap v678: 129 pgs: 129 active+clean; 894 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.8 MiB/s wr, 314 op/s 2024-04-24T09:13:06.552 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:13:06 smithi083 ceph-mon[27907]: pgmap v678: 129 pgs: 129 active+clean; 894 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.8 MiB/s wr, 314 op/s 2024-04-24T09:13:07.192 INFO:tasks.workunit.client.0.smithi083.stdout: 10 57058 6.62 MB/sec execute 338 sec latency 71.620 ms 2024-04-24T09:13:08.196 INFO:tasks.workunit.client.0.smithi083.stdout: 10 57164 6.61 MB/sec execute 339 sec latency 69.221 ms 2024-04-24T09:13:08.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:13:08 smithi138 ceph-mon[31441]: pgmap v679: 129 pgs: 129 active+clean; 894 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.8 MiB/s wr, 314 op/s 2024-04-24T09:13:08.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:13:08 smithi189 ceph-mon[31791]: pgmap v679: 129 pgs: 129 active+clean; 894 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.8 MiB/s wr, 314 op/s 2024-04-24T09:13:08.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:13:08 smithi083 ceph-mon[27907]: pgmap v679: 129 pgs: 129 active+clean; 894 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.8 MiB/s wr, 314 op/s 2024-04-24T09:13:08.988 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:08.988+0000 7fa103c00640 1 -- 172.21.15.83:0/255092909 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa0fc10c700 msgr2=0x7fa0fc109000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:08.988 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:08.988+0000 7fa103c00640 1 --2- 172.21.15.83:0/255092909 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa0fc10c700 0x7fa0fc109000 secure :-1 s=READY pgs=2596 cs=0 l=1 rev1=1 crypto rx=0x7fa0f4009ca0 tx=0x7fa0f402a8c0 comp rx=0 tx=0).stop 2024-04-24T09:13:08.988 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:08.988+0000 7fa103c00640 1 -- 172.21.15.83:0/255092909 shutdown_connections 2024-04-24T09:13:08.989 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:08.988+0000 7fa103c00640 1 --2- 172.21.15.83:0/255092909 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa0fc117090 0x7fa0fc119480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:08.989 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:08.988+0000 7fa103c00640 1 --2- 172.21.15.83:0/255092909 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa0fc109540 0x7fa0fc1099a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:08.989 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:08.988+0000 7fa103c00640 1 --2- 172.21.15.83:0/255092909 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa0fc10c700 0x7fa0fc109000 unknown :-1 s=CLOSED pgs=2596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:08.989 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:08.988+0000 7fa103c00640 1 -- 172.21.15.83:0/255092909 >> 172.21.15.83:0/255092909 conn(0x7fa0fc06d0d0 msgr2=0x7fa0fc06d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:08.989 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:08.988+0000 7fa103c00640 1 -- 172.21.15.83:0/255092909 shutdown_connections 2024-04-24T09:13:08.989 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:08.988+0000 7fa103c00640 1 -- 172.21.15.83:0/255092909 wait complete. 2024-04-24T09:13:08.989 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:08.989+0000 7fa103c00640 1 Processor -- start 2024-04-24T09:13:08.990 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:08.989+0000 7fa103c00640 1 -- start start 2024-04-24T09:13:08.990 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:08.990+0000 7fa103c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa0fc109540 0x7fa0fc1a32c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:08.990 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:08.990+0000 7fa103c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa0fc10c700 0x7fa0fc1a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:08.990 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:08.990+0000 7fa103c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa0fc117090 0x7fa0fc1a7b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:08.990 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:08.990+0000 7fa103c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa0fc11cd40 con 0x7fa0fc10c700 2024-04-24T09:13:08.990 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:08.990+0000 7fa103c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa0fc11cbc0 con 0x7fa0fc117090 2024-04-24T09:13:08.991 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:08.990+0000 7fa103c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa0fc11cec0 con 0x7fa0fc109540 2024-04-24T09:13:08.991 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:08.990+0000 7fa0fbe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa0fc10c700 0x7fa0fc1a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:08.992 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:08.990+0000 7fa0fbe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa0fc10c700 0x7fa0fc1a3800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40396/0 (socket says 172.21.15.83:40396) 2024-04-24T09:13:08.992 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:08.990+0000 7fa0fbe00640 1 -- 172.21.15.83:0/2445039141 learned_addr learned my addr 172.21.15.83:0/2445039141 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:13:08.992 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:08.991+0000 7fa0fbe00640 1 -- 172.21.15.83:0/2445039141 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa0fc109540 msgr2=0x7fa0fc1a32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:08.992 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:08.991+0000 7fa0fbe00640 1 --2- 172.21.15.83:0/2445039141 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa0fc109540 0x7fa0fc1a32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:08.992 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:08.991+0000 7fa0fbe00640 1 -- 172.21.15.83:0/2445039141 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa0fc117090 msgr2=0x7fa0fc1a7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:08.992 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:08.991+0000 7fa0fbe00640 1 --2- 172.21.15.83:0/2445039141 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa0fc117090 0x7fa0fc1a7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:08.992 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:08.991+0000 7fa0fbe00640 1 -- 172.21.15.83:0/2445039141 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0fc1a82b0 con 0x7fa0fc10c700 2024-04-24T09:13:08.992 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:08.991+0000 7fa0fbe00640 1 --2- 172.21.15.83:0/2445039141 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa0fc10c700 0x7fa0fc1a3800 secure :-1 s=READY pgs=2597 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec00b4a0 tx=0x7fa0ec00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:08.993 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:08.992+0000 7fa0f9600640 1 -- 172.21.15.83:0/2445039141 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0ec014070 con 0x7fa0fc10c700 2024-04-24T09:13:08.993 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:08.992+0000 7fa0f9600640 1 -- 172.21.15.83:0/2445039141 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa0ec010ac0 con 0x7fa0fc10c700 2024-04-24T09:13:08.993 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:08.992+0000 7fa0f9600640 1 -- 172.21.15.83:0/2445039141 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0ec017c30 con 0x7fa0fc10c700 2024-04-24T09:13:08.993 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:08.992+0000 7fa103c00640 1 -- 172.21.15.83:0/2445039141 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0fc1a85a0 con 0x7fa0fc10c700 2024-04-24T09:13:08.993 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:08.992+0000 7fa103c00640 1 -- 172.21.15.83:0/2445039141 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0fc1afe30 con 0x7fa0fc10c700 2024-04-24T09:13:08.996 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:08.996+0000 7fa0f9600640 1 -- 172.21.15.83:0/2445039141 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa0ec017070 con 0x7fa0fc10c700 2024-04-24T09:13:08.997 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:08.997+0000 7fa106970740 1 Processor -- start 2024-04-24T09:13:08.997 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:08.997+0000 7fa106970740 1 -- start start 2024-04-24T09:13:08.998 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:08.997+0000 7fa106970740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5603ee7551b0 0x5603ee755590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:08.998 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:08.997+0000 7fa106970740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5603ee755ad0 0x5603ee75fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:08.998 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:08.997+0000 7fa106970740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5603ee760390 0x5603ee762780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:08.998 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:08.998+0000 7fa106970740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5603ee6bb900 con 0x5603ee7551b0 2024-04-24T09:13:08.998 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:08.998+0000 7fa106970740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5603ee661f20 con 0x5603ee760390 2024-04-24T09:13:08.998 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:08.998+0000 7fa106970740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5603ee6b78c0 con 0x5603ee755ad0 2024-04-24T09:13:08.999 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:08.998+0000 7fa101c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5603ee7551b0 0x5603ee755590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:08.999 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:08.998+0000 7fa101c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5603ee7551b0 0x5603ee755590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40400/0 (socket says 172.21.15.83:40400) 2024-04-24T09:13:08.999 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:08.998+0000 7fa101c00640 1 -- 172.21.15.83:0/2870888087 learned_addr learned my addr 172.21.15.83:0/2870888087 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:13:09.000 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:08.998+0000 7fa0f9600640 1 --2- 172.21.15.83:0/2445039141 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa0e00a3e40 0x7fa0e00a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:09.000 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:08.998+0000 7fa101200640 1 --2- 172.21.15.83:0/2870888087 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5603ee755ad0 0x5603ee75fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:09.000 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:08.998+0000 7fa0f9600640 1 -- 172.21.15.83:0/2445039141 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 9671+0+0 (secure 0 0 0) 0x7fa0ec0d0300 con 0x7fa0fc10c700 2024-04-24T09:13:09.000 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:08.998+0000 7fa0fbe00640 1 --2- 172.21.15.83:0/2870888087 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5603ee760390 0x5603ee762780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:09.000 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:08.999+0000 7fa101c00640 1 -- 172.21.15.83:0/2870888087 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5603ee755ad0 msgr2=0x5603ee75fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:09.000 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:08.999+0000 7fa101c00640 1 --2- 172.21.15.83:0/2870888087 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5603ee755ad0 0x5603ee75fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:09.000 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:08.999+0000 7fa101c00640 1 -- 172.21.15.83:0/2870888087 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5603ee760390 msgr2=0x5603ee762780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:09.000 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:08.999+0000 7fa101c00640 1 --2- 172.21.15.83:0/2870888087 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5603ee760390 0x5603ee762780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:09.000 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:08.999+0000 7fa101c00640 1 -- 172.21.15.83:0/2870888087 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603ee686e10 con 0x5603ee7551b0 2024-04-24T09:13:09.000 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:08.999+0000 7fa0fbe00640 1 --2- 172.21.15.83:0/2870888087 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5603ee760390 0x5603ee762780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:13:09.001 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.000+0000 7fa101c00640 1 --2- 172.21.15.83:0/2870888087 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5603ee7551b0 0x5603ee755590 secure :-1 s=READY pgs=2598 cs=0 l=1 rev1=1 crypto rx=0x7fa0e8009be0 tx=0x7fa0e800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8216c39b40eb71 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:09.001 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.000+0000 7fa101c00640 1 --2- 172.21.15.83:0/2445039141 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa0e00a3e40 0x7fa0e00a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:09.001 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.001+0000 7fa101c00640 1 --2- 172.21.15.83:0/2445039141 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa0e00a3e40 0x7fa0e00a6300 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fa0e800ec20 tx=0x7fa0e8014000 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:09.001 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.001+0000 7fa0f2000640 1 -- 172.21.15.83:0/2870888087 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e801a640 con 0x5603ee7551b0 2024-04-24T09:13:09.001 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.001+0000 7fa0f2000640 1 -- 172.21.15.83:0/2870888087 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa0e8009cd0 con 0x5603ee7551b0 2024-04-24T09:13:09.001 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.001+0000 7fa0f2000640 1 -- 172.21.15.83:0/2870888087 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e80249a0 con 0x5603ee7551b0 2024-04-24T09:13:09.002 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.002+0000 7fa106970740 1 -- 172.21.15.83:0/2870888087 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5603ee7551b0 msgr2=0x5603ee755590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:09.003 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.002+0000 7fa106970740 1 --2- 172.21.15.83:0/2870888087 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5603ee7551b0 0x5603ee755590 secure :-1 s=READY pgs=2598 cs=0 l=1 rev1=1 crypto rx=0x7fa0e8009be0 tx=0x7fa0e800e720 comp rx=0 tx=0).stop 2024-04-24T09:13:09.003 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.002+0000 7fa106970740 1 -- 172.21.15.83:0/2870888087 shutdown_connections 2024-04-24T09:13:09.003 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.002+0000 7fa106970740 1 --2- 172.21.15.83:0/2870888087 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5603ee760390 0x5603ee762780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:09.003 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.002+0000 7fa106970740 1 --2- 172.21.15.83:0/2870888087 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5603ee755ad0 0x5603ee75fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:09.003 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.002+0000 7fa106970740 1 --2- 172.21.15.83:0/2870888087 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5603ee7551b0 0x5603ee755590 unknown :-1 s=CLOSED pgs=2598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:09.003 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.002+0000 7fa106970740 1 -- 172.21.15.83:0/2870888087 >> 172.21.15.83:0/2870888087 conn(0x5603ee7516c0 msgr2=0x5603ee751aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:09.003 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.003+0000 7fa106970740 1 -- 172.21.15.83:0/2870888087 shutdown_connections 2024-04-24T09:13:09.003 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.003+0000 7fa106970740 1 -- 172.21.15.83:0/2870888087 wait complete. 2024-04-24T09:13:09.004 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.003+0000 7fa106970740 1 -- start start 2024-04-24T09:13:09.004 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.003+0000 7fa106970740 1 Processor -- start 2024-04-24T09:13:09.005 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.004+0000 7fa106970740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5603ee7551b0 0x5603ee75d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:09.005 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.004+0000 7fa106970740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5603ee755ad0 0x5603ee75d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:09.005 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.004+0000 7fa106970740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5603ee760390 0x5603ee75ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:09.005 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.004+0000 7fa106970740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5603ee762fc0 con 0x5603ee760390 2024-04-24T09:13:09.005 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.004+0000 7fa106970740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5603ee6752c0 con 0x5603ee7551b0 2024-04-24T09:13:09.005 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.004+0000 7fa106970740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5603ee763140 con 0x5603ee755ad0 2024-04-24T09:13:09.005 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.004+0000 7fa106970740 10 client.0 _ll_register_callbacks cb 0x5603ee3cda00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:13:09.005 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.004+0000 7fa101200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5603ee7551b0 0x5603ee75d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:09.005 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.004+0000 7fa101c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5603ee760390 0x5603ee75ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:09.006 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.005+0000 7fa101200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5603ee7551b0 0x5603ee75d330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:56984/0 (socket says 172.21.15.83:56984) 2024-04-24T09:13:09.006 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.005+0000 7fa101200640 1 -- 172.21.15.83:0/971321763 learned_addr learned my addr 172.21.15.83:0/971321763 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:13:09.006 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.005+0000 7fa0fbe00640 1 --2- 172.21.15.83:0/971321763 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5603ee755ad0 0x5603ee75d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:09.006 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.005+0000 7fa101c00640 1 -- 172.21.15.83:0/971321763 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5603ee755ad0 msgr2=0x5603ee75d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:09.007 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.005+0000 7fa101c00640 1 --2- 172.21.15.83:0/971321763 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5603ee755ad0 0x5603ee75d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:09.007 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.005+0000 7fa101c00640 1 -- 172.21.15.83:0/971321763 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5603ee7551b0 msgr2=0x5603ee75d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:09.007 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.005+0000 7fa101c00640 1 --2- 172.21.15.83:0/971321763 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5603ee7551b0 0x5603ee75d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:09.007 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.005+0000 7fa101c00640 1 -- 172.21.15.83:0/971321763 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603ee685fd0 con 0x5603ee760390 2024-04-24T09:13:09.007 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.006+0000 7fa101c00640 1 --2- 172.21.15.83:0/971321763 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5603ee760390 0x5603ee75ddb0 secure :-1 s=READY pgs=2599 cs=0 l=1 rev1=1 crypto rx=0x7fa0e8009e80 tx=0x7fa0e8009910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37e9b62841c61ddc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:09.007 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.006+0000 7fa0e6a00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:13:09.007 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.006+0000 7fa0e6a00640 1 -- 172.21.15.83:0/971321763 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e8004450 con 0x5603ee760390 2024-04-24T09:13:09.007 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.006+0000 7fa0e6a00640 1 -- 172.21.15.83:0/971321763 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9671+0+0 (secure 0 0 0) 0x7fa0e80246c0 con 0x5603ee760390 2024-04-24T09:13:09.007 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.007+0000 7fa106970740 1 -- 172.21.15.83:0/971321763 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0e8009cd0 con 0x5603ee760390 2024-04-24T09:13:09.008 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.007+0000 7fa0e6a00640 1 -- 172.21.15.83:0/971321763 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7fa0e80248c0 con 0x5603ee760390 2024-04-24T09:13:09.008 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.007+0000 7fa106970740 10 client.22941 fetch_fsmap learned FSMap version 64 2024-04-24T09:13:09.008 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.007+0000 7fa106970740 1 -- 172.21.15.83:0/971321763 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x5603ee3ce210 con 0x5603ee760390 2024-04-24T09:13:09.008 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.007+0000 7fa0e6a00640 1 -- 172.21.15.83:0/971321763 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa0e8009cd0 con 0x5603ee760390 2024-04-24T09:13:09.009 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.008+0000 7fa0e6a00640 1 -- 172.21.15.83:0/971321763 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 64) ==== 8374+0+0 (secure 0 0 0) 0x7fa0e8036030 con 0x5603ee760390 2024-04-24T09:13:09.009 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.009+0000 7fa106970740 10 client.22941 fetch_fsmap finished waiting for FSMap version 64 2024-04-24T09:13:09.009 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.009+0000 7fa106970740 10 client.22941 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:13:09.009 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.009+0000 7fa106970740 20 client.22941 populate_metadata read hostname 'smithi083' 2024-04-24T09:13:09.009 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.009+0000 7fa106970740 1 --2- 172.21.15.83:0/971321763 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5603ee770490 0x5603ee772880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:09.010 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.009+0000 7fa106970740 4 client.22941 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:13:09.010 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.009+0000 7fa106970740 1 -- 172.21.15.83:0/971321763 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5603ee6c9320 con 0x5603ee770490 2024-04-24T09:13:09.010 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.009+0000 7fa101200640 1 --2- 172.21.15.83:0/971321763 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5603ee770490 0x5603ee772880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:09.011 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.011+0000 7fa101200640 1 --2- 172.21.15.83:0/971321763 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5603ee770490 0x5603ee772880 crc :-1 s=READY pgs=531 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae79d68147ff0618 server_cookie=11817d15d270bdf8 in_seq=0 out_seq=0 2024-04-24T09:13:09.011 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.011+0000 7fa0e6a00640 10 client.22941 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:13:09.012 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.012+0000 7fa0e6a00640 1 -- 172.21.15.83:0/971321763 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5603ee6c9320 con 0x5603ee770490 2024-04-24T09:13:09.012 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.012+0000 7fa0e6a00640 10 client.22941 handle_command_reply: tid=0 2024-04-24T09:13:09.012 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.012+0000 7fa106970740 1 client.22941 shutdown 2024-04-24T09:13:09.013 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.013+0000 7fa106970740 1 -- 172.21.15.83:0/971321763 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5603ee760390 msgr2=0x5603ee75ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:09.013 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.013+0000 7fa106970740 1 --2- 172.21.15.83:0/971321763 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5603ee760390 0x5603ee75ddb0 secure :-1 s=READY pgs=2599 cs=0 l=1 rev1=1 crypto rx=0x7fa0e8009e80 tx=0x7fa0e8009910 comp rx=0 tx=0).stop 2024-04-24T09:13:09.013 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.013+0000 7fa106970740 1 -- 172.21.15.83:0/971321763 shutdown_connections 2024-04-24T09:13:09.013 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.013+0000 7fa106970740 1 --2- 172.21.15.83:0/971321763 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5603ee770490 0x5603ee772880 crc :-1 s=READY pgs=531 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:09.013 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.013+0000 7fa106970740 1 --2- 172.21.15.83:0/971321763 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5603ee760390 0x5603ee75ddb0 unknown :-1 s=CLOSED pgs=2599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:09.013 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.013+0000 7fa0e6a00640 0 client.22941 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:13:09.014 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.013+0000 7fa106970740 1 --2- 172.21.15.83:0/971321763 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5603ee755ad0 0x5603ee75d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:09.014 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.013+0000 7fa106970740 1 --2- 172.21.15.83:0/971321763 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5603ee7551b0 0x5603ee75d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:09.014 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.013+0000 7fa106970740 1 -- 172.21.15.83:0/971321763 >> 172.21.15.83:0/971321763 conn(0x5603ee7516c0 msgr2=0x5603ee7573f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:09.014 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.014+0000 7fa106970740 1 -- 172.21.15.83:0/971321763 shutdown_connections 2024-04-24T09:13:09.014 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.014+0000 7fa106970740 1 -- 172.21.15.83:0/971321763 wait complete. 2024-04-24T09:13:09.015 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.015+0000 7fa106970740 20 client.22941 trim_cache size 0 max 16384 2024-04-24T09:13:09.051 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.051+0000 7fa106970740 1 Processor -- start 2024-04-24T09:13:09.051 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.051+0000 7fa106970740 1 -- start start 2024-04-24T09:13:09.052 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.052+0000 7fa106970740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5603ee7551b0 0x5603ee76be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:09.053 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.053+0000 7fa106970740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5603ee755ad0 0x5603ee75f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:09.053 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.053+0000 7fa106970740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5603ee760390 0x5603ee777eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:09.053 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.053+0000 7fa106970740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa0fc1b5ed0 con 0x5603ee7551b0 2024-04-24T09:13:09.053 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.053+0000 7fa106970740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa0fc1b7aa0 con 0x5603ee760390 2024-04-24T09:13:09.054 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.053+0000 7fa106970740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa0fc1b65a0 con 0x5603ee755ad0 2024-04-24T09:13:09.054 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.053+0000 7fa101200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5603ee7551b0 0x5603ee76be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:09.055 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.053+0000 7fa101200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5603ee7551b0 0x5603ee76be80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40410/0 (socket says 172.21.15.83:40410) 2024-04-24T09:13:09.056 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.054+0000 7fa0fbe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5603ee755ad0 0x5603ee75f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:09.056 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.054+0000 7fa0fbe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5603ee755ad0 0x5603ee75f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:52860/0 (socket says 172.21.15.83:52860) 2024-04-24T09:13:09.056 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.054+0000 7fa0fbe00640 1 -- 172.21.15.83:0/280481740 learned_addr learned my addr 172.21.15.83:0/280481740 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:13:09.056 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.055+0000 7fa0fbe00640 1 -- 172.21.15.83:0/280481740 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5603ee760390 msgr2=0x5603ee777eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:09.056 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.055+0000 7fa0fbe00640 1 --2- 172.21.15.83:0/280481740 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5603ee760390 0x5603ee777eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:09.056 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.055+0000 7fa0fbe00640 1 -- 172.21.15.83:0/280481740 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5603ee7551b0 msgr2=0x5603ee76be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:09.056 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.055+0000 7fa0fbe00640 1 --2- 172.21.15.83:0/280481740 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5603ee7551b0 0x5603ee76be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:09.056 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.055+0000 7fa0fbe00640 1 -- 172.21.15.83:0/280481740 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603ee49a620 con 0x5603ee755ad0 2024-04-24T09:13:09.056 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.055+0000 7fa101200640 1 --2- 172.21.15.83:0/280481740 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5603ee7551b0 0x5603ee76be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:13:09.057 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.056+0000 7fa0fbe00640 1 --2- 172.21.15.83:0/280481740 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5603ee755ad0 0x5603ee75f270 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec0cd6e0 tx=0x7fa0ec0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=551ac26acb3a51d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:09.057 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.056+0000 7fa0f1600640 1 -- 172.21.15.83:0/280481740 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0ec01e040 con 0x5603ee755ad0 2024-04-24T09:13:09.057 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.056+0000 7fa0f1600640 1 -- 172.21.15.83:0/280481740 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa0ec0a4b40 con 0x5603ee755ad0 2024-04-24T09:13:09.057 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.057+0000 7fa0f1600640 1 -- 172.21.15.83:0/280481740 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0ec0986b0 con 0x5603ee755ad0 2024-04-24T09:13:09.058 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.058+0000 7fa106970740 1 -- 172.21.15.83:0/280481740 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5603ee755ad0 msgr2=0x5603ee75f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:09.058 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.058+0000 7fa106970740 1 --2- 172.21.15.83:0/280481740 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5603ee755ad0 0x5603ee75f270 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec0cd6e0 tx=0x7fa0ec0046c0 comp rx=0 tx=0).stop 2024-04-24T09:13:09.059 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.058+0000 7fa106970740 1 -- 172.21.15.83:0/280481740 shutdown_connections 2024-04-24T09:13:09.059 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.058+0000 7fa106970740 1 --2- 172.21.15.83:0/280481740 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5603ee760390 0x5603ee777eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:09.059 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.058+0000 7fa106970740 1 --2- 172.21.15.83:0/280481740 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5603ee755ad0 0x5603ee75f270 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:09.059 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.058+0000 7fa106970740 1 --2- 172.21.15.83:0/280481740 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5603ee7551b0 0x5603ee76be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:09.059 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.058+0000 7fa106970740 1 -- 172.21.15.83:0/280481740 >> 172.21.15.83:0/280481740 conn(0x5603ee7516c0 msgr2=0x5603ee75eb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:09.059 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.059+0000 7fa106970740 1 -- 172.21.15.83:0/280481740 shutdown_connections 2024-04-24T09:13:09.060 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.059+0000 7fa106970740 1 -- 172.21.15.83:0/280481740 wait complete. 2024-04-24T09:13:09.060 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.059+0000 7fa106970740 1 -- start start 2024-04-24T09:13:09.060 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.059+0000 7fa106970740 1 Processor -- start 2024-04-24T09:13:09.060 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.060+0000 7fa106970740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5603ee7551b0 0x5603ee7687f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:09.060 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.060+0000 7fa106970740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5603ee755ad0 0x5603ee763b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:09.060 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.060+0000 7fa106970740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5603ee760390 0x5603ee7640e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:09.061 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.061+0000 7fa0fbe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5603ee755ad0 0x5603ee763b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:09.061 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.061+0000 7fa0fbe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5603ee755ad0 0x5603ee763b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40414/0 (socket says 172.21.15.83:40414) 2024-04-24T09:13:09.061 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.061+0000 7fa0fbe00640 1 -- 172.21.15.83:0/3649979926 learned_addr learned my addr 172.21.15.83:0/3649979926 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:13:09.061 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.061+0000 7fa101200640 1 --2- 172.21.15.83:0/3649979926 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5603ee7551b0 0x5603ee7687f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:09.062 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.061+0000 7fa106970740 1 -- 172.21.15.83:0/3649979926 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa0fc1b73a0 con 0x5603ee755ad0 2024-04-24T09:13:09.062 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.061+0000 7fa106970740 1 -- 172.21.15.83:0/3649979926 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa0fc1b6cd0 con 0x5603ee7551b0 2024-04-24T09:13:09.062 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.061+0000 7fa106970740 1 -- 172.21.15.83:0/3649979926 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa0fc1b3ef0 con 0x5603ee760390 2024-04-24T09:13:09.062 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.061+0000 7fa106970740 10 client.0 _ll_register_callbacks cb 0x7fa0e8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:13:09.062 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.061+0000 7fa101c00640 1 --2- 172.21.15.83:0/3649979926 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5603ee760390 0x5603ee7640e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:09.062 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.062+0000 7fa101c00640 1 -- 172.21.15.83:0/3649979926 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5603ee7551b0 msgr2=0x5603ee7687f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:09.062 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.062+0000 7fa101c00640 1 --2- 172.21.15.83:0/3649979926 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5603ee7551b0 0x5603ee7687f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:09.063 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.062+0000 7fa101c00640 1 -- 172.21.15.83:0/3649979926 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5603ee755ad0 msgr2=0x5603ee763b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:09.063 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.062+0000 7fa101c00640 1 --2- 172.21.15.83:0/3649979926 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5603ee755ad0 0x5603ee763b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:09.063 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.062+0000 7fa101c00640 1 -- 172.21.15.83:0/3649979926 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603ee764910 con 0x5603ee760390 2024-04-24T09:13:09.067 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.062+0000 7fa101c00640 1 --2- 172.21.15.83:0/3649979926 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5603ee760390 0x5603ee7640e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa0e800a010 tx=0x7fa0e802d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb5ec588a87d01fd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:09.068 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.062+0000 7fa0e6a00640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-04-24T09:13:09.068 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.063+0000 7fa0e6a00640 1 -- 172.21.15.83:0/3649979926 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e8045060 con 0x5603ee760390 2024-04-24T09:13:09.068 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.063+0000 7fa0e6a00640 1 -- 172.21.15.83:0/3649979926 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9671+0+0 (secure 0 0 0) 0x7fa0e80035f0 con 0x5603ee760390 2024-04-24T09:13:09.068 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.063+0000 7fa106970740 1 -- 172.21.15.83:0/3649979926 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0ec0a4b40 con 0x5603ee760390 2024-04-24T09:13:09.068 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.063+0000 7fa0e6a00640 1 -- 172.21.15.83:0/3649979926 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7fa0e8024bd0 con 0x5603ee760390 2024-04-24T09:13:09.068 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.064+0000 7fa106970740 10 client.29809 fetch_fsmap learned FSMap version 64 2024-04-24T09:13:09.068 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.064+0000 7fa106970740 1 -- 172.21.15.83:0/3649979926 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x5603ee7d6720 con 0x5603ee760390 2024-04-24T09:13:09.068 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.064+0000 7fa0e6a00640 1 -- 172.21.15.83:0/3649979926 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa0ec0a4b40 con 0x5603ee760390 2024-04-24T09:13:09.068 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.065+0000 7fa0e6a00640 1 -- 172.21.15.83:0/3649979926 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 64) ==== 8374+0+0 (secure 0 0 0) 0x7fa0e8036040 con 0x5603ee760390 2024-04-24T09:13:09.068 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.065+0000 7fa106970740 10 client.29809 fetch_fsmap finished waiting for FSMap version 64 2024-04-24T09:13:09.068 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.065+0000 7fa106970740 10 client.29809 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:13:09.068 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.065+0000 7fa106970740 20 client.29809 populate_metadata read hostname 'smithi083' 2024-04-24T09:13:09.068 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.065+0000 7fa106970740 1 --2- 172.21.15.83:0/3649979926 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5603ee7d6d90 0x5603ee7d91b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:09.069 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.065+0000 7fa106970740 4 client.29809 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T09:13:09.069 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.065+0000 7fa106970740 1 -- 172.21.15.83:0/3649979926 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5603ee6c3190 con 0x5603ee7d6d90 2024-04-24T09:13:09.069 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.065+0000 7fa101200640 1 --2- 172.21.15.83:0/3649979926 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5603ee7d6d90 0x5603ee7d91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:09.069 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.066+0000 7fa101200640 1 --2- 172.21.15.83:0/3649979926 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5603ee7d6d90 0x5603ee7d91b0 crc :-1 s=READY pgs=532 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57324624ee6e95bc server_cookie=3f1de2a6099f7bac in_seq=0 out_seq=0 2024-04-24T09:13:09.069 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.066+0000 7fa0e6a00640 10 client.29809 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:13:09.197 INFO:tasks.workunit.client.0.smithi083.stdout: 10 57251 6.62 MB/sec execute 340 sec latency 358.540 ms 2024-04-24T09:13:09.236 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.236+0000 7fa0e6a00640 1 -- 172.21.15.83:0/3649979926 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5603ee6c3190 con 0x5603ee7d6d90 2024-04-24T09:13:09.236 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.236+0000 7fa0e6a00640 10 client.29809 handle_command_reply: tid=0 2024-04-24T09:13:09.237 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.236+0000 7fa106970740 1 client.29809 shutdown 2024-04-24T09:13:09.239 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.237+0000 7fa106970740 1 -- 172.21.15.83:0/3649979926 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5603ee760390 msgr2=0x5603ee7640e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:09.239 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.237+0000 7fa106970740 1 --2- 172.21.15.83:0/3649979926 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5603ee760390 0x5603ee7640e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa0e800a010 tx=0x7fa0e802d000 comp rx=0 tx=0).stop 2024-04-24T09:13:09.239 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.239+0000 7fa106970740 1 -- 172.21.15.83:0/3649979926 shutdown_connections 2024-04-24T09:13:09.240 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.239+0000 7fa106970740 1 --2- 172.21.15.83:0/3649979926 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5603ee7d6d90 0x5603ee7d91b0 crc :-1 s=READY pgs=532 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:09.240 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.239+0000 7fa106970740 1 --2- 172.21.15.83:0/3649979926 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5603ee760390 0x5603ee7640e0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:09.240 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.239+0000 7fa106970740 1 --2- 172.21.15.83:0/3649979926 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5603ee755ad0 0x5603ee763b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:09.240 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.239+0000 7fa106970740 1 --2- 172.21.15.83:0/3649979926 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5603ee7551b0 0x5603ee7687f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:09.240 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.239+0000 7fa106970740 1 -- 172.21.15.83:0/3649979926 >> 172.21.15.83:0/3649979926 conn(0x5603ee7516c0 msgr2=0x5603ee75f3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:09.240 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.239+0000 7fa0e6a00640 0 client.29809 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:13:09.240 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.239+0000 7fa106970740 1 -- 172.21.15.83:0/3649979926 shutdown_connections 2024-04-24T09:13:09.240 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.240+0000 7fa106970740 1 -- 172.21.15.83:0/3649979926 wait complete. 2024-04-24T09:13:09.240 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.240+0000 7fa106970740 20 client.29809 trim_cache size 0 max 16384 2024-04-24T09:13:09.241 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T09:13:09.241 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T09:13:09.241 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T09:13:09.241 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T09:13:09.243 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.241+0000 7fa103c00640 1 -- 172.21.15.83:0/2445039141 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa0e00a3e40 msgr2=0x7fa0e00a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:09.243 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.241+0000 7fa103c00640 1 --2- 172.21.15.83:0/2445039141 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa0e00a3e40 0x7fa0e00a6300 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fa0e800ec20 tx=0x7fa0e8014000 comp rx=0 tx=0).stop 2024-04-24T09:13:09.243 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.241+0000 7fa103c00640 1 -- 172.21.15.83:0/2445039141 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa0fc10c700 msgr2=0x7fa0fc1a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:09.243 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.241+0000 7fa103c00640 1 --2- 172.21.15.83:0/2445039141 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa0fc10c700 0x7fa0fc1a3800 secure :-1 s=READY pgs=2597 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec00b4a0 tx=0x7fa0ec00b960 comp rx=0 tx=0).stop 2024-04-24T09:13:09.244 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.241+0000 7fa103c00640 1 -- 172.21.15.83:0/2445039141 shutdown_connections 2024-04-24T09:13:09.244 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.241+0000 7fa103c00640 1 --2- 172.21.15.83:0/2445039141 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa0e00a3e40 0x7fa0e00a6300 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:09.244 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.241+0000 7fa103c00640 1 --2- 172.21.15.83:0/2445039141 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa0fc117090 0x7fa0fc1a7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:09.244 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.241+0000 7fa103c00640 1 --2- 172.21.15.83:0/2445039141 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa0fc10c700 0x7fa0fc1a3800 unknown :-1 s=CLOSED pgs=2597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:09.244 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.241+0000 7fa103c00640 1 --2- 172.21.15.83:0/2445039141 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa0fc109540 0x7fa0fc1a32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:09.244 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.241+0000 7fa103c00640 1 -- 172.21.15.83:0/2445039141 >> 172.21.15.83:0/2445039141 conn(0x7fa0fc06d0d0 msgr2=0x7fa0fc117470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:09.244 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.242+0000 7fa103c00640 1 -- 172.21.15.83:0/2445039141 shutdown_connections 2024-04-24T09:13:09.244 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:09.242+0000 7fa103c00640 1 -- 172.21.15.83:0/2445039141 wait complete. 2024-04-24T09:13:10.197 INFO:tasks.workunit.client.0.smithi083.stdout: 10 57385 6.63 MB/sec execute 341 sec latency 135.139 ms 2024-04-24T09:13:10.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:13:10 smithi138 ceph-mon[31441]: pgmap v680: 129 pgs: 129 active+clean; 891 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.3 MiB/s wr, 282 op/s 2024-04-24T09:13:10.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:13:10 smithi138 ceph-mon[31441]: osdmap e126: 12 total, 12 up, 12 in 2024-04-24T09:13:10.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:13:10 smithi189 ceph-mon[31791]: pgmap v680: 129 pgs: 129 active+clean; 891 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.3 MiB/s wr, 282 op/s 2024-04-24T09:13:10.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:13:10 smithi189 ceph-mon[31791]: osdmap e126: 12 total, 12 up, 12 in 2024-04-24T09:13:10.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:13:10 smithi083 ceph-mon[27907]: pgmap v680: 129 pgs: 129 active+clean; 891 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.3 MiB/s wr, 282 op/s 2024-04-24T09:13:10.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:13:10 smithi083 ceph-mon[27907]: osdmap e126: 12 total, 12 up, 12 in 2024-04-24T09:13:11.196 INFO:tasks.workunit.client.0.smithi083.stdout: 10 57463 6.61 MB/sec execute 342 sec latency 459.135 ms 2024-04-24T09:13:12.196 INFO:tasks.workunit.client.0.smithi083.stdout: 10 57598 6.62 MB/sec execute 343 sec latency 110.632 ms 2024-04-24T09:13:12.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:13:12 smithi138 ceph-mon[31441]: pgmap v682: 129 pgs: 129 active+clean; 891 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 166 op/s 2024-04-24T09:13:12.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:13:12 smithi189 ceph-mon[31791]: pgmap v682: 129 pgs: 129 active+clean; 891 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 166 op/s 2024-04-24T09:13:12.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:13:12 smithi083 ceph-mon[27907]: pgmap v682: 129 pgs: 129 active+clean; 891 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 166 op/s 2024-04-24T09:13:13.196 INFO:tasks.workunit.client.0.smithi083.stdout: 10 57717 6.61 MB/sec execute 344 sec latency 442.221 ms 2024-04-24T09:13:14.196 INFO:tasks.workunit.client.0.smithi083.stdout: 10 57832 6.60 MB/sec execute 345 sec latency 90.251 ms 2024-04-24T09:13:14.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:13:14 smithi138 ceph-mon[31441]: pgmap v683: 129 pgs: 129 active+clean; 899 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.6 MiB/s wr, 284 op/s 2024-04-24T09:13:14.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:13:14 smithi189 ceph-mon[31791]: pgmap v683: 129 pgs: 129 active+clean; 899 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.6 MiB/s wr, 284 op/s 2024-04-24T09:13:14.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:13:14 smithi083 ceph-mon[27907]: pgmap v683: 129 pgs: 129 active+clean; 899 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.6 MiB/s wr, 284 op/s 2024-04-24T09:13:15.197 INFO:tasks.workunit.client.0.smithi083.stdout: 10 57981 6.61 MB/sec execute 346 sec latency 93.334 ms 2024-04-24T09:13:16.197 INFO:tasks.workunit.client.0.smithi083.stdout: 10 58074 6.61 MB/sec execute 347 sec latency 396.904 ms 2024-04-24T09:13:16.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:13:16 smithi138 ceph-mon[31441]: pgmap v684: 129 pgs: 129 active+clean; 899 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.5 MiB/s wr, 281 op/s 2024-04-24T09:13:16.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:13:16 smithi189 ceph-mon[31791]: pgmap v684: 129 pgs: 129 active+clean; 899 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.5 MiB/s wr, 281 op/s 2024-04-24T09:13:16.552 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:13:16 smithi083 ceph-mon[27907]: pgmap v684: 129 pgs: 129 active+clean; 899 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.5 MiB/s wr, 281 op/s 2024-04-24T09:13:17.197 INFO:tasks.workunit.client.0.smithi083.stdout: 10 58235 6.62 MB/sec execute 348 sec latency 82.591 ms 2024-04-24T09:13:17.426 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T09:13:17.426 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T09:13:17.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.710+0000 7f3edb400640 1 -- 172.21.15.83:0/859674712 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3ed4104de0 msgr2=0x7f3ed4108e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:17.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.710+0000 7f3edb400640 1 --2- 172.21.15.83:0/859674712 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3ed4104de0 0x7f3ed4108e20 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc009900 tx=0x7f3ebc02a3f0 comp rx=0 tx=0).stop 2024-04-24T09:13:17.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.711+0000 7f3edb400640 1 -- 172.21.15.83:0/859674712 shutdown_connections 2024-04-24T09:13:17.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.711+0000 7f3edb400640 1 --2- 172.21.15.83:0/859674712 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ed4109500 0x7f3ed41110d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:17.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.711+0000 7f3edb400640 1 --2- 172.21.15.83:0/859674712 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3ed4104de0 0x7f3ed4108e20 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:17.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.711+0000 7f3edb400640 1 --2- 172.21.15.83:0/859674712 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3ed4104430 0x7f3ed4104810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:17.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.711+0000 7f3edb400640 1 -- 172.21.15.83:0/859674712 >> 172.21.15.83:0/859674712 conn(0x7f3ed40ffcb0 msgr2=0x7f3ed41020d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:17.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.711+0000 7f3edb400640 1 -- 172.21.15.83:0/859674712 shutdown_connections 2024-04-24T09:13:17.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.712+0000 7f3edb400640 1 -- 172.21.15.83:0/859674712 wait complete. 2024-04-24T09:13:17.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.713+0000 7f3edb400640 1 Processor -- start 2024-04-24T09:13:17.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.713+0000 7f3edb400640 1 -- start start 2024-04-24T09:13:17.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.713+0000 7f3edb400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3ed4104430 0x7f3ed4110c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:17.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.713+0000 7f3edb400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3ed4109500 0x7f3ed411c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:17.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.714+0000 7f3edb400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ed4111170 0x7f3ed410ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:17.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.714+0000 7f3edb400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed40767d0 con 0x7f3ed4111170 2024-04-24T09:13:17.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.714+0000 7f3edb400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed4076650 con 0x7f3ed4104430 2024-04-24T09:13:17.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.714+0000 7f3edb400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed4076950 con 0x7f3ed4109500 2024-04-24T09:13:17.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.714+0000 7f3ed9400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ed4111170 0x7f3ed410ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:17.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.714+0000 7f3ed9400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ed4111170 0x7f3ed410ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40438/0 (socket says 172.21.15.83:40438) 2024-04-24T09:13:17.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.714+0000 7f3ed9400640 1 -- 172.21.15.83:0/2025580665 learned_addr learned my addr 172.21.15.83:0/2025580665 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:13:17.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.714+0000 7f3ed8a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3ed4104430 0x7f3ed4110c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:17.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.714+0000 7f3ed3e00640 1 --2- 172.21.15.83:0/2025580665 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3ed4109500 0x7f3ed411c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:17.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.715+0000 7f3ed9400640 1 -- 172.21.15.83:0/2025580665 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3ed4109500 msgr2=0x7f3ed411c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:17.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.715+0000 7f3ed9400640 1 --2- 172.21.15.83:0/2025580665 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3ed4109500 0x7f3ed411c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:17.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.715+0000 7f3ed9400640 1 -- 172.21.15.83:0/2025580665 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3ed4104430 msgr2=0x7f3ed4110c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:17.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.715+0000 7f3ed9400640 1 --2- 172.21.15.83:0/2025580665 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3ed4104430 0x7f3ed4110c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:17.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.715+0000 7f3ed9400640 1 -- 172.21.15.83:0/2025580665 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ed410f500 con 0x7f3ed4111170 2024-04-24T09:13:17.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.715+0000 7f3ed9400640 1 --2- 172.21.15.83:0/2025580665 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ed4111170 0x7f3ed410ed30 secure :-1 s=READY pgs=2600 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc00eb10 tx=0x7f3ecc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:17.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.717+0000 7f3ed1600640 1 -- 172.21.15.83:0/2025580665 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ecc004b20 con 0x7f3ed4111170 2024-04-24T09:13:17.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.717+0000 7f3ed1600640 1 -- 172.21.15.83:0/2025580665 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3ecc004cc0 con 0x7f3ed4111170 2024-04-24T09:13:17.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.717+0000 7f3edb400640 1 -- 172.21.15.83:0/2025580665 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ed410f7f0 con 0x7f3ed4111170 2024-04-24T09:13:17.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.717+0000 7f3ed1600640 1 -- 172.21.15.83:0/2025580665 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ecc010690 con 0x7f3ed4111170 2024-04-24T09:13:17.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.717+0000 7f3edb400640 1 -- 172.21.15.83:0/2025580665 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ed41b4640 con 0x7f3ed4111170 2024-04-24T09:13:17.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.721+0000 7f3ede3b0740 1 Processor -- start 2024-04-24T09:13:17.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.722+0000 7f3ed1600640 1 -- 172.21.15.83:0/2025580665 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3ecc010830 con 0x7f3ed4111170 2024-04-24T09:13:17.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.722+0000 7f3ede3b0740 1 -- start start 2024-04-24T09:13:17.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.722+0000 7f3ede3b0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562db529a860 0x562db529ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:17.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.722+0000 7f3ede3b0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562db529b180 0x562db52a5310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:17.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.722+0000 7f3ede3b0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562db52a5850 0x562db52a7c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:17.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.722+0000 7f3ede3b0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562db52001f0 con 0x562db529b180 2024-04-24T09:13:17.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.722+0000 7f3ede3b0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562db51a6350 con 0x562db52a5850 2024-04-24T09:13:17.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.722+0000 7f3ede3b0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x562db51fc1b0 con 0x562db529a860 2024-04-24T09:13:17.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.725+0000 7f3ed3e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562db529a860 0x562db529ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:17.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.725+0000 7f3ed3e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562db52a5850 0x562db52a7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:17.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.725+0000 7f3ed3e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562db52a5850 0x562db52a7c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:57042/0 (socket says 172.21.15.83:57042) 2024-04-24T09:13:17.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.725+0000 7f3ed3e00640 1 -- 172.21.15.83:0/957780269 learned_addr learned my addr 172.21.15.83:0/957780269 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:13:17.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.726+0000 7f3ed8a00640 1 --2- 172.21.15.83:0/957780269 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562db529b180 0x562db52a5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:17.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.727+0000 7f3ed1600640 1 --2- 172.21.15.83:0/2025580665 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f3eb40a3f20 0x7f3eb40a63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:17.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.727+0000 7f3ed1600640 1 -- 172.21.15.83:0/2025580665 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 9583+0+0 (secure 0 0 0) 0x7f3ecc0d11d0 con 0x7f3ed4111170 2024-04-24T09:13:17.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.727+0000 7f3ed8a00640 1 -- 172.21.15.83:0/957780269 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562db529a860 msgr2=0x562db529ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:17.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.727+0000 7f3ed8a00640 1 --2- 172.21.15.83:0/957780269 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562db529a860 0x562db529ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:17.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.727+0000 7f3ed8a00640 1 -- 172.21.15.83:0/957780269 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562db52a5850 msgr2=0x562db52a7c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:17.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.727+0000 7f3ed8a00640 1 --2- 172.21.15.83:0/957780269 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562db52a5850 0x562db52a7c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:17.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.727+0000 7f3ed8a00640 1 -- 172.21.15.83:0/957780269 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562db51faf50 con 0x562db529b180 2024-04-24T09:13:17.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.728+0000 7f3ed9400640 1 --2- 172.21.15.83:0/2025580665 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f3eb40a3f20 0x7f3eb40a63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:17.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.729+0000 7f3ed8a00640 1 --2- 172.21.15.83:0/957780269 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562db529b180 0x562db52a5310 secure :-1 s=READY pgs=2601 cs=0 l=1 rev1=1 crypto rx=0x7f3ec4009c00 tx=0x7f3ec4011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abdc5f877298b125 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:17.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.729+0000 7f3ed9400640 1 --2- 172.21.15.83:0/2025580665 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f3eb40a3f20 0x7f3eb40a63e0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc0049d0 tx=0x7f3ecc005b40 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:17.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.730+0000 7f3ec9c00640 1 -- 172.21.15.83:0/957780269 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec4004230 con 0x562db529b180 2024-04-24T09:13:17.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.730+0000 7f3ec9c00640 1 -- 172.21.15.83:0/957780269 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3ec40043d0 con 0x562db529b180 2024-04-24T09:13:17.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.730+0000 7f3ec9c00640 1 -- 172.21.15.83:0/957780269 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec4004230 con 0x562db529b180 2024-04-24T09:13:17.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.731+0000 7f3ede3b0740 1 -- 172.21.15.83:0/957780269 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562db529b180 msgr2=0x562db52a5310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:17.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.731+0000 7f3ede3b0740 1 --2- 172.21.15.83:0/957780269 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562db529b180 0x562db52a5310 secure :-1 s=READY pgs=2601 cs=0 l=1 rev1=1 crypto rx=0x7f3ec4009c00 tx=0x7f3ec4011810 comp rx=0 tx=0).stop 2024-04-24T09:13:17.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.732+0000 7f3ede3b0740 1 -- 172.21.15.83:0/957780269 shutdown_connections 2024-04-24T09:13:17.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.732+0000 7f3ede3b0740 1 --2- 172.21.15.83:0/957780269 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562db52a5850 0x562db52a7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:17.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.732+0000 7f3ede3b0740 1 --2- 172.21.15.83:0/957780269 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562db529b180 0x562db52a5310 unknown :-1 s=CLOSED pgs=2601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:17.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.732+0000 7f3ede3b0740 1 --2- 172.21.15.83:0/957780269 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562db529a860 0x562db529ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:17.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.732+0000 7f3ede3b0740 1 -- 172.21.15.83:0/957780269 >> 172.21.15.83:0/957780269 conn(0x562db5296b80 msgr2=0x562db5296f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:17.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.732+0000 7f3ede3b0740 1 -- 172.21.15.83:0/957780269 shutdown_connections 2024-04-24T09:13:17.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.732+0000 7f3ede3b0740 1 -- 172.21.15.83:0/957780269 wait complete. 2024-04-24T09:13:17.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.732+0000 7f3ede3b0740 1 -- start start 2024-04-24T09:13:17.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.733+0000 7f3ede3b0740 1 Processor -- start 2024-04-24T09:13:17.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.733+0000 7f3ede3b0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562db529a860 0x562db52a2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:17.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.733+0000 7f3ede3b0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562db529b180 0x562db52a2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:17.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.733+0000 7f3ede3b0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562db52a5850 0x562db52a3200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:17.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.733+0000 7f3ede3b0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562db52a8640 con 0x562db529a860 2024-04-24T09:13:17.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.733+0000 7f3ede3b0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562db51b9690 con 0x562db52a5850 2024-04-24T09:13:17.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.733+0000 7f3ede3b0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x562db52a87c0 con 0x562db529b180 2024-04-24T09:13:17.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.733+0000 7f3ede3b0740 10 client.0 _ll_register_callbacks cb 0x562db51f5650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:13:17.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.733+0000 7f3ed3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562db529a860 0x562db52a2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:17.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.734+0000 7f3ed8a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562db529b180 0x562db52a2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:17.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.734+0000 7f3ed8a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562db529b180 0x562db52a2cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:52902/0 (socket says 172.21.15.83:52902) 2024-04-24T09:13:17.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.734+0000 7f3ed8a00640 1 -- 172.21.15.83:0/2659830193 learned_addr learned my addr 172.21.15.83:0/2659830193 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:13:17.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.734+0000 7f3ed3e00640 1 -- 172.21.15.83:0/2659830193 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562db529b180 msgr2=0x562db52a2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:17.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.734+0000 7f3ed3e00640 1 --2- 172.21.15.83:0/2659830193 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562db529b180 0x562db52a2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:17.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.734+0000 7f3ed3e00640 1 -- 172.21.15.83:0/2659830193 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562db52a5850 msgr2=0x562db52a3200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:17.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.734+0000 7f3ed3e00640 1 --2- 172.21.15.83:0/2659830193 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562db52a5850 0x562db52a3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:17.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.734+0000 7f3ed3e00640 1 -- 172.21.15.83:0/2659830193 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562db51b8920 con 0x562db529a860 2024-04-24T09:13:17.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.734+0000 7f3ed3e00640 1 --2- 172.21.15.83:0/2659830193 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562db529a860 0x562db52a2780 secure :-1 s=READY pgs=2602 cs=0 l=1 rev1=1 crypto rx=0x562db5297d10 tx=0x7f3ebc005e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76a905184a78c48b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:17.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.734+0000 7f3ec2000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:13:17.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.735+0000 7f3ec2000640 1 -- 172.21.15.83:0/2659830193 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ebc035040 con 0x562db529a860 2024-04-24T09:13:17.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.735+0000 7f3ec2000640 1 -- 172.21.15.83:0/2659830193 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9583+0+0 (secure 0 0 0) 0x7f3ebc003a00 con 0x562db529a860 2024-04-24T09:13:17.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.735+0000 7f3ede3b0740 1 -- 172.21.15.83:0/2659830193 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ec40043d0 con 0x562db529a860 2024-04-24T09:13:17.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.735+0000 7f3ec2000640 1 -- 172.21.15.83:0/2659830193 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f3ebc004490 con 0x562db529a860 2024-04-24T09:13:17.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.735+0000 7f3ede3b0740 10 client.22956 fetch_fsmap learned FSMap version 64 2024-04-24T09:13:17.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.735+0000 7f3ede3b0740 1 -- 172.21.15.83:0/2659830193 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x562db4fdc7d0 con 0x562db529a860 2024-04-24T09:13:17.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.735+0000 7f3ec2000640 1 -- 172.21.15.83:0/2659830193 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3ec40043d0 con 0x562db529a860 2024-04-24T09:13:17.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.736+0000 7f3ec2000640 1 -- 172.21.15.83:0/2659830193 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 64) ==== 8374+0+0 (secure 0 0 0) 0x7f3ebc045070 con 0x562db529a860 2024-04-24T09:13:17.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.736+0000 7f3ede3b0740 10 client.22956 fetch_fsmap finished waiting for FSMap version 64 2024-04-24T09:13:17.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.736+0000 7f3ede3b0740 10 client.22956 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:13:17.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.736+0000 7f3ede3b0740 20 client.22956 populate_metadata read hostname 'smithi083' 2024-04-24T09:13:17.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.736+0000 7f3ede3b0740 1 --2- 172.21.15.83:0/2659830193 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x562db52b5bc0 0x562db52b8020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:17.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.736+0000 7f3ede3b0740 4 client.22956 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:13:17.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.736+0000 7f3ede3b0740 1 -- 172.21.15.83:0/2659830193 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562db520dc10 con 0x562db52b5bc0 2024-04-24T09:13:17.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.736+0000 7f3ed8a00640 1 --2- 172.21.15.83:0/2659830193 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x562db52b5bc0 0x562db52b8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:17.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.737+0000 7f3ed8a00640 1 --2- 172.21.15.83:0/2659830193 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x562db52b5bc0 0x562db52b8020 crc :-1 s=READY pgs=533 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c931c0889e0e01b server_cookie=da0b25e7bba21d36 in_seq=0 out_seq=0 2024-04-24T09:13:17.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.737+0000 7f3ec2000640 10 client.22956 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:13:17.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.738+0000 7f3ec2000640 1 -- 172.21.15.83:0/2659830193 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562db520dc10 con 0x562db52b5bc0 2024-04-24T09:13:17.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.738+0000 7f3ec2000640 10 client.22956 handle_command_reply: tid=0 2024-04-24T09:13:17.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.738+0000 7f3ede3b0740 1 client.22956 shutdown 2024-04-24T09:13:17.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.738+0000 7f3ede3b0740 1 -- 172.21.15.83:0/2659830193 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562db529a860 msgr2=0x562db52a2780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:17.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.738+0000 7f3ede3b0740 1 --2- 172.21.15.83:0/2659830193 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562db529a860 0x562db52a2780 secure :-1 s=READY pgs=2602 cs=0 l=1 rev1=1 crypto rx=0x562db5297d10 tx=0x7f3ebc005e70 comp rx=0 tx=0).stop 2024-04-24T09:13:17.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.738+0000 7f3ede3b0740 1 -- 172.21.15.83:0/2659830193 shutdown_connections 2024-04-24T09:13:17.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.738+0000 7f3ede3b0740 1 --2- 172.21.15.83:0/2659830193 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x562db52b5bc0 0x562db52b8020 crc :-1 s=READY pgs=533 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:17.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.738+0000 7f3ede3b0740 1 --2- 172.21.15.83:0/2659830193 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562db52a5850 0x562db52a3200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:17.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.738+0000 7f3ede3b0740 1 --2- 172.21.15.83:0/2659830193 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562db529b180 0x562db52a2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:17.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.738+0000 7f3ede3b0740 1 --2- 172.21.15.83:0/2659830193 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562db529a860 0x562db52a2780 unknown :-1 s=CLOSED pgs=2602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:17.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.738+0000 7f3ede3b0740 1 -- 172.21.15.83:0/2659830193 >> 172.21.15.83:0/2659830193 conn(0x562db5296b80 msgr2=0x562db529c980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:17.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.738+0000 7f3ec2000640 0 client.22956 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:13:17.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.738+0000 7f3ede3b0740 1 -- 172.21.15.83:0/2659830193 shutdown_connections 2024-04-24T09:13:17.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.738+0000 7f3ede3b0740 1 -- 172.21.15.83:0/2659830193 wait complete. 2024-04-24T09:13:17.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.738+0000 7f3ede3b0740 20 client.22956 trim_cache size 0 max 16384 2024-04-24T09:13:17.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.757+0000 7f3ede3b0740 1 Processor -- start 2024-04-24T09:13:17.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.757+0000 7f3ede3b0740 1 -- start start 2024-04-24T09:13:17.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.757+0000 7f3ede3b0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562db529a860 0x562db52b1500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:17.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.757+0000 7f3ede3b0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562db529b180 0x562db52a1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:17.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.757+0000 7f3ede3b0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562db52a5850 0x562db5295aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:17.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.757+0000 7f3ede3b0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed41be4a0 con 0x562db52a5850 2024-04-24T09:13:17.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.757+0000 7f3ede3b0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed41c0070 con 0x562db529a860 2024-04-24T09:13:17.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.757+0000 7f3ede3b0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed41beb70 con 0x562db529b180 2024-04-24T09:13:17.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.758+0000 7f3ed3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562db52a5850 0x562db5295aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:17.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.758+0000 7f3ed3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562db52a5850 0x562db5295aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40468/0 (socket says 172.21.15.83:40468) 2024-04-24T09:13:17.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.758+0000 7f3ed3e00640 1 -- 172.21.15.83:0/4173465085 learned_addr learned my addr 172.21.15.83:0/4173465085 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:13:17.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.758+0000 7f3ed8a00640 1 --2- 172.21.15.83:0/4173465085 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562db529b180 0x562db52a1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:17.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.758+0000 7f3ed3e00640 1 --2- 172.21.15.83:0/4173465085 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562db529a860 0x562db52b1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:17.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.758+0000 7f3ed3e00640 1 -- 172.21.15.83:0/4173465085 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562db529b180 msgr2=0x562db52a1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:17.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.758+0000 7f3ed3e00640 1 --2- 172.21.15.83:0/4173465085 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562db529b180 0x562db52a1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:17.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.758+0000 7f3ed3e00640 1 -- 172.21.15.83:0/4173465085 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562db529a860 msgr2=0x562db52b1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:17.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.758+0000 7f3ed3e00640 1 --2- 172.21.15.83:0/4173465085 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562db529a860 0x562db52b1500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:17.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.759+0000 7f3ed3e00640 1 -- 172.21.15.83:0/4173465085 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562db4f123f0 con 0x562db52a5850 2024-04-24T09:13:17.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.759+0000 7f3ed3e00640 1 --2- 172.21.15.83:0/4173465085 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562db52a5850 0x562db5295aa0 secure :-1 s=READY pgs=2603 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc03d4f0 tx=0x7f3ebc02a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a095e6da2078faed server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:17.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.759+0000 7f3ec9200640 1 -- 172.21.15.83:0/4173465085 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ebc035070 con 0x562db52a5850 2024-04-24T09:13:17.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.759+0000 7f3ec9200640 1 -- 172.21.15.83:0/4173465085 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3ebc003800 con 0x562db52a5850 2024-04-24T09:13:17.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.759+0000 7f3ec9200640 1 -- 172.21.15.83:0/4173465085 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ebc03d910 con 0x562db52a5850 2024-04-24T09:13:17.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.760+0000 7f3ede3b0740 1 -- 172.21.15.83:0/4173465085 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562db52a5850 msgr2=0x562db5295aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:17.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.760+0000 7f3ede3b0740 1 --2- 172.21.15.83:0/4173465085 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562db52a5850 0x562db5295aa0 secure :-1 s=READY pgs=2603 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc03d4f0 tx=0x7f3ebc02a3f0 comp rx=0 tx=0).stop 2024-04-24T09:13:17.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.760+0000 7f3ede3b0740 1 -- 172.21.15.83:0/4173465085 shutdown_connections 2024-04-24T09:13:17.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.760+0000 7f3ede3b0740 1 --2- 172.21.15.83:0/4173465085 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562db52a5850 0x562db5295aa0 unknown :-1 s=CLOSED pgs=2603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:17.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.760+0000 7f3ede3b0740 1 --2- 172.21.15.83:0/4173465085 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562db529b180 0x562db52a1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:17.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.760+0000 7f3ede3b0740 1 --2- 172.21.15.83:0/4173465085 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562db529a860 0x562db52b1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:17.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.760+0000 7f3ede3b0740 1 -- 172.21.15.83:0/4173465085 >> 172.21.15.83:0/4173465085 conn(0x562db5296b80 msgr2=0x562db5295390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:17.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.760+0000 7f3ede3b0740 1 -- 172.21.15.83:0/4173465085 shutdown_connections 2024-04-24T09:13:17.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.760+0000 7f3ede3b0740 1 -- 172.21.15.83:0/4173465085 wait complete. 2024-04-24T09:13:17.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.760+0000 7f3ede3b0740 1 -- start start 2024-04-24T09:13:17.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.761+0000 7f3ede3b0740 1 Processor -- start 2024-04-24T09:13:17.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.761+0000 7f3ede3b0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562db529a860 0x562db529f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:17.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.761+0000 7f3ede3b0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562db529b180 0x562db529f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:17.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.761+0000 7f3ede3b0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562db52a5850 0x562db529fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:17.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.761+0000 7f3ede3b0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed41bf970 con 0x562db529a860 2024-04-24T09:13:17.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.761+0000 7f3ede3b0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed41bf2a0 con 0x562db529b180 2024-04-24T09:13:17.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.761+0000 7f3ede3b0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed41bc4a0 con 0x562db52a5850 2024-04-24T09:13:17.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.761+0000 7f3ede3b0740 10 client.0 _ll_register_callbacks cb 0x7f3ec4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:13:17.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.761+0000 7f3ed3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562db529a860 0x562db529f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:17.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.761+0000 7f3ed3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562db529a860 0x562db529f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40472/0 (socket says 172.21.15.83:40472) 2024-04-24T09:13:17.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.761+0000 7f3ed3e00640 1 -- 172.21.15.83:0/2049409719 learned_addr learned my addr 172.21.15.83:0/2049409719 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:13:17.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.762+0000 7f3ed8a00640 1 --2- 172.21.15.83:0/2049409719 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562db529b180 0x562db529f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:17.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.762+0000 7f3ed3e00640 1 -- 172.21.15.83:0/2049409719 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562db52a5850 msgr2=0x562db529fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:17.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.762+0000 7f3ed3e00640 1 --2- 172.21.15.83:0/2049409719 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562db52a5850 0x562db529fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:17.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.762+0000 7f3ed3e00640 1 -- 172.21.15.83:0/2049409719 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562db529b180 msgr2=0x562db529f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:17.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.762+0000 7f3ed3e00640 1 --2- 172.21.15.83:0/2049409719 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562db529b180 0x562db529f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:17.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.762+0000 7f3ed3e00640 1 -- 172.21.15.83:0/2049409719 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562db4ed20f0 con 0x562db529a860 2024-04-24T09:13:17.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.762+0000 7f3ed3e00640 1 --2- 172.21.15.83:0/2049409719 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562db529a860 0x562db529f450 secure :-1 s=READY pgs=2604 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc031d70 tx=0x7f3ebc02a930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f012cd6a8490a8ad server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:17.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.762+0000 7f3ec2000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:13:17.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.762+0000 7f3ec2000640 1 -- 172.21.15.83:0/2049409719 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ebc035040 con 0x562db529a860 2024-04-24T09:13:17.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.762+0000 7f3ec2000640 1 -- 172.21.15.83:0/2049409719 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9583+0+0 (secure 0 0 0) 0x7f3ebc0318b0 con 0x562db529a860 2024-04-24T09:13:17.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.762+0000 7f3ede3b0740 1 -- 172.21.15.83:0/2049409719 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ebc003800 con 0x562db529a860 2024-04-24T09:13:17.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.762+0000 7f3ec2000640 1 -- 172.21.15.83:0/2049409719 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f3ebc03d6f0 con 0x562db529a860 2024-04-24T09:13:17.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.762+0000 7f3ede3b0740 10 client.22962 fetch_fsmap learned FSMap version 64 2024-04-24T09:13:17.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.762+0000 7f3ede3b0740 1 -- 172.21.15.83:0/2049409719 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x562db51c5810 con 0x562db529a860 2024-04-24T09:13:17.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.763+0000 7f3ec2000640 1 -- 172.21.15.83:0/2049409719 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3ebc003800 con 0x562db529a860 2024-04-24T09:13:17.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.763+0000 7f3ec2000640 1 -- 172.21.15.83:0/2049409719 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 64) ==== 8374+0+0 (secure 0 0 0) 0x7f3ebc045040 con 0x562db529a860 2024-04-24T09:13:17.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.763+0000 7f3ede3b0740 10 client.22962 fetch_fsmap finished waiting for FSMap version 64 2024-04-24T09:13:17.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.763+0000 7f3ede3b0740 10 client.22962 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:13:17.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.763+0000 7f3ede3b0740 20 client.22962 populate_metadata read hostname 'smithi083' 2024-04-24T09:13:17.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.763+0000 7f3ede3b0740 1 --2- 172.21.15.83:0/2049409719 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x562db531c720 0x562db531eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:17.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.763+0000 7f3ede3b0740 4 client.22962 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T09:13:17.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.763+0000 7f3ede3b0740 1 -- 172.21.15.83:0/2049409719 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562db5207a80 con 0x562db531c720 2024-04-24T09:13:17.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.764+0000 7f3ed8a00640 1 --2- 172.21.15.83:0/2049409719 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x562db531c720 0x562db531eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:17.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.764+0000 7f3ed8a00640 1 --2- 172.21.15.83:0/2049409719 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x562db531c720 0x562db531eb40 crc :-1 s=READY pgs=534 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7360b203d50d3fc0 server_cookie=f21aa0b19bf11248 in_seq=0 out_seq=0 2024-04-24T09:13:17.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.764+0000 7f3ec2000640 10 client.22962 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:13:17.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.765+0000 7f3ec2000640 1 -- 172.21.15.83:0/2049409719 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x562db5207a80 con 0x562db531c720 2024-04-24T09:13:17.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.765+0000 7f3ec2000640 10 client.22962 handle_command_reply: tid=0 2024-04-24T09:13:17.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.765+0000 7f3ede3b0740 1 client.22962 shutdown 2024-04-24T09:13:17.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.765+0000 7f3ede3b0740 1 -- 172.21.15.83:0/2049409719 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562db529a860 msgr2=0x562db529f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:17.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.765+0000 7f3ede3b0740 1 --2- 172.21.15.83:0/2049409719 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562db529a860 0x562db529f450 secure :-1 s=READY pgs=2604 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc031d70 tx=0x7f3ebc02a930 comp rx=0 tx=0).stop 2024-04-24T09:13:17.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.765+0000 7f3ede3b0740 1 -- 172.21.15.83:0/2049409719 shutdown_connections 2024-04-24T09:13:17.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.765+0000 7f3ede3b0740 1 --2- 172.21.15.83:0/2049409719 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x562db531c720 0x562db531eb40 crc :-1 s=READY pgs=534 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:17.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.765+0000 7f3ede3b0740 1 --2- 172.21.15.83:0/2049409719 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562db52a5850 0x562db529fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:17.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.765+0000 7f3ede3b0740 1 --2- 172.21.15.83:0/2049409719 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562db529b180 0x562db529f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:17.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.765+0000 7f3ec2000640 0 client.22962 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:13:17.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.765+0000 7f3ede3b0740 1 --2- 172.21.15.83:0/2049409719 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562db529a860 0x562db529f450 unknown :-1 s=CLOSED pgs=2604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:17.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.765+0000 7f3ede3b0740 1 -- 172.21.15.83:0/2049409719 >> 172.21.15.83:0/2049409719 conn(0x562db5296b80 msgr2=0x562db52a17a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:17.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.765+0000 7f3ede3b0740 1 -- 172.21.15.83:0/2049409719 shutdown_connections 2024-04-24T09:13:17.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.765+0000 7f3ede3b0740 1 -- 172.21.15.83:0/2049409719 wait complete. 2024-04-24T09:13:17.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.765+0000 7f3ede3b0740 20 client.22962 trim_cache size 0 max 16384 2024-04-24T09:13:17.769 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T09:13:17.769 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-04-24T09:13:17.769 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-04-24T09:13:17.769 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T09:13:17.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.766+0000 7f3edb400640 1 -- 172.21.15.83:0/2025580665 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f3eb40a3f20 msgr2=0x7f3eb40a63e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:17.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.766+0000 7f3edb400640 1 --2- 172.21.15.83:0/2025580665 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f3eb40a3f20 0x7f3eb40a63e0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc0049d0 tx=0x7f3ecc005b40 comp rx=0 tx=0).stop 2024-04-24T09:13:17.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.766+0000 7f3edb400640 1 -- 172.21.15.83:0/2025580665 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ed4111170 msgr2=0x7f3ed410ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:17.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.766+0000 7f3edb400640 1 --2- 172.21.15.83:0/2025580665 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ed4111170 0x7f3ed410ed30 secure :-1 s=READY pgs=2600 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc00eb10 tx=0x7f3ecc00c2d0 comp rx=0 tx=0).stop 2024-04-24T09:13:17.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.766+0000 7f3edb400640 1 -- 172.21.15.83:0/2025580665 shutdown_connections 2024-04-24T09:13:17.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.766+0000 7f3edb400640 1 --2- 172.21.15.83:0/2025580665 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f3eb40a3f20 0x7f3eb40a63e0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:17.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.766+0000 7f3edb400640 1 --2- 172.21.15.83:0/2025580665 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ed4111170 0x7f3ed410ed30 unknown :-1 s=CLOSED pgs=2600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:17.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.766+0000 7f3edb400640 1 --2- 172.21.15.83:0/2025580665 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3ed4109500 0x7f3ed411c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:17.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.766+0000 7f3edb400640 1 --2- 172.21.15.83:0/2025580665 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3ed4104430 0x7f3ed4110c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:17.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.766+0000 7f3edb400640 1 -- 172.21.15.83:0/2025580665 >> 172.21.15.83:0/2025580665 conn(0x7f3ed40ffcb0 msgr2=0x7f3ed41010c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:17.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.766+0000 7f3edb400640 1 -- 172.21.15.83:0/2025580665 shutdown_connections 2024-04-24T09:13:17.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:17.766+0000 7f3edb400640 1 -- 172.21.15.83:0/2025580665 wait complete. 2024-04-24T09:13:17.789 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T09:13:17.789 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T09:13:18.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:18.130+0000 7f1bce400640 1 -- 172.21.15.83:0/97749671 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1bc00051c0 msgr2=0x7f1bc0005640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:18.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:18.130+0000 7f1bce400640 1 --2- 172.21.15.83:0/97749671 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1bc00051c0 0x7f1bc0005640 secure :-1 s=READY pgs=2605 cs=0 l=1 rev1=1 crypto rx=0x7f1bb4009900 tx=0x7f1bb402a3f0 comp rx=0 tx=0).stop 2024-04-24T09:13:18.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:18.131+0000 7f1bce400640 1 -- 172.21.15.83:0/97749671 shutdown_connections 2024-04-24T09:13:18.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:18.131+0000 7f1bce400640 1 --2- 172.21.15.83:0/97749671 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1bc00986f0 0x7f1bc0098b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:18.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:18.131+0000 7f1bce400640 1 --2- 172.21.15.83:0/97749671 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1bc00051c0 0x7f1bc0005640 unknown :-1 s=CLOSED pgs=2605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:18.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:18.131+0000 7f1bce400640 1 --2- 172.21.15.83:0/97749671 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1bc009ad00 0x7f1bc0004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:18.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:18.131+0000 7f1bce400640 1 -- 172.21.15.83:0/97749671 >> 172.21.15.83:0/97749671 conn(0x7f1bc000a750 msgr2=0x7f1bc000ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:18.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:18.131+0000 7f1bce400640 1 -- 172.21.15.83:0/97749671 shutdown_connections 2024-04-24T09:13:18.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:18.131+0000 7f1bce400640 1 -- 172.21.15.83:0/97749671 wait complete. 2024-04-24T09:13:18.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:18.132+0000 7f1bce400640 1 Processor -- start 2024-04-24T09:13:18.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:18.133+0000 7f1bce400640 1 -- start start 2024-04-24T09:13:18.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:18.133+0000 7f1bce400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1bc00051c0 0x7f1bc0149640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:18.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:18.133+0000 7f1bce400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1bc00986f0 0x7f1bc0149b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:18.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:18.133+0000 7f1bce400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1bc009ad00 0x7f1bc01437b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:18.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:18.133+0000 7f1bce400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1bc00bd4f0 con 0x7f1bc009ad00 2024-04-24T09:13:18.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:18.133+0000 7f1bce400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1bc00bd370 con 0x7f1bc00051c0 2024-04-24T09:13:18.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:18.133+0000 7f1bce400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1bc00bd670 con 0x7f1bc00986f0 2024-04-24T09:13:18.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:18.134+0000 7f1bcd000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1bc00051c0 0x7f1bc0149640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:18.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:18.134+0000 7f1bc7e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1bc00986f0 0x7f1bc0149b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:18.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:18.134+0000 7f1bcd000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1bc00051c0 0x7f1bc0149640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:57098/0 (socket says 172.21.15.83:57098) 2024-04-24T09:13:18.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:18.134+0000 7f1bcd000640 1 -- 172.21.15.83:0/1823097789 learned_addr learned my addr 172.21.15.83:0/1823097789 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:13:18.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:18.134+0000 7f1bcda00640 1 --2- 172.21.15.83:0/1823097789 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1bc009ad00 0x7f1bc01437b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:18.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:18.134+0000 7f1bc7e00640 1 -- 172.21.15.83:0/1823097789 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1bc00051c0 msgr2=0x7f1bc0149640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:18.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:18.134+0000 7f1bc7e00640 1 --2- 172.21.15.83:0/1823097789 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1bc00051c0 0x7f1bc0149640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:18.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:18.134+0000 7f1bc7e00640 1 -- 172.21.15.83:0/1823097789 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1bc009ad00 msgr2=0x7f1bc01437b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:18.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:18.134+0000 7f1bc7e00640 1 --2- 172.21.15.83:0/1823097789 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1bc009ad00 0x7f1bc01437b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:18.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:18.134+0000 7f1bc7e00640 1 -- 172.21.15.83:0/1823097789 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bc0143f80 con 0x7f1bc00986f0 2024-04-24T09:13:18.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:18.135+0000 7f1bc7e00640 1 --2- 172.21.15.83:0/1823097789 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1bc00986f0 0x7f1bc0149b80 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1bb402a9e0 tx=0x7f1bb4003cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:18.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:18.136+0000 7f1bc5600640 1 -- 172.21.15.83:0/1823097789 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bb403f070 con 0x7f1bc00986f0 2024-04-24T09:13:18.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:18.136+0000 7f1bce400640 1 -- 172.21.15.83:0/1823097789 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bc0144270 con 0x7f1bc00986f0 2024-04-24T09:13:18.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:18.136+0000 7f1bce400640 1 -- 172.21.15.83:0/1823097789 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bc0150490 con 0x7f1bc00986f0 2024-04-24T09:13:18.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:18.137+0000 7f1bc5600640 1 -- 172.21.15.83:0/1823097789 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1bb4003e40 con 0x7f1bc00986f0 2024-04-24T09:13:18.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:18.137+0000 7f1bc5600640 1 -- 172.21.15.83:0/1823097789 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bb403f070 con 0x7f1bc00986f0 2024-04-24T09:13:18.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:18.140+0000 7f1bbaa00640 1 -- 172.21.15.83:0/1823097789 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ba4005290 con 0x7f1bc00986f0 2024-04-24T09:13:18.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:18.145+0000 7f1bc5600640 1 -- 172.21.15.83:0/1823097789 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1bb4042070 con 0x7f1bc00986f0 2024-04-24T09:13:18.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:18.147+0000 7f1bc5600640 1 --2- 172.21.15.83:0/1823097789 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f1bb00a3fd0 0x7f1bb00a6490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:18.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:18.147+0000 7f1bc5600640 1 -- 172.21.15.83:0/1823097789 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 9583+0+0 (secure 0 0 0) 0x7f1bb40bb320 con 0x7f1bc00986f0 2024-04-24T09:13:18.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:18.148+0000 7f1bcd000640 1 --2- 172.21.15.83:0/1823097789 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f1bb00a3fd0 0x7f1bb00a6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:18.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:18.149+0000 7f1bcd000640 1 --2- 172.21.15.83:0/1823097789 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f1bb00a3fd0 0x7f1bb00a6490 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f1bc8051000 tx=0x7f1bc806c040 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:18.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:18.150+0000 7f1bc5600640 1 -- 172.21.15.83:0/1823097789 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1bb40bd980 con 0x7f1bc00986f0 2024-04-24T09:13:18.197 INFO:tasks.workunit.client.0.smithi083.stdout: 10 58331 6.62 MB/sec execute 349 sec latency 120.736 ms 2024-04-24T09:13:18.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:13:18 smithi138 ceph-mon[31441]: pgmap v685: 129 pgs: 129 active+clean; 899 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.5 MiB/s wr, 281 op/s 2024-04-24T09:13:18.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:13:18 smithi189 ceph-mon[31791]: pgmap v685: 129 pgs: 129 active+clean; 899 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.5 MiB/s wr, 281 op/s 2024-04-24T09:13:18.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:13:18 smithi083 ceph-mon[27907]: pgmap v685: 129 pgs: 129 active+clean; 899 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.5 MiB/s wr, 281 op/s 2024-04-24T09:13:18.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:18.582+0000 7f1bbaa00640 1 -- 172.21.15.83:0/1823097789 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1ba4005580 con 0x7f1bc00986f0 2024-04-24T09:13:18.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:18.584+0000 7f1bc5600640 1 -- 172.21.15.83:0/1823097789 <== mon.1 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) ==== 94+0+11948 (secure 0 0 0) 0x7f1bb40bd280 con 0x7f1bc00986f0 2024-04-24T09:13:18.584 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T09:13:18.585 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":64,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/358872257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":358872257},{"type":"v1","addr":"172.21.15.83:6835","nonce":358872257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3157070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3157070851},{"type":"v1","addr":"172.21.15.83:6837","nonce":3157070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":64,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T08:48:29.422576+0000","modified":"2024-04-24T09:12:33.175729+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34520,"mds_1":34493,"mds_2":34451,"mds_3":34427,"mds_4":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14658":{"gid":14658,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6839/4085470844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":4085470844},{"type":"v1","addr":"172.21.15.83:6839","nonce":4085470844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14676":{"gid":14676,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6841/2634754210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":2634754210},{"type":"v1","addr":"172.21.15.83:6841","nonce":2634754210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24382":{"gid":24382,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/1859921310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1859921310},{"type":"v1","addr":"172.21.15.189:6833","nonce":1859921310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/2241923197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2241923197},{"type":"v1","addr":"172.21.15.189:6835","nonce":2241923197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.189:6839/2548965336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":2548965336},{"type":"v1","addr":"172.21.15.189:6839","nonce":2548965336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26686":{"gid":26686,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6837/1678817161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":1678817161},{"type":"v1","addr":"172.21.15.189:6837","nonce":1678817161}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.138:6833/1157967658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1157967658},{"type":"v1","addr":"172.21.15.138:6833","nonce":1157967658}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.138:6835/1909219758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1909219758},{"type":"v1","addr":"172.21.15.138:6835","nonce":1909219758}]},"join_fscid":-1,"export_targets":[0,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34493":{"gid":34493,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.138:6837/900566754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":900566754},{"type":"v1","addr":"172.21.15.138:6837","nonce":900566754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34520":{"gid":34520,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.138:6839/2236932393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2236932393},{"type":"v1","addr":"172.21.15.138:6839","nonce":2236932393}]},"join_fscid":-1,"export_targets":[4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34520,"qdb_cluster":[34451,34493,34427,34520,24454]},"id":1}]} 2024-04-24T09:13:18.585 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 64 2024-04-24T09:13:18.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:18.587+0000 7f1bbaa00640 1 -- 172.21.15.83:0/1823097789 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f1bb00a3fd0 msgr2=0x7f1bb00a6490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:18.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:18.587+0000 7f1bbaa00640 1 --2- 172.21.15.83:0/1823097789 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f1bb00a3fd0 0x7f1bb00a6490 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f1bc8051000 tx=0x7f1bc806c040 comp rx=0 tx=0).stop 2024-04-24T09:13:18.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:18.587+0000 7f1bbaa00640 1 -- 172.21.15.83:0/1823097789 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1bc00986f0 msgr2=0x7f1bc0149b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:18.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:18.587+0000 7f1bbaa00640 1 --2- 172.21.15.83:0/1823097789 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1bc00986f0 0x7f1bc0149b80 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1bb402a9e0 tx=0x7f1bb4003cd0 comp rx=0 tx=0).stop 2024-04-24T09:13:18.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:18.587+0000 7f1bbaa00640 1 -- 172.21.15.83:0/1823097789 shutdown_connections 2024-04-24T09:13:18.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:18.587+0000 7f1bbaa00640 1 --2- 172.21.15.83:0/1823097789 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f1bb00a3fd0 0x7f1bb00a6490 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:18.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:18.587+0000 7f1bbaa00640 1 --2- 172.21.15.83:0/1823097789 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1bc009ad00 0x7f1bc01437b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:18.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:18.587+0000 7f1bbaa00640 1 --2- 172.21.15.83:0/1823097789 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1bc00986f0 0x7f1bc0149b80 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:18.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:18.587+0000 7f1bbaa00640 1 --2- 172.21.15.83:0/1823097789 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1bc00051c0 0x7f1bc0149640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:18.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:18.587+0000 7f1bbaa00640 1 -- 172.21.15.83:0/1823097789 >> 172.21.15.83:0/1823097789 conn(0x7f1bc000a750 msgr2=0x7f1bc00976d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:18.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:18.588+0000 7f1bbaa00640 1 -- 172.21.15.83:0/1823097789 shutdown_connections 2024-04-24T09:13:18.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:18.589+0000 7f1bbaa00640 1 -- 172.21.15.83:0/1823097789 wait complete. 2024-04-24T09:13:18.615 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-24T09:13:19.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.004+0000 7fae53200640 1 -- 172.21.15.83:0/925028624 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae541093f0 msgr2=0x7fae54110fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:19.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.004+0000 7fae53200640 1 --2- 172.21.15.83:0/925028624 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae541093f0 0x7fae54110fc0 secure :-1 s=READY pgs=2606 cs=0 l=1 rev1=1 crypto rx=0x7fae4c009ca0 tx=0x7fae4c02a8c0 comp rx=0 tx=0).stop 2024-04-24T09:13:19.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.004+0000 7fae53200640 1 -- 172.21.15.83:0/925028624 shutdown_connections 2024-04-24T09:13:19.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.004+0000 7fae53200640 1 --2- 172.21.15.83:0/925028624 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae541093f0 0x7fae54110fc0 unknown :-1 s=CLOSED pgs=2606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:19.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.004+0000 7fae53200640 1 --2- 172.21.15.83:0/925028624 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae54104cd0 0x7fae54108d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:19.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.004+0000 7fae53200640 1 --2- 172.21.15.83:0/925028624 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fae54104320 0x7fae54104700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:19.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.004+0000 7fae53200640 1 -- 172.21.15.83:0/925028624 >> 172.21.15.83:0/925028624 conn(0x7fae540ffb40 msgr2=0x7fae54101f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:19.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.004+0000 7fae53200640 1 -- 172.21.15.83:0/925028624 shutdown_connections 2024-04-24T09:13:19.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.004+0000 7fae53200640 1 -- 172.21.15.83:0/925028624 wait complete. 2024-04-24T09:13:19.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.005+0000 7fae53200640 1 Processor -- start 2024-04-24T09:13:19.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.006+0000 7fae53200640 1 -- start start 2024-04-24T09:13:19.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.006+0000 7fae53200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fae54104320 0x7fae541a0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:19.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.006+0000 7fae53200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae541093f0 0x7fae541a0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:19.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.006+0000 7fae53200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae5419a6c0 0x7fae5419ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:19.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.006+0000 7fae53200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fae54114360 con 0x7fae5419a6c0 2024-04-24T09:13:19.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.006+0000 7fae53200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fae541141e0 con 0x7fae541093f0 2024-04-24T09:13:19.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.006+0000 7fae53200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fae541144e0 con 0x7fae54104320 2024-04-24T09:13:19.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.007+0000 7fae52800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae5419a6c0 0x7fae5419ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:19.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.007+0000 7fae51400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae541093f0 0x7fae541a0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:19.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.007+0000 7fae52800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae5419a6c0 0x7fae5419ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34998/0 (socket says 172.21.15.83:34998) 2024-04-24T09:13:19.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.007+0000 7fae51400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae541093f0 0x7fae541a0b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:37914/0 (socket says 172.21.15.83:37914) 2024-04-24T09:13:19.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.007+0000 7fae52800640 1 -- 172.21.15.83:0/4019711505 learned_addr learned my addr 172.21.15.83:0/4019711505 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:13:19.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.007+0000 7fae51e00640 1 --2- 172.21.15.83:0/4019711505 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fae54104320 0x7fae541a0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:19.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.007+0000 7fae51400640 1 -- 172.21.15.83:0/4019711505 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fae54104320 msgr2=0x7fae541a0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:19.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.007+0000 7fae51400640 1 --2- 172.21.15.83:0/4019711505 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fae54104320 0x7fae541a0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:19.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.007+0000 7fae51400640 1 -- 172.21.15.83:0/4019711505 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae5419a6c0 msgr2=0x7fae5419ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:19.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.007+0000 7fae51400640 1 --2- 172.21.15.83:0/4019711505 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae5419a6c0 0x7fae5419ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:19.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.007+0000 7fae51400640 1 -- 172.21.15.83:0/4019711505 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae5419b2c0 con 0x7fae541093f0 2024-04-24T09:13:19.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.008+0000 7fae51400640 1 --2- 172.21.15.83:0/4019711505 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae541093f0 0x7fae541a0b40 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fae4000e9c0 tx=0x7fae4000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:19.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.008+0000 7fae4aa00640 1 -- 172.21.15.83:0/4019711505 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae4000ccc0 con 0x7fae541093f0 2024-04-24T09:13:19.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.008+0000 7fae4aa00640 1 -- 172.21.15.83:0/4019711505 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fae400108e0 con 0x7fae541093f0 2024-04-24T09:13:19.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.008+0000 7fae4aa00640 1 -- 172.21.15.83:0/4019711505 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae400187e0 con 0x7fae541093f0 2024-04-24T09:13:19.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.008+0000 7fae53200640 1 -- 172.21.15.83:0/4019711505 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae541a73a0 con 0x7fae541093f0 2024-04-24T09:13:19.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.008+0000 7fae53200640 1 -- 172.21.15.83:0/4019711505 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae541a7750 con 0x7fae541093f0 2024-04-24T09:13:19.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.013+0000 7fae5bc17740 1 Processor -- start 2024-04-24T09:13:19.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.013+0000 7fae5bc17740 1 -- start start 2024-04-24T09:13:19.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.014+0000 7fae5bc17740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556733394b40 0x556733394f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:19.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.014+0000 7fae5bc17740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556733395460 0x55673339f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:19.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.014+0000 7fae5bc17740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55673339fb30 0x5567333a1f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:19.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.014+0000 7fae5bc17740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5567332fa1d0 con 0x556733395460 2024-04-24T09:13:19.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.014+0000 7fae5bc17740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5567332a0450 con 0x556733394b40 2024-04-24T09:13:19.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.014+0000 7fae5bc17740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5567332f6190 con 0x55673339fb30 2024-04-24T09:13:19.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.014+0000 7fae51e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556733395460 0x55673339f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:19.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.014+0000 7fae52800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556733394b40 0x556733394f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:19.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.014+0000 7fae52800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556733394b40 0x556733394f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:37918/0 (socket says 172.21.15.83:37918) 2024-04-24T09:13:19.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.014+0000 7fae52800640 1 -- 172.21.15.83:0/2619552685 learned_addr learned my addr 172.21.15.83:0/2619552685 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:13:19.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.015+0000 7fae52800640 1 -- 172.21.15.83:0/2619552685 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55673339fb30 msgr2=0x5567333a1f20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T09:13:19.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.015+0000 7fae52800640 1 --2- 172.21.15.83:0/2619552685 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55673339fb30 0x5567333a1f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:19.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.015+0000 7fae52800640 1 -- 172.21.15.83:0/2619552685 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556733395460 msgr2=0x55673339f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:19.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.015+0000 7fae52800640 1 --2- 172.21.15.83:0/2619552685 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556733395460 0x55673339f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:19.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.015+0000 7fae52800640 1 -- 172.21.15.83:0/2619552685 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55673335d310 con 0x556733394b40 2024-04-24T09:13:19.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.015+0000 7fae4aa00640 1 -- 172.21.15.83:0/4019711505 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fae4001f020 con 0x7fae541093f0 2024-04-24T09:13:19.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.015+0000 7fae51e00640 1 --2- 172.21.15.83:0/2619552685 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556733395460 0x55673339f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:13:19.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.016+0000 7fae52800640 1 --2- 172.21.15.83:0/2619552685 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556733394b40 0x556733394f20 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fae4c009a20 tx=0x7fae4c005ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d715f4bae897dc2b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:19.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.016+0000 7fae3f400640 1 -- 172.21.15.83:0/2619552685 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae4c031cc0 con 0x556733394b40 2024-04-24T09:13:19.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.017+0000 7fae3f400640 1 -- 172.21.15.83:0/2619552685 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fae4c036040 con 0x556733394b40 2024-04-24T09:13:19.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.017+0000 7fae3f400640 1 -- 172.21.15.83:0/2619552685 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae4c0042a0 con 0x556733394b40 2024-04-24T09:13:19.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.017+0000 7fae4aa00640 1 --2- 172.21.15.83:0/4019711505 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fae380a3de0 0x7fae380a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:19.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.017+0000 7fae4aa00640 1 -- 172.21.15.83:0/4019711505 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 9583+0+0 (secure 0 0 0) 0x7fae400d0d90 con 0x7fae541093f0 2024-04-24T09:13:19.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.018+0000 7fae5bc17740 1 -- 172.21.15.83:0/2619552685 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556733394b40 msgr2=0x556733394f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:19.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.018+0000 7fae5bc17740 1 --2- 172.21.15.83:0/2619552685 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556733394b40 0x556733394f20 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fae4c009a20 tx=0x7fae4c005ef0 comp rx=0 tx=0).stop 2024-04-24T09:13:19.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.018+0000 7fae51e00640 1 --2- 172.21.15.83:0/4019711505 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fae380a3de0 0x7fae380a62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:19.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.018+0000 7fae5bc17740 1 -- 172.21.15.83:0/2619552685 shutdown_connections 2024-04-24T09:13:19.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.018+0000 7fae5bc17740 1 --2- 172.21.15.83:0/2619552685 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55673339fb30 0x5567333a1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:19.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.018+0000 7fae5bc17740 1 --2- 172.21.15.83:0/2619552685 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556733395460 0x55673339f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:19.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.018+0000 7fae5bc17740 1 --2- 172.21.15.83:0/2619552685 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556733394b40 0x556733394f20 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:19.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.018+0000 7fae5bc17740 1 -- 172.21.15.83:0/2619552685 >> 172.21.15.83:0/2619552685 conn(0x556733390e60 msgr2=0x556733391240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:19.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.019+0000 7fae5bc17740 1 -- 172.21.15.83:0/2619552685 shutdown_connections 2024-04-24T09:13:19.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.019+0000 7fae5bc17740 1 -- 172.21.15.83:0/2619552685 wait complete. 2024-04-24T09:13:19.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.019+0000 7fae51e00640 1 --2- 172.21.15.83:0/4019711505 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fae380a3de0 0x7fae380a62a0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x556733396100 tx=0x7fae4400b7b0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:19.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.019+0000 7fae5bc17740 1 -- start start 2024-04-24T09:13:19.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.020+0000 7fae5bc17740 1 Processor -- start 2024-04-24T09:13:19.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.020+0000 7fae5bc17740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556733394b40 0x55673339c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:19.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.020+0000 7fae5bc17740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556733395460 0x55673339cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:19.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.020+0000 7fae5bc17740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55673339fb30 0x55673339d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:19.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.020+0000 7fae5bc17740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5567333ab170 con 0x556733394b40 2024-04-24T09:13:19.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.020+0000 7fae5bc17740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5567332b3810 con 0x556733395460 2024-04-24T09:13:19.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.020+0000 7fae5bc17740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5567333ab2f0 con 0x55673339fb30 2024-04-24T09:13:19.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.020+0000 7fae5bc17740 10 client.0 _ll_register_callbacks cb 0x556733281140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:13:19.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.021+0000 7fae51400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556733395460 0x55673339cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:19.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.021+0000 7fae51400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556733395460 0x55673339cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:37932/0 (socket says 172.21.15.83:37932) 2024-04-24T09:13:19.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.021+0000 7fae51400640 1 -- 172.21.15.83:0/621639436 learned_addr learned my addr 172.21.15.83:0/621639436 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:13:19.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.021+0000 7fae51e00640 1 --2- 172.21.15.83:0/621639436 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556733394b40 0x55673339c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:19.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.021+0000 7fae51400640 1 -- 172.21.15.83:0/621639436 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55673339fb30 msgr2=0x55673339d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:19.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.021+0000 7fae51400640 1 --2- 172.21.15.83:0/621639436 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55673339fb30 0x55673339d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:19.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.021+0000 7fae51400640 1 -- 172.21.15.83:0/621639436 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556733394b40 msgr2=0x55673339c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:19.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.021+0000 7fae51400640 1 --2- 172.21.15.83:0/621639436 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556733394b40 0x55673339c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:19.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.021+0000 7fae51400640 1 -- 172.21.15.83:0/621639436 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567332c43f0 con 0x556733395460 2024-04-24T09:13:19.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.021+0000 7fae51e00640 1 --2- 172.21.15.83:0/621639436 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556733394b40 0x55673339c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:13:19.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.022+0000 7fae51400640 1 --2- 172.21.15.83:0/621639436 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556733395460 0x55673339cea0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fae4009c8a0 tx=0x7fae40010ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d484512123cdef3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:19.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.022+0000 7fae33e00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:13:19.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.022+0000 7fae33e00640 1 -- 172.21.15.83:0/621639436 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae400a0cc0 con 0x556733395460 2024-04-24T09:13:19.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.022+0000 7fae5bc17740 1 -- 172.21.15.83:0/621639436 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae4c036040 con 0x556733395460 2024-04-24T09:13:19.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.022+0000 7fae33e00640 1 -- 172.21.15.83:0/621639436 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9583+0+0 (secure 0 0 0) 0x7fae400870f0 con 0x556733395460 2024-04-24T09:13:19.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.022+0000 7fae33e00640 1 -- 172.21.15.83:0/621639436 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7fae400872f0 con 0x556733395460 2024-04-24T09:13:19.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.022+0000 7fae5bc17740 10 client.40028 fetch_fsmap learned FSMap version 64 2024-04-24T09:13:19.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.022+0000 7fae5bc17740 1 -- 172.21.15.83:0/621639436 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55673300c3f0 con 0x556733395460 2024-04-24T09:13:19.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.023+0000 7fae33e00640 1 -- 172.21.15.83:0/621639436 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fae4c036040 con 0x556733395460 2024-04-24T09:13:19.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.023+0000 7fae33e00640 1 -- 172.21.15.83:0/621639436 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 64) ==== 8374+0+0 (secure 0 0 0) 0x7fae4009bd60 con 0x556733395460 2024-04-24T09:13:19.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.023+0000 7fae5bc17740 10 client.40028 fetch_fsmap finished waiting for FSMap version 64 2024-04-24T09:13:19.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.023+0000 7fae5bc17740 10 client.40028 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-24T09:13:19.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.023+0000 7fae5bc17740 20 client.40028 populate_metadata read hostname 'smithi083' 2024-04-24T09:13:19.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.023+0000 7fae5bc17740 1 --2- 172.21.15.83:0/621639436 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x5567333b85e0 0x5567333baa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:19.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.023+0000 7fae5bc17740 4 client.40028 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:13:19.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.023+0000 7fae5bc17740 1 -- 172.21.15.83:0/621639436 --> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556733307bf0 con 0x5567333b85e0 2024-04-24T09:13:19.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.024+0000 7fae51e00640 1 --2- 172.21.15.83:0/621639436 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x5567333b85e0 0x5567333baa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:19.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.025+0000 7fae33e00640 10 client.40028 ms_handle_connect on v2:172.21.15.189:6838/2548965336 2024-04-24T09:13:19.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.025+0000 7fae51e00640 1 --2- 172.21.15.83:0/621639436 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x5567333b85e0 0x5567333baa40 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e1872e5dddd0f8ff server_cookie=7819219d1378d1d1 in_seq=0 out_seq=0 2024-04-24T09:13:19.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.028+0000 7fae33e00640 1 -- 172.21.15.83:0/621639436 <== mds.4 v2:172.21.15.189:6838/2548965336 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556733307bf0 con 0x5567333b85e0 2024-04-24T09:13:19.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.028+0000 7fae33e00640 10 client.40028 handle_command_reply: tid=0 2024-04-24T09:13:19.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.028+0000 7fae5bc17740 1 client.40028 shutdown 2024-04-24T09:13:19.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.028+0000 7fae5bc17740 1 -- 172.21.15.83:0/621639436 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556733395460 msgr2=0x55673339cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:19.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.028+0000 7fae5bc17740 1 --2- 172.21.15.83:0/621639436 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556733395460 0x55673339cea0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fae4009c8a0 tx=0x7fae40010ec0 comp rx=0 tx=0).stop 2024-04-24T09:13:19.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.029+0000 7fae5bc17740 1 -- 172.21.15.83:0/621639436 shutdown_connections 2024-04-24T09:13:19.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.029+0000 7fae5bc17740 1 --2- 172.21.15.83:0/621639436 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x5567333b85e0 0x5567333baa40 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:19.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.029+0000 7fae5bc17740 1 --2- 172.21.15.83:0/621639436 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55673339fb30 0x55673339d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:19.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.029+0000 7fae5bc17740 1 --2- 172.21.15.83:0/621639436 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556733395460 0x55673339cea0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:19.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.029+0000 7fae5bc17740 1 --2- 172.21.15.83:0/621639436 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556733394b40 0x55673339c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:19.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.029+0000 7fae5bc17740 1 -- 172.21.15.83:0/621639436 >> 172.21.15.83:0/621639436 conn(0x556733390e60 msgr2=0x556733396b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:19.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.029+0000 7fae33e00640 0 client.40028 ms_handle_reset on v2:172.21.15.189:6838/2548965336 2024-04-24T09:13:19.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.029+0000 7fae5bc17740 1 -- 172.21.15.83:0/621639436 shutdown_connections 2024-04-24T09:13:19.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.029+0000 7fae5bc17740 1 -- 172.21.15.83:0/621639436 wait complete. 2024-04-24T09:13:19.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.029+0000 7fae5bc17740 20 client.40028 trim_cache size 0 max 16384 2024-04-24T09:13:19.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.053+0000 7fae5bc17740 1 Processor -- start 2024-04-24T09:13:19.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.053+0000 7fae5bc17740 1 -- start start 2024-04-24T09:13:19.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.053+0000 7fae5bc17740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556733394b40 0x5567333972e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:19.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.053+0000 7fae5bc17740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556733395460 0x55673339d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:19.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.053+0000 7fae5bc17740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55673339fb30 0x5567333c0310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:19.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.053+0000 7fae5bc17740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fae541ad240 con 0x556733395460 2024-04-24T09:13:19.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.053+0000 7fae5bc17740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fae541aee10 con 0x556733394b40 2024-04-24T09:13:19.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.053+0000 7fae5bc17740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fae541ad910 con 0x55673339fb30 2024-04-24T09:13:19.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.054+0000 7fae51e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556733394b40 0x5567333972e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:19.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.054+0000 7fae51e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556733394b40 0x5567333972e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:37944/0 (socket says 172.21.15.83:37944) 2024-04-24T09:13:19.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.054+0000 7fae51e00640 1 -- 172.21.15.83:0/3853201586 learned_addr learned my addr 172.21.15.83:0/3853201586 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:13:19.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.054+0000 7fae51400640 1 --2- 172.21.15.83:0/3853201586 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556733395460 0x55673339d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:19.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.054+0000 7fae51e00640 1 -- 172.21.15.83:0/3853201586 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55673339fb30 msgr2=0x5567333c0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:19.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.054+0000 7fae51e00640 1 --2- 172.21.15.83:0/3853201586 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55673339fb30 0x5567333c0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:19.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.054+0000 7fae51e00640 1 -- 172.21.15.83:0/3853201586 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556733395460 msgr2=0x55673339d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:19.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.054+0000 7fae51e00640 1 --2- 172.21.15.83:0/3853201586 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556733395460 0x55673339d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:19.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.054+0000 7fae51e00640 1 -- 172.21.15.83:0/3853201586 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567330d67d0 con 0x556733394b40 2024-04-24T09:13:19.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.054+0000 7fae51400640 1 --2- 172.21.15.83:0/3853201586 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556733395460 0x55673339d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:13:19.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.055+0000 7fae51e00640 1 --2- 172.21.15.83:0/3853201586 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556733394b40 0x5567333972e0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fae44011e60 tx=0x7fae44014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ad0314b69699822 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:19.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.055+0000 7fae3ea00640 1 -- 172.21.15.83:0/3853201586 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae44028330 con 0x556733394b40 2024-04-24T09:13:19.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.055+0000 7fae3ea00640 1 -- 172.21.15.83:0/3853201586 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fae4401f420 con 0x556733394b40 2024-04-24T09:13:19.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.055+0000 7fae3ea00640 1 -- 172.21.15.83:0/3853201586 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae4401e670 con 0x556733394b40 2024-04-24T09:13:19.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.056+0000 7fae5bc17740 1 -- 172.21.15.83:0/3853201586 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556733394b40 msgr2=0x5567333972e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:19.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.056+0000 7fae5bc17740 1 --2- 172.21.15.83:0/3853201586 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556733394b40 0x5567333972e0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fae44011e60 tx=0x7fae44014000 comp rx=0 tx=0).stop 2024-04-24T09:13:19.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.056+0000 7fae5bc17740 1 -- 172.21.15.83:0/3853201586 shutdown_connections 2024-04-24T09:13:19.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.056+0000 7fae5bc17740 1 --2- 172.21.15.83:0/3853201586 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55673339fb30 0x5567333c0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:19.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.056+0000 7fae5bc17740 1 --2- 172.21.15.83:0/3853201586 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556733395460 0x55673339d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:19.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.056+0000 7fae5bc17740 1 --2- 172.21.15.83:0/3853201586 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556733394b40 0x5567333972e0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:19.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.056+0000 7fae5bc17740 1 -- 172.21.15.83:0/3853201586 >> 172.21.15.83:0/3853201586 conn(0x556733390e60 msgr2=0x55673338f6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:19.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.057+0000 7fae5bc17740 1 -- 172.21.15.83:0/3853201586 shutdown_connections 2024-04-24T09:13:19.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.057+0000 7fae5bc17740 1 -- 172.21.15.83:0/3853201586 wait complete. 2024-04-24T09:13:19.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.057+0000 7fae5bc17740 1 -- start start 2024-04-24T09:13:19.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.057+0000 7fae5bc17740 1 Processor -- start 2024-04-24T09:13:19.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.058+0000 7fae5bc17740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556733394b40 0x5567333b0840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:19.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.058+0000 7fae5bc17740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556733395460 0x5567333a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:19.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.058+0000 7fae5bc17740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55673339fb30 0x5567333a0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:19.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.058+0000 7fae5bc17740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fae541ae710 con 0x556733395460 2024-04-24T09:13:19.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.058+0000 7fae5bc17740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fae541ae040 con 0x55673339fb30 2024-04-24T09:13:19.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.058+0000 7fae5bc17740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fae541ab260 con 0x556733394b40 2024-04-24T09:13:19.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.058+0000 7fae5bc17740 10 client.0 _ll_register_callbacks cb 0x7fae40098bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:13:19.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.058+0000 7fae51400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556733395460 0x5567333a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:19.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.058+0000 7fae52800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55673339fb30 0x5567333a0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:19.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.058+0000 7fae51400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556733395460 0x5567333a05f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35020/0 (socket says 172.21.15.83:35020) 2024-04-24T09:13:19.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.059+0000 7fae51400640 1 -- 172.21.15.83:0/176497680 learned_addr learned my addr 172.21.15.83:0/176497680 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:13:19.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.059+0000 7fae52800640 1 -- 172.21.15.83:0/176497680 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556733394b40 msgr2=0x5567333b0840 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-24T09:13:19.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.059+0000 7fae52800640 1 --2- 172.21.15.83:0/176497680 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556733394b40 0x5567333b0840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:19.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.059+0000 7fae52800640 1 -- 172.21.15.83:0/176497680 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556733395460 msgr2=0x5567333a05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:19.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.059+0000 7fae52800640 1 --2- 172.21.15.83:0/176497680 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556733395460 0x5567333a05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:19.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.059+0000 7fae52800640 1 -- 172.21.15.83:0/176497680 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567333a12a0 con 0x55673339fb30 2024-04-24T09:13:19.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.059+0000 7fae51400640 1 --2- 172.21.15.83:0/176497680 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556733395460 0x5567333a05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:13:19.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.060+0000 7fae52800640 1 --2- 172.21.15.83:0/176497680 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55673339fb30 0x5567333a0b30 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fae4c005e30 tx=0x7fae4c009800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c5f58975de6773b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:19.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.060+0000 7fae33e00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:13:19.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.060+0000 7fae33e00640 1 -- 172.21.15.83:0/176497680 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae4c031450 con 0x55673339fb30 2024-04-24T09:13:19.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.060+0000 7fae33e00640 1 -- 172.21.15.83:0/176497680 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9583+0+0 (secure 0 0 0) 0x7fae4c0315f0 con 0x55673339fb30 2024-04-24T09:13:19.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.060+0000 7fae5bc17740 1 -- 172.21.15.83:0/176497680 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae4401f420 con 0x55673339fb30 2024-04-24T09:13:19.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.061+0000 7fae33e00640 1 -- 172.21.15.83:0/176497680 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7fae4c057050 con 0x55673339fb30 2024-04-24T09:13:19.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.061+0000 7fae5bc17740 10 client.40034 fetch_fsmap learned FSMap version 64 2024-04-24T09:13:19.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.061+0000 7fae5bc17740 1 -- 172.21.15.83:0/176497680 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55673342fb90 con 0x55673339fb30 2024-04-24T09:13:19.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.061+0000 7fae33e00640 1 -- 172.21.15.83:0/176497680 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fae4401f420 con 0x55673339fb30 2024-04-24T09:13:19.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.061+0000 7fae33e00640 1 -- 172.21.15.83:0/176497680 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 64) ==== 8374+0+0 (secure 0 0 0) 0x7fae4c004520 con 0x55673339fb30 2024-04-24T09:13:19.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.061+0000 7fae5bc17740 10 client.40034 fetch_fsmap finished waiting for FSMap version 64 2024-04-24T09:13:19.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.061+0000 7fae5bc17740 10 client.40034 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-24T09:13:19.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.061+0000 7fae5bc17740 20 client.40034 populate_metadata read hostname 'smithi083' 2024-04-24T09:13:19.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.062+0000 7fae5bc17740 1 --2- 172.21.15.83:0/176497680 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55673342fe20 0x556733432210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:19.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.062+0000 7fae5bc17740 4 client.40034 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:13:19.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.062+0000 7fae5bc17740 1 -- 172.21.15.83:0/176497680 --> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556733301a60 con 0x55673342fe20 2024-04-24T09:13:19.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.062+0000 7fae51e00640 1 --2- 172.21.15.83:0/176497680 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55673342fe20 0x556733432210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:19.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.063+0000 7fae33e00640 10 client.40034 ms_handle_connect on v2:172.21.15.189:6838/2548965336 2024-04-24T09:13:19.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.063+0000 7fae51e00640 1 --2- 172.21.15.83:0/176497680 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55673342fe20 0x556733432210 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=bb119df696efdb7a server_cookie=a3a85981f47a1aa4 in_seq=0 out_seq=0 2024-04-24T09:13:19.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.064+0000 7fae33e00640 1 -- 172.21.15.83:0/176497680 <== mds.4 v2:172.21.15.189:6838/2548965336 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556733301a60 con 0x55673342fe20 2024-04-24T09:13:19.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.064+0000 7fae33e00640 10 client.40034 handle_command_reply: tid=0 2024-04-24T09:13:19.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.065+0000 7fae5bc17740 1 client.40034 shutdown 2024-04-24T09:13:19.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.065+0000 7fae5bc17740 1 -- 172.21.15.83:0/176497680 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55673339fb30 msgr2=0x5567333a0b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:19.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.065+0000 7fae5bc17740 1 --2- 172.21.15.83:0/176497680 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55673339fb30 0x5567333a0b30 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fae4c005e30 tx=0x7fae4c009800 comp rx=0 tx=0).stop 2024-04-24T09:13:19.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.066+0000 7fae5bc17740 1 -- 172.21.15.83:0/176497680 shutdown_connections 2024-04-24T09:13:19.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.066+0000 7fae5bc17740 1 --2- 172.21.15.83:0/176497680 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55673342fe20 0x556733432210 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:19.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.066+0000 7fae5bc17740 1 --2- 172.21.15.83:0/176497680 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55673339fb30 0x5567333a0b30 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:19.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.066+0000 7fae5bc17740 1 --2- 172.21.15.83:0/176497680 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556733395460 0x5567333a05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:19.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.066+0000 7fae5bc17740 1 --2- 172.21.15.83:0/176497680 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556733394b40 0x5567333b0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:19.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.066+0000 7fae5bc17740 1 -- 172.21.15.83:0/176497680 >> 172.21.15.83:0/176497680 conn(0x556733390e60 msgr2=0x55673338ff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:19.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.066+0000 7fae33e00640 0 client.40034 ms_handle_reset on v2:172.21.15.189:6838/2548965336 2024-04-24T09:13:19.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.066+0000 7fae5bc17740 1 -- 172.21.15.83:0/176497680 shutdown_connections 2024-04-24T09:13:19.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.066+0000 7fae5bc17740 1 -- 172.21.15.83:0/176497680 wait complete. 2024-04-24T09:13:19.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.066+0000 7fae5bc17740 20 client.40034 trim_cache size 0 max 16384 2024-04-24T09:13:19.067 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:13:19.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.067+0000 7fae53200640 1 -- 172.21.15.83:0/4019711505 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fae380a3de0 msgr2=0x7fae380a62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:19.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.067+0000 7fae53200640 1 --2- 172.21.15.83:0/4019711505 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fae380a3de0 0x7fae380a62a0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x556733396100 tx=0x7fae4400b7b0 comp rx=0 tx=0).stop 2024-04-24T09:13:19.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.067+0000 7fae53200640 1 -- 172.21.15.83:0/4019711505 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae541093f0 msgr2=0x7fae541a0b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:19.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.067+0000 7fae53200640 1 --2- 172.21.15.83:0/4019711505 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae541093f0 0x7fae541a0b40 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fae4000e9c0 tx=0x7fae4000ee80 comp rx=0 tx=0).stop 2024-04-24T09:13:19.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.067+0000 7fae53200640 1 -- 172.21.15.83:0/4019711505 shutdown_connections 2024-04-24T09:13:19.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.067+0000 7fae53200640 1 --2- 172.21.15.83:0/4019711505 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fae380a3de0 0x7fae380a62a0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:19.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.067+0000 7fae53200640 1 --2- 172.21.15.83:0/4019711505 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae5419a6c0 0x7fae5419ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:19.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.067+0000 7fae53200640 1 --2- 172.21.15.83:0/4019711505 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae541093f0 0x7fae541a0b40 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:19.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.067+0000 7fae53200640 1 --2- 172.21.15.83:0/4019711505 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fae54104320 0x7fae541a0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:19.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.067+0000 7fae53200640 1 -- 172.21.15.83:0/4019711505 >> 172.21.15.83:0/4019711505 conn(0x7fae540ffb40 msgr2=0x7fae54100e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:19.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.068+0000 7fae53200640 1 -- 172.21.15.83:0/4019711505 shutdown_connections 2024-04-24T09:13:19.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.068+0000 7fae53200640 1 -- 172.21.15.83:0/4019711505 wait complete. 2024-04-24T09:13:19.092 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-24T09:13:19.197 INFO:tasks.workunit.client.0.smithi083.stdout: 10 58419 6.60 MB/sec execute 350 sec latency 103.913 ms 2024-04-24T09:13:19.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.350+0000 7f4eff200640 1 -- 172.21.15.83:0/3783455245 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f00101000 msgr2=0x7f4f001013e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:19.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.350+0000 7f4eff200640 1 --2- 172.21.15.83:0/3783455245 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f00101000 0x7f4f001013e0 secure :-1 s=READY pgs=2607 cs=0 l=1 rev1=1 crypto rx=0x7f4ef0009900 tx=0x7f4ef002a410 comp rx=0 tx=0).stop 2024-04-24T09:13:19.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.350+0000 7f4eff200640 1 -- 172.21.15.83:0/3783455245 shutdown_connections 2024-04-24T09:13:19.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.350+0000 7f4eff200640 1 --2- 172.21.15.83:0/3783455245 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4f001060d0 0x7f4f0010dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:19.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.350+0000 7f4eff200640 1 --2- 172.21.15.83:0/3783455245 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4f001019b0 0x7f4f001059f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:19.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.350+0000 7f4eff200640 1 --2- 172.21.15.83:0/3783455245 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f00101000 0x7f4f001013e0 unknown :-1 s=CLOSED pgs=2607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:19.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.350+0000 7f4eff200640 1 -- 172.21.15.83:0/3783455245 >> 172.21.15.83:0/3783455245 conn(0x7f4f000fc880 msgr2=0x7f4f000feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:19.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.351+0000 7f4eff200640 1 -- 172.21.15.83:0/3783455245 shutdown_connections 2024-04-24T09:13:19.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.351+0000 7f4eff200640 1 -- 172.21.15.83:0/3783455245 wait complete. 2024-04-24T09:13:19.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.352+0000 7f4eff200640 1 Processor -- start 2024-04-24T09:13:19.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.352+0000 7f4eff200640 1 -- start start 2024-04-24T09:13:19.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.352+0000 7f4eff200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4f00101000 0x7f4f001a05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:19.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.352+0000 7f4eff200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4f001060d0 0x7f4f001a0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:19.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.352+0000 7f4eff200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f0019a670 0x7f4f0019aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:19.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.353+0000 7f4eff200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4f00112170 con 0x7f4f0019a670 2024-04-24T09:13:19.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.353+0000 7f4eff200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4f00111ff0 con 0x7f4f00101000 2024-04-24T09:13:19.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.353+0000 7f4eff200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4f001122f0 con 0x7f4f001060d0 2024-04-24T09:13:19.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.353+0000 7f4efe800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f0019a670 0x7f4f0019aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:19.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.353+0000 7f4efe800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f0019a670 0x7f4f0019aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35048/0 (socket says 172.21.15.83:35048) 2024-04-24T09:13:19.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.353+0000 7f4efe800640 1 -- 172.21.15.83:0/2880740898 learned_addr learned my addr 172.21.15.83:0/2880740898 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:13:19.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.353+0000 7f4efde00640 1 --2- 172.21.15.83:0/2880740898 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4f00101000 0x7f4f001a05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:19.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.353+0000 7f4efd400640 1 --2- 172.21.15.83:0/2880740898 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4f001060d0 0x7f4f001a0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:19.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.354+0000 7f4efe800640 1 -- 172.21.15.83:0/2880740898 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4f001060d0 msgr2=0x7f4f001a0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:19.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.354+0000 7f4efe800640 1 --2- 172.21.15.83:0/2880740898 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4f001060d0 0x7f4f001a0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:19.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.354+0000 7f4efe800640 1 -- 172.21.15.83:0/2880740898 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4f00101000 msgr2=0x7f4f001a05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:19.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.354+0000 7f4efe800640 1 --2- 172.21.15.83:0/2880740898 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4f00101000 0x7f4f001a05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:19.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.354+0000 7f4efe800640 1 -- 172.21.15.83:0/2880740898 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f0019b270 con 0x7f4f0019a670 2024-04-24T09:13:19.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.354+0000 7f4efe800640 1 --2- 172.21.15.83:0/2880740898 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f0019a670 0x7f4f0019aaf0 secure :-1 s=READY pgs=2608 cs=0 l=1 rev1=1 crypto rx=0x7f4ef800cce0 tx=0x7f4ef800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:19.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.354+0000 7f4ef6a00640 1 -- 172.21.15.83:0/2880740898 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef800f450 con 0x7f4f0019a670 2024-04-24T09:13:19.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.354+0000 7f4ef6a00640 1 -- 172.21.15.83:0/2880740898 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4ef800faa0 con 0x7f4f0019a670 2024-04-24T09:13:19.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.354+0000 7f4eff200640 1 -- 172.21.15.83:0/2880740898 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f001a7350 con 0x7f4f0019a670 2024-04-24T09:13:19.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.355+0000 7f4ef6a00640 1 -- 172.21.15.83:0/2880740898 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef8018b70 con 0x7f4f0019a670 2024-04-24T09:13:19.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.355+0000 7f4eff200640 1 -- 172.21.15.83:0/2880740898 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f001a7750 con 0x7f4f0019a670 2024-04-24T09:13:19.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.359+0000 7f4f0798f740 1 Processor -- start 2024-04-24T09:13:19.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.359+0000 7f4ef6a00640 1 -- 172.21.15.83:0/2880740898 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4ef8018d10 con 0x7f4f0019a670 2024-04-24T09:13:19.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.360+0000 7f4f0798f740 1 -- start start 2024-04-24T09:13:19.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.360+0000 7f4f0798f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a070bdbb40 0x55a070bdbf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:19.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.360+0000 7f4f0798f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a070bdc460 0x55a070be65f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:19.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.360+0000 7f4f0798f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a070be6b30 0x55a070be8f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:19.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.360+0000 7f4f0798f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a070b411d0 con 0x55a070be6b30 2024-04-24T09:13:19.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.360+0000 7f4f0798f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a070ae7450 con 0x55a070bdc460 2024-04-24T09:13:19.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.360+0000 7f4f0798f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a070b3d190 con 0x55a070bdbb40 2024-04-24T09:13:19.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.361+0000 7f4efd400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a070be6b30 0x55a070be8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:19.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.361+0000 7f4efd400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a070be6b30 0x55a070be8f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35062/0 (socket says 172.21.15.83:35062) 2024-04-24T09:13:19.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.361+0000 7f4efd400640 1 -- 172.21.15.83:0/1785372202 learned_addr learned my addr 172.21.15.83:0/1785372202 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:13:19.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.361+0000 7f4efd400640 1 --2- 172.21.15.83:0/1785372202 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a070bdbb40 0x55a070bdbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:19.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.361+0000 7f4efde00640 1 --2- 172.21.15.83:0/1785372202 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a070bdc460 0x55a070be65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:19.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.361+0000 7f4efd400640 1 -- 172.21.15.83:0/1785372202 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a070bdbb40 msgr2=0x55a070bdbf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:19.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.361+0000 7f4efd400640 1 --2- 172.21.15.83:0/1785372202 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a070bdbb40 0x55a070bdbf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:19.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.361+0000 7f4efd400640 1 -- 172.21.15.83:0/1785372202 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a070bdc460 msgr2=0x55a070be65f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:19.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.361+0000 7f4efd400640 1 --2- 172.21.15.83:0/1785372202 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a070bdc460 0x55a070be65f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:19.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.361+0000 7f4efd400640 1 -- 172.21.15.83:0/1785372202 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a070ba4310 con 0x55a070be6b30 2024-04-24T09:13:19.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.361+0000 7f4efde00640 1 --2- 172.21.15.83:0/1785372202 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a070bdc460 0x55a070be65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:13:19.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.362+0000 7f4efd400640 1 --2- 172.21.15.83:0/1785372202 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a070be6b30 0x55a070be8f20 secure :-1 s=READY pgs=2609 cs=0 l=1 rev1=1 crypto rx=0x7f4eec0125a0 tx=0x7f4eec0096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28f5aa240d1447b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:19.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.362+0000 7f4ef6a00640 1 --2- 172.21.15.83:0/2880740898 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f4ee40a3da0 0x7f4ee40a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:19.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.362+0000 7f4eeb000640 1 -- 172.21.15.83:0/1785372202 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eec010710 con 0x55a070be6b30 2024-04-24T09:13:19.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.362+0000 7f4eeb000640 1 -- 172.21.15.83:0/1785372202 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4eec010d60 con 0x55a070be6b30 2024-04-24T09:13:19.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.362+0000 7f4eeb000640 1 -- 172.21.15.83:0/1785372202 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eec01dd00 con 0x55a070be6b30 2024-04-24T09:13:19.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.362+0000 7f4ef6a00640 1 -- 172.21.15.83:0/2880740898 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 9583+0+0 (secure 0 0 0) 0x7f4ef80d0fe0 con 0x7f4f0019a670 2024-04-24T09:13:19.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.363+0000 7f4efde00640 1 --2- 172.21.15.83:0/2880740898 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f4ee40a3da0 0x7f4ee40a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:19.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.363+0000 7f4f0798f740 1 -- 172.21.15.83:0/1785372202 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a070be6b30 msgr2=0x55a070be8f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:19.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.363+0000 7f4f0798f740 1 --2- 172.21.15.83:0/1785372202 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a070be6b30 0x55a070be8f20 secure :-1 s=READY pgs=2609 cs=0 l=1 rev1=1 crypto rx=0x7f4eec0125a0 tx=0x7f4eec0096c0 comp rx=0 tx=0).stop 2024-04-24T09:13:19.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.364+0000 7f4f0798f740 1 -- 172.21.15.83:0/1785372202 shutdown_connections 2024-04-24T09:13:19.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.364+0000 7f4f0798f740 1 --2- 172.21.15.83:0/1785372202 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a070be6b30 0x55a070be8f20 unknown :-1 s=CLOSED pgs=2609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:19.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.364+0000 7f4f0798f740 1 --2- 172.21.15.83:0/1785372202 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a070bdc460 0x55a070be65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:19.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.364+0000 7f4f0798f740 1 --2- 172.21.15.83:0/1785372202 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a070bdbb40 0x55a070bdbf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:19.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.364+0000 7f4f0798f740 1 -- 172.21.15.83:0/1785372202 >> 172.21.15.83:0/1785372202 conn(0x55a070bd7e60 msgr2=0x55a070bd8240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:19.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.364+0000 7f4efde00640 1 --2- 172.21.15.83:0/2880740898 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f4ee40a3da0 0x7f4ee40a6260 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f4ef0000c00 tx=0x7f4ef0005a20 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:19.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.364+0000 7f4f0798f740 1 -- 172.21.15.83:0/1785372202 shutdown_connections 2024-04-24T09:13:19.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.364+0000 7f4f0798f740 1 -- 172.21.15.83:0/1785372202 wait complete. 2024-04-24T09:13:19.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.365+0000 7f4f0798f740 1 -- start start 2024-04-24T09:13:19.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.365+0000 7f4f0798f740 1 Processor -- start 2024-04-24T09:13:19.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.366+0000 7f4f0798f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a070bdbb40 0x55a070be3910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:19.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.367+0000 7f4f0798f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a070bdc460 0x55a070be3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:19.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.367+0000 7f4f0798f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a070be6b30 0x55a070be4390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:19.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.367+0000 7f4f0798f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a070be99c0 con 0x55a070bdbb40 2024-04-24T09:13:19.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.367+0000 7f4f0798f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a070afa810 con 0x55a070be6b30 2024-04-24T09:13:19.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.367+0000 7f4f0798f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a070be9b40 con 0x55a070bdc460 2024-04-24T09:13:19.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.367+0000 7f4f0798f740 10 client.0 _ll_register_callbacks cb 0x55a070ac8140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:13:19.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.368+0000 7f4efe800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a070be6b30 0x55a070be4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:19.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.368+0000 7f4efe800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a070be6b30 0x55a070be4390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:37988/0 (socket says 172.21.15.83:37988) 2024-04-24T09:13:19.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.368+0000 7f4efe800640 1 -- 172.21.15.83:0/1328449282 learned_addr learned my addr 172.21.15.83:0/1328449282 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:13:19.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.368+0000 7f4efd400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a070bdc460 0x55a070be3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:19.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.368+0000 7f4efe800640 1 -- 172.21.15.83:0/1328449282 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a070bdc460 msgr2=0x55a070be3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:19.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.368+0000 7f4efe800640 1 --2- 172.21.15.83:0/1328449282 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a070bdc460 0x55a070be3e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:19.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.368+0000 7f4efe800640 1 -- 172.21.15.83:0/1328449282 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a070bdbb40 msgr2=0x55a070be3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:19.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.368+0000 7f4efe800640 1 --2- 172.21.15.83:0/1328449282 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a070bdbb40 0x55a070be3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:19.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.368+0000 7f4efe800640 1 -- 172.21.15.83:0/1328449282 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a070b0b3f0 con 0x55a070be6b30 2024-04-24T09:13:19.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.369+0000 7f4efe800640 1 --2- 172.21.15.83:0/1328449282 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a070be6b30 0x55a070be4390 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f4ef80a52a0 tx=0x7f4ef809c580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2ea2adcaba291a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:19.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.369+0000 7f4edf400640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:13:19.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.369+0000 7f4edf400640 1 -- 172.21.15.83:0/1328449282 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef80b6060 con 0x55a070be6b30 2024-04-24T09:13:19.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.370+0000 7f4edf400640 1 -- 172.21.15.83:0/1328449282 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9583+0+0 (secure 0 0 0) 0x7f4ef800fc50 con 0x55a070be6b30 2024-04-24T09:13:19.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.370+0000 7f4f0798f740 1 -- 172.21.15.83:0/1328449282 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4eec010d60 con 0x55a070be6b30 2024-04-24T09:13:19.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.370+0000 7f4edf400640 1 -- 172.21.15.83:0/1328449282 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f4ef80a0f10 con 0x55a070be6b30 2024-04-24T09:13:19.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.371+0000 7f4f0798f740 10 client.40046 fetch_fsmap learned FSMap version 64 2024-04-24T09:13:19.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.371+0000 7f4f0798f740 1 -- 172.21.15.83:0/1328449282 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55a0708533f0 con 0x55a070be6b30 2024-04-24T09:13:19.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.371+0000 7f4edf400640 1 -- 172.21.15.83:0/1328449282 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4eec010d60 con 0x55a070be6b30 2024-04-24T09:13:19.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.371+0000 7f4edf400640 1 -- 172.21.15.83:0/1328449282 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 64) ==== 8374+0+0 (secure 0 0 0) 0x7f4ef809bfb0 con 0x55a070be6b30 2024-04-24T09:13:19.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.372+0000 7f4f0798f740 10 client.40046 fetch_fsmap finished waiting for FSMap version 64 2024-04-24T09:13:19.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.372+0000 7f4f0798f740 10 client.40046 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-24T09:13:19.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.372+0000 7f4f0798f740 20 client.40046 populate_metadata read hostname 'smithi083' 2024-04-24T09:13:19.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.372+0000 7f4f0798f740 1 --2- 172.21.15.83:0/1328449282 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55a070bff600 0x55a070c019f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:19.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.372+0000 7f4f0798f740 4 client.40046 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:13:19.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.372+0000 7f4f0798f740 1 -- 172.21.15.83:0/1328449282 --> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a070b4ebf0 con 0x55a070bff600 2024-04-24T09:13:19.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.372+0000 7f4efde00640 1 --2- 172.21.15.83:0/1328449282 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55a070bff600 0x55a070c019f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:19.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.373+0000 7f4edf400640 10 client.40046 ms_handle_connect on v2:172.21.15.138:6832/1157967658 2024-04-24T09:13:19.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.373+0000 7f4efde00640 1 --2- 172.21.15.83:0/1328449282 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55a070bff600 0x55a070c019f0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d925408954ce7379 server_cookie=51f8d2380580c6de in_seq=0 out_seq=0 2024-04-24T09:13:19.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.375+0000 7f4edf400640 1 -- 172.21.15.83:0/1328449282 <== mds.3 v2:172.21.15.138:6832/1157967658 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a070b4ebf0 con 0x55a070bff600 2024-04-24T09:13:19.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.375+0000 7f4edf400640 10 client.40046 handle_command_reply: tid=0 2024-04-24T09:13:19.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.375+0000 7f4f0798f740 1 client.40046 shutdown 2024-04-24T09:13:19.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.375+0000 7f4f0798f740 1 -- 172.21.15.83:0/1328449282 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a070be6b30 msgr2=0x55a070be4390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:19.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.375+0000 7f4f0798f740 1 --2- 172.21.15.83:0/1328449282 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a070be6b30 0x55a070be4390 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f4ef80a52a0 tx=0x7f4ef809c580 comp rx=0 tx=0).stop 2024-04-24T09:13:19.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.375+0000 7f4f0798f740 1 -- 172.21.15.83:0/1328449282 shutdown_connections 2024-04-24T09:13:19.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.375+0000 7f4f0798f740 1 --2- 172.21.15.83:0/1328449282 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a070be6b30 0x55a070be4390 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:19.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.375+0000 7f4f0798f740 1 --2- 172.21.15.83:0/1328449282 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55a070bff600 0x55a070c019f0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:19.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.375+0000 7f4f0798f740 1 --2- 172.21.15.83:0/1328449282 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a070bdc460 0x55a070be3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:19.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.375+0000 7f4f0798f740 1 --2- 172.21.15.83:0/1328449282 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a070bdbb40 0x55a070be3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:19.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.375+0000 7f4f0798f740 1 -- 172.21.15.83:0/1328449282 >> 172.21.15.83:0/1328449282 conn(0x55a070bd7e60 msgr2=0x55a070bddc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:19.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.375+0000 7f4edf400640 0 client.40046 ms_handle_reset on v2:172.21.15.138:6832/1157967658 2024-04-24T09:13:19.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.375+0000 7f4f0798f740 1 -- 172.21.15.83:0/1328449282 shutdown_connections 2024-04-24T09:13:19.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.375+0000 7f4f0798f740 1 -- 172.21.15.83:0/1328449282 wait complete. 2024-04-24T09:13:19.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.376+0000 7f4f0798f740 20 client.40046 trim_cache size 0 max 16384 2024-04-24T09:13:19.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.403+0000 7f4f0798f740 1 Processor -- start 2024-04-24T09:13:19.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.404+0000 7f4f0798f740 1 -- start start 2024-04-24T09:13:19.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.404+0000 7f4f0798f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a070bdbb40 0x55a070bd6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:19.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.404+0000 7f4f0798f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a070bdc460 0x55a070c072c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:19.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.404+0000 7f4f0798f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a070be6b30 0x55a070c07800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:19.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.404+0000 7f4f0798f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4f001ad150 con 0x55a070bdc460 2024-04-24T09:13:19.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.404+0000 7f4f0798f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4f001aed20 con 0x55a070be6b30 2024-04-24T09:13:19.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.404+0000 7f4f0798f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4f001ad820 con 0x55a070bdbb40 2024-04-24T09:13:19.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.405+0000 7f4efd400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a070bdc460 0x55a070c072c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:19.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.405+0000 7f4efd400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a070bdc460 0x55a070c072c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35084/0 (socket says 172.21.15.83:35084) 2024-04-24T09:13:19.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.405+0000 7f4efd400640 1 -- 172.21.15.83:0/2650338237 learned_addr learned my addr 172.21.15.83:0/2650338237 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:13:19.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.405+0000 7f4efde00640 1 --2- 172.21.15.83:0/2650338237 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a070bdbb40 0x55a070bd6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:19.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.405+0000 7f4efe800640 1 --2- 172.21.15.83:0/2650338237 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a070be6b30 0x55a070c07800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:19.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.405+0000 7f4efe800640 1 -- 172.21.15.83:0/2650338237 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a070bdbb40 msgr2=0x55a070bd6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:19.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.406+0000 7f4efe800640 1 --2- 172.21.15.83:0/2650338237 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a070bdbb40 0x55a070bd6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:19.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.406+0000 7f4efe800640 1 -- 172.21.15.83:0/2650338237 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a070bdc460 msgr2=0x55a070c072c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:19.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.406+0000 7f4efe800640 1 --2- 172.21.15.83:0/2650338237 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a070bdc460 0x55a070c072c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:19.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.406+0000 7f4efe800640 1 -- 172.21.15.83:0/2650338237 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a07091d7d0 con 0x55a070be6b30 2024-04-24T09:13:19.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.406+0000 7f4efd400640 1 --2- 172.21.15.83:0/2650338237 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a070bdc460 0x55a070c072c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:13:19.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.406+0000 7f4efde00640 1 --2- 172.21.15.83:0/2650338237 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a070bdbb40 0x55a070bd6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:13:19.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.406+0000 7f4efe800640 1 --2- 172.21.15.83:0/2650338237 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a070be6b30 0x55a070c07800 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f4ef80ce3c0 tx=0x7f4ef809c5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=856c30aa6e2fa530 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:19.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.407+0000 7f4eea600640 1 -- 172.21.15.83:0/2650338237 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef8098c00 con 0x55a070be6b30 2024-04-24T09:13:19.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.407+0000 7f4eea600640 1 -- 172.21.15.83:0/2650338237 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4ef80a57d0 con 0x55a070be6b30 2024-04-24T09:13:19.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.407+0000 7f4eea600640 1 -- 172.21.15.83:0/2650338237 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef80991d0 con 0x55a070be6b30 2024-04-24T09:13:19.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.408+0000 7f4f0798f740 1 -- 172.21.15.83:0/2650338237 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a070be6b30 msgr2=0x55a070c07800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:19.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.408+0000 7f4f0798f740 1 --2- 172.21.15.83:0/2650338237 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a070be6b30 0x55a070c07800 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f4ef80ce3c0 tx=0x7f4ef809c5f0 comp rx=0 tx=0).stop 2024-04-24T09:13:19.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.408+0000 7f4f0798f740 1 -- 172.21.15.83:0/2650338237 shutdown_connections 2024-04-24T09:13:19.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.408+0000 7f4f0798f740 1 --2- 172.21.15.83:0/2650338237 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a070be6b30 0x55a070c07800 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:19.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.408+0000 7f4f0798f740 1 --2- 172.21.15.83:0/2650338237 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a070bdc460 0x55a070c072c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:19.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.408+0000 7f4f0798f740 1 --2- 172.21.15.83:0/2650338237 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a070bdbb40 0x55a070bd6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:19.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.408+0000 7f4f0798f740 1 -- 172.21.15.83:0/2650338237 >> 172.21.15.83:0/2650338237 conn(0x55a070bd7e60 msgr2=0x55a070bd66c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:19.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.408+0000 7f4f0798f740 1 -- 172.21.15.83:0/2650338237 shutdown_connections 2024-04-24T09:13:19.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.408+0000 7f4f0798f740 1 -- 172.21.15.83:0/2650338237 wait complete. 2024-04-24T09:13:19.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.409+0000 7f4f0798f740 1 -- start start 2024-04-24T09:13:19.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.409+0000 7f4f0798f740 1 Processor -- start 2024-04-24T09:13:19.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.409+0000 7f4f0798f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a070bdbb40 0x55a070be7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:19.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.409+0000 7f4f0798f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a070bdc460 0x55a070be7f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:19.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.409+0000 7f4f0798f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a070be6b30 0x55a070be8490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:19.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.410+0000 7f4f0798f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4f001ae620 con 0x55a070bdc460 2024-04-24T09:13:19.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.410+0000 7f4f0798f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4f001adf50 con 0x55a070be6b30 2024-04-24T09:13:19.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.410+0000 7f4f0798f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4f001ab190 con 0x55a070bdbb40 2024-04-24T09:13:19.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.410+0000 7f4efe800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a070be6b30 0x55a070be8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:19.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.410+0000 7f4efe800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a070be6b30 0x55a070be8490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:38000/0 (socket says 172.21.15.83:38000) 2024-04-24T09:13:19.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.410+0000 7f4efe800640 1 -- 172.21.15.83:0/3064324886 learned_addr learned my addr 172.21.15.83:0/3064324886 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:13:19.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.410+0000 7f4efd400640 1 --2- 172.21.15.83:0/3064324886 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a070bdc460 0x55a070be7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:19.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.410+0000 7f4f0798f740 10 client.0 _ll_register_callbacks cb 0x7f4ef80077a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:13:19.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.410+0000 7f4efde00640 1 --2- 172.21.15.83:0/3064324886 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a070bdbb40 0x55a070be7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:19.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.411+0000 7f4efd400640 1 -- 172.21.15.83:0/3064324886 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a070bdbb40 msgr2=0x55a070be7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:19.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.411+0000 7f4efd400640 1 --2- 172.21.15.83:0/3064324886 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a070bdbb40 0x55a070be7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:19.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.411+0000 7f4efd400640 1 -- 172.21.15.83:0/3064324886 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a070be6b30 msgr2=0x55a070be8490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:19.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.411+0000 7f4efd400640 1 --2- 172.21.15.83:0/3064324886 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a070be6b30 0x55a070be8490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:19.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.411+0000 7f4efd400640 1 -- 172.21.15.83:0/3064324886 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a070bf5a40 con 0x55a070bdc460 2024-04-24T09:13:19.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.411+0000 7f4efe800640 1 --2- 172.21.15.83:0/3064324886 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a070be6b30 0x55a070be8490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:13:19.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.411+0000 7f4efd400640 1 --2- 172.21.15.83:0/3064324886 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a070bdc460 0x55a070be7f50 secure :-1 s=READY pgs=2610 cs=0 l=1 rev1=1 crypto rx=0x55a070bcbcc0 tx=0x7f4eec005610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d7a327f8d2a78e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:19.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.411+0000 7f4edf400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:13:19.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.411+0000 7f4edf400640 1 -- 172.21.15.83:0/3064324886 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eec02b020 con 0x55a070bdc460 2024-04-24T09:13:19.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.412+0000 7f4edf400640 1 -- 172.21.15.83:0/3064324886 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9583+0+0 (secure 0 0 0) 0x7f4eec004680 con 0x55a070bdc460 2024-04-24T09:13:19.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.412+0000 7f4f0798f740 1 -- 172.21.15.83:0/3064324886 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ef80a57d0 con 0x55a070bdc460 2024-04-24T09:13:19.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.412+0000 7f4edf400640 1 -- 172.21.15.83:0/3064324886 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f4eec005850 con 0x55a070bdc460 2024-04-24T09:13:19.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.412+0000 7f4f0798f740 10 client.22995 fetch_fsmap learned FSMap version 64 2024-04-24T09:13:19.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.412+0000 7f4f0798f740 1 -- 172.21.15.83:0/3064324886 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55a070c65fd0 con 0x55a070bdc460 2024-04-24T09:13:19.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.412+0000 7f4edf400640 1 -- 172.21.15.83:0/3064324886 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4ef80a57d0 con 0x55a070bdc460 2024-04-24T09:13:19.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.413+0000 7f4edf400640 1 -- 172.21.15.83:0/3064324886 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 64) ==== 8374+0+0 (secure 0 0 0) 0x7f4eec01c070 con 0x55a070bdc460 2024-04-24T09:13:19.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.413+0000 7f4f0798f740 10 client.22995 fetch_fsmap finished waiting for FSMap version 64 2024-04-24T09:13:19.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.413+0000 7f4f0798f740 10 client.22995 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-24T09:13:19.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.413+0000 7f4f0798f740 20 client.22995 populate_metadata read hostname 'smithi083' 2024-04-24T09:13:19.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.413+0000 7f4f0798f740 1 --2- 172.21.15.83:0/3064324886 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55a070c66570 0x55a070c68990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:19.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.413+0000 7f4f0798f740 4 client.22995 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:13:19.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.413+0000 7f4f0798f740 1 -- 172.21.15.83:0/3064324886 --> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a070b48a60 con 0x55a070c66570 2024-04-24T09:13:19.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.413+0000 7f4efde00640 1 --2- 172.21.15.83:0/3064324886 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55a070c66570 0x55a070c68990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:19.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.414+0000 7f4efde00640 1 --2- 172.21.15.83:0/3064324886 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55a070c66570 0x55a070c68990 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=32788896c4bd77b8 server_cookie=ee8f693c585227d4 in_seq=0 out_seq=0 2024-04-24T09:13:19.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.414+0000 7f4edf400640 10 client.22995 ms_handle_connect on v2:172.21.15.138:6832/1157967658 2024-04-24T09:13:19.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.414+0000 7f4edf400640 1 -- 172.21.15.83:0/3064324886 <== mds.3 v2:172.21.15.138:6832/1157967658 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a070b48a60 con 0x55a070c66570 2024-04-24T09:13:19.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.414+0000 7f4edf400640 10 client.22995 handle_command_reply: tid=0 2024-04-24T09:13:19.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.414+0000 7f4f0798f740 1 client.22995 shutdown 2024-04-24T09:13:19.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.415+0000 7f4f0798f740 1 -- 172.21.15.83:0/3064324886 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a070bdc460 msgr2=0x55a070be7f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:19.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.415+0000 7f4f0798f740 1 --2- 172.21.15.83:0/3064324886 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a070bdc460 0x55a070be7f50 secure :-1 s=READY pgs=2610 cs=0 l=1 rev1=1 crypto rx=0x55a070bcbcc0 tx=0x7f4eec005610 comp rx=0 tx=0).stop 2024-04-24T09:13:19.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.415+0000 7f4f0798f740 1 -- 172.21.15.83:0/3064324886 shutdown_connections 2024-04-24T09:13:19.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.415+0000 7f4f0798f740 1 --2- 172.21.15.83:0/3064324886 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a070be6b30 0x55a070be8490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:19.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.415+0000 7f4f0798f740 1 --2- 172.21.15.83:0/3064324886 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a070bdc460 0x55a070be7f50 unknown :-1 s=CLOSED pgs=2610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:19.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.415+0000 7f4f0798f740 1 --2- 172.21.15.83:0/3064324886 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55a070c66570 0x55a070c68990 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:19.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.415+0000 7f4f0798f740 1 --2- 172.21.15.83:0/3064324886 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a070bdbb40 0x55a070be7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:19.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.415+0000 7f4edf400640 0 client.22995 ms_handle_reset on v2:172.21.15.138:6832/1157967658 2024-04-24T09:13:19.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.415+0000 7f4f0798f740 1 -- 172.21.15.83:0/3064324886 >> 172.21.15.83:0/3064324886 conn(0x55a070bd7e60 msgr2=0x55a070be4140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:19.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.415+0000 7f4f0798f740 1 -- 172.21.15.83:0/3064324886 shutdown_connections 2024-04-24T09:13:19.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.416+0000 7f4f0798f740 1 -- 172.21.15.83:0/3064324886 wait complete. 2024-04-24T09:13:19.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.416+0000 7f4f0798f740 20 client.22995 trim_cache size 0 max 16384 2024-04-24T09:13:19.419 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:13:19.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.417+0000 7f4eff200640 1 -- 172.21.15.83:0/2880740898 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f4ee40a3da0 msgr2=0x7f4ee40a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:19.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.417+0000 7f4eff200640 1 --2- 172.21.15.83:0/2880740898 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f4ee40a3da0 0x7f4ee40a6260 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f4ef0000c00 tx=0x7f4ef0005a20 comp rx=0 tx=0).stop 2024-04-24T09:13:19.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.417+0000 7f4eff200640 1 -- 172.21.15.83:0/2880740898 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f0019a670 msgr2=0x7f4f0019aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:19.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.417+0000 7f4eff200640 1 --2- 172.21.15.83:0/2880740898 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f0019a670 0x7f4f0019aaf0 secure :-1 s=READY pgs=2608 cs=0 l=1 rev1=1 crypto rx=0x7f4ef800cce0 tx=0x7f4ef800a620 comp rx=0 tx=0).stop 2024-04-24T09:13:19.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.417+0000 7f4eff200640 1 -- 172.21.15.83:0/2880740898 shutdown_connections 2024-04-24T09:13:19.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.417+0000 7f4eff200640 1 --2- 172.21.15.83:0/2880740898 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f4ee40a3da0 0x7f4ee40a6260 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:19.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.417+0000 7f4eff200640 1 --2- 172.21.15.83:0/2880740898 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f0019a670 0x7f4f0019aaf0 unknown :-1 s=CLOSED pgs=2608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:19.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.417+0000 7f4eff200640 1 --2- 172.21.15.83:0/2880740898 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4f001060d0 0x7f4f001a0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:19.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.417+0000 7f4eff200640 1 --2- 172.21.15.83:0/2880740898 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4f00101000 0x7f4f001a05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:19.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.417+0000 7f4eff200640 1 -- 172.21.15.83:0/2880740898 >> 172.21.15.83:0/2880740898 conn(0x7f4f000fc880 msgr2=0x7f4f000fdb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:19.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.417+0000 7f4eff200640 1 -- 172.21.15.83:0/2880740898 shutdown_connections 2024-04-24T09:13:19.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.417+0000 7f4eff200640 1 -- 172.21.15.83:0/2880740898 wait complete. 2024-04-24T09:13:19.437 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:13:19 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/1823097789' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:13:19.440 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T09:13:19.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:13:19 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/1823097789' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:13:19.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:13:19 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/1823097789' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:13:19.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.698+0000 7fe7c2200640 1 -- 172.21.15.83:0/510856498 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7bc103980 msgr2=0x7fe7bc10be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:19.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.698+0000 7fe7c2200640 1 --2- 172.21.15.83:0/510856498 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7bc103980 0x7fe7bc10be60 secure :-1 s=READY pgs=2611 cs=0 l=1 rev1=1 crypto rx=0x7fe7a8009900 tx=0x7fe7a802a490 comp rx=0 tx=0).stop 2024-04-24T09:13:19.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.699+0000 7fe7c2200640 1 -- 172.21.15.83:0/510856498 shutdown_connections 2024-04-24T09:13:19.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.699+0000 7fe7c2200640 1 --2- 172.21.15.83:0/510856498 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe7bc10c480 0x7fe7bc10e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:19.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.699+0000 7fe7c2200640 1 --2- 172.21.15.83:0/510856498 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7bc103980 0x7fe7bc10be60 unknown :-1 s=CLOSED pgs=2611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:19.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.699+0000 7fe7c2200640 1 --2- 172.21.15.83:0/510856498 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7bc103060 0x7fe7bc103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:19.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.699+0000 7fe7c2200640 1 -- 172.21.15.83:0/510856498 >> 172.21.15.83:0/510856498 conn(0x7fe7bc0fc880 msgr2=0x7fe7bc0feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:19.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.699+0000 7fe7c2200640 1 -- 172.21.15.83:0/510856498 shutdown_connections 2024-04-24T09:13:19.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.699+0000 7fe7c2200640 1 -- 172.21.15.83:0/510856498 wait complete. 2024-04-24T09:13:19.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.700+0000 7fe7c2200640 1 Processor -- start 2024-04-24T09:13:19.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.700+0000 7fe7c2200640 1 -- start start 2024-04-24T09:13:19.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.701+0000 7fe7c2200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe7bc103060 0x7fe7bc10bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:19.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.701+0000 7fe7c2200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7bc103980 0x7fe7bc108e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:19.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.701+0000 7fe7c2200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7bc10c480 0x7fe7bc109450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:19.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.701+0000 7fe7c2200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe7bc114380 con 0x7fe7bc103980 2024-04-24T09:13:19.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.701+0000 7fe7c2200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe7bc114200 con 0x7fe7bc10c480 2024-04-24T09:13:19.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.701+0000 7fe7c2200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe7bc114500 con 0x7fe7bc103060 2024-04-24T09:13:19.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.701+0000 7fe7bbe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7bc103980 0x7fe7bc108e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:19.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.701+0000 7fe7bbe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7bc103980 0x7fe7bc108e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35114/0 (socket says 172.21.15.83:35114) 2024-04-24T09:13:19.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.701+0000 7fe7bbe00640 1 -- 172.21.15.83:0/2123195043 learned_addr learned my addr 172.21.15.83:0/2123195043 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:13:19.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.702+0000 7fe7c1800640 1 --2- 172.21.15.83:0/2123195043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7bc10c480 0x7fe7bc109450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:19.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.702+0000 7fe7bbe00640 1 -- 172.21.15.83:0/2123195043 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe7bc103060 msgr2=0x7fe7bc10bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:19.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.702+0000 7fe7bbe00640 1 --2- 172.21.15.83:0/2123195043 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe7bc103060 0x7fe7bc10bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:19.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.702+0000 7fe7bbe00640 1 -- 172.21.15.83:0/2123195043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7bc10c480 msgr2=0x7fe7bc109450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:19.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.702+0000 7fe7bbe00640 1 --2- 172.21.15.83:0/2123195043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7bc10c480 0x7fe7bc109450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:19.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.702+0000 7fe7bbe00640 1 -- 172.21.15.83:0/2123195043 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7bc1a0860 con 0x7fe7bc103980 2024-04-24T09:13:19.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.703+0000 7fe7bbe00640 1 --2- 172.21.15.83:0/2123195043 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7bc103980 0x7fe7bc108e60 secure :-1 s=READY pgs=2612 cs=0 l=1 rev1=1 crypto rx=0x7fe7a8005ab0 tx=0x7fe7a8003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:19.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.704+0000 7fe7b9600640 1 -- 172.21.15.83:0/2123195043 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a803f070 con 0x7fe7bc103980 2024-04-24T09:13:19.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.704+0000 7fe7b9600640 1 -- 172.21.15.83:0/2123195043 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe7a8031b50 con 0x7fe7bc103980 2024-04-24T09:13:19.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.704+0000 7fe7b9600640 1 -- 172.21.15.83:0/2123195043 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a8035030 con 0x7fe7bc103980 2024-04-24T09:13:19.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.706+0000 7fe7c2200640 1 -- 172.21.15.83:0/2123195043 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7bc1a0af0 con 0x7fe7bc103980 2024-04-24T09:13:19.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.706+0000 7fe7c2200640 1 -- 172.21.15.83:0/2123195043 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7bc1a0f90 con 0x7fe7bc103980 2024-04-24T09:13:19.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.710+0000 7fe7c660c740 1 Processor -- start 2024-04-24T09:13:19.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.710+0000 7fe7c660c740 1 -- start start 2024-04-24T09:13:19.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.710+0000 7fe7c660c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5615e87bdb40 0x5615e87bdf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:19.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.710+0000 7fe7c660c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5615e87be460 0x5615e87c85f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:19.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.710+0000 7fe7c660c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5615e87c8b30 0x5615e87caf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:19.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.710+0000 7fe7c660c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5615e87231d0 con 0x5615e87bdb40 2024-04-24T09:13:19.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.710+0000 7fe7c660c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5615e86c9450 con 0x5615e87c8b30 2024-04-24T09:13:19.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.710+0000 7fe7c660c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5615e871f190 con 0x5615e87be460 2024-04-24T09:13:19.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.711+0000 7fe7c1800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5615e87bdb40 0x5615e87bdf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:19.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.711+0000 7fe7c1800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5615e87bdb40 0x5615e87bdf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35124/0 (socket says 172.21.15.83:35124) 2024-04-24T09:13:19.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.711+0000 7fe7c1800640 1 -- 172.21.15.83:0/816150001 learned_addr learned my addr 172.21.15.83:0/816150001 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:13:19.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.712+0000 7fe7c1800640 1 -- 172.21.15.83:0/816150001 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5615e87be460 msgr2=0x5615e87c85f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:19.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.712+0000 7fe7c1800640 1 --2- 172.21.15.83:0/816150001 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5615e87be460 0x5615e87c85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:19.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.712+0000 7fe7c1800640 1 -- 172.21.15.83:0/816150001 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5615e87c8b30 msgr2=0x5615e87caf20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T09:13:19.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.712+0000 7fe7c1800640 1 --2- 172.21.15.83:0/816150001 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5615e87c8b30 0x5615e87caf20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:19.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.712+0000 7fe7c1800640 1 -- 172.21.15.83:0/816150001 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615e8786310 con 0x5615e87bdb40 2024-04-24T09:13:19.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.712+0000 7fe7b9600640 1 -- 172.21.15.83:0/2123195043 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe7a8042070 con 0x7fe7bc103980 2024-04-24T09:13:19.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.713+0000 7fe7c1800640 1 --2- 172.21.15.83:0/816150001 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5615e87bdb40 0x5615e87bdf20 secure :-1 s=READY pgs=2613 cs=0 l=1 rev1=1 crypto rx=0x7fe7b4009e60 tx=0x7fe7b4010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2649c06acecb17c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:19.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.713+0000 7fe7ae000640 1 -- 172.21.15.83:0/816150001 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b4016070 con 0x5615e87bdb40 2024-04-24T09:13:19.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.714+0000 7fe7ae000640 1 -- 172.21.15.83:0/816150001 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe7b4004d20 con 0x5615e87bdb40 2024-04-24T09:13:19.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.714+0000 7fe7ae000640 1 -- 172.21.15.83:0/816150001 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b4012740 con 0x5615e87bdb40 2024-04-24T09:13:19.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.715+0000 7fe7c660c740 1 -- 172.21.15.83:0/816150001 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5615e87bdb40 msgr2=0x5615e87bdf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:19.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.715+0000 7fe7c660c740 1 --2- 172.21.15.83:0/816150001 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5615e87bdb40 0x5615e87bdf20 secure :-1 s=READY pgs=2613 cs=0 l=1 rev1=1 crypto rx=0x7fe7b4009e60 tx=0x7fe7b4010740 comp rx=0 tx=0).stop 2024-04-24T09:13:19.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.715+0000 7fe7c660c740 1 -- 172.21.15.83:0/816150001 shutdown_connections 2024-04-24T09:13:19.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.715+0000 7fe7c660c740 1 --2- 172.21.15.83:0/816150001 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5615e87c8b30 0x5615e87caf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:19.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.715+0000 7fe7c660c740 1 --2- 172.21.15.83:0/816150001 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5615e87be460 0x5615e87c85f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:19.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.715+0000 7fe7c660c740 1 --2- 172.21.15.83:0/816150001 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5615e87bdb40 0x5615e87bdf20 unknown :-1 s=CLOSED pgs=2613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:19.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.715+0000 7fe7c660c740 1 -- 172.21.15.83:0/816150001 >> 172.21.15.83:0/816150001 conn(0x5615e87b9e60 msgr2=0x5615e87ba240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:19.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.715+0000 7fe7b9600640 1 --2- 172.21.15.83:0/2123195043 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe7a40a3de0 0x7fe7a40a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:19.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.716+0000 7fe7c660c740 1 -- 172.21.15.83:0/816150001 shutdown_connections 2024-04-24T09:13:19.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.716+0000 7fe7b9600640 1 -- 172.21.15.83:0/2123195043 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 9583+0+0 (secure 0 0 0) 0x7fe7a80f1360 con 0x7fe7bc103980 2024-04-24T09:13:19.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.716+0000 7fe7c660c740 1 -- 172.21.15.83:0/816150001 wait complete. 2024-04-24T09:13:19.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.716+0000 7fe7c0e00640 1 --2- 172.21.15.83:0/2123195043 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe7a40a3de0 0x7fe7a40a62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:19.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.716+0000 7fe7c660c740 1 -- start start 2024-04-24T09:13:19.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.717+0000 7fe7c660c740 1 Processor -- start 2024-04-24T09:13:19.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.717+0000 7fe7c0e00640 1 --2- 172.21.15.83:0/2123195043 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe7a40a3de0 0x7fe7a40a62a0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fe7b0005d80 tx=0x7fe7b0005c90 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:19.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.717+0000 7fe7c660c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5615e87bdb40 0x5615e87c8570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:19.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.717+0000 7fe7c660c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5615e87be460 0x5615e87c5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:19.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.717+0000 7fe7c660c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5615e87c8b30 0x5615e87c5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:19.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.717+0000 7fe7c660c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5615e87cb9c0 con 0x5615e87bdb40 2024-04-24T09:13:19.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.717+0000 7fe7c660c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5615e86dc810 con 0x5615e87be460 2024-04-24T09:13:19.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.717+0000 7fe7c660c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5615e87cbb40 con 0x5615e87c8b30 2024-04-24T09:13:19.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.717+0000 7fe7c660c740 10 client.0 _ll_register_callbacks cb 0x5615e86aa140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:13:19.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.718+0000 7fe7c1800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5615e87c8b30 0x5615e87c5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:19.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.718+0000 7fe7c1800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5615e87c8b30 0x5615e87c5fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:47788/0 (socket says 172.21.15.83:47788) 2024-04-24T09:13:19.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.718+0000 7fe7c1800640 1 -- 172.21.15.83:0/1184990098 learned_addr learned my addr 172.21.15.83:0/1184990098 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:13:19.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.718+0000 7fe7bbe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5615e87be460 0x5615e87c5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:19.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.718+0000 7fe7c0e00640 1 --2- 172.21.15.83:0/1184990098 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5615e87bdb40 0x5615e87c8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:19.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.718+0000 7fe7c1800640 1 -- 172.21.15.83:0/1184990098 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5615e87be460 msgr2=0x5615e87c5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:19.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.718+0000 7fe7c1800640 1 --2- 172.21.15.83:0/1184990098 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5615e87be460 0x5615e87c5860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:19.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.718+0000 7fe7c1800640 1 -- 172.21.15.83:0/1184990098 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5615e87bdb40 msgr2=0x5615e87c8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:19.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.719+0000 7fe7c1800640 1 --2- 172.21.15.83:0/1184990098 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5615e87bdb40 0x5615e87c8570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:19.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.719+0000 7fe7c1800640 1 -- 172.21.15.83:0/1184990098 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615e86ed3f0 con 0x5615e87c8b30 2024-04-24T09:13:19.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.719+0000 7fe7c0e00640 1 --2- 172.21.15.83:0/1184990098 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5615e87bdb40 0x5615e87c8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:13:19.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.719+0000 7fe7c1800640 1 --2- 172.21.15.83:0/1184990098 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5615e87c8b30 0x5615e87c5fa0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe7b40059a0 tx=0x7fe7b4009a80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b827fe6232beba10 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:19.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.719+0000 7fe7a2a00640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-04-24T09:13:19.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.720+0000 7fe7a2a00640 1 -- 172.21.15.83:0/1184990098 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b4016040 con 0x5615e87c8b30 2024-04-24T09:13:19.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.720+0000 7fe7c660c740 1 -- 172.21.15.83:0/1184990098 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7b4004d20 con 0x5615e87c8b30 2024-04-24T09:13:19.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.720+0000 7fe7a2a00640 1 -- 172.21.15.83:0/1184990098 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9583+0+0 (secure 0 0 0) 0x7fe7b4005420 con 0x5615e87c8b30 2024-04-24T09:13:19.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.720+0000 7fe7a2a00640 1 -- 172.21.15.83:0/1184990098 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7fe7b40124e0 con 0x5615e87c8b30 2024-04-24T09:13:19.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.720+0000 7fe7c660c740 10 client.29857 fetch_fsmap learned FSMap version 64 2024-04-24T09:13:19.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.720+0000 7fe7c660c740 1 -- 172.21.15.83:0/1184990098 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x5615e84353f0 con 0x5615e87c8b30 2024-04-24T09:13:19.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.723+0000 7fe7a2a00640 1 -- 172.21.15.83:0/1184990098 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe7b4004d20 con 0x5615e87c8b30 2024-04-24T09:13:19.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.724+0000 7fe7a2a00640 1 -- 172.21.15.83:0/1184990098 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 64) ==== 8374+0+0 (secure 0 0 0) 0x7fe7b402b030 con 0x5615e87c8b30 2024-04-24T09:13:19.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.724+0000 7fe7c660c740 10 client.29857 fetch_fsmap finished waiting for FSMap version 64 2024-04-24T09:13:19.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.724+0000 7fe7c660c740 10 client.29857 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-24T09:13:19.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.724+0000 7fe7c660c740 20 client.29857 populate_metadata read hostname 'smithi083' 2024-04-24T09:13:19.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.724+0000 7fe7c660c740 1 --2- 172.21.15.83:0/1184990098 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x5615e87d8c50 0x5615e87db040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:19.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.724+0000 7fe7c660c740 4 client.29857 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:13:19.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.724+0000 7fe7c660c740 1 -- 172.21.15.83:0/1184990098 --> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5615e8730bf0 con 0x5615e87d8c50 2024-04-24T09:13:19.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.726+0000 7fe7c0e00640 1 --2- 172.21.15.83:0/1184990098 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x5615e87d8c50 0x5615e87db040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:19.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.729+0000 7fe7c0e00640 1 --2- 172.21.15.83:0/1184990098 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x5615e87d8c50 0x5615e87db040 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=160f179f1973893d server_cookie=f3b62e8b2c7e614d in_seq=0 out_seq=0 2024-04-24T09:13:19.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.729+0000 7fe7a2a00640 10 client.29857 ms_handle_connect on v2:172.21.15.138:6834/1909219758 2024-04-24T09:13:19.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.732+0000 7fe7a2a00640 1 -- 172.21.15.83:0/1184990098 <== mds.2 v2:172.21.15.138:6834/1909219758 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5615e8730bf0 con 0x5615e87d8c50 2024-04-24T09:13:19.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.732+0000 7fe7a2a00640 10 client.29857 handle_command_reply: tid=0 2024-04-24T09:13:19.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.732+0000 7fe7c660c740 1 client.29857 shutdown 2024-04-24T09:13:19.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.732+0000 7fe7c660c740 1 -- 172.21.15.83:0/1184990098 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5615e87c8b30 msgr2=0x5615e87c5fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:19.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.732+0000 7fe7c660c740 1 --2- 172.21.15.83:0/1184990098 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5615e87c8b30 0x5615e87c5fa0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe7b40059a0 tx=0x7fe7b4009a80 comp rx=0 tx=0).stop 2024-04-24T09:13:19.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.732+0000 7fe7c660c740 1 -- 172.21.15.83:0/1184990098 shutdown_connections 2024-04-24T09:13:19.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.732+0000 7fe7c660c740 1 --2- 172.21.15.83:0/1184990098 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x5615e87d8c50 0x5615e87db040 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:19.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.732+0000 7fe7c660c740 1 --2- 172.21.15.83:0/1184990098 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5615e87c8b30 0x5615e87c5fa0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:19.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.732+0000 7fe7c660c740 1 --2- 172.21.15.83:0/1184990098 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5615e87be460 0x5615e87c5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:19.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.732+0000 7fe7a2a00640 0 client.29857 ms_handle_reset on v2:172.21.15.138:6834/1909219758 2024-04-24T09:13:19.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.732+0000 7fe7c660c740 1 --2- 172.21.15.83:0/1184990098 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5615e87bdb40 0x5615e87c8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:19.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.732+0000 7fe7c660c740 1 -- 172.21.15.83:0/1184990098 >> 172.21.15.83:0/1184990098 conn(0x5615e87b9e60 msgr2=0x5615e87bfb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:19.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.733+0000 7fe7c660c740 1 -- 172.21.15.83:0/1184990098 shutdown_connections 2024-04-24T09:13:19.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.733+0000 7fe7c660c740 1 -- 172.21.15.83:0/1184990098 wait complete. 2024-04-24T09:13:19.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.733+0000 7fe7c660c740 20 client.29857 trim_cache size 0 max 16384 2024-04-24T09:13:19.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.759+0000 7fe7c660c740 1 Processor -- start 2024-04-24T09:13:19.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.759+0000 7fe7c660c740 1 -- start start 2024-04-24T09:13:19.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.760+0000 7fe7c660c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5615e87bdb40 0x5615e87d48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:19.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.760+0000 7fe7c660c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5615e87be460 0x5615e87c80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:19.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.760+0000 7fe7c660c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5615e87c8b30 0x5615e87c85e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:19.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.760+0000 7fe7c660c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe7bc1be2d0 con 0x5615e87be460 2024-04-24T09:13:19.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.760+0000 7fe7c660c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe7bc1bfea0 con 0x5615e87c8b30 2024-04-24T09:13:19.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.760+0000 7fe7c660c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe7bc1be9a0 con 0x5615e87bdb40 2024-04-24T09:13:19.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.760+0000 7fe7bbe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5615e87be460 0x5615e87c80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:19.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.760+0000 7fe7c0e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5615e87bdb40 0x5615e87d48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:19.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.760+0000 7fe7bbe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5615e87be460 0x5615e87c80a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35144/0 (socket says 172.21.15.83:35144) 2024-04-24T09:13:19.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.760+0000 7fe7c0e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5615e87bdb40 0x5615e87d48b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:47804/0 (socket says 172.21.15.83:47804) 2024-04-24T09:13:19.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.760+0000 7fe7bbe00640 1 -- 172.21.15.83:0/2665372482 learned_addr learned my addr 172.21.15.83:0/2665372482 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:13:19.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.760+0000 7fe7c1800640 1 --2- 172.21.15.83:0/2665372482 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5615e87c8b30 0x5615e87c85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:19.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.761+0000 7fe7bbe00640 1 -- 172.21.15.83:0/2665372482 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5615e87bdb40 msgr2=0x5615e87d48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:19.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.761+0000 7fe7bbe00640 1 --2- 172.21.15.83:0/2665372482 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5615e87bdb40 0x5615e87d48b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:19.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.761+0000 7fe7bbe00640 1 -- 172.21.15.83:0/2665372482 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5615e87c8b30 msgr2=0x5615e87c85e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:19.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.761+0000 7fe7bbe00640 1 --2- 172.21.15.83:0/2665372482 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5615e87c8b30 0x5615e87c85e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:19.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.761+0000 7fe7bbe00640 1 -- 172.21.15.83:0/2665372482 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615e84ff7d0 con 0x5615e87be460 2024-04-24T09:13:19.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.761+0000 7fe7bbe00640 1 --2- 172.21.15.83:0/2665372482 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5615e87be460 0x5615e87c80a0 secure :-1 s=READY pgs=2614 cs=0 l=1 rev1=1 crypto rx=0x7fe7a80b9340 tx=0x7fe7a80bc900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56f89135d629fb6b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:19.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.762+0000 7fe7ad600640 1 -- 172.21.15.83:0/2665372482 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a80d5070 con 0x5615e87be460 2024-04-24T09:13:19.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.762+0000 7fe7ad600640 1 -- 172.21.15.83:0/2665372482 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe7a80c1290 con 0x5615e87be460 2024-04-24T09:13:19.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.762+0000 7fe7ad600640 1 -- 172.21.15.83:0/2665372482 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a80ee910 con 0x5615e87be460 2024-04-24T09:13:19.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.763+0000 7fe7c660c740 1 -- 172.21.15.83:0/2665372482 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5615e87be460 msgr2=0x5615e87c80a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:19.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.763+0000 7fe7c660c740 1 --2- 172.21.15.83:0/2665372482 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5615e87be460 0x5615e87c80a0 secure :-1 s=READY pgs=2614 cs=0 l=1 rev1=1 crypto rx=0x7fe7a80b9340 tx=0x7fe7a80bc900 comp rx=0 tx=0).stop 2024-04-24T09:13:19.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.763+0000 7fe7c660c740 1 -- 172.21.15.83:0/2665372482 shutdown_connections 2024-04-24T09:13:19.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.763+0000 7fe7c660c740 1 --2- 172.21.15.83:0/2665372482 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5615e87c8b30 0x5615e87c85e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:19.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.763+0000 7fe7c660c740 1 --2- 172.21.15.83:0/2665372482 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5615e87be460 0x5615e87c80a0 unknown :-1 s=CLOSED pgs=2614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:19.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.763+0000 7fe7c660c740 1 --2- 172.21.15.83:0/2665372482 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5615e87bdb40 0x5615e87d48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:19.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.763+0000 7fe7c660c740 1 -- 172.21.15.83:0/2665372482 >> 172.21.15.83:0/2665372482 conn(0x5615e87b9e60 msgr2=0x5615e87b86c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:19.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.763+0000 7fe7c660c740 1 -- 172.21.15.83:0/2665372482 shutdown_connections 2024-04-24T09:13:19.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.764+0000 7fe7c660c740 1 -- 172.21.15.83:0/2665372482 wait complete. 2024-04-24T09:13:19.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.764+0000 7fe7c660c740 1 -- start start 2024-04-24T09:13:19.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.765+0000 7fe7c660c740 1 Processor -- start 2024-04-24T09:13:19.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.765+0000 7fe7c660c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5615e87bdb40 0x5615e87d1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:19.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.765+0000 7fe7c660c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5615e87c8b30 0x5615e87c2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:19.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.765+0000 7fe7c660c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5615e87c3230 0x5615e87c9870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:19.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.765+0000 7fe7c660c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe7bc1bf7a0 con 0x5615e87c3230 2024-04-24T09:13:19.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.765+0000 7fe7c660c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe7bc1bf0d0 con 0x5615e87bdb40 2024-04-24T09:13:19.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.765+0000 7fe7c660c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe7bc1bc2d0 con 0x5615e87c8b30 2024-04-24T09:13:19.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.765+0000 7fe7c660c740 10 client.0 _ll_register_callbacks cb 0x7fe7b4009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:13:19.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.766+0000 7fe7bbe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5615e87c8b30 0x5615e87c2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:19.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.766+0000 7fe7c1800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5615e87c3230 0x5615e87c9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:19.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.766+0000 7fe7bbe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5615e87c8b30 0x5615e87c2ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:47810/0 (socket says 172.21.15.83:47810) 2024-04-24T09:13:19.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.766+0000 7fe7bbe00640 1 -- 172.21.15.83:0/1020768091 learned_addr learned my addr 172.21.15.83:0/1020768091 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:13:19.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.766+0000 7fe7c0e00640 1 --2- 172.21.15.83:0/1020768091 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5615e87bdb40 0x5615e87d1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:19.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.766+0000 7fe7bbe00640 1 -- 172.21.15.83:0/1020768091 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5615e87bdb40 msgr2=0x5615e87d1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:19.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.766+0000 7fe7bbe00640 1 --2- 172.21.15.83:0/1020768091 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5615e87bdb40 0x5615e87d1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:19.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.766+0000 7fe7bbe00640 1 -- 172.21.15.83:0/1020768091 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5615e87c3230 msgr2=0x5615e87c9870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:19.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.766+0000 7fe7bbe00640 1 --2- 172.21.15.83:0/1020768091 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5615e87c3230 0x5615e87c9870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:19.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.766+0000 7fe7bbe00640 1 -- 172.21.15.83:0/1020768091 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615e87c9db0 con 0x5615e87c8b30 2024-04-24T09:13:19.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.766+0000 7fe7c0e00640 1 --2- 172.21.15.83:0/1020768091 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5615e87bdb40 0x5615e87d1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:13:19.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.767+0000 7fe7c1800640 1 --2- 172.21.15.83:0/1020768091 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5615e87c3230 0x5615e87c9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:13:19.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.767+0000 7fe7bbe00640 1 --2- 172.21.15.83:0/1020768091 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5615e87c8b30 0x5615e87c2ca0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe7a8004650 tx=0x7fe7a802a490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9ddae982a705405 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:19.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.767+0000 7fe7a2a00640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-04-24T09:13:19.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.768+0000 7fe7a2a00640 1 -- 172.21.15.83:0/1020768091 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a80d5070 con 0x5615e87c8b30 2024-04-24T09:13:19.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.768+0000 7fe7a2a00640 1 -- 172.21.15.83:0/1020768091 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9583+0+0 (secure 0 0 0) 0x7fe7a80b95c0 con 0x5615e87c8b30 2024-04-24T09:13:19.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.768+0000 7fe7c660c740 1 -- 172.21.15.83:0/1020768091 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7a80c1290 con 0x5615e87c8b30 2024-04-24T09:13:19.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.769+0000 7fe7a2a00640 1 -- 172.21.15.83:0/1020768091 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7fe7a80ee910 con 0x5615e87c8b30 2024-04-24T09:13:19.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.769+0000 7fe7c660c740 10 client.29863 fetch_fsmap learned FSMap version 64 2024-04-24T09:13:19.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.769+0000 7fe7c660c740 1 -- 172.21.15.83:0/1020768091 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x5615e88505c0 con 0x5615e87c8b30 2024-04-24T09:13:19.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.770+0000 7fe7a2a00640 1 -- 172.21.15.83:0/1020768091 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe7a80c1290 con 0x5615e87c8b30 2024-04-24T09:13:19.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.770+0000 7fe7a2a00640 1 -- 172.21.15.83:0/1020768091 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 64) ==== 8374+0+0 (secure 0 0 0) 0x7fe7a8042360 con 0x5615e87c8b30 2024-04-24T09:13:19.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.770+0000 7fe7c660c740 10 client.29863 fetch_fsmap finished waiting for FSMap version 64 2024-04-24T09:13:19.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.770+0000 7fe7c660c740 10 client.29863 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-24T09:13:19.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.770+0000 7fe7c660c740 20 client.29863 populate_metadata read hostname 'smithi083' 2024-04-24T09:13:19.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.770+0000 7fe7c660c740 1 --2- 172.21.15.83:0/1020768091 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x5615e8850d60 0x5615e8853150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:19.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.770+0000 7fe7c660c740 4 client.29863 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:13:19.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.770+0000 7fe7c660c740 1 -- 172.21.15.83:0/1020768091 --> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5615e872aa60 con 0x5615e8850d60 2024-04-24T09:13:19.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.770+0000 7fe7c0e00640 1 --2- 172.21.15.83:0/1020768091 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x5615e8850d60 0x5615e8853150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:19.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.771+0000 7fe7c0e00640 1 --2- 172.21.15.83:0/1020768091 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x5615e8850d60 0x5615e8853150 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=da79c32976141716 server_cookie=2e2868f98566b744 in_seq=0 out_seq=0 2024-04-24T09:13:19.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.771+0000 7fe7a2a00640 10 client.29863 ms_handle_connect on v2:172.21.15.138:6834/1909219758 2024-04-24T09:13:19.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.771+0000 7fe7a2a00640 1 -- 172.21.15.83:0/1020768091 <== mds.2 v2:172.21.15.138:6834/1909219758 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5615e872aa60 con 0x5615e8850d60 2024-04-24T09:13:19.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.771+0000 7fe7a2a00640 10 client.29863 handle_command_reply: tid=0 2024-04-24T09:13:19.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.771+0000 7fe7c660c740 1 client.29863 shutdown 2024-04-24T09:13:19.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.771+0000 7fe7c660c740 1 -- 172.21.15.83:0/1020768091 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5615e87c8b30 msgr2=0x5615e87c2ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:19.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.771+0000 7fe7c660c740 1 --2- 172.21.15.83:0/1020768091 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5615e87c8b30 0x5615e87c2ca0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe7a8004650 tx=0x7fe7a802a490 comp rx=0 tx=0).stop 2024-04-24T09:13:19.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.772+0000 7fe7c660c740 1 -- 172.21.15.83:0/1020768091 shutdown_connections 2024-04-24T09:13:19.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.772+0000 7fe7c660c740 1 --2- 172.21.15.83:0/1020768091 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x5615e8850d60 0x5615e8853150 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:19.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.772+0000 7fe7c660c740 1 --2- 172.21.15.83:0/1020768091 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5615e87c3230 0x5615e87c9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:19.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.772+0000 7fe7c660c740 1 --2- 172.21.15.83:0/1020768091 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5615e87c8b30 0x5615e87c2ca0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:19.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.772+0000 7fe7c660c740 1 --2- 172.21.15.83:0/1020768091 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5615e87bdb40 0x5615e87d1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:19.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.772+0000 7fe7a2a00640 0 client.29863 ms_handle_reset on v2:172.21.15.138:6834/1909219758 2024-04-24T09:13:19.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.772+0000 7fe7c660c740 1 -- 172.21.15.83:0/1020768091 >> 172.21.15.83:0/1020768091 conn(0x5615e87b9e60 msgr2=0x5615e87b8f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:19.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.772+0000 7fe7c660c740 1 -- 172.21.15.83:0/1020768091 shutdown_connections 2024-04-24T09:13:19.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.772+0000 7fe7c660c740 1 -- 172.21.15.83:0/1020768091 wait complete. 2024-04-24T09:13:19.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.772+0000 7fe7c660c740 20 client.29863 trim_cache size 0 max 16384 2024-04-24T09:13:19.777 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:13:19.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.773+0000 7fe7c2200640 1 -- 172.21.15.83:0/2123195043 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe7a40a3de0 msgr2=0x7fe7a40a62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:19.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.773+0000 7fe7c2200640 1 --2- 172.21.15.83:0/2123195043 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe7a40a3de0 0x7fe7a40a62a0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fe7b0005d80 tx=0x7fe7b0005c90 comp rx=0 tx=0).stop 2024-04-24T09:13:19.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.773+0000 7fe7c2200640 1 -- 172.21.15.83:0/2123195043 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7bc103980 msgr2=0x7fe7bc108e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:19.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.773+0000 7fe7c2200640 1 --2- 172.21.15.83:0/2123195043 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7bc103980 0x7fe7bc108e60 secure :-1 s=READY pgs=2612 cs=0 l=1 rev1=1 crypto rx=0x7fe7a8005ab0 tx=0x7fe7a8003c20 comp rx=0 tx=0).stop 2024-04-24T09:13:19.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.773+0000 7fe7c2200640 1 -- 172.21.15.83:0/2123195043 shutdown_connections 2024-04-24T09:13:19.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.773+0000 7fe7c2200640 1 --2- 172.21.15.83:0/2123195043 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe7a40a3de0 0x7fe7a40a62a0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:19.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.773+0000 7fe7c2200640 1 --2- 172.21.15.83:0/2123195043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7bc10c480 0x7fe7bc109450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:19.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.773+0000 7fe7c2200640 1 --2- 172.21.15.83:0/2123195043 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7bc103980 0x7fe7bc108e60 unknown :-1 s=CLOSED pgs=2612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:19.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.773+0000 7fe7c2200640 1 --2- 172.21.15.83:0/2123195043 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe7bc103060 0x7fe7bc10bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:19.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.773+0000 7fe7c2200640 1 -- 172.21.15.83:0/2123195043 >> 172.21.15.83:0/2123195043 conn(0x7fe7bc0fc880 msgr2=0x7fe7bc0fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:19.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.773+0000 7fe7c2200640 1 -- 172.21.15.83:0/2123195043 shutdown_connections 2024-04-24T09:13:19.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:19.773+0000 7fe7c2200640 1 -- 172.21.15.83:0/2123195043 wait complete. 2024-04-24T09:13:19.796 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T09:13:20.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.046+0000 7f6991e00640 1 -- 172.21.15.83:0/4006366191 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f698c109580 msgr2=0x7f698c111150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:20.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.046+0000 7f6991e00640 1 --2- 172.21.15.83:0/4006366191 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f698c109580 0x7f698c111150 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f6984009be0 tx=0x7f698402a6d0 comp rx=0 tx=0).stop 2024-04-24T09:13:20.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.046+0000 7f6991e00640 1 -- 172.21.15.83:0/4006366191 shutdown_connections 2024-04-24T09:13:20.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.046+0000 7f6991e00640 1 --2- 172.21.15.83:0/4006366191 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f698c109580 0x7f698c111150 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:20.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.046+0000 7f6991e00640 1 --2- 172.21.15.83:0/4006366191 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f698c104e60 0x7f698c108ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:20.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.046+0000 7f6991e00640 1 --2- 172.21.15.83:0/4006366191 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f698c1044b0 0x7f698c104890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:20.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.046+0000 7f6991e00640 1 -- 172.21.15.83:0/4006366191 >> 172.21.15.83:0/4006366191 conn(0x7f698c0ffd90 msgr2=0x7f698c1021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:20.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.047+0000 7f6991e00640 1 -- 172.21.15.83:0/4006366191 shutdown_connections 2024-04-24T09:13:20.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.047+0000 7f6991e00640 1 -- 172.21.15.83:0/4006366191 wait complete. 2024-04-24T09:13:20.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.048+0000 7f6991e00640 1 Processor -- start 2024-04-24T09:13:20.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.049+0000 7f6991e00640 1 -- start start 2024-04-24T09:13:20.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.049+0000 7f6991e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f698c1044b0 0x7f698c1a06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:20.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.049+0000 7f6991e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f698c109580 0x7f698c1a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:20.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.049+0000 7f6991e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f698c19a850 0x7f698c19ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:20.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.049+0000 7f6991e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f698c1144d0 con 0x7f698c1044b0 2024-04-24T09:13:20.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.049+0000 7f6991e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f698c114350 con 0x7f698c109580 2024-04-24T09:13:20.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.049+0000 7f6991e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f698c114650 con 0x7f698c19a850 2024-04-24T09:13:20.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.049+0000 7f698aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f698c1044b0 0x7f698c1a06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:20.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.049+0000 7f698aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f698c1044b0 0x7f698c1a06e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35180/0 (socket says 172.21.15.83:35180) 2024-04-24T09:13:20.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.050+0000 7f698aa00640 1 -- 172.21.15.83:0/2960192827 learned_addr learned my addr 172.21.15.83:0/2960192827 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:13:20.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.049+0000 7f698a000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f698c109580 0x7f698c1a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:20.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.050+0000 7f698b400640 1 --2- 172.21.15.83:0/2960192827 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f698c19a850 0x7f698c19ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:20.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.050+0000 7f698a000640 1 -- 172.21.15.83:0/2960192827 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f698c19a850 msgr2=0x7f698c19ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:20.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.050+0000 7f698a000640 1 --2- 172.21.15.83:0/2960192827 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f698c19a850 0x7f698c19ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:20.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.050+0000 7f698a000640 1 -- 172.21.15.83:0/2960192827 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f698c1044b0 msgr2=0x7f698c1a06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:20.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.050+0000 7f698a000640 1 --2- 172.21.15.83:0/2960192827 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f698c1044b0 0x7f698c1a06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:20.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.050+0000 7f698a000640 1 -- 172.21.15.83:0/2960192827 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f698c19b3e0 con 0x7f698c109580 2024-04-24T09:13:20.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.051+0000 7f698a000640 1 --2- 172.21.15.83:0/2960192827 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f698c109580 0x7f698c1a0c20 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f6978005fd0 tx=0x7f697800ec20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:20.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.051+0000 7f6990a00640 1 -- 172.21.15.83:0/2960192827 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f697800ccc0 con 0x7f698c109580 2024-04-24T09:13:20.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.051+0000 7f6990a00640 1 -- 172.21.15.83:0/2960192827 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f697800ee70 con 0x7f698c109580 2024-04-24T09:13:20.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.051+0000 7f6990a00640 1 -- 172.21.15.83:0/2960192827 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6978018750 con 0x7f698c109580 2024-04-24T09:13:20.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.051+0000 7f6991e00640 1 -- 172.21.15.83:0/2960192827 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f698c19b6d0 con 0x7f698c109580 2024-04-24T09:13:20.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.052+0000 7f6991e00640 1 -- 172.21.15.83:0/2960192827 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f698c1a7860 con 0x7f698c109580 2024-04-24T09:13:20.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.056+0000 7f6990a00640 1 -- 172.21.15.83:0/2960192827 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6978010430 con 0x7f698c109580 2024-04-24T09:13:20.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.056+0000 7f6994c2c740 1 Processor -- start 2024-04-24T09:13:20.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.056+0000 7f6994c2c740 1 -- start start 2024-04-24T09:13:20.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.057+0000 7f6994c2c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bfb0bd160 0x562bfb0bd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:20.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.057+0000 7f6994c2c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bfb0bda80 0x562bfb0c7c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:20.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.057+0000 7f6994c2c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562bfb0c8150 0x562bfb0ca540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:20.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.057+0000 7f6994c2c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562bfb01a1d0 con 0x562bfb0bd160 2024-04-24T09:13:20.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.057+0000 7f6994c2c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562bfafc0450 con 0x562bfb0c8150 2024-04-24T09:13:20.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.057+0000 7f6994c2c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x562bfb016190 con 0x562bfb0bda80 2024-04-24T09:13:20.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.058+0000 7f698b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bfb0bd160 0x562bfb0bd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:20.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.058+0000 7f698b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bfb0bd160 0x562bfb0bd540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35194/0 (socket says 172.21.15.83:35194) 2024-04-24T09:13:20.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.058+0000 7f698b400640 1 -- 172.21.15.83:0/3983107280 learned_addr learned my addr 172.21.15.83:0/3983107280 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:13:20.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.058+0000 7f698aa00640 1 --2- 172.21.15.83:0/3983107280 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bfb0bda80 0x562bfb0c7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:20.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.058+0000 7f698a000640 1 --2- 172.21.15.83:0/3983107280 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562bfb0c8150 0x562bfb0ca540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:20.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.058+0000 7f698b400640 1 -- 172.21.15.83:0/3983107280 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bfb0bda80 msgr2=0x562bfb0c7c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:20.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.058+0000 7f698b400640 1 --2- 172.21.15.83:0/3983107280 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bfb0bda80 0x562bfb0c7c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:20.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.058+0000 7f698b400640 1 -- 172.21.15.83:0/3983107280 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562bfb0c8150 msgr2=0x562bfb0ca540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:20.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.058+0000 7f698b400640 1 --2- 172.21.15.83:0/3983107280 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562bfb0c8150 0x562bfb0ca540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:20.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.058+0000 7f698b400640 1 -- 172.21.15.83:0/3983107280 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562bfb07d310 con 0x562bfb0bd160 2024-04-24T09:13:20.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.058+0000 7f6990a00640 1 --2- 172.21.15.83:0/2960192827 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f69700a3d20 0x7f69700a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:20.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.059+0000 7f698b400640 1 --2- 172.21.15.83:0/3983107280 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bfb0bd160 0x562bfb0bd540 secure :-1 s=READY pgs=2615 cs=0 l=1 rev1=1 crypto rx=0x7f698402acc0 tx=0x7f69840303a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90b98ae0c7b5a809 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:20.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.059+0000 7f6990a00640 1 -- 172.21.15.83:0/2960192827 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 9583+0+0 (secure 0 0 0) 0x7f69780d0e20 con 0x7f698c109580 2024-04-24T09:13:20.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.059+0000 7f698aa00640 1 --2- 172.21.15.83:0/2960192827 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f69700a3d20 0x7f69700a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:20.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.059+0000 7f6977e00640 1 -- 172.21.15.83:0/3983107280 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6984040030 con 0x562bfb0bd160 2024-04-24T09:13:20.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.059+0000 7f6977e00640 1 -- 172.21.15.83:0/3983107280 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6984037040 con 0x562bfb0bd160 2024-04-24T09:13:20.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.059+0000 7f6977e00640 1 -- 172.21.15.83:0/3983107280 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6984036070 con 0x562bfb0bd160 2024-04-24T09:13:20.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.060+0000 7f6994c2c740 1 -- 172.21.15.83:0/3983107280 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bfb0bd160 msgr2=0x562bfb0bd540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:20.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.060+0000 7f6994c2c740 1 --2- 172.21.15.83:0/3983107280 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bfb0bd160 0x562bfb0bd540 secure :-1 s=READY pgs=2615 cs=0 l=1 rev1=1 crypto rx=0x7f698402acc0 tx=0x7f69840303a0 comp rx=0 tx=0).stop 2024-04-24T09:13:20.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.060+0000 7f698aa00640 1 --2- 172.21.15.83:0/2960192827 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f69700a3d20 0x7f69700a61e0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x562bfb0be720 tx=0x7f697c005f90 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:20.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.060+0000 7f6994c2c740 1 -- 172.21.15.83:0/3983107280 shutdown_connections 2024-04-24T09:13:20.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.060+0000 7f6994c2c740 1 --2- 172.21.15.83:0/3983107280 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562bfb0c8150 0x562bfb0ca540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:20.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.060+0000 7f6994c2c740 1 --2- 172.21.15.83:0/3983107280 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bfb0bda80 0x562bfb0c7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:20.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.060+0000 7f6994c2c740 1 --2- 172.21.15.83:0/3983107280 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bfb0bd160 0x562bfb0bd540 unknown :-1 s=CLOSED pgs=2615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:20.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.060+0000 7f6994c2c740 1 -- 172.21.15.83:0/3983107280 >> 172.21.15.83:0/3983107280 conn(0x562bfb0b9480 msgr2=0x562bfb0b9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:20.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.060+0000 7f6994c2c740 1 -- 172.21.15.83:0/3983107280 shutdown_connections 2024-04-24T09:13:20.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.060+0000 7f6994c2c740 1 -- 172.21.15.83:0/3983107280 wait complete. 2024-04-24T09:13:20.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.061+0000 7f6994c2c740 1 -- start start 2024-04-24T09:13:20.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.061+0000 7f6994c2c740 1 Processor -- start 2024-04-24T09:13:20.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.062+0000 7f6994c2c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bfb0bd160 0x562bfb0c7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:20.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.062+0000 7f6994c2c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562bfb0bda80 0x562bfb0c4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:20.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.062+0000 7f6994c2c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bfb0c8150 0x562bfb0c5600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:20.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.062+0000 7f6994c2c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562bfb0caf80 con 0x562bfb0bd160 2024-04-24T09:13:20.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.062+0000 7f6994c2c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562bfafd3810 con 0x562bfb0bda80 2024-04-24T09:13:20.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.062+0000 7f6994c2c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x562bfb0cb100 con 0x562bfb0c8150 2024-04-24T09:13:20.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.062+0000 7f6994c2c740 10 client.0 _ll_register_callbacks cb 0x562bfafa1140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:13:20.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.062+0000 7f698aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bfb0bd160 0x562bfb0c7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:20.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.062+0000 7f698aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bfb0bd160 0x562bfb0c7bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35204/0 (socket says 172.21.15.83:35204) 2024-04-24T09:13:20.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.062+0000 7f698aa00640 1 -- 172.21.15.83:0/1049459661 learned_addr learned my addr 172.21.15.83:0/1049459661 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:13:20.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.063+0000 7f698a000640 1 --2- 172.21.15.83:0/1049459661 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562bfb0bda80 0x562bfb0c4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:20.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.063+0000 7f698b400640 1 --2- 172.21.15.83:0/1049459661 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bfb0c8150 0x562bfb0c5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:20.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.063+0000 7f698aa00640 1 -- 172.21.15.83:0/1049459661 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bfb0c8150 msgr2=0x562bfb0c5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:20.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.063+0000 7f698aa00640 1 --2- 172.21.15.83:0/1049459661 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bfb0c8150 0x562bfb0c5600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:20.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.063+0000 7f698aa00640 1 -- 172.21.15.83:0/1049459661 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562bfb0bda80 msgr2=0x562bfb0c4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:20.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.063+0000 7f698aa00640 1 --2- 172.21.15.83:0/1049459661 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562bfb0bda80 0x562bfb0c4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:20.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.063+0000 7f698aa00640 1 -- 172.21.15.83:0/1049459661 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562bfafe43f0 con 0x562bfb0bd160 2024-04-24T09:13:20.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.064+0000 7f698aa00640 1 --2- 172.21.15.83:0/1049459661 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bfb0bd160 0x562bfb0c7bd0 secure :-1 s=READY pgs=2616 cs=0 l=1 rev1=1 crypto rx=0x7f697c01ea00 tx=0x7f697c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44bf33f136daa3c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:20.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.064+0000 7f6974c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:13:20.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.064+0000 7f6974c00640 1 -- 172.21.15.83:0/1049459661 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f697c02e060 con 0x562bfb0bd160 2024-04-24T09:13:20.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.064+0000 7f6994c2c740 1 -- 172.21.15.83:0/1049459661 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6984037040 con 0x562bfb0bd160 2024-04-24T09:13:20.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.064+0000 7f6974c00640 1 -- 172.21.15.83:0/1049459661 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9583+0+0 (secure 0 0 0) 0x7f697c018030 con 0x562bfb0bd160 2024-04-24T09:13:20.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.064+0000 7f6974c00640 1 -- 172.21.15.83:0/1049459661 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f697c029420 con 0x562bfb0bd160 2024-04-24T09:13:20.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.065+0000 7f6994c2c740 10 client.23025 fetch_fsmap learned FSMap version 64 2024-04-24T09:13:20.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.065+0000 7f6994c2c740 1 -- 172.21.15.83:0/1049459661 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x562bfad2c3f0 con 0x562bfb0bd160 2024-04-24T09:13:20.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.065+0000 7f6974c00640 1 -- 172.21.15.83:0/1049459661 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6984037040 con 0x562bfb0bd160 2024-04-24T09:13:20.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.067+0000 7f6974c00640 1 -- 172.21.15.83:0/1049459661 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 64) ==== 8374+0+0 (secure 0 0 0) 0x7f697c00f600 con 0x562bfb0bd160 2024-04-24T09:13:20.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.067+0000 7f6994c2c740 10 client.23025 fetch_fsmap finished waiting for FSMap version 64 2024-04-24T09:13:20.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.067+0000 7f6994c2c740 10 client.23025 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-24T09:13:20.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.067+0000 7f6994c2c740 20 client.23025 populate_metadata read hostname 'smithi083' 2024-04-24T09:13:20.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.067+0000 7f6994c2c740 1 --2- 172.21.15.83:0/1049459661 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x562bfb0d8300 0x562bfb0da750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:20.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.067+0000 7f6994c2c740 4 client.23025 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:13:20.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.067+0000 7f6994c2c740 1 -- 172.21.15.83:0/1049459661 --> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562bfb027bf0 con 0x562bfb0d8300 2024-04-24T09:13:20.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.067+0000 7f698a000640 1 --2- 172.21.15.83:0/1049459661 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x562bfb0d8300 0x562bfb0da750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:20.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.068+0000 7f698a000640 1 --2- 172.21.15.83:0/1049459661 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x562bfb0d8300 0x562bfb0da750 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=87fe7c4e8931851 server_cookie=62fdd47b011c0a57 in_seq=0 out_seq=0 2024-04-24T09:13:20.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.068+0000 7f6974c00640 10 client.23025 ms_handle_connect on v2:172.21.15.138:6836/900566754 2024-04-24T09:13:20.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.070+0000 7f6974c00640 1 -- 172.21.15.83:0/1049459661 <== mds.1 v2:172.21.15.138:6836/900566754 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562bfb027bf0 con 0x562bfb0d8300 2024-04-24T09:13:20.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.070+0000 7f6974c00640 10 client.23025 handle_command_reply: tid=0 2024-04-24T09:13:20.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.070+0000 7f6994c2c740 1 client.23025 shutdown 2024-04-24T09:13:20.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.071+0000 7f6994c2c740 1 -- 172.21.15.83:0/1049459661 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bfb0bd160 msgr2=0x562bfb0c7bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:20.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.071+0000 7f6994c2c740 1 --2- 172.21.15.83:0/1049459661 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bfb0bd160 0x562bfb0c7bd0 secure :-1 s=READY pgs=2616 cs=0 l=1 rev1=1 crypto rx=0x7f697c01ea00 tx=0x7f697c004730 comp rx=0 tx=0).stop 2024-04-24T09:13:20.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.071+0000 7f6994c2c740 1 -- 172.21.15.83:0/1049459661 shutdown_connections 2024-04-24T09:13:20.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.071+0000 7f6994c2c740 1 --2- 172.21.15.83:0/1049459661 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x562bfb0d8300 0x562bfb0da750 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:20.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.071+0000 7f6994c2c740 1 --2- 172.21.15.83:0/1049459661 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bfb0c8150 0x562bfb0c5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:20.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.071+0000 7f6994c2c740 1 --2- 172.21.15.83:0/1049459661 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562bfb0bda80 0x562bfb0c4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:20.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.071+0000 7f6994c2c740 1 --2- 172.21.15.83:0/1049459661 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bfb0bd160 0x562bfb0c7bd0 unknown :-1 s=CLOSED pgs=2616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:20.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.071+0000 7f6974c00640 0 client.23025 ms_handle_reset on v2:172.21.15.138:6836/900566754 2024-04-24T09:13:20.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.071+0000 7f6994c2c740 1 -- 172.21.15.83:0/1049459661 >> 172.21.15.83:0/1049459661 conn(0x562bfb0b9480 msgr2=0x562bfb0bf280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:20.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.071+0000 7f6994c2c740 1 -- 172.21.15.83:0/1049459661 shutdown_connections 2024-04-24T09:13:20.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.071+0000 7f6994c2c740 1 -- 172.21.15.83:0/1049459661 wait complete. 2024-04-24T09:13:20.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.071+0000 7f6994c2c740 20 client.23025 trim_cache size 0 max 16384 2024-04-24T09:13:20.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.095+0000 7f6994c2c740 1 Processor -- start 2024-04-24T09:13:20.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.095+0000 7f6994c2c740 1 -- start start 2024-04-24T09:13:20.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.095+0000 7f6994c2c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bfb0bda80 0x562bfb0bf8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:20.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.095+0000 7f6994c2c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562bfb0c8150 0x562bfb0c7700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:20.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.096+0000 7f6994c2c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bfb0c7c40 0x562bfb0e0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:20.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.096+0000 7f6994c2c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f698c1ad510 con 0x562bfb0bda80 2024-04-24T09:13:20.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.096+0000 7f6994c2c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f698c1af0e0 con 0x562bfb0c8150 2024-04-24T09:13:20.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.096+0000 7f6994c2c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f698c1adbe0 con 0x562bfb0c7c40 2024-04-24T09:13:20.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.096+0000 7f698aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bfb0bda80 0x562bfb0bf8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:20.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.096+0000 7f698a000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562bfb0c8150 0x562bfb0c7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:20.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.096+0000 7f698a000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562bfb0c8150 0x562bfb0c7700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:38080/0 (socket says 172.21.15.83:38080) 2024-04-24T09:13:20.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.096+0000 7f698aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bfb0bda80 0x562bfb0bf8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35220/0 (socket says 172.21.15.83:35220) 2024-04-24T09:13:20.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.096+0000 7f698aa00640 1 -- 172.21.15.83:0/3751575366 learned_addr learned my addr 172.21.15.83:0/3751575366 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:13:20.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.096+0000 7f698b400640 1 --2- 172.21.15.83:0/3751575366 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bfb0c7c40 0x562bfb0e0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:20.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.097+0000 7f698aa00640 1 -- 172.21.15.83:0/3751575366 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bfb0c7c40 msgr2=0x562bfb0e0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:20.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.097+0000 7f698aa00640 1 --2- 172.21.15.83:0/3751575366 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bfb0c7c40 0x562bfb0e0020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:20.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.097+0000 7f698aa00640 1 -- 172.21.15.83:0/3751575366 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562bfb0c8150 msgr2=0x562bfb0c7700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:20.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.097+0000 7f698aa00640 1 --2- 172.21.15.83:0/3751575366 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562bfb0c8150 0x562bfb0c7700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:20.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.097+0000 7f698aa00640 1 -- 172.21.15.83:0/3751575366 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562bfadf67d0 con 0x562bfb0bda80 2024-04-24T09:13:20.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.098+0000 7f698aa00640 1 --2- 172.21.15.83:0/3751575366 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bfb0bda80 0x562bfb0bf8c0 secure :-1 s=READY pgs=2617 cs=0 l=1 rev1=1 crypto rx=0x7f697c012cb0 tx=0x7f697c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3793907113924ff8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:20.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.098+0000 7f6977400640 1 -- 172.21.15.83:0/3751575366 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f697c00f5c0 con 0x562bfb0bda80 2024-04-24T09:13:20.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.098+0000 7f6977400640 1 -- 172.21.15.83:0/3751575366 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f697c03e070 con 0x562bfb0bda80 2024-04-24T09:13:20.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.098+0000 7f6977400640 1 -- 172.21.15.83:0/3751575366 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f697c039480 con 0x562bfb0bda80 2024-04-24T09:13:20.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.099+0000 7f6994c2c740 1 -- 172.21.15.83:0/3751575366 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bfb0bda80 msgr2=0x562bfb0bf8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:20.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.099+0000 7f6994c2c740 1 --2- 172.21.15.83:0/3751575366 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bfb0bda80 0x562bfb0bf8c0 secure :-1 s=READY pgs=2617 cs=0 l=1 rev1=1 crypto rx=0x7f697c012cb0 tx=0x7f697c000bc0 comp rx=0 tx=0).stop 2024-04-24T09:13:20.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.101+0000 7f6994c2c740 1 -- 172.21.15.83:0/3751575366 shutdown_connections 2024-04-24T09:13:20.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.101+0000 7f6994c2c740 1 --2- 172.21.15.83:0/3751575366 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bfb0c7c40 0x562bfb0e0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:20.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.101+0000 7f6994c2c740 1 --2- 172.21.15.83:0/3751575366 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562bfb0c8150 0x562bfb0c7700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:20.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.101+0000 7f6994c2c740 1 --2- 172.21.15.83:0/3751575366 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bfb0bda80 0x562bfb0bf8c0 unknown :-1 s=CLOSED pgs=2617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:20.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.101+0000 7f6994c2c740 1 -- 172.21.15.83:0/3751575366 >> 172.21.15.83:0/3751575366 conn(0x562bfb0b9480 msgr2=0x562bfb0afdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:20.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.101+0000 7f6994c2c740 1 -- 172.21.15.83:0/3751575366 shutdown_connections 2024-04-24T09:13:20.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.101+0000 7f6994c2c740 1 -- 172.21.15.83:0/3751575366 wait complete. 2024-04-24T09:13:20.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.102+0000 7f6994c2c740 1 -- start start 2024-04-24T09:13:20.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.102+0000 7f6994c2c740 1 Processor -- start 2024-04-24T09:13:20.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.102+0000 7f6994c2c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562bfb0bda80 0x562bfb0d0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:20.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.102+0000 7f6994c2c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bfb0c7c40 0x562bfb0d0890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:20.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.102+0000 7f6994c2c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bfb0c8150 0x562bfb0c1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:20.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.102+0000 7f6994c2c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f698c1ae9e0 con 0x562bfb0c8150 2024-04-24T09:13:20.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.102+0000 7f6994c2c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f698c1ae310 con 0x562bfb0bda80 2024-04-24T09:13:20.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.102+0000 7f6994c2c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f698c1ab510 con 0x562bfb0c7c40 2024-04-24T09:13:20.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.103+0000 7f698aa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562bfb0bda80 0x562bfb0d0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:20.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.103+0000 7f698aa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562bfb0bda80 0x562bfb0d0350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:38084/0 (socket says 172.21.15.83:38084) 2024-04-24T09:13:20.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.103+0000 7f698aa00640 1 -- 172.21.15.83:0/2454484510 learned_addr learned my addr 172.21.15.83:0/2454484510 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:13:20.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.103+0000 7f698a000640 1 --2- 172.21.15.83:0/2454484510 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bfb0c7c40 0x562bfb0d0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:20.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.103+0000 7f698b400640 1 --2- 172.21.15.83:0/2454484510 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bfb0c8150 0x562bfb0c1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:20.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.103+0000 7f6994c2c740 10 client.0 _ll_register_callbacks cb 0x7f697c01e890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:13:20.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.104+0000 7f698aa00640 1 -- 172.21.15.83:0/2454484510 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bfb0c7c40 msgr2=0x562bfb0d0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:20.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.104+0000 7f698aa00640 1 --2- 172.21.15.83:0/2454484510 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bfb0c7c40 0x562bfb0d0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:20.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.104+0000 7f698aa00640 1 -- 172.21.15.83:0/2454484510 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bfb0c8150 msgr2=0x562bfb0c1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:20.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.104+0000 7f698aa00640 1 --2- 172.21.15.83:0/2454484510 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bfb0c8150 0x562bfb0c1130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:20.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.104+0000 7f698aa00640 1 -- 172.21.15.83:0/2454484510 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562bfb0c1740 con 0x562bfb0bda80 2024-04-24T09:13:20.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.104+0000 7f698b400640 1 --2- 172.21.15.83:0/2454484510 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bfb0c8150 0x562bfb0c1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:13:20.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.105+0000 7f698aa00640 1 --2- 172.21.15.83:0/2454484510 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562bfb0bda80 0x562bfb0d0350 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f697c005820 tx=0x7f697c009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b212e90926c2738 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:20.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.105+0000 7f6974c00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:13:20.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.105+0000 7f6974c00640 1 -- 172.21.15.83:0/2454484510 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f697c02e020 con 0x562bfb0bda80 2024-04-24T09:13:20.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.105+0000 7f6974c00640 1 -- 172.21.15.83:0/2454484510 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9583+0+0 (secure 0 0 0) 0x7f697c024040 con 0x562bfb0bda80 2024-04-24T09:13:20.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.105+0000 7f6994c2c740 1 -- 172.21.15.83:0/2454484510 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f697c03e070 con 0x562bfb0bda80 2024-04-24T09:13:20.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.106+0000 7f6974c00640 1 -- 172.21.15.83:0/2454484510 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f697c0297d0 con 0x562bfb0bda80 2024-04-24T09:13:20.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.106+0000 7f6994c2c740 10 client.40085 fetch_fsmap learned FSMap version 64 2024-04-24T09:13:20.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.106+0000 7f6994c2c740 1 -- 172.21.15.83:0/2454484510 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x562bfb0c2a20 con 0x562bfb0bda80 2024-04-24T09:13:20.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.106+0000 7f6974c00640 1 -- 172.21.15.83:0/2454484510 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f697c03e070 con 0x562bfb0bda80 2024-04-24T09:13:20.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.106+0000 7f6974c00640 1 -- 172.21.15.83:0/2454484510 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 64) ==== 8374+0+0 (secure 0 0 0) 0x7f697c004ad0 con 0x562bfb0bda80 2024-04-24T09:13:20.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.106+0000 7f6994c2c740 10 client.40085 fetch_fsmap finished waiting for FSMap version 64 2024-04-24T09:13:20.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.106+0000 7f6994c2c740 10 client.40085 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-24T09:13:20.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.106+0000 7f6994c2c740 20 client.40085 populate_metadata read hostname 'smithi083' 2024-04-24T09:13:20.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.107+0000 7f6994c2c740 1 --2- 172.21.15.83:0/2454484510 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x562bfb13f8a0 0x562bfb141c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:20.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.107+0000 7f6994c2c740 4 client.40085 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:13:20.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.107+0000 7f6994c2c740 1 -- 172.21.15.83:0/2454484510 --> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562bfb021a60 con 0x562bfb13f8a0 2024-04-24T09:13:20.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.107+0000 7f698a000640 1 --2- 172.21.15.83:0/2454484510 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x562bfb13f8a0 0x562bfb141c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:20.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.108+0000 7f698a000640 1 --2- 172.21.15.83:0/2454484510 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x562bfb13f8a0 0x562bfb141c90 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=944580d9ed0930a6 server_cookie=2e24993b90fe7054 in_seq=0 out_seq=0 2024-04-24T09:13:20.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.108+0000 7f6974c00640 10 client.40085 ms_handle_connect on v2:172.21.15.138:6836/900566754 2024-04-24T09:13:20.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.108+0000 7f6974c00640 1 -- 172.21.15.83:0/2454484510 <== mds.1 v2:172.21.15.138:6836/900566754 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562bfb021a60 con 0x562bfb13f8a0 2024-04-24T09:13:20.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.108+0000 7f6974c00640 10 client.40085 handle_command_reply: tid=0 2024-04-24T09:13:20.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.108+0000 7f6994c2c740 1 client.40085 shutdown 2024-04-24T09:13:20.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.109+0000 7f6994c2c740 1 -- 172.21.15.83:0/2454484510 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562bfb0bda80 msgr2=0x562bfb0d0350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:20.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.109+0000 7f6994c2c740 1 --2- 172.21.15.83:0/2454484510 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562bfb0bda80 0x562bfb0d0350 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f697c005820 tx=0x7f697c009c20 comp rx=0 tx=0).stop 2024-04-24T09:13:20.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.109+0000 7f6994c2c740 1 -- 172.21.15.83:0/2454484510 shutdown_connections 2024-04-24T09:13:20.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.109+0000 7f6994c2c740 1 --2- 172.21.15.83:0/2454484510 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bfb0c8150 0x562bfb0c1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:20.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.109+0000 7f6994c2c740 1 --2- 172.21.15.83:0/2454484510 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x562bfb13f8a0 0x562bfb141c90 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:20.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.109+0000 7f6994c2c740 1 --2- 172.21.15.83:0/2454484510 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bfb0c7c40 0x562bfb0d0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:20.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.109+0000 7f6994c2c740 1 --2- 172.21.15.83:0/2454484510 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562bfb0bda80 0x562bfb0d0350 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:20.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.109+0000 7f6994c2c740 1 -- 172.21.15.83:0/2454484510 >> 172.21.15.83:0/2454484510 conn(0x562bfb0b9480 msgr2=0x562bfb0c3660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:20.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.109+0000 7f6974c00640 0 client.40085 ms_handle_reset on v2:172.21.15.138:6836/900566754 2024-04-24T09:13:20.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.110+0000 7f6994c2c740 1 -- 172.21.15.83:0/2454484510 shutdown_connections 2024-04-24T09:13:20.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.110+0000 7f6994c2c740 1 -- 172.21.15.83:0/2454484510 wait complete. 2024-04-24T09:13:20.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.110+0000 7f6994c2c740 20 client.40085 trim_cache size 0 max 16384 2024-04-24T09:13:20.112 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:13:20.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.111+0000 7f6991e00640 1 -- 172.21.15.83:0/2960192827 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f69700a3d20 msgr2=0x7f69700a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:20.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.111+0000 7f6991e00640 1 --2- 172.21.15.83:0/2960192827 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f69700a3d20 0x7f69700a61e0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x562bfb0be720 tx=0x7f697c005f90 comp rx=0 tx=0).stop 2024-04-24T09:13:20.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.111+0000 7f6991e00640 1 -- 172.21.15.83:0/2960192827 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f698c109580 msgr2=0x7f698c1a0c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:20.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.111+0000 7f6991e00640 1 --2- 172.21.15.83:0/2960192827 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f698c109580 0x7f698c1a0c20 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f6978005fd0 tx=0x7f697800ec20 comp rx=0 tx=0).stop 2024-04-24T09:13:20.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.111+0000 7f6991e00640 1 -- 172.21.15.83:0/2960192827 shutdown_connections 2024-04-24T09:13:20.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.111+0000 7f6991e00640 1 --2- 172.21.15.83:0/2960192827 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f69700a3d20 0x7f69700a61e0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:20.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.111+0000 7f6991e00640 1 --2- 172.21.15.83:0/2960192827 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f698c19a850 0x7f698c19ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:20.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.111+0000 7f6991e00640 1 --2- 172.21.15.83:0/2960192827 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f698c109580 0x7f698c1a0c20 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:20.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.111+0000 7f6991e00640 1 --2- 172.21.15.83:0/2960192827 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f698c1044b0 0x7f698c1a06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:20.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.111+0000 7f6991e00640 1 -- 172.21.15.83:0/2960192827 >> 172.21.15.83:0/2960192827 conn(0x7f698c0ffd90 msgr2=0x7f698c1010a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:20.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.113+0000 7f6991e00640 1 -- 172.21.15.83:0/2960192827 shutdown_connections 2024-04-24T09:13:20.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.113+0000 7f6991e00640 1 -- 172.21.15.83:0/2960192827 wait complete. 2024-04-24T09:13:20.145 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T09:13:20.197 INFO:tasks.workunit.client.0.smithi083.stdout: 10 58565 6.63 MB/sec execute 351 sec latency 97.733 ms 2024-04-24T09:13:20.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.413+0000 7fd3ab400640 1 -- 172.21.15.83:0/825428560 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3a4100de0 msgr2=0x7fd3a41011c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:20.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.413+0000 7fd3ab400640 1 --2- 172.21.15.83:0/825428560 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3a4100de0 0x7fd3a41011c0 secure :-1 s=READY pgs=2618 cs=0 l=1 rev1=1 crypto rx=0x7fd398009900 tx=0x7fd39802a3f0 comp rx=0 tx=0).stop 2024-04-24T09:13:20.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.413+0000 7fd3ab400640 1 -- 172.21.15.83:0/825428560 shutdown_connections 2024-04-24T09:13:20.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.413+0000 7fd3ab400640 1 --2- 172.21.15.83:0/825428560 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd3a410e990 0x7fd3a4110d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:20.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.413+0000 7fd3ab400640 1 --2- 172.21.15.83:0/825428560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd3a4101700 0x7fd3a410e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:20.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.413+0000 7fd3ab400640 1 --2- 172.21.15.83:0/825428560 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3a4100de0 0x7fd3a41011c0 unknown :-1 s=CLOSED pgs=2618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:20.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.413+0000 7fd3ab400640 1 -- 172.21.15.83:0/825428560 >> 172.21.15.83:0/825428560 conn(0x7fd3a4077280 msgr2=0x7fd3a4077690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:20.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.414+0000 7fd3ab400640 1 -- 172.21.15.83:0/825428560 shutdown_connections 2024-04-24T09:13:20.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.414+0000 7fd3ab400640 1 -- 172.21.15.83:0/825428560 wait complete. 2024-04-24T09:13:20.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.415+0000 7fd3ab400640 1 Processor -- start 2024-04-24T09:13:20.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.415+0000 7fd3ab400640 1 -- start start 2024-04-24T09:13:20.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.415+0000 7fd3ab400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3a4100de0 0x7fd3a410ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:20.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.415+0000 7fd3ab400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd3a4101700 0x7fd3a410d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:20.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.415+0000 7fd3ab400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd3a410e990 0x7fd3a41aa9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:20.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.415+0000 7fd3ab400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd3a41040a0 con 0x7fd3a4100de0 2024-04-24T09:13:20.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.415+0000 7fd3ab400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd3a4103f20 con 0x7fd3a410e990 2024-04-24T09:13:20.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.415+0000 7fd3ab400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd3a4104220 con 0x7fd3a4101700 2024-04-24T09:13:20.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.416+0000 7fd3a8a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3a4100de0 0x7fd3a410ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:20.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.416+0000 7fd3a9400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd3a410e990 0x7fd3a41aa9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:20.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.416+0000 7fd3a9400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd3a410e990 0x7fd3a41aa9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:38102/0 (socket says 172.21.15.83:38102) 2024-04-24T09:13:20.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.416+0000 7fd3a9400640 1 -- 172.21.15.83:0/3256812871 learned_addr learned my addr 172.21.15.83:0/3256812871 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:13:20.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.416+0000 7fd3a3e00640 1 --2- 172.21.15.83:0/3256812871 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd3a4101700 0x7fd3a410d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:20.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.416+0000 7fd3a9400640 1 -- 172.21.15.83:0/3256812871 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd3a4101700 msgr2=0x7fd3a410d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:20.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.416+0000 7fd3a9400640 1 --2- 172.21.15.83:0/3256812871 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd3a4101700 0x7fd3a410d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:20.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.416+0000 7fd3a9400640 1 -- 172.21.15.83:0/3256812871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3a4100de0 msgr2=0x7fd3a410ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:20.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.416+0000 7fd3a9400640 1 --2- 172.21.15.83:0/3256812871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3a4100de0 0x7fd3a410ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:20.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.416+0000 7fd3a9400640 1 -- 172.21.15.83:0/3256812871 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3a41ab110 con 0x7fd3a410e990 2024-04-24T09:13:20.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.417+0000 7fd3a9400640 1 --2- 172.21.15.83:0/3256812871 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd3a410e990 0x7fd3a41aa9c0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fd39c00eb10 tx=0x7fd39c00c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:20.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.417+0000 7fd3a1600640 1 -- 172.21.15.83:0/3256812871 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd39c004b20 con 0x7fd3a410e990 2024-04-24T09:13:20.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.417+0000 7fd3ab400640 1 -- 172.21.15.83:0/3256812871 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3a41ab400 con 0x7fd3a410e990 2024-04-24T09:13:20.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.417+0000 7fd3ab400640 1 -- 172.21.15.83:0/3256812871 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3a41ab950 con 0x7fd3a410e990 2024-04-24T09:13:20.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.417+0000 7fd3a1600640 1 -- 172.21.15.83:0/3256812871 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd39c004cc0 con 0x7fd3a410e990 2024-04-24T09:13:20.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.417+0000 7fd3a1600640 1 -- 172.21.15.83:0/3256812871 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd39c010720 con 0x7fd3a410e990 2024-04-24T09:13:20.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.422+0000 7fd3a1600640 1 -- 172.21.15.83:0/3256812871 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd39c004fe0 con 0x7fd3a410e990 2024-04-24T09:13:20.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.422+0000 7fd3ae219740 1 Processor -- start 2024-04-24T09:13:20.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.423+0000 7fd3ae219740 1 -- start start 2024-04-24T09:13:20.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.423+0000 7fd3ae219740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b4ac669950 0x55b4ac669d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:20.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.423+0000 7fd3ae219740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b4ac66a270 0x55b4ac674400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:20.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.423+0000 7fd3ae219740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b4ac674940 0x55b4ac676d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:20.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.423+0000 7fd3ae219740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55b4ac5cf1d0 con 0x55b4ac66a270 2024-04-24T09:13:20.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.423+0000 7fd3ae219740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b4ac575450 con 0x55b4ac669950 2024-04-24T09:13:20.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.423+0000 7fd3ae219740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55b4ac5cb190 con 0x55b4ac674940 2024-04-24T09:13:20.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.424+0000 7fd3a8a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b4ac66a270 0x55b4ac674400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:20.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.424+0000 7fd3a3e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b4ac669950 0x55b4ac669d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:20.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.424+0000 7fd3a3e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b4ac669950 0x55b4ac669d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:38118/0 (socket says 172.21.15.83:38118) 2024-04-24T09:13:20.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.424+0000 7fd3a3e00640 1 -- 172.21.15.83:0/3329507993 learned_addr learned my addr 172.21.15.83:0/3329507993 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:13:20.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.424+0000 7fd3a3e00640 1 --2- 172.21.15.83:0/3329507993 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b4ac674940 0x55b4ac676d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:20.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.424+0000 7fd3a3e00640 1 -- 172.21.15.83:0/3329507993 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b4ac674940 msgr2=0x55b4ac676d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:20.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.424+0000 7fd3a3e00640 1 --2- 172.21.15.83:0/3329507993 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b4ac674940 0x55b4ac676d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:20.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.424+0000 7fd3a3e00640 1 -- 172.21.15.83:0/3329507993 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b4ac66a270 msgr2=0x55b4ac674400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:20.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.424+0000 7fd3a3e00640 1 --2- 172.21.15.83:0/3329507993 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b4ac66a270 0x55b4ac674400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:20.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.425+0000 7fd3a3e00640 1 -- 172.21.15.83:0/3329507993 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4ac632310 con 0x55b4ac669950 2024-04-24T09:13:20.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.425+0000 7fd3a8a00640 1 --2- 172.21.15.83:0/3329507993 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b4ac66a270 0x55b4ac674400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:13:20.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.425+0000 7fd3a3e00640 1 --2- 172.21.15.83:0/3329507993 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b4ac669950 0x55b4ac669d30 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fd390009de0 tx=0x7fd390009730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2e438f8df22559b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:20.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.425+0000 7fd395c00640 1 -- 172.21.15.83:0/3329507993 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd390012e60 con 0x55b4ac669950 2024-04-24T09:13:20.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.425+0000 7fd3a1600640 1 --2- 172.21.15.83:0/3256812871 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fd38c0a3d50 0x7fd38c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:20.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.425+0000 7fd395c00640 1 -- 172.21.15.83:0/3329507993 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd390004220 con 0x55b4ac669950 2024-04-24T09:13:20.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.425+0000 7fd395c00640 1 -- 172.21.15.83:0/3329507993 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd39000f7c0 con 0x55b4ac669950 2024-04-24T09:13:20.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.426+0000 7fd3a1600640 1 -- 172.21.15.83:0/3256812871 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 9583+0+0 (secure 0 0 0) 0x7fd39c0d1350 con 0x7fd3a410e990 2024-04-24T09:13:20.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.426+0000 7fd3a8a00640 1 --2- 172.21.15.83:0/3256812871 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fd38c0a3d50 0x7fd38c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:20.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.427+0000 7fd3ae219740 1 -- 172.21.15.83:0/3329507993 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b4ac669950 msgr2=0x55b4ac669d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:20.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.427+0000 7fd3ae219740 1 --2- 172.21.15.83:0/3329507993 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b4ac669950 0x55b4ac669d30 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fd390009de0 tx=0x7fd390009730 comp rx=0 tx=0).stop 2024-04-24T09:13:20.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.427+0000 7fd3ae219740 1 -- 172.21.15.83:0/3329507993 shutdown_connections 2024-04-24T09:13:20.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.427+0000 7fd3ae219740 1 --2- 172.21.15.83:0/3329507993 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b4ac674940 0x55b4ac676d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:20.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.427+0000 7fd3ae219740 1 --2- 172.21.15.83:0/3329507993 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b4ac66a270 0x55b4ac674400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:20.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.427+0000 7fd3ae219740 1 --2- 172.21.15.83:0/3329507993 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b4ac669950 0x55b4ac669d30 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:20.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.427+0000 7fd3ae219740 1 -- 172.21.15.83:0/3329507993 >> 172.21.15.83:0/3329507993 conn(0x55b4ac665c70 msgr2=0x55b4ac666050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:20.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.428+0000 7fd3a8a00640 1 --2- 172.21.15.83:0/3256812871 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fd38c0a3d50 0x7fd38c0a6210 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fd398006010 tx=0x7fd398005a20 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:20.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.428+0000 7fd3ae219740 1 -- 172.21.15.83:0/3329507993 shutdown_connections 2024-04-24T09:13:20.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.428+0000 7fd3ae219740 1 -- 172.21.15.83:0/3329507993 wait complete. 2024-04-24T09:13:20.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.429+0000 7fd3ae219740 1 -- start start 2024-04-24T09:13:20.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.429+0000 7fd3ae219740 1 Processor -- start 2024-04-24T09:13:20.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.430+0000 7fd3ae219740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b4ac669950 0x55b4ac6743a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:20.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.430+0000 7fd3ae219740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b4ac66a270 0x55b4ac671690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:20.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.430+0000 7fd3ae219740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b4ac674940 0x55b4ac671db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:20.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.430+0000 7fd3ae219740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55b4ac6777d0 con 0x55b4ac674940 2024-04-24T09:13:20.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.430+0000 7fd3ae219740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b4ac588810 con 0x55b4ac669950 2024-04-24T09:13:20.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.430+0000 7fd3ae219740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55b4ac677950 con 0x55b4ac66a270 2024-04-24T09:13:20.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.431+0000 7fd3ae219740 10 client.0 _ll_register_callbacks cb 0x55b4ac556140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:13:20.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.431+0000 7fd3a9400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b4ac674940 0x55b4ac671db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:20.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.431+0000 7fd3a9400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b4ac674940 0x55b4ac671db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35270/0 (socket says 172.21.15.83:35270) 2024-04-24T09:13:20.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.431+0000 7fd3a9400640 1 -- 172.21.15.83:0/2243419653 learned_addr learned my addr 172.21.15.83:0/2243419653 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:13:20.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.431+0000 7fd3a3e00640 1 --2- 172.21.15.83:0/2243419653 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b4ac66a270 0x55b4ac671690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:20.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.431+0000 7fd3a8a00640 1 --2- 172.21.15.83:0/2243419653 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b4ac669950 0x55b4ac6743a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:20.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.432+0000 7fd3a9400640 1 -- 172.21.15.83:0/2243419653 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b4ac66a270 msgr2=0x55b4ac671690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:20.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.432+0000 7fd3a9400640 1 --2- 172.21.15.83:0/2243419653 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b4ac66a270 0x55b4ac671690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:20.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.432+0000 7fd3a9400640 1 -- 172.21.15.83:0/2243419653 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b4ac669950 msgr2=0x55b4ac6743a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:20.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.432+0000 7fd3a9400640 1 --2- 172.21.15.83:0/2243419653 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b4ac669950 0x55b4ac6743a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:20.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.432+0000 7fd3a9400640 1 -- 172.21.15.83:0/2243419653 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4ac5993f0 con 0x55b4ac674940 2024-04-24T09:13:20.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.432+0000 7fd3a9400640 1 --2- 172.21.15.83:0/2243419653 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b4ac674940 0x55b4ac671db0 secure :-1 s=READY pgs=2619 cs=0 l=1 rev1=1 crypto rx=0x7fd39c0a5610 tx=0x7fd39c09c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ef4603a9fe273df server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:20.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.432+0000 7fd38a000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:13:20.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.433+0000 7fd38a000640 1 -- 172.21.15.83:0/2243419653 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd39c0b5020 con 0x55b4ac674940 2024-04-24T09:13:20.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.433+0000 7fd38a000640 1 -- 172.21.15.83:0/2243419653 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9583+0+0 (secure 0 0 0) 0x7fd39c09d130 con 0x55b4ac674940 2024-04-24T09:13:20.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.433+0000 7fd3ae219740 1 -- 172.21.15.83:0/2243419653 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd390004220 con 0x55b4ac674940 2024-04-24T09:13:20.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.433+0000 7fd38a000640 1 -- 172.21.15.83:0/2243419653 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7fd39c0b6450 con 0x55b4ac674940 2024-04-24T09:13:20.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.433+0000 7fd3ae219740 10 client.23040 fetch_fsmap learned FSMap version 64 2024-04-24T09:13:20.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.433+0000 7fd3ae219740 1 -- 172.21.15.83:0/2243419653 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55b4ac2e13f0 con 0x55b4ac674940 2024-04-24T09:13:20.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.434+0000 7fd38a000640 1 -- 172.21.15.83:0/2243419653 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd390004220 con 0x55b4ac674940 2024-04-24T09:13:20.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.434+0000 7fd38a000640 1 -- 172.21.15.83:0/2243419653 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 64) ==== 8374+0+0 (secure 0 0 0) 0x7fd39c0a56b0 con 0x55b4ac674940 2024-04-24T09:13:20.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.434+0000 7fd3ae219740 10 client.23040 fetch_fsmap finished waiting for FSMap version 64 2024-04-24T09:13:20.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.434+0000 7fd3ae219740 10 client.23040 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:13:20.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.434+0000 7fd3ae219740 20 client.23040 populate_metadata read hostname 'smithi083' 2024-04-24T09:13:20.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.434+0000 7fd3ae219740 1 --2- 172.21.15.83:0/2243419653 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55b4ac684c10 0x55b4ac687030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:20.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.434+0000 7fd3ae219740 4 client.23040 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:13:20.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.434+0000 7fd3ae219740 1 -- 172.21.15.83:0/2243419653 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b4ac5dcbf0 con 0x55b4ac684c10 2024-04-24T09:13:20.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.435+0000 7fd3a8a00640 1 --2- 172.21.15.83:0/2243419653 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55b4ac684c10 0x55b4ac687030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:20.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.435+0000 7fd3a8a00640 1 --2- 172.21.15.83:0/2243419653 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55b4ac684c10 0x55b4ac687030 crc :-1 s=READY pgs=535 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42d67eb594d8cc1f server_cookie=964d90a52372a469 in_seq=0 out_seq=0 2024-04-24T09:13:20.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.435+0000 7fd38a000640 10 client.23040 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:13:20.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.436+0000 7fd38a000640 1 -- 172.21.15.83:0/2243419653 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b4ac5dcbf0 con 0x55b4ac684c10 2024-04-24T09:13:20.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.436+0000 7fd38a000640 10 client.23040 handle_command_reply: tid=0 2024-04-24T09:13:20.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.436+0000 7fd3ae219740 1 client.23040 shutdown 2024-04-24T09:13:20.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.437+0000 7fd3ae219740 1 -- 172.21.15.83:0/2243419653 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b4ac674940 msgr2=0x55b4ac671db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:20.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.437+0000 7fd3ae219740 1 --2- 172.21.15.83:0/2243419653 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b4ac674940 0x55b4ac671db0 secure :-1 s=READY pgs=2619 cs=0 l=1 rev1=1 crypto rx=0x7fd39c0a5610 tx=0x7fd39c09c960 comp rx=0 tx=0).stop 2024-04-24T09:13:20.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.437+0000 7fd3ae219740 1 -- 172.21.15.83:0/2243419653 shutdown_connections 2024-04-24T09:13:20.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.437+0000 7fd3ae219740 1 --2- 172.21.15.83:0/2243419653 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55b4ac684c10 0x55b4ac687030 crc :-1 s=READY pgs=535 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:20.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.437+0000 7fd3ae219740 1 --2- 172.21.15.83:0/2243419653 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b4ac674940 0x55b4ac671db0 unknown :-1 s=CLOSED pgs=2619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:20.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.437+0000 7fd3ae219740 1 --2- 172.21.15.83:0/2243419653 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b4ac66a270 0x55b4ac671690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:20.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.437+0000 7fd38a000640 0 client.23040 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:13:20.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.437+0000 7fd3ae219740 1 --2- 172.21.15.83:0/2243419653 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b4ac669950 0x55b4ac6743a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:20.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.437+0000 7fd3ae219740 1 -- 172.21.15.83:0/2243419653 >> 172.21.15.83:0/2243419653 conn(0x55b4ac665c70 msgr2=0x55b4ac66b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:20.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.437+0000 7fd3ae219740 1 -- 172.21.15.83:0/2243419653 shutdown_connections 2024-04-24T09:13:20.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.437+0000 7fd3ae219740 1 -- 172.21.15.83:0/2243419653 wait complete. 2024-04-24T09:13:20.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.437+0000 7fd3ae219740 20 client.23040 trim_cache size 0 max 16384 2024-04-24T09:13:20.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:13:20 smithi138 ceph-mon[31441]: pgmap v686: 129 pgs: 129 active+clean; 912 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.9 MiB/s wr, 249 op/s 2024-04-24T09:13:20.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:13:20 smithi189 ceph-mon[31791]: pgmap v686: 129 pgs: 129 active+clean; 912 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.9 MiB/s wr, 249 op/s 2024-04-24T09:13:20.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.460+0000 7fd3ae219740 1 Processor -- start 2024-04-24T09:13:20.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.461+0000 7fd3ae219740 1 -- start start 2024-04-24T09:13:20.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.461+0000 7fd3ae219740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b4ac669950 0x55b4ac680720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:20.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.461+0000 7fd3ae219740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b4ac66a270 0x55b4ac66c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:20.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.461+0000 7fd3ae219740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b4ac674940 0x55b4ac673ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:20.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.461+0000 7fd3ae219740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd384010100 con 0x55b4ac674940 2024-04-24T09:13:20.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.461+0000 7fd3ae219740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd384011cd0 con 0x55b4ac669950 2024-04-24T09:13:20.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.461+0000 7fd3ae219740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd3840107d0 con 0x55b4ac66a270 2024-04-24T09:13:20.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.461+0000 7fd3a8a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b4ac669950 0x55b4ac680720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:20.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.461+0000 7fd3a8a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b4ac669950 0x55b4ac680720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:38142/0 (socket says 172.21.15.83:38142) 2024-04-24T09:13:20.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.461+0000 7fd3a8a00640 1 -- 172.21.15.83:0/408242470 learned_addr learned my addr 172.21.15.83:0/408242470 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:13:20.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.461+0000 7fd3a9400640 1 --2- 172.21.15.83:0/408242470 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b4ac674940 0x55b4ac673ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:20.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.462+0000 7fd3a3e00640 1 --2- 172.21.15.83:0/408242470 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b4ac66a270 0x55b4ac66c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:20.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.462+0000 7fd3a8a00640 1 -- 172.21.15.83:0/408242470 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b4ac66a270 msgr2=0x55b4ac66c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:20.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.462+0000 7fd3a8a00640 1 --2- 172.21.15.83:0/408242470 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b4ac66a270 0x55b4ac66c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:20.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.462+0000 7fd3a8a00640 1 -- 172.21.15.83:0/408242470 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b4ac674940 msgr2=0x55b4ac673ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:20.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.462+0000 7fd3a8a00640 1 --2- 172.21.15.83:0/408242470 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b4ac674940 0x55b4ac673ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:20.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.462+0000 7fd3a8a00640 1 -- 172.21.15.83:0/408242470 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4ac66a0a0 con 0x55b4ac669950 2024-04-24T09:13:20.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.462+0000 7fd3a9400640 1 --2- 172.21.15.83:0/408242470 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b4ac674940 0x55b4ac673ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:13:20.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.462+0000 7fd3a8a00640 1 --2- 172.21.15.83:0/408242470 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b4ac669950 0x55b4ac680720 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fd39802a910 tx=0x7fd39803f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f86ac25870534808 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:20.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.463+0000 7fd395200640 1 -- 172.21.15.83:0/408242470 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd398035050 con 0x55b4ac669950 2024-04-24T09:13:20.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.463+0000 7fd395200640 1 -- 172.21.15.83:0/408242470 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd398056070 con 0x55b4ac669950 2024-04-24T09:13:20.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.463+0000 7fd395200640 1 -- 172.21.15.83:0/408242470 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3980046c0 con 0x55b4ac669950 2024-04-24T09:13:20.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.464+0000 7fd3ae219740 1 -- 172.21.15.83:0/408242470 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b4ac669950 msgr2=0x55b4ac680720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:20.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.464+0000 7fd3ae219740 1 --2- 172.21.15.83:0/408242470 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b4ac669950 0x55b4ac680720 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fd39802a910 tx=0x7fd39803f000 comp rx=0 tx=0).stop 2024-04-24T09:13:20.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.464+0000 7fd3ae219740 1 -- 172.21.15.83:0/408242470 shutdown_connections 2024-04-24T09:13:20.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.464+0000 7fd3ae219740 1 --2- 172.21.15.83:0/408242470 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b4ac674940 0x55b4ac673ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:20.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.464+0000 7fd3ae219740 1 --2- 172.21.15.83:0/408242470 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b4ac66a270 0x55b4ac66c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:20.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.464+0000 7fd3ae219740 1 --2- 172.21.15.83:0/408242470 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b4ac669950 0x55b4ac680720 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:20.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.464+0000 7fd3ae219740 1 -- 172.21.15.83:0/408242470 >> 172.21.15.83:0/408242470 conn(0x55b4ac665c70 msgr2=0x55b4ac66b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:20.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.464+0000 7fd3ae219740 1 -- 172.21.15.83:0/408242470 shutdown_connections 2024-04-24T09:13:20.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.465+0000 7fd3ae219740 1 -- 172.21.15.83:0/408242470 wait complete. 2024-04-24T09:13:20.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.465+0000 7fd3ae219740 1 -- start start 2024-04-24T09:13:20.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.465+0000 7fd3ae219740 1 Processor -- start 2024-04-24T09:13:20.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.466+0000 7fd3ae219740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b4ac669950 0x55b4ac6758e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:20.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.466+0000 7fd3ae219740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b4ac66a270 0x55b4ac675e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:20.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.466+0000 7fd3ae219740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b4ac674940 0x55b4ac676360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:20.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.466+0000 7fd3ae219740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd3840115d0 con 0x55b4ac669950 2024-04-24T09:13:20.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.466+0000 7fd3ae219740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd384010f00 con 0x55b4ac674940 2024-04-24T09:13:20.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.466+0000 7fd3ae219740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd38400e100 con 0x55b4ac66a270 2024-04-24T09:13:20.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.466+0000 7fd3ae219740 10 client.0 _ll_register_callbacks cb 0x7fd39c004a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:13:20.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.466+0000 7fd3a8a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b4ac669950 0x55b4ac6758e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:20.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.466+0000 7fd3a8a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b4ac669950 0x55b4ac6758e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35284/0 (socket says 172.21.15.83:35284) 2024-04-24T09:13:20.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.466+0000 7fd3a8a00640 1 -- 172.21.15.83:0/3172862100 learned_addr learned my addr 172.21.15.83:0/3172862100 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:13:20.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.466+0000 7fd3a9400640 1 --2- 172.21.15.83:0/3172862100 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b4ac674940 0x55b4ac676360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:20.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.467+0000 7fd3a8a00640 1 -- 172.21.15.83:0/3172862100 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b4ac66a270 msgr2=0x55b4ac675e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:20.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.467+0000 7fd3a8a00640 1 --2- 172.21.15.83:0/3172862100 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b4ac66a270 0x55b4ac675e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:20.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.467+0000 7fd3a8a00640 1 -- 172.21.15.83:0/3172862100 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b4ac674940 msgr2=0x55b4ac676360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:20.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.467+0000 7fd3a8a00640 1 --2- 172.21.15.83:0/3172862100 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b4ac674940 0x55b4ac676360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:20.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.467+0000 7fd3a8a00640 1 -- 172.21.15.83:0/3172862100 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4ac3ab7d0 con 0x55b4ac669950 2024-04-24T09:13:20.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.467+0000 7fd3a8a00640 1 --2- 172.21.15.83:0/3172862100 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b4ac669950 0x55b4ac6758e0 secure :-1 s=READY pgs=2620 cs=0 l=1 rev1=1 crypto rx=0x7fd398053040 tx=0x7fd398009230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9848d8994c1a571 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:20.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.467+0000 7fd38a000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:13:20.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.468+0000 7fd38a000640 1 -- 172.21.15.83:0/3172862100 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd39805f060 con 0x55b4ac669950 2024-04-24T09:13:20.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.468+0000 7fd38a000640 1 -- 172.21.15.83:0/3172862100 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9583+0+0 (secure 0 0 0) 0x7fd3980046c0 con 0x55b4ac669950 2024-04-24T09:13:20.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.468+0000 7fd3ae219740 1 -- 172.21.15.83:0/3172862100 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd398056070 con 0x55b4ac669950 2024-04-24T09:13:20.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.468+0000 7fd38a000640 1 -- 172.21.15.83:0/3172862100 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7fd398066050 con 0x55b4ac669950 2024-04-24T09:13:20.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.468+0000 7fd3ae219740 10 client.23046 fetch_fsmap learned FSMap version 64 2024-04-24T09:13:20.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.468+0000 7fd3ae219740 1 -- 172.21.15.83:0/3172862100 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55b4ac6eb4f0 con 0x55b4ac669950 2024-04-24T09:13:20.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.469+0000 7fd38a000640 1 -- 172.21.15.83:0/3172862100 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd398056070 con 0x55b4ac669950 2024-04-24T09:13:20.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.469+0000 7fd38a000640 1 -- 172.21.15.83:0/3172862100 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 64) ==== 8374+0+0 (secure 0 0 0) 0x7fd3980506b0 con 0x55b4ac669950 2024-04-24T09:13:20.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.469+0000 7fd3ae219740 10 client.23046 fetch_fsmap finished waiting for FSMap version 64 2024-04-24T09:13:20.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.469+0000 7fd3ae219740 10 client.23046 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:13:20.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.469+0000 7fd3ae219740 20 client.23046 populate_metadata read hostname 'smithi083' 2024-04-24T09:13:20.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.469+0000 7fd3ae219740 1 --2- 172.21.15.83:0/3172862100 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55b4ac6ebbb0 0x55b4ac6edfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:20.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.469+0000 7fd3ae219740 4 client.23046 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:13:20.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.469+0000 7fd3ae219740 1 -- 172.21.15.83:0/3172862100 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b4ac5d6a60 con 0x55b4ac6ebbb0 2024-04-24T09:13:20.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.470+0000 7fd3a3e00640 1 --2- 172.21.15.83:0/3172862100 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55b4ac6ebbb0 0x55b4ac6edfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:20.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.470+0000 7fd38a000640 10 client.23046 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:13:20.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.470+0000 7fd3a3e00640 1 --2- 172.21.15.83:0/3172862100 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55b4ac6ebbb0 0x55b4ac6edfd0 crc :-1 s=READY pgs=536 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2692fd1e5e4137e server_cookie=8ef62f2acd4f019e in_seq=0 out_seq=0 2024-04-24T09:13:20.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.472+0000 7fd38a000640 1 -- 172.21.15.83:0/3172862100 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b4ac5d6a60 con 0x55b4ac6ebbb0 2024-04-24T09:13:20.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.472+0000 7fd38a000640 10 client.23046 handle_command_reply: tid=0 2024-04-24T09:13:20.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.472+0000 7fd3ae219740 1 client.23046 shutdown 2024-04-24T09:13:20.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.473+0000 7fd3ae219740 1 -- 172.21.15.83:0/3172862100 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b4ac669950 msgr2=0x55b4ac6758e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:20.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.473+0000 7fd3ae219740 1 --2- 172.21.15.83:0/3172862100 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b4ac669950 0x55b4ac6758e0 secure :-1 s=READY pgs=2620 cs=0 l=1 rev1=1 crypto rx=0x7fd398053040 tx=0x7fd398009230 comp rx=0 tx=0).stop 2024-04-24T09:13:20.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.473+0000 7fd3ae219740 1 -- 172.21.15.83:0/3172862100 shutdown_connections 2024-04-24T09:13:20.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.473+0000 7fd3ae219740 1 --2- 172.21.15.83:0/3172862100 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55b4ac6ebbb0 0x55b4ac6edfd0 crc :-1 s=READY pgs=536 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:20.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.473+0000 7fd3ae219740 1 --2- 172.21.15.83:0/3172862100 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b4ac674940 0x55b4ac676360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:20.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.473+0000 7fd38a000640 0 client.23046 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:13:20.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.473+0000 7fd3ae219740 1 --2- 172.21.15.83:0/3172862100 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b4ac66a270 0x55b4ac675e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:20.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.473+0000 7fd3ae219740 1 --2- 172.21.15.83:0/3172862100 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b4ac669950 0x55b4ac6758e0 unknown :-1 s=CLOSED pgs=2620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:20.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.473+0000 7fd3ae219740 1 -- 172.21.15.83:0/3172862100 >> 172.21.15.83:0/3172862100 conn(0x55b4ac665c70 msgr2=0x55b4ac67f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:20.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.473+0000 7fd3ae219740 1 -- 172.21.15.83:0/3172862100 shutdown_connections 2024-04-24T09:13:20.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.473+0000 7fd3ae219740 1 -- 172.21.15.83:0/3172862100 wait complete. 2024-04-24T09:13:20.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.473+0000 7fd3ae219740 20 client.23046 trim_cache size 0 max 16384 2024-04-24T09:13:20.475 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:13:20.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.475+0000 7fd3ab400640 1 -- 172.21.15.83:0/3256812871 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fd38c0a3d50 msgr2=0x7fd38c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:20.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.475+0000 7fd3ab400640 1 --2- 172.21.15.83:0/3256812871 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fd38c0a3d50 0x7fd38c0a6210 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fd398006010 tx=0x7fd398005a20 comp rx=0 tx=0).stop 2024-04-24T09:13:20.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.475+0000 7fd3ab400640 1 -- 172.21.15.83:0/3256812871 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd3a410e990 msgr2=0x7fd3a41aa9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:20.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.475+0000 7fd3ab400640 1 --2- 172.21.15.83:0/3256812871 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd3a410e990 0x7fd3a41aa9c0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fd39c00eb10 tx=0x7fd39c00c2d0 comp rx=0 tx=0).stop 2024-04-24T09:13:20.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.475+0000 7fd3ab400640 1 -- 172.21.15.83:0/3256812871 shutdown_connections 2024-04-24T09:13:20.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.475+0000 7fd3ab400640 1 --2- 172.21.15.83:0/3256812871 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fd38c0a3d50 0x7fd38c0a6210 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:20.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.475+0000 7fd3ab400640 1 --2- 172.21.15.83:0/3256812871 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd3a410e990 0x7fd3a41aa9c0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:20.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.475+0000 7fd3ab400640 1 --2- 172.21.15.83:0/3256812871 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd3a4101700 0x7fd3a410d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:20.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.475+0000 7fd3ab400640 1 --2- 172.21.15.83:0/3256812871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3a4100de0 0x7fd3a410ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:20.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.475+0000 7fd3ab400640 1 -- 172.21.15.83:0/3256812871 >> 172.21.15.83:0/3256812871 conn(0x7fd3a4077280 msgr2=0x7fd3a4110000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:20.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.475+0000 7fd3ab400640 1 -- 172.21.15.83:0/3256812871 shutdown_connections 2024-04-24T09:13:20.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:20.475+0000 7fd3ab400640 1 -- 172.21.15.83:0/3256812871 wait complete. 2024-04-24T09:13:20.495 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:13:20 smithi083 ceph-mon[27907]: pgmap v686: 129 pgs: 129 active+clean; 912 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.9 MiB/s wr, 249 op/s 2024-04-24T09:13:21.197 INFO:tasks.workunit.client.0.smithi083.stdout: 10 58632 6.62 MB/sec execute 352 sec latency 451.266 ms 2024-04-24T09:13:21.498 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T09:13:21.498 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T09:13:21.499 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T09:13:21.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.771+0000 7f7d8aa00640 1 -- 172.21.15.83:0/752748475 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d84102240 msgr2=0x7f7d84102620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:21.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.771+0000 7f7d8aa00640 1 --2- 172.21.15.83:0/752748475 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d84102240 0x7f7d84102620 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f7d74009900 tx=0x7f7d7402a3f0 comp rx=0 tx=0).stop 2024-04-24T09:13:21.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.771+0000 7f7d8aa00640 1 -- 172.21.15.83:0/752748475 shutdown_connections 2024-04-24T09:13:21.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.771+0000 7f7d8aa00640 1 --2- 172.21.15.83:0/752748475 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7d84107310 0x7f7d8410eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:21.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.771+0000 7f7d8aa00640 1 --2- 172.21.15.83:0/752748475 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d84102bf0 0x7f7d84106c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:21.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.771+0000 7f7d8aa00640 1 --2- 172.21.15.83:0/752748475 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d84102240 0x7f7d84102620 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:21.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.771+0000 7f7d8aa00640 1 -- 172.21.15.83:0/752748475 >> 172.21.15.83:0/752748475 conn(0x7f7d84079450 msgr2=0x7f7d84079860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:21.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.771+0000 7f7d8aa00640 1 -- 172.21.15.83:0/752748475 shutdown_connections 2024-04-24T09:13:21.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.771+0000 7f7d8aa00640 1 -- 172.21.15.83:0/752748475 wait complete. 2024-04-24T09:13:21.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.772+0000 7f7d8aa00640 1 Processor -- start 2024-04-24T09:13:21.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.772+0000 7f7d8aa00640 1 -- start start 2024-04-24T09:13:21.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.773+0000 7f7d8aa00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7d84102240 0x7f7d841a06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:21.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.773+0000 7f7d8aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d84107310 0x7f7d841a0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:21.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.773+0000 7f7d8aa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d8419a880 0x7f7d8419ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:21.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.773+0000 7f7d83e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7d84102240 0x7f7d841a06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:21.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.773+0000 7f7d83e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7d84102240 0x7f7d841a06d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:47924/0 (socket says 172.21.15.83:47924) 2024-04-24T09:13:21.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.773+0000 7f7d83e00640 1 -- 172.21.15.83:0/3018761583 learned_addr learned my addr 172.21.15.83:0/3018761583 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:13:21.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.773+0000 7f7d8aa00640 1 -- 172.21.15.83:0/3018761583 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7d841144b0 con 0x7f7d84107310 2024-04-24T09:13:21.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.773+0000 7f7d8aa00640 1 -- 172.21.15.83:0/3018761583 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7d84114330 con 0x7f7d8419a880 2024-04-24T09:13:21.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.773+0000 7f7d8aa00640 1 -- 172.21.15.83:0/3018761583 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7d84114630 con 0x7f7d84102240 2024-04-24T09:13:21.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.774+0000 7f7d7be00640 1 --2- 172.21.15.83:0/3018761583 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d84107310 0x7f7d841a0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:21.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.774+0000 7f7d83e00640 1 -- 172.21.15.83:0/3018761583 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d8419a880 msgr2=0x7f7d8419ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:21.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.774+0000 7f7d83e00640 1 --2- 172.21.15.83:0/3018761583 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d8419a880 0x7f7d8419ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:21.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.774+0000 7f7d83e00640 1 -- 172.21.15.83:0/3018761583 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d84107310 msgr2=0x7f7d841a0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:21.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.774+0000 7f7d83e00640 1 --2- 172.21.15.83:0/3018761583 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d84107310 0x7f7d841a0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:21.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.774+0000 7f7d83e00640 1 -- 172.21.15.83:0/3018761583 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d8419b3a0 con 0x7f7d84102240 2024-04-24T09:13:21.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.774+0000 7f7d7be00640 1 --2- 172.21.15.83:0/3018761583 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d84107310 0x7f7d841a0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:13:21.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.774+0000 7f7d83e00640 1 --2- 172.21.15.83:0/3018761583 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7d84102240 0x7f7d841a06d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7d74005940 tx=0x7f7d740040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:21.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.775+0000 7f7d81600640 1 -- 172.21.15.83:0/3018761583 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d7403f030 con 0x7f7d84102240 2024-04-24T09:13:21.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.775+0000 7f7d8aa00640 1 -- 172.21.15.83:0/3018761583 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d8419b630 con 0x7f7d84102240 2024-04-24T09:13:21.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.775+0000 7f7d81600640 1 -- 172.21.15.83:0/3018761583 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7d740044c0 con 0x7f7d84102240 2024-04-24T09:13:21.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.775+0000 7f7d81600640 1 -- 172.21.15.83:0/3018761583 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d74035070 con 0x7f7d84102240 2024-04-24T09:13:21.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.775+0000 7f7d8aa00640 1 -- 172.21.15.83:0/3018761583 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d841a7880 con 0x7f7d84102240 2024-04-24T09:13:21.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.781+0000 7f7d8d781740 1 Processor -- start 2024-04-24T09:13:21.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.781+0000 7f7d8d781740 1 -- start start 2024-04-24T09:13:21.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.782+0000 7f7d8d781740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef18ff18a0 0x55ef18ff1c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:21.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.782+0000 7f7d8d781740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef18ff21c0 0x55ef18ffc350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:21.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.782+0000 7f7d8d781740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef18ffc890 0x55ef18ffec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:21.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.782+0000 7f7d8d781740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ef18f4ee70 con 0x55ef18ff21c0 2024-04-24T09:13:21.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.782+0000 7f7d8d781740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ef18ef5380 con 0x55ef18ff18a0 2024-04-24T09:13:21.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.782+0000 7f7d8d781740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55ef18f4ae30 con 0x55ef18ffc890 2024-04-24T09:13:21.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.782+0000 7f7d81600640 1 -- 172.21.15.83:0/3018761583 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7d74042030 con 0x7f7d84102240 2024-04-24T09:13:21.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.782+0000 7f7d83e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef18ff21c0 0x55ef18ffc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:21.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.783+0000 7f7d83e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef18ff21c0 0x55ef18ffc350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35304/0 (socket says 172.21.15.83:35304) 2024-04-24T09:13:21.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.783+0000 7f7d83e00640 1 -- 172.21.15.83:0/3668177053 learned_addr learned my addr 172.21.15.83:0/3668177053 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:13:21.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.783+0000 7f7d88a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef18ff18a0 0x55ef18ff1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:21.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.783+0000 7f7d7be00640 1 --2- 172.21.15.83:0/3668177053 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef18ffc890 0x55ef18ffec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:21.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.784+0000 7f7d88a00640 1 -- 172.21.15.83:0/3668177053 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef18ffc890 msgr2=0x55ef18ffec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:21.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.784+0000 7f7d88a00640 1 --2- 172.21.15.83:0/3668177053 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef18ffc890 0x55ef18ffec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:21.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.784+0000 7f7d88a00640 1 -- 172.21.15.83:0/3668177053 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef18ff21c0 msgr2=0x55ef18ffc350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:21.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.784+0000 7f7d88a00640 1 --2- 172.21.15.83:0/3668177053 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef18ff21c0 0x55ef18ffc350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:21.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.784+0000 7f7d88a00640 1 -- 172.21.15.83:0/3668177053 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef18d297e0 con 0x55ef18ff18a0 2024-04-24T09:13:21.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.784+0000 7f7d88a00640 1 --2- 172.21.15.83:0/3668177053 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef18ff18a0 0x55ef18ff1c80 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c009d70 tx=0x7f7d7c00e720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=791c713a102b2d3d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:21.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.785+0000 7f7d79200640 1 -- 172.21.15.83:0/3668177053 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d7c004b20 con 0x55ef18ff18a0 2024-04-24T09:13:21.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.785+0000 7f7d79200640 1 -- 172.21.15.83:0/3668177053 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7d7c003e20 con 0x55ef18ff18a0 2024-04-24T09:13:21.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.785+0000 7f7d79200640 1 -- 172.21.15.83:0/3668177053 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d7c0047c0 con 0x55ef18ff18a0 2024-04-24T09:13:21.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.785+0000 7f7d81600640 1 --2- 172.21.15.83:0/3018761583 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f7d680a3e20 0x7f7d680a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:21.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.785+0000 7f7d81600640 1 -- 172.21.15.83:0/3018761583 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 9583+0+0 (secure 0 0 0) 0x7f7d740f0ae0 con 0x7f7d84102240 2024-04-24T09:13:21.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.786+0000 7f7d83e00640 1 --2- 172.21.15.83:0/3018761583 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f7d680a3e20 0x7f7d680a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:21.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.789+0000 7f7d83e00640 1 --2- 172.21.15.83:0/3018761583 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f7d680a3e20 0x7f7d680a62e0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f7d740bc300 tx=0x7f7d7402a3f0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:21.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.789+0000 7f7d8d781740 1 -- 172.21.15.83:0/3668177053 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef18ff18a0 msgr2=0x55ef18ff1c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:21.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.789+0000 7f7d8d781740 1 --2- 172.21.15.83:0/3668177053 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef18ff18a0 0x55ef18ff1c80 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c009d70 tx=0x7f7d7c00e720 comp rx=0 tx=0).stop 2024-04-24T09:13:21.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.789+0000 7f7d8d781740 1 -- 172.21.15.83:0/3668177053 shutdown_connections 2024-04-24T09:13:21.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.789+0000 7f7d8d781740 1 --2- 172.21.15.83:0/3668177053 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef18ffc890 0x55ef18ffec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:21.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.789+0000 7f7d8d781740 1 --2- 172.21.15.83:0/3668177053 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef18ff21c0 0x55ef18ffc350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:21.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.790+0000 7f7d8d781740 1 --2- 172.21.15.83:0/3668177053 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef18ff18a0 0x55ef18ff1c80 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:21.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.790+0000 7f7d8d781740 1 -- 172.21.15.83:0/3668177053 >> 172.21.15.83:0/3668177053 conn(0x55ef18fedbc0 msgr2=0x55ef18fedfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:21.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.790+0000 7f7d8d781740 1 -- 172.21.15.83:0/3668177053 shutdown_connections 2024-04-24T09:13:21.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.791+0000 7f7d8d781740 1 -- 172.21.15.83:0/3668177053 wait complete. 2024-04-24T09:13:21.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.792+0000 7f7d8d781740 1 -- start start 2024-04-24T09:13:21.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.793+0000 7f7d8d781740 1 Processor -- start 2024-04-24T09:13:21.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.794+0000 7f7d8d781740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef18ff18a0 0x55ef18ff97a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:21.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.794+0000 7f7d8d781740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef18ff21c0 0x55ef18ff9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:21.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.794+0000 7f7d8d781740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef18ffc890 0x55ef18ffa220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:21.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.794+0000 7f7d8d781740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ef19010690 con 0x55ef18ff21c0 2024-04-24T09:13:21.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.794+0000 7f7d8d781740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ef18f08660 con 0x55ef18ffc890 2024-04-24T09:13:21.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.794+0000 7f7d8d781740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55ef19010810 con 0x55ef18ff18a0 2024-04-24T09:13:21.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.794+0000 7f7d8d781740 10 client.0 _ll_register_callbacks cb 0x55ef18ef6aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:13:21.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.795+0000 7f7d88a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef18ff18a0 0x55ef18ff97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:21.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.795+0000 7f7d88a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef18ff18a0 0x55ef18ff97a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:47948/0 (socket says 172.21.15.83:47948) 2024-04-24T09:13:21.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.795+0000 7f7d88a00640 1 -- 172.21.15.83:0/1431110256 learned_addr learned my addr 172.21.15.83:0/1431110256 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:13:21.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.795+0000 7f7d88a00640 1 --2- 172.21.15.83:0/1431110256 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef18ffc890 0x55ef18ffa220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:21.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.795+0000 7f7d7be00640 1 --2- 172.21.15.83:0/1431110256 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef18ff21c0 0x55ef18ff9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:21.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.795+0000 7f7d88a00640 1 -- 172.21.15.83:0/1431110256 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef18ffc890 msgr2=0x55ef18ffa220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:21.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.795+0000 7f7d88a00640 1 --2- 172.21.15.83:0/1431110256 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef18ffc890 0x55ef18ffa220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:21.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.795+0000 7f7d88a00640 1 -- 172.21.15.83:0/1431110256 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef18ff21c0 msgr2=0x55ef18ff9ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:21.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.795+0000 7f7d88a00640 1 --2- 172.21.15.83:0/1431110256 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef18ff21c0 0x55ef18ff9ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:21.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.796+0000 7f7d88a00640 1 -- 172.21.15.83:0/1431110256 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef18d89f70 con 0x55ef18ff18a0 2024-04-24T09:13:21.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.796+0000 7f7d7be00640 1 --2- 172.21.15.83:0/1431110256 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef18ff21c0 0x55ef18ff9ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:13:21.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.796+0000 7f7d88a00640 1 --2- 172.21.15.83:0/1431110256 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef18ff18a0 0x55ef18ff97a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c009370 tx=0x7f7d7c01c040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50224c1136d808b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:21.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.796+0000 7f7d71600640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-04-24T09:13:21.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.797+0000 7f7d71600640 1 -- 172.21.15.83:0/1431110256 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d7c003b70 con 0x55ef18ff18a0 2024-04-24T09:13:21.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.797+0000 7f7d71600640 1 -- 172.21.15.83:0/1431110256 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9583+0+0 (secure 0 0 0) 0x7f7d7c004b20 con 0x55ef18ff18a0 2024-04-24T09:13:21.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.797+0000 7f7d8d781740 1 -- 172.21.15.83:0/1431110256 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d7c003e20 con 0x55ef18ff18a0 2024-04-24T09:13:21.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.798+0000 7f7d71600640 1 -- 172.21.15.83:0/1431110256 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f7d7c018470 con 0x55ef18ff18a0 2024-04-24T09:13:21.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.798+0000 7f7d8d781740 10 client.29905 fetch_fsmap learned FSMap version 64 2024-04-24T09:13:21.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.798+0000 7f7d8d781740 1 -- 172.21.15.83:0/1431110256 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55ef18c21170 con 0x55ef18ff18a0 2024-04-24T09:13:21.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.798+0000 7f7d71600640 1 -- 172.21.15.83:0/1431110256 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7d7c003e20 con 0x55ef18ff18a0 2024-04-24T09:13:21.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.799+0000 7f7d71600640 1 -- 172.21.15.83:0/1431110256 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 64) ==== 8374+0+0 (secure 0 0 0) 0x7f7d7c0104c0 con 0x55ef18ff18a0 2024-04-24T09:13:21.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.799+0000 7f7d8d781740 10 client.29905 fetch_fsmap finished waiting for FSMap version 64 2024-04-24T09:13:21.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.799+0000 7f7d8d781740 10 client.29905 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:13:21.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.799+0000 7f7d8d781740 20 client.29905 populate_metadata read hostname 'smithi083' 2024-04-24T09:13:21.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.799+0000 7f7d8d781740 1 --2- 172.21.15.83:0/1431110256 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ef18fe8db0 0x55ef18feb200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:21.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.799+0000 7f7d8d781740 4 client.29905 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:13:21.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.799+0000 7f7d8d781740 1 -- 172.21.15.83:0/1431110256 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ef18f5c890 con 0x55ef18fe8db0 2024-04-24T09:13:21.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.799+0000 7f7d7be00640 1 --2- 172.21.15.83:0/1431110256 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ef18fe8db0 0x55ef18feb200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:21.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.801+0000 7f7d7be00640 1 --2- 172.21.15.83:0/1431110256 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ef18fe8db0 0x55ef18feb200 crc :-1 s=READY pgs=537 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a80b6923ea1c89f8 server_cookie=2ca484d197864a26 in_seq=0 out_seq=0 2024-04-24T09:13:21.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.801+0000 7f7d71600640 10 client.29905 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:13:21.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.802+0000 7f7d71600640 1 -- 172.21.15.83:0/1431110256 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ef18f5c890 con 0x55ef18fe8db0 2024-04-24T09:13:21.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.802+0000 7f7d71600640 10 client.29905 handle_command_reply: tid=0 2024-04-24T09:13:21.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.802+0000 7f7d8d781740 1 client.29905 shutdown 2024-04-24T09:13:21.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.802+0000 7f7d8d781740 1 -- 172.21.15.83:0/1431110256 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef18ff18a0 msgr2=0x55ef18ff97a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:21.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.802+0000 7f7d8d781740 1 --2- 172.21.15.83:0/1431110256 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef18ff18a0 0x55ef18ff97a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c009370 tx=0x7f7d7c01c040 comp rx=0 tx=0).stop 2024-04-24T09:13:21.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.802+0000 7f7d8d781740 1 -- 172.21.15.83:0/1431110256 shutdown_connections 2024-04-24T09:13:21.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.802+0000 7f7d8d781740 1 --2- 172.21.15.83:0/1431110256 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ef18fe8db0 0x55ef18feb200 crc :-1 s=READY pgs=537 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:21.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.802+0000 7f7d8d781740 1 --2- 172.21.15.83:0/1431110256 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef18ffc890 0x55ef18ffa220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:21.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.802+0000 7f7d8d781740 1 --2- 172.21.15.83:0/1431110256 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef18ff21c0 0x55ef18ff9ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:21.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.802+0000 7f7d8d781740 1 --2- 172.21.15.83:0/1431110256 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef18ff18a0 0x55ef18ff97a0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:21.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.802+0000 7f7d8d781740 1 -- 172.21.15.83:0/1431110256 >> 172.21.15.83:0/1431110256 conn(0x55ef18fedbc0 msgr2=0x55ef18fe4510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:21.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.802+0000 7f7d71600640 0 client.29905 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:13:21.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.803+0000 7f7d8d781740 1 -- 172.21.15.83:0/1431110256 shutdown_connections 2024-04-24T09:13:21.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.803+0000 7f7d8d781740 1 -- 172.21.15.83:0/1431110256 wait complete. 2024-04-24T09:13:21.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.803+0000 7f7d8d781740 20 client.29905 trim_cache size 0 max 16384 2024-04-24T09:13:21.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.833+0000 7f7d8d781740 1 Processor -- start 2024-04-24T09:13:21.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.833+0000 7f7d8d781740 1 -- start start 2024-04-24T09:13:21.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.834+0000 7f7d8d781740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef18ff18a0 0x55ef19019550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:21.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.834+0000 7f7d8d781740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef18ff21c0 0x55ef18ffa040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:21.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.834+0000 7f7d8d781740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef18ffc890 0x55ef18ff3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:21.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.834+0000 7f7d8d781740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7d84071360 con 0x55ef18ff21c0 2024-04-24T09:13:21.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.834+0000 7f7d8d781740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7d84072f30 con 0x55ef18ffc890 2024-04-24T09:13:21.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.834+0000 7f7d8d781740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7d84071a30 con 0x55ef18ff18a0 2024-04-24T09:13:21.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.834+0000 7f7d7be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef18ff21c0 0x55ef18ffa040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:21.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.834+0000 7f7d7be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef18ff21c0 0x55ef18ffa040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35320/0 (socket says 172.21.15.83:35320) 2024-04-24T09:13:21.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.834+0000 7f7d7be00640 1 -- 172.21.15.83:0/3324013814 learned_addr learned my addr 172.21.15.83:0/3324013814 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:13:21.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.835+0000 7f7d88a00640 1 --2- 172.21.15.83:0/3324013814 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef18ff18a0 0x55ef19019550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:21.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.835+0000 7f7d7be00640 1 -- 172.21.15.83:0/3324013814 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef18ff18a0 msgr2=0x55ef19019550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:21.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.835+0000 7f7d7be00640 1 --2- 172.21.15.83:0/3324013814 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef18ff18a0 0x55ef19019550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:21.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.835+0000 7f7d7be00640 1 -- 172.21.15.83:0/3324013814 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef18ffc890 msgr2=0x55ef18ff3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:21.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.835+0000 7f7d7be00640 1 --2- 172.21.15.83:0/3324013814 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef18ffc890 0x55ef18ff3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:21.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.835+0000 7f7d7be00640 1 -- 172.21.15.83:0/3324013814 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef18ff8750 con 0x55ef18ff21c0 2024-04-24T09:13:21.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.835+0000 7f7d88a00640 1 --2- 172.21.15.83:0/3324013814 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef18ff18a0 0x55ef19019550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:13:21.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.835+0000 7f7d7be00640 1 --2- 172.21.15.83:0/3324013814 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef18ff21c0 0x55ef18ffa040 secure :-1 s=READY pgs=2621 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c009770 tx=0x7f7d6c014840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c31b6005682a548 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:21.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.836+0000 7f7d73e00640 1 -- 172.21.15.83:0/3324013814 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d6c0142d0 con 0x55ef18ff21c0 2024-04-24T09:13:21.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.836+0000 7f7d73e00640 1 -- 172.21.15.83:0/3324013814 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7d6c0054d0 con 0x55ef18ff21c0 2024-04-24T09:13:21.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.836+0000 7f7d73e00640 1 -- 172.21.15.83:0/3324013814 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d6c01e050 con 0x55ef18ff21c0 2024-04-24T09:13:21.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.837+0000 7f7d8d781740 1 -- 172.21.15.83:0/3324013814 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef18ff21c0 msgr2=0x55ef18ffa040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:21.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.837+0000 7f7d8d781740 1 --2- 172.21.15.83:0/3324013814 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef18ff21c0 0x55ef18ffa040 secure :-1 s=READY pgs=2621 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c009770 tx=0x7f7d6c014840 comp rx=0 tx=0).stop 2024-04-24T09:13:21.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.837+0000 7f7d8d781740 1 -- 172.21.15.83:0/3324013814 shutdown_connections 2024-04-24T09:13:21.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.837+0000 7f7d8d781740 1 --2- 172.21.15.83:0/3324013814 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef18ffc890 0x55ef18ff3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:21.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.837+0000 7f7d8d781740 1 --2- 172.21.15.83:0/3324013814 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef18ff21c0 0x55ef18ffa040 unknown :-1 s=CLOSED pgs=2621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:21.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.837+0000 7f7d8d781740 1 --2- 172.21.15.83:0/3324013814 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef18ff18a0 0x55ef19019550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:21.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.837+0000 7f7d8d781740 1 -- 172.21.15.83:0/3324013814 >> 172.21.15.83:0/3324013814 conn(0x55ef18fedbc0 msgr2=0x55ef18ff2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:21.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.837+0000 7f7d8d781740 1 -- 172.21.15.83:0/3324013814 shutdown_connections 2024-04-24T09:13:21.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.837+0000 7f7d8d781740 1 -- 172.21.15.83:0/3324013814 wait complete. 2024-04-24T09:13:21.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.837+0000 7f7d8d781740 1 -- start start 2024-04-24T09:13:21.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.838+0000 7f7d8d781740 1 Processor -- start 2024-04-24T09:13:21.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.838+0000 7f7d8d781740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef18ff18a0 0x55ef18ff6680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:21.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.838+0000 7f7d8d781740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef18ffc890 0x55ef18ff6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:21.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.838+0000 7f7d8d781740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef18ff7100 0x55ef19013ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:21.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.838+0000 7f7d8d781740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7d84072830 con 0x55ef18ff7100 2024-04-24T09:13:21.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.838+0000 7f7d8d781740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7d84072160 con 0x55ef18ffc890 2024-04-24T09:13:21.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.838+0000 7f7d8d781740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7d8406f360 con 0x55ef18ff18a0 2024-04-24T09:13:21.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.838+0000 7f7d8d781740 10 client.0 _ll_register_callbacks cb 0x7f7d7c004610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:13:21.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.839+0000 7f7d88a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef18ff7100 0x55ef19013ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:21.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.839+0000 7f7d88a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef18ff7100 0x55ef19013ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35324/0 (socket says 172.21.15.83:35324) 2024-04-24T09:13:21.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.839+0000 7f7d88a00640 1 -- 172.21.15.83:0/1446543164 learned_addr learned my addr 172.21.15.83:0/1446543164 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:13:21.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.839+0000 7f7d7be00640 1 --2- 172.21.15.83:0/1446543164 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef18ffc890 0x55ef18ff6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:21.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.839+0000 7f7d88a00640 1 --2- 172.21.15.83:0/1446543164 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef18ff18a0 0x55ef18ff6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:21.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.839+0000 7f7d88a00640 1 -- 172.21.15.83:0/1446543164 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef18ff18a0 msgr2=0x55ef18ff6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:21.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.839+0000 7f7d88a00640 1 --2- 172.21.15.83:0/1446543164 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef18ff18a0 0x55ef18ff6680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:21.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.839+0000 7f7d88a00640 1 -- 172.21.15.83:0/1446543164 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef18ffc890 msgr2=0x55ef18ff6bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:21.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.839+0000 7f7d88a00640 1 --2- 172.21.15.83:0/1446543164 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef18ffc890 0x55ef18ff6bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:21.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.839+0000 7f7d88a00640 1 -- 172.21.15.83:0/1446543164 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef190145b0 con 0x55ef18ff7100 2024-04-24T09:13:21.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.840+0000 7f7d88a00640 1 --2- 172.21.15.83:0/1446543164 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef18ff7100 0x55ef19013ef0 secure :-1 s=READY pgs=2622 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c010aa0 tx=0x7f7d7c00e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f989be879e6d5d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:21.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.840+0000 7f7d71600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:13:21.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.840+0000 7f7d71600640 1 -- 172.21.15.83:0/1446543164 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d7c00ee30 con 0x55ef18ff7100 2024-04-24T09:13:21.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.840+0000 7f7d71600640 1 -- 172.21.15.83:0/1446543164 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9583+0+0 (secure 0 0 0) 0x7f7d7c010430 con 0x55ef18ff7100 2024-04-24T09:13:21.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.841+0000 7f7d8d781740 1 -- 172.21.15.83:0/1446543164 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d6c0054d0 con 0x55ef18ff7100 2024-04-24T09:13:21.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.841+0000 7f7d71600640 1 -- 172.21.15.83:0/1446543164 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f7d7c010630 con 0x55ef18ff7100 2024-04-24T09:13:21.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.841+0000 7f7d8d781740 10 client.23058 fetch_fsmap learned FSMap version 64 2024-04-24T09:13:21.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.841+0000 7f7d8d781740 1 -- 172.21.15.83:0/1446543164 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55ef1908ccd0 con 0x55ef18ff7100 2024-04-24T09:13:21.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.841+0000 7f7d71600640 1 -- 172.21.15.83:0/1446543164 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7d6c0054d0 con 0x55ef18ff7100 2024-04-24T09:13:21.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.841+0000 7f7d71600640 1 -- 172.21.15.83:0/1446543164 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 64) ==== 8374+0+0 (secure 0 0 0) 0x7f7d7c018920 con 0x55ef18ff7100 2024-04-24T09:13:21.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.842+0000 7f7d8d781740 10 client.23058 fetch_fsmap finished waiting for FSMap version 64 2024-04-24T09:13:21.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.842+0000 7f7d8d781740 10 client.23058 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:13:21.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.842+0000 7f7d8d781740 20 client.23058 populate_metadata read hostname 'smithi083' 2024-04-24T09:13:21.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.842+0000 7f7d8d781740 1 --2- 172.21.15.83:0/1446543164 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ef1908d340 0x55ef1908f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:21.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.842+0000 7f7d8d781740 4 client.23058 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T09:13:21.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.842+0000 7f7d8d781740 1 -- 172.21.15.83:0/1446543164 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ef18f56700 con 0x55ef1908d340 2024-04-24T09:13:21.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.843+0000 7f7d7be00640 1 --2- 172.21.15.83:0/1446543164 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ef1908d340 0x55ef1908f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:21.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.844+0000 7f7d7be00640 1 --2- 172.21.15.83:0/1446543164 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ef1908d340 0x55ef1908f760 crc :-1 s=READY pgs=538 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1fb752e57a69c906 server_cookie=24e6f78a5b6fd50a in_seq=0 out_seq=0 2024-04-24T09:13:21.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.844+0000 7f7d71600640 10 client.23058 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:13:21.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.845+0000 7f7d71600640 1 -- 172.21.15.83:0/1446543164 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ef18f56700 con 0x55ef1908d340 2024-04-24T09:13:21.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.845+0000 7f7d71600640 10 client.23058 handle_command_reply: tid=0 2024-04-24T09:13:21.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.845+0000 7f7d8d781740 1 client.23058 shutdown 2024-04-24T09:13:21.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.845+0000 7f7d8d781740 1 -- 172.21.15.83:0/1446543164 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef18ff7100 msgr2=0x55ef19013ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:21.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.845+0000 7f7d8d781740 1 --2- 172.21.15.83:0/1446543164 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef18ff7100 0x55ef19013ef0 secure :-1 s=READY pgs=2622 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c010aa0 tx=0x7f7d7c00e720 comp rx=0 tx=0).stop 2024-04-24T09:13:21.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.846+0000 7f7d8d781740 1 -- 172.21.15.83:0/1446543164 shutdown_connections 2024-04-24T09:13:21.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.846+0000 7f7d8d781740 1 --2- 172.21.15.83:0/1446543164 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ef1908d340 0x55ef1908f760 crc :-1 s=READY pgs=538 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:21.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.846+0000 7f7d8d781740 1 --2- 172.21.15.83:0/1446543164 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef18ff7100 0x55ef19013ef0 unknown :-1 s=CLOSED pgs=2622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:21.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.846+0000 7f7d8d781740 1 --2- 172.21.15.83:0/1446543164 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef18ffc890 0x55ef18ff6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:21.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.846+0000 7f7d8d781740 1 --2- 172.21.15.83:0/1446543164 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef18ff18a0 0x55ef18ff6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:21.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.846+0000 7f7d8d781740 1 -- 172.21.15.83:0/1446543164 >> 172.21.15.83:0/1446543164 conn(0x55ef18fedbc0 msgr2=0x55ef18fe4510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:21.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.846+0000 7f7d71600640 0 client.23058 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:13:21.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.846+0000 7f7d8d781740 1 -- 172.21.15.83:0/1446543164 shutdown_connections 2024-04-24T09:13:21.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.847+0000 7f7d8d781740 1 -- 172.21.15.83:0/1446543164 wait complete. 2024-04-24T09:13:21.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.847+0000 7f7d8d781740 20 client.23058 trim_cache size 0 max 16384 2024-04-24T09:13:21.848 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T09:13:21.848 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-04-24T09:13:21.848 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "57eba8c5-264d-49ec-9eb8-a560a2ae52b4", 2024-04-24T09:13:21.848 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-04-24T09:13:21.849 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T09:13:21.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.848+0000 7f7d8aa00640 1 -- 172.21.15.83:0/3018761583 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f7d680a3e20 msgr2=0x7f7d680a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:21.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.848+0000 7f7d8aa00640 1 --2- 172.21.15.83:0/3018761583 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f7d680a3e20 0x7f7d680a62e0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f7d740bc300 tx=0x7f7d7402a3f0 comp rx=0 tx=0).stop 2024-04-24T09:13:21.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.848+0000 7f7d8aa00640 1 -- 172.21.15.83:0/3018761583 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7d84102240 msgr2=0x7f7d841a06d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:21.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.848+0000 7f7d8aa00640 1 --2- 172.21.15.83:0/3018761583 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7d84102240 0x7f7d841a06d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7d74005940 tx=0x7f7d740040c0 comp rx=0 tx=0).stop 2024-04-24T09:13:21.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.848+0000 7f7d8aa00640 1 -- 172.21.15.83:0/3018761583 shutdown_connections 2024-04-24T09:13:21.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.848+0000 7f7d8aa00640 1 --2- 172.21.15.83:0/3018761583 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f7d680a3e20 0x7f7d680a62e0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:21.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.848+0000 7f7d8aa00640 1 --2- 172.21.15.83:0/3018761583 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d8419a880 0x7f7d8419ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:21.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.848+0000 7f7d8aa00640 1 --2- 172.21.15.83:0/3018761583 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d84107310 0x7f7d841a0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:21.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.848+0000 7f7d8aa00640 1 --2- 172.21.15.83:0/3018761583 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7d84102240 0x7f7d841a06d0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:21.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.848+0000 7f7d8aa00640 1 -- 172.21.15.83:0/3018761583 >> 172.21.15.83:0/3018761583 conn(0x7f7d84079450 msgr2=0x7f7d84100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:21.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.848+0000 7f7d8aa00640 1 -- 172.21.15.83:0/3018761583 shutdown_connections 2024-04-24T09:13:21.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:21.849+0000 7f7d8aa00640 1 -- 172.21.15.83:0/3018761583 wait complete. 2024-04-24T09:13:21.873 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T09:13:21.874 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T09:13:22.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.149+0000 7f7eb1e00640 1 -- 172.21.15.83:0/2706995138 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7eac076530 msgr2=0x7f7eac076910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:22.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.149+0000 7f7eb1e00640 1 --2- 172.21.15.83:0/2706995138 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7eac076530 0x7f7eac076910 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7ea0009930 tx=0x7f7ea002a420 comp rx=0 tx=0).stop 2024-04-24T09:13:22.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.149+0000 7f7eb1e00640 1 -- 172.21.15.83:0/2706995138 shutdown_connections 2024-04-24T09:13:22.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.149+0000 7f7eb1e00640 1 --2- 172.21.15.83:0/2706995138 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7eac112d40 0x7f7eac115130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:22.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.149+0000 7f7eb1e00640 1 --2- 172.21.15.83:0/2706995138 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7eac076e50 0x7f7eac112800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:22.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.149+0000 7f7eb1e00640 1 --2- 172.21.15.83:0/2706995138 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7eac076530 0x7f7eac076910 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:22.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.149+0000 7f7eb1e00640 1 -- 172.21.15.83:0/2706995138 >> 172.21.15.83:0/2706995138 conn(0x7f7eac0ffb70 msgr2=0x7f7eac101f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:22.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.149+0000 7f7eb1e00640 1 -- 172.21.15.83:0/2706995138 shutdown_connections 2024-04-24T09:13:22.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.149+0000 7f7eb1e00640 1 -- 172.21.15.83:0/2706995138 wait complete. 2024-04-24T09:13:22.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.150+0000 7f7eb1e00640 1 Processor -- start 2024-04-24T09:13:22.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.151+0000 7f7eb1e00640 1 -- start start 2024-04-24T09:13:22.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.151+0000 7f7eb1e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7eac076530 0x7f7eac1a4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:22.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.151+0000 7f7eb1e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7eac076e50 0x7f7eac1a4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:22.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.151+0000 7f7eb1e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7eac112d40 0x7f7eac19eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:22.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.151+0000 7f7eb1e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7eac118790 con 0x7f7eac076e50 2024-04-24T09:13:22.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.151+0000 7f7eb1e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7eac118610 con 0x7f7eac112d40 2024-04-24T09:13:22.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.151+0000 7f7eb1e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7eac118910 con 0x7f7eac076530 2024-04-24T09:13:22.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.152+0000 7f7eb0a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7eac076530 0x7f7eac1a4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:22.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.152+0000 7f7eb0a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7eac076530 0x7f7eac1a4900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:47990/0 (socket says 172.21.15.83:47990) 2024-04-24T09:13:22.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.152+0000 7f7eb0a00640 1 -- 172.21.15.83:0/1970676582 learned_addr learned my addr 172.21.15.83:0/1970676582 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:13:22.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.152+0000 7f7eabe00640 1 --2- 172.21.15.83:0/1970676582 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7eac076e50 0x7f7eac1a4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:22.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.152+0000 7f7eb1400640 1 --2- 172.21.15.83:0/1970676582 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7eac112d40 0x7f7eac19eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:22.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.152+0000 7f7eb0a00640 1 -- 172.21.15.83:0/1970676582 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7eac112d40 msgr2=0x7f7eac19eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:22.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.152+0000 7f7eb0a00640 1 --2- 172.21.15.83:0/1970676582 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7eac112d40 0x7f7eac19eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:22.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.152+0000 7f7eb0a00640 1 -- 172.21.15.83:0/1970676582 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7eac076e50 msgr2=0x7f7eac1a4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:22.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.152+0000 7f7eb0a00640 1 --2- 172.21.15.83:0/1970676582 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7eac076e50 0x7f7eac1a4e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:22.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.152+0000 7f7eb0a00640 1 -- 172.21.15.83:0/1970676582 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7eac19f230 con 0x7f7eac076530 2024-04-24T09:13:22.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.153+0000 7f7eb0a00640 1 --2- 172.21.15.83:0/1970676582 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7eac076530 0x7f7eac1a4900 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7ea0005ab0 tx=0x7f7ea0003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:22.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.153+0000 7f7ea9600640 1 -- 172.21.15.83:0/1970676582 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea003f070 con 0x7f7eac076530 2024-04-24T09:13:22.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.153+0000 7f7ea9600640 1 -- 172.21.15.83:0/1970676582 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7ea0031b50 con 0x7f7eac076530 2024-04-24T09:13:22.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.153+0000 7f7ea9600640 1 -- 172.21.15.83:0/1970676582 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea0035030 con 0x7f7eac076530 2024-04-24T09:13:22.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.153+0000 7f7eb1e00640 1 -- 172.21.15.83:0/1970676582 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7eac19f4c0 con 0x7f7eac076530 2024-04-24T09:13:22.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.153+0000 7f7eb1e00640 1 -- 172.21.15.83:0/1970676582 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7eac1ab7c0 con 0x7f7eac076530 2024-04-24T09:13:22.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.158+0000 7f7eb6214740 1 Processor -- start 2024-04-24T09:13:22.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.158+0000 7f7ea9600640 1 -- 172.21.15.83:0/1970676582 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7ea0042070 con 0x7f7eac076530 2024-04-24T09:13:22.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.158+0000 7f7eb6214740 1 -- start start 2024-04-24T09:13:22.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.159+0000 7f7eb6214740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5601a999fa30 0x5601a999fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:22.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.159+0000 7f7eb6214740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5601a99a0350 0x5601a99aa4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:22.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.159+0000 7f7eb6214740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5601a99aaa20 0x5601a99ace10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:22.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.159+0000 7f7eb6214740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5601a99051f0 con 0x5601a999fa30 2024-04-24T09:13:22.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.159+0000 7f7eb6214740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5601a98ab350 con 0x5601a99aaa20 2024-04-24T09:13:22.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.159+0000 7f7eb6214740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5601a99011b0 con 0x5601a99a0350 2024-04-24T09:13:22.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.159+0000 7f7eb1400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5601a999fa30 0x5601a999fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:22.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.160+0000 7f7eb1400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5601a999fa30 0x5601a999fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35360/0 (socket says 172.21.15.83:35360) 2024-04-24T09:13:22.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.160+0000 7f7eb1400640 1 -- 172.21.15.83:0/4208030823 learned_addr learned my addr 172.21.15.83:0/4208030823 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:13:22.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.160+0000 7f7eabe00640 1 --2- 172.21.15.83:0/4208030823 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5601a99aaa20 0x5601a99ace10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:22.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.160+0000 7f7eabe00640 1 -- 172.21.15.83:0/4208030823 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5601a99a0350 msgr2=0x5601a99aa4e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-24T09:13:22.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.160+0000 7f7eabe00640 1 --2- 172.21.15.83:0/4208030823 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5601a99a0350 0x5601a99aa4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:22.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.160+0000 7f7eabe00640 1 -- 172.21.15.83:0/4208030823 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5601a999fa30 msgr2=0x5601a999fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:22.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.160+0000 7f7eabe00640 1 --2- 172.21.15.83:0/4208030823 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5601a999fa30 0x5601a999fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:22.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.160+0000 7f7eabe00640 1 -- 172.21.15.83:0/4208030823 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601a98fff50 con 0x5601a99aaa20 2024-04-24T09:13:22.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.160+0000 7f7eb1400640 1 --2- 172.21.15.83:0/4208030823 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5601a999fa30 0x5601a999fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:13:22.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.160+0000 7f7ea9600640 1 --2- 172.21.15.83:0/1970676582 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f7e940a3e20 0x7f7e940a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:22.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.161+0000 7f7eabe00640 1 --2- 172.21.15.83:0/4208030823 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5601a99aaa20 0x5601a99ace10 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f7eac1a02a0 tx=0x7f7e9800f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bff7d475cf5e9e36 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:22.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.161+0000 7f7ea9600640 1 -- 172.21.15.83:0/1970676582 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 9583+0+0 (secure 0 0 0) 0x7f7ea00f11e0 con 0x7f7eac076530 2024-04-24T09:13:22.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.161+0000 7f7eb1400640 1 --2- 172.21.15.83:0/1970676582 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f7e940a3e20 0x7f7e940a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:22.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.161+0000 7f7e9dc00640 1 -- 172.21.15.83:0/4208030823 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e98005220 con 0x5601a99aaa20 2024-04-24T09:13:22.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.161+0000 7f7e9dc00640 1 -- 172.21.15.83:0/4208030823 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7e98004c50 con 0x5601a99aaa20 2024-04-24T09:13:22.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.161+0000 7f7e9dc00640 1 -- 172.21.15.83:0/4208030823 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e98018770 con 0x5601a99aaa20 2024-04-24T09:13:22.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.162+0000 7f7eb6214740 1 -- 172.21.15.83:0/4208030823 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5601a99aaa20 msgr2=0x5601a99ace10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:22.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.162+0000 7f7eb6214740 1 --2- 172.21.15.83:0/4208030823 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5601a99aaa20 0x5601a99ace10 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f7eac1a02a0 tx=0x7f7e9800f790 comp rx=0 tx=0).stop 2024-04-24T09:13:22.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.162+0000 7f7eb1400640 1 --2- 172.21.15.83:0/1970676582 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f7e940a3e20 0x7f7e940a62e0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f7ea4009fd0 tx=0x7f7ea4009500 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:22.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.162+0000 7f7eb6214740 1 -- 172.21.15.83:0/4208030823 shutdown_connections 2024-04-24T09:13:22.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.162+0000 7f7eb6214740 1 --2- 172.21.15.83:0/4208030823 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5601a99aaa20 0x5601a99ace10 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:22.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.162+0000 7f7eb6214740 1 --2- 172.21.15.83:0/4208030823 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5601a99a0350 0x5601a99aa4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:22.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.162+0000 7f7eb6214740 1 --2- 172.21.15.83:0/4208030823 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5601a999fa30 0x5601a999fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:22.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.162+0000 7f7eb6214740 1 -- 172.21.15.83:0/4208030823 >> 172.21.15.83:0/4208030823 conn(0x5601a999bd50 msgr2=0x5601a999c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:22.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.162+0000 7f7eb6214740 1 -- 172.21.15.83:0/4208030823 shutdown_connections 2024-04-24T09:13:22.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.162+0000 7f7eb6214740 1 -- 172.21.15.83:0/4208030823 wait complete. 2024-04-24T09:13:22.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.163+0000 7f7eb6214740 1 -- start start 2024-04-24T09:13:22.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.163+0000 7f7eb6214740 1 Processor -- start 2024-04-24T09:13:22.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.164+0000 7f7eb6214740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5601a999fa30 0x5601a99a79d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:22.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.164+0000 7f7eb6214740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5601a99a0350 0x5601a99a7f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:22.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.164+0000 7f7eb6214740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5601a99aaa20 0x5601a99a8450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:22.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.164+0000 7f7eb6214740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5601a99ad860 con 0x5601a99a0350 2024-04-24T09:13:22.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.164+0000 7f7eb6214740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5601a98be690 con 0x5601a99aaa20 2024-04-24T09:13:22.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.164+0000 7f7eb6214740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5601a99ad9e0 con 0x5601a999fa30 2024-04-24T09:13:22.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.164+0000 7f7eb6214740 10 client.0 _ll_register_callbacks cb 0x5601a98fa650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:13:22.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.164+0000 7f7eabe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5601a99a0350 0x5601a99a7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:22.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.164+0000 7f7eabe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5601a99a0350 0x5601a99a7f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35370/0 (socket says 172.21.15.83:35370) 2024-04-24T09:13:22.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.164+0000 7f7eabe00640 1 -- 172.21.15.83:0/261477582 learned_addr learned my addr 172.21.15.83:0/261477582 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:13:22.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.164+0000 7f7eb0a00640 1 --2- 172.21.15.83:0/261477582 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5601a999fa30 0x5601a99a79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:22.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.165+0000 7f7eb1400640 1 --2- 172.21.15.83:0/261477582 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5601a99aaa20 0x5601a99a8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:22.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.165+0000 7f7eabe00640 1 -- 172.21.15.83:0/261477582 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5601a999fa30 msgr2=0x5601a99a79d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:22.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.165+0000 7f7eabe00640 1 --2- 172.21.15.83:0/261477582 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5601a999fa30 0x5601a99a79d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:22.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.165+0000 7f7eabe00640 1 -- 172.21.15.83:0/261477582 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5601a99aaa20 msgr2=0x5601a99a8450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:22.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.165+0000 7f7eabe00640 1 --2- 172.21.15.83:0/261477582 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5601a99aaa20 0x5601a99a8450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:22.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.165+0000 7f7eabe00640 1 -- 172.21.15.83:0/261477582 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601a98bd920 con 0x5601a99a0350 2024-04-24T09:13:22.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.165+0000 7f7eabe00640 1 --2- 172.21.15.83:0/261477582 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5601a99a0350 0x5601a99a7f10 secure :-1 s=READY pgs=2623 cs=0 l=1 rev1=1 crypto rx=0x7f7e98005bd0 tx=0x7f7e98005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4733430fd05e5a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:22.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.165+0000 7f7e92000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:13:22.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.166+0000 7f7e92000640 1 -- 172.21.15.83:0/261477582 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e98027070 con 0x5601a99a0350 2024-04-24T09:13:22.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.166+0000 7f7e92000640 1 -- 172.21.15.83:0/261477582 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9583+0+0 (secure 0 0 0) 0x7f7e980186c0 con 0x5601a99a0350 2024-04-24T09:13:22.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.166+0000 7f7eb6214740 1 -- 172.21.15.83:0/261477582 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e98004c50 con 0x5601a99a0350 2024-04-24T09:13:22.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.166+0000 7f7e92000640 1 -- 172.21.15.83:0/261477582 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f7e98033050 con 0x5601a99a0350 2024-04-24T09:13:22.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.166+0000 7f7eb6214740 10 client.23067 fetch_fsmap learned FSMap version 64 2024-04-24T09:13:22.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.166+0000 7f7eb6214740 1 -- 172.21.15.83:0/261477582 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x5601a96e17d0 con 0x5601a99a0350 2024-04-24T09:13:22.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.167+0000 7f7e92000640 1 -- 172.21.15.83:0/261477582 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7e98004c50 con 0x5601a99a0350 2024-04-24T09:13:22.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.167+0000 7f7e92000640 1 -- 172.21.15.83:0/261477582 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 64) ==== 8374+0+0 (secure 0 0 0) 0x7f7e98023430 con 0x5601a99a0350 2024-04-24T09:13:22.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.167+0000 7f7eb6214740 10 client.23067 fetch_fsmap finished waiting for FSMap version 64 2024-04-24T09:13:22.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.167+0000 7f7eb6214740 10 client.23067 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:13:22.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.167+0000 7f7eb6214740 20 client.23067 populate_metadata read hostname 'smithi083' 2024-04-24T09:13:22.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.167+0000 7f7eb6214740 1 --2- 172.21.15.83:0/261477582 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5601a99badc0 0x5601a99bd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:22.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.167+0000 7f7eb6214740 4 client.23067 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:13:22.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.167+0000 7f7eb6214740 1 -- 172.21.15.83:0/261477582 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5601a9912c10 con 0x5601a99badc0 2024-04-24T09:13:22.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.167+0000 7f7eb0a00640 1 --2- 172.21.15.83:0/261477582 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5601a99badc0 0x5601a99bd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:22.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.168+0000 7f7eb0a00640 1 --2- 172.21.15.83:0/261477582 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5601a99badc0 0x5601a99bd1b0 crc :-1 s=READY pgs=539 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c157d4ce554c445b server_cookie=19a3651605f5d64f in_seq=0 out_seq=0 2024-04-24T09:13:22.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.168+0000 7f7e92000640 10 client.23067 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:13:22.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.169+0000 7f7e92000640 1 -- 172.21.15.83:0/261477582 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5601a9912c10 con 0x5601a99badc0 2024-04-24T09:13:22.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.169+0000 7f7e92000640 10 client.23067 handle_command_reply: tid=0 2024-04-24T09:13:22.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.169+0000 7f7eb6214740 1 client.23067 shutdown 2024-04-24T09:13:22.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.169+0000 7f7eb6214740 1 -- 172.21.15.83:0/261477582 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5601a99a0350 msgr2=0x5601a99a7f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:22.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.169+0000 7f7eb6214740 1 --2- 172.21.15.83:0/261477582 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5601a99a0350 0x5601a99a7f10 secure :-1 s=READY pgs=2623 cs=0 l=1 rev1=1 crypto rx=0x7f7e98005bd0 tx=0x7f7e98005030 comp rx=0 tx=0).stop 2024-04-24T09:13:22.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.170+0000 7f7eb6214740 1 -- 172.21.15.83:0/261477582 shutdown_connections 2024-04-24T09:13:22.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.170+0000 7f7eb6214740 1 --2- 172.21.15.83:0/261477582 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5601a99badc0 0x5601a99bd1b0 crc :-1 s=READY pgs=539 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:22.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.170+0000 7f7eb6214740 1 --2- 172.21.15.83:0/261477582 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5601a99aaa20 0x5601a99a8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:22.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.170+0000 7f7eb6214740 1 --2- 172.21.15.83:0/261477582 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5601a99a0350 0x5601a99a7f10 unknown :-1 s=CLOSED pgs=2623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:22.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.170+0000 7f7eb6214740 1 --2- 172.21.15.83:0/261477582 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5601a999fa30 0x5601a99a79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:22.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.170+0000 7f7eb6214740 1 -- 172.21.15.83:0/261477582 >> 172.21.15.83:0/261477582 conn(0x5601a999bd50 msgr2=0x5601a999afd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:22.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.170+0000 7f7e92000640 0 client.23067 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:13:22.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.170+0000 7f7eb6214740 1 -- 172.21.15.83:0/261477582 shutdown_connections 2024-04-24T09:13:22.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.170+0000 7f7eb6214740 1 -- 172.21.15.83:0/261477582 wait complete. 2024-04-24T09:13:22.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.170+0000 7f7eb6214740 20 client.23067 trim_cache size 0 max 16384 2024-04-24T09:13:22.197 INFO:tasks.workunit.client.0.smithi083.stdout: 10 58766 6.62 MB/sec execute 353 sec latency 475.852 ms 2024-04-24T09:13:22.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.211+0000 7f7eb6214740 1 Processor -- start 2024-04-24T09:13:22.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.211+0000 7f7eb6214740 1 -- start start 2024-04-24T09:13:22.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.212+0000 7f7eb6214740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5601a999fa30 0x5601a99b6780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:22.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.212+0000 7f7eb6214740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5601a99a0350 0x5601a99a21d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:22.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.212+0000 7f7eb6214740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5601a99aaa20 0x5601a99c28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:22.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.212+0000 7f7eb6214740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7eac1b59e0 con 0x5601a99aaa20 2024-04-24T09:13:22.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.212+0000 7f7eb6214740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7eac1b75b0 con 0x5601a999fa30 2024-04-24T09:13:22.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.212+0000 7f7eb6214740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7eac1b60b0 con 0x5601a99a0350 2024-04-24T09:13:22.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.213+0000 7f7eb0a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5601a999fa30 0x5601a99b6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:22.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.213+0000 7f7eb0a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5601a999fa30 0x5601a99b6780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:38244/0 (socket says 172.21.15.83:38244) 2024-04-24T09:13:22.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.213+0000 7f7eb0a00640 1 -- 172.21.15.83:0/177548745 learned_addr learned my addr 172.21.15.83:0/177548745 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:13:22.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.213+0000 7f7eabe00640 1 --2- 172.21.15.83:0/177548745 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5601a99a0350 0x5601a99a21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:22.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.214+0000 7f7eb0a00640 1 -- 172.21.15.83:0/177548745 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5601a99a0350 msgr2=0x5601a99a21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:22.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.214+0000 7f7eb0a00640 1 --2- 172.21.15.83:0/177548745 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5601a99a0350 0x5601a99a21d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:22.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.214+0000 7f7eb0a00640 1 -- 172.21.15.83:0/177548745 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5601a99aaa20 msgr2=0x5601a99c28a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T09:13:22.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.214+0000 7f7eb1400640 1 --2- 172.21.15.83:0/177548745 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5601a99aaa20 0x5601a99c28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:22.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.214+0000 7f7eb0a00640 1 --2- 172.21.15.83:0/177548745 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5601a99aaa20 0x5601a99c28a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:22.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.214+0000 7f7eb0a00640 1 -- 172.21.15.83:0/177548745 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601a96173f0 con 0x5601a999fa30 2024-04-24T09:13:22.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.214+0000 7f7eb1400640 1 --2- 172.21.15.83:0/177548745 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5601a99aaa20 0x5601a99c28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:13:22.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.215+0000 7f7eb0a00640 1 --2- 172.21.15.83:0/177548745 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5601a999fa30 0x5601a99b6780 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f7ea00b96d0 tx=0x7f7ea00dd040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d32183f5ae3a2e85 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:22.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.215+0000 7f7e9d200640 1 -- 172.21.15.83:0/177548745 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea00d5070 con 0x5601a999fa30 2024-04-24T09:13:22.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.215+0000 7f7e9d200640 1 -- 172.21.15.83:0/177548745 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7ea00d6260 con 0x5601a999fa30 2024-04-24T09:13:22.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.215+0000 7f7e9d200640 1 -- 172.21.15.83:0/177548745 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea00ee730 con 0x5601a999fa30 2024-04-24T09:13:22.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.216+0000 7f7eb6214740 1 -- 172.21.15.83:0/177548745 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5601a999fa30 msgr2=0x5601a99b6780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:22.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.216+0000 7f7eb6214740 1 --2- 172.21.15.83:0/177548745 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5601a999fa30 0x5601a99b6780 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f7ea00b96d0 tx=0x7f7ea00dd040 comp rx=0 tx=0).stop 2024-04-24T09:13:22.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.216+0000 7f7eb6214740 1 -- 172.21.15.83:0/177548745 shutdown_connections 2024-04-24T09:13:22.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.216+0000 7f7eb6214740 1 --2- 172.21.15.83:0/177548745 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5601a99aaa20 0x5601a99c28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:22.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.216+0000 7f7eb6214740 1 --2- 172.21.15.83:0/177548745 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5601a99a0350 0x5601a99a21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:22.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.216+0000 7f7eb6214740 1 --2- 172.21.15.83:0/177548745 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5601a999fa30 0x5601a99b6780 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:22.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.216+0000 7f7eb6214740 1 -- 172.21.15.83:0/177548745 >> 172.21.15.83:0/177548745 conn(0x5601a999bd50 msgr2=0x5601a999afd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:22.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.216+0000 7f7eb6214740 1 -- 172.21.15.83:0/177548745 shutdown_connections 2024-04-24T09:13:22.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.216+0000 7f7eb6214740 1 -- 172.21.15.83:0/177548745 wait complete. 2024-04-24T09:13:22.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.217+0000 7f7eb6214740 1 -- start start 2024-04-24T09:13:22.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.217+0000 7f7eb6214740 1 Processor -- start 2024-04-24T09:13:22.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.218+0000 7f7eb6214740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5601a999fa30 0x5601a99b2f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:22.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.218+0000 7f7eb6214740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5601a99a0350 0x5601a99a4260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:22.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.218+0000 7f7eb6214740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5601a99aaa20 0x5601a99a47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:22.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.218+0000 7f7eabe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5601a99a0350 0x5601a99a4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:22.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.218+0000 7f7eb6214740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7eac1b6eb0 con 0x5601a999fa30 2024-04-24T09:13:22.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.218+0000 7f7eb6214740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7eac1b67e0 con 0x5601a99aaa20 2024-04-24T09:13:22.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.218+0000 7f7eb6214740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7eac1b39e0 con 0x5601a99a0350 2024-04-24T09:13:22.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.219+0000 7f7eabe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5601a99a0350 0x5601a99a4260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:48010/0 (socket says 172.21.15.83:48010) 2024-04-24T09:13:22.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.219+0000 7f7eabe00640 1 -- 172.21.15.83:0/2490733463 learned_addr learned my addr 172.21.15.83:0/2490733463 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:13:22.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.219+0000 7f7eb1400640 1 --2- 172.21.15.83:0/2490733463 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5601a99aaa20 0x5601a99a47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:22.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.219+0000 7f7eb6214740 10 client.0 _ll_register_callbacks cb 0x7f7ea4009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:13:22.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.219+0000 7f7eb0a00640 1 --2- 172.21.15.83:0/2490733463 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5601a999fa30 0x5601a99b2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:22.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.219+0000 7f7eabe00640 1 -- 172.21.15.83:0/2490733463 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5601a99aaa20 msgr2=0x5601a99a47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:22.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.219+0000 7f7eabe00640 1 --2- 172.21.15.83:0/2490733463 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5601a99aaa20 0x5601a99a47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:22.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.219+0000 7f7eabe00640 1 -- 172.21.15.83:0/2490733463 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5601a999fa30 msgr2=0x5601a99b2f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:22.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.220+0000 7f7eabe00640 1 --2- 172.21.15.83:0/2490733463 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5601a999fa30 0x5601a99b2f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:22.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.220+0000 7f7eabe00640 1 -- 172.21.15.83:0/2490733463 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601a95d70f0 con 0x5601a99a0350 2024-04-24T09:13:22.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.220+0000 7f7eb0a00640 1 --2- 172.21.15.83:0/2490733463 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5601a999fa30 0x5601a99b2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:13:22.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.220+0000 7f7eabe00640 1 --2- 172.21.15.83:0/2490733463 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5601a99a0350 0x5601a99a4260 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x5601a99a8fa0 tx=0x7f7e9800a2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75c3a2dca6e5aa47 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:22.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.220+0000 7f7e92000640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-04-24T09:13:22.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.220+0000 7f7e92000640 1 -- 172.21.15.83:0/2490733463 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e98027040 con 0x5601a99a0350 2024-04-24T09:13:22.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.220+0000 7f7e92000640 1 -- 172.21.15.83:0/2490733463 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9583+0+0 (secure 0 0 0) 0x7f7e98023430 con 0x5601a99a0350 2024-04-24T09:13:22.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.220+0000 7f7eb6214740 1 -- 172.21.15.83:0/2490733463 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5601a990ca80 con 0x5601a99a0350 2024-04-24T09:13:22.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.221+0000 7f7e92000640 1 -- 172.21.15.83:0/2490733463 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f7e98046050 con 0x5601a99a0350 2024-04-24T09:13:22.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.221+0000 7f7eb6214740 10 client.29923 fetch_fsmap learned FSMap version 64 2024-04-24T09:13:22.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.221+0000 7f7eb6214740 1 -- 172.21.15.83:0/2490733463 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x5601a98ca810 con 0x5601a99a0350 2024-04-24T09:13:22.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.221+0000 7f7e92000640 1 -- 172.21.15.83:0/2490733463 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x5601a990ca80 con 0x5601a99a0350 2024-04-24T09:13:22.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.222+0000 7f7e92000640 1 -- 172.21.15.83:0/2490733463 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 64) ==== 8374+0+0 (secure 0 0 0) 0x7f7e980049d0 con 0x5601a99a0350 2024-04-24T09:13:22.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.222+0000 7f7eb6214740 10 client.29923 fetch_fsmap finished waiting for FSMap version 64 2024-04-24T09:13:22.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.222+0000 7f7eb6214740 10 client.29923 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:13:22.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.222+0000 7f7eb6214740 20 client.29923 populate_metadata read hostname 'smithi083' 2024-04-24T09:13:22.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.222+0000 7f7eb6214740 1 --2- 172.21.15.83:0/2490733463 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5601a9a21320 0x5601a9a23740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:22.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.222+0000 7f7eb6214740 4 client.29923 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T09:13:22.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.222+0000 7f7eb6214740 1 -- 172.21.15.83:0/2490733463 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5601a9888360 con 0x5601a9a21320 2024-04-24T09:13:22.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.223+0000 7f7eb0a00640 1 --2- 172.21.15.83:0/2490733463 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5601a9a21320 0x5601a9a23740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:22.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.223+0000 7f7eb0a00640 1 --2- 172.21.15.83:0/2490733463 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5601a9a21320 0x5601a9a23740 crc :-1 s=READY pgs=540 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8ac0ffd4deed6ee server_cookie=49e0a41aceea5189 in_seq=0 out_seq=0 2024-04-24T09:13:22.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.223+0000 7f7e92000640 10 client.29923 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:13:22.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.224+0000 7f7e92000640 1 -- 172.21.15.83:0/2490733463 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5601a9888360 con 0x5601a9a21320 2024-04-24T09:13:22.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.224+0000 7f7e92000640 10 client.29923 handle_command_reply: tid=0 2024-04-24T09:13:22.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.224+0000 7f7eb6214740 1 client.29923 shutdown 2024-04-24T09:13:22.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.225+0000 7f7eb6214740 1 -- 172.21.15.83:0/2490733463 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5601a99a0350 msgr2=0x5601a99a4260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:22.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.225+0000 7f7eb6214740 1 --2- 172.21.15.83:0/2490733463 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5601a99a0350 0x5601a99a4260 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x5601a99a8fa0 tx=0x7f7e9800a2d0 comp rx=0 tx=0).stop 2024-04-24T09:13:22.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.225+0000 7f7eb6214740 1 -- 172.21.15.83:0/2490733463 shutdown_connections 2024-04-24T09:13:22.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.225+0000 7f7eb6214740 1 --2- 172.21.15.83:0/2490733463 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5601a9a21320 0x5601a9a23740 crc :-1 s=READY pgs=540 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:22.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.225+0000 7f7eb6214740 1 --2- 172.21.15.83:0/2490733463 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5601a99aaa20 0x5601a99a47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:22.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.225+0000 7f7eb6214740 1 --2- 172.21.15.83:0/2490733463 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5601a99a0350 0x5601a99a4260 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:22.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.225+0000 7f7eb6214740 1 --2- 172.21.15.83:0/2490733463 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5601a999fa30 0x5601a99b2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:22.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.225+0000 7f7eb6214740 1 -- 172.21.15.83:0/2490733463 >> 172.21.15.83:0/2490733463 conn(0x5601a999bd50 msgr2=0x5601a99a7970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:22.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.225+0000 7f7e92000640 0 client.29923 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:13:22.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.226+0000 7f7eb6214740 1 -- 172.21.15.83:0/2490733463 shutdown_connections 2024-04-24T09:13:22.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.226+0000 7f7eb6214740 1 -- 172.21.15.83:0/2490733463 wait complete. 2024-04-24T09:13:22.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.226+0000 7f7eb6214740 20 client.29923 trim_cache size 0 max 16384 2024-04-24T09:13:22.227 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T09:13:22.227 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T09:13:22.228 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-04-24T09:13:22.228 INFO:teuthology.orchestra.run.smithi083.stdout: "57eba8c5-264d-49ec-9eb8-a560a2ae52b4": { 2024-04-24T09:13:22.228 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-04-24T09:13:22.228 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "57eba8c5-264d-49ec-9eb8-a560a2ae52b4", 2024-04-24T09:13:22.228 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-04-24T09:13:22.228 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:13:22.228 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:13:22.228 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T09:13:22.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.227+0000 7f7eb1e00640 1 -- 172.21.15.83:0/1970676582 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f7e940a3e20 msgr2=0x7f7e940a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:22.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.227+0000 7f7eb1e00640 1 --2- 172.21.15.83:0/1970676582 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f7e940a3e20 0x7f7e940a62e0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f7ea4009fd0 tx=0x7f7ea4009500 comp rx=0 tx=0).stop 2024-04-24T09:13:22.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.227+0000 7f7eb1e00640 1 -- 172.21.15.83:0/1970676582 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7eac076530 msgr2=0x7f7eac1a4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:22.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.227+0000 7f7eb1e00640 1 --2- 172.21.15.83:0/1970676582 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7eac076530 0x7f7eac1a4900 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7ea0005ab0 tx=0x7f7ea0003b90 comp rx=0 tx=0).stop 2024-04-24T09:13:22.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.227+0000 7f7eb1e00640 1 -- 172.21.15.83:0/1970676582 shutdown_connections 2024-04-24T09:13:22.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.227+0000 7f7eb1e00640 1 --2- 172.21.15.83:0/1970676582 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f7e940a3e20 0x7f7e940a62e0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:22.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.227+0000 7f7eb1e00640 1 --2- 172.21.15.83:0/1970676582 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7eac112d40 0x7f7eac19eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:22.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.227+0000 7f7eb1e00640 1 --2- 172.21.15.83:0/1970676582 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7eac076e50 0x7f7eac1a4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:22.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.227+0000 7f7eb1e00640 1 --2- 172.21.15.83:0/1970676582 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7eac076530 0x7f7eac1a4900 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:22.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.227+0000 7f7eb1e00640 1 -- 172.21.15.83:0/1970676582 >> 172.21.15.83:0/1970676582 conn(0x7f7eac0ffb70 msgr2=0x7f7eac101f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:22.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.228+0000 7f7eb1e00640 1 -- 172.21.15.83:0/1970676582 shutdown_connections 2024-04-24T09:13:22.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:22.228+0000 7f7eb1e00640 1 -- 172.21.15.83:0/1970676582 wait complete. 2024-04-24T09:13:22.249 INFO:tasks.cephfs.filesystem:scrub status for tag:57eba8c5-264d-49ec-9eb8-a560a2ae52b4 - {'path': '/', 'tag': '57eba8c5-264d-49ec-9eb8-a560a2ae52b4', 'options': 'recursive,force'} 2024-04-24T09:13:22.432 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.431+0000 7ff13c400640 1 -- 172.21.15.83:0/1119043673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff134108fb0 msgr2=0x7ff1341154c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:22.432 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.431+0000 7ff13c400640 1 --2- 172.21.15.83:0/1119043673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff134108fb0 0x7ff1341154c0 secure :-1 s=READY pgs=2624 cs=0 l=1 rev1=1 crypto rx=0x7ff12c009cd0 tx=0x7ff12c02a910 comp rx=0 tx=0).stop 2024-04-24T09:13:22.432 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.431+0000 7ff13c400640 1 -- 172.21.15.83:0/1119043673 shutdown_connections 2024-04-24T09:13:22.432 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.431+0000 7ff13c400640 1 --2- 172.21.15.83:0/1119043673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff134108fb0 0x7ff1341154c0 unknown :-1 s=CLOSED pgs=2624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:22.432 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.431+0000 7ff13c400640 1 --2- 172.21.15.83:0/1119043673 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1341085f0 0x7ff134108a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:22.432 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.431+0000 7ff13c400640 1 --2- 172.21.15.83:0/1119043673 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff13410e690 0x7ff13410ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:22.433 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.431+0000 7ff13c400640 1 -- 172.21.15.83:0/1119043673 >> 172.21.15.83:0/1119043673 conn(0x7ff134076590 msgr2=0x7ff1340789b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:22.433 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.432+0000 7ff13c400640 1 -- 172.21.15.83:0/1119043673 shutdown_connections 2024-04-24T09:13:22.433 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.432+0000 7ff13c400640 1 -- 172.21.15.83:0/1119043673 wait complete. 2024-04-24T09:13:22.433 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.433+0000 7ff13c400640 1 Processor -- start 2024-04-24T09:13:22.433 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.433+0000 7ff13c400640 1 -- start start 2024-04-24T09:13:22.435 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.434+0000 7ff13c400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff1341085f0 0x7ff1341a4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:22.435 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.434+0000 7ff13c400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff134108fb0 0x7ff1341a4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:22.435 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.434+0000 7ff13c400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff13410e690 0x7ff13419eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:22.435 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.434+0000 7ff13c400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff1341188d0 con 0x7ff1341085f0 2024-04-24T09:13:22.435 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.434+0000 7ff13c400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff134118750 con 0x7ff13410e690 2024-04-24T09:13:22.435 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.434+0000 7ff13c400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff134118a50 con 0x7ff134108fb0 2024-04-24T09:13:22.435 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.434+0000 7ff139a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff1341085f0 0x7ff1341a4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:22.435 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.434+0000 7ff139a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff1341085f0 0x7ff1341a4900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35386/0 (socket says 172.21.15.83:35386) 2024-04-24T09:13:22.435 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.434+0000 7ff139a00640 1 -- 172.21.15.83:0/712692028 learned_addr learned my addr 172.21.15.83:0/712692028 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:13:22.435 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.434+0000 7ff139000640 1 --2- 172.21.15.83:0/712692028 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff134108fb0 0x7ff1341a4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:22.435 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.435+0000 7ff13a400640 1 --2- 172.21.15.83:0/712692028 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff13410e690 0x7ff13419eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:22.436 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.435+0000 7ff139a00640 1 -- 172.21.15.83:0/712692028 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff134108fb0 msgr2=0x7ff1341a4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:22.436 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.435+0000 7ff139a00640 1 --2- 172.21.15.83:0/712692028 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff134108fb0 0x7ff1341a4e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:22.436 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.435+0000 7ff139a00640 1 -- 172.21.15.83:0/712692028 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff13410e690 msgr2=0x7ff13419eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:22.437 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.435+0000 7ff139a00640 1 --2- 172.21.15.83:0/712692028 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff13410e690 0x7ff13419eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:22.437 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.435+0000 7ff139a00640 1 -- 172.21.15.83:0/712692028 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff13419f230 con 0x7ff1341085f0 2024-04-24T09:13:22.437 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.436+0000 7ff139a00640 1 --2- 172.21.15.83:0/712692028 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff1341085f0 0x7ff1341a4900 secure :-1 s=READY pgs=2625 cs=0 l=1 rev1=1 crypto rx=0x7ff12800d770 tx=0x7ff12800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:22.437 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.436+0000 7ff132000640 1 -- 172.21.15.83:0/712692028 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff128017070 con 0x7ff1341085f0 2024-04-24T09:13:22.437 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.436+0000 7ff132000640 1 -- 172.21.15.83:0/712692028 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff128009c30 con 0x7ff1341085f0 2024-04-24T09:13:22.437 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.436+0000 7ff13c400640 1 -- 172.21.15.83:0/712692028 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff13419f520 con 0x7ff1341085f0 2024-04-24T09:13:22.437 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.436+0000 7ff132000640 1 -- 172.21.15.83:0/712692028 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff128003d00 con 0x7ff1341085f0 2024-04-24T09:13:22.437 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.436+0000 7ff13c400640 1 -- 172.21.15.83:0/712692028 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1341ab7c0 con 0x7ff1341085f0 2024-04-24T09:13:22.440 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.440+0000 7ff13f3c5740 1 Processor -- start 2024-04-24T09:13:22.442 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.441+0000 7ff13f3c5740 1 -- start start 2024-04-24T09:13:22.442 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.441+0000 7ff132000640 1 -- 172.21.15.83:0/712692028 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff128003ea0 con 0x7ff1341085f0 2024-04-24T09:13:22.442 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.441+0000 7ff13f3c5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562e86a2a1b0 0x562e86a2a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:22.442 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.441+0000 7ff13f3c5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562e86a2aad0 0x562e86a34e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:22.442 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.441+0000 7ff13f3c5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562e86a35390 0x562e86a37780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:22.443 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.441+0000 7ff13f3c5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562e86990900 con 0x562e86a2aad0 2024-04-24T09:13:22.443 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.441+0000 7ff13f3c5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562e86936f20 con 0x562e86a35390 2024-04-24T09:13:22.443 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.441+0000 7ff13f3c5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x562e8698c8c0 con 0x562e86a2a1b0 2024-04-24T09:13:22.443 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.441+0000 7ff139000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562e86a35390 0x562e86a37780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:22.443 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.441+0000 7ff139000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562e86a35390 0x562e86a37780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:38290/0 (socket says 172.21.15.83:38290) 2024-04-24T09:13:22.443 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.441+0000 7ff139000640 1 -- 172.21.15.83:0/3140536721 learned_addr learned my addr 172.21.15.83:0/3140536721 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:13:22.443 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.442+0000 7ff13a400640 1 --2- 172.21.15.83:0/3140536721 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562e86a2a1b0 0x562e86a2a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:22.443 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.442+0000 7ff139a00640 1 --2- 172.21.15.83:0/3140536721 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562e86a2aad0 0x562e86a34e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:22.443 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.442+0000 7ff139000640 1 -- 172.21.15.83:0/3140536721 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562e86a2a1b0 msgr2=0x562e86a2a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:22.443 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.442+0000 7ff139000640 1 --2- 172.21.15.83:0/3140536721 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562e86a2a1b0 0x562e86a2a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:22.443 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.442+0000 7ff139000640 1 -- 172.21.15.83:0/3140536721 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562e86a2aad0 msgr2=0x562e86a34e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:22.443 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.442+0000 7ff139000640 1 --2- 172.21.15.83:0/3140536721 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562e86a2aad0 0x562e86a34e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:22.443 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.442+0000 7ff139000640 1 -- 172.21.15.83:0/3140536721 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e8695be10 con 0x562e86a35390 2024-04-24T09:13:22.444 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.442+0000 7ff13a400640 1 --2- 172.21.15.83:0/3140536721 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562e86a2a1b0 0x562e86a2a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:13:22.445 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.442+0000 7ff139000640 1 --2- 172.21.15.83:0/3140536721 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562e86a35390 0x562e86a37780 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7ff120009c60 tx=0x7ff120011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd49b20c8d6d402d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:22.445 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.443+0000 7ff126a00640 1 -- 172.21.15.83:0/3140536721 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff120004230 con 0x562e86a35390 2024-04-24T09:13:22.445 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.443+0000 7ff126a00640 1 -- 172.21.15.83:0/3140536721 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff1200043d0 con 0x562e86a35390 2024-04-24T09:13:22.445 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.443+0000 7ff126a00640 1 -- 172.21.15.83:0/3140536721 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff120004d30 con 0x562e86a35390 2024-04-24T09:13:22.445 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.443+0000 7ff132000640 1 --2- 172.21.15.83:0/712692028 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff11c0a3ec0 0x7ff11c0a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:22.446 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.443+0000 7ff132000640 1 -- 172.21.15.83:0/712692028 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 9583+0+0 (secure 0 0 0) 0x7ff1280d0870 con 0x7ff1341085f0 2024-04-24T09:13:22.446 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.443+0000 7ff13a400640 1 --2- 172.21.15.83:0/712692028 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff11c0a3ec0 0x7ff11c0a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:22.446 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.444+0000 7ff13f3c5740 1 -- 172.21.15.83:0/3140536721 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562e86a35390 msgr2=0x562e86a37780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:22.446 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.444+0000 7ff13f3c5740 1 --2- 172.21.15.83:0/3140536721 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562e86a35390 0x562e86a37780 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7ff120009c60 tx=0x7ff120011810 comp rx=0 tx=0).stop 2024-04-24T09:13:22.446 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.444+0000 7ff13a400640 1 --2- 172.21.15.83:0/712692028 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff11c0a3ec0 0x7ff11c0a6380 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7ff12c004af0 tx=0x7ff12c005d80 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:22.446 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.444+0000 7ff13f3c5740 1 -- 172.21.15.83:0/3140536721 shutdown_connections 2024-04-24T09:13:22.446 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.444+0000 7ff13f3c5740 1 --2- 172.21.15.83:0/3140536721 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562e86a35390 0x562e86a37780 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:22.446 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.445+0000 7ff13f3c5740 1 --2- 172.21.15.83:0/3140536721 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562e86a2aad0 0x562e86a34e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:22.446 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.445+0000 7ff13f3c5740 1 --2- 172.21.15.83:0/3140536721 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562e86a2a1b0 0x562e86a2a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:22.446 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.445+0000 7ff13f3c5740 1 -- 172.21.15.83:0/3140536721 >> 172.21.15.83:0/3140536721 conn(0x562e86a266c0 msgr2=0x562e86a26aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:22.446 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.445+0000 7ff13f3c5740 1 -- 172.21.15.83:0/3140536721 shutdown_connections 2024-04-24T09:13:22.447 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.445+0000 7ff13f3c5740 1 -- 172.21.15.83:0/3140536721 wait complete. 2024-04-24T09:13:22.448 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.446+0000 7ff13f3c5740 1 -- start start 2024-04-24T09:13:22.448 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.446+0000 7ff13f3c5740 1 Processor -- start 2024-04-24T09:13:22.449 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.446+0000 7ff13f3c5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562e86a2a1b0 0x562e86a322f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:22.449 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.446+0000 7ff13f3c5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562e86a2aad0 0x562e86a32830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:22.449 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.447+0000 7ff13f3c5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562e86a35390 0x562e86a32d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:22.449 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.447+0000 7ff13f3c5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562e86a48fd0 con 0x562e86a2a1b0 2024-04-24T09:13:22.449 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.447+0000 7ff13f3c5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562e8694a2c0 con 0x562e86a35390 2024-04-24T09:13:22.449 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.447+0000 7ff13f3c5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x562e86a49150 con 0x562e86a2aad0 2024-04-24T09:13:22.449 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.447+0000 7ff13f3c5740 10 client.0 _ll_register_callbacks cb 0x562e866a2a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:13:22.449 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.447+0000 7ff139a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562e86a2a1b0 0x562e86a322f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:22.449 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.447+0000 7ff139a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562e86a2a1b0 0x562e86a322f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35392/0 (socket says 172.21.15.83:35392) 2024-04-24T09:13:22.449 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.447+0000 7ff139a00640 1 -- 172.21.15.83:0/865234565 learned_addr learned my addr 172.21.15.83:0/865234565 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:13:22.450 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.447+0000 7ff13a400640 1 --2- 172.21.15.83:0/865234565 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562e86a35390 0x562e86a32d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:22.450 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.447+0000 7ff139a00640 1 -- 172.21.15.83:0/865234565 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562e86a2aad0 msgr2=0x562e86a32830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:22.450 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.447+0000 7ff139000640 1 --2- 172.21.15.83:0/865234565 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562e86a2aad0 0x562e86a32830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:22.450 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.447+0000 7ff139a00640 1 --2- 172.21.15.83:0/865234565 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562e86a2aad0 0x562e86a32830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:22.450 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.447+0000 7ff139a00640 1 -- 172.21.15.83:0/865234565 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562e86a35390 msgr2=0x562e86a32d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:22.450 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.447+0000 7ff139a00640 1 --2- 172.21.15.83:0/865234565 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562e86a35390 0x562e86a32d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:22.450 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.447+0000 7ff139a00640 1 -- 172.21.15.83:0/865234565 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e8695afd0 con 0x562e86a2a1b0 2024-04-24T09:13:22.450 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.448+0000 7ff13a400640 1 --2- 172.21.15.83:0/865234565 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562e86a35390 0x562e86a32d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:13:22.450 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.448+0000 7ff139000640 1 --2- 172.21.15.83:0/865234565 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562e86a2aad0 0x562e86a32830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:13:22.450 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.448+0000 7ff139a00640 1 --2- 172.21.15.83:0/865234565 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562e86a2a1b0 0x562e86a322f0 secure :-1 s=READY pgs=2626 cs=0 l=1 rev1=1 crypto rx=0x7ff12809c140 tx=0x7ff12809c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c40d8b579a3b6762 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:22.450 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.448+0000 7ff11b400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:13:22.450 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.448+0000 7ff11b400640 1 -- 172.21.15.83:0/865234565 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff128020050 con 0x562e86a2a1b0 2024-04-24T09:13:22.451 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.448+0000 7ff11b400640 1 -- 172.21.15.83:0/865234565 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9583+0+0 (secure 0 0 0) 0x7ff128098ab0 con 0x562e86a2a1b0 2024-04-24T09:13:22.451 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.448+0000 7ff13f3c5740 1 -- 172.21.15.83:0/865234565 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1200043d0 con 0x562e86a2a1b0 2024-04-24T09:13:22.452 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.449+0000 7ff11b400640 1 -- 172.21.15.83:0/865234565 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7ff12809c440 con 0x562e86a2a1b0 2024-04-24T09:13:22.452 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.449+0000 7ff13f3c5740 10 client.23082 fetch_fsmap learned FSMap version 64 2024-04-24T09:13:22.452 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.449+0000 7ff13f3c5740 1 -- 172.21.15.83:0/865234565 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x562e866a3210 con 0x562e86a2a1b0 2024-04-24T09:13:22.452 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.450+0000 7ff11b400640 1 -- 172.21.15.83:0/865234565 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff1200043d0 con 0x562e86a2a1b0 2024-04-24T09:13:22.452 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.450+0000 7ff11b400640 1 -- 172.21.15.83:0/865234565 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 64) ==== 8374+0+0 (secure 0 0 0) 0x7ff1280d0070 con 0x562e86a2a1b0 2024-04-24T09:13:22.452 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.450+0000 7ff13f3c5740 10 client.23082 fetch_fsmap finished waiting for FSMap version 64 2024-04-24T09:13:22.452 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.450+0000 7ff13f3c5740 10 client.23082 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:13:22.452 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.450+0000 7ff13f3c5740 20 client.23082 populate_metadata read hostname 'smithi083' 2024-04-24T09:13:22.452 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.450+0000 7ff13f3c5740 1 --2- 172.21.15.83:0/865234565 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x562e86a56580 0x562e86a589b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:22.452 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.451+0000 7ff13f3c5740 4 client.23082 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:13:22.452 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.451+0000 7ff13f3c5740 1 -- 172.21.15.83:0/865234565 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562e8699e320 con 0x562e86a56580 2024-04-24T09:13:22.452 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.451+0000 7ff139000640 1 --2- 172.21.15.83:0/865234565 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x562e86a56580 0x562e86a589b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:22.453 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.452+0000 7ff139000640 1 --2- 172.21.15.83:0/865234565 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x562e86a56580 0x562e86a589b0 crc :-1 s=READY pgs=541 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3cbc0cf5c8fe62b5 server_cookie=3c343f0fa3602bae in_seq=0 out_seq=0 2024-04-24T09:13:22.453 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.452+0000 7ff11b400640 10 client.23082 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:13:22.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:13:22 smithi138 ceph-mon[31441]: pgmap v687: 129 pgs: 129 active+clean; 912 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.5 MiB/s wr, 222 op/s 2024-04-24T09:13:22.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:13:22 smithi138 ceph-mon[31441]: scrub queued for path: / 2024-04-24T09:13:22.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:13:22 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:13:22.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:13:22 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:13:22.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:13:22 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:13:22.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:13:22 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:13:22.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:13:22 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:13:22.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:13:22 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:13:22.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:13:22 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:13:22.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:13:22 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:13:22.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:13:22 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:13:22.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:13:22 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:13:22.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:13:22 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:13:22.457 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.455+0000 7ff11b400640 1 -- 172.21.15.83:0/865234565 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562e8699e320 con 0x562e86a56580 2024-04-24T09:13:22.457 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.455+0000 7ff11b400640 10 client.23082 handle_command_reply: tid=0 2024-04-24T09:13:22.457 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.455+0000 7ff13f3c5740 1 client.23082 shutdown 2024-04-24T09:13:22.458 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.456+0000 7ff13f3c5740 1 -- 172.21.15.83:0/865234565 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562e86a2a1b0 msgr2=0x562e86a322f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:22.458 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.456+0000 7ff13f3c5740 1 --2- 172.21.15.83:0/865234565 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562e86a2a1b0 0x562e86a322f0 secure :-1 s=READY pgs=2626 cs=0 l=1 rev1=1 crypto rx=0x7ff12809c140 tx=0x7ff12809c2d0 comp rx=0 tx=0).stop 2024-04-24T09:13:22.458 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.456+0000 7ff13f3c5740 1 -- 172.21.15.83:0/865234565 shutdown_connections 2024-04-24T09:13:22.458 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.456+0000 7ff13f3c5740 1 --2- 172.21.15.83:0/865234565 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x562e86a56580 0x562e86a589b0 crc :-1 s=READY pgs=541 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:22.458 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.456+0000 7ff13f3c5740 1 --2- 172.21.15.83:0/865234565 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562e86a35390 0x562e86a32d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:22.458 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.456+0000 7ff13f3c5740 1 --2- 172.21.15.83:0/865234565 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562e86a2aad0 0x562e86a32830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:22.458 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.456+0000 7ff13f3c5740 1 --2- 172.21.15.83:0/865234565 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562e86a2a1b0 0x562e86a322f0 unknown :-1 s=CLOSED pgs=2626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:22.458 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.456+0000 7ff11b400640 0 client.23082 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:13:22.458 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.456+0000 7ff13f3c5740 1 -- 172.21.15.83:0/865234565 >> 172.21.15.83:0/865234565 conn(0x562e86a266c0 msgr2=0x562e86a2c440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:22.458 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.456+0000 7ff13f3c5740 1 -- 172.21.15.83:0/865234565 shutdown_connections 2024-04-24T09:13:22.458 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.457+0000 7ff13f3c5740 1 -- 172.21.15.83:0/865234565 wait complete. 2024-04-24T09:13:22.458 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.457+0000 7ff13f3c5740 20 client.23082 trim_cache size 0 max 16384 2024-04-24T09:13:22.459 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:13:22 smithi189 ceph-mon[31791]: pgmap v687: 129 pgs: 129 active+clean; 912 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.5 MiB/s wr, 222 op/s 2024-04-24T09:13:22.459 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:13:22 smithi189 ceph-mon[31791]: scrub queued for path: / 2024-04-24T09:13:22.459 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:13:22 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:13:22.459 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:13:22 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:13:22.459 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:13:22 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:13:22.459 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:13:22 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:13:22.459 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:13:22 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:13:22.459 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:13:22 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:13:22.459 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:13:22 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:13:22.460 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:13:22 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:13:22.460 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:13:22 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:13:22.460 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:13:22 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:13:22.460 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:13:22 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:13:22.484 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.483+0000 7ff13f3c5740 1 Processor -- start 2024-04-24T09:13:22.484 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.484+0000 7ff13f3c5740 1 -- start start 2024-04-24T09:13:22.485 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.484+0000 7ff13f3c5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562e86a2a1b0 0x562e86a2cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:22.485 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.484+0000 7ff13f3c5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562e86a2aad0 0x562e86a32b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:22.485 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.484+0000 7ff13f3c5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562e86a35390 0x562e86a5e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:22.485 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.484+0000 7ff13a400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562e86a35390 0x562e86a5e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:22.485 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.485+0000 7ff13a400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562e86a35390 0x562e86a5e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35402/0 (socket says 172.21.15.83:35402) 2024-04-24T09:13:22.485 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.485+0000 7ff13a400640 1 -- 172.21.15.83:0/1810404936 learned_addr learned my addr 172.21.15.83:0/1810404936 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:13:22.485 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.485+0000 7ff13f3c5740 1 -- 172.21.15.83:0/1810404936 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff1341ad300 con 0x562e86a35390 2024-04-24T09:13:22.485 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.485+0000 7ff13f3c5740 1 -- 172.21.15.83:0/1810404936 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff1341aeed0 con 0x562e86a2aad0 2024-04-24T09:13:22.486 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.485+0000 7ff13f3c5740 1 -- 172.21.15.83:0/1810404936 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff1341ad9d0 con 0x562e86a2a1b0 2024-04-24T09:13:22.486 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.485+0000 7ff139000640 1 --2- 172.21.15.83:0/1810404936 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562e86a2aad0 0x562e86a32b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:22.486 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.485+0000 7ff139a00640 1 --2- 172.21.15.83:0/1810404936 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562e86a2a1b0 0x562e86a2cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:22.486 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.485+0000 7ff13a400640 1 -- 172.21.15.83:0/1810404936 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562e86a2a1b0 msgr2=0x562e86a2cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:22.486 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.485+0000 7ff13a400640 1 --2- 172.21.15.83:0/1810404936 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562e86a2a1b0 0x562e86a2cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:22.486 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.485+0000 7ff13a400640 1 -- 172.21.15.83:0/1810404936 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562e86a2aad0 msgr2=0x562e86a32b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:22.486 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.485+0000 7ff13a400640 1 --2- 172.21.15.83:0/1810404936 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562e86a2aad0 0x562e86a32b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:22.486 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.485+0000 7ff13a400640 1 -- 172.21.15.83:0/1810404936 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e8676f620 con 0x562e86a35390 2024-04-24T09:13:22.487 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.486+0000 7ff13a400640 1 --2- 172.21.15.83:0/1810404936 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562e86a35390 0x562e86a5e2c0 secure :-1 s=READY pgs=2627 cs=0 l=1 rev1=1 crypto rx=0x7ff12c035000 tx=0x7ff12c041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a10ee6262e16ab61 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:22.487 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.486+0000 7ff126000640 1 -- 172.21.15.83:0/1810404936 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff12c031660 con 0x562e86a35390 2024-04-24T09:13:22.487 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.486+0000 7ff126000640 1 -- 172.21.15.83:0/1810404936 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff12c036050 con 0x562e86a35390 2024-04-24T09:13:22.487 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.486+0000 7ff126000640 1 -- 172.21.15.83:0/1810404936 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff12c004450 con 0x562e86a35390 2024-04-24T09:13:22.487 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.487+0000 7ff13f3c5740 1 -- 172.21.15.83:0/1810404936 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562e86a35390 msgr2=0x562e86a5e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:22.487 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.487+0000 7ff13f3c5740 1 --2- 172.21.15.83:0/1810404936 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562e86a35390 0x562e86a5e2c0 secure :-1 s=READY pgs=2627 cs=0 l=1 rev1=1 crypto rx=0x7ff12c035000 tx=0x7ff12c041000 comp rx=0 tx=0).stop 2024-04-24T09:13:22.488 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.487+0000 7ff13f3c5740 1 -- 172.21.15.83:0/1810404936 shutdown_connections 2024-04-24T09:13:22.488 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.487+0000 7ff13f3c5740 1 --2- 172.21.15.83:0/1810404936 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562e86a35390 0x562e86a5e2c0 unknown :-1 s=CLOSED pgs=2627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:22.488 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.487+0000 7ff13f3c5740 1 --2- 172.21.15.83:0/1810404936 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562e86a2aad0 0x562e86a32b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:22.488 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.487+0000 7ff13f3c5740 1 --2- 172.21.15.83:0/1810404936 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562e86a2a1b0 0x562e86a2cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:22.488 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.487+0000 7ff13f3c5740 1 -- 172.21.15.83:0/1810404936 >> 172.21.15.83:0/1810404936 conn(0x562e86a266c0 msgr2=0x562e86a25840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:22.488 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.488+0000 7ff13f3c5740 1 -- 172.21.15.83:0/1810404936 shutdown_connections 2024-04-24T09:13:22.489 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.488+0000 7ff13f3c5740 1 -- 172.21.15.83:0/1810404936 wait complete. 2024-04-24T09:13:22.489 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.488+0000 7ff13f3c5740 1 -- start start 2024-04-24T09:13:22.489 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.489+0000 7ff13f3c5740 1 Processor -- start 2024-04-24T09:13:22.490 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.489+0000 7ff13f3c5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562e86a2a1b0 0x562e86a2f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:22.490 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.489+0000 7ff13f3c5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562e86a2aad0 0x562e86a2f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:22.490 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.489+0000 7ff13f3c5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562e86a35390 0x562e86a2fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:22.491 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.489+0000 7ff13f3c5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff1341ae7d0 con 0x562e86a2a1b0 2024-04-24T09:13:22.491 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.489+0000 7ff13f3c5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff1341ae100 con 0x562e86a2aad0 2024-04-24T09:13:22.491 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.489+0000 7ff13f3c5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff13404fd90 con 0x562e86a35390 2024-04-24T09:13:22.491 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.489+0000 7ff139a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562e86a2a1b0 0x562e86a2f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:22.491 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.489+0000 7ff13f3c5740 10 client.0 _ll_register_callbacks cb 0x7ff1280987b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:13:22.491 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.490+0000 7ff139a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562e86a2a1b0 0x562e86a2f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35416/0 (socket says 172.21.15.83:35416) 2024-04-24T09:13:22.491 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.490+0000 7ff139a00640 1 -- 172.21.15.83:0/3512593217 learned_addr learned my addr 172.21.15.83:0/3512593217 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:13:22.491 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.490+0000 7ff139000640 1 --2- 172.21.15.83:0/3512593217 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562e86a2aad0 0x562e86a2f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:22.491 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.490+0000 7ff139a00640 1 -- 172.21.15.83:0/3512593217 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562e86a35390 msgr2=0x562e86a2fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:22.491 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.490+0000 7ff13a400640 1 --2- 172.21.15.83:0/3512593217 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562e86a35390 0x562e86a2fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:22.491 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.490+0000 7ff139a00640 1 --2- 172.21.15.83:0/3512593217 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562e86a35390 0x562e86a2fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:22.491 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.490+0000 7ff139a00640 1 -- 172.21.15.83:0/3512593217 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562e86a2aad0 msgr2=0x562e86a2f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:22.491 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.490+0000 7ff139a00640 1 --2- 172.21.15.83:0/3512593217 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562e86a2aad0 0x562e86a2f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:22.491 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.490+0000 7ff139a00640 1 -- 172.21.15.83:0/3512593217 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e86accce0 con 0x562e86a2a1b0 2024-04-24T09:13:22.492 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.490+0000 7ff13a400640 1 --2- 172.21.15.83:0/3512593217 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562e86a35390 0x562e86a2fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:13:22.493 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.491+0000 7ff139a00640 1 --2- 172.21.15.83:0/3512593217 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562e86a2a1b0 0x562e86a2f130 secure :-1 s=READY pgs=2628 cs=0 l=1 rev1=1 crypto rx=0x7ff12809a6f0 tx=0x7ff1280cdff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=805a1b78361ca7e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:22.493 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.491+0000 7ff11b400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:13:22.493 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.491+0000 7ff11b400640 1 -- 172.21.15.83:0/3512593217 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff128020070 con 0x562e86a2a1b0 2024-04-24T09:13:22.493 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.491+0000 7ff11b400640 1 -- 172.21.15.83:0/3512593217 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9583+0+0 (secure 0 0 0) 0x7ff1280d0570 con 0x562e86a2a1b0 2024-04-24T09:13:22.493 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.491+0000 7ff13f3c5740 1 -- 172.21.15.83:0/3512593217 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff12c036050 con 0x562e86a2a1b0 2024-04-24T09:13:22.493 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.491+0000 7ff11b400640 1 -- 172.21.15.83:0/3512593217 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7ff12809bda0 con 0x562e86a2a1b0 2024-04-24T09:13:22.493 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.491+0000 7ff13f3c5740 10 client.23088 fetch_fsmap learned FSMap version 64 2024-04-24T09:13:22.494 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.491+0000 7ff13f3c5740 1 -- 172.21.15.83:0/3512593217 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x562e86accce0 con 0x562e86a2a1b0 2024-04-24T09:13:22.494 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.492+0000 7ff11b400640 1 -- 172.21.15.83:0/3512593217 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff12c036050 con 0x562e86a2a1b0 2024-04-24T09:13:22.494 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.492+0000 7ff11b400640 1 -- 172.21.15.83:0/3512593217 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 64) ==== 8374+0+0 (secure 0 0 0) 0x7ff1280d0040 con 0x562e86a2a1b0 2024-04-24T09:13:22.494 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.492+0000 7ff13f3c5740 10 client.23088 fetch_fsmap finished waiting for FSMap version 64 2024-04-24T09:13:22.494 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.492+0000 7ff13f3c5740 10 client.23088 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:13:22.494 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.492+0000 7ff13f3c5740 20 client.23088 populate_metadata read hostname 'smithi083' 2024-04-24T09:13:22.494 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.492+0000 7ff13f3c5740 1 --2- 172.21.15.83:0/3512593217 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x562e86ace450 0x562e86ad0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:22.494 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.492+0000 7ff13f3c5740 4 client.23088 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T09:13:22.494 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.492+0000 7ff13f3c5740 1 -- 172.21.15.83:0/3512593217 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562e86998190 con 0x562e86ace450 2024-04-24T09:13:22.494 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.492+0000 7ff139000640 1 --2- 172.21.15.83:0/3512593217 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x562e86ace450 0x562e86ad0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:22.494 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.493+0000 7ff139000640 1 --2- 172.21.15.83:0/3512593217 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x562e86ace450 0x562e86ad0870 crc :-1 s=READY pgs=542 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b3791f04dc0abc7 server_cookie=454d74b22b1c1f23 in_seq=0 out_seq=0 2024-04-24T09:13:22.494 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.493+0000 7ff11b400640 10 client.23088 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:13:22.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:13:22 smithi083 ceph-mon[27907]: pgmap v687: 129 pgs: 129 active+clean; 912 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.5 MiB/s wr, 222 op/s 2024-04-24T09:13:22.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:13:22 smithi083 ceph-mon[27907]: scrub queued for path: / 2024-04-24T09:13:22.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:13:22 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:13:22.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:13:22 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:13:22.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:13:22 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:13:22.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:13:22 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:13:22.555 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:13:22 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:13:22.555 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:13:22 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:13:22.555 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:13:22 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:13:22.555 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:13:22 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:13:22.555 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:13:22 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:13:22.555 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:13:22 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:13:22.555 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:13:22 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:13:22.627 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.627+0000 7ff11b400640 1 -- 172.21.15.83:0/3512593217 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562e86998190 con 0x562e86ace450 2024-04-24T09:13:22.628 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.627+0000 7ff11b400640 10 client.23088 handle_command_reply: tid=0 2024-04-24T09:13:22.628 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.627+0000 7ff13f3c5740 1 client.23088 shutdown 2024-04-24T09:13:22.629 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.628+0000 7ff13f3c5740 1 -- 172.21.15.83:0/3512593217 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562e86a2a1b0 msgr2=0x562e86a2f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:22.629 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.628+0000 7ff13f3c5740 1 --2- 172.21.15.83:0/3512593217 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562e86a2a1b0 0x562e86a2f130 secure :-1 s=READY pgs=2628 cs=0 l=1 rev1=1 crypto rx=0x7ff12809a6f0 tx=0x7ff1280cdff0 comp rx=0 tx=0).stop 2024-04-24T09:13:22.629 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.628+0000 7ff13f3c5740 1 -- 172.21.15.83:0/3512593217 shutdown_connections 2024-04-24T09:13:22.629 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.628+0000 7ff13f3c5740 1 --2- 172.21.15.83:0/3512593217 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x562e86ace450 0x562e86ad0870 crc :-1 s=READY pgs=542 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:22.629 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.628+0000 7ff13f3c5740 1 --2- 172.21.15.83:0/3512593217 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562e86a35390 0x562e86a2fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:22.629 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.628+0000 7ff11b400640 0 client.23088 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:13:22.629 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.628+0000 7ff13f3c5740 1 --2- 172.21.15.83:0/3512593217 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562e86a2aad0 0x562e86a2f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:22.629 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.628+0000 7ff13f3c5740 1 --2- 172.21.15.83:0/3512593217 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562e86a2a1b0 0x562e86a2f130 unknown :-1 s=CLOSED pgs=2628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:22.629 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.628+0000 7ff13f3c5740 1 -- 172.21.15.83:0/3512593217 >> 172.21.15.83:0/3512593217 conn(0x562e86a266c0 msgr2=0x562e86a33b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:22.629 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.629+0000 7ff13f3c5740 1 -- 172.21.15.83:0/3512593217 shutdown_connections 2024-04-24T09:13:22.631 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.630+0000 7ff13f3c5740 1 -- 172.21.15.83:0/3512593217 wait complete. 2024-04-24T09:13:22.631 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.631+0000 7ff13f3c5740 20 client.23088 trim_cache size 0 max 16384 2024-04-24T09:13:22.631 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T09:13:22.631 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T09:13:22.632 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T09:13:22.632 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T09:13:22.633 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.633+0000 7ff13c400640 1 -- 172.21.15.83:0/712692028 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff11c0a3ec0 msgr2=0x7ff11c0a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:22.633 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.633+0000 7ff13c400640 1 --2- 172.21.15.83:0/712692028 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff11c0a3ec0 0x7ff11c0a6380 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7ff12c004af0 tx=0x7ff12c005d80 comp rx=0 tx=0).stop 2024-04-24T09:13:22.633 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.633+0000 7ff13c400640 1 -- 172.21.15.83:0/712692028 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff1341085f0 msgr2=0x7ff1341a4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:22.633 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.633+0000 7ff13c400640 1 --2- 172.21.15.83:0/712692028 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff1341085f0 0x7ff1341a4900 secure :-1 s=READY pgs=2625 cs=0 l=1 rev1=1 crypto rx=0x7ff12800d770 tx=0x7ff12800dc30 comp rx=0 tx=0).stop 2024-04-24T09:13:22.633 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.633+0000 7ff13c400640 1 -- 172.21.15.83:0/712692028 shutdown_connections 2024-04-24T09:13:22.633 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.633+0000 7ff13c400640 1 --2- 172.21.15.83:0/712692028 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff11c0a3ec0 0x7ff11c0a6380 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:22.634 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.633+0000 7ff13c400640 1 --2- 172.21.15.83:0/712692028 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff13410e690 0x7ff13419eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:22.634 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.633+0000 7ff13c400640 1 --2- 172.21.15.83:0/712692028 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff134108fb0 0x7ff1341a4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:22.634 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.633+0000 7ff13c400640 1 --2- 172.21.15.83:0/712692028 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff1341085f0 0x7ff1341a4900 unknown :-1 s=CLOSED pgs=2625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:22.634 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.633+0000 7ff13c400640 1 -- 172.21.15.83:0/712692028 >> 172.21.15.83:0/712692028 conn(0x7ff134076590 msgr2=0x7ff134077aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:22.634 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.633+0000 7ff13c400640 1 -- 172.21.15.83:0/712692028 shutdown_connections 2024-04-24T09:13:22.634 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:22.633+0000 7ff13c400640 1 -- 172.21.15.83:0/712692028 wait complete. 2024-04-24T09:13:23.197 INFO:tasks.workunit.client.0.smithi083.stdout: 10 58861 6.61 MB/sec execute 354 sec latency 106.226 ms 2024-04-24T09:13:23.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:13:23 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:13:23.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:13:23 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:13:23.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:13:23 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:13:23.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:13:23 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:13:23.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:13:23 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:13:23.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:13:23 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:13:23.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:13:23 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:13:23.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:13:23 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:13:23.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:13:23 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:13:23.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:13:23 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:13:23.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:13:23 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:13:23.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:13:23 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:13:23.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:13:23 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:13:23.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:13:23 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:13:23.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:13:23 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:13:23.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:13:23 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:13:23.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:13:23 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:13:23.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:13:23 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:13:24.197 INFO:tasks.workunit.client.0.smithi083.stdout: 10 58968 6.61 MB/sec execute 355 sec latency 97.977 ms 2024-04-24T09:13:24.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:13:24 smithi138 ceph-mon[31441]: pgmap v688: 129 pgs: 129 active+clean; 921 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.0 MiB/s wr, 360 op/s 2024-04-24T09:13:24.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:13:24 smithi189 ceph-mon[31791]: pgmap v688: 129 pgs: 129 active+clean; 921 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.0 MiB/s wr, 360 op/s 2024-04-24T09:13:24.552 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:13:24 smithi083 ceph-mon[27907]: pgmap v688: 129 pgs: 129 active+clean; 921 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.0 MiB/s wr, 360 op/s 2024-04-24T09:13:25.198 INFO:tasks.workunit.client.0.smithi083.stdout: 10 59126 6.62 MB/sec execute 356 sec latency 78.660 ms 2024-04-24T09:13:26.198 INFO:tasks.workunit.client.0.smithi083.stdout: 10 59229 6.62 MB/sec execute 357 sec latency 254.300 ms 2024-04-24T09:13:26.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:13:26 smithi138 ceph-mon[31441]: pgmap v689: 129 pgs: 129 active+clean; 921 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.5 MiB/s wr, 237 op/s 2024-04-24T09:13:26.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:13:26 smithi189 ceph-mon[31791]: pgmap v689: 129 pgs: 129 active+clean; 921 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.5 MiB/s wr, 237 op/s 2024-04-24T09:13:26.552 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:13:26 smithi083 ceph-mon[27907]: pgmap v689: 129 pgs: 129 active+clean; 921 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.5 MiB/s wr, 237 op/s 2024-04-24T09:13:27.198 INFO:tasks.workunit.client.0.smithi083.stdout: 10 59376 6.62 MB/sec execute 358 sec latency 78.519 ms 2024-04-24T09:13:28.199 INFO:tasks.workunit.client.0.smithi083.stdout: 10 59494 6.62 MB/sec execute 359 sec latency 60.421 ms 2024-04-24T09:13:28.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:13:28 smithi138 ceph-mon[31441]: pgmap v690: 129 pgs: 129 active+clean; 921 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.5 MiB/s wr, 237 op/s 2024-04-24T09:13:28.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:13:28 smithi189 ceph-mon[31791]: pgmap v690: 129 pgs: 129 active+clean; 921 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.5 MiB/s wr, 237 op/s 2024-04-24T09:13:28.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:13:28 smithi083 ceph-mon[27907]: pgmap v690: 129 pgs: 129 active+clean; 921 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.5 MiB/s wr, 237 op/s 2024-04-24T09:13:29.199 INFO:tasks.workunit.client.0.smithi083.stdout: 10 59596 6.61 MB/sec execute 360 sec latency 82.202 ms 2024-04-24T09:13:30.199 INFO:tasks.workunit.client.0.smithi083.stdout: 10 59705 6.60 MB/sec execute 361 sec latency 91.101 ms 2024-04-24T09:13:30.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:13:30 smithi138 ceph-mon[31441]: pgmap v691: 129 pgs: 129 active+clean; 931 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.6 MiB/s wr, 331 op/s 2024-04-24T09:13:30.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:13:30 smithi189 ceph-mon[31791]: pgmap v691: 129 pgs: 129 active+clean; 931 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.6 MiB/s wr, 331 op/s 2024-04-24T09:13:30.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:13:30 smithi083 ceph-mon[27907]: pgmap v691: 129 pgs: 129 active+clean; 931 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.6 MiB/s wr, 331 op/s 2024-04-24T09:13:31.199 INFO:tasks.workunit.client.0.smithi083.stdout: 10 59871 6.62 MB/sec execute 362 sec latency 80.827 ms 2024-04-24T09:13:32.199 INFO:tasks.workunit.client.0.smithi083.stdout: 10 60023 6.62 MB/sec execute 363 sec latency 209.200 ms 2024-04-24T09:13:32.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:13:32 smithi138 ceph-mon[31441]: pgmap v692: 129 pgs: 129 active+clean; 931 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.8 MiB/s wr, 247 op/s 2024-04-24T09:13:32.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:13:32 smithi189 ceph-mon[31791]: pgmap v692: 129 pgs: 129 active+clean; 931 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.8 MiB/s wr, 247 op/s 2024-04-24T09:13:32.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:13:32 smithi083 ceph-mon[27907]: pgmap v692: 129 pgs: 129 active+clean; 931 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.8 MiB/s wr, 247 op/s 2024-04-24T09:13:33.199 INFO:tasks.workunit.client.0.smithi083.stdout: 10 60166 6.63 MB/sec execute 364 sec latency 97.177 ms 2024-04-24T09:13:33.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:13:33 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:13:33.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:13:33 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:13:33.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:13:33 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:13:33.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:13:33 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:13:33.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:13:33 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:13:33.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:13:33 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:13:33.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:13:33 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:13:33.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:13:33 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:13:33.459 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:13:33 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:13:33.459 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:13:33 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:13:33.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:13:33 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:13:33.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:13:33 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:13:33.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:13:33 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:13:33.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:13:33 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:13:33.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:13:33 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:13:34.199 INFO:tasks.workunit.client.0.smithi083.stdout: 10 60266 6.62 MB/sec execute 365 sec latency 77.084 ms 2024-04-24T09:13:34.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:13:34 smithi138 ceph-mon[31441]: pgmap v693: 129 pgs: 129 active+clean; 939 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.7 MiB/s wr, 360 op/s 2024-04-24T09:13:34.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:13:34 smithi189 ceph-mon[31791]: pgmap v693: 129 pgs: 129 active+clean; 939 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.7 MiB/s wr, 360 op/s 2024-04-24T09:13:34.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:13:34 smithi083 ceph-mon[27907]: pgmap v693: 129 pgs: 129 active+clean; 939 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.7 MiB/s wr, 360 op/s 2024-04-24T09:13:35.199 INFO:tasks.workunit.client.0.smithi083.stdout: 10 60384 6.61 MB/sec execute 366 sec latency 60.495 ms 2024-04-24T09:13:35.820 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.820+0000 7fabb7200640 1 -- 172.21.15.83:0/2790656885 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fabb8107140 msgr2=0x7fabb810ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:35.820 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.820+0000 7fabb7200640 1 --2- 172.21.15.83:0/2790656885 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fabb8107140 0x7fabb810ed10 secure :-1 s=READY pgs=2629 cs=0 l=1 rev1=1 crypto rx=0x7fabb0009ca0 tx=0x7fabb002ecd0 comp rx=0 tx=0).stop 2024-04-24T09:13:35.820 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.820+0000 7fabb7200640 1 -- 172.21.15.83:0/2790656885 shutdown_connections 2024-04-24T09:13:35.820 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.820+0000 7fabb7200640 1 --2- 172.21.15.83:0/2790656885 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fabb8107140 0x7fabb810ed10 unknown :-1 s=CLOSED pgs=2629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:35.820 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.820+0000 7fabb7200640 1 --2- 172.21.15.83:0/2790656885 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fabb8102a20 0x7fabb8106a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:35.820 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.820+0000 7fabb7200640 1 --2- 172.21.15.83:0/2790656885 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fabb8102070 0x7fabb8102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:35.821 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.820+0000 7fabb7200640 1 -- 172.21.15.83:0/2790656885 >> 172.21.15.83:0/2790656885 conn(0x7fabb80fd950 msgr2=0x7fabb80ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:35.821 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.821+0000 7fabb7200640 1 -- 172.21.15.83:0/2790656885 shutdown_connections 2024-04-24T09:13:35.821 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.821+0000 7fabb7200640 1 -- 172.21.15.83:0/2790656885 wait complete. 2024-04-24T09:13:35.821 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.822+0000 7fabb7200640 1 Processor -- start 2024-04-24T09:13:35.821 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.822+0000 7fabb7200640 1 -- start start 2024-04-24T09:13:35.822 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.822+0000 7fabb7200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fabb8102070 0x7fabb81963e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:35.823 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.823+0000 7fabb7200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fabb8107140 0x7fabb8196920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:35.823 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.823+0000 7fabb7200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fabb819aca0 0x7fabb819b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:35.823 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.823+0000 7fabb7200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fabb810fe80 con 0x7fabb8102070 2024-04-24T09:13:35.823 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.823+0000 7fabb7200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fabb810fd00 con 0x7fabb819aca0 2024-04-24T09:13:35.823 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.823+0000 7fabb7200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fabb8110000 con 0x7fabb8107140 2024-04-24T09:13:35.823 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.823+0000 7fabb5e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fabb8102070 0x7fabb81963e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:35.823 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.823+0000 7fabb5e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fabb8102070 0x7fabb81963e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36248/0 (socket says 172.21.15.83:36248) 2024-04-24T09:13:35.823 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.823+0000 7fabb5e00640 1 -- 172.21.15.83:0/4273699044 learned_addr learned my addr 172.21.15.83:0/4273699044 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:13:35.824 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.823+0000 7fabb6800640 1 --2- 172.21.15.83:0/4273699044 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fabb819aca0 0x7fabb819b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:35.824 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.823+0000 7fabb5400640 1 --2- 172.21.15.83:0/4273699044 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fabb8107140 0x7fabb8196920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:35.824 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.824+0000 7fabb5e00640 1 -- 172.21.15.83:0/4273699044 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fabb8107140 msgr2=0x7fabb8196920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:35.824 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.824+0000 7fabb5e00640 1 --2- 172.21.15.83:0/4273699044 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fabb8107140 0x7fabb8196920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:35.824 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.824+0000 7fabb5e00640 1 -- 172.21.15.83:0/4273699044 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fabb819aca0 msgr2=0x7fabb819b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:35.824 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.824+0000 7fabb5e00640 1 --2- 172.21.15.83:0/4273699044 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fabb819aca0 0x7fabb819b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:35.824 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.824+0000 7fabb5e00640 1 -- 172.21.15.83:0/4273699044 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabb819b790 con 0x7fabb8102070 2024-04-24T09:13:35.825 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.824+0000 7fabb5e00640 1 --2- 172.21.15.83:0/4273699044 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fabb8102070 0x7fabb81963e0 secure :-1 s=READY pgs=2630 cs=0 l=1 rev1=1 crypto rx=0x7faba800e9f0 tx=0x7faba800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:35.825 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.824+0000 7fabaea00640 1 -- 172.21.15.83:0/4273699044 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba800cd80 con 0x7fabb8102070 2024-04-24T09:13:35.825 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.824+0000 7fabb7200640 1 -- 172.21.15.83:0/4273699044 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabb819ba80 con 0x7fabb8102070 2024-04-24T09:13:35.825 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.824+0000 7fabaea00640 1 -- 172.21.15.83:0/4273699044 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faba80054d0 con 0x7fabb8102070 2024-04-24T09:13:35.825 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.825+0000 7fabaea00640 1 -- 172.21.15.83:0/4273699044 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba8010490 con 0x7fabb8102070 2024-04-24T09:13:35.825 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.825+0000 7fabb7200640 1 -- 172.21.15.83:0/4273699044 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabb81a3210 con 0x7fabb8102070 2024-04-24T09:13:35.828 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.829+0000 7fabaea00640 1 -- 172.21.15.83:0/4273699044 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7faba8004680 con 0x7fabb8102070 2024-04-24T09:13:35.829 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.829+0000 7fabbff83740 1 Processor -- start 2024-04-24T09:13:35.829 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.830+0000 7fabbff83740 1 -- start start 2024-04-24T09:13:35.829 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.830+0000 7fabbff83740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f6b84183c0 0x55f6b84187a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:35.829 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.830+0000 7fabbff83740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6b8418ce0 0x55f6b8423060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:35.829 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.830+0000 7fabbff83740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6b84235a0 0x55f6b8425990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:35.829 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.830+0000 7fabbff83740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55f6b837e900 con 0x55f6b84235a0 2024-04-24T09:13:35.829 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.830+0000 7fabbff83740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f6b8324f20 con 0x55f6b8418ce0 2024-04-24T09:13:35.830 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.830+0000 7fabbff83740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55f6b837a8c0 con 0x55f6b84183c0 2024-04-24T09:13:35.831 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.831+0000 7fabaea00640 1 --2- 172.21.15.83:0/4273699044 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fab9c0a3bf0 0x7fab9c0a60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:35.831 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.831+0000 7fabb5e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6b8418ce0 0x55f6b8423060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:35.831 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.831+0000 7fabb5e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6b8418ce0 0x55f6b8423060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:47582/0 (socket says 172.21.15.83:47582) 2024-04-24T09:13:35.831 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.831+0000 7fabb5e00640 1 -- 172.21.15.83:0/1401784773 learned_addr learned my addr 172.21.15.83:0/1401784773 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:13:35.831 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.831+0000 7fabb6800640 1 --2- 172.21.15.83:0/1401784773 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f6b84183c0 0x55f6b84187a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:35.831 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.831+0000 7fabaea00640 1 -- 172.21.15.83:0/4273699044 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 9583+0+0 (secure 0 0 0) 0x7faba80d0fe0 con 0x7fabb8102070 2024-04-24T09:13:35.831 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.831+0000 7fabb5400640 1 --2- 172.21.15.83:0/1401784773 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6b84235a0 0x55f6b8425990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:35.831 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.831+0000 7fabb5e00640 1 -- 172.21.15.83:0/1401784773 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f6b84183c0 msgr2=0x55f6b84187a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:35.831 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.831+0000 7fabb5e00640 1 --2- 172.21.15.83:0/1401784773 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f6b84183c0 0x55f6b84187a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:35.831 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.831+0000 7fabb5e00640 1 -- 172.21.15.83:0/1401784773 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6b84235a0 msgr2=0x55f6b8425990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:35.831 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.831+0000 7fabb5e00640 1 --2- 172.21.15.83:0/1401784773 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6b84235a0 0x55f6b8425990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:35.832 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.831+0000 7fabb5e00640 1 -- 172.21.15.83:0/1401784773 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6b8349e10 con 0x55f6b8418ce0 2024-04-24T09:13:35.832 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.831+0000 7fabb6800640 1 --2- 172.21.15.83:0/4273699044 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fab9c0a3bf0 0x7fab9c0a60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:35.832 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.831+0000 7fabb5400640 1 --2- 172.21.15.83:0/1401784773 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6b84235a0 0x55f6b8425990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:13:35.832 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.832+0000 7fabb5e00640 1 --2- 172.21.15.83:0/1401784773 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6b8418ce0 0x55f6b8423060 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7faba80ce210 tx=0x7faba8087650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0fd5fbd113eca7e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:35.833 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.832+0000 7faba3400640 1 -- 172.21.15.83:0/1401784773 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba8005680 con 0x55f6b8418ce0 2024-04-24T09:13:35.833 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.832+0000 7faba3400640 1 -- 172.21.15.83:0/1401784773 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faba809de50 con 0x55f6b8418ce0 2024-04-24T09:13:35.833 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.832+0000 7faba3400640 1 -- 172.21.15.83:0/1401784773 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba8099190 con 0x55f6b8418ce0 2024-04-24T09:13:35.833 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.833+0000 7fabbff83740 1 -- 172.21.15.83:0/1401784773 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6b8418ce0 msgr2=0x55f6b8423060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:35.833 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.833+0000 7fabb6800640 1 --2- 172.21.15.83:0/4273699044 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fab9c0a3bf0 0x7fab9c0a60b0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fabb0006010 tx=0x7fabb00095c0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:35.833 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.833+0000 7fabbff83740 1 --2- 172.21.15.83:0/1401784773 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6b8418ce0 0x55f6b8423060 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7faba80ce210 tx=0x7faba8087650 comp rx=0 tx=0).stop 2024-04-24T09:13:35.834 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.834+0000 7fabbff83740 1 -- 172.21.15.83:0/1401784773 shutdown_connections 2024-04-24T09:13:35.834 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.834+0000 7fabbff83740 1 --2- 172.21.15.83:0/1401784773 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6b84235a0 0x55f6b8425990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:35.834 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.834+0000 7fabbff83740 1 --2- 172.21.15.83:0/1401784773 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6b8418ce0 0x55f6b8423060 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:35.834 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.834+0000 7fabbff83740 1 --2- 172.21.15.83:0/1401784773 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f6b84183c0 0x55f6b84187a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:35.834 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.834+0000 7fabbff83740 1 -- 172.21.15.83:0/1401784773 >> 172.21.15.83:0/1401784773 conn(0x55f6b84148d0 msgr2=0x55f6b8414cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:35.834 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.834+0000 7fabbff83740 1 -- 172.21.15.83:0/1401784773 shutdown_connections 2024-04-24T09:13:35.834 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.834+0000 7fabbff83740 1 -- 172.21.15.83:0/1401784773 wait complete. 2024-04-24T09:13:35.834 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.834+0000 7fabbff83740 1 -- start start 2024-04-24T09:13:35.834 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.835+0000 7fabbff83740 1 Processor -- start 2024-04-24T09:13:35.836 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.835+0000 7fabbff83740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6b84183c0 0x55f6b8420660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:35.836 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.835+0000 7fabbff83740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f6b8418ce0 0x55f6b8420ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:35.836 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.835+0000 7fabbff83740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6b84235a0 0x55f6b84210e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:35.836 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.835+0000 7fabbff83740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55f6b8426130 con 0x55f6b84183c0 2024-04-24T09:13:35.836 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.835+0000 7fabbff83740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f6b83382c0 con 0x55f6b84235a0 2024-04-24T09:13:35.836 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.835+0000 7fabbff83740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55f6b84262b0 con 0x55f6b8418ce0 2024-04-24T09:13:35.836 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.835+0000 7fabbff83740 10 client.0 _ll_register_callbacks cb 0x55f6b8090a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:13:35.836 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.836+0000 7fabb6800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6b84235a0 0x55f6b84210e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:35.836 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.836+0000 7fabb6800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6b84235a0 0x55f6b84210e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:47590/0 (socket says 172.21.15.83:47590) 2024-04-24T09:13:35.836 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.836+0000 7fabb6800640 1 -- 172.21.15.83:0/1158665946 learned_addr learned my addr 172.21.15.83:0/1158665946 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:13:35.836 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.836+0000 7fabb5e00640 1 --2- 172.21.15.83:0/1158665946 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6b84183c0 0x55f6b8420660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:35.837 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.836+0000 7fabb6800640 1 -- 172.21.15.83:0/1158665946 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f6b8418ce0 msgr2=0x55f6b8420ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:35.837 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.836+0000 7fabb6800640 1 --2- 172.21.15.83:0/1158665946 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f6b8418ce0 0x55f6b8420ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:35.837 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.836+0000 7fabb6800640 1 -- 172.21.15.83:0/1158665946 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6b84183c0 msgr2=0x55f6b8420660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:35.837 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.836+0000 7fabb6800640 1 --2- 172.21.15.83:0/1158665946 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6b84183c0 0x55f6b8420660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:35.837 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.836+0000 7fabb6800640 1 -- 172.21.15.83:0/1158665946 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6b8348fd0 con 0x55f6b84235a0 2024-04-24T09:13:35.837 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.836+0000 7fabb5e00640 1 --2- 172.21.15.83:0/1158665946 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6b84183c0 0x55f6b8420660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:13:35.837 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.837+0000 7fabb6800640 1 --2- 172.21.15.83:0/1158665946 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6b84235a0 0x55f6b84210e0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fabb0039000 tx=0x7fabb0031040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a14efe890d186141 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:35.837 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.837+0000 7fab97e00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:13:35.839 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.837+0000 7fab97e00640 1 -- 172.21.15.83:0/1158665946 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb0053070 con 0x55f6b84235a0 2024-04-24T09:13:35.839 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.837+0000 7fabbff83740 1 -- 172.21.15.83:0/1158665946 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faba809de50 con 0x55f6b84235a0 2024-04-24T09:13:35.839 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.837+0000 7fab97e00640 1 -- 172.21.15.83:0/1158665946 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9583+0+0 (secure 0 0 0) 0x7fabb002fa20 con 0x55f6b84235a0 2024-04-24T09:13:35.839 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.837+0000 7fab97e00640 1 -- 172.21.15.83:0/1158665946 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7fabb005f050 con 0x55f6b84235a0 2024-04-24T09:13:35.839 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.837+0000 7fabbff83740 10 client.40160 fetch_fsmap learned FSMap version 64 2024-04-24T09:13:35.839 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.837+0000 7fabbff83740 1 -- 172.21.15.83:0/1158665946 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55f6b8091210 con 0x55f6b84235a0 2024-04-24T09:13:35.839 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.838+0000 7fab97e00640 1 -- 172.21.15.83:0/1158665946 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faba809de50 con 0x55f6b84235a0 2024-04-24T09:13:35.840 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.838+0000 7fab97e00640 1 -- 172.21.15.83:0/1158665946 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 64) ==== 8374+0+0 (secure 0 0 0) 0x7fabb0042040 con 0x55f6b84235a0 2024-04-24T09:13:35.840 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.838+0000 7fabbff83740 10 client.40160 fetch_fsmap finished waiting for FSMap version 64 2024-04-24T09:13:35.840 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.838+0000 7fabbff83740 10 client.40160 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:13:35.840 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.838+0000 7fabbff83740 20 client.40160 populate_metadata read hostname 'smithi083' 2024-04-24T09:13:35.840 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.838+0000 7fabbff83740 1 --2- 172.21.15.83:0/1158665946 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55f6b84337a0 0x55f6b8435bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:35.840 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.838+0000 7fabbff83740 4 client.40160 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:13:35.840 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.838+0000 7fabbff83740 1 -- 172.21.15.83:0/1158665946 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f6b838c320 con 0x55f6b84337a0 2024-04-24T09:13:35.840 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.839+0000 7fabb5e00640 1 --2- 172.21.15.83:0/1158665946 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55f6b84337a0 0x55f6b8435bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:35.840 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.839+0000 7fabb5e00640 1 --2- 172.21.15.83:0/1158665946 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55f6b84337a0 0x55f6b8435bd0 crc :-1 s=READY pgs=543 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4b46fb31154afe2 server_cookie=aa1a6668ae0bcf61 in_seq=0 out_seq=0 2024-04-24T09:13:35.840 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.839+0000 7fab97e00640 10 client.40160 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:13:35.842 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.840+0000 7fab97e00640 1 -- 172.21.15.83:0/1158665946 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f6b838c320 con 0x55f6b84337a0 2024-04-24T09:13:35.842 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.840+0000 7fab97e00640 10 client.40160 handle_command_reply: tid=0 2024-04-24T09:13:35.842 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.841+0000 7fabbff83740 1 client.40160 shutdown 2024-04-24T09:13:35.842 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.841+0000 7fabbff83740 1 -- 172.21.15.83:0/1158665946 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6b84235a0 msgr2=0x55f6b84210e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:35.842 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.841+0000 7fabbff83740 1 --2- 172.21.15.83:0/1158665946 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6b84235a0 0x55f6b84210e0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fabb0039000 tx=0x7fabb0031040 comp rx=0 tx=0).stop 2024-04-24T09:13:35.842 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.841+0000 7fabbff83740 1 -- 172.21.15.83:0/1158665946 shutdown_connections 2024-04-24T09:13:35.842 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.841+0000 7fabbff83740 1 --2- 172.21.15.83:0/1158665946 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55f6b84337a0 0x55f6b8435bd0 crc :-1 s=READY pgs=543 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:35.842 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.841+0000 7fabbff83740 1 --2- 172.21.15.83:0/1158665946 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6b84235a0 0x55f6b84210e0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:35.842 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.841+0000 7fabbff83740 1 --2- 172.21.15.83:0/1158665946 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f6b8418ce0 0x55f6b8420ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:35.842 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.841+0000 7fabbff83740 1 --2- 172.21.15.83:0/1158665946 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6b84183c0 0x55f6b8420660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:35.842 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.841+0000 7fab97e00640 0 client.40160 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:13:35.842 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.841+0000 7fabbff83740 1 -- 172.21.15.83:0/1158665946 >> 172.21.15.83:0/1158665946 conn(0x55f6b84148d0 msgr2=0x55f6b8413a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:35.842 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.841+0000 7fabbff83740 1 -- 172.21.15.83:0/1158665946 shutdown_connections 2024-04-24T09:13:35.843 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.841+0000 7fabbff83740 1 -- 172.21.15.83:0/1158665946 wait complete. 2024-04-24T09:13:35.843 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.841+0000 7fabbff83740 20 client.40160 trim_cache size 0 max 16384 2024-04-24T09:13:35.865 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.866+0000 7fabbff83740 1 Processor -- start 2024-04-24T09:13:35.865 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.866+0000 7fabbff83740 1 -- start start 2024-04-24T09:13:35.866 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.866+0000 7fabbff83740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6b84183c0 0x55f6b8422d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:35.866 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.866+0000 7fabbff83740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f6b8418ce0 0x55f6b841ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:35.866 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.866+0000 7fabbff83740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6b84235a0 0x55f6b8413910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:35.866 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.866+0000 7fabbff83740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fabb81acf40 con 0x55f6b84183c0 2024-04-24T09:13:35.866 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.866+0000 7fabbff83740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fabb81aeb10 con 0x55f6b84235a0 2024-04-24T09:13:35.866 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.866+0000 7fabbff83740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fabb81ad610 con 0x55f6b8418ce0 2024-04-24T09:13:35.867 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.867+0000 7fabb5e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6b84183c0 0x55f6b8422d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:35.867 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.867+0000 7fabb5e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6b84183c0 0x55f6b8422d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36282/0 (socket says 172.21.15.83:36282) 2024-04-24T09:13:35.867 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.867+0000 7fabb5e00640 1 -- 172.21.15.83:0/1858149874 learned_addr learned my addr 172.21.15.83:0/1858149874 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:13:35.867 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.867+0000 7fabb6800640 1 --2- 172.21.15.83:0/1858149874 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6b84235a0 0x55f6b8413910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:35.868 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.867+0000 7fabb5400640 1 --2- 172.21.15.83:0/1858149874 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f6b8418ce0 0x55f6b841ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:35.868 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.867+0000 7fabb5e00640 1 -- 172.21.15.83:0/1858149874 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f6b8418ce0 msgr2=0x55f6b841ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:35.868 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.867+0000 7fabb5e00640 1 --2- 172.21.15.83:0/1858149874 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f6b8418ce0 0x55f6b841ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:35.868 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.867+0000 7fabb5e00640 1 -- 172.21.15.83:0/1858149874 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6b84235a0 msgr2=0x55f6b8413910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:35.868 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.867+0000 7fabb5e00640 1 --2- 172.21.15.83:0/1858149874 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6b84235a0 0x55f6b8413910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:35.868 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.867+0000 7fabb5e00640 1 -- 172.21.15.83:0/1858149874 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6b8418b10 con 0x55f6b84183c0 2024-04-24T09:13:35.868 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.868+0000 7fabb5e00640 1 --2- 172.21.15.83:0/1858149874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6b84183c0 0x55f6b8422d90 secure :-1 s=READY pgs=2631 cs=0 l=1 rev1=1 crypto rx=0x7faba809cab0 tx=0x7faba809c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9601f846dea07ef0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:35.868 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.868+0000 7faba2a00640 1 -- 172.21.15.83:0/1858149874 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba8005680 con 0x55f6b84183c0 2024-04-24T09:13:35.868 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.868+0000 7faba2a00640 1 -- 172.21.15.83:0/1858149874 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faba80b5070 con 0x55f6b84183c0 2024-04-24T09:13:35.868 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.868+0000 7faba2a00640 1 -- 172.21.15.83:0/1858149874 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba8099190 con 0x55f6b84183c0 2024-04-24T09:13:35.869 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.869+0000 7fabbff83740 1 -- 172.21.15.83:0/1858149874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6b84183c0 msgr2=0x55f6b8422d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:35.869 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.869+0000 7fabbff83740 1 --2- 172.21.15.83:0/1858149874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6b84183c0 0x55f6b8422d90 secure :-1 s=READY pgs=2631 cs=0 l=1 rev1=1 crypto rx=0x7faba809cab0 tx=0x7faba809c760 comp rx=0 tx=0).stop 2024-04-24T09:13:35.869 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.869+0000 7fabbff83740 1 -- 172.21.15.83:0/1858149874 shutdown_connections 2024-04-24T09:13:35.869 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.869+0000 7fabbff83740 1 --2- 172.21.15.83:0/1858149874 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6b84235a0 0x55f6b8413910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:35.870 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.869+0000 7fabbff83740 1 --2- 172.21.15.83:0/1858149874 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f6b8418ce0 0x55f6b841ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:35.870 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.869+0000 7fabbff83740 1 --2- 172.21.15.83:0/1858149874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6b84183c0 0x55f6b8422d90 unknown :-1 s=CLOSED pgs=2631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:35.870 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.869+0000 7fabbff83740 1 -- 172.21.15.83:0/1858149874 >> 172.21.15.83:0/1858149874 conn(0x55f6b84148d0 msgr2=0x55f6b8413200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:35.870 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.869+0000 7fabbff83740 1 -- 172.21.15.83:0/1858149874 shutdown_connections 2024-04-24T09:13:35.870 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.869+0000 7fabbff83740 1 -- 172.21.15.83:0/1858149874 wait complete. 2024-04-24T09:13:35.870 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.870+0000 7fabbff83740 1 -- start start 2024-04-24T09:13:35.870 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.870+0000 7fabbff83740 1 Processor -- start 2024-04-24T09:13:35.872 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.871+0000 7fabbff83740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f6b84183c0 0x55f6b841da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:35.872 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.871+0000 7fabbff83740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6b8418ce0 0x55f6b841df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:35.872 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.871+0000 7fabbff83740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6b84235a0 0x55f6b84a1560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:35.873 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.871+0000 7fabbff83740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fabb81ae410 con 0x55f6b84235a0 2024-04-24T09:13:35.873 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.871+0000 7fabbff83740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fabb81add40 con 0x55f6b8418ce0 2024-04-24T09:13:35.873 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.871+0000 7fabbff83740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fabb81aaf40 con 0x55f6b84183c0 2024-04-24T09:13:35.873 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.871+0000 7fabbff83740 10 client.0 _ll_register_callbacks cb 0x7faba8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:13:35.873 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.871+0000 7fabb5400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6b8418ce0 0x55f6b841df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:35.873 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.871+0000 7fabb5400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6b8418ce0 0x55f6b841df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:47608/0 (socket says 172.21.15.83:47608) 2024-04-24T09:13:35.873 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.871+0000 7fabb5400640 1 -- 172.21.15.83:0/1496363314 learned_addr learned my addr 172.21.15.83:0/1496363314 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:13:35.873 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.871+0000 7fabb6800640 1 --2- 172.21.15.83:0/1496363314 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6b84235a0 0x55f6b84a1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:35.873 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.871+0000 7fabb5e00640 1 --2- 172.21.15.83:0/1496363314 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f6b84183c0 0x55f6b841da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:35.873 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.872+0000 7fabb5400640 1 -- 172.21.15.83:0/1496363314 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f6b84183c0 msgr2=0x55f6b841da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:35.873 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.872+0000 7fabb5400640 1 --2- 172.21.15.83:0/1496363314 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f6b84183c0 0x55f6b841da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:35.873 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.872+0000 7fabb5400640 1 -- 172.21.15.83:0/1496363314 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6b84235a0 msgr2=0x55f6b84a1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:35.874 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.872+0000 7fabb5400640 1 --2- 172.21.15.83:0/1496363314 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6b84235a0 0x55f6b84a1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:35.874 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.872+0000 7fabb5400640 1 -- 172.21.15.83:0/1496363314 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6b815d620 con 0x55f6b8418ce0 2024-04-24T09:13:35.874 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.872+0000 7fabb5400640 1 --2- 172.21.15.83:0/1496363314 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6b8418ce0 0x55f6b841df50 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x55f6b8415a60 tx=0x7faba4013b70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2c4ed3db25674f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:35.874 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.872+0000 7fab97e00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:13:35.874 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.872+0000 7fab97e00640 1 -- 172.21.15.83:0/1496363314 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba4011b40 con 0x55f6b8418ce0 2024-04-24T09:13:35.874 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.873+0000 7fab97e00640 1 -- 172.21.15.83:0/1496363314 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9583+0+0 (secure 0 0 0) 0x7faba4011ce0 con 0x55f6b8418ce0 2024-04-24T09:13:35.874 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.873+0000 7fabbff83740 1 -- 172.21.15.83:0/1496363314 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faba80b5070 con 0x55f6b8418ce0 2024-04-24T09:13:35.874 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.873+0000 7fab97e00640 1 -- 172.21.15.83:0/1496363314 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7faba40253f0 con 0x55f6b8418ce0 2024-04-24T09:13:35.874 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.873+0000 7fabbff83740 10 client.40166 fetch_fsmap learned FSMap version 64 2024-04-24T09:13:35.874 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.873+0000 7fabbff83740 1 -- 172.21.15.83:0/1496363314 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55f6b84a2ba0 con 0x55f6b8418ce0 2024-04-24T09:13:35.875 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.873+0000 7fab97e00640 1 -- 172.21.15.83:0/1496363314 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faba80b5070 con 0x55f6b8418ce0 2024-04-24T09:13:35.875 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.874+0000 7fab97e00640 1 -- 172.21.15.83:0/1496363314 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 64) ==== 8374+0+0 (secure 0 0 0) 0x7faba4024030 con 0x55f6b8418ce0 2024-04-24T09:13:35.875 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.874+0000 7fabbff83740 10 client.40166 fetch_fsmap finished waiting for FSMap version 64 2024-04-24T09:13:35.875 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.874+0000 7fabbff83740 10 client.40166 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:13:35.876 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.874+0000 7fabbff83740 20 client.40166 populate_metadata read hostname 'smithi083' 2024-04-24T09:13:35.876 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.874+0000 7fabbff83740 1 --2- 172.21.15.83:0/1496363314 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55f6b84a33d0 0x55f6b84a57f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:35.876 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.874+0000 7fabbff83740 4 client.40166 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T09:13:35.876 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.874+0000 7fabbff83740 1 -- 172.21.15.83:0/1496363314 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f6b8386190 con 0x55f6b84a33d0 2024-04-24T09:13:35.876 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.875+0000 7fabb5e00640 1 --2- 172.21.15.83:0/1496363314 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55f6b84a33d0 0x55f6b84a57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:35.876 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.875+0000 7fabb5e00640 1 --2- 172.21.15.83:0/1496363314 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55f6b84a33d0 0x55f6b84a57f0 crc :-1 s=READY pgs=544 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5ab2706187527dd server_cookie=95491c3890957681 in_seq=0 out_seq=0 2024-04-24T09:13:35.876 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:35.875+0000 7fab97e00640 10 client.40166 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:13:36.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:13:35 smithi083 ceph-mon[27907]: pgmap v694: 129 pgs: 129 active+clean; 939 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.9 MiB/s wr, 207 op/s 2024-04-24T09:13:36.079 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:36.079+0000 7fab97e00640 1 -- 172.21.15.83:0/1496363314 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f6b8386190 con 0x55f6b84a33d0 2024-04-24T09:13:36.079 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:36.079+0000 7fab97e00640 10 client.40166 handle_command_reply: tid=0 2024-04-24T09:13:36.079 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:36.079+0000 7fabbff83740 1 client.40166 shutdown 2024-04-24T09:13:36.080 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:36.080+0000 7fabbff83740 1 -- 172.21.15.83:0/1496363314 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6b8418ce0 msgr2=0x55f6b841df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:36.080 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:36.080+0000 7fabbff83740 1 --2- 172.21.15.83:0/1496363314 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6b8418ce0 0x55f6b841df50 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x55f6b8415a60 tx=0x7faba4013b70 comp rx=0 tx=0).stop 2024-04-24T09:13:36.080 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:36.081+0000 7fabbff83740 1 -- 172.21.15.83:0/1496363314 shutdown_connections 2024-04-24T09:13:36.080 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:36.081+0000 7fabbff83740 1 --2- 172.21.15.83:0/1496363314 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55f6b84a33d0 0x55f6b84a57f0 crc :-1 s=READY pgs=544 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:36.081 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:36.081+0000 7fabbff83740 1 --2- 172.21.15.83:0/1496363314 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f6b84235a0 0x55f6b84a1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:36.081 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:36.081+0000 7fabbff83740 1 --2- 172.21.15.83:0/1496363314 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6b8418ce0 0x55f6b841df50 secure :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0x55f6b8415a60 tx=0x7faba4013b70 comp rx=0 tx=0).stop 2024-04-24T09:13:36.081 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:36.081+0000 7fabbff83740 1 --2- 172.21.15.83:0/1496363314 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f6b84183c0 0x55f6b841da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:36.081 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:36.081+0000 7fabbff83740 1 -- 172.21.15.83:0/1496363314 >> 172.21.15.83:0/1496363314 conn(0x55f6b84148d0 msgr2=0x55f6b8413a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:36.081 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:36.081+0000 7fab97e00640 0 client.40166 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:13:36.081 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:36.081+0000 7fabbff83740 1 -- 172.21.15.83:0/1496363314 shutdown_connections 2024-04-24T09:13:36.081 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:36.081+0000 7fabbff83740 1 -- 172.21.15.83:0/1496363314 wait complete. 2024-04-24T09:13:36.081 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:36.081+0000 7fabbff83740 20 client.40166 trim_cache size 0 max 16384 2024-04-24T09:13:36.082 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T09:13:36.082 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T09:13:36.082 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T09:13:36.082 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T09:13:36.083 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:36.082+0000 7fabb7200640 1 -- 172.21.15.83:0/4273699044 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fab9c0a3bf0 msgr2=0x7fab9c0a60b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:36.083 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:36.082+0000 7fabb7200640 1 --2- 172.21.15.83:0/4273699044 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fab9c0a3bf0 0x7fab9c0a60b0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fabb0006010 tx=0x7fabb00095c0 comp rx=0 tx=0).stop 2024-04-24T09:13:36.083 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:36.082+0000 7fabb7200640 1 -- 172.21.15.83:0/4273699044 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fabb8102070 msgr2=0x7fabb81963e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:36.083 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:36.082+0000 7fabb7200640 1 --2- 172.21.15.83:0/4273699044 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fabb8102070 0x7fabb81963e0 secure :-1 s=READY pgs=2630 cs=0 l=1 rev1=1 crypto rx=0x7faba800e9f0 tx=0x7faba800edf0 comp rx=0 tx=0).stop 2024-04-24T09:13:36.084 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:36.083+0000 7fabb7200640 1 -- 172.21.15.83:0/4273699044 shutdown_connections 2024-04-24T09:13:36.084 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:36.083+0000 7fabb7200640 1 --2- 172.21.15.83:0/4273699044 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fab9c0a3bf0 0x7fab9c0a60b0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:36.084 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:36.083+0000 7fabb7200640 1 --2- 172.21.15.83:0/4273699044 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fabb819aca0 0x7fabb819b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:36.084 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:36.083+0000 7fabb7200640 1 --2- 172.21.15.83:0/4273699044 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fabb8107140 0x7fabb8196920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:36.084 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:36.083+0000 7fabb7200640 1 --2- 172.21.15.83:0/4273699044 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fabb8102070 0x7fabb81963e0 unknown :-1 s=CLOSED pgs=2630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:36.084 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:36.083+0000 7fabb7200640 1 -- 172.21.15.83:0/4273699044 >> 172.21.15.83:0/4273699044 conn(0x7fabb80fd950 msgr2=0x7fabb80fed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:36.084 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:36.083+0000 7fabb7200640 1 -- 172.21.15.83:0/4273699044 shutdown_connections 2024-04-24T09:13:36.084 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:36.083+0000 7fabb7200640 1 -- 172.21.15.83:0/4273699044 wait complete. 2024-04-24T09:13:36.199 INFO:tasks.workunit.client.0.smithi083.stdout: 10 60522 6.64 MB/sec execute 367 sec latency 160.866 ms 2024-04-24T09:13:36.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:13:35 smithi138 ceph-mon[31441]: pgmap v694: 129 pgs: 129 active+clean; 939 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.9 MiB/s wr, 207 op/s 2024-04-24T09:13:36.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:13:35 smithi189 ceph-mon[31791]: pgmap v694: 129 pgs: 129 active+clean; 939 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.9 MiB/s wr, 207 op/s 2024-04-24T09:13:37.200 INFO:tasks.workunit.client.0.smithi083.stdout: 10 60646 6.63 MB/sec execute 368 sec latency 68.056 ms 2024-04-24T09:13:38.200 INFO:tasks.workunit.client.0.smithi083.stdout: 10 60793 6.63 MB/sec execute 369 sec latency 89.367 ms 2024-04-24T09:13:38.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:13:38 smithi083 ceph-mon[27907]: pgmap v695: 129 pgs: 129 active+clean; 939 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.9 MiB/s wr, 207 op/s 2024-04-24T09:13:38.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:13:38 smithi138 ceph-mon[31441]: pgmap v695: 129 pgs: 129 active+clean; 939 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.9 MiB/s wr, 207 op/s 2024-04-24T09:13:38.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:13:38 smithi189 ceph-mon[31791]: pgmap v695: 129 pgs: 129 active+clean; 939 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.9 MiB/s wr, 207 op/s 2024-04-24T09:13:39.199 INFO:tasks.workunit.client.0.smithi083.stdout: 10 60923 6.63 MB/sec execute 370 sec latency 70.534 ms 2024-04-24T09:13:40.200 INFO:tasks.workunit.client.0.smithi083.stdout: 10 61019 6.61 MB/sec execute 371 sec latency 89.756 ms 2024-04-24T09:13:40.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:13:40 smithi083 ceph-mon[27907]: pgmap v696: 129 pgs: 129 active+clean; 945 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.8 MiB/s wr, 361 op/s 2024-04-24T09:13:40.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:13:40 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:13:40.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:13:40 smithi138 ceph-mon[31441]: pgmap v696: 129 pgs: 129 active+clean; 945 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.8 MiB/s wr, 361 op/s 2024-04-24T09:13:40.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:13:40 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:13:40.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:13:40 smithi189 ceph-mon[31791]: pgmap v696: 129 pgs: 129 active+clean; 945 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.8 MiB/s wr, 361 op/s 2024-04-24T09:13:40.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:13:40 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:13:41.200 INFO:tasks.workunit.client.0.smithi083.stdout: 10 61101 6.61 MB/sec execute 372 sec latency 261.523 ms 2024-04-24T09:13:42.200 INFO:tasks.workunit.client.0.smithi083.stdout: 10 61238 6.62 MB/sec execute 373 sec latency 71.522 ms 2024-04-24T09:13:42.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:13:42 smithi083 ceph-mon[27907]: pgmap v697: 129 pgs: 129 active+clean; 945 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.8 MiB/s wr, 266 op/s 2024-04-24T09:13:42.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:13:42 smithi138 ceph-mon[31441]: pgmap v697: 129 pgs: 129 active+clean; 945 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.8 MiB/s wr, 266 op/s 2024-04-24T09:13:42.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:13:42 smithi189 ceph-mon[31791]: pgmap v697: 129 pgs: 129 active+clean; 945 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.8 MiB/s wr, 266 op/s 2024-04-24T09:13:43.200 INFO:tasks.workunit.client.0.smithi083.stdout: 10 61393 6.62 MB/sec execute 374 sec latency 78.285 ms 2024-04-24T09:13:43.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:13:43 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:13:43.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:13:43 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:13:43.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:13:43 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:13:43.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:13:43 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:13:43.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:13:43 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:13:43.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:13:43 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:13:43.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:13:43 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:13:43.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:13:43 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:13:43.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:13:43 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:13:44.200 INFO:tasks.workunit.client.0.smithi083.stdout: 10 61517 6.62 MB/sec execute 375 sec latency 96.346 ms 2024-04-24T09:13:44.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:13:44 smithi138 ceph-mon[31441]: pgmap v698: 129 pgs: 129 active+clean; 955 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.7 MiB/s wr, 365 op/s 2024-04-24T09:13:44.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:13:44 smithi189 ceph-mon[31791]: pgmap v698: 129 pgs: 129 active+clean; 955 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.7 MiB/s wr, 365 op/s 2024-04-24T09:13:44.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:13:44 smithi083 ceph-mon[27907]: pgmap v698: 129 pgs: 129 active+clean; 955 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.7 MiB/s wr, 365 op/s 2024-04-24T09:13:45.200 INFO:tasks.workunit.client.0.smithi083.stdout: 10 61620 6.61 MB/sec execute 376 sec latency 83.724 ms 2024-04-24T09:13:46.200 INFO:tasks.workunit.client.0.smithi083.stdout: 10 61739 6.63 MB/sec execute 377 sec latency 274.737 ms 2024-04-24T09:13:46.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:13:46 smithi138 ceph-mon[31441]: pgmap v699: 129 pgs: 129 active+clean; 955 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.8 MiB/s wr, 252 op/s 2024-04-24T09:13:46.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:13:46 smithi189 ceph-mon[31791]: pgmap v699: 129 pgs: 129 active+clean; 955 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.8 MiB/s wr, 252 op/s 2024-04-24T09:13:46.552 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:13:46 smithi083 ceph-mon[27907]: pgmap v699: 129 pgs: 129 active+clean; 955 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.8 MiB/s wr, 252 op/s 2024-04-24T09:13:47.200 INFO:tasks.workunit.client.0.smithi083.stdout: 10 61876 6.64 MB/sec execute 378 sec latency 380.525 ms 2024-04-24T09:13:48.200 INFO:tasks.workunit.client.0.smithi083.stdout: 10 62019 6.64 MB/sec execute 379 sec latency 84.038 ms 2024-04-24T09:13:48.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:13:48 smithi138 ceph-mon[31441]: pgmap v700: 129 pgs: 129 active+clean; 955 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.8 MiB/s wr, 252 op/s 2024-04-24T09:13:48.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:13:48 smithi189 ceph-mon[31791]: pgmap v700: 129 pgs: 129 active+clean; 955 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.8 MiB/s wr, 252 op/s 2024-04-24T09:13:48.555 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:13:48 smithi083 ceph-mon[27907]: pgmap v700: 129 pgs: 129 active+clean; 955 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.8 MiB/s wr, 252 op/s 2024-04-24T09:13:49.200 INFO:tasks.workunit.client.0.smithi083.stdout: 10 62122 6.64 MB/sec execute 380 sec latency 86.996 ms 2024-04-24T09:13:49.275 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.274+0000 7f344ca00640 1 -- 172.21.15.83:0/1952110518 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3448102290 msgr2=0x7f3448102670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:49.275 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.274+0000 7f344ca00640 1 --2- 172.21.15.83:0/1952110518 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3448102290 0x7f3448102670 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3438009900 tx=0x7f343802a3f0 comp rx=0 tx=0).stop 2024-04-24T09:13:49.275 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.274+0000 7f344ca00640 1 -- 172.21.15.83:0/1952110518 shutdown_connections 2024-04-24T09:13:49.275 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.274+0000 7f344ca00640 1 --2- 172.21.15.83:0/1952110518 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3448107360 0x7f344810ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:49.275 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.274+0000 7f344ca00640 1 --2- 172.21.15.83:0/1952110518 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3448102c40 0x7f3448106c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:49.275 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.274+0000 7f344ca00640 1 --2- 172.21.15.83:0/1952110518 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3448102290 0x7f3448102670 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:49.275 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.274+0000 7f344ca00640 1 -- 172.21.15.83:0/1952110518 >> 172.21.15.83:0/1952110518 conn(0x7f34480fdab0 msgr2=0x7f34480ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:49.275 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.275+0000 7f344ca00640 1 -- 172.21.15.83:0/1952110518 shutdown_connections 2024-04-24T09:13:49.275 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.275+0000 7f344ca00640 1 -- 172.21.15.83:0/1952110518 wait complete. 2024-04-24T09:13:49.276 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.276+0000 7f344ca00640 1 Processor -- start 2024-04-24T09:13:49.276 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.276+0000 7f344ca00640 1 -- start start 2024-04-24T09:13:49.277 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.276+0000 7f344ca00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3448102290 0x7f3448074300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:49.277 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.277+0000 7f344ca00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3448107360 0x7f3448074840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:49.277 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.277+0000 7f344ca00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3448074d80 0x7f34481a6960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:49.277 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.277+0000 7f344ca00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f34481122f0 con 0x7f3448102290 2024-04-24T09:13:49.277 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.277+0000 7f344ca00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3448112170 con 0x7f3448074d80 2024-04-24T09:13:49.277 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.277+0000 7f344ca00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3448112470 con 0x7f3448107360 2024-04-24T09:13:49.278 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.277+0000 7f3446800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3448074d80 0x7f34481a6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:49.278 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.277+0000 7f3446800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3448074d80 0x7f34481a6960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:60570/0 (socket says 172.21.15.83:60570) 2024-04-24T09:13:49.278 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.277+0000 7f3446800640 1 -- 172.21.15.83:0/3392215378 learned_addr learned my addr 172.21.15.83:0/3392215378 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:13:49.278 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.277+0000 7f3445400640 1 --2- 172.21.15.83:0/3392215378 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3448107360 0x7f3448074840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:49.278 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.278+0000 7f3445e00640 1 --2- 172.21.15.83:0/3392215378 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3448102290 0x7f3448074300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:49.279 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.278+0000 7f3446800640 1 -- 172.21.15.83:0/3392215378 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3448107360 msgr2=0x7f3448074840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:49.279 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.278+0000 7f3446800640 1 --2- 172.21.15.83:0/3392215378 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3448107360 0x7f3448074840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:49.279 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.278+0000 7f3446800640 1 -- 172.21.15.83:0/3392215378 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3448102290 msgr2=0x7f3448074300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:49.279 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.278+0000 7f3446800640 1 --2- 172.21.15.83:0/3392215378 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3448102290 0x7f3448074300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:49.279 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.278+0000 7f3446800640 1 -- 172.21.15.83:0/3392215378 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34481a7060 con 0x7f3448074d80 2024-04-24T09:13:49.279 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.279+0000 7f3445e00640 1 --2- 172.21.15.83:0/3392215378 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3448102290 0x7f3448074300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:13:49.279 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.279+0000 7f3446800640 1 --2- 172.21.15.83:0/3392215378 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3448074d80 0x7f34481a6960 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f344000efd0 tx=0x7f344000c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:49.280 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.279+0000 7f343ea00640 1 -- 172.21.15.83:0/3392215378 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3440003b70 con 0x7f3448074d80 2024-04-24T09:13:49.280 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.279+0000 7f344ca00640 1 -- 172.21.15.83:0/3392215378 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34481a7350 con 0x7f3448074d80 2024-04-24T09:13:49.280 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.279+0000 7f344ca00640 1 -- 172.21.15.83:0/3392215378 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34481a78a0 con 0x7f3448074d80 2024-04-24T09:13:49.280 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.280+0000 7f343ea00640 1 -- 172.21.15.83:0/3392215378 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3440003d10 con 0x7f3448074d80 2024-04-24T09:13:49.280 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.280+0000 7f343ea00640 1 -- 172.21.15.83:0/3392215378 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3440010980 con 0x7f3448074d80 2024-04-24T09:13:49.284 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.284+0000 7f344f79e740 1 Processor -- start 2024-04-24T09:13:49.284 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.284+0000 7f343ea00640 1 -- 172.21.15.83:0/3392215378 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3440004020 con 0x7f3448074d80 2024-04-24T09:13:49.285 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.285+0000 7f344f79e740 1 -- start start 2024-04-24T09:13:49.285 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.285+0000 7f344f79e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef2e6891b0 0x55ef2e689590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:49.285 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.285+0000 7f344f79e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef2e689ad0 0x55ef2e693e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:49.286 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.285+0000 7f344f79e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef2e694390 0x55ef2e696780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:49.286 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.285+0000 7f344f79e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ef2e5ef900 con 0x55ef2e6891b0 2024-04-24T09:13:49.286 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.285+0000 7f344f79e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ef2e595f20 con 0x55ef2e689ad0 2024-04-24T09:13:49.286 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.285+0000 7f344f79e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55ef2e5eb8c0 con 0x55ef2e694390 2024-04-24T09:13:49.286 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.286+0000 7f3445400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef2e6891b0 0x55ef2e689590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:49.286 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.286+0000 7f3445400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef2e694390 0x55ef2e696780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:49.286 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.286+0000 7f3445e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef2e689ad0 0x55ef2e693e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:49.287 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.286+0000 7f3445400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef2e694390 0x55ef2e696780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:40842/0 (socket says 172.21.15.83:40842) 2024-04-24T09:13:49.287 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.286+0000 7f3445400640 1 -- 172.21.15.83:0/3558799470 learned_addr learned my addr 172.21.15.83:0/3558799470 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:13:49.287 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.286+0000 7f3445e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef2e689ad0 0x55ef2e693e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:60574/0 (socket says 172.21.15.83:60574) 2024-04-24T09:13:49.287 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.287+0000 7f3445400640 1 -- 172.21.15.83:0/3558799470 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef2e689ad0 msgr2=0x55ef2e693e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:49.287 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.287+0000 7f3445400640 1 --2- 172.21.15.83:0/3558799470 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef2e689ad0 0x55ef2e693e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:49.288 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.287+0000 7f3445400640 1 -- 172.21.15.83:0/3558799470 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef2e6891b0 msgr2=0x55ef2e689590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:49.288 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.287+0000 7f3445400640 1 --2- 172.21.15.83:0/3558799470 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef2e6891b0 0x55ef2e689590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:49.288 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.287+0000 7f3445400640 1 -- 172.21.15.83:0/3558799470 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef2e5bae10 con 0x55ef2e694390 2024-04-24T09:13:49.288 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.287+0000 7f3445400640 1 --2- 172.21.15.83:0/3558799470 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef2e694390 0x55ef2e696780 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3434012750 tx=0x7f3434005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ca0aaadfdc8bf50 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:49.288 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.287+0000 7f3433000640 1 -- 172.21.15.83:0/3558799470 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3434012110 con 0x55ef2e694390 2024-04-24T09:13:49.288 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.288+0000 7f343ea00640 1 --2- 172.21.15.83:0/3392215378 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f342c0a3e70 0x7f342c0a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:49.288 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.288+0000 7f3433000640 1 -- 172.21.15.83:0/3558799470 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f343400fc70 con 0x55ef2e694390 2024-04-24T09:13:49.288 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.288+0000 7f3433000640 1 -- 172.21.15.83:0/3558799470 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34340108f0 con 0x55ef2e694390 2024-04-24T09:13:49.289 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.288+0000 7f343ea00640 1 -- 172.21.15.83:0/3392215378 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 9583+0+0 (secure 0 0 0) 0x7f34400d0ec0 con 0x7f3448074d80 2024-04-24T09:13:49.289 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.288+0000 7f3445e00640 1 --2- 172.21.15.83:0/3392215378 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f342c0a3e70 0x7f342c0a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:49.290 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.289+0000 7f344f79e740 1 -- 172.21.15.83:0/3558799470 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef2e694390 msgr2=0x55ef2e696780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:49.290 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.289+0000 7f344f79e740 1 --2- 172.21.15.83:0/3558799470 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef2e694390 0x55ef2e696780 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3434012750 tx=0x7f3434005f90 comp rx=0 tx=0).stop 2024-04-24T09:13:49.290 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.289+0000 7f344f79e740 1 -- 172.21.15.83:0/3558799470 shutdown_connections 2024-04-24T09:13:49.290 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.289+0000 7f344f79e740 1 --2- 172.21.15.83:0/3558799470 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef2e694390 0x55ef2e696780 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:49.290 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.289+0000 7f344f79e740 1 --2- 172.21.15.83:0/3558799470 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef2e689ad0 0x55ef2e693e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:49.290 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.289+0000 7f344f79e740 1 --2- 172.21.15.83:0/3558799470 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef2e6891b0 0x55ef2e689590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:49.290 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.289+0000 7f344f79e740 1 -- 172.21.15.83:0/3558799470 >> 172.21.15.83:0/3558799470 conn(0x55ef2e6856c0 msgr2=0x55ef2e685aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:49.290 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.289+0000 7f3445e00640 1 --2- 172.21.15.83:0/3392215378 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f342c0a3e70 0x7f342c0a6330 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f3438005ab0 tx=0x7f3438005a20 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:49.290 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.289+0000 7f344f79e740 1 -- 172.21.15.83:0/3558799470 shutdown_connections 2024-04-24T09:13:49.290 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.289+0000 7f344f79e740 1 -- 172.21.15.83:0/3558799470 wait complete. 2024-04-24T09:13:49.290 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.289+0000 7f344f79e740 1 -- start start 2024-04-24T09:13:49.291 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.290+0000 7f344f79e740 1 Processor -- start 2024-04-24T09:13:49.292 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.290+0000 7f344f79e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef2e6891b0 0x55ef2e691320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:49.292 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.290+0000 7f344f79e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef2e689ad0 0x55ef2e691860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:49.292 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.290+0000 7f344f79e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef2e694390 0x55ef2e691da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:49.292 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.290+0000 7f344f79e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ef2e696fc0 con 0x55ef2e689ad0 2024-04-24T09:13:49.293 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.290+0000 7f344f79e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ef2e5a92c0 con 0x55ef2e694390 2024-04-24T09:13:49.293 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.290+0000 7f344f79e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55ef2e697140 con 0x55ef2e6891b0 2024-04-24T09:13:49.293 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.291+0000 7f344f79e740 10 client.0 _ll_register_callbacks cb 0x55ef2e301a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:13:49.293 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.291+0000 7f3445400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef2e689ad0 0x55ef2e691860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:49.293 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.291+0000 7f3446800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef2e694390 0x55ef2e691da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:49.293 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.291+0000 7f3445e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef2e6891b0 0x55ef2e691320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:49.293 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.291+0000 7f3445e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef2e6891b0 0x55ef2e691320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:40844/0 (socket says 172.21.15.83:40844) 2024-04-24T09:13:49.293 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.291+0000 7f3445e00640 1 -- 172.21.15.83:0/3914398101 learned_addr learned my addr 172.21.15.83:0/3914398101 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:13:49.293 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.291+0000 7f3445400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef2e689ad0 0x55ef2e691860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44856/0 (socket says 172.21.15.83:44856) 2024-04-24T09:13:49.293 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.291+0000 7f3446800640 1 --2- 172.21.15.83:0/3914398101 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef2e694390 0x55ef2e691da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:60576/0 (socket says 172.21.15.83:60576) 2024-04-24T09:13:49.293 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.291+0000 7f3445e00640 1 -- 172.21.15.83:0/3914398101 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef2e694390 msgr2=0x55ef2e691da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:49.293 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.291+0000 7f3445e00640 1 --2- 172.21.15.83:0/3914398101 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef2e694390 0x55ef2e691da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:49.293 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.291+0000 7f3445e00640 1 -- 172.21.15.83:0/3914398101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef2e689ad0 msgr2=0x55ef2e691860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:49.293 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.292+0000 7f3445e00640 1 --2- 172.21.15.83:0/3914398101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef2e689ad0 0x55ef2e691860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:49.294 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.292+0000 7f3445e00640 1 -- 172.21.15.83:0/3914398101 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef2e5b9fd0 con 0x55ef2e6891b0 2024-04-24T09:13:49.294 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.292+0000 7f3445400640 1 --2- 172.21.15.83:0/3914398101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef2e689ad0 0x55ef2e691860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:13:49.294 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.292+0000 7f3445e00640 1 --2- 172.21.15.83:0/3914398101 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef2e6891b0 0x55ef2e691320 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3438033000 tx=0x7f3438009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33a2ae76ac4dca04 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:49.295 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.292+0000 7f3427400640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-04-24T09:13:49.295 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.292+0000 7f3427400640 1 -- 172.21.15.83:0/3914398101 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3438035040 con 0x55ef2e6891b0 2024-04-24T09:13:49.295 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.293+0000 7f3427400640 1 -- 172.21.15.83:0/3914398101 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9583+0+0 (secure 0 0 0) 0x7f34380037f0 con 0x55ef2e6891b0 2024-04-24T09:13:49.296 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.293+0000 7f344f79e740 1 -- 172.21.15.83:0/3914398101 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f343400fc70 con 0x55ef2e6891b0 2024-04-24T09:13:49.296 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.293+0000 7f3427400640 1 -- 172.21.15.83:0/3914398101 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f34380039f0 con 0x55ef2e6891b0 2024-04-24T09:13:49.296 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.293+0000 7f344f79e740 10 client.29959 fetch_fsmap learned FSMap version 64 2024-04-24T09:13:49.296 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.293+0000 7f344f79e740 1 -- 172.21.15.83:0/3914398101 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55ef2e302210 con 0x55ef2e6891b0 2024-04-24T09:13:49.296 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.294+0000 7f3427400640 1 -- 172.21.15.83:0/3914398101 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f343400fc70 con 0x55ef2e6891b0 2024-04-24T09:13:49.296 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.294+0000 7f3427400640 1 -- 172.21.15.83:0/3914398101 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 64) ==== 8374+0+0 (secure 0 0 0) 0x7f3438044030 con 0x55ef2e6891b0 2024-04-24T09:13:49.296 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.294+0000 7f344f79e740 10 client.29959 fetch_fsmap finished waiting for FSMap version 64 2024-04-24T09:13:49.296 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.294+0000 7f344f79e740 10 client.29959 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:13:49.296 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.294+0000 7f344f79e740 20 client.29959 populate_metadata read hostname 'smithi083' 2024-04-24T09:13:49.296 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.294+0000 7f344f79e740 1 --2- 172.21.15.83:0/3914398101 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ef2e6a45b0 0x55ef2e6a6a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:49.296 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.294+0000 7f344f79e740 4 client.29959 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:13:49.296 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.294+0000 7f344f79e740 1 -- 172.21.15.83:0/3914398101 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ef2e5fd320 con 0x55ef2e6a45b0 2024-04-24T09:13:49.296 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.295+0000 7f3445400640 1 --2- 172.21.15.83:0/3914398101 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ef2e6a45b0 0x55ef2e6a6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:49.297 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.296+0000 7f3427400640 10 client.29959 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:13:49.297 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.296+0000 7f3445400640 1 --2- 172.21.15.83:0/3914398101 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ef2e6a45b0 0x55ef2e6a6a20 crc :-1 s=READY pgs=545 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21be3315151f28fb server_cookie=9a24c2755e243ce in_seq=0 out_seq=0 2024-04-24T09:13:49.297 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.297+0000 7f3427400640 1 -- 172.21.15.83:0/3914398101 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ef2e5fd320 con 0x55ef2e6a45b0 2024-04-24T09:13:49.297 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.297+0000 7f3427400640 10 client.29959 handle_command_reply: tid=0 2024-04-24T09:13:49.297 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.297+0000 7f344f79e740 1 client.29959 shutdown 2024-04-24T09:13:49.298 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.298+0000 7f344f79e740 1 -- 172.21.15.83:0/3914398101 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef2e6891b0 msgr2=0x55ef2e691320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:49.298 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.298+0000 7f344f79e740 1 --2- 172.21.15.83:0/3914398101 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef2e6891b0 0x55ef2e691320 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3438033000 tx=0x7f3438009260 comp rx=0 tx=0).stop 2024-04-24T09:13:49.299 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.298+0000 7f344f79e740 1 -- 172.21.15.83:0/3914398101 shutdown_connections 2024-04-24T09:13:49.299 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.298+0000 7f344f79e740 1 --2- 172.21.15.83:0/3914398101 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ef2e6a45b0 0x55ef2e6a6a20 crc :-1 s=READY pgs=545 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:49.299 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.298+0000 7f344f79e740 1 --2- 172.21.15.83:0/3914398101 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef2e694390 0x55ef2e691da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:49.299 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.298+0000 7f344f79e740 1 --2- 172.21.15.83:0/3914398101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef2e689ad0 0x55ef2e691860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:49.299 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.298+0000 7f344f79e740 1 --2- 172.21.15.83:0/3914398101 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef2e6891b0 0x55ef2e691320 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:49.299 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.298+0000 7f344f79e740 1 -- 172.21.15.83:0/3914398101 >> 172.21.15.83:0/3914398101 conn(0x55ef2e6856c0 msgr2=0x55ef2e68b440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:49.299 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.298+0000 7f3427400640 0 client.29959 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:13:49.299 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.298+0000 7f344f79e740 1 -- 172.21.15.83:0/3914398101 shutdown_connections 2024-04-24T09:13:49.299 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.298+0000 7f344f79e740 1 -- 172.21.15.83:0/3914398101 wait complete. 2024-04-24T09:13:49.299 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.298+0000 7f344f79e740 20 client.29959 trim_cache size 0 max 16384 2024-04-24T09:13:49.324 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.324+0000 7f344f79e740 1 Processor -- start 2024-04-24T09:13:49.326 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.325+0000 7f344f79e740 1 -- start start 2024-04-24T09:13:49.326 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.325+0000 7f344f79e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef2e6891b0 0x55ef2e69fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:49.326 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.325+0000 7f344f79e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef2e689ad0 0x55ef2e68bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:49.326 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.325+0000 7f344f79e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef2e694390 0x55ef2e6846c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:49.326 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.325+0000 7f344f79e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3428010110 con 0x55ef2e689ad0 2024-04-24T09:13:49.326 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.325+0000 7f344f79e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3428011ce0 con 0x55ef2e694390 2024-04-24T09:13:49.326 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.325+0000 7f344f79e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f34280107e0 con 0x55ef2e6891b0 2024-04-24T09:13:49.326 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.326+0000 7f3445e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef2e6891b0 0x55ef2e69fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:49.327 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.326+0000 7f3445400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef2e689ad0 0x55ef2e68bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:49.327 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.326+0000 7f3445400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef2e689ad0 0x55ef2e68bbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44872/0 (socket says 172.21.15.83:44872) 2024-04-24T09:13:49.327 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.326+0000 7f3445400640 1 -- 172.21.15.83:0/4234089107 learned_addr learned my addr 172.21.15.83:0/4234089107 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:13:49.327 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.326+0000 7f3446800640 1 --2- 172.21.15.83:0/4234089107 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef2e694390 0x55ef2e6846c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:49.327 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.326+0000 7f3445400640 1 -- 172.21.15.83:0/4234089107 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef2e6891b0 msgr2=0x55ef2e69fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:49.327 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.326+0000 7f3445400640 1 --2- 172.21.15.83:0/4234089107 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef2e6891b0 0x55ef2e69fe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:49.327 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.326+0000 7f3445400640 1 -- 172.21.15.83:0/4234089107 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef2e694390 msgr2=0x55ef2e6846c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:49.328 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.326+0000 7f3445400640 1 --2- 172.21.15.83:0/4234089107 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef2e694390 0x55ef2e6846c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:49.328 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.326+0000 7f3445400640 1 -- 172.21.15.83:0/4234089107 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef2e6a6d90 con 0x55ef2e689ad0 2024-04-24T09:13:49.328 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.327+0000 7f3445400640 1 --2- 172.21.15.83:0/4234089107 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef2e689ad0 0x55ef2e68bbc0 secure :-1 s=READY pgs=2632 cs=0 l=1 rev1=1 crypto rx=0x7f34340104a0 tx=0x7f3434014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=844ef0ca240b4ba6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:49.328 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.327+0000 7f3432600640 1 -- 172.21.15.83:0/4234089107 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f343402d070 con 0x55ef2e689ad0 2024-04-24T09:13:49.328 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.327+0000 7f3432600640 1 -- 172.21.15.83:0/4234089107 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3434016040 con 0x55ef2e689ad0 2024-04-24T09:13:49.329 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.327+0000 7f3432600640 1 -- 172.21.15.83:0/4234089107 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3434005040 con 0x55ef2e689ad0 2024-04-24T09:13:49.329 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.328+0000 7f344f79e740 1 -- 172.21.15.83:0/4234089107 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef2e689ad0 msgr2=0x55ef2e68bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:49.329 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.328+0000 7f344f79e740 1 --2- 172.21.15.83:0/4234089107 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef2e689ad0 0x55ef2e68bbc0 secure :-1 s=READY pgs=2632 cs=0 l=1 rev1=1 crypto rx=0x7f34340104a0 tx=0x7f3434014040 comp rx=0 tx=0).stop 2024-04-24T09:13:49.329 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.329+0000 7f344f79e740 1 -- 172.21.15.83:0/4234089107 shutdown_connections 2024-04-24T09:13:49.329 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.329+0000 7f344f79e740 1 --2- 172.21.15.83:0/4234089107 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef2e694390 0x55ef2e6846c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:49.330 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.329+0000 7f344f79e740 1 --2- 172.21.15.83:0/4234089107 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef2e689ad0 0x55ef2e68bbc0 unknown :-1 s=CLOSED pgs=2632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:49.330 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.329+0000 7f344f79e740 1 --2- 172.21.15.83:0/4234089107 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef2e6891b0 0x55ef2e69fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:49.330 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.329+0000 7f344f79e740 1 -- 172.21.15.83:0/4234089107 >> 172.21.15.83:0/4234089107 conn(0x55ef2e6856c0 msgr2=0x55ef2e683fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:49.330 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.329+0000 7f344f79e740 1 -- 172.21.15.83:0/4234089107 shutdown_connections 2024-04-24T09:13:49.330 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.329+0000 7f344f79e740 1 -- 172.21.15.83:0/4234089107 wait complete. 2024-04-24T09:13:49.330 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.330+0000 7f344f79e740 1 -- start start 2024-04-24T09:13:49.330 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.330+0000 7f344f79e740 1 Processor -- start 2024-04-24T09:13:49.331 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.331+0000 7f344f79e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef2e6891b0 0x55ef2e697540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:49.331 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.331+0000 7f344f79e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef2e689ad0 0x55ef2e697a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:49.331 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.331+0000 7f344f79e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef2e694390 0x55ef2e708b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:49.332 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.331+0000 7f3445e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef2e6891b0 0x55ef2e697540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:49.332 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.331+0000 7f3445e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef2e6891b0 0x55ef2e697540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:60590/0 (socket says 172.21.15.83:60590) 2024-04-24T09:13:49.332 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.331+0000 7f3445400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef2e689ad0 0x55ef2e697a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:49.332 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.331+0000 7f3445400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef2e689ad0 0x55ef2e697a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:40856/0 (socket says 172.21.15.83:40856) 2024-04-24T09:13:49.332 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.331+0000 7f3446800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef2e694390 0x55ef2e708b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:49.332 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.331+0000 7f3446800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef2e694390 0x55ef2e708b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44874/0 (socket says 172.21.15.83:44874) 2024-04-24T09:13:49.332 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.332+0000 7f3445e00640 1 -- 172.21.15.83:0/1878606314 learned_addr learned my addr 172.21.15.83:0/1878606314 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:13:49.332 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.332+0000 7f344f79e740 1 -- 172.21.15.83:0/1878606314 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f34280115e0 con 0x55ef2e694390 2024-04-24T09:13:49.333 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.332+0000 7f344f79e740 1 -- 172.21.15.83:0/1878606314 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3428010f10 con 0x55ef2e6891b0 2024-04-24T09:13:49.333 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.332+0000 7f344f79e740 1 -- 172.21.15.83:0/1878606314 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f342800e110 con 0x55ef2e689ad0 2024-04-24T09:13:49.333 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.333+0000 7f344f79e740 10 client.0 _ll_register_callbacks cb 0x7f3438009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:13:49.333 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.333+0000 7f3446800640 1 -- 172.21.15.83:0/1878606314 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef2e689ad0 msgr2=0x55ef2e697a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:49.334 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.333+0000 7f3446800640 1 --2- 172.21.15.83:0/1878606314 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef2e689ad0 0x55ef2e697a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:49.334 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.333+0000 7f3446800640 1 -- 172.21.15.83:0/1878606314 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef2e6891b0 msgr2=0x55ef2e697540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:49.338 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.333+0000 7f3446800640 1 --2- 172.21.15.83:0/1878606314 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef2e6891b0 0x55ef2e697540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:49.338 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.333+0000 7f3446800640 1 -- 172.21.15.83:0/1878606314 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef2e3ce620 con 0x55ef2e694390 2024-04-24T09:13:49.338 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.333+0000 7f3445e00640 1 --2- 172.21.15.83:0/1878606314 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef2e6891b0 0x55ef2e697540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:13:49.338 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.333+0000 7f3445400640 1 --2- 172.21.15.83:0/1878606314 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef2e689ad0 0x55ef2e697a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:13:49.338 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.334+0000 7f3446800640 1 --2- 172.21.15.83:0/1878606314 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef2e694390 0x55ef2e708b50 secure :-1 s=READY pgs=2633 cs=0 l=1 rev1=1 crypto rx=0x7f34400ce190 tx=0x7f344009c340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63019e9f72a4e4c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:49.339 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.334+0000 7f3427400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:13:49.340 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.334+0000 7f3427400640 1 -- 172.21.15.83:0/1878606314 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34400bbda0 con 0x55ef2e694390 2024-04-24T09:13:49.340 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.334+0000 7f3427400640 1 -- 172.21.15.83:0/1878606314 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9583+0+0 (secure 0 0 0) 0x7f34400870d0 con 0x55ef2e694390 2024-04-24T09:13:49.340 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.334+0000 7f344f79e740 1 -- 172.21.15.83:0/1878606314 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3434016040 con 0x55ef2e694390 2024-04-24T09:13:49.340 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.335+0000 7f3427400640 1 -- 172.21.15.83:0/1878606314 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f344009c9e0 con 0x55ef2e694390 2024-04-24T09:13:49.341 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.335+0000 7f344f79e740 10 client.23118 fetch_fsmap learned FSMap version 64 2024-04-24T09:13:49.341 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.335+0000 7f344f79e740 1 -- 172.21.15.83:0/1878606314 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55ef2e70a210 con 0x55ef2e694390 2024-04-24T09:13:49.341 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.336+0000 7f3427400640 1 -- 172.21.15.83:0/1878606314 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3434016040 con 0x55ef2e694390 2024-04-24T09:13:49.341 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.336+0000 7f3427400640 1 -- 172.21.15.83:0/1878606314 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 64) ==== 8374+0+0 (secure 0 0 0) 0x7f344001f020 con 0x55ef2e694390 2024-04-24T09:13:49.341 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.336+0000 7f344f79e740 10 client.23118 fetch_fsmap finished waiting for FSMap version 64 2024-04-24T09:13:49.341 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.336+0000 7f344f79e740 10 client.23118 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:13:49.341 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.336+0000 7f344f79e740 20 client.23118 populate_metadata read hostname 'smithi083' 2024-04-24T09:13:49.341 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.336+0000 7f344f79e740 1 --2- 172.21.15.83:0/1878606314 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ef2e70a8e0 0x55ef2e70cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:49.341 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.336+0000 7f344f79e740 4 client.23118 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T09:13:49.341 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.336+0000 7f344f79e740 1 -- 172.21.15.83:0/1878606314 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ef2e5f7190 con 0x55ef2e70a8e0 2024-04-24T09:13:49.341 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.337+0000 7f3445e00640 1 --2- 172.21.15.83:0/1878606314 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ef2e70a8e0 0x55ef2e70cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:49.341 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.338+0000 7f3445e00640 1 --2- 172.21.15.83:0/1878606314 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ef2e70a8e0 0x55ef2e70cd00 crc :-1 s=READY pgs=546 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13c56a673c840e5a server_cookie=5167b0fa978ab202 in_seq=0 out_seq=0 2024-04-24T09:13:49.341 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.338+0000 7f3427400640 10 client.23118 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:13:49.580 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.580+0000 7f3427400640 1 -- 172.21.15.83:0/1878606314 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ef2e5f7190 con 0x55ef2e70a8e0 2024-04-24T09:13:49.581 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.580+0000 7f3427400640 10 client.23118 handle_command_reply: tid=0 2024-04-24T09:13:49.581 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.580+0000 7f344f79e740 1 client.23118 shutdown 2024-04-24T09:13:49.581 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.580+0000 7f344f79e740 1 -- 172.21.15.83:0/1878606314 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef2e694390 msgr2=0x55ef2e708b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:49.581 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.580+0000 7f344f79e740 1 --2- 172.21.15.83:0/1878606314 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef2e694390 0x55ef2e708b50 secure :-1 s=READY pgs=2633 cs=0 l=1 rev1=1 crypto rx=0x7f34400ce190 tx=0x7f344009c340 comp rx=0 tx=0).stop 2024-04-24T09:13:49.582 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.580+0000 7f344f79e740 1 -- 172.21.15.83:0/1878606314 shutdown_connections 2024-04-24T09:13:49.582 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.580+0000 7f344f79e740 1 --2- 172.21.15.83:0/1878606314 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ef2e70a8e0 0x55ef2e70cd00 crc :-1 s=READY pgs=546 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:49.582 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.581+0000 7f344f79e740 1 --2- 172.21.15.83:0/1878606314 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef2e694390 0x55ef2e708b50 unknown :-1 s=CLOSED pgs=2633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:49.582 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.581+0000 7f344f79e740 1 --2- 172.21.15.83:0/1878606314 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef2e689ad0 0x55ef2e697a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:49.582 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.581+0000 7f344f79e740 1 --2- 172.21.15.83:0/1878606314 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef2e6891b0 0x55ef2e697540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:49.582 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.581+0000 7f3427400640 0 client.23118 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:13:49.583 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.581+0000 7f344f79e740 1 -- 172.21.15.83:0/1878606314 >> 172.21.15.83:0/1878606314 conn(0x55ef2e6856c0 msgr2=0x55ef2e68f7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:49.583 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.581+0000 7f344f79e740 1 -- 172.21.15.83:0/1878606314 shutdown_connections 2024-04-24T09:13:49.583 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.581+0000 7f344f79e740 1 -- 172.21.15.83:0/1878606314 wait complete. 2024-04-24T09:13:49.583 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.581+0000 7f344f79e740 20 client.23118 trim_cache size 0 max 16384 2024-04-24T09:13:49.583 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T09:13:49.583 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T09:13:49.583 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T09:13:49.583 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T09:13:49.584 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.583+0000 7f344ca00640 1 -- 172.21.15.83:0/3392215378 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f342c0a3e70 msgr2=0x7f342c0a6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:49.584 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.583+0000 7f344ca00640 1 --2- 172.21.15.83:0/3392215378 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f342c0a3e70 0x7f342c0a6330 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f3438005ab0 tx=0x7f3438005a20 comp rx=0 tx=0).stop 2024-04-24T09:13:49.584 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.583+0000 7f344ca00640 1 -- 172.21.15.83:0/3392215378 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3448074d80 msgr2=0x7f34481a6960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:49.584 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.583+0000 7f344ca00640 1 --2- 172.21.15.83:0/3392215378 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3448074d80 0x7f34481a6960 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f344000efd0 tx=0x7f344000c6a0 comp rx=0 tx=0).stop 2024-04-24T09:13:49.584 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.583+0000 7f344ca00640 1 -- 172.21.15.83:0/3392215378 shutdown_connections 2024-04-24T09:13:49.584 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.583+0000 7f344ca00640 1 --2- 172.21.15.83:0/3392215378 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f342c0a3e70 0x7f342c0a6330 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:49.584 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.583+0000 7f344ca00640 1 --2- 172.21.15.83:0/3392215378 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3448074d80 0x7f34481a6960 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:49.585 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.583+0000 7f344ca00640 1 --2- 172.21.15.83:0/3392215378 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3448107360 0x7f3448074840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:49.585 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.583+0000 7f344ca00640 1 --2- 172.21.15.83:0/3392215378 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3448102290 0x7f3448074300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:49.585 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.583+0000 7f344ca00640 1 -- 172.21.15.83:0/3392215378 >> 172.21.15.83:0/3392215378 conn(0x7f34480fdab0 msgr2=0x7f34480ffe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:49.585 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.583+0000 7f344ca00640 1 -- 172.21.15.83:0/3392215378 shutdown_connections 2024-04-24T09:13:49.585 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:13:49.583+0000 7f344ca00640 1 -- 172.21.15.83:0/3392215378 wait complete. 2024-04-24T09:13:50.200 INFO:tasks.workunit.client.0.smithi083.stdout: 10 62213 6.63 MB/sec execute 381 sec latency 141.254 ms 2024-04-24T09:13:50.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:13:50 smithi138 ceph-mon[31441]: pgmap v701: 129 pgs: 129 active+clean; 966 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.0 MiB/s wr, 349 op/s 2024-04-24T09:13:50.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:13:50 smithi189 ceph-mon[31791]: pgmap v701: 129 pgs: 129 active+clean; 966 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.0 MiB/s wr, 349 op/s 2024-04-24T09:13:50.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:13:50 smithi083 ceph-mon[27907]: pgmap v701: 129 pgs: 129 active+clean; 966 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.0 MiB/s wr, 349 op/s 2024-04-24T09:13:51.201 INFO:tasks.workunit.client.0.smithi083.stdout: 10 62337 6.63 MB/sec execute 382 sec latency 91.674 ms 2024-04-24T09:13:52.201 INFO:tasks.workunit.client.0.smithi083.stdout: 10 62440 6.62 MB/sec execute 383 sec latency 490.322 ms 2024-04-24T09:13:52.249 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T09:13:52.250 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T09:13:52.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:13:52 smithi138 ceph-mon[31441]: pgmap v702: 129 pgs: 129 active+clean; 966 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.1 MiB/s wr, 196 op/s 2024-04-24T09:13:52.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:13:52 smithi189 ceph-mon[31791]: pgmap v702: 129 pgs: 129 active+clean; 966 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.1 MiB/s wr, 196 op/s 2024-04-24T09:13:52.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:13:52 smithi083 ceph-mon[27907]: pgmap v702: 129 pgs: 129 active+clean; 966 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.1 MiB/s wr, 196 op/s 2024-04-24T09:13:52.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:52.980+0000 7f0dd7800640 1 -- 172.21.15.83:0/3090227023 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0dd0107310 msgr2=0x7f0dd010eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:52.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:52.980+0000 7f0dd7800640 1 --2- 172.21.15.83:0/3090227023 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0dd0107310 0x7f0dd010eee0 secure :-1 s=READY pgs=2634 cs=0 l=1 rev1=1 crypto rx=0x7f0dc8009be0 tx=0x7f0dc802a6e0 comp rx=0 tx=0).stop 2024-04-24T09:13:52.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:52.981+0000 7f0dd7800640 1 -- 172.21.15.83:0/3090227023 shutdown_connections 2024-04-24T09:13:52.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:52.981+0000 7f0dd7800640 1 --2- 172.21.15.83:0/3090227023 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0dd0107310 0x7f0dd010eee0 unknown :-1 s=CLOSED pgs=2634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:52.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:52.981+0000 7f0dd7800640 1 --2- 172.21.15.83:0/3090227023 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0dd0102bf0 0x7f0dd0106c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:52.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:52.981+0000 7f0dd7800640 1 --2- 172.21.15.83:0/3090227023 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0dd0102240 0x7f0dd0102620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:52.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:52.981+0000 7f0dd7800640 1 -- 172.21.15.83:0/3090227023 >> 172.21.15.83:0/3090227023 conn(0x7f0dd0079450 msgr2=0x7f0dd0079860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:52.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:52.981+0000 7f0dd7800640 1 -- 172.21.15.83:0/3090227023 shutdown_connections 2024-04-24T09:13:52.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:52.981+0000 7f0dd7800640 1 -- 172.21.15.83:0/3090227023 wait complete. 2024-04-24T09:13:52.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:52.982+0000 7f0dd7800640 1 Processor -- start 2024-04-24T09:13:52.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:52.982+0000 7f0dd7800640 1 -- start start 2024-04-24T09:13:52.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:52.983+0000 7f0dd7800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0dd0102240 0x7f0dd01965a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:52.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:52.983+0000 7f0dd7800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0dd0107310 0x7f0dd0196ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:52.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:52.983+0000 7f0dd7800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0dd019ae60 0x7f0dd019b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:52.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:52.983+0000 7f0dd7800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0dd0110080 con 0x7f0dd0102240 2024-04-24T09:13:52.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:52.983+0000 7f0dd7800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0dd010ff00 con 0x7f0dd0107310 2024-04-24T09:13:52.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:52.983+0000 7f0dd7800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0dd0110200 con 0x7f0dd019ae60 2024-04-24T09:13:52.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:52.983+0000 7f0dcfe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0dd0107310 0x7f0dd0196ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:52.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:52.983+0000 7f0dd4e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0dd0102240 0x7f0dd01965a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:52.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:52.983+0000 7f0dcfe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0dd0107310 0x7f0dd0196ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:60596/0 (socket says 172.21.15.83:60596) 2024-04-24T09:13:52.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:52.983+0000 7f0dcfe00640 1 -- 172.21.15.83:0/76522420 learned_addr learned my addr 172.21.15.83:0/76522420 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:13:52.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:52.983+0000 7f0dd5800640 1 --2- 172.21.15.83:0/76522420 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0dd019ae60 0x7f0dd019b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:52.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:52.984+0000 7f0dcfe00640 1 -- 172.21.15.83:0/76522420 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0dd019ae60 msgr2=0x7f0dd019b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:52.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:52.984+0000 7f0dcfe00640 1 --2- 172.21.15.83:0/76522420 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0dd019ae60 0x7f0dd019b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:52.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:52.984+0000 7f0dcfe00640 1 -- 172.21.15.83:0/76522420 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0dd0102240 msgr2=0x7f0dd01965a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:52.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:52.984+0000 7f0dcfe00640 1 --2- 172.21.15.83:0/76522420 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0dd0102240 0x7f0dd01965a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:52.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:52.984+0000 7f0dcfe00640 1 -- 172.21.15.83:0/76522420 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dd019b950 con 0x7f0dd0107310 2024-04-24T09:13:52.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:52.984+0000 7f0dd4e00640 1 --2- 172.21.15.83:0/76522420 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0dd0102240 0x7f0dd01965a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:13:52.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:52.984+0000 7f0dcfe00640 1 --2- 172.21.15.83:0/76522420 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0dd0107310 0x7f0dd0196ae0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc00e9c0 tx=0x7f0dbc00edc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:52.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:52.985+0000 7f0dcd600640 1 -- 172.21.15.83:0/76522420 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dbc00ccc0 con 0x7f0dd0107310 2024-04-24T09:13:52.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:52.985+0000 7f0dcd600640 1 -- 172.21.15.83:0/76522420 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0dbc0054d0 con 0x7f0dd0107310 2024-04-24T09:13:52.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:52.985+0000 7f0dd7800640 1 -- 172.21.15.83:0/76522420 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dd019bc40 con 0x7f0dd0107310 2024-04-24T09:13:52.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:52.985+0000 7f0dcd600640 1 -- 172.21.15.83:0/76522420 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dbc010430 con 0x7f0dd0107310 2024-04-24T09:13:52.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:52.985+0000 7f0dd7800640 1 -- 172.21.15.83:0/76522420 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dd01a3430 con 0x7f0dd0107310 2024-04-24T09:13:52.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:52.989+0000 7f0dda7d6740 1 Processor -- start 2024-04-24T09:13:52.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:52.989+0000 7f0dda7d6740 1 -- start start 2024-04-24T09:13:52.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:52.990+0000 7f0dda7d6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56005d661070 0x56005d661450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:52.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:52.990+0000 7f0dda7d6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56005d661990 0x56005d66bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:52.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:52.990+0000 7f0dda7d6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56005d66c060 0x56005d66e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:52.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:52.990+0000 7f0dda7d6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56005d5be1f0 con 0x56005d661070 2024-04-24T09:13:52.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:52.990+0000 7f0dda7d6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56005d564350 con 0x56005d661990 2024-04-24T09:13:52.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:52.990+0000 7f0dda7d6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56005d5ba1b0 con 0x56005d66c060 2024-04-24T09:13:52.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:52.991+0000 7f0dd4e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56005d661990 0x56005d66bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:52.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:52.991+0000 7f0dd4e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56005d661990 0x56005d66bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:60598/0 (socket says 172.21.15.83:60598) 2024-04-24T09:13:52.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:52.991+0000 7f0dd4e00640 1 -- 172.21.15.83:0/232610418 learned_addr learned my addr 172.21.15.83:0/232610418 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:13:52.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:52.991+0000 7f0dd5800640 1 --2- 172.21.15.83:0/232610418 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56005d661070 0x56005d661450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:52.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:52.991+0000 7f0dcd600640 1 -- 172.21.15.83:0/76522420 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0dbc004f30 con 0x7f0dd0107310 2024-04-24T09:13:52.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:52.992+0000 7f0dcfe00640 1 --2- 172.21.15.83:0/232610418 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56005d66c060 0x56005d66e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:52.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:52.992+0000 7f0dd4e00640 1 -- 172.21.15.83:0/232610418 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56005d66c060 msgr2=0x56005d66e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:52.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:52.992+0000 7f0dd4e00640 1 --2- 172.21.15.83:0/232610418 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56005d66c060 0x56005d66e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:52.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:52.992+0000 7f0dd4e00640 1 -- 172.21.15.83:0/232610418 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56005d661070 msgr2=0x56005d661450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:52.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:52.993+0000 7f0dd4e00640 1 --2- 172.21.15.83:0/232610418 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56005d661070 0x56005d661450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:52.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:52.993+0000 7f0dcd600640 1 --2- 172.21.15.83:0/76522420 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f0db80a3d90 0x7f0db80a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:52.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:52.993+0000 7f0dd4e00640 1 -- 172.21.15.83:0/232610418 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56005d5b8f50 con 0x56005d661990 2024-04-24T09:13:52.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:52.994+0000 7f0dcd600640 1 -- 172.21.15.83:0/76522420 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 9583+0+0 (secure 0 0 0) 0x7f0dbc0d1270 con 0x7f0dd0107310 2024-04-24T09:13:52.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:52.994+0000 7f0dd5800640 1 --2- 172.21.15.83:0/232610418 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56005d661070 0x56005d661450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:13:52.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:52.994+0000 7f0dcfe00640 1 --2- 172.21.15.83:0/76522420 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f0db80a3d90 0x7f0db80a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:52.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:52.994+0000 7f0dd4e00640 1 --2- 172.21.15.83:0/232610418 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56005d661990 0x56005d66bb20 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f0dc4009fd0 tx=0x7f0dc40119a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=396644df6ccf0548 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:52.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:52.995+0000 7f0dc2000640 1 -- 172.21.15.83:0/232610418 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc401e070 con 0x56005d661990 2024-04-24T09:13:52.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:52.995+0000 7f0dcfe00640 1 --2- 172.21.15.83:0/76522420 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f0db80a3d90 0x7f0db80a6250 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x56005d65e520 tx=0x7f0dbc00efb0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:52.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:52.995+0000 7f0dc2000640 1 -- 172.21.15.83:0/232610418 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0dc400fa70 con 0x56005d661990 2024-04-24T09:13:52.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:52.995+0000 7f0dc2000640 1 -- 172.21.15.83:0/232610418 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc4019b20 con 0x56005d661990 2024-04-24T09:13:52.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:52.996+0000 7f0dda7d6740 1 -- 172.21.15.83:0/232610418 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56005d661990 msgr2=0x56005d66bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:52.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:52.996+0000 7f0dda7d6740 1 --2- 172.21.15.83:0/232610418 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56005d661990 0x56005d66bb20 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f0dc4009fd0 tx=0x7f0dc40119a0 comp rx=0 tx=0).stop 2024-04-24T09:13:52.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:52.996+0000 7f0dda7d6740 1 -- 172.21.15.83:0/232610418 shutdown_connections 2024-04-24T09:13:52.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:52.996+0000 7f0dda7d6740 1 --2- 172.21.15.83:0/232610418 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56005d66c060 0x56005d66e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:52.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:52.996+0000 7f0dda7d6740 1 --2- 172.21.15.83:0/232610418 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56005d661990 0x56005d66bb20 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:52.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:52.996+0000 7f0dda7d6740 1 --2- 172.21.15.83:0/232610418 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56005d661070 0x56005d661450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:52.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:52.996+0000 7f0dda7d6740 1 -- 172.21.15.83:0/232610418 >> 172.21.15.83:0/232610418 conn(0x56005d65d390 msgr2=0x56005d65d770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:52.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:52.997+0000 7f0dda7d6740 1 -- 172.21.15.83:0/232610418 shutdown_connections 2024-04-24T09:13:52.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:52.997+0000 7f0dda7d6740 1 -- 172.21.15.83:0/232610418 wait complete. 2024-04-24T09:13:52.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:52.997+0000 7f0dda7d6740 1 -- start start 2024-04-24T09:13:52.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:52.998+0000 7f0dda7d6740 1 Processor -- start 2024-04-24T09:13:52.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:52.998+0000 7f0dda7d6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56005d661070 0x56005d668f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:52.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:52.998+0000 7f0dda7d6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56005d661990 0x56005d6694c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:52.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:52.998+0000 7f0dda7d6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56005d66c060 0x56005d669a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:52.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:52.998+0000 7f0dda7d6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56005d66ee50 con 0x56005d661070 2024-04-24T09:13:52.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:52.998+0000 7f0dda7d6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56005d577690 con 0x56005d661990 2024-04-24T09:13:52.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:52.998+0000 7f0dda7d6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56005d66efd0 con 0x56005d66c060 2024-04-24T09:13:52.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:52.998+0000 7f0dda7d6740 10 client.0 _ll_register_callbacks cb 0x56005d5b3650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:13:52.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:52.999+0000 7f0dd5800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56005d661070 0x56005d668f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:53.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:52.999+0000 7f0dd5800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56005d661070 0x56005d668f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44908/0 (socket says 172.21.15.83:44908) 2024-04-24T09:13:53.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:52.999+0000 7f0dd5800640 1 -- 172.21.15.83:0/130534256 learned_addr learned my addr 172.21.15.83:0/130534256 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:13:53.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:52.999+0000 7f0dd4e00640 1 --2- 172.21.15.83:0/130534256 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56005d661990 0x56005d6694c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:53.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:52.999+0000 7f0dd4e00640 1 -- 172.21.15.83:0/130534256 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56005d66c060 msgr2=0x56005d669a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:53.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.000+0000 7f0dd4e00640 1 --2- 172.21.15.83:0/130534256 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56005d66c060 0x56005d669a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:53.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.000+0000 7f0dd4e00640 1 -- 172.21.15.83:0/130534256 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56005d661070 msgr2=0x56005d668f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:53.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.000+0000 7f0dd4e00640 1 --2- 172.21.15.83:0/130534256 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56005d661070 0x56005d668f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:53.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.000+0000 7f0dd4e00640 1 -- 172.21.15.83:0/130534256 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56005d576920 con 0x56005d661990 2024-04-24T09:13:53.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.000+0000 7f0dd5800640 1 --2- 172.21.15.83:0/130534256 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56005d661070 0x56005d668f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:13:53.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.000+0000 7f0dd4e00640 1 --2- 172.21.15.83:0/130534256 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56005d661990 0x56005d6694c0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f0dc401b000 tx=0x7f0dc400f7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6616e54f3a32205 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:53.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.000+0000 7f0db6a00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:13:53.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.001+0000 7f0db6a00640 1 -- 172.21.15.83:0/130534256 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc401e040 con 0x56005d661990 2024-04-24T09:13:53.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.001+0000 7f0db6a00640 1 -- 172.21.15.83:0/130534256 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9583+0+0 (secure 0 0 0) 0x7f0dc4005040 con 0x56005d661990 2024-04-24T09:13:53.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.001+0000 7f0dda7d6740 1 -- 172.21.15.83:0/130534256 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0dc400fa70 con 0x56005d661990 2024-04-24T09:13:53.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.001+0000 7f0db6a00640 1 -- 172.21.15.83:0/130534256 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f0dc4009ce0 con 0x56005d661990 2024-04-24T09:13:53.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.001+0000 7f0dda7d6740 10 client.40193 fetch_fsmap learned FSMap version 64 2024-04-24T09:13:53.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.001+0000 7f0dda7d6740 1 -- 172.21.15.83:0/130534256 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x56005d39a7d0 con 0x56005d661990 2024-04-24T09:13:53.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.002+0000 7f0db6a00640 1 -- 172.21.15.83:0/130534256 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0dc400fa70 con 0x56005d661990 2024-04-24T09:13:53.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.002+0000 7f0db6a00640 1 -- 172.21.15.83:0/130534256 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 64) ==== 8374+0+0 (secure 0 0 0) 0x7f0dc4021030 con 0x56005d661990 2024-04-24T09:13:53.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.002+0000 7f0dda7d6740 10 client.40193 fetch_fsmap finished waiting for FSMap version 64 2024-04-24T09:13:53.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.002+0000 7f0dda7d6740 10 client.40193 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:13:53.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.002+0000 7f0dda7d6740 20 client.40193 populate_metadata read hostname 'smithi083' 2024-04-24T09:13:53.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.002+0000 7f0dda7d6740 1 --2- 172.21.15.83:0/130534256 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56005d67c000 0x56005d67e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:53.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.002+0000 7f0dda7d6740 4 client.40193 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:13:53.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.002+0000 7f0dda7d6740 1 -- 172.21.15.83:0/130534256 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56005d5cbc10 con 0x56005d67c000 2024-04-24T09:13:53.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.003+0000 7f0dd5800640 1 --2- 172.21.15.83:0/130534256 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56005d67c000 0x56005d67e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:53.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.004+0000 7f0dd5800640 1 --2- 172.21.15.83:0/130534256 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56005d67c000 0x56005d67e3f0 crc :-1 s=READY pgs=547 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5786850f23357ee server_cookie=ddd0c0ebf113fc3c in_seq=0 out_seq=0 2024-04-24T09:13:53.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.004+0000 7f0db6a00640 10 client.40193 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:13:53.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.006+0000 7f0db6a00640 1 -- 172.21.15.83:0/130534256 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56005d5cbc10 con 0x56005d67c000 2024-04-24T09:13:53.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.006+0000 7f0db6a00640 10 client.40193 handle_command_reply: tid=0 2024-04-24T09:13:53.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.006+0000 7f0dda7d6740 1 client.40193 shutdown 2024-04-24T09:13:53.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.006+0000 7f0dda7d6740 1 -- 172.21.15.83:0/130534256 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56005d661990 msgr2=0x56005d6694c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:53.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.006+0000 7f0dda7d6740 1 --2- 172.21.15.83:0/130534256 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56005d661990 0x56005d6694c0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f0dc401b000 tx=0x7f0dc400f7f0 comp rx=0 tx=0).stop 2024-04-24T09:13:53.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.006+0000 7f0dda7d6740 1 -- 172.21.15.83:0/130534256 shutdown_connections 2024-04-24T09:13:53.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.006+0000 7f0dda7d6740 1 --2- 172.21.15.83:0/130534256 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56005d67c000 0x56005d67e3f0 crc :-1 s=READY pgs=547 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:53.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.006+0000 7f0dda7d6740 1 --2- 172.21.15.83:0/130534256 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56005d66c060 0x56005d669a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:53.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.006+0000 7f0dda7d6740 1 --2- 172.21.15.83:0/130534256 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56005d661990 0x56005d6694c0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:53.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.006+0000 7f0db6a00640 0 client.40193 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:13:53.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.006+0000 7f0dda7d6740 1 --2- 172.21.15.83:0/130534256 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56005d661070 0x56005d668f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:53.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.006+0000 7f0dda7d6740 1 -- 172.21.15.83:0/130534256 >> 172.21.15.83:0/130534256 conn(0x56005d65d390 msgr2=0x56005d663090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:53.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.006+0000 7f0dda7d6740 1 -- 172.21.15.83:0/130534256 shutdown_connections 2024-04-24T09:13:53.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.007+0000 7f0dda7d6740 1 -- 172.21.15.83:0/130534256 wait complete. 2024-04-24T09:13:53.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.007+0000 7f0dda7d6740 20 client.40193 trim_cache size 0 max 16384 2024-04-24T09:13:53.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.037+0000 7f0dda7d6740 1 Processor -- start 2024-04-24T09:13:53.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.037+0000 7f0dda7d6740 1 -- start start 2024-04-24T09:13:53.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.037+0000 7f0dda7d6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56005d661070 0x56005d677d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:53.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.038+0000 7f0dda7d6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56005d661990 0x56005d6697d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:53.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.038+0000 7f0dda7d6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56005d66c060 0x56005d683c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:53.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.038+0000 7f0dda7d6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0dd01ad560 con 0x56005d661990 2024-04-24T09:13:53.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.038+0000 7f0dda7d6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0dd01af130 con 0x56005d66c060 2024-04-24T09:13:53.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.038+0000 7f0dda7d6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0dd01adc30 con 0x56005d661070 2024-04-24T09:13:53.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.038+0000 7f0dd5800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56005d66c060 0x56005d683c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:53.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.038+0000 7f0dd5800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56005d66c060 0x56005d683c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:60626/0 (socket says 172.21.15.83:60626) 2024-04-24T09:13:53.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.038+0000 7f0dd5800640 1 -- 172.21.15.83:0/3783696651 learned_addr learned my addr 172.21.15.83:0/3783696651 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:13:53.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.038+0000 7f0dd5800640 1 --2- 172.21.15.83:0/3783696651 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56005d661070 0x56005d677d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:53.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.038+0000 7f0dd4e00640 1 --2- 172.21.15.83:0/3783696651 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56005d661990 0x56005d6697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:53.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.039+0000 7f0dd5800640 1 -- 172.21.15.83:0/3783696651 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56005d661070 msgr2=0x56005d677d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:53.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.039+0000 7f0dd5800640 1 --2- 172.21.15.83:0/3783696651 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56005d661070 0x56005d677d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:53.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.039+0000 7f0dd5800640 1 -- 172.21.15.83:0/3783696651 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56005d661990 msgr2=0x56005d6697d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:53.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.039+0000 7f0dd5800640 1 --2- 172.21.15.83:0/3783696651 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56005d661990 0x56005d6697d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:53.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.039+0000 7f0dd5800640 1 -- 172.21.15.83:0/3783696651 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56005d66e7c0 con 0x56005d66c060 2024-04-24T09:13:53.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.040+0000 7f0dd5800640 1 --2- 172.21.15.83:0/3783696651 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56005d66c060 0x56005d683c50 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f0dc8006010 tx=0x7f0dc8005bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dea78d8819df37d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:53.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.040+0000 7f0dc1600640 1 -- 172.21.15.83:0/3783696651 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc8036040 con 0x56005d66c060 2024-04-24T09:13:53.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.040+0000 7f0dc1600640 1 -- 172.21.15.83:0/3783696651 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0dc8037040 con 0x56005d66c060 2024-04-24T09:13:53.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.040+0000 7f0dc1600640 1 -- 172.21.15.83:0/3783696651 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc80316a0 con 0x56005d66c060 2024-04-24T09:13:53.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.041+0000 7f0dda7d6740 1 -- 172.21.15.83:0/3783696651 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56005d66c060 msgr2=0x56005d683c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:53.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.041+0000 7f0dda7d6740 1 --2- 172.21.15.83:0/3783696651 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56005d66c060 0x56005d683c50 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f0dc8006010 tx=0x7f0dc8005bd0 comp rx=0 tx=0).stop 2024-04-24T09:13:53.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.041+0000 7f0dda7d6740 1 -- 172.21.15.83:0/3783696651 shutdown_connections 2024-04-24T09:13:53.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.041+0000 7f0dda7d6740 1 --2- 172.21.15.83:0/3783696651 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56005d66c060 0x56005d683c50 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:53.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.041+0000 7f0dda7d6740 1 --2- 172.21.15.83:0/3783696651 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56005d661990 0x56005d6697d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:53.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.041+0000 7f0dda7d6740 1 --2- 172.21.15.83:0/3783696651 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56005d661070 0x56005d677d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:53.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.041+0000 7f0dda7d6740 1 -- 172.21.15.83:0/3783696651 >> 172.21.15.83:0/3783696651 conn(0x56005d65d390 msgr2=0x56005d667ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:53.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.041+0000 7f0dda7d6740 1 -- 172.21.15.83:0/3783696651 shutdown_connections 2024-04-24T09:13:53.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.041+0000 7f0dda7d6740 1 -- 172.21.15.83:0/3783696651 wait complete. 2024-04-24T09:13:53.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.042+0000 7f0dda7d6740 1 -- start start 2024-04-24T09:13:53.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.042+0000 7f0dda7d6740 1 Processor -- start 2024-04-24T09:13:53.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.042+0000 7f0dda7d6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56005d661070 0x56005d674220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:53.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.042+0000 7f0dda7d6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56005d661990 0x56005d674760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:53.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.043+0000 7f0dda7d6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56005d66c060 0x56005d6dd690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:53.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.043+0000 7f0dda7d6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0dd01aea30 con 0x56005d661070 2024-04-24T09:13:53.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.043+0000 7f0dda7d6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0dd01ae360 con 0x56005d661990 2024-04-24T09:13:53.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.043+0000 7f0dda7d6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0dd01ab560 con 0x56005d66c060 2024-04-24T09:13:53.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.043+0000 7f0dda7d6740 10 client.0 _ll_register_callbacks cb 0x7f0dc401c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:13:53.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.043+0000 7f0dd5800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56005d661070 0x56005d674220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:53.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.043+0000 7f0dd5800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56005d661070 0x56005d674220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44926/0 (socket says 172.21.15.83:44926) 2024-04-24T09:13:53.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.043+0000 7f0dd5800640 1 -- 172.21.15.83:0/1097514406 learned_addr learned my addr 172.21.15.83:0/1097514406 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:13:53.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.043+0000 7f0dd4e00640 1 --2- 172.21.15.83:0/1097514406 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56005d661990 0x56005d674760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:53.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.043+0000 7f0dd4e00640 1 -- 172.21.15.83:0/1097514406 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56005d66c060 msgr2=0x56005d6dd690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:53.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.043+0000 7f0dd4e00640 1 --2- 172.21.15.83:0/1097514406 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56005d66c060 0x56005d6dd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:53.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.043+0000 7f0dd4e00640 1 -- 172.21.15.83:0/1097514406 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56005d661070 msgr2=0x56005d674220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:53.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.043+0000 7f0dd4e00640 1 --2- 172.21.15.83:0/1097514406 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56005d661070 0x56005d674220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:53.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.043+0000 7f0dd4e00640 1 -- 172.21.15.83:0/1097514406 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56005d2d03f0 con 0x56005d661990 2024-04-24T09:13:53.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.044+0000 7f0dd4e00640 1 --2- 172.21.15.83:0/1097514406 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56005d661990 0x56005d674760 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f0dc4010010 tx=0x7f0dc400f430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ebd7645de4a2e139 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:53.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.044+0000 7f0db6a00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:13:53.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.044+0000 7f0db6a00640 1 -- 172.21.15.83:0/1097514406 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc4036070 con 0x56005d661990 2024-04-24T09:13:53.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.044+0000 7f0db6a00640 1 -- 172.21.15.83:0/1097514406 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9583+0+0 (secure 0 0 0) 0x7f0dc400f9d0 con 0x56005d661990 2024-04-24T09:13:53.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.044+0000 7f0dda7d6740 1 -- 172.21.15.83:0/1097514406 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0dc8037040 con 0x56005d661990 2024-04-24T09:13:53.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.045+0000 7f0db6a00640 1 -- 172.21.15.83:0/1097514406 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f0dc403e050 con 0x56005d661990 2024-04-24T09:13:53.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.045+0000 7f0dda7d6740 10 client.40199 fetch_fsmap learned FSMap version 64 2024-04-24T09:13:53.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.045+0000 7f0dda7d6740 1 -- 172.21.15.83:0/1097514406 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x56005d2900f0 con 0x56005d661990 2024-04-24T09:13:53.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.045+0000 7f0db6a00640 1 -- 172.21.15.83:0/1097514406 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0dc8037040 con 0x56005d661990 2024-04-24T09:13:53.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.046+0000 7f0db6a00640 1 -- 172.21.15.83:0/1097514406 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 64) ==== 8374+0+0 (secure 0 0 0) 0x7f0dc4039030 con 0x56005d661990 2024-04-24T09:13:53.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.046+0000 7f0dda7d6740 10 client.40199 fetch_fsmap finished waiting for FSMap version 64 2024-04-24T09:13:53.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.046+0000 7f0dda7d6740 10 client.40199 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:13:53.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.046+0000 7f0dda7d6740 20 client.40199 populate_metadata read hostname 'smithi083' 2024-04-24T09:13:53.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.046+0000 7f0dda7d6740 1 --2- 172.21.15.83:0/1097514406 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56005d6f3520 0x56005d6f5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:53.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.046+0000 7f0dda7d6740 4 client.40199 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T09:13:53.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.046+0000 7f0dda7d6740 1 -- 172.21.15.83:0/1097514406 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56005d5c5a80 con 0x56005d6f3520 2024-04-24T09:13:53.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.048+0000 7f0dd5800640 1 --2- 172.21.15.83:0/1097514406 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56005d6f3520 0x56005d6f5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:53.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.050+0000 7f0dd5800640 1 --2- 172.21.15.83:0/1097514406 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56005d6f3520 0x56005d6f5940 crc :-1 s=READY pgs=548 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cdf2e81f55029b59 server_cookie=7b7754f7c0fb3e94 in_seq=0 out_seq=0 2024-04-24T09:13:53.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.050+0000 7f0db6a00640 10 client.40199 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:13:53.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.052+0000 7f0db6a00640 1 -- 172.21.15.83:0/1097514406 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56005d5c5a80 con 0x56005d6f3520 2024-04-24T09:13:53.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.052+0000 7f0db6a00640 10 client.40199 handle_command_reply: tid=0 2024-04-24T09:13:53.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.052+0000 7f0dda7d6740 1 client.40199 shutdown 2024-04-24T09:13:53.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.052+0000 7f0dda7d6740 1 -- 172.21.15.83:0/1097514406 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56005d661990 msgr2=0x56005d674760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:53.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.052+0000 7f0dda7d6740 1 --2- 172.21.15.83:0/1097514406 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56005d661990 0x56005d674760 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f0dc4010010 tx=0x7f0dc400f430 comp rx=0 tx=0).stop 2024-04-24T09:13:53.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.053+0000 7f0dda7d6740 1 -- 172.21.15.83:0/1097514406 shutdown_connections 2024-04-24T09:13:53.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.053+0000 7f0dda7d6740 1 --2- 172.21.15.83:0/1097514406 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56005d6f3520 0x56005d6f5940 crc :-1 s=READY pgs=548 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:53.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.053+0000 7f0dda7d6740 1 --2- 172.21.15.83:0/1097514406 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56005d66c060 0x56005d6dd690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:53.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.053+0000 7f0dda7d6740 1 --2- 172.21.15.83:0/1097514406 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56005d661990 0x56005d674760 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:53.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.053+0000 7f0dda7d6740 1 --2- 172.21.15.83:0/1097514406 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56005d661070 0x56005d674220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:53.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.053+0000 7f0db6a00640 0 client.40199 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:13:53.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.053+0000 7f0dda7d6740 1 -- 172.21.15.83:0/1097514406 >> 172.21.15.83:0/1097514406 conn(0x56005d65d390 msgr2=0x56005d653c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:53.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.053+0000 7f0dda7d6740 1 -- 172.21.15.83:0/1097514406 shutdown_connections 2024-04-24T09:13:53.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.053+0000 7f0dda7d6740 1 -- 172.21.15.83:0/1097514406 wait complete. 2024-04-24T09:13:53.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.053+0000 7f0dda7d6740 20 client.40199 trim_cache size 0 max 16384 2024-04-24T09:13:53.056 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T09:13:53.056 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-04-24T09:13:53.056 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-04-24T09:13:53.056 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T09:13:53.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.054+0000 7f0dd7800640 1 -- 172.21.15.83:0/76522420 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f0db80a3d90 msgr2=0x7f0db80a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:53.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.054+0000 7f0dd7800640 1 --2- 172.21.15.83:0/76522420 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f0db80a3d90 0x7f0db80a6250 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x56005d65e520 tx=0x7f0dbc00efb0 comp rx=0 tx=0).stop 2024-04-24T09:13:53.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.054+0000 7f0dd7800640 1 -- 172.21.15.83:0/76522420 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0dd0107310 msgr2=0x7f0dd0196ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:53.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.054+0000 7f0dd7800640 1 --2- 172.21.15.83:0/76522420 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0dd0107310 0x7f0dd0196ae0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc00e9c0 tx=0x7f0dbc00edc0 comp rx=0 tx=0).stop 2024-04-24T09:13:53.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.054+0000 7f0dd7800640 1 -- 172.21.15.83:0/76522420 shutdown_connections 2024-04-24T09:13:53.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.054+0000 7f0dd7800640 1 --2- 172.21.15.83:0/76522420 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f0db80a3d90 0x7f0db80a6250 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:53.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.054+0000 7f0dd7800640 1 --2- 172.21.15.83:0/76522420 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0dd019ae60 0x7f0dd019b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:53.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.054+0000 7f0dd7800640 1 --2- 172.21.15.83:0/76522420 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0dd0107310 0x7f0dd0196ae0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:53.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.054+0000 7f0dd7800640 1 --2- 172.21.15.83:0/76522420 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0dd0102240 0x7f0dd01965a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:53.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.054+0000 7f0dd7800640 1 -- 172.21.15.83:0/76522420 >> 172.21.15.83:0/76522420 conn(0x7f0dd0079450 msgr2=0x7f0dd00fff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:53.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.054+0000 7f0dd7800640 1 -- 172.21.15.83:0/76522420 shutdown_connections 2024-04-24T09:13:53.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.054+0000 7f0dd7800640 1 -- 172.21.15.83:0/76522420 wait complete. 2024-04-24T09:13:53.077 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T09:13:53.077 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T09:13:53.201 INFO:tasks.workunit.client.0.smithi083.stdout: 10 62522 6.61 MB/sec execute 384 sec latency 1138.043 ms 2024-04-24T09:13:53.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.326+0000 7f29e0e00640 1 -- 172.21.15.83:0/2597171806 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29dc069400 msgr2=0x7f29dc0697e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:53.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.326+0000 7f29e0e00640 1 --2- 172.21.15.83:0/2597171806 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29dc069400 0x7f29dc0697e0 secure :-1 s=READY pgs=2635 cs=0 l=1 rev1=1 crypto rx=0x7f29cc009900 tx=0x7f29cc02a3f0 comp rx=0 tx=0).stop 2024-04-24T09:13:53.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.327+0000 7f29e0e00640 1 -- 172.21.15.83:0/2597171806 shutdown_connections 2024-04-24T09:13:53.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.327+0000 7f29e0e00640 1 --2- 172.21.15.83:0/2597171806 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f29dc1071a0 0x7f29dc10ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:53.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.327+0000 7f29e0e00640 1 --2- 172.21.15.83:0/2597171806 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f29dc069db0 0x7f29dc106a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:53.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.327+0000 7f29e0e00640 1 --2- 172.21.15.83:0/2597171806 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29dc069400 0x7f29dc0697e0 unknown :-1 s=CLOSED pgs=2635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:53.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.327+0000 7f29e0e00640 1 -- 172.21.15.83:0/2597171806 >> 172.21.15.83:0/2597171806 conn(0x7f29dc076840 msgr2=0x7f29dc076c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:53.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.327+0000 7f29e0e00640 1 -- 172.21.15.83:0/2597171806 shutdown_connections 2024-04-24T09:13:53.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.327+0000 7f29e0e00640 1 -- 172.21.15.83:0/2597171806 wait complete. 2024-04-24T09:13:53.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.328+0000 7f29e0e00640 1 Processor -- start 2024-04-24T09:13:53.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.328+0000 7f29e0e00640 1 -- start start 2024-04-24T09:13:53.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.328+0000 7f29e0e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f29dc069400 0x7f29dc10cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:53.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.328+0000 7f29e0e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f29dc1071a0 0x7f29dc10d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:53.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.329+0000 7f29e0e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29dc1a35e0 0x7f29dc1a3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:53.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.329+0000 7f29e0e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f29dc112120 con 0x7f29dc1a35e0 2024-04-24T09:13:53.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.329+0000 7f29e0e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f29dc111fa0 con 0x7f29dc069400 2024-04-24T09:13:53.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.329+0000 7f29e0e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f29dc1122a0 con 0x7f29dc1071a0 2024-04-24T09:13:53.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.329+0000 7f29dbe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29dc1a35e0 0x7f29dc1a3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:53.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.329+0000 7f29db400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f29dc069400 0x7f29dc10cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:53.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.329+0000 7f29db400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f29dc069400 0x7f29dc10cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:60636/0 (socket says 172.21.15.83:60636) 2024-04-24T09:13:53.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.329+0000 7f29db400640 1 -- 172.21.15.83:0/1803289117 learned_addr learned my addr 172.21.15.83:0/1803289117 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:13:53.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.329+0000 7f29daa00640 1 --2- 172.21.15.83:0/1803289117 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f29dc1071a0 0x7f29dc10d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:53.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.329+0000 7f29db400640 1 -- 172.21.15.83:0/1803289117 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f29dc1071a0 msgr2=0x7f29dc10d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:53.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.329+0000 7f29db400640 1 --2- 172.21.15.83:0/1803289117 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f29dc1071a0 0x7f29dc10d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:53.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.329+0000 7f29db400640 1 -- 172.21.15.83:0/1803289117 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29dc1a35e0 msgr2=0x7f29dc1a3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:53.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.329+0000 7f29db400640 1 --2- 172.21.15.83:0/1803289117 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29dc1a35e0 0x7f29dc1a3a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:53.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.329+0000 7f29db400640 1 -- 172.21.15.83:0/1803289117 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29dc1a40d0 con 0x7f29dc069400 2024-04-24T09:13:53.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.329+0000 7f29dbe00640 1 --2- 172.21.15.83:0/1803289117 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29dc1a35e0 0x7f29dc1a3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:13:53.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.330+0000 7f29db400640 1 --2- 172.21.15.83:0/1803289117 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f29dc069400 0x7f29dc10cf60 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f29cc005ab0 tx=0x7f29cc003cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:53.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.330+0000 7f29d3e00640 1 -- 172.21.15.83:0/1803289117 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29cc03f070 con 0x7f29dc069400 2024-04-24T09:13:53.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.330+0000 7f29d3e00640 1 -- 172.21.15.83:0/1803289117 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f29cc031d60 con 0x7f29dc069400 2024-04-24T09:13:53.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.330+0000 7f29d3e00640 1 -- 172.21.15.83:0/1803289117 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29cc035030 con 0x7f29dc069400 2024-04-24T09:13:53.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.330+0000 7f29e0e00640 1 -- 172.21.15.83:0/1803289117 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29dc1a4360 con 0x7f29dc069400 2024-04-24T09:13:53.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.331+0000 7f29e0e00640 1 -- 172.21.15.83:0/1803289117 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29dc1abad0 con 0x7f29dc069400 2024-04-24T09:13:53.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.332+0000 7f29e0e00640 1 -- 172.21.15.83:0/1803289117 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29dc068d40 con 0x7f29dc069400 2024-04-24T09:13:53.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.336+0000 7f29d3e00640 1 -- 172.21.15.83:0/1803289117 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f29cc042070 con 0x7f29dc069400 2024-04-24T09:13:53.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.337+0000 7f29d3e00640 1 --2- 172.21.15.83:0/1803289117 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f29c00a3da0 0x7f29c00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:53.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.338+0000 7f29d3e00640 1 -- 172.21.15.83:0/1803289117 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 9583+0+0 (secure 0 0 0) 0x7f29cc0f18c0 con 0x7f29dc069400 2024-04-24T09:13:53.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.338+0000 7f29daa00640 1 --2- 172.21.15.83:0/1803289117 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f29c00a3da0 0x7f29c00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:53.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.339+0000 7f29daa00640 1 --2- 172.21.15.83:0/1803289117 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f29c00a3da0 0x7f29c00a6260 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f29c4005fd0 tx=0x7f29c4005e20 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:53.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.344+0000 7f29d3e00640 1 -- 172.21.15.83:0/1803289117 <== mon.2 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f29cc0bdae0 con 0x7f29dc069400 2024-04-24T09:13:53.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.781+0000 7f29e0e00640 1 -- 172.21.15.83:0/1803289117 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f29dc19ffe0 con 0x7f29dc069400 2024-04-24T09:13:53.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.782+0000 7f29d3e00640 1 -- 172.21.15.83:0/1803289117 <== mon.2 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) ==== 94+0+11948 (secure 0 0 0) 0x7f29cc0bd3e0 con 0x7f29dc069400 2024-04-24T09:13:53.783 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T09:13:53.783 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":64,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/358872257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":358872257},{"type":"v1","addr":"172.21.15.83:6835","nonce":358872257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3157070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3157070851},{"type":"v1","addr":"172.21.15.83:6837","nonce":3157070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":64,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T08:48:29.422576+0000","modified":"2024-04-24T09:12:33.175729+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34520,"mds_1":34493,"mds_2":34451,"mds_3":34427,"mds_4":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14658":{"gid":14658,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6839/4085470844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":4085470844},{"type":"v1","addr":"172.21.15.83:6839","nonce":4085470844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14676":{"gid":14676,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6841/2634754210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":2634754210},{"type":"v1","addr":"172.21.15.83:6841","nonce":2634754210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24382":{"gid":24382,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/1859921310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1859921310},{"type":"v1","addr":"172.21.15.189:6833","nonce":1859921310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/2241923197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2241923197},{"type":"v1","addr":"172.21.15.189:6835","nonce":2241923197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.189:6839/2548965336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":2548965336},{"type":"v1","addr":"172.21.15.189:6839","nonce":2548965336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26686":{"gid":26686,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6837/1678817161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":1678817161},{"type":"v1","addr":"172.21.15.189:6837","nonce":1678817161}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.138:6833/1157967658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1157967658},{"type":"v1","addr":"172.21.15.138:6833","nonce":1157967658}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.138:6835/1909219758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1909219758},{"type":"v1","addr":"172.21.15.138:6835","nonce":1909219758}]},"join_fscid":-1,"export_targets":[0,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34493":{"gid":34493,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.138:6837/900566754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":900566754},{"type":"v1","addr":"172.21.15.138:6837","nonce":900566754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34520":{"gid":34520,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.138:6839/2236932393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2236932393},{"type":"v1","addr":"172.21.15.138:6839","nonce":2236932393}]},"join_fscid":-1,"export_targets":[4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34520,"qdb_cluster":[34451,34493,34427,34520,24454]},"id":1}]} 2024-04-24T09:13:53.784 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 64 2024-04-24T09:13:53.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.785+0000 7f29e0e00640 1 -- 172.21.15.83:0/1803289117 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f29c00a3da0 msgr2=0x7f29c00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:53.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.785+0000 7f29e0e00640 1 --2- 172.21.15.83:0/1803289117 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f29c00a3da0 0x7f29c00a6260 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f29c4005fd0 tx=0x7f29c4005e20 comp rx=0 tx=0).stop 2024-04-24T09:13:53.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.785+0000 7f29e0e00640 1 -- 172.21.15.83:0/1803289117 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f29dc069400 msgr2=0x7f29dc10cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:53.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.785+0000 7f29e0e00640 1 --2- 172.21.15.83:0/1803289117 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f29dc069400 0x7f29dc10cf60 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f29cc005ab0 tx=0x7f29cc003cd0 comp rx=0 tx=0).stop 2024-04-24T09:13:53.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.786+0000 7f29e0e00640 1 -- 172.21.15.83:0/1803289117 shutdown_connections 2024-04-24T09:13:53.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.786+0000 7f29e0e00640 1 --2- 172.21.15.83:0/1803289117 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f29c00a3da0 0x7f29c00a6260 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:53.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.786+0000 7f29e0e00640 1 --2- 172.21.15.83:0/1803289117 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29dc1a35e0 0x7f29dc1a3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:53.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.786+0000 7f29e0e00640 1 --2- 172.21.15.83:0/1803289117 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f29dc1071a0 0x7f29dc10d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:53.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.786+0000 7f29e0e00640 1 --2- 172.21.15.83:0/1803289117 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f29dc069400 0x7f29dc10cf60 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:53.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.786+0000 7f29e0e00640 1 -- 172.21.15.83:0/1803289117 >> 172.21.15.83:0/1803289117 conn(0x7f29dc076840 msgr2=0x7f29dc105b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:53.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.786+0000 7f29e0e00640 1 -- 172.21.15.83:0/1803289117 shutdown_connections 2024-04-24T09:13:53.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:53.786+0000 7f29e0e00640 1 -- 172.21.15.83:0/1803289117 wait complete. 2024-04-24T09:13:53.809 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-24T09:13:54.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.056+0000 7fc3efe00640 1 -- 172.21.15.83:0/3799736945 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3e810e8e0 msgr2=0x7fc3e8110cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:54.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.056+0000 7fc3efe00640 1 --2- 172.21.15.83:0/3799736945 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3e810e8e0 0x7fc3e8110cd0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fc3e0009ca0 tx=0x7fc3e002a950 comp rx=0 tx=0).stop 2024-04-24T09:13:54.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.056+0000 7fc3efe00640 1 -- 172.21.15.83:0/3799736945 shutdown_connections 2024-04-24T09:13:54.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.056+0000 7fc3efe00640 1 --2- 172.21.15.83:0/3799736945 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3e810e8e0 0x7fc3e8110cd0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:54.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.056+0000 7fc3efe00640 1 --2- 172.21.15.83:0/3799736945 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc3e8074bf0 0x7fc3e810e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:54.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.056+0000 7fc3efe00640 1 --2- 172.21.15.83:0/3799736945 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3e80742d0 0x7fc3e80746b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:54.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.056+0000 7fc3efe00640 1 -- 172.21.15.83:0/3799736945 >> 172.21.15.83:0/3799736945 conn(0x7fc3e8079450 msgr2=0x7fc3e8079860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:54.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.057+0000 7fc3efe00640 1 -- 172.21.15.83:0/3799736945 shutdown_connections 2024-04-24T09:13:54.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.057+0000 7fc3efe00640 1 -- 172.21.15.83:0/3799736945 wait complete. 2024-04-24T09:13:54.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.058+0000 7fc3efe00640 1 Processor -- start 2024-04-24T09:13:54.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.058+0000 7fc3efe00640 1 -- start start 2024-04-24T09:13:54.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.058+0000 7fc3efe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3e80742d0 0x7fc3e81a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:54.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.058+0000 7fc3efe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3e8074bf0 0x7fc3e81a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:54.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.058+0000 7fc3efe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc3e810e8e0 0x7fc3e819a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:54.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.058+0000 7fc3efe00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e8114470 con 0x7fc3e80742d0 2024-04-24T09:13:54.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.058+0000 7fc3efe00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e81142f0 con 0x7fc3e8074bf0 2024-04-24T09:13:54.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.058+0000 7fc3efe00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e81145f0 con 0x7fc3e810e8e0 2024-04-24T09:13:54.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.059+0000 7fc3ed400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3e80742d0 0x7fc3e81a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:54.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.059+0000 7fc3ed400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3e80742d0 0x7fc3e81a06b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44966/0 (socket says 172.21.15.83:44966) 2024-04-24T09:13:54.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.059+0000 7fc3ed400640 1 -- 172.21.15.83:0/2826690388 learned_addr learned my addr 172.21.15.83:0/2826690388 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:13:54.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.059+0000 7fc3eca00640 1 --2- 172.21.15.83:0/2826690388 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3e8074bf0 0x7fc3e81a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:54.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.059+0000 7fc3ed400640 1 -- 172.21.15.83:0/2826690388 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc3e810e8e0 msgr2=0x7fc3e819a860 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-24T09:13:54.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.059+0000 7fc3ed400640 1 --2- 172.21.15.83:0/2826690388 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc3e810e8e0 0x7fc3e819a860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:54.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.059+0000 7fc3ed400640 1 -- 172.21.15.83:0/2826690388 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3e8074bf0 msgr2=0x7fc3e81a0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:54.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.060+0000 7fc3ed400640 1 --2- 172.21.15.83:0/2826690388 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3e8074bf0 0x7fc3e81a0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:54.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.060+0000 7fc3ed400640 1 -- 172.21.15.83:0/2826690388 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3e819afe0 con 0x7fc3e80742d0 2024-04-24T09:13:54.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.060+0000 7fc3eca00640 1 --2- 172.21.15.83:0/2826690388 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3e8074bf0 0x7fc3e81a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:13:54.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.060+0000 7fc3ed400640 1 --2- 172.21.15.83:0/2826690388 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3e80742d0 0x7fc3e81a06b0 secure :-1 s=READY pgs=2636 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc00d950 tx=0x7fc3dc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:54.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.061+0000 7fc3e6000640 1 -- 172.21.15.83:0/2826690388 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3dc017070 con 0x7fc3e80742d0 2024-04-24T09:13:54.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.061+0000 7fc3e6000640 1 -- 172.21.15.83:0/2826690388 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc3dc009bb0 con 0x7fc3e80742d0 2024-04-24T09:13:54.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.061+0000 7fc3e6000640 1 -- 172.21.15.83:0/2826690388 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3dc00f620 con 0x7fc3e80742d0 2024-04-24T09:13:54.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.061+0000 7fc3efe00640 1 -- 172.21.15.83:0/2826690388 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3e819b2d0 con 0x7fc3e80742d0 2024-04-24T09:13:54.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.061+0000 7fc3efe00640 1 -- 172.21.15.83:0/2826690388 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3e81a7570 con 0x7fc3e80742d0 2024-04-24T09:13:54.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.067+0000 7fc3e6000640 1 -- 172.21.15.83:0/2826690388 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc3dc010460 con 0x7fc3e80742d0 2024-04-24T09:13:54.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.069+0000 7fc3e6000640 1 --2- 172.21.15.83:0/2826690388 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fc3d00a3de0 0x7fc3d00a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:54.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.069+0000 7fc3f2c4b740 1 Processor -- start 2024-04-24T09:13:54.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.069+0000 7fc3e6000640 1 -- 172.21.15.83:0/2826690388 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 9583+0+0 (secure 0 0 0) 0x7fc3dc0d00e0 con 0x7fc3e80742d0 2024-04-24T09:13:54.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.069+0000 7fc3f2c4b740 1 -- start start 2024-04-24T09:13:54.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.069+0000 7fc3eca00640 1 --2- 172.21.15.83:0/2826690388 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fc3d00a3de0 0x7fc3d00a62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:54.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.070+0000 7fc3f2c4b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5595f479a950 0x5595f479ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:54.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.070+0000 7fc3f2c4b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5595f479b270 0x5595f47a5400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:54.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.070+0000 7fc3f2c4b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595f47a5940 0x5595f47a7d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:54.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.070+0000 7fc3f2c4b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5595f47001d0 con 0x5595f47a5940 2024-04-24T09:13:54.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.070+0000 7fc3f2c4b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5595f46a6450 con 0x5595f479a950 2024-04-24T09:13:54.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.070+0000 7fc3f2c4b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5595f46fc190 con 0x5595f479b270 2024-04-24T09:13:54.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.070+0000 7fc3ed400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5595f479a950 0x5595f479ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:54.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.070+0000 7fc3ed400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5595f479a950 0x5595f479ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:60656/0 (socket says 172.21.15.83:60656) 2024-04-24T09:13:54.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.070+0000 7fc3ed400640 1 -- 172.21.15.83:0/2725000786 learned_addr learned my addr 172.21.15.83:0/2725000786 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:13:54.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.071+0000 7fc3eca00640 1 --2- 172.21.15.83:0/2725000786 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5595f479b270 0x5595f47a5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:54.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.071+0000 7fc3ede00640 1 --2- 172.21.15.83:0/2725000786 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595f47a5940 0x5595f47a7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:54.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.071+0000 7fc3ed400640 1 -- 172.21.15.83:0/2725000786 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5595f479b270 msgr2=0x5595f47a5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:54.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.071+0000 7fc3ed400640 1 --2- 172.21.15.83:0/2725000786 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5595f479b270 0x5595f47a5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:54.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.071+0000 7fc3ed400640 1 -- 172.21.15.83:0/2725000786 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595f47a5940 msgr2=0x5595f47a7d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:54.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.071+0000 7fc3ed400640 1 --2- 172.21.15.83:0/2725000786 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595f47a5940 0x5595f47a7d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:54.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.071+0000 7fc3ed400640 1 -- 172.21.15.83:0/2725000786 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595f4763310 con 0x5595f479a950 2024-04-24T09:13:54.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.071+0000 7fc3eca00640 1 --2- 172.21.15.83:0/2826690388 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fc3d00a3de0 0x7fc3d00a62a0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fc3e8110690 tx=0x7fc3d400bf90 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:54.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.071+0000 7fc3ed400640 1 --2- 172.21.15.83:0/2725000786 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5595f479a950 0x5595f479ad30 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc005fd0 tx=0x7fc3dc09c440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70d40c6e9fa9df6b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:54.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.072+0000 7fc3da600640 1 -- 172.21.15.83:0/2725000786 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3dc020070 con 0x5595f479a950 2024-04-24T09:13:54.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.072+0000 7fc3da600640 1 -- 172.21.15.83:0/2725000786 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc3dc09ffb0 con 0x5595f479a950 2024-04-24T09:13:54.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.072+0000 7fc3da600640 1 -- 172.21.15.83:0/2725000786 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3dc0cdb00 con 0x5595f479a950 2024-04-24T09:13:54.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.073+0000 7fc3f2c4b740 1 -- 172.21.15.83:0/2725000786 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5595f479a950 msgr2=0x5595f479ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:54.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.073+0000 7fc3f2c4b740 1 --2- 172.21.15.83:0/2725000786 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5595f479a950 0x5595f479ad30 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc005fd0 tx=0x7fc3dc09c440 comp rx=0 tx=0).stop 2024-04-24T09:13:54.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.073+0000 7fc3f2c4b740 1 -- 172.21.15.83:0/2725000786 shutdown_connections 2024-04-24T09:13:54.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.073+0000 7fc3f2c4b740 1 --2- 172.21.15.83:0/2725000786 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595f47a5940 0x5595f47a7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:54.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.073+0000 7fc3f2c4b740 1 --2- 172.21.15.83:0/2725000786 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5595f479b270 0x5595f47a5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:54.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.073+0000 7fc3f2c4b740 1 --2- 172.21.15.83:0/2725000786 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5595f479a950 0x5595f479ad30 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:54.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.073+0000 7fc3f2c4b740 1 -- 172.21.15.83:0/2725000786 >> 172.21.15.83:0/2725000786 conn(0x5595f4796c70 msgr2=0x5595f4797050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:54.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.073+0000 7fc3f2c4b740 1 -- 172.21.15.83:0/2725000786 shutdown_connections 2024-04-24T09:13:54.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.073+0000 7fc3f2c4b740 1 -- 172.21.15.83:0/2725000786 wait complete. 2024-04-24T09:13:54.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.074+0000 7fc3f2c4b740 1 -- start start 2024-04-24T09:13:54.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.074+0000 7fc3f2c4b740 1 Processor -- start 2024-04-24T09:13:54.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.075+0000 7fc3f2c4b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5595f479a950 0x5595f47a53f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:54.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.075+0000 7fc3f2c4b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5595f479b270 0x5595f47a26e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:54.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.075+0000 7fc3f2c4b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595f47a5940 0x5595f47a2e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:54.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.075+0000 7fc3f2c4b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5595f47a8710 con 0x5595f47a5940 2024-04-24T09:13:54.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.075+0000 7fc3f2c4b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5595f46b9810 con 0x5595f479a950 2024-04-24T09:13:54.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.075+0000 7fc3f2c4b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5595f47a8890 con 0x5595f479b270 2024-04-24T09:13:54.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.075+0000 7fc3f2c4b740 10 client.0 _ll_register_callbacks cb 0x5595f4687140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:13:54.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.075+0000 7fc3ed400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5595f479a950 0x5595f47a53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:54.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.075+0000 7fc3ed400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5595f479a950 0x5595f47a53f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:60668/0 (socket says 172.21.15.83:60668) 2024-04-24T09:13:54.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.075+0000 7fc3ed400640 1 -- 172.21.15.83:0/3431703645 learned_addr learned my addr 172.21.15.83:0/3431703645 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:13:54.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.075+0000 7fc3ede00640 1 --2- 172.21.15.83:0/3431703645 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595f47a5940 0x5595f47a2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:54.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.076+0000 7fc3eca00640 1 --2- 172.21.15.83:0/3431703645 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5595f479b270 0x5595f47a26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:54.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.076+0000 7fc3ed400640 1 -- 172.21.15.83:0/3431703645 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5595f479b270 msgr2=0x5595f47a26e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:54.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.076+0000 7fc3ed400640 1 --2- 172.21.15.83:0/3431703645 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5595f479b270 0x5595f47a26e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:54.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.076+0000 7fc3ed400640 1 -- 172.21.15.83:0/3431703645 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595f47a5940 msgr2=0x5595f47a2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:54.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.076+0000 7fc3ed400640 1 --2- 172.21.15.83:0/3431703645 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595f47a5940 0x5595f47a2e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:54.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.076+0000 7fc3ed400640 1 -- 172.21.15.83:0/3431703645 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595f46ca3f0 con 0x5595f479a950 2024-04-24T09:13:54.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.076+0000 7fc3ed400640 1 --2- 172.21.15.83:0/3431703645 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5595f479a950 0x5595f47a53f0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc098060 tx=0x7fc3dc09bb00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=379c5fc998388916 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:54.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.076+0000 7fc3cea00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:13:54.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.076+0000 7fc3cea00640 1 -- 172.21.15.83:0/3431703645 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3dc020070 con 0x5595f479a950 2024-04-24T09:13:54.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.076+0000 7fc3cea00640 1 -- 172.21.15.83:0/3431703645 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9583+0+0 (secure 0 0 0) 0x7fc3dc09b050 con 0x5595f479a950 2024-04-24T09:13:54.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.076+0000 7fc3f2c4b740 1 -- 172.21.15.83:0/3431703645 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3dc09ffb0 con 0x5595f479a950 2024-04-24T09:13:54.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.077+0000 7fc3cea00640 1 -- 172.21.15.83:0/3431703645 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7fc3dc0ac050 con 0x5595f479a950 2024-04-24T09:13:54.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.077+0000 7fc3f2c4b740 10 client.40217 fetch_fsmap learned FSMap version 64 2024-04-24T09:13:54.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.077+0000 7fc3f2c4b740 1 -- 172.21.15.83:0/3431703645 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x5595f44123f0 con 0x5595f479a950 2024-04-24T09:13:54.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.077+0000 7fc3cea00640 1 -- 172.21.15.83:0/3431703645 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc3dc09ffb0 con 0x5595f479a950 2024-04-24T09:13:54.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.077+0000 7fc3cea00640 1 -- 172.21.15.83:0/3431703645 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 64) ==== 8374+0+0 (secure 0 0 0) 0x7fc3dc097b50 con 0x5595f479a950 2024-04-24T09:13:54.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.077+0000 7fc3f2c4b740 10 client.40217 fetch_fsmap finished waiting for FSMap version 64 2024-04-24T09:13:54.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.077+0000 7fc3f2c4b740 10 client.40217 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-24T09:13:54.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.077+0000 7fc3f2c4b740 20 client.40217 populate_metadata read hostname 'smithi083' 2024-04-24T09:13:54.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.077+0000 7fc3f2c4b740 1 --2- 172.21.15.83:0/3431703645 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x5595f47b5970 0x5595f47b7d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:54.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.077+0000 7fc3f2c4b740 4 client.40217 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:13:54.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.077+0000 7fc3f2c4b740 1 -- 172.21.15.83:0/3431703645 --> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5595f470dbf0 con 0x5595f47b5970 2024-04-24T09:13:54.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.078+0000 7fc3eca00640 1 --2- 172.21.15.83:0/3431703645 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x5595f47b5970 0x5595f47b7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:54.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.079+0000 7fc3eca00640 1 --2- 172.21.15.83:0/3431703645 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x5595f47b5970 0x5595f47b7d60 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9ac2d0566ceee6cb server_cookie=104d056aa00962a8 in_seq=0 out_seq=0 2024-04-24T09:13:54.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.079+0000 7fc3cea00640 10 client.40217 ms_handle_connect on v2:172.21.15.189:6838/2548965336 2024-04-24T09:13:54.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.081+0000 7fc3cea00640 1 -- 172.21.15.83:0/3431703645 <== mds.4 v2:172.21.15.189:6838/2548965336 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5595f470dbf0 con 0x5595f47b5970 2024-04-24T09:13:54.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.081+0000 7fc3cea00640 10 client.40217 handle_command_reply: tid=0 2024-04-24T09:13:54.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.081+0000 7fc3f2c4b740 1 client.40217 shutdown 2024-04-24T09:13:54.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.082+0000 7fc3f2c4b740 1 -- 172.21.15.83:0/3431703645 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5595f479a950 msgr2=0x5595f47a53f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:54.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.082+0000 7fc3f2c4b740 1 --2- 172.21.15.83:0/3431703645 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5595f479a950 0x5595f47a53f0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc098060 tx=0x7fc3dc09bb00 comp rx=0 tx=0).stop 2024-04-24T09:13:54.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.082+0000 7fc3f2c4b740 1 -- 172.21.15.83:0/3431703645 shutdown_connections 2024-04-24T09:13:54.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.082+0000 7fc3f2c4b740 1 --2- 172.21.15.83:0/3431703645 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x5595f47b5970 0x5595f47b7d60 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:54.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.082+0000 7fc3f2c4b740 1 --2- 172.21.15.83:0/3431703645 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595f47a5940 0x5595f47a2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:54.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.082+0000 7fc3f2c4b740 1 --2- 172.21.15.83:0/3431703645 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5595f479b270 0x5595f47a26e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:54.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.082+0000 7fc3cea00640 0 client.40217 ms_handle_reset on v2:172.21.15.189:6838/2548965336 2024-04-24T09:13:54.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.082+0000 7fc3f2c4b740 1 --2- 172.21.15.83:0/3431703645 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5595f479a950 0x5595f47a53f0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:54.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.082+0000 7fc3f2c4b740 1 -- 172.21.15.83:0/3431703645 >> 172.21.15.83:0/3431703645 conn(0x5595f4796c70 msgr2=0x5595f479ca70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:54.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.082+0000 7fc3f2c4b740 1 -- 172.21.15.83:0/3431703645 shutdown_connections 2024-04-24T09:13:54.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.082+0000 7fc3f2c4b740 1 -- 172.21.15.83:0/3431703645 wait complete. 2024-04-24T09:13:54.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.082+0000 7fc3f2c4b740 20 client.40217 trim_cache size 0 max 16384 2024-04-24T09:13:54.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.105+0000 7fc3f2c4b740 1 Processor -- start 2024-04-24T09:13:54.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.105+0000 7fc3f2c4b740 1 -- start start 2024-04-24T09:13:54.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.106+0000 7fc3f2c4b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5595f479a950 0x5595f47a1620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:54.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.106+0000 7fc3f2c4b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5595f479b270 0x5595f4795c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:54.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.106+0000 7fc3f2c4b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595f47a5940 0x5595f47a4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:54.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.106+0000 7fc3f2c4b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e81b5dd0 con 0x5595f47a5940 2024-04-24T09:13:54.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.106+0000 7fc3f2c4b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e81b79a0 con 0x5595f479a950 2024-04-24T09:13:54.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.106+0000 7fc3f2c4b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e81b64a0 con 0x5595f479b270 2024-04-24T09:13:54.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.106+0000 7fc3ede00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595f47a5940 0x5595f47a4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:54.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.106+0000 7fc3ede00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595f47a5940 0x5595f47a4f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44990/0 (socket says 172.21.15.83:44990) 2024-04-24T09:13:54.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.106+0000 7fc3ede00640 1 -- 172.21.15.83:0/672704311 learned_addr learned my addr 172.21.15.83:0/672704311 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:13:54.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.106+0000 7fc3ed400640 1 --2- 172.21.15.83:0/672704311 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5595f479a950 0x5595f47a1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:54.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.107+0000 7fc3eca00640 1 --2- 172.21.15.83:0/672704311 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5595f479b270 0x5595f4795c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:54.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.107+0000 7fc3ed400640 1 -- 172.21.15.83:0/672704311 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5595f479b270 msgr2=0x5595f4795c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:54.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.107+0000 7fc3ed400640 1 --2- 172.21.15.83:0/672704311 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5595f479b270 0x5595f4795c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:54.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.107+0000 7fc3ed400640 1 -- 172.21.15.83:0/672704311 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595f47a5940 msgr2=0x5595f47a4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:54.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.107+0000 7fc3ed400640 1 --2- 172.21.15.83:0/672704311 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595f47a5940 0x5595f47a4f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:54.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.107+0000 7fc3ed400640 1 -- 172.21.15.83:0/672704311 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595f44dc7d0 con 0x5595f479a950 2024-04-24T09:13:54.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.107+0000 7fc3ede00640 1 --2- 172.21.15.83:0/672704311 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595f47a5940 0x5595f47a4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:13:54.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.107+0000 7fc3ed400640 1 --2- 172.21.15.83:0/672704311 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5595f479a950 0x5595f47a1620 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc099eb0 tx=0x7fc3dc009f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca9768960a4eee3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:54.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.107+0000 7fc3d9c00640 1 -- 172.21.15.83:0/672704311 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3dc020070 con 0x5595f479a950 2024-04-24T09:13:54.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.107+0000 7fc3d9c00640 1 -- 172.21.15.83:0/672704311 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc3dc0b0400 con 0x5595f479a950 2024-04-24T09:13:54.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.108+0000 7fc3d9c00640 1 -- 172.21.15.83:0/672704311 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3dc0108a0 con 0x5595f479a950 2024-04-24T09:13:54.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.108+0000 7fc3f2c4b740 1 -- 172.21.15.83:0/672704311 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5595f479a950 msgr2=0x5595f47a1620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:54.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.108+0000 7fc3f2c4b740 1 --2- 172.21.15.83:0/672704311 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5595f479a950 0x5595f47a1620 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc099eb0 tx=0x7fc3dc009f60 comp rx=0 tx=0).stop 2024-04-24T09:13:54.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.109+0000 7fc3f2c4b740 1 -- 172.21.15.83:0/672704311 shutdown_connections 2024-04-24T09:13:54.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.109+0000 7fc3f2c4b740 1 --2- 172.21.15.83:0/672704311 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595f47a5940 0x5595f47a4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:54.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.109+0000 7fc3f2c4b740 1 --2- 172.21.15.83:0/672704311 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5595f479b270 0x5595f4795c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:54.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.109+0000 7fc3f2c4b740 1 --2- 172.21.15.83:0/672704311 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5595f479a950 0x5595f47a1620 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:54.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.109+0000 7fc3f2c4b740 1 -- 172.21.15.83:0/672704311 >> 172.21.15.83:0/672704311 conn(0x5595f4796c70 msgr2=0x5595f4795560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:54.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.109+0000 7fc3f2c4b740 1 -- 172.21.15.83:0/672704311 shutdown_connections 2024-04-24T09:13:54.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.109+0000 7fc3f2c4b740 1 -- 172.21.15.83:0/672704311 wait complete. 2024-04-24T09:13:54.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.109+0000 7fc3f2c4b740 1 -- start start 2024-04-24T09:13:54.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.110+0000 7fc3f2c4b740 1 Processor -- start 2024-04-24T09:13:54.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.110+0000 7fc3f2c4b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5595f479a950 0x5595f479f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:54.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.110+0000 7fc3f2c4b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595f479b270 0x5595f479fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:54.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.110+0000 7fc3f2c4b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5595f47a5940 0x5595f47a0110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:54.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.110+0000 7fc3f2c4b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e81b72a0 con 0x5595f479b270 2024-04-24T09:13:54.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.110+0000 7fc3f2c4b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e81b6bd0 con 0x5595f479a950 2024-04-24T09:13:54.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.110+0000 7fc3f2c4b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e81b3df0 con 0x5595f47a5940 2024-04-24T09:13:54.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.111+0000 7fc3f2c4b740 10 client.0 _ll_register_callbacks cb 0x7fc3dc09b5f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:13:54.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.111+0000 7fc3ed400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5595f479a950 0x5595f479f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:54.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.111+0000 7fc3ed400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5595f479a950 0x5595f479f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:60680/0 (socket says 172.21.15.83:60680) 2024-04-24T09:13:54.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.111+0000 7fc3ed400640 1 -- 172.21.15.83:0/164643978 learned_addr learned my addr 172.21.15.83:0/164643978 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:13:54.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.111+0000 7fc3ede00640 1 --2- 172.21.15.83:0/164643978 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5595f47a5940 0x5595f47a0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:54.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.111+0000 7fc3eca00640 1 --2- 172.21.15.83:0/164643978 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595f479b270 0x5595f479fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:54.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.111+0000 7fc3ed400640 1 -- 172.21.15.83:0/164643978 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5595f47a5940 msgr2=0x5595f47a0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:54.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.112+0000 7fc3ed400640 1 --2- 172.21.15.83:0/164643978 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5595f47a5940 0x5595f47a0110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:54.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.112+0000 7fc3ed400640 1 -- 172.21.15.83:0/164643978 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595f479b270 msgr2=0x5595f479fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:54.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.112+0000 7fc3ed400640 1 --2- 172.21.15.83:0/164643978 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595f479b270 0x5595f479fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:54.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.112+0000 7fc3ed400640 1 -- 172.21.15.83:0/164643978 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595f481ae10 con 0x5595f479a950 2024-04-24T09:13:54.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.112+0000 7fc3eca00640 1 --2- 172.21.15.83:0/164643978 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595f479b270 0x5595f479fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:13:54.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.112+0000 7fc3ed400640 1 --2- 172.21.15.83:0/164643978 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5595f479a950 0x5595f479f670 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc098010 tx=0x7fc3dc0b5230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d557df4da2aea03a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:54.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.112+0000 7fc3cea00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:13:54.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.112+0000 7fc3cea00640 1 -- 172.21.15.83:0/164643978 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3dc020070 con 0x5595f479a950 2024-04-24T09:13:54.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.113+0000 7fc3cea00640 1 -- 172.21.15.83:0/164643978 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9583+0+0 (secure 0 0 0) 0x7fc3dc0108a0 con 0x5595f479a950 2024-04-24T09:13:54.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.113+0000 7fc3f2c4b740 1 -- 172.21.15.83:0/164643978 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3dc0b0400 con 0x5595f479a950 2024-04-24T09:13:54.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.113+0000 7fc3cea00640 1 -- 172.21.15.83:0/164643978 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7fc3dc09ffb0 con 0x5595f479a950 2024-04-24T09:13:54.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.113+0000 7fc3f2c4b740 10 client.40223 fetch_fsmap learned FSMap version 64 2024-04-24T09:13:54.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.113+0000 7fc3f2c4b740 1 -- 172.21.15.83:0/164643978 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x5595f481ae10 con 0x5595f479a950 2024-04-24T09:13:54.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.114+0000 7fc3cea00640 1 -- 172.21.15.83:0/164643978 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc3dc0b0400 con 0x5595f479a950 2024-04-24T09:13:54.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.114+0000 7fc3cea00640 1 -- 172.21.15.83:0/164643978 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 64) ==== 8374+0+0 (secure 0 0 0) 0x7fc3dc097950 con 0x5595f479a950 2024-04-24T09:13:54.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.114+0000 7fc3f2c4b740 10 client.40223 fetch_fsmap finished waiting for FSMap version 64 2024-04-24T09:13:54.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.114+0000 7fc3f2c4b740 10 client.40223 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-24T09:13:54.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.114+0000 7fc3f2c4b740 20 client.40223 populate_metadata read hostname 'smithi083' 2024-04-24T09:13:54.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.114+0000 7fc3f2c4b740 1 --2- 172.21.15.83:0/164643978 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x5595f481c310 0x5595f481e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:54.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.114+0000 7fc3f2c4b740 4 client.40223 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:13:54.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.114+0000 7fc3f2c4b740 1 -- 172.21.15.83:0/164643978 --> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5595f4707a60 con 0x5595f481c310 2024-04-24T09:13:54.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.115+0000 7fc3eca00640 1 --2- 172.21.15.83:0/164643978 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x5595f481c310 0x5595f481e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:54.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.116+0000 7fc3eca00640 1 --2- 172.21.15.83:0/164643978 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x5595f481c310 0x5595f481e700 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=358bbed67d0da7ba server_cookie=1cbba3e3531b2963 in_seq=0 out_seq=0 2024-04-24T09:13:54.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.116+0000 7fc3cea00640 10 client.40223 ms_handle_connect on v2:172.21.15.189:6838/2548965336 2024-04-24T09:13:54.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.117+0000 7fc3cea00640 1 -- 172.21.15.83:0/164643978 <== mds.4 v2:172.21.15.189:6838/2548965336 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5595f4707a60 con 0x5595f481c310 2024-04-24T09:13:54.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.117+0000 7fc3cea00640 10 client.40223 handle_command_reply: tid=0 2024-04-24T09:13:54.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.117+0000 7fc3f2c4b740 1 client.40223 shutdown 2024-04-24T09:13:54.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.117+0000 7fc3f2c4b740 1 -- 172.21.15.83:0/164643978 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5595f479a950 msgr2=0x5595f479f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:54.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.117+0000 7fc3f2c4b740 1 --2- 172.21.15.83:0/164643978 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5595f479a950 0x5595f479f670 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc098010 tx=0x7fc3dc0b5230 comp rx=0 tx=0).stop 2024-04-24T09:13:54.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.117+0000 7fc3f2c4b740 1 -- 172.21.15.83:0/164643978 shutdown_connections 2024-04-24T09:13:54.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.117+0000 7fc3f2c4b740 1 --2- 172.21.15.83:0/164643978 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x5595f481c310 0x5595f481e700 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:54.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.117+0000 7fc3f2c4b740 1 --2- 172.21.15.83:0/164643978 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5595f47a5940 0x5595f47a0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:54.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.117+0000 7fc3f2c4b740 1 --2- 172.21.15.83:0/164643978 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5595f479b270 0x5595f479fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:54.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.117+0000 7fc3cea00640 0 client.40223 ms_handle_reset on v2:172.21.15.189:6838/2548965336 2024-04-24T09:13:54.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.117+0000 7fc3f2c4b740 1 --2- 172.21.15.83:0/164643978 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5595f479a950 0x5595f479f670 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:54.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.117+0000 7fc3f2c4b740 1 -- 172.21.15.83:0/164643978 >> 172.21.15.83:0/164643978 conn(0x5595f4796c70 msgr2=0x5595f4795df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:54.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.118+0000 7fc3f2c4b740 1 -- 172.21.15.83:0/164643978 shutdown_connections 2024-04-24T09:13:54.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.118+0000 7fc3f2c4b740 1 -- 172.21.15.83:0/164643978 wait complete. 2024-04-24T09:13:54.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.118+0000 7fc3f2c4b740 20 client.40223 trim_cache size 0 max 16384 2024-04-24T09:13:54.119 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:13:54.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.119+0000 7fc3efe00640 1 -- 172.21.15.83:0/2826690388 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fc3d00a3de0 msgr2=0x7fc3d00a62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:54.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.119+0000 7fc3efe00640 1 --2- 172.21.15.83:0/2826690388 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fc3d00a3de0 0x7fc3d00a62a0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fc3e8110690 tx=0x7fc3d400bf90 comp rx=0 tx=0).stop 2024-04-24T09:13:54.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.119+0000 7fc3efe00640 1 -- 172.21.15.83:0/2826690388 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3e80742d0 msgr2=0x7fc3e81a06b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:54.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.119+0000 7fc3efe00640 1 --2- 172.21.15.83:0/2826690388 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3e80742d0 0x7fc3e81a06b0 secure :-1 s=READY pgs=2636 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc00d950 tx=0x7fc3dc00de10 comp rx=0 tx=0).stop 2024-04-24T09:13:54.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.119+0000 7fc3efe00640 1 -- 172.21.15.83:0/2826690388 shutdown_connections 2024-04-24T09:13:54.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.119+0000 7fc3efe00640 1 --2- 172.21.15.83:0/2826690388 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fc3d00a3de0 0x7fc3d00a62a0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:54.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.119+0000 7fc3efe00640 1 --2- 172.21.15.83:0/2826690388 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc3e810e8e0 0x7fc3e819a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:54.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.119+0000 7fc3efe00640 1 --2- 172.21.15.83:0/2826690388 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3e8074bf0 0x7fc3e81a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:54.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.119+0000 7fc3efe00640 1 --2- 172.21.15.83:0/2826690388 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3e80742d0 0x7fc3e81a06b0 unknown :-1 s=CLOSED pgs=2636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:54.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.119+0000 7fc3efe00640 1 -- 172.21.15.83:0/2826690388 >> 172.21.15.83:0/2826690388 conn(0x7fc3e8079450 msgr2=0x7fc3e80ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:54.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.119+0000 7fc3efe00640 1 -- 172.21.15.83:0/2826690388 shutdown_connections 2024-04-24T09:13:54.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.119+0000 7fc3efe00640 1 -- 172.21.15.83:0/2826690388 wait complete. 2024-04-24T09:13:54.143 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-24T09:13:54.201 INFO:tasks.workunit.client.0.smithi083.stdout: 10 62632 6.61 MB/sec execute 385 sec latency 274.618 ms 2024-04-24T09:13:54.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.388+0000 7f534e800640 1 -- 172.21.15.83:0/2399545709 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53481093f0 msgr2=0x7f5348110fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:54.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.388+0000 7f534e800640 1 --2- 172.21.15.83:0/2399545709 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53481093f0 0x7f5348110fc0 secure :-1 s=READY pgs=2637 cs=0 l=1 rev1=1 crypto rx=0x7f5340009ca0 tx=0x7f534002a8e0 comp rx=0 tx=0).stop 2024-04-24T09:13:54.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.389+0000 7f534e800640 1 -- 172.21.15.83:0/2399545709 shutdown_connections 2024-04-24T09:13:54.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.389+0000 7f534e800640 1 --2- 172.21.15.83:0/2399545709 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53481093f0 0x7f5348110fc0 unknown :-1 s=CLOSED pgs=2637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:54.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.389+0000 7f534e800640 1 --2- 172.21.15.83:0/2399545709 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5348104cd0 0x7f5348108d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:54.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.389+0000 7f534e800640 1 --2- 172.21.15.83:0/2399545709 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5348104320 0x7f5348104700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:54.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.389+0000 7f534e800640 1 -- 172.21.15.83:0/2399545709 >> 172.21.15.83:0/2399545709 conn(0x7f53480ffb40 msgr2=0x7f5348101f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:54.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.390+0000 7f534e800640 1 -- 172.21.15.83:0/2399545709 shutdown_connections 2024-04-24T09:13:54.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.390+0000 7f534e800640 1 -- 172.21.15.83:0/2399545709 wait complete. 2024-04-24T09:13:54.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.391+0000 7f534e800640 1 Processor -- start 2024-04-24T09:13:54.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.392+0000 7f534e800640 1 -- start start 2024-04-24T09:13:54.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.392+0000 7f534e800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5348104320 0x7f53481a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:54.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.392+0000 7f534e800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f53481093f0 0x7f53481a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:54.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.392+0000 7f534e800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f534819a7a0 0x7f534819ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:54.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.392+0000 7f534e800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5348114400 con 0x7f5348104320 2024-04-24T09:13:54.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.392+0000 7f534e800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5348114280 con 0x7f534819a7a0 2024-04-24T09:13:54.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.392+0000 7f534e800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5348114580 con 0x7f53481093f0 2024-04-24T09:13:54.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.393+0000 7f534de00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f534819a7a0 0x7f534819ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:54.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.393+0000 7f534de00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f534819a7a0 0x7f534819ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:60698/0 (socket says 172.21.15.83:60698) 2024-04-24T09:13:54.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.393+0000 7f534de00640 1 -- 172.21.15.83:0/2655523757 learned_addr learned my addr 172.21.15.83:0/2655523757 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:13:54.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.393+0000 7f534ca00640 1 --2- 172.21.15.83:0/2655523757 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f53481093f0 0x7f53481a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:54.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.394+0000 7f534ca00640 1 -- 172.21.15.83:0/2655523757 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f534819a7a0 msgr2=0x7f534819ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:54.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.394+0000 7f534ca00640 1 --2- 172.21.15.83:0/2655523757 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f534819a7a0 0x7f534819ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:54.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.394+0000 7f534ca00640 1 -- 172.21.15.83:0/2655523757 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5348104320 msgr2=0x7f53481a0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:54.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.394+0000 7f534ca00640 1 --2- 172.21.15.83:0/2655523757 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5348104320 0x7f53481a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:54.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.394+0000 7f534ca00640 1 -- 172.21.15.83:0/2655523757 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f534819b330 con 0x7f53481093f0 2024-04-24T09:13:54.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.394+0000 7f534ca00640 1 --2- 172.21.15.83:0/2655523757 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f53481093f0 0x7f53481a0b70 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f533c00d6a0 tx=0x7f533c00db60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:54.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.395+0000 7f5346000640 1 -- 172.21.15.83:0/2655523757 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f533c017070 con 0x7f53481093f0 2024-04-24T09:13:54.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.395+0000 7f534e800640 1 -- 172.21.15.83:0/2655523757 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f534819b620 con 0x7f53481093f0 2024-04-24T09:13:54.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.395+0000 7f534e800640 1 -- 172.21.15.83:0/2655523757 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53481a7850 con 0x7f53481093f0 2024-04-24T09:13:54.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.395+0000 7f5346000640 1 -- 172.21.15.83:0/2655523757 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f533c009820 con 0x7f53481093f0 2024-04-24T09:13:54.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.395+0000 7f5346000640 1 -- 172.21.15.83:0/2655523757 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f533c017070 con 0x7f53481093f0 2024-04-24T09:13:54.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.400+0000 7f5352b73740 1 Processor -- start 2024-04-24T09:13:54.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.400+0000 7f5352b73740 1 -- start start 2024-04-24T09:13:54.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.400+0000 7f5352b73740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bfe4dceb40 0x55bfe4dcef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:54.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.400+0000 7f5352b73740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bfe4dcf460 0x55bfe4dd95f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:54.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.400+0000 7f5352b73740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bfe4dd9b30 0x55bfe4ddbf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:54.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.400+0000 7f5352b73740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55bfe4d341d0 con 0x55bfe4dd9b30 2024-04-24T09:13:54.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.400+0000 7f5352b73740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55bfe4cda450 con 0x55bfe4dceb40 2024-04-24T09:13:54.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.400+0000 7f5352b73740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55bfe4d30190 con 0x55bfe4dcf460 2024-04-24T09:13:54.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.401+0000 7f534de00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bfe4dceb40 0x55bfe4dcef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:54.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.401+0000 7f534de00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bfe4dceb40 0x55bfe4dcef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:60708/0 (socket says 172.21.15.83:60708) 2024-04-24T09:13:54.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.401+0000 7f534de00640 1 -- 172.21.15.83:0/4142692736 learned_addr learned my addr 172.21.15.83:0/4142692736 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:13:54.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.401+0000 7f534d400640 1 --2- 172.21.15.83:0/4142692736 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bfe4dcf460 0x55bfe4dd95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:54.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.401+0000 7f534de00640 1 -- 172.21.15.83:0/4142692736 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bfe4dcf460 msgr2=0x55bfe4dd95f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:54.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.401+0000 7f534de00640 1 --2- 172.21.15.83:0/4142692736 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bfe4dcf460 0x55bfe4dd95f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:54.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.401+0000 7f534de00640 1 -- 172.21.15.83:0/4142692736 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bfe4dd9b30 msgr2=0x55bfe4ddbf20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T09:13:54.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.401+0000 7f534de00640 1 --2- 172.21.15.83:0/4142692736 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bfe4dd9b30 0x55bfe4ddbf20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:54.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.401+0000 7f534de00640 1 -- 172.21.15.83:0/4142692736 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bfe4d97310 con 0x55bfe4dceb40 2024-04-24T09:13:54.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.402+0000 7f534de00640 1 --2- 172.21.15.83:0/4142692736 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bfe4dceb40 0x55bfe4dcef20 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f5340031620 tx=0x7f5340031650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd260746e2bae2db server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:54.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.402+0000 7f533a600640 1 -- 172.21.15.83:0/4142692736 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5340031ac0 con 0x55bfe4dceb40 2024-04-24T09:13:54.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.403+0000 7f533a600640 1 -- 172.21.15.83:0/4142692736 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5340036040 con 0x55bfe4dceb40 2024-04-24T09:13:54.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.403+0000 7f533a600640 1 -- 172.21.15.83:0/4142692736 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53400046f0 con 0x55bfe4dceb40 2024-04-24T09:13:54.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.403+0000 7f5346000640 1 -- 172.21.15.83:0/2655523757 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f533c005220 con 0x7f53481093f0 2024-04-24T09:13:54.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.405+0000 7f5352b73740 1 -- 172.21.15.83:0/4142692736 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bfe4dceb40 msgr2=0x55bfe4dcef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:54.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.405+0000 7f5352b73740 1 --2- 172.21.15.83:0/4142692736 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bfe4dceb40 0x55bfe4dcef20 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f5340031620 tx=0x7f5340031650 comp rx=0 tx=0).stop 2024-04-24T09:13:54.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.406+0000 7f5352b73740 1 -- 172.21.15.83:0/4142692736 shutdown_connections 2024-04-24T09:13:54.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.406+0000 7f5352b73740 1 --2- 172.21.15.83:0/4142692736 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bfe4dd9b30 0x55bfe4ddbf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:54.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.406+0000 7f5352b73740 1 --2- 172.21.15.83:0/4142692736 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bfe4dcf460 0x55bfe4dd95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:54.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.406+0000 7f5352b73740 1 --2- 172.21.15.83:0/4142692736 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bfe4dceb40 0x55bfe4dcef20 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:54.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.406+0000 7f5352b73740 1 -- 172.21.15.83:0/4142692736 >> 172.21.15.83:0/4142692736 conn(0x55bfe4dcae60 msgr2=0x55bfe4dcb240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:54.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.406+0000 7f5352b73740 1 -- 172.21.15.83:0/4142692736 shutdown_connections 2024-04-24T09:13:54.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.406+0000 7f5352b73740 1 -- 172.21.15.83:0/4142692736 wait complete. 2024-04-24T09:13:54.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.407+0000 7f5346000640 1 --2- 172.21.15.83:0/2655523757 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f53300ac640 0x7f53300aeb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:54.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.407+0000 7f5352b73740 1 -- start start 2024-04-24T09:13:54.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.407+0000 7f5346000640 1 -- 172.21.15.83:0/2655523757 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 9583+0+0 (secure 0 0 0) 0x7f533c0d0080 con 0x7f53481093f0 2024-04-24T09:13:54.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.407+0000 7f5352b73740 1 Processor -- start 2024-04-24T09:13:54.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.407+0000 7f534de00640 1 --2- 172.21.15.83:0/2655523757 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f53300ac640 0x7f53300aeb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:54.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.408+0000 7f5352b73740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bfe4dceb40 0x55bfe4dd6930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:54.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.408+0000 7f5352b73740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bfe4dcf460 0x55bfe4dd6e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:54.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.408+0000 7f5352b73740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bfe4dd9b30 0x55bfe4dd73b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:54.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.408+0000 7f5352b73740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55bfe4ded970 con 0x55bfe4dd9b30 2024-04-24T09:13:54.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.408+0000 7f5352b73740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55bfe4ced810 con 0x55bfe4dceb40 2024-04-24T09:13:54.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.408+0000 7f5352b73740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55bfe4dedaf0 con 0x55bfe4dcf460 2024-04-24T09:13:54.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.409+0000 7f534de00640 1 --2- 172.21.15.83:0/2655523757 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f53300ac640 0x7f53300aeb00 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f5340009620 tx=0x7f534002a8e0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:54.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.409+0000 7f5352b73740 10 client.0 _ll_register_callbacks cb 0x55bfe4cbb140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:13:54.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.410+0000 7f534d400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bfe4dceb40 0x55bfe4dd6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:54.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.410+0000 7f534d400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bfe4dceb40 0x55bfe4dd6930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:60714/0 (socket says 172.21.15.83:60714) 2024-04-24T09:13:54.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.410+0000 7f534d400640 1 -- 172.21.15.83:0/3259627718 learned_addr learned my addr 172.21.15.83:0/3259627718 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:13:54.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.410+0000 7f534de00640 1 --2- 172.21.15.83:0/3259627718 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bfe4dd9b30 0x55bfe4dd73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:54.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.410+0000 7f534ca00640 1 --2- 172.21.15.83:0/3259627718 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bfe4dcf460 0x55bfe4dd6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:54.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.411+0000 7f534d400640 1 -- 172.21.15.83:0/3259627718 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bfe4dcf460 msgr2=0x55bfe4dd6e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:54.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.411+0000 7f534d400640 1 --2- 172.21.15.83:0/3259627718 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bfe4dcf460 0x55bfe4dd6e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:54.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.411+0000 7f534d400640 1 -- 172.21.15.83:0/3259627718 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bfe4dd9b30 msgr2=0x55bfe4dd73b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:54.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.411+0000 7f534d400640 1 --2- 172.21.15.83:0/3259627718 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bfe4dd9b30 0x55bfe4dd73b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:54.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.411+0000 7f534d400640 1 -- 172.21.15.83:0/3259627718 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bfe4cfe3f0 con 0x55bfe4dceb40 2024-04-24T09:13:54.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.412+0000 7f534de00640 1 --2- 172.21.15.83:0/3259627718 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bfe4dd9b30 0x55bfe4dd73b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:13:54.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.413+0000 7f534d400640 1 --2- 172.21.15.83:0/3259627718 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bfe4dceb40 0x55bfe4dd6930 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f533400d210 tx=0x7f533400d6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0d33bb1149e8cea server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:54.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.413+0000 7f532ea00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:13:54.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.414+0000 7f532ea00640 1 -- 172.21.15.83:0/3259627718 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53340125f0 con 0x55bfe4dceb40 2024-04-24T09:13:54.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.415+0000 7f532ea00640 1 -- 172.21.15.83:0/3259627718 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9583+0+0 (secure 0 0 0) 0x7f5334012790 con 0x55bfe4dceb40 2024-04-24T09:13:54.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.415+0000 7f5352b73740 1 -- 172.21.15.83:0/3259627718 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5340036040 con 0x55bfe4dceb40 2024-04-24T09:13:54.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.415+0000 7f532ea00640 1 -- 172.21.15.83:0/3259627718 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f5334012990 con 0x55bfe4dceb40 2024-04-24T09:13:54.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.416+0000 7f5352b73740 10 client.40235 fetch_fsmap learned FSMap version 64 2024-04-24T09:13:54.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.416+0000 7f5352b73740 1 -- 172.21.15.83:0/3259627718 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55bfe4a463f0 con 0x55bfe4dceb40 2024-04-24T09:13:54.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.416+0000 7f532ea00640 1 -- 172.21.15.83:0/3259627718 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5340036040 con 0x55bfe4dceb40 2024-04-24T09:13:54.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.417+0000 7f532ea00640 1 -- 172.21.15.83:0/3259627718 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 64) ==== 8374+0+0 (secure 0 0 0) 0x7f533401d450 con 0x55bfe4dceb40 2024-04-24T09:13:54.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.417+0000 7f5352b73740 10 client.40235 fetch_fsmap finished waiting for FSMap version 64 2024-04-24T09:13:54.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.417+0000 7f5352b73740 10 client.40235 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-24T09:13:54.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.417+0000 7f5352b73740 20 client.40235 populate_metadata read hostname 'smithi083' 2024-04-24T09:13:54.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.417+0000 7f5352b73740 1 --2- 172.21.15.83:0/3259627718 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55bfe4e1cee0 0x55bfe4e1f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:54.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.417+0000 7f5352b73740 4 client.40235 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:13:54.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.417+0000 7f5352b73740 1 -- 172.21.15.83:0/3259627718 --> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bfe4d41bf0 con 0x55bfe4e1cee0 2024-04-24T09:13:54.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.418+0000 7f534ca00640 1 --2- 172.21.15.83:0/3259627718 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55bfe4e1cee0 0x55bfe4e1f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:54.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.419+0000 7f532ea00640 10 client.40235 ms_handle_connect on v2:172.21.15.138:6832/1157967658 2024-04-24T09:13:54.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.419+0000 7f534ca00640 1 --2- 172.21.15.83:0/3259627718 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55bfe4e1cee0 0x55bfe4e1f2d0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a5e6c4da8b736e06 server_cookie=44b873104033bfb6 in_seq=0 out_seq=0 2024-04-24T09:13:54.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.422+0000 7f532ea00640 1 -- 172.21.15.83:0/3259627718 <== mds.3 v2:172.21.15.138:6832/1157967658 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bfe4d41bf0 con 0x55bfe4e1cee0 2024-04-24T09:13:54.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.422+0000 7f532ea00640 10 client.40235 handle_command_reply: tid=0 2024-04-24T09:13:54.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.422+0000 7f5352b73740 1 client.40235 shutdown 2024-04-24T09:13:54.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.425+0000 7f5352b73740 1 -- 172.21.15.83:0/3259627718 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bfe4dceb40 msgr2=0x55bfe4dd6930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:54.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.425+0000 7f5352b73740 1 --2- 172.21.15.83:0/3259627718 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bfe4dceb40 0x55bfe4dd6930 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f533400d210 tx=0x7f533400d6d0 comp rx=0 tx=0).stop 2024-04-24T09:13:54.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.425+0000 7f5352b73740 1 -- 172.21.15.83:0/3259627718 shutdown_connections 2024-04-24T09:13:54.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.425+0000 7f5352b73740 1 --2- 172.21.15.83:0/3259627718 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bfe4dd9b30 0x55bfe4dd73b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:54.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.425+0000 7f5352b73740 1 --2- 172.21.15.83:0/3259627718 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55bfe4e1cee0 0x55bfe4e1f2d0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:54.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.425+0000 7f532ea00640 0 client.40235 ms_handle_reset on v2:172.21.15.138:6832/1157967658 2024-04-24T09:13:54.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.425+0000 7f5352b73740 1 --2- 172.21.15.83:0/3259627718 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bfe4dcf460 0x55bfe4dd6e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:54.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.425+0000 7f5352b73740 1 --2- 172.21.15.83:0/3259627718 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bfe4dceb40 0x55bfe4dd6930 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:54.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.425+0000 7f5352b73740 1 -- 172.21.15.83:0/3259627718 >> 172.21.15.83:0/3259627718 conn(0x55bfe4dcae60 msgr2=0x55bfe4dd0b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:54.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.426+0000 7f5352b73740 1 -- 172.21.15.83:0/3259627718 shutdown_connections 2024-04-24T09:13:54.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.426+0000 7f5352b73740 1 -- 172.21.15.83:0/3259627718 wait complete. 2024-04-24T09:13:54.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.426+0000 7f5352b73740 20 client.40235 trim_cache size 0 max 16384 2024-04-24T09:13:54.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.452+0000 7f5352b73740 1 Processor -- start 2024-04-24T09:13:54.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.452+0000 7f5352b73740 1 -- start start 2024-04-24T09:13:54.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.452+0000 7f5352b73740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bfe4dcf460 0x55bfe4dd12a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:54.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.452+0000 7f5352b73740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bfe4dd9b30 0x55bfe4dd7220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:54.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.452+0000 7f5352b73740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bfe4dd9170 0x55bfe4e24bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:54.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.452+0000 7f5352b73740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5328010100 con 0x55bfe4dd9170 2024-04-24T09:13:54.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.452+0000 7f5352b73740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5328011cd0 con 0x55bfe4dcf460 2024-04-24T09:13:54.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.452+0000 7f5352b73740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f53280107d0 con 0x55bfe4dd9b30 2024-04-24T09:13:54.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.453+0000 7f534ca00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bfe4dd9b30 0x55bfe4dd7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:54.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.453+0000 7f534d400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bfe4dcf460 0x55bfe4dd12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:54.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.453+0000 7f534ca00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bfe4dd9b30 0x55bfe4dd7220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:41060/0 (socket says 172.21.15.83:41060) 2024-04-24T09:13:54.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.453+0000 7f534ca00640 1 -- 172.21.15.83:0/1730901156 learned_addr learned my addr 172.21.15.83:0/1730901156 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:13:54.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.453+0000 7f534de00640 1 --2- 172.21.15.83:0/1730901156 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bfe4dd9170 0x55bfe4e24bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:54.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.453+0000 7f534de00640 1 -- 172.21.15.83:0/1730901156 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bfe4dd9b30 msgr2=0x55bfe4dd7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:54.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.453+0000 7f534de00640 1 --2- 172.21.15.83:0/1730901156 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bfe4dd9b30 0x55bfe4dd7220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:54.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.453+0000 7f534de00640 1 -- 172.21.15.83:0/1730901156 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bfe4dcf460 msgr2=0x55bfe4dd12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:54.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.453+0000 7f534de00640 1 --2- 172.21.15.83:0/1730901156 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bfe4dcf460 0x55bfe4dd12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:54.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.453+0000 7f534de00640 1 -- 172.21.15.83:0/1730901156 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bfe4b107d0 con 0x55bfe4dd9170 2024-04-24T09:13:54.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.454+0000 7f534de00640 1 --2- 172.21.15.83:0/1730901156 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bfe4dd9170 0x55bfe4e24bb0 secure :-1 s=READY pgs=2638 cs=0 l=1 rev1=1 crypto rx=0x7f5340004350 tx=0x7f5340031500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7a8919eb941f313 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:54.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.454+0000 7f5339c00640 1 -- 172.21.15.83:0/1730901156 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5340003970 con 0x55bfe4dd9170 2024-04-24T09:13:54.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.455+0000 7f5339c00640 1 -- 172.21.15.83:0/1730901156 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5340036050 con 0x55bfe4dd9170 2024-04-24T09:13:54.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:13:54 smithi189 ceph-mon[31791]: pgmap v703: 129 pgs: 129 active+clean; 971 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.7 MiB/s wr, 322 op/s 2024-04-24T09:13:54.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:13:54 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/1803289117' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:13:54.457 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:13:54 smithi138 ceph-mon[31441]: pgmap v703: 129 pgs: 129 active+clean; 971 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.7 MiB/s wr, 322 op/s 2024-04-24T09:13:54.457 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:13:54 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/1803289117' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:13:54.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.455+0000 7f5339c00640 1 -- 172.21.15.83:0/1730901156 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5340030ab0 con 0x55bfe4dd9170 2024-04-24T09:13:54.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.455+0000 7f5352b73740 1 -- 172.21.15.83:0/1730901156 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bfe4dd9170 msgr2=0x55bfe4e24bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:54.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.455+0000 7f5352b73740 1 --2- 172.21.15.83:0/1730901156 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bfe4dd9170 0x55bfe4e24bb0 secure :-1 s=READY pgs=2638 cs=0 l=1 rev1=1 crypto rx=0x7f5340004350 tx=0x7f5340031500 comp rx=0 tx=0).stop 2024-04-24T09:13:54.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.456+0000 7f5352b73740 1 -- 172.21.15.83:0/1730901156 shutdown_connections 2024-04-24T09:13:54.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.456+0000 7f5352b73740 1 --2- 172.21.15.83:0/1730901156 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bfe4dd9170 0x55bfe4e24bb0 unknown :-1 s=CLOSED pgs=2638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:54.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.456+0000 7f5352b73740 1 --2- 172.21.15.83:0/1730901156 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bfe4dd9b30 0x55bfe4dd7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:54.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.456+0000 7f5352b73740 1 --2- 172.21.15.83:0/1730901156 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bfe4dcf460 0x55bfe4dd12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:54.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.456+0000 7f5352b73740 1 -- 172.21.15.83:0/1730901156 >> 172.21.15.83:0/1730901156 conn(0x55bfe4dcae60 msgr2=0x55bfe4dc96c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:54.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.456+0000 7f5352b73740 1 -- 172.21.15.83:0/1730901156 shutdown_connections 2024-04-24T09:13:54.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.456+0000 7f5352b73740 1 -- 172.21.15.83:0/1730901156 wait complete. 2024-04-24T09:13:54.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.456+0000 7f5352b73740 1 -- start start 2024-04-24T09:13:54.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.457+0000 7f5352b73740 1 Processor -- start 2024-04-24T09:13:54.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.457+0000 7f5352b73740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bfe4dcf460 0x55bfe4df30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:54.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.457+0000 7f5352b73740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bfe4dd9170 0x55bfe4dda530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:54.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.457+0000 7f5352b73740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bfe4dd9b30 0x55bfe4ddaa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:54.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.457+0000 7f5352b73740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f53280115d0 con 0x55bfe4dd9b30 2024-04-24T09:13:54.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.457+0000 7f5352b73740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5328010f00 con 0x55bfe4dcf460 2024-04-24T09:13:54.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.457+0000 7f5352b73740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f532800e100 con 0x55bfe4dd9170 2024-04-24T09:13:54.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.457+0000 7f5352b73740 10 client.0 _ll_register_callbacks cb 0x7f533400b160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:13:54.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.458+0000 7f534ca00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bfe4dd9170 0x55bfe4dda530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:54.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.458+0000 7f534ca00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bfe4dd9170 0x55bfe4dda530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:41066/0 (socket says 172.21.15.83:41066) 2024-04-24T09:13:54.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.458+0000 7f534de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bfe4dd9b30 0x55bfe4ddaa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:54.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.458+0000 7f534de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bfe4dd9b30 0x55bfe4ddaa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45020/0 (socket says 172.21.15.83:45020) 2024-04-24T09:13:54.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.458+0000 7f534de00640 1 -- 172.21.15.83:0/715231550 learned_addr learned my addr 172.21.15.83:0/715231550 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:13:54.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.458+0000 7f534d400640 1 --2- 172.21.15.83:0/715231550 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bfe4dcf460 0x55bfe4df30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:54.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.459+0000 7f534ca00640 1 -- 172.21.15.83:0/715231550 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bfe4dcf460 msgr2=0x55bfe4df30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:54.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.459+0000 7f534ca00640 1 --2- 172.21.15.83:0/715231550 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bfe4dcf460 0x55bfe4df30b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:54.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.459+0000 7f534ca00640 1 -- 172.21.15.83:0/715231550 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bfe4dd9b30 msgr2=0x55bfe4ddaa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:54.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.459+0000 7f534ca00640 1 --2- 172.21.15.83:0/715231550 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bfe4dd9b30 0x55bfe4ddaa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:54.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.459+0000 7f534ca00640 1 -- 172.21.15.83:0/715231550 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bfe4ddb130 con 0x55bfe4dd9170 2024-04-24T09:13:54.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.459+0000 7f534ca00640 1 --2- 172.21.15.83:0/715231550 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bfe4dd9170 0x55bfe4dda530 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f533c009610 tx=0x7f533c005cf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18701d5c6f6f2cf5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:54.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.459+0000 7f532ea00640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-04-24T09:13:54.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.460+0000 7f532ea00640 1 -- 172.21.15.83:0/715231550 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f533c014070 con 0x55bfe4dd9170 2024-04-24T09:13:54.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.460+0000 7f532ea00640 1 -- 172.21.15.83:0/715231550 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9583+0+0 (secure 0 0 0) 0x7f533c0cd630 con 0x55bfe4dd9170 2024-04-24T09:13:54.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.460+0000 7f5352b73740 1 -- 172.21.15.83:0/715231550 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5340036050 con 0x55bfe4dd9170 2024-04-24T09:13:54.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.461+0000 7f532ea00640 1 -- 172.21.15.83:0/715231550 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f533c020040 con 0x55bfe4dd9170 2024-04-24T09:13:54.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.461+0000 7f5352b73740 10 client.30004 fetch_fsmap learned FSMap version 64 2024-04-24T09:13:54.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.461+0000 7f5352b73740 1 -- 172.21.15.83:0/715231550 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55bfe4e83420 con 0x55bfe4dd9170 2024-04-24T09:13:54.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.461+0000 7f532ea00640 1 -- 172.21.15.83:0/715231550 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5340036050 con 0x55bfe4dd9170 2024-04-24T09:13:54.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.462+0000 7f532ea00640 1 -- 172.21.15.83:0/715231550 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 64) ==== 8374+0+0 (secure 0 0 0) 0x7f533c0cdb00 con 0x55bfe4dd9170 2024-04-24T09:13:54.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.462+0000 7f5352b73740 10 client.30004 fetch_fsmap finished waiting for FSMap version 64 2024-04-24T09:13:54.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.462+0000 7f5352b73740 10 client.30004 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-24T09:13:54.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.462+0000 7f5352b73740 20 client.30004 populate_metadata read hostname 'smithi083' 2024-04-24T09:13:54.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.462+0000 7f5352b73740 1 --2- 172.21.15.83:0/715231550 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55bfe4e83a30 0x55bfe4e85e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:54.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.462+0000 7f5352b73740 4 client.30004 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:13:54.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.462+0000 7f5352b73740 1 -- 172.21.15.83:0/715231550 --> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bfe4d3ba60 con 0x55bfe4e83a30 2024-04-24T09:13:54.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.462+0000 7f534d400640 1 --2- 172.21.15.83:0/715231550 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55bfe4e83a30 0x55bfe4e85e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:54.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.464+0000 7f534d400640 1 --2- 172.21.15.83:0/715231550 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55bfe4e83a30 0x55bfe4e85e50 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7533b266553dc598 server_cookie=6374950de7faaca9 in_seq=0 out_seq=0 2024-04-24T09:13:54.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.464+0000 7f532ea00640 10 client.30004 ms_handle_connect on v2:172.21.15.138:6832/1157967658 2024-04-24T09:13:54.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.464+0000 7f532ea00640 1 -- 172.21.15.83:0/715231550 <== mds.3 v2:172.21.15.138:6832/1157967658 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bfe4d3ba60 con 0x55bfe4e83a30 2024-04-24T09:13:54.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.464+0000 7f532ea00640 10 client.30004 handle_command_reply: tid=0 2024-04-24T09:13:54.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.464+0000 7f5352b73740 1 client.30004 shutdown 2024-04-24T09:13:54.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.465+0000 7f5352b73740 1 -- 172.21.15.83:0/715231550 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bfe4dd9170 msgr2=0x55bfe4dda530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:54.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.465+0000 7f5352b73740 1 --2- 172.21.15.83:0/715231550 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bfe4dd9170 0x55bfe4dda530 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f533c009610 tx=0x7f533c005cf0 comp rx=0 tx=0).stop 2024-04-24T09:13:54.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.465+0000 7f5352b73740 1 -- 172.21.15.83:0/715231550 shutdown_connections 2024-04-24T09:13:54.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.465+0000 7f5352b73740 1 --2- 172.21.15.83:0/715231550 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bfe4dd9b30 0x55bfe4ddaa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:54.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.465+0000 7f5352b73740 1 --2- 172.21.15.83:0/715231550 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55bfe4e83a30 0x55bfe4e85e50 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:54.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.465+0000 7f5352b73740 1 --2- 172.21.15.83:0/715231550 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bfe4dd9170 0x55bfe4dda530 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:54.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.465+0000 7f5352b73740 1 --2- 172.21.15.83:0/715231550 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bfe4dcf460 0x55bfe4df30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:54.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.465+0000 7f532ea00640 0 client.30004 ms_handle_reset on v2:172.21.15.138:6832/1157967658 2024-04-24T09:13:54.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.465+0000 7f5352b73740 1 -- 172.21.15.83:0/715231550 >> 172.21.15.83:0/715231550 conn(0x55bfe4dcae60 msgr2=0x55bfe4dd89f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:54.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.465+0000 7f5352b73740 1 -- 172.21.15.83:0/715231550 shutdown_connections 2024-04-24T09:13:54.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.465+0000 7f5352b73740 1 -- 172.21.15.83:0/715231550 wait complete. 2024-04-24T09:13:54.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.466+0000 7f5352b73740 20 client.30004 trim_cache size 0 max 16384 2024-04-24T09:13:54.469 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:13:54.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.466+0000 7f534e800640 1 -- 172.21.15.83:0/2655523757 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f53300ac640 msgr2=0x7f53300aeb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:54.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.466+0000 7f534e800640 1 --2- 172.21.15.83:0/2655523757 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f53300ac640 0x7f53300aeb00 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f5340009620 tx=0x7f534002a8e0 comp rx=0 tx=0).stop 2024-04-24T09:13:54.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.466+0000 7f534e800640 1 -- 172.21.15.83:0/2655523757 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f53481093f0 msgr2=0x7f53481a0b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:54.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.466+0000 7f534e800640 1 --2- 172.21.15.83:0/2655523757 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f53481093f0 0x7f53481a0b70 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f533c00d6a0 tx=0x7f533c00db60 comp rx=0 tx=0).stop 2024-04-24T09:13:54.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.467+0000 7f534e800640 1 -- 172.21.15.83:0/2655523757 shutdown_connections 2024-04-24T09:13:54.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.467+0000 7f534e800640 1 --2- 172.21.15.83:0/2655523757 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f53300ac640 0x7f53300aeb00 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:54.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.467+0000 7f534e800640 1 --2- 172.21.15.83:0/2655523757 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f534819a7a0 0x7f534819ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:54.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.467+0000 7f534e800640 1 --2- 172.21.15.83:0/2655523757 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f53481093f0 0x7f53481a0b70 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:54.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.467+0000 7f534e800640 1 --2- 172.21.15.83:0/2655523757 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5348104320 0x7f53481a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:54.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.467+0000 7f534e800640 1 -- 172.21.15.83:0/2655523757 >> 172.21.15.83:0/2655523757 conn(0x7f53480ffb40 msgr2=0x7f5348101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:54.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.467+0000 7f534e800640 1 -- 172.21.15.83:0/2655523757 shutdown_connections 2024-04-24T09:13:54.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.467+0000 7f534e800640 1 -- 172.21.15.83:0/2655523757 wait complete. 2024-04-24T09:13:54.490 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:13:54 smithi083 ceph-mon[27907]: pgmap v703: 129 pgs: 129 active+clean; 971 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.7 MiB/s wr, 322 op/s 2024-04-24T09:13:54.490 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:13:54 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/1803289117' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:13:54.493 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T09:13:54.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.759+0000 7f8ee2000640 1 -- 172.21.15.83:0/3618392061 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8edc101770 msgr2=0x7f8edc105c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:54.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.759+0000 7f8ee2000640 1 --2- 172.21.15.83:0/3618392061 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8edc101770 0x7f8edc105c30 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f8ec8009860 tx=0x7f8ec802a3d0 comp rx=0 tx=0).stop 2024-04-24T09:13:54.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.759+0000 7f8ee2000640 1 -- 172.21.15.83:0/3618392061 shutdown_connections 2024-04-24T09:13:54.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.759+0000 7f8ee2000640 1 --2- 172.21.15.83:0/3618392061 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8edc106250 0x7f8edc1066e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:54.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.759+0000 7f8ee2000640 1 --2- 172.21.15.83:0/3618392061 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8edc101770 0x7f8edc105c30 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:54.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.759+0000 7f8ee2000640 1 --2- 172.21.15.83:0/3618392061 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8edc100e50 0x7f8edc101230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:54.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.759+0000 7f8ee2000640 1 -- 172.21.15.83:0/3618392061 >> 172.21.15.83:0/3618392061 conn(0x7f8edc0fc880 msgr2=0x7f8edc0feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:54.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.759+0000 7f8ee2000640 1 -- 172.21.15.83:0/3618392061 shutdown_connections 2024-04-24T09:13:54.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.759+0000 7f8ee2000640 1 -- 172.21.15.83:0/3618392061 wait complete. 2024-04-24T09:13:54.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.761+0000 7f8ee2000640 1 Processor -- start 2024-04-24T09:13:54.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.761+0000 7f8ee2000640 1 -- start start 2024-04-24T09:13:54.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.762+0000 7f8ee2000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8edc100e50 0x7f8edc1a48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:54.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.762+0000 7f8ee2000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8edc101770 0x7f8edc1a4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:54.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.762+0000 7f8ee2000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8edc106250 0x7f8edc19ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:54.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.762+0000 7f8ee2000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8edc1186a0 con 0x7f8edc106250 2024-04-24T09:13:54.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.762+0000 7f8ee2000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8edc118520 con 0x7f8edc100e50 2024-04-24T09:13:54.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.762+0000 7f8ee2000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8edc118820 con 0x7f8edc101770 2024-04-24T09:13:54.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.762+0000 7f8edbe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8edc101770 0x7f8edc1a4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:54.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.763+0000 7f8ee0c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8edc100e50 0x7f8edc1a48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:54.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.763+0000 7f8edbe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8edc101770 0x7f8edc1a4df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:41082/0 (socket says 172.21.15.83:41082) 2024-04-24T09:13:54.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.763+0000 7f8edbe00640 1 -- 172.21.15.83:0/1601540684 learned_addr learned my addr 172.21.15.83:0/1601540684 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:13:54.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.763+0000 7f8ee0c00640 1 -- 172.21.15.83:0/1601540684 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8edc101770 msgr2=0x7f8edc1a4df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:54.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.764+0000 7f8ee0c00640 1 --2- 172.21.15.83:0/1601540684 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8edc101770 0x7f8edc1a4df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:54.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.764+0000 7f8ee0c00640 1 -- 172.21.15.83:0/1601540684 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8edc106250 msgr2=0x7f8edc19ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:54.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.764+0000 7f8ee0c00640 1 --2- 172.21.15.83:0/1601540684 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8edc106250 0x7f8edc19ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:54.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.764+0000 7f8ee0c00640 1 -- 172.21.15.83:0/1601540684 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8edc19f1e0 con 0x7f8edc100e50 2024-04-24T09:13:54.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.764+0000 7f8ee0c00640 1 --2- 172.21.15.83:0/1601540684 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8edc100e50 0x7f8edc1a48b0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f8ed000c970 tx=0x7f8ed000ce30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:54.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.765+0000 7f8ed9600640 1 -- 172.21.15.83:0/1601540684 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed0007bb0 con 0x7f8edc100e50 2024-04-24T09:13:54.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.765+0000 7f8ed9600640 1 -- 172.21.15.83:0/1601540684 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8ed0007d50 con 0x7f8edc100e50 2024-04-24T09:13:54.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.765+0000 7f8ed9600640 1 -- 172.21.15.83:0/1601540684 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed0018760 con 0x7f8edc100e50 2024-04-24T09:13:54.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.765+0000 7f8ee2000640 1 -- 172.21.15.83:0/1601540684 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8edc19f4d0 con 0x7f8edc100e50 2024-04-24T09:13:54.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.766+0000 7f8ee2000640 1 -- 172.21.15.83:0/1601540684 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8edc1ab770 con 0x7f8edc100e50 2024-04-24T09:13:54.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.770+0000 7f8ee6394740 1 Processor -- start 2024-04-24T09:13:54.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.770+0000 7f8ee6394740 1 -- start start 2024-04-24T09:13:54.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.771+0000 7f8ee6394740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cf32a0db40 0x55cf32a0df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:54.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.771+0000 7f8ee6394740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf32a0e460 0x55cf32a185f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:54.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.771+0000 7f8ee6394740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf32a18b30 0x55cf32a1af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:54.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.771+0000 7f8ee6394740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55cf329731d0 con 0x55cf32a18b30 2024-04-24T09:13:54.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.771+0000 7f8ee6394740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55cf32919450 con 0x55cf32a0e460 2024-04-24T09:13:54.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.771+0000 7f8ee6394740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55cf3296f190 con 0x55cf32a0db40 2024-04-24T09:13:54.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.772+0000 7f8ee1600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cf32a0db40 0x55cf32a0df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:54.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.772+0000 7f8ee1600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cf32a0db40 0x55cf32a0df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:41096/0 (socket says 172.21.15.83:41096) 2024-04-24T09:13:54.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.772+0000 7f8ee1600640 1 -- 172.21.15.83:0/3489093980 learned_addr learned my addr 172.21.15.83:0/3489093980 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:13:54.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.772+0000 7f8edbe00640 1 --2- 172.21.15.83:0/3489093980 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf32a18b30 0x55cf32a1af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:54.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.773+0000 7f8ee1600640 1 -- 172.21.15.83:0/3489093980 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf32a0e460 msgr2=0x55cf32a185f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T09:13:54.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.773+0000 7f8ee1600640 1 --2- 172.21.15.83:0/3489093980 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf32a0e460 0x55cf32a185f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:54.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.773+0000 7f8ee1600640 1 -- 172.21.15.83:0/3489093980 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf32a18b30 msgr2=0x55cf32a1af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:54.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.773+0000 7f8ee1600640 1 --2- 172.21.15.83:0/3489093980 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf32a18b30 0x55cf32a1af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:54.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.773+0000 7f8ee1600640 1 -- 172.21.15.83:0/3489093980 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf329d6310 con 0x55cf32a0db40 2024-04-24T09:13:54.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.773+0000 7f8edbe00640 1 --2- 172.21.15.83:0/3489093980 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf32a18b30 0x55cf32a1af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:13:54.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.773+0000 7f8ed9600640 1 -- 172.21.15.83:0/1601540684 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8ed000f450 con 0x7f8edc100e50 2024-04-24T09:13:54.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.774+0000 7f8ee1600640 1 --2- 172.21.15.83:0/3489093980 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cf32a0db40 0x55cf32a0df20 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8ed4005c40 tx=0x7f8ed400e7c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f7bddb07c3fc877 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:54.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.774+0000 7f8ecdc00640 1 -- 172.21.15.83:0/3489093980 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed4003b70 con 0x55cf32a0db40 2024-04-24T09:13:54.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.774+0000 7f8ecdc00640 1 -- 172.21.15.83:0/3489093980 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8ed4003d10 con 0x55cf32a0db40 2024-04-24T09:13:54.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.774+0000 7f8ecdc00640 1 -- 172.21.15.83:0/3489093980 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed400cba0 con 0x55cf32a0db40 2024-04-24T09:13:54.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.776+0000 7f8ee6394740 1 -- 172.21.15.83:0/3489093980 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cf32a0db40 msgr2=0x55cf32a0df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:54.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.776+0000 7f8ee6394740 1 --2- 172.21.15.83:0/3489093980 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cf32a0db40 0x55cf32a0df20 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8ed4005c40 tx=0x7f8ed400e7c0 comp rx=0 tx=0).stop 2024-04-24T09:13:54.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.776+0000 7f8ed9600640 1 --2- 172.21.15.83:0/1601540684 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f8ec40a3e70 0x7f8ec40a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:54.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.777+0000 7f8ee6394740 1 -- 172.21.15.83:0/3489093980 shutdown_connections 2024-04-24T09:13:54.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.777+0000 7f8ee6394740 1 --2- 172.21.15.83:0/3489093980 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf32a18b30 0x55cf32a1af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:54.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.777+0000 7f8ee6394740 1 --2- 172.21.15.83:0/3489093980 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf32a0e460 0x55cf32a185f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:54.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.777+0000 7f8ee6394740 1 --2- 172.21.15.83:0/3489093980 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cf32a0db40 0x55cf32a0df20 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:54.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.777+0000 7f8ee6394740 1 -- 172.21.15.83:0/3489093980 >> 172.21.15.83:0/3489093980 conn(0x55cf32a09e60 msgr2=0x55cf32a0a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:54.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.777+0000 7f8ee6394740 1 -- 172.21.15.83:0/3489093980 shutdown_connections 2024-04-24T09:13:54.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.777+0000 7f8ee1600640 1 --2- 172.21.15.83:0/1601540684 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f8ec40a3e70 0x7f8ec40a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:54.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.777+0000 7f8ed9600640 1 -- 172.21.15.83:0/1601540684 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 9583+0+0 (secure 0 0 0) 0x7f8ed00d0bb0 con 0x7f8edc100e50 2024-04-24T09:13:54.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.778+0000 7f8ee6394740 1 -- 172.21.15.83:0/3489093980 wait complete. 2024-04-24T09:13:54.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.778+0000 7f8ee1600640 1 --2- 172.21.15.83:0/1601540684 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f8ec40a3e70 0x7f8ec40a6330 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f8ed400c800 tx=0x7f8ed4004a40 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:54.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.779+0000 7f8ee6394740 1 -- start start 2024-04-24T09:13:54.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.779+0000 7f8ee6394740 1 Processor -- start 2024-04-24T09:13:54.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.779+0000 7f8ee6394740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf32a0db40 0x55cf32a185e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:54.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.779+0000 7f8ee6394740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cf32a0e460 0x55cf32a158d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:54.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.779+0000 7f8ee6394740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf32a18b30 0x55cf32a16010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:54.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.779+0000 7f8ee6394740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55cf32a1b9c0 con 0x55cf32a18b30 2024-04-24T09:13:54.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.779+0000 7f8ee6394740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55cf3292c810 con 0x55cf32a0db40 2024-04-24T09:13:54.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.779+0000 7f8ee6394740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55cf32a1bb40 con 0x55cf32a0e460 2024-04-24T09:13:54.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.780+0000 7f8ee6394740 10 client.0 _ll_register_callbacks cb 0x55cf328fa140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:13:54.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.780+0000 7f8ee1600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf32a18b30 0x55cf32a16010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:54.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.780+0000 7f8ee0c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf32a0db40 0x55cf32a185e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:54.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.780+0000 7f8ee0c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf32a0db40 0x55cf32a185e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:60760/0 (socket says 172.21.15.83:60760) 2024-04-24T09:13:54.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.780+0000 7f8ee0c00640 1 -- 172.21.15.83:0/1416984565 learned_addr learned my addr 172.21.15.83:0/1416984565 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:13:54.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.780+0000 7f8edbe00640 1 --2- 172.21.15.83:0/1416984565 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cf32a0e460 0x55cf32a158d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:54.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.781+0000 7f8ee1600640 1 -- 172.21.15.83:0/1416984565 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cf32a0e460 msgr2=0x55cf32a158d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:54.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.781+0000 7f8ee1600640 1 --2- 172.21.15.83:0/1416984565 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cf32a0e460 0x55cf32a158d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:54.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.781+0000 7f8ee1600640 1 -- 172.21.15.83:0/1416984565 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf32a0db40 msgr2=0x55cf32a185e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:54.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.781+0000 7f8ee1600640 1 --2- 172.21.15.83:0/1416984565 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf32a0db40 0x55cf32a185e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:54.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.781+0000 7f8ee1600640 1 -- 172.21.15.83:0/1416984565 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf3293d3f0 con 0x55cf32a18b30 2024-04-24T09:13:54.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.781+0000 7f8edbe00640 1 --2- 172.21.15.83:0/1416984565 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cf32a0e460 0x55cf32a158d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:13:54.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.781+0000 7f8ee1600640 1 --2- 172.21.15.83:0/1416984565 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf32a18b30 0x55cf32a16010 secure :-1 s=READY pgs=2639 cs=0 l=1 rev1=1 crypto rx=0x7f8ed4009840 tx=0x7f8ed400c700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2898b463e073fdd9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:54.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.781+0000 7f8ec2000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:13:54.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.781+0000 7f8ec2000640 1 -- 172.21.15.83:0/1416984565 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed4030070 con 0x55cf32a18b30 2024-04-24T09:13:54.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.782+0000 7f8ec2000640 1 -- 172.21.15.83:0/1416984565 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9583+0+0 (secure 0 0 0) 0x7f8ed4022650 con 0x55cf32a18b30 2024-04-24T09:13:54.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.782+0000 7f8ee6394740 1 -- 172.21.15.83:0/1416984565 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ed4003d10 con 0x55cf32a18b30 2024-04-24T09:13:54.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.782+0000 7f8ec2000640 1 -- 172.21.15.83:0/1416984565 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f8ed403c050 con 0x55cf32a18b30 2024-04-24T09:13:54.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.782+0000 7f8ee6394740 10 client.23178 fetch_fsmap learned FSMap version 64 2024-04-24T09:13:54.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.782+0000 7f8ee6394740 1 -- 172.21.15.83:0/1416984565 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55cf326853f0 con 0x55cf32a18b30 2024-04-24T09:13:54.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.782+0000 7f8ec2000640 1 -- 172.21.15.83:0/1416984565 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8ed4003d10 con 0x55cf32a18b30 2024-04-24T09:13:54.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.783+0000 7f8ec2000640 1 -- 172.21.15.83:0/1416984565 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 64) ==== 8374+0+0 (secure 0 0 0) 0x7f8ed402b070 con 0x55cf32a18b30 2024-04-24T09:13:54.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.783+0000 7f8ee6394740 10 client.23178 fetch_fsmap finished waiting for FSMap version 64 2024-04-24T09:13:54.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.783+0000 7f8ee6394740 10 client.23178 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-24T09:13:54.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.783+0000 7f8ee6394740 20 client.23178 populate_metadata read hostname 'smithi083' 2024-04-24T09:13:54.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.783+0000 7f8ee6394740 1 --2- 172.21.15.83:0/1416984565 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55cf32a28d00 0x55cf32a2b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:54.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.783+0000 7f8ee6394740 4 client.23178 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:13:54.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.783+0000 7f8ee6394740 1 -- 172.21.15.83:0/1416984565 --> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cf32980bf0 con 0x55cf32a28d00 2024-04-24T09:13:54.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.784+0000 7f8ee0c00640 1 --2- 172.21.15.83:0/1416984565 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55cf32a28d00 0x55cf32a2b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:54.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.787+0000 7f8ee0c00640 1 --2- 172.21.15.83:0/1416984565 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55cf32a28d00 0x55cf32a2b0f0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a2c185e8937962d8 server_cookie=24bb552bfabccbb2 in_seq=0 out_seq=0 2024-04-24T09:13:54.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.787+0000 7f8ec2000640 10 client.23178 ms_handle_connect on v2:172.21.15.138:6834/1909219758 2024-04-24T09:13:54.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.789+0000 7f8ec2000640 1 -- 172.21.15.83:0/1416984565 <== mds.2 v2:172.21.15.138:6834/1909219758 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cf32980bf0 con 0x55cf32a28d00 2024-04-24T09:13:54.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.789+0000 7f8ec2000640 10 client.23178 handle_command_reply: tid=0 2024-04-24T09:13:54.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.789+0000 7f8ee6394740 1 client.23178 shutdown 2024-04-24T09:13:54.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.789+0000 7f8ee6394740 1 -- 172.21.15.83:0/1416984565 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf32a18b30 msgr2=0x55cf32a16010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:54.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.789+0000 7f8ee6394740 1 --2- 172.21.15.83:0/1416984565 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf32a18b30 0x55cf32a16010 secure :-1 s=READY pgs=2639 cs=0 l=1 rev1=1 crypto rx=0x7f8ed4009840 tx=0x7f8ed400c700 comp rx=0 tx=0).stop 2024-04-24T09:13:54.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.790+0000 7f8ee6394740 1 -- 172.21.15.83:0/1416984565 shutdown_connections 2024-04-24T09:13:54.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.790+0000 7f8ee6394740 1 --2- 172.21.15.83:0/1416984565 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55cf32a28d00 0x55cf32a2b0f0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:54.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.790+0000 7f8ee6394740 1 --2- 172.21.15.83:0/1416984565 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf32a18b30 0x55cf32a16010 unknown :-1 s=CLOSED pgs=2639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:54.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.790+0000 7f8ee6394740 1 --2- 172.21.15.83:0/1416984565 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cf32a0e460 0x55cf32a158d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:54.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.790+0000 7f8ee6394740 1 --2- 172.21.15.83:0/1416984565 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf32a0db40 0x55cf32a185e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:54.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.790+0000 7f8ee6394740 1 -- 172.21.15.83:0/1416984565 >> 172.21.15.83:0/1416984565 conn(0x55cf32a09e60 msgr2=0x55cf32a0fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:54.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.790+0000 7f8ec2000640 0 client.23178 ms_handle_reset on v2:172.21.15.138:6834/1909219758 2024-04-24T09:13:54.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.792+0000 7f8ee6394740 1 -- 172.21.15.83:0/1416984565 shutdown_connections 2024-04-24T09:13:54.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.792+0000 7f8ee6394740 1 -- 172.21.15.83:0/1416984565 wait complete. 2024-04-24T09:13:54.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.793+0000 7f8ee6394740 20 client.23178 trim_cache size 0 max 16384 2024-04-24T09:13:54.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.825+0000 7f8ee6394740 1 Processor -- start 2024-04-24T09:13:54.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.825+0000 7f8ee6394740 1 -- start start 2024-04-24T09:13:54.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.825+0000 7f8ee6394740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf32a0db40 0x55cf32a248e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:54.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.825+0000 7f8ee6394740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf32a0e460 0x55cf32a18110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:54.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.825+0000 7f8ee6394740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cf32a18b30 0x55cf32a30880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:54.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.825+0000 7f8ee6394740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8edc1b5a80 con 0x55cf32a0db40 2024-04-24T09:13:54.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.825+0000 7f8ee6394740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8edc1b7650 con 0x55cf32a0e460 2024-04-24T09:13:54.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.825+0000 7f8ee6394740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8edc1b6150 con 0x55cf32a18b30 2024-04-24T09:13:54.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.826+0000 7f8edbe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf32a0e460 0x55cf32a18110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:54.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.826+0000 7f8ee1600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cf32a18b30 0x55cf32a30880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:54.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.827+0000 7f8edbe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf32a0e460 0x55cf32a18110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:60762/0 (socket says 172.21.15.83:60762) 2024-04-24T09:13:54.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.827+0000 7f8edbe00640 1 -- 172.21.15.83:0/485676579 learned_addr learned my addr 172.21.15.83:0/485676579 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:13:54.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.827+0000 7f8ee0c00640 1 --2- 172.21.15.83:0/485676579 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf32a0db40 0x55cf32a248e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:54.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.828+0000 7f8edbe00640 1 -- 172.21.15.83:0/485676579 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cf32a18b30 msgr2=0x55cf32a30880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:54.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.828+0000 7f8edbe00640 1 --2- 172.21.15.83:0/485676579 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cf32a18b30 0x55cf32a30880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:54.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.828+0000 7f8edbe00640 1 -- 172.21.15.83:0/485676579 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf32a0db40 msgr2=0x55cf32a248e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:54.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.828+0000 7f8edbe00640 1 --2- 172.21.15.83:0/485676579 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf32a0db40 0x55cf32a248e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:54.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.828+0000 7f8edbe00640 1 -- 172.21.15.83:0/485676579 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf32a0e290 con 0x55cf32a0e460 2024-04-24T09:13:54.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.829+0000 7f8edbe00640 1 --2- 172.21.15.83:0/485676579 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf32a0e460 0x55cf32a18110 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f8ec802a8f0 tx=0x7f8ec80306a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9fcc6731c9e6d9e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:54.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.831+0000 7f8ecd200640 1 -- 172.21.15.83:0/485676579 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec8004050 con 0x55cf32a0e460 2024-04-24T09:13:54.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.832+0000 7f8ecd200640 1 -- 172.21.15.83:0/485676579 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8ec8030e20 con 0x55cf32a0e460 2024-04-24T09:13:54.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.832+0000 7f8ecd200640 1 -- 172.21.15.83:0/485676579 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec80055d0 con 0x55cf32a0e460 2024-04-24T09:13:54.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.832+0000 7f8ee6394740 1 -- 172.21.15.83:0/485676579 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf32a0e460 msgr2=0x55cf32a18110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:54.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.832+0000 7f8ee6394740 1 --2- 172.21.15.83:0/485676579 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf32a0e460 0x55cf32a18110 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f8ec802a8f0 tx=0x7f8ec80306a0 comp rx=0 tx=0).stop 2024-04-24T09:13:54.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.833+0000 7f8ee6394740 1 -- 172.21.15.83:0/485676579 shutdown_connections 2024-04-24T09:13:54.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.833+0000 7f8ee6394740 1 --2- 172.21.15.83:0/485676579 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cf32a18b30 0x55cf32a30880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:54.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.834+0000 7f8ee6394740 1 --2- 172.21.15.83:0/485676579 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf32a0e460 0x55cf32a18110 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:54.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.834+0000 7f8ee6394740 1 --2- 172.21.15.83:0/485676579 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf32a0db40 0x55cf32a248e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:54.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.834+0000 7f8ee6394740 1 -- 172.21.15.83:0/485676579 >> 172.21.15.83:0/485676579 conn(0x55cf32a09e60 msgr2=0x55cf32a14900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:54.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.834+0000 7f8ee6394740 1 -- 172.21.15.83:0/485676579 shutdown_connections 2024-04-24T09:13:54.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.835+0000 7f8ee6394740 1 -- 172.21.15.83:0/485676579 wait complete. 2024-04-24T09:13:54.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.835+0000 7f8ee6394740 1 -- start start 2024-04-24T09:13:54.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.836+0000 7f8ee6394740 1 Processor -- start 2024-04-24T09:13:54.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.837+0000 7f8ee6394740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf32a0db40 0x55cf32a135c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:54.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.838+0000 7f8ee6394740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf32a0e460 0x55cf32a20df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:54.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.838+0000 7f8ee6394740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cf32a18b30 0x55cf32a19ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:54.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.838+0000 7f8ee6394740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8edc1b6f50 con 0x55cf32a0db40 2024-04-24T09:13:54.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.838+0000 7f8ee6394740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8edc1b6880 con 0x55cf32a0e460 2024-04-24T09:13:54.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.838+0000 7f8ee6394740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8edc1b3a80 con 0x55cf32a18b30 2024-04-24T09:13:54.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.838+0000 7f8ee6394740 10 client.0 _ll_register_callbacks cb 0x7f8edc06a1f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:13:54.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.838+0000 7f8edbe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf32a0e460 0x55cf32a20df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:54.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.838+0000 7f8edbe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf32a0e460 0x55cf32a20df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:60766/0 (socket says 172.21.15.83:60766) 2024-04-24T09:13:54.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.838+0000 7f8edbe00640 1 -- 172.21.15.83:0/3729869695 learned_addr learned my addr 172.21.15.83:0/3729869695 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:13:54.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.839+0000 7f8ee0c00640 1 --2- 172.21.15.83:0/3729869695 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf32a0db40 0x55cf32a135c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:54.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.839+0000 7f8ee1600640 1 --2- 172.21.15.83:0/3729869695 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cf32a18b30 0x55cf32a19ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:54.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.839+0000 7f8edbe00640 1 -- 172.21.15.83:0/3729869695 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cf32a18b30 msgr2=0x55cf32a19ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:54.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.839+0000 7f8edbe00640 1 --2- 172.21.15.83:0/3729869695 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cf32a18b30 0x55cf32a19ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:54.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.840+0000 7f8edbe00640 1 -- 172.21.15.83:0/3729869695 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf32a0db40 msgr2=0x55cf32a135c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:54.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.840+0000 7f8edbe00640 1 --2- 172.21.15.83:0/3729869695 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf32a0db40 0x55cf32a135c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:54.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.840+0000 7f8edbe00640 1 -- 172.21.15.83:0/3729869695 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf3274f7d0 con 0x55cf32a0e460 2024-04-24T09:13:54.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.840+0000 7f8ee0c00640 1 --2- 172.21.15.83:0/3729869695 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf32a0db40 0x55cf32a135c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:13:54.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.841+0000 7f8edbe00640 1 --2- 172.21.15.83:0/3729869695 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf32a0e460 0x55cf32a20df0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f8ec8045000 tx=0x7f8ec8030850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4fd27613c543b642 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:54.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.841+0000 7f8ec2000640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:13:54.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.841+0000 7f8ec2000640 1 -- 172.21.15.83:0/3729869695 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec8004320 con 0x55cf32a0e460 2024-04-24T09:13:54.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.841+0000 7f8ec2000640 1 -- 172.21.15.83:0/3729869695 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9583+0+0 (secure 0 0 0) 0x7f8ec80051d0 con 0x55cf32a0e460 2024-04-24T09:13:54.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.841+0000 7f8ee6394740 1 -- 172.21.15.83:0/3729869695 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ec8030e20 con 0x55cf32a0e460 2024-04-24T09:13:54.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.842+0000 7f8ec2000640 1 -- 172.21.15.83:0/3729869695 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f8ec8059050 con 0x55cf32a0e460 2024-04-24T09:13:54.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.842+0000 7f8ee6394740 10 client.40256 fetch_fsmap learned FSMap version 64 2024-04-24T09:13:54.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.842+0000 7f8ee6394740 1 -- 172.21.15.83:0/3729869695 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55cf32a8f600 con 0x55cf32a0e460 2024-04-24T09:13:54.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.843+0000 7f8ec2000640 1 -- 172.21.15.83:0/3729869695 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8ec8030e20 con 0x55cf32a0e460 2024-04-24T09:13:54.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.844+0000 7f8ec2000640 1 -- 172.21.15.83:0/3729869695 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 64) ==== 8374+0+0 (secure 0 0 0) 0x7f8ec80046c0 con 0x55cf32a0e460 2024-04-24T09:13:54.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.844+0000 7f8ee6394740 10 client.40256 fetch_fsmap finished waiting for FSMap version 64 2024-04-24T09:13:54.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.844+0000 7f8ee6394740 10 client.40256 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-24T09:13:54.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.844+0000 7f8ee6394740 20 client.40256 populate_metadata read hostname 'smithi083' 2024-04-24T09:13:54.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.844+0000 7f8ee6394740 1 --2- 172.21.15.83:0/3729869695 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55cf32a8fcd0 0x55cf32a920f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:54.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.844+0000 7f8ee6394740 4 client.40256 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:13:54.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.844+0000 7f8ee6394740 1 -- 172.21.15.83:0/3729869695 --> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cf3297aa60 con 0x55cf32a8fcd0 2024-04-24T09:13:54.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.845+0000 7f8ee0c00640 1 --2- 172.21.15.83:0/3729869695 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55cf32a8fcd0 0x55cf32a920f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:54.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.846+0000 7f8ee0c00640 1 --2- 172.21.15.83:0/3729869695 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55cf32a8fcd0 0x55cf32a920f0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=186a125156095ac1 server_cookie=5ef1a471990146f1 in_seq=0 out_seq=0 2024-04-24T09:13:54.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.846+0000 7f8ec2000640 10 client.40256 ms_handle_connect on v2:172.21.15.138:6834/1909219758 2024-04-24T09:13:54.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.846+0000 7f8ec2000640 1 -- 172.21.15.83:0/3729869695 <== mds.2 v2:172.21.15.138:6834/1909219758 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cf3297aa60 con 0x55cf32a8fcd0 2024-04-24T09:13:54.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.846+0000 7f8ec2000640 10 client.40256 handle_command_reply: tid=0 2024-04-24T09:13:54.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.846+0000 7f8ee6394740 1 client.40256 shutdown 2024-04-24T09:13:54.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.847+0000 7f8ee6394740 1 -- 172.21.15.83:0/3729869695 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf32a0e460 msgr2=0x55cf32a20df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:54.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.847+0000 7f8ee6394740 1 --2- 172.21.15.83:0/3729869695 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf32a0e460 0x55cf32a20df0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f8ec8045000 tx=0x7f8ec8030850 comp rx=0 tx=0).stop 2024-04-24T09:13:54.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.847+0000 7f8ee6394740 1 -- 172.21.15.83:0/3729869695 shutdown_connections 2024-04-24T09:13:54.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.847+0000 7f8ee6394740 1 --2- 172.21.15.83:0/3729869695 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55cf32a8fcd0 0x55cf32a920f0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:54.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.847+0000 7f8ec2000640 0 client.40256 ms_handle_reset on v2:172.21.15.138:6834/1909219758 2024-04-24T09:13:54.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.847+0000 7f8ee6394740 1 --2- 172.21.15.83:0/3729869695 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cf32a18b30 0x55cf32a19ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:54.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.847+0000 7f8ee6394740 1 --2- 172.21.15.83:0/3729869695 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf32a0e460 0x55cf32a20df0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:54.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.847+0000 7f8ee6394740 1 --2- 172.21.15.83:0/3729869695 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf32a0db40 0x55cf32a135c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:54.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.847+0000 7f8ee6394740 1 -- 172.21.15.83:0/3729869695 >> 172.21.15.83:0/3729869695 conn(0x55cf32a09e60 msgr2=0x55cf32a086c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:54.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.848+0000 7f8ee6394740 1 -- 172.21.15.83:0/3729869695 shutdown_connections 2024-04-24T09:13:54.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.848+0000 7f8ee6394740 1 -- 172.21.15.83:0/3729869695 wait complete. 2024-04-24T09:13:54.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.849+0000 7f8ee6394740 20 client.40256 trim_cache size 0 max 16384 2024-04-24T09:13:54.849 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:13:54.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.850+0000 7f8ee2000640 1 -- 172.21.15.83:0/1601540684 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f8ec40a3e70 msgr2=0x7f8ec40a6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:54.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.850+0000 7f8ee2000640 1 --2- 172.21.15.83:0/1601540684 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f8ec40a3e70 0x7f8ec40a6330 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f8ed400c800 tx=0x7f8ed4004a40 comp rx=0 tx=0).stop 2024-04-24T09:13:54.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.850+0000 7f8ee2000640 1 -- 172.21.15.83:0/1601540684 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8edc100e50 msgr2=0x7f8edc1a48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:54.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.850+0000 7f8ee2000640 1 --2- 172.21.15.83:0/1601540684 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8edc100e50 0x7f8edc1a48b0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f8ed000c970 tx=0x7f8ed000ce30 comp rx=0 tx=0).stop 2024-04-24T09:13:54.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.850+0000 7f8ee2000640 1 -- 172.21.15.83:0/1601540684 shutdown_connections 2024-04-24T09:13:54.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.850+0000 7f8ee2000640 1 --2- 172.21.15.83:0/1601540684 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f8ec40a3e70 0x7f8ec40a6330 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:54.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.850+0000 7f8ee2000640 1 --2- 172.21.15.83:0/1601540684 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8edc106250 0x7f8edc19ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:54.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.850+0000 7f8ee2000640 1 --2- 172.21.15.83:0/1601540684 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8edc101770 0x7f8edc1a4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:54.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.850+0000 7f8ee2000640 1 --2- 172.21.15.83:0/1601540684 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8edc100e50 0x7f8edc1a48b0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:54.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.850+0000 7f8ee2000640 1 -- 172.21.15.83:0/1601540684 >> 172.21.15.83:0/1601540684 conn(0x7f8edc0fc880 msgr2=0x7f8edc0fd8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:54.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.850+0000 7f8ee2000640 1 -- 172.21.15.83:0/1601540684 shutdown_connections 2024-04-24T09:13:54.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:54.850+0000 7f8ee2000640 1 -- 172.21.15.83:0/1601540684 wait complete. 2024-04-24T09:13:54.872 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T09:13:55.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.132+0000 7f21b9400640 1 -- 172.21.15.83:0/3777898118 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21b4030d20 con 0x7f21bc10c480 2024-04-24T09:13:55.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.132+0000 7f21bb200640 1 -- 172.21.15.83:0/3777898118 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f21bc10c480 msgr2=0x7f21bc10e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:55.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.132+0000 7f21bb200640 1 --2- 172.21.15.83:0/3777898118 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f21bc10c480 0x7f21bc10e920 secure :-1 s=READY pgs=2640 cs=0 l=1 rev1=1 crypto rx=0x7f21b4009be0 tx=0x7f21b402a750 comp rx=0 tx=0).stop 2024-04-24T09:13:55.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.133+0000 7f21bb200640 1 -- 172.21.15.83:0/3777898118 shutdown_connections 2024-04-24T09:13:55.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.133+0000 7f21bb200640 1 --2- 172.21.15.83:0/3777898118 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f21bc10c480 0x7f21bc10e920 unknown :-1 s=CLOSED pgs=2640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:55.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.133+0000 7f21bb200640 1 --2- 172.21.15.83:0/3777898118 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f21bc103980 0x7f21bc10be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:55.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.133+0000 7f21bb200640 1 --2- 172.21.15.83:0/3777898118 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f21bc103060 0x7f21bc103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:55.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.133+0000 7f21bb200640 1 -- 172.21.15.83:0/3777898118 >> 172.21.15.83:0/3777898118 conn(0x7f21bc0fc880 msgr2=0x7f21bc0feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:55.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.133+0000 7f21bb200640 1 -- 172.21.15.83:0/3777898118 shutdown_connections 2024-04-24T09:13:55.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.133+0000 7f21bb200640 1 -- 172.21.15.83:0/3777898118 wait complete. 2024-04-24T09:13:55.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.134+0000 7f21bb200640 1 Processor -- start 2024-04-24T09:13:55.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.135+0000 7f21bb200640 1 -- start start 2024-04-24T09:13:55.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.135+0000 7f21bb200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f21bc103060 0x7f21bc10b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:55.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.135+0000 7f21bb200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f21bc103980 0x7f21bc109b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:55.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.135+0000 7f21bb200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f21bc10c480 0x7f21bc10a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:55.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.135+0000 7f21bb200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f21bc10fdc0 con 0x7f21bc103060 2024-04-24T09:13:55.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.135+0000 7f21bb200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f21bc10fc40 con 0x7f21bc10c480 2024-04-24T09:13:55.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.135+0000 7f21bb200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f21bc10ff40 con 0x7f21bc103980 2024-04-24T09:13:55.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.136+0000 7f21ba800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f21bc10c480 0x7f21bc10a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:55.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.136+0000 7f21ba800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f21bc10c480 0x7f21bc10a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:60784/0 (socket says 172.21.15.83:60784) 2024-04-24T09:13:55.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.136+0000 7f21ba800640 1 -- 172.21.15.83:0/2920021818 learned_addr learned my addr 172.21.15.83:0/2920021818 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:13:55.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.136+0000 7f21b9e00640 1 --2- 172.21.15.83:0/2920021818 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f21bc103060 0x7f21bc10b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:55.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.137+0000 7f21b3e00640 1 --2- 172.21.15.83:0/2920021818 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f21bc103980 0x7f21bc109b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:55.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.137+0000 7f21ba800640 1 -- 172.21.15.83:0/2920021818 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f21bc103980 msgr2=0x7f21bc109b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:55.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.137+0000 7f21ba800640 1 --2- 172.21.15.83:0/2920021818 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f21bc103980 0x7f21bc109b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:55.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.137+0000 7f21ba800640 1 -- 172.21.15.83:0/2920021818 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f21bc103060 msgr2=0x7f21bc10b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:55.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.137+0000 7f21ba800640 1 --2- 172.21.15.83:0/2920021818 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f21bc103060 0x7f21bc10b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:55.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.137+0000 7f21ba800640 1 -- 172.21.15.83:0/2920021818 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21bc10a6a0 con 0x7f21bc10c480 2024-04-24T09:13:55.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.137+0000 7f21b3e00640 1 --2- 172.21.15.83:0/2920021818 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f21bc103980 0x7f21bc109b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:13:55.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.137+0000 7f21ba800640 1 --2- 172.21.15.83:0/2920021818 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f21bc10c480 0x7f21bc10a060 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f21b4005cc0 tx=0x7f21b4003c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:55.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.138+0000 7f21b2a00640 1 -- 172.21.15.83:0/2920021818 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21b4041020 con 0x7f21bc10c480 2024-04-24T09:13:55.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.138+0000 7f21bb200640 1 -- 172.21.15.83:0/2920021818 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21bc10a930 con 0x7f21bc10c480 2024-04-24T09:13:55.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.138+0000 7f21b2a00640 1 -- 172.21.15.83:0/2920021818 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f21b4037040 con 0x7f21bc10c480 2024-04-24T09:13:55.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.138+0000 7f21b2a00640 1 -- 172.21.15.83:0/2920021818 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21b4036070 con 0x7f21bc10c480 2024-04-24T09:13:55.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.138+0000 7f21bb200640 1 -- 172.21.15.83:0/2920021818 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21bc1ab6f0 con 0x7f21bc10c480 2024-04-24T09:13:55.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.142+0000 7f21c37fd740 1 Processor -- start 2024-04-24T09:13:55.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.144+0000 7f21b2a00640 1 -- 172.21.15.83:0/2920021818 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f21b4044020 con 0x7f21bc10c480 2024-04-24T09:13:55.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.144+0000 7f21c37fd740 1 -- start start 2024-04-24T09:13:55.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.144+0000 7f21c37fd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642eda8bb40 0x5642eda8bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:55.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.144+0000 7f21c37fd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642eda8c460 0x5642eda965f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:55.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.144+0000 7f21c37fd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5642eda96b30 0x5642eda98f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:55.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.144+0000 7f21c37fd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5642ed9f11d0 con 0x5642eda8bb40 2024-04-24T09:13:55.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.144+0000 7f21c37fd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5642ed997450 con 0x5642eda8c460 2024-04-24T09:13:55.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.145+0000 7f21c37fd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5642ed9ed190 con 0x5642eda96b30 2024-04-24T09:13:55.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.145+0000 7f21b9e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642eda8c460 0x5642eda965f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:55.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.145+0000 7f21b3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642eda8bb40 0x5642eda8bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:55.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.145+0000 7f21b9e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642eda8c460 0x5642eda965f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:60786/0 (socket says 172.21.15.83:60786) 2024-04-24T09:13:55.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.145+0000 7f21b9e00640 1 -- 172.21.15.83:0/2155200694 learned_addr learned my addr 172.21.15.83:0/2155200694 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:13:55.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.146+0000 7f21b3e00640 1 --2- 172.21.15.83:0/2155200694 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5642eda96b30 0x5642eda98f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:55.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.146+0000 7f21b9e00640 1 -- 172.21.15.83:0/2155200694 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5642eda96b30 msgr2=0x5642eda98f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:55.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.146+0000 7f21b9e00640 1 --2- 172.21.15.83:0/2155200694 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5642eda96b30 0x5642eda98f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:55.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.146+0000 7f21b9e00640 1 -- 172.21.15.83:0/2155200694 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642eda8bb40 msgr2=0x5642eda8bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:55.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.146+0000 7f21b9e00640 1 --2- 172.21.15.83:0/2155200694 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642eda8bb40 0x5642eda8bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:55.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.146+0000 7f21b9e00640 1 -- 172.21.15.83:0/2155200694 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642eda54310 con 0x5642eda8c460 2024-04-24T09:13:55.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.146+0000 7f21b3e00640 1 --2- 172.21.15.83:0/2155200694 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642eda8bb40 0x5642eda8bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:13:55.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.146+0000 7f21b2a00640 1 --2- 172.21.15.83:0/2920021818 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f21a00a3da0 0x7f21a00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:55.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.146+0000 7f21b3e00640 1 --2- 172.21.15.83:0/2920021818 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f21a00a3da0 0x7f21a00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:55.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.147+0000 7f21b2a00640 1 -- 172.21.15.83:0/2920021818 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 9583+0+0 (secure 0 0 0) 0x7f21b40f1470 con 0x7f21bc10c480 2024-04-24T09:13:55.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.147+0000 7f21b9e00640 1 --2- 172.21.15.83:0/2155200694 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642eda8c460 0x5642eda965f0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f21ac009cd0 tx=0x7f21ac011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73dbe4a88394114b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:55.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.147+0000 7f21b3e00640 1 --2- 172.21.15.83:0/2920021818 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f21a00a3da0 0x7f21a00a6260 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f21a4009fd0 tx=0x7f21a40095d0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:55.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.147+0000 7f21ab400640 1 -- 172.21.15.83:0/2155200694 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21ac01e070 con 0x5642eda8c460 2024-04-24T09:13:55.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.148+0000 7f21ab400640 1 -- 172.21.15.83:0/2155200694 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f21ac019ae0 con 0x5642eda8c460 2024-04-24T09:13:55.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.148+0000 7f21ab400640 1 -- 172.21.15.83:0/2155200694 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21ac00ea20 con 0x5642eda8c460 2024-04-24T09:13:55.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.148+0000 7f21c37fd740 1 -- 172.21.15.83:0/2155200694 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642eda8c460 msgr2=0x5642eda965f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:55.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.148+0000 7f21c37fd740 1 --2- 172.21.15.83:0/2155200694 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642eda8c460 0x5642eda965f0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f21ac009cd0 tx=0x7f21ac011810 comp rx=0 tx=0).stop 2024-04-24T09:13:55.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.149+0000 7f21c37fd740 1 -- 172.21.15.83:0/2155200694 shutdown_connections 2024-04-24T09:13:55.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.149+0000 7f21c37fd740 1 --2- 172.21.15.83:0/2155200694 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5642eda96b30 0x5642eda98f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:55.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.149+0000 7f21c37fd740 1 --2- 172.21.15.83:0/2155200694 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642eda8c460 0x5642eda965f0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:55.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.149+0000 7f21c37fd740 1 --2- 172.21.15.83:0/2155200694 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642eda8bb40 0x5642eda8bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:55.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.149+0000 7f21c37fd740 1 -- 172.21.15.83:0/2155200694 >> 172.21.15.83:0/2155200694 conn(0x5642eda87e60 msgr2=0x5642eda88240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:55.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.149+0000 7f21c37fd740 1 -- 172.21.15.83:0/2155200694 shutdown_connections 2024-04-24T09:13:55.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.149+0000 7f21c37fd740 1 -- 172.21.15.83:0/2155200694 wait complete. 2024-04-24T09:13:55.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.149+0000 7f21c37fd740 1 -- start start 2024-04-24T09:13:55.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.149+0000 7f21c37fd740 1 Processor -- start 2024-04-24T09:13:55.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.150+0000 7f21c37fd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642eda8bb40 0x5642eda96560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:55.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.150+0000 7f21c37fd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642eda8c460 0x5642eda93850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:55.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.150+0000 7f21c37fd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5642eda96b30 0x5642eda93f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:55.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.150+0000 7f21c37fd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5642edaaa9d0 con 0x5642eda8bb40 2024-04-24T09:13:55.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.150+0000 7f21c37fd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5642ed9aa810 con 0x5642eda8c460 2024-04-24T09:13:55.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.150+0000 7f21c37fd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5642edaaab50 con 0x5642eda96b30 2024-04-24T09:13:55.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.150+0000 7f21c37fd740 10 client.0 _ll_register_callbacks cb 0x5642ed978140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:13:55.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.150+0000 7f21b9e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642eda8bb40 0x5642eda96560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:55.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.151+0000 7f21b9e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642eda8bb40 0x5642eda96560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45104/0 (socket says 172.21.15.83:45104) 2024-04-24T09:13:55.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.151+0000 7f21b9e00640 1 -- 172.21.15.83:0/457130898 learned_addr learned my addr 172.21.15.83:0/457130898 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:13:55.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.151+0000 7f21ba800640 1 --2- 172.21.15.83:0/457130898 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5642eda96b30 0x5642eda93f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:55.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.151+0000 7f21b3e00640 1 --2- 172.21.15.83:0/457130898 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642eda8c460 0x5642eda93850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:55.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.151+0000 7f21b9e00640 1 -- 172.21.15.83:0/457130898 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5642eda96b30 msgr2=0x5642eda93f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:55.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.151+0000 7f21b9e00640 1 --2- 172.21.15.83:0/457130898 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5642eda96b30 0x5642eda93f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:55.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.151+0000 7f21b9e00640 1 -- 172.21.15.83:0/457130898 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642eda8c460 msgr2=0x5642eda93850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:55.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.151+0000 7f21b9e00640 1 --2- 172.21.15.83:0/457130898 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642eda8c460 0x5642eda93850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:55.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.151+0000 7f21b9e00640 1 -- 172.21.15.83:0/457130898 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642ed9bb3f0 con 0x5642eda8bb40 2024-04-24T09:13:55.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.151+0000 7f21ba800640 1 --2- 172.21.15.83:0/457130898 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5642eda96b30 0x5642eda93f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:13:55.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.152+0000 7f21b9e00640 1 --2- 172.21.15.83:0/457130898 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642eda8bb40 0x5642eda96560 secure :-1 s=READY pgs=2641 cs=0 l=1 rev1=1 crypto rx=0x7f21ac009690 tx=0x7f21ac005df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f8d57a712b67f59 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:55.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.152+0000 7f219fe00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:13:55.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.152+0000 7f219fe00640 1 -- 172.21.15.83:0/457130898 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21ac01e050 con 0x5642eda8bb40 2024-04-24T09:13:55.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.152+0000 7f219fe00640 1 -- 172.21.15.83:0/457130898 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9583+0+0 (secure 0 0 0) 0x7f21ac005450 con 0x5642eda8bb40 2024-04-24T09:13:55.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.152+0000 7f21c37fd740 1 -- 172.21.15.83:0/457130898 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21ac019ae0 con 0x5642eda8bb40 2024-04-24T09:13:55.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.152+0000 7f219fe00640 1 -- 172.21.15.83:0/457130898 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f21ac033050 con 0x5642eda8bb40 2024-04-24T09:13:55.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.153+0000 7f21c37fd740 10 client.23193 fetch_fsmap learned FSMap version 64 2024-04-24T09:13:55.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.153+0000 7f21c37fd740 1 -- 172.21.15.83:0/457130898 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x5642ed7033f0 con 0x5642eda8bb40 2024-04-24T09:13:55.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.153+0000 7f219fe00640 1 -- 172.21.15.83:0/457130898 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f21ac019ae0 con 0x5642eda8bb40 2024-04-24T09:13:55.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.153+0000 7f219fe00640 1 -- 172.21.15.83:0/457130898 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 64) ==== 8374+0+0 (secure 0 0 0) 0x7f21ac004750 con 0x5642eda8bb40 2024-04-24T09:13:55.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.153+0000 7f21c37fd740 10 client.23193 fetch_fsmap finished waiting for FSMap version 64 2024-04-24T09:13:55.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.153+0000 7f21c37fd740 10 client.23193 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-24T09:13:55.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.154+0000 7f21c37fd740 20 client.23193 populate_metadata read hostname 'smithi083' 2024-04-24T09:13:55.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.154+0000 7f21c37fd740 1 --2- 172.21.15.83:0/457130898 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x5642edab7c20 0x5642edaba010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:55.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.154+0000 7f21c37fd740 4 client.23193 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:13:55.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.154+0000 7f21c37fd740 1 -- 172.21.15.83:0/457130898 --> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5642ed9febf0 con 0x5642edab7c20 2024-04-24T09:13:55.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.154+0000 7f21b3e00640 1 --2- 172.21.15.83:0/457130898 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x5642edab7c20 0x5642edaba010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:55.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.155+0000 7f21b3e00640 1 --2- 172.21.15.83:0/457130898 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x5642edab7c20 0x5642edaba010 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e4b02a9f40bf6bac server_cookie=9117b3d411e1d7a7 in_seq=0 out_seq=0 2024-04-24T09:13:55.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.155+0000 7f219fe00640 10 client.23193 ms_handle_connect on v2:172.21.15.138:6836/900566754 2024-04-24T09:13:55.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.157+0000 7f219fe00640 1 -- 172.21.15.83:0/457130898 <== mds.1 v2:172.21.15.138:6836/900566754 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5642ed9febf0 con 0x5642edab7c20 2024-04-24T09:13:55.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.157+0000 7f219fe00640 10 client.23193 handle_command_reply: tid=0 2024-04-24T09:13:55.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.158+0000 7f21c37fd740 1 client.23193 shutdown 2024-04-24T09:13:55.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.158+0000 7f21c37fd740 1 -- 172.21.15.83:0/457130898 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642eda8bb40 msgr2=0x5642eda96560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:55.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.158+0000 7f21c37fd740 1 --2- 172.21.15.83:0/457130898 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642eda8bb40 0x5642eda96560 secure :-1 s=READY pgs=2641 cs=0 l=1 rev1=1 crypto rx=0x7f21ac009690 tx=0x7f21ac005df0 comp rx=0 tx=0).stop 2024-04-24T09:13:55.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.158+0000 7f21c37fd740 1 -- 172.21.15.83:0/457130898 shutdown_connections 2024-04-24T09:13:55.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.158+0000 7f21c37fd740 1 --2- 172.21.15.83:0/457130898 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x5642edab7c20 0x5642edaba010 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:55.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.158+0000 7f21c37fd740 1 --2- 172.21.15.83:0/457130898 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5642eda96b30 0x5642eda93f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:55.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.158+0000 7f21c37fd740 1 --2- 172.21.15.83:0/457130898 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642eda8c460 0x5642eda93850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:55.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.158+0000 7f21c37fd740 1 --2- 172.21.15.83:0/457130898 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642eda8bb40 0x5642eda96560 unknown :-1 s=CLOSED pgs=2641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:55.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.158+0000 7f219fe00640 0 client.23193 ms_handle_reset on v2:172.21.15.138:6836/900566754 2024-04-24T09:13:55.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.158+0000 7f21c37fd740 1 -- 172.21.15.83:0/457130898 >> 172.21.15.83:0/457130898 conn(0x5642eda87e60 msgr2=0x5642eda8db60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:55.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.159+0000 7f21c37fd740 1 -- 172.21.15.83:0/457130898 shutdown_connections 2024-04-24T09:13:55.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.159+0000 7f21c37fd740 1 -- 172.21.15.83:0/457130898 wait complete. 2024-04-24T09:13:55.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.159+0000 7f21c37fd740 20 client.23193 trim_cache size 0 max 16384 2024-04-24T09:13:55.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.182+0000 7f21c37fd740 1 Processor -- start 2024-04-24T09:13:55.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.182+0000 7f21c37fd740 1 -- start start 2024-04-24T09:13:55.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.182+0000 7f21c37fd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642eda8bb40 0x5642eda92790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:55.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.182+0000 7f21c37fd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5642eda8c460 0x5642eda86dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:55.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.182+0000 7f21c37fd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642eda96b30 0x5642eda96090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:55.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.182+0000 7f21c37fd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f21bc1ad300 con 0x5642eda8bb40 2024-04-24T09:13:55.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.182+0000 7f21c37fd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f21bc1aeed0 con 0x5642eda96b30 2024-04-24T09:13:55.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.182+0000 7f21c37fd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f21bc1ad9d0 con 0x5642eda8c460 2024-04-24T09:13:55.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.182+0000 7f21b9e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642eda8bb40 0x5642eda92790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:55.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.182+0000 7f21b9e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642eda8bb40 0x5642eda92790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45112/0 (socket says 172.21.15.83:45112) 2024-04-24T09:13:55.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.182+0000 7f21b9e00640 1 -- 172.21.15.83:0/4059002579 learned_addr learned my addr 172.21.15.83:0/4059002579 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:13:55.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.182+0000 7f21ba800640 1 --2- 172.21.15.83:0/4059002579 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642eda96b30 0x5642eda96090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:55.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.183+0000 7f21b3e00640 1 --2- 172.21.15.83:0/4059002579 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5642eda8c460 0x5642eda86dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:55.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.183+0000 7f21b9e00640 1 -- 172.21.15.83:0/4059002579 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5642eda8c460 msgr2=0x5642eda86dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:55.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.183+0000 7f21b9e00640 1 --2- 172.21.15.83:0/4059002579 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5642eda8c460 0x5642eda86dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:55.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.183+0000 7f21b9e00640 1 -- 172.21.15.83:0/4059002579 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642eda96b30 msgr2=0x5642eda96090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:55.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.183+0000 7f21b9e00640 1 --2- 172.21.15.83:0/4059002579 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642eda96b30 0x5642eda96090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:55.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.183+0000 7f21b9e00640 1 -- 172.21.15.83:0/4059002579 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642ed7cd7d0 con 0x5642eda8bb40 2024-04-24T09:13:55.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.183+0000 7f21b9e00640 1 --2- 172.21.15.83:0/4059002579 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642eda8bb40 0x5642eda92790 secure :-1 s=READY pgs=2642 cs=0 l=1 rev1=1 crypto rx=0x7f21ac011850 tx=0x7f21ac009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb6b024c6f1fa3a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:55.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.184+0000 7f21aaa00640 1 -- 172.21.15.83:0/4059002579 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21ac03c070 con 0x5642eda8bb40 2024-04-24T09:13:55.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.184+0000 7f21aaa00640 1 -- 172.21.15.83:0/4059002579 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f21ac02a070 con 0x5642eda8bb40 2024-04-24T09:13:55.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.184+0000 7f21aaa00640 1 -- 172.21.15.83:0/4059002579 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21ac01e070 con 0x5642eda8bb40 2024-04-24T09:13:55.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.185+0000 7f21c37fd740 1 -- 172.21.15.83:0/4059002579 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642eda8bb40 msgr2=0x5642eda92790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:55.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.185+0000 7f21c37fd740 1 --2- 172.21.15.83:0/4059002579 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642eda8bb40 0x5642eda92790 secure :-1 s=READY pgs=2642 cs=0 l=1 rev1=1 crypto rx=0x7f21ac011850 tx=0x7f21ac009bd0 comp rx=0 tx=0).stop 2024-04-24T09:13:55.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.185+0000 7f21c37fd740 1 -- 172.21.15.83:0/4059002579 shutdown_connections 2024-04-24T09:13:55.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.185+0000 7f21c37fd740 1 --2- 172.21.15.83:0/4059002579 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642eda96b30 0x5642eda96090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:55.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.185+0000 7f21c37fd740 1 --2- 172.21.15.83:0/4059002579 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5642eda8c460 0x5642eda86dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:55.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.185+0000 7f21c37fd740 1 --2- 172.21.15.83:0/4059002579 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642eda8bb40 0x5642eda92790 unknown :-1 s=CLOSED pgs=2642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:55.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.185+0000 7f21c37fd740 1 -- 172.21.15.83:0/4059002579 >> 172.21.15.83:0/4059002579 conn(0x5642eda87e60 msgr2=0x5642eda866c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:55.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.185+0000 7f21c37fd740 1 -- 172.21.15.83:0/4059002579 shutdown_connections 2024-04-24T09:13:55.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.185+0000 7f21c37fd740 1 -- 172.21.15.83:0/4059002579 wait complete. 2024-04-24T09:13:55.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.186+0000 7f21c37fd740 1 -- start start 2024-04-24T09:13:55.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.186+0000 7f21c37fd740 1 Processor -- start 2024-04-24T09:13:55.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.187+0000 7f21c37fd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642eda8bb40 0x5642eda974d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:55.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.187+0000 7f21c37fd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642eda8c460 0x5642eda97a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:55.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.187+0000 7f21c37fd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5642eda96b30 0x5642edaaeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:55.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.187+0000 7f21c37fd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f21bc1ae7d0 con 0x5642eda8bb40 2024-04-24T09:13:55.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.187+0000 7f21c37fd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f21bc1ae100 con 0x5642eda8c460 2024-04-24T09:13:55.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.187+0000 7f21c37fd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f21bc10b000 con 0x5642eda96b30 2024-04-24T09:13:55.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.187+0000 7f21c37fd740 10 client.0 _ll_register_callbacks cb 0x7f21ac01c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:13:55.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.187+0000 7f21b9e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642eda8bb40 0x5642eda974d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:55.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.188+0000 7f21b3e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642eda8c460 0x5642eda97a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:55.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.188+0000 7f21b3e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642eda8c460 0x5642eda97a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:60800/0 (socket says 172.21.15.83:60800) 2024-04-24T09:13:55.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.188+0000 7f21b3e00640 1 -- 172.21.15.83:0/3019535228 learned_addr learned my addr 172.21.15.83:0/3019535228 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:13:55.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.188+0000 7f21ba800640 1 --2- 172.21.15.83:0/3019535228 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5642eda96b30 0x5642edaaeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:55.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.188+0000 7f21b3e00640 1 -- 172.21.15.83:0/3019535228 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5642eda96b30 msgr2=0x5642edaaeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:55.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.188+0000 7f21b3e00640 1 --2- 172.21.15.83:0/3019535228 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5642eda96b30 0x5642edaaeda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:55.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.188+0000 7f21b3e00640 1 -- 172.21.15.83:0/3019535228 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642eda8bb40 msgr2=0x5642eda974d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:55.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.188+0000 7f21b3e00640 1 --2- 172.21.15.83:0/3019535228 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642eda8bb40 0x5642eda974d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:55.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.188+0000 7f21b3e00640 1 -- 172.21.15.83:0/3019535228 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642edaaf2e0 con 0x5642eda8c460 2024-04-24T09:13:55.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.189+0000 7f21b3e00640 1 --2- 172.21.15.83:0/3019535228 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642eda8c460 0x5642eda97a10 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f21bc10b560 tx=0x7f21a40046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dbb0295e261ac2a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:55.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.189+0000 7f219fe00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:13:55.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.189+0000 7f219fe00640 1 -- 172.21.15.83:0/3019535228 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21a4030060 con 0x5642eda8c460 2024-04-24T09:13:55.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.189+0000 7f219fe00640 1 -- 172.21.15.83:0/3019535228 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9583+0+0 (secure 0 0 0) 0x7f21a402bd50 con 0x5642eda8c460 2024-04-24T09:13:55.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.189+0000 7f21c37fd740 1 -- 172.21.15.83:0/3019535228 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21ac02a070 con 0x5642eda8c460 2024-04-24T09:13:55.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.190+0000 7f219fe00640 1 -- 172.21.15.83:0/3019535228 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f21a4037050 con 0x5642eda8c460 2024-04-24T09:13:55.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.190+0000 7f21c37fd740 10 client.40274 fetch_fsmap learned FSMap version 64 2024-04-24T09:13:55.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.190+0000 7f21c37fd740 1 -- 172.21.15.83:0/3019535228 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x5642edab0650 con 0x5642eda8c460 2024-04-24T09:13:55.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.190+0000 7f219fe00640 1 -- 172.21.15.83:0/3019535228 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f21ac02a070 con 0x5642eda8c460 2024-04-24T09:13:55.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.191+0000 7f219fe00640 1 -- 172.21.15.83:0/3019535228 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 64) ==== 8374+0+0 (secure 0 0 0) 0x7f21a4004be0 con 0x5642eda8c460 2024-04-24T09:13:55.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.191+0000 7f21c37fd740 10 client.40274 fetch_fsmap finished waiting for FSMap version 64 2024-04-24T09:13:55.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.191+0000 7f21c37fd740 10 client.40274 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-24T09:13:55.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.191+0000 7f21c37fd740 20 client.40274 populate_metadata read hostname 'smithi083' 2024-04-24T09:13:55.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.191+0000 7f21c37fd740 1 --2- 172.21.15.83:0/3019535228 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x5642edb1e520 0x5642edb20940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:55.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.191+0000 7f21c37fd740 4 client.40274 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:13:55.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.191+0000 7f21c37fd740 1 -- 172.21.15.83:0/3019535228 --> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5642ed9f8a60 con 0x5642edb1e520 2024-04-24T09:13:55.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.192+0000 7f21b9e00640 1 --2- 172.21.15.83:0/3019535228 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x5642edb1e520 0x5642edb20940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:55.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.193+0000 7f21b9e00640 1 --2- 172.21.15.83:0/3019535228 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x5642edb1e520 0x5642edb20940 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c88c834adfd0b2e2 server_cookie=769ab65b740b4ada in_seq=0 out_seq=0 2024-04-24T09:13:55.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.193+0000 7f219fe00640 10 client.40274 ms_handle_connect on v2:172.21.15.138:6836/900566754 2024-04-24T09:13:55.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.194+0000 7f219fe00640 1 -- 172.21.15.83:0/3019535228 <== mds.1 v2:172.21.15.138:6836/900566754 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5642ed9f8a60 con 0x5642edb1e520 2024-04-24T09:13:55.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.194+0000 7f219fe00640 10 client.40274 handle_command_reply: tid=0 2024-04-24T09:13:55.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.194+0000 7f21c37fd740 1 client.40274 shutdown 2024-04-24T09:13:55.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.195+0000 7f21c37fd740 1 -- 172.21.15.83:0/3019535228 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642eda8c460 msgr2=0x5642eda97a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:55.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.195+0000 7f21c37fd740 1 --2- 172.21.15.83:0/3019535228 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642eda8c460 0x5642eda97a10 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f21bc10b560 tx=0x7f21a40046c0 comp rx=0 tx=0).stop 2024-04-24T09:13:55.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.195+0000 7f21c37fd740 1 -- 172.21.15.83:0/3019535228 shutdown_connections 2024-04-24T09:13:55.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.195+0000 7f21c37fd740 1 --2- 172.21.15.83:0/3019535228 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x5642edb1e520 0x5642edb20940 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:55.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.195+0000 7f21c37fd740 1 --2- 172.21.15.83:0/3019535228 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5642eda96b30 0x5642edaaeda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:55.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.195+0000 7f219fe00640 0 client.40274 ms_handle_reset on v2:172.21.15.138:6836/900566754 2024-04-24T09:13:55.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.195+0000 7f21c37fd740 1 --2- 172.21.15.83:0/3019535228 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642eda8c460 0x5642eda97a10 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:55.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.195+0000 7f21c37fd740 1 --2- 172.21.15.83:0/3019535228 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642eda8bb40 0x5642eda974d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:55.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.195+0000 7f21c37fd740 1 -- 172.21.15.83:0/3019535228 >> 172.21.15.83:0/3019535228 conn(0x5642eda87e60 msgr2=0x5642eda86f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:55.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.196+0000 7f21c37fd740 1 -- 172.21.15.83:0/3019535228 shutdown_connections 2024-04-24T09:13:55.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.196+0000 7f21c37fd740 1 -- 172.21.15.83:0/3019535228 wait complete. 2024-04-24T09:13:55.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.196+0000 7f21c37fd740 20 client.40274 trim_cache size 0 max 16384 2024-04-24T09:13:55.197 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:13:55.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.197+0000 7f21bb200640 1 -- 172.21.15.83:0/2920021818 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f21a00a3da0 msgr2=0x7f21a00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:55.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.197+0000 7f21bb200640 1 --2- 172.21.15.83:0/2920021818 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f21a00a3da0 0x7f21a00a6260 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f21a4009fd0 tx=0x7f21a40095d0 comp rx=0 tx=0).stop 2024-04-24T09:13:55.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.197+0000 7f21bb200640 1 -- 172.21.15.83:0/2920021818 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f21bc10c480 msgr2=0x7f21bc10a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:55.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.197+0000 7f21bb200640 1 --2- 172.21.15.83:0/2920021818 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f21bc10c480 0x7f21bc10a060 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f21b4005cc0 tx=0x7f21b4003c90 comp rx=0 tx=0).stop 2024-04-24T09:13:55.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.197+0000 7f21bb200640 1 -- 172.21.15.83:0/2920021818 shutdown_connections 2024-04-24T09:13:55.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.197+0000 7f21bb200640 1 --2- 172.21.15.83:0/2920021818 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f21a00a3da0 0x7f21a00a6260 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:55.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.198+0000 7f21bb200640 1 --2- 172.21.15.83:0/2920021818 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f21bc10c480 0x7f21bc10a060 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:55.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.198+0000 7f21bb200640 1 --2- 172.21.15.83:0/2920021818 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f21bc103980 0x7f21bc109b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:55.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.198+0000 7f21bb200640 1 --2- 172.21.15.83:0/2920021818 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f21bc103060 0x7f21bc10b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:55.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.198+0000 7f21bb200640 1 -- 172.21.15.83:0/2920021818 >> 172.21.15.83:0/2920021818 conn(0x7f21bc0fc880 msgr2=0x7f21bc0fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:55.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.198+0000 7f21bb200640 1 -- 172.21.15.83:0/2920021818 shutdown_connections 2024-04-24T09:13:55.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.198+0000 7f21bb200640 1 -- 172.21.15.83:0/2920021818 wait complete. 2024-04-24T09:13:55.201 INFO:tasks.workunit.client.0.smithi083.stdout: 10 62739 6.60 MB/sec execute 386 sec latency 166.249 ms 2024-04-24T09:13:55.222 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T09:13:55.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.514+0000 7f0a23200640 1 -- 172.21.15.83:0/3317846894 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0a1c0a5700 msgr2=0x7f0a1c0b7530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:55.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.514+0000 7f0a23200640 1 --2- 172.21.15.83:0/3317846894 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0a1c0a5700 0x7f0a1c0b7530 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0a10009900 tx=0x7f0a1002a480 comp rx=0 tx=0).stop 2024-04-24T09:13:55.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.514+0000 7f0a23200640 1 -- 172.21.15.83:0/3317846894 shutdown_connections 2024-04-24T09:13:55.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.514+0000 7f0a23200640 1 --2- 172.21.15.83:0/3317846894 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a1c0b7a70 0x7f0a1c0b9e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:55.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.514+0000 7f0a23200640 1 --2- 172.21.15.83:0/3317846894 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0a1c0a5700 0x7f0a1c0b7530 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:55.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.514+0000 7f0a23200640 1 --2- 172.21.15.83:0/3317846894 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a1c0a4de0 0x7f0a1c0a51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:55.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.514+0000 7f0a23200640 1 -- 172.21.15.83:0/3317846894 >> 172.21.15.83:0/3317846894 conn(0x7f0a1c01a640 msgr2=0x7f0a1c01aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:55.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.514+0000 7f0a23200640 1 -- 172.21.15.83:0/3317846894 shutdown_connections 2024-04-24T09:13:55.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.514+0000 7f0a23200640 1 -- 172.21.15.83:0/3317846894 wait complete. 2024-04-24T09:13:55.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.515+0000 7f0a23200640 1 Processor -- start 2024-04-24T09:13:55.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.516+0000 7f0a23200640 1 -- start start 2024-04-24T09:13:55.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.516+0000 7f0a23200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a1c0a4de0 0x7f0a1c1496f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:55.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.516+0000 7f0a23200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a1c0a5700 0x7f0a1c149c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:55.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.516+0000 7f0a23200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0a1c0b7a70 0x7f0a1c1438a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:55.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.516+0000 7f0a23200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0a1c0bd5c0 con 0x7f0a1c0a5700 2024-04-24T09:13:55.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.516+0000 7f0a23200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0a1c0bd440 con 0x7f0a1c0a4de0 2024-04-24T09:13:55.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.516+0000 7f0a23200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0a1c0bd740 con 0x7f0a1c0b7a70 2024-04-24T09:13:55.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.517+0000 7f0a21e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a1c0a4de0 0x7f0a1c1496f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:55.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.517+0000 7f0a21400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a1c0a5700 0x7f0a1c149c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:55.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.517+0000 7f0a21400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a1c0a5700 0x7f0a1c149c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45152/0 (socket says 172.21.15.83:45152) 2024-04-24T09:13:55.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.517+0000 7f0a21400640 1 -- 172.21.15.83:0/145694509 learned_addr learned my addr 172.21.15.83:0/145694509 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:13:55.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.517+0000 7f0a22800640 1 --2- 172.21.15.83:0/145694509 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0a1c0b7a70 0x7f0a1c1438a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:55.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.518+0000 7f0a21e00640 1 -- 172.21.15.83:0/145694509 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0a1c0b7a70 msgr2=0x7f0a1c1438a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:55.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.518+0000 7f0a21e00640 1 --2- 172.21.15.83:0/145694509 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0a1c0b7a70 0x7f0a1c1438a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:55.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.518+0000 7f0a21e00640 1 -- 172.21.15.83:0/145694509 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a1c0a5700 msgr2=0x7f0a1c149c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:55.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.518+0000 7f0a21e00640 1 --2- 172.21.15.83:0/145694509 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a1c0a5700 0x7f0a1c149c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:55.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.518+0000 7f0a21e00640 1 -- 172.21.15.83:0/145694509 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a1c144020 con 0x7f0a1c0a4de0 2024-04-24T09:13:55.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.518+0000 7f0a22800640 1 --2- 172.21.15.83:0/145694509 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0a1c0b7a70 0x7f0a1c1438a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:13:55.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.518+0000 7f0a21e00640 1 --2- 172.21.15.83:0/145694509 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a1c0a4de0 0x7f0a1c1496f0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f0a1400e9c0 tx=0x7f0a1400edc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:55.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.519+0000 7f0a1aa00640 1 -- 172.21.15.83:0/145694509 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a1400ccc0 con 0x7f0a1c0a4de0 2024-04-24T09:13:55.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.519+0000 7f0a1aa00640 1 -- 172.21.15.83:0/145694509 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0a140035f0 con 0x7f0a1c0a4de0 2024-04-24T09:13:55.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.519+0000 7f0a1aa00640 1 -- 172.21.15.83:0/145694509 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a1400ce60 con 0x7f0a1c0a4de0 2024-04-24T09:13:55.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.519+0000 7f0a23200640 1 -- 172.21.15.83:0/145694509 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a1c144310 con 0x7f0a1c0a4de0 2024-04-24T09:13:55.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.519+0000 7f0a23200640 1 -- 172.21.15.83:0/145694509 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a1c1505b0 con 0x7f0a1c0a4de0 2024-04-24T09:13:55.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.523+0000 7f0a1aa00640 1 -- 172.21.15.83:0/145694509 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0a140037a0 con 0x7f0a1c0a4de0 2024-04-24T09:13:55.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.524+0000 7f0a2b82c740 1 Processor -- start 2024-04-24T09:13:55.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.524+0000 7f0a2b82c740 1 -- start start 2024-04-24T09:13:55.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.525+0000 7f0a2b82c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f009d45b40 0x55f009d45f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:55.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.525+0000 7f0a2b82c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f009d46460 0x55f009d505f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:55.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.525+0000 7f0a2b82c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f009d50b30 0x55f009d52f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:55.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.525+0000 7f0a2b82c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55f009cab1d0 con 0x55f009d50b30 2024-04-24T09:13:55.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.525+0000 7f0a2b82c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f009c51450 con 0x55f009d45b40 2024-04-24T09:13:55.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.525+0000 7f0a2b82c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55f009ca7190 con 0x55f009d46460 2024-04-24T09:13:55.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.525+0000 7f0a22800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f009d45b40 0x55f009d45f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:55.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.525+0000 7f0a22800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f009d45b40 0x55f009d45f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:60828/0 (socket says 172.21.15.83:60828) 2024-04-24T09:13:55.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.525+0000 7f0a1aa00640 1 --2- 172.21.15.83:0/145694509 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f0a080a3f20 0x7f0a080a63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:55.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.525+0000 7f0a22800640 1 -- 172.21.15.83:0/1991728745 learned_addr learned my addr 172.21.15.83:0/1991728745 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:13:55.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.525+0000 7f0a21e00640 1 --2- 172.21.15.83:0/1991728745 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f009d46460 0x55f009d505f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:55.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.525+0000 7f0a21400640 1 --2- 172.21.15.83:0/1991728745 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f009d50b30 0x55f009d52f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:55.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.526+0000 7f0a1aa00640 1 -- 172.21.15.83:0/145694509 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 9583+0+0 (secure 0 0 0) 0x7f0a14010430 con 0x7f0a1c0a4de0 2024-04-24T09:13:55.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.526+0000 7f0a22800640 1 --2- 172.21.15.83:0/145694509 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f0a080a3f20 0x7f0a080a63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:55.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.526+0000 7f0a21400640 1 -- 172.21.15.83:0/1991728745 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f009d46460 msgr2=0x55f009d505f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:55.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.526+0000 7f0a21400640 1 --2- 172.21.15.83:0/1991728745 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f009d46460 0x55f009d505f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:55.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.526+0000 7f0a21400640 1 -- 172.21.15.83:0/1991728745 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f009d45b40 msgr2=0x55f009d45f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:55.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.526+0000 7f0a21400640 1 --2- 172.21.15.83:0/1991728745 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f009d45b40 0x55f009d45f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:55.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.526+0000 7f0a21400640 1 -- 172.21.15.83:0/1991728745 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f009d0e310 con 0x55f009d50b30 2024-04-24T09:13:55.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.527+0000 7f0a21400640 1 --2- 172.21.15.83:0/1991728745 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f009d50b30 0x55f009d52f20 secure :-1 s=READY pgs=2643 cs=0 l=1 rev1=1 crypto rx=0x7f0a10000c00 tx=0x7f0a10005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d5bee3e477ea266 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:55.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.527+0000 7f0a22800640 1 --2- 172.21.15.83:0/145694509 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f0a080a3f20 0x7f0a080a63e0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f0a24051000 tx=0x7f0a240699f0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:55.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.527+0000 7f0a0f000640 1 -- 172.21.15.83:0/1991728745 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a10040070 con 0x55f009d50b30 2024-04-24T09:13:55.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.528+0000 7f0a0f000640 1 -- 172.21.15.83:0/1991728745 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0a100054d0 con 0x55f009d50b30 2024-04-24T09:13:55.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.528+0000 7f0a0f000640 1 -- 172.21.15.83:0/1991728745 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a10035040 con 0x55f009d50b30 2024-04-24T09:13:55.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.529+0000 7f0a2b82c740 1 -- 172.21.15.83:0/1991728745 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f009d50b30 msgr2=0x55f009d52f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:55.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.529+0000 7f0a2b82c740 1 --2- 172.21.15.83:0/1991728745 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f009d50b30 0x55f009d52f20 secure :-1 s=READY pgs=2643 cs=0 l=1 rev1=1 crypto rx=0x7f0a10000c00 tx=0x7f0a10005c00 comp rx=0 tx=0).stop 2024-04-24T09:13:55.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.530+0000 7f0a2b82c740 1 -- 172.21.15.83:0/1991728745 shutdown_connections 2024-04-24T09:13:55.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.530+0000 7f0a2b82c740 1 --2- 172.21.15.83:0/1991728745 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f009d50b30 0x55f009d52f20 unknown :-1 s=CLOSED pgs=2643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:55.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.530+0000 7f0a2b82c740 1 --2- 172.21.15.83:0/1991728745 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f009d46460 0x55f009d505f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:55.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.530+0000 7f0a2b82c740 1 --2- 172.21.15.83:0/1991728745 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f009d45b40 0x55f009d45f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:55.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.530+0000 7f0a2b82c740 1 -- 172.21.15.83:0/1991728745 >> 172.21.15.83:0/1991728745 conn(0x55f009d41e60 msgr2=0x55f009d42240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:55.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.531+0000 7f0a2b82c740 1 -- 172.21.15.83:0/1991728745 shutdown_connections 2024-04-24T09:13:55.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.532+0000 7f0a2b82c740 1 -- 172.21.15.83:0/1991728745 wait complete. 2024-04-24T09:13:55.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.532+0000 7f0a2b82c740 1 -- start start 2024-04-24T09:13:55.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.532+0000 7f0a2b82c740 1 Processor -- start 2024-04-24T09:13:55.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.533+0000 7f0a2b82c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f009d45b40 0x55f009d50550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:55.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.533+0000 7f0a2b82c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f009d46460 0x55f009d4d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:55.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.533+0000 7f0a2b82c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f009d50b30 0x55f009d4de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:55.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.533+0000 7f0a2b82c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55f009d53960 con 0x55f009d50b30 2024-04-24T09:13:55.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.534+0000 7f0a2b82c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f009c64810 con 0x55f009d46460 2024-04-24T09:13:55.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.534+0000 7f0a2b82c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55f009d53ae0 con 0x55f009d45b40 2024-04-24T09:13:55.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.534+0000 7f0a2b82c740 10 client.0 _ll_register_callbacks cb 0x55f009c32140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:13:55.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.534+0000 7f0a21400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f009d46460 0x55f009d4d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:55.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.534+0000 7f0a21e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f009d45b40 0x55f009d50550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:55.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.534+0000 7f0a21e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f009d45b40 0x55f009d50550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:41174/0 (socket says 172.21.15.83:41174) 2024-04-24T09:13:55.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.534+0000 7f0a21e00640 1 -- 172.21.15.83:0/1244732068 learned_addr learned my addr 172.21.15.83:0/1244732068 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:13:55.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.535+0000 7f0a21e00640 1 -- 172.21.15.83:0/1244732068 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f009d46460 msgr2=0x55f009d4d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:55.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.535+0000 7f0a21e00640 1 --2- 172.21.15.83:0/1244732068 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f009d46460 0x55f009d4d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:55.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.535+0000 7f0a21e00640 1 -- 172.21.15.83:0/1244732068 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f009d50b30 msgr2=0x55f009d4de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:55.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.535+0000 7f0a21e00640 1 --2- 172.21.15.83:0/1244732068 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f009d50b30 0x55f009d4de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:55.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.535+0000 7f0a21e00640 1 -- 172.21.15.83:0/1244732068 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f009c753f0 con 0x55f009d45b40 2024-04-24T09:13:55.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.536+0000 7f0a21e00640 1 --2- 172.21.15.83:0/1244732068 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f009d45b40 0x55f009d50550 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0a140992f0 tx=0x7f0a14015f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7db531870eba70dc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:55.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.536+0000 7f0a03400640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-04-24T09:13:55.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.536+0000 7f0a03400640 1 -- 172.21.15.83:0/1244732068 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a14085e60 con 0x55f009d45b40 2024-04-24T09:13:55.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.536+0000 7f0a03400640 1 -- 172.21.15.83:0/1244732068 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9583+0+0 (secure 0 0 0) 0x7f0a1409cd80 con 0x55f009d45b40 2024-04-24T09:13:55.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.536+0000 7f0a2b82c740 1 -- 172.21.15.83:0/1244732068 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a100054d0 con 0x55f009d45b40 2024-04-24T09:13:55.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.537+0000 7f0a03400640 1 -- 172.21.15.83:0/1244732068 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f0a1409d020 con 0x55f009d45b40 2024-04-24T09:13:55.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.537+0000 7f0a2b82c740 10 client.30046 fetch_fsmap learned FSMap version 64 2024-04-24T09:13:55.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.537+0000 7f0a2b82c740 1 -- 172.21.15.83:0/1244732068 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55f0099bd3f0 con 0x55f009d45b40 2024-04-24T09:13:55.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.537+0000 7f0a03400640 1 -- 172.21.15.83:0/1244732068 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0a100054d0 con 0x55f009d45b40 2024-04-24T09:13:55.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.538+0000 7f0a03400640 1 -- 172.21.15.83:0/1244732068 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 64) ==== 8374+0+0 (secure 0 0 0) 0x7f0a140970a0 con 0x55f009d45b40 2024-04-24T09:13:55.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.538+0000 7f0a2b82c740 10 client.30046 fetch_fsmap finished waiting for FSMap version 64 2024-04-24T09:13:55.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.538+0000 7f0a2b82c740 10 client.30046 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:13:55.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.538+0000 7f0a2b82c740 20 client.30046 populate_metadata read hostname 'smithi083' 2024-04-24T09:13:55.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.538+0000 7f0a2b82c740 1 --2- 172.21.15.83:0/1244732068 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55f009d60c30 0x55f009d630a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:55.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.538+0000 7f0a2b82c740 4 client.30046 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:13:55.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.538+0000 7f0a2b82c740 1 -- 172.21.15.83:0/1244732068 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f009cb8bf0 con 0x55f009d60c30 2024-04-24T09:13:55.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.539+0000 7f0a21400640 1 --2- 172.21.15.83:0/1244732068 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55f009d60c30 0x55f009d630a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:55.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.539+0000 7f0a21400640 1 --2- 172.21.15.83:0/1244732068 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55f009d60c30 0x55f009d630a0 crc :-1 s=READY pgs=549 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9630767c36a65b17 server_cookie=219b81674a2bc28b in_seq=0 out_seq=0 2024-04-24T09:13:55.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.539+0000 7f0a03400640 10 client.30046 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:13:55.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.540+0000 7f0a03400640 1 -- 172.21.15.83:0/1244732068 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f009cb8bf0 con 0x55f009d60c30 2024-04-24T09:13:55.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.540+0000 7f0a03400640 10 client.30046 handle_command_reply: tid=0 2024-04-24T09:13:55.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.540+0000 7f0a2b82c740 1 client.30046 shutdown 2024-04-24T09:13:55.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.541+0000 7f0a2b82c740 1 -- 172.21.15.83:0/1244732068 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f009d45b40 msgr2=0x55f009d50550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:55.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.541+0000 7f0a2b82c740 1 --2- 172.21.15.83:0/1244732068 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f009d45b40 0x55f009d50550 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0a140992f0 tx=0x7f0a14015f80 comp rx=0 tx=0).stop 2024-04-24T09:13:55.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.541+0000 7f0a2b82c740 1 -- 172.21.15.83:0/1244732068 shutdown_connections 2024-04-24T09:13:55.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.541+0000 7f0a2b82c740 1 --2- 172.21.15.83:0/1244732068 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55f009d60c30 0x55f009d630a0 crc :-1 s=READY pgs=549 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:55.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.541+0000 7f0a2b82c740 1 --2- 172.21.15.83:0/1244732068 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f009d50b30 0x55f009d4de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:55.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.541+0000 7f0a2b82c740 1 --2- 172.21.15.83:0/1244732068 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f009d46460 0x55f009d4d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:55.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.541+0000 7f0a2b82c740 1 --2- 172.21.15.83:0/1244732068 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f009d45b40 0x55f009d50550 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:55.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.541+0000 7f0a2b82c740 1 -- 172.21.15.83:0/1244732068 >> 172.21.15.83:0/1244732068 conn(0x55f009d41e60 msgr2=0x55f009d47c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:55.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.541+0000 7f0a03400640 0 client.30046 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:13:55.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.541+0000 7f0a2b82c740 1 -- 172.21.15.83:0/1244732068 shutdown_connections 2024-04-24T09:13:55.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.541+0000 7f0a2b82c740 1 -- 172.21.15.83:0/1244732068 wait complete. 2024-04-24T09:13:55.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.541+0000 7f0a2b82c740 20 client.30046 trim_cache size 0 max 16384 2024-04-24T09:13:55.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.563+0000 7f0a2b82c740 1 Processor -- start 2024-04-24T09:13:55.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.563+0000 7f0a2b82c740 1 -- start start 2024-04-24T09:13:55.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.564+0000 7f0a2b82c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f009d45b40 0x55f009d5c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:55.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.564+0000 7f0a2b82c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f009d46460 0x55f009d50080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:55.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.564+0000 7f0a2b82c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f009d50b30 0x55f009d68970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:55.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.564+0000 7f0a2b82c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0a1c15a900 con 0x55f009d46460 2024-04-24T09:13:55.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.564+0000 7f0a2b82c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0a1c15c4d0 con 0x55f009d45b40 2024-04-24T09:13:55.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.564+0000 7f0a2b82c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0a1c15afd0 con 0x55f009d50b30 2024-04-24T09:13:55.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.564+0000 7f0a21e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f009d45b40 0x55f009d5c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:55.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.564+0000 7f0a21400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f009d46460 0x55f009d50080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:55.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.564+0000 7f0a21e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f009d45b40 0x55f009d5c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:60842/0 (socket says 172.21.15.83:60842) 2024-04-24T09:13:55.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.564+0000 7f0a21e00640 1 -- 172.21.15.83:0/4195217066 learned_addr learned my addr 172.21.15.83:0/4195217066 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:13:55.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.564+0000 7f0a22800640 1 --2- 172.21.15.83:0/4195217066 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f009d50b30 0x55f009d68970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:55.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.565+0000 7f0a21400640 1 -- 172.21.15.83:0/4195217066 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f009d50b30 msgr2=0x55f009d68970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:55.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.565+0000 7f0a21400640 1 --2- 172.21.15.83:0/4195217066 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f009d50b30 0x55f009d68970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:55.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.565+0000 7f0a21400640 1 -- 172.21.15.83:0/4195217066 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f009d45b40 msgr2=0x55f009d5c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:55.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.565+0000 7f0a21400640 1 --2- 172.21.15.83:0/4195217066 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f009d45b40 0x55f009d5c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:55.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.565+0000 7f0a21400640 1 -- 172.21.15.83:0/4195217066 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f009a877d0 con 0x55f009d46460 2024-04-24T09:13:55.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.565+0000 7f0a21e00640 1 --2- 172.21.15.83:0/4195217066 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f009d45b40 0x55f009d5c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:13:55.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.566+0000 7f0a21400640 1 --2- 172.21.15.83:0/4195217066 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f009d46460 0x55f009d50080 secure :-1 s=READY pgs=2644 cs=0 l=1 rev1=1 crypto rx=0x55f009d3d9d0 tx=0x7f0a10004b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2a838e0d9beaa8d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:55.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.566+0000 7f0a0e600640 1 -- 172.21.15.83:0/4195217066 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a10055070 con 0x55f009d46460 2024-04-24T09:13:55.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.566+0000 7f0a0e600640 1 -- 172.21.15.83:0/4195217066 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0a100049a0 con 0x55f009d46460 2024-04-24T09:13:55.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.566+0000 7f0a0e600640 1 -- 172.21.15.83:0/4195217066 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a10035040 con 0x55f009d46460 2024-04-24T09:13:55.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.567+0000 7f0a2b82c740 1 -- 172.21.15.83:0/4195217066 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f009d46460 msgr2=0x55f009d50080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:55.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.567+0000 7f0a2b82c740 1 --2- 172.21.15.83:0/4195217066 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f009d46460 0x55f009d50080 secure :-1 s=READY pgs=2644 cs=0 l=1 rev1=1 crypto rx=0x55f009d3d9d0 tx=0x7f0a10004b70 comp rx=0 tx=0).stop 2024-04-24T09:13:55.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.567+0000 7f0a2b82c740 1 -- 172.21.15.83:0/4195217066 shutdown_connections 2024-04-24T09:13:55.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.567+0000 7f0a2b82c740 1 --2- 172.21.15.83:0/4195217066 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f009d50b30 0x55f009d68970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:55.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.567+0000 7f0a2b82c740 1 --2- 172.21.15.83:0/4195217066 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f009d46460 0x55f009d50080 unknown :-1 s=CLOSED pgs=2644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:55.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.567+0000 7f0a2b82c740 1 --2- 172.21.15.83:0/4195217066 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f009d45b40 0x55f009d5c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:55.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.567+0000 7f0a2b82c740 1 -- 172.21.15.83:0/4195217066 >> 172.21.15.83:0/4195217066 conn(0x55f009d41e60 msgr2=0x55f009d47380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:55.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.568+0000 7f0a2b82c740 1 -- 172.21.15.83:0/4195217066 shutdown_connections 2024-04-24T09:13:55.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.568+0000 7f0a2b82c740 1 -- 172.21.15.83:0/4195217066 wait complete. 2024-04-24T09:13:55.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.568+0000 7f0a2b82c740 1 -- start start 2024-04-24T09:13:55.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.569+0000 7f0a2b82c740 1 Processor -- start 2024-04-24T09:13:55.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.569+0000 7f0a2b82c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f009d45b40 0x55f009d58d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:55.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.569+0000 7f0a2b82c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f009d46460 0x55f009d59270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:55.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.570+0000 7f0a2b82c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f009d50b30 0x55f009d51d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:55.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.570+0000 7f0a2b82c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0a1c15bdd0 con 0x55f009d50b30 2024-04-24T09:13:55.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.570+0000 7f0a2b82c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0a1c15b700 con 0x55f009d46460 2024-04-24T09:13:55.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.570+0000 7f0a2b82c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0a1c158900 con 0x55f009d45b40 2024-04-24T09:13:55.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.570+0000 7f0a2b82c740 10 client.0 _ll_register_callbacks cb 0x7f0a14009660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:13:55.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.570+0000 7f0a21e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f009d45b40 0x55f009d58d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:55.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.570+0000 7f0a21e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f009d45b40 0x55f009d58d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:41196/0 (socket says 172.21.15.83:41196) 2024-04-24T09:13:55.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.570+0000 7f0a21e00640 1 -- 172.21.15.83:0/3267124574 learned_addr learned my addr 172.21.15.83:0/3267124574 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:13:55.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.570+0000 7f0a21400640 1 --2- 172.21.15.83:0/3267124574 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f009d46460 0x55f009d59270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:55.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.570+0000 7f0a22800640 1 --2- 172.21.15.83:0/3267124574 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f009d50b30 0x55f009d51d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:55.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.571+0000 7f0a21e00640 1 -- 172.21.15.83:0/3267124574 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f009d46460 msgr2=0x55f009d59270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:55.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.571+0000 7f0a21e00640 1 --2- 172.21.15.83:0/3267124574 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f009d46460 0x55f009d59270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:55.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.571+0000 7f0a21e00640 1 -- 172.21.15.83:0/3267124574 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f009d50b30 msgr2=0x55f009d51d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:55.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.571+0000 7f0a21e00640 1 --2- 172.21.15.83:0/3267124574 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f009d50b30 0x55f009d51d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:55.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.571+0000 7f0a21e00640 1 -- 172.21.15.83:0/3267124574 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f009d522b0 con 0x55f009d45b40 2024-04-24T09:13:55.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.571+0000 7f0a22800640 1 --2- 172.21.15.83:0/3267124574 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f009d50b30 0x55f009d51d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:13:55.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.571+0000 7f0a21e00640 1 --2- 172.21.15.83:0/3267124574 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f009d45b40 0x55f009d58d30 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0a140992f0 tx=0x7f0a1409cdb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=82a405b7b25b6fa server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:55.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.571+0000 7f0a03400640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-04-24T09:13:55.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.572+0000 7f0a03400640 1 -- 172.21.15.83:0/3267124574 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a14066060 con 0x55f009d45b40 2024-04-24T09:13:55.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.572+0000 7f0a03400640 1 -- 172.21.15.83:0/3267124574 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9583+0+0 (secure 0 0 0) 0x7f0a140970d0 con 0x55f009d45b40 2024-04-24T09:13:55.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.572+0000 7f0a2b82c740 1 -- 172.21.15.83:0/3267124574 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a100049a0 con 0x55f009d45b40 2024-04-24T09:13:55.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.572+0000 7f0a03400640 1 -- 172.21.15.83:0/3267124574 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f0a14085e60 con 0x55f009d45b40 2024-04-24T09:13:55.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.572+0000 7f0a2b82c740 10 client.30052 fetch_fsmap learned FSMap version 64 2024-04-24T09:13:55.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.572+0000 7f0a2b82c740 1 -- 172.21.15.83:0/3267124574 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55f009dc79c0 con 0x55f009d45b40 2024-04-24T09:13:55.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.573+0000 7f0a03400640 1 -- 172.21.15.83:0/3267124574 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0a100049a0 con 0x55f009d45b40 2024-04-24T09:13:55.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.574+0000 7f0a03400640 1 -- 172.21.15.83:0/3267124574 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 64) ==== 8374+0+0 (secure 0 0 0) 0x7f0a14067250 con 0x55f009d45b40 2024-04-24T09:13:55.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.574+0000 7f0a2b82c740 10 client.30052 fetch_fsmap finished waiting for FSMap version 64 2024-04-24T09:13:55.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.574+0000 7f0a2b82c740 10 client.30052 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:13:55.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.574+0000 7f0a2b82c740 20 client.30052 populate_metadata read hostname 'smithi083' 2024-04-24T09:13:55.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.574+0000 7f0a2b82c740 1 --2- 172.21.15.83:0/3267124574 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55f009dc7fb0 0x55f009dca3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:55.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.574+0000 7f0a2b82c740 4 client.30052 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:13:55.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.574+0000 7f0a2b82c740 1 -- 172.21.15.83:0/3267124574 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f009cb2a60 con 0x55f009dc7fb0 2024-04-24T09:13:55.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.574+0000 7f0a21400640 1 --2- 172.21.15.83:0/3267124574 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55f009dc7fb0 0x55f009dca3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:55.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.575+0000 7f0a21400640 1 --2- 172.21.15.83:0/3267124574 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55f009dc7fb0 0x55f009dca3d0 crc :-1 s=READY pgs=550 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ade01b5270c5dce4 server_cookie=527f6be88d0e85ed in_seq=0 out_seq=0 2024-04-24T09:13:55.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.575+0000 7f0a03400640 10 client.30052 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:13:55.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.576+0000 7f0a03400640 1 -- 172.21.15.83:0/3267124574 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f009cb2a60 con 0x55f009dc7fb0 2024-04-24T09:13:55.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.576+0000 7f0a03400640 10 client.30052 handle_command_reply: tid=0 2024-04-24T09:13:55.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.576+0000 7f0a2b82c740 1 client.30052 shutdown 2024-04-24T09:13:55.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.576+0000 7f0a2b82c740 1 -- 172.21.15.83:0/3267124574 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f009d45b40 msgr2=0x55f009d58d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:55.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.576+0000 7f0a2b82c740 1 --2- 172.21.15.83:0/3267124574 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f009d45b40 0x55f009d58d30 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0a140992f0 tx=0x7f0a1409cdb0 comp rx=0 tx=0).stop 2024-04-24T09:13:55.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.576+0000 7f0a2b82c740 1 -- 172.21.15.83:0/3267124574 shutdown_connections 2024-04-24T09:13:55.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.576+0000 7f0a2b82c740 1 --2- 172.21.15.83:0/3267124574 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55f009dc7fb0 0x55f009dca3d0 crc :-1 s=READY pgs=550 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:55.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.576+0000 7f0a2b82c740 1 --2- 172.21.15.83:0/3267124574 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f009d50b30 0x55f009d51d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:55.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.576+0000 7f0a2b82c740 1 --2- 172.21.15.83:0/3267124574 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f009d46460 0x55f009d59270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:55.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.576+0000 7f0a2b82c740 1 --2- 172.21.15.83:0/3267124574 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f009d45b40 0x55f009d58d30 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:55.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.576+0000 7f0a2b82c740 1 -- 172.21.15.83:0/3267124574 >> 172.21.15.83:0/3267124574 conn(0x55f009d41e60 msgr2=0x55f009d47bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:55.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.576+0000 7f0a03400640 0 client.30052 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:13:55.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.577+0000 7f0a2b82c740 1 -- 172.21.15.83:0/3267124574 shutdown_connections 2024-04-24T09:13:55.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.577+0000 7f0a2b82c740 1 -- 172.21.15.83:0/3267124574 wait complete. 2024-04-24T09:13:55.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.577+0000 7f0a2b82c740 20 client.30052 trim_cache size 0 max 16384 2024-04-24T09:13:55.578 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:13:55.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.578+0000 7f0a23200640 1 -- 172.21.15.83:0/145694509 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f0a080a3f20 msgr2=0x7f0a080a63e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:55.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.578+0000 7f0a23200640 1 --2- 172.21.15.83:0/145694509 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f0a080a3f20 0x7f0a080a63e0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f0a24051000 tx=0x7f0a240699f0 comp rx=0 tx=0).stop 2024-04-24T09:13:55.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.578+0000 7f0a23200640 1 -- 172.21.15.83:0/145694509 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a1c0a4de0 msgr2=0x7f0a1c1496f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:55.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.578+0000 7f0a23200640 1 --2- 172.21.15.83:0/145694509 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a1c0a4de0 0x7f0a1c1496f0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f0a1400e9c0 tx=0x7f0a1400edc0 comp rx=0 tx=0).stop 2024-04-24T09:13:55.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.578+0000 7f0a23200640 1 -- 172.21.15.83:0/145694509 shutdown_connections 2024-04-24T09:13:55.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.578+0000 7f0a23200640 1 --2- 172.21.15.83:0/145694509 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f0a080a3f20 0x7f0a080a63e0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:55.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.578+0000 7f0a23200640 1 --2- 172.21.15.83:0/145694509 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0a1c0b7a70 0x7f0a1c1438a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:55.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.578+0000 7f0a23200640 1 --2- 172.21.15.83:0/145694509 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a1c0a5700 0x7f0a1c149c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:55.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.578+0000 7f0a23200640 1 --2- 172.21.15.83:0/145694509 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a1c0a4de0 0x7f0a1c1496f0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:55.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.578+0000 7f0a23200640 1 -- 172.21.15.83:0/145694509 >> 172.21.15.83:0/145694509 conn(0x7f0a1c01a640 msgr2=0x7f0a1c0b5520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:55.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.578+0000 7f0a23200640 1 -- 172.21.15.83:0/145694509 shutdown_connections 2024-04-24T09:13:55.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:55.579+0000 7f0a23200640 1 -- 172.21.15.83:0/145694509 wait complete. 2024-04-24T09:13:56.201 INFO:tasks.workunit.client.0.smithi083.stdout: 10 62844 6.59 MB/sec execute 387 sec latency 107.637 ms 2024-04-24T09:13:56.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:13:56 smithi138 ceph-mon[31441]: pgmap v704: 129 pgs: 129 active+clean; 971 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.8 MiB/s wr, 224 op/s 2024-04-24T09:13:56.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:13:56 smithi189 ceph-mon[31791]: pgmap v704: 129 pgs: 129 active+clean; 971 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.8 MiB/s wr, 224 op/s 2024-04-24T09:13:56.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:13:56 smithi083 ceph-mon[27907]: pgmap v704: 129 pgs: 129 active+clean; 971 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.8 MiB/s wr, 224 op/s 2024-04-24T09:13:56.602 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T09:13:56.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T09:13:56.603 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T09:13:56.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.885+0000 7fa825c00640 1 -- 172.21.15.83:0/3266227735 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa820104320 msgr2=0x7fa820104700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:56.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.885+0000 7fa825c00640 1 --2- 172.21.15.83:0/3266227735 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa820104320 0x7fa820104700 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fa810009900 tx=0x7fa81002a440 comp rx=0 tx=0).stop 2024-04-24T09:13:56.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.885+0000 7fa825c00640 1 -- 172.21.15.83:0/3266227735 shutdown_connections 2024-04-24T09:13:56.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.885+0000 7fa825c00640 1 --2- 172.21.15.83:0/3266227735 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa8201093f0 0x7fa820110fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:56.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.885+0000 7fa825c00640 1 --2- 172.21.15.83:0/3266227735 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa820104cd0 0x7fa820108d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:56.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.885+0000 7fa825c00640 1 --2- 172.21.15.83:0/3266227735 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa820104320 0x7fa820104700 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:56.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.885+0000 7fa825c00640 1 -- 172.21.15.83:0/3266227735 >> 172.21.15.83:0/3266227735 conn(0x7fa8200ffb40 msgr2=0x7fa820101f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:56.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.885+0000 7fa825c00640 1 -- 172.21.15.83:0/3266227735 shutdown_connections 2024-04-24T09:13:56.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.885+0000 7fa825c00640 1 -- 172.21.15.83:0/3266227735 wait complete. 2024-04-24T09:13:56.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.887+0000 7fa825c00640 1 Processor -- start 2024-04-24T09:13:56.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.887+0000 7fa825c00640 1 -- start start 2024-04-24T09:13:56.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.887+0000 7fa825c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa820104320 0x7fa820110f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:56.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.888+0000 7fa825c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa8201093f0 0x7fa82010e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:56.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.888+0000 7fa825c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa82010e5a0 0x7fa82010ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:56.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.888+0000 7fa81fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa820104320 0x7fa820110f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:56.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.888+0000 7fa81fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa820104320 0x7fa820110f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45192/0 (socket says 172.21.15.83:45192) 2024-04-24T09:13:56.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.888+0000 7fa825c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa820114380 con 0x7fa820104320 2024-04-24T09:13:56.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.888+0000 7fa825c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa820114200 con 0x7fa8201093f0 2024-04-24T09:13:56.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.888+0000 7fa825c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa820114500 con 0x7fa82010e5a0 2024-04-24T09:13:56.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.888+0000 7fa81fe00640 1 -- 172.21.15.83:0/3865927207 learned_addr learned my addr 172.21.15.83:0/3865927207 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:13:56.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.888+0000 7fa81f400640 1 --2- 172.21.15.83:0/3865927207 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa8201093f0 0x7fa82010e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:56.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.889+0000 7fa81f400640 1 -- 172.21.15.83:0/3865927207 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa82010e5a0 msgr2=0x7fa82010ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:56.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.889+0000 7fa81f400640 1 --2- 172.21.15.83:0/3865927207 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa82010e5a0 0x7fa82010ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:56.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.889+0000 7fa81f400640 1 -- 172.21.15.83:0/3865927207 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa820104320 msgr2=0x7fa820110f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:56.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.889+0000 7fa81f400640 1 --2- 172.21.15.83:0/3865927207 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa820104320 0x7fa820110f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:56.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.889+0000 7fa81f400640 1 -- 172.21.15.83:0/3865927207 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8201ab350 con 0x7fa8201093f0 2024-04-24T09:13:56.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.889+0000 7fa81f400640 1 --2- 172.21.15.83:0/3865927207 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa8201093f0 0x7fa82010e060 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fa80c00e9c0 tx=0x7fa80c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:56.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.890+0000 7fa81cc00640 1 -- 172.21.15.83:0/3865927207 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa80c00cd80 con 0x7fa8201093f0 2024-04-24T09:13:56.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.890+0000 7fa81cc00640 1 -- 172.21.15.83:0/3865927207 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa80c0035f0 con 0x7fa8201093f0 2024-04-24T09:13:56.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.890+0000 7fa81cc00640 1 -- 172.21.15.83:0/3865927207 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa80c010690 con 0x7fa8201093f0 2024-04-24T09:13:56.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.890+0000 7fa825c00640 1 -- 172.21.15.83:0/3865927207 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8201ab640 con 0x7fa8201093f0 2024-04-24T09:13:56.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.890+0000 7fa825c00640 1 -- 172.21.15.83:0/3865927207 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8201abb40 con 0x7fa8201093f0 2024-04-24T09:13:56.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.895+0000 7fa82a1b8740 1 Processor -- start 2024-04-24T09:13:56.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.895+0000 7fa82a1b8740 1 -- start start 2024-04-24T09:13:56.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.896+0000 7fa82a1b8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a88de6b270 0x55a88de6b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:56.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.896+0000 7fa82a1b8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a88de6bb90 0x55a88de75d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:56.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.896+0000 7fa82a1b8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a88de76260 0x55a88de78650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:56.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.896+0000 7fa82a1b8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a88ddd0e70 con 0x55a88de6bb90 2024-04-24T09:13:56.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.896+0000 7fa82a1b8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a88dd77380 con 0x55a88de6b270 2024-04-24T09:13:56.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.896+0000 7fa82a1b8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a88ddcce30 con 0x55a88de76260 2024-04-24T09:13:56.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.896+0000 7fa825200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a88de6b270 0x55a88de6b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:56.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.896+0000 7fa825200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a88de6b270 0x55a88de6b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:60882/0 (socket says 172.21.15.83:60882) 2024-04-24T09:13:56.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.896+0000 7fa825200640 1 -- 172.21.15.83:0/1539021570 learned_addr learned my addr 172.21.15.83:0/1539021570 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:13:56.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.896+0000 7fa825200640 1 -- 172.21.15.83:0/1539021570 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a88de76260 msgr2=0x55a88de78650 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T09:13:56.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.896+0000 7fa825200640 1 --2- 172.21.15.83:0/1539021570 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a88de76260 0x55a88de78650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:56.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.896+0000 7fa825200640 1 -- 172.21.15.83:0/1539021570 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a88de6bb90 msgr2=0x55a88de75d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:56.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.896+0000 7fa825200640 1 --2- 172.21.15.83:0/1539021570 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a88de6bb90 0x55a88de75d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:56.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.896+0000 7fa825200640 1 -- 172.21.15.83:0/1539021570 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a88dbab7e0 con 0x55a88de6b270 2024-04-24T09:13:56.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.897+0000 7fa825200640 1 --2- 172.21.15.83:0/1539021570 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a88de6b270 0x55a88de6b650 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fa818009d50 tx=0x7fa81800d6b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3947d6ded1b8f784 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:56.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.897+0000 7fa815600640 1 -- 172.21.15.83:0/1539021570 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa818003970 con 0x55a88de6b270 2024-04-24T09:13:56.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.898+0000 7fa815600640 1 -- 172.21.15.83:0/1539021570 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa818004020 con 0x55a88de6b270 2024-04-24T09:13:56.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.898+0000 7fa815600640 1 -- 172.21.15.83:0/1539021570 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa818010700 con 0x55a88de6b270 2024-04-24T09:13:56.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.898+0000 7fa81cc00640 1 -- 172.21.15.83:0/3865927207 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa80c0037a0 con 0x7fa8201093f0 2024-04-24T09:13:56.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.899+0000 7fa82a1b8740 1 -- 172.21.15.83:0/1539021570 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a88de6b270 msgr2=0x55a88de6b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:56.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.899+0000 7fa82a1b8740 1 --2- 172.21.15.83:0/1539021570 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a88de6b270 0x55a88de6b650 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fa818009d50 tx=0x7fa81800d6b0 comp rx=0 tx=0).stop 2024-04-24T09:13:56.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.900+0000 7fa82a1b8740 1 -- 172.21.15.83:0/1539021570 shutdown_connections 2024-04-24T09:13:56.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.900+0000 7fa82a1b8740 1 --2- 172.21.15.83:0/1539021570 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a88de76260 0x55a88de78650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:56.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.900+0000 7fa82a1b8740 1 --2- 172.21.15.83:0/1539021570 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a88de6bb90 0x55a88de75d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:56.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.900+0000 7fa82a1b8740 1 --2- 172.21.15.83:0/1539021570 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a88de6b270 0x55a88de6b650 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:56.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.900+0000 7fa82a1b8740 1 -- 172.21.15.83:0/1539021570 >> 172.21.15.83:0/1539021570 conn(0x55a88de67590 msgr2=0x55a88de67970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:56.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.901+0000 7fa82a1b8740 1 -- 172.21.15.83:0/1539021570 shutdown_connections 2024-04-24T09:13:56.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.901+0000 7fa81cc00640 1 --2- 172.21.15.83:0/3865927207 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa8040a3e00 0x7fa8040a62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:56.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.902+0000 7fa81cc00640 1 -- 172.21.15.83:0/3865927207 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 9583+0+0 (secure 0 0 0) 0x7fa80c0d0ef0 con 0x7fa8201093f0 2024-04-24T09:13:56.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.902+0000 7fa82a1b8740 1 -- 172.21.15.83:0/1539021570 wait complete. 2024-04-24T09:13:56.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.903+0000 7fa82a1b8740 1 -- start start 2024-04-24T09:13:56.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.903+0000 7fa82a1b8740 1 Processor -- start 2024-04-24T09:13:56.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.903+0000 7fa81fe00640 1 --2- 172.21.15.83:0/3865927207 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa8040a3e00 0x7fa8040a62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:56.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.905+0000 7fa81fe00640 1 --2- 172.21.15.83:0/3865927207 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa8040a3e00 0x7fa8040a62c0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x55a88de6c830 tx=0x7fa810005850 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:56.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.905+0000 7fa82a1b8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a88de6b270 0x55a88de73210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:56.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.905+0000 7fa82a1b8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a88de6bb90 0x55a88de73750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:56.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.905+0000 7fa82a1b8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a88de76260 0x55a88de73c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:56.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.905+0000 7fa81fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a88de6b270 0x55a88de73210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:56.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.905+0000 7fa81fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a88de6b270 0x55a88de73210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:60894/0 (socket says 172.21.15.83:60894) 2024-04-24T09:13:56.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.905+0000 7fa81fe00640 1 -- 172.21.15.83:0/1916312160 learned_addr learned my addr 172.21.15.83:0/1916312160 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:13:56.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.905+0000 7fa82a1b8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a88de79050 con 0x55a88de6bb90 2024-04-24T09:13:56.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.906+0000 7fa82a1b8740 1 -- 172.21.15.83:0/1916312160 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a88dd8a660 con 0x55a88de6b270 2024-04-24T09:13:56.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.906+0000 7fa82a1b8740 1 -- 172.21.15.83:0/1916312160 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a88de791d0 con 0x55a88de76260 2024-04-24T09:13:56.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.906+0000 7fa82a1b8740 10 client.0 _ll_register_callbacks cb 0x55a88dd78aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:13:56.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.906+0000 7fa81f400640 1 --2- 172.21.15.83:0/1916312160 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a88de6bb90 0x55a88de73750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:56.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.906+0000 7fa825200640 1 --2- 172.21.15.83:0/1916312160 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a88de76260 0x55a88de73c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:56.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.906+0000 7fa81f400640 1 -- 172.21.15.83:0/1916312160 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a88de76260 msgr2=0x55a88de73c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:56.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.906+0000 7fa81f400640 1 --2- 172.21.15.83:0/1916312160 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a88de76260 0x55a88de73c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:56.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.906+0000 7fa81f400640 1 -- 172.21.15.83:0/1916312160 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a88de6b270 msgr2=0x55a88de73210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:56.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.906+0000 7fa81f400640 1 --2- 172.21.15.83:0/1916312160 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a88de6b270 0x55a88de73210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:56.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.906+0000 7fa81f400640 1 -- 172.21.15.83:0/1916312160 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a88dc0bf70 con 0x55a88de6bb90 2024-04-24T09:13:56.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.907+0000 7fa80a000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:13:56.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.907+0000 7fa81f400640 1 --2- 172.21.15.83:0/1916312160 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a88de6bb90 0x55a88de73750 secure :-1 s=READY pgs=2645 cs=0 l=1 rev1=1 crypto rx=0x7fa80c0ce190 tx=0x7fa80c09c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29c109bc59c8f41e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:56.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.907+0000 7fa80a000640 1 -- 172.21.15.83:0/1916312160 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa80c0a0dc0 con 0x55a88de6bb90 2024-04-24T09:13:56.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.907+0000 7fa80a000640 1 -- 172.21.15.83:0/1916312160 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9583+0+0 (secure 0 0 0) 0x7fa80c099080 con 0x55a88de6bb90 2024-04-24T09:13:56.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.908+0000 7fa82a1b8740 1 -- 172.21.15.83:0/1916312160 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa818004020 con 0x55a88de6bb90 2024-04-24T09:13:56.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.908+0000 7fa80a000640 1 -- 172.21.15.83:0/1916312160 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7fa80c0a0dc0 con 0x55a88de6bb90 2024-04-24T09:13:56.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.908+0000 7fa82a1b8740 10 client.23223 fetch_fsmap learned FSMap version 64 2024-04-24T09:13:56.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.908+0000 7fa82a1b8740 1 -- 172.21.15.83:0/1916312160 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55a88daa3170 con 0x55a88de6bb90 2024-04-24T09:13:56.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.909+0000 7fa80a000640 1 -- 172.21.15.83:0/1916312160 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa818004020 con 0x55a88de6bb90 2024-04-24T09:13:56.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.910+0000 7fa80a000640 1 -- 172.21.15.83:0/1916312160 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 64) ==== 8374+0+0 (secure 0 0 0) 0x7fa80c0bbbd0 con 0x55a88de6bb90 2024-04-24T09:13:56.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.910+0000 7fa82a1b8740 10 client.23223 fetch_fsmap finished waiting for FSMap version 64 2024-04-24T09:13:56.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.910+0000 7fa82a1b8740 10 client.23223 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:13:56.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.910+0000 7fa82a1b8740 20 client.23223 populate_metadata read hostname 'smithi083' 2024-04-24T09:13:56.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.910+0000 7fa82a1b8740 1 --2- 172.21.15.83:0/1916312160 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55a88de866b0 0x55a88de88aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:56.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.910+0000 7fa82a1b8740 4 client.23223 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:13:56.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.910+0000 7fa82a1b8740 1 -- 172.21.15.83:0/1916312160 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a88ddde890 con 0x55a88de866b0 2024-04-24T09:13:56.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.910+0000 7fa81fe00640 1 --2- 172.21.15.83:0/1916312160 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55a88de866b0 0x55a88de88aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:56.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.911+0000 7fa81fe00640 1 --2- 172.21.15.83:0/1916312160 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55a88de866b0 0x55a88de88aa0 crc :-1 s=READY pgs=551 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c676c06522caa842 server_cookie=dd7b7a43354555b0 in_seq=0 out_seq=0 2024-04-24T09:13:56.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.911+0000 7fa80a000640 10 client.23223 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:13:56.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.912+0000 7fa80a000640 1 -- 172.21.15.83:0/1916312160 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a88ddde890 con 0x55a88de866b0 2024-04-24T09:13:56.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.912+0000 7fa80a000640 10 client.23223 handle_command_reply: tid=0 2024-04-24T09:13:56.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.913+0000 7fa82a1b8740 1 client.23223 shutdown 2024-04-24T09:13:56.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.913+0000 7fa82a1b8740 1 -- 172.21.15.83:0/1916312160 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a88de6bb90 msgr2=0x55a88de73750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:56.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.913+0000 7fa82a1b8740 1 --2- 172.21.15.83:0/1916312160 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a88de6bb90 0x55a88de73750 secure :-1 s=READY pgs=2645 cs=0 l=1 rev1=1 crypto rx=0x7fa80c0ce190 tx=0x7fa80c09c460 comp rx=0 tx=0).stop 2024-04-24T09:13:56.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.913+0000 7fa82a1b8740 1 -- 172.21.15.83:0/1916312160 shutdown_connections 2024-04-24T09:13:56.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.913+0000 7fa82a1b8740 1 --2- 172.21.15.83:0/1916312160 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55a88de866b0 0x55a88de88aa0 crc :-1 s=READY pgs=551 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:56.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.913+0000 7fa82a1b8740 1 --2- 172.21.15.83:0/1916312160 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a88de76260 0x55a88de73c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:56.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.913+0000 7fa82a1b8740 1 --2- 172.21.15.83:0/1916312160 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a88de6bb90 0x55a88de73750 unknown :-1 s=CLOSED pgs=2645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:56.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.913+0000 7fa82a1b8740 1 --2- 172.21.15.83:0/1916312160 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a88de6b270 0x55a88de73210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:56.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.913+0000 7fa82a1b8740 1 -- 172.21.15.83:0/1916312160 >> 172.21.15.83:0/1916312160 conn(0x55a88de67590 msgr2=0x55a88de6d290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:56.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.913+0000 7fa80a000640 0 client.23223 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:13:56.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.914+0000 7fa82a1b8740 1 -- 172.21.15.83:0/1916312160 shutdown_connections 2024-04-24T09:13:56.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.914+0000 7fa82a1b8740 1 -- 172.21.15.83:0/1916312160 wait complete. 2024-04-24T09:13:56.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.914+0000 7fa82a1b8740 20 client.23223 trim_cache size 0 max 16384 2024-04-24T09:13:56.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.948+0000 7fa82a1b8740 1 Processor -- start 2024-04-24T09:13:56.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.948+0000 7fa82a1b8740 1 -- start start 2024-04-24T09:13:56.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.949+0000 7fa82a1b8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a88de6b270 0x55a88de81f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:56.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.949+0000 7fa82a1b8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a88de6bb90 0x55a88de6d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:56.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.949+0000 7fa82a1b8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a88de76260 0x55a88de8e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:56.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.949+0000 7fa82a1b8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa8201b5b30 con 0x55a88de6bb90 2024-04-24T09:13:56.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.949+0000 7fa82a1b8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa8201b7700 con 0x55a88de76260 2024-04-24T09:13:56.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.949+0000 7fa82a1b8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa8201b6200 con 0x55a88de6b270 2024-04-24T09:13:56.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.950+0000 7fa81fe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a88de6b270 0x55a88de81f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:56.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.950+0000 7fa81fe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a88de6b270 0x55a88de81f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:41218/0 (socket says 172.21.15.83:41218) 2024-04-24T09:13:56.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.950+0000 7fa81fe00640 1 -- 172.21.15.83:0/1097446283 learned_addr learned my addr 172.21.15.83:0/1097446283 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:13:56.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.950+0000 7fa81f400640 1 --2- 172.21.15.83:0/1097446283 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a88de6bb90 0x55a88de6d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:56.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.950+0000 7fa825200640 1 --2- 172.21.15.83:0/1097446283 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a88de76260 0x55a88de8e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:56.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.950+0000 7fa81fe00640 1 -- 172.21.15.83:0/1097446283 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a88de76260 msgr2=0x55a88de8e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:56.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.950+0000 7fa81fe00640 1 --2- 172.21.15.83:0/1097446283 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a88de76260 0x55a88de8e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:56.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.950+0000 7fa81fe00640 1 -- 172.21.15.83:0/1097446283 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a88de6bb90 msgr2=0x55a88de6d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:56.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.950+0000 7fa81fe00640 1 --2- 172.21.15.83:0/1097446283 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a88de6bb90 0x55a88de6d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:56.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.950+0000 7fa81fe00640 1 -- 172.21.15.83:0/1097446283 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a88de6eea0 con 0x55a88de6b270 2024-04-24T09:13:56.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.951+0000 7fa81f400640 1 --2- 172.21.15.83:0/1097446283 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a88de6bb90 0x55a88de6d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:13:56.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.951+0000 7fa81fe00640 1 --2- 172.21.15.83:0/1097446283 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a88de6b270 0x55a88de81f10 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa8100058c0 tx=0x7fa81003f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=729131ad6a65e26a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:56.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.952+0000 7fa814c00640 1 -- 172.21.15.83:0/1097446283 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa81004c110 con 0x55a88de6b270 2024-04-24T09:13:56.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.952+0000 7fa814c00640 1 -- 172.21.15.83:0/1097446283 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa81004cd40 con 0x55a88de6b270 2024-04-24T09:13:56.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.952+0000 7fa814c00640 1 -- 172.21.15.83:0/1097446283 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa810031440 con 0x55a88de6b270 2024-04-24T09:13:56.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.953+0000 7fa82a1b8740 1 -- 172.21.15.83:0/1097446283 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a88de6b270 msgr2=0x55a88de81f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:56.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.953+0000 7fa82a1b8740 1 --2- 172.21.15.83:0/1097446283 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a88de6b270 0x55a88de81f10 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa8100058c0 tx=0x7fa81003f000 comp rx=0 tx=0).stop 2024-04-24T09:13:56.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.953+0000 7fa82a1b8740 1 -- 172.21.15.83:0/1097446283 shutdown_connections 2024-04-24T09:13:56.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.953+0000 7fa82a1b8740 1 --2- 172.21.15.83:0/1097446283 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a88de76260 0x55a88de8e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:56.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.953+0000 7fa82a1b8740 1 --2- 172.21.15.83:0/1097446283 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a88de6bb90 0x55a88de6d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:56.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.953+0000 7fa82a1b8740 1 --2- 172.21.15.83:0/1097446283 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a88de6b270 0x55a88de81f10 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:56.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.953+0000 7fa82a1b8740 1 -- 172.21.15.83:0/1097446283 >> 172.21.15.83:0/1097446283 conn(0x55a88de67590 msgr2=0x55a88de74a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:56.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.954+0000 7fa82a1b8740 1 -- 172.21.15.83:0/1097446283 shutdown_connections 2024-04-24T09:13:56.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.954+0000 7fa82a1b8740 1 -- 172.21.15.83:0/1097446283 wait complete. 2024-04-24T09:13:56.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.954+0000 7fa82a1b8740 1 -- start start 2024-04-24T09:13:56.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.954+0000 7fa82a1b8740 1 Processor -- start 2024-04-24T09:13:56.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.955+0000 7fa82a1b8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a88de6b270 0x55a88de7e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:56.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.955+0000 7fa82a1b8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a88de6bb90 0x55a88de7e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:56.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.955+0000 7fa82a1b8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a88de76260 0x55a88de7b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:56.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.955+0000 7fa82a1b8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa8201b7000 con 0x55a88de6bb90 2024-04-24T09:13:56.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.955+0000 7fa82a1b8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa8201b6930 con 0x55a88de76260 2024-04-24T09:13:56.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.955+0000 7fa82a1b8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa8201b3b30 con 0x55a88de6b270 2024-04-24T09:13:56.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.955+0000 7fa82a1b8740 10 client.0 _ll_register_callbacks cb 0x7fa80c003d20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:13:56.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.955+0000 7fa81f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a88de6bb90 0x55a88de7e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:56.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.955+0000 7fa81f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a88de6bb90 0x55a88de7e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45220/0 (socket says 172.21.15.83:45220) 2024-04-24T09:13:56.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.955+0000 7fa81f400640 1 -- 172.21.15.83:0/1603624766 learned_addr learned my addr 172.21.15.83:0/1603624766 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:13:56.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.956+0000 7fa825200640 1 --2- 172.21.15.83:0/1603624766 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a88de76260 0x55a88de7b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:56.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.956+0000 7fa81f400640 1 -- 172.21.15.83:0/1603624766 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a88de6b270 msgr2=0x55a88de7e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:56.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.956+0000 7fa81f400640 1 --2- 172.21.15.83:0/1603624766 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a88de6b270 0x55a88de7e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:56.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.956+0000 7fa81f400640 1 -- 172.21.15.83:0/1603624766 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a88de76260 msgr2=0x55a88de7b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:56.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.956+0000 7fa81f400640 1 --2- 172.21.15.83:0/1603624766 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a88de76260 0x55a88de7b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:56.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.956+0000 7fa81f400640 1 -- 172.21.15.83:0/1603624766 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a88de7ba30 con 0x55a88de6bb90 2024-04-24T09:13:56.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.956+0000 7fa81f400640 1 --2- 172.21.15.83:0/1603624766 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a88de6bb90 0x55a88de7e960 secure :-1 s=READY pgs=2646 cs=0 l=1 rev1=1 crypto rx=0x7fa80c09c710 tx=0x7fa80c098760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2bb3ca6fa2cc864 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:56.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.956+0000 7fa80a000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:13:56.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.957+0000 7fa80a000640 1 -- 172.21.15.83:0/1603624766 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa80c09c550 con 0x55a88de6bb90 2024-04-24T09:13:56.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.957+0000 7fa80a000640 1 -- 172.21.15.83:0/1603624766 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9583+0+0 (secure 0 0 0) 0x7fa80c0ce440 con 0x55a88de6bb90 2024-04-24T09:13:56.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.957+0000 7fa82a1b8740 1 -- 172.21.15.83:0/1603624766 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa81004cd40 con 0x55a88de6bb90 2024-04-24T09:13:56.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.958+0000 7fa80a000640 1 -- 172.21.15.83:0/1603624766 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7fa80c0ce640 con 0x55a88de6bb90 2024-04-24T09:13:56.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.958+0000 7fa82a1b8740 10 client.23229 fetch_fsmap learned FSMap version 64 2024-04-24T09:13:56.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.958+0000 7fa82a1b8740 1 -- 172.21.15.83:0/1603624766 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55a88deed530 con 0x55a88de6bb90 2024-04-24T09:13:56.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.958+0000 7fa80a000640 1 -- 172.21.15.83:0/1603624766 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa81004cd40 con 0x55a88de6bb90 2024-04-24T09:13:56.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.959+0000 7fa80a000640 1 -- 172.21.15.83:0/1603624766 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 64) ==== 8374+0+0 (secure 0 0 0) 0x7fa80c0ad020 con 0x55a88de6bb90 2024-04-24T09:13:56.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.959+0000 7fa82a1b8740 10 client.23229 fetch_fsmap finished waiting for FSMap version 64 2024-04-24T09:13:56.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.959+0000 7fa82a1b8740 10 client.23229 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:13:56.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.959+0000 7fa82a1b8740 20 client.23229 populate_metadata read hostname 'smithi083' 2024-04-24T09:13:56.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.959+0000 7fa82a1b8740 1 --2- 172.21.15.83:0/1603624766 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55a88deedca0 0x55a88def00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:56.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.959+0000 7fa82a1b8740 4 client.23229 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T09:13:56.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.959+0000 7fa82a1b8740 1 -- 172.21.15.83:0/1603624766 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a88ddd8700 con 0x55a88deedca0 2024-04-24T09:13:56.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.959+0000 7fa81fe00640 1 --2- 172.21.15.83:0/1603624766 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55a88deedca0 0x55a88def00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:56.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.960+0000 7fa81fe00640 1 --2- 172.21.15.83:0/1603624766 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55a88deedca0 0x55a88def00c0 crc :-1 s=READY pgs=552 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45d6a9bc88e9d439 server_cookie=75014ac6fba532c9 in_seq=0 out_seq=0 2024-04-24T09:13:56.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.960+0000 7fa80a000640 10 client.23229 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:13:56.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.962+0000 7fa80a000640 1 -- 172.21.15.83:0/1603624766 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a88ddd8700 con 0x55a88deedca0 2024-04-24T09:13:56.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.962+0000 7fa80a000640 10 client.23229 handle_command_reply: tid=0 2024-04-24T09:13:56.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.963+0000 7fa82a1b8740 1 client.23229 shutdown 2024-04-24T09:13:56.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.963+0000 7fa82a1b8740 1 -- 172.21.15.83:0/1603624766 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a88de6bb90 msgr2=0x55a88de7e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:56.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.963+0000 7fa82a1b8740 1 --2- 172.21.15.83:0/1603624766 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a88de6bb90 0x55a88de7e960 secure :-1 s=READY pgs=2646 cs=0 l=1 rev1=1 crypto rx=0x7fa80c09c710 tx=0x7fa80c098760 comp rx=0 tx=0).stop 2024-04-24T09:13:56.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.963+0000 7fa82a1b8740 1 -- 172.21.15.83:0/1603624766 shutdown_connections 2024-04-24T09:13:56.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.963+0000 7fa82a1b8740 1 --2- 172.21.15.83:0/1603624766 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55a88deedca0 0x55a88def00c0 crc :-1 s=READY pgs=552 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:56.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.963+0000 7fa82a1b8740 1 --2- 172.21.15.83:0/1603624766 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a88de76260 0x55a88de7b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:56.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.963+0000 7fa82a1b8740 1 --2- 172.21.15.83:0/1603624766 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a88de6bb90 0x55a88de7e960 unknown :-1 s=CLOSED pgs=2646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:56.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.963+0000 7fa82a1b8740 1 --2- 172.21.15.83:0/1603624766 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a88de6b270 0x55a88de7e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:56.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.963+0000 7fa82a1b8740 1 -- 172.21.15.83:0/1603624766 >> 172.21.15.83:0/1603624766 conn(0x55a88de67590 msgr2=0x55a88de752d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:56.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.963+0000 7fa80a000640 0 client.23229 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:13:56.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.963+0000 7fa82a1b8740 1 -- 172.21.15.83:0/1603624766 shutdown_connections 2024-04-24T09:13:56.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.964+0000 7fa82a1b8740 1 -- 172.21.15.83:0/1603624766 wait complete. 2024-04-24T09:13:56.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.964+0000 7fa82a1b8740 20 client.23229 trim_cache size 0 max 16384 2024-04-24T09:13:56.965 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T09:13:56.965 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-04-24T09:13:56.965 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "64546e1c-5611-455f-b73a-ad4a542174cc", 2024-04-24T09:13:56.965 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-04-24T09:13:56.965 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T09:13:56.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.965+0000 7fa825c00640 1 -- 172.21.15.83:0/3865927207 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa8040a3e00 msgr2=0x7fa8040a62c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:56.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.965+0000 7fa825c00640 1 --2- 172.21.15.83:0/3865927207 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa8040a3e00 0x7fa8040a62c0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x55a88de6c830 tx=0x7fa810005850 comp rx=0 tx=0).stop 2024-04-24T09:13:56.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.965+0000 7fa825c00640 1 -- 172.21.15.83:0/3865927207 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa8201093f0 msgr2=0x7fa82010e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:56.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.965+0000 7fa825c00640 1 --2- 172.21.15.83:0/3865927207 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa8201093f0 0x7fa82010e060 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fa80c00e9c0 tx=0x7fa80c00ee80 comp rx=0 tx=0).stop 2024-04-24T09:13:56.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.965+0000 7fa825c00640 1 -- 172.21.15.83:0/3865927207 shutdown_connections 2024-04-24T09:13:56.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.965+0000 7fa825c00640 1 --2- 172.21.15.83:0/3865927207 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa8040a3e00 0x7fa8040a62c0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:56.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.965+0000 7fa825c00640 1 --2- 172.21.15.83:0/3865927207 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa82010e5a0 0x7fa82010ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:56.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.965+0000 7fa825c00640 1 --2- 172.21.15.83:0/3865927207 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa8201093f0 0x7fa82010e060 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:56.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.965+0000 7fa825c00640 1 --2- 172.21.15.83:0/3865927207 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa820104320 0x7fa820110f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:56.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.965+0000 7fa825c00640 1 -- 172.21.15.83:0/3865927207 >> 172.21.15.83:0/3865927207 conn(0x7fa8200ffb40 msgr2=0x7fa820100f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:56.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.965+0000 7fa825c00640 1 -- 172.21.15.83:0/3865927207 shutdown_connections 2024-04-24T09:13:56.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:56.965+0000 7fa825c00640 1 -- 172.21.15.83:0/3865927207 wait complete. 2024-04-24T09:13:56.995 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T09:13:56.995 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T09:13:57.201 INFO:tasks.workunit.client.0.smithi083.stdout: 10 62939 6.58 MB/sec execute 388 sec latency 118.418 ms 2024-04-24T09:13:57.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.248+0000 7f817a000640 1 -- 172.21.15.83:0/1198529623 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8174074bf0 msgr2=0x7f817410e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:57.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.248+0000 7f817a000640 1 --2- 172.21.15.83:0/1198529623 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8174074bf0 0x7f817410e3a0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f8160009860 tx=0x7f816002a3d0 comp rx=0 tx=0).stop 2024-04-24T09:13:57.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.248+0000 7f817a000640 1 -- 172.21.15.83:0/1198529623 shutdown_connections 2024-04-24T09:13:57.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.248+0000 7f817a000640 1 --2- 172.21.15.83:0/1198529623 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f817410e8e0 0x7f8174110cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:57.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.248+0000 7f817a000640 1 --2- 172.21.15.83:0/1198529623 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8174074bf0 0x7f817410e3a0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:57.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.248+0000 7f817a000640 1 --2- 172.21.15.83:0/1198529623 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81740742d0 0x7f81740746b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:57.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.248+0000 7f817a000640 1 -- 172.21.15.83:0/1198529623 >> 172.21.15.83:0/1198529623 conn(0x7f8174079450 msgr2=0x7f8174079860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:57.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.248+0000 7f817a000640 1 -- 172.21.15.83:0/1198529623 shutdown_connections 2024-04-24T09:13:57.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.249+0000 7f817a000640 1 -- 172.21.15.83:0/1198529623 wait complete. 2024-04-24T09:13:57.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.250+0000 7f817a000640 1 Processor -- start 2024-04-24T09:13:57.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.250+0000 7f817a000640 1 -- start start 2024-04-24T09:13:57.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.250+0000 7f817a000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81740742d0 0x7f8174107ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:57.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.251+0000 7f817a000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8174074bf0 0x7f8174108010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:57.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.251+0000 7f817a000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f817410e8e0 0x7f8174102be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:57.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.251+0000 7f817a000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8174114480 con 0x7f81740742d0 2024-04-24T09:13:57.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.251+0000 7f817a000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8174114300 con 0x7f8174074bf0 2024-04-24T09:13:57.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.251+0000 7f817a000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8174114600 con 0x7f817410e8e0 2024-04-24T09:13:57.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.251+0000 7f8173e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f817410e8e0 0x7f8174102be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:57.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.251+0000 7f8173e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f817410e8e0 0x7f8174102be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:41254/0 (socket says 172.21.15.83:41254) 2024-04-24T09:13:57.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.252+0000 7f8173400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81740742d0 0x7f8174107ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:57.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.252+0000 7f8173e00640 1 -- 172.21.15.83:0/363311489 learned_addr learned my addr 172.21.15.83:0/363311489 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:13:57.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.252+0000 7f8173e00640 1 -- 172.21.15.83:0/363311489 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8174074bf0 msgr2=0x7f8174108010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:57.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.252+0000 7f8172a00640 1 --2- 172.21.15.83:0/363311489 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8174074bf0 0x7f8174108010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:57.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.252+0000 7f8173e00640 1 --2- 172.21.15.83:0/363311489 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8174074bf0 0x7f8174108010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:57.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.252+0000 7f8173e00640 1 -- 172.21.15.83:0/363311489 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81740742d0 msgr2=0x7f8174107ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:57.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.252+0000 7f8173e00640 1 --2- 172.21.15.83:0/363311489 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81740742d0 0x7f8174107ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:57.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.252+0000 7f8173e00640 1 -- 172.21.15.83:0/363311489 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81741034a0 con 0x7f817410e8e0 2024-04-24T09:13:57.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.253+0000 7f8173400640 1 --2- 172.21.15.83:0/363311489 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81740742d0 0x7f8174107ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:13:57.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.253+0000 7f8172a00640 1 --2- 172.21.15.83:0/363311489 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8174074bf0 0x7f8174108010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:13:57.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.253+0000 7f8173e00640 1 --2- 172.21.15.83:0/363311489 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f817410e8e0 0x7f8174102be0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f816c00e9d0 tx=0x7f816c00ee90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:57.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.253+0000 7f816be00640 1 -- 172.21.15.83:0/363311489 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f816c00cd80 con 0x7f817410e8e0 2024-04-24T09:13:57.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.254+0000 7f817a000640 1 -- 172.21.15.83:0/363311489 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8174103790 con 0x7f817410e8e0 2024-04-24T09:13:57.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.254+0000 7f816be00640 1 -- 172.21.15.83:0/363311489 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f816c005020 con 0x7f817410e8e0 2024-04-24T09:13:57.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.254+0000 7f816be00640 1 -- 172.21.15.83:0/363311489 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f816c0116d0 con 0x7f817410e8e0 2024-04-24T09:13:57.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.254+0000 7f817a000640 1 -- 172.21.15.83:0/363311489 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f817410d9c0 con 0x7f817410e8e0 2024-04-24T09:13:57.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.258+0000 7f817cf92740 1 Processor -- start 2024-04-24T09:13:57.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.259+0000 7f816be00640 1 -- 172.21.15.83:0/363311489 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f816c0051d0 con 0x7f817410e8e0 2024-04-24T09:13:57.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.259+0000 7f817cf92740 1 -- start start 2024-04-24T09:13:57.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.260+0000 7f817cf92740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581508ea860 0x5581508eac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:57.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.260+0000 7f817cf92740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5581508eb180 0x5581508f5310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:57.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.260+0000 7f817cf92740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5581508f5850 0x5581508f7c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:57.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.260+0000 7f817cf92740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5581508501f0 con 0x5581508ea860 2024-04-24T09:13:57.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.260+0000 7f817cf92740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5581507f6350 con 0x5581508f5850 2024-04-24T09:13:57.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.260+0000 7f817cf92740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55815084c1b0 con 0x5581508eb180 2024-04-24T09:13:57.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.261+0000 7f8173400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5581508eb180 0x5581508f5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:57.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.261+0000 7f8173400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5581508eb180 0x5581508f5310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:41264/0 (socket says 172.21.15.83:41264) 2024-04-24T09:13:57.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.261+0000 7f8173400640 1 -- 172.21.15.83:0/2784050710 learned_addr learned my addr 172.21.15.83:0/2784050710 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:13:57.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.261+0000 7f8172a00640 1 --2- 172.21.15.83:0/2784050710 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581508ea860 0x5581508eac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:57.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.261+0000 7f8173400640 1 -- 172.21.15.83:0/2784050710 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5581508f5850 msgr2=0x5581508f7c40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T09:13:57.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.261+0000 7f8173400640 1 --2- 172.21.15.83:0/2784050710 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5581508f5850 0x5581508f7c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:57.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.261+0000 7f8173400640 1 -- 172.21.15.83:0/2784050710 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581508ea860 msgr2=0x5581508eac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:57.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.261+0000 7f8173400640 1 --2- 172.21.15.83:0/2784050710 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581508ea860 0x5581508eac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:57.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.261+0000 7f8173400640 1 -- 172.21.15.83:0/2784050710 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55815084af50 con 0x5581508eb180 2024-04-24T09:13:57.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.262+0000 7f8173400640 1 --2- 172.21.15.83:0/2784050710 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5581508eb180 0x5581508f5310 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f81640114f0 tx=0x7f81640119b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=77f03826182d7041 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:57.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.262+0000 7f816be00640 1 --2- 172.21.15.83:0/363311489 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f81580a3d20 0x7f81580a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:57.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.262+0000 7f815fe00640 1 -- 172.21.15.83:0/2784050710 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8164005020 con 0x5581508eb180 2024-04-24T09:13:57.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.262+0000 7f816be00640 1 -- 172.21.15.83:0/363311489 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 9583+0+0 (secure 0 0 0) 0x7f816c0d0e30 con 0x7f817410e8e0 2024-04-24T09:13:57.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.263+0000 7f815fe00640 1 -- 172.21.15.83:0/2784050710 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f81640047a0 con 0x5581508eb180 2024-04-24T09:13:57.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.263+0000 7f815fe00640 1 -- 172.21.15.83:0/2784050710 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f816400f820 con 0x5581508eb180 2024-04-24T09:13:57.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.263+0000 7f8172a00640 1 --2- 172.21.15.83:0/363311489 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f81580a3d20 0x7f81580a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:57.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.263+0000 7f817cf92740 1 -- 172.21.15.83:0/2784050710 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5581508eb180 msgr2=0x5581508f5310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:57.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.263+0000 7f817cf92740 1 --2- 172.21.15.83:0/2784050710 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5581508eb180 0x5581508f5310 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f81640114f0 tx=0x7f81640119b0 comp rx=0 tx=0).stop 2024-04-24T09:13:57.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.264+0000 7f8172a00640 1 --2- 172.21.15.83:0/363311489 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f81580a3d20 0x7f81580a61e0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f8160000c00 tx=0x7f8160005c90 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:57.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.264+0000 7f817cf92740 1 -- 172.21.15.83:0/2784050710 shutdown_connections 2024-04-24T09:13:57.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.264+0000 7f817cf92740 1 --2- 172.21.15.83:0/2784050710 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5581508f5850 0x5581508f7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:57.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.264+0000 7f817cf92740 1 --2- 172.21.15.83:0/2784050710 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5581508eb180 0x5581508f5310 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:57.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.264+0000 7f817cf92740 1 --2- 172.21.15.83:0/2784050710 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581508ea860 0x5581508eac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:57.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.264+0000 7f817cf92740 1 -- 172.21.15.83:0/2784050710 >> 172.21.15.83:0/2784050710 conn(0x5581508e6b80 msgr2=0x5581508e6f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:57.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.264+0000 7f817cf92740 1 -- 172.21.15.83:0/2784050710 shutdown_connections 2024-04-24T09:13:57.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.264+0000 7f817cf92740 1 -- 172.21.15.83:0/2784050710 wait complete. 2024-04-24T09:13:57.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.264+0000 7f817cf92740 1 -- start start 2024-04-24T09:13:57.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.265+0000 7f817cf92740 1 Processor -- start 2024-04-24T09:13:57.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.265+0000 7f817cf92740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5581508ea860 0x5581508f2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:57.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.266+0000 7f817cf92740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581508eb180 0x5581508f2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:57.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.266+0000 7f817cf92740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5581508f5850 0x5581508f3200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:57.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.266+0000 7f817cf92740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5581508f8640 con 0x5581508eb180 2024-04-24T09:13:57.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.266+0000 7f817cf92740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558150809690 con 0x5581508f5850 2024-04-24T09:13:57.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.266+0000 7f817cf92740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5581508f87c0 con 0x5581508ea860 2024-04-24T09:13:57.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.266+0000 7f817cf92740 10 client.0 _ll_register_callbacks cb 0x558150845650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:13:57.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.266+0000 7f8173400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5581508ea860 0x5581508f2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:57.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.266+0000 7f8173400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5581508ea860 0x5581508f2780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:41276/0 (socket says 172.21.15.83:41276) 2024-04-24T09:13:57.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.266+0000 7f8173400640 1 -- 172.21.15.83:0/1534247634 learned_addr learned my addr 172.21.15.83:0/1534247634 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:13:57.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.266+0000 7f8172a00640 1 --2- 172.21.15.83:0/1534247634 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581508eb180 0x5581508f2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:57.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.266+0000 7f8173e00640 1 --2- 172.21.15.83:0/1534247634 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5581508f5850 0x5581508f3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:57.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.267+0000 7f8173400640 1 -- 172.21.15.83:0/1534247634 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5581508f5850 msgr2=0x5581508f3200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:57.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.267+0000 7f8173400640 1 --2- 172.21.15.83:0/1534247634 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5581508f5850 0x5581508f3200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:57.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.267+0000 7f8173400640 1 -- 172.21.15.83:0/1534247634 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581508eb180 msgr2=0x5581508f2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:57.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.267+0000 7f8173400640 1 --2- 172.21.15.83:0/1534247634 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581508eb180 0x5581508f2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:57.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.267+0000 7f8173400640 1 -- 172.21.15.83:0/1534247634 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558150808920 con 0x5581508ea860 2024-04-24T09:13:57.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.267+0000 7f8172a00640 1 --2- 172.21.15.83:0/1534247634 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581508eb180 0x5581508f2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:13:57.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.267+0000 7f8173400640 1 --2- 172.21.15.83:0/1534247634 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5581508ea860 0x5581508f2780 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8164009c80 tx=0x7f8164005160 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9cb4552b13b84d5c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:57.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.267+0000 7f815cc00640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-04-24T09:13:57.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.268+0000 7f815cc00640 1 -- 172.21.15.83:0/1534247634 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8164028070 con 0x5581508ea860 2024-04-24T09:13:57.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.268+0000 7f817cf92740 1 -- 172.21.15.83:0/1534247634 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81640047a0 con 0x5581508ea860 2024-04-24T09:13:57.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.268+0000 7f815cc00640 1 -- 172.21.15.83:0/1534247634 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9583+0+0 (secure 0 0 0) 0x7f816400f5c0 con 0x5581508ea860 2024-04-24T09:13:57.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.269+0000 7f815cc00640 1 -- 172.21.15.83:0/1534247634 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f8164034050 con 0x5581508ea860 2024-04-24T09:13:57.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.269+0000 7f817cf92740 10 client.30073 fetch_fsmap learned FSMap version 64 2024-04-24T09:13:57.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.269+0000 7f817cf92740 1 -- 172.21.15.83:0/1534247634 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55815062c7d0 con 0x5581508ea860 2024-04-24T09:13:57.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.269+0000 7f815cc00640 1 -- 172.21.15.83:0/1534247634 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f81640047a0 con 0x5581508ea860 2024-04-24T09:13:57.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.271+0000 7f815cc00640 1 -- 172.21.15.83:0/1534247634 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 64) ==== 8374+0+0 (secure 0 0 0) 0x7f816400f7c0 con 0x5581508ea860 2024-04-24T09:13:57.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.271+0000 7f817cf92740 10 client.30073 fetch_fsmap finished waiting for FSMap version 64 2024-04-24T09:13:57.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.271+0000 7f817cf92740 10 client.30073 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:13:57.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.271+0000 7f817cf92740 20 client.30073 populate_metadata read hostname 'smithi083' 2024-04-24T09:13:57.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.272+0000 7f817cf92740 1 --2- 172.21.15.83:0/1534247634 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x558150905ac0 0x558150907eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:57.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.272+0000 7f817cf92740 4 client.30073 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:13:57.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.272+0000 7f817cf92740 1 -- 172.21.15.83:0/1534247634 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55815085dc10 con 0x558150905ac0 2024-04-24T09:13:57.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.272+0000 7f8172a00640 1 --2- 172.21.15.83:0/1534247634 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x558150905ac0 0x558150907eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:57.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.273+0000 7f8172a00640 1 --2- 172.21.15.83:0/1534247634 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x558150905ac0 0x558150907eb0 crc :-1 s=READY pgs=553 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=390fe1959f07b531 server_cookie=aacb24b9528604db in_seq=0 out_seq=0 2024-04-24T09:13:57.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.273+0000 7f815cc00640 10 client.30073 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:13:57.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.275+0000 7f815cc00640 1 -- 172.21.15.83:0/1534247634 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55815085dc10 con 0x558150905ac0 2024-04-24T09:13:57.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.275+0000 7f815cc00640 10 client.30073 handle_command_reply: tid=0 2024-04-24T09:13:57.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.275+0000 7f817cf92740 1 client.30073 shutdown 2024-04-24T09:13:57.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.275+0000 7f817cf92740 1 -- 172.21.15.83:0/1534247634 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5581508ea860 msgr2=0x5581508f2780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:57.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.275+0000 7f817cf92740 1 --2- 172.21.15.83:0/1534247634 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5581508ea860 0x5581508f2780 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8164009c80 tx=0x7f8164005160 comp rx=0 tx=0).stop 2024-04-24T09:13:57.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.275+0000 7f817cf92740 1 -- 172.21.15.83:0/1534247634 shutdown_connections 2024-04-24T09:13:57.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.275+0000 7f817cf92740 1 --2- 172.21.15.83:0/1534247634 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x558150905ac0 0x558150907eb0 crc :-1 s=READY pgs=553 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:57.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.275+0000 7f817cf92740 1 --2- 172.21.15.83:0/1534247634 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5581508f5850 0x5581508f3200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:57.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.275+0000 7f815cc00640 0 client.30073 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:13:57.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.275+0000 7f817cf92740 1 --2- 172.21.15.83:0/1534247634 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581508eb180 0x5581508f2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:57.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.275+0000 7f817cf92740 1 --2- 172.21.15.83:0/1534247634 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5581508ea860 0x5581508f2780 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:57.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.275+0000 7f817cf92740 1 -- 172.21.15.83:0/1534247634 >> 172.21.15.83:0/1534247634 conn(0x5581508e6b80 msgr2=0x5581508ec980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:57.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.275+0000 7f817cf92740 1 -- 172.21.15.83:0/1534247634 shutdown_connections 2024-04-24T09:13:57.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.275+0000 7f817cf92740 1 -- 172.21.15.83:0/1534247634 wait complete. 2024-04-24T09:13:57.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.276+0000 7f817cf92740 20 client.30073 trim_cache size 0 max 16384 2024-04-24T09:13:57.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:13:57 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:13:57.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.308+0000 7f817cf92740 1 Processor -- start 2024-04-24T09:13:57.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.308+0000 7f817cf92740 1 -- start start 2024-04-24T09:13:57.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.309+0000 7f817cf92740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5581508ea860 0x558150901500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:57.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.309+0000 7f817cf92740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581508eb180 0x5581508ed190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:57.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.309+0000 7f817cf92740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5581508f5850 0x5581508f3020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:57.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.309+0000 7f817cf92740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f81741be580 con 0x5581508eb180 2024-04-24T09:13:57.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.309+0000 7f817cf92740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f81741c0150 con 0x5581508f5850 2024-04-24T09:13:57.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.309+0000 7f817cf92740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f81741bec50 con 0x5581508ea860 2024-04-24T09:13:57.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.309+0000 7f8172a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581508eb180 0x5581508ed190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:57.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.309+0000 7f8172a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581508eb180 0x5581508ed190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45282/0 (socket says 172.21.15.83:45282) 2024-04-24T09:13:57.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.309+0000 7f8172a00640 1 -- 172.21.15.83:0/2752871066 learned_addr learned my addr 172.21.15.83:0/2752871066 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:13:57.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.309+0000 7f8173400640 1 --2- 172.21.15.83:0/2752871066 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5581508ea860 0x558150901500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:57.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.309+0000 7f8173e00640 1 --2- 172.21.15.83:0/2752871066 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5581508f5850 0x5581508f3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:57.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.310+0000 7f8172a00640 1 -- 172.21.15.83:0/2752871066 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5581508ea860 msgr2=0x558150901500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:57.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.310+0000 7f8172a00640 1 --2- 172.21.15.83:0/2752871066 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5581508ea860 0x558150901500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:57.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.310+0000 7f8172a00640 1 -- 172.21.15.83:0/2752871066 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5581508f5850 msgr2=0x5581508f3020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:57.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.310+0000 7f8172a00640 1 --2- 172.21.15.83:0/2752871066 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5581508f5850 0x5581508f3020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:57.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.310+0000 7f8172a00640 1 -- 172.21.15.83:0/2752871066 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558150908220 con 0x5581508eb180 2024-04-24T09:13:57.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.311+0000 7f8172a00640 1 --2- 172.21.15.83:0/2752871066 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581508eb180 0x5581508ed190 secure :-1 s=READY pgs=2647 cs=0 l=1 rev1=1 crypto rx=0x7f8160009830 tx=0x7f816003f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d97fe93ad3560ba5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:57.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.311+0000 7f815f400640 1 -- 172.21.15.83:0/2752871066 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8160005650 con 0x5581508eb180 2024-04-24T09:13:57.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.311+0000 7f815f400640 1 -- 172.21.15.83:0/2752871066 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8160004030 con 0x5581508eb180 2024-04-24T09:13:57.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.311+0000 7f815f400640 1 -- 172.21.15.83:0/2752871066 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8160052730 con 0x5581508eb180 2024-04-24T09:13:57.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.312+0000 7f817cf92740 1 -- 172.21.15.83:0/2752871066 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581508eb180 msgr2=0x5581508ed190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:57.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.312+0000 7f817cf92740 1 --2- 172.21.15.83:0/2752871066 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581508eb180 0x5581508ed190 secure :-1 s=READY pgs=2647 cs=0 l=1 rev1=1 crypto rx=0x7f8160009830 tx=0x7f816003f000 comp rx=0 tx=0).stop 2024-04-24T09:13:57.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.312+0000 7f817cf92740 1 -- 172.21.15.83:0/2752871066 shutdown_connections 2024-04-24T09:13:57.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.312+0000 7f817cf92740 1 --2- 172.21.15.83:0/2752871066 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5581508f5850 0x5581508f3020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:57.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.312+0000 7f817cf92740 1 --2- 172.21.15.83:0/2752871066 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581508eb180 0x5581508ed190 secure :-1 s=CLOSED pgs=2647 cs=0 l=1 rev1=1 crypto rx=0x7f8160009830 tx=0x7f816003f000 comp rx=0 tx=0).stop 2024-04-24T09:13:57.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.312+0000 7f817cf92740 1 --2- 172.21.15.83:0/2752871066 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5581508ea860 0x558150901500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:57.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.312+0000 7f817cf92740 1 -- 172.21.15.83:0/2752871066 >> 172.21.15.83:0/2752871066 conn(0x5581508e6b80 msgr2=0x5581508e5390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:57.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.313+0000 7f817cf92740 1 -- 172.21.15.83:0/2752871066 shutdown_connections 2024-04-24T09:13:57.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.313+0000 7f817cf92740 1 -- 172.21.15.83:0/2752871066 wait complete. 2024-04-24T09:13:57.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.313+0000 7f817cf92740 1 -- start start 2024-04-24T09:13:57.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.313+0000 7f817cf92740 1 Processor -- start 2024-04-24T09:13:57.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.314+0000 7f817cf92740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5581508ea860 0x5581508f89d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:57.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.314+0000 7f817cf92740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581508f5850 0x5581508f8f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:57.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.314+0000 7f817cf92740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5581508f9450 0x558150969ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:57.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.314+0000 7f817cf92740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f81741bfa50 con 0x5581508f5850 2024-04-24T09:13:57.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.314+0000 7f817cf92740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f81741bf380 con 0x5581508ea860 2024-04-24T09:13:57.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.314+0000 7f817cf92740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f81741bc580 con 0x5581508f9450 2024-04-24T09:13:57.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.314+0000 7f817cf92740 10 client.0 _ll_register_callbacks cb 0x7f8164009f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:13:57.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.314+0000 7f8172a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581508f5850 0x5581508f8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:57.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.314+0000 7f8172a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581508f5850 0x5581508f8f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45286/0 (socket says 172.21.15.83:45286) 2024-04-24T09:13:57.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.314+0000 7f8172a00640 1 -- 172.21.15.83:0/2690115514 learned_addr learned my addr 172.21.15.83:0/2690115514 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:13:57.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.315+0000 7f8173400640 1 --2- 172.21.15.83:0/2690115514 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5581508ea860 0x5581508f89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:57.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.315+0000 7f8172a00640 1 -- 172.21.15.83:0/2690115514 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5581508f9450 msgr2=0x558150969ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:57.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.315+0000 7f8172a00640 1 --2- 172.21.15.83:0/2690115514 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5581508f9450 0x558150969ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:57.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.315+0000 7f8172a00640 1 -- 172.21.15.83:0/2690115514 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5581508ea860 msgr2=0x5581508f89d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:13:57.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.315+0000 7f8172a00640 1 --2- 172.21.15.83:0/2690115514 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5581508ea860 0x5581508f89d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:57.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.315+0000 7f8172a00640 1 -- 172.21.15.83:0/2690115514 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581505623f0 con 0x5581508f5850 2024-04-24T09:13:57.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.315+0000 7f8173400640 1 --2- 172.21.15.83:0/2690115514 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5581508ea860 0x5581508f89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:13:57.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.315+0000 7f8172a00640 1 --2- 172.21.15.83:0/2690115514 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581508f5850 0x5581508f8f10 secure :-1 s=READY pgs=2648 cs=0 l=1 rev1=1 crypto rx=0x7f8160005940 tx=0x7f8160030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4603ce07af88b315 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:13:57.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.315+0000 7f815cc00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:13:57.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.316+0000 7f815cc00640 1 -- 172.21.15.83:0/2690115514 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8160060060 con 0x5581508f5850 2024-04-24T09:13:57.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.316+0000 7f815cc00640 1 -- 172.21.15.83:0/2690115514 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9583+0+0 (secure 0 0 0) 0x7f81600041e0 con 0x5581508f5850 2024-04-24T09:13:57.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.316+0000 7f817cf92740 1 -- 172.21.15.83:0/2690115514 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8160004030 con 0x5581508f5850 2024-04-24T09:13:57.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.317+0000 7f815cc00640 1 -- 172.21.15.83:0/2690115514 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f8160067050 con 0x5581508f5850 2024-04-24T09:13:57.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.317+0000 7f817cf92740 10 client.23247 fetch_fsmap learned FSMap version 64 2024-04-24T09:13:57.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.317+0000 7f817cf92740 1 -- 172.21.15.83:0/2690115514 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x5581505220f0 con 0x5581508f5850 2024-04-24T09:13:57.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.317+0000 7f815cc00640 1 -- 172.21.15.83:0/2690115514 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f81600043e0 con 0x5581508f5850 2024-04-24T09:13:57.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.318+0000 7f815cc00640 1 -- 172.21.15.83:0/2690115514 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 64) ==== 8374+0+0 (secure 0 0 0) 0x7f8160003890 con 0x5581508f5850 2024-04-24T09:13:57.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.318+0000 7f817cf92740 10 client.23247 fetch_fsmap finished waiting for FSMap version 64 2024-04-24T09:13:57.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.318+0000 7f817cf92740 10 client.23247 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:13:57.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.318+0000 7f817cf92740 20 client.23247 populate_metadata read hostname 'smithi083' 2024-04-24T09:13:57.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.318+0000 7f817cf92740 1 --2- 172.21.15.83:0/2690115514 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55815097ca40 0x55815097ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:13:57.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.318+0000 7f817cf92740 4 client.23247 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T09:13:57.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.318+0000 7f817cf92740 1 -- 172.21.15.83:0/2690115514 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f8160004030 con 0x55815097ca40 2024-04-24T09:13:57.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.318+0000 7f8173400640 1 --2- 172.21.15.83:0/2690115514 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55815097ca40 0x55815097ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:13:57.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.319+0000 7f8173400640 1 --2- 172.21.15.83:0/2690115514 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55815097ca40 0x55815097ee60 crc :-1 s=READY pgs=554 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b918e6233ef97bd1 server_cookie=e70990cdade0c306 in_seq=0 out_seq=0 2024-04-24T09:13:57.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.319+0000 7f815cc00640 10 client.23247 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:13:57.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.319+0000 7f815cc00640 1 -- 172.21.15.83:0/2690115514 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x7f8160004030 con 0x55815097ca40 2024-04-24T09:13:57.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.319+0000 7f815cc00640 10 client.23247 handle_command_reply: tid=0 2024-04-24T09:13:57.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.319+0000 7f817cf92740 1 client.23247 shutdown 2024-04-24T09:13:57.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.320+0000 7f817cf92740 1 -- 172.21.15.83:0/2690115514 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581508f5850 msgr2=0x5581508f8f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:57.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.320+0000 7f817cf92740 1 --2- 172.21.15.83:0/2690115514 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581508f5850 0x5581508f8f10 secure :-1 s=READY pgs=2648 cs=0 l=1 rev1=1 crypto rx=0x7f8160005940 tx=0x7f8160030d80 comp rx=0 tx=0).stop 2024-04-24T09:13:57.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.320+0000 7f817cf92740 1 -- 172.21.15.83:0/2690115514 shutdown_connections 2024-04-24T09:13:57.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.320+0000 7f817cf92740 1 --2- 172.21.15.83:0/2690115514 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55815097ca40 0x55815097ee60 crc :-1 s=READY pgs=554 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:57.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.320+0000 7f817cf92740 1 --2- 172.21.15.83:0/2690115514 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5581508f9450 0x558150969ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:57.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.320+0000 7f817cf92740 1 --2- 172.21.15.83:0/2690115514 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581508f5850 0x5581508f8f10 unknown :-1 s=CLOSED pgs=2648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:57.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.320+0000 7f815cc00640 0 client.23247 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:13:57.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.320+0000 7f817cf92740 1 --2- 172.21.15.83:0/2690115514 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5581508ea860 0x5581508f89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:57.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.320+0000 7f817cf92740 1 -- 172.21.15.83:0/2690115514 >> 172.21.15.83:0/2690115514 conn(0x5581508e6b80 msgr2=0x5581508e5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:57.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.320+0000 7f817cf92740 1 -- 172.21.15.83:0/2690115514 shutdown_connections 2024-04-24T09:13:57.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.320+0000 7f817cf92740 1 -- 172.21.15.83:0/2690115514 wait complete. 2024-04-24T09:13:57.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.320+0000 7f817cf92740 20 client.23247 trim_cache size 0 max 16384 2024-04-24T09:13:57.322 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T09:13:57.322 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T09:13:57.322 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-04-24T09:13:57.322 INFO:teuthology.orchestra.run.smithi083.stdout: "64546e1c-5611-455f-b73a-ad4a542174cc": { 2024-04-24T09:13:57.322 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-04-24T09:13:57.322 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "64546e1c-5611-455f-b73a-ad4a542174cc", 2024-04-24T09:13:57.322 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-04-24T09:13:57.323 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:13:57.323 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:13:57.323 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T09:13:57.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.321+0000 7f817a000640 1 -- 172.21.15.83:0/363311489 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f81580a3d20 msgr2=0x7f81580a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:57.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.321+0000 7f817a000640 1 --2- 172.21.15.83:0/363311489 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f81580a3d20 0x7f81580a61e0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f8160000c00 tx=0x7f8160005c90 comp rx=0 tx=0).stop 2024-04-24T09:13:57.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.321+0000 7f817a000640 1 -- 172.21.15.83:0/363311489 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f817410e8e0 msgr2=0x7f8174102be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:13:57.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.321+0000 7f817a000640 1 --2- 172.21.15.83:0/363311489 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f817410e8e0 0x7f8174102be0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f816c00e9d0 tx=0x7f816c00ee90 comp rx=0 tx=0).stop 2024-04-24T09:13:57.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.322+0000 7f817a000640 1 -- 172.21.15.83:0/363311489 shutdown_connections 2024-04-24T09:13:57.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.322+0000 7f817a000640 1 --2- 172.21.15.83:0/363311489 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f81580a3d20 0x7f81580a61e0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:57.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.322+0000 7f817a000640 1 --2- 172.21.15.83:0/363311489 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f817410e8e0 0x7f8174102be0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:57.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.322+0000 7f817a000640 1 --2- 172.21.15.83:0/363311489 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8174074bf0 0x7f8174108010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:57.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.322+0000 7f817a000640 1 --2- 172.21.15.83:0/363311489 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81740742d0 0x7f8174107ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:13:57.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.322+0000 7f817a000640 1 -- 172.21.15.83:0/363311489 >> 172.21.15.83:0/363311489 conn(0x7f8174079450 msgr2=0x7f81740fefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:13:57.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.322+0000 7f817a000640 1 -- 172.21.15.83:0/363311489 shutdown_connections 2024-04-24T09:13:57.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:13:57.322+0000 7f817a000640 1 -- 172.21.15.83:0/363311489 wait complete. 2024-04-24T09:13:57.347 INFO:tasks.cephfs.filesystem:scrub status for tag:64546e1c-5611-455f-b73a-ad4a542174cc - {'path': '/', 'tag': '64546e1c-5611-455f-b73a-ad4a542174cc', 'options': 'recursive,force'} 2024-04-24T09:13:57.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:13:57 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:13:57.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:13:57 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:13:58.201 INFO:tasks.workunit.client.0.smithi083.stdout: 10 63059 6.57 MB/sec execute 389 sec latency 93.067 ms 2024-04-24T09:13:58.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:13:58 smithi138 ceph-mon[31441]: pgmap v705: 129 pgs: 129 active+clean; 971 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.8 MiB/s wr, 224 op/s 2024-04-24T09:13:58.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:13:58 smithi138 ceph-mon[31441]: scrub queued for path: / 2024-04-24T09:13:58.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:13:58 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:13:58.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:13:58 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:13:58.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:13:58 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:13:58.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:13:58 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:13:58.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:13:58 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:13:58.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:13:58 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:13:58.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:13:58 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:13:58.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:13:58 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:13:58.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:13:58 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:13:58.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:13:58 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:13:58.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:13:58 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:13:58.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:13:58 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:13:58.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:13:58 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:13:58.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:13:58 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:13:58.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:13:58 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:13:58.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:13:58 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:13:58.457 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:13:58 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:13:58.457 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:13:58 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:13:58.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:13:58 smithi189 ceph-mon[31791]: pgmap v705: 129 pgs: 129 active+clean; 971 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.8 MiB/s wr, 224 op/s 2024-04-24T09:13:58.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:13:58 smithi189 ceph-mon[31791]: scrub queued for path: / 2024-04-24T09:13:58.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:13:58 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:13:58.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:13:58 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:13:58.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:13:58 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:13:58.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:13:58 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:13:58.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:13:58 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:13:58.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:13:58 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:13:58.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:13:58 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:13:58.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:13:58 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:13:58.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:13:58 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:13:58.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:13:58 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:13:58.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:13:58 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:13:58.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:13:58 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:13:58.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:13:58 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:13:58.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:13:58 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:13:58.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:13:58 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:13:58.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:13:58 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:13:58.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:13:58 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:13:58.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:13:58 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:13:58.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:13:58 smithi083 ceph-mon[27907]: pgmap v705: 129 pgs: 129 active+clean; 971 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.8 MiB/s wr, 224 op/s 2024-04-24T09:13:58.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:13:58 smithi083 ceph-mon[27907]: scrub queued for path: / 2024-04-24T09:13:58.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:13:58 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:13:58.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:13:58 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:13:58.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:13:58 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:13:58.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:13:58 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:13:58.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:13:58 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:13:58.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:13:58 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:13:58.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:13:58 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:13:58.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:13:58 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:13:58.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:13:58 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:13:58.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:13:58 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:13:58.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:13:58 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:13:58.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:13:58 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:13:58.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:13:58 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:13:58.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:13:58 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:13:58.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:13:58 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:13:58.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:13:58 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:13:58.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:13:58 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:13:58.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:13:58 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:13:59.203 INFO:tasks.workunit.client.0.smithi083.stdout: 10 63186 6.57 MB/sec execute 390 sec latency 83.867 ms 2024-04-24T09:14:00.203 INFO:tasks.workunit.client.0.smithi083.stdout: 10 63345 6.59 MB/sec execute 391 sec latency 73.903 ms 2024-04-24T09:14:00.552 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:14:00 smithi083 ceph-mon[27907]: pgmap v706: 129 pgs: 129 active+clean; 975 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.3 MiB/s wr, 353 op/s 2024-04-24T09:14:00.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:14:00 smithi138 ceph-mon[31441]: pgmap v706: 129 pgs: 129 active+clean; 975 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.3 MiB/s wr, 353 op/s 2024-04-24T09:14:00.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:14:00 smithi189 ceph-mon[31791]: pgmap v706: 129 pgs: 129 active+clean; 975 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.3 MiB/s wr, 353 op/s 2024-04-24T09:14:01.203 INFO:tasks.workunit.client.0.smithi083.stdout: 10 63480 6.59 MB/sec execute 392 sec latency 102.900 ms 2024-04-24T09:14:02.203 INFO:tasks.workunit.client.0.smithi083.stdout: 10 63629 6.59 MB/sec execute 393 sec latency 71.460 ms 2024-04-24T09:14:02.509 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:14:02 smithi189 ceph-mon[31791]: pgmap v707: 129 pgs: 129 active+clean; 975 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.1 MiB/s wr, 255 op/s 2024-04-24T09:14:02.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:14:02 smithi083 ceph-mon[27907]: pgmap v707: 129 pgs: 129 active+clean; 975 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.1 MiB/s wr, 255 op/s 2024-04-24T09:14:02.704 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:14:02 smithi138 ceph-mon[31441]: pgmap v707: 129 pgs: 129 active+clean; 975 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.1 MiB/s wr, 255 op/s 2024-04-24T09:14:02.854 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.854+0000 7f18d8200640 1 -- 172.21.15.83:0/1497941158 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18d0102280 msgr2=0x7f18d0102660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:02.854 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.854+0000 7f18d8200640 1 --2- 172.21.15.83:0/1497941158 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18d0102280 0x7f18d0102660 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f18c4009900 tx=0x7f18c402e820 comp rx=0 tx=0).stop 2024-04-24T09:14:02.854 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.854+0000 7f18d8200640 1 -- 172.21.15.83:0/1497941158 shutdown_connections 2024-04-24T09:14:02.855 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.854+0000 7f18d8200640 1 --2- 172.21.15.83:0/1497941158 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f18d0107350 0x7f18d010ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:02.855 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.854+0000 7f18d8200640 1 --2- 172.21.15.83:0/1497941158 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18d0102c30 0x7f18d0106c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:02.855 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.854+0000 7f18d8200640 1 --2- 172.21.15.83:0/1497941158 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18d0102280 0x7f18d0102660 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:02.855 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.854+0000 7f18d8200640 1 -- 172.21.15.83:0/1497941158 >> 172.21.15.83:0/1497941158 conn(0x7f18d0079490 msgr2=0x7f18d00798a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:14:02.855 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.854+0000 7f18d8200640 1 -- 172.21.15.83:0/1497941158 shutdown_connections 2024-04-24T09:14:02.855 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.854+0000 7f18d8200640 1 -- 172.21.15.83:0/1497941158 wait complete. 2024-04-24T09:14:02.855 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.855+0000 7f18d8200640 1 Processor -- start 2024-04-24T09:14:02.856 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.855+0000 7f18d8200640 1 -- start start 2024-04-24T09:14:02.856 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.856+0000 7f18d8200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18d0102280 0x7f18d019ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:02.856 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.856+0000 7f18d8200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f18d0107350 0x7f18d019f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:02.856 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.856+0000 7f18d8200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18d01a35e0 0x7f18d01a3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:02.856 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.856+0000 7f18d8200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f18d0118900 con 0x7f18d0102280 2024-04-24T09:14:02.857 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.856+0000 7f18d5800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18d0102280 0x7f18d019ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:02.857 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.856+0000 7f18d5800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18d0102280 0x7f18d019ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49794/0 (socket says 172.21.15.83:49794) 2024-04-24T09:14:02.857 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.856+0000 7f18d5800640 1 -- 172.21.15.83:0/1688264120 learned_addr learned my addr 172.21.15.83:0/1688264120 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:14:02.857 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.856+0000 7f18d8200640 1 -- 172.21.15.83:0/1688264120 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f18d0118780 con 0x7f18d01a35e0 2024-04-24T09:14:02.857 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.856+0000 7f18d8200640 1 -- 172.21.15.83:0/1688264120 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f18d0118a80 con 0x7f18d0107350 2024-04-24T09:14:02.857 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.856+0000 7f18d6200640 1 --2- 172.21.15.83:0/1688264120 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18d01a35e0 0x7f18d01a3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:02.858 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.856+0000 7f18d4e00640 1 --2- 172.21.15.83:0/1688264120 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f18d0107350 0x7f18d019f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:02.858 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.857+0000 7f18d4e00640 1 -- 172.21.15.83:0/1688264120 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18d01a35e0 msgr2=0x7f18d01a3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:02.858 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.857+0000 7f18d4e00640 1 --2- 172.21.15.83:0/1688264120 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18d01a35e0 0x7f18d01a3a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:02.858 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.857+0000 7f18d4e00640 1 -- 172.21.15.83:0/1688264120 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18d0102280 msgr2=0x7f18d019ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:02.858 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.857+0000 7f18d4e00640 1 --2- 172.21.15.83:0/1688264120 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18d0102280 0x7f18d019ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:02.858 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.857+0000 7f18d4e00640 1 -- 172.21.15.83:0/1688264120 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18d01a40d0 con 0x7f18d0107350 2024-04-24T09:14:02.858 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.857+0000 7f18d4e00640 1 --2- 172.21.15.83:0/1688264120 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f18d0107350 0x7f18d019f260 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f18bc00d6a0 tx=0x7f18bc00db60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:14:02.858 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.858+0000 7f18ce000640 1 -- 172.21.15.83:0/1688264120 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18bc017070 con 0x7f18d0107350 2024-04-24T09:14:02.859 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.858+0000 7f18d8200640 1 -- 172.21.15.83:0/1688264120 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18d01a43c0 con 0x7f18d0107350 2024-04-24T09:14:02.859 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.858+0000 7f18d8200640 1 -- 172.21.15.83:0/1688264120 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18d006bd90 con 0x7f18d0107350 2024-04-24T09:14:02.859 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.858+0000 7f18ce000640 1 -- 172.21.15.83:0/1688264120 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f18bc009820 con 0x7f18d0107350 2024-04-24T09:14:02.859 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.858+0000 7f18ce000640 1 -- 172.21.15.83:0/1688264120 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18bc004c40 con 0x7f18d0107350 2024-04-24T09:14:02.862 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.862+0000 7f18daf78740 1 Processor -- start 2024-04-24T09:14:02.862 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.862+0000 7f18daf78740 1 -- start start 2024-04-24T09:14:02.864 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.863+0000 7f18daf78740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0a3e1f1b0 0x55e0a3e1f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:02.864 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.863+0000 7f18ce000640 1 -- 172.21.15.83:0/1688264120 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f18bc005220 con 0x7f18d0107350 2024-04-24T09:14:02.864 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.863+0000 7f18daf78740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e0a3e1fad0 0x55e0a3e29e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:02.864 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.863+0000 7f18daf78740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e0a3e2a390 0x55e0a3e2c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:02.864 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.863+0000 7f18daf78740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e0a3d85900 con 0x55e0a3e1fad0 2024-04-24T09:14:02.864 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.863+0000 7f18daf78740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e0a3d2bf20 con 0x55e0a3e1f1b0 2024-04-24T09:14:02.864 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.863+0000 7f18daf78740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55e0a3d818c0 con 0x55e0a3e2a390 2024-04-24T09:14:02.864 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.863+0000 7f18d4e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e0a3e2a390 0x55e0a3e2c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:02.864 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.863+0000 7f18d4e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e0a3e2a390 0x55e0a3e2c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:59536/0 (socket says 172.21.15.83:59536) 2024-04-24T09:14:02.864 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.863+0000 7f18d4e00640 1 -- 172.21.15.83:0/3898684650 learned_addr learned my addr 172.21.15.83:0/3898684650 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:14:02.865 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.864+0000 7f18d5800640 1 --2- 172.21.15.83:0/3898684650 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e0a3e1fad0 0x55e0a3e29e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:02.865 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.864+0000 7f18ce000640 1 --2- 172.21.15.83:0/1688264120 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f18b80a3ec0 0x7f18b80a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:02.865 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.865+0000 7f18d5800640 1 -- 172.21.15.83:0/3898684650 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e0a3e2a390 msgr2=0x55e0a3e2c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:14:02.865 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.865+0000 7f18d5800640 1 --2- 172.21.15.83:0/3898684650 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e0a3e2a390 0x55e0a3e2c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:02.866 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.865+0000 7f18d5800640 1 -- 172.21.15.83:0/3898684650 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0a3e1f1b0 msgr2=0x55e0a3e1f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:14:02.866 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.865+0000 7f18ce000640 1 -- 172.21.15.83:0/1688264120 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 9671+0+0 (secure 0 0 0) 0x7f18bc0d0000 con 0x7f18d0107350 2024-04-24T09:14:02.866 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.865+0000 7f18d6200640 1 --2- 172.21.15.83:0/3898684650 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0a3e1f1b0 0x55e0a3e1f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:02.866 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.865+0000 7f18d5800640 1 --2- 172.21.15.83:0/3898684650 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0a3e1f1b0 0x55e0a3e1f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:02.866 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.865+0000 7f18d5800640 1 -- 172.21.15.83:0/3898684650 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0a3d50e10 con 0x55e0a3e1fad0 2024-04-24T09:14:02.866 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.865+0000 7f18d6200640 1 --2- 172.21.15.83:0/3898684650 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0a3e1f1b0 0x55e0a3e1f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:14:02.867 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.866+0000 7f18d5800640 1 --2- 172.21.15.83:0/3898684650 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e0a3e1fad0 0x55e0a3e29e50 secure :-1 s=READY pgs=2649 cs=0 l=1 rev1=1 crypto rx=0x7f18c4005ab0 tx=0x7f18c40048a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8277c3f8ffeb2df server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:14:02.867 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.866+0000 7f18d6200640 1 --2- 172.21.15.83:0/1688264120 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f18b80a3ec0 0x7f18b80a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:02.867 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.866+0000 7f18c2a00640 1 -- 172.21.15.83:0/3898684650 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18c4045020 con 0x55e0a3e1fad0 2024-04-24T09:14:02.867 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.866+0000 7f18c2a00640 1 -- 172.21.15.83:0/3898684650 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f18c4040900 con 0x55e0a3e1fad0 2024-04-24T09:14:02.867 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.866+0000 7f18c2a00640 1 -- 172.21.15.83:0/3898684650 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18c4036650 con 0x55e0a3e1fad0 2024-04-24T09:14:02.868 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.867+0000 7f18daf78740 1 -- 172.21.15.83:0/3898684650 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e0a3e1fad0 msgr2=0x55e0a3e29e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:02.868 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.867+0000 7f18daf78740 1 --2- 172.21.15.83:0/3898684650 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e0a3e1fad0 0x55e0a3e29e50 secure :-1 s=READY pgs=2649 cs=0 l=1 rev1=1 crypto rx=0x7f18c4005ab0 tx=0x7f18c40048a0 comp rx=0 tx=0).stop 2024-04-24T09:14:02.868 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.867+0000 7f18d6200640 1 --2- 172.21.15.83:0/1688264120 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f18b80a3ec0 0x7f18b80a6380 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f18c8005fd0 tx=0x7f18c8004ab0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:14:02.868 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.867+0000 7f18daf78740 1 -- 172.21.15.83:0/3898684650 shutdown_connections 2024-04-24T09:14:02.868 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.867+0000 7f18daf78740 1 --2- 172.21.15.83:0/3898684650 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e0a3e2a390 0x55e0a3e2c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:02.868 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.867+0000 7f18daf78740 1 --2- 172.21.15.83:0/3898684650 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e0a3e1fad0 0x55e0a3e29e50 unknown :-1 s=CLOSED pgs=2649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:02.868 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.867+0000 7f18daf78740 1 --2- 172.21.15.83:0/3898684650 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0a3e1f1b0 0x55e0a3e1f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:02.868 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.867+0000 7f18daf78740 1 -- 172.21.15.83:0/3898684650 >> 172.21.15.83:0/3898684650 conn(0x55e0a3e1b6c0 msgr2=0x55e0a3e1baa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:14:02.868 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.867+0000 7f18daf78740 1 -- 172.21.15.83:0/3898684650 shutdown_connections 2024-04-24T09:14:02.869 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.867+0000 7f18daf78740 1 -- 172.21.15.83:0/3898684650 wait complete. 2024-04-24T09:14:02.869 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.868+0000 7f18daf78740 1 -- start start 2024-04-24T09:14:02.870 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.868+0000 7f18daf78740 1 Processor -- start 2024-04-24T09:14:02.870 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.869+0000 7f18daf78740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e0a3e1f1b0 0x55e0a3e27290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:02.870 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.869+0000 7f18daf78740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e0a3e1fad0 0x55e0a3e277d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:02.871 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.869+0000 7f18daf78740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0a3e2a390 0x55e0a3e27d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:02.871 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.869+0000 7f18daf78740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e0a3e2cf60 con 0x55e0a3e1fad0 2024-04-24T09:14:02.871 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.869+0000 7f18daf78740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e0a3d3f2c0 con 0x55e0a3e2a390 2024-04-24T09:14:02.871 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.869+0000 7f18daf78740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55e0a3e2d0e0 con 0x55e0a3e1f1b0 2024-04-24T09:14:02.871 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.869+0000 7f18daf78740 10 client.0 _ll_register_callbacks cb 0x55e0a3a97a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:14:02.871 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.870+0000 7f18d4e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e0a3e1fad0 0x55e0a3e277d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:02.871 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.870+0000 7f18d5800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e0a3e1f1b0 0x55e0a3e27290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:02.871 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.870+0000 7f18d4e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e0a3e1fad0 0x55e0a3e277d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49808/0 (socket says 172.21.15.83:49808) 2024-04-24T09:14:02.871 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.870+0000 7f18d4e00640 1 -- 172.21.15.83:0/2266105873 learned_addr learned my addr 172.21.15.83:0/2266105873 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:14:02.871 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.870+0000 7f18d6200640 1 --2- 172.21.15.83:0/2266105873 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0a3e2a390 0x55e0a3e27d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:02.871 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.870+0000 7f18d4e00640 1 -- 172.21.15.83:0/2266105873 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e0a3e1f1b0 msgr2=0x55e0a3e27290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:14:02.871 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.870+0000 7f18d4e00640 1 --2- 172.21.15.83:0/2266105873 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e0a3e1f1b0 0x55e0a3e27290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:02.871 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.870+0000 7f18d4e00640 1 -- 172.21.15.83:0/2266105873 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0a3e2a390 msgr2=0x55e0a3e27d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:14:02.871 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.870+0000 7f18d4e00640 1 --2- 172.21.15.83:0/2266105873 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0a3e2a390 0x55e0a3e27d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:02.872 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.870+0000 7f18d4e00640 1 -- 172.21.15.83:0/2266105873 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0a3d4ffd0 con 0x55e0a3e1fad0 2024-04-24T09:14:02.873 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.871+0000 7f18d4e00640 1 --2- 172.21.15.83:0/2266105873 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e0a3e1fad0 0x55e0a3e277d0 secure :-1 s=READY pgs=2650 cs=0 l=1 rev1=1 crypto rx=0x7f18bc098540 tx=0x7f18bc0cd2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1c02de2990ade14 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:14:02.873 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.871+0000 7f18b7400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:14:02.873 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.871+0000 7f18b7400640 1 -- 172.21.15.83:0/2266105873 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18bc020040 con 0x55e0a3e1fad0 2024-04-24T09:14:02.873 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.871+0000 7f18b7400640 1 -- 172.21.15.83:0/2266105873 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(127..127 src has 1..127) ==== 9671+0+0 (secure 0 0 0) 0x7f18bc0cd4f0 con 0x55e0a3e1fad0 2024-04-24T09:14:02.873 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.871+0000 7f18daf78740 1 -- 172.21.15.83:0/2266105873 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18c4040900 con 0x55e0a3e1fad0 2024-04-24T09:14:02.874 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.872+0000 7f18b7400640 1 -- 172.21.15.83:0/2266105873 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f18bc09b4d0 con 0x55e0a3e1fad0 2024-04-24T09:14:02.874 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.872+0000 7f18daf78740 10 client.23259 fetch_fsmap learned FSMap version 64 2024-04-24T09:14:02.874 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.872+0000 7f18daf78740 1 -- 172.21.15.83:0/2266105873 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55e0a3a98210 con 0x55e0a3e1fad0 2024-04-24T09:14:02.874 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.872+0000 7f18b7400640 1 -- 172.21.15.83:0/2266105873 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f18c4040900 con 0x55e0a3e1fad0 2024-04-24T09:14:02.874 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.872+0000 7f18b7400640 1 -- 172.21.15.83:0/2266105873 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 64) ==== 8374+0+0 (secure 0 0 0) 0x7f18bc0b8070 con 0x55e0a3e1fad0 2024-04-24T09:14:02.874 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.873+0000 7f18daf78740 10 client.23259 fetch_fsmap finished waiting for FSMap version 64 2024-04-24T09:14:02.874 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.873+0000 7f18daf78740 10 client.23259 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:14:02.874 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.873+0000 7f18daf78740 20 client.23259 populate_metadata read hostname 'smithi083' 2024-04-24T09:14:02.874 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.873+0000 7f18daf78740 1 --2- 172.21.15.83:0/2266105873 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55e0a3e3a9a0 0x55e0a3e3cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:02.874 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.873+0000 7f18daf78740 4 client.23259 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:14:02.874 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.873+0000 7f18daf78740 1 -- 172.21.15.83:0/2266105873 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e0a3d93320 con 0x55e0a3e3a9a0 2024-04-24T09:14:02.875 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.873+0000 7f18d5800640 1 --2- 172.21.15.83:0/2266105873 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55e0a3e3a9a0 0x55e0a3e3cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:02.876 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.874+0000 7f18d5800640 1 --2- 172.21.15.83:0/2266105873 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55e0a3e3a9a0 0x55e0a3e3cde0 crc :-1 s=READY pgs=555 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7270879da9ee89a8 server_cookie=19d13b2387be9ee5 in_seq=0 out_seq=0 2024-04-24T09:14:02.876 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.874+0000 7f18b7400640 10 client.23259 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:14:02.876 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.875+0000 7f18b7400640 1 -- 172.21.15.83:0/2266105873 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e0a3d93320 con 0x55e0a3e3a9a0 2024-04-24T09:14:02.876 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.875+0000 7f18b7400640 10 client.23259 handle_command_reply: tid=0 2024-04-24T09:14:02.876 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.875+0000 7f18daf78740 1 client.23259 shutdown 2024-04-24T09:14:02.876 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.875+0000 7f18daf78740 1 -- 172.21.15.83:0/2266105873 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e0a3e1fad0 msgr2=0x55e0a3e277d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:02.877 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.875+0000 7f18daf78740 1 --2- 172.21.15.83:0/2266105873 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e0a3e1fad0 0x55e0a3e277d0 secure :-1 s=READY pgs=2650 cs=0 l=1 rev1=1 crypto rx=0x7f18bc098540 tx=0x7f18bc0cd2e0 comp rx=0 tx=0).stop 2024-04-24T09:14:02.877 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.875+0000 7f18daf78740 1 -- 172.21.15.83:0/2266105873 shutdown_connections 2024-04-24T09:14:02.877 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.876+0000 7f18daf78740 1 --2- 172.21.15.83:0/2266105873 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55e0a3e3a9a0 0x55e0a3e3cde0 crc :-1 s=READY pgs=555 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:02.877 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.876+0000 7f18daf78740 1 --2- 172.21.15.83:0/2266105873 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0a3e2a390 0x55e0a3e27d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:02.877 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.876+0000 7f18daf78740 1 --2- 172.21.15.83:0/2266105873 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e0a3e1fad0 0x55e0a3e277d0 unknown :-1 s=CLOSED pgs=2650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:02.877 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.876+0000 7f18daf78740 1 --2- 172.21.15.83:0/2266105873 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e0a3e1f1b0 0x55e0a3e27290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:02.877 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.876+0000 7f18daf78740 1 -- 172.21.15.83:0/2266105873 >> 172.21.15.83:0/2266105873 conn(0x55e0a3e1b6c0 msgr2=0x55e0a3e21440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:14:02.877 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.876+0000 7f18b7400640 0 client.23259 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:14:02.877 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.876+0000 7f18daf78740 1 -- 172.21.15.83:0/2266105873 shutdown_connections 2024-04-24T09:14:02.877 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.876+0000 7f18daf78740 1 -- 172.21.15.83:0/2266105873 wait complete. 2024-04-24T09:14:02.877 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.876+0000 7f18daf78740 20 client.23259 trim_cache size 0 max 16384 2024-04-24T09:14:02.903 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.903+0000 7f18daf78740 1 Processor -- start 2024-04-24T09:14:02.904 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.903+0000 7f18daf78740 1 -- start start 2024-04-24T09:14:02.904 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.903+0000 7f18daf78740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0a3e1f1b0 0x55e0a3e35e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:02.904 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.903+0000 7f18daf78740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e0a3e1fad0 0x55e0a3e426f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:02.904 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.903+0000 7f18daf78740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e0a3e2a390 0x55e0a3e42c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:02.904 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.903+0000 7f18daf78740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f18b0010300 con 0x55e0a3e1fad0 2024-04-24T09:14:02.904 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.903+0000 7f18daf78740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f18b0011ed0 con 0x55e0a3e1f1b0 2024-04-24T09:14:02.904 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.903+0000 7f18daf78740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f18b00109d0 con 0x55e0a3e2a390 2024-04-24T09:14:02.905 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.904+0000 7f18d5800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0a3e1f1b0 0x55e0a3e35e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:02.905 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.904+0000 7f18d5800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0a3e1f1b0 0x55e0a3e35e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:52668/0 (socket says 172.21.15.83:52668) 2024-04-24T09:14:02.905 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.904+0000 7f18d5800640 1 -- 172.21.15.83:0/1396279775 learned_addr learned my addr 172.21.15.83:0/1396279775 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:14:02.905 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.904+0000 7f18d6200640 1 --2- 172.21.15.83:0/1396279775 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e0a3e2a390 0x55e0a3e42c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:02.906 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.904+0000 7f18d4e00640 1 --2- 172.21.15.83:0/1396279775 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e0a3e1fad0 0x55e0a3e426f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:02.906 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.904+0000 7f18d5800640 1 -- 172.21.15.83:0/1396279775 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e0a3e2a390 msgr2=0x55e0a3e42c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:14:02.906 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.904+0000 7f18d5800640 1 --2- 172.21.15.83:0/1396279775 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e0a3e2a390 0x55e0a3e42c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:02.907 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.904+0000 7f18d5800640 1 -- 172.21.15.83:0/1396279775 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e0a3e1fad0 msgr2=0x55e0a3e426f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:14:02.907 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.904+0000 7f18d5800640 1 --2- 172.21.15.83:0/1396279775 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e0a3e1fad0 0x55e0a3e426f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:02.907 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.904+0000 7f18d5800640 1 -- 172.21.15.83:0/1396279775 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0a3e20580 con 0x55e0a3e1f1b0 2024-04-24T09:14:02.907 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.904+0000 7f18d6200640 1 --2- 172.21.15.83:0/1396279775 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e0a3e2a390 0x55e0a3e42c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:14:02.907 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.905+0000 7f18d5800640 1 --2- 172.21.15.83:0/1396279775 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0a3e1f1b0 0x55e0a3e35e80 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f18c4009230 tx=0x7f18c4005a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=236f38bc2830cbf server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:14:02.909 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.905+0000 7f18c2000640 1 -- 172.21.15.83:0/1396279775 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18c4045040 con 0x55e0a3e1f1b0 2024-04-24T09:14:02.909 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.905+0000 7f18c2000640 1 -- 172.21.15.83:0/1396279775 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f18c4058030 con 0x55e0a3e1f1b0 2024-04-24T09:14:02.909 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.905+0000 7f18c2000640 1 -- 172.21.15.83:0/1396279775 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18c4036d20 con 0x55e0a3e1f1b0 2024-04-24T09:14:02.910 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.906+0000 7f18daf78740 1 -- 172.21.15.83:0/1396279775 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0a3e1f1b0 msgr2=0x55e0a3e35e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:02.910 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.906+0000 7f18daf78740 1 --2- 172.21.15.83:0/1396279775 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0a3e1f1b0 0x55e0a3e35e80 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f18c4009230 tx=0x7f18c4005a20 comp rx=0 tx=0).stop 2024-04-24T09:14:02.910 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.906+0000 7f18daf78740 1 -- 172.21.15.83:0/1396279775 shutdown_connections 2024-04-24T09:14:02.910 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.906+0000 7f18daf78740 1 --2- 172.21.15.83:0/1396279775 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e0a3e2a390 0x55e0a3e42c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:02.910 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.906+0000 7f18daf78740 1 --2- 172.21.15.83:0/1396279775 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e0a3e1fad0 0x55e0a3e426f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:02.910 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.906+0000 7f18daf78740 1 --2- 172.21.15.83:0/1396279775 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0a3e1f1b0 0x55e0a3e35e80 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:02.910 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.906+0000 7f18daf78740 1 -- 172.21.15.83:0/1396279775 >> 172.21.15.83:0/1396279775 conn(0x55e0a3e1b6c0 msgr2=0x55e0a3e22570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:14:02.910 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.906+0000 7f18daf78740 1 -- 172.21.15.83:0/1396279775 shutdown_connections 2024-04-24T09:14:02.910 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.907+0000 7f18daf78740 1 -- 172.21.15.83:0/1396279775 wait complete. 2024-04-24T09:14:02.910 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.907+0000 7f18daf78740 1 -- start start 2024-04-24T09:14:02.910 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.907+0000 7f18daf78740 1 Processor -- start 2024-04-24T09:14:02.910 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.908+0000 7f18daf78740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0a3e1f1b0 0x55e0a3e99340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:02.911 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.908+0000 7f18daf78740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e0a3e1fad0 0x55e0a3e99880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:02.911 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.908+0000 7f18daf78740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e0a3e2a390 0x55e0a3e2d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:02.911 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.908+0000 7f18daf78740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f18b00117d0 con 0x55e0a3e2a390 2024-04-24T09:14:02.911 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.908+0000 7f18daf78740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f18b0011100 con 0x55e0a3e1f1b0 2024-04-24T09:14:02.911 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.908+0000 7f18daf78740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f18b000e300 con 0x55e0a3e1fad0 2024-04-24T09:14:02.911 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.908+0000 7f18daf78740 10 client.0 _ll_register_callbacks cb 0x7f18bc0cd210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:14:02.911 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.908+0000 7f18d6200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e0a3e2a390 0x55e0a3e2d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:02.911 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.908+0000 7f18d6200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e0a3e2a390 0x55e0a3e2d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49820/0 (socket says 172.21.15.83:49820) 2024-04-24T09:14:02.911 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.908+0000 7f18d6200640 1 -- 172.21.15.83:0/3187508731 learned_addr learned my addr 172.21.15.83:0/3187508731 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:14:02.911 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.908+0000 7f18d5800640 1 --2- 172.21.15.83:0/3187508731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0a3e1f1b0 0x55e0a3e99340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:02.911 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.908+0000 7f18d4e00640 1 --2- 172.21.15.83:0/3187508731 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e0a3e1fad0 0x55e0a3e99880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:02.911 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.909+0000 7f18d6200640 1 -- 172.21.15.83:0/3187508731 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e0a3e1fad0 msgr2=0x55e0a3e99880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:14:02.911 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.909+0000 7f18d6200640 1 --2- 172.21.15.83:0/3187508731 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e0a3e1fad0 0x55e0a3e99880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:02.911 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.909+0000 7f18d6200640 1 -- 172.21.15.83:0/3187508731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0a3e1f1b0 msgr2=0x55e0a3e99340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:14:02.911 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.909+0000 7f18d6200640 1 --2- 172.21.15.83:0/3187508731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0a3e1f1b0 0x55e0a3e99340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:02.912 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.909+0000 7f18d6200640 1 -- 172.21.15.83:0/3187508731 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0a3b64620 con 0x55e0a3e2a390 2024-04-24T09:14:02.912 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.909+0000 7f18d4e00640 1 --2- 172.21.15.83:0/3187508731 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e0a3e1fad0 0x55e0a3e99880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:14:02.912 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.909+0000 7f18d5800640 1 --2- 172.21.15.83:0/3187508731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0a3e1f1b0 0x55e0a3e99340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:14:02.912 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.909+0000 7f18d6200640 1 --2- 172.21.15.83:0/3187508731 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e0a3e2a390 0x55e0a3e2d760 secure :-1 s=READY pgs=2651 cs=0 l=1 rev1=1 crypto rx=0x7f18c80128e0 tx=0x7f18c8009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9460f90003755dbe server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:14:02.912 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.909+0000 7f18b7400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:14:02.913 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.909+0000 7f18b7400640 1 -- 172.21.15.83:0/3187508731 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18c8020bd0 con 0x55e0a3e2a390 2024-04-24T09:14:02.913 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.910+0000 7f18b7400640 1 -- 172.21.15.83:0/3187508731 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(127..127 src has 1..127) ==== 9671+0+0 (secure 0 0 0) 0x7f18c8020d70 con 0x55e0a3e2a390 2024-04-24T09:14:02.913 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.910+0000 7f18daf78740 1 -- 172.21.15.83:0/3187508731 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18c4058030 con 0x55e0a3e2a390 2024-04-24T09:14:02.913 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.910+0000 7f18b7400640 1 -- 172.21.15.83:0/3187508731 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f18c801c430 con 0x55e0a3e2a390 2024-04-24T09:14:02.913 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.910+0000 7f18daf78740 10 client.23265 fetch_fsmap learned FSMap version 64 2024-04-24T09:14:02.914 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.910+0000 7f18daf78740 1 -- 172.21.15.83:0/3187508731 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55e0a3ea0940 con 0x55e0a3e2a390 2024-04-24T09:14:02.914 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.910+0000 7f18b7400640 1 -- 172.21.15.83:0/3187508731 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f18c4058030 con 0x55e0a3e2a390 2024-04-24T09:14:02.914 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.910+0000 7f18b7400640 1 -- 172.21.15.83:0/3187508731 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 64) ==== 8374+0+0 (secure 0 0 0) 0x7f18c801e6d0 con 0x55e0a3e2a390 2024-04-24T09:14:02.914 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.911+0000 7f18daf78740 10 client.23265 fetch_fsmap finished waiting for FSMap version 64 2024-04-24T09:14:02.914 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.911+0000 7f18daf78740 10 client.23265 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:14:02.914 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.911+0000 7f18daf78740 20 client.23265 populate_metadata read hostname 'smithi083' 2024-04-24T09:14:02.914 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.911+0000 7f18daf78740 1 --2- 172.21.15.83:0/3187508731 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55e0a3ea1040 0x55e0a3ea3480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:02.914 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.911+0000 7f18daf78740 4 client.23265 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T09:14:02.914 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.911+0000 7f18daf78740 1 -- 172.21.15.83:0/3187508731 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e0a3d8d190 con 0x55e0a3ea1040 2024-04-24T09:14:02.914 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.911+0000 7f18d5800640 1 --2- 172.21.15.83:0/3187508731 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55e0a3ea1040 0x55e0a3ea3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:02.914 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.912+0000 7f18d5800640 1 --2- 172.21.15.83:0/3187508731 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55e0a3ea1040 0x55e0a3ea3480 crc :-1 s=READY pgs=556 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86d27301a9b158c5 server_cookie=d05549d7dd7446a0 in_seq=0 out_seq=0 2024-04-24T09:14:02.914 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:02.912+0000 7f18b7400640 10 client.23265 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:14:03.038 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:03.037+0000 7f18b7400640 1 -- 172.21.15.83:0/3187508731 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e0a3d8d190 con 0x55e0a3ea1040 2024-04-24T09:14:03.038 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:03.037+0000 7f18b7400640 10 client.23265 handle_command_reply: tid=0 2024-04-24T09:14:03.038 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:03.037+0000 7f18daf78740 1 client.23265 shutdown 2024-04-24T09:14:03.038 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:03.037+0000 7f18daf78740 1 -- 172.21.15.83:0/3187508731 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e0a3e2a390 msgr2=0x55e0a3e2d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:03.038 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:03.037+0000 7f18daf78740 1 --2- 172.21.15.83:0/3187508731 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e0a3e2a390 0x55e0a3e2d760 secure :-1 s=READY pgs=2651 cs=0 l=1 rev1=1 crypto rx=0x7f18c80128e0 tx=0x7f18c8009500 comp rx=0 tx=0).stop 2024-04-24T09:14:03.038 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:03.037+0000 7f18daf78740 1 -- 172.21.15.83:0/3187508731 shutdown_connections 2024-04-24T09:14:03.038 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:03.037+0000 7f18daf78740 1 --2- 172.21.15.83:0/3187508731 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55e0a3ea1040 0x55e0a3ea3480 crc :-1 s=READY pgs=556 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:03.038 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:03.037+0000 7f18daf78740 1 --2- 172.21.15.83:0/3187508731 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e0a3e2a390 0x55e0a3e2d760 unknown :-1 s=CLOSED pgs=2651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:03.039 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:03.038+0000 7f18daf78740 1 --2- 172.21.15.83:0/3187508731 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e0a3e1fad0 0x55e0a3e99880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:03.039 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:03.038+0000 7f18daf78740 1 --2- 172.21.15.83:0/3187508731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0a3e1f1b0 0x55e0a3e99340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:03.039 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:03.038+0000 7f18daf78740 1 -- 172.21.15.83:0/3187508731 >> 172.21.15.83:0/3187508731 conn(0x55e0a3e1b6c0 msgr2=0x55e0a3e27250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:14:03.039 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:03.038+0000 7f18b7400640 0 client.23265 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:14:03.039 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:03.038+0000 7f18daf78740 1 -- 172.21.15.83:0/3187508731 shutdown_connections 2024-04-24T09:14:03.039 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:03.038+0000 7f18daf78740 1 -- 172.21.15.83:0/3187508731 wait complete. 2024-04-24T09:14:03.040 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:03.038+0000 7f18daf78740 20 client.23265 trim_cache size 0 max 16384 2024-04-24T09:14:03.040 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T09:14:03.040 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T09:14:03.040 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T09:14:03.040 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T09:14:03.040 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:03.040+0000 7f18d8200640 1 -- 172.21.15.83:0/1688264120 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f18b80a3ec0 msgr2=0x7f18b80a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:03.040 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:03.040+0000 7f18d8200640 1 --2- 172.21.15.83:0/1688264120 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f18b80a3ec0 0x7f18b80a6380 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f18c8005fd0 tx=0x7f18c8004ab0 comp rx=0 tx=0).stop 2024-04-24T09:14:03.041 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:03.040+0000 7f18d8200640 1 -- 172.21.15.83:0/1688264120 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f18d0107350 msgr2=0x7f18d019f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:03.041 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:03.040+0000 7f18d8200640 1 --2- 172.21.15.83:0/1688264120 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f18d0107350 0x7f18d019f260 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f18bc00d6a0 tx=0x7f18bc00db60 comp rx=0 tx=0).stop 2024-04-24T09:14:03.041 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:03.041+0000 7f18d8200640 1 -- 172.21.15.83:0/1688264120 shutdown_connections 2024-04-24T09:14:03.041 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:03.041+0000 7f18d8200640 1 --2- 172.21.15.83:0/1688264120 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f18b80a3ec0 0x7f18b80a6380 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:03.041 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:03.041+0000 7f18d8200640 1 --2- 172.21.15.83:0/1688264120 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18d01a35e0 0x7f18d01a3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:03.041 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:03.041+0000 7f18d8200640 1 --2- 172.21.15.83:0/1688264120 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f18d0107350 0x7f18d019f260 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:03.041 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:03.041+0000 7f18d8200640 1 --2- 172.21.15.83:0/1688264120 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18d0102280 0x7f18d019ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:03.041 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:03.041+0000 7f18d8200640 1 -- 172.21.15.83:0/1688264120 >> 172.21.15.83:0/1688264120 conn(0x7f18d0079490 msgr2=0x7f18d00ffdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:14:03.042 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:03.041+0000 7f18d8200640 1 -- 172.21.15.83:0/1688264120 shutdown_connections 2024-04-24T09:14:03.042 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:03.041+0000 7f18d8200640 1 -- 172.21.15.83:0/1688264120 wait complete. 2024-04-24T09:14:03.203 INFO:tasks.workunit.client.0.smithi083.stdout: 10 63735 6.59 MB/sec execute 394 sec latency 116.307 ms 2024-04-24T09:14:03.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:14:03 smithi083 ceph-mon[27907]: osdmap e127: 12 total, 12 up, 12 in 2024-04-24T09:14:03.706 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:14:03 smithi138 ceph-mon[31441]: osdmap e127: 12 total, 12 up, 12 in 2024-04-24T09:14:03.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:14:03 smithi189 ceph-mon[31791]: osdmap e127: 12 total, 12 up, 12 in 2024-04-24T09:14:04.203 INFO:tasks.workunit.client.0.smithi083.stdout: 10 63851 6.59 MB/sec execute 395 sec latency 64.196 ms 2024-04-24T09:14:04.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:14:04 smithi083 ceph-mon[27907]: pgmap v709: 129 pgs: 12 active+clean+snaptrim_wait, 7 active+clean+snaptrim, 110 active+clean; 968 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.1 MiB/s wr, 291 op/s 2024-04-24T09:14:04.704 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:14:04 smithi138 ceph-mon[31441]: pgmap v709: 129 pgs: 12 active+clean+snaptrim_wait, 7 active+clean+snaptrim, 110 active+clean; 968 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.1 MiB/s wr, 291 op/s 2024-04-24T09:14:04.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:14:04 smithi189 ceph-mon[31791]: pgmap v709: 129 pgs: 12 active+clean+snaptrim_wait, 7 active+clean+snaptrim, 110 active+clean; 968 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.1 MiB/s wr, 291 op/s 2024-04-24T09:14:05.203 INFO:tasks.workunit.client.0.smithi083.stdout: 10 63972 6.59 MB/sec execute 396 sec latency 77.673 ms 2024-04-24T09:14:06.204 INFO:tasks.workunit.client.0.smithi083.stdout: 10 64139 6.60 MB/sec execute 397 sec latency 77.147 ms 2024-04-24T09:14:06.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:14:06 smithi083 ceph-mon[27907]: pgmap v710: 129 pgs: 12 active+clean+snaptrim_wait, 7 active+clean+snaptrim, 110 active+clean; 968 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.1 MiB/s wr, 291 op/s 2024-04-24T09:14:06.704 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:14:06 smithi138 ceph-mon[31441]: pgmap v710: 129 pgs: 12 active+clean+snaptrim_wait, 7 active+clean+snaptrim, 110 active+clean; 968 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.1 MiB/s wr, 291 op/s 2024-04-24T09:14:06.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:14:06 smithi189 ceph-mon[31791]: pgmap v710: 129 pgs: 12 active+clean+snaptrim_wait, 7 active+clean+snaptrim, 110 active+clean; 968 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.1 MiB/s wr, 291 op/s 2024-04-24T09:14:07.203 INFO:tasks.workunit.client.0.smithi083.stdout: 10 64259 6.61 MB/sec execute 398 sec latency 333.020 ms 2024-04-24T09:14:07.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:14:07 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:14:07.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:14:07 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:14:07.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:14:07 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:14:07.704 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:14:07 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:14:07.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:14:07 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:14:07.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:14:07 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:14:07.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:14:07 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:14:07.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:14:07 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:14:07.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:14:07 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:14:08.204 INFO:tasks.workunit.client.0.smithi083.stdout: 10 64354 6.60 MB/sec execute 399 sec latency 267.573 ms 2024-04-24T09:14:08.555 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:14:08 smithi083 ceph-mon[27907]: pgmap v711: 129 pgs: 12 active+clean+snaptrim_wait, 7 active+clean+snaptrim, 110 active+clean; 968 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.1 MiB/s wr, 291 op/s 2024-04-24T09:14:08.555 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:14:08 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:14:08.555 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:14:08 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:14:08.704 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:14:08 smithi138 ceph-mon[31441]: pgmap v711: 129 pgs: 12 active+clean+snaptrim_wait, 7 active+clean+snaptrim, 110 active+clean; 968 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.1 MiB/s wr, 291 op/s 2024-04-24T09:14:08.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:14:08 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:14:08.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:14:08 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:14:08.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:14:08 smithi189 ceph-mon[31791]: pgmap v711: 129 pgs: 12 active+clean+snaptrim_wait, 7 active+clean+snaptrim, 110 active+clean; 968 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.1 MiB/s wr, 291 op/s 2024-04-24T09:14:08.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:14:08 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:14:08.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:14:08 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:14:09.204 INFO:tasks.workunit.client.0.smithi083.stdout: 10 64487 6.61 MB/sec execute 400 sec latency 743.797 ms 2024-04-24T09:14:10.211 INFO:tasks.workunit.client.0.smithi083.stdout: 10 64649 6.63 MB/sec execute 401 sec latency 126.592 ms 2024-04-24T09:14:10.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:14:10 smithi083 ceph-mon[27907]: pgmap v712: 129 pgs: 129 active+clean; 980 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.5 MiB/s wr, 272 op/s 2024-04-24T09:14:10.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:14:10 smithi083 ceph-mon[27907]: osdmap e128: 12 total, 12 up, 12 in 2024-04-24T09:14:10.706 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:14:10 smithi138 ceph-mon[31441]: pgmap v712: 129 pgs: 129 active+clean; 980 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.5 MiB/s wr, 272 op/s 2024-04-24T09:14:10.706 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:14:10 smithi138 ceph-mon[31441]: osdmap e128: 12 total, 12 up, 12 in 2024-04-24T09:14:10.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:14:10 smithi189 ceph-mon[31791]: pgmap v712: 129 pgs: 129 active+clean; 980 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.5 MiB/s wr, 272 op/s 2024-04-24T09:14:10.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:14:10 smithi189 ceph-mon[31791]: osdmap e128: 12 total, 12 up, 12 in 2024-04-24T09:14:11.204 INFO:tasks.workunit.client.0.smithi083.stdout: 10 64737 6.62 MB/sec execute 402 sec latency 300.374 ms 2024-04-24T09:14:12.204 INFO:tasks.workunit.client.0.smithi083.stdout: 10 64836 6.62 MB/sec execute 403 sec latency 128.619 ms 2024-04-24T09:14:12.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:14:12 smithi083 ceph-mon[27907]: pgmap v714: 129 pgs: 129 active+clean; 980 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.2 MiB/s wr, 312 op/s 2024-04-24T09:14:12.704 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:14:12 smithi138 ceph-mon[31441]: pgmap v714: 129 pgs: 129 active+clean; 980 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.2 MiB/s wr, 312 op/s 2024-04-24T09:14:12.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:14:12 smithi189 ceph-mon[31791]: pgmap v714: 129 pgs: 129 active+clean; 980 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.2 MiB/s wr, 312 op/s 2024-04-24T09:14:13.204 INFO:tasks.workunit.client.0.smithi083.stdout: 10 64964 6.63 MB/sec execute 404 sec latency 67.652 ms 2024-04-24T09:14:14.204 INFO:tasks.workunit.client.0.smithi083.stdout: 10 65086 6.64 MB/sec execute 405 sec latency 93.008 ms 2024-04-24T09:14:14.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:14:14 smithi083 ceph-mon[27907]: pgmap v715: 129 pgs: 129 active+clean; 996 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.9 MiB/s wr, 238 op/s 2024-04-24T09:14:14.704 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:14:14 smithi138 ceph-mon[31441]: pgmap v715: 129 pgs: 129 active+clean; 996 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.9 MiB/s wr, 238 op/s 2024-04-24T09:14:14.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:14:14 smithi189 ceph-mon[31791]: pgmap v715: 129 pgs: 129 active+clean; 996 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.9 MiB/s wr, 238 op/s 2024-04-24T09:14:15.204 INFO:tasks.workunit.client.0.smithi083.stdout: 10 65215 6.63 MB/sec execute 406 sec latency 101.192 ms 2024-04-24T09:14:16.204 INFO:tasks.workunit.client.0.smithi083.stdout: 10 65302 6.62 MB/sec execute 407 sec latency 99.332 ms 2024-04-24T09:14:16.220 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.221+0000 7f14df200640 1 -- 172.21.15.83:0/43236190 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14e0107140 msgr2=0x7f14e010ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:16.221 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.221+0000 7f14df200640 1 --2- 172.21.15.83:0/43236190 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14e0107140 0x7f14e010ed10 secure :-1 s=READY pgs=2652 cs=0 l=1 rev1=1 crypto rx=0x7f14d8009ca0 tx=0x7f14d802a8c0 comp rx=0 tx=0).stop 2024-04-24T09:14:16.221 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.221+0000 7f14df200640 1 -- 172.21.15.83:0/43236190 shutdown_connections 2024-04-24T09:14:16.221 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.221+0000 7f14df200640 1 --2- 172.21.15.83:0/43236190 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14e0107140 0x7f14e010ed10 unknown :-1 s=CLOSED pgs=2652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:16.221 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.221+0000 7f14df200640 1 --2- 172.21.15.83:0/43236190 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f14e0102a20 0x7f14e0106a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:16.221 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.221+0000 7f14df200640 1 --2- 172.21.15.83:0/43236190 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f14e0102070 0x7f14e0102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:16.221 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.221+0000 7f14df200640 1 -- 172.21.15.83:0/43236190 >> 172.21.15.83:0/43236190 conn(0x7f14e00fd950 msgr2=0x7f14e00ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:14:16.221 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.221+0000 7f14df200640 1 -- 172.21.15.83:0/43236190 shutdown_connections 2024-04-24T09:14:16.221 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.221+0000 7f14df200640 1 -- 172.21.15.83:0/43236190 wait complete. 2024-04-24T09:14:16.222 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.222+0000 7f14df200640 1 Processor -- start 2024-04-24T09:14:16.222 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.222+0000 7f14df200640 1 -- start start 2024-04-24T09:14:16.222 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.223+0000 7f14df200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14e0102070 0x7f14e01a0540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:16.223 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.223+0000 7f14df200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f14e0107140 0x7f14e01a0a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:16.223 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.223+0000 7f14df200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f14e019a6a0 0x7f14e019ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:16.223 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.223+0000 7f14df200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f14e0112100 con 0x7f14e0102070 2024-04-24T09:14:16.223 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.223+0000 7f14df200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f14e0111f80 con 0x7f14e019a6a0 2024-04-24T09:14:16.223 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.223+0000 7f14df200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f14e0112280 con 0x7f14e0107140 2024-04-24T09:14:16.224 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.224+0000 7f14de800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f14e019a6a0 0x7f14e019ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:16.224 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.224+0000 7f14de800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f14e019a6a0 0x7f14e019ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:42958/0 (socket says 172.21.15.83:42958) 2024-04-24T09:14:16.224 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.224+0000 7f14de800640 1 -- 172.21.15.83:0/1593011404 learned_addr learned my addr 172.21.15.83:0/1593011404 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:14:16.224 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.224+0000 7f14dde00640 1 --2- 172.21.15.83:0/1593011404 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14e0102070 0x7f14e01a0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:16.224 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.224+0000 7f14dd400640 1 --2- 172.21.15.83:0/1593011404 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f14e0107140 0x7f14e01a0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:16.224 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.224+0000 7f14de800640 1 -- 172.21.15.83:0/1593011404 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f14e0107140 msgr2=0x7f14e01a0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:16.224 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.224+0000 7f14de800640 1 --2- 172.21.15.83:0/1593011404 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f14e0107140 0x7f14e01a0a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:16.224 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.224+0000 7f14de800640 1 -- 172.21.15.83:0/1593011404 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14e0102070 msgr2=0x7f14e01a0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:16.224 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.224+0000 7f14de800640 1 --2- 172.21.15.83:0/1593011404 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14e0102070 0x7f14e01a0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:16.224 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.224+0000 7f14de800640 1 -- 172.21.15.83:0/1593011404 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14e019b2a0 con 0x7f14e019a6a0 2024-04-24T09:14:16.224 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.224+0000 7f14dd400640 1 --2- 172.21.15.83:0/1593011404 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f14e0107140 0x7f14e01a0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:14:16.225 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.225+0000 7f14de800640 1 --2- 172.21.15.83:0/1593011404 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f14e019a6a0 0x7f14e019ab20 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f14d80098d0 tx=0x7f14d8032fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:14:16.225 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.225+0000 7f14d6a00640 1 -- 172.21.15.83:0/1593011404 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d8003c70 con 0x7f14e019a6a0 2024-04-24T09:14:16.225 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.225+0000 7f14d6a00640 1 -- 172.21.15.83:0/1593011404 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f14d8003e10 con 0x7f14e019a6a0 2024-04-24T09:14:16.225 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.225+0000 7f14d6a00640 1 -- 172.21.15.83:0/1593011404 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d8030bf0 con 0x7f14e019a6a0 2024-04-24T09:14:16.225 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.225+0000 7f14df200640 1 -- 172.21.15.83:0/1593011404 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14e019b530 con 0x7f14e019a6a0 2024-04-24T09:14:16.225 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.226+0000 7f14df200640 1 -- 172.21.15.83:0/1593011404 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14e0108ff0 con 0x7f14e019a6a0 2024-04-24T09:14:16.229 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.230+0000 7f14e7a03740 1 Processor -- start 2024-04-24T09:14:16.231 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.232+0000 7f14d6a00640 1 -- 172.21.15.83:0/1593011404 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f14d8004040 con 0x7f14e019a6a0 2024-04-24T09:14:16.231 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.232+0000 7f14e7a03740 1 -- start start 2024-04-24T09:14:16.232 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.232+0000 7f14e7a03740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a3d1a093c0 0x55a3d1a097a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:16.232 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.233+0000 7f14e7a03740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a3d1a09ce0 0x55a3d1a14060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:16.232 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.233+0000 7f14e7a03740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a3d1a145a0 0x55a3d1a16990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:16.232 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.233+0000 7f14e7a03740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a3d196f900 con 0x55a3d1a093c0 2024-04-24T09:14:16.233 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.233+0000 7f14e7a03740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a3d1915f20 con 0x55a3d1a145a0 2024-04-24T09:14:16.233 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.233+0000 7f14e7a03740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a3d196b8c0 con 0x55a3d1a09ce0 2024-04-24T09:14:16.233 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.233+0000 7f14dd400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a3d1a093c0 0x55a3d1a097a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:16.233 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.233+0000 7f14dd400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a3d1a093c0 0x55a3d1a097a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55122/0 (socket says 172.21.15.83:55122) 2024-04-24T09:14:16.233 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.233+0000 7f14dd400640 1 -- 172.21.15.83:0/2932741304 learned_addr learned my addr 172.21.15.83:0/2932741304 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:14:16.233 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.233+0000 7f14dde00640 1 --2- 172.21.15.83:0/2932741304 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a3d1a09ce0 0x55a3d1a14060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:16.233 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.234+0000 7f14dd400640 1 -- 172.21.15.83:0/2932741304 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a3d1a09ce0 msgr2=0x55a3d1a14060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:14:16.233 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.234+0000 7f14dd400640 1 --2- 172.21.15.83:0/2932741304 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a3d1a09ce0 0x55a3d1a14060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:16.234 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.234+0000 7f14dd400640 1 -- 172.21.15.83:0/2932741304 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a3d1a145a0 msgr2=0x55a3d1a16990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:14:16.234 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.234+0000 7f14dd400640 1 --2- 172.21.15.83:0/2932741304 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a3d1a145a0 0x55a3d1a16990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:16.234 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.234+0000 7f14dd400640 1 -- 172.21.15.83:0/2932741304 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3d193ae10 con 0x55a3d1a093c0 2024-04-24T09:14:16.234 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.234+0000 7f14d6a00640 1 --2- 172.21.15.83:0/1593011404 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f14c40a3ee0 0x7f14c40a63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:16.235 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.234+0000 7f14d6a00640 1 -- 172.21.15.83:0/1593011404 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9583+0+0 (secure 0 0 0) 0x7f14d80fab00 con 0x7f14e019a6a0 2024-04-24T09:14:16.235 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.235+0000 7f14dd400640 1 --2- 172.21.15.83:0/2932741304 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a3d1a093c0 0x55a3d1a097a0 secure :-1 s=READY pgs=2653 cs=0 l=1 rev1=1 crypto rx=0x7f14cc009c00 tx=0x7f14cc00e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9abca3d7428a982 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:14:16.235 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.235+0000 7f14dde00640 1 --2- 172.21.15.83:0/1593011404 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f14c40a3ee0 0x7f14c40a63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:16.235 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.235+0000 7f14cb000640 1 -- 172.21.15.83:0/2932741304 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14cc016070 con 0x55a3d1a093c0 2024-04-24T09:14:16.235 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.235+0000 7f14cb000640 1 -- 172.21.15.83:0/2932741304 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f14cc010880 con 0x55a3d1a093c0 2024-04-24T09:14:16.235 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.235+0000 7f14cb000640 1 -- 172.21.15.83:0/2932741304 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14cc016070 con 0x55a3d1a093c0 2024-04-24T09:14:16.236 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.237+0000 7f14e7a03740 1 -- 172.21.15.83:0/2932741304 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a3d1a093c0 msgr2=0x55a3d1a097a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:16.236 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.237+0000 7f14e7a03740 1 --2- 172.21.15.83:0/2932741304 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a3d1a093c0 0x55a3d1a097a0 secure :-1 s=READY pgs=2653 cs=0 l=1 rev1=1 crypto rx=0x7f14cc009c00 tx=0x7f14cc00e6c0 comp rx=0 tx=0).stop 2024-04-24T09:14:16.236 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.237+0000 7f14dde00640 1 --2- 172.21.15.83:0/1593011404 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f14c40a3ee0 0x7f14c40a63a0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x55a3d1a0a960 tx=0x7f14d0005f90 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:14:16.237 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.237+0000 7f14e7a03740 1 -- 172.21.15.83:0/2932741304 shutdown_connections 2024-04-24T09:14:16.237 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.237+0000 7f14e7a03740 1 --2- 172.21.15.83:0/2932741304 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a3d1a145a0 0x55a3d1a16990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:16.237 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.237+0000 7f14e7a03740 1 --2- 172.21.15.83:0/2932741304 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a3d1a09ce0 0x55a3d1a14060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:16.237 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.237+0000 7f14e7a03740 1 --2- 172.21.15.83:0/2932741304 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a3d1a093c0 0x55a3d1a097a0 unknown :-1 s=CLOSED pgs=2653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:16.237 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.237+0000 7f14e7a03740 1 -- 172.21.15.83:0/2932741304 >> 172.21.15.83:0/2932741304 conn(0x55a3d1a058d0 msgr2=0x55a3d1a05cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:14:16.237 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.237+0000 7f14e7a03740 1 -- 172.21.15.83:0/2932741304 shutdown_connections 2024-04-24T09:14:16.238 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.238+0000 7f14e7a03740 1 -- 172.21.15.83:0/2932741304 wait complete. 2024-04-24T09:14:16.238 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.239+0000 7f14e7a03740 1 -- start start 2024-04-24T09:14:16.238 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.239+0000 7f14e7a03740 1 Processor -- start 2024-04-24T09:14:16.239 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.239+0000 7f14e7a03740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a3d1a093c0 0x55a3d1a115d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:16.239 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.239+0000 7f14e7a03740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a3d1a09ce0 0x55a3d1a11b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:16.239 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.239+0000 7f14e7a03740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a3d1a145a0 0x55a3d1a12050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:16.239 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.239+0000 7f14e7a03740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a3d1a170e0 con 0x55a3d1a145a0 2024-04-24T09:14:16.239 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.240+0000 7f14e7a03740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a3d19292c0 con 0x55a3d1a093c0 2024-04-24T09:14:16.239 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.240+0000 7f14e7a03740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a3d1a17260 con 0x55a3d1a09ce0 2024-04-24T09:14:16.239 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.240+0000 7f14e7a03740 10 client.0 _ll_register_callbacks cb 0x55a3d1681a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:14:16.240 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.240+0000 7f14dd400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a3d1a09ce0 0x55a3d1a11b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:16.240 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.240+0000 7f14de800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a3d1a145a0 0x55a3d1a12050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:16.240 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.240+0000 7f14de800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a3d1a145a0 0x55a3d1a12050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55128/0 (socket says 172.21.15.83:55128) 2024-04-24T09:14:16.240 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.241+0000 7f14de800640 1 -- 172.21.15.83:0/3033087113 learned_addr learned my addr 172.21.15.83:0/3033087113 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:14:16.241 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.241+0000 7f14de800640 1 -- 172.21.15.83:0/3033087113 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a3d1a09ce0 msgr2=0x55a3d1a11b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:14:16.241 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.241+0000 7f14de800640 1 --2- 172.21.15.83:0/3033087113 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a3d1a09ce0 0x55a3d1a11b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:16.241 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.241+0000 7f14de800640 1 -- 172.21.15.83:0/3033087113 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a3d1a093c0 msgr2=0x55a3d1a115d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-24T09:14:16.241 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.241+0000 7f14de800640 1 --2- 172.21.15.83:0/3033087113 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a3d1a093c0 0x55a3d1a115d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:16.241 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.241+0000 7f14de800640 1 -- 172.21.15.83:0/3033087113 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3d1939fd0 con 0x55a3d1a145a0 2024-04-24T09:14:16.241 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.242+0000 7f14de800640 1 --2- 172.21.15.83:0/3033087113 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a3d1a145a0 0x55a3d1a12050 secure :-1 s=READY pgs=2654 cs=0 l=1 rev1=1 crypto rx=0x7f14d8009430 tx=0x7f14d80305c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3eae4428cb417272 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:14:16.241 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.242+0000 7f14bf400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:14:16.242 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.242+0000 7f14bf400640 1 -- 172.21.15.83:0/3033087113 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d80c6730 con 0x55a3d1a145a0 2024-04-24T09:14:16.242 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.242+0000 7f14bf400640 1 -- 172.21.15.83:0/3033087113 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(128..128 src has 1..128) ==== 9583+0+0 (secure 0 0 0) 0x7f14d80c2da0 con 0x55a3d1a145a0 2024-04-24T09:14:16.242 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.242+0000 7f14e7a03740 1 -- 172.21.15.83:0/3033087113 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14cc010880 con 0x55a3d1a145a0 2024-04-24T09:14:16.242 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.243+0000 7f14bf400640 1 -- 172.21.15.83:0/3033087113 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f14d80caae0 con 0x55a3d1a145a0 2024-04-24T09:14:16.242 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.243+0000 7f14e7a03740 10 client.23277 fetch_fsmap learned FSMap version 64 2024-04-24T09:14:16.243 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.243+0000 7f14e7a03740 1 -- 172.21.15.83:0/3033087113 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55a3d1682210 con 0x55a3d1a145a0 2024-04-24T09:14:16.243 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.243+0000 7f14bf400640 1 -- 172.21.15.83:0/3033087113 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f14cc010880 con 0x55a3d1a145a0 2024-04-24T09:14:16.244 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.243+0000 7f14bf400640 1 -- 172.21.15.83:0/3033087113 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 64) ==== 8374+0+0 (secure 0 0 0) 0x7f14d80c6990 con 0x55a3d1a145a0 2024-04-24T09:14:16.244 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.243+0000 7f14e7a03740 10 client.23277 fetch_fsmap finished waiting for FSMap version 64 2024-04-24T09:14:16.244 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.243+0000 7f14e7a03740 10 client.23277 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:14:16.244 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.243+0000 7f14e7a03740 20 client.23277 populate_metadata read hostname 'smithi083' 2024-04-24T09:14:16.244 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.244+0000 7f14e7a03740 1 --2- 172.21.15.83:0/3033087113 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55a3d1a35720 0x55a3d1a37b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:16.244 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.244+0000 7f14e7a03740 4 client.23277 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:14:16.244 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.244+0000 7f14e7a03740 1 -- 172.21.15.83:0/3033087113 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a3d197d320 con 0x55a3d1a35720 2024-04-24T09:14:16.245 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.246+0000 7f14dde00640 1 --2- 172.21.15.83:0/3033087113 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55a3d1a35720 0x55a3d1a37b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:16.246 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.247+0000 7f14dde00640 1 --2- 172.21.15.83:0/3033087113 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55a3d1a35720 0x55a3d1a37b70 crc :-1 s=READY pgs=557 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b8ab79012086f96 server_cookie=b05202f9ef3d8c51 in_seq=0 out_seq=0 2024-04-24T09:14:16.246 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.247+0000 7f14bf400640 10 client.23277 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:14:16.252 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.253+0000 7f14bf400640 1 -- 172.21.15.83:0/3033087113 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a3d197d320 con 0x55a3d1a35720 2024-04-24T09:14:16.253 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.253+0000 7f14bf400640 10 client.23277 handle_command_reply: tid=0 2024-04-24T09:14:16.253 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.253+0000 7f14e7a03740 1 client.23277 shutdown 2024-04-24T09:14:16.253 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.253+0000 7f14e7a03740 1 -- 172.21.15.83:0/3033087113 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a3d1a145a0 msgr2=0x55a3d1a12050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:16.253 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.253+0000 7f14e7a03740 1 --2- 172.21.15.83:0/3033087113 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a3d1a145a0 0x55a3d1a12050 secure :-1 s=READY pgs=2654 cs=0 l=1 rev1=1 crypto rx=0x7f14d8009430 tx=0x7f14d80305c0 comp rx=0 tx=0).stop 2024-04-24T09:14:16.254 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.254+0000 7f14e7a03740 1 -- 172.21.15.83:0/3033087113 shutdown_connections 2024-04-24T09:14:16.254 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.254+0000 7f14e7a03740 1 --2- 172.21.15.83:0/3033087113 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55a3d1a35720 0x55a3d1a37b70 crc :-1 s=READY pgs=557 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:16.254 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.254+0000 7f14e7a03740 1 --2- 172.21.15.83:0/3033087113 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a3d1a145a0 0x55a3d1a12050 unknown :-1 s=CLOSED pgs=2654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:16.254 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.254+0000 7f14e7a03740 1 --2- 172.21.15.83:0/3033087113 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a3d1a09ce0 0x55a3d1a11b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:16.254 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.254+0000 7f14e7a03740 1 --2- 172.21.15.83:0/3033087113 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a3d1a093c0 0x55a3d1a115d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:16.254 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.254+0000 7f14bf400640 0 client.23277 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:14:16.254 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.254+0000 7f14e7a03740 1 -- 172.21.15.83:0/3033087113 >> 172.21.15.83:0/3033087113 conn(0x55a3d1a058d0 msgr2=0x55a3d1a0b650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:14:16.254 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.254+0000 7f14e7a03740 1 -- 172.21.15.83:0/3033087113 shutdown_connections 2024-04-24T09:14:16.254 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.254+0000 7f14e7a03740 1 -- 172.21.15.83:0/3033087113 wait complete. 2024-04-24T09:14:16.254 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.255+0000 7f14e7a03740 20 client.23277 trim_cache size 0 max 16384 2024-04-24T09:14:16.278 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.279+0000 7f14e7a03740 1 Processor -- start 2024-04-24T09:14:16.278 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.279+0000 7f14e7a03740 1 -- start start 2024-04-24T09:14:16.279 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.280+0000 7f14e7a03740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a3d1a093c0 0x55a3d1a13d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:16.279 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.280+0000 7f14e7a03740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a3d1a09ce0 0x55a3d1a05210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:16.279 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.280+0000 7f14e7a03740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a3d1a145a0 0x55a3d1a0bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:16.279 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.280+0000 7f14e7a03740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f14e00718e0 con 0x55a3d1a09ce0 2024-04-24T09:14:16.279 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.280+0000 7f14e7a03740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f14e00734b0 con 0x55a3d1a145a0 2024-04-24T09:14:16.279 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.280+0000 7f14e7a03740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f14e0071fb0 con 0x55a3d1a093c0 2024-04-24T09:14:16.280 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.281+0000 7f14de800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a3d1a145a0 0x55a3d1a0bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:16.280 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.281+0000 7f14de800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a3d1a145a0 0x55a3d1a0bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:42978/0 (socket says 172.21.15.83:42978) 2024-04-24T09:14:16.280 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.281+0000 7f14de800640 1 -- 172.21.15.83:0/422419251 learned_addr learned my addr 172.21.15.83:0/422419251 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:14:16.280 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.281+0000 7f14dde00640 1 --2- 172.21.15.83:0/422419251 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a3d1a093c0 0x55a3d1a13d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:16.280 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.281+0000 7f14dd400640 1 --2- 172.21.15.83:0/422419251 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a3d1a09ce0 0x55a3d1a05210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:16.281 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.281+0000 7f14de800640 1 -- 172.21.15.83:0/422419251 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a3d1a093c0 msgr2=0x55a3d1a13d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:14:16.281 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.281+0000 7f14de800640 1 --2- 172.21.15.83:0/422419251 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a3d1a093c0 0x55a3d1a13d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:16.281 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.281+0000 7f14de800640 1 -- 172.21.15.83:0/422419251 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a3d1a09ce0 msgr2=0x55a3d1a05210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:14:16.281 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.281+0000 7f14de800640 1 --2- 172.21.15.83:0/422419251 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a3d1a09ce0 0x55a3d1a05210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:16.281 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.281+0000 7f14de800640 1 -- 172.21.15.83:0/422419251 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3d1a105e0 con 0x55a3d1a145a0 2024-04-24T09:14:16.281 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.281+0000 7f14dd400640 1 --2- 172.21.15.83:0/422419251 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a3d1a09ce0 0x55a3d1a05210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:14:16.281 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.282+0000 7f14de800640 1 --2- 172.21.15.83:0/422419251 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a3d1a145a0 0x55a3d1a0bd80 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f14d80c66c0 tx=0x7f14d80c4c40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=916c5a1e7b720907 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:14:16.287 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.288+0000 7f14ca600640 1 -- 172.21.15.83:0/422419251 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d80c6270 con 0x55a3d1a145a0 2024-04-24T09:14:16.287 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.288+0000 7f14ca600640 1 -- 172.21.15.83:0/422419251 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f14d8036040 con 0x55a3d1a145a0 2024-04-24T09:14:16.287 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.288+0000 7f14ca600640 1 -- 172.21.15.83:0/422419251 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d80c6990 con 0x55a3d1a145a0 2024-04-24T09:14:16.288 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.289+0000 7f14e7a03740 1 -- 172.21.15.83:0/422419251 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a3d1a145a0 msgr2=0x55a3d1a0bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:16.288 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.289+0000 7f14e7a03740 1 --2- 172.21.15.83:0/422419251 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a3d1a145a0 0x55a3d1a0bd80 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f14d80c66c0 tx=0x7f14d80c4c40 comp rx=0 tx=0).stop 2024-04-24T09:14:16.288 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.289+0000 7f14e7a03740 1 -- 172.21.15.83:0/422419251 shutdown_connections 2024-04-24T09:14:16.288 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.289+0000 7f14e7a03740 1 --2- 172.21.15.83:0/422419251 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a3d1a145a0 0x55a3d1a0bd80 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:16.289 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.289+0000 7f14e7a03740 1 --2- 172.21.15.83:0/422419251 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a3d1a09ce0 0x55a3d1a05210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:16.289 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.289+0000 7f14e7a03740 1 --2- 172.21.15.83:0/422419251 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a3d1a093c0 0x55a3d1a13d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:16.289 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.289+0000 7f14e7a03740 1 -- 172.21.15.83:0/422419251 >> 172.21.15.83:0/422419251 conn(0x55a3d1a058d0 msgr2=0x55a3d1a11e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:14:16.289 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.289+0000 7f14e7a03740 1 -- 172.21.15.83:0/422419251 shutdown_connections 2024-04-24T09:14:16.289 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.289+0000 7f14e7a03740 1 -- 172.21.15.83:0/422419251 wait complete. 2024-04-24T09:14:16.290 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.290+0000 7f14e7a03740 1 -- start start 2024-04-24T09:14:16.290 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.290+0000 7f14e7a03740 1 Processor -- start 2024-04-24T09:14:16.290 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.290+0000 7f14e7a03740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a3d1a093c0 0x55a3d1a97570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:16.290 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.290+0000 7f14e7a03740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a3d1a09ce0 0x55a3d1a1c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:16.290 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.290+0000 7f14e7a03740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a3d1a145a0 0x55a3d1a1cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:16.291 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.290+0000 7f14e7a03740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f14e0072db0 con 0x55a3d1a093c0 2024-04-24T09:14:16.291 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.290+0000 7f14e7a03740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f14e00726e0 con 0x55a3d1a09ce0 2024-04-24T09:14:16.291 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.290+0000 7f14e7a03740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f14e006f920 con 0x55a3d1a145a0 2024-04-24T09:14:16.291 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.291+0000 7f14e7a03740 10 client.0 _ll_register_callbacks cb 0x7f14d80099a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:14:16.291 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.291+0000 7f14dd400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a3d1a09ce0 0x55a3d1a1c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:16.291 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.291+0000 7f14dde00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a3d1a093c0 0x55a3d1a97570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:16.291 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.291+0000 7f14dde00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a3d1a093c0 0x55a3d1a97570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55146/0 (socket says 172.21.15.83:55146) 2024-04-24T09:14:16.291 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.291+0000 7f14dde00640 1 -- 172.21.15.83:0/123083731 learned_addr learned my addr 172.21.15.83:0/123083731 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:14:16.291 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.291+0000 7f14de800640 1 --2- 172.21.15.83:0/123083731 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a3d1a145a0 0x55a3d1a1cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:16.292 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.292+0000 7f14dde00640 1 -- 172.21.15.83:0/123083731 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a3d1a145a0 msgr2=0x55a3d1a1cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:14:16.292 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.292+0000 7f14dde00640 1 --2- 172.21.15.83:0/123083731 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a3d1a145a0 0x55a3d1a1cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:16.292 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.292+0000 7f14dde00640 1 -- 172.21.15.83:0/123083731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a3d1a09ce0 msgr2=0x55a3d1a1c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:14:16.292 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.292+0000 7f14dde00640 1 --2- 172.21.15.83:0/123083731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a3d1a09ce0 0x55a3d1a1c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:16.293 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.292+0000 7f14dde00640 1 -- 172.21.15.83:0/123083731 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3d1a16d00 con 0x55a3d1a093c0 2024-04-24T09:14:16.293 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.292+0000 7f14dd400640 1 --2- 172.21.15.83:0/123083731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a3d1a09ce0 0x55a3d1a1c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:14:16.293 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.292+0000 7f14dde00640 1 --2- 172.21.15.83:0/123083731 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a3d1a093c0 0x55a3d1a97570 secure :-1 s=READY pgs=2655 cs=0 l=1 rev1=1 crypto rx=0x7f14d0009460 tx=0x7f14d0015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf83e016f7cdf0ff server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:14:16.293 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.292+0000 7f14bf400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:14:16.293 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.293+0000 7f14bf400640 1 -- 172.21.15.83:0/123083731 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d0027600 con 0x55a3d1a093c0 2024-04-24T09:14:16.293 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.293+0000 7f14bf400640 1 -- 172.21.15.83:0/123083731 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(128..128 src has 1..128) ==== 9583+0+0 (secure 0 0 0) 0x7f14d00277a0 con 0x55a3d1a093c0 2024-04-24T09:14:16.293 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.293+0000 7f14e7a03740 1 -- 172.21.15.83:0/123083731 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14d8036040 con 0x55a3d1a093c0 2024-04-24T09:14:16.293 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.293+0000 7f14bf400640 1 -- 172.21.15.83:0/123083731 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f14d00035f0 con 0x55a3d1a093c0 2024-04-24T09:14:16.294 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.293+0000 7f14e7a03740 10 client.23283 fetch_fsmap learned FSMap version 64 2024-04-24T09:14:16.294 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.294+0000 7f14e7a03740 1 -- 172.21.15.83:0/123083731 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55a3d174e620 con 0x55a3d1a093c0 2024-04-24T09:14:16.294 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.294+0000 7f14bf400640 1 -- 172.21.15.83:0/123083731 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f14d8036040 con 0x55a3d1a093c0 2024-04-24T09:14:16.294 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.294+0000 7f14bf400640 1 -- 172.21.15.83:0/123083731 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 64) ==== 8374+0+0 (secure 0 0 0) 0x7f14d00307f0 con 0x55a3d1a093c0 2024-04-24T09:14:16.294 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.294+0000 7f14e7a03740 10 client.23283 fetch_fsmap finished waiting for FSMap version 64 2024-04-24T09:14:16.294 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.294+0000 7f14e7a03740 10 client.23283 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:14:16.294 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.294+0000 7f14e7a03740 20 client.23283 populate_metadata read hostname 'smithi083' 2024-04-24T09:14:16.295 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.294+0000 7f14e7a03740 1 --2- 172.21.15.83:0/123083731 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55a3d1aad650 0x55a3d1aafa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:16.295 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.294+0000 7f14e7a03740 4 client.23283 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T09:14:16.295 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.294+0000 7f14e7a03740 1 -- 172.21.15.83:0/123083731 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a3d1977190 con 0x55a3d1aad650 2024-04-24T09:14:16.295 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.295+0000 7f14dd400640 1 --2- 172.21.15.83:0/123083731 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55a3d1aad650 0x55a3d1aafa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:16.295 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.296+0000 7f14bf400640 10 client.23283 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:14:16.295 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.296+0000 7f14dd400640 1 --2- 172.21.15.83:0/123083731 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55a3d1aad650 0x55a3d1aafa70 crc :-1 s=READY pgs=558 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc4cc46856b5a1c2 server_cookie=3853b6e51ecb0beb in_seq=0 out_seq=0 2024-04-24T09:14:16.499 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.499+0000 7f14bf400640 1 -- 172.21.15.83:0/123083731 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a3d1977190 con 0x55a3d1aad650 2024-04-24T09:14:16.499 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.499+0000 7f14bf400640 10 client.23283 handle_command_reply: tid=0 2024-04-24T09:14:16.499 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.499+0000 7f14e7a03740 1 client.23283 shutdown 2024-04-24T09:14:16.499 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.500+0000 7f14e7a03740 1 -- 172.21.15.83:0/123083731 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a3d1a093c0 msgr2=0x55a3d1a97570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:16.499 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.500+0000 7f14e7a03740 1 --2- 172.21.15.83:0/123083731 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a3d1a093c0 0x55a3d1a97570 secure :-1 s=READY pgs=2655 cs=0 l=1 rev1=1 crypto rx=0x7f14d0009460 tx=0x7f14d0015040 comp rx=0 tx=0).stop 2024-04-24T09:14:16.500 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.500+0000 7f14e7a03740 1 -- 172.21.15.83:0/123083731 shutdown_connections 2024-04-24T09:14:16.500 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.500+0000 7f14e7a03740 1 --2- 172.21.15.83:0/123083731 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55a3d1aad650 0x55a3d1aafa70 crc :-1 s=READY pgs=558 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:16.500 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.500+0000 7f14e7a03740 1 --2- 172.21.15.83:0/123083731 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a3d1a145a0 0x55a3d1a1cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:16.500 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.500+0000 7f14e7a03740 1 --2- 172.21.15.83:0/123083731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a3d1a09ce0 0x55a3d1a1c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:16.500 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.500+0000 7f14bf400640 0 client.23283 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:14:16.500 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.500+0000 7f14e7a03740 1 --2- 172.21.15.83:0/123083731 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a3d1a093c0 0x55a3d1a97570 unknown :-1 s=CLOSED pgs=2655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:16.501 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.500+0000 7f14e7a03740 1 -- 172.21.15.83:0/123083731 >> 172.21.15.83:0/123083731 conn(0x55a3d1a058d0 msgr2=0x55a3d1a1f050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:14:16.501 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.501+0000 7f14e7a03740 1 -- 172.21.15.83:0/123083731 shutdown_connections 2024-04-24T09:14:16.501 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.501+0000 7f14e7a03740 1 -- 172.21.15.83:0/123083731 wait complete. 2024-04-24T09:14:16.501 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.501+0000 7f14e7a03740 20 client.23283 trim_cache size 0 max 16384 2024-04-24T09:14:16.501 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T09:14:16.501 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T09:14:16.501 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T09:14:16.501 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T09:14:16.502 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.502+0000 7f14df200640 1 -- 172.21.15.83:0/1593011404 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f14c40a3ee0 msgr2=0x7f14c40a63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:16.502 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.502+0000 7f14df200640 1 --2- 172.21.15.83:0/1593011404 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f14c40a3ee0 0x7f14c40a63a0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x55a3d1a0a960 tx=0x7f14d0005f90 comp rx=0 tx=0).stop 2024-04-24T09:14:16.502 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.502+0000 7f14df200640 1 -- 172.21.15.83:0/1593011404 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f14e019a6a0 msgr2=0x7f14e019ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:16.502 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.502+0000 7f14df200640 1 --2- 172.21.15.83:0/1593011404 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f14e019a6a0 0x7f14e019ab20 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f14d80098d0 tx=0x7f14d8032fd0 comp rx=0 tx=0).stop 2024-04-24T09:14:16.502 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.503+0000 7f14df200640 1 -- 172.21.15.83:0/1593011404 shutdown_connections 2024-04-24T09:14:16.502 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.503+0000 7f14df200640 1 --2- 172.21.15.83:0/1593011404 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f14c40a3ee0 0x7f14c40a63a0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:16.502 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.503+0000 7f14df200640 1 --2- 172.21.15.83:0/1593011404 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f14e019a6a0 0x7f14e019ab20 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:16.502 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.503+0000 7f14df200640 1 --2- 172.21.15.83:0/1593011404 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f14e0107140 0x7f14e01a0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:16.502 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.503+0000 7f14df200640 1 --2- 172.21.15.83:0/1593011404 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14e0102070 0x7f14e01a0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:16.502 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.503+0000 7f14df200640 1 -- 172.21.15.83:0/1593011404 >> 172.21.15.83:0/1593011404 conn(0x7f14e00fd950 msgr2=0x7f14e010ea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:14:16.503 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.503+0000 7f14df200640 1 -- 172.21.15.83:0/1593011404 shutdown_connections 2024-04-24T09:14:16.503 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:16.503+0000 7f14df200640 1 -- 172.21.15.83:0/1593011404 wait complete. 2024-04-24T09:14:16.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:14:16 smithi083 ceph-mon[27907]: pgmap v716: 129 pgs: 129 active+clean; 996 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.9 MiB/s wr, 238 op/s 2024-04-24T09:14:16.704 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:14:16 smithi138 ceph-mon[31441]: pgmap v716: 129 pgs: 129 active+clean; 996 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.9 MiB/s wr, 238 op/s 2024-04-24T09:14:16.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:14:16 smithi189 ceph-mon[31791]: pgmap v716: 129 pgs: 129 active+clean; 996 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.9 MiB/s wr, 238 op/s 2024-04-24T09:14:17.204 INFO:tasks.workunit.client.0.smithi083.stdout: 10 65383 6.61 MB/sec execute 408 sec latency 129.300 ms 2024-04-24T09:14:18.204 INFO:tasks.workunit.client.0.smithi083.stdout: 10 65500 6.62 MB/sec execute 409 sec latency 83.371 ms 2024-04-24T09:14:18.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:14:18 smithi083 ceph-mon[27907]: pgmap v717: 129 pgs: 129 active+clean; 996 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.9 MiB/s wr, 238 op/s 2024-04-24T09:14:18.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:14:18 smithi138 ceph-mon[31441]: pgmap v717: 129 pgs: 129 active+clean; 996 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.9 MiB/s wr, 238 op/s 2024-04-24T09:14:18.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:14:18 smithi189 ceph-mon[31791]: pgmap v717: 129 pgs: 129 active+clean; 996 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.9 MiB/s wr, 238 op/s 2024-04-24T09:14:19.204 INFO:tasks.workunit.client.0.smithi083.stdout: 10 65608 6.61 MB/sec execute 410 sec latency 81.541 ms 2024-04-24T09:14:20.204 INFO:tasks.workunit.client.0.smithi083.stdout: 10 65726 6.61 MB/sec execute 411 sec latency 74.209 ms 2024-04-24T09:14:20.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:14:20 smithi083 ceph-mon[27907]: pgmap v718: 129 pgs: 129 active+clean; 1004 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.3 MiB/s wr, 284 op/s 2024-04-24T09:14:20.704 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:14:20 smithi138 ceph-mon[31441]: pgmap v718: 129 pgs: 129 active+clean; 1004 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.3 MiB/s wr, 284 op/s 2024-04-24T09:14:20.706 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:14:20 smithi189 ceph-mon[31791]: pgmap v718: 129 pgs: 129 active+clean; 1004 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.3 MiB/s wr, 284 op/s 2024-04-24T09:14:21.204 INFO:tasks.workunit.client.0.smithi083.stdout: 10 65838 6.60 MB/sec execute 412 sec latency 83.135 ms 2024-04-24T09:14:22.205 INFO:tasks.workunit.client.0.smithi083.stdout: 10 65914 6.60 MB/sec execute 413 sec latency 415.729 ms 2024-04-24T09:14:22.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:14:22 smithi083 ceph-mon[27907]: pgmap v719: 129 pgs: 129 active+clean; 1004 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.8 MiB/s wr, 254 op/s 2024-04-24T09:14:22.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:14:22 smithi138 ceph-mon[31441]: pgmap v719: 129 pgs: 129 active+clean; 1004 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.8 MiB/s wr, 254 op/s 2024-04-24T09:14:22.706 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:14:22 smithi189 ceph-mon[31791]: pgmap v719: 129 pgs: 129 active+clean; 1004 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.8 MiB/s wr, 254 op/s 2024-04-24T09:14:23.205 INFO:tasks.workunit.client.0.smithi083.stdout: 10 66004 6.59 MB/sec execute 414 sec latency 204.896 ms 2024-04-24T09:14:24.205 INFO:tasks.workunit.client.0.smithi083.stdout: 10 66121 6.58 MB/sec execute 415 sec latency 98.308 ms 2024-04-24T09:14:24.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:14:24 smithi138 ceph-mon[31441]: pgmap v720: 129 pgs: 129 active+clean; 1008 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 6.1 MiB/s wr, 341 op/s 2024-04-24T09:14:24.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:14:24 smithi138 ceph-mon[31441]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:14:24.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:14:24 smithi189 ceph-mon[31791]: pgmap v720: 129 pgs: 129 active+clean; 1008 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 6.1 MiB/s wr, 341 op/s 2024-04-24T09:14:24.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:14:24 smithi189 ceph-mon[31791]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:14:24.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:14:24 smithi083 ceph-mon[27907]: pgmap v720: 129 pgs: 129 active+clean; 1008 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 6.1 MiB/s wr, 341 op/s 2024-04-24T09:14:24.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:14:24 smithi083 ceph-mon[27907]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:14:25.205 INFO:tasks.workunit.client.0.smithi083.stdout: 10 66255 6.59 MB/sec execute 416 sec latency 124.270 ms 2024-04-24T09:14:26.205 INFO:tasks.workunit.client.0.smithi083.stdout: 10 66379 6.58 MB/sec execute 417 sec latency 103.223 ms 2024-04-24T09:14:26.704 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:14:26 smithi138 ceph-mon[31441]: pgmap v721: 129 pgs: 129 active+clean; 1008 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.9 MiB/s wr, 257 op/s 2024-04-24T09:14:26.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:14:26 smithi189 ceph-mon[31791]: pgmap v721: 129 pgs: 129 active+clean; 1008 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.9 MiB/s wr, 257 op/s 2024-04-24T09:14:26.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:14:26 smithi083 ceph-mon[27907]: pgmap v721: 129 pgs: 129 active+clean; 1008 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.9 MiB/s wr, 257 op/s 2024-04-24T09:14:27.205 INFO:tasks.workunit.client.0.smithi083.stdout: 10 66498 6.57 MB/sec execute 418 sec latency 90.586 ms 2024-04-24T09:14:27.349 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T09:14:27.349 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T09:14:27.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.614+0000 7feb70000640 1 -- 172.21.15.83:0/3923776050 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb68109520 msgr2=0x7feb681110f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:27.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.614+0000 7feb70000640 1 --2- 172.21.15.83:0/3923776050 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb68109520 0x7feb681110f0 secure :-1 s=READY pgs=2656 cs=0 l=1 rev1=1 crypto rx=0x7feb60009ca0 tx=0x7feb6002a8b0 comp rx=0 tx=0).stop 2024-04-24T09:14:27.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.615+0000 7feb70000640 1 -- 172.21.15.83:0/3923776050 shutdown_connections 2024-04-24T09:14:27.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.615+0000 7feb70000640 1 --2- 172.21.15.83:0/3923776050 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb68109520 0x7feb681110f0 unknown :-1 s=CLOSED pgs=2656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:27.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.615+0000 7feb70000640 1 --2- 172.21.15.83:0/3923776050 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feb68104e00 0x7feb68108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:27.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.615+0000 7feb70000640 1 --2- 172.21.15.83:0/3923776050 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb68104450 0x7feb68104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:27.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.615+0000 7feb70000640 1 -- 172.21.15.83:0/3923776050 >> 172.21.15.83:0/3923776050 conn(0x7feb680ffc70 msgr2=0x7feb68102090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:14:27.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.615+0000 7feb70000640 1 -- 172.21.15.83:0/3923776050 shutdown_connections 2024-04-24T09:14:27.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.615+0000 7feb70000640 1 -- 172.21.15.83:0/3923776050 wait complete. 2024-04-24T09:14:27.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.617+0000 7feb70000640 1 Processor -- start 2024-04-24T09:14:27.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.617+0000 7feb70000640 1 -- start start 2024-04-24T09:14:27.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.618+0000 7feb70000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feb68104450 0x7feb681a06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:27.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.618+0000 7feb70000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb68109520 0x7feb681a0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:27.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.618+0000 7feb70000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb6819a880 0x7feb6819ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:27.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.618+0000 7feb70000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7feb681144e0 con 0x7feb68109520 2024-04-24T09:14:27.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.618+0000 7feb70000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7feb68114360 con 0x7feb6819a880 2024-04-24T09:14:27.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.618+0000 7feb70000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7feb68114660 con 0x7feb68104450 2024-04-24T09:14:27.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.618+0000 7feb6d600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feb68104450 0x7feb681a06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:27.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.619+0000 7feb6d600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feb68104450 0x7feb681a06d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:57218/0 (socket says 172.21.15.83:57218) 2024-04-24T09:14:27.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.619+0000 7feb6d600640 1 -- 172.21.15.83:0/2560160333 learned_addr learned my addr 172.21.15.83:0/2560160333 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:14:27.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.619+0000 7feb6cc00640 1 --2- 172.21.15.83:0/2560160333 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb68109520 0x7feb681a0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:27.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.619+0000 7feb6e000640 1 --2- 172.21.15.83:0/2560160333 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb6819a880 0x7feb6819ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:27.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.620+0000 7feb6e000640 1 -- 172.21.15.83:0/2560160333 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feb68104450 msgr2=0x7feb681a06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:27.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.620+0000 7feb6e000640 1 --2- 172.21.15.83:0/2560160333 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feb68104450 0x7feb681a06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:27.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.620+0000 7feb6e000640 1 -- 172.21.15.83:0/2560160333 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb68109520 msgr2=0x7feb681a0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:27.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.620+0000 7feb6e000640 1 --2- 172.21.15.83:0/2560160333 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb68109520 0x7feb681a0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:27.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.620+0000 7feb6e000640 1 -- 172.21.15.83:0/2560160333 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb6819b3a0 con 0x7feb6819a880 2024-04-24T09:14:27.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.621+0000 7feb6e000640 1 --2- 172.21.15.83:0/2560160333 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb6819a880 0x7feb6819ad30 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7feb60009bf0 tx=0x7feb60030d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:14:27.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.621+0000 7feb66000640 1 -- 172.21.15.83:0/2560160333 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb60004040 con 0x7feb6819a880 2024-04-24T09:14:27.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.621+0000 7feb66000640 1 -- 172.21.15.83:0/2560160333 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feb60048070 con 0x7feb6819a880 2024-04-24T09:14:27.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.621+0000 7feb66000640 1 -- 172.21.15.83:0/2560160333 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb60031ae0 con 0x7feb6819a880 2024-04-24T09:14:27.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.621+0000 7feb70000640 1 -- 172.21.15.83:0/2560160333 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb6819b630 con 0x7feb6819a880 2024-04-24T09:14:27.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.622+0000 7feb70000640 1 -- 172.21.15.83:0/2560160333 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb68076670 con 0x7feb6819a880 2024-04-24T09:14:27.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.627+0000 7feb72e44740 1 Processor -- start 2024-04-24T09:14:27.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.629+0000 7feb66000640 1 -- 172.21.15.83:0/2560160333 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7feb60031c80 con 0x7feb6819a880 2024-04-24T09:14:27.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.629+0000 7feb72e44740 1 -- start start 2024-04-24T09:14:27.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.630+0000 7feb72e44740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f74eec860 0x556f74eecc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:27.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.630+0000 7feb72e44740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f74eed180 0x556f74ef7310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:27.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.630+0000 7feb72e44740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f74ef7850 0x556f74ef9c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:27.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.630+0000 7feb72e44740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556f74e521f0 con 0x556f74eed180 2024-04-24T09:14:27.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.630+0000 7feb72e44740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556f74df8350 con 0x556f74eec860 2024-04-24T09:14:27.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.630+0000 7feb72e44740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x556f74e4e1b0 con 0x556f74ef7850 2024-04-24T09:14:27.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.631+0000 7feb6cc00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f74eec860 0x556f74eecc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:27.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.631+0000 7feb6cc00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f74eec860 0x556f74eecc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:53018/0 (socket says 172.21.15.83:53018) 2024-04-24T09:14:27.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.631+0000 7feb6cc00640 1 -- 172.21.15.83:0/2118353453 learned_addr learned my addr 172.21.15.83:0/2118353453 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:14:27.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.631+0000 7feb6d600640 1 --2- 172.21.15.83:0/2118353453 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f74eed180 0x556f74ef7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:27.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.631+0000 7feb66000640 1 --2- 172.21.15.83:0/2560160333 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7feb500a3d60 0x7feb500a6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:27.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.631+0000 7feb6cc00640 1 --2- 172.21.15.83:0/2118353453 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f74ef7850 0x556f74ef9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:27.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.631+0000 7feb66000640 1 -- 172.21.15.83:0/2560160333 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9583+0+0 (secure 0 0 0) 0x7feb600faa20 con 0x7feb6819a880 2024-04-24T09:14:27.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.631+0000 7feb6e000640 1 --2- 172.21.15.83:0/2560160333 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7feb500a3d60 0x7feb500a6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:27.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.632+0000 7feb6d600640 1 -- 172.21.15.83:0/2118353453 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f74ef7850 msgr2=0x556f74ef9c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:14:27.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.632+0000 7feb6d600640 1 --2- 172.21.15.83:0/2118353453 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f74ef7850 0x556f74ef9c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:27.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.632+0000 7feb6d600640 1 -- 172.21.15.83:0/2118353453 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f74eec860 msgr2=0x556f74eecc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:14:27.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.632+0000 7feb6d600640 1 --2- 172.21.15.83:0/2118353453 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f74eec860 0x556f74eecc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:27.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.632+0000 7feb6d600640 1 -- 172.21.15.83:0/2118353453 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f74e4cf50 con 0x556f74eed180 2024-04-24T09:14:27.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.632+0000 7feb6d600640 1 --2- 172.21.15.83:0/2118353453 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f74eed180 0x556f74ef7310 secure :-1 s=READY pgs=2657 cs=0 l=1 rev1=1 crypto rx=0x7feb5c005fd0 tx=0x7feb5c00bbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70e2e85abe7f7758 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:14:27.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.632+0000 7feb6e000640 1 --2- 172.21.15.83:0/2560160333 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7feb500a3d60 0x7feb500a6220 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7feb600c6640 tx=0x7feb600095c0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:14:27.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.633+0000 7feb5a600640 1 -- 172.21.15.83:0/2118353453 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb5c003bd0 con 0x556f74eed180 2024-04-24T09:14:27.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.633+0000 7feb5a600640 1 -- 172.21.15.83:0/2118353453 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feb5c015070 con 0x556f74eed180 2024-04-24T09:14:27.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.633+0000 7feb5a600640 1 -- 172.21.15.83:0/2118353453 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb5c004cc0 con 0x556f74eed180 2024-04-24T09:14:27.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.633+0000 7feb72e44740 1 -- 172.21.15.83:0/2118353453 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f74eed180 msgr2=0x556f74ef7310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:27.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.633+0000 7feb72e44740 1 --2- 172.21.15.83:0/2118353453 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f74eed180 0x556f74ef7310 secure :-1 s=READY pgs=2657 cs=0 l=1 rev1=1 crypto rx=0x7feb5c005fd0 tx=0x7feb5c00bbb0 comp rx=0 tx=0).stop 2024-04-24T09:14:27.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.634+0000 7feb72e44740 1 -- 172.21.15.83:0/2118353453 shutdown_connections 2024-04-24T09:14:27.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.634+0000 7feb72e44740 1 --2- 172.21.15.83:0/2118353453 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f74ef7850 0x556f74ef9c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:27.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.634+0000 7feb72e44740 1 --2- 172.21.15.83:0/2118353453 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f74eed180 0x556f74ef7310 unknown :-1 s=CLOSED pgs=2657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:27.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.634+0000 7feb72e44740 1 --2- 172.21.15.83:0/2118353453 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f74eec860 0x556f74eecc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:27.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.634+0000 7feb72e44740 1 -- 172.21.15.83:0/2118353453 >> 172.21.15.83:0/2118353453 conn(0x556f74ee8b80 msgr2=0x556f74ee8f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:14:27.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.634+0000 7feb72e44740 1 -- 172.21.15.83:0/2118353453 shutdown_connections 2024-04-24T09:14:27.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.634+0000 7feb72e44740 1 -- 172.21.15.83:0/2118353453 wait complete. 2024-04-24T09:14:27.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.634+0000 7feb72e44740 1 -- start start 2024-04-24T09:14:27.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.635+0000 7feb72e44740 1 Processor -- start 2024-04-24T09:14:27.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.635+0000 7feb72e44740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f74eec860 0x556f74ef4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:27.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.635+0000 7feb72e44740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f74eed180 0x556f74ef4c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:27.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.635+0000 7feb72e44740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f74ef7850 0x556f74ef51c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:27.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.635+0000 7feb72e44740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556f74efa640 con 0x556f74ef7850 2024-04-24T09:14:27.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.635+0000 7feb72e44740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556f74e0b690 con 0x556f74eed180 2024-04-24T09:14:27.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.635+0000 7feb72e44740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x556f74efa7c0 con 0x556f74eec860 2024-04-24T09:14:27.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.635+0000 7feb72e44740 10 client.0 _ll_register_callbacks cb 0x556f74e47650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:14:27.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.636+0000 7feb6cc00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f74eec860 0x556f74ef4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:27.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.637+0000 7feb6d600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f74eed180 0x556f74ef4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:27.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.637+0000 7feb6d600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f74eed180 0x556f74ef4c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:53024/0 (socket says 172.21.15.83:53024) 2024-04-24T09:14:27.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.637+0000 7feb6d600640 1 -- 172.21.15.83:0/2765891919 learned_addr learned my addr 172.21.15.83:0/2765891919 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:14:27.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.637+0000 7feb6cc00640 1 --2- 172.21.15.83:0/2765891919 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f74ef7850 0x556f74ef51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:27.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.638+0000 7feb6d600640 1 -- 172.21.15.83:0/2765891919 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f74eec860 msgr2=0x556f74ef4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:14:27.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.638+0000 7feb6d600640 1 --2- 172.21.15.83:0/2765891919 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f74eec860 0x556f74ef4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:27.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.638+0000 7feb6d600640 1 -- 172.21.15.83:0/2765891919 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f74ef7850 msgr2=0x556f74ef51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:14:27.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.638+0000 7feb6d600640 1 --2- 172.21.15.83:0/2765891919 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f74ef7850 0x556f74ef51c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:27.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.638+0000 7feb6d600640 1 -- 172.21.15.83:0/2765891919 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f74e0a920 con 0x556f74eed180 2024-04-24T09:14:27.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.638+0000 7feb6cc00640 1 --2- 172.21.15.83:0/2765891919 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f74ef7850 0x556f74ef51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:14:27.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.639+0000 7feb6d600640 1 --2- 172.21.15.83:0/2765891919 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f74eed180 0x556f74ef4c80 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7feb5c000c00 tx=0x7feb5c005760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=700dd672f8be916d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:14:27.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.639+0000 7feb4ea00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:14:27.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.639+0000 7feb4ea00640 1 -- 172.21.15.83:0/2765891919 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb5c0041e0 con 0x556f74eed180 2024-04-24T09:14:27.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.640+0000 7feb4ea00640 1 -- 172.21.15.83:0/2765891919 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(128..128 src has 1..128) ==== 9583+0+0 (secure 0 0 0) 0x7feb5c003de0 con 0x556f74eed180 2024-04-24T09:14:27.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.640+0000 7feb72e44740 1 -- 172.21.15.83:0/2765891919 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb5c015070 con 0x556f74eed180 2024-04-24T09:14:27.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.640+0000 7feb4ea00640 1 -- 172.21.15.83:0/2765891919 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7feb5c011440 con 0x556f74eed180 2024-04-24T09:14:27.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.641+0000 7feb72e44740 10 client.40361 fetch_fsmap learned FSMap version 65 2024-04-24T09:14:27.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.641+0000 7feb72e44740 1 -- 172.21.15.83:0/2765891919 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x556f74c2e7d0 con 0x556f74eed180 2024-04-24T09:14:27.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.642+0000 7feb4ea00640 1 -- 172.21.15.83:0/2765891919 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feb5c015070 con 0x556f74eed180 2024-04-24T09:14:27.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.642+0000 7feb4ea00640 1 -- 172.21.15.83:0/2765891919 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7feb5c025820 con 0x556f74eed180 2024-04-24T09:14:27.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.643+0000 7feb72e44740 10 client.40361 fetch_fsmap finished waiting for FSMap version 65 2024-04-24T09:14:27.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.643+0000 7feb72e44740 10 client.40361 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:14:27.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.643+0000 7feb72e44740 20 client.40361 populate_metadata read hostname 'smithi083' 2024-04-24T09:14:27.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.643+0000 7feb72e44740 1 --2- 172.21.15.83:0/2765891919 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556f74f07a50 0x556f74f09e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:27.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.643+0000 7feb72e44740 4 client.40361 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:14:27.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.643+0000 7feb72e44740 1 -- 172.21.15.83:0/2765891919 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556f74e5fc10 con 0x556f74f07a50 2024-04-24T09:14:27.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.644+0000 7feb6cc00640 1 --2- 172.21.15.83:0/2765891919 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556f74f07a50 0x556f74f09e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:27.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.645+0000 7feb6cc00640 1 --2- 172.21.15.83:0/2765891919 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556f74f07a50 0x556f74f09e40 crc :-1 s=READY pgs=559 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1260ca18d7a0444 server_cookie=572e044c3f81c3a3 in_seq=0 out_seq=0 2024-04-24T09:14:27.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.645+0000 7feb4ea00640 10 client.40361 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:14:27.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.646+0000 7feb4ea00640 1 -- 172.21.15.83:0/2765891919 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556f74e5fc10 con 0x556f74f07a50 2024-04-24T09:14:27.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.646+0000 7feb4ea00640 10 client.40361 handle_command_reply: tid=0 2024-04-24T09:14:27.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.646+0000 7feb72e44740 1 client.40361 shutdown 2024-04-24T09:14:27.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.647+0000 7feb72e44740 1 -- 172.21.15.83:0/2765891919 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f74eed180 msgr2=0x556f74ef4c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:27.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.647+0000 7feb72e44740 1 --2- 172.21.15.83:0/2765891919 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f74eed180 0x556f74ef4c80 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7feb5c000c00 tx=0x7feb5c005760 comp rx=0 tx=0).stop 2024-04-24T09:14:27.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.647+0000 7feb72e44740 1 -- 172.21.15.83:0/2765891919 shutdown_connections 2024-04-24T09:14:27.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.647+0000 7feb72e44740 1 --2- 172.21.15.83:0/2765891919 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556f74f07a50 0x556f74f09e40 crc :-1 s=READY pgs=559 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:27.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.647+0000 7feb4ea00640 0 client.40361 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:14:27.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.647+0000 7feb72e44740 1 --2- 172.21.15.83:0/2765891919 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f74ef7850 0x556f74ef51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:27.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.647+0000 7feb72e44740 1 --2- 172.21.15.83:0/2765891919 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f74eed180 0x556f74ef4c80 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:27.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.647+0000 7feb72e44740 1 --2- 172.21.15.83:0/2765891919 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f74eec860 0x556f74ef4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:27.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.647+0000 7feb72e44740 1 -- 172.21.15.83:0/2765891919 >> 172.21.15.83:0/2765891919 conn(0x556f74ee8b80 msgr2=0x556f74eee880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:14:27.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.647+0000 7feb72e44740 1 -- 172.21.15.83:0/2765891919 shutdown_connections 2024-04-24T09:14:27.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.648+0000 7feb72e44740 1 -- 172.21.15.83:0/2765891919 wait complete. 2024-04-24T09:14:27.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.648+0000 7feb72e44740 20 client.40361 trim_cache size 0 max 16384 2024-04-24T09:14:27.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.682+0000 7feb72e44740 1 Processor -- start 2024-04-24T09:14:27.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.682+0000 7feb72e44740 1 -- start start 2024-04-24T09:14:27.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.683+0000 7feb72e44740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f74eec860 0x556f74f03530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:27.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.683+0000 7feb72e44740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f74eed180 0x556f74eeeb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:27.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.683+0000 7feb72e44740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f74ef7850 0x556f74ef6680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:27.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.683+0000 7feb72e44740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7feb681ad480 con 0x556f74ef7850 2024-04-24T09:14:27.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.683+0000 7feb72e44740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7feb681af020 con 0x556f74eec860 2024-04-24T09:14:27.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.683+0000 7feb72e44740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7feb681adb50 con 0x556f74eed180 2024-04-24T09:14:27.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.683+0000 7feb6cc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f74ef7850 0x556f74ef6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:27.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.683+0000 7feb6cc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f74ef7850 0x556f74ef6680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51580/0 (socket says 172.21.15.83:51580) 2024-04-24T09:14:27.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.683+0000 7feb6cc00640 1 -- 172.21.15.83:0/2890121678 learned_addr learned my addr 172.21.15.83:0/2890121678 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:14:27.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.683+0000 7feb6cc00640 1 --2- 172.21.15.83:0/2890121678 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f74eec860 0x556f74f03530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:27.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.683+0000 7feb6d600640 1 --2- 172.21.15.83:0/2890121678 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f74eed180 0x556f74eeeb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:27.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.684+0000 7feb6cc00640 1 -- 172.21.15.83:0/2890121678 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f74eed180 msgr2=0x556f74eeeb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:14:27.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.684+0000 7feb6cc00640 1 --2- 172.21.15.83:0/2890121678 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f74eed180 0x556f74eeeb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:27.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.684+0000 7feb6cc00640 1 -- 172.21.15.83:0/2890121678 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f74eec860 msgr2=0x556f74f03530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:14:27.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.684+0000 7feb6cc00640 1 --2- 172.21.15.83:0/2890121678 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f74eec860 0x556f74f03530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:27.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.684+0000 7feb6cc00640 1 -- 172.21.15.83:0/2890121678 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f74b643f0 con 0x556f74ef7850 2024-04-24T09:14:27.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.684+0000 7feb6d600640 1 --2- 172.21.15.83:0/2890121678 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f74eed180 0x556f74eeeb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:14:27.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.685+0000 7feb6cc00640 1 --2- 172.21.15.83:0/2890121678 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f74ef7850 0x556f74ef6680 secure :-1 s=READY pgs=2658 cs=0 l=1 rev1=1 crypto rx=0x7feb54017700 tx=0x7feb540094d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9ccfaedf9ab0e29 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:14:27.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.685+0000 7feb59c00640 1 -- 172.21.15.83:0/2890121678 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb54017dc0 con 0x556f74ef7850 2024-04-24T09:14:27.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.685+0000 7feb59c00640 1 -- 172.21.15.83:0/2890121678 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x556f74e5fc10 con 0x556f74ef7850 2024-04-24T09:14:27.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.685+0000 7feb59c00640 1 -- 172.21.15.83:0/2890121678 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb54023770 con 0x556f74ef7850 2024-04-24T09:14:27.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.686+0000 7feb72e44740 1 -- 172.21.15.83:0/2890121678 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f74ef7850 msgr2=0x556f74ef6680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:27.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.686+0000 7feb72e44740 1 --2- 172.21.15.83:0/2890121678 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f74ef7850 0x556f74ef6680 secure :-1 s=READY pgs=2658 cs=0 l=1 rev1=1 crypto rx=0x7feb54017700 tx=0x7feb540094d0 comp rx=0 tx=0).stop 2024-04-24T09:14:27.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.686+0000 7feb72e44740 1 -- 172.21.15.83:0/2890121678 shutdown_connections 2024-04-24T09:14:27.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.686+0000 7feb72e44740 1 --2- 172.21.15.83:0/2890121678 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f74ef7850 0x556f74ef6680 unknown :-1 s=CLOSED pgs=2658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:27.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.686+0000 7feb72e44740 1 --2- 172.21.15.83:0/2890121678 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f74eed180 0x556f74eeeb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:27.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.686+0000 7feb72e44740 1 --2- 172.21.15.83:0/2890121678 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f74eec860 0x556f74f03530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:27.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.686+0000 7feb72e44740 1 -- 172.21.15.83:0/2890121678 >> 172.21.15.83:0/2890121678 conn(0x556f74ee8b80 msgr2=0x556f74ef5f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:14:27.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.687+0000 7feb72e44740 1 -- 172.21.15.83:0/2890121678 shutdown_connections 2024-04-24T09:14:27.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.687+0000 7feb72e44740 1 -- 172.21.15.83:0/2890121678 wait complete. 2024-04-24T09:14:27.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.687+0000 7feb72e44740 1 -- start start 2024-04-24T09:14:27.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.688+0000 7feb72e44740 1 Processor -- start 2024-04-24T09:14:27.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.689+0000 7feb72e44740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f74eec860 0x556f74ef1470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:27.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.689+0000 7feb72e44740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f74eed180 0x556f74ef19b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:27.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.689+0000 7feb72e44740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f74ef7850 0x556f74ef1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:27.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.690+0000 7feb72e44740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7feb681ae920 con 0x556f74eec860 2024-04-24T09:14:27.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.690+0000 7feb72e44740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7feb681ae250 con 0x556f74ef7850 2024-04-24T09:14:27.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.690+0000 7feb72e44740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7feb681ab480 con 0x556f74eed180 2024-04-24T09:14:27.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.690+0000 7feb6d600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f74eed180 0x556f74ef19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:27.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.690+0000 7feb6d600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f74eed180 0x556f74ef19b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:57270/0 (socket says 172.21.15.83:57270) 2024-04-24T09:14:27.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.690+0000 7feb6d600640 1 -- 172.21.15.83:0/3003906185 learned_addr learned my addr 172.21.15.83:0/3003906185 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:14:27.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.690+0000 7feb6cc00640 1 --2- 172.21.15.83:0/3003906185 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f74eec860 0x556f74ef1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:27.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.690+0000 7feb72e44740 10 client.0 _ll_register_callbacks cb 0x7feb5c004420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:14:27.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.691+0000 7feb6d600640 1 -- 172.21.15.83:0/3003906185 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f74ef7850 msgr2=0x556f74ef1ef0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T09:14:27.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.691+0000 7feb6d600640 1 --2- 172.21.15.83:0/3003906185 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f74ef7850 0x556f74ef1ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:27.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.691+0000 7feb6d600640 1 -- 172.21.15.83:0/3003906185 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f74eec860 msgr2=0x556f74ef1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:14:27.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.691+0000 7feb6d600640 1 --2- 172.21.15.83:0/3003906185 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f74eec860 0x556f74ef1470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:27.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.691+0000 7feb6d600640 1 -- 172.21.15.83:0/3003906185 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f74b240f0 con 0x556f74eed180 2024-04-24T09:14:27.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.691+0000 7feb6cc00640 1 --2- 172.21.15.83:0/3003906185 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f74eec860 0x556f74ef1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:14:27.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.691+0000 7feb6d600640 1 --2- 172.21.15.83:0/3003906185 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f74eed180 0x556f74ef19b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7feb5c00bbf0 tx=0x7feb5c026ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb9a1e22b7fc008 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:14:27.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.692+0000 7feb4ea00640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-04-24T09:14:27.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.692+0000 7feb4ea00640 1 -- 172.21.15.83:0/3003906185 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb5c003e60 con 0x556f74eed180 2024-04-24T09:14:27.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.693+0000 7feb4ea00640 1 -- 172.21.15.83:0/3003906185 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(128..128 src has 1..128) ==== 9583+0+0 (secure 0 0 0) 0x7feb5c034040 con 0x556f74eed180 2024-04-24T09:14:27.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.693+0000 7feb72e44740 1 -- 172.21.15.83:0/3003906185 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556f74e5fc10 con 0x556f74eed180 2024-04-24T09:14:27.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.693+0000 7feb4ea00640 1 -- 172.21.15.83:0/3003906185 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7feb5c005250 con 0x556f74eed180 2024-04-24T09:14:27.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.693+0000 7feb72e44740 10 client.30121 fetch_fsmap learned FSMap version 65 2024-04-24T09:14:27.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.693+0000 7feb72e44740 1 -- 172.21.15.83:0/3003906185 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x556f74e17810 con 0x556f74eed180 2024-04-24T09:14:27.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.694+0000 7feb4ea00640 1 -- 172.21.15.83:0/3003906185 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x556f74e5fc10 con 0x556f74eed180 2024-04-24T09:14:27.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.694+0000 7feb4ea00640 1 -- 172.21.15.83:0/3003906185 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7feb5c004780 con 0x556f74eed180 2024-04-24T09:14:27.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.694+0000 7feb72e44740 10 client.30121 fetch_fsmap finished waiting for FSMap version 65 2024-04-24T09:14:27.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.694+0000 7feb72e44740 10 client.30121 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:14:27.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.694+0000 7feb72e44740 20 client.30121 populate_metadata read hostname 'smithi083' 2024-04-24T09:14:27.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.694+0000 7feb72e44740 1 --2- 172.21.15.83:0/3003906185 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556f74f07a50 0x556f74f70700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:27.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.694+0000 7feb72e44740 4 client.30121 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T09:14:27.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.694+0000 7feb72e44740 1 -- 172.21.15.83:0/3003906185 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556f74e59a80 con 0x556f74f07a50 2024-04-24T09:14:27.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.695+0000 7feb6cc00640 1 --2- 172.21.15.83:0/3003906185 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556f74f07a50 0x556f74f70700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:27.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.696+0000 7feb6cc00640 1 --2- 172.21.15.83:0/3003906185 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556f74f07a50 0x556f74f70700 crc :-1 s=READY pgs=560 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4b00fc3d2f9a7c8 server_cookie=16e3814908ec133b in_seq=0 out_seq=0 2024-04-24T09:14:27.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.696+0000 7feb4ea00640 10 client.30121 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:14:27.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.696+0000 7feb4ea00640 1 -- 172.21.15.83:0/3003906185 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x556f74e59a80 con 0x556f74f07a50 2024-04-24T09:14:27.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.696+0000 7feb4ea00640 10 client.30121 handle_command_reply: tid=0 2024-04-24T09:14:27.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.696+0000 7feb72e44740 1 client.30121 shutdown 2024-04-24T09:14:27.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.696+0000 7feb72e44740 1 -- 172.21.15.83:0/3003906185 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f74eed180 msgr2=0x556f74ef19b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:27.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.696+0000 7feb72e44740 1 --2- 172.21.15.83:0/3003906185 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f74eed180 0x556f74ef19b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7feb5c00bbf0 tx=0x7feb5c026ef0 comp rx=0 tx=0).stop 2024-04-24T09:14:27.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.696+0000 7feb72e44740 1 -- 172.21.15.83:0/3003906185 shutdown_connections 2024-04-24T09:14:27.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.696+0000 7feb72e44740 1 --2- 172.21.15.83:0/3003906185 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556f74f07a50 0x556f74f70700 crc :-1 s=READY pgs=560 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:27.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.696+0000 7feb72e44740 1 --2- 172.21.15.83:0/3003906185 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f74ef7850 0x556f74ef1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:27.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.696+0000 7feb72e44740 1 --2- 172.21.15.83:0/3003906185 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556f74eed180 0x556f74ef19b0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:27.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.696+0000 7feb72e44740 1 --2- 172.21.15.83:0/3003906185 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f74eec860 0x556f74ef1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:27.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.696+0000 7feb72e44740 1 -- 172.21.15.83:0/3003906185 >> 172.21.15.83:0/3003906185 conn(0x556f74ee8b80 msgr2=0x556f74ef6800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:14:27.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.697+0000 7feb4ea00640 0 client.30121 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:14:27.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.697+0000 7feb72e44740 1 -- 172.21.15.83:0/3003906185 shutdown_connections 2024-04-24T09:14:27.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.698+0000 7feb72e44740 1 -- 172.21.15.83:0/3003906185 wait complete. 2024-04-24T09:14:27.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.698+0000 7feb72e44740 20 client.30121 trim_cache size 0 max 16384 2024-04-24T09:14:27.698 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T09:14:27.698 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-04-24T09:14:27.698 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-04-24T09:14:27.698 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T09:14:27.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.699+0000 7feb70000640 1 -- 172.21.15.83:0/2560160333 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7feb500a3d60 msgr2=0x7feb500a6220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:27.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.699+0000 7feb70000640 1 --2- 172.21.15.83:0/2560160333 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7feb500a3d60 0x7feb500a6220 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7feb600c6640 tx=0x7feb600095c0 comp rx=0 tx=0).stop 2024-04-24T09:14:27.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.699+0000 7feb70000640 1 -- 172.21.15.83:0/2560160333 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb6819a880 msgr2=0x7feb6819ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:27.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.699+0000 7feb70000640 1 --2- 172.21.15.83:0/2560160333 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb6819a880 0x7feb6819ad30 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7feb60009bf0 tx=0x7feb60030d10 comp rx=0 tx=0).stop 2024-04-24T09:14:27.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.699+0000 7feb70000640 1 -- 172.21.15.83:0/2560160333 shutdown_connections 2024-04-24T09:14:27.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.699+0000 7feb70000640 1 --2- 172.21.15.83:0/2560160333 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7feb500a3d60 0x7feb500a6220 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:27.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.699+0000 7feb70000640 1 --2- 172.21.15.83:0/2560160333 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb6819a880 0x7feb6819ad30 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:27.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.699+0000 7feb70000640 1 --2- 172.21.15.83:0/2560160333 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb68109520 0x7feb681a0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:27.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.699+0000 7feb70000640 1 --2- 172.21.15.83:0/2560160333 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feb68104450 0x7feb681a06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:27.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.699+0000 7feb70000640 1 -- 172.21.15.83:0/2560160333 >> 172.21.15.83:0/2560160333 conn(0x7feb680ffc70 msgr2=0x7feb68101050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:14:27.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.700+0000 7feb70000640 1 -- 172.21.15.83:0/2560160333 shutdown_connections 2024-04-24T09:14:27.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.700+0000 7feb70000640 1 -- 172.21.15.83:0/2560160333 wait complete. 2024-04-24T09:14:27.722 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T09:14:27.723 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T09:14:27.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.980+0000 7f62afe00640 1 -- 172.21.15.83:0/2092224525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62b01071a0 msgr2=0x7f62b010ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:27.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.980+0000 7f62afe00640 1 --2- 172.21.15.83:0/2092224525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62b01071a0 0x7f62b010ed40 secure :-1 s=READY pgs=2659 cs=0 l=1 rev1=1 crypto rx=0x7f62a8009ca0 tx=0x7f62a802a8c0 comp rx=0 tx=0).stop 2024-04-24T09:14:27.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.980+0000 7f62afe00640 1 -- 172.21.15.83:0/2092224525 shutdown_connections 2024-04-24T09:14:27.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.980+0000 7f62afe00640 1 --2- 172.21.15.83:0/2092224525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62b01071a0 0x7f62b010ed40 unknown :-1 s=CLOSED pgs=2659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:27.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.980+0000 7f62afe00640 1 --2- 172.21.15.83:0/2092224525 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f62b0069db0 0x7f62b0106a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:27.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.980+0000 7f62afe00640 1 --2- 172.21.15.83:0/2092224525 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f62b0069400 0x7f62b00697e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:27.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.980+0000 7f62afe00640 1 -- 172.21.15.83:0/2092224525 >> 172.21.15.83:0/2092224525 conn(0x7f62b0076840 msgr2=0x7f62b0076c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:14:27.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.980+0000 7f62afe00640 1 -- 172.21.15.83:0/2092224525 shutdown_connections 2024-04-24T09:14:27.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.980+0000 7f62afe00640 1 -- 172.21.15.83:0/2092224525 wait complete. 2024-04-24T09:14:27.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.982+0000 7f62afe00640 1 Processor -- start 2024-04-24T09:14:27.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.982+0000 7f62afe00640 1 -- start start 2024-04-24T09:14:27.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.982+0000 7f62afe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f62b0069400 0x7f62b010c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:27.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.982+0000 7f62afe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62b01071a0 0x7f62b010c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:27.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.982+0000 7f62afe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f62b010caf0 0x7f62b01aad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:27.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.982+0000 7f62afe00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f62b0102d10 con 0x7f62b01071a0 2024-04-24T09:14:27.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.982+0000 7f62afe00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f62b0102b90 con 0x7f62b010caf0 2024-04-24T09:14:27.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.982+0000 7f62afe00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f62b0102e90 con 0x7f62b0069400 2024-04-24T09:14:27.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.983+0000 7f62ae000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62b01071a0 0x7f62b010c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:27.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.983+0000 7f62aea00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f62b0069400 0x7f62b010c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:27.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.983+0000 7f62aea00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f62b0069400 0x7f62b010c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:57288/0 (socket says 172.21.15.83:57288) 2024-04-24T09:14:27.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.983+0000 7f62aea00640 1 -- 172.21.15.83:0/1656776448 learned_addr learned my addr 172.21.15.83:0/1656776448 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:14:27.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.983+0000 7f62af400640 1 --2- 172.21.15.83:0/1656776448 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f62b010caf0 0x7f62b01aad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:27.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.983+0000 7f62ae000640 1 -- 172.21.15.83:0/1656776448 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f62b0069400 msgr2=0x7f62b010c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:27.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.983+0000 7f62ae000640 1 --2- 172.21.15.83:0/1656776448 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f62b0069400 0x7f62b010c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:27.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.983+0000 7f62ae000640 1 -- 172.21.15.83:0/1656776448 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f62b010caf0 msgr2=0x7f62b01aad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:27.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.983+0000 7f62ae000640 1 --2- 172.21.15.83:0/1656776448 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f62b010caf0 0x7f62b01aad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:27.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.983+0000 7f62ae000640 1 -- 172.21.15.83:0/1656776448 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62b01ab330 con 0x7f62b01071a0 2024-04-24T09:14:27.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.983+0000 7f62af400640 1 --2- 172.21.15.83:0/1656776448 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f62b010caf0 0x7f62b01aad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:14:27.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.984+0000 7f62ae000640 1 --2- 172.21.15.83:0/1656776448 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62b01071a0 0x7f62b010c5b0 secure :-1 s=READY pgs=2660 cs=0 l=1 rev1=1 crypto rx=0x7f629c00e9c0 tx=0x7f629c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:14:27.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.984+0000 7f62a7400640 1 -- 172.21.15.83:0/1656776448 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f629c00cd80 con 0x7f62b01071a0 2024-04-24T09:14:27.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.984+0000 7f62a7400640 1 -- 172.21.15.83:0/1656776448 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f629c0035f0 con 0x7f62b01071a0 2024-04-24T09:14:27.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.984+0000 7f62afe00640 1 -- 172.21.15.83:0/1656776448 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62b01ab620 con 0x7f62b01071a0 2024-04-24T09:14:27.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.984+0000 7f62a7400640 1 -- 172.21.15.83:0/1656776448 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f629c010690 con 0x7f62b01071a0 2024-04-24T09:14:27.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.986+0000 7f62afe00640 1 -- 172.21.15.83:0/1656776448 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62b01abbc0 con 0x7f62b01071a0 2024-04-24T09:14:27.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.990+0000 7f62afe00640 1 -- 172.21.15.83:0/1656776448 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62b0103550 con 0x7f62b01071a0 2024-04-24T09:14:27.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.990+0000 7f62a7400640 1 -- 172.21.15.83:0/1656776448 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f629c0037a0 con 0x7f62b01071a0 2024-04-24T09:14:27.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.993+0000 7f62a7400640 1 --2- 172.21.15.83:0/1656776448 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f62940a3de0 0x7f62940a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:27.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.993+0000 7f62a7400640 1 -- 172.21.15.83:0/1656776448 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9583+0+0 (secure 0 0 0) 0x7f629c0d0eb0 con 0x7f62b01071a0 2024-04-24T09:14:27.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.994+0000 7f62aea00640 1 --2- 172.21.15.83:0/1656776448 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f62940a3de0 0x7f62940a62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:27.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:27.995+0000 7f62aea00640 1 --2- 172.21.15.83:0/1656776448 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f62940a3de0 0x7f62940a62a0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f62a0009c90 tx=0x7f62a0005f90 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:14:28.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.003+0000 7f62a7400640 1 -- 172.21.15.83:0/1656776448 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f629c09d130 con 0x7f62b01071a0 2024-04-24T09:14:28.205 INFO:tasks.workunit.client.0.smithi083.stdout: 10 66615 6.57 MB/sec execute 419 sec latency 125.963 ms 2024-04-24T09:14:28.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.422+0000 7f62afe00640 1 -- 172.21.15.83:0/1656776448 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f62b0068d40 con 0x7f62b01071a0 2024-04-24T09:14:28.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.423+0000 7f62a7400640 1 -- 172.21.15.83:0/1656776448 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) ==== 94+0+11946 (secure 0 0 0) 0x7f629c09ca30 con 0x7f62b01071a0 2024-04-24T09:14:28.423 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T09:14:28.423 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":65,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/358872257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":358872257},{"type":"v1","addr":"172.21.15.83:6835","nonce":358872257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3157070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3157070851},{"type":"v1","addr":"172.21.15.83:6837","nonce":3157070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":65,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T08:48:29.422576+0000","modified":"2024-04-24T09:14:23.179979+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34520,"mds_1":34493,"mds_2":34451,"mds_3":34427,"mds_4":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14658":{"gid":14658,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6839/4085470844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":4085470844},{"type":"v1","addr":"172.21.15.83:6839","nonce":4085470844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14676":{"gid":14676,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6841/2634754210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":2634754210},{"type":"v1","addr":"172.21.15.83:6841","nonce":2634754210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24382":{"gid":24382,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/1859921310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1859921310},{"type":"v1","addr":"172.21.15.189:6833","nonce":1859921310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/2241923197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2241923197},{"type":"v1","addr":"172.21.15.189:6835","nonce":2241923197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.189:6839/2548965336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":2548965336},{"type":"v1","addr":"172.21.15.189:6839","nonce":2548965336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26686":{"gid":26686,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6837/1678817161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":1678817161},{"type":"v1","addr":"172.21.15.189:6837","nonce":1678817161}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.138:6833/1157967658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1157967658},{"type":"v1","addr":"172.21.15.138:6833","nonce":1157967658}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.138:6835/1909219758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1909219758},{"type":"v1","addr":"172.21.15.138:6835","nonce":1909219758}]},"join_fscid":-1,"export_targets":[4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34493":{"gid":34493,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.138:6837/900566754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":900566754},{"type":"v1","addr":"172.21.15.138:6837","nonce":900566754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34520":{"gid":34520,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.138:6839/2236932393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2236932393},{"type":"v1","addr":"172.21.15.138:6839","nonce":2236932393}]},"join_fscid":-1,"export_targets":[4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34520,"qdb_cluster":[24454,34520,34427,34451,34493]},"id":1}]} 2024-04-24T09:14:28.423 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 65 2024-04-24T09:14:28.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.426+0000 7f62afe00640 1 -- 172.21.15.83:0/1656776448 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f62940a3de0 msgr2=0x7f62940a62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:28.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.426+0000 7f62afe00640 1 --2- 172.21.15.83:0/1656776448 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f62940a3de0 0x7f62940a62a0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f62a0009c90 tx=0x7f62a0005f90 comp rx=0 tx=0).stop 2024-04-24T09:14:28.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.426+0000 7f62afe00640 1 -- 172.21.15.83:0/1656776448 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62b01071a0 msgr2=0x7f62b010c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:28.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.426+0000 7f62afe00640 1 --2- 172.21.15.83:0/1656776448 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62b01071a0 0x7f62b010c5b0 secure :-1 s=READY pgs=2660 cs=0 l=1 rev1=1 crypto rx=0x7f629c00e9c0 tx=0x7f629c00ee80 comp rx=0 tx=0).stop 2024-04-24T09:14:28.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.427+0000 7f62afe00640 1 -- 172.21.15.83:0/1656776448 shutdown_connections 2024-04-24T09:14:28.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.427+0000 7f62afe00640 1 --2- 172.21.15.83:0/1656776448 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f62940a3de0 0x7f62940a62a0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:28.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.427+0000 7f62afe00640 1 --2- 172.21.15.83:0/1656776448 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f62b010caf0 0x7f62b01aad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:28.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.427+0000 7f62afe00640 1 --2- 172.21.15.83:0/1656776448 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62b01071a0 0x7f62b010c5b0 unknown :-1 s=CLOSED pgs=2660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:28.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.427+0000 7f62afe00640 1 --2- 172.21.15.83:0/1656776448 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f62b0069400 0x7f62b010c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:28.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.427+0000 7f62afe00640 1 -- 172.21.15.83:0/1656776448 >> 172.21.15.83:0/1656776448 conn(0x7f62b0076840 msgr2=0x7f62b00ff350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:14:28.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.427+0000 7f62afe00640 1 -- 172.21.15.83:0/1656776448 shutdown_connections 2024-04-24T09:14:28.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.427+0000 7f62afe00640 1 -- 172.21.15.83:0/1656776448 wait complete. 2024-04-24T09:14:28.450 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-24T09:14:28.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:14:28 smithi138 ceph-mon[31441]: pgmap v722: 129 pgs: 129 active+clean; 1008 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.9 MiB/s wr, 257 op/s 2024-04-24T09:14:28.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:14:28 smithi189 ceph-mon[31791]: pgmap v722: 129 pgs: 129 active+clean; 1008 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.9 MiB/s wr, 257 op/s 2024-04-24T09:14:28.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.723+0000 7f1201800640 1 -- 172.21.15.83:0/1039076286 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11fc1019b0 msgr2=0x7f11fc1059f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:28.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.723+0000 7f1201800640 1 --2- 172.21.15.83:0/1039076286 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11fc1019b0 0x7f11fc1059f0 secure :-1 s=READY pgs=2661 cs=0 l=1 rev1=1 crypto rx=0x7f11e8009900 tx=0x7f11e802a3f0 comp rx=0 tx=0).stop 2024-04-24T09:14:28.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.723+0000 7f1201800640 1 -- 172.21.15.83:0/1039076286 shutdown_connections 2024-04-24T09:14:28.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.723+0000 7f1201800640 1 --2- 172.21.15.83:0/1039076286 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11fc1060d0 0x7f11fc069190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:28.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.723+0000 7f1201800640 1 --2- 172.21.15.83:0/1039076286 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11fc1019b0 0x7f11fc1059f0 unknown :-1 s=CLOSED pgs=2661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:28.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.723+0000 7f1201800640 1 --2- 172.21.15.83:0/1039076286 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f11fc101000 0x7f11fc1013e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:28.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.723+0000 7f1201800640 1 -- 172.21.15.83:0/1039076286 >> 172.21.15.83:0/1039076286 conn(0x7f11fc0fc880 msgr2=0x7f11fc0feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:14:28.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.723+0000 7f1201800640 1 -- 172.21.15.83:0/1039076286 shutdown_connections 2024-04-24T09:14:28.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.723+0000 7f1201800640 1 -- 172.21.15.83:0/1039076286 wait complete. 2024-04-24T09:14:28.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.724+0000 7f1201800640 1 Processor -- start 2024-04-24T09:14:28.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.724+0000 7f1201800640 1 -- start start 2024-04-24T09:14:28.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.725+0000 7f1201800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11fc101000 0x7f11fc113ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:28.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.725+0000 7f1201800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f11fc1060d0 0x7f11fc110dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:28.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.725+0000 7f1201800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11fc111310 0x7f11fc1117a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:28.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.725+0000 7f1201800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f11fc114440 con 0x7f11fc101000 2024-04-24T09:14:28.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.725+0000 7f1201800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f11fc1142c0 con 0x7f11fc111310 2024-04-24T09:14:28.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.725+0000 7f1201800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f11fc1145c0 con 0x7f11fc1060d0 2024-04-24T09:14:28.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.725+0000 7f11fbe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11fc101000 0x7f11fc113ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:28.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.725+0000 7f1200e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11fc111310 0x7f11fc1117a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:28.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.725+0000 7f11fbe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11fc101000 0x7f11fc113ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51622/0 (socket says 172.21.15.83:51622) 2024-04-24T09:14:28.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.725+0000 7f1200e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11fc111310 0x7f11fc1117a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:53072/0 (socket says 172.21.15.83:53072) 2024-04-24T09:14:28.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.725+0000 7f11fbe00640 1 -- 172.21.15.83:0/2769768551 learned_addr learned my addr 172.21.15.83:0/2769768551 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:14:28.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.726+0000 7f11fb400640 1 --2- 172.21.15.83:0/2769768551 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f11fc1060d0 0x7f11fc110dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:28.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.726+0000 7f1200e00640 1 -- 172.21.15.83:0/2769768551 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f11fc1060d0 msgr2=0x7f11fc110dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:28.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.726+0000 7f1200e00640 1 --2- 172.21.15.83:0/2769768551 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f11fc1060d0 0x7f11fc110dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:28.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.726+0000 7f1200e00640 1 -- 172.21.15.83:0/2769768551 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11fc101000 msgr2=0x7f11fc113ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:28.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.726+0000 7f1200e00640 1 --2- 172.21.15.83:0/2769768551 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11fc101000 0x7f11fc113ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:28.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.726+0000 7f1200e00640 1 -- 172.21.15.83:0/2769768551 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11fc1ab360 con 0x7f11fc111310 2024-04-24T09:14:28.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.726+0000 7f11fb400640 1 --2- 172.21.15.83:0/2769768551 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f11fc1060d0 0x7f11fc110dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:14:28.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.726+0000 7f11fbe00640 1 --2- 172.21.15.83:0/2769768551 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11fc101000 0x7f11fc113ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:14:28.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.727+0000 7f1200e00640 1 --2- 172.21.15.83:0/2769768551 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11fc111310 0x7f11fc1117a0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f11f400ca30 tx=0x7f11f400cef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:14:28.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.727+0000 7f11f8c00640 1 -- 172.21.15.83:0/2769768551 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f4007d90 con 0x7f11fc111310 2024-04-24T09:14:28.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.727+0000 7f1201800640 1 -- 172.21.15.83:0/2769768551 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11fc1ab650 con 0x7f11fc111310 2024-04-24T09:14:28.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.727+0000 7f11f8c00640 1 -- 172.21.15.83:0/2769768551 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f11f4005020 con 0x7f11fc111310 2024-04-24T09:14:28.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.727+0000 7f1201800640 1 -- 172.21.15.83:0/2769768551 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11fc1abb70 con 0x7f11fc111310 2024-04-24T09:14:28.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.727+0000 7f11f8c00640 1 -- 172.21.15.83:0/2769768551 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f401d060 con 0x7f11fc111310 2024-04-24T09:14:28.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.730+0000 7f1205c0f740 1 Processor -- start 2024-04-24T09:14:28.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.731+0000 7f11f8c00640 1 -- 172.21.15.83:0/2769768551 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f11f400f690 con 0x7f11fc111310 2024-04-24T09:14:28.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.731+0000 7f1205c0f740 1 -- start start 2024-04-24T09:14:28.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.731+0000 7f1205c0f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e795225b40 0x55e795225f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:28.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.732+0000 7f1205c0f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e795226460 0x55e7952305f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:28.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.732+0000 7f1205c0f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e795230b30 0x55e795232f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:28.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.732+0000 7f1205c0f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e79518b1d0 con 0x55e795225b40 2024-04-24T09:14:28.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.732+0000 7f1205c0f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e795131450 con 0x55e795230b30 2024-04-24T09:14:28.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.732+0000 7f1205c0f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55e795187190 con 0x55e795226460 2024-04-24T09:14:28.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.732+0000 7f11fbe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e795226460 0x55e7952305f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:28.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.732+0000 7f11fbe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e795226460 0x55e7952305f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:57316/0 (socket says 172.21.15.83:57316) 2024-04-24T09:14:28.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.732+0000 7f11fbe00640 1 -- 172.21.15.83:0/745004180 learned_addr learned my addr 172.21.15.83:0/745004180 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:14:28.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.732+0000 7f11fb400640 1 --2- 172.21.15.83:0/745004180 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e795225b40 0x55e795225f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:28.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.732+0000 7f11fb400640 1 --2- 172.21.15.83:0/745004180 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e795230b30 0x55e795232f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:28.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.733+0000 7f11fbe00640 1 -- 172.21.15.83:0/745004180 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e795230b30 msgr2=0x55e795232f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:14:28.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.733+0000 7f11fbe00640 1 --2- 172.21.15.83:0/745004180 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e795230b30 0x55e795232f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:28.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.733+0000 7f11fbe00640 1 -- 172.21.15.83:0/745004180 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e795225b40 msgr2=0x55e795225f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:14:28.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.733+0000 7f11fbe00640 1 --2- 172.21.15.83:0/745004180 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e795225b40 0x55e795225f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:28.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.733+0000 7f11fbe00640 1 -- 172.21.15.83:0/745004180 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7951ee310 con 0x55e795226460 2024-04-24T09:14:28.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.733+0000 7f11f8c00640 1 --2- 172.21.15.83:0/2769768551 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f11e00a3e20 0x7f11e00a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:28.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.733+0000 7f11f8c00640 1 -- 172.21.15.83:0/2769768551 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9583+0+0 (secure 0 0 0) 0x7f11f40d0d30 con 0x7f11fc111310 2024-04-24T09:14:28.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.733+0000 7f11fbe00640 1 --2- 172.21.15.83:0/745004180 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e795226460 0x55e7952305f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f11ec010560 tx=0x7f11ec010a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ac41ec1b86e9799 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:14:28.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.734+0000 7f11fb400640 1 --2- 172.21.15.83:0/2769768551 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f11e00a3e20 0x7f11e00a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:28.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.734+0000 7f11f1600640 1 -- 172.21.15.83:0/745004180 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11ec00e7d0 con 0x55e795226460 2024-04-24T09:14:28.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.734+0000 7f11f1600640 1 -- 172.21.15.83:0/745004180 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f11ec00ee20 con 0x55e795226460 2024-04-24T09:14:28.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.734+0000 7f11f1600640 1 -- 172.21.15.83:0/745004180 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11ec01ce40 con 0x55e795226460 2024-04-24T09:14:28.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.735+0000 7f1205c0f740 1 -- 172.21.15.83:0/745004180 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e795226460 msgr2=0x55e7952305f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:28.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.735+0000 7f11fb400640 1 --2- 172.21.15.83:0/2769768551 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f11e00a3e20 0x7f11e00a62e0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f11e8006010 tx=0x7f11e8000bc0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:14:28.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.735+0000 7f1205c0f740 1 --2- 172.21.15.83:0/745004180 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e795226460 0x55e7952305f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f11ec010560 tx=0x7f11ec010a20 comp rx=0 tx=0).stop 2024-04-24T09:14:28.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.735+0000 7f1205c0f740 1 -- 172.21.15.83:0/745004180 shutdown_connections 2024-04-24T09:14:28.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.735+0000 7f1205c0f740 1 --2- 172.21.15.83:0/745004180 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e795230b30 0x55e795232f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:28.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.735+0000 7f1205c0f740 1 --2- 172.21.15.83:0/745004180 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e795226460 0x55e7952305f0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:28.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.735+0000 7f1205c0f740 1 --2- 172.21.15.83:0/745004180 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e795225b40 0x55e795225f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:28.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.735+0000 7f1205c0f740 1 -- 172.21.15.83:0/745004180 >> 172.21.15.83:0/745004180 conn(0x55e795221e60 msgr2=0x55e795222240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:14:28.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.735+0000 7f1205c0f740 1 -- 172.21.15.83:0/745004180 shutdown_connections 2024-04-24T09:14:28.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.735+0000 7f1205c0f740 1 -- 172.21.15.83:0/745004180 wait complete. 2024-04-24T09:14:28.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.736+0000 7f1205c0f740 1 -- start start 2024-04-24T09:14:28.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.736+0000 7f1205c0f740 1 Processor -- start 2024-04-24T09:14:28.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.737+0000 7f1205c0f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e795225b40 0x55e7952304b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:28.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.737+0000 7f1205c0f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e795226460 0x55e79522d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:28.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.737+0000 7f1205c0f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e795230b30 0x55e79522dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:28.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.737+0000 7f1205c0f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e7952449d0 con 0x55e795230b30 2024-04-24T09:14:28.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.737+0000 7f1205c0f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e795144810 con 0x55e795226460 2024-04-24T09:14:28.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.737+0000 7f1205c0f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55e795244b50 con 0x55e795225b40 2024-04-24T09:14:28.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.737+0000 7f1205c0f740 10 client.0 _ll_register_callbacks cb 0x55e795112140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:14:28.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.738+0000 7f11fb400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e795226460 0x55e79522d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:28.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.738+0000 7f11fb400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e795226460 0x55e79522d7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:53094/0 (socket says 172.21.15.83:53094) 2024-04-24T09:14:28.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.738+0000 7f11fb400640 1 -- 172.21.15.83:0/1953122019 learned_addr learned my addr 172.21.15.83:0/1953122019 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:14:28.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.738+0000 7f1200e00640 1 --2- 172.21.15.83:0/1953122019 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e795230b30 0x55e79522dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:28.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.738+0000 7f11fbe00640 1 --2- 172.21.15.83:0/1953122019 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e795225b40 0x55e7952304b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:28.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.738+0000 7f11fb400640 1 -- 172.21.15.83:0/1953122019 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e795225b40 msgr2=0x55e7952304b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:14:28.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.738+0000 7f11fb400640 1 --2- 172.21.15.83:0/1953122019 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e795225b40 0x55e7952304b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:28.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.738+0000 7f11fb400640 1 -- 172.21.15.83:0/1953122019 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e795230b30 msgr2=0x55e79522dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:14:28.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.738+0000 7f11fb400640 1 --2- 172.21.15.83:0/1953122019 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e795230b30 0x55e79522dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:28.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.738+0000 7f11fb400640 1 -- 172.21.15.83:0/1953122019 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7951553f0 con 0x55e795226460 2024-04-24T09:14:28.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.738+0000 7f11fbe00640 1 --2- 172.21.15.83:0/1953122019 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e795225b40 0x55e7952304b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:14:28.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.738+0000 7f1200e00640 1 --2- 172.21.15.83:0/1953122019 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e795230b30 0x55e79522dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:14:28.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.739+0000 7f11fb400640 1 --2- 172.21.15.83:0/1953122019 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e795226460 0x55e79522d7a0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f11e8033000 tx=0x7f11e802a3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ad47abe3c3cbcce server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:14:28.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.739+0000 7f11e6000640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:14:28.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.739+0000 7f11e6000640 1 -- 172.21.15.83:0/1953122019 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11e8035040 con 0x55e795226460 2024-04-24T09:14:28.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.739+0000 7f11e6000640 1 -- 172.21.15.83:0/1953122019 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(128..128 src has 1..128) ==== 9583+0+0 (secure 0 0 0) 0x7f11e8003a00 con 0x55e795226460 2024-04-24T09:14:28.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.739+0000 7f1205c0f740 1 -- 172.21.15.83:0/1953122019 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11ec00ee20 con 0x55e795226460 2024-04-24T09:14:28.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.740+0000 7f11e6000640 1 -- 172.21.15.83:0/1953122019 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f11e8003c00 con 0x55e795226460 2024-04-24T09:14:28.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.740+0000 7f1205c0f740 10 client.40376 fetch_fsmap learned FSMap version 65 2024-04-24T09:14:28.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.740+0000 7f1205c0f740 1 -- 172.21.15.83:0/1953122019 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x55e794e9d3f0 con 0x55e795226460 2024-04-24T09:14:28.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.741+0000 7f11e6000640 1 -- 172.21.15.83:0/1953122019 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f11ec00ee20 con 0x55e795226460 2024-04-24T09:14:28.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.741+0000 7f11e6000640 1 -- 172.21.15.83:0/1953122019 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7f11e803a040 con 0x55e795226460 2024-04-24T09:14:28.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.741+0000 7f1205c0f740 10 client.40376 fetch_fsmap finished waiting for FSMap version 65 2024-04-24T09:14:28.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.741+0000 7f1205c0f740 10 client.40376 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-24T09:14:28.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.741+0000 7f1205c0f740 20 client.40376 populate_metadata read hostname 'smithi083' 2024-04-24T09:14:28.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.741+0000 7f1205c0f740 1 --2- 172.21.15.83:0/1953122019 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55e795251c70 0x55e7952540f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:28.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.741+0000 7f1205c0f740 4 client.40376 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:14:28.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.741+0000 7f1205c0f740 1 -- 172.21.15.83:0/1953122019 --> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e795198bf0 con 0x55e795251c70 2024-04-24T09:14:28.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.742+0000 7f11fbe00640 1 --2- 172.21.15.83:0/1953122019 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55e795251c70 0x55e7952540f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:28.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.743+0000 7f11fbe00640 1 --2- 172.21.15.83:0/1953122019 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55e795251c70 0x55e7952540f0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5242b611ea75322e server_cookie=6add123d043e327e in_seq=0 out_seq=0 2024-04-24T09:14:28.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.743+0000 7f11e6000640 10 client.40376 ms_handle_connect on v2:172.21.15.189:6838/2548965336 2024-04-24T09:14:28.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.746+0000 7f11e6000640 1 -- 172.21.15.83:0/1953122019 <== mds.4 v2:172.21.15.189:6838/2548965336 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e795198bf0 con 0x55e795251c70 2024-04-24T09:14:28.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.746+0000 7f11e6000640 10 client.40376 handle_command_reply: tid=0 2024-04-24T09:14:28.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.746+0000 7f1205c0f740 1 client.40376 shutdown 2024-04-24T09:14:28.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.746+0000 7f1205c0f740 1 -- 172.21.15.83:0/1953122019 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e795226460 msgr2=0x55e79522d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:28.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.746+0000 7f1205c0f740 1 --2- 172.21.15.83:0/1953122019 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e795226460 0x55e79522d7a0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f11e8033000 tx=0x7f11e802a3f0 comp rx=0 tx=0).stop 2024-04-24T09:14:28.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.746+0000 7f1205c0f740 1 -- 172.21.15.83:0/1953122019 shutdown_connections 2024-04-24T09:14:28.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.746+0000 7f1205c0f740 1 --2- 172.21.15.83:0/1953122019 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55e795251c70 0x55e7952540f0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:28.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.746+0000 7f1205c0f740 1 --2- 172.21.15.83:0/1953122019 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e795230b30 0x55e79522dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:28.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.746+0000 7f1205c0f740 1 --2- 172.21.15.83:0/1953122019 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e795226460 0x55e79522d7a0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:28.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.746+0000 7f1205c0f740 1 --2- 172.21.15.83:0/1953122019 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e795225b40 0x55e7952304b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:28.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.746+0000 7f1205c0f740 1 -- 172.21.15.83:0/1953122019 >> 172.21.15.83:0/1953122019 conn(0x55e795221e60 msgr2=0x55e795227b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:14:28.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.746+0000 7f11e6000640 0 client.40376 ms_handle_reset on v2:172.21.15.189:6838/2548965336 2024-04-24T09:14:28.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.747+0000 7f1205c0f740 1 -- 172.21.15.83:0/1953122019 shutdown_connections 2024-04-24T09:14:28.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.747+0000 7f1205c0f740 1 -- 172.21.15.83:0/1953122019 wait complete. 2024-04-24T09:14:28.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.747+0000 7f1205c0f740 20 client.40376 trim_cache size 0 max 16384 2024-04-24T09:14:28.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.773+0000 7f1205c0f740 1 Processor -- start 2024-04-24T09:14:28.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.773+0000 7f1205c0f740 1 -- start start 2024-04-24T09:14:28.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.773+0000 7f1205c0f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e795225b40 0x55e79524d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:28.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.774+0000 7f1205c0f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e795226460 0x55e79522ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:28.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.774+0000 7f1205c0f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e795230b30 0x55e795259a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:28.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.774+0000 7f1205c0f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f11dc010100 con 0x55e795225b40 2024-04-24T09:14:28.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.774+0000 7f1205c0f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f11dc011ca0 con 0x55e795226460 2024-04-24T09:14:28.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.774+0000 7f1205c0f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f11dc0107d0 con 0x55e795230b30 2024-04-24T09:14:28.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.774+0000 7f11fbe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e795225b40 0x55e79524d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:28.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.774+0000 7f11fbe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e795225b40 0x55e79524d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51634/0 (socket says 172.21.15.83:51634) 2024-04-24T09:14:28.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.774+0000 7f11fbe00640 1 -- 172.21.15.83:0/1877403861 learned_addr learned my addr 172.21.15.83:0/1877403861 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:14:28.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.774+0000 7f1200e00640 1 --2- 172.21.15.83:0/1877403861 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e795230b30 0x55e795259a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:28.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.775+0000 7f11fbe00640 1 -- 172.21.15.83:0/1877403861 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e795230b30 msgr2=0x55e795259a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:14:28.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.775+0000 7f11fbe00640 1 --2- 172.21.15.83:0/1877403861 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e795230b30 0x55e795259a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:28.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.775+0000 7f11fbe00640 1 -- 172.21.15.83:0/1877403861 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e795226460 msgr2=0x55e79522ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:14:28.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.775+0000 7f11fbe00640 1 --2- 172.21.15.83:0/1877403861 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e795226460 0x55e79522ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:28.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.775+0000 7f11fbe00640 1 -- 172.21.15.83:0/1877403861 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e79522e090 con 0x55e795225b40 2024-04-24T09:14:28.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.776+0000 7f11fbe00640 1 --2- 172.21.15.83:0/1877403861 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e795225b40 0x55e79524d8f0 secure :-1 s=READY pgs=2662 cs=0 l=1 rev1=1 crypto rx=0x7f11ec010530 tx=0x7f11ec01c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19a6472e807dbae7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:14:28.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.776+0000 7f11f0c00640 1 -- 172.21.15.83:0/1877403861 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11ec01cad0 con 0x55e795225b40 2024-04-24T09:14:28.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.777+0000 7f11f0c00640 1 -- 172.21.15.83:0/1877403861 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f11ec028900 con 0x55e795225b40 2024-04-24T09:14:28.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.777+0000 7f11f0c00640 1 -- 172.21.15.83:0/1877403861 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11ec005610 con 0x55e795225b40 2024-04-24T09:14:28.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.777+0000 7f1205c0f740 1 -- 172.21.15.83:0/1877403861 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e795225b40 msgr2=0x55e79524d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:28.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.777+0000 7f1205c0f740 1 --2- 172.21.15.83:0/1877403861 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e795225b40 0x55e79524d8f0 secure :-1 s=READY pgs=2662 cs=0 l=1 rev1=1 crypto rx=0x7f11ec010530 tx=0x7f11ec01c9a0 comp rx=0 tx=0).stop 2024-04-24T09:14:28.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.778+0000 7f1205c0f740 1 -- 172.21.15.83:0/1877403861 shutdown_connections 2024-04-24T09:14:28.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.778+0000 7f1205c0f740 1 --2- 172.21.15.83:0/1877403861 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e795230b30 0x55e795259a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:28.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.778+0000 7f1205c0f740 1 --2- 172.21.15.83:0/1877403861 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e795226460 0x55e79522ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:28.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.778+0000 7f1205c0f740 1 --2- 172.21.15.83:0/1877403861 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e795225b40 0x55e79524d8f0 unknown :-1 s=CLOSED pgs=2662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:28.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.778+0000 7f1205c0f740 1 -- 172.21.15.83:0/1877403861 >> 172.21.15.83:0/1877403861 conn(0x55e795221e60 msgr2=0x55e795228a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:14:28.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.778+0000 7f1205c0f740 1 -- 172.21.15.83:0/1877403861 shutdown_connections 2024-04-24T09:14:28.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.778+0000 7f1205c0f740 1 -- 172.21.15.83:0/1877403861 wait complete. 2024-04-24T09:14:28.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.778+0000 7f1205c0f740 1 -- start start 2024-04-24T09:14:28.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.779+0000 7f1205c0f740 1 Processor -- start 2024-04-24T09:14:28.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.779+0000 7f1205c0f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e795225b40 0x55e79522b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:28.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.779+0000 7f1205c0f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e795226460 0x55e795249e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:28.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.779+0000 7f1205c0f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e795230b30 0x55e795231ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:28.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.779+0000 7f1205c0f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f11dc0115a0 con 0x55e795230b30 2024-04-24T09:14:28.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.779+0000 7f1205c0f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f11dc010ed0 con 0x55e795225b40 2024-04-24T09:14:28.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.779+0000 7f1205c0f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f11dc00e100 con 0x55e795226460 2024-04-24T09:14:28.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.779+0000 7f1205c0f740 10 client.0 _ll_register_callbacks cb 0x7f11e8004540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:14:28.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.779+0000 7f11fbe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e795225b40 0x55e79522b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:28.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.780+0000 7f1200e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e795230b30 0x55e795231ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:28.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.780+0000 7f11fbe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e795225b40 0x55e79522b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:53112/0 (socket says 172.21.15.83:53112) 2024-04-24T09:14:28.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.780+0000 7f11fbe00640 1 -- 172.21.15.83:0/30684130 learned_addr learned my addr 172.21.15.83:0/30684130 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:14:28.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.780+0000 7f11fb400640 1 --2- 172.21.15.83:0/30684130 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e795226460 0x55e795249e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:28.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.780+0000 7f11fbe00640 1 -- 172.21.15.83:0/30684130 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e795226460 msgr2=0x55e795249e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:14:28.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.780+0000 7f11fbe00640 1 --2- 172.21.15.83:0/30684130 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e795226460 0x55e795249e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:28.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.780+0000 7f11fbe00640 1 -- 172.21.15.83:0/30684130 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e795230b30 msgr2=0x55e795231ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:14:28.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.780+0000 7f11fbe00640 1 --2- 172.21.15.83:0/30684130 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e795230b30 0x55e795231ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:28.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.780+0000 7f11fbe00640 1 -- 172.21.15.83:0/30684130 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e795226290 con 0x55e795225b40 2024-04-24T09:14:28.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.781+0000 7f11fbe00640 1 --2- 172.21.15.83:0/30684130 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e795225b40 0x55e79522b4a0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f11ec00bc60 tx=0x7f11ec01c510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=773d61e960da8c8d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:14:28.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.781+0000 7f11e6000640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:14:28.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.781+0000 7f11e6000640 1 -- 172.21.15.83:0/30684130 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11ec037060 con 0x55e795225b40 2024-04-24T09:14:28.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.781+0000 7f11e6000640 1 -- 172.21.15.83:0/30684130 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(128..128 src has 1..128) ==== 9583+0+0 (secure 0 0 0) 0x7f11ec005590 con 0x55e795225b40 2024-04-24T09:14:28.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.781+0000 7f1205c0f740 1 -- 172.21.15.83:0/30684130 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11ec028900 con 0x55e795225b40 2024-04-24T09:14:28.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.781+0000 7f11e6000640 1 -- 172.21.15.83:0/30684130 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f11ec00bdd0 con 0x55e795225b40 2024-04-24T09:14:28.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.782+0000 7f1205c0f740 10 client.40379 fetch_fsmap learned FSMap version 65 2024-04-24T09:14:28.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.782+0000 7f1205c0f740 1 -- 172.21.15.83:0/30684130 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x55e794f677d0 con 0x55e795225b40 2024-04-24T09:14:28.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.782+0000 7f11e6000640 1 -- 172.21.15.83:0/30684130 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f11ec028900 con 0x55e795225b40 2024-04-24T09:14:28.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.782+0000 7f11e6000640 1 -- 172.21.15.83:0/30684130 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7f11ec031070 con 0x55e795225b40 2024-04-24T09:14:28.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.783+0000 7f1205c0f740 10 client.40379 fetch_fsmap finished waiting for FSMap version 65 2024-04-24T09:14:28.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.783+0000 7f1205c0f740 10 client.40379 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-24T09:14:28.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.783+0000 7f1205c0f740 20 client.40379 populate_metadata read hostname 'smithi083' 2024-04-24T09:14:28.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.783+0000 7f1205c0f740 1 --2- 172.21.15.83:0/30684130 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55e7952b8aa0 0x55e7952baec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:28.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.783+0000 7f1205c0f740 4 client.40379 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:14:28.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.783+0000 7f1205c0f740 1 -- 172.21.15.83:0/30684130 --> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e795192a60 con 0x55e7952b8aa0 2024-04-24T09:14:28.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.783+0000 7f11fb400640 1 --2- 172.21.15.83:0/30684130 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55e7952b8aa0 0x55e7952baec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:28.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.784+0000 7f11fb400640 1 --2- 172.21.15.83:0/30684130 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55e7952b8aa0 0x55e7952baec0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1d5c574ebd5f099b server_cookie=a1651c41e4b2622c in_seq=0 out_seq=0 2024-04-24T09:14:28.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.785+0000 7f11e6000640 10 client.40379 ms_handle_connect on v2:172.21.15.189:6838/2548965336 2024-04-24T09:14:28.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.785+0000 7f11e6000640 1 -- 172.21.15.83:0/30684130 <== mds.4 v2:172.21.15.189:6838/2548965336 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e795192a60 con 0x55e7952b8aa0 2024-04-24T09:14:28.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.785+0000 7f11e6000640 10 client.40379 handle_command_reply: tid=0 2024-04-24T09:14:28.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.785+0000 7f1205c0f740 1 client.40379 shutdown 2024-04-24T09:14:28.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.785+0000 7f1205c0f740 1 -- 172.21.15.83:0/30684130 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e795225b40 msgr2=0x55e79522b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:28.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.785+0000 7f1205c0f740 1 --2- 172.21.15.83:0/30684130 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e795225b40 0x55e79522b4a0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f11ec00bc60 tx=0x7f11ec01c510 comp rx=0 tx=0).stop 2024-04-24T09:14:28.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.785+0000 7f1205c0f740 1 -- 172.21.15.83:0/30684130 shutdown_connections 2024-04-24T09:14:28.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.785+0000 7f1205c0f740 1 --2- 172.21.15.83:0/30684130 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55e7952b8aa0 0x55e7952baec0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:28.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.785+0000 7f1205c0f740 1 --2- 172.21.15.83:0/30684130 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e795230b30 0x55e795231ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:28.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.785+0000 7f1205c0f740 1 --2- 172.21.15.83:0/30684130 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e795226460 0x55e795249e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:28.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.785+0000 7f1205c0f740 1 --2- 172.21.15.83:0/30684130 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e795225b40 0x55e79522b4a0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:28.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.785+0000 7f11e6000640 0 client.40379 ms_handle_reset on v2:172.21.15.189:6838/2548965336 2024-04-24T09:14:28.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.785+0000 7f1205c0f740 1 -- 172.21.15.83:0/30684130 >> 172.21.15.83:0/30684130 conn(0x55e795221e60 msgr2=0x55e7952206c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:14:28.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.785+0000 7f1205c0f740 1 -- 172.21.15.83:0/30684130 shutdown_connections 2024-04-24T09:14:28.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.785+0000 7f1205c0f740 1 -- 172.21.15.83:0/30684130 wait complete. 2024-04-24T09:14:28.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.786+0000 7f1205c0f740 20 client.40379 trim_cache size 0 max 16384 2024-04-24T09:14:28.786 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:14:28.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.786+0000 7f1201800640 1 -- 172.21.15.83:0/2769768551 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f11e00a3e20 msgr2=0x7f11e00a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:28.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.786+0000 7f1201800640 1 --2- 172.21.15.83:0/2769768551 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f11e00a3e20 0x7f11e00a62e0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f11e8006010 tx=0x7f11e8000bc0 comp rx=0 tx=0).stop 2024-04-24T09:14:28.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.786+0000 7f1201800640 1 -- 172.21.15.83:0/2769768551 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11fc111310 msgr2=0x7f11fc1117a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:28.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.786+0000 7f1201800640 1 --2- 172.21.15.83:0/2769768551 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11fc111310 0x7f11fc1117a0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f11f400ca30 tx=0x7f11f400cef0 comp rx=0 tx=0).stop 2024-04-24T09:14:28.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.787+0000 7f1201800640 1 -- 172.21.15.83:0/2769768551 shutdown_connections 2024-04-24T09:14:28.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.787+0000 7f1201800640 1 --2- 172.21.15.83:0/2769768551 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f11e00a3e20 0x7f11e00a62e0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:28.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.787+0000 7f1201800640 1 --2- 172.21.15.83:0/2769768551 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11fc111310 0x7f11fc1117a0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:28.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.787+0000 7f1201800640 1 --2- 172.21.15.83:0/2769768551 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f11fc1060d0 0x7f11fc110dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:28.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.787+0000 7f1201800640 1 --2- 172.21.15.83:0/2769768551 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11fc101000 0x7f11fc113ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:28.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.787+0000 7f1201800640 1 -- 172.21.15.83:0/2769768551 >> 172.21.15.83:0/2769768551 conn(0x7f11fc0fc880 msgr2=0x7f11fc10dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:14:28.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.787+0000 7f1201800640 1 -- 172.21.15.83:0/2769768551 shutdown_connections 2024-04-24T09:14:28.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:28.787+0000 7f1201800640 1 -- 172.21.15.83:0/2769768551 wait complete. 2024-04-24T09:14:28.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:14:28 smithi083 ceph-mon[27907]: pgmap v722: 129 pgs: 129 active+clean; 1008 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.9 MiB/s wr, 257 op/s 2024-04-24T09:14:28.809 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-24T09:14:29.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.088+0000 7fe8f7e00640 1 -- 172.21.15.83:0/3234605557 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8f8104320 msgr2=0x7fe8f8104700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:29.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.088+0000 7fe8f7e00640 1 --2- 172.21.15.83:0/3234605557 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8f8104320 0x7fe8f8104700 secure :-1 s=READY pgs=2663 cs=0 l=1 rev1=1 crypto rx=0x7fe8e4009900 tx=0x7fe8e402a3f0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.089+0000 7fe8f7e00640 1 -- 172.21.15.83:0/3234605557 shutdown_connections 2024-04-24T09:14:29.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.089+0000 7fe8f7e00640 1 --2- 172.21.15.83:0/3234605557 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe8f81093f0 0x7fe8f8110fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.089+0000 7fe8f7e00640 1 --2- 172.21.15.83:0/3234605557 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe8f8104cd0 0x7fe8f8108d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.089+0000 7fe8f7e00640 1 --2- 172.21.15.83:0/3234605557 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8f8104320 0x7fe8f8104700 unknown :-1 s=CLOSED pgs=2663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.089+0000 7fe8f7e00640 1 -- 172.21.15.83:0/3234605557 >> 172.21.15.83:0/3234605557 conn(0x7fe8f80ffb40 msgr2=0x7fe8f8101f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:14:29.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.089+0000 7fe8f7e00640 1 -- 172.21.15.83:0/3234605557 shutdown_connections 2024-04-24T09:14:29.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.089+0000 7fe8f7e00640 1 -- 172.21.15.83:0/3234605557 wait complete. 2024-04-24T09:14:29.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.090+0000 7fe8f7e00640 1 Processor -- start 2024-04-24T09:14:29.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.090+0000 7fe8f7e00640 1 -- start start 2024-04-24T09:14:29.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.091+0000 7fe8f7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8f8104320 0x7fe8f81a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:29.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.091+0000 7fe8f7e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe8f81093f0 0x7fe8f81a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:29.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.091+0000 7fe8f7e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe8f819a720 0x7fe8f819aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:29.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.091+0000 7fe8f7e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f81143a0 con 0x7fe8f8104320 2024-04-24T09:14:29.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.091+0000 7fe8f7e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f8114220 con 0x7fe8f81093f0 2024-04-24T09:14:29.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.091+0000 7fe8f7e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f8114520 con 0x7fe8f819a720 2024-04-24T09:14:29.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.091+0000 7fe8f6000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe8f81093f0 0x7fe8f81a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:29.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.091+0000 7fe8f6a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8f8104320 0x7fe8f81a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:29.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.091+0000 7fe8f6000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe8f81093f0 0x7fe8f81a0ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:35284/0 (socket says 172.21.15.83:35284) 2024-04-24T09:14:29.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.091+0000 7fe8f6000640 1 -- 172.21.15.83:0/3995396015 learned_addr learned my addr 172.21.15.83:0/3995396015 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:14:29.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.092+0000 7fe8f7400640 1 --2- 172.21.15.83:0/3995396015 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe8f819a720 0x7fe8f819aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:29.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.092+0000 7fe8f6a00640 1 -- 172.21.15.83:0/3995396015 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe8f819a720 msgr2=0x7fe8f819aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:29.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.092+0000 7fe8f6a00640 1 --2- 172.21.15.83:0/3995396015 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe8f819a720 0x7fe8f819aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.092+0000 7fe8f6a00640 1 -- 172.21.15.83:0/3995396015 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe8f81093f0 msgr2=0x7fe8f81a0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:29.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.092+0000 7fe8f6a00640 1 --2- 172.21.15.83:0/3995396015 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe8f81093f0 0x7fe8f81a0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.092+0000 7fe8f6a00640 1 -- 172.21.15.83:0/3995396015 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8f819b320 con 0x7fe8f8104320 2024-04-24T09:14:29.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.092+0000 7fe8f6a00640 1 --2- 172.21.15.83:0/3995396015 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8f8104320 0x7fe8f81a0660 secure :-1 s=READY pgs=2664 cs=0 l=1 rev1=1 crypto rx=0x7fe8e402a9e0 tx=0x7fe8e4003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:14:29.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.093+0000 7fe8ef400640 1 -- 172.21.15.83:0/3995396015 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e403f070 con 0x7fe8f8104320 2024-04-24T09:14:29.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.093+0000 7fe8f7e00640 1 -- 172.21.15.83:0/3995396015 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8f819b5b0 con 0x7fe8f8104320 2024-04-24T09:14:29.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.093+0000 7fe8ef400640 1 -- 172.21.15.83:0/3995396015 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe8e4031d60 con 0x7fe8f8104320 2024-04-24T09:14:29.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.093+0000 7fe8ef400640 1 -- 172.21.15.83:0/3995396015 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e4035030 con 0x7fe8f8104320 2024-04-24T09:14:29.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.093+0000 7fe8f7e00640 1 -- 172.21.15.83:0/3995396015 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8f81a7740 con 0x7fe8f8104320 2024-04-24T09:14:29.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.096+0000 7fe90041a740 1 Processor -- start 2024-04-24T09:14:29.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.096+0000 7fe90041a740 1 -- start start 2024-04-24T09:14:29.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.097+0000 7fe90041a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56524d09eb40 0x56524d09ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:29.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.097+0000 7fe90041a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56524d09f460 0x56524d0a95f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:29.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.097+0000 7fe90041a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56524d0a9b30 0x56524d0abf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:29.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.097+0000 7fe90041a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56524d0041d0 con 0x56524d0a9b30 2024-04-24T09:14:29.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.097+0000 7fe90041a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56524cfaa450 con 0x56524d09f460 2024-04-24T09:14:29.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.097+0000 7fe90041a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56524d000190 con 0x56524d09eb40 2024-04-24T09:14:29.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.097+0000 7fe8f6000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56524d0a9b30 0x56524d0abf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:29.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.097+0000 7fe8ef400640 1 -- 172.21.15.83:0/3995396015 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe8e4042070 con 0x7fe8f8104320 2024-04-24T09:14:29.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.097+0000 7fe8f6000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56524d0a9b30 0x56524d0abf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37016/0 (socket says 172.21.15.83:37016) 2024-04-24T09:14:29.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.097+0000 7fe8f6000640 1 -- 172.21.15.83:0/3882737038 learned_addr learned my addr 172.21.15.83:0/3882737038 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:14:29.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.097+0000 7fe8f7400640 1 --2- 172.21.15.83:0/3882737038 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56524d09eb40 0x56524d09ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:29.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.098+0000 7fe8f6a00640 1 --2- 172.21.15.83:0/3882737038 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56524d09f460 0x56524d0a95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:29.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.098+0000 7fe8f6000640 1 -- 172.21.15.83:0/3882737038 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56524d09eb40 msgr2=0x56524d09ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:14:29.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.098+0000 7fe8f6000640 1 --2- 172.21.15.83:0/3882737038 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56524d09eb40 0x56524d09ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.098+0000 7fe8f6000640 1 -- 172.21.15.83:0/3882737038 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56524d09f460 msgr2=0x56524d0a95f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:14:29.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.098+0000 7fe8f6000640 1 --2- 172.21.15.83:0/3882737038 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56524d09f460 0x56524d0a95f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.098+0000 7fe8f6000640 1 -- 172.21.15.83:0/3882737038 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56524d067310 con 0x56524d0a9b30 2024-04-24T09:14:29.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.098+0000 7fe8f6000640 1 --2- 172.21.15.83:0/3882737038 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56524d0a9b30 0x56524d0abf20 secure :-1 s=READY pgs=2665 cs=0 l=1 rev1=1 crypto rx=0x7fe8e800ac70 tx=0x7fe8e800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75e430f759068f37 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:14:29.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.099+0000 7fe8e3400640 1 -- 172.21.15.83:0/3882737038 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e801d070 con 0x56524d0a9b30 2024-04-24T09:14:29.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.099+0000 7fe8e3400640 1 -- 172.21.15.83:0/3882737038 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe8e8018af0 con 0x56524d0a9b30 2024-04-24T09:14:29.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.099+0000 7fe8e3400640 1 -- 172.21.15.83:0/3882737038 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e8010a80 con 0x56524d0a9b30 2024-04-24T09:14:29.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.099+0000 7fe8ef400640 1 --2- 172.21.15.83:0/3995396015 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe8dc0a3da0 0x7fe8dc0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:29.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.100+0000 7fe8f7400640 1 --2- 172.21.15.83:0/3995396015 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe8dc0a3da0 0x7fe8dc0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:29.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.100+0000 7fe90041a740 1 -- 172.21.15.83:0/3882737038 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56524d0a9b30 msgr2=0x56524d0abf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:29.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.100+0000 7fe90041a740 1 --2- 172.21.15.83:0/3882737038 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56524d0a9b30 0x56524d0abf20 secure :-1 s=READY pgs=2665 cs=0 l=1 rev1=1 crypto rx=0x7fe8e800ac70 tx=0x7fe8e800f790 comp rx=0 tx=0).stop 2024-04-24T09:14:29.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.101+0000 7fe8ef400640 1 -- 172.21.15.83:0/3995396015 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9583+0+0 (secure 0 0 0) 0x7fe8e40f0ff0 con 0x7fe8f8104320 2024-04-24T09:14:29.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.101+0000 7fe8f7400640 1 --2- 172.21.15.83:0/3995396015 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe8dc0a3da0 0x7fe8dc0a6260 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fe8f0005fd0 tx=0x7fe8f00096e0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:14:29.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.101+0000 7fe90041a740 1 -- 172.21.15.83:0/3882737038 shutdown_connections 2024-04-24T09:14:29.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.101+0000 7fe90041a740 1 --2- 172.21.15.83:0/3882737038 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56524d0a9b30 0x56524d0abf20 unknown :-1 s=CLOSED pgs=2665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.101+0000 7fe90041a740 1 --2- 172.21.15.83:0/3882737038 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56524d09f460 0x56524d0a95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.101+0000 7fe90041a740 1 --2- 172.21.15.83:0/3882737038 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56524d09eb40 0x56524d09ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.101+0000 7fe90041a740 1 -- 172.21.15.83:0/3882737038 >> 172.21.15.83:0/3882737038 conn(0x56524d09ae60 msgr2=0x56524d09b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:14:29.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.102+0000 7fe90041a740 1 -- 172.21.15.83:0/3882737038 shutdown_connections 2024-04-24T09:14:29.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.102+0000 7fe90041a740 1 -- 172.21.15.83:0/3882737038 wait complete. 2024-04-24T09:14:29.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.102+0000 7fe90041a740 1 -- start start 2024-04-24T09:14:29.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.103+0000 7fe90041a740 1 Processor -- start 2024-04-24T09:14:29.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.103+0000 7fe90041a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56524d09eb40 0x56524d0a9530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:29.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.103+0000 7fe90041a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56524d09f460 0x56524d0a6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:29.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.103+0000 7fe90041a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56524d0a9b30 0x56524d0a6e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:29.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.103+0000 7fe90041a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56524d0ac9c0 con 0x56524d09eb40 2024-04-24T09:14:29.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.103+0000 7fe90041a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56524cfbd810 con 0x56524d09f460 2024-04-24T09:14:29.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.103+0000 7fe90041a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56524d0acb40 con 0x56524d0a9b30 2024-04-24T09:14:29.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.103+0000 7fe90041a740 10 client.0 _ll_register_callbacks cb 0x56524cf8b140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:14:29.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.104+0000 7fe8f6000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56524d09f460 0x56524d0a6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:29.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.104+0000 7fe8f6000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56524d09f460 0x56524d0a6820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:35302/0 (socket says 172.21.15.83:35302) 2024-04-24T09:14:29.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.104+0000 7fe8f6000640 1 -- 172.21.15.83:0/708160664 learned_addr learned my addr 172.21.15.83:0/708160664 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:14:29.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.104+0000 7fe8f7400640 1 --2- 172.21.15.83:0/708160664 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56524d0a9b30 0x56524d0a6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:29.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.104+0000 7fe8f6000640 1 -- 172.21.15.83:0/708160664 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56524d0a9b30 msgr2=0x56524d0a6e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:14:29.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.104+0000 7fe8f6000640 1 --2- 172.21.15.83:0/708160664 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56524d0a9b30 0x56524d0a6e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.104+0000 7fe8f6000640 1 -- 172.21.15.83:0/708160664 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56524d09eb40 msgr2=0x56524d0a9530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:14:29.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.104+0000 7fe8f6a00640 1 --2- 172.21.15.83:0/708160664 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56524d09eb40 0x56524d0a9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:29.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.104+0000 7fe8f6000640 1 --2- 172.21.15.83:0/708160664 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56524d09eb40 0x56524d0a9530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.104+0000 7fe8f6000640 1 -- 172.21.15.83:0/708160664 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56524cfce3f0 con 0x56524d09f460 2024-04-24T09:14:29.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.104+0000 7fe8f6a00640 1 --2- 172.21.15.83:0/708160664 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56524d09eb40 0x56524d0a9530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:14:29.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.105+0000 7fe8f6000640 1 --2- 172.21.15.83:0/708160664 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56524d09f460 0x56524d0a6820 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fe8e800a870 tx=0x7fe8e8005ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a76239b5a61685 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:14:29.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.105+0000 7fe8d7e00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:14:29.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.106+0000 7fe8d7e00640 1 -- 172.21.15.83:0/708160664 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e801d050 con 0x56524d09f460 2024-04-24T09:14:29.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.106+0000 7fe8d7e00640 1 -- 172.21.15.83:0/708160664 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(128..128 src has 1..128) ==== 9583+0+0 (secure 0 0 0) 0x7fe8e8005240 con 0x56524d09f460 2024-04-24T09:14:29.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.106+0000 7fe90041a740 1 -- 172.21.15.83:0/708160664 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8e8018af0 con 0x56524d09f460 2024-04-24T09:14:29.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.107+0000 7fe8d7e00640 1 -- 172.21.15.83:0/708160664 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7fe8e8010430 con 0x56524d09f460 2024-04-24T09:14:29.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.107+0000 7fe90041a740 10 client.40391 fetch_fsmap learned FSMap version 65 2024-04-24T09:14:29.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.107+0000 7fe90041a740 1 -- 172.21.15.83:0/708160664 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x56524cd163f0 con 0x56524d09f460 2024-04-24T09:14:29.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.108+0000 7fe8d7e00640 1 -- 172.21.15.83:0/708160664 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe8e8018af0 con 0x56524d09f460 2024-04-24T09:14:29.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.108+0000 7fe8d7e00640 1 -- 172.21.15.83:0/708160664 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7fe8e80047c0 con 0x56524d09f460 2024-04-24T09:14:29.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.108+0000 7fe90041a740 10 client.40391 fetch_fsmap finished waiting for FSMap version 65 2024-04-24T09:14:29.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.108+0000 7fe90041a740 10 client.40391 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-24T09:14:29.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.108+0000 7fe90041a740 20 client.40391 populate_metadata read hostname 'smithi083' 2024-04-24T09:14:29.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.108+0000 7fe90041a740 1 --2- 172.21.15.83:0/708160664 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x56524d0b9c70 0x56524d0bc060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:29.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.108+0000 7fe90041a740 4 client.40391 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:14:29.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.109+0000 7fe90041a740 1 -- 172.21.15.83:0/708160664 --> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56524d011bf0 con 0x56524d0b9c70 2024-04-24T09:14:29.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.109+0000 7fe8f6a00640 1 --2- 172.21.15.83:0/708160664 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x56524d0b9c70 0x56524d0bc060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:29.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.110+0000 7fe8f6a00640 1 --2- 172.21.15.83:0/708160664 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x56524d0b9c70 0x56524d0bc060 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=939c120c885c0011 server_cookie=fb8f414f85d863a7 in_seq=0 out_seq=0 2024-04-24T09:14:29.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.110+0000 7fe8d7e00640 10 client.40391 ms_handle_connect on v2:172.21.15.138:6832/1157967658 2024-04-24T09:14:29.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.110+0000 7fe8d7e00640 1 -- 172.21.15.83:0/708160664 <== mds.3 v2:172.21.15.138:6832/1157967658 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56524d011bf0 con 0x56524d0b9c70 2024-04-24T09:14:29.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.111+0000 7fe8d7e00640 10 client.40391 handle_command_reply: tid=0 2024-04-24T09:14:29.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.111+0000 7fe90041a740 1 client.40391 shutdown 2024-04-24T09:14:29.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.111+0000 7fe90041a740 1 -- 172.21.15.83:0/708160664 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56524d09f460 msgr2=0x56524d0a6820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:29.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.111+0000 7fe90041a740 1 --2- 172.21.15.83:0/708160664 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56524d09f460 0x56524d0a6820 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fe8e800a870 tx=0x7fe8e8005ec0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.111+0000 7fe90041a740 1 -- 172.21.15.83:0/708160664 shutdown_connections 2024-04-24T09:14:29.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.111+0000 7fe90041a740 1 --2- 172.21.15.83:0/708160664 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x56524d0b9c70 0x56524d0bc060 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.111+0000 7fe90041a740 1 --2- 172.21.15.83:0/708160664 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56524d0a9b30 0x56524d0a6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.111+0000 7fe90041a740 1 --2- 172.21.15.83:0/708160664 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56524d09f460 0x56524d0a6820 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.111+0000 7fe90041a740 1 --2- 172.21.15.83:0/708160664 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56524d09eb40 0x56524d0a9530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.111+0000 7fe90041a740 1 -- 172.21.15.83:0/708160664 >> 172.21.15.83:0/708160664 conn(0x56524d09ae60 msgr2=0x56524d0a0c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:14:29.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.111+0000 7fe8d7e00640 0 client.40391 ms_handle_reset on v2:172.21.15.138:6832/1157967658 2024-04-24T09:14:29.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.111+0000 7fe90041a740 1 -- 172.21.15.83:0/708160664 shutdown_connections 2024-04-24T09:14:29.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.111+0000 7fe90041a740 1 -- 172.21.15.83:0/708160664 wait complete. 2024-04-24T09:14:29.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.112+0000 7fe90041a740 20 client.40391 trim_cache size 0 max 16384 2024-04-24T09:14:29.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.136+0000 7fe90041a740 1 Processor -- start 2024-04-24T09:14:29.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.136+0000 7fe90041a740 1 -- start start 2024-04-24T09:14:29.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.136+0000 7fe90041a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56524d09eb40 0x56524d0b58b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:29.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.136+0000 7fe90041a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56524d0a9b30 0x56524d0a9060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:29.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.136+0000 7fe90041a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56524d0a95a0 0x56524d0c1660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:29.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.136+0000 7fe90041a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f81b15e0 con 0x56524d09eb40 2024-04-24T09:14:29.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.136+0000 7fe90041a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f81b3180 con 0x56524d0a95a0 2024-04-24T09:14:29.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.136+0000 7fe90041a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f81b1cb0 con 0x56524d0a9b30 2024-04-24T09:14:29.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.137+0000 7fe8f6a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56524d09eb40 0x56524d0b58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:29.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.137+0000 7fe8f7400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56524d0a95a0 0x56524d0c1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:29.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.137+0000 7fe8f6a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56524d09eb40 0x56524d0b58b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37026/0 (socket says 172.21.15.83:37026) 2024-04-24T09:14:29.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.137+0000 7fe8f6a00640 1 -- 172.21.15.83:0/1953226699 learned_addr learned my addr 172.21.15.83:0/1953226699 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:14:29.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.137+0000 7fe8f6000640 1 --2- 172.21.15.83:0/1953226699 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56524d0a9b30 0x56524d0a9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:29.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.138+0000 7fe8f6a00640 1 -- 172.21.15.83:0/1953226699 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56524d0a9b30 msgr2=0x56524d0a9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:14:29.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.138+0000 7fe8f6a00640 1 --2- 172.21.15.83:0/1953226699 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56524d0a9b30 0x56524d0a9060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.138+0000 7fe8f6a00640 1 -- 172.21.15.83:0/1953226699 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56524d0a95a0 msgr2=0x56524d0c1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:14:29.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.138+0000 7fe8f6a00640 1 --2- 172.21.15.83:0/1953226699 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56524d0a95a0 0x56524d0c1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.138+0000 7fe8f6a00640 1 -- 172.21.15.83:0/1953226699 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56524cde07d0 con 0x56524d09eb40 2024-04-24T09:14:29.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.138+0000 7fe8f7400640 1 --2- 172.21.15.83:0/1953226699 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56524d0a95a0 0x56524d0c1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:14:29.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.138+0000 7fe8f6a00640 1 --2- 172.21.15.83:0/1953226699 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56524d09eb40 0x56524d0b58b0 secure :-1 s=READY pgs=2666 cs=0 l=1 rev1=1 crypto rx=0x7fe8e4006010 tx=0x7fe8e402a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e0178bced380b40 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:14:29.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.139+0000 7fe8e2a00640 1 -- 172.21.15.83:0/1953226699 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e4049040 con 0x56524d09eb40 2024-04-24T09:14:29.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.139+0000 7fe8e2a00640 1 -- 172.21.15.83:0/1953226699 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe8e40bd210 con 0x56524d09eb40 2024-04-24T09:14:29.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.139+0000 7fe8e2a00640 1 -- 172.21.15.83:0/1953226699 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e40baea0 con 0x56524d09eb40 2024-04-24T09:14:29.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.140+0000 7fe90041a740 1 -- 172.21.15.83:0/1953226699 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56524d09eb40 msgr2=0x56524d0b58b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:29.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.140+0000 7fe90041a740 1 --2- 172.21.15.83:0/1953226699 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56524d09eb40 0x56524d0b58b0 secure :-1 s=READY pgs=2666 cs=0 l=1 rev1=1 crypto rx=0x7fe8e4006010 tx=0x7fe8e402a3f0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.140+0000 7fe90041a740 1 -- 172.21.15.83:0/1953226699 shutdown_connections 2024-04-24T09:14:29.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.140+0000 7fe90041a740 1 --2- 172.21.15.83:0/1953226699 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56524d0a95a0 0x56524d0c1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.140+0000 7fe90041a740 1 --2- 172.21.15.83:0/1953226699 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56524d0a9b30 0x56524d0a9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.140+0000 7fe90041a740 1 --2- 172.21.15.83:0/1953226699 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56524d09eb40 0x56524d0b58b0 unknown :-1 s=CLOSED pgs=2666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.140+0000 7fe90041a740 1 -- 172.21.15.83:0/1953226699 >> 172.21.15.83:0/1953226699 conn(0x56524d09ae60 msgr2=0x56524d09b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:14:29.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.140+0000 7fe90041a740 1 -- 172.21.15.83:0/1953226699 shutdown_connections 2024-04-24T09:14:29.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.140+0000 7fe90041a740 1 -- 172.21.15.83:0/1953226699 wait complete. 2024-04-24T09:14:29.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.141+0000 7fe90041a740 1 -- start start 2024-04-24T09:14:29.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.141+0000 7fe90041a740 1 Processor -- start 2024-04-24T09:14:29.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.141+0000 7fe90041a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56524d09eb40 0x56524d11d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:29.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.141+0000 7fe90041a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56524d0a95a0 0x56524d11d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:29.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.141+0000 7fe90041a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56524d0a9b30 0x56524d0b1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:29.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.142+0000 7fe90041a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f81b2a80 con 0x56524d0a95a0 2024-04-24T09:14:29.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.142+0000 7fe90041a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f81b23b0 con 0x56524d09eb40 2024-04-24T09:14:29.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.142+0000 7fe90041a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f81af5e0 con 0x56524d0a9b30 2024-04-24T09:14:29.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.142+0000 7fe90041a740 10 client.0 _ll_register_callbacks cb 0x7fe8e801b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:14:29.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.142+0000 7fe8f6a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56524d09eb40 0x56524d11d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:29.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.142+0000 7fe8f6a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56524d09eb40 0x56524d11d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:35314/0 (socket says 172.21.15.83:35314) 2024-04-24T09:14:29.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.142+0000 7fe8f6a00640 1 -- 172.21.15.83:0/3616801103 learned_addr learned my addr 172.21.15.83:0/3616801103 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:14:29.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.143+0000 7fe8f6a00640 1 -- 172.21.15.83:0/3616801103 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56524d0a9b30 msgr2=0x56524d0b1db0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T09:14:29.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.143+0000 7fe8f6a00640 1 --2- 172.21.15.83:0/3616801103 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56524d0a9b30 0x56524d0b1db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.143+0000 7fe8f6a00640 1 -- 172.21.15.83:0/3616801103 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56524d0a95a0 msgr2=0x56524d11d7c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T09:14:29.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.143+0000 7fe8f6a00640 1 --2- 172.21.15.83:0/3616801103 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56524d0a95a0 0x56524d11d7c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.143+0000 7fe8f6a00640 1 -- 172.21.15.83:0/3616801103 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56524d0b25e0 con 0x56524d09eb40 2024-04-24T09:14:29.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.143+0000 7fe8f6a00640 1 --2- 172.21.15.83:0/3616801103 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56524d09eb40 0x56524d11d280 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fe8e40b8f70 tx=0x7fe8e40045e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7160bebb47bff36 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:14:29.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.143+0000 7fe8d7e00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:14:29.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.144+0000 7fe8d7e00640 1 -- 172.21.15.83:0/3616801103 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e4049070 con 0x56524d09eb40 2024-04-24T09:14:29.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.144+0000 7fe8d7e00640 1 -- 172.21.15.83:0/3616801103 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(128..128 src has 1..128) ==== 9583+0+0 (secure 0 0 0) 0x7fe8e40b9180 con 0x56524d09eb40 2024-04-24T09:14:29.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.145+0000 7fe90041a740 1 -- 172.21.15.83:0/3616801103 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8e40bd210 con 0x56524d09eb40 2024-04-24T09:14:29.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.145+0000 7fe8d7e00640 1 -- 172.21.15.83:0/3616801103 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7fe8e40c0ec0 con 0x56524d09eb40 2024-04-24T09:14:29.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.145+0000 7fe90041a740 10 client.40397 fetch_fsmap learned FSMap version 65 2024-04-24T09:14:29.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.145+0000 7fe90041a740 1 -- 172.21.15.83:0/3616801103 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x56524d1286e0 con 0x56524d09eb40 2024-04-24T09:14:29.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.145+0000 7fe8d7e00640 1 -- 172.21.15.83:0/3616801103 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe8e40c5780 con 0x56524d09eb40 2024-04-24T09:14:29.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.146+0000 7fe8d7e00640 1 -- 172.21.15.83:0/3616801103 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7fe8e404c060 con 0x56524d09eb40 2024-04-24T09:14:29.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.146+0000 7fe90041a740 10 client.40397 fetch_fsmap finished waiting for FSMap version 65 2024-04-24T09:14:29.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.146+0000 7fe90041a740 10 client.40397 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-24T09:14:29.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.146+0000 7fe90041a740 20 client.40397 populate_metadata read hostname 'smithi083' 2024-04-24T09:14:29.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.146+0000 7fe90041a740 1 --2- 172.21.15.83:0/3616801103 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x56524d128b60 0x56524d12af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:29.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.146+0000 7fe90041a740 4 client.40397 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:14:29.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.146+0000 7fe90041a740 1 -- 172.21.15.83:0/3616801103 --> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fe8e40bd210 con 0x56524d128b60 2024-04-24T09:14:29.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.147+0000 7fe8f6000640 1 --2- 172.21.15.83:0/3616801103 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x56524d128b60 0x56524d12af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:29.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.148+0000 7fe8f6000640 1 --2- 172.21.15.83:0/3616801103 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x56524d128b60 0x56524d12af50 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=891a030e7117a5de server_cookie=4e2ae88e0d6939d3 in_seq=0 out_seq=0 2024-04-24T09:14:29.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.148+0000 7fe8d7e00640 10 client.40397 ms_handle_connect on v2:172.21.15.138:6832/1157967658 2024-04-24T09:14:29.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.148+0000 7fe8d7e00640 1 -- 172.21.15.83:0/3616801103 <== mds.3 v2:172.21.15.138:6832/1157967658 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fe8e40bd210 con 0x56524d128b60 2024-04-24T09:14:29.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.148+0000 7fe8d7e00640 10 client.40397 handle_command_reply: tid=0 2024-04-24T09:14:29.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.148+0000 7fe90041a740 1 client.40397 shutdown 2024-04-24T09:14:29.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.148+0000 7fe90041a740 1 -- 172.21.15.83:0/3616801103 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56524d09eb40 msgr2=0x56524d11d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:29.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.148+0000 7fe90041a740 1 --2- 172.21.15.83:0/3616801103 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56524d09eb40 0x56524d11d280 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fe8e40b8f70 tx=0x7fe8e40045e0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.148+0000 7fe90041a740 1 -- 172.21.15.83:0/3616801103 shutdown_connections 2024-04-24T09:14:29.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.148+0000 7fe90041a740 1 --2- 172.21.15.83:0/3616801103 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x56524d128b60 0x56524d12af50 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.148+0000 7fe90041a740 1 --2- 172.21.15.83:0/3616801103 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56524d0a9b30 0x56524d0b1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.148+0000 7fe90041a740 1 --2- 172.21.15.83:0/3616801103 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56524d0a95a0 0x56524d11d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.148+0000 7fe90041a740 1 --2- 172.21.15.83:0/3616801103 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56524d09eb40 0x56524d11d280 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.148+0000 7fe90041a740 1 -- 172.21.15.83:0/3616801103 >> 172.21.15.83:0/3616801103 conn(0x56524d09ae60 msgr2=0x56524d0996c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:14:29.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.149+0000 7fe8d7e00640 0 client.40397 ms_handle_reset on v2:172.21.15.138:6832/1157967658 2024-04-24T09:14:29.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.149+0000 7fe90041a740 1 -- 172.21.15.83:0/3616801103 shutdown_connections 2024-04-24T09:14:29.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.149+0000 7fe90041a740 1 -- 172.21.15.83:0/3616801103 wait complete. 2024-04-24T09:14:29.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.149+0000 7fe90041a740 20 client.40397 trim_cache size 0 max 16384 2024-04-24T09:14:29.149 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:14:29.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.150+0000 7fe8f7e00640 1 -- 172.21.15.83:0/3995396015 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe8dc0a3da0 msgr2=0x7fe8dc0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:29.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.150+0000 7fe8f7e00640 1 --2- 172.21.15.83:0/3995396015 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe8dc0a3da0 0x7fe8dc0a6260 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fe8f0005fd0 tx=0x7fe8f00096e0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.150+0000 7fe8f7e00640 1 -- 172.21.15.83:0/3995396015 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8f8104320 msgr2=0x7fe8f81a0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:29.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.150+0000 7fe8f7e00640 1 --2- 172.21.15.83:0/3995396015 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8f8104320 0x7fe8f81a0660 secure :-1 s=READY pgs=2664 cs=0 l=1 rev1=1 crypto rx=0x7fe8e402a9e0 tx=0x7fe8e4003cd0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.151+0000 7fe8f7e00640 1 -- 172.21.15.83:0/3995396015 shutdown_connections 2024-04-24T09:14:29.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.151+0000 7fe8f7e00640 1 --2- 172.21.15.83:0/3995396015 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe8dc0a3da0 0x7fe8dc0a6260 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.151+0000 7fe8f7e00640 1 --2- 172.21.15.83:0/3995396015 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe8f819a720 0x7fe8f819aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.151+0000 7fe8f7e00640 1 --2- 172.21.15.83:0/3995396015 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe8f81093f0 0x7fe8f81a0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.151+0000 7fe8f7e00640 1 --2- 172.21.15.83:0/3995396015 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8f8104320 0x7fe8f81a0660 secure :-1 s=CLOSED pgs=2664 cs=0 l=1 rev1=1 crypto rx=0x7fe8e402a9e0 tx=0x7fe8e4003cd0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.151+0000 7fe8f7e00640 1 -- 172.21.15.83:0/3995396015 >> 172.21.15.83:0/3995396015 conn(0x7fe8f80ffb40 msgr2=0x7fe8f8100e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:14:29.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.151+0000 7fe8f7e00640 1 -- 172.21.15.83:0/3995396015 shutdown_connections 2024-04-24T09:14:29.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.151+0000 7fe8f7e00640 1 -- 172.21.15.83:0/3995396015 wait complete. 2024-04-24T09:14:29.175 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T09:14:29.205 INFO:tasks.workunit.client.0.smithi083.stdout: 10 66723 6.56 MB/sec execute 420 sec latency 113.685 ms 2024-04-24T09:14:29.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.440+0000 7f0142e00640 1 -- 172.21.15.83:0/810732319 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f013c101770 msgr2=0x7f013c105c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:29.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.440+0000 7f0142e00640 1 --2- 172.21.15.83:0/810732319 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f013c101770 0x7f013c105c30 secure :-1 s=READY pgs=2667 cs=0 l=1 rev1=1 crypto rx=0x7f0128009900 tx=0x7f012802a470 comp rx=0 tx=0).stop 2024-04-24T09:14:29.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.440+0000 7f0142e00640 1 -- 172.21.15.83:0/810732319 shutdown_connections 2024-04-24T09:14:29.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.440+0000 7f0142e00640 1 --2- 172.21.15.83:0/810732319 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f013c106250 0x7f013c1066e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.440+0000 7f0142e00640 1 --2- 172.21.15.83:0/810732319 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f013c101770 0x7f013c105c30 unknown :-1 s=CLOSED pgs=2667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.440+0000 7f0142e00640 1 --2- 172.21.15.83:0/810732319 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f013c100e50 0x7f013c101230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.440+0000 7f0142e00640 1 -- 172.21.15.83:0/810732319 >> 172.21.15.83:0/810732319 conn(0x7f013c0fc880 msgr2=0x7f013c0feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:14:29.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.440+0000 7f0142e00640 1 -- 172.21.15.83:0/810732319 shutdown_connections 2024-04-24T09:14:29.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.440+0000 7f0142e00640 1 -- 172.21.15.83:0/810732319 wait complete. 2024-04-24T09:14:29.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.441+0000 7f0142e00640 1 Processor -- start 2024-04-24T09:14:29.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.441+0000 7f0142e00640 1 -- start start 2024-04-24T09:14:29.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.442+0000 7f0142e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f013c100e50 0x7f013c1a4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:29.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.442+0000 7f0142e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f013c101770 0x7f013c1a4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:29.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.442+0000 7f0142e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f013c106250 0x7f013c19eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:29.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.442+0000 7f0142e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f013c118750 con 0x7f013c106250 2024-04-24T09:14:29.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.442+0000 7f0142e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f013c1185d0 con 0x7f013c100e50 2024-04-24T09:14:29.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.442+0000 7f0142e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f013c1188d0 con 0x7f013c101770 2024-04-24T09:14:29.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.442+0000 7f0142400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f013c106250 0x7f013c19eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:29.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.442+0000 7f0142400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f013c106250 0x7f013c19eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37050/0 (socket says 172.21.15.83:37050) 2024-04-24T09:14:29.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.442+0000 7f0142400640 1 -- 172.21.15.83:0/3962333887 learned_addr learned my addr 172.21.15.83:0/3962333887 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:14:29.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.442+0000 7f0141a00640 1 --2- 172.21.15.83:0/3962333887 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f013c100e50 0x7f013c1a4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:29.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.443+0000 7f0141000640 1 --2- 172.21.15.83:0/3962333887 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f013c101770 0x7f013c1a4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:29.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.443+0000 7f0142400640 1 -- 172.21.15.83:0/3962333887 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f013c101770 msgr2=0x7f013c1a4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:29.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.443+0000 7f0142400640 1 --2- 172.21.15.83:0/3962333887 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f013c101770 0x7f013c1a4e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.443+0000 7f0142400640 1 -- 172.21.15.83:0/3962333887 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f013c100e50 msgr2=0x7f013c1a4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:29.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.443+0000 7f0142400640 1 --2- 172.21.15.83:0/3962333887 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f013c100e50 0x7f013c1a4940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.443+0000 7f0142400640 1 -- 172.21.15.83:0/3962333887 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f013c19f270 con 0x7f013c106250 2024-04-24T09:14:29.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.443+0000 7f0142400640 1 --2- 172.21.15.83:0/3962333887 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f013c106250 0x7f013c19eaf0 secure :-1 s=READY pgs=2668 cs=0 l=1 rev1=1 crypto rx=0x7f013400cad0 tx=0x7f013400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:14:29.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.444+0000 7f013a000640 1 -- 172.21.15.83:0/3962333887 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f013400ae20 con 0x7f013c106250 2024-04-24T09:14:29.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.444+0000 7f013a000640 1 -- 172.21.15.83:0/3962333887 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0134005020 con 0x7f013c106250 2024-04-24T09:14:29.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.444+0000 7f013a000640 1 -- 172.21.15.83:0/3962333887 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f013400f6b0 con 0x7f013c106250 2024-04-24T09:14:29.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.444+0000 7f0142e00640 1 -- 172.21.15.83:0/3962333887 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f013c19f560 con 0x7f013c106250 2024-04-24T09:14:29.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.444+0000 7f0142e00640 1 -- 172.21.15.83:0/3962333887 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f013c1ab800 con 0x7f013c106250 2024-04-24T09:14:29.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.448+0000 7f014725d740 1 Processor -- start 2024-04-24T09:14:29.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.450+0000 7f013a000640 1 -- 172.21.15.83:0/3962333887 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f01340051d0 con 0x7f013c106250 2024-04-24T09:14:29.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.450+0000 7f014725d740 1 -- start start 2024-04-24T09:14:29.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.451+0000 7f014725d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56338765eb40 0x56338765ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:29.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.451+0000 7f014725d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56338765f460 0x5633876695f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:29.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.451+0000 7f014725d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563387669b30 0x56338766bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:29.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.451+0000 7f014725d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5633875c41d0 con 0x56338765f460 2024-04-24T09:14:29.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.451+0000 7f014725d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56338756a450 con 0x56338765eb40 2024-04-24T09:14:29.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.451+0000 7f014725d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5633875c0190 con 0x563387669b30 2024-04-24T09:14:29.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.451+0000 7f0141000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56338765eb40 0x56338765ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:29.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.451+0000 7f0141a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56338765f460 0x5633876695f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:29.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.452+0000 7f0141a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56338765f460 0x5633876695f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37058/0 (socket says 172.21.15.83:37058) 2024-04-24T09:14:29.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.452+0000 7f0141a00640 1 -- 172.21.15.83:0/887983873 learned_addr learned my addr 172.21.15.83:0/887983873 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:14:29.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.452+0000 7f0141000640 1 --2- 172.21.15.83:0/887983873 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563387669b30 0x56338766bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:29.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.452+0000 7f013a000640 1 --2- 172.21.15.83:0/3962333887 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f01240a3e20 0x7f01240a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:29.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.452+0000 7f0141a00640 1 -- 172.21.15.83:0/887983873 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563387669b30 msgr2=0x56338766bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:14:29.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.452+0000 7f0141a00640 1 --2- 172.21.15.83:0/887983873 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563387669b30 0x56338766bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.452+0000 7f0141a00640 1 -- 172.21.15.83:0/887983873 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56338765eb40 msgr2=0x56338765ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:14:29.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.452+0000 7f0141a00640 1 --2- 172.21.15.83:0/887983873 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56338765eb40 0x56338765ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.452+0000 7f0141a00640 1 -- 172.21.15.83:0/887983873 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563387627310 con 0x56338765f460 2024-04-24T09:14:29.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.452+0000 7f013a000640 1 -- 172.21.15.83:0/3962333887 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9583+0+0 (secure 0 0 0) 0x7f01340d0960 con 0x7f013c106250 2024-04-24T09:14:29.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.453+0000 7f0142400640 1 --2- 172.21.15.83:0/3962333887 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f01240a3e20 0x7f01240a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:29.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.453+0000 7f0141a00640 1 --2- 172.21.15.83:0/887983873 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56338765f460 0x5633876695f0 secure :-1 s=READY pgs=2669 cs=0 l=1 rev1=1 crypto rx=0x7f0130009c80 tx=0x7f01300117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=278e8a405ff3625f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:14:29.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.453+0000 7f012ea00640 1 -- 172.21.15.83:0/887983873 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0130012450 con 0x56338765f460 2024-04-24T09:14:29.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.453+0000 7f012ea00640 1 -- 172.21.15.83:0/887983873 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0130012aa0 con 0x56338765f460 2024-04-24T09:14:29.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.454+0000 7f012ea00640 1 -- 172.21.15.83:0/887983873 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f013000fb00 con 0x56338765f460 2024-04-24T09:14:29.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.454+0000 7f014725d740 1 -- 172.21.15.83:0/887983873 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56338765f460 msgr2=0x5633876695f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:29.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.454+0000 7f014725d740 1 --2- 172.21.15.83:0/887983873 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56338765f460 0x5633876695f0 secure :-1 s=READY pgs=2669 cs=0 l=1 rev1=1 crypto rx=0x7f0130009c80 tx=0x7f01300117b0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.455+0000 7f0142400640 1 --2- 172.21.15.83:0/3962333887 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f01240a3e20 0x7f01240a62e0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f013400a920 tx=0x7f013400a890 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:14:29.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.455+0000 7f014725d740 1 -- 172.21.15.83:0/887983873 shutdown_connections 2024-04-24T09:14:29.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.455+0000 7f014725d740 1 --2- 172.21.15.83:0/887983873 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563387669b30 0x56338766bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.455+0000 7f014725d740 1 --2- 172.21.15.83:0/887983873 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56338765f460 0x5633876695f0 unknown :-1 s=CLOSED pgs=2669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.455+0000 7f014725d740 1 --2- 172.21.15.83:0/887983873 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56338765eb40 0x56338765ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.455+0000 7f014725d740 1 -- 172.21.15.83:0/887983873 >> 172.21.15.83:0/887983873 conn(0x56338765ae60 msgr2=0x56338765b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:14:29.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.455+0000 7f014725d740 1 -- 172.21.15.83:0/887983873 shutdown_connections 2024-04-24T09:14:29.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.455+0000 7f014725d740 1 -- 172.21.15.83:0/887983873 wait complete. 2024-04-24T09:14:29.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.456+0000 7f014725d740 1 -- start start 2024-04-24T09:14:29.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.456+0000 7f014725d740 1 Processor -- start 2024-04-24T09:14:29.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.457+0000 7f014725d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56338765eb40 0x563387669540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:29.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.457+0000 7f014725d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56338765f460 0x563387666830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:29.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.457+0000 7f014725d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563387669b30 0x563387666e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:29.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.457+0000 7f014725d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56338766ca10 con 0x56338765f460 2024-04-24T09:14:29.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.457+0000 7f014725d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56338757d810 con 0x563387669b30 2024-04-24T09:14:29.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.457+0000 7f014725d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56338766cb90 con 0x56338765eb40 2024-04-24T09:14:29.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.457+0000 7f014725d740 10 client.0 _ll_register_callbacks cb 0x56338754b140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:14:29.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.458+0000 7f0141000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563387669b30 0x563387666e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:29.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.458+0000 7f0141a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56338765f460 0x563387666830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:29.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.458+0000 7f0141a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56338765f460 0x563387666830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37066/0 (socket says 172.21.15.83:37066) 2024-04-24T09:14:29.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.458+0000 7f0141a00640 1 -- 172.21.15.83:0/1897696850 learned_addr learned my addr 172.21.15.83:0/1897696850 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:14:29.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.458+0000 7f0141000640 1 --2- 172.21.15.83:0/1897696850 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56338765eb40 0x563387669540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:29.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.458+0000 7f0141a00640 1 -- 172.21.15.83:0/1897696850 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56338765eb40 msgr2=0x563387669540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:14:29.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.458+0000 7f0141a00640 1 --2- 172.21.15.83:0/1897696850 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56338765eb40 0x563387669540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.458+0000 7f0141a00640 1 -- 172.21.15.83:0/1897696850 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563387669b30 msgr2=0x563387666e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:14:29.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.458+0000 7f0141a00640 1 --2- 172.21.15.83:0/1897696850 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563387669b30 0x563387666e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.458+0000 7f0141a00640 1 -- 172.21.15.83:0/1897696850 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56338758e3f0 con 0x56338765f460 2024-04-24T09:14:29.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.459+0000 7f0123400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:14:29.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.459+0000 7f0141a00640 1 --2- 172.21.15.83:0/1897696850 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56338765f460 0x563387666830 secure :-1 s=READY pgs=2670 cs=0 l=1 rev1=1 crypto rx=0x7f0130009790 tx=0x7f0130009d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b46a9502b1190d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:14:29.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.459+0000 7f0123400640 1 -- 172.21.15.83:0/1897696850 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0130028020 con 0x56338765f460 2024-04-24T09:14:29.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.459+0000 7f0123400640 1 -- 172.21.15.83:0/1897696850 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(128..128 src has 1..128) ==== 9583+0+0 (secure 0 0 0) 0x7f0130004310 con 0x56338765f460 2024-04-24T09:14:29.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.460+0000 7f014725d740 1 -- 172.21.15.83:0/1897696850 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0130012aa0 con 0x56338765f460 2024-04-24T09:14:29.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.460+0000 7f0123400640 1 -- 172.21.15.83:0/1897696850 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f0130028020 con 0x56338765f460 2024-04-24T09:14:29.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.460+0000 7f014725d740 10 client.23346 fetch_fsmap learned FSMap version 65 2024-04-24T09:14:29.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.460+0000 7f014725d740 1 -- 172.21.15.83:0/1897696850 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x5633872d63f0 con 0x56338765f460 2024-04-24T09:14:29.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.461+0000 7f0123400640 1 -- 172.21.15.83:0/1897696850 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0130012aa0 con 0x56338765f460 2024-04-24T09:14:29.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.461+0000 7f0123400640 1 -- 172.21.15.83:0/1897696850 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7f013000fcd0 con 0x56338765f460 2024-04-24T09:14:29.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.461+0000 7f014725d740 10 client.23346 fetch_fsmap finished waiting for FSMap version 65 2024-04-24T09:14:29.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.461+0000 7f014725d740 10 client.23346 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-24T09:14:29.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.461+0000 7f014725d740 20 client.23346 populate_metadata read hostname 'smithi083' 2024-04-24T09:14:29.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.461+0000 7f014725d740 1 --2- 172.21.15.83:0/1897696850 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x563387679c80 0x56338767c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:29.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.461+0000 7f014725d740 4 client.23346 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:14:29.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.461+0000 7f014725d740 1 -- 172.21.15.83:0/1897696850 --> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5633875d1bf0 con 0x563387679c80 2024-04-24T09:14:29.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.462+0000 7f0141000640 1 --2- 172.21.15.83:0/1897696850 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x563387679c80 0x56338767c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:29.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.463+0000 7f0141000640 1 --2- 172.21.15.83:0/1897696850 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x563387679c80 0x56338767c070 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4d14d293efc7f072 server_cookie=abfbcf5511e52b2c in_seq=0 out_seq=0 2024-04-24T09:14:29.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.463+0000 7f0123400640 10 client.23346 ms_handle_connect on v2:172.21.15.138:6834/1909219758 2024-04-24T09:14:29.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.464+0000 7f0123400640 1 -- 172.21.15.83:0/1897696850 <== mds.2 v2:172.21.15.138:6834/1909219758 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5633875d1bf0 con 0x563387679c80 2024-04-24T09:14:29.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.464+0000 7f0123400640 10 client.23346 handle_command_reply: tid=0 2024-04-24T09:14:29.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.464+0000 7f014725d740 1 client.23346 shutdown 2024-04-24T09:14:29.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.464+0000 7f014725d740 1 -- 172.21.15.83:0/1897696850 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56338765f460 msgr2=0x563387666830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:29.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.464+0000 7f014725d740 1 --2- 172.21.15.83:0/1897696850 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56338765f460 0x563387666830 secure :-1 s=READY pgs=2670 cs=0 l=1 rev1=1 crypto rx=0x7f0130009790 tx=0x7f0130009d70 comp rx=0 tx=0).stop 2024-04-24T09:14:29.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.464+0000 7f014725d740 1 -- 172.21.15.83:0/1897696850 shutdown_connections 2024-04-24T09:14:29.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.464+0000 7f014725d740 1 --2- 172.21.15.83:0/1897696850 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x563387679c80 0x56338767c070 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.464+0000 7f014725d740 1 --2- 172.21.15.83:0/1897696850 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563387669b30 0x563387666e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.464+0000 7f014725d740 1 --2- 172.21.15.83:0/1897696850 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56338765f460 0x563387666830 unknown :-1 s=CLOSED pgs=2670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.464+0000 7f014725d740 1 --2- 172.21.15.83:0/1897696850 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56338765eb40 0x563387669540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.464+0000 7f014725d740 1 -- 172.21.15.83:0/1897696850 >> 172.21.15.83:0/1897696850 conn(0x56338765ae60 msgr2=0x563387660b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:14:29.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.464+0000 7f0123400640 0 client.23346 ms_handle_reset on v2:172.21.15.138:6834/1909219758 2024-04-24T09:14:29.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.465+0000 7f014725d740 1 -- 172.21.15.83:0/1897696850 shutdown_connections 2024-04-24T09:14:29.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.465+0000 7f014725d740 1 -- 172.21.15.83:0/1897696850 wait complete. 2024-04-24T09:14:29.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.465+0000 7f014725d740 20 client.23346 trim_cache size 0 max 16384 2024-04-24T09:14:29.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.490+0000 7f014725d740 1 Processor -- start 2024-04-24T09:14:29.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.491+0000 7f014725d740 1 -- start start 2024-04-24T09:14:29.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.491+0000 7f014725d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56338765eb40 0x5633876758d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:29.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.491+0000 7f014725d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563387669b30 0x563387659dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:29.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.491+0000 7f014725d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563387669070 0x5633876694b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:29.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.491+0000 7f014725d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f013c1b5be0 con 0x56338765eb40 2024-04-24T09:14:29.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.491+0000 7f014725d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f013c1b7780 con 0x563387669070 2024-04-24T09:14:29.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.491+0000 7f014725d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f013c1b62b0 con 0x563387669b30 2024-04-24T09:14:29.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.492+0000 7f0141000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56338765eb40 0x5633876758d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:29.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.492+0000 7f0141000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56338765eb40 0x5633876758d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37074/0 (socket says 172.21.15.83:37074) 2024-04-24T09:14:29.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.492+0000 7f0141000640 1 -- 172.21.15.83:0/2660985024 learned_addr learned my addr 172.21.15.83:0/2660985024 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:14:29.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.492+0000 7f0141000640 1 --2- 172.21.15.83:0/2660985024 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563387669070 0x5633876694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:29.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.493+0000 7f0141a00640 1 --2- 172.21.15.83:0/2660985024 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563387669b30 0x563387659dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:29.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.493+0000 7f0141000640 1 -- 172.21.15.83:0/2660985024 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563387669b30 msgr2=0x563387659dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:14:29.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.493+0000 7f0141000640 1 --2- 172.21.15.83:0/2660985024 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563387669b30 0x563387659dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.493+0000 7f0141000640 1 -- 172.21.15.83:0/2660985024 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563387669070 msgr2=0x5633876694b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:14:29.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.493+0000 7f0141000640 1 --2- 172.21.15.83:0/2660985024 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563387669070 0x5633876694b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.493+0000 7f0141000640 1 -- 172.21.15.83:0/2660985024 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633873a07d0 con 0x56338765eb40 2024-04-24T09:14:29.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.493+0000 7f0141000640 1 --2- 172.21.15.83:0/2660985024 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56338765eb40 0x5633876758d0 secure :-1 s=READY pgs=2671 cs=0 l=1 rev1=1 crypto rx=0x7f0128009230 tx=0x7f0128031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=480c69ceadbc91b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:14:29.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.494+0000 7f012e000640 1 -- 172.21.15.83:0/2660985024 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0128035040 con 0x56338765eb40 2024-04-24T09:14:29.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.494+0000 7f012e000640 1 -- 172.21.15.83:0/2660985024 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f01280054e0 con 0x56338765eb40 2024-04-24T09:14:29.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.494+0000 7f012e000640 1 -- 172.21.15.83:0/2660985024 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0128003db0 con 0x56338765eb40 2024-04-24T09:14:29.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.495+0000 7f014725d740 1 -- 172.21.15.83:0/2660985024 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56338765eb40 msgr2=0x5633876758d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:29.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.495+0000 7f014725d740 1 --2- 172.21.15.83:0/2660985024 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56338765eb40 0x5633876758d0 secure :-1 s=READY pgs=2671 cs=0 l=1 rev1=1 crypto rx=0x7f0128009230 tx=0x7f0128031400 comp rx=0 tx=0).stop 2024-04-24T09:14:29.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.495+0000 7f014725d740 1 -- 172.21.15.83:0/2660985024 shutdown_connections 2024-04-24T09:14:29.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.495+0000 7f014725d740 1 --2- 172.21.15.83:0/2660985024 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563387669070 0x5633876694b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.495+0000 7f014725d740 1 --2- 172.21.15.83:0/2660985024 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563387669b30 0x563387659dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.495+0000 7f014725d740 1 --2- 172.21.15.83:0/2660985024 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56338765eb40 0x5633876758d0 unknown :-1 s=CLOSED pgs=2671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.495+0000 7f014725d740 1 -- 172.21.15.83:0/2660985024 >> 172.21.15.83:0/2660985024 conn(0x56338765ae60 msgr2=0x5633876596c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:14:29.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.495+0000 7f014725d740 1 -- 172.21.15.83:0/2660985024 shutdown_connections 2024-04-24T09:14:29.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.495+0000 7f014725d740 1 -- 172.21.15.83:0/2660985024 wait complete. 2024-04-24T09:14:29.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.496+0000 7f014725d740 1 -- start start 2024-04-24T09:14:29.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.496+0000 7f014725d740 1 Processor -- start 2024-04-24T09:14:29.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.497+0000 7f014725d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56338765eb40 0x56338766a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:29.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.497+0000 7f014725d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563387669070 0x56338766ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:29.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.497+0000 7f014725d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563387669b30 0x56338766b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:29.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.497+0000 7f014725d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f013c1b7080 con 0x56338765eb40 2024-04-24T09:14:29.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.497+0000 7f014725d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f013c1b69b0 con 0x563387669070 2024-04-24T09:14:29.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.497+0000 7f014725d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f013c1b3be0 con 0x563387669b30 2024-04-24T09:14:29.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.497+0000 7f014725d740 10 client.0 _ll_register_callbacks cb 0x7f013c06a1f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:14:29.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.497+0000 7f0141000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56338765eb40 0x56338766a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:29.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.497+0000 7f0141000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56338765eb40 0x56338766a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37076/0 (socket says 172.21.15.83:37076) 2024-04-24T09:14:29.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.497+0000 7f0141a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563387669070 0x56338766ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:29.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.497+0000 7f0141000640 1 -- 172.21.15.83:0/3993514290 learned_addr learned my addr 172.21.15.83:0/3993514290 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:14:29.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.498+0000 7f0141000640 1 -- 172.21.15.83:0/3993514290 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563387669b30 msgr2=0x56338766b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:14:29.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.498+0000 7f0141000640 1 --2- 172.21.15.83:0/3993514290 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563387669b30 0x56338766b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.498+0000 7f0141000640 1 -- 172.21.15.83:0/3993514290 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563387669070 msgr2=0x56338766ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:14:29.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.498+0000 7f0141000640 1 --2- 172.21.15.83:0/3993514290 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563387669070 0x56338766ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.498+0000 7f0141000640 1 -- 172.21.15.83:0/3993514290 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563387670080 con 0x56338765eb40 2024-04-24T09:14:29.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.498+0000 7f0141000640 1 --2- 172.21.15.83:0/3993514290 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56338765eb40 0x56338766a830 secure :-1 s=READY pgs=2672 cs=0 l=1 rev1=1 crypto rx=0x7f0128003a90 tx=0x7f012804d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f751447e13b1382 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:14:29.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.498+0000 7f0123400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:14:29.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.498+0000 7f0123400640 1 -- 172.21.15.83:0/3993514290 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0128035040 con 0x56338765eb40 2024-04-24T09:14:29.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.498+0000 7f0123400640 1 -- 172.21.15.83:0/3993514290 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(128..128 src has 1..128) ==== 9583+0+0 (secure 0 0 0) 0x7f0128003d30 con 0x56338765eb40 2024-04-24T09:14:29.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.498+0000 7f014725d740 1 -- 172.21.15.83:0/3993514290 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01280054e0 con 0x56338765eb40 2024-04-24T09:14:29.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.499+0000 7f0123400640 1 -- 172.21.15.83:0/3993514290 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f012805f050 con 0x56338765eb40 2024-04-24T09:14:29.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.499+0000 7f014725d740 10 client.23352 fetch_fsmap learned FSMap version 65 2024-04-24T09:14:29.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.499+0000 7f014725d740 1 -- 172.21.15.83:0/3993514290 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x5633876e0280 con 0x56338765eb40 2024-04-24T09:14:29.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.499+0000 7f0123400640 1 -- 172.21.15.83:0/3993514290 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f01280054e0 con 0x56338765eb40 2024-04-24T09:14:29.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.499+0000 7f0123400640 1 -- 172.21.15.83:0/3993514290 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7f0128055030 con 0x56338765eb40 2024-04-24T09:14:29.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.500+0000 7f014725d740 10 client.23352 fetch_fsmap finished waiting for FSMap version 65 2024-04-24T09:14:29.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.500+0000 7f014725d740 10 client.23352 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-24T09:14:29.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.500+0000 7f014725d740 20 client.23352 populate_metadata read hostname 'smithi083' 2024-04-24T09:14:29.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.500+0000 7f014725d740 1 --2- 172.21.15.83:0/3993514290 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x5633876e08f0 0x5633876e2d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:29.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.500+0000 7f014725d740 4 client.23352 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:14:29.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.500+0000 7f014725d740 1 -- 172.21.15.83:0/3993514290 --> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5633875cba60 con 0x5633876e08f0 2024-04-24T09:14:29.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.500+0000 7f0141a00640 1 --2- 172.21.15.83:0/3993514290 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x5633876e08f0 0x5633876e2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:29.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.501+0000 7f0123400640 10 client.23352 ms_handle_connect on v2:172.21.15.138:6834/1909219758 2024-04-24T09:14:29.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.501+0000 7f0141a00640 1 --2- 172.21.15.83:0/3993514290 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x5633876e08f0 0x5633876e2d10 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=18c30798f543d847 server_cookie=d255342e6658492e in_seq=0 out_seq=0 2024-04-24T09:14:29.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.502+0000 7f0123400640 1 -- 172.21.15.83:0/3993514290 <== mds.2 v2:172.21.15.138:6834/1909219758 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5633875cba60 con 0x5633876e08f0 2024-04-24T09:14:29.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.502+0000 7f0123400640 10 client.23352 handle_command_reply: tid=0 2024-04-24T09:14:29.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.502+0000 7f014725d740 1 client.23352 shutdown 2024-04-24T09:14:29.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.503+0000 7f014725d740 1 -- 172.21.15.83:0/3993514290 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56338765eb40 msgr2=0x56338766a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:29.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.503+0000 7f014725d740 1 --2- 172.21.15.83:0/3993514290 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56338765eb40 0x56338766a830 secure :-1 s=READY pgs=2672 cs=0 l=1 rev1=1 crypto rx=0x7f0128003a90 tx=0x7f012804d520 comp rx=0 tx=0).stop 2024-04-24T09:14:29.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.503+0000 7f014725d740 1 -- 172.21.15.83:0/3993514290 shutdown_connections 2024-04-24T09:14:29.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.503+0000 7f014725d740 1 --2- 172.21.15.83:0/3993514290 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x5633876e08f0 0x5633876e2d10 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.503+0000 7f014725d740 1 --2- 172.21.15.83:0/3993514290 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563387669b30 0x56338766b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.503+0000 7f014725d740 1 --2- 172.21.15.83:0/3993514290 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563387669070 0x56338766ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.503+0000 7f014725d740 1 --2- 172.21.15.83:0/3993514290 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56338765eb40 0x56338766a830 unknown :-1 s=CLOSED pgs=2672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.503+0000 7f014725d740 1 -- 172.21.15.83:0/3993514290 >> 172.21.15.83:0/3993514290 conn(0x56338765ae60 msgr2=0x5633876688f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:14:29.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.503+0000 7f0123400640 0 client.23352 ms_handle_reset on v2:172.21.15.138:6834/1909219758 2024-04-24T09:14:29.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.503+0000 7f014725d740 1 -- 172.21.15.83:0/3993514290 shutdown_connections 2024-04-24T09:14:29.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.503+0000 7f014725d740 1 -- 172.21.15.83:0/3993514290 wait complete. 2024-04-24T09:14:29.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.503+0000 7f014725d740 20 client.23352 trim_cache size 0 max 16384 2024-04-24T09:14:29.504 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:14:29.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.504+0000 7f0142e00640 1 -- 172.21.15.83:0/3962333887 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f01240a3e20 msgr2=0x7f01240a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:29.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.504+0000 7f0142e00640 1 --2- 172.21.15.83:0/3962333887 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f01240a3e20 0x7f01240a62e0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f013400a920 tx=0x7f013400a890 comp rx=0 tx=0).stop 2024-04-24T09:14:29.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.504+0000 7f0142e00640 1 -- 172.21.15.83:0/3962333887 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f013c106250 msgr2=0x7f013c19eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:29.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.504+0000 7f0142e00640 1 --2- 172.21.15.83:0/3962333887 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f013c106250 0x7f013c19eaf0 secure :-1 s=READY pgs=2668 cs=0 l=1 rev1=1 crypto rx=0x7f013400cad0 tx=0x7f013400cf90 comp rx=0 tx=0).stop 2024-04-24T09:14:29.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.504+0000 7f0142e00640 1 -- 172.21.15.83:0/3962333887 shutdown_connections 2024-04-24T09:14:29.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.504+0000 7f0142e00640 1 --2- 172.21.15.83:0/3962333887 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f01240a3e20 0x7f01240a62e0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.504+0000 7f0142e00640 1 --2- 172.21.15.83:0/3962333887 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f013c106250 0x7f013c19eaf0 unknown :-1 s=CLOSED pgs=2668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.504+0000 7f0142e00640 1 --2- 172.21.15.83:0/3962333887 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f013c101770 0x7f013c1a4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.504+0000 7f0142e00640 1 --2- 172.21.15.83:0/3962333887 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f013c100e50 0x7f013c1a4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.504+0000 7f0142e00640 1 -- 172.21.15.83:0/3962333887 >> 172.21.15.83:0/3962333887 conn(0x7f013c0fc880 msgr2=0x7f013c0fd920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:14:29.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.505+0000 7f0142e00640 1 -- 172.21.15.83:0/3962333887 shutdown_connections 2024-04-24T09:14:29.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.505+0000 7f0142e00640 1 -- 172.21.15.83:0/3962333887 wait complete. 2024-04-24T09:14:29.523 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:14:29 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/1656776448' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:14:29.527 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T09:14:29.699 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.699+0000 7faed4600640 1 -- 172.21.15.83:0/2168715448 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faecc107090 msgr2=0x7faecc107470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:29.699 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.699+0000 7faed4600640 1 --2- 172.21.15.83:0/2168715448 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faecc107090 0x7faecc107470 secure :-1 s=READY pgs=2673 cs=0 l=1 rev1=1 crypto rx=0x7faec0009930 tx=0x7faec002a420 comp rx=0 tx=0).stop 2024-04-24T09:14:29.699 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.699+0000 7faed4600640 1 -- 172.21.15.83:0/2168715448 shutdown_connections 2024-04-24T09:14:29.699 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.699+0000 7faed4600640 1 --2- 172.21.15.83:0/2168715448 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faecc101bf0 0x7faecc10dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.699 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.699+0000 7faed4600640 1 --2- 172.21.15.83:0/2168715448 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faecc101250 0x7faecc1016b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.699 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.699+0000 7faed4600640 1 --2- 172.21.15.83:0/2168715448 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faecc107090 0x7faecc107470 unknown :-1 s=CLOSED pgs=2673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.699 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.699+0000 7faed4600640 1 -- 172.21.15.83:0/2168715448 >> 172.21.15.83:0/2168715448 conn(0x7faecc076c30 msgr2=0x7faecc077040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:14:29.699 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.700+0000 7faed4600640 1 -- 172.21.15.83:0/2168715448 shutdown_connections 2024-04-24T09:14:29.699 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.700+0000 7faed4600640 1 -- 172.21.15.83:0/2168715448 wait complete. 2024-04-24T09:14:29.700 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.701+0000 7faed4600640 1 Processor -- start 2024-04-24T09:14:29.700 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.701+0000 7faed4600640 1 -- start start 2024-04-24T09:14:29.701 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.702+0000 7faed4600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faecc101250 0x7faecc1a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:29.701 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.702+0000 7faed4600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faecc101bf0 0x7faecc1a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:29.701 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.702+0000 7faed4600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faecc107090 0x7faecc19a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:29.701 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.702+0000 7faed4600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7faecc110120 con 0x7faecc107090 2024-04-24T09:14:29.701 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.702+0000 7faed4600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7faecc10ffa0 con 0x7faecc101bf0 2024-04-24T09:14:29.702 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.702+0000 7faed4600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7faecc1102a0 con 0x7faecc101250 2024-04-24T09:14:29.702 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.703+0000 7faed2600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faecc107090 0x7faecc19a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:29.702 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.703+0000 7faed1c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faecc101250 0x7faecc1a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:29.702 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.703+0000 7faed2600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faecc107090 0x7faecc19a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37092/0 (socket says 172.21.15.83:37092) 2024-04-24T09:14:29.702 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.703+0000 7faed2600640 1 -- 172.21.15.83:0/551712868 learned_addr learned my addr 172.21.15.83:0/551712868 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:14:29.702 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.703+0000 7faed1200640 1 --2- 172.21.15.83:0/551712868 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faecc101bf0 0x7faecc1a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:29.703 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.704+0000 7faed2600640 1 -- 172.21.15.83:0/551712868 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faecc101250 msgr2=0x7faecc1a06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:29.703 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.704+0000 7faed2600640 1 --2- 172.21.15.83:0/551712868 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faecc101250 0x7faecc1a06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.703 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.704+0000 7faed2600640 1 -- 172.21.15.83:0/551712868 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faecc101bf0 msgr2=0x7faecc1a0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:29.703 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.704+0000 7faed2600640 1 --2- 172.21.15.83:0/551712868 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faecc101bf0 0x7faecc1a0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.703 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.704+0000 7faed2600640 1 -- 172.21.15.83:0/551712868 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faecc19b030 con 0x7faecc107090 2024-04-24T09:14:29.703 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.704+0000 7faed1c00640 1 --2- 172.21.15.83:0/551712868 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faecc101250 0x7faecc1a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:14:29.704 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.704+0000 7faed2600640 1 --2- 172.21.15.83:0/551712868 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faecc107090 0x7faecc19a770 secure :-1 s=READY pgs=2674 cs=0 l=1 rev1=1 crypto rx=0x7faec400cad0 tx=0x7faec400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:14:29.704 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.705+0000 7faeca000640 1 -- 172.21.15.83:0/551712868 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec400ae20 con 0x7faecc107090 2024-04-24T09:14:29.704 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.705+0000 7faeca000640 1 -- 172.21.15.83:0/551712868 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faec4005020 con 0x7faecc107090 2024-04-24T09:14:29.704 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.705+0000 7faeca000640 1 -- 172.21.15.83:0/551712868 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec400f6b0 con 0x7faecc107090 2024-04-24T09:14:29.704 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.705+0000 7faed4600640 1 -- 172.21.15.83:0/551712868 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faecc19b2c0 con 0x7faecc107090 2024-04-24T09:14:29.705 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.705+0000 7faed4600640 1 -- 172.21.15.83:0/551712868 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faecc196290 con 0x7faecc107090 2024-04-24T09:14:29.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:14:29 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/1656776448' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:14:29.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:14:29 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/1656776448' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:14:29.711 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.711+0000 7faed738f740 1 Processor -- start 2024-04-24T09:14:29.711 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.711+0000 7faeca000640 1 -- 172.21.15.83:0/551712868 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7faec40051d0 con 0x7faecc107090 2024-04-24T09:14:29.711 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.712+0000 7faed738f740 1 -- start start 2024-04-24T09:14:29.711 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.712+0000 7faed738f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a4587f1b0 0x560a4587f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:29.711 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.712+0000 7faed738f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560a4587fad0 0x560a45889e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:29.711 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.712+0000 7faed738f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560a4588a390 0x560a4588c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:29.712 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.712+0000 7faed738f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x560a457e5900 con 0x560a4587f1b0 2024-04-24T09:14:29.712 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.712+0000 7faed738f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560a4578bf20 con 0x560a4588a390 2024-04-24T09:14:29.712 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.712+0000 7faed738f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x560a457e18c0 con 0x560a4587fad0 2024-04-24T09:14:29.712 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.713+0000 7faed1c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560a4587fad0 0x560a45889e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:29.712 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.713+0000 7faed1c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560a4587fad0 0x560a45889e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:45120/0 (socket says 172.21.15.83:45120) 2024-04-24T09:14:29.712 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.713+0000 7faed1c00640 1 -- 172.21.15.83:0/2893636948 learned_addr learned my addr 172.21.15.83:0/2893636948 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:14:29.713 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.713+0000 7faed1200640 1 --2- 172.21.15.83:0/2893636948 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a4587f1b0 0x560a4587f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:29.713 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.714+0000 7faed1200640 1 --2- 172.21.15.83:0/2893636948 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560a4588a390 0x560a4588c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:29.713 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.714+0000 7faed1c00640 1 -- 172.21.15.83:0/2893636948 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560a4588a390 msgr2=0x560a4588c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:14:29.713 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.714+0000 7faed1c00640 1 --2- 172.21.15.83:0/2893636948 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560a4588a390 0x560a4588c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.713 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.714+0000 7faed1c00640 1 -- 172.21.15.83:0/2893636948 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a4587f1b0 msgr2=0x560a4587f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:14:29.714 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.714+0000 7faed1c00640 1 --2- 172.21.15.83:0/2893636948 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a4587f1b0 0x560a4587f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.714 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.714+0000 7faed1c00640 1 -- 172.21.15.83:0/2893636948 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a457b0e10 con 0x560a4587fad0 2024-04-24T09:14:29.714 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.715+0000 7faeca000640 1 --2- 172.21.15.83:0/551712868 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7faeb40a3e90 0x7faeb40a6350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:29.714 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.715+0000 7faeca000640 1 -- 172.21.15.83:0/551712868 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9583+0+0 (secure 0 0 0) 0x7faec40d07b0 con 0x7faecc107090 2024-04-24T09:14:29.715 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.715+0000 7faed1c00640 1 --2- 172.21.15.83:0/2893636948 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560a4587fad0 0x560a45889e50 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7faec002aa10 tx=0x7faec0004610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7eb97cf58ce39a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:14:29.715 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.715+0000 7faed1200640 1 --2- 172.21.15.83:0/2893636948 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a4587f1b0 0x560a4587f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:14:29.715 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.716+0000 7faebea00640 1 -- 172.21.15.83:0/2893636948 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec0035040 con 0x560a4587fad0 2024-04-24T09:14:29.715 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.716+0000 7faed1200640 1 --2- 172.21.15.83:0/551712868 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7faeb40a3e90 0x7faeb40a6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:29.715 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.716+0000 7faebea00640 1 -- 172.21.15.83:0/2893636948 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faec0004760 con 0x560a4587fad0 2024-04-24T09:14:29.715 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.716+0000 7faebea00640 1 -- 172.21.15.83:0/2893636948 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec00039c0 con 0x560a4587fad0 2024-04-24T09:14:29.716 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.717+0000 7faed1200640 1 --2- 172.21.15.83:0/551712868 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7faeb40a3e90 0x7faeb40a6350 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x560a4587cae0 tx=0x7faeb8005850 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:14:29.717 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.717+0000 7faed738f740 1 -- 172.21.15.83:0/2893636948 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560a4587fad0 msgr2=0x560a45889e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:29.717 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.717+0000 7faed738f740 1 --2- 172.21.15.83:0/2893636948 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560a4587fad0 0x560a45889e50 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7faec002aa10 tx=0x7faec0004610 comp rx=0 tx=0).stop 2024-04-24T09:14:29.717 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.717+0000 7faed738f740 1 -- 172.21.15.83:0/2893636948 shutdown_connections 2024-04-24T09:14:29.717 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.717+0000 7faed738f740 1 --2- 172.21.15.83:0/2893636948 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560a4588a390 0x560a4588c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.717 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.717+0000 7faed738f740 1 --2- 172.21.15.83:0/2893636948 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560a4587fad0 0x560a45889e50 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.717 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.717+0000 7faed738f740 1 --2- 172.21.15.83:0/2893636948 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a4587f1b0 0x560a4587f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.717 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.717+0000 7faed738f740 1 -- 172.21.15.83:0/2893636948 >> 172.21.15.83:0/2893636948 conn(0x560a4587b6c0 msgr2=0x560a4587baa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:14:29.717 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.717+0000 7faed738f740 1 -- 172.21.15.83:0/2893636948 shutdown_connections 2024-04-24T09:14:29.717 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.718+0000 7faed738f740 1 -- 172.21.15.83:0/2893636948 wait complete. 2024-04-24T09:14:29.718 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.718+0000 7faed738f740 1 -- start start 2024-04-24T09:14:29.718 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.718+0000 7faed738f740 1 Processor -- start 2024-04-24T09:14:29.718 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.719+0000 7faed738f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a4587f1b0 0x560a458872e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:29.718 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.719+0000 7faed738f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560a4587fad0 0x560a45887820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:29.718 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.719+0000 7faed738f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560a4588a390 0x560a45887d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:29.718 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.719+0000 7faed738f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x560a4588cf60 con 0x560a4587f1b0 2024-04-24T09:14:29.719 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.719+0000 7faed738f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560a4579f2c0 con 0x560a4588a390 2024-04-24T09:14:29.719 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.719+0000 7faed738f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x560a4588d0e0 con 0x560a4587fad0 2024-04-24T09:14:29.719 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.719+0000 7faed738f740 10 client.0 _ll_register_callbacks cb 0x560a454f7a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:14:29.720 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.720+0000 7faed1c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a4587f1b0 0x560a458872e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:29.720 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.720+0000 7faed1c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a4587f1b0 0x560a458872e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37112/0 (socket says 172.21.15.83:37112) 2024-04-24T09:14:29.720 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.720+0000 7faed1c00640 1 -- 172.21.15.83:0/1879822749 learned_addr learned my addr 172.21.15.83:0/1879822749 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:14:29.720 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.720+0000 7faed1c00640 1 -- 172.21.15.83:0/1879822749 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560a4587fad0 msgr2=0x560a45887820 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T09:14:29.720 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.720+0000 7faed1c00640 1 --2- 172.21.15.83:0/1879822749 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560a4587fad0 0x560a45887820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.720 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.720+0000 7faed1c00640 1 -- 172.21.15.83:0/1879822749 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560a4588a390 msgr2=0x560a45887d60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-24T09:14:29.720 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.720+0000 7faed1c00640 1 --2- 172.21.15.83:0/1879822749 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560a4588a390 0x560a45887d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.720 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.720+0000 7faed1c00640 1 -- 172.21.15.83:0/1879822749 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a457affd0 con 0x560a4587f1b0 2024-04-24T09:14:29.720 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.720+0000 7faed1c00640 1 --2- 172.21.15.83:0/1879822749 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a4587f1b0 0x560a458872e0 secure :-1 s=READY pgs=2675 cs=0 l=1 rev1=1 crypto rx=0x7faec0009850 tx=0x7faec0005630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a32ceb43278149 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:14:29.721 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.721+0000 7faeb3400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:14:29.721 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.721+0000 7faeb3400640 1 -- 172.21.15.83:0/1879822749 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec0052060 con 0x560a4587f1b0 2024-04-24T09:14:29.721 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.721+0000 7faeb3400640 1 -- 172.21.15.83:0/1879822749 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(128..128 src has 1..128) ==== 9583+0+0 (secure 0 0 0) 0x7faec0004b20 con 0x560a4587f1b0 2024-04-24T09:14:29.721 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.721+0000 7faed738f740 1 -- 172.21.15.83:0/1879822749 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faec0004760 con 0x560a4587f1b0 2024-04-24T09:14:29.721 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.722+0000 7faeb3400640 1 -- 172.21.15.83:0/1879822749 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7faec0059050 con 0x560a4587f1b0 2024-04-24T09:14:29.722 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.722+0000 7faed738f740 10 client.23364 fetch_fsmap learned FSMap version 65 2024-04-24T09:14:29.722 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.722+0000 7faed738f740 1 -- 172.21.15.83:0/1879822749 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x560a454f8210 con 0x560a4587f1b0 2024-04-24T09:14:29.722 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.722+0000 7faeb3400640 1 -- 172.21.15.83:0/1879822749 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faec0004760 con 0x560a4587f1b0 2024-04-24T09:14:29.722 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.723+0000 7faeb3400640 1 -- 172.21.15.83:0/1879822749 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7faec00314a0 con 0x560a4587f1b0 2024-04-24T09:14:29.723 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.723+0000 7faed738f740 10 client.23364 fetch_fsmap finished waiting for FSMap version 65 2024-04-24T09:14:29.723 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.723+0000 7faed738f740 10 client.23364 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:14:29.723 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.723+0000 7faed738f740 20 client.23364 populate_metadata read hostname 'smithi083' 2024-04-24T09:14:29.723 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.723+0000 7faed738f740 1 --2- 172.21.15.83:0/1879822749 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x560a4589a0a0 0x560a4589c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:29.723 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.723+0000 7faed738f740 4 client.23364 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:14:29.723 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.723+0000 7faed738f740 1 -- 172.21.15.83:0/1879822749 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560a457f3320 con 0x560a4589a0a0 2024-04-24T09:14:29.724 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.724+0000 7faed1200640 1 --2- 172.21.15.83:0/1879822749 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x560a4589a0a0 0x560a4589c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:29.724 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.725+0000 7faed1200640 1 --2- 172.21.15.83:0/1879822749 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x560a4589a0a0 0x560a4589c490 crc :-1 s=READY pgs=561 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=adbd2a75056ae439 server_cookie=e8bf26f5183734e in_seq=0 out_seq=0 2024-04-24T09:14:29.724 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.725+0000 7faeb3400640 10 client.23364 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:14:29.726 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.726+0000 7faeb3400640 1 -- 172.21.15.83:0/1879822749 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560a457f3320 con 0x560a4589a0a0 2024-04-24T09:14:29.726 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.726+0000 7faeb3400640 10 client.23364 handle_command_reply: tid=0 2024-04-24T09:14:29.726 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.726+0000 7faed738f740 1 client.23364 shutdown 2024-04-24T09:14:29.726 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.727+0000 7faed738f740 1 -- 172.21.15.83:0/1879822749 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a4587f1b0 msgr2=0x560a458872e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:29.727 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.727+0000 7faed738f740 1 --2- 172.21.15.83:0/1879822749 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a4587f1b0 0x560a458872e0 secure :-1 s=READY pgs=2675 cs=0 l=1 rev1=1 crypto rx=0x7faec0009850 tx=0x7faec0005630 comp rx=0 tx=0).stop 2024-04-24T09:14:29.728 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.727+0000 7faed738f740 1 -- 172.21.15.83:0/1879822749 shutdown_connections 2024-04-24T09:14:29.728 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.727+0000 7faed738f740 1 --2- 172.21.15.83:0/1879822749 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x560a4589a0a0 0x560a4589c490 crc :-1 s=READY pgs=561 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.728 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.727+0000 7faed738f740 1 --2- 172.21.15.83:0/1879822749 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560a4588a390 0x560a45887d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.728 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.727+0000 7faed738f740 1 --2- 172.21.15.83:0/1879822749 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560a4587fad0 0x560a45887820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.729 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.727+0000 7faed738f740 1 --2- 172.21.15.83:0/1879822749 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a4587f1b0 0x560a458872e0 unknown :-1 s=CLOSED pgs=2675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.729 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.727+0000 7faed738f740 1 -- 172.21.15.83:0/1879822749 >> 172.21.15.83:0/1879822749 conn(0x560a4587b6c0 msgr2=0x560a45881440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:14:29.729 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.727+0000 7faeb3400640 0 client.23364 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:14:29.729 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.728+0000 7faed738f740 1 -- 172.21.15.83:0/1879822749 shutdown_connections 2024-04-24T09:14:29.729 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.728+0000 7faed738f740 1 -- 172.21.15.83:0/1879822749 wait complete. 2024-04-24T09:14:29.729 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.728+0000 7faed738f740 20 client.23364 trim_cache size 0 max 16384 2024-04-24T09:14:29.759 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.760+0000 7faed738f740 1 Processor -- start 2024-04-24T09:14:29.759 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.760+0000 7faed738f740 1 -- start start 2024-04-24T09:14:29.762 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.760+0000 7faed738f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560a4587f1b0 0x560a45895e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:29.762 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.760+0000 7faed738f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a4587fad0 0x560a4587a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:29.762 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.760+0000 7faed738f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560a4588a390 0x560a458a1a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:29.762 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.760+0000 7faed738f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7faecc1b5c20 con 0x560a4587fad0 2024-04-24T09:14:29.762 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.760+0000 7faed738f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7faecc1b77c0 con 0x560a4587f1b0 2024-04-24T09:14:29.763 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.760+0000 7faed738f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7faecc1b62f0 con 0x560a4588a390 2024-04-24T09:14:29.763 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.761+0000 7faed1c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560a4587f1b0 0x560a45895e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:29.763 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.761+0000 7faed1c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560a4587f1b0 0x560a45895e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:35448/0 (socket says 172.21.15.83:35448) 2024-04-24T09:14:29.763 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.761+0000 7faed1c00640 1 -- 172.21.15.83:0/2900258845 learned_addr learned my addr 172.21.15.83:0/2900258845 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:14:29.763 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.761+0000 7faed1200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a4587fad0 0x560a4587a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:29.763 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.761+0000 7faed2600640 1 --2- 172.21.15.83:0/2900258845 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560a4588a390 0x560a458a1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:29.763 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.762+0000 7faed1200640 1 -- 172.21.15.83:0/2900258845 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560a4588a390 msgr2=0x560a458a1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:14:29.764 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.762+0000 7faed1200640 1 --2- 172.21.15.83:0/2900258845 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560a4588a390 0x560a458a1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.764 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.762+0000 7faed1200640 1 -- 172.21.15.83:0/2900258845 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560a4587f1b0 msgr2=0x560a45895e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:14:29.764 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.762+0000 7faed1200640 1 --2- 172.21.15.83:0/2900258845 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560a4587f1b0 0x560a45895e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.764 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.762+0000 7faed1200640 1 -- 172.21.15.83:0/2900258845 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a455c4620 con 0x560a4587fad0 2024-04-24T09:14:29.764 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.762+0000 7faed2600640 1 --2- 172.21.15.83:0/2900258845 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560a4588a390 0x560a458a1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:14:29.764 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.762+0000 7faed1c00640 1 --2- 172.21.15.83:0/2900258845 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560a4587f1b0 0x560a45895e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:14:29.764 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.762+0000 7faed1200640 1 --2- 172.21.15.83:0/2900258845 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a4587fad0 0x560a4587a6c0 secure :-1 s=READY pgs=2676 cs=0 l=1 rev1=1 crypto rx=0x560a458888b0 tx=0x7faeb80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72d4e7a324151c26 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:14:29.764 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.762+0000 7faebe000640 1 -- 172.21.15.83:0/2900258845 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeb8029b50 con 0x560a4587fad0 2024-04-24T09:14:29.764 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.763+0000 7faebe000640 1 -- 172.21.15.83:0/2900258845 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faeb8005020 con 0x560a4587fad0 2024-04-24T09:14:29.764 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.763+0000 7faebe000640 1 -- 172.21.15.83:0/2900258845 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeb80292d0 con 0x560a4587fad0 2024-04-24T09:14:29.765 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.763+0000 7faed738f740 1 -- 172.21.15.83:0/2900258845 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a4587fad0 msgr2=0x560a4587a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:29.766 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.763+0000 7faed738f740 1 --2- 172.21.15.83:0/2900258845 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a4587fad0 0x560a4587a6c0 secure :-1 s=READY pgs=2676 cs=0 l=1 rev1=1 crypto rx=0x560a458888b0 tx=0x7faeb80046c0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.766 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.764+0000 7faed738f740 1 -- 172.21.15.83:0/2900258845 shutdown_connections 2024-04-24T09:14:29.766 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.764+0000 7faed738f740 1 --2- 172.21.15.83:0/2900258845 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560a4588a390 0x560a458a1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.766 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.764+0000 7faed738f740 1 --2- 172.21.15.83:0/2900258845 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a4587fad0 0x560a4587a6c0 unknown :-1 s=CLOSED pgs=2676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.766 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.764+0000 7faed738f740 1 --2- 172.21.15.83:0/2900258845 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560a4587f1b0 0x560a45895e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.766 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.764+0000 7faed738f740 1 -- 172.21.15.83:0/2900258845 >> 172.21.15.83:0/2900258845 conn(0x560a4587b6c0 msgr2=0x560a45879fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:14:29.766 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.765+0000 7faed738f740 1 -- 172.21.15.83:0/2900258845 shutdown_connections 2024-04-24T09:14:29.766 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.765+0000 7faed738f740 1 -- 172.21.15.83:0/2900258845 wait complete. 2024-04-24T09:14:29.766 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.766+0000 7faed738f740 1 -- start start 2024-04-24T09:14:29.766 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.766+0000 7faed738f740 1 Processor -- start 2024-04-24T09:14:29.768 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.767+0000 7faed738f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560a4587f1b0 0x560a45884120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:29.768 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.767+0000 7faed738f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a4587fad0 0x560a45884660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:29.768 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.767+0000 7faed738f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560a4588a390 0x560a45884ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:29.768 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.768+0000 7faed738f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7faecc1b70c0 con 0x560a4587fad0 2024-04-24T09:14:29.768 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.768+0000 7faed738f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7faecc1b69f0 con 0x560a4587f1b0 2024-04-24T09:14:29.768 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.768+0000 7faed738f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7faecc1b3c20 con 0x560a4588a390 2024-04-24T09:14:29.768 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.768+0000 7faed1c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560a4587f1b0 0x560a45884120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:29.768 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.768+0000 7faed738f740 10 client.0 _ll_register_callbacks cb 0x7faeb8009e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:14:29.768 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.768+0000 7faed1c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560a4587f1b0 0x560a45884120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:35460/0 (socket says 172.21.15.83:35460) 2024-04-24T09:14:29.769 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.768+0000 7faed1c00640 1 -- 172.21.15.83:0/1628660124 learned_addr learned my addr 172.21.15.83:0/1628660124 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:14:29.769 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.768+0000 7faed2600640 1 --2- 172.21.15.83:0/1628660124 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560a4588a390 0x560a45884ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:29.769 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.768+0000 7faed1c00640 1 -- 172.21.15.83:0/1628660124 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560a4588a390 msgr2=0x560a45884ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:14:29.769 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.768+0000 7faed1c00640 1 --2- 172.21.15.83:0/1628660124 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560a4588a390 0x560a45884ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.769 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.768+0000 7faed1c00640 1 -- 172.21.15.83:0/1628660124 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a4587fad0 msgr2=0x560a45884660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:14:29.769 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.768+0000 7faed1c00640 1 --2- 172.21.15.83:0/1628660124 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a4587fad0 0x560a45884660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.769 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.768+0000 7faed1c00640 1 -- 172.21.15.83:0/1628660124 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a458ff480 con 0x560a4587f1b0 2024-04-24T09:14:29.770 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.769+0000 7faed1c00640 1 --2- 172.21.15.83:0/1628660124 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560a4587f1b0 0x560a45884120 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7faec0031840 tx=0x7faec00316d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4123abeb154a3c58 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:14:29.770 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.769+0000 7faeb3400640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:14:29.770 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.770+0000 7faeb3400640 1 -- 172.21.15.83:0/1628660124 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec0052040 con 0x560a4587f1b0 2024-04-24T09:14:29.770 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.770+0000 7faeb3400640 1 -- 172.21.15.83:0/1628660124 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(128..128 src has 1..128) ==== 9583+0+0 (secure 0 0 0) 0x7faec0004b20 con 0x560a4587f1b0 2024-04-24T09:14:29.770 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.770+0000 7faed738f740 1 -- 172.21.15.83:0/1628660124 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faeb8005020 con 0x560a4587f1b0 2024-04-24T09:14:29.770 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.770+0000 7faeb3400640 1 -- 172.21.15.83:0/1628660124 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7faec0035070 con 0x560a4587f1b0 2024-04-24T09:14:29.771 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.770+0000 7faed738f740 10 client.40430 fetch_fsmap learned FSMap version 65 2024-04-24T09:14:29.771 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.771+0000 7faed738f740 1 -- 172.21.15.83:0/1628660124 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x560a459006c0 con 0x560a4587f1b0 2024-04-24T09:14:29.771 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.771+0000 7faeb3400640 1 -- 172.21.15.83:0/1628660124 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faeb8005020 con 0x560a4587f1b0 2024-04-24T09:14:29.771 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.771+0000 7faeb3400640 1 -- 172.21.15.83:0/1628660124 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7faec0003aa0 con 0x560a4587f1b0 2024-04-24T09:14:29.771 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.771+0000 7faed738f740 10 client.40430 fetch_fsmap finished waiting for FSMap version 65 2024-04-24T09:14:29.772 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.772+0000 7faed738f740 10 client.40430 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:14:29.772 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.772+0000 7faed738f740 20 client.40430 populate_metadata read hostname 'smithi083' 2024-04-24T09:14:29.772 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.772+0000 7faed738f740 1 --2- 172.21.15.83:0/1628660124 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x560a45900db0 0x560a459031d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:29.772 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.772+0000 7faed738f740 4 client.40430 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T09:14:29.772 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.772+0000 7faed738f740 1 -- 172.21.15.83:0/1628660124 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560a457ed190 con 0x560a45900db0 2024-04-24T09:14:29.773 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.773+0000 7faed1200640 1 --2- 172.21.15.83:0/1628660124 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x560a45900db0 0x560a459031d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:29.773 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.773+0000 7faed1200640 1 --2- 172.21.15.83:0/1628660124 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x560a45900db0 0x560a459031d0 crc :-1 s=READY pgs=562 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a02e82751a364651 server_cookie=74919679b217ad15 in_seq=0 out_seq=0 2024-04-24T09:14:29.773 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.773+0000 7faeb3400640 10 client.40430 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:14:29.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.849+0000 7fe9a6c00640 1 -- 172.21.15.83:0/3188829083 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe9a0108e40 msgr2=0x7fe9a01092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:29.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.849+0000 7fe9a6c00640 1 --2- 172.21.15.83:0/3188829083 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe9a0108e40 0x7fe9a01092c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe98c009890 tx=0x7fe98c02a370 comp rx=0 tx=0).stop 2024-04-24T09:14:29.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.850+0000 7fe9a6c00640 1 -- 172.21.15.83:0/3188829083 shutdown_connections 2024-04-24T09:14:29.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.850+0000 7fe9a6c00640 1 --2- 172.21.15.83:0/3188829083 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9a0109800 0x7fe9a0115360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.850+0000 7fe9a6c00640 1 --2- 172.21.15.83:0/3188829083 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe9a0108e40 0x7fe9a01092c0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.850+0000 7fe9a6c00640 1 --2- 172.21.15.83:0/3188829083 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe9a010e5a0 0x7fe9a010e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.850+0000 7fe9a6c00640 1 -- 172.21.15.83:0/3188829083 >> 172.21.15.83:0/3188829083 conn(0x7fe9a0077a20 msgr2=0x7fe9a0079e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:14:29.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.852+0000 7fe9a6c00640 1 -- 172.21.15.83:0/3188829083 shutdown_connections 2024-04-24T09:14:29.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.853+0000 7fe9a6c00640 1 -- 172.21.15.83:0/3188829083 wait complete. 2024-04-24T09:14:29.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.857+0000 7fe9a6c00640 1 Processor -- start 2024-04-24T09:14:29.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.859+0000 7fe9a6c00640 1 -- start start 2024-04-24T09:14:29.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.860+0000 7fe9a6c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe9a0108e40 0x7fe9a006c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:29.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.860+0000 7fe9a6c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9a0109800 0x7fe9a006c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:29.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.860+0000 7fe9a6c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe9a010e5a0 0x7fe9a006cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:29.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.860+0000 7fe99fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe9a0108e40 0x7fe9a006c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:29.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.860+0000 7fe99fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe9a0108e40 0x7fe9a006c0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:35470/0 (socket says 172.21.15.83:35470) 2024-04-24T09:14:29.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.861+0000 7fe99f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9a0109800 0x7fe9a006c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:29.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.861+0000 7fe99f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9a0109800 0x7fe9a006c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37164/0 (socket says 172.21.15.83:37164) 2024-04-24T09:14:29.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.861+0000 7fe9a6c00640 1 -- 172.21.15.83:0/1455258361 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a0118760 con 0x7fe9a0109800 2024-04-24T09:14:29.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.861+0000 7fe9a6c00640 1 -- 172.21.15.83:0/1455258361 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a01185e0 con 0x7fe9a0108e40 2024-04-24T09:14:29.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.861+0000 7fe9a6c00640 1 -- 172.21.15.83:0/1455258361 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a01188e0 con 0x7fe9a010e5a0 2024-04-24T09:14:29.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.861+0000 7fe99fe00640 1 -- 172.21.15.83:0/1455258361 learned_addr learned my addr 172.21.15.83:0/1455258361 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:14:29.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.863+0000 7fe99fe00640 1 -- 172.21.15.83:0/1455258361 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe9a010e5a0 msgr2=0x7fe9a006cb40 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-24T09:14:29.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.863+0000 7fe99fe00640 1 --2- 172.21.15.83:0/1455258361 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe9a010e5a0 0x7fe9a006cb40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.863+0000 7fe99fe00640 1 -- 172.21.15.83:0/1455258361 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9a0109800 msgr2=0x7fe9a006c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:29.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.863+0000 7fe99fe00640 1 --2- 172.21.15.83:0/1455258361 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9a0109800 0x7fe9a006c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.863+0000 7fe99fe00640 1 -- 172.21.15.83:0/1455258361 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9a01b37c0 con 0x7fe9a0108e40 2024-04-24T09:14:29.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.864+0000 7fe99fe00640 1 --2- 172.21.15.83:0/1455258361 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe9a0108e40 0x7fe9a006c0c0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fe99800ff50 tx=0x7fe99800d760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:14:29.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.865+0000 7fe99cc00640 1 -- 172.21.15.83:0/1455258361 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe99800c7a0 con 0x7fe9a0108e40 2024-04-24T09:14:29.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.865+0000 7fe99cc00640 1 -- 172.21.15.83:0/1455258361 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe99800cdf0 con 0x7fe9a0108e40 2024-04-24T09:14:29.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.865+0000 7fe99cc00640 1 -- 172.21.15.83:0/1455258361 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe998016d60 con 0x7fe9a0108e40 2024-04-24T09:14:29.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.865+0000 7fe9a6c00640 1 -- 172.21.15.83:0/1455258361 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9a01b3ab0 con 0x7fe9a0108e40 2024-04-24T09:14:29.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.865+0000 7fe9a6c00640 1 -- 172.21.15.83:0/1455258361 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9a01b3f80 con 0x7fe9a0108e40 2024-04-24T09:14:29.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.876+0000 7fe9a99e0740 1 Processor -- start 2024-04-24T09:14:29.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.876+0000 7fe9a99e0740 1 -- start start 2024-04-24T09:14:29.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.877+0000 7fe9a99e0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561df813cb40 0x561df813cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:29.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.877+0000 7fe9a99e0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561df813d460 0x561df81475f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:29.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.877+0000 7fe9a99e0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561df8147b30 0x561df8149f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:29.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.877+0000 7fe9a99e0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561df80a21d0 con 0x561df813d460 2024-04-24T09:14:29.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.877+0000 7fe9a99e0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x561df8048450 con 0x561df8147b30 2024-04-24T09:14:29.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.877+0000 7fe9a99e0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x561df809e190 con 0x561df813cb40 2024-04-24T09:14:29.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.877+0000 7fe99cc00640 1 -- 172.21.15.83:0/1455258361 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe9980203a0 con 0x7fe9a0108e40 2024-04-24T09:14:29.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.877+0000 7fe9a4c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561df813cb40 0x561df813cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:29.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.878+0000 7fe9a4c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561df813cb40 0x561df813cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:45164/0 (socket says 172.21.15.83:45164) 2024-04-24T09:14:29.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.878+0000 7fe9a4c00640 1 -- 172.21.15.83:0/198267347 learned_addr learned my addr 172.21.15.83:0/198267347 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:14:29.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.878+0000 7fe9a4c00640 1 -- 172.21.15.83:0/198267347 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561df8147b30 msgr2=0x561df8149f20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-24T09:14:29.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.878+0000 7fe9a4c00640 1 --2- 172.21.15.83:0/198267347 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561df8147b30 0x561df8149f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.878+0000 7fe9a4c00640 1 -- 172.21.15.83:0/198267347 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561df813d460 msgr2=0x561df81475f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T09:14:29.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.878+0000 7fe9a4c00640 1 --2- 172.21.15.83:0/198267347 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561df813d460 0x561df81475f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.878+0000 7fe9a4c00640 1 -- 172.21.15.83:0/198267347 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561df8105310 con 0x561df813cb40 2024-04-24T09:14:29.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.879+0000 7fe9a4c00640 1 --2- 172.21.15.83:0/198267347 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561df813cb40 0x561df813cf20 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe994009a80 tx=0x7fe994009f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62b612a1b42bb4ae server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:14:29.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.880+0000 7fe991200640 1 -- 172.21.15.83:0/198267347 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe99400ec90 con 0x561df813cb40 2024-04-24T09:14:29.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.880+0000 7fe991200640 1 -- 172.21.15.83:0/198267347 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe99400ee30 con 0x561df813cb40 2024-04-24T09:14:29.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.880+0000 7fe991200640 1 -- 172.21.15.83:0/198267347 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe994010430 con 0x561df813cb40 2024-04-24T09:14:29.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.882+0000 7fe9a99e0740 1 -- 172.21.15.83:0/198267347 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561df813cb40 msgr2=0x561df813cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:29.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.882+0000 7fe9a99e0740 1 --2- 172.21.15.83:0/198267347 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561df813cb40 0x561df813cf20 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe994009a80 tx=0x7fe994009f40 comp rx=0 tx=0).stop 2024-04-24T09:14:29.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.882+0000 7fe9a99e0740 1 -- 172.21.15.83:0/198267347 shutdown_connections 2024-04-24T09:14:29.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.882+0000 7fe9a99e0740 1 --2- 172.21.15.83:0/198267347 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561df8147b30 0x561df8149f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.882+0000 7fe9a99e0740 1 --2- 172.21.15.83:0/198267347 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561df813d460 0x561df81475f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.882+0000 7fe9a99e0740 1 --2- 172.21.15.83:0/198267347 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561df813cb40 0x561df813cf20 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.882+0000 7fe9a99e0740 1 -- 172.21.15.83:0/198267347 >> 172.21.15.83:0/198267347 conn(0x561df8138e60 msgr2=0x561df8139240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:14:29.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.883+0000 7fe9a99e0740 1 -- 172.21.15.83:0/198267347 shutdown_connections 2024-04-24T09:14:29.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.883+0000 7fe9a99e0740 1 -- 172.21.15.83:0/198267347 wait complete. 2024-04-24T09:14:29.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.884+0000 7fe9a99e0740 1 -- start start 2024-04-24T09:14:29.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.885+0000 7fe9a99e0740 1 Processor -- start 2024-04-24T09:14:29.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.886+0000 7fe99cc00640 1 --2- 172.21.15.83:0/1455258361 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe9880a3db0 0x7fe9880a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:29.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.886+0000 7fe99cc00640 1 -- 172.21.15.83:0/1455258361 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9583+0+0 (secure 0 0 0) 0x7fe9980d15e0 con 0x7fe9a0108e40 2024-04-24T09:14:29.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.887+0000 7fe9a99e0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561df813cb40 0x561df81475e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:29.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.887+0000 7fe9a99e0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561df813d460 0x561df81448d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:29.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.887+0000 7fe9a99e0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561df8147b30 0x561df8145010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:29.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.887+0000 7fe9a99e0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561df814a9c0 con 0x561df8147b30 2024-04-24T09:14:29.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.887+0000 7fe9a99e0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x561df805b810 con 0x561df813d460 2024-04-24T09:14:29.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.887+0000 7fe9a99e0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x561df814ab40 con 0x561df813cb40 2024-04-24T09:14:29.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.887+0000 7fe9a99e0740 10 client.0 _ll_register_callbacks cb 0x561df8029140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:14:29.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.888+0000 7fe9a4c00640 1 --2- 172.21.15.83:0/1455258361 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe9880a3db0 0x7fe9880a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:29.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.888+0000 7fe99fe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561df813cb40 0x561df81475e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:29.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.888+0000 7fe99fe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561df813cb40 0x561df81475e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:45176/0 (socket says 172.21.15.83:45176) 2024-04-24T09:14:29.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.888+0000 7fe99fe00640 1 -- 172.21.15.83:0/2361112779 learned_addr learned my addr 172.21.15.83:0/2361112779 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:14:29.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.889+0000 7fe99fe00640 1 -- 172.21.15.83:0/2361112779 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561df813d460 msgr2=0x561df81448d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:14:29.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.889+0000 7fe99fe00640 1 --2- 172.21.15.83:0/2361112779 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561df813d460 0x561df81448d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.889+0000 7fe99fe00640 1 -- 172.21.15.83:0/2361112779 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561df8147b30 msgr2=0x561df8145010 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T09:14:29.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.889+0000 7fe9a4c00640 1 --2- 172.21.15.83:0/2361112779 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561df8147b30 0x561df8145010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:29.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.889+0000 7fe99fe00640 1 --2- 172.21.15.83:0/2361112779 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561df8147b30 0x561df8145010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.889+0000 7fe99fe00640 1 -- 172.21.15.83:0/2361112779 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561df806c3f0 con 0x561df813cb40 2024-04-24T09:14:29.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.890+0000 7fe99fe00640 1 --2- 172.21.15.83:0/2361112779 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561df813cb40 0x561df81475e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe9980ce970 tx=0x7fe99809cdd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8cb907895c59e623 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:14:29.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.890+0000 7fe985600640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-04-24T09:14:29.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.890+0000 7fe9a4c00640 1 --2- 172.21.15.83:0/1455258361 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe9880a3db0 0x7fe9880a6270 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fe994004ad0 tx=0x7fe994004b30 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:14:29.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.891+0000 7fe985600640 1 -- 172.21.15.83:0/2361112779 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9980b5070 con 0x561df813cb40 2024-04-24T09:14:29.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.891+0000 7fe985600640 1 -- 172.21.15.83:0/2361112779 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(128..128 src has 1..128) ==== 9583+0+0 (secure 0 0 0) 0x7fe99809d3c0 con 0x561df813cb40 2024-04-24T09:14:29.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.892+0000 7fe9a99e0740 1 -- 172.21.15.83:0/2361112779 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe99400ee30 con 0x561df813cb40 2024-04-24T09:14:29.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.893+0000 7fe985600640 1 -- 172.21.15.83:0/2361112779 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7fe9980a1510 con 0x561df813cb40 2024-04-24T09:14:29.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.893+0000 7fe9a99e0740 10 client.30184 fetch_fsmap learned FSMap version 65 2024-04-24T09:14:29.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.893+0000 7fe9a99e0740 1 -- 172.21.15.83:0/2361112779 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x561df7db43f0 con 0x561df813cb40 2024-04-24T09:14:29.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.894+0000 7fe985600640 1 -- 172.21.15.83:0/2361112779 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe99400ee30 con 0x561df813cb40 2024-04-24T09:14:29.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.896+0000 7fe985600640 1 -- 172.21.15.83:0/2361112779 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7fe99807f030 con 0x561df813cb40 2024-04-24T09:14:29.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.896+0000 7fe9a99e0740 10 client.30184 fetch_fsmap finished waiting for FSMap version 65 2024-04-24T09:14:29.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.896+0000 7fe9a99e0740 10 client.30184 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-24T09:14:29.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.896+0000 7fe9a99e0740 20 client.30184 populate_metadata read hostname 'smithi083' 2024-04-24T09:14:29.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.896+0000 7fe9a99e0740 1 --2- 172.21.15.83:0/2361112779 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x561df8157d70 0x561df815a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:29.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.896+0000 7fe9a99e0740 4 client.30184 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:14:29.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.896+0000 7fe9a99e0740 1 -- 172.21.15.83:0/2361112779 --> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561df80afbf0 con 0x561df8157d70 2024-04-24T09:14:29.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.897+0000 7fe99f400640 1 --2- 172.21.15.83:0/2361112779 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x561df8157d70 0x561df815a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:29.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.898+0000 7fe99f400640 1 --2- 172.21.15.83:0/2361112779 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x561df8157d70 0x561df815a160 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=490e1e8b57b746b6 server_cookie=727bf7cb87a48100 in_seq=0 out_seq=0 2024-04-24T09:14:29.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.898+0000 7fe985600640 10 client.30184 ms_handle_connect on v2:172.21.15.138:6836/900566754 2024-04-24T09:14:29.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.900+0000 7fe985600640 1 -- 172.21.15.83:0/2361112779 <== mds.1 v2:172.21.15.138:6836/900566754 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561df80afbf0 con 0x561df8157d70 2024-04-24T09:14:29.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.900+0000 7fe985600640 10 client.30184 handle_command_reply: tid=0 2024-04-24T09:14:29.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.900+0000 7fe9a99e0740 1 client.30184 shutdown 2024-04-24T09:14:29.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.901+0000 7fe9a99e0740 1 -- 172.21.15.83:0/2361112779 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561df813cb40 msgr2=0x561df81475e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:29.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.901+0000 7fe9a99e0740 1 --2- 172.21.15.83:0/2361112779 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561df813cb40 0x561df81475e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe9980ce970 tx=0x7fe99809cdd0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.901+0000 7fe9a99e0740 1 -- 172.21.15.83:0/2361112779 shutdown_connections 2024-04-24T09:14:29.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.901+0000 7fe9a99e0740 1 --2- 172.21.15.83:0/2361112779 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561df8147b30 0x561df8145010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.901+0000 7fe9a99e0740 1 --2- 172.21.15.83:0/2361112779 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561df813d460 0x561df81448d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.901+0000 7fe9a99e0740 1 --2- 172.21.15.83:0/2361112779 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x561df8157d70 0x561df815a160 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.901+0000 7fe9a99e0740 1 --2- 172.21.15.83:0/2361112779 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561df813cb40 0x561df81475e0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.901+0000 7fe9a99e0740 1 -- 172.21.15.83:0/2361112779 >> 172.21.15.83:0/2361112779 conn(0x561df8138e60 msgr2=0x561df813eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:14:29.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.901+0000 7fe985600640 0 client.30184 ms_handle_reset on v2:172.21.15.138:6836/900566754 2024-04-24T09:14:29.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.901+0000 7fe9a99e0740 1 -- 172.21.15.83:0/2361112779 shutdown_connections 2024-04-24T09:14:29.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.901+0000 7fe9a99e0740 1 -- 172.21.15.83:0/2361112779 wait complete. 2024-04-24T09:14:29.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.903+0000 7fe9a99e0740 20 client.30184 trim_cache size 0 max 16384 2024-04-24T09:14:29.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.930+0000 7fe9a99e0740 1 Processor -- start 2024-04-24T09:14:29.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.930+0000 7fe9a99e0740 1 -- start start 2024-04-24T09:14:29.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.930+0000 7fe9a99e0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561df813cb40 0x561df81538e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:29.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.930+0000 7fe9a99e0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561df813d460 0x561df8147110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:29.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.930+0000 7fe9a99e0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561df8147b30 0x561df815f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:29.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.930+0000 7fe9a99e0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe9800100e0 con 0x561df813cb40 2024-04-24T09:14:29.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.930+0000 7fe9a99e0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe980011c80 con 0x561df8147b30 2024-04-24T09:14:29.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.930+0000 7fe9a99e0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe9800107b0 con 0x561df813d460 2024-04-24T09:14:29.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.931+0000 7fe9a4c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561df8147b30 0x561df815f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:29.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.931+0000 7fe9a4c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561df8147b30 0x561df815f9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:35486/0 (socket says 172.21.15.83:35486) 2024-04-24T09:14:29.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.931+0000 7fe9a4c00640 1 -- 172.21.15.83:0/1152405814 learned_addr learned my addr 172.21.15.83:0/1152405814 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:14:29.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.931+0000 7fe99f400640 1 --2- 172.21.15.83:0/1152405814 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561df813d460 0x561df8147110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:29.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.931+0000 7fe99fe00640 1 --2- 172.21.15.83:0/1152405814 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561df813cb40 0x561df81538e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:29.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.932+0000 7fe9a4c00640 1 -- 172.21.15.83:0/1152405814 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561df813d460 msgr2=0x561df8147110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:14:29.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.932+0000 7fe9a4c00640 1 --2- 172.21.15.83:0/1152405814 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561df813d460 0x561df8147110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.932+0000 7fe9a4c00640 1 -- 172.21.15.83:0/1152405814 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561df813cb40 msgr2=0x561df81538e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:14:29.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.932+0000 7fe9a4c00640 1 --2- 172.21.15.83:0/1152405814 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561df813cb40 0x561df81538e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.932+0000 7fe9a4c00640 1 -- 172.21.15.83:0/1152405814 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561df8145380 con 0x561df8147b30 2024-04-24T09:14:29.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.932+0000 7fe99f400640 1 --2- 172.21.15.83:0/1152405814 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561df813d460 0x561df8147110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:14:29.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.932+0000 7fe99fe00640 1 --2- 172.21.15.83:0/1152405814 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561df813cb40 0x561df81538e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:14:29.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.932+0000 7fe9a4c00640 1 --2- 172.21.15.83:0/1152405814 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561df8147b30 0x561df815f9f0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fe994009510 tx=0x7fe994009f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f9355700551b340 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:14:29.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.933+0000 7fe987e00640 1 -- 172.21.15.83:0/1152405814 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9940035f0 con 0x561df8147b30 2024-04-24T09:14:29.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.933+0000 7fe987e00640 1 -- 172.21.15.83:0/1152405814 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe994003c40 con 0x561df8147b30 2024-04-24T09:14:29.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.933+0000 7fe987e00640 1 -- 172.21.15.83:0/1152405814 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe994004fd0 con 0x561df8147b30 2024-04-24T09:14:29.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.934+0000 7fe9a99e0740 1 -- 172.21.15.83:0/1152405814 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561df8147b30 msgr2=0x561df815f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:29.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.934+0000 7fe9a99e0740 1 --2- 172.21.15.83:0/1152405814 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561df8147b30 0x561df815f9f0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fe994009510 tx=0x7fe994009f40 comp rx=0 tx=0).stop 2024-04-24T09:14:29.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.934+0000 7fe9a99e0740 1 -- 172.21.15.83:0/1152405814 shutdown_connections 2024-04-24T09:14:29.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.934+0000 7fe9a99e0740 1 --2- 172.21.15.83:0/1152405814 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561df8147b30 0x561df815f9f0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.934+0000 7fe9a99e0740 1 --2- 172.21.15.83:0/1152405814 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561df813d460 0x561df8147110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.934+0000 7fe9a99e0740 1 --2- 172.21.15.83:0/1152405814 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561df813cb40 0x561df81538e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.934+0000 7fe9a99e0740 1 -- 172.21.15.83:0/1152405814 >> 172.21.15.83:0/1152405814 conn(0x561df8138e60 msgr2=0x561df813fb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:14:29.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.934+0000 7fe9a99e0740 1 -- 172.21.15.83:0/1152405814 shutdown_connections 2024-04-24T09:14:29.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.934+0000 7fe9a99e0740 1 -- 172.21.15.83:0/1152405814 wait complete. 2024-04-24T09:14:29.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.934+0000 7fe9a99e0740 1 -- start start 2024-04-24T09:14:29.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.935+0000 7fe9a99e0740 1 Processor -- start 2024-04-24T09:14:29.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.936+0000 7fe9a99e0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561df813cb40 0x561df8150240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:29.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.936+0000 7fe9a99e0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561df813d460 0x561df8141bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:29.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.936+0000 7fe9a99e0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561df8147b30 0x561df8142160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:29.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.936+0000 7fe99fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561df813cb40 0x561df8150240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:29.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.936+0000 7fe99fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561df813cb40 0x561df8150240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37206/0 (socket says 172.21.15.83:37206) 2024-04-24T09:14:29.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.936+0000 7fe99f400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561df813d460 0x561df8141bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:29.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.936+0000 7fe99f400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561df813d460 0x561df8141bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:45204/0 (socket says 172.21.15.83:45204) 2024-04-24T09:14:29.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.936+0000 7fe99fe00640 1 -- 172.21.15.83:0/1832378228 learned_addr learned my addr 172.21.15.83:0/1832378228 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:14:29.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.936+0000 7fe9a99e0740 1 -- 172.21.15.83:0/1832378228 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe980011580 con 0x561df813cb40 2024-04-24T09:14:29.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.936+0000 7fe9a99e0740 1 -- 172.21.15.83:0/1832378228 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe980010eb0 con 0x561df8147b30 2024-04-24T09:14:29.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.936+0000 7fe9a99e0740 1 -- 172.21.15.83:0/1832378228 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe98000e0e0 con 0x561df813d460 2024-04-24T09:14:29.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.937+0000 7fe9a4c00640 1 --2- 172.21.15.83:0/1832378228 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561df8147b30 0x561df8142160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:29.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.937+0000 7fe9a99e0740 10 client.0 _ll_register_callbacks cb 0x7fe99809cfa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:14:29.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.937+0000 7fe99f400640 1 -- 172.21.15.83:0/1832378228 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561df8147b30 msgr2=0x561df8142160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:14:29.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.937+0000 7fe99f400640 1 --2- 172.21.15.83:0/1832378228 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561df8147b30 0x561df8142160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.937+0000 7fe99f400640 1 -- 172.21.15.83:0/1832378228 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561df813cb40 msgr2=0x561df8150240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:14:29.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.937+0000 7fe99f400640 1 --2- 172.21.15.83:0/1832378228 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561df813cb40 0x561df8150240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.937+0000 7fe99f400640 1 -- 172.21.15.83:0/1832378228 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561df7e7e7d0 con 0x561df813d460 2024-04-24T09:14:29.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.937+0000 7fe99fe00640 1 --2- 172.21.15.83:0/1832378228 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561df813cb40 0x561df8150240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:14:29.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.938+0000 7fe99f400640 1 --2- 172.21.15.83:0/1832378228 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561df813d460 0x561df8141bc0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe98c035040 tx=0x7fe98c030c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85c52a310f9799ec server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:14:29.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.938+0000 7fe985600640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-04-24T09:14:29.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.938+0000 7fe985600640 1 -- 172.21.15.83:0/1832378228 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe98c049070 con 0x561df813d460 2024-04-24T09:14:29.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.938+0000 7fe985600640 1 -- 172.21.15.83:0/1832378228 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(128..128 src has 1..128) ==== 9583+0+0 (secure 0 0 0) 0x7fe98c030680 con 0x561df813d460 2024-04-24T09:14:29.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.938+0000 7fe9a99e0740 1 -- 172.21.15.83:0/1832378228 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe994003c40 con 0x561df813d460 2024-04-24T09:14:29.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.938+0000 7fe985600640 1 -- 172.21.15.83:0/1832378228 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7fe98c054050 con 0x561df813d460 2024-04-24T09:14:29.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.938+0000 7fe9a99e0740 10 client.30190 fetch_fsmap learned FSMap version 65 2024-04-24T09:14:29.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.938+0000 7fe9a99e0740 1 -- 172.21.15.83:0/1832378228 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x561df81e0270 con 0x561df813d460 2024-04-24T09:14:29.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.939+0000 7fe985600640 1 -- 172.21.15.83:0/1832378228 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe994003c40 con 0x561df813d460 2024-04-24T09:14:29.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.939+0000 7fe985600640 1 -- 172.21.15.83:0/1832378228 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7fe98c03e040 con 0x561df813d460 2024-04-24T09:14:29.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.939+0000 7fe9a99e0740 10 client.30190 fetch_fsmap finished waiting for FSMap version 65 2024-04-24T09:14:29.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.939+0000 7fe9a99e0740 10 client.30190 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-24T09:14:29.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.939+0000 7fe9a99e0740 20 client.30190 populate_metadata read hostname 'smithi083' 2024-04-24T09:14:29.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.939+0000 7fe9a99e0740 1 --2- 172.21.15.83:0/1832378228 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x561df81e09a0 0x561df81e2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:29.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.939+0000 7fe9a99e0740 4 client.30190 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:14:29.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.939+0000 7fe9a99e0740 1 -- 172.21.15.83:0/1832378228 --> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561df80a9a60 con 0x561df81e09a0 2024-04-24T09:14:29.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.940+0000 7fe99fe00640 1 --2- 172.21.15.83:0/1832378228 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x561df81e09a0 0x561df81e2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:29.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.940+0000 7fe99fe00640 1 --2- 172.21.15.83:0/1832378228 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x561df81e09a0 0x561df81e2dc0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b53e892ee4497bb0 server_cookie=d056b74798ddf75c in_seq=0 out_seq=0 2024-04-24T09:14:29.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.940+0000 7fe985600640 10 client.30190 ms_handle_connect on v2:172.21.15.138:6836/900566754 2024-04-24T09:14:29.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.941+0000 7fe985600640 1 -- 172.21.15.83:0/1832378228 <== mds.1 v2:172.21.15.138:6836/900566754 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561df80a9a60 con 0x561df81e09a0 2024-04-24T09:14:29.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.941+0000 7fe985600640 10 client.30190 handle_command_reply: tid=0 2024-04-24T09:14:29.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.941+0000 7fe9a99e0740 1 client.30190 shutdown 2024-04-24T09:14:29.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.941+0000 7fe9a99e0740 1 -- 172.21.15.83:0/1832378228 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561df813d460 msgr2=0x561df8141bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:29.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.941+0000 7fe9a99e0740 1 --2- 172.21.15.83:0/1832378228 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561df813d460 0x561df8141bc0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe98c035040 tx=0x7fe98c030c60 comp rx=0 tx=0).stop 2024-04-24T09:14:29.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.942+0000 7fe9a99e0740 1 -- 172.21.15.83:0/1832378228 shutdown_connections 2024-04-24T09:14:29.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.942+0000 7fe9a99e0740 1 --2- 172.21.15.83:0/1832378228 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561df8147b30 0x561df8142160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.942+0000 7fe9a99e0740 1 --2- 172.21.15.83:0/1832378228 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x561df81e09a0 0x561df81e2dc0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.942+0000 7fe9a99e0740 1 --2- 172.21.15.83:0/1832378228 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561df813d460 0x561df8141bc0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.942+0000 7fe9a99e0740 1 --2- 172.21.15.83:0/1832378228 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561df813cb40 0x561df8150240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.942+0000 7fe985600640 0 client.30190 ms_handle_reset on v2:172.21.15.138:6836/900566754 2024-04-24T09:14:29.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.942+0000 7fe9a99e0740 1 -- 172.21.15.83:0/1832378228 >> 172.21.15.83:0/1832378228 conn(0x561df8138e60 msgr2=0x561df813e290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:14:29.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.942+0000 7fe9a99e0740 1 -- 172.21.15.83:0/1832378228 shutdown_connections 2024-04-24T09:14:29.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.942+0000 7fe9a99e0740 1 -- 172.21.15.83:0/1832378228 wait complete. 2024-04-24T09:14:29.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.942+0000 7fe9a99e0740 20 client.30190 trim_cache size 0 max 16384 2024-04-24T09:14:29.944 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:14:29.945 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.943+0000 7faeb3400640 1 -- 172.21.15.83:0/1628660124 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560a457ed190 con 0x560a45900db0 2024-04-24T09:14:29.945 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.943+0000 7faeb3400640 10 client.40430 handle_command_reply: tid=0 2024-04-24T09:14:29.945 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.943+0000 7faed738f740 1 client.40430 shutdown 2024-04-24T09:14:29.945 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.943+0000 7faed738f740 1 -- 172.21.15.83:0/1628660124 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560a4587f1b0 msgr2=0x560a45884120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:29.945 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.943+0000 7faed738f740 1 --2- 172.21.15.83:0/1628660124 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560a4587f1b0 0x560a45884120 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7faec0031840 tx=0x7faec00316d0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.943+0000 7fe9a6c00640 1 -- 172.21.15.83:0/1455258361 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe9880a3db0 msgr2=0x7fe9880a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:29.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.943+0000 7fe9a6c00640 1 --2- 172.21.15.83:0/1455258361 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe9880a3db0 0x7fe9880a6270 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fe994004ad0 tx=0x7fe994004b30 comp rx=0 tx=0).stop 2024-04-24T09:14:29.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.943+0000 7fe9a6c00640 1 -- 172.21.15.83:0/1455258361 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe9a0108e40 msgr2=0x7fe9a006c0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:29.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.943+0000 7fe9a6c00640 1 --2- 172.21.15.83:0/1455258361 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe9a0108e40 0x7fe9a006c0c0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fe99800ff50 tx=0x7fe99800d760 comp rx=0 tx=0).stop 2024-04-24T09:14:29.946 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.943+0000 7faed738f740 1 -- 172.21.15.83:0/1628660124 shutdown_connections 2024-04-24T09:14:29.946 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.943+0000 7faed738f740 1 --2- 172.21.15.83:0/1628660124 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x560a45900db0 0x560a459031d0 crc :-1 s=READY pgs=562 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.943+0000 7fe9a6c00640 1 -- 172.21.15.83:0/1455258361 shutdown_connections 2024-04-24T09:14:29.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.943+0000 7fe9a6c00640 1 --2- 172.21.15.83:0/1455258361 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe9880a3db0 0x7fe9880a6270 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.943+0000 7fe9a6c00640 1 --2- 172.21.15.83:0/1455258361 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe9a010e5a0 0x7fe9a006cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.943+0000 7fe9a6c00640 1 --2- 172.21.15.83:0/1455258361 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9a0109800 0x7fe9a006c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.943+0000 7fe9a6c00640 1 --2- 172.21.15.83:0/1455258361 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe9a0108e40 0x7fe9a006c0c0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.943+0000 7fe9a6c00640 1 -- 172.21.15.83:0/1455258361 >> 172.21.15.83:0/1455258361 conn(0x7fe9a0077a20 msgr2=0x7fe9a007a4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:14:29.947 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.943+0000 7faed738f740 1 --2- 172.21.15.83:0/1628660124 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560a4588a390 0x560a45884ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.947 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.943+0000 7faed738f740 1 --2- 172.21.15.83:0/1628660124 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a4587fad0 0x560a45884660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.947 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.943+0000 7faed738f740 1 --2- 172.21.15.83:0/1628660124 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560a4587f1b0 0x560a45884120 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.947 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.943+0000 7faed738f740 1 -- 172.21.15.83:0/1628660124 >> 172.21.15.83:0/1628660124 conn(0x560a4587b6c0 msgr2=0x560a45887b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:14:29.947 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.943+0000 7faeb3400640 0 client.40430 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:14:29.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.943+0000 7fe9a6c00640 1 -- 172.21.15.83:0/1455258361 shutdown_connections 2024-04-24T09:14:29.948 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.944+0000 7faed738f740 1 -- 172.21.15.83:0/1628660124 shutdown_connections 2024-04-24T09:14:29.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:29.944+0000 7fe9a6c00640 1 -- 172.21.15.83:0/1455258361 wait complete. 2024-04-24T09:14:29.949 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.944+0000 7faed738f740 1 -- 172.21.15.83:0/1628660124 wait complete. 2024-04-24T09:14:29.949 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.944+0000 7faed738f740 20 client.40430 trim_cache size 0 max 16384 2024-04-24T09:14:29.949 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T09:14:29.949 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T09:14:29.949 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T09:14:29.949 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T09:14:29.950 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.945+0000 7faed4600640 1 -- 172.21.15.83:0/551712868 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7faeb40a3e90 msgr2=0x7faeb40a6350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:29.950 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.945+0000 7faed4600640 1 --2- 172.21.15.83:0/551712868 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7faeb40a3e90 0x7faeb40a6350 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x560a4587cae0 tx=0x7faeb8005850 comp rx=0 tx=0).stop 2024-04-24T09:14:29.950 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.945+0000 7faed4600640 1 -- 172.21.15.83:0/551712868 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faecc107090 msgr2=0x7faecc19a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:29.950 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.945+0000 7faed4600640 1 --2- 172.21.15.83:0/551712868 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faecc107090 0x7faecc19a770 secure :-1 s=READY pgs=2674 cs=0 l=1 rev1=1 crypto rx=0x7faec400cad0 tx=0x7faec400cf90 comp rx=0 tx=0).stop 2024-04-24T09:14:29.950 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.945+0000 7faed4600640 1 -- 172.21.15.83:0/551712868 shutdown_connections 2024-04-24T09:14:29.950 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.945+0000 7faed4600640 1 --2- 172.21.15.83:0/551712868 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7faeb40a3e90 0x7faeb40a6350 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.950 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.945+0000 7faed4600640 1 --2- 172.21.15.83:0/551712868 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faecc107090 0x7faecc19a770 unknown :-1 s=CLOSED pgs=2674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.950 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.945+0000 7faed4600640 1 --2- 172.21.15.83:0/551712868 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faecc101bf0 0x7faecc1a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.950 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.945+0000 7faed4600640 1 --2- 172.21.15.83:0/551712868 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faecc101250 0x7faecc1a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:29.951 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.946+0000 7faed4600640 1 -- 172.21.15.83:0/551712868 >> 172.21.15.83:0/551712868 conn(0x7faecc076c30 msgr2=0x7faecc0fffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:14:29.951 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.946+0000 7faed4600640 1 -- 172.21.15.83:0/551712868 shutdown_connections 2024-04-24T09:14:29.951 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:29.946+0000 7faed4600640 1 -- 172.21.15.83:0/551712868 wait complete. 2024-04-24T09:14:29.967 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T09:14:30.207 INFO:tasks.workunit.client.0.smithi083.stdout: 10 66827 6.56 MB/sec execute 421 sec latency 141.497 ms 2024-04-24T09:14:30.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.218+0000 7ff227200640 1 -- 172.21.15.83:0/83611373 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2200a26d0 msgr2=0x7ff2200a4ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:30.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.218+0000 7ff227200640 1 --2- 172.21.15.83:0/83611373 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2200a26d0 0x7ff2200a4ac0 secure :-1 s=READY pgs=2677 cs=0 l=1 rev1=1 crypto rx=0x7ff228067d00 tx=0x7ff228096890 comp rx=0 tx=0).stop 2024-04-24T09:14:30.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.218+0000 7ff227200640 1 -- 172.21.15.83:0/83611373 shutdown_connections 2024-04-24T09:14:30.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.218+0000 7ff227200640 1 --2- 172.21.15.83:0/83611373 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff2200a26d0 0x7ff2200a4ac0 unknown :-1 s=CLOSED pgs=2677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:30.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.218+0000 7ff227200640 1 --2- 172.21.15.83:0/83611373 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff220094ab0 0x7ff2200a2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:30.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.218+0000 7ff227200640 1 --2- 172.21.15.83:0/83611373 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff220094190 0x7ff220094570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:30.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.218+0000 7ff227200640 1 -- 172.21.15.83:0/83611373 >> 172.21.15.83:0/83611373 conn(0x7ff220009a80 msgr2=0x7ff220009e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:14:30.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.219+0000 7ff227200640 1 -- 172.21.15.83:0/83611373 shutdown_connections 2024-04-24T09:14:30.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.219+0000 7ff227200640 1 -- 172.21.15.83:0/83611373 wait complete. 2024-04-24T09:14:30.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.220+0000 7ff227200640 1 Processor -- start 2024-04-24T09:14:30.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.220+0000 7ff227200640 1 -- start start 2024-04-24T09:14:30.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.221+0000 7ff227200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff220094190 0x7ff22012e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:30.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.221+0000 7ff227200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff220094ab0 0x7ff22012ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:30.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.221+0000 7ff227200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff2200a26d0 0x7ff220132e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:30.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.221+0000 7ff227200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff2200a8130 con 0x7ff220094ab0 2024-04-24T09:14:30.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.221+0000 7ff227200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff2200a7fb0 con 0x7ff220094190 2024-04-24T09:14:30.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.221+0000 7ff227200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff2200a82b0 con 0x7ff2200a26d0 2024-04-24T09:14:30.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.221+0000 7ff225e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff220094190 0x7ff22012e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:30.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.221+0000 7ff225e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff220094190 0x7ff22012e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:35524/0 (socket says 172.21.15.83:35524) 2024-04-24T09:14:30.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.221+0000 7ff225e00640 1 -- 172.21.15.83:0/1887167580 learned_addr learned my addr 172.21.15.83:0/1887167580 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:14:30.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.221+0000 7ff225400640 1 --2- 172.21.15.83:0/1887167580 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff220094ab0 0x7ff22012ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:30.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.221+0000 7ff226800640 1 --2- 172.21.15.83:0/1887167580 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff2200a26d0 0x7ff220132e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:30.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.222+0000 7ff225400640 1 -- 172.21.15.83:0/1887167580 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff2200a26d0 msgr2=0x7ff220132e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:30.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.222+0000 7ff225400640 1 --2- 172.21.15.83:0/1887167580 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff2200a26d0 0x7ff220132e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:30.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.222+0000 7ff225400640 1 -- 172.21.15.83:0/1887167580 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff220094190 msgr2=0x7ff22012e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:30.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.222+0000 7ff225400640 1 --2- 172.21.15.83:0/1887167580 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff220094190 0x7ff22012e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:30.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.222+0000 7ff225400640 1 -- 172.21.15.83:0/1887167580 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff220133580 con 0x7ff220094ab0 2024-04-24T09:14:30.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.223+0000 7ff225400640 1 --2- 172.21.15.83:0/1887167580 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff220094ab0 0x7ff22012ead0 secure :-1 s=READY pgs=2678 cs=0 l=1 rev1=1 crypto rx=0x7ff21400c970 tx=0x7ff21400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:14:30.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.223+0000 7ff21ea00640 1 -- 172.21.15.83:0/1887167580 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff214007bb0 con 0x7ff220094ab0 2024-04-24T09:14:30.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.223+0000 7ff21ea00640 1 -- 172.21.15.83:0/1887167580 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff214007d50 con 0x7ff220094ab0 2024-04-24T09:14:30.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.223+0000 7ff21ea00640 1 -- 172.21.15.83:0/1887167580 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff214018780 con 0x7ff220094ab0 2024-04-24T09:14:30.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.223+0000 7ff227200640 1 -- 172.21.15.83:0/1887167580 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff220133870 con 0x7ff220094ab0 2024-04-24T09:14:30.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.223+0000 7ff227200640 1 -- 172.21.15.83:0/1887167580 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff220097d40 con 0x7ff220094ab0 2024-04-24T09:14:30.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.228+0000 7ff22fbbc740 1 Processor -- start 2024-04-24T09:14:30.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.228+0000 7ff22fbbc740 1 -- start start 2024-04-24T09:14:30.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.229+0000 7ff22fbbc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f37b9fb40 0x560f37b9ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:30.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.229+0000 7ff22fbbc740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560f37ba0460 0x560f37baa5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:30.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.229+0000 7ff22fbbc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f37baab30 0x560f37bacf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:30.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.229+0000 7ff22fbbc740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x560f37b051d0 con 0x560f37baab30 2024-04-24T09:14:30.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.229+0000 7ff22fbbc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560f37aab450 con 0x560f37b9fb40 2024-04-24T09:14:30.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.229+0000 7ff22fbbc740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x560f37b01190 con 0x560f37ba0460 2024-04-24T09:14:30.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.230+0000 7ff226800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f37b9fb40 0x560f37b9ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:30.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.230+0000 7ff21ea00640 1 -- 172.21.15.83:0/1887167580 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff21400f450 con 0x7ff220094ab0 2024-04-24T09:14:30.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.230+0000 7ff226800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f37b9fb40 0x560f37b9ff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:35536/0 (socket says 172.21.15.83:35536) 2024-04-24T09:14:30.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.232+0000 7ff226800640 1 -- 172.21.15.83:0/4018616863 learned_addr learned my addr 172.21.15.83:0/4018616863 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:14:30.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.230+0000 7ff225e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560f37ba0460 0x560f37baa5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:30.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.230+0000 7ff225400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f37baab30 0x560f37bacf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:30.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.232+0000 7ff21ea00640 1 --2- 172.21.15.83:0/1887167580 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff20c0a3da0 0x7ff20c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:30.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.232+0000 7ff225400640 1 -- 172.21.15.83:0/4018616863 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560f37ba0460 msgr2=0x560f37baa5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:14:30.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.232+0000 7ff225400640 1 --2- 172.21.15.83:0/4018616863 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560f37ba0460 0x560f37baa5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:30.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.232+0000 7ff225400640 1 -- 172.21.15.83:0/4018616863 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f37b9fb40 msgr2=0x560f37b9ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:14:30.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.232+0000 7ff225400640 1 --2- 172.21.15.83:0/4018616863 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f37b9fb40 0x560f37b9ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:30.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.232+0000 7ff225400640 1 -- 172.21.15.83:0/4018616863 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f37b68310 con 0x560f37baab30 2024-04-24T09:14:30.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.233+0000 7ff226800640 1 --2- 172.21.15.83:0/4018616863 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f37b9fb40 0x560f37b9ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:14:30.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.233+0000 7ff21ea00640 1 -- 172.21.15.83:0/1887167580 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9583+0+0 (secure 0 0 0) 0x7ff2140d0c50 con 0x7ff220094ab0 2024-04-24T09:14:30.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.233+0000 7ff225400640 1 --2- 172.21.15.83:0/4018616863 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f37baab30 0x560f37bacf20 secure :-1 s=READY pgs=2679 cs=0 l=1 rev1=1 crypto rx=0x7ff2140cdf20 tx=0x7ff214066ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=787d4e3673404317 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:14:30.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.233+0000 7ff226800640 1 --2- 172.21.15.83:0/1887167580 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff20c0a3da0 0x7ff20c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:30.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.235+0000 7ff226800640 1 --2- 172.21.15.83:0/1887167580 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff20c0a3da0 0x7ff20c0a6260 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7ff228096e80 tx=0x7ff2280684d0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:14:30.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.235+0000 7ff213400640 1 -- 172.21.15.83:0/4018616863 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff21409c2b0 con 0x560f37baab30 2024-04-24T09:14:30.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.235+0000 7ff213400640 1 -- 172.21.15.83:0/4018616863 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff21400fa90 con 0x560f37baab30 2024-04-24T09:14:30.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.235+0000 7ff213400640 1 -- 172.21.15.83:0/4018616863 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff21409c0d0 con 0x560f37baab30 2024-04-24T09:14:30.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.236+0000 7ff22fbbc740 1 -- 172.21.15.83:0/4018616863 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f37baab30 msgr2=0x560f37bacf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:30.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.236+0000 7ff22fbbc740 1 --2- 172.21.15.83:0/4018616863 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f37baab30 0x560f37bacf20 secure :-1 s=READY pgs=2679 cs=0 l=1 rev1=1 crypto rx=0x7ff2140cdf20 tx=0x7ff214066ea0 comp rx=0 tx=0).stop 2024-04-24T09:14:30.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.237+0000 7ff22fbbc740 1 -- 172.21.15.83:0/4018616863 shutdown_connections 2024-04-24T09:14:30.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.237+0000 7ff22fbbc740 1 --2- 172.21.15.83:0/4018616863 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f37baab30 0x560f37bacf20 unknown :-1 s=CLOSED pgs=2679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:30.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.237+0000 7ff22fbbc740 1 --2- 172.21.15.83:0/4018616863 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560f37ba0460 0x560f37baa5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:30.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.237+0000 7ff22fbbc740 1 --2- 172.21.15.83:0/4018616863 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f37b9fb40 0x560f37b9ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:30.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.237+0000 7ff22fbbc740 1 -- 172.21.15.83:0/4018616863 >> 172.21.15.83:0/4018616863 conn(0x560f37b9be60 msgr2=0x560f37b9c240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:14:30.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.237+0000 7ff22fbbc740 1 -- 172.21.15.83:0/4018616863 shutdown_connections 2024-04-24T09:14:30.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.237+0000 7ff22fbbc740 1 -- 172.21.15.83:0/4018616863 wait complete. 2024-04-24T09:14:30.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.238+0000 7ff22fbbc740 1 -- start start 2024-04-24T09:14:30.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.238+0000 7ff22fbbc740 1 Processor -- start 2024-04-24T09:14:30.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.238+0000 7ff22fbbc740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560f37b9fb40 0x560f37baa4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:30.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.238+0000 7ff22fbbc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f37ba0460 0x560f37ba77e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:30.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.238+0000 7ff22fbbc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f37baab30 0x560f37ba7df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:30.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.239+0000 7ff22fbbc740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x560f37bad960 con 0x560f37baab30 2024-04-24T09:14:30.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.239+0000 7ff22fbbc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560f37abe810 con 0x560f37ba0460 2024-04-24T09:14:30.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.239+0000 7ff22fbbc740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x560f37badae0 con 0x560f37b9fb40 2024-04-24T09:14:30.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.239+0000 7ff22fbbc740 10 client.0 _ll_register_callbacks cb 0x560f37a8c140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:14:30.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.239+0000 7ff225400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f37ba0460 0x560f37ba77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:30.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.239+0000 7ff225400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f37ba0460 0x560f37ba77e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:35542/0 (socket says 172.21.15.83:35542) 2024-04-24T09:14:30.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.239+0000 7ff225400640 1 -- 172.21.15.83:0/3658420629 learned_addr learned my addr 172.21.15.83:0/3658420629 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:14:30.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.239+0000 7ff225400640 1 -- 172.21.15.83:0/3658420629 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560f37b9fb40 msgr2=0x560f37baa4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:14:30.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.239+0000 7ff225400640 1 --2- 172.21.15.83:0/3658420629 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560f37b9fb40 0x560f37baa4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:30.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.239+0000 7ff225400640 1 -- 172.21.15.83:0/3658420629 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f37baab30 msgr2=0x560f37ba7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:14:30.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.239+0000 7ff226800640 1 --2- 172.21.15.83:0/3658420629 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f37baab30 0x560f37ba7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:30.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.240+0000 7ff225400640 1 --2- 172.21.15.83:0/3658420629 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f37baab30 0x560f37ba7df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:30.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.240+0000 7ff225400640 1 -- 172.21.15.83:0/3658420629 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f37acf3f0 con 0x560f37ba0460 2024-04-24T09:14:30.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.240+0000 7ff226800640 1 --2- 172.21.15.83:0/3658420629 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f37baab30 0x560f37ba7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:14:30.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.240+0000 7ff225400640 1 --2- 172.21.15.83:0/3658420629 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f37ba0460 0x560f37ba77e0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7ff2140a4eb0 tx=0x7ff21400fc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8239837192e5646e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:14:30.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.240+0000 7ff207e00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:14:30.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.240+0000 7ff207e00640 1 -- 172.21.15.83:0/3658420629 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff21409c0d0 con 0x560f37ba0460 2024-04-24T09:14:30.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.240+0000 7ff207e00640 1 -- 172.21.15.83:0/3658420629 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(128..128 src has 1..128) ==== 9583+0+0 (secure 0 0 0) 0x7ff21400f740 con 0x560f37ba0460 2024-04-24T09:14:30.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.241+0000 7ff22fbbc740 1 -- 172.21.15.83:0/3658420629 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560f37b12bf0 con 0x560f37ba0460 2024-04-24T09:14:30.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.241+0000 7ff207e00640 1 -- 172.21.15.83:0/3658420629 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7ff21409c2b0 con 0x560f37ba0460 2024-04-24T09:14:30.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.241+0000 7ff22fbbc740 10 client.40454 fetch_fsmap learned FSMap version 65 2024-04-24T09:14:30.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.241+0000 7ff22fbbc740 1 -- 172.21.15.83:0/3658420629 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x560f378173f0 con 0x560f37ba0460 2024-04-24T09:14:30.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.241+0000 7ff207e00640 1 -- 172.21.15.83:0/3658420629 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x560f37b12bf0 con 0x560f37ba0460 2024-04-24T09:14:30.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.241+0000 7ff207e00640 1 -- 172.21.15.83:0/3658420629 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7ff214098de0 con 0x560f37ba0460 2024-04-24T09:14:30.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.241+0000 7ff22fbbc740 10 client.40454 fetch_fsmap finished waiting for FSMap version 65 2024-04-24T09:14:30.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.241+0000 7ff22fbbc740 10 client.40454 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:14:30.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.242+0000 7ff22fbbc740 20 client.40454 populate_metadata read hostname 'smithi083' 2024-04-24T09:14:30.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.242+0000 7ff22fbbc740 1 --2- 172.21.15.83:0/3658420629 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x560f37bbab40 0x560f37bbcf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:30.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.242+0000 7ff22fbbc740 4 client.40454 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:14:30.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.242+0000 7ff22fbbc740 1 -- 172.21.15.83:0/3658420629 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560f37b0ca60 con 0x560f37bbab40 2024-04-24T09:14:30.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.242+0000 7ff225e00640 1 --2- 172.21.15.83:0/3658420629 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x560f37bbab40 0x560f37bbcf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:30.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.243+0000 7ff225e00640 1 --2- 172.21.15.83:0/3658420629 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x560f37bbab40 0x560f37bbcf30 crc :-1 s=READY pgs=563 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2163b406abf202ed server_cookie=bb982bef187b0f59 in_seq=0 out_seq=0 2024-04-24T09:14:30.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.243+0000 7ff207e00640 10 client.40454 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:14:30.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.245+0000 7ff207e00640 1 -- 172.21.15.83:0/3658420629 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560f37b0ca60 con 0x560f37bbab40 2024-04-24T09:14:30.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.245+0000 7ff207e00640 10 client.40454 handle_command_reply: tid=0 2024-04-24T09:14:30.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.245+0000 7ff22fbbc740 1 client.40454 shutdown 2024-04-24T09:14:30.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.245+0000 7ff22fbbc740 1 -- 172.21.15.83:0/3658420629 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f37ba0460 msgr2=0x560f37ba77e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:30.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.245+0000 7ff22fbbc740 1 --2- 172.21.15.83:0/3658420629 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f37ba0460 0x560f37ba77e0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7ff2140a4eb0 tx=0x7ff21400fc80 comp rx=0 tx=0).stop 2024-04-24T09:14:30.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.245+0000 7ff22fbbc740 1 -- 172.21.15.83:0/3658420629 shutdown_connections 2024-04-24T09:14:30.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.245+0000 7ff22fbbc740 1 --2- 172.21.15.83:0/3658420629 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x560f37bbab40 0x560f37bbcf30 crc :-1 s=READY pgs=563 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:30.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.245+0000 7ff22fbbc740 1 --2- 172.21.15.83:0/3658420629 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f37baab30 0x560f37ba7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:30.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.245+0000 7ff22fbbc740 1 --2- 172.21.15.83:0/3658420629 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f37ba0460 0x560f37ba77e0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:30.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.245+0000 7ff22fbbc740 1 --2- 172.21.15.83:0/3658420629 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560f37b9fb40 0x560f37baa4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:30.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.245+0000 7ff207e00640 0 client.40454 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:14:30.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.245+0000 7ff22fbbc740 1 -- 172.21.15.83:0/3658420629 >> 172.21.15.83:0/3658420629 conn(0x560f37b9be60 msgr2=0x560f37ba1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:14:30.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.246+0000 7ff22fbbc740 1 -- 172.21.15.83:0/3658420629 shutdown_connections 2024-04-24T09:14:30.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.246+0000 7ff22fbbc740 1 -- 172.21.15.83:0/3658420629 wait complete. 2024-04-24T09:14:30.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.246+0000 7ff22fbbc740 20 client.40454 trim_cache size 0 max 16384 2024-04-24T09:14:30.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.271+0000 7ff22fbbc740 1 Processor -- start 2024-04-24T09:14:30.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.272+0000 7ff22fbbc740 1 -- start start 2024-04-24T09:14:30.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.272+0000 7ff22fbbc740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560f37b9fb40 0x560f37bb68b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:30.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.272+0000 7ff22fbbc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f37ba0460 0x560f37ba67c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:30.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.272+0000 7ff22fbbc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f37baab30 0x560f37baa020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:30.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.272+0000 7ff22fbbc740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff2201496a0 con 0x560f37ba0460 2024-04-24T09:14:30.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.272+0000 7ff22fbbc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff22014b240 con 0x560f37baab30 2024-04-24T09:14:30.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.272+0000 7ff22fbbc740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff220149d70 con 0x560f37b9fb40 2024-04-24T09:14:30.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.272+0000 7ff225400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f37ba0460 0x560f37ba67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:30.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.272+0000 7ff225400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f37ba0460 0x560f37ba67c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37248/0 (socket says 172.21.15.83:37248) 2024-04-24T09:14:30.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.272+0000 7ff225400640 1 -- 172.21.15.83:0/3889659536 learned_addr learned my addr 172.21.15.83:0/3889659536 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:14:30.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.273+0000 7ff225e00640 1 --2- 172.21.15.83:0/3889659536 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560f37b9fb40 0x560f37bb68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:30.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.273+0000 7ff226800640 1 --2- 172.21.15.83:0/3889659536 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f37baab30 0x560f37baa020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:30.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.273+0000 7ff225400640 1 -- 172.21.15.83:0/3889659536 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560f37b9fb40 msgr2=0x560f37bb68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:14:30.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.273+0000 7ff225400640 1 --2- 172.21.15.83:0/3889659536 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560f37b9fb40 0x560f37bb68b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:30.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.273+0000 7ff225400640 1 -- 172.21.15.83:0/3889659536 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f37baab30 msgr2=0x560f37baa020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:14:30.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.273+0000 7ff225400640 1 --2- 172.21.15.83:0/3889659536 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f37baab30 0x560f37baa020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:30.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.273+0000 7ff225400640 1 -- 172.21.15.83:0/3889659536 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f378e17d0 con 0x560f37ba0460 2024-04-24T09:14:30.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.273+0000 7ff226800640 1 --2- 172.21.15.83:0/3889659536 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f37baab30 0x560f37baa020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:14:30.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.273+0000 7ff225400640 1 --2- 172.21.15.83:0/3889659536 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f37ba0460 0x560f37ba67c0 secure :-1 s=READY pgs=2680 cs=0 l=1 rev1=1 crypto rx=0x7ff214007590 tx=0x7ff21409ce70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3421e36fe92ec726 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:14:30.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.274+0000 7ff212a00640 1 -- 172.21.15.83:0/3889659536 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff21409c0d0 con 0x560f37ba0460 2024-04-24T09:14:30.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.274+0000 7ff212a00640 1 -- 172.21.15.83:0/3889659536 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff2140a53e0 con 0x560f37ba0460 2024-04-24T09:14:30.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.274+0000 7ff212a00640 1 -- 172.21.15.83:0/3889659536 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2140a4f50 con 0x560f37ba0460 2024-04-24T09:14:30.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.275+0000 7ff22fbbc740 1 -- 172.21.15.83:0/3889659536 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f37ba0460 msgr2=0x560f37ba67c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:30.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.275+0000 7ff22fbbc740 1 --2- 172.21.15.83:0/3889659536 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f37ba0460 0x560f37ba67c0 secure :-1 s=READY pgs=2680 cs=0 l=1 rev1=1 crypto rx=0x7ff214007590 tx=0x7ff21409ce70 comp rx=0 tx=0).stop 2024-04-24T09:14:30.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.275+0000 7ff22fbbc740 1 -- 172.21.15.83:0/3889659536 shutdown_connections 2024-04-24T09:14:30.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.275+0000 7ff22fbbc740 1 --2- 172.21.15.83:0/3889659536 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f37baab30 0x560f37baa020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:30.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.275+0000 7ff22fbbc740 1 --2- 172.21.15.83:0/3889659536 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f37ba0460 0x560f37ba67c0 unknown :-1 s=CLOSED pgs=2680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:30.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.275+0000 7ff22fbbc740 1 --2- 172.21.15.83:0/3889659536 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560f37b9fb40 0x560f37bb68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:30.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.275+0000 7ff22fbbc740 1 -- 172.21.15.83:0/3889659536 >> 172.21.15.83:0/3889659536 conn(0x560f37b9be60 msgr2=0x560f37ba60b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:14:30.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.275+0000 7ff22fbbc740 1 -- 172.21.15.83:0/3889659536 shutdown_connections 2024-04-24T09:14:30.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.275+0000 7ff22fbbc740 1 -- 172.21.15.83:0/3889659536 wait complete. 2024-04-24T09:14:30.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.276+0000 7ff22fbbc740 1 -- start start 2024-04-24T09:14:30.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.276+0000 7ff22fbbc740 1 Processor -- start 2024-04-24T09:14:30.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.276+0000 7ff22fbbc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f37b9fb40 0x560f37baaf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:30.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.276+0000 7ff22fbbc740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560f37ba0460 0x560f37bab450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:30.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.276+0000 7ff22fbbc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f37baab30 0x560f37bab990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:30.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.276+0000 7ff22fbbc740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff22014ab40 con 0x560f37b9fb40 2024-04-24T09:14:30.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.276+0000 7ff22fbbc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff22014a470 con 0x560f37baab30 2024-04-24T09:14:30.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.276+0000 7ff22fbbc740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff220147700 con 0x560f37ba0460 2024-04-24T09:14:30.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.276+0000 7ff22fbbc740 10 client.0 _ll_register_callbacks cb 0x7ff22806b040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:14:30.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.277+0000 7ff226800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f37baab30 0x560f37bab990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:30.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.277+0000 7ff226800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f37baab30 0x560f37bab990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:35562/0 (socket says 172.21.15.83:35562) 2024-04-24T09:14:30.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.277+0000 7ff226800640 1 -- 172.21.15.83:0/1372298292 learned_addr learned my addr 172.21.15.83:0/1372298292 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:14:30.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.277+0000 7ff225e00640 1 --2- 172.21.15.83:0/1372298292 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f37b9fb40 0x560f37baaf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:30.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.277+0000 7ff225400640 1 --2- 172.21.15.83:0/1372298292 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560f37ba0460 0x560f37bab450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:30.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.277+0000 7ff226800640 1 -- 172.21.15.83:0/1372298292 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560f37ba0460 msgr2=0x560f37bab450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:14:30.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.277+0000 7ff226800640 1 --2- 172.21.15.83:0/1372298292 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560f37ba0460 0x560f37bab450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:30.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.277+0000 7ff226800640 1 -- 172.21.15.83:0/1372298292 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f37b9fb40 msgr2=0x560f37baaf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:14:30.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.278+0000 7ff226800640 1 --2- 172.21.15.83:0/1372298292 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f37b9fb40 0x560f37baaf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:30.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.278+0000 7ff226800640 1 -- 172.21.15.83:0/1372298292 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f37bb1fe0 con 0x560f37baab30 2024-04-24T09:14:30.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.278+0000 7ff225e00640 1 --2- 172.21.15.83:0/1372298292 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f37b9fb40 0x560f37baaf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:14:30.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.278+0000 7ff226800640 1 --2- 172.21.15.83:0/1372298292 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f37baab30 0x560f37bab990 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7ff228000b60 tx=0x7ff22806a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed87f8d93da8bf82 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:14:30.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.278+0000 7ff207e00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:14:30.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.278+0000 7ff207e00640 1 -- 172.21.15.83:0/1372298292 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2280bc070 con 0x560f37baab30 2024-04-24T09:14:30.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.279+0000 7ff207e00640 1 -- 172.21.15.83:0/1372298292 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(128..128 src has 1..128) ==== 9583+0+0 (secure 0 0 0) 0x7ff2280a4040 con 0x560f37baab30 2024-04-24T09:14:30.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.279+0000 7ff22fbbc740 1 -- 172.21.15.83:0/1372298292 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2140a53e0 con 0x560f37baab30 2024-04-24T09:14:30.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.279+0000 7ff207e00640 1 -- 172.21.15.83:0/1372298292 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7ff228068b30 con 0x560f37baab30 2024-04-24T09:14:30.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.279+0000 7ff22fbbc740 10 client.40457 fetch_fsmap learned FSMap version 65 2024-04-24T09:14:30.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.279+0000 7ff22fbbc740 1 -- 172.21.15.83:0/1372298292 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x560f37bb3230 con 0x560f37baab30 2024-04-24T09:14:30.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.279+0000 7ff207e00640 1 -- 172.21.15.83:0/1372298292 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff2140a53e0 con 0x560f37baab30 2024-04-24T09:14:30.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.280+0000 7ff207e00640 1 -- 172.21.15.83:0/1372298292 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7ff2280c1460 con 0x560f37baab30 2024-04-24T09:14:30.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.280+0000 7ff22fbbc740 10 client.40457 fetch_fsmap finished waiting for FSMap version 65 2024-04-24T09:14:30.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.280+0000 7ff22fbbc740 10 client.40457 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:14:30.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.280+0000 7ff22fbbc740 20 client.40457 populate_metadata read hostname 'smithi083' 2024-04-24T09:14:30.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.280+0000 7ff22fbbc740 1 --2- 172.21.15.83:0/1372298292 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x560f37c31ef0 0x560f37c34310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:30.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.281+0000 7ff22fbbc740 4 client.40457 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:14:30.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.281+0000 7ff22fbbc740 1 -- 172.21.15.83:0/1372298292 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560f37a884d0 con 0x560f37c31ef0 2024-04-24T09:14:30.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.281+0000 7ff225e00640 1 --2- 172.21.15.83:0/1372298292 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x560f37c31ef0 0x560f37c34310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:30.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.282+0000 7ff225e00640 1 --2- 172.21.15.83:0/1372298292 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x560f37c31ef0 0x560f37c34310 crc :-1 s=READY pgs=564 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8a11edaf1605a97 server_cookie=8a54ad3c512220c7 in_seq=0 out_seq=0 2024-04-24T09:14:30.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.282+0000 7ff207e00640 10 client.40457 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:14:30.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.282+0000 7ff207e00640 1 -- 172.21.15.83:0/1372298292 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560f37a884d0 con 0x560f37c31ef0 2024-04-24T09:14:30.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.282+0000 7ff207e00640 10 client.40457 handle_command_reply: tid=0 2024-04-24T09:14:30.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.282+0000 7ff22fbbc740 1 client.40457 shutdown 2024-04-24T09:14:30.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.282+0000 7ff22fbbc740 1 -- 172.21.15.83:0/1372298292 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f37baab30 msgr2=0x560f37bab990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:30.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.282+0000 7ff22fbbc740 1 --2- 172.21.15.83:0/1372298292 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f37baab30 0x560f37bab990 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7ff228000b60 tx=0x7ff22806a410 comp rx=0 tx=0).stop 2024-04-24T09:14:30.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.282+0000 7ff22fbbc740 1 -- 172.21.15.83:0/1372298292 shutdown_connections 2024-04-24T09:14:30.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.282+0000 7ff22fbbc740 1 --2- 172.21.15.83:0/1372298292 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x560f37c31ef0 0x560f37c34310 crc :-1 s=READY pgs=564 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:30.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.282+0000 7ff22fbbc740 1 --2- 172.21.15.83:0/1372298292 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560f37baab30 0x560f37bab990 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:30.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.282+0000 7ff22fbbc740 1 --2- 172.21.15.83:0/1372298292 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560f37ba0460 0x560f37bab450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:30.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.282+0000 7ff22fbbc740 1 --2- 172.21.15.83:0/1372298292 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f37b9fb40 0x560f37baaf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:30.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.282+0000 7ff22fbbc740 1 -- 172.21.15.83:0/1372298292 >> 172.21.15.83:0/1372298292 conn(0x560f37b9be60 msgr2=0x560f37ba6940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:14:30.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.282+0000 7ff207e00640 0 client.40457 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:14:30.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.282+0000 7ff22fbbc740 1 -- 172.21.15.83:0/1372298292 shutdown_connections 2024-04-24T09:14:30.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.283+0000 7ff22fbbc740 1 -- 172.21.15.83:0/1372298292 wait complete. 2024-04-24T09:14:30.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.283+0000 7ff22fbbc740 20 client.40457 trim_cache size 0 max 16384 2024-04-24T09:14:30.283 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:14:30.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.283+0000 7ff227200640 1 -- 172.21.15.83:0/1887167580 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff20c0a3da0 msgr2=0x7ff20c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:30.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.283+0000 7ff227200640 1 --2- 172.21.15.83:0/1887167580 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff20c0a3da0 0x7ff20c0a6260 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7ff228096e80 tx=0x7ff2280684d0 comp rx=0 tx=0).stop 2024-04-24T09:14:30.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.283+0000 7ff227200640 1 -- 172.21.15.83:0/1887167580 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff220094ab0 msgr2=0x7ff22012ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:30.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.283+0000 7ff227200640 1 --2- 172.21.15.83:0/1887167580 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff220094ab0 0x7ff22012ead0 secure :-1 s=READY pgs=2678 cs=0 l=1 rev1=1 crypto rx=0x7ff21400c970 tx=0x7ff21400ce30 comp rx=0 tx=0).stop 2024-04-24T09:14:30.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.284+0000 7ff227200640 1 -- 172.21.15.83:0/1887167580 shutdown_connections 2024-04-24T09:14:30.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.284+0000 7ff227200640 1 --2- 172.21.15.83:0/1887167580 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff20c0a3da0 0x7ff20c0a6260 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:30.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.284+0000 7ff227200640 1 --2- 172.21.15.83:0/1887167580 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff2200a26d0 0x7ff220132e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:30.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.284+0000 7ff227200640 1 --2- 172.21.15.83:0/1887167580 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff220094ab0 0x7ff22012ead0 unknown :-1 s=CLOSED pgs=2678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:30.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.284+0000 7ff227200640 1 --2- 172.21.15.83:0/1887167580 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff220094190 0x7ff22012e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:30.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.284+0000 7ff227200640 1 -- 172.21.15.83:0/1887167580 >> 172.21.15.83:0/1887167580 conn(0x7ff220009a80 msgr2=0x7ff2200a2bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:14:30.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.284+0000 7ff227200640 1 -- 172.21.15.83:0/1887167580 shutdown_connections 2024-04-24T09:14:30.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:30.284+0000 7ff227200640 1 -- 172.21.15.83:0/1887167580 wait complete. 2024-04-24T09:14:30.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:14:30 smithi138 ceph-mon[31441]: pgmap v723: 129 pgs: 129 active+clean; 1010 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.1 MiB/s wr, 364 op/s 2024-04-24T09:14:30.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:14:30 smithi189 ceph-mon[31791]: pgmap v723: 129 pgs: 129 active+clean; 1010 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.1 MiB/s wr, 364 op/s 2024-04-24T09:14:30.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:14:30 smithi083 ceph-mon[27907]: pgmap v723: 129 pgs: 129 active+clean; 1010 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.1 MiB/s wr, 364 op/s 2024-04-24T09:14:31.207 INFO:tasks.workunit.client.0.smithi083.stdout: 10 66958 6.56 MB/sec execute 422 sec latency 88.722 ms 2024-04-24T09:14:31.314 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T09:14:31.315 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T09:14:31.315 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T09:14:31.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.568+0000 7fa211e00640 1 -- 172.21.15.83:0/4179916149 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa20c1078c0 msgr2=0x7fa20c100570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:31.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.568+0000 7fa211e00640 1 --2- 172.21.15.83:0/4179916149 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa20c1078c0 0x7fa20c100570 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc009900 tx=0x7fa1fc02a3f0 comp rx=0 tx=0).stop 2024-04-24T09:14:31.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.568+0000 7fa211e00640 1 -- 172.21.15.83:0/4179916149 shutdown_connections 2024-04-24T09:14:31.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.568+0000 7fa211e00640 1 --2- 172.21.15.83:0/4179916149 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa20c100c50 0x7fa20c104a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:31.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.568+0000 7fa211e00640 1 --2- 172.21.15.83:0/4179916149 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa20c1078c0 0x7fa20c100570 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:31.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.568+0000 7fa211e00640 1 --2- 172.21.15.83:0/4179916149 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa20c106f10 0x7fa20c1072f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:31.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.568+0000 7fa211e00640 1 -- 172.21.15.83:0/4179916149 >> 172.21.15.83:0/4179916149 conn(0x7fa20c077500 msgr2=0x7fa20c0fe880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:14:31.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.568+0000 7fa211e00640 1 -- 172.21.15.83:0/4179916149 shutdown_connections 2024-04-24T09:14:31.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.569+0000 7fa211e00640 1 -- 172.21.15.83:0/4179916149 wait complete. 2024-04-24T09:14:31.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.570+0000 7fa211e00640 1 Processor -- start 2024-04-24T09:14:31.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.570+0000 7fa211e00640 1 -- start start 2024-04-24T09:14:31.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.570+0000 7fa211e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa20c100c50 0x7fa20c115ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:31.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.570+0000 7fa211e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa20c106f10 0x7fa20c10eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:31.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.570+0000 7fa211e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa20c1078c0 0x7fa20c10f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:31.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.571+0000 7fa211e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa20c1188d0 con 0x7fa20c1078c0 2024-04-24T09:14:31.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.571+0000 7fa211e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa20c118750 con 0x7fa20c100c50 2024-04-24T09:14:31.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.571+0000 7fa211e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa20c118a50 con 0x7fa20c106f10 2024-04-24T09:14:31.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.571+0000 7fa20aa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa20c100c50 0x7fa20c115ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:31.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.571+0000 7fa20aa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa20c100c50 0x7fa20c115ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:35574/0 (socket says 172.21.15.83:35574) 2024-04-24T09:14:31.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.571+0000 7fa20b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa20c1078c0 0x7fa20c10f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:31.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.571+0000 7fa20aa00640 1 -- 172.21.15.83:0/2532501017 learned_addr learned my addr 172.21.15.83:0/2532501017 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:14:31.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.571+0000 7fa20a000640 1 --2- 172.21.15.83:0/2532501017 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa20c106f10 0x7fa20c10eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:31.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.572+0000 7fa20aa00640 1 -- 172.21.15.83:0/2532501017 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa20c106f10 msgr2=0x7fa20c10eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:31.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.572+0000 7fa20aa00640 1 --2- 172.21.15.83:0/2532501017 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa20c106f10 0x7fa20c10eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:31.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.572+0000 7fa20aa00640 1 -- 172.21.15.83:0/2532501017 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa20c1078c0 msgr2=0x7fa20c10f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:31.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.572+0000 7fa20aa00640 1 --2- 172.21.15.83:0/2532501017 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa20c1078c0 0x7fa20c10f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:31.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.572+0000 7fa20aa00640 1 -- 172.21.15.83:0/2532501017 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa20c10f680 con 0x7fa20c100c50 2024-04-24T09:14:31.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.572+0000 7fa20a000640 1 --2- 172.21.15.83:0/2532501017 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa20c106f10 0x7fa20c10eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:14:31.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.572+0000 7fa20b400640 1 --2- 172.21.15.83:0/2532501017 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa20c1078c0 0x7fa20c10f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:14:31.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.572+0000 7fa20aa00640 1 --2- 172.21.15.83:0/2532501017 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa20c100c50 0x7fa20c115ad0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fa1f400e9f0 tx=0x7fa1f400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:14:31.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.572+0000 7fa210a00640 1 -- 172.21.15.83:0/2532501017 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f400cd80 con 0x7fa20c100c50 2024-04-24T09:14:31.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.573+0000 7fa210a00640 1 -- 172.21.15.83:0/2532501017 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa1f40054d0 con 0x7fa20c100c50 2024-04-24T09:14:31.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.573+0000 7fa210a00640 1 -- 172.21.15.83:0/2532501017 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f4010490 con 0x7fa20c100c50 2024-04-24T09:14:31.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.573+0000 7fa211e00640 1 -- 172.21.15.83:0/2532501017 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa20c10f970 con 0x7fa20c100c50 2024-04-24T09:14:31.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.573+0000 7fa211e00640 1 -- 172.21.15.83:0/2532501017 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa20c113970 con 0x7fa20c100c50 2024-04-24T09:14:31.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.577+0000 7fa210a00640 1 -- 172.21.15.83:0/2532501017 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa1f4005020 con 0x7fa20c100c50 2024-04-24T09:14:31.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.578+0000 7fa214be0740 1 Processor -- start 2024-04-24T09:14:31.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.579+0000 7fa214be0740 1 -- start start 2024-04-24T09:14:31.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.579+0000 7fa214be0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558877063090 0x558877063470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:31.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.579+0000 7fa214be0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588770639b0 0x55887706db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:31.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.579+0000 7fa214be0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55887706e080 0x558877070470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:31.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.579+0000 7fa214be0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x558876fc8e70 con 0x5588770639b0 2024-04-24T09:14:31.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.579+0000 7fa214be0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558876f6f380 con 0x558877063090 2024-04-24T09:14:31.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.579+0000 7fa214be0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x558876fc4e30 con 0x55887706e080 2024-04-24T09:14:31.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.579+0000 7fa210a00640 1 --2- 172.21.15.83:0/2532501017 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa1f00a3e20 0x7fa1f00a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:31.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.580+0000 7fa20a000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55887706e080 0x558877070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:31.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.580+0000 7fa20a000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55887706e080 0x558877070470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:45310/0 (socket says 172.21.15.83:45310) 2024-04-24T09:14:31.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.580+0000 7fa20a000640 1 -- 172.21.15.83:0/678451563 learned_addr learned my addr 172.21.15.83:0/678451563 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:14:31.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.580+0000 7fa210a00640 1 -- 172.21.15.83:0/2532501017 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9583+0+0 (secure 0 0 0) 0x7fa1f40d0ff0 con 0x7fa20c100c50 2024-04-24T09:14:31.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.580+0000 7fa20a000640 1 -- 172.21.15.83:0/678451563 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558877063090 msgr2=0x558877063470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:14:31.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.580+0000 7fa20a000640 1 --2- 172.21.15.83:0/678451563 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558877063090 0x558877063470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:31.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.580+0000 7fa20a000640 1 -- 172.21.15.83:0/678451563 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588770639b0 msgr2=0x55887706db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:14:31.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.581+0000 7fa20aa00640 1 --2- 172.21.15.83:0/678451563 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588770639b0 0x55887706db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:31.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.581+0000 7fa20a000640 1 --2- 172.21.15.83:0/678451563 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588770639b0 0x55887706db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:31.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.581+0000 7fa20b400640 1 --2- 172.21.15.83:0/2532501017 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa1f00a3e20 0x7fa1f00a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:31.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.581+0000 7fa20a000640 1 -- 172.21.15.83:0/678451563 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558876da37e0 con 0x55887706e080 2024-04-24T09:14:31.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.581+0000 7fa20aa00640 1 --2- 172.21.15.83:0/678451563 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588770639b0 0x55887706db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:14:31.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.581+0000 7fa20a000640 1 --2- 172.21.15.83:0/678451563 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55887706e080 0x558877070470 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc02a910 tx=0x7fa1fc004170 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1121b12bb166cc4c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:14:31.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.582+0000 7fa1fbe00640 1 -- 172.21.15.83:0/678451563 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1fc03f070 con 0x55887706e080 2024-04-24T09:14:31.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.582+0000 7fa20b400640 1 --2- 172.21.15.83:0/2532501017 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa1f00a3e20 0x7fa1f00a62e0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fa20400f6e0 tx=0x7fa204007e30 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:14:31.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.582+0000 7fa1fbe00640 1 -- 172.21.15.83:0/678451563 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa1fc0054f0 con 0x55887706e080 2024-04-24T09:14:31.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.582+0000 7fa1fbe00640 1 -- 172.21.15.83:0/678451563 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1fc035030 con 0x55887706e080 2024-04-24T09:14:31.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.583+0000 7fa214be0740 1 -- 172.21.15.83:0/678451563 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55887706e080 msgr2=0x558877070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:31.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.583+0000 7fa214be0740 1 --2- 172.21.15.83:0/678451563 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55887706e080 0x558877070470 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc02a910 tx=0x7fa1fc004170 comp rx=0 tx=0).stop 2024-04-24T09:14:31.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.583+0000 7fa214be0740 1 -- 172.21.15.83:0/678451563 shutdown_connections 2024-04-24T09:14:31.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.583+0000 7fa214be0740 1 --2- 172.21.15.83:0/678451563 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55887706e080 0x558877070470 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:31.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.583+0000 7fa214be0740 1 --2- 172.21.15.83:0/678451563 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5588770639b0 0x55887706db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:31.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.583+0000 7fa214be0740 1 --2- 172.21.15.83:0/678451563 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558877063090 0x558877063470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:31.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.583+0000 7fa214be0740 1 -- 172.21.15.83:0/678451563 >> 172.21.15.83:0/678451563 conn(0x55887705f3b0 msgr2=0x55887705f790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:14:31.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.584+0000 7fa214be0740 1 -- 172.21.15.83:0/678451563 shutdown_connections 2024-04-24T09:14:31.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.584+0000 7fa214be0740 1 -- 172.21.15.83:0/678451563 wait complete. 2024-04-24T09:14:31.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.584+0000 7fa214be0740 1 -- start start 2024-04-24T09:14:31.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.585+0000 7fa214be0740 1 Processor -- start 2024-04-24T09:14:31.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.585+0000 7fa214be0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558877063090 0x55887706b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:31.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.586+0000 7fa214be0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5588770639b0 0x55887706b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:31.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.586+0000 7fa214be0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55887706e080 0x55887706bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:31.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.586+0000 7fa214be0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x558877070e10 con 0x558877063090 2024-04-24T09:14:31.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.586+0000 7fa20a000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5588770639b0 0x55887706b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:31.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.586+0000 7fa20a000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5588770639b0 0x55887706b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:35584/0 (socket says 172.21.15.83:35584) 2024-04-24T09:14:31.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.586+0000 7fa20a000640 1 -- 172.21.15.83:0/2884345934 learned_addr learned my addr 172.21.15.83:0/2884345934 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:14:31.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.586+0000 7fa214be0740 1 -- 172.21.15.83:0/2884345934 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558876f82660 con 0x5588770639b0 2024-04-24T09:14:31.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.586+0000 7fa214be0740 1 -- 172.21.15.83:0/2884345934 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x558877070f90 con 0x55887706e080 2024-04-24T09:14:31.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.586+0000 7fa20b400640 1 --2- 172.21.15.83:0/2884345934 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55887706e080 0x55887706bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:31.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.587+0000 7fa20aa00640 1 --2- 172.21.15.83:0/2884345934 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558877063090 0x55887706b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:31.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.587+0000 7fa214be0740 10 client.0 _ll_register_callbacks cb 0x558876f70aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:14:31.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.587+0000 7fa20a000640 1 -- 172.21.15.83:0/2884345934 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55887706e080 msgr2=0x55887706bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:14:31.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.587+0000 7fa20a000640 1 --2- 172.21.15.83:0/2884345934 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55887706e080 0x55887706bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:31.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.587+0000 7fa20a000640 1 -- 172.21.15.83:0/2884345934 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558877063090 msgr2=0x55887706b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:14:31.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.587+0000 7fa20a000640 1 --2- 172.21.15.83:0/2884345934 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558877063090 0x55887706b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:31.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.587+0000 7fa20a000640 1 -- 172.21.15.83:0/2884345934 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558876e03f70 con 0x5588770639b0 2024-04-24T09:14:31.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.588+0000 7fa20aa00640 1 --2- 172.21.15.83:0/2884345934 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558877063090 0x55887706b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:14:31.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.588+0000 7fa20a000640 1 --2- 172.21.15.83:0/2884345934 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5588770639b0 0x55887706b5d0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc031440 tx=0x7fa1fc0048f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cdfa2bf78ed3bfd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:14:31.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.588+0000 7fa1f8c00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:14:31.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.588+0000 7fa1f8c00640 1 -- 172.21.15.83:0/2884345934 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1fc04c070 con 0x5588770639b0 2024-04-24T09:14:31.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.589+0000 7fa1f8c00640 1 -- 172.21.15.83:0/2884345934 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(128..128 src has 1..128) ==== 9583+0+0 (secure 0 0 0) 0x7fa1fc004f20 con 0x5588770639b0 2024-04-24T09:14:31.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.589+0000 7fa214be0740 1 -- 172.21.15.83:0/2884345934 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1fc0054f0 con 0x5588770639b0 2024-04-24T09:14:31.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.589+0000 7fa1f8c00640 1 -- 172.21.15.83:0/2884345934 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7fa1fc056050 con 0x5588770639b0 2024-04-24T09:14:31.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.589+0000 7fa214be0740 10 client.40466 fetch_fsmap learned FSMap version 65 2024-04-24T09:14:31.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.589+0000 7fa214be0740 1 -- 172.21.15.83:0/2884345934 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x558876c9b170 con 0x5588770639b0 2024-04-24T09:14:31.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.590+0000 7fa1f8c00640 1 -- 172.21.15.83:0/2884345934 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa1fc0054f0 con 0x5588770639b0 2024-04-24T09:14:31.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.590+0000 7fa1f8c00640 1 -- 172.21.15.83:0/2884345934 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7fa1fc0314a0 con 0x5588770639b0 2024-04-24T09:14:31.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.595+0000 7fa214be0740 10 client.40466 fetch_fsmap finished waiting for FSMap version 65 2024-04-24T09:14:31.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.595+0000 7fa214be0740 10 client.40466 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:14:31.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.595+0000 7fa214be0740 20 client.40466 populate_metadata read hostname 'smithi083' 2024-04-24T09:14:31.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.595+0000 7fa214be0740 1 --2- 172.21.15.83:0/2884345934 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55887708f220 0x558877091610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:31.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.595+0000 7fa214be0740 4 client.40466 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:14:31.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.595+0000 7fa214be0740 1 -- 172.21.15.83:0/2884345934 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558876fd6890 con 0x55887708f220 2024-04-24T09:14:31.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.596+0000 7fa20aa00640 1 --2- 172.21.15.83:0/2884345934 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55887708f220 0x558877091610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:31.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.599+0000 7fa20aa00640 1 --2- 172.21.15.83:0/2884345934 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55887708f220 0x558877091610 crc :-1 s=READY pgs=565 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8b46f09f8fda5aa server_cookie=3718069e3855c121 in_seq=0 out_seq=0 2024-04-24T09:14:31.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.599+0000 7fa1f8c00640 10 client.40466 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:14:31.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.602+0000 7fa1f8c00640 1 -- 172.21.15.83:0/2884345934 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558876fd6890 con 0x55887708f220 2024-04-24T09:14:31.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.603+0000 7fa1f8c00640 10 client.40466 handle_command_reply: tid=0 2024-04-24T09:14:31.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.603+0000 7fa214be0740 1 client.40466 shutdown 2024-04-24T09:14:31.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.604+0000 7fa214be0740 1 -- 172.21.15.83:0/2884345934 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5588770639b0 msgr2=0x55887706b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:31.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.604+0000 7fa214be0740 1 --2- 172.21.15.83:0/2884345934 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5588770639b0 0x55887706b5d0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc031440 tx=0x7fa1fc0048f0 comp rx=0 tx=0).stop 2024-04-24T09:14:31.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.605+0000 7fa214be0740 1 -- 172.21.15.83:0/2884345934 shutdown_connections 2024-04-24T09:14:31.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.605+0000 7fa214be0740 1 --2- 172.21.15.83:0/2884345934 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55887708f220 0x558877091610 crc :-1 s=READY pgs=565 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:31.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.605+0000 7fa214be0740 1 --2- 172.21.15.83:0/2884345934 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55887706e080 0x55887706bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:31.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.605+0000 7fa214be0740 1 --2- 172.21.15.83:0/2884345934 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5588770639b0 0x55887706b5d0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:31.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.605+0000 7fa214be0740 1 --2- 172.21.15.83:0/2884345934 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558877063090 0x55887706b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:31.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.605+0000 7fa214be0740 1 -- 172.21.15.83:0/2884345934 >> 172.21.15.83:0/2884345934 conn(0x55887705f3b0 msgr2=0x5588770650b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:14:31.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.605+0000 7fa1f8c00640 0 client.40466 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:14:31.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.605+0000 7fa214be0740 1 -- 172.21.15.83:0/2884345934 shutdown_connections 2024-04-24T09:14:31.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.606+0000 7fa214be0740 1 -- 172.21.15.83:0/2884345934 wait complete. 2024-04-24T09:14:31.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.606+0000 7fa214be0740 20 client.40466 trim_cache size 0 max 16384 2024-04-24T09:14:31.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.635+0000 7fa214be0740 1 Processor -- start 2024-04-24T09:14:31.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.635+0000 7fa214be0740 1 -- start start 2024-04-24T09:14:31.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.635+0000 7fa214be0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558877063090 0x55887705ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:31.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.635+0000 7fa214be0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588770639b0 0x558877096c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:31.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.635+0000 7fa214be0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55887706e080 0x558877097150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:31.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.635+0000 7fa214be0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa20c1b5b40 con 0x55887706e080 2024-04-24T09:14:31.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.635+0000 7fa214be0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa20c1b76e0 con 0x558877063090 2024-04-24T09:14:31.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.635+0000 7fa214be0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa20c1b6210 con 0x5588770639b0 2024-04-24T09:14:31.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.636+0000 7fa20b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55887706e080 0x558877097150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:31.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.636+0000 7fa20b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55887706e080 0x558877097150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37290/0 (socket says 172.21.15.83:37290) 2024-04-24T09:14:31.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.636+0000 7fa20b400640 1 -- 172.21.15.83:0/1631191548 learned_addr learned my addr 172.21.15.83:0/1631191548 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:14:31.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.636+0000 7fa20aa00640 1 --2- 172.21.15.83:0/1631191548 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558877063090 0x55887705ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:31.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.636+0000 7fa20a000640 1 --2- 172.21.15.83:0/1631191548 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588770639b0 0x558877096c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:31.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.637+0000 7fa20b400640 1 -- 172.21.15.83:0/1631191548 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588770639b0 msgr2=0x558877096c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:14:31.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.637+0000 7fa20b400640 1 --2- 172.21.15.83:0/1631191548 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588770639b0 0x558877096c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:31.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.637+0000 7fa20b400640 1 -- 172.21.15.83:0/1631191548 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558877063090 msgr2=0x55887705ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:14:31.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.637+0000 7fa20b400640 1 --2- 172.21.15.83:0/1631191548 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558877063090 0x55887705ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:31.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.637+0000 7fa20b400640 1 -- 172.21.15.83:0/1631191548 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588770707e0 con 0x55887706e080 2024-04-24T09:14:31.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.637+0000 7fa20aa00640 1 --2- 172.21.15.83:0/1631191548 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558877063090 0x55887705ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:14:31.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.637+0000 7fa20b400640 1 --2- 172.21.15.83:0/1631191548 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55887706e080 0x558877097150 secure :-1 s=READY pgs=2681 cs=0 l=1 rev1=1 crypto rx=0x7fa204010ae0 tx=0x7fa204012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fe202431e9c7c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:14:31.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.638+0000 7fa1fb400640 1 -- 172.21.15.83:0/1631191548 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa204004b20 con 0x55887706e080 2024-04-24T09:14:31.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.638+0000 7fa1fb400640 1 -- 172.21.15.83:0/1631191548 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa204004cc0 con 0x55887706e080 2024-04-24T09:14:31.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.638+0000 7fa1fb400640 1 -- 172.21.15.83:0/1631191548 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa204005470 con 0x55887706e080 2024-04-24T09:14:31.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.639+0000 7fa214be0740 1 -- 172.21.15.83:0/1631191548 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55887706e080 msgr2=0x558877097150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:31.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.639+0000 7fa214be0740 1 --2- 172.21.15.83:0/1631191548 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55887706e080 0x558877097150 secure :-1 s=READY pgs=2681 cs=0 l=1 rev1=1 crypto rx=0x7fa204010ae0 tx=0x7fa204012000 comp rx=0 tx=0).stop 2024-04-24T09:14:31.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.639+0000 7fa214be0740 1 -- 172.21.15.83:0/1631191548 shutdown_connections 2024-04-24T09:14:31.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.639+0000 7fa214be0740 1 --2- 172.21.15.83:0/1631191548 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55887706e080 0x558877097150 unknown :-1 s=CLOSED pgs=2681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:31.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.639+0000 7fa214be0740 1 --2- 172.21.15.83:0/1631191548 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588770639b0 0x558877096c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:31.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.639+0000 7fa214be0740 1 --2- 172.21.15.83:0/1631191548 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558877063090 0x55887705ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:31.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.639+0000 7fa214be0740 1 -- 172.21.15.83:0/1631191548 >> 172.21.15.83:0/1631191548 conn(0x55887705f3b0 msgr2=0x55887706b8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:14:31.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.639+0000 7fa214be0740 1 -- 172.21.15.83:0/1631191548 shutdown_connections 2024-04-24T09:14:31.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.640+0000 7fa214be0740 1 -- 172.21.15.83:0/1631191548 wait complete. 2024-04-24T09:14:31.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.640+0000 7fa214be0740 1 -- start start 2024-04-24T09:14:31.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.640+0000 7fa214be0740 1 Processor -- start 2024-04-24T09:14:31.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.642+0000 7fa214be0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558877063090 0x558877067d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:31.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.642+0000 7fa214be0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5588770639b0 0x558877068260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:31.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.642+0000 7fa214be0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55887706e080 0x5588770687a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:31.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.642+0000 7fa20aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558877063090 0x558877067d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:31.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.642+0000 7fa20aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558877063090 0x558877067d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37292/0 (socket says 172.21.15.83:37292) 2024-04-24T09:14:31.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.643+0000 7fa20aa00640 1 -- 172.21.15.83:0/1978990032 learned_addr learned my addr 172.21.15.83:0/1978990032 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:14:31.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.643+0000 7fa214be0740 1 -- 172.21.15.83:0/1978990032 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa20c1b6fe0 con 0x558877063090 2024-04-24T09:14:31.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.643+0000 7fa214be0740 1 -- 172.21.15.83:0/1978990032 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa20c1b6910 con 0x5588770639b0 2024-04-24T09:14:31.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.643+0000 7fa214be0740 1 -- 172.21.15.83:0/1978990032 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa20c101580 con 0x55887706e080 2024-04-24T09:14:31.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.643+0000 7fa214be0740 10 client.0 _ll_register_callbacks cb 0x7fa1fc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:14:31.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.643+0000 7fa20b400640 1 --2- 172.21.15.83:0/1978990032 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55887706e080 0x5588770687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:31.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.643+0000 7fa20a000640 1 --2- 172.21.15.83:0/1978990032 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5588770639b0 0x558877068260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:31.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.644+0000 7fa20a000640 1 -- 172.21.15.83:0/1978990032 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55887706e080 msgr2=0x5588770687a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:14:31.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.644+0000 7fa20a000640 1 --2- 172.21.15.83:0/1978990032 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55887706e080 0x5588770687a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:31.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.644+0000 7fa20a000640 1 -- 172.21.15.83:0/1978990032 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558877063090 msgr2=0x558877067d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:14:31.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.644+0000 7fa20a000640 1 --2- 172.21.15.83:0/1978990032 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558877063090 0x558877067d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:31.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.644+0000 7fa20a000640 1 -- 172.21.15.83:0/1978990032 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588770f4480 con 0x5588770639b0 2024-04-24T09:14:31.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.645+0000 7fa20a000640 1 --2- 172.21.15.83:0/1978990032 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5588770639b0 0x558877068260 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc005ab0 tx=0x7fa1fc004c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a176ac37e670e15a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:14:31.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.645+0000 7fa1f8c00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:14:31.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.645+0000 7fa1f8c00640 1 -- 172.21.15.83:0/1978990032 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1fc060060 con 0x5588770639b0 2024-04-24T09:14:31.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.645+0000 7fa1f8c00640 1 -- 172.21.15.83:0/1978990032 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(128..128 src has 1..128) ==== 9583+0+0 (secure 0 0 0) 0x7fa1fc031d30 con 0x5588770639b0 2024-04-24T09:14:31.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.645+0000 7fa214be0740 1 -- 172.21.15.83:0/1978990032 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa204004cc0 con 0x5588770639b0 2024-04-24T09:14:31.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.645+0000 7fa1f8c00640 1 -- 172.21.15.83:0/1978990032 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7fa1fc067050 con 0x5588770639b0 2024-04-24T09:14:31.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.645+0000 7fa214be0740 10 client.40472 fetch_fsmap learned FSMap version 65 2024-04-24T09:14:31.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.645+0000 7fa214be0740 1 -- 172.21.15.83:0/1978990032 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x5588770f5760 con 0x5588770639b0 2024-04-24T09:14:31.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.646+0000 7fa1f8c00640 1 -- 172.21.15.83:0/1978990032 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa204004cc0 con 0x5588770639b0 2024-04-24T09:14:31.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.646+0000 7fa1f8c00640 1 -- 172.21.15.83:0/1978990032 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7fa1fc003a40 con 0x5588770639b0 2024-04-24T09:14:31.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.646+0000 7fa214be0740 10 client.40472 fetch_fsmap finished waiting for FSMap version 65 2024-04-24T09:14:31.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.646+0000 7fa214be0740 10 client.40472 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:14:31.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.646+0000 7fa214be0740 20 client.40472 populate_metadata read hostname 'smithi083' 2024-04-24T09:14:31.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.646+0000 7fa214be0740 1 --2- 172.21.15.83:0/1978990032 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5588770f5f40 0x5588770f8330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:31.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.646+0000 7fa214be0740 4 client.40472 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T09:14:31.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.646+0000 7fa214be0740 1 -- 172.21.15.83:0/1978990032 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x558876fd0700 con 0x5588770f5f40 2024-04-24T09:14:31.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.646+0000 7fa20aa00640 1 --2- 172.21.15.83:0/1978990032 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5588770f5f40 0x5588770f8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:31.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.647+0000 7fa1f8c00640 10 client.40472 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:14:31.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.647+0000 7fa20aa00640 1 --2- 172.21.15.83:0/1978990032 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5588770f5f40 0x5588770f8330 crc :-1 s=READY pgs=566 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a92b1135704414ae server_cookie=4d765ae7f4bc551e in_seq=0 out_seq=0 2024-04-24T09:14:31.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.648+0000 7fa1f8c00640 1 -- 172.21.15.83:0/1978990032 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x558876fd0700 con 0x5588770f5f40 2024-04-24T09:14:31.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.648+0000 7fa1f8c00640 10 client.40472 handle_command_reply: tid=0 2024-04-24T09:14:31.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.648+0000 7fa214be0740 1 client.40472 shutdown 2024-04-24T09:14:31.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.649+0000 7fa214be0740 1 -- 172.21.15.83:0/1978990032 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5588770639b0 msgr2=0x558877068260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:31.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.649+0000 7fa214be0740 1 --2- 172.21.15.83:0/1978990032 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5588770639b0 0x558877068260 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc005ab0 tx=0x7fa1fc004c60 comp rx=0 tx=0).stop 2024-04-24T09:14:31.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.649+0000 7fa214be0740 1 -- 172.21.15.83:0/1978990032 shutdown_connections 2024-04-24T09:14:31.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.649+0000 7fa214be0740 1 --2- 172.21.15.83:0/1978990032 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5588770f5f40 0x5588770f8330 crc :-1 s=READY pgs=566 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:31.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.649+0000 7fa214be0740 1 --2- 172.21.15.83:0/1978990032 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55887706e080 0x5588770687a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:31.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.649+0000 7fa214be0740 1 --2- 172.21.15.83:0/1978990032 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5588770639b0 0x558877068260 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:31.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.649+0000 7fa214be0740 1 --2- 172.21.15.83:0/1978990032 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558877063090 0x558877067d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:31.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.649+0000 7fa1f8c00640 0 client.40472 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:14:31.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.649+0000 7fa214be0740 1 -- 172.21.15.83:0/1978990032 >> 172.21.15.83:0/1978990032 conn(0x55887705f3b0 msgr2=0x55887706c8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:14:31.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.650+0000 7fa214be0740 1 -- 172.21.15.83:0/1978990032 shutdown_connections 2024-04-24T09:14:31.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.650+0000 7fa214be0740 1 -- 172.21.15.83:0/1978990032 wait complete. 2024-04-24T09:14:31.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.650+0000 7fa214be0740 20 client.40472 trim_cache size 0 max 16384 2024-04-24T09:14:31.651 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T09:14:31.651 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-04-24T09:14:31.651 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "eccb2691-2d88-47d9-a790-071cb67c2725", 2024-04-24T09:14:31.651 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-04-24T09:14:31.651 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T09:14:31.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.652+0000 7fa211e00640 1 -- 172.21.15.83:0/2532501017 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa1f00a3e20 msgr2=0x7fa1f00a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:31.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.652+0000 7fa211e00640 1 --2- 172.21.15.83:0/2532501017 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa1f00a3e20 0x7fa1f00a62e0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fa20400f6e0 tx=0x7fa204007e30 comp rx=0 tx=0).stop 2024-04-24T09:14:31.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.652+0000 7fa211e00640 1 -- 172.21.15.83:0/2532501017 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa20c100c50 msgr2=0x7fa20c115ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:31.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.652+0000 7fa211e00640 1 --2- 172.21.15.83:0/2532501017 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa20c100c50 0x7fa20c115ad0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fa1f400e9f0 tx=0x7fa1f400edf0 comp rx=0 tx=0).stop 2024-04-24T09:14:31.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.652+0000 7fa211e00640 1 -- 172.21.15.83:0/2532501017 shutdown_connections 2024-04-24T09:14:31.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.652+0000 7fa211e00640 1 --2- 172.21.15.83:0/2532501017 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa1f00a3e20 0x7fa1f00a62e0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:31.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.652+0000 7fa211e00640 1 --2- 172.21.15.83:0/2532501017 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa20c1078c0 0x7fa20c10f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:31.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.652+0000 7fa211e00640 1 --2- 172.21.15.83:0/2532501017 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa20c106f10 0x7fa20c10eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:31.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.652+0000 7fa211e00640 1 --2- 172.21.15.83:0/2532501017 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa20c100c50 0x7fa20c115ad0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:31.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.652+0000 7fa211e00640 1 -- 172.21.15.83:0/2532501017 >> 172.21.15.83:0/2532501017 conn(0x7fa20c077500 msgr2=0x7fa20c102aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:14:31.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.652+0000 7fa211e00640 1 -- 172.21.15.83:0/2532501017 shutdown_connections 2024-04-24T09:14:31.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.652+0000 7fa211e00640 1 -- 172.21.15.83:0/2532501017 wait complete. 2024-04-24T09:14:31.674 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T09:14:31.674 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T09:14:31.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.940+0000 7f7ffb200640 1 -- 172.21.15.83:0/1371746952 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ffc100c70 msgr2=0x7f7ffc101050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:31.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.940+0000 7f7ffb200640 1 --2- 172.21.15.83:0/1371746952 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ffc100c70 0x7f7ffc101050 secure :-1 s=READY pgs=2682 cs=0 l=1 rev1=1 crypto rx=0x7f7fec009ca0 tx=0x7f7fec02a8e0 comp rx=0 tx=0).stop 2024-04-24T09:14:31.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.941+0000 7f7ffb200640 1 -- 172.21.15.83:0/1371746952 shutdown_connections 2024-04-24T09:14:31.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.941+0000 7f7ffb200640 1 --2- 172.21.15.83:0/1371746952 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ffc10e820 0x7f7ffc110c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:31.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.941+0000 7f7ffb200640 1 --2- 172.21.15.83:0/1371746952 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7ffc101620 0x7f7ffc10e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:31.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.941+0000 7f7ffb200640 1 --2- 172.21.15.83:0/1371746952 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ffc100c70 0x7f7ffc101050 unknown :-1 s=CLOSED pgs=2682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:31.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.941+0000 7f7ffb200640 1 -- 172.21.15.83:0/1371746952 >> 172.21.15.83:0/1371746952 conn(0x7f7ffc0fc880 msgr2=0x7f7ffc0feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:14:31.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.941+0000 7f7ffb200640 1 -- 172.21.15.83:0/1371746952 shutdown_connections 2024-04-24T09:14:31.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.941+0000 7f7ffb200640 1 -- 172.21.15.83:0/1371746952 wait complete. 2024-04-24T09:14:31.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.943+0000 7f7ffb200640 1 Processor -- start 2024-04-24T09:14:31.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.944+0000 7f7ffb200640 1 -- start start 2024-04-24T09:14:31.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.945+0000 7f7ffb200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ffc100c70 0x7f7ffc1a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:31.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.946+0000 7f7ffb200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7ffc101620 0x7f7ffc1a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:31.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.946+0000 7f7ffb200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ffc10e820 0x7f7ffc19a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:31.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.946+0000 7f7ffb200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7ffc114410 con 0x7f7ffc100c70 2024-04-24T09:14:31.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.946+0000 7f7ffb200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7ffc114290 con 0x7f7ffc10e820 2024-04-24T09:14:31.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.946+0000 7f7ffb200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7ffc114590 con 0x7f7ffc101620 2024-04-24T09:14:31.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.946+0000 7f7ff1400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7ffc101620 0x7f7ffc1a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:31.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.947+0000 7f7ff1400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7ffc101620 0x7f7ffc1a0bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:45354/0 (socket says 172.21.15.83:45354) 2024-04-24T09:14:31.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.947+0000 7f7ff1400640 1 -- 172.21.15.83:0/2940332070 learned_addr learned my addr 172.21.15.83:0/2940332070 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:14:31.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.947+0000 7f7ffa800640 1 --2- 172.21.15.83:0/2940332070 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ffc10e820 0x7f7ffc19a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:31.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.947+0000 7f7ff9e00640 1 --2- 172.21.15.83:0/2940332070 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ffc100c70 0x7f7ffc1a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:31.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.947+0000 7f7ffa800640 1 -- 172.21.15.83:0/2940332070 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7ffc101620 msgr2=0x7f7ffc1a0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:31.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.947+0000 7f7ffa800640 1 --2- 172.21.15.83:0/2940332070 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7ffc101620 0x7f7ffc1a0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:31.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.947+0000 7f7ffa800640 1 -- 172.21.15.83:0/2940332070 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ffc100c70 msgr2=0x7f7ffc1a06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:31.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.947+0000 7f7ffa800640 1 --2- 172.21.15.83:0/2940332070 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ffc100c70 0x7f7ffc1a06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:31.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.947+0000 7f7ffa800640 1 -- 172.21.15.83:0/2940332070 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ffc19b030 con 0x7f7ffc10e820 2024-04-24T09:14:31.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.948+0000 7f7ffa800640 1 --2- 172.21.15.83:0/2940332070 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ffc10e820 0x7f7ffc19a8b0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc06b6b0 tx=0x7f7ff400b870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:14:31.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.949+0000 7f7ff3400640 1 -- 172.21.15.83:0/2940332070 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff4015020 con 0x7f7ffc10e820 2024-04-24T09:14:31.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.949+0000 7f7ff3400640 1 -- 172.21.15.83:0/2940332070 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7ff4011ce0 con 0x7f7ffc10e820 2024-04-24T09:14:31.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.949+0000 7f7ff3400640 1 -- 172.21.15.83:0/2940332070 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff4010d50 con 0x7f7ffc10e820 2024-04-24T09:14:31.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.949+0000 7f7ffb200640 1 -- 172.21.15.83:0/2940332070 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ffc19b320 con 0x7f7ffc10e820 2024-04-24T09:14:31.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.949+0000 7f7ffb200640 1 -- 172.21.15.83:0/2940332070 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ffc1a7570 con 0x7f7ffc10e820 2024-04-24T09:14:31.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.953+0000 7f8004029740 1 Processor -- start 2024-04-24T09:14:31.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.955+0000 7f7ff3400640 1 -- 172.21.15.83:0/2940332070 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7ff4018020 con 0x7f7ffc10e820 2024-04-24T09:14:31.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.955+0000 7f8004029740 1 -- start start 2024-04-24T09:14:31.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.955+0000 7f8004029740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a6fd08f780 0x55a6fd08fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:31.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.955+0000 7f8004029740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a6fd090130 0x55a6fd0a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:31.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.955+0000 7f8004029740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a6fd0a3230 0x55a6fd0a5620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:31.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.955+0000 7f8004029740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a6fcff51f0 con 0x55a6fd08f780 2024-04-24T09:14:31.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.955+0000 7f8004029740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a6fcf9b350 con 0x55a6fd0a3230 2024-04-24T09:14:31.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.955+0000 7f8004029740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a6fcff11b0 con 0x55a6fd090130 2024-04-24T09:14:31.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.956+0000 7f7ff1400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a6fd08f780 0x55a6fd08fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:31.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.956+0000 7f7ff1400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a6fd08f780 0x55a6fd08fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37316/0 (socket says 172.21.15.83:37316) 2024-04-24T09:14:31.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.956+0000 7f7ff1400640 1 -- 172.21.15.83:0/3949723943 learned_addr learned my addr 172.21.15.83:0/3949723943 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:14:31.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.956+0000 7f7ff1400640 1 --2- 172.21.15.83:0/3949723943 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a6fd0a3230 0x55a6fd0a5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:31.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.956+0000 7f7ff3400640 1 --2- 172.21.15.83:0/2940332070 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f7fe00a3e60 0x7f7fe00a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:31.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.957+0000 7f7ff1400640 1 -- 172.21.15.83:0/3949723943 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a6fd090130 msgr2=0x55a6fd0a2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:14:31.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.957+0000 7f7ff1400640 1 --2- 172.21.15.83:0/3949723943 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a6fd090130 0x55a6fd0a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:31.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.957+0000 7f7ff1400640 1 -- 172.21.15.83:0/3949723943 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a6fd08f780 msgr2=0x55a6fd08fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:14:31.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.957+0000 7f7ff1400640 1 --2- 172.21.15.83:0/3949723943 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a6fd08f780 0x55a6fd08fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:31.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.957+0000 7f7ff1400640 1 -- 172.21.15.83:0/3949723943 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6fcfeff50 con 0x55a6fd0a3230 2024-04-24T09:14:31.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.957+0000 7f7ff3400640 1 -- 172.21.15.83:0/2940332070 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9583+0+0 (secure 0 0 0) 0x7f7ff40d02d0 con 0x7f7ffc10e820 2024-04-24T09:14:31.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.957+0000 7f7ff1400640 1 --2- 172.21.15.83:0/3949723943 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a6fd0a3230 0x55a6fd0a5620 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f7fe400bfd0 tx=0x7f7fe400b410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4799a238d06d1ddf server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:14:31.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.957+0000 7f7ffa800640 1 --2- 172.21.15.83:0/2940332070 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f7fe00a3e60 0x7f7fe00a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:31.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.957+0000 7f7feb400640 1 -- 172.21.15.83:0/3949723943 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fe4005020 con 0x55a6fd0a3230 2024-04-24T09:14:31.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.958+0000 7f7feb400640 1 -- 172.21.15.83:0/3949723943 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7fe40047a0 con 0x55a6fd0a3230 2024-04-24T09:14:31.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.958+0000 7f7feb400640 1 -- 172.21.15.83:0/3949723943 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fe4009900 con 0x55a6fd0a3230 2024-04-24T09:14:31.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.959+0000 7f8004029740 1 -- 172.21.15.83:0/3949723943 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a6fd0a3230 msgr2=0x55a6fd0a5620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:31.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.959+0000 7f8004029740 1 --2- 172.21.15.83:0/3949723943 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a6fd0a3230 0x55a6fd0a5620 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f7fe400bfd0 tx=0x7f7fe400b410 comp rx=0 tx=0).stop 2024-04-24T09:14:31.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.959+0000 7f7ffa800640 1 --2- 172.21.15.83:0/2940332070 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f7fe00a3e60 0x7f7fe00a6320 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f7ff40cd710 tx=0x7f7ff40046e0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:14:31.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.959+0000 7f8004029740 1 -- 172.21.15.83:0/3949723943 shutdown_connections 2024-04-24T09:14:31.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.959+0000 7f8004029740 1 --2- 172.21.15.83:0/3949723943 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a6fd0a3230 0x55a6fd0a5620 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:31.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.959+0000 7f8004029740 1 --2- 172.21.15.83:0/3949723943 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a6fd090130 0x55a6fd0a2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:31.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.959+0000 7f8004029740 1 --2- 172.21.15.83:0/3949723943 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a6fd08f780 0x55a6fd08fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:31.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.959+0000 7f8004029740 1 -- 172.21.15.83:0/3949723943 >> 172.21.15.83:0/3949723943 conn(0x55a6fd08bd50 msgr2=0x55a6fd08c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:14:31.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.959+0000 7f8004029740 1 -- 172.21.15.83:0/3949723943 shutdown_connections 2024-04-24T09:14:31.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.959+0000 7f8004029740 1 -- 172.21.15.83:0/3949723943 wait complete. 2024-04-24T09:14:31.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.960+0000 7f8004029740 1 -- start start 2024-04-24T09:14:31.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.960+0000 7f8004029740 1 Processor -- start 2024-04-24T09:14:31.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.961+0000 7f8004029740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a6fd08f780 0x55a6fd0a02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:31.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.961+0000 7f8004029740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a6fd090130 0x55a6fd0a07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:31.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.961+0000 7f8004029740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a6fd0a3230 0x55a6fd0a0d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:31.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.961+0000 7f8004029740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a6fd0a5ff0 con 0x55a6fd08f780 2024-04-24T09:14:31.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.961+0000 7f8004029740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a6fcfae690 con 0x55a6fd0a3230 2024-04-24T09:14:31.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.961+0000 7f8004029740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a6fd0a6170 con 0x55a6fd090130 2024-04-24T09:14:31.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.961+0000 7f8004029740 10 client.0 _ll_register_callbacks cb 0x55a6fcfea650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:14:31.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.961+0000 7f7ff1400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a6fd08f780 0x55a6fd0a02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:31.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.961+0000 7f7ff1400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a6fd08f780 0x55a6fd0a02a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37330/0 (socket says 172.21.15.83:37330) 2024-04-24T09:14:31.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.961+0000 7f7ff9e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a6fd090130 0x55a6fd0a07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:31.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.961+0000 7f7ff1400640 1 -- 172.21.15.83:0/2668709708 learned_addr learned my addr 172.21.15.83:0/2668709708 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:14:31.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.962+0000 7f7ff1400640 1 --2- 172.21.15.83:0/2668709708 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a6fd0a3230 0x55a6fd0a0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:31.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.962+0000 7f7ff1400640 1 -- 172.21.15.83:0/2668709708 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a6fd090130 msgr2=0x55a6fd0a07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:14:31.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.962+0000 7f7ff1400640 1 --2- 172.21.15.83:0/2668709708 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a6fd090130 0x55a6fd0a07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:31.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.962+0000 7f7ff1400640 1 -- 172.21.15.83:0/2668709708 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a6fd0a3230 msgr2=0x55a6fd0a0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:14:31.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.962+0000 7f7ff1400640 1 --2- 172.21.15.83:0/2668709708 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a6fd0a3230 0x55a6fd0a0d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:31.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.962+0000 7f7ff1400640 1 -- 172.21.15.83:0/2668709708 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6fcfad920 con 0x55a6fd08f780 2024-04-24T09:14:31.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.963+0000 7f7ff1400640 1 --2- 172.21.15.83:0/2668709708 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a6fd08f780 0x55a6fd0a02a0 secure :-1 s=READY pgs=2683 cs=0 l=1 rev1=1 crypto rx=0x7f7fe400b6c0 tx=0x7f7fe40123f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c738ffbfc15c48b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:14:31.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.963+0000 7f7fdfe00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:14:31.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.963+0000 7f7fdfe00640 1 -- 172.21.15.83:0/2668709708 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fe4016070 con 0x55a6fd08f780 2024-04-24T09:14:31.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.963+0000 7f7fdfe00640 1 -- 172.21.15.83:0/2668709708 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(128..128 src has 1..128) ==== 9583+0+0 (secure 0 0 0) 0x7f7fe4004fa0 con 0x55a6fd08f780 2024-04-24T09:14:31.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.963+0000 7f8004029740 1 -- 172.21.15.83:0/2668709708 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7fe40047a0 con 0x55a6fd08f780 2024-04-24T09:14:31.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.964+0000 7f7fdfe00640 1 -- 172.21.15.83:0/2668709708 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f7fe4031050 con 0x55a6fd08f780 2024-04-24T09:14:31.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.964+0000 7f8004029740 10 client.23415 fetch_fsmap learned FSMap version 65 2024-04-24T09:14:31.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.964+0000 7f8004029740 1 -- 172.21.15.83:0/2668709708 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x55a6fcdd17d0 con 0x55a6fd08f780 2024-04-24T09:14:31.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.964+0000 7f7fdfe00640 1 -- 172.21.15.83:0/2668709708 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7fe40047a0 con 0x55a6fd08f780 2024-04-24T09:14:31.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.965+0000 7f7fdfe00640 1 -- 172.21.15.83:0/2668709708 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7f7fe4009ba0 con 0x55a6fd08f780 2024-04-24T09:14:31.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.965+0000 7f8004029740 10 client.23415 fetch_fsmap finished waiting for FSMap version 65 2024-04-24T09:14:31.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.965+0000 7f8004029740 10 client.23415 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:14:31.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.965+0000 7f8004029740 20 client.23415 populate_metadata read hostname 'smithi083' 2024-04-24T09:14:31.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.965+0000 7f8004029740 1 --2- 172.21.15.83:0/2668709708 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55a6fd0b3580 0x55a6fd0b5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:31.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.965+0000 7f8004029740 4 client.23415 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:14:31.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.965+0000 7f8004029740 1 -- 172.21.15.83:0/2668709708 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a6fd002c10 con 0x55a6fd0b3580 2024-04-24T09:14:31.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.965+0000 7f7ff9e00640 1 --2- 172.21.15.83:0/2668709708 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55a6fd0b3580 0x55a6fd0b5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:31.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.966+0000 7f7ff9e00640 1 --2- 172.21.15.83:0/2668709708 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55a6fd0b3580 0x55a6fd0b5970 crc :-1 s=READY pgs=567 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=250cf114d51bd6ba server_cookie=e636345348755744 in_seq=0 out_seq=0 2024-04-24T09:14:31.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.966+0000 7f7fdfe00640 10 client.23415 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:14:31.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.968+0000 7f7fdfe00640 1 -- 172.21.15.83:0/2668709708 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a6fd002c10 con 0x55a6fd0b3580 2024-04-24T09:14:31.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.968+0000 7f7fdfe00640 10 client.23415 handle_command_reply: tid=0 2024-04-24T09:14:31.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.968+0000 7f8004029740 1 client.23415 shutdown 2024-04-24T09:14:31.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.968+0000 7f8004029740 1 -- 172.21.15.83:0/2668709708 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a6fd08f780 msgr2=0x55a6fd0a02a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:31.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.968+0000 7f8004029740 1 --2- 172.21.15.83:0/2668709708 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a6fd08f780 0x55a6fd0a02a0 secure :-1 s=READY pgs=2683 cs=0 l=1 rev1=1 crypto rx=0x7f7fe400b6c0 tx=0x7f7fe40123f0 comp rx=0 tx=0).stop 2024-04-24T09:14:31.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.968+0000 7f8004029740 1 -- 172.21.15.83:0/2668709708 shutdown_connections 2024-04-24T09:14:31.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.968+0000 7f8004029740 1 --2- 172.21.15.83:0/2668709708 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55a6fd0b3580 0x55a6fd0b5970 crc :-1 s=READY pgs=567 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:31.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.968+0000 7f8004029740 1 --2- 172.21.15.83:0/2668709708 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a6fd0a3230 0x55a6fd0a0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:31.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.968+0000 7f8004029740 1 --2- 172.21.15.83:0/2668709708 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a6fd090130 0x55a6fd0a07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:31.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.968+0000 7f8004029740 1 --2- 172.21.15.83:0/2668709708 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a6fd08f780 0x55a6fd0a02a0 unknown :-1 s=CLOSED pgs=2683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:31.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.968+0000 7f8004029740 1 -- 172.21.15.83:0/2668709708 >> 172.21.15.83:0/2668709708 conn(0x55a6fd08bd50 msgr2=0x55a6fd091bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:14:31.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.968+0000 7f7fdfe00640 0 client.23415 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:14:31.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.969+0000 7f8004029740 1 -- 172.21.15.83:0/2668709708 shutdown_connections 2024-04-24T09:14:31.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.969+0000 7f8004029740 1 -- 172.21.15.83:0/2668709708 wait complete. 2024-04-24T09:14:31.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.969+0000 7f8004029740 20 client.23415 trim_cache size 0 max 16384 2024-04-24T09:14:31.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.997+0000 7f8004029740 1 Processor -- start 2024-04-24T09:14:32.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.998+0000 7f8004029740 1 -- start start 2024-04-24T09:14:32.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.998+0000 7f8004029740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a6fd08f780 0x55a6fd0aef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:32.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.998+0000 7f8004029740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a6fd090130 0x55a6fd0a0af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:32.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.998+0000 7f8004029740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a6fd0a3230 0x55a6fd09fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:32.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.998+0000 7f8004029740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7ffc1ad340 con 0x55a6fd0a3230 2024-04-24T09:14:32.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.998+0000 7f8004029740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7ffc1aeee0 con 0x55a6fd090130 2024-04-24T09:14:32.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.998+0000 7f8004029740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7ffc1ada10 con 0x55a6fd08f780 2024-04-24T09:14:32.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.998+0000 7f7ff9e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a6fd090130 0x55a6fd0a0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:32.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.998+0000 7f7ff9e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a6fd090130 0x55a6fd0a0af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:35648/0 (socket says 172.21.15.83:35648) 2024-04-24T09:14:32.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.998+0000 7f7ff9e00640 1 -- 172.21.15.83:0/2678217908 learned_addr learned my addr 172.21.15.83:0/2678217908 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:14:32.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.998+0000 7f7ff1400640 1 --2- 172.21.15.83:0/2678217908 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a6fd0a3230 0x55a6fd09fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:32.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.999+0000 7f7ff9e00640 1 -- 172.21.15.83:0/2678217908 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a6fd08f780 msgr2=0x55a6fd0aef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:14:32.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.999+0000 7f7ff9e00640 1 --2- 172.21.15.83:0/2678217908 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a6fd08f780 0x55a6fd0aef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:32.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.999+0000 7f7ff9e00640 1 -- 172.21.15.83:0/2678217908 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a6fd0a3230 msgr2=0x55a6fd09fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:14:32.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.999+0000 7f7ff9e00640 1 --2- 172.21.15.83:0/2678217908 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a6fd0a3230 0x55a6fd09fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:32.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.999+0000 7f7ff9e00640 1 -- 172.21.15.83:0/2678217908 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6fd0a5990 con 0x55a6fd090130 2024-04-24T09:14:32.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.999+0000 7f7ff1400640 1 --2- 172.21.15.83:0/2678217908 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a6fd0a3230 0x55a6fd09fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:14:32.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:31.999+0000 7f7ff9e00640 1 --2- 172.21.15.83:0/2678217908 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a6fd090130 0x55a6fd0a0af0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f7fec000f80 tx=0x7f7fec0316d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1b372d431a57757 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:14:32.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:32.000+0000 7f7feaa00640 1 -- 172.21.15.83:0/2678217908 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fec003b70 con 0x55a6fd090130 2024-04-24T09:14:32.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:32.000+0000 7f7feaa00640 1 -- 172.21.15.83:0/2678217908 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7fec036040 con 0x55a6fd090130 2024-04-24T09:14:32.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:32.000+0000 7f7feaa00640 1 -- 172.21.15.83:0/2678217908 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fec004040 con 0x55a6fd090130 2024-04-24T09:14:32.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:32.001+0000 7f8004029740 1 -- 172.21.15.83:0/2678217908 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a6fd090130 msgr2=0x55a6fd0a0af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:32.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:32.001+0000 7f8004029740 1 --2- 172.21.15.83:0/2678217908 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a6fd090130 0x55a6fd0a0af0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f7fec000f80 tx=0x7f7fec0316d0 comp rx=0 tx=0).stop 2024-04-24T09:14:32.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:32.001+0000 7f8004029740 1 -- 172.21.15.83:0/2678217908 shutdown_connections 2024-04-24T09:14:32.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:32.001+0000 7f8004029740 1 --2- 172.21.15.83:0/2678217908 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a6fd0a3230 0x55a6fd09fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:32.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:32.001+0000 7f8004029740 1 --2- 172.21.15.83:0/2678217908 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a6fd090130 0x55a6fd0a0af0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:32.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:32.001+0000 7f8004029740 1 --2- 172.21.15.83:0/2678217908 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a6fd08f780 0x55a6fd0aef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:32.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:32.001+0000 7f8004029740 1 -- 172.21.15.83:0/2678217908 >> 172.21.15.83:0/2678217908 conn(0x55a6fd08bd50 msgr2=0x55a6fd0a1ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:14:32.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:32.001+0000 7f8004029740 1 -- 172.21.15.83:0/2678217908 shutdown_connections 2024-04-24T09:14:32.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:32.001+0000 7f8004029740 1 -- 172.21.15.83:0/2678217908 wait complete. 2024-04-24T09:14:32.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:32.001+0000 7f8004029740 1 -- start start 2024-04-24T09:14:32.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:32.002+0000 7f8004029740 1 Processor -- start 2024-04-24T09:14:32.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:32.002+0000 7f8004029740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a6fd08f780 0x55a6fd0ab450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:32.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:32.002+0000 7f8004029740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a6fd090130 0x55a6fd0ab990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:32.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:32.002+0000 7f8004029740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a6fd0a3230 0x55a6fd09cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:32.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:32.002+0000 7f8004029740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7ffc1ae7e0 con 0x55a6fd08f780 2024-04-24T09:14:32.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:32.002+0000 7f8004029740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7ffc1ae110 con 0x55a6fd090130 2024-04-24T09:14:32.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:32.002+0000 7f8004029740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7ffc1ab340 con 0x55a6fd0a3230 2024-04-24T09:14:32.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:32.002+0000 7f8004029740 10 client.0 _ll_register_callbacks cb 0x7f7fe400bb60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:14:32.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:32.003+0000 7f7ff1400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a6fd08f780 0x55a6fd0ab450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:32.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:32.003+0000 7f7ff1400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a6fd08f780 0x55a6fd0ab450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37338/0 (socket says 172.21.15.83:37338) 2024-04-24T09:14:32.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:32.003+0000 7f7ff1400640 1 -- 172.21.15.83:0/1657078300 learned_addr learned my addr 172.21.15.83:0/1657078300 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:14:32.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:32.003+0000 7f7ff9e00640 1 --2- 172.21.15.83:0/1657078300 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a6fd090130 0x55a6fd0ab990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:32.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:32.003+0000 7f7ff1400640 1 -- 172.21.15.83:0/1657078300 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a6fd0a3230 msgr2=0x55a6fd09cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:14:32.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:32.003+0000 7f7ff1400640 1 --2- 172.21.15.83:0/1657078300 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a6fd0a3230 0x55a6fd09cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:32.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:32.003+0000 7f7ff1400640 1 -- 172.21.15.83:0/1657078300 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a6fd090130 msgr2=0x55a6fd0ab990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:14:32.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:32.003+0000 7f7ff1400640 1 --2- 172.21.15.83:0/1657078300 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a6fd090130 0x55a6fd0ab990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:32.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:32.003+0000 7f7ff1400640 1 -- 172.21.15.83:0/1657078300 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6fcd073f0 con 0x55a6fd08f780 2024-04-24T09:14:32.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:32.003+0000 7f7ff1400640 1 --2- 172.21.15.83:0/1657078300 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a6fd08f780 0x55a6fd0ab450 secure :-1 s=READY pgs=2684 cs=0 l=1 rev1=1 crypto rx=0x7f7fe40057c0 tx=0x7f7fe400b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4090f4d4d730cbc0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:14:32.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:32.004+0000 7f7fdfe00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:14:32.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:32.004+0000 7f7fdfe00640 1 -- 172.21.15.83:0/1657078300 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fe403a060 con 0x55a6fd08f780 2024-04-24T09:14:32.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:32.004+0000 7f7fdfe00640 1 -- 172.21.15.83:0/1657078300 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(128..128 src has 1..128) ==== 9583+0+0 (secure 0 0 0) 0x7f7fe400bc10 con 0x55a6fd08f780 2024-04-24T09:14:32.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:32.004+0000 7f8004029740 1 -- 172.21.15.83:0/1657078300 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7fec036040 con 0x55a6fd08f780 2024-04-24T09:14:32.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:32.004+0000 7f7fdfe00640 1 -- 172.21.15.83:0/1657078300 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f7fe4041050 con 0x55a6fd08f780 2024-04-24T09:14:32.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:32.004+0000 7f8004029740 10 client.23421 fetch_fsmap learned FSMap version 65 2024-04-24T09:14:32.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:32.004+0000 7f8004029740 1 -- 172.21.15.83:0/1657078300 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x55a6fccc70f0 con 0x55a6fd08f780 2024-04-24T09:14:32.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:32.005+0000 7f7fdfe00640 1 -- 172.21.15.83:0/1657078300 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7fec036040 con 0x55a6fd08f780 2024-04-24T09:14:32.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:32.005+0000 7f7fdfe00640 1 -- 172.21.15.83:0/1657078300 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 65) ==== 8370+0+0 (secure 0 0 0) 0x7f7fe4009b30 con 0x55a6fd08f780 2024-04-24T09:14:32.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:32.005+0000 7f8004029740 10 client.23421 fetch_fsmap finished waiting for FSMap version 65 2024-04-24T09:14:32.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:32.005+0000 7f8004029740 10 client.23421 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:14:32.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:32.005+0000 7f8004029740 20 client.23421 populate_metadata read hostname 'smithi083' 2024-04-24T09:14:32.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:32.005+0000 7f8004029740 1 --2- 172.21.15.83:0/1657078300 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55a6fd119850 0x55a6fd11bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:32.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:32.005+0000 7f8004029740 4 client.23421 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T09:14:32.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:32.005+0000 7f8004029740 1 -- 172.21.15.83:0/1657078300 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a6fcffca80 con 0x55a6fd119850 2024-04-24T09:14:32.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:32.006+0000 7f7ff9e00640 1 --2- 172.21.15.83:0/1657078300 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55a6fd119850 0x55a6fd11bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:32.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:32.007+0000 7f7ff9e00640 1 --2- 172.21.15.83:0/1657078300 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55a6fd119850 0x55a6fd11bc70 crc :-1 s=READY pgs=568 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d262676cab61d327 server_cookie=d854549f2fb182ff in_seq=0 out_seq=0 2024-04-24T09:14:32.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:32.007+0000 7f7fdfe00640 10 client.23421 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:14:32.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:32.007+0000 7f7fdfe00640 1 -- 172.21.15.83:0/1657078300 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55a6fcffca80 con 0x55a6fd119850 2024-04-24T09:14:32.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:32.007+0000 7f7fdfe00640 10 client.23421 handle_command_reply: tid=0 2024-04-24T09:14:32.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:32.007+0000 7f8004029740 1 client.23421 shutdown 2024-04-24T09:14:32.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:32.007+0000 7f8004029740 1 -- 172.21.15.83:0/1657078300 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a6fd08f780 msgr2=0x55a6fd0ab450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:32.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:32.007+0000 7f8004029740 1 --2- 172.21.15.83:0/1657078300 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a6fd08f780 0x55a6fd0ab450 secure :-1 s=READY pgs=2684 cs=0 l=1 rev1=1 crypto rx=0x7f7fe40057c0 tx=0x7f7fe400b7c0 comp rx=0 tx=0).stop 2024-04-24T09:14:32.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:32.007+0000 7f8004029740 1 -- 172.21.15.83:0/1657078300 shutdown_connections 2024-04-24T09:14:32.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:32.007+0000 7f8004029740 1 --2- 172.21.15.83:0/1657078300 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55a6fd119850 0x55a6fd11bc70 crc :-1 s=READY pgs=568 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:32.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:32.007+0000 7f8004029740 1 --2- 172.21.15.83:0/1657078300 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a6fd0a3230 0x55a6fd09cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:32.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:32.007+0000 7f8004029740 1 --2- 172.21.15.83:0/1657078300 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a6fd090130 0x55a6fd0ab990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:32.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:32.007+0000 7f8004029740 1 --2- 172.21.15.83:0/1657078300 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a6fd08f780 0x55a6fd0ab450 unknown :-1 s=CLOSED pgs=2684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:32.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:32.007+0000 7f8004029740 1 -- 172.21.15.83:0/1657078300 >> 172.21.15.83:0/1657078300 conn(0x55a6fd08bd50 msgr2=0x55a6fd0a08c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:14:32.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:32.007+0000 7f7fdfe00640 0 client.23421 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:14:32.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:32.008+0000 7f8004029740 1 -- 172.21.15.83:0/1657078300 shutdown_connections 2024-04-24T09:14:32.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:32.008+0000 7f8004029740 1 -- 172.21.15.83:0/1657078300 wait complete. 2024-04-24T09:14:32.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:32.008+0000 7f8004029740 20 client.23421 trim_cache size 0 max 16384 2024-04-24T09:14:32.113 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T09:14:32.113 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T09:14:32.113 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-04-24T09:14:32.113 INFO:teuthology.orchestra.run.smithi083.stdout: "eccb2691-2d88-47d9-a790-071cb67c2725": { 2024-04-24T09:14:32.113 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-04-24T09:14:32.113 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "eccb2691-2d88-47d9-a790-071cb67c2725", 2024-04-24T09:14:32.113 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-04-24T09:14:32.113 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:14:32.113 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:14:32.113 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T09:14:32.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:32.008+0000 7f7ffb200640 1 -- 172.21.15.83:0/2940332070 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f7fe00a3e60 msgr2=0x7f7fe00a6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:32.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:32.008+0000 7f7ffb200640 1 --2- 172.21.15.83:0/2940332070 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f7fe00a3e60 0x7f7fe00a6320 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f7ff40cd710 tx=0x7f7ff40046e0 comp rx=0 tx=0).stop 2024-04-24T09:14:32.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:32.009+0000 7f7ffb200640 1 -- 172.21.15.83:0/2940332070 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ffc10e820 msgr2=0x7f7ffc19a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:32.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:32.009+0000 7f7ffb200640 1 --2- 172.21.15.83:0/2940332070 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ffc10e820 0x7f7ffc19a8b0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc06b6b0 tx=0x7f7ff400b870 comp rx=0 tx=0).stop 2024-04-24T09:14:32.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:32.009+0000 7f7ffb200640 1 -- 172.21.15.83:0/2940332070 shutdown_connections 2024-04-24T09:14:32.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:32.009+0000 7f7ffb200640 1 --2- 172.21.15.83:0/2940332070 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f7fe00a3e60 0x7f7fe00a6320 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:32.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:32.009+0000 7f7ffb200640 1 --2- 172.21.15.83:0/2940332070 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ffc10e820 0x7f7ffc19a8b0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:32.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:32.009+0000 7f7ffb200640 1 --2- 172.21.15.83:0/2940332070 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7ffc101620 0x7f7ffc1a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:32.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:32.009+0000 7f7ffb200640 1 --2- 172.21.15.83:0/2940332070 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ffc100c70 0x7f7ffc1a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:32.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:32.009+0000 7f7ffb200640 1 -- 172.21.15.83:0/2940332070 >> 172.21.15.83:0/2940332070 conn(0x7f7ffc0fc880 msgr2=0x7f7ffc0fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:14:32.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:32.009+0000 7f7ffb200640 1 -- 172.21.15.83:0/2940332070 shutdown_connections 2024-04-24T09:14:32.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:14:32.009+0000 7f7ffb200640 1 -- 172.21.15.83:0/2940332070 wait complete. 2024-04-24T09:14:32.116 INFO:tasks.cephfs.filesystem:scrub status for tag:eccb2691-2d88-47d9-a790-071cb67c2725 - {'path': '/', 'tag': 'eccb2691-2d88-47d9-a790-071cb67c2725', 'options': 'recursive,force'} 2024-04-24T09:14:32.207 INFO:tasks.workunit.client.0.smithi083.stdout: 10 67085 6.56 MB/sec execute 423 sec latency 123.392 ms 2024-04-24T09:14:32.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:14:32 smithi138 ceph-mon[31441]: pgmap v724: 129 pgs: 129 active+clean; 1010 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.9 MiB/s wr, 212 op/s 2024-04-24T09:14:32.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:14:32 smithi138 ceph-mon[31441]: scrub queued for path: / 2024-04-24T09:14:32.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:14:32 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:14:32.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:14:32 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:14:32.706 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:14:32 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:14:32.706 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:14:32 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:14:32.706 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:14:32 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:14:32.706 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:14:32 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:14:32.706 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:14:32 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:14:32.706 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:14:32 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:14:32.706 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:14:32 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:14:32.706 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:14:32 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:14:32.706 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:14:32 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:14:32.706 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:14:32 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:14:32.706 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:14:32 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:14:32.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:14:32 smithi189 ceph-mon[31791]: pgmap v724: 129 pgs: 129 active+clean; 1010 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.9 MiB/s wr, 212 op/s 2024-04-24T09:14:32.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:14:32 smithi189 ceph-mon[31791]: scrub queued for path: / 2024-04-24T09:14:32.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:14:32 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:14:32.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:14:32 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:14:32.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:14:32 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:14:32.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:14:32 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:14:32.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:14:32 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:14:32.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:14:32 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:14:32.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:14:32 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:14:32.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:14:32 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:14:32.709 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:14:32 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:14:32.709 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:14:32 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:14:32.709 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:14:32 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:14:32.709 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:14:32 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:14:32.709 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:14:32 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:14:32.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:14:32 smithi083 ceph-mon[27907]: pgmap v724: 129 pgs: 129 active+clean; 1010 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.9 MiB/s wr, 212 op/s 2024-04-24T09:14:32.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:14:32 smithi083 ceph-mon[27907]: scrub queued for path: / 2024-04-24T09:14:32.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:14:32 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:14:32.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:14:32 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:14:32.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:14:32 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:14:32.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:14:32 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:14:32.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:14:32 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:14:32.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:14:32 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:14:32.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:14:32 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:14:32.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:14:32 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:14:32.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:14:32 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:14:32.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:14:32 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:14:32.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:14:32 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:14:32.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:14:32 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:14:32.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:14:32 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:14:33.207 INFO:tasks.workunit.client.0.smithi083.stdout: 10 67235 6.56 MB/sec execute 424 sec latency 91.714 ms 2024-04-24T09:14:33.704 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:14:33 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:14:33.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:14:33 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:14:33.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:14:33 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:14:33.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:14:33 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:14:33.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:14:33 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:14:33.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:14:33 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:14:33.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:14:33 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:14:33.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:14:33 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:14:33.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:14:33 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:14:33.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:14:33 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:14:33.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:14:33 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:14:33.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:14:33 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:14:33.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:14:33 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:14:33.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:14:33 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:14:33.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:14:33 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:14:33.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:14:33 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:14:33.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:14:33 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:14:33.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:14:33 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:14:34.207 INFO:tasks.workunit.client.0.smithi083.stdout: 10 67361 6.56 MB/sec execute 425 sec latency 371.245 ms 2024-04-24T09:14:34.704 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:14:34 smithi138 ceph-mon[31441]: pgmap v725: 129 pgs: 129 active+clean; 1018 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.5 MiB/s wr, 370 op/s 2024-04-24T09:14:34.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:14:34 smithi138 ceph-mon[31441]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:14:34.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:14:34 smithi189 ceph-mon[31791]: pgmap v725: 129 pgs: 129 active+clean; 1018 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.5 MiB/s wr, 370 op/s 2024-04-24T09:14:34.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:14:34 smithi189 ceph-mon[31791]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:14:34.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:14:34 smithi083 ceph-mon[27907]: pgmap v725: 129 pgs: 129 active+clean; 1018 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.5 MiB/s wr, 370 op/s 2024-04-24T09:14:34.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:14:34 smithi083 ceph-mon[27907]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:14:35.207 INFO:tasks.workunit.client.0.smithi083.stdout: 10 67471 6.56 MB/sec execute 426 sec latency 74.845 ms 2024-04-24T09:14:36.207 INFO:tasks.workunit.client.0.smithi083.stdout: 10 67581 6.56 MB/sec execute 427 sec latency 70.288 ms 2024-04-24T09:14:36.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:14:36 smithi138 ceph-mon[31441]: pgmap v726: 129 pgs: 129 active+clean; 1018 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.9 MiB/s wr, 265 op/s 2024-04-24T09:14:36.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:14:36 smithi189 ceph-mon[31791]: pgmap v726: 129 pgs: 129 active+clean; 1018 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.9 MiB/s wr, 265 op/s 2024-04-24T09:14:36.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:14:36 smithi083 ceph-mon[27907]: pgmap v726: 129 pgs: 129 active+clean; 1018 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.9 MiB/s wr, 265 op/s 2024-04-24T09:14:37.207 INFO:tasks.workunit.client.0.smithi083.stdout: 10 67742 6.57 MB/sec execute 428 sec latency 75.473 ms 2024-04-24T09:14:38.207 INFO:tasks.workunit.client.0.smithi083.stdout: 10 67897 6.58 MB/sec execute 429 sec latency 76.956 ms 2024-04-24T09:14:38.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:14:38 smithi138 ceph-mon[31441]: pgmap v727: 129 pgs: 129 active+clean; 1018 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.9 MiB/s wr, 265 op/s 2024-04-24T09:14:38.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:14:38 smithi189 ceph-mon[31791]: pgmap v727: 129 pgs: 129 active+clean; 1018 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.9 MiB/s wr, 265 op/s 2024-04-24T09:14:38.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:14:38 smithi083 ceph-mon[27907]: pgmap v727: 129 pgs: 129 active+clean; 1018 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.9 MiB/s wr, 265 op/s 2024-04-24T09:14:39.207 INFO:tasks.workunit.client.0.smithi083.stdout: 10 68050 6.59 MB/sec execute 430 sec latency 106.963 ms 2024-04-24T09:14:40.207 INFO:tasks.workunit.client.0.smithi083.stdout: 10 68184 6.60 MB/sec execute 431 sec latency 61.068 ms 2024-04-24T09:14:40.704 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:14:40 smithi138 ceph-mon[31441]: pgmap v728: 129 pgs: 129 active+clean; 1.0 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.8 MiB/s wr, 348 op/s 2024-04-24T09:14:40.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:14:40 smithi189 ceph-mon[31791]: pgmap v728: 129 pgs: 129 active+clean; 1.0 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.8 MiB/s wr, 348 op/s 2024-04-24T09:14:40.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:14:40 smithi083 ceph-mon[27907]: pgmap v728: 129 pgs: 129 active+clean; 1.0 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.8 MiB/s wr, 348 op/s 2024-04-24T09:14:41.208 INFO:tasks.workunit.client.0.smithi083.stdout: 10 68342 6.62 MB/sec execute 432 sec latency 70.647 ms 2024-04-24T09:14:42.208 INFO:tasks.workunit.client.0.smithi083.stdout: 10 68444 6.61 MB/sec execute 433 sec latency 296.177 ms 2024-04-24T09:14:42.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:14:42 smithi138 ceph-mon[31441]: pgmap v729: 129 pgs: 129 active+clean; 1.0 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.6 MiB/s wr, 240 op/s 2024-04-24T09:14:42.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:14:42 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:14:42.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:14:42 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:14:42.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:14:42 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:14:42.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:14:42 smithi189 ceph-mon[31791]: pgmap v729: 129 pgs: 129 active+clean; 1.0 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.6 MiB/s wr, 240 op/s 2024-04-24T09:14:42.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:14:42 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:14:42.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:14:42 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:14:42.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:14:42 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:14:42.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:14:42 smithi083 ceph-mon[27907]: pgmap v729: 129 pgs: 129 active+clean; 1.0 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.6 MiB/s wr, 240 op/s 2024-04-24T09:14:42.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:14:42 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:14:42.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:14:42 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:14:42.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:14:42 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:14:43.209 INFO:tasks.workunit.client.0.smithi083.stdout: 10 68535 6.61 MB/sec execute 434 sec latency 94.708 ms 2024-04-24T09:14:43.704 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:14:43 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:14:43.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:14:43 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:14:43.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:14:43 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:14:43.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:14:43 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:14:43.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:14:43 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:14:43.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:14:43 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:14:43.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:14:43 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:14:43.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:14:43 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:14:43.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:14:43 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:14:43.982 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:43.981+0000 7fad11e00640 1 -- 172.21.15.83:0/2749984165 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad0c100a90 msgr2=0x7fad0c10ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:43.982 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:43.981+0000 7fad11e00640 1 --2- 172.21.15.83:0/2749984165 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad0c100a90 0x7fad0c10ed90 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fad04009be0 tx=0x7fad0402a6c0 comp rx=0 tx=0).stop 2024-04-24T09:14:43.982 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:43.981+0000 7fad11e00640 1 -- 172.21.15.83:0/2749984165 shutdown_connections 2024-04-24T09:14:43.982 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:43.981+0000 7fad11e00640 1 --2- 172.21.15.83:0/2749984165 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad0c100a90 0x7fad0c10ed90 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:43.983 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:43.981+0000 7fad11e00640 1 --2- 172.21.15.83:0/2749984165 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad0c1000d0 0x7fad0c100550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:43.983 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:43.981+0000 7fad11e00640 1 --2- 172.21.15.83:0/2749984165 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fad0c107120 0x7fad0c107500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:43.983 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:43.981+0000 7fad11e00640 1 -- 172.21.15.83:0/2749984165 >> 172.21.15.83:0/2749984165 conn(0x7fad0c077000 msgr2=0x7fad0c0fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:14:43.983 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:43.981+0000 7fad11e00640 1 -- 172.21.15.83:0/2749984165 shutdown_connections 2024-04-24T09:14:43.983 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:43.981+0000 7fad11e00640 1 -- 172.21.15.83:0/2749984165 wait complete. 2024-04-24T09:14:43.984 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:43.982+0000 7fad11e00640 1 Processor -- start 2024-04-24T09:14:43.984 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:43.983+0000 7fad11e00640 1 -- start start 2024-04-24T09:14:43.984 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:43.983+0000 7fad11e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fad0c1000d0 0x7fad0c1a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:43.984 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:43.983+0000 7fad11e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad0c100a90 0x7fad0c1a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:43.984 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:43.983+0000 7fad11e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad0c107120 0x7fad0c19a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:43.984 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:43.983+0000 7fad11e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fad0c112240 con 0x7fad0c107120 2024-04-24T09:14:43.984 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:43.983+0000 7fad11e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fad0c1120c0 con 0x7fad0c100a90 2024-04-24T09:14:43.984 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:43.983+0000 7fad11e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fad0c1123c0 con 0x7fad0c1000d0 2024-04-24T09:14:43.984 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:43.984+0000 7fad0aa00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fad0c1000d0 0x7fad0c1a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:43.985 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:43.984+0000 7fad0aa00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fad0c1000d0 0x7fad0c1a0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:60368/0 (socket says 172.21.15.83:60368) 2024-04-24T09:14:43.985 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:43.984+0000 7fad0aa00640 1 -- 172.21.15.83:0/2010859161 learned_addr learned my addr 172.21.15.83:0/2010859161 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:14:43.985 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:43.984+0000 7fad02000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad0c100a90 0x7fad0c1a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:43.985 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:43.984+0000 7fad0b400640 1 --2- 172.21.15.83:0/2010859161 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad0c107120 0x7fad0c19a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:43.985 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:43.985+0000 7fad02000640 1 -- 172.21.15.83:0/2010859161 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fad0c1000d0 msgr2=0x7fad0c1a0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:43.985 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:43.985+0000 7fad02000640 1 --2- 172.21.15.83:0/2010859161 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fad0c1000d0 0x7fad0c1a0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:43.985 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:43.985+0000 7fad02000640 1 -- 172.21.15.83:0/2010859161 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad0c107120 msgr2=0x7fad0c19a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:43.985 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:43.985+0000 7fad02000640 1 --2- 172.21.15.83:0/2010859161 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad0c107120 0x7fad0c19a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:43.986 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:43.985+0000 7fad02000640 1 -- 172.21.15.83:0/2010859161 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad0c19b090 con 0x7fad0c100a90 2024-04-24T09:14:43.986 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:43.985+0000 7fad0aa00640 1 --2- 172.21.15.83:0/2010859161 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fad0c1000d0 0x7fad0c1a0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:14:43.986 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:43.985+0000 7fad0b400640 1 --2- 172.21.15.83:0/2010859161 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad0c107120 0x7fad0c19a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:14:43.986 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:43.985+0000 7fad02000640 1 --2- 172.21.15.83:0/2010859161 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad0c100a90 0x7fad0c1a0bc0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7facf4005fd0 tx=0x7facf400dde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:14:43.986 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:43.986+0000 7fad10a00640 1 -- 172.21.15.83:0/2010859161 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf4009c30 con 0x7fad0c100a90 2024-04-24T09:14:43.986 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:43.986+0000 7fad10a00640 1 -- 172.21.15.83:0/2010859161 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7facf400be30 con 0x7fad0c100a90 2024-04-24T09:14:43.986 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:43.986+0000 7fad10a00640 1 -- 172.21.15.83:0/2010859161 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf4004a00 con 0x7fad0c100a90 2024-04-24T09:14:43.987 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:43.986+0000 7fad11e00640 1 -- 172.21.15.83:0/2010859161 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad0c19b320 con 0x7fad0c100a90 2024-04-24T09:14:43.987 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:43.986+0000 7fad11e00640 1 -- 172.21.15.83:0/2010859161 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad0c107e50 con 0x7fad0c100a90 2024-04-24T09:14:43.991 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:43.991+0000 7fad14dc0740 1 Processor -- start 2024-04-24T09:14:43.991 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:43.991+0000 7fad14dc0740 1 -- start start 2024-04-24T09:14:43.992 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:43.991+0000 7fad14dc0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ea584461b0 0x55ea58446590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:43.992 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:43.991+0000 7fad14dc0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ea58446ad0 0x55ea58450e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:43.992 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:43.991+0000 7fad14dc0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ea58451390 0x55ea58453780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:43.992 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:43.991+0000 7fad14dc0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ea583ac900 con 0x55ea58446ad0 2024-04-24T09:14:43.992 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:43.991+0000 7fad14dc0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ea58352f20 con 0x55ea58451390 2024-04-24T09:14:43.992 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:43.991+0000 7fad14dc0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55ea583a88c0 con 0x55ea584461b0 2024-04-24T09:14:43.993 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:43.992+0000 7fad10a00640 1 -- 172.21.15.83:0/2010859161 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7facf4004ba0 con 0x7fad0c100a90 2024-04-24T09:14:43.993 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:43.992+0000 7fad0b400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ea584461b0 0x55ea58446590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:43.993 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:43.992+0000 7fad02000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ea58451390 0x55ea58453780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:43.993 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:43.992+0000 7fad0aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ea58446ad0 0x55ea58450e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:43.993 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:43.992+0000 7fad0aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ea58446ad0 0x55ea58450e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52468/0 (socket says 172.21.15.83:52468) 2024-04-24T09:14:43.993 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:43.992+0000 7fad0aa00640 1 -- 172.21.15.83:0/3136216070 learned_addr learned my addr 172.21.15.83:0/3136216070 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:14:43.994 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:43.993+0000 7fad02000640 1 -- 172.21.15.83:0/3136216070 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ea584461b0 msgr2=0x55ea58446590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:14:43.994 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:43.993+0000 7fad02000640 1 --2- 172.21.15.83:0/3136216070 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ea584461b0 0x55ea58446590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:43.994 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:43.993+0000 7fad02000640 1 -- 172.21.15.83:0/3136216070 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ea58446ad0 msgr2=0x55ea58450e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:14:43.994 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:43.993+0000 7fad02000640 1 --2- 172.21.15.83:0/3136216070 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ea58446ad0 0x55ea58450e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:43.994 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:43.993+0000 7fad02000640 1 -- 172.21.15.83:0/3136216070 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea58377e10 con 0x55ea58451390 2024-04-24T09:14:43.995 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:43.993+0000 7fad0aa00640 1 --2- 172.21.15.83:0/3136216070 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ea58446ad0 0x55ea58450e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:14:43.995 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:43.993+0000 7fad02000640 1 --2- 172.21.15.83:0/3136216070 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ea58451390 0x55ea58453780 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7facf40cdbb0 tx=0x7facf40d6cb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=452122da706b302b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:14:43.995 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:43.994+0000 7fad10a00640 1 --2- 172.21.15.83:0/2010859161 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7facf00a3f70 0x7facf00a6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:43.995 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:43.994+0000 7facfbe00640 1 -- 172.21.15.83:0/3136216070 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf40df070 con 0x55ea58451390 2024-04-24T09:14:43.995 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:43.994+0000 7facfbe00640 1 -- 172.21.15.83:0/3136216070 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7facf40d8c60 con 0x55ea58451390 2024-04-24T09:14:43.995 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:43.994+0000 7facfbe00640 1 -- 172.21.15.83:0/3136216070 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf40d2330 con 0x55ea58451390 2024-04-24T09:14:43.995 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:43.994+0000 7fad10a00640 1 -- 172.21.15.83:0/2010859161 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9583+0+0 (secure 0 0 0) 0x7facf40d08e0 con 0x7fad0c100a90 2024-04-24T09:14:43.995 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:43.995+0000 7fad0b400640 1 --2- 172.21.15.83:0/2010859161 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7facf00a3f70 0x7facf00a6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:43.996 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:43.995+0000 7fad14dc0740 1 -- 172.21.15.83:0/3136216070 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ea58451390 msgr2=0x55ea58453780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:43.996 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:43.995+0000 7fad14dc0740 1 --2- 172.21.15.83:0/3136216070 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ea58451390 0x55ea58453780 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7facf40cdbb0 tx=0x7facf40d6cb0 comp rx=0 tx=0).stop 2024-04-24T09:14:43.996 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:43.996+0000 7fad0b400640 1 --2- 172.21.15.83:0/2010859161 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7facf00a3f70 0x7facf00a6430 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fad04004af0 tx=0x7fad04040040 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:14:43.997 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:43.996+0000 7fad14dc0740 1 -- 172.21.15.83:0/3136216070 shutdown_connections 2024-04-24T09:14:43.997 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:43.996+0000 7fad14dc0740 1 --2- 172.21.15.83:0/3136216070 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ea58451390 0x55ea58453780 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:43.997 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:43.996+0000 7fad14dc0740 1 --2- 172.21.15.83:0/3136216070 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ea58446ad0 0x55ea58450e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:43.997 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:43.996+0000 7fad14dc0740 1 --2- 172.21.15.83:0/3136216070 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ea584461b0 0x55ea58446590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:43.997 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:43.996+0000 7fad14dc0740 1 -- 172.21.15.83:0/3136216070 >> 172.21.15.83:0/3136216070 conn(0x55ea584426c0 msgr2=0x55ea58442aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:14:43.997 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:43.996+0000 7fad14dc0740 1 -- 172.21.15.83:0/3136216070 shutdown_connections 2024-04-24T09:14:43.997 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:43.996+0000 7fad14dc0740 1 -- 172.21.15.83:0/3136216070 wait complete. 2024-04-24T09:14:43.997 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:43.997+0000 7fad14dc0740 1 -- start start 2024-04-24T09:14:43.998 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:43.997+0000 7fad14dc0740 1 Processor -- start 2024-04-24T09:14:43.998 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:43.998+0000 7fad14dc0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ea584461b0 0x55ea5844e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:43.998 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:43.998+0000 7fad14dc0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ea58446ad0 0x55ea5844e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:43.998 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:43.998+0000 7fad14dc0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ea58451390 0x55ea5844edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:43.998 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:43.998+0000 7fad14dc0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ea58453fc0 con 0x55ea584461b0 2024-04-24T09:14:43.999 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:43.998+0000 7fad14dc0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ea583662c0 con 0x55ea58446ad0 2024-04-24T09:14:43.999 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:43.998+0000 7fad14dc0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55ea58454140 con 0x55ea58451390 2024-04-24T09:14:43.999 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:43.998+0000 7fad14dc0740 10 client.0 _ll_register_callbacks cb 0x55ea580bea00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:14:44.000 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:43.998+0000 7fad0b400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ea58451390 0x55ea5844edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:44.000 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:43.998+0000 7fad0b400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ea58451390 0x55ea5844edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:60394/0 (socket says 172.21.15.83:60394) 2024-04-24T09:14:44.000 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:43.998+0000 7fad0b400640 1 -- 172.21.15.83:0/1051274131 learned_addr learned my addr 172.21.15.83:0/1051274131 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:14:44.000 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:43.998+0000 7fad02000640 1 --2- 172.21.15.83:0/1051274131 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ea58446ad0 0x55ea5844e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:44.000 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:43.998+0000 7fad0aa00640 1 --2- 172.21.15.83:0/1051274131 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ea584461b0 0x55ea5844e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:44.000 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:43.999+0000 7fad0b400640 1 -- 172.21.15.83:0/1051274131 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ea58446ad0 msgr2=0x55ea5844e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:14:44.000 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:43.999+0000 7fad0b400640 1 --2- 172.21.15.83:0/1051274131 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ea58446ad0 0x55ea5844e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:44.000 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:43.999+0000 7fad0b400640 1 -- 172.21.15.83:0/1051274131 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ea584461b0 msgr2=0x55ea5844e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:14:44.000 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:43.999+0000 7fad0b400640 1 --2- 172.21.15.83:0/1051274131 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ea584461b0 0x55ea5844e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:44.000 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:43.999+0000 7fad0b400640 1 -- 172.21.15.83:0/1051274131 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea58376fd0 con 0x55ea58451390 2024-04-24T09:14:44.000 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:43.999+0000 7fad0b400640 1 --2- 172.21.15.83:0/1051274131 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ea58451390 0x55ea5844edf0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fad0403ef10 tx=0x7fad0402a6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75f39ed242b36195 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:14:44.000 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:43.999+0000 7facf8c00640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-04-24T09:14:44.001 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.000+0000 7facf8c00640 1 -- 172.21.15.83:0/1051274131 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad04030d20 con 0x55ea58451390 2024-04-24T09:14:44.001 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.000+0000 7fad14dc0740 1 -- 172.21.15.83:0/1051274131 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7facf40d8c60 con 0x55ea58451390 2024-04-24T09:14:44.001 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.000+0000 7facf8c00640 1 -- 172.21.15.83:0/1051274131 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(128..128 src has 1..128) ==== 9583+0+0 (secure 0 0 0) 0x7fad04031440 con 0x55ea58451390 2024-04-24T09:14:44.002 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.001+0000 7facf8c00640 1 -- 172.21.15.83:0/1051274131 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7fad04063050 con 0x55ea58451390 2024-04-24T09:14:44.002 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.001+0000 7fad14dc0740 10 client.30238 fetch_fsmap learned FSMap version 66 2024-04-24T09:14:44.002 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.001+0000 7fad14dc0740 1 -- 172.21.15.83:0/1051274131 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x55ea580bf210 con 0x55ea58451390 2024-04-24T09:14:44.002 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.001+0000 7facf8c00640 1 -- 172.21.15.83:0/1051274131 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7facf40d8c60 con 0x55ea58451390 2024-04-24T09:14:44.003 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.002+0000 7facf8c00640 1 -- 172.21.15.83:0/1051274131 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 66) ==== 8366+0+0 (secure 0 0 0) 0x7fad040043b0 con 0x55ea58451390 2024-04-24T09:14:44.003 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.002+0000 7fad14dc0740 10 client.30238 fetch_fsmap finished waiting for FSMap version 66 2024-04-24T09:14:44.003 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.002+0000 7fad14dc0740 10 client.30238 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:14:44.003 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.002+0000 7fad14dc0740 20 client.30238 populate_metadata read hostname 'smithi083' 2024-04-24T09:14:44.003 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.002+0000 7fad14dc0740 1 --2- 172.21.15.83:0/1051274131 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ea58461570 0x55ea58463960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:44.003 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.002+0000 7fad14dc0740 4 client.30238 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:14:44.003 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.002+0000 7fad14dc0740 1 -- 172.21.15.83:0/1051274131 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ea583ba320 con 0x55ea58461570 2024-04-24T09:14:44.003 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.002+0000 7fad0aa00640 1 --2- 172.21.15.83:0/1051274131 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ea58461570 0x55ea58463960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:44.004 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.003+0000 7fad0aa00640 1 --2- 172.21.15.83:0/1051274131 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ea58461570 0x55ea58463960 crc :-1 s=READY pgs=569 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=906c67acb8659d0c server_cookie=dd6771976f0acfe3 in_seq=0 out_seq=0 2024-04-24T09:14:44.004 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.003+0000 7facf8c00640 10 client.30238 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:14:44.005 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.004+0000 7facf8c00640 1 -- 172.21.15.83:0/1051274131 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ea583ba320 con 0x55ea58461570 2024-04-24T09:14:44.006 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.004+0000 7facf8c00640 10 client.30238 handle_command_reply: tid=0 2024-04-24T09:14:44.006 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.004+0000 7fad14dc0740 1 client.30238 shutdown 2024-04-24T09:14:44.006 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.005+0000 7fad14dc0740 1 -- 172.21.15.83:0/1051274131 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ea58451390 msgr2=0x55ea5844edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:44.006 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.005+0000 7fad14dc0740 1 --2- 172.21.15.83:0/1051274131 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ea58451390 0x55ea5844edf0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fad0403ef10 tx=0x7fad0402a6c0 comp rx=0 tx=0).stop 2024-04-24T09:14:44.006 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.005+0000 7fad14dc0740 1 -- 172.21.15.83:0/1051274131 shutdown_connections 2024-04-24T09:14:44.006 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.005+0000 7fad14dc0740 1 --2- 172.21.15.83:0/1051274131 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ea58461570 0x55ea58463960 crc :-1 s=READY pgs=569 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:44.006 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.005+0000 7fad14dc0740 1 --2- 172.21.15.83:0/1051274131 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ea58451390 0x55ea5844edf0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:44.006 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.005+0000 7fad14dc0740 1 --2- 172.21.15.83:0/1051274131 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ea58446ad0 0x55ea5844e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:44.006 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.005+0000 7fad14dc0740 1 --2- 172.21.15.83:0/1051274131 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ea584461b0 0x55ea5844e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:44.006 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.005+0000 7fad14dc0740 1 -- 172.21.15.83:0/1051274131 >> 172.21.15.83:0/1051274131 conn(0x55ea584426c0 msgr2=0x55ea58448440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:14:44.006 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.005+0000 7facf8c00640 0 client.30238 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:14:44.006 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.005+0000 7fad14dc0740 1 -- 172.21.15.83:0/1051274131 shutdown_connections 2024-04-24T09:14:44.007 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.005+0000 7fad14dc0740 1 -- 172.21.15.83:0/1051274131 wait complete. 2024-04-24T09:14:44.007 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.006+0000 7fad14dc0740 20 client.30238 trim_cache size 0 max 16384 2024-04-24T09:14:44.039 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.038+0000 7fad14dc0740 1 Processor -- start 2024-04-24T09:14:44.040 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.038+0000 7fad14dc0740 1 -- start start 2024-04-24T09:14:44.040 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.038+0000 7fad14dc0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ea584461b0 0x55ea5845ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:44.040 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.038+0000 7fad14dc0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ea58446ad0 0x55ea58448b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:44.040 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.038+0000 7fad14dc0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ea58451390 0x55ea58469270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:44.040 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.038+0000 7fad14dc0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fad0c1ad3d0 con 0x55ea58446ad0 2024-04-24T09:14:44.040 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.038+0000 7fad14dc0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fad0c1aef40 con 0x55ea584461b0 2024-04-24T09:14:44.040 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.038+0000 7fad14dc0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fad0c1adaa0 con 0x55ea58451390 2024-04-24T09:14:44.040 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.039+0000 7fad02000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ea58446ad0 0x55ea58448b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:44.040 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.039+0000 7fad0b400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ea58451390 0x55ea58469270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:44.040 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.039+0000 7fad0b400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ea58451390 0x55ea58469270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:60396/0 (socket says 172.21.15.83:60396) 2024-04-24T09:14:44.041 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.039+0000 7fad0b400640 1 -- 172.21.15.83:0/1946596256 learned_addr learned my addr 172.21.15.83:0/1946596256 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:14:44.041 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.039+0000 7fad0aa00640 1 --2- 172.21.15.83:0/1946596256 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ea584461b0 0x55ea5845ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:44.041 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.040+0000 7fad0b400640 1 -- 172.21.15.83:0/1946596256 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ea584461b0 msgr2=0x55ea5845ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:14:44.041 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.040+0000 7fad0b400640 1 --2- 172.21.15.83:0/1946596256 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ea584461b0 0x55ea5845ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:44.042 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.040+0000 7fad0b400640 1 -- 172.21.15.83:0/1946596256 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ea58446ad0 msgr2=0x55ea58448b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:14:44.042 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.040+0000 7fad0b400640 1 --2- 172.21.15.83:0/1946596256 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ea58446ad0 0x55ea58448b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:44.042 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.040+0000 7fad0b400640 1 -- 172.21.15.83:0/1946596256 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea5843dea0 con 0x55ea58451390 2024-04-24T09:14:44.042 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.040+0000 7fad0b400640 1 --2- 172.21.15.83:0/1946596256 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ea58451390 0x55ea58469270 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fad04009890 tx=0x7fad04003dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a67142ba11f17ed server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:14:44.042 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.041+0000 7facfb400640 1 -- 172.21.15.83:0/1946596256 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad04030d20 con 0x55ea58451390 2024-04-24T09:14:44.042 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.041+0000 7facfb400640 1 -- 172.21.15.83:0/1946596256 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fad0406c070 con 0x55ea58451390 2024-04-24T09:14:44.042 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.041+0000 7facfb400640 1 -- 172.21.15.83:0/1946596256 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad040319a0 con 0x55ea58451390 2024-04-24T09:14:44.043 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.042+0000 7fad14dc0740 1 -- 172.21.15.83:0/1946596256 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ea58451390 msgr2=0x55ea58469270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:44.043 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.042+0000 7fad14dc0740 1 --2- 172.21.15.83:0/1946596256 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ea58451390 0x55ea58469270 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fad04009890 tx=0x7fad04003dc0 comp rx=0 tx=0).stop 2024-04-24T09:14:44.043 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.042+0000 7fad14dc0740 1 -- 172.21.15.83:0/1946596256 shutdown_connections 2024-04-24T09:14:44.043 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.042+0000 7fad14dc0740 1 --2- 172.21.15.83:0/1946596256 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ea58451390 0x55ea58469270 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:44.043 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.042+0000 7fad14dc0740 1 --2- 172.21.15.83:0/1946596256 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ea58446ad0 0x55ea58448b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:44.043 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.042+0000 7fad14dc0740 1 --2- 172.21.15.83:0/1946596256 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ea584461b0 0x55ea5845ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:44.043 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.042+0000 7fad14dc0740 1 -- 172.21.15.83:0/1946596256 >> 172.21.15.83:0/1946596256 conn(0x55ea584426c0 msgr2=0x55ea58449670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:14:44.043 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.043+0000 7fad14dc0740 1 -- 172.21.15.83:0/1946596256 shutdown_connections 2024-04-24T09:14:44.043 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.043+0000 7fad14dc0740 1 -- 172.21.15.83:0/1946596256 wait complete. 2024-04-24T09:14:44.044 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.043+0000 7fad14dc0740 1 -- start start 2024-04-24T09:14:44.044 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.043+0000 7fad14dc0740 1 Processor -- start 2024-04-24T09:14:44.044 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.044+0000 7fad14dc0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ea584461b0 0x55ea58459780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:44.044 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.044+0000 7fad14dc0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ea58446ad0 0x55ea5844ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:44.044 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.044+0000 7fad14dc0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ea58451390 0x55ea5844b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:44.045 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.044+0000 7fad14dc0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fad0c1ae840 con 0x55ea58446ad0 2024-04-24T09:14:44.045 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.044+0000 7fad14dc0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fad0c1ae170 con 0x55ea584461b0 2024-04-24T09:14:44.045 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.044+0000 7fad14dc0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fad0c1ab3d0 con 0x55ea58451390 2024-04-24T09:14:44.045 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.044+0000 7fad14dc0740 10 client.0 _ll_register_callbacks cb 0x7fad04009a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:14:44.046 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.044+0000 7fad0aa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ea584461b0 0x55ea58459780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:44.046 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.044+0000 7fad0aa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ea584461b0 0x55ea58459780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:59490/0 (socket says 172.21.15.83:59490) 2024-04-24T09:14:44.046 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.044+0000 7fad0aa00640 1 -- 172.21.15.83:0/3050497896 learned_addr learned my addr 172.21.15.83:0/3050497896 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:14:44.046 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.044+0000 7fad0b400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ea58451390 0x55ea5844b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:44.046 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.045+0000 7fad02000640 1 --2- 172.21.15.83:0/3050497896 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ea58446ad0 0x55ea5844ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:44.046 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.045+0000 7fad0aa00640 1 -- 172.21.15.83:0/3050497896 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ea58451390 msgr2=0x55ea5844b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:14:44.046 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.045+0000 7fad0aa00640 1 --2- 172.21.15.83:0/3050497896 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ea58451390 0x55ea5844b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:44.046 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.045+0000 7fad0aa00640 1 -- 172.21.15.83:0/3050497896 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ea58446ad0 msgr2=0x55ea5844ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:14:44.046 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.045+0000 7fad0aa00640 1 --2- 172.21.15.83:0/3050497896 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ea58446ad0 0x55ea5844ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:44.046 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.045+0000 7fad0aa00640 1 -- 172.21.15.83:0/3050497896 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea5818b620 con 0x55ea584461b0 2024-04-24T09:14:44.046 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.045+0000 7fad0b400640 1 --2- 172.21.15.83:0/3050497896 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ea58451390 0x55ea5844b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:14:44.046 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.046+0000 7fad0aa00640 1 --2- 172.21.15.83:0/3050497896 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ea584461b0 0x55ea58459780 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7facfc000c00 tx=0x7facfc0039c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ddaf537bcc6a09ba server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:14:44.047 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.046+0000 7facf8c00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:14:44.048 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.046+0000 7facf8c00640 1 -- 172.21.15.83:0/3050497896 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facfc020620 con 0x55ea584461b0 2024-04-24T09:14:44.048 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.046+0000 7fad14dc0740 1 -- 172.21.15.83:0/3050497896 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad0406c070 con 0x55ea584461b0 2024-04-24T09:14:44.048 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.046+0000 7facf8c00640 1 -- 172.21.15.83:0/3050497896 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(128..128 src has 1..128) ==== 9583+0+0 (secure 0 0 0) 0x7facfc0207c0 con 0x55ea584461b0 2024-04-24T09:14:44.048 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.046+0000 7facf8c00640 1 -- 172.21.15.83:0/3050497896 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7facfc02e050 con 0x55ea584461b0 2024-04-24T09:14:44.048 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.047+0000 7fad14dc0740 10 client.40502 fetch_fsmap learned FSMap version 66 2024-04-24T09:14:44.048 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.047+0000 7fad14dc0740 1 -- 172.21.15.83:0/3050497896 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x55ea584c79a0 con 0x55ea584461b0 2024-04-24T09:14:44.048 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.047+0000 7facf8c00640 1 -- 172.21.15.83:0/3050497896 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7facfc024030 con 0x55ea584461b0 2024-04-24T09:14:44.048 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.047+0000 7facf8c00640 1 -- 172.21.15.83:0/3050497896 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 66) ==== 8366+0+0 (secure 0 0 0) 0x7facfc004c90 con 0x55ea584461b0 2024-04-24T09:14:44.048 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.047+0000 7fad14dc0740 10 client.40502 fetch_fsmap finished waiting for FSMap version 66 2024-04-24T09:14:44.048 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.047+0000 7fad14dc0740 10 client.40502 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:14:44.048 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.047+0000 7fad14dc0740 20 client.40502 populate_metadata read hostname 'smithi083' 2024-04-24T09:14:44.048 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.047+0000 7fad14dc0740 1 --2- 172.21.15.83:0/3050497896 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ea584c8110 0x55ea584ca500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:44.048 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.047+0000 7fad14dc0740 4 client.40502 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T09:14:44.049 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.047+0000 7fad14dc0740 1 -- 172.21.15.83:0/3050497896 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fad0406c070 con 0x55ea584c8110 2024-04-24T09:14:44.049 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.048+0000 7fad02000640 1 --2- 172.21.15.83:0/3050497896 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ea584c8110 0x55ea584ca500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:44.049 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.049+0000 7facf8c00640 10 client.40502 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:14:44.049 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.049+0000 7fad02000640 1 --2- 172.21.15.83:0/3050497896 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ea584c8110 0x55ea584ca500 crc :-1 s=READY pgs=570 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7e0f492c431bbe4 server_cookie=a7a80453f0caf572 in_seq=0 out_seq=0 2024-04-24T09:14:44.208 INFO:tasks.workunit.client.0.smithi083.stdout: 10 68615 6.60 MB/sec execute 435 sec latency 118.419 ms 2024-04-24T09:14:44.222 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.222+0000 7facf8c00640 1 -- 172.21.15.83:0/3050497896 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7fad0406c070 con 0x55ea584c8110 2024-04-24T09:14:44.222 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.222+0000 7facf8c00640 10 client.40502 handle_command_reply: tid=0 2024-04-24T09:14:44.222 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.222+0000 7fad14dc0740 1 client.40502 shutdown 2024-04-24T09:14:44.222 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.222+0000 7fad14dc0740 1 -- 172.21.15.83:0/3050497896 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ea584461b0 msgr2=0x55ea58459780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:44.223 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.222+0000 7fad14dc0740 1 --2- 172.21.15.83:0/3050497896 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ea584461b0 0x55ea58459780 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7facfc000c00 tx=0x7facfc0039c0 comp rx=0 tx=0).stop 2024-04-24T09:14:44.223 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.222+0000 7fad14dc0740 1 -- 172.21.15.83:0/3050497896 shutdown_connections 2024-04-24T09:14:44.223 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.222+0000 7fad14dc0740 1 --2- 172.21.15.83:0/3050497896 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55ea584c8110 0x55ea584ca500 crc :-1 s=READY pgs=570 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:44.224 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.222+0000 7fad14dc0740 1 --2- 172.21.15.83:0/3050497896 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ea58451390 0x55ea5844b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:44.224 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.222+0000 7fad14dc0740 1 --2- 172.21.15.83:0/3050497896 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ea58446ad0 0x55ea5844ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:44.224 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.222+0000 7fad14dc0740 1 --2- 172.21.15.83:0/3050497896 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ea584461b0 0x55ea58459780 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:44.224 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.222+0000 7facf8c00640 0 client.40502 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:14:44.224 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.222+0000 7fad14dc0740 1 -- 172.21.15.83:0/3050497896 >> 172.21.15.83:0/3050497896 conn(0x55ea584426c0 msgr2=0x55ea58440fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:14:44.224 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.223+0000 7fad14dc0740 1 -- 172.21.15.83:0/3050497896 shutdown_connections 2024-04-24T09:14:44.224 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.223+0000 7fad14dc0740 1 -- 172.21.15.83:0/3050497896 wait complete. 2024-04-24T09:14:44.224 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.223+0000 7fad14dc0740 20 client.40502 trim_cache size 0 max 16384 2024-04-24T09:14:44.225 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T09:14:44.225 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T09:14:44.225 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T09:14:44.225 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T09:14:44.227 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.226+0000 7fad11e00640 1 -- 172.21.15.83:0/2010859161 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7facf00a3f70 msgr2=0x7facf00a6430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:44.227 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.227+0000 7fad11e00640 1 --2- 172.21.15.83:0/2010859161 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7facf00a3f70 0x7facf00a6430 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fad04004af0 tx=0x7fad04040040 comp rx=0 tx=0).stop 2024-04-24T09:14:44.227 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.227+0000 7fad11e00640 1 -- 172.21.15.83:0/2010859161 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad0c100a90 msgr2=0x7fad0c1a0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:44.228 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.227+0000 7fad11e00640 1 --2- 172.21.15.83:0/2010859161 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad0c100a90 0x7fad0c1a0bc0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7facf4005fd0 tx=0x7facf400dde0 comp rx=0 tx=0).stop 2024-04-24T09:14:44.228 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.227+0000 7fad11e00640 1 -- 172.21.15.83:0/2010859161 shutdown_connections 2024-04-24T09:14:44.228 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.227+0000 7fad11e00640 1 --2- 172.21.15.83:0/2010859161 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7facf00a3f70 0x7facf00a6430 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:44.228 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.227+0000 7fad11e00640 1 --2- 172.21.15.83:0/2010859161 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad0c107120 0x7fad0c19a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:44.228 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.227+0000 7fad11e00640 1 --2- 172.21.15.83:0/2010859161 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad0c100a90 0x7fad0c1a0bc0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:44.228 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.227+0000 7fad11e00640 1 --2- 172.21.15.83:0/2010859161 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fad0c1000d0 0x7fad0c1a0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:44.228 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.227+0000 7fad11e00640 1 -- 172.21.15.83:0/2010859161 >> 172.21.15.83:0/2010859161 conn(0x7fad0c077000 msgr2=0x7fad0c103070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:14:44.228 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.227+0000 7fad11e00640 1 -- 172.21.15.83:0/2010859161 shutdown_connections 2024-04-24T09:14:44.228 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:44.227+0000 7fad11e00640 1 -- 172.21.15.83:0/2010859161 wait complete. 2024-04-24T09:14:44.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:14:44 smithi138 ceph-mon[31441]: pgmap v730: 129 pgs: 129 active+clean; 1.0 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.0 MiB/s wr, 325 op/s 2024-04-24T09:14:44.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:14:44 smithi189 ceph-mon[31791]: pgmap v730: 129 pgs: 129 active+clean; 1.0 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.0 MiB/s wr, 325 op/s 2024-04-24T09:14:44.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:14:44 smithi083 ceph-mon[27907]: pgmap v730: 129 pgs: 129 active+clean; 1.0 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.0 MiB/s wr, 325 op/s 2024-04-24T09:14:45.208 INFO:tasks.workunit.client.0.smithi083.stdout: 10 68720 6.60 MB/sec execute 436 sec latency 151.988 ms 2024-04-24T09:14:45.704 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:14:45 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:14:45.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:14:45 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:14:45.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:14:45 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:14:45.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:14:45 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:14:45.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:14:45 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:14:45.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:14:45 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:14:45.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:14:45 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:14:45.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:14:45 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:14:45.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:14:45 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:14:46.208 INFO:tasks.workunit.client.0.smithi083.stdout: 10 68859 6.61 MB/sec execute 437 sec latency 173.818 ms 2024-04-24T09:14:46.704 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:14:46 smithi138 ceph-mon[31441]: pgmap v731: 129 pgs: 129 active+clean; 1.0 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.4 MiB/s wr, 168 op/s 2024-04-24T09:14:46.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:14:46 smithi189 ceph-mon[31791]: pgmap v731: 129 pgs: 129 active+clean; 1.0 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.4 MiB/s wr, 168 op/s 2024-04-24T09:14:46.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:14:46 smithi083 ceph-mon[27907]: pgmap v731: 129 pgs: 129 active+clean; 1.0 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.4 MiB/s wr, 168 op/s 2024-04-24T09:14:47.208 INFO:tasks.workunit.client.0.smithi083.stdout: 10 68972 6.60 MB/sec execute 438 sec latency 122.065 ms 2024-04-24T09:14:48.208 INFO:tasks.workunit.client.0.smithi083.stdout: 10 69093 6.61 MB/sec execute 439 sec latency 74.695 ms 2024-04-24T09:14:48.704 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:14:48 smithi138 ceph-mon[31441]: pgmap v732: 129 pgs: 129 active+clean; 1.0 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.4 MiB/s wr, 168 op/s 2024-04-24T09:14:48.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:14:48 smithi189 ceph-mon[31791]: pgmap v732: 129 pgs: 129 active+clean; 1.0 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.4 MiB/s wr, 168 op/s 2024-04-24T09:14:48.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:14:48 smithi083 ceph-mon[27907]: pgmap v732: 129 pgs: 129 active+clean; 1.0 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.4 MiB/s wr, 168 op/s 2024-04-24T09:14:49.209 INFO:tasks.workunit.client.0.smithi083.stdout: 10 69199 6.60 MB/sec execute 440 sec latency 77.974 ms 2024-04-24T09:14:50.208 INFO:tasks.workunit.client.0.smithi083.stdout: 10 69312 6.61 MB/sec execute 441 sec latency 183.796 ms 2024-04-24T09:14:50.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:14:50 smithi138 ceph-mon[31441]: pgmap v733: 129 pgs: 129 active+clean; 1.0 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.6 MiB/s wr, 298 op/s 2024-04-24T09:14:50.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:14:50 smithi189 ceph-mon[31791]: pgmap v733: 129 pgs: 129 active+clean; 1.0 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.6 MiB/s wr, 298 op/s 2024-04-24T09:14:50.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:14:50 smithi083 ceph-mon[27907]: pgmap v733: 129 pgs: 129 active+clean; 1.0 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.6 MiB/s wr, 298 op/s 2024-04-24T09:14:51.208 INFO:tasks.workunit.client.0.smithi083.stdout: 10 69386 6.59 MB/sec execute 442 sec latency 348.843 ms 2024-04-24T09:14:52.209 INFO:tasks.workunit.client.0.smithi083.stdout: 10 69493 6.58 MB/sec execute 443 sec latency 87.621 ms 2024-04-24T09:14:52.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:14:52 smithi138 ceph-mon[31441]: pgmap v734: 129 pgs: 129 active+clean; 1.0 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.6 MiB/s wr, 215 op/s 2024-04-24T09:14:52.706 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:14:52 smithi189 ceph-mon[31791]: pgmap v734: 129 pgs: 129 active+clean; 1.0 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.6 MiB/s wr, 215 op/s 2024-04-24T09:14:52.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:14:52 smithi083 ceph-mon[27907]: pgmap v734: 129 pgs: 129 active+clean; 1.0 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.6 MiB/s wr, 215 op/s 2024-04-24T09:14:53.208 INFO:tasks.workunit.client.0.smithi083.stdout: 10 69623 6.58 MB/sec execute 444 sec latency 98.393 ms 2024-04-24T09:14:54.208 INFO:tasks.workunit.client.0.smithi083.stdout: 10 69731 6.57 MB/sec execute 445 sec latency 92.438 ms 2024-04-24T09:14:54.704 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:14:54 smithi138 ceph-mon[31441]: pgmap v735: 129 pgs: 129 active+clean; 1.0 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 6.2 MiB/s wr, 316 op/s 2024-04-24T09:14:54.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:14:54 smithi138 ceph-mon[31441]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:14:54.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:14:54 smithi189 ceph-mon[31791]: pgmap v735: 129 pgs: 129 active+clean; 1.0 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 6.2 MiB/s wr, 316 op/s 2024-04-24T09:14:54.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:14:54 smithi189 ceph-mon[31791]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:14:54.802 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:14:54 smithi083 ceph-mon[27907]: pgmap v735: 129 pgs: 129 active+clean; 1.0 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 6.2 MiB/s wr, 316 op/s 2024-04-24T09:14:54.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:14:54 smithi083 ceph-mon[27907]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:14:55.209 INFO:tasks.workunit.client.0.smithi083.stdout: 10 69844 6.57 MB/sec execute 446 sec latency 216.576 ms 2024-04-24T09:14:55.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:14:55 smithi083 ceph-mon[27907]: pgmap v736: 129 pgs: 129 active+clean; 1.0 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.8 MiB/s wr, 231 op/s 2024-04-24T09:14:55.954 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:14:55 smithi138 ceph-mon[31441]: pgmap v736: 129 pgs: 129 active+clean; 1.0 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.8 MiB/s wr, 231 op/s 2024-04-24T09:14:55.956 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:14:55 smithi189 ceph-mon[31791]: pgmap v736: 129 pgs: 129 active+clean; 1.0 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.8 MiB/s wr, 231 op/s 2024-04-24T09:14:56.208 INFO:tasks.workunit.client.0.smithi083.stdout: 10 69965 6.56 MB/sec execute 447 sec latency 311.051 ms 2024-04-24T09:14:57.209 INFO:tasks.workunit.client.0.smithi083.stdout: 10 70088 6.55 MB/sec execute 448 sec latency 77.617 ms 2024-04-24T09:14:57.442 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.442+0000 7fedb9400640 1 -- 172.21.15.83:0/259936404 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedb4104e10 msgr2=0x7fedb4108e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:57.442 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.442+0000 7fedb9400640 1 --2- 172.21.15.83:0/259936404 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedb4104e10 0x7fedb4108e50 secure :-1 s=READY pgs=2685 cs=0 l=1 rev1=1 crypto rx=0x7feda0009900 tx=0x7feda002a3f0 comp rx=0 tx=0).stop 2024-04-24T09:14:57.442 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.442+0000 7fedb9400640 1 -- 172.21.15.83:0/259936404 shutdown_connections 2024-04-24T09:14:57.442 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.442+0000 7fedb9400640 1 --2- 172.21.15.83:0/259936404 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fedb4109530 0x7fedb4111100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:57.442 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.442+0000 7fedb9400640 1 --2- 172.21.15.83:0/259936404 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedb4104e10 0x7fedb4108e50 unknown :-1 s=CLOSED pgs=2685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:57.442 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.442+0000 7fedb9400640 1 --2- 172.21.15.83:0/259936404 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fedb4104460 0x7fedb4104840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:57.442 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.442+0000 7fedb9400640 1 -- 172.21.15.83:0/259936404 >> 172.21.15.83:0/259936404 conn(0x7fedb40ffd00 msgr2=0x7fedb4102120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:14:57.442 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.442+0000 7fedb9400640 1 -- 172.21.15.83:0/259936404 shutdown_connections 2024-04-24T09:14:57.442 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.443+0000 7fedb9400640 1 -- 172.21.15.83:0/259936404 wait complete. 2024-04-24T09:14:57.443 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.444+0000 7fedb9400640 1 Processor -- start 2024-04-24T09:14:57.443 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.444+0000 7fedb9400640 1 -- start start 2024-04-24T09:14:57.444 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.445+0000 7fedb9400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fedb4104460 0x7fedb41a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:57.444 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.445+0000 7fedb9400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fedb4109530 0x7fedb41a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:57.444 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.445+0000 7fedb9400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedb419a7a0 0x7fedb419ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:57.444 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.445+0000 7fedb9400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fedb41144a0 con 0x7fedb419a7a0 2024-04-24T09:14:57.444 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.445+0000 7fedb9400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fedb4114320 con 0x7fedb4104460 2024-04-24T09:14:57.444 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.445+0000 7fedb9400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fedb4114620 con 0x7fedb4109530 2024-04-24T09:14:57.445 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.445+0000 7fedb3200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedb419a7a0 0x7fedb419ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:57.445 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.445+0000 7fedb3200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedb419a7a0 0x7fedb419ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35828/0 (socket says 172.21.15.83:35828) 2024-04-24T09:14:57.445 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.445+0000 7fedb2800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fedb4104460 0x7fedb41a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:57.445 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.445+0000 7fedb3200640 1 -- 172.21.15.83:0/3685433739 learned_addr learned my addr 172.21.15.83:0/3685433739 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:14:57.445 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.445+0000 7fedb1e00640 1 --2- 172.21.15.83:0/3685433739 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fedb4109530 0x7fedb41a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:57.445 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.445+0000 7fedb2800640 1 -- 172.21.15.83:0/3685433739 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fedb4109530 msgr2=0x7fedb41a0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:57.445 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.445+0000 7fedb2800640 1 --2- 172.21.15.83:0/3685433739 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fedb4109530 0x7fedb41a0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:57.446 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.446+0000 7fedb2800640 1 -- 172.21.15.83:0/3685433739 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedb419a7a0 msgr2=0x7fedb419ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:57.446 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.446+0000 7fedb2800640 1 --2- 172.21.15.83:0/3685433739 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedb419a7a0 0x7fedb419ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:57.446 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.446+0000 7fedb2800640 1 -- 172.21.15.83:0/3685433739 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedb419b330 con 0x7fedb4104460 2024-04-24T09:14:57.446 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.446+0000 7fedb3200640 1 --2- 172.21.15.83:0/3685433739 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedb419a7a0 0x7fedb419ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:14:57.446 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.446+0000 7fedb2800640 1 --2- 172.21.15.83:0/3685433739 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fedb4104460 0x7fedb41a0630 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7feda400e9f0 tx=0x7feda400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:14:57.447 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.446+0000 7fedab400640 1 -- 172.21.15.83:0/3685433739 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feda400ccc0 con 0x7fedb4104460 2024-04-24T09:14:57.447 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.446+0000 7fedab400640 1 -- 172.21.15.83:0/3685433739 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feda40108e0 con 0x7fedb4104460 2024-04-24T09:14:57.447 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.446+0000 7fedab400640 1 -- 172.21.15.83:0/3685433739 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feda4018880 con 0x7fedb4104460 2024-04-24T09:14:57.447 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.447+0000 7fedb9400640 1 -- 172.21.15.83:0/3685433739 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedb419b620 con 0x7fedb4104460 2024-04-24T09:14:57.447 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.447+0000 7fedb9400640 1 -- 172.21.15.83:0/3685433739 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedb41a7780 con 0x7fedb4104460 2024-04-24T09:14:57.452 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.451+0000 7fedab400640 1 -- 172.21.15.83:0/3685433739 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7feda4010a90 con 0x7fedb4104460 2024-04-24T09:14:57.452 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.452+0000 7fedbc27e740 1 Processor -- start 2024-04-24T09:14:57.452 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.452+0000 7fedbc27e740 1 -- start start 2024-04-24T09:14:57.452 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.452+0000 7fedbc27e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b34774e880 0x55b34774ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:57.452 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.452+0000 7fedbc27e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b34774f1a0 0x55b34774a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:57.453 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.452+0000 7fedbc27e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b347762610 0x55b347764a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:57.453 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.452+0000 7fedbc27e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55b3476ac900 con 0x55b34774e880 2024-04-24T09:14:57.453 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.452+0000 7fedbc27e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b347652f20 con 0x55b34774f1a0 2024-04-24T09:14:57.453 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.452+0000 7fedbc27e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55b3476a88c0 con 0x55b347762610 2024-04-24T09:14:57.454 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.453+0000 7fedb2800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b34774f1a0 0x55b34774a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:57.454 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.453+0000 7fedb3200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b34774e880 0x55b34774ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:57.454 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.453+0000 7fedb2800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b34774f1a0 0x55b34774a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:55786/0 (socket says 172.21.15.83:55786) 2024-04-24T09:14:57.454 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.453+0000 7fedb2800640 1 -- 172.21.15.83:0/1139721870 learned_addr learned my addr 172.21.15.83:0/1139721870 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:14:57.454 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.453+0000 7fedb1e00640 1 --2- 172.21.15.83:0/1139721870 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b347762610 0x55b347764a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:57.454 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.453+0000 7fedab400640 1 --2- 172.21.15.83:0/3685433739 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fed980a3f20 0x7fed980a63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:57.454 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.453+0000 7fedab400640 1 -- 172.21.15.83:0/3685433739 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9583+0+0 (secure 0 0 0) 0x7feda40d0ed0 con 0x7fedb4104460 2024-04-24T09:14:57.454 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.454+0000 7fedb2800640 1 -- 172.21.15.83:0/1139721870 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b347762610 msgr2=0x55b347764a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:14:57.455 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.454+0000 7fedb2800640 1 --2- 172.21.15.83:0/1139721870 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b347762610 0x55b347764a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:57.455 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.454+0000 7fedb2800640 1 -- 172.21.15.83:0/1139721870 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b34774e880 msgr2=0x55b34774ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:14:57.455 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.454+0000 7fedb2800640 1 --2- 172.21.15.83:0/1139721870 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b34774e880 0x55b34774ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:57.455 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.454+0000 7fedb2800640 1 -- 172.21.15.83:0/1139721870 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b347677e10 con 0x55b34774f1a0 2024-04-24T09:14:57.455 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.454+0000 7fedb3200640 1 --2- 172.21.15.83:0/1139721870 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b34774e880 0x55b34774ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:14:57.455 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.454+0000 7fedb3200640 1 --2- 172.21.15.83:0/3685433739 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fed980a3f20 0x7fed980a63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:57.455 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.454+0000 7fedb2800640 1 --2- 172.21.15.83:0/1139721870 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b34774f1a0 0x55b34774a0d0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7feda409c9e0 tx=0x7feda4009700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85886541e5401df2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:14:57.455 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.455+0000 7fed9f400640 1 -- 172.21.15.83:0/1139721870 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feda40bd0d0 con 0x55b34774f1a0 2024-04-24T09:14:57.456 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.455+0000 7fed9f400640 1 -- 172.21.15.83:0/1139721870 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feda40d4630 con 0x55b34774f1a0 2024-04-24T09:14:57.456 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.455+0000 7fed9f400640 1 -- 172.21.15.83:0/1139721870 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feda40990c0 con 0x55b34774f1a0 2024-04-24T09:14:57.456 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.456+0000 7fedb3200640 1 --2- 172.21.15.83:0/3685433739 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fed980a3f20 0x7fed980a63e0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fedac009370 tx=0x7fedac014040 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:14:57.457 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.456+0000 7fedbc27e740 1 -- 172.21.15.83:0/1139721870 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b34774f1a0 msgr2=0x55b34774a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:57.457 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.456+0000 7fedbc27e740 1 --2- 172.21.15.83:0/1139721870 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b34774f1a0 0x55b34774a0d0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7feda409c9e0 tx=0x7feda4009700 comp rx=0 tx=0).stop 2024-04-24T09:14:57.457 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.456+0000 7fedbc27e740 1 -- 172.21.15.83:0/1139721870 shutdown_connections 2024-04-24T09:14:57.457 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.456+0000 7fedbc27e740 1 --2- 172.21.15.83:0/1139721870 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b347762610 0x55b347764a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:57.457 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.456+0000 7fedbc27e740 1 --2- 172.21.15.83:0/1139721870 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b34774f1a0 0x55b34774a0d0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:57.457 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.456+0000 7fedbc27e740 1 --2- 172.21.15.83:0/1139721870 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b34774e880 0x55b34774ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:57.457 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.456+0000 7fedbc27e740 1 -- 172.21.15.83:0/1139721870 >> 172.21.15.83:0/1139721870 conn(0x55b34774aed0 msgr2=0x55b34774b2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:14:57.457 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.456+0000 7fedbc27e740 1 -- 172.21.15.83:0/1139721870 shutdown_connections 2024-04-24T09:14:57.457 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.456+0000 7fedbc27e740 1 -- 172.21.15.83:0/1139721870 wait complete. 2024-04-24T09:14:57.457 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.457+0000 7fedbc27e740 1 -- start start 2024-04-24T09:14:57.457 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.457+0000 7fedbc27e740 1 Processor -- start 2024-04-24T09:14:57.458 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.459+0000 7fedbc27e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b34774e880 0x55b347747590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:57.458 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.459+0000 7fedbc27e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b34774f1a0 0x55b347747ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:57.459 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.459+0000 7fedbc27e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b347762610 0x55b347748010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:57.459 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.459+0000 7fedbc27e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55b347764f40 con 0x55b34774f1a0 2024-04-24T09:14:57.459 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.459+0000 7fedbc27e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b3476662c0 con 0x55b347762610 2024-04-24T09:14:57.459 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.459+0000 7fedbc27e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55b3477650c0 con 0x55b34774e880 2024-04-24T09:14:57.459 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.459+0000 7fedbc27e740 10 client.0 _ll_register_callbacks cb 0x55b3473bea00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:14:57.459 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.459+0000 7fedb3200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b347762610 0x55b347748010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:57.459 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.459+0000 7fedb1e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b34774f1a0 0x55b347747ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:57.459 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.459+0000 7fedb1e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b34774f1a0 0x55b347747ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35852/0 (socket says 172.21.15.83:35852) 2024-04-24T09:14:57.459 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.459+0000 7fedb1e00640 1 -- 172.21.15.83:0/2230098329 learned_addr learned my addr 172.21.15.83:0/2230098329 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:14:57.460 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.459+0000 7fedb2800640 1 --2- 172.21.15.83:0/2230098329 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b34774e880 0x55b347747590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:57.460 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.460+0000 7fedb3200640 1 -- 172.21.15.83:0/2230098329 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b34774e880 msgr2=0x55b347747590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:14:57.460 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.460+0000 7fedb3200640 1 --2- 172.21.15.83:0/2230098329 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b34774e880 0x55b347747590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:57.460 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.460+0000 7fedb3200640 1 -- 172.21.15.83:0/2230098329 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b34774f1a0 msgr2=0x55b347747ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:14:57.461 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.460+0000 7fedb3200640 1 --2- 172.21.15.83:0/2230098329 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b34774f1a0 0x55b347747ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:57.461 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.460+0000 7fedb3200640 1 -- 172.21.15.83:0/2230098329 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b347676fd0 con 0x55b347762610 2024-04-24T09:14:57.461 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.460+0000 7fedb1e00640 1 --2- 172.21.15.83:0/2230098329 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b34774f1a0 0x55b347747ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:14:57.461 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.461+0000 7fedb3200640 1 --2- 172.21.15.83:0/2230098329 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b347762610 0x55b347748010 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fedac009fd0 tx=0x7fedac004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=358c2f6436c6dac7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:14:57.461 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.461+0000 7fed93e00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:14:57.461 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.461+0000 7fed93e00640 1 -- 172.21.15.83:0/2230098329 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedac003b70 con 0x55b347762610 2024-04-24T09:14:57.461 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.461+0000 7fedbc27e740 1 -- 172.21.15.83:0/2230098329 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b3476ba320 con 0x55b347762610 2024-04-24T09:14:57.462 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.462+0000 7fed93e00640 1 -- 172.21.15.83:0/2230098329 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(128..128 src has 1..128) ==== 9583+0+0 (secure 0 0 0) 0x7fedac003d10 con 0x55b347762610 2024-04-24T09:14:57.462 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.462+0000 7fed93e00640 1 -- 172.21.15.83:0/2230098329 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7fedac031050 con 0x55b347762610 2024-04-24T09:14:57.462 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.462+0000 7fedbc27e740 10 client.40514 fetch_fsmap learned FSMap version 67 2024-04-24T09:14:57.462 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.462+0000 7fedbc27e740 1 -- 172.21.15.83:0/2230098329 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) -- 0x55b3473bf210 con 0x55b347762610 2024-04-24T09:14:57.463 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.463+0000 7fed93e00640 1 -- 172.21.15.83:0/2230098329 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55b3476ba320 con 0x55b347762610 2024-04-24T09:14:57.463 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.463+0000 7fed93e00640 1 -- 172.21.15.83:0/2230098329 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 67) ==== 8370+0+0 (secure 0 0 0) 0x7fedac026490 con 0x55b347762610 2024-04-24T09:14:57.463 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.463+0000 7fedbc27e740 10 client.40514 fetch_fsmap finished waiting for FSMap version 67 2024-04-24T09:14:57.463 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.463+0000 7fedbc27e740 10 client.40514 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:14:57.463 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.463+0000 7fedbc27e740 20 client.40514 populate_metadata read hostname 'smithi083' 2024-04-24T09:14:57.463 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.463+0000 7fedbc27e740 1 --2- 172.21.15.83:0/2230098329 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55b347772660 0x55b347774a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:57.463 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.463+0000 7fedbc27e740 4 client.40514 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:14:57.463 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.463+0000 7fedbc27e740 1 -- 172.21.15.83:0/2230098329 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b3476b4190 con 0x55b347772660 2024-04-24T09:14:57.464 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.464+0000 7fedb2800640 1 --2- 172.21.15.83:0/2230098329 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55b347772660 0x55b347774a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:57.464 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.465+0000 7fedb2800640 1 --2- 172.21.15.83:0/2230098329 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55b347772660 0x55b347774a90 crc :-1 s=READY pgs=571 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bde11cbe14ca9a87 server_cookie=99ed66727a0e9e08 in_seq=0 out_seq=0 2024-04-24T09:14:57.464 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.465+0000 7fed93e00640 10 client.40514 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:14:57.465 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.466+0000 7fed93e00640 1 -- 172.21.15.83:0/2230098329 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b3476b4190 con 0x55b347772660 2024-04-24T09:14:57.465 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.466+0000 7fed93e00640 10 client.40514 handle_command_reply: tid=0 2024-04-24T09:14:57.465 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.466+0000 7fedbc27e740 1 client.40514 shutdown 2024-04-24T09:14:57.466 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.467+0000 7fedbc27e740 1 -- 172.21.15.83:0/2230098329 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b347762610 msgr2=0x55b347748010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:57.466 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.467+0000 7fedbc27e740 1 --2- 172.21.15.83:0/2230098329 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b347762610 0x55b347748010 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fedac009fd0 tx=0x7fedac004a40 comp rx=0 tx=0).stop 2024-04-24T09:14:57.467 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.467+0000 7fedbc27e740 1 -- 172.21.15.83:0/2230098329 shutdown_connections 2024-04-24T09:14:57.467 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.467+0000 7fedbc27e740 1 --2- 172.21.15.83:0/2230098329 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55b347772660 0x55b347774a90 crc :-1 s=READY pgs=571 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:57.467 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.467+0000 7fedbc27e740 1 --2- 172.21.15.83:0/2230098329 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b347762610 0x55b347748010 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:57.467 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.467+0000 7fedbc27e740 1 --2- 172.21.15.83:0/2230098329 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b34774f1a0 0x55b347747ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:57.467 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.467+0000 7fedbc27e740 1 --2- 172.21.15.83:0/2230098329 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b34774e880 0x55b347747590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:57.467 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.467+0000 7fedbc27e740 1 -- 172.21.15.83:0/2230098329 >> 172.21.15.83:0/2230098329 conn(0x55b34774aed0 msgr2=0x55b347750c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:14:57.467 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.467+0000 7fed93e00640 0 client.40514 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:14:57.467 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.467+0000 7fedbc27e740 1 -- 172.21.15.83:0/2230098329 shutdown_connections 2024-04-24T09:14:57.467 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.467+0000 7fedbc27e740 1 -- 172.21.15.83:0/2230098329 wait complete. 2024-04-24T09:14:57.467 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.467+0000 7fedbc27e740 20 client.40514 trim_cache size 0 max 16384 2024-04-24T09:14:57.494 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.494+0000 7fedbc27e740 1 Processor -- start 2024-04-24T09:14:57.495 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.495+0000 7fedbc27e740 1 -- start start 2024-04-24T09:14:57.495 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.495+0000 7fedbc27e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b34774e880 0x55b3477416c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:57.495 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.495+0000 7fedbc27e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b34774f1a0 0x55b34776de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:57.495 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.495+0000 7fedbc27e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b347762610 0x55b34777a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:57.495 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.495+0000 7fedbc27e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fedb41ad400 con 0x55b347762610 2024-04-24T09:14:57.495 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.495+0000 7fedbc27e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fedb41aefa0 con 0x55b34774e880 2024-04-24T09:14:57.495 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.495+0000 7fedbc27e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fedb41adad0 con 0x55b34774f1a0 2024-04-24T09:14:57.495 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.495+0000 7fedb2800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b34774e880 0x55b3477416c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:57.495 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.495+0000 7fedb3200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b347762610 0x55b34777a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:57.496 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.495+0000 7fedb2800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b34774e880 0x55b3477416c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:55812/0 (socket says 172.21.15.83:55812) 2024-04-24T09:14:57.496 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.495+0000 7fedb2800640 1 -- 172.21.15.83:0/164433647 learned_addr learned my addr 172.21.15.83:0/164433647 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:14:57.496 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.495+0000 7fedb1e00640 1 --2- 172.21.15.83:0/164433647 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b34774f1a0 0x55b34776de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:57.497 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.496+0000 7fedb3200640 1 -- 172.21.15.83:0/164433647 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b34774f1a0 msgr2=0x55b34776de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:14:57.498 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.496+0000 7fedb3200640 1 --2- 172.21.15.83:0/164433647 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b34774f1a0 0x55b34776de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:57.498 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.496+0000 7fedb3200640 1 -- 172.21.15.83:0/164433647 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b34774e880 msgr2=0x55b3477416c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:14:57.498 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.496+0000 7fedb3200640 1 --2- 172.21.15.83:0/164433647 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b34774e880 0x55b3477416c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:57.498 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.496+0000 7fedb3200640 1 -- 172.21.15.83:0/164433647 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b34748b620 con 0x55b347762610 2024-04-24T09:14:57.498 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.496+0000 7fedb3200640 1 --2- 172.21.15.83:0/164433647 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b347762610 0x55b34777a3a0 secure :-1 s=READY pgs=2686 cs=0 l=1 rev1=1 crypto rx=0x7fedac00e820 tx=0x7fedac01bc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37fbfec122356529 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:14:57.498 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.497+0000 7fed9ea00640 1 -- 172.21.15.83:0/164433647 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedac00ee90 con 0x55b347762610 2024-04-24T09:14:57.498 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.497+0000 7fed9ea00640 1 -- 172.21.15.83:0/164433647 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fedac026cd0 con 0x55b347762610 2024-04-24T09:14:57.498 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.497+0000 7fed9ea00640 1 -- 172.21.15.83:0/164433647 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedac035d40 con 0x55b347762610 2024-04-24T09:14:57.498 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.498+0000 7fedbc27e740 1 -- 172.21.15.83:0/164433647 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b347762610 msgr2=0x55b34777a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:57.498 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.498+0000 7fedbc27e740 1 --2- 172.21.15.83:0/164433647 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b347762610 0x55b34777a3a0 secure :-1 s=READY pgs=2686 cs=0 l=1 rev1=1 crypto rx=0x7fedac00e820 tx=0x7fedac01bc70 comp rx=0 tx=0).stop 2024-04-24T09:14:57.498 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.498+0000 7fedbc27e740 1 -- 172.21.15.83:0/164433647 shutdown_connections 2024-04-24T09:14:57.498 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.498+0000 7fedbc27e740 1 --2- 172.21.15.83:0/164433647 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b347762610 0x55b34777a3a0 unknown :-1 s=CLOSED pgs=2686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:57.499 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.498+0000 7fedbc27e740 1 --2- 172.21.15.83:0/164433647 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b34774f1a0 0x55b34776de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:57.499 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.498+0000 7fedbc27e740 1 --2- 172.21.15.83:0/164433647 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b34774e880 0x55b3477416c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:57.499 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.498+0000 7fedbc27e740 1 -- 172.21.15.83:0/164433647 >> 172.21.15.83:0/164433647 conn(0x55b34774aed0 msgr2=0x55b347740fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:14:57.499 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.498+0000 7fedbc27e740 1 -- 172.21.15.83:0/164433647 shutdown_connections 2024-04-24T09:14:57.499 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.498+0000 7fedbc27e740 1 -- 172.21.15.83:0/164433647 wait complete. 2024-04-24T09:14:57.499 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.498+0000 7fedbc27e740 1 -- start start 2024-04-24T09:14:57.499 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.499+0000 7fedbc27e740 1 Processor -- start 2024-04-24T09:14:57.501 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.499+0000 7fedbc27e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b34774e880 0x55b3477442e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:57.501 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.499+0000 7fedbc27e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b34774f1a0 0x55b347744820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:57.501 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.499+0000 7fedbc27e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b347762610 0x55b347744d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:57.501 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.499+0000 7fedbc27e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fedb41ae8a0 con 0x55b34774f1a0 2024-04-24T09:14:57.501 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.499+0000 7fedbc27e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fedb41ae1d0 con 0x55b34774e880 2024-04-24T09:14:57.501 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.499+0000 7fedbc27e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fedb41ab420 con 0x55b347762610 2024-04-24T09:14:57.501 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.499+0000 7fedbc27e740 10 client.0 _ll_register_callbacks cb 0x7fedac009440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:14:57.501 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.500+0000 7fedb2800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b34774e880 0x55b3477442e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:57.501 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.500+0000 7fedb1e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b34774f1a0 0x55b347744820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:57.501 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.500+0000 7fedb1e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b34774f1a0 0x55b347744820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35872/0 (socket says 172.21.15.83:35872) 2024-04-24T09:14:57.502 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.500+0000 7fedb2800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b34774e880 0x55b3477442e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:55814/0 (socket says 172.21.15.83:55814) 2024-04-24T09:14:57.502 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.500+0000 7fedb1e00640 1 -- 172.21.15.83:0/569268150 learned_addr learned my addr 172.21.15.83:0/569268150 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:14:57.502 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.500+0000 7fedb3200640 1 --2- 172.21.15.83:0/569268150 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b347762610 0x55b347744d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:57.502 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.500+0000 7fedb1e00640 1 -- 172.21.15.83:0/569268150 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b347762610 msgr2=0x55b347744d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:14:57.502 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.500+0000 7fedb1e00640 1 --2- 172.21.15.83:0/569268150 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b347762610 0x55b347744d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:57.502 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.501+0000 7fedb1e00640 1 -- 172.21.15.83:0/569268150 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b34774e880 msgr2=0x55b3477442e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:14:57.502 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.501+0000 7fedb1e00640 1 --2- 172.21.15.83:0/569268150 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b34774e880 0x55b3477442e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:57.502 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.501+0000 7fedb1e00640 1 -- 172.21.15.83:0/569268150 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3477d7c40 con 0x55b34774f1a0 2024-04-24T09:14:57.502 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.501+0000 7fedb2800640 1 --2- 172.21.15.83:0/569268150 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b34774e880 0x55b3477442e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:14:57.502 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.501+0000 7fedb1e00640 1 --2- 172.21.15.83:0/569268150 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b34774f1a0 0x55b347744820 secure :-1 s=READY pgs=2687 cs=0 l=1 rev1=1 crypto rx=0x7fedb406b790 tx=0x7feda0005610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27025d6a7c7d2b99 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:14:57.503 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.501+0000 7fed93e00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:14:57.504 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.502+0000 7fed93e00640 1 -- 172.21.15.83:0/569268150 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feda0035040 con 0x55b34774f1a0 2024-04-24T09:14:57.504 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.502+0000 7fed93e00640 1 -- 172.21.15.83:0/569268150 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(128..128 src has 1..128) ==== 9583+0+0 (secure 0 0 0) 0x7feda00048d0 con 0x55b34774f1a0 2024-04-24T09:14:57.504 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.502+0000 7fedbc27e740 1 -- 172.21.15.83:0/569268150 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fedac026cd0 con 0x55b34774f1a0 2024-04-24T09:14:57.504 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.502+0000 7fed93e00640 1 -- 172.21.15.83:0/569268150 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7feda0004af0 con 0x55b34774f1a0 2024-04-24T09:14:57.504 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.502+0000 7fedbc27e740 10 client.23457 fetch_fsmap learned FSMap version 67 2024-04-24T09:14:57.505 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.502+0000 7fedbc27e740 1 -- 172.21.15.83:0/569268150 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) -- 0x55b3477d8f80 con 0x55b34774f1a0 2024-04-24T09:14:57.505 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.503+0000 7fed93e00640 1 -- 172.21.15.83:0/569268150 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fedac026cd0 con 0x55b34774f1a0 2024-04-24T09:14:57.505 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.503+0000 7fed93e00640 1 -- 172.21.15.83:0/569268150 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 67) ==== 8370+0+0 (secure 0 0 0) 0x7feda00314a0 con 0x55b34774f1a0 2024-04-24T09:14:57.505 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.503+0000 7fedbc27e740 10 client.23457 fetch_fsmap finished waiting for FSMap version 67 2024-04-24T09:14:57.505 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.503+0000 7fedbc27e740 10 client.23457 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:14:57.505 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.503+0000 7fedbc27e740 20 client.23457 populate_metadata read hostname 'smithi083' 2024-04-24T09:14:57.505 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.503+0000 7fedbc27e740 1 --2- 172.21.15.83:0/569268150 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55b3477d9800 0x55b3477dbc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:14:57.505 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.503+0000 7fedbc27e740 4 client.23457 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T09:14:57.505 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.503+0000 7fedbc27e740 1 -- 172.21.15.83:0/569268150 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b3476304b0 con 0x55b3477d9800 2024-04-24T09:14:57.505 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.503+0000 7fedb2800640 1 --2- 172.21.15.83:0/569268150 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55b3477d9800 0x55b3477dbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:14:57.505 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.504+0000 7fed93e00640 10 client.23457 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:14:57.505 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.504+0000 7fedb2800640 1 --2- 172.21.15.83:0/569268150 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55b3477d9800 0x55b3477dbc20 crc :-1 s=READY pgs=572 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e67e82515d44dcfa server_cookie=2dbc1022749f03af in_seq=0 out_seq=0 2024-04-24T09:14:57.667 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.667+0000 7fed93e00640 1 -- 172.21.15.83:0/569268150 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b3476304b0 con 0x55b3477d9800 2024-04-24T09:14:57.667 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.667+0000 7fed93e00640 10 client.23457 handle_command_reply: tid=0 2024-04-24T09:14:57.667 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.667+0000 7fedbc27e740 1 client.23457 shutdown 2024-04-24T09:14:57.667 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.667+0000 7fedbc27e740 1 -- 172.21.15.83:0/569268150 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b34774f1a0 msgr2=0x55b347744820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:57.667 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.667+0000 7fedbc27e740 1 --2- 172.21.15.83:0/569268150 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b34774f1a0 0x55b347744820 secure :-1 s=READY pgs=2687 cs=0 l=1 rev1=1 crypto rx=0x7fedb406b790 tx=0x7feda0005610 comp rx=0 tx=0).stop 2024-04-24T09:14:57.668 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.668+0000 7fedbc27e740 1 -- 172.21.15.83:0/569268150 shutdown_connections 2024-04-24T09:14:57.668 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.668+0000 7fedbc27e740 1 --2- 172.21.15.83:0/569268150 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55b3477d9800 0x55b3477dbc20 crc :-1 s=READY pgs=572 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:57.668 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.668+0000 7fedbc27e740 1 --2- 172.21.15.83:0/569268150 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b347762610 0x55b347744d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:57.668 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.668+0000 7fedbc27e740 1 --2- 172.21.15.83:0/569268150 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b34774f1a0 0x55b347744820 unknown :-1 s=CLOSED pgs=2687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:57.668 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.668+0000 7fedbc27e740 1 --2- 172.21.15.83:0/569268150 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b34774e880 0x55b3477442e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:57.668 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.668+0000 7fedbc27e740 1 -- 172.21.15.83:0/569268150 >> 172.21.15.83:0/569268150 conn(0x55b34774aed0 msgr2=0x55b347747550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:14:57.668 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.668+0000 7fed93e00640 0 client.23457 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:14:57.668 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.668+0000 7fedbc27e740 1 -- 172.21.15.83:0/569268150 shutdown_connections 2024-04-24T09:14:57.668 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.668+0000 7fedbc27e740 1 -- 172.21.15.83:0/569268150 wait complete. 2024-04-24T09:14:57.668 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.669+0000 7fedbc27e740 20 client.23457 trim_cache size 0 max 16384 2024-04-24T09:14:57.669 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T09:14:57.669 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T09:14:57.669 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T09:14:57.669 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T09:14:57.670 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.669+0000 7fedb9400640 1 -- 172.21.15.83:0/3685433739 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fed980a3f20 msgr2=0x7fed980a63e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:57.670 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.669+0000 7fedb9400640 1 --2- 172.21.15.83:0/3685433739 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fed980a3f20 0x7fed980a63e0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fedac009370 tx=0x7fedac014040 comp rx=0 tx=0).stop 2024-04-24T09:14:57.670 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.669+0000 7fedb9400640 1 -- 172.21.15.83:0/3685433739 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fedb4104460 msgr2=0x7fedb41a0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:14:57.670 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.669+0000 7fedb9400640 1 --2- 172.21.15.83:0/3685433739 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fedb4104460 0x7fedb41a0630 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7feda400e9f0 tx=0x7feda400eeb0 comp rx=0 tx=0).stop 2024-04-24T09:14:57.670 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.670+0000 7fedb9400640 1 -- 172.21.15.83:0/3685433739 shutdown_connections 2024-04-24T09:14:57.670 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.670+0000 7fedb9400640 1 --2- 172.21.15.83:0/3685433739 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fed980a3f20 0x7fed980a63e0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:57.670 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.670+0000 7fedb9400640 1 --2- 172.21.15.83:0/3685433739 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fedb419a7a0 0x7fedb419ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:57.670 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.670+0000 7fedb9400640 1 --2- 172.21.15.83:0/3685433739 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fedb4109530 0x7fedb41a0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:57.670 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.670+0000 7fedb9400640 1 --2- 172.21.15.83:0/3685433739 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fedb4104460 0x7fedb41a0630 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:14:57.670 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.670+0000 7fedb9400640 1 -- 172.21.15.83:0/3685433739 >> 172.21.15.83:0/3685433739 conn(0x7fedb40ffd00 msgr2=0x7fedb4102020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:14:57.671 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.670+0000 7fedb9400640 1 -- 172.21.15.83:0/3685433739 shutdown_connections 2024-04-24T09:14:57.671 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:14:57.670+0000 7fedb9400640 1 -- 172.21.15.83:0/3685433739 wait complete. 2024-04-24T09:14:58.209 INFO:tasks.workunit.client.0.smithi083.stdout: 10 70216 6.56 MB/sec execute 449 sec latency 149.377 ms 2024-04-24T09:14:58.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:14:58 smithi083 ceph-mon[27907]: pgmap v737: 129 pgs: 129 active+clean; 1.0 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.8 MiB/s wr, 231 op/s 2024-04-24T09:14:58.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:14:58 smithi138 ceph-mon[31441]: pgmap v737: 129 pgs: 129 active+clean; 1.0 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.8 MiB/s wr, 231 op/s 2024-04-24T09:14:58.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:14:58 smithi189 ceph-mon[31791]: pgmap v737: 129 pgs: 129 active+clean; 1.0 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.8 MiB/s wr, 231 op/s 2024-04-24T09:14:59.209 INFO:tasks.workunit.client.0.smithi083.stdout: 10 70356 6.57 MB/sec execute 450 sec latency 75.699 ms 2024-04-24T09:15:00.209 INFO:tasks.workunit.client.0.smithi083.stdout: 10 70496 6.56 MB/sec execute 451 sec latency 73.030 ms 2024-04-24T09:15:00.319 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:15:00 smithi083 ceph-mon[27907]: pgmap v738: 129 pgs: 129 active+clean; 1.0 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.4 MiB/s wr, 388 op/s 2024-04-24T09:15:00.319 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:15:00 smithi083 ceph-mon[27907]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:15:00.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:15:00 smithi138 ceph-mon[31441]: pgmap v738: 129 pgs: 129 active+clean; 1.0 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.4 MiB/s wr, 388 op/s 2024-04-24T09:15:00.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:15:00 smithi138 ceph-mon[31441]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:15:00.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:15:00 smithi189 ceph-mon[31791]: pgmap v738: 129 pgs: 129 active+clean; 1.0 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.4 MiB/s wr, 388 op/s 2024-04-24T09:15:00.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:15:00 smithi189 ceph-mon[31791]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:15:01.209 INFO:tasks.workunit.client.0.smithi083.stdout: 10 70573 6.55 MB/sec execute 452 sec latency 491.048 ms 2024-04-24T09:15:02.116 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T09:15:02.117 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T09:15:02.209 INFO:tasks.workunit.client.0.smithi083.stdout: 10 70573 6.54 MB/sec execute 453 sec latency 1491.152 ms 2024-04-24T09:15:02.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:15:02 smithi138 ceph-mon[31441]: pgmap v739: 129 pgs: 129 active+clean; 1.0 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.2 MiB/s wr, 257 op/s 2024-04-24T09:15:02.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:15:02 smithi189 ceph-mon[31791]: pgmap v739: 129 pgs: 129 active+clean; 1.0 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.2 MiB/s wr, 257 op/s 2024-04-24T09:15:02.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:15:02 smithi083 ceph-mon[27907]: pgmap v739: 129 pgs: 129 active+clean; 1.0 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.2 MiB/s wr, 257 op/s 2024-04-24T09:15:03.209 INFO:tasks.workunit.client.0.smithi083.stdout: 10 70573 6.52 MB/sec execute 454 sec latency 2491.295 ms 2024-04-24T09:15:03.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.272+0000 7f9d5a600640 1 -- 172.21.15.83:0/4124569189 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d54108fd0 msgr2=0x7f9d54115490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:03.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.272+0000 7f9d5a600640 1 --2- 172.21.15.83:0/4124569189 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d54108fd0 0x7f9d54115490 secure :-1 s=READY pgs=2688 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c009930 tx=0x7f9d4c02a420 comp rx=0 tx=0).stop 2024-04-24T09:15:03.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.272+0000 7f9d5a600640 1 -- 172.21.15.83:0/4124569189 shutdown_connections 2024-04-24T09:15:03.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.272+0000 7f9d5a600640 1 --2- 172.21.15.83:0/4124569189 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d54108fd0 0x7f9d54115490 unknown :-1 s=CLOSED pgs=2688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:03.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.272+0000 7f9d5a600640 1 --2- 172.21.15.83:0/4124569189 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d54108610 0x7f9d54108a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:03.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.272+0000 7f9d5a600640 1 --2- 172.21.15.83:0/4124569189 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9d5410e660 0x7f9d5410ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:03.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.272+0000 7f9d5a600640 1 -- 172.21.15.83:0/4124569189 >> 172.21.15.83:0/4124569189 conn(0x7f9d54076540 msgr2=0x7f9d54078960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:03.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.272+0000 7f9d5a600640 1 -- 172.21.15.83:0/4124569189 shutdown_connections 2024-04-24T09:15:03.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.272+0000 7f9d5a600640 1 -- 172.21.15.83:0/4124569189 wait complete. 2024-04-24T09:15:03.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.273+0000 7f9d5a600640 1 Processor -- start 2024-04-24T09:15:03.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.273+0000 7f9d5a600640 1 -- start start 2024-04-24T09:15:03.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.274+0000 7f9d5a600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d54108610 0x7f9d5419edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:03.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.274+0000 7f9d5a600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d54108fd0 0x7f9d5419f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:03.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.274+0000 7f9d5a600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9d5410e660 0x7f9d541a3620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:03.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.274+0000 7f9d5a600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9d54118850 con 0x7f9d54108fd0 2024-04-24T09:15:03.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.274+0000 7f9d5a600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9d541186d0 con 0x7f9d54108610 2024-04-24T09:15:03.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.274+0000 7f9d5a600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9d541189d0 con 0x7f9d5410e660 2024-04-24T09:15:03.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.274+0000 7f9d4aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d54108fd0 0x7f9d5419f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:03.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.274+0000 7f9d4aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d54108fd0 0x7f9d5419f2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54754/0 (socket says 172.21.15.83:54754) 2024-04-24T09:15:03.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.274+0000 7f9d4aa00640 1 -- 172.21.15.83:0/2030820346 learned_addr learned my addr 172.21.15.83:0/2030820346 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:15:03.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.274+0000 7f9d53400640 1 --2- 172.21.15.83:0/2030820346 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d54108610 0x7f9d5419edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:03.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.274+0000 7f9d53e00640 1 --2- 172.21.15.83:0/2030820346 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9d5410e660 0x7f9d541a3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:03.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.275+0000 7f9d4aa00640 1 -- 172.21.15.83:0/2030820346 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9d5410e660 msgr2=0x7f9d541a3620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:03.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.275+0000 7f9d4aa00640 1 --2- 172.21.15.83:0/2030820346 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9d5410e660 0x7f9d541a3620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:03.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.275+0000 7f9d4aa00640 1 -- 172.21.15.83:0/2030820346 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d54108610 msgr2=0x7f9d5419edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:03.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.275+0000 7f9d4aa00640 1 --2- 172.21.15.83:0/2030820346 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d54108610 0x7f9d5419edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:03.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.275+0000 7f9d4aa00640 1 -- 172.21.15.83:0/2030820346 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d541a3d50 con 0x7f9d54108fd0 2024-04-24T09:15:03.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.275+0000 7f9d53400640 1 --2- 172.21.15.83:0/2030820346 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d54108610 0x7f9d5419edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:15:03.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.275+0000 7f9d4aa00640 1 --2- 172.21.15.83:0/2030820346 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d54108fd0 0x7f9d5419f2f0 secure :-1 s=READY pgs=2689 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c00d950 tx=0x7f9d3c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:03.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.275+0000 7f9d50c00640 1 -- 172.21.15.83:0/2030820346 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d3c017070 con 0x7f9d54108fd0 2024-04-24T09:15:03.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.275+0000 7f9d50c00640 1 -- 172.21.15.83:0/2030820346 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9d3c009c30 con 0x7f9d54108fd0 2024-04-24T09:15:03.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.275+0000 7f9d5a600640 1 -- 172.21.15.83:0/2030820346 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d541a4040 con 0x7f9d54108fd0 2024-04-24T09:15:03.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.275+0000 7f9d50c00640 1 -- 172.21.15.83:0/2030820346 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d3c00f620 con 0x7f9d54108fd0 2024-04-24T09:15:03.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.275+0000 7f9d5a600640 1 -- 172.21.15.83:0/2030820346 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d541ab8d0 con 0x7f9d54108fd0 2024-04-24T09:15:03.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.279+0000 7f9d50c00640 1 -- 172.21.15.83:0/2030820346 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9d3c0109a0 con 0x7f9d54108fd0 2024-04-24T09:15:03.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.279+0000 7f9d5d37e740 1 Processor -- start 2024-04-24T09:15:03.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.279+0000 7f9d5d37e740 1 -- start start 2024-04-24T09:15:03.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.279+0000 7f9d5d37e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5629d38b7860 0x5629d38b7c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:03.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.279+0000 7f9d5d37e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5629d38b8180 0x5629d38c2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:03.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.280+0000 7f9d5d37e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5629d38c2850 0x5629d38c4c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:03.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.280+0000 7f9d5d37e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5629d381d1f0 con 0x5629d38c2850 2024-04-24T09:15:03.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.280+0000 7f9d5d37e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5629d37c3350 con 0x5629d38b7860 2024-04-24T09:15:03.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.280+0000 7f9d5d37e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5629d38191b0 con 0x5629d38b8180 2024-04-24T09:15:03.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.280+0000 7f9d4aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5629d38c2850 0x5629d38c4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:03.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.280+0000 7f9d4aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5629d38c2850 0x5629d38c4c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54764/0 (socket says 172.21.15.83:54764) 2024-04-24T09:15:03.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.280+0000 7f9d4aa00640 1 -- 172.21.15.83:0/3426132346 learned_addr learned my addr 172.21.15.83:0/3426132346 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:15:03.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.280+0000 7f9d53e00640 1 --2- 172.21.15.83:0/3426132346 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5629d38b7860 0x5629d38b7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:03.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.280+0000 7f9d53400640 1 --2- 172.21.15.83:0/3426132346 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5629d38b8180 0x5629d38c2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:03.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.280+0000 7f9d4aa00640 1 -- 172.21.15.83:0/3426132346 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5629d38b8180 msgr2=0x5629d38c2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:03.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.281+0000 7f9d4aa00640 1 --2- 172.21.15.83:0/3426132346 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5629d38b8180 0x5629d38c2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:03.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.281+0000 7f9d4aa00640 1 -- 172.21.15.83:0/3426132346 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5629d38b7860 msgr2=0x5629d38b7c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:03.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.281+0000 7f9d4aa00640 1 --2- 172.21.15.83:0/3426132346 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5629d38b7860 0x5629d38b7c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:03.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.281+0000 7f9d4aa00640 1 -- 172.21.15.83:0/3426132346 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629d3817f50 con 0x5629d38c2850 2024-04-24T09:15:03.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.281+0000 7f9d53e00640 1 --2- 172.21.15.83:0/3426132346 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5629d38b7860 0x5629d38b7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:15:03.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.281+0000 7f9d53400640 1 --2- 172.21.15.83:0/3426132346 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5629d38b8180 0x5629d38c2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:15:03.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.281+0000 7f9d50c00640 1 --2- 172.21.15.83:0/2030820346 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f9d380a3d20 0x7f9d380a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:03.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.281+0000 7f9d50c00640 1 -- 172.21.15.83:0/2030820346 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9583+0+0 (secure 0 0 0) 0x7f9d3c0d0930 con 0x7f9d54108fd0 2024-04-24T09:15:03.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.281+0000 7f9d4aa00640 1 --2- 172.21.15.83:0/3426132346 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5629d38c2850 0x5629d38c4c40 secure :-1 s=READY pgs=2690 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c0cdb40 tx=0x7f9d3c0d8670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d3536200c1661a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:03.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.281+0000 7f9d53e00640 1 --2- 172.21.15.83:0/2030820346 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f9d380a3d20 0x7f9d380a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:03.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.281+0000 7f9d48c00640 1 -- 172.21.15.83:0/3426132346 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d3c020020 con 0x5629d38c2850 2024-04-24T09:15:03.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.281+0000 7f9d48c00640 1 -- 172.21.15.83:0/3426132346 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9d3c09cb50 con 0x5629d38c2850 2024-04-24T09:15:03.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.281+0000 7f9d48c00640 1 -- 172.21.15.83:0/3426132346 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d3c09bf90 con 0x5629d38c2850 2024-04-24T09:15:03.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.282+0000 7f9d5d37e740 1 -- 172.21.15.83:0/3426132346 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5629d38c2850 msgr2=0x5629d38c4c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:03.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.282+0000 7f9d53e00640 1 --2- 172.21.15.83:0/2030820346 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f9d380a3d20 0x7f9d380a61e0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c031440 tx=0x7f9d4c0058d0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:03.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.282+0000 7f9d5d37e740 1 --2- 172.21.15.83:0/3426132346 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5629d38c2850 0x5629d38c4c40 secure :-1 s=READY pgs=2690 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c0cdb40 tx=0x7f9d3c0d8670 comp rx=0 tx=0).stop 2024-04-24T09:15:03.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.282+0000 7f9d5d37e740 1 -- 172.21.15.83:0/3426132346 shutdown_connections 2024-04-24T09:15:03.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.282+0000 7f9d5d37e740 1 --2- 172.21.15.83:0/3426132346 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5629d38c2850 0x5629d38c4c40 unknown :-1 s=CLOSED pgs=2690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:03.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.282+0000 7f9d5d37e740 1 --2- 172.21.15.83:0/3426132346 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5629d38b8180 0x5629d38c2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:03.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.282+0000 7f9d5d37e740 1 --2- 172.21.15.83:0/3426132346 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5629d38b7860 0x5629d38b7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:03.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.282+0000 7f9d5d37e740 1 -- 172.21.15.83:0/3426132346 >> 172.21.15.83:0/3426132346 conn(0x5629d38b3b80 msgr2=0x5629d38b3f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:03.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.283+0000 7f9d5d37e740 1 -- 172.21.15.83:0/3426132346 shutdown_connections 2024-04-24T09:15:03.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.283+0000 7f9d5d37e740 1 -- 172.21.15.83:0/3426132346 wait complete. 2024-04-24T09:15:03.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.283+0000 7f9d5d37e740 1 -- start start 2024-04-24T09:15:03.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.283+0000 7f9d5d37e740 1 Processor -- start 2024-04-24T09:15:03.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.284+0000 7f9d5d37e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5629d38b7860 0x5629d38bf750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:03.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.284+0000 7f9d5d37e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5629d38b8180 0x5629d38bfc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:03.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.284+0000 7f9d5d37e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5629d38c2850 0x5629d38c01d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:03.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.284+0000 7f9d5d37e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5629d38c55e0 con 0x5629d38b8180 2024-04-24T09:15:03.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.284+0000 7f9d5d37e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5629d37d6690 con 0x5629d38b7860 2024-04-24T09:15:03.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.284+0000 7f9d5d37e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5629d38c5760 con 0x5629d38c2850 2024-04-24T09:15:03.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.284+0000 7f9d5d37e740 10 client.0 _ll_register_callbacks cb 0x5629d3812650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:15:03.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.284+0000 7f9d4aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5629d38b8180 0x5629d38bfc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:03.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.284+0000 7f9d4aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5629d38b8180 0x5629d38bfc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54772/0 (socket says 172.21.15.83:54772) 2024-04-24T09:15:03.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.284+0000 7f9d4aa00640 1 -- 172.21.15.83:0/1978605165 learned_addr learned my addr 172.21.15.83:0/1978605165 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:15:03.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.284+0000 7f9d53400640 1 --2- 172.21.15.83:0/1978605165 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5629d38b7860 0x5629d38bf750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:03.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.284+0000 7f9d53e00640 1 --2- 172.21.15.83:0/1978605165 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5629d38c2850 0x5629d38c01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:03.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.284+0000 7f9d4aa00640 1 -- 172.21.15.83:0/1978605165 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5629d38c2850 msgr2=0x5629d38c01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:03.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.284+0000 7f9d4aa00640 1 --2- 172.21.15.83:0/1978605165 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5629d38c2850 0x5629d38c01d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:03.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.284+0000 7f9d4aa00640 1 -- 172.21.15.83:0/1978605165 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5629d38b7860 msgr2=0x5629d38bf750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:03.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.284+0000 7f9d4aa00640 1 --2- 172.21.15.83:0/1978605165 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5629d38b7860 0x5629d38bf750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:03.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.285+0000 7f9d4aa00640 1 -- 172.21.15.83:0/1978605165 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629d37d5920 con 0x5629d38b8180 2024-04-24T09:15:03.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.285+0000 7f9d53e00640 1 --2- 172.21.15.83:0/1978605165 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5629d38c2850 0x5629d38c01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:15:03.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.285+0000 7f9d4aa00640 1 --2- 172.21.15.83:0/1978605165 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5629d38b8180 0x5629d38bfc90 secure :-1 s=READY pgs=2691 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c0058c0 tx=0x7f9d3c0d6960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=676cba6a88d4149e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:03.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.285+0000 7f9d41600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:15:03.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.285+0000 7f9d41600640 1 -- 172.21.15.83:0/1978605165 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d3c020070 con 0x5629d38b8180 2024-04-24T09:15:03.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.285+0000 7f9d41600640 1 -- 172.21.15.83:0/1978605165 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(128..128 src has 1..128) ==== 9583+0+0 (secure 0 0 0) 0x7f9d3c0d6cb0 con 0x5629d38b8180 2024-04-24T09:15:03.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.285+0000 7f9d5d37e740 1 -- 172.21.15.83:0/1978605165 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d3c09cb50 con 0x5629d38b8180 2024-04-24T09:15:03.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.285+0000 7f9d41600640 1 -- 172.21.15.83:0/1978605165 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7f9d3c09bdb0 con 0x5629d38b8180 2024-04-24T09:15:03.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.285+0000 7f9d5d37e740 10 client.23469 fetch_fsmap learned FSMap version 68 2024-04-24T09:15:03.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.285+0000 7f9d5d37e740 1 -- 172.21.15.83:0/1978605165 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) -- 0x5629d35f97d0 con 0x5629d38b8180 2024-04-24T09:15:03.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.286+0000 7f9d41600640 1 -- 172.21.15.83:0/1978605165 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9d3c09cb50 con 0x5629d38b8180 2024-04-24T09:15:03.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.286+0000 7f9d41600640 1 -- 172.21.15.83:0/1978605165 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 68) ==== 8374+0+0 (secure 0 0 0) 0x7f9d3c0cde80 con 0x5629d38b8180 2024-04-24T09:15:03.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.286+0000 7f9d5d37e740 10 client.23469 fetch_fsmap finished waiting for FSMap version 68 2024-04-24T09:15:03.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.286+0000 7f9d5d37e740 10 client.23469 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:15:03.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.286+0000 7f9d5d37e740 20 client.23469 populate_metadata read hostname 'smithi083' 2024-04-24T09:15:03.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.286+0000 7f9d5d37e740 1 --2- 172.21.15.83:0/1978605165 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5629d38d2990 0x5629d38d4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:03.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.286+0000 7f9d5d37e740 4 client.23469 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:15:03.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.286+0000 7f9d5d37e740 1 -- 172.21.15.83:0/1978605165 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5629d382ac10 con 0x5629d38d2990 2024-04-24T09:15:03.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.287+0000 7f9d53400640 1 --2- 172.21.15.83:0/1978605165 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5629d38d2990 0x5629d38d4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:03.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.288+0000 7f9d41600640 10 client.23469 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:15:03.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.288+0000 7f9d53400640 1 --2- 172.21.15.83:0/1978605165 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5629d38d2990 0x5629d38d4d80 crc :-1 s=READY pgs=573 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=321b9136974793f3 server_cookie=9b30b4cec89c9b91 in_seq=0 out_seq=0 2024-04-24T09:15:03.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.289+0000 7f9d41600640 1 -- 172.21.15.83:0/1978605165 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5629d382ac10 con 0x5629d38d2990 2024-04-24T09:15:03.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.289+0000 7f9d41600640 10 client.23469 handle_command_reply: tid=0 2024-04-24T09:15:03.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.290+0000 7f9d5d37e740 1 client.23469 shutdown 2024-04-24T09:15:03.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.290+0000 7f9d5d37e740 1 -- 172.21.15.83:0/1978605165 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5629d38b8180 msgr2=0x5629d38bfc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:03.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.290+0000 7f9d5d37e740 1 --2- 172.21.15.83:0/1978605165 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5629d38b8180 0x5629d38bfc90 secure :-1 s=READY pgs=2691 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c0058c0 tx=0x7f9d3c0d6960 comp rx=0 tx=0).stop 2024-04-24T09:15:03.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.290+0000 7f9d5d37e740 1 -- 172.21.15.83:0/1978605165 shutdown_connections 2024-04-24T09:15:03.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.290+0000 7f9d5d37e740 1 --2- 172.21.15.83:0/1978605165 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5629d38d2990 0x5629d38d4d80 crc :-1 s=READY pgs=573 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:03.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.290+0000 7f9d5d37e740 1 --2- 172.21.15.83:0/1978605165 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5629d38c2850 0x5629d38c01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:03.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.290+0000 7f9d5d37e740 1 --2- 172.21.15.83:0/1978605165 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5629d38b8180 0x5629d38bfc90 unknown :-1 s=CLOSED pgs=2691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:03.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.290+0000 7f9d41600640 0 client.23469 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:15:03.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.290+0000 7f9d5d37e740 1 --2- 172.21.15.83:0/1978605165 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5629d38b7860 0x5629d38bf750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:03.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.290+0000 7f9d5d37e740 1 -- 172.21.15.83:0/1978605165 >> 172.21.15.83:0/1978605165 conn(0x5629d38b3b80 msgr2=0x5629d38b9980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:03.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.290+0000 7f9d5d37e740 1 -- 172.21.15.83:0/1978605165 shutdown_connections 2024-04-24T09:15:03.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.290+0000 7f9d5d37e740 1 -- 172.21.15.83:0/1978605165 wait complete. 2024-04-24T09:15:03.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.290+0000 7f9d5d37e740 20 client.23469 trim_cache size 0 max 16384 2024-04-24T09:15:03.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.310+0000 7f9d5d37e740 1 Processor -- start 2024-04-24T09:15:03.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.310+0000 7f9d5d37e740 1 -- start start 2024-04-24T09:15:03.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.310+0000 7f9d5d37e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5629d38b7860 0x5629d38be690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:03.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.310+0000 7f9d5d37e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5629d38b8180 0x5629d38b9fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:03.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.310+0000 7f9d5d37e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5629d38c2850 0x5629d38d7da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:03.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.310+0000 7f9d5d37e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9d541b5c70 con 0x5629d38c2850 2024-04-24T09:15:03.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.310+0000 7f9d5d37e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9d541b7840 con 0x5629d38b8180 2024-04-24T09:15:03.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.310+0000 7f9d5d37e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9d541b6340 con 0x5629d38b7860 2024-04-24T09:15:03.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.311+0000 7f9d53e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5629d38c2850 0x5629d38d7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:03.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.311+0000 7f9d53e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5629d38c2850 0x5629d38d7da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54776/0 (socket says 172.21.15.83:54776) 2024-04-24T09:15:03.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.311+0000 7f9d53e00640 1 -- 172.21.15.83:0/1423674178 learned_addr learned my addr 172.21.15.83:0/1423674178 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:15:03.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.311+0000 7f9d53400640 1 --2- 172.21.15.83:0/1423674178 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5629d38b7860 0x5629d38be690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:03.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.311+0000 7f9d4aa00640 1 --2- 172.21.15.83:0/1423674178 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5629d38b8180 0x5629d38b9fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:03.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.311+0000 7f9d53e00640 1 -- 172.21.15.83:0/1423674178 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5629d38b7860 msgr2=0x5629d38be690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:03.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.311+0000 7f9d53e00640 1 --2- 172.21.15.83:0/1423674178 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5629d38b7860 0x5629d38be690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:03.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.311+0000 7f9d53e00640 1 -- 172.21.15.83:0/1423674178 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5629d38b8180 msgr2=0x5629d38b9fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:03.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.311+0000 7f9d53e00640 1 --2- 172.21.15.83:0/1423674178 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5629d38b8180 0x5629d38b9fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:03.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.311+0000 7f9d53e00640 1 -- 172.21.15.83:0/1423674178 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629d352f3f0 con 0x5629d38c2850 2024-04-24T09:15:03.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.312+0000 7f9d53e00640 1 --2- 172.21.15.83:0/1423674178 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5629d38c2850 0x5629d38d7da0 secure :-1 s=READY pgs=2692 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c009930 tx=0x7f9d4c03d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a10aca5292a965ba server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:03.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.312+0000 7f9d43e00640 1 -- 172.21.15.83:0/1423674178 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d4c04c070 con 0x5629d38c2850 2024-04-24T09:15:03.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.312+0000 7f9d43e00640 1 -- 172.21.15.83:0/1423674178 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9d4c0054f0 con 0x5629d38c2850 2024-04-24T09:15:03.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.312+0000 7f9d43e00640 1 -- 172.21.15.83:0/1423674178 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d4c035050 con 0x5629d38c2850 2024-04-24T09:15:03.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.313+0000 7f9d5d37e740 1 -- 172.21.15.83:0/1423674178 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5629d38c2850 msgr2=0x5629d38d7da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:03.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.313+0000 7f9d5d37e740 1 --2- 172.21.15.83:0/1423674178 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5629d38c2850 0x5629d38d7da0 secure :-1 s=READY pgs=2692 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c009930 tx=0x7f9d4c03d000 comp rx=0 tx=0).stop 2024-04-24T09:15:03.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.313+0000 7f9d5d37e740 1 -- 172.21.15.83:0/1423674178 shutdown_connections 2024-04-24T09:15:03.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.313+0000 7f9d5d37e740 1 --2- 172.21.15.83:0/1423674178 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5629d38c2850 0x5629d38d7da0 unknown :-1 s=CLOSED pgs=2692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:03.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.313+0000 7f9d5d37e740 1 --2- 172.21.15.83:0/1423674178 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5629d38b8180 0x5629d38b9fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:03.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.313+0000 7f9d5d37e740 1 --2- 172.21.15.83:0/1423674178 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5629d38b7860 0x5629d38be690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:03.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.313+0000 7f9d5d37e740 1 -- 172.21.15.83:0/1423674178 >> 172.21.15.83:0/1423674178 conn(0x5629d38b3b80 msgr2=0x5629d38b2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:03.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.313+0000 7f9d5d37e740 1 -- 172.21.15.83:0/1423674178 shutdown_connections 2024-04-24T09:15:03.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.313+0000 7f9d5d37e740 1 -- 172.21.15.83:0/1423674178 wait complete. 2024-04-24T09:15:03.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.313+0000 7f9d5d37e740 1 -- start start 2024-04-24T09:15:03.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.313+0000 7f9d5d37e740 1 Processor -- start 2024-04-24T09:15:03.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.314+0000 7f9d5d37e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5629d38b7860 0x5629d38d8280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:03.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.314+0000 7f9d5d37e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5629d38b8180 0x5629d38d5660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:03.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.314+0000 7f9d5d37e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5629d38c2850 0x5629d38d5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:03.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.314+0000 7f9d5d37e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9d541b7140 con 0x5629d38b7860 2024-04-24T09:15:03.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.314+0000 7f9d5d37e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9d541b6a70 con 0x5629d38b8180 2024-04-24T09:15:03.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.314+0000 7f9d5d37e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9d541b3c70 con 0x5629d38c2850 2024-04-24T09:15:03.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.314+0000 7f9d5d37e740 10 client.0 _ll_register_callbacks cb 0x7f9d3c0d6a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:15:03.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.314+0000 7f9d53400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5629d38b7860 0x5629d38d8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:03.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.314+0000 7f9d53400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5629d38b7860 0x5629d38d8280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54788/0 (socket says 172.21.15.83:54788) 2024-04-24T09:15:03.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.314+0000 7f9d53400640 1 -- 172.21.15.83:0/3606058477 learned_addr learned my addr 172.21.15.83:0/3606058477 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:15:03.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.314+0000 7f9d4aa00640 1 --2- 172.21.15.83:0/3606058477 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5629d38b8180 0x5629d38d5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:03.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.314+0000 7f9d53e00640 1 --2- 172.21.15.83:0/3606058477 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5629d38c2850 0x5629d38d5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:03.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.315+0000 7f9d53400640 1 -- 172.21.15.83:0/3606058477 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5629d38c2850 msgr2=0x5629d38d5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:03.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.315+0000 7f9d53400640 1 --2- 172.21.15.83:0/3606058477 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5629d38c2850 0x5629d38d5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:03.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.315+0000 7f9d53400640 1 -- 172.21.15.83:0/3606058477 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5629d38b8180 msgr2=0x5629d38d5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:03.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.315+0000 7f9d53400640 1 --2- 172.21.15.83:0/3606058477 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5629d38b8180 0x5629d38d5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:03.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.315+0000 7f9d53400640 1 -- 172.21.15.83:0/3606058477 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629d34ef0f0 con 0x5629d38b7860 2024-04-24T09:15:03.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.315+0000 7f9d4aa00640 1 --2- 172.21.15.83:0/3606058477 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5629d38b8180 0x5629d38d5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:15:03.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.315+0000 7f9d53e00640 1 --2- 172.21.15.83:0/3606058477 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5629d38c2850 0x5629d38d5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:15:03.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.315+0000 7f9d53400640 1 --2- 172.21.15.83:0/3606058477 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5629d38b7860 0x5629d38d8280 secure :-1 s=READY pgs=2693 cs=0 l=1 rev1=1 crypto rx=0x7f9d44009f70 tx=0x7f9d44012cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b986a69588519bb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:03.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.315+0000 7f9d41600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:15:03.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.315+0000 7f9d41600640 1 -- 172.21.15.83:0/3606058477 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d44010b00 con 0x5629d38b7860 2024-04-24T09:15:03.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.315+0000 7f9d41600640 1 -- 172.21.15.83:0/3606058477 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(128..128 src has 1..128) ==== 9583+0+0 (secure 0 0 0) 0x7f9d44010ca0 con 0x5629d38b7860 2024-04-24T09:15:03.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.315+0000 7f9d5d37e740 1 -- 172.21.15.83:0/3606058477 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d4c0054f0 con 0x5629d38b7860 2024-04-24T09:15:03.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.315+0000 7f9d41600640 1 -- 172.21.15.83:0/3606058477 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7f9d4401f390 con 0x5629d38b7860 2024-04-24T09:15:03.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.315+0000 7f9d5d37e740 10 client.23475 fetch_fsmap learned FSMap version 68 2024-04-24T09:15:03.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.315+0000 7f9d5d37e740 1 -- 172.21.15.83:0/3606058477 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) -- 0x5629d37e2810 con 0x5629d38b7860 2024-04-24T09:15:03.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.316+0000 7f9d41600640 1 -- 172.21.15.83:0/3606058477 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9d4c0054f0 con 0x5629d38b7860 2024-04-24T09:15:03.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.316+0000 7f9d41600640 1 -- 172.21.15.83:0/3606058477 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 68) ==== 8374+0+0 (secure 0 0 0) 0x7f9d4400f040 con 0x5629d38b7860 2024-04-24T09:15:03.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.316+0000 7f9d5d37e740 10 client.23475 fetch_fsmap finished waiting for FSMap version 68 2024-04-24T09:15:03.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.316+0000 7f9d5d37e740 10 client.23475 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:15:03.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.316+0000 7f9d5d37e740 20 client.23475 populate_metadata read hostname 'smithi083' 2024-04-24T09:15:03.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.316+0000 7f9d5d37e740 1 --2- 172.21.15.83:0/3606058477 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5629d39390f0 0x5629d393b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:03.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.316+0000 7f9d5d37e740 4 client.23475 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T09:15:03.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.316+0000 7f9d5d37e740 1 -- 172.21.15.83:0/3606058477 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5629d3824a80 con 0x5629d39390f0 2024-04-24T09:15:03.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.317+0000 7f9d4aa00640 1 --2- 172.21.15.83:0/3606058477 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5629d39390f0 0x5629d393b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:03.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.318+0000 7f9d4aa00640 1 --2- 172.21.15.83:0/3606058477 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5629d39390f0 0x5629d393b510 crc :-1 s=READY pgs=574 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1871854928057e79 server_cookie=2d836d3ee8e58332 in_seq=0 out_seq=0 2024-04-24T09:15:03.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.318+0000 7f9d41600640 10 client.23475 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:15:03.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.318+0000 7f9d41600640 1 -- 172.21.15.83:0/3606058477 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5629d3824a80 con 0x5629d39390f0 2024-04-24T09:15:03.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.318+0000 7f9d41600640 10 client.23475 handle_command_reply: tid=0 2024-04-24T09:15:03.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.318+0000 7f9d5d37e740 1 client.23475 shutdown 2024-04-24T09:15:03.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.319+0000 7f9d5d37e740 1 -- 172.21.15.83:0/3606058477 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5629d38b7860 msgr2=0x5629d38d8280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:03.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.319+0000 7f9d5d37e740 1 --2- 172.21.15.83:0/3606058477 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5629d38b7860 0x5629d38d8280 secure :-1 s=READY pgs=2693 cs=0 l=1 rev1=1 crypto rx=0x7f9d44009f70 tx=0x7f9d44012cb0 comp rx=0 tx=0).stop 2024-04-24T09:15:03.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.319+0000 7f9d5d37e740 1 -- 172.21.15.83:0/3606058477 shutdown_connections 2024-04-24T09:15:03.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.319+0000 7f9d5d37e740 1 --2- 172.21.15.83:0/3606058477 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5629d39390f0 0x5629d393b510 crc :-1 s=READY pgs=574 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:03.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.319+0000 7f9d5d37e740 1 --2- 172.21.15.83:0/3606058477 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5629d38c2850 0x5629d38d5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:03.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.319+0000 7f9d5d37e740 1 --2- 172.21.15.83:0/3606058477 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5629d38b8180 0x5629d38d5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:03.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.319+0000 7f9d5d37e740 1 --2- 172.21.15.83:0/3606058477 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5629d38b7860 0x5629d38d8280 unknown :-1 s=CLOSED pgs=2693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:03.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.319+0000 7f9d41600640 0 client.23475 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:15:03.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.319+0000 7f9d5d37e740 1 -- 172.21.15.83:0/3606058477 >> 172.21.15.83:0/3606058477 conn(0x5629d38b3b80 msgr2=0x5629d38bf650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:03.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.319+0000 7f9d5d37e740 1 -- 172.21.15.83:0/3606058477 shutdown_connections 2024-04-24T09:15:03.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.319+0000 7f9d5d37e740 1 -- 172.21.15.83:0/3606058477 wait complete. 2024-04-24T09:15:03.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.319+0000 7f9d5d37e740 20 client.23475 trim_cache size 0 max 16384 2024-04-24T09:15:03.322 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T09:15:03.322 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-04-24T09:15:03.322 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-04-24T09:15:03.322 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T09:15:03.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.320+0000 7f9d5a600640 1 -- 172.21.15.83:0/2030820346 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f9d380a3d20 msgr2=0x7f9d380a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:03.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.320+0000 7f9d5a600640 1 --2- 172.21.15.83:0/2030820346 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f9d380a3d20 0x7f9d380a61e0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c031440 tx=0x7f9d4c0058d0 comp rx=0 tx=0).stop 2024-04-24T09:15:03.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.320+0000 7f9d5a600640 1 -- 172.21.15.83:0/2030820346 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d54108fd0 msgr2=0x7f9d5419f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:03.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.320+0000 7f9d5a600640 1 --2- 172.21.15.83:0/2030820346 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d54108fd0 0x7f9d5419f2f0 secure :-1 s=READY pgs=2689 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c00d950 tx=0x7f9d3c00de10 comp rx=0 tx=0).stop 2024-04-24T09:15:03.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.320+0000 7f9d5a600640 1 -- 172.21.15.83:0/2030820346 shutdown_connections 2024-04-24T09:15:03.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.320+0000 7f9d5a600640 1 --2- 172.21.15.83:0/2030820346 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f9d380a3d20 0x7f9d380a61e0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:03.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.320+0000 7f9d5a600640 1 --2- 172.21.15.83:0/2030820346 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9d5410e660 0x7f9d541a3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:03.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.320+0000 7f9d5a600640 1 --2- 172.21.15.83:0/2030820346 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d54108fd0 0x7f9d5419f2f0 unknown :-1 s=CLOSED pgs=2689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:03.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.320+0000 7f9d5a600640 1 --2- 172.21.15.83:0/2030820346 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d54108610 0x7f9d5419edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:03.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.320+0000 7f9d5a600640 1 -- 172.21.15.83:0/2030820346 >> 172.21.15.83:0/2030820346 conn(0x7f9d54076540 msgr2=0x7f9d54077d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:03.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.320+0000 7f9d5a600640 1 -- 172.21.15.83:0/2030820346 shutdown_connections 2024-04-24T09:15:03.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.320+0000 7f9d5a600640 1 -- 172.21.15.83:0/2030820346 wait complete. 2024-04-24T09:15:03.337 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T09:15:03.337 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T09:15:03.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.557+0000 7f1a2ec00640 1 -- 172.21.15.83:0/1571734921 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1a2810ef60 msgr2=0x7f1a2811d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:03.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.557+0000 7f1a2ec00640 1 --2- 172.21.15.83:0/1571734921 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1a2810ef60 0x7f1a2811d850 secure :-1 s=READY pgs=2694 cs=0 l=1 rev1=1 crypto rx=0x7f1a20009ca0 tx=0x7f1a2002a8c0 comp rx=0 tx=0).stop 2024-04-24T09:15:03.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.558+0000 7f1a2ec00640 1 -- 172.21.15.83:0/1571734921 shutdown_connections 2024-04-24T09:15:03.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.558+0000 7f1a2ec00640 1 --2- 172.21.15.83:0/1571734921 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1a2810ef60 0x7f1a2811d850 unknown :-1 s=CLOSED pgs=2694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:03.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.558+0000 7f1a2ec00640 1 --2- 172.21.15.83:0/1571734921 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a28110bd0 0x7f1a2810e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:03.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.558+0000 7f1a2ec00640 1 --2- 172.21.15.83:0/1571734921 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1a28110760 0x7f1a2810e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:03.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.558+0000 7f1a2ec00640 1 -- 172.21.15.83:0/1571734921 >> 172.21.15.83:0/1571734921 conn(0x7f1a2806d120 msgr2=0x7f1a2806d530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:03.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.559+0000 7f1a2ec00640 1 -- 172.21.15.83:0/1571734921 shutdown_connections 2024-04-24T09:15:03.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.559+0000 7f1a2ec00640 1 -- 172.21.15.83:0/1571734921 wait complete. 2024-04-24T09:15:03.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.561+0000 7f1a2ec00640 1 Processor -- start 2024-04-24T09:15:03.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.561+0000 7f1a2ec00640 1 -- start start 2024-04-24T09:15:03.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.562+0000 7f1a2ec00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a28110760 0x7f1a281a7640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:03.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.562+0000 7f1a2ec00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1a281a7b80 0x7f1a281abf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:03.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.562+0000 7f1a2ec00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1a281ac440 0x7f1a281ac8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:03.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.562+0000 7f1a2ec00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1a28121100 con 0x7f1a281a7b80 2024-04-24T09:15:03.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.562+0000 7f1a2ec00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1a28120f80 con 0x7f1a28110760 2024-04-24T09:15:03.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.562+0000 7f1a2ec00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1a28121280 con 0x7f1a281ac440 2024-04-24T09:15:03.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.562+0000 7f1a27e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a28110760 0x7f1a281a7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:03.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.562+0000 7f1a27e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a28110760 0x7f1a281a7640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:37112/0 (socket says 172.21.15.83:37112) 2024-04-24T09:15:03.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.562+0000 7f1a27e00640 1 -- 172.21.15.83:0/1212299431 learned_addr learned my addr 172.21.15.83:0/1212299431 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:15:03.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.562+0000 7f1a27400640 1 --2- 172.21.15.83:0/1212299431 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1a281a7b80 0x7f1a281abf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:03.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.563+0000 7f1a2cc00640 1 --2- 172.21.15.83:0/1212299431 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1a281ac440 0x7f1a281ac8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:03.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.563+0000 7f1a27e00640 1 -- 172.21.15.83:0/1212299431 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1a281ac440 msgr2=0x7f1a281ac8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:03.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.563+0000 7f1a27e00640 1 --2- 172.21.15.83:0/1212299431 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1a281ac440 0x7f1a281ac8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:03.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.563+0000 7f1a27e00640 1 -- 172.21.15.83:0/1212299431 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1a281a7b80 msgr2=0x7f1a281abf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:03.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.563+0000 7f1a27e00640 1 --2- 172.21.15.83:0/1212299431 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1a281a7b80 0x7f1a281abf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:03.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.563+0000 7f1a27e00640 1 -- 172.21.15.83:0/1212299431 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a281b41b0 con 0x7f1a28110760 2024-04-24T09:15:03.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.566+0000 7f1a27e00640 1 --2- 172.21.15.83:0/1212299431 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a28110760 0x7f1a281a7640 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f1a1400b4a0 tx=0x7f1a1400b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:03.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.566+0000 7f1a24c00640 1 -- 172.21.15.83:0/1212299431 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a14014070 con 0x7f1a28110760 2024-04-24T09:15:03.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.566+0000 7f1a24c00640 1 -- 172.21.15.83:0/1212299431 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1a14010ac0 con 0x7f1a28110760 2024-04-24T09:15:03.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.566+0000 7f1a24c00640 1 -- 172.21.15.83:0/1212299431 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a14017c30 con 0x7f1a28110760 2024-04-24T09:15:03.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.567+0000 7f1a2ec00640 1 -- 172.21.15.83:0/1212299431 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a281b43e0 con 0x7f1a28110760 2024-04-24T09:15:03.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.567+0000 7f1a2ec00640 1 -- 172.21.15.83:0/1212299431 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a281b48b0 con 0x7f1a28110760 2024-04-24T09:15:03.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.569+0000 7f1a2ec00640 1 -- 172.21.15.83:0/1212299431 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a28068e20 con 0x7f1a28110760 2024-04-24T09:15:03.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.574+0000 7f1a24c00640 1 -- 172.21.15.83:0/1212299431 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1a14017070 con 0x7f1a28110760 2024-04-24T09:15:03.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.577+0000 7f1a24c00640 1 --2- 172.21.15.83:0/1212299431 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f1a0c0a3da0 0x7f1a0c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:03.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.577+0000 7f1a24c00640 1 -- 172.21.15.83:0/1212299431 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9583+0+0 (secure 0 0 0) 0x7f1a140d0250 con 0x7f1a28110760 2024-04-24T09:15:03.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.578+0000 7f1a27400640 1 --2- 172.21.15.83:0/1212299431 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f1a0c0a3da0 0x7f1a0c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:03.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.578+0000 7f1a24c00640 1 -- 172.21.15.83:0/1212299431 <== mon.2 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1a14098550 con 0x7f1a28110760 2024-04-24T09:15:03.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:03.579+0000 7f1a27400640 1 --2- 172.21.15.83:0/1212299431 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f1a0c0a3da0 0x7f1a0c0a6260 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f1a1800bbe0 tx=0x7f1a1800b390 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:04.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.012+0000 7f1a2ec00640 1 -- 172.21.15.83:0/1212299431 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1a280008d0 con 0x7f1a28110760 2024-04-24T09:15:04.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.014+0000 7f1a24c00640 1 -- 172.21.15.83:0/1212299431 <== mon.2 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) ==== 94+0+11948 (secure 0 0 0) 0x7f1a1409c5e0 con 0x7f1a28110760 2024-04-24T09:15:04.017 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T09:15:04.017 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":68,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/358872257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":358872257},{"type":"v1","addr":"172.21.15.83:6835","nonce":358872257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3157070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3157070851},{"type":"v1","addr":"172.21.15.83:6837","nonce":3157070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":68,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T08:48:29.422576+0000","modified":"2024-04-24T09:14:58.180660+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34520,"mds_1":34493,"mds_2":34451,"mds_3":34427,"mds_4":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14658":{"gid":14658,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6839/4085470844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":4085470844},{"type":"v1","addr":"172.21.15.83:6839","nonce":4085470844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14676":{"gid":14676,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6841/2634754210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":2634754210},{"type":"v1","addr":"172.21.15.83:6841","nonce":2634754210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24382":{"gid":24382,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/1859921310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1859921310},{"type":"v1","addr":"172.21.15.189:6833","nonce":1859921310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/2241923197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2241923197},{"type":"v1","addr":"172.21.15.189:6835","nonce":2241923197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.189:6839/2548965336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":2548965336},{"type":"v1","addr":"172.21.15.189:6839","nonce":2548965336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26686":{"gid":26686,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6837/1678817161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":1678817161},{"type":"v1","addr":"172.21.15.189:6837","nonce":1678817161}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.138:6833/1157967658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1157967658},{"type":"v1","addr":"172.21.15.138:6833","nonce":1157967658}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.138:6835/1909219758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1909219758},{"type":"v1","addr":"172.21.15.138:6835","nonce":1909219758}]},"join_fscid":-1,"export_targets":[0,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34493":{"gid":34493,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.138:6837/900566754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":900566754},{"type":"v1","addr":"172.21.15.138:6837","nonce":900566754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34520":{"gid":34520,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.138:6839/2236932393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2236932393},{"type":"v1","addr":"172.21.15.138:6839","nonce":2236932393}]},"join_fscid":-1,"export_targets":[4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34520,"qdb_cluster":[34451,34493,34427,34520,24454]},"id":1}]} 2024-04-24T09:15:04.018 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 68 2024-04-24T09:15:04.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.018+0000 7f1a2ec00640 1 -- 172.21.15.83:0/1212299431 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f1a0c0a3da0 msgr2=0x7f1a0c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:04.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.018+0000 7f1a2ec00640 1 --2- 172.21.15.83:0/1212299431 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f1a0c0a3da0 0x7f1a0c0a6260 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f1a1800bbe0 tx=0x7f1a1800b390 comp rx=0 tx=0).stop 2024-04-24T09:15:04.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.018+0000 7f1a2ec00640 1 -- 172.21.15.83:0/1212299431 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a28110760 msgr2=0x7f1a281a7640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:04.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.018+0000 7f1a2ec00640 1 --2- 172.21.15.83:0/1212299431 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a28110760 0x7f1a281a7640 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f1a1400b4a0 tx=0x7f1a1400b960 comp rx=0 tx=0).stop 2024-04-24T09:15:04.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.018+0000 7f1a2ec00640 1 -- 172.21.15.83:0/1212299431 shutdown_connections 2024-04-24T09:15:04.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.018+0000 7f1a2ec00640 1 --2- 172.21.15.83:0/1212299431 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f1a0c0a3da0 0x7f1a0c0a6260 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:04.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.018+0000 7f1a2ec00640 1 --2- 172.21.15.83:0/1212299431 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1a281ac440 0x7f1a281ac8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:04.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.018+0000 7f1a2ec00640 1 --2- 172.21.15.83:0/1212299431 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1a281a7b80 0x7f1a281abf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:04.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.018+0000 7f1a2ec00640 1 --2- 172.21.15.83:0/1212299431 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a28110760 0x7f1a281a7640 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:04.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.018+0000 7f1a2ec00640 1 -- 172.21.15.83:0/1212299431 >> 172.21.15.83:0/1212299431 conn(0x7f1a2806d120 msgr2=0x7f1a280715d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:04.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.018+0000 7f1a2ec00640 1 -- 172.21.15.83:0/1212299431 shutdown_connections 2024-04-24T09:15:04.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.019+0000 7f1a2ec00640 1 -- 172.21.15.83:0/1212299431 wait complete. 2024-04-24T09:15:04.044 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-24T09:15:04.209 INFO:tasks.workunit.client.0.smithi083.stdout: 10 70655 6.53 MB/sec execute 455 sec latency 2864.985 ms 2024-04-24T09:15:04.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:15:04 smithi083 ceph-mon[27907]: pgmap v740: 129 pgs: 129 active+clean; 1.0 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.6 MiB/s wr, 339 op/s 2024-04-24T09:15:04.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:15:04 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/1212299431' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:15:04.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.361+0000 7f721ee00640 1 -- 172.21.15.83:0/2702207729 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72181093f0 msgr2=0x7f7218110fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:04.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.361+0000 7f721ee00640 1 --2- 172.21.15.83:0/2702207729 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72181093f0 0x7f7218110fc0 secure :-1 s=READY pgs=2695 cs=0 l=1 rev1=1 crypto rx=0x7f7210009ca0 tx=0x7f721002a8c0 comp rx=0 tx=0).stop 2024-04-24T09:15:04.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.361+0000 7f721ee00640 1 -- 172.21.15.83:0/2702207729 shutdown_connections 2024-04-24T09:15:04.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.361+0000 7f721ee00640 1 --2- 172.21.15.83:0/2702207729 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72181093f0 0x7f7218110fc0 unknown :-1 s=CLOSED pgs=2695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:04.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.361+0000 7f721ee00640 1 --2- 172.21.15.83:0/2702207729 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7218104cd0 0x7f7218108d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:04.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.361+0000 7f721ee00640 1 --2- 172.21.15.83:0/2702207729 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7218104320 0x7f7218104700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:04.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.361+0000 7f721ee00640 1 -- 172.21.15.83:0/2702207729 >> 172.21.15.83:0/2702207729 conn(0x7f72180ffb40 msgr2=0x7f7218101f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:04.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.361+0000 7f721ee00640 1 -- 172.21.15.83:0/2702207729 shutdown_connections 2024-04-24T09:15:04.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.362+0000 7f721ee00640 1 -- 172.21.15.83:0/2702207729 wait complete. 2024-04-24T09:15:04.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.363+0000 7f721ee00640 1 Processor -- start 2024-04-24T09:15:04.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.363+0000 7f721ee00640 1 -- start start 2024-04-24T09:15:04.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.363+0000 7f721ee00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7218104320 0x7f7218110f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:04.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.363+0000 7f721ee00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f72181093f0 0x7f721810e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:04.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.363+0000 7f721ee00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f721810e620 0x7f721810eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:04.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.363+0000 7f721ee00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7218114360 con 0x7f721810e620 2024-04-24T09:15:04.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.363+0000 7f721ee00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f72181141e0 con 0x7f72181093f0 2024-04-24T09:15:04.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.363+0000 7f721ee00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f72181144e0 con 0x7f7218104320 2024-04-24T09:15:04.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.364+0000 7f721e400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f721810e620 0x7f721810eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:04.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.364+0000 7f721e400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f721810e620 0x7f721810eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54834/0 (socket says 172.21.15.83:54834) 2024-04-24T09:15:04.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.364+0000 7f721e400640 1 -- 172.21.15.83:0/3096967918 learned_addr learned my addr 172.21.15.83:0/3096967918 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:15:04.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.364+0000 7f721d000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f72181093f0 0x7f721810e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:04.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.364+0000 7f721da00640 1 --2- 172.21.15.83:0/3096967918 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7218104320 0x7f7218110f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:04.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.364+0000 7f721e400640 1 -- 172.21.15.83:0/3096967918 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7218104320 msgr2=0x7f7218110f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:04.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.364+0000 7f721e400640 1 --2- 172.21.15.83:0/3096967918 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7218104320 0x7f7218110f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:04.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.364+0000 7f721e400640 1 -- 172.21.15.83:0/3096967918 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f72181093f0 msgr2=0x7f721810e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:04.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.364+0000 7f721e400640 1 --2- 172.21.15.83:0/3096967918 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f72181093f0 0x7f721810e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:04.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.364+0000 7f721e400640 1 -- 172.21.15.83:0/3096967918 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72181ab330 con 0x7f721810e620 2024-04-24T09:15:04.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.365+0000 7f721e400640 1 --2- 172.21.15.83:0/3096967918 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f721810e620 0x7f721810eab0 secure :-1 s=READY pgs=2696 cs=0 l=1 rev1=1 crypto rx=0x7f7210031040 tx=0x7f7210032f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:04.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.365+0000 7f7216000640 1 -- 172.21.15.83:0/3096967918 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7210003b70 con 0x7f721810e620 2024-04-24T09:15:04.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.365+0000 7f7216000640 1 -- 172.21.15.83:0/3096967918 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7210003d10 con 0x7f721810e620 2024-04-24T09:15:04.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.365+0000 7f721ee00640 1 -- 172.21.15.83:0/3096967918 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72181ab5c0 con 0x7f721810e620 2024-04-24T09:15:04.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.365+0000 7f7216000640 1 -- 172.21.15.83:0/3096967918 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7210030a50 con 0x7f721810e620 2024-04-24T09:15:04.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.365+0000 7f721ee00640 1 -- 172.21.15.83:0/3096967918 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72181abb60 con 0x7f721810e620 2024-04-24T09:15:04.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.370+0000 7f72233d3740 1 Processor -- start 2024-04-24T09:15:04.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.370+0000 7f7216000640 1 -- 172.21.15.83:0/3096967918 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7210030bf0 con 0x7f721810e620 2024-04-24T09:15:04.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.370+0000 7f72233d3740 1 -- start start 2024-04-24T09:15:04.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.370+0000 7f72233d3740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5616b497ab40 0x5616b497af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:04.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.370+0000 7f72233d3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5616b497b460 0x5616b49855f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:04.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.370+0000 7f72233d3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5616b4985b30 0x5616b4987f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:04.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.370+0000 7f72233d3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5616b48e01d0 con 0x5616b4985b30 2024-04-24T09:15:04.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.370+0000 7f72233d3740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5616b4886450 con 0x5616b497ab40 2024-04-24T09:15:04.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.370+0000 7f72233d3740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5616b48dc190 con 0x5616b497b460 2024-04-24T09:15:04.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.371+0000 7f721d000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5616b497ab40 0x5616b497af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:04.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.371+0000 7f721d000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5616b497ab40 0x5616b497af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:37152/0 (socket says 172.21.15.83:37152) 2024-04-24T09:15:04.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.371+0000 7f721d000640 1 -- 172.21.15.83:0/796046693 learned_addr learned my addr 172.21.15.83:0/796046693 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:15:04.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.371+0000 7f721da00640 1 --2- 172.21.15.83:0/796046693 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5616b497b460 0x5616b49855f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:04.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.371+0000 7f721d000640 1 --2- 172.21.15.83:0/796046693 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5616b4985b30 0x5616b4987f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:04.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.372+0000 7f7216000640 1 --2- 172.21.15.83:0/3096967918 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f72000a3e60 0x7f72000a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:04.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.372+0000 7f721d000640 1 -- 172.21.15.83:0/796046693 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5616b497b460 msgr2=0x5616b49855f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:04.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.372+0000 7f721d000640 1 --2- 172.21.15.83:0/796046693 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5616b497b460 0x5616b49855f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:04.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.372+0000 7f721d000640 1 -- 172.21.15.83:0/796046693 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5616b4985b30 msgr2=0x5616b4987f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:04.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.372+0000 7f721d000640 1 --2- 172.21.15.83:0/796046693 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5616b4985b30 0x5616b4987f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:04.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.372+0000 7f721d000640 1 -- 172.21.15.83:0/796046693 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616b4943310 con 0x5616b497ab40 2024-04-24T09:15:04.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.372+0000 7f721da00640 1 --2- 172.21.15.83:0/796046693 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5616b497b460 0x5616b49855f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:15:04.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.372+0000 7f7216000640 1 -- 172.21.15.83:0/3096967918 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9583+0+0 (secure 0 0 0) 0x7f72100f1f20 con 0x7f721810e620 2024-04-24T09:15:04.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.372+0000 7f721d000640 1 --2- 172.21.15.83:0/796046693 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5616b497ab40 0x5616b497af20 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f7204011740 tx=0x7f72040046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a070b15d8b58cf4a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:04.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.372+0000 7f721e400640 1 --2- 172.21.15.83:0/3096967918 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f72000a3e60 0x7f72000a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:04.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.373+0000 7f720aa00640 1 -- 172.21.15.83:0/796046693 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7204004050 con 0x5616b497ab40 2024-04-24T09:15:04.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.373+0000 7f720aa00640 1 -- 172.21.15.83:0/796046693 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f720400fc70 con 0x5616b497ab40 2024-04-24T09:15:04.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.373+0000 7f720aa00640 1 -- 172.21.15.83:0/796046693 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7204003a50 con 0x5616b497ab40 2024-04-24T09:15:04.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.374+0000 7f721e400640 1 --2- 172.21.15.83:0/3096967918 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f72000a3e60 0x7f72000a6320 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f72100ba640 tx=0x7f721004e000 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:04.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.374+0000 7f72233d3740 1 -- 172.21.15.83:0/796046693 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5616b497ab40 msgr2=0x5616b497af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:04.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.374+0000 7f72233d3740 1 --2- 172.21.15.83:0/796046693 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5616b497ab40 0x5616b497af20 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f7204011740 tx=0x7f72040046c0 comp rx=0 tx=0).stop 2024-04-24T09:15:04.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.374+0000 7f72233d3740 1 -- 172.21.15.83:0/796046693 shutdown_connections 2024-04-24T09:15:04.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.374+0000 7f72233d3740 1 --2- 172.21.15.83:0/796046693 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5616b4985b30 0x5616b4987f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:04.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.374+0000 7f72233d3740 1 --2- 172.21.15.83:0/796046693 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5616b497b460 0x5616b49855f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:04.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.374+0000 7f72233d3740 1 --2- 172.21.15.83:0/796046693 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5616b497ab40 0x5616b497af20 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:04.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.374+0000 7f72233d3740 1 -- 172.21.15.83:0/796046693 >> 172.21.15.83:0/796046693 conn(0x5616b4976e60 msgr2=0x5616b4977240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:04.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.374+0000 7f72233d3740 1 -- 172.21.15.83:0/796046693 shutdown_connections 2024-04-24T09:15:04.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.375+0000 7f72233d3740 1 -- 172.21.15.83:0/796046693 wait complete. 2024-04-24T09:15:04.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.375+0000 7f72233d3740 1 -- start start 2024-04-24T09:15:04.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.376+0000 7f72233d3740 1 Processor -- start 2024-04-24T09:15:04.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.376+0000 7f72233d3740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5616b497ab40 0x5616b4982950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:04.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.376+0000 7f72233d3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5616b497b460 0x5616b4982e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:04.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.376+0000 7f72233d3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5616b4985b30 0x5616b49833d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:04.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.376+0000 7f72233d3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5616b49889c0 con 0x5616b497b460 2024-04-24T09:15:04.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.376+0000 7f72233d3740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5616b4899810 con 0x5616b497ab40 2024-04-24T09:15:04.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.376+0000 7f72233d3740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5616b4988b40 con 0x5616b4985b30 2024-04-24T09:15:04.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.377+0000 7f72233d3740 10 client.0 _ll_register_callbacks cb 0x5616b4867140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:15:04.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.377+0000 7f721da00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5616b497b460 0x5616b4982e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:04.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.377+0000 7f721d000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5616b497ab40 0x5616b4982950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:04.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.377+0000 7f721da00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5616b497b460 0x5616b4982e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54856/0 (socket says 172.21.15.83:54856) 2024-04-24T09:15:04.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.377+0000 7f721da00640 1 -- 172.21.15.83:0/838597690 learned_addr learned my addr 172.21.15.83:0/838597690 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:15:04.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.378+0000 7f721d000640 1 -- 172.21.15.83:0/838597690 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5616b4985b30 msgr2=0x5616b49833d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:04.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.378+0000 7f721d000640 1 --2- 172.21.15.83:0/838597690 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5616b4985b30 0x5616b49833d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:04.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.378+0000 7f721d000640 1 -- 172.21.15.83:0/838597690 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5616b497b460 msgr2=0x5616b4982e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:04.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.378+0000 7f721d000640 1 --2- 172.21.15.83:0/838597690 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5616b497b460 0x5616b4982e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:04.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.378+0000 7f721d000640 1 -- 172.21.15.83:0/838597690 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616b48aa3f0 con 0x5616b497ab40 2024-04-24T09:15:04.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.378+0000 7f721da00640 1 --2- 172.21.15.83:0/838597690 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5616b497b460 0x5616b4982e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:15:04.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.378+0000 7f721d000640 1 --2- 172.21.15.83:0/838597690 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5616b497ab40 0x5616b4982950 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f72040097a0 tx=0x7f7204005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c02eb988f675d093 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:04.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.378+0000 7f71ff400640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:15:04.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.378+0000 7f71ff400640 1 -- 172.21.15.83:0/838597690 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f720401f3f0 con 0x5616b497ab40 2024-04-24T09:15:04.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.378+0000 7f71ff400640 1 -- 172.21.15.83:0/838597690 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(128..128 src has 1..128) ==== 9583+0+0 (secure 0 0 0) 0x7f720401f590 con 0x5616b497ab40 2024-04-24T09:15:04.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.379+0000 7f72233d3740 1 -- 172.21.15.83:0/838597690 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f720400fc70 con 0x5616b497ab40 2024-04-24T09:15:04.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.379+0000 7f71ff400640 1 -- 172.21.15.83:0/838597690 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7f7204031050 con 0x5616b497ab40 2024-04-24T09:15:04.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.379+0000 7f72233d3740 10 client.40553 fetch_fsmap learned FSMap version 68 2024-04-24T09:15:04.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.379+0000 7f72233d3740 1 -- 172.21.15.83:0/838597690 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) -- 0x5616b45f23f0 con 0x5616b497ab40 2024-04-24T09:15:04.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.380+0000 7f71ff400640 1 -- 172.21.15.83:0/838597690 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f720400fc70 con 0x5616b497ab40 2024-04-24T09:15:04.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.380+0000 7f71ff400640 1 -- 172.21.15.83:0/838597690 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 68) ==== 8374+0+0 (secure 0 0 0) 0x7f720401e9a0 con 0x5616b497ab40 2024-04-24T09:15:04.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.380+0000 7f72233d3740 10 client.40553 fetch_fsmap finished waiting for FSMap version 68 2024-04-24T09:15:04.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.380+0000 7f72233d3740 10 client.40553 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-24T09:15:04.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.381+0000 7f72233d3740 20 client.40553 populate_metadata read hostname 'smithi083' 2024-04-24T09:15:04.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.381+0000 7f72233d3740 1 --2- 172.21.15.83:0/838597690 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x5616b4995ca0 0x5616b4998090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:04.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.381+0000 7f72233d3740 4 client.40553 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:15:04.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.381+0000 7f72233d3740 1 -- 172.21.15.83:0/838597690 --> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5616b48edbf0 con 0x5616b4995ca0 2024-04-24T09:15:04.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.381+0000 7f721da00640 1 --2- 172.21.15.83:0/838597690 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x5616b4995ca0 0x5616b4998090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:04.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.383+0000 7f721da00640 1 --2- 172.21.15.83:0/838597690 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x5616b4995ca0 0x5616b4998090 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=960aaf1e56b59fc4 server_cookie=fd55969706d0a2a4 in_seq=0 out_seq=0 2024-04-24T09:15:04.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.383+0000 7f71ff400640 10 client.40553 ms_handle_connect on v2:172.21.15.189:6838/2548965336 2024-04-24T09:15:04.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.386+0000 7f71ff400640 1 -- 172.21.15.83:0/838597690 <== mds.4 v2:172.21.15.189:6838/2548965336 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5616b48edbf0 con 0x5616b4995ca0 2024-04-24T09:15:04.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.386+0000 7f71ff400640 10 client.40553 handle_command_reply: tid=0 2024-04-24T09:15:04.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.386+0000 7f72233d3740 1 client.40553 shutdown 2024-04-24T09:15:04.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.387+0000 7f72233d3740 1 -- 172.21.15.83:0/838597690 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5616b497ab40 msgr2=0x5616b4982950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:04.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.387+0000 7f72233d3740 1 --2- 172.21.15.83:0/838597690 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5616b497ab40 0x5616b4982950 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f72040097a0 tx=0x7f7204005740 comp rx=0 tx=0).stop 2024-04-24T09:15:04.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.387+0000 7f72233d3740 1 -- 172.21.15.83:0/838597690 shutdown_connections 2024-04-24T09:15:04.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.387+0000 7f72233d3740 1 --2- 172.21.15.83:0/838597690 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x5616b4995ca0 0x5616b4998090 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:04.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.387+0000 7f72233d3740 1 --2- 172.21.15.83:0/838597690 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5616b4985b30 0x5616b49833d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:04.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.387+0000 7f72233d3740 1 --2- 172.21.15.83:0/838597690 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5616b497b460 0x5616b4982e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:04.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.387+0000 7f72233d3740 1 --2- 172.21.15.83:0/838597690 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5616b497ab40 0x5616b4982950 secure :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f72040097a0 tx=0x7f7204005740 comp rx=0 tx=0).stop 2024-04-24T09:15:04.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.387+0000 7f72233d3740 1 -- 172.21.15.83:0/838597690 >> 172.21.15.83:0/838597690 conn(0x5616b4976e60 msgr2=0x5616b497cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:04.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.387+0000 7f71ff400640 0 client.40553 ms_handle_reset on v2:172.21.15.189:6838/2548965336 2024-04-24T09:15:04.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.387+0000 7f72233d3740 1 -- 172.21.15.83:0/838597690 shutdown_connections 2024-04-24T09:15:04.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.387+0000 7f72233d3740 1 -- 172.21.15.83:0/838597690 wait complete. 2024-04-24T09:15:04.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.387+0000 7f72233d3740 20 client.40553 trim_cache size 0 max 16384 2024-04-24T09:15:04.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.409+0000 7f72233d3740 1 Processor -- start 2024-04-24T09:15:04.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.409+0000 7f72233d3740 1 -- start start 2024-04-24T09:15:04.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.410+0000 7f72233d3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5616b497b460 0x5616b49918b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:04.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.410+0000 7f72233d3740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5616b4985b30 0x5616b497d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:04.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.410+0000 7f72233d3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5616b4981ab0 0x5616b4975dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:04.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.410+0000 7f72233d3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f72181b5970 con 0x5616b4981ab0 2024-04-24T09:15:04.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.410+0000 7f72233d3740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f72181b7540 con 0x5616b4985b30 2024-04-24T09:15:04.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.410+0000 7f72233d3740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f72181b6040 con 0x5616b497b460 2024-04-24T09:15:04.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.410+0000 7f721d000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5616b4981ab0 0x5616b4975dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:04.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.410+0000 7f721d000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5616b4981ab0 0x5616b4975dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54866/0 (socket says 172.21.15.83:54866) 2024-04-24T09:15:04.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.410+0000 7f721d000640 1 -- 172.21.15.83:0/2123550527 learned_addr learned my addr 172.21.15.83:0/2123550527 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:15:04.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.410+0000 7f721da00640 1 --2- 172.21.15.83:0/2123550527 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5616b4985b30 0x5616b497d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:04.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.410+0000 7f721d000640 1 --2- 172.21.15.83:0/2123550527 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5616b497b460 0x5616b49918b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:04.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.411+0000 7f721d000640 1 -- 172.21.15.83:0/2123550527 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5616b497b460 msgr2=0x5616b49918b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:04.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.411+0000 7f721d000640 1 --2- 172.21.15.83:0/2123550527 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5616b497b460 0x5616b49918b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:04.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.411+0000 7f721d000640 1 -- 172.21.15.83:0/2123550527 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5616b4985b30 msgr2=0x5616b497d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:04.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.411+0000 7f721d000640 1 --2- 172.21.15.83:0/2123550527 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5616b4985b30 0x5616b497d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:04.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.411+0000 7f721d000640 1 -- 172.21.15.83:0/2123550527 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616b497b290 con 0x5616b4981ab0 2024-04-24T09:15:04.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.411+0000 7f721d000640 1 --2- 172.21.15.83:0/2123550527 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5616b4981ab0 0x5616b4975dd0 secure :-1 s=READY pgs=2697 cs=0 l=1 rev1=1 crypto rx=0x7f72040059f0 tx=0x7f72040035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab47002ad79eca4f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:04.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.412+0000 7f720a000640 1 -- 172.21.15.83:0/2123550527 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7204023070 con 0x5616b4981ab0 2024-04-24T09:15:04.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.412+0000 7f720a000640 1 -- 172.21.15.83:0/2123550527 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f720400fbf0 con 0x5616b4981ab0 2024-04-24T09:15:04.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.412+0000 7f720a000640 1 -- 172.21.15.83:0/2123550527 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f720401fd20 con 0x5616b4981ab0 2024-04-24T09:15:04.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.413+0000 7f72233d3740 1 -- 172.21.15.83:0/2123550527 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5616b4981ab0 msgr2=0x5616b4975dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:04.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.413+0000 7f72233d3740 1 --2- 172.21.15.83:0/2123550527 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5616b4981ab0 0x5616b4975dd0 secure :-1 s=READY pgs=2697 cs=0 l=1 rev1=1 crypto rx=0x7f72040059f0 tx=0x7f72040035f0 comp rx=0 tx=0).stop 2024-04-24T09:15:04.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.413+0000 7f72233d3740 1 -- 172.21.15.83:0/2123550527 shutdown_connections 2024-04-24T09:15:04.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.413+0000 7f72233d3740 1 --2- 172.21.15.83:0/2123550527 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5616b4981ab0 0x5616b4975dd0 unknown :-1 s=CLOSED pgs=2697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:04.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.413+0000 7f72233d3740 1 --2- 172.21.15.83:0/2123550527 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5616b4985b30 0x5616b497d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:04.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.413+0000 7f72233d3740 1 --2- 172.21.15.83:0/2123550527 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5616b497b460 0x5616b49918b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:04.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.413+0000 7f72233d3740 1 -- 172.21.15.83:0/2123550527 >> 172.21.15.83:0/2123550527 conn(0x5616b4976e60 msgr2=0x5616b49756c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:04.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.413+0000 7f72233d3740 1 -- 172.21.15.83:0/2123550527 shutdown_connections 2024-04-24T09:15:04.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.413+0000 7f72233d3740 1 -- 172.21.15.83:0/2123550527 wait complete. 2024-04-24T09:15:04.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.413+0000 7f72233d3740 1 -- start start 2024-04-24T09:15:04.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.414+0000 7f72233d3740 1 Processor -- start 2024-04-24T09:15:04.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.414+0000 7f72233d3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5616b497b460 0x5616b49878f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:04.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.414+0000 7f72233d3740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5616b4981ab0 0x5616b4985f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:04.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.414+0000 7f72233d3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5616b4985b30 0x5616b4986500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:04.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.414+0000 7f72233d3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f72181b6e40 con 0x5616b497b460 2024-04-24T09:15:04.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.414+0000 7f72233d3740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f72181b6770 con 0x5616b4981ab0 2024-04-24T09:15:04.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.414+0000 7f72233d3740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f72181b3970 con 0x5616b4985b30 2024-04-24T09:15:04.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.414+0000 7f72233d3740 10 client.0 _ll_register_callbacks cb 0x7f7204009de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:15:04.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.415+0000 7f721da00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5616b4981ab0 0x5616b4985f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:04.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.415+0000 7f721d000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5616b497b460 0x5616b49878f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:04.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.415+0000 7f721da00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5616b4981ab0 0x5616b4985f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:37162/0 (socket says 172.21.15.83:37162) 2024-04-24T09:15:04.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.415+0000 7f721da00640 1 -- 172.21.15.83:0/3521995514 learned_addr learned my addr 172.21.15.83:0/3521995514 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:15:04.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.415+0000 7f721da00640 1 -- 172.21.15.83:0/3521995514 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5616b4985b30 msgr2=0x5616b4986500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:04.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.415+0000 7f721da00640 1 --2- 172.21.15.83:0/3521995514 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5616b4985b30 0x5616b4986500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:04.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.415+0000 7f721da00640 1 -- 172.21.15.83:0/3521995514 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5616b497b460 msgr2=0x5616b49878f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:04.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.415+0000 7f721da00640 1 --2- 172.21.15.83:0/3521995514 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5616b497b460 0x5616b49878f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:04.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.415+0000 7f721da00640 1 -- 172.21.15.83:0/3521995514 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616b46bc7d0 con 0x5616b4981ab0 2024-04-24T09:15:04.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.415+0000 7f721d000640 1 --2- 172.21.15.83:0/3521995514 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5616b497b460 0x5616b49878f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:15:04.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.416+0000 7f71ff400640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:15:04.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.416+0000 7f721da00640 1 --2- 172.21.15.83:0/3521995514 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5616b4981ab0 0x5616b4985f10 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x5616b4983f20 tx=0x7f720c012b00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76cabfed16d3839a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:04.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.416+0000 7f71ff400640 1 -- 172.21.15.83:0/3521995514 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f720c0211c0 con 0x5616b4981ab0 2024-04-24T09:15:04.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.416+0000 7f72233d3740 1 -- 172.21.15.83:0/3521995514 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f720400fbf0 con 0x5616b4981ab0 2024-04-24T09:15:04.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.416+0000 7f71ff400640 1 -- 172.21.15.83:0/3521995514 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(128..128 src has 1..128) ==== 9583+0+0 (secure 0 0 0) 0x7f720c021360 con 0x5616b4981ab0 2024-04-24T09:15:04.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.417+0000 7f71ff400640 1 -- 172.21.15.83:0/3521995514 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7f720c021560 con 0x5616b4981ab0 2024-04-24T09:15:04.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.417+0000 7f72233d3740 10 client.40559 fetch_fsmap learned FSMap version 68 2024-04-24T09:15:04.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.417+0000 7f72233d3740 1 -- 172.21.15.83:0/3521995514 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) -- 0x5616b498dd90 con 0x5616b4981ab0 2024-04-24T09:15:04.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.417+0000 7f71ff400640 1 -- 172.21.15.83:0/3521995514 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f720c02f030 con 0x5616b4981ab0 2024-04-24T09:15:04.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.417+0000 7f71ff400640 1 -- 172.21.15.83:0/3521995514 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 68) ==== 8374+0+0 (secure 0 0 0) 0x7f720c028020 con 0x5616b4981ab0 2024-04-24T09:15:04.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.418+0000 7f72233d3740 10 client.40559 fetch_fsmap finished waiting for FSMap version 68 2024-04-24T09:15:04.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.418+0000 7f72233d3740 10 client.40559 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-24T09:15:04.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.418+0000 7f72233d3740 20 client.40559 populate_metadata read hostname 'smithi083' 2024-04-24T09:15:04.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.418+0000 7f72233d3740 1 --2- 172.21.15.83:0/3521995514 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x5616b49fc070 0x5616b49fe460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:04.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.418+0000 7f72233d3740 4 client.40559 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:15:04.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.418+0000 7f72233d3740 1 -- 172.21.15.83:0/3521995514 --> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5616b48e7a60 con 0x5616b49fc070 2024-04-24T09:15:04.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.418+0000 7f721d000640 1 --2- 172.21.15.83:0/3521995514 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x5616b49fc070 0x5616b49fe460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:04.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.419+0000 7f721d000640 1 --2- 172.21.15.83:0/3521995514 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x5616b49fc070 0x5616b49fe460 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=903aded2510c7fd6 server_cookie=d809080e91bf2dc7 in_seq=0 out_seq=0 2024-04-24T09:15:04.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.419+0000 7f71ff400640 10 client.40559 ms_handle_connect on v2:172.21.15.189:6838/2548965336 2024-04-24T09:15:04.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.420+0000 7f71ff400640 1 -- 172.21.15.83:0/3521995514 <== mds.4 v2:172.21.15.189:6838/2548965336 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5616b48e7a60 con 0x5616b49fc070 2024-04-24T09:15:04.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.420+0000 7f71ff400640 10 client.40559 handle_command_reply: tid=0 2024-04-24T09:15:04.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.420+0000 7f72233d3740 1 client.40559 shutdown 2024-04-24T09:15:04.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.420+0000 7f72233d3740 1 -- 172.21.15.83:0/3521995514 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5616b4981ab0 msgr2=0x5616b4985f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:04.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.420+0000 7f72233d3740 1 --2- 172.21.15.83:0/3521995514 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5616b4981ab0 0x5616b4985f10 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x5616b4983f20 tx=0x7f720c012b00 comp rx=0 tx=0).stop 2024-04-24T09:15:04.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.420+0000 7f72233d3740 1 -- 172.21.15.83:0/3521995514 shutdown_connections 2024-04-24T09:15:04.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.420+0000 7f72233d3740 1 --2- 172.21.15.83:0/3521995514 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x5616b49fc070 0x5616b49fe460 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:04.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.420+0000 7f72233d3740 1 --2- 172.21.15.83:0/3521995514 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5616b4985b30 0x5616b4986500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:04.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.420+0000 7f72233d3740 1 --2- 172.21.15.83:0/3521995514 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5616b4981ab0 0x5616b4985f10 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:04.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.420+0000 7f72233d3740 1 --2- 172.21.15.83:0/3521995514 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5616b497b460 0x5616b49878f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:04.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.420+0000 7f71ff400640 0 client.40559 ms_handle_reset on v2:172.21.15.189:6838/2548965336 2024-04-24T09:15:04.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.420+0000 7f72233d3740 1 -- 172.21.15.83:0/3521995514 >> 172.21.15.83:0/3521995514 conn(0x5616b4976e60 msgr2=0x5616b4975f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:04.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.420+0000 7f72233d3740 1 -- 172.21.15.83:0/3521995514 shutdown_connections 2024-04-24T09:15:04.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.421+0000 7f72233d3740 1 -- 172.21.15.83:0/3521995514 wait complete. 2024-04-24T09:15:04.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.421+0000 7f72233d3740 20 client.40559 trim_cache size 0 max 16384 2024-04-24T09:15:04.421 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:15:04.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.422+0000 7f721ee00640 1 -- 172.21.15.83:0/3096967918 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f72000a3e60 msgr2=0x7f72000a6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:04.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.422+0000 7f721ee00640 1 --2- 172.21.15.83:0/3096967918 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f72000a3e60 0x7f72000a6320 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f72100ba640 tx=0x7f721004e000 comp rx=0 tx=0).stop 2024-04-24T09:15:04.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.422+0000 7f721ee00640 1 -- 172.21.15.83:0/3096967918 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f721810e620 msgr2=0x7f721810eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:04.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.422+0000 7f721ee00640 1 --2- 172.21.15.83:0/3096967918 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f721810e620 0x7f721810eab0 secure :-1 s=READY pgs=2696 cs=0 l=1 rev1=1 crypto rx=0x7f7210031040 tx=0x7f7210032f80 comp rx=0 tx=0).stop 2024-04-24T09:15:04.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.423+0000 7f721ee00640 1 -- 172.21.15.83:0/3096967918 shutdown_connections 2024-04-24T09:15:04.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.423+0000 7f721ee00640 1 --2- 172.21.15.83:0/3096967918 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f72000a3e60 0x7f72000a6320 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:04.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.423+0000 7f721ee00640 1 --2- 172.21.15.83:0/3096967918 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f721810e620 0x7f721810eab0 unknown :-1 s=CLOSED pgs=2696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:04.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.423+0000 7f721ee00640 1 --2- 172.21.15.83:0/3096967918 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f72181093f0 0x7f721810e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:04.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.423+0000 7f721ee00640 1 --2- 172.21.15.83:0/3096967918 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7218104320 0x7f7218110f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:04.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.423+0000 7f721ee00640 1 -- 172.21.15.83:0/3096967918 >> 172.21.15.83:0/3096967918 conn(0x7f72180ffb40 msgr2=0x7f7218100e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:04.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.423+0000 7f721ee00640 1 -- 172.21.15.83:0/3096967918 shutdown_connections 2024-04-24T09:15:04.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.423+0000 7f721ee00640 1 -- 172.21.15.83:0/3096967918 wait complete. 2024-04-24T09:15:04.447 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-24T09:15:04.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:15:04 smithi138 ceph-mon[31441]: pgmap v740: 129 pgs: 129 active+clean; 1.0 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.6 MiB/s wr, 339 op/s 2024-04-24T09:15:04.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:15:04 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/1212299431' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:15:04.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:15:04 smithi189 ceph-mon[31791]: pgmap v740: 129 pgs: 129 active+clean; 1.0 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.6 MiB/s wr, 339 op/s 2024-04-24T09:15:04.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:15:04 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/1212299431' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:15:04.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.801+0000 7f78f8200640 1 -- 172.21.15.83:0/3212489976 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f78f0112ee0 msgr2=0x7f78f01152d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:04.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.801+0000 7f78f8200640 1 --2- 172.21.15.83:0/3212489976 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f78f0112ee0 0x7f78f01152d0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f78e8009c10 tx=0x7f78e802a6e0 comp rx=0 tx=0).stop 2024-04-24T09:15:04.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.801+0000 7f78f8200640 1 -- 172.21.15.83:0/3212489976 shutdown_connections 2024-04-24T09:15:04.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.801+0000 7f78f8200640 1 --2- 172.21.15.83:0/3212489976 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f78f0112ee0 0x7f78f01152d0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:04.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.801+0000 7f78f8200640 1 --2- 172.21.15.83:0/3212489976 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f78f0076e50 0x7f78f01129a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:04.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.801+0000 7f78f8200640 1 --2- 172.21.15.83:0/3212489976 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78f0076530 0x7f78f0076910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:04.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.801+0000 7f78f8200640 1 -- 172.21.15.83:0/3212489976 >> 172.21.15.83:0/3212489976 conn(0x7f78f00ffd90 msgr2=0x7f78f01021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:04.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.802+0000 7f78f8200640 1 -- 172.21.15.83:0/3212489976 shutdown_connections 2024-04-24T09:15:04.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.802+0000 7f78f8200640 1 -- 172.21.15.83:0/3212489976 wait complete. 2024-04-24T09:15:04.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.803+0000 7f78f8200640 1 Processor -- start 2024-04-24T09:15:04.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.803+0000 7f78f8200640 1 -- start start 2024-04-24T09:15:04.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.803+0000 7f78f8200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f78f0076530 0x7f78f00738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:04.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.803+0000 7f78f8200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f78f0076e50 0x7f78f006c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:04.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.803+0000 7f78f8200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78f0112ee0 0x7f78f006ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:04.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.803+0000 7f78f8200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f78f01188f0 con 0x7f78f0112ee0 2024-04-24T09:15:04.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.803+0000 7f78f8200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f78f0118770 con 0x7f78f0076530 2024-04-24T09:15:04.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.803+0000 7f78f8200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f78f0118a70 con 0x7f78f0076e50 2024-04-24T09:15:04.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.804+0000 7f78f5800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f78f0076530 0x7f78f00738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:04.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.804+0000 7f78f5800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f78f0076530 0x7f78f00738d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:37184/0 (socket says 172.21.15.83:37184) 2024-04-24T09:15:04.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.804+0000 7f78f5800640 1 -- 172.21.15.83:0/2880577173 learned_addr learned my addr 172.21.15.83:0/2880577173 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:15:04.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.804+0000 7f78f6200640 1 --2- 172.21.15.83:0/2880577173 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78f0112ee0 0x7f78f006ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:04.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.804+0000 7f78f4e00640 1 --2- 172.21.15.83:0/2880577173 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f78f0076e50 0x7f78f006c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:04.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.804+0000 7f78f5800640 1 -- 172.21.15.83:0/2880577173 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f78f0076e50 msgr2=0x7f78f006c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:04.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.804+0000 7f78f5800640 1 --2- 172.21.15.83:0/2880577173 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f78f0076e50 0x7f78f006c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:04.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.804+0000 7f78f5800640 1 -- 172.21.15.83:0/2880577173 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78f0112ee0 msgr2=0x7f78f006ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:04.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.805+0000 7f78f5800640 1 --2- 172.21.15.83:0/2880577173 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78f0112ee0 0x7f78f006ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:04.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.805+0000 7f78f5800640 1 -- 172.21.15.83:0/2880577173 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78f006d530 con 0x7f78f0076530 2024-04-24T09:15:04.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.805+0000 7f78f6200640 1 --2- 172.21.15.83:0/2880577173 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78f0112ee0 0x7f78f006ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:15:04.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.805+0000 7f78f5800640 1 --2- 172.21.15.83:0/2880577173 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f78f0076530 0x7f78f00738d0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f78e400e9f0 tx=0x7f78e400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:04.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.805+0000 7f78ee000640 1 -- 172.21.15.83:0/2880577173 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78e400cd80 con 0x7f78f0076530 2024-04-24T09:15:04.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.805+0000 7f78f8200640 1 -- 172.21.15.83:0/2880577173 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78f00713f0 con 0x7f78f0076530 2024-04-24T09:15:04.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.806+0000 7f78f8200640 1 -- 172.21.15.83:0/2880577173 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78f0071990 con 0x7f78f0076530 2024-04-24T09:15:04.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.806+0000 7f78ee000640 1 -- 172.21.15.83:0/2880577173 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f78e40108e0 con 0x7f78f0076530 2024-04-24T09:15:04.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.806+0000 7f78ee000640 1 -- 172.21.15.83:0/2880577173 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78e4018920 con 0x7f78f0076530 2024-04-24T09:15:04.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.810+0000 7f78faf64740 1 Processor -- start 2024-04-24T09:15:04.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.810+0000 7f78faf64740 1 -- start start 2024-04-24T09:15:04.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.811+0000 7f78faf64740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55914f8cb950 0x55914f8cbd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:04.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.811+0000 7f78faf64740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55914f8cc270 0x55914f8d6400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:04.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.811+0000 7f78faf64740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55914f8d6940 0x55914f8d8d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:04.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.811+0000 7f78faf64740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55914f8311d0 con 0x55914f8cb950 2024-04-24T09:15:04.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.811+0000 7f78faf64740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55914f7d7450 con 0x55914f8d6940 2024-04-24T09:15:04.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.811+0000 7f78faf64740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55914f82d190 con 0x55914f8cc270 2024-04-24T09:15:04.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.811+0000 7f78ee000640 1 -- 172.21.15.83:0/2880577173 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f78e4018ac0 con 0x7f78f0076530 2024-04-24T09:15:04.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.811+0000 7f78f6200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55914f8cb950 0x55914f8cbd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:04.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.811+0000 7f78f6200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55914f8cb950 0x55914f8cbd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54896/0 (socket says 172.21.15.83:54896) 2024-04-24T09:15:04.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.811+0000 7f78f6200640 1 -- 172.21.15.83:0/1394653171 learned_addr learned my addr 172.21.15.83:0/1394653171 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:15:04.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.811+0000 7f78f5800640 1 --2- 172.21.15.83:0/1394653171 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55914f8cc270 0x55914f8d6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:04.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.812+0000 7f78f4e00640 1 --2- 172.21.15.83:0/1394653171 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55914f8d6940 0x55914f8d8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:04.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.812+0000 7f78f6200640 1 -- 172.21.15.83:0/1394653171 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55914f8cc270 msgr2=0x55914f8d6400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:04.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.812+0000 7f78f6200640 1 --2- 172.21.15.83:0/1394653171 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55914f8cc270 0x55914f8d6400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:04.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.812+0000 7f78f6200640 1 -- 172.21.15.83:0/1394653171 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55914f8d6940 msgr2=0x55914f8d8d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:04.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.812+0000 7f78f6200640 1 --2- 172.21.15.83:0/1394653171 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55914f8d6940 0x55914f8d8d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:04.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.812+0000 7f78f6200640 1 -- 172.21.15.83:0/1394653171 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55914f894310 con 0x55914f8cb950 2024-04-24T09:15:04.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.812+0000 7f78f4e00640 1 --2- 172.21.15.83:0/1394653171 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55914f8d6940 0x55914f8d8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:15:04.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.813+0000 7f78ee000640 1 --2- 172.21.15.83:0/2880577173 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f78d80a3e20 0x7f78d80a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:04.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.813+0000 7f78ee000640 1 -- 172.21.15.83:0/2880577173 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9583+0+0 (secure 0 0 0) 0x7f78e40d0f80 con 0x7f78f0076530 2024-04-24T09:15:04.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.813+0000 7f78f5800640 1 --2- 172.21.15.83:0/2880577173 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f78d80a3e20 0x7f78d80a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:04.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.813+0000 7f78f6200640 1 --2- 172.21.15.83:0/1394653171 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55914f8cb950 0x55914f8cbd30 secure :-1 s=READY pgs=2698 cs=0 l=1 rev1=1 crypto rx=0x7f78e8034040 tx=0x7f78e8030630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3643a6364d9caa37 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:04.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.814+0000 7f78e2a00640 1 -- 172.21.15.83:0/1394653171 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78e8041020 con 0x55914f8cb950 2024-04-24T09:15:04.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.814+0000 7f78e2a00640 1 -- 172.21.15.83:0/1394653171 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f78e8037040 con 0x55914f8cb950 2024-04-24T09:15:04.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.814+0000 7f78e2a00640 1 -- 172.21.15.83:0/1394653171 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78e8041020 con 0x55914f8cb950 2024-04-24T09:15:04.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.814+0000 7f78f5800640 1 --2- 172.21.15.83:0/2880577173 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f78d80a3e20 0x7f78d80a62e0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x55914f8ccf10 tx=0x7f78e4009310 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:04.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.815+0000 7f78faf64740 1 -- 172.21.15.83:0/1394653171 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55914f8cb950 msgr2=0x55914f8cbd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:04.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.815+0000 7f78faf64740 1 --2- 172.21.15.83:0/1394653171 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55914f8cb950 0x55914f8cbd30 secure :-1 s=READY pgs=2698 cs=0 l=1 rev1=1 crypto rx=0x7f78e8034040 tx=0x7f78e8030630 comp rx=0 tx=0).stop 2024-04-24T09:15:04.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.815+0000 7f78faf64740 1 -- 172.21.15.83:0/1394653171 shutdown_connections 2024-04-24T09:15:04.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.815+0000 7f78faf64740 1 --2- 172.21.15.83:0/1394653171 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55914f8d6940 0x55914f8d8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:04.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.815+0000 7f78faf64740 1 --2- 172.21.15.83:0/1394653171 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55914f8cc270 0x55914f8d6400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:04.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.815+0000 7f78faf64740 1 --2- 172.21.15.83:0/1394653171 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55914f8cb950 0x55914f8cbd30 unknown :-1 s=CLOSED pgs=2698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:04.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.815+0000 7f78faf64740 1 -- 172.21.15.83:0/1394653171 >> 172.21.15.83:0/1394653171 conn(0x55914f8c7c70 msgr2=0x55914f8c8050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:04.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.816+0000 7f78faf64740 1 -- 172.21.15.83:0/1394653171 shutdown_connections 2024-04-24T09:15:04.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.816+0000 7f78faf64740 1 -- 172.21.15.83:0/1394653171 wait complete. 2024-04-24T09:15:04.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.816+0000 7f78faf64740 1 -- start start 2024-04-24T09:15:04.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.816+0000 7f78faf64740 1 Processor -- start 2024-04-24T09:15:04.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.817+0000 7f78faf64740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55914f8cb950 0x55914f8d6370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:04.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.817+0000 7f78faf64740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55914f8cc270 0x55914f8d3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:04.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.817+0000 7f78faf64740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55914f8d6940 0x55914f8d3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:04.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.817+0000 7f78faf64740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55914f8d9770 con 0x55914f8cc270 2024-04-24T09:15:04.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.817+0000 7f78faf64740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55914f7ea810 con 0x55914f8d6940 2024-04-24T09:15:04.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.817+0000 7f78faf64740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55914f8d98f0 con 0x55914f8cb950 2024-04-24T09:15:04.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.817+0000 7f78faf64740 10 client.0 _ll_register_callbacks cb 0x55914f7b8140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:15:04.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.818+0000 7f78f4e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55914f8cc270 0x55914f8d3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:04.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.818+0000 7f78f4e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55914f8cc270 0x55914f8d3660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54910/0 (socket says 172.21.15.83:54910) 2024-04-24T09:15:04.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.818+0000 7f78f6200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55914f8cb950 0x55914f8d6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:04.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.818+0000 7f78f4e00640 1 -- 172.21.15.83:0/2336294412 learned_addr learned my addr 172.21.15.83:0/2336294412 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:15:04.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.818+0000 7f78f6200640 1 --2- 172.21.15.83:0/2336294412 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55914f8d6940 0x55914f8d3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:04.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.818+0000 7f78f4e00640 1 -- 172.21.15.83:0/2336294412 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55914f8cb950 msgr2=0x55914f8d6370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:04.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.818+0000 7f78f4e00640 1 --2- 172.21.15.83:0/2336294412 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55914f8cb950 0x55914f8d6370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:04.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.818+0000 7f78f4e00640 1 -- 172.21.15.83:0/2336294412 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55914f8d6940 msgr2=0x55914f8d3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:04.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.818+0000 7f78f4e00640 1 --2- 172.21.15.83:0/2336294412 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55914f8d6940 0x55914f8d3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:04.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.818+0000 7f78f4e00640 1 -- 172.21.15.83:0/2336294412 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55914f7fb3f0 con 0x55914f8cc270 2024-04-24T09:15:04.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.818+0000 7f78f6200640 1 --2- 172.21.15.83:0/2336294412 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55914f8d6940 0x55914f8d3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:15:04.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.819+0000 7f78f4e00640 1 --2- 172.21.15.83:0/2336294412 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55914f8cc270 0x55914f8d3660 secure :-1 s=READY pgs=2699 cs=0 l=1 rev1=1 crypto rx=0x7f78dc009930 tx=0x7f78dc009df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=448cc30cd8811406 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:04.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.819+0000 7f78d7400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:15:04.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.820+0000 7f78d7400640 1 -- 172.21.15.83:0/2336294412 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78dc01b470 con 0x55914f8cc270 2024-04-24T09:15:04.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.820+0000 7f78d7400640 1 -- 172.21.15.83:0/2336294412 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(128..128 src has 1..128) ==== 9583+0+0 (secure 0 0 0) 0x7f78dc012450 con 0x55914f8cc270 2024-04-24T09:15:04.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.820+0000 7f78faf64740 1 -- 172.21.15.83:0/2336294412 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78e8037040 con 0x55914f8cc270 2024-04-24T09:15:04.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.820+0000 7f78d7400640 1 -- 172.21.15.83:0/2336294412 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7f78dc012690 con 0x55914f8cc270 2024-04-24T09:15:04.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.820+0000 7f78faf64740 10 client.23508 fetch_fsmap learned FSMap version 68 2024-04-24T09:15:04.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.820+0000 7f78faf64740 1 -- 172.21.15.83:0/2336294412 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) -- 0x55914f5433f0 con 0x55914f8cc270 2024-04-24T09:15:04.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.821+0000 7f78d7400640 1 -- 172.21.15.83:0/2336294412 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f78e8037040 con 0x55914f8cc270 2024-04-24T09:15:04.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.821+0000 7f78d7400640 1 -- 172.21.15.83:0/2336294412 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 68) ==== 8374+0+0 (secure 0 0 0) 0x7f78dc01bb10 con 0x55914f8cc270 2024-04-24T09:15:04.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.821+0000 7f78faf64740 10 client.23508 fetch_fsmap finished waiting for FSMap version 68 2024-04-24T09:15:04.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.821+0000 7f78faf64740 10 client.23508 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-24T09:15:04.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.821+0000 7f78faf64740 20 client.23508 populate_metadata read hostname 'smithi083' 2024-04-24T09:15:04.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.821+0000 7f78faf64740 1 --2- 172.21.15.83:0/2336294412 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55914f8e69e0 0x55914f8e8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:04.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.821+0000 7f78faf64740 4 client.23508 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:15:04.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.821+0000 7f78faf64740 1 -- 172.21.15.83:0/2336294412 --> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55914f83ebf0 con 0x55914f8e69e0 2024-04-24T09:15:04.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.822+0000 7f78f6200640 1 --2- 172.21.15.83:0/2336294412 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55914f8e69e0 0x55914f8e8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:04.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.823+0000 7f78f6200640 1 --2- 172.21.15.83:0/2336294412 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55914f8e69e0 0x55914f8e8e00 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4393d2a1ad84d319 server_cookie=10a03b624cb4e4ae in_seq=0 out_seq=0 2024-04-24T09:15:04.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.823+0000 7f78d7400640 10 client.23508 ms_handle_connect on v2:172.21.15.138:6832/1157967658 2024-04-24T09:15:04.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.823+0000 7f78d7400640 1 -- 172.21.15.83:0/2336294412 <== mds.3 v2:172.21.15.138:6832/1157967658 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55914f83ebf0 con 0x55914f8e69e0 2024-04-24T09:15:04.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.824+0000 7f78d7400640 10 client.23508 handle_command_reply: tid=0 2024-04-24T09:15:04.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.824+0000 7f78faf64740 1 client.23508 shutdown 2024-04-24T09:15:04.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.824+0000 7f78faf64740 1 -- 172.21.15.83:0/2336294412 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55914f8cc270 msgr2=0x55914f8d3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:04.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.824+0000 7f78faf64740 1 --2- 172.21.15.83:0/2336294412 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55914f8cc270 0x55914f8d3660 secure :-1 s=READY pgs=2699 cs=0 l=1 rev1=1 crypto rx=0x7f78dc009930 tx=0x7f78dc009df0 comp rx=0 tx=0).stop 2024-04-24T09:15:04.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.824+0000 7f78faf64740 1 -- 172.21.15.83:0/2336294412 shutdown_connections 2024-04-24T09:15:04.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.824+0000 7f78faf64740 1 --2- 172.21.15.83:0/2336294412 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55914f8d6940 0x55914f8d3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:04.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.824+0000 7f78faf64740 1 --2- 172.21.15.83:0/2336294412 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55914f8cc270 0x55914f8d3660 unknown :-1 s=CLOSED pgs=2699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:04.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.824+0000 7f78faf64740 1 --2- 172.21.15.83:0/2336294412 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55914f8e69e0 0x55914f8e8e00 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:04.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.824+0000 7f78faf64740 1 --2- 172.21.15.83:0/2336294412 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55914f8cb950 0x55914f8d6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:04.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.824+0000 7f78d7400640 0 client.23508 ms_handle_reset on v2:172.21.15.138:6832/1157967658 2024-04-24T09:15:04.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.824+0000 7f78faf64740 1 -- 172.21.15.83:0/2336294412 >> 172.21.15.83:0/2336294412 conn(0x55914f8c7c70 msgr2=0x55914f8cd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:04.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.824+0000 7f78faf64740 1 -- 172.21.15.83:0/2336294412 shutdown_connections 2024-04-24T09:15:04.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.825+0000 7f78faf64740 1 -- 172.21.15.83:0/2336294412 wait complete. 2024-04-24T09:15:04.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.825+0000 7f78faf64740 20 client.23508 trim_cache size 0 max 16384 2024-04-24T09:15:04.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.847+0000 7f78faf64740 1 Processor -- start 2024-04-24T09:15:04.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.847+0000 7f78faf64740 1 -- start start 2024-04-24T09:15:04.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.847+0000 7f78faf64740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55914f8cb950 0x55914f8e26c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:04.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.847+0000 7f78faf64740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55914f8cc270 0x55914f8d5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:04.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.847+0000 7f78faf64740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55914f8d6940 0x55914f8d63e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:04.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.847+0000 7f78faf64740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f78d0010100 con 0x55914f8d6940 2024-04-24T09:15:04.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.847+0000 7f78faf64740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f78d0011cd0 con 0x55914f8cb950 2024-04-24T09:15:04.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.847+0000 7f78faf64740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f78d00107d0 con 0x55914f8cc270 2024-04-24T09:15:04.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.847+0000 7f78f6200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55914f8cb950 0x55914f8e26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:04.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.848+0000 7f78f6200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55914f8cb950 0x55914f8e26c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:37218/0 (socket says 172.21.15.83:37218) 2024-04-24T09:15:04.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.848+0000 7f78f6200640 1 -- 172.21.15.83:0/2932737857 learned_addr learned my addr 172.21.15.83:0/2932737857 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:15:04.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.848+0000 7f78f6200640 1 --2- 172.21.15.83:0/2932737857 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55914f8d6940 0x55914f8d63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:04.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.848+0000 7f78f4e00640 1 --2- 172.21.15.83:0/2932737857 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55914f8cc270 0x55914f8d5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:04.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.848+0000 7f78f6200640 1 -- 172.21.15.83:0/2932737857 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55914f8cc270 msgr2=0x55914f8d5ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:04.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.848+0000 7f78f6200640 1 --2- 172.21.15.83:0/2932737857 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55914f8cc270 0x55914f8d5ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:04.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.848+0000 7f78f6200640 1 -- 172.21.15.83:0/2932737857 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55914f8d6940 msgr2=0x55914f8d63e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:04.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.848+0000 7f78f6200640 1 --2- 172.21.15.83:0/2932737857 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55914f8d6940 0x55914f8d63e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:04.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.848+0000 7f78f6200640 1 -- 172.21.15.83:0/2932737857 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55914f8d4110 con 0x55914f8cb950 2024-04-24T09:15:04.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.849+0000 7f78f6200640 1 --2- 172.21.15.83:0/2932737857 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55914f8cb950 0x55914f8e26c0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f78e802acd0 tx=0x7f78e803f040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f765a65287873c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:04.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.849+0000 7f78e2000640 1 -- 172.21.15.83:0/2932737857 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78e8041070 con 0x55914f8cb950 2024-04-24T09:15:04.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.849+0000 7f78e2000640 1 -- 172.21.15.83:0/2932737857 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f78e8030920 con 0x55914f8cb950 2024-04-24T09:15:04.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.849+0000 7f78e2000640 1 -- 172.21.15.83:0/2932737857 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78e8036070 con 0x55914f8cb950 2024-04-24T09:15:04.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.850+0000 7f78faf64740 1 -- 172.21.15.83:0/2932737857 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55914f8cb950 msgr2=0x55914f8e26c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:04.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.850+0000 7f78faf64740 1 --2- 172.21.15.83:0/2932737857 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55914f8cb950 0x55914f8e26c0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f78e802acd0 tx=0x7f78e803f040 comp rx=0 tx=0).stop 2024-04-24T09:15:04.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.850+0000 7f78faf64740 1 -- 172.21.15.83:0/2932737857 shutdown_connections 2024-04-24T09:15:04.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.850+0000 7f78faf64740 1 --2- 172.21.15.83:0/2932737857 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55914f8d6940 0x55914f8d63e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:04.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.850+0000 7f78faf64740 1 --2- 172.21.15.83:0/2932737857 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55914f8cc270 0x55914f8d5ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:04.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.850+0000 7f78faf64740 1 --2- 172.21.15.83:0/2932737857 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55914f8cb950 0x55914f8e26c0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:04.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.850+0000 7f78faf64740 1 -- 172.21.15.83:0/2932737857 >> 172.21.15.83:0/2932737857 conn(0x55914f8c7c70 msgr2=0x55914f8cd0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:04.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.850+0000 7f78faf64740 1 -- 172.21.15.83:0/2932737857 shutdown_connections 2024-04-24T09:15:04.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.850+0000 7f78faf64740 1 -- 172.21.15.83:0/2932737857 wait complete. 2024-04-24T09:15:04.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.850+0000 7f78faf64740 1 -- start start 2024-04-24T09:15:04.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.851+0000 7f78faf64740 1 Processor -- start 2024-04-24T09:15:04.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.851+0000 7f78faf64740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55914f8cb950 0x55914f8d74d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:04.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.851+0000 7f78faf64740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55914f8cc270 0x55914f8d7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:04.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.851+0000 7f78faf64740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55914f8d6940 0x55914f8ddbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:04.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.851+0000 7f78faf64740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f78d00115d0 con 0x55914f8cb950 2024-04-24T09:15:04.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.851+0000 7f78faf64740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f78d0010f00 con 0x55914f8cc270 2024-04-24T09:15:04.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.851+0000 7f78faf64740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f78d000e100 con 0x55914f8d6940 2024-04-24T09:15:04.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.851+0000 7f78faf64740 10 client.0 _ll_register_callbacks cb 0x7f78dc004370 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:15:04.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.851+0000 7f78f4e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55914f8cc270 0x55914f8d7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:04.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.851+0000 7f78f4e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55914f8cc270 0x55914f8d7a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:37222/0 (socket says 172.21.15.83:37222) 2024-04-24T09:15:04.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.851+0000 7f78f4e00640 1 -- 172.21.15.83:0/611587062 learned_addr learned my addr 172.21.15.83:0/611587062 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:15:04.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.852+0000 7f78f6200640 1 --2- 172.21.15.83:0/611587062 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55914f8cb950 0x55914f8d74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:04.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.852+0000 7f78f6200640 1 --2- 172.21.15.83:0/611587062 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55914f8d6940 0x55914f8ddbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:04.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.852+0000 7f78f4e00640 1 -- 172.21.15.83:0/611587062 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55914f8d6940 msgr2=0x55914f8ddbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:04.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.852+0000 7f78f4e00640 1 --2- 172.21.15.83:0/611587062 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55914f8d6940 0x55914f8ddbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:04.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.852+0000 7f78f4e00640 1 -- 172.21.15.83:0/611587062 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55914f8cb950 msgr2=0x55914f8d74d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:04.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.852+0000 7f78f4e00640 1 --2- 172.21.15.83:0/611587062 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55914f8cb950 0x55914f8d74d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:04.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.852+0000 7f78f4e00640 1 -- 172.21.15.83:0/611587062 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55914f60d7d0 con 0x55914f8cc270 2024-04-24T09:15:04.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.852+0000 7f78f4e00640 1 --2- 172.21.15.83:0/611587062 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55914f8cc270 0x55914f8d7a10 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f78dc000c00 tx=0x7f78dc0039c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3cfb05e194931b2e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:04.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.852+0000 7f78d7400640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:15:04.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.853+0000 7f78d7400640 1 -- 172.21.15.83:0/611587062 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78dc012a40 con 0x55914f8cc270 2024-04-24T09:15:04.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.853+0000 7f78faf64740 1 -- 172.21.15.83:0/611587062 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78e8030920 con 0x55914f8cc270 2024-04-24T09:15:04.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.853+0000 7f78d7400640 1 -- 172.21.15.83:0/611587062 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(128..128 src has 1..128) ==== 9583+0+0 (secure 0 0 0) 0x7f78dc012be0 con 0x55914f8cc270 2024-04-24T09:15:04.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.854+0000 7f78d7400640 1 -- 172.21.15.83:0/611587062 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7f78dc03b050 con 0x55914f8cc270 2024-04-24T09:15:04.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.854+0000 7f78faf64740 10 client.40574 fetch_fsmap learned FSMap version 68 2024-04-24T09:15:04.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.854+0000 7f78faf64740 1 -- 172.21.15.83:0/611587062 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) -- 0x55914f8df2b0 con 0x55914f8cc270 2024-04-24T09:15:04.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.854+0000 7f78d7400640 1 -- 172.21.15.83:0/611587062 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f78e8030920 con 0x55914f8cc270 2024-04-24T09:15:04.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.855+0000 7f78d7400640 1 -- 172.21.15.83:0/611587062 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 68) ==== 8374+0+0 (secure 0 0 0) 0x7f78dc011570 con 0x55914f8cc270 2024-04-24T09:15:04.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.855+0000 7f78faf64740 10 client.40574 fetch_fsmap finished waiting for FSMap version 68 2024-04-24T09:15:04.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.855+0000 7f78faf64740 10 client.40574 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-24T09:15:04.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.855+0000 7f78faf64740 20 client.40574 populate_metadata read hostname 'smithi083' 2024-04-24T09:15:04.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.855+0000 7f78faf64740 1 --2- 172.21.15.83:0/611587062 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55914f94d3a0 0x55914f94f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:04.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.855+0000 7f78faf64740 4 client.40574 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:15:04.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.855+0000 7f78faf64740 1 -- 172.21.15.83:0/611587062 --> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55914f838a60 con 0x55914f94d3a0 2024-04-24T09:15:04.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.855+0000 7f78f6200640 1 --2- 172.21.15.83:0/611587062 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55914f94d3a0 0x55914f94f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:04.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.856+0000 7f78d7400640 10 client.40574 ms_handle_connect on v2:172.21.15.138:6832/1157967658 2024-04-24T09:15:04.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.856+0000 7f78f6200640 1 --2- 172.21.15.83:0/611587062 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55914f94d3a0 0x55914f94f7c0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a6bf11eef96b3451 server_cookie=200de2fcdf2b76aa in_seq=0 out_seq=0 2024-04-24T09:15:04.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.856+0000 7f78d7400640 1 -- 172.21.15.83:0/611587062 <== mds.3 v2:172.21.15.138:6832/1157967658 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55914f838a60 con 0x55914f94d3a0 2024-04-24T09:15:04.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.856+0000 7f78d7400640 10 client.40574 handle_command_reply: tid=0 2024-04-24T09:15:04.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.856+0000 7f78faf64740 1 client.40574 shutdown 2024-04-24T09:15:04.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.857+0000 7f78faf64740 1 -- 172.21.15.83:0/611587062 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55914f8cc270 msgr2=0x55914f8d7a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:04.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.857+0000 7f78faf64740 1 --2- 172.21.15.83:0/611587062 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55914f8cc270 0x55914f8d7a10 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f78dc000c00 tx=0x7f78dc0039c0 comp rx=0 tx=0).stop 2024-04-24T09:15:04.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.857+0000 7f78faf64740 1 -- 172.21.15.83:0/611587062 shutdown_connections 2024-04-24T09:15:04.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.857+0000 7f78faf64740 1 --2- 172.21.15.83:0/611587062 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55914f94d3a0 0x55914f94f7c0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:04.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.857+0000 7f78faf64740 1 --2- 172.21.15.83:0/611587062 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55914f8d6940 0x55914f8ddbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:04.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.857+0000 7f78faf64740 1 --2- 172.21.15.83:0/611587062 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55914f8cc270 0x55914f8d7a10 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:04.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.857+0000 7f78faf64740 1 --2- 172.21.15.83:0/611587062 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55914f8cb950 0x55914f8d74d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:04.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.857+0000 7f78d7400640 0 client.40574 ms_handle_reset on v2:172.21.15.138:6832/1157967658 2024-04-24T09:15:04.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.857+0000 7f78faf64740 1 -- 172.21.15.83:0/611587062 >> 172.21.15.83:0/611587062 conn(0x55914f8c7c70 msgr2=0x55914f8cd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:04.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.857+0000 7f78faf64740 1 -- 172.21.15.83:0/611587062 shutdown_connections 2024-04-24T09:15:04.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.857+0000 7f78faf64740 1 -- 172.21.15.83:0/611587062 wait complete. 2024-04-24T09:15:04.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.857+0000 7f78faf64740 20 client.40574 trim_cache size 0 max 16384 2024-04-24T09:15:04.858 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:15:04.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.858+0000 7f78f8200640 1 -- 172.21.15.83:0/2880577173 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f78d80a3e20 msgr2=0x7f78d80a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:04.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.858+0000 7f78f8200640 1 --2- 172.21.15.83:0/2880577173 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f78d80a3e20 0x7f78d80a62e0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x55914f8ccf10 tx=0x7f78e4009310 comp rx=0 tx=0).stop 2024-04-24T09:15:04.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.858+0000 7f78f8200640 1 -- 172.21.15.83:0/2880577173 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f78f0076530 msgr2=0x7f78f00738d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:04.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.858+0000 7f78f8200640 1 --2- 172.21.15.83:0/2880577173 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f78f0076530 0x7f78f00738d0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f78e400e9f0 tx=0x7f78e400eeb0 comp rx=0 tx=0).stop 2024-04-24T09:15:04.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.858+0000 7f78f8200640 1 -- 172.21.15.83:0/2880577173 shutdown_connections 2024-04-24T09:15:04.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.858+0000 7f78f8200640 1 --2- 172.21.15.83:0/2880577173 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f78d80a3e20 0x7f78d80a62e0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:04.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.858+0000 7f78f8200640 1 --2- 172.21.15.83:0/2880577173 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78f0112ee0 0x7f78f006ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:04.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.858+0000 7f78f8200640 1 --2- 172.21.15.83:0/2880577173 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f78f0076e50 0x7f78f006c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:04.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.858+0000 7f78f8200640 1 --2- 172.21.15.83:0/2880577173 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f78f0076530 0x7f78f00738d0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:04.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.858+0000 7f78f8200640 1 -- 172.21.15.83:0/2880577173 >> 172.21.15.83:0/2880577173 conn(0x7f78f00ffd90 msgr2=0x7f78f0102180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:04.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.859+0000 7f78f8200640 1 -- 172.21.15.83:0/2880577173 shutdown_connections 2024-04-24T09:15:04.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:04.859+0000 7f78f8200640 1 -- 172.21.15.83:0/2880577173 wait complete. 2024-04-24T09:15:04.879 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T09:15:05.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.137+0000 7fc8a2800640 1 -- 172.21.15.83:0/3714873845 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc89c104e60 msgr2=0x7fc89c108ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:05.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.137+0000 7fc8a2800640 1 --2- 172.21.15.83:0/3714873845 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc89c104e60 0x7fc89c108ea0 secure :-1 s=READY pgs=2700 cs=0 l=1 rev1=1 crypto rx=0x7fc888009860 tx=0x7fc88802a340 comp rx=0 tx=0).stop 2024-04-24T09:15:05.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.138+0000 7fc8a2800640 1 -- 172.21.15.83:0/3714873845 shutdown_connections 2024-04-24T09:15:05.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.138+0000 7fc8a2800640 1 --2- 172.21.15.83:0/3714873845 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc89c109580 0x7fc89c111150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:05.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.138+0000 7fc8a2800640 1 --2- 172.21.15.83:0/3714873845 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc89c104e60 0x7fc89c108ea0 unknown :-1 s=CLOSED pgs=2700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:05.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.138+0000 7fc8a2800640 1 --2- 172.21.15.83:0/3714873845 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc89c1044b0 0x7fc89c104890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:05.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.138+0000 7fc8a2800640 1 -- 172.21.15.83:0/3714873845 >> 172.21.15.83:0/3714873845 conn(0x7fc89c0ffd90 msgr2=0x7fc89c1021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:05.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.138+0000 7fc8a2800640 1 -- 172.21.15.83:0/3714873845 shutdown_connections 2024-04-24T09:15:05.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.138+0000 7fc8a2800640 1 -- 172.21.15.83:0/3714873845 wait complete. 2024-04-24T09:15:05.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.139+0000 7fc8a2800640 1 Processor -- start 2024-04-24T09:15:05.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.139+0000 7fc8a2800640 1 -- start start 2024-04-24T09:15:05.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.140+0000 7fc8a2800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc89c1044b0 0x7fc89c076510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:05.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.140+0000 7fc8a2800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc89c109580 0x7fc89c076a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:05.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.140+0000 7fc8a2800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc89c076f90 0x7fc89c1ab120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:05.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.140+0000 7fc8a2800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc89c1144a0 con 0x7fc89c076f90 2024-04-24T09:15:05.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.140+0000 7fc8a2800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc89c114320 con 0x7fc89c1044b0 2024-04-24T09:15:05.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.140+0000 7fc8a2800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc89c114620 con 0x7fc89c109580 2024-04-24T09:15:05.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.140+0000 7fc89b400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc89c1044b0 0x7fc89c076510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:05.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.140+0000 7fc89be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc89c076f90 0x7fc89c1ab120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:05.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.140+0000 7fc89b400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc89c1044b0 0x7fc89c076510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:37248/0 (socket says 172.21.15.83:37248) 2024-04-24T09:15:05.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.140+0000 7fc89b400640 1 -- 172.21.15.83:0/1117302128 learned_addr learned my addr 172.21.15.83:0/1117302128 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:15:05.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.141+0000 7fc89aa00640 1 --2- 172.21.15.83:0/1117302128 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc89c109580 0x7fc89c076a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:05.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.141+0000 7fc89be00640 1 -- 172.21.15.83:0/1117302128 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc89c109580 msgr2=0x7fc89c076a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:05.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.141+0000 7fc89be00640 1 --2- 172.21.15.83:0/1117302128 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc89c109580 0x7fc89c076a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:05.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.141+0000 7fc89be00640 1 -- 172.21.15.83:0/1117302128 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc89c1044b0 msgr2=0x7fc89c076510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:05.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.141+0000 7fc89be00640 1 --2- 172.21.15.83:0/1117302128 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc89c1044b0 0x7fc89c076510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:05.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.141+0000 7fc89be00640 1 -- 172.21.15.83:0/1117302128 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc89c1ab660 con 0x7fc89c076f90 2024-04-24T09:15:05.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.141+0000 7fc89b400640 1 --2- 172.21.15.83:0/1117302128 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc89c1044b0 0x7fc89c076510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:15:05.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.141+0000 7fc89be00640 1 --2- 172.21.15.83:0/1117302128 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc89c076f90 0x7fc89c1ab120 secure :-1 s=READY pgs=2701 cs=0 l=1 rev1=1 crypto rx=0x7fc88c00c710 tx=0x7fc88c00cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:05.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.142+0000 7fc893e00640 1 -- 172.21.15.83:0/1117302128 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc88c004220 con 0x7fc89c076f90 2024-04-24T09:15:05.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.142+0000 7fc893e00640 1 -- 172.21.15.83:0/1117302128 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc88c00ce20 con 0x7fc89c076f90 2024-04-24T09:15:05.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.142+0000 7fc893e00640 1 -- 172.21.15.83:0/1117302128 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc88c003db0 con 0x7fc89c076f90 2024-04-24T09:15:05.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.142+0000 7fc8a2800640 1 -- 172.21.15.83:0/1117302128 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc89c1ab8f0 con 0x7fc89c076f90 2024-04-24T09:15:05.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.143+0000 7fc8a2800640 1 -- 172.21.15.83:0/1117302128 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc89c1abdc0 con 0x7fc89c076f90 2024-04-24T09:15:05.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.147+0000 7fc893e00640 1 -- 172.21.15.83:0/1117302128 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc88c0079b0 con 0x7fc89c076f90 2024-04-24T09:15:05.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.147+0000 7fc8a5631740 1 Processor -- start 2024-04-24T09:15:05.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.148+0000 7fc8a5631740 1 -- start start 2024-04-24T09:15:05.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.148+0000 7fc8a5631740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5605da740950 0x5605da740d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:05.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.148+0000 7fc8a5631740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5605da741270 0x5605da74b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:05.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.148+0000 7fc8a5631740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605da74b940 0x5605da74dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:05.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.148+0000 7fc8a5631740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5605da6a61d0 con 0x5605da740950 2024-04-24T09:15:05.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.148+0000 7fc8a5631740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5605da64c450 con 0x5605da74b940 2024-04-24T09:15:05.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.148+0000 7fc8a5631740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5605da6a2190 con 0x5605da741270 2024-04-24T09:15:05.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.149+0000 7fc89aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5605da740950 0x5605da740d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:05.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.149+0000 7fc89b400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5605da741270 0x5605da74b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:05.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.149+0000 7fc89aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5605da740950 0x5605da740d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54976/0 (socket says 172.21.15.83:54976) 2024-04-24T09:15:05.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.149+0000 7fc89aa00640 1 -- 172.21.15.83:0/1408936576 learned_addr learned my addr 172.21.15.83:0/1408936576 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:15:05.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.149+0000 7fc89aa00640 1 --2- 172.21.15.83:0/1408936576 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605da74b940 0x5605da74dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:05.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.149+0000 7fc893e00640 1 --2- 172.21.15.83:0/1117302128 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fc8800a3da0 0x7fc8800a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:05.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.149+0000 7fc89aa00640 1 -- 172.21.15.83:0/1408936576 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5605da741270 msgr2=0x5605da74b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:05.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.149+0000 7fc89aa00640 1 --2- 172.21.15.83:0/1408936576 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5605da741270 0x5605da74b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:05.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.149+0000 7fc89aa00640 1 -- 172.21.15.83:0/1408936576 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605da74b940 msgr2=0x5605da74dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:05.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.149+0000 7fc89aa00640 1 --2- 172.21.15.83:0/1408936576 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605da74b940 0x5605da74dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:05.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.149+0000 7fc89aa00640 1 -- 172.21.15.83:0/1408936576 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605da709310 con 0x5605da740950 2024-04-24T09:15:05.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.149+0000 7fc89be00640 1 --2- 172.21.15.83:0/1117302128 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fc8800a3da0 0x7fc8800a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:05.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.150+0000 7fc89aa00640 1 --2- 172.21.15.83:0/1408936576 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5605da740950 0x5605da740d30 secure :-1 s=READY pgs=2702 cs=0 l=1 rev1=1 crypto rx=0x7fc88802a930 tx=0x7fc8880094b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86b967f151cab466 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:05.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.150+0000 7fc893e00640 1 -- 172.21.15.83:0/1117302128 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9583+0+0 (secure 0 0 0) 0x7fc88c0d04c0 con 0x7fc89c076f90 2024-04-24T09:15:05.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.150+0000 7fc890c00640 1 -- 172.21.15.83:0/1408936576 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc888046070 con 0x5605da740950 2024-04-24T09:15:05.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.150+0000 7fc890c00640 1 -- 172.21.15.83:0/1408936576 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc888030900 con 0x5605da740950 2024-04-24T09:15:05.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.150+0000 7fc890c00640 1 -- 172.21.15.83:0/1408936576 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc888037040 con 0x5605da740950 2024-04-24T09:15:05.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.151+0000 7fc8a5631740 1 -- 172.21.15.83:0/1408936576 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5605da740950 msgr2=0x5605da740d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:05.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.151+0000 7fc8a5631740 1 --2- 172.21.15.83:0/1408936576 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5605da740950 0x5605da740d30 secure :-1 s=READY pgs=2702 cs=0 l=1 rev1=1 crypto rx=0x7fc88802a930 tx=0x7fc8880094b0 comp rx=0 tx=0).stop 2024-04-24T09:15:05.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.151+0000 7fc89be00640 1 --2- 172.21.15.83:0/1117302128 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fc8800a3da0 0x7fc8800a6260 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fc88c007fd0 tx=0x7fc88c0044e0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:05.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.151+0000 7fc8a5631740 1 -- 172.21.15.83:0/1408936576 shutdown_connections 2024-04-24T09:15:05.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.151+0000 7fc8a5631740 1 --2- 172.21.15.83:0/1408936576 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605da74b940 0x5605da74dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:05.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.151+0000 7fc8a5631740 1 --2- 172.21.15.83:0/1408936576 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5605da741270 0x5605da74b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:05.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.151+0000 7fc8a5631740 1 --2- 172.21.15.83:0/1408936576 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5605da740950 0x5605da740d30 unknown :-1 s=CLOSED pgs=2702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:05.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.151+0000 7fc8a5631740 1 -- 172.21.15.83:0/1408936576 >> 172.21.15.83:0/1408936576 conn(0x5605da73cc70 msgr2=0x5605da73d050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:05.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.152+0000 7fc8a5631740 1 -- 172.21.15.83:0/1408936576 shutdown_connections 2024-04-24T09:15:05.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.152+0000 7fc8a5631740 1 -- 172.21.15.83:0/1408936576 wait complete. 2024-04-24T09:15:05.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.152+0000 7fc8a5631740 1 -- start start 2024-04-24T09:15:05.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.152+0000 7fc8a5631740 1 Processor -- start 2024-04-24T09:15:05.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.153+0000 7fc8a5631740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605da740950 0x5605da748710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:05.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.153+0000 7fc8a5631740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5605da741270 0x5605da748c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:05.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.153+0000 7fc8a5631740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5605da74b940 0x5605da749190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:05.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.153+0000 7fc8a5631740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5605da74e770 con 0x5605da74b940 2024-04-24T09:15:05.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.153+0000 7fc8a5631740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5605da65f810 con 0x5605da740950 2024-04-24T09:15:05.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.153+0000 7fc8a5631740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5605da74e8f0 con 0x5605da741270 2024-04-24T09:15:05.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.153+0000 7fc8a5631740 10 client.0 _ll_register_callbacks cb 0x5605da62d140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:15:05.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.153+0000 7fc89b400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5605da741270 0x5605da748c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:05.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.153+0000 7fc89aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5605da74b940 0x5605da749190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:05.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.153+0000 7fc89b400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5605da741270 0x5605da748c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:60336/0 (socket says 172.21.15.83:60336) 2024-04-24T09:15:05.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.153+0000 7fc89b400640 1 -- 172.21.15.83:0/2516012605 learned_addr learned my addr 172.21.15.83:0/2516012605 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:15:05.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.153+0000 7fc89aa00640 1 --2- 172.21.15.83:0/2516012605 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605da740950 0x5605da748710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:05.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.154+0000 7fc89b400640 1 -- 172.21.15.83:0/2516012605 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605da740950 msgr2=0x5605da748710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:05.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.154+0000 7fc89b400640 1 --2- 172.21.15.83:0/2516012605 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605da740950 0x5605da748710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:05.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.154+0000 7fc89b400640 1 -- 172.21.15.83:0/2516012605 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5605da74b940 msgr2=0x5605da749190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:05.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.154+0000 7fc89b400640 1 --2- 172.21.15.83:0/2516012605 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5605da74b940 0x5605da749190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:05.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.154+0000 7fc89b400640 1 -- 172.21.15.83:0/2516012605 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605da6703f0 con 0x5605da741270 2024-04-24T09:15:05.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.154+0000 7fc89aa00640 1 --2- 172.21.15.83:0/2516012605 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5605da74b940 0x5605da749190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:15:05.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.155+0000 7fc89b400640 1 --2- 172.21.15.83:0/2516012605 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5605da741270 0x5605da748c50 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc89400d690 tx=0x7fc89400db50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9274d4da9eb3b86e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:05.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.155+0000 7fc885600640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-04-24T09:15:05.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.156+0000 7fc885600640 1 -- 172.21.15.83:0/2516012605 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc894003b70 con 0x5605da741270 2024-04-24T09:15:05.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.156+0000 7fc8a5631740 1 -- 172.21.15.83:0/2516012605 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc888030900 con 0x5605da741270 2024-04-24T09:15:05.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.156+0000 7fc885600640 1 -- 172.21.15.83:0/2516012605 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(128..128 src has 1..128) ==== 9583+0+0 (secure 0 0 0) 0x7fc894003d10 con 0x5605da741270 2024-04-24T09:15:05.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.156+0000 7fc885600640 1 -- 172.21.15.83:0/2516012605 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7fc894026050 con 0x5605da741270 2024-04-24T09:15:05.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.157+0000 7fc8a5631740 10 client.30310 fetch_fsmap learned FSMap version 68 2024-04-24T09:15:05.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.157+0000 7fc8a5631740 1 -- 172.21.15.83:0/2516012605 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) -- 0x5605da3b83f0 con 0x5605da741270 2024-04-24T09:15:05.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.157+0000 7fc885600640 1 -- 172.21.15.83:0/2516012605 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc888030900 con 0x5605da741270 2024-04-24T09:15:05.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.157+0000 7fc885600640 1 -- 172.21.15.83:0/2516012605 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 68) ==== 8374+0+0 (secure 0 0 0) 0x7fc89400b040 con 0x5605da741270 2024-04-24T09:15:05.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.158+0000 7fc8a5631740 10 client.30310 fetch_fsmap finished waiting for FSMap version 68 2024-04-24T09:15:05.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.158+0000 7fc8a5631740 10 client.30310 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-24T09:15:05.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.158+0000 7fc8a5631740 20 client.30310 populate_metadata read hostname 'smithi083' 2024-04-24T09:15:05.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.158+0000 7fc8a5631740 1 --2- 172.21.15.83:0/2516012605 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x5605da75bc10 0x5605da75e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:05.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.158+0000 7fc8a5631740 4 client.30310 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:15:05.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.158+0000 7fc8a5631740 1 -- 172.21.15.83:0/2516012605 --> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5605da6b3bf0 con 0x5605da75bc10 2024-04-24T09:15:05.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.158+0000 7fc89aa00640 1 --2- 172.21.15.83:0/2516012605 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x5605da75bc10 0x5605da75e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:05.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.159+0000 7fc885600640 10 client.30310 ms_handle_connect on v2:172.21.15.138:6834/1909219758 2024-04-24T09:15:05.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.159+0000 7fc89aa00640 1 --2- 172.21.15.83:0/2516012605 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x5605da75bc10 0x5605da75e050 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6e9a31b56003f8a9 server_cookie=90a12a6c474e28d in_seq=0 out_seq=0 2024-04-24T09:15:05.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.160+0000 7fc885600640 1 -- 172.21.15.83:0/2516012605 <== mds.2 v2:172.21.15.138:6834/1909219758 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5605da6b3bf0 con 0x5605da75bc10 2024-04-24T09:15:05.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.160+0000 7fc885600640 10 client.30310 handle_command_reply: tid=0 2024-04-24T09:15:05.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.160+0000 7fc8a5631740 1 client.30310 shutdown 2024-04-24T09:15:05.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.161+0000 7fc8a5631740 1 -- 172.21.15.83:0/2516012605 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5605da741270 msgr2=0x5605da748c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:05.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.161+0000 7fc8a5631740 1 --2- 172.21.15.83:0/2516012605 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5605da741270 0x5605da748c50 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc89400d690 tx=0x7fc89400db50 comp rx=0 tx=0).stop 2024-04-24T09:15:05.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.161+0000 7fc8a5631740 1 -- 172.21.15.83:0/2516012605 shutdown_connections 2024-04-24T09:15:05.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.161+0000 7fc8a5631740 1 --2- 172.21.15.83:0/2516012605 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x5605da75bc10 0x5605da75e050 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:05.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.161+0000 7fc8a5631740 1 --2- 172.21.15.83:0/2516012605 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5605da74b940 0x5605da749190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:05.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.161+0000 7fc8a5631740 1 --2- 172.21.15.83:0/2516012605 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5605da741270 0x5605da748c50 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:05.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.161+0000 7fc8a5631740 1 --2- 172.21.15.83:0/2516012605 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605da740950 0x5605da748710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:05.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.161+0000 7fc885600640 0 client.30310 ms_handle_reset on v2:172.21.15.138:6834/1909219758 2024-04-24T09:15:05.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.161+0000 7fc8a5631740 1 -- 172.21.15.83:0/2516012605 >> 172.21.15.83:0/2516012605 conn(0x5605da73cc70 msgr2=0x5605da742a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:05.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.161+0000 7fc8a5631740 1 -- 172.21.15.83:0/2516012605 shutdown_connections 2024-04-24T09:15:05.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.162+0000 7fc8a5631740 1 -- 172.21.15.83:0/2516012605 wait complete. 2024-04-24T09:15:05.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.162+0000 7fc8a5631740 20 client.30310 trim_cache size 0 max 16384 2024-04-24T09:15:05.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.187+0000 7fc8a5631740 1 Processor -- start 2024-04-24T09:15:05.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.187+0000 7fc8a5631740 1 -- start start 2024-04-24T09:15:05.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.187+0000 7fc8a5631740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605da740950 0x5605da757630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:05.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.187+0000 7fc8a5631740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5605da741270 0x5605da763960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:05.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.187+0000 7fc8a5631740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5605da74b940 0x5605da763ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:05.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.187+0000 7fc8a5631740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc89c1b5b70 con 0x5605da741270 2024-04-24T09:15:05.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.187+0000 7fc8a5631740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc89c1b7740 con 0x5605da740950 2024-04-24T09:15:05.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.187+0000 7fc8a5631740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc89c1b6240 con 0x5605da74b940 2024-04-24T09:15:05.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.188+0000 7fc89b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5605da741270 0x5605da763960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:05.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.188+0000 7fc89aa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605da740950 0x5605da757630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:05.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.188+0000 7fc89aa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605da740950 0x5605da757630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:37272/0 (socket says 172.21.15.83:37272) 2024-04-24T09:15:05.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.188+0000 7fc89aa00640 1 -- 172.21.15.83:0/1580829993 learned_addr learned my addr 172.21.15.83:0/1580829993 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:15:05.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.188+0000 7fc89b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5605da741270 0x5605da763960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54994/0 (socket says 172.21.15.83:54994) 2024-04-24T09:15:05.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.188+0000 7fc89aa00640 1 --2- 172.21.15.83:0/1580829993 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5605da74b940 0x5605da763ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:05.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.188+0000 7fc89aa00640 1 -- 172.21.15.83:0/1580829993 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5605da74b940 msgr2=0x5605da763ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:05.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.188+0000 7fc89aa00640 1 --2- 172.21.15.83:0/1580829993 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5605da74b940 0x5605da763ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:05.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.188+0000 7fc89aa00640 1 -- 172.21.15.83:0/1580829993 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5605da741270 msgr2=0x5605da763960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:05.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.188+0000 7fc89aa00640 1 --2- 172.21.15.83:0/1580829993 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5605da741270 0x5605da763960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:05.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.188+0000 7fc89aa00640 1 -- 172.21.15.83:0/1580829993 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605da4827d0 con 0x5605da740950 2024-04-24T09:15:05.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.189+0000 7fc89aa00640 1 --2- 172.21.15.83:0/1580829993 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605da740950 0x5605da757630 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fc888005380 tx=0x7fc888005310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af923dd695ed64c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:05.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.189+0000 7fc887e00640 1 -- 172.21.15.83:0/1580829993 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc888037040 con 0x5605da740950 2024-04-24T09:15:05.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.189+0000 7fc887e00640 1 -- 172.21.15.83:0/1580829993 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc888030900 con 0x5605da740950 2024-04-24T09:15:05.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.189+0000 7fc887e00640 1 -- 172.21.15.83:0/1580829993 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc888046040 con 0x5605da740950 2024-04-24T09:15:05.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.190+0000 7fc8a5631740 1 -- 172.21.15.83:0/1580829993 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605da740950 msgr2=0x5605da757630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:05.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.190+0000 7fc8a5631740 1 --2- 172.21.15.83:0/1580829993 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605da740950 0x5605da757630 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fc888005380 tx=0x7fc888005310 comp rx=0 tx=0).stop 2024-04-24T09:15:05.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.191+0000 7fc8a5631740 1 -- 172.21.15.83:0/1580829993 shutdown_connections 2024-04-24T09:15:05.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.191+0000 7fc8a5631740 1 --2- 172.21.15.83:0/1580829993 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5605da74b940 0x5605da763ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:05.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.191+0000 7fc8a5631740 1 --2- 172.21.15.83:0/1580829993 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5605da741270 0x5605da763960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:05.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.191+0000 7fc8a5631740 1 --2- 172.21.15.83:0/1580829993 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605da740950 0x5605da757630 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:05.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.191+0000 7fc8a5631740 1 -- 172.21.15.83:0/1580829993 >> 172.21.15.83:0/1580829993 conn(0x5605da73cc70 msgr2=0x5605da73b560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:05.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.191+0000 7fc8a5631740 1 -- 172.21.15.83:0/1580829993 shutdown_connections 2024-04-24T09:15:05.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.191+0000 7fc8a5631740 1 -- 172.21.15.83:0/1580829993 wait complete. 2024-04-24T09:15:05.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.191+0000 7fc8a5631740 1 -- start start 2024-04-24T09:15:05.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.192+0000 7fc8a5631740 1 Processor -- start 2024-04-24T09:15:05.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.192+0000 7fc8a5631740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5605da740950 0x5605da745190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:05.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.192+0000 7fc8a5631740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5605da741270 0x5605da7456d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:05.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.192+0000 7fc8a5631740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605da74b940 0x5605da752b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:05.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.192+0000 7fc8a5631740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc89c1b7040 con 0x5605da740950 2024-04-24T09:15:05.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.192+0000 7fc8a5631740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc89c1b6970 con 0x5605da74b940 2024-04-24T09:15:05.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.192+0000 7fc8a5631740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc89c1b3b70 con 0x5605da741270 2024-04-24T09:15:05.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.192+0000 7fc8a5631740 10 client.0 _ll_register_callbacks cb 0x5605da74b890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:15:05.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.193+0000 7fc89b400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5605da741270 0x5605da7456d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:05.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.193+0000 7fc89aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5605da740950 0x5605da745190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:05.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.193+0000 7fc89aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5605da740950 0x5605da745190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54998/0 (socket says 172.21.15.83:54998) 2024-04-24T09:15:05.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.193+0000 7fc89aa00640 1 -- 172.21.15.83:0/802016292 learned_addr learned my addr 172.21.15.83:0/802016292 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:15:05.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.193+0000 7fc89aa00640 1 --2- 172.21.15.83:0/802016292 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605da74b940 0x5605da752b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:05.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.194+0000 7fc89aa00640 1 -- 172.21.15.83:0/802016292 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5605da741270 msgr2=0x5605da7456d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:05.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.194+0000 7fc89aa00640 1 --2- 172.21.15.83:0/802016292 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5605da741270 0x5605da7456d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:05.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.194+0000 7fc89aa00640 1 -- 172.21.15.83:0/802016292 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605da74b940 msgr2=0x5605da752b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:05.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.194+0000 7fc89aa00640 1 --2- 172.21.15.83:0/802016292 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605da74b940 0x5605da752b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:05.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.194+0000 7fc89aa00640 1 -- 172.21.15.83:0/802016292 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605da753080 con 0x5605da740950 2024-04-24T09:15:05.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.194+0000 7fc89aa00640 1 --2- 172.21.15.83:0/802016292 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5605da740950 0x5605da745190 secure :-1 s=READY pgs=2703 cs=0 l=1 rev1=1 crypto rx=0x7fc888004600 tx=0x7fc888054630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=214c9fb355df6832 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:05.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.194+0000 7fc885600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:15:05.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.195+0000 7fc885600640 1 -- 172.21.15.83:0/802016292 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc888037070 con 0x5605da740950 2024-04-24T09:15:05.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.195+0000 7fc885600640 1 -- 172.21.15.83:0/802016292 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(128..128 src has 1..128) ==== 9583+0+0 (secure 0 0 0) 0x7fc888054800 con 0x5605da740950 2024-04-24T09:15:05.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.195+0000 7fc8a5631740 1 -- 172.21.15.83:0/802016292 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc888030900 con 0x5605da740950 2024-04-24T09:15:05.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.195+0000 7fc885600640 1 -- 172.21.15.83:0/802016292 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7fc888046040 con 0x5605da740950 2024-04-24T09:15:05.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.195+0000 7fc8a5631740 10 client.23532 fetch_fsmap learned FSMap version 68 2024-04-24T09:15:05.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.195+0000 7fc8a5631740 1 -- 172.21.15.83:0/802016292 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) -- 0x5605da754350 con 0x5605da740950 2024-04-24T09:15:05.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.197+0000 7fc885600640 1 -- 172.21.15.83:0/802016292 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc888030900 con 0x5605da740950 2024-04-24T09:15:05.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.197+0000 7fc885600640 1 -- 172.21.15.83:0/802016292 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 68) ==== 8374+0+0 (secure 0 0 0) 0x7fc88804c040 con 0x5605da740950 2024-04-24T09:15:05.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.197+0000 7fc8a5631740 10 client.23532 fetch_fsmap finished waiting for FSMap version 68 2024-04-24T09:15:05.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.197+0000 7fc8a5631740 10 client.23532 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-24T09:15:05.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.197+0000 7fc8a5631740 20 client.23532 populate_metadata read hostname 'smithi083' 2024-04-24T09:15:05.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.197+0000 7fc8a5631740 1 --2- 172.21.15.83:0/802016292 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x5605da7c21d0 0x5605da7c45c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:05.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.197+0000 7fc8a5631740 4 client.23532 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:15:05.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.197+0000 7fc8a5631740 1 -- 172.21.15.83:0/802016292 --> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5605da6ada60 con 0x5605da7c21d0 2024-04-24T09:15:05.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.198+0000 7fc89b400640 1 --2- 172.21.15.83:0/802016292 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x5605da7c21d0 0x5605da7c45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:05.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.198+0000 7fc885600640 10 client.23532 ms_handle_connect on v2:172.21.15.138:6834/1909219758 2024-04-24T09:15:05.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.198+0000 7fc89b400640 1 --2- 172.21.15.83:0/802016292 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x5605da7c21d0 0x5605da7c45c0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d6991bce618270b server_cookie=6a478a24d460514f in_seq=0 out_seq=0 2024-04-24T09:15:05.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.199+0000 7fc885600640 1 -- 172.21.15.83:0/802016292 <== mds.2 v2:172.21.15.138:6834/1909219758 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5605da6ada60 con 0x5605da7c21d0 2024-04-24T09:15:05.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.199+0000 7fc885600640 10 client.23532 handle_command_reply: tid=0 2024-04-24T09:15:05.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.199+0000 7fc8a5631740 1 client.23532 shutdown 2024-04-24T09:15:05.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.199+0000 7fc8a5631740 1 -- 172.21.15.83:0/802016292 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5605da740950 msgr2=0x5605da745190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:05.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.199+0000 7fc8a5631740 1 --2- 172.21.15.83:0/802016292 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5605da740950 0x5605da745190 secure :-1 s=READY pgs=2703 cs=0 l=1 rev1=1 crypto rx=0x7fc888004600 tx=0x7fc888054630 comp rx=0 tx=0).stop 2024-04-24T09:15:05.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.199+0000 7fc8a5631740 1 -- 172.21.15.83:0/802016292 shutdown_connections 2024-04-24T09:15:05.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.199+0000 7fc8a5631740 1 --2- 172.21.15.83:0/802016292 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x5605da7c21d0 0x5605da7c45c0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:05.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.199+0000 7fc8a5631740 1 --2- 172.21.15.83:0/802016292 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605da74b940 0x5605da752b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:05.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.199+0000 7fc8a5631740 1 --2- 172.21.15.83:0/802016292 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5605da741270 0x5605da7456d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:05.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.199+0000 7fc885600640 0 client.23532 ms_handle_reset on v2:172.21.15.138:6834/1909219758 2024-04-24T09:15:05.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.199+0000 7fc8a5631740 1 --2- 172.21.15.83:0/802016292 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5605da740950 0x5605da745190 unknown :-1 s=CLOSED pgs=2703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:05.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.199+0000 7fc8a5631740 1 -- 172.21.15.83:0/802016292 >> 172.21.15.83:0/802016292 conn(0x5605da73cc70 msgr2=0x5605da746eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:05.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.200+0000 7fc8a5631740 1 -- 172.21.15.83:0/802016292 shutdown_connections 2024-04-24T09:15:05.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.200+0000 7fc8a5631740 1 -- 172.21.15.83:0/802016292 wait complete. 2024-04-24T09:15:05.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.200+0000 7fc8a5631740 20 client.23532 trim_cache size 0 max 16384 2024-04-24T09:15:05.200 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:15:05.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.201+0000 7fc8a2800640 1 -- 172.21.15.83:0/1117302128 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fc8800a3da0 msgr2=0x7fc8800a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:05.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.201+0000 7fc8a2800640 1 --2- 172.21.15.83:0/1117302128 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fc8800a3da0 0x7fc8800a6260 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fc88c007fd0 tx=0x7fc88c0044e0 comp rx=0 tx=0).stop 2024-04-24T09:15:05.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.201+0000 7fc8a2800640 1 -- 172.21.15.83:0/1117302128 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc89c076f90 msgr2=0x7fc89c1ab120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:05.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.201+0000 7fc8a2800640 1 --2- 172.21.15.83:0/1117302128 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc89c076f90 0x7fc89c1ab120 secure :-1 s=READY pgs=2701 cs=0 l=1 rev1=1 crypto rx=0x7fc88c00c710 tx=0x7fc88c00cbd0 comp rx=0 tx=0).stop 2024-04-24T09:15:05.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.201+0000 7fc8a2800640 1 -- 172.21.15.83:0/1117302128 shutdown_connections 2024-04-24T09:15:05.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.201+0000 7fc8a2800640 1 --2- 172.21.15.83:0/1117302128 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fc8800a3da0 0x7fc8800a6260 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:05.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.201+0000 7fc8a2800640 1 --2- 172.21.15.83:0/1117302128 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc89c076f90 0x7fc89c1ab120 unknown :-1 s=CLOSED pgs=2701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:05.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.201+0000 7fc8a2800640 1 --2- 172.21.15.83:0/1117302128 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc89c109580 0x7fc89c076a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:05.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.201+0000 7fc8a2800640 1 --2- 172.21.15.83:0/1117302128 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc89c1044b0 0x7fc89c076510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:05.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.201+0000 7fc8a2800640 1 -- 172.21.15.83:0/1117302128 >> 172.21.15.83:0/1117302128 conn(0x7fc89c0ffd90 msgr2=0x7fc89c101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:05.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.201+0000 7fc8a2800640 1 -- 172.21.15.83:0/1117302128 shutdown_connections 2024-04-24T09:15:05.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.201+0000 7fc8a2800640 1 -- 172.21.15.83:0/1117302128 wait complete. 2024-04-24T09:15:05.209 INFO:tasks.workunit.client.0.smithi083.stdout: 10 70777 6.53 MB/sec execute 456 sec latency 88.311 ms 2024-04-24T09:15:05.220 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T09:15:05.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.476+0000 7ff058400640 1 -- 172.21.15.83:0/2479424331 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff050109580 msgr2=0x7ff050111150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:05.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.476+0000 7ff058400640 1 --2- 172.21.15.83:0/2479424331 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff050109580 0x7ff050111150 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7ff048009be0 tx=0x7ff04802a6c0 comp rx=0 tx=0).stop 2024-04-24T09:15:05.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.476+0000 7ff058400640 1 -- 172.21.15.83:0/2479424331 shutdown_connections 2024-04-24T09:15:05.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.476+0000 7ff058400640 1 --2- 172.21.15.83:0/2479424331 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff050109580 0x7ff050111150 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:05.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.476+0000 7ff058400640 1 --2- 172.21.15.83:0/2479424331 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff050104e60 0x7ff050108ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:05.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.476+0000 7ff058400640 1 --2- 172.21.15.83:0/2479424331 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff0501044b0 0x7ff050104890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:05.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.476+0000 7ff058400640 1 -- 172.21.15.83:0/2479424331 >> 172.21.15.83:0/2479424331 conn(0x7ff0500ffd90 msgr2=0x7ff0501021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:05.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.476+0000 7ff058400640 1 -- 172.21.15.83:0/2479424331 shutdown_connections 2024-04-24T09:15:05.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.476+0000 7ff058400640 1 -- 172.21.15.83:0/2479424331 wait complete. 2024-04-24T09:15:05.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.477+0000 7ff058400640 1 Processor -- start 2024-04-24T09:15:05.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.478+0000 7ff058400640 1 -- start start 2024-04-24T09:15:05.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.478+0000 7ff058400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff0501044b0 0x7ff0501a06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:05.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.478+0000 7ff058400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff050109580 0x7ff0501a0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:05.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.478+0000 7ff058400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff05019a8d0 0x7ff05019ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:05.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.478+0000 7ff058400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff0501144f0 con 0x7ff05019a8d0 2024-04-24T09:15:05.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.478+0000 7ff058400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff050114370 con 0x7ff0501044b0 2024-04-24T09:15:05.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.478+0000 7ff058400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff050114670 con 0x7ff050109580 2024-04-24T09:15:05.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.478+0000 7ff056400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff05019a8d0 0x7ff05019ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:05.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.478+0000 7ff055a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff0501044b0 0x7ff0501a06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:05.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.478+0000 7ff056400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff05019a8d0 0x7ff05019ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55012/0 (socket says 172.21.15.83:55012) 2024-04-24T09:15:05.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.478+0000 7ff056400640 1 -- 172.21.15.83:0/2564573200 learned_addr learned my addr 172.21.15.83:0/2564573200 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:15:05.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.479+0000 7ff055000640 1 --2- 172.21.15.83:0/2564573200 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff050109580 0x7ff0501a0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:05.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.479+0000 7ff055a00640 1 -- 172.21.15.83:0/2564573200 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff050109580 msgr2=0x7ff0501a0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:05.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.479+0000 7ff055a00640 1 --2- 172.21.15.83:0/2564573200 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff050109580 0x7ff0501a0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:05.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.479+0000 7ff055a00640 1 -- 172.21.15.83:0/2564573200 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff05019a8d0 msgr2=0x7ff05019ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:05.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.479+0000 7ff055a00640 1 --2- 172.21.15.83:0/2564573200 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff05019a8d0 0x7ff05019ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:05.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.479+0000 7ff055a00640 1 -- 172.21.15.83:0/2564573200 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff05019b3f0 con 0x7ff0501044b0 2024-04-24T09:15:05.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.479+0000 7ff056400640 1 --2- 172.21.15.83:0/2564573200 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff05019a8d0 0x7ff05019ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:15:05.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.479+0000 7ff055a00640 1 --2- 172.21.15.83:0/2564573200 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff0501044b0 0x7ff0501a06d0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7ff04400e790 tx=0x7ff04400ec50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:05.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.480+0000 7ff04e000640 1 -- 172.21.15.83:0/2564573200 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff04400eea0 con 0x7ff0501044b0 2024-04-24T09:15:05.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.480+0000 7ff058400640 1 -- 172.21.15.83:0/2564573200 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff05019b6e0 con 0x7ff0501044b0 2024-04-24T09:15:05.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.480+0000 7ff04e000640 1 -- 172.21.15.83:0/2564573200 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff04400ccc0 con 0x7ff0501044b0 2024-04-24T09:15:05.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.480+0000 7ff04e000640 1 -- 172.21.15.83:0/2564573200 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff044018710 con 0x7ff0501044b0 2024-04-24T09:15:05.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.480+0000 7ff058400640 1 -- 172.21.15.83:0/2564573200 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0501a7930 con 0x7ff0501044b0 2024-04-24T09:15:05.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.484+0000 7ff05b27c740 1 Processor -- start 2024-04-24T09:15:05.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.484+0000 7ff05b27c740 1 -- start start 2024-04-24T09:15:05.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.484+0000 7ff05b27c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560761042950 0x560761042d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:05.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.484+0000 7ff05b27c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560761043270 0x56076104d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:05.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.484+0000 7ff05b27c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56076104d940 0x56076104fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:05.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.484+0000 7ff05b27c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x560760fa81d0 con 0x560761043270 2024-04-24T09:15:05.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.484+0000 7ff05b27c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560760f4e450 con 0x560761042950 2024-04-24T09:15:05.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.484+0000 7ff05b27c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x560760fa4190 con 0x56076104d940 2024-04-24T09:15:05.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.485+0000 7ff056400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560761042950 0x560761042d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:05.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.485+0000 7ff055000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56076104d940 0x56076104fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:05.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.485+0000 7ff056400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560761042950 0x560761042d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:37292/0 (socket says 172.21.15.83:37292) 2024-04-24T09:15:05.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.485+0000 7ff056400640 1 -- 172.21.15.83:0/2284287025 learned_addr learned my addr 172.21.15.83:0/2284287025 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:15:05.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.485+0000 7ff04e000640 1 -- 172.21.15.83:0/2564573200 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff0440188b0 con 0x7ff0501044b0 2024-04-24T09:15:05.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.485+0000 7ff055a00640 1 --2- 172.21.15.83:0/2284287025 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560761043270 0x56076104d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:05.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.486+0000 7ff055000640 1 -- 172.21.15.83:0/2284287025 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560761042950 msgr2=0x560761042d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:05.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.486+0000 7ff055000640 1 --2- 172.21.15.83:0/2284287025 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560761042950 0x560761042d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:05.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.486+0000 7ff055000640 1 -- 172.21.15.83:0/2284287025 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560761043270 msgr2=0x56076104d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:05.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.486+0000 7ff055000640 1 --2- 172.21.15.83:0/2284287025 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560761043270 0x56076104d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:05.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.486+0000 7ff055000640 1 -- 172.21.15.83:0/2284287025 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56076100b310 con 0x56076104d940 2024-04-24T09:15:05.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.486+0000 7ff056400640 1 --2- 172.21.15.83:0/2284287025 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560761042950 0x560761042d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:15:05.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.487+0000 7ff055000640 1 --2- 172.21.15.83:0/2284287025 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56076104d940 0x56076104fd30 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff05019c070 tx=0x7ff03c009f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29683169c967f2c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:05.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.488+0000 7ff04e000640 1 --2- 172.21.15.83:0/2564573200 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff0380a3e20 0x7ff0380a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:05.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.488+0000 7ff042a00640 1 -- 172.21.15.83:0/2284287025 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff03c011d70 con 0x56076104d940 2024-04-24T09:15:05.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.488+0000 7ff04e000640 1 -- 172.21.15.83:0/2564573200 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9583+0+0 (secure 0 0 0) 0x7ff0440d0d50 con 0x7ff0501044b0 2024-04-24T09:15:05.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.488+0000 7ff042a00640 1 -- 172.21.15.83:0/2284287025 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff03c012900 con 0x56076104d940 2024-04-24T09:15:05.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.488+0000 7ff042a00640 1 -- 172.21.15.83:0/2284287025 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff03c00f8e0 con 0x56076104d940 2024-04-24T09:15:05.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.488+0000 7ff056400640 1 --2- 172.21.15.83:0/2564573200 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff0380a3e20 0x7ff0380a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:05.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.489+0000 7ff05b27c740 1 -- 172.21.15.83:0/2284287025 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56076104d940 msgr2=0x56076104fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:05.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.489+0000 7ff05b27c740 1 --2- 172.21.15.83:0/2284287025 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56076104d940 0x56076104fd30 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff05019c070 tx=0x7ff03c009f50 comp rx=0 tx=0).stop 2024-04-24T09:15:05.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.489+0000 7ff056400640 1 --2- 172.21.15.83:0/2564573200 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff0380a3e20 0x7ff0380a62e0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7ff048005f00 tx=0x7ff048005e30 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:05.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.489+0000 7ff05b27c740 1 -- 172.21.15.83:0/2284287025 shutdown_connections 2024-04-24T09:15:05.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.489+0000 7ff05b27c740 1 --2- 172.21.15.83:0/2284287025 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56076104d940 0x56076104fd30 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:05.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.489+0000 7ff05b27c740 1 --2- 172.21.15.83:0/2284287025 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560761043270 0x56076104d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:05.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.489+0000 7ff05b27c740 1 --2- 172.21.15.83:0/2284287025 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560761042950 0x560761042d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:05.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.489+0000 7ff05b27c740 1 -- 172.21.15.83:0/2284287025 >> 172.21.15.83:0/2284287025 conn(0x56076103ec70 msgr2=0x56076103f050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:05.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.489+0000 7ff05b27c740 1 -- 172.21.15.83:0/2284287025 shutdown_connections 2024-04-24T09:15:05.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.490+0000 7ff05b27c740 1 -- 172.21.15.83:0/2284287025 wait complete. 2024-04-24T09:15:05.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.490+0000 7ff05b27c740 1 -- start start 2024-04-24T09:15:05.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.490+0000 7ff05b27c740 1 Processor -- start 2024-04-24T09:15:05.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.491+0000 7ff05b27c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560761042950 0x56076104d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:05.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.491+0000 7ff05b27c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560761043270 0x56076104a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:05.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.491+0000 7ff05b27c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56076104d940 0x56076104ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:05.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.491+0000 7ff05b27c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5607610507d0 con 0x56076104d940 2024-04-24T09:15:05.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.491+0000 7ff05b27c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560760f61810 con 0x560761042950 2024-04-24T09:15:05.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.491+0000 7ff05b27c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x560761050950 con 0x560761043270 2024-04-24T09:15:05.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.491+0000 7ff05b27c740 10 client.0 _ll_register_callbacks cb 0x560760f2f140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:15:05.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.491+0000 7ff056400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56076104d940 0x56076104ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:05.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.491+0000 7ff056400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56076104d940 0x56076104ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55036/0 (socket says 172.21.15.83:55036) 2024-04-24T09:15:05.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.491+0000 7ff056400640 1 -- 172.21.15.83:0/428209048 learned_addr learned my addr 172.21.15.83:0/428209048 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:15:05.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.492+0000 7ff055000640 1 --2- 172.21.15.83:0/428209048 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560761043270 0x56076104a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:05.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.492+0000 7ff056400640 1 -- 172.21.15.83:0/428209048 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560761043270 msgr2=0x56076104a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:05.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.492+0000 7ff056400640 1 --2- 172.21.15.83:0/428209048 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560761043270 0x56076104a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:05.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.492+0000 7ff056400640 1 -- 172.21.15.83:0/428209048 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560761042950 msgr2=0x56076104d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:05.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.492+0000 7ff056400640 1 --2- 172.21.15.83:0/428209048 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560761042950 0x56076104d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:05.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.492+0000 7ff056400640 1 -- 172.21.15.83:0/428209048 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560760f723f0 con 0x56076104d940 2024-04-24T09:15:05.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.492+0000 7ff055000640 1 --2- 172.21.15.83:0/428209048 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560761043270 0x56076104a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:15:05.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.493+0000 7ff056400640 1 --2- 172.21.15.83:0/428209048 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56076104d940 0x56076104ada0 secure :-1 s=READY pgs=2704 cs=0 l=1 rev1=1 crypto rx=0x7ff048030f80 tx=0x7ff04802a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3440d9fe4a4cf7f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:05.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.493+0000 7ff037400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:15:05.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.493+0000 7ff037400640 1 -- 172.21.15.83:0/428209048 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff048050070 con 0x56076104d940 2024-04-24T09:15:05.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.493+0000 7ff037400640 1 -- 172.21.15.83:0/428209048 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(128..128 src has 1..128) ==== 9583+0+0 (secure 0 0 0) 0x7ff048004040 con 0x56076104d940 2024-04-24T09:15:05.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.493+0000 7ff05b27c740 1 -- 172.21.15.83:0/428209048 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff03c012900 con 0x56076104d940 2024-04-24T09:15:05.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.494+0000 7ff037400640 1 -- 172.21.15.83:0/428209048 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7ff04805a050 con 0x56076104d940 2024-04-24T09:15:05.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.494+0000 7ff05b27c740 10 client.23544 fetch_fsmap learned FSMap version 68 2024-04-24T09:15:05.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.494+0000 7ff05b27c740 1 -- 172.21.15.83:0/428209048 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) -- 0x560760cba3f0 con 0x56076104d940 2024-04-24T09:15:05.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.495+0000 7ff037400640 1 -- 172.21.15.83:0/428209048 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff03c012900 con 0x56076104d940 2024-04-24T09:15:05.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.495+0000 7ff037400640 1 -- 172.21.15.83:0/428209048 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 68) ==== 8374+0+0 (secure 0 0 0) 0x7ff04803c040 con 0x56076104d940 2024-04-24T09:15:05.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.495+0000 7ff05b27c740 10 client.23544 fetch_fsmap finished waiting for FSMap version 68 2024-04-24T09:15:05.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.495+0000 7ff05b27c740 10 client.23544 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-24T09:15:05.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.495+0000 7ff05b27c740 20 client.23544 populate_metadata read hostname 'smithi083' 2024-04-24T09:15:05.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.495+0000 7ff05b27c740 1 --2- 172.21.15.83:0/428209048 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x5607610664c0 0x560761068980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:05.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.495+0000 7ff05b27c740 4 client.23544 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:15:05.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.495+0000 7ff05b27c740 1 -- 172.21.15.83:0/428209048 --> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560760fb5bf0 con 0x5607610664c0 2024-04-24T09:15:05.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.496+0000 7ff055a00640 1 --2- 172.21.15.83:0/428209048 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x5607610664c0 0x560761068980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:05.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.497+0000 7ff055a00640 1 --2- 172.21.15.83:0/428209048 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x5607610664c0 0x560761068980 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=26f92947c4263071 server_cookie=b1c4449d51db3190 in_seq=0 out_seq=0 2024-04-24T09:15:05.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.497+0000 7ff037400640 10 client.23544 ms_handle_connect on v2:172.21.15.138:6836/900566754 2024-04-24T09:15:05.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.498+0000 7ff037400640 1 -- 172.21.15.83:0/428209048 <== mds.1 v2:172.21.15.138:6836/900566754 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560760fb5bf0 con 0x5607610664c0 2024-04-24T09:15:05.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.498+0000 7ff037400640 10 client.23544 handle_command_reply: tid=0 2024-04-24T09:15:05.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.498+0000 7ff05b27c740 1 client.23544 shutdown 2024-04-24T09:15:05.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.499+0000 7ff05b27c740 1 -- 172.21.15.83:0/428209048 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56076104d940 msgr2=0x56076104ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:05.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.499+0000 7ff05b27c740 1 --2- 172.21.15.83:0/428209048 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56076104d940 0x56076104ada0 secure :-1 s=READY pgs=2704 cs=0 l=1 rev1=1 crypto rx=0x7ff048030f80 tx=0x7ff04802a6c0 comp rx=0 tx=0).stop 2024-04-24T09:15:05.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.499+0000 7ff05b27c740 1 -- 172.21.15.83:0/428209048 shutdown_connections 2024-04-24T09:15:05.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.499+0000 7ff05b27c740 1 --2- 172.21.15.83:0/428209048 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56076104d940 0x56076104ada0 unknown :-1 s=CLOSED pgs=2704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:05.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.499+0000 7ff05b27c740 1 --2- 172.21.15.83:0/428209048 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x5607610664c0 0x560761068980 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:05.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.499+0000 7ff05b27c740 1 --2- 172.21.15.83:0/428209048 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560761043270 0x56076104a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:05.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.499+0000 7ff05b27c740 1 --2- 172.21.15.83:0/428209048 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560761042950 0x56076104d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:05.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.499+0000 7ff05b27c740 1 -- 172.21.15.83:0/428209048 >> 172.21.15.83:0/428209048 conn(0x56076103ec70 msgr2=0x560761044970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:05.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.499+0000 7ff037400640 0 client.23544 ms_handle_reset on v2:172.21.15.138:6836/900566754 2024-04-24T09:15:05.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.499+0000 7ff05b27c740 1 -- 172.21.15.83:0/428209048 shutdown_connections 2024-04-24T09:15:05.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.499+0000 7ff05b27c740 1 -- 172.21.15.83:0/428209048 wait complete. 2024-04-24T09:15:05.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.500+0000 7ff05b27c740 20 client.23544 trim_cache size 0 max 16384 2024-04-24T09:15:05.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.527+0000 7ff05b27c740 1 Processor -- start 2024-04-24T09:15:05.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.527+0000 7ff05b27c740 1 -- start start 2024-04-24T09:15:05.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.528+0000 7ff05b27c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560761042950 0x5607610450f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:05.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.528+0000 7ff05b27c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560761043270 0x56076104cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:05.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.528+0000 7ff05b27c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56076104d940 0x56076104d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:05.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.528+0000 7ff05b27c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff0300100b0 con 0x560761042950 2024-04-24T09:15:05.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.528+0000 7ff05b27c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff030011c80 con 0x560761043270 2024-04-24T09:15:05.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.528+0000 7ff05b27c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff030010780 con 0x56076104d940 2024-04-24T09:15:05.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.528+0000 7ff055a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560761042950 0x5607610450f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:05.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.528+0000 7ff055a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560761042950 0x5607610450f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55052/0 (socket says 172.21.15.83:55052) 2024-04-24T09:15:05.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.528+0000 7ff055000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560761043270 0x56076104cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:05.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.528+0000 7ff055a00640 1 -- 172.21.15.83:0/4134471082 learned_addr learned my addr 172.21.15.83:0/4134471082 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:15:05.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.528+0000 7ff056400640 1 --2- 172.21.15.83:0/4134471082 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56076104d940 0x56076104d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:05.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.529+0000 7ff055000640 1 -- 172.21.15.83:0/4134471082 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56076104d940 msgr2=0x56076104d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:05.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.529+0000 7ff055000640 1 --2- 172.21.15.83:0/4134471082 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56076104d940 0x56076104d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:05.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.529+0000 7ff055000640 1 -- 172.21.15.83:0/4134471082 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560761042950 msgr2=0x5607610450f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:05.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.529+0000 7ff055000640 1 --2- 172.21.15.83:0/4134471082 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560761042950 0x5607610450f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:05.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.529+0000 7ff055000640 1 -- 172.21.15.83:0/4134471082 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560760d847d0 con 0x560761043270 2024-04-24T09:15:05.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.529+0000 7ff055a00640 1 --2- 172.21.15.83:0/4134471082 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560761042950 0x5607610450f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:15:05.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.529+0000 7ff055000640 1 --2- 172.21.15.83:0/4134471082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560761043270 0x56076104cea0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x560761049ec0 tx=0x7ff03c009800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b0b4ea8ba86ac40 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:05.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.530+0000 7ff042000640 1 -- 172.21.15.83:0/4134471082 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff03c011e70 con 0x560761043270 2024-04-24T09:15:05.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.530+0000 7ff042000640 1 -- 172.21.15.83:0/4134471082 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff03c0054f0 con 0x560761043270 2024-04-24T09:15:05.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.530+0000 7ff042000640 1 -- 172.21.15.83:0/4134471082 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff03c003e50 con 0x560761043270 2024-04-24T09:15:05.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.531+0000 7ff05b27c740 1 -- 172.21.15.83:0/4134471082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560761043270 msgr2=0x56076104cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:05.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.531+0000 7ff05b27c740 1 --2- 172.21.15.83:0/4134471082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560761043270 0x56076104cea0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x560761049ec0 tx=0x7ff03c009800 comp rx=0 tx=0).stop 2024-04-24T09:15:05.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.531+0000 7ff05b27c740 1 -- 172.21.15.83:0/4134471082 shutdown_connections 2024-04-24T09:15:05.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.531+0000 7ff05b27c740 1 --2- 172.21.15.83:0/4134471082 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56076104d940 0x56076104d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:05.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.531+0000 7ff05b27c740 1 --2- 172.21.15.83:0/4134471082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560761043270 0x56076104cea0 secure :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0x560761049ec0 tx=0x7ff03c009800 comp rx=0 tx=0).stop 2024-04-24T09:15:05.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.531+0000 7ff05b27c740 1 --2- 172.21.15.83:0/4134471082 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560761042950 0x5607610450f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:05.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.531+0000 7ff05b27c740 1 -- 172.21.15.83:0/4134471082 >> 172.21.15.83:0/4134471082 conn(0x56076103ec70 msgr2=0x560761045980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:05.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.532+0000 7ff05b27c740 1 -- 172.21.15.83:0/4134471082 shutdown_connections 2024-04-24T09:15:05.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.532+0000 7ff05b27c740 1 -- 172.21.15.83:0/4134471082 wait complete. 2024-04-24T09:15:05.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.532+0000 7ff05b27c740 1 -- start start 2024-04-24T09:15:05.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.532+0000 7ff05b27c740 1 Processor -- start 2024-04-24T09:15:05.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.533+0000 7ff05b27c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560761042950 0x56076105e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:05.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.533+0000 7ff05b27c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56076104d940 0x560761047900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:05.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.533+0000 7ff05b27c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560761047e40 0x5607610482c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:05.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.533+0000 7ff05b27c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff030011580 con 0x560761047e40 2024-04-24T09:15:05.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.533+0000 7ff05b27c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff030010eb0 con 0x56076104d940 2024-04-24T09:15:05.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.533+0000 7ff05b27c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff03000e0b0 con 0x560761042950 2024-04-24T09:15:05.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.533+0000 7ff05b27c740 10 client.0 _ll_register_callbacks cb 0x7ff048043070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:15:05.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.533+0000 7ff055000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56076104d940 0x560761047900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:05.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.533+0000 7ff056400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560761047e40 0x5607610482c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:05.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.533+0000 7ff055000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56076104d940 0x560761047900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:37340/0 (socket says 172.21.15.83:37340) 2024-04-24T09:15:05.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.533+0000 7ff055000640 1 -- 172.21.15.83:0/1094029110 learned_addr learned my addr 172.21.15.83:0/1094029110 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:15:05.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.533+0000 7ff055a00640 1 --2- 172.21.15.83:0/1094029110 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560761042950 0x56076105e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:05.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.534+0000 7ff055000640 1 -- 172.21.15.83:0/1094029110 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560761042950 msgr2=0x56076105e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:05.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.534+0000 7ff055000640 1 --2- 172.21.15.83:0/1094029110 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560761042950 0x56076105e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:05.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.534+0000 7ff055000640 1 -- 172.21.15.83:0/1094029110 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560761047e40 msgr2=0x5607610482c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:05.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.534+0000 7ff055000640 1 --2- 172.21.15.83:0/1094029110 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560761047e40 0x5607610482c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:05.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.534+0000 7ff055000640 1 -- 172.21.15.83:0/1094029110 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56076105ed20 con 0x56076104d940 2024-04-24T09:15:05.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.534+0000 7ff055a00640 1 --2- 172.21.15.83:0/1094029110 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560761042950 0x56076105e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:15:05.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.534+0000 7ff056400640 1 --2- 172.21.15.83:0/1094029110 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560761047e40 0x5607610482c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:15:05.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.534+0000 7ff055000640 1 --2- 172.21.15.83:0/1094029110 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56076104d940 0x560761047900 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7ff03c01c040 tx=0x7ff03c0058d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5454846ba05ae98 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:05.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.534+0000 7ff037400640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:15:05.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.535+0000 7ff037400640 1 -- 172.21.15.83:0/1094029110 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff03c0052c0 con 0x56076104d940 2024-04-24T09:15:05.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.535+0000 7ff037400640 1 -- 172.21.15.83:0/1094029110 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(128..128 src has 1..128) ==== 9583+0+0 (secure 0 0 0) 0x7ff03c003bd0 con 0x56076104d940 2024-04-24T09:15:05.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.535+0000 7ff05b27c740 1 -- 172.21.15.83:0/1094029110 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff03c0054f0 con 0x56076104d940 2024-04-24T09:15:05.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.535+0000 7ff037400640 1 -- 172.21.15.83:0/1094029110 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7ff03c03f050 con 0x56076104d940 2024-04-24T09:15:05.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.535+0000 7ff05b27c740 10 client.40607 fetch_fsmap learned FSMap version 68 2024-04-24T09:15:05.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.535+0000 7ff05b27c740 1 -- 172.21.15.83:0/1094029110 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) -- 0x5607610ccb20 con 0x56076104d940 2024-04-24T09:15:05.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.536+0000 7ff037400640 1 -- 172.21.15.83:0/1094029110 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff03c0054f0 con 0x56076104d940 2024-04-24T09:15:05.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.536+0000 7ff037400640 1 -- 172.21.15.83:0/1094029110 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 68) ==== 8374+0+0 (secure 0 0 0) 0x7ff03c024420 con 0x56076104d940 2024-04-24T09:15:05.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.536+0000 7ff05b27c740 10 client.40607 fetch_fsmap finished waiting for FSMap version 68 2024-04-24T09:15:05.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.536+0000 7ff05b27c740 10 client.40607 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-24T09:15:05.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.536+0000 7ff05b27c740 20 client.40607 populate_metadata read hostname 'smithi083' 2024-04-24T09:15:05.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.536+0000 7ff05b27c740 1 --2- 172.21.15.83:0/1094029110 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x5607610ccfd0 0x5607610cf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:05.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.536+0000 7ff05b27c740 4 client.40607 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:15:05.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.536+0000 7ff05b27c740 1 -- 172.21.15.83:0/1094029110 --> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560760fafa60 con 0x5607610ccfd0 2024-04-24T09:15:05.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.537+0000 7ff055a00640 1 --2- 172.21.15.83:0/1094029110 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x5607610ccfd0 0x5607610cf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:05.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.537+0000 7ff055a00640 1 --2- 172.21.15.83:0/1094029110 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x5607610ccfd0 0x5607610cf3f0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=90fa26f5ea41855d server_cookie=b33f79014b4783fd in_seq=0 out_seq=0 2024-04-24T09:15:05.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.537+0000 7ff037400640 10 client.40607 ms_handle_connect on v2:172.21.15.138:6836/900566754 2024-04-24T09:15:05.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.538+0000 7ff037400640 1 -- 172.21.15.83:0/1094029110 <== mds.1 v2:172.21.15.138:6836/900566754 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560760fafa60 con 0x5607610ccfd0 2024-04-24T09:15:05.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.538+0000 7ff037400640 10 client.40607 handle_command_reply: tid=0 2024-04-24T09:15:05.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.538+0000 7ff05b27c740 1 client.40607 shutdown 2024-04-24T09:15:05.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.538+0000 7ff05b27c740 1 -- 172.21.15.83:0/1094029110 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56076104d940 msgr2=0x560761047900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:05.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.538+0000 7ff05b27c740 1 --2- 172.21.15.83:0/1094029110 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56076104d940 0x560761047900 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7ff03c01c040 tx=0x7ff03c0058d0 comp rx=0 tx=0).stop 2024-04-24T09:15:05.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.538+0000 7ff05b27c740 1 -- 172.21.15.83:0/1094029110 shutdown_connections 2024-04-24T09:15:05.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.538+0000 7ff05b27c740 1 --2- 172.21.15.83:0/1094029110 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560761047e40 0x5607610482c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:05.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.538+0000 7ff05b27c740 1 --2- 172.21.15.83:0/1094029110 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56076104d940 0x560761047900 secure :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7ff03c01c040 tx=0x7ff03c0058d0 comp rx=0 tx=0).stop 2024-04-24T09:15:05.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.538+0000 7ff05b27c740 1 --2- 172.21.15.83:0/1094029110 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x5607610ccfd0 0x5607610cf3f0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:05.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.538+0000 7ff05b27c740 1 --2- 172.21.15.83:0/1094029110 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560761042950 0x56076105e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:05.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.538+0000 7ff05b27c740 1 -- 172.21.15.83:0/1094029110 >> 172.21.15.83:0/1094029110 conn(0x56076103ec70 msgr2=0x560761048f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:05.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.538+0000 7ff037400640 0 client.40607 ms_handle_reset on v2:172.21.15.138:6836/900566754 2024-04-24T09:15:05.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.538+0000 7ff05b27c740 1 -- 172.21.15.83:0/1094029110 shutdown_connections 2024-04-24T09:15:05.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.539+0000 7ff05b27c740 1 -- 172.21.15.83:0/1094029110 wait complete. 2024-04-24T09:15:05.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.539+0000 7ff05b27c740 20 client.40607 trim_cache size 0 max 16384 2024-04-24T09:15:05.539 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:15:05.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.539+0000 7ff058400640 1 -- 172.21.15.83:0/2564573200 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff0380a3e20 msgr2=0x7ff0380a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:05.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.539+0000 7ff058400640 1 --2- 172.21.15.83:0/2564573200 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff0380a3e20 0x7ff0380a62e0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7ff048005f00 tx=0x7ff048005e30 comp rx=0 tx=0).stop 2024-04-24T09:15:05.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.539+0000 7ff058400640 1 -- 172.21.15.83:0/2564573200 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff0501044b0 msgr2=0x7ff0501a06d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:05.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.539+0000 7ff058400640 1 --2- 172.21.15.83:0/2564573200 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff0501044b0 0x7ff0501a06d0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7ff04400e790 tx=0x7ff04400ec50 comp rx=0 tx=0).stop 2024-04-24T09:15:05.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.540+0000 7ff058400640 1 -- 172.21.15.83:0/2564573200 shutdown_connections 2024-04-24T09:15:05.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.540+0000 7ff058400640 1 --2- 172.21.15.83:0/2564573200 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff0380a3e20 0x7ff0380a62e0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:05.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.540+0000 7ff058400640 1 --2- 172.21.15.83:0/2564573200 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff05019a8d0 0x7ff05019ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:05.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.540+0000 7ff058400640 1 --2- 172.21.15.83:0/2564573200 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff050109580 0x7ff0501a0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:05.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.540+0000 7ff058400640 1 --2- 172.21.15.83:0/2564573200 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff0501044b0 0x7ff0501a06d0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:05.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.540+0000 7ff058400640 1 -- 172.21.15.83:0/2564573200 >> 172.21.15.83:0/2564573200 conn(0x7ff0500ffd90 msgr2=0x7ff050101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:05.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.540+0000 7ff058400640 1 -- 172.21.15.83:0/2564573200 shutdown_connections 2024-04-24T09:15:05.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.540+0000 7ff058400640 1 -- 172.21.15.83:0/2564573200 wait complete. 2024-04-24T09:15:05.564 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T09:15:05.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.816+0000 7f46eca00640 1 -- 172.21.15.83:0/3663255894 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46e8100c70 msgr2=0x7f46e8101050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:05.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.816+0000 7f46eca00640 1 --2- 172.21.15.83:0/3663255894 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46e8100c70 0x7f46e8101050 secure :-1 s=READY pgs=2705 cs=0 l=1 rev1=1 crypto rx=0x7f46d8009900 tx=0x7f46d802a3f0 comp rx=0 tx=0).stop 2024-04-24T09:15:05.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.816+0000 7f46eca00640 1 -- 172.21.15.83:0/3663255894 shutdown_connections 2024-04-24T09:15:05.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.816+0000 7f46eca00640 1 --2- 172.21.15.83:0/3663255894 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46e810e820 0x7f46e8110c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:05.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.816+0000 7f46eca00640 1 --2- 172.21.15.83:0/3663255894 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f46e8101620 0x7f46e810e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:05.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.816+0000 7f46eca00640 1 --2- 172.21.15.83:0/3663255894 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46e8100c70 0x7f46e8101050 unknown :-1 s=CLOSED pgs=2705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:05.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.816+0000 7f46eca00640 1 -- 172.21.15.83:0/3663255894 >> 172.21.15.83:0/3663255894 conn(0x7f46e80fc880 msgr2=0x7f46e80feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:05.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.816+0000 7f46eca00640 1 -- 172.21.15.83:0/3663255894 shutdown_connections 2024-04-24T09:15:05.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.816+0000 7f46eca00640 1 -- 172.21.15.83:0/3663255894 wait complete. 2024-04-24T09:15:05.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.818+0000 7f46eca00640 1 Processor -- start 2024-04-24T09:15:05.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.818+0000 7f46eca00640 1 -- start start 2024-04-24T09:15:05.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.818+0000 7f46eca00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46e8100c70 0x7f46e8108510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:05.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.818+0000 7f46eca00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46e8101620 0x7f46e810e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:05.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.818+0000 7f46eca00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f46e810e820 0x7f46e8108bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:05.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.818+0000 7f46eca00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f46e8103180 con 0x7f46e8100c70 2024-04-24T09:15:05.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.818+0000 7f46eca00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f46e8103000 con 0x7f46e8101620 2024-04-24T09:15:05.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.818+0000 7f46eca00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f46e8103300 con 0x7f46e810e820 2024-04-24T09:15:05.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.819+0000 7f46e6a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46e8101620 0x7f46e810e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:05.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.819+0000 7f46e7400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46e8100c70 0x7f46e8108510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:05.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.819+0000 7f46e7400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46e8100c70 0x7f46e8108510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55082/0 (socket says 172.21.15.83:55082) 2024-04-24T09:15:05.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.819+0000 7f46e7400640 1 -- 172.21.15.83:0/2246919015 learned_addr learned my addr 172.21.15.83:0/2246919015 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:15:05.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.819+0000 7f46e7e00640 1 --2- 172.21.15.83:0/2246919015 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f46e810e820 0x7f46e8108bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:05.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.819+0000 7f46e7400640 1 -- 172.21.15.83:0/2246919015 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f46e810e820 msgr2=0x7f46e8108bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:05.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.819+0000 7f46e7400640 1 --2- 172.21.15.83:0/2246919015 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f46e810e820 0x7f46e8108bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:05.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.819+0000 7f46e7400640 1 -- 172.21.15.83:0/2246919015 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46e8101620 msgr2=0x7f46e810e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:05.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.819+0000 7f46e7400640 1 --2- 172.21.15.83:0/2246919015 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46e8101620 0x7f46e810e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:05.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.819+0000 7f46e7400640 1 -- 172.21.15.83:0/2246919015 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46e8103cc0 con 0x7f46e8100c70 2024-04-24T09:15:05.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.820+0000 7f46e7400640 1 --2- 172.21.15.83:0/2246919015 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46e8100c70 0x7f46e8108510 secure :-1 s=READY pgs=2706 cs=0 l=1 rev1=1 crypto rx=0x7f46d8031440 tx=0x7f46d8003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:05.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.820+0000 7f46dfe00640 1 -- 172.21.15.83:0/2246919015 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46d803f070 con 0x7f46e8100c70 2024-04-24T09:15:05.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.820+0000 7f46eca00640 1 -- 172.21.15.83:0/2246919015 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46e8103ef0 con 0x7f46e8100c70 2024-04-24T09:15:05.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.820+0000 7f46eca00640 1 -- 172.21.15.83:0/2246919015 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46e8104430 con 0x7f46e8100c70 2024-04-24T09:15:05.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.821+0000 7f46dfe00640 1 -- 172.21.15.83:0/2246919015 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f46d8003e40 con 0x7f46e8100c70 2024-04-24T09:15:05.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.821+0000 7f46dfe00640 1 -- 172.21.15.83:0/2246919015 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46d8035030 con 0x7f46e8100c70 2024-04-24T09:15:05.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.825+0000 7f46f0f86740 1 Processor -- start 2024-04-24T09:15:05.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.825+0000 7f46f0f86740 1 -- start start 2024-04-24T09:15:05.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.826+0000 7f46f0f86740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565575a9ab40 0x565575a9af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:05.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.826+0000 7f46f0f86740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565575a9b460 0x565575aa55f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:05.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.826+0000 7f46f0f86740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565575aa5b30 0x565575aa7f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:05.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.826+0000 7f46f0f86740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x565575a001d0 con 0x565575aa5b30 2024-04-24T09:15:05.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.826+0000 7f46f0f86740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5655759a6450 con 0x565575a9b460 2024-04-24T09:15:05.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.826+0000 7f46f0f86740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5655759fc190 con 0x565575a9ab40 2024-04-24T09:15:05.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.826+0000 7f46dfe00640 1 -- 172.21.15.83:0/2246919015 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f46d8042070 con 0x7f46e8100c70 2024-04-24T09:15:05.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.826+0000 7f46e7e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565575a9ab40 0x565575a9af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:05.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.826+0000 7f46e7e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565575a9ab40 0x565575a9af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:60442/0 (socket says 172.21.15.83:60442) 2024-04-24T09:15:05.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.826+0000 7f46e7e00640 1 -- 172.21.15.83:0/4073788945 learned_addr learned my addr 172.21.15.83:0/4073788945 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:15:05.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.827+0000 7f46e6a00640 1 --2- 172.21.15.83:0/4073788945 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565575aa5b30 0x565575aa7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:05.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.827+0000 7f46e7400640 1 --2- 172.21.15.83:0/4073788945 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565575a9b460 0x565575aa55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:05.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.827+0000 7f46e7e00640 1 -- 172.21.15.83:0/4073788945 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565575a9b460 msgr2=0x565575aa55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:05.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.827+0000 7f46e7e00640 1 --2- 172.21.15.83:0/4073788945 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565575a9b460 0x565575aa55f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:05.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.827+0000 7f46e7e00640 1 -- 172.21.15.83:0/4073788945 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565575aa5b30 msgr2=0x565575aa7f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:05.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.827+0000 7f46e7e00640 1 --2- 172.21.15.83:0/4073788945 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565575aa5b30 0x565575aa7f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:05.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.827+0000 7f46e7e00640 1 -- 172.21.15.83:0/4073788945 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565575a63310 con 0x565575a9ab40 2024-04-24T09:15:05.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.827+0000 7f46e6a00640 1 --2- 172.21.15.83:0/4073788945 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565575aa5b30 0x565575aa7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:15:05.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.828+0000 7f46e7e00640 1 --2- 172.21.15.83:0/4073788945 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565575a9ab40 0x565575a9af20 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f46e0009e00 tx=0x7f46e0010740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2be771e420620eb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:05.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.828+0000 7f46d3e00640 1 -- 172.21.15.83:0/4073788945 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e0016070 con 0x565575a9ab40 2024-04-24T09:15:05.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.828+0000 7f46d3e00640 1 -- 172.21.15.83:0/4073788945 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f46e0004d20 con 0x565575a9ab40 2024-04-24T09:15:05.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.828+0000 7f46d3e00640 1 -- 172.21.15.83:0/4073788945 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e0012690 con 0x565575a9ab40 2024-04-24T09:15:05.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.829+0000 7f46dfe00640 1 --2- 172.21.15.83:0/2246919015 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f46cc0a3e60 0x7f46cc0a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:05.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.829+0000 7f46dfe00640 1 -- 172.21.15.83:0/2246919015 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9583+0+0 (secure 0 0 0) 0x7f46d80f12c0 con 0x7f46e8100c70 2024-04-24T09:15:05.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.829+0000 7f46e7400640 1 --2- 172.21.15.83:0/2246919015 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f46cc0a3e60 0x7f46cc0a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:05.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.829+0000 7f46f0f86740 1 -- 172.21.15.83:0/4073788945 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565575a9ab40 msgr2=0x565575a9af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:05.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.829+0000 7f46f0f86740 1 --2- 172.21.15.83:0/4073788945 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565575a9ab40 0x565575a9af20 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f46e0009e00 tx=0x7f46e0010740 comp rx=0 tx=0).stop 2024-04-24T09:15:05.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.829+0000 7f46f0f86740 1 -- 172.21.15.83:0/4073788945 shutdown_connections 2024-04-24T09:15:05.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.829+0000 7f46f0f86740 1 --2- 172.21.15.83:0/4073788945 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565575aa5b30 0x565575aa7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:05.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.829+0000 7f46f0f86740 1 --2- 172.21.15.83:0/4073788945 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565575a9b460 0x565575aa55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:05.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.829+0000 7f46f0f86740 1 --2- 172.21.15.83:0/4073788945 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565575a9ab40 0x565575a9af20 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:05.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.829+0000 7f46f0f86740 1 -- 172.21.15.83:0/4073788945 >> 172.21.15.83:0/4073788945 conn(0x565575a96e60 msgr2=0x565575a97240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:05.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.830+0000 7f46f0f86740 1 -- 172.21.15.83:0/4073788945 shutdown_connections 2024-04-24T09:15:05.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.830+0000 7f46e7400640 1 --2- 172.21.15.83:0/2246919015 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f46cc0a3e60 0x7f46cc0a6320 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x565575a9c100 tx=0x7f46d8009260 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:05.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.830+0000 7f46f0f86740 1 -- 172.21.15.83:0/4073788945 wait complete. 2024-04-24T09:15:05.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.830+0000 7f46f0f86740 1 -- start start 2024-04-24T09:15:05.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.831+0000 7f46f0f86740 1 Processor -- start 2024-04-24T09:15:05.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.831+0000 7f46f0f86740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565575a9ab40 0x565575aa5510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:05.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.831+0000 7f46f0f86740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565575a9b460 0x565575aa2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:05.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.831+0000 7f46f0f86740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565575aa5b30 0x565575aa2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:05.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.831+0000 7f46f0f86740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x565575aa89c0 con 0x565575a9ab40 2024-04-24T09:15:05.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.831+0000 7f46f0f86740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5655759b9810 con 0x565575aa5b30 2024-04-24T09:15:05.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.831+0000 7f46f0f86740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x565575aa8b40 con 0x565575a9b460 2024-04-24T09:15:05.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.831+0000 7f46f0f86740 10 client.0 _ll_register_callbacks cb 0x565575987140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:15:05.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.832+0000 7f46e7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565575a9ab40 0x565575aa5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:05.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.832+0000 7f46e6a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565575a9b460 0x565575aa2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:05.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.832+0000 7f46e7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565575a9ab40 0x565575aa5510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55094/0 (socket says 172.21.15.83:55094) 2024-04-24T09:15:05.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.832+0000 7f46e7e00640 1 -- 172.21.15.83:0/684764264 learned_addr learned my addr 172.21.15.83:0/684764264 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:15:05.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.832+0000 7f46e7e00640 1 --2- 172.21.15.83:0/684764264 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565575aa5b30 0x565575aa2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:05.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.833+0000 7f46e6a00640 1 -- 172.21.15.83:0/684764264 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565575aa5b30 msgr2=0x565575aa2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:05.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.833+0000 7f46e6a00640 1 --2- 172.21.15.83:0/684764264 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565575aa5b30 0x565575aa2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:05.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.833+0000 7f46e6a00640 1 -- 172.21.15.83:0/684764264 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565575a9ab40 msgr2=0x565575aa5510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:05.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.833+0000 7f46e6a00640 1 --2- 172.21.15.83:0/684764264 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565575a9ab40 0x565575aa5510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:05.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.833+0000 7f46e6a00640 1 -- 172.21.15.83:0/684764264 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5655759ca3f0 con 0x565575a9b460 2024-04-24T09:15:05.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.833+0000 7f46e7e00640 1 --2- 172.21.15.83:0/684764264 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565575a9ab40 0x565575aa5510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:15:05.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.833+0000 7f46e6a00640 1 --2- 172.21.15.83:0/684764264 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565575a9b460 0x565575aa2800 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f46e80ffdc0 tx=0x7f46d400bc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e81071337eb1505 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:05.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.833+0000 7f46d0c00640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-04-24T09:15:05.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.834+0000 7f46d0c00640 1 -- 172.21.15.83:0/684764264 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46d401b440 con 0x565575a9b460 2024-04-24T09:15:05.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.834+0000 7f46f0f86740 1 -- 172.21.15.83:0/684764264 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46e0004d20 con 0x565575a9b460 2024-04-24T09:15:05.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.834+0000 7f46d0c00640 1 -- 172.21.15.83:0/684764264 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(128..128 src has 1..128) ==== 9583+0+0 (secure 0 0 0) 0x7f46d401b5e0 con 0x565575a9b460 2024-04-24T09:15:05.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.835+0000 7f46d0c00640 1 -- 172.21.15.83:0/684764264 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7f46d4024050 con 0x565575a9b460 2024-04-24T09:15:05.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.835+0000 7f46f0f86740 10 client.30340 fetch_fsmap learned FSMap version 68 2024-04-24T09:15:05.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.835+0000 7f46f0f86740 1 -- 172.21.15.83:0/684764264 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) -- 0x5655757123f0 con 0x565575a9b460 2024-04-24T09:15:05.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.835+0000 7f46d0c00640 1 -- 172.21.15.83:0/684764264 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f46e0004d20 con 0x565575a9b460 2024-04-24T09:15:05.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.835+0000 7f46d0c00640 1 -- 172.21.15.83:0/684764264 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 68) ==== 8374+0+0 (secure 0 0 0) 0x7f46d4019020 con 0x565575a9b460 2024-04-24T09:15:05.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.835+0000 7f46f0f86740 10 client.30340 fetch_fsmap finished waiting for FSMap version 68 2024-04-24T09:15:05.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.835+0000 7f46f0f86740 10 client.30340 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:15:05.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.835+0000 7f46f0f86740 20 client.30340 populate_metadata read hostname 'smithi083' 2024-04-24T09:15:05.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.835+0000 7f46f0f86740 1 --2- 172.21.15.83:0/684764264 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x565575ab5db0 0x565575ab81a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:05.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.835+0000 7f46f0f86740 4 client.30340 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:15:05.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.835+0000 7f46f0f86740 1 -- 172.21.15.83:0/684764264 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565575a0dbf0 con 0x565575ab5db0 2024-04-24T09:15:05.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.836+0000 7f46e7e00640 1 --2- 172.21.15.83:0/684764264 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x565575ab5db0 0x565575ab81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:05.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.837+0000 7f46e7e00640 1 --2- 172.21.15.83:0/684764264 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x565575ab5db0 0x565575ab81a0 crc :-1 s=READY pgs=575 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5fb6f92f4c68bf02 server_cookie=71e1d108793f8304 in_seq=0 out_seq=0 2024-04-24T09:15:05.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.837+0000 7f46d0c00640 10 client.30340 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:15:05.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.838+0000 7f46d0c00640 1 -- 172.21.15.83:0/684764264 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x565575a0dbf0 con 0x565575ab5db0 2024-04-24T09:15:05.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.838+0000 7f46d0c00640 10 client.30340 handle_command_reply: tid=0 2024-04-24T09:15:05.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.838+0000 7f46f0f86740 1 client.30340 shutdown 2024-04-24T09:15:05.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.839+0000 7f46f0f86740 1 -- 172.21.15.83:0/684764264 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565575a9b460 msgr2=0x565575aa2800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:05.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.839+0000 7f46f0f86740 1 --2- 172.21.15.83:0/684764264 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565575a9b460 0x565575aa2800 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f46e80ffdc0 tx=0x7f46d400bc80 comp rx=0 tx=0).stop 2024-04-24T09:15:05.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.839+0000 7f46f0f86740 1 -- 172.21.15.83:0/684764264 shutdown_connections 2024-04-24T09:15:05.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.839+0000 7f46f0f86740 1 --2- 172.21.15.83:0/684764264 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x565575ab5db0 0x565575ab81a0 crc :-1 s=READY pgs=575 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:05.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.839+0000 7f46f0f86740 1 --2- 172.21.15.83:0/684764264 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565575aa5b30 0x565575aa2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:05.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.839+0000 7f46f0f86740 1 --2- 172.21.15.83:0/684764264 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565575a9b460 0x565575aa2800 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:05.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.839+0000 7f46d0c00640 0 client.30340 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:15:05.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.839+0000 7f46f0f86740 1 --2- 172.21.15.83:0/684764264 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565575a9ab40 0x565575aa5510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:05.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.839+0000 7f46f0f86740 1 -- 172.21.15.83:0/684764264 >> 172.21.15.83:0/684764264 conn(0x565575a96e60 msgr2=0x565575a9cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:05.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.839+0000 7f46f0f86740 1 -- 172.21.15.83:0/684764264 shutdown_connections 2024-04-24T09:15:05.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.839+0000 7f46f0f86740 1 -- 172.21.15.83:0/684764264 wait complete. 2024-04-24T09:15:05.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.839+0000 7f46f0f86740 20 client.30340 trim_cache size 0 max 16384 2024-04-24T09:15:05.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.869+0000 7f46f0f86740 1 Processor -- start 2024-04-24T09:15:05.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.870+0000 7f46f0f86740 1 -- start start 2024-04-24T09:15:05.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.870+0000 7f46f0f86740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565575a9ab40 0x565575ab1910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:05.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.871+0000 7f46e7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565575a9ab40 0x565575ab1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:05.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.871+0000 7f46e7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565575a9ab40 0x565575ab1910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55100/0 (socket says 172.21.15.83:55100) 2024-04-24T09:15:05.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.871+0000 7f46e7e00640 1 -- 172.21.15.83:0/3852011994 learned_addr learned my addr 172.21.15.83:0/3852011994 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:15:05.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.871+0000 7f46f0f86740 1 --2- 172.21.15.83:0/3852011994 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565575aa5b30 0x565575aa5040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:05.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.871+0000 7f46f0f86740 1 --2- 172.21.15.83:0/3852011994 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565575aa5580 0x565575abda90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:05.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.871+0000 7f46f0f86740 1 -- 172.21.15.83:0/3852011994 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f46c8010100 con 0x565575a9ab40 2024-04-24T09:15:05.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.871+0000 7f46f0f86740 1 -- 172.21.15.83:0/3852011994 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f46c8011cd0 con 0x565575aa5b30 2024-04-24T09:15:05.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.871+0000 7f46f0f86740 1 -- 172.21.15.83:0/3852011994 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f46c80107d0 con 0x565575aa5580 2024-04-24T09:15:05.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.872+0000 7f46e6a00640 1 --2- 172.21.15.83:0/3852011994 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565575aa5b30 0x565575aa5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:05.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.872+0000 7f46e6a00640 1 -- 172.21.15.83:0/3852011994 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565575aa5580 msgr2=0x565575abda90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:05.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.872+0000 7f46e6a00640 1 --2- 172.21.15.83:0/3852011994 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565575aa5580 0x565575abda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:05.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.872+0000 7f46e6a00640 1 -- 172.21.15.83:0/3852011994 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565575a9ab40 msgr2=0x565575ab1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:05.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.872+0000 7f46e6a00640 1 --2- 172.21.15.83:0/3852011994 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565575a9ab40 0x565575ab1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:05.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.872+0000 7f46e6a00640 1 -- 172.21.15.83:0/3852011994 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565575a9b290 con 0x565575aa5b30 2024-04-24T09:15:05.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.873+0000 7f46e7e00640 1 --2- 172.21.15.83:0/3852011994 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565575a9ab40 0x565575ab1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:15:05.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.874+0000 7f46e6a00640 1 --2- 172.21.15.83:0/3852011994 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565575aa5b30 0x565575aa5040 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x565575aa38c0 tx=0x7f46d4011f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c4762298bddd6397 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:05.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.875+0000 7f46d3400640 1 -- 172.21.15.83:0/3852011994 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46d402c070 con 0x565575aa5b30 2024-04-24T09:15:05.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.875+0000 7f46d3400640 1 -- 172.21.15.83:0/3852011994 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f46d4005850 con 0x565575aa5b30 2024-04-24T09:15:05.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.875+0000 7f46d3400640 1 -- 172.21.15.83:0/3852011994 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46d4011440 con 0x565575aa5b30 2024-04-24T09:15:05.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.876+0000 7f46f0f86740 1 -- 172.21.15.83:0/3852011994 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565575aa5b30 msgr2=0x565575aa5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:05.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.876+0000 7f46f0f86740 1 --2- 172.21.15.83:0/3852011994 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565575aa5b30 0x565575aa5040 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x565575aa38c0 tx=0x7f46d4011f90 comp rx=0 tx=0).stop 2024-04-24T09:15:05.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.877+0000 7f46f0f86740 1 -- 172.21.15.83:0/3852011994 shutdown_connections 2024-04-24T09:15:05.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.877+0000 7f46f0f86740 1 --2- 172.21.15.83:0/3852011994 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565575aa5580 0x565575abda90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:05.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.877+0000 7f46f0f86740 1 --2- 172.21.15.83:0/3852011994 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565575aa5b30 0x565575aa5040 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:05.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.877+0000 7f46f0f86740 1 --2- 172.21.15.83:0/3852011994 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565575a9ab40 0x565575ab1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:05.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.877+0000 7f46f0f86740 1 -- 172.21.15.83:0/3852011994 >> 172.21.15.83:0/3852011994 conn(0x565575a96e60 msgr2=0x565575a97240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:05.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.877+0000 7f46f0f86740 1 -- 172.21.15.83:0/3852011994 shutdown_connections 2024-04-24T09:15:05.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.877+0000 7f46f0f86740 1 -- 172.21.15.83:0/3852011994 wait complete. 2024-04-24T09:15:05.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.877+0000 7f46f0f86740 1 -- start start 2024-04-24T09:15:05.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.878+0000 7f46f0f86740 1 Processor -- start 2024-04-24T09:15:05.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.878+0000 7f46f0f86740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565575a9ab40 0x565575aae290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:05.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.878+0000 7f46f0f86740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565575aa5580 0x565575aa6510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:05.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.878+0000 7f46f0f86740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565575aa5b30 0x565575aa6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:05.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.878+0000 7f46f0f86740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f46c80115d0 con 0x565575aa5580 2024-04-24T09:15:05.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.879+0000 7f46f0f86740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f46c8010f00 con 0x565575aa5b30 2024-04-24T09:15:05.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.879+0000 7f46f0f86740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f46c800e100 con 0x565575a9ab40 2024-04-24T09:15:05.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.879+0000 7f46f0f86740 10 client.0 _ll_register_callbacks cb 0x7f46d4004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:15:05.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.879+0000 7f46e6a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565575aa5580 0x565575aa6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:05.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.879+0000 7f46e6a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565575aa5580 0x565575aa6510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55116/0 (socket says 172.21.15.83:55116) 2024-04-24T09:15:05.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.879+0000 7f46e6a00640 1 -- 172.21.15.83:0/3775788507 learned_addr learned my addr 172.21.15.83:0/3775788507 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:15:05.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.879+0000 7f46e7e00640 1 --2- 172.21.15.83:0/3775788507 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565575aa5b30 0x565575aa6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:05.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.879+0000 7f46e7e00640 1 --2- 172.21.15.83:0/3775788507 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565575a9ab40 0x565575aae290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:05.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.880+0000 7f46e7e00640 1 -- 172.21.15.83:0/3775788507 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565575aa5b30 msgr2=0x565575aa6ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:05.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.880+0000 7f46e7e00640 1 --2- 172.21.15.83:0/3775788507 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565575aa5b30 0x565575aa6ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:05.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.880+0000 7f46e7e00640 1 -- 172.21.15.83:0/3775788507 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565575aa5580 msgr2=0x565575aa6510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:05.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.880+0000 7f46e7e00640 1 --2- 172.21.15.83:0/3775788507 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565575aa5580 0x565575aa6510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:05.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.880+0000 7f46e7e00640 1 -- 172.21.15.83:0/3775788507 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5655757dc7d0 con 0x565575a9ab40 2024-04-24T09:15:05.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.880+0000 7f46e6a00640 1 --2- 172.21.15.83:0/3775788507 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565575aa5580 0x565575aa6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:15:05.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.881+0000 7f46e7e00640 1 --2- 172.21.15.83:0/3775788507 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565575a9ab40 0x565575aae290 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x565575ab2d20 tx=0x7f46e0009a00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12d9964b4ace6950 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:05.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.881+0000 7f46d0c00640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-04-24T09:15:05.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.881+0000 7f46d0c00640 1 -- 172.21.15.83:0/3775788507 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e0022040 con 0x565575a9ab40 2024-04-24T09:15:05.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.881+0000 7f46d0c00640 1 -- 172.21.15.83:0/3775788507 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(128..128 src has 1..128) ==== 9583+0+0 (secure 0 0 0) 0x7f46e002de00 con 0x565575a9ab40 2024-04-24T09:15:05.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.881+0000 7f46f0f86740 1 -- 172.21.15.83:0/3775788507 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46d4005850 con 0x565575a9ab40 2024-04-24T09:15:05.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.882+0000 7f46d0c00640 1 -- 172.21.15.83:0/3775788507 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7f46e0016040 con 0x565575a9ab40 2024-04-24T09:15:05.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.882+0000 7f46f0f86740 10 client.30343 fetch_fsmap learned FSMap version 68 2024-04-24T09:15:05.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.882+0000 7f46f0f86740 1 -- 172.21.15.83:0/3775788507 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) -- 0x565575b1c300 con 0x565575a9ab40 2024-04-24T09:15:05.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.883+0000 7f46d0c00640 1 -- 172.21.15.83:0/3775788507 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f46d4005850 con 0x565575a9ab40 2024-04-24T09:15:05.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.883+0000 7f46d0c00640 1 -- 172.21.15.83:0/3775788507 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 68) ==== 8374+0+0 (secure 0 0 0) 0x7f46e000f040 con 0x565575a9ab40 2024-04-24T09:15:05.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.883+0000 7f46f0f86740 10 client.30343 fetch_fsmap finished waiting for FSMap version 68 2024-04-24T09:15:05.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.883+0000 7f46f0f86740 10 client.30343 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:15:05.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.883+0000 7f46f0f86740 20 client.30343 populate_metadata read hostname 'smithi083' 2024-04-24T09:15:05.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.883+0000 7f46f0f86740 1 --2- 172.21.15.83:0/3775788507 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x565575b1c740 0x565575b1eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:05.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.883+0000 7f46f0f86740 4 client.30343 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:15:05.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.883+0000 7f46f0f86740 1 -- 172.21.15.83:0/3775788507 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "damage ls"}) -- 0x565575a07a60 con 0x565575b1c740 2024-04-24T09:15:05.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.883+0000 7f46e6a00640 1 --2- 172.21.15.83:0/3775788507 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x565575b1c740 0x565575b1eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:05.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.884+0000 7f46e6a00640 1 --2- 172.21.15.83:0/3775788507 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x565575b1c740 0x565575b1eb60 crc :-1 s=READY pgs=576 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a7a86f23a7a2af2 server_cookie=a5c75db7f098f66d in_seq=0 out_seq=0 2024-04-24T09:15:05.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.884+0000 7f46d0c00640 10 client.30343 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:15:05.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.884+0000 7f46d0c00640 1 -- 172.21.15.83:0/3775788507 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x565575a07a60 con 0x565575b1c740 2024-04-24T09:15:05.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.884+0000 7f46d0c00640 10 client.30343 handle_command_reply: tid=0 2024-04-24T09:15:05.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.884+0000 7f46f0f86740 1 client.30343 shutdown 2024-04-24T09:15:05.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.885+0000 7f46f0f86740 1 -- 172.21.15.83:0/3775788507 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565575a9ab40 msgr2=0x565575aae290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:05.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.885+0000 7f46f0f86740 1 --2- 172.21.15.83:0/3775788507 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565575a9ab40 0x565575aae290 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x565575ab2d20 tx=0x7f46e0009a00 comp rx=0 tx=0).stop 2024-04-24T09:15:05.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.885+0000 7f46f0f86740 1 -- 172.21.15.83:0/3775788507 shutdown_connections 2024-04-24T09:15:05.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.885+0000 7f46f0f86740 1 --2- 172.21.15.83:0/3775788507 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x565575b1c740 0x565575b1eb60 crc :-1 s=READY pgs=576 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:05.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.885+0000 7f46f0f86740 1 --2- 172.21.15.83:0/3775788507 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565575aa5b30 0x565575aa6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:05.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.885+0000 7f46d0c00640 0 client.30343 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:15:05.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.885+0000 7f46f0f86740 1 --2- 172.21.15.83:0/3775788507 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565575aa5580 0x565575aa6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:05.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.885+0000 7f46f0f86740 1 --2- 172.21.15.83:0/3775788507 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565575a9ab40 0x565575aae290 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:05.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.885+0000 7f46f0f86740 1 -- 172.21.15.83:0/3775788507 >> 172.21.15.83:0/3775788507 conn(0x565575a96e60 msgr2=0x565575a956c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:05.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.885+0000 7f46f0f86740 1 -- 172.21.15.83:0/3775788507 shutdown_connections 2024-04-24T09:15:05.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.886+0000 7f46f0f86740 1 -- 172.21.15.83:0/3775788507 wait complete. 2024-04-24T09:15:05.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.886+0000 7f46f0f86740 20 client.30343 trim_cache size 0 max 16384 2024-04-24T09:15:05.886 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:15:05.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.886+0000 7f46eca00640 1 -- 172.21.15.83:0/2246919015 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f46cc0a3e60 msgr2=0x7f46cc0a6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:05.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.886+0000 7f46eca00640 1 --2- 172.21.15.83:0/2246919015 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f46cc0a3e60 0x7f46cc0a6320 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x565575a9c100 tx=0x7f46d8009260 comp rx=0 tx=0).stop 2024-04-24T09:15:05.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.887+0000 7f46eca00640 1 -- 172.21.15.83:0/2246919015 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46e8100c70 msgr2=0x7f46e8108510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:05.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.887+0000 7f46eca00640 1 --2- 172.21.15.83:0/2246919015 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46e8100c70 0x7f46e8108510 secure :-1 s=READY pgs=2706 cs=0 l=1 rev1=1 crypto rx=0x7f46d8031440 tx=0x7f46d8003cd0 comp rx=0 tx=0).stop 2024-04-24T09:15:05.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.887+0000 7f46eca00640 1 -- 172.21.15.83:0/2246919015 shutdown_connections 2024-04-24T09:15:05.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.887+0000 7f46eca00640 1 --2- 172.21.15.83:0/2246919015 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f46cc0a3e60 0x7f46cc0a6320 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:05.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.887+0000 7f46eca00640 1 --2- 172.21.15.83:0/2246919015 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f46e810e820 0x7f46e8108bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:05.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.887+0000 7f46eca00640 1 --2- 172.21.15.83:0/2246919015 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46e8101620 0x7f46e810e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:05.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.887+0000 7f46eca00640 1 --2- 172.21.15.83:0/2246919015 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46e8100c70 0x7f46e8108510 unknown :-1 s=CLOSED pgs=2706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:05.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.887+0000 7f46eca00640 1 -- 172.21.15.83:0/2246919015 >> 172.21.15.83:0/2246919015 conn(0x7f46e80fc880 msgr2=0x7f46e80fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:05.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.887+0000 7f46eca00640 1 -- 172.21.15.83:0/2246919015 shutdown_connections 2024-04-24T09:15:05.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:05.887+0000 7f46eca00640 1 -- 172.21.15.83:0/2246919015 wait complete. 2024-04-24T09:15:06.209 INFO:tasks.workunit.client.0.smithi083.stdout: 10 70904 6.54 MB/sec execute 457 sec latency 144.819 ms 2024-04-24T09:15:06.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:15:06 smithi083 ceph-mon[27907]: pgmap v741: 129 pgs: 129 active+clean; 1.0 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.0 MiB/s wr, 238 op/s 2024-04-24T09:15:06.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:15:06 smithi138 ceph-mon[31441]: pgmap v741: 129 pgs: 129 active+clean; 1.0 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.0 MiB/s wr, 238 op/s 2024-04-24T09:15:06.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:15:06 smithi189 ceph-mon[31791]: pgmap v741: 129 pgs: 129 active+clean; 1.0 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.0 MiB/s wr, 238 op/s 2024-04-24T09:15:06.912 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T09:15:06.912 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T09:15:06.912 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T09:15:07.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.186+0000 7f8a4f200640 1 -- 172.21.15.83:0/2311647747 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a481060d0 msgr2=0x7f8a4810dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:07.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.186+0000 7f8a4f200640 1 --2- 172.21.15.83:0/2311647747 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a481060d0 0x7f8a4810dca0 secure :-1 s=READY pgs=2707 cs=0 l=1 rev1=1 crypto rx=0x7f8a44009be0 tx=0x7f8a4402a6c0 comp rx=0 tx=0).stop 2024-04-24T09:15:07.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.186+0000 7f8a4f200640 1 -- 172.21.15.83:0/2311647747 shutdown_connections 2024-04-24T09:15:07.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.187+0000 7f8a4f200640 1 --2- 172.21.15.83:0/2311647747 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a481060d0 0x7f8a4810dca0 unknown :-1 s=CLOSED pgs=2707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:07.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.187+0000 7f8a4f200640 1 --2- 172.21.15.83:0/2311647747 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a481019b0 0x7f8a481059f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:07.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.187+0000 7f8a4f200640 1 --2- 172.21.15.83:0/2311647747 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8a48101000 0x7f8a481013e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:07.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.187+0000 7f8a4f200640 1 -- 172.21.15.83:0/2311647747 >> 172.21.15.83:0/2311647747 conn(0x7f8a480fc880 msgr2=0x7f8a480feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:07.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.187+0000 7f8a4f200640 1 -- 172.21.15.83:0/2311647747 shutdown_connections 2024-04-24T09:15:07.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.187+0000 7f8a4f200640 1 -- 172.21.15.83:0/2311647747 wait complete. 2024-04-24T09:15:07.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.188+0000 7f8a4f200640 1 Processor -- start 2024-04-24T09:15:07.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.188+0000 7f8a4f200640 1 -- start start 2024-04-24T09:15:07.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.188+0000 7f8a4f200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a48101000 0x7f8a481a05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:07.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.188+0000 7f8a4f200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8a481060d0 0x7f8a481a0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:07.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.189+0000 7f8a4f200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a4819a6d0 0x7f8a4819ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:07.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.189+0000 7f8a4f200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8a4810ff30 con 0x7f8a48101000 2024-04-24T09:15:07.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.189+0000 7f8a4f200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8a4810fdb0 con 0x7f8a4819a6d0 2024-04-24T09:15:07.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.189+0000 7f8a4f200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8a481100b0 con 0x7f8a481060d0 2024-04-24T09:15:07.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.189+0000 7f8a4de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a48101000 0x7f8a481a05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:07.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.189+0000 7f8a4de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a48101000 0x7f8a481a05c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55140/0 (socket says 172.21.15.83:55140) 2024-04-24T09:15:07.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.189+0000 7f8a4de00640 1 -- 172.21.15.83:0/2399393007 learned_addr learned my addr 172.21.15.83:0/2399393007 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:15:07.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.189+0000 7f8a4d400640 1 --2- 172.21.15.83:0/2399393007 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8a481060d0 0x7f8a481a0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:07.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.189+0000 7f8a4e800640 1 --2- 172.21.15.83:0/2399393007 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a4819a6d0 0x7f8a4819ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:07.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.189+0000 7f8a4de00640 1 -- 172.21.15.83:0/2399393007 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8a481060d0 msgr2=0x7f8a481a0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:07.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.189+0000 7f8a4de00640 1 --2- 172.21.15.83:0/2399393007 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8a481060d0 0x7f8a481a0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:07.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.189+0000 7f8a4de00640 1 -- 172.21.15.83:0/2399393007 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a4819a6d0 msgr2=0x7f8a4819ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:07.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.189+0000 7f8a4de00640 1 --2- 172.21.15.83:0/2399393007 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a4819a6d0 0x7f8a4819ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:07.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.189+0000 7f8a4de00640 1 -- 172.21.15.83:0/2399393007 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a4819b2d0 con 0x7f8a48101000 2024-04-24T09:15:07.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.189+0000 7f8a4e800640 1 --2- 172.21.15.83:0/2399393007 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a4819a6d0 0x7f8a4819ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:15:07.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.190+0000 7f8a4de00640 1 --2- 172.21.15.83:0/2399393007 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a48101000 0x7f8a481a05c0 secure :-1 s=READY pgs=2708 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c00e9c0 tx=0x7f8a3c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:07.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.190+0000 7f8a42a00640 1 -- 172.21.15.83:0/2399393007 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a3c00ccc0 con 0x7f8a48101000 2024-04-24T09:15:07.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.190+0000 7f8a4f200640 1 -- 172.21.15.83:0/2399393007 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a48107c50 con 0x7f8a48101000 2024-04-24T09:15:07.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.190+0000 7f8a42a00640 1 -- 172.21.15.83:0/2399393007 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8a3c0054d0 con 0x7f8a48101000 2024-04-24T09:15:07.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.190+0000 7f8a4f200640 1 -- 172.21.15.83:0/2399393007 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a481081f0 con 0x7f8a48101000 2024-04-24T09:15:07.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.191+0000 7f8a42a00640 1 -- 172.21.15.83:0/2399393007 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a3c010430 con 0x7f8a48101000 2024-04-24T09:15:07.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.195+0000 7f8a535bc740 1 Processor -- start 2024-04-24T09:15:07.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.196+0000 7f8a535bc740 1 -- start start 2024-04-24T09:15:07.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.196+0000 7f8a535bc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2136d5270 0x55d2136d5650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:07.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.196+0000 7f8a535bc740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d2136d5b90 0x55d2136dfd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:07.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.196+0000 7f8a535bc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d2136e0260 0x55d2136e2650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:07.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.196+0000 7f8a535bc740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d21363ae70 con 0x55d2136e0260 2024-04-24T09:15:07.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.196+0000 7f8a535bc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d2135e1380 con 0x55d2136d5270 2024-04-24T09:15:07.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.196+0000 7f8a535bc740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d213636e30 con 0x55d2136d5b90 2024-04-24T09:15:07.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.196+0000 7f8a4e800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2136d5270 0x55d2136d5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:07.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.197+0000 7f8a4d400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d2136e0260 0x55d2136e2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:07.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.197+0000 7f8a4e800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2136d5270 0x55d2136d5650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:37404/0 (socket says 172.21.15.83:37404) 2024-04-24T09:15:07.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.197+0000 7f8a4e800640 1 -- 172.21.15.83:0/38831381 learned_addr learned my addr 172.21.15.83:0/38831381 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:15:07.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.197+0000 7f8a42a00640 1 -- 172.21.15.83:0/2399393007 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8a3c0105d0 con 0x7f8a48101000 2024-04-24T09:15:07.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.197+0000 7f8a4e800640 1 -- 172.21.15.83:0/38831381 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d2136d5b90 msgr2=0x55d2136dfd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:07.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.197+0000 7f8a4e800640 1 --2- 172.21.15.83:0/38831381 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d2136d5b90 0x55d2136dfd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:07.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.197+0000 7f8a4e800640 1 -- 172.21.15.83:0/38831381 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d2136e0260 msgr2=0x55d2136e2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:07.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.197+0000 7f8a4e800640 1 --2- 172.21.15.83:0/38831381 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d2136e0260 0x55d2136e2650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:07.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.197+0000 7f8a4e800640 1 -- 172.21.15.83:0/38831381 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2134157e0 con 0x55d2136d5270 2024-04-24T09:15:07.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.198+0000 7f8a4e800640 1 --2- 172.21.15.83:0/38831381 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2136d5270 0x55d2136d5650 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f8a44004af0 tx=0x7f8a440304b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a40b1974193e6cfd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:07.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.198+0000 7f8a42a00640 1 --2- 172.21.15.83:0/2399393007 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f8a300a3e10 0x7f8a300a62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:07.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.198+0000 7f8a3b000640 1 -- 172.21.15.83:0/38831381 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a44040030 con 0x55d2136d5270 2024-04-24T09:15:07.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.198+0000 7f8a3b000640 1 -- 172.21.15.83:0/38831381 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8a44037040 con 0x55d2136d5270 2024-04-24T09:15:07.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.198+0000 7f8a3b000640 1 -- 172.21.15.83:0/38831381 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a44036070 con 0x55d2136d5270 2024-04-24T09:15:07.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.199+0000 7f8a42a00640 1 -- 172.21.15.83:0/2399393007 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9583+0+0 (secure 0 0 0) 0x7f8a3c0d0ec0 con 0x7f8a48101000 2024-04-24T09:15:07.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.199+0000 7f8a4de00640 1 --2- 172.21.15.83:0/2399393007 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f8a300a3e10 0x7f8a300a62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:07.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.200+0000 7f8a4de00640 1 --2- 172.21.15.83:0/2399393007 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f8a300a3e10 0x7f8a300a62d0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x55d2136d6830 tx=0x7f8a3c09c7b0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:07.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.200+0000 7f8a535bc740 1 -- 172.21.15.83:0/38831381 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2136d5270 msgr2=0x55d2136d5650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:07.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.200+0000 7f8a535bc740 1 --2- 172.21.15.83:0/38831381 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2136d5270 0x55d2136d5650 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f8a44004af0 tx=0x7f8a440304b0 comp rx=0 tx=0).stop 2024-04-24T09:15:07.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.200+0000 7f8a535bc740 1 -- 172.21.15.83:0/38831381 shutdown_connections 2024-04-24T09:15:07.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.200+0000 7f8a535bc740 1 --2- 172.21.15.83:0/38831381 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d2136e0260 0x55d2136e2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:07.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.200+0000 7f8a535bc740 1 --2- 172.21.15.83:0/38831381 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d2136d5b90 0x55d2136dfd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:07.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.200+0000 7f8a535bc740 1 --2- 172.21.15.83:0/38831381 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2136d5270 0x55d2136d5650 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:07.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.200+0000 7f8a535bc740 1 -- 172.21.15.83:0/38831381 >> 172.21.15.83:0/38831381 conn(0x55d2136d1590 msgr2=0x55d2136d1970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:07.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.200+0000 7f8a535bc740 1 -- 172.21.15.83:0/38831381 shutdown_connections 2024-04-24T09:15:07.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.200+0000 7f8a535bc740 1 -- 172.21.15.83:0/38831381 wait complete. 2024-04-24T09:15:07.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.201+0000 7f8a535bc740 1 -- start start 2024-04-24T09:15:07.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.201+0000 7f8a535bc740 1 Processor -- start 2024-04-24T09:15:07.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.202+0000 7f8a535bc740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d2136d5270 0x55d2136dd130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:07.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.202+0000 7f8a535bc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d2136d5b90 0x55d2136dd670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:07.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.202+0000 7f8a535bc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2136e0260 0x55d2136ddbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:07.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.202+0000 7f8a535bc740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d2136e2ff0 con 0x55d2136d5b90 2024-04-24T09:15:07.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.202+0000 7f8a535bc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d2135f4660 con 0x55d2136e0260 2024-04-24T09:15:07.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.202+0000 7f8a535bc740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d2136e3170 con 0x55d2136d5270 2024-04-24T09:15:07.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.202+0000 7f8a4e800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d2136d5270 0x55d2136dd130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:07.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.202+0000 7f8a535bc740 10 client.0 _ll_register_callbacks cb 0x55d2135e2aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:15:07.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.203+0000 7f8a4e800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d2136d5270 0x55d2136dd130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:60478/0 (socket says 172.21.15.83:60478) 2024-04-24T09:15:07.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.203+0000 7f8a4e800640 1 -- 172.21.15.83:0/1231170812 learned_addr learned my addr 172.21.15.83:0/1231170812 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:15:07.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.203+0000 7f8a4d400640 1 --2- 172.21.15.83:0/1231170812 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d2136d5b90 0x55d2136dd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:07.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.203+0000 7f8a4e800640 1 --2- 172.21.15.83:0/1231170812 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2136e0260 0x55d2136ddbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:07.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.203+0000 7f8a4e800640 1 -- 172.21.15.83:0/1231170812 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2136e0260 msgr2=0x55d2136ddbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:07.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.203+0000 7f8a4e800640 1 --2- 172.21.15.83:0/1231170812 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2136e0260 0x55d2136ddbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:07.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.203+0000 7f8a4e800640 1 -- 172.21.15.83:0/1231170812 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d2136d5b90 msgr2=0x55d2136dd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:07.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.203+0000 7f8a4e800640 1 --2- 172.21.15.83:0/1231170812 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d2136d5b90 0x55d2136dd670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:07.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.203+0000 7f8a4e800640 1 -- 172.21.15.83:0/1231170812 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d213475f70 con 0x55d2136d5270 2024-04-24T09:15:07.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.204+0000 7f8a4e800640 1 --2- 172.21.15.83:0/1231170812 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d2136d5270 0x55d2136dd130 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8a44006010 tx=0x7f8a44030c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=79174c376cbe32ff server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:07.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.204+0000 7f8a2f400640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-04-24T09:15:07.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.204+0000 7f8a2f400640 1 -- 172.21.15.83:0/1231170812 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a44040070 con 0x55d2136d5270 2024-04-24T09:15:07.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.204+0000 7f8a2f400640 1 -- 172.21.15.83:0/1231170812 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(128..128 src has 1..128) ==== 9583+0+0 (secure 0 0 0) 0x7f8a44003b80 con 0x55d2136d5270 2024-04-24T09:15:07.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.204+0000 7f8a535bc740 1 -- 172.21.15.83:0/1231170812 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a44037040 con 0x55d2136d5270 2024-04-24T09:15:07.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.205+0000 7f8a2f400640 1 -- 172.21.15.83:0/1231170812 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7f8a44036040 con 0x55d2136d5270 2024-04-24T09:15:07.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.205+0000 7f8a535bc740 10 client.30352 fetch_fsmap learned FSMap version 68 2024-04-24T09:15:07.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.205+0000 7f8a535bc740 1 -- 172.21.15.83:0/1231170812 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) -- 0x55d21330d170 con 0x55d2136d5270 2024-04-24T09:15:07.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.205+0000 7f8a2f400640 1 -- 172.21.15.83:0/1231170812 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8a44037040 con 0x55d2136d5270 2024-04-24T09:15:07.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.206+0000 7f8a2f400640 1 -- 172.21.15.83:0/1231170812 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 68) ==== 8374+0+0 (secure 0 0 0) 0x7f8a44031440 con 0x55d2136d5270 2024-04-24T09:15:07.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.206+0000 7f8a535bc740 10 client.30352 fetch_fsmap finished waiting for FSMap version 68 2024-04-24T09:15:07.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.206+0000 7f8a535bc740 10 client.30352 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:15:07.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.206+0000 7f8a535bc740 20 client.30352 populate_metadata read hostname 'smithi083' 2024-04-24T09:15:07.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.206+0000 7f8a535bc740 1 --2- 172.21.15.83:0/1231170812 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d2136f03d0 0x55d2136f27c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:07.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.206+0000 7f8a535bc740 4 client.30352 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:15:07.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.206+0000 7f8a535bc740 1 -- 172.21.15.83:0/1231170812 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d213648890 con 0x55d2136f03d0 2024-04-24T09:15:07.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.206+0000 7f8a4d400640 1 --2- 172.21.15.83:0/1231170812 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d2136f03d0 0x55d2136f27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:07.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.207+0000 7f8a4d400640 1 --2- 172.21.15.83:0/1231170812 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d2136f03d0 0x55d2136f27c0 crc :-1 s=READY pgs=577 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=adbe0fc37b04586b server_cookie=f7a02d878b9d4fce in_seq=0 out_seq=0 2024-04-24T09:15:07.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.207+0000 7f8a2f400640 10 client.30352 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:15:07.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.209+0000 7f8a2f400640 1 -- 172.21.15.83:0/1231170812 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d213648890 con 0x55d2136f03d0 2024-04-24T09:15:07.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.209+0000 7f8a2f400640 10 client.30352 handle_command_reply: tid=0 2024-04-24T09:15:07.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.209+0000 7f8a535bc740 1 client.30352 shutdown 2024-04-24T09:15:07.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.209+0000 7f8a535bc740 1 -- 172.21.15.83:0/1231170812 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d2136d5270 msgr2=0x55d2136dd130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:07.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.209+0000 7f8a535bc740 1 --2- 172.21.15.83:0/1231170812 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d2136d5270 0x55d2136dd130 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8a44006010 tx=0x7f8a44030c90 comp rx=0 tx=0).stop 2024-04-24T09:15:07.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.209+0000 7f8a535bc740 1 -- 172.21.15.83:0/1231170812 shutdown_connections 2024-04-24T09:15:07.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.209+0000 7f8a535bc740 1 --2- 172.21.15.83:0/1231170812 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d2136f03d0 0x55d2136f27c0 crc :-1 s=READY pgs=577 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:07.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.209+0000 7f8a535bc740 1 --2- 172.21.15.83:0/1231170812 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2136e0260 0x55d2136ddbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:07.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.209+0000 7f8a535bc740 1 --2- 172.21.15.83:0/1231170812 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d2136d5b90 0x55d2136dd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:07.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.209+0000 7f8a535bc740 1 --2- 172.21.15.83:0/1231170812 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d2136d5270 0x55d2136dd130 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:07.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.209+0000 7f8a2f400640 0 client.30352 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:15:07.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.209+0000 7f8a535bc740 1 -- 172.21.15.83:0/1231170812 >> 172.21.15.83:0/1231170812 conn(0x55d2136d1590 msgr2=0x55d2136d7250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:07.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.210+0000 7f8a535bc740 1 -- 172.21.15.83:0/1231170812 shutdown_connections 2024-04-24T09:15:07.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.210+0000 7f8a535bc740 1 -- 172.21.15.83:0/1231170812 wait complete. 2024-04-24T09:15:07.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.210+0000 7f8a535bc740 20 client.30352 trim_cache size 0 max 16384 2024-04-24T09:15:07.210 INFO:tasks.workunit.client.0.smithi083.stdout: 10 71050 6.54 MB/sec execute 458 sec latency 70.432 ms 2024-04-24T09:15:07.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.237+0000 7f8a535bc740 1 Processor -- start 2024-04-24T09:15:07.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.237+0000 7f8a535bc740 1 -- start start 2024-04-24T09:15:07.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.237+0000 7f8a535bc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2136d5270 0x55d2136d79d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:07.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.237+0000 7f8a535bc740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d2136d5b90 0x55d2136ebb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:07.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.237+0000 7f8a535bc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d2136e0260 0x55d2136da3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:07.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.237+0000 7f8a535bc740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8a280100c0 con 0x55d2136e0260 2024-04-24T09:15:07.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.237+0000 7f8a535bc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8a28011c90 con 0x55d2136d5270 2024-04-24T09:15:07.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.237+0000 7f8a535bc740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8a28010790 con 0x55d2136d5b90 2024-04-24T09:15:07.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.238+0000 7f8a4e800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2136d5270 0x55d2136d79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:07.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.238+0000 7f8a4d400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d2136d5b90 0x55d2136ebb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:07.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.238+0000 7f8a4e800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2136d5270 0x55d2136d79d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:37416/0 (socket says 172.21.15.83:37416) 2024-04-24T09:15:07.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.238+0000 7f8a4d400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d2136d5b90 0x55d2136ebb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:60480/0 (socket says 172.21.15.83:60480) 2024-04-24T09:15:07.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.238+0000 7f8a4e800640 1 -- 172.21.15.83:0/2070313372 learned_addr learned my addr 172.21.15.83:0/2070313372 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:15:07.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.238+0000 7f8a4e800640 1 --2- 172.21.15.83:0/2070313372 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d2136e0260 0x55d2136da3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:07.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.238+0000 7f8a4e800640 1 -- 172.21.15.83:0/2070313372 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d2136d5b90 msgr2=0x55d2136ebb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:07.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.238+0000 7f8a4e800640 1 --2- 172.21.15.83:0/2070313372 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d2136d5b90 0x55d2136ebb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:07.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.238+0000 7f8a4e800640 1 -- 172.21.15.83:0/2070313372 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d2136e0260 msgr2=0x55d2136da3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:07.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.238+0000 7f8a4e800640 1 --2- 172.21.15.83:0/2070313372 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d2136e0260 0x55d2136da3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:07.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.238+0000 7f8a4e800640 1 -- 172.21.15.83:0/2070313372 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2136d59c0 con 0x55d2136d5270 2024-04-24T09:15:07.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.239+0000 7f8a4e800640 1 --2- 172.21.15.83:0/2070313372 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2136d5270 0x55d2136d79d0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f8a44002230 tx=0x7f8a4402a6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1083fdfa511eec81 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:07.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.239+0000 7f8a3a600640 1 -- 172.21.15.83:0/2070313372 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a44040070 con 0x55d2136d5270 2024-04-24T09:15:07.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.240+0000 7f8a3a600640 1 -- 172.21.15.83:0/2070313372 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8a44041040 con 0x55d2136d5270 2024-04-24T09:15:07.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.240+0000 7f8a3a600640 1 -- 172.21.15.83:0/2070313372 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a44031be0 con 0x55d2136d5270 2024-04-24T09:15:07.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.241+0000 7f8a535bc740 1 -- 172.21.15.83:0/2070313372 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2136d5270 msgr2=0x55d2136d79d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:07.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.241+0000 7f8a535bc740 1 --2- 172.21.15.83:0/2070313372 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2136d5270 0x55d2136d79d0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f8a44002230 tx=0x7f8a4402a6c0 comp rx=0 tx=0).stop 2024-04-24T09:15:07.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.241+0000 7f8a535bc740 1 -- 172.21.15.83:0/2070313372 shutdown_connections 2024-04-24T09:15:07.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.241+0000 7f8a535bc740 1 --2- 172.21.15.83:0/2070313372 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d2136e0260 0x55d2136da3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:07.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.241+0000 7f8a535bc740 1 --2- 172.21.15.83:0/2070313372 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d2136d5b90 0x55d2136ebb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:07.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.241+0000 7f8a535bc740 1 --2- 172.21.15.83:0/2070313372 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2136d5270 0x55d2136d79d0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:07.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.241+0000 7f8a535bc740 1 -- 172.21.15.83:0/2070313372 >> 172.21.15.83:0/2070313372 conn(0x55d2136d1590 msgr2=0x55d2136eb450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:07.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.241+0000 7f8a535bc740 1 -- 172.21.15.83:0/2070313372 shutdown_connections 2024-04-24T09:15:07.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.241+0000 7f8a535bc740 1 -- 172.21.15.83:0/2070313372 wait complete. 2024-04-24T09:15:07.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.242+0000 7f8a535bc740 1 -- start start 2024-04-24T09:15:07.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.242+0000 7f8a535bc740 1 Processor -- start 2024-04-24T09:15:07.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.243+0000 7f8a535bc740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d2136d5270 0x55d2136f5900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:07.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.243+0000 7f8a535bc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2136d5b90 0x55d2136f3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:07.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.243+0000 7f8a535bc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d2136e0260 0x55d2136f41e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:07.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.243+0000 7f8a535bc740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8a28011590 con 0x55d2136e0260 2024-04-24T09:15:07.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.243+0000 7f8a535bc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8a28010ec0 con 0x55d2136d5b90 2024-04-24T09:15:07.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.243+0000 7f8a535bc740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8a2800e0c0 con 0x55d2136d5270 2024-04-24T09:15:07.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.243+0000 7f8a535bc740 10 client.0 _ll_register_callbacks cb 0x7f8a34009530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:15:07.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.243+0000 7f8a4e800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d2136e0260 0x55d2136f41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:07.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.243+0000 7f8a4e800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d2136e0260 0x55d2136f41e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55184/0 (socket says 172.21.15.83:55184) 2024-04-24T09:15:07.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.243+0000 7f8a4e800640 1 -- 172.21.15.83:0/4115161012 learned_addr learned my addr 172.21.15.83:0/4115161012 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:15:07.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.243+0000 7f8a4e800640 1 --2- 172.21.15.83:0/4115161012 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d2136d5270 0x55d2136f5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:07.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.244+0000 7f8a4d400640 1 --2- 172.21.15.83:0/4115161012 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2136d5b90 0x55d2136f3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:07.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.244+0000 7f8a4e800640 1 -- 172.21.15.83:0/4115161012 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d2136d5270 msgr2=0x55d2136f5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:07.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.244+0000 7f8a4e800640 1 --2- 172.21.15.83:0/4115161012 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d2136d5270 0x55d2136f5900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:07.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.244+0000 7f8a4e800640 1 -- 172.21.15.83:0/4115161012 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2136d5b90 msgr2=0x55d2136f3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:07.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.244+0000 7f8a4e800640 1 --2- 172.21.15.83:0/4115161012 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2136d5b90 0x55d2136f3ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:07.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.244+0000 7f8a4e800640 1 -- 172.21.15.83:0/4115161012 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2136f4720 con 0x55d2136e0260 2024-04-24T09:15:07.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.244+0000 7f8a4e800640 1 --2- 172.21.15.83:0/4115161012 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d2136e0260 0x55d2136f41e0 secure :-1 s=READY pgs=2709 cs=0 l=1 rev1=1 crypto rx=0x7f8a4402abe0 tx=0x7f8a44048040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cff2dbdc62b8c908 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:07.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.244+0000 7f8a2f400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:15:07.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.244+0000 7f8a2f400640 1 -- 172.21.15.83:0/4115161012 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a44040070 con 0x55d2136e0260 2024-04-24T09:15:07.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.245+0000 7f8a2f400640 1 -- 172.21.15.83:0/4115161012 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(128..128 src has 1..128) ==== 9583+0+0 (secure 0 0 0) 0x7f8a44031c90 con 0x55d2136e0260 2024-04-24T09:15:07.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.245+0000 7f8a535bc740 1 -- 172.21.15.83:0/4115161012 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a44041040 con 0x55d2136e0260 2024-04-24T09:15:07.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.245+0000 7f8a2f400640 1 -- 172.21.15.83:0/4115161012 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7f8a44004b40 con 0x55d2136e0260 2024-04-24T09:15:07.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.245+0000 7f8a535bc740 10 client.23586 fetch_fsmap learned FSMap version 68 2024-04-24T09:15:07.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.245+0000 7f8a535bc740 1 -- 172.21.15.83:0/4115161012 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) -- 0x55d213757330 con 0x55d2136e0260 2024-04-24T09:15:07.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.246+0000 7f8a2f400640 1 -- 172.21.15.83:0/4115161012 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8a44041040 con 0x55d2136e0260 2024-04-24T09:15:07.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.246+0000 7f8a2f400640 1 -- 172.21.15.83:0/4115161012 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 68) ==== 8374+0+0 (secure 0 0 0) 0x7f8a44005800 con 0x55d2136e0260 2024-04-24T09:15:07.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.246+0000 7f8a535bc740 10 client.23586 fetch_fsmap finished waiting for FSMap version 68 2024-04-24T09:15:07.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.246+0000 7f8a535bc740 10 client.23586 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:15:07.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.246+0000 7f8a535bc740 20 client.23586 populate_metadata read hostname 'smithi083' 2024-04-24T09:15:07.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.246+0000 7f8a535bc740 1 --2- 172.21.15.83:0/4115161012 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d213757830 0x55d213759c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:07.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.246+0000 7f8a535bc740 4 client.23586 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T09:15:07.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.246+0000 7f8a535bc740 1 -- 172.21.15.83:0/4115161012 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55d213642700 con 0x55d213757830 2024-04-24T09:15:07.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.246+0000 7f8a4d400640 1 --2- 172.21.15.83:0/4115161012 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d213757830 0x55d213759c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:07.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.247+0000 7f8a4d400640 1 --2- 172.21.15.83:0/4115161012 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d213757830 0x55d213759c50 crc :-1 s=READY pgs=578 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2989c1a2ffe91006 server_cookie=f047d00d343c935c in_seq=0 out_seq=0 2024-04-24T09:15:07.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.247+0000 7f8a2f400640 10 client.23586 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:15:07.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.249+0000 7f8a2f400640 1 -- 172.21.15.83:0/4115161012 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d213642700 con 0x55d213757830 2024-04-24T09:15:07.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.249+0000 7f8a2f400640 10 client.23586 handle_command_reply: tid=0 2024-04-24T09:15:07.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.249+0000 7f8a535bc740 1 client.23586 shutdown 2024-04-24T09:15:07.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.249+0000 7f8a535bc740 1 -- 172.21.15.83:0/4115161012 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d2136e0260 msgr2=0x55d2136f41e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:07.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.249+0000 7f8a535bc740 1 --2- 172.21.15.83:0/4115161012 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d2136e0260 0x55d2136f41e0 secure :-1 s=READY pgs=2709 cs=0 l=1 rev1=1 crypto rx=0x7f8a4402abe0 tx=0x7f8a44048040 comp rx=0 tx=0).stop 2024-04-24T09:15:07.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.249+0000 7f8a535bc740 1 -- 172.21.15.83:0/4115161012 shutdown_connections 2024-04-24T09:15:07.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.249+0000 7f8a535bc740 1 --2- 172.21.15.83:0/4115161012 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d213757830 0x55d213759c50 crc :-1 s=READY pgs=578 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:07.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.249+0000 7f8a535bc740 1 --2- 172.21.15.83:0/4115161012 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d2136e0260 0x55d2136f41e0 unknown :-1 s=CLOSED pgs=2709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:07.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.249+0000 7f8a2f400640 0 client.23586 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:15:07.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.249+0000 7f8a535bc740 1 --2- 172.21.15.83:0/4115161012 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2136d5b90 0x55d2136f3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:07.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.249+0000 7f8a535bc740 1 --2- 172.21.15.83:0/4115161012 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d2136d5270 0x55d2136f5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:07.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.249+0000 7f8a535bc740 1 -- 172.21.15.83:0/4115161012 >> 172.21.15.83:0/4115161012 conn(0x55d2136d1590 msgr2=0x55d2136ebce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:07.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.250+0000 7f8a535bc740 1 -- 172.21.15.83:0/4115161012 shutdown_connections 2024-04-24T09:15:07.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.250+0000 7f8a535bc740 1 -- 172.21.15.83:0/4115161012 wait complete. 2024-04-24T09:15:07.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.250+0000 7f8a535bc740 20 client.23586 trim_cache size 0 max 16384 2024-04-24T09:15:07.250 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T09:15:07.251 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-04-24T09:15:07.251 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "bed76abd-1385-4070-a993-e3482666ecdb", 2024-04-24T09:15:07.251 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-04-24T09:15:07.251 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T09:15:07.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.251+0000 7f8a4f200640 1 -- 172.21.15.83:0/2399393007 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f8a300a3e10 msgr2=0x7f8a300a62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:07.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.251+0000 7f8a4f200640 1 --2- 172.21.15.83:0/2399393007 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f8a300a3e10 0x7f8a300a62d0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x55d2136d6830 tx=0x7f8a3c09c7b0 comp rx=0 tx=0).stop 2024-04-24T09:15:07.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.251+0000 7f8a4f200640 1 -- 172.21.15.83:0/2399393007 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a48101000 msgr2=0x7f8a481a05c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:07.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.251+0000 7f8a4f200640 1 --2- 172.21.15.83:0/2399393007 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a48101000 0x7f8a481a05c0 secure :-1 s=READY pgs=2708 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c00e9c0 tx=0x7f8a3c00ee80 comp rx=0 tx=0).stop 2024-04-24T09:15:07.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.251+0000 7f8a4f200640 1 -- 172.21.15.83:0/2399393007 shutdown_connections 2024-04-24T09:15:07.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.251+0000 7f8a4f200640 1 --2- 172.21.15.83:0/2399393007 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f8a300a3e10 0x7f8a300a62d0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:07.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.251+0000 7f8a4f200640 1 --2- 172.21.15.83:0/2399393007 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a4819a6d0 0x7f8a4819ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:07.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.251+0000 7f8a4f200640 1 --2- 172.21.15.83:0/2399393007 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8a481060d0 0x7f8a481a0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:07.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.251+0000 7f8a4f200640 1 --2- 172.21.15.83:0/2399393007 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a48101000 0x7f8a481a05c0 unknown :-1 s=CLOSED pgs=2708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:07.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.251+0000 7f8a4f200640 1 -- 172.21.15.83:0/2399393007 >> 172.21.15.83:0/2399393007 conn(0x7f8a480fc880 msgr2=0x7f8a480fdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:07.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.252+0000 7f8a4f200640 1 -- 172.21.15.83:0/2399393007 shutdown_connections 2024-04-24T09:15:07.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.252+0000 7f8a4f200640 1 -- 172.21.15.83:0/2399393007 wait complete. 2024-04-24T09:15:07.275 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T09:15:07.275 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T09:15:07.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.547+0000 7f3323800640 1 -- 172.21.15.83:0/3306567523 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f331c074bf0 msgr2=0x7f331c112b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:07.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.547+0000 7f3323800640 1 --2- 172.21.15.83:0/3306567523 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f331c074bf0 0x7f331c112b70 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f3308009900 tx=0x7f330802a480 comp rx=0 tx=0).stop 2024-04-24T09:15:07.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.547+0000 7f3323800640 1 -- 172.21.15.83:0/3306567523 shutdown_connections 2024-04-24T09:15:07.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.547+0000 7f3323800640 1 --2- 172.21.15.83:0/3306567523 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f331c1130b0 0x7f331c1154a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:07.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.547+0000 7f3323800640 1 --2- 172.21.15.83:0/3306567523 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f331c074bf0 0x7f331c112b70 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:07.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.547+0000 7f3323800640 1 --2- 172.21.15.83:0/3306567523 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f331c0742d0 0x7f331c0746b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:07.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.547+0000 7f3323800640 1 -- 172.21.15.83:0/3306567523 >> 172.21.15.83:0/3306567523 conn(0x7f331c079450 msgr2=0x7f331c079860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:07.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.547+0000 7f3323800640 1 -- 172.21.15.83:0/3306567523 shutdown_connections 2024-04-24T09:15:07.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.547+0000 7f3323800640 1 -- 172.21.15.83:0/3306567523 wait complete. 2024-04-24T09:15:07.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.548+0000 7f3323800640 1 Processor -- start 2024-04-24T09:15:07.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.549+0000 7f3323800640 1 -- start start 2024-04-24T09:15:07.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.549+0000 7f3323800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f331c0742d0 0x7f331c19ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:07.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.549+0000 7f3323800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f331c074bf0 0x7f331c19f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:07.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.549+0000 7f3323800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f331c1130b0 0x7f331c1a3730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:07.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.549+0000 7f3323800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f331c1188d0 con 0x7f331c074bf0 2024-04-24T09:15:07.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.549+0000 7f3323800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f331c118750 con 0x7f331c0742d0 2024-04-24T09:15:07.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.549+0000 7f3323800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f331c118a50 con 0x7f331c1130b0 2024-04-24T09:15:07.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.550+0000 7f331be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f331c074bf0 0x7f331c19f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:07.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.550+0000 7f331be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f331c074bf0 0x7f331c19f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55200/0 (socket says 172.21.15.83:55200) 2024-04-24T09:15:07.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.550+0000 7f331be00640 1 -- 172.21.15.83:0/2142272635 learned_addr learned my addr 172.21.15.83:0/2142272635 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:15:07.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.550+0000 7f3320e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f331c0742d0 0x7f331c19ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:07.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.550+0000 7f3321800640 1 --2- 172.21.15.83:0/2142272635 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f331c1130b0 0x7f331c1a3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:07.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.550+0000 7f3320e00640 1 -- 172.21.15.83:0/2142272635 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f331c1130b0 msgr2=0x7f331c1a3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:07.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.550+0000 7f3320e00640 1 --2- 172.21.15.83:0/2142272635 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f331c1130b0 0x7f331c1a3730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:07.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.550+0000 7f3320e00640 1 -- 172.21.15.83:0/2142272635 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f331c074bf0 msgr2=0x7f331c19f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:07.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.550+0000 7f3320e00640 1 --2- 172.21.15.83:0/2142272635 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f331c074bf0 0x7f331c19f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:07.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.550+0000 7f3320e00640 1 -- 172.21.15.83:0/2142272635 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f331c1a3e60 con 0x7f331c0742d0 2024-04-24T09:15:07.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.550+0000 7f3321800640 1 --2- 172.21.15.83:0/2142272635 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f331c1130b0 0x7f331c1a3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:15:07.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.551+0000 7f3320e00640 1 --2- 172.21.15.83:0/2142272635 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f331c0742d0 0x7f331c19ee70 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f331000d6d0 tx=0x7f331000db90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:07.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.551+0000 7f3319600640 1 -- 172.21.15.83:0/2142272635 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3310017070 con 0x7f331c0742d0 2024-04-24T09:15:07.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.551+0000 7f3319600640 1 -- 172.21.15.83:0/2142272635 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3310009820 con 0x7f331c0742d0 2024-04-24T09:15:07.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.551+0000 7f3319600640 1 -- 172.21.15.83:0/2142272635 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3310004c40 con 0x7f331c0742d0 2024-04-24T09:15:07.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.551+0000 7f3323800640 1 -- 172.21.15.83:0/2142272635 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f331c1a4150 con 0x7f331c0742d0 2024-04-24T09:15:07.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.551+0000 7f3323800640 1 -- 172.21.15.83:0/2142272635 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f331c1ab9e0 con 0x7f331c0742d0 2024-04-24T09:15:07.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.555+0000 7f33265fa740 1 Processor -- start 2024-04-24T09:15:07.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.556+0000 7f33265fa740 1 -- start start 2024-04-24T09:15:07.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.556+0000 7f3319600640 1 -- 172.21.15.83:0/2142272635 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f33100099d0 con 0x7f331c0742d0 2024-04-24T09:15:07.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.556+0000 7f33265fa740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56471f9a8860 0x56471f9a8c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:07.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.556+0000 7f33265fa740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56471f9a9180 0x56471f9b3310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:07.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.556+0000 7f33265fa740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56471f9b3850 0x56471f9b5c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:07.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.556+0000 7f33265fa740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56471f90e1f0 con 0x56471f9a8860 2024-04-24T09:15:07.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.556+0000 7f33265fa740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56471f8b4350 con 0x56471f9a9180 2024-04-24T09:15:07.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.556+0000 7f33265fa740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56471f90a1b0 con 0x56471f9b3850 2024-04-24T09:15:07.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.556+0000 7f3321800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56471f9a8860 0x56471f9a8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:07.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.557+0000 7f3321800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56471f9a8860 0x56471f9a8c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55210/0 (socket says 172.21.15.83:55210) 2024-04-24T09:15:07.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.557+0000 7f3321800640 1 -- 172.21.15.83:0/3691759917 learned_addr learned my addr 172.21.15.83:0/3691759917 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:15:07.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.557+0000 7f3320e00640 1 --2- 172.21.15.83:0/3691759917 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56471f9a9180 0x56471f9b3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:07.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.557+0000 7f331be00640 1 --2- 172.21.15.83:0/3691759917 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56471f9b3850 0x56471f9b5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:07.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.557+0000 7f3321800640 1 -- 172.21.15.83:0/3691759917 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56471f9b3850 msgr2=0x56471f9b5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:07.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.557+0000 7f3321800640 1 --2- 172.21.15.83:0/3691759917 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56471f9b3850 0x56471f9b5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:07.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.557+0000 7f3321800640 1 -- 172.21.15.83:0/3691759917 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56471f9a9180 msgr2=0x56471f9b3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:07.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.557+0000 7f3321800640 1 --2- 172.21.15.83:0/3691759917 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56471f9a9180 0x56471f9b3310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:07.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.557+0000 7f3321800640 1 -- 172.21.15.83:0/3691759917 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56471f908f50 con 0x56471f9a8860 2024-04-24T09:15:07.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.557+0000 7f3321800640 1 --2- 172.21.15.83:0/3691759917 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56471f9a8860 0x56471f9a8c40 secure :-1 s=READY pgs=2710 cs=0 l=1 rev1=1 crypto rx=0x7f3314009fc0 tx=0x7f3314010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f838031669fd8a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:07.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.558+0000 7f330e000640 1 -- 172.21.15.83:0/3691759917 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3314011700 con 0x56471f9a8860 2024-04-24T09:15:07.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.558+0000 7f330e000640 1 -- 172.21.15.83:0/3691759917 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3314011d50 con 0x56471f9a8860 2024-04-24T09:15:07.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.558+0000 7f330e000640 1 -- 172.21.15.83:0/3691759917 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f331401ce50 con 0x56471f9a8860 2024-04-24T09:15:07.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.558+0000 7f3319600640 1 --2- 172.21.15.83:0/2142272635 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f33040a3da0 0x7f33040a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:07.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.558+0000 7f3319600640 1 -- 172.21.15.83:0/2142272635 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9583+0+0 (secure 0 0 0) 0x7f33100d0500 con 0x7f331c0742d0 2024-04-24T09:15:07.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.558+0000 7f3320e00640 1 --2- 172.21.15.83:0/2142272635 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f33040a3da0 0x7f33040a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:07.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.559+0000 7f33265fa740 1 -- 172.21.15.83:0/3691759917 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56471f9a8860 msgr2=0x56471f9a8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:07.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.559+0000 7f33265fa740 1 --2- 172.21.15.83:0/3691759917 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56471f9a8860 0x56471f9a8c40 secure :-1 s=READY pgs=2710 cs=0 l=1 rev1=1 crypto rx=0x7f3314009fc0 tx=0x7f3314010860 comp rx=0 tx=0).stop 2024-04-24T09:15:07.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.559+0000 7f33265fa740 1 -- 172.21.15.83:0/3691759917 shutdown_connections 2024-04-24T09:15:07.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.559+0000 7f33265fa740 1 --2- 172.21.15.83:0/3691759917 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56471f9b3850 0x56471f9b5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:07.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.559+0000 7f33265fa740 1 --2- 172.21.15.83:0/3691759917 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56471f9a9180 0x56471f9b3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:07.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.559+0000 7f33265fa740 1 --2- 172.21.15.83:0/3691759917 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56471f9a8860 0x56471f9a8c40 unknown :-1 s=CLOSED pgs=2710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:07.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.559+0000 7f33265fa740 1 -- 172.21.15.83:0/3691759917 >> 172.21.15.83:0/3691759917 conn(0x56471f9a4b80 msgr2=0x56471f9a4f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:07.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.560+0000 7f33265fa740 1 -- 172.21.15.83:0/3691759917 shutdown_connections 2024-04-24T09:15:07.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.560+0000 7f3320e00640 1 --2- 172.21.15.83:0/2142272635 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f33040a3da0 0x7f33040a6260 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x56471f9a9e20 tx=0x7f3310009f30 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:07.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.560+0000 7f33265fa740 1 -- 172.21.15.83:0/3691759917 wait complete. 2024-04-24T09:15:07.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.560+0000 7f33265fa740 1 -- start start 2024-04-24T09:15:07.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.560+0000 7f33265fa740 1 Processor -- start 2024-04-24T09:15:07.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.561+0000 7f33265fa740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56471f9a8860 0x56471f9b0840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:07.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.561+0000 7f33265fa740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56471f9a9180 0x56471f9b0d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:07.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.561+0000 7f33265fa740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56471f9b3850 0x56471f9b12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:07.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.561+0000 7f33265fa740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56471f9b6640 con 0x56471f9b3850 2024-04-24T09:15:07.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.561+0000 7f33265fa740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56471f8c7690 con 0x56471f9a8860 2024-04-24T09:15:07.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.561+0000 7f33265fa740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56471f9b67c0 con 0x56471f9a9180 2024-04-24T09:15:07.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.561+0000 7f33265fa740 10 client.0 _ll_register_callbacks cb 0x56471f903650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:15:07.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.561+0000 7f3321800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56471f9b3850 0x56471f9b12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:07.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.561+0000 7f3321800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56471f9b3850 0x56471f9b12c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55216/0 (socket says 172.21.15.83:55216) 2024-04-24T09:15:07.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.561+0000 7f3321800640 1 -- 172.21.15.83:0/3139210864 learned_addr learned my addr 172.21.15.83:0/3139210864 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:15:07.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.562+0000 7f331be00640 1 --2- 172.21.15.83:0/3139210864 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56471f9a9180 0x56471f9b0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:07.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.562+0000 7f3321800640 1 --2- 172.21.15.83:0/3139210864 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56471f9a8860 0x56471f9b0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:07.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.562+0000 7f3321800640 1 -- 172.21.15.83:0/3139210864 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56471f9a9180 msgr2=0x56471f9b0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:07.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.562+0000 7f3321800640 1 --2- 172.21.15.83:0/3139210864 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56471f9a9180 0x56471f9b0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:07.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.562+0000 7f3321800640 1 -- 172.21.15.83:0/3139210864 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56471f9a8860 msgr2=0x56471f9b0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:07.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.562+0000 7f3321800640 1 --2- 172.21.15.83:0/3139210864 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56471f9a8860 0x56471f9b0840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:07.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.562+0000 7f3321800640 1 -- 172.21.15.83:0/3139210864 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56471f8c6920 con 0x56471f9b3850 2024-04-24T09:15:07.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.563+0000 7f3321800640 1 --2- 172.21.15.83:0/3139210864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56471f9b3850 0x56471f9b12c0 secure :-1 s=READY pgs=2711 cs=0 l=1 rev1=1 crypto rx=0x7f331401c7d0 tx=0x7f331401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c2224fa1acc4504 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:07.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.563+0000 7f3302a00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:15:07.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.563+0000 7f3302a00640 1 -- 172.21.15.83:0/3139210864 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3314004c40 con 0x56471f9b3850 2024-04-24T09:15:07.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.563+0000 7f33265fa740 1 -- 172.21.15.83:0/3139210864 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3314011d50 con 0x56471f9b3850 2024-04-24T09:15:07.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.563+0000 7f3302a00640 1 -- 172.21.15.83:0/3139210864 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(128..128 src has 1..128) ==== 9583+0+0 (secure 0 0 0) 0x7f3314004de0 con 0x56471f9b3850 2024-04-24T09:15:07.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.563+0000 7f3302a00640 1 -- 172.21.15.83:0/3139210864 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7f3314032050 con 0x56471f9b3850 2024-04-24T09:15:07.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.563+0000 7f33265fa740 10 client.23598 fetch_fsmap learned FSMap version 68 2024-04-24T09:15:07.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.563+0000 7f33265fa740 1 -- 172.21.15.83:0/3139210864 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) -- 0x56471f6ea7d0 con 0x56471f9b3850 2024-04-24T09:15:07.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.564+0000 7f3302a00640 1 -- 172.21.15.83:0/3139210864 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3314011d50 con 0x56471f9b3850 2024-04-24T09:15:07.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.564+0000 7f3302a00640 1 -- 172.21.15.83:0/3139210864 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 68) ==== 8374+0+0 (secure 0 0 0) 0x7f3314028400 con 0x56471f9b3850 2024-04-24T09:15:07.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.564+0000 7f33265fa740 10 client.23598 fetch_fsmap finished waiting for FSMap version 68 2024-04-24T09:15:07.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.564+0000 7f33265fa740 10 client.23598 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:15:07.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.564+0000 7f33265fa740 20 client.23598 populate_metadata read hostname 'smithi083' 2024-04-24T09:15:07.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.564+0000 7f33265fa740 1 --2- 172.21.15.83:0/3139210864 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56471f9c3b50 0x56471f9c5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:07.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.564+0000 7f33265fa740 4 client.23598 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:15:07.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.564+0000 7f33265fa740 1 -- 172.21.15.83:0/3139210864 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56471f91bc10 con 0x56471f9c3b50 2024-04-24T09:15:07.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.565+0000 7f331be00640 1 --2- 172.21.15.83:0/3139210864 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56471f9c3b50 0x56471f9c5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:07.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.565+0000 7f331be00640 1 --2- 172.21.15.83:0/3139210864 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56471f9c3b50 0x56471f9c5f40 crc :-1 s=READY pgs=579 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac3837810ca9cc22 server_cookie=cf73f4da4dd3fbf2 in_seq=0 out_seq=0 2024-04-24T09:15:07.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.565+0000 7f3302a00640 10 client.23598 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:15:07.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.566+0000 7f3302a00640 1 -- 172.21.15.83:0/3139210864 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56471f91bc10 con 0x56471f9c3b50 2024-04-24T09:15:07.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.566+0000 7f3302a00640 10 client.23598 handle_command_reply: tid=0 2024-04-24T09:15:07.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.567+0000 7f33265fa740 1 client.23598 shutdown 2024-04-24T09:15:07.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.567+0000 7f33265fa740 1 -- 172.21.15.83:0/3139210864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56471f9b3850 msgr2=0x56471f9b12c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:07.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.567+0000 7f33265fa740 1 --2- 172.21.15.83:0/3139210864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56471f9b3850 0x56471f9b12c0 secure :-1 s=READY pgs=2711 cs=0 l=1 rev1=1 crypto rx=0x7f331401c7d0 tx=0x7f331401c9f0 comp rx=0 tx=0).stop 2024-04-24T09:15:07.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.567+0000 7f33265fa740 1 -- 172.21.15.83:0/3139210864 shutdown_connections 2024-04-24T09:15:07.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.567+0000 7f33265fa740 1 --2- 172.21.15.83:0/3139210864 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56471f9c3b50 0x56471f9c5f40 crc :-1 s=READY pgs=579 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:07.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.567+0000 7f33265fa740 1 --2- 172.21.15.83:0/3139210864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56471f9b3850 0x56471f9b12c0 unknown :-1 s=CLOSED pgs=2711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:07.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.567+0000 7f33265fa740 1 --2- 172.21.15.83:0/3139210864 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56471f9a9180 0x56471f9b0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:07.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.567+0000 7f33265fa740 1 --2- 172.21.15.83:0/3139210864 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56471f9a8860 0x56471f9b0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:07.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.567+0000 7f33265fa740 1 -- 172.21.15.83:0/3139210864 >> 172.21.15.83:0/3139210864 conn(0x56471f9a4b80 msgr2=0x56471f9aa840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:07.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.567+0000 7f3302a00640 0 client.23598 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:15:07.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.568+0000 7f33265fa740 1 -- 172.21.15.83:0/3139210864 shutdown_connections 2024-04-24T09:15:07.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.568+0000 7f33265fa740 1 -- 172.21.15.83:0/3139210864 wait complete. 2024-04-24T09:15:07.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.568+0000 7f33265fa740 20 client.23598 trim_cache size 0 max 16384 2024-04-24T09:15:07.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.601+0000 7f33265fa740 1 Processor -- start 2024-04-24T09:15:07.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.601+0000 7f33265fa740 1 -- start start 2024-04-24T09:15:07.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.601+0000 7f33265fa740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56471f9a8860 0x56471f9bf530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:07.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.601+0000 7f33265fa740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56471f9a9180 0x56471f9cb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:07.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.601+0000 7f33265fa740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56471f9b3850 0x56471f9cbb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:07.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.601+0000 7f33265fa740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f331c1b5dd0 con 0x56471f9a8860 2024-04-24T09:15:07.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.601+0000 7f33265fa740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f331c1b79a0 con 0x56471f9b3850 2024-04-24T09:15:07.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.601+0000 7f33265fa740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f331c1b64a0 con 0x56471f9a9180 2024-04-24T09:15:07.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.602+0000 7f3321800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56471f9a8860 0x56471f9bf530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:07.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.602+0000 7f3321800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56471f9a8860 0x56471f9bf530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55220/0 (socket says 172.21.15.83:55220) 2024-04-24T09:15:07.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.602+0000 7f3321800640 1 -- 172.21.15.83:0/2236886357 learned_addr learned my addr 172.21.15.83:0/2236886357 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:15:07.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.602+0000 7f331be00640 1 --2- 172.21.15.83:0/2236886357 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56471f9a9180 0x56471f9cb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:07.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.603+0000 7f3321800640 1 --2- 172.21.15.83:0/2236886357 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56471f9b3850 0x56471f9cbb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:07.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.603+0000 7f3321800640 1 -- 172.21.15.83:0/2236886357 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56471f9a9180 msgr2=0x56471f9cb5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:07.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.603+0000 7f3321800640 1 --2- 172.21.15.83:0/2236886357 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56471f9a9180 0x56471f9cb5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:07.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.603+0000 7f3321800640 1 -- 172.21.15.83:0/2236886357 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56471f9b3850 msgr2=0x56471f9cbb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:07.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.603+0000 7f3321800640 1 --2- 172.21.15.83:0/2236886357 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56471f9b3850 0x56471f9cbb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:07.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.603+0000 7f3321800640 1 -- 172.21.15.83:0/2236886357 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56471f6203f0 con 0x56471f9a8860 2024-04-24T09:15:07.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.603+0000 7f331be00640 1 --2- 172.21.15.83:0/2236886357 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56471f9a9180 0x56471f9cb5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:15:07.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.603+0000 7f3321800640 1 --2- 172.21.15.83:0/2236886357 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56471f9a8860 0x56471f9bf530 secure :-1 s=READY pgs=2712 cs=0 l=1 rev1=1 crypto rx=0x56471f9cc050 tx=0x7f3314005e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6638df2aa6a4f2e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:07.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.604+0000 7f330d600640 1 -- 172.21.15.83:0/2236886357 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f331401c800 con 0x56471f9a8860 2024-04-24T09:15:07.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.604+0000 7f330d600640 1 -- 172.21.15.83:0/2236886357 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3314028400 con 0x56471f9a8860 2024-04-24T09:15:07.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.604+0000 7f330d600640 1 -- 172.21.15.83:0/2236886357 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3314005020 con 0x56471f9a8860 2024-04-24T09:15:07.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.605+0000 7f33265fa740 1 -- 172.21.15.83:0/2236886357 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56471f9a8860 msgr2=0x56471f9bf530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:07.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.605+0000 7f33265fa740 1 --2- 172.21.15.83:0/2236886357 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56471f9a8860 0x56471f9bf530 secure :-1 s=READY pgs=2712 cs=0 l=1 rev1=1 crypto rx=0x56471f9cc050 tx=0x7f3314005e70 comp rx=0 tx=0).stop 2024-04-24T09:15:07.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.606+0000 7f33265fa740 1 -- 172.21.15.83:0/2236886357 shutdown_connections 2024-04-24T09:15:07.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.606+0000 7f33265fa740 1 --2- 172.21.15.83:0/2236886357 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56471f9b3850 0x56471f9cbb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:07.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.606+0000 7f33265fa740 1 --2- 172.21.15.83:0/2236886357 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56471f9a9180 0x56471f9cb5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:07.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.606+0000 7f33265fa740 1 --2- 172.21.15.83:0/2236886357 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56471f9a8860 0x56471f9bf530 unknown :-1 s=CLOSED pgs=2712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:07.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.606+0000 7f33265fa740 1 -- 172.21.15.83:0/2236886357 >> 172.21.15.83:0/2236886357 conn(0x56471f9a4b80 msgr2=0x56471f9b1090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:07.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.606+0000 7f33265fa740 1 -- 172.21.15.83:0/2236886357 shutdown_connections 2024-04-24T09:15:07.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.606+0000 7f33265fa740 1 -- 172.21.15.83:0/2236886357 wait complete. 2024-04-24T09:15:07.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.607+0000 7f33265fa740 1 -- start start 2024-04-24T09:15:07.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.607+0000 7f33265fa740 1 Processor -- start 2024-04-24T09:15:07.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.608+0000 7f33265fa740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56471f9a8860 0x56471fa25730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:07.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.608+0000 7f33265fa740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56471f9a9180 0x56471f9bba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:07.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.608+0000 7f33265fa740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56471f9b3850 0x56471f9bbf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:07.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.608+0000 7f33265fa740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f331c1b72a0 con 0x56471f9a9180 2024-04-24T09:15:07.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.608+0000 7f33265fa740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f331c1b6bd0 con 0x56471f9b3850 2024-04-24T09:15:07.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.608+0000 7f33265fa740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f331c1b3dd0 con 0x56471f9a8860 2024-04-24T09:15:07.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.608+0000 7f33265fa740 10 client.0 _ll_register_callbacks cb 0x7f3314009e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:15:07.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.608+0000 7f331be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56471f9a9180 0x56471f9bba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:07.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.609+0000 7f331be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56471f9a9180 0x56471f9bba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55228/0 (socket says 172.21.15.83:55228) 2024-04-24T09:15:07.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.609+0000 7f331be00640 1 -- 172.21.15.83:0/2063790168 learned_addr learned my addr 172.21.15.83:0/2063790168 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:15:07.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.609+0000 7f3321800640 1 --2- 172.21.15.83:0/2063790168 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56471f9b3850 0x56471f9bbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:07.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.609+0000 7f3321800640 1 --2- 172.21.15.83:0/2063790168 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56471f9a8860 0x56471fa25730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:07.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.609+0000 7f331be00640 1 -- 172.21.15.83:0/2063790168 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56471f9a8860 msgr2=0x56471fa25730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:07.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.609+0000 7f331be00640 1 --2- 172.21.15.83:0/2063790168 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56471f9a8860 0x56471fa25730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:07.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.609+0000 7f331be00640 1 -- 172.21.15.83:0/2063790168 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56471f9b3850 msgr2=0x56471f9bbf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:07.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.609+0000 7f331be00640 1 --2- 172.21.15.83:0/2063790168 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56471f9b3850 0x56471f9bbf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:07.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.609+0000 7f331be00640 1 -- 172.21.15.83:0/2063790168 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56471f5e00f0 con 0x56471f9a9180 2024-04-24T09:15:07.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.609+0000 7f3321800640 1 --2- 172.21.15.83:0/2063790168 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56471f9a8860 0x56471fa25730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:15:07.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.610+0000 7f331be00640 1 --2- 172.21.15.83:0/2063790168 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56471f9a9180 0x56471f9bba40 secure :-1 s=READY pgs=2713 cs=0 l=1 rev1=1 crypto rx=0x7f330802a4c0 tx=0x7f33080044a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a38dbf67cfedff5f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:07.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.610+0000 7f3302a00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:15:07.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.610+0000 7f3302a00640 1 -- 172.21.15.83:0/2063790168 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3308049070 con 0x56471f9a9180 2024-04-24T09:15:07.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.611+0000 7f3302a00640 1 -- 172.21.15.83:0/2063790168 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(128..128 src has 1..128) ==== 9583+0+0 (secure 0 0 0) 0x7f3308005020 con 0x56471f9a9180 2024-04-24T09:15:07.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.611+0000 7f33265fa740 1 -- 172.21.15.83:0/2063790168 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3314028400 con 0x56471f9a9180 2024-04-24T09:15:07.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.612+0000 7f3302a00640 1 -- 172.21.15.83:0/2063790168 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7f3308052050 con 0x56471f9a9180 2024-04-24T09:15:07.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.612+0000 7f33265fa740 10 client.23604 fetch_fsmap learned FSMap version 68 2024-04-24T09:15:07.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.612+0000 7f33265fa740 1 -- 172.21.15.83:0/2063790168 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) -- 0x56471f8d3810 con 0x56471f9a9180 2024-04-24T09:15:07.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.613+0000 7f3302a00640 1 -- 172.21.15.83:0/2063790168 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3314028400 con 0x56471f9a9180 2024-04-24T09:15:07.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.613+0000 7f3302a00640 1 -- 172.21.15.83:0/2063790168 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 68) ==== 8374+0+0 (secure 0 0 0) 0x7f33080047a0 con 0x56471f9a9180 2024-04-24T09:15:07.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.613+0000 7f33265fa740 10 client.23604 fetch_fsmap finished waiting for FSMap version 68 2024-04-24T09:15:07.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.613+0000 7f33265fa740 10 client.23604 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:15:07.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.613+0000 7f33265fa740 20 client.23604 populate_metadata read hostname 'smithi083' 2024-04-24T09:15:07.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.613+0000 7f33265fa740 1 --2- 172.21.15.83:0/2063790168 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56471fa3b830 0x56471fa3dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:07.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.613+0000 7f33265fa740 4 client.23604 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T09:15:07.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.613+0000 7f33265fa740 1 -- 172.21.15.83:0/2063790168 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56471f915a80 con 0x56471fa3b830 2024-04-24T09:15:07.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.613+0000 7f3321800640 1 --2- 172.21.15.83:0/2063790168 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56471fa3b830 0x56471fa3dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:07.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.614+0000 7f3302a00640 10 client.23604 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:15:07.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.614+0000 7f3321800640 1 --2- 172.21.15.83:0/2063790168 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56471fa3b830 0x56471fa3dc50 crc :-1 s=READY pgs=580 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=128fbfdf6cbd29cc server_cookie=7a55e3045000b288 in_seq=0 out_seq=0 2024-04-24T09:15:07.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.615+0000 7f3302a00640 1 -- 172.21.15.83:0/2063790168 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56471f915a80 con 0x56471fa3b830 2024-04-24T09:15:07.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.615+0000 7f3302a00640 10 client.23604 handle_command_reply: tid=0 2024-04-24T09:15:07.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.615+0000 7f33265fa740 1 client.23604 shutdown 2024-04-24T09:15:07.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.615+0000 7f33265fa740 1 -- 172.21.15.83:0/2063790168 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56471f9a9180 msgr2=0x56471f9bba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:07.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.615+0000 7f33265fa740 1 --2- 172.21.15.83:0/2063790168 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56471f9a9180 0x56471f9bba40 secure :-1 s=READY pgs=2713 cs=0 l=1 rev1=1 crypto rx=0x7f330802a4c0 tx=0x7f33080044a0 comp rx=0 tx=0).stop 2024-04-24T09:15:07.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.615+0000 7f33265fa740 1 -- 172.21.15.83:0/2063790168 shutdown_connections 2024-04-24T09:15:07.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.615+0000 7f33265fa740 1 --2- 172.21.15.83:0/2063790168 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56471fa3b830 0x56471fa3dc50 crc :-1 s=READY pgs=580 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:07.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.615+0000 7f33265fa740 1 --2- 172.21.15.83:0/2063790168 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56471f9b3850 0x56471f9bbf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:07.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.615+0000 7f33265fa740 1 --2- 172.21.15.83:0/2063790168 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56471f9a9180 0x56471f9bba40 unknown :-1 s=CLOSED pgs=2713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:07.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.615+0000 7f33265fa740 1 --2- 172.21.15.83:0/2063790168 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56471f9a8860 0x56471fa25730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:07.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.615+0000 7f33265fa740 1 -- 172.21.15.83:0/2063790168 >> 172.21.15.83:0/2063790168 conn(0x56471f9a4b80 msgr2=0x56471f9a3390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:07.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.615+0000 7f3302a00640 0 client.23604 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:15:07.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.616+0000 7f33265fa740 1 -- 172.21.15.83:0/2063790168 shutdown_connections 2024-04-24T09:15:07.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.616+0000 7f33265fa740 1 -- 172.21.15.83:0/2063790168 wait complete. 2024-04-24T09:15:07.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.616+0000 7f33265fa740 20 client.23604 trim_cache size 0 max 16384 2024-04-24T09:15:07.616 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T09:15:07.616 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T09:15:07.616 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-04-24T09:15:07.616 INFO:teuthology.orchestra.run.smithi083.stdout: "bed76abd-1385-4070-a993-e3482666ecdb": { 2024-04-24T09:15:07.617 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-04-24T09:15:07.617 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "bed76abd-1385-4070-a993-e3482666ecdb", 2024-04-24T09:15:07.617 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-04-24T09:15:07.617 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:15:07.617 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:15:07.617 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T09:15:07.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.617+0000 7f3323800640 1 -- 172.21.15.83:0/2142272635 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f33040a3da0 msgr2=0x7f33040a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:07.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.617+0000 7f3323800640 1 --2- 172.21.15.83:0/2142272635 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f33040a3da0 0x7f33040a6260 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x56471f9a9e20 tx=0x7f3310009f30 comp rx=0 tx=0).stop 2024-04-24T09:15:07.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.617+0000 7f3323800640 1 -- 172.21.15.83:0/2142272635 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f331c0742d0 msgr2=0x7f331c19ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:07.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.617+0000 7f3323800640 1 --2- 172.21.15.83:0/2142272635 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f331c0742d0 0x7f331c19ee70 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f331000d6d0 tx=0x7f331000db90 comp rx=0 tx=0).stop 2024-04-24T09:15:07.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.617+0000 7f3323800640 1 -- 172.21.15.83:0/2142272635 shutdown_connections 2024-04-24T09:15:07.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.617+0000 7f3323800640 1 --2- 172.21.15.83:0/2142272635 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f33040a3da0 0x7f33040a6260 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:07.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.617+0000 7f3323800640 1 --2- 172.21.15.83:0/2142272635 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f331c1130b0 0x7f331c1a3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:07.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.617+0000 7f3323800640 1 --2- 172.21.15.83:0/2142272635 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f331c074bf0 0x7f331c19f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:07.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.617+0000 7f3323800640 1 --2- 172.21.15.83:0/2142272635 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f331c0742d0 0x7f331c19ee70 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:07.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.617+0000 7f3323800640 1 -- 172.21.15.83:0/2142272635 >> 172.21.15.83:0/2142272635 conn(0x7f331c079450 msgr2=0x7f331c0fefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:07.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.617+0000 7f3323800640 1 -- 172.21.15.83:0/2142272635 shutdown_connections 2024-04-24T09:15:07.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:07.617+0000 7f3323800640 1 -- 172.21.15.83:0/2142272635 wait complete. 2024-04-24T09:15:07.637 INFO:tasks.cephfs.filesystem:scrub status for tag:bed76abd-1385-4070-a993-e3482666ecdb - {'path': '/', 'tag': 'bed76abd-1385-4070-a993-e3482666ecdb', 'options': 'recursive,force'} 2024-04-24T09:15:08.209 INFO:tasks.workunit.client.0.smithi083.stdout: 10 71169 6.54 MB/sec execute 459 sec latency 95.411 ms 2024-04-24T09:15:08.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:15:08 smithi138 ceph-mon[31441]: pgmap v742: 129 pgs: 129 active+clean; 1.0 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.0 MiB/s wr, 238 op/s 2024-04-24T09:15:08.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:15:08 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:15:08.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:15:08 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:15:08.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:15:08 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:15:08.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:15:08 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:15:08.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:15:08 smithi189 ceph-mon[31791]: pgmap v742: 129 pgs: 129 active+clean; 1.0 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.0 MiB/s wr, 238 op/s 2024-04-24T09:15:08.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:15:08 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:15:08.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:15:08 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:15:08.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:15:08 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:15:08.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:15:08 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:15:08.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:15:08 smithi083 ceph-mon[27907]: pgmap v742: 129 pgs: 129 active+clean; 1.0 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.0 MiB/s wr, 238 op/s 2024-04-24T09:15:08.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:15:08 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:15:08.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:15:08 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:15:08.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:15:08 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:15:08.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:15:08 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:15:09.210 INFO:tasks.workunit.client.0.smithi083.stdout: 10 71260 6.53 MB/sec execute 460 sec latency 114.108 ms 2024-04-24T09:15:09.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:15:09 smithi138 ceph-mon[31441]: scrub queued for path: / 2024-04-24T09:15:09.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:15:09 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:15:09.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:15:09 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:15:09.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:15:09 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:15:09.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:15:09 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:15:09.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:15:09 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:15:09.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:15:09 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:15:09.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:15:09 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:15:09.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:15:09 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:15:09.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:15:09 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:15:09.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:15:09 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:15:09.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:15:09 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:15:09.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:15:09 smithi138 ceph-mon[31441]: osdmap e129: 12 total, 12 up, 12 in 2024-04-24T09:15:09.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:15:09 smithi189 ceph-mon[31791]: scrub queued for path: / 2024-04-24T09:15:09.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:15:09 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:15:09.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:15:09 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:15:09.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:15:09 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:15:09.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:15:09 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:15:09.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:15:09 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:15:09.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:15:09 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:15:09.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:15:09 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:15:09.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:15:09 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:15:09.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:15:09 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:15:09.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:15:09 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:15:09.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:15:09 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:15:09.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:15:09 smithi189 ceph-mon[31791]: osdmap e129: 12 total, 12 up, 12 in 2024-04-24T09:15:09.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:15:09 smithi083 ceph-mon[27907]: scrub queued for path: / 2024-04-24T09:15:09.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:15:09 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:15:09.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:15:09 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:15:09.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:15:09 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:15:09.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:15:09 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:15:09.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:15:09 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:15:09.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:15:09 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:15:09.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:15:09 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:15:09.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:15:09 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:15:09.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:15:09 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:15:09.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:15:09 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:15:09.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:15:09 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:15:09.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:15:09 smithi083 ceph-mon[27907]: osdmap e129: 12 total, 12 up, 12 in 2024-04-24T09:15:10.210 INFO:tasks.workunit.client.0.smithi083.stdout: 10 71382 6.54 MB/sec execute 461 sec latency 196.279 ms 2024-04-24T09:15:10.356 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:15:10 smithi138 ceph-mon[31441]: pgmap v744: 129 pgs: 129 active+clean; 1.1 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.5 MiB/s wr, 251 op/s 2024-04-24T09:15:10.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:15:10 smithi189 ceph-mon[31791]: pgmap v744: 129 pgs: 129 active+clean; 1.1 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.5 MiB/s wr, 251 op/s 2024-04-24T09:15:10.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:15:10 smithi083 ceph-mon[27907]: pgmap v744: 129 pgs: 129 active+clean; 1.1 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.5 MiB/s wr, 251 op/s 2024-04-24T09:15:11.150 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.150+0000 7f7701c00640 1 -- 172.21.15.83:0/468685842 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f76fc109370 msgr2=0x7f76fc110f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:11.150 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.150+0000 7f7701c00640 1 --2- 172.21.15.83:0/468685842 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f76fc109370 0x7f76fc110f40 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f76f4009ca0 tx=0x7f76f402a8c0 comp rx=0 tx=0).stop 2024-04-24T09:15:11.150 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.151+0000 7f7701c00640 1 -- 172.21.15.83:0/468685842 shutdown_connections 2024-04-24T09:15:11.151 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.151+0000 7f7701c00640 1 --2- 172.21.15.83:0/468685842 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f76fc109370 0x7f76fc110f40 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:11.151 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.151+0000 7f7701c00640 1 --2- 172.21.15.83:0/468685842 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76fc104c50 0x7f76fc108c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:11.151 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.151+0000 7f7701c00640 1 --2- 172.21.15.83:0/468685842 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f76fc1042a0 0x7f76fc104680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:11.151 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.151+0000 7f7701c00640 1 -- 172.21.15.83:0/468685842 >> 172.21.15.83:0/468685842 conn(0x7f76fc0ffae0 msgr2=0x7f76fc101f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:11.151 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.152+0000 7f7701c00640 1 -- 172.21.15.83:0/468685842 shutdown_connections 2024-04-24T09:15:11.152 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.152+0000 7f7701c00640 1 -- 172.21.15.83:0/468685842 wait complete. 2024-04-24T09:15:11.153 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.154+0000 7f7701c00640 1 Processor -- start 2024-04-24T09:15:11.153 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.154+0000 7f7701c00640 1 -- start start 2024-04-24T09:15:11.154 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.155+0000 7f7701c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f76fc1042a0 0x7f76fc10e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:11.155 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.155+0000 7f7701c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f76fc109370 0x7f76fc10e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:11.155 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.155+0000 7f7701c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76fc10eb10 0x7f76fc1a07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:11.155 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.155+0000 7f7701c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f76fc1142c0 con 0x7f76fc10eb10 2024-04-24T09:15:11.155 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.155+0000 7f7701c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f76fc114140 con 0x7f76fc1042a0 2024-04-24T09:15:11.155 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.155+0000 7f7701c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f76fc114440 con 0x7f76fc109370 2024-04-24T09:15:11.155 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.155+0000 7f7701200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76fc10eb10 0x7f76fc1a07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:11.155 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.155+0000 7f7701200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76fc10eb10 0x7f76fc1a07e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33772/0 (socket says 172.21.15.83:33772) 2024-04-24T09:15:11.155 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.155+0000 7f7701200640 1 -- 172.21.15.83:0/4025745500 learned_addr learned my addr 172.21.15.83:0/4025745500 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:15:11.155 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.155+0000 7f76fbe00640 1 --2- 172.21.15.83:0/4025745500 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f76fc1042a0 0x7f76fc10e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:11.155 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.155+0000 7f76fb400640 1 --2- 172.21.15.83:0/4025745500 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f76fc109370 0x7f76fc10e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:11.156 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.156+0000 7f7701200640 1 -- 172.21.15.83:0/4025745500 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f76fc109370 msgr2=0x7f76fc10e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:11.156 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.156+0000 7f7701200640 1 --2- 172.21.15.83:0/4025745500 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f76fc109370 0x7f76fc10e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:11.156 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.156+0000 7f7701200640 1 -- 172.21.15.83:0/4025745500 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f76fc1042a0 msgr2=0x7f76fc10e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:11.156 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.156+0000 7f7701200640 1 --2- 172.21.15.83:0/4025745500 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f76fc1042a0 0x7f76fc10e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:11.157 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.156+0000 7f7701200640 1 -- 172.21.15.83:0/4025745500 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76fc1a0d20 con 0x7f76fc10eb10 2024-04-24T09:15:11.157 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.156+0000 7f7701200640 1 --2- 172.21.15.83:0/4025745500 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76fc10eb10 0x7f76fc1a07e0 secure :-1 s=READY pgs=2714 cs=0 l=1 rev1=1 crypto rx=0x7f76f4009dc0 tx=0x7f76f4032ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:11.157 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.157+0000 7f76f8c00640 1 -- 172.21.15.83:0/4025745500 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76f4003c30 con 0x7f76fc10eb10 2024-04-24T09:15:11.157 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.157+0000 7f76f8c00640 1 -- 172.21.15.83:0/4025745500 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f76f4003dd0 con 0x7f76fc10eb10 2024-04-24T09:15:11.157 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.157+0000 7f7701c00640 1 -- 172.21.15.83:0/4025745500 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76fc1a0fb0 con 0x7f76fc10eb10 2024-04-24T09:15:11.157 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.157+0000 7f76f8c00640 1 -- 172.21.15.83:0/4025745500 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76f4030aa0 con 0x7f76fc10eb10 2024-04-24T09:15:11.158 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.157+0000 7f7701c00640 1 -- 172.21.15.83:0/4025745500 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76fc1a1550 con 0x7f76fc10eb10 2024-04-24T09:15:11.160 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.161+0000 7f770617d740 1 Processor -- start 2024-04-24T09:15:11.161 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.162+0000 7f76f8c00640 1 -- 172.21.15.83:0/4025745500 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f76f4004680 con 0x7f76fc10eb10 2024-04-24T09:15:11.161 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.162+0000 7f770617d740 1 -- start start 2024-04-24T09:15:11.162 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.162+0000 7f770617d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a78591a3c0 0x55a78591a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:11.162 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.162+0000 7f770617d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a78591ace0 0x55a785925060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:11.162 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.162+0000 7f770617d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7859255a0 0x55a785927990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:11.162 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.162+0000 7f770617d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a785880900 con 0x55a78591a3c0 2024-04-24T09:15:11.162 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.162+0000 7f770617d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a785826f20 con 0x55a7859255a0 2024-04-24T09:15:11.162 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.162+0000 7f770617d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a78587c8c0 con 0x55a78591ace0 2024-04-24T09:15:11.162 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.163+0000 7f76fb400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a78591a3c0 0x55a78591a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:11.162 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.163+0000 7f76fb400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a78591a3c0 0x55a78591a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33784/0 (socket says 172.21.15.83:33784) 2024-04-24T09:15:11.162 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.163+0000 7f76fb400640 1 -- 172.21.15.83:0/964334212 learned_addr learned my addr 172.21.15.83:0/964334212 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:15:11.163 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.163+0000 7f76fb400640 1 --2- 172.21.15.83:0/964334212 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7859255a0 0x55a785927990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:11.163 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.163+0000 7f76fbe00640 1 --2- 172.21.15.83:0/964334212 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a78591ace0 0x55a785925060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:11.164 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.163+0000 7f76f8c00640 1 --2- 172.21.15.83:0/4025745500 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f76e00a3d90 0x7f76e00a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:11.164 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.164+0000 7f76f8c00640 1 -- 172.21.15.83:0/4025745500 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 9671+0+0 (secure 0 0 0) 0x7f76f40f2230 con 0x7f76fc10eb10 2024-04-24T09:15:11.164 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.164+0000 7f7701200640 1 --2- 172.21.15.83:0/4025745500 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f76e00a3d90 0x7f76e00a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:11.164 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.164+0000 7f76fbe00640 1 -- 172.21.15.83:0/964334212 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7859255a0 msgr2=0x55a785927990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:11.164 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.164+0000 7f76fbe00640 1 --2- 172.21.15.83:0/964334212 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7859255a0 0x55a785927990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:11.164 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.164+0000 7f76fbe00640 1 -- 172.21.15.83:0/964334212 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a78591a3c0 msgr2=0x55a78591a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:11.164 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.164+0000 7f76fbe00640 1 --2- 172.21.15.83:0/964334212 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a78591a3c0 0x55a78591a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:11.164 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.164+0000 7f76fbe00640 1 -- 172.21.15.83:0/964334212 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a78584be10 con 0x55a78591ace0 2024-04-24T09:15:11.164 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.164+0000 7f7701200640 1 --2- 172.21.15.83:0/4025745500 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f76e00a3d90 0x7f76e00a6250 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f76f40ef610 tx=0x7f76f4004970 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:11.165 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.164+0000 7f76fbe00640 1 --2- 172.21.15.83:0/964334212 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a78591ace0 0x55a785925060 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f76ec009c20 tx=0x7f76ec011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e82027c58da94fd4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:11.165 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.165+0000 7f76f1600640 1 -- 172.21.15.83:0/964334212 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76ec009dd0 con 0x55a78591ace0 2024-04-24T09:15:11.165 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.165+0000 7f76f1600640 1 -- 172.21.15.83:0/964334212 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f76ec005020 con 0x55a78591ace0 2024-04-24T09:15:11.165 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.165+0000 7f76f1600640 1 -- 172.21.15.83:0/964334212 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76ec0049e0 con 0x55a78591ace0 2024-04-24T09:15:11.165 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.166+0000 7f770617d740 1 -- 172.21.15.83:0/964334212 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a78591ace0 msgr2=0x55a785925060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:11.165 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.166+0000 7f770617d740 1 --2- 172.21.15.83:0/964334212 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a78591ace0 0x55a785925060 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f76ec009c20 tx=0x7f76ec011810 comp rx=0 tx=0).stop 2024-04-24T09:15:11.166 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.166+0000 7f770617d740 1 -- 172.21.15.83:0/964334212 shutdown_connections 2024-04-24T09:15:11.166 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.166+0000 7f770617d740 1 --2- 172.21.15.83:0/964334212 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7859255a0 0x55a785927990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:11.166 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.166+0000 7f770617d740 1 --2- 172.21.15.83:0/964334212 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a78591ace0 0x55a785925060 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:11.166 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.166+0000 7f770617d740 1 --2- 172.21.15.83:0/964334212 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a78591a3c0 0x55a78591a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:11.166 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.166+0000 7f770617d740 1 -- 172.21.15.83:0/964334212 >> 172.21.15.83:0/964334212 conn(0x55a7859168d0 msgr2=0x55a785916cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:11.166 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.166+0000 7f770617d740 1 -- 172.21.15.83:0/964334212 shutdown_connections 2024-04-24T09:15:11.166 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.166+0000 7f770617d740 1 -- 172.21.15.83:0/964334212 wait complete. 2024-04-24T09:15:11.167 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.167+0000 7f770617d740 1 -- start start 2024-04-24T09:15:11.167 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.167+0000 7f770617d740 1 Processor -- start 2024-04-24T09:15:11.167 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.168+0000 7f770617d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a78591a3c0 0x55a785922620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:11.167 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.168+0000 7f770617d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a78591ace0 0x55a785922b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:11.167 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.168+0000 7f770617d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7859255a0 0x55a7859230a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:11.167 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.168+0000 7f770617d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a785928140 con 0x55a7859255a0 2024-04-24T09:15:11.167 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.168+0000 7f770617d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a78583a2c0 con 0x55a78591a3c0 2024-04-24T09:15:11.167 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.168+0000 7f770617d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a7859282c0 con 0x55a78591ace0 2024-04-24T09:15:11.168 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.168+0000 7f770617d740 10 client.0 _ll_register_callbacks cb 0x55a785592a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:15:11.168 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.168+0000 7f76fb400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a78591a3c0 0x55a785922620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:11.169 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.168+0000 7f76fbe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a78591ace0 0x55a785922b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:11.169 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.168+0000 7f76fb400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a78591a3c0 0x55a785922620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:53300/0 (socket says 172.21.15.83:53300) 2024-04-24T09:15:11.169 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.168+0000 7f76fb400640 1 -- 172.21.15.83:0/3299638449 learned_addr learned my addr 172.21.15.83:0/3299638449 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:15:11.169 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.168+0000 7f76fb400640 1 --2- 172.21.15.83:0/3299638449 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7859255a0 0x55a7859230a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:11.169 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.169+0000 7f76fbe00640 1 -- 172.21.15.83:0/3299638449 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a78591a3c0 msgr2=0x55a785922620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:11.169 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.169+0000 7f76fbe00640 1 --2- 172.21.15.83:0/3299638449 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a78591a3c0 0x55a785922620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:11.169 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.169+0000 7f76fbe00640 1 -- 172.21.15.83:0/3299638449 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7859255a0 msgr2=0x55a7859230a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:11.169 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.169+0000 7f76fbe00640 1 --2- 172.21.15.83:0/3299638449 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7859255a0 0x55a7859230a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:11.169 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.169+0000 7f76fbe00640 1 -- 172.21.15.83:0/3299638449 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a78584afd0 con 0x55a78591ace0 2024-04-24T09:15:11.169 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.169+0000 7f76fb400640 1 --2- 172.21.15.83:0/3299638449 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a78591a3c0 0x55a785922620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:15:11.169 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.169+0000 7f76fbe00640 1 --2- 172.21.15.83:0/3299638449 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a78591ace0 0x55a785922b60 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f76ec01b040 tx=0x7f76ec0098f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=90976b8b47cfd8af server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:11.169 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.169+0000 7f76e6000640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-04-24T09:15:11.170 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.170+0000 7f76e6000640 1 -- 172.21.15.83:0/3299638449 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76ec028020 con 0x55a78591ace0 2024-04-24T09:15:11.170 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.170+0000 7f770617d740 1 -- 172.21.15.83:0/3299638449 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76ec005020 con 0x55a78591ace0 2024-04-24T09:15:11.170 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.170+0000 7f76e6000640 1 -- 172.21.15.83:0/3299638449 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(129..129 src has 1..129) ==== 9671+0+0 (secure 0 0 0) 0x7f76ec016040 con 0x55a78591ace0 2024-04-24T09:15:11.171 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.170+0000 7f76e6000640 1 -- 172.21.15.83:0/3299638449 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7f76ec033050 con 0x55a78591ace0 2024-04-24T09:15:11.171 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.171+0000 7f770617d740 10 client.30385 fetch_fsmap learned FSMap version 68 2024-04-24T09:15:11.171 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.171+0000 7f770617d740 1 -- 172.21.15.83:0/3299638449 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) -- 0x55a785593210 con 0x55a78591ace0 2024-04-24T09:15:11.171 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.171+0000 7f76e6000640 1 -- 172.21.15.83:0/3299638449 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f76ec005020 con 0x55a78591ace0 2024-04-24T09:15:11.172 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.172+0000 7f76e6000640 1 -- 172.21.15.83:0/3299638449 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 68) ==== 8374+0+0 (secure 0 0 0) 0x7f76ec0035f0 con 0x55a78591ace0 2024-04-24T09:15:11.172 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.172+0000 7f770617d740 10 client.30385 fetch_fsmap finished waiting for FSMap version 68 2024-04-24T09:15:11.172 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.172+0000 7f770617d740 10 client.30385 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:15:11.172 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.172+0000 7f770617d740 20 client.30385 populate_metadata read hostname 'smithi083' 2024-04-24T09:15:11.172 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.172+0000 7f770617d740 1 --2- 172.21.15.83:0/3299638449 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55a785935710 0x55a785937b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:11.172 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.172+0000 7f770617d740 4 client.30385 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:15:11.172 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.172+0000 7f770617d740 1 -- 172.21.15.83:0/3299638449 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a78588e320 con 0x55a785935710 2024-04-24T09:15:11.172 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.173+0000 7f76fb400640 1 --2- 172.21.15.83:0/3299638449 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55a785935710 0x55a785937b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:11.172 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.173+0000 7f76fb400640 1 --2- 172.21.15.83:0/3299638449 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55a785935710 0x55a785937b00 crc :-1 s=READY pgs=581 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=109bb5e57b175787 server_cookie=edc71b2b1e78a177 in_seq=0 out_seq=0 2024-04-24T09:15:11.173 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.173+0000 7f76e6000640 10 client.30385 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:15:11.174 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.175+0000 7f76e6000640 1 -- 172.21.15.83:0/3299638449 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a78588e320 con 0x55a785935710 2024-04-24T09:15:11.174 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.175+0000 7f76e6000640 10 client.30385 handle_command_reply: tid=0 2024-04-24T09:15:11.174 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.175+0000 7f770617d740 1 client.30385 shutdown 2024-04-24T09:15:11.174 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.175+0000 7f770617d740 1 -- 172.21.15.83:0/3299638449 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a78591ace0 msgr2=0x55a785922b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:11.174 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.175+0000 7f770617d740 1 --2- 172.21.15.83:0/3299638449 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a78591ace0 0x55a785922b60 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f76ec01b040 tx=0x7f76ec0098f0 comp rx=0 tx=0).stop 2024-04-24T09:15:11.175 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.175+0000 7f770617d740 1 -- 172.21.15.83:0/3299638449 shutdown_connections 2024-04-24T09:15:11.175 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.175+0000 7f770617d740 1 --2- 172.21.15.83:0/3299638449 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55a785935710 0x55a785937b00 crc :-1 s=READY pgs=581 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:11.175 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.175+0000 7f770617d740 1 --2- 172.21.15.83:0/3299638449 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7859255a0 0x55a7859230a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:11.175 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.175+0000 7f770617d740 1 --2- 172.21.15.83:0/3299638449 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a78591ace0 0x55a785922b60 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:11.175 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.175+0000 7f76e6000640 0 client.30385 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:15:11.175 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.175+0000 7f770617d740 1 --2- 172.21.15.83:0/3299638449 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a78591a3c0 0x55a785922620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:11.175 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.175+0000 7f770617d740 1 -- 172.21.15.83:0/3299638449 >> 172.21.15.83:0/3299638449 conn(0x55a7859168d0 msgr2=0x55a78591c650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:11.176 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.175+0000 7f770617d740 1 -- 172.21.15.83:0/3299638449 shutdown_connections 2024-04-24T09:15:11.176 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.176+0000 7f770617d740 1 -- 172.21.15.83:0/3299638449 wait complete. 2024-04-24T09:15:11.176 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.176+0000 7f770617d740 20 client.30385 trim_cache size 0 max 16384 2024-04-24T09:15:11.199 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.199+0000 7f770617d740 1 Processor -- start 2024-04-24T09:15:11.199 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.199+0000 7f770617d740 1 -- start start 2024-04-24T09:15:11.199 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.199+0000 7f770617d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a78591a3c0 0x55a785924d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:11.199 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.199+0000 7f770617d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7859255a0 0x55a7859310c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:11.199 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.199+0000 7f770617d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a78593d340 0x55a78593d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:11.199 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.199+0000 7f770617d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f76fc1be400 con 0x55a7859255a0 2024-04-24T09:15:11.199 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.199+0000 7f770617d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f76fc1bffd0 con 0x55a78591a3c0 2024-04-24T09:15:11.199 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.199+0000 7f770617d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f76fc1bead0 con 0x55a78593d340 2024-04-24T09:15:11.200 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.200+0000 7f76fb400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a78591a3c0 0x55a785924d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:11.201 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.200+0000 7f76fb400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a78591a3c0 0x55a785924d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:53316/0 (socket says 172.21.15.83:53316) 2024-04-24T09:15:11.201 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.200+0000 7f76fb400640 1 -- 172.21.15.83:0/2398629179 learned_addr learned my addr 172.21.15.83:0/2398629179 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:15:11.201 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.200+0000 7f76fbe00640 1 --2- 172.21.15.83:0/2398629179 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7859255a0 0x55a7859310c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:11.201 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.200+0000 7f76fb400640 1 --2- 172.21.15.83:0/2398629179 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a78593d340 0x55a78593d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:11.201 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.200+0000 7f76fb400640 1 -- 172.21.15.83:0/2398629179 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a78593d340 msgr2=0x55a78593d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:11.201 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.200+0000 7f76fb400640 1 --2- 172.21.15.83:0/2398629179 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a78593d340 0x55a78593d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:11.201 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.200+0000 7f76fb400640 1 -- 172.21.15.83:0/2398629179 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7859255a0 msgr2=0x55a7859310c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:11.201 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.200+0000 7f76fb400640 1 --2- 172.21.15.83:0/2398629179 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7859255a0 0x55a7859310c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:11.201 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.200+0000 7f76fb400640 1 -- 172.21.15.83:0/2398629179 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a78565f620 con 0x55a78591a3c0 2024-04-24T09:15:11.201 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.201+0000 7f76fb400640 1 --2- 172.21.15.83:0/2398629179 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a78591a3c0 0x55a785924d50 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f76e80139c0 tx=0x7f76e80046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c192eb23a2ce79cf server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:11.201 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.201+0000 7f76f0c00640 1 -- 172.21.15.83:0/2398629179 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e8027e60 con 0x55a78591a3c0 2024-04-24T09:15:11.201 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.201+0000 7f76f0c00640 1 -- 172.21.15.83:0/2398629179 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f76e8013180 con 0x55a78591a3c0 2024-04-24T09:15:11.201 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.201+0000 7f76f0c00640 1 -- 172.21.15.83:0/2398629179 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e80277b0 con 0x55a78591a3c0 2024-04-24T09:15:11.202 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.202+0000 7f770617d740 1 -- 172.21.15.83:0/2398629179 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a78591a3c0 msgr2=0x55a785924d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:11.202 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.202+0000 7f770617d740 1 --2- 172.21.15.83:0/2398629179 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a78591a3c0 0x55a785924d50 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f76e80139c0 tx=0x7f76e80046c0 comp rx=0 tx=0).stop 2024-04-24T09:15:11.202 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.202+0000 7f770617d740 1 -- 172.21.15.83:0/2398629179 shutdown_connections 2024-04-24T09:15:11.202 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.202+0000 7f770617d740 1 --2- 172.21.15.83:0/2398629179 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a78593d340 0x55a78593d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:11.202 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.202+0000 7f770617d740 1 --2- 172.21.15.83:0/2398629179 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7859255a0 0x55a7859310c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:11.203 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.202+0000 7f770617d740 1 --2- 172.21.15.83:0/2398629179 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a78591a3c0 0x55a785924d50 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:11.203 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.202+0000 7f770617d740 1 -- 172.21.15.83:0/2398629179 >> 172.21.15.83:0/2398629179 conn(0x55a7859168d0 msgr2=0x55a785922e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:11.203 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.202+0000 7f770617d740 1 -- 172.21.15.83:0/2398629179 shutdown_connections 2024-04-24T09:15:11.203 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.202+0000 7f770617d740 1 -- 172.21.15.83:0/2398629179 wait complete. 2024-04-24T09:15:11.203 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.203+0000 7f770617d740 1 -- start start 2024-04-24T09:15:11.203 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.203+0000 7f770617d740 1 Processor -- start 2024-04-24T09:15:11.203 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.203+0000 7f770617d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a78591a3c0 0x55a78592d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:11.203 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.203+0000 7f770617d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7859255a0 0x55a78592db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:11.204 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.204+0000 7f76fb400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a78591a3c0 0x55a78592d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:11.204 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.204+0000 7f76fbe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7859255a0 0x55a78592db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:11.204 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.204+0000 7f76fbe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7859255a0 0x55a78592db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:53320/0 (socket says 172.21.15.83:53320) 2024-04-24T09:15:11.204 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.204+0000 7f770617d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a78593d340 0x55a78592a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:11.204 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.204+0000 7f76fbe00640 1 -- 172.21.15.83:0/2819741572 learned_addr learned my addr 172.21.15.83:0/2819741572 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:15:11.204 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.204+0000 7f770617d740 1 -- 172.21.15.83:0/2819741572 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f76fc1bf8d0 con 0x55a78593d340 2024-04-24T09:15:11.204 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.204+0000 7f770617d740 1 -- 172.21.15.83:0/2819741572 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f76fc1bf200 con 0x55a7859255a0 2024-04-24T09:15:11.204 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.204+0000 7f770617d740 1 -- 172.21.15.83:0/2819741572 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f76fc1bc400 con 0x55a78591a3c0 2024-04-24T09:15:11.204 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.204+0000 7f770617d740 10 client.0 _ll_register_callbacks cb 0x7f76e80099c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:15:11.205 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.205+0000 7f76fbe00640 1 -- 172.21.15.83:0/2819741572 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a78591a3c0 msgr2=0x55a78592d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:11.205 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.205+0000 7f76fbe00640 1 --2- 172.21.15.83:0/2819741572 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a78591a3c0 0x55a78592d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:11.205 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.205+0000 7f76fbe00640 1 -- 172.21.15.83:0/2819741572 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a78593d340 msgr2=0x55a78592a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:11.205 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.205+0000 7f76fbe00640 1 --2- 172.21.15.83:0/2819741572 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a78593d340 0x55a78592a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:11.205 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.205+0000 7f76fbe00640 1 -- 172.21.15.83:0/2819741572 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a78592af30 con 0x55a7859255a0 2024-04-24T09:15:11.205 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.206+0000 7f76fbe00640 1 --2- 172.21.15.83:0/2819741572 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7859255a0 0x55a78592db10 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f76ec006010 tx=0x7f76ec005c40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e955d883e09fe3a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:11.206 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.206+0000 7f76e6000640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:15:11.206 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.206+0000 7f76e6000640 1 -- 172.21.15.83:0/2819741572 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76ec028040 con 0x55a7859255a0 2024-04-24T09:15:11.206 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.206+0000 7f770617d740 1 -- 172.21.15.83:0/2819741572 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76e8013180 con 0x55a7859255a0 2024-04-24T09:15:11.206 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.206+0000 7f76e6000640 1 -- 172.21.15.83:0/2819741572 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(129..129 src has 1..129) ==== 9671+0+0 (secure 0 0 0) 0x7f76ec003c50 con 0x55a7859255a0 2024-04-24T09:15:11.207 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.207+0000 7f76e6000640 1 -- 172.21.15.83:0/2819741572 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7f76ec03f050 con 0x55a7859255a0 2024-04-24T09:15:11.207 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.207+0000 7f770617d740 10 client.40676 fetch_fsmap learned FSMap version 68 2024-04-24T09:15:11.207 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.207+0000 7f770617d740 1 -- 172.21.15.83:0/2819741572 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) -- 0x55a78599c850 con 0x55a7859255a0 2024-04-24T09:15:11.207 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.207+0000 7f76e6000640 1 -- 172.21.15.83:0/2819741572 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f76e8013180 con 0x55a7859255a0 2024-04-24T09:15:11.207 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.208+0000 7f76e6000640 1 -- 172.21.15.83:0/2819741572 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 68) ==== 8374+0+0 (secure 0 0 0) 0x7f76ec024a70 con 0x55a7859255a0 2024-04-24T09:15:11.208 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.208+0000 7f770617d740 10 client.40676 fetch_fsmap finished waiting for FSMap version 68 2024-04-24T09:15:11.208 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.208+0000 7f770617d740 10 client.40676 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:15:11.208 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.208+0000 7f770617d740 20 client.40676 populate_metadata read hostname 'smithi083' 2024-04-24T09:15:11.208 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.208+0000 7f770617d740 1 --2- 172.21.15.83:0/2819741572 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55a78599cf40 0x55a78599f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:11.208 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.208+0000 7f770617d740 4 client.40676 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T09:15:11.208 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.208+0000 7f770617d740 1 -- 172.21.15.83:0/2819741572 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a785888190 con 0x55a78599cf40 2024-04-24T09:15:11.208 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.208+0000 7f76fb400640 1 --2- 172.21.15.83:0/2819741572 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55a78599cf40 0x55a78599f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:11.209 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.210+0000 7f76fb400640 1 --2- 172.21.15.83:0/2819741572 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55a78599cf40 0x55a78599f360 crc :-1 s=READY pgs=582 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3ad549f226a9874 server_cookie=6760aa3ed910e3d9 in_seq=0 out_seq=0 2024-04-24T09:15:11.209 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.210+0000 7f76e6000640 10 client.40676 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:15:11.210 INFO:tasks.workunit.client.0.smithi083.stdout: 10 71497 6.55 MB/sec execute 462 sec latency 324.452 ms 2024-04-24T09:15:11.337 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.338+0000 7f76e6000640 1 -- 172.21.15.83:0/2819741572 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a785888190 con 0x55a78599cf40 2024-04-24T09:15:11.338 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.338+0000 7f76e6000640 10 client.40676 handle_command_reply: tid=0 2024-04-24T09:15:11.338 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.338+0000 7f770617d740 1 client.40676 shutdown 2024-04-24T09:15:11.338 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.338+0000 7f770617d740 1 -- 172.21.15.83:0/2819741572 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7859255a0 msgr2=0x55a78592db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:11.338 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.338+0000 7f770617d740 1 --2- 172.21.15.83:0/2819741572 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7859255a0 0x55a78592db10 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f76ec006010 tx=0x7f76ec005c40 comp rx=0 tx=0).stop 2024-04-24T09:15:11.338 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.338+0000 7f770617d740 1 -- 172.21.15.83:0/2819741572 shutdown_connections 2024-04-24T09:15:11.338 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.338+0000 7f770617d740 1 --2- 172.21.15.83:0/2819741572 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55a78599cf40 0x55a78599f360 crc :-1 s=READY pgs=582 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:11.338 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.338+0000 7f770617d740 1 --2- 172.21.15.83:0/2819741572 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a78593d340 0x55a78592a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:11.338 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.338+0000 7f770617d740 1 --2- 172.21.15.83:0/2819741572 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7859255a0 0x55a78592db10 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:11.338 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.338+0000 7f770617d740 1 --2- 172.21.15.83:0/2819741572 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a78591a3c0 0x55a78592d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:11.338 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.338+0000 7f770617d740 1 -- 172.21.15.83:0/2819741572 >> 172.21.15.83:0/2819741572 conn(0x55a7859168d0 msgr2=0x55a78593c330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:11.339 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.338+0000 7f76e6000640 0 client.40676 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:15:11.339 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.339+0000 7f770617d740 1 -- 172.21.15.83:0/2819741572 shutdown_connections 2024-04-24T09:15:11.339 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.339+0000 7f770617d740 1 -- 172.21.15.83:0/2819741572 wait complete. 2024-04-24T09:15:11.339 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.339+0000 7f770617d740 20 client.40676 trim_cache size 0 max 16384 2024-04-24T09:15:11.340 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T09:15:11.340 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T09:15:11.340 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T09:15:11.340 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T09:15:11.340 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.341+0000 7f7701c00640 1 -- 172.21.15.83:0/4025745500 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f76e00a3d90 msgr2=0x7f76e00a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:11.340 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.341+0000 7f7701c00640 1 --2- 172.21.15.83:0/4025745500 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f76e00a3d90 0x7f76e00a6250 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f76f40ef610 tx=0x7f76f4004970 comp rx=0 tx=0).stop 2024-04-24T09:15:11.341 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.341+0000 7f7701c00640 1 -- 172.21.15.83:0/4025745500 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76fc10eb10 msgr2=0x7f76fc1a07e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:11.341 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.341+0000 7f7701c00640 1 --2- 172.21.15.83:0/4025745500 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76fc10eb10 0x7f76fc1a07e0 secure :-1 s=READY pgs=2714 cs=0 l=1 rev1=1 crypto rx=0x7f76f4009dc0 tx=0x7f76f4032ff0 comp rx=0 tx=0).stop 2024-04-24T09:15:11.341 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.341+0000 7f7701c00640 1 -- 172.21.15.83:0/4025745500 shutdown_connections 2024-04-24T09:15:11.341 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.341+0000 7f7701c00640 1 --2- 172.21.15.83:0/4025745500 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f76e00a3d90 0x7f76e00a6250 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:11.341 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.341+0000 7f7701c00640 1 --2- 172.21.15.83:0/4025745500 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76fc10eb10 0x7f76fc1a07e0 unknown :-1 s=CLOSED pgs=2714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:11.341 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.341+0000 7f7701c00640 1 --2- 172.21.15.83:0/4025745500 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f76fc109370 0x7f76fc10e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:11.341 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.341+0000 7f7701c00640 1 --2- 172.21.15.83:0/4025745500 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f76fc1042a0 0x7f76fc10e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:11.341 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.341+0000 7f7701c00640 1 -- 172.21.15.83:0/4025745500 >> 172.21.15.83:0/4025745500 conn(0x7f76fc0ffae0 msgr2=0x7f76fc100e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:11.342 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.342+0000 7f7701c00640 1 -- 172.21.15.83:0/4025745500 shutdown_connections 2024-04-24T09:15:11.342 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:11.342+0000 7f7701c00640 1 -- 172.21.15.83:0/4025745500 wait complete. 2024-04-24T09:15:12.210 INFO:tasks.workunit.client.0.smithi083.stdout: 10 71624 6.55 MB/sec execute 463 sec latency 118.842 ms 2024-04-24T09:15:12.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:15:12 smithi138 ceph-mon[31441]: pgmap v745: 129 pgs: 129 active+clean; 1.1 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.5 MiB/s wr, 251 op/s 2024-04-24T09:15:12.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:15:12 smithi189 ceph-mon[31791]: pgmap v745: 129 pgs: 129 active+clean; 1.1 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.5 MiB/s wr, 251 op/s 2024-04-24T09:15:12.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:15:12 smithi083 ceph-mon[27907]: pgmap v745: 129 pgs: 129 active+clean; 1.1 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.5 MiB/s wr, 251 op/s 2024-04-24T09:15:13.210 INFO:tasks.workunit.client.0.smithi083.stdout: 10 71771 6.55 MB/sec execute 464 sec latency 79.472 ms 2024-04-24T09:15:14.210 INFO:tasks.workunit.client.0.smithi083.stdout: 10 71862 6.54 MB/sec execute 465 sec latency 89.631 ms 2024-04-24T09:15:14.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:15:14 smithi138 ceph-mon[31441]: pgmap v746: 129 pgs: 129 active+clean; 1.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.5 MiB/s wr, 289 op/s 2024-04-24T09:15:14.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:15:14 smithi189 ceph-mon[31791]: pgmap v746: 129 pgs: 129 active+clean; 1.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.5 MiB/s wr, 289 op/s 2024-04-24T09:15:14.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:15:14 smithi083 ceph-mon[27907]: pgmap v746: 129 pgs: 129 active+clean; 1.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.5 MiB/s wr, 289 op/s 2024-04-24T09:15:15.210 INFO:tasks.workunit.client.0.smithi083.stdout: 10 71948 6.53 MB/sec execute 466 sec latency 100.413 ms 2024-04-24T09:15:16.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:15:15 smithi138 ceph-mon[31441]: osdmap e130: 12 total, 12 up, 12 in 2024-04-24T09:15:16.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:15:15 smithi138 ceph-mon[31441]: pgmap v748: 129 pgs: 129 active+clean; 1.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.6 MiB/s wr, 361 op/s 2024-04-24T09:15:16.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:15:15 smithi189 ceph-mon[31791]: osdmap e130: 12 total, 12 up, 12 in 2024-04-24T09:15:16.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:15:15 smithi189 ceph-mon[31791]: pgmap v748: 129 pgs: 129 active+clean; 1.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.6 MiB/s wr, 361 op/s 2024-04-24T09:15:16.210 INFO:tasks.workunit.client.0.smithi083.stdout: 10 72086 6.54 MB/sec execute 467 sec latency 90.561 ms 2024-04-24T09:15:16.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:15:15 smithi083 ceph-mon[27907]: osdmap e130: 12 total, 12 up, 12 in 2024-04-24T09:15:16.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:15:15 smithi083 ceph-mon[27907]: pgmap v748: 129 pgs: 129 active+clean; 1.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.6 MiB/s wr, 361 op/s 2024-04-24T09:15:17.210 INFO:tasks.workunit.client.0.smithi083.stdout: 10 72236 6.54 MB/sec execute 468 sec latency 87.320 ms 2024-04-24T09:15:18.210 INFO:tasks.workunit.client.0.smithi083.stdout: 10 72376 6.54 MB/sec execute 469 sec latency 106.880 ms 2024-04-24T09:15:18.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:15:18 smithi138 ceph-mon[31441]: pgmap v749: 129 pgs: 129 active+clean; 1.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.9 MiB/s wr, 151 op/s 2024-04-24T09:15:18.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:15:18 smithi189 ceph-mon[31791]: pgmap v749: 129 pgs: 129 active+clean; 1.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.9 MiB/s wr, 151 op/s 2024-04-24T09:15:18.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:15:18 smithi083 ceph-mon[27907]: pgmap v749: 129 pgs: 129 active+clean; 1.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.9 MiB/s wr, 151 op/s 2024-04-24T09:15:19.210 INFO:tasks.workunit.client.0.smithi083.stdout: 10 72521 6.55 MB/sec execute 470 sec latency 81.069 ms 2024-04-24T09:15:19.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:15:19 smithi138 ceph-mon[31441]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:15:19.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:15:19 smithi189 ceph-mon[31791]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:15:19.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:15:19 smithi083 ceph-mon[27907]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:15:20.210 INFO:tasks.workunit.client.0.smithi083.stdout: 10 72622 6.55 MB/sec execute 471 sec latency 102.490 ms 2024-04-24T09:15:20.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:15:20 smithi138 ceph-mon[31441]: pgmap v750: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.1 MiB/s wr, 254 op/s 2024-04-24T09:15:20.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:15:20 smithi189 ceph-mon[31791]: pgmap v750: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.1 MiB/s wr, 254 op/s 2024-04-24T09:15:20.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:15:20 smithi083 ceph-mon[27907]: pgmap v750: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.1 MiB/s wr, 254 op/s 2024-04-24T09:15:21.210 INFO:tasks.workunit.client.0.smithi083.stdout: 10 72732 6.54 MB/sec execute 472 sec latency 84.955 ms 2024-04-24T09:15:22.210 INFO:tasks.workunit.client.0.smithi083.stdout: 10 72905 6.56 MB/sec execute 473 sec latency 77.428 ms 2024-04-24T09:15:22.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:15:22 smithi138 ceph-mon[31441]: pgmap v751: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.1 MiB/s wr, 254 op/s 2024-04-24T09:15:22.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:15:22 smithi189 ceph-mon[31791]: pgmap v751: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.1 MiB/s wr, 254 op/s 2024-04-24T09:15:22.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:15:22 smithi083 ceph-mon[27907]: pgmap v751: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.1 MiB/s wr, 254 op/s 2024-04-24T09:15:23.210 INFO:tasks.workunit.client.0.smithi083.stdout: 10 73033 6.55 MB/sec execute 474 sec latency 171.997 ms 2024-04-24T09:15:23.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:15:23 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:15:23.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:15:23 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:15:23.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:15:23 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:15:23.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:15:23 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:15:23.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:15:23 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:15:23.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:15:23 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:15:23.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:15:23 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:15:23.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:15:23 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:15:23.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:15:23 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:15:23.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:15:23 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:15:23.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:15:23 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:15:23.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:15:23 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:15:24.210 INFO:tasks.workunit.client.0.smithi083.stdout: 10 73160 6.55 MB/sec execute 475 sec latency 127.945 ms 2024-04-24T09:15:24.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:15:24 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:15:24.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:15:24 smithi138 ceph-mon[31441]: pgmap v752: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.0 MiB/s wr, 244 op/s 2024-04-24T09:15:24.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:15:24 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:15:24.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:15:24 smithi189 ceph-mon[31791]: pgmap v752: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.0 MiB/s wr, 244 op/s 2024-04-24T09:15:24.552 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.551+0000 7f7297800640 1 -- 172.21.15.83:0/2111560914 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7290109520 msgr2=0x7f72901110f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:24.552 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.551+0000 7f7297800640 1 --2- 172.21.15.83:0/2111560914 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7290109520 0x7f72901110f0 secure :-1 s=READY pgs=2715 cs=0 l=1 rev1=1 crypto rx=0x7f7288009ca0 tx=0x7f728802a8c0 comp rx=0 tx=0).stop 2024-04-24T09:15:24.552 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.551+0000 7f7297800640 1 -- 172.21.15.83:0/2111560914 shutdown_connections 2024-04-24T09:15:24.552 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.551+0000 7f7297800640 1 --2- 172.21.15.83:0/2111560914 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7290109520 0x7f72901110f0 unknown :-1 s=CLOSED pgs=2715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:24.552 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.551+0000 7f7297800640 1 --2- 172.21.15.83:0/2111560914 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7290104e00 0x7f7290108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:24.552 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.551+0000 7f7297800640 1 --2- 172.21.15.83:0/2111560914 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7290104450 0x7f7290104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:24.552 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.551+0000 7f7297800640 1 -- 172.21.15.83:0/2111560914 >> 172.21.15.83:0/2111560914 conn(0x7f72900ffcf0 msgr2=0x7f7290102110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:24.552 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.551+0000 7f7297800640 1 -- 172.21.15.83:0/2111560914 shutdown_connections 2024-04-24T09:15:24.552 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.551+0000 7f7297800640 1 -- 172.21.15.83:0/2111560914 wait complete. 2024-04-24T09:15:24.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:15:24 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:15:24.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:15:24 smithi083 ceph-mon[27907]: pgmap v752: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.0 MiB/s wr, 244 op/s 2024-04-24T09:15:24.554 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.553+0000 7f7297800640 1 Processor -- start 2024-04-24T09:15:24.554 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.553+0000 7f7297800640 1 -- start start 2024-04-24T09:15:24.554 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.553+0000 7f7297800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7290104450 0x7f7290110f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:24.554 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.553+0000 7f7297800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7290109520 0x7f729010dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:24.555 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.553+0000 7f7297800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f729010e4e0 0x7f729010e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:24.555 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.553+0000 7f7297800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f72901144e0 con 0x7f729010e4e0 2024-04-24T09:15:24.555 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.553+0000 7f7297800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7290114360 con 0x7f7290104450 2024-04-24T09:15:24.555 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.553+0000 7f7297800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7290114660 con 0x7f7290109520 2024-04-24T09:15:24.555 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.554+0000 7f7294e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7290104450 0x7f7290110f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:24.555 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.554+0000 7f7294e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7290104450 0x7f7290110f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:38860/0 (socket says 172.21.15.83:38860) 2024-04-24T09:15:24.555 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.554+0000 7f7294e00640 1 -- 172.21.15.83:0/432407924 learned_addr learned my addr 172.21.15.83:0/432407924 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:15:24.555 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.554+0000 7f7295800640 1 --2- 172.21.15.83:0/432407924 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f729010e4e0 0x7f729010e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:24.555 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.554+0000 7f728fe00640 1 --2- 172.21.15.83:0/432407924 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7290109520 0x7f729010dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:24.555 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.554+0000 7f7294e00640 1 -- 172.21.15.83:0/432407924 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7290109520 msgr2=0x7f729010dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:24.555 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.554+0000 7f7294e00640 1 --2- 172.21.15.83:0/432407924 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7290109520 0x7f729010dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:24.555 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.554+0000 7f7294e00640 1 -- 172.21.15.83:0/432407924 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f729010e4e0 msgr2=0x7f729010e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:24.555 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.554+0000 7f7294e00640 1 --2- 172.21.15.83:0/432407924 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f729010e4e0 0x7f729010e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:24.555 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.554+0000 7f7294e00640 1 -- 172.21.15.83:0/432407924 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72901ab3b0 con 0x7f7290104450 2024-04-24T09:15:24.556 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.555+0000 7f7294e00640 1 --2- 172.21.15.83:0/432407924 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7290104450 0x7f7290110f00 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f728400d7a0 tx=0x7f728400dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:24.556 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.555+0000 7f728d600640 1 -- 172.21.15.83:0/432407924 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7284017070 con 0x7f7290104450 2024-04-24T09:15:24.557 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.555+0000 7f728d600640 1 -- 172.21.15.83:0/432407924 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f72840054d0 con 0x7f7290104450 2024-04-24T09:15:24.557 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.555+0000 7f728d600640 1 -- 172.21.15.83:0/432407924 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7284004cd0 con 0x7f7290104450 2024-04-24T09:15:24.557 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.555+0000 7f7297800640 1 -- 172.21.15.83:0/432407924 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72901ab6a0 con 0x7f7290104450 2024-04-24T09:15:24.557 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.555+0000 7f7297800640 1 -- 172.21.15.83:0/432407924 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72901abba0 con 0x7f7290104450 2024-04-24T09:15:24.559 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.559+0000 7f729a607740 1 Processor -- start 2024-04-24T09:15:24.559 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.559+0000 7f729a607740 1 -- start start 2024-04-24T09:15:24.560 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.559+0000 7f729a607740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c7a7a6b1b0 0x55c7a7a6b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:24.560 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.559+0000 7f729a607740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7a7a6bad0 0x55c7a7a75e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:24.560 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.559+0000 7f729a607740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c7a7a76390 0x55c7a7a78780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:24.560 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.559+0000 7f729a607740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c7a79d1900 con 0x55c7a7a6bad0 2024-04-24T09:15:24.560 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.559+0000 7f729a607740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c7a7977f20 con 0x55c7a7a76390 2024-04-24T09:15:24.560 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.559+0000 7f729a607740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55c7a79cd8c0 con 0x55c7a7a6b1b0 2024-04-24T09:15:24.561 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.560+0000 7f728d600640 1 -- 172.21.15.83:0/432407924 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7284004e70 con 0x7f7290104450 2024-04-24T09:15:24.561 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.560+0000 7f728fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c7a7a76390 0x55c7a7a78780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:24.561 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.560+0000 7f728fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c7a7a76390 0x55c7a7a78780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:38866/0 (socket says 172.21.15.83:38866) 2024-04-24T09:15:24.561 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.560+0000 7f728fe00640 1 -- 172.21.15.83:0/4056511590 learned_addr learned my addr 172.21.15.83:0/4056511590 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:15:24.561 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.560+0000 7f7295800640 1 --2- 172.21.15.83:0/4056511590 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c7a7a6b1b0 0x55c7a7a6b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:24.561 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.560+0000 7f7294e00640 1 --2- 172.21.15.83:0/4056511590 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7a7a6bad0 0x55c7a7a75e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:24.561 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.561+0000 7f728fe00640 1 -- 172.21.15.83:0/4056511590 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c7a7a6b1b0 msgr2=0x55c7a7a6b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:24.561 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.561+0000 7f728fe00640 1 --2- 172.21.15.83:0/4056511590 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c7a7a6b1b0 0x55c7a7a6b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:24.562 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.561+0000 7f728fe00640 1 -- 172.21.15.83:0/4056511590 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7a7a6bad0 msgr2=0x55c7a7a75e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:24.562 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.561+0000 7f728fe00640 1 --2- 172.21.15.83:0/4056511590 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7a7a6bad0 0x55c7a7a75e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:24.562 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.561+0000 7f728fe00640 1 -- 172.21.15.83:0/4056511590 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7a799ce10 con 0x55c7a7a76390 2024-04-24T09:15:24.562 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.561+0000 7f7294e00640 1 --2- 172.21.15.83:0/4056511590 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7a7a6bad0 0x55c7a7a75e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:15:24.563 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.561+0000 7f728fe00640 1 --2- 172.21.15.83:0/4056511590 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c7a7a76390 0x55c7a7a78780 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f727c009b30 tx=0x7f727c0113e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c6f229ff6da3dd8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:24.564 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.562+0000 7f7282000640 1 -- 172.21.15.83:0/4056511590 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f727c011e10 con 0x55c7a7a76390 2024-04-24T09:15:24.564 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.562+0000 7f7282000640 1 -- 172.21.15.83:0/4056511590 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f727c016070 con 0x55c7a7a76390 2024-04-24T09:15:24.564 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.562+0000 7f7282000640 1 -- 172.21.15.83:0/4056511590 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f727c01b470 con 0x55c7a7a76390 2024-04-24T09:15:24.564 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.562+0000 7f728d600640 1 --2- 172.21.15.83:0/432407924 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f72780a3df0 0x7f72780a62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:24.564 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.562+0000 7f728d600640 1 -- 172.21.15.83:0/432407924 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 9583+0+0 (secure 0 0 0) 0x7f72840d06a0 con 0x7f7290104450 2024-04-24T09:15:24.564 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.562+0000 7f7295800640 1 --2- 172.21.15.83:0/432407924 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f72780a3df0 0x7f72780a62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:24.564 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.563+0000 7f729a607740 1 -- 172.21.15.83:0/4056511590 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c7a7a76390 msgr2=0x55c7a7a78780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:24.564 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.563+0000 7f729a607740 1 --2- 172.21.15.83:0/4056511590 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c7a7a76390 0x55c7a7a78780 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f727c009b30 tx=0x7f727c0113e0 comp rx=0 tx=0).stop 2024-04-24T09:15:24.564 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.563+0000 7f729a607740 1 -- 172.21.15.83:0/4056511590 shutdown_connections 2024-04-24T09:15:24.564 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.563+0000 7f729a607740 1 --2- 172.21.15.83:0/4056511590 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c7a7a76390 0x55c7a7a78780 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:24.564 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.563+0000 7f729a607740 1 --2- 172.21.15.83:0/4056511590 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7a7a6bad0 0x55c7a7a75e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:24.564 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.563+0000 7f729a607740 1 --2- 172.21.15.83:0/4056511590 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c7a7a6b1b0 0x55c7a7a6b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:24.564 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.563+0000 7f729a607740 1 -- 172.21.15.83:0/4056511590 >> 172.21.15.83:0/4056511590 conn(0x55c7a7a676c0 msgr2=0x55c7a7a67aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:24.564 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.563+0000 7f729a607740 1 -- 172.21.15.83:0/4056511590 shutdown_connections 2024-04-24T09:15:24.565 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.563+0000 7f729a607740 1 -- 172.21.15.83:0/4056511590 wait complete. 2024-04-24T09:15:24.565 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.563+0000 7f7295800640 1 --2- 172.21.15.83:0/432407924 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f72780a3df0 0x7f72780a62b0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f7288006010 tx=0x7f7288004a60 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:24.565 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.564+0000 7f729a607740 1 -- start start 2024-04-24T09:15:24.565 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.564+0000 7f729a607740 1 Processor -- start 2024-04-24T09:15:24.566 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.564+0000 7f729a607740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c7a7a6b1b0 0x55c7a7a73300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:24.566 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.564+0000 7f729a607740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c7a7a6bad0 0x55c7a7a73840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:24.566 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.564+0000 7f729a607740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7a7a76390 0x55c7a7a73d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:24.567 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.564+0000 7f729a607740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c7a7a78fc0 con 0x55c7a7a76390 2024-04-24T09:15:24.567 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.564+0000 7f729a607740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c7a798b2c0 con 0x55c7a7a6b1b0 2024-04-24T09:15:24.567 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.564+0000 7f729a607740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55c7a7a79140 con 0x55c7a7a6bad0 2024-04-24T09:15:24.567 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.565+0000 7f729a607740 10 client.0 _ll_register_callbacks cb 0x55c7a76e3a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:15:24.567 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.565+0000 7f728fe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c7a7a6bad0 0x55c7a7a73840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:24.567 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.565+0000 7f728fe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c7a7a6bad0 0x55c7a7a73840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:36514/0 (socket says 172.21.15.83:36514) 2024-04-24T09:15:24.567 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.565+0000 7f728fe00640 1 -- 172.21.15.83:0/1466775497 learned_addr learned my addr 172.21.15.83:0/1466775497 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:15:24.567 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.565+0000 7f7295800640 1 --2- 172.21.15.83:0/1466775497 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7a7a76390 0x55c7a7a73d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:24.567 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.565+0000 7f7294e00640 1 --2- 172.21.15.83:0/1466775497 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c7a7a6b1b0 0x55c7a7a73300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:24.567 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.565+0000 7f728fe00640 1 -- 172.21.15.83:0/1466775497 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c7a7a6b1b0 msgr2=0x55c7a7a73300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:24.567 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.566+0000 7f728fe00640 1 --2- 172.21.15.83:0/1466775497 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c7a7a6b1b0 0x55c7a7a73300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:24.568 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.566+0000 7f728fe00640 1 -- 172.21.15.83:0/1466775497 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7a7a76390 msgr2=0x55c7a7a73d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:24.568 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.566+0000 7f728fe00640 1 --2- 172.21.15.83:0/1466775497 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7a7a76390 0x55c7a7a73d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:24.568 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.566+0000 7f728fe00640 1 -- 172.21.15.83:0/1466775497 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7a799bfd0 con 0x55c7a7a6bad0 2024-04-24T09:15:24.568 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.566+0000 7f7295800640 1 --2- 172.21.15.83:0/1466775497 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7a7a76390 0x55c7a7a73d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:15:24.568 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.566+0000 7f7294e00640 1 --2- 172.21.15.83:0/1466775497 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c7a7a6b1b0 0x55c7a7a73300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:15:24.568 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.566+0000 7f728fe00640 1 --2- 172.21.15.83:0/1466775497 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c7a7a6bad0 0x55c7a7a73840 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f727c011e20 tx=0x7f727c011e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aae71f892bf4f518 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:24.568 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.566+0000 7f7276a00640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-04-24T09:15:24.570 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.567+0000 7f7276a00640 1 -- 172.21.15.83:0/1466775497 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f727c003800 con 0x55c7a7a6bad0 2024-04-24T09:15:24.570 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.567+0000 7f729a607740 1 -- 172.21.15.83:0/1466775497 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f727c016070 con 0x55c7a7a6bad0 2024-04-24T09:15:24.570 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.567+0000 7f7276a00640 1 -- 172.21.15.83:0/1466775497 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(130..130 src has 1..130) ==== 9583+0+0 (secure 0 0 0) 0x7f727c004050 con 0x55c7a7a6bad0 2024-04-24T09:15:24.570 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.567+0000 7f7276a00640 1 -- 172.21.15.83:0/1466775497 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f727c030050 con 0x55c7a7a6bad0 2024-04-24T09:15:24.570 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.568+0000 7f729a607740 10 client.30403 fetch_fsmap learned FSMap version 69 2024-04-24T09:15:24.570 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.568+0000 7f729a607740 1 -- 172.21.15.83:0/1466775497 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x55c7a76e4210 con 0x55c7a7a6bad0 2024-04-24T09:15:24.570 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.568+0000 7f7276a00640 1 -- 172.21.15.83:0/1466775497 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f727c016070 con 0x55c7a7a6bad0 2024-04-24T09:15:24.570 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.568+0000 7f7276a00640 1 -- 172.21.15.83:0/1466775497 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 69) ==== 8370+0+0 (secure 0 0 0) 0x7f727c01bc20 con 0x55c7a7a6bad0 2024-04-24T09:15:24.570 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.569+0000 7f729a607740 10 client.30403 fetch_fsmap finished waiting for FSMap version 69 2024-04-24T09:15:24.571 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.569+0000 7f729a607740 10 client.30403 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:15:24.571 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.569+0000 7f729a607740 20 client.30403 populate_metadata read hostname 'smithi083' 2024-04-24T09:15:24.571 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.569+0000 7f729a607740 1 --2- 172.21.15.83:0/1466775497 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55c7a7a86400 0x55c7a7a887f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:24.571 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.569+0000 7f729a607740 4 client.30403 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:15:24.571 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.569+0000 7f729a607740 1 -- 172.21.15.83:0/1466775497 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c7a79df320 con 0x55c7a7a86400 2024-04-24T09:15:24.571 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.570+0000 7f7294e00640 1 --2- 172.21.15.83:0/1466775497 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55c7a7a86400 0x55c7a7a887f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:24.571 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.571+0000 7f7294e00640 1 --2- 172.21.15.83:0/1466775497 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55c7a7a86400 0x55c7a7a887f0 crc :-1 s=READY pgs=583 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31c31ca43a0f2b65 server_cookie=f5e7d48f36ccae5b in_seq=0 out_seq=0 2024-04-24T09:15:24.572 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.571+0000 7f7276a00640 10 client.30403 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:15:24.572 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.572+0000 7f7276a00640 1 -- 172.21.15.83:0/1466775497 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c7a79df320 con 0x55c7a7a86400 2024-04-24T09:15:24.572 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.572+0000 7f7276a00640 10 client.30403 handle_command_reply: tid=0 2024-04-24T09:15:24.572 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.572+0000 7f729a607740 1 client.30403 shutdown 2024-04-24T09:15:24.573 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.572+0000 7f729a607740 1 -- 172.21.15.83:0/1466775497 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c7a7a6bad0 msgr2=0x55c7a7a73840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:24.573 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.572+0000 7f729a607740 1 --2- 172.21.15.83:0/1466775497 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c7a7a6bad0 0x55c7a7a73840 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f727c011e20 tx=0x7f727c011e50 comp rx=0 tx=0).stop 2024-04-24T09:15:24.573 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.573+0000 7f729a607740 1 -- 172.21.15.83:0/1466775497 shutdown_connections 2024-04-24T09:15:24.573 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.573+0000 7f729a607740 1 --2- 172.21.15.83:0/1466775497 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55c7a7a86400 0x55c7a7a887f0 crc :-1 s=READY pgs=583 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:24.573 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.573+0000 7f729a607740 1 --2- 172.21.15.83:0/1466775497 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7a7a76390 0x55c7a7a73d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:24.573 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.573+0000 7f729a607740 1 --2- 172.21.15.83:0/1466775497 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c7a7a6bad0 0x55c7a7a73840 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:24.574 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.573+0000 7f729a607740 1 --2- 172.21.15.83:0/1466775497 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c7a7a6b1b0 0x55c7a7a73300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:24.574 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.573+0000 7f729a607740 1 -- 172.21.15.83:0/1466775497 >> 172.21.15.83:0/1466775497 conn(0x55c7a7a676c0 msgr2=0x55c7a7a6d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:24.574 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.573+0000 7f7276a00640 0 client.30403 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:15:24.574 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.573+0000 7f729a607740 1 -- 172.21.15.83:0/1466775497 shutdown_connections 2024-04-24T09:15:24.574 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.573+0000 7f729a607740 1 -- 172.21.15.83:0/1466775497 wait complete. 2024-04-24T09:15:24.574 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.574+0000 7f729a607740 20 client.30403 trim_cache size 0 max 16384 2024-04-24T09:15:24.598 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.598+0000 7f729a607740 1 Processor -- start 2024-04-24T09:15:24.598 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.598+0000 7f729a607740 1 -- start start 2024-04-24T09:15:24.599 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.598+0000 7f729a607740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c7a7a6b1b0 0x55c7a7a66fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:24.599 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.598+0000 7f729a607740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c7a7a6bad0 0x55c7a7a8b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:24.599 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.598+0000 7f729a607740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7a7a76390 0x55c7a7a8bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:24.599 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.598+0000 7f729a607740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f72901b5b70 con 0x55c7a7a76390 2024-04-24T09:15:24.599 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.598+0000 7f729a607740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f72901b7710 con 0x55c7a7a6bad0 2024-04-24T09:15:24.599 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.598+0000 7f729a607740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f72901b6240 con 0x55c7a7a6b1b0 2024-04-24T09:15:24.600 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.599+0000 7f7295800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7a7a76390 0x55c7a7a8bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:24.600 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.599+0000 7f728fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c7a7a6bad0 0x55c7a7a8b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:24.600 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.599+0000 7f728fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c7a7a6bad0 0x55c7a7a8b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:38892/0 (socket says 172.21.15.83:38892) 2024-04-24T09:15:24.600 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.599+0000 7f728fe00640 1 -- 172.21.15.83:0/901922695 learned_addr learned my addr 172.21.15.83:0/901922695 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:15:24.600 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.599+0000 7f7294e00640 1 --2- 172.21.15.83:0/901922695 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c7a7a6b1b0 0x55c7a7a66fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:24.601 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.600+0000 7f728fe00640 1 -- 172.21.15.83:0/901922695 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c7a7a6b1b0 msgr2=0x55c7a7a66fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:24.601 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.600+0000 7f728fe00640 1 --2- 172.21.15.83:0/901922695 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c7a7a6b1b0 0x55c7a7a66fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:24.601 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.600+0000 7f728fe00640 1 -- 172.21.15.83:0/901922695 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7a7a76390 msgr2=0x55c7a7a8bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:24.601 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.600+0000 7f728fe00640 1 --2- 172.21.15.83:0/901922695 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7a7a76390 0x55c7a7a8bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:24.601 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.600+0000 7f728fe00640 1 -- 172.21.15.83:0/901922695 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7a77b0620 con 0x55c7a7a6bad0 2024-04-24T09:15:24.601 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.600+0000 7f7295800640 1 --2- 172.21.15.83:0/901922695 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7a7a76390 0x55c7a7a8bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:15:24.601 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.600+0000 7f728fe00640 1 --2- 172.21.15.83:0/901922695 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c7a7a6bad0 0x55c7a7a8b810 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f727c012610 tx=0x7f727c012710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4286dfc23fa746b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:24.601 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.600+0000 7f7281600640 1 -- 172.21.15.83:0/901922695 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f727c01f070 con 0x55c7a7a6bad0 2024-04-24T09:15:24.601 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.600+0000 7f7281600640 1 -- 172.21.15.83:0/901922695 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f727c016070 con 0x55c7a7a6bad0 2024-04-24T09:15:24.602 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.601+0000 7f7281600640 1 -- 172.21.15.83:0/901922695 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f727c0052d0 con 0x55c7a7a6bad0 2024-04-24T09:15:24.602 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.601+0000 7f729a607740 1 -- 172.21.15.83:0/901922695 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c7a7a6bad0 msgr2=0x55c7a7a8b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:24.603 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.601+0000 7f729a607740 1 --2- 172.21.15.83:0/901922695 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c7a7a6bad0 0x55c7a7a8b810 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f727c012610 tx=0x7f727c012710 comp rx=0 tx=0).stop 2024-04-24T09:15:24.603 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.601+0000 7f729a607740 1 -- 172.21.15.83:0/901922695 shutdown_connections 2024-04-24T09:15:24.603 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.601+0000 7f729a607740 1 --2- 172.21.15.83:0/901922695 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7a7a76390 0x55c7a7a8bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:24.603 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.601+0000 7f729a607740 1 --2- 172.21.15.83:0/901922695 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c7a7a6bad0 0x55c7a7a8b810 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:24.603 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.601+0000 7f729a607740 1 --2- 172.21.15.83:0/901922695 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c7a7a6b1b0 0x55c7a7a66fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:24.603 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.601+0000 7f729a607740 1 -- 172.21.15.83:0/901922695 >> 172.21.15.83:0/901922695 conn(0x55c7a7a676c0 msgr2=0x55c7a7a81c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:24.603 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.601+0000 7f729a607740 1 -- 172.21.15.83:0/901922695 shutdown_connections 2024-04-24T09:15:24.603 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.602+0000 7f729a607740 1 -- 172.21.15.83:0/901922695 wait complete. 2024-04-24T09:15:24.603 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.602+0000 7f729a607740 1 -- start start 2024-04-24T09:15:24.603 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.602+0000 7f729a607740 1 Processor -- start 2024-04-24T09:15:24.605 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.603+0000 7f729a607740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c7a7a6b1b0 0x55c7a7a8bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:24.605 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.603+0000 7f729a607740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7a7a6bad0 0x55c7a7a890a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:24.605 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.603+0000 7f729a607740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c7a7a76390 0x55c7a7a897e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:24.605 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.603+0000 7f729a607740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f72901b7040 con 0x55c7a7a6bad0 2024-04-24T09:15:24.605 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.603+0000 7f729a607740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f72901b6970 con 0x55c7a7a76390 2024-04-24T09:15:24.605 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.603+0000 7f729a607740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f72901b3b70 con 0x55c7a7a6b1b0 2024-04-24T09:15:24.605 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.603+0000 7f729a607740 10 client.0 _ll_register_callbacks cb 0x7f7288009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:15:24.605 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.603+0000 7f7295800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c7a7a76390 0x55c7a7a897e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:24.605 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.603+0000 7f7295800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c7a7a76390 0x55c7a7a897e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:38896/0 (socket says 172.21.15.83:38896) 2024-04-24T09:15:24.606 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.603+0000 7f7295800640 1 -- 172.21.15.83:0/609180380 learned_addr learned my addr 172.21.15.83:0/609180380 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:15:24.606 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.603+0000 7f728fe00640 1 --2- 172.21.15.83:0/609180380 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7a7a6bad0 0x55c7a7a890a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:24.606 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.603+0000 7f7294e00640 1 --2- 172.21.15.83:0/609180380 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c7a7a6b1b0 0x55c7a7a8bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:24.606 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.604+0000 7f728fe00640 1 -- 172.21.15.83:0/609180380 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c7a7a6b1b0 msgr2=0x55c7a7a8bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:24.606 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.604+0000 7f728fe00640 1 --2- 172.21.15.83:0/609180380 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c7a7a6b1b0 0x55c7a7a8bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:24.606 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.604+0000 7f728fe00640 1 -- 172.21.15.83:0/609180380 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c7a7a76390 msgr2=0x55c7a7a897e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:24.606 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.604+0000 7f728fe00640 1 --2- 172.21.15.83:0/609180380 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c7a7a76390 0x55c7a7a897e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:24.606 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.604+0000 7f728fe00640 1 -- 172.21.15.83:0/609180380 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7a7aeb5e0 con 0x55c7a7a6bad0 2024-04-24T09:15:24.606 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.604+0000 7f7295800640 1 --2- 172.21.15.83:0/609180380 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c7a7a76390 0x55c7a7a897e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:15:24.606 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.604+0000 7f728fe00640 1 --2- 172.21.15.83:0/609180380 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7a7a6bad0 0x55c7a7a890a0 secure :-1 s=READY pgs=2716 cs=0 l=1 rev1=1 crypto rx=0x7f727c006010 tx=0x7f727c012c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a16c0b0972e37e5e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:24.606 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.604+0000 7f7276a00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:15:24.606 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.604+0000 7f7276a00640 1 -- 172.21.15.83:0/609180380 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f727c039060 con 0x55c7a7a6bad0 2024-04-24T09:15:24.607 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.604+0000 7f7276a00640 1 -- 172.21.15.83:0/609180380 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(130..130 src has 1..130) ==== 9583+0+0 (secure 0 0 0) 0x7f727c003a10 con 0x55c7a7a6bad0 2024-04-24T09:15:24.607 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.605+0000 7f729a607740 1 -- 172.21.15.83:0/609180380 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f727c016070 con 0x55c7a7a6bad0 2024-04-24T09:15:24.608 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.605+0000 7f7276a00640 1 -- 172.21.15.83:0/609180380 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f727c01f040 con 0x55c7a7a6bad0 2024-04-24T09:15:24.608 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.605+0000 7f729a607740 10 client.23637 fetch_fsmap learned FSMap version 69 2024-04-24T09:15:24.608 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.605+0000 7f729a607740 1 -- 172.21.15.83:0/609180380 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x55c7a7aec790 con 0x55c7a7a6bad0 2024-04-24T09:15:24.608 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.605+0000 7f7276a00640 1 -- 172.21.15.83:0/609180380 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f727c016070 con 0x55c7a7a6bad0 2024-04-24T09:15:24.608 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.606+0000 7f7276a00640 1 -- 172.21.15.83:0/609180380 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 69) ==== 8370+0+0 (secure 0 0 0) 0x7f727c027420 con 0x55c7a7a6bad0 2024-04-24T09:15:24.608 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.606+0000 7f729a607740 10 client.23637 fetch_fsmap finished waiting for FSMap version 69 2024-04-24T09:15:24.608 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.606+0000 7f729a607740 10 client.23637 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:15:24.608 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.606+0000 7f729a607740 20 client.23637 populate_metadata read hostname 'smithi083' 2024-04-24T09:15:24.608 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.606+0000 7f729a607740 1 --2- 172.21.15.83:0/609180380 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55c7a7aece20 0x55c7a7aef240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:24.608 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.606+0000 7f729a607740 4 client.23637 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T09:15:24.608 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.606+0000 7f729a607740 1 -- 172.21.15.83:0/609180380 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c7a79d9190 con 0x55c7a7aece20 2024-04-24T09:15:24.608 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.607+0000 7f7294e00640 1 --2- 172.21.15.83:0/609180380 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55c7a7aece20 0x55c7a7aef240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:24.609 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.609+0000 7f7294e00640 1 --2- 172.21.15.83:0/609180380 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55c7a7aece20 0x55c7a7aef240 crc :-1 s=READY pgs=584 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1799a8c2f3c6eb3a server_cookie=c4a90b3a599f46fa in_seq=0 out_seq=0 2024-04-24T09:15:24.609 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.609+0000 7f7276a00640 10 client.23637 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:15:24.763 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.763+0000 7f7276a00640 1 -- 172.21.15.83:0/609180380 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c7a79d9190 con 0x55c7a7aece20 2024-04-24T09:15:24.763 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.763+0000 7f7276a00640 10 client.23637 handle_command_reply: tid=0 2024-04-24T09:15:24.763 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.763+0000 7f729a607740 1 client.23637 shutdown 2024-04-24T09:15:24.764 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.763+0000 7f729a607740 1 -- 172.21.15.83:0/609180380 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7a7a6bad0 msgr2=0x55c7a7a890a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:24.764 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.763+0000 7f729a607740 1 --2- 172.21.15.83:0/609180380 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7a7a6bad0 0x55c7a7a890a0 secure :-1 s=READY pgs=2716 cs=0 l=1 rev1=1 crypto rx=0x7f727c006010 tx=0x7f727c012c10 comp rx=0 tx=0).stop 2024-04-24T09:15:24.764 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.764+0000 7f729a607740 1 -- 172.21.15.83:0/609180380 shutdown_connections 2024-04-24T09:15:24.765 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.764+0000 7f729a607740 1 --2- 172.21.15.83:0/609180380 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55c7a7aece20 0x55c7a7aef240 crc :-1 s=READY pgs=584 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:24.765 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.764+0000 7f729a607740 1 --2- 172.21.15.83:0/609180380 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c7a7a76390 0x55c7a7a897e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:24.765 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.764+0000 7f729a607740 1 --2- 172.21.15.83:0/609180380 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7a7a6bad0 0x55c7a7a890a0 unknown :-1 s=CLOSED pgs=2716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:24.765 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.764+0000 7f729a607740 1 --2- 172.21.15.83:0/609180380 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c7a7a6b1b0 0x55c7a7a8bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:24.765 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.764+0000 7f729a607740 1 -- 172.21.15.83:0/609180380 >> 172.21.15.83:0/609180380 conn(0x55c7a7a676c0 msgr2=0x55c7a7a732c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:24.765 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.764+0000 7f7276a00640 0 client.23637 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:15:24.765 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.765+0000 7f729a607740 1 -- 172.21.15.83:0/609180380 shutdown_connections 2024-04-24T09:15:24.765 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.765+0000 7f729a607740 1 -- 172.21.15.83:0/609180380 wait complete. 2024-04-24T09:15:24.766 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.765+0000 7f729a607740 20 client.23637 trim_cache size 0 max 16384 2024-04-24T09:15:24.767 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T09:15:24.767 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T09:15:24.767 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T09:15:24.767 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T09:15:24.768 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.767+0000 7f7297800640 1 -- 172.21.15.83:0/432407924 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f72780a3df0 msgr2=0x7f72780a62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:24.768 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.767+0000 7f7297800640 1 --2- 172.21.15.83:0/432407924 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f72780a3df0 0x7f72780a62b0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f7288006010 tx=0x7f7288004a60 comp rx=0 tx=0).stop 2024-04-24T09:15:24.768 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.767+0000 7f7297800640 1 -- 172.21.15.83:0/432407924 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7290104450 msgr2=0x7f7290110f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:24.768 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.767+0000 7f7297800640 1 --2- 172.21.15.83:0/432407924 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7290104450 0x7f7290110f00 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f728400d7a0 tx=0x7f728400dc60 comp rx=0 tx=0).stop 2024-04-24T09:15:24.768 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.767+0000 7f7297800640 1 -- 172.21.15.83:0/432407924 shutdown_connections 2024-04-24T09:15:24.768 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.767+0000 7f7297800640 1 --2- 172.21.15.83:0/432407924 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f72780a3df0 0x7f72780a62b0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:24.768 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.767+0000 7f7297800640 1 --2- 172.21.15.83:0/432407924 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f729010e4e0 0x7f729010e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:24.768 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.767+0000 7f7297800640 1 --2- 172.21.15.83:0/432407924 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7290109520 0x7f729010dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:24.768 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.767+0000 7f7297800640 1 --2- 172.21.15.83:0/432407924 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7290104450 0x7f7290110f00 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:24.768 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.767+0000 7f7297800640 1 -- 172.21.15.83:0/432407924 >> 172.21.15.83:0/432407924 conn(0x7f72900ffcf0 msgr2=0x7f7290100790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:24.768 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.768+0000 7f7297800640 1 -- 172.21.15.83:0/432407924 shutdown_connections 2024-04-24T09:15:24.768 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:24.768+0000 7f7297800640 1 -- 172.21.15.83:0/432407924 wait complete. 2024-04-24T09:15:25.210 INFO:tasks.workunit.client.0.smithi083.stdout: 10 73281 6.55 MB/sec execute 476 sec latency 128.925 ms 2024-04-24T09:15:26.210 INFO:tasks.workunit.client.0.smithi083.stdout: 10 73365 6.54 MB/sec execute 477 sec latency 288.073 ms 2024-04-24T09:15:26.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:15:26 smithi138 ceph-mon[31441]: pgmap v753: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.9 MiB/s wr, 239 op/s 2024-04-24T09:15:26.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:15:26 smithi189 ceph-mon[31791]: pgmap v753: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.9 MiB/s wr, 239 op/s 2024-04-24T09:15:26.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:15:26 smithi083 ceph-mon[27907]: pgmap v753: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.9 MiB/s wr, 239 op/s 2024-04-24T09:15:27.210 INFO:tasks.workunit.client.0.smithi083.stdout: 10 73484 6.54 MB/sec execute 478 sec latency 186.042 ms 2024-04-24T09:15:28.211 INFO:tasks.workunit.client.0.smithi083.stdout: 10 73618 6.54 MB/sec execute 479 sec latency 76.750 ms 2024-04-24T09:15:28.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:15:28 smithi138 ceph-mon[31441]: pgmap v754: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.2 MiB/s wr, 203 op/s 2024-04-24T09:15:28.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:15:28 smithi189 ceph-mon[31791]: pgmap v754: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.2 MiB/s wr, 203 op/s 2024-04-24T09:15:28.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:15:28 smithi083 ceph-mon[27907]: pgmap v754: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.2 MiB/s wr, 203 op/s 2024-04-24T09:15:29.211 INFO:tasks.workunit.client.0.smithi083.stdout: 10 73759 6.55 MB/sec execute 480 sec latency 84.343 ms 2024-04-24T09:15:30.211 INFO:tasks.workunit.client.0.smithi083.stdout: 10 73862 6.54 MB/sec execute 481 sec latency 379.372 ms 2024-04-24T09:15:30.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:15:30 smithi138 ceph-mon[31441]: pgmap v755: 129 pgs: 129 active+clean; 1.1 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.9 MiB/s wr, 340 op/s 2024-04-24T09:15:30.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:15:30 smithi189 ceph-mon[31791]: pgmap v755: 129 pgs: 129 active+clean; 1.1 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.9 MiB/s wr, 340 op/s 2024-04-24T09:15:30.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:15:30 smithi083 ceph-mon[27907]: pgmap v755: 129 pgs: 129 active+clean; 1.1 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.9 MiB/s wr, 340 op/s 2024-04-24T09:15:31.211 INFO:tasks.workunit.client.0.smithi083.stdout: 10 73940 6.53 MB/sec execute 482 sec latency 411.441 ms 2024-04-24T09:15:32.211 INFO:tasks.workunit.client.0.smithi083.stdout: 10 74064 6.53 MB/sec execute 483 sec latency 251.940 ms 2024-04-24T09:15:32.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:15:32 smithi138 ceph-mon[31441]: pgmap v756: 129 pgs: 129 active+clean; 1.1 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.8 MiB/s wr, 241 op/s 2024-04-24T09:15:32.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:15:32 smithi189 ceph-mon[31791]: pgmap v756: 129 pgs: 129 active+clean; 1.1 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.8 MiB/s wr, 241 op/s 2024-04-24T09:15:32.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:15:32 smithi083 ceph-mon[27907]: pgmap v756: 129 pgs: 129 active+clean; 1.1 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.8 MiB/s wr, 241 op/s 2024-04-24T09:15:33.211 INFO:tasks.workunit.client.0.smithi083.stdout: 10 74203 6.54 MB/sec execute 484 sec latency 626.057 ms 2024-04-24T09:15:34.211 INFO:tasks.workunit.client.0.smithi083.stdout: 10 74353 6.55 MB/sec execute 485 sec latency 88.433 ms 2024-04-24T09:15:34.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:15:34 smithi138 ceph-mon[31441]: pgmap v757: 129 pgs: 129 active+clean; 1.1 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.6 MiB/s wr, 327 op/s 2024-04-24T09:15:34.459 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:15:34 smithi189 ceph-mon[31791]: pgmap v757: 129 pgs: 129 active+clean; 1.1 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.6 MiB/s wr, 327 op/s 2024-04-24T09:15:34.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:15:34 smithi083 ceph-mon[27907]: pgmap v757: 129 pgs: 129 active+clean; 1.1 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.6 MiB/s wr, 327 op/s 2024-04-24T09:15:35.211 INFO:tasks.workunit.client.0.smithi083.stdout: 10 74466 6.55 MB/sec execute 486 sec latency 77.092 ms 2024-04-24T09:15:36.212 INFO:tasks.workunit.client.0.smithi083.stdout: 10 74586 6.55 MB/sec execute 487 sec latency 128.907 ms 2024-04-24T09:15:36.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:15:36 smithi138 ceph-mon[31441]: pgmap v758: 129 pgs: 129 active+clean; 1.1 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.6 MiB/s wr, 223 op/s 2024-04-24T09:15:36.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:15:36 smithi189 ceph-mon[31791]: pgmap v758: 129 pgs: 129 active+clean; 1.1 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.6 MiB/s wr, 223 op/s 2024-04-24T09:15:36.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:15:36 smithi083 ceph-mon[27907]: pgmap v758: 129 pgs: 129 active+clean; 1.1 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.6 MiB/s wr, 223 op/s 2024-04-24T09:15:37.212 INFO:tasks.workunit.client.0.smithi083.stdout: 10 74699 6.55 MB/sec execute 488 sec latency 144.936 ms 2024-04-24T09:15:37.638 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T09:15:37.639 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T09:15:37.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:37.981+0000 7f5f1e400640 1 -- 172.21.15.83:0/3462662527 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f18109580 msgr2=0x7f5f18111150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:37.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:37.981+0000 7f5f1e400640 1 --2- 172.21.15.83:0/3462662527 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f18109580 0x7f5f18111150 secure :-1 s=READY pgs=2717 cs=0 l=1 rev1=1 crypto rx=0x7f5f10009be0 tx=0x7f5f1002a6e0 comp rx=0 tx=0).stop 2024-04-24T09:15:37.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:37.981+0000 7f5f1e400640 1 -- 172.21.15.83:0/3462662527 shutdown_connections 2024-04-24T09:15:37.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:37.981+0000 7f5f1e400640 1 --2- 172.21.15.83:0/3462662527 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f18109580 0x7f5f18111150 unknown :-1 s=CLOSED pgs=2717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:37.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:37.981+0000 7f5f1e400640 1 --2- 172.21.15.83:0/3462662527 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f18104e60 0x7f5f18108ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:37.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:37.981+0000 7f5f1e400640 1 --2- 172.21.15.83:0/3462662527 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5f181044b0 0x7f5f18104890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:37.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:37.981+0000 7f5f1e400640 1 -- 172.21.15.83:0/3462662527 >> 172.21.15.83:0/3462662527 conn(0x7f5f180ffd90 msgr2=0x7f5f181021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:37.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:37.981+0000 7f5f1e400640 1 -- 172.21.15.83:0/3462662527 shutdown_connections 2024-04-24T09:15:37.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:37.982+0000 7f5f1e400640 1 -- 172.21.15.83:0/3462662527 wait complete. 2024-04-24T09:15:37.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:37.983+0000 7f5f1e400640 1 Processor -- start 2024-04-24T09:15:37.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:37.983+0000 7f5f1e400640 1 -- start start 2024-04-24T09:15:37.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:37.983+0000 7f5f1e400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5f181044b0 0x7f5f18076510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:37.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:37.984+0000 7f5f1e400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f18109580 0x7f5f18076a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:37.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:37.984+0000 7f5f1e400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f18076f90 0x7f5f181ab020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:37.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:37.984+0000 7f5f1e400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5f181144f0 con 0x7f5f18076f90 2024-04-24T09:15:37.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:37.984+0000 7f5f1e400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5f18114370 con 0x7f5f18109580 2024-04-24T09:15:37.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:37.984+0000 7f5f1e400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5f18114670 con 0x7f5f181044b0 2024-04-24T09:15:37.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:37.984+0000 7f5f16a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f18109580 0x7f5f18076a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:37.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:37.984+0000 7f5f17e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f18076f90 0x7f5f181ab020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:37.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:37.984+0000 7f5f17e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f18076f90 0x7f5f181ab020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51454/0 (socket says 172.21.15.83:51454) 2024-04-24T09:15:37.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:37.984+0000 7f5f17e00640 1 -- 172.21.15.83:0/117777451 learned_addr learned my addr 172.21.15.83:0/117777451 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:15:37.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:37.984+0000 7f5f17400640 1 --2- 172.21.15.83:0/117777451 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5f181044b0 0x7f5f18076510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:37.986 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:37.984+0000 7fb72f200640 1 -- 172.21.15.83:0/2575700267 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb728104290 msgr2=0x7fb728104670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:37.986 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:37.984+0000 7fb72f200640 1 --2- 172.21.15.83:0/2575700267 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb728104290 0x7fb728104670 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fb71c009900 tx=0x7fb71c02a400 comp rx=0 tx=0).stop 2024-04-24T09:15:37.986 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:37.985+0000 7fb72f200640 1 -- 172.21.15.83:0/2575700267 shutdown_connections 2024-04-24T09:15:37.986 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:37.985+0000 7fb72f200640 1 --2- 172.21.15.83:0/2575700267 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb728109360 0x7fb728110f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:37.986 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:37.985+0000 7fb72f200640 1 --2- 172.21.15.83:0/2575700267 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb728104c40 0x7fb728108c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:37.986 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:37.985+0000 7fb72f200640 1 --2- 172.21.15.83:0/2575700267 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb728104290 0x7fb728104670 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:37.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:37.985+0000 7f5f17e00640 1 -- 172.21.15.83:0/117777451 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5f181044b0 msgr2=0x7f5f18076510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:37.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:37.985+0000 7f5f17e00640 1 --2- 172.21.15.83:0/117777451 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5f181044b0 0x7f5f18076510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:37.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:37.985+0000 7f5f17e00640 1 -- 172.21.15.83:0/117777451 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f18109580 msgr2=0x7f5f18076a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:37.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:37.985+0000 7f5f17e00640 1 --2- 172.21.15.83:0/117777451 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f18109580 0x7f5f18076a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:37.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:37.985+0000 7f5f17e00640 1 -- 172.21.15.83:0/117777451 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f181ab560 con 0x7f5f18076f90 2024-04-24T09:15:37.987 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:37.985+0000 7fb72f200640 1 -- 172.21.15.83:0/2575700267 >> 172.21.15.83:0/2575700267 conn(0x7fb7280ffb10 msgr2=0x7fb728101f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:37.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:37.986+0000 7f5f17e00640 1 --2- 172.21.15.83:0/117777451 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f18076f90 0x7f5f181ab020 secure :-1 s=READY pgs=2718 cs=0 l=1 rev1=1 crypto rx=0x7f5f10031040 tx=0x7f5f10031440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:37.988 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:37.986+0000 7fb72f200640 1 -- 172.21.15.83:0/2575700267 shutdown_connections 2024-04-24T09:15:37.988 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:37.986+0000 7fb72f200640 1 -- 172.21.15.83:0/2575700267 wait complete. 2024-04-24T09:15:37.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:37.986+0000 7f5f0fe00640 1 -- 172.21.15.83:0/117777451 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f10041020 con 0x7f5f18076f90 2024-04-24T09:15:37.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:37.986+0000 7f5f0fe00640 1 -- 172.21.15.83:0/117777451 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5f10031830 con 0x7f5f18076f90 2024-04-24T09:15:37.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:37.986+0000 7f5f0fe00640 1 -- 172.21.15.83:0/117777451 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f10036040 con 0x7f5f18076f90 2024-04-24T09:15:37.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:37.986+0000 7f5f1e400640 1 -- 172.21.15.83:0/117777451 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f181ab7f0 con 0x7f5f18076f90 2024-04-24T09:15:37.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:37.986+0000 7f5f1e400640 1 -- 172.21.15.83:0/117777451 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f181abc60 con 0x7f5f18076f90 2024-04-24T09:15:37.990 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:37.987+0000 7fb72f200640 1 Processor -- start 2024-04-24T09:15:37.990 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:37.988+0000 7fb72f200640 1 -- start start 2024-04-24T09:15:37.990 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:37.988+0000 7fb72f200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb728104290 0x7fb72810fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:37.990 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:37.988+0000 7fb72f200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb728109360 0x7fb72810ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:37.990 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:37.988+0000 7fb72f200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb72806fa60 0x7fb72806fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:37.990 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:37.989+0000 7fb72f200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb728077610 con 0x7fb728104290 2024-04-24T09:15:37.990 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:37.989+0000 7fb72f200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb728077490 con 0x7fb72806fa60 2024-04-24T09:15:37.990 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:37.989+0000 7fb72f200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb728077790 con 0x7fb728109360 2024-04-24T09:15:37.991 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:37.989+0000 7fb72de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb728104290 0x7fb72810fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:37.991 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:37.989+0000 7fb72de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb728104290 0x7fb72810fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51462/0 (socket says 172.21.15.83:51462) 2024-04-24T09:15:37.991 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:37.989+0000 7fb72de00640 1 -- 172.21.15.83:0/3940913502 learned_addr learned my addr 172.21.15.83:0/3940913502 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:15:37.991 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:37.989+0000 7fb72e800640 1 --2- 172.21.15.83:0/3940913502 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb72806fa60 0x7fb72806fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:37.991 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:37.989+0000 7fb72d400640 1 --2- 172.21.15.83:0/3940913502 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb728109360 0x7fb72810ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:37.991 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:37.989+0000 7fb72de00640 1 -- 172.21.15.83:0/3940913502 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb728109360 msgr2=0x7fb72810ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:37.991 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:37.989+0000 7fb72de00640 1 --2- 172.21.15.83:0/3940913502 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb728109360 0x7fb72810ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:37.991 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:37.989+0000 7fb72de00640 1 -- 172.21.15.83:0/3940913502 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb72806fa60 msgr2=0x7fb72806fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:37.991 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:37.989+0000 7fb72de00640 1 --2- 172.21.15.83:0/3940913502 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb72806fa60 0x7fb72806fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:37.991 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:37.989+0000 7fb72de00640 1 -- 172.21.15.83:0/3940913502 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb728070520 con 0x7fb728104290 2024-04-24T09:15:37.991 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:37.989+0000 7fb72e800640 1 --2- 172.21.15.83:0/3940913502 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb72806fa60 0x7fb72806fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:15:37.991 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:37.990+0000 7fb72de00640 1 --2- 172.21.15.83:0/3940913502 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb728104290 0x7fb72810fa30 secure :-1 s=READY pgs=2719 cs=0 l=1 rev1=1 crypto rx=0x7fb71c005ab0 tx=0x7fb71c003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:37.992 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:37.990+0000 7fb722a00640 1 -- 172.21.15.83:0/3940913502 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb71c03f070 con 0x7fb728104290 2024-04-24T09:15:37.992 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:37.990+0000 7fb722a00640 1 -- 172.21.15.83:0/3940913502 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb71c031b50 con 0x7fb728104290 2024-04-24T09:15:37.992 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:37.990+0000 7fb722a00640 1 -- 172.21.15.83:0/3940913502 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb71c035030 con 0x7fb728104290 2024-04-24T09:15:37.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:37.991+0000 7f5f21275740 1 Processor -- start 2024-04-24T09:15:37.992 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:37.991+0000 7fb72f200640 1 -- 172.21.15.83:0/3940913502 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7280707b0 con 0x7fb728104290 2024-04-24T09:15:37.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:37.991+0000 7f5f0fe00640 1 -- 172.21.15.83:0/117777451 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5f10044020 con 0x7f5f18076f90 2024-04-24T09:15:37.993 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:37.991+0000 7fb72f200640 1 -- 172.21.15.83:0/3940913502 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb728072d10 con 0x7fb728104290 2024-04-24T09:15:37.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:37.991+0000 7f5f21275740 1 -- start start 2024-04-24T09:15:37.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:37.992+0000 7f5f21275740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559f24a9f860 0x559f24a9fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:37.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:37.992+0000 7f5f21275740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f24aa0180 0x559f24aaa310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:37.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:37.992+0000 7f5f21275740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f24aaa850 0x559f24aacc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:37.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:37.992+0000 7f5f21275740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559f24a051f0 con 0x559f24aa0180 2024-04-24T09:15:37.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:37.992+0000 7f5f21275740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559f249ab350 con 0x559f24aaa850 2024-04-24T09:15:37.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:37.992+0000 7f5f21275740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559f24a011b0 con 0x559f24a9f860 2024-04-24T09:15:37.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:37.992+0000 7f5f16a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f24aaa850 0x559f24aacc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:37.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:37.993+0000 7f5f16a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f24aaa850 0x559f24aacc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:58728/0 (socket says 172.21.15.83:58728) 2024-04-24T09:15:37.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:37.993+0000 7f5f16a00640 1 -- 172.21.15.83:0/1867729605 learned_addr learned my addr 172.21.15.83:0/1867729605 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:15:37.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:37.993+0000 7f5f16a00640 1 --2- 172.21.15.83:0/1867729605 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559f24a9f860 0x559f24a9fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:37.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:37.993+0000 7f5f17400640 1 --2- 172.21.15.83:0/1867729605 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f24aa0180 0x559f24aaa310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:37.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:37.993+0000 7f5f16a00640 1 -- 172.21.15.83:0/1867729605 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559f24a9f860 msgr2=0x559f24a9fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:37.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:37.993+0000 7f5f16a00640 1 --2- 172.21.15.83:0/1867729605 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559f24a9f860 0x559f24a9fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:37.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:37.993+0000 7f5f16a00640 1 -- 172.21.15.83:0/1867729605 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f24aa0180 msgr2=0x559f24aaa310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:37.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:37.993+0000 7f5f16a00640 1 --2- 172.21.15.83:0/1867729605 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f24aa0180 0x559f24aaa310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:37.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:37.993+0000 7f5f16a00640 1 -- 172.21.15.83:0/1867729605 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f249fff50 con 0x559f24aaa850 2024-04-24T09:15:37.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:37.993+0000 7f5f17400640 1 --2- 172.21.15.83:0/1867729605 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f24aa0180 0x559f24aaa310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:15:37.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:37.993+0000 7f5f0fe00640 1 --2- 172.21.15.83:0/117777451 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f5efc0a3da0 0x7f5efc0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:37.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:37.994+0000 7f5f16a00640 1 --2- 172.21.15.83:0/1867729605 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f24aaa850 0x559f24aacc40 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f5f04013780 tx=0x7f5f04009620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=760dbb7c26ea672e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:37.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:37.994+0000 7f5f0fe00640 1 -- 172.21.15.83:0/117777451 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 9583+0+0 (secure 0 0 0) 0x7f5f100f1c10 con 0x7f5f18076f90 2024-04-24T09:15:37.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:37.994+0000 7f5f17400640 1 --2- 172.21.15.83:0/117777451 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f5efc0a3da0 0x7f5efc0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:37.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:37.994+0000 7f5f0cc00640 1 -- 172.21.15.83:0/1867729605 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f04013dc0 con 0x559f24aaa850 2024-04-24T09:15:37.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:37.994+0000 7f5f0cc00640 1 -- 172.21.15.83:0/1867729605 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5f04005020 con 0x559f24aaa850 2024-04-24T09:15:37.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:37.994+0000 7f5f0cc00640 1 -- 172.21.15.83:0/1867729605 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f040047a0 con 0x559f24aaa850 2024-04-24T09:15:37.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:37.995+0000 7f5f21275740 1 -- 172.21.15.83:0/1867729605 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f24aaa850 msgr2=0x559f24aacc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:37.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:37.995+0000 7f5f21275740 1 --2- 172.21.15.83:0/1867729605 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f24aaa850 0x559f24aacc40 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f5f04013780 tx=0x7f5f04009620 comp rx=0 tx=0).stop 2024-04-24T09:15:37.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:37.995+0000 7f5f17400640 1 --2- 172.21.15.83:0/117777451 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f5efc0a3da0 0x7f5efc0a6260 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x559f24aa0e20 tx=0x7f5f08005f90 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:37.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:37.995+0000 7f5f21275740 1 -- 172.21.15.83:0/1867729605 shutdown_connections 2024-04-24T09:15:37.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:37.995+0000 7f5f21275740 1 --2- 172.21.15.83:0/1867729605 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f24aaa850 0x559f24aacc40 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:37.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:37.995+0000 7f5f21275740 1 --2- 172.21.15.83:0/1867729605 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f24aa0180 0x559f24aaa310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:37.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:37.995+0000 7f5f21275740 1 --2- 172.21.15.83:0/1867729605 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559f24a9f860 0x559f24a9fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:37.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:37.995+0000 7f5f21275740 1 -- 172.21.15.83:0/1867729605 >> 172.21.15.83:0/1867729605 conn(0x559f24a9bb80 msgr2=0x559f24a9bf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:37.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:37.995+0000 7f5f21275740 1 -- 172.21.15.83:0/1867729605 shutdown_connections 2024-04-24T09:15:37.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:37.996+0000 7f5f21275740 1 -- 172.21.15.83:0/1867729605 wait complete. 2024-04-24T09:15:37.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:37.996+0000 7f5f21275740 1 -- start start 2024-04-24T09:15:37.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:37.997+0000 7f5f21275740 1 Processor -- start 2024-04-24T09:15:37.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:37.997+0000 7f5f21275740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f24a9f860 0x559f24aa7870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:37.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:37.997+0000 7f5f21275740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559f24aa0180 0x559f24aa7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:37.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:37.997+0000 7f5f21275740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f24aaa850 0x559f24aa82f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:37.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:37.997+0000 7f5f21275740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559f24aad640 con 0x559f24a9f860 2024-04-24T09:15:37.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:37.997+0000 7f5f21275740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559f249be690 con 0x559f24aaa850 2024-04-24T09:15:37.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:37.997+0000 7f5f21275740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559f24aad7c0 con 0x559f24aa0180 2024-04-24T09:15:37.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:37.997+0000 7f5f21275740 10 client.0 _ll_register_callbacks cb 0x559f249fa650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:15:37.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:37.998+0000 7f5f17400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f24a9f860 0x559f24aa7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:37.999 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:37.998+0000 7fb722a00640 1 -- 172.21.15.83:0/3940913502 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb71c042070 con 0x7fb728104290 2024-04-24T09:15:38.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:37.998+0000 7f5f17400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f24a9f860 0x559f24aa7870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51472/0 (socket says 172.21.15.83:51472) 2024-04-24T09:15:38.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:37.998+0000 7f5f17400640 1 -- 172.21.15.83:0/3538126628 learned_addr learned my addr 172.21.15.83:0/3538126628 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:15:38.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:37.998+0000 7f5f17e00640 1 --2- 172.21.15.83:0/3538126628 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f24aaa850 0x559f24aa82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:38.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:37.998+0000 7f5f17400640 1 -- 172.21.15.83:0/3538126628 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559f24aa0180 msgr2=0x559f24aa7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:38.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:37.998+0000 7f5f17400640 1 --2- 172.21.15.83:0/3538126628 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559f24aa0180 0x559f24aa7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:38.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:37.998+0000 7f5f17400640 1 -- 172.21.15.83:0/3538126628 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f24aaa850 msgr2=0x559f24aa82f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:38.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:37.998+0000 7f5f17400640 1 --2- 172.21.15.83:0/3538126628 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f24aaa850 0x559f24aa82f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:38.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:37.998+0000 7f5f17400640 1 -- 172.21.15.83:0/3538126628 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f249bd920 con 0x559f24a9f860 2024-04-24T09:15:38.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:37.998+0000 7f5f17e00640 1 --2- 172.21.15.83:0/3538126628 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f24aaa850 0x559f24aa82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:15:38.001 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:37.999+0000 7fb73356f740 1 Processor -- start 2024-04-24T09:15:38.001 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:37.999+0000 7fb73356f740 1 -- start start 2024-04-24T09:15:38.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:37.999+0000 7f5f17400640 1 --2- 172.21.15.83:0/3538126628 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f24a9f860 0x559f24aa7870 secure :-1 s=READY pgs=2720 cs=0 l=1 rev1=1 crypto rx=0x7f5f0801e9d0 tx=0x7f5f08004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b985550f5b145078 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:38.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:37.999+0000 7f5f01600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:15:38.002 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:37.999+0000 7fb73356f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e39ca813c0 0x55e39ca817a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:38.002 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:37.999+0000 7fb73356f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e39ca81ce0 0x55e39ca8c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:38.002 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:37.999+0000 7fb73356f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e39ca8c5a0 0x55e39ca8e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:38.002 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:37.999+0000 7fb73356f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e39c9e7900 con 0x55e39ca8c5a0 2024-04-24T09:15:38.002 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:37.999+0000 7fb73356f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e39c98df20 con 0x55e39ca81ce0 2024-04-24T09:15:38.002 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:37.999+0000 7fb73356f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55e39c9e38c0 con 0x55e39ca813c0 2024-04-24T09:15:38.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:37.999+0000 7f5f01600640 1 -- 172.21.15.83:0/3538126628 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f08009db0 con 0x559f24a9f860 2024-04-24T09:15:38.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:37.999+0000 7f5f01600640 1 -- 172.21.15.83:0/3538126628 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(130..130 src has 1..130) ==== 9583+0+0 (secure 0 0 0) 0x7f5f08018030 con 0x559f24a9f860 2024-04-24T09:15:38.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.000+0000 7f5f21275740 1 -- 172.21.15.83:0/3538126628 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f04005020 con 0x559f24a9f860 2024-04-24T09:15:38.003 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.000+0000 7fb72e800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e39ca813c0 0x55e39ca817a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:38.003 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.000+0000 7fb72e800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e39ca813c0 0x55e39ca817a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:47252/0 (socket says 172.21.15.83:47252) 2024-04-24T09:15:38.003 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.000+0000 7fb72e800640 1 -- 172.21.15.83:0/3698734228 learned_addr learned my addr 172.21.15.83:0/3698734228 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:15:38.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.000+0000 7f5f01600640 1 -- 172.21.15.83:0/3538126628 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f5f08004020 con 0x559f24a9f860 2024-04-24T09:15:38.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.000+0000 7f5f21275740 10 client.23655 fetch_fsmap learned FSMap version 69 2024-04-24T09:15:38.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.000+0000 7f5f21275740 1 -- 172.21.15.83:0/3538126628 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x559f247e17d0 con 0x559f24a9f860 2024-04-24T09:15:38.005 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.000+0000 7fb722a00640 1 --2- 172.21.15.83:0/3940913502 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb7100a3e60 0x7fb7100a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:38.005 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.000+0000 7fb72e800640 1 --2- 172.21.15.83:0/3940913502 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb7100a3e60 0x7fb7100a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:38.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.000+0000 7f5f01600640 1 -- 172.21.15.83:0/3538126628 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5f04005020 con 0x559f24a9f860 2024-04-24T09:15:38.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.001+0000 7f5f01600640 1 -- 172.21.15.83:0/3538126628 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 69) ==== 8370+0+0 (secure 0 0 0) 0x7f5f08029420 con 0x559f24a9f860 2024-04-24T09:15:38.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.001+0000 7f5f21275740 10 client.23655 fetch_fsmap finished waiting for FSMap version 69 2024-04-24T09:15:38.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.001+0000 7f5f21275740 10 client.23655 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:15:38.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.001+0000 7f5f21275740 20 client.23655 populate_metadata read hostname 'smithi083' 2024-04-24T09:15:38.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.001+0000 7f5f21275740 1 --2- 172.21.15.83:0/3538126628 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x559f24acbd20 0x559f24ace1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:38.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.001+0000 7f5f21275740 4 client.23655 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:15:38.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.001+0000 7f5f21275740 1 -- 172.21.15.83:0/3538126628 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559f24a12c10 con 0x559f24acbd20 2024-04-24T09:15:38.006 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.001+0000 7fb72d400640 1 --2- 172.21.15.83:0/3698734228 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e39ca8c5a0 0x55e39ca8e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:38.006 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.001+0000 7fb722a00640 1 -- 172.21.15.83:0/3940913502 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 9583+0+0 (secure 0 0 0) 0x7fb71c0f1000 con 0x7fb728104290 2024-04-24T09:15:38.006 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.001+0000 7fb72d400640 1 -- 172.21.15.83:0/3698734228 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e39ca813c0 msgr2=0x55e39ca817a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:38.006 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.001+0000 7fb72d400640 1 --2- 172.21.15.83:0/3698734228 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e39ca813c0 0x55e39ca817a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:38.006 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.001+0000 7fb72d400640 1 -- 172.21.15.83:0/3698734228 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e39ca81ce0 msgr2=0x55e39ca8c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:38.006 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.001+0000 7fb72d400640 1 --2- 172.21.15.83:0/3698734228 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e39ca81ce0 0x55e39ca8c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:38.006 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.001+0000 7fb72d400640 1 -- 172.21.15.83:0/3698734228 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e39c9b2e10 con 0x55e39ca8c5a0 2024-04-24T09:15:38.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.002+0000 7f5f16a00640 1 --2- 172.21.15.83:0/3538126628 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x559f24acbd20 0x559f24ace1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:38.007 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.002+0000 7fb72d400640 1 --2- 172.21.15.83:0/3698734228 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e39ca8c5a0 0x55e39ca8e990 secure :-1 s=READY pgs=2721 cs=0 l=1 rev1=1 crypto rx=0x7fb72806b4c0 tx=0x7fb71800e350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7f39f83370ede55 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:38.007 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.002+0000 7fb72e800640 1 --2- 172.21.15.83:0/3940913502 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb7100a3e60 0x7fb7100a6320 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fb724013fd0 tx=0x7fb724013460 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:38.007 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.002+0000 7fb717000640 1 -- 172.21.15.83:0/3698734228 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb718004020 con 0x55e39ca8c5a0 2024-04-24T09:15:38.008 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.003+0000 7fb717000640 1 -- 172.21.15.83:0/3698734228 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb7180054d0 con 0x55e39ca8c5a0 2024-04-24T09:15:38.008 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.003+0000 7fb717000640 1 -- 172.21.15.83:0/3698734228 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb718004c40 con 0x55e39ca8c5a0 2024-04-24T09:15:38.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.003+0000 7f5f01600640 10 client.23655 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:15:38.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.003+0000 7f5f16a00640 1 --2- 172.21.15.83:0/3538126628 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x559f24acbd20 0x559f24ace1c0 crc :-1 s=READY pgs=585 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc3e1fdff4484927 server_cookie=857ad8117215be05 in_seq=0 out_seq=0 2024-04-24T09:15:38.009 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.004+0000 7fb73356f740 1 -- 172.21.15.83:0/3698734228 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e39ca8c5a0 msgr2=0x55e39ca8e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:38.009 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.004+0000 7fb73356f740 1 --2- 172.21.15.83:0/3698734228 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e39ca8c5a0 0x55e39ca8e990 secure :-1 s=READY pgs=2721 cs=0 l=1 rev1=1 crypto rx=0x7fb72806b4c0 tx=0x7fb71800e350 comp rx=0 tx=0).stop 2024-04-24T09:15:38.009 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.004+0000 7fb73356f740 1 -- 172.21.15.83:0/3698734228 shutdown_connections 2024-04-24T09:15:38.009 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.004+0000 7fb73356f740 1 --2- 172.21.15.83:0/3698734228 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e39ca8c5a0 0x55e39ca8e990 unknown :-1 s=CLOSED pgs=2721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:38.009 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.004+0000 7fb73356f740 1 --2- 172.21.15.83:0/3698734228 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e39ca81ce0 0x55e39ca8c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:38.009 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.004+0000 7fb73356f740 1 --2- 172.21.15.83:0/3698734228 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e39ca813c0 0x55e39ca817a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:38.009 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.004+0000 7fb73356f740 1 -- 172.21.15.83:0/3698734228 >> 172.21.15.83:0/3698734228 conn(0x55e39ca7d8d0 msgr2=0x55e39ca7dcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:38.009 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.004+0000 7fb73356f740 1 -- 172.21.15.83:0/3698734228 shutdown_connections 2024-04-24T09:15:38.009 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.004+0000 7fb73356f740 1 -- 172.21.15.83:0/3698734228 wait complete. 2024-04-24T09:15:38.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.005+0000 7f5f01600640 1 -- 172.21.15.83:0/3538126628 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559f24a12c10 con 0x559f24acbd20 2024-04-24T09:15:38.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.005+0000 7f5f01600640 10 client.23655 handle_command_reply: tid=0 2024-04-24T09:15:38.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.005+0000 7f5f21275740 1 client.23655 shutdown 2024-04-24T09:15:38.010 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.005+0000 7fb73356f740 1 -- start start 2024-04-24T09:15:38.010 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.005+0000 7fb73356f740 1 Processor -- start 2024-04-24T09:15:38.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.005+0000 7f5f21275740 1 -- 172.21.15.83:0/3538126628 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f24a9f860 msgr2=0x559f24aa7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:38.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.005+0000 7f5f21275740 1 --2- 172.21.15.83:0/3538126628 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f24a9f860 0x559f24aa7870 secure :-1 s=READY pgs=2720 cs=0 l=1 rev1=1 crypto rx=0x7f5f0801e9d0 tx=0x7f5f08004730 comp rx=0 tx=0).stop 2024-04-24T09:15:38.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.005+0000 7f5f21275740 1 -- 172.21.15.83:0/3538126628 shutdown_connections 2024-04-24T09:15:38.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.005+0000 7f5f21275740 1 --2- 172.21.15.83:0/3538126628 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x559f24acbd20 0x559f24ace1c0 crc :-1 s=READY pgs=585 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:38.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.005+0000 7f5f21275740 1 --2- 172.21.15.83:0/3538126628 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f24aaa850 0x559f24aa82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:38.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.005+0000 7f5f21275740 1 --2- 172.21.15.83:0/3538126628 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559f24aa0180 0x559f24aa7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:38.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.005+0000 7f5f21275740 1 --2- 172.21.15.83:0/3538126628 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f24a9f860 0x559f24aa7870 unknown :-1 s=CLOSED pgs=2720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:38.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.005+0000 7f5f21275740 1 -- 172.21.15.83:0/3538126628 >> 172.21.15.83:0/3538126628 conn(0x559f24a9bb80 msgr2=0x559f24aa1980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:38.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.005+0000 7f5f01600640 0 client.23655 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:15:38.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.006+0000 7f5f21275740 1 -- 172.21.15.83:0/3538126628 shutdown_connections 2024-04-24T09:15:38.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.006+0000 7f5f21275740 1 -- 172.21.15.83:0/3538126628 wait complete. 2024-04-24T09:15:38.012 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.006+0000 7fb73356f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e39ca813c0 0x55e39ca896b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:38.012 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.006+0000 7fb73356f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e39ca81ce0 0x55e39ca89bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:38.012 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.006+0000 7fb73356f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e39ca8c5a0 0x55e39ca8a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:38.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.006+0000 7f5f21275740 20 client.23655 trim_cache size 0 max 16384 2024-04-24T09:15:38.015 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.006+0000 7fb73356f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e39ca8f140 con 0x55e39ca813c0 2024-04-24T09:15:38.016 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.006+0000 7fb73356f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e39c9a12c0 con 0x55e39ca8c5a0 2024-04-24T09:15:38.016 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.006+0000 7fb73356f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55e39ca8f2c0 con 0x55e39ca81ce0 2024-04-24T09:15:38.016 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.006+0000 7fb73356f740 10 client.0 _ll_register_callbacks cb 0x55e39c6f9a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:15:38.016 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.006+0000 7fb72de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e39ca813c0 0x55e39ca896b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:38.016 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.006+0000 7fb72de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e39ca813c0 0x55e39ca896b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51492/0 (socket says 172.21.15.83:51492) 2024-04-24T09:15:38.016 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.006+0000 7fb72de00640 1 -- 172.21.15.83:0/2231914555 learned_addr learned my addr 172.21.15.83:0/2231914555 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:15:38.016 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.007+0000 7fb72e800640 1 --2- 172.21.15.83:0/2231914555 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e39ca8c5a0 0x55e39ca8a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:38.016 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.007+0000 7fb72de00640 1 -- 172.21.15.83:0/2231914555 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e39ca81ce0 msgr2=0x55e39ca89bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:38.016 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.007+0000 7fb72d400640 1 --2- 172.21.15.83:0/2231914555 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e39ca81ce0 0x55e39ca89bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:38.016 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.007+0000 7fb72de00640 1 --2- 172.21.15.83:0/2231914555 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e39ca81ce0 0x55e39ca89bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:38.016 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.007+0000 7fb72de00640 1 -- 172.21.15.83:0/2231914555 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e39ca8c5a0 msgr2=0x55e39ca8a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:38.016 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.007+0000 7fb72de00640 1 --2- 172.21.15.83:0/2231914555 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e39ca8c5a0 0x55e39ca8a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:38.017 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.007+0000 7fb72de00640 1 -- 172.21.15.83:0/2231914555 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e39c9b1fd0 con 0x55e39ca813c0 2024-04-24T09:15:38.017 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.007+0000 7fb72e800640 1 --2- 172.21.15.83:0/2231914555 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e39ca8c5a0 0x55e39ca8a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:15:38.017 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.007+0000 7fb72d400640 1 --2- 172.21.15.83:0/2231914555 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e39ca81ce0 0x55e39ca89bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:15:38.017 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.007+0000 7fb72de00640 1 --2- 172.21.15.83:0/2231914555 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e39ca813c0 0x55e39ca896b0 secure :-1 s=READY pgs=2722 cs=0 l=1 rev1=1 crypto rx=0x7fb71c004650 tx=0x7fb71c0bc540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=151a2dceff028222 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:38.017 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.008+0000 7fb70b400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:15:38.017 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.008+0000 7fb70b400640 1 -- 172.21.15.83:0/2231914555 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb71c049050 con 0x55e39ca813c0 2024-04-24T09:15:38.017 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.008+0000 7fb70b400640 1 -- 172.21.15.83:0/2231914555 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(130..130 src has 1..130) ==== 9583+0+0 (secure 0 0 0) 0x7fb71c0b9190 con 0x55e39ca813c0 2024-04-24T09:15:38.017 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.008+0000 7fb73356f740 1 -- 172.21.15.83:0/2231914555 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7180054d0 con 0x55e39ca813c0 2024-04-24T09:15:38.017 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.008+0000 7fb70b400640 1 -- 172.21.15.83:0/2231914555 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7fb71c0c0ed0 con 0x55e39ca813c0 2024-04-24T09:15:38.017 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.008+0000 7fb73356f740 10 client.23661 fetch_fsmap learned FSMap version 69 2024-04-24T09:15:38.017 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.008+0000 7fb73356f740 1 -- 172.21.15.83:0/2231914555 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x55e39c6fa210 con 0x55e39ca813c0 2024-04-24T09:15:38.017 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.009+0000 7fb70b400640 1 -- 172.21.15.83:0/2231914555 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb7180054d0 con 0x55e39ca813c0 2024-04-24T09:15:38.017 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.009+0000 7fb70b400640 1 -- 172.21.15.83:0/2231914555 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 69) ==== 8370+0+0 (secure 0 0 0) 0x7fb71c0d8050 con 0x55e39ca813c0 2024-04-24T09:15:38.017 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.009+0000 7fb73356f740 10 client.23661 fetch_fsmap finished waiting for FSMap version 69 2024-04-24T09:15:38.017 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.009+0000 7fb73356f740 10 client.23661 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:15:38.018 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.009+0000 7fb73356f740 20 client.23661 populate_metadata read hostname 'smithi083' 2024-04-24T09:15:38.018 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.009+0000 7fb73356f740 1 --2- 172.21.15.83:0/2231914555 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55e39ca9c750 0x55e39ca9eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:38.018 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.009+0000 7fb73356f740 4 client.23661 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:15:38.018 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.009+0000 7fb73356f740 1 -- 172.21.15.83:0/2231914555 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e39c9f5320 con 0x55e39ca9c750 2024-04-24T09:15:38.018 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.010+0000 7fb72d400640 1 --2- 172.21.15.83:0/2231914555 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55e39ca9c750 0x55e39ca9eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:38.018 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.010+0000 7fb72d400640 1 --2- 172.21.15.83:0/2231914555 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55e39ca9c750 0x55e39ca9eb80 crc :-1 s=READY pgs=586 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f519a627f9744d1c server_cookie=2a2df1bcb3fc048d in_seq=0 out_seq=0 2024-04-24T09:15:38.018 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.010+0000 7fb70b400640 10 client.23661 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:15:38.018 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.011+0000 7fb70b400640 1 -- 172.21.15.83:0/2231914555 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e39c9f5320 con 0x55e39ca9c750 2024-04-24T09:15:38.018 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.011+0000 7fb70b400640 10 client.23661 handle_command_reply: tid=0 2024-04-24T09:15:38.018 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.011+0000 7fb73356f740 1 client.23661 shutdown 2024-04-24T09:15:38.018 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.012+0000 7fb73356f740 1 -- 172.21.15.83:0/2231914555 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e39ca813c0 msgr2=0x55e39ca896b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:38.018 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.012+0000 7fb73356f740 1 --2- 172.21.15.83:0/2231914555 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e39ca813c0 0x55e39ca896b0 secure :-1 s=READY pgs=2722 cs=0 l=1 rev1=1 crypto rx=0x7fb71c004650 tx=0x7fb71c0bc540 comp rx=0 tx=0).stop 2024-04-24T09:15:38.018 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.012+0000 7fb73356f740 1 -- 172.21.15.83:0/2231914555 shutdown_connections 2024-04-24T09:15:38.022 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.012+0000 7fb73356f740 1 --2- 172.21.15.83:0/2231914555 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55e39ca9c750 0x55e39ca9eb80 crc :-1 s=READY pgs=586 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:38.022 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.012+0000 7fb73356f740 1 --2- 172.21.15.83:0/2231914555 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e39ca8c5a0 0x55e39ca8a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:38.022 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.012+0000 7fb73356f740 1 --2- 172.21.15.83:0/2231914555 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e39ca81ce0 0x55e39ca89bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:38.022 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.012+0000 7fb73356f740 1 --2- 172.21.15.83:0/2231914555 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e39ca813c0 0x55e39ca896b0 unknown :-1 s=CLOSED pgs=2722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:38.022 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.012+0000 7fb70b400640 0 client.23661 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:15:38.022 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.012+0000 7fb73356f740 1 -- 172.21.15.83:0/2231914555 >> 172.21.15.83:0/2231914555 conn(0x55e39ca7d8d0 msgr2=0x55e39ca83600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:38.022 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.012+0000 7fb73356f740 1 -- 172.21.15.83:0/2231914555 shutdown_connections 2024-04-24T09:15:38.022 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.012+0000 7fb73356f740 1 -- 172.21.15.83:0/2231914555 wait complete. 2024-04-24T09:15:38.022 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.012+0000 7fb73356f740 20 client.23661 trim_cache size 0 max 16384 2024-04-24T09:15:38.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.045+0000 7f5f21275740 1 Processor -- start 2024-04-24T09:15:38.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.045+0000 7f5f21275740 1 -- start start 2024-04-24T09:15:38.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.046+0000 7f5f21275740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559f24a9f860 0x559f24ab6500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:38.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.046+0000 7f5f21275740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f24aa0180 0x559f24aa67b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:38.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.046+0000 7f5f21275740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f24aaa850 0x559f24ad3a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:38.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.046+0000 7f5f21275740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5f181ad420 con 0x559f24aa0180 2024-04-24T09:15:38.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.046+0000 7f5f21275740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5f181aefc0 con 0x559f24aaa850 2024-04-24T09:15:38.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.046+0000 7f5f21275740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5f181adaf0 con 0x559f24a9f860 2024-04-24T09:15:38.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.046+0000 7f5f16a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f24aa0180 0x559f24aa67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:38.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.046+0000 7f5f16a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f24aa0180 0x559f24aa67b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51508/0 (socket says 172.21.15.83:51508) 2024-04-24T09:15:38.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.046+0000 7f5f16a00640 1 -- 172.21.15.83:0/393278437 learned_addr learned my addr 172.21.15.83:0/393278437 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:15:38.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.046+0000 7f5f17e00640 1 --2- 172.21.15.83:0/393278437 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f24aaa850 0x559f24ad3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:38.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.047+0000 7f5f17400640 1 --2- 172.21.15.83:0/393278437 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559f24a9f860 0x559f24ab6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:38.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.047+0000 7f5f16a00640 1 -- 172.21.15.83:0/393278437 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559f24a9f860 msgr2=0x559f24ab6500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:38.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.047+0000 7f5f16a00640 1 --2- 172.21.15.83:0/393278437 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559f24a9f860 0x559f24ab6500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:38.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.047+0000 7f5f16a00640 1 -- 172.21.15.83:0/393278437 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f24aaa850 msgr2=0x559f24ad3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:38.048 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.047+0000 7fb73356f740 1 Processor -- start 2024-04-24T09:15:38.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.047+0000 7f5f16a00640 1 --2- 172.21.15.83:0/393278437 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f24aaa850 0x559f24ad3a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:38.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.047+0000 7f5f16a00640 1 -- 172.21.15.83:0/393278437 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f247173f0 con 0x559f24aa0180 2024-04-24T09:15:38.049 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.047+0000 7fb73356f740 1 -- start start 2024-04-24T09:15:38.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.047+0000 7f5f16a00640 1 --2- 172.21.15.83:0/393278437 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f24aa0180 0x559f24aa67b0 secure :-1 s=READY pgs=2723 cs=0 l=1 rev1=1 crypto rx=0x7f5f04009690 tx=0x7f5f04009970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3494f3856bd94ed server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:38.049 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.047+0000 7fb73356f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e39ca813c0 0x55e39ca980c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:38.050 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.047+0000 7fb73356f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e39ca81ce0 0x55e39ca83d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:38.050 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.047+0000 7fb73356f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e39ca8c5a0 0x55e39ca89d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:38.050 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.047+0000 7fb73356f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb7281be210 con 0x55e39ca81ce0 2024-04-24T09:15:38.050 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.047+0000 7fb73356f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb7281bfdb0 con 0x55e39ca8c5a0 2024-04-24T09:15:38.050 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.047+0000 7fb73356f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb7281be8e0 con 0x55e39ca813c0 2024-04-24T09:15:38.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.048+0000 7f5f03e00640 1 -- 172.21.15.83:0/393278437 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f04032070 con 0x559f24aa0180 2024-04-24T09:15:38.050 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.048+0000 7fb72d400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e39ca81ce0 0x55e39ca83d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:38.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.048+0000 7f5f03e00640 1 -- 172.21.15.83:0/393278437 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5f040054e0 con 0x559f24aa0180 2024-04-24T09:15:38.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.048+0000 7f5f03e00640 1 -- 172.21.15.83:0/393278437 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f040038e0 con 0x559f24aa0180 2024-04-24T09:15:38.052 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.048+0000 7fb72d400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e39ca81ce0 0x55e39ca83d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51510/0 (socket says 172.21.15.83:51510) 2024-04-24T09:15:38.052 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.048+0000 7fb72de00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e39ca813c0 0x55e39ca980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:38.052 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.048+0000 7fb72d400640 1 -- 172.21.15.83:0/281107041 learned_addr learned my addr 172.21.15.83:0/281107041 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:15:38.052 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.048+0000 7fb72e800640 1 --2- 172.21.15.83:0/281107041 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e39ca8c5a0 0x55e39ca89d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:38.052 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.048+0000 7fb72d400640 1 -- 172.21.15.83:0/281107041 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e39ca813c0 msgr2=0x55e39ca980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:38.052 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.048+0000 7fb72d400640 1 --2- 172.21.15.83:0/281107041 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e39ca813c0 0x55e39ca980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:38.052 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.048+0000 7fb72d400640 1 -- 172.21.15.83:0/281107041 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e39ca8c5a0 msgr2=0x55e39ca89d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:38.052 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.048+0000 7fb72d400640 1 --2- 172.21.15.83:0/281107041 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e39ca8c5a0 0x55e39ca89d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:38.052 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.049+0000 7fb72d400640 1 -- 172.21.15.83:0/281107041 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e39ca8ed00 con 0x55e39ca81ce0 2024-04-24T09:15:38.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.049+0000 7f5f21275740 1 -- 172.21.15.83:0/393278437 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f24aa0180 msgr2=0x559f24aa67b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:38.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.049+0000 7f5f21275740 1 --2- 172.21.15.83:0/393278437 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f24aa0180 0x559f24aa67b0 secure :-1 s=READY pgs=2723 cs=0 l=1 rev1=1 crypto rx=0x7f5f04009690 tx=0x7f5f04009970 comp rx=0 tx=0).stop 2024-04-24T09:15:38.053 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.049+0000 7fb72d400640 1 --2- 172.21.15.83:0/281107041 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e39ca81ce0 0x55e39ca83d80 secure :-1 s=READY pgs=2724 cs=0 l=1 rev1=1 crypto rx=0x7fb7180057c0 tx=0x7fb71800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac61c94cde495545 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:38.053 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.049+0000 7fb716600640 1 -- 172.21.15.83:0/281107041 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb718004220 con 0x55e39ca81ce0 2024-04-24T09:15:38.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.049+0000 7f5f21275740 1 -- 172.21.15.83:0/393278437 shutdown_connections 2024-04-24T09:15:38.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.049+0000 7f5f21275740 1 --2- 172.21.15.83:0/393278437 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f24aaa850 0x559f24ad3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:38.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.049+0000 7f5f21275740 1 --2- 172.21.15.83:0/393278437 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f24aa0180 0x559f24aa67b0 unknown :-1 s=CLOSED pgs=2723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:38.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.049+0000 7f5f21275740 1 --2- 172.21.15.83:0/393278437 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559f24a9f860 0x559f24ab6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:38.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.049+0000 7f5f21275740 1 -- 172.21.15.83:0/393278437 >> 172.21.15.83:0/393278437 conn(0x559f24a9bb80 msgr2=0x559f24aa80c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:38.054 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.050+0000 7fb716600640 1 -- 172.21.15.83:0/281107041 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb7180043c0 con 0x55e39ca81ce0 2024-04-24T09:15:38.054 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.050+0000 7fb716600640 1 -- 172.21.15.83:0/281107041 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb718004220 con 0x55e39ca81ce0 2024-04-24T09:15:38.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.050+0000 7f5f21275740 1 -- 172.21.15.83:0/393278437 shutdown_connections 2024-04-24T09:15:38.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.050+0000 7f5f21275740 1 -- 172.21.15.83:0/393278437 wait complete. 2024-04-24T09:15:38.055 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.050+0000 7fb73356f740 1 -- 172.21.15.83:0/281107041 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e39ca81ce0 msgr2=0x55e39ca83d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:38.055 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.050+0000 7fb73356f740 1 --2- 172.21.15.83:0/281107041 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e39ca81ce0 0x55e39ca83d80 secure :-1 s=READY pgs=2724 cs=0 l=1 rev1=1 crypto rx=0x7fb7180057c0 tx=0x7fb71800ef20 comp rx=0 tx=0).stop 2024-04-24T09:15:38.055 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.051+0000 7fb73356f740 1 -- 172.21.15.83:0/281107041 shutdown_connections 2024-04-24T09:15:38.055 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.051+0000 7fb73356f740 1 --2- 172.21.15.83:0/281107041 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e39ca8c5a0 0x55e39ca89d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:38.056 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.051+0000 7fb73356f740 1 --2- 172.21.15.83:0/281107041 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e39ca81ce0 0x55e39ca83d80 unknown :-1 s=CLOSED pgs=2724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:38.056 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.051+0000 7fb73356f740 1 --2- 172.21.15.83:0/281107041 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e39ca813c0 0x55e39ca980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:38.056 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.051+0000 7fb73356f740 1 -- 172.21.15.83:0/281107041 >> 172.21.15.83:0/281107041 conn(0x55e39ca7d8d0 msgr2=0x55e39ca89670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:38.056 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.051+0000 7fb73356f740 1 -- 172.21.15.83:0/281107041 shutdown_connections 2024-04-24T09:15:38.056 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.051+0000 7fb73356f740 1 -- 172.21.15.83:0/281107041 wait complete. 2024-04-24T09:15:38.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.050+0000 7f5f21275740 1 -- start start 2024-04-24T09:15:38.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.051+0000 7f5f21275740 1 Processor -- start 2024-04-24T09:15:38.056 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.051+0000 7fb73356f740 1 -- start start 2024-04-24T09:15:38.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.051+0000 7f5f21275740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f24a9f860 0x559f24aae370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:38.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.051+0000 7f5f21275740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559f24aa0180 0x559f24aae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:38.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.051+0000 7f5f21275740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f24aaa850 0x559f24aaedf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:38.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.051+0000 7f5f21275740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5f181ae8f0 con 0x559f24a9f860 2024-04-24T09:15:38.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.051+0000 7f5f21275740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5f181ae220 con 0x559f24aaa850 2024-04-24T09:15:38.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.051+0000 7f5f21275740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5f181a24a0 con 0x559f24aa0180 2024-04-24T09:15:38.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.051+0000 7f5f21275740 10 client.0 _ll_register_callbacks cb 0x7f5f08009660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:15:38.058 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.051+0000 7fb73356f740 1 Processor -- start 2024-04-24T09:15:38.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.052+0000 7f5f17400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f24a9f860 0x559f24aae370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:38.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.052+0000 7f5f17400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f24a9f860 0x559f24aae370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51524/0 (socket says 172.21.15.83:51524) 2024-04-24T09:15:38.058 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.052+0000 7fb73356f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e39ca813c0 0x55e39ca94a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:38.058 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.052+0000 7fb73356f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e39ca81ce0 0x55e39ca8fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:38.058 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.052+0000 7fb73356f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e39ca8c5a0 0x55e39ca90320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:38.059 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.052+0000 7fb73356f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb7281bf6e0 con 0x55e39ca81ce0 2024-04-24T09:15:38.059 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.052+0000 7fb73356f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb7281bf010 con 0x55e39ca8c5a0 2024-04-24T09:15:38.059 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.052+0000 7fb73356f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb7281bc210 con 0x55e39ca813c0 2024-04-24T09:15:38.059 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.052+0000 7fb73356f740 10 client.0 _ll_register_callbacks cb 0x7fb71c005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:15:38.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.052+0000 7f5f16a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559f24aa0180 0x559f24aae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:38.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.052+0000 7f5f17400640 1 -- 172.21.15.83:0/2746362924 learned_addr learned my addr 172.21.15.83:0/2746362924 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:15:38.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.052+0000 7f5f17e00640 1 --2- 172.21.15.83:0/2746362924 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f24aaa850 0x559f24aaedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:38.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.053+0000 7f5f17400640 1 -- 172.21.15.83:0/2746362924 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559f24aa0180 msgr2=0x559f24aae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:38.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.053+0000 7f5f17400640 1 --2- 172.21.15.83:0/2746362924 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559f24aa0180 0x559f24aae8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:38.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.053+0000 7f5f17400640 1 -- 172.21.15.83:0/2746362924 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f24aaa850 msgr2=0x559f24aaedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:38.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.053+0000 7f5f17400640 1 --2- 172.21.15.83:0/2746362924 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f24aaa850 0x559f24aaedf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:38.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.053+0000 7f5f17400640 1 -- 172.21.15.83:0/2746362924 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f246d70f0 con 0x559f24a9f860 2024-04-24T09:15:38.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.053+0000 7f5f16a00640 1 --2- 172.21.15.83:0/2746362924 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559f24aa0180 0x559f24aae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:15:38.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.054+0000 7f5f17400640 1 --2- 172.21.15.83:0/2746362924 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f24a9f860 0x559f24aae370 secure :-1 s=READY pgs=2725 cs=0 l=1 rev1=1 crypto rx=0x7f5f08029890 tx=0x7f5f0801e8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83b12c2d3ad41ea2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:38.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.054+0000 7f5f01600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:15:38.060 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.053+0000 7fb72e800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e39ca8c5a0 0x55e39ca90320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:38.061 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.053+0000 7fb72de00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e39ca813c0 0x55e39ca94a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:38.061 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.053+0000 7fb72e800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e39ca8c5a0 0x55e39ca90320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:58778/0 (socket says 172.21.15.83:58778) 2024-04-24T09:15:38.061 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.053+0000 7fb72e800640 1 -- 172.21.15.83:0/1091073750 learned_addr learned my addr 172.21.15.83:0/1091073750 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:15:38.061 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.054+0000 7fb72e800640 1 -- 172.21.15.83:0/1091073750 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e39ca813c0 msgr2=0x55e39ca94a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:38.061 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.054+0000 7fb72e800640 1 --2- 172.21.15.83:0/1091073750 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e39ca813c0 0x55e39ca94a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:38.061 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.054+0000 7fb72e800640 1 -- 172.21.15.83:0/1091073750 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e39ca81ce0 msgr2=0x55e39ca8fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:38.061 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.053+0000 7fb72d400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e39ca81ce0 0x55e39ca8fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:38.061 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.054+0000 7fb72e800640 1 --2- 172.21.15.83:0/1091073750 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e39ca81ce0 0x55e39ca8fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:38.061 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.054+0000 7fb72e800640 1 -- 172.21.15.83:0/1091073750 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e39c7c6620 con 0x55e39ca8c5a0 2024-04-24T09:15:38.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.054+0000 7f5f01600640 1 -- 172.21.15.83:0/2746362924 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f08037030 con 0x559f24a9f860 2024-04-24T09:15:38.062 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.054+0000 7fb72d400640 1 --2- 172.21.15.83:0/1091073750 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e39ca81ce0 0x55e39ca8fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:15:38.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.054+0000 7f5f01600640 1 -- 172.21.15.83:0/2746362924 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(130..130 src has 1..130) ==== 9583+0+0 (secure 0 0 0) 0x7f5f08024040 con 0x559f24a9f860 2024-04-24T09:15:38.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.054+0000 7f5f21275740 1 -- 172.21.15.83:0/2746362924 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f040054e0 con 0x559f24a9f860 2024-04-24T09:15:38.063 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.054+0000 7fb72e800640 1 --2- 172.21.15.83:0/1091073750 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e39ca8c5a0 0x55e39ca90320 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fb72400e730 tx=0x7fb724005bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef962e989b454979 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:38.063 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.054+0000 7fb70b400640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:15:38.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.055+0000 7f5f01600640 1 -- 172.21.15.83:0/2746362924 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f5f080041c0 con 0x559f24a9f860 2024-04-24T09:15:38.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.055+0000 7f5f21275740 10 client.23670 fetch_fsmap learned FSMap version 69 2024-04-24T09:15:38.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.055+0000 7f5f21275740 1 -- 172.21.15.83:0/2746362924 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x559f249ca810 con 0x559f24a9f860 2024-04-24T09:15:38.064 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.055+0000 7fb70b400640 1 -- 172.21.15.83:0/1091073750 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb724010640 con 0x55e39ca8c5a0 2024-04-24T09:15:38.064 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.055+0000 7fb70b400640 1 -- 172.21.15.83:0/1091073750 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(130..130 src has 1..130) ==== 9583+0+0 (secure 0 0 0) 0x7fb7240107e0 con 0x55e39ca8c5a0 2024-04-24T09:15:38.064 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.055+0000 7fb73356f740 1 -- 172.21.15.83:0/1091073750 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7180043c0 con 0x55e39ca8c5a0 2024-04-24T09:15:38.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.055+0000 7f5f01600640 1 -- 172.21.15.83:0/2746362924 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5f040054e0 con 0x559f24a9f860 2024-04-24T09:15:38.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.055+0000 7f5f01600640 1 -- 172.21.15.83:0/2746362924 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 69) ==== 8370+0+0 (secure 0 0 0) 0x7f5f0803d540 con 0x559f24a9f860 2024-04-24T09:15:38.065 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.056+0000 7fb70b400640 1 -- 172.21.15.83:0/1091073750 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7fb7240109e0 con 0x55e39ca8c5a0 2024-04-24T09:15:38.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.056+0000 7f5f21275740 10 client.23670 fetch_fsmap finished waiting for FSMap version 69 2024-04-24T09:15:38.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.056+0000 7f5f21275740 10 client.23670 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:15:38.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.056+0000 7f5f21275740 20 client.23670 populate_metadata read hostname 'smithi083' 2024-04-24T09:15:38.066 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.056+0000 7fb73356f740 10 client.40727 fetch_fsmap learned FSMap version 69 2024-04-24T09:15:38.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.056+0000 7f5f21275740 1 --2- 172.21.15.83:0/2746362924 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x559f24b32950 0x559f24b34d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:38.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.056+0000 7f5f21275740 4 client.23670 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T09:15:38.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.056+0000 7f5f21275740 1 -- 172.21.15.83:0/2746362924 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559f24a0ca80 con 0x559f24b32950 2024-04-24T09:15:38.066 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.056+0000 7fb73356f740 1 -- 172.21.15.83:0/1091073750 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x55e39cb13bf0 con 0x55e39ca8c5a0 2024-04-24T09:15:38.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.056+0000 7f5f16a00640 1 --2- 172.21.15.83:0/2746362924 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x559f24b32950 0x559f24b34d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:38.067 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.057+0000 7fb70b400640 1 -- 172.21.15.83:0/1091073750 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb7180043c0 con 0x55e39ca8c5a0 2024-04-24T09:15:38.067 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.057+0000 7fb70b400640 1 -- 172.21.15.83:0/1091073750 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 69) ==== 8370+0+0 (secure 0 0 0) 0x7fb72402bad0 con 0x55e39ca8c5a0 2024-04-24T09:15:38.067 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.057+0000 7fb73356f740 10 client.40727 fetch_fsmap finished waiting for FSMap version 69 2024-04-24T09:15:38.067 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.057+0000 7fb73356f740 10 client.40727 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:15:38.068 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.057+0000 7fb73356f740 20 client.40727 populate_metadata read hostname 'smithi083' 2024-04-24T09:15:38.068 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.057+0000 7fb73356f740 1 --2- 172.21.15.83:0/1091073750 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55e39cb143f0 0x55e39cb16830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:38.068 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.057+0000 7fb73356f740 4 client.40727 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T09:15:38.068 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.057+0000 7fb73356f740 1 -- 172.21.15.83:0/1091073750 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e39c9ef190 con 0x55e39cb143f0 2024-04-24T09:15:38.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.057+0000 7f5f16a00640 1 --2- 172.21.15.83:0/2746362924 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x559f24b32950 0x559f24b34d70 crc :-1 s=READY pgs=587 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1618909ff51c0691 server_cookie=5db2b791899a2cea in_seq=0 out_seq=0 2024-04-24T09:15:38.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.057+0000 7f5f01600640 10 client.23670 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:15:38.069 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.057+0000 7fb72de00640 1 --2- 172.21.15.83:0/1091073750 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55e39cb143f0 0x55e39cb16830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:38.069 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.058+0000 7fb70b400640 10 client.40727 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:15:38.069 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.058+0000 7fb72de00640 1 --2- 172.21.15.83:0/1091073750 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55e39cb143f0 0x55e39cb16830 crc :-1 s=READY pgs=588 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=538f4410f73fbc39 server_cookie=8a4eb6464208dba5 in_seq=0 out_seq=0 2024-04-24T09:15:38.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.059+0000 7f5f01600640 1 -- 172.21.15.83:0/2746362924 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x559f24a0ca80 con 0x559f24b32950 2024-04-24T09:15:38.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.059+0000 7f5f01600640 10 client.23670 handle_command_reply: tid=0 2024-04-24T09:15:38.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.059+0000 7f5f21275740 1 client.23670 shutdown 2024-04-24T09:15:38.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.060+0000 7f5f21275740 1 -- 172.21.15.83:0/2746362924 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f24a9f860 msgr2=0x559f24aae370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:38.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.060+0000 7f5f21275740 1 --2- 172.21.15.83:0/2746362924 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f24a9f860 0x559f24aae370 secure :-1 s=READY pgs=2725 cs=0 l=1 rev1=1 crypto rx=0x7f5f08029890 tx=0x7f5f0801e8b0 comp rx=0 tx=0).stop 2024-04-24T09:15:38.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.060+0000 7f5f21275740 1 -- 172.21.15.83:0/2746362924 shutdown_connections 2024-04-24T09:15:38.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.060+0000 7f5f21275740 1 --2- 172.21.15.83:0/2746362924 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x559f24b32950 0x559f24b34d70 crc :-1 s=READY pgs=587 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:38.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.060+0000 7f5f21275740 1 --2- 172.21.15.83:0/2746362924 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f24aaa850 0x559f24aaedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:38.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.060+0000 7f5f21275740 1 --2- 172.21.15.83:0/2746362924 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559f24aa0180 0x559f24aae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:38.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.060+0000 7f5f21275740 1 --2- 172.21.15.83:0/2746362924 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f24a9f860 0x559f24aae370 unknown :-1 s=CLOSED pgs=2725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:38.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.060+0000 7f5f21275740 1 -- 172.21.15.83:0/2746362924 >> 172.21.15.83:0/2746362924 conn(0x559f24a9bb80 msgr2=0x559f24aa7810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:38.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.060+0000 7f5f01600640 0 client.23670 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:15:38.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.060+0000 7f5f21275740 1 -- 172.21.15.83:0/2746362924 shutdown_connections 2024-04-24T09:15:38.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.060+0000 7f5f21275740 1 -- 172.21.15.83:0/2746362924 wait complete. 2024-04-24T09:15:38.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.061+0000 7f5f21275740 20 client.23670 trim_cache size 0 max 16384 2024-04-24T09:15:38.071 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T09:15:38.071 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-04-24T09:15:38.071 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-04-24T09:15:38.071 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T09:15:38.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.061+0000 7f5f1e400640 1 -- 172.21.15.83:0/117777451 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f5efc0a3da0 msgr2=0x7f5efc0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:38.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.062+0000 7f5f1e400640 1 --2- 172.21.15.83:0/117777451 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f5efc0a3da0 0x7f5efc0a6260 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x559f24aa0e20 tx=0x7f5f08005f90 comp rx=0 tx=0).stop 2024-04-24T09:15:38.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.062+0000 7f5f1e400640 1 -- 172.21.15.83:0/117777451 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f18076f90 msgr2=0x7f5f181ab020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:38.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.062+0000 7f5f1e400640 1 --2- 172.21.15.83:0/117777451 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f18076f90 0x7f5f181ab020 secure :-1 s=READY pgs=2718 cs=0 l=1 rev1=1 crypto rx=0x7f5f10031040 tx=0x7f5f10031440 comp rx=0 tx=0).stop 2024-04-24T09:15:38.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.062+0000 7f5f1e400640 1 -- 172.21.15.83:0/117777451 shutdown_connections 2024-04-24T09:15:38.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.062+0000 7f5f1e400640 1 --2- 172.21.15.83:0/117777451 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f5efc0a3da0 0x7f5efc0a6260 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:38.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.062+0000 7f5f1e400640 1 --2- 172.21.15.83:0/117777451 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f18076f90 0x7f5f181ab020 unknown :-1 s=CLOSED pgs=2718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:38.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.062+0000 7f5f1e400640 1 --2- 172.21.15.83:0/117777451 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f18109580 0x7f5f18076a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:38.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.062+0000 7f5f1e400640 1 --2- 172.21.15.83:0/117777451 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5f181044b0 0x7f5f18076510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:38.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.062+0000 7f5f1e400640 1 -- 172.21.15.83:0/117777451 >> 172.21.15.83:0/117777451 conn(0x7f5f180ffd90 msgr2=0x7f5f181020e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:38.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.062+0000 7f5f1e400640 1 -- 172.21.15.83:0/117777451 shutdown_connections 2024-04-24T09:15:38.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.062+0000 7f5f1e400640 1 -- 172.21.15.83:0/117777451 wait complete. 2024-04-24T09:15:38.085 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T09:15:38.085 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T09:15:38.214 INFO:tasks.workunit.client.0.smithi083.stdout: 10 74826 6.55 MB/sec execute 489 sec latency 138.615 ms 2024-04-24T09:15:38.296 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.295+0000 7fb70b400640 1 -- 172.21.15.83:0/1091073750 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e39c9ef190 con 0x55e39cb143f0 2024-04-24T09:15:38.296 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.295+0000 7fb70b400640 10 client.40727 handle_command_reply: tid=0 2024-04-24T09:15:38.296 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.295+0000 7fb73356f740 1 client.40727 shutdown 2024-04-24T09:15:38.296 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.295+0000 7fb73356f740 1 -- 172.21.15.83:0/1091073750 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e39ca8c5a0 msgr2=0x55e39ca90320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:38.296 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.295+0000 7fb73356f740 1 --2- 172.21.15.83:0/1091073750 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e39ca8c5a0 0x55e39ca90320 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fb72400e730 tx=0x7fb724005bd0 comp rx=0 tx=0).stop 2024-04-24T09:15:38.297 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.296+0000 7fb73356f740 1 -- 172.21.15.83:0/1091073750 shutdown_connections 2024-04-24T09:15:38.297 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.296+0000 7fb73356f740 1 --2- 172.21.15.83:0/1091073750 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55e39cb143f0 0x55e39cb16830 crc :-1 s=READY pgs=588 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:38.297 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.296+0000 7fb73356f740 1 --2- 172.21.15.83:0/1091073750 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e39ca8c5a0 0x55e39ca90320 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:38.297 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.296+0000 7fb73356f740 1 --2- 172.21.15.83:0/1091073750 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e39ca81ce0 0x55e39ca8fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:38.297 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.296+0000 7fb73356f740 1 --2- 172.21.15.83:0/1091073750 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e39ca813c0 0x55e39ca94a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:38.297 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.296+0000 7fb70b400640 0 client.40727 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:15:38.297 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.296+0000 7fb73356f740 1 -- 172.21.15.83:0/1091073750 >> 172.21.15.83:0/1091073750 conn(0x55e39ca7d8d0 msgr2=0x55e39ca7ca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:38.297 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.296+0000 7fb73356f740 1 -- 172.21.15.83:0/1091073750 shutdown_connections 2024-04-24T09:15:38.297 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.296+0000 7fb73356f740 1 -- 172.21.15.83:0/1091073750 wait complete. 2024-04-24T09:15:38.297 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.296+0000 7fb73356f740 20 client.40727 trim_cache size 0 max 16384 2024-04-24T09:15:38.298 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T09:15:38.298 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T09:15:38.298 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T09:15:38.298 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T09:15:38.298 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.297+0000 7fb72f200640 1 -- 172.21.15.83:0/3940913502 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb7100a3e60 msgr2=0x7fb7100a6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:38.299 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.297+0000 7fb72f200640 1 --2- 172.21.15.83:0/3940913502 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb7100a3e60 0x7fb7100a6320 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fb724013fd0 tx=0x7fb724013460 comp rx=0 tx=0).stop 2024-04-24T09:15:38.299 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.297+0000 7fb72f200640 1 -- 172.21.15.83:0/3940913502 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb728104290 msgr2=0x7fb72810fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:38.299 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.297+0000 7fb72f200640 1 --2- 172.21.15.83:0/3940913502 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb728104290 0x7fb72810fa30 secure :-1 s=READY pgs=2719 cs=0 l=1 rev1=1 crypto rx=0x7fb71c005ab0 tx=0x7fb71c003c20 comp rx=0 tx=0).stop 2024-04-24T09:15:38.299 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.297+0000 7fb72f200640 1 -- 172.21.15.83:0/3940913502 shutdown_connections 2024-04-24T09:15:38.299 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.297+0000 7fb72f200640 1 --2- 172.21.15.83:0/3940913502 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb7100a3e60 0x7fb7100a6320 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:38.299 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.297+0000 7fb72f200640 1 --2- 172.21.15.83:0/3940913502 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb72806fa60 0x7fb72806fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:38.299 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.297+0000 7fb72f200640 1 --2- 172.21.15.83:0/3940913502 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb728109360 0x7fb72810ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:38.299 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.297+0000 7fb72f200640 1 --2- 172.21.15.83:0/3940913502 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb728104290 0x7fb72810fa30 unknown :-1 s=CLOSED pgs=2719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:38.299 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.297+0000 7fb72f200640 1 -- 172.21.15.83:0/3940913502 >> 172.21.15.83:0/3940913502 conn(0x7fb7280ffb10 msgr2=0x7fb728100ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:38.299 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.298+0000 7fb72f200640 1 -- 172.21.15.83:0/3940913502 shutdown_connections 2024-04-24T09:15:38.299 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:38.298+0000 7fb72f200640 1 -- 172.21.15.83:0/3940913502 wait complete. 2024-04-24T09:15:38.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.411+0000 7f455de00640 1 -- 172.21.15.83:0/3979603222 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4558104c40 msgr2=0x7f4558108c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:38.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.411+0000 7f455de00640 1 --2- 172.21.15.83:0/3979603222 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4558104c40 0x7f4558108c80 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f4544009900 tx=0x7f454402a3f0 comp rx=0 tx=0).stop 2024-04-24T09:15:38.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.412+0000 7f455de00640 1 -- 172.21.15.83:0/3979603222 shutdown_connections 2024-04-24T09:15:38.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.412+0000 7f455de00640 1 --2- 172.21.15.83:0/3979603222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4558109360 0x7f4558110f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:38.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.412+0000 7f455de00640 1 --2- 172.21.15.83:0/3979603222 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4558104c40 0x7f4558108c80 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:38.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.412+0000 7f455de00640 1 --2- 172.21.15.83:0/3979603222 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4558104290 0x7f4558104670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:38.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.412+0000 7f455de00640 1 -- 172.21.15.83:0/3979603222 >> 172.21.15.83:0/3979603222 conn(0x7f45580ffad0 msgr2=0x7f4558101ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:38.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.412+0000 7f455de00640 1 -- 172.21.15.83:0/3979603222 shutdown_connections 2024-04-24T09:15:38.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.412+0000 7f455de00640 1 -- 172.21.15.83:0/3979603222 wait complete. 2024-04-24T09:15:38.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.414+0000 7f455de00640 1 Processor -- start 2024-04-24T09:15:38.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.414+0000 7f455de00640 1 -- start start 2024-04-24T09:15:38.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.415+0000 7f455de00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4558104290 0x7f45581a05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:38.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.415+0000 7f455de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4558109360 0x7f45581a0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:38.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.415+0000 7f455de00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f455819a670 0x7f455819aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:38.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.415+0000 7f455de00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f45581142d0 con 0x7f4558109360 2024-04-24T09:15:38.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.415+0000 7f455de00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4558114150 con 0x7f455819a670 2024-04-24T09:15:38.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.415+0000 7f455de00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4558114450 con 0x7f4558104290 2024-04-24T09:15:38.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.415+0000 7f455ca00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4558104290 0x7f45581a05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:38.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.416+0000 7f4557e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4558109360 0x7f45581a0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:38.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.416+0000 7f455d400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f455819a670 0x7f455819aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:38.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.416+0000 7f455ca00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4558104290 0x7f45581a05b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:47334/0 (socket says 172.21.15.83:47334) 2024-04-24T09:15:38.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.416+0000 7f455ca00640 1 -- 172.21.15.83:0/1075645305 learned_addr learned my addr 172.21.15.83:0/1075645305 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:15:38.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.416+0000 7f455d400640 1 -- 172.21.15.83:0/1075645305 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4558104290 msgr2=0x7f45581a05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:38.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.416+0000 7f455d400640 1 --2- 172.21.15.83:0/1075645305 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4558104290 0x7f45581a05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:38.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.416+0000 7f455d400640 1 -- 172.21.15.83:0/1075645305 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4558109360 msgr2=0x7f45581a0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:38.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.416+0000 7f455d400640 1 --2- 172.21.15.83:0/1075645305 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4558109360 0x7f45581a0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:38.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.416+0000 7f455d400640 1 -- 172.21.15.83:0/1075645305 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f455819b270 con 0x7f455819a670 2024-04-24T09:15:38.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.417+0000 7f455d400640 1 --2- 172.21.15.83:0/1075645305 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f455819a670 0x7f455819aaf0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f4550009fd0 tx=0x7f455000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:38.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.417+0000 7f4555600640 1 -- 172.21.15.83:0/1075645305 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f455000ce20 con 0x7f455819a670 2024-04-24T09:15:38.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.417+0000 7f455de00640 1 -- 172.21.15.83:0/1075645305 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45581a7350 con 0x7f455819a670 2024-04-24T09:15:38.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.417+0000 7f4555600640 1 -- 172.21.15.83:0/1075645305 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4550005020 con 0x7f455819a670 2024-04-24T09:15:38.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.417+0000 7f4555600640 1 -- 172.21.15.83:0/1075645305 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4550010690 con 0x7f455819a670 2024-04-24T09:15:38.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.417+0000 7f455de00640 1 -- 172.21.15.83:0/1075645305 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4558076440 con 0x7f455819a670 2024-04-24T09:15:38.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.419+0000 7f455de00640 1 -- 172.21.15.83:0/1075645305 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4558105760 con 0x7f455819a670 2024-04-24T09:15:38.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.421+0000 7f4555600640 1 -- 172.21.15.83:0/1075645305 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f45500051d0 con 0x7f455819a670 2024-04-24T09:15:38.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.423+0000 7f4555600640 1 --2- 172.21.15.83:0/1075645305 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f45400a3da0 0x7f45400a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:38.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.423+0000 7f4555600640 1 -- 172.21.15.83:0/1075645305 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 9583+0+0 (secure 0 0 0) 0x7f45500d1240 con 0x7f455819a670 2024-04-24T09:15:38.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.423+0000 7f455ca00640 1 --2- 172.21.15.83:0/1075645305 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f45400a3da0 0x7f45400a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:38.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.425+0000 7f455ca00640 1 --2- 172.21.15.83:0/1075645305 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f45400a3da0 0x7f45400a6260 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f454c005fd0 tx=0x7f454c00b480 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:38.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.431+0000 7f4555600640 1 -- 172.21.15.83:0/1075645305 <== mon.2 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f455009d460 con 0x7f455819a670 2024-04-24T09:15:38.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:15:38 smithi138 ceph-mon[31441]: pgmap v759: 129 pgs: 129 active+clean; 1.1 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.6 MiB/s wr, 223 op/s 2024-04-24T09:15:38.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:15:38 smithi189 ceph-mon[31791]: pgmap v759: 129 pgs: 129 active+clean; 1.1 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.6 MiB/s wr, 223 op/s 2024-04-24T09:15:38.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:15:38 smithi083 ceph-mon[27907]: pgmap v759: 129 pgs: 129 active+clean; 1.1 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.6 MiB/s wr, 223 op/s 2024-04-24T09:15:38.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.848+0000 7f455de00640 1 -- 172.21.15.83:0/1075645305 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4558104c40 con 0x7f455819a670 2024-04-24T09:15:38.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.849+0000 7f4555600640 1 -- 172.21.15.83:0/1075645305 <== mon.2 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) ==== 94+0+11947 (secure 0 0 0) 0x7f455009cd60 con 0x7f455819a670 2024-04-24T09:15:38.850 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T09:15:38.850 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":69,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/358872257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":358872257},{"type":"v1","addr":"172.21.15.83:6835","nonce":358872257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3157070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3157070851},{"type":"v1","addr":"172.21.15.83:6837","nonce":3157070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":69,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T08:48:29.422576+0000","modified":"2024-04-24T09:15:17.129816+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34520,"mds_1":34493,"mds_2":34451,"mds_3":34427,"mds_4":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14658":{"gid":14658,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6839/4085470844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":4085470844},{"type":"v1","addr":"172.21.15.83:6839","nonce":4085470844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14676":{"gid":14676,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6841/2634754210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":2634754210},{"type":"v1","addr":"172.21.15.83:6841","nonce":2634754210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24382":{"gid":24382,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/1859921310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1859921310},{"type":"v1","addr":"172.21.15.189:6833","nonce":1859921310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/2241923197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2241923197},{"type":"v1","addr":"172.21.15.189:6835","nonce":2241923197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.189:6839/2548965336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":2548965336},{"type":"v1","addr":"172.21.15.189:6839","nonce":2548965336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26686":{"gid":26686,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6837/1678817161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":1678817161},{"type":"v1","addr":"172.21.15.189:6837","nonce":1678817161}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.138:6833/1157967658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1157967658},{"type":"v1","addr":"172.21.15.138:6833","nonce":1157967658}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.138:6835/1909219758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1909219758},{"type":"v1","addr":"172.21.15.138:6835","nonce":1909219758}]},"join_fscid":-1,"export_targets":[0,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34493":{"gid":34493,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.138:6837/900566754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":900566754},{"type":"v1","addr":"172.21.15.138:6837","nonce":900566754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34520":{"gid":34520,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.138:6839/2236932393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2236932393},{"type":"v1","addr":"172.21.15.138:6839","nonce":2236932393}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34520,"qdb_cluster":[24454,34520,34427,34451,34493]},"id":1}]} 2024-04-24T09:15:38.850 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 69 2024-04-24T09:15:38.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.853+0000 7f455de00640 1 -- 172.21.15.83:0/1075645305 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f45400a3da0 msgr2=0x7f45400a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:38.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.853+0000 7f455de00640 1 --2- 172.21.15.83:0/1075645305 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f45400a3da0 0x7f45400a6260 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f454c005fd0 tx=0x7f454c00b480 comp rx=0 tx=0).stop 2024-04-24T09:15:38.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.853+0000 7f455de00640 1 -- 172.21.15.83:0/1075645305 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f455819a670 msgr2=0x7f455819aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:38.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.853+0000 7f455de00640 1 --2- 172.21.15.83:0/1075645305 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f455819a670 0x7f455819aaf0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f4550009fd0 tx=0x7f455000ef10 comp rx=0 tx=0).stop 2024-04-24T09:15:38.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.853+0000 7f455de00640 1 -- 172.21.15.83:0/1075645305 shutdown_connections 2024-04-24T09:15:38.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.853+0000 7f455de00640 1 --2- 172.21.15.83:0/1075645305 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f45400a3da0 0x7f45400a6260 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:38.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.853+0000 7f455de00640 1 --2- 172.21.15.83:0/1075645305 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f455819a670 0x7f455819aaf0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:38.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.853+0000 7f455de00640 1 --2- 172.21.15.83:0/1075645305 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4558109360 0x7f45581a0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:38.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.853+0000 7f455de00640 1 --2- 172.21.15.83:0/1075645305 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4558104290 0x7f45581a05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:38.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.853+0000 7f455de00640 1 -- 172.21.15.83:0/1075645305 >> 172.21.15.83:0/1075645305 conn(0x7f45580ffad0 msgr2=0x7f4558100e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:38.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.853+0000 7f455de00640 1 -- 172.21.15.83:0/1075645305 shutdown_connections 2024-04-24T09:15:38.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:38.853+0000 7f455de00640 1 -- 172.21.15.83:0/1075645305 wait complete. 2024-04-24T09:15:38.879 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-24T09:15:39.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.154+0000 7f5e45800640 1 -- 172.21.15.83:0/3990564514 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e40104cd0 msgr2=0x7f5e40108d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:39.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.154+0000 7f5e45800640 1 --2- 172.21.15.83:0/3990564514 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e40104cd0 0x7f5e40108d10 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c009860 tx=0x7f5e2c02a340 comp rx=0 tx=0).stop 2024-04-24T09:15:39.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.154+0000 7f5e45800640 1 -- 172.21.15.83:0/3990564514 shutdown_connections 2024-04-24T09:15:39.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.154+0000 7f5e45800640 1 --2- 172.21.15.83:0/3990564514 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5e401093f0 0x7f5e40110fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:39.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.154+0000 7f5e45800640 1 --2- 172.21.15.83:0/3990564514 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e40104cd0 0x7f5e40108d10 secure :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c009860 tx=0x7f5e2c02a340 comp rx=0 tx=0).stop 2024-04-24T09:15:39.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.154+0000 7f5e45800640 1 --2- 172.21.15.83:0/3990564514 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5e40104320 0x7f5e40104700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:39.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.154+0000 7f5e45800640 1 -- 172.21.15.83:0/3990564514 >> 172.21.15.83:0/3990564514 conn(0x7f5e400ffb40 msgr2=0x7f5e40101f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:39.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.154+0000 7f5e45800640 1 -- 172.21.15.83:0/3990564514 shutdown_connections 2024-04-24T09:15:39.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.154+0000 7f5e45800640 1 -- 172.21.15.83:0/3990564514 wait complete. 2024-04-24T09:15:39.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.156+0000 7f5e45800640 1 Processor -- start 2024-04-24T09:15:39.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.158+0000 7f5e45800640 1 -- start start 2024-04-24T09:15:39.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.159+0000 7f5e45800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e40104320 0x7f5e401a0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:39.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.159+0000 7f5e45800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5e401093f0 0x7f5e401a0c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:39.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.159+0000 7f5e45800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5e4019a8f0 0x7f5e4019ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:39.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.159+0000 7f5e45800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5e40114330 con 0x7f5e401093f0 2024-04-24T09:15:39.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.159+0000 7f5e45800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5e401141b0 con 0x7f5e40104320 2024-04-24T09:15:39.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.159+0000 7f5e45800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5e401144b0 con 0x7f5e4019a8f0 2024-04-24T09:15:39.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.160+0000 7f5e3f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5e401093f0 0x7f5e401a0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:39.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.160+0000 7f5e3fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e40104320 0x7f5e401a0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:39.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.161+0000 7f5e3f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5e401093f0 0x7f5e401a0c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43080/0 (socket says 172.21.15.83:43080) 2024-04-24T09:15:39.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.161+0000 7f5e3f400640 1 -- 172.21.15.83:0/63088014 learned_addr learned my addr 172.21.15.83:0/63088014 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:15:39.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.160+0000 7f5e44e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5e4019a8f0 0x7f5e4019ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:39.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.162+0000 7f5e3f400640 1 -- 172.21.15.83:0/63088014 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5e4019a8f0 msgr2=0x7f5e4019ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:39.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.162+0000 7f5e3f400640 1 --2- 172.21.15.83:0/63088014 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5e4019a8f0 0x7f5e4019ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:39.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.162+0000 7f5e3f400640 1 -- 172.21.15.83:0/63088014 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e40104320 msgr2=0x7f5e401a0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:39.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.162+0000 7f5e3f400640 1 --2- 172.21.15.83:0/63088014 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e40104320 0x7f5e401a0740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:39.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.162+0000 7f5e3f400640 1 -- 172.21.15.83:0/63088014 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e4019b410 con 0x7f5e401093f0 2024-04-24T09:15:39.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.162+0000 7f5e44e00640 1 --2- 172.21.15.83:0/63088014 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5e4019a8f0 0x7f5e4019ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:15:39.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.162+0000 7f5e3fe00640 1 --2- 172.21.15.83:0/63088014 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e40104320 0x7f5e401a0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:15:39.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.163+0000 7f5e3f400640 1 --2- 172.21.15.83:0/63088014 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5e401093f0 0x7f5e401a0c80 secure :-1 s=READY pgs=2726 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c0303f0 tx=0x7f5e2c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:39.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.166+0000 7f5e3cc00640 1 -- 172.21.15.83:0/63088014 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e2c037050 con 0x7f5e401093f0 2024-04-24T09:15:39.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.166+0000 7f5e3cc00640 1 -- 172.21.15.83:0/63088014 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5e2c038040 con 0x7f5e401093f0 2024-04-24T09:15:39.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.166+0000 7f5e3cc00640 1 -- 172.21.15.83:0/63088014 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e2c03e950 con 0x7f5e401093f0 2024-04-24T09:15:39.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.167+0000 7f5e45800640 1 -- 172.21.15.83:0/63088014 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e4019b6a0 con 0x7f5e401093f0 2024-04-24T09:15:39.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.168+0000 7f5e45800640 1 -- 172.21.15.83:0/63088014 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e40076680 con 0x7f5e401093f0 2024-04-24T09:15:39.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.177+0000 7f5e49de3740 1 Processor -- start 2024-04-24T09:15:39.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.178+0000 7f5e49de3740 1 -- start start 2024-04-24T09:15:39.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.178+0000 7f5e49de3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a1aa094b40 0x55a1aa094f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:39.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.178+0000 7f5e49de3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1aa095460 0x55a1aa09f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:39.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.178+0000 7f5e49de3740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1aa09fb30 0x55a1aa0a1f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:39.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.178+0000 7f5e49de3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a1a9ffa1d0 con 0x55a1aa095460 2024-04-24T09:15:39.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.178+0000 7f5e49de3740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a1a9fa0450 con 0x55a1aa09fb30 2024-04-24T09:15:39.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.178+0000 7f5e49de3740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a1a9ff6190 con 0x55a1aa094b40 2024-04-24T09:15:39.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.179+0000 7f5e44e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a1aa094b40 0x55a1aa094f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:39.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.179+0000 7f5e44e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a1aa094b40 0x55a1aa094f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:33832/0 (socket says 172.21.15.83:33832) 2024-04-24T09:15:39.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.179+0000 7f5e44e00640 1 -- 172.21.15.83:0/4076551469 learned_addr learned my addr 172.21.15.83:0/4076551469 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:15:39.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.180+0000 7f5e3fe00640 1 --2- 172.21.15.83:0/4076551469 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1aa095460 0x55a1aa09f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:39.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.180+0000 7f5e44e00640 1 -- 172.21.15.83:0/4076551469 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1aa09fb30 msgr2=0x55a1aa0a1f20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T09:15:39.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.180+0000 7f5e44e00640 1 --2- 172.21.15.83:0/4076551469 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1aa09fb30 0x55a1aa0a1f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:39.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.180+0000 7f5e44e00640 1 -- 172.21.15.83:0/4076551469 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1aa095460 msgr2=0x55a1aa09f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:39.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.180+0000 7f5e44e00640 1 --2- 172.21.15.83:0/4076551469 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1aa095460 0x55a1aa09f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:39.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.180+0000 7f5e44e00640 1 -- 172.21.15.83:0/4076551469 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1aa05d310 con 0x55a1aa094b40 2024-04-24T09:15:39.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.180+0000 7f5e3cc00640 1 -- 172.21.15.83:0/63088014 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5e2c045020 con 0x7f5e401093f0 2024-04-24T09:15:39.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.180+0000 7f5e44e00640 1 --2- 172.21.15.83:0/4076551469 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a1aa094b40 0x55a1aa094f20 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5e3000bfb0 tx=0x7f5e30010860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c408d4c12f70a30 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:39.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.181+0000 7f5e35600640 1 -- 172.21.15.83:0/4076551469 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e30022710 con 0x55a1aa094b40 2024-04-24T09:15:39.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.181+0000 7f5e35600640 1 -- 172.21.15.83:0/4076551469 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5e30022d60 con 0x55a1aa094b40 2024-04-24T09:15:39.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.181+0000 7f5e35600640 1 -- 172.21.15.83:0/4076551469 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e3002b380 con 0x55a1aa094b40 2024-04-24T09:15:39.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.184+0000 7f5e49de3740 1 -- 172.21.15.83:0/4076551469 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a1aa094b40 msgr2=0x55a1aa094f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:39.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.184+0000 7f5e49de3740 1 --2- 172.21.15.83:0/4076551469 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a1aa094b40 0x55a1aa094f20 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5e3000bfb0 tx=0x7f5e30010860 comp rx=0 tx=0).stop 2024-04-24T09:15:39.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.184+0000 7f5e3cc00640 1 --2- 172.21.15.83:0/63088014 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f5e240a3e20 0x7f5e240a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:39.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.184+0000 7f5e3cc00640 1 -- 172.21.15.83:0/63088014 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 9583+0+0 (secure 0 0 0) 0x7f5e2c0f09c0 con 0x7f5e401093f0 2024-04-24T09:15:39.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.185+0000 7f5e44e00640 1 --2- 172.21.15.83:0/63088014 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f5e240a3e20 0x7f5e240a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:39.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.186+0000 7f5e49de3740 1 -- 172.21.15.83:0/4076551469 shutdown_connections 2024-04-24T09:15:39.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.186+0000 7f5e49de3740 1 --2- 172.21.15.83:0/4076551469 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1aa09fb30 0x55a1aa0a1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:39.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.186+0000 7f5e49de3740 1 --2- 172.21.15.83:0/4076551469 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1aa095460 0x55a1aa09f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:39.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.186+0000 7f5e49de3740 1 --2- 172.21.15.83:0/4076551469 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a1aa094b40 0x55a1aa094f20 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:39.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.186+0000 7f5e49de3740 1 -- 172.21.15.83:0/4076551469 >> 172.21.15.83:0/4076551469 conn(0x55a1aa090e60 msgr2=0x55a1aa091240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:39.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.186+0000 7f5e49de3740 1 -- 172.21.15.83:0/4076551469 shutdown_connections 2024-04-24T09:15:39.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.186+0000 7f5e44e00640 1 --2- 172.21.15.83:0/63088014 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f5e240a3e20 0x7f5e240a62e0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f5e3002da50 tx=0x7f5e30024040 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:39.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.186+0000 7f5e49de3740 1 -- 172.21.15.83:0/4076551469 wait complete. 2024-04-24T09:15:39.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.187+0000 7f5e49de3740 1 -- start start 2024-04-24T09:15:39.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.187+0000 7f5e49de3740 1 Processor -- start 2024-04-24T09:15:39.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.187+0000 7f5e49de3740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1aa094b40 0x55a1aa09c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:39.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.187+0000 7f5e49de3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a1aa095460 0x55a1aa09ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:39.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.187+0000 7f5e49de3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1aa09fb30 0x55a1aa09d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:39.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.187+0000 7f5e49de3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a1aa0a29c0 con 0x55a1aa09fb30 2024-04-24T09:15:39.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.187+0000 7f5e49de3740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a1a9fb3810 con 0x55a1aa094b40 2024-04-24T09:15:39.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.188+0000 7f5e49de3740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a1aa0a2b40 con 0x55a1aa095460 2024-04-24T09:15:39.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.188+0000 7f5e49de3740 10 client.0 _ll_register_callbacks cb 0x55a1a9f81140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:15:39.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.188+0000 7f5e44e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1aa09fb30 0x55a1aa09d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:39.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.188+0000 7f5e3f400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a1aa095460 0x55a1aa09ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:39.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.188+0000 7f5e3f400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a1aa095460 0x55a1aa09ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:33838/0 (socket says 172.21.15.83:33838) 2024-04-24T09:15:39.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.188+0000 7f5e3f400640 1 -- 172.21.15.83:0/4008755970 learned_addr learned my addr 172.21.15.83:0/4008755970 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:15:39.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.188+0000 7f5e3fe00640 1 --2- 172.21.15.83:0/4008755970 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1aa094b40 0x55a1aa09c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:39.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.188+0000 7f5e3f400640 1 -- 172.21.15.83:0/4008755970 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1aa094b40 msgr2=0x55a1aa09c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:39.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.188+0000 7f5e3f400640 1 --2- 172.21.15.83:0/4008755970 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1aa094b40 0x55a1aa09c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:39.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.188+0000 7f5e3f400640 1 -- 172.21.15.83:0/4008755970 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1aa09fb30 msgr2=0x55a1aa09d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:39.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.188+0000 7f5e3f400640 1 --2- 172.21.15.83:0/4008755970 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1aa09fb30 0x55a1aa09d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:39.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.189+0000 7f5e3f400640 1 -- 172.21.15.83:0/4008755970 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1a9fc43f0 con 0x55a1aa095460 2024-04-24T09:15:39.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.189+0000 7f5e3f400640 1 --2- 172.21.15.83:0/4008755970 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a1aa095460 0x55a1aa09ce60 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c0bc2f0 tx=0x7f5e2c0bc3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2795f674ae778714 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:39.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.189+0000 7f5e2a000640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-04-24T09:15:39.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.189+0000 7f5e2a000640 1 -- 172.21.15.83:0/4008755970 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e2c0ee120 con 0x55a1aa095460 2024-04-24T09:15:39.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.189+0000 7f5e49de3740 1 -- 172.21.15.83:0/4008755970 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e30022d60 con 0x55a1aa095460 2024-04-24T09:15:39.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.189+0000 7f5e2a000640 1 -- 172.21.15.83:0/4008755970 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(130..130 src has 1..130) ==== 9583+0+0 (secure 0 0 0) 0x7f5e2c0c4fd0 con 0x55a1aa095460 2024-04-24T09:15:39.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.189+0000 7f5e2a000640 1 -- 172.21.15.83:0/4008755970 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f5e2c0da050 con 0x55a1aa095460 2024-04-24T09:15:39.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.189+0000 7f5e49de3740 10 client.30451 fetch_fsmap learned FSMap version 69 2024-04-24T09:15:39.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.189+0000 7f5e49de3740 1 -- 172.21.15.83:0/4008755970 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x55a1a9d0c3f0 con 0x55a1aa095460 2024-04-24T09:15:39.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.190+0000 7f5e2a000640 1 -- 172.21.15.83:0/4008755970 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5e30022d60 con 0x55a1aa095460 2024-04-24T09:15:39.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.190+0000 7f5e2a000640 1 -- 172.21.15.83:0/4008755970 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 69) ==== 8370+0+0 (secure 0 0 0) 0x7f5e2c0bba40 con 0x55a1aa095460 2024-04-24T09:15:39.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.190+0000 7f5e49de3740 10 client.30451 fetch_fsmap finished waiting for FSMap version 69 2024-04-24T09:15:39.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.190+0000 7f5e49de3740 10 client.30451 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-24T09:15:39.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.190+0000 7f5e49de3740 20 client.30451 populate_metadata read hostname 'smithi083' 2024-04-24T09:15:39.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.190+0000 7f5e49de3740 1 --2- 172.21.15.83:0/4008755970 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55a1aa0c0d20 0x55a1aa0c3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:39.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.190+0000 7f5e49de3740 4 client.30451 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:15:39.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.190+0000 7f5e49de3740 1 -- 172.21.15.83:0/4008755970 --> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a1aa007bf0 con 0x55a1aa0c0d20 2024-04-24T09:15:39.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.191+0000 7f5e3fe00640 1 --2- 172.21.15.83:0/4008755970 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55a1aa0c0d20 0x55a1aa0c3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:39.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.192+0000 7f5e3fe00640 1 --2- 172.21.15.83:0/4008755970 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55a1aa0c0d20 0x55a1aa0c3110 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d2d42c2a03f1aae3 server_cookie=2a42393a1a6de769 in_seq=0 out_seq=0 2024-04-24T09:15:39.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.192+0000 7f5e2a000640 10 client.30451 ms_handle_connect on v2:172.21.15.189:6838/2548965336 2024-04-24T09:15:39.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.193+0000 7f5e2a000640 1 -- 172.21.15.83:0/4008755970 <== mds.4 v2:172.21.15.189:6838/2548965336 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a1aa007bf0 con 0x55a1aa0c0d20 2024-04-24T09:15:39.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.193+0000 7f5e2a000640 10 client.30451 handle_command_reply: tid=0 2024-04-24T09:15:39.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.193+0000 7f5e49de3740 1 client.30451 shutdown 2024-04-24T09:15:39.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.193+0000 7f5e49de3740 1 -- 172.21.15.83:0/4008755970 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a1aa095460 msgr2=0x55a1aa09ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:39.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.193+0000 7f5e49de3740 1 --2- 172.21.15.83:0/4008755970 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a1aa095460 0x55a1aa09ce60 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c0bc2f0 tx=0x7f5e2c0bc3c0 comp rx=0 tx=0).stop 2024-04-24T09:15:39.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.194+0000 7f5e49de3740 1 -- 172.21.15.83:0/4008755970 shutdown_connections 2024-04-24T09:15:39.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.194+0000 7f5e49de3740 1 --2- 172.21.15.83:0/4008755970 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55a1aa0c0d20 0x55a1aa0c3110 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:39.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.194+0000 7f5e49de3740 1 --2- 172.21.15.83:0/4008755970 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1aa09fb30 0x55a1aa09d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:39.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.194+0000 7f5e49de3740 1 --2- 172.21.15.83:0/4008755970 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a1aa095460 0x55a1aa09ce60 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:39.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.194+0000 7f5e49de3740 1 --2- 172.21.15.83:0/4008755970 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1aa094b40 0x55a1aa09c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:39.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.194+0000 7f5e49de3740 1 -- 172.21.15.83:0/4008755970 >> 172.21.15.83:0/4008755970 conn(0x55a1aa090e60 msgr2=0x55a1aa096b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:39.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.194+0000 7f5e2a000640 0 client.30451 ms_handle_reset on v2:172.21.15.189:6838/2548965336 2024-04-24T09:15:39.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.195+0000 7f5e49de3740 1 -- 172.21.15.83:0/4008755970 shutdown_connections 2024-04-24T09:15:39.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.195+0000 7f5e49de3740 1 -- 172.21.15.83:0/4008755970 wait complete. 2024-04-24T09:15:39.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.195+0000 7f5e49de3740 20 client.30451 trim_cache size 0 max 16384 2024-04-24T09:15:39.213 INFO:tasks.workunit.client.0.smithi083.stdout: 10 74948 6.57 MB/sec execute 490 sec latency 124.234 ms 2024-04-24T09:15:39.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.217+0000 7f5e49de3740 1 Processor -- start 2024-04-24T09:15:39.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.217+0000 7f5e49de3740 1 -- start start 2024-04-24T09:15:39.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.218+0000 7f5e49de3740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1aa094b40 0x55a1aa0906d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:39.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.218+0000 7f5e49de3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1aa095460 0x55a1aa09d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:39.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.218+0000 7f5e49de3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a1aa09fb30 0x55a1aa0c89c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:39.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.218+0000 7f5e49de3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5e40071c10 con 0x55a1aa095460 2024-04-24T09:15:39.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.218+0000 7f5e49de3740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5e400737b0 con 0x55a1aa094b40 2024-04-24T09:15:39.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.218+0000 7f5e49de3740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5e400722e0 con 0x55a1aa09fb30 2024-04-24T09:15:39.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.218+0000 7f5e3f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1aa095460 0x55a1aa09d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:39.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.218+0000 7f5e3f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1aa095460 0x55a1aa09d1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43114/0 (socket says 172.21.15.83:43114) 2024-04-24T09:15:39.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.218+0000 7f5e3f400640 1 -- 172.21.15.83:0/2723589904 learned_addr learned my addr 172.21.15.83:0/2723589904 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:15:39.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.218+0000 7f5e44e00640 1 --2- 172.21.15.83:0/2723589904 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a1aa09fb30 0x55a1aa0c89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:39.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.218+0000 7f5e3fe00640 1 --2- 172.21.15.83:0/2723589904 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1aa094b40 0x55a1aa0906d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:39.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.219+0000 7f5e3f400640 1 -- 172.21.15.83:0/2723589904 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a1aa09fb30 msgr2=0x55a1aa0c89c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:39.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.219+0000 7f5e3f400640 1 --2- 172.21.15.83:0/2723589904 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a1aa09fb30 0x55a1aa0c89c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:39.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.219+0000 7f5e3f400640 1 -- 172.21.15.83:0/2723589904 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1aa094b40 msgr2=0x55a1aa0906d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:39.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.219+0000 7f5e3f400640 1 --2- 172.21.15.83:0/2723589904 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1aa094b40 0x55a1aa0906d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:39.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.219+0000 7f5e3f400640 1 -- 172.21.15.83:0/2723589904 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1a9dd67d0 con 0x55a1aa095460 2024-04-24T09:15:39.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.219+0000 7f5e3f400640 1 --2- 172.21.15.83:0/2723589904 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1aa095460 0x55a1aa09d1c0 secure :-1 s=READY pgs=2727 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c009230 tx=0x7f5e2c0bc4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a74876626f8a1dc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:39.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.220+0000 7f5e34c00640 1 -- 172.21.15.83:0/2723589904 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e2c0ee120 con 0x55a1aa095460 2024-04-24T09:15:39.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.220+0000 7f5e34c00640 1 -- 172.21.15.83:0/2723589904 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5e2c0328f0 con 0x55a1aa095460 2024-04-24T09:15:39.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.220+0000 7f5e34c00640 1 -- 172.21.15.83:0/2723589904 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e2c0bc130 con 0x55a1aa095460 2024-04-24T09:15:39.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.221+0000 7f5e49de3740 1 -- 172.21.15.83:0/2723589904 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1aa095460 msgr2=0x55a1aa09d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:39.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.221+0000 7f5e49de3740 1 --2- 172.21.15.83:0/2723589904 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1aa095460 0x55a1aa09d1c0 secure :-1 s=READY pgs=2727 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c009230 tx=0x7f5e2c0bc4f0 comp rx=0 tx=0).stop 2024-04-24T09:15:39.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.221+0000 7f5e49de3740 1 -- 172.21.15.83:0/2723589904 shutdown_connections 2024-04-24T09:15:39.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.221+0000 7f5e49de3740 1 --2- 172.21.15.83:0/2723589904 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a1aa09fb30 0x55a1aa0c89c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:39.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.221+0000 7f5e49de3740 1 --2- 172.21.15.83:0/2723589904 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1aa095460 0x55a1aa09d1c0 unknown :-1 s=CLOSED pgs=2727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:39.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.221+0000 7f5e49de3740 1 --2- 172.21.15.83:0/2723589904 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1aa094b40 0x55a1aa0906d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:39.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.221+0000 7f5e49de3740 1 -- 172.21.15.83:0/2723589904 >> 172.21.15.83:0/2723589904 conn(0x55a1aa090e60 msgr2=0x55a1aa096290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:39.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.222+0000 7f5e49de3740 1 -- 172.21.15.83:0/2723589904 shutdown_connections 2024-04-24T09:15:39.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.222+0000 7f5e49de3740 1 -- 172.21.15.83:0/2723589904 wait complete. 2024-04-24T09:15:39.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.222+0000 7f5e49de3740 1 -- start start 2024-04-24T09:15:39.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.222+0000 7f5e49de3740 1 Processor -- start 2024-04-24T09:15:39.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.223+0000 7f5e49de3740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1aa094b40 0x55a1aa09bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:39.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.223+0000 7f5e49de3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a1aa095460 0x55a1aa09c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:39.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.223+0000 7f5e49de3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1aa09fb30 0x55a1aa09c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:39.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.223+0000 7f5e49de3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5e400730e0 con 0x55a1aa09fb30 2024-04-24T09:15:39.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.223+0000 7f5e49de3740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5e40072a10 con 0x55a1aa094b40 2024-04-24T09:15:39.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.223+0000 7f5e49de3740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5e4006fc10 con 0x55a1aa095460 2024-04-24T09:15:39.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.223+0000 7f5e49de3740 10 client.0 _ll_register_callbacks cb 0x7f5e2c032f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:15:39.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.224+0000 7f5e3fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1aa094b40 0x55a1aa09bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:39.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.224+0000 7f5e3fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1aa094b40 0x55a1aa09bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:58468/0 (socket says 172.21.15.83:58468) 2024-04-24T09:15:39.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.224+0000 7f5e3fe00640 1 -- 172.21.15.83:0/3494817196 learned_addr learned my addr 172.21.15.83:0/3494817196 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:15:39.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.224+0000 7f5e3f400640 1 --2- 172.21.15.83:0/3494817196 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a1aa095460 0x55a1aa09c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:39.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.224+0000 7f5e44e00640 1 --2- 172.21.15.83:0/3494817196 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1aa09fb30 0x55a1aa09c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:39.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.224+0000 7f5e3fe00640 1 -- 172.21.15.83:0/3494817196 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a1aa095460 msgr2=0x55a1aa09c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:39.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.224+0000 7f5e3fe00640 1 --2- 172.21.15.83:0/3494817196 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a1aa095460 0x55a1aa09c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:39.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.224+0000 7f5e3fe00640 1 -- 172.21.15.83:0/3494817196 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1aa09fb30 msgr2=0x55a1aa09c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:39.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.224+0000 7f5e3fe00640 1 --2- 172.21.15.83:0/3494817196 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1aa09fb30 0x55a1aa09c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:39.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.224+0000 7f5e3fe00640 1 -- 172.21.15.83:0/3494817196 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1aa0a6ef0 con 0x55a1aa094b40 2024-04-24T09:15:39.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.225+0000 7f5e44e00640 1 --2- 172.21.15.83:0/3494817196 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1aa09fb30 0x55a1aa09c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:15:39.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.226+0000 7f5e3fe00640 1 --2- 172.21.15.83:0/3494817196 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1aa094b40 0x55a1aa09bc70 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f5e38009fd0 tx=0x7f5e38014e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b784ba09f79adc8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:39.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.226+0000 7f5e2a000640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:15:39.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.226+0000 7f5e2a000640 1 -- 172.21.15.83:0/3494817196 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e3801e510 con 0x55a1aa094b40 2024-04-24T09:15:39.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.226+0000 7f5e49de3740 1 -- 172.21.15.83:0/3494817196 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e2c0328f0 con 0x55a1aa094b40 2024-04-24T09:15:39.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.226+0000 7f5e2a000640 1 -- 172.21.15.83:0/3494817196 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(130..130 src has 1..130) ==== 9583+0+0 (secure 0 0 0) 0x7f5e3801e6b0 con 0x55a1aa094b40 2024-04-24T09:15:39.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.227+0000 7f5e2a000640 1 -- 172.21.15.83:0/3494817196 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f5e38030050 con 0x55a1aa094b40 2024-04-24T09:15:39.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.227+0000 7f5e49de3740 10 client.40748 fetch_fsmap learned FSMap version 69 2024-04-24T09:15:39.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.227+0000 7f5e49de3740 1 -- 172.21.15.83:0/3494817196 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x55a1aa0a82a0 con 0x55a1aa094b40 2024-04-24T09:15:39.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.227+0000 7f5e2a000640 1 -- 172.21.15.83:0/3494817196 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5e2c0328f0 con 0x55a1aa094b40 2024-04-24T09:15:39.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.227+0000 7f5e2a000640 1 -- 172.21.15.83:0/3494817196 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 69) ==== 8370+0+0 (secure 0 0 0) 0x7f5e38012910 con 0x55a1aa094b40 2024-04-24T09:15:39.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.227+0000 7f5e49de3740 10 client.40748 fetch_fsmap finished waiting for FSMap version 69 2024-04-24T09:15:39.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.227+0000 7f5e49de3740 10 client.40748 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-24T09:15:39.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.227+0000 7f5e49de3740 20 client.40748 populate_metadata read hostname 'smithi083' 2024-04-24T09:15:39.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.227+0000 7f5e49de3740 1 --2- 172.21.15.83:0/3494817196 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55a1aa127370 0x55a1aa129790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:39.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.227+0000 7f5e49de3740 4 client.40748 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:15:39.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.227+0000 7f5e49de3740 1 -- 172.21.15.83:0/3494817196 --> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a1aa001a60 con 0x55a1aa127370 2024-04-24T09:15:39.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.228+0000 7f5e3f400640 1 --2- 172.21.15.83:0/3494817196 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55a1aa127370 0x55a1aa129790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:39.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.228+0000 7f5e3f400640 1 --2- 172.21.15.83:0/3494817196 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55a1aa127370 0x55a1aa129790 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b081e06d7b68ffe server_cookie=79b175aa7c8407db in_seq=0 out_seq=0 2024-04-24T09:15:39.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.229+0000 7f5e2a000640 10 client.40748 ms_handle_connect on v2:172.21.15.189:6838/2548965336 2024-04-24T09:15:39.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.229+0000 7f5e2a000640 1 -- 172.21.15.83:0/3494817196 <== mds.4 v2:172.21.15.189:6838/2548965336 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a1aa001a60 con 0x55a1aa127370 2024-04-24T09:15:39.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.229+0000 7f5e2a000640 10 client.40748 handle_command_reply: tid=0 2024-04-24T09:15:39.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.229+0000 7f5e49de3740 1 client.40748 shutdown 2024-04-24T09:15:39.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.229+0000 7f5e49de3740 1 -- 172.21.15.83:0/3494817196 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1aa094b40 msgr2=0x55a1aa09bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:39.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.229+0000 7f5e49de3740 1 --2- 172.21.15.83:0/3494817196 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1aa094b40 0x55a1aa09bc70 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f5e38009fd0 tx=0x7f5e38014e50 comp rx=0 tx=0).stop 2024-04-24T09:15:39.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.229+0000 7f5e49de3740 1 -- 172.21.15.83:0/3494817196 shutdown_connections 2024-04-24T09:15:39.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.229+0000 7f5e49de3740 1 --2- 172.21.15.83:0/3494817196 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55a1aa127370 0x55a1aa129790 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:39.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.229+0000 7f5e49de3740 1 --2- 172.21.15.83:0/3494817196 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1aa09fb30 0x55a1aa09c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:39.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.229+0000 7f5e49de3740 1 --2- 172.21.15.83:0/3494817196 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a1aa095460 0x55a1aa09c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:39.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.229+0000 7f5e49de3740 1 --2- 172.21.15.83:0/3494817196 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1aa094b40 0x55a1aa09bc70 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:39.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.229+0000 7f5e49de3740 1 -- 172.21.15.83:0/3494817196 >> 172.21.15.83:0/3494817196 conn(0x55a1aa090e60 msgr2=0x55a1aa09e150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:39.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.229+0000 7f5e2a000640 0 client.40748 ms_handle_reset on v2:172.21.15.189:6838/2548965336 2024-04-24T09:15:39.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.229+0000 7f5e49de3740 1 -- 172.21.15.83:0/3494817196 shutdown_connections 2024-04-24T09:15:39.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.230+0000 7f5e49de3740 1 -- 172.21.15.83:0/3494817196 wait complete. 2024-04-24T09:15:39.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.230+0000 7f5e49de3740 20 client.40748 trim_cache size 0 max 16384 2024-04-24T09:15:39.231 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:15:39.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.230+0000 7f5e45800640 1 -- 172.21.15.83:0/63088014 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f5e240a3e20 msgr2=0x7f5e240a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:39.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.230+0000 7f5e45800640 1 --2- 172.21.15.83:0/63088014 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f5e240a3e20 0x7f5e240a62e0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f5e3002da50 tx=0x7f5e30024040 comp rx=0 tx=0).stop 2024-04-24T09:15:39.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.231+0000 7f5e45800640 1 -- 172.21.15.83:0/63088014 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5e401093f0 msgr2=0x7f5e401a0c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:39.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.231+0000 7f5e45800640 1 --2- 172.21.15.83:0/63088014 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5e401093f0 0x7f5e401a0c80 secure :-1 s=READY pgs=2726 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c0303f0 tx=0x7f5e2c0042f0 comp rx=0 tx=0).stop 2024-04-24T09:15:39.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.231+0000 7f5e45800640 1 -- 172.21.15.83:0/63088014 shutdown_connections 2024-04-24T09:15:39.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.231+0000 7f5e45800640 1 --2- 172.21.15.83:0/63088014 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f5e240a3e20 0x7f5e240a62e0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:39.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.231+0000 7f5e45800640 1 --2- 172.21.15.83:0/63088014 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5e4019a8f0 0x7f5e4019ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:39.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.231+0000 7f5e45800640 1 --2- 172.21.15.83:0/63088014 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5e401093f0 0x7f5e401a0c80 unknown :-1 s=CLOSED pgs=2726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:39.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.231+0000 7f5e45800640 1 --2- 172.21.15.83:0/63088014 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e40104320 0x7f5e401a0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:39.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.231+0000 7f5e45800640 1 -- 172.21.15.83:0/63088014 >> 172.21.15.83:0/63088014 conn(0x7f5e400ffb40 msgr2=0x7f5e401005e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:39.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.231+0000 7f5e45800640 1 -- 172.21.15.83:0/63088014 shutdown_connections 2024-04-24T09:15:39.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.231+0000 7f5e45800640 1 -- 172.21.15.83:0/63088014 wait complete. 2024-04-24T09:15:39.254 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-24T09:15:39.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:15:39 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/1075645305' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:15:39.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:15:39 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/1075645305' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:15:39.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.497+0000 7fd4a6600640 1 -- 172.21.15.83:0/2834947329 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4a01071e0 msgr2=0x7fd4a010edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:39.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.497+0000 7fd4a6600640 1 --2- 172.21.15.83:0/2834947329 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4a01071e0 0x7fd4a010edb0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fd498009ca0 tx=0x7fd49802a8c0 comp rx=0 tx=0).stop 2024-04-24T09:15:39.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.498+0000 7fd4a6600640 1 -- 172.21.15.83:0/2834947329 shutdown_connections 2024-04-24T09:15:39.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.498+0000 7fd4a6600640 1 --2- 172.21.15.83:0/2834947329 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4a01071e0 0x7fd4a010edb0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:39.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.498+0000 7fd4a6600640 1 --2- 172.21.15.83:0/2834947329 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd4a0102ac0 0x7fd4a0106b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:39.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.498+0000 7fd4a6600640 1 --2- 172.21.15.83:0/2834947329 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4a0102110 0x7fd4a01024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:39.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.498+0000 7fd4a6600640 1 -- 172.21.15.83:0/2834947329 >> 172.21.15.83:0/2834947329 conn(0x7fd4a00fd9d0 msgr2=0x7fd4a00ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:39.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.498+0000 7fd4a6600640 1 -- 172.21.15.83:0/2834947329 shutdown_connections 2024-04-24T09:15:39.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.498+0000 7fd4a6600640 1 -- 172.21.15.83:0/2834947329 wait complete. 2024-04-24T09:15:39.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.499+0000 7fd4a6600640 1 Processor -- start 2024-04-24T09:15:39.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.499+0000 7fd4a6600640 1 -- start start 2024-04-24T09:15:39.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.499+0000 7fd4a6600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd4a0102110 0x7fd4a0074300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:39.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.500+0000 7fd4a6600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4a01071e0 0x7fd4a0074840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:39.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.500+0000 7fd4a6600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4a0075fa0 0x7fd4a0074d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:39.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.500+0000 7fd4a6600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd4a01121a0 con 0x7fd4a0075fa0 2024-04-24T09:15:39.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.500+0000 7fd4a6600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd4a0112020 con 0x7fd4a01071e0 2024-04-24T09:15:39.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.500+0000 7fd4a6600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd4a0112320 con 0x7fd4a0102110 2024-04-24T09:15:39.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.500+0000 7fd4a5c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4a0075fa0 0x7fd4a0074d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:39.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.500+0000 7fd4a5c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4a0075fa0 0x7fd4a0074d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43138/0 (socket says 172.21.15.83:43138) 2024-04-24T09:15:39.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.500+0000 7fd4a5c00640 1 -- 172.21.15.83:0/1290453070 learned_addr learned my addr 172.21.15.83:0/1290453070 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:15:39.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.500+0000 7fd4a5200640 1 --2- 172.21.15.83:0/1290453070 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd4a0102110 0x7fd4a0074300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:39.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.501+0000 7fd4a5c00640 1 -- 172.21.15.83:0/1290453070 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd4a0102110 msgr2=0x7fd4a0074300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:39.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.501+0000 7fd4a5c00640 1 --2- 172.21.15.83:0/1290453070 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd4a0102110 0x7fd4a0074300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:39.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.501+0000 7fd4a5c00640 1 -- 172.21.15.83:0/1290453070 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4a01071e0 msgr2=0x7fd4a0074840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:39.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.501+0000 7fd4a5c00640 1 --2- 172.21.15.83:0/1290453070 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4a01071e0 0x7fd4a0074840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:39.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.501+0000 7fd4a5c00640 1 -- 172.21.15.83:0/1290453070 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4a01ab3b0 con 0x7fd4a0075fa0 2024-04-24T09:15:39.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.501+0000 7fd4a5c00640 1 --2- 172.21.15.83:0/1290453070 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4a0075fa0 0x7fd4a0074d80 secure :-1 s=READY pgs=2728 cs=0 l=1 rev1=1 crypto rx=0x7fd498005e30 tx=0x7fd4980093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:39.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.501+0000 7fd49d600640 1 -- 172.21.15.83:0/1290453070 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd498003c70 con 0x7fd4a0075fa0 2024-04-24T09:15:39.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.502+0000 7fd4a6600640 1 -- 172.21.15.83:0/1290453070 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4a01ab640 con 0x7fd4a0075fa0 2024-04-24T09:15:39.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.502+0000 7fd49d600640 1 -- 172.21.15.83:0/1290453070 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd498003e10 con 0x7fd4a0075fa0 2024-04-24T09:15:39.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.502+0000 7fd49d600640 1 -- 172.21.15.83:0/1290453070 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd498030c10 con 0x7fd4a0075fa0 2024-04-24T09:15:39.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.502+0000 7fd4a6600640 1 -- 172.21.15.83:0/1290453070 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4a01abbe0 con 0x7fd4a0075fa0 2024-04-24T09:15:39.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.506+0000 7fd4aaa69740 1 Processor -- start 2024-04-24T09:15:39.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.507+0000 7fd49d600640 1 -- 172.21.15.83:0/1290453070 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd498004040 con 0x7fd4a0075fa0 2024-04-24T09:15:39.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.507+0000 7fd4aaa69740 1 -- start start 2024-04-24T09:15:39.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.507+0000 7fd4aaa69740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563d4570fb40 0x563d4570ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:39.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.507+0000 7fd4aaa69740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d45710460 0x563d4571a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:39.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.507+0000 7fd4aaa69740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563d4571ab30 0x563d4571cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:39.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.507+0000 7fd4aaa69740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x563d456751d0 con 0x563d4570fb40 2024-04-24T09:15:39.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.507+0000 7fd4aaa69740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563d4561b450 con 0x563d45710460 2024-04-24T09:15:39.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.507+0000 7fd4aaa69740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x563d45671190 con 0x563d4571ab30 2024-04-24T09:15:39.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.508+0000 7fd4a5200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d45710460 0x563d4571a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:39.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.508+0000 7fd49fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563d4570fb40 0x563d4570ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:39.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.508+0000 7fd4a5200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d45710460 0x563d4571a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:58488/0 (socket says 172.21.15.83:58488) 2024-04-24T09:15:39.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.508+0000 7fd4a5200640 1 -- 172.21.15.83:0/2425613687 learned_addr learned my addr 172.21.15.83:0/2425613687 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:15:39.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.508+0000 7fd49fe00640 1 --2- 172.21.15.83:0/2425613687 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563d4571ab30 0x563d4571cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:39.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.508+0000 7fd4a5200640 1 -- 172.21.15.83:0/2425613687 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563d4571ab30 msgr2=0x563d4571cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:39.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.508+0000 7fd4a5200640 1 --2- 172.21.15.83:0/2425613687 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563d4571ab30 0x563d4571cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:39.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.508+0000 7fd4a5200640 1 -- 172.21.15.83:0/2425613687 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563d4570fb40 msgr2=0x563d4570ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:39.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.508+0000 7fd4a5200640 1 --2- 172.21.15.83:0/2425613687 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563d4570fb40 0x563d4570ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:39.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.508+0000 7fd4a5200640 1 -- 172.21.15.83:0/2425613687 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d456d8310 con 0x563d45710460 2024-04-24T09:15:39.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.509+0000 7fd4a5200640 1 --2- 172.21.15.83:0/2425613687 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d45710460 0x563d4571a5f0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fd494009cf0 tx=0x7fd494011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a5b76218be3f8ee server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:39.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.509+0000 7fd492000640 1 -- 172.21.15.83:0/2425613687 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd494004230 con 0x563d45710460 2024-04-24T09:15:39.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.509+0000 7fd49d600640 1 --2- 172.21.15.83:0/1290453070 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fd4880a3d20 0x7fd4880a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:39.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.509+0000 7fd492000640 1 -- 172.21.15.83:0/2425613687 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd4940043d0 con 0x563d45710460 2024-04-24T09:15:39.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.509+0000 7fd492000640 1 -- 172.21.15.83:0/2425613687 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd494019820 con 0x563d45710460 2024-04-24T09:15:39.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.510+0000 7fd49d600640 1 -- 172.21.15.83:0/1290453070 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 9583+0+0 (secure 0 0 0) 0x7fd4980f2170 con 0x7fd4a0075fa0 2024-04-24T09:15:39.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.510+0000 7fd49fe00640 1 --2- 172.21.15.83:0/1290453070 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fd4880a3d20 0x7fd4880a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:39.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.510+0000 7fd4aaa69740 1 -- 172.21.15.83:0/2425613687 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d45710460 msgr2=0x563d4571a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:39.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.510+0000 7fd4aaa69740 1 --2- 172.21.15.83:0/2425613687 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d45710460 0x563d4571a5f0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fd494009cf0 tx=0x7fd494011810 comp rx=0 tx=0).stop 2024-04-24T09:15:39.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.510+0000 7fd4aaa69740 1 -- 172.21.15.83:0/2425613687 shutdown_connections 2024-04-24T09:15:39.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.510+0000 7fd4aaa69740 1 --2- 172.21.15.83:0/2425613687 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563d4571ab30 0x563d4571cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:39.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.510+0000 7fd4aaa69740 1 --2- 172.21.15.83:0/2425613687 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d45710460 0x563d4571a5f0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:39.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.510+0000 7fd4aaa69740 1 --2- 172.21.15.83:0/2425613687 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563d4570fb40 0x563d4570ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:39.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.510+0000 7fd4aaa69740 1 -- 172.21.15.83:0/2425613687 >> 172.21.15.83:0/2425613687 conn(0x563d4570be60 msgr2=0x563d4570c240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:39.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.511+0000 7fd4aaa69740 1 -- 172.21.15.83:0/2425613687 shutdown_connections 2024-04-24T09:15:39.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.511+0000 7fd4aaa69740 1 -- 172.21.15.83:0/2425613687 wait complete. 2024-04-24T09:15:39.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.511+0000 7fd49fe00640 1 --2- 172.21.15.83:0/1290453070 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fd4880a3d20 0x7fd4880a61e0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x563d4570cff0 tx=0x7fd48c00d320 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:39.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.512+0000 7fd4aaa69740 1 -- start start 2024-04-24T09:15:39.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.512+0000 7fd4aaa69740 1 Processor -- start 2024-04-24T09:15:39.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.513+0000 7fd4aaa69740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563d4570fb40 0x563d4571a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:39.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.513+0000 7fd4aaa69740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d45710460 0x563d457178c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:39.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.513+0000 7fd4aaa69740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563d4571ab30 0x563d45717fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:39.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.513+0000 7fd4aaa69740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x563d4572e9d0 con 0x563d4571ab30 2024-04-24T09:15:39.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.513+0000 7fd4aaa69740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563d4562e810 con 0x563d45710460 2024-04-24T09:15:39.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.513+0000 7fd4aaa69740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x563d4572eb50 con 0x563d4570fb40 2024-04-24T09:15:39.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.513+0000 7fd4aaa69740 10 client.0 _ll_register_callbacks cb 0x563d455fc140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:15:39.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.513+0000 7fd4a5c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563d4571ab30 0x563d45717fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:39.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.513+0000 7fd4a5c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563d4571ab30 0x563d45717fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43160/0 (socket says 172.21.15.83:43160) 2024-04-24T09:15:39.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.513+0000 7fd4a5c00640 1 -- 172.21.15.83:0/2094025988 learned_addr learned my addr 172.21.15.83:0/2094025988 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:15:39.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.513+0000 7fd49fe00640 1 --2- 172.21.15.83:0/2094025988 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d45710460 0x563d457178c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:39.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.513+0000 7fd4a5200640 1 --2- 172.21.15.83:0/2094025988 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563d4570fb40 0x563d4571a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:39.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.514+0000 7fd4a5c00640 1 -- 172.21.15.83:0/2094025988 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563d4570fb40 msgr2=0x563d4571a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:39.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.514+0000 7fd4a5c00640 1 --2- 172.21.15.83:0/2094025988 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563d4570fb40 0x563d4571a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:39.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.514+0000 7fd4a5c00640 1 -- 172.21.15.83:0/2094025988 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d45710460 msgr2=0x563d457178c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:39.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.514+0000 7fd4a5c00640 1 --2- 172.21.15.83:0/2094025988 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d45710460 0x563d457178c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:39.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.514+0000 7fd4a5c00640 1 -- 172.21.15.83:0/2094025988 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d4563f3f0 con 0x563d4571ab30 2024-04-24T09:15:39.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.514+0000 7fd4a5c00640 1 --2- 172.21.15.83:0/2094025988 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563d4571ab30 0x563d45717fb0 secure :-1 s=READY pgs=2729 cs=0 l=1 rev1=1 crypto rx=0x7fd4980ba200 tx=0x7fd498009a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7de091bc4dabff server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:39.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.514+0000 7fd486a00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:15:39.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.514+0000 7fd486a00640 1 -- 172.21.15.83:0/2094025988 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4980d7060 con 0x563d4571ab30 2024-04-24T09:15:39.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.514+0000 7fd486a00640 1 -- 172.21.15.83:0/2094025988 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(130..130 src has 1..130) ==== 9583+0+0 (secure 0 0 0) 0x7fd4980ef6c0 con 0x563d4571ab30 2024-04-24T09:15:39.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.515+0000 7fd4aaa69740 1 -- 172.21.15.83:0/2094025988 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd4940043d0 con 0x563d4571ab30 2024-04-24T09:15:39.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.515+0000 7fd486a00640 1 -- 172.21.15.83:0/2094025988 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7fd4980dc050 con 0x563d4571ab30 2024-04-24T09:15:39.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.515+0000 7fd4aaa69740 10 client.23703 fetch_fsmap learned FSMap version 69 2024-04-24T09:15:39.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.515+0000 7fd4aaa69740 1 -- 172.21.15.83:0/2094025988 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x563d453873f0 con 0x563d4571ab30 2024-04-24T09:15:39.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.515+0000 7fd486a00640 1 -- 172.21.15.83:0/2094025988 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd4940043d0 con 0x563d4571ab30 2024-04-24T09:15:39.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.516+0000 7fd486a00640 1 -- 172.21.15.83:0/2094025988 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 69) ==== 8370+0+0 (secure 0 0 0) 0x7fd4980c6610 con 0x563d4571ab30 2024-04-24T09:15:39.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.516+0000 7fd4aaa69740 10 client.23703 fetch_fsmap finished waiting for FSMap version 69 2024-04-24T09:15:39.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.516+0000 7fd4aaa69740 10 client.23703 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-24T09:15:39.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.516+0000 7fd4aaa69740 20 client.23703 populate_metadata read hostname 'smithi083' 2024-04-24T09:15:39.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.516+0000 7fd4aaa69740 1 --2- 172.21.15.83:0/2094025988 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x563d4573bcb0 0x563d4573e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:39.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.516+0000 7fd4aaa69740 4 client.23703 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:15:39.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.516+0000 7fd4aaa69740 1 -- 172.21.15.83:0/2094025988 --> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563d45682bf0 con 0x563d4573bcb0 2024-04-24T09:15:39.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.516+0000 7fd4a5200640 1 --2- 172.21.15.83:0/2094025988 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x563d4573bcb0 0x563d4573e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:39.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.517+0000 7fd4a5200640 1 --2- 172.21.15.83:0/2094025988 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x563d4573bcb0 0x563d4573e0e0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1d9131cd5874ede2 server_cookie=ebb6a43c18d9f143 in_seq=0 out_seq=0 2024-04-24T09:15:39.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.517+0000 7fd486a00640 10 client.23703 ms_handle_connect on v2:172.21.15.138:6832/1157967658 2024-04-24T09:15:39.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.518+0000 7fd486a00640 1 -- 172.21.15.83:0/2094025988 <== mds.3 v2:172.21.15.138:6832/1157967658 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563d45682bf0 con 0x563d4573bcb0 2024-04-24T09:15:39.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.518+0000 7fd486a00640 10 client.23703 handle_command_reply: tid=0 2024-04-24T09:15:39.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.518+0000 7fd4aaa69740 1 client.23703 shutdown 2024-04-24T09:15:39.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.519+0000 7fd4aaa69740 1 -- 172.21.15.83:0/2094025988 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563d4571ab30 msgr2=0x563d45717fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:39.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.519+0000 7fd4aaa69740 1 --2- 172.21.15.83:0/2094025988 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563d4571ab30 0x563d45717fb0 secure :-1 s=READY pgs=2729 cs=0 l=1 rev1=1 crypto rx=0x7fd4980ba200 tx=0x7fd498009a00 comp rx=0 tx=0).stop 2024-04-24T09:15:39.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.519+0000 7fd4aaa69740 1 -- 172.21.15.83:0/2094025988 shutdown_connections 2024-04-24T09:15:39.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.519+0000 7fd4aaa69740 1 --2- 172.21.15.83:0/2094025988 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563d4571ab30 0x563d45717fb0 unknown :-1 s=CLOSED pgs=2729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:39.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.519+0000 7fd4aaa69740 1 --2- 172.21.15.83:0/2094025988 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d45710460 0x563d457178c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:39.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.519+0000 7fd4aaa69740 1 --2- 172.21.15.83:0/2094025988 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x563d4573bcb0 0x563d4573e0e0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:39.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.519+0000 7fd4aaa69740 1 --2- 172.21.15.83:0/2094025988 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563d4570fb40 0x563d4571a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:39.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.519+0000 7fd4aaa69740 1 -- 172.21.15.83:0/2094025988 >> 172.21.15.83:0/2094025988 conn(0x563d4570be60 msgr2=0x563d45711b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:39.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.519+0000 7fd486a00640 0 client.23703 ms_handle_reset on v2:172.21.15.138:6832/1157967658 2024-04-24T09:15:39.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.519+0000 7fd4aaa69740 1 -- 172.21.15.83:0/2094025988 shutdown_connections 2024-04-24T09:15:39.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.519+0000 7fd4aaa69740 1 -- 172.21.15.83:0/2094025988 wait complete. 2024-04-24T09:15:39.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.519+0000 7fd4aaa69740 20 client.23703 trim_cache size 0 max 16384 2024-04-24T09:15:39.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.544+0000 7fd4aaa69740 1 Processor -- start 2024-04-24T09:15:39.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.544+0000 7fd4aaa69740 1 -- start start 2024-04-24T09:15:39.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.544+0000 7fd4aaa69740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563d4570fb40 0x563d45717880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:39.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.544+0000 7fd4aaa69740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563d45710460 0x563d457378c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:39.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.544+0000 7fd4aaa69740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d4571ab30 0x563d4571a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:39.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.544+0000 7fd4aaa69740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd4a01ad320 con 0x563d45710460 2024-04-24T09:15:39.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.544+0000 7fd4aaa69740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd4a01aeec0 con 0x563d4571ab30 2024-04-24T09:15:39.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.544+0000 7fd4aaa69740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd4a01ad9f0 con 0x563d4570fb40 2024-04-24T09:15:39.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.545+0000 7fd49fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563d45710460 0x563d457378c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:39.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.545+0000 7fd49fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563d45710460 0x563d457378c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43168/0 (socket says 172.21.15.83:43168) 2024-04-24T09:15:39.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.545+0000 7fd49fe00640 1 -- 172.21.15.83:0/3895279473 learned_addr learned my addr 172.21.15.83:0/3895279473 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:15:39.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.545+0000 7fd4a5200640 1 --2- 172.21.15.83:0/3895279473 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563d4570fb40 0x563d45717880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:39.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.545+0000 7fd4a5c00640 1 --2- 172.21.15.83:0/3895279473 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d4571ab30 0x563d4571a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:39.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.545+0000 7fd49fe00640 1 -- 172.21.15.83:0/3895279473 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563d4570fb40 msgr2=0x563d45717880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:39.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.545+0000 7fd49fe00640 1 --2- 172.21.15.83:0/3895279473 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563d4570fb40 0x563d45717880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:39.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.545+0000 7fd49fe00640 1 -- 172.21.15.83:0/3895279473 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d4571ab30 msgr2=0x563d4571a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:39.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.545+0000 7fd49fe00640 1 --2- 172.21.15.83:0/3895279473 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d4571ab30 0x563d4571a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:39.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.545+0000 7fd49fe00640 1 -- 172.21.15.83:0/3895279473 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d45710290 con 0x563d45710460 2024-04-24T09:15:39.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.546+0000 7fd49fe00640 1 --2- 172.21.15.83:0/3895279473 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563d45710460 0x563d457378c0 secure :-1 s=READY pgs=2730 cs=0 l=1 rev1=1 crypto rx=0x7fd4a006b560 tx=0x7fd48c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc86ce46ba896838 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:39.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.546+0000 7fd491600640 1 -- 172.21.15.83:0/3895279473 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd48c018070 con 0x563d45710460 2024-04-24T09:15:39.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.546+0000 7fd491600640 1 -- 172.21.15.83:0/3895279473 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd48c0278c0 con 0x563d45710460 2024-04-24T09:15:39.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.546+0000 7fd491600640 1 -- 172.21.15.83:0/3895279473 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd48c01da20 con 0x563d45710460 2024-04-24T09:15:39.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.547+0000 7fd4aaa69740 1 -- 172.21.15.83:0/3895279473 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563d45710460 msgr2=0x563d457378c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:39.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.547+0000 7fd4aaa69740 1 --2- 172.21.15.83:0/3895279473 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563d45710460 0x563d457378c0 secure :-1 s=READY pgs=2730 cs=0 l=1 rev1=1 crypto rx=0x7fd4a006b560 tx=0x7fd48c014000 comp rx=0 tx=0).stop 2024-04-24T09:15:39.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.547+0000 7fd4aaa69740 1 -- 172.21.15.83:0/3895279473 shutdown_connections 2024-04-24T09:15:39.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.547+0000 7fd4aaa69740 1 --2- 172.21.15.83:0/3895279473 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d4571ab30 0x563d4571a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:39.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.547+0000 7fd4aaa69740 1 --2- 172.21.15.83:0/3895279473 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563d45710460 0x563d457378c0 unknown :-1 s=CLOSED pgs=2730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:39.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.547+0000 7fd4aaa69740 1 --2- 172.21.15.83:0/3895279473 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563d4570fb40 0x563d45717880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:39.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.547+0000 7fd4aaa69740 1 -- 172.21.15.83:0/3895279473 >> 172.21.15.83:0/3895279473 conn(0x563d4570be60 msgr2=0x563d4570c240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:39.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.547+0000 7fd4aaa69740 1 -- 172.21.15.83:0/3895279473 shutdown_connections 2024-04-24T09:15:39.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.547+0000 7fd4aaa69740 1 -- 172.21.15.83:0/3895279473 wait complete. 2024-04-24T09:15:39.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.548+0000 7fd4aaa69740 1 -- start start 2024-04-24T09:15:39.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.548+0000 7fd4aaa69740 1 Processor -- start 2024-04-24T09:15:39.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.549+0000 7fd4aaa69740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563d4570fb40 0x563d45733dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:39.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.549+0000 7fd4aaa69740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d45710460 0x563d45734310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:39.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.549+0000 7fd4aaa69740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563d4571ab30 0x563d4579f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:39.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.549+0000 7fd4aaa69740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd4a01ae7f0 con 0x563d4570fb40 2024-04-24T09:15:39.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.549+0000 7fd4aaa69740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd4a01ae120 con 0x563d45710460 2024-04-24T09:15:39.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.549+0000 7fd4aaa69740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd4a0073650 con 0x563d4571ab30 2024-04-24T09:15:39.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.549+0000 7fd4aaa69740 10 client.0 _ll_register_callbacks cb 0x7fd4980306d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:15:39.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.549+0000 7fd49fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d45710460 0x563d45734310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:39.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.549+0000 7fd49fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d45710460 0x563d45734310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:58526/0 (socket says 172.21.15.83:58526) 2024-04-24T09:15:39.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.549+0000 7fd49fe00640 1 -- 172.21.15.83:0/362066536 learned_addr learned my addr 172.21.15.83:0/362066536 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:15:39.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.549+0000 7fd4a5c00640 1 --2- 172.21.15.83:0/362066536 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563d4571ab30 0x563d4579f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:39.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.550+0000 7fd4a5200640 1 --2- 172.21.15.83:0/362066536 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563d4570fb40 0x563d45733dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:39.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.550+0000 7fd49fe00640 1 -- 172.21.15.83:0/362066536 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563d4571ab30 msgr2=0x563d4579f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:39.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.550+0000 7fd49fe00640 1 --2- 172.21.15.83:0/362066536 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563d4571ab30 0x563d4579f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:39.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.550+0000 7fd49fe00640 1 -- 172.21.15.83:0/362066536 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563d4570fb40 msgr2=0x563d45733dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:39.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.550+0000 7fd49fe00640 1 --2- 172.21.15.83:0/362066536 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563d4570fb40 0x563d45733dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:39.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.550+0000 7fd49fe00640 1 -- 172.21.15.83:0/362066536 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d454517d0 con 0x563d45710460 2024-04-24T09:15:39.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.550+0000 7fd4a5200640 1 --2- 172.21.15.83:0/362066536 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563d4570fb40 0x563d45733dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:15:39.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.550+0000 7fd49fe00640 1 --2- 172.21.15.83:0/362066536 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d45710460 0x563d45734310 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fd48c004700 tx=0x7fd48c0276c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa9524c1c0efb940 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:39.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.550+0000 7fd486a00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:15:39.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.550+0000 7fd486a00640 1 -- 172.21.15.83:0/362066536 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd48c00de10 con 0x563d45710460 2024-04-24T09:15:39.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.550+0000 7fd4aaa69740 1 -- 172.21.15.83:0/362066536 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd48c0278c0 con 0x563d45710460 2024-04-24T09:15:39.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.551+0000 7fd486a00640 1 -- 172.21.15.83:0/362066536 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(130..130 src has 1..130) ==== 9583+0+0 (secure 0 0 0) 0x7fd48c027c80 con 0x563d45710460 2024-04-24T09:15:39.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.551+0000 7fd486a00640 1 -- 172.21.15.83:0/362066536 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7fd48c03d050 con 0x563d45710460 2024-04-24T09:15:39.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.551+0000 7fd4aaa69740 10 client.40763 fetch_fsmap learned FSMap version 69 2024-04-24T09:15:39.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.551+0000 7fd4aaa69740 1 -- 172.21.15.83:0/362066536 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x563d457147b0 con 0x563d45710460 2024-04-24T09:15:39.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.551+0000 7fd486a00640 1 -- 172.21.15.83:0/362066536 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd48c0278c0 con 0x563d45710460 2024-04-24T09:15:39.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.551+0000 7fd486a00640 1 -- 172.21.15.83:0/362066536 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 69) ==== 8370+0+0 (secure 0 0 0) 0x7fd48c0047a0 con 0x563d45710460 2024-04-24T09:15:39.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.552+0000 7fd4aaa69740 10 client.40763 fetch_fsmap finished waiting for FSMap version 69 2024-04-24T09:15:39.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.552+0000 7fd4aaa69740 10 client.40763 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-24T09:15:39.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.552+0000 7fd4aaa69740 20 client.40763 populate_metadata read hostname 'smithi083' 2024-04-24T09:15:39.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.552+0000 7fd4aaa69740 1 --2- 172.21.15.83:0/362066536 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x563d45714f30 0x563d45715340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:39.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.552+0000 7fd4aaa69740 4 client.40763 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:15:39.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.552+0000 7fd4aaa69740 1 -- 172.21.15.83:0/362066536 --> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563d4567ca60 con 0x563d45714f30 2024-04-24T09:15:39.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.552+0000 7fd4a5200640 1 --2- 172.21.15.83:0/362066536 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x563d45714f30 0x563d45715340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:39.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:15:39 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/1075645305' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:15:39.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.553+0000 7fd4a5200640 1 --2- 172.21.15.83:0/362066536 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x563d45714f30 0x563d45715340 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b725ad2738820cf9 server_cookie=711af6207dbf8328 in_seq=0 out_seq=0 2024-04-24T09:15:39.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.553+0000 7fd486a00640 10 client.40763 ms_handle_connect on v2:172.21.15.138:6832/1157967658 2024-04-24T09:15:39.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.554+0000 7fd486a00640 1 -- 172.21.15.83:0/362066536 <== mds.3 v2:172.21.15.138:6832/1157967658 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563d4567ca60 con 0x563d45714f30 2024-04-24T09:15:39.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.554+0000 7fd486a00640 10 client.40763 handle_command_reply: tid=0 2024-04-24T09:15:39.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.554+0000 7fd4aaa69740 1 client.40763 shutdown 2024-04-24T09:15:39.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.554+0000 7fd4aaa69740 1 -- 172.21.15.83:0/362066536 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d45710460 msgr2=0x563d45734310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:39.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.554+0000 7fd4aaa69740 1 --2- 172.21.15.83:0/362066536 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d45710460 0x563d45734310 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fd48c004700 tx=0x7fd48c0276c0 comp rx=0 tx=0).stop 2024-04-24T09:15:39.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.554+0000 7fd4aaa69740 1 -- 172.21.15.83:0/362066536 shutdown_connections 2024-04-24T09:15:39.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.554+0000 7fd4aaa69740 1 --2- 172.21.15.83:0/362066536 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x563d45714f30 0x563d45715340 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:39.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.554+0000 7fd4aaa69740 1 --2- 172.21.15.83:0/362066536 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563d4571ab30 0x563d4579f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:39.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.554+0000 7fd4aaa69740 1 --2- 172.21.15.83:0/362066536 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d45710460 0x563d45734310 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:39.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.554+0000 7fd4aaa69740 1 --2- 172.21.15.83:0/362066536 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563d4570fb40 0x563d45733dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:39.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.554+0000 7fd4aaa69740 1 -- 172.21.15.83:0/362066536 >> 172.21.15.83:0/362066536 conn(0x563d4570be60 msgr2=0x563d4570a6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:39.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.554+0000 7fd486a00640 0 client.40763 ms_handle_reset on v2:172.21.15.138:6832/1157967658 2024-04-24T09:15:39.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.554+0000 7fd4aaa69740 1 -- 172.21.15.83:0/362066536 shutdown_connections 2024-04-24T09:15:39.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.554+0000 7fd4aaa69740 1 -- 172.21.15.83:0/362066536 wait complete. 2024-04-24T09:15:39.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.555+0000 7fd4aaa69740 20 client.40763 trim_cache size 0 max 16384 2024-04-24T09:15:39.557 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:15:39.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.555+0000 7fd4a6600640 1 -- 172.21.15.83:0/1290453070 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fd4880a3d20 msgr2=0x7fd4880a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:39.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.556+0000 7fd4a6600640 1 --2- 172.21.15.83:0/1290453070 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fd4880a3d20 0x7fd4880a61e0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x563d4570cff0 tx=0x7fd48c00d320 comp rx=0 tx=0).stop 2024-04-24T09:15:39.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.556+0000 7fd4a6600640 1 -- 172.21.15.83:0/1290453070 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4a0075fa0 msgr2=0x7fd4a0074d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:39.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.556+0000 7fd4a6600640 1 --2- 172.21.15.83:0/1290453070 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4a0075fa0 0x7fd4a0074d80 secure :-1 s=READY pgs=2728 cs=0 l=1 rev1=1 crypto rx=0x7fd498005e30 tx=0x7fd4980093d0 comp rx=0 tx=0).stop 2024-04-24T09:15:39.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.556+0000 7fd4a6600640 1 -- 172.21.15.83:0/1290453070 shutdown_connections 2024-04-24T09:15:39.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.556+0000 7fd4a6600640 1 --2- 172.21.15.83:0/1290453070 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fd4880a3d20 0x7fd4880a61e0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:39.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.556+0000 7fd4a6600640 1 --2- 172.21.15.83:0/1290453070 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4a0075fa0 0x7fd4a0074d80 unknown :-1 s=CLOSED pgs=2728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:39.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.556+0000 7fd4a6600640 1 --2- 172.21.15.83:0/1290453070 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4a01071e0 0x7fd4a0074840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:39.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.556+0000 7fd4a6600640 1 --2- 172.21.15.83:0/1290453070 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd4a0102110 0x7fd4a0074300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:39.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.556+0000 7fd4a6600640 1 -- 172.21.15.83:0/1290453070 >> 172.21.15.83:0/1290453070 conn(0x7fd4a00fd9d0 msgr2=0x7fd4a00ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:39.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.556+0000 7fd4a6600640 1 -- 172.21.15.83:0/1290453070 shutdown_connections 2024-04-24T09:15:39.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.556+0000 7fd4a6600640 1 -- 172.21.15.83:0/1290453070 wait complete. 2024-04-24T09:15:39.578 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T09:15:39.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.819+0000 7fb5fb200640 1 -- 172.21.15.83:0/3318137216 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5fc104320 msgr2=0x7fb5fc104700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:39.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.819+0000 7fb5fb200640 1 --2- 172.21.15.83:0/3318137216 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5fc104320 0x7fb5fc104700 secure :-1 s=READY pgs=2731 cs=0 l=1 rev1=1 crypto rx=0x7fb5f4009ca0 tx=0x7fb5f402a8b0 comp rx=0 tx=0).stop 2024-04-24T09:15:39.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.819+0000 7fb5fb200640 1 -- 172.21.15.83:0/3318137216 shutdown_connections 2024-04-24T09:15:39.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.819+0000 7fb5fb200640 1 --2- 172.21.15.83:0/3318137216 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5fc1093f0 0x7fb5fc110fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:39.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.819+0000 7fb5fb200640 1 --2- 172.21.15.83:0/3318137216 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb5fc104cd0 0x7fb5fc108d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:39.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.819+0000 7fb5fb200640 1 --2- 172.21.15.83:0/3318137216 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5fc104320 0x7fb5fc104700 unknown :-1 s=CLOSED pgs=2731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:39.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.819+0000 7fb5fb200640 1 -- 172.21.15.83:0/3318137216 >> 172.21.15.83:0/3318137216 conn(0x7fb5fc0ffb40 msgr2=0x7fb5fc101f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:39.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.819+0000 7fb5fb200640 1 -- 172.21.15.83:0/3318137216 shutdown_connections 2024-04-24T09:15:39.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.819+0000 7fb5fb200640 1 -- 172.21.15.83:0/3318137216 wait complete. 2024-04-24T09:15:39.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.820+0000 7fb5fb200640 1 Processor -- start 2024-04-24T09:15:39.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.821+0000 7fb5fb200640 1 -- start start 2024-04-24T09:15:39.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.821+0000 7fb5fb200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5fc1093f0 0x7fb5fc1a07d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:39.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.821+0000 7fb5fb200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5fc1a0d10 0x7fb5fc19a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:39.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.821+0000 7fb5fb200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb5fc19af10 0x7fb5fc19b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:39.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.821+0000 7fb5fb200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb5fc114380 con 0x7fb5fc1a0d10 2024-04-24T09:15:39.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.821+0000 7fb5fb200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb5fc114200 con 0x7fb5fc1093f0 2024-04-24T09:15:39.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.821+0000 7fb5fb200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb5fc114500 con 0x7fb5fc19af10 2024-04-24T09:15:39.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.821+0000 7fb5f9400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5fc1a0d10 0x7fb5fc19a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:39.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.821+0000 7fb5f9e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5fc1093f0 0x7fb5fc1a07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:39.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.822+0000 7fb5f9e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5fc1093f0 0x7fb5fc1a07d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:58550/0 (socket says 172.21.15.83:58550) 2024-04-24T09:15:39.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.822+0000 7fb5f9e00640 1 -- 172.21.15.83:0/4066847217 learned_addr learned my addr 172.21.15.83:0/4066847217 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:15:39.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.821+0000 7fb5f9400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5fc1a0d10 0x7fb5fc19a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43190/0 (socket says 172.21.15.83:43190) 2024-04-24T09:15:39.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.822+0000 7fb5fa800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb5fc19af10 0x7fb5fc19b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:39.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.822+0000 7fb5f9400640 1 -- 172.21.15.83:0/4066847217 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb5fc19af10 msgr2=0x7fb5fc19b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:39.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.822+0000 7fb5f9400640 1 --2- 172.21.15.83:0/4066847217 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb5fc19af10 0x7fb5fc19b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:39.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.822+0000 7fb5f9400640 1 -- 172.21.15.83:0/4066847217 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5fc1093f0 msgr2=0x7fb5fc1a07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:39.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.822+0000 7fb5f9400640 1 --2- 172.21.15.83:0/4066847217 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5fc1093f0 0x7fb5fc1a07d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:39.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.822+0000 7fb5f9400640 1 -- 172.21.15.83:0/4066847217 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5fc1a7690 con 0x7fb5fc1a0d10 2024-04-24T09:15:39.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.822+0000 7fb5f9e00640 1 --2- 172.21.15.83:0/4066847217 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5fc1093f0 0x7fb5fc1a07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:15:39.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.823+0000 7fb5f9400640 1 --2- 172.21.15.83:0/4066847217 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5fc1a0d10 0x7fb5fc19a9d0 secure :-1 s=READY pgs=2732 cs=0 l=1 rev1=1 crypto rx=0x7fb5e800d6a0 tx=0x7fb5e800db60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:39.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.823+0000 7fb5f2a00640 1 -- 172.21.15.83:0/4066847217 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e8017070 con 0x7fb5fc1a0d10 2024-04-24T09:15:39.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.823+0000 7fb5f2a00640 1 -- 172.21.15.83:0/4066847217 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb5e8009820 con 0x7fb5fc1a0d10 2024-04-24T09:15:39.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.823+0000 7fb5f2a00640 1 -- 172.21.15.83:0/4066847217 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e8004c40 con 0x7fb5fc1a0d10 2024-04-24T09:15:39.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.823+0000 7fb5fb200640 1 -- 172.21.15.83:0/4066847217 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5fc1a78c0 con 0x7fb5fc1a0d10 2024-04-24T09:15:39.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.823+0000 7fb5fb200640 1 -- 172.21.15.83:0/4066847217 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5fc1a7dc0 con 0x7fb5fc1a0d10 2024-04-24T09:15:39.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.827+0000 7fb603e0e740 1 Processor -- start 2024-04-24T09:15:39.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.828+0000 7fb603e0e740 1 -- start start 2024-04-24T09:15:39.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.828+0000 7fb603e0e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f78bc4b40 0x558f78bc4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:39.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.828+0000 7fb603e0e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f78bc5460 0x558f78bcf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:39.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.828+0000 7fb603e0e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558f78bcfb30 0x558f78bd1f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:39.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.828+0000 7fb603e0e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x558f78b2a1d0 con 0x558f78bcfb30 2024-04-24T09:15:39.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.828+0000 7fb603e0e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558f78ad0450 con 0x558f78bc4b40 2024-04-24T09:15:39.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.828+0000 7fb603e0e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x558f78b26190 con 0x558f78bc5460 2024-04-24T09:15:39.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.828+0000 7fb5f2a00640 1 -- 172.21.15.83:0/4066847217 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb5e8005220 con 0x7fb5fc1a0d10 2024-04-24T09:15:39.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.828+0000 7fb5fa800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f78bc4b40 0x558f78bc4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:39.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.828+0000 7fb5fa800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f78bc4b40 0x558f78bc4f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:58558/0 (socket says 172.21.15.83:58558) 2024-04-24T09:15:39.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.828+0000 7fb5fa800640 1 -- 172.21.15.83:0/3720621074 learned_addr learned my addr 172.21.15.83:0/3720621074 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:15:39.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.829+0000 7fb5f9e00640 1 --2- 172.21.15.83:0/3720621074 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f78bc5460 0x558f78bcf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:39.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.829+0000 7fb5f9400640 1 --2- 172.21.15.83:0/3720621074 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558f78bcfb30 0x558f78bd1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:39.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.829+0000 7fb5fa800640 1 -- 172.21.15.83:0/3720621074 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f78bc5460 msgr2=0x558f78bcf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:39.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.829+0000 7fb5fa800640 1 --2- 172.21.15.83:0/3720621074 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f78bc5460 0x558f78bcf5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:39.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.829+0000 7fb5fa800640 1 -- 172.21.15.83:0/3720621074 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558f78bcfb30 msgr2=0x558f78bd1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:39.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.829+0000 7fb5fa800640 1 --2- 172.21.15.83:0/3720621074 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558f78bcfb30 0x558f78bd1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:39.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.829+0000 7fb5fa800640 1 -- 172.21.15.83:0/3720621074 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f78b8d310 con 0x558f78bc4b40 2024-04-24T09:15:39.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.829+0000 7fb5f9e00640 1 --2- 172.21.15.83:0/3720621074 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f78bc5460 0x558f78bcf5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:15:39.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.829+0000 7fb5f9400640 1 --2- 172.21.15.83:0/3720621074 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558f78bcfb30 0x558f78bd1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:15:39.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.829+0000 7fb5fa800640 1 --2- 172.21.15.83:0/3720621074 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f78bc4b40 0x558f78bc4f20 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec00b6b0 tx=0x7fb5ec004610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85ef298e9968439e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:39.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.829+0000 7fb5e7400640 1 -- 172.21.15.83:0/3720621074 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5ec003b20 con 0x558f78bc4b40 2024-04-24T09:15:39.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.829+0000 7fb5e7400640 1 -- 172.21.15.83:0/3720621074 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb5ec015070 con 0x558f78bc4b40 2024-04-24T09:15:39.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.830+0000 7fb5e7400640 1 -- 172.21.15.83:0/3720621074 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5ec011640 con 0x558f78bc4b40 2024-04-24T09:15:39.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.830+0000 7fb5f2a00640 1 --2- 172.21.15.83:0/4066847217 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb5e00a3d20 0x7fb5e00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:39.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.830+0000 7fb5f2a00640 1 -- 172.21.15.83:0/4066847217 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 9583+0+0 (secure 0 0 0) 0x7fb5e80d0470 con 0x7fb5fc1a0d10 2024-04-24T09:15:39.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.830+0000 7fb603e0e740 1 -- 172.21.15.83:0/3720621074 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f78bc4b40 msgr2=0x558f78bc4f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:39.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.830+0000 7fb603e0e740 1 --2- 172.21.15.83:0/3720621074 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f78bc4b40 0x558f78bc4f20 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec00b6b0 tx=0x7fb5ec004610 comp rx=0 tx=0).stop 2024-04-24T09:15:39.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.831+0000 7fb5f9e00640 1 --2- 172.21.15.83:0/4066847217 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb5e00a3d20 0x7fb5e00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:39.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.831+0000 7fb603e0e740 1 -- 172.21.15.83:0/3720621074 shutdown_connections 2024-04-24T09:15:39.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.831+0000 7fb603e0e740 1 --2- 172.21.15.83:0/3720621074 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558f78bcfb30 0x558f78bd1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:39.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.831+0000 7fb603e0e740 1 --2- 172.21.15.83:0/3720621074 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f78bc5460 0x558f78bcf5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:39.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.831+0000 7fb603e0e740 1 --2- 172.21.15.83:0/3720621074 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f78bc4b40 0x558f78bc4f20 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:39.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.831+0000 7fb603e0e740 1 -- 172.21.15.83:0/3720621074 >> 172.21.15.83:0/3720621074 conn(0x558f78bc0e60 msgr2=0x558f78bc1240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:39.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.831+0000 7fb603e0e740 1 -- 172.21.15.83:0/3720621074 shutdown_connections 2024-04-24T09:15:39.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.831+0000 7fb603e0e740 1 -- 172.21.15.83:0/3720621074 wait complete. 2024-04-24T09:15:39.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.832+0000 7fb603e0e740 1 -- start start 2024-04-24T09:15:39.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.832+0000 7fb603e0e740 1 Processor -- start 2024-04-24T09:15:39.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.832+0000 7fb5f9e00640 1 --2- 172.21.15.83:0/4066847217 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb5e00a3d20 0x7fb5e00a61e0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x558f78bc6100 tx=0x7fb5f4009940 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:39.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.833+0000 7fb603e0e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f78bc4b40 0x558f78bcf5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:39.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.833+0000 7fb603e0e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558f78bc5460 0x558f78bcc8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:39.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.833+0000 7fb603e0e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f78bcfb30 0x558f78bccfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:39.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.833+0000 7fb603e0e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x558f78bd29c0 con 0x558f78bc5460 2024-04-24T09:15:39.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.833+0000 7fb603e0e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558f78ae3810 con 0x558f78bcfb30 2024-04-24T09:15:39.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.833+0000 7fb603e0e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x558f78bd2b40 con 0x558f78bc4b40 2024-04-24T09:15:39.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.833+0000 7fb603e0e740 10 client.0 _ll_register_callbacks cb 0x558f78ab1140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:15:39.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.833+0000 7fb5fa800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f78bcfb30 0x558f78bccfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:39.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.833+0000 7fb5fa800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f78bcfb30 0x558f78bccfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:58568/0 (socket says 172.21.15.83:58568) 2024-04-24T09:15:39.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.833+0000 7fb5fa800640 1 -- 172.21.15.83:0/76048887 learned_addr learned my addr 172.21.15.83:0/76048887 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:15:39.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.833+0000 7fb5f9400640 1 --2- 172.21.15.83:0/76048887 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558f78bc5460 0x558f78bcc8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:39.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.834+0000 7fb5f9e00640 1 --2- 172.21.15.83:0/76048887 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f78bc4b40 0x558f78bcf5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:39.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.834+0000 7fb5fa800640 1 -- 172.21.15.83:0/76048887 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f78bc4b40 msgr2=0x558f78bcf5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:39.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.834+0000 7fb5fa800640 1 --2- 172.21.15.83:0/76048887 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f78bc4b40 0x558f78bcf5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:39.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.834+0000 7fb5fa800640 1 -- 172.21.15.83:0/76048887 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558f78bc5460 msgr2=0x558f78bcc8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:39.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.834+0000 7fb5fa800640 1 --2- 172.21.15.83:0/76048887 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558f78bc5460 0x558f78bcc8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:39.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.834+0000 7fb5fa800640 1 -- 172.21.15.83:0/76048887 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f78af43f0 con 0x558f78bcfb30 2024-04-24T09:15:39.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.834+0000 7fb5fa800640 1 --2- 172.21.15.83:0/76048887 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f78bcfb30 0x558f78bccfa0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec004290 tx=0x7fb5ec01a6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e247b9189c76c4d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:39.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.834+0000 7fb5dbe00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:15:39.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.835+0000 7fb5dbe00640 1 -- 172.21.15.83:0/76048887 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5ec028060 con 0x558f78bcfb30 2024-04-24T09:15:39.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.835+0000 7fb5dbe00640 1 -- 172.21.15.83:0/76048887 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(130..130 src has 1..130) ==== 9583+0+0 (secure 0 0 0) 0x7fb5ec003920 con 0x558f78bcfb30 2024-04-24T09:15:39.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.835+0000 7fb603e0e740 1 -- 172.21.15.83:0/76048887 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5ec015070 con 0x558f78bcfb30 2024-04-24T09:15:39.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.835+0000 7fb5dbe00640 1 -- 172.21.15.83:0/76048887 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7fb5ec003b20 con 0x558f78bcfb30 2024-04-24T09:15:39.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.835+0000 7fb603e0e740 10 client.40775 fetch_fsmap learned FSMap version 69 2024-04-24T09:15:39.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.835+0000 7fb603e0e740 1 -- 172.21.15.83:0/76048887 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x558f7883c3f0 con 0x558f78bcfb30 2024-04-24T09:15:39.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.836+0000 7fb5dbe00640 1 -- 172.21.15.83:0/76048887 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb5ec015070 con 0x558f78bcfb30 2024-04-24T09:15:39.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.836+0000 7fb5dbe00640 1 -- 172.21.15.83:0/76048887 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 69) ==== 8370+0+0 (secure 0 0 0) 0x7fb5ec0104d0 con 0x558f78bcfb30 2024-04-24T09:15:39.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.836+0000 7fb603e0e740 10 client.40775 fetch_fsmap finished waiting for FSMap version 69 2024-04-24T09:15:39.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.836+0000 7fb603e0e740 10 client.40775 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-24T09:15:39.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.836+0000 7fb603e0e740 20 client.40775 populate_metadata read hostname 'smithi083' 2024-04-24T09:15:39.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.836+0000 7fb603e0e740 1 --2- 172.21.15.83:0/76048887 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x558f78bdfc50 0x558f78be2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:39.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.836+0000 7fb603e0e740 4 client.40775 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:15:39.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.836+0000 7fb603e0e740 1 -- 172.21.15.83:0/76048887 --> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558f78b37bf0 con 0x558f78bdfc50 2024-04-24T09:15:39.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.837+0000 7fb5f9e00640 1 --2- 172.21.15.83:0/76048887 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x558f78bdfc50 0x558f78be2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:39.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.838+0000 7fb5f9e00640 1 --2- 172.21.15.83:0/76048887 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x558f78bdfc50 0x558f78be2040 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8d3ca11659f7706c server_cookie=3cc12ac668bab27f in_seq=0 out_seq=0 2024-04-24T09:15:39.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.838+0000 7fb5dbe00640 10 client.40775 ms_handle_connect on v2:172.21.15.138:6834/1909219758 2024-04-24T09:15:39.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.839+0000 7fb5dbe00640 1 -- 172.21.15.83:0/76048887 <== mds.2 v2:172.21.15.138:6834/1909219758 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558f78b37bf0 con 0x558f78bdfc50 2024-04-24T09:15:39.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.839+0000 7fb5dbe00640 10 client.40775 handle_command_reply: tid=0 2024-04-24T09:15:39.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.839+0000 7fb603e0e740 1 client.40775 shutdown 2024-04-24T09:15:39.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.839+0000 7fb603e0e740 1 -- 172.21.15.83:0/76048887 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f78bcfb30 msgr2=0x558f78bccfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:39.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.839+0000 7fb603e0e740 1 --2- 172.21.15.83:0/76048887 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f78bcfb30 0x558f78bccfa0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec004290 tx=0x7fb5ec01a6a0 comp rx=0 tx=0).stop 2024-04-24T09:15:39.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.839+0000 7fb603e0e740 1 -- 172.21.15.83:0/76048887 shutdown_connections 2024-04-24T09:15:39.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.839+0000 7fb603e0e740 1 --2- 172.21.15.83:0/76048887 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x558f78bdfc50 0x558f78be2040 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:39.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.840+0000 7fb603e0e740 1 --2- 172.21.15.83:0/76048887 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f78bcfb30 0x558f78bccfa0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:39.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.840+0000 7fb603e0e740 1 --2- 172.21.15.83:0/76048887 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558f78bc5460 0x558f78bcc8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:39.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.840+0000 7fb603e0e740 1 --2- 172.21.15.83:0/76048887 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f78bc4b40 0x558f78bcf5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:39.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.840+0000 7fb603e0e740 1 -- 172.21.15.83:0/76048887 >> 172.21.15.83:0/76048887 conn(0x558f78bc0e60 msgr2=0x558f78bc6c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:39.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.840+0000 7fb5dbe00640 0 client.40775 ms_handle_reset on v2:172.21.15.138:6834/1909219758 2024-04-24T09:15:39.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.840+0000 7fb603e0e740 1 -- 172.21.15.83:0/76048887 shutdown_connections 2024-04-24T09:15:39.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.840+0000 7fb603e0e740 1 -- 172.21.15.83:0/76048887 wait complete. 2024-04-24T09:15:39.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.840+0000 7fb603e0e740 20 client.40775 trim_cache size 0 max 16384 2024-04-24T09:15:39.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.864+0000 7fb603e0e740 1 Processor -- start 2024-04-24T09:15:39.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.864+0000 7fb603e0e740 1 -- start start 2024-04-24T09:15:39.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.864+0000 7fb603e0e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558f78bc4b40 0x558f78bdb880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:39.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.865+0000 7fb603e0e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f78bc5460 0x558f78bcf0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:39.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.865+0000 7fb603e0e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f78bcfb30 0x558f78be7610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:39.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.865+0000 7fb603e0e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb5fc1b1cf0 con 0x558f78bc4b40 2024-04-24T09:15:39.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.865+0000 7fb603e0e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb5fc1b3890 con 0x558f78bcfb30 2024-04-24T09:15:39.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.865+0000 7fb603e0e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb5fc1b23c0 con 0x558f78bc5460 2024-04-24T09:15:39.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.865+0000 7fb5f9e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558f78bc4b40 0x558f78bdb880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:39.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.865+0000 7fb5f9e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558f78bc4b40 0x558f78bdb880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43212/0 (socket says 172.21.15.83:43212) 2024-04-24T09:15:39.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.865+0000 7fb5f9e00640 1 -- 172.21.15.83:0/2165959206 learned_addr learned my addr 172.21.15.83:0/2165959206 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:15:39.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.865+0000 7fb5fa800640 1 --2- 172.21.15.83:0/2165959206 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f78bcfb30 0x558f78be7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:39.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.865+0000 7fb5f9400640 1 --2- 172.21.15.83:0/2165959206 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f78bc5460 0x558f78bcf0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:39.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.866+0000 7fb5fa800640 1 -- 172.21.15.83:0/2165959206 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f78bc5460 msgr2=0x558f78bcf0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:39.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.866+0000 7fb5fa800640 1 --2- 172.21.15.83:0/2165959206 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f78bc5460 0x558f78bcf0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:39.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.866+0000 7fb5fa800640 1 -- 172.21.15.83:0/2165959206 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558f78bc4b40 msgr2=0x558f78bdb880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:39.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.866+0000 7fb5fa800640 1 --2- 172.21.15.83:0/2165959206 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558f78bc4b40 0x558f78bdb880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:39.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.866+0000 7fb5fa800640 1 -- 172.21.15.83:0/2165959206 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f789067d0 con 0x558f78bcfb30 2024-04-24T09:15:39.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.866+0000 7fb5f9e00640 1 --2- 172.21.15.83:0/2165959206 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558f78bc4b40 0x558f78bdb880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:15:39.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.866+0000 7fb5fa800640 1 --2- 172.21.15.83:0/2165959206 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f78bcfb30 0x558f78be7610 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec000c00 tx=0x7fb5ec01ac50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6bebfa0bf32c226a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:39.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.867+0000 7fb5e6a00640 1 -- 172.21.15.83:0/2165959206 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5ec028040 con 0x558f78bcfb30 2024-04-24T09:15:39.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.867+0000 7fb5e6a00640 1 -- 172.21.15.83:0/2165959206 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb5ec015070 con 0x558f78bcfb30 2024-04-24T09:15:39.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.867+0000 7fb5e6a00640 1 -- 172.21.15.83:0/2165959206 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5ec003f10 con 0x558f78bcfb30 2024-04-24T09:15:39.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.868+0000 7fb603e0e740 1 -- 172.21.15.83:0/2165959206 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f78bcfb30 msgr2=0x558f78be7610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:39.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.868+0000 7fb603e0e740 1 --2- 172.21.15.83:0/2165959206 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f78bcfb30 0x558f78be7610 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec000c00 tx=0x7fb5ec01ac50 comp rx=0 tx=0).stop 2024-04-24T09:15:39.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.868+0000 7fb603e0e740 1 -- 172.21.15.83:0/2165959206 shutdown_connections 2024-04-24T09:15:39.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.868+0000 7fb603e0e740 1 --2- 172.21.15.83:0/2165959206 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f78bcfb30 0x558f78be7610 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:39.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.868+0000 7fb603e0e740 1 --2- 172.21.15.83:0/2165959206 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f78bc5460 0x558f78bcf0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:39.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.868+0000 7fb603e0e740 1 --2- 172.21.15.83:0/2165959206 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558f78bc4b40 0x558f78bdb880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:39.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.868+0000 7fb603e0e740 1 -- 172.21.15.83:0/2165959206 >> 172.21.15.83:0/2165959206 conn(0x558f78bc0e60 msgr2=0x558f78bd1e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:39.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.868+0000 7fb603e0e740 1 -- 172.21.15.83:0/2165959206 shutdown_connections 2024-04-24T09:15:39.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.869+0000 7fb603e0e740 1 -- 172.21.15.83:0/2165959206 wait complete. 2024-04-24T09:15:39.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.869+0000 7fb603e0e740 1 -- start start 2024-04-24T09:15:39.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.869+0000 7fb603e0e740 1 Processor -- start 2024-04-24T09:15:39.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.870+0000 7fb603e0e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f78bc4b40 0x558f78bd81c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:39.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.870+0000 7fb603e0e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f78bc5460 0x558f78bc92e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:39.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.870+0000 7fb603e0e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558f78bcfb30 0x558f78bc98a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:39.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.870+0000 7fb603e0e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb5fc1b31c0 con 0x558f78bcfb30 2024-04-24T09:15:39.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.870+0000 7fb603e0e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb5fc1b2af0 con 0x558f78bc5460 2024-04-24T09:15:39.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.870+0000 7fb603e0e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb5fc1afcf0 con 0x558f78bc4b40 2024-04-24T09:15:39.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.870+0000 7fb603e0e740 10 client.0 _ll_register_callbacks cb 0x7fb5ec004510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:15:39.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.870+0000 7fb5f9400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f78bc5460 0x558f78bc92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:39.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.870+0000 7fb5f9400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f78bc5460 0x558f78bc92e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:58580/0 (socket says 172.21.15.83:58580) 2024-04-24T09:15:39.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.870+0000 7fb5f9400640 1 -- 172.21.15.83:0/1048860784 learned_addr learned my addr 172.21.15.83:0/1048860784 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:15:39.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.870+0000 7fb5fa800640 1 --2- 172.21.15.83:0/1048860784 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558f78bcfb30 0x558f78bc98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:39.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.871+0000 7fb5f9e00640 1 --2- 172.21.15.83:0/1048860784 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f78bc4b40 0x558f78bd81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:39.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.871+0000 7fb5fa800640 1 -- 172.21.15.83:0/1048860784 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f78bc4b40 msgr2=0x558f78bd81c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:39.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.871+0000 7fb5fa800640 1 --2- 172.21.15.83:0/1048860784 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f78bc4b40 0x558f78bd81c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:39.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.871+0000 7fb5fa800640 1 -- 172.21.15.83:0/1048860784 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f78bc5460 msgr2=0x558f78bc92e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:39.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.871+0000 7fb5fa800640 1 --2- 172.21.15.83:0/1048860784 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f78bc5460 0x558f78bc92e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:39.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.871+0000 7fb5fa800640 1 -- 172.21.15.83:0/1048860784 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f78bca0d0 con 0x558f78bcfb30 2024-04-24T09:15:39.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.871+0000 7fb5f9400640 1 --2- 172.21.15.83:0/1048860784 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f78bc5460 0x558f78bc92e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:15:39.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.871+0000 7fb5f9e00640 1 --2- 172.21.15.83:0/1048860784 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f78bc4b40 0x558f78bd81c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:15:39.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.872+0000 7fb5fa800640 1 --2- 172.21.15.83:0/1048860784 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558f78bcfb30 0x558f78bc98a0 secure :-1 s=READY pgs=2733 cs=0 l=1 rev1=1 crypto rx=0x558f78bbcad0 tx=0x7fb5ec0055d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd441a872f462310 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:39.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.872+0000 7fb5dbe00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:15:39.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.872+0000 7fb5dbe00640 1 -- 172.21.15.83:0/1048860784 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5ec028070 con 0x558f78bcfb30 2024-04-24T09:15:39.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.872+0000 7fb5dbe00640 1 -- 172.21.15.83:0/1048860784 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(130..130 src has 1..130) ==== 9583+0+0 (secure 0 0 0) 0x7fb5ec003920 con 0x558f78bcfb30 2024-04-24T09:15:39.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.872+0000 7fb603e0e740 1 -- 172.21.15.83:0/1048860784 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5ec015070 con 0x558f78bcfb30 2024-04-24T09:15:39.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.872+0000 7fb5dbe00640 1 -- 172.21.15.83:0/1048860784 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7fb5ec043050 con 0x558f78bcfb30 2024-04-24T09:15:39.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.872+0000 7fb603e0e740 10 client.23721 fetch_fsmap learned FSMap version 69 2024-04-24T09:15:39.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.872+0000 7fb603e0e740 1 -- 172.21.15.83:0/1048860784 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x558f78c4e780 con 0x558f78bcfb30 2024-04-24T09:15:39.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.873+0000 7fb5dbe00640 1 -- 172.21.15.83:0/1048860784 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb5ec015070 con 0x558f78bcfb30 2024-04-24T09:15:39.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.873+0000 7fb5dbe00640 1 -- 172.21.15.83:0/1048860784 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 69) ==== 8370+0+0 (secure 0 0 0) 0x7fb5ec011a80 con 0x558f78bcfb30 2024-04-24T09:15:39.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.873+0000 7fb603e0e740 10 client.23721 fetch_fsmap finished waiting for FSMap version 69 2024-04-24T09:15:39.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.873+0000 7fb603e0e740 10 client.23721 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-24T09:15:39.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.873+0000 7fb603e0e740 20 client.23721 populate_metadata read hostname 'smithi083' 2024-04-24T09:15:39.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.873+0000 7fb603e0e740 1 --2- 172.21.15.83:0/1048860784 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x558f78c4ede0 0x558f78c511d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:39.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.873+0000 7fb603e0e740 4 client.23721 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:15:39.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.873+0000 7fb603e0e740 1 -- 172.21.15.83:0/1048860784 --> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558f78b31a60 con 0x558f78c4ede0 2024-04-24T09:15:39.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.874+0000 7fb5f9e00640 1 --2- 172.21.15.83:0/1048860784 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x558f78c4ede0 0x558f78c511d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:39.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.874+0000 7fb5f9e00640 1 --2- 172.21.15.83:0/1048860784 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x558f78c4ede0 0x558f78c511d0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7616d3c9d0fb57c4 server_cookie=7e3a0d91b1bb6bed in_seq=0 out_seq=0 2024-04-24T09:15:39.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.874+0000 7fb5dbe00640 10 client.23721 ms_handle_connect on v2:172.21.15.138:6834/1909219758 2024-04-24T09:15:39.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.875+0000 7fb5dbe00640 1 -- 172.21.15.83:0/1048860784 <== mds.2 v2:172.21.15.138:6834/1909219758 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558f78b31a60 con 0x558f78c4ede0 2024-04-24T09:15:39.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.875+0000 7fb5dbe00640 10 client.23721 handle_command_reply: tid=0 2024-04-24T09:15:39.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.875+0000 7fb603e0e740 1 client.23721 shutdown 2024-04-24T09:15:39.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.875+0000 7fb603e0e740 1 -- 172.21.15.83:0/1048860784 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558f78bcfb30 msgr2=0x558f78bc98a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:39.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.875+0000 7fb603e0e740 1 --2- 172.21.15.83:0/1048860784 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558f78bcfb30 0x558f78bc98a0 secure :-1 s=READY pgs=2733 cs=0 l=1 rev1=1 crypto rx=0x558f78bbcad0 tx=0x7fb5ec0055d0 comp rx=0 tx=0).stop 2024-04-24T09:15:39.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.875+0000 7fb603e0e740 1 -- 172.21.15.83:0/1048860784 shutdown_connections 2024-04-24T09:15:39.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.875+0000 7fb603e0e740 1 --2- 172.21.15.83:0/1048860784 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x558f78c4ede0 0x558f78c511d0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:39.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.875+0000 7fb603e0e740 1 --2- 172.21.15.83:0/1048860784 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558f78bcfb30 0x558f78bc98a0 unknown :-1 s=CLOSED pgs=2733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:39.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.875+0000 7fb603e0e740 1 --2- 172.21.15.83:0/1048860784 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f78bc5460 0x558f78bc92e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:39.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.875+0000 7fb5dbe00640 0 client.23721 ms_handle_reset on v2:172.21.15.138:6834/1909219758 2024-04-24T09:15:39.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.875+0000 7fb603e0e740 1 --2- 172.21.15.83:0/1048860784 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f78bc4b40 0x558f78bd81c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:39.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.875+0000 7fb603e0e740 1 -- 172.21.15.83:0/1048860784 >> 172.21.15.83:0/1048860784 conn(0x558f78bc0e60 msgr2=0x558f78bd1f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:39.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.875+0000 7fb603e0e740 1 -- 172.21.15.83:0/1048860784 shutdown_connections 2024-04-24T09:15:39.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.875+0000 7fb603e0e740 1 -- 172.21.15.83:0/1048860784 wait complete. 2024-04-24T09:15:39.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.876+0000 7fb603e0e740 20 client.23721 trim_cache size 0 max 16384 2024-04-24T09:15:39.880 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:15:39.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.876+0000 7fb5fb200640 1 -- 172.21.15.83:0/4066847217 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb5e00a3d20 msgr2=0x7fb5e00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:39.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.877+0000 7fb5fb200640 1 --2- 172.21.15.83:0/4066847217 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb5e00a3d20 0x7fb5e00a61e0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x558f78bc6100 tx=0x7fb5f4009940 comp rx=0 tx=0).stop 2024-04-24T09:15:39.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.877+0000 7fb5fb200640 1 -- 172.21.15.83:0/4066847217 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5fc1a0d10 msgr2=0x7fb5fc19a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:39.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.877+0000 7fb5fb200640 1 --2- 172.21.15.83:0/4066847217 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5fc1a0d10 0x7fb5fc19a9d0 secure :-1 s=READY pgs=2732 cs=0 l=1 rev1=1 crypto rx=0x7fb5e800d6a0 tx=0x7fb5e800db60 comp rx=0 tx=0).stop 2024-04-24T09:15:39.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.877+0000 7fb5fb200640 1 -- 172.21.15.83:0/4066847217 shutdown_connections 2024-04-24T09:15:39.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.877+0000 7fb5fb200640 1 --2- 172.21.15.83:0/4066847217 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb5e00a3d20 0x7fb5e00a61e0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:39.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.877+0000 7fb5fb200640 1 --2- 172.21.15.83:0/4066847217 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb5fc19af10 0x7fb5fc19b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:39.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.877+0000 7fb5fb200640 1 --2- 172.21.15.83:0/4066847217 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5fc1a0d10 0x7fb5fc19a9d0 unknown :-1 s=CLOSED pgs=2732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:39.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.877+0000 7fb5fb200640 1 --2- 172.21.15.83:0/4066847217 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5fc1093f0 0x7fb5fc1a07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:39.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.877+0000 7fb5fb200640 1 -- 172.21.15.83:0/4066847217 >> 172.21.15.83:0/4066847217 conn(0x7fb5fc0ffb40 msgr2=0x7fb5fc100f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:39.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.877+0000 7fb5fb200640 1 -- 172.21.15.83:0/4066847217 shutdown_connections 2024-04-24T09:15:39.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:39.877+0000 7fb5fb200640 1 -- 172.21.15.83:0/4066847217 wait complete. 2024-04-24T09:15:39.899 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T09:15:40.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.182+0000 7f5f9aa00640 1 -- 172.21.15.83:0/1203876625 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5f9410a9a0 msgr2=0x7f5f9410ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:40.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.182+0000 7f5f9aa00640 1 --2- 172.21.15.83:0/1203876625 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5f9410a9a0 0x7f5f9410ad80 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c009c10 tx=0x7f5f8c02a6e0 comp rx=0 tx=0).stop 2024-04-24T09:15:40.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.183+0000 7f5f9aa00640 1 -- 172.21.15.83:0/1203876625 shutdown_connections 2024-04-24T09:15:40.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.183+0000 7f5f9aa00640 1 --2- 172.21.15.83:0/1203876625 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f94113f50 0x7f5f94116340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:40.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.183+0000 7f5f9aa00640 1 --2- 172.21.15.83:0/1203876625 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f9410b2c0 0x7f5f94113a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:40.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.183+0000 7f5f9aa00640 1 --2- 172.21.15.83:0/1203876625 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5f9410a9a0 0x7f5f9410ad80 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:40.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.183+0000 7f5f9aa00640 1 -- 172.21.15.83:0/1203876625 >> 172.21.15.83:0/1203876625 conn(0x7f5f9406d3e0 msgr2=0x7f5f9406d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:40.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.184+0000 7f5f9aa00640 1 -- 172.21.15.83:0/1203876625 shutdown_connections 2024-04-24T09:15:40.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.184+0000 7f5f9aa00640 1 -- 172.21.15.83:0/1203876625 wait complete. 2024-04-24T09:15:40.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.185+0000 7f5f9aa00640 1 Processor -- start 2024-04-24T09:15:40.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.187+0000 7f5f9aa00640 1 -- start start 2024-04-24T09:15:40.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.188+0000 7f5f9aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f9410a9a0 0x7f5f9419edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:40.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.188+0000 7f5f9aa00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5f9410b2c0 0x7f5f9419f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:40.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.188+0000 7f5f9aa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f94113f50 0x7f5f941a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:40.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.188+0000 7f5f9aa00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5f94118870 con 0x7f5f9410a9a0 2024-04-24T09:15:40.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.188+0000 7f5f9aa00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5f941186f0 con 0x7f5f94113f50 2024-04-24T09:15:40.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.188+0000 7f5f9aa00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5f941189f0 con 0x7f5f9410b2c0 2024-04-24T09:15:40.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.189+0000 7f5f98c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5f9410b2c0 0x7f5f9419f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:40.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.189+0000 7f5f98c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5f9410b2c0 0x7f5f9419f2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:33992/0 (socket says 172.21.15.83:33992) 2024-04-24T09:15:40.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.189+0000 7f5f98c00640 1 -- 172.21.15.83:0/1251042928 learned_addr learned my addr 172.21.15.83:0/1251042928 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:15:40.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.189+0000 7f5f99600640 1 --2- 172.21.15.83:0/1251042928 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f9410a9a0 0x7f5f9419edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:40.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.189+0000 7f5f9a000640 1 --2- 172.21.15.83:0/1251042928 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f94113f50 0x7f5f941a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:40.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.189+0000 7f5f9a000640 1 -- 172.21.15.83:0/1251042928 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5f9410b2c0 msgr2=0x7f5f9419f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:40.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.189+0000 7f5f9a000640 1 --2- 172.21.15.83:0/1251042928 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5f9410b2c0 0x7f5f9419f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:40.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.189+0000 7f5f9a000640 1 -- 172.21.15.83:0/1251042928 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f9410a9a0 msgr2=0x7f5f9419edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:40.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.189+0000 7f5f9a000640 1 --2- 172.21.15.83:0/1251042928 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f9410a9a0 0x7f5f9419edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:40.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.189+0000 7f5f9a000640 1 -- 172.21.15.83:0/1251042928 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f941a3da0 con 0x7f5f94113f50 2024-04-24T09:15:40.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.191+0000 7f5f9a000640 1 --2- 172.21.15.83:0/1251042928 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f94113f50 0x7f5f941a3670 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f5f8800a840 tx=0x7f5f8800ad00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:40.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.192+0000 7f5f92000640 1 -- 172.21.15.83:0/1251042928 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f880079b0 con 0x7f5f94113f50 2024-04-24T09:15:40.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.192+0000 7f5f92000640 1 -- 172.21.15.83:0/1251042928 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5f88005020 con 0x7f5f94113f50 2024-04-24T09:15:40.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.192+0000 7f5f92000640 1 -- 172.21.15.83:0/1251042928 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f88004ba0 con 0x7f5f94113f50 2024-04-24T09:15:40.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.192+0000 7f5f9aa00640 1 -- 172.21.15.83:0/1251042928 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f941a4090 con 0x7f5f94113f50 2024-04-24T09:15:40.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.193+0000 7f5f9aa00640 1 -- 172.21.15.83:0/1251042928 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f941ab920 con 0x7f5f94113f50 2024-04-24T09:15:40.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.201+0000 7f5f9eded740 1 Processor -- start 2024-04-24T09:15:40.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.203+0000 7f5f92000640 1 -- 172.21.15.83:0/1251042928 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5f880051d0 con 0x7f5f94113f50 2024-04-24T09:15:40.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.205+0000 7f5f9eded740 1 -- start start 2024-04-24T09:15:40.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.205+0000 7f5f9eded740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5640cc57f350 0x5640cc57f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:40.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.205+0000 7f5f9eded740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640cc57fc70 0x5640cc589e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:40.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.205+0000 7f5f9eded740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5640cc58a340 0x5640cc58c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:40.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.205+0000 7f5f9eded740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5640cc4dc1d0 con 0x5640cc58a340 2024-04-24T09:15:40.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.205+0000 7f5f9eded740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5640cc482450 con 0x5640cc57fc70 2024-04-24T09:15:40.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.205+0000 7f5f9eded740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5640cc4d8190 con 0x5640cc57f350 2024-04-24T09:15:40.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.206+0000 7f5f92000640 1 --2- 172.21.15.83:0/1251042928 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f5f7c0a3e60 0x7f5f7c0a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:40.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.206+0000 7f5f99600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640cc57fc70 0x5640cc589e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:40.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.206+0000 7f5f99600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640cc57fc70 0x5640cc589e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:58606/0 (socket says 172.21.15.83:58606) 2024-04-24T09:15:40.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.206+0000 7f5f99600640 1 -- 172.21.15.83:0/514774340 learned_addr learned my addr 172.21.15.83:0/514774340 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:15:40.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.207+0000 7f5f92000640 1 -- 172.21.15.83:0/1251042928 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 9583+0+0 (secure 0 0 0) 0x7f5f880d0020 con 0x7f5f94113f50 2024-04-24T09:15:40.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.208+0000 7f5f99600640 1 -- 172.21.15.83:0/514774340 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5640cc57f350 msgr2=0x5640cc57f730 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-24T09:15:40.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.208+0000 7f5f99600640 1 --2- 172.21.15.83:0/514774340 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5640cc57f350 0x5640cc57f730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:40.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.208+0000 7f5f99600640 1 -- 172.21.15.83:0/514774340 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5640cc58a340 msgr2=0x5640cc58c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:40.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.208+0000 7f5f98c00640 1 --2- 172.21.15.83:0/514774340 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5640cc58a340 0x5640cc58c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:40.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.208+0000 7f5f99600640 1 --2- 172.21.15.83:0/514774340 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5640cc58a340 0x5640cc58c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:40.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.208+0000 7f5f99600640 1 -- 172.21.15.83:0/514774340 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640cc53f310 con 0x5640cc57fc70 2024-04-24T09:15:40.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.208+0000 7f5f98c00640 1 --2- 172.21.15.83:0/514774340 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5640cc58a340 0x5640cc58c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:15:40.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.209+0000 7f5f9a000640 1 --2- 172.21.15.83:0/1251042928 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f5f7c0a3e60 0x7f5f7c0a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:40.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.210+0000 7f5f99600640 1 --2- 172.21.15.83:0/514774340 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640cc57fc70 0x5640cc589e00 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c02acd0 tx=0x7f5f8c031530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59b2b80ae1d836c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:40.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.210+0000 7f5f9a000640 1 --2- 172.21.15.83:0/1251042928 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f5f7c0a3e60 0x7f5f7c0a6320 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f5f88007590 tx=0x7f5f880056e0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:40.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.211+0000 7f5f86600640 1 -- 172.21.15.83:0/514774340 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f8c041030 con 0x5640cc57fc70 2024-04-24T09:15:40.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.211+0000 7f5f86600640 1 -- 172.21.15.83:0/514774340 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5f8c037040 con 0x5640cc57fc70 2024-04-24T09:15:40.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.211+0000 7f5f86600640 1 -- 172.21.15.83:0/514774340 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f8c036040 con 0x5640cc57fc70 2024-04-24T09:15:40.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.212+0000 7f5f9eded740 1 -- 172.21.15.83:0/514774340 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640cc57fc70 msgr2=0x5640cc589e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:40.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.212+0000 7f5f9eded740 1 --2- 172.21.15.83:0/514774340 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640cc57fc70 0x5640cc589e00 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c02acd0 tx=0x7f5f8c031530 comp rx=0 tx=0).stop 2024-04-24T09:15:40.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.214+0000 7f5f9eded740 1 -- 172.21.15.83:0/514774340 shutdown_connections 2024-04-24T09:15:40.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.214+0000 7f5f9eded740 1 --2- 172.21.15.83:0/514774340 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5640cc58a340 0x5640cc58c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:40.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.214+0000 7f5f9eded740 1 --2- 172.21.15.83:0/514774340 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640cc57fc70 0x5640cc589e00 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:40.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.214+0000 7f5f9eded740 1 --2- 172.21.15.83:0/514774340 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5640cc57f350 0x5640cc57f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:40.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.214+0000 7f5f9eded740 1 -- 172.21.15.83:0/514774340 >> 172.21.15.83:0/514774340 conn(0x5640cc57b670 msgr2=0x5640cc57ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:40.214 INFO:tasks.workunit.client.0.smithi083.stdout: 10 75079 6.56 MB/sec execute 491 sec latency 111.366 ms 2024-04-24T09:15:40.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.214+0000 7f5f9eded740 1 -- 172.21.15.83:0/514774340 shutdown_connections 2024-04-24T09:15:40.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.214+0000 7f5f9eded740 1 -- 172.21.15.83:0/514774340 wait complete. 2024-04-24T09:15:40.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.214+0000 7f5f9eded740 1 -- start start 2024-04-24T09:15:40.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.214+0000 7f5f9eded740 1 Processor -- start 2024-04-24T09:15:40.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.215+0000 7f5f9eded740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5640cc57f350 0x5640cc587180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:40.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.215+0000 7f5f9eded740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640cc57fc70 0x5640cc5876c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:40.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.215+0000 7f5f9eded740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5640cc58a340 0x5640cc587c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:40.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.215+0000 7f5f9eded740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5640cc58d170 con 0x5640cc58a340 2024-04-24T09:15:40.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.215+0000 7f5f9eded740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5640cc495810 con 0x5640cc57fc70 2024-04-24T09:15:40.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.215+0000 7f5f9eded740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5640cc58d2f0 con 0x5640cc57f350 2024-04-24T09:15:40.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.215+0000 7f5f9eded740 10 client.0 _ll_register_callbacks cb 0x5640cc463140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:15:40.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.216+0000 7f5f99600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640cc57fc70 0x5640cc5876c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:40.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.216+0000 7f5f99600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640cc57fc70 0x5640cc5876c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:58614/0 (socket says 172.21.15.83:58614) 2024-04-24T09:15:40.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.216+0000 7f5f98c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5640cc57f350 0x5640cc587180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:40.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.216+0000 7f5f99600640 1 -- 172.21.15.83:0/3618081199 learned_addr learned my addr 172.21.15.83:0/3618081199 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:15:40.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.216+0000 7f5f98c00640 1 --2- 172.21.15.83:0/3618081199 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5640cc58a340 0x5640cc587c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:40.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.217+0000 7f5f98c00640 1 -- 172.21.15.83:0/3618081199 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5640cc57f350 msgr2=0x5640cc587180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:40.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.217+0000 7f5f98c00640 1 --2- 172.21.15.83:0/3618081199 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5640cc57f350 0x5640cc587180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:40.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.217+0000 7f5f98c00640 1 -- 172.21.15.83:0/3618081199 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640cc57fc70 msgr2=0x5640cc5876c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:40.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.217+0000 7f5f98c00640 1 --2- 172.21.15.83:0/3618081199 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640cc57fc70 0x5640cc5876c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:40.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.217+0000 7f5f98c00640 1 -- 172.21.15.83:0/3618081199 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640cc4a63f0 con 0x5640cc58a340 2024-04-24T09:15:40.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.217+0000 7f5f98c00640 1 --2- 172.21.15.83:0/3618081199 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5640cc58a340 0x5640cc587c00 secure :-1 s=READY pgs=2734 cs=0 l=1 rev1=1 crypto rx=0x7f5f80005ab0 tx=0x7f5f80009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc761aaae9cc04ef server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:40.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.217+0000 7f5f7aa00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:15:40.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.217+0000 7f5f7aa00640 1 -- 172.21.15.83:0/3618081199 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f8000ca70 con 0x5640cc58a340 2024-04-24T09:15:40.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.217+0000 7f5f7aa00640 1 -- 172.21.15.83:0/3618081199 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(130..130 src has 1..130) ==== 9583+0+0 (secure 0 0 0) 0x7f5f8000cc10 con 0x5640cc58a340 2024-04-24T09:15:40.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.218+0000 7f5f9eded740 1 -- 172.21.15.83:0/3618081199 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f8c037040 con 0x5640cc58a340 2024-04-24T09:15:40.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.218+0000 7f5f7aa00640 1 -- 172.21.15.83:0/3618081199 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f5f80027050 con 0x5640cc58a340 2024-04-24T09:15:40.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.218+0000 7f5f9eded740 10 client.23730 fetch_fsmap learned FSMap version 69 2024-04-24T09:15:40.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.218+0000 7f5f9eded740 1 -- 172.21.15.83:0/3618081199 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x5640cc1ee3f0 con 0x5640cc58a340 2024-04-24T09:15:40.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.218+0000 7f5f7aa00640 1 -- 172.21.15.83:0/3618081199 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5f8c037040 con 0x5640cc58a340 2024-04-24T09:15:40.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.219+0000 7f5f7aa00640 1 -- 172.21.15.83:0/3618081199 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 69) ==== 8370+0+0 (secure 0 0 0) 0x7f5f80003b50 con 0x5640cc58a340 2024-04-24T09:15:40.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.219+0000 7f5f9eded740 10 client.23730 fetch_fsmap finished waiting for FSMap version 69 2024-04-24T09:15:40.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.219+0000 7f5f9eded740 10 client.23730 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-24T09:15:40.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.219+0000 7f5f9eded740 20 client.23730 populate_metadata read hostname 'smithi083' 2024-04-24T09:15:40.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.219+0000 7f5f9eded740 1 --2- 172.21.15.83:0/3618081199 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x5640cc59a4d0 0x5640cc59c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:40.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.219+0000 7f5f9eded740 4 client.23730 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:15:40.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.219+0000 7f5f9eded740 1 -- 172.21.15.83:0/3618081199 --> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5640cc4e9bf0 con 0x5640cc59a4d0 2024-04-24T09:15:40.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.220+0000 7f5f99600640 1 --2- 172.21.15.83:0/3618081199 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x5640cc59a4d0 0x5640cc59c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:40.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.221+0000 7f5f99600640 1 --2- 172.21.15.83:0/3618081199 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x5640cc59a4d0 0x5640cc59c8f0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=91387fade3ad52cd server_cookie=254b3adcc6de82ca in_seq=0 out_seq=0 2024-04-24T09:15:40.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.221+0000 7f5f7aa00640 10 client.23730 ms_handle_connect on v2:172.21.15.138:6836/900566754 2024-04-24T09:15:40.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.223+0000 7f5f7aa00640 1 -- 172.21.15.83:0/3618081199 <== mds.1 v2:172.21.15.138:6836/900566754 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5640cc4e9bf0 con 0x5640cc59a4d0 2024-04-24T09:15:40.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.223+0000 7f5f7aa00640 10 client.23730 handle_command_reply: tid=0 2024-04-24T09:15:40.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.223+0000 7f5f9eded740 1 client.23730 shutdown 2024-04-24T09:15:40.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.224+0000 7f5f9eded740 1 -- 172.21.15.83:0/3618081199 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5640cc58a340 msgr2=0x5640cc587c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:40.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.224+0000 7f5f9eded740 1 --2- 172.21.15.83:0/3618081199 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5640cc58a340 0x5640cc587c00 secure :-1 s=READY pgs=2734 cs=0 l=1 rev1=1 crypto rx=0x7f5f80005ab0 tx=0x7f5f80009260 comp rx=0 tx=0).stop 2024-04-24T09:15:40.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.224+0000 7f5f9eded740 1 -- 172.21.15.83:0/3618081199 shutdown_connections 2024-04-24T09:15:40.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.224+0000 7f5f9eded740 1 --2- 172.21.15.83:0/3618081199 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5640cc58a340 0x5640cc587c00 unknown :-1 s=CLOSED pgs=2734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:40.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.224+0000 7f5f9eded740 1 --2- 172.21.15.83:0/3618081199 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640cc57fc70 0x5640cc5876c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:40.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.224+0000 7f5f9eded740 1 --2- 172.21.15.83:0/3618081199 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x5640cc59a4d0 0x5640cc59c8f0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:40.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.224+0000 7f5f9eded740 1 --2- 172.21.15.83:0/3618081199 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5640cc57f350 0x5640cc587180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:40.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.224+0000 7f5f7aa00640 0 client.23730 ms_handle_reset on v2:172.21.15.138:6836/900566754 2024-04-24T09:15:40.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.224+0000 7f5f9eded740 1 -- 172.21.15.83:0/3618081199 >> 172.21.15.83:0/3618081199 conn(0x5640cc57b670 msgr2=0x5640cc581470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:40.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.224+0000 7f5f9eded740 1 -- 172.21.15.83:0/3618081199 shutdown_connections 2024-04-24T09:15:40.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.224+0000 7f5f9eded740 1 -- 172.21.15.83:0/3618081199 wait complete. 2024-04-24T09:15:40.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.225+0000 7f5f9eded740 20 client.23730 trim_cache size 0 max 16384 2024-04-24T09:15:40.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.249+0000 7f5f9eded740 1 Processor -- start 2024-04-24T09:15:40.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.249+0000 7f5f9eded740 1 -- start start 2024-04-24T09:15:40.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.249+0000 7f5f9eded740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5640cc57f350 0x5640cc596030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:40.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.249+0000 7f5f9eded740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640cc57fc70 0x5640cc587a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:40.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.249+0000 7f5f9eded740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5640cc58a340 0x5640cc5811f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:40.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.249+0000 7f5f9eded740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5f941b5a50 con 0x5640cc57f350 2024-04-24T09:15:40.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.249+0000 7f5f9eded740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5f941b75f0 con 0x5640cc57fc70 2024-04-24T09:15:40.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.249+0000 7f5f9eded740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5f941b6120 con 0x5640cc58a340 2024-04-24T09:15:40.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.250+0000 7f5f98c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5640cc57f350 0x5640cc596030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:40.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.250+0000 7f5f98c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5640cc57f350 0x5640cc596030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43294/0 (socket says 172.21.15.83:43294) 2024-04-24T09:15:40.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.250+0000 7f5f98c00640 1 -- 172.21.15.83:0/1689057291 learned_addr learned my addr 172.21.15.83:0/1689057291 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:15:40.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.250+0000 7f5f99600640 1 --2- 172.21.15.83:0/1689057291 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640cc57fc70 0x5640cc587a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:40.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.250+0000 7f5f98c00640 1 -- 172.21.15.83:0/1689057291 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5640cc58a340 msgr2=0x5640cc5811f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:40.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.250+0000 7f5f98c00640 1 --2- 172.21.15.83:0/1689057291 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5640cc58a340 0x5640cc5811f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:40.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.250+0000 7f5f98c00640 1 -- 172.21.15.83:0/1689057291 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640cc57fc70 msgr2=0x5640cc587a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:40.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.250+0000 7f5f98c00640 1 --2- 172.21.15.83:0/1689057291 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640cc57fc70 0x5640cc587a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:40.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.250+0000 7f5f98c00640 1 -- 172.21.15.83:0/1689057291 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640cc2b87d0 con 0x5640cc57f350 2024-04-24T09:15:40.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.250+0000 7f5f98c00640 1 --2- 172.21.15.83:0/1689057291 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5640cc57f350 0x5640cc596030 secure :-1 s=READY pgs=2735 cs=0 l=1 rev1=1 crypto rx=0x7f5f80005940 tx=0x7f5f80009690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc5bcc31ec096ada server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:40.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.251+0000 7f5f85c00640 1 -- 172.21.15.83:0/1689057291 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f80009710 con 0x5640cc57f350 2024-04-24T09:15:40.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.251+0000 7f5f85c00640 1 -- 172.21.15.83:0/1689057291 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5f8000f680 con 0x5640cc57f350 2024-04-24T09:15:40.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.251+0000 7f5f85c00640 1 -- 172.21.15.83:0/1689057291 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f800052d0 con 0x5640cc57f350 2024-04-24T09:15:40.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.253+0000 7f5f9eded740 1 -- 172.21.15.83:0/1689057291 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5640cc57f350 msgr2=0x5640cc596030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:40.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.253+0000 7f5f9eded740 1 --2- 172.21.15.83:0/1689057291 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5640cc57f350 0x5640cc596030 secure :-1 s=READY pgs=2735 cs=0 l=1 rev1=1 crypto rx=0x7f5f80005940 tx=0x7f5f80009690 comp rx=0 tx=0).stop 2024-04-24T09:15:40.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.253+0000 7f5f9eded740 1 -- 172.21.15.83:0/1689057291 shutdown_connections 2024-04-24T09:15:40.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.253+0000 7f5f9eded740 1 --2- 172.21.15.83:0/1689057291 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5640cc58a340 0x5640cc5811f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:40.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.253+0000 7f5f9eded740 1 --2- 172.21.15.83:0/1689057291 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640cc57fc70 0x5640cc587a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:40.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.253+0000 7f5f9eded740 1 --2- 172.21.15.83:0/1689057291 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5640cc57f350 0x5640cc596030 unknown :-1 s=CLOSED pgs=2735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:40.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.253+0000 7f5f9eded740 1 -- 172.21.15.83:0/1689057291 >> 172.21.15.83:0/1689057291 conn(0x5640cc57b670 msgr2=0x5640cc580ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:40.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.253+0000 7f5f9eded740 1 -- 172.21.15.83:0/1689057291 shutdown_connections 2024-04-24T09:15:40.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.253+0000 7f5f9eded740 1 -- 172.21.15.83:0/1689057291 wait complete. 2024-04-24T09:15:40.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.253+0000 7f5f9eded740 1 -- start start 2024-04-24T09:15:40.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.253+0000 7f5f9eded740 1 Processor -- start 2024-04-24T09:15:40.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.254+0000 7f5f9eded740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5640cc57f350 0x5640cc592540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:40.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.254+0000 7f5f9eded740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5640cc57fc70 0x5640cc592a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:40.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.254+0000 7f5f9eded740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640cc58a340 0x5640cc5833f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:40.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.254+0000 7f5f9eded740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5f941b6f20 con 0x5640cc57f350 2024-04-24T09:15:40.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.254+0000 7f5f9eded740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5f941b6850 con 0x5640cc58a340 2024-04-24T09:15:40.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.254+0000 7f5f9eded740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5f941b3a50 con 0x5640cc57fc70 2024-04-24T09:15:40.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.254+0000 7f5f9eded740 10 client.0 _ll_register_callbacks cb 0x7f5f80009a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:15:40.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.254+0000 7f5f98c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5640cc57f350 0x5640cc592540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:40.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.254+0000 7f5f98c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5640cc57f350 0x5640cc592540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43298/0 (socket says 172.21.15.83:43298) 2024-04-24T09:15:40.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.254+0000 7f5f98c00640 1 -- 172.21.15.83:0/4280902322 learned_addr learned my addr 172.21.15.83:0/4280902322 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:15:40.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.254+0000 7f5f99600640 1 --2- 172.21.15.83:0/4280902322 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5640cc57fc70 0x5640cc592a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:40.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.255+0000 7f5f98c00640 1 --2- 172.21.15.83:0/4280902322 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640cc58a340 0x5640cc5833f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:40.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.255+0000 7f5f98c00640 1 -- 172.21.15.83:0/4280902322 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5640cc57fc70 msgr2=0x5640cc592a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:40.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.255+0000 7f5f98c00640 1 --2- 172.21.15.83:0/4280902322 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5640cc57fc70 0x5640cc592a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:40.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.255+0000 7f5f98c00640 1 -- 172.21.15.83:0/4280902322 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640cc58a340 msgr2=0x5640cc5833f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:40.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.255+0000 7f5f98c00640 1 --2- 172.21.15.83:0/4280902322 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640cc58a340 0x5640cc5833f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:40.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.255+0000 7f5f98c00640 1 -- 172.21.15.83:0/4280902322 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640cc583930 con 0x5640cc57f350 2024-04-24T09:15:40.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.255+0000 7f5f98c00640 1 --2- 172.21.15.83:0/4280902322 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5640cc57f350 0x5640cc592540 secure :-1 s=READY pgs=2736 cs=0 l=1 rev1=1 crypto rx=0x7f5f80009cc0 tx=0x7f5f8000fe30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e1fb1190ccab56e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:40.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.255+0000 7f5f7aa00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:15:40.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.256+0000 7f5f7aa00640 1 -- 172.21.15.83:0/4280902322 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f8000fb90 con 0x5640cc57f350 2024-04-24T09:15:40.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.256+0000 7f5f7aa00640 1 -- 172.21.15.83:0/4280902322 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(130..130 src has 1..130) ==== 9583+0+0 (secure 0 0 0) 0x7f5f8000ca70 con 0x5640cc57f350 2024-04-24T09:15:40.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.256+0000 7f5f9eded740 1 -- 172.21.15.83:0/4280902322 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f8000f680 con 0x5640cc57f350 2024-04-24T09:15:40.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.256+0000 7f5f7aa00640 1 -- 172.21.15.83:0/4280902322 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f5f8000cc70 con 0x5640cc57f350 2024-04-24T09:15:40.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.256+0000 7f5f9eded740 10 client.23736 fetch_fsmap learned FSMap version 69 2024-04-24T09:15:40.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.256+0000 7f5f9eded740 1 -- 172.21.15.83:0/4280902322 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x5640cc6001b0 con 0x5640cc57f350 2024-04-24T09:15:40.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.257+0000 7f5f7aa00640 1 -- 172.21.15.83:0/4280902322 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5f8000f680 con 0x5640cc57f350 2024-04-24T09:15:40.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.257+0000 7f5f7aa00640 1 -- 172.21.15.83:0/4280902322 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 69) ==== 8370+0+0 (secure 0 0 0) 0x7f5f800039d0 con 0x5640cc57f350 2024-04-24T09:15:40.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.257+0000 7f5f9eded740 10 client.23736 fetch_fsmap finished waiting for FSMap version 69 2024-04-24T09:15:40.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.257+0000 7f5f9eded740 10 client.23736 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-24T09:15:40.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.257+0000 7f5f9eded740 20 client.23736 populate_metadata read hostname 'smithi083' 2024-04-24T09:15:40.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.257+0000 7f5f9eded740 1 --2- 172.21.15.83:0/4280902322 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x5640cc600860 0x5640cc602c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:40.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.257+0000 7f5f9eded740 4 client.23736 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:15:40.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.257+0000 7f5f9eded740 1 -- 172.21.15.83:0/4280902322 --> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5640cc4e3a60 con 0x5640cc600860 2024-04-24T09:15:40.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.258+0000 7f5f99600640 1 --2- 172.21.15.83:0/4280902322 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x5640cc600860 0x5640cc602c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:40.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.259+0000 7f5f99600640 1 --2- 172.21.15.83:0/4280902322 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x5640cc600860 0x5640cc602c80 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=18d60c7736411c2c server_cookie=258c2e8396536653 in_seq=0 out_seq=0 2024-04-24T09:15:40.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.259+0000 7f5f7aa00640 10 client.23736 ms_handle_connect on v2:172.21.15.138:6836/900566754 2024-04-24T09:15:40.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.262+0000 7f5f7aa00640 1 -- 172.21.15.83:0/4280902322 <== mds.1 v2:172.21.15.138:6836/900566754 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5640cc4e3a60 con 0x5640cc600860 2024-04-24T09:15:40.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.262+0000 7f5f7aa00640 10 client.23736 handle_command_reply: tid=0 2024-04-24T09:15:40.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.262+0000 7f5f9eded740 1 client.23736 shutdown 2024-04-24T09:15:40.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.262+0000 7f5f9eded740 1 -- 172.21.15.83:0/4280902322 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5640cc57f350 msgr2=0x5640cc592540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:40.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.262+0000 7f5f9eded740 1 --2- 172.21.15.83:0/4280902322 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5640cc57f350 0x5640cc592540 secure :-1 s=READY pgs=2736 cs=0 l=1 rev1=1 crypto rx=0x7f5f80009cc0 tx=0x7f5f8000fe30 comp rx=0 tx=0).stop 2024-04-24T09:15:40.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.262+0000 7f5f9eded740 1 -- 172.21.15.83:0/4280902322 shutdown_connections 2024-04-24T09:15:40.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.262+0000 7f5f9eded740 1 --2- 172.21.15.83:0/4280902322 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640cc58a340 0x5640cc5833f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:40.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.262+0000 7f5f9eded740 1 --2- 172.21.15.83:0/4280902322 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x5640cc600860 0x5640cc602c80 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:40.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.262+0000 7f5f9eded740 1 --2- 172.21.15.83:0/4280902322 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5640cc57fc70 0x5640cc592a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:40.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.262+0000 7f5f9eded740 1 --2- 172.21.15.83:0/4280902322 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5640cc57f350 0x5640cc592540 unknown :-1 s=CLOSED pgs=2736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:40.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.262+0000 7f5f9eded740 1 -- 172.21.15.83:0/4280902322 >> 172.21.15.83:0/4280902322 conn(0x5640cc57b670 msgr2=0x5640cc571f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:40.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.262+0000 7f5f7aa00640 0 client.23736 ms_handle_reset on v2:172.21.15.138:6836/900566754 2024-04-24T09:15:40.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.263+0000 7f5f9eded740 1 -- 172.21.15.83:0/4280902322 shutdown_connections 2024-04-24T09:15:40.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.263+0000 7f5f9eded740 1 -- 172.21.15.83:0/4280902322 wait complete. 2024-04-24T09:15:40.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.263+0000 7f5f9eded740 20 client.23736 trim_cache size 0 max 16384 2024-04-24T09:15:40.264 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:15:40.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.264+0000 7f5f9aa00640 1 -- 172.21.15.83:0/1251042928 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f5f7c0a3e60 msgr2=0x7f5f7c0a6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:40.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.264+0000 7f5f9aa00640 1 --2- 172.21.15.83:0/1251042928 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f5f7c0a3e60 0x7f5f7c0a6320 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f5f88007590 tx=0x7f5f880056e0 comp rx=0 tx=0).stop 2024-04-24T09:15:40.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.264+0000 7f5f9aa00640 1 -- 172.21.15.83:0/1251042928 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f94113f50 msgr2=0x7f5f941a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:40.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.264+0000 7f5f9aa00640 1 --2- 172.21.15.83:0/1251042928 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f94113f50 0x7f5f941a3670 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f5f8800a840 tx=0x7f5f8800ad00 comp rx=0 tx=0).stop 2024-04-24T09:15:40.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.264+0000 7f5f9aa00640 1 -- 172.21.15.83:0/1251042928 shutdown_connections 2024-04-24T09:15:40.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.264+0000 7f5f9aa00640 1 --2- 172.21.15.83:0/1251042928 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f5f7c0a3e60 0x7f5f7c0a6320 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:40.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.264+0000 7f5f9aa00640 1 --2- 172.21.15.83:0/1251042928 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f94113f50 0x7f5f941a3670 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:40.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.264+0000 7f5f9aa00640 1 --2- 172.21.15.83:0/1251042928 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5f9410b2c0 0x7f5f9419f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:40.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.264+0000 7f5f9aa00640 1 --2- 172.21.15.83:0/1251042928 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f9410a9a0 0x7f5f9419edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:40.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.264+0000 7f5f9aa00640 1 -- 172.21.15.83:0/1251042928 >> 172.21.15.83:0/1251042928 conn(0x7f5f9406d3e0 msgr2=0x7f5f94070c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:40.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.264+0000 7f5f9aa00640 1 -- 172.21.15.83:0/1251042928 shutdown_connections 2024-04-24T09:15:40.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.264+0000 7f5f9aa00640 1 -- 172.21.15.83:0/1251042928 wait complete. 2024-04-24T09:15:40.286 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T09:15:40.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:15:40 smithi138 ceph-mon[31441]: pgmap v760: 129 pgs: 129 active+clean; 1.1 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.0 MiB/s wr, 331 op/s 2024-04-24T09:15:40.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:15:40 smithi189 ceph-mon[31791]: pgmap v760: 129 pgs: 129 active+clean; 1.1 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.0 MiB/s wr, 331 op/s 2024-04-24T09:15:40.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.544+0000 7f49b3200640 1 -- 172.21.15.83:0/1214172420 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49b4102110 msgr2=0x7f49b41024f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:40.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.544+0000 7f49b3200640 1 --2- 172.21.15.83:0/1214172420 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49b4102110 0x7f49b41024f0 secure :-1 s=READY pgs=2737 cs=0 l=1 rev1=1 crypto rx=0x7f49a4009900 tx=0x7f49a402a410 comp rx=0 tx=0).stop 2024-04-24T09:15:40.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.544+0000 7f49b3200640 1 -- 172.21.15.83:0/1214172420 shutdown_connections 2024-04-24T09:15:40.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.544+0000 7f49b3200640 1 --2- 172.21.15.83:0/1214172420 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f49b41071e0 0x7f49b410edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:40.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.544+0000 7f49b3200640 1 --2- 172.21.15.83:0/1214172420 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f49b4102ac0 0x7f49b4106b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:40.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.544+0000 7f49b3200640 1 --2- 172.21.15.83:0/1214172420 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49b4102110 0x7f49b41024f0 unknown :-1 s=CLOSED pgs=2737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:40.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.544+0000 7f49b3200640 1 -- 172.21.15.83:0/1214172420 >> 172.21.15.83:0/1214172420 conn(0x7f49b40fd9d0 msgr2=0x7f49b40ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:40.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.544+0000 7f49b3200640 1 -- 172.21.15.83:0/1214172420 shutdown_connections 2024-04-24T09:15:40.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.544+0000 7f49b3200640 1 -- 172.21.15.83:0/1214172420 wait complete. 2024-04-24T09:15:40.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.545+0000 7f49b3200640 1 Processor -- start 2024-04-24T09:15:40.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.545+0000 7f49b3200640 1 -- start start 2024-04-24T09:15:40.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.545+0000 7f49b3200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f49b4102110 0x7f49b4198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:40.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.545+0000 7f49b3200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f49b41071e0 0x7f49b4198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:40.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.545+0000 7f49b3200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49b419cf80 0x7f49b419d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:40.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.545+0000 7f49b3200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f49b4112170 con 0x7f49b419cf80 2024-04-24T09:15:40.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.545+0000 7f49b3200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f49b4111ff0 con 0x7f49b4102110 2024-04-24T09:15:40.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.545+0000 7f49b3200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f49b41122f0 con 0x7f49b41071e0 2024-04-24T09:15:40.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.546+0000 7f49b2800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49b419cf80 0x7f49b419d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:40.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.546+0000 7f49b2800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49b419cf80 0x7f49b419d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43302/0 (socket says 172.21.15.83:43302) 2024-04-24T09:15:40.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.546+0000 7f49b2800640 1 -- 172.21.15.83:0/2272832235 learned_addr learned my addr 172.21.15.83:0/2272832235 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:15:40.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.546+0000 7f49b1400640 1 --2- 172.21.15.83:0/2272832235 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f49b41071e0 0x7f49b4198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:40.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.546+0000 7f49b1e00640 1 --2- 172.21.15.83:0/2272832235 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f49b4102110 0x7f49b4198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:40.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.546+0000 7f49b2800640 1 -- 172.21.15.83:0/2272832235 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f49b41071e0 msgr2=0x7f49b4198c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:40.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.546+0000 7f49b2800640 1 --2- 172.21.15.83:0/2272832235 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f49b41071e0 0x7f49b4198c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:40.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.546+0000 7f49b2800640 1 -- 172.21.15.83:0/2272832235 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f49b4102110 msgr2=0x7f49b4198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:40.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.546+0000 7f49b2800640 1 --2- 172.21.15.83:0/2272832235 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f49b4102110 0x7f49b4198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:40.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.546+0000 7f49b2800640 1 -- 172.21.15.83:0/2272832235 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49b419da70 con 0x7f49b419cf80 2024-04-24T09:15:40.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.547+0000 7f49b2800640 1 --2- 172.21.15.83:0/2272832235 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49b419cf80 0x7f49b419d430 secure :-1 s=READY pgs=2738 cs=0 l=1 rev1=1 crypto rx=0x7f49ac00eaa0 tx=0x7f49ac00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:40.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.547+0000 7f49aaa00640 1 -- 172.21.15.83:0/2272832235 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49ac00cdb0 con 0x7f49b419cf80 2024-04-24T09:15:40.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.547+0000 7f49b3200640 1 -- 172.21.15.83:0/2272832235 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49b419dd60 con 0x7f49b419cf80 2024-04-24T09:15:40.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.547+0000 7f49aaa00640 1 -- 172.21.15.83:0/2272832235 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f49ac005020 con 0x7f49b419cf80 2024-04-24T09:15:40.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.548+0000 7f49aaa00640 1 -- 172.21.15.83:0/2272832235 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49ac00cdb0 con 0x7f49b419cf80 2024-04-24T09:15:40.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.548+0000 7f49b3200640 1 -- 172.21.15.83:0/2272832235 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49b4074550 con 0x7f49b419cf80 2024-04-24T09:15:40.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.552+0000 7f49aaa00640 1 -- 172.21.15.83:0/2272832235 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f49ac0051d0 con 0x7f49b419cf80 2024-04-24T09:15:40.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.552+0000 7f49bbf99740 1 Processor -- start 2024-04-24T09:15:40.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:15:40 smithi083 ceph-mon[27907]: pgmap v760: 129 pgs: 129 active+clean; 1.1 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.0 MiB/s wr, 331 op/s 2024-04-24T09:15:40.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.554+0000 7f49bbf99740 1 -- start start 2024-04-24T09:15:40.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.554+0000 7f49aaa00640 1 --2- 172.21.15.83:0/2272832235 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f49980a4020 0x7f49980a64e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:40.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.554+0000 7f49aaa00640 1 -- 172.21.15.83:0/2272832235 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 9583+0+0 (secure 0 0 0) 0x7f49ac0d0550 con 0x7f49b419cf80 2024-04-24T09:15:40.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.555+0000 7f49bbf99740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559daf10bb40 0x559daf10bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:40.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.555+0000 7f49bbf99740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559daf10c460 0x559daf1165f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:40.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.555+0000 7f49bbf99740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559daf116b30 0x559daf118f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:40.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.555+0000 7f49bbf99740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559daf0711d0 con 0x559daf10bb40 2024-04-24T09:15:40.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.555+0000 7f49bbf99740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559daf017450 con 0x559daf10c460 2024-04-24T09:15:40.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.555+0000 7f49bbf99740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559daf06d190 con 0x559daf116b30 2024-04-24T09:15:40.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.555+0000 7f49b1400640 1 --2- 172.21.15.83:0/2272832235 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f49980a4020 0x7f49980a64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:40.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.555+0000 7f49b2800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559daf116b30 0x559daf118f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:40.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.555+0000 7f49b2800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559daf116b30 0x559daf118f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:34034/0 (socket says 172.21.15.83:34034) 2024-04-24T09:15:40.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.555+0000 7f49b2800640 1 -- 172.21.15.83:0/906140887 learned_addr learned my addr 172.21.15.83:0/906140887 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:15:40.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.555+0000 7f49b1e00640 1 --2- 172.21.15.83:0/906140887 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559daf10bb40 0x559daf10bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:40.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.556+0000 7f49b2800640 1 -- 172.21.15.83:0/906140887 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559daf10c460 msgr2=0x559daf1165f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T09:15:40.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.556+0000 7f49b2800640 1 --2- 172.21.15.83:0/906140887 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559daf10c460 0x559daf1165f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:40.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.556+0000 7f49b2800640 1 -- 172.21.15.83:0/906140887 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559daf10bb40 msgr2=0x559daf10bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:40.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.556+0000 7f49b2800640 1 --2- 172.21.15.83:0/906140887 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559daf10bb40 0x559daf10bf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:40.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.556+0000 7f49b2800640 1 -- 172.21.15.83:0/906140887 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559daf0d4310 con 0x559daf116b30 2024-04-24T09:15:40.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.556+0000 7f49b2800640 1 --2- 172.21.15.83:0/906140887 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559daf116b30 0x559daf118f20 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f49ac0a47b0 tx=0x7f49ac09c230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d2b6fcfc8c738b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:40.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.557+0000 7f499f400640 1 -- 172.21.15.83:0/906140887 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49ac01f040 con 0x559daf116b30 2024-04-24T09:15:40.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.557+0000 7f499f400640 1 -- 172.21.15.83:0/906140887 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f49ac0a00d0 con 0x559daf116b30 2024-04-24T09:15:40.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.557+0000 7f499f400640 1 -- 172.21.15.83:0/906140887 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49ac0d00b0 con 0x559daf116b30 2024-04-24T09:15:40.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.558+0000 7f49bbf99740 1 -- 172.21.15.83:0/906140887 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559daf116b30 msgr2=0x559daf118f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:40.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.558+0000 7f49bbf99740 1 --2- 172.21.15.83:0/906140887 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559daf116b30 0x559daf118f20 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f49ac0a47b0 tx=0x7f49ac09c230 comp rx=0 tx=0).stop 2024-04-24T09:15:40.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.558+0000 7f49b1400640 1 --2- 172.21.15.83:0/2272832235 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f49980a4020 0x7f49980a64e0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f49b40fdef0 tx=0x7f49a000b390 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:40.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.558+0000 7f49bbf99740 1 -- 172.21.15.83:0/906140887 shutdown_connections 2024-04-24T09:15:40.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.558+0000 7f49bbf99740 1 --2- 172.21.15.83:0/906140887 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559daf116b30 0x559daf118f20 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:40.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.558+0000 7f49bbf99740 1 --2- 172.21.15.83:0/906140887 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559daf10c460 0x559daf1165f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:40.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.558+0000 7f49bbf99740 1 --2- 172.21.15.83:0/906140887 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559daf10bb40 0x559daf10bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:40.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.558+0000 7f49bbf99740 1 -- 172.21.15.83:0/906140887 >> 172.21.15.83:0/906140887 conn(0x559daf107e60 msgr2=0x559daf108240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:40.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.558+0000 7f49bbf99740 1 -- 172.21.15.83:0/906140887 shutdown_connections 2024-04-24T09:15:40.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.558+0000 7f49bbf99740 1 -- 172.21.15.83:0/906140887 wait complete. 2024-04-24T09:15:40.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.559+0000 7f49bbf99740 1 -- start start 2024-04-24T09:15:40.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.559+0000 7f49bbf99740 1 Processor -- start 2024-04-24T09:15:40.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.560+0000 7f49bbf99740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559daf10bb40 0x559daf113900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:40.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.560+0000 7f49bbf99740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559daf10c460 0x559daf113e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:40.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.560+0000 7f49bbf99740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559daf116b30 0x559daf114380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:40.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.560+0000 7f49bbf99740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559daf1199c0 con 0x559daf116b30 2024-04-24T09:15:40.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.560+0000 7f49bbf99740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559daf02a810 con 0x559daf10bb40 2024-04-24T09:15:40.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.560+0000 7f49bbf99740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559daf119b40 con 0x559daf10c460 2024-04-24T09:15:40.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.560+0000 7f49bbf99740 10 client.0 _ll_register_callbacks cb 0x559daeff8140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:15:40.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.560+0000 7f49b2800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559daf116b30 0x559daf114380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:40.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.560+0000 7f49b2800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559daf116b30 0x559daf114380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43312/0 (socket says 172.21.15.83:43312) 2024-04-24T09:15:40.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.560+0000 7f49b2800640 1 -- 172.21.15.83:0/2163950839 learned_addr learned my addr 172.21.15.83:0/2163950839 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:15:40.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.560+0000 7f49b1e00640 1 --2- 172.21.15.83:0/2163950839 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559daf10bb40 0x559daf113900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:40.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.560+0000 7f49b1400640 1 --2- 172.21.15.83:0/2163950839 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559daf10c460 0x559daf113e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:40.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.561+0000 7f49b2800640 1 -- 172.21.15.83:0/2163950839 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559daf10c460 msgr2=0x559daf113e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:40.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.561+0000 7f49b2800640 1 --2- 172.21.15.83:0/2163950839 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559daf10c460 0x559daf113e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:40.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.561+0000 7f49b2800640 1 -- 172.21.15.83:0/2163950839 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559daf10bb40 msgr2=0x559daf113900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:40.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.561+0000 7f49b2800640 1 --2- 172.21.15.83:0/2163950839 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559daf10bb40 0x559daf113900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:40.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.561+0000 7f49b2800640 1 -- 172.21.15.83:0/2163950839 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559daf03b3f0 con 0x559daf116b30 2024-04-24T09:15:40.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.561+0000 7f49b2800640 1 --2- 172.21.15.83:0/2163950839 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559daf116b30 0x559daf114380 secure :-1 s=READY pgs=2739 cs=0 l=1 rev1=1 crypto rx=0x7f49ac09be20 tx=0x7f49ac09c020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5fb892251e51355 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:40.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.561+0000 7f4993e00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:15:40.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.562+0000 7f4993e00640 1 -- 172.21.15.83:0/2163950839 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49ac01f070 con 0x559daf116b30 2024-04-24T09:15:40.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.562+0000 7f49bbf99740 1 -- 172.21.15.83:0/2163950839 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49ac0a00d0 con 0x559daf116b30 2024-04-24T09:15:40.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.562+0000 7f4993e00640 1 -- 172.21.15.83:0/2163950839 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(130..130 src has 1..130) ==== 9583+0+0 (secure 0 0 0) 0x7f49ac0d0130 con 0x559daf116b30 2024-04-24T09:15:40.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.562+0000 7f4993e00640 1 -- 172.21.15.83:0/2163950839 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f49ac0d0330 con 0x559daf116b30 2024-04-24T09:15:40.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.562+0000 7f49bbf99740 10 client.23745 fetch_fsmap learned FSMap version 69 2024-04-24T09:15:40.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.562+0000 7f49bbf99740 1 -- 172.21.15.83:0/2163950839 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x559daed833f0 con 0x559daf116b30 2024-04-24T09:15:40.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.563+0000 7f4993e00640 1 -- 172.21.15.83:0/2163950839 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f49ac0a00d0 con 0x559daf116b30 2024-04-24T09:15:40.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.563+0000 7f4993e00640 1 -- 172.21.15.83:0/2163950839 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 69) ==== 8370+0+0 (secure 0 0 0) 0x7f49ac022060 con 0x559daf116b30 2024-04-24T09:15:40.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.563+0000 7f49bbf99740 10 client.23745 fetch_fsmap finished waiting for FSMap version 69 2024-04-24T09:15:40.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.563+0000 7f49bbf99740 10 client.23745 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:15:40.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.563+0000 7f49bbf99740 20 client.23745 populate_metadata read hostname 'smithi083' 2024-04-24T09:15:40.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.563+0000 7f49bbf99740 1 --2- 172.21.15.83:0/2163950839 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x559daf126c70 0x559daf129060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:40.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.563+0000 7f49bbf99740 4 client.23745 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:15:40.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.563+0000 7f49bbf99740 1 -- 172.21.15.83:0/2163950839 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559daf07ebf0 con 0x559daf126c70 2024-04-24T09:15:40.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.564+0000 7f49b1e00640 1 --2- 172.21.15.83:0/2163950839 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x559daf126c70 0x559daf129060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:40.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.565+0000 7f49b1e00640 1 --2- 172.21.15.83:0/2163950839 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x559daf126c70 0x559daf129060 crc :-1 s=READY pgs=589 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b85c3c80fda24a96 server_cookie=ce9c4eeeb59640dc in_seq=0 out_seq=0 2024-04-24T09:15:40.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.565+0000 7f4993e00640 10 client.23745 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:15:40.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.567+0000 7f4993e00640 1 -- 172.21.15.83:0/2163950839 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559daf07ebf0 con 0x559daf126c70 2024-04-24T09:15:40.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.567+0000 7f4993e00640 10 client.23745 handle_command_reply: tid=0 2024-04-24T09:15:40.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.567+0000 7f49bbf99740 1 client.23745 shutdown 2024-04-24T09:15:40.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.568+0000 7f49bbf99740 1 -- 172.21.15.83:0/2163950839 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559daf116b30 msgr2=0x559daf114380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:40.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.568+0000 7f49bbf99740 1 --2- 172.21.15.83:0/2163950839 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559daf116b30 0x559daf114380 secure :-1 s=READY pgs=2739 cs=0 l=1 rev1=1 crypto rx=0x7f49ac09be20 tx=0x7f49ac09c020 comp rx=0 tx=0).stop 2024-04-24T09:15:40.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.568+0000 7f49bbf99740 1 -- 172.21.15.83:0/2163950839 shutdown_connections 2024-04-24T09:15:40.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.568+0000 7f49bbf99740 1 --2- 172.21.15.83:0/2163950839 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x559daf126c70 0x559daf129060 crc :-1 s=READY pgs=589 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:40.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.568+0000 7f49bbf99740 1 --2- 172.21.15.83:0/2163950839 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559daf116b30 0x559daf114380 unknown :-1 s=CLOSED pgs=2739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:40.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.568+0000 7f49bbf99740 1 --2- 172.21.15.83:0/2163950839 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559daf10c460 0x559daf113e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:40.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.568+0000 7f4993e00640 0 client.23745 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:15:40.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.568+0000 7f49bbf99740 1 --2- 172.21.15.83:0/2163950839 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559daf10bb40 0x559daf113900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:40.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.568+0000 7f49bbf99740 1 -- 172.21.15.83:0/2163950839 >> 172.21.15.83:0/2163950839 conn(0x559daf107e60 msgr2=0x559daf10dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:40.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.568+0000 7f49bbf99740 1 -- 172.21.15.83:0/2163950839 shutdown_connections 2024-04-24T09:15:40.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.568+0000 7f49bbf99740 1 -- 172.21.15.83:0/2163950839 wait complete. 2024-04-24T09:15:40.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.569+0000 7f49bbf99740 20 client.23745 trim_cache size 0 max 16384 2024-04-24T09:15:40.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.591+0000 7f49bbf99740 1 Processor -- start 2024-04-24T09:15:40.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.591+0000 7f49bbf99740 1 -- start start 2024-04-24T09:15:40.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.592+0000 7f49bbf99740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559daf10bb40 0x559daf1141a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:40.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.592+0000 7f49bbf99740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559daf10c460 0x559daf1127b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:40.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.592+0000 7f49bbf99740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559daf116b30 0x559daf110b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:40.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.592+0000 7f49bbf99740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f49b41b1680 con 0x559daf10c460 2024-04-24T09:15:40.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.592+0000 7f49bbf99740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f49b41b3220 con 0x559daf10bb40 2024-04-24T09:15:40.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.592+0000 7f49bbf99740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f49b41b1d50 con 0x559daf116b30 2024-04-24T09:15:40.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.592+0000 7f49b1e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559daf10bb40 0x559daf1141a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:40.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.592+0000 7f49b1e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559daf10bb40 0x559daf1141a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:58680/0 (socket says 172.21.15.83:58680) 2024-04-24T09:15:40.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.592+0000 7f49b1e00640 1 -- 172.21.15.83:0/1999381266 learned_addr learned my addr 172.21.15.83:0/1999381266 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:15:40.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.593+0000 7f49b2800640 1 --2- 172.21.15.83:0/1999381266 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559daf116b30 0x559daf110b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:40.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.593+0000 7f49b1400640 1 --2- 172.21.15.83:0/1999381266 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559daf10c460 0x559daf1127b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:40.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.593+0000 7f49b1e00640 1 -- 172.21.15.83:0/1999381266 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559daf116b30 msgr2=0x559daf110b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:40.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.593+0000 7f49b1e00640 1 --2- 172.21.15.83:0/1999381266 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559daf116b30 0x559daf110b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:40.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.593+0000 7f49b1e00640 1 -- 172.21.15.83:0/1999381266 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559daf10c460 msgr2=0x559daf1127b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:40.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.593+0000 7f49b1e00640 1 --2- 172.21.15.83:0/1999381266 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559daf10c460 0x559daf1127b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:40.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.593+0000 7f49b1e00640 1 -- 172.21.15.83:0/1999381266 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559daee4d7d0 con 0x559daf10bb40 2024-04-24T09:15:40.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.593+0000 7f49b1400640 1 --2- 172.21.15.83:0/1999381266 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559daf10c460 0x559daf1127b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:15:40.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.594+0000 7f49b1e00640 1 --2- 172.21.15.83:0/1999381266 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559daf10bb40 0x559daf1141a0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f49a4009230 tx=0x7f49a4005a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37c595073a60a72d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:40.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.594+0000 7f499ea00640 1 -- 172.21.15.83:0/1999381266 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49a4035040 con 0x559daf10bb40 2024-04-24T09:15:40.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.594+0000 7f499ea00640 1 -- 172.21.15.83:0/1999381266 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f49a40054f0 con 0x559daf10bb40 2024-04-24T09:15:40.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.594+0000 7f499ea00640 1 -- 172.21.15.83:0/1999381266 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49a4045550 con 0x559daf10bb40 2024-04-24T09:15:40.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.595+0000 7f49bbf99740 1 -- 172.21.15.83:0/1999381266 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559daf10bb40 msgr2=0x559daf1141a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:40.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.595+0000 7f49bbf99740 1 --2- 172.21.15.83:0/1999381266 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559daf10bb40 0x559daf1141a0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f49a4009230 tx=0x7f49a4005a20 comp rx=0 tx=0).stop 2024-04-24T09:15:40.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.596+0000 7f49bbf99740 1 -- 172.21.15.83:0/1999381266 shutdown_connections 2024-04-24T09:15:40.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.596+0000 7f49bbf99740 1 --2- 172.21.15.83:0/1999381266 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559daf116b30 0x559daf110b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:40.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.596+0000 7f49bbf99740 1 --2- 172.21.15.83:0/1999381266 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559daf10c460 0x559daf1127b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:40.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.596+0000 7f49bbf99740 1 --2- 172.21.15.83:0/1999381266 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559daf10bb40 0x559daf1141a0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:40.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.596+0000 7f49bbf99740 1 -- 172.21.15.83:0/1999381266 >> 172.21.15.83:0/1999381266 conn(0x559daf107e60 msgr2=0x559daf1120a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:40.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.597+0000 7f49bbf99740 1 -- 172.21.15.83:0/1999381266 shutdown_connections 2024-04-24T09:15:40.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.597+0000 7f49bbf99740 1 -- 172.21.15.83:0/1999381266 wait complete. 2024-04-24T09:15:40.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.598+0000 7f49bbf99740 1 -- start start 2024-04-24T09:15:40.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.598+0000 7f49bbf99740 1 Processor -- start 2024-04-24T09:15:40.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.599+0000 7f49bbf99740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559daf10bb40 0x559daf12c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:40.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.599+0000 7f49bbf99740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559daf10c460 0x559daf110250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:40.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.599+0000 7f49bbf99740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559daf116b30 0x559daf110790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:40.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.600+0000 7f49bbf99740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f49b41b2b50 con 0x559daf116b30 2024-04-24T09:15:40.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.600+0000 7f49bbf99740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f49b41b2480 con 0x559daf10bb40 2024-04-24T09:15:40.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.600+0000 7f49bbf99740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f49b41af680 con 0x559daf10c460 2024-04-24T09:15:40.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.600+0000 7f49b1e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559daf10bb40 0x559daf12c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:40.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.600+0000 7f49b1e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559daf10bb40 0x559daf12c360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:58682/0 (socket says 172.21.15.83:58682) 2024-04-24T09:15:40.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.600+0000 7f49b1e00640 1 -- 172.21.15.83:0/2776252022 learned_addr learned my addr 172.21.15.83:0/2776252022 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:15:40.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.600+0000 7f49b1400640 1 --2- 172.21.15.83:0/2776252022 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559daf10c460 0x559daf110250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:40.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.600+0000 7f49bbf99740 10 client.0 _ll_register_callbacks cb 0x559daf116a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:15:40.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.600+0000 7f49b2800640 1 --2- 172.21.15.83:0/2776252022 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559daf116b30 0x559daf110790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:40.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.601+0000 7f49b2800640 1 -- 172.21.15.83:0/2776252022 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559daf10c460 msgr2=0x559daf110250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:40.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.601+0000 7f49b2800640 1 --2- 172.21.15.83:0/2776252022 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559daf10c460 0x559daf110250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:40.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.601+0000 7f49b2800640 1 -- 172.21.15.83:0/2776252022 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559daf10bb40 msgr2=0x559daf12c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:40.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.601+0000 7f49b2800640 1 --2- 172.21.15.83:0/2776252022 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559daf10bb40 0x559daf12c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:40.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.601+0000 7f49b2800640 1 -- 172.21.15.83:0/2776252022 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559daf110e50 con 0x559daf116b30 2024-04-24T09:15:40.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.602+0000 7f49b2800640 1 --2- 172.21.15.83:0/2776252022 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559daf116b30 0x559daf110790 secure :-1 s=READY pgs=2740 cs=0 l=1 rev1=1 crypto rx=0x7f49ac098580 tx=0x7f49ac005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5461b588e424f1c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:40.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.602+0000 7f4993e00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:15:40.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.602+0000 7f4993e00640 1 -- 172.21.15.83:0/2776252022 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49ac01f040 con 0x559daf116b30 2024-04-24T09:15:40.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.603+0000 7f4993e00640 1 -- 172.21.15.83:0/2776252022 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(130..130 src has 1..130) ==== 9583+0+0 (secure 0 0 0) 0x7f49ac098790 con 0x559daf116b30 2024-04-24T09:15:40.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.603+0000 7f49bbf99740 1 -- 172.21.15.83:0/2776252022 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49a40054f0 con 0x559daf116b30 2024-04-24T09:15:40.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.603+0000 7f4993e00640 1 -- 172.21.15.83:0/2776252022 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f49ac0b0b10 con 0x559daf116b30 2024-04-24T09:15:40.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.603+0000 7f49bbf99740 10 client.23751 fetch_fsmap learned FSMap version 69 2024-04-24T09:15:40.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.604+0000 7f49bbf99740 1 -- 172.21.15.83:0/2776252022 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x559daf18cd10 con 0x559daf116b30 2024-04-24T09:15:40.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.604+0000 7f4993e00640 1 -- 172.21.15.83:0/2776252022 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f49a40054f0 con 0x559daf116b30 2024-04-24T09:15:40.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.604+0000 7f4993e00640 1 -- 172.21.15.83:0/2776252022 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 69) ==== 8370+0+0 (secure 0 0 0) 0x7f49ac00f040 con 0x559daf116b30 2024-04-24T09:15:40.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.605+0000 7f49bbf99740 10 client.23751 fetch_fsmap finished waiting for FSMap version 69 2024-04-24T09:15:40.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.605+0000 7f49bbf99740 10 client.23751 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:15:40.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.605+0000 7f49bbf99740 20 client.23751 populate_metadata read hostname 'smithi083' 2024-04-24T09:15:40.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.605+0000 7f49bbf99740 1 --2- 172.21.15.83:0/2776252022 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x559daf18d240 0x559daf18f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:40.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.605+0000 7f49bbf99740 4 client.23751 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:15:40.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.605+0000 7f49bbf99740 1 -- 172.21.15.83:0/2776252022 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559daf078a60 con 0x559daf18d240 2024-04-24T09:15:40.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.605+0000 7f49b1e00640 1 --2- 172.21.15.83:0/2776252022 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x559daf18d240 0x559daf18f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:40.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.606+0000 7f49b1e00640 1 --2- 172.21.15.83:0/2776252022 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x559daf18d240 0x559daf18f630 crc :-1 s=READY pgs=590 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19be36e74714b17a server_cookie=607ef221da9acd68 in_seq=0 out_seq=0 2024-04-24T09:15:40.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.607+0000 7f4993e00640 10 client.23751 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:15:40.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.607+0000 7f4993e00640 1 -- 172.21.15.83:0/2776252022 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559daf078a60 con 0x559daf18d240 2024-04-24T09:15:40.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.607+0000 7f4993e00640 10 client.23751 handle_command_reply: tid=0 2024-04-24T09:15:40.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.607+0000 7f49bbf99740 1 client.23751 shutdown 2024-04-24T09:15:40.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.608+0000 7f49bbf99740 1 -- 172.21.15.83:0/2776252022 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559daf116b30 msgr2=0x559daf110790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:40.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.608+0000 7f49bbf99740 1 --2- 172.21.15.83:0/2776252022 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559daf116b30 0x559daf110790 secure :-1 s=READY pgs=2740 cs=0 l=1 rev1=1 crypto rx=0x7f49ac098580 tx=0x7f49ac005840 comp rx=0 tx=0).stop 2024-04-24T09:15:40.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.608+0000 7f49bbf99740 1 -- 172.21.15.83:0/2776252022 shutdown_connections 2024-04-24T09:15:40.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.608+0000 7f49bbf99740 1 --2- 172.21.15.83:0/2776252022 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x559daf18d240 0x559daf18f630 crc :-1 s=READY pgs=590 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:40.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.608+0000 7f49bbf99740 1 --2- 172.21.15.83:0/2776252022 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559daf116b30 0x559daf110790 unknown :-1 s=CLOSED pgs=2740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:40.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.608+0000 7f49bbf99740 1 --2- 172.21.15.83:0/2776252022 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559daf10c460 0x559daf110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:40.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.608+0000 7f49bbf99740 1 --2- 172.21.15.83:0/2776252022 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559daf10bb40 0x559daf12c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:40.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.608+0000 7f49bbf99740 1 -- 172.21.15.83:0/2776252022 >> 172.21.15.83:0/2776252022 conn(0x559daf107e60 msgr2=0x559daf112930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:40.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.608+0000 7f4993e00640 0 client.23751 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:15:40.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.609+0000 7f49bbf99740 1 -- 172.21.15.83:0/2776252022 shutdown_connections 2024-04-24T09:15:40.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.609+0000 7f49bbf99740 1 -- 172.21.15.83:0/2776252022 wait complete. 2024-04-24T09:15:40.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.609+0000 7f49bbf99740 20 client.23751 trim_cache size 0 max 16384 2024-04-24T09:15:40.609 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:15:40.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.610+0000 7f49b3200640 1 -- 172.21.15.83:0/2272832235 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f49980a4020 msgr2=0x7f49980a64e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:40.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.610+0000 7f49b3200640 1 --2- 172.21.15.83:0/2272832235 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f49980a4020 0x7f49980a64e0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f49b40fdef0 tx=0x7f49a000b390 comp rx=0 tx=0).stop 2024-04-24T09:15:40.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.610+0000 7f49b3200640 1 -- 172.21.15.83:0/2272832235 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49b419cf80 msgr2=0x7f49b419d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:40.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.610+0000 7f49b3200640 1 --2- 172.21.15.83:0/2272832235 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49b419cf80 0x7f49b419d430 secure :-1 s=READY pgs=2738 cs=0 l=1 rev1=1 crypto rx=0x7f49ac00eaa0 tx=0x7f49ac00ef60 comp rx=0 tx=0).stop 2024-04-24T09:15:40.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.610+0000 7f49b3200640 1 -- 172.21.15.83:0/2272832235 shutdown_connections 2024-04-24T09:15:40.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.610+0000 7f49b3200640 1 --2- 172.21.15.83:0/2272832235 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f49980a4020 0x7f49980a64e0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:40.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.610+0000 7f49b3200640 1 --2- 172.21.15.83:0/2272832235 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49b419cf80 0x7f49b419d430 unknown :-1 s=CLOSED pgs=2738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:40.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.610+0000 7f49b3200640 1 --2- 172.21.15.83:0/2272832235 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f49b41071e0 0x7f49b4198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:40.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.610+0000 7f49b3200640 1 --2- 172.21.15.83:0/2272832235 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f49b4102110 0x7f49b4198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:40.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.610+0000 7f49b3200640 1 -- 172.21.15.83:0/2272832235 >> 172.21.15.83:0/2272832235 conn(0x7f49b40fd9d0 msgr2=0x7f49b40fedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:40.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.610+0000 7f49b3200640 1 -- 172.21.15.83:0/2272832235 shutdown_connections 2024-04-24T09:15:40.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:40.611+0000 7f49b3200640 1 -- 172.21.15.83:0/2272832235 wait complete. 2024-04-24T09:15:41.213 INFO:tasks.workunit.client.0.smithi083.stdout: 10 75171 6.55 MB/sec execute 492 sec latency 108.291 ms 2024-04-24T09:15:41.634 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T09:15:41.634 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T09:15:41.634 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T09:15:41.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.881+0000 7fbbba400640 1 -- 172.21.15.83:0/3870444075 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbb410c480 msgr2=0x7fbbb410e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:41.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.881+0000 7fbbba400640 1 --2- 172.21.15.83:0/3870444075 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbb410c480 0x7fbbb410e920 secure :-1 s=READY pgs=2741 cs=0 l=1 rev1=1 crypto rx=0x7fbbac009ca0 tx=0x7fbbac02a950 comp rx=0 tx=0).stop 2024-04-24T09:15:41.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.881+0000 7fbbba400640 1 -- 172.21.15.83:0/3870444075 shutdown_connections 2024-04-24T09:15:41.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.881+0000 7fbbba400640 1 --2- 172.21.15.83:0/3870444075 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbb410c480 0x7fbbb410e920 unknown :-1 s=CLOSED pgs=2741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:41.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.881+0000 7fbbba400640 1 --2- 172.21.15.83:0/3870444075 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbbb4103980 0x7fbbb410be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:41.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.881+0000 7fbbba400640 1 --2- 172.21.15.83:0/3870444075 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbbb4103060 0x7fbbb4103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:41.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.881+0000 7fbbba400640 1 -- 172.21.15.83:0/3870444075 >> 172.21.15.83:0/3870444075 conn(0x7fbbb40fc880 msgr2=0x7fbbb40feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:41.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.882+0000 7fbbba400640 1 -- 172.21.15.83:0/3870444075 shutdown_connections 2024-04-24T09:15:41.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.882+0000 7fbbba400640 1 -- 172.21.15.83:0/3870444075 wait complete. 2024-04-24T09:15:41.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.883+0000 7fbbba400640 1 Processor -- start 2024-04-24T09:15:41.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.883+0000 7fbbba400640 1 -- start start 2024-04-24T09:15:41.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.883+0000 7fbbba400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbbb4103060 0x7fbbb4198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:41.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.883+0000 7fbbba400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbb4103980 0x7fbbb4198ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:41.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.883+0000 7fbbba400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbbb410c480 0x7fbbb419ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:41.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.884+0000 7fbbba400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb4112160 con 0x7fbbb4103980 2024-04-24T09:15:41.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.884+0000 7fbbba400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb4111fe0 con 0x7fbbb410c480 2024-04-24T09:15:41.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.884+0000 7fbbba400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb41122e0 con 0x7fbbb4103060 2024-04-24T09:15:41.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.884+0000 7fbbb3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbb4103980 0x7fbbb4198ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:41.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.884+0000 7fbbb3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbb4103980 0x7fbbb4198ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43364/0 (socket says 172.21.15.83:43364) 2024-04-24T09:15:41.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.884+0000 7fbbb3e00640 1 -- 172.21.15.83:0/2183075627 learned_addr learned my addr 172.21.15.83:0/2183075627 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:15:41.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.884+0000 7fbbb9000640 1 --2- 172.21.15.83:0/2183075627 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbbb4103060 0x7fbbb4198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:41.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.884+0000 7fbbb9a00640 1 --2- 172.21.15.83:0/2183075627 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbbb410c480 0x7fbbb419ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:41.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.885+0000 7fbbb3e00640 1 -- 172.21.15.83:0/2183075627 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbbb4103060 msgr2=0x7fbbb4198660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:41.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.885+0000 7fbbb3e00640 1 --2- 172.21.15.83:0/2183075627 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbbb4103060 0x7fbbb4198660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:41.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.885+0000 7fbbb3e00640 1 -- 172.21.15.83:0/2183075627 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbbb410c480 msgr2=0x7fbbb419ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:41.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.885+0000 7fbbb3e00640 1 --2- 172.21.15.83:0/2183075627 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbbb410c480 0x7fbbb419ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:41.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.885+0000 7fbbb3e00640 1 -- 172.21.15.83:0/2183075627 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbb419d600 con 0x7fbbb4103980 2024-04-24T09:15:41.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.885+0000 7fbbb3e00640 1 --2- 172.21.15.83:0/2183075627 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbb4103980 0x7fbbb4198ba0 secure :-1 s=READY pgs=2742 cs=0 l=1 rev1=1 crypto rx=0x7fbba000c970 tx=0x7fbba000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:41.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.885+0000 7fbbb1600640 1 -- 172.21.15.83:0/2183075627 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbba0007bb0 con 0x7fbbb4103980 2024-04-24T09:15:41.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.885+0000 7fbbb1600640 1 -- 172.21.15.83:0/2183075627 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbba0007d50 con 0x7fbbb4103980 2024-04-24T09:15:41.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.885+0000 7fbbb1600640 1 -- 172.21.15.83:0/2183075627 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbba0004e10 con 0x7fbbb4103980 2024-04-24T09:15:41.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.885+0000 7fbbba400640 1 -- 172.21.15.83:0/2183075627 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbb419d8f0 con 0x7fbbb4103980 2024-04-24T09:15:41.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.886+0000 7fbbba400640 1 -- 172.21.15.83:0/2183075627 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbb4100be0 con 0x7fbbb4103980 2024-04-24T09:15:41.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.890+0000 7fbbbe808740 1 Processor -- start 2024-04-24T09:15:41.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.891+0000 7fbbbe808740 1 -- start start 2024-04-24T09:15:41.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.891+0000 7fbbbe808740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560549a43270 0x560549a43650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:41.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.891+0000 7fbbbe808740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560549a43b90 0x560549a4dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:41.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.891+0000 7fbbbe808740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560549a4e260 0x560549a50650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:41.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.891+0000 7fbbbe808740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5605499a8e70 con 0x560549a4e260 2024-04-24T09:15:41.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.891+0000 7fbbbe808740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56054994f380 con 0x560549a43270 2024-04-24T09:15:41.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.891+0000 7fbbbe808740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5605499a4e30 con 0x560549a43b90 2024-04-24T09:15:41.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.891+0000 7fbbb1600640 1 -- 172.21.15.83:0/2183075627 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fbba0005020 con 0x7fbbb4103980 2024-04-24T09:15:41.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.891+0000 7fbbb9a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560549a43270 0x560549a43650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:41.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.891+0000 7fbbb9a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560549a43270 0x560549a43650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:58712/0 (socket says 172.21.15.83:58712) 2024-04-24T09:15:41.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.891+0000 7fbbb9a00640 1 -- 172.21.15.83:0/1987485187 learned_addr learned my addr 172.21.15.83:0/1987485187 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:15:41.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.892+0000 7fbbb9000640 1 --2- 172.21.15.83:0/1987485187 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560549a43b90 0x560549a4dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:41.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.892+0000 7fbbb3e00640 1 --2- 172.21.15.83:0/1987485187 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560549a4e260 0x560549a50650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:41.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.892+0000 7fbbb9a00640 1 -- 172.21.15.83:0/1987485187 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560549a43b90 msgr2=0x560549a4dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:41.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.892+0000 7fbbb9a00640 1 --2- 172.21.15.83:0/1987485187 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560549a43b90 0x560549a4dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:41.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.892+0000 7fbbb9a00640 1 -- 172.21.15.83:0/1987485187 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560549a4e260 msgr2=0x560549a50650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:41.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.892+0000 7fbbb9a00640 1 --2- 172.21.15.83:0/1987485187 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560549a4e260 0x560549a50650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:41.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.893+0000 7fbbb9a00640 1 -- 172.21.15.83:0/1987485187 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605497837e0 con 0x560549a43270 2024-04-24T09:15:41.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.893+0000 7fbbb3e00640 1 --2- 172.21.15.83:0/1987485187 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560549a4e260 0x560549a50650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:15:41.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.893+0000 7fbbb9a00640 1 --2- 172.21.15.83:0/1987485187 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560549a43270 0x560549a43650 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fbbac009ca0 tx=0x7fbbac02f670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f834f75cd9ebc16 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:41.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.894+0000 7fbbb1600640 1 --2- 172.21.15.83:0/2183075627 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fbb9c0a3da0 0x7fbb9c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:41.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.894+0000 7fbbb1600640 1 -- 172.21.15.83:0/2183075627 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 9583+0+0 (secure 0 0 0) 0x7fbba00d0970 con 0x7fbbb4103980 2024-04-24T09:15:41.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.894+0000 7fbba6000640 1 -- 172.21.15.83:0/1987485187 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbac048070 con 0x560549a43270 2024-04-24T09:15:41.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.894+0000 7fbba6000640 1 -- 172.21.15.83:0/1987485187 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbbac003e20 con 0x560549a43270 2024-04-24T09:15:41.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.894+0000 7fbbb9000640 1 --2- 172.21.15.83:0/2183075627 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fbb9c0a3da0 0x7fbb9c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:41.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.894+0000 7fbba6000640 1 -- 172.21.15.83:0/1987485187 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbac034040 con 0x560549a43270 2024-04-24T09:15:41.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.895+0000 7fbbbe808740 1 -- 172.21.15.83:0/1987485187 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560549a43270 msgr2=0x560549a43650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:41.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.895+0000 7fbbb9000640 1 --2- 172.21.15.83:0/2183075627 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fbb9c0a3da0 0x7fbb9c0a6260 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x560549a44830 tx=0x7fbba800d500 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:41.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.895+0000 7fbbbe808740 1 --2- 172.21.15.83:0/1987485187 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560549a43270 0x560549a43650 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fbbac009ca0 tx=0x7fbbac02f670 comp rx=0 tx=0).stop 2024-04-24T09:15:41.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.896+0000 7fbbbe808740 1 -- 172.21.15.83:0/1987485187 shutdown_connections 2024-04-24T09:15:41.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.896+0000 7fbbbe808740 1 --2- 172.21.15.83:0/1987485187 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560549a4e260 0x560549a50650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:41.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.896+0000 7fbbbe808740 1 --2- 172.21.15.83:0/1987485187 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560549a43b90 0x560549a4dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:41.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.896+0000 7fbbbe808740 1 --2- 172.21.15.83:0/1987485187 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560549a43270 0x560549a43650 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:41.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.896+0000 7fbbbe808740 1 -- 172.21.15.83:0/1987485187 >> 172.21.15.83:0/1987485187 conn(0x560549a3f590 msgr2=0x560549a3f970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:41.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.898+0000 7fbbbe808740 1 -- 172.21.15.83:0/1987485187 shutdown_connections 2024-04-24T09:15:41.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.898+0000 7fbbbe808740 1 -- 172.21.15.83:0/1987485187 wait complete. 2024-04-24T09:15:41.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.899+0000 7fbbbe808740 1 -- start start 2024-04-24T09:15:41.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.899+0000 7fbbbe808740 1 Processor -- start 2024-04-24T09:15:41.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.899+0000 7fbbbe808740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560549a43270 0x560549a4b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:41.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.899+0000 7fbbbe808740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560549a43b90 0x560549a4b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:41.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.900+0000 7fbbbe808740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560549a4e260 0x560549a4bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:41.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.900+0000 7fbbbe808740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x560549a50ff0 con 0x560549a43b90 2024-04-24T09:15:41.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.900+0000 7fbbbe808740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560549962660 con 0x560549a4e260 2024-04-24T09:15:41.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.900+0000 7fbbbe808740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x560549a51170 con 0x560549a43270 2024-04-24T09:15:41.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.900+0000 7fbbbe808740 10 client.0 _ll_register_callbacks cb 0x560549950aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:15:41.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.900+0000 7fbbb9a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560549a4e260 0x560549a4bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:41.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.900+0000 7fbbb9000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560549a43270 0x560549a4b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:41.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.900+0000 7fbbb9000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560549a43270 0x560549a4b220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:34092/0 (socket says 172.21.15.83:34092) 2024-04-24T09:15:41.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.900+0000 7fbbb9000640 1 -- 172.21.15.83:0/3274778659 learned_addr learned my addr 172.21.15.83:0/3274778659 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:15:41.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.900+0000 7fbbb3e00640 1 --2- 172.21.15.83:0/3274778659 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560549a43b90 0x560549a4b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:41.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.901+0000 7fbbb3e00640 1 -- 172.21.15.83:0/3274778659 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560549a43270 msgr2=0x560549a4b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:41.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.901+0000 7fbbb3e00640 1 --2- 172.21.15.83:0/3274778659 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560549a43270 0x560549a4b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:41.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.901+0000 7fbbb3e00640 1 -- 172.21.15.83:0/3274778659 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560549a4e260 msgr2=0x560549a4bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:41.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.901+0000 7fbbb3e00640 1 --2- 172.21.15.83:0/3274778659 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560549a4e260 0x560549a4bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:41.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.901+0000 7fbbb3e00640 1 -- 172.21.15.83:0/3274778659 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605497e3f70 con 0x560549a43b90 2024-04-24T09:15:41.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.901+0000 7fbbb9a00640 1 --2- 172.21.15.83:0/3274778659 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560549a4e260 0x560549a4bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:15:41.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.901+0000 7fbbb3e00640 1 --2- 172.21.15.83:0/3274778659 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560549a43b90 0x560549a4b760 secure :-1 s=READY pgs=2743 cs=0 l=1 rev1=1 crypto rx=0x7fbba00cdc40 tx=0x7fbba00d62c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbc134d6e3e6bd0d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:41.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.901+0000 7fbb9aa00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:15:41.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.902+0000 7fbb9aa00640 1 -- 172.21.15.83:0/3274778659 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbba001d040 con 0x560549a43b90 2024-04-24T09:15:41.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.902+0000 7fbb9aa00640 1 -- 172.21.15.83:0/3274778659 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(130..130 src has 1..130) ==== 9583+0+0 (secure 0 0 0) 0x7fbba0005510 con 0x560549a43b90 2024-04-24T09:15:41.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.902+0000 7fbbbe808740 1 -- 172.21.15.83:0/3274778659 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbbac003e20 con 0x560549a43b90 2024-04-24T09:15:41.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.902+0000 7fbb9aa00640 1 -- 172.21.15.83:0/3274778659 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7fbba009bdf0 con 0x560549a43b90 2024-04-24T09:15:41.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.902+0000 7fbbbe808740 10 client.23763 fetch_fsmap learned FSMap version 69 2024-04-24T09:15:41.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.902+0000 7fbbbe808740 1 -- 172.21.15.83:0/3274778659 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x56054967b170 con 0x560549a43b90 2024-04-24T09:15:41.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.903+0000 7fbb9aa00640 1 -- 172.21.15.83:0/3274778659 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbbac003e20 con 0x560549a43b90 2024-04-24T09:15:41.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.903+0000 7fbb9aa00640 1 -- 172.21.15.83:0/3274778659 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 69) ==== 8370+0+0 (secure 0 0 0) 0x7fbba00981e0 con 0x560549a43b90 2024-04-24T09:15:41.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.903+0000 7fbbbe808740 10 client.23763 fetch_fsmap finished waiting for FSMap version 69 2024-04-24T09:15:41.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.903+0000 7fbbbe808740 10 client.23763 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:15:41.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.903+0000 7fbbbe808740 20 client.23763 populate_metadata read hostname 'smithi083' 2024-04-24T09:15:41.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.903+0000 7fbbbe808740 1 --2- 172.21.15.83:0/3274778659 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x560549a5e500 0x560549a608f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:41.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.903+0000 7fbbbe808740 4 client.23763 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:15:41.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.903+0000 7fbbbe808740 1 -- 172.21.15.83:0/3274778659 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5605499b6890 con 0x560549a5e500 2024-04-24T09:15:41.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.904+0000 7fbbb9000640 1 --2- 172.21.15.83:0/3274778659 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x560549a5e500 0x560549a608f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:41.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.904+0000 7fbbb9000640 1 --2- 172.21.15.83:0/3274778659 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x560549a5e500 0x560549a608f0 crc :-1 s=READY pgs=591 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88c8610f1399132f server_cookie=76f11de8440260f5 in_seq=0 out_seq=0 2024-04-24T09:15:41.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.904+0000 7fbb9aa00640 10 client.23763 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:15:41.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.905+0000 7fbb9aa00640 1 -- 172.21.15.83:0/3274778659 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5605499b6890 con 0x560549a5e500 2024-04-24T09:15:41.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.905+0000 7fbb9aa00640 10 client.23763 handle_command_reply: tid=0 2024-04-24T09:15:41.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.905+0000 7fbbbe808740 1 client.23763 shutdown 2024-04-24T09:15:41.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.906+0000 7fbbbe808740 1 -- 172.21.15.83:0/3274778659 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560549a43b90 msgr2=0x560549a4b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:41.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.906+0000 7fbbbe808740 1 --2- 172.21.15.83:0/3274778659 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560549a43b90 0x560549a4b760 secure :-1 s=READY pgs=2743 cs=0 l=1 rev1=1 crypto rx=0x7fbba00cdc40 tx=0x7fbba00d62c0 comp rx=0 tx=0).stop 2024-04-24T09:15:41.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.906+0000 7fbbbe808740 1 -- 172.21.15.83:0/3274778659 shutdown_connections 2024-04-24T09:15:41.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.906+0000 7fbbbe808740 1 --2- 172.21.15.83:0/3274778659 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x560549a5e500 0x560549a608f0 crc :-1 s=READY pgs=591 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:41.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.906+0000 7fbbbe808740 1 --2- 172.21.15.83:0/3274778659 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560549a4e260 0x560549a4bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:41.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.906+0000 7fbbbe808740 1 --2- 172.21.15.83:0/3274778659 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560549a43b90 0x560549a4b760 unknown :-1 s=CLOSED pgs=2743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:41.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.906+0000 7fbbbe808740 1 --2- 172.21.15.83:0/3274778659 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560549a43270 0x560549a4b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:41.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.906+0000 7fbbbe808740 1 -- 172.21.15.83:0/3274778659 >> 172.21.15.83:0/3274778659 conn(0x560549a3f590 msgr2=0x560549a45390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:41.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.906+0000 7fbb9aa00640 0 client.23763 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:15:41.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.906+0000 7fbbbe808740 1 -- 172.21.15.83:0/3274778659 shutdown_connections 2024-04-24T09:15:41.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.906+0000 7fbbbe808740 1 -- 172.21.15.83:0/3274778659 wait complete. 2024-04-24T09:15:41.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.906+0000 7fbbbe808740 20 client.23763 trim_cache size 0 max 16384 2024-04-24T09:15:41.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.939+0000 7fbbbe808740 1 Processor -- start 2024-04-24T09:15:41.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.939+0000 7fbbbe808740 1 -- start start 2024-04-24T09:15:41.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.939+0000 7fbbbe808740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560549a43270 0x560549a4a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:41.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.939+0000 7fbbbe808740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560549a43b90 0x560549a59eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:41.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.939+0000 7fbbbe808740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560549a4e260 0x560549a66180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:41.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.939+0000 7fbbbe808740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb41b1530 con 0x560549a43270 2024-04-24T09:15:41.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.939+0000 7fbbbe808740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb41b30d0 con 0x560549a43b90 2024-04-24T09:15:41.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.939+0000 7fbbbe808740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb41b1c00 con 0x560549a4e260 2024-04-24T09:15:41.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.940+0000 7fbbb9000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560549a43270 0x560549a4a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:41.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.940+0000 7fbbb9000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560549a43270 0x560549a4a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43400/0 (socket says 172.21.15.83:43400) 2024-04-24T09:15:41.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.940+0000 7fbbb9000640 1 -- 172.21.15.83:0/2566373159 learned_addr learned my addr 172.21.15.83:0/2566373159 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:15:41.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.940+0000 7fbbb3e00640 1 --2- 172.21.15.83:0/2566373159 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560549a43b90 0x560549a59eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:41.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.940+0000 7fbbb9a00640 1 --2- 172.21.15.83:0/2566373159 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560549a4e260 0x560549a66180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:41.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.940+0000 7fbbb9000640 1 -- 172.21.15.83:0/2566373159 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560549a4e260 msgr2=0x560549a66180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:41.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.940+0000 7fbbb9000640 1 --2- 172.21.15.83:0/2566373159 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560549a4e260 0x560549a66180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:41.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.940+0000 7fbbb9000640 1 -- 172.21.15.83:0/2566373159 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560549a43b90 msgr2=0x560549a59eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:41.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.940+0000 7fbbb9000640 1 --2- 172.21.15.83:0/2566373159 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560549a43b90 0x560549a59eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:41.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.940+0000 7fbbb9000640 1 -- 172.21.15.83:0/2566373159 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560549a50650 con 0x560549a43270 2024-04-24T09:15:41.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.941+0000 7fbbb9000640 1 --2- 172.21.15.83:0/2566373159 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560549a43270 0x560549a4a160 secure :-1 s=READY pgs=2744 cs=0 l=1 rev1=1 crypto rx=0x7fbba8004630 tx=0x7fbba8015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eba1d937eef43606 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:41.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.941+0000 7fbba5600640 1 -- 172.21.15.83:0/2566373159 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbba8028030 con 0x560549a43270 2024-04-24T09:15:41.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.941+0000 7fbba5600640 1 -- 172.21.15.83:0/2566373159 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbba8028680 con 0x560549a43270 2024-04-24T09:15:41.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.941+0000 7fbba5600640 1 -- 172.21.15.83:0/2566373159 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbba8027a40 con 0x560549a43270 2024-04-24T09:15:41.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.942+0000 7fbbbe808740 1 -- 172.21.15.83:0/2566373159 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560549a43270 msgr2=0x560549a4a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:41.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.942+0000 7fbbbe808740 1 --2- 172.21.15.83:0/2566373159 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560549a43270 0x560549a4a160 secure :-1 s=READY pgs=2744 cs=0 l=1 rev1=1 crypto rx=0x7fbba8004630 tx=0x7fbba8015000 comp rx=0 tx=0).stop 2024-04-24T09:15:41.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.942+0000 7fbbbe808740 1 -- 172.21.15.83:0/2566373159 shutdown_connections 2024-04-24T09:15:41.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.942+0000 7fbbbe808740 1 --2- 172.21.15.83:0/2566373159 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560549a4e260 0x560549a66180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:41.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.942+0000 7fbbbe808740 1 --2- 172.21.15.83:0/2566373159 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560549a43b90 0x560549a59eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:41.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.942+0000 7fbbbe808740 1 --2- 172.21.15.83:0/2566373159 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560549a43270 0x560549a4a160 unknown :-1 s=CLOSED pgs=2744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:41.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.942+0000 7fbbbe808740 1 -- 172.21.15.83:0/2566373159 >> 172.21.15.83:0/2566373159 conn(0x560549a3f590 msgr2=0x560549a3e6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:41.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.943+0000 7fbbbe808740 1 -- 172.21.15.83:0/2566373159 shutdown_connections 2024-04-24T09:15:41.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.943+0000 7fbbbe808740 1 -- 172.21.15.83:0/2566373159 wait complete. 2024-04-24T09:15:41.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.943+0000 7fbbbe808740 1 -- start start 2024-04-24T09:15:41.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.943+0000 7fbbbe808740 1 Processor -- start 2024-04-24T09:15:41.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.944+0000 7fbbbe808740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560549a43270 0x560549a48040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:41.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.944+0000 7fbbbe808740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560549a43b90 0x560549a48580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:41.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.944+0000 7fbbbe808740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560549a4e260 0x560549a48ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:41.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.944+0000 7fbbbe808740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb41b2a00 con 0x560549a4e260 2024-04-24T09:15:41.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.944+0000 7fbbbe808740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb41b2330 con 0x560549a43b90 2024-04-24T09:15:41.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.944+0000 7fbbbe808740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb41af530 con 0x560549a43270 2024-04-24T09:15:41.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.944+0000 7fbbbe808740 10 client.0 _ll_register_callbacks cb 0x7fbba00d3f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:15:41.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.944+0000 7fbbb3e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560549a43b90 0x560549a48580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:41.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.944+0000 7fbbb9a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560549a4e260 0x560549a48ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:41.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.944+0000 7fbbb3e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560549a43b90 0x560549a48580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:58748/0 (socket says 172.21.15.83:58748) 2024-04-24T09:15:41.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.944+0000 7fbbb3e00640 1 -- 172.21.15.83:0/4292822118 learned_addr learned my addr 172.21.15.83:0/4292822118 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:15:41.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.945+0000 7fbbb9000640 1 --2- 172.21.15.83:0/4292822118 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560549a43270 0x560549a48040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:41.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.945+0000 7fbbb9a00640 1 -- 172.21.15.83:0/4292822118 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560549a43270 msgr2=0x560549a48040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:41.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.945+0000 7fbbb9a00640 1 --2- 172.21.15.83:0/4292822118 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560549a43270 0x560549a48040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:41.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.945+0000 7fbbb9a00640 1 -- 172.21.15.83:0/4292822118 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560549a43b90 msgr2=0x560549a48580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:41.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.945+0000 7fbbb9a00640 1 --2- 172.21.15.83:0/4292822118 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560549a43b90 0x560549a48580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:41.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.945+0000 7fbbb9a00640 1 -- 172.21.15.83:0/4292822118 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560549ac3b90 con 0x560549a4e260 2024-04-24T09:15:41.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.945+0000 7fbbb3e00640 1 --2- 172.21.15.83:0/4292822118 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560549a43b90 0x560549a48580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:15:41.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.945+0000 7fbbb9a00640 1 --2- 172.21.15.83:0/4292822118 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560549a4e260 0x560549a48ac0 secure :-1 s=READY pgs=2745 cs=0 l=1 rev1=1 crypto rx=0x7fbbac009bf0 tx=0x7fbbac02a950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acd825cd91d5d463 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:41.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.945+0000 7fbb9aa00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:15:41.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.946+0000 7fbb9aa00640 1 -- 172.21.15.83:0/4292822118 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbac048050 con 0x560549a4e260 2024-04-24T09:15:41.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.946+0000 7fbb9aa00640 1 -- 172.21.15.83:0/4292822118 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(130..130 src has 1..130) ==== 9583+0+0 (secure 0 0 0) 0x7fbbac030650 con 0x560549a4e260 2024-04-24T09:15:41.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.946+0000 7fbbbe808740 1 -- 172.21.15.83:0/4292822118 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbba8028680 con 0x560549a4e260 2024-04-24T09:15:41.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.946+0000 7fbb9aa00640 1 -- 172.21.15.83:0/4292822118 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7fbbac05c050 con 0x560549a4e260 2024-04-24T09:15:41.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.946+0000 7fbbbe808740 10 client.23769 fetch_fsmap learned FSMap version 69 2024-04-24T09:15:41.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.946+0000 7fbbbe808740 1 -- 172.21.15.83:0/4292822118 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x560549ac4e90 con 0x560549a4e260 2024-04-24T09:15:41.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.947+0000 7fbb9aa00640 1 -- 172.21.15.83:0/4292822118 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbba8028680 con 0x560549a4e260 2024-04-24T09:15:41.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.947+0000 7fbb9aa00640 1 -- 172.21.15.83:0/4292822118 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 69) ==== 8370+0+0 (secure 0 0 0) 0x7fbbac057030 con 0x560549a4e260 2024-04-24T09:15:41.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.947+0000 7fbbbe808740 10 client.23769 fetch_fsmap finished waiting for FSMap version 69 2024-04-24T09:15:41.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.947+0000 7fbbbe808740 10 client.23769 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:15:41.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.947+0000 7fbbbe808740 20 client.23769 populate_metadata read hostname 'smithi083' 2024-04-24T09:15:41.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.947+0000 7fbbbe808740 1 --2- 172.21.15.83:0/4292822118 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x560549ac5170 0x560549ac7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:41.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.947+0000 7fbbbe808740 4 client.23769 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T09:15:41.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.947+0000 7fbbbe808740 1 -- 172.21.15.83:0/4292822118 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5605499b0700 con 0x560549ac5170 2024-04-24T09:15:41.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.948+0000 7fbbb9000640 1 --2- 172.21.15.83:0/4292822118 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x560549ac5170 0x560549ac7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:41.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.948+0000 7fbbb9000640 1 --2- 172.21.15.83:0/4292822118 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x560549ac5170 0x560549ac7560 crc :-1 s=READY pgs=592 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e791ce668cd0119 server_cookie=af58c3f51bf1c4df in_seq=0 out_seq=0 2024-04-24T09:15:41.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.948+0000 7fbb9aa00640 10 client.23769 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:15:41.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.950+0000 7fbb9aa00640 1 -- 172.21.15.83:0/4292822118 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5605499b0700 con 0x560549ac5170 2024-04-24T09:15:41.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.950+0000 7fbb9aa00640 10 client.23769 handle_command_reply: tid=0 2024-04-24T09:15:41.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.950+0000 7fbbbe808740 1 client.23769 shutdown 2024-04-24T09:15:41.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.950+0000 7fbbbe808740 1 -- 172.21.15.83:0/4292822118 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560549a4e260 msgr2=0x560549a48ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:41.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.950+0000 7fbbbe808740 1 --2- 172.21.15.83:0/4292822118 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560549a4e260 0x560549a48ac0 secure :-1 s=READY pgs=2745 cs=0 l=1 rev1=1 crypto rx=0x7fbbac009bf0 tx=0x7fbbac02a950 comp rx=0 tx=0).stop 2024-04-24T09:15:41.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.951+0000 7fbbbe808740 1 -- 172.21.15.83:0/4292822118 shutdown_connections 2024-04-24T09:15:41.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.951+0000 7fbbbe808740 1 --2- 172.21.15.83:0/4292822118 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x560549ac5170 0x560549ac7560 crc :-1 s=READY pgs=592 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:41.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.951+0000 7fbbbe808740 1 --2- 172.21.15.83:0/4292822118 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560549a4e260 0x560549a48ac0 unknown :-1 s=CLOSED pgs=2745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:41.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.951+0000 7fbbbe808740 1 --2- 172.21.15.83:0/4292822118 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560549a43b90 0x560549a48580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:41.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.951+0000 7fbbbe808740 1 --2- 172.21.15.83:0/4292822118 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560549a43270 0x560549a48040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:41.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.951+0000 7fbbbe808740 1 -- 172.21.15.83:0/4292822118 >> 172.21.15.83:0/4292822118 conn(0x560549a3f590 msgr2=0x560549a4b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:41.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.951+0000 7fbb9aa00640 0 client.23769 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:15:41.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.951+0000 7fbbbe808740 1 -- 172.21.15.83:0/4292822118 shutdown_connections 2024-04-24T09:15:41.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.951+0000 7fbbbe808740 1 -- 172.21.15.83:0/4292822118 wait complete. 2024-04-24T09:15:41.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.951+0000 7fbbbe808740 20 client.23769 trim_cache size 0 max 16384 2024-04-24T09:15:41.951 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T09:15:41.952 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-04-24T09:15:41.952 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "e20170f4-34cb-4e1d-af18-ad695e3b4c53", 2024-04-24T09:15:41.952 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-04-24T09:15:41.952 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T09:15:41.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.952+0000 7fbbba400640 1 -- 172.21.15.83:0/2183075627 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fbb9c0a3da0 msgr2=0x7fbb9c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:41.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.952+0000 7fbbba400640 1 --2- 172.21.15.83:0/2183075627 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fbb9c0a3da0 0x7fbb9c0a6260 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x560549a44830 tx=0x7fbba800d500 comp rx=0 tx=0).stop 2024-04-24T09:15:41.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.952+0000 7fbbba400640 1 -- 172.21.15.83:0/2183075627 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbb4103980 msgr2=0x7fbbb4198ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:41.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.952+0000 7fbbba400640 1 --2- 172.21.15.83:0/2183075627 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbb4103980 0x7fbbb4198ba0 secure :-1 s=READY pgs=2742 cs=0 l=1 rev1=1 crypto rx=0x7fbba000c970 tx=0x7fbba000ce30 comp rx=0 tx=0).stop 2024-04-24T09:15:41.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.952+0000 7fbbba400640 1 -- 172.21.15.83:0/2183075627 shutdown_connections 2024-04-24T09:15:41.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.952+0000 7fbbba400640 1 --2- 172.21.15.83:0/2183075627 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fbb9c0a3da0 0x7fbb9c0a6260 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:41.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.952+0000 7fbbba400640 1 --2- 172.21.15.83:0/2183075627 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbbb410c480 0x7fbbb419ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:41.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.952+0000 7fbbba400640 1 --2- 172.21.15.83:0/2183075627 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbb4103980 0x7fbbb4198ba0 unknown :-1 s=CLOSED pgs=2742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:41.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.952+0000 7fbbba400640 1 --2- 172.21.15.83:0/2183075627 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbbb4103060 0x7fbbb4198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:41.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.952+0000 7fbbba400640 1 -- 172.21.15.83:0/2183075627 >> 172.21.15.83:0/2183075627 conn(0x7fbbb40fc880 msgr2=0x7fbbb40fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:41.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.953+0000 7fbbba400640 1 -- 172.21.15.83:0/2183075627 shutdown_connections 2024-04-24T09:15:41.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:41.953+0000 7fbbba400640 1 -- 172.21.15.83:0/2183075627 wait complete. 2024-04-24T09:15:41.975 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T09:15:41.975 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T09:15:42.214 INFO:tasks.workunit.client.0.smithi083.stdout: 10 75261 6.54 MB/sec execute 493 sec latency 108.334 ms 2024-04-24T09:15:42.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.264+0000 7f08bfe00640 1 -- 172.21.15.83:0/596236409 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f08c0102110 msgr2=0x7f08c01024f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:42.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.264+0000 7f08bfe00640 1 --2- 172.21.15.83:0/596236409 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f08c0102110 0x7f08c01024f0 secure :-1 s=READY pgs=2746 cs=0 l=1 rev1=1 crypto rx=0x7f08b0009860 tx=0x7f08b002a340 comp rx=0 tx=0).stop 2024-04-24T09:15:42.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.264+0000 7f08bfe00640 1 -- 172.21.15.83:0/596236409 shutdown_connections 2024-04-24T09:15:42.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.264+0000 7f08bfe00640 1 --2- 172.21.15.83:0/596236409 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f08c01071e0 0x7f08c010edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:42.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.264+0000 7f08bfe00640 1 --2- 172.21.15.83:0/596236409 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f08c0102ac0 0x7f08c0106b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:42.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.264+0000 7f08bfe00640 1 --2- 172.21.15.83:0/596236409 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f08c0102110 0x7f08c01024f0 unknown :-1 s=CLOSED pgs=2746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:42.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.264+0000 7f08bfe00640 1 -- 172.21.15.83:0/596236409 >> 172.21.15.83:0/596236409 conn(0x7f08c00fd9d0 msgr2=0x7f08c00ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:42.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.265+0000 7f08bfe00640 1 -- 172.21.15.83:0/596236409 shutdown_connections 2024-04-24T09:15:42.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.265+0000 7f08bfe00640 1 -- 172.21.15.83:0/596236409 wait complete. 2024-04-24T09:15:42.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.266+0000 7f08bfe00640 1 Processor -- start 2024-04-24T09:15:42.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.266+0000 7f08bfe00640 1 -- start start 2024-04-24T09:15:42.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.267+0000 7f08bfe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f08c0102110 0x7f08c010e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:42.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.267+0000 7f08bfe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f08c01071e0 0x7f08c010e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:42.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.267+0000 7f08bfe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f08c01a3680 0x7f08c01a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:42.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.267+0000 7f08bfe00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f08c0112140 con 0x7f08c01a3680 2024-04-24T09:15:42.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.267+0000 7f08bfe00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f08c0111fc0 con 0x7f08c01071e0 2024-04-24T09:15:42.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.267+0000 7f08bfe00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f08c01122c0 con 0x7f08c0102110 2024-04-24T09:15:42.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.267+0000 7f08bf400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f08c01a3680 0x7f08c01a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:42.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.267+0000 7f08be000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f08c01071e0 0x7f08c010e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:42.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.267+0000 7f08bf400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f08c01a3680 0x7f08c01a3b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43440/0 (socket says 172.21.15.83:43440) 2024-04-24T09:15:42.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.267+0000 7f08bf400640 1 -- 172.21.15.83:0/271456355 learned_addr learned my addr 172.21.15.83:0/271456355 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:15:42.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.267+0000 7f08bea00640 1 --2- 172.21.15.83:0/271456355 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f08c0102110 0x7f08c010e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:42.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.268+0000 7f08bf400640 1 -- 172.21.15.83:0/271456355 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f08c0102110 msgr2=0x7f08c010e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:42.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.268+0000 7f08bf400640 1 --2- 172.21.15.83:0/271456355 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f08c0102110 0x7f08c010e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:42.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.268+0000 7f08bf400640 1 -- 172.21.15.83:0/271456355 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f08c01071e0 msgr2=0x7f08c010e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:42.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.268+0000 7f08bf400640 1 --2- 172.21.15.83:0/271456355 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f08c01071e0 0x7f08c010e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:42.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.268+0000 7f08bf400640 1 -- 172.21.15.83:0/271456355 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08c01a41a0 con 0x7f08c01a3680 2024-04-24T09:15:42.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.268+0000 7f08bf400640 1 --2- 172.21.15.83:0/271456355 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f08c01a3680 0x7f08c01a3b30 secure :-1 s=READY pgs=2747 cs=0 l=1 rev1=1 crypto rx=0x7f08b800efc0 tx=0x7f08b800c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:42.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.268+0000 7f08b7400640 1 -- 172.21.15.83:0/271456355 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b8003970 con 0x7f08c01a3680 2024-04-24T09:15:42.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.268+0000 7f08bfe00640 1 -- 172.21.15.83:0/271456355 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08c01a4490 con 0x7f08c01a3680 2024-04-24T09:15:42.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.268+0000 7f08b7400640 1 -- 172.21.15.83:0/271456355 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f08b8004020 con 0x7f08c01a3680 2024-04-24T09:15:42.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.268+0000 7f08b7400640 1 -- 172.21.15.83:0/271456355 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b800b940 con 0x7f08c01a3680 2024-04-24T09:15:42.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.268+0000 7f08bfe00640 1 -- 172.21.15.83:0/271456355 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08c01abc80 con 0x7f08c01a3680 2024-04-24T09:15:42.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.273+0000 7f08c8631740 1 Processor -- start 2024-04-24T09:15:42.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.275+0000 7f08b7400640 1 -- 172.21.15.83:0/271456355 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f08b801d020 con 0x7f08c01a3680 2024-04-24T09:15:42.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.275+0000 7f08c8631740 1 -- start start 2024-04-24T09:15:42.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.275+0000 7f08c8631740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5609ddc55a30 0x5609ddc55e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:42.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.275+0000 7f08c8631740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5609ddc56350 0x5609ddc604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:42.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.275+0000 7f08c8631740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5609ddc60a20 0x5609ddc62e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:42.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.275+0000 7f08c8631740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5609ddbbb1f0 con 0x5609ddc60a20 2024-04-24T09:15:42.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.275+0000 7f08c8631740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5609ddb61350 con 0x5609ddc56350 2024-04-24T09:15:42.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.275+0000 7f08c8631740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5609ddbb71b0 con 0x5609ddc55a30 2024-04-24T09:15:42.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.276+0000 7f08be000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5609ddc55a30 0x5609ddc55e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:42.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.276+0000 7f08be000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5609ddc55a30 0x5609ddc55e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:34134/0 (socket says 172.21.15.83:34134) 2024-04-24T09:15:42.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.276+0000 7f08be000640 1 -- 172.21.15.83:0/3532598469 learned_addr learned my addr 172.21.15.83:0/3532598469 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:15:42.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.276+0000 7f08be000640 1 --2- 172.21.15.83:0/3532598469 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5609ddc60a20 0x5609ddc62e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:42.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.277+0000 7f08b7400640 1 --2- 172.21.15.83:0/271456355 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f08a40a3f20 0x7f08a40a63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:42.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.277+0000 7f08be000640 1 -- 172.21.15.83:0/3532598469 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5609ddc56350 msgr2=0x5609ddc604e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:42.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.277+0000 7f08be000640 1 --2- 172.21.15.83:0/3532598469 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5609ddc56350 0x5609ddc604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:42.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.277+0000 7f08be000640 1 -- 172.21.15.83:0/3532598469 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5609ddc60a20 msgr2=0x5609ddc62e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:42.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.277+0000 7f08be000640 1 --2- 172.21.15.83:0/3532598469 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5609ddc60a20 0x5609ddc62e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:42.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.277+0000 7f08be000640 1 -- 172.21.15.83:0/3532598469 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609ddbb5f50 con 0x5609ddc55a30 2024-04-24T09:15:42.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.277+0000 7f08b7400640 1 -- 172.21.15.83:0/271456355 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 9583+0+0 (secure 0 0 0) 0x7f08b800b6e0 con 0x7f08c01a3680 2024-04-24T09:15:42.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.278+0000 7f08be000640 1 --2- 172.21.15.83:0/3532598469 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5609ddc55a30 0x5609ddc55e10 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f08ac012750 tx=0x7f08ac005c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9cc1e25d06e37238 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:42.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.278+0000 7f08bf400640 1 --2- 172.21.15.83:0/271456355 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f08a40a3f20 0x7f08a40a63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:42.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.279+0000 7f08abe00640 1 -- 172.21.15.83:0/3532598469 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08ac004030 con 0x5609ddc55a30 2024-04-24T09:15:42.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.279+0000 7f08abe00640 1 -- 172.21.15.83:0/3532598469 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f08ac0054d0 con 0x5609ddc55a30 2024-04-24T09:15:42.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.279+0000 7f08abe00640 1 -- 172.21.15.83:0/3532598469 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08ac004a00 con 0x5609ddc55a30 2024-04-24T09:15:42.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.280+0000 7f08c8631740 1 -- 172.21.15.83:0/3532598469 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5609ddc55a30 msgr2=0x5609ddc55e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:42.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.280+0000 7f08c8631740 1 --2- 172.21.15.83:0/3532598469 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5609ddc55a30 0x5609ddc55e10 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f08ac012750 tx=0x7f08ac005c90 comp rx=0 tx=0).stop 2024-04-24T09:15:42.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.280+0000 7f08bf400640 1 --2- 172.21.15.83:0/271456355 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f08a40a3f20 0x7f08a40a63e0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f08b80a4b00 tx=0x7f08b8004a40 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:42.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.280+0000 7f08c8631740 1 -- 172.21.15.83:0/3532598469 shutdown_connections 2024-04-24T09:15:42.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.280+0000 7f08c8631740 1 --2- 172.21.15.83:0/3532598469 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5609ddc60a20 0x5609ddc62e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:42.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.280+0000 7f08c8631740 1 --2- 172.21.15.83:0/3532598469 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5609ddc56350 0x5609ddc604e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:42.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.280+0000 7f08c8631740 1 --2- 172.21.15.83:0/3532598469 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5609ddc55a30 0x5609ddc55e10 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:42.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.280+0000 7f08c8631740 1 -- 172.21.15.83:0/3532598469 >> 172.21.15.83:0/3532598469 conn(0x5609ddc51d50 msgr2=0x5609ddc52130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:42.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.281+0000 7f08c8631740 1 -- 172.21.15.83:0/3532598469 shutdown_connections 2024-04-24T09:15:42.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.281+0000 7f08c8631740 1 -- 172.21.15.83:0/3532598469 wait complete. 2024-04-24T09:15:42.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.281+0000 7f08c8631740 1 -- start start 2024-04-24T09:15:42.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.282+0000 7f08c8631740 1 Processor -- start 2024-04-24T09:15:42.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.283+0000 7f08c8631740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5609ddc55a30 0x5609ddc5da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:42.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.283+0000 7f08c8631740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5609ddc56350 0x5609ddc5dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:42.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.283+0000 7f08c8631740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5609ddc60a20 0x5609ddc5e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:42.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.283+0000 7f08c8631740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5609ddc63810 con 0x5609ddc60a20 2024-04-24T09:15:42.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.283+0000 7f08c8631740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5609ddb74690 con 0x5609ddc56350 2024-04-24T09:15:42.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.283+0000 7f08c8631740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5609ddc63990 con 0x5609ddc55a30 2024-04-24T09:15:42.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.283+0000 7f08c8631740 10 client.0 _ll_register_callbacks cb 0x5609ddbb0650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:15:42.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.284+0000 7f08bea00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5609ddc56350 0x5609ddc5dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:42.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.285+0000 7f08be000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5609ddc55a30 0x5609ddc5da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:42.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.285+0000 7f08be000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5609ddc55a30 0x5609ddc5da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:34136/0 (socket says 172.21.15.83:34136) 2024-04-24T09:15:42.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.285+0000 7f08be000640 1 -- 172.21.15.83:0/2812309197 learned_addr learned my addr 172.21.15.83:0/2812309197 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:15:42.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.285+0000 7f08be000640 1 --2- 172.21.15.83:0/2812309197 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5609ddc60a20 0x5609ddc5e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:42.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.286+0000 7f08be000640 1 -- 172.21.15.83:0/2812309197 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5609ddc56350 msgr2=0x5609ddc5dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:42.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.286+0000 7f08be000640 1 --2- 172.21.15.83:0/2812309197 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5609ddc56350 0x5609ddc5dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:42.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.286+0000 7f08be000640 1 -- 172.21.15.83:0/2812309197 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5609ddc60a20 msgr2=0x5609ddc5e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:42.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.286+0000 7f08be000640 1 --2- 172.21.15.83:0/2812309197 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5609ddc60a20 0x5609ddc5e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:42.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.286+0000 7f08be000640 1 -- 172.21.15.83:0/2812309197 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609ddb73920 con 0x5609ddc55a30 2024-04-24T09:15:42.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.287+0000 7f08be000640 1 --2- 172.21.15.83:0/2812309197 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5609ddc55a30 0x5609ddc5da60 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f08ac0057c0 tx=0x7f08ac00b5b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a709a23c36f645e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:42.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.287+0000 7f08a8c00640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-04-24T09:15:42.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.287+0000 7f08a8c00640 1 -- 172.21.15.83:0/2812309197 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08ac028040 con 0x5609ddc55a30 2024-04-24T09:15:42.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.287+0000 7f08a8c00640 1 -- 172.21.15.83:0/2812309197 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(130..130 src has 1..130) ==== 9583+0+0 (secure 0 0 0) 0x7f08ac0048c0 con 0x5609ddc55a30 2024-04-24T09:15:42.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.287+0000 7f08c8631740 1 -- 172.21.15.83:0/2812309197 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08ac0054d0 con 0x5609ddc55a30 2024-04-24T09:15:42.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.288+0000 7f08a8c00640 1 -- 172.21.15.83:0/2812309197 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f08ac035050 con 0x5609ddc55a30 2024-04-24T09:15:42.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.288+0000 7f08c8631740 10 client.30550 fetch_fsmap learned FSMap version 69 2024-04-24T09:15:42.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.288+0000 7f08c8631740 1 -- 172.21.15.83:0/2812309197 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x5609dd9977d0 con 0x5609ddc55a30 2024-04-24T09:15:42.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.288+0000 7f08a8c00640 1 -- 172.21.15.83:0/2812309197 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f08ac0054d0 con 0x5609ddc55a30 2024-04-24T09:15:42.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.289+0000 7f08a8c00640 1 -- 172.21.15.83:0/2812309197 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 69) ==== 8370+0+0 (secure 0 0 0) 0x7f08ac02e4e0 con 0x5609ddc55a30 2024-04-24T09:15:42.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.289+0000 7f08c8631740 10 client.30550 fetch_fsmap finished waiting for FSMap version 69 2024-04-24T09:15:42.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.289+0000 7f08c8631740 10 client.30550 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:15:42.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.289+0000 7f08c8631740 20 client.30550 populate_metadata read hostname 'smithi083' 2024-04-24T09:15:42.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.289+0000 7f08c8631740 1 --2- 172.21.15.83:0/2812309197 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5609ddc81d00 0x5609ddc840f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:42.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.289+0000 7f08c8631740 4 client.30550 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:15:42.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.289+0000 7f08c8631740 1 -- 172.21.15.83:0/2812309197 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5609ddbc8c10 con 0x5609ddc81d00 2024-04-24T09:15:42.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.289+0000 7f08bea00640 1 --2- 172.21.15.83:0/2812309197 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5609ddc81d00 0x5609ddc840f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:42.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.290+0000 7f08bea00640 1 --2- 172.21.15.83:0/2812309197 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5609ddc81d00 0x5609ddc840f0 crc :-1 s=READY pgs=593 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4917a08fb7ff298d server_cookie=c1669ab768d7b4c4 in_seq=0 out_seq=0 2024-04-24T09:15:42.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.290+0000 7f08a8c00640 10 client.30550 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:15:42.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.292+0000 7f08a8c00640 1 -- 172.21.15.83:0/2812309197 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5609ddbc8c10 con 0x5609ddc81d00 2024-04-24T09:15:42.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.292+0000 7f08a8c00640 10 client.30550 handle_command_reply: tid=0 2024-04-24T09:15:42.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.292+0000 7f08c8631740 1 client.30550 shutdown 2024-04-24T09:15:42.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.292+0000 7f08c8631740 1 -- 172.21.15.83:0/2812309197 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5609ddc55a30 msgr2=0x5609ddc5da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:42.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.292+0000 7f08c8631740 1 --2- 172.21.15.83:0/2812309197 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5609ddc55a30 0x5609ddc5da60 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f08ac0057c0 tx=0x7f08ac00b5b0 comp rx=0 tx=0).stop 2024-04-24T09:15:42.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.293+0000 7f08c8631740 1 -- 172.21.15.83:0/2812309197 shutdown_connections 2024-04-24T09:15:42.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.293+0000 7f08c8631740 1 --2- 172.21.15.83:0/2812309197 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5609ddc81d00 0x5609ddc840f0 crc :-1 s=READY pgs=593 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:42.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.293+0000 7f08a8c00640 0 client.30550 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:15:42.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.293+0000 7f08c8631740 1 --2- 172.21.15.83:0/2812309197 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5609ddc60a20 0x5609ddc5e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:42.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.293+0000 7f08c8631740 1 --2- 172.21.15.83:0/2812309197 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5609ddc56350 0x5609ddc5dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:42.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.293+0000 7f08c8631740 1 --2- 172.21.15.83:0/2812309197 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5609ddc55a30 0x5609ddc5da60 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:42.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.293+0000 7f08c8631740 1 -- 172.21.15.83:0/2812309197 >> 172.21.15.83:0/2812309197 conn(0x5609ddc51d50 msgr2=0x5609ddc57b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:42.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.293+0000 7f08c8631740 1 -- 172.21.15.83:0/2812309197 shutdown_connections 2024-04-24T09:15:42.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.293+0000 7f08c8631740 1 -- 172.21.15.83:0/2812309197 wait complete. 2024-04-24T09:15:42.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.293+0000 7f08c8631740 20 client.30550 trim_cache size 0 max 16384 2024-04-24T09:15:42.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:15:42 smithi083 ceph-mon[27907]: pgmap v761: 129 pgs: 129 active+clean; 1.1 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.3 MiB/s wr, 194 op/s 2024-04-24T09:15:42.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:15:42 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:15:42.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.325+0000 7f08c8631740 1 Processor -- start 2024-04-24T09:15:42.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.325+0000 7f08c8631740 1 -- start start 2024-04-24T09:15:42.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.325+0000 7f08c8631740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5609ddc55a30 0x5609ddc6c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:42.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.325+0000 7f08c8631740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5609ddc56350 0x5609ddc5e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:42.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.325+0000 7f08c8631740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5609ddc60a20 0x5609ddc89720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:42.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.325+0000 7f08c8631740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f08c01ad260 con 0x5609ddc60a20 2024-04-24T09:15:42.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.325+0000 7f08c8631740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f08c01aee00 con 0x5609ddc56350 2024-04-24T09:15:42.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.325+0000 7f08c8631740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f08c01ad930 con 0x5609ddc55a30 2024-04-24T09:15:42.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.326+0000 7f08be000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5609ddc60a20 0x5609ddc89720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:42.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.326+0000 7f08be000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5609ddc55a30 0x5609ddc6c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:42.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.326+0000 7f08bea00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5609ddc56350 0x5609ddc5e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:42.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.326+0000 7f08be000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5609ddc55a30 0x5609ddc6c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:34144/0 (socket says 172.21.15.83:34144) 2024-04-24T09:15:42.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.326+0000 7f08be000640 1 -- 172.21.15.83:0/3531374098 learned_addr learned my addr 172.21.15.83:0/3531374098 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:15:42.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.327+0000 7f08bea00640 1 -- 172.21.15.83:0/3531374098 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5609ddc55a30 msgr2=0x5609ddc6c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:42.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.327+0000 7f08bea00640 1 --2- 172.21.15.83:0/3531374098 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5609ddc55a30 0x5609ddc6c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:42.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.327+0000 7f08bea00640 1 -- 172.21.15.83:0/3531374098 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5609ddc60a20 msgr2=0x5609ddc89720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:42.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.327+0000 7f08bea00640 1 --2- 172.21.15.83:0/3531374098 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5609ddc60a20 0x5609ddc89720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:42.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.327+0000 7f08bea00640 1 -- 172.21.15.83:0/3531374098 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609dd8cd3f0 con 0x5609ddc56350 2024-04-24T09:15:42.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.327+0000 7f08be000640 1 --2- 172.21.15.83:0/3531374098 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5609ddc60a20 0x5609ddc89720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:15:42.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.327+0000 7f08bea00640 1 --2- 172.21.15.83:0/3531374098 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5609ddc56350 0x5609ddc5e340 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f08b002a860 tx=0x7f08b002a8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=369b6dd45222383d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:42.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.327+0000 7f08ab400640 1 -- 172.21.15.83:0/3531374098 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b0047070 con 0x5609ddc56350 2024-04-24T09:15:42.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.328+0000 7f08ab400640 1 -- 172.21.15.83:0/3531374098 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f08b0005040 con 0x5609ddc56350 2024-04-24T09:15:42.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.328+0000 7f08ab400640 1 -- 172.21.15.83:0/3531374098 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b0037040 con 0x5609ddc56350 2024-04-24T09:15:42.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.328+0000 7f08c8631740 1 -- 172.21.15.83:0/3531374098 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5609ddc56350 msgr2=0x5609ddc5e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:42.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.329+0000 7f08c8631740 1 --2- 172.21.15.83:0/3531374098 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5609ddc56350 0x5609ddc5e340 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f08b002a860 tx=0x7f08b002a8e0 comp rx=0 tx=0).stop 2024-04-24T09:15:42.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.329+0000 7f08c8631740 1 -- 172.21.15.83:0/3531374098 shutdown_connections 2024-04-24T09:15:42.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.329+0000 7f08c8631740 1 --2- 172.21.15.83:0/3531374098 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5609ddc60a20 0x5609ddc89720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:42.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.329+0000 7f08c8631740 1 --2- 172.21.15.83:0/3531374098 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5609ddc56350 0x5609ddc5e340 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:42.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.329+0000 7f08c8631740 1 --2- 172.21.15.83:0/3531374098 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5609ddc55a30 0x5609ddc6c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:42.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.329+0000 7f08c8631740 1 -- 172.21.15.83:0/3531374098 >> 172.21.15.83:0/3531374098 conn(0x5609ddc51d50 msgr2=0x5609ddc50fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:42.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.329+0000 7f08c8631740 1 -- 172.21.15.83:0/3531374098 shutdown_connections 2024-04-24T09:15:42.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.329+0000 7f08c8631740 1 -- 172.21.15.83:0/3531374098 wait complete. 2024-04-24T09:15:42.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.329+0000 7f08c8631740 1 -- start start 2024-04-24T09:15:42.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.330+0000 7f08c8631740 1 Processor -- start 2024-04-24T09:15:42.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.330+0000 7f08c8631740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5609ddc55a30 0x5609ddc68e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:42.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.330+0000 7f08c8631740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5609ddc56350 0x5609ddc661d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:42.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.330+0000 7f08c8631740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5609ddc60a20 0x5609ddc66710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:42.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.330+0000 7f08c8631740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f08c01ae730 con 0x5609ddc60a20 2024-04-24T09:15:42.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.330+0000 7f08c8631740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f08c01ae060 con 0x5609ddc55a30 2024-04-24T09:15:42.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.330+0000 7f08c8631740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f08c0073520 con 0x5609ddc56350 2024-04-24T09:15:42.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.330+0000 7f08c8631740 10 client.0 _ll_register_callbacks cb 0x7f08ac00bd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:15:42.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.331+0000 7f08be000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5609ddc55a30 0x5609ddc68e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:42.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.331+0000 7f08be000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5609ddc60a20 0x5609ddc66710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:42.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.331+0000 7f08bea00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5609ddc56350 0x5609ddc661d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:42.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.331+0000 7f08be000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5609ddc60a20 0x5609ddc66710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43482/0 (socket says 172.21.15.83:43482) 2024-04-24T09:15:42.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.331+0000 7f08be000640 1 -- 172.21.15.83:0/3911099426 learned_addr learned my addr 172.21.15.83:0/3911099426 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:15:42.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.332+0000 7f08be000640 1 -- 172.21.15.83:0/3911099426 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5609ddc56350 msgr2=0x5609ddc661d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:42.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.332+0000 7f08be000640 1 --2- 172.21.15.83:0/3911099426 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5609ddc56350 0x5609ddc661d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:42.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.332+0000 7f08be000640 1 -- 172.21.15.83:0/3911099426 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5609ddc55a30 msgr2=0x5609ddc68e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:42.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.332+0000 7f08be000640 1 --2- 172.21.15.83:0/3911099426 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5609ddc55a30 0x5609ddc68e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:42.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.332+0000 7f08be000640 1 -- 172.21.15.83:0/3911099426 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609dd88d0f0 con 0x5609ddc60a20 2024-04-24T09:15:42.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.332+0000 7f08be000640 1 --2- 172.21.15.83:0/3911099426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5609ddc60a20 0x5609ddc66710 secure :-1 s=READY pgs=2748 cs=0 l=1 rev1=1 crypto rx=0x7f08ac006010 tx=0x7f08ac00b670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0494927d5a2141d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:42.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.332+0000 7f08a8c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:15:42.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.332+0000 7f08a8c00640 1 -- 172.21.15.83:0/3911099426 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08ac005080 con 0x5609ddc60a20 2024-04-24T09:15:42.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.333+0000 7f08a8c00640 1 -- 172.21.15.83:0/3911099426 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(130..130 src has 1..130) ==== 9583+0+0 (secure 0 0 0) 0x7f08ac02e3c0 con 0x5609ddc60a20 2024-04-24T09:15:42.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.333+0000 7f08c8631740 1 -- 172.21.15.83:0/3911099426 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08b0005040 con 0x5609ddc60a20 2024-04-24T09:15:42.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.333+0000 7f08a8c00640 1 -- 172.21.15.83:0/3911099426 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f08ac040050 con 0x5609ddc60a20 2024-04-24T09:15:42.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.333+0000 7f08c8631740 10 client.23787 fetch_fsmap learned FSMap version 69 2024-04-24T09:15:42.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.333+0000 7f08c8631740 1 -- 172.21.15.83:0/3911099426 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x5609ddb80810 con 0x5609ddc60a20 2024-04-24T09:15:42.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.334+0000 7f08a8c00640 1 -- 172.21.15.83:0/3911099426 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f08b0005040 con 0x5609ddc60a20 2024-04-24T09:15:42.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.334+0000 7f08a8c00640 1 -- 172.21.15.83:0/3911099426 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 69) ==== 8370+0+0 (secure 0 0 0) 0x7f08ac0035f0 con 0x5609ddc60a20 2024-04-24T09:15:42.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.334+0000 7f08c8631740 10 client.23787 fetch_fsmap finished waiting for FSMap version 69 2024-04-24T09:15:42.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.334+0000 7f08c8631740 10 client.23787 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:15:42.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.334+0000 7f08c8631740 20 client.23787 populate_metadata read hostname 'smithi083' 2024-04-24T09:15:42.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.334+0000 7f08c8631740 1 --2- 172.21.15.83:0/3911099426 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5609ddce81c0 0x5609ddcea5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:42.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.334+0000 7f08c8631740 4 client.23787 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T09:15:42.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.334+0000 7f08c8631740 1 -- 172.21.15.83:0/3911099426 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5609ddbc2a80 con 0x5609ddce81c0 2024-04-24T09:15:42.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.335+0000 7f08bea00640 1 --2- 172.21.15.83:0/3911099426 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5609ddce81c0 0x5609ddcea5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:42.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.335+0000 7f08bea00640 1 --2- 172.21.15.83:0/3911099426 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5609ddce81c0 0x5609ddcea5e0 crc :-1 s=READY pgs=594 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fdeb8a6a6dee58e1 server_cookie=e75f3448994afc9 in_seq=0 out_seq=0 2024-04-24T09:15:42.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.335+0000 7f08a8c00640 10 client.23787 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:15:42.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.336+0000 7f08a8c00640 1 -- 172.21.15.83:0/3911099426 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5609ddbc2a80 con 0x5609ddce81c0 2024-04-24T09:15:42.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.336+0000 7f08a8c00640 10 client.23787 handle_command_reply: tid=0 2024-04-24T09:15:42.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.336+0000 7f08c8631740 1 client.23787 shutdown 2024-04-24T09:15:42.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.336+0000 7f08c8631740 1 -- 172.21.15.83:0/3911099426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5609ddc60a20 msgr2=0x5609ddc66710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:42.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.336+0000 7f08c8631740 1 --2- 172.21.15.83:0/3911099426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5609ddc60a20 0x5609ddc66710 secure :-1 s=READY pgs=2748 cs=0 l=1 rev1=1 crypto rx=0x7f08ac006010 tx=0x7f08ac00b670 comp rx=0 tx=0).stop 2024-04-24T09:15:42.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.336+0000 7f08c8631740 1 -- 172.21.15.83:0/3911099426 shutdown_connections 2024-04-24T09:15:42.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.336+0000 7f08c8631740 1 --2- 172.21.15.83:0/3911099426 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5609ddce81c0 0x5609ddcea5e0 crc :-1 s=READY pgs=594 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:42.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.336+0000 7f08c8631740 1 --2- 172.21.15.83:0/3911099426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5609ddc60a20 0x5609ddc66710 unknown :-1 s=CLOSED pgs=2748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:42.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.336+0000 7f08c8631740 1 --2- 172.21.15.83:0/3911099426 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5609ddc56350 0x5609ddc661d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:42.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.336+0000 7f08c8631740 1 --2- 172.21.15.83:0/3911099426 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5609ddc55a30 0x5609ddc68e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:42.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.336+0000 7f08c8631740 1 -- 172.21.15.83:0/3911099426 >> 172.21.15.83:0/3911099426 conn(0x5609ddc51d50 msgr2=0x5609ddc5fb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:42.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.336+0000 7f08a8c00640 0 client.23787 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:15:42.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.337+0000 7f08c8631740 1 -- 172.21.15.83:0/3911099426 shutdown_connections 2024-04-24T09:15:42.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.337+0000 7f08c8631740 1 -- 172.21.15.83:0/3911099426 wait complete. 2024-04-24T09:15:42.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.337+0000 7f08c8631740 20 client.23787 trim_cache size 0 max 16384 2024-04-24T09:15:42.337 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T09:15:42.337 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T09:15:42.338 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-04-24T09:15:42.338 INFO:teuthology.orchestra.run.smithi083.stdout: "e20170f4-34cb-4e1d-af18-ad695e3b4c53": { 2024-04-24T09:15:42.338 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-04-24T09:15:42.338 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "e20170f4-34cb-4e1d-af18-ad695e3b4c53", 2024-04-24T09:15:42.338 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-04-24T09:15:42.339 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:15:42.339 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:15:42.339 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T09:15:42.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.338+0000 7f08bfe00640 1 -- 172.21.15.83:0/271456355 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f08a40a3f20 msgr2=0x7f08a40a63e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:42.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.338+0000 7f08bfe00640 1 --2- 172.21.15.83:0/271456355 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f08a40a3f20 0x7f08a40a63e0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f08b80a4b00 tx=0x7f08b8004a40 comp rx=0 tx=0).stop 2024-04-24T09:15:42.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.338+0000 7f08bfe00640 1 -- 172.21.15.83:0/271456355 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f08c01a3680 msgr2=0x7f08c01a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:42.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.338+0000 7f08bfe00640 1 --2- 172.21.15.83:0/271456355 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f08c01a3680 0x7f08c01a3b30 secure :-1 s=READY pgs=2747 cs=0 l=1 rev1=1 crypto rx=0x7f08b800efc0 tx=0x7f08b800c800 comp rx=0 tx=0).stop 2024-04-24T09:15:42.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.338+0000 7f08bfe00640 1 -- 172.21.15.83:0/271456355 shutdown_connections 2024-04-24T09:15:42.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.338+0000 7f08bfe00640 1 --2- 172.21.15.83:0/271456355 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f08a40a3f20 0x7f08a40a63e0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:42.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.338+0000 7f08bfe00640 1 --2- 172.21.15.83:0/271456355 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f08c01a3680 0x7f08c01a3b30 unknown :-1 s=CLOSED pgs=2747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:42.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.338+0000 7f08bfe00640 1 --2- 172.21.15.83:0/271456355 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f08c01071e0 0x7f08c010e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:42.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.338+0000 7f08bfe00640 1 --2- 172.21.15.83:0/271456355 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f08c0102110 0x7f08c010e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:42.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.338+0000 7f08bfe00640 1 -- 172.21.15.83:0/271456355 >> 172.21.15.83:0/271456355 conn(0x7f08c00fd9d0 msgr2=0x7f08c00fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:42.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.338+0000 7f08bfe00640 1 -- 172.21.15.83:0/271456355 shutdown_connections 2024-04-24T09:15:42.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:15:42.339+0000 7f08bfe00640 1 -- 172.21.15.83:0/271456355 wait complete. 2024-04-24T09:15:42.361 INFO:tasks.cephfs.filesystem:scrub status for tag:e20170f4-34cb-4e1d-af18-ad695e3b4c53 - {'path': '/', 'tag': 'e20170f4-34cb-4e1d-af18-ad695e3b4c53', 'options': 'recursive,force'} 2024-04-24T09:15:42.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:15:42 smithi138 ceph-mon[31441]: pgmap v761: 129 pgs: 129 active+clean; 1.1 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.3 MiB/s wr, 194 op/s 2024-04-24T09:15:42.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:15:42 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:15:42.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:15:42 smithi189 ceph-mon[31791]: pgmap v761: 129 pgs: 129 active+clean; 1.1 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.3 MiB/s wr, 194 op/s 2024-04-24T09:15:42.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:15:42 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:15:43.214 INFO:tasks.workunit.client.0.smithi083.stdout: 10 75392 6.55 MB/sec execute 494 sec latency 90.153 ms 2024-04-24T09:15:43.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:15:43 smithi138 ceph-mon[31441]: scrub queued for path: / 2024-04-24T09:15:43.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:15:43 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:15:43.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:15:43 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:15:43.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:15:43 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:15:43.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:15:43 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:15:43.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:15:43 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:15:43.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:15:43 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:15:43.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:15:43 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:15:43.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:15:43 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:15:43.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:15:43 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:15:43.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:15:43 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:15:43.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:15:43 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:15:43.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:15:43 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:15:43.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:15:43 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:15:43.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:15:43 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:15:43.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:15:43 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:15:43.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:15:43 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:15:43.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:15:43 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:15:43.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:15:43 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:15:43.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:15:43 smithi189 ceph-mon[31791]: scrub queued for path: / 2024-04-24T09:15:43.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:15:43 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:15:43.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:15:43 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:15:43.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:15:43 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:15:43.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:15:43 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:15:43.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:15:43 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:15:43.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:15:43 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:15:43.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:15:43 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:15:43.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:15:43 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:15:43.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:15:43 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:15:43.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:15:43 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:15:43.459 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:15:43 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:15:43.459 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:15:43 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:15:43.459 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:15:43 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:15:43.459 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:15:43 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:15:43.459 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:15:43 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:15:43.459 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:15:43 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:15:43.459 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:15:43 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:15:43.459 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:15:43 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:15:43.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:15:43 smithi083 ceph-mon[27907]: scrub queued for path: / 2024-04-24T09:15:43.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:15:43 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:15:43.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:15:43 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:15:43.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:15:43 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:15:43.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:15:43 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:15:43.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:15:43 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:15:43.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:15:43 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:15:43.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:15:43 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:15:43.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:15:43 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:15:43.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:15:43 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:15:43.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:15:43 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:15:43.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:15:43 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:15:43.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:15:43 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:15:43.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:15:43 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:15:43.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:15:43 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:15:43.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:15:43 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:15:43.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:15:43 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:15:43.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:15:43 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:15:43.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:15:43 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:15:44.214 INFO:tasks.workunit.client.0.smithi083.stdout: 10 75512 6.54 MB/sec execute 495 sec latency 79.192 ms 2024-04-24T09:15:44.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:15:44 smithi138 ceph-mon[31441]: pgmap v762: 129 pgs: 129 active+clean; 1.1 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.2 MiB/s wr, 381 op/s 2024-04-24T09:15:44.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:15:44 smithi189 ceph-mon[31791]: pgmap v762: 129 pgs: 129 active+clean; 1.1 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.2 MiB/s wr, 381 op/s 2024-04-24T09:15:44.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:15:44 smithi083 ceph-mon[27907]: pgmap v762: 129 pgs: 129 active+clean; 1.1 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.2 MiB/s wr, 381 op/s 2024-04-24T09:15:45.214 INFO:tasks.workunit.client.0.smithi083.stdout: 10 75647 6.55 MB/sec execute 496 sec latency 99.267 ms 2024-04-24T09:15:46.215 INFO:tasks.workunit.client.0.smithi083.stdout: 10 75755 6.54 MB/sec execute 497 sec latency 121.885 ms 2024-04-24T09:15:46.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:15:46 smithi138 ceph-mon[31441]: pgmap v763: 129 pgs: 129 active+clean; 1.1 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.3 MiB/s wr, 295 op/s 2024-04-24T09:15:46.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:15:46 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:15:46.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:15:46 smithi189 ceph-mon[31791]: pgmap v763: 129 pgs: 129 active+clean; 1.1 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.3 MiB/s wr, 295 op/s 2024-04-24T09:15:46.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:15:46 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:15:46.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:15:46 smithi083 ceph-mon[27907]: pgmap v763: 129 pgs: 129 active+clean; 1.1 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.3 MiB/s wr, 295 op/s 2024-04-24T09:15:46.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:15:46 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:15:47.214 INFO:tasks.workunit.client.0.smithi083.stdout: 10 75844 6.54 MB/sec execute 498 sec latency 236.390 ms 2024-04-24T09:15:48.215 INFO:tasks.workunit.client.0.smithi083.stdout: 10 75948 6.53 MB/sec execute 499 sec latency 96.137 ms 2024-04-24T09:15:48.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:15:48 smithi083 ceph-mon[27907]: pgmap v764: 129 pgs: 129 active+clean; 1.1 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.3 MiB/s wr, 295 op/s 2024-04-24T09:15:48.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:15:48 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:15:48.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:15:48 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:15:48.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:15:48 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:15:48.704 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:15:48 smithi138 ceph-mon[31441]: pgmap v764: 129 pgs: 129 active+clean; 1.1 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.3 MiB/s wr, 295 op/s 2024-04-24T09:15:48.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:15:48 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:15:48.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:15:48 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:15:48.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:15:48 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:15:48.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:15:48 smithi189 ceph-mon[31791]: pgmap v764: 129 pgs: 129 active+clean; 1.1 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.3 MiB/s wr, 295 op/s 2024-04-24T09:15:48.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:15:48 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:15:48.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:15:48 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:15:48.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:15:48 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:15:49.215 INFO:tasks.workunit.client.0.smithi083.stdout: 10 76061 6.53 MB/sec execute 500 sec latency 120.673 ms 2024-04-24T09:15:50.215 INFO:tasks.workunit.client.0.smithi083.stdout: 10 76179 6.52 MB/sec execute 501 sec latency 108.920 ms 2024-04-24T09:15:50.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:15:50 smithi083 ceph-mon[27907]: pgmap v765: 129 pgs: 129 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 6.1 MiB/s wr, 387 op/s 2024-04-24T09:15:50.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:15:50 smithi138 ceph-mon[31441]: pgmap v765: 129 pgs: 129 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 6.1 MiB/s wr, 387 op/s 2024-04-24T09:15:50.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:15:50 smithi189 ceph-mon[31791]: pgmap v765: 129 pgs: 129 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 6.1 MiB/s wr, 387 op/s 2024-04-24T09:15:51.215 INFO:tasks.workunit.client.0.smithi083.stdout: 10 76280 6.52 MB/sec execute 502 sec latency 350.481 ms 2024-04-24T09:15:51.486 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.486+0000 7f66eba00640 1 -- 172.21.15.83:0/1505322099 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66e4117090 msgr2=0x7f66e4119480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:51.486 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.486+0000 7f66eba00640 1 --2- 172.21.15.83:0/1505322099 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66e4117090 0x7f66e4119480 secure :-1 s=READY pgs=2749 cs=0 l=1 rev1=1 crypto rx=0x7f66dc009ca0 tx=0x7f66dc02a8c0 comp rx=0 tx=0).stop 2024-04-24T09:15:51.486 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.486+0000 7f66eba00640 1 -- 172.21.15.83:0/1505322099 shutdown_connections 2024-04-24T09:15:51.486 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.486+0000 7f66eba00640 1 --2- 172.21.15.83:0/1505322099 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66e4117090 0x7f66e4119480 unknown :-1 s=CLOSED pgs=2749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:51.486 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.486+0000 7f66eba00640 1 --2- 172.21.15.83:0/1505322099 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f66e4109540 0x7f66e41099a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:51.486 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.486+0000 7f66eba00640 1 --2- 172.21.15.83:0/1505322099 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f66e410c700 0x7f66e4109000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:51.486 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.486+0000 7f66eba00640 1 -- 172.21.15.83:0/1505322099 >> 172.21.15.83:0/1505322099 conn(0x7f66e406d0d0 msgr2=0x7f66e406d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:51.486 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.487+0000 7f66eba00640 1 -- 172.21.15.83:0/1505322099 shutdown_connections 2024-04-24T09:15:51.487 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.487+0000 7f66eba00640 1 -- 172.21.15.83:0/1505322099 wait complete. 2024-04-24T09:15:51.487 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.488+0000 7f66eba00640 1 Processor -- start 2024-04-24T09:15:51.488 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.488+0000 7f66eba00640 1 -- start start 2024-04-24T09:15:51.488 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.489+0000 7f66eba00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66e4109540 0x7f66e41a3120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:51.488 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.489+0000 7f66eba00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f66e410c700 0x7f66e41a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:51.488 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.489+0000 7f66eba00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f66e4117090 0x7f66e41a79e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:51.489 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.489+0000 7f66eba00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f66e411cce0 con 0x7f66e4109540 2024-04-24T09:15:51.489 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.489+0000 7f66eba00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f66e411cb60 con 0x7f66e4117090 2024-04-24T09:15:51.489 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.489+0000 7f66eba00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f66e411ce60 con 0x7f66e410c700 2024-04-24T09:15:51.489 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.489+0000 7f66e9000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66e4109540 0x7f66e41a3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:51.489 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.489+0000 7f66e9000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66e4109540 0x7f66e41a3120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57706/0 (socket says 172.21.15.83:57706) 2024-04-24T09:15:51.489 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.489+0000 7f66e9000640 1 -- 172.21.15.83:0/3523970368 learned_addr learned my addr 172.21.15.83:0/3523970368 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:15:51.489 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.489+0000 7f66e3e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f66e410c700 0x7f66e41a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:51.489 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.490+0000 7f66e9a00640 1 --2- 172.21.15.83:0/3523970368 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f66e4117090 0x7f66e41a79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:51.490 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.490+0000 7f66e9000640 1 -- 172.21.15.83:0/3523970368 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f66e410c700 msgr2=0x7f66e41a3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:51.490 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.490+0000 7f66e9000640 1 --2- 172.21.15.83:0/3523970368 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f66e410c700 0x7f66e41a3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:51.491 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.490+0000 7f66e9000640 1 -- 172.21.15.83:0/3523970368 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f66e4117090 msgr2=0x7f66e41a79e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:51.491 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.490+0000 7f66e9000640 1 --2- 172.21.15.83:0/3523970368 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f66e4117090 0x7f66e41a79e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:51.491 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.490+0000 7f66e9000640 1 -- 172.21.15.83:0/3523970368 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66e41a8110 con 0x7f66e4109540 2024-04-24T09:15:51.491 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.490+0000 7f66e9a00640 1 --2- 172.21.15.83:0/3523970368 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f66e4117090 0x7f66e41a79e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:15:51.491 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.491+0000 7f66e9000640 1 --2- 172.21.15.83:0/3523970368 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66e4109540 0x7f66e41a3120 secure :-1 s=READY pgs=2750 cs=0 l=1 rev1=1 crypto rx=0x7f66d800e9f0 tx=0x7f66d800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:51.491 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.491+0000 7f66e1600640 1 -- 172.21.15.83:0/3523970368 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d800cd80 con 0x7f66e4109540 2024-04-24T09:15:51.491 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.491+0000 7f66e1600640 1 -- 172.21.15.83:0/3523970368 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f66d80108e0 con 0x7f66e4109540 2024-04-24T09:15:51.491 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.491+0000 7f66eba00640 1 -- 172.21.15.83:0/3523970368 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66e41a8400 con 0x7f66e4109540 2024-04-24T09:15:51.491 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.491+0000 7f66e1600640 1 -- 172.21.15.83:0/3523970368 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d8018920 con 0x7f66e4109540 2024-04-24T09:15:51.491 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.491+0000 7f66eba00640 1 -- 172.21.15.83:0/3523970368 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66e41afc90 con 0x7f66e4109540 2024-04-24T09:15:51.495 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.496+0000 7f66ee7de740 1 Processor -- start 2024-04-24T09:15:51.496 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.496+0000 7f66ee7de740 1 -- start start 2024-04-24T09:15:51.496 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.497+0000 7f66ee7de740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556836a5a1b0 0x556836a5a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:51.496 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.497+0000 7f66ee7de740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556836a5aad0 0x556836a64e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:51.496 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.497+0000 7f66ee7de740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556836a65390 0x556836a67780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:51.496 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.497+0000 7f66ee7de740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5568369c0900 con 0x556836a5a1b0 2024-04-24T09:15:51.496 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.497+0000 7f66ee7de740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556836966f20 con 0x556836a65390 2024-04-24T09:15:51.496 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.497+0000 7f66ee7de740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5568369bc8c0 con 0x556836a5aad0 2024-04-24T09:15:51.497 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.498+0000 7f66e9a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556836a5a1b0 0x556836a5a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:51.498 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.498+0000 7f66e9a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556836a5a1b0 0x556836a5a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57722/0 (socket says 172.21.15.83:57722) 2024-04-24T09:15:51.498 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.499+0000 7f66e9a00640 1 -- 172.21.15.83:0/2165445267 learned_addr learned my addr 172.21.15.83:0/2165445267 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:15:51.498 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.499+0000 7f66e1600640 1 -- 172.21.15.83:0/3523970368 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f66d8010a90 con 0x7f66e4109540 2024-04-24T09:15:51.498 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.498+0000 7f66e3e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556836a65390 0x556836a67780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:51.498 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.499+0000 7f66e9000640 1 --2- 172.21.15.83:0/2165445267 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556836a5aad0 0x556836a64e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:51.499 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.500+0000 7f66e3e00640 1 -- 172.21.15.83:0/2165445267 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556836a5aad0 msgr2=0x556836a64e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:51.499 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.500+0000 7f66e3e00640 1 --2- 172.21.15.83:0/2165445267 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556836a5aad0 0x556836a64e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:51.499 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.500+0000 7f66e3e00640 1 -- 172.21.15.83:0/2165445267 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556836a5a1b0 msgr2=0x556836a5a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:51.499 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.500+0000 7f66e3e00640 1 --2- 172.21.15.83:0/2165445267 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556836a5a1b0 0x556836a5a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:51.499 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.500+0000 7f66e3e00640 1 -- 172.21.15.83:0/2165445267 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55683698be10 con 0x556836a65390 2024-04-24T09:15:51.500 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.500+0000 7f66e3e00640 1 --2- 172.21.15.83:0/2165445267 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556836a65390 0x556836a67780 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f66d000bbc0 tx=0x7f66d00107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7149a0556c8d39c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:51.500 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.500+0000 7f66d6000640 1 -- 172.21.15.83:0/2165445267 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d0004030 con 0x556836a65390 2024-04-24T09:15:51.500 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.500+0000 7f66d6000640 1 -- 172.21.15.83:0/2165445267 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f66d00035f0 con 0x556836a65390 2024-04-24T09:15:51.500 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.501+0000 7f66d6000640 1 -- 172.21.15.83:0/2165445267 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d00116a0 con 0x556836a65390 2024-04-24T09:15:51.501 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.501+0000 7f66ee7de740 1 -- 172.21.15.83:0/2165445267 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556836a65390 msgr2=0x556836a67780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:51.501 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.501+0000 7f66ee7de740 1 --2- 172.21.15.83:0/2165445267 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556836a65390 0x556836a67780 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f66d000bbc0 tx=0x7f66d00107a0 comp rx=0 tx=0).stop 2024-04-24T09:15:51.501 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.502+0000 7f66ee7de740 1 -- 172.21.15.83:0/2165445267 shutdown_connections 2024-04-24T09:15:51.501 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.502+0000 7f66ee7de740 1 --2- 172.21.15.83:0/2165445267 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556836a65390 0x556836a67780 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:51.501 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.502+0000 7f66ee7de740 1 --2- 172.21.15.83:0/2165445267 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556836a5aad0 0x556836a64e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:51.501 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.502+0000 7f66ee7de740 1 --2- 172.21.15.83:0/2165445267 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556836a5a1b0 0x556836a5a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:51.501 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.502+0000 7f66ee7de740 1 -- 172.21.15.83:0/2165445267 >> 172.21.15.83:0/2165445267 conn(0x556836a566c0 msgr2=0x556836a56aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:51.501 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.502+0000 7f66ee7de740 1 -- 172.21.15.83:0/2165445267 shutdown_connections 2024-04-24T09:15:51.501 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.502+0000 7f66ee7de740 1 -- 172.21.15.83:0/2165445267 wait complete. 2024-04-24T09:15:51.502 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.502+0000 7f66ee7de740 1 -- start start 2024-04-24T09:15:51.502 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.502+0000 7f66ee7de740 1 Processor -- start 2024-04-24T09:15:51.502 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.503+0000 7f66e1600640 1 --2- 172.21.15.83:0/3523970368 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f66cc0a3dd0 0x7f66cc0a6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:51.502 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.503+0000 7f66ee7de740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556836a5a1b0 0x556836a622f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:51.502 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.503+0000 7f66ee7de740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556836a5aad0 0x556836a62830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:51.502 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.503+0000 7f66ee7de740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556836a65390 0x556836a62d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:51.503 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.503+0000 7f66ee7de740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556836a67fc0 con 0x556836a65390 2024-04-24T09:15:51.503 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.503+0000 7f66ee7de740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55683697a2c0 con 0x556836a5aad0 2024-04-24T09:15:51.503 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.503+0000 7f66ee7de740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x556836a68140 con 0x556836a5a1b0 2024-04-24T09:15:51.503 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.503+0000 7f66ee7de740 10 client.0 _ll_register_callbacks cb 0x5568366d2a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:15:51.503 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.503+0000 7f66e1600640 1 -- 172.21.15.83:0/3523970368 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 9583+0+0 (secure 0 0 0) 0x7f66d80d0e60 con 0x7f66e4109540 2024-04-24T09:15:51.504 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.503+0000 7f66e9a00640 1 --2- 172.21.15.83:0/3523970368 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f66cc0a3dd0 0x7f66cc0a6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:51.504 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.504+0000 7f66e9000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556836a5a1b0 0x556836a622f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:51.504 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.504+0000 7f66e3e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556836a5aad0 0x556836a62830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:51.504 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.504+0000 7f66e3e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556836a5aad0 0x556836a62830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:42696/0 (socket says 172.21.15.83:42696) 2024-04-24T09:15:51.504 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.504+0000 7f66e9a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556836a65390 0x556836a62d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:51.504 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.504+0000 7f66e3e00640 1 -- 172.21.15.83:0/2862949878 learned_addr learned my addr 172.21.15.83:0/2862949878 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:15:51.504 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.504+0000 7f66e3e00640 1 -- 172.21.15.83:0/2862949878 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556836a5a1b0 msgr2=0x556836a622f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:51.504 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.504+0000 7f66e3e00640 1 --2- 172.21.15.83:0/2862949878 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556836a5a1b0 0x556836a622f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:51.504 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.504+0000 7f66e3e00640 1 -- 172.21.15.83:0/2862949878 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556836a65390 msgr2=0x556836a62d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:51.504 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.504+0000 7f66e3e00640 1 --2- 172.21.15.83:0/2862949878 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556836a65390 0x556836a62d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:51.504 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.504+0000 7f66e3e00640 1 -- 172.21.15.83:0/2862949878 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55683698afd0 con 0x556836a5aad0 2024-04-24T09:15:51.504 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.505+0000 7f66e3e00640 1 --2- 172.21.15.83:0/2862949878 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556836a5aad0 0x556836a62830 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f66d0011010 tx=0x7f66d000bfa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=481d2ad3c559ec34 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:51.505 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.505+0000 7f66caa00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:15:51.506 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.505+0000 7f66caa00640 1 -- 172.21.15.83:0/2862949878 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d002a060 con 0x556836a5aad0 2024-04-24T09:15:51.506 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.505+0000 7f66caa00640 1 -- 172.21.15.83:0/2862949878 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(130..130 src has 1..130) ==== 9583+0+0 (secure 0 0 0) 0x7f66d00039b0 con 0x556836a5aad0 2024-04-24T09:15:51.506 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.505+0000 7f66ee7de740 1 -- 172.21.15.83:0/2862949878 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66d00035f0 con 0x556836a5aad0 2024-04-24T09:15:51.506 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.505+0000 7f66e9a00640 1 --2- 172.21.15.83:0/3523970368 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f66cc0a3dd0 0x7f66cc0a6290 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f66dc0319e0 tx=0x7f66dc009650 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:51.506 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.505+0000 7f66caa00640 1 -- 172.21.15.83:0/2862949878 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f66d0004000 con 0x556836a5aad0 2024-04-24T09:15:51.506 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.505+0000 7f66ee7de740 10 client.40856 fetch_fsmap learned FSMap version 69 2024-04-24T09:15:51.506 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.505+0000 7f66ee7de740 1 -- 172.21.15.83:0/2862949878 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x5568366d3210 con 0x556836a5aad0 2024-04-24T09:15:51.506 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.506+0000 7f66caa00640 1 -- 172.21.15.83:0/2862949878 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f66d00035f0 con 0x556836a5aad0 2024-04-24T09:15:51.506 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.506+0000 7f66caa00640 1 -- 172.21.15.83:0/2862949878 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 69) ==== 8370+0+0 (secure 0 0 0) 0x7f66d00046c0 con 0x556836a5aad0 2024-04-24T09:15:51.506 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.506+0000 7f66ee7de740 10 client.40856 fetch_fsmap finished waiting for FSMap version 69 2024-04-24T09:15:51.507 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.506+0000 7f66ee7de740 10 client.40856 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:15:51.507 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.507+0000 7f66ee7de740 20 client.40856 populate_metadata read hostname 'smithi083' 2024-04-24T09:15:51.507 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.507+0000 7f66ee7de740 1 --2- 172.21.15.83:0/2862949878 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556836a7db90 0x556836a7ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:51.507 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.507+0000 7f66ee7de740 4 client.40856 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:15:51.507 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.507+0000 7f66ee7de740 1 -- 172.21.15.83:0/2862949878 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5568369ce320 con 0x556836a7db90 2024-04-24T09:15:51.507 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.507+0000 7f66e9000640 1 --2- 172.21.15.83:0/2862949878 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556836a7db90 0x556836a7ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:51.507 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.508+0000 7f66e9000640 1 --2- 172.21.15.83:0/2862949878 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556836a7db90 0x556836a7ff80 crc :-1 s=READY pgs=595 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6253cab94d203b0 server_cookie=12adc818d3a0f70a in_seq=0 out_seq=0 2024-04-24T09:15:51.508 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.508+0000 7f66caa00640 10 client.40856 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:15:51.509 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.510+0000 7f66caa00640 1 -- 172.21.15.83:0/2862949878 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5568369ce320 con 0x556836a7db90 2024-04-24T09:15:51.509 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.510+0000 7f66caa00640 10 client.40856 handle_command_reply: tid=0 2024-04-24T09:15:51.509 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.510+0000 7f66ee7de740 1 client.40856 shutdown 2024-04-24T09:15:51.510 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.510+0000 7f66ee7de740 1 -- 172.21.15.83:0/2862949878 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556836a5aad0 msgr2=0x556836a62830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:51.510 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.510+0000 7f66ee7de740 1 --2- 172.21.15.83:0/2862949878 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556836a5aad0 0x556836a62830 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f66d0011010 tx=0x7f66d000bfa0 comp rx=0 tx=0).stop 2024-04-24T09:15:51.510 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.510+0000 7f66ee7de740 1 -- 172.21.15.83:0/2862949878 shutdown_connections 2024-04-24T09:15:51.510 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.510+0000 7f66ee7de740 1 --2- 172.21.15.83:0/2862949878 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556836a7db90 0x556836a7ff80 crc :-1 s=READY pgs=595 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:51.510 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.510+0000 7f66ee7de740 1 --2- 172.21.15.83:0/2862949878 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556836a65390 0x556836a62d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:51.510 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.510+0000 7f66ee7de740 1 --2- 172.21.15.83:0/2862949878 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556836a5aad0 0x556836a62830 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:51.510 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.510+0000 7f66caa00640 0 client.40856 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:15:51.510 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.510+0000 7f66ee7de740 1 --2- 172.21.15.83:0/2862949878 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556836a5a1b0 0x556836a622f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:51.511 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.510+0000 7f66ee7de740 1 -- 172.21.15.83:0/2862949878 >> 172.21.15.83:0/2862949878 conn(0x556836a566c0 msgr2=0x556836a5c440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:51.511 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.510+0000 7f66ee7de740 1 -- 172.21.15.83:0/2862949878 shutdown_connections 2024-04-24T09:15:51.511 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.511+0000 7f66ee7de740 1 -- 172.21.15.83:0/2862949878 wait complete. 2024-04-24T09:15:51.511 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.511+0000 7f66ee7de740 20 client.40856 trim_cache size 0 max 16384 2024-04-24T09:15:51.531 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.531+0000 7f66ee7de740 1 Processor -- start 2024-04-24T09:15:51.531 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.531+0000 7f66ee7de740 1 -- start start 2024-04-24T09:15:51.532 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.531+0000 7f66ee7de740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556836a5a1b0 0x556836a70e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:51.532 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.531+0000 7f66ee7de740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556836a5aad0 0x556836a855e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:51.532 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.532+0000 7f66ee7de740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556836a65390 0x556836a85b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:51.532 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.532+0000 7f66ee7de740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f66e41b5c90 con 0x556836a5a1b0 2024-04-24T09:15:51.532 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.532+0000 7f66ee7de740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f66e41b7830 con 0x556836a5aad0 2024-04-24T09:15:51.532 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.532+0000 7f66ee7de740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f66e41b6360 con 0x556836a65390 2024-04-24T09:15:51.532 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.532+0000 7f66e9000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556836a5a1b0 0x556836a70e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:51.532 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.532+0000 7f66e9000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556836a5a1b0 0x556836a70e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57732/0 (socket says 172.21.15.83:57732) 2024-04-24T09:15:51.532 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.532+0000 7f66e9000640 1 -- 172.21.15.83:0/893238445 learned_addr learned my addr 172.21.15.83:0/893238445 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:15:51.532 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.532+0000 7f66e3e00640 1 --2- 172.21.15.83:0/893238445 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556836a5aad0 0x556836a855e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:51.532 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.532+0000 7f66e9000640 1 -- 172.21.15.83:0/893238445 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556836a65390 msgr2=0x556836a85b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:51.532 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.532+0000 7f66e9000640 1 --2- 172.21.15.83:0/893238445 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556836a65390 0x556836a85b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:51.532 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.532+0000 7f66e9000640 1 -- 172.21.15.83:0/893238445 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556836a5aad0 msgr2=0x556836a855e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:51.532 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.532+0000 7f66e9000640 1 --2- 172.21.15.83:0/893238445 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556836a5aad0 0x556836a855e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:51.533 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.532+0000 7f66e9000640 1 -- 172.21.15.83:0/893238445 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55683679f620 con 0x556836a5a1b0 2024-04-24T09:15:51.533 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.532+0000 7f66e9000640 1 --2- 172.21.15.83:0/893238445 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556836a5a1b0 0x556836a70e80 secure :-1 s=READY pgs=2751 cs=0 l=1 rev1=1 crypto rx=0x7f66d8098e90 tx=0x7f66d80d48a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4542e71a419a6928 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:51.540 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.533+0000 7f66d5600640 1 -- 172.21.15.83:0/893238445 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d809c570 con 0x556836a5a1b0 2024-04-24T09:15:51.540 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.533+0000 7f66d5600640 1 -- 172.21.15.83:0/893238445 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f66d8010de0 con 0x556836a5a1b0 2024-04-24T09:15:51.540 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.533+0000 7f66d5600640 1 -- 172.21.15.83:0/893238445 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d809c390 con 0x556836a5a1b0 2024-04-24T09:15:51.540 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.533+0000 7f66ee7de740 1 -- 172.21.15.83:0/893238445 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556836a5a1b0 msgr2=0x556836a70e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:51.540 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.533+0000 7f66ee7de740 1 --2- 172.21.15.83:0/893238445 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556836a5a1b0 0x556836a70e80 secure :-1 s=READY pgs=2751 cs=0 l=1 rev1=1 crypto rx=0x7f66d8098e90 tx=0x7f66d80d48a0 comp rx=0 tx=0).stop 2024-04-24T09:15:51.540 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.533+0000 7f66ee7de740 1 -- 172.21.15.83:0/893238445 shutdown_connections 2024-04-24T09:15:51.540 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.533+0000 7f66ee7de740 1 --2- 172.21.15.83:0/893238445 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556836a65390 0x556836a85b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:51.540 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.533+0000 7f66ee7de740 1 --2- 172.21.15.83:0/893238445 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556836a5aad0 0x556836a855e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:51.540 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.533+0000 7f66ee7de740 1 --2- 172.21.15.83:0/893238445 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556836a5a1b0 0x556836a70e80 unknown :-1 s=CLOSED pgs=2751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:51.540 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.534+0000 7f66ee7de740 1 -- 172.21.15.83:0/893238445 >> 172.21.15.83:0/893238445 conn(0x556836a566c0 msgr2=0x556836a5d670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:51.541 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.534+0000 7f66ee7de740 1 -- 172.21.15.83:0/893238445 shutdown_connections 2024-04-24T09:15:51.541 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.534+0000 7f66ee7de740 1 -- 172.21.15.83:0/893238445 wait complete. 2024-04-24T09:15:51.541 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.534+0000 7f66ee7de740 1 -- start start 2024-04-24T09:15:51.541 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.534+0000 7f66ee7de740 1 Processor -- start 2024-04-24T09:15:51.541 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.534+0000 7f66ee7de740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556836a5a1b0 0x556836a68d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:51.541 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.535+0000 7f66ee7de740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556836a5aad0 0x556836a69240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:51.541 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.535+0000 7f66ee7de740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556836a65390 0x556836a69780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:51.541 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.535+0000 7f66ee7de740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f66e41b7160 con 0x556836a5aad0 2024-04-24T09:15:51.541 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.535+0000 7f66ee7de740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f66e41b6a90 con 0x556836a65390 2024-04-24T09:15:51.541 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.535+0000 7f66ee7de740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f66e41b3cb0 con 0x556836a5a1b0 2024-04-24T09:15:51.541 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.535+0000 7f66ee7de740 10 client.0 _ll_register_callbacks cb 0x7f66dc009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:15:51.541 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.535+0000 7f66e3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556836a5aad0 0x556836a69240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:51.542 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.535+0000 7f66e3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556836a5aad0 0x556836a69240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57740/0 (socket says 172.21.15.83:57740) 2024-04-24T09:15:51.542 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.535+0000 7f66e3e00640 1 -- 172.21.15.83:0/1787892342 learned_addr learned my addr 172.21.15.83:0/1787892342 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:15:51.542 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.535+0000 7f66e9a00640 1 --2- 172.21.15.83:0/1787892342 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556836a65390 0x556836a69780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:51.542 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.535+0000 7f66e3e00640 1 -- 172.21.15.83:0/1787892342 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556836a5a1b0 msgr2=0x556836a68d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:51.542 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.535+0000 7f66e3e00640 1 --2- 172.21.15.83:0/1787892342 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556836a5a1b0 0x556836a68d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:51.542 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.535+0000 7f66e3e00640 1 -- 172.21.15.83:0/1787892342 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556836a65390 msgr2=0x556836a69780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:15:51.542 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.535+0000 7f66e3e00640 1 --2- 172.21.15.83:0/1787892342 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556836a65390 0x556836a69780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:51.542 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.535+0000 7f66e3e00640 1 -- 172.21.15.83:0/1787892342 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556836ae2e50 con 0x556836a5aad0 2024-04-24T09:15:51.542 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.535+0000 7f66e3e00640 1 --2- 172.21.15.83:0/1787892342 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556836a5aad0 0x556836a69240 secure :-1 s=READY pgs=2752 cs=0 l=1 rev1=1 crypto rx=0x7f66d000c010 tx=0x7f66d0005ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d2483257bab6f11 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:15:51.542 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.535+0000 7f66caa00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:15:51.542 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.536+0000 7f66caa00640 1 -- 172.21.15.83:0/1787892342 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d002a040 con 0x556836a5aad0 2024-04-24T09:15:51.542 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.536+0000 7f66caa00640 1 -- 172.21.15.83:0/1787892342 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(130..130 src has 1..130) ==== 9583+0+0 (secure 0 0 0) 0x7f66d00041a0 con 0x556836a5aad0 2024-04-24T09:15:51.542 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.536+0000 7f66ee7de740 1 -- 172.21.15.83:0/1787892342 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5568369c8190 con 0x556836a5aad0 2024-04-24T09:15:51.542 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.536+0000 7f66caa00640 1 -- 172.21.15.83:0/1787892342 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f66d003f050 con 0x556836a5aad0 2024-04-24T09:15:51.543 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.536+0000 7f66ee7de740 10 client.23805 fetch_fsmap learned FSMap version 69 2024-04-24T09:15:51.543 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.536+0000 7f66ee7de740 1 -- 172.21.15.83:0/1787892342 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x556836ae40a0 con 0x556836a5aad0 2024-04-24T09:15:51.543 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.536+0000 7f66caa00640 1 -- 172.21.15.83:0/1787892342 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x5568369c8190 con 0x556836a5aad0 2024-04-24T09:15:51.543 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.536+0000 7f66caa00640 1 -- 172.21.15.83:0/1787892342 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 69) ==== 8370+0+0 (secure 0 0 0) 0x7f66d0004fe0 con 0x556836a5aad0 2024-04-24T09:15:51.543 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.536+0000 7f66ee7de740 10 client.23805 fetch_fsmap finished waiting for FSMap version 69 2024-04-24T09:15:51.543 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.536+0000 7f66ee7de740 10 client.23805 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:15:51.543 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.536+0000 7f66ee7de740 20 client.23805 populate_metadata read hostname 'smithi083' 2024-04-24T09:15:51.543 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.536+0000 7f66ee7de740 1 --2- 172.21.15.83:0/1787892342 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556836ae4730 0x556836ae6b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:15:51.543 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.536+0000 7f66ee7de740 4 client.23805 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T09:15:51.543 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.536+0000 7f66ee7de740 1 -- 172.21.15.83:0/1787892342 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5568369444b0 con 0x556836ae4730 2024-04-24T09:15:51.543 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.537+0000 7f66e9000640 1 --2- 172.21.15.83:0/1787892342 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556836ae4730 0x556836ae6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:15:51.543 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.537+0000 7f66e9000640 1 --2- 172.21.15.83:0/1787892342 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556836ae4730 0x556836ae6b50 crc :-1 s=READY pgs=596 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5256473c7365e80b server_cookie=4017635bfe9bd349 in_seq=0 out_seq=0 2024-04-24T09:15:51.543 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.537+0000 7f66caa00640 10 client.23805 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:15:51.703 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.702+0000 7f66caa00640 1 -- 172.21.15.83:0/1787892342 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5568369444b0 con 0x556836ae4730 2024-04-24T09:15:51.703 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.702+0000 7f66caa00640 10 client.23805 handle_command_reply: tid=0 2024-04-24T09:15:51.703 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.702+0000 7f66ee7de740 1 client.23805 shutdown 2024-04-24T09:15:51.703 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.703+0000 7f66ee7de740 1 -- 172.21.15.83:0/1787892342 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556836a5aad0 msgr2=0x556836a69240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:51.703 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.703+0000 7f66ee7de740 1 --2- 172.21.15.83:0/1787892342 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556836a5aad0 0x556836a69240 secure :-1 s=READY pgs=2752 cs=0 l=1 rev1=1 crypto rx=0x7f66d000c010 tx=0x7f66d0005ae0 comp rx=0 tx=0).stop 2024-04-24T09:15:51.703 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.703+0000 7f66ee7de740 1 -- 172.21.15.83:0/1787892342 shutdown_connections 2024-04-24T09:15:51.703 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.703+0000 7f66ee7de740 1 --2- 172.21.15.83:0/1787892342 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556836ae4730 0x556836ae6b50 crc :-1 s=READY pgs=596 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:51.703 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.703+0000 7f66ee7de740 1 --2- 172.21.15.83:0/1787892342 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556836a65390 0x556836a69780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:51.703 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.703+0000 7f66ee7de740 1 --2- 172.21.15.83:0/1787892342 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556836a5aad0 0x556836a69240 unknown :-1 s=CLOSED pgs=2752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:51.703 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.703+0000 7f66caa00640 0 client.23805 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:15:51.704 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.703+0000 7f66ee7de740 1 --2- 172.21.15.83:0/1787892342 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556836a5a1b0 0x556836a68d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:51.704 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.703+0000 7f66ee7de740 1 -- 172.21.15.83:0/1787892342 >> 172.21.15.83:0/1787892342 conn(0x556836a566c0 msgr2=0x556836a5bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:51.704 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.703+0000 7f66ee7de740 1 -- 172.21.15.83:0/1787892342 shutdown_connections 2024-04-24T09:15:51.704 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.703+0000 7f66ee7de740 1 -- 172.21.15.83:0/1787892342 wait complete. 2024-04-24T09:15:51.704 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.703+0000 7f66ee7de740 20 client.23805 trim_cache size 0 max 16384 2024-04-24T09:15:51.704 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T09:15:51.704 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T09:15:51.704 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T09:15:51.704 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T09:15:51.705 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.704+0000 7f66eba00640 1 -- 172.21.15.83:0/3523970368 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f66cc0a3dd0 msgr2=0x7f66cc0a6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:51.705 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.704+0000 7f66eba00640 1 --2- 172.21.15.83:0/3523970368 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f66cc0a3dd0 0x7f66cc0a6290 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f66dc0319e0 tx=0x7f66dc009650 comp rx=0 tx=0).stop 2024-04-24T09:15:51.705 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.704+0000 7f66eba00640 1 -- 172.21.15.83:0/3523970368 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66e4109540 msgr2=0x7f66e41a3120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:15:51.705 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.704+0000 7f66eba00640 1 --2- 172.21.15.83:0/3523970368 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66e4109540 0x7f66e41a3120 secure :-1 s=READY pgs=2750 cs=0 l=1 rev1=1 crypto rx=0x7f66d800e9f0 tx=0x7f66d800eeb0 comp rx=0 tx=0).stop 2024-04-24T09:15:51.705 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.704+0000 7f66eba00640 1 -- 172.21.15.83:0/3523970368 shutdown_connections 2024-04-24T09:15:51.706 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.704+0000 7f66eba00640 1 --2- 172.21.15.83:0/3523970368 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f66cc0a3dd0 0x7f66cc0a6290 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:51.706 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.704+0000 7f66eba00640 1 --2- 172.21.15.83:0/3523970368 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f66e4117090 0x7f66e41a79e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:51.706 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.704+0000 7f66eba00640 1 --2- 172.21.15.83:0/3523970368 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f66e410c700 0x7f66e41a3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:51.706 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.704+0000 7f66eba00640 1 --2- 172.21.15.83:0/3523970368 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66e4109540 0x7f66e41a3120 unknown :-1 s=CLOSED pgs=2750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:15:51.706 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.704+0000 7f66eba00640 1 -- 172.21.15.83:0/3523970368 >> 172.21.15.83:0/3523970368 conn(0x7f66e406d0d0 msgr2=0x7f66e4117470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:15:51.706 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.705+0000 7f66eba00640 1 -- 172.21.15.83:0/3523970368 shutdown_connections 2024-04-24T09:15:51.706 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:15:51.705+0000 7f66eba00640 1 -- 172.21.15.83:0/3523970368 wait complete. 2024-04-24T09:15:52.215 INFO:tasks.workunit.client.0.smithi083.stdout: 10 76401 6.52 MB/sec execute 503 sec latency 125.652 ms 2024-04-24T09:15:52.552 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:15:52 smithi083 ceph-mon[27907]: pgmap v766: 129 pgs: 129 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.7 MiB/s wr, 278 op/s 2024-04-24T09:15:52.704 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:15:52 smithi138 ceph-mon[31441]: pgmap v766: 129 pgs: 129 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.7 MiB/s wr, 278 op/s 2024-04-24T09:15:52.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:15:52 smithi189 ceph-mon[31791]: pgmap v766: 129 pgs: 129 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.7 MiB/s wr, 278 op/s 2024-04-24T09:15:53.215 INFO:tasks.workunit.client.0.smithi083.stdout: 10 76533 6.52 MB/sec execute 504 sec latency 108.821 ms 2024-04-24T09:15:53.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:15:53 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:15:53.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:15:53 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:15:53.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:15:53 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:15:53.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:15:53 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:15:53.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:15:53 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:15:53.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:15:53 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:15:53.706 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:15:53 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:15:53.706 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:15:53 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:15:53.706 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:15:53 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:15:53.706 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:15:53 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:15:53.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:15:53 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:15:53.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:15:53 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:15:53.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:15:53 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:15:53.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:15:53 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:15:53.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:15:53 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:15:54.215 INFO:tasks.workunit.client.0.smithi083.stdout: 10 76679 6.52 MB/sec execute 505 sec latency 109.203 ms 2024-04-24T09:15:54.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:15:54 smithi083 ceph-mon[27907]: pgmap v767: 129 pgs: 129 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.1 MiB/s wr, 401 op/s 2024-04-24T09:15:54.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:15:54 smithi138 ceph-mon[31441]: pgmap v767: 129 pgs: 129 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.1 MiB/s wr, 401 op/s 2024-04-24T09:15:54.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:15:54 smithi189 ceph-mon[31791]: pgmap v767: 129 pgs: 129 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.1 MiB/s wr, 401 op/s 2024-04-24T09:15:55.215 INFO:tasks.workunit.client.0.smithi083.stdout: 10 76797 6.51 MB/sec execute 506 sec latency 87.664 ms 2024-04-24T09:15:56.215 INFO:tasks.workunit.client.0.smithi083.stdout: 10 76937 6.53 MB/sec execute 507 sec latency 103.993 ms 2024-04-24T09:15:56.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:15:56 smithi083 ceph-mon[27907]: pgmap v768: 129 pgs: 129 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.2 MiB/s wr, 214 op/s 2024-04-24T09:15:56.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:15:56 smithi138 ceph-mon[31441]: pgmap v768: 129 pgs: 129 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.2 MiB/s wr, 214 op/s 2024-04-24T09:15:56.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:15:56 smithi189 ceph-mon[31791]: pgmap v768: 129 pgs: 129 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.2 MiB/s wr, 214 op/s 2024-04-24T09:15:57.215 INFO:tasks.workunit.client.0.smithi083.stdout: 10 77069 6.53 MB/sec execute 508 sec latency 114.787 ms 2024-04-24T09:15:58.215 INFO:tasks.workunit.client.0.smithi083.stdout: 10 77237 6.54 MB/sec execute 509 sec latency 120.051 ms 2024-04-24T09:15:58.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:15:58 smithi083 ceph-mon[27907]: pgmap v769: 129 pgs: 129 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.2 MiB/s wr, 214 op/s 2024-04-24T09:15:58.704 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:15:58 smithi138 ceph-mon[31441]: pgmap v769: 129 pgs: 129 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.2 MiB/s wr, 214 op/s 2024-04-24T09:15:58.706 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:15:58 smithi189 ceph-mon[31791]: pgmap v769: 129 pgs: 129 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.2 MiB/s wr, 214 op/s 2024-04-24T09:15:59.215 INFO:tasks.workunit.client.0.smithi083.stdout: 10 77371 6.54 MB/sec execute 510 sec latency 121.044 ms 2024-04-24T09:16:00.215 INFO:tasks.workunit.client.0.smithi083.stdout: 10 77503 6.54 MB/sec execute 511 sec latency 93.039 ms 2024-04-24T09:16:00.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:16:00 smithi083 ceph-mon[27907]: pgmap v770: 129 pgs: 129 active+clean; 1.2 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.4 MiB/s wr, 304 op/s 2024-04-24T09:16:00.704 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:16:00 smithi138 ceph-mon[31441]: pgmap v770: 129 pgs: 129 active+clean; 1.2 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.4 MiB/s wr, 304 op/s 2024-04-24T09:16:00.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:16:00 smithi189 ceph-mon[31791]: pgmap v770: 129 pgs: 129 active+clean; 1.2 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.4 MiB/s wr, 304 op/s 2024-04-24T09:16:01.216 INFO:tasks.workunit.client.0.smithi083.stdout: 10 77577 6.55 MB/sec execute 512 sec latency 517.813 ms 2024-04-24T09:16:02.216 INFO:tasks.workunit.client.0.smithi083.stdout: 10 77604 6.54 MB/sec execute 513 sec latency 730.282 ms 2024-04-24T09:16:02.510 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:16:02 smithi189 ceph-mon[31791]: pgmap v771: 129 pgs: 129 active+clean; 1.2 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.6 MiB/s wr, 212 op/s 2024-04-24T09:16:02.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:16:02 smithi083 ceph-mon[27907]: pgmap v771: 129 pgs: 129 active+clean; 1.2 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.6 MiB/s wr, 212 op/s 2024-04-24T09:16:02.704 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:16:02 smithi138 ceph-mon[31441]: pgmap v771: 129 pgs: 129 active+clean; 1.2 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.6 MiB/s wr, 212 op/s 2024-04-24T09:16:03.215 INFO:tasks.workunit.client.0.smithi083.stdout: 10 77721 6.53 MB/sec execute 514 sec latency 693.093 ms 2024-04-24T09:16:04.216 INFO:tasks.workunit.client.0.smithi083.stdout: 10 77828 6.54 MB/sec execute 515 sec latency 61.298 ms 2024-04-24T09:16:04.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:16:04 smithi083 ceph-mon[27907]: pgmap v772: 129 pgs: 129 active+clean; 1.2 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.3 MiB/s wr, 306 op/s 2024-04-24T09:16:04.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:16:04 smithi138 ceph-mon[31441]: pgmap v772: 129 pgs: 129 active+clean; 1.2 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.3 MiB/s wr, 306 op/s 2024-04-24T09:16:04.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:16:04 smithi189 ceph-mon[31791]: pgmap v772: 129 pgs: 129 active+clean; 1.2 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.3 MiB/s wr, 306 op/s 2024-04-24T09:16:04.887 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.885+0000 7f17d5600640 1 -- 172.21.15.83:0/1421139398 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f17d0076530 msgr2=0x7f17d0076910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:04.887 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.885+0000 7f17d5600640 1 --2- 172.21.15.83:0/1421139398 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f17d0076530 0x7f17d0076910 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f17c0009930 tx=0x7f17c002a420 comp rx=0 tx=0).stop 2024-04-24T09:16:04.887 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.886+0000 7f17d5600640 1 -- 172.21.15.83:0/1421139398 shutdown_connections 2024-04-24T09:16:04.888 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.886+0000 7f17d5600640 1 --2- 172.21.15.83:0/1421139398 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17d0112ea0 0x7f17d0115290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:04.888 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.886+0000 7f17d5600640 1 --2- 172.21.15.83:0/1421139398 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f17d0076e50 0x7f17d0112960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:04.888 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.886+0000 7f17d5600640 1 --2- 172.21.15.83:0/1421139398 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f17d0076530 0x7f17d0076910 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:04.888 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.886+0000 7f17d5600640 1 -- 172.21.15.83:0/1421139398 >> 172.21.15.83:0/1421139398 conn(0x7f17d00ffd30 msgr2=0x7f17d0102150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:04.888 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.886+0000 7f17d5600640 1 -- 172.21.15.83:0/1421139398 shutdown_connections 2024-04-24T09:16:04.888 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.886+0000 7f17d5600640 1 -- 172.21.15.83:0/1421139398 wait complete. 2024-04-24T09:16:04.891 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.887+0000 7f17d5600640 1 Processor -- start 2024-04-24T09:16:04.891 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.888+0000 7f17d5600640 1 -- start start 2024-04-24T09:16:04.891 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.888+0000 7f17d5600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f17d0076530 0x7f17d019ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:04.891 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.888+0000 7f17d5600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f17d0076e50 0x7f17d019f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:04.891 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.888+0000 7f17d5600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17d0112ea0 0x7f17d01a3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:04.892 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.888+0000 7f17d5600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f17d0118900 con 0x7f17d0112ea0 2024-04-24T09:16:04.892 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.888+0000 7f17d5600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f17d0118780 con 0x7f17d0076e50 2024-04-24T09:16:04.892 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.888+0000 7f17d5600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f17d0118a80 con 0x7f17d0076530 2024-04-24T09:16:04.892 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.889+0000 7f17ce000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f17d0076e50 0x7f17d019f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:04.892 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.889+0000 7f17ce000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f17d0076e50 0x7f17d019f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:49490/0 (socket says 172.21.15.83:49490) 2024-04-24T09:16:04.892 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.889+0000 7f17ce000640 1 -- 172.21.15.83:0/3098882458 learned_addr learned my addr 172.21.15.83:0/3098882458 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:16:04.892 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.889+0000 7f17cea00640 1 --2- 172.21.15.83:0/3098882458 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f17d0076530 0x7f17d019ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:04.893 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.889+0000 7f17cf400640 1 --2- 172.21.15.83:0/3098882458 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17d0112ea0 0x7f17d01a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:04.893 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.890+0000 7f17cea00640 1 -- 172.21.15.83:0/3098882458 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f17d0076e50 msgr2=0x7f17d019f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:04.893 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.890+0000 7f17cea00640 1 --2- 172.21.15.83:0/3098882458 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f17d0076e50 0x7f17d019f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:04.893 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.890+0000 7f17cea00640 1 -- 172.21.15.83:0/3098882458 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17d0112ea0 msgr2=0x7f17d01a3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:04.893 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.890+0000 7f17cea00640 1 --2- 172.21.15.83:0/3098882458 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17d0112ea0 0x7f17d01a3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:04.893 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.890+0000 7f17cea00640 1 -- 172.21.15.83:0/3098882458 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17d01a3e30 con 0x7f17d0076530 2024-04-24T09:16:04.893 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.891+0000 7f17cea00640 1 --2- 172.21.15.83:0/3098882458 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f17d0076530 0x7f17d019ee40 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f17c002a940 tx=0x7f17c0003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:04.895 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.891+0000 7f17c7400640 1 -- 172.21.15.83:0/3098882458 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c003f070 con 0x7f17d0076530 2024-04-24T09:16:04.895 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.891+0000 7f17c7400640 1 -- 172.21.15.83:0/3098882458 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f17c0031b60 con 0x7f17d0076530 2024-04-24T09:16:04.895 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.891+0000 7f17c7400640 1 -- 172.21.15.83:0/3098882458 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c0035030 con 0x7f17d0076530 2024-04-24T09:16:04.895 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.892+0000 7f17d5600640 1 -- 172.21.15.83:0/3098882458 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17d01a40c0 con 0x7f17d0076530 2024-04-24T09:16:04.895 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.893+0000 7f17d5600640 1 -- 172.21.15.83:0/3098882458 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17d01a4530 con 0x7f17d0076530 2024-04-24T09:16:04.898 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.897+0000 7f17d83f3740 1 Processor -- start 2024-04-24T09:16:04.899 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.898+0000 7f17d83f3740 1 -- start start 2024-04-24T09:16:04.899 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.898+0000 7f17d83f3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5633e1bd11b0 0x5633e1bd1590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:04.899 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.898+0000 7f17d83f3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5633e1bd1ad0 0x5633e1bdbe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:04.899 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.898+0000 7f17d83f3740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5633e1bdc390 0x5633e1bde780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:04.900 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.898+0000 7f17d83f3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5633e1b37900 con 0x5633e1bd11b0 2024-04-24T09:16:04.900 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.898+0000 7f17d83f3740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5633e1addf20 con 0x5633e1bdc390 2024-04-24T09:16:04.900 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.898+0000 7f17d83f3740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5633e1b338c0 con 0x5633e1bd1ad0 2024-04-24T09:16:04.901 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.898+0000 7f17cf400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5633e1bd11b0 0x5633e1bd1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:04.901 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.899+0000 7f17cf400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5633e1bd11b0 0x5633e1bd1590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37966/0 (socket says 172.21.15.83:37966) 2024-04-24T09:16:04.901 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.899+0000 7f17cf400640 1 -- 172.21.15.83:0/186969408 learned_addr learned my addr 172.21.15.83:0/186969408 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:16:04.901 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.899+0000 7f17c7400640 1 -- 172.21.15.83:0/3098882458 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f17c0042070 con 0x7f17d0076530 2024-04-24T09:16:04.902 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.899+0000 7f17ce000640 1 --2- 172.21.15.83:0/186969408 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5633e1bdc390 0x5633e1bde780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:04.902 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.899+0000 7f17cf400640 1 -- 172.21.15.83:0/186969408 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5633e1bd1ad0 msgr2=0x5633e1bdbe50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T09:16:04.902 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.899+0000 7f17cf400640 1 --2- 172.21.15.83:0/186969408 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5633e1bd1ad0 0x5633e1bdbe50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:04.902 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.899+0000 7f17cf400640 1 -- 172.21.15.83:0/186969408 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5633e1bdc390 msgr2=0x5633e1bde780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:04.902 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.899+0000 7f17cf400640 1 --2- 172.21.15.83:0/186969408 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5633e1bdc390 0x5633e1bde780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:04.902 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.899+0000 7f17cf400640 1 -- 172.21.15.83:0/186969408 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633e1b02e10 con 0x5633e1bd11b0 2024-04-24T09:16:04.902 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.899+0000 7f17ce000640 1 --2- 172.21.15.83:0/186969408 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5633e1bdc390 0x5633e1bde780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:16:04.903 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.899+0000 7f17cf400640 1 --2- 172.21.15.83:0/186969408 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5633e1bd11b0 0x5633e1bd1590 secure :-1 s=READY pgs=2753 cs=0 l=1 rev1=1 crypto rx=0x7f17c8009f30 tx=0x7f17c80118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af6ddb96ed227fc9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:04.903 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.900+0000 7f17bb400640 1 -- 172.21.15.83:0/186969408 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c8004d20 con 0x5633e1bd11b0 2024-04-24T09:16:04.903 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.900+0000 7f17bb400640 1 -- 172.21.15.83:0/186969408 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f17c8004050 con 0x5633e1bd11b0 2024-04-24T09:16:04.903 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.900+0000 7f17bb400640 1 -- 172.21.15.83:0/186969408 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c8005260 con 0x5633e1bd11b0 2024-04-24T09:16:04.909 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.901+0000 7f17c7400640 1 --2- 172.21.15.83:0/3098882458 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f17b40a3e40 0x7f17b40a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:04.909 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.901+0000 7f17c7400640 1 -- 172.21.15.83:0/3098882458 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 9583+0+0 (secure 0 0 0) 0x7f17c00f1200 con 0x7f17d0076530 2024-04-24T09:16:04.909 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.901+0000 7f17d83f3740 1 -- 172.21.15.83:0/186969408 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5633e1bd11b0 msgr2=0x5633e1bd1590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:04.910 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.901+0000 7f17d83f3740 1 --2- 172.21.15.83:0/186969408 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5633e1bd11b0 0x5633e1bd1590 secure :-1 s=READY pgs=2753 cs=0 l=1 rev1=1 crypto rx=0x7f17c8009f30 tx=0x7f17c80118d0 comp rx=0 tx=0).stop 2024-04-24T09:16:04.910 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.901+0000 7f17cea00640 1 --2- 172.21.15.83:0/3098882458 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f17b40a3e40 0x7f17b40a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:04.910 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.902+0000 7f17d83f3740 1 -- 172.21.15.83:0/186969408 shutdown_connections 2024-04-24T09:16:04.910 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.902+0000 7f17d83f3740 1 --2- 172.21.15.83:0/186969408 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5633e1bdc390 0x5633e1bde780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:04.910 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.902+0000 7f17d83f3740 1 --2- 172.21.15.83:0/186969408 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5633e1bd1ad0 0x5633e1bdbe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:04.911 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.902+0000 7f17d83f3740 1 --2- 172.21.15.83:0/186969408 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5633e1bd11b0 0x5633e1bd1590 unknown :-1 s=CLOSED pgs=2753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:04.911 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.902+0000 7f17d83f3740 1 -- 172.21.15.83:0/186969408 >> 172.21.15.83:0/186969408 conn(0x5633e1bcd6c0 msgr2=0x5633e1bcdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:04.911 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.902+0000 7f17cea00640 1 --2- 172.21.15.83:0/3098882458 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f17b40a3e40 0x7f17b40a6300 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x5633e1bd2750 tx=0x7f17c00095f0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:04.911 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.902+0000 7f17d83f3740 1 -- 172.21.15.83:0/186969408 shutdown_connections 2024-04-24T09:16:04.911 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.902+0000 7f17d83f3740 1 -- 172.21.15.83:0/186969408 wait complete. 2024-04-24T09:16:04.911 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.903+0000 7f17d83f3740 1 -- start start 2024-04-24T09:16:04.911 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.903+0000 7f17d83f3740 1 Processor -- start 2024-04-24T09:16:04.911 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.904+0000 7f17d83f3740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5633e1bd11b0 0x5633e1bd92d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:04.912 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.904+0000 7f17d83f3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5633e1bd1ad0 0x5633e1bd9810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:04.912 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.904+0000 7f17d83f3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5633e1bdc390 0x5633e1bd9d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:04.912 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.904+0000 7f17d83f3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5633e1bdefc0 con 0x5633e1bdc390 2024-04-24T09:16:04.912 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.904+0000 7f17d83f3740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5633e1af12c0 con 0x5633e1bd11b0 2024-04-24T09:16:04.912 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.904+0000 7f17d83f3740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5633e1bdf140 con 0x5633e1bd1ad0 2024-04-24T09:16:04.912 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.904+0000 7f17d83f3740 10 client.0 _ll_register_callbacks cb 0x5633e1849a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:16:04.912 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.905+0000 7f17cf400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5633e1bdc390 0x5633e1bd9d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:04.912 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.905+0000 7f17cf400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5633e1bd11b0 0x5633e1bd92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:04.912 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.905+0000 7f17cf400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5633e1bdc390 0x5633e1bd9d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37978/0 (socket says 172.21.15.83:37978) 2024-04-24T09:16:04.913 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.905+0000 7f17cf400640 1 -- 172.21.15.83:0/416569121 learned_addr learned my addr 172.21.15.83:0/416569121 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:16:04.913 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.905+0000 7f17ce000640 1 --2- 172.21.15.83:0/416569121 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5633e1bd1ad0 0x5633e1bd9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:04.913 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.905+0000 7f17cf400640 1 -- 172.21.15.83:0/416569121 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5633e1bd1ad0 msgr2=0x5633e1bd9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:04.913 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.905+0000 7f17cf400640 1 --2- 172.21.15.83:0/416569121 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5633e1bd1ad0 0x5633e1bd9810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:04.913 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.905+0000 7f17cf400640 1 -- 172.21.15.83:0/416569121 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5633e1bd11b0 msgr2=0x5633e1bd92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:04.913 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.905+0000 7f17cf400640 1 --2- 172.21.15.83:0/416569121 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5633e1bd11b0 0x5633e1bd92d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:04.913 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.905+0000 7f17cf400640 1 -- 172.21.15.83:0/416569121 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633e1b01fd0 con 0x5633e1bdc390 2024-04-24T09:16:04.913 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.905+0000 7f17ce000640 1 --2- 172.21.15.83:0/416569121 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5633e1bd1ad0 0x5633e1bd9810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:16:04.913 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.906+0000 7f17cf400640 1 --2- 172.21.15.83:0/416569121 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5633e1bdc390 0x5633e1bd9d50 secure :-1 s=READY pgs=2754 cs=0 l=1 rev1=1 crypto rx=0x7f17c8004650 tx=0x7f17c8004910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d25e096c6e04a723 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:04.914 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.906+0000 7f17afe00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:16:04.914 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.906+0000 7f17afe00640 1 -- 172.21.15.83:0/416569121 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c802d060 con 0x5633e1bdc390 2024-04-24T09:16:04.914 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.906+0000 7f17afe00640 1 -- 172.21.15.83:0/416569121 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(130..130 src has 1..130) ==== 9583+0+0 (secure 0 0 0) 0x7f17c800f690 con 0x5633e1bdc390 2024-04-24T09:16:04.914 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.906+0000 7f17d83f3740 1 -- 172.21.15.83:0/416569121 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17c8004050 con 0x5633e1bdc390 2024-04-24T09:16:04.914 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.906+0000 7f17afe00640 1 -- 172.21.15.83:0/416569121 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f17c800f890 con 0x5633e1bdc390 2024-04-24T09:16:04.914 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.906+0000 7f17d83f3740 10 client.23817 fetch_fsmap learned FSMap version 69 2024-04-24T09:16:04.914 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.907+0000 7f17d83f3740 1 -- 172.21.15.83:0/416569121 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x5633e184a210 con 0x5633e1bdc390 2024-04-24T09:16:04.914 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.907+0000 7f17afe00640 1 -- 172.21.15.83:0/416569121 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f17c8004050 con 0x5633e1bdc390 2024-04-24T09:16:04.915 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.907+0000 7f17afe00640 1 -- 172.21.15.83:0/416569121 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 69) ==== 8370+0+0 (secure 0 0 0) 0x7f17c8025040 con 0x5633e1bdc390 2024-04-24T09:16:04.915 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.907+0000 7f17d83f3740 10 client.23817 fetch_fsmap finished waiting for FSMap version 69 2024-04-24T09:16:04.915 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.907+0000 7f17d83f3740 10 client.23817 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:16:04.915 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.907+0000 7f17d83f3740 20 client.23817 populate_metadata read hostname 'smithi083' 2024-04-24T09:16:04.915 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.907+0000 7f17d83f3740 1 --2- 172.21.15.83:0/416569121 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5633e1bec420 0x5633e1bee810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:04.915 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.907+0000 7f17d83f3740 4 client.23817 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:16:04.916 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.907+0000 7f17d83f3740 1 -- 172.21.15.83:0/416569121 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5633e1b45320 con 0x5633e1bec420 2024-04-24T09:16:04.916 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.908+0000 7f17ce000640 1 --2- 172.21.15.83:0/416569121 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5633e1bec420 0x5633e1bee810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:04.916 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.909+0000 7f17ce000640 1 --2- 172.21.15.83:0/416569121 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5633e1bec420 0x5633e1bee810 crc :-1 s=READY pgs=597 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7b4f399b2fa08bd server_cookie=52da0ec18896a404 in_seq=0 out_seq=0 2024-04-24T09:16:04.916 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.909+0000 7f17afe00640 10 client.23817 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:16:04.919 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.909+0000 7f17afe00640 1 -- 172.21.15.83:0/416569121 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5633e1b45320 con 0x5633e1bec420 2024-04-24T09:16:04.919 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.909+0000 7f17afe00640 10 client.23817 handle_command_reply: tid=0 2024-04-24T09:16:04.919 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.910+0000 7f17d83f3740 1 client.23817 shutdown 2024-04-24T09:16:04.919 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.910+0000 7f17d83f3740 1 -- 172.21.15.83:0/416569121 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5633e1bdc390 msgr2=0x5633e1bd9d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:04.920 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.910+0000 7f17d83f3740 1 --2- 172.21.15.83:0/416569121 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5633e1bdc390 0x5633e1bd9d50 secure :-1 s=READY pgs=2754 cs=0 l=1 rev1=1 crypto rx=0x7f17c8004650 tx=0x7f17c8004910 comp rx=0 tx=0).stop 2024-04-24T09:16:04.920 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.910+0000 7f17d83f3740 1 -- 172.21.15.83:0/416569121 shutdown_connections 2024-04-24T09:16:04.920 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.910+0000 7f17d83f3740 1 --2- 172.21.15.83:0/416569121 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5633e1bec420 0x5633e1bee810 crc :-1 s=READY pgs=597 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:04.920 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.910+0000 7f17d83f3740 1 --2- 172.21.15.83:0/416569121 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5633e1bdc390 0x5633e1bd9d50 unknown :-1 s=CLOSED pgs=2754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:04.920 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.910+0000 7f17d83f3740 1 --2- 172.21.15.83:0/416569121 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5633e1bd1ad0 0x5633e1bd9810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:04.920 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.910+0000 7f17d83f3740 1 --2- 172.21.15.83:0/416569121 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5633e1bd11b0 0x5633e1bd92d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:04.920 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.910+0000 7f17afe00640 0 client.23817 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:16:04.920 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.910+0000 7f17d83f3740 1 -- 172.21.15.83:0/416569121 >> 172.21.15.83:0/416569121 conn(0x5633e1bcd6c0 msgr2=0x5633e1bd33f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:04.921 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.911+0000 7f17d83f3740 1 -- 172.21.15.83:0/416569121 shutdown_connections 2024-04-24T09:16:04.921 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.911+0000 7f17d83f3740 1 -- 172.21.15.83:0/416569121 wait complete. 2024-04-24T09:16:04.921 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.911+0000 7f17d83f3740 20 client.23817 trim_cache size 0 max 16384 2024-04-24T09:16:04.940 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.938+0000 7f17d83f3740 1 Processor -- start 2024-04-24T09:16:04.940 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.938+0000 7f17d83f3740 1 -- start start 2024-04-24T09:16:04.941 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.938+0000 7f17d83f3740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5633e1bd11b0 0x5633e1bd3b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:04.941 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.938+0000 7f17d83f3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5633e1bd1ad0 0x5633e1bd9b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:04.941 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.938+0000 7f17d83f3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5633e1bdc390 0x5633e1bf3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:04.941 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.938+0000 7f17d83f3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f17d01be450 con 0x5633e1bdc390 2024-04-24T09:16:04.943 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.938+0000 7f17d83f3740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f17d01bfff0 con 0x5633e1bd11b0 2024-04-24T09:16:04.943 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.938+0000 7f17d83f3740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f17d01beb20 con 0x5633e1bd1ad0 2024-04-24T09:16:04.943 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.938+0000 7f17cf400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5633e1bd11b0 0x5633e1bd3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:04.943 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.939+0000 7f17cf400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5633e1bd11b0 0x5633e1bd3b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:49514/0 (socket says 172.21.15.83:49514) 2024-04-24T09:16:04.943 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.939+0000 7f17cf400640 1 -- 172.21.15.83:0/2960297929 learned_addr learned my addr 172.21.15.83:0/2960297929 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:16:04.943 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.939+0000 7f17cf400640 1 --2- 172.21.15.83:0/2960297929 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5633e1bdc390 0x5633e1bf3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:04.943 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.939+0000 7f17ce000640 1 --2- 172.21.15.83:0/2960297929 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5633e1bd1ad0 0x5633e1bd9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:04.943 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.939+0000 7f17cf400640 1 -- 172.21.15.83:0/2960297929 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5633e1bd1ad0 msgr2=0x5633e1bd9b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:04.943 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.939+0000 7f17cf400640 1 --2- 172.21.15.83:0/2960297929 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5633e1bd1ad0 0x5633e1bd9b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:04.944 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.939+0000 7f17cf400640 1 -- 172.21.15.83:0/2960297929 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5633e1bdc390 msgr2=0x5633e1bf3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:04.944 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.939+0000 7f17cf400640 1 --2- 172.21.15.83:0/2960297929 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5633e1bdc390 0x5633e1bf3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:04.944 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.939+0000 7f17cf400640 1 -- 172.21.15.83:0/2960297929 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633e1916620 con 0x5633e1bd11b0 2024-04-24T09:16:04.944 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.940+0000 7f17cf400640 1 --2- 172.21.15.83:0/2960297929 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5633e1bd11b0 0x5633e1bd3b70 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f17c8004590 tx=0x7f17c8009230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8fba9b8c09c8c02 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:04.948 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.940+0000 7f17baa00640 1 -- 172.21.15.83:0/2960297929 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c802d070 con 0x5633e1bd11b0 2024-04-24T09:16:04.948 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.941+0000 7f17baa00640 1 -- 172.21.15.83:0/2960297929 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f17c8011dd0 con 0x5633e1bd11b0 2024-04-24T09:16:04.948 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.941+0000 7f17baa00640 1 -- 172.21.15.83:0/2960297929 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c800fa30 con 0x5633e1bd11b0 2024-04-24T09:16:04.948 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.941+0000 7f17d83f3740 1 -- 172.21.15.83:0/2960297929 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5633e1bd11b0 msgr2=0x5633e1bd3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:04.948 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.941+0000 7f17d83f3740 1 --2- 172.21.15.83:0/2960297929 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5633e1bd11b0 0x5633e1bd3b70 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f17c8004590 tx=0x7f17c8009230 comp rx=0 tx=0).stop 2024-04-24T09:16:04.948 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.942+0000 7f17d83f3740 1 -- 172.21.15.83:0/2960297929 shutdown_connections 2024-04-24T09:16:04.948 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.942+0000 7f17d83f3740 1 --2- 172.21.15.83:0/2960297929 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5633e1bdc390 0x5633e1bf3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:04.948 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.942+0000 7f17d83f3740 1 --2- 172.21.15.83:0/2960297929 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5633e1bd1ad0 0x5633e1bd9b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:04.948 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.942+0000 7f17d83f3740 1 --2- 172.21.15.83:0/2960297929 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5633e1bd11b0 0x5633e1bd3b70 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:04.948 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.942+0000 7f17d83f3740 1 -- 172.21.15.83:0/2960297929 >> 172.21.15.83:0/2960297929 conn(0x5633e1bcd6c0 msgr2=0x5633e1bcc840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:04.949 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.942+0000 7f17d83f3740 1 -- 172.21.15.83:0/2960297929 shutdown_connections 2024-04-24T09:16:04.949 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.942+0000 7f17d83f3740 1 -- 172.21.15.83:0/2960297929 wait complete. 2024-04-24T09:16:04.949 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.942+0000 7f17d83f3740 1 -- start start 2024-04-24T09:16:04.949 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.943+0000 7f17d83f3740 1 Processor -- start 2024-04-24T09:16:04.949 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.943+0000 7f17d83f3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5633e1bd11b0 0x5633e1be4390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:04.949 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.943+0000 7f17d83f3740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5633e1bd1ad0 0x5633e1be48d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:04.949 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.943+0000 7f17d83f3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5633e1bdc390 0x5633e1bd5540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:04.949 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.943+0000 7f17d83f3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f17d01bf920 con 0x5633e1bdc390 2024-04-24T09:16:04.949 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.943+0000 7f17d83f3740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f17d01bf250 con 0x5633e1bd1ad0 2024-04-24T09:16:04.949 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.943+0000 7f17d83f3740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f17d01bc4d0 con 0x5633e1bd11b0 2024-04-24T09:16:04.949 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.943+0000 7f17d83f3740 10 client.0 _ll_register_callbacks cb 0x7f17c8009e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:16:04.949 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.944+0000 7f17ce000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5633e1bd1ad0 0x5633e1be48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:04.949 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.944+0000 7f17ce000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5633e1bd1ad0 0x5633e1be48d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:49530/0 (socket says 172.21.15.83:49530) 2024-04-24T09:16:04.949 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.944+0000 7f17ce000640 1 -- 172.21.15.83:0/55877672 learned_addr learned my addr 172.21.15.83:0/55877672 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:16:04.950 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.944+0000 7f17ce000640 1 -- 172.21.15.83:0/55877672 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5633e1bd11b0 msgr2=0x5633e1be4390 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T09:16:04.950 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.944+0000 7f17ce000640 1 --2- 172.21.15.83:0/55877672 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5633e1bd11b0 0x5633e1be4390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:04.950 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.944+0000 7f17ce000640 1 -- 172.21.15.83:0/55877672 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5633e1bdc390 msgr2=0x5633e1bd5540 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T09:16:04.950 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.944+0000 7f17ce000640 1 --2- 172.21.15.83:0/55877672 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5633e1bdc390 0x5633e1bd5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:04.950 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.944+0000 7f17ce000640 1 -- 172.21.15.83:0/55877672 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633e1bd5a80 con 0x5633e1bd1ad0 2024-04-24T09:16:04.950 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.945+0000 7f17ce000640 1 --2- 172.21.15.83:0/55877672 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5633e1bd1ad0 0x5633e1be48d0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f17bc000c00 tx=0x7f17bc0119b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=face00be69f185f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:04.950 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.945+0000 7f17afe00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:16:04.950 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.945+0000 7f17afe00640 1 -- 172.21.15.83:0/55877672 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17bc01d940 con 0x5633e1bd1ad0 2024-04-24T09:16:04.950 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.945+0000 7f17d83f3740 1 -- 172.21.15.83:0/55877672 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17c8011dd0 con 0x5633e1bd1ad0 2024-04-24T09:16:04.950 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.945+0000 7f17afe00640 1 -- 172.21.15.83:0/55877672 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(130..130 src has 1..130) ==== 9583+0+0 (secure 0 0 0) 0x7f17bc005020 con 0x5633e1bd1ad0 2024-04-24T09:16:04.950 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.946+0000 7f17afe00640 1 -- 172.21.15.83:0/55877672 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f17bc02c050 con 0x5633e1bd1ad0 2024-04-24T09:16:04.951 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.946+0000 7f17d83f3740 10 client.40880 fetch_fsmap learned FSMap version 69 2024-04-24T09:16:04.951 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.946+0000 7f17d83f3740 1 -- 172.21.15.83:0/55877672 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x5633e1c51ef0 con 0x5633e1bd1ad0 2024-04-24T09:16:04.951 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.946+0000 7f17afe00640 1 -- 172.21.15.83:0/55877672 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f17c8011dd0 con 0x5633e1bd1ad0 2024-04-24T09:16:04.951 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.946+0000 7f17afe00640 1 -- 172.21.15.83:0/55877672 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 69) ==== 8370+0+0 (secure 0 0 0) 0x7f17bc010490 con 0x5633e1bd1ad0 2024-04-24T09:16:04.951 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.946+0000 7f17d83f3740 10 client.40880 fetch_fsmap finished waiting for FSMap version 69 2024-04-24T09:16:04.951 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.946+0000 7f17d83f3740 10 client.40880 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:16:04.951 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.946+0000 7f17d83f3740 20 client.40880 populate_metadata read hostname 'smithi083' 2024-04-24T09:16:04.951 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.946+0000 7f17d83f3740 1 --2- 172.21.15.83:0/55877672 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5633e1bec420 0x5633e1c54690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:04.951 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.946+0000 7f17d83f3740 4 client.40880 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T09:16:04.951 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.946+0000 7f17d83f3740 1 -- 172.21.15.83:0/55877672 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5633e1b3f190 con 0x5633e1bec420 2024-04-24T09:16:04.951 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.947+0000 7f17cf400640 1 --2- 172.21.15.83:0/55877672 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5633e1bec420 0x5633e1c54690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:04.951 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.947+0000 7f17cf400640 1 --2- 172.21.15.83:0/55877672 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5633e1bec420 0x5633e1c54690 crc :-1 s=READY pgs=598 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66ae67cc6d89582f server_cookie=191ccc6c7eeef846 in_seq=0 out_seq=0 2024-04-24T09:16:04.951 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:04.947+0000 7f17afe00640 10 client.40880 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:16:05.145 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:05.144+0000 7f17afe00640 1 -- 172.21.15.83:0/55877672 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5633e1b3f190 con 0x5633e1bec420 2024-04-24T09:16:05.145 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:05.144+0000 7f17afe00640 10 client.40880 handle_command_reply: tid=0 2024-04-24T09:16:05.145 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:05.145+0000 7f17d83f3740 1 client.40880 shutdown 2024-04-24T09:16:05.147 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:05.147+0000 7f17d83f3740 1 -- 172.21.15.83:0/55877672 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5633e1bd1ad0 msgr2=0x5633e1be48d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:05.147 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:05.147+0000 7f17d83f3740 1 --2- 172.21.15.83:0/55877672 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5633e1bd1ad0 0x5633e1be48d0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f17bc000c00 tx=0x7f17bc0119b0 comp rx=0 tx=0).stop 2024-04-24T09:16:05.147 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:05.147+0000 7f17d83f3740 1 -- 172.21.15.83:0/55877672 shutdown_connections 2024-04-24T09:16:05.147 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:05.147+0000 7f17d83f3740 1 --2- 172.21.15.83:0/55877672 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5633e1bec420 0x5633e1c54690 crc :-1 s=READY pgs=598 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:05.148 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:05.147+0000 7f17d83f3740 1 --2- 172.21.15.83:0/55877672 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5633e1bdc390 0x5633e1bd5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:05.148 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:05.147+0000 7f17d83f3740 1 --2- 172.21.15.83:0/55877672 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5633e1bd1ad0 0x5633e1be48d0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:05.148 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:05.147+0000 7f17d83f3740 1 --2- 172.21.15.83:0/55877672 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5633e1bd11b0 0x5633e1be4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:05.148 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:05.147+0000 7f17d83f3740 1 -- 172.21.15.83:0/55877672 >> 172.21.15.83:0/55877672 conn(0x5633e1bcd6c0 msgr2=0x5633e1bdab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:05.148 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:05.147+0000 7f17afe00640 0 client.40880 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:16:05.148 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:05.147+0000 7f17d83f3740 1 -- 172.21.15.83:0/55877672 shutdown_connections 2024-04-24T09:16:05.148 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:05.148+0000 7f17d83f3740 1 -- 172.21.15.83:0/55877672 wait complete. 2024-04-24T09:16:05.149 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:05.148+0000 7f17d83f3740 20 client.40880 trim_cache size 0 max 16384 2024-04-24T09:16:05.149 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T09:16:05.150 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T09:16:05.150 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T09:16:05.150 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T09:16:05.151 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:05.150+0000 7f17d5600640 1 -- 172.21.15.83:0/3098882458 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f17b40a3e40 msgr2=0x7f17b40a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:05.151 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:05.151+0000 7f17d5600640 1 --2- 172.21.15.83:0/3098882458 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f17b40a3e40 0x7f17b40a6300 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x5633e1bd2750 tx=0x7f17c00095f0 comp rx=0 tx=0).stop 2024-04-24T09:16:05.151 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:05.151+0000 7f17d5600640 1 -- 172.21.15.83:0/3098882458 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f17d0076530 msgr2=0x7f17d019ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:05.151 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:05.151+0000 7f17d5600640 1 --2- 172.21.15.83:0/3098882458 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f17d0076530 0x7f17d019ee40 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f17c002a940 tx=0x7f17c0003c10 comp rx=0 tx=0).stop 2024-04-24T09:16:05.152 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:05.151+0000 7f17d5600640 1 -- 172.21.15.83:0/3098882458 shutdown_connections 2024-04-24T09:16:05.152 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:05.151+0000 7f17d5600640 1 --2- 172.21.15.83:0/3098882458 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f17b40a3e40 0x7f17b40a6300 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:05.152 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:05.151+0000 7f17d5600640 1 --2- 172.21.15.83:0/3098882458 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17d0112ea0 0x7f17d01a3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:05.152 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:05.151+0000 7f17d5600640 1 --2- 172.21.15.83:0/3098882458 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f17d0076e50 0x7f17d019f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:05.152 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:05.151+0000 7f17d5600640 1 --2- 172.21.15.83:0/3098882458 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f17d0076530 0x7f17d019ee40 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:05.153 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:05.151+0000 7f17d5600640 1 -- 172.21.15.83:0/3098882458 >> 172.21.15.83:0/3098882458 conn(0x7f17d00ffd30 msgr2=0x7f17d0102120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:05.153 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:05.151+0000 7f17d5600640 1 -- 172.21.15.83:0/3098882458 shutdown_connections 2024-04-24T09:16:05.153 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:05.152+0000 7f17d5600640 1 -- 172.21.15.83:0/3098882458 wait complete. 2024-04-24T09:16:05.216 INFO:tasks.workunit.client.0.smithi083.stdout: 10 77916 6.53 MB/sec execute 516 sec latency 134.193 ms 2024-04-24T09:16:06.216 INFO:tasks.workunit.client.0.smithi083.stdout: 10 78037 6.53 MB/sec execute 517 sec latency 104.851 ms 2024-04-24T09:16:06.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:16:06 smithi083 ceph-mon[27907]: pgmap v773: 129 pgs: 129 active+clean; 1.2 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.9 MiB/s wr, 183 op/s 2024-04-24T09:16:06.704 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:16:06 smithi138 ceph-mon[31441]: pgmap v773: 129 pgs: 129 active+clean; 1.2 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.9 MiB/s wr, 183 op/s 2024-04-24T09:16:06.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:16:06 smithi189 ceph-mon[31791]: pgmap v773: 129 pgs: 129 active+clean; 1.2 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.9 MiB/s wr, 183 op/s 2024-04-24T09:16:07.216 INFO:tasks.workunit.client.0.smithi083.stdout: 10 78148 6.54 MB/sec execute 518 sec latency 432.071 ms 2024-04-24T09:16:08.216 INFO:tasks.workunit.client.0.smithi083.stdout: 10 78254 6.54 MB/sec execute 519 sec latency 119.840 ms 2024-04-24T09:16:08.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:16:08 smithi083 ceph-mon[27907]: pgmap v774: 129 pgs: 129 active+clean; 1.2 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.9 MiB/s wr, 183 op/s 2024-04-24T09:16:08.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:16:08 smithi083 ceph-mon[27907]: osdmap e131: 12 total, 12 up, 12 in 2024-04-24T09:16:08.704 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:16:08 smithi138 ceph-mon[31441]: pgmap v774: 129 pgs: 129 active+clean; 1.2 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.9 MiB/s wr, 183 op/s 2024-04-24T09:16:08.704 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:16:08 smithi138 ceph-mon[31441]: osdmap e131: 12 total, 12 up, 12 in 2024-04-24T09:16:08.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:16:08 smithi189 ceph-mon[31791]: pgmap v774: 129 pgs: 129 active+clean; 1.2 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.9 MiB/s wr, 183 op/s 2024-04-24T09:16:08.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:16:08 smithi189 ceph-mon[31791]: osdmap e131: 12 total, 12 up, 12 in 2024-04-24T09:16:09.216 INFO:tasks.workunit.client.0.smithi083.stdout: 10 78389 6.54 MB/sec execute 520 sec latency 84.548 ms 2024-04-24T09:16:10.216 INFO:tasks.workunit.client.0.smithi083.stdout: 10 78482 6.53 MB/sec execute 521 sec latency 101.033 ms 2024-04-24T09:16:10.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:16:10 smithi083 ceph-mon[27907]: pgmap v776: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 1.2 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.9 MiB/s wr, 252 op/s 2024-04-24T09:16:10.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:16:10 smithi138 ceph-mon[31441]: pgmap v776: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 1.2 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.9 MiB/s wr, 252 op/s 2024-04-24T09:16:10.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:16:10 smithi189 ceph-mon[31791]: pgmap v776: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 1.2 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.9 MiB/s wr, 252 op/s 2024-04-24T09:16:11.217 INFO:tasks.workunit.client.0.smithi083.stdout: 10 78592 6.53 MB/sec execute 522 sec latency 84.451 ms 2024-04-24T09:16:12.217 INFO:tasks.workunit.client.0.smithi083.stdout: 10 78720 6.53 MB/sec execute 523 sec latency 97.073 ms 2024-04-24T09:16:12.362 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T09:16:12.363 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T09:16:12.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:16:12 smithi083 ceph-mon[27907]: pgmap v777: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 1.2 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.9 MiB/s wr, 252 op/s 2024-04-24T09:16:12.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.616+0000 7fbfcac00640 1 -- 172.21.15.83:0/2005298821 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbfc4104e60 msgr2=0x7fbfc4108ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:12.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.616+0000 7fbfcac00640 1 --2- 172.21.15.83:0/2005298821 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbfc4104e60 0x7fbfc4108ea0 secure :-1 s=READY pgs=2755 cs=0 l=1 rev1=1 crypto rx=0x7fbfb0009900 tx=0x7fbfb002a3f0 comp rx=0 tx=0).stop 2024-04-24T09:16:12.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.616+0000 7fbfcac00640 1 -- 172.21.15.83:0/2005298821 shutdown_connections 2024-04-24T09:16:12.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.616+0000 7fbfcac00640 1 --2- 172.21.15.83:0/2005298821 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbfc4109580 0x7fbfc4111150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:12.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.616+0000 7fbfcac00640 1 --2- 172.21.15.83:0/2005298821 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbfc4104e60 0x7fbfc4108ea0 unknown :-1 s=CLOSED pgs=2755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:12.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.616+0000 7fbfcac00640 1 --2- 172.21.15.83:0/2005298821 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbfc41044b0 0x7fbfc4104890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:12.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.616+0000 7fbfcac00640 1 -- 172.21.15.83:0/2005298821 >> 172.21.15.83:0/2005298821 conn(0x7fbfc40ffd90 msgr2=0x7fbfc41021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:12.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.616+0000 7fbfcac00640 1 -- 172.21.15.83:0/2005298821 shutdown_connections 2024-04-24T09:16:12.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.616+0000 7fbfcac00640 1 -- 172.21.15.83:0/2005298821 wait complete. 2024-04-24T09:16:12.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.617+0000 7fbfcac00640 1 Processor -- start 2024-04-24T09:16:12.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.618+0000 7fbfcac00640 1 -- start start 2024-04-24T09:16:12.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.618+0000 7fbfcac00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbfc41044b0 0x7fbfc4076510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:12.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.618+0000 7fbfcac00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbfc4109580 0x7fbfc4076a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:12.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.618+0000 7fbfcac00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbfc4076f90 0x7fbfc41ab080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:12.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.618+0000 7fbfcac00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc4114510 con 0x7fbfc4076f90 2024-04-24T09:16:12.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.618+0000 7fbfcac00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc4114390 con 0x7fbfc4109580 2024-04-24T09:16:12.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.618+0000 7fbfcac00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc4114690 con 0x7fbfc41044b0 2024-04-24T09:16:12.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.619+0000 7fbfc3400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbfc4109580 0x7fbfc4076a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:12.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.619+0000 7fbfc3400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbfc4109580 0x7fbfc4076a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:51162/0 (socket says 172.21.15.83:51162) 2024-04-24T09:16:12.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.619+0000 7fbfc3400640 1 -- 172.21.15.83:0/2636711822 learned_addr learned my addr 172.21.15.83:0/2636711822 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:16:12.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.619+0000 7fbfc3e00640 1 --2- 172.21.15.83:0/2636711822 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbfc41044b0 0x7fbfc4076510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:12.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.619+0000 7fbfc8c00640 1 --2- 172.21.15.83:0/2636711822 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbfc4076f90 0x7fbfc41ab080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:12.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.620+0000 7fbfc3400640 1 -- 172.21.15.83:0/2636711822 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbfc41044b0 msgr2=0x7fbfc4076510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:12.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.620+0000 7fbfc3400640 1 --2- 172.21.15.83:0/2636711822 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbfc41044b0 0x7fbfc4076510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:12.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.620+0000 7fbfc3400640 1 -- 172.21.15.83:0/2636711822 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbfc4076f90 msgr2=0x7fbfc41ab080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:12.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.620+0000 7fbfc3400640 1 --2- 172.21.15.83:0/2636711822 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbfc4076f90 0x7fbfc41ab080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:12.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.620+0000 7fbfc3400640 1 -- 172.21.15.83:0/2636711822 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfc41ab5c0 con 0x7fbfc4109580 2024-04-24T09:16:12.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.620+0000 7fbfc3400640 1 --2- 172.21.15.83:0/2636711822 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbfc4109580 0x7fbfc4076a50 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fbfb0005ab0 tx=0x7fbfb0003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:12.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.621+0000 7fbfc0c00640 1 -- 172.21.15.83:0/2636711822 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb003f070 con 0x7fbfc4109580 2024-04-24T09:16:12.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.621+0000 7fbfcac00640 1 -- 172.21.15.83:0/2636711822 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfc41ab7f0 con 0x7fbfc4109580 2024-04-24T09:16:12.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.621+0000 7fbfc0c00640 1 -- 172.21.15.83:0/2636711822 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbfb0031b50 con 0x7fbfc4109580 2024-04-24T09:16:12.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.621+0000 7fbfc0c00640 1 -- 172.21.15.83:0/2636711822 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb0035030 con 0x7fbfc4109580 2024-04-24T09:16:12.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.621+0000 7fbfcac00640 1 -- 172.21.15.83:0/2636711822 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfc41abd00 con 0x7fbfc4109580 2024-04-24T09:16:12.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.625+0000 7fbfcd987740 1 Processor -- start 2024-04-24T09:16:12.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.625+0000 7fbfcd987740 1 -- start start 2024-04-24T09:16:12.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.626+0000 7fbfcd987740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565103ab9860 0x565103ab9c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:12.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.626+0000 7fbfcd987740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565103aba180 0x565103ac4310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:12.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.626+0000 7fbfcd987740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565103ac4850 0x565103ac6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:12.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.626+0000 7fbfcd987740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x565103a1f1f0 con 0x565103ac4850 2024-04-24T09:16:12.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.626+0000 7fbfcd987740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5651039c5350 con 0x565103ab9860 2024-04-24T09:16:12.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.626+0000 7fbfcd987740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x565103a1b1b0 con 0x565103aba180 2024-04-24T09:16:12.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.626+0000 7fbfc8c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565103ab9860 0x565103ab9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:12.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.626+0000 7fbfc3e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565103aba180 0x565103ac4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:12.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.626+0000 7fbfc8c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565103ab9860 0x565103ab9c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:51178/0 (socket says 172.21.15.83:51178) 2024-04-24T09:16:12.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.626+0000 7fbfc8c00640 1 -- 172.21.15.83:0/751240260 learned_addr learned my addr 172.21.15.83:0/751240260 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:16:12.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.627+0000 7fbfc3e00640 1 -- 172.21.15.83:0/751240260 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565103ab9860 msgr2=0x565103ab9c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:12.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.627+0000 7fbfc3e00640 1 --2- 172.21.15.83:0/751240260 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565103ab9860 0x565103ab9c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:12.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.627+0000 7fbfc3e00640 1 -- 172.21.15.83:0/751240260 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565103ac4850 msgr2=0x565103ac6c40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T09:16:12.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.627+0000 7fbfc3e00640 1 --2- 172.21.15.83:0/751240260 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565103ac4850 0x565103ac6c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:12.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.627+0000 7fbfc3e00640 1 -- 172.21.15.83:0/751240260 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565103a19f50 con 0x565103aba180 2024-04-24T09:16:12.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.628+0000 7fbfc3e00640 1 --2- 172.21.15.83:0/751240260 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565103aba180 0x565103ac4310 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fbfb4009c20 tx=0x7fbfb4011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fac3ad041e1d2429 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:12.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.628+0000 7fbfc0c00640 1 -- 172.21.15.83:0/2636711822 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fbfb0042070 con 0x7fbfc4109580 2024-04-24T09:16:12.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.628+0000 7fbfb9200640 1 -- 172.21.15.83:0/751240260 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb4009dd0 con 0x565103aba180 2024-04-24T09:16:12.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.629+0000 7fbfb9200640 1 -- 172.21.15.83:0/751240260 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbfb4005020 con 0x565103aba180 2024-04-24T09:16:12.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.629+0000 7fbfb9200640 1 -- 172.21.15.83:0/751240260 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb40049e0 con 0x565103aba180 2024-04-24T09:16:12.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.630+0000 7fbfcd987740 1 -- 172.21.15.83:0/751240260 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565103aba180 msgr2=0x565103ac4310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:12.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.630+0000 7fbfcd987740 1 --2- 172.21.15.83:0/751240260 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565103aba180 0x565103ac4310 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fbfb4009c20 tx=0x7fbfb4011810 comp rx=0 tx=0).stop 2024-04-24T09:16:12.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.630+0000 7fbfcd987740 1 -- 172.21.15.83:0/751240260 shutdown_connections 2024-04-24T09:16:12.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.630+0000 7fbfcd987740 1 --2- 172.21.15.83:0/751240260 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565103ac4850 0x565103ac6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:12.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.630+0000 7fbfcd987740 1 --2- 172.21.15.83:0/751240260 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565103aba180 0x565103ac4310 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:12.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.630+0000 7fbfcd987740 1 --2- 172.21.15.83:0/751240260 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565103ab9860 0x565103ab9c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:12.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.630+0000 7fbfcd987740 1 -- 172.21.15.83:0/751240260 >> 172.21.15.83:0/751240260 conn(0x565103ab5b80 msgr2=0x565103ab5f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:12.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.630+0000 7fbfcd987740 1 -- 172.21.15.83:0/751240260 shutdown_connections 2024-04-24T09:16:12.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.630+0000 7fbfcd987740 1 -- 172.21.15.83:0/751240260 wait complete. 2024-04-24T09:16:12.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.630+0000 7fbfc0c00640 1 --2- 172.21.15.83:0/2636711822 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fbfa80a3c30 0x7fbfa80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:12.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.631+0000 7fbfc0c00640 1 -- 172.21.15.83:0/2636711822 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 9671+0+0 (secure 0 0 0) 0x7fbfb00f0ff0 con 0x7fbfc4109580 2024-04-24T09:16:12.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.631+0000 7fbfcd987740 1 -- start start 2024-04-24T09:16:12.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.631+0000 7fbfcd987740 1 Processor -- start 2024-04-24T09:16:12.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.631+0000 7fbfc3e00640 1 --2- 172.21.15.83:0/2636711822 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fbfa80a3c30 0x7fbfa80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:12.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.632+0000 7fbfcd987740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565103ab9860 0x565103ac1880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:12.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.632+0000 7fbfcd987740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565103aba180 0x565103ac1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:12.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.632+0000 7fbfcd987740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565103ac4850 0x565103ac2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:12.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.632+0000 7fbfcd987740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x565103ac7640 con 0x565103aba180 2024-04-24T09:16:12.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.632+0000 7fbfcd987740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5651039d8690 con 0x565103ab9860 2024-04-24T09:16:12.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.632+0000 7fbfcd987740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x565103ac77c0 con 0x565103ac4850 2024-04-24T09:16:12.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.632+0000 7fbfcd987740 10 client.0 _ll_register_callbacks cb 0x565103a14650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:16:12.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.632+0000 7fbfc3400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565103aba180 0x565103ac1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:12.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.632+0000 7fbfc3400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565103aba180 0x565103ac1dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51750/0 (socket says 172.21.15.83:51750) 2024-04-24T09:16:12.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.632+0000 7fbfc3400640 1 -- 172.21.15.83:0/792637730 learned_addr learned my addr 172.21.15.83:0/792637730 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:16:12.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.632+0000 7fbfc8c00640 1 --2- 172.21.15.83:0/792637730 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565103ac4850 0x565103ac2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:12.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.633+0000 7fbfc3400640 1 -- 172.21.15.83:0/792637730 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565103ac4850 msgr2=0x565103ac2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:12.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.633+0000 7fbfc3400640 1 --2- 172.21.15.83:0/792637730 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565103ac4850 0x565103ac2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:12.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.633+0000 7fbfc3400640 1 -- 172.21.15.83:0/792637730 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565103ab9860 msgr2=0x565103ac1880 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T09:16:12.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.633+0000 7fbfc3400640 1 --2- 172.21.15.83:0/792637730 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565103ab9860 0x565103ac1880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:12.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.633+0000 7fbfc3400640 1 -- 172.21.15.83:0/792637730 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651039d7920 con 0x565103aba180 2024-04-24T09:16:12.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.633+0000 7fbfc8c00640 1 --2- 172.21.15.83:0/792637730 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565103ac4850 0x565103ac2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:16:12.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.633+0000 7fbfc3400640 1 --2- 172.21.15.83:0/792637730 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565103aba180 0x565103ac1dc0 secure :-1 s=READY pgs=2756 cs=0 l=1 rev1=1 crypto rx=0x7fbfb00ee320 tx=0x7fbfb00bc470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9776c2d58e3ab26 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:12.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.633+0000 7fbfad600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:16:12.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.633+0000 7fbfc3e00640 1 --2- 172.21.15.83:0/2636711822 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fbfa80a3c30 0x7fbfa80a60f0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fbfb40099b0 tx=0x7fbfb4000bc0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:12.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.634+0000 7fbfad600640 1 -- 172.21.15.83:0/792637730 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb0049050 con 0x565103aba180 2024-04-24T09:16:12.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.634+0000 7fbfad600640 1 -- 172.21.15.83:0/792637730 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(131..131 src has 1..131) ==== 9671+0+0 (secure 0 0 0) 0x7fbfb00ee540 con 0x565103aba180 2024-04-24T09:16:12.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.634+0000 7fbfcd987740 1 -- 172.21.15.83:0/792637730 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbfb4005020 con 0x565103aba180 2024-04-24T09:16:12.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.634+0000 7fbfad600640 1 -- 172.21.15.83:0/792637730 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7fbfb00c0ec0 con 0x565103aba180 2024-04-24T09:16:12.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.634+0000 7fbfcd987740 10 client.23829 fetch_fsmap learned FSMap version 69 2024-04-24T09:16:12.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.634+0000 7fbfcd987740 1 -- 172.21.15.83:0/792637730 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x5651037fb7d0 con 0x565103aba180 2024-04-24T09:16:12.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.635+0000 7fbfad600640 1 -- 172.21.15.83:0/792637730 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbfb4005020 con 0x565103aba180 2024-04-24T09:16:12.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.635+0000 7fbfad600640 1 -- 172.21.15.83:0/792637730 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 69) ==== 8370+0+0 (secure 0 0 0) 0x7fbfb00d8070 con 0x565103aba180 2024-04-24T09:16:12.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.635+0000 7fbfcd987740 10 client.23829 fetch_fsmap finished waiting for FSMap version 69 2024-04-24T09:16:12.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.635+0000 7fbfcd987740 10 client.23829 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:16:12.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.635+0000 7fbfcd987740 20 client.23829 populate_metadata read hostname 'smithi083' 2024-04-24T09:16:12.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.635+0000 7fbfcd987740 1 --2- 172.21.15.83:0/792637730 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x565103ad4af0 0x565103ad6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:12.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.635+0000 7fbfcd987740 4 client.23829 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:16:12.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.635+0000 7fbfcd987740 1 -- 172.21.15.83:0/792637730 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565103a2cc10 con 0x565103ad4af0 2024-04-24T09:16:12.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.636+0000 7fbfc3e00640 1 --2- 172.21.15.83:0/792637730 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x565103ad4af0 0x565103ad6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:12.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.637+0000 7fbfc3e00640 1 --2- 172.21.15.83:0/792637730 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x565103ad4af0 0x565103ad6ee0 crc :-1 s=READY pgs=599 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77c0fda0df94769f server_cookie=b7a726c71f2bd418 in_seq=0 out_seq=0 2024-04-24T09:16:12.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.637+0000 7fbfad600640 10 client.23829 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:16:12.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.638+0000 7fbfad600640 1 -- 172.21.15.83:0/792637730 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x565103a2cc10 con 0x565103ad4af0 2024-04-24T09:16:12.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.638+0000 7fbfad600640 10 client.23829 handle_command_reply: tid=0 2024-04-24T09:16:12.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.638+0000 7fbfcd987740 1 client.23829 shutdown 2024-04-24T09:16:12.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.638+0000 7fbfcd987740 1 -- 172.21.15.83:0/792637730 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565103aba180 msgr2=0x565103ac1dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:12.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.638+0000 7fbfcd987740 1 --2- 172.21.15.83:0/792637730 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565103aba180 0x565103ac1dc0 secure :-1 s=READY pgs=2756 cs=0 l=1 rev1=1 crypto rx=0x7fbfb00ee320 tx=0x7fbfb00bc470 comp rx=0 tx=0).stop 2024-04-24T09:16:12.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.638+0000 7fbfcd987740 1 -- 172.21.15.83:0/792637730 shutdown_connections 2024-04-24T09:16:12.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.638+0000 7fbfcd987740 1 --2- 172.21.15.83:0/792637730 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x565103ad4af0 0x565103ad6ee0 crc :-1 s=READY pgs=599 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:12.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.638+0000 7fbfcd987740 1 --2- 172.21.15.83:0/792637730 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565103ac4850 0x565103ac2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:12.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.639+0000 7fbfad600640 0 client.23829 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:16:12.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.638+0000 7fbfcd987740 1 --2- 172.21.15.83:0/792637730 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565103aba180 0x565103ac1dc0 unknown :-1 s=CLOSED pgs=2756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:12.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.639+0000 7fbfcd987740 1 --2- 172.21.15.83:0/792637730 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565103ab9860 0x565103ac1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:12.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.639+0000 7fbfcd987740 1 -- 172.21.15.83:0/792637730 >> 172.21.15.83:0/792637730 conn(0x565103ab5b80 msgr2=0x565103abb980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:12.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.639+0000 7fbfcd987740 1 -- 172.21.15.83:0/792637730 shutdown_connections 2024-04-24T09:16:12.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.639+0000 7fbfcd987740 1 -- 172.21.15.83:0/792637730 wait complete. 2024-04-24T09:16:12.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.639+0000 7fbfcd987740 20 client.23829 trim_cache size 0 max 16384 2024-04-24T09:16:12.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.670+0000 7fbfcd987740 1 Processor -- start 2024-04-24T09:16:12.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.670+0000 7fbfcd987740 1 -- start start 2024-04-24T09:16:12.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.670+0000 7fbfcd987740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565103ab9860 0x565103ad0500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:12.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.670+0000 7fbfcd987740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565103aba180 0x565103ac07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:12.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.670+0000 7fbfcd987740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565103ac4850 0x565103ac2120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:12.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.670+0000 7fbfcd987740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc41b5e20 con 0x565103ab9860 2024-04-24T09:16:12.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.670+0000 7fbfcd987740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc41b79c0 con 0x565103ac4850 2024-04-24T09:16:12.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.670+0000 7fbfcd987740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc41b64f0 con 0x565103aba180 2024-04-24T09:16:12.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.671+0000 7fbfc3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565103ab9860 0x565103ad0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:12.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.671+0000 7fbfc3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565103ab9860 0x565103ad0500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51752/0 (socket says 172.21.15.83:51752) 2024-04-24T09:16:12.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.671+0000 7fbfc3e00640 1 -- 172.21.15.83:0/1679579702 learned_addr learned my addr 172.21.15.83:0/1679579702 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:16:12.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.671+0000 7fbfc8c00640 1 --2- 172.21.15.83:0/1679579702 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565103ac4850 0x565103ac2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:12.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.671+0000 7fbfc3400640 1 --2- 172.21.15.83:0/1679579702 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565103aba180 0x565103ac07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:12.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.671+0000 7fbfc3e00640 1 -- 172.21.15.83:0/1679579702 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565103aba180 msgr2=0x565103ac07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:12.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.671+0000 7fbfc3e00640 1 --2- 172.21.15.83:0/1679579702 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565103aba180 0x565103ac07e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:12.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.671+0000 7fbfc3e00640 1 -- 172.21.15.83:0/1679579702 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565103ac4850 msgr2=0x565103ac2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:12.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.671+0000 7fbfc3e00640 1 --2- 172.21.15.83:0/1679579702 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565103ac4850 0x565103ac2120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:12.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.672+0000 7fbfc3e00640 1 -- 172.21.15.83:0/1679579702 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651037313f0 con 0x565103ab9860 2024-04-24T09:16:12.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.672+0000 7fbfc3e00640 1 --2- 172.21.15.83:0/1679579702 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565103ab9860 0x565103ad0500 secure :-1 s=READY pgs=2757 cs=0 l=1 rev1=1 crypto rx=0x7fbfb4011d10 tx=0x7fbfb4022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbf3dabc9d6ebf9e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:12.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.673+0000 7fbfafe00640 1 -- 172.21.15.83:0/1679579702 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb403a030 con 0x565103ab9860 2024-04-24T09:16:12.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.673+0000 7fbfafe00640 1 -- 172.21.15.83:0/1679579702 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbfb4029040 con 0x565103ab9860 2024-04-24T09:16:12.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.673+0000 7fbfafe00640 1 -- 172.21.15.83:0/1679579702 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb4030530 con 0x565103ab9860 2024-04-24T09:16:12.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.674+0000 7fbfcd987740 1 -- 172.21.15.83:0/1679579702 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565103ab9860 msgr2=0x565103ad0500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:12.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.674+0000 7fbfcd987740 1 --2- 172.21.15.83:0/1679579702 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565103ab9860 0x565103ad0500 secure :-1 s=READY pgs=2757 cs=0 l=1 rev1=1 crypto rx=0x7fbfb4011d10 tx=0x7fbfb4022000 comp rx=0 tx=0).stop 2024-04-24T09:16:12.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.674+0000 7fbfcd987740 1 -- 172.21.15.83:0/1679579702 shutdown_connections 2024-04-24T09:16:12.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.674+0000 7fbfcd987740 1 --2- 172.21.15.83:0/1679579702 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565103ac4850 0x565103ac2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:12.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.674+0000 7fbfcd987740 1 --2- 172.21.15.83:0/1679579702 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565103aba180 0x565103ac07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:12.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.674+0000 7fbfcd987740 1 --2- 172.21.15.83:0/1679579702 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565103ab9860 0x565103ad0500 unknown :-1 s=CLOSED pgs=2757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:12.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.674+0000 7fbfcd987740 1 -- 172.21.15.83:0/1679579702 >> 172.21.15.83:0/1679579702 conn(0x565103ab5b80 msgr2=0x565103ab4c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:12.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.674+0000 7fbfcd987740 1 -- 172.21.15.83:0/1679579702 shutdown_connections 2024-04-24T09:16:12.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.674+0000 7fbfcd987740 1 -- 172.21.15.83:0/1679579702 wait complete. 2024-04-24T09:16:12.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.675+0000 7fbfcd987740 1 -- start start 2024-04-24T09:16:12.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.675+0000 7fbfcd987740 1 Processor -- start 2024-04-24T09:16:12.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.676+0000 7fbfcd987740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565103ab9860 0x565103abe920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:12.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.676+0000 7fbfcd987740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565103aba180 0x565103abee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:12.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.676+0000 7fbfcd987740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565103ac4850 0x565103b39f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:12.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.676+0000 7fbfcd987740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc41b72f0 con 0x565103aba180 2024-04-24T09:16:12.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.676+0000 7fbfcd987740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc41b6c20 con 0x565103ab9860 2024-04-24T09:16:12.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.676+0000 7fbfcd987740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc41b3e20 con 0x565103ac4850 2024-04-24T09:16:12.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.676+0000 7fbfcd987740 10 client.0 _ll_register_callbacks cb 0x7fbfb0005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:16:12.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.677+0000 7fbfc3e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565103ab9860 0x565103abe920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:12.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.677+0000 7fbfc3e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565103ab9860 0x565103abe920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:51200/0 (socket says 172.21.15.83:51200) 2024-04-24T09:16:12.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.677+0000 7fbfc3e00640 1 -- 172.21.15.83:0/3546549939 learned_addr learned my addr 172.21.15.83:0/3546549939 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:16:12.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.677+0000 7fbfc8c00640 1 --2- 172.21.15.83:0/3546549939 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565103ac4850 0x565103b39f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:12.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.677+0000 7fbfc3e00640 1 -- 172.21.15.83:0/3546549939 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565103ac4850 msgr2=0x565103b39f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:12.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.677+0000 7fbfc3e00640 1 --2- 172.21.15.83:0/3546549939 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565103ac4850 0x565103b39f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:12.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.677+0000 7fbfc3e00640 1 -- 172.21.15.83:0/3546549939 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565103aba180 msgr2=0x565103abee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:12.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.677+0000 7fbfc3e00640 1 --2- 172.21.15.83:0/3546549939 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565103aba180 0x565103abee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:12.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.677+0000 7fbfc3e00640 1 -- 172.21.15.83:0/3546549939 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651036f10f0 con 0x565103ab9860 2024-04-24T09:16:12.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.678+0000 7fbfc3e00640 1 --2- 172.21.15.83:0/3546549939 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565103ab9860 0x565103abe920 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fbfb40096e0 tx=0x7fbfb40050c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2155c27c85893f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:12.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.678+0000 7fbfad600640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:16:12.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.679+0000 7fbfad600640 1 -- 172.21.15.83:0/3546549939 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb403a070 con 0x565103ab9860 2024-04-24T09:16:12.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.679+0000 7fbfad600640 1 -- 172.21.15.83:0/3546549939 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(131..131 src has 1..131) ==== 9671+0+0 (secure 0 0 0) 0x7fbfb4016040 con 0x565103ab9860 2024-04-24T09:16:12.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.679+0000 7fbfcd987740 1 -- 172.21.15.83:0/3546549939 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbfb4029040 con 0x565103ab9860 2024-04-24T09:16:12.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.679+0000 7fbfad600640 1 -- 172.21.15.83:0/3546549939 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7fbfb4032250 con 0x565103ab9860 2024-04-24T09:16:12.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.679+0000 7fbfcd987740 10 client.40895 fetch_fsmap learned FSMap version 69 2024-04-24T09:16:12.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.679+0000 7fbfcd987740 1 -- 172.21.15.83:0/3546549939 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x5651039e4810 con 0x565103ab9860 2024-04-24T09:16:12.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.680+0000 7fbfad600640 1 -- 172.21.15.83:0/3546549939 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbfb4029040 con 0x565103ab9860 2024-04-24T09:16:12.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.680+0000 7fbfad600640 1 -- 172.21.15.83:0/3546549939 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 69) ==== 8370+0+0 (secure 0 0 0) 0x7fbfb402e040 con 0x565103ab9860 2024-04-24T09:16:12.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.680+0000 7fbfcd987740 10 client.40895 fetch_fsmap finished waiting for FSMap version 69 2024-04-24T09:16:12.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.680+0000 7fbfcd987740 10 client.40895 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:16:12.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.680+0000 7fbfcd987740 20 client.40895 populate_metadata read hostname 'smithi083' 2024-04-24T09:16:12.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.681+0000 7fbfcd987740 1 --2- 172.21.15.83:0/3546549939 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x565103b4c6f0 0x565103b4eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:12.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.681+0000 7fbfcd987740 4 client.40895 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T09:16:12.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.681+0000 7fbfcd987740 1 -- 172.21.15.83:0/3546549939 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub status"}) -- 0x565103a26a80 con 0x565103b4c6f0 2024-04-24T09:16:12.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.681+0000 7fbfc3400640 1 --2- 172.21.15.83:0/3546549939 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x565103b4c6f0 0x565103b4eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:12.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.683+0000 7fbfc3400640 1 --2- 172.21.15.83:0/3546549939 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x565103b4c6f0 0x565103b4eb10 crc :-1 s=READY pgs=600 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2e4f84de142753d server_cookie=792824911052896f in_seq=0 out_seq=0 2024-04-24T09:16:12.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.684+0000 7fbfad600640 10 client.40895 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:16:12.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.684+0000 7fbfad600640 1 -- 172.21.15.83:0/3546549939 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x565103a26a80 con 0x565103b4c6f0 2024-04-24T09:16:12.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.684+0000 7fbfad600640 10 client.40895 handle_command_reply: tid=0 2024-04-24T09:16:12.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.684+0000 7fbfcd987740 1 client.40895 shutdown 2024-04-24T09:16:12.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.684+0000 7fbfcd987740 1 -- 172.21.15.83:0/3546549939 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565103ab9860 msgr2=0x565103abe920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:12.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.684+0000 7fbfcd987740 1 --2- 172.21.15.83:0/3546549939 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565103ab9860 0x565103abe920 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fbfb40096e0 tx=0x7fbfb40050c0 comp rx=0 tx=0).stop 2024-04-24T09:16:12.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.684+0000 7fbfcd987740 1 -- 172.21.15.83:0/3546549939 shutdown_connections 2024-04-24T09:16:12.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.685+0000 7fbfcd987740 1 --2- 172.21.15.83:0/3546549939 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x565103b4c6f0 0x565103b4eb10 crc :-1 s=READY pgs=600 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:12.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.685+0000 7fbfcd987740 1 --2- 172.21.15.83:0/3546549939 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565103ac4850 0x565103b39f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:12.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.685+0000 7fbfad600640 0 client.40895 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:16:12.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.685+0000 7fbfcd987740 1 --2- 172.21.15.83:0/3546549939 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565103aba180 0x565103abee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:12.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.685+0000 7fbfcd987740 1 --2- 172.21.15.83:0/3546549939 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565103ab9860 0x565103abe920 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:12.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.685+0000 7fbfcd987740 1 -- 172.21.15.83:0/3546549939 >> 172.21.15.83:0/3546549939 conn(0x565103ab5b80 msgr2=0x565103abafb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:12.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.685+0000 7fbfcd987740 1 -- 172.21.15.83:0/3546549939 shutdown_connections 2024-04-24T09:16:12.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.685+0000 7fbfcd987740 1 -- 172.21.15.83:0/3546549939 wait complete. 2024-04-24T09:16:12.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.685+0000 7fbfcd987740 20 client.40895 trim_cache size 0 max 16384 2024-04-24T09:16:12.687 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T09:16:12.687 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-04-24T09:16:12.687 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-04-24T09:16:12.687 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T09:16:12.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.686+0000 7fbfcac00640 1 -- 172.21.15.83:0/2636711822 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fbfa80a3c30 msgr2=0x7fbfa80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:12.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.686+0000 7fbfcac00640 1 --2- 172.21.15.83:0/2636711822 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fbfa80a3c30 0x7fbfa80a60f0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fbfb40099b0 tx=0x7fbfb4000bc0 comp rx=0 tx=0).stop 2024-04-24T09:16:12.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.686+0000 7fbfcac00640 1 -- 172.21.15.83:0/2636711822 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbfc4109580 msgr2=0x7fbfc4076a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:12.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.686+0000 7fbfcac00640 1 --2- 172.21.15.83:0/2636711822 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbfc4109580 0x7fbfc4076a50 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fbfb0005ab0 tx=0x7fbfb0003b90 comp rx=0 tx=0).stop 2024-04-24T09:16:12.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.687+0000 7fbfcac00640 1 -- 172.21.15.83:0/2636711822 shutdown_connections 2024-04-24T09:16:12.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.687+0000 7fbfcac00640 1 --2- 172.21.15.83:0/2636711822 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fbfa80a3c30 0x7fbfa80a60f0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:12.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.687+0000 7fbfcac00640 1 --2- 172.21.15.83:0/2636711822 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbfc4076f90 0x7fbfc41ab080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:12.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.687+0000 7fbfcac00640 1 --2- 172.21.15.83:0/2636711822 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbfc4109580 0x7fbfc4076a50 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:12.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.687+0000 7fbfcac00640 1 --2- 172.21.15.83:0/2636711822 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbfc41044b0 0x7fbfc4076510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:12.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.687+0000 7fbfcac00640 1 -- 172.21.15.83:0/2636711822 >> 172.21.15.83:0/2636711822 conn(0x7fbfc40ffd90 msgr2=0x7fbfc4101160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:12.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.687+0000 7fbfcac00640 1 -- 172.21.15.83:0/2636711822 shutdown_connections 2024-04-24T09:16:12.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.687+0000 7fbfcac00640 1 -- 172.21.15.83:0/2636711822 wait complete. 2024-04-24T09:16:12.704 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:16:12 smithi138 ceph-mon[31441]: pgmap v777: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 1.2 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.9 MiB/s wr, 252 op/s 2024-04-24T09:16:12.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:16:12 smithi189 ceph-mon[31791]: pgmap v777: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 1.2 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.9 MiB/s wr, 252 op/s 2024-04-24T09:16:12.709 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T09:16:12.709 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T09:16:12.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.966+0000 7ff101400640 1 -- 172.21.15.83:0/2104312406 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff0fc104890 msgr2=0x7ff0fc104d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:12.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.966+0000 7ff101400640 1 --2- 172.21.15.83:0/2104312406 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff0fc104890 0x7ff0fc104d40 secure :-1 s=READY pgs=2758 cs=0 l=1 rev1=1 crypto rx=0x7ff0ec009900 tx=0x7ff0ec02a3f0 comp rx=0 tx=0).stop 2024-04-24T09:16:12.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.966+0000 7ff101400640 1 -- 172.21.15.83:0/2104312406 shutdown_connections 2024-04-24T09:16:12.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.966+0000 7ff101400640 1 --2- 172.21.15.83:0/2104312406 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff0fc104890 0x7ff0fc104d40 unknown :-1 s=CLOSED pgs=2758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:12.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.966+0000 7ff101400640 1 --2- 172.21.15.83:0/2104312406 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff0fc1071b0 0x7ff0fc0696d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:12.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.966+0000 7ff101400640 1 --2- 172.21.15.83:0/2104312406 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff0fc106d40 0x7ff0fc069190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:12.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.966+0000 7ff101400640 1 -- 172.21.15.83:0/2104312406 >> 172.21.15.83:0/2104312406 conn(0x7ff0fc076840 msgr2=0x7ff0fc076c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:12.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.967+0000 7ff101400640 1 -- 172.21.15.83:0/2104312406 shutdown_connections 2024-04-24T09:16:12.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.967+0000 7ff101400640 1 -- 172.21.15.83:0/2104312406 wait complete. 2024-04-24T09:16:12.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.968+0000 7ff101400640 1 Processor -- start 2024-04-24T09:16:12.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.968+0000 7ff101400640 1 -- start start 2024-04-24T09:16:12.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.969+0000 7ff101400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff0fc104890 0x7ff0fc1a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:12.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.969+0000 7ff101400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff0fc106d40 0x7ff0fc1a4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:12.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.969+0000 7ff101400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff0fc1071b0 0x7ff0fc19e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:12.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.969+0000 7ff101400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff0fc118630 con 0x7ff0fc1071b0 2024-04-24T09:16:12.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.969+0000 7ff101400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff0fc1184b0 con 0x7ff0fc104890 2024-04-24T09:16:12.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.969+0000 7ff101400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff0fc1187b0 con 0x7ff0fc106d40 2024-04-24T09:16:12.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.969+0000 7ff100a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff0fc1071b0 0x7ff0fc19e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:12.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.969+0000 7ff100a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff0fc1071b0 0x7ff0fc19e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51790/0 (socket says 172.21.15.83:51790) 2024-04-24T09:16:12.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.969+0000 7ff100a00640 1 -- 172.21.15.83:0/478839531 learned_addr learned my addr 172.21.15.83:0/478839531 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:16:12.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.969+0000 7ff0fb400640 1 --2- 172.21.15.83:0/478839531 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff0fc106d40 0x7ff0fc1a4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:12.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.970+0000 7ff0fbe00640 1 --2- 172.21.15.83:0/478839531 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff0fc104890 0x7ff0fc1a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:12.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.970+0000 7ff100a00640 1 -- 172.21.15.83:0/478839531 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff0fc106d40 msgr2=0x7ff0fc1a4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:12.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.970+0000 7ff100a00640 1 --2- 172.21.15.83:0/478839531 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff0fc106d40 0x7ff0fc1a4cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:12.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.970+0000 7ff100a00640 1 -- 172.21.15.83:0/478839531 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff0fc104890 msgr2=0x7ff0fc1a4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:12.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.970+0000 7ff100a00640 1 --2- 172.21.15.83:0/478839531 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff0fc104890 0x7ff0fc1a4770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:12.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.970+0000 7ff100a00640 1 -- 172.21.15.83:0/478839531 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0fc19f0f0 con 0x7ff0fc1071b0 2024-04-24T09:16:12.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.970+0000 7ff100a00640 1 --2- 172.21.15.83:0/478839531 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff0fc1071b0 0x7ff0fc19e970 secure :-1 s=READY pgs=2759 cs=0 l=1 rev1=1 crypto rx=0x7ff0ec02a910 tx=0x7ff0ec003c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:12.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.971+0000 7ff0f8c00640 1 -- 172.21.15.83:0/478839531 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0ec03f070 con 0x7ff0fc1071b0 2024-04-24T09:16:12.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.971+0000 7ff0f8c00640 1 -- 172.21.15.83:0/478839531 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff0ec003e20 con 0x7ff0fc1071b0 2024-04-24T09:16:12.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.971+0000 7ff0f8c00640 1 -- 172.21.15.83:0/478839531 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0ec035030 con 0x7ff0fc1071b0 2024-04-24T09:16:12.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.971+0000 7ff101400640 1 -- 172.21.15.83:0/478839531 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0fc19f3e0 con 0x7ff0fc1071b0 2024-04-24T09:16:12.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.971+0000 7ff101400640 1 -- 172.21.15.83:0/478839531 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0fc1ab630 con 0x7ff0fc1071b0 2024-04-24T09:16:12.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.972+0000 7ff101400640 1 -- 172.21.15.83:0/478839531 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0fc068d40 con 0x7ff0fc1071b0 2024-04-24T09:16:12.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.976+0000 7ff0f8c00640 1 -- 172.21.15.83:0/478839531 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff0ec042070 con 0x7ff0fc1071b0 2024-04-24T09:16:12.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.980+0000 7ff0f8c00640 1 --2- 172.21.15.83:0/478839531 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff0dc0a3e70 0x7ff0dc0a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:12.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.980+0000 7ff0f8c00640 1 -- 172.21.15.83:0/478839531 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 9671+0+0 (secure 0 0 0) 0x7ff0ec0f1a40 con 0x7ff0fc1071b0 2024-04-24T09:16:12.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.980+0000 7ff0fbe00640 1 --2- 172.21.15.83:0/478839531 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff0dc0a3e70 0x7ff0dc0a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:12.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.982+0000 7ff0fbe00640 1 --2- 172.21.15.83:0/478839531 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff0dc0a3e70 0x7ff0dc0a6330 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc1018d0 tx=0x7ff0f4009500 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:12.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:12.982+0000 7ff0f8c00640 1 -- 172.21.15.83:0/478839531 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff0ec049070 con 0x7ff0fc1071b0 2024-04-24T09:16:13.217 INFO:tasks.workunit.client.0.smithi083.stdout: 10 78831 6.53 MB/sec execute 524 sec latency 92.712 ms 2024-04-24T09:16:13.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.445+0000 7ff101400640 1 -- 172.21.15.83:0/478839531 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff0fc19fe50 con 0x7ff0fc1071b0 2024-04-24T09:16:13.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.446+0000 7ff0f8c00640 1 -- 172.21.15.83:0/478839531 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) ==== 94+0+11947 (secure 0 0 0) 0x7ff0fc19fe50 con 0x7ff0fc1071b0 2024-04-24T09:16:13.447 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T09:16:13.447 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":69,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/358872257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":358872257},{"type":"v1","addr":"172.21.15.83:6835","nonce":358872257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3157070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3157070851},{"type":"v1","addr":"172.21.15.83:6837","nonce":3157070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":69,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T08:48:29.422576+0000","modified":"2024-04-24T09:15:17.129816+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34520,"mds_1":34493,"mds_2":34451,"mds_3":34427,"mds_4":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14658":{"gid":14658,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6839/4085470844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":4085470844},{"type":"v1","addr":"172.21.15.83:6839","nonce":4085470844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14676":{"gid":14676,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6841/2634754210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":2634754210},{"type":"v1","addr":"172.21.15.83:6841","nonce":2634754210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24382":{"gid":24382,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/1859921310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1859921310},{"type":"v1","addr":"172.21.15.189:6833","nonce":1859921310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/2241923197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2241923197},{"type":"v1","addr":"172.21.15.189:6835","nonce":2241923197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.189:6839/2548965336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":2548965336},{"type":"v1","addr":"172.21.15.189:6839","nonce":2548965336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26686":{"gid":26686,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6837/1678817161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":1678817161},{"type":"v1","addr":"172.21.15.189:6837","nonce":1678817161}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.138:6833/1157967658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1157967658},{"type":"v1","addr":"172.21.15.138:6833","nonce":1157967658}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.138:6835/1909219758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1909219758},{"type":"v1","addr":"172.21.15.138:6835","nonce":1909219758}]},"join_fscid":-1,"export_targets":[0,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34493":{"gid":34493,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.138:6837/900566754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":900566754},{"type":"v1","addr":"172.21.15.138:6837","nonce":900566754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34520":{"gid":34520,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.138:6839/2236932393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2236932393},{"type":"v1","addr":"172.21.15.138:6839","nonce":2236932393}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34520,"qdb_cluster":[24454,34520,34427,34451,34493]},"id":1}]} 2024-04-24T09:16:13.447 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 69 2024-04-24T09:16:13.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.450+0000 7ff101400640 1 -- 172.21.15.83:0/478839531 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff0dc0a3e70 msgr2=0x7ff0dc0a6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:13.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.450+0000 7ff101400640 1 --2- 172.21.15.83:0/478839531 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff0dc0a3e70 0x7ff0dc0a6330 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc1018d0 tx=0x7ff0f4009500 comp rx=0 tx=0).stop 2024-04-24T09:16:13.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.450+0000 7ff101400640 1 -- 172.21.15.83:0/478839531 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff0fc1071b0 msgr2=0x7ff0fc19e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:13.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.450+0000 7ff101400640 1 --2- 172.21.15.83:0/478839531 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff0fc1071b0 0x7ff0fc19e970 secure :-1 s=READY pgs=2759 cs=0 l=1 rev1=1 crypto rx=0x7ff0ec02a910 tx=0x7ff0ec003c00 comp rx=0 tx=0).stop 2024-04-24T09:16:13.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.450+0000 7ff101400640 1 -- 172.21.15.83:0/478839531 shutdown_connections 2024-04-24T09:16:13.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.450+0000 7ff101400640 1 --2- 172.21.15.83:0/478839531 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff0dc0a3e70 0x7ff0dc0a6330 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:13.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.451+0000 7ff101400640 1 --2- 172.21.15.83:0/478839531 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff0fc1071b0 0x7ff0fc19e970 unknown :-1 s=CLOSED pgs=2759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:13.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.451+0000 7ff101400640 1 --2- 172.21.15.83:0/478839531 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff0fc106d40 0x7ff0fc1a4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:13.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.451+0000 7ff101400640 1 --2- 172.21.15.83:0/478839531 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff0fc104890 0x7ff0fc1a4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:13.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.451+0000 7ff101400640 1 -- 172.21.15.83:0/478839531 >> 172.21.15.83:0/478839531 conn(0x7ff0fc076840 msgr2=0x7ff0fc101d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:13.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.451+0000 7ff101400640 1 -- 172.21.15.83:0/478839531 shutdown_connections 2024-04-24T09:16:13.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.451+0000 7ff101400640 1 -- 172.21.15.83:0/478839531 wait complete. 2024-04-24T09:16:13.480 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-24T09:16:13.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.733+0000 7faa71a00640 1 -- 172.21.15.83:0/3613004692 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa6c108e40 msgr2=0x7faa6c1092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:13.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.733+0000 7faa71a00640 1 --2- 172.21.15.83:0/3613004692 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa6c108e40 0x7faa6c1092c0 secure :-1 s=READY pgs=2760 cs=0 l=1 rev1=1 crypto rx=0x7faa58009900 tx=0x7faa5802a3f0 comp rx=0 tx=0).stop 2024-04-24T09:16:13.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.733+0000 7faa71a00640 1 -- 172.21.15.83:0/3613004692 shutdown_connections 2024-04-24T09:16:13.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.733+0000 7faa71a00640 1 --2- 172.21.15.83:0/3613004692 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa6c109800 0x7faa6c115360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:13.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.733+0000 7faa71a00640 1 --2- 172.21.15.83:0/3613004692 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa6c108e40 0x7faa6c1092c0 secure :-1 s=CLOSED pgs=2760 cs=0 l=1 rev1=1 crypto rx=0x7faa58009900 tx=0x7faa5802a3f0 comp rx=0 tx=0).stop 2024-04-24T09:16:13.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.733+0000 7faa71a00640 1 --2- 172.21.15.83:0/3613004692 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faa6c10e5a0 0x7faa6c10e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:13.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.733+0000 7faa71a00640 1 -- 172.21.15.83:0/3613004692 >> 172.21.15.83:0/3613004692 conn(0x7faa6c077a20 msgr2=0x7faa6c079e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:13.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.733+0000 7faa71a00640 1 -- 172.21.15.83:0/3613004692 shutdown_connections 2024-04-24T09:16:13.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.733+0000 7faa71a00640 1 -- 172.21.15.83:0/3613004692 wait complete. 2024-04-24T09:16:13.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.734+0000 7faa71a00640 1 Processor -- start 2024-04-24T09:16:13.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.735+0000 7faa71a00640 1 -- start start 2024-04-24T09:16:13.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.735+0000 7faa71a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa6c109800 0x7faa6c19ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:13.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.735+0000 7faa71a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faa6c10e5a0 0x7faa6c19f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:13.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.735+0000 7faa71a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa6c1a36d0 0x7faa6c1a3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:13.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.735+0000 7faa71a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7faa6c118760 con 0x7faa6c109800 2024-04-24T09:16:13.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.735+0000 7faa71a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7faa6c1185e0 con 0x7faa6c1a36d0 2024-04-24T09:16:13.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.735+0000 7faa71a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7faa6c1188e0 con 0x7faa6c10e5a0 2024-04-24T09:16:13.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.736+0000 7faa71000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa6c1a36d0 0x7faa6c1a3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:13.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.736+0000 7faa71000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa6c1a36d0 0x7faa6c1a3b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:51238/0 (socket says 172.21.15.83:51238) 2024-04-24T09:16:13.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.736+0000 7faa71000640 1 -- 172.21.15.83:0/2269346814 learned_addr learned my addr 172.21.15.83:0/2269346814 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:16:13.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.736+0000 7faa6b400640 1 --2- 172.21.15.83:0/2269346814 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faa6c10e5a0 0x7faa6c19f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:13.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.737+0000 7faa71000640 1 -- 172.21.15.83:0/2269346814 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faa6c10e5a0 msgr2=0x7faa6c19f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:13.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.737+0000 7faa71000640 1 --2- 172.21.15.83:0/2269346814 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faa6c10e5a0 0x7faa6c19f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:13.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.737+0000 7faa6be00640 1 --2- 172.21.15.83:0/2269346814 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa6c109800 0x7faa6c19ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:13.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.737+0000 7faa71000640 1 -- 172.21.15.83:0/2269346814 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa6c109800 msgr2=0x7faa6c19ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:13.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.737+0000 7faa71000640 1 --2- 172.21.15.83:0/2269346814 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa6c109800 0x7faa6c19ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:13.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.737+0000 7faa71000640 1 -- 172.21.15.83:0/2269346814 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa6c1a41c0 con 0x7faa6c1a36d0 2024-04-24T09:16:13.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.737+0000 7faa6b400640 1 --2- 172.21.15.83:0/2269346814 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faa6c10e5a0 0x7faa6c19f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:16:13.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.737+0000 7faa6be00640 1 --2- 172.21.15.83:0/2269346814 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa6c109800 0x7faa6c19ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:16:13.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.737+0000 7faa71000640 1 --2- 172.21.15.83:0/2269346814 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa6c1a36d0 0x7faa6c1a3b80 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7faa6400ea60 tx=0x7faa6400ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:13.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.737+0000 7faa68c00640 1 -- 172.21.15.83:0/2269346814 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa6400ce40 con 0x7faa6c1a36d0 2024-04-24T09:16:13.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.737+0000 7faa71a00640 1 -- 172.21.15.83:0/2269346814 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa6c1a44b0 con 0x7faa6c1a36d0 2024-04-24T09:16:13.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.737+0000 7faa68c00640 1 -- 172.21.15.83:0/2269346814 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faa64005020 con 0x7faa6c1a36d0 2024-04-24T09:16:13.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.737+0000 7faa68c00640 1 -- 172.21.15.83:0/2269346814 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa64010670 con 0x7faa6c1a36d0 2024-04-24T09:16:13.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.738+0000 7faa71a00640 1 -- 172.21.15.83:0/2269346814 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa6c1abca0 con 0x7faa6c1a36d0 2024-04-24T09:16:13.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.742+0000 7faa68c00640 1 -- 172.21.15.83:0/2269346814 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7faa640051d0 con 0x7faa6c1a36d0 2024-04-24T09:16:13.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.744+0000 7faa68c00640 1 --2- 172.21.15.83:0/2269346814 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7faa500a3de0 0x7faa500a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:13.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.744+0000 7faa68c00640 1 -- 172.21.15.83:0/2269346814 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 9671+0+0 (secure 0 0 0) 0x7faa640d12d0 con 0x7faa6c1a36d0 2024-04-24T09:16:13.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.745+0000 7faa75fc5740 1 Processor -- start 2024-04-24T09:16:13.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.745+0000 7faa6be00640 1 --2- 172.21.15.83:0/2269346814 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7faa500a3de0 0x7faa500a62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:13.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.745+0000 7faa75fc5740 1 -- start start 2024-04-24T09:16:13.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.745+0000 7faa75fc5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dd97377b40 0x55dd97377f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:13.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.745+0000 7faa75fc5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd97378460 0x55dd973825f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:13.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.745+0000 7faa75fc5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd97382b30 0x55dd97384f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:13.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.745+0000 7faa75fc5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55dd972dd1d0 con 0x55dd97382b30 2024-04-24T09:16:13.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.745+0000 7faa75fc5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55dd97283450 con 0x55dd97377b40 2024-04-24T09:16:13.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.745+0000 7faa75fc5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55dd972d9190 con 0x55dd97378460 2024-04-24T09:16:13.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.746+0000 7faa71000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd97382b30 0x55dd97384f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:13.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.746+0000 7faa6be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dd97377b40 0x55dd97377f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:13.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.746+0000 7faa71000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd97382b30 0x55dd97384f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51828/0 (socket says 172.21.15.83:51828) 2024-04-24T09:16:13.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.746+0000 7faa71000640 1 -- 172.21.15.83:0/3983867749 learned_addr learned my addr 172.21.15.83:0/3983867749 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:16:13.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.746+0000 7faa6b400640 1 --2- 172.21.15.83:0/3983867749 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd97378460 0x55dd973825f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:13.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.746+0000 7faa71000640 1 -- 172.21.15.83:0/3983867749 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd97378460 msgr2=0x55dd973825f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:13.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.746+0000 7faa71000640 1 --2- 172.21.15.83:0/3983867749 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd97378460 0x55dd973825f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:13.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.746+0000 7faa71000640 1 -- 172.21.15.83:0/3983867749 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dd97377b40 msgr2=0x55dd97377f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:13.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.746+0000 7faa71000640 1 --2- 172.21.15.83:0/3983867749 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dd97377b40 0x55dd97377f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:13.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.746+0000 7faa71000640 1 -- 172.21.15.83:0/3983867749 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd97340310 con 0x55dd97382b30 2024-04-24T09:16:13.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.747+0000 7faa6be00640 1 --2- 172.21.15.83:0/2269346814 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7faa500a3de0 0x7faa500a62a0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7faa5c015420 tx=0x7faa5c015820 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:13.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.747+0000 7faa71000640 1 --2- 172.21.15.83:0/3983867749 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd97382b30 0x55dd97384f20 secure :-1 s=READY pgs=2761 cs=0 l=1 rev1=1 crypto rx=0x7faa64009730 tx=0x7faa6409ce50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f19cfc55ac847b2f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:13.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.747+0000 7faa61600640 1 -- 172.21.15.83:0/3983867749 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa640b6070 con 0x55dd97382b30 2024-04-24T09:16:13.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.748+0000 7faa61600640 1 -- 172.21.15.83:0/3983867749 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faa64016dd0 con 0x55dd97382b30 2024-04-24T09:16:13.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.748+0000 7faa61600640 1 -- 172.21.15.83:0/3983867749 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa64099400 con 0x55dd97382b30 2024-04-24T09:16:13.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.749+0000 7faa75fc5740 1 -- 172.21.15.83:0/3983867749 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd97382b30 msgr2=0x55dd97384f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:13.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.749+0000 7faa75fc5740 1 --2- 172.21.15.83:0/3983867749 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd97382b30 0x55dd97384f20 secure :-1 s=READY pgs=2761 cs=0 l=1 rev1=1 crypto rx=0x7faa64009730 tx=0x7faa6409ce50 comp rx=0 tx=0).stop 2024-04-24T09:16:13.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.749+0000 7faa75fc5740 1 -- 172.21.15.83:0/3983867749 shutdown_connections 2024-04-24T09:16:13.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.749+0000 7faa75fc5740 1 --2- 172.21.15.83:0/3983867749 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd97382b30 0x55dd97384f20 unknown :-1 s=CLOSED pgs=2761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:13.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.749+0000 7faa75fc5740 1 --2- 172.21.15.83:0/3983867749 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd97378460 0x55dd973825f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:13.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.749+0000 7faa75fc5740 1 --2- 172.21.15.83:0/3983867749 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dd97377b40 0x55dd97377f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:13.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.749+0000 7faa75fc5740 1 -- 172.21.15.83:0/3983867749 >> 172.21.15.83:0/3983867749 conn(0x55dd97373e60 msgr2=0x55dd97374240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:13.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.749+0000 7faa75fc5740 1 -- 172.21.15.83:0/3983867749 shutdown_connections 2024-04-24T09:16:13.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.749+0000 7faa75fc5740 1 -- 172.21.15.83:0/3983867749 wait complete. 2024-04-24T09:16:13.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.750+0000 7faa75fc5740 1 -- start start 2024-04-24T09:16:13.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.750+0000 7faa75fc5740 1 Processor -- start 2024-04-24T09:16:13.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.751+0000 7faa75fc5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd97377b40 0x55dd973825e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:13.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.751+0000 7faa75fc5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd97378460 0x55dd9737f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:13.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.751+0000 7faa75fc5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dd97382b30 0x55dd97380010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:13.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.751+0000 7faa75fc5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55dd973859c0 con 0x55dd97377b40 2024-04-24T09:16:13.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.751+0000 7faa75fc5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55dd97296810 con 0x55dd97382b30 2024-04-24T09:16:13.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.751+0000 7faa75fc5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55dd97385b40 con 0x55dd97378460 2024-04-24T09:16:13.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.751+0000 7faa75fc5740 10 client.0 _ll_register_callbacks cb 0x55dd97264140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:16:13.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.751+0000 7faa71000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dd97382b30 0x55dd97380010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:13.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.751+0000 7faa71000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dd97382b30 0x55dd97380010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:51264/0 (socket says 172.21.15.83:51264) 2024-04-24T09:16:13.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.751+0000 7faa71000640 1 -- 172.21.15.83:0/2240094909 learned_addr learned my addr 172.21.15.83:0/2240094909 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:16:13.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.751+0000 7faa6b400640 1 --2- 172.21.15.83:0/2240094909 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd97378460 0x55dd9737f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:13.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.752+0000 7faa71000640 1 -- 172.21.15.83:0/2240094909 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd97378460 msgr2=0x55dd9737f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:13.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.752+0000 7faa71000640 1 --2- 172.21.15.83:0/2240094909 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd97378460 0x55dd9737f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:13.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.752+0000 7faa71000640 1 -- 172.21.15.83:0/2240094909 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd97377b40 msgr2=0x55dd973825e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:13.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.752+0000 7faa71000640 1 --2- 172.21.15.83:0/2240094909 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd97377b40 0x55dd973825e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:13.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.752+0000 7faa71000640 1 -- 172.21.15.83:0/2240094909 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd972a73f0 con 0x55dd97382b30 2024-04-24T09:16:13.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.752+0000 7faa71000640 1 --2- 172.21.15.83:0/2240094909 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dd97382b30 0x55dd97380010 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7faa640991f0 tx=0x7faa640096c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8493ec9c84d8ce46 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:13.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.752+0000 7faa56000640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:16:13.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.752+0000 7faa56000640 1 -- 172.21.15.83:0/2240094909 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa640ba070 con 0x55dd97382b30 2024-04-24T09:16:13.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.752+0000 7faa56000640 1 -- 172.21.15.83:0/2240094909 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(131..131 src has 1..131) ==== 9671+0+0 (secure 0 0 0) 0x7faa64099400 con 0x55dd97382b30 2024-04-24T09:16:13.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.752+0000 7faa75fc5740 1 -- 172.21.15.83:0/2240094909 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa64016dd0 con 0x55dd97382b30 2024-04-24T09:16:13.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.753+0000 7faa56000640 1 -- 172.21.15.83:0/2240094909 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7faa640b6040 con 0x55dd97382b30 2024-04-24T09:16:13.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.753+0000 7faa75fc5740 10 client.40913 fetch_fsmap learned FSMap version 69 2024-04-24T09:16:13.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.753+0000 7faa75fc5740 1 -- 172.21.15.83:0/2240094909 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x55dd96fef3f0 con 0x55dd97382b30 2024-04-24T09:16:13.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.754+0000 7faa56000640 1 -- 172.21.15.83:0/2240094909 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faa64016dd0 con 0x55dd97382b30 2024-04-24T09:16:13.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.754+0000 7faa56000640 1 -- 172.21.15.83:0/2240094909 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 69) ==== 8370+0+0 (secure 0 0 0) 0x7faa640bd0d0 con 0x55dd97382b30 2024-04-24T09:16:13.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.754+0000 7faa75fc5740 10 client.40913 fetch_fsmap finished waiting for FSMap version 69 2024-04-24T09:16:13.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.754+0000 7faa75fc5740 10 client.40913 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-24T09:16:13.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.754+0000 7faa75fc5740 20 client.40913 populate_metadata read hostname 'smithi083' 2024-04-24T09:16:13.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.755+0000 7faa75fc5740 1 --2- 172.21.15.83:0/2240094909 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55dd97392bf0 0x55dd97394fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:13.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.755+0000 7faa75fc5740 4 client.40913 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:16:13.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.755+0000 7faa75fc5740 1 -- 172.21.15.83:0/2240094909 --> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dd972eabf0 con 0x55dd97392bf0 2024-04-24T09:16:13.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.755+0000 7faa6be00640 1 --2- 172.21.15.83:0/2240094909 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55dd97392bf0 0x55dd97394fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:13.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.759+0000 7faa6be00640 1 --2- 172.21.15.83:0/2240094909 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55dd97392bf0 0x55dd97394fe0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=df67d3fc3580f223 server_cookie=3d1bb3f0d71718a5 in_seq=0 out_seq=0 2024-04-24T09:16:13.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.759+0000 7faa56000640 10 client.40913 ms_handle_connect on v2:172.21.15.189:6838/2548965336 2024-04-24T09:16:13.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.763+0000 7faa56000640 1 -- 172.21.15.83:0/2240094909 <== mds.4 v2:172.21.15.189:6838/2548965336 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55dd972eabf0 con 0x55dd97392bf0 2024-04-24T09:16:13.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.763+0000 7faa56000640 10 client.40913 handle_command_reply: tid=0 2024-04-24T09:16:13.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.763+0000 7faa75fc5740 1 client.40913 shutdown 2024-04-24T09:16:13.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.764+0000 7faa75fc5740 1 -- 172.21.15.83:0/2240094909 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dd97382b30 msgr2=0x55dd97380010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:13.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.764+0000 7faa75fc5740 1 --2- 172.21.15.83:0/2240094909 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dd97382b30 0x55dd97380010 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7faa640991f0 tx=0x7faa640096c0 comp rx=0 tx=0).stop 2024-04-24T09:16:13.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.764+0000 7faa75fc5740 1 -- 172.21.15.83:0/2240094909 shutdown_connections 2024-04-24T09:16:13.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.764+0000 7faa75fc5740 1 --2- 172.21.15.83:0/2240094909 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55dd97392bf0 0x55dd97394fe0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:13.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.764+0000 7faa75fc5740 1 --2- 172.21.15.83:0/2240094909 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dd97382b30 0x55dd97380010 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:13.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.764+0000 7faa75fc5740 1 --2- 172.21.15.83:0/2240094909 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd97378460 0x55dd9737f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:13.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.764+0000 7faa75fc5740 1 --2- 172.21.15.83:0/2240094909 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd97377b40 0x55dd973825e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:13.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.764+0000 7faa56000640 0 client.40913 ms_handle_reset on v2:172.21.15.189:6838/2548965336 2024-04-24T09:16:13.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.764+0000 7faa75fc5740 1 -- 172.21.15.83:0/2240094909 >> 172.21.15.83:0/2240094909 conn(0x55dd97373e60 msgr2=0x55dd97379c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:13.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.765+0000 7faa75fc5740 1 -- 172.21.15.83:0/2240094909 shutdown_connections 2024-04-24T09:16:13.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.765+0000 7faa75fc5740 1 -- 172.21.15.83:0/2240094909 wait complete. 2024-04-24T09:16:13.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.765+0000 7faa75fc5740 20 client.40913 trim_cache size 0 max 16384 2024-04-24T09:16:13.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.795+0000 7faa75fc5740 1 Processor -- start 2024-04-24T09:16:13.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.795+0000 7faa75fc5740 1 -- start start 2024-04-24T09:16:13.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.796+0000 7faa75fc5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd97377b40 0x55dd9737a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:13.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.796+0000 7faa75fc5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd97378460 0x55dd97382110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:13.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.796+0000 7faa75fc5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dd97382b30 0x55dd97383f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:13.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.796+0000 7faa75fc5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7faa6c1b6190 con 0x55dd97377b40 2024-04-24T09:16:13.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.796+0000 7faa75fc5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7faa6c1b7d30 con 0x55dd97382b30 2024-04-24T09:16:13.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.796+0000 7faa75fc5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7faa6c1b6860 con 0x55dd97378460 2024-04-24T09:16:13.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.796+0000 7faa6be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd97377b40 0x55dd9737a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:13.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.796+0000 7faa6be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd97377b40 0x55dd9737a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51850/0 (socket says 172.21.15.83:51850) 2024-04-24T09:16:13.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.796+0000 7faa6be00640 1 -- 172.21.15.83:0/3758640304 learned_addr learned my addr 172.21.15.83:0/3758640304 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:16:13.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.797+0000 7faa6b400640 1 --2- 172.21.15.83:0/3758640304 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd97378460 0x55dd97382110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:13.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.797+0000 7faa71000640 1 --2- 172.21.15.83:0/3758640304 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dd97382b30 0x55dd97383f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:13.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.797+0000 7faa6be00640 1 -- 172.21.15.83:0/3758640304 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd97378460 msgr2=0x55dd97382110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:13.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.797+0000 7faa6be00640 1 --2- 172.21.15.83:0/3758640304 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd97378460 0x55dd97382110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:13.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.797+0000 7faa6be00640 1 -- 172.21.15.83:0/3758640304 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dd97382b30 msgr2=0x55dd97383f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:13.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.797+0000 7faa6be00640 1 --2- 172.21.15.83:0/3758640304 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dd97382b30 0x55dd97383f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:13.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.797+0000 7faa6be00640 1 -- 172.21.15.83:0/3758640304 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd970b97d0 con 0x55dd97377b40 2024-04-24T09:16:13.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.797+0000 7faa6be00640 1 --2- 172.21.15.83:0/3758640304 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd97377b40 0x55dd9737a2e0 secure :-1 s=READY pgs=2762 cs=0 l=1 rev1=1 crypto rx=0x7faa5c009310 tx=0x7faa5c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=869a954003113a1d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:13.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.798+0000 7faa60c00640 1 -- 172.21.15.83:0/3758640304 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa5c0151e0 con 0x55dd97377b40 2024-04-24T09:16:13.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.798+0000 7faa60c00640 1 -- 172.21.15.83:0/3758640304 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faa5c012db0 con 0x55dd97377b40 2024-04-24T09:16:13.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.798+0000 7faa60c00640 1 -- 172.21.15.83:0/3758640304 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa5c0272d0 con 0x55dd97377b40 2024-04-24T09:16:13.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.799+0000 7faa75fc5740 1 -- 172.21.15.83:0/3758640304 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd97377b40 msgr2=0x55dd9737a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:13.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.799+0000 7faa75fc5740 1 --2- 172.21.15.83:0/3758640304 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd97377b40 0x55dd9737a2e0 secure :-1 s=READY pgs=2762 cs=0 l=1 rev1=1 crypto rx=0x7faa5c009310 tx=0x7faa5c018000 comp rx=0 tx=0).stop 2024-04-24T09:16:13.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.799+0000 7faa75fc5740 1 -- 172.21.15.83:0/3758640304 shutdown_connections 2024-04-24T09:16:13.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.799+0000 7faa75fc5740 1 --2- 172.21.15.83:0/3758640304 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dd97382b30 0x55dd97383f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:13.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.799+0000 7faa75fc5740 1 --2- 172.21.15.83:0/3758640304 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd97378460 0x55dd97382110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:13.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.799+0000 7faa75fc5740 1 --2- 172.21.15.83:0/3758640304 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd97377b40 0x55dd9737a2e0 unknown :-1 s=CLOSED pgs=2762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:13.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.799+0000 7faa75fc5740 1 -- 172.21.15.83:0/3758640304 >> 172.21.15.83:0/3758640304 conn(0x55dd97373e60 msgr2=0x55dd9737ab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:13.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.799+0000 7faa75fc5740 1 -- 172.21.15.83:0/3758640304 shutdown_connections 2024-04-24T09:16:13.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.799+0000 7faa75fc5740 1 -- 172.21.15.83:0/3758640304 wait complete. 2024-04-24T09:16:13.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.800+0000 7faa75fc5740 1 -- start start 2024-04-24T09:16:13.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.800+0000 7faa75fc5740 1 Processor -- start 2024-04-24T09:16:13.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.801+0000 7faa75fc5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd97377b40 0x55dd97393970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:13.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.801+0000 7faa75fc5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd97378460 0x55dd97393eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:13.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.801+0000 7faa75fc5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dd97382b30 0x55dd973943f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:13.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.801+0000 7faa75fc5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7faa6c1b7660 con 0x55dd97378460 2024-04-24T09:16:13.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.801+0000 7faa75fc5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7faa6c1b6f90 con 0x55dd97382b30 2024-04-24T09:16:13.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.801+0000 7faa75fc5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7faa6c1b4130 con 0x55dd97377b40 2024-04-24T09:16:13.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.801+0000 7faa75fc5740 10 client.0 _ll_register_callbacks cb 0x7faa640b4070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:16:13.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.802+0000 7faa6b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd97378460 0x55dd97393eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:13.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.802+0000 7faa6be00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd97377b40 0x55dd97393970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:13.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.802+0000 7faa6b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd97378460 0x55dd97393eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51860/0 (socket says 172.21.15.83:51860) 2024-04-24T09:16:13.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.802+0000 7faa6b400640 1 -- 172.21.15.83:0/3548376722 learned_addr learned my addr 172.21.15.83:0/3548376722 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:16:13.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.802+0000 7faa71000640 1 --2- 172.21.15.83:0/3548376722 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dd97382b30 0x55dd973943f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:13.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.802+0000 7faa71000640 1 -- 172.21.15.83:0/3548376722 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd97377b40 msgr2=0x55dd97393970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:13.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.802+0000 7faa71000640 1 --2- 172.21.15.83:0/3548376722 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd97377b40 0x55dd97393970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:13.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.802+0000 7faa71000640 1 -- 172.21.15.83:0/3548376722 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd97378460 msgr2=0x55dd97393eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:13.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.802+0000 7faa71000640 1 --2- 172.21.15.83:0/3548376722 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd97378460 0x55dd97393eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:13.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.802+0000 7faa71000640 1 -- 172.21.15.83:0/3548376722 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd973f7dc0 con 0x55dd97382b30 2024-04-24T09:16:13.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.803+0000 7faa71000640 1 --2- 172.21.15.83:0/3548376722 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dd97382b30 0x55dd973943f0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7faa640bd4a0 tx=0x7faa64099600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=111792a280d9dda8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:13.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.803+0000 7faa56000640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:16:13.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.803+0000 7faa56000640 1 -- 172.21.15.83:0/3548376722 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa640ba070 con 0x55dd97382b30 2024-04-24T09:16:13.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.803+0000 7faa56000640 1 -- 172.21.15.83:0/3548376722 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(131..131 src has 1..131) ==== 9671+0+0 (secure 0 0 0) 0x7faa64099400 con 0x55dd97382b30 2024-04-24T09:16:13.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.803+0000 7faa75fc5740 1 -- 172.21.15.83:0/3548376722 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa5c012db0 con 0x55dd97382b30 2024-04-24T09:16:13.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.804+0000 7faa56000640 1 -- 172.21.15.83:0/3548376722 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7faa6409cb60 con 0x55dd97382b30 2024-04-24T09:16:13.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.804+0000 7faa75fc5740 10 client.40919 fetch_fsmap learned FSMap version 69 2024-04-24T09:16:13.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.804+0000 7faa75fc5740 1 -- 172.21.15.83:0/3548376722 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x55dd973f8fa0 con 0x55dd97382b30 2024-04-24T09:16:13.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.804+0000 7faa56000640 1 -- 172.21.15.83:0/3548376722 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faa5c012db0 con 0x55dd97382b30 2024-04-24T09:16:13.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.805+0000 7faa56000640 1 -- 172.21.15.83:0/3548376722 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 69) ==== 8370+0+0 (secure 0 0 0) 0x7faa640a5570 con 0x55dd97382b30 2024-04-24T09:16:13.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.805+0000 7faa75fc5740 10 client.40919 fetch_fsmap finished waiting for FSMap version 69 2024-04-24T09:16:13.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.805+0000 7faa75fc5740 10 client.40919 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-24T09:16:13.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.805+0000 7faa75fc5740 20 client.40919 populate_metadata read hostname 'smithi083' 2024-04-24T09:16:13.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.805+0000 7faa75fc5740 1 --2- 172.21.15.83:0/3548376722 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55dd973f94d0 0x55dd973fb8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:13.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.805+0000 7faa75fc5740 4 client.40919 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:16:13.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.805+0000 7faa75fc5740 1 -- 172.21.15.83:0/3548376722 --> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dd972e4a60 con 0x55dd973f94d0 2024-04-24T09:16:13.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.806+0000 7faa6be00640 1 --2- 172.21.15.83:0/3548376722 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55dd973f94d0 0x55dd973fb8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:13.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.807+0000 7faa6be00640 1 --2- 172.21.15.83:0/3548376722 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55dd973f94d0 0x55dd973fb8c0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8aee1daf02077ba3 server_cookie=b98c97329405579d in_seq=0 out_seq=0 2024-04-24T09:16:13.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.807+0000 7faa56000640 10 client.40919 ms_handle_connect on v2:172.21.15.189:6838/2548965336 2024-04-24T09:16:13.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.810+0000 7faa56000640 1 -- 172.21.15.83:0/3548376722 <== mds.4 v2:172.21.15.189:6838/2548965336 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dd972e4a60 con 0x55dd973f94d0 2024-04-24T09:16:13.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.810+0000 7faa56000640 10 client.40919 handle_command_reply: tid=0 2024-04-24T09:16:13.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.810+0000 7faa75fc5740 1 client.40919 shutdown 2024-04-24T09:16:13.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.811+0000 7faa75fc5740 1 -- 172.21.15.83:0/3548376722 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dd97382b30 msgr2=0x55dd973943f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:13.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.811+0000 7faa75fc5740 1 --2- 172.21.15.83:0/3548376722 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dd97382b30 0x55dd973943f0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7faa640bd4a0 tx=0x7faa64099600 comp rx=0 tx=0).stop 2024-04-24T09:16:13.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.811+0000 7faa75fc5740 1 -- 172.21.15.83:0/3548376722 shutdown_connections 2024-04-24T09:16:13.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.811+0000 7faa75fc5740 1 --2- 172.21.15.83:0/3548376722 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55dd973f94d0 0x55dd973fb8c0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:13.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.811+0000 7faa75fc5740 1 --2- 172.21.15.83:0/3548376722 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dd97382b30 0x55dd973943f0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:13.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.811+0000 7faa75fc5740 1 --2- 172.21.15.83:0/3548376722 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd97378460 0x55dd97393eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:13.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.811+0000 7faa75fc5740 1 --2- 172.21.15.83:0/3548376722 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd97377b40 0x55dd97393970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:13.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.811+0000 7faa75fc5740 1 -- 172.21.15.83:0/3548376722 >> 172.21.15.83:0/3548376722 conn(0x55dd97373e60 msgr2=0x55dd9738dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:13.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.811+0000 7faa56000640 0 client.40919 ms_handle_reset on v2:172.21.15.189:6838/2548965336 2024-04-24T09:16:13.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.812+0000 7faa75fc5740 1 -- 172.21.15.83:0/3548376722 shutdown_connections 2024-04-24T09:16:13.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.812+0000 7faa75fc5740 1 -- 172.21.15.83:0/3548376722 wait complete. 2024-04-24T09:16:13.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.812+0000 7faa75fc5740 20 client.40919 trim_cache size 0 max 16384 2024-04-24T09:16:13.814 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:16:13.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.813+0000 7faa71a00640 1 -- 172.21.15.83:0/2269346814 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7faa500a3de0 msgr2=0x7faa500a62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:13.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.813+0000 7faa71a00640 1 --2- 172.21.15.83:0/2269346814 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7faa500a3de0 0x7faa500a62a0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7faa5c015420 tx=0x7faa5c015820 comp rx=0 tx=0).stop 2024-04-24T09:16:13.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.813+0000 7faa71a00640 1 -- 172.21.15.83:0/2269346814 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa6c1a36d0 msgr2=0x7faa6c1a3b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:13.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.813+0000 7faa71a00640 1 --2- 172.21.15.83:0/2269346814 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa6c1a36d0 0x7faa6c1a3b80 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7faa6400ea60 tx=0x7faa6400ef20 comp rx=0 tx=0).stop 2024-04-24T09:16:13.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.814+0000 7faa71a00640 1 -- 172.21.15.83:0/2269346814 shutdown_connections 2024-04-24T09:16:13.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.814+0000 7faa71a00640 1 --2- 172.21.15.83:0/2269346814 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7faa500a3de0 0x7faa500a62a0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:13.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.814+0000 7faa71a00640 1 --2- 172.21.15.83:0/2269346814 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa6c1a36d0 0x7faa6c1a3b80 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:13.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.814+0000 7faa71a00640 1 --2- 172.21.15.83:0/2269346814 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faa6c10e5a0 0x7faa6c19f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:13.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.814+0000 7faa71a00640 1 --2- 172.21.15.83:0/2269346814 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa6c109800 0x7faa6c19ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:13.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.814+0000 7faa71a00640 1 -- 172.21.15.83:0/2269346814 >> 172.21.15.83:0/2269346814 conn(0x7faa6c077a20 msgr2=0x7faa6c07a490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:13.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.814+0000 7faa71a00640 1 -- 172.21.15.83:0/2269346814 shutdown_connections 2024-04-24T09:16:13.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:13.814+0000 7faa71a00640 1 -- 172.21.15.83:0/2269346814 wait complete. 2024-04-24T09:16:13.839 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-24T09:16:14.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.096+0000 7fb018200640 1 -- 172.21.15.83:0/3452967138 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb010076e50 msgr2=0x7fb010112940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:14.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.096+0000 7fb018200640 1 --2- 172.21.15.83:0/3452967138 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb010076e50 0x7fb010112940 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fb008009930 tx=0x7fb00802a420 comp rx=0 tx=0).stop 2024-04-24T09:16:14.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.096+0000 7fb018200640 1 -- 172.21.15.83:0/3452967138 shutdown_connections 2024-04-24T09:16:14.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.096+0000 7fb018200640 1 --2- 172.21.15.83:0/3452967138 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb010112e80 0x7fb010115270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:14.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.096+0000 7fb018200640 1 --2- 172.21.15.83:0/3452967138 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb010076e50 0x7fb010112940 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:14.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.096+0000 7fb018200640 1 --2- 172.21.15.83:0/3452967138 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb010076530 0x7fb010076910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:14.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.096+0000 7fb018200640 1 -- 172.21.15.83:0/3452967138 >> 172.21.15.83:0/3452967138 conn(0x7fb0100ffc70 msgr2=0x7fb010102090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:14.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.096+0000 7fb018200640 1 -- 172.21.15.83:0/3452967138 shutdown_connections 2024-04-24T09:16:14.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.096+0000 7fb018200640 1 -- 172.21.15.83:0/3452967138 wait complete. 2024-04-24T09:16:14.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.098+0000 7fb018200640 1 Processor -- start 2024-04-24T09:16:14.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.098+0000 7fb018200640 1 -- start start 2024-04-24T09:16:14.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.098+0000 7fb018200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb010076530 0x7fb0101125b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:14.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.098+0000 7fb018200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb010076e50 0x7fb01010b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:14.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.098+0000 7fb018200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb010112e80 0x7fb01010bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:14.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.098+0000 7fb018200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb0101188e0 con 0x7fb010112e80 2024-04-24T09:16:14.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.098+0000 7fb018200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb010118760 con 0x7fb010076530 2024-04-24T09:16:14.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.098+0000 7fb018200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb010118a60 con 0x7fb010076e50 2024-04-24T09:16:14.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.099+0000 7fb016200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb010112e80 0x7fb01010bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:14.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.099+0000 7fb015800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb010076530 0x7fb0101125b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:14.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.099+0000 7fb016200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb010112e80 0x7fb01010bb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51876/0 (socket says 172.21.15.83:51876) 2024-04-24T09:16:14.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.099+0000 7fb016200640 1 -- 172.21.15.83:0/3845415338 learned_addr learned my addr 172.21.15.83:0/3845415338 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:16:14.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.099+0000 7fb014e00640 1 --2- 172.21.15.83:0/3845415338 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb010076e50 0x7fb01010b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:14.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.100+0000 7fb015800640 1 -- 172.21.15.83:0/3845415338 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb010076e50 msgr2=0x7fb01010b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:14.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.100+0000 7fb015800640 1 --2- 172.21.15.83:0/3845415338 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb010076e50 0x7fb01010b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:14.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.100+0000 7fb015800640 1 -- 172.21.15.83:0/3845415338 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb010112e80 msgr2=0x7fb01010bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:14.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.100+0000 7fb015800640 1 --2- 172.21.15.83:0/3845415338 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb010112e80 0x7fb01010bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:14.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.100+0000 7fb015800640 1 -- 172.21.15.83:0/3845415338 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb01010c2a0 con 0x7fb010076530 2024-04-24T09:16:14.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.101+0000 7fb015800640 1 --2- 172.21.15.83:0/3845415338 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb010076530 0x7fb0101125b0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7faffc00d6d0 tx=0x7faffc00db90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:14.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.101+0000 7fb00e000640 1 -- 172.21.15.83:0/3845415338 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faffc017070 con 0x7fb010076530 2024-04-24T09:16:14.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.101+0000 7fb018200640 1 -- 172.21.15.83:0/3845415338 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0101100d0 con 0x7fb010076530 2024-04-24T09:16:14.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.101+0000 7fb018200640 1 -- 172.21.15.83:0/3845415338 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb010110640 con 0x7fb010076530 2024-04-24T09:16:14.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.102+0000 7fb00e000640 1 -- 172.21.15.83:0/3845415338 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faffc009820 con 0x7fb010076530 2024-04-24T09:16:14.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.102+0000 7fb00e000640 1 -- 172.21.15.83:0/3845415338 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faffc004c40 con 0x7fb010076530 2024-04-24T09:16:14.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.106+0000 7fb01b013740 1 Processor -- start 2024-04-24T09:16:14.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.106+0000 7fb01b013740 1 -- start start 2024-04-24T09:16:14.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.107+0000 7fb01b013740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608154d7950 0x5608154d7d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:14.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.107+0000 7fb01b013740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5608154d8270 0x5608154e2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:14.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.107+0000 7fb01b013740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5608154e2940 0x5608154e4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:14.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.107+0000 7fb01b013740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56081543d1d0 con 0x5608154d7950 2024-04-24T09:16:14.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.107+0000 7fb01b013740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5608153e3450 con 0x5608154d8270 2024-04-24T09:16:14.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.107+0000 7fb01b013740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x560815439190 con 0x5608154e2940 2024-04-24T09:16:14.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.107+0000 7fb014e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5608154e2940 0x5608154e4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:14.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.107+0000 7fb014e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5608154e2940 0x5608154e4d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:35406/0 (socket says 172.21.15.83:35406) 2024-04-24T09:16:14.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.107+0000 7fb014e00640 1 -- 172.21.15.83:0/1642937736 learned_addr learned my addr 172.21.15.83:0/1642937736 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:16:14.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.107+0000 7fb016200640 1 --2- 172.21.15.83:0/1642937736 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608154d7950 0x5608154d7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:14.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.108+0000 7fb00e000640 1 -- 172.21.15.83:0/3845415338 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7faffc010460 con 0x7fb010076530 2024-04-24T09:16:14.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.108+0000 7fb014e00640 1 -- 172.21.15.83:0/1642937736 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5608154d8270 msgr2=0x5608154e2400 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-24T09:16:14.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.108+0000 7fb014e00640 1 --2- 172.21.15.83:0/1642937736 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5608154d8270 0x5608154e2400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:14.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.108+0000 7fb014e00640 1 -- 172.21.15.83:0/1642937736 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608154d7950 msgr2=0x5608154d7d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:14.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.108+0000 7fb014e00640 1 --2- 172.21.15.83:0/1642937736 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608154d7950 0x5608154d7d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:14.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.108+0000 7fb014e00640 1 -- 172.21.15.83:0/1642937736 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608154a0310 con 0x5608154e2940 2024-04-24T09:16:14.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.108+0000 7fb014e00640 1 --2- 172.21.15.83:0/1642937736 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5608154e2940 0x5608154e4d30 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb00802aa10 tx=0x7fb008004310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c46925a894b5ec9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:14.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.109+0000 7fb006a00640 1 -- 172.21.15.83:0/1642937736 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb008035040 con 0x5608154e2940 2024-04-24T09:16:14.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.109+0000 7fb00e000640 1 --2- 172.21.15.83:0/3845415338 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7faff40a3da0 0x7faff40a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:14.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.109+0000 7fb006a00640 1 -- 172.21.15.83:0/1642937736 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb008004750 con 0x5608154e2940 2024-04-24T09:16:14.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.110+0000 7fb006a00640 1 -- 172.21.15.83:0/1642937736 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb008003850 con 0x5608154e2940 2024-04-24T09:16:14.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.110+0000 7fb00e000640 1 -- 172.21.15.83:0/3845415338 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 9671+0+0 (secure 0 0 0) 0x7faffc0d0350 con 0x7fb010076530 2024-04-24T09:16:14.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.110+0000 7fb016200640 1 --2- 172.21.15.83:0/3845415338 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7faff40a3da0 0x7faff40a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:14.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.110+0000 7fb01b013740 1 -- 172.21.15.83:0/1642937736 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5608154e2940 msgr2=0x5608154e4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:14.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.110+0000 7fb01b013740 1 --2- 172.21.15.83:0/1642937736 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5608154e2940 0x5608154e4d30 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb00802aa10 tx=0x7fb008004310 comp rx=0 tx=0).stop 2024-04-24T09:16:14.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.111+0000 7fb01b013740 1 -- 172.21.15.83:0/1642937736 shutdown_connections 2024-04-24T09:16:14.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.111+0000 7fb01b013740 1 --2- 172.21.15.83:0/1642937736 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5608154e2940 0x5608154e4d30 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:14.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.111+0000 7fb01b013740 1 --2- 172.21.15.83:0/1642937736 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5608154d8270 0x5608154e2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:14.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.111+0000 7fb01b013740 1 --2- 172.21.15.83:0/1642937736 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608154d7950 0x5608154d7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:14.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.111+0000 7fb01b013740 1 -- 172.21.15.83:0/1642937736 >> 172.21.15.83:0/1642937736 conn(0x5608154d3c70 msgr2=0x5608154d4050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:14.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.111+0000 7fb016200640 1 --2- 172.21.15.83:0/3845415338 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7faff40a3da0 0x7faff40a6260 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fb000007f00 tx=0x7fb000007540 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:14.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.111+0000 7fb01b013740 1 -- 172.21.15.83:0/1642937736 shutdown_connections 2024-04-24T09:16:14.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.111+0000 7fb01b013740 1 -- 172.21.15.83:0/1642937736 wait complete. 2024-04-24T09:16:14.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.112+0000 7fb01b013740 1 -- start start 2024-04-24T09:16:14.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.112+0000 7fb01b013740 1 Processor -- start 2024-04-24T09:16:14.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.113+0000 7fb01b013740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5608154d7950 0x5608154e22d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:14.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.113+0000 7fb01b013740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5608154d8270 0x5608154df5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:14.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.113+0000 7fb01b013740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608154e2940 0x5608154dfb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:14.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.113+0000 7fb01b013740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5608154e57c0 con 0x5608154e2940 2024-04-24T09:16:14.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.113+0000 7fb01b013740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5608153f6810 con 0x5608154d8270 2024-04-24T09:16:14.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.113+0000 7fb01b013740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5608154e5940 con 0x5608154d7950 2024-04-24T09:16:14.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.113+0000 7fb01b013740 10 client.0 _ll_register_callbacks cb 0x5608153c4140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:16:14.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.114+0000 7fb016200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608154e2940 0x5608154dfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:14.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.114+0000 7fb015800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5608154d7950 0x5608154e22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:14.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.114+0000 7fb015800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5608154d7950 0x5608154e22d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:35418/0 (socket says 172.21.15.83:35418) 2024-04-24T09:16:14.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.114+0000 7fb015800640 1 -- 172.21.15.83:0/3866770266 learned_addr learned my addr 172.21.15.83:0/3866770266 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:16:14.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.114+0000 7fb014e00640 1 --2- 172.21.15.83:0/3866770266 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5608154d8270 0x5608154df5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:14.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.114+0000 7fb015800640 1 -- 172.21.15.83:0/3866770266 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5608154d8270 msgr2=0x5608154df5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:14.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.114+0000 7fb015800640 1 --2- 172.21.15.83:0/3866770266 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5608154d8270 0x5608154df5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:14.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.114+0000 7fb015800640 1 -- 172.21.15.83:0/3866770266 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608154e2940 msgr2=0x5608154dfb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:14.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.114+0000 7fb015800640 1 --2- 172.21.15.83:0/3866770266 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608154e2940 0x5608154dfb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:14.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.114+0000 7fb015800640 1 -- 172.21.15.83:0/3866770266 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608154073f0 con 0x5608154d7950 2024-04-24T09:16:14.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.114+0000 7fb015800640 1 --2- 172.21.15.83:0/3866770266 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5608154d7950 0x5608154e22d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7faffc0048b0 tx=0x7faffc0a0220 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc6ceb2233d232f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:14.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.115+0000 7faffb400640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-04-24T09:16:14.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.115+0000 7faffb400640 1 -- 172.21.15.83:0/3866770266 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faffc020040 con 0x5608154d7950 2024-04-24T09:16:14.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.115+0000 7faffb400640 1 -- 172.21.15.83:0/3866770266 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(131..131 src has 1..131) ==== 9671+0+0 (secure 0 0 0) 0x7faffc098590 con 0x5608154d7950 2024-04-24T09:16:14.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.115+0000 7fb01b013740 1 -- 172.21.15.83:0/3866770266 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb008004750 con 0x5608154d7950 2024-04-24T09:16:14.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.115+0000 7faffb400640 1 -- 172.21.15.83:0/3866770266 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7faffc0b5200 con 0x5608154d7950 2024-04-24T09:16:14.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.115+0000 7fb01b013740 10 client.30628 fetch_fsmap learned FSMap version 69 2024-04-24T09:16:14.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.115+0000 7fb01b013740 1 -- 172.21.15.83:0/3866770266 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x56081514f3f0 con 0x5608154d7950 2024-04-24T09:16:14.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.116+0000 7faffb400640 1 -- 172.21.15.83:0/3866770266 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb008004750 con 0x5608154d7950 2024-04-24T09:16:14.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.116+0000 7faffb400640 1 -- 172.21.15.83:0/3866770266 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 69) ==== 8370+0+0 (secure 0 0 0) 0x7faffc09ade0 con 0x5608154d7950 2024-04-24T09:16:14.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.116+0000 7fb01b013740 10 client.30628 fetch_fsmap finished waiting for FSMap version 69 2024-04-24T09:16:14.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.116+0000 7fb01b013740 10 client.30628 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-24T09:16:14.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.116+0000 7fb01b013740 20 client.30628 populate_metadata read hostname 'smithi083' 2024-04-24T09:16:14.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.116+0000 7fb01b013740 1 --2- 172.21.15.83:0/3866770266 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x5608154f2ac0 0x5608154f4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:14.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.116+0000 7fb01b013740 4 client.30628 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:16:14.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.116+0000 7fb01b013740 1 -- 172.21.15.83:0/3866770266 --> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56081544abf0 con 0x5608154f2ac0 2024-04-24T09:16:14.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.117+0000 7fb014e00640 1 --2- 172.21.15.83:0/3866770266 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x5608154f2ac0 0x5608154f4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:14.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.118+0000 7faffb400640 10 client.30628 ms_handle_connect on v2:172.21.15.138:6832/1157967658 2024-04-24T09:16:14.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.118+0000 7fb014e00640 1 --2- 172.21.15.83:0/3866770266 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x5608154f2ac0 0x5608154f4eb0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2d1becd3d443a2da server_cookie=65b5ad8ade53ba1b in_seq=0 out_seq=0 2024-04-24T09:16:14.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.120+0000 7faffb400640 1 -- 172.21.15.83:0/3866770266 <== mds.3 v2:172.21.15.138:6832/1157967658 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56081544abf0 con 0x5608154f2ac0 2024-04-24T09:16:14.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.120+0000 7faffb400640 10 client.30628 handle_command_reply: tid=0 2024-04-24T09:16:14.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.120+0000 7fb01b013740 1 client.30628 shutdown 2024-04-24T09:16:14.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.121+0000 7fb01b013740 1 -- 172.21.15.83:0/3866770266 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5608154d7950 msgr2=0x5608154e22d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:14.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.121+0000 7fb01b013740 1 --2- 172.21.15.83:0/3866770266 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5608154d7950 0x5608154e22d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7faffc0048b0 tx=0x7faffc0a0220 comp rx=0 tx=0).stop 2024-04-24T09:16:14.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.121+0000 7fb01b013740 1 -- 172.21.15.83:0/3866770266 shutdown_connections 2024-04-24T09:16:14.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.121+0000 7fb01b013740 1 --2- 172.21.15.83:0/3866770266 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608154e2940 0x5608154dfb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:14.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.121+0000 7fb01b013740 1 --2- 172.21.15.83:0/3866770266 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5608154d8270 0x5608154df5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:14.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.121+0000 7fb01b013740 1 --2- 172.21.15.83:0/3866770266 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x5608154f2ac0 0x5608154f4eb0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:14.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.121+0000 7fb01b013740 1 --2- 172.21.15.83:0/3866770266 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5608154d7950 0x5608154e22d0 secure :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7faffc0048b0 tx=0x7faffc0a0220 comp rx=0 tx=0).stop 2024-04-24T09:16:14.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.121+0000 7fb01b013740 1 -- 172.21.15.83:0/3866770266 >> 172.21.15.83:0/3866770266 conn(0x5608154d3c70 msgr2=0x5608154d2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:14.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.121+0000 7faffb400640 0 client.30628 ms_handle_reset on v2:172.21.15.138:6832/1157967658 2024-04-24T09:16:14.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.121+0000 7fb01b013740 1 -- 172.21.15.83:0/3866770266 shutdown_connections 2024-04-24T09:16:14.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.121+0000 7fb01b013740 1 -- 172.21.15.83:0/3866770266 wait complete. 2024-04-24T09:16:14.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.121+0000 7fb01b013740 20 client.30628 trim_cache size 0 max 16384 2024-04-24T09:16:14.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.147+0000 7fb01b013740 1 Processor -- start 2024-04-24T09:16:14.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.147+0000 7fb01b013740 1 -- start start 2024-04-24T09:16:14.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.148+0000 7fb01b013740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5608154d8270 0x5608154ee760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:14.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.148+0000 7fb01b013740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608154e2940 0x5608154de520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:14.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.148+0000 7fb01b013740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5608154df710 0x5608154e1e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:14.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.148+0000 7fb01b013740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7faff0010260 con 0x5608154e2940 2024-04-24T09:16:14.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.148+0000 7fb01b013740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7faff0011e00 con 0x5608154df710 2024-04-24T09:16:14.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.148+0000 7fb01b013740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7faff0010930 con 0x5608154d8270 2024-04-24T09:16:14.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.148+0000 7fb014e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608154e2940 0x5608154de520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:14.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.148+0000 7fb016200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5608154df710 0x5608154e1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:14.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.148+0000 7fb014e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608154e2940 0x5608154de520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51916/0 (socket says 172.21.15.83:51916) 2024-04-24T09:16:14.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.148+0000 7fb014e00640 1 -- 172.21.15.83:0/2562527351 learned_addr learned my addr 172.21.15.83:0/2562527351 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:16:14.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.148+0000 7fb015800640 1 --2- 172.21.15.83:0/2562527351 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5608154d8270 0x5608154ee760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:14.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.149+0000 7fb016200640 1 -- 172.21.15.83:0/2562527351 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5608154d8270 msgr2=0x5608154ee760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:14.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.149+0000 7fb016200640 1 --2- 172.21.15.83:0/2562527351 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5608154d8270 0x5608154ee760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:14.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.149+0000 7fb016200640 1 -- 172.21.15.83:0/2562527351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608154e2940 msgr2=0x5608154de520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:14.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.149+0000 7fb016200640 1 --2- 172.21.15.83:0/2562527351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608154e2940 0x5608154de520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:14.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.149+0000 7fb016200640 1 -- 172.21.15.83:0/2562527351 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608152197d0 con 0x5608154df710 2024-04-24T09:16:14.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.149+0000 7fb014e00640 1 --2- 172.21.15.83:0/2562527351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608154e2940 0x5608154de520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:16:14.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.150+0000 7fb016200640 1 --2- 172.21.15.83:0/2562527351 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5608154df710 0x5608154e1e00 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fb000007fd0 tx=0x7fb000018000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea407a39b532838a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:14.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.150+0000 7fb006000640 1 -- 172.21.15.83:0/2562527351 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb00002e070 con 0x5608154df710 2024-04-24T09:16:14.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.150+0000 7fb006000640 1 -- 172.21.15.83:0/2562527351 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb00001cd90 con 0x5608154df710 2024-04-24T09:16:14.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.150+0000 7fb006000640 1 -- 172.21.15.83:0/2562527351 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb00000f680 con 0x5608154df710 2024-04-24T09:16:14.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.151+0000 7fb01b013740 1 -- 172.21.15.83:0/2562527351 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5608154df710 msgr2=0x5608154e1e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:14.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.151+0000 7fb01b013740 1 --2- 172.21.15.83:0/2562527351 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5608154df710 0x5608154e1e00 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fb000007fd0 tx=0x7fb000018000 comp rx=0 tx=0).stop 2024-04-24T09:16:14.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.152+0000 7fb01b013740 1 -- 172.21.15.83:0/2562527351 shutdown_connections 2024-04-24T09:16:14.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.152+0000 7fb01b013740 1 --2- 172.21.15.83:0/2562527351 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5608154df710 0x5608154e1e00 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:14.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.152+0000 7fb01b013740 1 --2- 172.21.15.83:0/2562527351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608154e2940 0x5608154de520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:14.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.152+0000 7fb01b013740 1 --2- 172.21.15.83:0/2562527351 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5608154d8270 0x5608154ee760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:14.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.152+0000 7fb01b013740 1 -- 172.21.15.83:0/2562527351 >> 172.21.15.83:0/2562527351 conn(0x5608154d3c70 msgr2=0x5608154d90a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:14.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.152+0000 7fb01b013740 1 -- 172.21.15.83:0/2562527351 shutdown_connections 2024-04-24T09:16:14.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.152+0000 7fb01b013740 1 -- 172.21.15.83:0/2562527351 wait complete. 2024-04-24T09:16:14.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.152+0000 7fb01b013740 1 -- start start 2024-04-24T09:16:14.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.152+0000 7fb01b013740 1 Processor -- start 2024-04-24T09:16:14.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.154+0000 7fb01b013740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5608154d8270 0x5608154eac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:14.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.154+0000 7fb01b013740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608154e2940 0x5608154eb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:14.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.154+0000 7fb01b013740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5608154dd430 0x5608154e3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:14.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.154+0000 7fb01b013740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7faff0011730 con 0x5608154e2940 2024-04-24T09:16:14.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.154+0000 7fb01b013740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7faff0011060 con 0x5608154d8270 2024-04-24T09:16:14.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.154+0000 7fb01b013740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7faff000e260 con 0x5608154dd430 2024-04-24T09:16:14.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.154+0000 7fb01b013740 10 client.0 _ll_register_callbacks cb 0x7faffc0981c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:16:14.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.154+0000 7fb014e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608154e2940 0x5608154eb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:14.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.155+0000 7fb014e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608154e2940 0x5608154eb1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51918/0 (socket says 172.21.15.83:51918) 2024-04-24T09:16:14.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.155+0000 7fb014e00640 1 -- 172.21.15.83:0/230780397 learned_addr learned my addr 172.21.15.83:0/230780397 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:16:14.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.155+0000 7fb015800640 1 --2- 172.21.15.83:0/230780397 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5608154d8270 0x5608154eac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:14.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.155+0000 7fb016200640 1 --2- 172.21.15.83:0/230780397 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5608154dd430 0x5608154e3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:14.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.155+0000 7fb014e00640 1 -- 172.21.15.83:0/230780397 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5608154dd430 msgr2=0x5608154e3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:14.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.155+0000 7fb014e00640 1 --2- 172.21.15.83:0/230780397 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5608154dd430 0x5608154e3a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:14.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.155+0000 7fb014e00640 1 -- 172.21.15.83:0/230780397 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5608154d8270 msgr2=0x5608154eac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:14.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.155+0000 7fb014e00640 1 --2- 172.21.15.83:0/230780397 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5608154d8270 0x5608154eac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:14.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.156+0000 7fb014e00640 1 -- 172.21.15.83:0/230780397 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608154e4240 con 0x5608154e2940 2024-04-24T09:16:14.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.156+0000 7fb016200640 1 --2- 172.21.15.83:0/230780397 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5608154dd430 0x5608154e3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:16:14.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.156+0000 7fb015800640 1 --2- 172.21.15.83:0/230780397 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5608154d8270 0x5608154eac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:16:14.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.156+0000 7fb014e00640 1 --2- 172.21.15.83:0/230780397 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608154e2940 0x5608154eb1b0 secure :-1 s=READY pgs=2763 cs=0 l=1 rev1=1 crypto rx=0x5608154d4e00 tx=0x7fb008005e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=299d8d88a6f870f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:14.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.156+0000 7faffb400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:16:14.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.157+0000 7faffb400640 1 -- 172.21.15.83:0/230780397 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb008035040 con 0x5608154e2940 2024-04-24T09:16:14.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.157+0000 7faffb400640 1 -- 172.21.15.83:0/230780397 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(131..131 src has 1..131) ==== 9671+0+0 (secure 0 0 0) 0x7fb008054280 con 0x5608154e2940 2024-04-24T09:16:14.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.157+0000 7fb01b013740 1 -- 172.21.15.83:0/230780397 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb00001cd90 con 0x5608154e2940 2024-04-24T09:16:14.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.157+0000 7faffb400640 1 -- 172.21.15.83:0/230780397 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7fb008054480 con 0x5608154e2940 2024-04-24T09:16:14.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.157+0000 7fb01b013740 10 client.23865 fetch_fsmap learned FSMap version 69 2024-04-24T09:16:14.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.157+0000 7fb01b013740 1 -- 172.21.15.83:0/230780397 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x56081556ab90 con 0x5608154e2940 2024-04-24T09:16:14.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.158+0000 7faffb400640 1 -- 172.21.15.83:0/230780397 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb00001cd90 con 0x5608154e2940 2024-04-24T09:16:14.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.158+0000 7faffb400640 1 -- 172.21.15.83:0/230780397 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 69) ==== 8370+0+0 (secure 0 0 0) 0x7fb008031c40 con 0x5608154e2940 2024-04-24T09:16:14.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.158+0000 7fb01b013740 10 client.23865 fetch_fsmap finished waiting for FSMap version 69 2024-04-24T09:16:14.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.158+0000 7fb01b013740 10 client.23865 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-24T09:16:14.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.158+0000 7fb01b013740 20 client.23865 populate_metadata read hostname 'smithi083' 2024-04-24T09:16:14.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.158+0000 7fb01b013740 1 --2- 172.21.15.83:0/230780397 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x56081556aed0 0x56081556d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:14.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.158+0000 7fb01b013740 4 client.23865 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:16:14.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.158+0000 7fb01b013740 1 -- 172.21.15.83:0/230780397 --> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560815444a60 con 0x56081556aed0 2024-04-24T09:16:14.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.159+0000 7fb015800640 1 --2- 172.21.15.83:0/230780397 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x56081556aed0 0x56081556d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:14.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.161+0000 7fb015800640 1 --2- 172.21.15.83:0/230780397 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x56081556aed0 0x56081556d2c0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=476a50da8c1c7e83 server_cookie=84400dc1819051a2 in_seq=0 out_seq=0 2024-04-24T09:16:14.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.161+0000 7faffb400640 10 client.23865 ms_handle_connect on v2:172.21.15.138:6832/1157967658 2024-04-24T09:16:14.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.163+0000 7faffb400640 1 -- 172.21.15.83:0/230780397 <== mds.3 v2:172.21.15.138:6832/1157967658 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560815444a60 con 0x56081556aed0 2024-04-24T09:16:14.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.163+0000 7faffb400640 10 client.23865 handle_command_reply: tid=0 2024-04-24T09:16:14.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.163+0000 7fb01b013740 1 client.23865 shutdown 2024-04-24T09:16:14.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.163+0000 7fb01b013740 1 -- 172.21.15.83:0/230780397 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608154e2940 msgr2=0x5608154eb1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:14.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.163+0000 7fb01b013740 1 --2- 172.21.15.83:0/230780397 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608154e2940 0x5608154eb1b0 secure :-1 s=READY pgs=2763 cs=0 l=1 rev1=1 crypto rx=0x5608154d4e00 tx=0x7fb008005e30 comp rx=0 tx=0).stop 2024-04-24T09:16:14.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.164+0000 7fb01b013740 1 -- 172.21.15.83:0/230780397 shutdown_connections 2024-04-24T09:16:14.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.164+0000 7fb01b013740 1 --2- 172.21.15.83:0/230780397 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x56081556aed0 0x56081556d2c0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:14.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.164+0000 7fb01b013740 1 --2- 172.21.15.83:0/230780397 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5608154dd430 0x5608154e3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:14.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.164+0000 7fb01b013740 1 --2- 172.21.15.83:0/230780397 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608154e2940 0x5608154eb1b0 unknown :-1 s=CLOSED pgs=2763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:14.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.164+0000 7fb01b013740 1 --2- 172.21.15.83:0/230780397 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5608154d8270 0x5608154eac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:14.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.164+0000 7fb01b013740 1 -- 172.21.15.83:0/230780397 >> 172.21.15.83:0/230780397 conn(0x5608154d3c70 msgr2=0x5608154d9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:14.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.164+0000 7faffb400640 0 client.23865 ms_handle_reset on v2:172.21.15.138:6832/1157967658 2024-04-24T09:16:14.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.164+0000 7fb01b013740 1 -- 172.21.15.83:0/230780397 shutdown_connections 2024-04-24T09:16:14.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.164+0000 7fb01b013740 1 -- 172.21.15.83:0/230780397 wait complete. 2024-04-24T09:16:14.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.165+0000 7fb01b013740 20 client.23865 trim_cache size 0 max 16384 2024-04-24T09:16:14.166 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:16:14.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.166+0000 7fb018200640 1 -- 172.21.15.83:0/3845415338 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7faff40a3da0 msgr2=0x7faff40a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:14.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.166+0000 7fb018200640 1 --2- 172.21.15.83:0/3845415338 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7faff40a3da0 0x7faff40a6260 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fb000007f00 tx=0x7fb000007540 comp rx=0 tx=0).stop 2024-04-24T09:16:14.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.166+0000 7fb018200640 1 -- 172.21.15.83:0/3845415338 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb010076530 msgr2=0x7fb0101125b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:14.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.166+0000 7fb018200640 1 --2- 172.21.15.83:0/3845415338 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb010076530 0x7fb0101125b0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7faffc00d6d0 tx=0x7faffc00db90 comp rx=0 tx=0).stop 2024-04-24T09:16:14.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.167+0000 7fb018200640 1 -- 172.21.15.83:0/3845415338 shutdown_connections 2024-04-24T09:16:14.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.167+0000 7fb018200640 1 --2- 172.21.15.83:0/3845415338 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7faff40a3da0 0x7faff40a6260 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:14.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.167+0000 7fb018200640 1 --2- 172.21.15.83:0/3845415338 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb010112e80 0x7fb01010bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:14.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.167+0000 7fb018200640 1 --2- 172.21.15.83:0/3845415338 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb010076e50 0x7fb01010b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:14.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.167+0000 7fb018200640 1 --2- 172.21.15.83:0/3845415338 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb010076530 0x7fb0101125b0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:14.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.167+0000 7fb018200640 1 -- 172.21.15.83:0/3845415338 >> 172.21.15.83:0/3845415338 conn(0x7fb0100ffc70 msgr2=0x7fb010102060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:14.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.168+0000 7fb018200640 1 -- 172.21.15.83:0/3845415338 shutdown_connections 2024-04-24T09:16:14.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.169+0000 7fb018200640 1 -- 172.21.15.83:0/3845415338 wait complete. 2024-04-24T09:16:14.216 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T09:16:14.217 INFO:tasks.workunit.client.0.smithi083.stdout: 10 78945 6.53 MB/sec execute 525 sec latency 123.003 ms 2024-04-24T09:16:14.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.468+0000 7f73f3000640 1 -- 172.21.15.83:0/2481669132 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f73ec1030a0 msgr2=0x7f73ec103480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:14.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.468+0000 7f73f3000640 1 --2- 172.21.15.83:0/2481669132 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f73ec1030a0 0x7f73ec103480 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f73e0009900 tx=0x7f73e002a480 comp rx=0 tx=0).stop 2024-04-24T09:16:14.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.468+0000 7f73f3000640 1 -- 172.21.15.83:0/2481669132 shutdown_connections 2024-04-24T09:16:14.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.468+0000 7f73f3000640 1 --2- 172.21.15.83:0/2481669132 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f73ec10c470 0x7f73ec10e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:14.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.468+0000 7f73f3000640 1 --2- 172.21.15.83:0/2481669132 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f73ec1039c0 0x7f73ec10bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:14.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.468+0000 7f73f3000640 1 --2- 172.21.15.83:0/2481669132 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f73ec1030a0 0x7f73ec103480 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:14.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.468+0000 7f73f3000640 1 -- 172.21.15.83:0/2481669132 >> 172.21.15.83:0/2481669132 conn(0x7f73ec0fc880 msgr2=0x7f73ec0feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:14.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.468+0000 7f73f3000640 1 -- 172.21.15.83:0/2481669132 shutdown_connections 2024-04-24T09:16:14.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.468+0000 7f73f3000640 1 -- 172.21.15.83:0/2481669132 wait complete. 2024-04-24T09:16:14.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.470+0000 7f73f3000640 1 Processor -- start 2024-04-24T09:16:14.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.470+0000 7f73f3000640 1 -- start start 2024-04-24T09:16:14.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.470+0000 7f73f3000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f73ec1030a0 0x7f73ec19a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:14.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.470+0000 7f73f3000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f73ec1039c0 0x7f73ec19abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:14.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.470+0000 7f73f3000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f73ec10c470 0x7f73ec19ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:14.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.470+0000 7f73f3000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f73ec112150 con 0x7f73ec10c470 2024-04-24T09:16:14.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.470+0000 7f73f3000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f73ec111fd0 con 0x7f73ec1039c0 2024-04-24T09:16:14.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.470+0000 7f73f3000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f73ec1122d0 con 0x7f73ec1030a0 2024-04-24T09:16:14.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.471+0000 7f73f2600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f73ec10c470 0x7f73ec19ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:14.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.471+0000 7f73f1c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f73ec1030a0 0x7f73ec19a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:14.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.471+0000 7f73f1200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f73ec1039c0 0x7f73ec19abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:14.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.471+0000 7f73f1c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f73ec1030a0 0x7f73ec19a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:35450/0 (socket says 172.21.15.83:35450) 2024-04-24T09:16:14.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.471+0000 7f73f1c00640 1 -- 172.21.15.83:0/3195141997 learned_addr learned my addr 172.21.15.83:0/3195141997 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:16:14.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.471+0000 7f73f1200640 1 -- 172.21.15.83:0/3195141997 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f73ec1030a0 msgr2=0x7f73ec19a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:14.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.471+0000 7f73f1200640 1 --2- 172.21.15.83:0/3195141997 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f73ec1030a0 0x7f73ec19a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:14.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.472+0000 7f73f1200640 1 -- 172.21.15.83:0/3195141997 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f73ec10c470 msgr2=0x7f73ec19ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:14.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.472+0000 7f73f1200640 1 --2- 172.21.15.83:0/3195141997 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f73ec10c470 0x7f73ec19ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:14.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.472+0000 7f73f1200640 1 -- 172.21.15.83:0/3195141997 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73ec19f6a0 con 0x7f73ec1039c0 2024-04-24T09:16:14.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.472+0000 7f73f1200640 1 --2- 172.21.15.83:0/3195141997 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f73ec1039c0 0x7f73ec19abf0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f73d800d720 tx=0x7f73d800dbe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:14.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.473+0000 7f73ea000640 1 -- 172.21.15.83:0/3195141997 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d8017070 con 0x7f73ec1039c0 2024-04-24T09:16:14.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.473+0000 7f73ea000640 1 -- 172.21.15.83:0/3195141997 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f73d800de30 con 0x7f73ec1039c0 2024-04-24T09:16:14.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.473+0000 7f73ea000640 1 -- 172.21.15.83:0/3195141997 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d80047a0 con 0x7f73ec1039c0 2024-04-24T09:16:14.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.473+0000 7f73f3000640 1 -- 172.21.15.83:0/3195141997 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73ec19f990 con 0x7f73ec1039c0 2024-04-24T09:16:14.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.473+0000 7f73f3000640 1 -- 172.21.15.83:0/3195141997 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73ec103e40 con 0x7f73ec1039c0 2024-04-24T09:16:14.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.477+0000 7f73f7386740 1 Processor -- start 2024-04-24T09:16:14.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.477+0000 7f73f7386740 1 -- start start 2024-04-24T09:16:14.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.477+0000 7f73f7386740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c6fe38b40 0x563c6fe38f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:14.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.477+0000 7f73f7386740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c6fe39460 0x563c6fe435f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:14.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.477+0000 7f73f7386740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c6fe43b30 0x563c6fe45f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:14.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.477+0000 7f73f7386740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x563c6fd9e1d0 con 0x563c6fe38b40 2024-04-24T09:16:14.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.477+0000 7f73f7386740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563c6fd44450 con 0x563c6fe39460 2024-04-24T09:16:14.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.477+0000 7f73f7386740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x563c6fd9a190 con 0x563c6fe43b30 2024-04-24T09:16:14.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.478+0000 7f73f1c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c6fe39460 0x563c6fe435f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:14.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.478+0000 7f73f1c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c6fe39460 0x563c6fe435f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:51360/0 (socket says 172.21.15.83:51360) 2024-04-24T09:16:14.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.478+0000 7f73f1c00640 1 -- 172.21.15.83:0/1135193491 learned_addr learned my addr 172.21.15.83:0/1135193491 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:16:14.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.478+0000 7f73f2600640 1 --2- 172.21.15.83:0/1135193491 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c6fe38b40 0x563c6fe38f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:14.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.478+0000 7f73f1c00640 1 -- 172.21.15.83:0/1135193491 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c6fe43b30 msgr2=0x563c6fe45f20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T09:16:14.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.478+0000 7f73f1c00640 1 --2- 172.21.15.83:0/1135193491 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c6fe43b30 0x563c6fe45f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:14.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.478+0000 7f73f1c00640 1 -- 172.21.15.83:0/1135193491 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c6fe38b40 msgr2=0x563c6fe38f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:14.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.478+0000 7f73f1c00640 1 --2- 172.21.15.83:0/1135193491 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c6fe38b40 0x563c6fe38f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:14.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.478+0000 7f73f1c00640 1 -- 172.21.15.83:0/1135193491 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c6fe01310 con 0x563c6fe39460 2024-04-24T09:16:14.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.478+0000 7f73f2600640 1 --2- 172.21.15.83:0/1135193491 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c6fe38b40 0x563c6fe38f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:16:14.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.479+0000 7f73ea000640 1 -- 172.21.15.83:0/3195141997 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f73d80099d0 con 0x7f73ec1039c0 2024-04-24T09:16:14.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.479+0000 7f73f1c00640 1 --2- 172.21.15.83:0/1135193491 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c6fe39460 0x563c6fe435f0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f73e0004200 tx=0x7f73e0000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5bcab7dd44e7827 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:14.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.479+0000 7f73dea00640 1 -- 172.21.15.83:0/1135193491 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e0035040 con 0x563c6fe39460 2024-04-24T09:16:14.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.480+0000 7f73dea00640 1 -- 172.21.15.83:0/1135193491 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f73e0003ca0 con 0x563c6fe39460 2024-04-24T09:16:14.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.480+0000 7f73dea00640 1 -- 172.21.15.83:0/1135193491 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e003c440 con 0x563c6fe39460 2024-04-24T09:16:14.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.480+0000 7f73f7386740 1 -- 172.21.15.83:0/1135193491 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c6fe39460 msgr2=0x563c6fe435f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:14.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.480+0000 7f73f7386740 1 --2- 172.21.15.83:0/1135193491 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c6fe39460 0x563c6fe435f0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f73e0004200 tx=0x7f73e0000bc0 comp rx=0 tx=0).stop 2024-04-24T09:16:14.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.481+0000 7f73f7386740 1 -- 172.21.15.83:0/1135193491 shutdown_connections 2024-04-24T09:16:14.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.481+0000 7f73f7386740 1 --2- 172.21.15.83:0/1135193491 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c6fe43b30 0x563c6fe45f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:14.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.481+0000 7f73f7386740 1 --2- 172.21.15.83:0/1135193491 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c6fe39460 0x563c6fe435f0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:14.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.481+0000 7f73f7386740 1 --2- 172.21.15.83:0/1135193491 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c6fe38b40 0x563c6fe38f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:14.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.481+0000 7f73f7386740 1 -- 172.21.15.83:0/1135193491 >> 172.21.15.83:0/1135193491 conn(0x563c6fe34e60 msgr2=0x563c6fe35240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:14.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.481+0000 7f73f7386740 1 -- 172.21.15.83:0/1135193491 shutdown_connections 2024-04-24T09:16:14.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.481+0000 7f73ea000640 1 --2- 172.21.15.83:0/3195141997 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f73d40a3d80 0x7f73d40a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:14.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.481+0000 7f73f7386740 1 -- 172.21.15.83:0/1135193491 wait complete. 2024-04-24T09:16:14.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.481+0000 7f73ea000640 1 -- 172.21.15.83:0/3195141997 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 9671+0+0 (secure 0 0 0) 0x7f73d80d0750 con 0x7f73ec1039c0 2024-04-24T09:16:14.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.481+0000 7f73f1c00640 1 --2- 172.21.15.83:0/3195141997 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f73d40a3d80 0x7f73d40a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:14.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.482+0000 7f73f7386740 1 -- start start 2024-04-24T09:16:14.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.482+0000 7f73f7386740 1 Processor -- start 2024-04-24T09:16:14.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.482+0000 7f73f1c00640 1 --2- 172.21.15.83:0/3195141997 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f73d40a3d80 0x7f73d40a6240 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f73e0031b00 tx=0x7f73e002a480 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:14.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.482+0000 7f73f7386740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c6fe38b40 0x563c6fe435a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:14.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.482+0000 7f73f7386740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c6fe39460 0x563c6fe40890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:14.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.482+0000 7f73f7386740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c6fe43b30 0x563c6fe40fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:14.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.482+0000 7f73f7386740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x563c6fe46960 con 0x563c6fe43b30 2024-04-24T09:16:14.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.483+0000 7f73f7386740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563c6fd57810 con 0x563c6fe39460 2024-04-24T09:16:14.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.483+0000 7f73f7386740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x563c6fe46ae0 con 0x563c6fe38b40 2024-04-24T09:16:14.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.483+0000 7f73f7386740 10 client.0 _ll_register_callbacks cb 0x563c6fd25140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:16:14.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.483+0000 7f73f2600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c6fe43b30 0x563c6fe40fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:14.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.483+0000 7f73f2600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c6fe43b30 0x563c6fe40fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51950/0 (socket says 172.21.15.83:51950) 2024-04-24T09:16:14.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.483+0000 7f73f2600640 1 -- 172.21.15.83:0/2863542853 learned_addr learned my addr 172.21.15.83:0/2863542853 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:16:14.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.483+0000 7f73f1c00640 1 --2- 172.21.15.83:0/2863542853 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c6fe38b40 0x563c6fe435a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:14.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.483+0000 7f73f1200640 1 --2- 172.21.15.83:0/2863542853 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c6fe39460 0x563c6fe40890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:14.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.483+0000 7f73f2600640 1 -- 172.21.15.83:0/2863542853 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c6fe38b40 msgr2=0x563c6fe435a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:14.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.483+0000 7f73f2600640 1 --2- 172.21.15.83:0/2863542853 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c6fe38b40 0x563c6fe435a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:14.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.483+0000 7f73f2600640 1 -- 172.21.15.83:0/2863542853 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c6fe39460 msgr2=0x563c6fe40890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:14.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.483+0000 7f73f2600640 1 --2- 172.21.15.83:0/2863542853 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c6fe39460 0x563c6fe40890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:14.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.483+0000 7f73f2600640 1 -- 172.21.15.83:0/2863542853 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c6fd683f0 con 0x563c6fe43b30 2024-04-24T09:16:14.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.483+0000 7f73f1200640 1 --2- 172.21.15.83:0/2863542853 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c6fe39460 0x563c6fe40890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:16:14.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.484+0000 7f73f2600640 1 --2- 172.21.15.83:0/2863542853 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c6fe43b30 0x563c6fe40fd0 secure :-1 s=READY pgs=2764 cs=0 l=1 rev1=1 crypto rx=0x7f73e4012700 tx=0x7f73e4012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cc1eb444b73d784 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:14.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.484+0000 7f73d3400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:16:14.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.484+0000 7f73d3400640 1 -- 172.21.15.83:0/2863542853 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e401f060 con 0x563c6fe43b30 2024-04-24T09:16:14.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.484+0000 7f73d3400640 1 -- 172.21.15.83:0/2863542853 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(131..131 src has 1..131) ==== 9671+0+0 (secure 0 0 0) 0x7f73e4010dc0 con 0x563c6fe43b30 2024-04-24T09:16:14.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.484+0000 7f73f7386740 1 -- 172.21.15.83:0/2863542853 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73e0003ca0 con 0x563c6fe43b30 2024-04-24T09:16:14.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.485+0000 7f73d3400640 1 -- 172.21.15.83:0/2863542853 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f73e4026050 con 0x563c6fe43b30 2024-04-24T09:16:14.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.485+0000 7f73f7386740 10 client.23877 fetch_fsmap learned FSMap version 69 2024-04-24T09:16:14.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.485+0000 7f73f7386740 1 -- 172.21.15.83:0/2863542853 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x563c6fab03f0 con 0x563c6fe43b30 2024-04-24T09:16:14.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.485+0000 7f73d3400640 1 -- 172.21.15.83:0/2863542853 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f73e0003ca0 con 0x563c6fe43b30 2024-04-24T09:16:14.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.485+0000 7f73d3400640 1 -- 172.21.15.83:0/2863542853 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 69) ==== 8370+0+0 (secure 0 0 0) 0x7f73e4019030 con 0x563c6fe43b30 2024-04-24T09:16:14.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.485+0000 7f73f7386740 10 client.23877 fetch_fsmap finished waiting for FSMap version 69 2024-04-24T09:16:14.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.485+0000 7f73f7386740 10 client.23877 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-24T09:16:14.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.486+0000 7f73f7386740 20 client.23877 populate_metadata read hostname 'smithi083' 2024-04-24T09:16:14.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.486+0000 7f73f7386740 1 --2- 172.21.15.83:0/2863542853 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x563c6fe64d70 0x563c6fe671d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:14.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.486+0000 7f73f7386740 4 client.23877 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:16:14.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.486+0000 7f73f7386740 1 -- 172.21.15.83:0/2863542853 --> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563c6fdabbf0 con 0x563c6fe64d70 2024-04-24T09:16:14.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.486+0000 7f73f1c00640 1 --2- 172.21.15.83:0/2863542853 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x563c6fe64d70 0x563c6fe671d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:14.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.487+0000 7f73f1c00640 1 --2- 172.21.15.83:0/2863542853 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x563c6fe64d70 0x563c6fe671d0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=78506270699dc9ad server_cookie=c4dd747dbaa33f33 in_seq=0 out_seq=0 2024-04-24T09:16:14.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.487+0000 7f73d3400640 10 client.23877 ms_handle_connect on v2:172.21.15.138:6834/1909219758 2024-04-24T09:16:14.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.488+0000 7f73d3400640 1 -- 172.21.15.83:0/2863542853 <== mds.2 v2:172.21.15.138:6834/1909219758 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563c6fdabbf0 con 0x563c6fe64d70 2024-04-24T09:16:14.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.488+0000 7f73d3400640 10 client.23877 handle_command_reply: tid=0 2024-04-24T09:16:14.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.488+0000 7f73f7386740 1 client.23877 shutdown 2024-04-24T09:16:14.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.489+0000 7f73f7386740 1 -- 172.21.15.83:0/2863542853 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c6fe43b30 msgr2=0x563c6fe40fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:14.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.489+0000 7f73f7386740 1 --2- 172.21.15.83:0/2863542853 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c6fe43b30 0x563c6fe40fd0 secure :-1 s=READY pgs=2764 cs=0 l=1 rev1=1 crypto rx=0x7f73e4012700 tx=0x7f73e4012bc0 comp rx=0 tx=0).stop 2024-04-24T09:16:14.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.489+0000 7f73f7386740 1 -- 172.21.15.83:0/2863542853 shutdown_connections 2024-04-24T09:16:14.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.489+0000 7f73f7386740 1 --2- 172.21.15.83:0/2863542853 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x563c6fe64d70 0x563c6fe671d0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:14.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.489+0000 7f73f7386740 1 --2- 172.21.15.83:0/2863542853 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c6fe43b30 0x563c6fe40fd0 unknown :-1 s=CLOSED pgs=2764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:14.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.489+0000 7f73f7386740 1 --2- 172.21.15.83:0/2863542853 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c6fe39460 0x563c6fe40890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:14.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.489+0000 7f73f7386740 1 --2- 172.21.15.83:0/2863542853 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c6fe38b40 0x563c6fe435a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:14.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.489+0000 7f73d3400640 0 client.23877 ms_handle_reset on v2:172.21.15.138:6834/1909219758 2024-04-24T09:16:14.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.489+0000 7f73f7386740 1 -- 172.21.15.83:0/2863542853 >> 172.21.15.83:0/2863542853 conn(0x563c6fe34e60 msgr2=0x563c6fe3ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:14.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.489+0000 7f73f7386740 1 -- 172.21.15.83:0/2863542853 shutdown_connections 2024-04-24T09:16:14.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.489+0000 7f73f7386740 1 -- 172.21.15.83:0/2863542853 wait complete. 2024-04-24T09:16:14.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.489+0000 7f73f7386740 20 client.23877 trim_cache size 0 max 16384 2024-04-24T09:16:14.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.511+0000 7f73f7386740 1 Processor -- start 2024-04-24T09:16:14.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.512+0000 7f73f7386740 1 -- start start 2024-04-24T09:16:14.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.512+0000 7f73f7386740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c6fe38b40 0x563c6fe3b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:14.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.512+0000 7f73f7386740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c6fe39460 0x563c6fe430d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:14.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.512+0000 7f73f7386740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c6fe43b30 0x563c6fe6caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:14.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.512+0000 7f73f7386740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f73ec1ad0b0 con 0x563c6fe39460 2024-04-24T09:16:14.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.512+0000 7f73f7386740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f73ec1aec50 con 0x563c6fe43b30 2024-04-24T09:16:14.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.512+0000 7f73f7386740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f73ec1ad780 con 0x563c6fe38b40 2024-04-24T09:16:14.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.512+0000 7f73f1200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c6fe39460 0x563c6fe430d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:14.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.512+0000 7f73f1200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c6fe39460 0x563c6fe430d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51956/0 (socket says 172.21.15.83:51956) 2024-04-24T09:16:14.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.512+0000 7f73f2600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c6fe43b30 0x563c6fe6caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:14.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.513+0000 7f73f1200640 1 -- 172.21.15.83:0/1607801959 learned_addr learned my addr 172.21.15.83:0/1607801959 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:16:14.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.513+0000 7f73f1c00640 1 --2- 172.21.15.83:0/1607801959 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c6fe38b40 0x563c6fe3b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:14.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.513+0000 7f73f1200640 1 -- 172.21.15.83:0/1607801959 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c6fe38b40 msgr2=0x563c6fe3b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:14.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.513+0000 7f73f1200640 1 --2- 172.21.15.83:0/1607801959 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c6fe38b40 0x563c6fe3b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:14.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.513+0000 7f73f1200640 1 -- 172.21.15.83:0/1607801959 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c6fe43b30 msgr2=0x563c6fe6caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:14.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.513+0000 7f73f1200640 1 --2- 172.21.15.83:0/1607801959 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c6fe43b30 0x563c6fe6caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:14.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.513+0000 7f73f1200640 1 -- 172.21.15.83:0/1607801959 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c6fb7a7d0 con 0x563c6fe39460 2024-04-24T09:16:14.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.514+0000 7f73f1200640 1 --2- 172.21.15.83:0/1607801959 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c6fe39460 0x563c6fe430d0 secure :-1 s=READY pgs=2765 cs=0 l=1 rev1=1 crypto rx=0x7f73d8009610 tx=0x7f73d80057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b81e5989c09a49f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:14.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.514+0000 7f73de000640 1 -- 172.21.15.83:0/1607801959 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d80cf040 con 0x563c6fe39460 2024-04-24T09:16:14.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.514+0000 7f73de000640 1 -- 172.21.15.83:0/1607801959 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f73d80a0620 con 0x563c6fe39460 2024-04-24T09:16:14.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.514+0000 7f73de000640 1 -- 172.21.15.83:0/1607801959 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d80cf040 con 0x563c6fe39460 2024-04-24T09:16:14.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.515+0000 7f73f7386740 1 -- 172.21.15.83:0/1607801959 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c6fe39460 msgr2=0x563c6fe430d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:14.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.515+0000 7f73f7386740 1 --2- 172.21.15.83:0/1607801959 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c6fe39460 0x563c6fe430d0 secure :-1 s=READY pgs=2765 cs=0 l=1 rev1=1 crypto rx=0x7f73d8009610 tx=0x7f73d80057b0 comp rx=0 tx=0).stop 2024-04-24T09:16:14.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.515+0000 7f73f7386740 1 -- 172.21.15.83:0/1607801959 shutdown_connections 2024-04-24T09:16:14.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.515+0000 7f73f7386740 1 --2- 172.21.15.83:0/1607801959 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c6fe43b30 0x563c6fe6caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:14.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.515+0000 7f73f7386740 1 --2- 172.21.15.83:0/1607801959 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c6fe39460 0x563c6fe430d0 unknown :-1 s=CLOSED pgs=2765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:14.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.515+0000 7f73f7386740 1 --2- 172.21.15.83:0/1607801959 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c6fe38b40 0x563c6fe3b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:14.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.516+0000 7f73f7386740 1 -- 172.21.15.83:0/1607801959 >> 172.21.15.83:0/1607801959 conn(0x563c6fe34e60 msgr2=0x563c6fe3f1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:14.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.516+0000 7f73f7386740 1 -- 172.21.15.83:0/1607801959 shutdown_connections 2024-04-24T09:16:14.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.516+0000 7f73f7386740 1 -- 172.21.15.83:0/1607801959 wait complete. 2024-04-24T09:16:14.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.517+0000 7f73f7386740 1 -- start start 2024-04-24T09:16:14.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.517+0000 7f73f7386740 1 Processor -- start 2024-04-24T09:16:14.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.517+0000 7f73f7386740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c6fe38b40 0x563c6fe4c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:14.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.518+0000 7f73f7386740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c6fe39460 0x563c6fe3dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:14.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.518+0000 7f73f7386740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c6fe43b30 0x563c6fe3e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:14.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.518+0000 7f73f7386740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f73ec1ae580 con 0x563c6fe38b40 2024-04-24T09:16:14.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.518+0000 7f73f7386740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f73ec1adeb0 con 0x563c6fe39460 2024-04-24T09:16:14.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.518+0000 7f73f7386740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f73ec1ab110 con 0x563c6fe43b30 2024-04-24T09:16:14.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.518+0000 7f73f7386740 10 client.0 _ll_register_callbacks cb 0x7f73e4009a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:16:14.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.518+0000 7f73f1200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c6fe39460 0x563c6fe3dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:14.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.518+0000 7f73f1200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c6fe39460 0x563c6fe3dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:51384/0 (socket says 172.21.15.83:51384) 2024-04-24T09:16:14.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.518+0000 7f73f1200640 1 -- 172.21.15.83:0/3650609204 learned_addr learned my addr 172.21.15.83:0/3650609204 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:16:14.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.519+0000 7f73f1c00640 1 --2- 172.21.15.83:0/3650609204 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c6fe38b40 0x563c6fe4c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:14.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.519+0000 7f73f2600640 1 --2- 172.21.15.83:0/3650609204 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c6fe43b30 0x563c6fe3e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:14.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.519+0000 7f73f1200640 1 -- 172.21.15.83:0/3650609204 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c6fe43b30 msgr2=0x563c6fe3e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:14.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.519+0000 7f73f1200640 1 --2- 172.21.15.83:0/3650609204 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c6fe43b30 0x563c6fe3e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:14.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.519+0000 7f73f1200640 1 -- 172.21.15.83:0/3650609204 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c6fe38b40 msgr2=0x563c6fe4c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:14.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.519+0000 7f73f1200640 1 --2- 172.21.15.83:0/3650609204 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c6fe38b40 0x563c6fe4c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:14.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.519+0000 7f73f1200640 1 -- 172.21.15.83:0/3650609204 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c6fe448b0 con 0x563c6fe39460 2024-04-24T09:16:14.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.519+0000 7f73f1c00640 1 --2- 172.21.15.83:0/3650609204 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c6fe38b40 0x563c6fe4c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:16:14.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.519+0000 7f73f1200640 1 --2- 172.21.15.83:0/3650609204 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c6fe39460 0x563c6fe3dd10 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f73d80986d0 tx=0x7f73d8009f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f99d629f30f9f2c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:14.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.519+0000 7f73d3400640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:16:14.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.519+0000 7f73d3400640 1 -- 172.21.15.83:0/3650609204 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d80ba070 con 0x563c6fe39460 2024-04-24T09:16:14.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.520+0000 7f73d3400640 1 -- 172.21.15.83:0/3650609204 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(131..131 src has 1..131) ==== 9671+0+0 (secure 0 0 0) 0x7f73d80988e0 con 0x563c6fe39460 2024-04-24T09:16:14.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.520+0000 7f73f7386740 1 -- 172.21.15.83:0/3650609204 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73d80a0620 con 0x563c6fe39460 2024-04-24T09:16:14.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.520+0000 7f73d3400640 1 -- 172.21.15.83:0/3650609204 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f73d80b0050 con 0x563c6fe39460 2024-04-24T09:16:14.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.520+0000 7f73f7386740 10 client.40952 fetch_fsmap learned FSMap version 69 2024-04-24T09:16:14.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.520+0000 7f73f7386740 1 -- 172.21.15.83:0/3650609204 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x563c6fedc5b0 con 0x563c6fe39460 2024-04-24T09:16:14.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.520+0000 7f73d3400640 1 -- 172.21.15.83:0/3650609204 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f73d80a0620 con 0x563c6fe39460 2024-04-24T09:16:14.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.520+0000 7f73d3400640 1 -- 172.21.15.83:0/3650609204 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 69) ==== 8370+0+0 (secure 0 0 0) 0x7f73d80a0170 con 0x563c6fe39460 2024-04-24T09:16:14.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.520+0000 7f73f7386740 10 client.40952 fetch_fsmap finished waiting for FSMap version 69 2024-04-24T09:16:14.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.520+0000 7f73f7386740 10 client.40952 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-24T09:16:14.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.520+0000 7f73f7386740 20 client.40952 populate_metadata read hostname 'smithi083' 2024-04-24T09:16:14.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.521+0000 7f73f7386740 1 --2- 172.21.15.83:0/3650609204 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x563c6fedccd0 0x563c6fedf0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:14.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.521+0000 7f73f7386740 4 client.40952 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:16:14.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.521+0000 7f73f7386740 1 -- 172.21.15.83:0/3650609204 --> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563c6fda5a60 con 0x563c6fedccd0 2024-04-24T09:16:14.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.521+0000 7f73f1c00640 1 --2- 172.21.15.83:0/3650609204 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x563c6fedccd0 0x563c6fedf0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:14.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.522+0000 7f73f1c00640 1 --2- 172.21.15.83:0/3650609204 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x563c6fedccd0 0x563c6fedf0f0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a8d84f227f0841db server_cookie=f809866e128ef65b in_seq=0 out_seq=0 2024-04-24T09:16:14.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.522+0000 7f73d3400640 10 client.40952 ms_handle_connect on v2:172.21.15.138:6834/1909219758 2024-04-24T09:16:14.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.522+0000 7f73d3400640 1 -- 172.21.15.83:0/3650609204 <== mds.2 v2:172.21.15.138:6834/1909219758 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563c6fda5a60 con 0x563c6fedccd0 2024-04-24T09:16:14.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.522+0000 7f73d3400640 10 client.40952 handle_command_reply: tid=0 2024-04-24T09:16:14.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.522+0000 7f73f7386740 1 client.40952 shutdown 2024-04-24T09:16:14.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.522+0000 7f73f7386740 1 -- 172.21.15.83:0/3650609204 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c6fe39460 msgr2=0x563c6fe3dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:14.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.522+0000 7f73f7386740 1 --2- 172.21.15.83:0/3650609204 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c6fe39460 0x563c6fe3dd10 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f73d80986d0 tx=0x7f73d8009f50 comp rx=0 tx=0).stop 2024-04-24T09:16:14.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.522+0000 7f73f7386740 1 -- 172.21.15.83:0/3650609204 shutdown_connections 2024-04-24T09:16:14.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.523+0000 7f73f7386740 1 --2- 172.21.15.83:0/3650609204 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x563c6fedccd0 0x563c6fedf0f0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:14.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.523+0000 7f73d3400640 0 client.40952 ms_handle_reset on v2:172.21.15.138:6834/1909219758 2024-04-24T09:16:14.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.523+0000 7f73f7386740 1 --2- 172.21.15.83:0/3650609204 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c6fe43b30 0x563c6fe3e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:14.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.523+0000 7f73f7386740 1 --2- 172.21.15.83:0/3650609204 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c6fe39460 0x563c6fe3dd10 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:14.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.523+0000 7f73f7386740 1 --2- 172.21.15.83:0/3650609204 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c6fe38b40 0x563c6fe4c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:14.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.523+0000 7f73f7386740 1 -- 172.21.15.83:0/3650609204 >> 172.21.15.83:0/3650609204 conn(0x563c6fe34e60 msgr2=0x563c6fe336c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:14.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.523+0000 7f73f7386740 1 -- 172.21.15.83:0/3650609204 shutdown_connections 2024-04-24T09:16:14.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.523+0000 7f73f7386740 1 -- 172.21.15.83:0/3650609204 wait complete. 2024-04-24T09:16:14.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.523+0000 7f73f7386740 20 client.40952 trim_cache size 0 max 16384 2024-04-24T09:16:14.526 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:16:14.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.524+0000 7f73f3000640 1 -- 172.21.15.83:0/3195141997 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f73d40a3d80 msgr2=0x7f73d40a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:14.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.524+0000 7f73f3000640 1 --2- 172.21.15.83:0/3195141997 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f73d40a3d80 0x7f73d40a6240 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f73e0031b00 tx=0x7f73e002a480 comp rx=0 tx=0).stop 2024-04-24T09:16:14.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.524+0000 7f73f3000640 1 -- 172.21.15.83:0/3195141997 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f73ec1039c0 msgr2=0x7f73ec19abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:14.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.524+0000 7f73f3000640 1 --2- 172.21.15.83:0/3195141997 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f73ec1039c0 0x7f73ec19abf0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f73d800d720 tx=0x7f73d800dbe0 comp rx=0 tx=0).stop 2024-04-24T09:16:14.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.524+0000 7f73f3000640 1 -- 172.21.15.83:0/3195141997 shutdown_connections 2024-04-24T09:16:14.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.524+0000 7f73f3000640 1 --2- 172.21.15.83:0/3195141997 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f73d40a3d80 0x7f73d40a6240 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:14.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.524+0000 7f73f3000640 1 --2- 172.21.15.83:0/3195141997 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f73ec10c470 0x7f73ec19ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:14.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.524+0000 7f73f3000640 1 --2- 172.21.15.83:0/3195141997 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f73ec1039c0 0x7f73ec19abf0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:14.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.524+0000 7f73f3000640 1 --2- 172.21.15.83:0/3195141997 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f73ec1030a0 0x7f73ec19a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:14.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.524+0000 7f73f3000640 1 -- 172.21.15.83:0/3195141997 >> 172.21.15.83:0/3195141997 conn(0x7f73ec0fc880 msgr2=0x7f73ec10dd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:14.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.524+0000 7f73f3000640 1 -- 172.21.15.83:0/3195141997 shutdown_connections 2024-04-24T09:16:14.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.524+0000 7f73f3000640 1 -- 172.21.15.83:0/3195141997 wait complete. 2024-04-24T09:16:14.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:16:14 smithi083 ceph-mon[27907]: pgmap v778: 129 pgs: 129 active+clean; 1.2 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.3 MiB/s wr, 289 op/s 2024-04-24T09:16:14.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:16:14 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/478839531' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:16:14.548 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T09:16:14.704 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:16:14 smithi138 ceph-mon[31441]: pgmap v778: 129 pgs: 129 active+clean; 1.2 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.3 MiB/s wr, 289 op/s 2024-04-24T09:16:14.704 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:16:14 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/478839531' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:16:14.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:16:14 smithi189 ceph-mon[31791]: pgmap v778: 129 pgs: 129 active+clean; 1.2 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.3 MiB/s wr, 289 op/s 2024-04-24T09:16:14.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:16:14 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/478839531' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:16:14.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.805+0000 7f74a6000640 1 -- 172.21.15.83:0/1796018625 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74a0100b60 msgr2=0x7f74a0104ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:14.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.805+0000 7f74a6000640 1 --2- 172.21.15.83:0/1796018625 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74a0100b60 0x7f74a0104ee0 secure :-1 s=READY pgs=2766 cs=0 l=1 rev1=1 crypto rx=0x7f7490009900 tx=0x7f749002a3f0 comp rx=0 tx=0).stop 2024-04-24T09:16:14.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.805+0000 7f74a6000640 1 -- 172.21.15.83:0/1796018625 shutdown_connections 2024-04-24T09:16:14.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.805+0000 7f74a6000640 1 --2- 172.21.15.83:0/1796018625 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f74a010b180 0x7f74a010d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:14.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.805+0000 7f74a6000640 1 --2- 172.21.15.83:0/1796018625 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f74a0105420 0x7f74a0105880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:14.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.805+0000 7f74a6000640 1 --2- 172.21.15.83:0/1796018625 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74a0100b60 0x7f74a0104ee0 unknown :-1 s=CLOSED pgs=2766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:14.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.805+0000 7f74a6000640 1 -- 172.21.15.83:0/1796018625 >> 172.21.15.83:0/1796018625 conn(0x7f74a0077060 msgr2=0x7f74a00fe450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:14.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.806+0000 7f74a6000640 1 -- 172.21.15.83:0/1796018625 shutdown_connections 2024-04-24T09:16:14.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.806+0000 7f74a6000640 1 -- 172.21.15.83:0/1796018625 wait complete. 2024-04-24T09:16:14.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.807+0000 7f74a6000640 1 Processor -- start 2024-04-24T09:16:14.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.807+0000 7f74a6000640 1 -- start start 2024-04-24T09:16:14.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.808+0000 7f74a6000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f74a0100b60 0x7f74a01966f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:14.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.808+0000 7f74a6000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74a0105420 0x7f74a0196c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:14.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.808+0000 7f74a6000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f74a010b180 0x7f74a019afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:14.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.808+0000 7f74a6000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f74a0110150 con 0x7f74a0105420 2024-04-24T09:16:14.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.808+0000 7f74a6000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f74a010ffd0 con 0x7f74a0100b60 2024-04-24T09:16:14.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.808+0000 7f74a6000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f74a01102d0 con 0x7f74a010b180 2024-04-24T09:16:14.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.809+0000 7f749fe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f74a010b180 0x7f74a019afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:14.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.809+0000 7f749fe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f74a010b180 0x7f74a019afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:35504/0 (socket says 172.21.15.83:35504) 2024-04-24T09:16:14.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.809+0000 7f749fe00640 1 -- 172.21.15.83:0/1834767840 learned_addr learned my addr 172.21.15.83:0/1834767840 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:16:14.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.809+0000 7f749f400640 1 --2- 172.21.15.83:0/1834767840 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f74a0100b60 0x7f74a01966f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:14.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.808+0000 7f749ea00640 1 --2- 172.21.15.83:0/1834767840 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74a0105420 0x7f74a0196c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:14.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.810+0000 7f749fe00640 1 -- 172.21.15.83:0/1834767840 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f74a0100b60 msgr2=0x7f74a01966f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:14.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.810+0000 7f749fe00640 1 --2- 172.21.15.83:0/1834767840 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f74a0100b60 0x7f74a01966f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:14.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.810+0000 7f749fe00640 1 -- 172.21.15.83:0/1834767840 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74a0105420 msgr2=0x7f74a0196c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:14.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.810+0000 7f749fe00640 1 --2- 172.21.15.83:0/1834767840 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74a0105420 0x7f74a0196c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:14.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.810+0000 7f749fe00640 1 -- 172.21.15.83:0/1834767840 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74a019b6e0 con 0x7f74a010b180 2024-04-24T09:16:14.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.811+0000 7f749fe00640 1 --2- 172.21.15.83:0/1834767840 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f74a010b180 0x7f74a019afb0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f749800e960 tx=0x7f749800ee20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:14.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.817+0000 7f7497e00640 1 -- 172.21.15.83:0/1834767840 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7498004d20 con 0x7f74a010b180 2024-04-24T09:16:14.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.817+0000 7f7497e00640 1 -- 172.21.15.83:0/1834767840 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7498005020 con 0x7f74a010b180 2024-04-24T09:16:14.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.818+0000 7f74a6000640 1 -- 172.21.15.83:0/1834767840 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74a019b9d0 con 0x7f74a010b180 2024-04-24T09:16:14.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.818+0000 7f74a6000640 1 -- 172.21.15.83:0/1834767840 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74a01a3260 con 0x7f74a010b180 2024-04-24T09:16:14.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.818+0000 7f7497e00640 1 -- 172.21.15.83:0/1834767840 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74980116b0 con 0x7f74a010b180 2024-04-24T09:16:14.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.826+0000 7f7497e00640 1 -- 172.21.15.83:0/1834767840 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7498011850 con 0x7f74a010b180 2024-04-24T09:16:14.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.829+0000 7f74a8f90740 1 Processor -- start 2024-04-24T09:16:14.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.830+0000 7f74a8f90740 1 -- start start 2024-04-24T09:16:14.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.831+0000 7f74a8f90740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa974c6160 0x55fa974c6540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:14.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.831+0000 7f74a8f90740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fa974c6a80 0x55fa974d0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:14.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.831+0000 7f74a8f90740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa974d1150 0x55fa974d3540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:14.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.831+0000 7f74a8f90740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55fa974231d0 con 0x55fa974d1150 2024-04-24T09:16:14.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.831+0000 7f74a8f90740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55fa973c9450 con 0x55fa974c6a80 2024-04-24T09:16:14.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.831+0000 7f74a8f90740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55fa9741f190 con 0x55fa974c6160 2024-04-24T09:16:14.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.831+0000 7f749ea00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa974c6160 0x55fa974c6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:14.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.831+0000 7f749ea00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa974c6160 0x55fa974c6540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:35512/0 (socket says 172.21.15.83:35512) 2024-04-24T09:16:14.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.832+0000 7f749ea00640 1 -- 172.21.15.83:0/2839534482 learned_addr learned my addr 172.21.15.83:0/2839534482 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:16:14.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.832+0000 7f749ea00640 1 --2- 172.21.15.83:0/2839534482 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa974d1150 0x55fa974d3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:14.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.832+0000 7f749f400640 1 --2- 172.21.15.83:0/2839534482 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fa974c6a80 0x55fa974d0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:14.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.832+0000 7f7497e00640 1 --2- 172.21.15.83:0/1834767840 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f74840037c0 0x7f7484005c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:14.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.832+0000 7f749ea00640 1 -- 172.21.15.83:0/2839534482 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa974c6160 msgr2=0x55fa974c6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:14.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.832+0000 7f749ea00640 1 --2- 172.21.15.83:0/2839534482 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa974c6160 0x55fa974c6540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:14.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.832+0000 7f749ea00640 1 -- 172.21.15.83:0/2839534482 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fa974c6a80 msgr2=0x55fa974d0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:14.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.832+0000 7f749ea00640 1 --2- 172.21.15.83:0/2839534482 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fa974c6a80 0x55fa974d0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:14.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.832+0000 7f749ea00640 1 -- 172.21.15.83:0/2839534482 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa97486310 con 0x55fa974d1150 2024-04-24T09:16:14.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.833+0000 7f749fe00640 1 --2- 172.21.15.83:0/1834767840 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f74840037c0 0x7f7484005c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:14.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.833+0000 7f749ea00640 1 --2- 172.21.15.83:0/2839534482 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa974d1150 0x55fa974d3540 secure :-1 s=READY pgs=2767 cs=0 l=1 rev1=1 crypto rx=0x7f748c00adf0 tx=0x7f748c006da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7aa2ac50b1350874 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:14.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.833+0000 7f748be00640 1 -- 172.21.15.83:0/2839534482 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f748c01b070 con 0x55fa974d1150 2024-04-24T09:16:14.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.834+0000 7f748be00640 1 -- 172.21.15.83:0/2839534482 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f748c014c80 con 0x55fa974d1150 2024-04-24T09:16:14.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.834+0000 7f748be00640 1 -- 172.21.15.83:0/2839534482 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f748c0125f0 con 0x55fa974d1150 2024-04-24T09:16:14.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.834+0000 7f749fe00640 1 --2- 172.21.15.83:0/1834767840 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f74840037c0 0x7f7484005c80 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f749800c640 tx=0x7f7498013000 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:14.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.836+0000 7f74a8f90740 1 -- 172.21.15.83:0/2839534482 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa974d1150 msgr2=0x55fa974d3540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:14.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.836+0000 7f74a8f90740 1 --2- 172.21.15.83:0/2839534482 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa974d1150 0x55fa974d3540 secure :-1 s=READY pgs=2767 cs=0 l=1 rev1=1 crypto rx=0x7f748c00adf0 tx=0x7f748c006da0 comp rx=0 tx=0).stop 2024-04-24T09:16:14.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.836+0000 7f7497e00640 1 -- 172.21.15.83:0/1834767840 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 9583+0+0 (secure 0 0 0) 0x7f74980d1dd0 con 0x7f74a010b180 2024-04-24T09:16:14.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.836+0000 7f74a8f90740 1 -- 172.21.15.83:0/2839534482 shutdown_connections 2024-04-24T09:16:14.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.836+0000 7f74a8f90740 1 --2- 172.21.15.83:0/2839534482 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa974d1150 0x55fa974d3540 secure :-1 s=CLOSED pgs=2767 cs=0 l=1 rev1=1 crypto rx=0x7f748c00adf0 tx=0x7f748c006da0 comp rx=0 tx=0).stop 2024-04-24T09:16:14.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.836+0000 7f74a8f90740 1 --2- 172.21.15.83:0/2839534482 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fa974c6a80 0x55fa974d0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:14.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.836+0000 7f74a8f90740 1 --2- 172.21.15.83:0/2839534482 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa974c6160 0x55fa974c6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:14.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.836+0000 7f74a8f90740 1 -- 172.21.15.83:0/2839534482 >> 172.21.15.83:0/2839534482 conn(0x55fa974c2480 msgr2=0x55fa974c2860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:14.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.837+0000 7f74a8f90740 1 -- 172.21.15.83:0/2839534482 shutdown_connections 2024-04-24T09:16:14.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.838+0000 7f74a8f90740 1 -- 172.21.15.83:0/2839534482 wait complete. 2024-04-24T09:16:14.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.838+0000 7f74a8f90740 1 -- start start 2024-04-24T09:16:14.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.838+0000 7f74a8f90740 1 Processor -- start 2024-04-24T09:16:14.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.839+0000 7f74a8f90740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa974c6160 0x55fa974ce030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:14.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.839+0000 7f74a8f90740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fa974c6a80 0x55fa974ce570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:14.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.839+0000 7f74a8f90740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa974ceab0 0x55fa974ee4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:14.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.839+0000 7f74a8f90740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55fa974e4ff0 con 0x55fa974ceab0 2024-04-24T09:16:14.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.839+0000 7f74a8f90740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55fa973dc810 con 0x55fa974c6a80 2024-04-24T09:16:14.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.839+0000 7f74a8f90740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55fa974e5170 con 0x55fa974c6160 2024-04-24T09:16:14.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.839+0000 7f74a8f90740 10 client.0 _ll_register_callbacks cb 0x55fa973aa140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:16:14.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.839+0000 7f749f400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fa974c6a80 0x55fa974ce570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:14.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.839+0000 7f749f400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fa974c6a80 0x55fa974ce570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:51412/0 (socket says 172.21.15.83:51412) 2024-04-24T09:16:14.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.839+0000 7f749f400640 1 -- 172.21.15.83:0/3959655411 learned_addr learned my addr 172.21.15.83:0/3959655411 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:16:14.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.839+0000 7f749ea00640 1 --2- 172.21.15.83:0/3959655411 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa974c6160 0x55fa974ce030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:14.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.840+0000 7f749ea00640 1 --2- 172.21.15.83:0/3959655411 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa974ceab0 0x55fa974ee4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:14.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.840+0000 7f749f400640 1 -- 172.21.15.83:0/3959655411 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa974c6160 msgr2=0x55fa974ce030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:14.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.840+0000 7f749f400640 1 --2- 172.21.15.83:0/3959655411 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa974c6160 0x55fa974ce030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:14.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.840+0000 7f749f400640 1 -- 172.21.15.83:0/3959655411 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa974ceab0 msgr2=0x55fa974ee4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:14.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.840+0000 7f749f400640 1 --2- 172.21.15.83:0/3959655411 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa974ceab0 0x55fa974ee4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:14.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.840+0000 7f749f400640 1 -- 172.21.15.83:0/3959655411 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa973ed3f0 con 0x55fa974c6a80 2024-04-24T09:16:14.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.840+0000 7f749ea00640 1 --2- 172.21.15.83:0/3959655411 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa974ceab0 0x55fa974ee4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:16:14.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.841+0000 7f749f400640 1 --2- 172.21.15.83:0/3959655411 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fa974c6a80 0x55fa974ce570 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f7490005ab0 tx=0x7f74900052e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64784a6f18de0063 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:14.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.841+0000 7f7488c00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:16:14.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.841+0000 7f7488c00640 1 -- 172.21.15.83:0/3959655411 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7490035040 con 0x55fa974c6a80 2024-04-24T09:16:14.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.841+0000 7f74a8f90740 1 -- 172.21.15.83:0/3959655411 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f748c014c80 con 0x55fa974c6a80 2024-04-24T09:16:14.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.841+0000 7f7488c00640 1 -- 172.21.15.83:0/3959655411 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(132..132 src has 1..132) ==== 9583+0+0 (secure 0 0 0) 0x7f74900048c0 con 0x55fa974c6a80 2024-04-24T09:16:14.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.842+0000 7f7488c00640 1 -- 172.21.15.83:0/3959655411 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f7490004ac0 con 0x55fa974c6a80 2024-04-24T09:16:14.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.842+0000 7f74a8f90740 10 client.40964 fetch_fsmap learned FSMap version 69 2024-04-24T09:16:14.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.842+0000 7f74a8f90740 1 -- 172.21.15.83:0/3959655411 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x55fa971353f0 con 0x55fa974c6a80 2024-04-24T09:16:14.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.842+0000 7f7488c00640 1 -- 172.21.15.83:0/3959655411 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f748c014c80 con 0x55fa974c6a80 2024-04-24T09:16:14.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.842+0000 7f7488c00640 1 -- 172.21.15.83:0/3959655411 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 69) ==== 8370+0+0 (secure 0 0 0) 0x7f74900314a0 con 0x55fa974c6a80 2024-04-24T09:16:14.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.842+0000 7f74a8f90740 10 client.40964 fetch_fsmap finished waiting for FSMap version 69 2024-04-24T09:16:14.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.842+0000 7f74a8f90740 10 client.40964 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-24T09:16:14.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.842+0000 7f74a8f90740 20 client.40964 populate_metadata read hostname 'smithi083' 2024-04-24T09:16:14.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.842+0000 7f74a8f90740 1 --2- 172.21.15.83:0/3959655411 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x55fa974f2920 0x55fa974f4d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:14.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.842+0000 7f74a8f90740 4 client.40964 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:16:14.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.842+0000 7f74a8f90740 1 -- 172.21.15.83:0/3959655411 --> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fa97430bf0 con 0x55fa974f2920 2024-04-24T09:16:14.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.843+0000 7f749ea00640 1 --2- 172.21.15.83:0/3959655411 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x55fa974f2920 0x55fa974f4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:14.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.844+0000 7f749ea00640 1 --2- 172.21.15.83:0/3959655411 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x55fa974f2920 0x55fa974f4d10 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6c6826b9a1edd590 server_cookie=102af852297e0ff0 in_seq=0 out_seq=0 2024-04-24T09:16:14.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.844+0000 7f7488c00640 10 client.40964 ms_handle_connect on v2:172.21.15.138:6836/900566754 2024-04-24T09:16:14.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.845+0000 7f7488c00640 1 -- 172.21.15.83:0/3959655411 <== mds.1 v2:172.21.15.138:6836/900566754 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fa97430bf0 con 0x55fa974f2920 2024-04-24T09:16:14.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.845+0000 7f7488c00640 10 client.40964 handle_command_reply: tid=0 2024-04-24T09:16:14.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.845+0000 7f74a8f90740 1 client.40964 shutdown 2024-04-24T09:16:14.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.846+0000 7f74a8f90740 1 -- 172.21.15.83:0/3959655411 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fa974c6a80 msgr2=0x55fa974ce570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:14.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.846+0000 7f74a8f90740 1 --2- 172.21.15.83:0/3959655411 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fa974c6a80 0x55fa974ce570 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f7490005ab0 tx=0x7f74900052e0 comp rx=0 tx=0).stop 2024-04-24T09:16:14.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.846+0000 7f74a8f90740 1 -- 172.21.15.83:0/3959655411 shutdown_connections 2024-04-24T09:16:14.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.846+0000 7f74a8f90740 1 --2- 172.21.15.83:0/3959655411 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa974ceab0 0x55fa974ee4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:14.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.846+0000 7f74a8f90740 1 --2- 172.21.15.83:0/3959655411 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fa974c6a80 0x55fa974ce570 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:14.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.846+0000 7f74a8f90740 1 --2- 172.21.15.83:0/3959655411 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x55fa974f2920 0x55fa974f4d10 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:14.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.846+0000 7f74a8f90740 1 --2- 172.21.15.83:0/3959655411 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa974c6160 0x55fa974ce030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:14.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.846+0000 7f74a8f90740 1 -- 172.21.15.83:0/3959655411 >> 172.21.15.83:0/3959655411 conn(0x55fa974c2480 msgr2=0x55fa974b8df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:14.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.846+0000 7f7488c00640 0 client.40964 ms_handle_reset on v2:172.21.15.138:6836/900566754 2024-04-24T09:16:14.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.847+0000 7f74a8f90740 1 -- 172.21.15.83:0/3959655411 shutdown_connections 2024-04-24T09:16:14.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.847+0000 7f74a8f90740 1 -- 172.21.15.83:0/3959655411 wait complete. 2024-04-24T09:16:14.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.847+0000 7f74a8f90740 20 client.40964 trim_cache size 0 max 16384 2024-04-24T09:16:14.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.871+0000 7f74a8f90740 1 Processor -- start 2024-04-24T09:16:14.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.872+0000 7f74a8f90740 1 -- start start 2024-04-24T09:16:14.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.872+0000 7f74a8f90740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa974c6160 0x55fa974cd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:14.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.872+0000 7f74a8f90740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa974c6a80 0x55fa974fa5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:14.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.872+0000 7f74a8f90740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fa974ceab0 0x55fa974fab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:14.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.872+0000 7f74a8f90740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7480010100 con 0x55fa974c6a80 2024-04-24T09:16:14.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.872+0000 7f74a8f90740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7480011ca0 con 0x55fa974ceab0 2024-04-24T09:16:14.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.872+0000 7f74a8f90740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f74800107d0 con 0x55fa974c6160 2024-04-24T09:16:14.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.873+0000 7f749ea00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa974c6160 0x55fa974cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:14.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.873+0000 7f749ea00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa974c6160 0x55fa974cd010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:35526/0 (socket says 172.21.15.83:35526) 2024-04-24T09:16:14.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.873+0000 7f749f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa974c6a80 0x55fa974fa5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:14.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.873+0000 7f749ea00640 1 -- 172.21.15.83:0/688750546 learned_addr learned my addr 172.21.15.83:0/688750546 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:16:14.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.873+0000 7f749ea00640 1 -- 172.21.15.83:0/688750546 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fa974ceab0 msgr2=0x55fa974fab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:14.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.873+0000 7f749ea00640 1 --2- 172.21.15.83:0/688750546 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fa974ceab0 0x55fa974fab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:14.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.873+0000 7f749ea00640 1 -- 172.21.15.83:0/688750546 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa974c6a80 msgr2=0x55fa974fa5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:14.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.873+0000 7f749ea00640 1 --2- 172.21.15.83:0/688750546 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa974c6a80 0x55fa974fa5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:14.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.873+0000 7f749ea00640 1 -- 172.21.15.83:0/688750546 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa974f5080 con 0x55fa974c6160 2024-04-24T09:16:14.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.873+0000 7f749ea00640 1 --2- 172.21.15.83:0/688750546 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa974c6160 0x55fa974cd010 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f748c019040 tx=0x7f748c00a920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=727ef1828e4954e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:14.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.874+0000 7f748b400640 1 -- 172.21.15.83:0/688750546 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f748c01b040 con 0x55fa974c6160 2024-04-24T09:16:14.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.874+0000 7f748b400640 1 -- 172.21.15.83:0/688750546 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f748c01c040 con 0x55fa974c6160 2024-04-24T09:16:14.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.874+0000 7f748b400640 1 -- 172.21.15.83:0/688750546 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f748c0286b0 con 0x55fa974c6160 2024-04-24T09:16:14.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.875+0000 7f74a8f90740 1 -- 172.21.15.83:0/688750546 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa974c6160 msgr2=0x55fa974cd010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:14.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.875+0000 7f74a8f90740 1 --2- 172.21.15.83:0/688750546 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa974c6160 0x55fa974cd010 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f748c019040 tx=0x7f748c00a920 comp rx=0 tx=0).stop 2024-04-24T09:16:14.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.876+0000 7f74a8f90740 1 -- 172.21.15.83:0/688750546 shutdown_connections 2024-04-24T09:16:14.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.876+0000 7f74a8f90740 1 --2- 172.21.15.83:0/688750546 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fa974ceab0 0x55fa974fab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:14.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.876+0000 7f74a8f90740 1 --2- 172.21.15.83:0/688750546 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa974c6a80 0x55fa974fa5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:14.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.876+0000 7f74a8f90740 1 --2- 172.21.15.83:0/688750546 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa974c6160 0x55fa974cd010 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:14.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.876+0000 7f74a8f90740 1 -- 172.21.15.83:0/688750546 >> 172.21.15.83:0/688750546 conn(0x55fa974c2480 msgr2=0x55fa974cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:14.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.876+0000 7f74a8f90740 1 -- 172.21.15.83:0/688750546 shutdown_connections 2024-04-24T09:16:14.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.876+0000 7f74a8f90740 1 -- 172.21.15.83:0/688750546 wait complete. 2024-04-24T09:16:14.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.877+0000 7f74a8f90740 1 -- start start 2024-04-24T09:16:14.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.877+0000 7f74a8f90740 1 Processor -- start 2024-04-24T09:16:14.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.877+0000 7f74a8f90740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa974c6160 0x55fa974ea900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:14.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.877+0000 7f74a8f90740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fa974c6a80 0x55fa974cab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:14.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.878+0000 7f74a8f90740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa974ceab0 0x55fa974cb190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:14.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.878+0000 7f74a8f90740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f74800115d0 con 0x55fa974c6160 2024-04-24T09:16:14.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.878+0000 7f74a8f90740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7480010f00 con 0x55fa974c6a80 2024-04-24T09:16:14.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.878+0000 7f74a8f90740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f748000e100 con 0x55fa974ceab0 2024-04-24T09:16:14.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.878+0000 7f74a8f90740 10 client.0 _ll_register_callbacks cb 0x7f7490004550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:16:14.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.878+0000 7f749ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa974c6160 0x55fa974ea900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:14.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.878+0000 7f749ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa974c6160 0x55fa974ea900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52026/0 (socket says 172.21.15.83:52026) 2024-04-24T09:16:14.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.878+0000 7f749ea00640 1 -- 172.21.15.83:0/3005330074 learned_addr learned my addr 172.21.15.83:0/3005330074 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:16:14.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.878+0000 7f749f400640 1 --2- 172.21.15.83:0/3005330074 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fa974c6a80 0x55fa974cab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:14.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.878+0000 7f749ea00640 1 --2- 172.21.15.83:0/3005330074 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa974ceab0 0x55fa974cb190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:14.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.879+0000 7f749ea00640 1 -- 172.21.15.83:0/3005330074 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa974ceab0 msgr2=0x55fa974cb190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:14.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.879+0000 7f749ea00640 1 --2- 172.21.15.83:0/3005330074 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa974ceab0 0x55fa974cb190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:14.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.879+0000 7f749ea00640 1 -- 172.21.15.83:0/3005330074 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fa974c6a80 msgr2=0x55fa974cab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:14.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.879+0000 7f749ea00640 1 --2- 172.21.15.83:0/3005330074 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fa974c6a80 0x55fa974cab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:14.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.879+0000 7f749ea00640 1 -- 172.21.15.83:0/3005330074 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa971ff7d0 con 0x55fa974c6160 2024-04-24T09:16:14.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.880+0000 7f749ea00640 1 --2- 172.21.15.83:0/3005330074 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa974c6160 0x55fa974ea900 secure :-1 s=READY pgs=2768 cs=0 l=1 rev1=1 crypto rx=0x7f748c004810 tx=0x7f748c006a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db07d85cd3ff930b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:14.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.880+0000 7f7488c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:16:14.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.880+0000 7f7488c00640 1 -- 172.21.15.83:0/3005330074 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f748c01b040 con 0x55fa974c6160 2024-04-24T09:16:14.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.881+0000 7f74a8f90740 1 -- 172.21.15.83:0/3005330074 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f748c01c040 con 0x55fa974c6160 2024-04-24T09:16:14.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.881+0000 7f7488c00640 1 -- 172.21.15.83:0/3005330074 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(132..132 src has 1..132) ==== 9583+0+0 (secure 0 0 0) 0x7f748c016820 con 0x55fa974c6160 2024-04-24T09:16:14.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.881+0000 7f7488c00640 1 -- 172.21.15.83:0/3005330074 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f748c03c050 con 0x55fa974c6160 2024-04-24T09:16:14.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.881+0000 7f74a8f90740 10 client.23898 fetch_fsmap learned FSMap version 69 2024-04-24T09:16:14.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.881+0000 7f74a8f90740 1 -- 172.21.15.83:0/3005330074 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x55fa974b9620 con 0x55fa974c6160 2024-04-24T09:16:14.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.882+0000 7f7488c00640 1 -- 172.21.15.83:0/3005330074 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f748c01c040 con 0x55fa974c6160 2024-04-24T09:16:14.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.882+0000 7f7488c00640 1 -- 172.21.15.83:0/3005330074 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 69) ==== 8370+0+0 (secure 0 0 0) 0x7f748c031020 con 0x55fa974c6160 2024-04-24T09:16:14.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.882+0000 7f74a8f90740 10 client.23898 fetch_fsmap finished waiting for FSMap version 69 2024-04-24T09:16:14.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.882+0000 7f74a8f90740 10 client.23898 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-24T09:16:14.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.882+0000 7f74a8f90740 20 client.23898 populate_metadata read hostname 'smithi083' 2024-04-24T09:16:14.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.882+0000 7f74a8f90740 1 --2- 172.21.15.83:0/3005330074 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x55fa974b9a20 0x55fa974bbe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:14.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.882+0000 7f74a8f90740 4 client.23898 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:16:14.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.882+0000 7f74a8f90740 1 -- 172.21.15.83:0/3005330074 --> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fa9742aa60 con 0x55fa974b9a20 2024-04-24T09:16:14.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.883+0000 7f749f400640 1 --2- 172.21.15.83:0/3005330074 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x55fa974b9a20 0x55fa974bbe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:14.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.885+0000 7f749f400640 1 --2- 172.21.15.83:0/3005330074 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x55fa974b9a20 0x55fa974bbe40 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d968604fc178369f server_cookie=435e7ff654cd2acf in_seq=0 out_seq=0 2024-04-24T09:16:14.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.885+0000 7f7488c00640 10 client.23898 ms_handle_connect on v2:172.21.15.138:6836/900566754 2024-04-24T09:16:14.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.885+0000 7f7488c00640 1 -- 172.21.15.83:0/3005330074 <== mds.1 v2:172.21.15.138:6836/900566754 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fa9742aa60 con 0x55fa974b9a20 2024-04-24T09:16:14.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.885+0000 7f7488c00640 10 client.23898 handle_command_reply: tid=0 2024-04-24T09:16:14.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.886+0000 7f74a8f90740 1 client.23898 shutdown 2024-04-24T09:16:14.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.886+0000 7f74a8f90740 1 -- 172.21.15.83:0/3005330074 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa974c6160 msgr2=0x55fa974ea900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:14.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.886+0000 7f74a8f90740 1 --2- 172.21.15.83:0/3005330074 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa974c6160 0x55fa974ea900 secure :-1 s=READY pgs=2768 cs=0 l=1 rev1=1 crypto rx=0x7f748c004810 tx=0x7f748c006a30 comp rx=0 tx=0).stop 2024-04-24T09:16:14.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.886+0000 7f74a8f90740 1 -- 172.21.15.83:0/3005330074 shutdown_connections 2024-04-24T09:16:14.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.886+0000 7f74a8f90740 1 --2- 172.21.15.83:0/3005330074 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x55fa974b9a20 0x55fa974bbe40 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:14.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.886+0000 7f74a8f90740 1 --2- 172.21.15.83:0/3005330074 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa974ceab0 0x55fa974cb190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:14.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.886+0000 7f7488c00640 0 client.23898 ms_handle_reset on v2:172.21.15.138:6836/900566754 2024-04-24T09:16:14.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.886+0000 7f74a8f90740 1 --2- 172.21.15.83:0/3005330074 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fa974c6a80 0x55fa974cab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:14.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.886+0000 7f74a8f90740 1 --2- 172.21.15.83:0/3005330074 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa974c6160 0x55fa974ea900 unknown :-1 s=CLOSED pgs=2768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:14.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.886+0000 7f74a8f90740 1 -- 172.21.15.83:0/3005330074 >> 172.21.15.83:0/3005330074 conn(0x55fa974c2480 msgr2=0x55fa974b8560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:14.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.887+0000 7f74a8f90740 1 -- 172.21.15.83:0/3005330074 shutdown_connections 2024-04-24T09:16:14.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.887+0000 7f74a8f90740 1 -- 172.21.15.83:0/3005330074 wait complete. 2024-04-24T09:16:14.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.887+0000 7f74a8f90740 20 client.23898 trim_cache size 0 max 16384 2024-04-24T09:16:14.888 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:16:14.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.888+0000 7f74a6000640 1 -- 172.21.15.83:0/1834767840 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f74840037c0 msgr2=0x7f7484005c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:14.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.888+0000 7f74a6000640 1 --2- 172.21.15.83:0/1834767840 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f74840037c0 0x7f7484005c80 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f749800c640 tx=0x7f7498013000 comp rx=0 tx=0).stop 2024-04-24T09:16:14.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.888+0000 7f74a6000640 1 -- 172.21.15.83:0/1834767840 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f74a010b180 msgr2=0x7f74a019afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:14.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.888+0000 7f74a6000640 1 --2- 172.21.15.83:0/1834767840 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f74a010b180 0x7f74a019afb0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f749800e960 tx=0x7f749800ee20 comp rx=0 tx=0).stop 2024-04-24T09:16:14.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.888+0000 7f74a6000640 1 -- 172.21.15.83:0/1834767840 shutdown_connections 2024-04-24T09:16:14.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.888+0000 7f74a6000640 1 --2- 172.21.15.83:0/1834767840 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f74840037c0 0x7f7484005c80 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:14.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.888+0000 7f74a6000640 1 --2- 172.21.15.83:0/1834767840 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f74a010b180 0x7f74a019afb0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:14.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.888+0000 7f74a6000640 1 --2- 172.21.15.83:0/1834767840 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74a0105420 0x7f74a0196c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:14.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.888+0000 7f74a6000640 1 --2- 172.21.15.83:0/1834767840 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f74a0100b60 0x7f74a01966f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:14.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.888+0000 7f74a6000640 1 -- 172.21.15.83:0/1834767840 >> 172.21.15.83:0/1834767840 conn(0x7f74a0077060 msgr2=0x7f74a010c770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:14.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.888+0000 7f74a6000640 1 -- 172.21.15.83:0/1834767840 shutdown_connections 2024-04-24T09:16:14.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:14.888+0000 7f74a6000640 1 -- 172.21.15.83:0/1834767840 wait complete. 2024-04-24T09:16:14.910 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T09:16:15.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.152+0000 7f39fda00640 1 -- 172.21.15.83:0/819982057 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39f810e740 msgr2=0x7f39f8110b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:15.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.152+0000 7f39fda00640 1 --2- 172.21.15.83:0/819982057 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39f810e740 0x7f39f8110b50 secure :-1 s=READY pgs=2769 cs=0 l=1 rev1=1 crypto rx=0x7f39f0009ca0 tx=0x7f39f002a950 comp rx=0 tx=0).stop 2024-04-24T09:16:15.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.152+0000 7f39fda00640 1 -- 172.21.15.83:0/819982057 shutdown_connections 2024-04-24T09:16:15.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.152+0000 7f39fda00640 1 --2- 172.21.15.83:0/819982057 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39f810e740 0x7f39f8110b50 unknown :-1 s=CLOSED pgs=2769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:15.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.152+0000 7f39fda00640 1 --2- 172.21.15.83:0/819982057 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39f8074cf0 0x7f39f810e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:15.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.152+0000 7f39fda00640 1 --2- 172.21.15.83:0/819982057 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f39f80743d0 0x7f39f80747b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:15.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.152+0000 7f39fda00640 1 -- 172.21.15.83:0/819982057 >> 172.21.15.83:0/819982057 conn(0x7f39f80fd9d0 msgr2=0x7f39f80ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:15.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.152+0000 7f39fda00640 1 -- 172.21.15.83:0/819982057 shutdown_connections 2024-04-24T09:16:15.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.152+0000 7f39fda00640 1 -- 172.21.15.83:0/819982057 wait complete. 2024-04-24T09:16:15.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.153+0000 7f39fda00640 1 Processor -- start 2024-04-24T09:16:15.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.154+0000 7f39fda00640 1 -- start start 2024-04-24T09:16:15.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.154+0000 7f39fda00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39f80743d0 0x7f39f81a05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:15.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.154+0000 7f39fda00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39f8074cf0 0x7f39f81a0b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:15.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.154+0000 7f39fda00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f39f810e740 0x7f39f819a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:15.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.154+0000 7f39fda00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f39f8114360 con 0x7f39f8074cf0 2024-04-24T09:16:15.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.154+0000 7f39fda00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f39f81141e0 con 0x7f39f80743d0 2024-04-24T09:16:15.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.154+0000 7f39fda00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f39f81144e0 con 0x7f39f810e740 2024-04-24T09:16:15.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.155+0000 7f39f7e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39f80743d0 0x7f39f81a05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:15.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.155+0000 7f39f7e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39f80743d0 0x7f39f81a05e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:51448/0 (socket says 172.21.15.83:51448) 2024-04-24T09:16:15.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.155+0000 7f39f7e00640 1 -- 172.21.15.83:0/1327200262 learned_addr learned my addr 172.21.15.83:0/1327200262 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:16:15.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.155+0000 7f39f7400640 1 --2- 172.21.15.83:0/1327200262 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39f8074cf0 0x7f39f81a0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:15.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.155+0000 7f39fd000640 1 --2- 172.21.15.83:0/1327200262 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f39f810e740 0x7f39f819a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:15.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.155+0000 7f39f7e00640 1 -- 172.21.15.83:0/1327200262 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f39f810e740 msgr2=0x7f39f819a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:15.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.155+0000 7f39f7e00640 1 --2- 172.21.15.83:0/1327200262 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f39f810e740 0x7f39f819a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:15.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.155+0000 7f39f7e00640 1 -- 172.21.15.83:0/1327200262 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39f8074cf0 msgr2=0x7f39f81a0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:15.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.155+0000 7f39f7e00640 1 --2- 172.21.15.83:0/1327200262 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39f8074cf0 0x7f39f81a0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:15.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.155+0000 7f39f7e00640 1 -- 172.21.15.83:0/1327200262 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39f819aeb0 con 0x7f39f80743d0 2024-04-24T09:16:15.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.155+0000 7f39fd000640 1 --2- 172.21.15.83:0/1327200262 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f39f810e740 0x7f39f819a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:16:15.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.155+0000 7f39f7400640 1 --2- 172.21.15.83:0/1327200262 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39f8074cf0 0x7f39f81a0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:16:15.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.156+0000 7f39f7e00640 1 --2- 172.21.15.83:0/1327200262 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39f80743d0 0x7f39f81a05e0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f39e800d980 tx=0x7f39e800de40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:15.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.157+0000 7f39f4c00640 1 -- 172.21.15.83:0/1327200262 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39e8017030 con 0x7f39f80743d0 2024-04-24T09:16:15.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.157+0000 7f39f4c00640 1 -- 172.21.15.83:0/1327200262 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f39e8004020 con 0x7f39f80743d0 2024-04-24T09:16:15.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.157+0000 7f39fda00640 1 -- 172.21.15.83:0/1327200262 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39f819b1a0 con 0x7f39f80743d0 2024-04-24T09:16:15.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.157+0000 7f39fda00640 1 -- 172.21.15.83:0/1327200262 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39f806bc00 con 0x7f39f80743d0 2024-04-24T09:16:15.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.157+0000 7f39f4c00640 1 -- 172.21.15.83:0/1327200262 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39e8005640 con 0x7f39f80743d0 2024-04-24T09:16:15.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.162+0000 7f3a01e07740 1 Processor -- start 2024-04-24T09:16:15.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.163+0000 7f3a01e07740 1 -- start start 2024-04-24T09:16:15.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.163+0000 7f39f4c00640 1 -- 172.21.15.83:0/1327200262 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f39e80041d0 con 0x7f39f80743d0 2024-04-24T09:16:15.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.163+0000 7f3a01e07740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56412b5f5870 0x56412b5f5c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:15.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.163+0000 7f3a01e07740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56412b5f6220 0x56412b608e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:15.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.163+0000 7f3a01e07740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56412b609340 0x56412b60b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:15.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.163+0000 7f3a01e07740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56412b55b1d0 con 0x56412b5f5870 2024-04-24T09:16:15.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.163+0000 7f3a01e07740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56412b501450 con 0x56412b609340 2024-04-24T09:16:15.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.163+0000 7f3a01e07740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56412b557190 con 0x56412b5f6220 2024-04-24T09:16:15.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.164+0000 7f39fd000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56412b5f5870 0x56412b5f5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:15.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.164+0000 7f39fd000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56412b5f5870 0x56412b5f5c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52038/0 (socket says 172.21.15.83:52038) 2024-04-24T09:16:15.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.164+0000 7f39fd000640 1 -- 172.21.15.83:0/1571537731 learned_addr learned my addr 172.21.15.83:0/1571537731 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:16:15.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.166+0000 7f39f4c00640 1 --2- 172.21.15.83:0/1327200262 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f39dc0a3fc0 0x7f39dc0a6480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:15.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.166+0000 7f39f4c00640 1 -- 172.21.15.83:0/1327200262 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 9583+0+0 (secure 0 0 0) 0x7f39e80d0630 con 0x7f39f80743d0 2024-04-24T09:16:15.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.166+0000 7f39f7e00640 1 --2- 172.21.15.83:0/1571537731 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56412b5f6220 0x56412b608e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:15.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.167+0000 7f39f7400640 1 --2- 172.21.15.83:0/1571537731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56412b609340 0x56412b60b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:15.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.167+0000 7f39fd000640 1 --2- 172.21.15.83:0/1327200262 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f39dc0a3fc0 0x7f39dc0a6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:15.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.168+0000 7f39fd000640 1 -- 172.21.15.83:0/1571537731 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56412b5f6220 msgr2=0x56412b608e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:15.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.168+0000 7f39fd000640 1 --2- 172.21.15.83:0/1571537731 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56412b5f6220 0x56412b608e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:15.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.168+0000 7f39fd000640 1 -- 172.21.15.83:0/1571537731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56412b609340 msgr2=0x56412b60b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:15.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.168+0000 7f39fd000640 1 --2- 172.21.15.83:0/1571537731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56412b609340 0x56412b60b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:15.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.168+0000 7f39fd000640 1 -- 172.21.15.83:0/1571537731 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56412b5be310 con 0x56412b5f5870 2024-04-24T09:16:15.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.169+0000 7f39fd000640 1 --2- 172.21.15.83:0/1571537731 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56412b5f5870 0x56412b5f5c50 secure :-1 s=READY pgs=2770 cs=0 l=1 rev1=1 crypto rx=0x7f39f0004af0 tx=0x7f39f002f640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c5fe786a7341551 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:15.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.169+0000 7f39fd000640 1 --2- 172.21.15.83:0/1327200262 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f39dc0a3fc0 0x7f39dc0a6480 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f39f0005fa0 tx=0x7f39f0046000 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:15.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.172+0000 7f39ed600640 1 -- 172.21.15.83:0/1571537731 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39f004d070 con 0x56412b5f5870 2024-04-24T09:16:15.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.172+0000 7f39ed600640 1 -- 172.21.15.83:0/1571537731 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f39f002fec0 con 0x56412b5f5870 2024-04-24T09:16:15.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.172+0000 7f39ed600640 1 -- 172.21.15.83:0/1571537731 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39f004d070 con 0x56412b5f5870 2024-04-24T09:16:15.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.174+0000 7f3a01e07740 1 -- 172.21.15.83:0/1571537731 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56412b5f5870 msgr2=0x56412b5f5c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:15.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.174+0000 7f3a01e07740 1 --2- 172.21.15.83:0/1571537731 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56412b5f5870 0x56412b5f5c50 secure :-1 s=READY pgs=2770 cs=0 l=1 rev1=1 crypto rx=0x7f39f0004af0 tx=0x7f39f002f640 comp rx=0 tx=0).stop 2024-04-24T09:16:15.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.175+0000 7f3a01e07740 1 -- 172.21.15.83:0/1571537731 shutdown_connections 2024-04-24T09:16:15.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.175+0000 7f3a01e07740 1 --2- 172.21.15.83:0/1571537731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56412b609340 0x56412b60b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:15.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.175+0000 7f3a01e07740 1 --2- 172.21.15.83:0/1571537731 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56412b5f6220 0x56412b608e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:15.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.175+0000 7f3a01e07740 1 --2- 172.21.15.83:0/1571537731 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56412b5f5870 0x56412b5f5c50 unknown :-1 s=CLOSED pgs=2770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:15.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.175+0000 7f3a01e07740 1 -- 172.21.15.83:0/1571537731 >> 172.21.15.83:0/1571537731 conn(0x56412b5f1e60 msgr2=0x56412b5f2240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:15.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.176+0000 7f3a01e07740 1 -- 172.21.15.83:0/1571537731 shutdown_connections 2024-04-24T09:16:15.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.176+0000 7f3a01e07740 1 -- 172.21.15.83:0/1571537731 wait complete. 2024-04-24T09:16:15.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.177+0000 7f3a01e07740 1 -- start start 2024-04-24T09:16:15.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.177+0000 7f3a01e07740 1 Processor -- start 2024-04-24T09:16:15.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.178+0000 7f3a01e07740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56412b5f5870 0x56412b608d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:15.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.178+0000 7f3a01e07740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56412b5f6220 0x56412b606080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:15.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.178+0000 7f3a01e07740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56412b609340 0x56412b6067c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:15.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.178+0000 7f3a01e07740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56412b60c140 con 0x56412b609340 2024-04-24T09:16:15.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.178+0000 7f3a01e07740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56412b514810 con 0x56412b5f5870 2024-04-24T09:16:15.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.178+0000 7f3a01e07740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56412b60c2c0 con 0x56412b5f6220 2024-04-24T09:16:15.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.178+0000 7f3a01e07740 10 client.0 _ll_register_callbacks cb 0x56412b4e2140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:16:15.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.179+0000 7f39fd000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56412b609340 0x56412b6067c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:15.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.179+0000 7f39f7e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56412b5f5870 0x56412b608d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:15.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.179+0000 7f39f7e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56412b5f5870 0x56412b608d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:51468/0 (socket says 172.21.15.83:51468) 2024-04-24T09:16:15.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.179+0000 7f39f7e00640 1 -- 172.21.15.83:0/2375517114 learned_addr learned my addr 172.21.15.83:0/2375517114 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:16:15.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.179+0000 7f39fd000640 1 -- 172.21.15.83:0/2375517114 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56412b5f6220 msgr2=0x56412b606080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:15.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.179+0000 7f39fd000640 1 --2- 172.21.15.83:0/2375517114 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56412b5f6220 0x56412b606080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:15.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.179+0000 7f39fd000640 1 -- 172.21.15.83:0/2375517114 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56412b5f5870 msgr2=0x56412b608d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:15.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.179+0000 7f39fd000640 1 --2- 172.21.15.83:0/2375517114 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56412b5f5870 0x56412b608d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:15.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.179+0000 7f39fd000640 1 -- 172.21.15.83:0/2375517114 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56412b5253f0 con 0x56412b609340 2024-04-24T09:16:15.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.179+0000 7f39f7e00640 1 --2- 172.21.15.83:0/2375517114 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56412b5f5870 0x56412b608d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:16:15.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.180+0000 7f39fd000640 1 --2- 172.21.15.83:0/2375517114 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56412b609340 0x56412b6067c0 secure :-1 s=READY pgs=2771 cs=0 l=1 rev1=1 crypto rx=0x7f39f002ae70 tx=0x7f39f0004680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=923f1f051320e800 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:15.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.180+0000 7f39e2000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:16:15.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.181+0000 7f39e2000640 1 -- 172.21.15.83:0/2375517114 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39f0055020 con 0x56412b609340 2024-04-24T09:16:15.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.181+0000 7f39e2000640 1 -- 172.21.15.83:0/2375517114 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(132..132 src has 1..132) ==== 9583+0+0 (secure 0 0 0) 0x7f39f0039450 con 0x56412b609340 2024-04-24T09:16:15.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.181+0000 7f3a01e07740 1 -- 172.21.15.83:0/2375517114 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39f002fec0 con 0x56412b609340 2024-04-24T09:16:15.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.182+0000 7f39e2000640 1 -- 172.21.15.83:0/2375517114 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f39f004d040 con 0x56412b609340 2024-04-24T09:16:15.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.182+0000 7f3a01e07740 10 client.23910 fetch_fsmap learned FSMap version 69 2024-04-24T09:16:15.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.182+0000 7f3a01e07740 1 -- 172.21.15.83:0/2375517114 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x56412b26d3f0 con 0x56412b609340 2024-04-24T09:16:15.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.183+0000 7f39e2000640 1 -- 172.21.15.83:0/2375517114 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f39f002fec0 con 0x56412b609340 2024-04-24T09:16:15.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.184+0000 7f39e2000640 1 -- 172.21.15.83:0/2375517114 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 69) ==== 8370+0+0 (secure 0 0 0) 0x7f39f0030440 con 0x56412b609340 2024-04-24T09:16:15.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.184+0000 7f3a01e07740 10 client.23910 fetch_fsmap finished waiting for FSMap version 69 2024-04-24T09:16:15.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.184+0000 7f3a01e07740 10 client.23910 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:16:15.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.184+0000 7f3a01e07740 20 client.23910 populate_metadata read hostname 'smithi083' 2024-04-24T09:16:15.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.185+0000 7f3a01e07740 1 --2- 172.21.15.83:0/2375517114 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56412b6191a0 0x56412b61b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:15.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.185+0000 7f3a01e07740 4 client.23910 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:16:15.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.185+0000 7f3a01e07740 1 -- 172.21.15.83:0/2375517114 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56412b568bf0 con 0x56412b6191a0 2024-04-24T09:16:15.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.186+0000 7f39f7e00640 1 --2- 172.21.15.83:0/2375517114 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56412b6191a0 0x56412b61b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:15.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.187+0000 7f39e2000640 10 client.23910 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:16:15.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.187+0000 7f39f7e00640 1 --2- 172.21.15.83:0/2375517114 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56412b6191a0 0x56412b61b590 crc :-1 s=READY pgs=601 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a95a43e924da9741 server_cookie=3c9691748ff5ac45 in_seq=0 out_seq=0 2024-04-24T09:16:15.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.188+0000 7f39e2000640 1 -- 172.21.15.83:0/2375517114 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56412b568bf0 con 0x56412b6191a0 2024-04-24T09:16:15.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.188+0000 7f39e2000640 10 client.23910 handle_command_reply: tid=0 2024-04-24T09:16:15.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.188+0000 7f3a01e07740 1 client.23910 shutdown 2024-04-24T09:16:15.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.190+0000 7f3a01e07740 1 -- 172.21.15.83:0/2375517114 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56412b609340 msgr2=0x56412b6067c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:15.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.190+0000 7f3a01e07740 1 --2- 172.21.15.83:0/2375517114 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56412b609340 0x56412b6067c0 secure :-1 s=READY pgs=2771 cs=0 l=1 rev1=1 crypto rx=0x7f39f002ae70 tx=0x7f39f0004680 comp rx=0 tx=0).stop 2024-04-24T09:16:15.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.190+0000 7f3a01e07740 1 -- 172.21.15.83:0/2375517114 shutdown_connections 2024-04-24T09:16:15.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.190+0000 7f3a01e07740 1 --2- 172.21.15.83:0/2375517114 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56412b6191a0 0x56412b61b590 crc :-1 s=READY pgs=601 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:15.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.190+0000 7f3a01e07740 1 --2- 172.21.15.83:0/2375517114 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56412b609340 0x56412b6067c0 unknown :-1 s=CLOSED pgs=2771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:15.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.190+0000 7f3a01e07740 1 --2- 172.21.15.83:0/2375517114 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56412b5f6220 0x56412b606080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:15.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.190+0000 7f3a01e07740 1 --2- 172.21.15.83:0/2375517114 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56412b5f5870 0x56412b608d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:15.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.190+0000 7f3a01e07740 1 -- 172.21.15.83:0/2375517114 >> 172.21.15.83:0/2375517114 conn(0x56412b5f1e60 msgr2=0x56412b5f7cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:15.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.191+0000 7f39e2000640 0 client.23910 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:16:15.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.191+0000 7f3a01e07740 1 -- 172.21.15.83:0/2375517114 shutdown_connections 2024-04-24T09:16:15.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.191+0000 7f3a01e07740 1 -- 172.21.15.83:0/2375517114 wait complete. 2024-04-24T09:16:15.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.192+0000 7f3a01e07740 20 client.23910 trim_cache size 0 max 16384 2024-04-24T09:16:15.217 INFO:tasks.workunit.client.0.smithi083.stdout: 10 79043 6.53 MB/sec execute 526 sec latency 96.563 ms 2024-04-24T09:16:15.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.220+0000 7f3a01e07740 1 Processor -- start 2024-04-24T09:16:15.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.221+0000 7f3a01e07740 1 -- start start 2024-04-24T09:16:15.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.222+0000 7f3a01e07740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56412b5f5870 0x56412b6150a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:15.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.223+0000 7f3a01e07740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56412b5f6220 0x56412b6088c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:15.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.223+0000 7f3a01e07740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56412b609340 0x56412b608e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:15.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.223+0000 7f3a01e07740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f39d8010100 con 0x56412b5f5870 2024-04-24T09:16:15.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.223+0000 7f3a01e07740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f39d8011ca0 con 0x56412b609340 2024-04-24T09:16:15.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.223+0000 7f3a01e07740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f39d80107d0 con 0x56412b5f6220 2024-04-24T09:16:15.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.223+0000 7f39f7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56412b5f5870 0x56412b6150a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:15.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.223+0000 7f39f7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56412b5f5870 0x56412b6150a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52056/0 (socket says 172.21.15.83:52056) 2024-04-24T09:16:15.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.223+0000 7f39f7e00640 1 -- 172.21.15.83:0/31242073 learned_addr learned my addr 172.21.15.83:0/31242073 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:16:15.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.223+0000 7f39fd000640 1 --2- 172.21.15.83:0/31242073 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56412b609340 0x56412b608e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:15.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.223+0000 7f39f7400640 1 --2- 172.21.15.83:0/31242073 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56412b5f6220 0x56412b6088c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:15.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.224+0000 7f39f7e00640 1 -- 172.21.15.83:0/31242073 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56412b5f6220 msgr2=0x56412b6088c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:15.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.224+0000 7f39f7e00640 1 --2- 172.21.15.83:0/31242073 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56412b5f6220 0x56412b6088c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:15.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.224+0000 7f39f7e00640 1 -- 172.21.15.83:0/31242073 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56412b609340 msgr2=0x56412b608e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:15.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.224+0000 7f39f7e00640 1 --2- 172.21.15.83:0/31242073 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56412b609340 0x56412b608e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:15.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.224+0000 7f39f7e00640 1 -- 172.21.15.83:0/31242073 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56412b3377d0 con 0x56412b5f5870 2024-04-24T09:16:15.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.224+0000 7f39f7e00640 1 --2- 172.21.15.83:0/31242073 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56412b5f5870 0x56412b6150a0 secure :-1 s=READY pgs=2772 cs=0 l=1 rev1=1 crypto rx=0x7f39e80985b0 tx=0x7f39e8098520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54ffb3b2eadfe887 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:15.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.225+0000 7f39ecc00640 1 -- 172.21.15.83:0/31242073 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39e80a0500 con 0x56412b5f5870 2024-04-24T09:16:15.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.225+0000 7f39ecc00640 1 -- 172.21.15.83:0/31242073 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f39e80a4dc0 con 0x56412b5f5870 2024-04-24T09:16:15.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.225+0000 7f39ecc00640 1 -- 172.21.15.83:0/31242073 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39e80a4930 con 0x56412b5f5870 2024-04-24T09:16:15.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.226+0000 7f3a01e07740 1 -- 172.21.15.83:0/31242073 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56412b5f5870 msgr2=0x56412b6150a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:15.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.226+0000 7f3a01e07740 1 --2- 172.21.15.83:0/31242073 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56412b5f5870 0x56412b6150a0 secure :-1 s=READY pgs=2772 cs=0 l=1 rev1=1 crypto rx=0x7f39e80985b0 tx=0x7f39e8098520 comp rx=0 tx=0).stop 2024-04-24T09:16:15.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.226+0000 7f3a01e07740 1 -- 172.21.15.83:0/31242073 shutdown_connections 2024-04-24T09:16:15.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.226+0000 7f3a01e07740 1 --2- 172.21.15.83:0/31242073 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56412b609340 0x56412b608e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:15.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.226+0000 7f3a01e07740 1 --2- 172.21.15.83:0/31242073 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56412b5f6220 0x56412b6088c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:15.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.226+0000 7f3a01e07740 1 --2- 172.21.15.83:0/31242073 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56412b5f5870 0x56412b6150a0 unknown :-1 s=CLOSED pgs=2772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:15.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.226+0000 7f3a01e07740 1 -- 172.21.15.83:0/31242073 >> 172.21.15.83:0/31242073 conn(0x56412b5f1e60 msgr2=0x56412b6013e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:15.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.227+0000 7f3a01e07740 1 -- 172.21.15.83:0/31242073 shutdown_connections 2024-04-24T09:16:15.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.227+0000 7f3a01e07740 1 -- 172.21.15.83:0/31242073 wait complete. 2024-04-24T09:16:15.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.227+0000 7f3a01e07740 1 -- start start 2024-04-24T09:16:15.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.228+0000 7f3a01e07740 1 Processor -- start 2024-04-24T09:16:15.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.228+0000 7f3a01e07740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56412b5f5870 0x56412b6115b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:15.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.228+0000 7f3a01e07740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56412b5f6220 0x56412b611af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:15.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.228+0000 7f3a01e07740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56412b609340 0x56412b6022f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:15.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.228+0000 7f3a01e07740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f39d80115d0 con 0x56412b5f5870 2024-04-24T09:16:15.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.228+0000 7f3a01e07740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f39d8010f00 con 0x56412b609340 2024-04-24T09:16:15.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.228+0000 7f3a01e07740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f39d800e100 con 0x56412b5f6220 2024-04-24T09:16:15.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.228+0000 7f3a01e07740 10 client.0 _ll_register_callbacks cb 0x7f39f004b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:16:15.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.229+0000 7f39fd000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56412b609340 0x56412b6022f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:15.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.229+0000 7f39fd000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56412b609340 0x56412b6022f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:51476/0 (socket says 172.21.15.83:51476) 2024-04-24T09:16:15.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.229+0000 7f39fd000640 1 -- 172.21.15.83:0/3988261831 learned_addr learned my addr 172.21.15.83:0/3988261831 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:16:15.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.229+0000 7f39f7400640 1 --2- 172.21.15.83:0/3988261831 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56412b5f6220 0x56412b611af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:15.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.229+0000 7f39f7e00640 1 --2- 172.21.15.83:0/3988261831 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56412b5f5870 0x56412b6115b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:15.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.230+0000 7f39f7400640 1 -- 172.21.15.83:0/3988261831 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56412b609340 msgr2=0x56412b6022f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:15.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.230+0000 7f39f7400640 1 --2- 172.21.15.83:0/3988261831 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56412b609340 0x56412b6022f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:15.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.230+0000 7f39f7400640 1 -- 172.21.15.83:0/3988261831 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56412b5f5870 msgr2=0x56412b6115b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:15.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.230+0000 7f39f7400640 1 --2- 172.21.15.83:0/3988261831 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56412b5f5870 0x56412b6115b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:15.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.230+0000 7f39f7400640 1 -- 172.21.15.83:0/3988261831 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56412b602960 con 0x56412b5f6220 2024-04-24T09:16:15.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.230+0000 7f39f7e00640 1 --2- 172.21.15.83:0/3988261831 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56412b5f5870 0x56412b6115b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:16:15.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.230+0000 7f39f7400640 1 --2- 172.21.15.83:0/3988261831 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56412b5f6220 0x56412b611af0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x56412b60ba20 tx=0x7f39e4013830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e253aaa5bc85bea1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:15.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.230+0000 7f39e2000640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-04-24T09:16:15.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.231+0000 7f39e2000640 1 -- 172.21.15.83:0/3988261831 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39e4004260 con 0x56412b5f6220 2024-04-24T09:16:15.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.231+0000 7f39e2000640 1 -- 172.21.15.83:0/3988261831 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(132..132 src has 1..132) ==== 9583+0+0 (secure 0 0 0) 0x7f39e4004400 con 0x56412b5f6220 2024-04-24T09:16:15.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.231+0000 7f3a01e07740 1 -- 172.21.15.83:0/3988261831 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39e80a4dc0 con 0x56412b5f6220 2024-04-24T09:16:15.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.232+0000 7f39e2000640 1 -- 172.21.15.83:0/3988261831 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f39e40035f0 con 0x56412b5f6220 2024-04-24T09:16:15.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.232+0000 7f3a01e07740 10 client.30679 fetch_fsmap learned FSMap version 69 2024-04-24T09:16:15.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.232+0000 7f3a01e07740 1 -- 172.21.15.83:0/3988261831 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x56412b603ca0 con 0x56412b5f6220 2024-04-24T09:16:15.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.233+0000 7f39e2000640 1 -- 172.21.15.83:0/3988261831 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f39e80a4dc0 con 0x56412b5f6220 2024-04-24T09:16:15.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.233+0000 7f39e2000640 1 -- 172.21.15.83:0/3988261831 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 69) ==== 8370+0+0 (secure 0 0 0) 0x7f39e4027ae0 con 0x56412b5f6220 2024-04-24T09:16:15.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.233+0000 7f3a01e07740 10 client.30679 fetch_fsmap finished waiting for FSMap version 69 2024-04-24T09:16:15.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.233+0000 7f3a01e07740 10 client.30679 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:16:15.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.233+0000 7f3a01e07740 20 client.30679 populate_metadata read hostname 'smithi083' 2024-04-24T09:16:15.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.233+0000 7f3a01e07740 1 --2- 172.21.15.83:0/3988261831 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56412b6805d0 0x56412b6829f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:15.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.233+0000 7f3a01e07740 4 client.30679 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:16:15.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.233+0000 7f3a01e07740 1 -- 172.21.15.83:0/3988261831 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56412b562a60 con 0x56412b6805d0 2024-04-24T09:16:15.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.234+0000 7f39f7e00640 1 --2- 172.21.15.83:0/3988261831 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56412b6805d0 0x56412b6829f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:15.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.235+0000 7f39f7e00640 1 --2- 172.21.15.83:0/3988261831 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56412b6805d0 0x56412b6829f0 crc :-1 s=READY pgs=602 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3019af8e8c435f54 server_cookie=ee7e732ca580d722 in_seq=0 out_seq=0 2024-04-24T09:16:15.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.235+0000 7f39e2000640 10 client.30679 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:16:15.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.236+0000 7f39e2000640 1 -- 172.21.15.83:0/3988261831 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56412b562a60 con 0x56412b6805d0 2024-04-24T09:16:15.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.236+0000 7f39e2000640 10 client.30679 handle_command_reply: tid=0 2024-04-24T09:16:15.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.236+0000 7f3a01e07740 1 client.30679 shutdown 2024-04-24T09:16:15.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.236+0000 7f3a01e07740 1 -- 172.21.15.83:0/3988261831 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56412b5f6220 msgr2=0x56412b611af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:15.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.236+0000 7f3a01e07740 1 --2- 172.21.15.83:0/3988261831 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56412b5f6220 0x56412b611af0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x56412b60ba20 tx=0x7f39e4013830 comp rx=0 tx=0).stop 2024-04-24T09:16:15.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.236+0000 7f3a01e07740 1 -- 172.21.15.83:0/3988261831 shutdown_connections 2024-04-24T09:16:15.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.236+0000 7f3a01e07740 1 --2- 172.21.15.83:0/3988261831 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56412b6805d0 0x56412b6829f0 crc :-1 s=READY pgs=602 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:15.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.236+0000 7f3a01e07740 1 --2- 172.21.15.83:0/3988261831 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56412b609340 0x56412b6022f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:15.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.236+0000 7f3a01e07740 1 --2- 172.21.15.83:0/3988261831 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56412b5f6220 0x56412b611af0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:15.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.236+0000 7f3a01e07740 1 --2- 172.21.15.83:0/3988261831 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56412b5f5870 0x56412b6115b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:15.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.236+0000 7f3a01e07740 1 -- 172.21.15.83:0/3988261831 >> 172.21.15.83:0/3988261831 conn(0x56412b5f1e60 msgr2=0x56412b5f7420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:15.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.236+0000 7f39e2000640 0 client.30679 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:16:15.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.237+0000 7f3a01e07740 1 -- 172.21.15.83:0/3988261831 shutdown_connections 2024-04-24T09:16:15.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.237+0000 7f3a01e07740 1 -- 172.21.15.83:0/3988261831 wait complete. 2024-04-24T09:16:15.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.237+0000 7f3a01e07740 20 client.30679 trim_cache size 0 max 16384 2024-04-24T09:16:15.238 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:16:15.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.238+0000 7f39fda00640 1 -- 172.21.15.83:0/1327200262 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f39dc0a3fc0 msgr2=0x7f39dc0a6480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:15.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.238+0000 7f39fda00640 1 --2- 172.21.15.83:0/1327200262 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f39dc0a3fc0 0x7f39dc0a6480 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f39f0005fa0 tx=0x7f39f0046000 comp rx=0 tx=0).stop 2024-04-24T09:16:15.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.238+0000 7f39fda00640 1 -- 172.21.15.83:0/1327200262 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39f80743d0 msgr2=0x7f39f81a05e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:15.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.238+0000 7f39fda00640 1 --2- 172.21.15.83:0/1327200262 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39f80743d0 0x7f39f81a05e0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f39e800d980 tx=0x7f39e800de40 comp rx=0 tx=0).stop 2024-04-24T09:16:15.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.239+0000 7f39fda00640 1 -- 172.21.15.83:0/1327200262 shutdown_connections 2024-04-24T09:16:15.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.239+0000 7f39fda00640 1 --2- 172.21.15.83:0/1327200262 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f39dc0a3fc0 0x7f39dc0a6480 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:15.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.239+0000 7f39fda00640 1 --2- 172.21.15.83:0/1327200262 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f39f810e740 0x7f39f819a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:15.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.239+0000 7f39fda00640 1 --2- 172.21.15.83:0/1327200262 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39f8074cf0 0x7f39f81a0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:15.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.239+0000 7f39fda00640 1 --2- 172.21.15.83:0/1327200262 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39f80743d0 0x7f39f81a05e0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:15.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.239+0000 7f39fda00640 1 -- 172.21.15.83:0/1327200262 >> 172.21.15.83:0/1327200262 conn(0x7f39f80fd9d0 msgr2=0x7f39f80fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:15.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.239+0000 7f39fda00640 1 -- 172.21.15.83:0/1327200262 shutdown_connections 2024-04-24T09:16:15.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:15.239+0000 7f39fda00640 1 -- 172.21.15.83:0/1327200262 wait complete. 2024-04-24T09:16:16.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:16:15 smithi138 ceph-mon[31441]: osdmap e132: 12 total, 12 up, 12 in 2024-04-24T09:16:16.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:16:15 smithi138 ceph-mon[31441]: pgmap v780: 129 pgs: 129 active+clean; 1.2 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 6.7 MiB/s wr, 362 op/s 2024-04-24T09:16:16.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:16:15 smithi189 ceph-mon[31791]: osdmap e132: 12 total, 12 up, 12 in 2024-04-24T09:16:16.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:16:15 smithi189 ceph-mon[31791]: pgmap v780: 129 pgs: 129 active+clean; 1.2 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 6.7 MiB/s wr, 362 op/s 2024-04-24T09:16:16.218 INFO:tasks.workunit.client.0.smithi083.stdout: 10 79149 6.53 MB/sec execute 527 sec latency 107.937 ms 2024-04-24T09:16:16.268 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T09:16:16.268 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T09:16:16.268 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T09:16:16.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:16:15 smithi083 ceph-mon[27907]: osdmap e132: 12 total, 12 up, 12 in 2024-04-24T09:16:16.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:16:15 smithi083 ceph-mon[27907]: pgmap v780: 129 pgs: 129 active+clean; 1.2 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 6.7 MiB/s wr, 362 op/s 2024-04-24T09:16:16.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.511+0000 7f16e0600640 1 -- 172.21.15.83:0/549791414 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f16d8074bf0 msgr2=0x7f16d810e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:16.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.511+0000 7f16e0600640 1 --2- 172.21.15.83:0/549791414 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f16d8074bf0 0x7f16d810e3a0 secure :-1 s=READY pgs=2773 cs=0 l=1 rev1=1 crypto rx=0x7f16c4009900 tx=0x7f16c402a480 comp rx=0 tx=0).stop 2024-04-24T09:16:16.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.512+0000 7f16e0600640 1 -- 172.21.15.83:0/549791414 shutdown_connections 2024-04-24T09:16:16.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.512+0000 7f16e0600640 1 --2- 172.21.15.83:0/549791414 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16d810e8e0 0x7f16d8110cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:16.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.512+0000 7f16e0600640 1 --2- 172.21.15.83:0/549791414 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f16d8074bf0 0x7f16d810e3a0 unknown :-1 s=CLOSED pgs=2773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:16.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.512+0000 7f16e0600640 1 --2- 172.21.15.83:0/549791414 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f16d80742d0 0x7f16d80746b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:16.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.512+0000 7f16e0600640 1 -- 172.21.15.83:0/549791414 >> 172.21.15.83:0/549791414 conn(0x7f16d8079450 msgr2=0x7f16d8079860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:16.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.512+0000 7f16e0600640 1 -- 172.21.15.83:0/549791414 shutdown_connections 2024-04-24T09:16:16.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.512+0000 7f16e0600640 1 -- 172.21.15.83:0/549791414 wait complete. 2024-04-24T09:16:16.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.513+0000 7f16e0600640 1 Processor -- start 2024-04-24T09:16:16.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.514+0000 7f16e0600640 1 -- start start 2024-04-24T09:16:16.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.514+0000 7f16e0600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16d80742d0 0x7f16d81a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:16.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.514+0000 7f16e0600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f16d8074bf0 0x7f16d81a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:16.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.514+0000 7f16e0600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f16d810e8e0 0x7f16d819a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:16.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.514+0000 7f16e0600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f16d8114480 con 0x7f16d8074bf0 2024-04-24T09:16:16.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.514+0000 7f16e0600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f16d8114300 con 0x7f16d80742d0 2024-04-24T09:16:16.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.514+0000 7f16e0600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f16d8114600 con 0x7f16d810e8e0 2024-04-24T09:16:16.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.514+0000 7f16dd200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f16d8074bf0 0x7f16d81a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:16.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.514+0000 7f16ddc00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16d80742d0 0x7f16d81a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:16.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.514+0000 7f16dd200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f16d8074bf0 0x7f16d81a0bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52094/0 (socket says 172.21.15.83:52094) 2024-04-24T09:16:16.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.515+0000 7f16dd200640 1 -- 172.21.15.83:0/2060466929 learned_addr learned my addr 172.21.15.83:0/2060466929 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:16:16.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.515+0000 7f16de600640 1 --2- 172.21.15.83:0/2060466929 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f16d810e8e0 0x7f16d819a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:16.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.515+0000 7f16ddc00640 1 -- 172.21.15.83:0/2060466929 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f16d810e8e0 msgr2=0x7f16d819a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:16.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.515+0000 7f16ddc00640 1 --2- 172.21.15.83:0/2060466929 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f16d810e8e0 0x7f16d819a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:16.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.515+0000 7f16ddc00640 1 -- 172.21.15.83:0/2060466929 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f16d8074bf0 msgr2=0x7f16d81a0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:16.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.515+0000 7f16ddc00640 1 --2- 172.21.15.83:0/2060466929 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f16d8074bf0 0x7f16d81a0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:16.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.515+0000 7f16ddc00640 1 -- 172.21.15.83:0/2060466929 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16d819afe0 con 0x7f16d80742d0 2024-04-24T09:16:16.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.516+0000 7f16ddc00640 1 --2- 172.21.15.83:0/2060466929 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16d80742d0 0x7f16d81a06b0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f16cc005fd0 tx=0x7f16cc00dde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:16.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.516+0000 7f16d6000640 1 -- 172.21.15.83:0/2060466929 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16cc009c30 con 0x7f16d80742d0 2024-04-24T09:16:16.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.516+0000 7f16d6000640 1 -- 172.21.15.83:0/2060466929 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f16cc009dd0 con 0x7f16d80742d0 2024-04-24T09:16:16.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.516+0000 7f16e0600640 1 -- 172.21.15.83:0/2060466929 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16d819b2d0 con 0x7f16d80742d0 2024-04-24T09:16:16.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.516+0000 7f16d6000640 1 -- 172.21.15.83:0/2060466929 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16cc003e20 con 0x7f16d80742d0 2024-04-24T09:16:16.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.516+0000 7f16e0600640 1 -- 172.21.15.83:0/2060466929 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16d81a7570 con 0x7f16d80742d0 2024-04-24T09:16:16.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.520+0000 7f16e33a8740 1 Processor -- start 2024-04-24T09:16:16.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.521+0000 7f16e33a8740 1 -- start start 2024-04-24T09:16:16.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.521+0000 7f16e33a8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ff836e090 0x562ff836e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:16.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.521+0000 7f16e33a8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562ff836e9b0 0x562ff8378b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:16.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.521+0000 7f16e33a8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562ff8379080 0x562ff837b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:16.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.521+0000 7f16e33a8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562ff82d3e70 con 0x562ff836e9b0 2024-04-24T09:16:16.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.521+0000 7f16e33a8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562ff827a380 con 0x562ff836e090 2024-04-24T09:16:16.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.521+0000 7f16e33a8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x562ff82cfe30 con 0x562ff8379080 2024-04-24T09:16:16.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.521+0000 7f16de600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ff836e090 0x562ff836e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:16.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.522+0000 7f16d6000640 1 -- 172.21.15.83:0/2060466929 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f16cc0035f0 con 0x7f16d80742d0 2024-04-24T09:16:16.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.522+0000 7f16de600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ff836e090 0x562ff836e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:51496/0 (socket says 172.21.15.83:51496) 2024-04-24T09:16:16.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.522+0000 7f16de600640 1 -- 172.21.15.83:0/4175110735 learned_addr learned my addr 172.21.15.83:0/4175110735 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:16:16.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.522+0000 7f16dd200640 1 --2- 172.21.15.83:0/4175110735 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562ff8379080 0x562ff837b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:16.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.522+0000 7f16de600640 1 -- 172.21.15.83:0/4175110735 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562ff8379080 msgr2=0x562ff837b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:16.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.522+0000 7f16de600640 1 --2- 172.21.15.83:0/4175110735 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562ff8379080 0x562ff837b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:16.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.522+0000 7f16de600640 1 -- 172.21.15.83:0/4175110735 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562ff836e9b0 msgr2=0x562ff8378b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:16.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.522+0000 7f16de600640 1 --2- 172.21.15.83:0/4175110735 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562ff836e9b0 0x562ff8378b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:16.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.522+0000 7f16de600640 1 -- 172.21.15.83:0/4175110735 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ff80ae7e0 con 0x562ff836e090 2024-04-24T09:16:16.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.522+0000 7f16dd200640 1 --2- 172.21.15.83:0/4175110735 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562ff8379080 0x562ff837b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:16:16.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.523+0000 7f16de600640 1 --2- 172.21.15.83:0/4175110735 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ff836e090 0x562ff836e470 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f16d0009f00 tx=0x7f16d00107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d012b16336305589 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:16.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.523+0000 7f16caa00640 1 -- 172.21.15.83:0/4175110735 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16d0003b70 con 0x562ff836e090 2024-04-24T09:16:16.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.523+0000 7f16caa00640 1 -- 172.21.15.83:0/4175110735 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f16d0003d10 con 0x562ff836e090 2024-04-24T09:16:16.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.523+0000 7f16caa00640 1 -- 172.21.15.83:0/4175110735 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16d00119d0 con 0x562ff836e090 2024-04-24T09:16:16.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.524+0000 7f16e33a8740 1 -- 172.21.15.83:0/4175110735 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ff836e090 msgr2=0x562ff836e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:16.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.524+0000 7f16e33a8740 1 --2- 172.21.15.83:0/4175110735 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ff836e090 0x562ff836e470 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f16d0009f00 tx=0x7f16d00107a0 comp rx=0 tx=0).stop 2024-04-24T09:16:16.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.524+0000 7f16e33a8740 1 -- 172.21.15.83:0/4175110735 shutdown_connections 2024-04-24T09:16:16.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.524+0000 7f16e33a8740 1 --2- 172.21.15.83:0/4175110735 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562ff8379080 0x562ff837b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:16.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.524+0000 7f16e33a8740 1 --2- 172.21.15.83:0/4175110735 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562ff836e9b0 0x562ff8378b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:16.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.524+0000 7f16e33a8740 1 --2- 172.21.15.83:0/4175110735 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ff836e090 0x562ff836e470 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:16.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.524+0000 7f16e33a8740 1 -- 172.21.15.83:0/4175110735 >> 172.21.15.83:0/4175110735 conn(0x562ff836a3b0 msgr2=0x562ff836a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:16.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.525+0000 7f16e33a8740 1 -- 172.21.15.83:0/4175110735 shutdown_connections 2024-04-24T09:16:16.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.525+0000 7f16e33a8740 1 -- 172.21.15.83:0/4175110735 wait complete. 2024-04-24T09:16:16.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.525+0000 7f16e33a8740 1 -- start start 2024-04-24T09:16:16.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.525+0000 7f16d6000640 1 --2- 172.21.15.83:0/2060466929 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f16c00a3e00 0x7f16c00a62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:16.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.526+0000 7f16e33a8740 1 Processor -- start 2024-04-24T09:16:16.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.526+0000 7f16d6000640 1 -- 172.21.15.83:0/2060466929 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 9583+0+0 (secure 0 0 0) 0x7f16cc0d0740 con 0x7f16d80742d0 2024-04-24T09:16:16.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.526+0000 7f16de600640 1 --2- 172.21.15.83:0/2060466929 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f16c00a3e00 0x7f16c00a62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:16.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.526+0000 7f16e33a8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562ff836e090 0x562ff8376050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:16.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.526+0000 7f16e33a8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562ff836e9b0 0x562ff8376590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:16.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.526+0000 7f16e33a8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ff8379080 0x562ff8376ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:16.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.526+0000 7f16e33a8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562ff837be70 con 0x562ff836e9b0 2024-04-24T09:16:16.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.526+0000 7f16e33a8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562ff828d660 con 0x562ff8379080 2024-04-24T09:16:16.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.526+0000 7f16e33a8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x562ff837bff0 con 0x562ff836e090 2024-04-24T09:16:16.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.526+0000 7f16e33a8740 10 client.0 _ll_register_callbacks cb 0x562ff827baa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:16:16.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.527+0000 7f16dd200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562ff836e9b0 0x562ff8376590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:16.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.527+0000 7f16dd200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562ff836e9b0 0x562ff8376590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52120/0 (socket says 172.21.15.83:52120) 2024-04-24T09:16:16.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.527+0000 7f16dd200640 1 -- 172.21.15.83:0/1783226917 learned_addr learned my addr 172.21.15.83:0/1783226917 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:16:16.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.527+0000 7f16ddc00640 1 --2- 172.21.15.83:0/1783226917 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562ff836e090 0x562ff8376050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:16.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.527+0000 7f16de600640 1 --2- 172.21.15.83:0/2060466929 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f16c00a3e00 0x7f16c00a62c0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f16d0011010 tx=0x7f16d0013040 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:16.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.527+0000 7f16dd200640 1 -- 172.21.15.83:0/1783226917 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562ff836e090 msgr2=0x562ff8376050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:16.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.527+0000 7f16dd200640 1 --2- 172.21.15.83:0/1783226917 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562ff836e090 0x562ff8376050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:16.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.527+0000 7f16dd200640 1 -- 172.21.15.83:0/1783226917 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ff8379080 msgr2=0x562ff8376ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:16.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.527+0000 7f16dd200640 1 --2- 172.21.15.83:0/1783226917 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ff8379080 0x562ff8376ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:16.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.527+0000 7f16dd200640 1 -- 172.21.15.83:0/1783226917 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ff810ef70 con 0x562ff836e9b0 2024-04-24T09:16:16.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.528+0000 7f16dd200640 1 --2- 172.21.15.83:0/1783226917 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562ff836e9b0 0x562ff8376590 secure :-1 s=READY pgs=2774 cs=0 l=1 rev1=1 crypto rx=0x7f16c4009a20 tx=0x7f16c4031e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=674b1db0cbe998b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:16.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.528+0000 7f16bf400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:16:16.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.528+0000 7f16bf400640 1 -- 172.21.15.83:0/1783226917 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16c4035040 con 0x562ff836e9b0 2024-04-24T09:16:16.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.528+0000 7f16bf400640 1 -- 172.21.15.83:0/1783226917 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(132..132 src has 1..132) ==== 9583+0+0 (secure 0 0 0) 0x7f16c4004050 con 0x562ff836e9b0 2024-04-24T09:16:16.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.528+0000 7f16e33a8740 1 -- 172.21.15.83:0/1783226917 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16d0003d10 con 0x562ff836e9b0 2024-04-24T09:16:16.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.529+0000 7f16bf400640 1 -- 172.21.15.83:0/1783226917 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f16c404d050 con 0x562ff836e9b0 2024-04-24T09:16:16.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.529+0000 7f16e33a8740 10 client.23925 fetch_fsmap learned FSMap version 69 2024-04-24T09:16:16.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.529+0000 7f16e33a8740 1 -- 172.21.15.83:0/1783226917 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x562ff7fa6170 con 0x562ff836e9b0 2024-04-24T09:16:16.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.529+0000 7f16bf400640 1 -- 172.21.15.83:0/1783226917 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f16d0003d10 con 0x562ff836e9b0 2024-04-24T09:16:16.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.530+0000 7f16bf400640 1 -- 172.21.15.83:0/1783226917 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 69) ==== 8370+0+0 (secure 0 0 0) 0x7f16c40047a0 con 0x562ff836e9b0 2024-04-24T09:16:16.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.530+0000 7f16e33a8740 10 client.23925 fetch_fsmap finished waiting for FSMap version 69 2024-04-24T09:16:16.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.530+0000 7f16e33a8740 10 client.23925 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:16:16.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.530+0000 7f16e33a8740 20 client.23925 populate_metadata read hostname 'smithi083' 2024-04-24T09:16:16.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.530+0000 7f16e33a8740 1 --2- 172.21.15.83:0/1783226917 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x562ff83894a0 0x562ff838b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:16.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.530+0000 7f16e33a8740 4 client.23925 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:16:16.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.530+0000 7f16e33a8740 1 -- 172.21.15.83:0/1783226917 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562ff82e1890 con 0x562ff83894a0 2024-04-24T09:16:16.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.530+0000 7f16ddc00640 1 --2- 172.21.15.83:0/1783226917 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x562ff83894a0 0x562ff838b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:16.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.531+0000 7f16ddc00640 1 --2- 172.21.15.83:0/1783226917 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x562ff83894a0 0x562ff838b8c0 crc :-1 s=READY pgs=603 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d890d2e6b5d5d5c server_cookie=14310ce9efc9d001 in_seq=0 out_seq=0 2024-04-24T09:16:16.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.531+0000 7f16bf400640 10 client.23925 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:16:16.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.533+0000 7f16bf400640 1 -- 172.21.15.83:0/1783226917 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562ff82e1890 con 0x562ff83894a0 2024-04-24T09:16:16.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.533+0000 7f16bf400640 10 client.23925 handle_command_reply: tid=0 2024-04-24T09:16:16.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.533+0000 7f16e33a8740 1 client.23925 shutdown 2024-04-24T09:16:16.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.533+0000 7f16e33a8740 1 -- 172.21.15.83:0/1783226917 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562ff836e9b0 msgr2=0x562ff8376590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:16.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.533+0000 7f16e33a8740 1 --2- 172.21.15.83:0/1783226917 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562ff836e9b0 0x562ff8376590 secure :-1 s=READY pgs=2774 cs=0 l=1 rev1=1 crypto rx=0x7f16c4009a20 tx=0x7f16c4031e60 comp rx=0 tx=0).stop 2024-04-24T09:16:16.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.533+0000 7f16e33a8740 1 -- 172.21.15.83:0/1783226917 shutdown_connections 2024-04-24T09:16:16.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.533+0000 7f16e33a8740 1 --2- 172.21.15.83:0/1783226917 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x562ff83894a0 0x562ff838b8c0 crc :-1 s=READY pgs=603 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:16.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.533+0000 7f16e33a8740 1 --2- 172.21.15.83:0/1783226917 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ff8379080 0x562ff8376ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:16.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.533+0000 7f16e33a8740 1 --2- 172.21.15.83:0/1783226917 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562ff836e9b0 0x562ff8376590 unknown :-1 s=CLOSED pgs=2774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:16.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.533+0000 7f16e33a8740 1 --2- 172.21.15.83:0/1783226917 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562ff836e090 0x562ff8376050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:16.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.533+0000 7f16bf400640 0 client.23925 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:16:16.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.533+0000 7f16e33a8740 1 -- 172.21.15.83:0/1783226917 >> 172.21.15.83:0/1783226917 conn(0x562ff836a3b0 msgr2=0x562ff83700b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:16.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.534+0000 7f16e33a8740 1 -- 172.21.15.83:0/1783226917 shutdown_connections 2024-04-24T09:16:16.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.534+0000 7f16e33a8740 1 -- 172.21.15.83:0/1783226917 wait complete. 2024-04-24T09:16:16.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.534+0000 7f16e33a8740 20 client.23925 trim_cache size 0 max 16384 2024-04-24T09:16:16.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.568+0000 7f16e33a8740 1 Processor -- start 2024-04-24T09:16:16.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.568+0000 7f16e33a8740 1 -- start start 2024-04-24T09:16:16.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.568+0000 7f16e33a8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562ff836e090 0x562ff8384d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:16.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.568+0000 7f16e33a8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562ff836e9b0 0x562ff8376720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:16.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.568+0000 7f16e33a8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ff8379080 0x562ff8376c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:16.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.568+0000 7f16e33a8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f16d81b1870 con 0x562ff836e9b0 2024-04-24T09:16:16.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.568+0000 7f16e33a8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f16d81b3410 con 0x562ff8379080 2024-04-24T09:16:16.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.568+0000 7f16e33a8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f16d81b1f40 con 0x562ff836e090 2024-04-24T09:16:16.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.568+0000 7f16dd200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562ff836e9b0 0x562ff8376720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:16.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.569+0000 7f16dd200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562ff836e9b0 0x562ff8376720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52124/0 (socket says 172.21.15.83:52124) 2024-04-24T09:16:16.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.569+0000 7f16de600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ff8379080 0x562ff8376c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:16.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.569+0000 7f16dd200640 1 -- 172.21.15.83:0/3935394668 learned_addr learned my addr 172.21.15.83:0/3935394668 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:16:16.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.569+0000 7f16ddc00640 1 --2- 172.21.15.83:0/3935394668 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562ff836e090 0x562ff8384d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:16.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.569+0000 7f16dd200640 1 -- 172.21.15.83:0/3935394668 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562ff836e090 msgr2=0x562ff8384d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:16.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.569+0000 7f16dd200640 1 --2- 172.21.15.83:0/3935394668 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562ff836e090 0x562ff8384d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:16.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.569+0000 7f16dd200640 1 -- 172.21.15.83:0/3935394668 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ff8379080 msgr2=0x562ff8376c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:16.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.569+0000 7f16dd200640 1 --2- 172.21.15.83:0/3935394668 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ff8379080 0x562ff8376c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:16.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.569+0000 7f16dd200640 1 -- 172.21.15.83:0/3935394668 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ff838bf20 con 0x562ff836e9b0 2024-04-24T09:16:16.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.570+0000 7f16dd200640 1 --2- 172.21.15.83:0/3935394668 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562ff836e9b0 0x562ff8376720 secure :-1 s=READY pgs=2775 cs=0 l=1 rev1=1 crypto rx=0x7f16c4009850 tx=0x7f16c402a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46c998679f4453a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:16.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.570+0000 7f16ca000640 1 -- 172.21.15.83:0/3935394668 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16c4035030 con 0x562ff836e9b0 2024-04-24T09:16:16.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.570+0000 7f16ca000640 1 -- 172.21.15.83:0/3935394668 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f16c4040070 con 0x562ff836e9b0 2024-04-24T09:16:16.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.570+0000 7f16ca000640 1 -- 172.21.15.83:0/3935394668 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16c4005530 con 0x562ff836e9b0 2024-04-24T09:16:16.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.571+0000 7f16e33a8740 1 -- 172.21.15.83:0/3935394668 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562ff836e9b0 msgr2=0x562ff8376720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:16.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.571+0000 7f16e33a8740 1 --2- 172.21.15.83:0/3935394668 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562ff836e9b0 0x562ff8376720 secure :-1 s=READY pgs=2775 cs=0 l=1 rev1=1 crypto rx=0x7f16c4009850 tx=0x7f16c402a480 comp rx=0 tx=0).stop 2024-04-24T09:16:16.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.571+0000 7f16e33a8740 1 -- 172.21.15.83:0/3935394668 shutdown_connections 2024-04-24T09:16:16.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.571+0000 7f16e33a8740 1 --2- 172.21.15.83:0/3935394668 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ff8379080 0x562ff8376c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:16.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.571+0000 7f16e33a8740 1 --2- 172.21.15.83:0/3935394668 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562ff836e9b0 0x562ff8376720 unknown :-1 s=CLOSED pgs=2775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:16.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.571+0000 7f16e33a8740 1 --2- 172.21.15.83:0/3935394668 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562ff836e090 0x562ff8384d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:16.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.571+0000 7f16e33a8740 1 -- 172.21.15.83:0/3935394668 >> 172.21.15.83:0/3935394668 conn(0x562ff836a3b0 msgr2=0x562ff8376010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:16.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.571+0000 7f16e33a8740 1 -- 172.21.15.83:0/3935394668 shutdown_connections 2024-04-24T09:16:16.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.571+0000 7f16e33a8740 1 -- 172.21.15.83:0/3935394668 wait complete. 2024-04-24T09:16:16.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.572+0000 7f16e33a8740 1 -- start start 2024-04-24T09:16:16.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.572+0000 7f16e33a8740 1 Processor -- start 2024-04-24T09:16:16.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.572+0000 7f16e33a8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562ff836e090 0x562ff837cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:16.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.572+0000 7f16e33a8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ff836e9b0 0x562ff837d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:16.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.572+0000 7f16e33a8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562ff8379080 0x562ff837d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:16.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.572+0000 7f16e33a8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f16d81b2d40 con 0x562ff836e090 2024-04-24T09:16:16.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.572+0000 7f16e33a8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f16d81b2670 con 0x562ff836e9b0 2024-04-24T09:16:16.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.572+0000 7f16e33a8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f16d81af870 con 0x562ff8379080 2024-04-24T09:16:16.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.572+0000 7f16e33a8740 10 client.0 _ll_register_callbacks cb 0x7f16c4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:16:16.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.573+0000 7f16ddc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562ff836e090 0x562ff837cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:16.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.573+0000 7f16ddc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562ff836e090 0x562ff837cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52130/0 (socket says 172.21.15.83:52130) 2024-04-24T09:16:16.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.573+0000 7f16ddc00640 1 -- 172.21.15.83:0/3221829585 learned_addr learned my addr 172.21.15.83:0/3221829585 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:16:16.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.573+0000 7f16de600640 1 --2- 172.21.15.83:0/3221829585 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562ff8379080 0x562ff837d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:16.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.573+0000 7f16dd200640 1 --2- 172.21.15.83:0/3221829585 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ff836e9b0 0x562ff837d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:16.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.573+0000 7f16ddc00640 1 -- 172.21.15.83:0/3221829585 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562ff8379080 msgr2=0x562ff837d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:16.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.573+0000 7f16ddc00640 1 --2- 172.21.15.83:0/3221829585 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562ff8379080 0x562ff837d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:16.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.573+0000 7f16ddc00640 1 -- 172.21.15.83:0/3221829585 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ff836e9b0 msgr2=0x562ff837d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:16.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.573+0000 7f16ddc00640 1 --2- 172.21.15.83:0/3221829585 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ff836e9b0 0x562ff837d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:16.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.573+0000 7f16ddc00640 1 -- 172.21.15.83:0/3221829585 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ff83f71d0 con 0x562ff836e090 2024-04-24T09:16:16.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.574+0000 7f16ddc00640 1 --2- 172.21.15.83:0/3221829585 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562ff836e090 0x562ff837cbd0 secure :-1 s=READY pgs=2776 cs=0 l=1 rev1=1 crypto rx=0x7f16cc009610 tx=0x7f16cc00d8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49b5717a27858d78 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:16.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.574+0000 7f16bf400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:16:16.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.574+0000 7f16bf400640 1 -- 172.21.15.83:0/3221829585 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16cc0b4070 con 0x562ff836e090 2024-04-24T09:16:16.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.574+0000 7f16bf400640 1 -- 172.21.15.83:0/3221829585 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(132..132 src has 1..132) ==== 9583+0+0 (secure 0 0 0) 0x7f16cc0988d0 con 0x562ff836e090 2024-04-24T09:16:16.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.574+0000 7f16e33a8740 1 -- 172.21.15.83:0/3221829585 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16c4040070 con 0x562ff836e090 2024-04-24T09:16:16.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.575+0000 7f16bf400640 1 -- 172.21.15.83:0/3221829585 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f16cc020040 con 0x562ff836e090 2024-04-24T09:16:16.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.575+0000 7f16e33a8740 10 client.23931 fetch_fsmap learned FSMap version 69 2024-04-24T09:16:16.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.575+0000 7f16e33a8740 1 -- 172.21.15.83:0/3221829585 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x562ff83f8450 con 0x562ff836e090 2024-04-24T09:16:16.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.575+0000 7f16bf400640 1 -- 172.21.15.83:0/3221829585 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f16c4040070 con 0x562ff836e090 2024-04-24T09:16:16.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.575+0000 7f16bf400640 1 -- 172.21.15.83:0/3221829585 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 69) ==== 8370+0+0 (secure 0 0 0) 0x7f16cc0a0160 con 0x562ff836e090 2024-04-24T09:16:16.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.575+0000 7f16e33a8740 10 client.23931 fetch_fsmap finished waiting for FSMap version 69 2024-04-24T09:16:16.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.575+0000 7f16e33a8740 10 client.23931 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:16:16.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.575+0000 7f16e33a8740 20 client.23931 populate_metadata read hostname 'smithi083' 2024-04-24T09:16:16.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.575+0000 7f16e33a8740 1 --2- 172.21.15.83:0/3221829585 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x562ff83f8b70 0x562ff83faf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:16.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.575+0000 7f16e33a8740 4 client.23931 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T09:16:16.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.575+0000 7f16e33a8740 1 -- 172.21.15.83:0/3221829585 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x562ff82db700 con 0x562ff83f8b70 2024-04-24T09:16:16.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.575+0000 7f16dd200640 1 --2- 172.21.15.83:0/3221829585 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x562ff83f8b70 0x562ff83faf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:16.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.576+0000 7f16dd200640 1 --2- 172.21.15.83:0/3221829585 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x562ff83f8b70 0x562ff83faf90 crc :-1 s=READY pgs=604 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a541cd2ef107134a server_cookie=4d24c3d89cd9bbcf in_seq=0 out_seq=0 2024-04-24T09:16:16.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.576+0000 7f16bf400640 10 client.23931 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:16:16.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.577+0000 7f16bf400640 1 -- 172.21.15.83:0/3221829585 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562ff82db700 con 0x562ff83f8b70 2024-04-24T09:16:16.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.578+0000 7f16bf400640 10 client.23931 handle_command_reply: tid=0 2024-04-24T09:16:16.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.578+0000 7f16e33a8740 1 client.23931 shutdown 2024-04-24T09:16:16.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.578+0000 7f16e33a8740 1 -- 172.21.15.83:0/3221829585 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562ff836e090 msgr2=0x562ff837cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:16.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.578+0000 7f16e33a8740 1 --2- 172.21.15.83:0/3221829585 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562ff836e090 0x562ff837cbd0 secure :-1 s=READY pgs=2776 cs=0 l=1 rev1=1 crypto rx=0x7f16cc009610 tx=0x7f16cc00d8d0 comp rx=0 tx=0).stop 2024-04-24T09:16:16.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.578+0000 7f16e33a8740 1 -- 172.21.15.83:0/3221829585 shutdown_connections 2024-04-24T09:16:16.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.578+0000 7f16e33a8740 1 --2- 172.21.15.83:0/3221829585 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x562ff83f8b70 0x562ff83faf90 crc :-1 s=READY pgs=604 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:16.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.578+0000 7f16e33a8740 1 --2- 172.21.15.83:0/3221829585 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562ff8379080 0x562ff837d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:16.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.578+0000 7f16e33a8740 1 --2- 172.21.15.83:0/3221829585 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ff836e9b0 0x562ff837d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:16.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.578+0000 7f16e33a8740 1 --2- 172.21.15.83:0/3221829585 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562ff836e090 0x562ff837cbd0 unknown :-1 s=CLOSED pgs=2776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:16.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.578+0000 7f16e33a8740 1 -- 172.21.15.83:0/3221829585 >> 172.21.15.83:0/3221829585 conn(0x562ff836a3b0 msgr2=0x562ff83768a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:16.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.578+0000 7f16bf400640 0 client.23931 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:16:16.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.578+0000 7f16e33a8740 1 -- 172.21.15.83:0/3221829585 shutdown_connections 2024-04-24T09:16:16.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.578+0000 7f16e33a8740 1 -- 172.21.15.83:0/3221829585 wait complete. 2024-04-24T09:16:16.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.578+0000 7f16e33a8740 20 client.23931 trim_cache size 0 max 16384 2024-04-24T09:16:16.580 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T09:16:16.580 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-04-24T09:16:16.580 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "465e20cc-88c5-4f84-8ace-5ca9be6b1e93", 2024-04-24T09:16:16.580 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-04-24T09:16:16.580 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T09:16:16.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.579+0000 7f16e0600640 1 -- 172.21.15.83:0/2060466929 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f16c00a3e00 msgr2=0x7f16c00a62c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:16.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.579+0000 7f16e0600640 1 --2- 172.21.15.83:0/2060466929 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f16c00a3e00 0x7f16c00a62c0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f16d0011010 tx=0x7f16d0013040 comp rx=0 tx=0).stop 2024-04-24T09:16:16.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.579+0000 7f16e0600640 1 -- 172.21.15.83:0/2060466929 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16d80742d0 msgr2=0x7f16d81a06b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:16.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.579+0000 7f16e0600640 1 --2- 172.21.15.83:0/2060466929 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16d80742d0 0x7f16d81a06b0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f16cc005fd0 tx=0x7f16cc00dde0 comp rx=0 tx=0).stop 2024-04-24T09:16:16.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.580+0000 7f16e0600640 1 -- 172.21.15.83:0/2060466929 shutdown_connections 2024-04-24T09:16:16.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.580+0000 7f16e0600640 1 --2- 172.21.15.83:0/2060466929 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f16c00a3e00 0x7f16c00a62c0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:16.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.580+0000 7f16e0600640 1 --2- 172.21.15.83:0/2060466929 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f16d810e8e0 0x7f16d819a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:16.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.580+0000 7f16e0600640 1 --2- 172.21.15.83:0/2060466929 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f16d8074bf0 0x7f16d81a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:16.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.580+0000 7f16e0600640 1 --2- 172.21.15.83:0/2060466929 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16d80742d0 0x7f16d81a06b0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:16.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.580+0000 7f16e0600640 1 -- 172.21.15.83:0/2060466929 >> 172.21.15.83:0/2060466929 conn(0x7f16d8079450 msgr2=0x7f16d80fefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:16.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.580+0000 7f16e0600640 1 -- 172.21.15.83:0/2060466929 shutdown_connections 2024-04-24T09:16:16.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.580+0000 7f16e0600640 1 -- 172.21.15.83:0/2060466929 wait complete. 2024-04-24T09:16:16.602 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T09:16:16.602 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T09:16:16.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.898+0000 7f38d4c00640 1 -- 172.21.15.83:0/337303911 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f38d0104e60 msgr2=0x7f38d0108ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:16.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.898+0000 7f38d4c00640 1 --2- 172.21.15.83:0/337303911 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f38d0104e60 0x7f38d0108ea0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f38bc009900 tx=0x7f38bc02a3f0 comp rx=0 tx=0).stop 2024-04-24T09:16:16.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.898+0000 7f38d4c00640 1 -- 172.21.15.83:0/337303911 shutdown_connections 2024-04-24T09:16:16.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.898+0000 7f38d4c00640 1 --2- 172.21.15.83:0/337303911 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38d0109580 0x7f38d0111150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:16.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.898+0000 7f38d4c00640 1 --2- 172.21.15.83:0/337303911 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f38d0104e60 0x7f38d0108ea0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:16.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.898+0000 7f38d4c00640 1 --2- 172.21.15.83:0/337303911 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f38d01044b0 0x7f38d0104890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:16.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.898+0000 7f38d4c00640 1 -- 172.21.15.83:0/337303911 >> 172.21.15.83:0/337303911 conn(0x7f38d00ffd90 msgr2=0x7f38d01021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:16.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.899+0000 7f38d4c00640 1 -- 172.21.15.83:0/337303911 shutdown_connections 2024-04-24T09:16:16.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.899+0000 7f38d4c00640 1 -- 172.21.15.83:0/337303911 wait complete. 2024-04-24T09:16:16.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.900+0000 7f38d4c00640 1 Processor -- start 2024-04-24T09:16:16.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.900+0000 7f38d4c00640 1 -- start start 2024-04-24T09:16:16.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.900+0000 7f38d4c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38d01044b0 0x7f38d011a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:16.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.900+0000 7f38d4c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f38d0109580 0x7f38d01105f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:16.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.900+0000 7f38d4c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f38d0110b30 0x7f38d0110fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:16.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.900+0000 7f38d4c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f38d0078070 con 0x7f38d01044b0 2024-04-24T09:16:16.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.900+0000 7f38d4c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f38d0077ef0 con 0x7f38d0109580 2024-04-24T09:16:16.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.900+0000 7f38d4c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f38d00781f0 con 0x7f38d0110b30 2024-04-24T09:16:16.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.901+0000 7f38cde00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38d01044b0 0x7f38d011a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:16.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.901+0000 7f38cde00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38d01044b0 0x7f38d011a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52138/0 (socket says 172.21.15.83:52138) 2024-04-24T09:16:16.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.901+0000 7f38cde00640 1 -- 172.21.15.83:0/3801744651 learned_addr learned my addr 172.21.15.83:0/3801744651 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:16:16.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.901+0000 7f38cd400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f38d0109580 0x7f38d01105f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:16.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.901+0000 7f38ce800640 1 --2- 172.21.15.83:0/3801744651 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f38d0110b30 0x7f38d0110fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:16.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.901+0000 7f38cde00640 1 -- 172.21.15.83:0/3801744651 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f38d0110b30 msgr2=0x7f38d0110fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:16.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.901+0000 7f38cde00640 1 --2- 172.21.15.83:0/3801744651 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f38d0110b30 0x7f38d0110fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:16.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.901+0000 7f38cde00640 1 -- 172.21.15.83:0/3801744651 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f38d0109580 msgr2=0x7f38d01105f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:16.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.901+0000 7f38cde00640 1 --2- 172.21.15.83:0/3801744651 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f38d0109580 0x7f38d01105f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:16.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.901+0000 7f38cde00640 1 -- 172.21.15.83:0/3801744651 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38d0116210 con 0x7f38d01044b0 2024-04-24T09:16:16.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.901+0000 7f38cd400640 1 --2- 172.21.15.83:0/3801744651 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f38d0109580 0x7f38d01105f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:16:16.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.902+0000 7f38cde00640 1 --2- 172.21.15.83:0/3801744651 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38d01044b0 0x7f38d011a7d0 secure :-1 s=READY pgs=2777 cs=0 l=1 rev1=1 crypto rx=0x7f38c000d980 tx=0x7f38c000de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:16.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.902+0000 7f38c6a00640 1 -- 172.21.15.83:0/3801744651 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c0017030 con 0x7f38d01044b0 2024-04-24T09:16:16.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.902+0000 7f38c6a00640 1 -- 172.21.15.83:0/3801744651 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f38c0004020 con 0x7f38d01044b0 2024-04-24T09:16:16.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.902+0000 7f38d4c00640 1 -- 172.21.15.83:0/3801744651 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38d0116500 con 0x7f38d01044b0 2024-04-24T09:16:16.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.903+0000 7f38d4c00640 1 -- 172.21.15.83:0/3801744651 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38d0116a70 con 0x7f38d01044b0 2024-04-24T09:16:16.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.903+0000 7f38c6a00640 1 -- 172.21.15.83:0/3801744651 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c0005640 con 0x7f38d01044b0 2024-04-24T09:16:16.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.907+0000 7f38d79d1740 1 Processor -- start 2024-04-24T09:16:16.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.907+0000 7f38d79d1740 1 -- start start 2024-04-24T09:16:16.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.907+0000 7f38c6a00640 1 -- 172.21.15.83:0/3801744651 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f38c0009a30 con 0x7f38d01044b0 2024-04-24T09:16:16.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.908+0000 7f38d79d1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556fdf6ae860 0x556fdf6aec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:16.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.908+0000 7f38d79d1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556fdf6af180 0x556fdf6b9310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:16.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.908+0000 7f38d79d1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556fdf6b9850 0x556fdf6bbc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:16.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.908+0000 7f38d79d1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556fdf6141f0 con 0x556fdf6ae860 2024-04-24T09:16:16.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.908+0000 7f38d79d1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556fdf5ba350 con 0x556fdf6af180 2024-04-24T09:16:16.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.908+0000 7f38d79d1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x556fdf6101b0 con 0x556fdf6b9850 2024-04-24T09:16:16.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.908+0000 7f38ce800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556fdf6ae860 0x556fdf6aec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:16.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.908+0000 7f38ce800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556fdf6ae860 0x556fdf6aec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52146/0 (socket says 172.21.15.83:52146) 2024-04-24T09:16:16.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.908+0000 7f38ce800640 1 -- 172.21.15.83:0/4122987097 learned_addr learned my addr 172.21.15.83:0/4122987097 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:16:16.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.908+0000 7f38cd400640 1 --2- 172.21.15.83:0/4122987097 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556fdf6b9850 0x556fdf6bbc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:16.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.909+0000 7f38ce800640 1 -- 172.21.15.83:0/4122987097 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556fdf6b9850 msgr2=0x556fdf6bbc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:16.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.909+0000 7f38ce800640 1 --2- 172.21.15.83:0/4122987097 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556fdf6b9850 0x556fdf6bbc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:16.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.909+0000 7f38ce800640 1 -- 172.21.15.83:0/4122987097 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556fdf6af180 msgr2=0x556fdf6b9310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:16.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.909+0000 7f38ce800640 1 --2- 172.21.15.83:0/4122987097 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556fdf6af180 0x556fdf6b9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:16.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.909+0000 7f38ce800640 1 -- 172.21.15.83:0/4122987097 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556fdf60ef50 con 0x556fdf6ae860 2024-04-24T09:16:16.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.910+0000 7f38ce800640 1 --2- 172.21.15.83:0/4122987097 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556fdf6ae860 0x556fdf6aec40 secure :-1 s=READY pgs=2778 cs=0 l=1 rev1=1 crypto rx=0x7f38c8009f40 tx=0x7f38c8010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b26e76671a87b2fd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:16.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.910+0000 7f38bb000640 1 -- 172.21.15.83:0/4122987097 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c8003b70 con 0x556fdf6ae860 2024-04-24T09:16:16.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.910+0000 7f38bb000640 1 -- 172.21.15.83:0/4122987097 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f38c8003d10 con 0x556fdf6ae860 2024-04-24T09:16:16.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.910+0000 7f38c6a00640 1 --2- 172.21.15.83:0/3801744651 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f38b40a3f40 0x7f38b40a6400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:16.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.911+0000 7f38cde00640 1 --2- 172.21.15.83:0/3801744651 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f38b40a3f40 0x7f38b40a6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:16.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.911+0000 7f38c6a00640 1 -- 172.21.15.83:0/3801744651 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 9583+0+0 (secure 0 0 0) 0x7f38c00d0600 con 0x7f38d01044b0 2024-04-24T09:16:16.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.911+0000 7f38d79d1740 1 -- 172.21.15.83:0/4122987097 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556fdf6ae860 msgr2=0x556fdf6aec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:16.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.911+0000 7f38bb000640 1 -- 172.21.15.83:0/4122987097 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c80118a0 con 0x556fdf6ae860 2024-04-24T09:16:16.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.911+0000 7f38d79d1740 1 --2- 172.21.15.83:0/4122987097 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556fdf6ae860 0x556fdf6aec40 secure :-1 s=READY pgs=2778 cs=0 l=1 rev1=1 crypto rx=0x7f38c8009f40 tx=0x7f38c8010860 comp rx=0 tx=0).stop 2024-04-24T09:16:16.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.912+0000 7f38cde00640 1 --2- 172.21.15.83:0/3801744651 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f38b40a3f40 0x7f38b40a6400 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x556fdf6afe20 tx=0x7f38c0066780 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:16.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.912+0000 7f38d79d1740 1 -- 172.21.15.83:0/4122987097 shutdown_connections 2024-04-24T09:16:16.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.912+0000 7f38d79d1740 1 --2- 172.21.15.83:0/4122987097 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556fdf6b9850 0x556fdf6bbc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:16.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.912+0000 7f38d79d1740 1 --2- 172.21.15.83:0/4122987097 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556fdf6af180 0x556fdf6b9310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:16.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.912+0000 7f38d79d1740 1 --2- 172.21.15.83:0/4122987097 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556fdf6ae860 0x556fdf6aec40 unknown :-1 s=CLOSED pgs=2778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:16.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.912+0000 7f38d79d1740 1 -- 172.21.15.83:0/4122987097 >> 172.21.15.83:0/4122987097 conn(0x556fdf6aab80 msgr2=0x556fdf6aaf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:16.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.913+0000 7f38d79d1740 1 -- 172.21.15.83:0/4122987097 shutdown_connections 2024-04-24T09:16:16.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.913+0000 7f38d79d1740 1 -- 172.21.15.83:0/4122987097 wait complete. 2024-04-24T09:16:16.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.913+0000 7f38d79d1740 1 -- start start 2024-04-24T09:16:16.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.913+0000 7f38d79d1740 1 Processor -- start 2024-04-24T09:16:16.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.914+0000 7f38d79d1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556fdf6ae860 0x556fdf6b66a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:16.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.914+0000 7f38d79d1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556fdf6af180 0x556fdf6b6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:16.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.914+0000 7f38d79d1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556fdf6b9850 0x556fdf6b7120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:16.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.914+0000 7f38d79d1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556fdf6bc690 con 0x556fdf6af180 2024-04-24T09:16:16.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.914+0000 7f38d79d1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556fdf5cd690 con 0x556fdf6ae860 2024-04-24T09:16:16.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.914+0000 7f38d79d1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x556fdf6bc810 con 0x556fdf6b9850 2024-04-24T09:16:16.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.914+0000 7f38d79d1740 10 client.0 _ll_register_callbacks cb 0x556fdf609650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:16:16.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.914+0000 7f38ce800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556fdf6ae860 0x556fdf6b66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:16.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.914+0000 7f38ce800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556fdf6ae860 0x556fdf6b66a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:51544/0 (socket says 172.21.15.83:51544) 2024-04-24T09:16:16.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.914+0000 7f38ce800640 1 -- 172.21.15.83:0/1171264336 learned_addr learned my addr 172.21.15.83:0/1171264336 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:16:16.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.915+0000 7f38cd400640 1 --2- 172.21.15.83:0/1171264336 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556fdf6af180 0x556fdf6b6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:16.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.915+0000 7f38ce800640 1 --2- 172.21.15.83:0/1171264336 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556fdf6b9850 0x556fdf6b7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:16.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.915+0000 7f38ce800640 1 -- 172.21.15.83:0/1171264336 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556fdf6b9850 msgr2=0x556fdf6b7120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:16.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.915+0000 7f38ce800640 1 --2- 172.21.15.83:0/1171264336 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556fdf6b9850 0x556fdf6b7120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:16.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.915+0000 7f38ce800640 1 -- 172.21.15.83:0/1171264336 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556fdf6af180 msgr2=0x556fdf6b6be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:16.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.915+0000 7f38ce800640 1 --2- 172.21.15.83:0/1171264336 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556fdf6af180 0x556fdf6b6be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:16.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.915+0000 7f38ce800640 1 -- 172.21.15.83:0/1171264336 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556fdf5cc920 con 0x556fdf6ae860 2024-04-24T09:16:16.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.915+0000 7f38ce800640 1 --2- 172.21.15.83:0/1171264336 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556fdf6ae860 0x556fdf6b66a0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f38c8004750 tx=0x7f38c8013040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6cda281519d6fc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:16.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.915+0000 7f38af400640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:16:16.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.916+0000 7f38af400640 1 -- 172.21.15.83:0/1171264336 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c8011850 con 0x556fdf6ae860 2024-04-24T09:16:16.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.916+0000 7f38d79d1740 1 -- 172.21.15.83:0/1171264336 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38c8003d10 con 0x556fdf6ae860 2024-04-24T09:16:16.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.916+0000 7f38af400640 1 -- 172.21.15.83:0/1171264336 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(132..132 src has 1..132) ==== 9583+0+0 (secure 0 0 0) 0x7f38c80119f0 con 0x556fdf6ae860 2024-04-24T09:16:16.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.917+0000 7f38af400640 1 -- 172.21.15.83:0/1171264336 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f38c8011bf0 con 0x556fdf6ae860 2024-04-24T09:16:16.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.917+0000 7f38d79d1740 10 client.41009 fetch_fsmap learned FSMap version 69 2024-04-24T09:16:16.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.917+0000 7f38d79d1740 1 -- 172.21.15.83:0/1171264336 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x556fdf3f07d0 con 0x556fdf6ae860 2024-04-24T09:16:16.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.917+0000 7f38af400640 1 -- 172.21.15.83:0/1171264336 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f38c8003d10 con 0x556fdf6ae860 2024-04-24T09:16:16.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.918+0000 7f38af400640 1 -- 172.21.15.83:0/1171264336 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 69) ==== 8370+0+0 (secure 0 0 0) 0x7f38c8029040 con 0x556fdf6ae860 2024-04-24T09:16:16.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.918+0000 7f38d79d1740 10 client.41009 fetch_fsmap finished waiting for FSMap version 69 2024-04-24T09:16:16.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.918+0000 7f38d79d1740 10 client.41009 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:16:16.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.918+0000 7f38d79d1740 20 client.41009 populate_metadata read hostname 'smithi083' 2024-04-24T09:16:16.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.918+0000 7f38d79d1740 1 --2- 172.21.15.83:0/1171264336 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556fdf6c9aa0 0x556fdf6cbe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:16.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.918+0000 7f38d79d1740 4 client.41009 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:16:16.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.918+0000 7f38d79d1740 1 -- 172.21.15.83:0/1171264336 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556fdf621c10 con 0x556fdf6c9aa0 2024-04-24T09:16:16.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.918+0000 7f38cd400640 1 --2- 172.21.15.83:0/1171264336 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556fdf6c9aa0 0x556fdf6cbe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:16.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.919+0000 7f38af400640 10 client.41009 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:16:16.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.919+0000 7f38cd400640 1 --2- 172.21.15.83:0/1171264336 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556fdf6c9aa0 0x556fdf6cbe90 crc :-1 s=READY pgs=605 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9bcf0975fbdde01 server_cookie=833c97d2f136ec81 in_seq=0 out_seq=0 2024-04-24T09:16:16.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.920+0000 7f38af400640 1 -- 172.21.15.83:0/1171264336 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556fdf621c10 con 0x556fdf6c9aa0 2024-04-24T09:16:16.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.920+0000 7f38af400640 10 client.41009 handle_command_reply: tid=0 2024-04-24T09:16:16.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.920+0000 7f38d79d1740 1 client.41009 shutdown 2024-04-24T09:16:16.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.920+0000 7f38d79d1740 1 -- 172.21.15.83:0/1171264336 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556fdf6ae860 msgr2=0x556fdf6b66a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:16.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.920+0000 7f38d79d1740 1 --2- 172.21.15.83:0/1171264336 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556fdf6ae860 0x556fdf6b66a0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f38c8004750 tx=0x7f38c8013040 comp rx=0 tx=0).stop 2024-04-24T09:16:16.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.921+0000 7f38d79d1740 1 -- 172.21.15.83:0/1171264336 shutdown_connections 2024-04-24T09:16:16.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.921+0000 7f38d79d1740 1 --2- 172.21.15.83:0/1171264336 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556fdf6c9aa0 0x556fdf6cbe90 crc :-1 s=READY pgs=605 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:16.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.921+0000 7f38d79d1740 1 --2- 172.21.15.83:0/1171264336 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556fdf6b9850 0x556fdf6b7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:16.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.921+0000 7f38d79d1740 1 --2- 172.21.15.83:0/1171264336 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556fdf6af180 0x556fdf6b6be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:16.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.921+0000 7f38d79d1740 1 --2- 172.21.15.83:0/1171264336 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556fdf6ae860 0x556fdf6b66a0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:16.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.921+0000 7f38af400640 0 client.41009 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:16:16.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.921+0000 7f38d79d1740 1 -- 172.21.15.83:0/1171264336 >> 172.21.15.83:0/1171264336 conn(0x556fdf6aab80 msgr2=0x556fdf6b0860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:16.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.923+0000 7f38d79d1740 1 -- 172.21.15.83:0/1171264336 shutdown_connections 2024-04-24T09:16:16.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.923+0000 7f38d79d1740 1 -- 172.21.15.83:0/1171264336 wait complete. 2024-04-24T09:16:16.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.923+0000 7f38d79d1740 20 client.41009 trim_cache size 0 max 16384 2024-04-24T09:16:16.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.957+0000 7f38d79d1740 1 Processor -- start 2024-04-24T09:16:16.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.958+0000 7f38d79d1740 1 -- start start 2024-04-24T09:16:16.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.958+0000 7f38d79d1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556fdf6ae860 0x556fdf6c55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:16.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.958+0000 7f38d79d1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556fdf6af180 0x556fdf6b0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:16.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.958+0000 7f38d79d1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556fdf6b9850 0x556fdf6a9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:16.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.958+0000 7f38d79d1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f38d01be510 con 0x556fdf6ae860 2024-04-24T09:16:16.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.958+0000 7f38d79d1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f38d01c00b0 con 0x556fdf6b9850 2024-04-24T09:16:16.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.958+0000 7f38d79d1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f38d01bebe0 con 0x556fdf6af180 2024-04-24T09:16:16.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.958+0000 7f38ce800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556fdf6ae860 0x556fdf6c55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:16.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.958+0000 7f38ce800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556fdf6ae860 0x556fdf6c55d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52162/0 (socket says 172.21.15.83:52162) 2024-04-24T09:16:16.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.958+0000 7f38ce800640 1 -- 172.21.15.83:0/1004208487 learned_addr learned my addr 172.21.15.83:0/1004208487 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:16:16.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.959+0000 7f38ce800640 1 --2- 172.21.15.83:0/1004208487 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556fdf6b9850 0x556fdf6a9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:16.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.959+0000 7f38cd400640 1 --2- 172.21.15.83:0/1004208487 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556fdf6af180 0x556fdf6b0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:16.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.959+0000 7f38ce800640 1 -- 172.21.15.83:0/1004208487 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556fdf6af180 msgr2=0x556fdf6b0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:16.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.959+0000 7f38ce800640 1 --2- 172.21.15.83:0/1004208487 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556fdf6af180 0x556fdf6b0fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:16.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.959+0000 7f38ce800640 1 -- 172.21.15.83:0/1004208487 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556fdf6b9850 msgr2=0x556fdf6a9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:16.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.959+0000 7f38ce800640 1 --2- 172.21.15.83:0/1004208487 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556fdf6b9850 0x556fdf6a9aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:16.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.959+0000 7f38ce800640 1 -- 172.21.15.83:0/1004208487 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556fdf3263f0 con 0x556fdf6ae860 2024-04-24T09:16:16.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.959+0000 7f38ce800640 1 --2- 172.21.15.83:0/1004208487 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556fdf6ae860 0x556fdf6c55d0 secure :-1 s=READY pgs=2779 cs=0 l=1 rev1=1 crypto rx=0x7f38c8009f40 tx=0x7f38c8004870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bdd15b1b004c6e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:16.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.960+0000 7f38ba600640 1 -- 172.21.15.83:0/1004208487 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c8010d60 con 0x556fdf6ae860 2024-04-24T09:16:16.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.960+0000 7f38ba600640 1 -- 172.21.15.83:0/1004208487 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f38c8004e40 con 0x556fdf6ae860 2024-04-24T09:16:16.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.960+0000 7f38ba600640 1 -- 172.21.15.83:0/1004208487 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c8011850 con 0x556fdf6ae860 2024-04-24T09:16:16.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.961+0000 7f38d79d1740 1 -- 172.21.15.83:0/1004208487 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556fdf6ae860 msgr2=0x556fdf6c55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:16.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.961+0000 7f38d79d1740 1 --2- 172.21.15.83:0/1004208487 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556fdf6ae860 0x556fdf6c55d0 secure :-1 s=READY pgs=2779 cs=0 l=1 rev1=1 crypto rx=0x7f38c8009f40 tx=0x7f38c8004870 comp rx=0 tx=0).stop 2024-04-24T09:16:16.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.961+0000 7f38d79d1740 1 -- 172.21.15.83:0/1004208487 shutdown_connections 2024-04-24T09:16:16.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.961+0000 7f38d79d1740 1 --2- 172.21.15.83:0/1004208487 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556fdf6b9850 0x556fdf6a9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:16.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.961+0000 7f38d79d1740 1 --2- 172.21.15.83:0/1004208487 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556fdf6af180 0x556fdf6b0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:16.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.961+0000 7f38d79d1740 1 --2- 172.21.15.83:0/1004208487 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556fdf6ae860 0x556fdf6c55d0 unknown :-1 s=CLOSED pgs=2779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:16.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.961+0000 7f38d79d1740 1 -- 172.21.15.83:0/1004208487 >> 172.21.15.83:0/1004208487 conn(0x556fdf6aab80 msgr2=0x556fdf6a9390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:16.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.962+0000 7f38d79d1740 1 -- 172.21.15.83:0/1004208487 shutdown_connections 2024-04-24T09:16:16.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.962+0000 7f38d79d1740 1 -- 172.21.15.83:0/1004208487 wait complete. 2024-04-24T09:16:16.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.962+0000 7f38d79d1740 1 -- start start 2024-04-24T09:16:16.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.962+0000 7f38d79d1740 1 Processor -- start 2024-04-24T09:16:16.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.963+0000 7f38d79d1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556fdf6ae860 0x556fdf6c1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:16.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.963+0000 7f38d79d1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556fdf6af180 0x556fdf6bd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:16.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.963+0000 7f38d79d1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556fdf6b9850 0x556fdf6bd680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:16.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.963+0000 7f38d79d1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f38d01bf9e0 con 0x556fdf6af180 2024-04-24T09:16:16.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.963+0000 7f38d79d1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f38d01bf310 con 0x556fdf6b9850 2024-04-24T09:16:16.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.963+0000 7f38d79d1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f38d01bc510 con 0x556fdf6ae860 2024-04-24T09:16:16.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.963+0000 7f38d79d1740 10 client.0 _ll_register_callbacks cb 0x7f38bc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:16:16.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.963+0000 7f38cd400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556fdf6af180 0x556fdf6bd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:16.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.963+0000 7f38cd400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556fdf6af180 0x556fdf6bd140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52164/0 (socket says 172.21.15.83:52164) 2024-04-24T09:16:16.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.963+0000 7f38cd400640 1 -- 172.21.15.83:0/3323773503 learned_addr learned my addr 172.21.15.83:0/3323773503 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:16:16.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.964+0000 7f38ce800640 1 --2- 172.21.15.83:0/3323773503 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556fdf6ae860 0x556fdf6c1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:16.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.964+0000 7f38ce800640 1 --2- 172.21.15.83:0/3323773503 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556fdf6b9850 0x556fdf6bd680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:16.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.964+0000 7f38cd400640 1 -- 172.21.15.83:0/3323773503 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556fdf6ae860 msgr2=0x556fdf6c1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:16.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.964+0000 7f38cd400640 1 --2- 172.21.15.83:0/3323773503 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556fdf6ae860 0x556fdf6c1dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:16.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.964+0000 7f38cd400640 1 -- 172.21.15.83:0/3323773503 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556fdf6b9850 msgr2=0x556fdf6bd680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:16.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.964+0000 7f38cd400640 1 --2- 172.21.15.83:0/3323773503 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556fdf6b9850 0x556fdf6bd680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:16.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.964+0000 7f38cd400640 1 -- 172.21.15.83:0/3323773503 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556fdf2e60f0 con 0x556fdf6af180 2024-04-24T09:16:16.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.965+0000 7f38cd400640 1 --2- 172.21.15.83:0/3323773503 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556fdf6af180 0x556fdf6bd140 secure :-1 s=READY pgs=2780 cs=0 l=1 rev1=1 crypto rx=0x7f38d006b750 tx=0x7f38bc0051b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ed0ab6d100cb28b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:16.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.965+0000 7f38af400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:16:16.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.965+0000 7f38af400640 1 -- 172.21.15.83:0/3323773503 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38bc04b020 con 0x556fdf6af180 2024-04-24T09:16:16.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.965+0000 7f38af400640 1 -- 172.21.15.83:0/3323773503 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(132..132 src has 1..132) ==== 9583+0+0 (secure 0 0 0) 0x7f38bc047420 con 0x556fdf6af180 2024-04-24T09:16:16.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.965+0000 7f38d79d1740 1 -- 172.21.15.83:0/3323773503 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556fdf61ba80 con 0x556fdf6af180 2024-04-24T09:16:16.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.966+0000 7f38af400640 1 -- 172.21.15.83:0/3323773503 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f38bc035040 con 0x556fdf6af180 2024-04-24T09:16:16.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.966+0000 7f38d79d1740 10 client.23949 fetch_fsmap learned FSMap version 69 2024-04-24T09:16:16.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.966+0000 7f38d79d1740 1 -- 172.21.15.83:0/3323773503 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x556fdf5d9810 con 0x556fdf6af180 2024-04-24T09:16:16.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.966+0000 7f38af400640 1 -- 172.21.15.83:0/3323773503 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x556fdf61ba80 con 0x556fdf6af180 2024-04-24T09:16:16.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.966+0000 7f38af400640 1 -- 172.21.15.83:0/3323773503 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 69) ==== 8370+0+0 (secure 0 0 0) 0x7f38bc003680 con 0x556fdf6af180 2024-04-24T09:16:16.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.966+0000 7f38d79d1740 10 client.23949 fetch_fsmap finished waiting for FSMap version 69 2024-04-24T09:16:16.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.966+0000 7f38d79d1740 10 client.23949 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:16:16.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.966+0000 7f38d79d1740 20 client.23949 populate_metadata read hostname 'smithi083' 2024-04-24T09:16:16.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.966+0000 7f38d79d1740 1 --2- 172.21.15.83:0/3323773503 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556fdf730150 0x556fdf732570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:16.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.966+0000 7f38d79d1740 4 client.23949 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T09:16:16.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.966+0000 7f38d79d1740 1 -- 172.21.15.83:0/3323773503 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556fdf597360 con 0x556fdf730150 2024-04-24T09:16:16.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.967+0000 7f38ce800640 1 --2- 172.21.15.83:0/3323773503 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556fdf730150 0x556fdf732570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:16.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.968+0000 7f38ce800640 1 --2- 172.21.15.83:0/3323773503 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556fdf730150 0x556fdf732570 crc :-1 s=READY pgs=606 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b38151ab902f2364 server_cookie=a184401b5d0b0bab in_seq=0 out_seq=0 2024-04-24T09:16:16.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.968+0000 7f38af400640 10 client.23949 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:16:16.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.968+0000 7f38af400640 1 -- 172.21.15.83:0/3323773503 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x556fdf597360 con 0x556fdf730150 2024-04-24T09:16:16.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.968+0000 7f38af400640 10 client.23949 handle_command_reply: tid=0 2024-04-24T09:16:16.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.968+0000 7f38d79d1740 1 client.23949 shutdown 2024-04-24T09:16:16.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.968+0000 7f38d79d1740 1 -- 172.21.15.83:0/3323773503 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556fdf6af180 msgr2=0x556fdf6bd140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:16.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.968+0000 7f38d79d1740 1 --2- 172.21.15.83:0/3323773503 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556fdf6af180 0x556fdf6bd140 secure :-1 s=READY pgs=2780 cs=0 l=1 rev1=1 crypto rx=0x7f38d006b750 tx=0x7f38bc0051b0 comp rx=0 tx=0).stop 2024-04-24T09:16:16.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.968+0000 7f38d79d1740 1 -- 172.21.15.83:0/3323773503 shutdown_connections 2024-04-24T09:16:16.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.968+0000 7f38d79d1740 1 --2- 172.21.15.83:0/3323773503 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x556fdf730150 0x556fdf732570 crc :-1 s=READY pgs=606 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:16.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.968+0000 7f38d79d1740 1 --2- 172.21.15.83:0/3323773503 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556fdf6b9850 0x556fdf6bd680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:16.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.968+0000 7f38d79d1740 1 --2- 172.21.15.83:0/3323773503 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556fdf6af180 0x556fdf6bd140 unknown :-1 s=CLOSED pgs=2780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:16.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.968+0000 7f38d79d1740 1 --2- 172.21.15.83:0/3323773503 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556fdf6ae860 0x556fdf6c1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:16.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.968+0000 7f38d79d1740 1 -- 172.21.15.83:0/3323773503 >> 172.21.15.83:0/3323773503 conn(0x556fdf6aab80 msgr2=0x556fdf6b8760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:16.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.968+0000 7f38af400640 0 client.23949 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:16:16.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.969+0000 7f38d79d1740 1 -- 172.21.15.83:0/3323773503 shutdown_connections 2024-04-24T09:16:16.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.969+0000 7f38d79d1740 1 -- 172.21.15.83:0/3323773503 wait complete. 2024-04-24T09:16:16.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.969+0000 7f38d79d1740 20 client.23949 trim_cache size 0 max 16384 2024-04-24T09:16:16.970 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T09:16:16.971 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T09:16:16.971 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-04-24T09:16:16.971 INFO:teuthology.orchestra.run.smithi083.stdout: "465e20cc-88c5-4f84-8ace-5ca9be6b1e93": { 2024-04-24T09:16:16.971 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-04-24T09:16:16.971 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "465e20cc-88c5-4f84-8ace-5ca9be6b1e93", 2024-04-24T09:16:16.971 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-04-24T09:16:16.971 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:16:16.971 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:16:16.971 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T09:16:16.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.970+0000 7f38d4c00640 1 -- 172.21.15.83:0/3801744651 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f38b40a3f40 msgr2=0x7f38b40a6400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:16.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.970+0000 7f38d4c00640 1 --2- 172.21.15.83:0/3801744651 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f38b40a3f40 0x7f38b40a6400 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x556fdf6afe20 tx=0x7f38c0066780 comp rx=0 tx=0).stop 2024-04-24T09:16:16.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.970+0000 7f38d4c00640 1 -- 172.21.15.83:0/3801744651 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38d01044b0 msgr2=0x7f38d011a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:16.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.970+0000 7f38d4c00640 1 --2- 172.21.15.83:0/3801744651 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38d01044b0 0x7f38d011a7d0 secure :-1 s=READY pgs=2777 cs=0 l=1 rev1=1 crypto rx=0x7f38c000d980 tx=0x7f38c000de40 comp rx=0 tx=0).stop 2024-04-24T09:16:16.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.970+0000 7f38d4c00640 1 -- 172.21.15.83:0/3801744651 shutdown_connections 2024-04-24T09:16:16.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.970+0000 7f38d4c00640 1 --2- 172.21.15.83:0/3801744651 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f38b40a3f40 0x7f38b40a6400 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:16.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.970+0000 7f38d4c00640 1 --2- 172.21.15.83:0/3801744651 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f38d0110b30 0x7f38d0110fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:16.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.970+0000 7f38d4c00640 1 --2- 172.21.15.83:0/3801744651 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f38d0109580 0x7f38d01105f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:16.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.970+0000 7f38d4c00640 1 --2- 172.21.15.83:0/3801744651 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38d01044b0 0x7f38d011a7d0 unknown :-1 s=CLOSED pgs=2777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:16.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.970+0000 7f38d4c00640 1 -- 172.21.15.83:0/3801744651 >> 172.21.15.83:0/3801744651 conn(0x7f38d00ffd90 msgr2=0x7f38d0079870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:16.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.970+0000 7f38d4c00640 1 -- 172.21.15.83:0/3801744651 shutdown_connections 2024-04-24T09:16:16.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:16.970+0000 7f38d4c00640 1 -- 172.21.15.83:0/3801744651 wait complete. 2024-04-24T09:16:16.989 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:16:16 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:16:16.989 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:16:16 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:16:16.989 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:16:16 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:16:16.992 INFO:tasks.cephfs.filesystem:scrub status for tag:465e20cc-88c5-4f84-8ace-5ca9be6b1e93 - {'path': '/', 'tag': '465e20cc-88c5-4f84-8ace-5ca9be6b1e93', 'options': 'recursive,force'} 2024-04-24T09:16:17.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:16:16 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:16:17.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:16:16 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:16:17.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:16:16 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:16:17.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:16:16 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:16:17.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:16:16 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:16:17.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:16:16 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:16:17.217 INFO:tasks.workunit.client.0.smithi083.stdout: 10 79231 6.52 MB/sec execute 528 sec latency 268.181 ms 2024-04-24T09:16:18.204 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:16:17 smithi083 ceph-mon[27907]: scrub queued for path: / 2024-04-24T09:16:18.204 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:16:17 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:16:18.204 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:16:17 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:16:18.204 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:16:17 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:16:18.204 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:16:17 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:16:18.205 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:16:17 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:16:18.205 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:16:17 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:16:18.205 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:16:17 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:16:18.205 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:16:17 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:16:18.205 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:16:17 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:16:18.205 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:16:17 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:16:18.205 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:16:17 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:16:18.205 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:16:17 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:16:18.205 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:16:17 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:16:18.205 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:16:17 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:16:18.205 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:16:17 smithi083 ceph-mon[27907]: pgmap v781: 129 pgs: 129 active+clean; 1.2 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.5 MiB/s wr, 297 op/s 2024-04-24T09:16:18.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:16:17 smithi138 ceph-mon[31441]: scrub queued for path: / 2024-04-24T09:16:18.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:16:17 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:16:18.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:16:17 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:16:18.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:16:17 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:16:18.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:16:17 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:16:18.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:16:17 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:16:18.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:16:17 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:16:18.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:16:17 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:16:18.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:16:17 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:16:18.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:16:17 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:16:18.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:16:17 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:16:18.206 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:16:17 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:16:18.207 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:16:17 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:16:18.207 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:16:17 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:16:18.207 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:16:17 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:16:18.207 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:16:17 smithi138 ceph-mon[31441]: pgmap v781: 129 pgs: 129 active+clean; 1.2 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.5 MiB/s wr, 297 op/s 2024-04-24T09:16:18.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:16:17 smithi189 ceph-mon[31791]: scrub queued for path: / 2024-04-24T09:16:18.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:16:17 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:16:18.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:16:17 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:16:18.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:16:17 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:16:18.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:16:17 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:16:18.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:16:17 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:16:18.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:16:17 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:16:18.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:16:17 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:16:18.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:16:17 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:16:18.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:16:17 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:16:18.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:16:17 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:16:18.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:16:17 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:16:18.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:16:17 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:16:18.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:16:17 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:16:18.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:16:17 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:16:18.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:16:17 smithi189 ceph-mon[31791]: pgmap v781: 129 pgs: 129 active+clean; 1.2 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.5 MiB/s wr, 297 op/s 2024-04-24T09:16:18.218 INFO:tasks.workunit.client.0.smithi083.stdout: 10 79342 6.51 MB/sec execute 529 sec latency 78.498 ms 2024-04-24T09:16:18.343 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.343+0000 7f9a14a00640 1 -- 172.21.15.83:0/3135292535 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a1006e2f0 msgr2=0x7f9a10109000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:18.344 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.343+0000 7f9a14a00640 1 --2- 172.21.15.83:0/3135292535 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a1006e2f0 0x7f9a10109000 secure :-1 s=READY pgs=2781 cs=0 l=1 rev1=1 crypto rx=0x7f9a00009900 tx=0x7f9a0002a3f0 comp rx=0 tx=0).stop 2024-04-24T09:16:18.344 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.343+0000 7f9a14a00640 1 -- 172.21.15.83:0/3135292535 shutdown_connections 2024-04-24T09:16:18.344 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.343+0000 7f9a14a00640 1 --2- 172.21.15.83:0/3135292535 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9a10114d40 0x7f9a10117130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:18.344 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.343+0000 7f9a14a00640 1 --2- 172.21.15.83:0/3135292535 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a101095d0 0x7f9a10109a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:18.344 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.343+0000 7f9a14a00640 1 --2- 172.21.15.83:0/3135292535 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a1006e2f0 0x7f9a10109000 unknown :-1 s=CLOSED pgs=2781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:18.344 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.343+0000 7f9a14a00640 1 -- 172.21.15.83:0/3135292535 >> 172.21.15.83:0/3135292535 conn(0x7f9a1006cf40 msgr2=0x7f9a1006d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:18.344 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.343+0000 7f9a14a00640 1 -- 172.21.15.83:0/3135292535 shutdown_connections 2024-04-24T09:16:18.344 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.343+0000 7f9a14a00640 1 -- 172.21.15.83:0/3135292535 wait complete. 2024-04-24T09:16:18.345 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.345+0000 7f9a14a00640 1 Processor -- start 2024-04-24T09:16:18.345 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.345+0000 7f9a14a00640 1 -- start start 2024-04-24T09:16:18.346 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.345+0000 7f9a14a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a1006e2f0 0x7f9a10113b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:18.346 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.345+0000 7f9a14a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a101095d0 0x7f9a10114070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:18.346 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.345+0000 7f9a14a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9a10114d40 0x7f9a101abdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:18.346 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.345+0000 7f9a14a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9a1011a910 con 0x7f9a101095d0 2024-04-24T09:16:18.346 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.345+0000 7f9a14a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9a1011a790 con 0x7f9a1006e2f0 2024-04-24T09:16:18.347 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.345+0000 7f9a14a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9a1011aa90 con 0x7f9a10114d40 2024-04-24T09:16:18.347 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.345+0000 7f9a0de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a101095d0 0x7f9a10114070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:18.347 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.346+0000 7f9a0de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a101095d0 0x7f9a10114070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52186/0 (socket says 172.21.15.83:52186) 2024-04-24T09:16:18.347 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.346+0000 7f9a0e800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a1006e2f0 0x7f9a10113b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:18.347 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.346+0000 7f9a0de00640 1 -- 172.21.15.83:0/2197465260 learned_addr learned my addr 172.21.15.83:0/2197465260 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:16:18.347 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.346+0000 7f9a0f200640 1 --2- 172.21.15.83:0/2197465260 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9a10114d40 0x7f9a101abdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:18.347 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.346+0000 7f9a0de00640 1 -- 172.21.15.83:0/2197465260 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9a10114d40 msgr2=0x7f9a101abdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:18.347 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.346+0000 7f9a0de00640 1 --2- 172.21.15.83:0/2197465260 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9a10114d40 0x7f9a101abdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:18.347 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.346+0000 7f9a0de00640 1 -- 172.21.15.83:0/2197465260 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a1006e2f0 msgr2=0x7f9a10113b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:18.348 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.346+0000 7f9a0de00640 1 --2- 172.21.15.83:0/2197465260 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a1006e2f0 0x7f9a10113b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:18.348 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.346+0000 7f9a0de00640 1 -- 172.21.15.83:0/2197465260 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a101ac500 con 0x7f9a101095d0 2024-04-24T09:16:18.348 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.347+0000 7f9a0f200640 1 --2- 172.21.15.83:0/2197465260 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9a10114d40 0x7f9a101abdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:16:18.348 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.347+0000 7f9a0e800640 1 --2- 172.21.15.83:0/2197465260 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a1006e2f0 0x7f9a10113b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:16:18.348 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.347+0000 7f9a0de00640 1 --2- 172.21.15.83:0/2197465260 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a101095d0 0x7f9a10114070 secure :-1 s=READY pgs=2782 cs=0 l=1 rev1=1 crypto rx=0x7f99fc00e9c0 tx=0x7f99fc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:18.348 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.347+0000 7f9a07400640 1 -- 172.21.15.83:0/2197465260 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99fc00cd80 con 0x7f9a101095d0 2024-04-24T09:16:18.349 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.347+0000 7f9a07400640 1 -- 172.21.15.83:0/2197465260 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f99fc0035f0 con 0x7f9a101095d0 2024-04-24T09:16:18.349 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.347+0000 7f9a07400640 1 -- 172.21.15.83:0/2197465260 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99fc010690 con 0x7f9a101095d0 2024-04-24T09:16:18.349 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.347+0000 7f9a14a00640 1 -- 172.21.15.83:0/2197465260 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a101ac7f0 con 0x7f9a101095d0 2024-04-24T09:16:18.349 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.348+0000 7f9a14a00640 1 -- 172.21.15.83:0/2197465260 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a101b4080 con 0x7f9a101095d0 2024-04-24T09:16:18.352 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.352+0000 7f9a1820c740 1 Processor -- start 2024-04-24T09:16:18.352 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.352+0000 7f9a1820c740 1 -- start start 2024-04-24T09:16:18.352 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.352+0000 7f9a1820c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5559244253c0 0x5559244257a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:18.353 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.352+0000 7f9a07400640 1 -- 172.21.15.83:0/2197465260 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f99fc0037a0 con 0x7f9a101095d0 2024-04-24T09:16:18.353 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.352+0000 7f9a1820c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555924425ce0 0x555924430060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:18.353 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.352+0000 7f9a1820c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559244305a0 0x555924432990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:18.353 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.352+0000 7f9a1820c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55592438b900 con 0x555924425ce0 2024-04-24T09:16:18.353 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.352+0000 7f9a1820c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555924331f20 con 0x5559244253c0 2024-04-24T09:16:18.353 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.352+0000 7f9a1820c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5559243878c0 con 0x5559244305a0 2024-04-24T09:16:18.353 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.353+0000 7f9a0f200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5559244253c0 0x5559244257a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:18.353 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.353+0000 7f9a0f200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5559244253c0 0x5559244257a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:51598/0 (socket says 172.21.15.83:51598) 2024-04-24T09:16:18.353 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.353+0000 7f9a0e800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555924425ce0 0x555924430060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:18.354 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.353+0000 7f9a0f200640 1 -- 172.21.15.83:0/1744913513 learned_addr learned my addr 172.21.15.83:0/1744913513 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:16:18.354 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.353+0000 7f9a0de00640 1 --2- 172.21.15.83:0/1744913513 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559244305a0 0x555924432990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:18.354 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.353+0000 7f9a0f200640 1 -- 172.21.15.83:0/1744913513 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559244305a0 msgr2=0x555924432990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:18.354 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.353+0000 7f9a0f200640 1 --2- 172.21.15.83:0/1744913513 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559244305a0 0x555924432990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:18.354 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.353+0000 7f9a0f200640 1 -- 172.21.15.83:0/1744913513 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555924425ce0 msgr2=0x555924430060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:18.354 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.353+0000 7f9a0f200640 1 --2- 172.21.15.83:0/1744913513 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555924425ce0 0x555924430060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:18.355 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.353+0000 7f9a0f200640 1 -- 172.21.15.83:0/1744913513 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555924356e10 con 0x5559244253c0 2024-04-24T09:16:18.355 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.353+0000 7f9a0e800640 1 --2- 172.21.15.83:0/1744913513 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555924425ce0 0x555924430060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:16:18.355 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.354+0000 7f9a0f200640 1 --2- 172.21.15.83:0/1744913513 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5559244253c0 0x5559244257a0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f9a08009e10 tx=0x7f9a0800f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16e5087d518d05ea server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:18.355 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.354+0000 7f99fb400640 1 -- 172.21.15.83:0/1744913513 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a08004d20 con 0x5559244253c0 2024-04-24T09:16:18.356 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.354+0000 7f99fb400640 1 -- 172.21.15.83:0/1744913513 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9a080054d0 con 0x5559244253c0 2024-04-24T09:16:18.356 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.354+0000 7f99fb400640 1 -- 172.21.15.83:0/1744913513 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a08011800 con 0x5559244253c0 2024-04-24T09:16:18.357 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.354+0000 7f9a07400640 1 --2- 172.21.15.83:0/2197465260 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f99f40a3ee0 0x7f99f40a63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:18.357 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.355+0000 7f9a07400640 1 -- 172.21.15.83:0/2197465260 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 9583+0+0 (secure 0 0 0) 0x7f99fc0d0fc0 con 0x7f9a101095d0 2024-04-24T09:16:18.357 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.355+0000 7f9a0e800640 1 --2- 172.21.15.83:0/2197465260 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f99f40a3ee0 0x7f99f40a63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:18.357 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.355+0000 7f9a1820c740 1 -- 172.21.15.83:0/1744913513 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5559244253c0 msgr2=0x5559244257a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:18.357 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.355+0000 7f9a1820c740 1 --2- 172.21.15.83:0/1744913513 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5559244253c0 0x5559244257a0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f9a08009e10 tx=0x7f9a0800f730 comp rx=0 tx=0).stop 2024-04-24T09:16:18.357 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.355+0000 7f9a1820c740 1 -- 172.21.15.83:0/1744913513 shutdown_connections 2024-04-24T09:16:18.357 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.355+0000 7f9a1820c740 1 --2- 172.21.15.83:0/1744913513 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559244305a0 0x555924432990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:18.357 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.355+0000 7f9a1820c740 1 --2- 172.21.15.83:0/1744913513 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555924425ce0 0x555924430060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:18.357 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.355+0000 7f9a1820c740 1 --2- 172.21.15.83:0/1744913513 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5559244253c0 0x5559244257a0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:18.357 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.355+0000 7f9a1820c740 1 -- 172.21.15.83:0/1744913513 >> 172.21.15.83:0/1744913513 conn(0x5559244218d0 msgr2=0x555924421cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:18.357 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.356+0000 7f9a1820c740 1 -- 172.21.15.83:0/1744913513 shutdown_connections 2024-04-24T09:16:18.357 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.356+0000 7f9a1820c740 1 -- 172.21.15.83:0/1744913513 wait complete. 2024-04-24T09:16:18.357 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.356+0000 7f9a0e800640 1 --2- 172.21.15.83:0/2197465260 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f99f40a3ee0 0x7f99f40a63a0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f9a00005ab0 tx=0x7f9a00005a20 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:18.357 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.356+0000 7f9a1820c740 1 -- start start 2024-04-24T09:16:18.359 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.357+0000 7f9a1820c740 1 Processor -- start 2024-04-24T09:16:18.359 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.357+0000 7f9a1820c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559244253c0 0x55592442d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:18.359 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.357+0000 7f9a1820c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555924425ce0 0x55592442daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:18.359 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.357+0000 7f9a1820c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5559244305a0 0x55592442e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:18.359 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.357+0000 7f9a1820c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5559244330e0 con 0x5559244305a0 2024-04-24T09:16:18.359 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.357+0000 7f9a1820c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5559243452c0 con 0x555924425ce0 2024-04-24T09:16:18.359 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.357+0000 7f9a1820c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x555924433260 con 0x5559244253c0 2024-04-24T09:16:18.359 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.357+0000 7f9a1820c740 10 client.0 _ll_register_callbacks cb 0x55592409da00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:16:18.359 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.358+0000 7f9a0f200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5559244305a0 0x55592442e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:18.359 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.358+0000 7f9a0f200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5559244305a0 0x55592442e030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52206/0 (socket says 172.21.15.83:52206) 2024-04-24T09:16:18.359 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.358+0000 7f9a0f200640 1 -- 172.21.15.83:0/3538129209 learned_addr learned my addr 172.21.15.83:0/3538129209 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:16:18.360 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.358+0000 7f9a0e800640 1 --2- 172.21.15.83:0/3538129209 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559244253c0 0x55592442d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:18.360 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.358+0000 7f9a0de00640 1 --2- 172.21.15.83:0/3538129209 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555924425ce0 0x55592442daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:18.360 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.358+0000 7f9a0f200640 1 -- 172.21.15.83:0/3538129209 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559244253c0 msgr2=0x55592442d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:18.360 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.358+0000 7f9a0f200640 1 --2- 172.21.15.83:0/3538129209 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559244253c0 0x55592442d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:18.360 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.358+0000 7f9a0f200640 1 -- 172.21.15.83:0/3538129209 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555924425ce0 msgr2=0x55592442daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:18.360 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.358+0000 7f9a0f200640 1 --2- 172.21.15.83:0/3538129209 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555924425ce0 0x55592442daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:18.360 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.358+0000 7f9a0f200640 1 -- 172.21.15.83:0/3538129209 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555924355fd0 con 0x5559244305a0 2024-04-24T09:16:18.360 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.359+0000 7f9a0f200640 1 --2- 172.21.15.83:0/3538129209 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5559244305a0 0x55592442e030 secure :-1 s=READY pgs=2783 cs=0 l=1 rev1=1 crypto rx=0x7f9a08005230 tx=0x7f9a08005260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=215f0415c48ed43e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:18.360 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.359+0000 7f99efe00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:16:18.361 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.359+0000 7f99efe00640 1 -- 172.21.15.83:0/3538129209 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a08028060 con 0x5559244305a0 2024-04-24T09:16:18.361 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.359+0000 7f99efe00640 1 -- 172.21.15.83:0/3538129209 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(132..132 src has 1..132) ==== 9583+0+0 (secure 0 0 0) 0x7f9a08005890 con 0x5559244305a0 2024-04-24T09:16:18.361 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.359+0000 7f9a1820c740 1 -- 172.21.15.83:0/3538129209 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a080054d0 con 0x5559244305a0 2024-04-24T09:16:18.361 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.360+0000 7f99efe00640 1 -- 172.21.15.83:0/3538129209 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f9a0802f050 con 0x5559244305a0 2024-04-24T09:16:18.362 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.360+0000 7f9a1820c740 10 client.23961 fetch_fsmap learned FSMap version 69 2024-04-24T09:16:18.362 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.360+0000 7f9a1820c740 1 -- 172.21.15.83:0/3538129209 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x55592409e210 con 0x5559244305a0 2024-04-24T09:16:18.362 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.360+0000 7f99efe00640 1 -- 172.21.15.83:0/3538129209 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9a080054d0 con 0x5559244305a0 2024-04-24T09:16:18.362 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.360+0000 7f99efe00640 1 -- 172.21.15.83:0/3538129209 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 69) ==== 8370+0+0 (secure 0 0 0) 0x7f9a08011c70 con 0x5559244305a0 2024-04-24T09:16:18.362 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.360+0000 7f9a1820c740 10 client.23961 fetch_fsmap finished waiting for FSMap version 69 2024-04-24T09:16:18.362 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.360+0000 7f9a1820c740 10 client.23961 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:16:18.362 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.360+0000 7f9a1820c740 20 client.23961 populate_metadata read hostname 'smithi083' 2024-04-24T09:16:18.362 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.360+0000 7f9a1820c740 1 --2- 172.21.15.83:0/3538129209 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x555924440680 0x555924442a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:18.362 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.360+0000 7f9a1820c740 4 client.23961 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:16:18.362 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.360+0000 7f9a1820c740 1 -- 172.21.15.83:0/3538129209 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555924399320 con 0x555924440680 2024-04-24T09:16:18.362 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.361+0000 7f9a0e800640 1 --2- 172.21.15.83:0/3538129209 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x555924440680 0x555924442a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:18.363 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.362+0000 7f9a0e800640 1 --2- 172.21.15.83:0/3538129209 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x555924440680 0x555924442a70 crc :-1 s=READY pgs=607 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45f09c5dd79259dd server_cookie=b4632388dfda26d3 in_seq=0 out_seq=0 2024-04-24T09:16:18.364 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.362+0000 7f99efe00640 10 client.23961 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:16:18.364 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.364+0000 7f99efe00640 1 -- 172.21.15.83:0/3538129209 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555924399320 con 0x555924440680 2024-04-24T09:16:18.364 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.364+0000 7f99efe00640 10 client.23961 handle_command_reply: tid=0 2024-04-24T09:16:18.364 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.364+0000 7f9a1820c740 1 client.23961 shutdown 2024-04-24T09:16:18.365 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.365+0000 7f9a1820c740 1 -- 172.21.15.83:0/3538129209 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5559244305a0 msgr2=0x55592442e030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:18.365 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.365+0000 7f9a1820c740 1 --2- 172.21.15.83:0/3538129209 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5559244305a0 0x55592442e030 secure :-1 s=READY pgs=2783 cs=0 l=1 rev1=1 crypto rx=0x7f9a08005230 tx=0x7f9a08005260 comp rx=0 tx=0).stop 2024-04-24T09:16:18.365 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.365+0000 7f9a1820c740 1 -- 172.21.15.83:0/3538129209 shutdown_connections 2024-04-24T09:16:18.365 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.365+0000 7f9a1820c740 1 --2- 172.21.15.83:0/3538129209 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x555924440680 0x555924442a70 crc :-1 s=READY pgs=607 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:18.365 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.365+0000 7f99efe00640 0 client.23961 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:16:18.365 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.365+0000 7f9a1820c740 1 --2- 172.21.15.83:0/3538129209 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5559244305a0 0x55592442e030 unknown :-1 s=CLOSED pgs=2783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:18.365 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.365+0000 7f9a1820c740 1 --2- 172.21.15.83:0/3538129209 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555924425ce0 0x55592442daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:18.365 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.365+0000 7f9a1820c740 1 --2- 172.21.15.83:0/3538129209 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559244253c0 0x55592442d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:18.366 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.365+0000 7f9a1820c740 1 -- 172.21.15.83:0/3538129209 >> 172.21.15.83:0/3538129209 conn(0x5559244218d0 msgr2=0x555924427650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:18.366 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.365+0000 7f9a1820c740 1 -- 172.21.15.83:0/3538129209 shutdown_connections 2024-04-24T09:16:18.366 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.365+0000 7f9a1820c740 1 -- 172.21.15.83:0/3538129209 wait complete. 2024-04-24T09:16:18.366 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.366+0000 7f9a1820c740 20 client.23961 trim_cache size 0 max 16384 2024-04-24T09:16:18.396 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.396+0000 7f9a1820c740 1 Processor -- start 2024-04-24T09:16:18.397 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.396+0000 7f9a1820c740 1 -- start start 2024-04-24T09:16:18.397 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.396+0000 7f9a1820c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559244253c0 0x55592442fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:18.397 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.396+0000 7f9a1820c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555924425ce0 0x55592442ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:18.397 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.396+0000 7f9a1820c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5559244305a0 0x555924421210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:18.397 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.396+0000 7f9a1820c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9a101be2c0 con 0x555924425ce0 2024-04-24T09:16:18.397 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.396+0000 7f9a1820c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9a101bfe60 con 0x5559244305a0 2024-04-24T09:16:18.397 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.396+0000 7f9a1820c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9a101be990 con 0x5559244253c0 2024-04-24T09:16:18.397 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.397+0000 7f9a0de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555924425ce0 0x55592442ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:18.398 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.397+0000 7f9a0e800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559244253c0 0x55592442fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:18.398 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.397+0000 7f9a0e800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559244253c0 0x55592442fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:35790/0 (socket says 172.21.15.83:35790) 2024-04-24T09:16:18.398 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.398+0000 7f9a0e800640 1 -- 172.21.15.83:0/4057155993 learned_addr learned my addr 172.21.15.83:0/4057155993 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:16:18.398 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.397+0000 7f9a0de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555924425ce0 0x55592442ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52216/0 (socket says 172.21.15.83:52216) 2024-04-24T09:16:18.399 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.398+0000 7f9a0de00640 1 -- 172.21.15.83:0/4057155993 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559244253c0 msgr2=0x55592442fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:18.399 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.398+0000 7f9a0f200640 1 --2- 172.21.15.83:0/4057155993 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5559244305a0 0x555924421210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:18.399 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.399+0000 7f9a0de00640 1 --2- 172.21.15.83:0/4057155993 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559244253c0 0x55592442fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:18.399 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.399+0000 7f9a0de00640 1 -- 172.21.15.83:0/4057155993 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5559244305a0 msgr2=0x555924421210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:18.399 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.399+0000 7f9a0de00640 1 --2- 172.21.15.83:0/4057155993 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5559244305a0 0x555924421210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:18.400 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.399+0000 7f9a0de00640 1 -- 172.21.15.83:0/4057155993 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55592416a620 con 0x555924425ce0 2024-04-24T09:16:18.400 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.399+0000 7f9a0e800640 1 --2- 172.21.15.83:0/4057155993 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559244253c0 0x55592442fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:16:18.400 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.400+0000 7f9a0f200640 1 --2- 172.21.15.83:0/4057155993 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5559244305a0 0x555924421210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:16:18.400 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.400+0000 7f9a0de00640 1 --2- 172.21.15.83:0/4057155993 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555924425ce0 0x55592442ceb0 secure :-1 s=READY pgs=2784 cs=0 l=1 rev1=1 crypto rx=0x7f99fc0ce290 tx=0x7f99fc003e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c568047b4a5f9788 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:18.402 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.400+0000 7f99faa00640 1 -- 172.21.15.83:0/4057155993 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99fc003b10 con 0x555924425ce0 2024-04-24T09:16:18.402 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.402+0000 7f99faa00640 1 -- 172.21.15.83:0/4057155993 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f99fc09d1e0 con 0x555924425ce0 2024-04-24T09:16:18.403 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.402+0000 7f99faa00640 1 -- 172.21.15.83:0/4057155993 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99fc099150 con 0x555924425ce0 2024-04-24T09:16:18.404 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.403+0000 7f9a1820c740 1 -- 172.21.15.83:0/4057155993 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555924425ce0 msgr2=0x55592442ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:18.404 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.403+0000 7f9a1820c740 1 --2- 172.21.15.83:0/4057155993 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555924425ce0 0x55592442ceb0 secure :-1 s=READY pgs=2784 cs=0 l=1 rev1=1 crypto rx=0x7f99fc0ce290 tx=0x7f99fc003e10 comp rx=0 tx=0).stop 2024-04-24T09:16:18.405 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.404+0000 7f9a1820c740 1 -- 172.21.15.83:0/4057155993 shutdown_connections 2024-04-24T09:16:18.405 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.404+0000 7f9a1820c740 1 --2- 172.21.15.83:0/4057155993 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5559244305a0 0x555924421210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:18.405 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.404+0000 7f9a1820c740 1 --2- 172.21.15.83:0/4057155993 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555924425ce0 0x55592442ceb0 unknown :-1 s=CLOSED pgs=2784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:18.405 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.404+0000 7f9a1820c740 1 --2- 172.21.15.83:0/4057155993 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559244253c0 0x55592442fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:18.405 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.404+0000 7f9a1820c740 1 -- 172.21.15.83:0/4057155993 >> 172.21.15.83:0/4057155993 conn(0x5559244218d0 msgr2=0x555924427600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:18.405 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.404+0000 7f9a1820c740 1 -- 172.21.15.83:0/4057155993 shutdown_connections 2024-04-24T09:16:18.405 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.405+0000 7f9a1820c740 1 -- 172.21.15.83:0/4057155993 wait complete. 2024-04-24T09:16:18.405 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.405+0000 7f9a1820c740 1 -- start start 2024-04-24T09:16:18.406 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.405+0000 7f9a1820c740 1 Processor -- start 2024-04-24T09:16:18.406 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.406+0000 7f9a1820c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5559244253c0 0x555924438a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:18.406 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.406+0000 7f9a1820c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555924425ce0 0x55592442a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:18.407 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.406+0000 7f9a1820c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559244305a0 0x55592442a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:18.407 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.406+0000 7f9a1820c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9a101bf790 con 0x555924425ce0 2024-04-24T09:16:18.407 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.406+0000 7f9a1820c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9a101bf0c0 con 0x5559244253c0 2024-04-24T09:16:18.407 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.406+0000 7f9a1820c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9a101bc2c0 con 0x5559244305a0 2024-04-24T09:16:18.407 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.406+0000 7f9a1820c740 10 client.0 _ll_register_callbacks cb 0x7f9a08009c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:16:18.407 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.407+0000 7f9a0e800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5559244253c0 0x555924438a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:18.407 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.407+0000 7f9a0de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555924425ce0 0x55592442a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:18.408 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.407+0000 7f9a0e800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5559244253c0 0x555924438a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:51618/0 (socket says 172.21.15.83:51618) 2024-04-24T09:16:18.408 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.407+0000 7f9a0e800640 1 -- 172.21.15.83:0/4240456501 learned_addr learned my addr 172.21.15.83:0/4240456501 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:16:18.408 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.407+0000 7f9a0f200640 1 --2- 172.21.15.83:0/4240456501 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559244305a0 0x55592442a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:18.409 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.408+0000 7f9a0de00640 1 -- 172.21.15.83:0/4240456501 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559244305a0 msgr2=0x55592442a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:18.409 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.408+0000 7f9a0de00640 1 --2- 172.21.15.83:0/4240456501 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559244305a0 0x55592442a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:18.409 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.408+0000 7f9a0de00640 1 -- 172.21.15.83:0/4240456501 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5559244253c0 msgr2=0x555924438a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:18.409 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.408+0000 7f9a0de00640 1 --2- 172.21.15.83:0/4240456501 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5559244253c0 0x555924438a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:18.409 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.408+0000 7f9a0de00640 1 -- 172.21.15.83:0/4240456501 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559244a58b0 con 0x555924425ce0 2024-04-24T09:16:18.409 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.408+0000 7f9a0de00640 1 --2- 172.21.15.83:0/4240456501 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555924425ce0 0x55592442a120 secure :-1 s=READY pgs=2785 cs=0 l=1 rev1=1 crypto rx=0x7f99fc018000 tx=0x7f99fc003d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9351d74176d12017 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:18.409 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.408+0000 7f99efe00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:16:18.409 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.409+0000 7f99efe00640 1 -- 172.21.15.83:0/4240456501 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99fc0b5020 con 0x555924425ce0 2024-04-24T09:16:18.409 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.409+0000 7f99efe00640 1 -- 172.21.15.83:0/4240456501 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(132..132 src has 1..132) ==== 9583+0+0 (secure 0 0 0) 0x7f99fc0a54c0 con 0x555924425ce0 2024-04-24T09:16:18.410 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.409+0000 7f9a1820c740 1 -- 172.21.15.83:0/4240456501 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99fc09d1e0 con 0x555924425ce0 2024-04-24T09:16:18.410 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.409+0000 7f99efe00640 1 -- 172.21.15.83:0/4240456501 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f99fc09cae0 con 0x555924425ce0 2024-04-24T09:16:18.410 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.410+0000 7f9a1820c740 10 client.23967 fetch_fsmap learned FSMap version 69 2024-04-24T09:16:18.410 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.410+0000 7f9a1820c740 1 -- 172.21.15.83:0/4240456501 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x5559244a69a0 con 0x555924425ce0 2024-04-24T09:16:18.411 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.410+0000 7f99efe00640 1 -- 172.21.15.83:0/4240456501 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f99fc09d1e0 con 0x555924425ce0 2024-04-24T09:16:18.411 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.411+0000 7f99efe00640 1 -- 172.21.15.83:0/4240456501 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 69) ==== 8370+0+0 (secure 0 0 0) 0x7f99fc09b9a0 con 0x555924425ce0 2024-04-24T09:16:18.411 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.411+0000 7f9a1820c740 10 client.23967 fetch_fsmap finished waiting for FSMap version 69 2024-04-24T09:16:18.411 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.411+0000 7f9a1820c740 10 client.23967 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:16:18.412 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.411+0000 7f9a1820c740 20 client.23967 populate_metadata read hostname 'smithi083' 2024-04-24T09:16:18.412 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.411+0000 7f9a1820c740 1 --2- 172.21.15.83:0/4240456501 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5559244a7100 0x5559244a9520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:18.412 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.411+0000 7f9a1820c740 4 client.23967 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T09:16:18.412 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.411+0000 7f9a1820c740 1 -- 172.21.15.83:0/4240456501 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555924393190 con 0x5559244a7100 2024-04-24T09:16:18.412 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.411+0000 7f9a0e800640 1 --2- 172.21.15.83:0/4240456501 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5559244a7100 0x5559244a9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:18.412 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.412+0000 7f9a0e800640 1 --2- 172.21.15.83:0/4240456501 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5559244a7100 0x5559244a9520 crc :-1 s=READY pgs=608 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b63d0146e54c7ba5 server_cookie=d604127cf2a6bb26 in_seq=0 out_seq=0 2024-04-24T09:16:18.412 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.412+0000 7f99efe00640 10 client.23967 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:16:18.571 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.570+0000 7f99efe00640 1 -- 172.21.15.83:0/4240456501 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555924393190 con 0x5559244a7100 2024-04-24T09:16:18.571 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.570+0000 7f99efe00640 10 client.23967 handle_command_reply: tid=0 2024-04-24T09:16:18.571 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.570+0000 7f9a1820c740 1 client.23967 shutdown 2024-04-24T09:16:18.571 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.570+0000 7f9a1820c740 1 -- 172.21.15.83:0/4240456501 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555924425ce0 msgr2=0x55592442a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:18.571 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.570+0000 7f9a1820c740 1 --2- 172.21.15.83:0/4240456501 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555924425ce0 0x55592442a120 secure :-1 s=READY pgs=2785 cs=0 l=1 rev1=1 crypto rx=0x7f99fc018000 tx=0x7f99fc003d70 comp rx=0 tx=0).stop 2024-04-24T09:16:18.572 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.571+0000 7f9a1820c740 1 -- 172.21.15.83:0/4240456501 shutdown_connections 2024-04-24T09:16:18.572 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.571+0000 7f9a1820c740 1 --2- 172.21.15.83:0/4240456501 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5559244a7100 0x5559244a9520 crc :-1 s=READY pgs=608 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:18.572 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.571+0000 7f9a1820c740 1 --2- 172.21.15.83:0/4240456501 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559244305a0 0x55592442a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:18.572 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.571+0000 7f9a1820c740 1 --2- 172.21.15.83:0/4240456501 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555924425ce0 0x55592442a120 unknown :-1 s=CLOSED pgs=2785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:18.572 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.571+0000 7f9a1820c740 1 --2- 172.21.15.83:0/4240456501 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5559244253c0 0x555924438a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:18.572 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.571+0000 7f9a1820c740 1 -- 172.21.15.83:0/4240456501 >> 172.21.15.83:0/4240456501 conn(0x5559244218d0 msgr2=0x555924427600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:18.572 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.571+0000 7f99efe00640 0 client.23967 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:16:18.572 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.571+0000 7f9a1820c740 1 -- 172.21.15.83:0/4240456501 shutdown_connections 2024-04-24T09:16:18.572 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.571+0000 7f9a1820c740 1 -- 172.21.15.83:0/4240456501 wait complete. 2024-04-24T09:16:18.572 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.571+0000 7f9a1820c740 20 client.23967 trim_cache size 0 max 16384 2024-04-24T09:16:18.573 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T09:16:18.573 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T09:16:18.573 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T09:16:18.573 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T09:16:18.573 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.573+0000 7f9a14a00640 1 -- 172.21.15.83:0/2197465260 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f99f40a3ee0 msgr2=0x7f99f40a63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:18.574 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.573+0000 7f9a14a00640 1 --2- 172.21.15.83:0/2197465260 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f99f40a3ee0 0x7f99f40a63a0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f9a00005ab0 tx=0x7f9a00005a20 comp rx=0 tx=0).stop 2024-04-24T09:16:18.574 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.573+0000 7f9a14a00640 1 -- 172.21.15.83:0/2197465260 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a101095d0 msgr2=0x7f9a10114070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:18.574 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.573+0000 7f9a14a00640 1 --2- 172.21.15.83:0/2197465260 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a101095d0 0x7f9a10114070 secure :-1 s=READY pgs=2782 cs=0 l=1 rev1=1 crypto rx=0x7f99fc00e9c0 tx=0x7f99fc00ee80 comp rx=0 tx=0).stop 2024-04-24T09:16:18.574 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.573+0000 7f9a14a00640 1 -- 172.21.15.83:0/2197465260 shutdown_connections 2024-04-24T09:16:18.574 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.573+0000 7f9a14a00640 1 --2- 172.21.15.83:0/2197465260 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f99f40a3ee0 0x7f99f40a63a0 secure :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f9a00005ab0 tx=0x7f9a00005a20 comp rx=0 tx=0).stop 2024-04-24T09:16:18.574 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.573+0000 7f9a14a00640 1 --2- 172.21.15.83:0/2197465260 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9a10114d40 0x7f9a101abdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:18.574 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.573+0000 7f9a14a00640 1 --2- 172.21.15.83:0/2197465260 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a101095d0 0x7f9a10114070 unknown :-1 s=CLOSED pgs=2782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:18.574 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.573+0000 7f9a14a00640 1 --2- 172.21.15.83:0/2197465260 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a1006e2f0 0x7f9a10113b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:18.574 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.573+0000 7f9a14a00640 1 -- 172.21.15.83:0/2197465260 >> 172.21.15.83:0/2197465260 conn(0x7f9a1006cf40 msgr2=0x7f9a1010a9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:18.575 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.574+0000 7f9a14a00640 1 -- 172.21.15.83:0/2197465260 shutdown_connections 2024-04-24T09:16:18.575 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:18.574+0000 7f9a14a00640 1 -- 172.21.15.83:0/2197465260 wait complete. 2024-04-24T09:16:19.218 INFO:tasks.workunit.client.0.smithi083.stdout: 10 79452 6.50 MB/sec execute 530 sec latency 133.937 ms 2024-04-24T09:16:20.218 INFO:tasks.workunit.client.0.smithi083.stdout: 10 79571 6.49 MB/sec execute 531 sec latency 84.595 ms 2024-04-24T09:16:20.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:16:20 smithi138 ceph-mon[31441]: pgmap v782: 129 pgs: 129 active+clean; 1.2 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.4 MiB/s wr, 311 op/s 2024-04-24T09:16:20.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:16:20 smithi189 ceph-mon[31791]: pgmap v782: 129 pgs: 129 active+clean; 1.2 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.4 MiB/s wr, 311 op/s 2024-04-24T09:16:20.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:16:20 smithi083 ceph-mon[27907]: pgmap v782: 129 pgs: 129 active+clean; 1.2 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.4 MiB/s wr, 311 op/s 2024-04-24T09:16:21.218 INFO:tasks.workunit.client.0.smithi083.stdout: 10 79689 6.49 MB/sec execute 532 sec latency 214.696 ms 2024-04-24T09:16:22.218 INFO:tasks.workunit.client.0.smithi083.stdout: 10 79767 6.48 MB/sec execute 533 sec latency 372.354 ms 2024-04-24T09:16:22.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:16:22 smithi138 ceph-mon[31441]: pgmap v783: 129 pgs: 129 active+clean; 1.2 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.4 MiB/s wr, 311 op/s 2024-04-24T09:16:22.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:16:22 smithi189 ceph-mon[31791]: pgmap v783: 129 pgs: 129 active+clean; 1.2 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.4 MiB/s wr, 311 op/s 2024-04-24T09:16:22.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:16:22 smithi083 ceph-mon[27907]: pgmap v783: 129 pgs: 129 active+clean; 1.2 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.4 MiB/s wr, 311 op/s 2024-04-24T09:16:23.218 INFO:tasks.workunit.client.0.smithi083.stdout: 10 79889 6.48 MB/sec execute 534 sec latency 76.383 ms 2024-04-24T09:16:24.218 INFO:tasks.workunit.client.0.smithi083.stdout: 10 80012 6.47 MB/sec execute 535 sec latency 85.984 ms 2024-04-24T09:16:24.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:16:24 smithi138 ceph-mon[31441]: pgmap v784: 129 pgs: 129 active+clean; 1.2 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.7 MiB/s wr, 317 op/s 2024-04-24T09:16:24.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:16:24 smithi189 ceph-mon[31791]: pgmap v784: 129 pgs: 129 active+clean; 1.2 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.7 MiB/s wr, 317 op/s 2024-04-24T09:16:24.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:16:24 smithi083 ceph-mon[27907]: pgmap v784: 129 pgs: 129 active+clean; 1.2 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.7 MiB/s wr, 317 op/s 2024-04-24T09:16:25.218 INFO:tasks.workunit.client.0.smithi083.stdout: 10 80180 6.50 MB/sec execute 536 sec latency 88.952 ms 2024-04-24T09:16:26.220 INFO:tasks.workunit.client.0.smithi083.stdout: 10 80329 6.51 MB/sec execute 537 sec latency 84.832 ms 2024-04-24T09:16:26.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:16:26 smithi138 ceph-mon[31441]: pgmap v785: 129 pgs: 129 active+clean; 1.2 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.7 MiB/s wr, 311 op/s 2024-04-24T09:16:26.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:16:26 smithi189 ceph-mon[31791]: pgmap v785: 129 pgs: 129 active+clean; 1.2 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.7 MiB/s wr, 311 op/s 2024-04-24T09:16:26.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:16:26 smithi083 ceph-mon[27907]: pgmap v785: 129 pgs: 129 active+clean; 1.2 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.7 MiB/s wr, 311 op/s 2024-04-24T09:16:27.220 INFO:tasks.workunit.client.0.smithi083.stdout: 10 80475 6.51 MB/sec execute 538 sec latency 81.526 ms 2024-04-24T09:16:28.221 INFO:tasks.workunit.client.0.smithi083.stdout: 10 80601 6.51 MB/sec execute 539 sec latency 99.297 ms 2024-04-24T09:16:28.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:16:28 smithi138 ceph-mon[31441]: pgmap v786: 129 pgs: 129 active+clean; 1.2 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.1 MiB/s wr, 264 op/s 2024-04-24T09:16:28.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:16:28 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:16:28.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:16:28 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:16:28.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:16:28 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:16:28.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:16:28 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:16:28.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:16:28 smithi189 ceph-mon[31791]: pgmap v786: 129 pgs: 129 active+clean; 1.2 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.1 MiB/s wr, 264 op/s 2024-04-24T09:16:28.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:16:28 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:16:28.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:16:28 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:16:28.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:16:28 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:16:28.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:16:28 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:16:28.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:16:28 smithi083 ceph-mon[27907]: pgmap v786: 129 pgs: 129 active+clean; 1.2 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.1 MiB/s wr, 264 op/s 2024-04-24T09:16:28.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:16:28 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:16:28.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:16:28 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:16:28.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:16:28 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:16:28.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:16:28 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:16:29.220 INFO:tasks.workunit.client.0.smithi083.stdout: 10 80733 6.51 MB/sec execute 540 sec latency 79.207 ms 2024-04-24T09:16:29.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:16:29 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:16:29.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:16:29 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:16:29.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:16:29 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:16:30.220 INFO:tasks.workunit.client.0.smithi083.stdout: 10 80905 6.52 MB/sec execute 541 sec latency 71.403 ms 2024-04-24T09:16:30.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:16:30 smithi138 ceph-mon[31441]: pgmap v787: 129 pgs: 129 active+clean; 1.2 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.2 MiB/s wr, 358 op/s 2024-04-24T09:16:30.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:16:30 smithi189 ceph-mon[31791]: pgmap v787: 129 pgs: 129 active+clean; 1.2 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.2 MiB/s wr, 358 op/s 2024-04-24T09:16:30.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:16:30 smithi083 ceph-mon[27907]: pgmap v787: 129 pgs: 129 active+clean; 1.2 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.2 MiB/s wr, 358 op/s 2024-04-24T09:16:31.221 INFO:tasks.workunit.client.0.smithi083.stdout: 10 81043 6.52 MB/sec execute 542 sec latency 91.722 ms 2024-04-24T09:16:31.767 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.768+0000 7fb957600640 1 -- 172.21.15.83:0/3528658537 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb950109920 msgr2=0x7fb950114c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:31.767 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.768+0000 7fb957600640 1 --2- 172.21.15.83:0/3528658537 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb950109920 0x7fb950114c10 secure :-1 s=READY pgs=2786 cs=0 l=1 rev1=1 crypto rx=0x7fb93c009900 tx=0x7fb93c02a480 comp rx=0 tx=0).stop 2024-04-24T09:16:31.768 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.768+0000 7fb957600640 1 -- 172.21.15.83:0/3528658537 shutdown_connections 2024-04-24T09:16:31.768 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.768+0000 7fb957600640 1 --2- 172.21.15.83:0/3528658537 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb950115150 0x7fb950117540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:31.768 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.768+0000 7fb957600640 1 --2- 172.21.15.83:0/3528658537 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb950109920 0x7fb950114c10 unknown :-1 s=CLOSED pgs=2786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:31.768 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.768+0000 7fb957600640 1 --2- 172.21.15.83:0/3528658537 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb950109000 0x7fb9501093e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:31.768 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.768+0000 7fb957600640 1 -- 172.21.15.83:0/3528658537 >> 172.21.15.83:0/3528658537 conn(0x7fb95006d0d0 msgr2=0x7fb95006d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:31.768 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.768+0000 7fb957600640 1 -- 172.21.15.83:0/3528658537 shutdown_connections 2024-04-24T09:16:31.768 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.768+0000 7fb957600640 1 -- 172.21.15.83:0/3528658537 wait complete. 2024-04-24T09:16:31.768 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.769+0000 7fb957600640 1 Processor -- start 2024-04-24T09:16:31.769 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.770+0000 7fb957600640 1 -- start start 2024-04-24T09:16:31.769 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.770+0000 7fb957600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb950109000 0x7fb95010aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:31.769 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.770+0000 7fb957600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb950109920 0x7fb95010afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:31.770 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.770+0000 7fb957600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb950115150 0x7fb95010b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:31.770 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.770+0000 7fb957600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb95011aad0 con 0x7fb950109000 2024-04-24T09:16:31.770 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.770+0000 7fb957600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb95011a950 con 0x7fb950109920 2024-04-24T09:16:31.770 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.770+0000 7fb957600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb95011ac50 con 0x7fb950115150 2024-04-24T09:16:31.770 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.770+0000 7fb954c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb950109000 0x7fb95010aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:31.771 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.771+0000 7fb954c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb950109000 0x7fb95010aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40924/0 (socket says 172.21.15.83:40924) 2024-04-24T09:16:31.771 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.771+0000 7fb954c00640 1 -- 172.21.15.83:0/3123474243 learned_addr learned my addr 172.21.15.83:0/3123474243 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:16:31.771 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.771+0000 7fb955600640 1 --2- 172.21.15.83:0/3123474243 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb950115150 0x7fb95010b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:31.771 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.771+0000 7fb94fe00640 1 --2- 172.21.15.83:0/3123474243 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb950109920 0x7fb95010afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:31.771 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.771+0000 7fb954c00640 1 -- 172.21.15.83:0/3123474243 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb950115150 msgr2=0x7fb95010b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:31.771 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.771+0000 7fb954c00640 1 --2- 172.21.15.83:0/3123474243 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb950115150 0x7fb95010b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:31.771 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.771+0000 7fb954c00640 1 -- 172.21.15.83:0/3123474243 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb950109920 msgr2=0x7fb95010afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:31.771 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.771+0000 7fb954c00640 1 --2- 172.21.15.83:0/3123474243 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb950109920 0x7fb95010afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:31.771 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.771+0000 7fb954c00640 1 -- 172.21.15.83:0/3123474243 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9501af430 con 0x7fb950109000 2024-04-24T09:16:31.772 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.772+0000 7fb954c00640 1 --2- 172.21.15.83:0/3123474243 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb950109000 0x7fb95010aa70 secure :-1 s=READY pgs=2787 cs=0 l=1 rev1=1 crypto rx=0x7fb94400d770 tx=0x7fb94400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:31.772 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.772+0000 7fb94d600640 1 -- 172.21.15.83:0/3123474243 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb944017070 con 0x7fb950109000 2024-04-24T09:16:31.772 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.772+0000 7fb957600640 1 -- 172.21.15.83:0/3123474243 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9501af720 con 0x7fb950109000 2024-04-24T09:16:31.772 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.773+0000 7fb94d600640 1 -- 172.21.15.83:0/3123474243 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb9440054d0 con 0x7fb950109000 2024-04-24T09:16:31.773 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.773+0000 7fb957600640 1 -- 172.21.15.83:0/3123474243 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9501afc40 con 0x7fb950109000 2024-04-24T09:16:31.773 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.773+0000 7fb94d600640 1 -- 172.21.15.83:0/3123474243 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb944004cf0 con 0x7fb950109000 2024-04-24T09:16:31.778 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.779+0000 7fb95a44e740 1 Processor -- start 2024-04-24T09:16:31.779 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.780+0000 7fb94d600640 1 -- 172.21.15.83:0/3123474243 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb944004e90 con 0x7fb950109000 2024-04-24T09:16:31.780 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.781+0000 7fb95a44e740 1 -- start start 2024-04-24T09:16:31.781 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.781+0000 7fb95a44e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56541e30e1b0 0x56541e30e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:31.781 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.781+0000 7fb95a44e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56541e30ead0 0x56541e318e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:31.781 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.781+0000 7fb95a44e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56541e319390 0x56541e31b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:31.781 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.781+0000 7fb95a44e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56541e274900 con 0x56541e319390 2024-04-24T09:16:31.781 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.781+0000 7fb95a44e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56541e21af20 con 0x56541e30ead0 2024-04-24T09:16:31.781 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.781+0000 7fb95a44e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56541e2708c0 con 0x56541e30e1b0 2024-04-24T09:16:31.782 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.782+0000 7fb955600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56541e30e1b0 0x56541e30e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:31.782 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.782+0000 7fb955600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56541e30e1b0 0x56541e30e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:36728/0 (socket says 172.21.15.83:36728) 2024-04-24T09:16:31.782 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.782+0000 7fb955600640 1 -- 172.21.15.83:0/695434325 learned_addr learned my addr 172.21.15.83:0/695434325 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:16:31.782 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.783+0000 7fb94fe00640 1 --2- 172.21.15.83:0/695434325 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56541e319390 0x56541e31b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:31.782 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.783+0000 7fb954c00640 1 --2- 172.21.15.83:0/695434325 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56541e30ead0 0x56541e318e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:31.783 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.783+0000 7fb94d600640 1 --2- 172.21.15.83:0/3123474243 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb9380a3ef0 0x7fb9380a63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:31.783 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.783+0000 7fb94d600640 1 -- 172.21.15.83:0/3123474243 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 9583+0+0 (secure 0 0 0) 0x7fb9440d0620 con 0x7fb950109000 2024-04-24T09:16:31.783 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.784+0000 7fb955600640 1 -- 172.21.15.83:0/695434325 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56541e30ead0 msgr2=0x56541e318e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:31.784 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.784+0000 7fb955600640 1 --2- 172.21.15.83:0/695434325 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56541e30ead0 0x56541e318e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:31.784 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.784+0000 7fb955600640 1 -- 172.21.15.83:0/695434325 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56541e319390 msgr2=0x56541e31b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:31.784 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.784+0000 7fb955600640 1 --2- 172.21.15.83:0/695434325 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56541e319390 0x56541e31b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:31.784 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.784+0000 7fb955600640 1 -- 172.21.15.83:0/695434325 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56541e23fe10 con 0x56541e30e1b0 2024-04-24T09:16:31.784 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.784+0000 7fb955600640 1 --2- 172.21.15.83:0/3123474243 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb9380a3ef0 0x7fb9380a63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:31.784 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.784+0000 7fb94fe00640 1 --2- 172.21.15.83:0/695434325 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56541e319390 0x56541e31b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:16:31.784 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.784+0000 7fb955600640 1 --2- 172.21.15.83:0/695434325 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56541e30e1b0 0x56541e30e590 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb948011590 tx=0x7fb948011a50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17789eed0231913c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:31.784 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.785+0000 7fb955600640 1 --2- 172.21.15.83:0/3123474243 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb9380a3ef0 0x7fb9380a63b0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fb948022000 tx=0x7fb948023000 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:31.785 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.785+0000 7fb941c00640 1 -- 172.21.15.83:0/695434325 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb948017050 con 0x56541e30e1b0 2024-04-24T09:16:31.785 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.785+0000 7fb941c00640 1 -- 172.21.15.83:0/695434325 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb94801bb90 con 0x56541e30e1b0 2024-04-24T09:16:31.785 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.785+0000 7fb941c00640 1 -- 172.21.15.83:0/695434325 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb948025de0 con 0x56541e30e1b0 2024-04-24T09:16:31.785 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.786+0000 7fb95a44e740 1 -- 172.21.15.83:0/695434325 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56541e30e1b0 msgr2=0x56541e30e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:31.785 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.786+0000 7fb95a44e740 1 --2- 172.21.15.83:0/695434325 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56541e30e1b0 0x56541e30e590 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb948011590 tx=0x7fb948011a50 comp rx=0 tx=0).stop 2024-04-24T09:16:31.786 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.786+0000 7fb95a44e740 1 -- 172.21.15.83:0/695434325 shutdown_connections 2024-04-24T09:16:31.786 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.786+0000 7fb95a44e740 1 --2- 172.21.15.83:0/695434325 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56541e319390 0x56541e31b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:31.786 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.786+0000 7fb95a44e740 1 --2- 172.21.15.83:0/695434325 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56541e30ead0 0x56541e318e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:31.786 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.786+0000 7fb95a44e740 1 --2- 172.21.15.83:0/695434325 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56541e30e1b0 0x56541e30e590 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:31.786 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.786+0000 7fb95a44e740 1 -- 172.21.15.83:0/695434325 >> 172.21.15.83:0/695434325 conn(0x56541e30a6c0 msgr2=0x56541e30aaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:31.786 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.786+0000 7fb95a44e740 1 -- 172.21.15.83:0/695434325 shutdown_connections 2024-04-24T09:16:31.786 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.786+0000 7fb95a44e740 1 -- 172.21.15.83:0/695434325 wait complete. 2024-04-24T09:16:31.786 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.787+0000 7fb95a44e740 1 -- start start 2024-04-24T09:16:31.787 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.787+0000 7fb95a44e740 1 Processor -- start 2024-04-24T09:16:31.788 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.788+0000 7fb95a44e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56541e30e1b0 0x56541e3162b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:31.788 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.788+0000 7fb95a44e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56541e30ead0 0x56541e3167f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:31.788 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.788+0000 7fb95a44e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56541e319390 0x56541e316d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:31.788 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.788+0000 7fb95a44e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56541e31bfc0 con 0x56541e30e1b0 2024-04-24T09:16:31.788 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.788+0000 7fb95a44e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56541e22e2c0 con 0x56541e319390 2024-04-24T09:16:31.788 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.788+0000 7fb95a44e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56541e31c140 con 0x56541e30ead0 2024-04-24T09:16:31.788 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.788+0000 7fb95a44e740 10 client.0 _ll_register_callbacks cb 0x56541df86a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:16:31.788 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.788+0000 7fb955600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56541e319390 0x56541e316d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:31.788 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.788+0000 7fb955600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56541e319390 0x56541e316d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:46318/0 (socket says 172.21.15.83:46318) 2024-04-24T09:16:31.788 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.788+0000 7fb955600640 1 -- 172.21.15.83:0/3068934191 learned_addr learned my addr 172.21.15.83:0/3068934191 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:16:31.788 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.788+0000 7fb954c00640 1 --2- 172.21.15.83:0/3068934191 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56541e30e1b0 0x56541e3162b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:31.788 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.789+0000 7fb94fe00640 1 --2- 172.21.15.83:0/3068934191 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56541e30ead0 0x56541e3167f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:31.789 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.789+0000 7fb955600640 1 -- 172.21.15.83:0/3068934191 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56541e30ead0 msgr2=0x56541e3167f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:31.789 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.789+0000 7fb955600640 1 --2- 172.21.15.83:0/3068934191 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56541e30ead0 0x56541e3167f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:31.789 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.789+0000 7fb955600640 1 -- 172.21.15.83:0/3068934191 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56541e30e1b0 msgr2=0x56541e3162b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:31.789 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.789+0000 7fb955600640 1 --2- 172.21.15.83:0/3068934191 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56541e30e1b0 0x56541e3162b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:31.789 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.789+0000 7fb955600640 1 -- 172.21.15.83:0/3068934191 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56541e23efd0 con 0x56541e319390 2024-04-24T09:16:31.789 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.789+0000 7fb94fe00640 1 --2- 172.21.15.83:0/3068934191 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56541e30ead0 0x56541e3167f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:16:31.790 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.790+0000 7fb936000640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:16:31.790 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.790+0000 7fb955600640 1 --2- 172.21.15.83:0/3068934191 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56541e319390 0x56541e316d30 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fb948009a70 tx=0x7fb948009aa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6470d0f2c3bcf35 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:31.790 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.791+0000 7fb936000640 1 -- 172.21.15.83:0/3068934191 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb948033060 con 0x56541e319390 2024-04-24T09:16:31.791 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.791+0000 7fb936000640 1 -- 172.21.15.83:0/3068934191 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(132..132 src has 1..132) ==== 9583+0+0 (secure 0 0 0) 0x7fb948004a00 con 0x56541e319390 2024-04-24T09:16:31.791 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.792+0000 7fb95a44e740 1 -- 172.21.15.83:0/3068934191 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb94801bb90 con 0x56541e319390 2024-04-24T09:16:31.791 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.792+0000 7fb936000640 1 -- 172.21.15.83:0/3068934191 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7fb948017030 con 0x56541e319390 2024-04-24T09:16:31.791 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.792+0000 7fb95a44e740 10 client.41042 fetch_fsmap learned FSMap version 69 2024-04-24T09:16:31.792 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.792+0000 7fb95a44e740 1 -- 172.21.15.83:0/3068934191 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x56541df87210 con 0x56541e319390 2024-04-24T09:16:31.793 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.794+0000 7fb936000640 1 -- 172.21.15.83:0/3068934191 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb94801bb90 con 0x56541e319390 2024-04-24T09:16:31.793 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.794+0000 7fb936000640 1 -- 172.21.15.83:0/3068934191 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 69) ==== 8370+0+0 (secure 0 0 0) 0x7fb948005230 con 0x56541e319390 2024-04-24T09:16:31.794 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.795+0000 7fb95a44e740 10 client.41042 fetch_fsmap finished waiting for FSMap version 69 2024-04-24T09:16:31.794 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.795+0000 7fb95a44e740 10 client.41042 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:16:31.794 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.795+0000 7fb95a44e740 20 client.41042 populate_metadata read hostname 'smithi083' 2024-04-24T09:16:31.794 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.795+0000 7fb95a44e740 1 --2- 172.21.15.83:0/3068934191 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56541e329410 0x56541e32b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:31.794 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.795+0000 7fb95a44e740 4 client.41042 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:16:31.794 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.795+0000 7fb95a44e740 1 -- 172.21.15.83:0/3068934191 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56541e282320 con 0x56541e329410 2024-04-24T09:16:31.795 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.795+0000 7fb954c00640 1 --2- 172.21.15.83:0/3068934191 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56541e329410 0x56541e32b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:31.795 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.796+0000 7fb954c00640 1 --2- 172.21.15.83:0/3068934191 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56541e329410 0x56541e32b800 crc :-1 s=READY pgs=609 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d1700b12e03747d server_cookie=828730e3691a9c0c in_seq=0 out_seq=0 2024-04-24T09:16:31.795 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.796+0000 7fb936000640 10 client.41042 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:16:31.797 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.797+0000 7fb936000640 1 -- 172.21.15.83:0/3068934191 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56541e282320 con 0x56541e329410 2024-04-24T09:16:31.797 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.797+0000 7fb936000640 10 client.41042 handle_command_reply: tid=0 2024-04-24T09:16:31.797 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.797+0000 7fb95a44e740 1 client.41042 shutdown 2024-04-24T09:16:31.798 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.798+0000 7fb95a44e740 1 -- 172.21.15.83:0/3068934191 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56541e319390 msgr2=0x56541e316d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:31.798 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.798+0000 7fb95a44e740 1 --2- 172.21.15.83:0/3068934191 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56541e319390 0x56541e316d30 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fb948009a70 tx=0x7fb948009aa0 comp rx=0 tx=0).stop 2024-04-24T09:16:31.798 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.798+0000 7fb95a44e740 1 -- 172.21.15.83:0/3068934191 shutdown_connections 2024-04-24T09:16:31.798 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.798+0000 7fb95a44e740 1 --2- 172.21.15.83:0/3068934191 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56541e329410 0x56541e32b800 crc :-1 s=READY pgs=609 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:31.798 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.798+0000 7fb95a44e740 1 --2- 172.21.15.83:0/3068934191 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56541e319390 0x56541e316d30 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:31.799 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.798+0000 7fb95a44e740 1 --2- 172.21.15.83:0/3068934191 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56541e30ead0 0x56541e3167f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:31.799 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.798+0000 7fb95a44e740 1 --2- 172.21.15.83:0/3068934191 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56541e30e1b0 0x56541e3162b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:31.799 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.798+0000 7fb95a44e740 1 -- 172.21.15.83:0/3068934191 >> 172.21.15.83:0/3068934191 conn(0x56541e30a6c0 msgr2=0x56541e3103f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:31.799 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.798+0000 7fb936000640 0 client.41042 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:16:31.799 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.799+0000 7fb95a44e740 1 -- 172.21.15.83:0/3068934191 shutdown_connections 2024-04-24T09:16:31.799 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.799+0000 7fb95a44e740 1 -- 172.21.15.83:0/3068934191 wait complete. 2024-04-24T09:16:31.799 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.799+0000 7fb95a44e740 20 client.41042 trim_cache size 0 max 16384 2024-04-24T09:16:31.823 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.824+0000 7fb95a44e740 1 Processor -- start 2024-04-24T09:16:31.823 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.824+0000 7fb95a44e740 1 -- start start 2024-04-24T09:16:31.824 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.824+0000 7fb95a44e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56541e30e1b0 0x56541e324e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:31.824 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.824+0000 7fb95a44e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56541e30ead0 0x56541e316b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:31.824 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.824+0000 7fb95a44e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56541e319390 0x56541e3181f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:31.824 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.824+0000 7fb95a44e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb9501ba0b0 con 0x56541e30ead0 2024-04-24T09:16:31.824 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.824+0000 7fb95a44e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb9501bbc50 con 0x56541e30e1b0 2024-04-24T09:16:31.824 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.824+0000 7fb95a44e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb9501ba780 con 0x56541e319390 2024-04-24T09:16:31.825 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.825+0000 7fb94fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56541e30ead0 0x56541e316b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:31.825 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.825+0000 7fb954c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56541e30e1b0 0x56541e324e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:31.825 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.825+0000 7fb954c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56541e30e1b0 0x56541e324e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:46332/0 (socket says 172.21.15.83:46332) 2024-04-24T09:16:31.825 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.825+0000 7fb954c00640 1 -- 172.21.15.83:0/3976772089 learned_addr learned my addr 172.21.15.83:0/3976772089 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:16:31.825 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.825+0000 7fb955600640 1 --2- 172.21.15.83:0/3976772089 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56541e319390 0x56541e3181f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:31.825 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.825+0000 7fb954c00640 1 -- 172.21.15.83:0/3976772089 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56541e319390 msgr2=0x56541e3181f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:31.825 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.825+0000 7fb954c00640 1 --2- 172.21.15.83:0/3976772089 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56541e319390 0x56541e3181f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:31.826 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.825+0000 7fb954c00640 1 -- 172.21.15.83:0/3976772089 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56541e30ead0 msgr2=0x56541e316b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:31.826 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.825+0000 7fb954c00640 1 --2- 172.21.15.83:0/3976772089 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56541e30ead0 0x56541e316b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:31.826 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.825+0000 7fb954c00640 1 -- 172.21.15.83:0/3976772089 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56541e305ea0 con 0x56541e30e1b0 2024-04-24T09:16:31.826 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.826+0000 7fb954c00640 1 --2- 172.21.15.83:0/3976772089 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56541e30e1b0 0x56541e324e80 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fb94409c0d0 tx=0x7fb9440a0040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3362124209b455a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:31.827 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.826+0000 7fb941200640 1 -- 172.21.15.83:0/3976772089 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9440b4020 con 0x56541e30e1b0 2024-04-24T09:16:31.827 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.826+0000 7fb941200640 1 -- 172.21.15.83:0/3976772089 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb944009c30 con 0x56541e30e1b0 2024-04-24T09:16:31.827 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.826+0000 7fb941200640 1 -- 172.21.15.83:0/3976772089 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb944020040 con 0x56541e30e1b0 2024-04-24T09:16:31.827 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.827+0000 7fb95a44e740 1 -- 172.21.15.83:0/3976772089 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56541e30e1b0 msgr2=0x56541e324e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:31.827 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.827+0000 7fb95a44e740 1 --2- 172.21.15.83:0/3976772089 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56541e30e1b0 0x56541e324e80 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fb94409c0d0 tx=0x7fb9440a0040 comp rx=0 tx=0).stop 2024-04-24T09:16:31.827 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.827+0000 7fb95a44e740 1 -- 172.21.15.83:0/3976772089 shutdown_connections 2024-04-24T09:16:31.827 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.827+0000 7fb95a44e740 1 --2- 172.21.15.83:0/3976772089 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56541e319390 0x56541e3181f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:31.827 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.827+0000 7fb95a44e740 1 --2- 172.21.15.83:0/3976772089 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56541e30ead0 0x56541e316b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:31.827 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.827+0000 7fb95a44e740 1 --2- 172.21.15.83:0/3976772089 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56541e30e1b0 0x56541e324e80 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:31.827 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.827+0000 7fb95a44e740 1 -- 172.21.15.83:0/3976772089 >> 172.21.15.83:0/3976772089 conn(0x56541e30a6c0 msgr2=0x56541e317ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:31.827 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.828+0000 7fb95a44e740 1 -- 172.21.15.83:0/3976772089 shutdown_connections 2024-04-24T09:16:31.827 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.828+0000 7fb95a44e740 1 -- 172.21.15.83:0/3976772089 wait complete. 2024-04-24T09:16:31.829 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.828+0000 7fb95a44e740 1 -- start start 2024-04-24T09:16:31.829 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.829+0000 7fb95a44e740 1 Processor -- start 2024-04-24T09:16:31.829 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.829+0000 7fb95a44e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56541e30e1b0 0x56541e321580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:31.829 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.829+0000 7fb95a44e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56541e30ead0 0x56541e312b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:31.829 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.829+0000 7fb95a44e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56541e319390 0x56541e313040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:31.829 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.829+0000 7fb95a44e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb9501bb580 con 0x56541e30ead0 2024-04-24T09:16:31.829 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.829+0000 7fb95a44e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb9501baeb0 con 0x56541e319390 2024-04-24T09:16:31.830 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.829+0000 7fb95a44e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb9501b80b0 con 0x56541e30e1b0 2024-04-24T09:16:31.830 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.829+0000 7fb95a44e740 10 client.0 _ll_register_callbacks cb 0x7fb948011440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:16:31.830 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.830+0000 7fb955600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56541e319390 0x56541e313040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:31.830 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.830+0000 7fb94fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56541e30ead0 0x56541e312b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:31.830 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.830+0000 7fb954c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56541e30e1b0 0x56541e321580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:31.830 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.830+0000 7fb94fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56541e30ead0 0x56541e312b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40956/0 (socket says 172.21.15.83:40956) 2024-04-24T09:16:31.830 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.830+0000 7fb954c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56541e30e1b0 0x56541e321580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:36740/0 (socket says 172.21.15.83:36740) 2024-04-24T09:16:31.830 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.830+0000 7fb94fe00640 1 -- 172.21.15.83:0/2300875020 learned_addr learned my addr 172.21.15.83:0/2300875020 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:16:31.833 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.830+0000 7fb94fe00640 1 -- 172.21.15.83:0/2300875020 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56541e30e1b0 msgr2=0x56541e321580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:31.833 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.830+0000 7fb94fe00640 1 --2- 172.21.15.83:0/2300875020 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56541e30e1b0 0x56541e321580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:31.833 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.830+0000 7fb94fe00640 1 -- 172.21.15.83:0/2300875020 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56541e319390 msgr2=0x56541e313040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:31.834 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.830+0000 7fb94fe00640 1 --2- 172.21.15.83:0/2300875020 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56541e319390 0x56541e313040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:31.834 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.830+0000 7fb94fe00640 1 -- 172.21.15.83:0/2300875020 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56541e053620 con 0x56541e30ead0 2024-04-24T09:16:31.834 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.831+0000 7fb94fe00640 1 --2- 172.21.15.83:0/2300875020 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56541e30ead0 0x56541e312b00 secure :-1 s=READY pgs=2788 cs=0 l=1 rev1=1 crypto rx=0x7fb93c009230 tx=0x7fb93c0053b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8bdc4cea293ed97 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:31.834 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.831+0000 7fb936000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:16:31.834 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.831+0000 7fb936000640 1 -- 172.21.15.83:0/2300875020 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb93c035040 con 0x56541e30ead0 2024-04-24T09:16:31.834 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.831+0000 7fb936000640 1 -- 172.21.15.83:0/2300875020 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(132..132 src has 1..132) ==== 9583+0+0 (secure 0 0 0) 0x7fb93c0048c0 con 0x56541e30ead0 2024-04-24T09:16:31.834 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.831+0000 7fb95a44e740 1 -- 172.21.15.83:0/2300875020 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56541e27c190 con 0x56541e30ead0 2024-04-24T09:16:31.834 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.832+0000 7fb936000640 1 -- 172.21.15.83:0/2300875020 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7fb93c04e050 con 0x56541e30ead0 2024-04-24T09:16:31.834 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.832+0000 7fb95a44e740 10 client.23985 fetch_fsmap learned FSMap version 69 2024-04-24T09:16:31.834 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.832+0000 7fb95a44e740 1 -- 172.21.15.83:0/2300875020 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x56541e397900 con 0x56541e30ead0 2024-04-24T09:16:31.835 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.832+0000 7fb936000640 1 -- 172.21.15.83:0/2300875020 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x56541e27c190 con 0x56541e30ead0 2024-04-24T09:16:31.835 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.832+0000 7fb936000640 1 -- 172.21.15.83:0/2300875020 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 69) ==== 8370+0+0 (secure 0 0 0) 0x7fb93c043020 con 0x56541e30ead0 2024-04-24T09:16:31.835 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.832+0000 7fb95a44e740 10 client.23985 fetch_fsmap finished waiting for FSMap version 69 2024-04-24T09:16:31.835 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.832+0000 7fb95a44e740 10 client.23985 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:16:31.835 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.832+0000 7fb95a44e740 20 client.23985 populate_metadata read hostname 'smithi083' 2024-04-24T09:16:31.835 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.832+0000 7fb95a44e740 1 --2- 172.21.15.83:0/2300875020 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56541e398080 0x56541e39a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:31.835 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.832+0000 7fb95a44e740 4 client.23985 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T09:16:31.835 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.833+0000 7fb95a44e740 1 -- 172.21.15.83:0/2300875020 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56541e1f84b0 con 0x56541e398080 2024-04-24T09:16:31.836 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.833+0000 7fb954c00640 1 --2- 172.21.15.83:0/2300875020 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56541e398080 0x56541e39a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:31.836 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.834+0000 7fb936000640 10 client.23985 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:16:31.836 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.833+0000 7fb954c00640 1 --2- 172.21.15.83:0/2300875020 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56541e398080 0x56541e39a4a0 crc :-1 s=READY pgs=610 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a011fcc5f98ed2f8 server_cookie=80d983968d385cb7 in_seq=0 out_seq=0 2024-04-24T09:16:31.977 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.977+0000 7fb936000640 1 -- 172.21.15.83:0/2300875020 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56541e1f84b0 con 0x56541e398080 2024-04-24T09:16:31.977 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.977+0000 7fb936000640 10 client.23985 handle_command_reply: tid=0 2024-04-24T09:16:31.977 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.977+0000 7fb95a44e740 1 client.23985 shutdown 2024-04-24T09:16:31.977 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.978+0000 7fb95a44e740 1 -- 172.21.15.83:0/2300875020 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56541e30ead0 msgr2=0x56541e312b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:31.977 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.978+0000 7fb95a44e740 1 --2- 172.21.15.83:0/2300875020 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56541e30ead0 0x56541e312b00 secure :-1 s=READY pgs=2788 cs=0 l=1 rev1=1 crypto rx=0x7fb93c009230 tx=0x7fb93c0053b0 comp rx=0 tx=0).stop 2024-04-24T09:16:31.978 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.978+0000 7fb95a44e740 1 -- 172.21.15.83:0/2300875020 shutdown_connections 2024-04-24T09:16:31.978 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.978+0000 7fb95a44e740 1 --2- 172.21.15.83:0/2300875020 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56541e398080 0x56541e39a4a0 crc :-1 s=READY pgs=610 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:31.978 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.978+0000 7fb95a44e740 1 --2- 172.21.15.83:0/2300875020 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56541e319390 0x56541e313040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:31.978 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.978+0000 7fb95a44e740 1 --2- 172.21.15.83:0/2300875020 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56541e30ead0 0x56541e312b00 unknown :-1 s=CLOSED pgs=2788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:31.978 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.978+0000 7fb95a44e740 1 --2- 172.21.15.83:0/2300875020 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56541e30e1b0 0x56541e321580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:31.978 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.978+0000 7fb95a44e740 1 -- 172.21.15.83:0/2300875020 >> 172.21.15.83:0/2300875020 conn(0x56541e30a6c0 msgr2=0x56541e318370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:31.978 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.978+0000 7fb936000640 0 client.23985 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:16:31.979 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.978+0000 7fb95a44e740 1 -- 172.21.15.83:0/2300875020 shutdown_connections 2024-04-24T09:16:31.979 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.978+0000 7fb95a44e740 1 -- 172.21.15.83:0/2300875020 wait complete. 2024-04-24T09:16:31.979 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.979+0000 7fb95a44e740 20 client.23985 trim_cache size 0 max 16384 2024-04-24T09:16:31.979 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T09:16:31.979 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T09:16:31.979 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T09:16:31.979 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T09:16:31.980 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.979+0000 7fb957600640 1 -- 172.21.15.83:0/3123474243 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb9380a3ef0 msgr2=0x7fb9380a63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:31.980 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.979+0000 7fb957600640 1 --2- 172.21.15.83:0/3123474243 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb9380a3ef0 0x7fb9380a63b0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fb948022000 tx=0x7fb948023000 comp rx=0 tx=0).stop 2024-04-24T09:16:31.980 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.979+0000 7fb957600640 1 -- 172.21.15.83:0/3123474243 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb950109000 msgr2=0x7fb95010aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:31.980 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.979+0000 7fb957600640 1 --2- 172.21.15.83:0/3123474243 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb950109000 0x7fb95010aa70 secure :-1 s=READY pgs=2787 cs=0 l=1 rev1=1 crypto rx=0x7fb94400d770 tx=0x7fb94400dc30 comp rx=0 tx=0).stop 2024-04-24T09:16:31.980 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.980+0000 7fb957600640 1 -- 172.21.15.83:0/3123474243 shutdown_connections 2024-04-24T09:16:31.980 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.980+0000 7fb957600640 1 --2- 172.21.15.83:0/3123474243 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb9380a3ef0 0x7fb9380a63b0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:31.980 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.980+0000 7fb957600640 1 --2- 172.21.15.83:0/3123474243 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb950115150 0x7fb95010b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:31.980 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.980+0000 7fb957600640 1 --2- 172.21.15.83:0/3123474243 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb950109920 0x7fb95010afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:31.981 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.980+0000 7fb957600640 1 --2- 172.21.15.83:0/3123474243 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb950109000 0x7fb95010aa70 unknown :-1 s=CLOSED pgs=2787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:31.981 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.980+0000 7fb957600640 1 -- 172.21.15.83:0/3123474243 >> 172.21.15.83:0/3123474243 conn(0x7fb95006d0d0 msgr2=0x7fb950070cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:31.981 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.980+0000 7fb957600640 1 -- 172.21.15.83:0/3123474243 shutdown_connections 2024-04-24T09:16:31.981 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:31.980+0000 7fb957600640 1 -- 172.21.15.83:0/3123474243 wait complete. 2024-04-24T09:16:32.222 INFO:tasks.workunit.client.0.smithi083.stdout: 10 81111 6.51 MB/sec execute 543 sec latency 269.465 ms 2024-04-24T09:16:32.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:16:32 smithi138 ceph-mon[31441]: pgmap v788: 129 pgs: 129 active+clean; 1.2 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.6 MiB/s wr, 223 op/s 2024-04-24T09:16:32.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:16:32 smithi189 ceph-mon[31791]: pgmap v788: 129 pgs: 129 active+clean; 1.2 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.6 MiB/s wr, 223 op/s 2024-04-24T09:16:32.552 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:16:32 smithi083 ceph-mon[27907]: pgmap v788: 129 pgs: 129 active+clean; 1.2 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.6 MiB/s wr, 223 op/s 2024-04-24T09:16:33.221 INFO:tasks.workunit.client.0.smithi083.stdout: 10 81222 6.52 MB/sec execute 544 sec latency 340.451 ms 2024-04-24T09:16:34.221 INFO:tasks.workunit.client.0.smithi083.stdout: 10 81332 6.51 MB/sec execute 545 sec latency 76.951 ms 2024-04-24T09:16:34.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:16:34 smithi138 ceph-mon[31441]: pgmap v789: 129 pgs: 129 active+clean; 1.2 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.7 MiB/s wr, 328 op/s 2024-04-24T09:16:34.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:16:34 smithi189 ceph-mon[31791]: pgmap v789: 129 pgs: 129 active+clean; 1.2 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.7 MiB/s wr, 328 op/s 2024-04-24T09:16:34.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:16:34 smithi083 ceph-mon[27907]: pgmap v789: 129 pgs: 129 active+clean; 1.2 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.7 MiB/s wr, 328 op/s 2024-04-24T09:16:35.221 INFO:tasks.workunit.client.0.smithi083.stdout: 10 81480 6.53 MB/sec execute 546 sec latency 76.515 ms 2024-04-24T09:16:36.221 INFO:tasks.workunit.client.0.smithi083.stdout: 10 81621 6.52 MB/sec execute 547 sec latency 90.375 ms 2024-04-24T09:16:36.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:16:36 smithi138 ceph-mon[31441]: pgmap v790: 129 pgs: 129 active+clean; 1.2 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.1 MiB/s wr, 198 op/s 2024-04-24T09:16:36.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:16:36 smithi189 ceph-mon[31791]: pgmap v790: 129 pgs: 129 active+clean; 1.2 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.1 MiB/s wr, 198 op/s 2024-04-24T09:16:36.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:16:36 smithi083 ceph-mon[27907]: pgmap v790: 129 pgs: 129 active+clean; 1.2 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.1 MiB/s wr, 198 op/s 2024-04-24T09:16:37.221 INFO:tasks.workunit.client.0.smithi083.stdout: 10 81754 6.53 MB/sec execute 548 sec latency 87.397 ms 2024-04-24T09:16:38.221 INFO:tasks.workunit.client.0.smithi083.stdout: 10 81851 6.53 MB/sec execute 549 sec latency 88.143 ms 2024-04-24T09:16:38.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:16:38 smithi138 ceph-mon[31441]: pgmap v791: 129 pgs: 129 active+clean; 1.2 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.1 MiB/s wr, 198 op/s 2024-04-24T09:16:38.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:16:38 smithi189 ceph-mon[31791]: pgmap v791: 129 pgs: 129 active+clean; 1.2 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.1 MiB/s wr, 198 op/s 2024-04-24T09:16:38.552 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:16:38 smithi083 ceph-mon[27907]: pgmap v791: 129 pgs: 129 active+clean; 1.2 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.1 MiB/s wr, 198 op/s 2024-04-24T09:16:39.221 INFO:tasks.workunit.client.0.smithi083.stdout: 10 82002 6.54 MB/sec execute 550 sec latency 75.258 ms 2024-04-24T09:16:40.221 INFO:tasks.workunit.client.0.smithi083.stdout: 10 82114 6.53 MB/sec execute 551 sec latency 68.290 ms 2024-04-24T09:16:40.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:16:40 smithi138 ceph-mon[31441]: pgmap v792: 129 pgs: 129 active+clean; 1.2 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.6 MiB/s wr, 300 op/s 2024-04-24T09:16:40.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:16:40 smithi189 ceph-mon[31791]: pgmap v792: 129 pgs: 129 active+clean; 1.2 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.6 MiB/s wr, 300 op/s 2024-04-24T09:16:40.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:16:40 smithi083 ceph-mon[27907]: pgmap v792: 129 pgs: 129 active+clean; 1.2 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.6 MiB/s wr, 300 op/s 2024-04-24T09:16:41.221 INFO:tasks.workunit.client.0.smithi083.stdout: 10 82235 6.54 MB/sec execute 552 sec latency 81.922 ms 2024-04-24T09:16:42.221 INFO:tasks.workunit.client.0.smithi083.stdout: 10 82340 6.54 MB/sec execute 553 sec latency 67.937 ms 2024-04-24T09:16:42.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:16:42 smithi138 ceph-mon[31441]: pgmap v793: 129 pgs: 129 active+clean; 1.2 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.5 MiB/s wr, 206 op/s 2024-04-24T09:16:42.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:16:42 smithi189 ceph-mon[31791]: pgmap v793: 129 pgs: 129 active+clean; 1.2 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.5 MiB/s wr, 206 op/s 2024-04-24T09:16:42.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:16:42 smithi083 ceph-mon[27907]: pgmap v793: 129 pgs: 129 active+clean; 1.2 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.5 MiB/s wr, 206 op/s 2024-04-24T09:16:43.222 INFO:tasks.workunit.client.0.smithi083.stdout: 10 82454 6.53 MB/sec execute 554 sec latency 65.258 ms 2024-04-24T09:16:44.222 INFO:tasks.workunit.client.0.smithi083.stdout: 10 82571 6.53 MB/sec execute 555 sec latency 90.602 ms 2024-04-24T09:16:44.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:16:44 smithi138 ceph-mon[31441]: pgmap v794: 129 pgs: 129 active+clean; 1.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 6.8 MiB/s wr, 313 op/s 2024-04-24T09:16:44.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:16:44 smithi189 ceph-mon[31791]: pgmap v794: 129 pgs: 129 active+clean; 1.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 6.8 MiB/s wr, 313 op/s 2024-04-24T09:16:44.552 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:16:44 smithi083 ceph-mon[27907]: pgmap v794: 129 pgs: 129 active+clean; 1.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 6.8 MiB/s wr, 313 op/s 2024-04-24T09:16:45.166 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.165+0000 7f76ac800640 1 -- 172.21.15.83:0/513128206 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76a41053c0 msgr2=0x7f76a4105820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:45.166 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.165+0000 7f76ac800640 1 --2- 172.21.15.83:0/513128206 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76a41053c0 0x7f76a4105820 secure :-1 s=READY pgs=2789 cs=0 l=1 rev1=1 crypto rx=0x7f7694009900 tx=0x7f769402a400 comp rx=0 tx=0).stop 2024-04-24T09:16:45.166 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.165+0000 7f76ac800640 1 -- 172.21.15.83:0/513128206 shutdown_connections 2024-04-24T09:16:45.166 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.165+0000 7f76ac800640 1 --2- 172.21.15.83:0/513128206 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f76a410b120 0x7f76a410d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:45.166 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.165+0000 7f76ac800640 1 --2- 172.21.15.83:0/513128206 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76a41053c0 0x7f76a4105820 unknown :-1 s=CLOSED pgs=2789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:45.166 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.165+0000 7f76ac800640 1 --2- 172.21.15.83:0/513128206 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f76a4100ab0 0x7f76a4104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:45.166 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.165+0000 7f76ac800640 1 -- 172.21.15.83:0/513128206 >> 172.21.15.83:0/513128206 conn(0x7f76a4077000 msgr2=0x7f76a40fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:45.166 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.165+0000 7f76ac800640 1 -- 172.21.15.83:0/513128206 shutdown_connections 2024-04-24T09:16:45.167 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.166+0000 7f76ac800640 1 -- 172.21.15.83:0/513128206 wait complete. 2024-04-24T09:16:45.167 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.167+0000 7f76ac800640 1 Processor -- start 2024-04-24T09:16:45.168 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.167+0000 7f76ac800640 1 -- start start 2024-04-24T09:16:45.168 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.167+0000 7f76ac800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f76a4100ab0 0x7f76a41965d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:45.168 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.167+0000 7f76ac800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76a41053c0 0x7f76a4196b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:45.168 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.168+0000 7f76ac800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f76a410b120 0x7f76a419ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:45.168 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.168+0000 7f76ac800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f76a41100b0 con 0x7f76a41053c0 2024-04-24T09:16:45.168 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.168+0000 7f76ac800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f76a410ff30 con 0x7f76a4100ab0 2024-04-24T09:16:45.168 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.168+0000 7f76ac800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f76a4110230 con 0x7f76a410b120 2024-04-24T09:16:45.168 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.168+0000 7f76a9400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76a41053c0 0x7f76a4196b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:45.169 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.168+0000 7f76a9400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76a41053c0 0x7f76a4196b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42002/0 (socket says 172.21.15.83:42002) 2024-04-24T09:16:45.169 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.168+0000 7f76a9e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f76a4100ab0 0x7f76a41965d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:45.169 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.168+0000 7f76a9400640 1 -- 172.21.15.83:0/1334750179 learned_addr learned my addr 172.21.15.83:0/1334750179 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:16:45.169 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.169+0000 7f76aa800640 1 --2- 172.21.15.83:0/1334750179 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f76a410b120 0x7f76a419ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:45.170 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.170+0000 7f76a9400640 1 -- 172.21.15.83:0/1334750179 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f76a410b120 msgr2=0x7f76a419ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:45.170 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.170+0000 7f76a9400640 1 --2- 172.21.15.83:0/1334750179 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f76a410b120 0x7f76a419ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:45.170 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.170+0000 7f76a9400640 1 -- 172.21.15.83:0/1334750179 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f76a4100ab0 msgr2=0x7f76a41965d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:45.170 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.170+0000 7f76a9400640 1 --2- 172.21.15.83:0/1334750179 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f76a4100ab0 0x7f76a41965d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:45.170 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.170+0000 7f76a9400640 1 -- 172.21.15.83:0/1334750179 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76a419b5c0 con 0x7f76a41053c0 2024-04-24T09:16:45.171 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.170+0000 7f76a9400640 1 --2- 172.21.15.83:0/1334750179 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76a41053c0 0x7f76a4196b10 secure :-1 s=READY pgs=2790 cs=0 l=1 rev1=1 crypto rx=0x7f7694005ab0 tx=0x7f7694003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:45.171 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.171+0000 7f769ea00640 1 -- 172.21.15.83:0/1334750179 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f769403f070 con 0x7f76a41053c0 2024-04-24T09:16:45.172 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.171+0000 7f769ea00640 1 -- 172.21.15.83:0/1334750179 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7694031d60 con 0x7f76a41053c0 2024-04-24T09:16:45.172 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.171+0000 7f769ea00640 1 -- 172.21.15.83:0/1334750179 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7694035030 con 0x7f76a41053c0 2024-04-24T09:16:45.172 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.172+0000 7f76ac800640 1 -- 172.21.15.83:0/1334750179 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76a419b850 con 0x7f76a41053c0 2024-04-24T09:16:45.172 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.172+0000 7f76ac800640 1 -- 172.21.15.83:0/1334750179 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76a41a3140 con 0x7f76a41053c0 2024-04-24T09:16:45.178 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.177+0000 7f769ea00640 1 -- 172.21.15.83:0/1334750179 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7694042070 con 0x7f76a41053c0 2024-04-24T09:16:45.178 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.178+0000 7f76af773740 1 Processor -- start 2024-04-24T09:16:45.178 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.178+0000 7f76af773740 1 -- start start 2024-04-24T09:16:45.178 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.178+0000 7f76af773740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f709511b0 0x558f70951590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:45.179 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.178+0000 7f76af773740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558f70951ad0 0x558f7095be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:45.179 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.178+0000 7f76af773740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f7095c390 0x558f7095e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:45.179 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.178+0000 7f76af773740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x558f708b7900 con 0x558f70951ad0 2024-04-24T09:16:45.179 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.178+0000 7f76af773740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558f7085df20 con 0x558f7095c390 2024-04-24T09:16:45.179 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.178+0000 7f76af773740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x558f708b38c0 con 0x558f709511b0 2024-04-24T09:16:45.180 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.179+0000 7f76a9400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f7095c390 0x558f7095e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:45.180 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.179+0000 7f76a9400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f7095c390 0x558f7095e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:41550/0 (socket says 172.21.15.83:41550) 2024-04-24T09:16:45.180 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.179+0000 7f76a9400640 1 -- 172.21.15.83:0/3316355134 learned_addr learned my addr 172.21.15.83:0/3316355134 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:16:45.180 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.179+0000 7f76aa800640 1 --2- 172.21.15.83:0/3316355134 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f709511b0 0x558f70951590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:45.180 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.179+0000 7f76a9e00640 1 --2- 172.21.15.83:0/3316355134 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558f70951ad0 0x558f7095be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:45.180 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.179+0000 7f76a9400640 1 -- 172.21.15.83:0/3316355134 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f709511b0 msgr2=0x558f70951590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:45.180 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.179+0000 7f76a9400640 1 --2- 172.21.15.83:0/3316355134 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f709511b0 0x558f70951590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:45.180 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.179+0000 7f76a9400640 1 -- 172.21.15.83:0/3316355134 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558f70951ad0 msgr2=0x558f7095be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:45.180 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.179+0000 7f76a9400640 1 --2- 172.21.15.83:0/3316355134 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558f70951ad0 0x558f7095be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:45.180 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.179+0000 7f76a9400640 1 -- 172.21.15.83:0/3316355134 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f70882e10 con 0x558f7095c390 2024-04-24T09:16:45.180 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.180+0000 7f769ea00640 1 --2- 172.21.15.83:0/1334750179 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f768c0a82d0 0x7f768c0aa790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:45.181 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.180+0000 7f769ea00640 1 -- 172.21.15.83:0/1334750179 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 9583+0+0 (secure 0 0 0) 0x7f76940f1390 con 0x7f76a41053c0 2024-04-24T09:16:45.181 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.180+0000 7f76aa800640 1 --2- 172.21.15.83:0/1334750179 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f768c0a82d0 0x7f768c0aa790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:45.181 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.180+0000 7f76a9400640 1 --2- 172.21.15.83:0/3316355134 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f7095c390 0x558f7095e780 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f76940ee5a0 tx=0x7f76940f95a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=363c3a0eff1f3044 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:45.181 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.180+0000 7f7693000640 1 -- 172.21.15.83:0/3316355134 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7694049070 con 0x558f7095c390 2024-04-24T09:16:45.181 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.181+0000 7f7693000640 1 -- 172.21.15.83:0/3316355134 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f76940bcf60 con 0x558f7095c390 2024-04-24T09:16:45.181 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.181+0000 7f7693000640 1 -- 172.21.15.83:0/3316355134 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76940bcd30 con 0x558f7095c390 2024-04-24T09:16:45.182 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.182+0000 7f76af773740 1 -- 172.21.15.83:0/3316355134 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f7095c390 msgr2=0x558f7095e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:45.182 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.182+0000 7f76af773740 1 --2- 172.21.15.83:0/3316355134 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f7095c390 0x558f7095e780 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f76940ee5a0 tx=0x7f76940f95a0 comp rx=0 tx=0).stop 2024-04-24T09:16:45.182 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.182+0000 7f76aa800640 1 --2- 172.21.15.83:0/1334750179 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f768c0a82d0 0x7f768c0aa790 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f76a0005fd0 tx=0x7f76a0009600 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:45.182 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.182+0000 7f76af773740 1 -- 172.21.15.83:0/3316355134 shutdown_connections 2024-04-24T09:16:45.182 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.182+0000 7f76af773740 1 --2- 172.21.15.83:0/3316355134 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f7095c390 0x558f7095e780 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:45.182 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.182+0000 7f76af773740 1 --2- 172.21.15.83:0/3316355134 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558f70951ad0 0x558f7095be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:45.182 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.182+0000 7f76af773740 1 --2- 172.21.15.83:0/3316355134 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f709511b0 0x558f70951590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:45.183 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.182+0000 7f76af773740 1 -- 172.21.15.83:0/3316355134 >> 172.21.15.83:0/3316355134 conn(0x558f7094d6c0 msgr2=0x558f7094daa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:45.183 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.182+0000 7f76af773740 1 -- 172.21.15.83:0/3316355134 shutdown_connections 2024-04-24T09:16:45.183 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.182+0000 7f76af773740 1 -- 172.21.15.83:0/3316355134 wait complete. 2024-04-24T09:16:45.183 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.183+0000 7f76af773740 1 -- start start 2024-04-24T09:16:45.183 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.183+0000 7f76af773740 1 Processor -- start 2024-04-24T09:16:45.183 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.183+0000 7f76af773740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f709511b0 0x558f70959300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:45.184 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.183+0000 7f76af773740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558f70951ad0 0x558f70959840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:45.184 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.184+0000 7f76af773740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f7095c390 0x558f70959d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:45.184 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.184+0000 7f76af773740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x558f7095ed80 con 0x558f70951ad0 2024-04-24T09:16:45.184 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.184+0000 7f76af773740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558f708712c0 con 0x558f709511b0 2024-04-24T09:16:45.184 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.184+0000 7f76af773740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x558f7095ef00 con 0x558f7095c390 2024-04-24T09:16:45.184 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.184+0000 7f76af773740 10 client.0 _ll_register_callbacks cb 0x558f705c9a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:16:45.184 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.184+0000 7f76a9e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f709511b0 0x558f70959300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:45.185 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.184+0000 7f76a9e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f709511b0 0x558f70959300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:41556/0 (socket says 172.21.15.83:41556) 2024-04-24T09:16:45.185 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.184+0000 7f76a9e00640 1 -- 172.21.15.83:0/12005858 learned_addr learned my addr 172.21.15.83:0/12005858 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:16:45.185 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.184+0000 7f76a9400640 1 --2- 172.21.15.83:0/12005858 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558f70951ad0 0x558f70959840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:45.185 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.185+0000 7f76aa800640 1 --2- 172.21.15.83:0/12005858 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f7095c390 0x558f70959d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:45.185 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.185+0000 7f76a9e00640 1 -- 172.21.15.83:0/12005858 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f7095c390 msgr2=0x558f70959d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:45.185 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.185+0000 7f76a9e00640 1 --2- 172.21.15.83:0/12005858 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f7095c390 0x558f70959d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:45.186 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.185+0000 7f76a9e00640 1 -- 172.21.15.83:0/12005858 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558f70951ad0 msgr2=0x558f70959840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:45.186 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.185+0000 7f76a9e00640 1 --2- 172.21.15.83:0/12005858 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558f70951ad0 0x558f70959840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:45.186 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.185+0000 7f76a9e00640 1 -- 172.21.15.83:0/12005858 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f70881fd0 con 0x558f709511b0 2024-04-24T09:16:45.186 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.185+0000 7f76aa800640 1 --2- 172.21.15.83:0/12005858 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f7095c390 0x558f70959d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:16:45.186 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.185+0000 7f76a9400640 1 --2- 172.21.15.83:0/12005858 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558f70951ad0 0x558f70959840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:16:45.186 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.186+0000 7f76a9e00640 1 --2- 172.21.15.83:0/12005858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f709511b0 0x558f70959300 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f7698012700 tx=0x7f7698012bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2fd7ccd6c233ef86 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:45.186 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.186+0000 7f7687400640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:16:45.186 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.186+0000 7f7687400640 1 -- 172.21.15.83:0/12005858 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f769800f920 con 0x558f709511b0 2024-04-24T09:16:45.187 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.186+0000 7f7687400640 1 -- 172.21.15.83:0/12005858 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(132..132 src has 1..132) ==== 9583+0+0 (secure 0 0 0) 0x7f769800fac0 con 0x558f709511b0 2024-04-24T09:16:45.187 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.187+0000 7f76af773740 1 -- 172.21.15.83:0/12005858 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76940bcf60 con 0x558f709511b0 2024-04-24T09:16:45.187 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.187+0000 7f7687400640 1 -- 172.21.15.83:0/12005858 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f769800fcc0 con 0x558f709511b0 2024-04-24T09:16:45.188 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.187+0000 7f76af773740 10 client.41060 fetch_fsmap learned FSMap version 69 2024-04-24T09:16:45.188 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.188+0000 7f76af773740 1 -- 172.21.15.83:0/12005858 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x558f705ca210 con 0x558f709511b0 2024-04-24T09:16:45.188 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.188+0000 7f7687400640 1 -- 172.21.15.83:0/12005858 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f76940bcf60 con 0x558f709511b0 2024-04-24T09:16:45.188 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.188+0000 7f7687400640 1 -- 172.21.15.83:0/12005858 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 69) ==== 8370+0+0 (secure 0 0 0) 0x7f7698010430 con 0x558f709511b0 2024-04-24T09:16:45.189 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.188+0000 7f76af773740 10 client.41060 fetch_fsmap finished waiting for FSMap version 69 2024-04-24T09:16:45.189 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.188+0000 7f76af773740 10 client.41060 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:16:45.189 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.189+0000 7f76af773740 20 client.41060 populate_metadata read hostname 'smithi083' 2024-04-24T09:16:45.189 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.189+0000 7f76af773740 1 --2- 172.21.15.83:0/12005858 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x558f7096c240 0x558f7096e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:45.189 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.189+0000 7f76af773740 4 client.41060 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:16:45.189 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.189+0000 7f76af773740 1 -- 172.21.15.83:0/12005858 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558f708c5320 con 0x558f7096c240 2024-04-24T09:16:45.189 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.189+0000 7f76a9400640 1 --2- 172.21.15.83:0/12005858 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x558f7096c240 0x558f7096e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:45.190 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.190+0000 7f7687400640 10 client.41060 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:16:45.190 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.190+0000 7f76a9400640 1 --2- 172.21.15.83:0/12005858 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x558f7096c240 0x558f7096e6d0 crc :-1 s=READY pgs=611 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50bf10b71cc9d352 server_cookie=113e0d363cb30898 in_seq=0 out_seq=0 2024-04-24T09:16:45.191 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.191+0000 7f7687400640 1 -- 172.21.15.83:0/12005858 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558f708c5320 con 0x558f7096c240 2024-04-24T09:16:45.191 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.191+0000 7f7687400640 10 client.41060 handle_command_reply: tid=0 2024-04-24T09:16:45.191 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.191+0000 7f76af773740 1 client.41060 shutdown 2024-04-24T09:16:45.192 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.191+0000 7f76af773740 1 -- 172.21.15.83:0/12005858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f709511b0 msgr2=0x558f70959300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:45.192 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.191+0000 7f76af773740 1 --2- 172.21.15.83:0/12005858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f709511b0 0x558f70959300 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f7698012700 tx=0x7f7698012bc0 comp rx=0 tx=0).stop 2024-04-24T09:16:45.192 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.192+0000 7f76af773740 1 -- 172.21.15.83:0/12005858 shutdown_connections 2024-04-24T09:16:45.192 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.192+0000 7f76af773740 1 --2- 172.21.15.83:0/12005858 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x558f7096c240 0x558f7096e6d0 crc :-1 s=READY pgs=611 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:45.192 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.192+0000 7f76af773740 1 --2- 172.21.15.83:0/12005858 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f7095c390 0x558f70959d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:45.192 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.192+0000 7f76af773740 1 --2- 172.21.15.83:0/12005858 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558f70951ad0 0x558f70959840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:45.192 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.192+0000 7f76af773740 1 --2- 172.21.15.83:0/12005858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f709511b0 0x558f70959300 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:45.192 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.192+0000 7f7687400640 0 client.41060 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:16:45.192 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.192+0000 7f76af773740 1 -- 172.21.15.83:0/12005858 >> 172.21.15.83:0/12005858 conn(0x558f7094d6c0 msgr2=0x558f70953440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:45.193 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.192+0000 7f76af773740 1 -- 172.21.15.83:0/12005858 shutdown_connections 2024-04-24T09:16:45.193 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.192+0000 7f76af773740 1 -- 172.21.15.83:0/12005858 wait complete. 2024-04-24T09:16:45.193 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.192+0000 7f76af773740 20 client.41060 trim_cache size 0 max 16384 2024-04-24T09:16:45.219 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.219+0000 7f76af773740 1 Processor -- start 2024-04-24T09:16:45.219 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.219+0000 7f76af773740 1 -- start start 2024-04-24T09:16:45.220 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.219+0000 7f76af773740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558f709511b0 0x558f70967af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:45.220 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.219+0000 7f76af773740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f70951ad0 0x558f70973fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:45.220 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.219+0000 7f76af773740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f7095c390 0x558f70974520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:45.220 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.219+0000 7f76af773740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f76a41ad3c0 con 0x558f709511b0 2024-04-24T09:16:45.220 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.219+0000 7f76af773740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f76a41aef60 con 0x558f70951ad0 2024-04-24T09:16:45.220 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.219+0000 7f76af773740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f76a41ada90 con 0x558f7095c390 2024-04-24T09:16:45.222 INFO:tasks.workunit.client.0.smithi083.stdout: 10 82669 6.52 MB/sec execute 556 sec latency 280.412 ms 2024-04-24T09:16:45.222 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.220+0000 7f76a9e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558f709511b0 0x558f70967af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:45.222 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.220+0000 7f76a9e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558f709511b0 0x558f70967af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42022/0 (socket says 172.21.15.83:42022) 2024-04-24T09:16:45.222 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.220+0000 7f76a9e00640 1 -- 172.21.15.83:0/3010492729 learned_addr learned my addr 172.21.15.83:0/3010492729 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:16:45.222 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.220+0000 7f76aa800640 1 --2- 172.21.15.83:0/3010492729 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f7095c390 0x558f70974520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:45.222 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.220+0000 7f76a9e00640 1 -- 172.21.15.83:0/3010492729 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f7095c390 msgr2=0x558f70974520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:45.222 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.220+0000 7f76a9e00640 1 --2- 172.21.15.83:0/3010492729 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f7095c390 0x558f70974520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:45.222 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.220+0000 7f76a9e00640 1 -- 172.21.15.83:0/3010492729 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f70951ad0 msgr2=0x558f70973fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:45.222 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.220+0000 7f76a9e00640 1 --2- 172.21.15.83:0/3010492729 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f70951ad0 0x558f70973fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:45.222 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.220+0000 7f76a9e00640 1 -- 172.21.15.83:0/3010492729 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f70951900 con 0x558f709511b0 2024-04-24T09:16:45.222 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.221+0000 7f76a9e00640 1 --2- 172.21.15.83:0/3010492729 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558f709511b0 0x558f70967af0 secure :-1 s=READY pgs=2791 cs=0 l=1 rev1=1 crypto rx=0x7f7698001eb0 tx=0x7f7698009420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc036fbc1b2e9009 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:45.223 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.222+0000 7f7692600640 1 -- 172.21.15.83:0/3010492729 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f769802b030 con 0x558f709511b0 2024-04-24T09:16:45.223 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.222+0000 7f7692600640 1 -- 172.21.15.83:0/3010492729 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7698009dc0 con 0x558f709511b0 2024-04-24T09:16:45.223 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.222+0000 7f7692600640 1 -- 172.21.15.83:0/3010492729 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7698010ac0 con 0x558f709511b0 2024-04-24T09:16:45.224 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.223+0000 7f76af773740 1 -- 172.21.15.83:0/3010492729 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558f709511b0 msgr2=0x558f70967af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:45.224 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.223+0000 7f76af773740 1 --2- 172.21.15.83:0/3010492729 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558f709511b0 0x558f70967af0 secure :-1 s=READY pgs=2791 cs=0 l=1 rev1=1 crypto rx=0x7f7698001eb0 tx=0x7f7698009420 comp rx=0 tx=0).stop 2024-04-24T09:16:45.224 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.224+0000 7f76af773740 1 -- 172.21.15.83:0/3010492729 shutdown_connections 2024-04-24T09:16:45.224 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.224+0000 7f76af773740 1 --2- 172.21.15.83:0/3010492729 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f7095c390 0x558f70974520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:45.224 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.224+0000 7f76af773740 1 --2- 172.21.15.83:0/3010492729 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f70951ad0 0x558f70973fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:45.224 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.224+0000 7f76af773740 1 --2- 172.21.15.83:0/3010492729 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558f709511b0 0x558f70967af0 unknown :-1 s=CLOSED pgs=2791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:45.224 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.224+0000 7f76af773740 1 -- 172.21.15.83:0/3010492729 >> 172.21.15.83:0/3010492729 conn(0x558f7094d6c0 msgr2=0x558f70954670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:45.225 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.224+0000 7f76af773740 1 -- 172.21.15.83:0/3010492729 shutdown_connections 2024-04-24T09:16:45.225 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.225+0000 7f76af773740 1 -- 172.21.15.83:0/3010492729 wait complete. 2024-04-24T09:16:45.225 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.225+0000 7f76af773740 1 -- start start 2024-04-24T09:16:45.225 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.226+0000 7f76af773740 1 Processor -- start 2024-04-24T09:16:45.227 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.226+0000 7f76af773740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f70951ad0 0x558f709ca810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:45.227 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.226+0000 7f76af773740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f7095c390 0x558f709cad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:45.227 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.226+0000 7f76af773740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558f709cb290 0x558f709da060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:45.227 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.226+0000 7f76af773740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f76a41ae890 con 0x558f709cb290 2024-04-24T09:16:45.227 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.226+0000 7f76af773740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f76a41ae1c0 con 0x558f7095c390 2024-04-24T09:16:45.227 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.226+0000 7f76af773740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f76a41ab3c0 con 0x558f70951ad0 2024-04-24T09:16:45.227 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.226+0000 7f76af773740 10 client.0 _ll_register_callbacks cb 0x7f7698009850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:16:45.228 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.227+0000 7f76aa800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558f709cb290 0x558f709da060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:45.228 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.227+0000 7f76a9e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f70951ad0 0x558f709ca810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:45.228 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.227+0000 7f76aa800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558f709cb290 0x558f709da060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42032/0 (socket says 172.21.15.83:42032) 2024-04-24T09:16:45.228 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.227+0000 7f76aa800640 1 -- 172.21.15.83:0/4140816448 learned_addr learned my addr 172.21.15.83:0/4140816448 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:16:45.228 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.228+0000 7f76aa800640 1 -- 172.21.15.83:0/4140816448 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f70951ad0 msgr2=0x558f709ca810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:45.228 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.228+0000 7f76aa800640 1 --2- 172.21.15.83:0/4140816448 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f70951ad0 0x558f709ca810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:45.228 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.228+0000 7f76aa800640 1 -- 172.21.15.83:0/4140816448 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f7095c390 msgr2=0x558f709cad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:45.228 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.228+0000 7f76aa800640 1 --2- 172.21.15.83:0/4140816448 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f7095c390 0x558f709cad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:45.228 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.228+0000 7f76aa800640 1 -- 172.21.15.83:0/4140816448 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f70696620 con 0x558f709cb290 2024-04-24T09:16:45.229 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.229+0000 7f76aa800640 1 --2- 172.21.15.83:0/4140816448 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558f709cb290 0x558f709da060 secure :-1 s=READY pgs=2792 cs=0 l=1 rev1=1 crypto rx=0x7f76a0011e70 tx=0x7f76a0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa459164d1853065 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:45.229 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.229+0000 7f7687400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:16:45.229 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.229+0000 7f7687400640 1 -- 172.21.15.83:0/4140816448 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a0029420 con 0x558f709cb290 2024-04-24T09:16:45.230 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.230+0000 7f76af773740 1 -- 172.21.15.83:0/4140816448 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7698009dc0 con 0x558f709cb290 2024-04-24T09:16:45.230 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.230+0000 7f7687400640 1 -- 172.21.15.83:0/4140816448 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(132..132 src has 1..132) ==== 9583+0+0 (secure 0 0 0) 0x7f76a0025070 con 0x558f709cb290 2024-04-24T09:16:45.230 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.230+0000 7f7687400640 1 -- 172.21.15.83:0/4140816448 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f76a0029420 con 0x558f709cb290 2024-04-24T09:16:45.230 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.230+0000 7f76af773740 10 client.24003 fetch_fsmap learned FSMap version 69 2024-04-24T09:16:45.231 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.230+0000 7f76af773740 1 -- 172.21.15.83:0/4140816448 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x558f709db890 con 0x558f709cb290 2024-04-24T09:16:45.231 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.231+0000 7f7687400640 1 -- 172.21.15.83:0/4140816448 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7698009dc0 con 0x558f709cb290 2024-04-24T09:16:45.231 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.231+0000 7f7687400640 1 -- 172.21.15.83:0/4140816448 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 69) ==== 8370+0+0 (secure 0 0 0) 0x7f76a000f040 con 0x558f709cb290 2024-04-24T09:16:45.231 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.231+0000 7f76af773740 10 client.24003 fetch_fsmap finished waiting for FSMap version 69 2024-04-24T09:16:45.231 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.231+0000 7f76af773740 10 client.24003 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:16:45.232 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.231+0000 7f76af773740 20 client.24003 populate_metadata read hostname 'smithi083' 2024-04-24T09:16:45.232 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.231+0000 7f76af773740 1 --2- 172.21.15.83:0/4140816448 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x558f709dc040 0x558f709de480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:45.232 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.231+0000 7f76af773740 4 client.24003 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T09:16:45.232 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.231+0000 7f76af773740 1 -- 172.21.15.83:0/4140816448 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558f708bf190 con 0x558f709dc040 2024-04-24T09:16:45.232 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.232+0000 7f76a9e00640 1 --2- 172.21.15.83:0/4140816448 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x558f709dc040 0x558f709de480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:45.232 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.232+0000 7f76a9e00640 1 --2- 172.21.15.83:0/4140816448 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x558f709dc040 0x558f709de480 crc :-1 s=READY pgs=612 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65870d57f4df93d3 server_cookie=2dd26f866de6c2ae in_seq=0 out_seq=0 2024-04-24T09:16:45.232 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.232+0000 7f7687400640 10 client.24003 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:16:45.473 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.472+0000 7f7687400640 1 -- 172.21.15.83:0/4140816448 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558f708bf190 con 0x558f709dc040 2024-04-24T09:16:45.473 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.472+0000 7f7687400640 10 client.24003 handle_command_reply: tid=0 2024-04-24T09:16:45.473 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.472+0000 7f76af773740 1 client.24003 shutdown 2024-04-24T09:16:45.473 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.473+0000 7f76af773740 1 -- 172.21.15.83:0/4140816448 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558f709cb290 msgr2=0x558f709da060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:45.474 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.473+0000 7f76af773740 1 --2- 172.21.15.83:0/4140816448 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558f709cb290 0x558f709da060 secure :-1 s=READY pgs=2792 cs=0 l=1 rev1=1 crypto rx=0x7f76a0011e70 tx=0x7f76a0004a40 comp rx=0 tx=0).stop 2024-04-24T09:16:45.474 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.473+0000 7f76af773740 1 -- 172.21.15.83:0/4140816448 shutdown_connections 2024-04-24T09:16:45.474 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.473+0000 7f76af773740 1 --2- 172.21.15.83:0/4140816448 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x558f709dc040 0x558f709de480 crc :-1 s=READY pgs=612 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:45.474 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.473+0000 7f76af773740 1 --2- 172.21.15.83:0/4140816448 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558f709cb290 0x558f709da060 unknown :-1 s=CLOSED pgs=2792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:45.474 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.474+0000 7f7687400640 0 client.24003 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:16:45.474 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.474+0000 7f76af773740 1 --2- 172.21.15.83:0/4140816448 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f7095c390 0x558f709cad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:45.474 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.474+0000 7f76af773740 1 --2- 172.21.15.83:0/4140816448 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f70951ad0 0x558f709ca810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:45.474 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.474+0000 7f76af773740 1 -- 172.21.15.83:0/4140816448 >> 172.21.15.83:0/4140816448 conn(0x558f7094d6c0 msgr2=0x558f709592c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:45.475 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.474+0000 7f76af773740 1 -- 172.21.15.83:0/4140816448 shutdown_connections 2024-04-24T09:16:45.475 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.475+0000 7f76af773740 1 -- 172.21.15.83:0/4140816448 wait complete. 2024-04-24T09:16:45.475 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.475+0000 7f76af773740 20 client.24003 trim_cache size 0 max 16384 2024-04-24T09:16:45.476 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T09:16:45.476 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T09:16:45.477 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T09:16:45.477 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T09:16:45.477 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.476+0000 7f76ac800640 1 -- 172.21.15.83:0/1334750179 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f768c0a82d0 msgr2=0x7f768c0aa790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:45.477 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.476+0000 7f76ac800640 1 --2- 172.21.15.83:0/1334750179 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f768c0a82d0 0x7f768c0aa790 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f76a0005fd0 tx=0x7f76a0009600 comp rx=0 tx=0).stop 2024-04-24T09:16:45.477 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.476+0000 7f76ac800640 1 -- 172.21.15.83:0/1334750179 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76a41053c0 msgr2=0x7f76a4196b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:45.477 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.476+0000 7f76ac800640 1 --2- 172.21.15.83:0/1334750179 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76a41053c0 0x7f76a4196b10 secure :-1 s=READY pgs=2790 cs=0 l=1 rev1=1 crypto rx=0x7f7694005ab0 tx=0x7f7694003c80 comp rx=0 tx=0).stop 2024-04-24T09:16:45.478 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.477+0000 7f76ac800640 1 -- 172.21.15.83:0/1334750179 shutdown_connections 2024-04-24T09:16:45.478 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.477+0000 7f76ac800640 1 --2- 172.21.15.83:0/1334750179 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f768c0a82d0 0x7f768c0aa790 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:45.478 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.477+0000 7f76ac800640 1 --2- 172.21.15.83:0/1334750179 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f76a410b120 0x7f76a419ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:45.478 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.477+0000 7f76ac800640 1 --2- 172.21.15.83:0/1334750179 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76a41053c0 0x7f76a4196b10 unknown :-1 s=CLOSED pgs=2790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:45.478 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.477+0000 7f76ac800640 1 --2- 172.21.15.83:0/1334750179 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f76a4100ab0 0x7f76a41965d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:45.478 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.477+0000 7f76ac800640 1 -- 172.21.15.83:0/1334750179 >> 172.21.15.83:0/1334750179 conn(0x7f76a4077000 msgr2=0x7f76a410c550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:45.478 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.478+0000 7f76ac800640 1 -- 172.21.15.83:0/1334750179 shutdown_connections 2024-04-24T09:16:45.478 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:45.478+0000 7f76ac800640 1 -- 172.21.15.83:0/1334750179 wait complete. 2024-04-24T09:16:46.222 INFO:tasks.workunit.client.0.smithi083.stdout: 10 82780 6.51 MB/sec execute 557 sec latency 229.830 ms 2024-04-24T09:16:46.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:16:46 smithi138 ceph-mon[31441]: pgmap v795: 129 pgs: 129 active+clean; 1.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.7 MiB/s wr, 209 op/s 2024-04-24T09:16:46.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:16:46 smithi189 ceph-mon[31791]: pgmap v795: 129 pgs: 129 active+clean; 1.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.7 MiB/s wr, 209 op/s 2024-04-24T09:16:46.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:16:46 smithi083 ceph-mon[27907]: pgmap v795: 129 pgs: 129 active+clean; 1.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.7 MiB/s wr, 209 op/s 2024-04-24T09:16:46.993 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T09:16:46.993 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T09:16:47.222 INFO:tasks.workunit.client.0.smithi083.stdout: 10 82890 6.51 MB/sec execute 558 sec latency 145.020 ms 2024-04-24T09:16:47.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.279+0000 7f5673200640 1 -- 172.21.15.83:0/498123617 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5674100670 msgr2=0x7f5674100a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:47.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.279+0000 7f5673200640 1 --2- 172.21.15.83:0/498123617 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5674100670 0x7f5674100a50 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f5660009900 tx=0x7f566002a420 comp rx=0 tx=0).stop 2024-04-24T09:16:47.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.280+0000 7f5673200640 1 -- 172.21.15.83:0/498123617 shutdown_connections 2024-04-24T09:16:47.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.280+0000 7f5673200640 1 --2- 172.21.15.83:0/498123617 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5674107230 0x7f567410ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:47.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.280+0000 7f5673200640 1 --2- 172.21.15.83:0/498123617 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5674101020 0x7f5674106b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:47.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.280+0000 7f5673200640 1 --2- 172.21.15.83:0/498123617 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5674100670 0x7f5674100a50 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:47.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.280+0000 7f5673200640 1 -- 172.21.15.83:0/498123617 >> 172.21.15.83:0/498123617 conn(0x7f5674077390 msgr2=0x7f56740fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:47.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.280+0000 7f5673200640 1 -- 172.21.15.83:0/498123617 shutdown_connections 2024-04-24T09:16:47.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.280+0000 7f5673200640 1 -- 172.21.15.83:0/498123617 wait complete. 2024-04-24T09:16:47.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.282+0000 7f5673200640 1 Processor -- start 2024-04-24T09:16:47.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.282+0000 7f5673200640 1 -- start start 2024-04-24T09:16:47.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.282+0000 7f5673200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5674100670 0x7f5674073a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:47.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.282+0000 7f5673200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5674107230 0x7f5674101510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:47.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.283+0000 7f5673200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5674101a50 0x7f5674101ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:47.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.283+0000 7f5673200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f567406c320 con 0x7f5674100670 2024-04-24T09:16:47.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.283+0000 7f5671e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5674100670 0x7f5674073a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:47.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.283+0000 7f5673200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f567406c1a0 con 0x7f5674107230 2024-04-24T09:16:47.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.283+0000 7f5673200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f567406c4a0 con 0x7f5674101a50 2024-04-24T09:16:47.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.283+0000 7f5671e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5674100670 0x7f5674073a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42048/0 (socket says 172.21.15.83:42048) 2024-04-24T09:16:47.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.283+0000 7f5671e00640 1 -- 172.21.15.83:0/678159084 learned_addr learned my addr 172.21.15.83:0/678159084 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:16:47.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.283+0000 7f5672800640 1 --2- 172.21.15.83:0/678159084 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5674101a50 0x7f5674101ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:47.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.283+0000 7f5671e00640 1 -- 172.21.15.83:0/678159084 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5674101a50 msgr2=0x7f5674101ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:47.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.283+0000 7f5671e00640 1 --2- 172.21.15.83:0/678159084 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5674101a50 0x7f5674101ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:47.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.283+0000 7f5671e00640 1 -- 172.21.15.83:0/678159084 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5674107230 msgr2=0x7f5674101510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:47.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.283+0000 7f5671400640 1 --2- 172.21.15.83:0/678159084 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5674107230 0x7f5674101510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:47.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.284+0000 7f5671e00640 1 --2- 172.21.15.83:0/678159084 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5674107230 0x7f5674101510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:47.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.284+0000 7f5671e00640 1 -- 172.21.15.83:0/678159084 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56741b3da0 con 0x7f5674100670 2024-04-24T09:16:47.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.284+0000 7f5671400640 1 --2- 172.21.15.83:0/678159084 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5674107230 0x7f5674101510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:16:47.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.284+0000 7f5671e00640 1 --2- 172.21.15.83:0/678159084 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5674100670 0x7f5674073a50 secure :-1 s=READY pgs=2793 cs=0 l=1 rev1=1 crypto rx=0x7f566002aa10 tx=0x7f5660003e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:47.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.284+0000 7f566aa00640 1 -- 172.21.15.83:0/678159084 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5660040030 con 0x7f5674100670 2024-04-24T09:16:47.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.284+0000 7f566aa00640 1 -- 172.21.15.83:0/678159084 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f566003c8f0 con 0x7f5674100670 2024-04-24T09:16:47.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.284+0000 7f566aa00640 1 -- 172.21.15.83:0/678159084 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5660031860 con 0x7f5674100670 2024-04-24T09:16:47.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.285+0000 7f5673200640 1 -- 172.21.15.83:0/678159084 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56741b3f70 con 0x7f5674100670 2024-04-24T09:16:47.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.285+0000 7f5673200640 1 -- 172.21.15.83:0/678159084 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56741b4320 con 0x7f5674100670 2024-04-24T09:16:47.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.289+0000 7f566aa00640 1 -- 172.21.15.83:0/678159084 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5660043030 con 0x7f5674100670 2024-04-24T09:16:47.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.289+0000 7f567bf6e740 1 Processor -- start 2024-04-24T09:16:47.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.289+0000 7f567bf6e740 1 -- start start 2024-04-24T09:16:47.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.290+0000 7f567bf6e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d6bf511a30 0x55d6bf511e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:47.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.290+0000 7f567bf6e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6bf512350 0x55d6bf51c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:47.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.290+0000 7f567bf6e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d6bf51ca20 0x55d6bf51ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:47.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.290+0000 7f567bf6e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d6bf4771f0 con 0x55d6bf512350 2024-04-24T09:16:47.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.290+0000 7f567bf6e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d6bf41d350 con 0x55d6bf51ca20 2024-04-24T09:16:47.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.290+0000 7f567bf6e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d6bf4731b0 con 0x55d6bf511a30 2024-04-24T09:16:47.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.291+0000 7f5671e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6bf512350 0x55d6bf51c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:47.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.291+0000 7f5671400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d6bf51ca20 0x55d6bf51ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:47.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.291+0000 7f5671400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d6bf51ca20 0x55d6bf51ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:41604/0 (socket says 172.21.15.83:41604) 2024-04-24T09:16:47.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.291+0000 7f5671400640 1 -- 172.21.15.83:0/3920086052 learned_addr learned my addr 172.21.15.83:0/3920086052 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:16:47.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.291+0000 7f5672800640 1 --2- 172.21.15.83:0/3920086052 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d6bf511a30 0x55d6bf511e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:47.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.291+0000 7f5671400640 1 -- 172.21.15.83:0/3920086052 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d6bf511a30 msgr2=0x55d6bf511e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:47.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.291+0000 7f5671400640 1 --2- 172.21.15.83:0/3920086052 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d6bf511a30 0x55d6bf511e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:47.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.291+0000 7f5671400640 1 -- 172.21.15.83:0/3920086052 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6bf512350 msgr2=0x55d6bf51c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:47.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.292+0000 7f5671400640 1 --2- 172.21.15.83:0/3920086052 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6bf512350 0x55d6bf51c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:47.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.292+0000 7f5671400640 1 -- 172.21.15.83:0/3920086052 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6bf471f50 con 0x55d6bf51ca20 2024-04-24T09:16:47.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.292+0000 7f5671e00640 1 --2- 172.21.15.83:0/3920086052 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6bf512350 0x55d6bf51c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:16:47.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.292+0000 7f566aa00640 1 --2- 172.21.15.83:0/678159084 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f56580a3e60 0x7f56580a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:47.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.292+0000 7f566aa00640 1 -- 172.21.15.83:0/678159084 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 9583+0+0 (secure 0 0 0) 0x7f56600f15c0 con 0x7f5674100670 2024-04-24T09:16:47.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.292+0000 7f5671400640 1 --2- 172.21.15.83:0/3920086052 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d6bf51ca20 0x55d6bf51ee10 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f5664009bf0 tx=0x7f566400e6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b54dabe8a76e71de server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:47.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.292+0000 7f5672800640 1 --2- 172.21.15.83:0/678159084 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f56580a3e60 0x7f56580a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:47.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.293+0000 7f565f400640 1 -- 172.21.15.83:0/3920086052 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5664016070 con 0x55d6bf51ca20 2024-04-24T09:16:47.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.293+0000 7f565f400640 1 -- 172.21.15.83:0/3920086052 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f56640054d0 con 0x55d6bf51ca20 2024-04-24T09:16:47.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.293+0000 7f565f400640 1 -- 172.21.15.83:0/3920086052 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5664011470 con 0x55d6bf51ca20 2024-04-24T09:16:47.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.293+0000 7f567bf6e740 1 -- 172.21.15.83:0/3920086052 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d6bf51ca20 msgr2=0x55d6bf51ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:47.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.294+0000 7f567bf6e740 1 --2- 172.21.15.83:0/3920086052 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d6bf51ca20 0x55d6bf51ee10 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f5664009bf0 tx=0x7f566400e6c0 comp rx=0 tx=0).stop 2024-04-24T09:16:47.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.294+0000 7f5672800640 1 --2- 172.21.15.83:0/678159084 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f56580a3e60 0x7f56580a6320 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f566c005fd0 tx=0x7f566c009500 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:47.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.294+0000 7f567bf6e740 1 -- 172.21.15.83:0/3920086052 shutdown_connections 2024-04-24T09:16:47.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.294+0000 7f567bf6e740 1 --2- 172.21.15.83:0/3920086052 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d6bf51ca20 0x55d6bf51ee10 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:47.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.294+0000 7f567bf6e740 1 --2- 172.21.15.83:0/3920086052 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6bf512350 0x55d6bf51c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:47.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.294+0000 7f567bf6e740 1 --2- 172.21.15.83:0/3920086052 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d6bf511a30 0x55d6bf511e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:47.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.294+0000 7f567bf6e740 1 -- 172.21.15.83:0/3920086052 >> 172.21.15.83:0/3920086052 conn(0x55d6bf50dd50 msgr2=0x55d6bf50e130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:47.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.294+0000 7f567bf6e740 1 -- 172.21.15.83:0/3920086052 shutdown_connections 2024-04-24T09:16:47.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.294+0000 7f567bf6e740 1 -- 172.21.15.83:0/3920086052 wait complete. 2024-04-24T09:16:47.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.294+0000 7f567bf6e740 1 -- start start 2024-04-24T09:16:47.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.295+0000 7f567bf6e740 1 Processor -- start 2024-04-24T09:16:47.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.295+0000 7f567bf6e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d6bf511a30 0x55d6bf519980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:47.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.295+0000 7f567bf6e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d6bf512350 0x55d6bf519ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:47.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.295+0000 7f567bf6e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6bf51ca20 0x55d6bf51a400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:47.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.295+0000 7f567bf6e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d6bf51f810 con 0x55d6bf51ca20 2024-04-24T09:16:47.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.295+0000 7f567bf6e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d6bf430690 con 0x55d6bf511a30 2024-04-24T09:16:47.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.295+0000 7f567bf6e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d6bf51f990 con 0x55d6bf512350 2024-04-24T09:16:47.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.295+0000 7f567bf6e740 10 client.0 _ll_register_callbacks cb 0x55d6bf46c650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:16:47.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.296+0000 7f5671e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d6bf511a30 0x55d6bf519980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:47.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.296+0000 7f5671e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d6bf511a30 0x55d6bf519980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:41606/0 (socket says 172.21.15.83:41606) 2024-04-24T09:16:47.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.296+0000 7f5671e00640 1 -- 172.21.15.83:0/1206525904 learned_addr learned my addr 172.21.15.83:0/1206525904 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:16:47.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.296+0000 7f5672800640 1 --2- 172.21.15.83:0/1206525904 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6bf51ca20 0x55d6bf51a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:47.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.296+0000 7f5671400640 1 --2- 172.21.15.83:0/1206525904 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d6bf512350 0x55d6bf519ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:47.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.296+0000 7f5671e00640 1 -- 172.21.15.83:0/1206525904 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d6bf512350 msgr2=0x55d6bf519ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:47.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.296+0000 7f5671e00640 1 --2- 172.21.15.83:0/1206525904 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d6bf512350 0x55d6bf519ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:47.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.296+0000 7f5671e00640 1 -- 172.21.15.83:0/1206525904 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6bf51ca20 msgr2=0x55d6bf51a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:47.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.296+0000 7f5671e00640 1 --2- 172.21.15.83:0/1206525904 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6bf51ca20 0x55d6bf51a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:47.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.296+0000 7f5671e00640 1 -- 172.21.15.83:0/1206525904 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6bf42f920 con 0x55d6bf511a30 2024-04-24T09:16:47.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.296+0000 7f5671400640 1 --2- 172.21.15.83:0/1206525904 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d6bf512350 0x55d6bf519ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:16:47.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.297+0000 7f5671e00640 1 --2- 172.21.15.83:0/1206525904 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d6bf511a30 0x55d6bf519980 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f5660006010 tx=0x7f56600bb0f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=830f65644357cce2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:47.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.297+0000 7f5653e00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:16:47.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.297+0000 7f5653e00640 1 -- 172.21.15.83:0/1206525904 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f566004a040 con 0x55d6bf511a30 2024-04-24T09:16:47.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.297+0000 7f5653e00640 1 -- 172.21.15.83:0/1206525904 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(132..132 src has 1..132) ==== 9583+0+0 (secure 0 0 0) 0x7f56600f1170 con 0x55d6bf511a30 2024-04-24T09:16:47.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.297+0000 7f567bf6e740 1 -- 172.21.15.83:0/1206525904 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56640054d0 con 0x55d6bf511a30 2024-04-24T09:16:47.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.297+0000 7f5653e00640 1 -- 172.21.15.83:0/1206525904 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f56600b92f0 con 0x55d6bf511a30 2024-04-24T09:16:47.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.297+0000 7f567bf6e740 10 client.41069 fetch_fsmap learned FSMap version 69 2024-04-24T09:16:47.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.297+0000 7f567bf6e740 1 -- 172.21.15.83:0/1206525904 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x55d6bf2537d0 con 0x55d6bf511a30 2024-04-24T09:16:47.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.298+0000 7f5653e00640 1 -- 172.21.15.83:0/1206525904 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f56640054d0 con 0x55d6bf511a30 2024-04-24T09:16:47.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.298+0000 7f5653e00640 1 -- 172.21.15.83:0/1206525904 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 69) ==== 8370+0+0 (secure 0 0 0) 0x7f56600bd340 con 0x55d6bf511a30 2024-04-24T09:16:47.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.298+0000 7f567bf6e740 10 client.41069 fetch_fsmap finished waiting for FSMap version 69 2024-04-24T09:16:47.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.298+0000 7f567bf6e740 10 client.41069 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:16:47.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.298+0000 7f567bf6e740 20 client.41069 populate_metadata read hostname 'smithi083' 2024-04-24T09:16:47.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.298+0000 7f567bf6e740 1 --2- 172.21.15.83:0/1206525904 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d6bf52cd10 0x55d6bf52f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:47.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.298+0000 7f567bf6e740 4 client.41069 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:16:47.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.298+0000 7f567bf6e740 1 -- 172.21.15.83:0/1206525904 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d6bf484c10 con 0x55d6bf52cd10 2024-04-24T09:16:47.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.299+0000 7f5671400640 1 --2- 172.21.15.83:0/1206525904 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d6bf52cd10 0x55d6bf52f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:47.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.300+0000 7f5671400640 1 --2- 172.21.15.83:0/1206525904 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d6bf52cd10 0x55d6bf52f1b0 crc :-1 s=READY pgs=613 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd678c44bcb8aa6b server_cookie=26cba83bedf60070 in_seq=0 out_seq=0 2024-04-24T09:16:47.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.300+0000 7f5653e00640 10 client.41069 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:16:47.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.303+0000 7f5653e00640 1 -- 172.21.15.83:0/1206525904 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d6bf484c10 con 0x55d6bf52cd10 2024-04-24T09:16:47.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.303+0000 7f5653e00640 10 client.41069 handle_command_reply: tid=0 2024-04-24T09:16:47.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.303+0000 7f567bf6e740 1 client.41069 shutdown 2024-04-24T09:16:47.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.303+0000 7f567bf6e740 1 -- 172.21.15.83:0/1206525904 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d6bf511a30 msgr2=0x55d6bf519980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:47.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.303+0000 7f567bf6e740 1 --2- 172.21.15.83:0/1206525904 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d6bf511a30 0x55d6bf519980 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f5660006010 tx=0x7f56600bb0f0 comp rx=0 tx=0).stop 2024-04-24T09:16:47.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.303+0000 7f567bf6e740 1 -- 172.21.15.83:0/1206525904 shutdown_connections 2024-04-24T09:16:47.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.303+0000 7f567bf6e740 1 --2- 172.21.15.83:0/1206525904 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d6bf52cd10 0x55d6bf52f1b0 crc :-1 s=READY pgs=613 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:47.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.303+0000 7f567bf6e740 1 --2- 172.21.15.83:0/1206525904 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6bf51ca20 0x55d6bf51a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:47.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.303+0000 7f567bf6e740 1 --2- 172.21.15.83:0/1206525904 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d6bf512350 0x55d6bf519ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:47.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.303+0000 7f567bf6e740 1 --2- 172.21.15.83:0/1206525904 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d6bf511a30 0x55d6bf519980 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:47.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.303+0000 7f567bf6e740 1 -- 172.21.15.83:0/1206525904 >> 172.21.15.83:0/1206525904 conn(0x55d6bf50dd50 msgr2=0x55d6bf50cfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:47.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.303+0000 7f5653e00640 0 client.41069 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:16:47.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.304+0000 7f567bf6e740 1 -- 172.21.15.83:0/1206525904 shutdown_connections 2024-04-24T09:16:47.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.304+0000 7f567bf6e740 1 -- 172.21.15.83:0/1206525904 wait complete. 2024-04-24T09:16:47.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.304+0000 7f567bf6e740 20 client.41069 trim_cache size 0 max 16384 2024-04-24T09:16:47.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.351+0000 7f567bf6e740 1 Processor -- start 2024-04-24T09:16:47.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.351+0000 7f567bf6e740 1 -- start start 2024-04-24T09:16:47.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.352+0000 7f567bf6e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d6bf511a30 0x55d6bf528700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:47.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.352+0000 7f567bf6e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6bf512350 0x55d6bf534a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:47.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.352+0000 7f567bf6e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d6bf51ca20 0x55d6bf534fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:47.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.352+0000 7f567bf6e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f56741be290 con 0x55d6bf512350 2024-04-24T09:16:47.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.352+0000 7f567bf6e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f56741bfe30 con 0x55d6bf51ca20 2024-04-24T09:16:47.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.352+0000 7f567bf6e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f56741be960 con 0x55d6bf511a30 2024-04-24T09:16:47.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.353+0000 7f5671400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6bf512350 0x55d6bf534a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:47.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.353+0000 7f5672800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d6bf51ca20 0x55d6bf534fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:47.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.353+0000 7f5672800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d6bf51ca20 0x55d6bf534fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:41622/0 (socket says 172.21.15.83:41622) 2024-04-24T09:16:47.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.353+0000 7f5672800640 1 -- 172.21.15.83:0/1445188661 learned_addr learned my addr 172.21.15.83:0/1445188661 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:16:47.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.353+0000 7f5671400640 1 -- 172.21.15.83:0/1445188661 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d6bf511a30 msgr2=0x55d6bf528700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:47.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.353+0000 7f5671e00640 1 --2- 172.21.15.83:0/1445188661 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d6bf511a30 0x55d6bf528700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:47.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.353+0000 7f5671400640 1 --2- 172.21.15.83:0/1445188661 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d6bf511a30 0x55d6bf528700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:47.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.354+0000 7f5671400640 1 -- 172.21.15.83:0/1445188661 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d6bf51ca20 msgr2=0x55d6bf534fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:47.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.354+0000 7f5671400640 1 --2- 172.21.15.83:0/1445188661 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d6bf51ca20 0x55d6bf534fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:47.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.354+0000 7f5671400640 1 -- 172.21.15.83:0/1445188661 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6bf51f180 con 0x55d6bf512350 2024-04-24T09:16:47.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.354+0000 7f5671e00640 1 --2- 172.21.15.83:0/1445188661 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d6bf511a30 0x55d6bf528700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:16:47.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.354+0000 7f5671400640 1 --2- 172.21.15.83:0/1445188661 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6bf512350 0x55d6bf534a80 secure :-1 s=READY pgs=2794 cs=0 l=1 rev1=1 crypto rx=0x7f5664000c00 tx=0x7f5664009ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=558a8e4ca0dfa5f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:47.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.355+0000 7f565ea00640 1 -- 172.21.15.83:0/1445188661 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5664016040 con 0x55d6bf512350 2024-04-24T09:16:47.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.355+0000 7f565ea00640 1 -- 172.21.15.83:0/1445188661 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5664029170 con 0x55d6bf512350 2024-04-24T09:16:47.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.355+0000 7f565ea00640 1 -- 172.21.15.83:0/1445188661 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5664004f00 con 0x55d6bf512350 2024-04-24T09:16:47.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.356+0000 7f567bf6e740 1 -- 172.21.15.83:0/1445188661 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6bf512350 msgr2=0x55d6bf534a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:47.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.356+0000 7f567bf6e740 1 --2- 172.21.15.83:0/1445188661 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6bf512350 0x55d6bf534a80 secure :-1 s=READY pgs=2794 cs=0 l=1 rev1=1 crypto rx=0x7f5664000c00 tx=0x7f5664009ee0 comp rx=0 tx=0).stop 2024-04-24T09:16:47.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.356+0000 7f567bf6e740 1 -- 172.21.15.83:0/1445188661 shutdown_connections 2024-04-24T09:16:47.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.356+0000 7f567bf6e740 1 --2- 172.21.15.83:0/1445188661 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d6bf51ca20 0x55d6bf534fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:47.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.356+0000 7f567bf6e740 1 --2- 172.21.15.83:0/1445188661 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6bf512350 0x55d6bf534a80 secure :-1 s=CLOSED pgs=2794 cs=0 l=1 rev1=1 crypto rx=0x7f5664000c00 tx=0x7f5664009ee0 comp rx=0 tx=0).stop 2024-04-24T09:16:47.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.356+0000 7f567bf6e740 1 --2- 172.21.15.83:0/1445188661 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d6bf511a30 0x55d6bf528700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:47.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.356+0000 7f567bf6e740 1 -- 172.21.15.83:0/1445188661 >> 172.21.15.83:0/1445188661 conn(0x55d6bf50dd50 msgr2=0x55d6bf513a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:47.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.356+0000 7f567bf6e740 1 -- 172.21.15.83:0/1445188661 shutdown_connections 2024-04-24T09:16:47.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.357+0000 7f567bf6e740 1 -- 172.21.15.83:0/1445188661 wait complete. 2024-04-24T09:16:47.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.357+0000 7f567bf6e740 1 -- start start 2024-04-24T09:16:47.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.357+0000 7f567bf6e740 1 Processor -- start 2024-04-24T09:16:47.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.360+0000 7f567bf6e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d6bf511a30 0x55d6bf5168c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:47.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.360+0000 7f567bf6e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6bf51ca20 0x55d6bf516e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:47.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.362+0000 7f567bf6e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d6bf517340 0x55d6bf5922f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:47.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.362+0000 7f567bf6e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f56741bf760 con 0x55d6bf51ca20 2024-04-24T09:16:47.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.361+0000 7f5671e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d6bf511a30 0x55d6bf5168c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:47.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.363+0000 7f5671e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d6bf511a30 0x55d6bf5168c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:41626/0 (socket says 172.21.15.83:41626) 2024-04-24T09:16:47.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.363+0000 7f5671e00640 1 -- 172.21.15.83:0/3295072834 learned_addr learned my addr 172.21.15.83:0/3295072834 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:16:47.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.363+0000 7f567bf6e740 1 -- 172.21.15.83:0/3295072834 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f56741bf090 con 0x55d6bf511a30 2024-04-24T09:16:47.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.363+0000 7f567bf6e740 1 -- 172.21.15.83:0/3295072834 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f56741bc290 con 0x55d6bf517340 2024-04-24T09:16:47.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.363+0000 7f567bf6e740 10 client.0 _ll_register_callbacks cb 0x7f5660003730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:16:47.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.363+0000 7f5672800640 1 --2- 172.21.15.83:0/3295072834 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d6bf517340 0x55d6bf5922f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:47.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.363+0000 7f5671e00640 1 -- 172.21.15.83:0/3295072834 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d6bf517340 msgr2=0x55d6bf5922f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:47.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.363+0000 7f5671e00640 1 --2- 172.21.15.83:0/3295072834 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d6bf517340 0x55d6bf5922f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:47.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.363+0000 7f5671400640 1 --2- 172.21.15.83:0/3295072834 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6bf51ca20 0x55d6bf516e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:47.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.363+0000 7f5671e00640 1 -- 172.21.15.83:0/3295072834 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6bf51ca20 msgr2=0x55d6bf516e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:47.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.363+0000 7f5671e00640 1 --2- 172.21.15.83:0/3295072834 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6bf51ca20 0x55d6bf516e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:47.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.363+0000 7f5671e00640 1 -- 172.21.15.83:0/3295072834 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6bf1893f0 con 0x55d6bf511a30 2024-04-24T09:16:47.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.364+0000 7f5672800640 1 --2- 172.21.15.83:0/3295072834 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d6bf517340 0x55d6bf5922f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:16:47.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.364+0000 7f5671400640 1 --2- 172.21.15.83:0/3295072834 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6bf51ca20 0x55d6bf516e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:16:47.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.365+0000 7f5671e00640 1 --2- 172.21.15.83:0/3295072834 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d6bf511a30 0x55d6bf5168c0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f56600c5d70 tx=0x7f566003cf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67659c16e240544e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:47.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.365+0000 7f5653e00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:16:47.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.365+0000 7f5653e00640 1 -- 172.21.15.83:0/3295072834 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f566004a070 con 0x55d6bf511a30 2024-04-24T09:16:47.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.366+0000 7f567bf6e740 1 -- 172.21.15.83:0/3295072834 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5664029170 con 0x55d6bf511a30 2024-04-24T09:16:47.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.366+0000 7f5653e00640 1 -- 172.21.15.83:0/3295072834 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(132..132 src has 1..132) ==== 9583+0+0 (secure 0 0 0) 0x7f56600c58c0 con 0x55d6bf511a30 2024-04-24T09:16:47.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.366+0000 7f5653e00640 1 -- 172.21.15.83:0/3295072834 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f566004a070 con 0x55d6bf511a30 2024-04-24T09:16:47.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.367+0000 7f567bf6e740 10 client.41075 fetch_fsmap learned FSMap version 69 2024-04-24T09:16:47.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.367+0000 7f567bf6e740 1 -- 172.21.15.83:0/3295072834 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x55d6bf1490f0 con 0x55d6bf511a30 2024-04-24T09:16:47.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.368+0000 7f5653e00640 1 -- 172.21.15.83:0/3295072834 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5664029170 con 0x55d6bf511a30 2024-04-24T09:16:47.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.368+0000 7f5653e00640 1 -- 172.21.15.83:0/3295072834 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 69) ==== 8370+0+0 (secure 0 0 0) 0x7f56600bc530 con 0x55d6bf511a30 2024-04-24T09:16:47.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.368+0000 7f567bf6e740 10 client.41075 fetch_fsmap finished waiting for FSMap version 69 2024-04-24T09:16:47.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.368+0000 7f567bf6e740 10 client.41075 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:16:47.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.368+0000 7f567bf6e740 20 client.41075 populate_metadata read hostname 'smithi083' 2024-04-24T09:16:47.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.368+0000 7f567bf6e740 1 --2- 172.21.15.83:0/3295072834 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d6bf593d50 0x55d6bf596170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:47.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.368+0000 7f567bf6e740 4 client.41075 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T09:16:47.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.368+0000 7f567bf6e740 1 -- 172.21.15.83:0/3295072834 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d6bf47ea80 con 0x55d6bf593d50 2024-04-24T09:16:47.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.368+0000 7f5671400640 1 --2- 172.21.15.83:0/3295072834 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d6bf593d50 0x55d6bf596170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:47.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.369+0000 7f5653e00640 10 client.41075 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:16:47.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.370+0000 7f5671400640 1 --2- 172.21.15.83:0/3295072834 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d6bf593d50 0x55d6bf596170 crc :-1 s=READY pgs=614 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44cf88de1708775a server_cookie=b896016b1e19361b in_seq=0 out_seq=0 2024-04-24T09:16:47.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.370+0000 7f5653e00640 1 -- 172.21.15.83:0/3295072834 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55d6bf47ea80 con 0x55d6bf593d50 2024-04-24T09:16:47.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.370+0000 7f5653e00640 10 client.41075 handle_command_reply: tid=0 2024-04-24T09:16:47.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.370+0000 7f567bf6e740 1 client.41075 shutdown 2024-04-24T09:16:47.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.371+0000 7f567bf6e740 1 -- 172.21.15.83:0/3295072834 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d6bf511a30 msgr2=0x55d6bf5168c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:47.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.371+0000 7f567bf6e740 1 --2- 172.21.15.83:0/3295072834 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d6bf511a30 0x55d6bf5168c0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f56600c5d70 tx=0x7f566003cf10 comp rx=0 tx=0).stop 2024-04-24T09:16:47.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.371+0000 7f567bf6e740 1 -- 172.21.15.83:0/3295072834 shutdown_connections 2024-04-24T09:16:47.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.371+0000 7f567bf6e740 1 --2- 172.21.15.83:0/3295072834 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55d6bf593d50 0x55d6bf596170 crc :-1 s=READY pgs=614 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:47.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.371+0000 7f567bf6e740 1 --2- 172.21.15.83:0/3295072834 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d6bf517340 0x55d6bf5922f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:47.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.371+0000 7f567bf6e740 1 --2- 172.21.15.83:0/3295072834 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6bf51ca20 0x55d6bf516e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:47.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.371+0000 7f567bf6e740 1 --2- 172.21.15.83:0/3295072834 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d6bf511a30 0x55d6bf5168c0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:47.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.371+0000 7f567bf6e740 1 -- 172.21.15.83:0/3295072834 >> 172.21.15.83:0/3295072834 conn(0x55d6bf50dd50 msgr2=0x55d6bf51b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:47.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.371+0000 7f5653e00640 0 client.41075 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:16:47.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.372+0000 7f567bf6e740 1 -- 172.21.15.83:0/3295072834 shutdown_connections 2024-04-24T09:16:47.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.373+0000 7f567bf6e740 1 -- 172.21.15.83:0/3295072834 wait complete. 2024-04-24T09:16:47.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.373+0000 7f567bf6e740 20 client.41075 trim_cache size 0 max 16384 2024-04-24T09:16:47.375 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T09:16:47.375 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-04-24T09:16:47.375 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-04-24T09:16:47.375 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T09:16:47.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.374+0000 7f5673200640 1 -- 172.21.15.83:0/678159084 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f56580a3e60 msgr2=0x7f56580a6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:47.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.374+0000 7f5673200640 1 --2- 172.21.15.83:0/678159084 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f56580a3e60 0x7f56580a6320 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f566c005fd0 tx=0x7f566c009500 comp rx=0 tx=0).stop 2024-04-24T09:16:47.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.374+0000 7f5673200640 1 -- 172.21.15.83:0/678159084 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5674100670 msgr2=0x7f5674073a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:47.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.374+0000 7f5673200640 1 --2- 172.21.15.83:0/678159084 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5674100670 0x7f5674073a50 secure :-1 s=READY pgs=2793 cs=0 l=1 rev1=1 crypto rx=0x7f566002aa10 tx=0x7f5660003e90 comp rx=0 tx=0).stop 2024-04-24T09:16:47.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.375+0000 7f5673200640 1 -- 172.21.15.83:0/678159084 shutdown_connections 2024-04-24T09:16:47.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.375+0000 7f5673200640 1 --2- 172.21.15.83:0/678159084 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f56580a3e60 0x7f56580a6320 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:47.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.375+0000 7f5673200640 1 --2- 172.21.15.83:0/678159084 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5674101a50 0x7f5674101ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:47.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.375+0000 7f5673200640 1 --2- 172.21.15.83:0/678159084 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5674107230 0x7f5674101510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:47.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.375+0000 7f5673200640 1 --2- 172.21.15.83:0/678159084 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5674100670 0x7f5674073a50 unknown :-1 s=CLOSED pgs=2793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:47.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.375+0000 7f5673200640 1 -- 172.21.15.83:0/678159084 >> 172.21.15.83:0/678159084 conn(0x7f5674077390 msgr2=0x7f56741054c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:47.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.376+0000 7f5673200640 1 -- 172.21.15.83:0/678159084 shutdown_connections 2024-04-24T09:16:47.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.376+0000 7f5673200640 1 -- 172.21.15.83:0/678159084 wait complete. 2024-04-24T09:16:47.418 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T09:16:47.419 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T09:16:47.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.684+0000 7f264a000640 1 -- 172.21.15.83:0/3911416847 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2644104260 msgr2=0x7f2644104640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:47.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.684+0000 7f264a000640 1 --2- 172.21.15.83:0/3911416847 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2644104260 0x7f2644104640 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f2634009ca0 tx=0x7f263402a8c0 comp rx=0 tx=0).stop 2024-04-24T09:16:47.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.684+0000 7f264a000640 1 -- 172.21.15.83:0/3911416847 shutdown_connections 2024-04-24T09:16:47.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.684+0000 7f264a000640 1 --2- 172.21.15.83:0/3911416847 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2644109330 0x7f2644110f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:47.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.684+0000 7f264a000640 1 --2- 172.21.15.83:0/3911416847 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2644104c10 0x7f2644108c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:47.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.684+0000 7f264a000640 1 --2- 172.21.15.83:0/3911416847 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2644104260 0x7f2644104640 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:47.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.684+0000 7f264a000640 1 -- 172.21.15.83:0/3911416847 >> 172.21.15.83:0/3911416847 conn(0x7f26440ffaa0 msgr2=0x7f2644101ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:47.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.684+0000 7f264a000640 1 -- 172.21.15.83:0/3911416847 shutdown_connections 2024-04-24T09:16:47.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.684+0000 7f264a000640 1 -- 172.21.15.83:0/3911416847 wait complete. 2024-04-24T09:16:47.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.685+0000 7f264a000640 1 Processor -- start 2024-04-24T09:16:47.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.686+0000 7f264a000640 1 -- start start 2024-04-24T09:16:47.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.686+0000 7f264a000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2644104260 0x7f264410f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:47.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.686+0000 7f264a000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2644109330 0x7f264410f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:47.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.686+0000 7f264a000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f264410fe00 0x7f26441aaba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:47.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.686+0000 7f264a000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f26440770f0 con 0x7f2644104260 2024-04-24T09:16:47.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.686+0000 7f264a000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2644076f70 con 0x7f2644109330 2024-04-24T09:16:47.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.686+0000 7f264a000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2644077270 con 0x7f264410fe00 2024-04-24T09:16:47.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.686+0000 7f2648c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2644104260 0x7f264410f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:47.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.686+0000 7f2648c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2644104260 0x7f264410f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42126/0 (socket says 172.21.15.83:42126) 2024-04-24T09:16:47.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.686+0000 7f2648c00640 1 -- 172.21.15.83:0/1347070816 learned_addr learned my addr 172.21.15.83:0/1347070816 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:16:47.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.686+0000 7f2643e00640 1 --2- 172.21.15.83:0/1347070816 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2644109330 0x7f264410f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:47.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.687+0000 7f2649600640 1 --2- 172.21.15.83:0/1347070816 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f264410fe00 0x7f26441aaba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:47.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.687+0000 7f2648c00640 1 -- 172.21.15.83:0/1347070816 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f264410fe00 msgr2=0x7f26441aaba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:47.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.687+0000 7f2648c00640 1 --2- 172.21.15.83:0/1347070816 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f264410fe00 0x7f26441aaba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:47.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.687+0000 7f2648c00640 1 -- 172.21.15.83:0/1347070816 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2644109330 msgr2=0x7f264410f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:47.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.687+0000 7f2648c00640 1 --2- 172.21.15.83:0/1347070816 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2644109330 0x7f264410f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:47.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.687+0000 7f2648c00640 1 -- 172.21.15.83:0/1347070816 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26441ab320 con 0x7f2644104260 2024-04-24T09:16:47.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.687+0000 7f2648c00640 1 --2- 172.21.15.83:0/1347070816 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2644104260 0x7f264410f380 secure :-1 s=READY pgs=2795 cs=0 l=1 rev1=1 crypto rx=0x7f2634031040 tx=0x7f2634032ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:47.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.688+0000 7f2641600640 1 -- 172.21.15.83:0/1347070816 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2634003c70 con 0x7f2644104260 2024-04-24T09:16:47.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.688+0000 7f2641600640 1 -- 172.21.15.83:0/1347070816 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2634003e10 con 0x7f2644104260 2024-04-24T09:16:47.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.688+0000 7f2641600640 1 -- 172.21.15.83:0/1347070816 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2634030b30 con 0x7f2644104260 2024-04-24T09:16:47.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.688+0000 7f264a000640 1 -- 172.21.15.83:0/1347070816 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26441ab5b0 con 0x7f2644104260 2024-04-24T09:16:47.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.688+0000 7f264a000640 1 -- 172.21.15.83:0/1347070816 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26441abab0 con 0x7f2644104260 2024-04-24T09:16:47.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.690+0000 7f264a000640 1 -- 172.21.15.83:0/1347070816 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2644069190 con 0x7f2644104260 2024-04-24T09:16:47.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.693+0000 7f2641600640 1 -- 172.21.15.83:0/1347070816 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2634030cd0 con 0x7f2644104260 2024-04-24T09:16:47.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.697+0000 7f2641600640 1 --2- 172.21.15.83:0/1347070816 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f26280a3da0 0x7f26280a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:47.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.697+0000 7f2641600640 1 -- 172.21.15.83:0/1347070816 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 9583+0+0 (secure 0 0 0) 0x7f26340f2020 con 0x7f2644104260 2024-04-24T09:16:47.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.698+0000 7f2643e00640 1 --2- 172.21.15.83:0/1347070816 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f26280a3da0 0x7f26280a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:47.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.700+0000 7f2643e00640 1 --2- 172.21.15.83:0/1347070816 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f26280a3da0 0x7f26280a6260 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f2644110e10 tx=0x7f263000b390 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:47.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:47.704+0000 7f2641600640 1 -- 172.21.15.83:0/1347070816 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f26340be440 con 0x7f2644104260 2024-04-24T09:16:48.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.135+0000 7f264a000640 1 -- 172.21.15.83:0/1347070816 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2644104c10 con 0x7f2644104260 2024-04-24T09:16:48.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.136+0000 7f2641600640 1 -- 172.21.15.83:0/1347070816 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) ==== 94+0+11947 (secure 0 0 0) 0x7f26340bdd40 con 0x7f2644104260 2024-04-24T09:16:48.137 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T09:16:48.137 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":69,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/358872257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":358872257},{"type":"v1","addr":"172.21.15.83:6835","nonce":358872257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3157070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3157070851},{"type":"v1","addr":"172.21.15.83:6837","nonce":3157070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":69,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T08:48:29.422576+0000","modified":"2024-04-24T09:15:17.129816+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34520,"mds_1":34493,"mds_2":34451,"mds_3":34427,"mds_4":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14658":{"gid":14658,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6839/4085470844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":4085470844},{"type":"v1","addr":"172.21.15.83:6839","nonce":4085470844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14676":{"gid":14676,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6841/2634754210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":2634754210},{"type":"v1","addr":"172.21.15.83:6841","nonce":2634754210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24382":{"gid":24382,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/1859921310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1859921310},{"type":"v1","addr":"172.21.15.189:6833","nonce":1859921310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/2241923197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2241923197},{"type":"v1","addr":"172.21.15.189:6835","nonce":2241923197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.189:6839/2548965336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":2548965336},{"type":"v1","addr":"172.21.15.189:6839","nonce":2548965336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26686":{"gid":26686,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6837/1678817161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":1678817161},{"type":"v1","addr":"172.21.15.189:6837","nonce":1678817161}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.138:6833/1157967658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1157967658},{"type":"v1","addr":"172.21.15.138:6833","nonce":1157967658}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.138:6835/1909219758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1909219758},{"type":"v1","addr":"172.21.15.138:6835","nonce":1909219758}]},"join_fscid":-1,"export_targets":[0,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34493":{"gid":34493,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.138:6837/900566754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":900566754},{"type":"v1","addr":"172.21.15.138:6837","nonce":900566754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34520":{"gid":34520,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.138:6839/2236932393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2236932393},{"type":"v1","addr":"172.21.15.138:6839","nonce":2236932393}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34520,"qdb_cluster":[24454,34520,34427,34451,34493]},"id":1}]} 2024-04-24T09:16:48.137 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 69 2024-04-24T09:16:48.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.140+0000 7f264a000640 1 -- 172.21.15.83:0/1347070816 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f26280a3da0 msgr2=0x7f26280a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:48.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.140+0000 7f264a000640 1 --2- 172.21.15.83:0/1347070816 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f26280a3da0 0x7f26280a6260 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f2644110e10 tx=0x7f263000b390 comp rx=0 tx=0).stop 2024-04-24T09:16:48.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.140+0000 7f264a000640 1 -- 172.21.15.83:0/1347070816 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2644104260 msgr2=0x7f264410f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:48.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.140+0000 7f264a000640 1 --2- 172.21.15.83:0/1347070816 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2644104260 0x7f264410f380 secure :-1 s=READY pgs=2795 cs=0 l=1 rev1=1 crypto rx=0x7f2634031040 tx=0x7f2634032ff0 comp rx=0 tx=0).stop 2024-04-24T09:16:48.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.140+0000 7f264a000640 1 -- 172.21.15.83:0/1347070816 shutdown_connections 2024-04-24T09:16:48.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.140+0000 7f264a000640 1 --2- 172.21.15.83:0/1347070816 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f26280a3da0 0x7f26280a6260 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:48.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.140+0000 7f264a000640 1 --2- 172.21.15.83:0/1347070816 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f264410fe00 0x7f26441aaba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:48.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.140+0000 7f264a000640 1 --2- 172.21.15.83:0/1347070816 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2644109330 0x7f264410f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:48.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.140+0000 7f264a000640 1 --2- 172.21.15.83:0/1347070816 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2644104260 0x7f264410f380 unknown :-1 s=CLOSED pgs=2795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:48.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.140+0000 7f264a000640 1 -- 172.21.15.83:0/1347070816 >> 172.21.15.83:0/1347070816 conn(0x7f26440ffaa0 msgr2=0x7f2644100e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:48.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.140+0000 7f264a000640 1 -- 172.21.15.83:0/1347070816 shutdown_connections 2024-04-24T09:16:48.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.140+0000 7f264a000640 1 -- 172.21.15.83:0/1347070816 wait complete. 2024-04-24T09:16:48.168 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-24T09:16:48.224 INFO:tasks.workunit.client.0.smithi083.stdout: 10 82982 6.50 MB/sec execute 559 sec latency 98.056 ms 2024-04-24T09:16:48.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:16:48 smithi138 ceph-mon[31441]: pgmap v796: 129 pgs: 129 active+clean; 1.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.7 MiB/s wr, 209 op/s 2024-04-24T09:16:48.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:16:48 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:16:48.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:16:48 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/1347070816' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:16:48.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:16:48 smithi189 ceph-mon[31791]: pgmap v796: 129 pgs: 129 active+clean; 1.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.7 MiB/s wr, 209 op/s 2024-04-24T09:16:48.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:16:48 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:16:48.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:16:48 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/1347070816' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:16:48.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.481+0000 7f87d7200640 1 -- 172.21.15.83:0/3211933809 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87d8112eb0 msgr2=0x7f87d81152a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:48.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.481+0000 7f87d7200640 1 --2- 172.21.15.83:0/3211933809 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87d8112eb0 0x7f87d81152a0 secure :-1 s=READY pgs=2796 cs=0 l=1 rev1=1 crypto rx=0x7f87c8009900 tx=0x7f87c802a490 comp rx=0 tx=0).stop 2024-04-24T09:16:48.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.482+0000 7f87d7200640 1 -- 172.21.15.83:0/3211933809 shutdown_connections 2024-04-24T09:16:48.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.482+0000 7f87d7200640 1 --2- 172.21.15.83:0/3211933809 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87d8112eb0 0x7f87d81152a0 unknown :-1 s=CLOSED pgs=2796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:48.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.482+0000 7f87d7200640 1 --2- 172.21.15.83:0/3211933809 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f87d8074cc0 0x7f87d8112970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:48.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.482+0000 7f87d7200640 1 --2- 172.21.15.83:0/3211933809 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87d80743a0 0x7f87d8074780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:48.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.482+0000 7f87d7200640 1 -- 172.21.15.83:0/3211933809 >> 172.21.15.83:0/3211933809 conn(0x7f87d80fd9d0 msgr2=0x7f87d80ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:48.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.483+0000 7f87d7200640 1 -- 172.21.15.83:0/3211933809 shutdown_connections 2024-04-24T09:16:48.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.483+0000 7f87d7200640 1 -- 172.21.15.83:0/3211933809 wait complete. 2024-04-24T09:16:48.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.484+0000 7f87d7200640 1 Processor -- start 2024-04-24T09:16:48.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.485+0000 7f87d7200640 1 -- start start 2024-04-24T09:16:48.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.487+0000 7f87d7200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87d80743a0 0x7f87d81a48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:48.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.488+0000 7f87d7200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87d8074cc0 0x7f87d81a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:48.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.488+0000 7f87d5e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87d80743a0 0x7f87d81a48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:48.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.488+0000 7f87d5e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87d80743a0 0x7f87d81a48f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42152/0 (socket says 172.21.15.83:42152) 2024-04-24T09:16:48.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.488+0000 7f87d7200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f87d8112eb0 0x7f87d819eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:48.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.488+0000 7f87d5e00640 1 -- 172.21.15.83:0/2854710283 learned_addr learned my addr 172.21.15.83:0/2854710283 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:16:48.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.488+0000 7f87d7200640 1 -- 172.21.15.83:0/2854710283 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f87d8118750 con 0x7f87d80743a0 2024-04-24T09:16:48.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.488+0000 7f87d7200640 1 -- 172.21.15.83:0/2854710283 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f87d81185d0 con 0x7f87d8074cc0 2024-04-24T09:16:48.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.488+0000 7f87d7200640 1 -- 172.21.15.83:0/2854710283 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f87d81188d0 con 0x7f87d8112eb0 2024-04-24T09:16:48.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.488+0000 7f87d5400640 1 --2- 172.21.15.83:0/2854710283 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87d8074cc0 0x7f87d81a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:48.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.488+0000 7f87d6800640 1 --2- 172.21.15.83:0/2854710283 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f87d8112eb0 0x7f87d819eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:48.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.489+0000 7f87d6800640 1 -- 172.21.15.83:0/2854710283 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87d8074cc0 msgr2=0x7f87d81a4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:48.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.489+0000 7f87d6800640 1 --2- 172.21.15.83:0/2854710283 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87d8074cc0 0x7f87d81a4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:48.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.489+0000 7f87d6800640 1 -- 172.21.15.83:0/2854710283 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87d80743a0 msgr2=0x7f87d81a48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:48.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.489+0000 7f87d6800640 1 --2- 172.21.15.83:0/2854710283 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87d80743a0 0x7f87d81a48f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:48.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.489+0000 7f87d6800640 1 -- 172.21.15.83:0/2854710283 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87d819f270 con 0x7f87d8112eb0 2024-04-24T09:16:48.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.489+0000 7f87d5e00640 1 --2- 172.21.15.83:0/2854710283 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87d80743a0 0x7f87d81a48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:16:48.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.490+0000 7f87d6800640 1 --2- 172.21.15.83:0/2854710283 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f87d8112eb0 0x7f87d819eaf0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f87c8009230 tx=0x7f87c8003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:48.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.490+0000 7f87cea00640 1 -- 172.21.15.83:0/2854710283 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c803f070 con 0x7f87d8112eb0 2024-04-24T09:16:48.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.490+0000 7f87d7200640 1 -- 172.21.15.83:0/2854710283 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87d819f500 con 0x7f87d8112eb0 2024-04-24T09:16:48.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.490+0000 7f87d7200640 1 -- 172.21.15.83:0/2854710283 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87d81ab7b0 con 0x7f87d8112eb0 2024-04-24T09:16:48.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.492+0000 7f87cea00640 1 -- 172.21.15.83:0/2854710283 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f87c8003e40 con 0x7f87d8112eb0 2024-04-24T09:16:48.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.493+0000 7f87cea00640 1 -- 172.21.15.83:0/2854710283 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c8035030 con 0x7f87d8112eb0 2024-04-24T09:16:48.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.497+0000 7f87e001d740 1 Processor -- start 2024-04-24T09:16:48.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.498+0000 7f87cea00640 1 -- 172.21.15.83:0/2854710283 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f87c8042070 con 0x7f87d8112eb0 2024-04-24T09:16:48.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.499+0000 7f87e001d740 1 -- start start 2024-04-24T09:16:48.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.499+0000 7f87e001d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555edf978b40 0x555edf978f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:48.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.499+0000 7f87e001d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555edf979460 0x555edf9835f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:48.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.499+0000 7f87e001d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555edf983b30 0x555edf985f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:48.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.499+0000 7f87e001d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x555edf8de1d0 con 0x555edf978b40 2024-04-24T09:16:48.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.499+0000 7f87e001d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555edf884450 con 0x555edf983b30 2024-04-24T09:16:48.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.499+0000 7f87e001d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x555edf8da190 con 0x555edf979460 2024-04-24T09:16:48.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.500+0000 7f87d5e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555edf979460 0x555edf9835f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:48.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.500+0000 7f87d5e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555edf979460 0x555edf9835f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:58866/0 (socket says 172.21.15.83:58866) 2024-04-24T09:16:48.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.500+0000 7f87d5e00640 1 -- 172.21.15.83:0/1601570136 learned_addr learned my addr 172.21.15.83:0/1601570136 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:16:48.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.501+0000 7f87d5e00640 1 -- 172.21.15.83:0/1601570136 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555edf983b30 msgr2=0x555edf985f20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T09:16:48.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.501+0000 7f87d5e00640 1 --2- 172.21.15.83:0/1601570136 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555edf983b30 0x555edf985f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:48.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.501+0000 7f87d5e00640 1 -- 172.21.15.83:0/1601570136 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555edf978b40 msgr2=0x555edf978f20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T09:16:48.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.501+0000 7f87cea00640 1 --2- 172.21.15.83:0/2854710283 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f87bc0a3d20 0x7f87bc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:48.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.501+0000 7f87cea00640 1 -- 172.21.15.83:0/2854710283 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 9583+0+0 (secure 0 0 0) 0x7f87c80f0fc0 con 0x7f87d8112eb0 2024-04-24T09:16:48.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.502+0000 7f87d5400640 1 --2- 172.21.15.83:0/1601570136 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555edf978b40 0x555edf978f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:48.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.502+0000 7f87d5e00640 1 --2- 172.21.15.83:0/1601570136 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555edf978b40 0x555edf978f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:48.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.502+0000 7f87d5e00640 1 -- 172.21.15.83:0/1601570136 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555edf941310 con 0x555edf979460 2024-04-24T09:16:48.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.502+0000 7f87d5400640 1 --2- 172.21.15.83:0/1601570136 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555edf978b40 0x555edf978f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:16:48.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.502+0000 7f87d5e00640 1 --2- 172.21.15.83:0/1601570136 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555edf979460 0x555edf9835f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f87c40115f0 tx=0x7f87c4011ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2c0dbcf7c3e10bd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:48.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.502+0000 7f87d5400640 1 --2- 172.21.15.83:0/2854710283 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f87bc0a3d20 0x7f87bc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:48.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.502+0000 7f87c3400640 1 -- 172.21.15.83:0/1601570136 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c4004020 con 0x555edf979460 2024-04-24T09:16:48.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.503+0000 7f87c3400640 1 -- 172.21.15.83:0/1601570136 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f87c40054d0 con 0x555edf979460 2024-04-24T09:16:48.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.503+0000 7f87c3400640 1 -- 172.21.15.83:0/1601570136 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c4004e00 con 0x555edf979460 2024-04-24T09:16:48.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.504+0000 7f87e001d740 1 -- 172.21.15.83:0/1601570136 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555edf979460 msgr2=0x555edf9835f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:48.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.504+0000 7f87e001d740 1 --2- 172.21.15.83:0/1601570136 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555edf979460 0x555edf9835f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f87c40115f0 tx=0x7f87c4011ab0 comp rx=0 tx=0).stop 2024-04-24T09:16:48.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.504+0000 7f87d5400640 1 --2- 172.21.15.83:0/2854710283 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f87bc0a3d20 0x7f87bc0a61e0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f87d0009a00 tx=0x7f87d0004ab0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:48.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.504+0000 7f87e001d740 1 -- 172.21.15.83:0/1601570136 shutdown_connections 2024-04-24T09:16:48.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.504+0000 7f87e001d740 1 --2- 172.21.15.83:0/1601570136 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555edf983b30 0x555edf985f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:48.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.504+0000 7f87e001d740 1 --2- 172.21.15.83:0/1601570136 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555edf979460 0x555edf9835f0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:48.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.504+0000 7f87e001d740 1 --2- 172.21.15.83:0/1601570136 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555edf978b40 0x555edf978f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:48.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.504+0000 7f87e001d740 1 -- 172.21.15.83:0/1601570136 >> 172.21.15.83:0/1601570136 conn(0x555edf974e60 msgr2=0x555edf975240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:48.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.504+0000 7f87e001d740 1 -- 172.21.15.83:0/1601570136 shutdown_connections 2024-04-24T09:16:48.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.504+0000 7f87e001d740 1 -- 172.21.15.83:0/1601570136 wait complete. 2024-04-24T09:16:48.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.505+0000 7f87e001d740 1 -- start start 2024-04-24T09:16:48.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.505+0000 7f87e001d740 1 Processor -- start 2024-04-24T09:16:48.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.506+0000 7f87e001d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555edf978b40 0x555edf9835d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:48.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.506+0000 7f87e001d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555edf979460 0x555edf9808c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:48.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.506+0000 7f87e001d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555edf983b30 0x555edf980fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:48.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.506+0000 7f87e001d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x555edf9869c0 con 0x555edf979460 2024-04-24T09:16:48.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.506+0000 7f87e001d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555edf897810 con 0x555edf978b40 2024-04-24T09:16:48.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.506+0000 7f87e001d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x555edf986b40 con 0x555edf983b30 2024-04-24T09:16:48.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.506+0000 7f87e001d740 10 client.0 _ll_register_callbacks cb 0x555edf865140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:16:48.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.506+0000 7f87d5e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555edf978b40 0x555edf9835d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:48.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.506+0000 7f87d5400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555edf979460 0x555edf9808c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:48.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.506+0000 7f87d5400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555edf979460 0x555edf9808c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42172/0 (socket says 172.21.15.83:42172) 2024-04-24T09:16:48.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.506+0000 7f87d5400640 1 -- 172.21.15.83:0/228566678 learned_addr learned my addr 172.21.15.83:0/228566678 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:16:48.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.506+0000 7f87d6800640 1 --2- 172.21.15.83:0/228566678 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555edf983b30 0x555edf980fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:48.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.507+0000 7f87d5400640 1 -- 172.21.15.83:0/228566678 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555edf983b30 msgr2=0x555edf980fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:48.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.507+0000 7f87d5400640 1 --2- 172.21.15.83:0/228566678 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555edf983b30 0x555edf980fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:48.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.507+0000 7f87d5400640 1 -- 172.21.15.83:0/228566678 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555edf978b40 msgr2=0x555edf9835d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:48.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.507+0000 7f87d5400640 1 --2- 172.21.15.83:0/228566678 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555edf978b40 0x555edf9835d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:48.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.507+0000 7f87d5400640 1 -- 172.21.15.83:0/228566678 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555edf8a83f0 con 0x555edf979460 2024-04-24T09:16:48.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.507+0000 7f87d5400640 1 --2- 172.21.15.83:0/228566678 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555edf979460 0x555edf9808c0 secure :-1 s=READY pgs=2797 cs=0 l=1 rev1=1 crypto rx=0x7f87d0010820 tx=0x7f87d0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67608d0c4a179bc0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:48.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.507+0000 7f87b7e00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:16:48.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.508+0000 7f87b7e00640 1 -- 172.21.15.83:0/228566678 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d002c060 con 0x555edf979460 2024-04-24T09:16:48.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.508+0000 7f87b7e00640 1 -- 172.21.15.83:0/228566678 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(132..132 src has 1..132) ==== 9583+0+0 (secure 0 0 0) 0x7f87d0019070 con 0x555edf979460 2024-04-24T09:16:48.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.508+0000 7f87e001d740 1 -- 172.21.15.83:0/228566678 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87c40054d0 con 0x555edf979460 2024-04-24T09:16:48.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.508+0000 7f87b7e00640 1 -- 172.21.15.83:0/228566678 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f87d0010e60 con 0x555edf979460 2024-04-24T09:16:48.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.508+0000 7f87e001d740 10 client.24033 fetch_fsmap learned FSMap version 69 2024-04-24T09:16:48.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.508+0000 7f87e001d740 1 -- 172.21.15.83:0/228566678 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x555edf5f03f0 con 0x555edf979460 2024-04-24T09:16:48.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.509+0000 7f87b7e00640 1 -- 172.21.15.83:0/228566678 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f87c40054d0 con 0x555edf979460 2024-04-24T09:16:48.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.509+0000 7f87b7e00640 1 -- 172.21.15.83:0/228566678 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 69) ==== 8370+0+0 (secure 0 0 0) 0x7f87d001c420 con 0x555edf979460 2024-04-24T09:16:48.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.509+0000 7f87e001d740 10 client.24033 fetch_fsmap finished waiting for FSMap version 69 2024-04-24T09:16:48.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.509+0000 7f87e001d740 10 client.24033 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-24T09:16:48.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.509+0000 7f87e001d740 20 client.24033 populate_metadata read hostname 'smithi083' 2024-04-24T09:16:48.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.509+0000 7f87e001d740 1 --2- 172.21.15.83:0/228566678 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x555edf993d20 0x555edf9961c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:48.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.509+0000 7f87e001d740 4 client.24033 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:16:48.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.509+0000 7f87e001d740 1 -- 172.21.15.83:0/228566678 --> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555edf8ebbf0 con 0x555edf993d20 2024-04-24T09:16:48.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.510+0000 7f87d5e00640 1 --2- 172.21.15.83:0/228566678 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x555edf993d20 0x555edf9961c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:48.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.511+0000 7f87d5e00640 1 --2- 172.21.15.83:0/228566678 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x555edf993d20 0x555edf9961c0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f17e50fe71f96eea server_cookie=d1edfe2577dc0f2a in_seq=0 out_seq=0 2024-04-24T09:16:48.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.511+0000 7f87b7e00640 10 client.24033 ms_handle_connect on v2:172.21.15.189:6838/2548965336 2024-04-24T09:16:48.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.512+0000 7f87b7e00640 1 -- 172.21.15.83:0/228566678 <== mds.4 v2:172.21.15.189:6838/2548965336 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555edf8ebbf0 con 0x555edf993d20 2024-04-24T09:16:48.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.512+0000 7f87b7e00640 10 client.24033 handle_command_reply: tid=0 2024-04-24T09:16:48.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.512+0000 7f87e001d740 1 client.24033 shutdown 2024-04-24T09:16:48.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.513+0000 7f87e001d740 1 -- 172.21.15.83:0/228566678 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555edf979460 msgr2=0x555edf9808c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:48.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.513+0000 7f87e001d740 1 --2- 172.21.15.83:0/228566678 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555edf979460 0x555edf9808c0 secure :-1 s=READY pgs=2797 cs=0 l=1 rev1=1 crypto rx=0x7f87d0010820 tx=0x7f87d0004a40 comp rx=0 tx=0).stop 2024-04-24T09:16:48.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.513+0000 7f87e001d740 1 -- 172.21.15.83:0/228566678 shutdown_connections 2024-04-24T09:16:48.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.513+0000 7f87e001d740 1 --2- 172.21.15.83:0/228566678 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x555edf993d20 0x555edf9961c0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:48.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.513+0000 7f87e001d740 1 --2- 172.21.15.83:0/228566678 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555edf983b30 0x555edf980fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:48.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.513+0000 7f87e001d740 1 --2- 172.21.15.83:0/228566678 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555edf979460 0x555edf9808c0 unknown :-1 s=CLOSED pgs=2797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:48.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.513+0000 7f87e001d740 1 --2- 172.21.15.83:0/228566678 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555edf978b40 0x555edf9835d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:48.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.513+0000 7f87e001d740 1 -- 172.21.15.83:0/228566678 >> 172.21.15.83:0/228566678 conn(0x555edf974e60 msgr2=0x555edf97ab20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:48.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.513+0000 7f87b7e00640 0 client.24033 ms_handle_reset on v2:172.21.15.189:6838/2548965336 2024-04-24T09:16:48.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.513+0000 7f87e001d740 1 -- 172.21.15.83:0/228566678 shutdown_connections 2024-04-24T09:16:48.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.513+0000 7f87e001d740 1 -- 172.21.15.83:0/228566678 wait complete. 2024-04-24T09:16:48.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.513+0000 7f87e001d740 20 client.24033 trim_cache size 0 max 16384 2024-04-24T09:16:48.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.541+0000 7f87e001d740 1 Processor -- start 2024-04-24T09:16:48.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.541+0000 7f87e001d740 1 -- start start 2024-04-24T09:16:48.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.541+0000 7f87e001d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555edf978b40 0x555edf98f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:48.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.542+0000 7f87e001d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555edf979460 0x555edf983100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:48.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.542+0000 7f87e001d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555edf983b30 0x555edf99ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:48.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.542+0000 7f87e001d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f87b8010100 con 0x555edf978b40 2024-04-24T09:16:48.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.542+0000 7f87e001d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f87b8011ca0 con 0x555edf983b30 2024-04-24T09:16:48.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.542+0000 7f87e001d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f87b80107d0 con 0x555edf979460 2024-04-24T09:16:48.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.542+0000 7f87d5400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555edf979460 0x555edf983100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:48.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.542+0000 7f87d5400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555edf979460 0x555edf983100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:58894/0 (socket says 172.21.15.83:58894) 2024-04-24T09:16:48.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.542+0000 7f87d5400640 1 -- 172.21.15.83:0/1851190543 learned_addr learned my addr 172.21.15.83:0/1851190543 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:16:48.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.542+0000 7f87d6800640 1 --2- 172.21.15.83:0/1851190543 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555edf983b30 0x555edf99ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:48.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.542+0000 7f87d5e00640 1 --2- 172.21.15.83:0/1851190543 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555edf978b40 0x555edf98f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:48.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.543+0000 7f87d5400640 1 -- 172.21.15.83:0/1851190543 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555edf983b30 msgr2=0x555edf99ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:48.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.543+0000 7f87d5400640 1 --2- 172.21.15.83:0/1851190543 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555edf983b30 0x555edf99ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:48.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.543+0000 7f87d5400640 1 -- 172.21.15.83:0/1851190543 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555edf978b40 msgr2=0x555edf98f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:48.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.543+0000 7f87d5400640 1 --2- 172.21.15.83:0/1851190543 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555edf978b40 0x555edf98f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:48.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.543+0000 7f87d5400640 1 -- 172.21.15.83:0/1851190543 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555edf6ba7d0 con 0x555edf979460 2024-04-24T09:16:48.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.543+0000 7f87d5400640 1 --2- 172.21.15.83:0/1851190543 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555edf979460 0x555edf983100 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f87d001c9e0 tx=0x7f87d0009500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=768634646f9c545f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:48.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.543+0000 7f87c2a00640 1 -- 172.21.15.83:0/1851190543 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d0036080 con 0x555edf979460 2024-04-24T09:16:48.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.543+0000 7f87c2a00640 1 -- 172.21.15.83:0/1851190543 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f87d00134b0 con 0x555edf979460 2024-04-24T09:16:48.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.543+0000 7f87c2a00640 1 -- 172.21.15.83:0/1851190543 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d001ccc0 con 0x555edf979460 2024-04-24T09:16:48.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.544+0000 7f87e001d740 1 -- 172.21.15.83:0/1851190543 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555edf979460 msgr2=0x555edf983100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:48.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.544+0000 7f87e001d740 1 --2- 172.21.15.83:0/1851190543 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555edf979460 0x555edf983100 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f87d001c9e0 tx=0x7f87d0009500 comp rx=0 tx=0).stop 2024-04-24T09:16:48.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.544+0000 7f87e001d740 1 -- 172.21.15.83:0/1851190543 shutdown_connections 2024-04-24T09:16:48.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.544+0000 7f87e001d740 1 --2- 172.21.15.83:0/1851190543 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555edf983b30 0x555edf99ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:48.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.544+0000 7f87e001d740 1 --2- 172.21.15.83:0/1851190543 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555edf979460 0x555edf983100 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:48.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.544+0000 7f87e001d740 1 --2- 172.21.15.83:0/1851190543 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555edf978b40 0x555edf98f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:48.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.544+0000 7f87e001d740 1 -- 172.21.15.83:0/1851190543 >> 172.21.15.83:0/1851190543 conn(0x555edf974e60 msgr2=0x555edf97bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:48.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.545+0000 7f87e001d740 1 -- 172.21.15.83:0/1851190543 shutdown_connections 2024-04-24T09:16:48.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.545+0000 7f87e001d740 1 -- 172.21.15.83:0/1851190543 wait complete. 2024-04-24T09:16:48.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.545+0000 7f87e001d740 1 -- start start 2024-04-24T09:16:48.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.545+0000 7f87e001d740 1 Processor -- start 2024-04-24T09:16:48.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.546+0000 7f87e001d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555edf978b40 0x555edf984c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:48.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.546+0000 7f87e001d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555edf979460 0x555edf985170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:48.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.546+0000 7f87e001d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555edf983b30 0x555edf98a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:48.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.546+0000 7f87e001d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f87b80115d0 con 0x555edf978b40 2024-04-24T09:16:48.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.546+0000 7f87e001d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f87b8010f00 con 0x555edf983b30 2024-04-24T09:16:48.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.546+0000 7f87e001d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f87b800e100 con 0x555edf979460 2024-04-24T09:16:48.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.546+0000 7f87e001d740 10 client.0 _ll_register_callbacks cb 0x7f87d0009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:16:48.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.546+0000 7f87d5400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555edf979460 0x555edf985170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:48.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.546+0000 7f87d5400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555edf979460 0x555edf985170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:58908/0 (socket says 172.21.15.83:58908) 2024-04-24T09:16:48.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.546+0000 7f87d5400640 1 -- 172.21.15.83:0/856150746 learned_addr learned my addr 172.21.15.83:0/856150746 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:16:48.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.546+0000 7f87d6800640 1 --2- 172.21.15.83:0/856150746 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555edf983b30 0x555edf98a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:48.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.547+0000 7f87d5e00640 1 --2- 172.21.15.83:0/856150746 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555edf978b40 0x555edf984c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:48.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.547+0000 7f87d5400640 1 -- 172.21.15.83:0/856150746 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555edf983b30 msgr2=0x555edf98a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:48.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.547+0000 7f87d5400640 1 --2- 172.21.15.83:0/856150746 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555edf983b30 0x555edf98a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:48.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.547+0000 7f87d5400640 1 -- 172.21.15.83:0/856150746 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555edf978b40 msgr2=0x555edf984c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:48.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.547+0000 7f87d5400640 1 --2- 172.21.15.83:0/856150746 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555edf978b40 0x555edf984c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:48.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.547+0000 7f87d5400640 1 -- 172.21.15.83:0/856150746 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555edf985a30 con 0x555edf979460 2024-04-24T09:16:48.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.547+0000 7f87d5e00640 1 --2- 172.21.15.83:0/856150746 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555edf978b40 0x555edf984c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:16:48.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.547+0000 7f87d6800640 1 --2- 172.21.15.83:0/856150746 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555edf983b30 0x555edf98a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:16:48.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.547+0000 7f87d5400640 1 --2- 172.21.15.83:0/856150746 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555edf979460 0x555edf985170 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f87d001c910 tx=0x7f87d0012e10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39d65acd686e2687 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:48.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.547+0000 7f87b7e00640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-04-24T09:16:48.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.547+0000 7f87b7e00640 1 -- 172.21.15.83:0/856150746 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d002c040 con 0x555edf979460 2024-04-24T09:16:48.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.548+0000 7f87b7e00640 1 -- 172.21.15.83:0/856150746 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(132..132 src has 1..132) ==== 9583+0+0 (secure 0 0 0) 0x7f87d0023030 con 0x555edf979460 2024-04-24T09:16:48.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.548+0000 7f87e001d740 1 -- 172.21.15.83:0/856150746 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87d00134b0 con 0x555edf979460 2024-04-24T09:16:48.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.548+0000 7f87b7e00640 1 -- 172.21.15.83:0/856150746 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f87d0046050 con 0x555edf979460 2024-04-24T09:16:48.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.548+0000 7f87e001d740 10 client.30793 fetch_fsmap learned FSMap version 69 2024-04-24T09:16:48.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.548+0000 7f87e001d740 1 -- 172.21.15.83:0/856150746 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x555edf9fa900 con 0x555edf979460 2024-04-24T09:16:48.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.548+0000 7f87b7e00640 1 -- 172.21.15.83:0/856150746 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f87d00134b0 con 0x555edf979460 2024-04-24T09:16:48.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.548+0000 7f87b7e00640 1 -- 172.21.15.83:0/856150746 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 69) ==== 8370+0+0 (secure 0 0 0) 0x7f87d0005600 con 0x555edf979460 2024-04-24T09:16:48.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.548+0000 7f87e001d740 10 client.30793 fetch_fsmap finished waiting for FSMap version 69 2024-04-24T09:16:48.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.548+0000 7f87e001d740 10 client.30793 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-24T09:16:48.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.548+0000 7f87e001d740 20 client.30793 populate_metadata read hostname 'smithi083' 2024-04-24T09:16:48.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.548+0000 7f87e001d740 1 --2- 172.21.15.83:0/856150746 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x555edf9faee0 0x555edf9fd300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:48.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.548+0000 7f87e001d740 4 client.30793 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:16:48.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.548+0000 7f87e001d740 1 -- 172.21.15.83:0/856150746 --> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555edf8e5a60 con 0x555edf9faee0 2024-04-24T09:16:48.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.549+0000 7f87d5e00640 1 --2- 172.21.15.83:0/856150746 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x555edf9faee0 0x555edf9fd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:48.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.549+0000 7f87b7e00640 10 client.30793 ms_handle_connect on v2:172.21.15.189:6838/2548965336 2024-04-24T09:16:48.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.549+0000 7f87d5e00640 1 --2- 172.21.15.83:0/856150746 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x555edf9faee0 0x555edf9fd300 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=72d6a1d806de0e7a server_cookie=18823c63ba8cfbb3 in_seq=0 out_seq=0 2024-04-24T09:16:48.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.550+0000 7f87b7e00640 1 -- 172.21.15.83:0/856150746 <== mds.4 v2:172.21.15.189:6838/2548965336 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555edf8e5a60 con 0x555edf9faee0 2024-04-24T09:16:48.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.550+0000 7f87b7e00640 10 client.30793 handle_command_reply: tid=0 2024-04-24T09:16:48.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.550+0000 7f87e001d740 1 client.30793 shutdown 2024-04-24T09:16:48.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.551+0000 7f87e001d740 1 -- 172.21.15.83:0/856150746 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555edf979460 msgr2=0x555edf985170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:48.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.551+0000 7f87e001d740 1 --2- 172.21.15.83:0/856150746 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555edf979460 0x555edf985170 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f87d001c910 tx=0x7f87d0012e10 comp rx=0 tx=0).stop 2024-04-24T09:16:48.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.551+0000 7f87e001d740 1 -- 172.21.15.83:0/856150746 shutdown_connections 2024-04-24T09:16:48.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.551+0000 7f87e001d740 1 --2- 172.21.15.83:0/856150746 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x555edf9faee0 0x555edf9fd300 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:48.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.551+0000 7f87e001d740 1 --2- 172.21.15.83:0/856150746 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555edf983b30 0x555edf98a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:48.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.551+0000 7f87e001d740 1 --2- 172.21.15.83:0/856150746 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555edf979460 0x555edf985170 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:48.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.551+0000 7f87e001d740 1 --2- 172.21.15.83:0/856150746 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555edf978b40 0x555edf984c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:48.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.551+0000 7f87e001d740 1 -- 172.21.15.83:0/856150746 >> 172.21.15.83:0/856150746 conn(0x555edf974e60 msgr2=0x555edf9736c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:48.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.551+0000 7f87b7e00640 0 client.30793 ms_handle_reset on v2:172.21.15.189:6838/2548965336 2024-04-24T09:16:48.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.551+0000 7f87e001d740 1 -- 172.21.15.83:0/856150746 shutdown_connections 2024-04-24T09:16:48.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.551+0000 7f87e001d740 1 -- 172.21.15.83:0/856150746 wait complete. 2024-04-24T09:16:48.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.551+0000 7f87e001d740 20 client.30793 trim_cache size 0 max 16384 2024-04-24T09:16:48.553 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:16:48.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:16:48 smithi083 ceph-mon[27907]: pgmap v796: 129 pgs: 129 active+clean; 1.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.7 MiB/s wr, 209 op/s 2024-04-24T09:16:48.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:16:48 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:16:48.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:16:48 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/1347070816' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:16:48.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.552+0000 7f87d7200640 1 -- 172.21.15.83:0/2854710283 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f87bc0a3d20 msgr2=0x7f87bc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:48.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.552+0000 7f87d7200640 1 --2- 172.21.15.83:0/2854710283 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f87bc0a3d20 0x7f87bc0a61e0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f87d0009a00 tx=0x7f87d0004ab0 comp rx=0 tx=0).stop 2024-04-24T09:16:48.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.552+0000 7f87d7200640 1 -- 172.21.15.83:0/2854710283 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f87d8112eb0 msgr2=0x7f87d819eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:48.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.552+0000 7f87d7200640 1 --2- 172.21.15.83:0/2854710283 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f87d8112eb0 0x7f87d819eaf0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f87c8009230 tx=0x7f87c8003b90 comp rx=0 tx=0).stop 2024-04-24T09:16:48.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.553+0000 7f87d7200640 1 -- 172.21.15.83:0/2854710283 shutdown_connections 2024-04-24T09:16:48.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.553+0000 7f87d7200640 1 --2- 172.21.15.83:0/2854710283 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f87bc0a3d20 0x7f87bc0a61e0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:48.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.553+0000 7f87d7200640 1 --2- 172.21.15.83:0/2854710283 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f87d8112eb0 0x7f87d819eaf0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:48.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.553+0000 7f87d7200640 1 --2- 172.21.15.83:0/2854710283 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87d8074cc0 0x7f87d81a4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:48.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.553+0000 7f87d7200640 1 --2- 172.21.15.83:0/2854710283 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87d80743a0 0x7f87d81a48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:48.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.553+0000 7f87d7200640 1 -- 172.21.15.83:0/2854710283 >> 172.21.15.83:0/2854710283 conn(0x7f87d80fd9d0 msgr2=0x7f87d80ff2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:48.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.554+0000 7f87d7200640 1 -- 172.21.15.83:0/2854710283 shutdown_connections 2024-04-24T09:16:48.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.554+0000 7f87d7200640 1 -- 172.21.15.83:0/2854710283 wait complete. 2024-04-24T09:16:48.578 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-24T09:16:48.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.836+0000 7f8a3b200640 1 -- 172.21.15.83:0/3895763093 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a3411b6a0 msgr2=0x7f8a3411da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:48.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.836+0000 7f8a3b200640 1 --2- 172.21.15.83:0/3895763093 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a3411b6a0 0x7f8a3411da90 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c009ca0 tx=0x7f8a2c02a940 comp rx=0 tx=0).stop 2024-04-24T09:16:48.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.836+0000 7f8a3b200640 1 -- 172.21.15.83:0/3895763093 shutdown_connections 2024-04-24T09:16:48.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.836+0000 7f8a3b200640 1 --2- 172.21.15.83:0/3895763093 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a3411b6a0 0x7f8a3411da90 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:48.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.836+0000 7f8a3b200640 1 --2- 172.21.15.83:0/3895763093 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a3406e2e0 0x7f8a3411b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:48.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.836+0000 7f8a3b200640 1 --2- 172.21.15.83:0/3895763093 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8a3406d9c0 0x7f8a3406dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:48.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.836+0000 7f8a3b200640 1 -- 172.21.15.83:0/3895763093 >> 172.21.15.83:0/3895763093 conn(0x7f8a3406d030 msgr2=0x7f8a3410b010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:48.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.837+0000 7f8a3b200640 1 -- 172.21.15.83:0/3895763093 shutdown_connections 2024-04-24T09:16:48.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.837+0000 7f8a3b200640 1 -- 172.21.15.83:0/3895763093 wait complete. 2024-04-24T09:16:48.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.838+0000 7f8a3b200640 1 Processor -- start 2024-04-24T09:16:48.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.838+0000 7f8a3b200640 1 -- start start 2024-04-24T09:16:48.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.839+0000 7f8a3b200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8a3406d9c0 0x7f8a341a75d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:48.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.839+0000 7f8a3b200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a3406e2e0 0x7f8a341a7b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:48.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.839+0000 7f8a3b200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a3411b6a0 0x7f8a341abe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:48.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.839+0000 7f8a3b200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8a34121010 con 0x7f8a3406e2e0 2024-04-24T09:16:48.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.839+0000 7f8a3b200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8a34120e90 con 0x7f8a3411b6a0 2024-04-24T09:16:48.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.839+0000 7f8a3b200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8a34121190 con 0x7f8a3406d9c0 2024-04-24T09:16:48.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.839+0000 7f8a33e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8a3406d9c0 0x7f8a341a75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:48.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.839+0000 7f8a33e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8a3406d9c0 0x7f8a341a75d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:41222/0 (socket says 172.21.15.83:41222) 2024-04-24T09:16:48.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.839+0000 7f8a33e00640 1 -- 172.21.15.83:0/153751168 learned_addr learned my addr 172.21.15.83:0/153751168 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:16:48.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.839+0000 7f8a33400640 1 --2- 172.21.15.83:0/153751168 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a3406e2e0 0x7f8a341a7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:48.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.839+0000 7f8a39200640 1 --2- 172.21.15.83:0/153751168 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a3411b6a0 0x7f8a341abe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:48.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.840+0000 7f8a33400640 1 -- 172.21.15.83:0/153751168 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8a3406d9c0 msgr2=0x7f8a341a75d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:48.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.840+0000 7f8a33400640 1 --2- 172.21.15.83:0/153751168 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8a3406d9c0 0x7f8a341a75d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:48.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.840+0000 7f8a33400640 1 -- 172.21.15.83:0/153751168 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a3411b6a0 msgr2=0x7f8a341abe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:48.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.840+0000 7f8a33400640 1 --2- 172.21.15.83:0/153751168 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a3411b6a0 0x7f8a341abe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:48.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.840+0000 7f8a33400640 1 -- 172.21.15.83:0/153751168 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a341ac5c0 con 0x7f8a3406e2e0 2024-04-24T09:16:48.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.840+0000 7f8a33e00640 1 --2- 172.21.15.83:0/153751168 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8a3406d9c0 0x7f8a341a75d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:16:48.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.840+0000 7f8a33400640 1 --2- 172.21.15.83:0/153751168 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a3406e2e0 0x7f8a341a7b10 secure :-1 s=READY pgs=2798 cs=0 l=1 rev1=1 crypto rx=0x7f8a2000e9f0 tx=0x7f8a2000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:48.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.841+0000 7f8a30c00640 1 -- 172.21.15.83:0/153751168 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a2000cd80 con 0x7f8a3406e2e0 2024-04-24T09:16:48.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.841+0000 7f8a3b200640 1 -- 172.21.15.83:0/153751168 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a341ac8b0 con 0x7f8a3406e2e0 2024-04-24T09:16:48.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.841+0000 7f8a30c00640 1 -- 172.21.15.83:0/153751168 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8a200035f0 con 0x7f8a3406e2e0 2024-04-24T09:16:48.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.841+0000 7f8a30c00640 1 -- 172.21.15.83:0/153751168 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a20010690 con 0x7f8a3406e2e0 2024-04-24T09:16:48.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.841+0000 7f8a3b200640 1 -- 172.21.15.83:0/153751168 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a341b4140 con 0x7f8a3406e2e0 2024-04-24T09:16:48.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.848+0000 7f8a30c00640 1 -- 172.21.15.83:0/153751168 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8a200108f0 con 0x7f8a3406e2e0 2024-04-24T09:16:48.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.848+0000 7f8a3e1d6740 1 Processor -- start 2024-04-24T09:16:48.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.849+0000 7f8a3e1d6740 1 -- start start 2024-04-24T09:16:48.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.851+0000 7f8a3e1d6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bf72882950 0x55bf72882d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:48.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.851+0000 7f8a3e1d6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bf72883270 0x55bf7288d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:48.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.851+0000 7f8a3e1d6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bf7288d940 0x55bf7288fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:48.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.851+0000 7f8a3e1d6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55bf727e81d0 con 0x55bf72882950 2024-04-24T09:16:48.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.851+0000 7f8a3e1d6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55bf7278e450 con 0x55bf72883270 2024-04-24T09:16:48.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.851+0000 7f8a3e1d6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55bf727e4190 con 0x55bf7288d940 2024-04-24T09:16:48.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.852+0000 7f8a39200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bf72882950 0x55bf72882d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:48.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.853+0000 7f8a33e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bf72883270 0x55bf7288d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:48.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.853+0000 7f8a33e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bf72883270 0x55bf7288d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:43112/0 (socket says 172.21.15.83:43112) 2024-04-24T09:16:48.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.853+0000 7f8a33e00640 1 -- 172.21.15.83:0/2992460089 learned_addr learned my addr 172.21.15.83:0/2992460089 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:16:48.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.853+0000 7f8a30c00640 1 --2- 172.21.15.83:0/153751168 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f8a180a3da0 0x7f8a180a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:48.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.853+0000 7f8a33e00640 1 -- 172.21.15.83:0/2992460089 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bf7288d940 msgr2=0x55bf7288fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:48.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.853+0000 7f8a33e00640 1 --2- 172.21.15.83:0/2992460089 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bf7288d940 0x55bf7288fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:48.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.853+0000 7f8a33e00640 1 -- 172.21.15.83:0/2992460089 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bf72882950 msgr2=0x55bf72882d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:48.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.853+0000 7f8a33e00640 1 --2- 172.21.15.83:0/2992460089 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bf72882950 0x55bf72882d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:48.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.853+0000 7f8a33e00640 1 -- 172.21.15.83:0/2992460089 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf7284b310 con 0x55bf72883270 2024-04-24T09:16:48.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.853+0000 7f8a33e00640 1 --2- 172.21.15.83:0/2992460089 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bf72883270 0x55bf7288d400 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f8a24011650 tx=0x7f8a24009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c96806e4739c4aa2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:48.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.854+0000 7f8a39200640 1 --2- 172.21.15.83:0/153751168 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f8a180a3da0 0x7f8a180a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:48.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.854+0000 7f8a29600640 1 -- 172.21.15.83:0/2992460089 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a2401e070 con 0x55bf72883270 2024-04-24T09:16:48.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.854+0000 7f8a30c00640 1 -- 172.21.15.83:0/153751168 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 9583+0+0 (secure 0 0 0) 0x7f8a200d0e10 con 0x7f8a3406e2e0 2024-04-24T09:16:48.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.854+0000 7f8a29600640 1 -- 172.21.15.83:0/2992460089 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8a2400fb70 con 0x55bf72883270 2024-04-24T09:16:48.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.854+0000 7f8a29600640 1 -- 172.21.15.83:0/2992460089 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a24019bd0 con 0x55bf72883270 2024-04-24T09:16:48.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.856+0000 7f8a3e1d6740 1 -- 172.21.15.83:0/2992460089 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bf72883270 msgr2=0x55bf7288d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:48.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.856+0000 7f8a3e1d6740 1 --2- 172.21.15.83:0/2992460089 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bf72883270 0x55bf7288d400 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f8a24011650 tx=0x7f8a24009f90 comp rx=0 tx=0).stop 2024-04-24T09:16:48.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.856+0000 7f8a39200640 1 --2- 172.21.15.83:0/153751168 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f8a180a3da0 0x7f8a180a6260 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c005fe0 tx=0x7f8a2c02a940 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:48.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.857+0000 7f8a3e1d6740 1 -- 172.21.15.83:0/2992460089 shutdown_connections 2024-04-24T09:16:48.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.857+0000 7f8a3e1d6740 1 --2- 172.21.15.83:0/2992460089 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bf7288d940 0x55bf7288fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:48.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.857+0000 7f8a3e1d6740 1 --2- 172.21.15.83:0/2992460089 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bf72883270 0x55bf7288d400 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:48.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.857+0000 7f8a3e1d6740 1 --2- 172.21.15.83:0/2992460089 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bf72882950 0x55bf72882d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:48.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.857+0000 7f8a3e1d6740 1 -- 172.21.15.83:0/2992460089 >> 172.21.15.83:0/2992460089 conn(0x55bf7287ec70 msgr2=0x55bf7287f050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:48.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.857+0000 7f8a3e1d6740 1 -- 172.21.15.83:0/2992460089 shutdown_connections 2024-04-24T09:16:48.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.858+0000 7f8a3e1d6740 1 -- 172.21.15.83:0/2992460089 wait complete. 2024-04-24T09:16:48.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.858+0000 7f8a3e1d6740 1 -- start start 2024-04-24T09:16:48.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.859+0000 7f8a3e1d6740 1 Processor -- start 2024-04-24T09:16:48.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.860+0000 7f8a3e1d6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bf72882950 0x55bf7288d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:48.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.860+0000 7f8a3e1d6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bf72883270 0x55bf7288a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:48.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.860+0000 7f8a3e1d6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bf7288d940 0x55bf7288adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:48.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.860+0000 7f8a3e1d6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55bf728907d0 con 0x55bf72883270 2024-04-24T09:16:48.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.861+0000 7f8a33400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bf72883270 0x55bf7288a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:48.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.861+0000 7f8a3e1d6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55bf727a1810 con 0x55bf72882950 2024-04-24T09:16:48.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.861+0000 7f8a3e1d6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55bf72890950 con 0x55bf7288d940 2024-04-24T09:16:48.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.861+0000 7f8a33400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bf72883270 0x55bf7288a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35896/0 (socket says 172.21.15.83:35896) 2024-04-24T09:16:48.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.861+0000 7f8a33400640 1 -- 172.21.15.83:0/1320568556 learned_addr learned my addr 172.21.15.83:0/1320568556 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:16:48.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.861+0000 7f8a3e1d6740 10 client.0 _ll_register_callbacks cb 0x55bf7276f140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:16:48.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.861+0000 7f8a39200640 1 --2- 172.21.15.83:0/1320568556 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bf7288d940 0x55bf7288adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:48.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.861+0000 7f8a33e00640 1 --2- 172.21.15.83:0/1320568556 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bf72882950 0x55bf7288d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:48.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.862+0000 7f8a39200640 1 -- 172.21.15.83:0/1320568556 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bf72882950 msgr2=0x55bf7288d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:48.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.862+0000 7f8a39200640 1 --2- 172.21.15.83:0/1320568556 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bf72882950 0x55bf7288d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:48.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.862+0000 7f8a39200640 1 -- 172.21.15.83:0/1320568556 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bf72883270 msgr2=0x55bf7288a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:48.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.862+0000 7f8a39200640 1 --2- 172.21.15.83:0/1320568556 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bf72883270 0x55bf7288a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:48.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.862+0000 7f8a39200640 1 -- 172.21.15.83:0/1320568556 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf727b23f0 con 0x55bf7288d940 2024-04-24T09:16:48.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.862+0000 7f8a33e00640 1 --2- 172.21.15.83:0/1320568556 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bf72882950 0x55bf7288d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:16:48.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.862+0000 7f8a39200640 1 --2- 172.21.15.83:0/1320568556 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bf7288d940 0x55bf7288adb0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c005cc0 tx=0x7f8a2c0095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a586fdeb6cf79f8e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:48.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.862+0000 7f8a1e000640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-04-24T09:16:48.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.863+0000 7f8a1e000640 1 -- 172.21.15.83:0/1320568556 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a2c034040 con 0x55bf7288d940 2024-04-24T09:16:48.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.863+0000 7f8a1e000640 1 -- 172.21.15.83:0/1320568556 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(132..132 src has 1..132) ==== 9583+0+0 (secure 0 0 0) 0x7f8a2c004790 con 0x55bf7288d940 2024-04-24T09:16:48.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.863+0000 7f8a3e1d6740 1 -- 172.21.15.83:0/1320568556 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a2400fb70 con 0x55bf7288d940 2024-04-24T09:16:48.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.864+0000 7f8a1e000640 1 -- 172.21.15.83:0/1320568556 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f8a2c030640 con 0x55bf7288d940 2024-04-24T09:16:48.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.864+0000 7f8a3e1d6740 10 client.30802 fetch_fsmap learned FSMap version 69 2024-04-24T09:16:48.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.864+0000 7f8a3e1d6740 1 -- 172.21.15.83:0/1320568556 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x55bf724fa3f0 con 0x55bf7288d940 2024-04-24T09:16:48.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.864+0000 7f8a1e000640 1 -- 172.21.15.83:0/1320568556 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8a2400fb70 con 0x55bf7288d940 2024-04-24T09:16:48.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.865+0000 7f8a1e000640 1 -- 172.21.15.83:0/1320568556 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 69) ==== 8370+0+0 (secure 0 0 0) 0x7f8a2c047070 con 0x55bf7288d940 2024-04-24T09:16:48.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.865+0000 7f8a3e1d6740 10 client.30802 fetch_fsmap finished waiting for FSMap version 69 2024-04-24T09:16:48.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.865+0000 7f8a3e1d6740 10 client.30802 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-24T09:16:48.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.865+0000 7f8a3e1d6740 20 client.30802 populate_metadata read hostname 'smithi083' 2024-04-24T09:16:48.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.865+0000 7f8a3e1d6740 1 --2- 172.21.15.83:0/1320568556 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55bf7289dbb0 0x55bf7289ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:48.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.865+0000 7f8a3e1d6740 4 client.30802 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:16:48.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.865+0000 7f8a3e1d6740 1 -- 172.21.15.83:0/1320568556 --> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bf727f5bf0 con 0x55bf7289dbb0 2024-04-24T09:16:48.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.865+0000 7f8a33e00640 1 --2- 172.21.15.83:0/1320568556 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55bf7289dbb0 0x55bf7289ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:48.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.866+0000 7f8a33e00640 1 --2- 172.21.15.83:0/1320568556 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55bf7289dbb0 0x55bf7289ffc0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=cebac808d65dd225 server_cookie=ff4cd8739efdc58b in_seq=0 out_seq=0 2024-04-24T09:16:48.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.866+0000 7f8a1e000640 10 client.30802 ms_handle_connect on v2:172.21.15.138:6832/1157967658 2024-04-24T09:16:48.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.868+0000 7f8a1e000640 1 -- 172.21.15.83:0/1320568556 <== mds.3 v2:172.21.15.138:6832/1157967658 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bf727f5bf0 con 0x55bf7289dbb0 2024-04-24T09:16:48.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.868+0000 7f8a1e000640 10 client.30802 handle_command_reply: tid=0 2024-04-24T09:16:48.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.868+0000 7f8a3e1d6740 1 client.30802 shutdown 2024-04-24T09:16:48.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.868+0000 7f8a3e1d6740 1 -- 172.21.15.83:0/1320568556 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bf7288d940 msgr2=0x55bf7288adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:48.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.868+0000 7f8a3e1d6740 1 --2- 172.21.15.83:0/1320568556 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bf7288d940 0x55bf7288adb0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c005cc0 tx=0x7f8a2c0095c0 comp rx=0 tx=0).stop 2024-04-24T09:16:48.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.869+0000 7f8a3e1d6740 1 -- 172.21.15.83:0/1320568556 shutdown_connections 2024-04-24T09:16:48.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.869+0000 7f8a3e1d6740 1 --2- 172.21.15.83:0/1320568556 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55bf7289dbb0 0x55bf7289ffc0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:48.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.869+0000 7f8a1e000640 0 client.30802 ms_handle_reset on v2:172.21.15.138:6832/1157967658 2024-04-24T09:16:48.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.869+0000 7f8a3e1d6740 1 --2- 172.21.15.83:0/1320568556 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bf7288d940 0x55bf7288adb0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:48.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.869+0000 7f8a3e1d6740 1 --2- 172.21.15.83:0/1320568556 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bf72883270 0x55bf7288a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:48.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.869+0000 7f8a3e1d6740 1 --2- 172.21.15.83:0/1320568556 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bf72882950 0x55bf7288d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:48.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.869+0000 7f8a3e1d6740 1 -- 172.21.15.83:0/1320568556 >> 172.21.15.83:0/1320568556 conn(0x55bf7287ec70 msgr2=0x55bf7287ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:48.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.869+0000 7f8a3e1d6740 1 -- 172.21.15.83:0/1320568556 shutdown_connections 2024-04-24T09:16:48.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.869+0000 7f8a3e1d6740 1 -- 172.21.15.83:0/1320568556 wait complete. 2024-04-24T09:16:48.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.869+0000 7f8a3e1d6740 20 client.30802 trim_cache size 0 max 16384 2024-04-24T09:16:48.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.896+0000 7f8a3e1d6740 1 Processor -- start 2024-04-24T09:16:48.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.896+0000 7f8a3e1d6740 1 -- start start 2024-04-24T09:16:48.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.896+0000 7f8a3e1d6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bf72882950 0x55bf7287e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:48.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.896+0000 7f8a3e1d6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bf72883270 0x55bf7288cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:48.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.896+0000 7f8a3e1d6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bf7288d940 0x55bf728a58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:48.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.896+0000 7f8a3e1d6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8a341be480 con 0x55bf72883270 2024-04-24T09:16:48.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.896+0000 7f8a3e1d6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8a341c0020 con 0x55bf7288d940 2024-04-24T09:16:48.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.896+0000 7f8a3e1d6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8a341beb50 con 0x55bf72882950 2024-04-24T09:16:48.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.897+0000 7f8a39200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bf7288d940 0x55bf728a58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:48.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.897+0000 7f8a39200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bf7288d940 0x55bf728a58e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:43138/0 (socket says 172.21.15.83:43138) 2024-04-24T09:16:48.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.897+0000 7f8a39200640 1 -- 172.21.15.83:0/3657976080 learned_addr learned my addr 172.21.15.83:0/3657976080 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:16:48.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.897+0000 7f8a33400640 1 --2- 172.21.15.83:0/3657976080 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bf72883270 0x55bf7288cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:48.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.897+0000 7f8a33e00640 1 --2- 172.21.15.83:0/3657976080 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bf72882950 0x55bf7287e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:48.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.897+0000 7f8a39200640 1 -- 172.21.15.83:0/3657976080 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bf72882950 msgr2=0x55bf7287e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:48.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.897+0000 7f8a39200640 1 --2- 172.21.15.83:0/3657976080 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bf72882950 0x55bf7287e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:48.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.897+0000 7f8a39200640 1 -- 172.21.15.83:0/3657976080 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bf72883270 msgr2=0x55bf7288cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:48.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.897+0000 7f8a39200640 1 --2- 172.21.15.83:0/3657976080 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bf72883270 0x55bf7288cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:48.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.897+0000 7f8a39200640 1 -- 172.21.15.83:0/3657976080 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf725c47d0 con 0x55bf7288d940 2024-04-24T09:16:48.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.898+0000 7f8a33400640 1 --2- 172.21.15.83:0/3657976080 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bf72883270 0x55bf7288cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:16:48.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.898+0000 7f8a33e00640 1 --2- 172.21.15.83:0/3657976080 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bf72882950 0x55bf7287e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:16:48.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.898+0000 7f8a39200640 1 --2- 172.21.15.83:0/3657976080 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bf7288d940 0x55bf728a58e0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c02fe50 tx=0x7f8a2c009ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8779c228778f2fb7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:48.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.898+0000 7f8a28c00640 1 -- 172.21.15.83:0/3657976080 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a2c061030 con 0x55bf7288d940 2024-04-24T09:16:48.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.899+0000 7f8a28c00640 1 -- 172.21.15.83:0/3657976080 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8a2c044040 con 0x55bf7288d940 2024-04-24T09:16:48.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.899+0000 7f8a28c00640 1 -- 172.21.15.83:0/3657976080 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a2c0307e0 con 0x55bf7288d940 2024-04-24T09:16:48.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.899+0000 7f8a3e1d6740 1 -- 172.21.15.83:0/3657976080 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bf7288d940 msgr2=0x55bf728a58e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:48.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.899+0000 7f8a3e1d6740 1 --2- 172.21.15.83:0/3657976080 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bf7288d940 0x55bf728a58e0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c02fe50 tx=0x7f8a2c009ba0 comp rx=0 tx=0).stop 2024-04-24T09:16:48.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.900+0000 7f8a3e1d6740 1 -- 172.21.15.83:0/3657976080 shutdown_connections 2024-04-24T09:16:48.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.900+0000 7f8a3e1d6740 1 --2- 172.21.15.83:0/3657976080 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bf7288d940 0x55bf728a58e0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:48.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.900+0000 7f8a3e1d6740 1 --2- 172.21.15.83:0/3657976080 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bf72883270 0x55bf7288cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:48.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.900+0000 7f8a3e1d6740 1 --2- 172.21.15.83:0/3657976080 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bf72882950 0x55bf7287e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:48.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.900+0000 7f8a3e1d6740 1 -- 172.21.15.83:0/3657976080 >> 172.21.15.83:0/3657976080 conn(0x55bf7287ec70 msgr2=0x55bf7287f050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:48.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.900+0000 7f8a3e1d6740 1 -- 172.21.15.83:0/3657976080 shutdown_connections 2024-04-24T09:16:48.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.900+0000 7f8a3e1d6740 1 -- 172.21.15.83:0/3657976080 wait complete. 2024-04-24T09:16:48.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.900+0000 7f8a3e1d6740 1 -- start start 2024-04-24T09:16:48.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.901+0000 7f8a3e1d6740 1 Processor -- start 2024-04-24T09:16:48.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.901+0000 7f8a3e1d6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bf72882950 0x55bf7288e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:48.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.901+0000 7f8a3e1d6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bf72883270 0x55bf7288ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:48.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.901+0000 7f8a3e1d6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bf7288d940 0x55bf72894c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:48.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.901+0000 7f8a3e1d6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8a341bf950 con 0x55bf72883270 2024-04-24T09:16:48.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.901+0000 7f8a3e1d6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8a341bf280 con 0x55bf7288d940 2024-04-24T09:16:48.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.901+0000 7f8a3e1d6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8a341bc480 con 0x55bf72882950 2024-04-24T09:16:48.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.901+0000 7f8a3e1d6740 10 client.0 _ll_register_callbacks cb 0x7f8a24005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:16:48.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.902+0000 7f8a33400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bf72883270 0x55bf7288ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:48.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.902+0000 7f8a39200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bf7288d940 0x55bf72894c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:48.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.902+0000 7f8a33400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bf72883270 0x55bf7288ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35916/0 (socket says 172.21.15.83:35916) 2024-04-24T09:16:48.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.902+0000 7f8a33400640 1 -- 172.21.15.83:0/3867524636 learned_addr learned my addr 172.21.15.83:0/3867524636 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:16:48.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.902+0000 7f8a33e00640 1 --2- 172.21.15.83:0/3867524636 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bf72882950 0x55bf7288e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:48.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.903+0000 7f8a33e00640 1 -- 172.21.15.83:0/3867524636 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bf7288d940 msgr2=0x55bf72894c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:48.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.903+0000 7f8a33e00640 1 --2- 172.21.15.83:0/3867524636 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bf7288d940 0x55bf72894c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:48.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.903+0000 7f8a33e00640 1 -- 172.21.15.83:0/3867524636 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bf72883270 msgr2=0x55bf7288ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:48.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.903+0000 7f8a33e00640 1 --2- 172.21.15.83:0/3867524636 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bf72883270 0x55bf7288ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:48.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.903+0000 7f8a33e00640 1 -- 172.21.15.83:0/3867524636 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf728951f0 con 0x55bf72882950 2024-04-24T09:16:48.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.903+0000 7f8a33400640 1 --2- 172.21.15.83:0/3867524636 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bf72883270 0x55bf7288ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:16:48.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.903+0000 7f8a33e00640 1 --2- 172.21.15.83:0/3867524636 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bf72882950 0x55bf7288e510 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8a240197f0 tx=0x7f8a2400fca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b552efc5dad24f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:48.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.903+0000 7f8a1e000640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-04-24T09:16:48.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.903+0000 7f8a1e000640 1 -- 172.21.15.83:0/3867524636 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a2402f060 con 0x55bf72882950 2024-04-24T09:16:48.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.904+0000 7f8a1e000640 1 -- 172.21.15.83:0/3867524636 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(132..132 src has 1..132) ==== 9583+0+0 (secure 0 0 0) 0x7f8a24009c10 con 0x55bf72882950 2024-04-24T09:16:48.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.904+0000 7f8a3e1d6740 1 -- 172.21.15.83:0/3867524636 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a2c044040 con 0x55bf72882950 2024-04-24T09:16:48.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.904+0000 7f8a1e000640 1 -- 172.21.15.83:0/3867524636 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f8a2401e040 con 0x55bf72882950 2024-04-24T09:16:48.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.904+0000 7f8a3e1d6740 10 client.30808 fetch_fsmap learned FSMap version 69 2024-04-24T09:16:48.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.904+0000 7f8a3e1d6740 1 -- 172.21.15.83:0/3867524636 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x55bf728964b0 con 0x55bf72882950 2024-04-24T09:16:48.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.905+0000 7f8a1e000640 1 -- 172.21.15.83:0/3867524636 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8a2c044040 con 0x55bf72882950 2024-04-24T09:16:48.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.906+0000 7f8a1e000640 1 -- 172.21.15.83:0/3867524636 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 69) ==== 8370+0+0 (secure 0 0 0) 0x7f8a24019ac0 con 0x55bf72882950 2024-04-24T09:16:48.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.906+0000 7f8a3e1d6740 10 client.30808 fetch_fsmap finished waiting for FSMap version 69 2024-04-24T09:16:48.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.906+0000 7f8a3e1d6740 10 client.30808 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-24T09:16:48.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.906+0000 7f8a3e1d6740 20 client.30808 populate_metadata read hostname 'smithi083' 2024-04-24T09:16:48.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.906+0000 7f8a3e1d6740 1 --2- 172.21.15.83:0/3867524636 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55bf7289dbb0 0x55bf72917480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:48.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.906+0000 7f8a3e1d6740 4 client.30808 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:16:48.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.906+0000 7f8a3e1d6740 1 -- 172.21.15.83:0/3867524636 --> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bf727efa60 con 0x55bf7289dbb0 2024-04-24T09:16:48.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.907+0000 7f8a33400640 1 --2- 172.21.15.83:0/3867524636 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55bf7289dbb0 0x55bf72917480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:48.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.908+0000 7f8a1e000640 10 client.30808 ms_handle_connect on v2:172.21.15.138:6832/1157967658 2024-04-24T09:16:48.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.908+0000 7f8a33400640 1 --2- 172.21.15.83:0/3867524636 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55bf7289dbb0 0x55bf72917480 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7bca9d22f6fc1c57 server_cookie=d9579bf0a822d618 in_seq=0 out_seq=0 2024-04-24T09:16:48.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.908+0000 7f8a1e000640 1 -- 172.21.15.83:0/3867524636 <== mds.3 v2:172.21.15.138:6832/1157967658 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bf727efa60 con 0x55bf7289dbb0 2024-04-24T09:16:48.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.908+0000 7f8a1e000640 10 client.30808 handle_command_reply: tid=0 2024-04-24T09:16:48.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.909+0000 7f8a3e1d6740 1 client.30808 shutdown 2024-04-24T09:16:48.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.909+0000 7f8a3e1d6740 1 -- 172.21.15.83:0/3867524636 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bf72882950 msgr2=0x55bf7288e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:48.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.909+0000 7f8a3e1d6740 1 --2- 172.21.15.83:0/3867524636 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bf72882950 0x55bf7288e510 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8a240197f0 tx=0x7f8a2400fca0 comp rx=0 tx=0).stop 2024-04-24T09:16:48.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.909+0000 7f8a3e1d6740 1 -- 172.21.15.83:0/3867524636 shutdown_connections 2024-04-24T09:16:48.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.909+0000 7f8a3e1d6740 1 --2- 172.21.15.83:0/3867524636 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bf7288d940 0x55bf72894c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:48.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.909+0000 7f8a3e1d6740 1 --2- 172.21.15.83:0/3867524636 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bf72883270 0x55bf7288ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:48.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.909+0000 7f8a3e1d6740 1 --2- 172.21.15.83:0/3867524636 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55bf7289dbb0 0x55bf72917480 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:48.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.909+0000 7f8a3e1d6740 1 --2- 172.21.15.83:0/3867524636 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bf72882950 0x55bf7288e510 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:48.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.909+0000 7f8a3e1d6740 1 -- 172.21.15.83:0/3867524636 >> 172.21.15.83:0/3867524636 conn(0x55bf7287ec70 msgr2=0x55bf72889f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:48.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.909+0000 7f8a1e000640 0 client.30808 ms_handle_reset on v2:172.21.15.138:6832/1157967658 2024-04-24T09:16:48.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.909+0000 7f8a3e1d6740 1 -- 172.21.15.83:0/3867524636 shutdown_connections 2024-04-24T09:16:48.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.909+0000 7f8a3e1d6740 1 -- 172.21.15.83:0/3867524636 wait complete. 2024-04-24T09:16:48.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.910+0000 7f8a3e1d6740 20 client.30808 trim_cache size 0 max 16384 2024-04-24T09:16:48.911 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:16:48.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.910+0000 7f8a3b200640 1 -- 172.21.15.83:0/153751168 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f8a180a3da0 msgr2=0x7f8a180a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:48.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.910+0000 7f8a3b200640 1 --2- 172.21.15.83:0/153751168 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f8a180a3da0 0x7f8a180a6260 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c005fe0 tx=0x7f8a2c02a940 comp rx=0 tx=0).stop 2024-04-24T09:16:48.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.910+0000 7f8a3b200640 1 -- 172.21.15.83:0/153751168 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a3406e2e0 msgr2=0x7f8a341a7b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:48.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.910+0000 7f8a3b200640 1 --2- 172.21.15.83:0/153751168 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a3406e2e0 0x7f8a341a7b10 secure :-1 s=READY pgs=2798 cs=0 l=1 rev1=1 crypto rx=0x7f8a2000e9f0 tx=0x7f8a2000eeb0 comp rx=0 tx=0).stop 2024-04-24T09:16:48.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.911+0000 7f8a3b200640 1 -- 172.21.15.83:0/153751168 shutdown_connections 2024-04-24T09:16:48.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.911+0000 7f8a3b200640 1 --2- 172.21.15.83:0/153751168 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f8a180a3da0 0x7f8a180a6260 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:48.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.911+0000 7f8a3b200640 1 --2- 172.21.15.83:0/153751168 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a3411b6a0 0x7f8a341abe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:48.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.911+0000 7f8a3b200640 1 --2- 172.21.15.83:0/153751168 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8a3406e2e0 0x7f8a341a7b10 unknown :-1 s=CLOSED pgs=2798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:48.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.911+0000 7f8a3b200640 1 --2- 172.21.15.83:0/153751168 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8a3406d9c0 0x7f8a341a75d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:48.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.911+0000 7f8a3b200640 1 -- 172.21.15.83:0/153751168 >> 172.21.15.83:0/153751168 conn(0x7f8a3406d030 msgr2=0x7f8a3410b010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:48.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.911+0000 7f8a3b200640 1 -- 172.21.15.83:0/153751168 shutdown_connections 2024-04-24T09:16:48.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:48.911+0000 7f8a3b200640 1 -- 172.21.15.83:0/153751168 wait complete. 2024-04-24T09:16:48.935 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T09:16:49.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.203+0000 7f5bb2a00640 1 -- 172.21.15.83:0/2640126770 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5bac0742d0 msgr2=0x7f5bac0746b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:49.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.203+0000 7f5bb2a00640 1 --2- 172.21.15.83:0/2640126770 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5bac0742d0 0x7f5bac0746b0 secure :-1 s=READY pgs=2799 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c009930 tx=0x7f5b9c02a4d0 comp rx=0 tx=0).stop 2024-04-24T09:16:49.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.204+0000 7f5bb2a00640 1 -- 172.21.15.83:0/2640126770 shutdown_connections 2024-04-24T09:16:49.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.204+0000 7f5bb2a00640 1 --2- 172.21.15.83:0/2640126770 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5bac1130b0 0x7f5bac1154a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:49.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.204+0000 7f5bb2a00640 1 --2- 172.21.15.83:0/2640126770 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5bac074bf0 0x7f5bac112b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:49.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.204+0000 7f5bb2a00640 1 --2- 172.21.15.83:0/2640126770 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5bac0742d0 0x7f5bac0746b0 unknown :-1 s=CLOSED pgs=2799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:49.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.204+0000 7f5bb2a00640 1 -- 172.21.15.83:0/2640126770 >> 172.21.15.83:0/2640126770 conn(0x7f5bac079450 msgr2=0x7f5bac079860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:49.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.204+0000 7f5bb2a00640 1 -- 172.21.15.83:0/2640126770 shutdown_connections 2024-04-24T09:16:49.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.204+0000 7f5bb2a00640 1 -- 172.21.15.83:0/2640126770 wait complete. 2024-04-24T09:16:49.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.205+0000 7f5bb2a00640 1 Processor -- start 2024-04-24T09:16:49.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.205+0000 7f5bb2a00640 1 -- start start 2024-04-24T09:16:49.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.206+0000 7f5bb2a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5bac0742d0 0x7f5bac19eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:49.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.206+0000 7f5bb2a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5bac074bf0 0x7f5bac19f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:49.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.206+0000 7f5bb2a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5bac1130b0 0x7f5bac1a37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:49.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.206+0000 7f5bb2a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5bac118930 con 0x7f5bac0742d0 2024-04-24T09:16:49.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.206+0000 7f5bb2a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5bac1187b0 con 0x7f5bac1130b0 2024-04-24T09:16:49.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.206+0000 7f5bb2a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5bac118ab0 con 0x7f5bac074bf0 2024-04-24T09:16:49.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.207+0000 7f5babe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5bac0742d0 0x7f5bac19eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:49.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.207+0000 7f5babe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5bac0742d0 0x7f5bac19eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35948/0 (socket says 172.21.15.83:35948) 2024-04-24T09:16:49.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.207+0000 7f5babe00640 1 -- 172.21.15.83:0/2893739704 learned_addr learned my addr 172.21.15.83:0/2893739704 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:16:49.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.207+0000 7f5bb0a00640 1 --2- 172.21.15.83:0/2893739704 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5bac1130b0 0x7f5bac1a37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:49.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.207+0000 7f5bab400640 1 --2- 172.21.15.83:0/2893739704 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5bac074bf0 0x7f5bac19f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:49.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.207+0000 7f5bb0a00640 1 -- 172.21.15.83:0/2893739704 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5bac074bf0 msgr2=0x7f5bac19f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:49.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.207+0000 7f5bb0a00640 1 --2- 172.21.15.83:0/2893739704 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5bac074bf0 0x7f5bac19f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:49.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.207+0000 7f5bb0a00640 1 -- 172.21.15.83:0/2893739704 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5bac0742d0 msgr2=0x7f5bac19eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:49.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.207+0000 7f5bb0a00640 1 --2- 172.21.15.83:0/2893739704 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5bac0742d0 0x7f5bac19eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:49.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.207+0000 7f5bb0a00640 1 -- 172.21.15.83:0/2893739704 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bac1a3ee0 con 0x7f5bac1130b0 2024-04-24T09:16:49.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.208+0000 7f5bb0a00640 1 --2- 172.21.15.83:0/2893739704 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5bac1130b0 0x7f5bac1a37b0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f5ba4005fd0 tx=0x7f5ba400edd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:49.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.208+0000 7f5ba8c00640 1 -- 172.21.15.83:0/2893739704 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ba4004d20 con 0x7f5bac1130b0 2024-04-24T09:16:49.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.208+0000 7f5ba8c00640 1 -- 172.21.15.83:0/2893739704 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5ba4005020 con 0x7f5bac1130b0 2024-04-24T09:16:49.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.208+0000 7f5ba8c00640 1 -- 172.21.15.83:0/2893739704 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ba40116d0 con 0x7f5bac1130b0 2024-04-24T09:16:49.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.209+0000 7f5bb2a00640 1 -- 172.21.15.83:0/2893739704 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bac1a41d0 con 0x7f5bac1130b0 2024-04-24T09:16:49.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.209+0000 7f5bb2a00640 1 -- 172.21.15.83:0/2893739704 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bac1aba60 con 0x7f5bac1130b0 2024-04-24T09:16:49.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.214+0000 7f5ba8c00640 1 -- 172.21.15.83:0/2893739704 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5ba40051d0 con 0x7f5bac1130b0 2024-04-24T09:16:49.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.216+0000 7f5bb5990740 1 Processor -- start 2024-04-24T09:16:49.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.217+0000 7f5bb5990740 1 -- start start 2024-04-24T09:16:49.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.217+0000 7f5bb5990740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ddddea5950 0x55ddddea5d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:49.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.217+0000 7f5bb5990740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ddddea6270 0x55ddddeb0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:49.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.217+0000 7f5ba8c00640 1 --2- 172.21.15.83:0/2893739704 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f5b900a3de0 0x7f5b900a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:49.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.217+0000 7f5bb5990740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ddddeb0940 0x55ddddeb2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:49.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.217+0000 7f5bb5990740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55dddde0b1d0 con 0x55ddddea5950 2024-04-24T09:16:49.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.217+0000 7f5bb5990740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55dddddb1450 con 0x55ddddeb0940 2024-04-24T09:16:49.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.217+0000 7f5bb5990740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55dddde07190 con 0x55ddddea6270 2024-04-24T09:16:49.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.217+0000 7f5ba8c00640 1 -- 172.21.15.83:0/2893739704 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 9583+0+0 (secure 0 0 0) 0x7f5ba40d0d00 con 0x7f5bac1130b0 2024-04-24T09:16:49.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.218+0000 7f5bb0a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ddddeb0940 0x55ddddeb2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:49.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.218+0000 7f5bb0a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ddddeb0940 0x55ddddeb2d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:43172/0 (socket says 172.21.15.83:43172) 2024-04-24T09:16:49.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.218+0000 7f5bb0a00640 1 -- 172.21.15.83:0/3047630332 learned_addr learned my addr 172.21.15.83:0/3047630332 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:16:49.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.218+0000 7f5bab400640 1 --2- 172.21.15.83:0/3047630332 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ddddea5950 0x55ddddea5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:49.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.218+0000 7f5babe00640 1 --2- 172.21.15.83:0/2893739704 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f5b900a3de0 0x7f5b900a62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:49.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.218+0000 7f5bb0a00640 1 -- 172.21.15.83:0/3047630332 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ddddea6270 msgr2=0x55ddddeb0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:49.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.218+0000 7f5bb0a00640 1 --2- 172.21.15.83:0/3047630332 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ddddea6270 0x55ddddeb0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:49.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.218+0000 7f5bb0a00640 1 -- 172.21.15.83:0/3047630332 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ddddea5950 msgr2=0x55ddddea5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:49.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.218+0000 7f5bb0a00640 1 --2- 172.21.15.83:0/3047630332 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ddddea5950 0x55ddddea5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:49.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.218+0000 7f5bb0a00640 1 -- 172.21.15.83:0/3047630332 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dddde6e310 con 0x55ddddeb0940 2024-04-24T09:16:49.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.219+0000 7f5bb0a00640 1 --2- 172.21.15.83:0/3047630332 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ddddeb0940 0x55ddddeb2d30 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f5ba40a4f60 tx=0x7f5ba409c240 comp rx=0 tx=0).ready entity=mon.2 client_cookie=186a47caf175fc60 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:49.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.219+0000 7f5ba1200640 1 -- 172.21.15.83:0/3047630332 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ba401f040 con 0x55ddddeb0940 2024-04-24T09:16:49.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.219+0000 7f5ba1200640 1 -- 172.21.15.83:0/3047630332 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5ba40675c0 con 0x55ddddeb0940 2024-04-24T09:16:49.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.220+0000 7f5ba1200640 1 -- 172.21.15.83:0/3047630332 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ba40870d0 con 0x55ddddeb0940 2024-04-24T09:16:49.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.221+0000 7f5bb5990740 1 -- 172.21.15.83:0/3047630332 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ddddeb0940 msgr2=0x55ddddeb2d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:49.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.221+0000 7f5babe00640 1 --2- 172.21.15.83:0/2893739704 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f5b900a3de0 0x7f5b900a62a0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c000c00 tx=0x7f5b9c0095c0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:49.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.221+0000 7f5bb5990740 1 --2- 172.21.15.83:0/3047630332 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ddddeb0940 0x55ddddeb2d30 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f5ba40a4f60 tx=0x7f5ba409c240 comp rx=0 tx=0).stop 2024-04-24T09:16:49.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.221+0000 7f5bb5990740 1 -- 172.21.15.83:0/3047630332 shutdown_connections 2024-04-24T09:16:49.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.221+0000 7f5bb5990740 1 --2- 172.21.15.83:0/3047630332 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ddddeb0940 0x55ddddeb2d30 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:49.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.221+0000 7f5bb5990740 1 --2- 172.21.15.83:0/3047630332 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ddddea6270 0x55ddddeb0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:49.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.221+0000 7f5bb5990740 1 --2- 172.21.15.83:0/3047630332 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ddddea5950 0x55ddddea5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:49.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.221+0000 7f5bb5990740 1 -- 172.21.15.83:0/3047630332 >> 172.21.15.83:0/3047630332 conn(0x55ddddea1c70 msgr2=0x55ddddea2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:49.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.221+0000 7f5bb5990740 1 -- 172.21.15.83:0/3047630332 shutdown_connections 2024-04-24T09:16:49.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.221+0000 7f5bb5990740 1 -- 172.21.15.83:0/3047630332 wait complete. 2024-04-24T09:16:49.222 INFO:tasks.workunit.client.0.smithi083.stdout: 10 83073 6.50 MB/sec execute 560 sec latency 107.399 ms 2024-04-24T09:16:49.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.222+0000 7f5bb5990740 1 -- start start 2024-04-24T09:16:49.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.222+0000 7f5bb5990740 1 Processor -- start 2024-04-24T09:16:49.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.223+0000 7f5bb5990740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ddddea5950 0x55ddddead740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:49.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.223+0000 7f5bb5990740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ddddea6270 0x55ddddeadc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:49.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.223+0000 7f5bb5990740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ddddeb0940 0x55ddddeae1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:49.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.223+0000 7f5bb5990740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ddddeb3770 con 0x55ddddea6270 2024-04-24T09:16:49.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.223+0000 7f5bb5990740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55dddddc4810 con 0x55ddddea5950 2024-04-24T09:16:49.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.223+0000 7f5bb5990740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55ddddeb38f0 con 0x55ddddeb0940 2024-04-24T09:16:49.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.223+0000 7f5bb5990740 10 client.0 _ll_register_callbacks cb 0x55ddddd92140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:16:49.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.224+0000 7f5babe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ddddea5950 0x55ddddead740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:49.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.224+0000 7f5babe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ddddea5950 0x55ddddead740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:43184/0 (socket says 172.21.15.83:43184) 2024-04-24T09:16:49.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.224+0000 7f5babe00640 1 -- 172.21.15.83:0/3501595664 learned_addr learned my addr 172.21.15.83:0/3501595664 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:16:49.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.224+0000 7f5bb0a00640 1 --2- 172.21.15.83:0/3501595664 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ddddeb0940 0x55ddddeae1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:49.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.224+0000 7f5bab400640 1 --2- 172.21.15.83:0/3501595664 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ddddea6270 0x55ddddeadc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:49.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.225+0000 7f5bab400640 1 -- 172.21.15.83:0/3501595664 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ddddeb0940 msgr2=0x55ddddeae1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:49.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.225+0000 7f5bab400640 1 --2- 172.21.15.83:0/3501595664 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ddddeb0940 0x55ddddeae1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:49.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.225+0000 7f5bab400640 1 -- 172.21.15.83:0/3501595664 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ddddea5950 msgr2=0x55ddddead740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:49.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.225+0000 7f5bab400640 1 --2- 172.21.15.83:0/3501595664 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ddddea5950 0x55ddddead740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:49.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.225+0000 7f5bab400640 1 -- 172.21.15.83:0/3501595664 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dddddd53f0 con 0x55ddddea6270 2024-04-24T09:16:49.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.226+0000 7f5bb0a00640 1 --2- 172.21.15.83:0/3501595664 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ddddeb0940 0x55ddddeae1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:16:49.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.226+0000 7f5bab400640 1 --2- 172.21.15.83:0/3501595664 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ddddea6270 0x55ddddeadc80 secure :-1 s=READY pgs=2800 cs=0 l=1 rev1=1 crypto rx=0x7f5b98006aa0 tx=0x7f5b98010620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdb5184101c61c2a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:49.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.226+0000 7f5b95600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:16:49.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.227+0000 7f5b95600640 1 -- 172.21.15.83:0/3501595664 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b98018020 con 0x55ddddea6270 2024-04-24T09:16:49.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.227+0000 7f5b95600640 1 -- 172.21.15.83:0/3501595664 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(132..132 src has 1..132) ==== 9583+0+0 (secure 0 0 0) 0x7f5b98014680 con 0x55ddddea6270 2024-04-24T09:16:49.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.227+0000 7f5bb5990740 1 -- 172.21.15.83:0/3501595664 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ba40675c0 con 0x55ddddea6270 2024-04-24T09:16:49.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.228+0000 7f5b95600640 1 -- 172.21.15.83:0/3501595664 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f5b98023050 con 0x55ddddea6270 2024-04-24T09:16:49.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.228+0000 7f5bb5990740 10 client.24066 fetch_fsmap learned FSMap version 69 2024-04-24T09:16:49.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.228+0000 7f5bb5990740 1 -- 172.21.15.83:0/3501595664 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x55ddddb1d3f0 con 0x55ddddea6270 2024-04-24T09:16:49.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.229+0000 7f5b95600640 1 -- 172.21.15.83:0/3501595664 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5ba40675c0 con 0x55ddddea6270 2024-04-24T09:16:49.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.229+0000 7f5b95600640 1 -- 172.21.15.83:0/3501595664 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 69) ==== 8370+0+0 (secure 0 0 0) 0x7f5b98013940 con 0x55ddddea6270 2024-04-24T09:16:49.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.229+0000 7f5bb5990740 10 client.24066 fetch_fsmap finished waiting for FSMap version 69 2024-04-24T09:16:49.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.229+0000 7f5bb5990740 10 client.24066 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-24T09:16:49.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.229+0000 7f5bb5990740 20 client.24066 populate_metadata read hostname 'smithi083' 2024-04-24T09:16:49.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.229+0000 7f5bb5990740 1 --2- 172.21.15.83:0/3501595664 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55ddddec0cc0 0x55ddddec30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:49.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.229+0000 7f5bb5990740 4 client.24066 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:16:49.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.229+0000 7f5bb5990740 1 -- 172.21.15.83:0/3501595664 --> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dddde18bf0 con 0x55ddddec0cc0 2024-04-24T09:16:49.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.230+0000 7f5babe00640 1 --2- 172.21.15.83:0/3501595664 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55ddddec0cc0 0x55ddddec30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:49.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.231+0000 7f5babe00640 1 --2- 172.21.15.83:0/3501595664 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55ddddec0cc0 0x55ddddec30e0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1c31aa3d90794e7f server_cookie=ce6423486703b0a7 in_seq=0 out_seq=0 2024-04-24T09:16:49.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.231+0000 7f5b95600640 10 client.24066 ms_handle_connect on v2:172.21.15.138:6834/1909219758 2024-04-24T09:16:49.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.232+0000 7f5b95600640 1 -- 172.21.15.83:0/3501595664 <== mds.2 v2:172.21.15.138:6834/1909219758 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55dddde18bf0 con 0x55ddddec0cc0 2024-04-24T09:16:49.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.232+0000 7f5b95600640 10 client.24066 handle_command_reply: tid=0 2024-04-24T09:16:49.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.232+0000 7f5bb5990740 1 client.24066 shutdown 2024-04-24T09:16:49.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.232+0000 7f5bb5990740 1 -- 172.21.15.83:0/3501595664 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ddddea6270 msgr2=0x55ddddeadc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:49.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.232+0000 7f5bb5990740 1 --2- 172.21.15.83:0/3501595664 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ddddea6270 0x55ddddeadc80 secure :-1 s=READY pgs=2800 cs=0 l=1 rev1=1 crypto rx=0x7f5b98006aa0 tx=0x7f5b98010620 comp rx=0 tx=0).stop 2024-04-24T09:16:49.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.233+0000 7f5bb5990740 1 -- 172.21.15.83:0/3501595664 shutdown_connections 2024-04-24T09:16:49.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.233+0000 7f5bb5990740 1 --2- 172.21.15.83:0/3501595664 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55ddddec0cc0 0x55ddddec30e0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:49.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.233+0000 7f5bb5990740 1 --2- 172.21.15.83:0/3501595664 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ddddeb0940 0x55ddddeae1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:49.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.233+0000 7f5bb5990740 1 --2- 172.21.15.83:0/3501595664 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ddddea6270 0x55ddddeadc80 unknown :-1 s=CLOSED pgs=2800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:49.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.233+0000 7f5bb5990740 1 --2- 172.21.15.83:0/3501595664 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ddddea5950 0x55ddddead740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:49.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.233+0000 7f5b95600640 0 client.24066 ms_handle_reset on v2:172.21.15.138:6834/1909219758 2024-04-24T09:16:49.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.233+0000 7f5bb5990740 1 -- 172.21.15.83:0/3501595664 >> 172.21.15.83:0/3501595664 conn(0x55ddddea1c70 msgr2=0x55ddddea7a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:49.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.233+0000 7f5bb5990740 1 -- 172.21.15.83:0/3501595664 shutdown_connections 2024-04-24T09:16:49.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.234+0000 7f5bb5990740 1 -- 172.21.15.83:0/3501595664 wait complete. 2024-04-24T09:16:49.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.234+0000 7f5bb5990740 20 client.24066 trim_cache size 0 max 16384 2024-04-24T09:16:49.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.257+0000 7f5bb5990740 1 Processor -- start 2024-04-24T09:16:49.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.257+0000 7f5bb5990740 1 -- start start 2024-04-24T09:16:49.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.258+0000 7f5bb5990740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ddddea5950 0x55ddddea1570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:49.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.258+0000 7f5bb5990740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ddddeb0940 0x55ddddeadfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:49.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.258+0000 7f5bb5990740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ddddeacae0 0x55ddddec89f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:49.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.258+0000 7f5bb5990740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5bac1b5d20 con 0x55ddddeb0940 2024-04-24T09:16:49.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.258+0000 7f5bb5990740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5bac1b78c0 con 0x55ddddea5950 2024-04-24T09:16:49.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.258+0000 7f5bb5990740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5bac1b63f0 con 0x55ddddeacae0 2024-04-24T09:16:49.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.258+0000 7f5babe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ddddea5950 0x55ddddea1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:49.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.258+0000 7f5babe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ddddea5950 0x55ddddea1570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:43188/0 (socket says 172.21.15.83:43188) 2024-04-24T09:16:49.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.258+0000 7f5babe00640 1 -- 172.21.15.83:0/676407528 learned_addr learned my addr 172.21.15.83:0/676407528 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:16:49.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.258+0000 7f5bab400640 1 --2- 172.21.15.83:0/676407528 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ddddeb0940 0x55ddddeadfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:49.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.259+0000 7f5bb0a00640 1 --2- 172.21.15.83:0/676407528 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ddddeacae0 0x55ddddec89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:49.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.259+0000 7f5bab400640 1 -- 172.21.15.83:0/676407528 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ddddeacae0 msgr2=0x55ddddec89f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:49.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.259+0000 7f5bab400640 1 --2- 172.21.15.83:0/676407528 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ddddeacae0 0x55ddddec89f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:49.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.259+0000 7f5bab400640 1 -- 172.21.15.83:0/676407528 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ddddea5950 msgr2=0x55ddddea1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:49.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.259+0000 7f5bab400640 1 --2- 172.21.15.83:0/676407528 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ddddea5950 0x55ddddea1570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:49.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.259+0000 7f5bab400640 1 -- 172.21.15.83:0/676407528 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ddddbe77d0 con 0x55ddddeb0940 2024-04-24T09:16:49.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.259+0000 7f5bab400640 1 --2- 172.21.15.83:0/676407528 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ddddeb0940 0x55ddddeadfe0 secure :-1 s=READY pgs=2801 cs=0 l=1 rev1=1 crypto rx=0x55ddddeae800 tx=0x7f5b98004640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23a6dc5430fb5426 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:49.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.260+0000 7f5b97e00640 1 -- 172.21.15.83:0/676407528 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b9802b070 con 0x55ddddeb0940 2024-04-24T09:16:49.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.260+0000 7f5b97e00640 1 -- 172.21.15.83:0/676407528 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5b9800aab0 con 0x55ddddeb0940 2024-04-24T09:16:49.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.260+0000 7f5b97e00640 1 -- 172.21.15.83:0/676407528 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b98018040 con 0x55ddddeb0940 2024-04-24T09:16:49.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.261+0000 7f5bb5990740 1 -- 172.21.15.83:0/676407528 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ddddeb0940 msgr2=0x55ddddeadfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:49.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.261+0000 7f5bb5990740 1 --2- 172.21.15.83:0/676407528 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ddddeb0940 0x55ddddeadfe0 secure :-1 s=READY pgs=2801 cs=0 l=1 rev1=1 crypto rx=0x55ddddeae800 tx=0x7f5b98004640 comp rx=0 tx=0).stop 2024-04-24T09:16:49.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.261+0000 7f5bb5990740 1 -- 172.21.15.83:0/676407528 shutdown_connections 2024-04-24T09:16:49.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.261+0000 7f5bb5990740 1 --2- 172.21.15.83:0/676407528 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ddddeacae0 0x55ddddec89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:49.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.261+0000 7f5bb5990740 1 --2- 172.21.15.83:0/676407528 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ddddeb0940 0x55ddddeadfe0 unknown :-1 s=CLOSED pgs=2801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:49.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.261+0000 7f5bb5990740 1 --2- 172.21.15.83:0/676407528 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ddddea5950 0x55ddddea1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:49.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.261+0000 7f5bb5990740 1 -- 172.21.15.83:0/676407528 >> 172.21.15.83:0/676407528 conn(0x55ddddea1c70 msgr2=0x55ddddea7970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:49.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.261+0000 7f5bb5990740 1 -- 172.21.15.83:0/676407528 shutdown_connections 2024-04-24T09:16:49.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.261+0000 7f5bb5990740 1 -- 172.21.15.83:0/676407528 wait complete. 2024-04-24T09:16:49.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.262+0000 7f5bb5990740 1 -- start start 2024-04-24T09:16:49.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.262+0000 7f5bb5990740 1 Processor -- start 2024-04-24T09:16:49.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.262+0000 7f5bb5990740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ddddea5950 0x55ddddeb8f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:49.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.262+0000 7f5bb5990740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ddddeacae0 0x55ddddeb14e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:49.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.262+0000 7f5bb5990740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ddddeb0940 0x55ddddeb1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:49.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.262+0000 7f5bb5990740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5bac1b71f0 con 0x55ddddeacae0 2024-04-24T09:16:49.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.262+0000 7f5bb5990740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5bac1b6b20 con 0x55ddddea5950 2024-04-24T09:16:49.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.262+0000 7f5bb5990740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5bac1b3d20 con 0x55ddddeb0940 2024-04-24T09:16:49.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.262+0000 7f5bb5990740 10 client.0 _ll_register_callbacks cb 0x7f5b9800a5e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:16:49.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.263+0000 7f5babe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ddddea5950 0x55ddddeb8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:49.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.263+0000 7f5babe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ddddea5950 0x55ddddeb8f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:43202/0 (socket says 172.21.15.83:43202) 2024-04-24T09:16:49.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.263+0000 7f5babe00640 1 -- 172.21.15.83:0/803117303 learned_addr learned my addr 172.21.15.83:0/803117303 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:16:49.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.263+0000 7f5bab400640 1 --2- 172.21.15.83:0/803117303 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ddddeacae0 0x55ddddeb14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:49.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.263+0000 7f5bb0a00640 1 --2- 172.21.15.83:0/803117303 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ddddeb0940 0x55ddddeb1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:49.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.264+0000 7f5bab400640 1 -- 172.21.15.83:0/803117303 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ddddeb0940 msgr2=0x55ddddeb1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:49.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.264+0000 7f5bab400640 1 --2- 172.21.15.83:0/803117303 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ddddeb0940 0x55ddddeb1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:49.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.264+0000 7f5bab400640 1 -- 172.21.15.83:0/803117303 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ddddea5950 msgr2=0x55ddddeb8f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:49.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.264+0000 7f5bab400640 1 --2- 172.21.15.83:0/803117303 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ddddea5950 0x55ddddeb8f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:49.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.264+0000 7f5bab400640 1 -- 172.21.15.83:0/803117303 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ddddeb2250 con 0x55ddddeacae0 2024-04-24T09:16:49.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.264+0000 7f5bab400640 1 --2- 172.21.15.83:0/803117303 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ddddeacae0 0x55ddddeb14e0 secure :-1 s=READY pgs=2802 cs=0 l=1 rev1=1 crypto rx=0x55ddddeb08b0 tx=0x7f5b980148b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb8a80a615e237c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:49.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.264+0000 7f5b95600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:16:49.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.265+0000 7f5b95600640 1 -- 172.21.15.83:0/803117303 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b98035060 con 0x55ddddeacae0 2024-04-24T09:16:49.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.265+0000 7f5b95600640 1 -- 172.21.15.83:0/803117303 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(132..132 src has 1..132) ==== 9583+0+0 (secure 0 0 0) 0x7f5b980133d0 con 0x55ddddeacae0 2024-04-24T09:16:49.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.265+0000 7f5bb5990740 1 -- 172.21.15.83:0/803117303 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b9800aab0 con 0x55ddddeacae0 2024-04-24T09:16:49.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.265+0000 7f5b95600640 1 -- 172.21.15.83:0/803117303 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f5b9803c050 con 0x55ddddeacae0 2024-04-24T09:16:49.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.265+0000 7f5bb5990740 10 client.24072 fetch_fsmap learned FSMap version 69 2024-04-24T09:16:49.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.265+0000 7f5bb5990740 1 -- 172.21.15.83:0/803117303 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x55ddddf272c0 con 0x55ddddeacae0 2024-04-24T09:16:49.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.266+0000 7f5b95600640 1 -- 172.21.15.83:0/803117303 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5b9800aab0 con 0x55ddddeacae0 2024-04-24T09:16:49.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.266+0000 7f5b95600640 1 -- 172.21.15.83:0/803117303 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 69) ==== 8370+0+0 (secure 0 0 0) 0x7f5b98030910 con 0x55ddddeacae0 2024-04-24T09:16:49.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.266+0000 7f5bb5990740 10 client.24072 fetch_fsmap finished waiting for FSMap version 69 2024-04-24T09:16:49.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.266+0000 7f5bb5990740 10 client.24072 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-24T09:16:49.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.266+0000 7f5bb5990740 20 client.24072 populate_metadata read hostname 'smithi083' 2024-04-24T09:16:49.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.266+0000 7f5bb5990740 1 --2- 172.21.15.83:0/803117303 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55ddddf278c0 0x55ddddf29ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:49.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.266+0000 7f5bb5990740 4 client.24072 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:16:49.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.266+0000 7f5bb5990740 1 -- 172.21.15.83:0/803117303 --> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dddde12a60 con 0x55ddddf278c0 2024-04-24T09:16:49.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.267+0000 7f5babe00640 1 --2- 172.21.15.83:0/803117303 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55ddddf278c0 0x55ddddf29ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:49.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.267+0000 7f5babe00640 1 --2- 172.21.15.83:0/803117303 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55ddddf278c0 0x55ddddf29ce0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6b699993161e53ea server_cookie=e2f3c304d74567bb in_seq=0 out_seq=0 2024-04-24T09:16:49.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.267+0000 7f5b95600640 10 client.24072 ms_handle_connect on v2:172.21.15.138:6834/1909219758 2024-04-24T09:16:49.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.268+0000 7f5b95600640 1 -- 172.21.15.83:0/803117303 <== mds.2 v2:172.21.15.138:6834/1909219758 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dddde12a60 con 0x55ddddf278c0 2024-04-24T09:16:49.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.268+0000 7f5b95600640 10 client.24072 handle_command_reply: tid=0 2024-04-24T09:16:49.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.268+0000 7f5bb5990740 1 client.24072 shutdown 2024-04-24T09:16:49.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.269+0000 7f5bb5990740 1 -- 172.21.15.83:0/803117303 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ddddeacae0 msgr2=0x55ddddeb14e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:49.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.269+0000 7f5bb5990740 1 --2- 172.21.15.83:0/803117303 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ddddeacae0 0x55ddddeb14e0 secure :-1 s=READY pgs=2802 cs=0 l=1 rev1=1 crypto rx=0x55ddddeb08b0 tx=0x7f5b980148b0 comp rx=0 tx=0).stop 2024-04-24T09:16:49.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.269+0000 7f5bb5990740 1 -- 172.21.15.83:0/803117303 shutdown_connections 2024-04-24T09:16:49.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.269+0000 7f5bb5990740 1 --2- 172.21.15.83:0/803117303 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x55ddddf278c0 0x55ddddf29ce0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:49.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.269+0000 7f5bb5990740 1 --2- 172.21.15.83:0/803117303 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ddddeb0940 0x55ddddeb1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:49.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.269+0000 7f5bb5990740 1 --2- 172.21.15.83:0/803117303 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ddddeacae0 0x55ddddeb14e0 unknown :-1 s=CLOSED pgs=2802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:49.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.269+0000 7f5bb5990740 1 --2- 172.21.15.83:0/803117303 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ddddea5950 0x55ddddeb8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:49.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.269+0000 7f5bb5990740 1 -- 172.21.15.83:0/803117303 >> 172.21.15.83:0/803117303 conn(0x55ddddea1c70 msgr2=0x55ddddeaef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:49.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.269+0000 7f5b95600640 0 client.24072 ms_handle_reset on v2:172.21.15.138:6834/1909219758 2024-04-24T09:16:49.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.269+0000 7f5bb5990740 1 -- 172.21.15.83:0/803117303 shutdown_connections 2024-04-24T09:16:49.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.269+0000 7f5bb5990740 1 -- 172.21.15.83:0/803117303 wait complete. 2024-04-24T09:16:49.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.269+0000 7f5bb5990740 20 client.24072 trim_cache size 0 max 16384 2024-04-24T09:16:49.271 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:16:49.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.270+0000 7f5bb2a00640 1 -- 172.21.15.83:0/2893739704 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f5b900a3de0 msgr2=0x7f5b900a62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:49.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.270+0000 7f5bb2a00640 1 --2- 172.21.15.83:0/2893739704 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f5b900a3de0 0x7f5b900a62a0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c000c00 tx=0x7f5b9c0095c0 comp rx=0 tx=0).stop 2024-04-24T09:16:49.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.270+0000 7f5bb2a00640 1 -- 172.21.15.83:0/2893739704 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5bac1130b0 msgr2=0x7f5bac1a37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:49.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.270+0000 7f5bb2a00640 1 --2- 172.21.15.83:0/2893739704 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5bac1130b0 0x7f5bac1a37b0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f5ba4005fd0 tx=0x7f5ba400edd0 comp rx=0 tx=0).stop 2024-04-24T09:16:49.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.270+0000 7f5bb2a00640 1 -- 172.21.15.83:0/2893739704 shutdown_connections 2024-04-24T09:16:49.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.270+0000 7f5bb2a00640 1 --2- 172.21.15.83:0/2893739704 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f5b900a3de0 0x7f5b900a62a0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:49.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.270+0000 7f5bb2a00640 1 --2- 172.21.15.83:0/2893739704 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5bac1130b0 0x7f5bac1a37b0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:49.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.270+0000 7f5bb2a00640 1 --2- 172.21.15.83:0/2893739704 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5bac074bf0 0x7f5bac19f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:49.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.270+0000 7f5bb2a00640 1 --2- 172.21.15.83:0/2893739704 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5bac0742d0 0x7f5bac19eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:49.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.270+0000 7f5bb2a00640 1 -- 172.21.15.83:0/2893739704 >> 172.21.15.83:0/2893739704 conn(0x7f5bac079450 msgr2=0x7f5bac0ff010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:49.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.271+0000 7f5bb2a00640 1 -- 172.21.15.83:0/2893739704 shutdown_connections 2024-04-24T09:16:49.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.271+0000 7f5bb2a00640 1 -- 172.21.15.83:0/2893739704 wait complete. 2024-04-24T09:16:49.292 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T09:16:49.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.572+0000 7f3e6f200640 1 -- 172.21.15.83:0/4244847534 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e68101810 msgr2=0x7f3e68105eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:49.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.572+0000 7f3e6f200640 1 --2- 172.21.15.83:0/4244847534 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e68101810 0x7f3e68105eb0 secure :-1 s=READY pgs=2803 cs=0 l=1 rev1=1 crypto rx=0x7f3e50009860 tx=0x7f3e5002a3d0 comp rx=0 tx=0).stop 2024-04-24T09:16:49.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.573+0000 7f3e6f200640 1 -- 172.21.15.83:0/4244847534 shutdown_connections 2024-04-24T09:16:49.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.573+0000 7f3e6f200640 1 --2- 172.21.15.83:0/4244847534 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3e681063f0 0x7f3e681068a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:49.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.573+0000 7f3e6f200640 1 --2- 172.21.15.83:0/4244847534 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e68101810 0x7f3e68105eb0 unknown :-1 s=CLOSED pgs=2803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:49.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.573+0000 7f3e6f200640 1 --2- 172.21.15.83:0/4244847534 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3e68100ef0 0x7f3e681012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:49.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.573+0000 7f3e6f200640 1 -- 172.21.15.83:0/4244847534 >> 172.21.15.83:0/4244847534 conn(0x7f3e68078060 msgr2=0x7f3e68078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:49.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.573+0000 7f3e6f200640 1 -- 172.21.15.83:0/4244847534 shutdown_connections 2024-04-24T09:16:49.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.573+0000 7f3e6f200640 1 -- 172.21.15.83:0/4244847534 wait complete. 2024-04-24T09:16:49.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.574+0000 7f3e6f200640 1 Processor -- start 2024-04-24T09:16:49.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.574+0000 7f3e6f200640 1 -- start start 2024-04-24T09:16:49.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.575+0000 7f3e6f200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e68100ef0 0x7f3e6819ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:49.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.575+0000 7f3e6f200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3e68101810 0x7f3e6819f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:49.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.575+0000 7f3e6f200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3e681063f0 0x7f3e681a36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:49.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.575+0000 7f3e6f200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3e681188c0 con 0x7f3e68100ef0 2024-04-24T09:16:49.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.575+0000 7f3e6f200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3e68118740 con 0x7f3e68101810 2024-04-24T09:16:49.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.575+0000 7f3e6f200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3e68118a40 con 0x7f3e681063f0 2024-04-24T09:16:49.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.575+0000 7f3e67e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e68100ef0 0x7f3e6819ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:49.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.575+0000 7f3e67400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3e68101810 0x7f3e6819f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:49.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.575+0000 7f3e67e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e68100ef0 0x7f3e6819ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36012/0 (socket says 172.21.15.83:36012) 2024-04-24T09:16:49.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.575+0000 7f3e67e00640 1 -- 172.21.15.83:0/3332936905 learned_addr learned my addr 172.21.15.83:0/3332936905 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:16:49.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.575+0000 7f3e6d200640 1 --2- 172.21.15.83:0/3332936905 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3e681063f0 0x7f3e681a36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:49.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.576+0000 7f3e67e00640 1 -- 172.21.15.83:0/3332936905 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3e681063f0 msgr2=0x7f3e681a36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:49.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.576+0000 7f3e67e00640 1 --2- 172.21.15.83:0/3332936905 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3e681063f0 0x7f3e681a36e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:49.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.576+0000 7f3e67e00640 1 -- 172.21.15.83:0/3332936905 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3e68101810 msgr2=0x7f3e6819f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:49.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.576+0000 7f3e67e00640 1 --2- 172.21.15.83:0/3332936905 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3e68101810 0x7f3e6819f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:49.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.576+0000 7f3e67e00640 1 -- 172.21.15.83:0/3332936905 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e681a3e10 con 0x7f3e68100ef0 2024-04-24T09:16:49.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.576+0000 7f3e67e00640 1 --2- 172.21.15.83:0/3332936905 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e68100ef0 0x7f3e6819ee20 secure :-1 s=READY pgs=2804 cs=0 l=1 rev1=1 crypto rx=0x7f3e5800ebc0 tx=0x7f3e5800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:49.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.577+0000 7f3e64c00640 1 -- 172.21.15.83:0/3332936905 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e58010630 con 0x7f3e68100ef0 2024-04-24T09:16:49.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.577+0000 7f3e64c00640 1 -- 172.21.15.83:0/3332936905 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e58010c80 con 0x7f3e68100ef0 2024-04-24T09:16:49.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.577+0000 7f3e6f200640 1 -- 172.21.15.83:0/3332936905 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e681a4100 con 0x7f3e68100ef0 2024-04-24T09:16:49.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.577+0000 7f3e64c00640 1 -- 172.21.15.83:0/3332936905 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e58018b30 con 0x7f3e68100ef0 2024-04-24T09:16:49.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.577+0000 7f3e6f200640 1 -- 172.21.15.83:0/3332936905 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e681ab990 con 0x7f3e68100ef0 2024-04-24T09:16:49.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.582+0000 7f3e72023740 1 Processor -- start 2024-04-24T09:16:49.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.582+0000 7f3e64c00640 1 -- 172.21.15.83:0/3332936905 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3e58018cd0 con 0x7f3e68100ef0 2024-04-24T09:16:49.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.582+0000 7f3e72023740 1 -- start start 2024-04-24T09:16:49.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.583+0000 7f3e72023740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e4661ee950 0x55e4661eed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:49.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.583+0000 7f3e72023740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e4661ef270 0x55e4661f9400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:49.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.584+0000 7f3e72023740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e4661f9940 0x55e4661fbd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:49.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.584+0000 7f3e72023740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e4661541d0 con 0x55e4661f9940 2024-04-24T09:16:49.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.584+0000 7f3e72023740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e4660fa450 con 0x55e4661ef270 2024-04-24T09:16:49.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.584+0000 7f3e72023740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55e466150190 con 0x55e4661ee950 2024-04-24T09:16:49.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.584+0000 7f3e67400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e4661f9940 0x55e4661fbd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:49.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.584+0000 7f3e67e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e4661ef270 0x55e4661f9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:49.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.584+0000 7f3e6d200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e4661ee950 0x55e4661eed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:49.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.584+0000 7f3e6d200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e4661ee950 0x55e4661eed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:41324/0 (socket says 172.21.15.83:41324) 2024-04-24T09:16:49.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.584+0000 7f3e67400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e4661f9940 0x55e4661fbd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36022/0 (socket says 172.21.15.83:36022) 2024-04-24T09:16:49.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.584+0000 7f3e6d200640 1 -- 172.21.15.83:0/4139710638 learned_addr learned my addr 172.21.15.83:0/4139710638 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:16:49.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.585+0000 7f3e67400640 1 -- 172.21.15.83:0/4139710638 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e4661ee950 msgr2=0x55e4661eed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:49.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.585+0000 7f3e67400640 1 --2- 172.21.15.83:0/4139710638 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e4661ee950 0x55e4661eed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:49.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.585+0000 7f3e67400640 1 -- 172.21.15.83:0/4139710638 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e4661ef270 msgr2=0x55e4661f9400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:49.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.585+0000 7f3e67400640 1 --2- 172.21.15.83:0/4139710638 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e4661ef270 0x55e4661f9400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:49.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.585+0000 7f3e67400640 1 -- 172.21.15.83:0/4139710638 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4661b7310 con 0x55e4661f9940 2024-04-24T09:16:49.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.585+0000 7f3e64c00640 1 --2- 172.21.15.83:0/3332936905 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f3e4c0a3e20 0x7f3e4c0a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:49.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.586+0000 7f3e67400640 1 --2- 172.21.15.83:0/4139710638 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e4661f9940 0x55e4661fbd30 secure :-1 s=READY pgs=2805 cs=0 l=1 rev1=1 crypto rx=0x7f3e50009830 tx=0x7f3e500303a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79efabe5e44027ce server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:49.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.586+0000 7f3e64c00640 1 -- 172.21.15.83:0/3332936905 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 9583+0+0 (secure 0 0 0) 0x7f3e580d1270 con 0x7f3e68100ef0 2024-04-24T09:16:49.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.586+0000 7f3e5d600640 1 -- 172.21.15.83:0/4139710638 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e50030790 con 0x55e4661f9940 2024-04-24T09:16:49.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.586+0000 7f3e5d600640 1 -- 172.21.15.83:0/4139710638 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e50041030 con 0x55e4661f9940 2024-04-24T09:16:49.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.586+0000 7f3e5d600640 1 -- 172.21.15.83:0/4139710638 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e50005330 con 0x55e4661f9940 2024-04-24T09:16:49.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.587+0000 7f3e6d200640 1 --2- 172.21.15.83:0/3332936905 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f3e4c0a3e20 0x7f3e4c0a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:49.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.587+0000 7f3e72023740 1 -- 172.21.15.83:0/4139710638 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e4661f9940 msgr2=0x55e4661fbd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:49.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.587+0000 7f3e72023740 1 --2- 172.21.15.83:0/4139710638 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e4661f9940 0x55e4661fbd30 secure :-1 s=READY pgs=2805 cs=0 l=1 rev1=1 crypto rx=0x7f3e50009830 tx=0x7f3e500303a0 comp rx=0 tx=0).stop 2024-04-24T09:16:49.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.588+0000 7f3e72023740 1 -- 172.21.15.83:0/4139710638 shutdown_connections 2024-04-24T09:16:49.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.588+0000 7f3e72023740 1 --2- 172.21.15.83:0/4139710638 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e4661f9940 0x55e4661fbd30 unknown :-1 s=CLOSED pgs=2805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:49.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.588+0000 7f3e72023740 1 --2- 172.21.15.83:0/4139710638 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e4661ef270 0x55e4661f9400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:49.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.588+0000 7f3e72023740 1 --2- 172.21.15.83:0/4139710638 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e4661ee950 0x55e4661eed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:49.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.588+0000 7f3e72023740 1 -- 172.21.15.83:0/4139710638 >> 172.21.15.83:0/4139710638 conn(0x55e4661eac70 msgr2=0x55e4661eb050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:49.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.588+0000 7f3e6d200640 1 --2- 172.21.15.83:0/3332936905 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f3e4c0a3e20 0x7f3e4c0a62e0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f3e600096c0 tx=0x7f3e60009260 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:49.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.590+0000 7f3e72023740 1 -- 172.21.15.83:0/4139710638 shutdown_connections 2024-04-24T09:16:49.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.590+0000 7f3e72023740 1 -- 172.21.15.83:0/4139710638 wait complete. 2024-04-24T09:16:49.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.591+0000 7f3e72023740 1 -- start start 2024-04-24T09:16:49.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.591+0000 7f3e72023740 1 Processor -- start 2024-04-24T09:16:49.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.592+0000 7f3e72023740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e4661ee950 0x55e4661f9340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:49.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.592+0000 7f3e72023740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e4661ef270 0x55e4661f66a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:49.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.592+0000 7f3e72023740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e4661f9940 0x55e4661f6d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:49.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.592+0000 7f3e72023740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e4661fc770 con 0x55e4661f9940 2024-04-24T09:16:49.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.592+0000 7f3e72023740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e46610d810 con 0x55e4661ee950 2024-04-24T09:16:49.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.592+0000 7f3e72023740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55e4661fc8f0 con 0x55e4661ef270 2024-04-24T09:16:49.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.592+0000 7f3e72023740 10 client.0 _ll_register_callbacks cb 0x55e4660db140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:16:49.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.592+0000 7f3e6d200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e4661f9940 0x55e4661f6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:49.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.593+0000 7f3e67400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e4661ef270 0x55e4661f66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:49.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.593+0000 7f3e67400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e4661ef270 0x55e4661f66a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:41330/0 (socket says 172.21.15.83:41330) 2024-04-24T09:16:49.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.593+0000 7f3e67400640 1 -- 172.21.15.83:0/806066552 learned_addr learned my addr 172.21.15.83:0/806066552 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:16:49.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.593+0000 7f3e67e00640 1 --2- 172.21.15.83:0/806066552 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e4661ee950 0x55e4661f9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:49.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.594+0000 7f3e67400640 1 -- 172.21.15.83:0/806066552 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e4661ee950 msgr2=0x55e4661f9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:49.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.594+0000 7f3e67400640 1 --2- 172.21.15.83:0/806066552 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e4661ee950 0x55e4661f9340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:49.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.594+0000 7f3e67400640 1 -- 172.21.15.83:0/806066552 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e4661f9940 msgr2=0x55e4661f6d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:49.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.594+0000 7f3e67400640 1 --2- 172.21.15.83:0/806066552 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e4661f9940 0x55e4661f6d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:49.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.594+0000 7f3e67400640 1 -- 172.21.15.83:0/806066552 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e46611e3f0 con 0x55e4661ef270 2024-04-24T09:16:49.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.594+0000 7f3e6d200640 1 --2- 172.21.15.83:0/806066552 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e4661f9940 0x55e4661f6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:16:49.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.594+0000 7f3e67400640 1 --2- 172.21.15.83:0/806066552 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e4661ef270 0x55e4661f66a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3e50030f40 tx=0x7f3e500095b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ebc83fdea09515ce server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:49.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.594+0000 7f3e56000640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-04-24T09:16:49.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.595+0000 7f3e56000640 1 -- 172.21.15.83:0/806066552 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e50030c00 con 0x55e4661ef270 2024-04-24T09:16:49.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.595+0000 7f3e56000640 1 -- 172.21.15.83:0/806066552 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(132..132 src has 1..132) ==== 9583+0+0 (secure 0 0 0) 0x7f3e50004030 con 0x55e4661ef270 2024-04-24T09:16:49.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.595+0000 7f3e72023740 1 -- 172.21.15.83:0/806066552 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e50041030 con 0x55e4661ef270 2024-04-24T09:16:49.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.595+0000 7f3e56000640 1 -- 172.21.15.83:0/806066552 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f3e50004230 con 0x55e4661ef270 2024-04-24T09:16:49.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.595+0000 7f3e72023740 10 client.30832 fetch_fsmap learned FSMap version 69 2024-04-24T09:16:49.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.595+0000 7f3e72023740 1 -- 172.21.15.83:0/806066552 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x55e465e663f0 con 0x55e4661ef270 2024-04-24T09:16:49.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.596+0000 7f3e56000640 1 -- 172.21.15.83:0/806066552 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e50041030 con 0x55e4661ef270 2024-04-24T09:16:49.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.597+0000 7f3e56000640 1 -- 172.21.15.83:0/806066552 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 69) ==== 8370+0+0 (secure 0 0 0) 0x7f3e50004a00 con 0x55e4661ef270 2024-04-24T09:16:49.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.597+0000 7f3e72023740 10 client.30832 fetch_fsmap finished waiting for FSMap version 69 2024-04-24T09:16:49.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.597+0000 7f3e72023740 10 client.30832 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-24T09:16:49.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.597+0000 7f3e72023740 20 client.30832 populate_metadata read hostname 'smithi083' 2024-04-24T09:16:49.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.598+0000 7f3e72023740 1 --2- 172.21.15.83:0/806066552 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x55e466209950 0x55e46620bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:49.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.598+0000 7f3e72023740 4 client.30832 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:16:49.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.598+0000 7f3e72023740 1 -- 172.21.15.83:0/806066552 --> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e466161bf0 con 0x55e466209950 2024-04-24T09:16:49.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.598+0000 7f3e67e00640 1 --2- 172.21.15.83:0/806066552 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x55e466209950 0x55e46620bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:49.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.599+0000 7f3e67e00640 1 --2- 172.21.15.83:0/806066552 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x55e466209950 0x55e46620bd40 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7d138e6a298972d4 server_cookie=4644a543f06ae055 in_seq=0 out_seq=0 2024-04-24T09:16:49.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.599+0000 7f3e56000640 10 client.30832 ms_handle_connect on v2:172.21.15.138:6836/900566754 2024-04-24T09:16:49.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.600+0000 7f3e56000640 1 -- 172.21.15.83:0/806066552 <== mds.1 v2:172.21.15.138:6836/900566754 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e466161bf0 con 0x55e466209950 2024-04-24T09:16:49.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.600+0000 7f3e56000640 10 client.30832 handle_command_reply: tid=0 2024-04-24T09:16:49.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.601+0000 7f3e72023740 1 client.30832 shutdown 2024-04-24T09:16:49.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.601+0000 7f3e72023740 1 -- 172.21.15.83:0/806066552 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e4661ef270 msgr2=0x55e4661f66a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:49.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.601+0000 7f3e72023740 1 --2- 172.21.15.83:0/806066552 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e4661ef270 0x55e4661f66a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3e50030f40 tx=0x7f3e500095b0 comp rx=0 tx=0).stop 2024-04-24T09:16:49.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.601+0000 7f3e72023740 1 -- 172.21.15.83:0/806066552 shutdown_connections 2024-04-24T09:16:49.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.601+0000 7f3e72023740 1 --2- 172.21.15.83:0/806066552 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e4661f9940 0x55e4661f6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:49.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.601+0000 7f3e72023740 1 --2- 172.21.15.83:0/806066552 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x55e466209950 0x55e46620bd40 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:49.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.601+0000 7f3e72023740 1 --2- 172.21.15.83:0/806066552 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e4661ef270 0x55e4661f66a0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:49.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.601+0000 7f3e72023740 1 --2- 172.21.15.83:0/806066552 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e4661ee950 0x55e4661f9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:49.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.602+0000 7f3e72023740 1 -- 172.21.15.83:0/806066552 >> 172.21.15.83:0/806066552 conn(0x55e4661eac70 msgr2=0x55e4661f0a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:49.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.602+0000 7f3e56000640 0 client.30832 ms_handle_reset on v2:172.21.15.138:6836/900566754 2024-04-24T09:16:49.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.602+0000 7f3e72023740 1 -- 172.21.15.83:0/806066552 shutdown_connections 2024-04-24T09:16:49.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.602+0000 7f3e72023740 1 -- 172.21.15.83:0/806066552 wait complete. 2024-04-24T09:16:49.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.602+0000 7f3e72023740 20 client.30832 trim_cache size 0 max 16384 2024-04-24T09:16:49.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.627+0000 7f3e72023740 1 Processor -- start 2024-04-24T09:16:49.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.628+0000 7f3e72023740 1 -- start start 2024-04-24T09:16:49.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.628+0000 7f3e72023740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e4661ee950 0x55e4661f1190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:49.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.628+0000 7f3e72023740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e4661ef270 0x55e466205630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:49.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.628+0000 7f3e72023740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e4661f9940 0x55e4661f5e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:49.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.628+0000 7f3e72023740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3e681b5c70 con 0x55e4661f9940 2024-04-24T09:16:49.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.628+0000 7f3e72023740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3e681b7810 con 0x55e4661ee950 2024-04-24T09:16:49.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.628+0000 7f3e72023740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3e681b6340 con 0x55e4661ef270 2024-04-24T09:16:49.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.628+0000 7f3e6d200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e4661f9940 0x55e4661f5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:49.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.629+0000 7f3e6d200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e4661f9940 0x55e4661f5e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36040/0 (socket says 172.21.15.83:36040) 2024-04-24T09:16:49.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.629+0000 7f3e6d200640 1 -- 172.21.15.83:0/2530908872 learned_addr learned my addr 172.21.15.83:0/2530908872 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:16:49.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.629+0000 7f3e67e00640 1 --2- 172.21.15.83:0/2530908872 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e4661ee950 0x55e4661f1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:49.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.629+0000 7f3e67400640 1 --2- 172.21.15.83:0/2530908872 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e4661ef270 0x55e466205630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:49.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.629+0000 7f3e6d200640 1 -- 172.21.15.83:0/2530908872 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e4661ef270 msgr2=0x55e466205630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:49.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.629+0000 7f3e6d200640 1 --2- 172.21.15.83:0/2530908872 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e4661ef270 0x55e466205630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:49.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.629+0000 7f3e6d200640 1 -- 172.21.15.83:0/2530908872 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e4661ee950 msgr2=0x55e4661f1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:49.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.629+0000 7f3e6d200640 1 --2- 172.21.15.83:0/2530908872 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e4661ee950 0x55e4661f1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:49.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.629+0000 7f3e6d200640 1 -- 172.21.15.83:0/2530908872 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e465f307d0 con 0x55e4661f9940 2024-04-24T09:16:49.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.630+0000 7f3e6d200640 1 --2- 172.21.15.83:0/2530908872 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e4661f9940 0x55e4661f5e50 secure :-1 s=READY pgs=2806 cs=0 l=1 rev1=1 crypto rx=0x7f3e60009ec0 tx=0x7f3e60015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cd6f106d37aad72 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:49.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.630+0000 7f3e5cc00640 1 -- 172.21.15.83:0/2530908872 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e60010db0 con 0x55e4661f9940 2024-04-24T09:16:49.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.630+0000 7f3e5cc00640 1 -- 172.21.15.83:0/2530908872 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e6001c8e0 con 0x55e4661f9940 2024-04-24T09:16:49.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.630+0000 7f3e5cc00640 1 -- 172.21.15.83:0/2530908872 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e6001f9e0 con 0x55e4661f9940 2024-04-24T09:16:49.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.631+0000 7f3e72023740 1 -- 172.21.15.83:0/2530908872 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e4661f9940 msgr2=0x55e4661f5e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:49.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.631+0000 7f3e72023740 1 --2- 172.21.15.83:0/2530908872 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e4661f9940 0x55e4661f5e50 secure :-1 s=READY pgs=2806 cs=0 l=1 rev1=1 crypto rx=0x7f3e60009ec0 tx=0x7f3e60015000 comp rx=0 tx=0).stop 2024-04-24T09:16:49.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.632+0000 7f3e72023740 1 -- 172.21.15.83:0/2530908872 shutdown_connections 2024-04-24T09:16:49.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.632+0000 7f3e72023740 1 --2- 172.21.15.83:0/2530908872 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e4661f9940 0x55e4661f5e50 unknown :-1 s=CLOSED pgs=2806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:49.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.632+0000 7f3e72023740 1 --2- 172.21.15.83:0/2530908872 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e4661ef270 0x55e466205630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:49.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.632+0000 7f3e72023740 1 --2- 172.21.15.83:0/2530908872 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e4661ee950 0x55e4661f1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:49.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.632+0000 7f3e72023740 1 -- 172.21.15.83:0/2530908872 >> 172.21.15.83:0/2530908872 conn(0x55e4661eac70 msgr2=0x55e4661eb050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:49.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.632+0000 7f3e72023740 1 -- 172.21.15.83:0/2530908872 shutdown_connections 2024-04-24T09:16:49.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.632+0000 7f3e72023740 1 -- 172.21.15.83:0/2530908872 wait complete. 2024-04-24T09:16:49.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.632+0000 7f3e72023740 1 -- start start 2024-04-24T09:16:49.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.633+0000 7f3e72023740 1 Processor -- start 2024-04-24T09:16:49.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.633+0000 7f3e72023740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e4661ee950 0x55e4661fa510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:49.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.633+0000 7f3e72023740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e4661ef270 0x55e4661faa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:49.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.633+0000 7f3e72023740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e4661f9940 0x55e466200b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:49.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.633+0000 7f3e72023740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3e681b7140 con 0x55e4661ef270 2024-04-24T09:16:49.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.633+0000 7f3e72023740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3e681b6a70 con 0x55e4661f9940 2024-04-24T09:16:49.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.633+0000 7f3e72023740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3e681b3c70 con 0x55e4661ee950 2024-04-24T09:16:49.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.633+0000 7f3e72023740 10 client.0 _ll_register_callbacks cb 0x7f3e500096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:16:49.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.634+0000 7f3e6d200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e4661f9940 0x55e466200b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:49.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.634+0000 7f3e67400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e4661ef270 0x55e4661faa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:49.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.634+0000 7f3e6d200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e4661f9940 0x55e466200b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:43266/0 (socket says 172.21.15.83:43266) 2024-04-24T09:16:49.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.634+0000 7f3e6d200640 1 -- 172.21.15.83:0/2496702004 learned_addr learned my addr 172.21.15.83:0/2496702004 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:16:49.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.634+0000 7f3e67e00640 1 --2- 172.21.15.83:0/2496702004 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e4661ee950 0x55e4661fa510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:49.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.635+0000 7f3e67400640 1 -- 172.21.15.83:0/2496702004 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e4661ee950 msgr2=0x55e4661fa510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:49.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.635+0000 7f3e67400640 1 --2- 172.21.15.83:0/2496702004 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e4661ee950 0x55e4661fa510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:49.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.635+0000 7f3e67400640 1 -- 172.21.15.83:0/2496702004 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e4661f9940 msgr2=0x55e466200b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:49.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.635+0000 7f3e67400640 1 --2- 172.21.15.83:0/2496702004 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e4661f9940 0x55e466200b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:49.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.635+0000 7f3e67400640 1 -- 172.21.15.83:0/2496702004 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e466201100 con 0x55e4661ef270 2024-04-24T09:16:49.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.635+0000 7f3e67e00640 1 --2- 172.21.15.83:0/2496702004 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e4661ee950 0x55e4661fa510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:16:49.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.635+0000 7f3e67400640 1 --2- 172.21.15.83:0/2496702004 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e4661ef270 0x55e4661faa50 secure :-1 s=READY pgs=2807 cs=0 l=1 rev1=1 crypto rx=0x7f3e50004e20 tx=0x7f3e500058c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddd28077d699feb6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:49.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.635+0000 7f3e56000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:16:49.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.636+0000 7f3e56000640 1 -- 172.21.15.83:0/2496702004 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e500048d0 con 0x55e4661ef270 2024-04-24T09:16:49.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.636+0000 7f3e56000640 1 -- 172.21.15.83:0/2496702004 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(132..132 src has 1..132) ==== 9583+0+0 (secure 0 0 0) 0x7f3e50004a70 con 0x55e4661ef270 2024-04-24T09:16:49.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.636+0000 7f3e72023740 1 -- 172.21.15.83:0/2496702004 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e6001c8e0 con 0x55e4661ef270 2024-04-24T09:16:49.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.636+0000 7f3e56000640 1 -- 172.21.15.83:0/2496702004 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f3e50005010 con 0x55e4661ef270 2024-04-24T09:16:49.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.636+0000 7f3e72023740 10 client.24090 fetch_fsmap learned FSMap version 69 2024-04-24T09:16:49.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.636+0000 7f3e72023740 1 -- 172.21.15.83:0/2496702004 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x55e4662023a0 con 0x55e4661ef270 2024-04-24T09:16:49.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.637+0000 7f3e56000640 1 -- 172.21.15.83:0/2496702004 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e6001c8e0 con 0x55e4661ef270 2024-04-24T09:16:49.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.637+0000 7f3e56000640 1 -- 172.21.15.83:0/2496702004 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 69) ==== 8370+0+0 (secure 0 0 0) 0x7f3e5005e9e0 con 0x55e4661ef270 2024-04-24T09:16:49.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.637+0000 7f3e72023740 10 client.24090 fetch_fsmap finished waiting for FSMap version 69 2024-04-24T09:16:49.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.637+0000 7f3e72023740 10 client.24090 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-24T09:16:49.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.637+0000 7f3e72023740 20 client.24090 populate_metadata read hostname 'smithi083' 2024-04-24T09:16:49.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.637+0000 7f3e72023740 1 --2- 172.21.15.83:0/2496702004 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x55e46626ff60 0x55e466272380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:49.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.637+0000 7f3e72023740 4 client.24090 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:16:49.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.637+0000 7f3e72023740 1 -- 172.21.15.83:0/2496702004 --> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e46615ba60 con 0x55e46626ff60 2024-04-24T09:16:49.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.638+0000 7f3e67e00640 1 --2- 172.21.15.83:0/2496702004 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x55e46626ff60 0x55e466272380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:49.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.639+0000 7f3e67e00640 1 --2- 172.21.15.83:0/2496702004 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x55e46626ff60 0x55e466272380 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4f16733759740c44 server_cookie=95d7152060002f6 in_seq=0 out_seq=0 2024-04-24T09:16:49.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.639+0000 7f3e56000640 10 client.24090 ms_handle_connect on v2:172.21.15.138:6836/900566754 2024-04-24T09:16:49.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.639+0000 7f3e56000640 1 -- 172.21.15.83:0/2496702004 <== mds.1 v2:172.21.15.138:6836/900566754 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e46615ba60 con 0x55e46626ff60 2024-04-24T09:16:49.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.639+0000 7f3e56000640 10 client.24090 handle_command_reply: tid=0 2024-04-24T09:16:49.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.639+0000 7f3e72023740 1 client.24090 shutdown 2024-04-24T09:16:49.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.639+0000 7f3e72023740 1 -- 172.21.15.83:0/2496702004 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e4661ef270 msgr2=0x55e4661faa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:49.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.639+0000 7f3e72023740 1 --2- 172.21.15.83:0/2496702004 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e4661ef270 0x55e4661faa50 secure :-1 s=READY pgs=2807 cs=0 l=1 rev1=1 crypto rx=0x7f3e50004e20 tx=0x7f3e500058c0 comp rx=0 tx=0).stop 2024-04-24T09:16:49.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.640+0000 7f3e72023740 1 -- 172.21.15.83:0/2496702004 shutdown_connections 2024-04-24T09:16:49.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.640+0000 7f3e72023740 1 --2- 172.21.15.83:0/2496702004 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e4661f9940 0x55e466200b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:49.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.640+0000 7f3e72023740 1 --2- 172.21.15.83:0/2496702004 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e4661ef270 0x55e4661faa50 unknown :-1 s=CLOSED pgs=2807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:49.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.640+0000 7f3e72023740 1 --2- 172.21.15.83:0/2496702004 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x55e46626ff60 0x55e466272380 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:49.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.640+0000 7f3e72023740 1 --2- 172.21.15.83:0/2496702004 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e4661ee950 0x55e4661fa510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:49.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.640+0000 7f3e72023740 1 -- 172.21.15.83:0/2496702004 >> 172.21.15.83:0/2496702004 conn(0x55e4661eac70 msgr2=0x55e4661e9560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:49.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.640+0000 7f3e56000640 0 client.24090 ms_handle_reset on v2:172.21.15.138:6836/900566754 2024-04-24T09:16:49.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.640+0000 7f3e72023740 1 -- 172.21.15.83:0/2496702004 shutdown_connections 2024-04-24T09:16:49.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.640+0000 7f3e72023740 1 -- 172.21.15.83:0/2496702004 wait complete. 2024-04-24T09:16:49.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.640+0000 7f3e72023740 20 client.24090 trim_cache size 0 max 16384 2024-04-24T09:16:49.642 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:16:49.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.641+0000 7f3e6f200640 1 -- 172.21.15.83:0/3332936905 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f3e4c0a3e20 msgr2=0x7f3e4c0a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:49.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.641+0000 7f3e6f200640 1 --2- 172.21.15.83:0/3332936905 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f3e4c0a3e20 0x7f3e4c0a62e0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f3e600096c0 tx=0x7f3e60009260 comp rx=0 tx=0).stop 2024-04-24T09:16:49.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.641+0000 7f3e6f200640 1 -- 172.21.15.83:0/3332936905 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e68100ef0 msgr2=0x7f3e6819ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:49.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.641+0000 7f3e6f200640 1 --2- 172.21.15.83:0/3332936905 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e68100ef0 0x7f3e6819ee20 secure :-1 s=READY pgs=2804 cs=0 l=1 rev1=1 crypto rx=0x7f3e5800ebc0 tx=0x7f3e5800c6a0 comp rx=0 tx=0).stop 2024-04-24T09:16:49.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.642+0000 7f3e6f200640 1 -- 172.21.15.83:0/3332936905 shutdown_connections 2024-04-24T09:16:49.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.642+0000 7f3e6f200640 1 --2- 172.21.15.83:0/3332936905 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f3e4c0a3e20 0x7f3e4c0a62e0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:49.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.642+0000 7f3e6f200640 1 --2- 172.21.15.83:0/3332936905 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3e681063f0 0x7f3e681a36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:49.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.642+0000 7f3e6f200640 1 --2- 172.21.15.83:0/3332936905 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3e68101810 0x7f3e6819f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:49.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.642+0000 7f3e6f200640 1 --2- 172.21.15.83:0/3332936905 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e68100ef0 0x7f3e6819ee20 unknown :-1 s=CLOSED pgs=2804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:49.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.642+0000 7f3e6f200640 1 -- 172.21.15.83:0/3332936905 >> 172.21.15.83:0/3332936905 conn(0x7f3e68078060 msgr2=0x7f3e68103e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:49.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.642+0000 7f3e6f200640 1 -- 172.21.15.83:0/3332936905 shutdown_connections 2024-04-24T09:16:49.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.642+0000 7f3e6f200640 1 -- 172.21.15.83:0/3332936905 wait complete. 2024-04-24T09:16:49.668 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T09:16:49.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.943+0000 7ffabd800640 1 -- 172.21.15.83:0/2014627688 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffab8103060 msgr2=0x7ffab8103440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:49.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.943+0000 7ffabd800640 1 --2- 172.21.15.83:0/2014627688 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffab8103060 0x7ffab8103440 secure :-1 s=READY pgs=2808 cs=0 l=1 rev1=1 crypto rx=0x7ffaa8009900 tx=0x7ffaa802a490 comp rx=0 tx=0).stop 2024-04-24T09:16:49.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.943+0000 7ffabd800640 1 -- 172.21.15.83:0/2014627688 shutdown_connections 2024-04-24T09:16:49.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.943+0000 7ffabd800640 1 --2- 172.21.15.83:0/2014627688 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffab810c480 0x7ffab810e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:49.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.943+0000 7ffabd800640 1 --2- 172.21.15.83:0/2014627688 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffab8103980 0x7ffab810be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:49.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.943+0000 7ffabd800640 1 --2- 172.21.15.83:0/2014627688 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffab8103060 0x7ffab8103440 unknown :-1 s=CLOSED pgs=2808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:49.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.943+0000 7ffabd800640 1 -- 172.21.15.83:0/2014627688 >> 172.21.15.83:0/2014627688 conn(0x7ffab80fc880 msgr2=0x7ffab80feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:49.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.943+0000 7ffabd800640 1 -- 172.21.15.83:0/2014627688 shutdown_connections 2024-04-24T09:16:49.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.943+0000 7ffabd800640 1 -- 172.21.15.83:0/2014627688 wait complete. 2024-04-24T09:16:49.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.945+0000 7ffabd800640 1 Processor -- start 2024-04-24T09:16:49.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.945+0000 7ffabd800640 1 -- start start 2024-04-24T09:16:49.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.945+0000 7ffabd800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffab8103060 0x7ffab81a4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:49.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.945+0000 7ffabd800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffab8103980 0x7ffab81a4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:49.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.945+0000 7ffabd800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffab810c480 0x7ffab810aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:49.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.945+0000 7ffabd800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ffab81121c0 con 0x7ffab810c480 2024-04-24T09:16:49.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.945+0000 7ffabd800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ffab8112040 con 0x7ffab8103980 2024-04-24T09:16:49.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.945+0000 7ffabd800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ffab8112340 con 0x7ffab8103060 2024-04-24T09:16:49.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.946+0000 7ffab7400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffab8103980 0x7ffab81a4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:49.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.946+0000 7ffabce00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffab810c480 0x7ffab810aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:49.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.946+0000 7ffab7400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffab8103980 0x7ffab81a4e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:43290/0 (socket says 172.21.15.83:43290) 2024-04-24T09:16:49.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.946+0000 7ffab7400640 1 -- 172.21.15.83:0/2062003159 learned_addr learned my addr 172.21.15.83:0/2062003159 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:16:49.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.946+0000 7ffab7e00640 1 --2- 172.21.15.83:0/2062003159 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffab8103060 0x7ffab81a4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:49.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.947+0000 7ffab7400640 1 -- 172.21.15.83:0/2062003159 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffab8103060 msgr2=0x7ffab81a4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:49.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.947+0000 7ffab7400640 1 --2- 172.21.15.83:0/2062003159 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffab8103060 0x7ffab81a4940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:49.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.947+0000 7ffab7400640 1 -- 172.21.15.83:0/2062003159 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffab810c480 msgr2=0x7ffab810aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:49.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.947+0000 7ffab7400640 1 --2- 172.21.15.83:0/2062003159 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffab810c480 0x7ffab810aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:49.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.947+0000 7ffab7400640 1 -- 172.21.15.83:0/2062003159 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffab810b770 con 0x7ffab8103980 2024-04-24T09:16:49.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.947+0000 7ffab7400640 1 --2- 172.21.15.83:0/2062003159 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffab8103980 0x7ffab81a4e80 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7ffaa400d740 tx=0x7ffaa400dc00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:49.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.947+0000 7ffab4c00640 1 -- 172.21.15.83:0/2062003159 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaa4017070 con 0x7ffab8103980 2024-04-24T09:16:49.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.948+0000 7ffabd800640 1 -- 172.21.15.83:0/2062003159 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffab810ba60 con 0x7ffab8103980 2024-04-24T09:16:49.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.948+0000 7ffab4c00640 1 -- 172.21.15.83:0/2062003159 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffaa400de50 con 0x7ffab8103980 2024-04-24T09:16:49.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.948+0000 7ffabd800640 1 -- 172.21.15.83:0/2062003159 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffab81ab800 con 0x7ffab8103980 2024-04-24T09:16:49.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.949+0000 7ffab4c00640 1 -- 172.21.15.83:0/2062003159 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaa400f620 con 0x7ffab8103980 2024-04-24T09:16:49.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.953+0000 7ffac1d80740 1 Processor -- start 2024-04-24T09:16:49.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.953+0000 7ffac1d80740 1 -- start start 2024-04-24T09:16:49.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.954+0000 7ffac1d80740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0bdd3cb40 0x55f0bdd3cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:49.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.954+0000 7ffac1d80740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0bdd3d460 0x55f0bdd475f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:49.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.954+0000 7ffac1d80740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0bdd47b30 0x55f0bdd49f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:49.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.954+0000 7ffac1d80740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55f0bdca21d0 con 0x55f0bdd3cb40 2024-04-24T09:16:49.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.954+0000 7ffac1d80740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f0bdc48450 con 0x55f0bdd3d460 2024-04-24T09:16:49.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.954+0000 7ffac1d80740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55f0bdc9e190 con 0x55f0bdd47b30 2024-04-24T09:16:49.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.954+0000 7ffab7e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0bdd3d460 0x55f0bdd475f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:49.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.954+0000 7ffab7e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0bdd3d460 0x55f0bdd475f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:43302/0 (socket says 172.21.15.83:43302) 2024-04-24T09:16:49.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.954+0000 7ffab7e00640 1 -- 172.21.15.83:0/1045221328 learned_addr learned my addr 172.21.15.83:0/1045221328 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:16:49.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.954+0000 7ffab4c00640 1 -- 172.21.15.83:0/2062003159 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ffaa4009c30 con 0x7ffab8103980 2024-04-24T09:16:49.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.955+0000 7ffabce00640 1 --2- 172.21.15.83:0/1045221328 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0bdd3cb40 0x55f0bdd3cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:49.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.955+0000 7ffab7e00640 1 -- 172.21.15.83:0/1045221328 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0bdd47b30 msgr2=0x55f0bdd49f20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-24T09:16:49.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.955+0000 7ffab7e00640 1 --2- 172.21.15.83:0/1045221328 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0bdd47b30 0x55f0bdd49f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:49.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.955+0000 7ffab7e00640 1 -- 172.21.15.83:0/1045221328 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0bdd3cb40 msgr2=0x55f0bdd3cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:49.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.955+0000 7ffab7e00640 1 --2- 172.21.15.83:0/1045221328 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0bdd3cb40 0x55f0bdd3cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:49.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.955+0000 7ffab7e00640 1 -- 172.21.15.83:0/1045221328 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0bdd05310 con 0x55f0bdd3d460 2024-04-24T09:16:49.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.955+0000 7ffabce00640 1 --2- 172.21.15.83:0/1045221328 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0bdd3cb40 0x55f0bdd3cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:16:49.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.955+0000 7ffab7e00640 1 --2- 172.21.15.83:0/1045221328 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0bdd3d460 0x55f0bdd475f0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7ffaa8009230 tx=0x7ffaa8004390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4c4b22c280b6f74 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:49.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.956+0000 7ffaad600640 1 -- 172.21.15.83:0/1045221328 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaa8035030 con 0x55f0bdd3d460 2024-04-24T09:16:49.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.956+0000 7ffaad600640 1 -- 172.21.15.83:0/1045221328 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffaa80046c0 con 0x55f0bdd3d460 2024-04-24T09:16:49.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.956+0000 7ffaad600640 1 -- 172.21.15.83:0/1045221328 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaa8005d50 con 0x55f0bdd3d460 2024-04-24T09:16:49.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.957+0000 7ffab4c00640 1 --2- 172.21.15.83:0/2062003159 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ffa9c0a3d20 0x7ffa9c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:49.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.957+0000 7ffac1d80740 1 -- 172.21.15.83:0/1045221328 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0bdd3d460 msgr2=0x55f0bdd475f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:49.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.957+0000 7ffac1d80740 1 --2- 172.21.15.83:0/1045221328 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0bdd3d460 0x55f0bdd475f0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7ffaa8009230 tx=0x7ffaa8004390 comp rx=0 tx=0).stop 2024-04-24T09:16:49.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.957+0000 7ffab4c00640 1 -- 172.21.15.83:0/2062003159 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 9583+0+0 (secure 0 0 0) 0x7ffaa40d0100 con 0x7ffab8103980 2024-04-24T09:16:49.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.958+0000 7ffabce00640 1 --2- 172.21.15.83:0/2062003159 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ffa9c0a3d20 0x7ffa9c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:49.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.959+0000 7ffac1d80740 1 -- 172.21.15.83:0/1045221328 shutdown_connections 2024-04-24T09:16:49.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.959+0000 7ffac1d80740 1 --2- 172.21.15.83:0/1045221328 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0bdd47b30 0x55f0bdd49f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:49.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.959+0000 7ffac1d80740 1 --2- 172.21.15.83:0/1045221328 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0bdd3d460 0x55f0bdd475f0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:49.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.959+0000 7ffac1d80740 1 --2- 172.21.15.83:0/1045221328 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0bdd3cb40 0x55f0bdd3cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:49.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.959+0000 7ffac1d80740 1 -- 172.21.15.83:0/1045221328 >> 172.21.15.83:0/1045221328 conn(0x55f0bdd38e60 msgr2=0x55f0bdd39240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:49.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.959+0000 7ffac1d80740 1 -- 172.21.15.83:0/1045221328 shutdown_connections 2024-04-24T09:16:49.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.960+0000 7ffabce00640 1 --2- 172.21.15.83:0/2062003159 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ffa9c0a3d20 0x7ffa9c0a61e0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7ffab000a510 tx=0x7ffab0009650 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:49.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.960+0000 7ffac1d80740 1 -- 172.21.15.83:0/1045221328 wait complete. 2024-04-24T09:16:49.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.961+0000 7ffac1d80740 1 -- start start 2024-04-24T09:16:49.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.961+0000 7ffac1d80740 1 Processor -- start 2024-04-24T09:16:49.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.962+0000 7ffac1d80740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0bdd3cb40 0x55f0bdd44920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:49.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.962+0000 7ffac1d80740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0bdd3d460 0x55f0bdd44e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:49.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.962+0000 7ffac1d80740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0bdd47b30 0x55f0bdd453a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:49.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.962+0000 7ffac1d80740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55f0bdd4a960 con 0x55f0bdd3cb40 2024-04-24T09:16:49.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.962+0000 7ffac1d80740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f0bdc5b810 con 0x55f0bdd3d460 2024-04-24T09:16:49.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.962+0000 7ffac1d80740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55f0bdd4aae0 con 0x55f0bdd47b30 2024-04-24T09:16:49.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.962+0000 7ffac1d80740 10 client.0 _ll_register_callbacks cb 0x55f0bdc29140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:16:49.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.962+0000 7ffab7400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0bdd3d460 0x55f0bdd44e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:49.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.962+0000 7ffab7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0bdd3cb40 0x55f0bdd44920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:49.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.962+0000 7ffab7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0bdd3cb40 0x55f0bdd44920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36084/0 (socket says 172.21.15.83:36084) 2024-04-24T09:16:49.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.962+0000 7ffab7e00640 1 -- 172.21.15.83:0/41496375 learned_addr learned my addr 172.21.15.83:0/41496375 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:16:49.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.963+0000 7ffabce00640 1 --2- 172.21.15.83:0/41496375 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0bdd47b30 0x55f0bdd453a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:49.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.963+0000 7ffab7e00640 1 -- 172.21.15.83:0/41496375 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0bdd47b30 msgr2=0x55f0bdd453a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:49.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.963+0000 7ffab7e00640 1 --2- 172.21.15.83:0/41496375 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0bdd47b30 0x55f0bdd453a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:49.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.963+0000 7ffab7e00640 1 -- 172.21.15.83:0/41496375 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0bdd3d460 msgr2=0x55f0bdd44e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:49.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.963+0000 7ffab7e00640 1 --2- 172.21.15.83:0/41496375 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0bdd3d460 0x55f0bdd44e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:49.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.963+0000 7ffab7e00640 1 -- 172.21.15.83:0/41496375 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0bdc6c3f0 con 0x55f0bdd3cb40 2024-04-24T09:16:49.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.963+0000 7ffab7400640 1 --2- 172.21.15.83:0/41496375 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0bdd3d460 0x55f0bdd44e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:16:49.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.964+0000 7ffab7e00640 1 --2- 172.21.15.83:0/41496375 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0bdd3cb40 0x55f0bdd44920 secure :-1 s=READY pgs=2809 cs=0 l=1 rev1=1 crypto rx=0x7ffaa8005ab0 tx=0x7ffaa802a490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31b265d9a7b073a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:49.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.964+0000 7ffaa2000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:16:49.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.964+0000 7ffaa2000640 1 -- 172.21.15.83:0/41496375 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaa8035040 con 0x55f0bdd3cb40 2024-04-24T09:16:49.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.964+0000 7ffaa2000640 1 -- 172.21.15.83:0/41496375 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(132..132 src has 1..132) ==== 9583+0+0 (secure 0 0 0) 0x7ffaa803b630 con 0x55f0bdd3cb40 2024-04-24T09:16:49.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.965+0000 7ffac1d80740 1 -- 172.21.15.83:0/41496375 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffaa80046c0 con 0x55f0bdd3cb40 2024-04-24T09:16:49.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.967+0000 7ffaa2000640 1 -- 172.21.15.83:0/41496375 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7ffaa803b830 con 0x55f0bdd3cb40 2024-04-24T09:16:49.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.967+0000 7ffac1d80740 10 client.24099 fetch_fsmap learned FSMap version 69 2024-04-24T09:16:49.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.969+0000 7ffac1d80740 1 -- 172.21.15.83:0/41496375 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x55f0bd9b43f0 con 0x55f0bdd3cb40 2024-04-24T09:16:49.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.970+0000 7ffaa2000640 1 -- 172.21.15.83:0/41496375 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffaa80046c0 con 0x55f0bdd3cb40 2024-04-24T09:16:49.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.970+0000 7ffaa2000640 1 -- 172.21.15.83:0/41496375 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 69) ==== 8370+0+0 (secure 0 0 0) 0x7ffaa804b070 con 0x55f0bdd3cb40 2024-04-24T09:16:49.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.971+0000 7ffac1d80740 10 client.24099 fetch_fsmap finished waiting for FSMap version 69 2024-04-24T09:16:49.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.971+0000 7ffac1d80740 10 client.24099 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:16:49.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.971+0000 7ffac1d80740 20 client.24099 populate_metadata read hostname 'smithi083' 2024-04-24T09:16:49.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.971+0000 7ffac1d80740 1 --2- 172.21.15.83:0/41496375 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55f0bdd57b50 0x55f0bdd59f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:49.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.971+0000 7ffac1d80740 4 client.24099 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:16:49.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.971+0000 7ffac1d80740 1 -- 172.21.15.83:0/41496375 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f0bdcafbf0 con 0x55f0bdd57b50 2024-04-24T09:16:49.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.972+0000 7ffab7400640 1 --2- 172.21.15.83:0/41496375 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55f0bdd57b50 0x55f0bdd59f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:49.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.973+0000 7ffab7400640 1 --2- 172.21.15.83:0/41496375 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55f0bdd57b50 0x55f0bdd59f40 crc :-1 s=READY pgs=615 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d694740bfe880b server_cookie=224c024fcf27fff5 in_seq=0 out_seq=0 2024-04-24T09:16:49.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.974+0000 7ffaa2000640 10 client.24099 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:16:49.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.975+0000 7ffaa2000640 1 -- 172.21.15.83:0/41496375 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f0bdcafbf0 con 0x55f0bdd57b50 2024-04-24T09:16:49.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.975+0000 7ffaa2000640 10 client.24099 handle_command_reply: tid=0 2024-04-24T09:16:49.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.975+0000 7ffac1d80740 1 client.24099 shutdown 2024-04-24T09:16:49.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.975+0000 7ffac1d80740 1 -- 172.21.15.83:0/41496375 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0bdd3cb40 msgr2=0x55f0bdd44920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:49.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.975+0000 7ffac1d80740 1 --2- 172.21.15.83:0/41496375 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0bdd3cb40 0x55f0bdd44920 secure :-1 s=READY pgs=2809 cs=0 l=1 rev1=1 crypto rx=0x7ffaa8005ab0 tx=0x7ffaa802a490 comp rx=0 tx=0).stop 2024-04-24T09:16:49.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.975+0000 7ffac1d80740 1 -- 172.21.15.83:0/41496375 shutdown_connections 2024-04-24T09:16:49.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.975+0000 7ffac1d80740 1 --2- 172.21.15.83:0/41496375 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55f0bdd57b50 0x55f0bdd59f40 crc :-1 s=READY pgs=615 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:49.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.976+0000 7ffac1d80740 1 --2- 172.21.15.83:0/41496375 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0bdd47b30 0x55f0bdd453a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:49.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.976+0000 7ffac1d80740 1 --2- 172.21.15.83:0/41496375 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0bdd3d460 0x55f0bdd44e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:49.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.976+0000 7ffaa2000640 0 client.24099 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:16:49.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.976+0000 7ffac1d80740 1 --2- 172.21.15.83:0/41496375 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0bdd3cb40 0x55f0bdd44920 secure :-1 s=CLOSED pgs=2809 cs=0 l=1 rev1=1 crypto rx=0x7ffaa8005ab0 tx=0x7ffaa802a490 comp rx=0 tx=0).stop 2024-04-24T09:16:49.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.976+0000 7ffac1d80740 1 -- 172.21.15.83:0/41496375 >> 172.21.15.83:0/41496375 conn(0x55f0bdd38e60 msgr2=0x55f0bdd3eb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:49.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.977+0000 7ffac1d80740 1 -- 172.21.15.83:0/41496375 shutdown_connections 2024-04-24T09:16:49.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.977+0000 7ffac1d80740 1 -- 172.21.15.83:0/41496375 wait complete. 2024-04-24T09:16:49.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:49.977+0000 7ffac1d80740 20 client.24099 trim_cache size 0 max 16384 2024-04-24T09:16:50.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:50.013+0000 7ffac1d80740 1 Processor -- start 2024-04-24T09:16:50.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:50.013+0000 7ffac1d80740 1 -- start start 2024-04-24T09:16:50.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:50.016+0000 7ffac1d80740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0bdd3d460 0x55f0bdd3ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:50.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:50.017+0000 7ffac1d80740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0bdd47b30 0x55f0bdd3f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:50.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:50.017+0000 7ffac1d80740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0bdd47160 0x55f0bdd41b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:50.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:50.019+0000 7ffac1d80740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ffab81b5c10 con 0x55f0bdd47b30 2024-04-24T09:16:50.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:50.019+0000 7ffac1d80740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ffab81b77b0 con 0x55f0bdd47160 2024-04-24T09:16:50.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:50.019+0000 7ffac1d80740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ffab81b62e0 con 0x55f0bdd3d460 2024-04-24T09:16:50.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:50.019+0000 7ffab7e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0bdd3d460 0x55f0bdd3ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:50.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:50.019+0000 7ffabce00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0bdd47160 0x55f0bdd41b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:50.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:50.019+0000 7ffab7400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0bdd47b30 0x55f0bdd3f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:50.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:50.019+0000 7ffabce00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0bdd47160 0x55f0bdd41b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:43322/0 (socket says 172.21.15.83:43322) 2024-04-24T09:16:50.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:50.019+0000 7ffabce00640 1 -- 172.21.15.83:0/4249723548 learned_addr learned my addr 172.21.15.83:0/4249723548 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:16:50.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:50.022+0000 7ffabce00640 1 -- 172.21.15.83:0/4249723548 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0bdd3d460 msgr2=0x55f0bdd3ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:50.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:50.022+0000 7ffabce00640 1 --2- 172.21.15.83:0/4249723548 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0bdd3d460 0x55f0bdd3ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:50.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:50.022+0000 7ffabce00640 1 -- 172.21.15.83:0/4249723548 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0bdd47b30 msgr2=0x55f0bdd3f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:50.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:50.022+0000 7ffabce00640 1 --2- 172.21.15.83:0/4249723548 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0bdd47b30 0x55f0bdd3f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:50.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:50.022+0000 7ffabce00640 1 -- 172.21.15.83:0/4249723548 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0bda7e7d0 con 0x55f0bdd47160 2024-04-24T09:16:50.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:50.022+0000 7ffab7400640 1 --2- 172.21.15.83:0/4249723548 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0bdd47b30 0x55f0bdd3f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:16:50.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:50.023+0000 7ffabce00640 1 --2- 172.21.15.83:0/4249723548 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0bdd47160 0x55f0bdd41b80 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7ffab0017910 tx=0x7ffab0012000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b741901f05b64f20 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:50.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:50.025+0000 7ffaacc00640 1 -- 172.21.15.83:0/4249723548 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffab0028070 con 0x55f0bdd47160 2024-04-24T09:16:50.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:50.026+0000 7ffaacc00640 1 -- 172.21.15.83:0/4249723548 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffab0016c60 con 0x55f0bdd47160 2024-04-24T09:16:50.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:50.026+0000 7ffaacc00640 1 -- 172.21.15.83:0/4249723548 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffab0028070 con 0x55f0bdd47160 2024-04-24T09:16:50.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:50.027+0000 7ffac1d80740 1 -- 172.21.15.83:0/4249723548 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0bdd47160 msgr2=0x55f0bdd41b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:50.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:50.027+0000 7ffac1d80740 1 --2- 172.21.15.83:0/4249723548 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0bdd47160 0x55f0bdd41b80 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7ffab0017910 tx=0x7ffab0012000 comp rx=0 tx=0).stop 2024-04-24T09:16:50.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:50.028+0000 7ffac1d80740 1 -- 172.21.15.83:0/4249723548 shutdown_connections 2024-04-24T09:16:50.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:50.028+0000 7ffac1d80740 1 --2- 172.21.15.83:0/4249723548 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0bdd47160 0x55f0bdd41b80 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:50.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:50.028+0000 7ffac1d80740 1 --2- 172.21.15.83:0/4249723548 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0bdd47b30 0x55f0bdd3f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:50.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:50.028+0000 7ffac1d80740 1 --2- 172.21.15.83:0/4249723548 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0bdd3d460 0x55f0bdd3ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:50.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:50.028+0000 7ffac1d80740 1 -- 172.21.15.83:0/4249723548 >> 172.21.15.83:0/4249723548 conn(0x55f0bdd38e60 msgr2=0x55f0bdd535c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:50.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:50.028+0000 7ffac1d80740 1 -- 172.21.15.83:0/4249723548 shutdown_connections 2024-04-24T09:16:50.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:50.029+0000 7ffac1d80740 1 -- 172.21.15.83:0/4249723548 wait complete. 2024-04-24T09:16:50.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:50.029+0000 7ffac1d80740 1 -- start start 2024-04-24T09:16:50.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:50.029+0000 7ffac1d80740 1 Processor -- start 2024-04-24T09:16:50.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:50.030+0000 7ffac1d80740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0bdd3d460 0x55f0bdd5d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:50.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:50.031+0000 7ffac1d80740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0bdd47160 0x55f0bdd58530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:50.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:50.031+0000 7ffac1d80740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0bdd47b30 0x55f0bdd58a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:50.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:50.031+0000 7ffab7e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0bdd3d460 0x55f0bdd5d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:50.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:50.032+0000 7ffac1d80740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ffab81b70e0 con 0x55f0bdd47b30 2024-04-24T09:16:50.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:50.032+0000 7ffac1d80740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ffab81b6a10 con 0x55f0bdd47160 2024-04-24T09:16:50.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:50.032+0000 7ffac1d80740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ffab81b3c10 con 0x55f0bdd3d460 2024-04-24T09:16:50.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:50.032+0000 7ffac1d80740 10 client.0 _ll_register_callbacks cb 0x7ffaa8004260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:16:50.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:50.032+0000 7ffabce00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0bdd47b30 0x55f0bdd58a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:50.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:50.032+0000 7ffab7400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0bdd47160 0x55f0bdd58530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:50.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:50.032+0000 7ffabce00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0bdd47b30 0x55f0bdd58a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36106/0 (socket says 172.21.15.83:36106) 2024-04-24T09:16:50.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:50.032+0000 7ffabce00640 1 -- 172.21.15.83:0/1415397647 learned_addr learned my addr 172.21.15.83:0/1415397647 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:16:50.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:50.033+0000 7ffabce00640 1 -- 172.21.15.83:0/1415397647 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0bdd3d460 msgr2=0x55f0bdd5d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:50.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:50.033+0000 7ffabce00640 1 --2- 172.21.15.83:0/1415397647 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0bdd3d460 0x55f0bdd5d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:50.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:50.033+0000 7ffabce00640 1 -- 172.21.15.83:0/1415397647 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0bdd47160 msgr2=0x55f0bdd58530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:50.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:50.033+0000 7ffabce00640 1 --2- 172.21.15.83:0/1415397647 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0bdd47160 0x55f0bdd58530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:50.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:50.033+0000 7ffabce00640 1 -- 172.21.15.83:0/1415397647 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0bdd59030 con 0x55f0bdd47b30 2024-04-24T09:16:50.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:50.033+0000 7ffab7400640 1 --2- 172.21.15.83:0/1415397647 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0bdd47160 0x55f0bdd58530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:16:50.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:50.034+0000 7ffabce00640 1 --2- 172.21.15.83:0/1415397647 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0bdd47b30 0x55f0bdd58a70 secure :-1 s=READY pgs=2810 cs=0 l=1 rev1=1 crypto rx=0x7ffab00178e0 tx=0x7ffab00174c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dce89125dd83da12 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:50.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:50.034+0000 7ffaa2000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:16:50.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:50.035+0000 7ffaa2000640 1 -- 172.21.15.83:0/1415397647 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffab0028070 con 0x55f0bdd47b30 2024-04-24T09:16:50.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:50.035+0000 7ffaa2000640 1 -- 172.21.15.83:0/1415397647 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(132..132 src has 1..132) ==== 9583+0+0 (secure 0 0 0) 0x7ffab0023c90 con 0x55f0bdd47b30 2024-04-24T09:16:50.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:50.035+0000 7ffac1d80740 1 -- 172.21.15.83:0/1415397647 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffab0016c60 con 0x55f0bdd47b30 2024-04-24T09:16:50.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:50.035+0000 7ffaa2000640 1 -- 172.21.15.83:0/1415397647 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7ffab0036050 con 0x55f0bdd47b30 2024-04-24T09:16:50.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:50.035+0000 7ffac1d80740 10 client.24105 fetch_fsmap learned FSMap version 69 2024-04-24T09:16:50.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:50.035+0000 7ffac1d80740 1 -- 172.21.15.83:0/1415397647 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x55f0bddbdaf0 con 0x55f0bdd47b30 2024-04-24T09:16:50.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:50.036+0000 7ffaa2000640 1 -- 172.21.15.83:0/1415397647 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffab0016c60 con 0x55f0bdd47b30 2024-04-24T09:16:50.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:50.036+0000 7ffaa2000640 1 -- 172.21.15.83:0/1415397647 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 69) ==== 8370+0+0 (secure 0 0 0) 0x7ffab0022030 con 0x55f0bdd47b30 2024-04-24T09:16:50.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:50.036+0000 7ffac1d80740 10 client.24105 fetch_fsmap finished waiting for FSMap version 69 2024-04-24T09:16:50.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:50.037+0000 7ffac1d80740 10 client.24105 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:16:50.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:50.037+0000 7ffac1d80740 20 client.24105 populate_metadata read hostname 'smithi083' 2024-04-24T09:16:50.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:50.037+0000 7ffac1d80740 1 --2- 172.21.15.83:0/1415397647 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55f0bddbe070 0x55f0bddc0490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:50.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:50.037+0000 7ffac1d80740 4 client.24105 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:16:50.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:50.037+0000 7ffac1d80740 1 -- 172.21.15.83:0/1415397647 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f0bdca9a60 con 0x55f0bddbe070 2024-04-24T09:16:50.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:50.037+0000 7ffab7e00640 1 --2- 172.21.15.83:0/1415397647 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55f0bddbe070 0x55f0bddc0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:50.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:50.038+0000 7ffab7e00640 1 --2- 172.21.15.83:0/1415397647 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55f0bddbe070 0x55f0bddc0490 crc :-1 s=READY pgs=616 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d21c53433acf0406 server_cookie=a5eeebdb39c3f5b3 in_seq=0 out_seq=0 2024-04-24T09:16:50.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:50.038+0000 7ffaa2000640 10 client.24105 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:16:50.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:50.039+0000 7ffaa2000640 1 -- 172.21.15.83:0/1415397647 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f0bdca9a60 con 0x55f0bddbe070 2024-04-24T09:16:50.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:50.039+0000 7ffaa2000640 10 client.24105 handle_command_reply: tid=0 2024-04-24T09:16:50.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:50.039+0000 7ffac1d80740 1 client.24105 shutdown 2024-04-24T09:16:50.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:50.039+0000 7ffac1d80740 1 -- 172.21.15.83:0/1415397647 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0bdd47b30 msgr2=0x55f0bdd58a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:50.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:50.039+0000 7ffac1d80740 1 --2- 172.21.15.83:0/1415397647 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0bdd47b30 0x55f0bdd58a70 secure :-1 s=READY pgs=2810 cs=0 l=1 rev1=1 crypto rx=0x7ffab00178e0 tx=0x7ffab00174c0 comp rx=0 tx=0).stop 2024-04-24T09:16:50.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:50.039+0000 7ffac1d80740 1 -- 172.21.15.83:0/1415397647 shutdown_connections 2024-04-24T09:16:50.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:50.039+0000 7ffac1d80740 1 --2- 172.21.15.83:0/1415397647 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55f0bddbe070 0x55f0bddc0490 crc :-1 s=READY pgs=616 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:50.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:50.039+0000 7ffac1d80740 1 --2- 172.21.15.83:0/1415397647 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0bdd47b30 0x55f0bdd58a70 unknown :-1 s=CLOSED pgs=2810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:50.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:50.039+0000 7ffac1d80740 1 --2- 172.21.15.83:0/1415397647 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0bdd47160 0x55f0bdd58530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:50.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:50.039+0000 7ffac1d80740 1 --2- 172.21.15.83:0/1415397647 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0bdd3d460 0x55f0bdd5d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:50.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:50.039+0000 7ffac1d80740 1 -- 172.21.15.83:0/1415397647 >> 172.21.15.83:0/1415397647 conn(0x55f0bdd38e60 msgr2=0x55f0bdd448e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:50.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:50.040+0000 7ffaa2000640 0 client.24105 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:16:50.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:50.040+0000 7ffac1d80740 1 -- 172.21.15.83:0/1415397647 shutdown_connections 2024-04-24T09:16:50.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:50.040+0000 7ffac1d80740 1 -- 172.21.15.83:0/1415397647 wait complete. 2024-04-24T09:16:50.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:50.040+0000 7ffac1d80740 20 client.24105 trim_cache size 0 max 16384 2024-04-24T09:16:50.042 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:16:50.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:50.041+0000 7ffabd800640 1 -- 172.21.15.83:0/2062003159 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ffa9c0a3d20 msgr2=0x7ffa9c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:50.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:50.041+0000 7ffabd800640 1 --2- 172.21.15.83:0/2062003159 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ffa9c0a3d20 0x7ffa9c0a61e0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7ffab000a510 tx=0x7ffab0009650 comp rx=0 tx=0).stop 2024-04-24T09:16:50.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:50.041+0000 7ffabd800640 1 -- 172.21.15.83:0/2062003159 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffab8103980 msgr2=0x7ffab81a4e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:50.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:50.042+0000 7ffabd800640 1 --2- 172.21.15.83:0/2062003159 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffab8103980 0x7ffab81a4e80 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7ffaa400d740 tx=0x7ffaa400dc00 comp rx=0 tx=0).stop 2024-04-24T09:16:50.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:50.042+0000 7ffabd800640 1 -- 172.21.15.83:0/2062003159 shutdown_connections 2024-04-24T09:16:50.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:50.042+0000 7ffabd800640 1 --2- 172.21.15.83:0/2062003159 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ffa9c0a3d20 0x7ffa9c0a61e0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:50.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:50.042+0000 7ffabd800640 1 --2- 172.21.15.83:0/2062003159 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffab810c480 0x7ffab810aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:50.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:50.042+0000 7ffabd800640 1 --2- 172.21.15.83:0/2062003159 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffab8103980 0x7ffab81a4e80 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:50.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:50.042+0000 7ffabd800640 1 --2- 172.21.15.83:0/2062003159 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffab8103060 0x7ffab81a4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:50.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:50.042+0000 7ffabd800640 1 -- 172.21.15.83:0/2062003159 >> 172.21.15.83:0/2062003159 conn(0x7ffab80fc880 msgr2=0x7ffab8100be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:50.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:50.043+0000 7ffabd800640 1 -- 172.21.15.83:0/2062003159 shutdown_connections 2024-04-24T09:16:50.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:50.043+0000 7ffabd800640 1 -- 172.21.15.83:0/2062003159 wait complete. 2024-04-24T09:16:50.223 INFO:tasks.workunit.client.0.smithi083.stdout: 10 83167 6.49 MB/sec execute 561 sec latency 102.125 ms 2024-04-24T09:16:50.447 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:16:50 smithi138 ceph-mon[31441]: pgmap v797: 129 pgs: 129 active+clean; 1.2 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 6.2 MiB/s wr, 350 op/s 2024-04-24T09:16:50.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:16:50 smithi189 ceph-mon[31791]: pgmap v797: 129 pgs: 129 active+clean; 1.2 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 6.2 MiB/s wr, 350 op/s 2024-04-24T09:16:50.552 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:16:50 smithi083 ceph-mon[27907]: pgmap v797: 129 pgs: 129 active+clean; 1.2 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 6.2 MiB/s wr, 350 op/s 2024-04-24T09:16:51.069 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T09:16:51.070 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T09:16:51.070 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T09:16:51.223 INFO:tasks.workunit.client.0.smithi083.stdout: 10 83273 6.49 MB/sec execute 562 sec latency 145.225 ms 2024-04-24T09:16:51.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.372+0000 7f5b89a00640 1 -- 172.21.15.83:0/1361504111 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b841106a0 msgr2=0x7f5b84110a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:51.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.372+0000 7f5b89a00640 1 --2- 172.21.15.83:0/1361504111 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b841106a0 0x7f5b84110a80 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c009900 tx=0x7f5b6c02a490 comp rx=0 tx=0).stop 2024-04-24T09:16:51.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.373+0000 7f5b89a00640 1 -- 172.21.15.83:0/1361504111 shutdown_connections 2024-04-24T09:16:51.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.373+0000 7f5b89a00640 1 --2- 172.21.15.83:0/1361504111 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5b84109a80 0x7f5b8411d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:51.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.373+0000 7f5b89a00640 1 --2- 172.21.15.83:0/1361504111 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b84109000 0x7f5b84109460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:51.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.373+0000 7f5b89a00640 1 --2- 172.21.15.83:0/1361504111 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b841106a0 0x7f5b84110a80 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:51.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.373+0000 7f5b89a00640 1 -- 172.21.15.83:0/1361504111 >> 172.21.15.83:0/1361504111 conn(0x7f5b8406cf80 msgr2=0x7f5b8406d390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:51.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.376+0000 7f5b89a00640 1 -- 172.21.15.83:0/1361504111 shutdown_connections 2024-04-24T09:16:51.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.378+0000 7f5b89a00640 1 -- 172.21.15.83:0/1361504111 wait complete. 2024-04-24T09:16:51.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.379+0000 7f5b89a00640 1 Processor -- start 2024-04-24T09:16:51.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.380+0000 7f5b89a00640 1 -- start start 2024-04-24T09:16:51.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.382+0000 7f5b89a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5b84109000 0x7f5b841a7530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:51.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.383+0000 7f5b89a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b84109a80 0x7f5b841a7a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:51.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.383+0000 7f5b89a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b841106a0 0x7f5b841abd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:51.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.383+0000 7f5b89a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5b84120f30 con 0x7f5b84109a80 2024-04-24T09:16:51.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.383+0000 7f5b89a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5b84120db0 con 0x7f5b841106a0 2024-04-24T09:16:51.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.383+0000 7f5b89a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5b841210b0 con 0x7f5b84109000 2024-04-24T09:16:51.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.383+0000 7f5b83400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b84109a80 0x7f5b841a7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:51.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.383+0000 7f5b89000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b841106a0 0x7f5b841abd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:51.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.383+0000 7f5b83400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b84109a80 0x7f5b841a7a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36118/0 (socket says 172.21.15.83:36118) 2024-04-24T09:16:51.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.383+0000 7f5b83400640 1 -- 172.21.15.83:0/4044877312 learned_addr learned my addr 172.21.15.83:0/4044877312 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:16:51.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.384+0000 7f5b83400640 1 -- 172.21.15.83:0/4044877312 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5b84109000 msgr2=0x7f5b841a7530 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-24T09:16:51.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.384+0000 7f5b83400640 1 --2- 172.21.15.83:0/4044877312 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5b84109000 0x7f5b841a7530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:51.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.384+0000 7f5b83400640 1 -- 172.21.15.83:0/4044877312 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b841106a0 msgr2=0x7f5b841abd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:51.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.384+0000 7f5b83400640 1 --2- 172.21.15.83:0/4044877312 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b841106a0 0x7f5b841abd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:51.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.384+0000 7f5b83400640 1 -- 172.21.15.83:0/4044877312 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b841ac480 con 0x7f5b84109a80 2024-04-24T09:16:51.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.385+0000 7f5b83400640 1 --2- 172.21.15.83:0/4044877312 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b84109a80 0x7f5b841a7a70 secure :-1 s=READY pgs=2811 cs=0 l=1 rev1=1 crypto rx=0x7f5b7400eab0 tx=0x7f5b7400ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:51.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.385+0000 7f5b80c00640 1 -- 172.21.15.83:0/4044877312 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b7400ce40 con 0x7f5b84109a80 2024-04-24T09:16:51.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.385+0000 7f5b80c00640 1 -- 172.21.15.83:0/4044877312 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5b74011920 con 0x7f5b84109a80 2024-04-24T09:16:51.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.385+0000 7f5b80c00640 1 -- 172.21.15.83:0/4044877312 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b74010a00 con 0x7f5b84109a80 2024-04-24T09:16:51.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.385+0000 7f5b89a00640 1 -- 172.21.15.83:0/4044877312 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b841ac770 con 0x7f5b84109a80 2024-04-24T09:16:51.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.386+0000 7f5b89a00640 1 -- 172.21.15.83:0/4044877312 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b841b4000 con 0x7f5b84109a80 2024-04-24T09:16:51.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.391+0000 7f5b8dde1740 1 Processor -- start 2024-04-24T09:16:51.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.392+0000 7f5b80c00640 1 -- 172.21.15.83:0/4044877312 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5b74011ad0 con 0x7f5b84109a80 2024-04-24T09:16:51.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.396+0000 7f5b80c00640 1 --2- 172.21.15.83:0/4044877312 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f5b680a3ef0 0x7f5b680a63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:51.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.396+0000 7f5b80c00640 1 -- 172.21.15.83:0/4044877312 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 9583+0+0 (secure 0 0 0) 0x7f5b740d1260 con 0x7f5b84109a80 2024-04-24T09:16:51.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.396+0000 7f5b8dde1740 1 -- start start 2024-04-24T09:16:51.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.397+0000 7f5b89000640 1 --2- 172.21.15.83:0/4044877312 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f5b680a3ef0 0x7f5b680a63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:51.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.397+0000 7f5b8dde1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55864c64f270 0x55864c64f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:51.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.397+0000 7f5b8dde1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55864c64fb90 0x55864c659d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:51.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.397+0000 7f5b8dde1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55864c65a260 0x55864c65c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:51.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.397+0000 7f5b8dde1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55864c5b4e70 con 0x55864c65a260 2024-04-24T09:16:51.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.397+0000 7f5b8dde1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55864c55b380 con 0x55864c64fb90 2024-04-24T09:16:51.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.397+0000 7f5b8dde1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55864c5b0e30 con 0x55864c64f270 2024-04-24T09:16:51.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.398+0000 7f5b83400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55864c64fb90 0x55864c659d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:51.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.398+0000 7f5b83400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55864c64fb90 0x55864c659d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:43358/0 (socket says 172.21.15.83:43358) 2024-04-24T09:16:51.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.398+0000 7f5b83400640 1 -- 172.21.15.83:0/4216842807 learned_addr learned my addr 172.21.15.83:0/4216842807 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:16:51.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.398+0000 7f5b89000640 1 --2- 172.21.15.83:0/4216842807 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55864c65a260 0x55864c65c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:51.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.398+0000 7f5b83e00640 1 --2- 172.21.15.83:0/4216842807 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55864c64f270 0x55864c64f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:51.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.399+0000 7f5b83400640 1 -- 172.21.15.83:0/4216842807 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55864c64f270 msgr2=0x55864c64f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:51.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.399+0000 7f5b83400640 1 --2- 172.21.15.83:0/4216842807 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55864c64f270 0x55864c64f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:51.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.399+0000 7f5b83400640 1 -- 172.21.15.83:0/4216842807 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55864c65a260 msgr2=0x55864c65c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:51.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.399+0000 7f5b83400640 1 --2- 172.21.15.83:0/4216842807 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55864c65a260 0x55864c65c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:51.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.399+0000 7f5b83400640 1 -- 172.21.15.83:0/4216842807 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55864c38f7e0 con 0x55864c64fb90 2024-04-24T09:16:51.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.399+0000 7f5b89000640 1 --2- 172.21.15.83:0/4044877312 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f5b680a3ef0 0x7f5b680a63b0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c005fd0 tx=0x7f5b7c0137d0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:51.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.399+0000 7f5b83400640 1 --2- 172.21.15.83:0/4216842807 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55864c64fb90 0x55864c659d20 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f5b7409cd10 tx=0x7f5b7409c7b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b7a7ee8605b6f9c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:51.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.399+0000 7f5b79600640 1 -- 172.21.15.83:0/4216842807 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b740b6070 con 0x55864c64fb90 2024-04-24T09:16:51.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.400+0000 7f5b79600640 1 -- 172.21.15.83:0/4216842807 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5b7409cd80 con 0x55864c64fb90 2024-04-24T09:16:51.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.400+0000 7f5b79600640 1 -- 172.21.15.83:0/4216842807 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b740ce7b0 con 0x55864c64fb90 2024-04-24T09:16:51.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.401+0000 7f5b8dde1740 1 -- 172.21.15.83:0/4216842807 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55864c64fb90 msgr2=0x55864c659d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:51.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.401+0000 7f5b8dde1740 1 --2- 172.21.15.83:0/4216842807 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55864c64fb90 0x55864c659d20 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f5b7409cd10 tx=0x7f5b7409c7b0 comp rx=0 tx=0).stop 2024-04-24T09:16:51.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.401+0000 7f5b8dde1740 1 -- 172.21.15.83:0/4216842807 shutdown_connections 2024-04-24T09:16:51.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.401+0000 7f5b8dde1740 1 --2- 172.21.15.83:0/4216842807 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55864c65a260 0x55864c65c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:51.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.401+0000 7f5b8dde1740 1 --2- 172.21.15.83:0/4216842807 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55864c64fb90 0x55864c659d20 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:51.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.401+0000 7f5b8dde1740 1 --2- 172.21.15.83:0/4216842807 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55864c64f270 0x55864c64f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:51.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.401+0000 7f5b8dde1740 1 -- 172.21.15.83:0/4216842807 >> 172.21.15.83:0/4216842807 conn(0x55864c64b590 msgr2=0x55864c64b970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:51.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.402+0000 7f5b8dde1740 1 -- 172.21.15.83:0/4216842807 shutdown_connections 2024-04-24T09:16:51.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.402+0000 7f5b8dde1740 1 -- 172.21.15.83:0/4216842807 wait complete. 2024-04-24T09:16:51.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.403+0000 7f5b8dde1740 1 -- start start 2024-04-24T09:16:51.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.403+0000 7f5b8dde1740 1 Processor -- start 2024-04-24T09:16:51.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.405+0000 7f5b8dde1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55864c64f270 0x55864c657210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:51.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.405+0000 7f5b8dde1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55864c64fb90 0x55864c657750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:51.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.405+0000 7f5b8dde1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55864c65a260 0x55864c657c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:51.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.405+0000 7f5b8dde1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55864c66e100 con 0x55864c64f270 2024-04-24T09:16:51.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.405+0000 7f5b8dde1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55864c56e660 con 0x55864c65a260 2024-04-24T09:16:51.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.405+0000 7f5b8dde1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55864c66e280 con 0x55864c64fb90 2024-04-24T09:16:51.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.405+0000 7f5b8dde1740 10 client.0 _ll_register_callbacks cb 0x55864c55caa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:16:51.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.405+0000 7f5b83400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55864c64fb90 0x55864c657750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:51.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.405+0000 7f5b83400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55864c64fb90 0x55864c657750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:41448/0 (socket says 172.21.15.83:41448) 2024-04-24T09:16:51.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.405+0000 7f5b83400640 1 -- 172.21.15.83:0/2781874771 learned_addr learned my addr 172.21.15.83:0/2781874771 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:16:51.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.405+0000 7f5b89000640 1 --2- 172.21.15.83:0/2781874771 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55864c65a260 0x55864c657c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:51.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.406+0000 7f5b83e00640 1 --2- 172.21.15.83:0/2781874771 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55864c64f270 0x55864c657210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:51.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.406+0000 7f5b83400640 1 -- 172.21.15.83:0/2781874771 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55864c65a260 msgr2=0x55864c657c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:51.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.406+0000 7f5b83400640 1 --2- 172.21.15.83:0/2781874771 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55864c65a260 0x55864c657c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:51.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.406+0000 7f5b83400640 1 -- 172.21.15.83:0/2781874771 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55864c64f270 msgr2=0x55864c657210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:51.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.406+0000 7f5b83400640 1 --2- 172.21.15.83:0/2781874771 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55864c64f270 0x55864c657210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:51.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.406+0000 7f5b83400640 1 -- 172.21.15.83:0/2781874771 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55864c3eff70 con 0x55864c64fb90 2024-04-24T09:16:51.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.406+0000 7f5b83e00640 1 --2- 172.21.15.83:0/2781874771 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55864c64f270 0x55864c657210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:16:51.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.406+0000 7f5b83400640 1 --2- 172.21.15.83:0/2781874771 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55864c64fb90 0x55864c657750 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5b740a54c0 tx=0x7f5b74010480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a00a70361a72c2f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:51.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.406+0000 7f5b72000640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-04-24T09:16:51.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.407+0000 7f5b72000640 1 -- 172.21.15.83:0/2781874771 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b740ba070 con 0x55864c64fb90 2024-04-24T09:16:51.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.407+0000 7f5b72000640 1 -- 172.21.15.83:0/2781874771 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(132..132 src has 1..132) ==== 9583+0+0 (secure 0 0 0) 0x7f5b740a5560 con 0x55864c64fb90 2024-04-24T09:16:51.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.407+0000 7f5b8dde1740 1 -- 172.21.15.83:0/2781874771 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b7409cd80 con 0x55864c64fb90 2024-04-24T09:16:51.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.408+0000 7f5b72000640 1 -- 172.21.15.83:0/2781874771 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f5b740b6040 con 0x55864c64fb90 2024-04-24T09:16:51.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.408+0000 7f5b8dde1740 10 client.30862 fetch_fsmap learned FSMap version 69 2024-04-24T09:16:51.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.408+0000 7f5b8dde1740 1 -- 172.21.15.83:0/2781874771 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x55864c287170 con 0x55864c64fb90 2024-04-24T09:16:51.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.408+0000 7f5b72000640 1 -- 172.21.15.83:0/2781874771 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5b7409cd80 con 0x55864c64fb90 2024-04-24T09:16:51.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.408+0000 7f5b72000640 1 -- 172.21.15.83:0/2781874771 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 69) ==== 8370+0+0 (secure 0 0 0) 0x7f5b7400f040 con 0x55864c64fb90 2024-04-24T09:16:51.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.408+0000 7f5b8dde1740 10 client.30862 fetch_fsmap finished waiting for FSMap version 69 2024-04-24T09:16:51.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.409+0000 7f5b8dde1740 10 client.30862 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:16:51.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.409+0000 7f5b8dde1740 20 client.30862 populate_metadata read hostname 'smithi083' 2024-04-24T09:16:51.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.409+0000 7f5b8dde1740 1 --2- 172.21.15.83:0/2781874771 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55864c67b540 0x55864c67d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:51.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.409+0000 7f5b8dde1740 4 client.30862 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:16:51.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.409+0000 7f5b8dde1740 1 -- 172.21.15.83:0/2781874771 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55864c5c2890 con 0x55864c67b540 2024-04-24T09:16:51.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.409+0000 7f5b83e00640 1 --2- 172.21.15.83:0/2781874771 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55864c67b540 0x55864c67d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:51.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.410+0000 7f5b83e00640 1 --2- 172.21.15.83:0/2781874771 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55864c67b540 0x55864c67d930 crc :-1 s=READY pgs=617 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=484b681ffc15ca01 server_cookie=27ca109566e5f7d4 in_seq=0 out_seq=0 2024-04-24T09:16:51.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.410+0000 7f5b72000640 10 client.30862 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:16:51.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.411+0000 7f5b72000640 1 -- 172.21.15.83:0/2781874771 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55864c5c2890 con 0x55864c67b540 2024-04-24T09:16:51.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.411+0000 7f5b72000640 10 client.30862 handle_command_reply: tid=0 2024-04-24T09:16:51.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.412+0000 7f5b8dde1740 1 client.30862 shutdown 2024-04-24T09:16:51.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.412+0000 7f5b8dde1740 1 -- 172.21.15.83:0/2781874771 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55864c64fb90 msgr2=0x55864c657750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:51.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.412+0000 7f5b8dde1740 1 --2- 172.21.15.83:0/2781874771 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55864c64fb90 0x55864c657750 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5b740a54c0 tx=0x7f5b74010480 comp rx=0 tx=0).stop 2024-04-24T09:16:51.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.412+0000 7f5b8dde1740 1 -- 172.21.15.83:0/2781874771 shutdown_connections 2024-04-24T09:16:51.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.412+0000 7f5b8dde1740 1 --2- 172.21.15.83:0/2781874771 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55864c67b540 0x55864c67d930 crc :-1 s=READY pgs=617 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:51.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.412+0000 7f5b8dde1740 1 --2- 172.21.15.83:0/2781874771 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55864c65a260 0x55864c657c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:51.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.412+0000 7f5b8dde1740 1 --2- 172.21.15.83:0/2781874771 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55864c64fb90 0x55864c657750 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:51.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.412+0000 7f5b8dde1740 1 --2- 172.21.15.83:0/2781874771 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55864c64f270 0x55864c657210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:51.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.412+0000 7f5b8dde1740 1 -- 172.21.15.83:0/2781874771 >> 172.21.15.83:0/2781874771 conn(0x55864c64b590 msgr2=0x55864c651390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:51.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.413+0000 7f5b72000640 0 client.30862 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:16:51.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.413+0000 7f5b8dde1740 1 -- 172.21.15.83:0/2781874771 shutdown_connections 2024-04-24T09:16:51.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.413+0000 7f5b8dde1740 1 -- 172.21.15.83:0/2781874771 wait complete. 2024-04-24T09:16:51.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.413+0000 7f5b8dde1740 20 client.30862 trim_cache size 0 max 16384 2024-04-24T09:16:51.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.450+0000 7f5b8dde1740 1 Processor -- start 2024-04-24T09:16:51.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.450+0000 7f5b8dde1740 1 -- start start 2024-04-24T09:16:51.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.450+0000 7f5b8dde1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55864c64f270 0x55864c651a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:51.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.450+0000 7f5b8dde1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55864c64fb90 0x55864c657ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:51.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.450+0000 7f5b8dde1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55864c65a260 0x55864c680570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:51.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.451+0000 7f5b8dde1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5b841b5990 con 0x55864c64fb90 2024-04-24T09:16:51.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.451+0000 7f5b8dde1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5b841b7530 con 0x55864c64f270 2024-04-24T09:16:51.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.451+0000 7f5b8dde1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5b841b6060 con 0x55864c65a260 2024-04-24T09:16:51.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.451+0000 7f5b89000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55864c65a260 0x55864c680570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:51.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.451+0000 7f5b89000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55864c65a260 0x55864c680570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:41460/0 (socket says 172.21.15.83:41460) 2024-04-24T09:16:51.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.451+0000 7f5b89000640 1 -- 172.21.15.83:0/3393937735 learned_addr learned my addr 172.21.15.83:0/3393937735 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:16:51.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.451+0000 7f5b83400640 1 --2- 172.21.15.83:0/3393937735 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55864c64fb90 0x55864c657ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:51.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.452+0000 7f5b83400640 1 -- 172.21.15.83:0/3393937735 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55864c65a260 msgr2=0x55864c680570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:51.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.452+0000 7f5b83400640 1 --2- 172.21.15.83:0/3393937735 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55864c65a260 0x55864c680570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:51.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.452+0000 7f5b83400640 1 -- 172.21.15.83:0/3393937735 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55864c64f270 msgr2=0x55864c651a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:51.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.452+0000 7f5b83400640 1 --2- 172.21.15.83:0/3393937735 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55864c64f270 0x55864c651a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:51.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.452+0000 7f5b83400640 1 -- 172.21.15.83:0/3393937735 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55864c6529e0 con 0x55864c64fb90 2024-04-24T09:16:51.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.452+0000 7f5b89000640 1 --2- 172.21.15.83:0/3393937735 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55864c65a260 0x55864c680570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:16:51.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.453+0000 7f5b83400640 1 --2- 172.21.15.83:0/3393937735 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55864c64fb90 0x55864c657ab0 secure :-1 s=READY pgs=2812 cs=0 l=1 rev1=1 crypto rx=0x7f5b7409ca80 tx=0x7f5b7409c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4d4c617a6e31690 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:51.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.453+0000 7f5b78c00640 1 -- 172.21.15.83:0/3393937735 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b740ba070 con 0x55864c64fb90 2024-04-24T09:16:51.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.454+0000 7f5b78c00640 1 -- 172.21.15.83:0/3393937735 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5b7409c4c0 con 0x55864c64fb90 2024-04-24T09:16:51.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.454+0000 7f5b78c00640 1 -- 172.21.15.83:0/3393937735 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b740bd210 con 0x55864c64fb90 2024-04-24T09:16:51.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.455+0000 7f5b8dde1740 1 -- 172.21.15.83:0/3393937735 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55864c64fb90 msgr2=0x55864c657ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:51.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.455+0000 7f5b8dde1740 1 --2- 172.21.15.83:0/3393937735 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55864c64fb90 0x55864c657ab0 secure :-1 s=READY pgs=2812 cs=0 l=1 rev1=1 crypto rx=0x7f5b7409ca80 tx=0x7f5b7409c7b0 comp rx=0 tx=0).stop 2024-04-24T09:16:51.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.455+0000 7f5b8dde1740 1 -- 172.21.15.83:0/3393937735 shutdown_connections 2024-04-24T09:16:51.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.455+0000 7f5b8dde1740 1 --2- 172.21.15.83:0/3393937735 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55864c65a260 0x55864c680570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:51.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.455+0000 7f5b8dde1740 1 --2- 172.21.15.83:0/3393937735 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55864c64fb90 0x55864c657ab0 unknown :-1 s=CLOSED pgs=2812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:51.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.455+0000 7f5b8dde1740 1 --2- 172.21.15.83:0/3393937735 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55864c64f270 0x55864c651a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:51.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.455+0000 7f5b8dde1740 1 -- 172.21.15.83:0/3393937735 >> 172.21.15.83:0/3393937735 conn(0x55864c64b590 msgr2=0x55864c676d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:51.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.455+0000 7f5b8dde1740 1 -- 172.21.15.83:0/3393937735 shutdown_connections 2024-04-24T09:16:51.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.455+0000 7f5b8dde1740 1 -- 172.21.15.83:0/3393937735 wait complete. 2024-04-24T09:16:51.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.456+0000 7f5b8dde1740 1 -- start start 2024-04-24T09:16:51.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.456+0000 7f5b8dde1740 1 Processor -- start 2024-04-24T09:16:51.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.457+0000 7f5b8dde1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55864c64f270 0x55864c6808b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:51.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.457+0000 7f5b8dde1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55864c64fb90 0x55864c67dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:51.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.457+0000 7f5b8dde1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55864c65a260 0x55864c67e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:51.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.457+0000 7f5b8dde1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5b841b6e60 con 0x55864c65a260 2024-04-24T09:16:51.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.457+0000 7f5b8dde1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5b841b6790 con 0x55864c64f270 2024-04-24T09:16:51.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.457+0000 7f5b8dde1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5b84118f30 con 0x55864c64fb90 2024-04-24T09:16:51.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.457+0000 7f5b8dde1740 10 client.0 _ll_register_callbacks cb 0x7f5b740b4070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:16:51.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.457+0000 7f5b83e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55864c64f270 0x55864c6808b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:51.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.457+0000 7f5b83e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55864c64f270 0x55864c6808b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:43386/0 (socket says 172.21.15.83:43386) 2024-04-24T09:16:51.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.457+0000 7f5b83e00640 1 -- 172.21.15.83:0/3644907755 learned_addr learned my addr 172.21.15.83:0/3644907755 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:16:51.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.457+0000 7f5b83400640 1 --2- 172.21.15.83:0/3644907755 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55864c64fb90 0x55864c67dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:51.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.458+0000 7f5b83e00640 1 -- 172.21.15.83:0/3644907755 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55864c64fb90 msgr2=0x55864c67dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:51.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.458+0000 7f5b83e00640 1 --2- 172.21.15.83:0/3644907755 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55864c64fb90 0x55864c67dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:51.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.458+0000 7f5b83e00640 1 -- 172.21.15.83:0/3644907755 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55864c65a260 msgr2=0x55864c67e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:51.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.458+0000 7f5b83e00640 1 --2- 172.21.15.83:0/3644907755 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55864c65a260 0x55864c67e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:51.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.458+0000 7f5b83e00640 1 -- 172.21.15.83:0/3644907755 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55864c67e980 con 0x55864c64f270 2024-04-24T09:16:51.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.458+0000 7f5b83400640 1 --2- 172.21.15.83:0/3644907755 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55864c64fb90 0x55864c67dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:16:51.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.458+0000 7f5b83e00640 1 --2- 172.21.15.83:0/3644907755 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55864c64f270 0x55864c6808b0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c02a9b0 tx=0x7f5b6c004280 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4f88a395e96dc7a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:51.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.458+0000 7f5b72000640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:16:51.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.458+0000 7f5b72000640 1 -- 172.21.15.83:0/3644907755 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b6c035040 con 0x55864c64f270 2024-04-24T09:16:51.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.458+0000 7f5b72000640 1 -- 172.21.15.83:0/3644907755 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(132..132 src has 1..132) ==== 9583+0+0 (secure 0 0 0) 0x7f5b6c005240 con 0x55864c64f270 2024-04-24T09:16:51.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.459+0000 7f5b8dde1740 1 -- 172.21.15.83:0/3644907755 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b7409c4c0 con 0x55864c64f270 2024-04-24T09:16:51.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.459+0000 7f5b72000640 1 -- 172.21.15.83:0/3644907755 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f5b6c005440 con 0x55864c64f270 2024-04-24T09:16:51.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.459+0000 7f5b8dde1740 10 client.41180 fetch_fsmap learned FSMap version 69 2024-04-24T09:16:51.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.459+0000 7f5b8dde1740 1 -- 172.21.15.83:0/3644907755 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x55864c6e16d0 con 0x55864c64f270 2024-04-24T09:16:51.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.460+0000 7f5b72000640 1 -- 172.21.15.83:0/3644907755 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5b7409c4c0 con 0x55864c64f270 2024-04-24T09:16:51.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.460+0000 7f5b72000640 1 -- 172.21.15.83:0/3644907755 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 69) ==== 8370+0+0 (secure 0 0 0) 0x7f5b6c03a070 con 0x55864c64f270 2024-04-24T09:16:51.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.460+0000 7f5b8dde1740 10 client.41180 fetch_fsmap finished waiting for FSMap version 69 2024-04-24T09:16:51.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.460+0000 7f5b8dde1740 10 client.41180 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:16:51.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.460+0000 7f5b8dde1740 20 client.41180 populate_metadata read hostname 'smithi083' 2024-04-24T09:16:51.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.460+0000 7f5b8dde1740 1 --2- 172.21.15.83:0/3644907755 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55864c6e1ea0 0x55864c6e4290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:51.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.460+0000 7f5b8dde1740 4 client.41180 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T09:16:51.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.460+0000 7f5b8dde1740 1 -- 172.21.15.83:0/3644907755 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55864c5bc700 con 0x55864c6e1ea0 2024-04-24T09:16:51.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.461+0000 7f5b83400640 1 --2- 172.21.15.83:0/3644907755 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55864c6e1ea0 0x55864c6e4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:51.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.463+0000 7f5b83400640 1 --2- 172.21.15.83:0/3644907755 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55864c6e1ea0 0x55864c6e4290 crc :-1 s=READY pgs=618 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15e648092b6e075d server_cookie=271352a12edef0eb in_seq=0 out_seq=0 2024-04-24T09:16:51.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.463+0000 7f5b72000640 10 client.41180 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:16:51.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.465+0000 7f5b72000640 1 -- 172.21.15.83:0/3644907755 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55864c5bc700 con 0x55864c6e1ea0 2024-04-24T09:16:51.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.465+0000 7f5b72000640 10 client.41180 handle_command_reply: tid=0 2024-04-24T09:16:51.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.465+0000 7f5b8dde1740 1 client.41180 shutdown 2024-04-24T09:16:51.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.466+0000 7f5b8dde1740 1 -- 172.21.15.83:0/3644907755 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55864c64f270 msgr2=0x55864c6808b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:51.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.466+0000 7f5b8dde1740 1 --2- 172.21.15.83:0/3644907755 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55864c64f270 0x55864c6808b0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c02a9b0 tx=0x7f5b6c004280 comp rx=0 tx=0).stop 2024-04-24T09:16:51.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.466+0000 7f5b8dde1740 1 -- 172.21.15.83:0/3644907755 shutdown_connections 2024-04-24T09:16:51.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.466+0000 7f5b8dde1740 1 --2- 172.21.15.83:0/3644907755 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55864c6e1ea0 0x55864c6e4290 crc :-1 s=READY pgs=618 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:51.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.466+0000 7f5b8dde1740 1 --2- 172.21.15.83:0/3644907755 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55864c65a260 0x55864c67e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:51.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.466+0000 7f5b8dde1740 1 --2- 172.21.15.83:0/3644907755 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55864c64fb90 0x55864c67dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:51.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.466+0000 7f5b8dde1740 1 --2- 172.21.15.83:0/3644907755 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55864c64f270 0x55864c6808b0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:51.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.466+0000 7f5b8dde1740 1 -- 172.21.15.83:0/3644907755 >> 172.21.15.83:0/3644907755 conn(0x55864c64b590 msgr2=0x55864c649e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:51.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.466+0000 7f5b72000640 0 client.41180 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:16:51.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.466+0000 7f5b8dde1740 1 -- 172.21.15.83:0/3644907755 shutdown_connections 2024-04-24T09:16:51.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.466+0000 7f5b8dde1740 1 -- 172.21.15.83:0/3644907755 wait complete. 2024-04-24T09:16:51.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.466+0000 7f5b8dde1740 20 client.41180 trim_cache size 0 max 16384 2024-04-24T09:16:51.467 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T09:16:51.468 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-04-24T09:16:51.468 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "04e42236-ddde-448f-8d34-4fa6b0ac2baf", 2024-04-24T09:16:51.468 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-04-24T09:16:51.468 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T09:16:51.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.467+0000 7f5b89a00640 1 -- 172.21.15.83:0/4044877312 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f5b680a3ef0 msgr2=0x7f5b680a63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:51.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.467+0000 7f5b89a00640 1 --2- 172.21.15.83:0/4044877312 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f5b680a3ef0 0x7f5b680a63b0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c005fd0 tx=0x7f5b7c0137d0 comp rx=0 tx=0).stop 2024-04-24T09:16:51.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.468+0000 7f5b89a00640 1 -- 172.21.15.83:0/4044877312 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b84109a80 msgr2=0x7f5b841a7a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:51.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.468+0000 7f5b89a00640 1 --2- 172.21.15.83:0/4044877312 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b84109a80 0x7f5b841a7a70 secure :-1 s=READY pgs=2811 cs=0 l=1 rev1=1 crypto rx=0x7f5b7400eab0 tx=0x7f5b7400ef70 comp rx=0 tx=0).stop 2024-04-24T09:16:51.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.468+0000 7f5b89a00640 1 -- 172.21.15.83:0/4044877312 shutdown_connections 2024-04-24T09:16:51.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.468+0000 7f5b89a00640 1 --2- 172.21.15.83:0/4044877312 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f5b680a3ef0 0x7f5b680a63b0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:51.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.468+0000 7f5b89a00640 1 --2- 172.21.15.83:0/4044877312 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b841106a0 0x7f5b841abd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:51.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.468+0000 7f5b89a00640 1 --2- 172.21.15.83:0/4044877312 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b84109a80 0x7f5b841a7a70 secure :-1 s=CLOSED pgs=2811 cs=0 l=1 rev1=1 crypto rx=0x7f5b7400eab0 tx=0x7f5b7400ef70 comp rx=0 tx=0).stop 2024-04-24T09:16:51.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.468+0000 7f5b89a00640 1 --2- 172.21.15.83:0/4044877312 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5b84109000 0x7f5b841a7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:51.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.468+0000 7f5b89a00640 1 -- 172.21.15.83:0/4044877312 >> 172.21.15.83:0/4044877312 conn(0x7f5b8406cf80 msgr2=0x7f5b84070d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:51.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.468+0000 7f5b89a00640 1 -- 172.21.15.83:0/4044877312 shutdown_connections 2024-04-24T09:16:51.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.469+0000 7f5b89a00640 1 -- 172.21.15.83:0/4044877312 wait complete. 2024-04-24T09:16:51.493 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T09:16:51.493 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T09:16:51.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.762+0000 7f33f0a00640 1 -- 172.21.15.83:0/1300715924 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33ec076530 msgr2=0x7f33ec076910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:51.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.762+0000 7f33f0a00640 1 --2- 172.21.15.83:0/1300715924 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33ec076530 0x7f33ec076910 secure :-1 s=READY pgs=2813 cs=0 l=1 rev1=1 crypto rx=0x7f33dc009900 tx=0x7f33dc02a3f0 comp rx=0 tx=0).stop 2024-04-24T09:16:51.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.763+0000 7f33f0a00640 1 -- 172.21.15.83:0/1300715924 shutdown_connections 2024-04-24T09:16:51.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.763+0000 7f33f0a00640 1 --2- 172.21.15.83:0/1300715924 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f33ec112d50 0x7f33ec115140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:51.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.763+0000 7f33f0a00640 1 --2- 172.21.15.83:0/1300715924 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f33ec076e50 0x7f33ec112810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:51.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.763+0000 7f33f0a00640 1 --2- 172.21.15.83:0/1300715924 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33ec076530 0x7f33ec076910 unknown :-1 s=CLOSED pgs=2813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:51.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.763+0000 7f33f0a00640 1 -- 172.21.15.83:0/1300715924 >> 172.21.15.83:0/1300715924 conn(0x7f33ec0ffb40 msgr2=0x7f33ec101f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:51.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.763+0000 7f33f0a00640 1 -- 172.21.15.83:0/1300715924 shutdown_connections 2024-04-24T09:16:51.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.763+0000 7f33f0a00640 1 -- 172.21.15.83:0/1300715924 wait complete. 2024-04-24T09:16:51.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.764+0000 7f33f0a00640 1 Processor -- start 2024-04-24T09:16:51.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.765+0000 7f33f0a00640 1 -- start start 2024-04-24T09:16:51.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.766+0000 7f33f0a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f33ec076530 0x7f33ec19ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:51.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.766+0000 7f33f0a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f33ec076e50 0x7f33ec19f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:51.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.766+0000 7f33f0a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33ec112d50 0x7f33ec1a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:51.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.766+0000 7f33f0a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f33ec118830 con 0x7f33ec112d50 2024-04-24T09:16:51.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.766+0000 7f33f0a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f33ec1186b0 con 0x7f33ec076530 2024-04-24T09:16:51.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.766+0000 7f33f0a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f33ec1189b0 con 0x7f33ec076e50 2024-04-24T09:16:51.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.767+0000 7f33ea800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f33ec076530 0x7f33ec19ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:51.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.767+0000 7f33eb200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33ec112d50 0x7f33ec1a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:51.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.767+0000 7f33ea800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f33ec076530 0x7f33ec19ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:43400/0 (socket says 172.21.15.83:43400) 2024-04-24T09:16:51.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.767+0000 7f33ea800640 1 -- 172.21.15.83:0/2490704545 learned_addr learned my addr 172.21.15.83:0/2490704545 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:16:51.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.767+0000 7f33e9e00640 1 --2- 172.21.15.83:0/2490704545 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f33ec076e50 0x7f33ec19f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:51.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.767+0000 7f33ea800640 1 -- 172.21.15.83:0/2490704545 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f33ec076e50 msgr2=0x7f33ec19f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:51.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.767+0000 7f33ea800640 1 --2- 172.21.15.83:0/2490704545 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f33ec076e50 0x7f33ec19f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:51.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.767+0000 7f33ea800640 1 -- 172.21.15.83:0/2490704545 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33ec112d50 msgr2=0x7f33ec1a3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:51.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.767+0000 7f33ea800640 1 --2- 172.21.15.83:0/2490704545 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33ec112d50 0x7f33ec1a3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:51.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.767+0000 7f33ea800640 1 -- 172.21.15.83:0/2490704545 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33ec1a3d60 con 0x7f33ec076530 2024-04-24T09:16:51.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.768+0000 7f33ea800640 1 --2- 172.21.15.83:0/2490704545 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f33ec076530 0x7f33ec19ed70 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f33dc004770 tx=0x7f33dc003ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:51.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.768+0000 7f33e3400640 1 -- 172.21.15.83:0/2490704545 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33dc03f070 con 0x7f33ec076530 2024-04-24T09:16:51.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.768+0000 7f33f0a00640 1 -- 172.21.15.83:0/2490704545 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33ec1a3ff0 con 0x7f33ec076530 2024-04-24T09:16:51.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.768+0000 7f33e3400640 1 -- 172.21.15.83:0/2490704545 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f33dc03bd20 con 0x7f33ec076530 2024-04-24T09:16:51.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.768+0000 7f33e3400640 1 -- 172.21.15.83:0/2490704545 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33dc035030 con 0x7f33ec076530 2024-04-24T09:16:51.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.768+0000 7f33f0a00640 1 -- 172.21.15.83:0/2490704545 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33ec1ab8e0 con 0x7f33ec076530 2024-04-24T09:16:51.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.774+0000 7f33f43c1740 1 Processor -- start 2024-04-24T09:16:51.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.774+0000 7f33f43c1740 1 -- start start 2024-04-24T09:16:51.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.774+0000 7f33e3400640 1 -- 172.21.15.83:0/2490704545 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f33dc042070 con 0x7f33ec076530 2024-04-24T09:16:51.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.774+0000 7f33f43c1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f4d299ba30 0x55f4d299be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:51.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.774+0000 7f33f43c1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f4d299c350 0x55f4d29a64e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:51.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.774+0000 7f33f43c1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4d29a6a20 0x55f4d29a8e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:51.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.774+0000 7f33f43c1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55f4d29011f0 con 0x55f4d299ba30 2024-04-24T09:16:51.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.774+0000 7f33f43c1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f4d28a7350 con 0x55f4d29a6a20 2024-04-24T09:16:51.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.774+0000 7f33f43c1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55f4d28fd1b0 con 0x55f4d299c350 2024-04-24T09:16:51.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.775+0000 7f33eb200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f4d299ba30 0x55f4d299be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:51.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.775+0000 7f33eb200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f4d299ba30 0x55f4d299be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36162/0 (socket says 172.21.15.83:36162) 2024-04-24T09:16:51.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.775+0000 7f33eb200640 1 -- 172.21.15.83:0/3308307716 learned_addr learned my addr 172.21.15.83:0/3308307716 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:16:51.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.775+0000 7f33e9e00640 1 --2- 172.21.15.83:0/3308307716 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4d29a6a20 0x55f4d29a8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:51.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.775+0000 7f33eb200640 1 -- 172.21.15.83:0/3308307716 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f4d299c350 msgr2=0x55f4d29a64e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:51.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.775+0000 7f33eb200640 1 --2- 172.21.15.83:0/3308307716 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f4d299c350 0x55f4d29a64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:51.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.775+0000 7f33eb200640 1 -- 172.21.15.83:0/3308307716 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4d29a6a20 msgr2=0x55f4d29a8e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:51.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.775+0000 7f33eb200640 1 --2- 172.21.15.83:0/3308307716 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4d29a6a20 0x55f4d29a8e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:51.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.775+0000 7f33eb200640 1 -- 172.21.15.83:0/3308307716 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4d28fbf50 con 0x55f4d299ba30 2024-04-24T09:16:51.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.775+0000 7f33e9e00640 1 --2- 172.21.15.83:0/3308307716 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4d29a6a20 0x55f4d29a8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:16:51.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.776+0000 7f33eb200640 1 --2- 172.21.15.83:0/3308307716 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f4d299ba30 0x55f4d299be10 secure :-1 s=READY pgs=2814 cs=0 l=1 rev1=1 crypto rx=0x7f33e400a400 tx=0x7f33e4009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=191e41263e15e40c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:51.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.776+0000 7f33d7400640 1 -- 172.21.15.83:0/3308307716 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33e4013790 con 0x55f4d299ba30 2024-04-24T09:16:51.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.776+0000 7f33d7400640 1 -- 172.21.15.83:0/3308307716 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f33e4019070 con 0x55f4d299ba30 2024-04-24T09:16:51.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.776+0000 7f33d7400640 1 -- 172.21.15.83:0/3308307716 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33e4014c50 con 0x55f4d299ba30 2024-04-24T09:16:51.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.776+0000 7f33e3400640 1 --2- 172.21.15.83:0/2490704545 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f33d00a3da0 0x7f33d00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:51.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.776+0000 7f33e3400640 1 -- 172.21.15.83:0/2490704545 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 9583+0+0 (secure 0 0 0) 0x7f33dc0f10c0 con 0x7f33ec076530 2024-04-24T09:16:51.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.777+0000 7f33ea800640 1 --2- 172.21.15.83:0/2490704545 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f33d00a3da0 0x7f33d00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:51.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.777+0000 7f33f43c1740 1 -- 172.21.15.83:0/3308307716 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f4d299ba30 msgr2=0x55f4d299be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:51.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.777+0000 7f33f43c1740 1 --2- 172.21.15.83:0/3308307716 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f4d299ba30 0x55f4d299be10 secure :-1 s=READY pgs=2814 cs=0 l=1 rev1=1 crypto rx=0x7f33e400a400 tx=0x7f33e4009650 comp rx=0 tx=0).stop 2024-04-24T09:16:51.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.777+0000 7f33f43c1740 1 -- 172.21.15.83:0/3308307716 shutdown_connections 2024-04-24T09:16:51.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.777+0000 7f33f43c1740 1 --2- 172.21.15.83:0/3308307716 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4d29a6a20 0x55f4d29a8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:51.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.777+0000 7f33f43c1740 1 --2- 172.21.15.83:0/3308307716 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f4d299c350 0x55f4d29a64e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:51.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.777+0000 7f33f43c1740 1 --2- 172.21.15.83:0/3308307716 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f4d299ba30 0x55f4d299be10 unknown :-1 s=CLOSED pgs=2814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:51.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.778+0000 7f33f43c1740 1 -- 172.21.15.83:0/3308307716 >> 172.21.15.83:0/3308307716 conn(0x55f4d2997d50 msgr2=0x55f4d2998130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:51.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.778+0000 7f33ea800640 1 --2- 172.21.15.83:0/2490704545 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f33d00a3da0 0x7f33d00a6260 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x55f4d299cff0 tx=0x7f33dc009560 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:51.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.778+0000 7f33f43c1740 1 -- 172.21.15.83:0/3308307716 shutdown_connections 2024-04-24T09:16:51.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.778+0000 7f33f43c1740 1 -- 172.21.15.83:0/3308307716 wait complete. 2024-04-24T09:16:51.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.778+0000 7f33f43c1740 1 -- start start 2024-04-24T09:16:51.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.779+0000 7f33f43c1740 1 Processor -- start 2024-04-24T09:16:51.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.779+0000 7f33f43c1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f4d299ba30 0x55f4d29a3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:51.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.779+0000 7f33f43c1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f4d299c350 0x55f4d29a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:51.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.779+0000 7f33f43c1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4d29a6a20 0x55f4d29a44c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:51.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.779+0000 7f33f43c1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55f4d29a9810 con 0x55f4d299c350 2024-04-24T09:16:51.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.779+0000 7f33f43c1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f4d28ba690 con 0x55f4d29a6a20 2024-04-24T09:16:51.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.779+0000 7f33f43c1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55f4d29a9990 con 0x55f4d299ba30 2024-04-24T09:16:51.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.779+0000 7f33f43c1740 10 client.0 _ll_register_callbacks cb 0x55f4d28f6650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:16:51.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.780+0000 7f33eb200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4d29a6a20 0x55f4d29a44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:51.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.780+0000 7f33eb200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4d29a6a20 0x55f4d29a44c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:43418/0 (socket says 172.21.15.83:43418) 2024-04-24T09:16:51.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.780+0000 7f33eb200640 1 -- 172.21.15.83:0/718149993 learned_addr learned my addr 172.21.15.83:0/718149993 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:16:51.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.780+0000 7f33e9e00640 1 --2- 172.21.15.83:0/718149993 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f4d299c350 0x55f4d29a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:51.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.780+0000 7f33eb200640 1 --2- 172.21.15.83:0/718149993 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f4d299ba30 0x55f4d29a3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:51.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.780+0000 7f33eb200640 1 -- 172.21.15.83:0/718149993 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f4d299ba30 msgr2=0x55f4d29a3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:51.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.780+0000 7f33eb200640 1 --2- 172.21.15.83:0/718149993 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f4d299ba30 0x55f4d29a3a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:51.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.780+0000 7f33eb200640 1 -- 172.21.15.83:0/718149993 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f4d299c350 msgr2=0x55f4d29a3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:51.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.780+0000 7f33eb200640 1 --2- 172.21.15.83:0/718149993 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f4d299c350 0x55f4d29a3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:51.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.780+0000 7f33eb200640 1 -- 172.21.15.83:0/718149993 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4d28b9920 con 0x55f4d29a6a20 2024-04-24T09:16:51.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.781+0000 7f33eb200640 1 --2- 172.21.15.83:0/718149993 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4d29a6a20 0x55f4d29a44c0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f33e4009900 tx=0x7f33e4014830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=338813110717ee12 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:51.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.781+0000 7f33cbe00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:16:51.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.781+0000 7f33cbe00640 1 -- 172.21.15.83:0/718149993 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33e40139a0 con 0x55f4d29a6a20 2024-04-24T09:16:51.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.781+0000 7f33f43c1740 1 -- 172.21.15.83:0/718149993 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33e4019070 con 0x55f4d29a6a20 2024-04-24T09:16:51.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.781+0000 7f33cbe00640 1 -- 172.21.15.83:0/718149993 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(132..132 src has 1..132) ==== 9583+0+0 (secure 0 0 0) 0x7f33e4013b40 con 0x55f4d29a6a20 2024-04-24T09:16:51.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.782+0000 7f33cbe00640 1 -- 172.21.15.83:0/718149993 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f33e4013d60 con 0x55f4d29a6a20 2024-04-24T09:16:51.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.782+0000 7f33f43c1740 10 client.41186 fetch_fsmap learned FSMap version 69 2024-04-24T09:16:51.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.782+0000 7f33f43c1740 1 -- 172.21.15.83:0/718149993 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x55f4d26dd7d0 con 0x55f4d29a6a20 2024-04-24T09:16:51.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.782+0000 7f33cbe00640 1 -- 172.21.15.83:0/718149993 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f33e4019070 con 0x55f4d29a6a20 2024-04-24T09:16:51.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.782+0000 7f33cbe00640 1 -- 172.21.15.83:0/718149993 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 69) ==== 8370+0+0 (secure 0 0 0) 0x7f33e402e480 con 0x55f4d29a6a20 2024-04-24T09:16:51.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.782+0000 7f33f43c1740 10 client.41186 fetch_fsmap finished waiting for FSMap version 69 2024-04-24T09:16:51.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.782+0000 7f33f43c1740 10 client.41186 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:16:51.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.783+0000 7f33f43c1740 20 client.41186 populate_metadata read hostname 'smithi083' 2024-04-24T09:16:51.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.783+0000 7f33f43c1740 1 --2- 172.21.15.83:0/718149993 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55f4d29b6da0 0x55f4d29b9190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:51.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.783+0000 7f33f43c1740 4 client.41186 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:16:51.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.783+0000 7f33f43c1740 1 -- 172.21.15.83:0/718149993 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f4d290ec10 con 0x55f4d29b6da0 2024-04-24T09:16:51.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.783+0000 7f33e9e00640 1 --2- 172.21.15.83:0/718149993 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55f4d29b6da0 0x55f4d29b9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:51.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.784+0000 7f33e9e00640 1 --2- 172.21.15.83:0/718149993 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55f4d29b6da0 0x55f4d29b9190 crc :-1 s=READY pgs=619 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b34867555fd1644e server_cookie=8a3a73ab5e188b68 in_seq=0 out_seq=0 2024-04-24T09:16:51.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.784+0000 7f33cbe00640 10 client.41186 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:16:51.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.788+0000 7f33cbe00640 1 -- 172.21.15.83:0/718149993 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f4d290ec10 con 0x55f4d29b6da0 2024-04-24T09:16:51.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.788+0000 7f33cbe00640 10 client.41186 handle_command_reply: tid=0 2024-04-24T09:16:51.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.788+0000 7f33f43c1740 1 client.41186 shutdown 2024-04-24T09:16:51.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.788+0000 7f33f43c1740 1 -- 172.21.15.83:0/718149993 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4d29a6a20 msgr2=0x55f4d29a44c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:51.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.789+0000 7f33f43c1740 1 --2- 172.21.15.83:0/718149993 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4d29a6a20 0x55f4d29a44c0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f33e4009900 tx=0x7f33e4014830 comp rx=0 tx=0).stop 2024-04-24T09:16:51.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.789+0000 7f33f43c1740 1 -- 172.21.15.83:0/718149993 shutdown_connections 2024-04-24T09:16:51.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.789+0000 7f33f43c1740 1 --2- 172.21.15.83:0/718149993 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55f4d29b6da0 0x55f4d29b9190 crc :-1 s=READY pgs=619 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:51.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.789+0000 7f33f43c1740 1 --2- 172.21.15.83:0/718149993 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4d29a6a20 0x55f4d29a44c0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:51.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.789+0000 7f33f43c1740 1 --2- 172.21.15.83:0/718149993 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f4d299c350 0x55f4d29a3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:51.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.789+0000 7f33f43c1740 1 --2- 172.21.15.83:0/718149993 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f4d299ba30 0x55f4d29a3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:51.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.789+0000 7f33cbe00640 0 client.41186 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:16:51.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.789+0000 7f33f43c1740 1 -- 172.21.15.83:0/718149993 >> 172.21.15.83:0/718149993 conn(0x55f4d2997d50 msgr2=0x55f4d299db50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:51.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.789+0000 7f33f43c1740 1 -- 172.21.15.83:0/718149993 shutdown_connections 2024-04-24T09:16:51.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.789+0000 7f33f43c1740 1 -- 172.21.15.83:0/718149993 wait complete. 2024-04-24T09:16:51.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.789+0000 7f33f43c1740 20 client.41186 trim_cache size 0 max 16384 2024-04-24T09:16:51.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.821+0000 7f33f43c1740 1 Processor -- start 2024-04-24T09:16:51.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.822+0000 7f33f43c1740 1 -- start start 2024-04-24T09:16:51.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.822+0000 7f33f43c1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f4d299ba30 0x55f4d29a2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:51.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.822+0000 7f33f43c1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f4d299c350 0x55f4d29a42e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:51.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.822+0000 7f33f43c1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4d29a6a20 0x55f4d29be7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:51.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.822+0000 7f33f43c1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f33ec1b5ad0 con 0x55f4d299c350 2024-04-24T09:16:51.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.822+0000 7f33f43c1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f33ec1b7670 con 0x55f4d29a6a20 2024-04-24T09:16:51.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.822+0000 7f33f43c1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f33ec1b61a0 con 0x55f4d299ba30 2024-04-24T09:16:51.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.822+0000 7f33e9e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f4d299c350 0x55f4d29a42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:51.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.822+0000 7f33e9e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f4d299c350 0x55f4d29a42e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36178/0 (socket says 172.21.15.83:36178) 2024-04-24T09:16:51.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.822+0000 7f33e9e00640 1 -- 172.21.15.83:0/1526928887 learned_addr learned my addr 172.21.15.83:0/1526928887 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:16:51.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.823+0000 7f33eb200640 1 --2- 172.21.15.83:0/1526928887 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4d29a6a20 0x55f4d29be7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:51.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.823+0000 7f33eb200640 1 --2- 172.21.15.83:0/1526928887 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f4d299ba30 0x55f4d29a2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:51.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.823+0000 7f33e9e00640 1 -- 172.21.15.83:0/1526928887 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f4d299ba30 msgr2=0x55f4d29a2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:51.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.823+0000 7f33e9e00640 1 --2- 172.21.15.83:0/1526928887 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f4d299ba30 0x55f4d29a2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:51.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.823+0000 7f33e9e00640 1 -- 172.21.15.83:0/1526928887 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4d29a6a20 msgr2=0x55f4d29be7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:51.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.823+0000 7f33e9e00640 1 --2- 172.21.15.83:0/1526928887 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4d29a6a20 0x55f4d29be7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:51.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.823+0000 7f33e9e00640 1 -- 172.21.15.83:0/1526928887 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4d26133f0 con 0x55f4d299c350 2024-04-24T09:16:51.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.824+0000 7f33e9e00640 1 --2- 172.21.15.83:0/1526928887 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f4d299c350 0x55f4d29a42e0 secure :-1 s=READY pgs=2815 cs=0 l=1 rev1=1 crypto rx=0x55f4d29a32a0 tx=0x7f33d800b420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b3da3428a5deef8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:51.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.824+0000 7f33d6a00640 1 -- 172.21.15.83:0/1526928887 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33d801f070 con 0x55f4d299c350 2024-04-24T09:16:51.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.824+0000 7f33d6a00640 1 -- 172.21.15.83:0/1526928887 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f33d801bdc0 con 0x55f4d299c350 2024-04-24T09:16:51.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.824+0000 7f33d6a00640 1 -- 172.21.15.83:0/1526928887 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33d8011a20 con 0x55f4d299c350 2024-04-24T09:16:51.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.825+0000 7f33f43c1740 1 -- 172.21.15.83:0/1526928887 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f4d299c350 msgr2=0x55f4d29a42e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:51.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.825+0000 7f33f43c1740 1 --2- 172.21.15.83:0/1526928887 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f4d299c350 0x55f4d29a42e0 secure :-1 s=READY pgs=2815 cs=0 l=1 rev1=1 crypto rx=0x55f4d29a32a0 tx=0x7f33d800b420 comp rx=0 tx=0).stop 2024-04-24T09:16:51.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.825+0000 7f33f43c1740 1 -- 172.21.15.83:0/1526928887 shutdown_connections 2024-04-24T09:16:51.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.825+0000 7f33f43c1740 1 --2- 172.21.15.83:0/1526928887 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4d29a6a20 0x55f4d29be7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:51.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.825+0000 7f33f43c1740 1 --2- 172.21.15.83:0/1526928887 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f4d299c350 0x55f4d29a42e0 unknown :-1 s=CLOSED pgs=2815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:51.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.825+0000 7f33f43c1740 1 --2- 172.21.15.83:0/1526928887 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f4d299ba30 0x55f4d29a2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:51.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.825+0000 7f33f43c1740 1 -- 172.21.15.83:0/1526928887 >> 172.21.15.83:0/1526928887 conn(0x55f4d2997d50 msgr2=0x55f4d2996740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:51.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.825+0000 7f33f43c1740 1 -- 172.21.15.83:0/1526928887 shutdown_connections 2024-04-24T09:16:51.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.826+0000 7f33f43c1740 1 -- 172.21.15.83:0/1526928887 wait complete. 2024-04-24T09:16:51.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.826+0000 7f33f43c1740 1 -- start start 2024-04-24T09:16:51.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.826+0000 7f33f43c1740 1 Processor -- start 2024-04-24T09:16:51.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.827+0000 7f33f43c1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f4d299ba30 0x55f4d29a0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:51.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.827+0000 7f33f43c1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4d299c350 0x55f4d29a0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:51.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.827+0000 7f33f43c1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f4d29a6a20 0x55f4d29a1290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:51.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.827+0000 7f33f43c1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f33ec1b6fa0 con 0x55f4d299ba30 2024-04-24T09:16:51.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.827+0000 7f33f43c1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f33ec1b68d0 con 0x55f4d299c350 2024-04-24T09:16:51.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.827+0000 7f33f43c1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f33ec1b3ad0 con 0x55f4d29a6a20 2024-04-24T09:16:51.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.827+0000 7f33f43c1740 10 client.0 _ll_register_callbacks cb 0x7f33e4009e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:16:51.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.827+0000 7f33eb200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f4d299ba30 0x55f4d29a0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:51.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.827+0000 7f33eb200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f4d299ba30 0x55f4d29a0810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36194/0 (socket says 172.21.15.83:36194) 2024-04-24T09:16:51.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.827+0000 7f33eb200640 1 -- 172.21.15.83:0/1988256804 learned_addr learned my addr 172.21.15.83:0/1988256804 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:16:51.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.827+0000 7f33e9e00640 1 --2- 172.21.15.83:0/1988256804 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4d299c350 0x55f4d29a0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:51.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.828+0000 7f33eb200640 1 -- 172.21.15.83:0/1988256804 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f4d29a6a20 msgr2=0x55f4d29a1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:51.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.828+0000 7f33eb200640 1 --2- 172.21.15.83:0/1988256804 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f4d29a6a20 0x55f4d29a1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:51.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.828+0000 7f33eb200640 1 -- 172.21.15.83:0/1988256804 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4d299c350 msgr2=0x55f4d29a0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:51.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.828+0000 7f33eb200640 1 --2- 172.21.15.83:0/1988256804 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4d299c350 0x55f4d29a0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:51.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.828+0000 7f33eb200640 1 -- 172.21.15.83:0/1988256804 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4d25d30f0 con 0x55f4d299ba30 2024-04-24T09:16:51.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.828+0000 7f33e9e00640 1 --2- 172.21.15.83:0/1988256804 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4d299c350 0x55f4d29a0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:16:51.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.828+0000 7f33eb200640 1 --2- 172.21.15.83:0/1988256804 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f4d299ba30 0x55f4d29a0810 secure :-1 s=READY pgs=2816 cs=0 l=1 rev1=1 crypto rx=0x7f33e400bfd0 tx=0x7f33e4013f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d6d32cac290aaa5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:51.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.828+0000 7f33cbe00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:16:51.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.829+0000 7f33cbe00640 1 -- 172.21.15.83:0/1988256804 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33e400ad50 con 0x55f4d299ba30 2024-04-24T09:16:51.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.829+0000 7f33cbe00640 1 -- 172.21.15.83:0/1988256804 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(132..132 src has 1..132) ==== 9583+0+0 (secure 0 0 0) 0x7f33e4014420 con 0x55f4d299ba30 2024-04-24T09:16:51.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.829+0000 7f33f43c1740 1 -- 172.21.15.83:0/1988256804 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33d801bdc0 con 0x55f4d299ba30 2024-04-24T09:16:51.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.829+0000 7f33cbe00640 1 -- 172.21.15.83:0/1988256804 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f33e402ed20 con 0x55f4d299ba30 2024-04-24T09:16:51.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.829+0000 7f33f43c1740 10 client.24135 fetch_fsmap learned FSMap version 69 2024-04-24T09:16:51.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.829+0000 7f33f43c1740 1 -- 172.21.15.83:0/1988256804 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x55f4d28c6810 con 0x55f4d299ba30 2024-04-24T09:16:51.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.830+0000 7f33cbe00640 1 -- 172.21.15.83:0/1988256804 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f33d801bdc0 con 0x55f4d299ba30 2024-04-24T09:16:51.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.830+0000 7f33cbe00640 1 -- 172.21.15.83:0/1988256804 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 69) ==== 8370+0+0 (secure 0 0 0) 0x7f33e4036c40 con 0x55f4d299ba30 2024-04-24T09:16:51.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.830+0000 7f33f43c1740 10 client.24135 fetch_fsmap finished waiting for FSMap version 69 2024-04-24T09:16:51.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.830+0000 7f33f43c1740 10 client.24135 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:16:51.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.830+0000 7f33f43c1740 20 client.24135 populate_metadata read hostname 'smithi083' 2024-04-24T09:16:51.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.830+0000 7f33f43c1740 1 --2- 172.21.15.83:0/1988256804 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55f4d2a1d8c0 0x55f4d2a1fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:51.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.830+0000 7f33f43c1740 4 client.24135 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T09:16:51.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.830+0000 7f33f43c1740 1 -- 172.21.15.83:0/1988256804 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f4d2908a80 con 0x55f4d2a1d8c0 2024-04-24T09:16:51.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.831+0000 7f33e9e00640 1 --2- 172.21.15.83:0/1988256804 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55f4d2a1d8c0 0x55f4d2a1fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:51.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.831+0000 7f33cbe00640 10 client.24135 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:16:51.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.831+0000 7f33e9e00640 1 --2- 172.21.15.83:0/1988256804 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55f4d2a1d8c0 0x55f4d2a1fce0 crc :-1 s=READY pgs=620 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=384cdee01c45c9cc server_cookie=6bd0d45207050c8a in_seq=0 out_seq=0 2024-04-24T09:16:51.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.832+0000 7f33cbe00640 1 -- 172.21.15.83:0/1988256804 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55f4d2908a80 con 0x55f4d2a1d8c0 2024-04-24T09:16:51.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.832+0000 7f33cbe00640 10 client.24135 handle_command_reply: tid=0 2024-04-24T09:16:51.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.832+0000 7f33f43c1740 1 client.24135 shutdown 2024-04-24T09:16:51.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.832+0000 7f33f43c1740 1 -- 172.21.15.83:0/1988256804 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f4d299ba30 msgr2=0x55f4d29a0810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:51.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.832+0000 7f33f43c1740 1 --2- 172.21.15.83:0/1988256804 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f4d299ba30 0x55f4d29a0810 secure :-1 s=READY pgs=2816 cs=0 l=1 rev1=1 crypto rx=0x7f33e400bfd0 tx=0x7f33e4013f00 comp rx=0 tx=0).stop 2024-04-24T09:16:51.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.833+0000 7f33f43c1740 1 -- 172.21.15.83:0/1988256804 shutdown_connections 2024-04-24T09:16:51.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.833+0000 7f33f43c1740 1 --2- 172.21.15.83:0/1988256804 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55f4d2a1d8c0 0x55f4d2a1fce0 crc :-1 s=READY pgs=620 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:51.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.833+0000 7f33f43c1740 1 --2- 172.21.15.83:0/1988256804 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f4d29a6a20 0x55f4d29a1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:51.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.833+0000 7f33f43c1740 1 --2- 172.21.15.83:0/1988256804 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4d299c350 0x55f4d29a0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:51.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.833+0000 7f33f43c1740 1 --2- 172.21.15.83:0/1988256804 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f4d299ba30 0x55f4d29a0810 unknown :-1 s=CLOSED pgs=2816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:51.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.833+0000 7f33cbe00640 0 client.24135 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:16:51.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.833+0000 7f33f43c1740 1 -- 172.21.15.83:0/1988256804 >> 172.21.15.83:0/1988256804 conn(0x55f4d2997d50 msgr2=0x55f4d2996740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:51.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.833+0000 7f33f43c1740 1 -- 172.21.15.83:0/1988256804 shutdown_connections 2024-04-24T09:16:51.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.833+0000 7f33f43c1740 1 -- 172.21.15.83:0/1988256804 wait complete. 2024-04-24T09:16:51.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.833+0000 7f33f43c1740 20 client.24135 trim_cache size 0 max 16384 2024-04-24T09:16:51.834 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T09:16:51.834 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T09:16:51.834 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-04-24T09:16:51.834 INFO:teuthology.orchestra.run.smithi083.stdout: "04e42236-ddde-448f-8d34-4fa6b0ac2baf": { 2024-04-24T09:16:51.835 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-04-24T09:16:51.835 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "04e42236-ddde-448f-8d34-4fa6b0ac2baf", 2024-04-24T09:16:51.835 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-04-24T09:16:51.835 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:16:51.835 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:16:51.835 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T09:16:51.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.834+0000 7f33f0a00640 1 -- 172.21.15.83:0/2490704545 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f33d00a3da0 msgr2=0x7f33d00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:51.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.834+0000 7f33f0a00640 1 --2- 172.21.15.83:0/2490704545 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f33d00a3da0 0x7f33d00a6260 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x55f4d299cff0 tx=0x7f33dc009560 comp rx=0 tx=0).stop 2024-04-24T09:16:51.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.834+0000 7f33f0a00640 1 -- 172.21.15.83:0/2490704545 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f33ec076530 msgr2=0x7f33ec19ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:51.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.834+0000 7f33f0a00640 1 --2- 172.21.15.83:0/2490704545 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f33ec076530 0x7f33ec19ed70 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f33dc004770 tx=0x7f33dc003ca0 comp rx=0 tx=0).stop 2024-04-24T09:16:51.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.834+0000 7f33f0a00640 1 -- 172.21.15.83:0/2490704545 shutdown_connections 2024-04-24T09:16:51.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.834+0000 7f33f0a00640 1 --2- 172.21.15.83:0/2490704545 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f33d00a3da0 0x7f33d00a6260 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:51.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.834+0000 7f33f0a00640 1 --2- 172.21.15.83:0/2490704545 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33ec112d50 0x7f33ec1a3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:51.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.834+0000 7f33f0a00640 1 --2- 172.21.15.83:0/2490704545 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f33ec076e50 0x7f33ec19f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:51.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.834+0000 7f33f0a00640 1 --2- 172.21.15.83:0/2490704545 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f33ec076530 0x7f33ec19ed70 secure :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f33dc004770 tx=0x7f33dc003ca0 comp rx=0 tx=0).stop 2024-04-24T09:16:51.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.834+0000 7f33f0a00640 1 -- 172.21.15.83:0/2490704545 >> 172.21.15.83:0/2490704545 conn(0x7f33ec0ffb40 msgr2=0x7f33ec101f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:51.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.834+0000 7f33f0a00640 1 -- 172.21.15.83:0/2490704545 shutdown_connections 2024-04-24T09:16:51.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:16:51.834+0000 7f33f0a00640 1 -- 172.21.15.83:0/2490704545 wait complete. 2024-04-24T09:16:51.856 INFO:tasks.cephfs.filesystem:scrub status for tag:04e42236-ddde-448f-8d34-4fa6b0ac2baf - {'path': '/', 'tag': '04e42236-ddde-448f-8d34-4fa6b0ac2baf', 'options': 'recursive,force'} 2024-04-24T09:16:52.223 INFO:tasks.workunit.client.0.smithi083.stdout: 10 83408 6.51 MB/sec execute 563 sec latency 181.832 ms 2024-04-24T09:16:52.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:16:52 smithi138 ceph-mon[31441]: pgmap v798: 129 pgs: 129 active+clean; 1.2 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.7 MiB/s wr, 248 op/s 2024-04-24T09:16:52.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:16:52 smithi138 ceph-mon[31441]: scrub queued for path: / 2024-04-24T09:16:52.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:16:52 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:16:52.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:16:52 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:16:52.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:16:52 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:16:52.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:16:52 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:16:52.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:16:52 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:16:52.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:16:52 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:16:52.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:16:52 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:16:52.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:16:52 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:16:52.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:16:52 smithi189 ceph-mon[31791]: pgmap v798: 129 pgs: 129 active+clean; 1.2 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.7 MiB/s wr, 248 op/s 2024-04-24T09:16:52.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:16:52 smithi189 ceph-mon[31791]: scrub queued for path: / 2024-04-24T09:16:52.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:16:52 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:16:52.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:16:52 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:16:52.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:16:52 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:16:52.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:16:52 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:16:52.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:16:52 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:16:52.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:16:52 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:16:52.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:16:52 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:16:52.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:16:52 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:16:52.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:16:52 smithi083 ceph-mon[27907]: pgmap v798: 129 pgs: 129 active+clean; 1.2 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.7 MiB/s wr, 248 op/s 2024-04-24T09:16:52.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:16:52 smithi083 ceph-mon[27907]: scrub queued for path: / 2024-04-24T09:16:52.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:16:52 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:16:52.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:16:52 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:16:52.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:16:52 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:16:52.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:16:52 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:16:52.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:16:52 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:16:52.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:16:52 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:16:52.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:16:52 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:16:52.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:16:52 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:16:53.224 INFO:tasks.workunit.client.0.smithi083.stdout: 10 83495 6.51 MB/sec execute 564 sec latency 425.698 ms 2024-04-24T09:16:53.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:16:53 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:16:53.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:16:53 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:16:53.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:16:53 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:16:53.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:16:53 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:16:53.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:16:53 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:16:53.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:16:53 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:16:53.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:16:53 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:16:53.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:16:53 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:16:53.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:16:53 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:16:53.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:16:53 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:16:53.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:16:53 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:16:53.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:16:53 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:16:53.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:16:53 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:16:53.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:16:53 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:16:53.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:16:53 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:16:53.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:16:53 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:16:53.459 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:16:53 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:16:53.459 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:16:53 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:16:53.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:16:53 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:16:53.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:16:53 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:16:53.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:16:53 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:16:53.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:16:53 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:16:53.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:16:53 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:16:53.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:16:53 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:16:53.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:16:53 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:16:53.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:16:53 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:16:53.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:16:53 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:16:54.224 INFO:tasks.workunit.client.0.smithi083.stdout: 10 83577 6.50 MB/sec execute 565 sec latency 449.056 ms 2024-04-24T09:16:54.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:16:54 smithi138 ceph-mon[31441]: pgmap v799: 129 pgs: 129 active+clean; 1.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 5.6 MiB/s wr, 377 op/s 2024-04-24T09:16:54.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:16:54 smithi189 ceph-mon[31791]: pgmap v799: 129 pgs: 129 active+clean; 1.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 5.6 MiB/s wr, 377 op/s 2024-04-24T09:16:54.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:16:54 smithi083 ceph-mon[27907]: pgmap v799: 129 pgs: 129 active+clean; 1.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 5.6 MiB/s wr, 377 op/s 2024-04-24T09:16:55.224 INFO:tasks.workunit.client.0.smithi083.stdout: 10 83700 6.50 MB/sec execute 566 sec latency 239.832 ms 2024-04-24T09:16:56.225 INFO:tasks.workunit.client.0.smithi083.stdout: 10 83821 6.50 MB/sec execute 567 sec latency 75.490 ms 2024-04-24T09:16:56.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:16:56 smithi138 ceph-mon[31441]: pgmap v800: 129 pgs: 129 active+clean; 1.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.3 MiB/s wr, 270 op/s 2024-04-24T09:16:56.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:16:56 smithi189 ceph-mon[31791]: pgmap v800: 129 pgs: 129 active+clean; 1.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.3 MiB/s wr, 270 op/s 2024-04-24T09:16:56.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:16:56 smithi083 ceph-mon[27907]: pgmap v800: 129 pgs: 129 active+clean; 1.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.3 MiB/s wr, 270 op/s 2024-04-24T09:16:57.226 INFO:tasks.workunit.client.0.smithi083.stdout: 10 83934 6.49 MB/sec execute 568 sec latency 85.142 ms 2024-04-24T09:16:58.226 INFO:tasks.workunit.client.0.smithi083.stdout: 10 84051 6.50 MB/sec execute 569 sec latency 174.488 ms 2024-04-24T09:16:58.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:16:58 smithi138 ceph-mon[31441]: pgmap v801: 129 pgs: 129 active+clean; 1.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.3 MiB/s wr, 270 op/s 2024-04-24T09:16:58.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:16:58 smithi189 ceph-mon[31791]: pgmap v801: 129 pgs: 129 active+clean; 1.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.3 MiB/s wr, 270 op/s 2024-04-24T09:16:58.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:16:58 smithi083 ceph-mon[27907]: pgmap v801: 129 pgs: 129 active+clean; 1.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.3 MiB/s wr, 270 op/s 2024-04-24T09:16:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.669+0000 7f30d7e00640 1 -- 172.21.15.83:0/3248046775 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f30d8076530 msgr2=0x7f30d8076910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.669+0000 7f30d7e00640 1 --2- 172.21.15.83:0/3248046775 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f30d8076530 0x7f30d8076910 secure :-1 s=READY pgs=2817 cs=0 l=1 rev1=1 crypto rx=0x7f30d0009ca0 tx=0x7f30d002a8c0 comp rx=0 tx=0).stop 2024-04-24T09:16:58.670 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.670+0000 7f30d7e00640 1 -- 172.21.15.83:0/3248046775 shutdown_connections 2024-04-24T09:16:58.670 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.670+0000 7f30d7e00640 1 --2- 172.21.15.83:0/3248046775 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f30d8112cd0 0x7f30d81150c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:58.670 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.670+0000 7f30d7e00640 1 --2- 172.21.15.83:0/3248046775 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f30d8076e50 0x7f30d8112790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:58.670 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.670+0000 7f30d7e00640 1 --2- 172.21.15.83:0/3248046775 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f30d8076530 0x7f30d8076910 unknown :-1 s=CLOSED pgs=2817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:58.670 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.670+0000 7f30d7e00640 1 -- 172.21.15.83:0/3248046775 >> 172.21.15.83:0/3248046775 conn(0x7f30d80ffae0 msgr2=0x7f30d8101f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:58.671 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.670+0000 7f30d7e00640 1 -- 172.21.15.83:0/3248046775 shutdown_connections 2024-04-24T09:16:58.671 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.670+0000 7f30d7e00640 1 -- 172.21.15.83:0/3248046775 wait complete. 2024-04-24T09:16:58.672 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.672+0000 7f30d7e00640 1 Processor -- start 2024-04-24T09:16:58.672 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.672+0000 7f30d7e00640 1 -- start start 2024-04-24T09:16:58.673 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.672+0000 7f30d7e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f30d8076530 0x7f30d81a4950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:58.673 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.672+0000 7f30d7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f30d8076e50 0x7f30d81a4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:58.673 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.673+0000 7f30d7e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f30d8112cd0 0x7f30d819eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:58.673 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.673+0000 7f30d7e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f30d81187a0 con 0x7f30d8076e50 2024-04-24T09:16:58.673 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.673+0000 7f30d7e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f30d8118620 con 0x7f30d8076530 2024-04-24T09:16:58.673 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.673+0000 7f30d7e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f30d8118920 con 0x7f30d8112cd0 2024-04-24T09:16:58.674 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.673+0000 7f30d6a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f30d8076530 0x7f30d81a4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:58.674 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.673+0000 7f30d6a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f30d8076530 0x7f30d81a4950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:43454/0 (socket says 172.21.15.83:43454) 2024-04-24T09:16:58.674 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.673+0000 7f30d6a00640 1 -- 172.21.15.83:0/1945603668 learned_addr learned my addr 172.21.15.83:0/1945603668 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:16:58.674 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.673+0000 7f30d6000640 1 --2- 172.21.15.83:0/1945603668 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f30d8076e50 0x7f30d81a4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:58.674 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.674+0000 7f30d7400640 1 --2- 172.21.15.83:0/1945603668 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f30d8112cd0 0x7f30d819eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:58.675 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.674+0000 7f30d6000640 1 -- 172.21.15.83:0/1945603668 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f30d8112cd0 msgr2=0x7f30d819eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:58.675 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.674+0000 7f30d6000640 1 --2- 172.21.15.83:0/1945603668 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f30d8112cd0 0x7f30d819eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:58.675 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.674+0000 7f30d6000640 1 -- 172.21.15.83:0/1945603668 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f30d8076530 msgr2=0x7f30d81a4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:58.675 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.674+0000 7f30d6000640 1 --2- 172.21.15.83:0/1945603668 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f30d8076530 0x7f30d81a4950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:58.675 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.674+0000 7f30d6000640 1 -- 172.21.15.83:0/1945603668 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30d819f280 con 0x7f30d8076e50 2024-04-24T09:16:58.675 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.674+0000 7f30d6a00640 1 --2- 172.21.15.83:0/1945603668 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f30d8076530 0x7f30d81a4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:16:58.675 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.674+0000 7f30d6000640 1 --2- 172.21.15.83:0/1945603668 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f30d8076e50 0x7f30d81a4e90 secure :-1 s=READY pgs=2818 cs=0 l=1 rev1=1 crypto rx=0x7f30c800b3f0 tx=0x7f30c800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:58.675 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.675+0000 7f30cf400640 1 -- 172.21.15.83:0/1945603668 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c8015020 con 0x7f30d8076e50 2024-04-24T09:16:58.676 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.675+0000 7f30d7e00640 1 -- 172.21.15.83:0/1945603668 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30d819f570 con 0x7f30d8076e50 2024-04-24T09:16:58.676 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.675+0000 7f30cf400640 1 -- 172.21.15.83:0/1945603668 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f30c8011ce0 con 0x7f30d8076e50 2024-04-24T09:16:58.676 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.675+0000 7f30cf400640 1 -- 172.21.15.83:0/1945603668 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c8010d80 con 0x7f30d8076e50 2024-04-24T09:16:58.676 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.675+0000 7f30d7e00640 1 -- 172.21.15.83:0/1945603668 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30d81ab810 con 0x7f30d8076e50 2024-04-24T09:16:58.679 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.679+0000 7f30cf400640 1 -- 172.21.15.83:0/1945603668 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f30c8018020 con 0x7f30d8076e50 2024-04-24T09:16:58.681 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.681+0000 7f30e09b3740 1 Processor -- start 2024-04-24T09:16:58.681 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.681+0000 7f30e09b3740 1 -- start start 2024-04-24T09:16:58.681 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.681+0000 7f30cf400640 1 --2- 172.21.15.83:0/1945603668 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f30bc0a3de0 0x7f30bc0a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:58.681 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.681+0000 7f30e09b3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c82b76f3c0 0x55c82b76f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:58.682 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.681+0000 7f30e09b3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c82b76fce0 0x55c82b77a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:58.682 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.681+0000 7f30e09b3740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c82b77a5a0 0x55c82b77c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:58.682 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.681+0000 7f30e09b3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c82b6d5900 con 0x55c82b76f3c0 2024-04-24T09:16:58.682 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.681+0000 7f30e09b3740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c82b67bf20 con 0x55c82b77a5a0 2024-04-24T09:16:58.682 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.681+0000 7f30e09b3740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55c82b6d18c0 con 0x55c82b76fce0 2024-04-24T09:16:58.683 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.681+0000 7f30d6a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c82b76f3c0 0x55c82b76f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:58.683 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.681+0000 7f30cf400640 1 -- 172.21.15.83:0/1945603668 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 9583+0+0 (secure 0 0 0) 0x7f30c80d0100 con 0x7f30d8076e50 2024-04-24T09:16:58.683 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.682+0000 7f30d7400640 1 --2- 172.21.15.83:0/1945603668 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f30bc0a3de0 0x7f30bc0a62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:58.683 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.682+0000 7f30d6a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c82b76f3c0 0x55c82b76f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36222/0 (socket says 172.21.15.83:36222) 2024-04-24T09:16:58.683 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.682+0000 7f30d6a00640 1 -- 172.21.15.83:0/1479411631 learned_addr learned my addr 172.21.15.83:0/1479411631 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:16:58.684 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.682+0000 7f30d7400640 1 --2- 172.21.15.83:0/1479411631 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c82b77a5a0 0x55c82b77c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:58.684 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.682+0000 7f30d6000640 1 --2- 172.21.15.83:0/1479411631 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c82b76fce0 0x55c82b77a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:58.684 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.683+0000 7f30d7400640 1 --2- 172.21.15.83:0/1945603668 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f30bc0a3de0 0x7f30bc0a62a0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f30d806b1d0 tx=0x7f30c4011440 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:58.684 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.683+0000 7f30d6a00640 1 -- 172.21.15.83:0/1479411631 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c82b76fce0 msgr2=0x55c82b77a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:58.684 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.683+0000 7f30d6a00640 1 --2- 172.21.15.83:0/1479411631 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c82b76fce0 0x55c82b77a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:58.684 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.683+0000 7f30d6a00640 1 -- 172.21.15.83:0/1479411631 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c82b77a5a0 msgr2=0x55c82b77c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:58.684 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.683+0000 7f30d6a00640 1 --2- 172.21.15.83:0/1479411631 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c82b77a5a0 0x55c82b77c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:58.684 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.683+0000 7f30d6a00640 1 -- 172.21.15.83:0/1479411631 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c82b6a0e10 con 0x55c82b76f3c0 2024-04-24T09:16:58.684 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.683+0000 7f30d6a00640 1 --2- 172.21.15.83:0/1479411631 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c82b76f3c0 0x55c82b76f7a0 secure :-1 s=READY pgs=2819 cs=0 l=1 rev1=1 crypto rx=0x7f30d0009dc0 tx=0x7f30d0009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d5c624562feaa6b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:58.685 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.683+0000 7f30c3e00640 1 -- 172.21.15.83:0/1479411631 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30d0031d70 con 0x55c82b76f3c0 2024-04-24T09:16:58.685 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.683+0000 7f30c3e00640 1 -- 172.21.15.83:0/1479411631 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f30d0036030 con 0x55c82b76f3c0 2024-04-24T09:16:58.685 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.683+0000 7f30c3e00640 1 -- 172.21.15.83:0/1479411631 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30d0004860 con 0x55c82b76f3c0 2024-04-24T09:16:58.685 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.684+0000 7f30e09b3740 1 -- 172.21.15.83:0/1479411631 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c82b76f3c0 msgr2=0x55c82b76f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:58.685 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.685+0000 7f30e09b3740 1 --2- 172.21.15.83:0/1479411631 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c82b76f3c0 0x55c82b76f7a0 secure :-1 s=READY pgs=2819 cs=0 l=1 rev1=1 crypto rx=0x7f30d0009dc0 tx=0x7f30d0009a20 comp rx=0 tx=0).stop 2024-04-24T09:16:58.685 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.685+0000 7f30e09b3740 1 -- 172.21.15.83:0/1479411631 shutdown_connections 2024-04-24T09:16:58.685 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.685+0000 7f30e09b3740 1 --2- 172.21.15.83:0/1479411631 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c82b77a5a0 0x55c82b77c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:58.685 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.685+0000 7f30e09b3740 1 --2- 172.21.15.83:0/1479411631 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c82b76fce0 0x55c82b77a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:58.686 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.685+0000 7f30e09b3740 1 --2- 172.21.15.83:0/1479411631 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c82b76f3c0 0x55c82b76f7a0 unknown :-1 s=CLOSED pgs=2819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:58.686 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.685+0000 7f30e09b3740 1 -- 172.21.15.83:0/1479411631 >> 172.21.15.83:0/1479411631 conn(0x55c82b76b8d0 msgr2=0x55c82b76bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:58.686 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.685+0000 7f30e09b3740 1 -- 172.21.15.83:0/1479411631 shutdown_connections 2024-04-24T09:16:58.686 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.685+0000 7f30e09b3740 1 -- 172.21.15.83:0/1479411631 wait complete. 2024-04-24T09:16:58.686 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.685+0000 7f30e09b3740 1 -- start start 2024-04-24T09:16:58.686 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.686+0000 7f30e09b3740 1 Processor -- start 2024-04-24T09:16:58.687 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.686+0000 7f30e09b3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c82b76fce0 0x55c82b777650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:58.687 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.686+0000 7f30e09b3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c82b77a5a0 0x55c82b777b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:58.687 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.686+0000 7f30e09b3740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c82b7780d0 0x55c82b7863e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.686+0000 7f30e09b3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c82b77cff0 con 0x55c82b77a5a0 2024-04-24T09:16:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.686+0000 7f30e09b3740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c82b68f2c0 con 0x55c82b7780d0 2024-04-24T09:16:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.686+0000 7f30e09b3740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55c82b77d170 con 0x55c82b76fce0 2024-04-24T09:16:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.686+0000 7f30e09b3740 10 client.0 _ll_register_callbacks cb 0x55c82b3e7a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:16:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.687+0000 7f30d7400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c82b7780d0 0x55c82b7863e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.687+0000 7f30d6000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c82b77a5a0 0x55c82b777b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.687+0000 7f30d7400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c82b7780d0 0x55c82b7863e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:43472/0 (socket says 172.21.15.83:43472) 2024-04-24T09:16:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.687+0000 7f30d7400640 1 -- 172.21.15.83:0/3794077000 learned_addr learned my addr 172.21.15.83:0/3794077000 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:16:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.687+0000 7f30d6a00640 1 --2- 172.21.15.83:0/3794077000 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c82b76fce0 0x55c82b777650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.687+0000 7f30d6000640 1 -- 172.21.15.83:0/3794077000 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c82b76fce0 msgr2=0x55c82b777650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.687+0000 7f30d6000640 1 --2- 172.21.15.83:0/3794077000 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c82b76fce0 0x55c82b777650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.687+0000 7f30d6000640 1 -- 172.21.15.83:0/3794077000 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c82b7780d0 msgr2=0x55c82b7863e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.687+0000 7f30d6000640 1 --2- 172.21.15.83:0/3794077000 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c82b7780d0 0x55c82b7863e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.687+0000 7f30d6000640 1 -- 172.21.15.83:0/3794077000 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c82b69ffd0 con 0x55c82b77a5a0 2024-04-24T09:16:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.688+0000 7f30d6000640 1 --2- 172.21.15.83:0/3794077000 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c82b77a5a0 0x55c82b777b90 secure :-1 s=READY pgs=2820 cs=0 l=1 rev1=1 crypto rx=0x7f30c80cd540 tx=0x7f30c809c5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26314d2944bcf353 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.688+0000 7f30c0c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:16:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.688+0000 7f30c0c00640 1 -- 172.21.15.83:0/3794077000 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c80b5260 con 0x55c82b77a5a0 2024-04-24T09:16:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.688+0000 7f30c0c00640 1 -- 172.21.15.83:0/3794077000 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(132..132 src has 1..132) ==== 9583+0+0 (secure 0 0 0) 0x7f30c8098400 con 0x55c82b77a5a0 2024-04-24T09:16:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.688+0000 7f30e09b3740 1 -- 172.21.15.83:0/3794077000 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30d0036030 con 0x55c82b77a5a0 2024-04-24T09:16:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.689+0000 7f30c0c00640 1 -- 172.21.15.83:0/3794077000 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f30c807abb0 con 0x55c82b77a5a0 2024-04-24T09:16:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.689+0000 7f30e09b3740 10 client.24147 fetch_fsmap learned FSMap version 69 2024-04-24T09:16:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.689+0000 7f30e09b3740 1 -- 172.21.15.83:0/3794077000 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x55c82b3e8210 con 0x55c82b77a5a0 2024-04-24T09:16:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.689+0000 7f30c0c00640 1 -- 172.21.15.83:0/3794077000 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f30d0036030 con 0x55c82b77a5a0 2024-04-24T09:16:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.689+0000 7f30c0c00640 1 -- 172.21.15.83:0/3794077000 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 69) ==== 8370+0+0 (secure 0 0 0) 0x7f30c809b1e0 con 0x55c82b77a5a0 2024-04-24T09:16:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.689+0000 7f30e09b3740 10 client.24147 fetch_fsmap finished waiting for FSMap version 69 2024-04-24T09:16:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.689+0000 7f30e09b3740 10 client.24147 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:16:58.691 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.689+0000 7f30e09b3740 20 client.24147 populate_metadata read hostname 'smithi083' 2024-04-24T09:16:58.691 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.689+0000 7f30e09b3740 1 --2- 172.21.15.83:0/3794077000 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55c82b78aae0 0x55c82b78ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:58.691 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.689+0000 7f30e09b3740 4 client.24147 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:16:58.691 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.689+0000 7f30e09b3740 1 -- 172.21.15.83:0/3794077000 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c82b6e3320 con 0x55c82b78aae0 2024-04-24T09:16:58.691 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.690+0000 7f30d6a00640 1 --2- 172.21.15.83:0/3794077000 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55c82b78aae0 0x55c82b78ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:58.691 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.691+0000 7f30d6a00640 1 --2- 172.21.15.83:0/3794077000 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55c82b78aae0 0x55c82b78ced0 crc :-1 s=READY pgs=621 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74d37fe4ff96e746 server_cookie=bc8b9701eea27b2b in_seq=0 out_seq=0 2024-04-24T09:16:58.691 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.691+0000 7f30c0c00640 10 client.24147 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:16:58.693 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.692+0000 7f30c0c00640 1 -- 172.21.15.83:0/3794077000 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c82b6e3320 con 0x55c82b78aae0 2024-04-24T09:16:58.693 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.692+0000 7f30c0c00640 10 client.24147 handle_command_reply: tid=0 2024-04-24T09:16:58.693 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.692+0000 7f30e09b3740 1 client.24147 shutdown 2024-04-24T09:16:58.693 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.693+0000 7f30e09b3740 1 -- 172.21.15.83:0/3794077000 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c82b77a5a0 msgr2=0x55c82b777b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:58.693 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.693+0000 7f30e09b3740 1 --2- 172.21.15.83:0/3794077000 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c82b77a5a0 0x55c82b777b90 secure :-1 s=READY pgs=2820 cs=0 l=1 rev1=1 crypto rx=0x7f30c80cd540 tx=0x7f30c809c5d0 comp rx=0 tx=0).stop 2024-04-24T09:16:58.693 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.693+0000 7f30e09b3740 1 -- 172.21.15.83:0/3794077000 shutdown_connections 2024-04-24T09:16:58.693 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.693+0000 7f30e09b3740 1 --2- 172.21.15.83:0/3794077000 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55c82b78aae0 0x55c82b78ced0 crc :-1 s=READY pgs=621 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:58.693 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.693+0000 7f30e09b3740 1 --2- 172.21.15.83:0/3794077000 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c82b7780d0 0x55c82b7863e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:58.693 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.693+0000 7f30e09b3740 1 --2- 172.21.15.83:0/3794077000 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c82b77a5a0 0x55c82b777b90 unknown :-1 s=CLOSED pgs=2820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:58.694 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.693+0000 7f30e09b3740 1 --2- 172.21.15.83:0/3794077000 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c82b76fce0 0x55c82b777650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:58.694 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.693+0000 7f30e09b3740 1 -- 172.21.15.83:0/3794077000 >> 172.21.15.83:0/3794077000 conn(0x55c82b76b8d0 msgr2=0x55c82b771600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:58.694 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.693+0000 7f30c0c00640 0 client.24147 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:16:58.694 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.693+0000 7f30e09b3740 1 -- 172.21.15.83:0/3794077000 shutdown_connections 2024-04-24T09:16:58.694 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.693+0000 7f30e09b3740 1 -- 172.21.15.83:0/3794077000 wait complete. 2024-04-24T09:16:58.694 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.694+0000 7f30e09b3740 20 client.24147 trim_cache size 0 max 16384 2024-04-24T09:16:58.717 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.716+0000 7f30e09b3740 1 Processor -- start 2024-04-24T09:16:58.717 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.717+0000 7f30e09b3740 1 -- start start 2024-04-24T09:16:58.717 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.717+0000 7f30e09b3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c82b76fce0 0x55c82b7927e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:58.718 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.717+0000 7f30e09b3740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c82b7780d0 0x55c82b792d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:58.718 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.717+0000 7f30e09b3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c82b77a5a0 0x55c82b785eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:58.718 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.717+0000 7f30e09b3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f30b8010110 con 0x55c82b76fce0 2024-04-24T09:16:58.718 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.717+0000 7f30e09b3740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f30b8011cb0 con 0x55c82b7780d0 2024-04-24T09:16:58.718 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.717+0000 7f30e09b3740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f30b80107e0 con 0x55c82b77a5a0 2024-04-24T09:16:58.718 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.717+0000 7f30d6a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c82b76fce0 0x55c82b7927e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:58.719 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.718+0000 7f30d6a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c82b76fce0 0x55c82b7927e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36230/0 (socket says 172.21.15.83:36230) 2024-04-24T09:16:58.719 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.718+0000 7f30d6a00640 1 -- 172.21.15.83:0/3615686550 learned_addr learned my addr 172.21.15.83:0/3615686550 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:16:58.719 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.718+0000 7f30d6000640 1 --2- 172.21.15.83:0/3615686550 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c82b7780d0 0x55c82b792d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:58.719 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.718+0000 7f30d6a00640 1 -- 172.21.15.83:0/3615686550 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c82b77a5a0 msgr2=0x55c82b785eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:58.719 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.718+0000 7f30d6a00640 1 --2- 172.21.15.83:0/3615686550 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c82b77a5a0 0x55c82b785eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:58.719 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.718+0000 7f30d6a00640 1 -- 172.21.15.83:0/3615686550 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c82b7780d0 msgr2=0x55c82b792d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:58.719 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.718+0000 7f30d6a00640 1 --2- 172.21.15.83:0/3615686550 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c82b7780d0 0x55c82b792d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:58.719 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.718+0000 7f30d6a00640 1 -- 172.21.15.83:0/3615686550 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c82b77cd00 con 0x55c82b76fce0 2024-04-24T09:16:58.719 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.719+0000 7f30d6a00640 1 --2- 172.21.15.83:0/3615686550 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c82b76fce0 0x55c82b7927e0 secure :-1 s=READY pgs=2821 cs=0 l=1 rev1=1 crypto rx=0x7f30d0031930 tx=0x7f30d0004460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57bcb09feaee4ed8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:58.720 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.719+0000 7f30c3400640 1 -- 172.21.15.83:0/3615686550 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30d0031bf0 con 0x55c82b76fce0 2024-04-24T09:16:58.720 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.719+0000 7f30c3400640 1 -- 172.21.15.83:0/3615686550 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f30d0055e50 con 0x55c82b76fce0 2024-04-24T09:16:58.720 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.719+0000 7f30c3400640 1 -- 172.21.15.83:0/3615686550 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30d0003e00 con 0x55c82b76fce0 2024-04-24T09:16:58.720 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.720+0000 7f30e09b3740 1 -- 172.21.15.83:0/3615686550 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c82b76fce0 msgr2=0x55c82b7927e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:58.720 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.720+0000 7f30e09b3740 1 --2- 172.21.15.83:0/3615686550 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c82b76fce0 0x55c82b7927e0 secure :-1 s=READY pgs=2821 cs=0 l=1 rev1=1 crypto rx=0x7f30d0031930 tx=0x7f30d0004460 comp rx=0 tx=0).stop 2024-04-24T09:16:58.721 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.721+0000 7f30e09b3740 1 -- 172.21.15.83:0/3615686550 shutdown_connections 2024-04-24T09:16:58.721 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.721+0000 7f30e09b3740 1 --2- 172.21.15.83:0/3615686550 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c82b77a5a0 0x55c82b785eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:58.721 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.721+0000 7f30e09b3740 1 --2- 172.21.15.83:0/3615686550 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c82b7780d0 0x55c82b792d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:58.721 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.721+0000 7f30e09b3740 1 --2- 172.21.15.83:0/3615686550 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c82b76fce0 0x55c82b7927e0 unknown :-1 s=CLOSED pgs=2821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:58.721 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.721+0000 7f30e09b3740 1 -- 172.21.15.83:0/3615686550 >> 172.21.15.83:0/3615686550 conn(0x55c82b76b8d0 msgr2=0x55c82b775f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:58.722 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.721+0000 7f30e09b3740 1 -- 172.21.15.83:0/3615686550 shutdown_connections 2024-04-24T09:16:58.722 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.722+0000 7f30e09b3740 1 -- 172.21.15.83:0/3615686550 wait complete. 2024-04-24T09:16:58.723 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.722+0000 7f30e09b3740 1 -- start start 2024-04-24T09:16:58.723 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.722+0000 7f30e09b3740 1 Processor -- start 2024-04-24T09:16:58.724 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.723+0000 7f30e09b3740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c82b76fce0 0x55c82b782770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:58.724 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.723+0000 7f30e09b3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c82b7780d0 0x55c82b774050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:58.724 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.723+0000 7f30e09b3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c82b77a5a0 0x55c82b774590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:58.724 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.723+0000 7f30e09b3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f30b80115e0 con 0x55c82b7780d0 2024-04-24T09:16:58.724 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.723+0000 7f30e09b3740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f30b8010f10 con 0x55c82b76fce0 2024-04-24T09:16:58.724 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.723+0000 7f30e09b3740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f30b800e110 con 0x55c82b77a5a0 2024-04-24T09:16:58.724 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.723+0000 7f30e09b3740 10 client.0 _ll_register_callbacks cb 0x7f30c80baf50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:16:58.725 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.724+0000 7f30d6a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c82b76fce0 0x55c82b782770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:58.725 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.724+0000 7f30d6000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c82b7780d0 0x55c82b774050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:58.725 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.724+0000 7f30d6a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c82b76fce0 0x55c82b782770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:43490/0 (socket says 172.21.15.83:43490) 2024-04-24T09:16:58.725 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.724+0000 7f30d6000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c82b7780d0 0x55c82b774050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36246/0 (socket says 172.21.15.83:36246) 2024-04-24T09:16:58.725 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.724+0000 7f30d6000640 1 -- 172.21.15.83:0/1077466103 learned_addr learned my addr 172.21.15.83:0/1077466103 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:16:58.726 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.724+0000 7f30d7400640 1 --2- 172.21.15.83:0/1077466103 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c82b77a5a0 0x55c82b774590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:58.726 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.725+0000 7f30d6000640 1 -- 172.21.15.83:0/1077466103 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c82b77a5a0 msgr2=0x55c82b774590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:58.726 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.725+0000 7f30d6000640 1 --2- 172.21.15.83:0/1077466103 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c82b77a5a0 0x55c82b774590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:58.726 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.725+0000 7f30d6000640 1 -- 172.21.15.83:0/1077466103 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c82b76fce0 msgr2=0x55c82b782770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:16:58.726 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.725+0000 7f30d6000640 1 --2- 172.21.15.83:0/1077466103 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c82b76fce0 0x55c82b782770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:58.726 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.725+0000 7f30d6000640 1 -- 172.21.15.83:0/1077466103 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c82b4b4620 con 0x55c82b7780d0 2024-04-24T09:16:58.726 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.725+0000 7f30d6000640 1 --2- 172.21.15.83:0/1077466103 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c82b7780d0 0x55c82b774050 secure :-1 s=READY pgs=2822 cs=0 l=1 rev1=1 crypto rx=0x7f30c809ba90 tx=0x7f30c80cd930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8192e1441dbd3a1c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:16:58.726 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.725+0000 7f30c0c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:16:58.726 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.725+0000 7f30c0c00640 1 -- 172.21.15.83:0/1077466103 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c80b9060 con 0x55c82b7780d0 2024-04-24T09:16:58.726 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.725+0000 7f30c0c00640 1 -- 172.21.15.83:0/1077466103 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(132..132 src has 1..132) ==== 9583+0+0 (secure 0 0 0) 0x7f30c8098400 con 0x55c82b7780d0 2024-04-24T09:16:58.726 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.726+0000 7f30e09b3740 1 -- 172.21.15.83:0/1077466103 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30d0055e50 con 0x55c82b7780d0 2024-04-24T09:16:58.727 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.726+0000 7f30c0c00640 1 -- 172.21.15.83:0/1077466103 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f30c8098030 con 0x55c82b7780d0 2024-04-24T09:16:58.727 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.726+0000 7f30e09b3740 10 client.24153 fetch_fsmap learned FSMap version 69 2024-04-24T09:16:58.728 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.726+0000 7f30e09b3740 1 -- 172.21.15.83:0/1077466103 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x55c82b801f00 con 0x55c82b7780d0 2024-04-24T09:16:58.728 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.726+0000 7f30c0c00640 1 -- 172.21.15.83:0/1077466103 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f30d0055e50 con 0x55c82b7780d0 2024-04-24T09:16:58.728 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.727+0000 7f30c0c00640 1 -- 172.21.15.83:0/1077466103 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 69) ==== 8370+0+0 (secure 0 0 0) 0x7f30c8082020 con 0x55c82b7780d0 2024-04-24T09:16:58.728 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.727+0000 7f30e09b3740 10 client.24153 fetch_fsmap finished waiting for FSMap version 69 2024-04-24T09:16:58.728 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.727+0000 7f30e09b3740 10 client.24153 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:16:58.728 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.727+0000 7f30e09b3740 20 client.24153 populate_metadata read hostname 'smithi083' 2024-04-24T09:16:58.728 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.727+0000 7f30e09b3740 1 --2- 172.21.15.83:0/1077466103 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55c82b802490 0x55c82b8048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:16:58.728 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.727+0000 7f30e09b3740 4 client.24153 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T09:16:58.728 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.727+0000 7f30e09b3740 1 -- 172.21.15.83:0/1077466103 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c82b6dd190 con 0x55c82b802490 2024-04-24T09:16:58.728 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.727+0000 7f30d6a00640 1 --2- 172.21.15.83:0/1077466103 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55c82b802490 0x55c82b8048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:16:58.729 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.728+0000 7f30d6a00640 1 --2- 172.21.15.83:0/1077466103 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55c82b802490 0x55c82b8048b0 crc :-1 s=READY pgs=622 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de4d819f6bbf8b1f server_cookie=5f8c4b85e934cf57 in_seq=0 out_seq=0 2024-04-24T09:16:58.729 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:58.728+0000 7f30c0c00640 10 client.24153 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:16:59.226 INFO:tasks.workunit.client.0.smithi083.stdout: 10 84122 6.49 MB/sec execute 570 sec latency 596.049 ms 2024-04-24T09:16:59.268 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:59.268+0000 7f30c0c00640 1 -- 172.21.15.83:0/1077466103 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c82b6dd190 con 0x55c82b802490 2024-04-24T09:16:59.268 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:59.268+0000 7f30c0c00640 10 client.24153 handle_command_reply: tid=0 2024-04-24T09:16:59.269 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:59.268+0000 7f30e09b3740 1 client.24153 shutdown 2024-04-24T09:16:59.269 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:59.268+0000 7f30e09b3740 1 -- 172.21.15.83:0/1077466103 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c82b7780d0 msgr2=0x55c82b774050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:59.269 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:59.268+0000 7f30e09b3740 1 --2- 172.21.15.83:0/1077466103 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c82b7780d0 0x55c82b774050 secure :-1 s=READY pgs=2822 cs=0 l=1 rev1=1 crypto rx=0x7f30c809ba90 tx=0x7f30c80cd930 comp rx=0 tx=0).stop 2024-04-24T09:16:59.269 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:59.269+0000 7f30e09b3740 1 -- 172.21.15.83:0/1077466103 shutdown_connections 2024-04-24T09:16:59.269 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:59.269+0000 7f30e09b3740 1 --2- 172.21.15.83:0/1077466103 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55c82b802490 0x55c82b8048b0 crc :-1 s=READY pgs=622 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:59.269 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:59.269+0000 7f30e09b3740 1 --2- 172.21.15.83:0/1077466103 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c82b77a5a0 0x55c82b774590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:59.269 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:59.269+0000 7f30e09b3740 1 --2- 172.21.15.83:0/1077466103 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c82b7780d0 0x55c82b774050 unknown :-1 s=CLOSED pgs=2822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:59.269 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:59.269+0000 7f30c0c00640 0 client.24153 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:16:59.269 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:59.269+0000 7f30e09b3740 1 --2- 172.21.15.83:0/1077466103 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c82b76fce0 0x55c82b782770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:59.270 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:59.269+0000 7f30e09b3740 1 -- 172.21.15.83:0/1077466103 >> 172.21.15.83:0/1077466103 conn(0x55c82b76b8d0 msgr2=0x55c82b770d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:59.270 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:59.269+0000 7f30e09b3740 1 -- 172.21.15.83:0/1077466103 shutdown_connections 2024-04-24T09:16:59.270 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:59.270+0000 7f30e09b3740 1 -- 172.21.15.83:0/1077466103 wait complete. 2024-04-24T09:16:59.270 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:59.270+0000 7f30e09b3740 20 client.24153 trim_cache size 0 max 16384 2024-04-24T09:16:59.270 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T09:16:59.271 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T09:16:59.271 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T09:16:59.271 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T09:16:59.271 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:59.271+0000 7f30d7e00640 1 -- 172.21.15.83:0/1945603668 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f30bc0a3de0 msgr2=0x7f30bc0a62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:59.271 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:59.271+0000 7f30d7e00640 1 --2- 172.21.15.83:0/1945603668 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f30bc0a3de0 0x7f30bc0a62a0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f30d806b1d0 tx=0x7f30c4011440 comp rx=0 tx=0).stop 2024-04-24T09:16:59.271 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:59.271+0000 7f30d7e00640 1 -- 172.21.15.83:0/1945603668 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f30d8076e50 msgr2=0x7f30d81a4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:16:59.271 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:59.271+0000 7f30d7e00640 1 --2- 172.21.15.83:0/1945603668 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f30d8076e50 0x7f30d81a4e90 secure :-1 s=READY pgs=2818 cs=0 l=1 rev1=1 crypto rx=0x7f30c800b3f0 tx=0x7f30c800b8b0 comp rx=0 tx=0).stop 2024-04-24T09:16:59.272 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:59.271+0000 7f30d7e00640 1 -- 172.21.15.83:0/1945603668 shutdown_connections 2024-04-24T09:16:59.272 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:59.271+0000 7f30d7e00640 1 --2- 172.21.15.83:0/1945603668 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f30bc0a3de0 0x7f30bc0a62a0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:59.272 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:59.271+0000 7f30d7e00640 1 --2- 172.21.15.83:0/1945603668 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f30d8112cd0 0x7f30d819eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:59.272 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:59.271+0000 7f30d7e00640 1 --2- 172.21.15.83:0/1945603668 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f30d8076e50 0x7f30d81a4e90 unknown :-1 s=CLOSED pgs=2818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:59.272 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:59.271+0000 7f30d7e00640 1 --2- 172.21.15.83:0/1945603668 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f30d8076530 0x7f30d81a4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:16:59.272 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:59.271+0000 7f30d7e00640 1 -- 172.21.15.83:0/1945603668 >> 172.21.15.83:0/1945603668 conn(0x7f30d80ffae0 msgr2=0x7f30d8113940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:16:59.272 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:59.272+0000 7f30d7e00640 1 -- 172.21.15.83:0/1945603668 shutdown_connections 2024-04-24T09:16:59.272 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:16:59.272+0000 7f30d7e00640 1 -- 172.21.15.83:0/1945603668 wait complete. 2024-04-24T09:17:00.226 INFO:tasks.workunit.client.0.smithi083.stdout: 10 84237 6.49 MB/sec execute 571 sec latency 605.018 ms 2024-04-24T09:17:00.415 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:17:00 smithi083 ceph-mon[27907]: pgmap v802: 129 pgs: 129 active+clean; 1.3 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.8 MiB/s wr, 350 op/s 2024-04-24T09:17:00.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:17:00 smithi138 ceph-mon[31441]: pgmap v802: 129 pgs: 129 active+clean; 1.3 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.8 MiB/s wr, 350 op/s 2024-04-24T09:17:00.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:17:00 smithi189 ceph-mon[31791]: pgmap v802: 129 pgs: 129 active+clean; 1.3 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.8 MiB/s wr, 350 op/s 2024-04-24T09:17:01.226 INFO:tasks.workunit.client.0.smithi083.stdout: 10 84323 6.48 MB/sec execute 572 sec latency 225.235 ms 2024-04-24T09:17:02.226 INFO:tasks.workunit.client.0.smithi083.stdout: 10 84426 6.48 MB/sec execute 573 sec latency 143.401 ms 2024-04-24T09:17:02.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:17:02 smithi083 ceph-mon[27907]: pgmap v803: 129 pgs: 129 active+clean; 1.3 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.3 MiB/s wr, 209 op/s 2024-04-24T09:17:02.704 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:17:02 smithi138 ceph-mon[31441]: pgmap v803: 129 pgs: 129 active+clean; 1.3 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.3 MiB/s wr, 209 op/s 2024-04-24T09:17:02.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:17:02 smithi189 ceph-mon[31791]: pgmap v803: 129 pgs: 129 active+clean; 1.3 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.3 MiB/s wr, 209 op/s 2024-04-24T09:17:03.226 INFO:tasks.workunit.client.0.smithi083.stdout: 10 84500 6.48 MB/sec execute 574 sec latency 163.467 ms 2024-04-24T09:17:03.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:17:03 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:17:03.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:17:03 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:17:03.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:17:03 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:17:03.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:17:03 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:17:03.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:17:03 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:17:03.555 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:17:03 smithi083 ceph-mon[27907]: osdmap e133: 12 total, 12 up, 12 in 2024-04-24T09:17:03.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:17:03 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:17:03.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:17:03 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:17:03.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:17:03 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:17:03.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:17:03 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:17:03.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:17:03 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:17:03.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:17:03 smithi138 ceph-mon[31441]: osdmap e133: 12 total, 12 up, 12 in 2024-04-24T09:17:03.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:17:03 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:17:03.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:17:03 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:17:03.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:17:03 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:17:03.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:17:03 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:17:03.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:17:03 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:17:03.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:17:03 smithi189 ceph-mon[31791]: osdmap e133: 12 total, 12 up, 12 in 2024-04-24T09:17:04.227 INFO:tasks.workunit.client.0.smithi083.stdout: 10 84619 6.49 MB/sec execute 575 sec latency 117.525 ms 2024-04-24T09:17:04.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:17:04 smithi083 ceph-mon[27907]: pgmap v805: 129 pgs: 11 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 96 active+clean; 1.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.0 MiB/s wr, 238 op/s 2024-04-24T09:17:04.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:17:04 smithi138 ceph-mon[31441]: pgmap v805: 129 pgs: 11 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 96 active+clean; 1.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.0 MiB/s wr, 238 op/s 2024-04-24T09:17:04.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:17:04 smithi189 ceph-mon[31791]: pgmap v805: 129 pgs: 11 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 96 active+clean; 1.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.0 MiB/s wr, 238 op/s 2024-04-24T09:17:05.226 INFO:tasks.workunit.client.0.smithi083.stdout: 10 84732 6.49 MB/sec execute 576 sec latency 108.727 ms 2024-04-24T09:17:06.204 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:17:05 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:17:06.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:17:05 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:17:06.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:17:05 smithi138 ceph-mon[31441]: pgmap v806: 129 pgs: 11 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 96 active+clean; 1.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.0 MiB/s wr, 238 op/s 2024-04-24T09:17:06.206 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:17:05 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:17:06.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:17:05 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:17:06.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:17:05 smithi189 ceph-mon[31791]: pgmap v806: 129 pgs: 11 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 96 active+clean; 1.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.0 MiB/s wr, 238 op/s 2024-04-24T09:17:06.226 INFO:tasks.workunit.client.0.smithi083.stdout: 10 84863 6.48 MB/sec execute 577 sec latency 251.154 ms 2024-04-24T09:17:06.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:17:05 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:17:06.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:17:05 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:17:06.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:17:05 smithi083 ceph-mon[27907]: pgmap v806: 129 pgs: 11 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 96 active+clean; 1.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.0 MiB/s wr, 238 op/s 2024-04-24T09:17:07.226 INFO:tasks.workunit.client.0.smithi083.stdout: 10 84978 6.48 MB/sec execute 578 sec latency 180.381 ms 2024-04-24T09:17:08.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:17:07 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:17:08.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:17:07 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:17:08.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:17:07 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:17:08.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:17:07 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:17:08.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:17:07 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:17:08.053 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:17:07 smithi083 ceph-mon[27907]: pgmap v807: 129 pgs: 11 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 96 active+clean; 1.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.0 MiB/s wr, 238 op/s 2024-04-24T09:17:08.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:17:07 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:17:08.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:17:07 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:17:08.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:17:07 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:17:08.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:17:07 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:17:08.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:17:07 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:17:08.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:17:07 smithi138 ceph-mon[31441]: pgmap v807: 129 pgs: 11 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 96 active+clean; 1.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.0 MiB/s wr, 238 op/s 2024-04-24T09:17:08.209 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:17:07 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:17:08.209 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:17:07 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:17:08.209 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:17:07 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:17:08.209 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:17:07 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:17:08.209 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:17:07 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:17:08.209 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:17:07 smithi189 ceph-mon[31791]: pgmap v807: 129 pgs: 11 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 96 active+clean; 1.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.0 MiB/s wr, 238 op/s 2024-04-24T09:17:08.226 INFO:tasks.workunit.client.0.smithi083.stdout: 10 85092 6.48 MB/sec execute 579 sec latency 86.779 ms 2024-04-24T09:17:09.227 INFO:tasks.workunit.client.0.smithi083.stdout: 10 85220 6.48 MB/sec execute 580 sec latency 106.221 ms 2024-04-24T09:17:10.227 INFO:tasks.workunit.client.0.smithi083.stdout: 10 85339 6.49 MB/sec execute 581 sec latency 181.876 ms 2024-04-24T09:17:10.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:17:10 smithi138 ceph-mon[31441]: pgmap v808: 129 pgs: 129 active+clean; 1.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.7 MiB/s wr, 248 op/s 2024-04-24T09:17:10.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:17:10 smithi138 ceph-mon[31441]: osdmap e134: 12 total, 12 up, 12 in 2024-04-24T09:17:10.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:17:10 smithi189 ceph-mon[31791]: pgmap v808: 129 pgs: 129 active+clean; 1.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.7 MiB/s wr, 248 op/s 2024-04-24T09:17:10.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:17:10 smithi189 ceph-mon[31791]: osdmap e134: 12 total, 12 up, 12 in 2024-04-24T09:17:10.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:17:10 smithi083 ceph-mon[27907]: pgmap v808: 129 pgs: 129 active+clean; 1.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.7 MiB/s wr, 248 op/s 2024-04-24T09:17:10.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:17:10 smithi083 ceph-mon[27907]: osdmap e134: 12 total, 12 up, 12 in 2024-04-24T09:17:11.227 INFO:tasks.workunit.client.0.smithi083.stdout: 10 85431 6.49 MB/sec execute 582 sec latency 410.723 ms 2024-04-24T09:17:12.229 INFO:tasks.workunit.client.0.smithi083.stdout: 10 85525 6.49 MB/sec execute 583 sec latency 391.385 ms 2024-04-24T09:17:12.362 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:17:12 smithi138 ceph-mon[31441]: pgmap v810: 129 pgs: 129 active+clean; 1.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.4 MiB/s wr, 282 op/s 2024-04-24T09:17:12.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:17:12 smithi189 ceph-mon[31791]: pgmap v810: 129 pgs: 129 active+clean; 1.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.4 MiB/s wr, 282 op/s 2024-04-24T09:17:12.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:17:12 smithi083 ceph-mon[27907]: pgmap v810: 129 pgs: 129 active+clean; 1.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.4 MiB/s wr, 282 op/s 2024-04-24T09:17:13.229 INFO:tasks.workunit.client.0.smithi083.stdout: 10 85634 6.49 MB/sec execute 584 sec latency 83.697 ms 2024-04-24T09:17:13.895 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.894+0000 7f493ee00640 1 -- 172.21.15.83:0/3095220601 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4938109000 msgr2=0x7f49381093e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:13.895 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.894+0000 7f493ee00640 1 --2- 172.21.15.83:0/3095220601 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4938109000 0x7f49381093e0 secure :-1 s=READY pgs=2823 cs=0 l=1 rev1=1 crypto rx=0x7f4928009ca0 tx=0x7f492802a950 comp rx=0 tx=0).stop 2024-04-24T09:17:13.895 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.895+0000 7f493ee00640 1 -- 172.21.15.83:0/3095220601 shutdown_connections 2024-04-24T09:17:13.895 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.895+0000 7f493ee00640 1 --2- 172.21.15.83:0/3095220601 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4938115150 0x7f4938117540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:13.895 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.895+0000 7f493ee00640 1 --2- 172.21.15.83:0/3095220601 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4938109920 0x7f4938114c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:13.895 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.895+0000 7f493ee00640 1 --2- 172.21.15.83:0/3095220601 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4938109000 0x7f49381093e0 unknown :-1 s=CLOSED pgs=2823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:13.895 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.895+0000 7f493ee00640 1 -- 172.21.15.83:0/3095220601 >> 172.21.15.83:0/3095220601 conn(0x7f493806d560 msgr2=0x7f493806d970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:13.895 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.895+0000 7f493ee00640 1 -- 172.21.15.83:0/3095220601 shutdown_connections 2024-04-24T09:17:13.895 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.895+0000 7f493ee00640 1 -- 172.21.15.83:0/3095220601 wait complete. 2024-04-24T09:17:13.896 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.896+0000 7f493ee00640 1 Processor -- start 2024-04-24T09:17:13.896 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.896+0000 7f493ee00640 1 -- start start 2024-04-24T09:17:13.896 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.897+0000 7f493ee00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4938109000 0x7f49381a3130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:13.896 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.897+0000 7f493ee00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4938109920 0x7f49381a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:13.896 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.897+0000 7f493ee00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4938115150 0x7f49381a79f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:13.896 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.897+0000 7f493ee00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f493810b750 con 0x7f4938115150 2024-04-24T09:17:13.896 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.897+0000 7f493ee00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f493810b5d0 con 0x7f4938109000 2024-04-24T09:17:13.897 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.897+0000 7f493ee00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f493810b8d0 con 0x7f4938109920 2024-04-24T09:17:13.897 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.897+0000 7f4937e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4938109000 0x7f49381a3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:13.897 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.897+0000 7f493ce00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4938115150 0x7f49381a79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:13.897 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.897+0000 7f4937e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4938109000 0x7f49381a3130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:36662/0 (socket says 172.21.15.83:36662) 2024-04-24T09:17:13.897 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.897+0000 7f493ce00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4938115150 0x7f49381a79f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55802/0 (socket says 172.21.15.83:55802) 2024-04-24T09:17:13.897 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.897+0000 7f4937e00640 1 -- 172.21.15.83:0/893450958 learned_addr learned my addr 172.21.15.83:0/893450958 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:17:13.898 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.897+0000 7f4937400640 1 --2- 172.21.15.83:0/893450958 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4938109920 0x7f49381a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:13.898 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.898+0000 7f493ce00640 1 -- 172.21.15.83:0/893450958 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4938109920 msgr2=0x7f49381a3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:13.898 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.898+0000 7f493ce00640 1 --2- 172.21.15.83:0/893450958 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4938109920 0x7f49381a3670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:13.898 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.898+0000 7f493ce00640 1 -- 172.21.15.83:0/893450958 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4938109000 msgr2=0x7f49381a3130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:13.898 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.898+0000 7f493ce00640 1 --2- 172.21.15.83:0/893450958 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4938109000 0x7f49381a3130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:13.898 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.898+0000 7f493ce00640 1 -- 172.21.15.83:0/893450958 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49381a8120 con 0x7f4938115150 2024-04-24T09:17:13.899 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.898+0000 7f493ce00640 1 --2- 172.21.15.83:0/893450958 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4938115150 0x7f49381a79f0 secure :-1 s=READY pgs=2824 cs=0 l=1 rev1=1 crypto rx=0x7f493000e970 tx=0x7f493000ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:13.899 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.899+0000 7f4934c00640 1 -- 172.21.15.83:0/893450958 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f493000ccc0 con 0x7f4938115150 2024-04-24T09:17:13.899 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.899+0000 7f493ee00640 1 -- 172.21.15.83:0/893450958 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49381a8410 con 0x7f4938115150 2024-04-24T09:17:13.899 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.899+0000 7f4934c00640 1 -- 172.21.15.83:0/893450958 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4930005020 con 0x7f4938115150 2024-04-24T09:17:13.899 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.899+0000 7f4934c00640 1 -- 172.21.15.83:0/893450958 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4930004a00 con 0x7f4938115150 2024-04-24T09:17:13.899 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.899+0000 7f493ee00640 1 -- 172.21.15.83:0/893450958 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49381afca0 con 0x7f4938115150 2024-04-24T09:17:13.903 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.904+0000 7f4941c31740 1 Processor -- start 2024-04-24T09:17:13.905 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.905+0000 7f4934c00640 1 -- 172.21.15.83:0/893450958 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f49300051d0 con 0x7f4938115150 2024-04-24T09:17:13.905 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.905+0000 7f4941c31740 1 -- start start 2024-04-24T09:17:13.905 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.906+0000 7f4941c31740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605334811b0 0x560533481590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:13.906 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.906+0000 7f4941c31740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560533481ad0 0x56053348be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:13.906 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.906+0000 7f4941c31740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56053348c390 0x56053348e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:13.906 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.906+0000 7f4941c31740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5605333e7900 con 0x560533481ad0 2024-04-24T09:17:13.906 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.906+0000 7f4941c31740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56053338df20 con 0x5605334811b0 2024-04-24T09:17:13.906 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.906+0000 7f4941c31740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5605333e38c0 con 0x56053348c390 2024-04-24T09:17:13.906 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.906+0000 7f4937400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605334811b0 0x560533481590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:13.906 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.906+0000 7f4937400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605334811b0 0x560533481590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:36666/0 (socket says 172.21.15.83:36666) 2024-04-24T09:17:13.906 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.906+0000 7f4937400640 1 -- 172.21.15.83:0/167537308 learned_addr learned my addr 172.21.15.83:0/167537308 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:17:13.906 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.906+0000 7f4937e00640 1 --2- 172.21.15.83:0/167537308 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560533481ad0 0x56053348be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:13.907 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.907+0000 7f4937400640 1 --2- 172.21.15.83:0/167537308 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56053348c390 0x56053348e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:13.907 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.907+0000 7f4934c00640 1 --2- 172.21.15.83:0/893450958 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f491c0a3e40 0x7f491c0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:13.908 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.907+0000 7f4937400640 1 -- 172.21.15.83:0/167537308 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56053348c390 msgr2=0x56053348e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:13.908 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.907+0000 7f4937400640 1 --2- 172.21.15.83:0/167537308 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56053348c390 0x56053348e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:13.908 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.907+0000 7f4937400640 1 -- 172.21.15.83:0/167537308 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560533481ad0 msgr2=0x56053348be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:13.908 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.907+0000 7f4937400640 1 --2- 172.21.15.83:0/167537308 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560533481ad0 0x56053348be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:13.908 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.907+0000 7f4937400640 1 -- 172.21.15.83:0/167537308 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605333b2e10 con 0x5605334811b0 2024-04-24T09:17:13.908 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.907+0000 7f4934c00640 1 -- 172.21.15.83:0/893450958 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 9583+0+0 (secure 0 0 0) 0x7f49300d0ad0 con 0x7f4938115150 2024-04-24T09:17:13.908 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.908+0000 7f4937400640 1 --2- 172.21.15.83:0/167537308 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605334811b0 0x560533481590 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f49200058a0 tx=0x7f492000b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=215ea5761407dba3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:13.908 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.908+0000 7f493ce00640 1 --2- 172.21.15.83:0/893450958 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f491c0a3e40 0x7f491c0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:13.908 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.908+0000 7f492d600640 1 -- 172.21.15.83:0/167537308 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4920004030 con 0x5605334811b0 2024-04-24T09:17:13.908 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.908+0000 7f492d600640 1 -- 172.21.15.83:0/167537308 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f49200041d0 con 0x5605334811b0 2024-04-24T09:17:13.908 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.908+0000 7f492d600640 1 -- 172.21.15.83:0/167537308 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4920010460 con 0x5605334811b0 2024-04-24T09:17:13.909 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.909+0000 7f493ce00640 1 --2- 172.21.15.83:0/893450958 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f491c0a3e40 0x7f491c0a6300 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f4930098fc0 tx=0x7f4930005750 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:13.909 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.909+0000 7f4941c31740 1 -- 172.21.15.83:0/167537308 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605334811b0 msgr2=0x560533481590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:13.909 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.909+0000 7f4941c31740 1 --2- 172.21.15.83:0/167537308 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605334811b0 0x560533481590 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f49200058a0 tx=0x7f492000b840 comp rx=0 tx=0).stop 2024-04-24T09:17:13.909 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.909+0000 7f4941c31740 1 -- 172.21.15.83:0/167537308 shutdown_connections 2024-04-24T09:17:13.909 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.909+0000 7f4941c31740 1 --2- 172.21.15.83:0/167537308 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56053348c390 0x56053348e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:13.909 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.909+0000 7f4941c31740 1 --2- 172.21.15.83:0/167537308 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560533481ad0 0x56053348be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:13.909 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.909+0000 7f4941c31740 1 --2- 172.21.15.83:0/167537308 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605334811b0 0x560533481590 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:13.909 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.909+0000 7f4941c31740 1 -- 172.21.15.83:0/167537308 >> 172.21.15.83:0/167537308 conn(0x56053347d6c0 msgr2=0x56053347daa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:13.910 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.909+0000 7f4941c31740 1 -- 172.21.15.83:0/167537308 shutdown_connections 2024-04-24T09:17:13.910 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.910+0000 7f4941c31740 1 -- 172.21.15.83:0/167537308 wait complete. 2024-04-24T09:17:13.910 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.910+0000 7f4941c31740 1 -- start start 2024-04-24T09:17:13.910 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.911+0000 7f4941c31740 1 Processor -- start 2024-04-24T09:17:13.911 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.911+0000 7f4941c31740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5605334811b0 0x560533489350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:13.911 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.911+0000 7f4941c31740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560533481ad0 0x560533489890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:13.911 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.912+0000 7f4941c31740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56053348c390 0x560533489dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:13.911 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.912+0000 7f4941c31740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56053348efc0 con 0x5605334811b0 2024-04-24T09:17:13.911 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.912+0000 7f4941c31740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5605333a12c0 con 0x56053348c390 2024-04-24T09:17:13.911 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.912+0000 7f4941c31740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56053348f140 con 0x560533481ad0 2024-04-24T09:17:13.911 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.912+0000 7f4941c31740 10 client.0 _ll_register_callbacks cb 0x5605330f9a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:17:13.912 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.912+0000 7f4937400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56053348c390 0x560533489dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:13.912 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.912+0000 7f4937e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560533481ad0 0x560533489890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:13.912 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.912+0000 7f4937400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56053348c390 0x560533489dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:36678/0 (socket says 172.21.15.83:36678) 2024-04-24T09:17:13.912 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.912+0000 7f4937400640 1 -- 172.21.15.83:0/1697758950 learned_addr learned my addr 172.21.15.83:0/1697758950 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:17:13.912 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.912+0000 7f4937400640 1 --2- 172.21.15.83:0/1697758950 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5605334811b0 0x560533489350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:13.912 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.913+0000 7f4937400640 1 -- 172.21.15.83:0/1697758950 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560533481ad0 msgr2=0x560533489890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:13.912 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.913+0000 7f4937400640 1 --2- 172.21.15.83:0/1697758950 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560533481ad0 0x560533489890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:13.912 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.913+0000 7f4937400640 1 -- 172.21.15.83:0/1697758950 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5605334811b0 msgr2=0x560533489350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:13.913 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.913+0000 7f4937400640 1 --2- 172.21.15.83:0/1697758950 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5605334811b0 0x560533489350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:13.913 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.913+0000 7f4937400640 1 -- 172.21.15.83:0/1697758950 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605333b1fd0 con 0x56053348c390 2024-04-24T09:17:13.913 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.913+0000 7f4937400640 1 --2- 172.21.15.83:0/1697758950 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56053348c390 0x560533489dd0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f49200045d0 tx=0x7f4920003be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e55a7fad9b826e0c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:13.913 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.914+0000 7f4926000640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:17:13.914 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.914+0000 7f4926000640 1 -- 172.21.15.83:0/1697758950 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4920028040 con 0x56053348c390 2024-04-24T09:17:13.914 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.914+0000 7f4926000640 1 -- 172.21.15.83:0/1697758950 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(134..134 src has 1..134) ==== 9583+0+0 (secure 0 0 0) 0x7f49200037f0 con 0x56053348c390 2024-04-24T09:17:13.914 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.915+0000 7f4941c31740 1 -- 172.21.15.83:0/1697758950 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49200041d0 con 0x56053348c390 2024-04-24T09:17:13.914 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.915+0000 7f4926000640 1 -- 172.21.15.83:0/1697758950 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f4920012210 con 0x56053348c390 2024-04-24T09:17:13.914 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.915+0000 7f4941c31740 10 client.41222 fetch_fsmap learned FSMap version 69 2024-04-24T09:17:13.914 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.915+0000 7f4941c31740 1 -- 172.21.15.83:0/1697758950 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x5605330fa210 con 0x56053348c390 2024-04-24T09:17:13.915 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.916+0000 7f4926000640 1 -- 172.21.15.83:0/1697758950 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f49200041d0 con 0x56053348c390 2024-04-24T09:17:13.915 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.916+0000 7f4926000640 1 -- 172.21.15.83:0/1697758950 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 69) ==== 8370+0+0 (secure 0 0 0) 0x7f4920004860 con 0x56053348c390 2024-04-24T09:17:13.915 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.916+0000 7f4941c31740 10 client.41222 fetch_fsmap finished waiting for FSMap version 69 2024-04-24T09:17:13.915 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.916+0000 7f4941c31740 10 client.41222 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:17:13.915 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.916+0000 7f4941c31740 20 client.41222 populate_metadata read hostname 'smithi083' 2024-04-24T09:17:13.915 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.916+0000 7f4941c31740 1 --2- 172.21.15.83:0/1697758950 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5605334a4ba0 0x5605334a6f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:13.916 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.916+0000 7f4941c31740 4 client.41222 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:17:13.916 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.916+0000 7f4941c31740 1 -- 172.21.15.83:0/1697758950 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5605333f5320 con 0x5605334a4ba0 2024-04-24T09:17:13.916 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.917+0000 7f4937e00640 1 --2- 172.21.15.83:0/1697758950 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5605334a4ba0 0x5605334a6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:13.916 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.917+0000 7f4937e00640 1 --2- 172.21.15.83:0/1697758950 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5605334a4ba0 0x5605334a6f90 crc :-1 s=READY pgs=623 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a7c5b0e8a8c8393 server_cookie=c715ba82aabc0dec in_seq=0 out_seq=0 2024-04-24T09:17:13.916 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.917+0000 7f4926000640 10 client.41222 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:17:13.918 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.918+0000 7f4926000640 1 -- 172.21.15.83:0/1697758950 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5605333f5320 con 0x5605334a4ba0 2024-04-24T09:17:13.918 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.918+0000 7f4926000640 10 client.41222 handle_command_reply: tid=0 2024-04-24T09:17:13.918 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.918+0000 7f4941c31740 1 client.41222 shutdown 2024-04-24T09:17:13.918 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.919+0000 7f4941c31740 1 -- 172.21.15.83:0/1697758950 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56053348c390 msgr2=0x560533489dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:13.918 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.919+0000 7f4941c31740 1 --2- 172.21.15.83:0/1697758950 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56053348c390 0x560533489dd0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f49200045d0 tx=0x7f4920003be0 comp rx=0 tx=0).stop 2024-04-24T09:17:13.919 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.919+0000 7f4941c31740 1 -- 172.21.15.83:0/1697758950 shutdown_connections 2024-04-24T09:17:13.919 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.919+0000 7f4941c31740 1 --2- 172.21.15.83:0/1697758950 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5605334a4ba0 0x5605334a6f90 crc :-1 s=READY pgs=623 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:13.919 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.919+0000 7f4941c31740 1 --2- 172.21.15.83:0/1697758950 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56053348c390 0x560533489dd0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:13.919 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.919+0000 7f4941c31740 1 --2- 172.21.15.83:0/1697758950 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560533481ad0 0x560533489890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:13.919 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.919+0000 7f4941c31740 1 --2- 172.21.15.83:0/1697758950 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5605334811b0 0x560533489350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:13.919 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.919+0000 7f4941c31740 1 -- 172.21.15.83:0/1697758950 >> 172.21.15.83:0/1697758950 conn(0x56053347d6c0 msgr2=0x560533483440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:13.919 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.919+0000 7f4926000640 0 client.41222 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:17:13.919 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.919+0000 7f4941c31740 1 -- 172.21.15.83:0/1697758950 shutdown_connections 2024-04-24T09:17:13.919 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.919+0000 7f4941c31740 1 -- 172.21.15.83:0/1697758950 wait complete. 2024-04-24T09:17:13.919 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.919+0000 7f4941c31740 20 client.41222 trim_cache size 0 max 16384 2024-04-24T09:17:13.941 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.942+0000 7f4941c31740 1 Processor -- start 2024-04-24T09:17:13.941 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.942+0000 7f4941c31740 1 -- start start 2024-04-24T09:17:13.942 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.942+0000 7f4941c31740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5605334811b0 0x5605334a0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:13.942 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.942+0000 7f4941c31740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560533481ad0 0x5605334832c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:13.942 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.942+0000 7f4941c31740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56053348c390 0x5605334ac620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:13.942 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.942+0000 7f4941c31740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f49381b5bf0 con 0x5605334811b0 2024-04-24T09:17:13.942 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.942+0000 7f4941c31740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f49381b7790 con 0x56053348c390 2024-04-24T09:17:13.942 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.942+0000 7f4941c31740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f49381b62c0 con 0x560533481ad0 2024-04-24T09:17:13.942 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.943+0000 7f4937400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5605334811b0 0x5605334a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:13.942 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.943+0000 7f4937400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5605334811b0 0x5605334a0690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55824/0 (socket says 172.21.15.83:55824) 2024-04-24T09:17:13.942 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.943+0000 7f4937400640 1 -- 172.21.15.83:0/3583389967 learned_addr learned my addr 172.21.15.83:0/3583389967 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:17:13.942 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.943+0000 7f4937400640 1 --2- 172.21.15.83:0/3583389967 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56053348c390 0x5605334ac620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:13.943 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.943+0000 7f4937400640 1 -- 172.21.15.83:0/3583389967 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560533481ad0 msgr2=0x5605334832c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:13.943 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.943+0000 7f4937e00640 1 --2- 172.21.15.83:0/3583389967 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560533481ad0 0x5605334832c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:13.943 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.943+0000 7f4937400640 1 --2- 172.21.15.83:0/3583389967 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560533481ad0 0x5605334832c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:13.943 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.943+0000 7f4937400640 1 -- 172.21.15.83:0/3583389967 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56053348c390 msgr2=0x5605334ac620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:13.943 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.943+0000 7f4937400640 1 --2- 172.21.15.83:0/3583389967 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56053348c390 0x5605334ac620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:13.943 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.943+0000 7f4937400640 1 -- 172.21.15.83:0/3583389967 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605331c6620 con 0x5605334811b0 2024-04-24T09:17:13.944 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.944+0000 7f4937400640 1 --2- 172.21.15.83:0/3583389967 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5605334811b0 0x5605334a0690 secure :-1 s=READY pgs=2825 cs=0 l=1 rev1=1 crypto rx=0x7f492002bc10 tx=0x7f4920012530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f71a322fd544e17a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:13.944 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.944+0000 7f492cc00640 1 -- 172.21.15.83:0/3583389967 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f492001dbe0 con 0x5605334811b0 2024-04-24T09:17:13.944 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.944+0000 7f492cc00640 1 -- 172.21.15.83:0/3583389967 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4920004120 con 0x5605334811b0 2024-04-24T09:17:13.944 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.944+0000 7f492cc00640 1 -- 172.21.15.83:0/3583389967 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4920004400 con 0x5605334811b0 2024-04-24T09:17:13.944 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.945+0000 7f4941c31740 1 -- 172.21.15.83:0/3583389967 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5605334811b0 msgr2=0x5605334a0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:13.944 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.945+0000 7f4941c31740 1 --2- 172.21.15.83:0/3583389967 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5605334811b0 0x5605334a0690 secure :-1 s=READY pgs=2825 cs=0 l=1 rev1=1 crypto rx=0x7f492002bc10 tx=0x7f4920012530 comp rx=0 tx=0).stop 2024-04-24T09:17:13.944 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.945+0000 7f4941c31740 1 -- 172.21.15.83:0/3583389967 shutdown_connections 2024-04-24T09:17:13.944 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.945+0000 7f4941c31740 1 --2- 172.21.15.83:0/3583389967 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56053348c390 0x5605334ac620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:13.945 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.945+0000 7f4941c31740 1 --2- 172.21.15.83:0/3583389967 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560533481ad0 0x5605334832c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:13.945 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.945+0000 7f4941c31740 1 --2- 172.21.15.83:0/3583389967 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5605334811b0 0x5605334a0690 unknown :-1 s=CLOSED pgs=2825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:13.945 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.945+0000 7f4941c31740 1 -- 172.21.15.83:0/3583389967 >> 172.21.15.83:0/3583389967 conn(0x56053347d6c0 msgr2=0x560533482bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:13.945 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.945+0000 7f4941c31740 1 -- 172.21.15.83:0/3583389967 shutdown_connections 2024-04-24T09:17:13.945 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.945+0000 7f4941c31740 1 -- 172.21.15.83:0/3583389967 wait complete. 2024-04-24T09:17:13.945 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.946+0000 7f4941c31740 1 -- start start 2024-04-24T09:17:13.945 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.946+0000 7f4941c31740 1 Processor -- start 2024-04-24T09:17:13.946 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.947+0000 7f4941c31740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605334811b0 0x5605334860c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:13.947 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.947+0000 7f4941c31740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560533481ad0 0x560533486600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:13.947 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.947+0000 7f4941c31740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56053348c390 0x560533486b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:13.947 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.947+0000 7f4941c31740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f49381b70c0 con 0x560533481ad0 2024-04-24T09:17:13.947 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.947+0000 7f4941c31740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f49381b69f0 con 0x5605334811b0 2024-04-24T09:17:13.947 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.947+0000 7f4941c31740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f49381b3c10 con 0x56053348c390 2024-04-24T09:17:13.947 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.947+0000 7f4941c31740 10 client.0 _ll_register_callbacks cb 0x7f4928009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:17:13.947 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.947+0000 7f4937e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560533481ad0 0x560533486600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:13.947 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.947+0000 7f4937e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560533481ad0 0x560533486600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55834/0 (socket says 172.21.15.83:55834) 2024-04-24T09:17:13.947 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.947+0000 7f4937e00640 1 -- 172.21.15.83:0/2264035839 learned_addr learned my addr 172.21.15.83:0/2264035839 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:17:13.947 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.947+0000 7f4937400640 1 --2- 172.21.15.83:0/2264035839 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605334811b0 0x5605334860c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:13.947 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.948+0000 7f4937400640 1 --2- 172.21.15.83:0/2264035839 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56053348c390 0x560533486b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:13.948 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.948+0000 7f4937e00640 1 -- 172.21.15.83:0/2264035839 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56053348c390 msgr2=0x560533486b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:13.948 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.948+0000 7f4937e00640 1 --2- 172.21.15.83:0/2264035839 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56053348c390 0x560533486b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:13.948 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.948+0000 7f4937e00640 1 -- 172.21.15.83:0/2264035839 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605334811b0 msgr2=0x5605334860c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:13.948 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.948+0000 7f4937e00640 1 --2- 172.21.15.83:0/2264035839 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605334811b0 0x5605334860c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:13.948 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.948+0000 7f4937e00640 1 -- 172.21.15.83:0/2264035839 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560533509f40 con 0x560533481ad0 2024-04-24T09:17:13.949 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.948+0000 7f4937400640 1 --2- 172.21.15.83:0/2264035839 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56053348c390 0x560533486b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:17:13.949 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.948+0000 7f4937e00640 1 --2- 172.21.15.83:0/2264035839 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560533481ad0 0x560533486600 secure :-1 s=READY pgs=2826 cs=0 l=1 rev1=1 crypto rx=0x7f4928006010 tx=0x7f4928003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25f1ad17a36fb983 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:13.949 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.948+0000 7f4926000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:17:13.949 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.948+0000 7f4926000640 1 -- 172.21.15.83:0/2264035839 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4928034040 con 0x560533481ad0 2024-04-24T09:17:13.949 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.949+0000 7f4926000640 1 -- 172.21.15.83:0/2264035839 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(134..134 src has 1..134) ==== 9583+0+0 (secure 0 0 0) 0x7f492802f8c0 con 0x560533481ad0 2024-04-24T09:17:13.949 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.949+0000 7f4941c31740 1 -- 172.21.15.83:0/2264035839 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4920004120 con 0x560533481ad0 2024-04-24T09:17:13.949 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.949+0000 7f4926000640 1 -- 172.21.15.83:0/2264035839 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f492802fac0 con 0x560533481ad0 2024-04-24T09:17:13.949 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.949+0000 7f4941c31740 10 client.24171 fetch_fsmap learned FSMap version 69 2024-04-24T09:17:13.949 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.949+0000 7f4941c31740 1 -- 172.21.15.83:0/2264035839 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x56053350b290 con 0x560533481ad0 2024-04-24T09:17:13.950 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.949+0000 7f4926000640 1 -- 172.21.15.83:0/2264035839 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4920004120 con 0x560533481ad0 2024-04-24T09:17:13.950 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.950+0000 7f4926000640 1 -- 172.21.15.83:0/2264035839 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 69) ==== 8370+0+0 (secure 0 0 0) 0x7f49280481a0 con 0x560533481ad0 2024-04-24T09:17:13.950 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.950+0000 7f4941c31740 10 client.24171 fetch_fsmap finished waiting for FSMap version 69 2024-04-24T09:17:13.950 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.950+0000 7f4941c31740 10 client.24171 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:17:13.950 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.950+0000 7f4941c31740 20 client.24171 populate_metadata read hostname 'smithi083' 2024-04-24T09:17:13.950 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.950+0000 7f4941c31740 1 --2- 172.21.15.83:0/2264035839 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56053350ba90 0x56053350deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:13.950 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.950+0000 7f4941c31740 4 client.24171 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T09:17:13.950 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.950+0000 7f4941c31740 1 -- 172.21.15.83:0/2264035839 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5605333ef190 con 0x56053350ba90 2024-04-24T09:17:13.950 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.951+0000 7f4937400640 1 --2- 172.21.15.83:0/2264035839 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56053350ba90 0x56053350deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:13.951 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.951+0000 7f4937400640 1 --2- 172.21.15.83:0/2264035839 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56053350ba90 0x56053350deb0 crc :-1 s=READY pgs=624 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1894bf5c38ec011b server_cookie=5808a0a6dcb5462c in_seq=0 out_seq=0 2024-04-24T09:17:13.951 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:13.951+0000 7f4926000640 10 client.24171 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:17:14.093 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:14.094+0000 7f4926000640 1 -- 172.21.15.83:0/2264035839 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5605333ef190 con 0x56053350ba90 2024-04-24T09:17:14.093 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:14.094+0000 7f4926000640 10 client.24171 handle_command_reply: tid=0 2024-04-24T09:17:14.093 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:14.094+0000 7f4941c31740 1 client.24171 shutdown 2024-04-24T09:17:14.094 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:14.094+0000 7f4941c31740 1 -- 172.21.15.83:0/2264035839 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560533481ad0 msgr2=0x560533486600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:14.094 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:14.094+0000 7f4941c31740 1 --2- 172.21.15.83:0/2264035839 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560533481ad0 0x560533486600 secure :-1 s=READY pgs=2826 cs=0 l=1 rev1=1 crypto rx=0x7f4928006010 tx=0x7f4928003970 comp rx=0 tx=0).stop 2024-04-24T09:17:14.094 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:14.094+0000 7f4941c31740 1 -- 172.21.15.83:0/2264035839 shutdown_connections 2024-04-24T09:17:14.095 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:14.094+0000 7f4941c31740 1 --2- 172.21.15.83:0/2264035839 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x56053350ba90 0x56053350deb0 crc :-1 s=READY pgs=624 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:14.095 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:14.094+0000 7f4941c31740 1 --2- 172.21.15.83:0/2264035839 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56053348c390 0x560533486b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:14.095 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:14.094+0000 7f4941c31740 1 --2- 172.21.15.83:0/2264035839 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560533481ad0 0x560533486600 unknown :-1 s=CLOSED pgs=2826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:14.095 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:14.094+0000 7f4941c31740 1 --2- 172.21.15.83:0/2264035839 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605334811b0 0x5605334860c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:14.095 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:14.095+0000 7f4941c31740 1 -- 172.21.15.83:0/2264035839 >> 172.21.15.83:0/2264035839 conn(0x56053347d6c0 msgr2=0x56053348b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:14.095 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:14.095+0000 7f4926000640 0 client.24171 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:17:14.095 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:14.095+0000 7f4941c31740 1 -- 172.21.15.83:0/2264035839 shutdown_connections 2024-04-24T09:17:14.095 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:14.095+0000 7f4941c31740 1 -- 172.21.15.83:0/2264035839 wait complete. 2024-04-24T09:17:14.095 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:14.095+0000 7f4941c31740 20 client.24171 trim_cache size 0 max 16384 2024-04-24T09:17:14.095 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T09:17:14.096 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T09:17:14.096 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T09:17:14.096 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T09:17:14.096 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:14.096+0000 7f493ee00640 1 -- 172.21.15.83:0/893450958 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f491c0a3e40 msgr2=0x7f491c0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:14.096 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:14.096+0000 7f493ee00640 1 --2- 172.21.15.83:0/893450958 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f491c0a3e40 0x7f491c0a6300 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f4930098fc0 tx=0x7f4930005750 comp rx=0 tx=0).stop 2024-04-24T09:17:14.096 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:14.096+0000 7f493ee00640 1 -- 172.21.15.83:0/893450958 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4938115150 msgr2=0x7f49381a79f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:14.096 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:14.096+0000 7f493ee00640 1 --2- 172.21.15.83:0/893450958 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4938115150 0x7f49381a79f0 secure :-1 s=READY pgs=2824 cs=0 l=1 rev1=1 crypto rx=0x7f493000e970 tx=0x7f493000ee30 comp rx=0 tx=0).stop 2024-04-24T09:17:14.096 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:14.097+0000 7f493ee00640 1 -- 172.21.15.83:0/893450958 shutdown_connections 2024-04-24T09:17:14.096 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:14.097+0000 7f493ee00640 1 --2- 172.21.15.83:0/893450958 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f491c0a3e40 0x7f491c0a6300 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:14.096 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:14.097+0000 7f493ee00640 1 --2- 172.21.15.83:0/893450958 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4938115150 0x7f49381a79f0 unknown :-1 s=CLOSED pgs=2824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:14.097 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:14.097+0000 7f493ee00640 1 --2- 172.21.15.83:0/893450958 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4938109920 0x7f49381a3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:14.097 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:14.097+0000 7f493ee00640 1 --2- 172.21.15.83:0/893450958 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4938109000 0x7f49381a3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:14.097 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:14.097+0000 7f493ee00640 1 -- 172.21.15.83:0/893450958 >> 172.21.15.83:0/893450958 conn(0x7f493806d560 msgr2=0x7f4938117190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:14.097 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:14.097+0000 7f493ee00640 1 -- 172.21.15.83:0/893450958 shutdown_connections 2024-04-24T09:17:14.097 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:14.097+0000 7f493ee00640 1 -- 172.21.15.83:0/893450958 wait complete. 2024-04-24T09:17:14.229 INFO:tasks.workunit.client.0.smithi083.stdout: 10 85740 6.49 MB/sec execute 585 sec latency 119.437 ms 2024-04-24T09:17:14.421 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:17:14 smithi083 ceph-mon[27907]: pgmap v811: 129 pgs: 129 active+clean; 1.3 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.7 MiB/s wr, 215 op/s 2024-04-24T09:17:14.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:17:14 smithi138 ceph-mon[31441]: pgmap v811: 129 pgs: 129 active+clean; 1.3 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.7 MiB/s wr, 215 op/s 2024-04-24T09:17:14.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:17:14 smithi189 ceph-mon[31791]: pgmap v811: 129 pgs: 129 active+clean; 1.3 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.7 MiB/s wr, 215 op/s 2024-04-24T09:17:15.229 INFO:tasks.workunit.client.0.smithi083.stdout: 10 85840 6.48 MB/sec execute 586 sec latency 77.009 ms 2024-04-24T09:17:15.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:17:15 smithi083 ceph-mon[27907]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:17:15.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:17:15 smithi138 ceph-mon[31441]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:17:15.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:17:15 smithi189 ceph-mon[31791]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:17:16.229 INFO:tasks.workunit.client.0.smithi083.stdout: 10 85963 6.48 MB/sec execute 587 sec latency 156.862 ms 2024-04-24T09:17:16.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:17:16 smithi138 ceph-mon[31441]: pgmap v812: 129 pgs: 129 active+clean; 1.3 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.7 MiB/s wr, 215 op/s 2024-04-24T09:17:16.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:17:16 smithi189 ceph-mon[31791]: pgmap v812: 129 pgs: 129 active+clean; 1.3 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.7 MiB/s wr, 215 op/s 2024-04-24T09:17:16.552 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:17:16 smithi083 ceph-mon[27907]: pgmap v812: 129 pgs: 129 active+clean; 1.3 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.7 MiB/s wr, 215 op/s 2024-04-24T09:17:17.229 INFO:tasks.workunit.client.0.smithi083.stdout: 10 86082 6.47 MB/sec execute 588 sec latency 94.850 ms 2024-04-24T09:17:18.230 INFO:tasks.workunit.client.0.smithi083.stdout: 10 86213 6.47 MB/sec execute 589 sec latency 103.996 ms 2024-04-24T09:17:18.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:17:18 smithi138 ceph-mon[31441]: pgmap v813: 129 pgs: 129 active+clean; 1.3 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.7 MiB/s wr, 215 op/s 2024-04-24T09:17:18.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:17:18 smithi189 ceph-mon[31791]: pgmap v813: 129 pgs: 129 active+clean; 1.3 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.7 MiB/s wr, 215 op/s 2024-04-24T09:17:18.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:17:18 smithi083 ceph-mon[27907]: pgmap v813: 129 pgs: 129 active+clean; 1.3 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.7 MiB/s wr, 215 op/s 2024-04-24T09:17:19.230 INFO:tasks.workunit.client.0.smithi083.stdout: 10 86324 6.47 MB/sec execute 590 sec latency 80.912 ms 2024-04-24T09:17:20.230 INFO:tasks.workunit.client.0.smithi083.stdout: 10 86452 6.47 MB/sec execute 591 sec latency 73.874 ms 2024-04-24T09:17:20.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:17:20 smithi138 ceph-mon[31441]: pgmap v814: 129 pgs: 129 active+clean; 1.3 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.5 MiB/s wr, 265 op/s 2024-04-24T09:17:20.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:17:20 smithi189 ceph-mon[31791]: pgmap v814: 129 pgs: 129 active+clean; 1.3 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.5 MiB/s wr, 265 op/s 2024-04-24T09:17:20.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:17:20 smithi083 ceph-mon[27907]: pgmap v814: 129 pgs: 129 active+clean; 1.3 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.5 MiB/s wr, 265 op/s 2024-04-24T09:17:21.230 INFO:tasks.workunit.client.0.smithi083.stdout: 10 86587 6.47 MB/sec execute 592 sec latency 81.079 ms 2024-04-24T09:17:21.857 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T09:17:21.857 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T09:17:22.230 INFO:tasks.workunit.client.0.smithi083.stdout: 10 86715 6.48 MB/sec execute 593 sec latency 83.979 ms 2024-04-24T09:17:22.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:17:22 smithi083 ceph-mon[27907]: pgmap v815: 129 pgs: 129 active+clean; 1.3 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.0 MiB/s wr, 236 op/s 2024-04-24T09:17:22.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:17:22 smithi138 ceph-mon[31441]: pgmap v815: 129 pgs: 129 active+clean; 1.3 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.0 MiB/s wr, 236 op/s 2024-04-24T09:17:22.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:17:22 smithi189 ceph-mon[31791]: pgmap v815: 129 pgs: 129 active+clean; 1.3 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.0 MiB/s wr, 236 op/s 2024-04-24T09:17:22.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.690+0000 7f27f3200640 1 -- 172.21.15.83:0/2615883506 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f27f41060d0 msgr2=0x7f27f4069190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:22.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.690+0000 7f27f3200640 1 --2- 172.21.15.83:0/2615883506 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f27f41060d0 0x7f27f4069190 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f27dc009860 tx=0x7f27dc02a360 comp rx=0 tx=0).stop 2024-04-24T09:17:22.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.691+0000 7f27f3200640 1 -- 172.21.15.83:0/2615883506 shutdown_connections 2024-04-24T09:17:22.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.691+0000 7f27f3200640 1 --2- 172.21.15.83:0/2615883506 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f27f41060d0 0x7f27f4069190 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:22.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.691+0000 7f27f3200640 1 --2- 172.21.15.83:0/2615883506 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27f41019b0 0x7f27f41059f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:22.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.691+0000 7f27f3200640 1 --2- 172.21.15.83:0/2615883506 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f27f4101000 0x7f27f41013e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:22.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.691+0000 7f27f3200640 1 -- 172.21.15.83:0/2615883506 >> 172.21.15.83:0/2615883506 conn(0x7f27f40fc880 msgr2=0x7f27f40feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:22.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.691+0000 7f27f3200640 1 -- 172.21.15.83:0/2615883506 shutdown_connections 2024-04-24T09:17:22.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.691+0000 7f27f3200640 1 -- 172.21.15.83:0/2615883506 wait complete. 2024-04-24T09:17:22.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.693+0000 7f27f3200640 1 Processor -- start 2024-04-24T09:17:22.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.693+0000 7f27f3200640 1 -- start start 2024-04-24T09:17:22.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.693+0000 7f27f3200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f27f4101000 0x7f27f4113cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:22.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.693+0000 7f27f3200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f27f41060d0 0x7f27f4110db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:22.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.693+0000 7f27f3200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27f41112f0 0x7f27f4111760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:22.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.694+0000 7f27f3200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f27f4114440 con 0x7f27f41112f0 2024-04-24T09:17:22.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.694+0000 7f27f3200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f27f41142c0 con 0x7f27f4101000 2024-04-24T09:17:22.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.694+0000 7f27f3200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f27f41145c0 con 0x7f27f41060d0 2024-04-24T09:17:22.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.694+0000 7f27f1e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f27f4101000 0x7f27f4113cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:22.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.694+0000 7f27f1e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f27f4101000 0x7f27f4113cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:39098/0 (socket says 172.21.15.83:39098) 2024-04-24T09:17:22.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.694+0000 7f27f1e00640 1 -- 172.21.15.83:0/1760399545 learned_addr learned my addr 172.21.15.83:0/1760399545 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:17:22.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.695+0000 7f27f1400640 1 --2- 172.21.15.83:0/1760399545 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f27f41060d0 0x7f27f4110db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:22.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.695+0000 7f27f2800640 1 --2- 172.21.15.83:0/1760399545 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27f41112f0 0x7f27f4111760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:22.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.695+0000 7f27f1e00640 1 -- 172.21.15.83:0/1760399545 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f27f41060d0 msgr2=0x7f27f4110db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:22.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.695+0000 7f27f1e00640 1 --2- 172.21.15.83:0/1760399545 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f27f41060d0 0x7f27f4110db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:22.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.695+0000 7f27f1e00640 1 -- 172.21.15.83:0/1760399545 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27f41112f0 msgr2=0x7f27f4111760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:22.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.695+0000 7f27f1e00640 1 --2- 172.21.15.83:0/1760399545 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27f41112f0 0x7f27f4111760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:22.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.695+0000 7f27f1e00640 1 -- 172.21.15.83:0/1760399545 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27f41ab2b0 con 0x7f27f4101000 2024-04-24T09:17:22.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.695+0000 7f27f2800640 1 --2- 172.21.15.83:0/1760399545 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27f41112f0 0x7f27f4111760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:17:22.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.695+0000 7f27f1400640 1 --2- 172.21.15.83:0/1760399545 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f27f41060d0 0x7f27f4110db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:17:22.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.696+0000 7f27f1e00640 1 --2- 172.21.15.83:0/1760399545 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f27f4101000 0x7f27f4113cc0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f27ec00e9f0 tx=0x7f27ec00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:22.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.696+0000 7f27eaa00640 1 -- 172.21.15.83:0/1760399545 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27ec00cd80 con 0x7f27f4101000 2024-04-24T09:17:22.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.696+0000 7f27eaa00640 1 -- 172.21.15.83:0/1760399545 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f27ec0108e0 con 0x7f27f4101000 2024-04-24T09:17:22.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.696+0000 7f27f3200640 1 -- 172.21.15.83:0/1760399545 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27f41ab5a0 con 0x7f27f4101000 2024-04-24T09:17:22.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.696+0000 7f27eaa00640 1 -- 172.21.15.83:0/1760399545 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27ec018920 con 0x7f27f4101000 2024-04-24T09:17:22.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.696+0000 7f27f3200640 1 -- 172.21.15.83:0/1760399545 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27f41abaf0 con 0x7f27f4101000 2024-04-24T09:17:22.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.701+0000 7f27eaa00640 1 -- 172.21.15.83:0/1760399545 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f27ec010a90 con 0x7f27f4101000 2024-04-24T09:17:22.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.704+0000 7f27eaa00640 1 --2- 172.21.15.83:0/1760399545 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f27d80a3d20 0x7f27d80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:22.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.704+0000 7f27eaa00640 1 -- 172.21.15.83:0/1760399545 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 9583+0+0 (secure 0 0 0) 0x7f27ec0d10d0 con 0x7f27f4101000 2024-04-24T09:17:22.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.705+0000 7f27f1400640 1 --2- 172.21.15.83:0/1760399545 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f27d80a3d20 0x7f27d80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:22.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.706+0000 7f27f1400640 1 --2- 172.21.15.83:0/1760399545 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f27d80a3d20 0x7f27d80a61e0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f27e4005e30 tx=0x7f27e40092c0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:22.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.826+0000 7f27fbbdf740 1 Processor -- start 2024-04-24T09:17:22.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.826+0000 7f27fbbdf740 1 -- start start 2024-04-24T09:17:22.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.827+0000 7f27fbbdf740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9f2918a30 0x55e9f2918e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:22.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.827+0000 7f27fbbdf740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e9f2919350 0x55e9f29234e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:22.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.827+0000 7f27fbbdf740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9f2923a20 0x55e9f2925e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:22.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.827+0000 7f27fbbdf740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e9f287e1f0 con 0x55e9f2918a30 2024-04-24T09:17:22.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.827+0000 7f27fbbdf740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e9f2824350 con 0x55e9f2923a20 2024-04-24T09:17:22.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.827+0000 7f27fbbdf740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55e9f287a1b0 con 0x55e9f2919350 2024-04-24T09:17:22.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.827+0000 7f27f2800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9f2923a20 0x55e9f2925e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:22.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.827+0000 7f27f1e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9f2918a30 0x55e9f2918e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:22.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.827+0000 7f27f2800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9f2923a20 0x55e9f2925e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:39114/0 (socket says 172.21.15.83:39114) 2024-04-24T09:17:22.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.827+0000 7f27f2800640 1 -- 172.21.15.83:0/3635887112 learned_addr learned my addr 172.21.15.83:0/3635887112 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:17:22.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.828+0000 7f27f1400640 1 --2- 172.21.15.83:0/3635887112 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e9f2919350 0x55e9f29234e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:22.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.828+0000 7f27f2800640 1 -- 172.21.15.83:0/3635887112 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e9f2919350 msgr2=0x55e9f29234e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:22.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.828+0000 7f27f2800640 1 --2- 172.21.15.83:0/3635887112 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e9f2919350 0x55e9f29234e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:22.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.828+0000 7f27f2800640 1 -- 172.21.15.83:0/3635887112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9f2918a30 msgr2=0x55e9f2918e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:22.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.828+0000 7f27f2800640 1 --2- 172.21.15.83:0/3635887112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9f2918a30 0x55e9f2918e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:22.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.828+0000 7f27f2800640 1 -- 172.21.15.83:0/3635887112 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9f2878f50 con 0x55e9f2923a20 2024-04-24T09:17:22.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.828+0000 7f27f1e00640 1 --2- 172.21.15.83:0/3635887112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9f2918a30 0x55e9f2918e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:17:22.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.828+0000 7f27f2800640 1 --2- 172.21.15.83:0/3635887112 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9f2923a20 0x55e9f2925e10 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f27dc009780 tx=0x7f27dc009520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f88528917a43ff9d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:22.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.829+0000 7f27e3000640 1 -- 172.21.15.83:0/3635887112 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27dc037070 con 0x55e9f2923a20 2024-04-24T09:17:22.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.829+0000 7f27e3000640 1 -- 172.21.15.83:0/3635887112 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f27dc038040 con 0x55e9f2923a20 2024-04-24T09:17:22.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.829+0000 7f27e3000640 1 -- 172.21.15.83:0/3635887112 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27dc005280 con 0x55e9f2923a20 2024-04-24T09:17:22.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.830+0000 7f27fbbdf740 1 -- 172.21.15.83:0/3635887112 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9f2923a20 msgr2=0x55e9f2925e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:22.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.830+0000 7f27fbbdf740 1 --2- 172.21.15.83:0/3635887112 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9f2923a20 0x55e9f2925e10 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f27dc009780 tx=0x7f27dc009520 comp rx=0 tx=0).stop 2024-04-24T09:17:22.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.830+0000 7f27fbbdf740 1 -- 172.21.15.83:0/3635887112 shutdown_connections 2024-04-24T09:17:22.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.830+0000 7f27fbbdf740 1 --2- 172.21.15.83:0/3635887112 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9f2923a20 0x55e9f2925e10 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:22.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.830+0000 7f27fbbdf740 1 --2- 172.21.15.83:0/3635887112 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e9f2919350 0x55e9f29234e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:22.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.830+0000 7f27fbbdf740 1 --2- 172.21.15.83:0/3635887112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9f2918a30 0x55e9f2918e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:22.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.830+0000 7f27fbbdf740 1 -- 172.21.15.83:0/3635887112 >> 172.21.15.83:0/3635887112 conn(0x55e9f2914d50 msgr2=0x55e9f2915130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:22.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.830+0000 7f27fbbdf740 1 -- 172.21.15.83:0/3635887112 shutdown_connections 2024-04-24T09:17:22.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.830+0000 7f27fbbdf740 1 -- 172.21.15.83:0/3635887112 wait complete. 2024-04-24T09:17:22.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.831+0000 7f27fbbdf740 1 -- start start 2024-04-24T09:17:22.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.831+0000 7f27fbbdf740 1 Processor -- start 2024-04-24T09:17:22.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.832+0000 7f27fbbdf740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9f2918a30 0x55e9f2920960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:22.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.832+0000 7f27fbbdf740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9f2919350 0x55e9f2920ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:22.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.832+0000 7f27fbbdf740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e9f2923a20 0x55e9f29213e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:22.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.832+0000 7f27fbbdf740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e9f29267b0 con 0x55e9f2918a30 2024-04-24T09:17:22.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.832+0000 7f27fbbdf740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e9f2837690 con 0x55e9f2919350 2024-04-24T09:17:22.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.832+0000 7f27fbbdf740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55e9f2926930 con 0x55e9f2923a20 2024-04-24T09:17:22.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.832+0000 7f27fbbdf740 10 client.0 _ll_register_callbacks cb 0x55e9f2873650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:17:22.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.832+0000 7f27f1e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9f2918a30 0x55e9f2920960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:22.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.833+0000 7f27f1e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9f2918a30 0x55e9f2920960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35156/0 (socket says 172.21.15.83:35156) 2024-04-24T09:17:22.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.833+0000 7f27f1400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9f2919350 0x55e9f2920ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:22.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.833+0000 7f27f1e00640 1 -- 172.21.15.83:0/844925367 learned_addr learned my addr 172.21.15.83:0/844925367 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:17:22.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.833+0000 7f27f2800640 1 --2- 172.21.15.83:0/844925367 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e9f2923a20 0x55e9f29213e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:22.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.833+0000 7f27f1400640 1 -- 172.21.15.83:0/844925367 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e9f2923a20 msgr2=0x55e9f29213e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:22.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.833+0000 7f27f1400640 1 --2- 172.21.15.83:0/844925367 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e9f2923a20 0x55e9f29213e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:22.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.833+0000 7f27f1400640 1 -- 172.21.15.83:0/844925367 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9f2918a30 msgr2=0x55e9f2920960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:22.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.833+0000 7f27f1400640 1 --2- 172.21.15.83:0/844925367 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9f2918a30 0x55e9f2920960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:22.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.833+0000 7f27f1400640 1 -- 172.21.15.83:0/844925367 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9f2836920 con 0x55e9f2919350 2024-04-24T09:17:22.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.834+0000 7f27f1e00640 1 --2- 172.21.15.83:0/844925367 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9f2918a30 0x55e9f2920960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:17:22.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.834+0000 7f27f1400640 1 --2- 172.21.15.83:0/844925367 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9f2919350 0x55e9f2920ea0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f27e40093b0 tx=0x7f27e4004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de794112db5cfd2f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:22.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.834+0000 7f27d7400640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:17:22.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.834+0000 7f27d7400640 1 -- 172.21.15.83:0/844925367 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27e4003970 con 0x55e9f2919350 2024-04-24T09:17:22.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.834+0000 7f27d7400640 1 -- 172.21.15.83:0/844925367 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(134..134 src has 1..134) ==== 9583+0+0 (secure 0 0 0) 0x7f27e4003b10 con 0x55e9f2919350 2024-04-24T09:17:22.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.835+0000 7f27fbbdf740 1 -- 172.21.15.83:0/844925367 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27dc038040 con 0x55e9f2919350 2024-04-24T09:17:22.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.835+0000 7f27d7400640 1 -- 172.21.15.83:0/844925367 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7f27e4003d10 con 0x55e9f2919350 2024-04-24T09:17:22.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.835+0000 7f27fbbdf740 10 client.41240 fetch_fsmap learned FSMap version 70 2024-04-24T09:17:22.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.835+0000 7f27fbbdf740 1 -- 172.21.15.83:0/844925367 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) -- 0x55e9f265a7d0 con 0x55e9f2919350 2024-04-24T09:17:22.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.835+0000 7f27d7400640 1 -- 172.21.15.83:0/844925367 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f27e40044e0 con 0x55e9f2919350 2024-04-24T09:17:22.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.836+0000 7f27d7400640 1 -- 172.21.15.83:0/844925367 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 70) ==== 8366+0+0 (secure 0 0 0) 0x7f27e400b040 con 0x55e9f2919350 2024-04-24T09:17:22.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.836+0000 7f27fbbdf740 10 client.41240 fetch_fsmap finished waiting for FSMap version 70 2024-04-24T09:17:22.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.836+0000 7f27fbbdf740 10 client.41240 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:17:22.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.836+0000 7f27fbbdf740 20 client.41240 populate_metadata read hostname 'smithi083' 2024-04-24T09:17:22.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.836+0000 7f27fbbdf740 1 --2- 172.21.15.83:0/844925367 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55e9f2933de0 0x55e9f2936250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:22.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.836+0000 7f27fbbdf740 4 client.41240 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:17:22.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.836+0000 7f27fbbdf740 1 -- 172.21.15.83:0/844925367 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f27dc038040 con 0x55e9f2933de0 2024-04-24T09:17:22.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.836+0000 7f27f1e00640 1 --2- 172.21.15.83:0/844925367 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55e9f2933de0 0x55e9f2936250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:22.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.837+0000 7f27f1e00640 1 --2- 172.21.15.83:0/844925367 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55e9f2933de0 0x55e9f2936250 crc :-1 s=READY pgs=625 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2fb77f86b14d649f server_cookie=b27a2baa1bab6167 in_seq=0 out_seq=0 2024-04-24T09:17:22.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.837+0000 7f27d7400640 10 client.41240 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:17:22.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.839+0000 7f27d7400640 1 -- 172.21.15.83:0/844925367 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f27dc038040 con 0x55e9f2933de0 2024-04-24T09:17:22.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.839+0000 7f27d7400640 10 client.41240 handle_command_reply: tid=0 2024-04-24T09:17:22.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.839+0000 7f27fbbdf740 1 client.41240 shutdown 2024-04-24T09:17:22.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.840+0000 7f27fbbdf740 1 -- 172.21.15.83:0/844925367 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9f2919350 msgr2=0x55e9f2920ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:22.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.840+0000 7f27fbbdf740 1 --2- 172.21.15.83:0/844925367 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9f2919350 0x55e9f2920ea0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f27e40093b0 tx=0x7f27e4004ab0 comp rx=0 tx=0).stop 2024-04-24T09:17:22.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.840+0000 7f27fbbdf740 1 -- 172.21.15.83:0/844925367 shutdown_connections 2024-04-24T09:17:22.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.840+0000 7f27fbbdf740 1 --2- 172.21.15.83:0/844925367 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55e9f2933de0 0x55e9f2936250 crc :-1 s=READY pgs=625 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:22.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.840+0000 7f27fbbdf740 1 --2- 172.21.15.83:0/844925367 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e9f2923a20 0x55e9f29213e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:22.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.840+0000 7f27fbbdf740 1 --2- 172.21.15.83:0/844925367 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9f2919350 0x55e9f2920ea0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:22.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.840+0000 7f27fbbdf740 1 --2- 172.21.15.83:0/844925367 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9f2918a30 0x55e9f2920960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:22.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.840+0000 7f27fbbdf740 1 -- 172.21.15.83:0/844925367 >> 172.21.15.83:0/844925367 conn(0x55e9f2914d50 msgr2=0x55e9f291ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:22.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.840+0000 7f27d7400640 0 client.41240 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:17:22.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.841+0000 7f27fbbdf740 1 -- 172.21.15.83:0/844925367 shutdown_connections 2024-04-24T09:17:22.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.841+0000 7f27fbbdf740 1 -- 172.21.15.83:0/844925367 wait complete. 2024-04-24T09:17:22.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.841+0000 7f27fbbdf740 20 client.41240 trim_cache size 0 max 16384 2024-04-24T09:17:22.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.875+0000 7f27fbbdf740 1 Processor -- start 2024-04-24T09:17:22.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.875+0000 7f27fbbdf740 1 -- start start 2024-04-24T09:17:22.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.875+0000 7f27fbbdf740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9f2918a30 0x55e9f292f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:22.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.875+0000 7f27fbbdf740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e9f2919350 0x55e9f291b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:22.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.875+0000 7f27fbbdf740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9f2923a20 0x55e9f293bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:22.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.875+0000 7f27fbbdf740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f27f41b5a70 con 0x55e9f2923a20 2024-04-24T09:17:22.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.875+0000 7f27fbbdf740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f27f41b75e0 con 0x55e9f2918a30 2024-04-24T09:17:22.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.875+0000 7f27fbbdf740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f27f41b6140 con 0x55e9f2919350 2024-04-24T09:17:22.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.876+0000 7f27f1e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9f2918a30 0x55e9f292f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:22.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.876+0000 7f27f2800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9f2923a20 0x55e9f293bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:22.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.876+0000 7f27f1e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9f2918a30 0x55e9f292f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:39142/0 (socket says 172.21.15.83:39142) 2024-04-24T09:17:22.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.876+0000 7f27f2800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9f2923a20 0x55e9f293bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35160/0 (socket says 172.21.15.83:35160) 2024-04-24T09:17:22.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.876+0000 7f27f1e00640 1 -- 172.21.15.83:0/2877461682 learned_addr learned my addr 172.21.15.83:0/2877461682 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:17:22.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.876+0000 7f27f1400640 1 --2- 172.21.15.83:0/2877461682 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e9f2919350 0x55e9f291b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:22.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.876+0000 7f27f1e00640 1 -- 172.21.15.83:0/2877461682 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e9f2919350 msgr2=0x55e9f291b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:22.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.876+0000 7f27f1e00640 1 --2- 172.21.15.83:0/2877461682 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e9f2919350 0x55e9f291b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:22.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.876+0000 7f27f1e00640 1 -- 172.21.15.83:0/2877461682 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9f2923a20 msgr2=0x55e9f293bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:22.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.876+0000 7f27f1e00640 1 --2- 172.21.15.83:0/2877461682 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9f2923a20 0x55e9f293bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:22.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.876+0000 7f27f1e00640 1 -- 172.21.15.83:0/2877461682 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9f25903f0 con 0x55e9f2918a30 2024-04-24T09:17:22.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.876+0000 7f27f2800640 1 --2- 172.21.15.83:0/2877461682 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9f2923a20 0x55e9f293bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:17:22.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.877+0000 7f27f1e00640 1 --2- 172.21.15.83:0/2877461682 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9f2918a30 0x55e9f292f6c0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f27ec009380 tx=0x7f27ec009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e394b6e3e29703d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:22.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.877+0000 7f27e2600640 1 -- 172.21.15.83:0/2877461682 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27ec0b5070 con 0x55e9f2918a30 2024-04-24T09:17:22.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.877+0000 7f27e2600640 1 -- 172.21.15.83:0/2877461682 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f27ec099260 con 0x55e9f2918a30 2024-04-24T09:17:22.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.877+0000 7f27e2600640 1 -- 172.21.15.83:0/2877461682 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27ec09c040 con 0x55e9f2918a30 2024-04-24T09:17:22.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.878+0000 7f27fbbdf740 1 -- 172.21.15.83:0/2877461682 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9f2918a30 msgr2=0x55e9f292f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:22.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.878+0000 7f27fbbdf740 1 --2- 172.21.15.83:0/2877461682 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9f2918a30 0x55e9f292f6c0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f27ec009380 tx=0x7f27ec009310 comp rx=0 tx=0).stop 2024-04-24T09:17:22.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.879+0000 7f27fbbdf740 1 -- 172.21.15.83:0/2877461682 shutdown_connections 2024-04-24T09:17:22.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.879+0000 7f27fbbdf740 1 --2- 172.21.15.83:0/2877461682 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9f2923a20 0x55e9f293bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:22.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.879+0000 7f27fbbdf740 1 --2- 172.21.15.83:0/2877461682 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e9f2919350 0x55e9f291b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:22.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.879+0000 7f27fbbdf740 1 --2- 172.21.15.83:0/2877461682 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9f2918a30 0x55e9f292f6c0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:22.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.879+0000 7f27fbbdf740 1 -- 172.21.15.83:0/2877461682 >> 172.21.15.83:0/2877461682 conn(0x55e9f2914d50 msgr2=0x55e9f29211b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:22.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.879+0000 7f27fbbdf740 1 -- 172.21.15.83:0/2877461682 shutdown_connections 2024-04-24T09:17:22.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.880+0000 7f27fbbdf740 1 -- 172.21.15.83:0/2877461682 wait complete. 2024-04-24T09:17:22.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.880+0000 7f27fbbdf740 1 -- start start 2024-04-24T09:17:22.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.880+0000 7f27fbbdf740 1 Processor -- start 2024-04-24T09:17:22.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.881+0000 7f27fbbdf740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9f2918a30 0x55e9f291d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:22.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.881+0000 7f27fbbdf740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e9f2919350 0x55e9f291da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:22.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.881+0000 7f27fbbdf740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9f2923a20 0x55e9f291dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:22.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.881+0000 7f27fbbdf740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f27f41b6f10 con 0x55e9f2923a20 2024-04-24T09:17:22.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.881+0000 7f27fbbdf740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f27f41b6840 con 0x55e9f2918a30 2024-04-24T09:17:22.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.881+0000 7f27fbbdf740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f27f41b3a70 con 0x55e9f2919350 2024-04-24T09:17:22.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.881+0000 7f27fbbdf740 10 client.0 _ll_register_callbacks cb 0x7f27e4009b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:17:22.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.881+0000 7f27f1e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9f2918a30 0x55e9f291d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:22.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.881+0000 7f27f2800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9f2923a20 0x55e9f291dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:22.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.881+0000 7f27f2800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9f2923a20 0x55e9f291dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35168/0 (socket says 172.21.15.83:35168) 2024-04-24T09:17:22.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.881+0000 7f27f2800640 1 -- 172.21.15.83:0/42549597 learned_addr learned my addr 172.21.15.83:0/42549597 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:17:22.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.882+0000 7f27f1400640 1 --2- 172.21.15.83:0/42549597 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e9f2919350 0x55e9f291da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:22.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.882+0000 7f27f2800640 1 -- 172.21.15.83:0/42549597 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e9f2919350 msgr2=0x55e9f291da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:22.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.882+0000 7f27f2800640 1 --2- 172.21.15.83:0/42549597 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e9f2919350 0x55e9f291da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:22.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.882+0000 7f27f2800640 1 -- 172.21.15.83:0/42549597 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9f2918a30 msgr2=0x55e9f291d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:22.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.882+0000 7f27f2800640 1 --2- 172.21.15.83:0/42549597 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9f2918a30 0x55e9f291d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:22.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.882+0000 7f27f2800640 1 -- 172.21.15.83:0/42549597 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9f25500f0 con 0x55e9f2923a20 2024-04-24T09:17:22.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.882+0000 7f27f2800640 1 --2- 172.21.15.83:0/42549597 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9f2923a20 0x55e9f291dfd0 secure :-1 s=READY pgs=2827 cs=0 l=1 rev1=1 crypto rx=0x7f27dc041040 tx=0x7f27dc0044b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2729a539a1d110d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:22.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.883+0000 7f27d7400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:17:22.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.883+0000 7f27d7400640 1 -- 172.21.15.83:0/42549597 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27dc037040 con 0x55e9f2923a20 2024-04-24T09:17:22.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.883+0000 7f27d7400640 1 -- 172.21.15.83:0/42549597 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(134..134 src has 1..134) ==== 9583+0+0 (secure 0 0 0) 0x7f27dc030650 con 0x55e9f2923a20 2024-04-24T09:17:22.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.883+0000 7f27fbbdf740 1 -- 172.21.15.83:0/42549597 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27ec099260 con 0x55e9f2923a20 2024-04-24T09:17:22.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.883+0000 7f27d7400640 1 -- 172.21.15.83:0/42549597 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7f27dc02a9e0 con 0x55e9f2923a20 2024-04-24T09:17:22.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.883+0000 7f27fbbdf740 10 client.24186 fetch_fsmap learned FSMap version 70 2024-04-24T09:17:22.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.883+0000 7f27fbbdf740 1 -- 172.21.15.83:0/42549597 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) -- 0x55e9f2843810 con 0x55e9f2923a20 2024-04-24T09:17:22.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.884+0000 7f27d7400640 1 -- 172.21.15.83:0/42549597 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f27ec099260 con 0x55e9f2923a20 2024-04-24T09:17:22.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.884+0000 7f27d7400640 1 -- 172.21.15.83:0/42549597 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 70) ==== 8366+0+0 (secure 0 0 0) 0x7f27dc004a60 con 0x55e9f2923a20 2024-04-24T09:17:22.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.884+0000 7f27fbbdf740 10 client.24186 fetch_fsmap finished waiting for FSMap version 70 2024-04-24T09:17:22.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.884+0000 7f27fbbdf740 10 client.24186 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:17:22.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.884+0000 7f27fbbdf740 20 client.24186 populate_metadata read hostname 'smithi083' 2024-04-24T09:17:22.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.884+0000 7f27fbbdf740 1 --2- 172.21.15.83:0/42549597 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55e9f299a5e0 0x55e9f299c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:22.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.884+0000 7f27fbbdf740 4 client.24186 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T09:17:22.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.884+0000 7f27fbbdf740 1 -- 172.21.15.83:0/42549597 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e9f288bc10 con 0x55e9f299a5e0 2024-04-24T09:17:22.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.884+0000 7f27f1e00640 1 --2- 172.21.15.83:0/42549597 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55e9f299a5e0 0x55e9f299c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:22.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.885+0000 7f27d7400640 10 client.24186 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:17:22.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.885+0000 7f27f1e00640 1 --2- 172.21.15.83:0/42549597 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55e9f299a5e0 0x55e9f299c9d0 crc :-1 s=READY pgs=626 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=273c07ff8f72b9e6 server_cookie=df25aae7c75b9472 in_seq=0 out_seq=0 2024-04-24T09:17:22.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.885+0000 7f27d7400640 1 -- 172.21.15.83:0/42549597 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55e9f288bc10 con 0x55e9f299a5e0 2024-04-24T09:17:22.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.885+0000 7f27d7400640 10 client.24186 handle_command_reply: tid=0 2024-04-24T09:17:22.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.885+0000 7f27fbbdf740 1 client.24186 shutdown 2024-04-24T09:17:22.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.886+0000 7f27fbbdf740 1 -- 172.21.15.83:0/42549597 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9f2923a20 msgr2=0x55e9f291dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:22.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.886+0000 7f27fbbdf740 1 --2- 172.21.15.83:0/42549597 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9f2923a20 0x55e9f291dfd0 secure :-1 s=READY pgs=2827 cs=0 l=1 rev1=1 crypto rx=0x7f27dc041040 tx=0x7f27dc0044b0 comp rx=0 tx=0).stop 2024-04-24T09:17:22.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.886+0000 7f27fbbdf740 1 -- 172.21.15.83:0/42549597 shutdown_connections 2024-04-24T09:17:22.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.886+0000 7f27fbbdf740 1 --2- 172.21.15.83:0/42549597 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55e9f299a5e0 0x55e9f299c9d0 crc :-1 s=READY pgs=626 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:22.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.886+0000 7f27fbbdf740 1 --2- 172.21.15.83:0/42549597 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9f2923a20 0x55e9f291dfd0 unknown :-1 s=CLOSED pgs=2827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:22.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.886+0000 7f27fbbdf740 1 --2- 172.21.15.83:0/42549597 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e9f2919350 0x55e9f291da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:22.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.886+0000 7f27fbbdf740 1 --2- 172.21.15.83:0/42549597 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9f2918a30 0x55e9f291d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:22.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.886+0000 7f27d7400640 0 client.24186 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:17:22.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.886+0000 7f27fbbdf740 1 -- 172.21.15.83:0/42549597 >> 172.21.15.83:0/42549597 conn(0x55e9f2914d50 msgr2=0x55e9f2913740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:22.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.886+0000 7f27fbbdf740 1 -- 172.21.15.83:0/42549597 shutdown_connections 2024-04-24T09:17:22.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.886+0000 7f27fbbdf740 1 -- 172.21.15.83:0/42549597 wait complete. 2024-04-24T09:17:22.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.886+0000 7f27fbbdf740 20 client.24186 trim_cache size 0 max 16384 2024-04-24T09:17:22.888 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T09:17:22.888 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-04-24T09:17:22.888 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-04-24T09:17:22.888 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T09:17:22.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.887+0000 7f27f3200640 1 -- 172.21.15.83:0/1760399545 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f27d80a3d20 msgr2=0x7f27d80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:22.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.887+0000 7f27f3200640 1 --2- 172.21.15.83:0/1760399545 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f27d80a3d20 0x7f27d80a61e0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f27e4005e30 tx=0x7f27e40092c0 comp rx=0 tx=0).stop 2024-04-24T09:17:22.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.887+0000 7f27f3200640 1 -- 172.21.15.83:0/1760399545 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f27f4101000 msgr2=0x7f27f4113cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:22.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.887+0000 7f27f3200640 1 --2- 172.21.15.83:0/1760399545 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f27f4101000 0x7f27f4113cc0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f27ec00e9f0 tx=0x7f27ec00eeb0 comp rx=0 tx=0).stop 2024-04-24T09:17:22.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.887+0000 7f27f3200640 1 -- 172.21.15.83:0/1760399545 shutdown_connections 2024-04-24T09:17:22.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.887+0000 7f27f3200640 1 --2- 172.21.15.83:0/1760399545 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f27d80a3d20 0x7f27d80a61e0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:22.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.887+0000 7f27f3200640 1 --2- 172.21.15.83:0/1760399545 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27f41112f0 0x7f27f4111760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:22.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.887+0000 7f27f3200640 1 --2- 172.21.15.83:0/1760399545 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f27f41060d0 0x7f27f4110db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:22.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.887+0000 7f27f3200640 1 --2- 172.21.15.83:0/1760399545 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f27f4101000 0x7f27f4113cc0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:22.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.887+0000 7f27f3200640 1 -- 172.21.15.83:0/1760399545 >> 172.21.15.83:0/1760399545 conn(0x7f27f40fc880 msgr2=0x7f27f410dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:22.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.888+0000 7f27f3200640 1 -- 172.21.15.83:0/1760399545 shutdown_connections 2024-04-24T09:17:22.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:22.888+0000 7f27f3200640 1 -- 172.21.15.83:0/1760399545 wait complete. 2024-04-24T09:17:22.912 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T09:17:22.912 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T09:17:23.230 INFO:tasks.workunit.client.0.smithi083.stdout: 10 86838 6.48 MB/sec execute 594 sec latency 88.422 ms 2024-04-24T09:17:23.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:23.500+0000 7f2dae200640 1 -- 172.21.15.83:0/257896537 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2da8069400 msgr2=0x7f2da80697e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:23.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:23.500+0000 7f2dae200640 1 --2- 172.21.15.83:0/257896537 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2da8069400 0x7f2da80697e0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c009900 tx=0x7f2d9c02a400 comp rx=0 tx=0).stop 2024-04-24T09:17:23.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:23.501+0000 7f2dae200640 1 -- 172.21.15.83:0/257896537 shutdown_connections 2024-04-24T09:17:23.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:23.501+0000 7f2dae200640 1 --2- 172.21.15.83:0/257896537 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2da81071a0 0x7f2da810ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:23.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:23.501+0000 7f2dae200640 1 --2- 172.21.15.83:0/257896537 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2da8069db0 0x7f2da8106a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:23.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:23.501+0000 7f2dae200640 1 --2- 172.21.15.83:0/257896537 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2da8069400 0x7f2da80697e0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:23.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:23.501+0000 7f2dae200640 1 -- 172.21.15.83:0/257896537 >> 172.21.15.83:0/257896537 conn(0x7f2da8076840 msgr2=0x7f2da8076c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:23.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:23.502+0000 7f2dae200640 1 -- 172.21.15.83:0/257896537 shutdown_connections 2024-04-24T09:17:23.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:23.502+0000 7f2dae200640 1 -- 172.21.15.83:0/257896537 wait complete. 2024-04-24T09:17:23.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:23.504+0000 7f2dae200640 1 Processor -- start 2024-04-24T09:17:23.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:23.505+0000 7f2dae200640 1 -- start start 2024-04-24T09:17:23.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:23.505+0000 7f2dae200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2da8069400 0x7f2da8102b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:23.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:23.505+0000 7f2dae200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2da81071a0 0x7f2da81010c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:23.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:23.505+0000 7f2dae200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2da8101600 0x7f2da8101a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:23.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:23.505+0000 7f2dae200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2da8112180 con 0x7f2da8069400 2024-04-24T09:17:23.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:23.505+0000 7f2dae200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2da8112000 con 0x7f2da8101600 2024-04-24T09:17:23.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:23.505+0000 7f2dae200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2da8112300 con 0x7f2da81071a0 2024-04-24T09:17:23.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:23.506+0000 7f2dace00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2da8069400 0x7f2da8102b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:23.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:23.506+0000 7f2dace00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2da8069400 0x7f2da8102b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35190/0 (socket says 172.21.15.83:35190) 2024-04-24T09:17:23.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:23.506+0000 7f2dace00640 1 -- 172.21.15.83:0/236221312 learned_addr learned my addr 172.21.15.83:0/236221312 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:17:23.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:23.506+0000 7f2da7e00640 1 --2- 172.21.15.83:0/236221312 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2da81071a0 0x7f2da81010c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:23.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:23.506+0000 7f2dace00640 1 -- 172.21.15.83:0/236221312 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2da81071a0 msgr2=0x7f2da81010c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:23.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:23.506+0000 7f2dace00640 1 --2- 172.21.15.83:0/236221312 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2da81071a0 0x7f2da81010c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:23.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:23.506+0000 7f2dace00640 1 -- 172.21.15.83:0/236221312 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2da8101600 msgr2=0x7f2da8101a90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T09:17:23.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:23.507+0000 7f2dad800640 1 --2- 172.21.15.83:0/236221312 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2da8101600 0x7f2da8101a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:23.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:23.507+0000 7f2dace00640 1 --2- 172.21.15.83:0/236221312 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2da8101600 0x7f2da8101a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:23.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:23.507+0000 7f2dace00640 1 -- 172.21.15.83:0/236221312 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2da81af720 con 0x7f2da8069400 2024-04-24T09:17:23.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:23.507+0000 7f2dace00640 1 --2- 172.21.15.83:0/236221312 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2da8069400 0x7f2da8102b30 secure :-1 s=READY pgs=2828 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c02a9f0 tx=0x7f2d9c003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:23.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:23.508+0000 7f2da5600640 1 -- 172.21.15.83:0/236221312 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d9c03f070 con 0x7f2da8069400 2024-04-24T09:17:23.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:23.508+0000 7f2dae200640 1 -- 172.21.15.83:0/236221312 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2da81af9b0 con 0x7f2da8069400 2024-04-24T09:17:23.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:23.508+0000 7f2da5600640 1 -- 172.21.15.83:0/236221312 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2d9c031b50 con 0x7f2da8069400 2024-04-24T09:17:23.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:23.508+0000 7f2da5600640 1 -- 172.21.15.83:0/236221312 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d9c035030 con 0x7f2da8069400 2024-04-24T09:17:23.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:23.508+0000 7f2dae200640 1 -- 172.21.15.83:0/236221312 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2da81afef0 con 0x7f2da8069400 2024-04-24T09:17:23.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:23.511+0000 7f2dae200640 1 -- 172.21.15.83:0/236221312 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2da8103550 con 0x7f2da8069400 2024-04-24T09:17:23.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:23.515+0000 7f2da5600640 1 -- 172.21.15.83:0/236221312 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2d9c042070 con 0x7f2da8069400 2024-04-24T09:17:23.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:23.517+0000 7f2da5600640 1 --2- 172.21.15.83:0/236221312 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f2d900a3e20 0x7f2d900a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:23.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:23.518+0000 7f2da5600640 1 -- 172.21.15.83:0/236221312 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 9583+0+0 (secure 0 0 0) 0x7f2d9c0f16c0 con 0x7f2da8069400 2024-04-24T09:17:23.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:23.519+0000 7f2da7e00640 1 --2- 172.21.15.83:0/236221312 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f2d900a3e20 0x7f2d900a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:23.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:23.521+0000 7f2da7e00640 1 --2- 172.21.15.83:0/236221312 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f2d900a3e20 0x7f2d900a62e0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f2da8102880 tx=0x7f2d94005f90 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:23.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:23.526+0000 7f2da5600640 1 -- 172.21.15.83:0/236221312 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2d9c0bd8e0 con 0x7f2da8069400 2024-04-24T09:17:23.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:23.948+0000 7f2dae200640 1 -- 172.21.15.83:0/236221312 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2da8068d40 con 0x7f2da8069400 2024-04-24T09:17:23.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:23.949+0000 7f2da5600640 1 -- 172.21.15.83:0/236221312 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) ==== 94+0+11945 (secure 0 0 0) 0x7f2da8068d40 con 0x7f2da8069400 2024-04-24T09:17:23.950 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T09:17:23.950 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":70,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/358872257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":358872257},{"type":"v1","addr":"172.21.15.83:6835","nonce":358872257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3157070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3157070851},{"type":"v1","addr":"172.21.15.83:6837","nonce":3157070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":70,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T08:48:29.422576+0000","modified":"2024-04-24T09:17:13.321709+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34520,"mds_1":34493,"mds_2":34451,"mds_3":34427,"mds_4":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14658":{"gid":14658,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6839/4085470844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":4085470844},{"type":"v1","addr":"172.21.15.83:6839","nonce":4085470844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14676":{"gid":14676,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6841/2634754210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":2634754210},{"type":"v1","addr":"172.21.15.83:6841","nonce":2634754210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24382":{"gid":24382,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/1859921310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1859921310},{"type":"v1","addr":"172.21.15.189:6833","nonce":1859921310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/2241923197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2241923197},{"type":"v1","addr":"172.21.15.189:6835","nonce":2241923197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.189:6839/2548965336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":2548965336},{"type":"v1","addr":"172.21.15.189:6839","nonce":2548965336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26686":{"gid":26686,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6837/1678817161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":1678817161},{"type":"v1","addr":"172.21.15.189:6837","nonce":1678817161}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.138:6833/1157967658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1157967658},{"type":"v1","addr":"172.21.15.138:6833","nonce":1157967658}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.138:6835/1909219758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1909219758},{"type":"v1","addr":"172.21.15.138:6835","nonce":1909219758}]},"join_fscid":-1,"export_targets":[4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34493":{"gid":34493,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.138:6837/900566754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":900566754},{"type":"v1","addr":"172.21.15.138:6837","nonce":900566754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34520":{"gid":34520,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.138:6839/2236932393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2236932393},{"type":"v1","addr":"172.21.15.138:6839","nonce":2236932393}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34520,"qdb_cluster":[34493,34451,34427,24454,34520]},"id":1}]} 2024-04-24T09:17:23.950 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 70 2024-04-24T09:17:23.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:23.952+0000 7f2dae200640 1 -- 172.21.15.83:0/236221312 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f2d900a3e20 msgr2=0x7f2d900a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:23.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:23.952+0000 7f2dae200640 1 --2- 172.21.15.83:0/236221312 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f2d900a3e20 0x7f2d900a62e0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f2da8102880 tx=0x7f2d94005f90 comp rx=0 tx=0).stop 2024-04-24T09:17:23.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:23.952+0000 7f2dae200640 1 -- 172.21.15.83:0/236221312 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2da8069400 msgr2=0x7f2da8102b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:23.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:23.952+0000 7f2dae200640 1 --2- 172.21.15.83:0/236221312 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2da8069400 0x7f2da8102b30 secure :-1 s=READY pgs=2828 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c02a9f0 tx=0x7f2d9c003c80 comp rx=0 tx=0).stop 2024-04-24T09:17:23.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:23.953+0000 7f2dae200640 1 -- 172.21.15.83:0/236221312 shutdown_connections 2024-04-24T09:17:23.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:23.953+0000 7f2dae200640 1 --2- 172.21.15.83:0/236221312 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f2d900a3e20 0x7f2d900a62e0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:23.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:23.953+0000 7f2dae200640 1 --2- 172.21.15.83:0/236221312 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2da8101600 0x7f2da8101a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:23.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:23.953+0000 7f2dae200640 1 --2- 172.21.15.83:0/236221312 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2da81071a0 0x7f2da81010c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:23.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:23.953+0000 7f2dae200640 1 --2- 172.21.15.83:0/236221312 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2da8069400 0x7f2da8102b30 unknown :-1 s=CLOSED pgs=2828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:23.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:23.953+0000 7f2dae200640 1 -- 172.21.15.83:0/236221312 >> 172.21.15.83:0/236221312 conn(0x7f2da8076840 msgr2=0x7f2da80ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:23.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:23.953+0000 7f2dae200640 1 -- 172.21.15.83:0/236221312 shutdown_connections 2024-04-24T09:17:23.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:23.953+0000 7f2dae200640 1 -- 172.21.15.83:0/236221312 wait complete. 2024-04-24T09:17:23.978 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-24T09:17:24.230 INFO:tasks.workunit.client.0.smithi083.stdout: 10 86958 6.48 MB/sec execute 595 sec latency 108.410 ms 2024-04-24T09:17:24.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.251+0000 7f1ca1400640 1 -- 172.21.15.83:0/1479715810 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c9c100c60 msgr2=0x7f1c9c101040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:24.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.251+0000 7f1ca1400640 1 --2- 172.21.15.83:0/1479715810 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c9c100c60 0x7f1c9c101040 secure :-1 s=READY pgs=2829 cs=0 l=1 rev1=1 crypto rx=0x7f1c8c009900 tx=0x7f1c8c02a3f0 comp rx=0 tx=0).stop 2024-04-24T09:17:24.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.252+0000 7f1ca1400640 1 -- 172.21.15.83:0/1479715810 shutdown_connections 2024-04-24T09:17:24.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.252+0000 7f1ca1400640 1 --2- 172.21.15.83:0/1479715810 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1c9c10e810 0x7f1c9c110c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:24.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.252+0000 7f1ca1400640 1 --2- 172.21.15.83:0/1479715810 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c9c101610 0x7f1c9c10e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:24.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.252+0000 7f1ca1400640 1 --2- 172.21.15.83:0/1479715810 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c9c100c60 0x7f1c9c101040 unknown :-1 s=CLOSED pgs=2829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:24.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.252+0000 7f1ca1400640 1 -- 172.21.15.83:0/1479715810 >> 172.21.15.83:0/1479715810 conn(0x7f1c9c0fc880 msgr2=0x7f1c9c0feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:24.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.252+0000 7f1ca1400640 1 -- 172.21.15.83:0/1479715810 shutdown_connections 2024-04-24T09:17:24.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.252+0000 7f1ca1400640 1 -- 172.21.15.83:0/1479715810 wait complete. 2024-04-24T09:17:24.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.253+0000 7f1ca1400640 1 Processor -- start 2024-04-24T09:17:24.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.253+0000 7f1ca1400640 1 -- start start 2024-04-24T09:17:24.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.254+0000 7f1ca1400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c9c100c60 0x7f1c9c1a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:24.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.254+0000 7f1ca1400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1c9c101610 0x7f1c9c1a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:24.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.254+0000 7f1ca1400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c9c10e810 0x7f1c9c19a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:24.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.254+0000 7f1ca1400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1c9c1143b0 con 0x7f1c9c10e810 2024-04-24T09:17:24.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.254+0000 7f1ca1400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1c9c114230 con 0x7f1c9c100c60 2024-04-24T09:17:24.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.254+0000 7f1ca1400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1c9c114530 con 0x7f1c9c101610 2024-04-24T09:17:24.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.254+0000 7f1ca0a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c9c10e810 0x7f1c9c19a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:24.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.254+0000 7f1ca0a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c9c10e810 0x7f1c9c19a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35216/0 (socket says 172.21.15.83:35216) 2024-04-24T09:17:24.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.254+0000 7f1ca0a00640 1 -- 172.21.15.83:0/3423320783 learned_addr learned my addr 172.21.15.83:0/3423320783 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:17:24.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.255+0000 7f1c93400640 1 --2- 172.21.15.83:0/3423320783 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1c9c101610 0x7f1c9c1a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:24.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.255+0000 7f1ca0a00640 1 -- 172.21.15.83:0/3423320783 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1c9c101610 msgr2=0x7f1c9c1a0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:24.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.255+0000 7f1ca0a00640 1 --2- 172.21.15.83:0/3423320783 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1c9c101610 0x7f1c9c1a0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:24.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.255+0000 7f1ca0a00640 1 -- 172.21.15.83:0/3423320783 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c9c100c60 msgr2=0x7f1c9c1a0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:24.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.255+0000 7f1ca0a00640 1 --2- 172.21.15.83:0/3423320783 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c9c100c60 0x7f1c9c1a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:24.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.255+0000 7f1ca0a00640 1 -- 172.21.15.83:0/3423320783 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c9c19afd0 con 0x7f1c9c10e810 2024-04-24T09:17:24.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.255+0000 7f1ca0a00640 1 --2- 172.21.15.83:0/3423320783 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c9c10e810 0x7f1c9c19a710 secure :-1 s=READY pgs=2830 cs=0 l=1 rev1=1 crypto rx=0x7f1c94009fd0 tx=0x7f1c9400ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:24.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.256+0000 7f1c99600640 1 -- 172.21.15.83:0/3423320783 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c94004d20 con 0x7f1c9c10e810 2024-04-24T09:17:24.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.256+0000 7f1c99600640 1 -- 172.21.15.83:0/3423320783 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1c94005020 con 0x7f1c9c10e810 2024-04-24T09:17:24.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.256+0000 7f1c99600640 1 -- 172.21.15.83:0/3423320783 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c940116b0 con 0x7f1c9c10e810 2024-04-24T09:17:24.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.256+0000 7f1ca1400640 1 -- 172.21.15.83:0/3423320783 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c9c19b2c0 con 0x7f1c9c10e810 2024-04-24T09:17:24.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.256+0000 7f1ca1400640 1 -- 172.21.15.83:0/3423320783 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c9c1a75d0 con 0x7f1c9c10e810 2024-04-24T09:17:24.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.260+0000 7f1ca5977740 1 Processor -- start 2024-04-24T09:17:24.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.261+0000 7f1c99600640 1 -- 172.21.15.83:0/3423320783 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1c940051d0 con 0x7f1c9c10e810 2024-04-24T09:17:24.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.261+0000 7f1ca5977740 1 -- start start 2024-04-24T09:17:24.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.262+0000 7f1ca5977740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564d39c66b40 0x564d39c66f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:24.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.262+0000 7f1ca5977740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564d39c67460 0x564d39c715f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:24.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.262+0000 7f1ca5977740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564d39c71b30 0x564d39c73f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:24.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.262+0000 7f1ca5977740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x564d39bcc1d0 con 0x564d39c67460 2024-04-24T09:17:24.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.262+0000 7f1ca5977740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x564d39b72450 con 0x564d39c66b40 2024-04-24T09:17:24.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.262+0000 7f1ca5977740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x564d39bc8190 con 0x564d39c71b30 2024-04-24T09:17:24.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.262+0000 7f1c9be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564d39c67460 0x564d39c715f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:24.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.262+0000 7f1c93400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564d39c66b40 0x564d39c66f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:24.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.262+0000 7f1c93400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564d39c66b40 0x564d39c66f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:39194/0 (socket says 172.21.15.83:39194) 2024-04-24T09:17:24.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.262+0000 7f1c9be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564d39c67460 0x564d39c715f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35230/0 (socket says 172.21.15.83:35230) 2024-04-24T09:17:24.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.262+0000 7f1c93400640 1 -- 172.21.15.83:0/3736456557 learned_addr learned my addr 172.21.15.83:0/3736456557 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:17:24.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.263+0000 7f1c93400640 1 --2- 172.21.15.83:0/3736456557 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564d39c71b30 0x564d39c73f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:24.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.263+0000 7f1c93400640 1 -- 172.21.15.83:0/3736456557 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564d39c71b30 msgr2=0x564d39c73f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:24.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.263+0000 7f1c93400640 1 --2- 172.21.15.83:0/3736456557 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564d39c71b30 0x564d39c73f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:24.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.263+0000 7f1c93400640 1 -- 172.21.15.83:0/3736456557 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564d39c67460 msgr2=0x564d39c715f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:24.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.263+0000 7f1c93400640 1 --2- 172.21.15.83:0/3736456557 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564d39c67460 0x564d39c715f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:24.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.263+0000 7f1c93400640 1 -- 172.21.15.83:0/3736456557 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d39c2f310 con 0x564d39c66b40 2024-04-24T09:17:24.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.263+0000 7f1c9be00640 1 --2- 172.21.15.83:0/3736456557 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564d39c67460 0x564d39c715f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:17:24.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.263+0000 7f1c99600640 1 --2- 172.21.15.83:0/3423320783 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f1c800a3da0 0x7f1c800a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:24.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.263+0000 7f1c93400640 1 --2- 172.21.15.83:0/3736456557 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564d39c66b40 0x564d39c66f20 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f1c84009f90 tx=0x7f1c84009790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c636b5734ab3cfa server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:24.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.264+0000 7f1c99600640 1 -- 172.21.15.83:0/3423320783 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 9583+0+0 (secure 0 0 0) 0x7f1c940d11e0 con 0x7f1c9c10e810 2024-04-24T09:17:24.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.264+0000 7f1c9be00640 1 --2- 172.21.15.83:0/3423320783 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f1c800a3da0 0x7f1c800a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:24.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.264+0000 7f1c91200640 1 -- 172.21.15.83:0/3736456557 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c84004250 con 0x564d39c66b40 2024-04-24T09:17:24.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.264+0000 7f1c91200640 1 -- 172.21.15.83:0/3736456557 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1c840043f0 con 0x564d39c66b40 2024-04-24T09:17:24.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.264+0000 7f1c91200640 1 -- 172.21.15.83:0/3736456557 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c84010910 con 0x564d39c66b40 2024-04-24T09:17:24.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.265+0000 7f1ca5977740 1 -- 172.21.15.83:0/3736456557 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564d39c66b40 msgr2=0x564d39c66f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:24.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.265+0000 7f1ca5977740 1 --2- 172.21.15.83:0/3736456557 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564d39c66b40 0x564d39c66f20 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f1c84009f90 tx=0x7f1c84009790 comp rx=0 tx=0).stop 2024-04-24T09:17:24.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.265+0000 7f1c9be00640 1 --2- 172.21.15.83:0/3423320783 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f1c800a3da0 0x7f1c800a6260 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f1c8c0057c0 tx=0x7f1c8c031400 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:24.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.265+0000 7f1ca5977740 1 -- 172.21.15.83:0/3736456557 shutdown_connections 2024-04-24T09:17:24.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.265+0000 7f1ca5977740 1 --2- 172.21.15.83:0/3736456557 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564d39c71b30 0x564d39c73f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:24.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.265+0000 7f1ca5977740 1 --2- 172.21.15.83:0/3736456557 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564d39c67460 0x564d39c715f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:24.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.265+0000 7f1ca5977740 1 --2- 172.21.15.83:0/3736456557 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564d39c66b40 0x564d39c66f20 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:24.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.265+0000 7f1ca5977740 1 -- 172.21.15.83:0/3736456557 >> 172.21.15.83:0/3736456557 conn(0x564d39c62e60 msgr2=0x564d39c63240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:24.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.265+0000 7f1ca5977740 1 -- 172.21.15.83:0/3736456557 shutdown_connections 2024-04-24T09:17:24.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.265+0000 7f1ca5977740 1 -- 172.21.15.83:0/3736456557 wait complete. 2024-04-24T09:17:24.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.266+0000 7f1ca5977740 1 -- start start 2024-04-24T09:17:24.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.266+0000 7f1ca5977740 1 Processor -- start 2024-04-24T09:17:24.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.267+0000 7f1ca5977740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564d39c66b40 0x564d39c6e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:24.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.267+0000 7f1ca5977740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564d39c67460 0x564d39c6ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:24.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.267+0000 7f1ca5977740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564d39c71b30 0x564d39c6f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:24.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.267+0000 7f1ca5977740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x564d39c749c0 con 0x564d39c66b40 2024-04-24T09:17:24.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.267+0000 7f1ca5977740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x564d39b85810 con 0x564d39c67460 2024-04-24T09:17:24.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.267+0000 7f1ca5977740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x564d39c74b40 con 0x564d39c71b30 2024-04-24T09:17:24.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.267+0000 7f1ca5977740 10 client.0 _ll_register_callbacks cb 0x564d39b53140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:17:24.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.267+0000 7f1c93400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564d39c67460 0x564d39c6ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:24.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.267+0000 7f1c93400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564d39c67460 0x564d39c6ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:39202/0 (socket says 172.21.15.83:39202) 2024-04-24T09:17:24.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.267+0000 7f1c93400640 1 -- 172.21.15.83:0/2809061064 learned_addr learned my addr 172.21.15.83:0/2809061064 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:17:24.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.267+0000 7f1c9be00640 1 --2- 172.21.15.83:0/2809061064 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564d39c66b40 0x564d39c6e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:24.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.268+0000 7f1c93400640 1 -- 172.21.15.83:0/2809061064 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564d39c71b30 msgr2=0x564d39c6f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:24.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.268+0000 7f1c93400640 1 --2- 172.21.15.83:0/2809061064 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564d39c71b30 0x564d39c6f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:24.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.268+0000 7f1c93400640 1 -- 172.21.15.83:0/2809061064 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564d39c66b40 msgr2=0x564d39c6e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:24.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.268+0000 7f1c93400640 1 --2- 172.21.15.83:0/2809061064 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564d39c66b40 0x564d39c6e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:24.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.268+0000 7f1c93400640 1 -- 172.21.15.83:0/2809061064 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d39b963f0 con 0x564d39c67460 2024-04-24T09:17:24.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.268+0000 7f1c93400640 1 --2- 172.21.15.83:0/2809061064 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564d39c67460 0x564d39c6ee50 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f1c840052f0 tx=0x7f1c84005320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84dcecbe75043e65 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:24.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.268+0000 7f1c89600640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:17:24.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.268+0000 7f1c89600640 1 -- 172.21.15.83:0/2809061064 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c8402e060 con 0x564d39c67460 2024-04-24T09:17:24.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.268+0000 7f1c89600640 1 -- 172.21.15.83:0/2809061064 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(134..134 src has 1..134) ==== 9583+0+0 (secure 0 0 0) 0x7f1c840048c0 con 0x564d39c67460 2024-04-24T09:17:24.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.269+0000 7f1ca5977740 1 -- 172.21.15.83:0/2809061064 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c840043f0 con 0x564d39c67460 2024-04-24T09:17:24.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.269+0000 7f1c89600640 1 -- 172.21.15.83:0/2809061064 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7f1c84035050 con 0x564d39c67460 2024-04-24T09:17:24.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.269+0000 7f1ca5977740 10 client.41258 fetch_fsmap learned FSMap version 70 2024-04-24T09:17:24.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.269+0000 7f1ca5977740 1 -- 172.21.15.83:0/2809061064 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) -- 0x564d398de3f0 con 0x564d39c67460 2024-04-24T09:17:24.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.269+0000 7f1c89600640 1 -- 172.21.15.83:0/2809061064 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1c840043f0 con 0x564d39c67460 2024-04-24T09:17:24.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.270+0000 7f1c89600640 1 -- 172.21.15.83:0/2809061064 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 70) ==== 8366+0+0 (secure 0 0 0) 0x7f1c8401c020 con 0x564d39c67460 2024-04-24T09:17:24.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.270+0000 7f1ca5977740 10 client.41258 fetch_fsmap finished waiting for FSMap version 70 2024-04-24T09:17:24.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.270+0000 7f1ca5977740 10 client.41258 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-24T09:17:24.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.270+0000 7f1ca5977740 20 client.41258 populate_metadata read hostname 'smithi083' 2024-04-24T09:17:24.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.270+0000 7f1ca5977740 1 --2- 172.21.15.83:0/2809061064 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x564d39c8a470 0x564d39c8c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:24.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.270+0000 7f1ca5977740 4 client.41258 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:17:24.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.270+0000 7f1ca5977740 1 -- 172.21.15.83:0/2809061064 --> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564d39bd9bf0 con 0x564d39c8a470 2024-04-24T09:17:24.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.270+0000 7f1c9be00640 1 --2- 172.21.15.83:0/2809061064 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x564d39c8a470 0x564d39c8c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:24.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.272+0000 7f1c9be00640 1 --2- 172.21.15.83:0/2809061064 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x564d39c8a470 0x564d39c8c860 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e7eb65197ecb05e6 server_cookie=b5550fc171a959a0 in_seq=0 out_seq=0 2024-04-24T09:17:24.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.272+0000 7f1c89600640 10 client.41258 ms_handle_connect on v2:172.21.15.189:6838/2548965336 2024-04-24T09:17:24.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.274+0000 7f1c89600640 1 -- 172.21.15.83:0/2809061064 <== mds.4 v2:172.21.15.189:6838/2548965336 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564d39bd9bf0 con 0x564d39c8a470 2024-04-24T09:17:24.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.275+0000 7f1c89600640 10 client.41258 handle_command_reply: tid=0 2024-04-24T09:17:24.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.275+0000 7f1ca5977740 1 client.41258 shutdown 2024-04-24T09:17:24.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.275+0000 7f1ca5977740 1 -- 172.21.15.83:0/2809061064 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564d39c67460 msgr2=0x564d39c6ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:24.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.275+0000 7f1ca5977740 1 --2- 172.21.15.83:0/2809061064 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564d39c67460 0x564d39c6ee50 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f1c840052f0 tx=0x7f1c84005320 comp rx=0 tx=0).stop 2024-04-24T09:17:24.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.276+0000 7f1ca5977740 1 -- 172.21.15.83:0/2809061064 shutdown_connections 2024-04-24T09:17:24.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.276+0000 7f1ca5977740 1 --2- 172.21.15.83:0/2809061064 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x564d39c8a470 0x564d39c8c860 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:24.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.276+0000 7f1c89600640 0 client.41258 ms_handle_reset on v2:172.21.15.189:6838/2548965336 2024-04-24T09:17:24.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.276+0000 7f1ca5977740 1 --2- 172.21.15.83:0/2809061064 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564d39c71b30 0x564d39c6f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:24.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.276+0000 7f1ca5977740 1 --2- 172.21.15.83:0/2809061064 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564d39c67460 0x564d39c6ee50 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:24.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.276+0000 7f1ca5977740 1 --2- 172.21.15.83:0/2809061064 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564d39c66b40 0x564d39c6e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:24.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.276+0000 7f1ca5977740 1 -- 172.21.15.83:0/2809061064 >> 172.21.15.83:0/2809061064 conn(0x564d39c62e60 msgr2=0x564d39c68b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:24.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.276+0000 7f1ca5977740 1 -- 172.21.15.83:0/2809061064 shutdown_connections 2024-04-24T09:17:24.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.276+0000 7f1ca5977740 1 -- 172.21.15.83:0/2809061064 wait complete. 2024-04-24T09:17:24.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.276+0000 7f1ca5977740 20 client.41258 trim_cache size 0 max 16384 2024-04-24T09:17:24.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.300+0000 7f1ca5977740 1 Processor -- start 2024-04-24T09:17:24.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.300+0000 7f1ca5977740 1 -- start start 2024-04-24T09:17:24.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.300+0000 7f1ca5977740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564d39c66b40 0x564d39c91ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:24.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.300+0000 7f1ca5977740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564d39c67460 0x564d39c92530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:24.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.300+0000 7f1ca5977740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564d39c71b30 0x564d39c616c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:24.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.300+0000 7f1ca5977740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1c9c1ad380 con 0x564d39c67460 2024-04-24T09:17:24.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.300+0000 7f1ca5977740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1c9c1aeef0 con 0x564d39c66b40 2024-04-24T09:17:24.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.300+0000 7f1ca5977740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1c9c1ada50 con 0x564d39c71b30 2024-04-24T09:17:24.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.300+0000 7f1c93400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564d39c67460 0x564d39c92530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:24.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.300+0000 7f1c9be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564d39c66b40 0x564d39c91ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:24.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.300+0000 7f1c93400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564d39c67460 0x564d39c92530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35258/0 (socket says 172.21.15.83:35258) 2024-04-24T09:17:24.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.300+0000 7f1c9be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564d39c66b40 0x564d39c91ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:39218/0 (socket says 172.21.15.83:39218) 2024-04-24T09:17:24.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.300+0000 7f1c93400640 1 -- 172.21.15.83:0/2281544367 learned_addr learned my addr 172.21.15.83:0/2281544367 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:17:24.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.301+0000 7f1ca0a00640 1 --2- 172.21.15.83:0/2281544367 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564d39c71b30 0x564d39c616c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:24.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.301+0000 7f1c93400640 1 -- 172.21.15.83:0/2281544367 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564d39c71b30 msgr2=0x564d39c616c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:24.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.301+0000 7f1c93400640 1 --2- 172.21.15.83:0/2281544367 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564d39c71b30 0x564d39c616c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:24.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.301+0000 7f1c93400640 1 -- 172.21.15.83:0/2281544367 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564d39c66b40 msgr2=0x564d39c91ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:24.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.301+0000 7f1c93400640 1 --2- 172.21.15.83:0/2281544367 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564d39c66b40 0x564d39c91ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:24.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.301+0000 7f1c93400640 1 -- 172.21.15.83:0/2281544367 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d399a87d0 con 0x564d39c67460 2024-04-24T09:17:24.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.301+0000 7f1c9be00640 1 --2- 172.21.15.83:0/2281544367 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564d39c66b40 0x564d39c91ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:17:24.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.302+0000 7f1c93400640 1 --2- 172.21.15.83:0/2281544367 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564d39c67460 0x564d39c92530 secure :-1 s=READY pgs=2831 cs=0 l=1 rev1=1 crypto rx=0x7f1c84010420 tx=0x7f1c840035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=409de1fb78480c6e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:24.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.302+0000 7f1c8be00640 1 -- 172.21.15.83:0/2281544367 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c8402e070 con 0x564d39c67460 2024-04-24T09:17:24.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.302+0000 7f1c8be00640 1 -- 172.21.15.83:0/2281544367 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1c84005390 con 0x564d39c67460 2024-04-24T09:17:24.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:17:24 smithi083 ceph-mon[27907]: pgmap v816: 129 pgs: 129 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 6.1 MiB/s wr, 320 op/s 2024-04-24T09:17:24.304 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:17:24 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/236221312' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:17:24.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.302+0000 7f1c8be00640 1 -- 172.21.15.83:0/2281544367 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c840105e0 con 0x564d39c67460 2024-04-24T09:17:24.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.303+0000 7f1ca5977740 1 -- 172.21.15.83:0/2281544367 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564d39c67460 msgr2=0x564d39c92530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:24.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.303+0000 7f1ca5977740 1 --2- 172.21.15.83:0/2281544367 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564d39c67460 0x564d39c92530 secure :-1 s=READY pgs=2831 cs=0 l=1 rev1=1 crypto rx=0x7f1c84010420 tx=0x7f1c840035f0 comp rx=0 tx=0).stop 2024-04-24T09:17:24.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.304+0000 7f1ca5977740 1 -- 172.21.15.83:0/2281544367 shutdown_connections 2024-04-24T09:17:24.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.304+0000 7f1ca5977740 1 --2- 172.21.15.83:0/2281544367 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564d39c71b30 0x564d39c616c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:24.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.304+0000 7f1ca5977740 1 --2- 172.21.15.83:0/2281544367 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564d39c67460 0x564d39c92530 unknown :-1 s=CLOSED pgs=2831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:24.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.304+0000 7f1ca5977740 1 --2- 172.21.15.83:0/2281544367 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564d39c66b40 0x564d39c91ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:24.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.304+0000 7f1ca5977740 1 -- 172.21.15.83:0/2281544367 >> 172.21.15.83:0/2281544367 conn(0x564d39c62e60 msgr2=0x564d39c6d940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:24.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.304+0000 7f1ca5977740 1 -- 172.21.15.83:0/2281544367 shutdown_connections 2024-04-24T09:17:24.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.304+0000 7f1ca5977740 1 -- 172.21.15.83:0/2281544367 wait complete. 2024-04-24T09:17:24.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.305+0000 7f1ca5977740 1 -- start start 2024-04-24T09:17:24.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.305+0000 7f1ca5977740 1 Processor -- start 2024-04-24T09:17:24.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.306+0000 7f1ca5977740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564d39c66b40 0x564d39c72680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:24.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.306+0000 7f1ca5977740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564d39c67460 0x564d39c72bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:24.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.306+0000 7f1ca5977740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564d39c71b30 0x564d39c81650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:24.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.306+0000 7f1ca5977740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1c9c1ae820 con 0x564d39c66b40 2024-04-24T09:17:24.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.306+0000 7f1ca5977740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1c9c1ae150 con 0x564d39c67460 2024-04-24T09:17:24.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.306+0000 7f1ca5977740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1c9c1ab380 con 0x564d39c71b30 2024-04-24T09:17:24.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.306+0000 7f1ca5977740 10 client.0 _ll_register_callbacks cb 0x7f1c84005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:17:24.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.306+0000 7f1c9be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564d39c66b40 0x564d39c72680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:24.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.306+0000 7f1c9be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564d39c66b40 0x564d39c72680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35270/0 (socket says 172.21.15.83:35270) 2024-04-24T09:17:24.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.306+0000 7f1c9be00640 1 -- 172.21.15.83:0/3954772884 learned_addr learned my addr 172.21.15.83:0/3954772884 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:17:24.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.306+0000 7f1c93400640 1 --2- 172.21.15.83:0/3954772884 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564d39c67460 0x564d39c72bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:24.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.307+0000 7f1ca0a00640 1 --2- 172.21.15.83:0/3954772884 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564d39c71b30 0x564d39c81650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:24.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.307+0000 7f1c9be00640 1 -- 172.21.15.83:0/3954772884 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564d39c71b30 msgr2=0x564d39c81650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:24.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.307+0000 7f1c9be00640 1 --2- 172.21.15.83:0/3954772884 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564d39c71b30 0x564d39c81650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:24.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.307+0000 7f1c9be00640 1 -- 172.21.15.83:0/3954772884 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564d39c67460 msgr2=0x564d39c72bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:24.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.307+0000 7f1c9be00640 1 --2- 172.21.15.83:0/3954772884 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564d39c67460 0x564d39c72bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:24.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.307+0000 7f1c9be00640 1 -- 172.21.15.83:0/3954772884 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d39c81c10 con 0x564d39c66b40 2024-04-24T09:17:24.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.307+0000 7f1ca0a00640 1 --2- 172.21.15.83:0/3954772884 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564d39c71b30 0x564d39c81650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:17:24.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.307+0000 7f1c9be00640 1 --2- 172.21.15.83:0/3954772884 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564d39c66b40 0x564d39c72680 secure :-1 s=READY pgs=2832 cs=0 l=1 rev1=1 crypto rx=0x7f1c8c009900 tx=0x7f1c8c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4d20d4fc18a8fcb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:24.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.308+0000 7f1c89600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:17:24.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.308+0000 7f1c89600640 1 -- 172.21.15.83:0/3954772884 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c8c035040 con 0x564d39c66b40 2024-04-24T09:17:24.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.308+0000 7f1c89600640 1 -- 172.21.15.83:0/3954772884 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(134..134 src has 1..134) ==== 9583+0+0 (secure 0 0 0) 0x7f1c8c003b40 con 0x564d39c66b40 2024-04-24T09:17:24.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.308+0000 7f1ca5977740 1 -- 172.21.15.83:0/3954772884 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c84005390 con 0x564d39c66b40 2024-04-24T09:17:24.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.309+0000 7f1c89600640 1 -- 172.21.15.83:0/3954772884 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7f1c8c003d40 con 0x564d39c66b40 2024-04-24T09:17:24.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.309+0000 7f1ca5977740 10 client.24207 fetch_fsmap learned FSMap version 70 2024-04-24T09:17:24.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.309+0000 7f1ca5977740 1 -- 172.21.15.83:0/3954772884 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) -- 0x564d39c82ef0 con 0x564d39c66b40 2024-04-24T09:17:24.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.309+0000 7f1c89600640 1 -- 172.21.15.83:0/3954772884 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1c84005390 con 0x564d39c66b40 2024-04-24T09:17:24.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.309+0000 7f1c89600640 1 -- 172.21.15.83:0/3954772884 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 70) ==== 8366+0+0 (secure 0 0 0) 0x7f1c8c0316a0 con 0x564d39c66b40 2024-04-24T09:17:24.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.309+0000 7f1ca5977740 10 client.24207 fetch_fsmap finished waiting for FSMap version 70 2024-04-24T09:17:24.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.309+0000 7f1ca5977740 10 client.24207 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-24T09:17:24.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.309+0000 7f1ca5977740 20 client.24207 populate_metadata read hostname 'smithi083' 2024-04-24T09:17:24.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.309+0000 7f1ca5977740 1 --2- 172.21.15.83:0/3954772884 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x564d39cf0c30 0x564d39cf3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:24.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.309+0000 7f1ca5977740 4 client.24207 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:17:24.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.309+0000 7f1ca5977740 1 -- 172.21.15.83:0/3954772884 --> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564d39bd3a60 con 0x564d39cf0c30 2024-04-24T09:17:24.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.310+0000 7f1c93400640 1 --2- 172.21.15.83:0/3954772884 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x564d39cf0c30 0x564d39cf3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:24.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.311+0000 7f1c93400640 1 --2- 172.21.15.83:0/3954772884 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x564d39cf0c30 0x564d39cf3050 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=33ddfeee97a074f1 server_cookie=bae590b37f6a7b49 in_seq=0 out_seq=0 2024-04-24T09:17:24.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.311+0000 7f1c89600640 10 client.24207 ms_handle_connect on v2:172.21.15.189:6838/2548965336 2024-04-24T09:17:24.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.314+0000 7f1c89600640 1 -- 172.21.15.83:0/3954772884 <== mds.4 v2:172.21.15.189:6838/2548965336 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564d39bd3a60 con 0x564d39cf0c30 2024-04-24T09:17:24.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.314+0000 7f1c89600640 10 client.24207 handle_command_reply: tid=0 2024-04-24T09:17:24.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.314+0000 7f1ca5977740 1 client.24207 shutdown 2024-04-24T09:17:24.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.314+0000 7f1ca5977740 1 -- 172.21.15.83:0/3954772884 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564d39c66b40 msgr2=0x564d39c72680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:24.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.314+0000 7f1ca5977740 1 --2- 172.21.15.83:0/3954772884 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564d39c66b40 0x564d39c72680 secure :-1 s=READY pgs=2832 cs=0 l=1 rev1=1 crypto rx=0x7f1c8c009900 tx=0x7f1c8c009260 comp rx=0 tx=0).stop 2024-04-24T09:17:24.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.314+0000 7f1ca5977740 1 -- 172.21.15.83:0/3954772884 shutdown_connections 2024-04-24T09:17:24.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.314+0000 7f1ca5977740 1 --2- 172.21.15.83:0/3954772884 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x564d39cf0c30 0x564d39cf3050 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:24.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.314+0000 7f1ca5977740 1 --2- 172.21.15.83:0/3954772884 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564d39c71b30 0x564d39c81650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:24.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.314+0000 7f1c89600640 0 client.24207 ms_handle_reset on v2:172.21.15.189:6838/2548965336 2024-04-24T09:17:24.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.315+0000 7f1ca5977740 1 --2- 172.21.15.83:0/3954772884 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564d39c67460 0x564d39c72bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:24.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.315+0000 7f1ca5977740 1 --2- 172.21.15.83:0/3954772884 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564d39c66b40 0x564d39c72680 unknown :-1 s=CLOSED pgs=2832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:24.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.315+0000 7f1ca5977740 1 -- 172.21.15.83:0/3954772884 >> 172.21.15.83:0/3954772884 conn(0x564d39c62e60 msgr2=0x564d39c616c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:24.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.315+0000 7f1ca5977740 1 -- 172.21.15.83:0/3954772884 shutdown_connections 2024-04-24T09:17:24.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.315+0000 7f1ca5977740 1 -- 172.21.15.83:0/3954772884 wait complete. 2024-04-24T09:17:24.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.315+0000 7f1ca5977740 20 client.24207 trim_cache size 0 max 16384 2024-04-24T09:17:24.317 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:17:24.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.316+0000 7f1ca1400640 1 -- 172.21.15.83:0/3423320783 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f1c800a3da0 msgr2=0x7f1c800a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:24.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.316+0000 7f1ca1400640 1 --2- 172.21.15.83:0/3423320783 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f1c800a3da0 0x7f1c800a6260 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f1c8c0057c0 tx=0x7f1c8c031400 comp rx=0 tx=0).stop 2024-04-24T09:17:24.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.316+0000 7f1ca1400640 1 -- 172.21.15.83:0/3423320783 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c9c10e810 msgr2=0x7f1c9c19a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:24.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.316+0000 7f1ca1400640 1 --2- 172.21.15.83:0/3423320783 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c9c10e810 0x7f1c9c19a710 secure :-1 s=READY pgs=2830 cs=0 l=1 rev1=1 crypto rx=0x7f1c94009fd0 tx=0x7f1c9400ee60 comp rx=0 tx=0).stop 2024-04-24T09:17:24.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.316+0000 7f1ca1400640 1 -- 172.21.15.83:0/3423320783 shutdown_connections 2024-04-24T09:17:24.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.316+0000 7f1ca1400640 1 --2- 172.21.15.83:0/3423320783 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f1c800a3da0 0x7f1c800a6260 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:24.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.316+0000 7f1ca1400640 1 --2- 172.21.15.83:0/3423320783 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c9c10e810 0x7f1c9c19a710 unknown :-1 s=CLOSED pgs=2830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:24.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.316+0000 7f1ca1400640 1 --2- 172.21.15.83:0/3423320783 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1c9c101610 0x7f1c9c1a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:24.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.316+0000 7f1ca1400640 1 --2- 172.21.15.83:0/3423320783 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c9c100c60 0x7f1c9c1a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:24.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.316+0000 7f1ca1400640 1 -- 172.21.15.83:0/3423320783 >> 172.21.15.83:0/3423320783 conn(0x7f1c9c0fc880 msgr2=0x7f1c9c0fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:24.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.316+0000 7f1ca1400640 1 -- 172.21.15.83:0/3423320783 shutdown_connections 2024-04-24T09:17:24.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.316+0000 7f1ca1400640 1 -- 172.21.15.83:0/3423320783 wait complete. 2024-04-24T09:17:24.339 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-24T09:17:24.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:17:24 smithi138 ceph-mon[31441]: pgmap v816: 129 pgs: 129 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 6.1 MiB/s wr, 320 op/s 2024-04-24T09:17:24.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:17:24 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/236221312' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:17:24.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:17:24 smithi189 ceph-mon[31791]: pgmap v816: 129 pgs: 129 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 6.1 MiB/s wr, 320 op/s 2024-04-24T09:17:24.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:17:24 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/236221312' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:17:24.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.670+0000 7f144be00640 1 -- 172.21.15.83:0/4115323002 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f144c10e760 msgr2=0x7f144c110b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:24.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.670+0000 7f144be00640 1 --2- 172.21.15.83:0/4115323002 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f144c10e760 0x7f144c110b50 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f1444009ca0 tx=0x7f144402a950 comp rx=0 tx=0).stop 2024-04-24T09:17:24.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.670+0000 7f144be00640 1 -- 172.21.15.83:0/4115323002 shutdown_connections 2024-04-24T09:17:24.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.670+0000 7f144be00640 1 --2- 172.21.15.83:0/4115323002 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f144c10e760 0x7f144c110b50 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:24.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.670+0000 7f144be00640 1 --2- 172.21.15.83:0/4115323002 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f144c074cd0 0x7f144c10e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:24.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.670+0000 7f144be00640 1 --2- 172.21.15.83:0/4115323002 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f144c0743b0 0x7f144c074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:24.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.670+0000 7f144be00640 1 -- 172.21.15.83:0/4115323002 >> 172.21.15.83:0/4115323002 conn(0x7f144c0fd9d0 msgr2=0x7f144c0ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:24.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.671+0000 7f144be00640 1 -- 172.21.15.83:0/4115323002 shutdown_connections 2024-04-24T09:17:24.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.671+0000 7f144be00640 1 -- 172.21.15.83:0/4115323002 wait complete. 2024-04-24T09:17:24.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.672+0000 7f144be00640 1 Processor -- start 2024-04-24T09:17:24.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.672+0000 7f144be00640 1 -- start start 2024-04-24T09:17:24.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.673+0000 7f144be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f144c0743b0 0x7f144c1a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:24.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.673+0000 7f144be00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f144c074cd0 0x7f144c1a0b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:24.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.673+0000 7f144be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f144c10e760 0x7f144c19a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:24.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.673+0000 7f144be00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f144c114310 con 0x7f144c0743b0 2024-04-24T09:17:24.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.673+0000 7f144be00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f144c114190 con 0x7f144c10e760 2024-04-24T09:17:24.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.673+0000 7f144be00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f144c114490 con 0x7f144c074cd0 2024-04-24T09:17:24.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.673+0000 7f144aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f144c0743b0 0x7f144c1a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:24.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.673+0000 7f144aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f144c0743b0 0x7f144c1a05f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35278/0 (socket says 172.21.15.83:35278) 2024-04-24T09:17:24.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.673+0000 7f144aa00640 1 -- 172.21.15.83:0/1183656116 learned_addr learned my addr 172.21.15.83:0/1183656116 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:17:24.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.673+0000 7f144b400640 1 --2- 172.21.15.83:0/1183656116 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f144c10e760 0x7f144c19a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:24.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.673+0000 7f144a000640 1 --2- 172.21.15.83:0/1183656116 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f144c074cd0 0x7f144c1a0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:24.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.674+0000 7f144b400640 1 -- 172.21.15.83:0/1183656116 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f144c074cd0 msgr2=0x7f144c1a0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:24.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.674+0000 7f144b400640 1 --2- 172.21.15.83:0/1183656116 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f144c074cd0 0x7f144c1a0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:24.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.674+0000 7f144b400640 1 -- 172.21.15.83:0/1183656116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f144c0743b0 msgr2=0x7f144c1a05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:24.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.674+0000 7f144b400640 1 --2- 172.21.15.83:0/1183656116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f144c0743b0 0x7f144c1a05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:24.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.674+0000 7f144b400640 1 -- 172.21.15.83:0/1183656116 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f144c19afc0 con 0x7f144c10e760 2024-04-24T09:17:24.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.674+0000 7f144b400640 1 --2- 172.21.15.83:0/1183656116 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f144c10e760 0x7f144c19a700 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f1444005cc0 tx=0x7f144402f3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:24.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.674+0000 7f1443400640 1 -- 172.21.15.83:0/1183656116 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1444034040 con 0x7f144c10e760 2024-04-24T09:17:24.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.675+0000 7f1443400640 1 -- 172.21.15.83:0/1183656116 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f14440308f0 con 0x7f144c10e760 2024-04-24T09:17:24.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.675+0000 7f1443400640 1 -- 172.21.15.83:0/1183656116 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14440045e0 con 0x7f144c10e760 2024-04-24T09:17:24.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.675+0000 7f144be00640 1 -- 172.21.15.83:0/1183656116 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f144c19b250 con 0x7f144c10e760 2024-04-24T09:17:24.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.675+0000 7f144be00640 1 -- 172.21.15.83:0/1183656116 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f144c101db0 con 0x7f144c10e760 2024-04-24T09:17:24.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.679+0000 7f1454840740 1 Processor -- start 2024-04-24T09:17:24.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.679+0000 7f1443400640 1 -- 172.21.15.83:0/1183656116 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1444037020 con 0x7f144c10e760 2024-04-24T09:17:24.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.680+0000 7f1454840740 1 -- start start 2024-04-24T09:17:24.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.680+0000 7f1454840740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562394358b40 0x562394358f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:24.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.680+0000 7f1454840740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562394359460 0x5623943635f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:24.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.680+0000 7f1454840740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562394363b30 0x562394365f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:24.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.680+0000 7f1454840740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5623942be1d0 con 0x562394363b30 2024-04-24T09:17:24.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.680+0000 7f1454840740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562394264450 con 0x562394358b40 2024-04-24T09:17:24.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.680+0000 7f1454840740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5623942ba190 con 0x562394359460 2024-04-24T09:17:24.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.681+0000 7f144aa00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562394359460 0x5623943635f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:24.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.681+0000 7f144aa00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562394359460 0x5623943635f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:59034/0 (socket says 172.21.15.83:59034) 2024-04-24T09:17:24.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.681+0000 7f144aa00640 1 -- 172.21.15.83:0/2162160297 learned_addr learned my addr 172.21.15.83:0/2162160297 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:17:24.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.681+0000 7f144a000640 1 --2- 172.21.15.83:0/2162160297 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562394358b40 0x562394358f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:24.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.681+0000 7f144a000640 1 --2- 172.21.15.83:0/2162160297 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562394363b30 0x562394365f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:24.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.682+0000 7f144aa00640 1 -- 172.21.15.83:0/2162160297 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562394358b40 msgr2=0x562394358f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:24.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.682+0000 7f144aa00640 1 --2- 172.21.15.83:0/2162160297 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562394358b40 0x562394358f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:24.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.682+0000 7f144aa00640 1 -- 172.21.15.83:0/2162160297 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562394363b30 msgr2=0x562394365f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:24.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.682+0000 7f144aa00640 1 --2- 172.21.15.83:0/2162160297 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562394363b30 0x562394365f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:24.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.682+0000 7f144aa00640 1 -- 172.21.15.83:0/2162160297 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562394321310 con 0x562394359460 2024-04-24T09:17:24.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.682+0000 7f1443400640 1 --2- 172.21.15.83:0/1183656116 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f14300a3d20 0x7f14300a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:24.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.682+0000 7f1443400640 1 -- 172.21.15.83:0/1183656116 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 9583+0+0 (secure 0 0 0) 0x7f14440f1980 con 0x7f144c10e760 2024-04-24T09:17:24.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.683+0000 7f144a000640 1 --2- 172.21.15.83:0/1183656116 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f14300a3d20 0x7f14300a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:24.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.683+0000 7f144aa00640 1 --2- 172.21.15.83:0/2162160297 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562394359460 0x5623943635f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f143c00b9e0 tx=0x7f143c00b410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f44bcabae6aa17c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:24.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.683+0000 7f1437e00640 1 -- 172.21.15.83:0/2162160297 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f143c014ce0 con 0x562394359460 2024-04-24T09:17:24.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.683+0000 7f144a000640 1 --2- 172.21.15.83:0/1183656116 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f14300a3d20 0x7f14300a61e0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x562394355ff0 tx=0x7f1438005f90 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:24.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.684+0000 7f1437e00640 1 -- 172.21.15.83:0/2162160297 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f143c009d30 con 0x562394359460 2024-04-24T09:17:24.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.684+0000 7f1437e00640 1 -- 172.21.15.83:0/2162160297 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f143c005650 con 0x562394359460 2024-04-24T09:17:24.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.684+0000 7f1454840740 1 -- 172.21.15.83:0/2162160297 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562394359460 msgr2=0x5623943635f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:24.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.684+0000 7f1454840740 1 --2- 172.21.15.83:0/2162160297 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562394359460 0x5623943635f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f143c00b9e0 tx=0x7f143c00b410 comp rx=0 tx=0).stop 2024-04-24T09:17:24.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.685+0000 7f1454840740 1 -- 172.21.15.83:0/2162160297 shutdown_connections 2024-04-24T09:17:24.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.685+0000 7f1454840740 1 --2- 172.21.15.83:0/2162160297 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562394363b30 0x562394365f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:24.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.685+0000 7f1454840740 1 --2- 172.21.15.83:0/2162160297 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562394359460 0x5623943635f0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:24.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.685+0000 7f1454840740 1 --2- 172.21.15.83:0/2162160297 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562394358b40 0x562394358f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:24.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.685+0000 7f1454840740 1 -- 172.21.15.83:0/2162160297 >> 172.21.15.83:0/2162160297 conn(0x562394354e60 msgr2=0x562394355240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:24.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.685+0000 7f1454840740 1 -- 172.21.15.83:0/2162160297 shutdown_connections 2024-04-24T09:17:24.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.685+0000 7f1454840740 1 -- 172.21.15.83:0/2162160297 wait complete. 2024-04-24T09:17:24.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.686+0000 7f1454840740 1 -- start start 2024-04-24T09:17:24.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.686+0000 7f1454840740 1 Processor -- start 2024-04-24T09:17:24.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.687+0000 7f1454840740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562394358b40 0x562394363560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:24.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.687+0000 7f1454840740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562394359460 0x562394360850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:24.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.687+0000 7f1454840740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562394363b30 0x562394360f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:24.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.687+0000 7f1454840740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5623943669c0 con 0x562394358b40 2024-04-24T09:17:24.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.687+0000 7f1454840740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562394277810 con 0x562394359460 2024-04-24T09:17:24.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.687+0000 7f1454840740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x562394366b40 con 0x562394363b30 2024-04-24T09:17:24.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.687+0000 7f1454840740 10 client.0 _ll_register_callbacks cb 0x562394245140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:17:24.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.687+0000 7f144aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562394358b40 0x562394363560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:24.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.687+0000 7f144aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562394358b40 0x562394363560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35298/0 (socket says 172.21.15.83:35298) 2024-04-24T09:17:24.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.687+0000 7f144aa00640 1 -- 172.21.15.83:0/587378069 learned_addr learned my addr 172.21.15.83:0/587378069 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:17:24.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.687+0000 7f144a000640 1 --2- 172.21.15.83:0/587378069 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562394359460 0x562394360850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:24.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.688+0000 7f144b400640 1 --2- 172.21.15.83:0/587378069 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562394363b30 0x562394360f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:24.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.688+0000 7f144aa00640 1 -- 172.21.15.83:0/587378069 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562394363b30 msgr2=0x562394360f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:24.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.688+0000 7f144aa00640 1 --2- 172.21.15.83:0/587378069 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562394363b30 0x562394360f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:24.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.688+0000 7f144aa00640 1 -- 172.21.15.83:0/587378069 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562394359460 msgr2=0x562394360850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:24.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.688+0000 7f144aa00640 1 --2- 172.21.15.83:0/587378069 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562394359460 0x562394360850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:24.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.688+0000 7f144aa00640 1 -- 172.21.15.83:0/587378069 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623942883f0 con 0x562394358b40 2024-04-24T09:17:24.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.688+0000 7f144aa00640 1 --2- 172.21.15.83:0/587378069 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562394358b40 0x562394363560 secure :-1 s=READY pgs=2833 cs=0 l=1 rev1=1 crypto rx=0x7f143c017010 tx=0x7f143c014d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9defcd3b5d911aa server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:24.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.688+0000 7f1434c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:17:24.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.689+0000 7f1434c00640 1 -- 172.21.15.83:0/587378069 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f143c02e020 con 0x562394358b40 2024-04-24T09:17:24.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.689+0000 7f1434c00640 1 -- 172.21.15.83:0/587378069 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(134..134 src has 1..134) ==== 9583+0+0 (secure 0 0 0) 0x7f143c015200 con 0x562394358b40 2024-04-24T09:17:24.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.689+0000 7f1454840740 1 -- 172.21.15.83:0/587378069 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f143c009d30 con 0x562394358b40 2024-04-24T09:17:24.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.689+0000 7f1434c00640 1 -- 172.21.15.83:0/587378069 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7f143c025040 con 0x562394358b40 2024-04-24T09:17:24.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.689+0000 7f1454840740 10 client.24219 fetch_fsmap learned FSMap version 70 2024-04-24T09:17:24.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.689+0000 7f1454840740 1 -- 172.21.15.83:0/587378069 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) -- 0x562393fd03f0 con 0x562394358b40 2024-04-24T09:17:24.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.690+0000 7f1434c00640 1 -- 172.21.15.83:0/587378069 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f143c009d30 con 0x562394358b40 2024-04-24T09:17:24.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.690+0000 7f1434c00640 1 -- 172.21.15.83:0/587378069 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 70) ==== 8366+0+0 (secure 0 0 0) 0x7f143c01f070 con 0x562394358b40 2024-04-24T09:17:24.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.690+0000 7f1454840740 10 client.24219 fetch_fsmap finished waiting for FSMap version 70 2024-04-24T09:17:24.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.690+0000 7f1454840740 10 client.24219 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-24T09:17:24.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.690+0000 7f1454840740 20 client.24219 populate_metadata read hostname 'smithi083' 2024-04-24T09:17:24.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.690+0000 7f1454840740 1 --2- 172.21.15.83:0/587378069 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x562394373bc0 0x562394375fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:24.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.690+0000 7f1454840740 4 client.24219 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:17:24.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.690+0000 7f1454840740 1 -- 172.21.15.83:0/587378069 --> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5623942cbbf0 con 0x562394373bc0 2024-04-24T09:17:24.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.690+0000 7f144a000640 1 --2- 172.21.15.83:0/587378069 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x562394373bc0 0x562394375fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:24.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.691+0000 7f1434c00640 10 client.24219 ms_handle_connect on v2:172.21.15.138:6832/1157967658 2024-04-24T09:17:24.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.691+0000 7f144a000640 1 --2- 172.21.15.83:0/587378069 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x562394373bc0 0x562394375fb0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3a6406d010672751 server_cookie=7920113b6c183230 in_seq=0 out_seq=0 2024-04-24T09:17:24.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.693+0000 7f1434c00640 1 -- 172.21.15.83:0/587378069 <== mds.3 v2:172.21.15.138:6832/1157967658 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5623942cbbf0 con 0x562394373bc0 2024-04-24T09:17:24.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.693+0000 7f1434c00640 10 client.24219 handle_command_reply: tid=0 2024-04-24T09:17:24.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.693+0000 7f1454840740 1 client.24219 shutdown 2024-04-24T09:17:24.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.693+0000 7f1454840740 1 -- 172.21.15.83:0/587378069 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562394358b40 msgr2=0x562394363560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:24.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.693+0000 7f1454840740 1 --2- 172.21.15.83:0/587378069 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562394358b40 0x562394363560 secure :-1 s=READY pgs=2833 cs=0 l=1 rev1=1 crypto rx=0x7f143c017010 tx=0x7f143c014d10 comp rx=0 tx=0).stop 2024-04-24T09:17:24.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.693+0000 7f1454840740 1 -- 172.21.15.83:0/587378069 shutdown_connections 2024-04-24T09:17:24.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.693+0000 7f1454840740 1 --2- 172.21.15.83:0/587378069 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x562394373bc0 0x562394375fb0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:24.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.693+0000 7f1454840740 1 --2- 172.21.15.83:0/587378069 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562394363b30 0x562394360f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:24.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.693+0000 7f1454840740 1 --2- 172.21.15.83:0/587378069 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562394359460 0x562394360850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:24.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.693+0000 7f1454840740 1 --2- 172.21.15.83:0/587378069 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562394358b40 0x562394363560 unknown :-1 s=CLOSED pgs=2833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:24.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.693+0000 7f1434c00640 0 client.24219 ms_handle_reset on v2:172.21.15.138:6832/1157967658 2024-04-24T09:17:24.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.693+0000 7f1454840740 1 -- 172.21.15.83:0/587378069 >> 172.21.15.83:0/587378069 conn(0x562394354e60 msgr2=0x56239435ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:24.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.693+0000 7f1454840740 1 -- 172.21.15.83:0/587378069 shutdown_connections 2024-04-24T09:17:24.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.694+0000 7f1454840740 1 -- 172.21.15.83:0/587378069 wait complete. 2024-04-24T09:17:24.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.694+0000 7f1454840740 20 client.24219 trim_cache size 0 max 16384 2024-04-24T09:17:24.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.742+0000 7f1454840740 1 Processor -- start 2024-04-24T09:17:24.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.742+0000 7f1454840740 1 -- start start 2024-04-24T09:17:24.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.743+0000 7f1454840740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562394358b40 0x56239436f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:24.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.743+0000 7f1454840740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562394359460 0x562394363090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:24.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.743+0000 7f1454840740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562394363b30 0x5623943635d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:24.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.743+0000 7f1454840740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f144c1ad210 con 0x562394359460 2024-04-24T09:17:24.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.743+0000 7f1454840740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f144c1aed80 con 0x562394363b30 2024-04-24T09:17:24.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.743+0000 7f1454840740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f144c1ad8e0 con 0x562394358b40 2024-04-24T09:17:24.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.743+0000 7f144b400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562394363b30 0x5623943635d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:24.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.743+0000 7f144b400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562394363b30 0x5623943635d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:39284/0 (socket says 172.21.15.83:39284) 2024-04-24T09:17:24.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.743+0000 7f144b400640 1 -- 172.21.15.83:0/870824821 learned_addr learned my addr 172.21.15.83:0/870824821 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:17:24.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.743+0000 7f144a000640 1 --2- 172.21.15.83:0/870824821 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562394359460 0x562394363090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:24.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.744+0000 7f144aa00640 1 --2- 172.21.15.83:0/870824821 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562394358b40 0x56239436f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:24.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.744+0000 7f144a000640 1 -- 172.21.15.83:0/870824821 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562394358b40 msgr2=0x56239436f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:24.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.744+0000 7f144a000640 1 --2- 172.21.15.83:0/870824821 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562394358b40 0x56239436f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:24.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.744+0000 7f144a000640 1 -- 172.21.15.83:0/870824821 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562394363b30 msgr2=0x5623943635d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:24.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.744+0000 7f144a000640 1 --2- 172.21.15.83:0/870824821 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562394363b30 0x5623943635d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:24.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.744+0000 7f144a000640 1 -- 172.21.15.83:0/870824821 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562394359290 con 0x562394359460 2024-04-24T09:17:24.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.744+0000 7f144a000640 1 --2- 172.21.15.83:0/870824821 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562394359460 0x562394363090 secure :-1 s=READY pgs=2834 cs=0 l=1 rev1=1 crypto rx=0x562394361e20 tx=0x7f14380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5da24f409f7a926f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:24.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.745+0000 7f1437400640 1 -- 172.21.15.83:0/870824821 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1438034070 con 0x562394359460 2024-04-24T09:17:24.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.745+0000 7f1437400640 1 -- 172.21.15.83:0/870824821 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f143802e680 con 0x562394359460 2024-04-24T09:17:24.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.745+0000 7f1437400640 1 -- 172.21.15.83:0/870824821 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f143801fb70 con 0x562394359460 2024-04-24T09:17:24.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.746+0000 7f1454840740 1 -- 172.21.15.83:0/870824821 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562394359460 msgr2=0x562394363090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:24.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.746+0000 7f1454840740 1 --2- 172.21.15.83:0/870824821 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562394359460 0x562394363090 secure :-1 s=READY pgs=2834 cs=0 l=1 rev1=1 crypto rx=0x562394361e20 tx=0x7f14380046c0 comp rx=0 tx=0).stop 2024-04-24T09:17:24.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.747+0000 7f1454840740 1 -- 172.21.15.83:0/870824821 shutdown_connections 2024-04-24T09:17:24.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.747+0000 7f1454840740 1 --2- 172.21.15.83:0/870824821 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562394363b30 0x5623943635d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:24.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.747+0000 7f1454840740 1 --2- 172.21.15.83:0/870824821 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562394359460 0x562394363090 unknown :-1 s=CLOSED pgs=2834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:24.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.747+0000 7f1454840740 1 --2- 172.21.15.83:0/870824821 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562394358b40 0x56239436f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:24.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.747+0000 7f1454840740 1 -- 172.21.15.83:0/870824821 >> 172.21.15.83:0/870824821 conn(0x562394354e60 msgr2=0x562394355240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:24.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.747+0000 7f1454840740 1 -- 172.21.15.83:0/870824821 shutdown_connections 2024-04-24T09:17:24.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.747+0000 7f1454840740 1 -- 172.21.15.83:0/870824821 wait complete. 2024-04-24T09:17:24.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.748+0000 7f1454840740 1 -- start start 2024-04-24T09:17:24.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.748+0000 7f1454840740 1 Processor -- start 2024-04-24T09:17:24.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.749+0000 7f1454840740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562394358b40 0x5623943646e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:24.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.749+0000 7f1454840740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562394359460 0x562394364c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:24.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.749+0000 7f1454840740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562394363b30 0x56239436ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:24.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.749+0000 7f1454840740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f144c1ae6b0 con 0x562394359460 2024-04-24T09:17:24.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.749+0000 7f1454840740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f144c1adfe0 con 0x562394358b40 2024-04-24T09:17:24.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.749+0000 7f1454840740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f144c1ab210 con 0x562394363b30 2024-04-24T09:17:24.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.749+0000 7f1454840740 10 client.0 _ll_register_callbacks cb 0x7f1438009cd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:17:24.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.749+0000 7f144aa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562394358b40 0x5623943646e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:24.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.749+0000 7f144aa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562394358b40 0x5623943646e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:39286/0 (socket says 172.21.15.83:39286) 2024-04-24T09:17:24.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.749+0000 7f144aa00640 1 -- 172.21.15.83:0/4109221137 learned_addr learned my addr 172.21.15.83:0/4109221137 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:17:24.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.749+0000 7f144a000640 1 --2- 172.21.15.83:0/4109221137 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562394359460 0x562394364c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:24.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.749+0000 7f144b400640 1 --2- 172.21.15.83:0/4109221137 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562394363b30 0x56239436ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:24.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.750+0000 7f144aa00640 1 -- 172.21.15.83:0/4109221137 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562394363b30 msgr2=0x56239436ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:24.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.750+0000 7f144aa00640 1 --2- 172.21.15.83:0/4109221137 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562394363b30 0x56239436ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:24.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.750+0000 7f144aa00640 1 -- 172.21.15.83:0/4109221137 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562394359460 msgr2=0x562394364c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:24.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.750+0000 7f144aa00640 1 --2- 172.21.15.83:0/4109221137 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562394359460 0x562394364c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:24.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.750+0000 7f144aa00640 1 -- 172.21.15.83:0/4109221137 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56239409a7d0 con 0x562394358b40 2024-04-24T09:17:24.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.750+0000 7f144b400640 1 --2- 172.21.15.83:0/4109221137 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562394363b30 0x56239436ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:17:24.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.750+0000 7f144aa00640 1 --2- 172.21.15.83:0/4109221137 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562394358b40 0x5623943646e0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f143c009fd0 tx=0x7f143c00b830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5503cd6085331d32 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:24.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.750+0000 7f1434c00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:17:24.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.751+0000 7f1434c00640 1 -- 172.21.15.83:0/4109221137 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f143c013060 con 0x562394358b40 2024-04-24T09:17:24.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.751+0000 7f1434c00640 1 -- 172.21.15.83:0/4109221137 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(134..134 src has 1..134) ==== 9583+0+0 (secure 0 0 0) 0x7f143c02aa30 con 0x562394358b40 2024-04-24T09:17:24.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.751+0000 7f1454840740 1 -- 172.21.15.83:0/4109221137 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f143802e680 con 0x562394358b40 2024-04-24T09:17:24.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.751+0000 7f1434c00640 1 -- 172.21.15.83:0/4109221137 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7f143c02e040 con 0x562394358b40 2024-04-24T09:17:24.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.751+0000 7f1454840740 10 client.41282 fetch_fsmap learned FSMap version 70 2024-04-24T09:17:24.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.751+0000 7f1454840740 1 -- 172.21.15.83:0/4109221137 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) -- 0x56239436c520 con 0x562394358b40 2024-04-24T09:17:24.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.751+0000 7f1434c00640 1 -- 172.21.15.83:0/4109221137 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f143802e680 con 0x562394358b40 2024-04-24T09:17:24.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.752+0000 7f1434c00640 1 -- 172.21.15.83:0/4109221137 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 70) ==== 8366+0+0 (secure 0 0 0) 0x7f143c01f070 con 0x562394358b40 2024-04-24T09:17:24.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.752+0000 7f1454840740 10 client.41282 fetch_fsmap finished waiting for FSMap version 70 2024-04-24T09:17:24.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.752+0000 7f1454840740 10 client.41282 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-24T09:17:24.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.752+0000 7f1454840740 20 client.41282 populate_metadata read hostname 'smithi083' 2024-04-24T09:17:24.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.752+0000 7f1454840740 1 --2- 172.21.15.83:0/4109221137 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x5623943eb1d0 0x5623943ed5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:24.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.752+0000 7f1454840740 4 client.41282 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:17:24.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.752+0000 7f1454840740 1 -- 172.21.15.83:0/4109221137 --> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5623942c5a60 con 0x5623943eb1d0 2024-04-24T09:17:24.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.752+0000 7f144a000640 1 --2- 172.21.15.83:0/4109221137 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x5623943eb1d0 0x5623943ed5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:24.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.753+0000 7f144a000640 1 --2- 172.21.15.83:0/4109221137 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x5623943eb1d0 0x5623943ed5f0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=fb7b4bb4496513c5 server_cookie=a69c21e10e7aa621 in_seq=0 out_seq=0 2024-04-24T09:17:24.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.753+0000 7f1434c00640 10 client.41282 ms_handle_connect on v2:172.21.15.138:6832/1157967658 2024-04-24T09:17:24.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.753+0000 7f1434c00640 1 -- 172.21.15.83:0/4109221137 <== mds.3 v2:172.21.15.138:6832/1157967658 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5623942c5a60 con 0x5623943eb1d0 2024-04-24T09:17:24.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.753+0000 7f1434c00640 10 client.41282 handle_command_reply: tid=0 2024-04-24T09:17:24.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.753+0000 7f1454840740 1 client.41282 shutdown 2024-04-24T09:17:24.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.754+0000 7f1454840740 1 -- 172.21.15.83:0/4109221137 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562394358b40 msgr2=0x5623943646e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:24.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.754+0000 7f1454840740 1 --2- 172.21.15.83:0/4109221137 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562394358b40 0x5623943646e0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f143c009fd0 tx=0x7f143c00b830 comp rx=0 tx=0).stop 2024-04-24T09:17:24.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.754+0000 7f1454840740 1 -- 172.21.15.83:0/4109221137 shutdown_connections 2024-04-24T09:17:24.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.754+0000 7f1454840740 1 --2- 172.21.15.83:0/4109221137 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x5623943eb1d0 0x5623943ed5f0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:24.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.754+0000 7f1454840740 1 --2- 172.21.15.83:0/4109221137 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562394363b30 0x56239436ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:24.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.754+0000 7f1454840740 1 --2- 172.21.15.83:0/4109221137 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562394359460 0x562394364c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:24.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.754+0000 7f1454840740 1 --2- 172.21.15.83:0/4109221137 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562394358b40 0x5623943646e0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:24.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.754+0000 7f1454840740 1 -- 172.21.15.83:0/4109221137 >> 172.21.15.83:0/4109221137 conn(0x562394354e60 msgr2=0x5623943536c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:24.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.756+0000 7f1434c00640 0 client.41282 ms_handle_reset on v2:172.21.15.138:6832/1157967658 2024-04-24T09:17:24.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.756+0000 7f1454840740 1 -- 172.21.15.83:0/4109221137 shutdown_connections 2024-04-24T09:17:24.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.756+0000 7f1454840740 1 -- 172.21.15.83:0/4109221137 wait complete. 2024-04-24T09:17:24.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.757+0000 7f1454840740 20 client.41282 trim_cache size 0 max 16384 2024-04-24T09:17:24.757 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:17:24.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.758+0000 7f144be00640 1 -- 172.21.15.83:0/1183656116 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f14300a3d20 msgr2=0x7f14300a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:24.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.758+0000 7f144be00640 1 --2- 172.21.15.83:0/1183656116 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f14300a3d20 0x7f14300a61e0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x562394355ff0 tx=0x7f1438005f90 comp rx=0 tx=0).stop 2024-04-24T09:17:24.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.758+0000 7f144be00640 1 -- 172.21.15.83:0/1183656116 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f144c10e760 msgr2=0x7f144c19a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:24.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.758+0000 7f144be00640 1 --2- 172.21.15.83:0/1183656116 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f144c10e760 0x7f144c19a700 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f1444005cc0 tx=0x7f144402f3a0 comp rx=0 tx=0).stop 2024-04-24T09:17:24.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.759+0000 7f144be00640 1 -- 172.21.15.83:0/1183656116 shutdown_connections 2024-04-24T09:17:24.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.759+0000 7f144be00640 1 --2- 172.21.15.83:0/1183656116 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f14300a3d20 0x7f14300a61e0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:24.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.759+0000 7f144be00640 1 --2- 172.21.15.83:0/1183656116 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f144c10e760 0x7f144c19a700 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:24.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.759+0000 7f144be00640 1 --2- 172.21.15.83:0/1183656116 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f144c074cd0 0x7f144c1a0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:24.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.759+0000 7f144be00640 1 --2- 172.21.15.83:0/1183656116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f144c0743b0 0x7f144c1a05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:24.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.759+0000 7f144be00640 1 -- 172.21.15.83:0/1183656116 >> 172.21.15.83:0/1183656116 conn(0x7f144c0fd9d0 msgr2=0x7f144c0fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:24.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.759+0000 7f144be00640 1 -- 172.21.15.83:0/1183656116 shutdown_connections 2024-04-24T09:17:24.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:24.760+0000 7f144be00640 1 -- 172.21.15.83:0/1183656116 wait complete. 2024-04-24T09:17:24.783 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T09:17:25.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.038+0000 7f298b200640 1 -- 172.21.15.83:0/2627516112 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f298c100c70 msgr2=0x7f298c101050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:25.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.038+0000 7f298b200640 1 --2- 172.21.15.83:0/2627516112 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f298c100c70 0x7f298c101050 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f2984009ca0 tx=0x7f298402a8c0 comp rx=0 tx=0).stop 2024-04-24T09:17:25.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.038+0000 7f298b200640 1 -- 172.21.15.83:0/2627516112 shutdown_connections 2024-04-24T09:17:25.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.038+0000 7f298b200640 1 --2- 172.21.15.83:0/2627516112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f298c10e820 0x7f298c110c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:25.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.038+0000 7f298b200640 1 --2- 172.21.15.83:0/2627516112 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f298c101620 0x7f298c10e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:25.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.038+0000 7f298b200640 1 --2- 172.21.15.83:0/2627516112 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f298c100c70 0x7f298c101050 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:25.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.038+0000 7f298b200640 1 -- 172.21.15.83:0/2627516112 >> 172.21.15.83:0/2627516112 conn(0x7f298c0fc880 msgr2=0x7f298c0feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:25.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.038+0000 7f298b200640 1 -- 172.21.15.83:0/2627516112 shutdown_connections 2024-04-24T09:17:25.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.039+0000 7f298b200640 1 -- 172.21.15.83:0/2627516112 wait complete. 2024-04-24T09:17:25.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.040+0000 7f298b200640 1 Processor -- start 2024-04-24T09:17:25.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.040+0000 7f298b200640 1 -- start start 2024-04-24T09:17:25.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.040+0000 7f298b200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f298c100c70 0x7f298c1a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:25.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.040+0000 7f298b200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f298c101620 0x7f298c1a0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:25.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.040+0000 7f298b200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f298c10e820 0x7f298c19a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:25.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.040+0000 7f298b200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f298c1143c0 con 0x7f298c100c70 2024-04-24T09:17:25.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.040+0000 7f298b200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f298c114240 con 0x7f298c101620 2024-04-24T09:17:25.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.040+0000 7f298b200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f298c114540 con 0x7f298c10e820 2024-04-24T09:17:25.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.041+0000 7f2989e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f298c100c70 0x7f298c1a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:25.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.041+0000 7f2989e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f298c100c70 0x7f298c1a0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35330/0 (socket says 172.21.15.83:35330) 2024-04-24T09:17:25.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.041+0000 7f2989e00640 1 -- 172.21.15.83:0/1866572104 learned_addr learned my addr 172.21.15.83:0/1866572104 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:17:25.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.041+0000 7f2989400640 1 --2- 172.21.15.83:0/1866572104 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f298c101620 0x7f298c1a0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:25.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.041+0000 7f298a800640 1 --2- 172.21.15.83:0/1866572104 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f298c10e820 0x7f298c19a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:25.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.041+0000 7f2989e00640 1 -- 172.21.15.83:0/1866572104 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f298c10e820 msgr2=0x7f298c19a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:25.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.041+0000 7f2989e00640 1 --2- 172.21.15.83:0/1866572104 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f298c10e820 0x7f298c19a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:25.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.041+0000 7f2989e00640 1 -- 172.21.15.83:0/1866572104 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f298c101620 msgr2=0x7f298c1a0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:25.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.041+0000 7f2989e00640 1 --2- 172.21.15.83:0/1866572104 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f298c101620 0x7f298c1a0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:25.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.041+0000 7f2989e00640 1 -- 172.21.15.83:0/1866572104 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f298c19af10 con 0x7f298c100c70 2024-04-24T09:17:25.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.042+0000 7f2989e00640 1 --2- 172.21.15.83:0/1866572104 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f298c100c70 0x7f298c1a0640 secure :-1 s=READY pgs=2835 cs=0 l=1 rev1=1 crypto rx=0x7f2984031040 tx=0x7f2984032ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:25.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.042+0000 7f2982a00640 1 -- 172.21.15.83:0/1866572104 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2984003c70 con 0x7f298c100c70 2024-04-24T09:17:25.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.042+0000 7f2982a00640 1 -- 172.21.15.83:0/1866572104 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2984003e10 con 0x7f298c100c70 2024-04-24T09:17:25.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.042+0000 7f2982a00640 1 -- 172.21.15.83:0/1866572104 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2984030b30 con 0x7f298c100c70 2024-04-24T09:17:25.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.043+0000 7f298b200640 1 -- 172.21.15.83:0/1866572104 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f298c19b1a0 con 0x7f298c100c70 2024-04-24T09:17:25.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.043+0000 7f298b200640 1 -- 172.21.15.83:0/1866572104 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f298c06ba20 con 0x7f298c100c70 2024-04-24T09:17:25.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.047+0000 7f2993d7f740 1 Processor -- start 2024-04-24T09:17:25.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.048+0000 7f2993d7f740 1 -- start start 2024-04-24T09:17:25.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.048+0000 7f2993d7f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56522b7da890 0x56522b7dac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:25.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.048+0000 7f2982a00640 1 -- 172.21.15.83:0/1866572104 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2984030cd0 con 0x7f298c100c70 2024-04-24T09:17:25.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.048+0000 7f2993d7f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56522b7db240 0x56522b7ede00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:25.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.048+0000 7f2993d7f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56522b7ee340 0x56522b7f0730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:25.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.048+0000 7f2993d7f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56522b7401d0 con 0x56522b7db240 2024-04-24T09:17:25.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.048+0000 7f2993d7f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56522b6e6450 con 0x56522b7ee340 2024-04-24T09:17:25.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.048+0000 7f2993d7f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56522b73c190 con 0x56522b7da890 2024-04-24T09:17:25.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.049+0000 7f2989400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56522b7ee340 0x56522b7f0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:25.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.049+0000 7f2989400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56522b7ee340 0x56522b7f0730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:39322/0 (socket says 172.21.15.83:39322) 2024-04-24T09:17:25.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.049+0000 7f2989400640 1 -- 172.21.15.83:0/3022805683 learned_addr learned my addr 172.21.15.83:0/3022805683 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:17:25.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.049+0000 7f2989e00640 1 --2- 172.21.15.83:0/3022805683 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56522b7db240 0x56522b7ede00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:25.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.049+0000 7f298a800640 1 --2- 172.21.15.83:0/3022805683 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56522b7da890 0x56522b7dac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:25.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.049+0000 7f2989e00640 1 -- 172.21.15.83:0/3022805683 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56522b7da890 msgr2=0x56522b7dac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:25.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.049+0000 7f2989e00640 1 --2- 172.21.15.83:0/3022805683 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56522b7da890 0x56522b7dac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:25.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.049+0000 7f2989e00640 1 -- 172.21.15.83:0/3022805683 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56522b7ee340 msgr2=0x56522b7f0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:25.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.049+0000 7f2989e00640 1 --2- 172.21.15.83:0/3022805683 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56522b7ee340 0x56522b7f0730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:25.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.049+0000 7f2989e00640 1 -- 172.21.15.83:0/3022805683 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56522b7a3310 con 0x56522b7db240 2024-04-24T09:17:25.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.050+0000 7f2989e00640 1 --2- 172.21.15.83:0/3022805683 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56522b7db240 0x56522b7ede00 secure :-1 s=READY pgs=2836 cs=0 l=1 rev1=1 crypto rx=0x7f29840ef2e0 tx=0x7f29840f5f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25d4aa48429cbaea server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:25.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.051+0000 7f297b400640 1 -- 172.21.15.83:0/3022805683 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29840fa830 con 0x56522b7db240 2024-04-24T09:17:25.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.051+0000 7f2982a00640 1 --2- 172.21.15.83:0/1866572104 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f29700a3da0 0x7f29700a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:25.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.051+0000 7f297b400640 1 -- 172.21.15.83:0/3022805683 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f29840ff070 con 0x56522b7db240 2024-04-24T09:17:25.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.051+0000 7f2982a00640 1 -- 172.21.15.83:0/1866572104 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 9583+0+0 (secure 0 0 0) 0x7f29840f1f00 con 0x7f298c100c70 2024-04-24T09:17:25.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.051+0000 7f297b400640 1 -- 172.21.15.83:0/3022805683 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2984104de0 con 0x56522b7db240 2024-04-24T09:17:25.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.052+0000 7f298a800640 1 --2- 172.21.15.83:0/1866572104 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f29700a3da0 0x7f29700a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:25.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.052+0000 7f2993d7f740 1 -- 172.21.15.83:0/3022805683 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56522b7db240 msgr2=0x56522b7ede00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:25.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.052+0000 7f2993d7f740 1 --2- 172.21.15.83:0/3022805683 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56522b7db240 0x56522b7ede00 secure :-1 s=READY pgs=2836 cs=0 l=1 rev1=1 crypto rx=0x7f29840ef2e0 tx=0x7f29840f5f10 comp rx=0 tx=0).stop 2024-04-24T09:17:25.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.053+0000 7f2993d7f740 1 -- 172.21.15.83:0/3022805683 shutdown_connections 2024-04-24T09:17:25.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.053+0000 7f2993d7f740 1 --2- 172.21.15.83:0/3022805683 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56522b7ee340 0x56522b7f0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:25.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.053+0000 7f2993d7f740 1 --2- 172.21.15.83:0/3022805683 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56522b7db240 0x56522b7ede00 unknown :-1 s=CLOSED pgs=2836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:25.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.053+0000 7f2993d7f740 1 --2- 172.21.15.83:0/3022805683 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56522b7da890 0x56522b7dac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:25.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.053+0000 7f2993d7f740 1 -- 172.21.15.83:0/3022805683 >> 172.21.15.83:0/3022805683 conn(0x56522b7d6e60 msgr2=0x56522b7d7240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:25.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.053+0000 7f2993d7f740 1 -- 172.21.15.83:0/3022805683 shutdown_connections 2024-04-24T09:17:25.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.053+0000 7f298a800640 1 --2- 172.21.15.83:0/1866572104 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f29700a3da0 0x7f29700a6260 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f298c06b6b0 tx=0x7f297c005f90 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:25.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.054+0000 7f2993d7f740 1 -- 172.21.15.83:0/3022805683 wait complete. 2024-04-24T09:17:25.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.054+0000 7f2993d7f740 1 -- start start 2024-04-24T09:17:25.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.054+0000 7f2993d7f740 1 Processor -- start 2024-04-24T09:17:25.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.055+0000 7f2993d7f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56522b7da890 0x56522b7edd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:25.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.055+0000 7f2993d7f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56522b7db240 0x56522b7eb040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:25.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.055+0000 7f2993d7f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56522b7ee340 0x56522b7eb650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:25.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.055+0000 7f2993d7f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56522b7401d0 con 0x56522b7db240 2024-04-24T09:17:25.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.055+0000 7f2993d7f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56522b6f9810 con 0x56522b7ee340 2024-04-24T09:17:25.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.055+0000 7f2993d7f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56522b7f0ec0 con 0x56522b7da890 2024-04-24T09:17:25.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.056+0000 7f2993d7f740 10 client.0 _ll_register_callbacks cb 0x56522b6c7140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:17:25.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.056+0000 7f2989e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56522b7da890 0x56522b7edd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:25.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.056+0000 7f2989e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56522b7da890 0x56522b7edd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:59074/0 (socket says 172.21.15.83:59074) 2024-04-24T09:17:25.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.056+0000 7f2989e00640 1 -- 172.21.15.83:0/2080758353 learned_addr learned my addr 172.21.15.83:0/2080758353 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:17:25.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.056+0000 7f298a800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56522b7ee340 0x56522b7eb650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:25.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.056+0000 7f2989400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56522b7db240 0x56522b7eb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:25.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.057+0000 7f298a800640 1 -- 172.21.15.83:0/2080758353 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56522b7da890 msgr2=0x56522b7edd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:25.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.057+0000 7f298a800640 1 --2- 172.21.15.83:0/2080758353 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56522b7da890 0x56522b7edd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:25.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.057+0000 7f298a800640 1 -- 172.21.15.83:0/2080758353 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56522b7db240 msgr2=0x56522b7eb040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:25.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.057+0000 7f298a800640 1 --2- 172.21.15.83:0/2080758353 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56522b7db240 0x56522b7eb040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:25.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.057+0000 7f298a800640 1 -- 172.21.15.83:0/2080758353 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56522b70a3f0 con 0x56522b7ee340 2024-04-24T09:17:25.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.057+0000 7f2989400640 1 --2- 172.21.15.83:0/2080758353 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56522b7db240 0x56522b7eb040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:17:25.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.057+0000 7f298a800640 1 --2- 172.21.15.83:0/2080758353 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56522b7ee340 0x56522b7eb650 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f297c020550 tx=0x7f297c004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=87734e8e2889b587 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:25.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.057+0000 7f296fe00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:17:25.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.058+0000 7f296fe00640 1 -- 172.21.15.83:0/2080758353 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f297c02d060 con 0x56522b7ee340 2024-04-24T09:17:25.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.058+0000 7f296fe00640 1 -- 172.21.15.83:0/2080758353 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(134..134 src has 1..134) ==== 9583+0+0 (secure 0 0 0) 0x7f297c018030 con 0x56522b7ee340 2024-04-24T09:17:25.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.058+0000 7f2993d7f740 1 -- 172.21.15.83:0/2080758353 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29840ff070 con 0x56522b7ee340 2024-04-24T09:17:25.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.059+0000 7f296fe00640 1 -- 172.21.15.83:0/2080758353 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7f297c011650 con 0x56522b7ee340 2024-04-24T09:17:25.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.059+0000 7f2993d7f740 10 client.41294 fetch_fsmap learned FSMap version 70 2024-04-24T09:17:25.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.059+0000 7f2993d7f740 1 -- 172.21.15.83:0/2080758353 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) -- 0x56522b4523f0 con 0x56522b7ee340 2024-04-24T09:17:25.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.060+0000 7f296fe00640 1 -- 172.21.15.83:0/2080758353 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f29840ff070 con 0x56522b7ee340 2024-04-24T09:17:25.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.060+0000 7f296fe00640 1 -- 172.21.15.83:0/2080758353 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 70) ==== 8366+0+0 (secure 0 0 0) 0x7f297c0289f0 con 0x56522b7ee340 2024-04-24T09:17:25.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.060+0000 7f2993d7f740 10 client.41294 fetch_fsmap finished waiting for FSMap version 70 2024-04-24T09:17:25.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.060+0000 7f2993d7f740 10 client.41294 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-24T09:17:25.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.060+0000 7f2993d7f740 20 client.41294 populate_metadata read hostname 'smithi083' 2024-04-24T09:17:25.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.060+0000 7f2993d7f740 1 --2- 172.21.15.83:0/2080758353 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x56522b7fe150 0x56522b800560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:25.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.060+0000 7f2993d7f740 4 client.41294 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:17:25.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.060+0000 7f2993d7f740 1 -- 172.21.15.83:0/2080758353 --> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56522b74dbf0 con 0x56522b7fe150 2024-04-24T09:17:25.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.060+0000 7f2989e00640 1 --2- 172.21.15.83:0/2080758353 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x56522b7fe150 0x56522b800560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:25.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.061+0000 7f2989e00640 1 --2- 172.21.15.83:0/2080758353 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x56522b7fe150 0x56522b800560 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f5f9193c91ef0f6e server_cookie=ff4d88b47961b9b0 in_seq=0 out_seq=0 2024-04-24T09:17:25.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.061+0000 7f296fe00640 10 client.41294 ms_handle_connect on v2:172.21.15.138:6834/1909219758 2024-04-24T09:17:25.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.062+0000 7f296fe00640 1 -- 172.21.15.83:0/2080758353 <== mds.2 v2:172.21.15.138:6834/1909219758 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56522b74dbf0 con 0x56522b7fe150 2024-04-24T09:17:25.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.062+0000 7f296fe00640 10 client.41294 handle_command_reply: tid=0 2024-04-24T09:17:25.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.062+0000 7f2993d7f740 1 client.41294 shutdown 2024-04-24T09:17:25.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.063+0000 7f2993d7f740 1 -- 172.21.15.83:0/2080758353 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56522b7ee340 msgr2=0x56522b7eb650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:25.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.063+0000 7f2993d7f740 1 --2- 172.21.15.83:0/2080758353 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56522b7ee340 0x56522b7eb650 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f297c020550 tx=0x7f297c004730 comp rx=0 tx=0).stop 2024-04-24T09:17:25.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.063+0000 7f2993d7f740 1 -- 172.21.15.83:0/2080758353 shutdown_connections 2024-04-24T09:17:25.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.063+0000 7f2993d7f740 1 --2- 172.21.15.83:0/2080758353 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x56522b7fe150 0x56522b800560 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:25.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.063+0000 7f2993d7f740 1 --2- 172.21.15.83:0/2080758353 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56522b7ee340 0x56522b7eb650 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:25.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.063+0000 7f2993d7f740 1 --2- 172.21.15.83:0/2080758353 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56522b7db240 0x56522b7eb040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:25.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.063+0000 7f2993d7f740 1 --2- 172.21.15.83:0/2080758353 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56522b7da890 0x56522b7edd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:25.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.063+0000 7f2993d7f740 1 -- 172.21.15.83:0/2080758353 >> 172.21.15.83:0/2080758353 conn(0x56522b7d6e60 msgr2=0x56522b7dcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:25.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.063+0000 7f296fe00640 0 client.41294 ms_handle_reset on v2:172.21.15.138:6834/1909219758 2024-04-24T09:17:25.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.063+0000 7f2993d7f740 1 -- 172.21.15.83:0/2080758353 shutdown_connections 2024-04-24T09:17:25.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.063+0000 7f2993d7f740 1 -- 172.21.15.83:0/2080758353 wait complete. 2024-04-24T09:17:25.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.063+0000 7f2993d7f740 20 client.41294 trim_cache size 0 max 16384 2024-04-24T09:17:25.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.087+0000 7f2993d7f740 1 Processor -- start 2024-04-24T09:17:25.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.088+0000 7f2993d7f740 1 -- start start 2024-04-24T09:17:25.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.088+0000 7f2993d7f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56522b7da890 0x56522b7f9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:25.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.088+0000 7f2993d7f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56522b7db240 0x56522b7ed880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:25.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.088+0000 7f2993d7f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56522b7ee340 0x56522b805e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:25.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.088+0000 7f2993d7f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f298c0718b0 con 0x56522b7db240 2024-04-24T09:17:25.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.088+0000 7f2993d7f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f298c073420 con 0x56522b7da890 2024-04-24T09:17:25.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.088+0000 7f2993d7f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f298c071f80 con 0x56522b7ee340 2024-04-24T09:17:25.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.088+0000 7f2989400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56522b7db240 0x56522b7ed880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:25.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.088+0000 7f2989400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56522b7db240 0x56522b7ed880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35370/0 (socket says 172.21.15.83:35370) 2024-04-24T09:17:25.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.088+0000 7f2989400640 1 -- 172.21.15.83:0/322331227 learned_addr learned my addr 172.21.15.83:0/322331227 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:17:25.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.088+0000 7f2989e00640 1 --2- 172.21.15.83:0/322331227 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56522b7da890 0x56522b7f9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:25.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.089+0000 7f298a800640 1 --2- 172.21.15.83:0/322331227 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56522b7ee340 0x56522b805e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:25.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.089+0000 7f2989400640 1 -- 172.21.15.83:0/322331227 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56522b7ee340 msgr2=0x56522b805e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:25.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.089+0000 7f2989400640 1 --2- 172.21.15.83:0/322331227 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56522b7ee340 0x56522b805e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:25.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.089+0000 7f2989400640 1 -- 172.21.15.83:0/322331227 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56522b7da890 msgr2=0x56522b7f9ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:25.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.089+0000 7f2989400640 1 --2- 172.21.15.83:0/322331227 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56522b7da890 0x56522b7f9ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:25.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.089+0000 7f2989400640 1 -- 172.21.15.83:0/322331227 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56522b51c7d0 con 0x56522b7db240 2024-04-24T09:17:25.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.090+0000 7f2989400640 1 --2- 172.21.15.83:0/322331227 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56522b7db240 0x56522b7ed880 secure :-1 s=READY pgs=2837 cs=0 l=1 rev1=1 crypto rx=0x7f2974005fd0 tx=0x7f2974012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ca4eb11abf78ed5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:25.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.090+0000 7f297aa00640 1 -- 172.21.15.83:0/322331227 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2974020070 con 0x56522b7db240 2024-04-24T09:17:25.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.090+0000 7f297aa00640 1 -- 172.21.15.83:0/322331227 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2974010940 con 0x56522b7db240 2024-04-24T09:17:25.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.090+0000 7f297aa00640 1 -- 172.21.15.83:0/322331227 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f297401b8f0 con 0x56522b7db240 2024-04-24T09:17:25.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.091+0000 7f2993d7f740 1 -- 172.21.15.83:0/322331227 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56522b7db240 msgr2=0x56522b7ed880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:25.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.091+0000 7f2993d7f740 1 --2- 172.21.15.83:0/322331227 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56522b7db240 0x56522b7ed880 secure :-1 s=READY pgs=2837 cs=0 l=1 rev1=1 crypto rx=0x7f2974005fd0 tx=0x7f2974012820 comp rx=0 tx=0).stop 2024-04-24T09:17:25.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.091+0000 7f2993d7f740 1 -- 172.21.15.83:0/322331227 shutdown_connections 2024-04-24T09:17:25.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.091+0000 7f2993d7f740 1 --2- 172.21.15.83:0/322331227 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56522b7ee340 0x56522b805e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:25.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.091+0000 7f2993d7f740 1 --2- 172.21.15.83:0/322331227 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56522b7db240 0x56522b7ed880 unknown :-1 s=CLOSED pgs=2837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:25.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.091+0000 7f2993d7f740 1 --2- 172.21.15.83:0/322331227 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56522b7da890 0x56522b7f9ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:25.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.091+0000 7f2993d7f740 1 -- 172.21.15.83:0/322331227 >> 172.21.15.83:0/322331227 conn(0x56522b7d6e60 msgr2=0x56522b7eb000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:25.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.091+0000 7f2993d7f740 1 -- 172.21.15.83:0/322331227 shutdown_connections 2024-04-24T09:17:25.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.091+0000 7f2993d7f740 1 -- 172.21.15.83:0/322331227 wait complete. 2024-04-24T09:17:25.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.092+0000 7f2993d7f740 1 -- start start 2024-04-24T09:17:25.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.092+0000 7f2993d7f740 1 Processor -- start 2024-04-24T09:17:25.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.093+0000 7f2993d7f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56522b7da890 0x56522b7ee720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:25.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.093+0000 7f2993d7f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56522b7db240 0x56522b7eec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:25.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.093+0000 7f2993d7f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56522b7ee340 0x56522b7ef1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:25.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.093+0000 7f2993d7f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f298c072d50 con 0x56522b7db240 2024-04-24T09:17:25.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.093+0000 7f2993d7f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f298c072680 con 0x56522b7ee340 2024-04-24T09:17:25.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.093+0000 7f2993d7f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f298c06f8b0 con 0x56522b7da890 2024-04-24T09:17:25.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.093+0000 7f2993d7f740 10 client.0 _ll_register_callbacks cb 0x7f297c0098c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:17:25.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.093+0000 7f298a800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56522b7ee340 0x56522b7ef1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:25.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.094+0000 7f298a800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56522b7ee340 0x56522b7ef1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:39348/0 (socket says 172.21.15.83:39348) 2024-04-24T09:17:25.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.094+0000 7f298a800640 1 -- 172.21.15.83:0/1453048912 learned_addr learned my addr 172.21.15.83:0/1453048912 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:17:25.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.094+0000 7f2989e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56522b7da890 0x56522b7ee720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:25.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.094+0000 7f298a800640 1 -- 172.21.15.83:0/1453048912 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56522b7da890 msgr2=0x56522b7ee720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:25.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.094+0000 7f298a800640 1 --2- 172.21.15.83:0/1453048912 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56522b7da890 0x56522b7ee720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:25.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.094+0000 7f298a800640 1 -- 172.21.15.83:0/1453048912 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56522b7db240 msgr2=0x56522b7eec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:25.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.094+0000 7f298a800640 1 --2- 172.21.15.83:0/1453048912 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56522b7db240 0x56522b7eec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:25.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.094+0000 7f298a800640 1 -- 172.21.15.83:0/1453048912 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56522b7f53d0 con 0x56522b7ee340 2024-04-24T09:17:25.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.095+0000 7f298a800640 1 --2- 172.21.15.83:0/1453048912 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56522b7ee340 0x56522b7ef1a0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f297c028d90 tx=0x7f297c0202f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c139ab135d39832e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:25.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.095+0000 7f296fe00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:17:25.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.095+0000 7f296fe00640 1 -- 172.21.15.83:0/1453048912 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f297c037080 con 0x56522b7ee340 2024-04-24T09:17:25.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.095+0000 7f2993d7f740 1 -- 172.21.15.83:0/1453048912 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2974010940 con 0x56522b7ee340 2024-04-24T09:17:25.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.095+0000 7f296fe00640 1 -- 172.21.15.83:0/1453048912 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(134..134 src has 1..134) ==== 9583+0+0 (secure 0 0 0) 0x7f297c01dbd0 con 0x56522b7ee340 2024-04-24T09:17:25.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.096+0000 7f296fe00640 1 -- 172.21.15.83:0/1453048912 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7f297c02d020 con 0x56522b7ee340 2024-04-24T09:17:25.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.096+0000 7f2993d7f740 10 client.41300 fetch_fsmap learned FSMap version 70 2024-04-24T09:17:25.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.096+0000 7f2993d7f740 1 -- 172.21.15.83:0/1453048912 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) -- 0x56522b7f6690 con 0x56522b7ee340 2024-04-24T09:17:25.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.096+0000 7f296fe00640 1 -- 172.21.15.83:0/1453048912 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2974010940 con 0x56522b7ee340 2024-04-24T09:17:25.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.097+0000 7f296fe00640 1 -- 172.21.15.83:0/1453048912 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 70) ==== 8366+0+0 (secure 0 0 0) 0x7f297c0284b0 con 0x56522b7ee340 2024-04-24T09:17:25.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.097+0000 7f2993d7f740 10 client.41300 fetch_fsmap finished waiting for FSMap version 70 2024-04-24T09:17:25.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.097+0000 7f2993d7f740 10 client.41300 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-24T09:17:25.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.097+0000 7f2993d7f740 20 client.41300 populate_metadata read hostname 'smithi083' 2024-04-24T09:17:25.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.097+0000 7f2993d7f740 1 --2- 172.21.15.83:0/1453048912 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x56522b864870 0x56522b866c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:25.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.097+0000 7f2993d7f740 4 client.41300 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:17:25.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.097+0000 7f2993d7f740 1 -- 172.21.15.83:0/1453048912 --> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56522b747a60 con 0x56522b864870 2024-04-24T09:17:25.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.097+0000 7f2989e00640 1 --2- 172.21.15.83:0/1453048912 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x56522b864870 0x56522b866c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:25.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.098+0000 7f2989e00640 1 --2- 172.21.15.83:0/1453048912 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x56522b864870 0x56522b866c90 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9ee6f6e31f1c36cd server_cookie=2dbabec9479cb34e in_seq=0 out_seq=0 2024-04-24T09:17:25.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.098+0000 7f296fe00640 10 client.41300 ms_handle_connect on v2:172.21.15.138:6834/1909219758 2024-04-24T09:17:25.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.098+0000 7f296fe00640 1 -- 172.21.15.83:0/1453048912 <== mds.2 v2:172.21.15.138:6834/1909219758 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56522b747a60 con 0x56522b864870 2024-04-24T09:17:25.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.098+0000 7f296fe00640 10 client.41300 handle_command_reply: tid=0 2024-04-24T09:17:25.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.098+0000 7f2993d7f740 1 client.41300 shutdown 2024-04-24T09:17:25.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.099+0000 7f2993d7f740 1 -- 172.21.15.83:0/1453048912 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56522b7ee340 msgr2=0x56522b7ef1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:25.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.099+0000 7f2993d7f740 1 --2- 172.21.15.83:0/1453048912 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56522b7ee340 0x56522b7ef1a0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f297c028d90 tx=0x7f297c0202f0 comp rx=0 tx=0).stop 2024-04-24T09:17:25.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.099+0000 7f2993d7f740 1 -- 172.21.15.83:0/1453048912 shutdown_connections 2024-04-24T09:17:25.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.099+0000 7f2993d7f740 1 --2- 172.21.15.83:0/1453048912 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x56522b864870 0x56522b866c90 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:25.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.099+0000 7f2993d7f740 1 --2- 172.21.15.83:0/1453048912 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56522b7ee340 0x56522b7ef1a0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:25.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.099+0000 7f2993d7f740 1 --2- 172.21.15.83:0/1453048912 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56522b7db240 0x56522b7eec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:25.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.099+0000 7f2993d7f740 1 --2- 172.21.15.83:0/1453048912 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56522b7da890 0x56522b7ee720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:25.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.099+0000 7f296fe00640 0 client.41300 ms_handle_reset on v2:172.21.15.138:6834/1909219758 2024-04-24T09:17:25.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.099+0000 7f2993d7f740 1 -- 172.21.15.83:0/1453048912 >> 172.21.15.83:0/1453048912 conn(0x56522b7d6e60 msgr2=0x56522b7e9910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:25.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.101+0000 7f2993d7f740 1 -- 172.21.15.83:0/1453048912 shutdown_connections 2024-04-24T09:17:25.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.101+0000 7f2993d7f740 1 -- 172.21.15.83:0/1453048912 wait complete. 2024-04-24T09:17:25.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.102+0000 7f2993d7f740 20 client.41300 trim_cache size 0 max 16384 2024-04-24T09:17:25.102 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:17:25.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.103+0000 7f298b200640 1 -- 172.21.15.83:0/1866572104 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f29700a3da0 msgr2=0x7f29700a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:25.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.103+0000 7f298b200640 1 --2- 172.21.15.83:0/1866572104 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f29700a3da0 0x7f29700a6260 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f298c06b6b0 tx=0x7f297c005f90 comp rx=0 tx=0).stop 2024-04-24T09:17:25.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.103+0000 7f298b200640 1 -- 172.21.15.83:0/1866572104 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f298c100c70 msgr2=0x7f298c1a0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:25.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.103+0000 7f298b200640 1 --2- 172.21.15.83:0/1866572104 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f298c100c70 0x7f298c1a0640 secure :-1 s=READY pgs=2835 cs=0 l=1 rev1=1 crypto rx=0x7f2984031040 tx=0x7f2984032ff0 comp rx=0 tx=0).stop 2024-04-24T09:17:25.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.103+0000 7f298b200640 1 -- 172.21.15.83:0/1866572104 shutdown_connections 2024-04-24T09:17:25.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.103+0000 7f298b200640 1 --2- 172.21.15.83:0/1866572104 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f29700a3da0 0x7f29700a6260 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:25.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.103+0000 7f298b200640 1 --2- 172.21.15.83:0/1866572104 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f298c10e820 0x7f298c19a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:25.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.103+0000 7f298b200640 1 --2- 172.21.15.83:0/1866572104 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f298c101620 0x7f298c1a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:25.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.103+0000 7f298b200640 1 --2- 172.21.15.83:0/1866572104 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f298c100c70 0x7f298c1a0640 unknown :-1 s=CLOSED pgs=2835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:25.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.103+0000 7f298b200640 1 -- 172.21.15.83:0/1866572104 >> 172.21.15.83:0/1866572104 conn(0x7f298c0fc880 msgr2=0x7f298c10fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:25.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.104+0000 7f298b200640 1 -- 172.21.15.83:0/1866572104 shutdown_connections 2024-04-24T09:17:25.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.104+0000 7f298b200640 1 -- 172.21.15.83:0/1866572104 wait complete. 2024-04-24T09:17:25.131 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T09:17:25.230 INFO:tasks.workunit.client.0.smithi083.stdout: 10 87090 6.48 MB/sec execute 596 sec latency 102.931 ms 2024-04-24T09:17:25.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.395+0000 7f6c87400640 1 -- 172.21.15.83:0/171001680 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c8010e660 msgr2=0x7f6c8010ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:25.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.396+0000 7f6c87400640 1 --2- 172.21.15.83:0/171001680 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c8010e660 0x7f6c8010ea40 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f6c74009900 tx=0x7f6c7402a410 comp rx=0 tx=0).stop 2024-04-24T09:17:25.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.396+0000 7f6c87400640 1 -- 172.21.15.83:0/171001680 shutdown_connections 2024-04-24T09:17:25.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.396+0000 7f6c87400640 1 --2- 172.21.15.83:0/171001680 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c80108fd0 0x7f6c80115490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:25.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.396+0000 7f6c87400640 1 --2- 172.21.15.83:0/171001680 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6c80108610 0x7f6c80108a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:25.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.396+0000 7f6c87400640 1 --2- 172.21.15.83:0/171001680 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c8010e660 0x7f6c8010ea40 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:25.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.396+0000 7f6c87400640 1 -- 172.21.15.83:0/171001680 >> 172.21.15.83:0/171001680 conn(0x7f6c80076540 msgr2=0x7f6c80078960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:25.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.397+0000 7f6c87400640 1 -- 172.21.15.83:0/171001680 shutdown_connections 2024-04-24T09:17:25.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.397+0000 7f6c87400640 1 -- 172.21.15.83:0/171001680 wait complete. 2024-04-24T09:17:25.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.398+0000 7f6c87400640 1 Processor -- start 2024-04-24T09:17:25.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.398+0000 7f6c87400640 1 -- start start 2024-04-24T09:17:25.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.398+0000 7f6c87400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c80108610 0x7f6c8019ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:25.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.398+0000 7f6c87400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c80108fd0 0x7f6c8019f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:25.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.398+0000 7f6c87400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6c8010e660 0x7f6c801a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:25.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.398+0000 7f6c87400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6c801188e0 con 0x7f6c80108610 2024-04-24T09:17:25.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.398+0000 7f6c87400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6c80118760 con 0x7f6c80108fd0 2024-04-24T09:17:25.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.398+0000 7f6c87400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6c80118a60 con 0x7f6c8010e660 2024-04-24T09:17:25.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.399+0000 7f6c84a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c80108610 0x7f6c8019ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:25.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.399+0000 7f6c84a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c80108610 0x7f6c8019ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35390/0 (socket says 172.21.15.83:35390) 2024-04-24T09:17:25.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.399+0000 7f6c84a00640 1 -- 172.21.15.83:0/966016246 learned_addr learned my addr 172.21.15.83:0/966016246 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:17:25.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.400+0000 7f6c84a00640 1 -- 172.21.15.83:0/966016246 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6c8010e660 msgr2=0x7f6c801a3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:25.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.400+0000 7f6c84a00640 1 --2- 172.21.15.83:0/966016246 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6c8010e660 0x7f6c801a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:25.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.400+0000 7f6c84a00640 1 -- 172.21.15.83:0/966016246 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c80108fd0 msgr2=0x7f6c8019f340 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-24T09:17:25.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.400+0000 7f6c84a00640 1 --2- 172.21.15.83:0/966016246 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c80108fd0 0x7f6c8019f340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:25.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.400+0000 7f6c84a00640 1 -- 172.21.15.83:0/966016246 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c801a3da0 con 0x7f6c80108610 2024-04-24T09:17:25.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.400+0000 7f6c84a00640 1 --2- 172.21.15.83:0/966016246 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c80108610 0x7f6c8019ee00 secure :-1 s=READY pgs=2838 cs=0 l=1 rev1=1 crypto rx=0x7f6c7402a930 tx=0x7f6c74003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:25.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.401+0000 7f6c7d600640 1 -- 172.21.15.83:0/966016246 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c7403f070 con 0x7f6c80108610 2024-04-24T09:17:25.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.401+0000 7f6c7d600640 1 -- 172.21.15.83:0/966016246 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6c7403bb10 con 0x7f6c80108610 2024-04-24T09:17:25.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.401+0000 7f6c87400640 1 -- 172.21.15.83:0/966016246 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c801a4030 con 0x7f6c80108610 2024-04-24T09:17:25.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.401+0000 7f6c7d600640 1 -- 172.21.15.83:0/966016246 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c74035030 con 0x7f6c80108610 2024-04-24T09:17:25.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.401+0000 7f6c87400640 1 -- 172.21.15.83:0/966016246 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c801ab920 con 0x7f6c80108610 2024-04-24T09:17:25.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.409+0000 7f6c7d600640 1 -- 172.21.15.83:0/966016246 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6c74042070 con 0x7f6c80108610 2024-04-24T09:17:25.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.412+0000 7f6c7d600640 1 --2- 172.21.15.83:0/966016246 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f6c680a3e20 0x7f6c680a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:25.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.412+0000 7f6c7d600640 1 -- 172.21.15.83:0/966016246 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 9583+0+0 (secure 0 0 0) 0x7f6c740f1030 con 0x7f6c80108610 2024-04-24T09:17:25.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.413+0000 7f6c7fe00640 1 --2- 172.21.15.83:0/966016246 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f6c680a3e20 0x7f6c680a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:25.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.413+0000 7f6c8a170740 1 Processor -- start 2024-04-24T09:17:25.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.414+0000 7f6c8a170740 1 -- start start 2024-04-24T09:17:25.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.415+0000 7f6c7fe00640 1 --2- 172.21.15.83:0/966016246 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f6c680a3e20 0x7f6c680a62e0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f6c8010dd50 tx=0x7f6c6c00b370 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:25.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.415+0000 7f6c8a170740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560143ef5950 0x560143ef5d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:25.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.415+0000 7f6c8a170740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560143ef6270 0x560143f00400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:25.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.415+0000 7f6c8a170740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560143f00940 0x560143f02d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:25.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.415+0000 7f6c7fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560143ef6270 0x560143f00400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:25.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.416+0000 7f6c8a170740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x560143e5b1d0 con 0x560143ef6270 2024-04-24T09:17:25.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.416+0000 7f6c7fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560143ef6270 0x560143f00400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35400/0 (socket says 172.21.15.83:35400) 2024-04-24T09:17:25.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.416+0000 7f6c7fe00640 1 -- 172.21.15.83:0/179308865 learned_addr learned my addr 172.21.15.83:0/179308865 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:17:25.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.416+0000 7f6c8a170740 1 -- 172.21.15.83:0/179308865 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560143e01450 con 0x560143f00940 2024-04-24T09:17:25.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.416+0000 7f6c8a170740 1 -- 172.21.15.83:0/179308865 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x560143e57190 con 0x560143ef5950 2024-04-24T09:17:25.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.416+0000 7f6c85400640 1 --2- 172.21.15.83:0/179308865 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560143f00940 0x560143f02d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:25.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.417+0000 7f6c7fe00640 1 -- 172.21.15.83:0/179308865 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560143ef5950 msgr2=0x560143ef5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:25.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.417+0000 7f6c7fe00640 1 --2- 172.21.15.83:0/179308865 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560143ef5950 0x560143ef5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:25.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.417+0000 7f6c7fe00640 1 -- 172.21.15.83:0/179308865 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560143f00940 msgr2=0x560143f02d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:25.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.417+0000 7f6c7fe00640 1 --2- 172.21.15.83:0/179308865 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560143f00940 0x560143f02d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:25.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.417+0000 7f6c7fe00640 1 -- 172.21.15.83:0/179308865 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560143ebe310 con 0x560143ef6270 2024-04-24T09:17:25.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.418+0000 7f6c7fe00640 1 --2- 172.21.15.83:0/179308865 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560143ef6270 0x560143f00400 secure :-1 s=READY pgs=2839 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c00ddd0 tx=0x7f6c6c010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7d6920550c17dcb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:25.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.418+0000 7f6c71c00640 1 -- 172.21.15.83:0/179308865 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c6c01d440 con 0x560143ef6270 2024-04-24T09:17:25.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.419+0000 7f6c71c00640 1 -- 172.21.15.83:0/179308865 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6c6c022070 con 0x560143ef6270 2024-04-24T09:17:25.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.419+0000 7f6c71c00640 1 -- 172.21.15.83:0/179308865 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c6c00f8c0 con 0x560143ef6270 2024-04-24T09:17:25.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.420+0000 7f6c8a170740 1 -- 172.21.15.83:0/179308865 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560143ef6270 msgr2=0x560143f00400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:25.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.420+0000 7f6c8a170740 1 --2- 172.21.15.83:0/179308865 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560143ef6270 0x560143f00400 secure :-1 s=READY pgs=2839 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c00ddd0 tx=0x7f6c6c010000 comp rx=0 tx=0).stop 2024-04-24T09:17:25.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.421+0000 7f6c8a170740 1 -- 172.21.15.83:0/179308865 shutdown_connections 2024-04-24T09:17:25.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.421+0000 7f6c8a170740 1 --2- 172.21.15.83:0/179308865 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560143f00940 0x560143f02d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:25.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.421+0000 7f6c8a170740 1 --2- 172.21.15.83:0/179308865 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560143ef6270 0x560143f00400 unknown :-1 s=CLOSED pgs=2839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:25.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.421+0000 7f6c8a170740 1 --2- 172.21.15.83:0/179308865 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560143ef5950 0x560143ef5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:25.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.421+0000 7f6c8a170740 1 -- 172.21.15.83:0/179308865 >> 172.21.15.83:0/179308865 conn(0x560143ef1c70 msgr2=0x560143ef2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:25.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.421+0000 7f6c8a170740 1 -- 172.21.15.83:0/179308865 shutdown_connections 2024-04-24T09:17:25.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.421+0000 7f6c8a170740 1 -- 172.21.15.83:0/179308865 wait complete. 2024-04-24T09:17:25.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.422+0000 7f6c8a170740 1 -- start start 2024-04-24T09:17:25.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.423+0000 7f6c8a170740 1 Processor -- start 2024-04-24T09:17:25.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.428+0000 7f6c8a170740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560143ef5950 0x560143f003a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:25.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.429+0000 7f6c8a170740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560143ef6270 0x560143efd690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:25.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.429+0000 7f6c8a170740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560143f00940 0x560143efddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:25.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.429+0000 7f6c8a170740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x560143f03770 con 0x560143ef5950 2024-04-24T09:17:25.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.429+0000 7f6c8a170740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560143e14810 con 0x560143ef6270 2024-04-24T09:17:25.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.429+0000 7f6c8a170740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x560143f038f0 con 0x560143f00940 2024-04-24T09:17:25.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.429+0000 7f6c8a170740 10 client.0 _ll_register_callbacks cb 0x560143de2140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:17:25.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.429+0000 7f6c84a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560143ef5950 0x560143f003a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:25.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.429+0000 7f6c7fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560143ef6270 0x560143efd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:25.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.429+0000 7f6c84a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560143ef5950 0x560143f003a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35412/0 (socket says 172.21.15.83:35412) 2024-04-24T09:17:25.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.429+0000 7f6c84a00640 1 -- 172.21.15.83:0/4150493415 learned_addr learned my addr 172.21.15.83:0/4150493415 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:17:25.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.430+0000 7f6c7fe00640 1 -- 172.21.15.83:0/4150493415 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560143f00940 msgr2=0x560143efddb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-24T09:17:25.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.430+0000 7f6c7fe00640 1 --2- 172.21.15.83:0/4150493415 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560143f00940 0x560143efddb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:25.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.430+0000 7f6c7fe00640 1 -- 172.21.15.83:0/4150493415 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560143ef5950 msgr2=0x560143f003a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:25.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.431+0000 7f6c7fe00640 1 --2- 172.21.15.83:0/4150493415 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560143ef5950 0x560143f003a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:25.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.431+0000 7f6c7fe00640 1 -- 172.21.15.83:0/4150493415 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560143e253f0 con 0x560143ef6270 2024-04-24T09:17:25.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.431+0000 7f6c84a00640 1 --2- 172.21.15.83:0/4150493415 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560143ef5950 0x560143f003a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:17:25.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.435+0000 7f6c7fe00640 1 --2- 172.21.15.83:0/4150493415 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560143ef6270 0x560143efd690 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c012040 tx=0x7f6c6c00dec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64f3ade5e06e64a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:25.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.435+0000 7f6c66000640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:17:25.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.437+0000 7f6c66000640 1 -- 172.21.15.83:0/4150493415 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c6c00f800 con 0x560143ef6270 2024-04-24T09:17:25.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.437+0000 7f6c66000640 1 -- 172.21.15.83:0/4150493415 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(134..134 src has 1..134) ==== 9583+0+0 (secure 0 0 0) 0x7f6c6c00f9a0 con 0x560143ef6270 2024-04-24T09:17:25.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.438+0000 7f6c8a170740 1 -- 172.21.15.83:0/4150493415 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c6c022070 con 0x560143ef6270 2024-04-24T09:17:25.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.438+0000 7f6c66000640 1 -- 172.21.15.83:0/4150493415 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7f6c6c00fba0 con 0x560143ef6270 2024-04-24T09:17:25.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.438+0000 7f6c8a170740 10 client.41309 fetch_fsmap learned FSMap version 70 2024-04-24T09:17:25.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.438+0000 7f6c8a170740 1 -- 172.21.15.83:0/4150493415 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) -- 0x560143b6d3f0 con 0x560143ef6270 2024-04-24T09:17:25.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.439+0000 7f6c66000640 1 -- 172.21.15.83:0/4150493415 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6c6c022070 con 0x560143ef6270 2024-04-24T09:17:25.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.439+0000 7f6c66000640 1 -- 172.21.15.83:0/4150493415 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 70) ==== 8366+0+0 (secure 0 0 0) 0x7f6c6c03c420 con 0x560143ef6270 2024-04-24T09:17:25.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.439+0000 7f6c8a170740 10 client.41309 fetch_fsmap finished waiting for FSMap version 70 2024-04-24T09:17:25.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.439+0000 7f6c8a170740 10 client.41309 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-24T09:17:25.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.439+0000 7f6c8a170740 20 client.41309 populate_metadata read hostname 'smithi083' 2024-04-24T09:17:25.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.440+0000 7f6c8a170740 1 --2- 172.21.15.83:0/4150493415 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x560143f10b00 0x560143f12ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:25.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.440+0000 7f6c8a170740 4 client.41309 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:17:25.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.440+0000 7f6c8a170740 1 -- 172.21.15.83:0/4150493415 --> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560143e68bf0 con 0x560143f10b00 2024-04-24T09:17:25.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.441+0000 7f6c84a00640 1 --2- 172.21.15.83:0/4150493415 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x560143f10b00 0x560143f12ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:25.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.442+0000 7f6c84a00640 1 --2- 172.21.15.83:0/4150493415 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x560143f10b00 0x560143f12ef0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ec4103930fa305f7 server_cookie=b1658224b8f30f34 in_seq=0 out_seq=0 2024-04-24T09:17:25.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.442+0000 7f6c66000640 10 client.41309 ms_handle_connect on v2:172.21.15.138:6836/900566754 2024-04-24T09:17:25.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.443+0000 7f6c66000640 1 -- 172.21.15.83:0/4150493415 <== mds.1 v2:172.21.15.138:6836/900566754 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560143e68bf0 con 0x560143f10b00 2024-04-24T09:17:25.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.443+0000 7f6c66000640 10 client.41309 handle_command_reply: tid=0 2024-04-24T09:17:25.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.443+0000 7f6c8a170740 1 client.41309 shutdown 2024-04-24T09:17:25.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.444+0000 7f6c8a170740 1 -- 172.21.15.83:0/4150493415 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560143ef6270 msgr2=0x560143efd690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:25.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.444+0000 7f6c8a170740 1 --2- 172.21.15.83:0/4150493415 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560143ef6270 0x560143efd690 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c012040 tx=0x7f6c6c00dec0 comp rx=0 tx=0).stop 2024-04-24T09:17:25.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.444+0000 7f6c8a170740 1 -- 172.21.15.83:0/4150493415 shutdown_connections 2024-04-24T09:17:25.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.444+0000 7f6c8a170740 1 --2- 172.21.15.83:0/4150493415 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x560143f10b00 0x560143f12ef0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:25.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.444+0000 7f6c8a170740 1 --2- 172.21.15.83:0/4150493415 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560143f00940 0x560143efddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:25.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.444+0000 7f6c8a170740 1 --2- 172.21.15.83:0/4150493415 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560143ef6270 0x560143efd690 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:25.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.444+0000 7f6c8a170740 1 --2- 172.21.15.83:0/4150493415 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560143ef5950 0x560143f003a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:25.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.444+0000 7f6c8a170740 1 -- 172.21.15.83:0/4150493415 >> 172.21.15.83:0/4150493415 conn(0x560143ef1c70 msgr2=0x560143ef7950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:25.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.444+0000 7f6c66000640 0 client.41309 ms_handle_reset on v2:172.21.15.138:6836/900566754 2024-04-24T09:17:25.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.444+0000 7f6c8a170740 1 -- 172.21.15.83:0/4150493415 shutdown_connections 2024-04-24T09:17:25.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.444+0000 7f6c8a170740 1 -- 172.21.15.83:0/4150493415 wait complete. 2024-04-24T09:17:25.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.445+0000 7f6c8a170740 20 client.41309 trim_cache size 0 max 16384 2024-04-24T09:17:25.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.471+0000 7f6c8a170740 1 Processor -- start 2024-04-24T09:17:25.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.472+0000 7f6c8a170740 1 -- start start 2024-04-24T09:17:25.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.472+0000 7f6c8a170740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560143ef5950 0x560143ef80b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:25.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.472+0000 7f6c8a170740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560143ef6270 0x560143effed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:25.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.472+0000 7f6c8a170740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560143f00940 0x560143efa930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:25.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.472+0000 7f6c8a170740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6c801b5d30 con 0x560143f00940 2024-04-24T09:17:25.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.472+0000 7f6c8a170740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6c801b78a0 con 0x560143ef5950 2024-04-24T09:17:25.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.472+0000 7f6c8a170740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6c801b6400 con 0x560143ef6270 2024-04-24T09:17:25.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.473+0000 7f6c84a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560143ef5950 0x560143ef80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:25.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.473+0000 7f6c85400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560143f00940 0x560143efa930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:25.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.473+0000 7f6c7fe00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560143ef6270 0x560143effed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:25.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.473+0000 7f6c85400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560143f00940 0x560143efa930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35418/0 (socket says 172.21.15.83:35418) 2024-04-24T09:17:25.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.473+0000 7f6c85400640 1 -- 172.21.15.83:0/186868599 learned_addr learned my addr 172.21.15.83:0/186868599 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:17:25.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.474+0000 7f6c7fe00640 1 -- 172.21.15.83:0/186868599 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560143ef5950 msgr2=0x560143ef80b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:25.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.474+0000 7f6c7fe00640 1 --2- 172.21.15.83:0/186868599 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560143ef5950 0x560143ef80b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:25.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.474+0000 7f6c7fe00640 1 -- 172.21.15.83:0/186868599 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560143f00940 msgr2=0x560143efa930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:25.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.474+0000 7f6c7fe00640 1 --2- 172.21.15.83:0/186868599 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560143f00940 0x560143efa930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:25.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.474+0000 7f6c7fe00640 1 -- 172.21.15.83:0/186868599 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560143efe120 con 0x560143ef6270 2024-04-24T09:17:25.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.474+0000 7f6c85400640 1 --2- 172.21.15.83:0/186868599 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560143f00940 0x560143efa930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:17:25.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.475+0000 7f6c7fe00640 1 --2- 172.21.15.83:0/186868599 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560143ef6270 0x560143effed0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c005860 tx=0x7f6c6c010d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a022340bca9e720 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:25.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.475+0000 7f6c71200640 1 -- 172.21.15.83:0/186868599 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c6c03c420 con 0x560143ef6270 2024-04-24T09:17:25.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.475+0000 7f6c71200640 1 -- 172.21.15.83:0/186868599 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6c6c040070 con 0x560143ef6270 2024-04-24T09:17:25.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.475+0000 7f6c71200640 1 -- 172.21.15.83:0/186868599 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c6c005040 con 0x560143ef6270 2024-04-24T09:17:25.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.476+0000 7f6c8a170740 1 -- 172.21.15.83:0/186868599 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560143ef6270 msgr2=0x560143effed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:25.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.476+0000 7f6c8a170740 1 --2- 172.21.15.83:0/186868599 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560143ef6270 0x560143effed0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c005860 tx=0x7f6c6c010d30 comp rx=0 tx=0).stop 2024-04-24T09:17:25.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.476+0000 7f6c8a170740 1 -- 172.21.15.83:0/186868599 shutdown_connections 2024-04-24T09:17:25.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.476+0000 7f6c8a170740 1 --2- 172.21.15.83:0/186868599 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560143f00940 0x560143efa930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:25.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.476+0000 7f6c8a170740 1 --2- 172.21.15.83:0/186868599 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560143ef6270 0x560143effed0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:25.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.476+0000 7f6c8a170740 1 --2- 172.21.15.83:0/186868599 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560143ef5950 0x560143ef80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:25.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.476+0000 7f6c8a170740 1 -- 172.21.15.83:0/186868599 >> 172.21.15.83:0/186868599 conn(0x560143ef1c70 msgr2=0x560143efbf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:25.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.477+0000 7f6c8a170740 1 -- 172.21.15.83:0/186868599 shutdown_connections 2024-04-24T09:17:25.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.477+0000 7f6c8a170740 1 -- 172.21.15.83:0/186868599 wait complete. 2024-04-24T09:17:25.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.477+0000 7f6c8a170740 1 -- start start 2024-04-24T09:17:25.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.477+0000 7f6c8a170740 1 Processor -- start 2024-04-24T09:17:25.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.478+0000 7f6c8a170740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560143ef5950 0x560143f16400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:25.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.478+0000 7f6c8a170740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560143ef6270 0x560143efa0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:25.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.478+0000 7f6c8a170740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560143f00940 0x560143efa690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:25.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.478+0000 7f6c8a170740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6c801b71d0 con 0x560143ef5950 2024-04-24T09:17:25.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.478+0000 7f6c8a170740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6c801b6b00 con 0x560143ef6270 2024-04-24T09:17:25.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.478+0000 7f6c8a170740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6c801b3d30 con 0x560143f00940 2024-04-24T09:17:25.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.478+0000 7f6c8a170740 10 client.0 _ll_register_callbacks cb 0x7f6c6c00dca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:17:25.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.479+0000 7f6c84a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560143ef5950 0x560143f16400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:25.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.479+0000 7f6c84a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560143ef5950 0x560143f16400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35424/0 (socket says 172.21.15.83:35424) 2024-04-24T09:17:25.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.479+0000 7f6c84a00640 1 -- 172.21.15.83:0/2432369434 learned_addr learned my addr 172.21.15.83:0/2432369434 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:17:25.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.479+0000 7f6c7fe00640 1 --2- 172.21.15.83:0/2432369434 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560143ef6270 0x560143efa0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:25.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.479+0000 7f6c85400640 1 --2- 172.21.15.83:0/2432369434 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560143f00940 0x560143efa690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:25.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.479+0000 7f6c84a00640 1 -- 172.21.15.83:0/2432369434 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560143f00940 msgr2=0x560143efa690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:25.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.479+0000 7f6c84a00640 1 --2- 172.21.15.83:0/2432369434 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560143f00940 0x560143efa690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:25.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.479+0000 7f6c84a00640 1 -- 172.21.15.83:0/2432369434 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560143ef6270 msgr2=0x560143efa0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:25.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.479+0000 7f6c84a00640 1 --2- 172.21.15.83:0/2432369434 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560143ef6270 0x560143efa0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:25.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.479+0000 7f6c84a00640 1 -- 172.21.15.83:0/2432369434 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560143c377d0 con 0x560143ef5950 2024-04-24T09:17:25.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.480+0000 7f6c84a00640 1 --2- 172.21.15.83:0/2432369434 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560143ef5950 0x560143f16400 secure :-1 s=READY pgs=2840 cs=0 l=1 rev1=1 crypto rx=0x7f6c740b8fb0 tx=0x7f6c74009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9eaf97d1b9760f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:25.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.480+0000 7f6c66000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:17:25.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.480+0000 7f6c66000640 1 -- 172.21.15.83:0/2432369434 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c74049040 con 0x560143ef5950 2024-04-24T09:17:25.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.481+0000 7f6c66000640 1 -- 172.21.15.83:0/2432369434 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(134..134 src has 1..134) ==== 9583+0+0 (secure 0 0 0) 0x7f6c7403bcc0 con 0x560143ef5950 2024-04-24T09:17:25.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.481+0000 7f6c8a170740 1 -- 172.21.15.83:0/2432369434 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c6c040070 con 0x560143ef5950 2024-04-24T09:17:25.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.481+0000 7f6c66000640 1 -- 172.21.15.83:0/2432369434 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7f6c74031430 con 0x560143ef5950 2024-04-24T09:17:25.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.481+0000 7f6c8a170740 10 client.24258 fetch_fsmap learned FSMap version 70 2024-04-24T09:17:25.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.481+0000 7f6c8a170740 1 -- 172.21.15.83:0/2432369434 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) -- 0x560143f76bc0 con 0x560143ef5950 2024-04-24T09:17:25.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.482+0000 7f6c66000640 1 -- 172.21.15.83:0/2432369434 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6c6c040070 con 0x560143ef5950 2024-04-24T09:17:25.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.482+0000 7f6c66000640 1 -- 172.21.15.83:0/2432369434 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 70) ==== 8366+0+0 (secure 0 0 0) 0x7f6c7404c060 con 0x560143ef5950 2024-04-24T09:17:25.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.482+0000 7f6c8a170740 10 client.24258 fetch_fsmap finished waiting for FSMap version 70 2024-04-24T09:17:25.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.482+0000 7f6c8a170740 10 client.24258 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-24T09:17:25.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.482+0000 7f6c8a170740 20 client.24258 populate_metadata read hostname 'smithi083' 2024-04-24T09:17:25.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.482+0000 7f6c8a170740 1 --2- 172.21.15.83:0/2432369434 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x560143f77390 0x560143f797b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:25.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.482+0000 7f6c8a170740 4 client.24258 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:17:25.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.482+0000 7f6c8a170740 1 -- 172.21.15.83:0/2432369434 --> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560143f03280 con 0x560143f77390 2024-04-24T09:17:25.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.483+0000 7f6c7fe00640 1 --2- 172.21.15.83:0/2432369434 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x560143f77390 0x560143f797b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:25.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.484+0000 7f6c7fe00640 1 --2- 172.21.15.83:0/2432369434 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x560143f77390 0x560143f797b0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d642773e723edfd5 server_cookie=d37cba9a758e6995 in_seq=0 out_seq=0 2024-04-24T09:17:25.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.484+0000 7f6c66000640 10 client.24258 ms_handle_connect on v2:172.21.15.138:6836/900566754 2024-04-24T09:17:25.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.484+0000 7f6c66000640 1 -- 172.21.15.83:0/2432369434 <== mds.1 v2:172.21.15.138:6836/900566754 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560143f03280 con 0x560143f77390 2024-04-24T09:17:25.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.484+0000 7f6c66000640 10 client.24258 handle_command_reply: tid=0 2024-04-24T09:17:25.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.484+0000 7f6c8a170740 1 client.24258 shutdown 2024-04-24T09:17:25.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.486+0000 7f6c8a170740 1 -- 172.21.15.83:0/2432369434 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560143ef5950 msgr2=0x560143f16400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:25.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.486+0000 7f6c8a170740 1 --2- 172.21.15.83:0/2432369434 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560143ef5950 0x560143f16400 secure :-1 s=READY pgs=2840 cs=0 l=1 rev1=1 crypto rx=0x7f6c740b8fb0 tx=0x7f6c74009260 comp rx=0 tx=0).stop 2024-04-24T09:17:25.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.486+0000 7f6c8a170740 1 -- 172.21.15.83:0/2432369434 shutdown_connections 2024-04-24T09:17:25.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.486+0000 7f6c8a170740 1 --2- 172.21.15.83:0/2432369434 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x560143f77390 0x560143f797b0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:25.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.486+0000 7f6c8a170740 1 --2- 172.21.15.83:0/2432369434 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560143f00940 0x560143efa690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:25.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.487+0000 7f6c8a170740 1 --2- 172.21.15.83:0/2432369434 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560143ef6270 0x560143efa0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:25.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.487+0000 7f6c8a170740 1 --2- 172.21.15.83:0/2432369434 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560143ef5950 0x560143f16400 secure :-1 s=CLOSED pgs=2840 cs=0 l=1 rev1=1 crypto rx=0x7f6c740b8fb0 tx=0x7f6c74009260 comp rx=0 tx=0).stop 2024-04-24T09:17:25.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.487+0000 7f6c8a170740 1 -- 172.21.15.83:0/2432369434 >> 172.21.15.83:0/2432369434 conn(0x560143ef1c70 msgr2=0x560143f0bb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:25.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.487+0000 7f6c66000640 0 client.24258 ms_handle_reset on v2:172.21.15.138:6836/900566754 2024-04-24T09:17:25.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.487+0000 7f6c8a170740 1 -- 172.21.15.83:0/2432369434 shutdown_connections 2024-04-24T09:17:25.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.488+0000 7f6c8a170740 1 -- 172.21.15.83:0/2432369434 wait complete. 2024-04-24T09:17:25.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.488+0000 7f6c8a170740 20 client.24258 trim_cache size 0 max 16384 2024-04-24T09:17:25.490 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:17:25.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.491+0000 7f6c87400640 1 -- 172.21.15.83:0/966016246 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f6c680a3e20 msgr2=0x7f6c680a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:25.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.491+0000 7f6c87400640 1 --2- 172.21.15.83:0/966016246 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f6c680a3e20 0x7f6c680a62e0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f6c8010dd50 tx=0x7f6c6c00b370 comp rx=0 tx=0).stop 2024-04-24T09:17:25.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.491+0000 7f6c87400640 1 -- 172.21.15.83:0/966016246 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c80108610 msgr2=0x7f6c8019ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:25.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.491+0000 7f6c87400640 1 --2- 172.21.15.83:0/966016246 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c80108610 0x7f6c8019ee00 secure :-1 s=READY pgs=2838 cs=0 l=1 rev1=1 crypto rx=0x7f6c7402a930 tx=0x7f6c74003b90 comp rx=0 tx=0).stop 2024-04-24T09:17:25.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.491+0000 7f6c87400640 1 -- 172.21.15.83:0/966016246 shutdown_connections 2024-04-24T09:17:25.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.491+0000 7f6c87400640 1 --2- 172.21.15.83:0/966016246 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f6c680a3e20 0x7f6c680a62e0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:25.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.491+0000 7f6c87400640 1 --2- 172.21.15.83:0/966016246 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6c8010e660 0x7f6c801a3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:25.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.491+0000 7f6c87400640 1 --2- 172.21.15.83:0/966016246 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c80108fd0 0x7f6c8019f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:25.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.491+0000 7f6c87400640 1 --2- 172.21.15.83:0/966016246 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c80108610 0x7f6c8019ee00 unknown :-1 s=CLOSED pgs=2838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:25.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.491+0000 7f6c87400640 1 -- 172.21.15.83:0/966016246 >> 172.21.15.83:0/966016246 conn(0x7f6c80076540 msgr2=0x7f6c80077d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:25.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.492+0000 7f6c87400640 1 -- 172.21.15.83:0/966016246 shutdown_connections 2024-04-24T09:17:25.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.492+0000 7f6c87400640 1 -- 172.21.15.83:0/966016246 wait complete. 2024-04-24T09:17:25.522 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T09:17:25.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.774+0000 7fe53ca00640 1 -- 172.21.15.83:0/3362447592 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe538101770 msgr2=0x7fe538105c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:25.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.774+0000 7fe53ca00640 1 --2- 172.21.15.83:0/3362447592 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe538101770 0x7fe538105c30 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fe524009860 tx=0x7fe52402a3d0 comp rx=0 tx=0).stop 2024-04-24T09:17:25.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.775+0000 7fe53ca00640 1 -- 172.21.15.83:0/3362447592 shutdown_connections 2024-04-24T09:17:25.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.775+0000 7fe53ca00640 1 --2- 172.21.15.83:0/3362447592 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe538106250 0x7fe5381066e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:25.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.775+0000 7fe53ca00640 1 --2- 172.21.15.83:0/3362447592 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe538101770 0x7fe538105c30 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:25.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.775+0000 7fe53ca00640 1 --2- 172.21.15.83:0/3362447592 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe538100e50 0x7fe538101230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:25.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.775+0000 7fe53ca00640 1 -- 172.21.15.83:0/3362447592 >> 172.21.15.83:0/3362447592 conn(0x7fe5380fc880 msgr2=0x7fe5380feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:25.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.775+0000 7fe53ca00640 1 -- 172.21.15.83:0/3362447592 shutdown_connections 2024-04-24T09:17:25.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.775+0000 7fe53ca00640 1 -- 172.21.15.83:0/3362447592 wait complete. 2024-04-24T09:17:25.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.776+0000 7fe53ca00640 1 Processor -- start 2024-04-24T09:17:25.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.776+0000 7fe53ca00640 1 -- start start 2024-04-24T09:17:25.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.777+0000 7fe53ca00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe538100e50 0x7fe53819edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:25.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.777+0000 7fe53ca00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe538101770 0x7fe53819f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:25.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.777+0000 7fe53ca00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe538106250 0x7fe5381a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:25.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.777+0000 7fe53ca00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe5381186f0 con 0x7fe538101770 2024-04-24T09:17:25.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.777+0000 7fe53ca00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe538118570 con 0x7fe538106250 2024-04-24T09:17:25.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.777+0000 7fe53ca00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe538118870 con 0x7fe538100e50 2024-04-24T09:17:25.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.777+0000 7fe536a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe538101770 0x7fe53819f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:25.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.777+0000 7fe536a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe538101770 0x7fe53819f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35438/0 (socket says 172.21.15.83:35438) 2024-04-24T09:17:25.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.777+0000 7fe536a00640 1 -- 172.21.15.83:0/863227789 learned_addr learned my addr 172.21.15.83:0/863227789 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:17:25.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.777+0000 7fe537e00640 1 --2- 172.21.15.83:0/863227789 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe538106250 0x7fe5381a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:25.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.778+0000 7fe536a00640 1 -- 172.21.15.83:0/863227789 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe538100e50 msgr2=0x7fe53819edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:25.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.778+0000 7fe536a00640 1 --2- 172.21.15.83:0/863227789 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe538100e50 0x7fe53819edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:25.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.778+0000 7fe536a00640 1 -- 172.21.15.83:0/863227789 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe538106250 msgr2=0x7fe5381a3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:25.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.778+0000 7fe536a00640 1 --2- 172.21.15.83:0/863227789 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe538106250 0x7fe5381a3690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:25.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.778+0000 7fe536a00640 1 -- 172.21.15.83:0/863227789 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5381a3dc0 con 0x7fe538101770 2024-04-24T09:17:25.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.778+0000 7fe537e00640 1 --2- 172.21.15.83:0/863227789 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe538106250 0x7fe5381a3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:17:25.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.778+0000 7fe536a00640 1 --2- 172.21.15.83:0/863227789 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe538101770 0x7fe53819f310 secure :-1 s=READY pgs=2841 cs=0 l=1 rev1=1 crypto rx=0x7fe524000c00 tx=0x7fe524030790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:25.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.779+0000 7fe52fe00640 1 -- 172.21.15.83:0/863227789 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe524004460 con 0x7fe538101770 2024-04-24T09:17:25.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.779+0000 7fe52fe00640 1 -- 172.21.15.83:0/863227789 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe524005570 con 0x7fe538101770 2024-04-24T09:17:25.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.779+0000 7fe53ca00640 1 -- 172.21.15.83:0/863227789 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5381a4050 con 0x7fe538101770 2024-04-24T09:17:25.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.779+0000 7fe52fe00640 1 -- 172.21.15.83:0/863227789 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe524004920 con 0x7fe538101770 2024-04-24T09:17:25.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.779+0000 7fe53ca00640 1 -- 172.21.15.83:0/863227789 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5381ab940 con 0x7fe538101770 2024-04-24T09:17:25.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.784+0000 7fe540e4e740 1 Processor -- start 2024-04-24T09:17:25.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.784+0000 7fe540e4e740 1 -- start start 2024-04-24T09:17:25.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.784+0000 7fe540e4e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ed5688b40 0x564ed5688f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:25.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.784+0000 7fe52fe00640 1 -- 172.21.15.83:0/863227789 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe524039040 con 0x7fe538101770 2024-04-24T09:17:25.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.784+0000 7fe540e4e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564ed5689460 0x564ed56935f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:25.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.784+0000 7fe540e4e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564ed5693b30 0x564ed5695f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:25.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.784+0000 7fe540e4e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x564ed55ee1d0 con 0x564ed5688b40 2024-04-24T09:17:25.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.785+0000 7fe540e4e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x564ed5594450 con 0x564ed5693b30 2024-04-24T09:17:25.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.785+0000 7fe540e4e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x564ed55ea190 con 0x564ed5689460 2024-04-24T09:17:25.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.785+0000 7fe537e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ed5688b40 0x564ed5688f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:25.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.785+0000 7fe537e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ed5688b40 0x564ed5688f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35442/0 (socket says 172.21.15.83:35442) 2024-04-24T09:17:25.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.785+0000 7fe537e00640 1 -- 172.21.15.83:0/2434195539 learned_addr learned my addr 172.21.15.83:0/2434195539 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:17:25.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.785+0000 7fe536a00640 1 --2- 172.21.15.83:0/2434195539 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564ed5693b30 0x564ed5695f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:25.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.785+0000 7fe537e00640 1 -- 172.21.15.83:0/2434195539 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564ed5689460 msgr2=0x564ed56935f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:25.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.785+0000 7fe537e00640 1 --2- 172.21.15.83:0/2434195539 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564ed5689460 0x564ed56935f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:25.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.785+0000 7fe537e00640 1 -- 172.21.15.83:0/2434195539 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564ed5693b30 msgr2=0x564ed5695f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:25.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.785+0000 7fe537e00640 1 --2- 172.21.15.83:0/2434195539 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564ed5693b30 0x564ed5695f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:25.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.786+0000 7fe537e00640 1 -- 172.21.15.83:0/2434195539 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ed5651310 con 0x564ed5688b40 2024-04-24T09:17:25.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.786+0000 7fe537e00640 1 --2- 172.21.15.83:0/2434195539 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ed5688b40 0x564ed5688f20 secure :-1 s=READY pgs=2842 cs=0 l=1 rev1=1 crypto rx=0x7fe530009fc0 tx=0x7fe53000ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b82b750b79882d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:25.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.786+0000 7fe52fe00640 1 --2- 172.21.15.83:0/863227789 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe51c0a3da0 0x7fe51c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:25.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.787+0000 7fe52fe00640 1 -- 172.21.15.83:0/863227789 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 9583+0+0 (secure 0 0 0) 0x7fe5240f1910 con 0x7fe538101770 2024-04-24T09:17:25.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.787+0000 7fe523e00640 1 -- 172.21.15.83:0/2434195539 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe53001d070 con 0x564ed5688b40 2024-04-24T09:17:25.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.787+0000 7fe523e00640 1 -- 172.21.15.83:0/2434195539 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe530003e20 con 0x564ed5688b40 2024-04-24T09:17:25.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.787+0000 7fe523e00640 1 -- 172.21.15.83:0/2434195539 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5300046a0 con 0x564ed5688b40 2024-04-24T09:17:25.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.787+0000 7fe537400640 1 --2- 172.21.15.83:0/863227789 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe51c0a3da0 0x7fe51c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:25.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.788+0000 7fe540e4e740 1 -- 172.21.15.83:0/2434195539 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ed5688b40 msgr2=0x564ed5688f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:25.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.788+0000 7fe540e4e740 1 --2- 172.21.15.83:0/2434195539 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ed5688b40 0x564ed5688f20 secure :-1 s=READY pgs=2842 cs=0 l=1 rev1=1 crypto rx=0x7fe530009fc0 tx=0x7fe53000ec30 comp rx=0 tx=0).stop 2024-04-24T09:17:25.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.788+0000 7fe540e4e740 1 -- 172.21.15.83:0/2434195539 shutdown_connections 2024-04-24T09:17:25.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.788+0000 7fe540e4e740 1 --2- 172.21.15.83:0/2434195539 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564ed5693b30 0x564ed5695f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:25.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.788+0000 7fe540e4e740 1 --2- 172.21.15.83:0/2434195539 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564ed5689460 0x564ed56935f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:25.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.788+0000 7fe540e4e740 1 --2- 172.21.15.83:0/2434195539 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ed5688b40 0x564ed5688f20 unknown :-1 s=CLOSED pgs=2842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:25.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.788+0000 7fe540e4e740 1 -- 172.21.15.83:0/2434195539 >> 172.21.15.83:0/2434195539 conn(0x564ed5684e60 msgr2=0x564ed5685240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:25.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.789+0000 7fe537400640 1 --2- 172.21.15.83:0/863227789 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe51c0a3da0 0x7fe51c0a6260 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x564ed568a100 tx=0x7fe528005e50 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:25.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.789+0000 7fe540e4e740 1 -- 172.21.15.83:0/2434195539 shutdown_connections 2024-04-24T09:17:25.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.789+0000 7fe540e4e740 1 -- 172.21.15.83:0/2434195539 wait complete. 2024-04-24T09:17:25.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.790+0000 7fe540e4e740 1 -- start start 2024-04-24T09:17:25.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.790+0000 7fe540e4e740 1 Processor -- start 2024-04-24T09:17:25.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.791+0000 7fe540e4e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564ed5688b40 0x564ed5690930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:25.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.791+0000 7fe540e4e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ed5689460 0x564ed5690e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:25.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.791+0000 7fe540e4e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564ed5693b30 0x564ed56913b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:25.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.791+0000 7fe540e4e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x564ed56a79d0 con 0x564ed5689460 2024-04-24T09:17:25.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.791+0000 7fe540e4e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x564ed55a7810 con 0x564ed5688b40 2024-04-24T09:17:25.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.791+0000 7fe540e4e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x564ed56a7b50 con 0x564ed5693b30 2024-04-24T09:17:25.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.791+0000 7fe540e4e740 10 client.0 _ll_register_callbacks cb 0x564ed5575140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:17:25.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.791+0000 7fe536a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ed5689460 0x564ed5690e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:25.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.791+0000 7fe537e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564ed5693b30 0x564ed56913b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:25.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.791+0000 7fe537e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564ed5693b30 0x564ed56913b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:59168/0 (socket says 172.21.15.83:59168) 2024-04-24T09:17:25.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.791+0000 7fe537e00640 1 -- 172.21.15.83:0/2143565866 learned_addr learned my addr 172.21.15.83:0/2143565866 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:17:25.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.792+0000 7fe537400640 1 --2- 172.21.15.83:0/2143565866 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564ed5688b40 0x564ed5690930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:25.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.792+0000 7fe537400640 1 -- 172.21.15.83:0/2143565866 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564ed5693b30 msgr2=0x564ed56913b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:25.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.792+0000 7fe537400640 1 --2- 172.21.15.83:0/2143565866 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564ed5693b30 0x564ed56913b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:25.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.792+0000 7fe537400640 1 -- 172.21.15.83:0/2143565866 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ed5689460 msgr2=0x564ed5690e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:25.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.792+0000 7fe537400640 1 --2- 172.21.15.83:0/2143565866 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ed5689460 0x564ed5690e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:25.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.792+0000 7fe537400640 1 -- 172.21.15.83:0/2143565866 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ed55b83f0 con 0x564ed5688b40 2024-04-24T09:17:25.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.792+0000 7fe536a00640 1 --2- 172.21.15.83:0/2143565866 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ed5689460 0x564ed5690e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:17:25.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.793+0000 7fe537400640 1 --2- 172.21.15.83:0/2143565866 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564ed5688b40 0x564ed5690930 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fe528005d00 tx=0x7fe528005c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19a5aaa130345325 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:25.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.793+0000 7fe520c00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:17:25.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.793+0000 7fe520c00640 1 -- 172.21.15.83:0/2143565866 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe528021020 con 0x564ed5688b40 2024-04-24T09:17:25.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.793+0000 7fe540e4e740 1 -- 172.21.15.83:0/2143565866 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe530003e20 con 0x564ed5688b40 2024-04-24T09:17:25.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.794+0000 7fe520c00640 1 -- 172.21.15.83:0/2143565866 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(134..134 src has 1..134) ==== 9583+0+0 (secure 0 0 0) 0x7fe52801d640 con 0x564ed5688b40 2024-04-24T09:17:25.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.794+0000 7fe520c00640 1 -- 172.21.15.83:0/2143565866 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7fe528010e40 con 0x564ed5688b40 2024-04-24T09:17:25.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.794+0000 7fe540e4e740 10 client.41327 fetch_fsmap learned FSMap version 70 2024-04-24T09:17:25.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.794+0000 7fe540e4e740 1 -- 172.21.15.83:0/2143565866 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) -- 0x564ed53003f0 con 0x564ed5688b40 2024-04-24T09:17:25.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.795+0000 7fe520c00640 1 -- 172.21.15.83:0/2143565866 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe530003e20 con 0x564ed5688b40 2024-04-24T09:17:25.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.795+0000 7fe520c00640 1 -- 172.21.15.83:0/2143565866 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 70) ==== 8366+0+0 (secure 0 0 0) 0x7fe52800f950 con 0x564ed5688b40 2024-04-24T09:17:25.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.795+0000 7fe540e4e740 10 client.41327 fetch_fsmap finished waiting for FSMap version 70 2024-04-24T09:17:25.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.795+0000 7fe540e4e740 10 client.41327 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:17:25.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.795+0000 7fe540e4e740 20 client.41327 populate_metadata read hostname 'smithi083' 2024-04-24T09:17:25.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.795+0000 7fe540e4e740 1 --2- 172.21.15.83:0/2143565866 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x564ed56b4d70 0x564ed56b7160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:25.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.795+0000 7fe540e4e740 4 client.41327 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:17:25.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.795+0000 7fe540e4e740 1 -- 172.21.15.83:0/2143565866 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564ed55fbbf0 con 0x564ed56b4d70 2024-04-24T09:17:25.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.796+0000 7fe536a00640 1 --2- 172.21.15.83:0/2143565866 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x564ed56b4d70 0x564ed56b7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:25.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.797+0000 7fe536a00640 1 --2- 172.21.15.83:0/2143565866 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x564ed56b4d70 0x564ed56b7160 crc :-1 s=READY pgs=627 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c63a42651cede0e server_cookie=edcc943fed03608f in_seq=0 out_seq=0 2024-04-24T09:17:25.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.797+0000 7fe520c00640 10 client.41327 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:17:25.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.798+0000 7fe520c00640 1 -- 172.21.15.83:0/2143565866 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564ed55fbbf0 con 0x564ed56b4d70 2024-04-24T09:17:25.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.798+0000 7fe520c00640 10 client.41327 handle_command_reply: tid=0 2024-04-24T09:17:25.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.798+0000 7fe540e4e740 1 client.41327 shutdown 2024-04-24T09:17:25.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.799+0000 7fe540e4e740 1 -- 172.21.15.83:0/2143565866 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564ed5688b40 msgr2=0x564ed5690930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:25.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.799+0000 7fe540e4e740 1 --2- 172.21.15.83:0/2143565866 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564ed5688b40 0x564ed5690930 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fe528005d00 tx=0x7fe528005c90 comp rx=0 tx=0).stop 2024-04-24T09:17:25.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.799+0000 7fe540e4e740 1 -- 172.21.15.83:0/2143565866 shutdown_connections 2024-04-24T09:17:25.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.799+0000 7fe540e4e740 1 --2- 172.21.15.83:0/2143565866 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x564ed56b4d70 0x564ed56b7160 crc :-1 s=READY pgs=627 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:25.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.799+0000 7fe540e4e740 1 --2- 172.21.15.83:0/2143565866 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564ed5693b30 0x564ed56913b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:25.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.799+0000 7fe540e4e740 1 --2- 172.21.15.83:0/2143565866 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ed5689460 0x564ed5690e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:25.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.799+0000 7fe520c00640 0 client.41327 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:17:25.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.799+0000 7fe540e4e740 1 --2- 172.21.15.83:0/2143565866 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564ed5688b40 0x564ed5690930 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:25.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.799+0000 7fe540e4e740 1 -- 172.21.15.83:0/2143565866 >> 172.21.15.83:0/2143565866 conn(0x564ed5684e60 msgr2=0x564ed568ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:25.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.799+0000 7fe540e4e740 1 -- 172.21.15.83:0/2143565866 shutdown_connections 2024-04-24T09:17:25.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.799+0000 7fe540e4e740 1 -- 172.21.15.83:0/2143565866 wait complete. 2024-04-24T09:17:25.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.799+0000 7fe540e4e740 20 client.41327 trim_cache size 0 max 16384 2024-04-24T09:17:25.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.825+0000 7fe540e4e740 1 Processor -- start 2024-04-24T09:17:25.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.825+0000 7fe540e4e740 1 -- start start 2024-04-24T09:17:25.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.825+0000 7fe540e4e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ed5688b40 0x564ed56b0890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:25.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.825+0000 7fe540e4e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564ed5689460 0x564ed56911d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:25.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.825+0000 7fe540e4e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564ed5693b30 0x564ed56bca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:25.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.825+0000 7fe540e4e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe5381af570 con 0x564ed5688b40 2024-04-24T09:17:25.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.825+0000 7fe540e4e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe5381ad330 con 0x564ed5693b30 2024-04-24T09:17:25.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.825+0000 7fe540e4e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe5381aeea0 con 0x564ed5689460 2024-04-24T09:17:25.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.826+0000 7fe537400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ed5688b40 0x564ed56b0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:25.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.826+0000 7fe537400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ed5688b40 0x564ed56b0890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35460/0 (socket says 172.21.15.83:35460) 2024-04-24T09:17:25.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.826+0000 7fe537400640 1 -- 172.21.15.83:0/577999955 learned_addr learned my addr 172.21.15.83:0/577999955 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:17:25.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.826+0000 7fe536a00640 1 --2- 172.21.15.83:0/577999955 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564ed5689460 0x564ed56911d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:25.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.826+0000 7fe537e00640 1 --2- 172.21.15.83:0/577999955 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564ed5693b30 0x564ed56bca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:25.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.826+0000 7fe537400640 1 -- 172.21.15.83:0/577999955 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564ed5689460 msgr2=0x564ed56911d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:25.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.826+0000 7fe537400640 1 --2- 172.21.15.83:0/577999955 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564ed5689460 0x564ed56911d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:25.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.826+0000 7fe537400640 1 -- 172.21.15.83:0/577999955 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564ed5693b30 msgr2=0x564ed56bca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:25.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.826+0000 7fe537400640 1 --2- 172.21.15.83:0/577999955 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564ed5693b30 0x564ed56bca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:25.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.826+0000 7fe537400640 1 -- 172.21.15.83:0/577999955 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ed5689290 con 0x564ed5688b40 2024-04-24T09:17:25.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.827+0000 7fe537400640 1 --2- 172.21.15.83:0/577999955 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ed5688b40 0x564ed56b0890 secure :-1 s=READY pgs=2843 cs=0 l=1 rev1=1 crypto rx=0x7fe52800fda0 tx=0x7fe528005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5b231bd94fa1706 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:25.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.827+0000 7fe523400640 1 -- 172.21.15.83:0/577999955 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe528021040 con 0x564ed5688b40 2024-04-24T09:17:25.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.827+0000 7fe523400640 1 -- 172.21.15.83:0/577999955 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe528016030 con 0x564ed5688b40 2024-04-24T09:17:25.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.827+0000 7fe523400640 1 -- 172.21.15.83:0/577999955 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe528025760 con 0x564ed5688b40 2024-04-24T09:17:25.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.828+0000 7fe540e4e740 1 -- 172.21.15.83:0/577999955 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ed5688b40 msgr2=0x564ed56b0890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:25.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.828+0000 7fe540e4e740 1 --2- 172.21.15.83:0/577999955 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ed5688b40 0x564ed56b0890 secure :-1 s=READY pgs=2843 cs=0 l=1 rev1=1 crypto rx=0x7fe52800fda0 tx=0x7fe528005850 comp rx=0 tx=0).stop 2024-04-24T09:17:25.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.828+0000 7fe540e4e740 1 -- 172.21.15.83:0/577999955 shutdown_connections 2024-04-24T09:17:25.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.828+0000 7fe540e4e740 1 --2- 172.21.15.83:0/577999955 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564ed5693b30 0x564ed56bca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:25.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.828+0000 7fe540e4e740 1 --2- 172.21.15.83:0/577999955 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564ed5689460 0x564ed56911d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:25.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.828+0000 7fe540e4e740 1 --2- 172.21.15.83:0/577999955 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ed5688b40 0x564ed56b0890 unknown :-1 s=CLOSED pgs=2843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:25.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.828+0000 7fe540e4e740 1 -- 172.21.15.83:0/577999955 >> 172.21.15.83:0/577999955 conn(0x564ed5684e60 msgr2=0x564ed5683f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:25.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.829+0000 7fe540e4e740 1 -- 172.21.15.83:0/577999955 shutdown_connections 2024-04-24T09:17:25.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.829+0000 7fe540e4e740 1 -- 172.21.15.83:0/577999955 wait complete. 2024-04-24T09:17:25.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.829+0000 7fe540e4e740 1 -- start start 2024-04-24T09:17:25.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.829+0000 7fe540e4e740 1 Processor -- start 2024-04-24T09:17:25.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.830+0000 7fe540e4e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564ed5688b40 0x564ed5694700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:25.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.830+0000 7fe540e4e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ed5689460 0x564ed5694c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:25.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.830+0000 7fe540e4e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564ed5693b30 0x564ed56abda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:25.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.830+0000 7fe540e4e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe5381ae100 con 0x564ed5689460 2024-04-24T09:17:25.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.830+0000 7fe540e4e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe5381ada00 con 0x564ed5688b40 2024-04-24T09:17:25.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.830+0000 7fe540e4e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe5381ae7d0 con 0x564ed5693b30 2024-04-24T09:17:25.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.830+0000 7fe540e4e740 10 client.0 _ll_register_callbacks cb 0x7fe52800b2d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:17:25.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.830+0000 7fe536a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ed5689460 0x564ed5694c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:25.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.830+0000 7fe536a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ed5689460 0x564ed5694c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35472/0 (socket says 172.21.15.83:35472) 2024-04-24T09:17:25.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.830+0000 7fe536a00640 1 -- 172.21.15.83:0/3587235037 learned_addr learned my addr 172.21.15.83:0/3587235037 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:17:25.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.830+0000 7fe537400640 1 --2- 172.21.15.83:0/3587235037 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564ed5688b40 0x564ed5694700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:25.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.831+0000 7fe537e00640 1 --2- 172.21.15.83:0/3587235037 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564ed5693b30 0x564ed56abda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:25.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.831+0000 7fe536a00640 1 -- 172.21.15.83:0/3587235037 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564ed5693b30 msgr2=0x564ed56abda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:25.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.831+0000 7fe536a00640 1 --2- 172.21.15.83:0/3587235037 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564ed5693b30 0x564ed56abda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:25.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.831+0000 7fe536a00640 1 -- 172.21.15.83:0/3587235037 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564ed5688b40 msgr2=0x564ed5694700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:25.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.831+0000 7fe536a00640 1 --2- 172.21.15.83:0/3587235037 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564ed5688b40 0x564ed5694700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:25.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.831+0000 7fe536a00640 1 -- 172.21.15.83:0/3587235037 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ed53ca7d0 con 0x564ed5689460 2024-04-24T09:17:25.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.831+0000 7fe536a00640 1 --2- 172.21.15.83:0/3587235037 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ed5689460 0x564ed5694c40 secure :-1 s=READY pgs=2844 cs=0 l=1 rev1=1 crypto rx=0x7fe5240bd4b0 tx=0x7fe5240f74d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3e436ac47f5dfac server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:25.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.831+0000 7fe520c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:17:25.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.832+0000 7fe520c00640 1 -- 172.21.15.83:0/3587235037 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5240bce80 con 0x564ed5689460 2024-04-24T09:17:25.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.832+0000 7fe520c00640 1 -- 172.21.15.83:0/3587235037 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(134..134 src has 1..134) ==== 9583+0+0 (secure 0 0 0) 0x7fe524031620 con 0x564ed5689460 2024-04-24T09:17:25.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.832+0000 7fe540e4e740 1 -- 172.21.15.83:0/3587235037 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe528016030 con 0x564ed5689460 2024-04-24T09:17:25.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.832+0000 7fe520c00640 1 -- 172.21.15.83:0/3587235037 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7fe5240bdc20 con 0x564ed5689460 2024-04-24T09:17:25.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.832+0000 7fe540e4e740 10 client.24276 fetch_fsmap learned FSMap version 70 2024-04-24T09:17:25.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.832+0000 7fe540e4e740 1 -- 172.21.15.83:0/3587235037 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) -- 0x564ed56ad3e0 con 0x564ed5689460 2024-04-24T09:17:25.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.833+0000 7fe520c00640 1 -- 172.21.15.83:0/3587235037 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe5240a2e50 con 0x564ed5689460 2024-04-24T09:17:25.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.833+0000 7fe520c00640 1 -- 172.21.15.83:0/3587235037 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 70) ==== 8366+0+0 (secure 0 0 0) 0x7fe52404c030 con 0x564ed5689460 2024-04-24T09:17:25.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.833+0000 7fe540e4e740 10 client.24276 fetch_fsmap finished waiting for FSMap version 70 2024-04-24T09:17:25.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.833+0000 7fe540e4e740 10 client.24276 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:17:25.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.833+0000 7fe540e4e740 20 client.24276 populate_metadata read hostname 'smithi083' 2024-04-24T09:17:25.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.833+0000 7fe540e4e740 1 --2- 172.21.15.83:0/3587235037 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x564ed571b5a0 0x564ed571d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:25.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.833+0000 7fe540e4e740 4 client.24276 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:17:25.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.833+0000 7fe540e4e740 1 -- 172.21.15.83:0/3587235037 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564ed55f5a60 con 0x564ed571b5a0 2024-04-24T09:17:25.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.834+0000 7fe537400640 1 --2- 172.21.15.83:0/3587235037 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x564ed571b5a0 0x564ed571d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:25.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.835+0000 7fe537400640 1 --2- 172.21.15.83:0/3587235037 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x564ed571b5a0 0x564ed571d9c0 crc :-1 s=READY pgs=628 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6274362ced521b98 server_cookie=87658bb499da161a in_seq=0 out_seq=0 2024-04-24T09:17:25.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.835+0000 7fe520c00640 10 client.24276 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:17:25.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.835+0000 7fe520c00640 1 -- 172.21.15.83:0/3587235037 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564ed55f5a60 con 0x564ed571b5a0 2024-04-24T09:17:25.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.835+0000 7fe520c00640 10 client.24276 handle_command_reply: tid=0 2024-04-24T09:17:25.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.835+0000 7fe540e4e740 1 client.24276 shutdown 2024-04-24T09:17:25.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.836+0000 7fe540e4e740 1 -- 172.21.15.83:0/3587235037 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ed5689460 msgr2=0x564ed5694c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:25.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.836+0000 7fe540e4e740 1 --2- 172.21.15.83:0/3587235037 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ed5689460 0x564ed5694c40 secure :-1 s=READY pgs=2844 cs=0 l=1 rev1=1 crypto rx=0x7fe5240bd4b0 tx=0x7fe5240f74d0 comp rx=0 tx=0).stop 2024-04-24T09:17:25.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.836+0000 7fe540e4e740 1 -- 172.21.15.83:0/3587235037 shutdown_connections 2024-04-24T09:17:25.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.836+0000 7fe540e4e740 1 --2- 172.21.15.83:0/3587235037 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x564ed571b5a0 0x564ed571d9c0 crc :-1 s=READY pgs=628 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:25.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.836+0000 7fe520c00640 0 client.24276 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:17:25.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.836+0000 7fe540e4e740 1 --2- 172.21.15.83:0/3587235037 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564ed5693b30 0x564ed56abda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:25.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.836+0000 7fe540e4e740 1 --2- 172.21.15.83:0/3587235037 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ed5689460 0x564ed5694c40 unknown :-1 s=CLOSED pgs=2844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:25.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.836+0000 7fe540e4e740 1 --2- 172.21.15.83:0/3587235037 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564ed5688b40 0x564ed5694700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:25.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.836+0000 7fe540e4e740 1 -- 172.21.15.83:0/3587235037 >> 172.21.15.83:0/3587235037 conn(0x564ed5684e60 msgr2=0x564ed568a290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:25.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.836+0000 7fe540e4e740 1 -- 172.21.15.83:0/3587235037 shutdown_connections 2024-04-24T09:17:25.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.837+0000 7fe540e4e740 1 -- 172.21.15.83:0/3587235037 wait complete. 2024-04-24T09:17:25.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.837+0000 7fe540e4e740 20 client.24276 trim_cache size 0 max 16384 2024-04-24T09:17:25.838 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:17:25.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.837+0000 7fe53ca00640 1 -- 172.21.15.83:0/863227789 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe51c0a3da0 msgr2=0x7fe51c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:25.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.838+0000 7fe53ca00640 1 --2- 172.21.15.83:0/863227789 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe51c0a3da0 0x7fe51c0a6260 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x564ed568a100 tx=0x7fe528005e50 comp rx=0 tx=0).stop 2024-04-24T09:17:25.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.838+0000 7fe53ca00640 1 -- 172.21.15.83:0/863227789 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe538101770 msgr2=0x7fe53819f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:25.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.838+0000 7fe53ca00640 1 --2- 172.21.15.83:0/863227789 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe538101770 0x7fe53819f310 secure :-1 s=READY pgs=2841 cs=0 l=1 rev1=1 crypto rx=0x7fe524000c00 tx=0x7fe524030790 comp rx=0 tx=0).stop 2024-04-24T09:17:25.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.838+0000 7fe53ca00640 1 -- 172.21.15.83:0/863227789 shutdown_connections 2024-04-24T09:17:25.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.838+0000 7fe53ca00640 1 --2- 172.21.15.83:0/863227789 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe51c0a3da0 0x7fe51c0a6260 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:25.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.838+0000 7fe53ca00640 1 --2- 172.21.15.83:0/863227789 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe538106250 0x7fe5381a3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:25.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.838+0000 7fe53ca00640 1 --2- 172.21.15.83:0/863227789 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe538101770 0x7fe53819f310 unknown :-1 s=CLOSED pgs=2841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:25.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.838+0000 7fe53ca00640 1 --2- 172.21.15.83:0/863227789 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe538100e50 0x7fe53819edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:25.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.838+0000 7fe53ca00640 1 -- 172.21.15.83:0/863227789 >> 172.21.15.83:0/863227789 conn(0x7fe5380fc880 msgr2=0x7fe5380fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:25.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.838+0000 7fe53ca00640 1 -- 172.21.15.83:0/863227789 shutdown_connections 2024-04-24T09:17:25.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:25.838+0000 7fe53ca00640 1 -- 172.21.15.83:0/863227789 wait complete. 2024-04-24T09:17:26.230 INFO:tasks.workunit.client.0.smithi083.stdout: 10 87201 6.48 MB/sec execute 597 sec latency 111.135 ms 2024-04-24T09:17:26.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:17:26 smithi138 ceph-mon[31441]: pgmap v817: 129 pgs: 129 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.2 MiB/s wr, 229 op/s 2024-04-24T09:17:26.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:17:26 smithi189 ceph-mon[31791]: pgmap v817: 129 pgs: 129 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.2 MiB/s wr, 229 op/s 2024-04-24T09:17:26.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:17:26 smithi083 ceph-mon[27907]: pgmap v817: 129 pgs: 129 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.2 MiB/s wr, 229 op/s 2024-04-24T09:17:26.860 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T09:17:26.861 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T09:17:26.861 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T09:17:27.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.113+0000 7f2172200640 1 -- 172.21.15.83:0/1953631684 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f216c104cd0 msgr2=0x7f216c108d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:27.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.113+0000 7f2172200640 1 --2- 172.21.15.83:0/1953631684 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f216c104cd0 0x7f216c108d10 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f2158009900 tx=0x7f215802a400 comp rx=0 tx=0).stop 2024-04-24T09:17:27.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.114+0000 7f2172200640 1 -- 172.21.15.83:0/1953631684 shutdown_connections 2024-04-24T09:17:27.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.114+0000 7f2172200640 1 --2- 172.21.15.83:0/1953631684 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f216c1093f0 0x7f216c110fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:27.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.114+0000 7f2172200640 1 --2- 172.21.15.83:0/1953631684 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f216c104cd0 0x7f216c108d10 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:27.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.114+0000 7f2172200640 1 --2- 172.21.15.83:0/1953631684 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f216c104320 0x7f216c104700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:27.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.114+0000 7f2172200640 1 -- 172.21.15.83:0/1953631684 >> 172.21.15.83:0/1953631684 conn(0x7f216c0ffb40 msgr2=0x7f216c101f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:27.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.114+0000 7f2172200640 1 -- 172.21.15.83:0/1953631684 shutdown_connections 2024-04-24T09:17:27.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.114+0000 7f2172200640 1 -- 172.21.15.83:0/1953631684 wait complete. 2024-04-24T09:17:27.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.115+0000 7f2172200640 1 Processor -- start 2024-04-24T09:17:27.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.115+0000 7f2172200640 1 -- start start 2024-04-24T09:17:27.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.116+0000 7f2172200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f216c104320 0x7f216c1a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:27.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.116+0000 7f2172200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f216c1093f0 0x7f216c1a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:27.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.116+0000 7f2172200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f216c19a7d0 0x7f216c19ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:27.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.116+0000 7f2172200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f216c114310 con 0x7f216c1093f0 2024-04-24T09:17:27.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.116+0000 7f2172200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f216c114190 con 0x7f216c19a7d0 2024-04-24T09:17:27.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.116+0000 7f2172200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f216c114490 con 0x7f216c104320 2024-04-24T09:17:27.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.117+0000 7f2171800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f216c19a7d0 0x7f216c19ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:27.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.117+0000 7f216be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f216c1093f0 0x7f216c1a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:27.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.117+0000 7f2171800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f216c19a7d0 0x7f216c19ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:39486/0 (socket says 172.21.15.83:39486) 2024-04-24T09:17:27.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.117+0000 7f2171800640 1 -- 172.21.15.83:0/1658779458 learned_addr learned my addr 172.21.15.83:0/1658779458 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:17:27.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.117+0000 7f2170e00640 1 --2- 172.21.15.83:0/1658779458 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f216c104320 0x7f216c1a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:27.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.117+0000 7f216be00640 1 -- 172.21.15.83:0/1658779458 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f216c104320 msgr2=0x7f216c1a0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:27.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.117+0000 7f216be00640 1 --2- 172.21.15.83:0/1658779458 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f216c104320 0x7f216c1a0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:27.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.117+0000 7f216be00640 1 -- 172.21.15.83:0/1658779458 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f216c19a7d0 msgr2=0x7f216c19ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:27.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.117+0000 7f216be00640 1 --2- 172.21.15.83:0/1658779458 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f216c19a7d0 0x7f216c19ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:27.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.117+0000 7f216be00640 1 -- 172.21.15.83:0/1658779458 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f216c19b360 con 0x7f216c1093f0 2024-04-24T09:17:27.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.118+0000 7f216be00640 1 --2- 172.21.15.83:0/1658779458 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f216c1093f0 0x7f216c1a0b50 secure :-1 s=READY pgs=2845 cs=0 l=1 rev1=1 crypto rx=0x7f215802a920 tx=0x7f2158004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:27.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.118+0000 7f2169600640 1 -- 172.21.15.83:0/1658779458 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f215803f070 con 0x7f216c1093f0 2024-04-24T09:17:27.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.118+0000 7f2172200640 1 -- 172.21.15.83:0/1658779458 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f216c19b5f0 con 0x7f216c1093f0 2024-04-24T09:17:27.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.118+0000 7f2169600640 1 -- 172.21.15.83:0/1658779458 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2158003ca0 con 0x7f216c1093f0 2024-04-24T09:17:27.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.118+0000 7f2169600640 1 -- 172.21.15.83:0/1658779458 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2158035070 con 0x7f216c1093f0 2024-04-24T09:17:27.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.118+0000 7f2172200640 1 -- 172.21.15.83:0/1658779458 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f216c076760 con 0x7f216c1093f0 2024-04-24T09:17:27.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.122+0000 7f21765fd740 1 Processor -- start 2024-04-24T09:17:27.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.123+0000 7f21765fd740 1 -- start start 2024-04-24T09:17:27.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.123+0000 7f21765fd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570bf22d270 0x5570bf22d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:27.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.123+0000 7f21765fd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5570bf22db90 0x5570bf237d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:27.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.123+0000 7f21765fd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570bf238260 0x5570bf23a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:27.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.123+0000 7f21765fd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5570bf192e70 con 0x5570bf238260 2024-04-24T09:17:27.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.123+0000 7f21765fd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5570bf139380 con 0x5570bf22d270 2024-04-24T09:17:27.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.123+0000 7f21765fd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5570bf18ee30 con 0x5570bf22db90 2024-04-24T09:17:27.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.124+0000 7f2171800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570bf22d270 0x5570bf22d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:27.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.124+0000 7f2171800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570bf22d270 0x5570bf22d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:39492/0 (socket says 172.21.15.83:39492) 2024-04-24T09:17:27.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.124+0000 7f2171800640 1 -- 172.21.15.83:0/32422352 learned_addr learned my addr 172.21.15.83:0/32422352 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:17:27.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.124+0000 7f2169600640 1 -- 172.21.15.83:0/1658779458 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2158043400 con 0x7f216c1093f0 2024-04-24T09:17:27.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.124+0000 7f2171800640 1 -- 172.21.15.83:0/32422352 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5570bf22db90 msgr2=0x5570bf237d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:27.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.124+0000 7f2171800640 1 --2- 172.21.15.83:0/32422352 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5570bf22db90 0x5570bf237d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:27.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.124+0000 7f2171800640 1 -- 172.21.15.83:0/32422352 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570bf238260 msgr2=0x5570bf23a650 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T09:17:27.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.125+0000 7f216be00640 1 --2- 172.21.15.83:0/32422352 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570bf238260 0x5570bf23a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:27.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.125+0000 7f2171800640 1 --2- 172.21.15.83:0/32422352 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570bf238260 0x5570bf23a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:27.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.125+0000 7f2171800640 1 -- 172.21.15.83:0/32422352 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570bef6d7e0 con 0x5570bf22d270 2024-04-24T09:17:27.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.125+0000 7f216be00640 1 --2- 172.21.15.83:0/32422352 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570bf238260 0x5570bf23a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:17:27.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.126+0000 7f2171800640 1 --2- 172.21.15.83:0/32422352 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570bf22d270 0x5570bf22d650 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f2164009d20 tx=0x7f2164011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9aabe0afa9ab6d13 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:27.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.126+0000 7f215e000640 1 -- 172.21.15.83:0/32422352 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f216401e070 con 0x5570bf22d270 2024-04-24T09:17:27.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.126+0000 7f215e000640 1 -- 172.21.15.83:0/32422352 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2164019ae0 con 0x5570bf22d270 2024-04-24T09:17:27.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.126+0000 7f215e000640 1 -- 172.21.15.83:0/32422352 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f216400fa20 con 0x5570bf22d270 2024-04-24T09:17:27.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.127+0000 7f2169600640 1 --2- 172.21.15.83:0/1658779458 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f21540a3e20 0x7f21540a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:27.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.127+0000 7f2169600640 1 -- 172.21.15.83:0/1658779458 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 9583+0+0 (secure 0 0 0) 0x7f21580f1130 con 0x7f216c1093f0 2024-04-24T09:17:27.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.127+0000 7f2170e00640 1 --2- 172.21.15.83:0/1658779458 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f21540a3e20 0x7f21540a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:27.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.128+0000 7f21765fd740 1 -- 172.21.15.83:0/32422352 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570bf22d270 msgr2=0x5570bf22d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:27.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.128+0000 7f21765fd740 1 --2- 172.21.15.83:0/32422352 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570bf22d270 0x5570bf22d650 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f2164009d20 tx=0x7f2164011810 comp rx=0 tx=0).stop 2024-04-24T09:17:27.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.128+0000 7f21765fd740 1 -- 172.21.15.83:0/32422352 shutdown_connections 2024-04-24T09:17:27.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.128+0000 7f21765fd740 1 --2- 172.21.15.83:0/32422352 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570bf238260 0x5570bf23a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:27.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.128+0000 7f21765fd740 1 --2- 172.21.15.83:0/32422352 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5570bf22db90 0x5570bf237d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:27.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.128+0000 7f21765fd740 1 --2- 172.21.15.83:0/32422352 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570bf22d270 0x5570bf22d650 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:27.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.128+0000 7f21765fd740 1 -- 172.21.15.83:0/32422352 >> 172.21.15.83:0/32422352 conn(0x5570bf229590 msgr2=0x5570bf229970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:27.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.128+0000 7f2170e00640 1 --2- 172.21.15.83:0/1658779458 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f21540a3e20 0x7f21540a62e0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f2160005fd0 tx=0x7f2160008230 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:27.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.128+0000 7f21765fd740 1 -- 172.21.15.83:0/32422352 shutdown_connections 2024-04-24T09:17:27.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.128+0000 7f21765fd740 1 -- 172.21.15.83:0/32422352 wait complete. 2024-04-24T09:17:27.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.129+0000 7f21765fd740 1 -- start start 2024-04-24T09:17:27.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.129+0000 7f21765fd740 1 Processor -- start 2024-04-24T09:17:27.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.131+0000 7f21765fd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570bf22d270 0x5570bf2350c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:27.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.131+0000 7f21765fd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570bf22db90 0x5570bf235600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:27.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.131+0000 7f21765fd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5570bf238260 0x5570bf235b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:27.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.131+0000 7f21765fd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5570bf23b050 con 0x5570bf22d270 2024-04-24T09:17:27.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.131+0000 7f21765fd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5570bf14c660 con 0x5570bf22db90 2024-04-24T09:17:27.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.131+0000 7f21765fd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5570bf23b1d0 con 0x5570bf238260 2024-04-24T09:17:27.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.131+0000 7f21765fd740 10 client.0 _ll_register_callbacks cb 0x5570bf13aaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:17:27.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.131+0000 7f216be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570bf22db90 0x5570bf235600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:27.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.132+0000 7f2170e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570bf22d270 0x5570bf2350c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:27.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.132+0000 7f216be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570bf22db90 0x5570bf235600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:39494/0 (socket says 172.21.15.83:39494) 2024-04-24T09:17:27.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.132+0000 7f216be00640 1 -- 172.21.15.83:0/2042802783 learned_addr learned my addr 172.21.15.83:0/2042802783 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:17:27.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.132+0000 7f2170e00640 1 -- 172.21.15.83:0/2042802783 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5570bf238260 msgr2=0x5570bf235b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:27.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.132+0000 7f2170e00640 1 --2- 172.21.15.83:0/2042802783 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5570bf238260 0x5570bf235b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:27.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.132+0000 7f2170e00640 1 -- 172.21.15.83:0/2042802783 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570bf22db90 msgr2=0x5570bf235600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:27.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.132+0000 7f2170e00640 1 --2- 172.21.15.83:0/2042802783 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570bf22db90 0x5570bf235600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:27.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.132+0000 7f2170e00640 1 -- 172.21.15.83:0/2042802783 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570befcdf70 con 0x5570bf22d270 2024-04-24T09:17:27.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.132+0000 7f216be00640 1 --2- 172.21.15.83:0/2042802783 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570bf22db90 0x5570bf235600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:17:27.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.133+0000 7f2170e00640 1 --2- 172.21.15.83:0/2042802783 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570bf22d270 0x5570bf2350c0 secure :-1 s=READY pgs=2846 cs=0 l=1 rev1=1 crypto rx=0x7f2160005c40 tx=0x7f2160004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5be1f4cd2fd4177 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:27.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.133+0000 7f2152a00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:17:27.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.133+0000 7f2152a00640 1 -- 172.21.15.83:0/2042802783 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2160020070 con 0x5570bf22d270 2024-04-24T09:17:27.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.133+0000 7f2152a00640 1 -- 172.21.15.83:0/2042802783 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(134..134 src has 1..134) ==== 9583+0+0 (secure 0 0 0) 0x7f216000ec80 con 0x5570bf22d270 2024-04-24T09:17:27.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.133+0000 7f21765fd740 1 -- 172.21.15.83:0/2042802783 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2164019ae0 con 0x5570bf22d270 2024-04-24T09:17:27.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.134+0000 7f2152a00640 1 -- 172.21.15.83:0/2042802783 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7f216002c050 con 0x5570bf22d270 2024-04-24T09:17:27.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.134+0000 7f21765fd740 10 client.24285 fetch_fsmap learned FSMap version 70 2024-04-24T09:17:27.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.134+0000 7f21765fd740 1 -- 172.21.15.83:0/2042802783 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) -- 0x5570bee65170 con 0x5570bf22d270 2024-04-24T09:17:27.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.134+0000 7f2152a00640 1 -- 172.21.15.83:0/2042802783 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2164019ae0 con 0x5570bf22d270 2024-04-24T09:17:27.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.135+0000 7f2152a00640 1 -- 172.21.15.83:0/2042802783 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 70) ==== 8366+0+0 (secure 0 0 0) 0x7f21600243b0 con 0x5570bf22d270 2024-04-24T09:17:27.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.135+0000 7f21765fd740 10 client.24285 fetch_fsmap finished waiting for FSMap version 70 2024-04-24T09:17:27.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.135+0000 7f21765fd740 10 client.24285 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:17:27.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.135+0000 7f21765fd740 20 client.24285 populate_metadata read hostname 'smithi083' 2024-04-24T09:17:27.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.135+0000 7f21765fd740 1 --2- 172.21.15.83:0/2042802783 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5570bf2484b0 0x5570bf24a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:27.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.135+0000 7f21765fd740 4 client.24285 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:17:27.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.135+0000 7f21765fd740 1 -- 172.21.15.83:0/2042802783 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5570bf1a0890 con 0x5570bf2484b0 2024-04-24T09:17:27.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.135+0000 7f216be00640 1 --2- 172.21.15.83:0/2042802783 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5570bf2484b0 0x5570bf24a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:27.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.136+0000 7f216be00640 1 --2- 172.21.15.83:0/2042802783 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5570bf2484b0 0x5570bf24a930 crc :-1 s=READY pgs=629 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e01840803007dd00 server_cookie=45c981c449a81cd6 in_seq=0 out_seq=0 2024-04-24T09:17:27.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.136+0000 7f2152a00640 10 client.24285 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:17:27.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.137+0000 7f2152a00640 1 -- 172.21.15.83:0/2042802783 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5570bf1a0890 con 0x5570bf2484b0 2024-04-24T09:17:27.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.137+0000 7f2152a00640 10 client.24285 handle_command_reply: tid=0 2024-04-24T09:17:27.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.137+0000 7f21765fd740 1 client.24285 shutdown 2024-04-24T09:17:27.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.138+0000 7f21765fd740 1 -- 172.21.15.83:0/2042802783 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570bf22d270 msgr2=0x5570bf2350c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:27.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.138+0000 7f21765fd740 1 --2- 172.21.15.83:0/2042802783 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570bf22d270 0x5570bf2350c0 secure :-1 s=READY pgs=2846 cs=0 l=1 rev1=1 crypto rx=0x7f2160005c40 tx=0x7f2160004ab0 comp rx=0 tx=0).stop 2024-04-24T09:17:27.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.138+0000 7f21765fd740 1 -- 172.21.15.83:0/2042802783 shutdown_connections 2024-04-24T09:17:27.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.138+0000 7f21765fd740 1 --2- 172.21.15.83:0/2042802783 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5570bf2484b0 0x5570bf24a930 crc :-1 s=READY pgs=629 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:27.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.138+0000 7f21765fd740 1 --2- 172.21.15.83:0/2042802783 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5570bf238260 0x5570bf235b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:27.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.138+0000 7f21765fd740 1 --2- 172.21.15.83:0/2042802783 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570bf22db90 0x5570bf235600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:27.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.138+0000 7f21765fd740 1 --2- 172.21.15.83:0/2042802783 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570bf22d270 0x5570bf2350c0 unknown :-1 s=CLOSED pgs=2846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:27.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.138+0000 7f21765fd740 1 -- 172.21.15.83:0/2042802783 >> 172.21.15.83:0/2042802783 conn(0x5570bf229590 msgr2=0x5570bf22f290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:27.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.138+0000 7f2152a00640 0 client.24285 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:17:27.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.138+0000 7f21765fd740 1 -- 172.21.15.83:0/2042802783 shutdown_connections 2024-04-24T09:17:27.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.139+0000 7f21765fd740 1 -- 172.21.15.83:0/2042802783 wait complete. 2024-04-24T09:17:27.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.139+0000 7f21765fd740 20 client.24285 trim_cache size 0 max 16384 2024-04-24T09:17:27.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.176+0000 7f21765fd740 1 Processor -- start 2024-04-24T09:17:27.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.176+0000 7f21765fd740 1 -- start start 2024-04-24T09:17:27.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.177+0000 7f21765fd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5570bf22d270 0x5570bf243fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:27.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.177+0000 7f21765fd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570bf22db90 0x5570bf22f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:27.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.177+0000 7f21765fd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570bf238260 0x5570bf228570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:27.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.177+0000 7f21765fd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f214c0101d0 con 0x5570bf22db90 2024-04-24T09:17:27.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.177+0000 7f21765fd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f214c011d40 con 0x5570bf238260 2024-04-24T09:17:27.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.177+0000 7f21765fd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f214c0108a0 con 0x5570bf22d270 2024-04-24T09:17:27.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.178+0000 7f2171800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570bf238260 0x5570bf228570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:27.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.178+0000 7f2171800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570bf238260 0x5570bf228570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:39506/0 (socket says 172.21.15.83:39506) 2024-04-24T09:17:27.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.178+0000 7f2171800640 1 -- 172.21.15.83:0/3689306056 learned_addr learned my addr 172.21.15.83:0/3689306056 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:17:27.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.178+0000 7f2171800640 1 -- 172.21.15.83:0/3689306056 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5570bf22d270 msgr2=0x5570bf243fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-24T09:17:27.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.178+0000 7f2171800640 1 --2- 172.21.15.83:0/3689306056 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5570bf22d270 0x5570bf243fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:27.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.178+0000 7f2171800640 1 -- 172.21.15.83:0/3689306056 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570bf22db90 msgr2=0x5570bf22f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:27.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.178+0000 7f2171800640 1 --2- 172.21.15.83:0/3689306056 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570bf22db90 0x5570bf22f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:27.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.178+0000 7f2171800640 1 -- 172.21.15.83:0/3689306056 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570bf238090 con 0x5570bf238260 2024-04-24T09:17:27.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.179+0000 7f2171800640 1 --2- 172.21.15.83:0/3689306056 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570bf238260 0x5570bf228570 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f216401c040 tx=0x7f2164005040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fbb348c90015b96b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:27.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.179+0000 7f215d600640 1 -- 172.21.15.83:0/3689306056 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f216401e070 con 0x5570bf238260 2024-04-24T09:17:27.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.180+0000 7f215d600640 1 -- 172.21.15.83:0/3689306056 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2164028070 con 0x5570bf238260 2024-04-24T09:17:27.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.180+0000 7f215d600640 1 -- 172.21.15.83:0/3689306056 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21640236a0 con 0x5570bf238260 2024-04-24T09:17:27.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.181+0000 7f21765fd740 1 -- 172.21.15.83:0/3689306056 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570bf238260 msgr2=0x5570bf228570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:27.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.181+0000 7f21765fd740 1 --2- 172.21.15.83:0/3689306056 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570bf238260 0x5570bf228570 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f216401c040 tx=0x7f2164005040 comp rx=0 tx=0).stop 2024-04-24T09:17:27.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.181+0000 7f21765fd740 1 -- 172.21.15.83:0/3689306056 shutdown_connections 2024-04-24T09:17:27.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.181+0000 7f21765fd740 1 --2- 172.21.15.83:0/3689306056 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570bf238260 0x5570bf228570 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:27.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.181+0000 7f21765fd740 1 --2- 172.21.15.83:0/3689306056 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570bf22db90 0x5570bf22f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:27.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.181+0000 7f21765fd740 1 --2- 172.21.15.83:0/3689306056 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5570bf22d270 0x5570bf243fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:27.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.181+0000 7f21765fd740 1 -- 172.21.15.83:0/3689306056 >> 172.21.15.83:0/3689306056 conn(0x5570bf229590 msgr2=0x5570bf227e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:27.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.181+0000 7f21765fd740 1 -- 172.21.15.83:0/3689306056 shutdown_connections 2024-04-24T09:17:27.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.181+0000 7f21765fd740 1 -- 172.21.15.83:0/3689306056 wait complete. 2024-04-24T09:17:27.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.182+0000 7f21765fd740 1 -- start start 2024-04-24T09:17:27.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.182+0000 7f21765fd740 1 Processor -- start 2024-04-24T09:17:27.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.183+0000 7f21765fd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5570bf22d270 0x5570bf239410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:27.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.183+0000 7f21765fd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570bf22db90 0x5570bf239950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:27.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.183+0000 7f21765fd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570bf238260 0x5570bf23e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:27.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.183+0000 7f21765fd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f214c011670 con 0x5570bf22db90 2024-04-24T09:17:27.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.183+0000 7f21765fd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f214c010fa0 con 0x5570bf238260 2024-04-24T09:17:27.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.183+0000 7f21765fd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f214c00e1d0 con 0x5570bf22d270 2024-04-24T09:17:27.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.183+0000 7f21765fd740 10 client.0 _ll_register_callbacks cb 0x7f216000faa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:17:27.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.184+0000 7f216be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570bf22db90 0x5570bf239950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:27.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.184+0000 7f216be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570bf22db90 0x5570bf239950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35522/0 (socket says 172.21.15.83:35522) 2024-04-24T09:17:27.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.184+0000 7f216be00640 1 -- 172.21.15.83:0/3835164143 learned_addr learned my addr 172.21.15.83:0/3835164143 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:17:27.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.184+0000 7f2171800640 1 --2- 172.21.15.83:0/3835164143 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570bf238260 0x5570bf23e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:27.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.184+0000 7f2170e00640 1 --2- 172.21.15.83:0/3835164143 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5570bf22d270 0x5570bf239410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:27.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.184+0000 7f2171800640 1 -- 172.21.15.83:0/3835164143 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5570bf22d270 msgr2=0x5570bf239410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:27.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.184+0000 7f2171800640 1 --2- 172.21.15.83:0/3835164143 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5570bf22d270 0x5570bf239410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:27.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.184+0000 7f2171800640 1 -- 172.21.15.83:0/3835164143 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570bf22db90 msgr2=0x5570bf239950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:27.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.184+0000 7f2171800640 1 --2- 172.21.15.83:0/3835164143 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570bf22db90 0x5570bf239950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:27.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.184+0000 7f2171800640 1 -- 172.21.15.83:0/3835164143 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570bf23eee0 con 0x5570bf238260 2024-04-24T09:17:27.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.185+0000 7f2171800640 1 --2- 172.21.15.83:0/3835164143 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570bf238260 0x5570bf23e9a0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f21640050b0 tx=0x7f21640048b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b408d6b0106ce4bc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:27.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.185+0000 7f2152a00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:17:27.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.185+0000 7f2152a00640 1 -- 172.21.15.83:0/3835164143 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2164052070 con 0x5570bf238260 2024-04-24T09:17:27.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.186+0000 7f2152a00640 1 -- 172.21.15.83:0/3835164143 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(134..134 src has 1..134) ==== 9583+0+0 (secure 0 0 0) 0x7f2164023640 con 0x5570bf238260 2024-04-24T09:17:27.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.186+0000 7f21765fd740 1 -- 172.21.15.83:0/3835164143 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2164028070 con 0x5570bf238260 2024-04-24T09:17:27.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.186+0000 7f2152a00640 1 -- 172.21.15.83:0/3835164143 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7f216405c050 con 0x5570bf238260 2024-04-24T09:17:27.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.186+0000 7f21765fd740 10 client.41351 fetch_fsmap learned FSMap version 70 2024-04-24T09:17:27.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.186+0000 7f21765fd740 1 -- 172.21.15.83:0/3835164143 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) -- 0x5570bf2aea70 con 0x5570bf238260 2024-04-24T09:17:27.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.187+0000 7f2152a00640 1 -- 172.21.15.83:0/3835164143 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2164028070 con 0x5570bf238260 2024-04-24T09:17:27.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.187+0000 7f2152a00640 1 -- 172.21.15.83:0/3835164143 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 70) ==== 8366+0+0 (secure 0 0 0) 0x7f2164057440 con 0x5570bf238260 2024-04-24T09:17:27.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.187+0000 7f21765fd740 10 client.41351 fetch_fsmap finished waiting for FSMap version 70 2024-04-24T09:17:27.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.187+0000 7f21765fd740 10 client.41351 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:17:27.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.187+0000 7f21765fd740 20 client.41351 populate_metadata read hostname 'smithi083' 2024-04-24T09:17:27.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.187+0000 7f21765fd740 1 --2- 172.21.15.83:0/3835164143 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5570bf2af110 0x5570bf2b1530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:27.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.187+0000 7f21765fd740 4 client.41351 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T09:17:27.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.187+0000 7f21765fd740 1 -- 172.21.15.83:0/3835164143 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5570bf19a700 con 0x5570bf2af110 2024-04-24T09:17:27.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.188+0000 7f2170e00640 1 --2- 172.21.15.83:0/3835164143 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5570bf2af110 0x5570bf2b1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:27.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.188+0000 7f2152a00640 10 client.41351 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:17:27.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.188+0000 7f2170e00640 1 --2- 172.21.15.83:0/3835164143 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5570bf2af110 0x5570bf2b1530 crc :-1 s=READY pgs=630 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9af2793ade22a689 server_cookie=902c98a97f053ced in_seq=0 out_seq=0 2024-04-24T09:17:27.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.189+0000 7f2152a00640 1 -- 172.21.15.83:0/3835164143 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5570bf19a700 con 0x5570bf2af110 2024-04-24T09:17:27.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.189+0000 7f2152a00640 10 client.41351 handle_command_reply: tid=0 2024-04-24T09:17:27.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.189+0000 7f21765fd740 1 client.41351 shutdown 2024-04-24T09:17:27.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.190+0000 7f21765fd740 1 -- 172.21.15.83:0/3835164143 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570bf238260 msgr2=0x5570bf23e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:27.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.190+0000 7f21765fd740 1 --2- 172.21.15.83:0/3835164143 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570bf238260 0x5570bf23e9a0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f21640050b0 tx=0x7f21640048b0 comp rx=0 tx=0).stop 2024-04-24T09:17:27.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.190+0000 7f21765fd740 1 -- 172.21.15.83:0/3835164143 shutdown_connections 2024-04-24T09:17:27.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.190+0000 7f21765fd740 1 --2- 172.21.15.83:0/3835164143 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5570bf2af110 0x5570bf2b1530 crc :-1 s=READY pgs=630 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:27.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.190+0000 7f21765fd740 1 --2- 172.21.15.83:0/3835164143 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570bf238260 0x5570bf23e9a0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:27.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.190+0000 7f21765fd740 1 --2- 172.21.15.83:0/3835164143 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5570bf22db90 0x5570bf239950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:27.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.190+0000 7f21765fd740 1 --2- 172.21.15.83:0/3835164143 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5570bf22d270 0x5570bf239410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:27.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.190+0000 7f21765fd740 1 -- 172.21.15.83:0/3835164143 >> 172.21.15.83:0/3835164143 conn(0x5570bf229590 msgr2=0x5570bf2286f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:27.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.190+0000 7f2152a00640 0 client.41351 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:17:27.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.190+0000 7f21765fd740 1 -- 172.21.15.83:0/3835164143 shutdown_connections 2024-04-24T09:17:27.192 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T09:17:27.192 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-04-24T09:17:27.192 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "bd8e2894-a168-48cd-8991-67fabbe2e832", 2024-04-24T09:17:27.192 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-04-24T09:17:27.192 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T09:17:27.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.190+0000 7f21765fd740 1 -- 172.21.15.83:0/3835164143 wait complete. 2024-04-24T09:17:27.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.191+0000 7f21765fd740 20 client.41351 trim_cache size 0 max 16384 2024-04-24T09:17:27.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.193+0000 7f2172200640 1 -- 172.21.15.83:0/1658779458 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f21540a3e20 msgr2=0x7f21540a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:27.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.193+0000 7f2172200640 1 --2- 172.21.15.83:0/1658779458 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f21540a3e20 0x7f21540a62e0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f2160005fd0 tx=0x7f2160008230 comp rx=0 tx=0).stop 2024-04-24T09:17:27.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.193+0000 7f2172200640 1 -- 172.21.15.83:0/1658779458 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f216c1093f0 msgr2=0x7f216c1a0b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:27.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.193+0000 7f2172200640 1 --2- 172.21.15.83:0/1658779458 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f216c1093f0 0x7f216c1a0b50 secure :-1 s=READY pgs=2845 cs=0 l=1 rev1=1 crypto rx=0x7f215802a920 tx=0x7f2158004040 comp rx=0 tx=0).stop 2024-04-24T09:17:27.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.195+0000 7f2172200640 1 -- 172.21.15.83:0/1658779458 shutdown_connections 2024-04-24T09:17:27.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.195+0000 7f2172200640 1 --2- 172.21.15.83:0/1658779458 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f21540a3e20 0x7f21540a62e0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:27.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.195+0000 7f2172200640 1 --2- 172.21.15.83:0/1658779458 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f216c19a7d0 0x7f216c19ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:27.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.195+0000 7f2172200640 1 --2- 172.21.15.83:0/1658779458 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f216c1093f0 0x7f216c1a0b50 unknown :-1 s=CLOSED pgs=2845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:27.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.195+0000 7f2172200640 1 --2- 172.21.15.83:0/1658779458 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f216c104320 0x7f216c1a0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:27.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.195+0000 7f2172200640 1 -- 172.21.15.83:0/1658779458 >> 172.21.15.83:0/1658779458 conn(0x7f216c0ffb40 msgr2=0x7f216c100f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:27.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.196+0000 7f2172200640 1 -- 172.21.15.83:0/1658779458 shutdown_connections 2024-04-24T09:17:27.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.197+0000 7f2172200640 1 -- 172.21.15.83:0/1658779458 wait complete. 2024-04-24T09:17:27.227 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T09:17:27.228 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T09:17:27.238 INFO:tasks.workunit.client.0.smithi083.stdout: 10 87317 6.48 MB/sec execute 598 sec latency 101.743 ms 2024-04-24T09:17:27.300 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.300+0000 7fca90200640 1 -- 172.21.15.83:0/1518575708 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fca88100cf0 msgr2=0x7fca8810a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:27.301 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.300+0000 7fca90200640 1 --2- 172.21.15.83:0/1518575708 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fca88100cf0 0x7fca8810a120 secure :-1 s=READY pgs=2847 cs=0 l=1 rev1=1 crypto rx=0x7fca74009930 tx=0x7fca7402a420 comp rx=0 tx=0).stop 2024-04-24T09:17:27.301 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.300+0000 7fca90200640 1 -- 172.21.15.83:0/1518575708 shutdown_connections 2024-04-24T09:17:27.301 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.300+0000 7fca90200640 1 --2- 172.21.15.83:0/1518575708 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fca8810a850 0x7fca881111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:27.301 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.300+0000 7fca90200640 1 --2- 172.21.15.83:0/1518575708 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fca88100cf0 0x7fca8810a120 unknown :-1 s=CLOSED pgs=2847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:27.301 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.300+0000 7fca90200640 1 --2- 172.21.15.83:0/1518575708 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fca88100340 0x7fca88100720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:27.301 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.300+0000 7fca90200640 1 -- 172.21.15.83:0/1518575708 >> 172.21.15.83:0/1518575708 conn(0x7fca88077000 msgr2=0x7fca880fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:27.301 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.300+0000 7fca90200640 1 -- 172.21.15.83:0/1518575708 shutdown_connections 2024-04-24T09:17:27.301 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.300+0000 7fca90200640 1 -- 172.21.15.83:0/1518575708 wait complete. 2024-04-24T09:17:27.302 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.302+0000 7fca90200640 1 Processor -- start 2024-04-24T09:17:27.302 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.302+0000 7fca90200640 1 -- start start 2024-04-24T09:17:27.303 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.302+0000 7fca90200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fca88100340 0x7fca881a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:27.303 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.302+0000 7fca90200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fca8810a850 0x7fca881a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:27.303 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.302+0000 7fca90200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fca8819a770 0x7fca8819abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:27.303 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.302+0000 7fca90200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fca881144b0 con 0x7fca8819a770 2024-04-24T09:17:27.303 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.302+0000 7fca90200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fca88114330 con 0x7fca88100340 2024-04-24T09:17:27.303 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.302+0000 7fca90200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fca88114630 con 0x7fca8810a850 2024-04-24T09:17:27.304 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.303+0000 7fca8e200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fca8819a770 0x7fca8819abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:27.304 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.303+0000 7fca8e200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fca8819a770 0x7fca8819abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35546/0 (socket says 172.21.15.83:35546) 2024-04-24T09:17:27.304 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.303+0000 7fca8e200640 1 -- 172.21.15.83:0/1261128603 learned_addr learned my addr 172.21.15.83:0/1261128603 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:17:27.304 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.303+0000 7fca8ce00640 1 --2- 172.21.15.83:0/1261128603 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fca8810a850 0x7fca881a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:27.305 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.303+0000 7fca8d800640 1 --2- 172.21.15.83:0/1261128603 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fca88100340 0x7fca881a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:27.305 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.303+0000 7fca8e200640 1 -- 172.21.15.83:0/1261128603 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fca8810a850 msgr2=0x7fca881a0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:27.305 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.303+0000 7fca8e200640 1 --2- 172.21.15.83:0/1261128603 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fca8810a850 0x7fca881a0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:27.305 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.303+0000 7fca8e200640 1 -- 172.21.15.83:0/1261128603 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fca88100340 msgr2=0x7fca881a06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:27.305 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.303+0000 7fca8e200640 1 --2- 172.21.15.83:0/1261128603 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fca88100340 0x7fca881a06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:27.305 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.303+0000 7fca8e200640 1 -- 172.21.15.83:0/1261128603 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca8819b370 con 0x7fca8819a770 2024-04-24T09:17:27.305 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.304+0000 7fca8e200640 1 --2- 172.21.15.83:0/1261128603 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fca8819a770 0x7fca8819abf0 secure :-1 s=READY pgs=2848 cs=0 l=1 rev1=1 crypto rx=0x7fca8000eb10 tx=0x7fca8000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:27.305 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.304+0000 7fca86000640 1 -- 172.21.15.83:0/1261128603 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca80004b20 con 0x7fca8819a770 2024-04-24T09:17:27.305 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.304+0000 7fca86000640 1 -- 172.21.15.83:0/1261128603 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fca80004cc0 con 0x7fca8819a770 2024-04-24T09:17:27.305 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.304+0000 7fca90200640 1 -- 172.21.15.83:0/1261128603 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca881a7450 con 0x7fca8819a770 2024-04-24T09:17:27.306 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.304+0000 7fca90200640 1 -- 172.21.15.83:0/1261128603 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca881a7800 con 0x7fca8819a770 2024-04-24T09:17:27.306 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.304+0000 7fca86000640 1 -- 172.21.15.83:0/1261128603 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca80010690 con 0x7fca8819a770 2024-04-24T09:17:27.310 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.310+0000 7fca92ffe740 1 Processor -- start 2024-04-24T09:17:27.310 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.310+0000 7fca86000640 1 -- 172.21.15.83:0/1261128603 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fca80005020 con 0x7fca8819a770 2024-04-24T09:17:27.315 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.311+0000 7fca92ffe740 1 -- start start 2024-04-24T09:17:27.315 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.311+0000 7fca92ffe740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55919531c1b0 0x55919531c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:27.315 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.311+0000 7fca92ffe740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55919531cad0 0x559195326e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:27.315 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.311+0000 7fca92ffe740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559195327390 0x559195329780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:27.316 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.311+0000 7fca92ffe740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559195282900 con 0x55919531cad0 2024-04-24T09:17:27.316 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.311+0000 7fca92ffe740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559195228f20 con 0x55919531c1b0 2024-04-24T09:17:27.316 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.311+0000 7fca8d800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55919531cad0 0x559195326e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:27.316 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.314+0000 7fca92ffe740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55919527e8c0 con 0x559195327390 2024-04-24T09:17:27.316 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.314+0000 7fca8d800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55919531cad0 0x559195326e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35548/0 (socket says 172.21.15.83:35548) 2024-04-24T09:17:27.316 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.314+0000 7fca8d800640 1 -- 172.21.15.83:0/3556797198 learned_addr learned my addr 172.21.15.83:0/3556797198 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:17:27.316 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.314+0000 7fca8ce00640 1 --2- 172.21.15.83:0/3556797198 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55919531c1b0 0x55919531c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:27.316 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.314+0000 7fca86000640 1 --2- 172.21.15.83:0/1261128603 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fca700a3dc0 0x7fca700a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:27.316 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.314+0000 7fca8ce00640 1 --2- 172.21.15.83:0/3556797198 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559195327390 0x559195329780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:27.316 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.315+0000 7fca86000640 1 -- 172.21.15.83:0/1261128603 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 9583+0+0 (secure 0 0 0) 0x7fca800d1250 con 0x7fca8819a770 2024-04-24T09:17:27.316 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.315+0000 7fca8e200640 1 --2- 172.21.15.83:0/1261128603 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fca700a3dc0 0x7fca700a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:27.316 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.315+0000 7fca8ce00640 1 -- 172.21.15.83:0/3556797198 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559195327390 msgr2=0x559195329780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:27.316 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.315+0000 7fca8ce00640 1 --2- 172.21.15.83:0/3556797198 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559195327390 0x559195329780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:27.316 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.315+0000 7fca8ce00640 1 -- 172.21.15.83:0/3556797198 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55919531cad0 msgr2=0x559195326e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:27.317 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.315+0000 7fca8ce00640 1 --2- 172.21.15.83:0/3556797198 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55919531cad0 0x559195326e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:27.317 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.316+0000 7fca8e200640 1 --2- 172.21.15.83:0/1261128603 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fca700a3dc0 0x7fca700a6280 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fca800ce630 tx=0x7fca80009a70 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:27.317 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.316+0000 7fca8ce00640 1 -- 172.21.15.83:0/3556797198 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55919524de10 con 0x55919531c1b0 2024-04-24T09:17:27.317 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.316+0000 7fca8d800640 1 --2- 172.21.15.83:0/3556797198 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55919531cad0 0x559195326e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:17:27.317 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.316+0000 7fca8ce00640 1 --2- 172.21.15.83:0/3556797198 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55919531c1b0 0x55919531c590 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fca7402aa10 tx=0x7fca74005780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b5703b0ce0bb665 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:27.318 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.317+0000 7fca7aa00640 1 -- 172.21.15.83:0/3556797198 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca74035030 con 0x55919531c1b0 2024-04-24T09:17:27.318 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.317+0000 7fca7aa00640 1 -- 172.21.15.83:0/3556797198 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fca74003ca0 con 0x55919531c1b0 2024-04-24T09:17:27.318 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.317+0000 7fca7aa00640 1 -- 172.21.15.83:0/3556797198 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca7403d970 con 0x55919531c1b0 2024-04-24T09:17:27.319 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.319+0000 7fca92ffe740 1 -- 172.21.15.83:0/3556797198 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55919531c1b0 msgr2=0x55919531c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:27.319 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.319+0000 7fca92ffe740 1 --2- 172.21.15.83:0/3556797198 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55919531c1b0 0x55919531c590 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fca7402aa10 tx=0x7fca74005780 comp rx=0 tx=0).stop 2024-04-24T09:17:27.320 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.319+0000 7fca92ffe740 1 -- 172.21.15.83:0/3556797198 shutdown_connections 2024-04-24T09:17:27.320 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.319+0000 7fca92ffe740 1 --2- 172.21.15.83:0/3556797198 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559195327390 0x559195329780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:27.320 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.319+0000 7fca92ffe740 1 --2- 172.21.15.83:0/3556797198 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55919531cad0 0x559195326e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:27.320 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.319+0000 7fca92ffe740 1 --2- 172.21.15.83:0/3556797198 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55919531c1b0 0x55919531c590 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:27.320 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.319+0000 7fca92ffe740 1 -- 172.21.15.83:0/3556797198 >> 172.21.15.83:0/3556797198 conn(0x5591953186c0 msgr2=0x559195318aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:27.320 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.320+0000 7fca92ffe740 1 -- 172.21.15.83:0/3556797198 shutdown_connections 2024-04-24T09:17:27.321 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.320+0000 7fca92ffe740 1 -- 172.21.15.83:0/3556797198 wait complete. 2024-04-24T09:17:27.321 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.320+0000 7fca92ffe740 1 -- start start 2024-04-24T09:17:27.321 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.320+0000 7fca92ffe740 1 Processor -- start 2024-04-24T09:17:27.322 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.321+0000 7fca92ffe740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55919531c1b0 0x559195324410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:27.322 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.321+0000 7fca92ffe740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55919531cad0 0x559195324950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:27.322 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.321+0000 7fca92ffe740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559195327390 0x559195324e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:27.322 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.321+0000 7fca92ffe740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55919533af70 con 0x559195327390 2024-04-24T09:17:27.322 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.321+0000 7fca92ffe740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55919523c2c0 con 0x55919531cad0 2024-04-24T09:17:27.322 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.321+0000 7fca92ffe740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55919533b0f0 con 0x55919531c1b0 2024-04-24T09:17:27.322 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.321+0000 7fca92ffe740 10 client.0 _ll_register_callbacks cb 0x559194f94a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:17:27.322 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.322+0000 7fca8d800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55919531cad0 0x559195324950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:27.322 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.322+0000 7fca8ce00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559195327390 0x559195324e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:27.322 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.322+0000 7fca8d800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55919531cad0 0x559195324950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:39552/0 (socket says 172.21.15.83:39552) 2024-04-24T09:17:27.322 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.322+0000 7fca8d800640 1 -- 172.21.15.83:0/571167837 learned_addr learned my addr 172.21.15.83:0/571167837 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:17:27.324 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.322+0000 7fca8ce00640 1 --2- 172.21.15.83:0/571167837 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55919531c1b0 0x559195324410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:27.324 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.322+0000 7fca8ce00640 1 -- 172.21.15.83:0/571167837 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55919531c1b0 msgr2=0x559195324410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:27.324 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.322+0000 7fca8ce00640 1 --2- 172.21.15.83:0/571167837 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55919531c1b0 0x559195324410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:27.324 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.322+0000 7fca8ce00640 1 -- 172.21.15.83:0/571167837 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55919531cad0 msgr2=0x559195324950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:27.324 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.322+0000 7fca8ce00640 1 --2- 172.21.15.83:0/571167837 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55919531cad0 0x559195324950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:27.324 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.322+0000 7fca8ce00640 1 -- 172.21.15.83:0/571167837 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55919524cfd0 con 0x559195327390 2024-04-24T09:17:27.324 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.323+0000 7fca8ce00640 1 --2- 172.21.15.83:0/571167837 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559195327390 0x559195324e90 secure :-1 s=READY pgs=2849 cs=0 l=1 rev1=1 crypto rx=0x7fca7402a940 tx=0x7fca74005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8848b7a11afecee0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:27.324 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.323+0000 7fca6f400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:17:27.324 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.323+0000 7fca6f400640 1 -- 172.21.15.83:0/571167837 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca74051060 con 0x559195327390 2024-04-24T09:17:27.324 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.323+0000 7fca6f400640 1 -- 172.21.15.83:0/571167837 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(134..134 src has 1..134) ==== 9583+0+0 (secure 0 0 0) 0x7fca740037f0 con 0x559195327390 2024-04-24T09:17:27.324 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.324+0000 7fca92ffe740 1 -- 172.21.15.83:0/571167837 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca74003ca0 con 0x559195327390 2024-04-24T09:17:27.325 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.324+0000 7fca6f400640 1 -- 172.21.15.83:0/571167837 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7fca74035070 con 0x559195327390 2024-04-24T09:17:27.325 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.324+0000 7fca92ffe740 10 client.24300 fetch_fsmap learned FSMap version 70 2024-04-24T09:17:27.325 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.324+0000 7fca92ffe740 1 -- 172.21.15.83:0/571167837 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) -- 0x559194f95210 con 0x559195327390 2024-04-24T09:17:27.325 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.325+0000 7fca6f400640 1 -- 172.21.15.83:0/571167837 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fca74003ca0 con 0x559195327390 2024-04-24T09:17:27.326 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.325+0000 7fca6f400640 1 -- 172.21.15.83:0/571167837 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 70) ==== 8366+0+0 (secure 0 0 0) 0x7fca74049040 con 0x559195327390 2024-04-24T09:17:27.326 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.325+0000 7fca92ffe740 10 client.24300 fetch_fsmap finished waiting for FSMap version 70 2024-04-24T09:17:27.326 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.325+0000 7fca92ffe740 10 client.24300 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:17:27.326 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.325+0000 7fca92ffe740 20 client.24300 populate_metadata read hostname 'smithi083' 2024-04-24T09:17:27.326 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.326+0000 7fca92ffe740 1 --2- 172.21.15.83:0/571167837 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x559195348440 0x55919534a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:27.326 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.326+0000 7fca92ffe740 4 client.24300 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:17:27.326 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.326+0000 7fca92ffe740 1 -- 172.21.15.83:0/571167837 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559195290320 con 0x559195348440 2024-04-24T09:17:27.327 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.327+0000 7fca8d800640 1 --2- 172.21.15.83:0/571167837 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x559195348440 0x55919534a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:27.331 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.330+0000 7fca8d800640 1 --2- 172.21.15.83:0/571167837 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x559195348440 0x55919534a830 crc :-1 s=READY pgs=631 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9eadeaf2ddb9d40 server_cookie=5cfcfb9755e0a646 in_seq=0 out_seq=0 2024-04-24T09:17:27.331 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.331+0000 7fca6f400640 10 client.24300 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:17:27.339 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.339+0000 7fca6f400640 1 -- 172.21.15.83:0/571167837 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559195290320 con 0x559195348440 2024-04-24T09:17:27.340 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.339+0000 7fca6f400640 10 client.24300 handle_command_reply: tid=0 2024-04-24T09:17:27.340 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.339+0000 7fca92ffe740 1 client.24300 shutdown 2024-04-24T09:17:27.340 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.340+0000 7fca92ffe740 1 -- 172.21.15.83:0/571167837 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559195327390 msgr2=0x559195324e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:27.341 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.340+0000 7fca92ffe740 1 --2- 172.21.15.83:0/571167837 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559195327390 0x559195324e90 secure :-1 s=READY pgs=2849 cs=0 l=1 rev1=1 crypto rx=0x7fca7402a940 tx=0x7fca74005850 comp rx=0 tx=0).stop 2024-04-24T09:17:27.341 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.340+0000 7fca92ffe740 1 -- 172.21.15.83:0/571167837 shutdown_connections 2024-04-24T09:17:27.341 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.340+0000 7fca92ffe740 1 --2- 172.21.15.83:0/571167837 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x559195348440 0x55919534a830 crc :-1 s=READY pgs=631 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:27.341 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.340+0000 7fca92ffe740 1 --2- 172.21.15.83:0/571167837 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559195327390 0x559195324e90 unknown :-1 s=CLOSED pgs=2849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:27.341 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.340+0000 7fca92ffe740 1 --2- 172.21.15.83:0/571167837 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55919531cad0 0x559195324950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:27.341 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.340+0000 7fca92ffe740 1 --2- 172.21.15.83:0/571167837 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55919531c1b0 0x559195324410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:27.341 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.340+0000 7fca92ffe740 1 -- 172.21.15.83:0/571167837 >> 172.21.15.83:0/571167837 conn(0x5591953186c0 msgr2=0x55919531e440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:27.341 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.340+0000 7fca6f400640 0 client.24300 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:17:27.341 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.340+0000 7fca92ffe740 1 -- 172.21.15.83:0/571167837 shutdown_connections 2024-04-24T09:17:27.341 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.340+0000 7fca92ffe740 1 -- 172.21.15.83:0/571167837 wait complete. 2024-04-24T09:17:27.341 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.340+0000 7fca92ffe740 20 client.24300 trim_cache size 0 max 16384 2024-04-24T09:17:27.365 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.365+0000 7fca92ffe740 1 Processor -- start 2024-04-24T09:17:27.365 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.365+0000 7fca92ffe740 1 -- start start 2024-04-24T09:17:27.366 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.365+0000 7fca92ffe740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55919531c1b0 0x559195317fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:27.366 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.365+0000 7fca92ffe740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55919531cad0 0x559195343a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:27.366 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.365+0000 7fca92ffe740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559195327390 0x55919534d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:27.366 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.365+0000 7fca92ffe740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fca881a8e70 con 0x559195327390 2024-04-24T09:17:27.366 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.365+0000 7fca92ffe740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fca881aa9e0 con 0x55919531cad0 2024-04-24T09:17:27.366 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.365+0000 7fca92ffe740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fca881a9540 con 0x55919531c1b0 2024-04-24T09:17:27.367 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.366+0000 7fca8d800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55919531cad0 0x559195343a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:27.367 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.366+0000 7fca8ce00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55919531c1b0 0x559195317fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:27.367 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.366+0000 7fca8ce00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55919531c1b0 0x559195317fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:59268/0 (socket says 172.21.15.83:59268) 2024-04-24T09:17:27.367 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.366+0000 7fca8ce00640 1 -- 172.21.15.83:0/4230957877 learned_addr learned my addr 172.21.15.83:0/4230957877 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:17:27.367 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.366+0000 7fca8d800640 1 -- 172.21.15.83:0/4230957877 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55919531c1b0 msgr2=0x559195317fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:27.367 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.366+0000 7fca8d800640 1 --2- 172.21.15.83:0/4230957877 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55919531c1b0 0x559195317fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:27.367 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.366+0000 7fca8d800640 1 -- 172.21.15.83:0/4230957877 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559195327390 msgr2=0x55919534d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:27.367 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.366+0000 7fca8d800640 1 --2- 172.21.15.83:0/4230957877 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559195327390 0x55919534d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:27.367 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.366+0000 7fca8d800640 1 -- 172.21.15.83:0/4230957877 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559195061620 con 0x55919531cad0 2024-04-24T09:17:27.368 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.367+0000 7fca8d800640 1 --2- 172.21.15.83:0/4230957877 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55919531cad0 0x559195343a50 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x55919533b0e0 tx=0x7fca7c005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6715be336c9f075f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:27.369 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.368+0000 7fca7a000640 1 -- 172.21.15.83:0/4230957877 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca7c004220 con 0x55919531cad0 2024-04-24T09:17:27.369 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.369+0000 7fca7a000640 1 -- 172.21.15.83:0/4230957877 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fca7c025070 con 0x55919531cad0 2024-04-24T09:17:27.369 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.369+0000 7fca7a000640 1 -- 172.21.15.83:0/4230957877 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca7c004d70 con 0x55919531cad0 2024-04-24T09:17:27.370 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.370+0000 7fca92ffe740 1 -- 172.21.15.83:0/4230957877 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55919531cad0 msgr2=0x559195343a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:27.370 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.370+0000 7fca92ffe740 1 --2- 172.21.15.83:0/4230957877 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55919531cad0 0x559195343a50 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x55919533b0e0 tx=0x7fca7c005f90 comp rx=0 tx=0).stop 2024-04-24T09:17:27.370 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.370+0000 7fca92ffe740 1 -- 172.21.15.83:0/4230957877 shutdown_connections 2024-04-24T09:17:27.370 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.370+0000 7fca92ffe740 1 --2- 172.21.15.83:0/4230957877 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559195327390 0x55919534d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:27.370 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.370+0000 7fca92ffe740 1 --2- 172.21.15.83:0/4230957877 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55919531cad0 0x559195343a50 secure :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0x55919533b0e0 tx=0x7fca7c005f90 comp rx=0 tx=0).stop 2024-04-24T09:17:27.371 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.370+0000 7fca92ffe740 1 --2- 172.21.15.83:0/4230957877 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55919531c1b0 0x559195317fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:27.371 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.370+0000 7fca92ffe740 1 -- 172.21.15.83:0/4230957877 >> 172.21.15.83:0/4230957877 conn(0x5591953186c0 msgr2=0x559195343340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:27.371 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.370+0000 7fca92ffe740 1 -- 172.21.15.83:0/4230957877 shutdown_connections 2024-04-24T09:17:27.371 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.371+0000 7fca92ffe740 1 -- 172.21.15.83:0/4230957877 wait complete. 2024-04-24T09:17:27.372 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.372+0000 7fca92ffe740 1 -- start start 2024-04-24T09:17:27.372 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.372+0000 7fca92ffe740 1 Processor -- start 2024-04-24T09:17:27.374 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.374+0000 7fca92ffe740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55919531c1b0 0x55919534ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:27.374 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.374+0000 7fca92ffe740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559195327390 0x55919534b390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:27.375 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.374+0000 7fca92ffe740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55919534b8d0 0x5591953ad620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:27.375 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.374+0000 7fca92ffe740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fca881aa310 con 0x55919531c1b0 2024-04-24T09:17:27.375 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.374+0000 7fca92ffe740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fca881a9c40 con 0x55919534b8d0 2024-04-24T09:17:27.375 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.374+0000 7fca92ffe740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fca88073610 con 0x559195327390 2024-04-24T09:17:27.375 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.374+0000 7fca92ffe740 10 client.0 _ll_register_callbacks cb 0x7fca7c0096c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:17:27.378 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.374+0000 7fca8ce00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55919531c1b0 0x55919534ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:27.378 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.375+0000 7fca8d800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559195327390 0x55919534b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:27.378 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.377+0000 7fca8d800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559195327390 0x55919534b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:59282/0 (socket says 172.21.15.83:59282) 2024-04-24T09:17:27.378 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.377+0000 7fca8d800640 1 -- 172.21.15.83:0/1143621418 learned_addr learned my addr 172.21.15.83:0/1143621418 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:17:27.378 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.377+0000 7fca8d800640 1 -- 172.21.15.83:0/1143621418 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55919534b8d0 msgr2=0x5591953ad620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:27.378 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.377+0000 7fca8d800640 1 --2- 172.21.15.83:0/1143621418 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55919534b8d0 0x5591953ad620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:27.378 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.377+0000 7fca8d800640 1 -- 172.21.15.83:0/1143621418 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55919531c1b0 msgr2=0x55919534ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:27.378 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.378+0000 7fca8d800640 1 --2- 172.21.15.83:0/1143621418 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55919531c1b0 0x55919534ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:27.379 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.378+0000 7fca8d800640 1 -- 172.21.15.83:0/1143621418 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591953adb60 con 0x559195327390 2024-04-24T09:17:27.379 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.378+0000 7fca8ce00640 1 --2- 172.21.15.83:0/1143621418 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55919531c1b0 0x55919534ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:17:27.379 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.379+0000 7fca8d800640 1 --2- 172.21.15.83:0/1143621418 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559195327390 0x55919534b390 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fca7c014010 tx=0x7fca7c004200 comp rx=0 tx=0).ready entity=mon.1 client_cookie=80ccdcfe90506cba server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:27.379 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.379+0000 7fca6f400640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-04-24T09:17:27.380 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.380+0000 7fca6f400640 1 -- 172.21.15.83:0/1143621418 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca7c005220 con 0x559195327390 2024-04-24T09:17:27.380 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.380+0000 7fca6f400640 1 -- 172.21.15.83:0/1143621418 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(134..134 src has 1..134) ==== 9583+0+0 (secure 0 0 0) 0x7fca7c0053c0 con 0x559195327390 2024-04-24T09:17:27.382 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.381+0000 7fca92ffe740 1 -- 172.21.15.83:0/1143621418 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca7c025070 con 0x559195327390 2024-04-24T09:17:27.382 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.382+0000 7fca6f400640 1 -- 172.21.15.83:0/1143621418 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7fca7c0055c0 con 0x559195327390 2024-04-24T09:17:27.382 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.382+0000 7fca92ffe740 10 client.31006 fetch_fsmap learned FSMap version 70 2024-04-24T09:17:27.382 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.382+0000 7fca92ffe740 1 -- 172.21.15.83:0/1143621418 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) -- 0x5591953aed20 con 0x559195327390 2024-04-24T09:17:27.383 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.383+0000 7fca6f400640 1 -- 172.21.15.83:0/1143621418 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fca7c025070 con 0x559195327390 2024-04-24T09:17:27.383 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.383+0000 7fca6f400640 1 -- 172.21.15.83:0/1143621418 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 70) ==== 8366+0+0 (secure 0 0 0) 0x7fca7c031020 con 0x559195327390 2024-04-24T09:17:27.384 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.383+0000 7fca92ffe740 10 client.31006 fetch_fsmap finished waiting for FSMap version 70 2024-04-24T09:17:27.384 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.383+0000 7fca92ffe740 10 client.31006 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-24T09:17:27.384 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.383+0000 7fca92ffe740 20 client.31006 populate_metadata read hostname 'smithi083' 2024-04-24T09:17:27.384 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.383+0000 7fca92ffe740 1 --2- 172.21.15.83:0/1143621418 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5591953af340 0x5591953b1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:27.384 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.384+0000 7fca92ffe740 4 client.31006 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-24T09:17:27.384 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.384+0000 7fca92ffe740 1 -- 172.21.15.83:0/1143621418 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55919528a190 con 0x5591953af340 2024-04-24T09:17:27.385 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.385+0000 7fca8ce00640 1 --2- 172.21.15.83:0/1143621418 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5591953af340 0x5591953b1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:27.388 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.387+0000 7fca8ce00640 1 --2- 172.21.15.83:0/1143621418 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5591953af340 0x5591953b1760 crc :-1 s=READY pgs=632 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5788b482421e8bd server_cookie=7c7a130ef30bf743 in_seq=0 out_seq=0 2024-04-24T09:17:27.388 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.387+0000 7fca6f400640 10 client.31006 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:17:27.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.574+0000 7f0a9fc00640 1 -- 172.21.15.83:0/76226004 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0a98109b30 msgr2=0x7f0a98114c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:27.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.574+0000 7f0a9fc00640 1 --2- 172.21.15.83:0/76226004 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0a98109b30 0x7f0a98114c10 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0a84009900 tx=0x7f0a8402a410 comp rx=0 tx=0).stop 2024-04-24T09:17:27.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.576+0000 7f0a9fc00640 1 -- 172.21.15.83:0/76226004 shutdown_connections 2024-04-24T09:17:27.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.576+0000 7f0a9fc00640 1 --2- 172.21.15.83:0/76226004 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a98115150 0x7f0a98117540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:27.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.576+0000 7f0a9fc00640 1 --2- 172.21.15.83:0/76226004 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0a98109b30 0x7f0a98114c10 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:27.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.576+0000 7f0a9fc00640 1 --2- 172.21.15.83:0/76226004 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a98109210 0x7f0a981095f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:27.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.576+0000 7f0a9fc00640 1 -- 172.21.15.83:0/76226004 >> 172.21.15.83:0/76226004 conn(0x7f0a9806d110 msgr2=0x7f0a9806d520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:27.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.577+0000 7f0a9fc00640 1 -- 172.21.15.83:0/76226004 shutdown_connections 2024-04-24T09:17:27.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.577+0000 7f0a9fc00640 1 -- 172.21.15.83:0/76226004 wait complete. 2024-04-24T09:17:27.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.580+0000 7f0a9fc00640 1 Processor -- start 2024-04-24T09:17:27.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.581+0000 7f0a9fc00640 1 -- start start 2024-04-24T09:17:27.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.581+0000 7f0a9fc00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0a98109210 0x7f0a98112e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:27.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.581+0000 7f0a9fc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a98109b30 0x7f0a98113360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:27.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.581+0000 7f0a9fc00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a98115150 0x7f0a981138a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:27.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.581+0000 7f0a9fc00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0a98118950 con 0x7f0a98109b30 2024-04-24T09:17:27.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.581+0000 7f0a9fc00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0a981187d0 con 0x7f0a98115150 2024-04-24T09:17:27.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.581+0000 7f0a9fc00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0a98118ad0 con 0x7f0a98109210 2024-04-24T09:17:27.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.582+0000 7f0a9d200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0a98109210 0x7f0a98112e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:27.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.582+0000 7f0a9d200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0a98109210 0x7f0a98112e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.83:59306/0 (socket says 172.21.15.83:59306) 2024-04-24T09:17:27.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.582+0000 7f0a9d200640 1 -- 172.21.15.83:0/1194977099 learned_addr learned my addr 172.21.15.83:0/1194977099 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:17:27.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.582+0000 7f0a9dc00640 1 --2- 172.21.15.83:0/1194977099 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a98115150 0x7f0a981138a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:27.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.582+0000 7f0a97e00640 1 --2- 172.21.15.83:0/1194977099 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a98109b30 0x7f0a98113360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:27.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.583+0000 7f0a9d200640 1 -- 172.21.15.83:0/1194977099 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a98115150 msgr2=0x7f0a981138a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:27.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.583+0000 7f0a9d200640 1 --2- 172.21.15.83:0/1194977099 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a98115150 0x7f0a981138a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:27.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.583+0000 7f0a9d200640 1 -- 172.21.15.83:0/1194977099 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a98109b30 msgr2=0x7f0a98113360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:27.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.583+0000 7f0a9d200640 1 --2- 172.21.15.83:0/1194977099 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a98109b30 0x7f0a98113360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:27.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.583+0000 7f0a9d200640 1 -- 172.21.15.83:0/1194977099 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a981b38a0 con 0x7f0a98109210 2024-04-24T09:17:27.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.583+0000 7f0a97e00640 1 --2- 172.21.15.83:0/1194977099 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a98109b30 0x7f0a98113360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:17:27.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.584+0000 7f0a9d200640 1 --2- 172.21.15.83:0/1194977099 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0a98109210 0x7f0a98112e20 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c00c920 tx=0x7f0a8c00cde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:27.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.584+0000 7f0a95600640 1 -- 172.21.15.83:0/1194977099 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a8c007c70 con 0x7f0a98109210 2024-04-24T09:17:27.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.584+0000 7f0a95600640 1 -- 172.21.15.83:0/1194977099 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0a8c007e10 con 0x7f0a98109210 2024-04-24T09:17:27.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.584+0000 7f0a95600640 1 -- 172.21.15.83:0/1194977099 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a8c018820 con 0x7f0a98109210 2024-04-24T09:17:27.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.584+0000 7f0a9fc00640 1 -- 172.21.15.83:0/1194977099 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a981b3b90 con 0x7f0a98109210 2024-04-24T09:17:27.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.585+0000 7f0a9fc00640 1 -- 172.21.15.83:0/1194977099 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a981b4090 con 0x7f0a98109210 2024-04-24T09:17:27.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.590+0000 7f0a95600640 1 -- 172.21.15.83:0/1194977099 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0a8c0189c0 con 0x7f0a98109210 2024-04-24T09:17:27.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.590+0000 7f0aa29cc740 1 Processor -- start 2024-04-24T09:17:27.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.591+0000 7f0aa29cc740 1 -- start start 2024-04-24T09:17:27.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.591+0000 7f0aa29cc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5582d7727860 0x5582d7727c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:27.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.591+0000 7f0aa29cc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582d7728180 0x5582d7732310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:27.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.592+0000 7f0aa29cc740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5582d7732850 0x5582d7734c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:27.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.592+0000 7f0aa29cc740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5582d768d1f0 con 0x5582d7728180 2024-04-24T09:17:27.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.592+0000 7f0aa29cc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5582d7633350 con 0x5582d7727860 2024-04-24T09:17:27.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.592+0000 7f0aa29cc740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5582d76891b0 con 0x5582d7732850 2024-04-24T09:17:27.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.592+0000 7f0a9d200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582d7728180 0x5582d7732310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:27.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.592+0000 7f0a9d200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582d7728180 0x5582d7732310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35580/0 (socket says 172.21.15.83:35580) 2024-04-24T09:17:27.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.592+0000 7f0a9d200640 1 -- 172.21.15.83:0/1486242585 learned_addr learned my addr 172.21.15.83:0/1486242585 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:17:27.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.592+0000 7f0a9dc00640 1 --2- 172.21.15.83:0/1486242585 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5582d7727860 0x5582d7727c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:27.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.592+0000 7f0a97e00640 1 --2- 172.21.15.83:0/1486242585 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5582d7732850 0x5582d7734c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:27.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.593+0000 7f0a95600640 1 --2- 172.21.15.83:0/1194977099 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f0a800a3e20 0x7f0a800a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:27.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.593+0000 7f0a95600640 1 -- 172.21.15.83:0/1194977099 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 9583+0+0 (secure 0 0 0) 0x7f0a8c0d0e70 con 0x7f0a98109210 2024-04-24T09:17:27.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.593+0000 7f0a9d200640 1 -- 172.21.15.83:0/1486242585 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5582d7732850 msgr2=0x5582d7734c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:27.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.593+0000 7f0a9d200640 1 --2- 172.21.15.83:0/1486242585 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5582d7732850 0x5582d7734c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:27.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.593+0000 7f0a9d200640 1 -- 172.21.15.83:0/1486242585 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5582d7727860 msgr2=0x5582d7727c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:27.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.593+0000 7f0a9d200640 1 --2- 172.21.15.83:0/1486242585 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5582d7727860 0x5582d7727c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:27.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.593+0000 7f0a9d200640 1 -- 172.21.15.83:0/1486242585 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582d7687f50 con 0x5582d7728180 2024-04-24T09:17:27.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.593+0000 7f0a9dc00640 1 --2- 172.21.15.83:0/1194977099 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f0a800a3e20 0x7f0a800a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:27.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.594+0000 7f0a9d200640 1 --2- 172.21.15.83:0/1486242585 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582d7728180 0x5582d7732310 secure :-1 s=READY pgs=2850 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c0ce080 tx=0x7f0a8c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=533f07b1512969cc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:27.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.594+0000 7f0a8a000640 1 -- 172.21.15.83:0/1486242585 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a8c09c4d0 con 0x5582d7728180 2024-04-24T09:17:27.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.594+0000 7f0a8a000640 1 -- 172.21.15.83:0/1486242585 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0a8c00fb00 con 0x5582d7728180 2024-04-24T09:17:27.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.594+0000 7f0a8a000640 1 -- 172.21.15.83:0/1486242585 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a8c09c4d0 con 0x5582d7728180 2024-04-24T09:17:27.596 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.594+0000 7fca6f400640 1 -- 172.21.15.83:0/1143621418 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55919528a190 con 0x5591953af340 2024-04-24T09:17:27.596 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.595+0000 7fca6f400640 10 client.31006 handle_command_reply: tid=0 2024-04-24T09:17:27.596 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.595+0000 7fca92ffe740 1 client.31006 shutdown 2024-04-24T09:17:27.596 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.595+0000 7fca92ffe740 1 -- 172.21.15.83:0/1143621418 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559195327390 msgr2=0x55919534b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:27.596 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.595+0000 7fca92ffe740 1 --2- 172.21.15.83:0/1143621418 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559195327390 0x55919534b390 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fca7c014010 tx=0x7fca7c004200 comp rx=0 tx=0).stop 2024-04-24T09:17:27.596 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.595+0000 7fca92ffe740 1 -- 172.21.15.83:0/1143621418 shutdown_connections 2024-04-24T09:17:27.596 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.595+0000 7fca92ffe740 1 --2- 172.21.15.83:0/1143621418 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5591953af340 0x5591953b1760 crc :-1 s=READY pgs=632 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:27.596 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.595+0000 7fca92ffe740 1 --2- 172.21.15.83:0/1143621418 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55919534b8d0 0x5591953ad620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:27.596 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.595+0000 7fca92ffe740 1 --2- 172.21.15.83:0/1143621418 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559195327390 0x55919534b390 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:27.596 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.595+0000 7fca92ffe740 1 --2- 172.21.15.83:0/1143621418 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55919531c1b0 0x55919534ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:27.597 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.595+0000 7fca92ffe740 1 -- 172.21.15.83:0/1143621418 >> 172.21.15.83:0/1143621418 conn(0x5591953186c0 msgr2=0x559195343bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:27.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.595+0000 7f0aa29cc740 1 -- 172.21.15.83:0/1486242585 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582d7728180 msgr2=0x5582d7732310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:27.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.595+0000 7f0aa29cc740 1 --2- 172.21.15.83:0/1486242585 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582d7728180 0x5582d7732310 secure :-1 s=READY pgs=2850 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c0ce080 tx=0x7f0a8c000bc0 comp rx=0 tx=0).stop 2024-04-24T09:17:27.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.595+0000 7f0a9dc00640 1 --2- 172.21.15.83:0/1194977099 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f0a800a3e20 0x7f0a800a62e0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f0a90005fd0 tx=0x7f0a90004a40 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:27.597 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.595+0000 7fca6f400640 0 client.31006 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:17:27.597 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.596+0000 7fca92ffe740 1 -- 172.21.15.83:0/1143621418 shutdown_connections 2024-04-24T09:17:27.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.596+0000 7f0aa29cc740 1 -- 172.21.15.83:0/1486242585 shutdown_connections 2024-04-24T09:17:27.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.596+0000 7f0aa29cc740 1 --2- 172.21.15.83:0/1486242585 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5582d7732850 0x5582d7734c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:27.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.596+0000 7f0aa29cc740 1 --2- 172.21.15.83:0/1486242585 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582d7728180 0x5582d7732310 unknown :-1 s=CLOSED pgs=2850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:27.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.596+0000 7f0aa29cc740 1 --2- 172.21.15.83:0/1486242585 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5582d7727860 0x5582d7727c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:27.598 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.596+0000 7fca92ffe740 1 -- 172.21.15.83:0/1143621418 wait complete. 2024-04-24T09:17:27.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.596+0000 7f0aa29cc740 1 -- 172.21.15.83:0/1486242585 >> 172.21.15.83:0/1486242585 conn(0x5582d7723b80 msgr2=0x5582d7723f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:27.599 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.596+0000 7fca92ffe740 20 client.31006 trim_cache size 0 max 16384 2024-04-24T09:17:27.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.596+0000 7f0aa29cc740 1 -- 172.21.15.83:0/1486242585 shutdown_connections 2024-04-24T09:17:27.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.596+0000 7f0aa29cc740 1 -- 172.21.15.83:0/1486242585 wait complete. 2024-04-24T09:17:27.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.597+0000 7f0aa29cc740 1 -- start start 2024-04-24T09:17:27.600 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:{ 2024-04-24T09:17:27.600 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "message": "", 2024-04-24T09:17:27.600 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout: "return_code": 0 2024-04-24T09:17:27.600 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stdout:} 2024-04-24T09:17:27.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.597+0000 7f0aa29cc740 1 Processor -- start 2024-04-24T09:17:27.601 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.598+0000 7fca90200640 1 -- 172.21.15.83:0/1261128603 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fca700a3dc0 msgr2=0x7fca700a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:27.601 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.598+0000 7fca90200640 1 --2- 172.21.15.83:0/1261128603 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fca700a3dc0 0x7fca700a6280 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fca800ce630 tx=0x7fca80009a70 comp rx=0 tx=0).stop 2024-04-24T09:17:27.601 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.598+0000 7fca90200640 1 -- 172.21.15.83:0/1261128603 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fca8819a770 msgr2=0x7fca8819abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:27.601 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.598+0000 7fca90200640 1 --2- 172.21.15.83:0/1261128603 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fca8819a770 0x7fca8819abf0 secure :-1 s=READY pgs=2848 cs=0 l=1 rev1=1 crypto rx=0x7fca8000eb10 tx=0x7fca8000c2d0 comp rx=0 tx=0).stop 2024-04-24T09:17:27.601 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.598+0000 7fca90200640 1 -- 172.21.15.83:0/1261128603 shutdown_connections 2024-04-24T09:17:27.601 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.598+0000 7fca90200640 1 --2- 172.21.15.83:0/1261128603 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fca700a3dc0 0x7fca700a6280 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:27.601 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.598+0000 7fca90200640 1 --2- 172.21.15.83:0/1261128603 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fca8819a770 0x7fca8819abf0 unknown :-1 s=CLOSED pgs=2848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:27.601 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.598+0000 7fca90200640 1 --2- 172.21.15.83:0/1261128603 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fca8810a850 0x7fca881a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:27.601 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.598+0000 7fca90200640 1 --2- 172.21.15.83:0/1261128603 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fca88100340 0x7fca881a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:27.602 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.598+0000 7fca90200640 1 -- 172.21.15.83:0/1261128603 >> 172.21.15.83:0/1261128603 conn(0x7fca88077000 msgr2=0x7fca880fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:27.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.598+0000 7f0aa29cc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582d7727860 0x5582d772f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:27.602 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.598+0000 7fca90200640 1 -- 172.21.15.83:0/1261128603 shutdown_connections 2024-04-24T09:17:27.602 INFO:teuthology.task.background_exec.ubuntu@smithi083.front.sepia.ceph.com.smithi083.stderr:2024-04-24T09:17:27.598+0000 7fca90200640 1 -- 172.21.15.83:0/1261128603 wait complete. 2024-04-24T09:17:27.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.599+0000 7f0aa29cc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5582d7728180 0x5582d772fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:27.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.599+0000 7f0aa29cc740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5582d7732850 0x5582d77301e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:27.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.599+0000 7f0aa29cc740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5582d7735630 con 0x5582d7727860 2024-04-24T09:17:27.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.599+0000 7f0aa29cc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5582d7646690 con 0x5582d7728180 2024-04-24T09:17:27.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.599+0000 7f0aa29cc740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5582d77357b0 con 0x5582d7732850 2024-04-24T09:17:27.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.599+0000 7f0aa29cc740 10 client.0 _ll_register_callbacks cb 0x5582d7682650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:17:27.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.599+0000 7f0a9d200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582d7727860 0x5582d772f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:27.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.600+0000 7f0a9d200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582d7727860 0x5582d772f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35582/0 (socket says 172.21.15.83:35582) 2024-04-24T09:17:27.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.600+0000 7f0a9d200640 1 -- 172.21.15.83:0/3314208812 learned_addr learned my addr 172.21.15.83:0/3314208812 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:17:27.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.600+0000 7f0a97e00640 1 --2- 172.21.15.83:0/3314208812 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5582d7728180 0x5582d772fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:27.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.600+0000 7f0a9d200640 1 -- 172.21.15.83:0/3314208812 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5582d7732850 msgr2=0x5582d77301e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-24T09:17:27.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.600+0000 7f0a9d200640 1 --2- 172.21.15.83:0/3314208812 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5582d7732850 0x5582d77301e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:27.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.600+0000 7f0a9d200640 1 -- 172.21.15.83:0/3314208812 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5582d7728180 msgr2=0x5582d772fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:27.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.600+0000 7f0a9d200640 1 --2- 172.21.15.83:0/3314208812 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5582d7728180 0x5582d772fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:27.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.600+0000 7f0a9d200640 1 -- 172.21.15.83:0/3314208812 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582d7645920 con 0x5582d7727860 2024-04-24T09:17:27.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.600+0000 7f0a9d200640 1 --2- 172.21.15.83:0/3314208812 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582d7727860 0x5582d772f760 secure :-1 s=READY pgs=2851 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c09c920 tx=0x7f0a8c0a5420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d83c1e2244cfcbc4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:27.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.600+0000 7f0a7ea00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:17:27.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.601+0000 7f0a7ea00640 1 -- 172.21.15.83:0/3314208812 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a8c0b8070 con 0x5582d7727860 2024-04-24T09:17:27.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.601+0000 7f0a7ea00640 1 -- 172.21.15.83:0/3314208812 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(134..134 src has 1..134) ==== 9583+0+0 (secure 0 0 0) 0x7f0a8c0bbd60 con 0x5582d7727860 2024-04-24T09:17:27.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.601+0000 7f0aa29cc740 1 -- 172.21.15.83:0/3314208812 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5582d769ac10 con 0x5582d7727860 2024-04-24T09:17:27.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.601+0000 7f0a7ea00640 1 -- 172.21.15.83:0/3314208812 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7f0a8c0aa050 con 0x5582d7727860 2024-04-24T09:17:27.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.601+0000 7f0aa29cc740 10 client.24309 fetch_fsmap learned FSMap version 70 2024-04-24T09:17:27.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.601+0000 7f0aa29cc740 1 -- 172.21.15.83:0/3314208812 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) -- 0x5582d74697d0 con 0x5582d7727860 2024-04-24T09:17:27.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.602+0000 7f0a7ea00640 1 -- 172.21.15.83:0/3314208812 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x5582d769ac10 con 0x5582d7727860 2024-04-24T09:17:27.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.602+0000 7f0a7ea00640 1 -- 172.21.15.83:0/3314208812 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 70) ==== 8366+0+0 (secure 0 0 0) 0x7f0a8c0bb030 con 0x5582d7727860 2024-04-24T09:17:27.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.602+0000 7f0aa29cc740 10 client.24309 fetch_fsmap finished waiting for FSMap version 70 2024-04-24T09:17:27.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.602+0000 7f0aa29cc740 10 client.24309 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:17:27.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.602+0000 7f0aa29cc740 20 client.24309 populate_metadata read hostname 'smithi083' 2024-04-24T09:17:27.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.602+0000 7f0aa29cc740 1 --2- 172.21.15.83:0/3314208812 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5582d77427c0 0x5582d7744bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:27.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.602+0000 7f0aa29cc740 4 client.24309 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:17:27.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.602+0000 7f0aa29cc740 1 -- 172.21.15.83:0/3314208812 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5582d7694a80 con 0x5582d77427c0 2024-04-24T09:17:27.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.603+0000 7f0a97e00640 1 --2- 172.21.15.83:0/3314208812 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5582d77427c0 0x5582d7744bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:27.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.604+0000 7f0a97e00640 1 --2- 172.21.15.83:0/3314208812 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5582d77427c0 0x5582d7744bb0 crc :-1 s=READY pgs=633 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22c78b4e29f858e server_cookie=d7da98bac15b44f5 in_seq=0 out_seq=0 2024-04-24T09:17:27.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.604+0000 7f0a7ea00640 10 client.24309 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:17:27.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.605+0000 7f0a7ea00640 1 -- 172.21.15.83:0/3314208812 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5582d7694a80 con 0x5582d77427c0 2024-04-24T09:17:27.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.605+0000 7f0a7ea00640 10 client.24309 handle_command_reply: tid=0 2024-04-24T09:17:27.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.605+0000 7f0aa29cc740 1 client.24309 shutdown 2024-04-24T09:17:27.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.605+0000 7f0aa29cc740 1 -- 172.21.15.83:0/3314208812 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582d7727860 msgr2=0x5582d772f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:27.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.605+0000 7f0aa29cc740 1 --2- 172.21.15.83:0/3314208812 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582d7727860 0x5582d772f760 secure :-1 s=READY pgs=2851 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c09c920 tx=0x7f0a8c0a5420 comp rx=0 tx=0).stop 2024-04-24T09:17:27.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.606+0000 7f0aa29cc740 1 -- 172.21.15.83:0/3314208812 shutdown_connections 2024-04-24T09:17:27.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.606+0000 7f0aa29cc740 1 --2- 172.21.15.83:0/3314208812 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5582d77427c0 0x5582d7744bb0 crc :-1 s=READY pgs=633 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:27.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.606+0000 7f0aa29cc740 1 --2- 172.21.15.83:0/3314208812 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5582d7732850 0x5582d77301e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:27.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.606+0000 7f0aa29cc740 1 --2- 172.21.15.83:0/3314208812 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5582d7728180 0x5582d772fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:27.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.606+0000 7f0aa29cc740 1 --2- 172.21.15.83:0/3314208812 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582d7727860 0x5582d772f760 unknown :-1 s=CLOSED pgs=2851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:27.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.606+0000 7f0aa29cc740 1 -- 172.21.15.83:0/3314208812 >> 172.21.15.83:0/3314208812 conn(0x5582d7723b80 msgr2=0x5582d7729860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:27.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.606+0000 7f0a7ea00640 0 client.24309 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:17:27.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.606+0000 7f0aa29cc740 1 -- 172.21.15.83:0/3314208812 shutdown_connections 2024-04-24T09:17:27.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.606+0000 7f0aa29cc740 1 -- 172.21.15.83:0/3314208812 wait complete. 2024-04-24T09:17:27.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.607+0000 7f0aa29cc740 20 client.24309 trim_cache size 0 max 16384 2024-04-24T09:17:27.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.645+0000 7f0aa29cc740 1 Processor -- start 2024-04-24T09:17:27.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.645+0000 7f0aa29cc740 1 -- start start 2024-04-24T09:17:27.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.646+0000 7f0aa29cc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5582d7727860 0x5582d7729fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:27.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.646+0000 7f0aa29cc740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5582d7728180 0x5582d773e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:27.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.646+0000 7f0aa29cc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582d7732850 0x5582d7747bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:27.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.646+0000 7f0aa29cc740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0a981be460 con 0x5582d7732850 2024-04-24T09:17:27.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.646+0000 7f0aa29cc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0a981bffd0 con 0x5582d7727860 2024-04-24T09:17:27.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.646+0000 7f0aa29cc740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0a981beb30 con 0x5582d7728180 2024-04-24T09:17:27.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.646+0000 7f0a9dc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582d7732850 0x5582d7747bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:27.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.646+0000 7f0a9d200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5582d7727860 0x5582d7729fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:27.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.646+0000 7f0a9d200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5582d7727860 0x5582d7729fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:39630/0 (socket says 172.21.15.83:39630) 2024-04-24T09:17:27.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.647+0000 7f0a9d200640 1 -- 172.21.15.83:0/1953219611 learned_addr learned my addr 172.21.15.83:0/1953219611 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:17:27.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.647+0000 7f0a97e00640 1 --2- 172.21.15.83:0/1953219611 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5582d7728180 0x5582d773e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:27.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.647+0000 7f0a9dc00640 1 -- 172.21.15.83:0/1953219611 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5582d7728180 msgr2=0x5582d773e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:27.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.647+0000 7f0a9dc00640 1 --2- 172.21.15.83:0/1953219611 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5582d7728180 0x5582d773e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:27.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.647+0000 7f0a9dc00640 1 -- 172.21.15.83:0/1953219611 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5582d7727860 msgr2=0x5582d7729fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:27.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.647+0000 7f0a9dc00640 1 --2- 172.21.15.83:0/1953219611 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5582d7727860 0x5582d7729fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:27.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.647+0000 7f0a9dc00640 1 -- 172.21.15.83:0/1953219611 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582d739f3f0 con 0x5582d7732850 2024-04-24T09:17:27.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.647+0000 7f0a9d200640 1 --2- 172.21.15.83:0/1953219611 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5582d7727860 0x5582d7729fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:17:27.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.647+0000 7f0a9dc00640 1 --2- 172.21.15.83:0/1953219611 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582d7732850 0x5582d7747bd0 secure :-1 s=READY pgs=2852 cs=0 l=1 rev1=1 crypto rx=0x7f0a90013e60 tx=0x7f0a90016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e56cb788f37d7e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:27.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.648+0000 7f0a89600640 1 -- 172.21.15.83:0/1953219611 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a900044b0 con 0x5582d7732850 2024-04-24T09:17:27.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.648+0000 7f0a89600640 1 -- 172.21.15.83:0/1953219611 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0a90025020 con 0x5582d7732850 2024-04-24T09:17:27.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.648+0000 7f0a89600640 1 -- 172.21.15.83:0/1953219611 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a90011440 con 0x5582d7732850 2024-04-24T09:17:27.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.649+0000 7f0aa29cc740 1 -- 172.21.15.83:0/1953219611 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582d7732850 msgr2=0x5582d7747bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:27.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.649+0000 7f0aa29cc740 1 --2- 172.21.15.83:0/1953219611 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582d7732850 0x5582d7747bd0 secure :-1 s=READY pgs=2852 cs=0 l=1 rev1=1 crypto rx=0x7f0a90013e60 tx=0x7f0a90016000 comp rx=0 tx=0).stop 2024-04-24T09:17:27.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.649+0000 7f0aa29cc740 1 -- 172.21.15.83:0/1953219611 shutdown_connections 2024-04-24T09:17:27.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.649+0000 7f0aa29cc740 1 --2- 172.21.15.83:0/1953219611 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582d7732850 0x5582d7747bd0 unknown :-1 s=CLOSED pgs=2852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:27.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.649+0000 7f0aa29cc740 1 --2- 172.21.15.83:0/1953219611 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5582d7728180 0x5582d773e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:27.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.649+0000 7f0aa29cc740 1 --2- 172.21.15.83:0/1953219611 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5582d7727860 0x5582d7729fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:27.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.649+0000 7f0aa29cc740 1 -- 172.21.15.83:0/1953219611 >> 172.21.15.83:0/1953219611 conn(0x5582d7723b80 msgr2=0x5582d773da00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:27.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.649+0000 7f0aa29cc740 1 -- 172.21.15.83:0/1953219611 shutdown_connections 2024-04-24T09:17:27.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.649+0000 7f0aa29cc740 1 -- 172.21.15.83:0/1953219611 wait complete. 2024-04-24T09:17:27.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.650+0000 7f0aa29cc740 1 -- start start 2024-04-24T09:17:27.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.650+0000 7f0aa29cc740 1 Processor -- start 2024-04-24T09:17:27.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.650+0000 7f0aa29cc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5582d7727860 0x5582d7747e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:27.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.650+0000 7f0aa29cc740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5582d7728180 0x5582d7745170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:27.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.650+0000 7f0aa29cc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582d7732850 0x5582d77456b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:27.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.650+0000 7f0aa29cc740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0a981bf900 con 0x5582d7732850 2024-04-24T09:17:27.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.650+0000 7f0aa29cc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0a981bf230 con 0x5582d7727860 2024-04-24T09:17:27.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.650+0000 7f0aa29cc740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0a981bc460 con 0x5582d7728180 2024-04-24T09:17:27.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.650+0000 7f0aa29cc740 10 client.0 _ll_register_callbacks cb 0x7f0a900099c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:17:27.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.650+0000 7f0a9d200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5582d7727860 0x5582d7747e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:27.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.650+0000 7f0a9dc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582d7732850 0x5582d77456b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:27.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.650+0000 7f0a9d200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5582d7727860 0x5582d7747e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:39634/0 (socket says 172.21.15.83:39634) 2024-04-24T09:17:27.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.650+0000 7f0a9d200640 1 -- 172.21.15.83:0/1317352596 learned_addr learned my addr 172.21.15.83:0/1317352596 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:17:27.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.651+0000 7f0a97e00640 1 --2- 172.21.15.83:0/1317352596 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5582d7728180 0x5582d7745170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:27.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.651+0000 7f0a9d200640 1 -- 172.21.15.83:0/1317352596 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5582d7728180 msgr2=0x5582d7745170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:27.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.651+0000 7f0a9d200640 1 --2- 172.21.15.83:0/1317352596 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5582d7728180 0x5582d7745170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:27.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.651+0000 7f0a9d200640 1 -- 172.21.15.83:0/1317352596 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582d7732850 msgr2=0x5582d77456b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:27.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.651+0000 7f0a9d200640 1 --2- 172.21.15.83:0/1317352596 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582d7732850 0x5582d77456b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:27.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.651+0000 7f0a9d200640 1 -- 172.21.15.83:0/1317352596 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582d735f0f0 con 0x5582d7727860 2024-04-24T09:17:27.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.651+0000 7f0a9dc00640 1 --2- 172.21.15.83:0/1317352596 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582d7732850 0x5582d77456b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T09:17:27.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.651+0000 7f0a9d200640 1 --2- 172.21.15.83:0/1317352596 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5582d7727860 0x5582d7747e00 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c09ac40 tx=0x7f0a8c09c760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be07c89169e53eac server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:27.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.651+0000 7f0a7ea00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-24T09:17:27.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.652+0000 7f0a7ea00640 1 -- 172.21.15.83:0/1317352596 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a8c0b8050 con 0x5582d7727860 2024-04-24T09:17:27.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.652+0000 7f0a7ea00640 1 -- 172.21.15.83:0/1317352596 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(134..134 src has 1..134) ==== 9583+0+0 (secure 0 0 0) 0x7f0a8c09bde0 con 0x5582d7727860 2024-04-24T09:17:27.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.652+0000 7f0aa29cc740 1 -- 172.21.15.83:0/1317352596 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a90025020 con 0x5582d7727860 2024-04-24T09:17:27.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.652+0000 7f0a7ea00640 1 -- 172.21.15.83:0/1317352596 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7f0a8c018cb0 con 0x5582d7727860 2024-04-24T09:17:27.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.652+0000 7f0aa29cc740 10 client.41384 fetch_fsmap learned FSMap version 70 2024-04-24T09:17:27.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.652+0000 7f0aa29cc740 1 -- 172.21.15.83:0/1317352596 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) -- 0x5582d7652810 con 0x5582d7727860 2024-04-24T09:17:27.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.652+0000 7f0a7ea00640 1 -- 172.21.15.83:0/1317352596 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0a90025020 con 0x5582d7727860 2024-04-24T09:17:27.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.653+0000 7f0a7ea00640 1 -- 172.21.15.83:0/1317352596 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 70) ==== 8366+0+0 (secure 0 0 0) 0x7f0a8c0bb040 con 0x5582d7727860 2024-04-24T09:17:27.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.653+0000 7f0aa29cc740 10 client.41384 fetch_fsmap finished waiting for FSMap version 70 2024-04-24T09:17:27.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.653+0000 7f0aa29cc740 10 client.41384 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:17:27.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.653+0000 7f0aa29cc740 20 client.41384 populate_metadata read hostname 'smithi083' 2024-04-24T09:17:27.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.653+0000 7f0aa29cc740 1 --2- 172.21.15.83:0/1317352596 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5582d77a8810 0x5582d77aac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:27.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.653+0000 7f0aa29cc740 4 client.41384 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T09:17:27.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.653+0000 7f0aa29cc740 1 -- 172.21.15.83:0/1317352596 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5582d7610360 con 0x5582d77a8810 2024-04-24T09:17:27.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.654+0000 7f0a97e00640 1 --2- 172.21.15.83:0/1317352596 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5582d77a8810 0x5582d77aac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:27.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.655+0000 7f0a97e00640 1 --2- 172.21.15.83:0/1317352596 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5582d77a8810 0x5582d77aac00 crc :-1 s=READY pgs=634 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72404798eb39baaa server_cookie=2b462b51ea742b64 in_seq=0 out_seq=0 2024-04-24T09:17:27.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.655+0000 7f0a7ea00640 10 client.41384 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:17:27.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.655+0000 7f0a7ea00640 1 -- 172.21.15.83:0/1317352596 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5582d7610360 con 0x5582d77a8810 2024-04-24T09:17:27.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.655+0000 7f0a7ea00640 10 client.41384 handle_command_reply: tid=0 2024-04-24T09:17:27.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.655+0000 7f0aa29cc740 1 client.41384 shutdown 2024-04-24T09:17:27.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.655+0000 7f0aa29cc740 1 -- 172.21.15.83:0/1317352596 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5582d7727860 msgr2=0x5582d7747e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:27.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.655+0000 7f0aa29cc740 1 --2- 172.21.15.83:0/1317352596 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5582d7727860 0x5582d7747e00 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c09ac40 tx=0x7f0a8c09c760 comp rx=0 tx=0).stop 2024-04-24T09:17:27.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.656+0000 7f0aa29cc740 1 -- 172.21.15.83:0/1317352596 shutdown_connections 2024-04-24T09:17:27.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.656+0000 7f0aa29cc740 1 --2- 172.21.15.83:0/1317352596 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x5582d77a8810 0x5582d77aac00 crc :-1 s=READY pgs=634 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:27.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.656+0000 7f0aa29cc740 1 --2- 172.21.15.83:0/1317352596 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582d7732850 0x5582d77456b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:27.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.656+0000 7f0aa29cc740 1 --2- 172.21.15.83:0/1317352596 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5582d7728180 0x5582d7745170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:27.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.656+0000 7f0aa29cc740 1 --2- 172.21.15.83:0/1317352596 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5582d7727860 0x5582d7747e00 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:27.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.656+0000 7f0aa29cc740 1 -- 172.21.15.83:0/1317352596 >> 172.21.15.83:0/1317352596 conn(0x5582d7723b80 msgr2=0x5582d7731820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:27.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.656+0000 7f0a7ea00640 0 client.41384 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:17:27.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.656+0000 7f0aa29cc740 1 -- 172.21.15.83:0/1317352596 shutdown_connections 2024-04-24T09:17:27.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.656+0000 7f0aa29cc740 1 -- 172.21.15.83:0/1317352596 wait complete. 2024-04-24T09:17:27.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.657+0000 7f0aa29cc740 20 client.41384 trim_cache size 0 max 16384 2024-04-24T09:17:27.658 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T09:17:27.658 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T09:17:27.658 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-04-24T09:17:27.658 INFO:teuthology.orchestra.run.smithi083.stdout: "bd8e2894-a168-48cd-8991-67fabbe2e832": { 2024-04-24T09:17:27.658 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-04-24T09:17:27.658 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "bd8e2894-a168-48cd-8991-67fabbe2e832", 2024-04-24T09:17:27.658 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-04-24T09:17:27.658 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:17:27.658 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:17:27.659 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T09:17:27.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.657+0000 7f0a9fc00640 1 -- 172.21.15.83:0/1194977099 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f0a800a3e20 msgr2=0x7f0a800a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:27.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.657+0000 7f0a9fc00640 1 --2- 172.21.15.83:0/1194977099 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f0a800a3e20 0x7f0a800a62e0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f0a90005fd0 tx=0x7f0a90004a40 comp rx=0 tx=0).stop 2024-04-24T09:17:27.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.657+0000 7f0a9fc00640 1 -- 172.21.15.83:0/1194977099 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0a98109210 msgr2=0x7f0a98112e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:27.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.657+0000 7f0a9fc00640 1 --2- 172.21.15.83:0/1194977099 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0a98109210 0x7f0a98112e20 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c00c920 tx=0x7f0a8c00cde0 comp rx=0 tx=0).stop 2024-04-24T09:17:27.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.658+0000 7f0a9fc00640 1 -- 172.21.15.83:0/1194977099 shutdown_connections 2024-04-24T09:17:27.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.658+0000 7f0a9fc00640 1 --2- 172.21.15.83:0/1194977099 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f0a800a3e20 0x7f0a800a62e0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:27.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.658+0000 7f0a9fc00640 1 --2- 172.21.15.83:0/1194977099 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a98115150 0x7f0a981138a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:27.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.658+0000 7f0a9fc00640 1 --2- 172.21.15.83:0/1194977099 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a98109b30 0x7f0a98113360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:27.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.658+0000 7f0a9fc00640 1 --2- 172.21.15.83:0/1194977099 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0a98109210 0x7f0a98112e20 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:27.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.658+0000 7f0a9fc00640 1 -- 172.21.15.83:0/1194977099 >> 172.21.15.83:0/1194977099 conn(0x7f0a9806d110 msgr2=0x7f0a98071f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:27.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.658+0000 7f0a9fc00640 1 -- 172.21.15.83:0/1194977099 shutdown_connections 2024-04-24T09:17:27.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:27.658+0000 7f0a9fc00640 1 -- 172.21.15.83:0/1194977099 wait complete. 2024-04-24T09:17:27.686 INFO:tasks.cephfs.filesystem:scrub status for tag:bd8e2894-a168-48cd-8991-67fabbe2e832 - {'path': '/', 'tag': 'bd8e2894-a168-48cd-8991-67fabbe2e832', 'options': 'recursive,force'} 2024-04-24T09:17:28.230 INFO:tasks.workunit.client.0.smithi083.stdout: 10 87413 6.48 MB/sec execute 599 sec latency 197.176 ms 2024-04-24T09:17:28.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:17:28 smithi138 ceph-mon[31441]: pgmap v818: 129 pgs: 129 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.2 MiB/s wr, 229 op/s 2024-04-24T09:17:28.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:17:28 smithi138 ceph-mon[31441]: scrub queued for path: / 2024-04-24T09:17:28.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:17:28 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:17:28.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:17:28 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:17:28.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:17:28 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:17:28.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:17:28 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:17:28.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:17:28 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:17:28.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:17:28 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:17:28.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:17:28 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:17:28.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:17:28 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:17:28.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:17:28 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:17:28.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:17:28 smithi138 ceph-mon[31441]: scrub summary: active paths [/] 2024-04-24T09:17:28.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:17:28 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:17:28.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:17:28 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:17:28.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:17:28 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:17:28.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:17:28 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:17:28.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:17:28 smithi138 ceph-mon[31441]: scrub summary: idle+waiting paths [/] 2024-04-24T09:17:28.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:17:28 smithi189 ceph-mon[31791]: pgmap v818: 129 pgs: 129 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.2 MiB/s wr, 229 op/s 2024-04-24T09:17:28.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:17:28 smithi189 ceph-mon[31791]: scrub queued for path: / 2024-04-24T09:17:28.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:17:28 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:17:28.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:17:28 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:17:28.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:17:28 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:17:28.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:17:28 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:17:28.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:17:28 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:17:28.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:17:28 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:17:28.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:17:28 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:17:28.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:17:28 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:17:28.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:17:28 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:17:28.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:17:28 smithi189 ceph-mon[31791]: scrub summary: active paths [/] 2024-04-24T09:17:28.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:17:28 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:17:28.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:17:28 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:17:28.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:17:28 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:17:28.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:17:28 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:17:28.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:17:28 smithi189 ceph-mon[31791]: scrub summary: idle+waiting paths [/] 2024-04-24T09:17:28.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:17:28 smithi083 ceph-mon[27907]: pgmap v818: 129 pgs: 129 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.2 MiB/s wr, 229 op/s 2024-04-24T09:17:28.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:17:28 smithi083 ceph-mon[27907]: scrub queued for path: / 2024-04-24T09:17:28.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:17:28 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:17:28.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:17:28 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:17:28.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:17:28 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:17:28.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:17:28 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:17:28.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:17:28 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:17:28.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:17:28 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:17:28.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:17:28 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:17:28.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:17:28 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:17:28.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:17:28 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:17:28.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:17:28 smithi083 ceph-mon[27907]: scrub summary: active paths [/] 2024-04-24T09:17:28.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:17:28 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:17:28.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:17:28 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:17:28.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:17:28 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:17:28.555 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:17:28 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:17:28.555 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:17:28 smithi083 ceph-mon[27907]: scrub summary: idle+waiting paths [/] 2024-04-24T09:17:29.230 INFO:tasks.workunit.client.0.smithi083.stdout: 10 cleanup 600 sec 2024-04-24T09:17:30.230 INFO:tasks.workunit.client.0.smithi083.stdout: 10 cleanup 601 sec 2024-04-24T09:17:30.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:17:30 smithi138 ceph-mon[31441]: pgmap v819: 129 pgs: 129 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 6.2 MiB/s wr, 404 op/s 2024-04-24T09:17:30.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:17:30 smithi189 ceph-mon[31791]: pgmap v819: 129 pgs: 129 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 6.2 MiB/s wr, 404 op/s 2024-04-24T09:17:30.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:17:30 smithi083 ceph-mon[27907]: pgmap v819: 129 pgs: 129 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 6.2 MiB/s wr, 404 op/s 2024-04-24T09:17:31.230 INFO:tasks.workunit.client.0.smithi083.stdout: 4 cleanup 602 sec 2024-04-24T09:17:32.231 INFO:tasks.workunit.client.0.smithi083.stdout: 3 cleanup 603 sec 2024-04-24T09:17:32.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:17:32 smithi138 ceph-mon[31441]: pgmap v820: 129 pgs: 129 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.3 MiB/s wr, 274 op/s 2024-04-24T09:17:32.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:17:32 smithi138 ceph-mon[31441]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:17:32.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:17:32 smithi189 ceph-mon[31791]: pgmap v820: 129 pgs: 129 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.3 MiB/s wr, 274 op/s 2024-04-24T09:17:32.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:17:32 smithi189 ceph-mon[31791]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:17:32.552 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:17:32 smithi083 ceph-mon[27907]: pgmap v820: 129 pgs: 129 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.3 MiB/s wr, 274 op/s 2024-04-24T09:17:32.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:17:32 smithi083 ceph-mon[27907]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:17:33.231 INFO:tasks.workunit.client.0.smithi083.stdout: 1 cleanup 604 sec 2024-04-24T09:17:33.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:17:33 smithi138 ceph-mon[31441]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:17:33.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:17:33 smithi189 ceph-mon[31791]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:17:33.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:17:33 smithi083 ceph-mon[27907]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:17:34.183 INFO:tasks.workunit.client.0.smithi083.stdout: 0 cleanup 605 sec 2024-04-24T09:17:34.183 INFO:tasks.workunit.client.0.smithi083.stdout: 2024-04-24T09:17:34.184 INFO:tasks.workunit.client.0.smithi083.stdout: Operation Count AvgLat MaxLat 2024-04-24T09:17:34.184 INFO:tasks.workunit.client.0.smithi083.stdout: ---------------------------------------- 2024-04-24T09:17:34.184 INFO:tasks.workunit.client.0.smithi083.stdout: NTCreateX 124166 14.786 2819.482 2024-04-24T09:17:34.184 INFO:tasks.workunit.client.0.smithi083.stdout: Close 91178 0.279 331.171 2024-04-24T09:17:34.184 INFO:tasks.workunit.client.0.smithi083.stdout: Rename 5283 34.058 2832.061 2024-04-24T09:17:34.184 INFO:tasks.workunit.client.0.smithi083.stdout: Unlink 25129 12.836 2782.301 2024-04-24T09:17:34.184 INFO:tasks.workunit.client.0.smithi083.stdout: Qpathinfo 112440 9.947 2799.018 2024-04-24T09:17:34.184 INFO:tasks.workunit.client.0.smithi083.stdout: Qfileinfo 19623 0.568 295.246 2024-04-24T09:17:34.184 INFO:tasks.workunit.client.0.smithi083.stdout: Qfsinfo 20770 1.460 265.905 2024-04-24T09:17:34.184 INFO:tasks.workunit.client.0.smithi083.stdout: Sfileinfo 10130 4.712 388.003 2024-04-24T09:17:34.184 INFO:tasks.workunit.client.0.smithi083.stdout: Find 43665 51.392 2864.978 2024-04-24T09:17:34.184 INFO:tasks.workunit.client.0.smithi083.stdout: WriteX 61417 1.077 395.587 2024-04-24T09:17:34.184 INFO:tasks.workunit.client.0.smithi083.stdout: ReadX 195636 0.022 65.445 2024-04-24T09:17:34.184 INFO:tasks.workunit.client.0.smithi083.stdout: LockX 408 2.238 10.016 2024-04-24T09:17:34.184 INFO:tasks.workunit.client.0.smithi083.stdout: UnlockX 408 2.142 35.712 2024-04-24T09:17:34.185 INFO:tasks.workunit.client.0.smithi083.stdout: Flush 8713 12.255 474.049 2024-04-24T09:17:34.185 INFO:tasks.workunit.client.0.smithi083.stdout: 2024-04-24T09:17:34.185 INFO:tasks.workunit.client.0.smithi083.stdout:Throughput 6.4824 MB/sec 10 clients 10 procs max_latency=2864.985 ms 2024-04-24T09:17:34.186 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-24T09:17:34.186 DEBUG:teuthology.orchestra.run.smithi083:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-24T09:17:34.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:17:34 smithi138 ceph-mon[31441]: pgmap v821: 129 pgs: 129 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 6.2 MiB/s wr, 406 op/s 2024-04-24T09:17:34.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:17:34 smithi189 ceph-mon[31791]: pgmap v821: 129 pgs: 129 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 6.2 MiB/s wr, 406 op/s 2024-04-24T09:17:34.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:17:34 smithi083 ceph-mon[27907]: pgmap v821: 129 pgs: 129 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 6.2 MiB/s wr, 406 op/s 2024-04-24T09:17:36.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:17:36 smithi138 ceph-mon[31441]: pgmap v822: 129 pgs: 129 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.9 MiB/s wr, 307 op/s 2024-04-24T09:17:36.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:17:36 smithi138 ceph-mon[31441]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:17:36.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:17:36 smithi189 ceph-mon[31791]: pgmap v822: 129 pgs: 129 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.9 MiB/s wr, 307 op/s 2024-04-24T09:17:36.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:17:36 smithi189 ceph-mon[31791]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:17:36.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:17:36 smithi083 ceph-mon[27907]: pgmap v822: 129 pgs: 129 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.9 MiB/s wr, 307 op/s 2024-04-24T09:17:36.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:17:36 smithi083 ceph-mon[27907]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:17:38.288 INFO:tasks.workunit:Stopping ['suites/dbench.sh'] on client.0... 2024-04-24T09:17:38.289 DEBUG:teuthology.orchestra.run.smithi083:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-24T09:17:38.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:17:38 smithi083 ceph-mon[27907]: pgmap v823: 129 pgs: 129 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.9 MiB/s wr, 307 op/s 2024-04-24T09:17:38.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:17:38 smithi083 ceph-mon[27907]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:17:38.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:17:38 smithi138 ceph-mon[31441]: pgmap v823: 129 pgs: 129 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.9 MiB/s wr, 307 op/s 2024-04-24T09:17:38.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:17:38 smithi138 ceph-mon[31441]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:17:38.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:17:38 smithi189 ceph-mon[31791]: pgmap v823: 129 pgs: 129 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.9 MiB/s wr, 307 op/s 2024-04-24T09:17:38.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:17:38 smithi189 ceph-mon[31791]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:17:39.555 DEBUG:teuthology.parallel:result is None 2024-04-24T09:17:39.556 DEBUG:teuthology.orchestra.run.smithi083:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-04-24T09:17:39.661 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-04-24T09:17:39.662 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-04-24T09:17:39.672 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi083.front.sepia.ceph.com 2024-04-24T09:17:39.698 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-04-24T09:17:39.707 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-04-24T09:17:39.708 INFO:teuthology.task.exec:Executing custom commands... 2024-04-24T09:17:39.708 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi083.front.sepia.ceph.com 2024-04-24T09:17:39.708 DEBUG:teuthology.orchestra.run.smithi083:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-04-24T09:17:39.875 DEBUG:teuthology.orchestra.run.smithi083:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-04-24T09:17:39.974 DEBUG:teuthology.orchestra.run.smithi083:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-24T09:17:40.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:40.171+0000 7f0505200640 1 -- 172.21.15.83:0/2322595497 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05001042b0 msgr2=0x7f0500104690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:40.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:40.171+0000 7f0505200640 1 --2- 172.21.15.83:0/2322595497 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05001042b0 0x7f0500104690 secure :-1 s=READY pgs=2853 cs=0 l=1 rev1=1 crypto rx=0x7f04f0009900 tx=0x7f04f002a3f0 comp rx=0 tx=0).stop 2024-04-24T09:17:40.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:40.171+0000 7f0505200640 1 -- 172.21.15.83:0/2322595497 shutdown_connections 2024-04-24T09:17:40.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:40.171+0000 7f0505200640 1 --2- 172.21.15.83:0/2322595497 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0500109380 0x7f0500110f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:40.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:40.171+0000 7f0505200640 1 --2- 172.21.15.83:0/2322595497 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0500104c60 0x7f0500108ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:40.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:40.171+0000 7f0505200640 1 --2- 172.21.15.83:0/2322595497 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05001042b0 0x7f0500104690 unknown :-1 s=CLOSED pgs=2853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:40.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:40.171+0000 7f0505200640 1 -- 172.21.15.83:0/2322595497 >> 172.21.15.83:0/2322595497 conn(0x7f05000ffb10 msgr2=0x7f0500101f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:40.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:40.171+0000 7f0505200640 1 -- 172.21.15.83:0/2322595497 shutdown_connections 2024-04-24T09:17:40.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:40.171+0000 7f0505200640 1 -- 172.21.15.83:0/2322595497 wait complete. 2024-04-24T09:17:40.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:40.172+0000 7f0505200640 1 Processor -- start 2024-04-24T09:17:40.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:40.172+0000 7f0505200640 1 -- start start 2024-04-24T09:17:40.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:40.172+0000 7f0505200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05001042b0 0x7f05001a05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:40.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:40.172+0000 7f0505200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0500109380 0x7f05001a0b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:40.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:40.173+0000 7f0505200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f050019a6f0 0x7f050019ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:40.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:40.173+0000 7f0505200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0500114310 con 0x7f050019a6f0 2024-04-24T09:17:40.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:40.173+0000 7f0505200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0500114190 con 0x7f05001042b0 2024-04-24T09:17:40.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:40.173+0000 7f0505200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0500114490 con 0x7f0500109380 2024-04-24T09:17:40.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:40.173+0000 7f04ffe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f050019a6f0 0x7f050019ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:40.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:40.173+0000 7f04ffe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f050019a6f0 0x7f050019ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47300/0 (socket says 172.21.15.83:47300) 2024-04-24T09:17:40.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:40.173+0000 7f04ffe00640 1 -- 172.21.15.83:0/1181504869 learned_addr learned my addr 172.21.15.83:0/1181504869 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:17:40.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:40.173+0000 7f04ff400640 1 --2- 172.21.15.83:0/1181504869 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05001042b0 0x7f05001a05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:40.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:40.173+0000 7f04fea00640 1 --2- 172.21.15.83:0/1181504869 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0500109380 0x7f05001a0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:40.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:40.173+0000 7f04ffe00640 1 -- 172.21.15.83:0/1181504869 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0500109380 msgr2=0x7f05001a0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:40.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:40.173+0000 7f04ffe00640 1 --2- 172.21.15.83:0/1181504869 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0500109380 0x7f05001a0b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:40.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:40.173+0000 7f04ffe00640 1 -- 172.21.15.83:0/1181504869 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05001042b0 msgr2=0x7f05001a05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:40.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:40.173+0000 7f04ffe00640 1 --2- 172.21.15.83:0/1181504869 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05001042b0 0x7f05001a05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:40.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:40.173+0000 7f04ffe00640 1 -- 172.21.15.83:0/1181504869 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f050019b2f0 con 0x7f050019a6f0 2024-04-24T09:17:40.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:40.173+0000 7f04ff400640 1 --2- 172.21.15.83:0/1181504869 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05001042b0 0x7f05001a05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:17:40.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:40.174+0000 7f04ffe00640 1 --2- 172.21.15.83:0/1181504869 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f050019a6f0 0x7f050019ab70 secure :-1 s=READY pgs=2854 cs=0 l=1 rev1=1 crypto rx=0x7f04f800cad0 tx=0x7f04f800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:40.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:40.174+0000 7f04f7e00640 1 -- 172.21.15.83:0/1181504869 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f800ae20 con 0x7f050019a6f0 2024-04-24T09:17:40.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:40.174+0000 7f04f7e00640 1 -- 172.21.15.83:0/1181504869 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f04f8005020 con 0x7f050019a6f0 2024-04-24T09:17:40.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:40.174+0000 7f0505200640 1 -- 172.21.15.83:0/1181504869 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05001a7380 con 0x7f050019a6f0 2024-04-24T09:17:40.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:40.174+0000 7f04f7e00640 1 -- 172.21.15.83:0/1181504869 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f800f6b0 con 0x7f050019a6f0 2024-04-24T09:17:40.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:40.174+0000 7f0505200640 1 -- 172.21.15.83:0/1181504869 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05001a7730 con 0x7f050019a6f0 2024-04-24T09:17:40.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:40.175+0000 7f0505200640 1 -- 172.21.15.83:0/1181504869 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0500105780 con 0x7f050019a6f0 2024-04-24T09:17:40.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:40.177+0000 7f04f7e00640 1 -- 172.21.15.83:0/1181504869 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f04f80051d0 con 0x7f050019a6f0 2024-04-24T09:17:40.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:40.179+0000 7f04f7e00640 1 --2- 172.21.15.83:0/1181504869 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f04e40a3d10 0x7f04e40a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:40.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:40.179+0000 7f04f7e00640 1 -- 172.21.15.83:0/1181504869 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 9583+0+0 (secure 0 0 0) 0x7f04f80d0d20 con 0x7f050019a6f0 2024-04-24T09:17:40.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:40.179+0000 7f04ff400640 1 --2- 172.21.15.83:0/1181504869 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f04e40a3d10 0x7f04e40a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:40.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:40.180+0000 7f04ff400640 1 --2- 172.21.15.83:0/1181504869 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f04e40a3d10 0x7f04e40a61d0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f04f0005ab0 tx=0x7f04f0000bc0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:40.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:40.183+0000 7f04f7e00640 1 -- 172.21.15.83:0/1181504869 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f04f8014030 con 0x7f050019a6f0 2024-04-24T09:17:40.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:40.412+0000 7f0505200640 1 -- 172.21.15.83:0/1181504869 --> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f0500104c60 con 0x7f04e40a3d10 2024-04-24T09:17:40.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:40.434+0000 7f04f7e00640 1 -- 172.21.15.83:0/1181504869 <== mgr.15579 v2:172.21.15.83:6800/3594769126 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+351 (secure 0 0 0) 0x7f0500104c60 con 0x7f04e40a3d10 2024-04-24T09:17:40.434 INFO:teuthology.orchestra.run.smithi083.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-24T00:00:00", "created": "2024-04-24T08:51:46", "first": "2024-04-24T08:52:00", "last": "2024-04-24T09:17:00", "last_pruned": "2024-04-24T09:17:00", "created_count": 26, "pruned_count": 19, "active": true} 2024-04-24T09:17:40.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:40.436+0000 7f0505200640 1 -- 172.21.15.83:0/1181504869 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f04e40a3d10 msgr2=0x7f04e40a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:40.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:40.437+0000 7f0505200640 1 --2- 172.21.15.83:0/1181504869 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f04e40a3d10 0x7f04e40a61d0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f04f0005ab0 tx=0x7f04f0000bc0 comp rx=0 tx=0).stop 2024-04-24T09:17:40.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:40.437+0000 7f0505200640 1 -- 172.21.15.83:0/1181504869 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f050019a6f0 msgr2=0x7f050019ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:40.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:40.437+0000 7f0505200640 1 --2- 172.21.15.83:0/1181504869 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f050019a6f0 0x7f050019ab70 secure :-1 s=READY pgs=2854 cs=0 l=1 rev1=1 crypto rx=0x7f04f800cad0 tx=0x7f04f800cf90 comp rx=0 tx=0).stop 2024-04-24T09:17:40.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:40.437+0000 7f0505200640 1 -- 172.21.15.83:0/1181504869 shutdown_connections 2024-04-24T09:17:40.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:40.437+0000 7f0505200640 1 --2- 172.21.15.83:0/1181504869 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f04e40a3d10 0x7f04e40a61d0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:40.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:40.437+0000 7f0505200640 1 --2- 172.21.15.83:0/1181504869 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f050019a6f0 0x7f050019ab70 unknown :-1 s=CLOSED pgs=2854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:40.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:40.437+0000 7f0505200640 1 --2- 172.21.15.83:0/1181504869 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0500109380 0x7f05001a0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:40.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:40.437+0000 7f0505200640 1 --2- 172.21.15.83:0/1181504869 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05001042b0 0x7f05001a05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:40.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:40.437+0000 7f0505200640 1 -- 172.21.15.83:0/1181504869 >> 172.21.15.83:0/1181504869 conn(0x7f05000ffb10 msgr2=0x7f0500101e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:40.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:40.437+0000 7f0505200640 1 -- 172.21.15.83:0/1181504869 shutdown_connections 2024-04-24T09:17:40.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:40.437+0000 7f0505200640 1 -- 172.21.15.83:0/1181504869 wait complete. 2024-04-24T09:17:40.455 DEBUG:teuthology.orchestra.run.smithi083:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-24T09:17:40.507 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:17:40 smithi138 ceph-mon[31441]: pgmap v824: 129 pgs: 129 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.3 MiB/s wr, 396 op/s 2024-04-24T09:17:40.507 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:17:40 smithi138 ceph-mon[31441]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:17:40.524 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:17:40 smithi083 ceph-mon[27907]: pgmap v824: 129 pgs: 129 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.3 MiB/s wr, 396 op/s 2024-04-24T09:17:40.524 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:17:40 smithi083 ceph-mon[27907]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:17:40.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:40.653+0000 7fba49e00640 1 -- 172.21.15.83:0/2286554189 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba44076530 msgr2=0x7fba44076910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:40.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:40.653+0000 7fba49e00640 1 --2- 172.21.15.83:0/2286554189 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba44076530 0x7fba44076910 secure :-1 s=READY pgs=2855 cs=0 l=1 rev1=1 crypto rx=0x7fba34009900 tx=0x7fba3402a3f0 comp rx=0 tx=0).stop 2024-04-24T09:17:40.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:40.653+0000 7fba49e00640 1 -- 172.21.15.83:0/2286554189 shutdown_connections 2024-04-24T09:17:40.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:40.653+0000 7fba49e00640 1 --2- 172.21.15.83:0/2286554189 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba44112ea0 0x7fba44115290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:40.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:40.653+0000 7fba49e00640 1 --2- 172.21.15.83:0/2286554189 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fba44076e50 0x7fba44112960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:40.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:40.653+0000 7fba49e00640 1 --2- 172.21.15.83:0/2286554189 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba44076530 0x7fba44076910 unknown :-1 s=CLOSED pgs=2855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:40.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:40.653+0000 7fba49e00640 1 -- 172.21.15.83:0/2286554189 >> 172.21.15.83:0/2286554189 conn(0x7fba440ffd30 msgr2=0x7fba44102150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:40.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:40.654+0000 7fba49e00640 1 -- 172.21.15.83:0/2286554189 shutdown_connections 2024-04-24T09:17:40.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:40.654+0000 7fba49e00640 1 -- 172.21.15.83:0/2286554189 wait complete. 2024-04-24T09:17:40.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:40.654+0000 7fba49e00640 1 Processor -- start 2024-04-24T09:17:40.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:40.655+0000 7fba49e00640 1 -- start start 2024-04-24T09:17:40.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:40.655+0000 7fba49e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fba44076530 0x7fba4419eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:40.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:40.655+0000 7fba49e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba44076e50 0x7fba4419f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:40.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:40.655+0000 7fba49e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba44112ea0 0x7fba441a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:40.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:40.655+0000 7fba49e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fba44118960 con 0x7fba44076e50 2024-04-24T09:17:40.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:40.655+0000 7fba49e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fba441187e0 con 0x7fba44112ea0 2024-04-24T09:17:40.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:40.655+0000 7fba49e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fba44118ae0 con 0x7fba44076530 2024-04-24T09:17:40.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:40.655+0000 7fba42000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba44076e50 0x7fba4419f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:40.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:40.655+0000 7fba42000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba44076e50 0x7fba4419f420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47312/0 (socket says 172.21.15.83:47312) 2024-04-24T09:17:40.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:40.655+0000 7fba42000640 1 -- 172.21.15.83:0/921124320 learned_addr learned my addr 172.21.15.83:0/921124320 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:17:40.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:40.656+0000 7fba42a00640 1 --2- 172.21.15.83:0/921124320 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fba44076530 0x7fba4419eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:40.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:40.656+0000 7fba42000640 1 -- 172.21.15.83:0/921124320 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fba44076530 msgr2=0x7fba4419eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:40.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:40.656+0000 7fba42000640 1 --2- 172.21.15.83:0/921124320 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fba44076530 0x7fba4419eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:40.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:40.656+0000 7fba42000640 1 -- 172.21.15.83:0/921124320 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba44112ea0 msgr2=0x7fba441a3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:40.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:40.656+0000 7fba42000640 1 --2- 172.21.15.83:0/921124320 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba44112ea0 0x7fba441a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:40.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:40.656+0000 7fba42000640 1 -- 172.21.15.83:0/921124320 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba441a3e80 con 0x7fba44076e50 2024-04-24T09:17:40.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:40.656+0000 7fba42a00640 1 --2- 172.21.15.83:0/921124320 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fba44076530 0x7fba4419eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:17:40.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:40.656+0000 7fba42000640 1 --2- 172.21.15.83:0/921124320 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba44076e50 0x7fba4419f420 secure :-1 s=READY pgs=2856 cs=0 l=1 rev1=1 crypto rx=0x7fba3000b4a0 tx=0x7fba3000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:40.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:40.656+0000 7fba48a00640 1 -- 172.21.15.83:0/921124320 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba30014070 con 0x7fba44076e50 2024-04-24T09:17:40.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:40.656+0000 7fba48a00640 1 -- 172.21.15.83:0/921124320 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fba30010ac0 con 0x7fba44076e50 2024-04-24T09:17:40.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:40.656+0000 7fba49e00640 1 -- 172.21.15.83:0/921124320 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba441a4170 con 0x7fba44076e50 2024-04-24T09:17:40.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:40.656+0000 7fba48a00640 1 -- 172.21.15.83:0/921124320 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba30017c30 con 0x7fba44076e50 2024-04-24T09:17:40.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:40.656+0000 7fba49e00640 1 -- 172.21.15.83:0/921124320 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba441aba00 con 0x7fba44076e50 2024-04-24T09:17:40.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:40.658+0000 7fba49e00640 1 -- 172.21.15.83:0/921124320 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba4404fd90 con 0x7fba44076e50 2024-04-24T09:17:40.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:40.660+0000 7fba48a00640 1 -- 172.21.15.83:0/921124320 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fba30017070 con 0x7fba44076e50 2024-04-24T09:17:40.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:40.661+0000 7fba48a00640 1 --2- 172.21.15.83:0/921124320 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fba280a3e40 0x7fba280a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:40.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:40.662+0000 7fba48a00640 1 -- 172.21.15.83:0/921124320 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 9583+0+0 (secure 0 0 0) 0x7fba300cff50 con 0x7fba44076e50 2024-04-24T09:17:40.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:40.662+0000 7fba42a00640 1 --2- 172.21.15.83:0/921124320 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fba280a3e40 0x7fba280a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:40.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:40.663+0000 7fba42a00640 1 --2- 172.21.15.83:0/921124320 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fba280a3e40 0x7fba280a6300 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fba34005ab0 tx=0x7fba34000bc0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:40.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:40.667+0000 7fba48a00640 1 -- 172.21.15.83:0/921124320 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fba3009c280 con 0x7fba44076e50 2024-04-24T09:17:40.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:17:40 smithi189 ceph-mon[31791]: pgmap v824: 129 pgs: 129 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.3 MiB/s wr, 396 op/s 2024-04-24T09:17:40.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:17:40 smithi189 ceph-mon[31791]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:17:40.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:40.892+0000 7fba49e00640 1 -- 172.21.15.83:0/921124320 --> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7fba44062440 con 0x7fba280a3e40 2024-04-24T09:17:40.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:40.914+0000 7fba48a00640 1 -- 172.21.15.83:0/921124320 <== mgr.15579 v2:172.21.15.83:6800/3594769126 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7fba44062440 con 0x7fba280a3e40 2024-04-24T09:17:40.914 INFO:teuthology.orchestra.run.smithi083.stdout:/ 1m 6m3h 2024-04-24T09:17:40.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:40.917+0000 7fba49e00640 1 -- 172.21.15.83:0/921124320 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fba280a3e40 msgr2=0x7fba280a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:40.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:40.917+0000 7fba49e00640 1 --2- 172.21.15.83:0/921124320 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fba280a3e40 0x7fba280a6300 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fba34005ab0 tx=0x7fba34000bc0 comp rx=0 tx=0).stop 2024-04-24T09:17:40.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:40.917+0000 7fba49e00640 1 -- 172.21.15.83:0/921124320 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba44076e50 msgr2=0x7fba4419f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:40.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:40.917+0000 7fba49e00640 1 --2- 172.21.15.83:0/921124320 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba44076e50 0x7fba4419f420 secure :-1 s=READY pgs=2856 cs=0 l=1 rev1=1 crypto rx=0x7fba3000b4a0 tx=0x7fba3000b960 comp rx=0 tx=0).stop 2024-04-24T09:17:40.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:40.917+0000 7fba49e00640 1 -- 172.21.15.83:0/921124320 shutdown_connections 2024-04-24T09:17:40.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:40.917+0000 7fba49e00640 1 --2- 172.21.15.83:0/921124320 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fba280a3e40 0x7fba280a6300 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:40.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:40.917+0000 7fba49e00640 1 --2- 172.21.15.83:0/921124320 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba44112ea0 0x7fba441a3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:40.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:40.917+0000 7fba49e00640 1 --2- 172.21.15.83:0/921124320 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba44076e50 0x7fba4419f420 unknown :-1 s=CLOSED pgs=2856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:40.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:40.917+0000 7fba49e00640 1 --2- 172.21.15.83:0/921124320 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fba44076530 0x7fba4419eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:40.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:40.917+0000 7fba49e00640 1 -- 172.21.15.83:0/921124320 >> 172.21.15.83:0/921124320 conn(0x7fba440ffd30 msgr2=0x7fba44102120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:40.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:40.917+0000 7fba49e00640 1 -- 172.21.15.83:0/921124320 shutdown_connections 2024-04-24T09:17:40.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:40.917+0000 7fba49e00640 1 -- 172.21.15.83:0/921124320 wait complete. 2024-04-24T09:17:40.937 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-04-24T09:17:40.947 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-04-24T09:17:41.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:17:41 smithi083 ceph-mon[27907]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:17:41.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:17:41 smithi083 ceph-mon[27907]: from='client.24321 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:17:41.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:17:41 smithi138 ceph-mon[31441]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:17:41.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:17:41 smithi138 ceph-mon[31441]: from='client.24321 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:17:41.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:17:41 smithi189 ceph-mon[31791]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:17:41.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:17:41 smithi189 ceph-mon[31791]: from='client.24321 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:17:42.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:17:42 smithi083 ceph-mon[27907]: from='client.24327 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:17:42.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:17:42 smithi083 ceph-mon[27907]: pgmap v825: 129 pgs: 129 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.3 MiB/s wr, 221 op/s 2024-04-24T09:17:42.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:17:42 smithi083 ceph-mon[27907]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:17:42.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:17:42 smithi138 ceph-mon[31441]: from='client.24327 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:17:42.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:17:42 smithi138 ceph-mon[31441]: pgmap v825: 129 pgs: 129 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.3 MiB/s wr, 221 op/s 2024-04-24T09:17:42.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:17:42 smithi138 ceph-mon[31441]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:17:42.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:17:42 smithi189 ceph-mon[31791]: from='client.24327 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T09:17:42.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:17:42 smithi189 ceph-mon[31791]: pgmap v825: 129 pgs: 129 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.3 MiB/s wr, 221 op/s 2024-04-24T09:17:42.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:17:42 smithi189 ceph-mon[31791]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:17:43.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:17:43 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:17:43.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:17:43 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:17:43.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:17:43 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:17:43.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:17:43 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:17:43.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:17:43 smithi083 ceph-mon[27907]: scrub summary: idle 2024-04-24T09:17:43.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:17:43 smithi083 ceph-mon[27907]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:17:43.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:17:43 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:17:43.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:17:43 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:17:43.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:17:43 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:17:43.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:17:43 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:17:43.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:17:43 smithi138 ceph-mon[31441]: scrub summary: idle 2024-04-24T09:17:43.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:17:43 smithi138 ceph-mon[31441]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:17:43.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:17:43 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:17:43.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:17:43 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:17:43.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:17:43 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:17:43.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:17:43 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:17:43.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:17:43 smithi189 ceph-mon[31791]: scrub summary: idle 2024-04-24T09:17:43.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:17:43 smithi189 ceph-mon[31791]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:17:44.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:17:44 smithi083 ceph-mon[27907]: pgmap v826: 129 pgs: 129 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 238 op/s 2024-04-24T09:17:44.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:17:44 smithi138 ceph-mon[31441]: pgmap v826: 129 pgs: 129 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 238 op/s 2024-04-24T09:17:44.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:17:44 smithi189 ceph-mon[31791]: pgmap v826: 129 pgs: 129 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 238 op/s 2024-04-24T09:17:46.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:17:46 smithi083 ceph-mon[27907]: pgmap v827: 129 pgs: 129 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 591 KiB/s rd, 458 KiB/s wr, 105 op/s 2024-04-24T09:17:46.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:17:46 smithi138 ceph-mon[31441]: pgmap v827: 129 pgs: 129 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 591 KiB/s rd, 458 KiB/s wr, 105 op/s 2024-04-24T09:17:46.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:17:46 smithi189 ceph-mon[31791]: pgmap v827: 129 pgs: 129 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 591 KiB/s rd, 458 KiB/s wr, 105 op/s 2024-04-24T09:17:48.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:17:48 smithi083 ceph-mon[27907]: pgmap v828: 129 pgs: 129 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 591 KiB/s rd, 458 KiB/s wr, 105 op/s 2024-04-24T09:17:48.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:17:48 smithi138 ceph-mon[31441]: pgmap v828: 129 pgs: 129 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 591 KiB/s rd, 458 KiB/s wr, 105 op/s 2024-04-24T09:17:48.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:17:48 smithi189 ceph-mon[31791]: pgmap v828: 129 pgs: 129 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 591 KiB/s rd, 458 KiB/s wr, 105 op/s 2024-04-24T09:17:50.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:17:50 smithi138 ceph-mon[31441]: pgmap v829: 129 pgs: 129 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 601 KiB/s rd, 475 KiB/s wr, 113 op/s 2024-04-24T09:17:50.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:17:50 smithi138 ceph-mon[31441]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:17:50.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:17:50 smithi189 ceph-mon[31791]: pgmap v829: 129 pgs: 129 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 601 KiB/s rd, 475 KiB/s wr, 113 op/s 2024-04-24T09:17:50.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:17:50 smithi189 ceph-mon[31791]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:17:50.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:17:50 smithi083 ceph-mon[27907]: pgmap v829: 129 pgs: 129 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 601 KiB/s rd, 475 KiB/s wr, 113 op/s 2024-04-24T09:17:50.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:17:50 smithi083 ceph-mon[27907]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T09:17:52.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:17:52 smithi138 ceph-mon[31441]: pgmap v830: 129 pgs: 129 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 79 KiB/s wr, 24 op/s 2024-04-24T09:17:52.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:17:52 smithi189 ceph-mon[31791]: pgmap v830: 129 pgs: 129 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 79 KiB/s wr, 24 op/s 2024-04-24T09:17:52.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:17:52 smithi083 ceph-mon[27907]: pgmap v830: 129 pgs: 129 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 79 KiB/s wr, 24 op/s 2024-04-24T09:17:54.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:17:54 smithi138 ceph-mon[31441]: pgmap v831: 129 pgs: 129 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 79 KiB/s wr, 29 op/s 2024-04-24T09:17:54.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:17:54 smithi189 ceph-mon[31791]: pgmap v831: 129 pgs: 129 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 79 KiB/s wr, 29 op/s 2024-04-24T09:17:54.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:17:54 smithi083 ceph-mon[27907]: pgmap v831: 129 pgs: 129 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 79 KiB/s wr, 29 op/s 2024-04-24T09:17:56.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:17:56 smithi138 ceph-mon[31441]: pgmap v832: 129 pgs: 129 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-04-24T09:17:56.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:17:56 smithi189 ceph-mon[31791]: pgmap v832: 129 pgs: 129 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-04-24T09:17:56.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:17:56 smithi083 ceph-mon[27907]: pgmap v832: 129 pgs: 129 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-04-24T09:17:57.688 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T09:17:57.688 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T09:17:57.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.905+0000 7fb3de400640 1 -- 172.21.15.83:0/1864522300 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb3d8104320 msgr2=0x7fb3d8104700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:57.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.905+0000 7fb3de400640 1 --2- 172.21.15.83:0/1864522300 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb3d8104320 0x7fb3d8104700 secure :-1 s=READY pgs=2857 cs=0 l=1 rev1=1 crypto rx=0x7fb3c8009900 tx=0x7fb3c802a410 comp rx=0 tx=0).stop 2024-04-24T09:17:57.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.905+0000 7fb3de400640 1 -- 172.21.15.83:0/1864522300 shutdown_connections 2024-04-24T09:17:57.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.905+0000 7fb3de400640 1 --2- 172.21.15.83:0/1864522300 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3d81093f0 0x7fb3d8110fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:57.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.905+0000 7fb3de400640 1 --2- 172.21.15.83:0/1864522300 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb3d8104cd0 0x7fb3d8108d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:57.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.905+0000 7fb3de400640 1 --2- 172.21.15.83:0/1864522300 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb3d8104320 0x7fb3d8104700 unknown :-1 s=CLOSED pgs=2857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:57.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.905+0000 7fb3de400640 1 -- 172.21.15.83:0/1864522300 >> 172.21.15.83:0/1864522300 conn(0x7fb3d80ffb40 msgr2=0x7fb3d8101f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:57.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.905+0000 7fb3de400640 1 -- 172.21.15.83:0/1864522300 shutdown_connections 2024-04-24T09:17:57.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.905+0000 7fb3de400640 1 -- 172.21.15.83:0/1864522300 wait complete. 2024-04-24T09:17:57.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.906+0000 7fb3de400640 1 Processor -- start 2024-04-24T09:17:57.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.907+0000 7fb3de400640 1 -- start start 2024-04-24T09:17:57.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.907+0000 7fb3de400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb3d8104320 0x7fb3d81a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:57.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.907+0000 7fb3de400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3d81093f0 0x7fb3d81a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:57.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.907+0000 7fb3de400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb3d819a730 0x7fb3d819abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:57.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.907+0000 7fb3de400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d8114350 con 0x7fb3d8104320 2024-04-24T09:17:57.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.907+0000 7fb3de400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d81141d0 con 0x7fb3d81093f0 2024-04-24T09:17:57.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.907+0000 7fb3de400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d81144d0 con 0x7fb3d819a730 2024-04-24T09:17:57.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.907+0000 7fb3dd000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb3d8104320 0x7fb3d81a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:57.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.907+0000 7fb3dd000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb3d8104320 0x7fb3d81a0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60088/0 (socket says 172.21.15.83:60088) 2024-04-24T09:17:57.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.907+0000 7fb3dd000640 1 -- 172.21.15.83:0/2003090577 learned_addr learned my addr 172.21.15.83:0/2003090577 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:17:57.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.908+0000 7fb3cfe00640 1 --2- 172.21.15.83:0/2003090577 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3d81093f0 0x7fb3d81a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:57.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.908+0000 7fb3dda00640 1 --2- 172.21.15.83:0/2003090577 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb3d819a730 0x7fb3d819abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:57.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.908+0000 7fb3dd000640 1 -- 172.21.15.83:0/2003090577 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb3d819a730 msgr2=0x7fb3d819abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:57.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.908+0000 7fb3dd000640 1 --2- 172.21.15.83:0/2003090577 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb3d819a730 0x7fb3d819abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:57.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.908+0000 7fb3dd000640 1 -- 172.21.15.83:0/2003090577 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3d81093f0 msgr2=0x7fb3d81a0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:57.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.908+0000 7fb3dd000640 1 --2- 172.21.15.83:0/2003090577 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3d81093f0 0x7fb3d81a0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:57.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.908+0000 7fb3dd000640 1 -- 172.21.15.83:0/2003090577 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3d819b330 con 0x7fb3d8104320 2024-04-24T09:17:57.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.908+0000 7fb3cfe00640 1 --2- 172.21.15.83:0/2003090577 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3d81093f0 0x7fb3d81a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:17:57.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.908+0000 7fb3dd000640 1 --2- 172.21.15.83:0/2003090577 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb3d8104320 0x7fb3d81a0670 secure :-1 s=READY pgs=2858 cs=0 l=1 rev1=1 crypto rx=0x7fb3c8005940 tx=0x7fb3c8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:57.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.908+0000 7fb3d6000640 1 -- 172.21.15.83:0/2003090577 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3c803f030 con 0x7fb3d8104320 2024-04-24T09:17:57.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.909+0000 7fb3d6000640 1 -- 172.21.15.83:0/2003090577 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb3c8003aa0 con 0x7fb3d8104320 2024-04-24T09:17:57.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.909+0000 7fb3de400640 1 -- 172.21.15.83:0/2003090577 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3d819b5c0 con 0x7fb3d8104320 2024-04-24T09:17:57.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.909+0000 7fb3d6000640 1 -- 172.21.15.83:0/2003090577 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3c8035070 con 0x7fb3d8104320 2024-04-24T09:17:57.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.909+0000 7fb3de400640 1 -- 172.21.15.83:0/2003090577 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3d81a7700 con 0x7fb3d8104320 2024-04-24T09:17:57.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.912+0000 7fb3e27de740 1 Processor -- start 2024-04-24T09:17:57.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.912+0000 7fb3d6000640 1 -- 172.21.15.83:0/2003090577 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb3c8042030 con 0x7fb3d8104320 2024-04-24T09:17:57.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.912+0000 7fb3e27de740 1 -- start start 2024-04-24T09:17:57.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.912+0000 7fb3e27de740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f065092780 0x55f065092b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:57.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.912+0000 7fb3e27de740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f065093130 0x55f0650a5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:57.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.912+0000 7fb3e27de740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0650a6230 0x55f0650a8620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:57.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.912+0000 7fb3e27de740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55f064ff81f0 con 0x55f0650a6230 2024-04-24T09:17:57.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.912+0000 7fb3e27de740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f064f9e350 con 0x55f065093130 2024-04-24T09:17:57.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.912+0000 7fb3e27de740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55f064ff41b0 con 0x55f065092780 2024-04-24T09:17:57.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.913+0000 7fb3cfe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0650a6230 0x55f0650a8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:57.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.913+0000 7fb3cfe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0650a6230 0x55f0650a8620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60092/0 (socket says 172.21.15.83:60092) 2024-04-24T09:17:57.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.913+0000 7fb3cfe00640 1 -- 172.21.15.83:0/1408041285 learned_addr learned my addr 172.21.15.83:0/1408041285 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:17:57.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.913+0000 7fb3dda00640 1 --2- 172.21.15.83:0/1408041285 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f065092780 0x55f065092b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:57.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.913+0000 7fb3dd000640 1 --2- 172.21.15.83:0/1408041285 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f065093130 0x55f0650a5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:57.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.913+0000 7fb3cfe00640 1 -- 172.21.15.83:0/1408041285 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f065092780 msgr2=0x55f065092b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:57.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.913+0000 7fb3cfe00640 1 --2- 172.21.15.83:0/1408041285 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f065092780 0x55f065092b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:57.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.913+0000 7fb3cfe00640 1 -- 172.21.15.83:0/1408041285 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f065093130 msgr2=0x55f0650a5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:57.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.913+0000 7fb3cfe00640 1 --2- 172.21.15.83:0/1408041285 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f065093130 0x55f0650a5cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:57.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.913+0000 7fb3cfe00640 1 -- 172.21.15.83:0/1408041285 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f064ff2f50 con 0x55f0650a6230 2024-04-24T09:17:57.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.913+0000 7fb3dd000640 1 --2- 172.21.15.83:0/1408041285 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f065093130 0x55f0650a5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:17:57.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.913+0000 7fb3cfe00640 1 --2- 172.21.15.83:0/1408041285 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0650a6230 0x55f0650a8620 secure :-1 s=READY pgs=2859 cs=0 l=1 rev1=1 crypto rx=0x7fb3c000bce0 tx=0x7fb3c000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a067d61580a85a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:57.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.913+0000 7fb3d6000640 1 --2- 172.21.15.83:0/2003090577 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb3bc0a3d20 0x7fb3bc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:57.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.914+0000 7fb3d6000640 1 -- 172.21.15.83:0/2003090577 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 9583+0+0 (secure 0 0 0) 0x7fb3c80f14e0 con 0x7fb3d8104320 2024-04-24T09:17:57.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.914+0000 7fb3ce000640 1 -- 172.21.15.83:0/1408041285 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3c0015070 con 0x55f0650a6230 2024-04-24T09:17:57.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.914+0000 7fb3ce000640 1 -- 172.21.15.83:0/1408041285 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb3c001a910 con 0x55f0650a6230 2024-04-24T09:17:57.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.914+0000 7fb3ce000640 1 -- 172.21.15.83:0/1408041285 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3c0011990 con 0x55f0650a6230 2024-04-24T09:17:57.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.914+0000 7fb3dda00640 1 --2- 172.21.15.83:0/2003090577 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb3bc0a3d20 0x7fb3bc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:57.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.915+0000 7fb3e27de740 1 -- 172.21.15.83:0/1408041285 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0650a6230 msgr2=0x55f0650a8620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:57.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.915+0000 7fb3e27de740 1 --2- 172.21.15.83:0/1408041285 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0650a6230 0x55f0650a8620 secure :-1 s=READY pgs=2859 cs=0 l=1 rev1=1 crypto rx=0x7fb3c000bce0 tx=0x7fb3c000f730 comp rx=0 tx=0).stop 2024-04-24T09:17:57.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.915+0000 7fb3dda00640 1 --2- 172.21.15.83:0/2003090577 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb3bc0a3d20 0x7fb3bc0a61e0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fb3d0005fd0 tx=0x7fb3d0009750 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:57.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.915+0000 7fb3e27de740 1 -- 172.21.15.83:0/1408041285 shutdown_connections 2024-04-24T09:17:57.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.915+0000 7fb3e27de740 1 --2- 172.21.15.83:0/1408041285 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0650a6230 0x55f0650a8620 unknown :-1 s=CLOSED pgs=2859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:57.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.915+0000 7fb3e27de740 1 --2- 172.21.15.83:0/1408041285 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f065093130 0x55f0650a5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:57.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.915+0000 7fb3e27de740 1 --2- 172.21.15.83:0/1408041285 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f065092780 0x55f065092b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:57.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.915+0000 7fb3e27de740 1 -- 172.21.15.83:0/1408041285 >> 172.21.15.83:0/1408041285 conn(0x55f06508ed50 msgr2=0x55f06508f130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:57.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.915+0000 7fb3e27de740 1 -- 172.21.15.83:0/1408041285 shutdown_connections 2024-04-24T09:17:57.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.915+0000 7fb3e27de740 1 -- 172.21.15.83:0/1408041285 wait complete. 2024-04-24T09:17:57.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.915+0000 7fb3e27de740 1 -- start start 2024-04-24T09:17:57.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.915+0000 7fb3e27de740 1 Processor -- start 2024-04-24T09:17:57.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.916+0000 7fb3e27de740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f065092780 0x55f0650a3210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:57.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.916+0000 7fb3e27de740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f065093130 0x55f0650a3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:57.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.916+0000 7fb3e27de740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0650a6230 0x55f0650a3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:57.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.916+0000 7fb3e27de740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55f0650a8ff0 con 0x55f065093130 2024-04-24T09:17:57.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.916+0000 7fb3e27de740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f064fb1690 con 0x55f0650a6230 2024-04-24T09:17:57.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.916+0000 7fb3e27de740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55f0650a9170 con 0x55f065092780 2024-04-24T09:17:57.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.916+0000 7fb3e27de740 10 client.0 _ll_register_callbacks cb 0x55f064fed650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:17:57.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.916+0000 7fb3cfe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f065093130 0x55f0650a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:57.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.916+0000 7fb3cfe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f065093130 0x55f0650a3750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60096/0 (socket says 172.21.15.83:60096) 2024-04-24T09:17:57.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.916+0000 7fb3cfe00640 1 -- 172.21.15.83:0/3279414446 learned_addr learned my addr 172.21.15.83:0/3279414446 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:17:57.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.916+0000 7fb3dda00640 1 --2- 172.21.15.83:0/3279414446 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0650a6230 0x55f0650a3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:57.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.916+0000 7fb3dd000640 1 --2- 172.21.15.83:0/3279414446 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f065092780 0x55f0650a3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:57.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.917+0000 7fb3cfe00640 1 -- 172.21.15.83:0/3279414446 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f065092780 msgr2=0x55f0650a3210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:57.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.917+0000 7fb3cfe00640 1 --2- 172.21.15.83:0/3279414446 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f065092780 0x55f0650a3210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:57.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.917+0000 7fb3cfe00640 1 -- 172.21.15.83:0/3279414446 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0650a6230 msgr2=0x55f0650a3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:57.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.917+0000 7fb3cfe00640 1 --2- 172.21.15.83:0/3279414446 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0650a6230 0x55f0650a3c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:57.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.917+0000 7fb3cfe00640 1 -- 172.21.15.83:0/3279414446 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f064fb0920 con 0x55f065093130 2024-04-24T09:17:57.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.917+0000 7fb3dda00640 1 --2- 172.21.15.83:0/3279414446 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0650a6230 0x55f0650a3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:17:57.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.917+0000 7fb3dd000640 1 --2- 172.21.15.83:0/3279414446 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f065092780 0x55f0650a3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:17:57.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.917+0000 7fb3cfe00640 1 --2- 172.21.15.83:0/3279414446 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f065093130 0x55f0650a3750 secure :-1 s=READY pgs=2860 cs=0 l=1 rev1=1 crypto rx=0x7fb3c001a690 tx=0x7fb3c001a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0f7db16dcc48a4e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:57.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.917+0000 7fb3c6a00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:17:57.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.917+0000 7fb3c6a00640 1 -- 172.21.15.83:0/3279414446 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3c0015050 con 0x55f065093130 2024-04-24T09:17:57.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.917+0000 7fb3c6a00640 1 -- 172.21.15.83:0/3279414446 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(134..134 src has 1..134) ==== 9583+0+0 (secure 0 0 0) 0x7fb3c001acd0 con 0x55f065093130 2024-04-24T09:17:57.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.917+0000 7fb3e27de740 1 -- 172.21.15.83:0/3279414446 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3c001a910 con 0x55f065093130 2024-04-24T09:17:57.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.917+0000 7fb3c6a00640 1 -- 172.21.15.83:0/3279414446 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7fb3c0031050 con 0x55f065093130 2024-04-24T09:17:57.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.918+0000 7fb3e27de740 10 client.24339 fetch_fsmap learned FSMap version 79 2024-04-24T09:17:57.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.918+0000 7fb3e27de740 1 -- 172.21.15.83:0/3279414446 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=79,monmap=4+}) -- 0x55f064dd47d0 con 0x55f065093130 2024-04-24T09:17:57.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.918+0000 7fb3c6a00640 1 -- 172.21.15.83:0/3279414446 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb3c001a910 con 0x55f065093130 2024-04-24T09:17:57.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.918+0000 7fb3c6a00640 1 -- 172.21.15.83:0/3279414446 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 79) ==== 8398+0+0 (secure 0 0 0) 0x7fb3c002c030 con 0x55f065093130 2024-04-24T09:17:57.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.918+0000 7fb3e27de740 10 client.24339 fetch_fsmap finished waiting for FSMap version 79 2024-04-24T09:17:57.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.918+0000 7fb3e27de740 10 client.24339 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:17:57.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.918+0000 7fb3e27de740 20 client.24339 populate_metadata read hostname 'smithi083' 2024-04-24T09:17:57.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.918+0000 7fb3e27de740 1 --2- 172.21.15.83:0/3279414446 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55f0650b6410 0x55f0650b8800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:57.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.918+0000 7fb3e27de740 4 client.24339 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:17:57.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.918+0000 7fb3e27de740 1 -- 172.21.15.83:0/3279414446 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f065005c10 con 0x55f0650b6410 2024-04-24T09:17:57.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.919+0000 7fb3dd000640 1 --2- 172.21.15.83:0/3279414446 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55f0650b6410 0x55f0650b8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:57.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.920+0000 7fb3dd000640 1 --2- 172.21.15.83:0/3279414446 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55f0650b6410 0x55f0650b8800 crc :-1 s=READY pgs=635 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d7a204703d41699 server_cookie=4c9e36989bad26c5 in_seq=0 out_seq=0 2024-04-24T09:17:57.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.920+0000 7fb3c6a00640 10 client.24339 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:17:57.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.922+0000 7fb3c6a00640 1 -- 172.21.15.83:0/3279414446 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f065005c10 con 0x55f0650b6410 2024-04-24T09:17:57.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.922+0000 7fb3c6a00640 10 client.24339 handle_command_reply: tid=0 2024-04-24T09:17:57.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.922+0000 7fb3e27de740 1 client.24339 shutdown 2024-04-24T09:17:57.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.922+0000 7fb3e27de740 1 -- 172.21.15.83:0/3279414446 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f065093130 msgr2=0x55f0650a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:57.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.922+0000 7fb3e27de740 1 --2- 172.21.15.83:0/3279414446 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f065093130 0x55f0650a3750 secure :-1 s=READY pgs=2860 cs=0 l=1 rev1=1 crypto rx=0x7fb3c001a690 tx=0x7fb3c001a6c0 comp rx=0 tx=0).stop 2024-04-24T09:17:57.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.922+0000 7fb3e27de740 1 -- 172.21.15.83:0/3279414446 shutdown_connections 2024-04-24T09:17:57.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.922+0000 7fb3e27de740 1 --2- 172.21.15.83:0/3279414446 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55f0650b6410 0x55f0650b8800 crc :-1 s=READY pgs=635 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:57.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.922+0000 7fb3e27de740 1 --2- 172.21.15.83:0/3279414446 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0650a6230 0x55f0650a3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:57.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.922+0000 7fb3e27de740 1 --2- 172.21.15.83:0/3279414446 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f065093130 0x55f0650a3750 unknown :-1 s=CLOSED pgs=2860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:57.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.922+0000 7fb3e27de740 1 --2- 172.21.15.83:0/3279414446 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f065092780 0x55f0650a3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:57.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.922+0000 7fb3c6a00640 0 client.24339 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:17:57.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.922+0000 7fb3e27de740 1 -- 172.21.15.83:0/3279414446 >> 172.21.15.83:0/3279414446 conn(0x55f06508ed50 msgr2=0x55f065094b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:57.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.922+0000 7fb3e27de740 1 -- 172.21.15.83:0/3279414446 shutdown_connections 2024-04-24T09:17:57.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.922+0000 7fb3e27de740 1 -- 172.21.15.83:0/3279414446 wait complete. 2024-04-24T09:17:57.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.922+0000 7fb3e27de740 20 client.24339 trim_cache size 0 max 16384 2024-04-24T09:17:57.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.941+0000 7fb3e27de740 1 Processor -- start 2024-04-24T09:17:57.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.941+0000 7fb3e27de740 1 -- start start 2024-04-24T09:17:57.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.941+0000 7fb3e27de740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f065092780 0x55f0650b1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:57.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.941+0000 7fb3e27de740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f065093130 0x55f06508de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:57.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.941+0000 7fb3e27de740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0650a6230 0x55f0650bdf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:57.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.941+0000 7fb3e27de740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d81ad1d0 con 0x55f0650a6230 2024-04-24T09:17:57.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.941+0000 7fb3e27de740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d81aedd0 con 0x55f065093130 2024-04-24T09:17:57.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.941+0000 7fb3e27de740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d81ad8a0 con 0x55f065092780 2024-04-24T09:17:57.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.942+0000 7fb3dda00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0650a6230 0x55f0650bdf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:57.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.942+0000 7fb3dda00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0650a6230 0x55f0650bdf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60106/0 (socket says 172.21.15.83:60106) 2024-04-24T09:17:57.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.942+0000 7fb3dda00640 1 -- 172.21.15.83:0/1175198535 learned_addr learned my addr 172.21.15.83:0/1175198535 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:17:57.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.942+0000 7fb3dd000640 1 --2- 172.21.15.83:0/1175198535 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f065092780 0x55f0650b1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:57.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.942+0000 7fb3cfe00640 1 --2- 172.21.15.83:0/1175198535 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f065093130 0x55f06508de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:57.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.942+0000 7fb3dda00640 1 -- 172.21.15.83:0/1175198535 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f065092780 msgr2=0x55f0650b1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:57.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.942+0000 7fb3dda00640 1 --2- 172.21.15.83:0/1175198535 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f065092780 0x55f0650b1ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:57.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.942+0000 7fb3dda00640 1 -- 172.21.15.83:0/1175198535 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f065093130 msgr2=0x55f06508de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:57.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.942+0000 7fb3dda00640 1 --2- 172.21.15.83:0/1175198535 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f065093130 0x55f06508de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:57.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.942+0000 7fb3dda00640 1 -- 172.21.15.83:0/1175198535 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f06509e000 con 0x55f0650a6230 2024-04-24T09:17:57.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.942+0000 7fb3cfe00640 1 --2- 172.21.15.83:0/1175198535 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f065093130 0x55f06508de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:17:57.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.943+0000 7fb3dda00640 1 --2- 172.21.15.83:0/1175198535 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0650a6230 0x55f0650bdf20 secure :-1 s=READY pgs=2861 cs=0 l=1 rev1=1 crypto rx=0x7fb3d0012740 tx=0x7fb3d0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2912b45f71b5af89 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:57.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.943+0000 7fb3cd600640 1 -- 172.21.15.83:0/1175198535 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3d001e6f0 con 0x55f0650a6230 2024-04-24T09:17:57.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.943+0000 7fb3cd600640 1 -- 172.21.15.83:0/1175198535 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb3d001ed40 con 0x55f0650a6230 2024-04-24T09:17:57.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.943+0000 7fb3cd600640 1 -- 172.21.15.83:0/1175198535 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3d001fe50 con 0x55f0650a6230 2024-04-24T09:17:57.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.944+0000 7fb3e27de740 1 -- 172.21.15.83:0/1175198535 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0650a6230 msgr2=0x55f0650bdf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:57.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.944+0000 7fb3e27de740 1 --2- 172.21.15.83:0/1175198535 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0650a6230 0x55f0650bdf20 secure :-1 s=READY pgs=2861 cs=0 l=1 rev1=1 crypto rx=0x7fb3d0012740 tx=0x7fb3d0014000 comp rx=0 tx=0).stop 2024-04-24T09:17:57.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.944+0000 7fb3e27de740 1 -- 172.21.15.83:0/1175198535 shutdown_connections 2024-04-24T09:17:57.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.944+0000 7fb3e27de740 1 --2- 172.21.15.83:0/1175198535 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f0650a6230 0x55f0650bdf20 unknown :-1 s=CLOSED pgs=2861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:57.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.944+0000 7fb3e27de740 1 --2- 172.21.15.83:0/1175198535 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f065093130 0x55f06508de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:57.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.944+0000 7fb3e27de740 1 --2- 172.21.15.83:0/1175198535 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f065092780 0x55f0650b1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:57.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.944+0000 7fb3e27de740 1 -- 172.21.15.83:0/1175198535 >> 172.21.15.83:0/1175198535 conn(0x55f06508ed50 msgr2=0x55f06508d740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:57.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.944+0000 7fb3e27de740 1 -- 172.21.15.83:0/1175198535 shutdown_connections 2024-04-24T09:17:57.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.944+0000 7fb3e27de740 1 -- 172.21.15.83:0/1175198535 wait complete. 2024-04-24T09:17:57.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.944+0000 7fb3e27de740 1 -- start start 2024-04-24T09:17:57.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.945+0000 7fb3e27de740 1 Processor -- start 2024-04-24T09:17:57.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.945+0000 7fb3e27de740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f065092780 0x55f0650ae5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:57.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.945+0000 7fb3e27de740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f065093130 0x55f06509faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:57.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.945+0000 7fb3e27de740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0650a6230 0x55f06509ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:57.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.945+0000 7fb3e27de740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d81ae6d0 con 0x55f065092780 2024-04-24T09:17:57.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.945+0000 7fb3e27de740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d81adfa0 con 0x55f0650a6230 2024-04-24T09:17:57.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.945+0000 7fb3e27de740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d81ab1d0 con 0x55f065093130 2024-04-24T09:17:57.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.945+0000 7fb3e27de740 10 client.0 _ll_register_callbacks cb 0x7fb3c000bb60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:17:57.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.945+0000 7fb3dd000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f065092780 0x55f0650ae5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:57.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.945+0000 7fb3dd000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f065092780 0x55f0650ae5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60120/0 (socket says 172.21.15.83:60120) 2024-04-24T09:17:57.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.945+0000 7fb3dd000640 1 -- 172.21.15.83:0/3038668034 learned_addr learned my addr 172.21.15.83:0/3038668034 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:17:57.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.946+0000 7fb3cfe00640 1 --2- 172.21.15.83:0/3038668034 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f065093130 0x55f06509faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:57.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.946+0000 7fb3dda00640 1 --2- 172.21.15.83:0/3038668034 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0650a6230 0x55f06509ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:57.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.946+0000 7fb3dd000640 1 -- 172.21.15.83:0/3038668034 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f065093130 msgr2=0x55f06509faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:57.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.946+0000 7fb3dd000640 1 --2- 172.21.15.83:0/3038668034 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f065093130 0x55f06509faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:57.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.946+0000 7fb3dd000640 1 -- 172.21.15.83:0/3038668034 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0650a6230 msgr2=0x55f06509ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:57.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.946+0000 7fb3dd000640 1 --2- 172.21.15.83:0/3038668034 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0650a6230 0x55f06509ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:57.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.946+0000 7fb3dd000640 1 -- 172.21.15.83:0/3038668034 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f064d0a3f0 con 0x55f065092780 2024-04-24T09:17:57.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.946+0000 7fb3cfe00640 1 --2- 172.21.15.83:0/3038668034 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f065093130 0x55f06509faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:17:57.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.946+0000 7fb3dda00640 1 --2- 172.21.15.83:0/3038668034 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0650a6230 0x55f06509ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:17:57.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.946+0000 7fb3dd000640 1 --2- 172.21.15.83:0/3038668034 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f065092780 0x55f0650ae5f0 secure :-1 s=READY pgs=2862 cs=0 l=1 rev1=1 crypto rx=0x7fb3c8006010 tx=0x7fb3c80b93c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5122d1254517ff2a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:57.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.946+0000 7fb3c6a00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:17:57.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.946+0000 7fb3c6a00640 1 -- 172.21.15.83:0/3038668034 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3c8049040 con 0x55f065092780 2024-04-24T09:17:57.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.946+0000 7fb3c6a00640 1 -- 172.21.15.83:0/3038668034 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(134..134 src has 1..134) ==== 9583+0+0 (secure 0 0 0) 0x7fb3c8003c50 con 0x55f065092780 2024-04-24T09:17:57.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.946+0000 7fb3e27de740 1 -- 172.21.15.83:0/3038668034 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3d001ed40 con 0x55f065092780 2024-04-24T09:17:57.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.946+0000 7fb3c6a00640 1 -- 172.21.15.83:0/3038668034 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7fb3c80c13b0 con 0x55f065092780 2024-04-24T09:17:57.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.946+0000 7fb3e27de740 10 client.24345 fetch_fsmap learned FSMap version 79 2024-04-24T09:17:57.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.946+0000 7fb3e27de740 1 -- 172.21.15.83:0/3038668034 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=79,monmap=4+}) -- 0x55f064cca0f0 con 0x55f065092780 2024-04-24T09:17:57.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.947+0000 7fb3c6a00640 1 -- 172.21.15.83:0/3038668034 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb3d001ed40 con 0x55f065092780 2024-04-24T09:17:57.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.947+0000 7fb3c6a00640 1 -- 172.21.15.83:0/3038668034 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 79) ==== 8398+0+0 (secure 0 0 0) 0x7fb3c80eea30 con 0x55f065092780 2024-04-24T09:17:57.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.947+0000 7fb3e27de740 10 client.24345 fetch_fsmap finished waiting for FSMap version 79 2024-04-24T09:17:57.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.947+0000 7fb3e27de740 10 client.24345 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:17:57.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.947+0000 7fb3e27de740 20 client.24345 populate_metadata read hostname 'smithi083' 2024-04-24T09:17:57.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.947+0000 7fb3e27de740 1 --2- 172.21.15.83:0/3038668034 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55f06511c850 0x55f06511ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:57.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.947+0000 7fb3e27de740 4 client.24345 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T09:17:57.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.947+0000 7fb3e27de740 1 -- 172.21.15.83:0/3038668034 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f064fffa80 con 0x55f06511c850 2024-04-24T09:17:57.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.948+0000 7fb3cfe00640 1 --2- 172.21.15.83:0/3038668034 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55f06511c850 0x55f06511ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:57.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.948+0000 7fb3cfe00640 1 --2- 172.21.15.83:0/3038668034 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55f06511c850 0x55f06511ec70 crc :-1 s=READY pgs=636 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42e0e7826b7bbcc6 server_cookie=712231903340bfea in_seq=0 out_seq=0 2024-04-24T09:17:57.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.948+0000 7fb3c6a00640 10 client.24345 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:17:57.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.949+0000 7fb3c6a00640 1 -- 172.21.15.83:0/3038668034 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55f064fffa80 con 0x55f06511c850 2024-04-24T09:17:57.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.949+0000 7fb3c6a00640 10 client.24345 handle_command_reply: tid=0 2024-04-24T09:17:57.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.949+0000 7fb3e27de740 1 client.24345 shutdown 2024-04-24T09:17:57.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.949+0000 7fb3e27de740 1 -- 172.21.15.83:0/3038668034 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f065092780 msgr2=0x55f0650ae5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:57.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.949+0000 7fb3e27de740 1 --2- 172.21.15.83:0/3038668034 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f065092780 0x55f0650ae5f0 secure :-1 s=READY pgs=2862 cs=0 l=1 rev1=1 crypto rx=0x7fb3c8006010 tx=0x7fb3c80b93c0 comp rx=0 tx=0).stop 2024-04-24T09:17:57.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.949+0000 7fb3e27de740 1 -- 172.21.15.83:0/3038668034 shutdown_connections 2024-04-24T09:17:57.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.949+0000 7fb3e27de740 1 --2- 172.21.15.83:0/3038668034 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x55f06511c850 0x55f06511ec70 crc :-1 s=READY pgs=636 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:57.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.949+0000 7fb3e27de740 1 --2- 172.21.15.83:0/3038668034 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0650a6230 0x55f06509ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:57.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.949+0000 7fb3e27de740 1 --2- 172.21.15.83:0/3038668034 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f065093130 0x55f06509faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:57.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.949+0000 7fb3c6a00640 0 client.24345 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:17:57.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.949+0000 7fb3e27de740 1 --2- 172.21.15.83:0/3038668034 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f065092780 0x55f0650ae5f0 unknown :-1 s=CLOSED pgs=2862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:57.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.949+0000 7fb3e27de740 1 -- 172.21.15.83:0/3038668034 >> 172.21.15.83:0/3038668034 conn(0x55f06508ed50 msgr2=0x55f06508dfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:57.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.949+0000 7fb3e27de740 1 -- 172.21.15.83:0/3038668034 shutdown_connections 2024-04-24T09:17:57.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.949+0000 7fb3e27de740 1 -- 172.21.15.83:0/3038668034 wait complete. 2024-04-24T09:17:57.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.950+0000 7fb3e27de740 20 client.24345 trim_cache size 0 max 16384 2024-04-24T09:17:57.952 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T09:17:57.953 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-04-24T09:17:57.953 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-04-24T09:17:57.953 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T09:17:57.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.950+0000 7fb3de400640 1 -- 172.21.15.83:0/2003090577 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb3bc0a3d20 msgr2=0x7fb3bc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:57.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.950+0000 7fb3de400640 1 --2- 172.21.15.83:0/2003090577 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb3bc0a3d20 0x7fb3bc0a61e0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fb3d0005fd0 tx=0x7fb3d0009750 comp rx=0 tx=0).stop 2024-04-24T09:17:57.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.950+0000 7fb3de400640 1 -- 172.21.15.83:0/2003090577 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb3d8104320 msgr2=0x7fb3d81a0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:57.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.950+0000 7fb3de400640 1 --2- 172.21.15.83:0/2003090577 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb3d8104320 0x7fb3d81a0670 secure :-1 s=READY pgs=2858 cs=0 l=1 rev1=1 crypto rx=0x7fb3c8005940 tx=0x7fb3c8004270 comp rx=0 tx=0).stop 2024-04-24T09:17:57.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.950+0000 7fb3de400640 1 -- 172.21.15.83:0/2003090577 shutdown_connections 2024-04-24T09:17:57.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.950+0000 7fb3de400640 1 --2- 172.21.15.83:0/2003090577 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb3bc0a3d20 0x7fb3bc0a61e0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:57.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.950+0000 7fb3de400640 1 --2- 172.21.15.83:0/2003090577 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb3d819a730 0x7fb3d819abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:57.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.950+0000 7fb3de400640 1 --2- 172.21.15.83:0/2003090577 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3d81093f0 0x7fb3d81a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:57.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.950+0000 7fb3de400640 1 --2- 172.21.15.83:0/2003090577 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb3d8104320 0x7fb3d81a0670 unknown :-1 s=CLOSED pgs=2858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:57.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.950+0000 7fb3de400640 1 -- 172.21.15.83:0/2003090577 >> 172.21.15.83:0/2003090577 conn(0x7fb3d80ffb40 msgr2=0x7fb3d8100f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:57.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.950+0000 7fb3de400640 1 -- 172.21.15.83:0/2003090577 shutdown_connections 2024-04-24T09:17:57.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:57.951+0000 7fb3de400640 1 -- 172.21.15.83:0/2003090577 wait complete. 2024-04-24T09:17:57.968 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T09:17:57.969 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T09:17:58.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.167+0000 7f4a89200640 1 -- 172.21.15.83:0/909221150 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a84076530 msgr2=0x7f4a84076910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:58.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.167+0000 7f4a89200640 1 --2- 172.21.15.83:0/909221150 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a84076530 0x7f4a84076910 secure :-1 s=READY pgs=2863 cs=0 l=1 rev1=1 crypto rx=0x7f4a74009900 tx=0x7f4a7402a3f0 comp rx=0 tx=0).stop 2024-04-24T09:17:58.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.167+0000 7f4a89200640 1 -- 172.21.15.83:0/909221150 shutdown_connections 2024-04-24T09:17:58.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.167+0000 7f4a89200640 1 --2- 172.21.15.83:0/909221150 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4a84112ca0 0x7f4a84115090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:58.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.167+0000 7f4a89200640 1 --2- 172.21.15.83:0/909221150 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a84076e50 0x7f4a84112760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:58.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.167+0000 7f4a89200640 1 --2- 172.21.15.83:0/909221150 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a84076530 0x7f4a84076910 unknown :-1 s=CLOSED pgs=2863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:58.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.167+0000 7f4a89200640 1 -- 172.21.15.83:0/909221150 >> 172.21.15.83:0/909221150 conn(0x7f4a840ffad0 msgr2=0x7f4a84101ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:58.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.167+0000 7f4a89200640 1 -- 172.21.15.83:0/909221150 shutdown_connections 2024-04-24T09:17:58.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.167+0000 7f4a89200640 1 -- 172.21.15.83:0/909221150 wait complete. 2024-04-24T09:17:58.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.168+0000 7f4a89200640 1 Processor -- start 2024-04-24T09:17:58.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.168+0000 7f4a89200640 1 -- start start 2024-04-24T09:17:58.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.168+0000 7f4a89200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a84076530 0x7f4a841a4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:58.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.168+0000 7f4a89200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4a84076e50 0x7f4a841a4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:58.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.168+0000 7f4a89200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a84112ca0 0x7f4a8419ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:58.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.168+0000 7f4a89200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4a841186c0 con 0x7f4a84112ca0 2024-04-24T09:17:58.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.168+0000 7f4a89200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4a84118540 con 0x7f4a84076530 2024-04-24T09:17:58.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.168+0000 7f4a89200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4a84118840 con 0x7f4a84076e50 2024-04-24T09:17:58.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.169+0000 7f4a83e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a84112ca0 0x7f4a8419ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:58.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.169+0000 7f4a83e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a84112ca0 0x7f4a8419ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60130/0 (socket says 172.21.15.83:60130) 2024-04-24T09:17:58.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.169+0000 7f4a83e00640 1 -- 172.21.15.83:0/1046491100 learned_addr learned my addr 172.21.15.83:0/1046491100 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:17:58.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.169+0000 7f4a82a00640 1 --2- 172.21.15.83:0/1046491100 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4a84076e50 0x7f4a841a4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:58.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.169+0000 7f4a83400640 1 --2- 172.21.15.83:0/1046491100 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a84076530 0x7f4a841a4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:58.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.169+0000 7f4a83e00640 1 -- 172.21.15.83:0/1046491100 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4a84076e50 msgr2=0x7f4a841a4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:58.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.169+0000 7f4a83e00640 1 --2- 172.21.15.83:0/1046491100 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4a84076e50 0x7f4a841a4d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:58.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.169+0000 7f4a83e00640 1 -- 172.21.15.83:0/1046491100 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a84076530 msgr2=0x7f4a841a4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:58.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.169+0000 7f4a83e00640 1 --2- 172.21.15.83:0/1046491100 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a84076530 0x7f4a841a4840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:58.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.169+0000 7f4a83e00640 1 -- 172.21.15.83:0/1046491100 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a8419f1c0 con 0x7f4a84112ca0 2024-04-24T09:17:58.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.169+0000 7f4a83e00640 1 --2- 172.21.15.83:0/1046491100 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a84112ca0 0x7f4a8419ea40 secure :-1 s=READY pgs=2864 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c00cab0 tx=0x7f4a7c00cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:58.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.170+0000 7f4a7be00640 1 -- 172.21.15.83:0/1046491100 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a7c00ae20 con 0x7f4a84112ca0 2024-04-24T09:17:58.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.170+0000 7f4a7be00640 1 -- 172.21.15.83:0/1046491100 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4a7c00f900 con 0x7f4a84112ca0 2024-04-24T09:17:58.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.170+0000 7f4a7be00640 1 -- 172.21.15.83:0/1046491100 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a7c0189f0 con 0x7f4a84112ca0 2024-04-24T09:17:58.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.170+0000 7f4a89200640 1 -- 172.21.15.83:0/1046491100 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a8419f4b0 con 0x7f4a84112ca0 2024-04-24T09:17:58.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.170+0000 7f4a89200640 1 -- 172.21.15.83:0/1046491100 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a841ab700 con 0x7f4a84112ca0 2024-04-24T09:17:58.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.171+0000 7f4a89200640 1 -- 172.21.15.83:0/1046491100 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a84069190 con 0x7f4a84112ca0 2024-04-24T09:17:58.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.173+0000 7f4a7be00640 1 -- 172.21.15.83:0/1046491100 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4a7c00fab0 con 0x7f4a84112ca0 2024-04-24T09:17:58.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.174+0000 7f4a7be00640 1 --2- 172.21.15.83:0/1046491100 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f4a680a3d20 0x7f4a680a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:58.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.175+0000 7f4a7be00640 1 -- 172.21.15.83:0/1046491100 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 9583+0+0 (secure 0 0 0) 0x7f4a7c0d0fc0 con 0x7f4a84112ca0 2024-04-24T09:17:58.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.175+0000 7f4a83400640 1 --2- 172.21.15.83:0/1046491100 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f4a680a3d20 0x7f4a680a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:58.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.175+0000 7f4a83400640 1 --2- 172.21.15.83:0/1046491100 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f4a680a3d20 0x7f4a680a61e0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f4a74000c00 tx=0x7f4a74009610 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:58.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.179+0000 7f4a7be00640 1 -- 172.21.15.83:0/1046491100 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4a7c014030 con 0x7f4a84112ca0 2024-04-24T09:17:58.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.428+0000 7f4a89200640 1 -- 172.21.15.83:0/1046491100 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4a8402d620 con 0x7f4a84112ca0 2024-04-24T09:17:58.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.429+0000 7f4a7be00640 1 -- 172.21.15.83:0/1046491100 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) ==== 94+0+11957 (secure 0 0 0) 0x7f4a7c09d1e0 con 0x7f4a84112ca0 2024-04-24T09:17:58.429 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T09:17:58.429 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":79,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/358872257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":358872257},{"type":"v1","addr":"172.21.15.83:6835","nonce":358872257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3157070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3157070851},{"type":"v1","addr":"172.21.15.83:6837","nonce":3157070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":79,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T08:48:29.422576+0000","modified":"2024-04-24T09:17:48.323281+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34520,"mds_1":34493,"mds_2":34451,"mds_3":34427,"mds_4":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14658":{"gid":14658,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6839/4085470844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":4085470844},{"type":"v1","addr":"172.21.15.83:6839","nonce":4085470844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14676":{"gid":14676,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6841/2634754210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":2634754210},{"type":"v1","addr":"172.21.15.83:6841","nonce":2634754210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24382":{"gid":24382,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/1859921310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1859921310},{"type":"v1","addr":"172.21.15.189:6833","nonce":1859921310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/2241923197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2241923197},{"type":"v1","addr":"172.21.15.189:6835","nonce":2241923197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.189:6839/2548965336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":2548965336},{"type":"v1","addr":"172.21.15.189:6839","nonce":2548965336}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26686":{"gid":26686,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6837/1678817161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":1678817161},{"type":"v1","addr":"172.21.15.189:6837","nonce":1678817161}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.138:6833/1157967658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1157967658},{"type":"v1","addr":"172.21.15.138:6833","nonce":1157967658}]},"join_fscid":-1,"export_targets":[0,1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.138:6835/1909219758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1909219758},{"type":"v1","addr":"172.21.15.138:6835","nonce":1909219758}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34493":{"gid":34493,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.138:6837/900566754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":900566754},{"type":"v1","addr":"172.21.15.138:6837","nonce":900566754}]},"join_fscid":-1,"export_targets":[0,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34520":{"gid":34520,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.138:6839/2236932393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2236932393},{"type":"v1","addr":"172.21.15.138:6839","nonce":2236932393}]},"join_fscid":-1,"export_targets":[4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34520,"qdb_cluster":[34520,24454,34427,34493,34451]},"id":1}]} 2024-04-24T09:17:58.429 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 79 2024-04-24T09:17:58.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.432+0000 7f4a89200640 1 -- 172.21.15.83:0/1046491100 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f4a680a3d20 msgr2=0x7f4a680a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:58.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.432+0000 7f4a89200640 1 --2- 172.21.15.83:0/1046491100 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f4a680a3d20 0x7f4a680a61e0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f4a74000c00 tx=0x7f4a74009610 comp rx=0 tx=0).stop 2024-04-24T09:17:58.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.432+0000 7f4a89200640 1 -- 172.21.15.83:0/1046491100 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a84112ca0 msgr2=0x7f4a8419ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:58.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.432+0000 7f4a89200640 1 --2- 172.21.15.83:0/1046491100 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a84112ca0 0x7f4a8419ea40 secure :-1 s=READY pgs=2864 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c00cab0 tx=0x7f4a7c00cf70 comp rx=0 tx=0).stop 2024-04-24T09:17:58.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.432+0000 7f4a89200640 1 -- 172.21.15.83:0/1046491100 shutdown_connections 2024-04-24T09:17:58.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.432+0000 7f4a89200640 1 --2- 172.21.15.83:0/1046491100 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f4a680a3d20 0x7f4a680a61e0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:58.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.432+0000 7f4a89200640 1 --2- 172.21.15.83:0/1046491100 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a84112ca0 0x7f4a8419ea40 unknown :-1 s=CLOSED pgs=2864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:58.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.432+0000 7f4a89200640 1 --2- 172.21.15.83:0/1046491100 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4a84076e50 0x7f4a841a4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:58.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.432+0000 7f4a89200640 1 --2- 172.21.15.83:0/1046491100 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a84076530 0x7f4a841a4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:58.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.432+0000 7f4a89200640 1 -- 172.21.15.83:0/1046491100 >> 172.21.15.83:0/1046491100 conn(0x7f4a840ffad0 msgr2=0x7f4a841143c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:58.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.432+0000 7f4a89200640 1 -- 172.21.15.83:0/1046491100 shutdown_connections 2024-04-24T09:17:58.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.432+0000 7f4a89200640 1 -- 172.21.15.83:0/1046491100 wait complete. 2024-04-24T09:17:58.451 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-24T09:17:58.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.658+0000 7fa233800640 1 -- 172.21.15.83:0/3176330188 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa22c108fd0 msgr2=0x7fa22c115490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:58.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.658+0000 7fa233800640 1 --2- 172.21.15.83:0/3176330188 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa22c108fd0 0x7fa22c115490 secure :-1 s=READY pgs=2865 cs=0 l=1 rev1=1 crypto rx=0x7fa224009cd0 tx=0x7fa22402a8f0 comp rx=0 tx=0).stop 2024-04-24T09:17:58.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.658+0000 7fa233800640 1 -- 172.21.15.83:0/3176330188 shutdown_connections 2024-04-24T09:17:58.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.658+0000 7fa233800640 1 --2- 172.21.15.83:0/3176330188 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa22c108fd0 0x7fa22c115490 unknown :-1 s=CLOSED pgs=2865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:58.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.658+0000 7fa233800640 1 --2- 172.21.15.83:0/3176330188 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa22c108610 0x7fa22c108a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:58.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.659+0000 7fa233800640 1 --2- 172.21.15.83:0/3176330188 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa22c10e660 0x7fa22c10ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:58.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.659+0000 7fa233800640 1 -- 172.21.15.83:0/3176330188 >> 172.21.15.83:0/3176330188 conn(0x7fa22c076540 msgr2=0x7fa22c078960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:58.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.659+0000 7fa233800640 1 -- 172.21.15.83:0/3176330188 shutdown_connections 2024-04-24T09:17:58.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.659+0000 7fa233800640 1 -- 172.21.15.83:0/3176330188 wait complete. 2024-04-24T09:17:58.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.660+0000 7fa233800640 1 Processor -- start 2024-04-24T09:17:58.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.660+0000 7fa233800640 1 -- start start 2024-04-24T09:17:58.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.660+0000 7fa233800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa22c108610 0x7fa22c1a4930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:58.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.660+0000 7fa233800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa22c108fd0 0x7fa22c1a4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:58.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.660+0000 7fa233800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa22c10e660 0x7fa22c19eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:58.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.660+0000 7fa233800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa22c1188a0 con 0x7fa22c108610 2024-04-24T09:17:58.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.660+0000 7fa233800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa22c118720 con 0x7fa22c10e660 2024-04-24T09:17:58.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.660+0000 7fa233800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa22c118a20 con 0x7fa22c108fd0 2024-04-24T09:17:58.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.660+0000 7fa230e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa22c108610 0x7fa22c1a4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:58.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.661+0000 7fa230e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa22c108610 0x7fa22c1a4930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60154/0 (socket says 172.21.15.83:60154) 2024-04-24T09:17:58.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.661+0000 7fa230e00640 1 -- 172.21.15.83:0/90116403 learned_addr learned my addr 172.21.15.83:0/90116403 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:17:58.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.661+0000 7fa231800640 1 --2- 172.21.15.83:0/90116403 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa22c10e660 0x7fa22c19eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:58.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.661+0000 7fa22be00640 1 --2- 172.21.15.83:0/90116403 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa22c108fd0 0x7fa22c1a4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:58.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.661+0000 7fa230e00640 1 -- 172.21.15.83:0/90116403 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa22c108fd0 msgr2=0x7fa22c1a4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:58.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.661+0000 7fa230e00640 1 --2- 172.21.15.83:0/90116403 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa22c108fd0 0x7fa22c1a4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:58.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.661+0000 7fa230e00640 1 -- 172.21.15.83:0/90116403 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa22c10e660 msgr2=0x7fa22c19eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:58.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.661+0000 7fa230e00640 1 --2- 172.21.15.83:0/90116403 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa22c10e660 0x7fa22c19eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:58.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.661+0000 7fa230e00640 1 -- 172.21.15.83:0/90116403 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa22c19f260 con 0x7fa22c108610 2024-04-24T09:17:58.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.661+0000 7fa231800640 1 --2- 172.21.15.83:0/90116403 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa22c10e660 0x7fa22c19eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:17:58.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.661+0000 7fa230e00640 1 --2- 172.21.15.83:0/90116403 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa22c108610 0x7fa22c1a4930 secure :-1 s=READY pgs=2866 cs=0 l=1 rev1=1 crypto rx=0x7fa22000d770 tx=0x7fa22000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:58.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.662+0000 7fa229600640 1 -- 172.21.15.83:0/90116403 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa220017070 con 0x7fa22c108610 2024-04-24T09:17:58.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.662+0000 7fa229600640 1 -- 172.21.15.83:0/90116403 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa220009c30 con 0x7fa22c108610 2024-04-24T09:17:58.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.662+0000 7fa233800640 1 -- 172.21.15.83:0/90116403 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa22c19f550 con 0x7fa22c108610 2024-04-24T09:17:58.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.662+0000 7fa229600640 1 -- 172.21.15.83:0/90116403 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa220003d00 con 0x7fa22c108610 2024-04-24T09:17:58.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.662+0000 7fa233800640 1 -- 172.21.15.83:0/90116403 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa22c1ab7f0 con 0x7fa22c108610 2024-04-24T09:17:58.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.665+0000 7fa23666a740 1 Processor -- start 2024-04-24T09:17:58.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.665+0000 7fa229600640 1 -- 172.21.15.83:0/90116403 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa220003ea0 con 0x7fa22c108610 2024-04-24T09:17:58.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.665+0000 7fa23666a740 1 -- start start 2024-04-24T09:17:58.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.666+0000 7fa23666a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55efd4c96950 0x55efd4c96d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:58.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.666+0000 7fa23666a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55efd4c97270 0x55efd4ca1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:58.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.666+0000 7fa23666a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55efd4ca1940 0x55efd4ca3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:58.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.666+0000 7fa23666a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55efd4bfc1d0 con 0x55efd4ca1940 2024-04-24T09:17:58.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.666+0000 7fa23666a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55efd4ba2450 con 0x55efd4c97270 2024-04-24T09:17:58.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.666+0000 7fa23666a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55efd4bf8190 con 0x55efd4c96950 2024-04-24T09:17:58.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.666+0000 7fa22be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55efd4ca1940 0x55efd4ca3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:58.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.666+0000 7fa22be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55efd4ca1940 0x55efd4ca3d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60166/0 (socket says 172.21.15.83:60166) 2024-04-24T09:17:58.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.666+0000 7fa22be00640 1 -- 172.21.15.83:0/3492441656 learned_addr learned my addr 172.21.15.83:0/3492441656 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:17:58.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.666+0000 7fa230e00640 1 --2- 172.21.15.83:0/3492441656 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55efd4c97270 0x55efd4ca1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:58.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.666+0000 7fa231800640 1 --2- 172.21.15.83:0/3492441656 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55efd4c96950 0x55efd4c96d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:58.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.667+0000 7fa22be00640 1 -- 172.21.15.83:0/3492441656 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55efd4c96950 msgr2=0x55efd4c96d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:58.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.667+0000 7fa22be00640 1 --2- 172.21.15.83:0/3492441656 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55efd4c96950 0x55efd4c96d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:58.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.667+0000 7fa22be00640 1 -- 172.21.15.83:0/3492441656 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55efd4c97270 msgr2=0x55efd4ca1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:58.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.667+0000 7fa22be00640 1 --2- 172.21.15.83:0/3492441656 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55efd4c97270 0x55efd4ca1400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:58.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.667+0000 7fa22be00640 1 -- 172.21.15.83:0/3492441656 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55efd4c5f310 con 0x55efd4ca1940 2024-04-24T09:17:58.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.667+0000 7fa231800640 1 --2- 172.21.15.83:0/3492441656 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55efd4c96950 0x55efd4c96d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:17:58.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.667+0000 7fa229600640 1 --2- 172.21.15.83:0/90116403 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa2140a3da0 0x7fa2140a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:58.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.667+0000 7fa229600640 1 -- 172.21.15.83:0/90116403 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 9583+0+0 (secure 0 0 0) 0x7fa2200d09d0 con 0x7fa22c108610 2024-04-24T09:17:58.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.667+0000 7fa22be00640 1 --2- 172.21.15.83:0/3492441656 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55efd4ca1940 0x55efd4ca3d30 secure :-1 s=READY pgs=2867 cs=0 l=1 rev1=1 crypto rx=0x7fa21800cc00 tx=0x7fa21800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82d450f5f6cf9ada server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:58.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.667+0000 7fa231800640 1 --2- 172.21.15.83:0/90116403 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa2140a3da0 0x7fa2140a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:58.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.667+0000 7fa21e000640 1 -- 172.21.15.83:0/3492441656 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa218005220 con 0x55efd4ca1940 2024-04-24T09:17:58.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.668+0000 7fa21e000640 1 -- 172.21.15.83:0/3492441656 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa218004c50 con 0x55efd4ca1940 2024-04-24T09:17:58.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.668+0000 7fa21e000640 1 -- 172.21.15.83:0/3492441656 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa218010790 con 0x55efd4ca1940 2024-04-24T09:17:58.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.668+0000 7fa23666a740 1 -- 172.21.15.83:0/3492441656 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55efd4ca1940 msgr2=0x55efd4ca3d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:58.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.668+0000 7fa231800640 1 --2- 172.21.15.83:0/90116403 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa2140a3da0 0x7fa2140a6260 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fa224031040 tx=0x7fa224009360 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:58.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.668+0000 7fa23666a740 1 --2- 172.21.15.83:0/3492441656 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55efd4ca1940 0x55efd4ca3d30 secure :-1 s=READY pgs=2867 cs=0 l=1 rev1=1 crypto rx=0x7fa21800cc00 tx=0x7fa21800e720 comp rx=0 tx=0).stop 2024-04-24T09:17:58.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.669+0000 7fa23666a740 1 -- 172.21.15.83:0/3492441656 shutdown_connections 2024-04-24T09:17:58.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.669+0000 7fa23666a740 1 --2- 172.21.15.83:0/3492441656 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55efd4ca1940 0x55efd4ca3d30 unknown :-1 s=CLOSED pgs=2867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:58.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.669+0000 7fa23666a740 1 --2- 172.21.15.83:0/3492441656 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55efd4c97270 0x55efd4ca1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:58.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.669+0000 7fa23666a740 1 --2- 172.21.15.83:0/3492441656 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55efd4c96950 0x55efd4c96d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:58.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.669+0000 7fa23666a740 1 -- 172.21.15.83:0/3492441656 >> 172.21.15.83:0/3492441656 conn(0x55efd4c92c70 msgr2=0x55efd4c93050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:58.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.669+0000 7fa23666a740 1 -- 172.21.15.83:0/3492441656 shutdown_connections 2024-04-24T09:17:58.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.669+0000 7fa23666a740 1 -- 172.21.15.83:0/3492441656 wait complete. 2024-04-24T09:17:58.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.669+0000 7fa23666a740 1 -- start start 2024-04-24T09:17:58.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.669+0000 7fa23666a740 1 Processor -- start 2024-04-24T09:17:58.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.670+0000 7fa23666a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55efd4c96950 0x55efd4ca1320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:58.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.670+0000 7fa23666a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55efd4c97270 0x55efd4c9e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:58.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.670+0000 7fa23666a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55efd4ca1940 0x55efd4c9ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:58.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.670+0000 7fa23666a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55efd4ca47d0 con 0x55efd4c97270 2024-04-24T09:17:58.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.670+0000 7fa23666a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55efd4bb5810 con 0x55efd4ca1940 2024-04-24T09:17:58.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.670+0000 7fa23666a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55efd4ca4950 con 0x55efd4c96950 2024-04-24T09:17:58.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.670+0000 7fa23666a740 10 client.0 _ll_register_callbacks cb 0x55efd4b83140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:17:58.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.670+0000 7fa22be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55efd4c97270 0x55efd4c9e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:58.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.670+0000 7fa22be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55efd4c97270 0x55efd4c9e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60180/0 (socket says 172.21.15.83:60180) 2024-04-24T09:17:58.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.670+0000 7fa22be00640 1 -- 172.21.15.83:0/1517871749 learned_addr learned my addr 172.21.15.83:0/1517871749 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:17:58.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.670+0000 7fa231800640 1 --2- 172.21.15.83:0/1517871749 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55efd4ca1940 0x55efd4c9ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:58.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.670+0000 7fa230e00640 1 --2- 172.21.15.83:0/1517871749 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55efd4c96950 0x55efd4ca1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:58.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.670+0000 7fa22be00640 1 -- 172.21.15.83:0/1517871749 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55efd4c96950 msgr2=0x55efd4ca1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:58.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.670+0000 7fa22be00640 1 --2- 172.21.15.83:0/1517871749 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55efd4c96950 0x55efd4ca1320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:58.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.670+0000 7fa22be00640 1 -- 172.21.15.83:0/1517871749 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55efd4ca1940 msgr2=0x55efd4c9ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:58.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.670+0000 7fa22be00640 1 --2- 172.21.15.83:0/1517871749 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55efd4ca1940 0x55efd4c9ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:58.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.670+0000 7fa22be00640 1 -- 172.21.15.83:0/1517871749 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55efd4bc63f0 con 0x55efd4c97270 2024-04-24T09:17:58.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.671+0000 7fa231800640 1 --2- 172.21.15.83:0/1517871749 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55efd4ca1940 0x55efd4c9ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:17:58.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.671+0000 7fa22be00640 1 --2- 172.21.15.83:0/1517871749 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55efd4c97270 0x55efd4c9e610 secure :-1 s=READY pgs=2868 cs=0 l=1 rev1=1 crypto rx=0x7fa218010470 tx=0x7fa21800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f522d33fae2fe66 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:58.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.671+0000 7fa212a00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:17:58.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.671+0000 7fa212a00640 1 -- 172.21.15.83:0/1517871749 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa21802a060 con 0x55efd4c97270 2024-04-24T09:17:58.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.671+0000 7fa23666a740 1 -- 172.21.15.83:0/1517871749 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa218004c50 con 0x55efd4c97270 2024-04-24T09:17:58.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.671+0000 7fa212a00640 1 -- 172.21.15.83:0/1517871749 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(134..134 src has 1..134) ==== 9583+0+0 (secure 0 0 0) 0x7fa218005010 con 0x55efd4c97270 2024-04-24T09:17:58.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.671+0000 7fa212a00640 1 -- 172.21.15.83:0/1517871749 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7fa218031050 con 0x55efd4c97270 2024-04-24T09:17:58.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.671+0000 7fa23666a740 10 client.24363 fetch_fsmap learned FSMap version 79 2024-04-24T09:17:58.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.671+0000 7fa23666a740 1 -- 172.21.15.83:0/1517871749 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=79,monmap=4+}) -- 0x55efd490e3f0 con 0x55efd4c97270 2024-04-24T09:17:58.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.672+0000 7fa212a00640 1 -- 172.21.15.83:0/1517871749 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa218004c50 con 0x55efd4c97270 2024-04-24T09:17:58.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.672+0000 7fa212a00640 1 -- 172.21.15.83:0/1517871749 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 79) ==== 8398+0+0 (secure 0 0 0) 0x7fa21801a460 con 0x55efd4c97270 2024-04-24T09:17:58.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.672+0000 7fa23666a740 10 client.24363 fetch_fsmap finished waiting for FSMap version 79 2024-04-24T09:17:58.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.672+0000 7fa23666a740 10 client.24363 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-24T09:17:58.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.672+0000 7fa23666a740 20 client.24363 populate_metadata read hostname 'smithi083' 2024-04-24T09:17:58.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.672+0000 7fa23666a740 1 --2- 172.21.15.83:0/1517871749 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55efd4cb1a20 0x55efd4cb3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:58.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.672+0000 7fa23666a740 4 client.24363 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:17:58.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.672+0000 7fa23666a740 1 -- 172.21.15.83:0/1517871749 --> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55efd4c09bf0 con 0x55efd4cb1a20 2024-04-24T09:17:58.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.673+0000 7fa230e00640 1 --2- 172.21.15.83:0/1517871749 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55efd4cb1a20 0x55efd4cb3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:58.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.674+0000 7fa230e00640 1 --2- 172.21.15.83:0/1517871749 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55efd4cb1a20 0x55efd4cb3e10 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=aff379adf3a7f7f8 server_cookie=d7b8b65de0ba3ef0 in_seq=0 out_seq=0 2024-04-24T09:17:58.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.674+0000 7fa212a00640 10 client.24363 ms_handle_connect on v2:172.21.15.189:6838/2548965336 2024-04-24T09:17:58.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.676+0000 7fa212a00640 1 -- 172.21.15.83:0/1517871749 <== mds.4 v2:172.21.15.189:6838/2548965336 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55efd4c09bf0 con 0x55efd4cb1a20 2024-04-24T09:17:58.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.676+0000 7fa212a00640 10 client.24363 handle_command_reply: tid=0 2024-04-24T09:17:58.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.676+0000 7fa23666a740 1 client.24363 shutdown 2024-04-24T09:17:58.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.676+0000 7fa23666a740 1 -- 172.21.15.83:0/1517871749 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55efd4c97270 msgr2=0x55efd4c9e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:58.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.676+0000 7fa23666a740 1 --2- 172.21.15.83:0/1517871749 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55efd4c97270 0x55efd4c9e610 secure :-1 s=READY pgs=2868 cs=0 l=1 rev1=1 crypto rx=0x7fa218010470 tx=0x7fa21800c940 comp rx=0 tx=0).stop 2024-04-24T09:17:58.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.676+0000 7fa23666a740 1 -- 172.21.15.83:0/1517871749 shutdown_connections 2024-04-24T09:17:58.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.676+0000 7fa23666a740 1 --2- 172.21.15.83:0/1517871749 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55efd4cb1a20 0x55efd4cb3e10 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:58.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.676+0000 7fa23666a740 1 --2- 172.21.15.83:0/1517871749 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55efd4ca1940 0x55efd4c9ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:58.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.676+0000 7fa23666a740 1 --2- 172.21.15.83:0/1517871749 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55efd4c97270 0x55efd4c9e610 unknown :-1 s=CLOSED pgs=2868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:58.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.676+0000 7fa212a00640 0 client.24363 ms_handle_reset on v2:172.21.15.189:6838/2548965336 2024-04-24T09:17:58.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.676+0000 7fa23666a740 1 --2- 172.21.15.83:0/1517871749 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55efd4c96950 0x55efd4ca1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:58.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.676+0000 7fa23666a740 1 -- 172.21.15.83:0/1517871749 >> 172.21.15.83:0/1517871749 conn(0x55efd4c92c70 msgr2=0x55efd4c98a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:58.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.676+0000 7fa23666a740 1 -- 172.21.15.83:0/1517871749 shutdown_connections 2024-04-24T09:17:58.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.676+0000 7fa23666a740 1 -- 172.21.15.83:0/1517871749 wait complete. 2024-04-24T09:17:58.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.676+0000 7fa23666a740 20 client.24363 trim_cache size 0 max 16384 2024-04-24T09:17:58.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.691+0000 7fa23666a740 1 Processor -- start 2024-04-24T09:17:58.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.691+0000 7fa23666a740 1 -- start start 2024-04-24T09:17:58.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.691+0000 7fa23666a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55efd4c96950 0x55efd4cad6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:58.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.691+0000 7fa23666a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55efd4c97270 0x55efd4ca0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:58.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.691+0000 7fa23666a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55efd4ca1940 0x55efd4cb9550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:58.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.691+0000 7fa23666a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa22c1b5bc0 con 0x55efd4c96950 2024-04-24T09:17:58.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.691+0000 7fa23666a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa22c1b77c0 con 0x55efd4c97270 2024-04-24T09:17:58.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.691+0000 7fa23666a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa22c1b6290 con 0x55efd4ca1940 2024-04-24T09:17:58.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.692+0000 7fa230e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55efd4c96950 0x55efd4cad6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:58.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.692+0000 7fa230e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55efd4c96950 0x55efd4cad6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60182/0 (socket says 172.21.15.83:60182) 2024-04-24T09:17:58.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.692+0000 7fa230e00640 1 -- 172.21.15.83:0/3049967244 learned_addr learned my addr 172.21.15.83:0/3049967244 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:17:58.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.692+0000 7fa231800640 1 --2- 172.21.15.83:0/3049967244 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55efd4ca1940 0x55efd4cb9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:58.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.692+0000 7fa230e00640 1 -- 172.21.15.83:0/3049967244 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55efd4ca1940 msgr2=0x55efd4cb9550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:58.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.692+0000 7fa230e00640 1 --2- 172.21.15.83:0/3049967244 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55efd4ca1940 0x55efd4cb9550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:58.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.692+0000 7fa22be00640 1 --2- 172.21.15.83:0/3049967244 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55efd4c97270 0x55efd4ca0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:58.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.692+0000 7fa230e00640 1 -- 172.21.15.83:0/3049967244 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55efd4c97270 msgr2=0x55efd4ca0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:58.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.692+0000 7fa230e00640 1 --2- 172.21.15.83:0/3049967244 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55efd4c97270 0x55efd4ca0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:58.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.692+0000 7fa230e00640 1 -- 172.21.15.83:0/3049967244 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55efd49d87d0 con 0x55efd4c96950 2024-04-24T09:17:58.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.692+0000 7fa231800640 1 --2- 172.21.15.83:0/3049967244 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55efd4ca1940 0x55efd4cb9550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:17:58.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.692+0000 7fa22be00640 1 --2- 172.21.15.83:0/3049967244 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55efd4c97270 0x55efd4ca0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:17:58.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.692+0000 7fa230e00640 1 --2- 172.21.15.83:0/3049967244 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55efd4c96950 0x55efd4cad6c0 secure :-1 s=READY pgs=2869 cs=0 l=1 rev1=1 crypto rx=0x7fa220009610 tx=0x7fa2200d2e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12a0338b5caf9d95 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:58.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.693+0000 7fa21d600640 1 -- 172.21.15.83:0/3049967244 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa220020040 con 0x55efd4c96950 2024-04-24T09:17:58.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.693+0000 7fa21d600640 1 -- 172.21.15.83:0/3049967244 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa220009de0 con 0x55efd4c96950 2024-04-24T09:17:58.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.693+0000 7fa21d600640 1 -- 172.21.15.83:0/3049967244 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa220098b60 con 0x55efd4c96950 2024-04-24T09:17:58.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.693+0000 7fa23666a740 1 -- 172.21.15.83:0/3049967244 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55efd4c96950 msgr2=0x55efd4cad6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:58.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.693+0000 7fa23666a740 1 --2- 172.21.15.83:0/3049967244 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55efd4c96950 0x55efd4cad6c0 secure :-1 s=READY pgs=2869 cs=0 l=1 rev1=1 crypto rx=0x7fa220009610 tx=0x7fa2200d2e20 comp rx=0 tx=0).stop 2024-04-24T09:17:58.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.693+0000 7fa23666a740 1 -- 172.21.15.83:0/3049967244 shutdown_connections 2024-04-24T09:17:58.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.693+0000 7fa23666a740 1 --2- 172.21.15.83:0/3049967244 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55efd4ca1940 0x55efd4cb9550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:58.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.693+0000 7fa23666a740 1 --2- 172.21.15.83:0/3049967244 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55efd4c97270 0x55efd4ca0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:58.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.693+0000 7fa23666a740 1 --2- 172.21.15.83:0/3049967244 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55efd4c96950 0x55efd4cad6c0 unknown :-1 s=CLOSED pgs=2869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:58.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.693+0000 7fa23666a740 1 -- 172.21.15.83:0/3049967244 >> 172.21.15.83:0/3049967244 conn(0x55efd4c92c70 msgr2=0x55efd4ca3be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:58.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.694+0000 7fa23666a740 1 -- 172.21.15.83:0/3049967244 shutdown_connections 2024-04-24T09:17:58.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.694+0000 7fa23666a740 1 -- 172.21.15.83:0/3049967244 wait complete. 2024-04-24T09:17:58.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.694+0000 7fa23666a740 1 -- start start 2024-04-24T09:17:58.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.694+0000 7fa23666a740 1 Processor -- start 2024-04-24T09:17:58.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.695+0000 7fa23666a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55efd4c97270 0x55efd4ca2530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:58.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.695+0000 7fa23666a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55efd4ca1940 0x55efd4ca2a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:58.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.695+0000 7fa23666a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55efd4ca2fb0 0x55efd4ca8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:58.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.695+0000 7fa23666a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa22c1b70c0 con 0x55efd4ca2fb0 2024-04-24T09:17:58.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.695+0000 7fa23666a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa22c1b6990 con 0x55efd4c97270 2024-04-24T09:17:58.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.695+0000 7fa23666a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa22c1b3bc0 con 0x55efd4ca1940 2024-04-24T09:17:58.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.695+0000 7fa23666a740 10 client.0 _ll_register_callbacks cb 0x7fa218014070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:17:58.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.695+0000 7fa231800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55efd4ca2fb0 0x55efd4ca8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:58.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.695+0000 7fa231800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55efd4ca2fb0 0x55efd4ca8bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60198/0 (socket says 172.21.15.83:60198) 2024-04-24T09:17:58.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.695+0000 7fa231800640 1 -- 172.21.15.83:0/3860488979 learned_addr learned my addr 172.21.15.83:0/3860488979 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:17:58.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.695+0000 7fa230e00640 1 --2- 172.21.15.83:0/3860488979 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55efd4c97270 0x55efd4ca2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:58.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.695+0000 7fa22be00640 1 --2- 172.21.15.83:0/3860488979 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55efd4ca1940 0x55efd4ca2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:58.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.695+0000 7fa231800640 1 -- 172.21.15.83:0/3860488979 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55efd4ca1940 msgr2=0x55efd4ca2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:58.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.695+0000 7fa231800640 1 --2- 172.21.15.83:0/3860488979 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55efd4ca1940 0x55efd4ca2a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:58.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.695+0000 7fa231800640 1 -- 172.21.15.83:0/3860488979 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55efd4c97270 msgr2=0x55efd4ca2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:58.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.695+0000 7fa231800640 1 --2- 172.21.15.83:0/3860488979 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55efd4c97270 0x55efd4ca2530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:58.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.695+0000 7fa231800640 1 -- 172.21.15.83:0/3860488979 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55efd4ca92c0 con 0x55efd4ca2fb0 2024-04-24T09:17:58.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.695+0000 7fa22be00640 1 --2- 172.21.15.83:0/3860488979 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55efd4ca1940 0x55efd4ca2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:17:58.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.696+0000 7fa231800640 1 --2- 172.21.15.83:0/3860488979 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55efd4ca2fb0 0x55efd4ca8bd0 secure :-1 s=READY pgs=2870 cs=0 l=1 rev1=1 crypto rx=0x7fa224009ca0 tx=0x7fa2240095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb3b43ec92e708d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:58.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.696+0000 7fa212a00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:17:58.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.696+0000 7fa212a00640 1 -- 172.21.15.83:0/3860488979 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa224003fc0 con 0x55efd4ca2fb0 2024-04-24T09:17:58.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.696+0000 7fa212a00640 1 -- 172.21.15.83:0/3860488979 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(134..134 src has 1..134) ==== 9583+0+0 (secure 0 0 0) 0x7fa224004160 con 0x55efd4ca2fb0 2024-04-24T09:17:58.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.696+0000 7fa23666a740 1 -- 172.21.15.83:0/3860488979 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa220009de0 con 0x55efd4ca2fb0 2024-04-24T09:17:58.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.696+0000 7fa212a00640 1 -- 172.21.15.83:0/3860488979 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7fa224004360 con 0x55efd4ca2fb0 2024-04-24T09:17:58.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.696+0000 7fa23666a740 10 client.24369 fetch_fsmap learned FSMap version 79 2024-04-24T09:17:58.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.696+0000 7fa23666a740 1 -- 172.21.15.83:0/3860488979 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=79,monmap=4+}) -- 0x55efd4d17dc0 con 0x55efd4ca2fb0 2024-04-24T09:17:58.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.696+0000 7fa212a00640 1 -- 172.21.15.83:0/3860488979 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa220009de0 con 0x55efd4ca2fb0 2024-04-24T09:17:58.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.697+0000 7fa212a00640 1 -- 172.21.15.83:0/3860488979 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 79) ==== 8398+0+0 (secure 0 0 0) 0x7fa224044040 con 0x55efd4ca2fb0 2024-04-24T09:17:58.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.697+0000 7fa23666a740 10 client.24369 fetch_fsmap finished waiting for FSMap version 79 2024-04-24T09:17:58.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.697+0000 7fa23666a740 10 client.24369 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-04-24T09:17:58.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.697+0000 7fa23666a740 20 client.24369 populate_metadata read hostname 'smithi083' 2024-04-24T09:17:58.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.697+0000 7fa23666a740 1 --2- 172.21.15.83:0/3860488979 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55efd4d18480 0x55efd4d1a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:58.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.697+0000 7fa23666a740 4 client.24369 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:17:58.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.697+0000 7fa23666a740 1 -- 172.21.15.83:0/3860488979 --> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55efd4c03a60 con 0x55efd4d18480 2024-04-24T09:17:58.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.697+0000 7fa230e00640 1 --2- 172.21.15.83:0/3860488979 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55efd4d18480 0x55efd4d1a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:58.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.698+0000 7fa230e00640 1 --2- 172.21.15.83:0/3860488979 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55efd4d18480 0x55efd4d1a8a0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d394a27eb8215f3b server_cookie=d9e234123c0bf4d5 in_seq=0 out_seq=0 2024-04-24T09:17:58.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.698+0000 7fa212a00640 10 client.24369 ms_handle_connect on v2:172.21.15.189:6838/2548965336 2024-04-24T09:17:58.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.699+0000 7fa212a00640 1 -- 172.21.15.83:0/3860488979 <== mds.4 v2:172.21.15.189:6838/2548965336 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55efd4c03a60 con 0x55efd4d18480 2024-04-24T09:17:58.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.699+0000 7fa212a00640 10 client.24369 handle_command_reply: tid=0 2024-04-24T09:17:58.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.699+0000 7fa23666a740 1 client.24369 shutdown 2024-04-24T09:17:58.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.699+0000 7fa23666a740 1 -- 172.21.15.83:0/3860488979 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55efd4ca2fb0 msgr2=0x55efd4ca8bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:58.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.699+0000 7fa23666a740 1 --2- 172.21.15.83:0/3860488979 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55efd4ca2fb0 0x55efd4ca8bd0 secure :-1 s=READY pgs=2870 cs=0 l=1 rev1=1 crypto rx=0x7fa224009ca0 tx=0x7fa2240095c0 comp rx=0 tx=0).stop 2024-04-24T09:17:58.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.699+0000 7fa23666a740 1 -- 172.21.15.83:0/3860488979 shutdown_connections 2024-04-24T09:17:58.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.699+0000 7fa23666a740 1 --2- 172.21.15.83:0/3860488979 >> [v2:172.21.15.189:6838/2548965336,v1:172.21.15.189:6839/2548965336] conn(0x55efd4d18480 0x55efd4d1a8a0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:58.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.699+0000 7fa23666a740 1 --2- 172.21.15.83:0/3860488979 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55efd4ca2fb0 0x55efd4ca8bd0 unknown :-1 s=CLOSED pgs=2870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:58.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.699+0000 7fa23666a740 1 --2- 172.21.15.83:0/3860488979 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55efd4ca1940 0x55efd4ca2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:58.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.699+0000 7fa23666a740 1 --2- 172.21.15.83:0/3860488979 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55efd4c97270 0x55efd4ca2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:58.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.699+0000 7fa23666a740 1 -- 172.21.15.83:0/3860488979 >> 172.21.15.83:0/3860488979 conn(0x55efd4c92c70 msgr2=0x55efd4c91560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:58.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.699+0000 7fa212a00640 0 client.24369 ms_handle_reset on v2:172.21.15.189:6838/2548965336 2024-04-24T09:17:58.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.699+0000 7fa23666a740 1 -- 172.21.15.83:0/3860488979 shutdown_connections 2024-04-24T09:17:58.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.699+0000 7fa23666a740 1 -- 172.21.15.83:0/3860488979 wait complete. 2024-04-24T09:17:58.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.699+0000 7fa23666a740 20 client.24369 trim_cache size 0 max 16384 2024-04-24T09:17:58.702 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:17:58.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.700+0000 7fa233800640 1 -- 172.21.15.83:0/90116403 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa2140a3da0 msgr2=0x7fa2140a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:58.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.700+0000 7fa233800640 1 --2- 172.21.15.83:0/90116403 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa2140a3da0 0x7fa2140a6260 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fa224031040 tx=0x7fa224009360 comp rx=0 tx=0).stop 2024-04-24T09:17:58.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.700+0000 7fa233800640 1 -- 172.21.15.83:0/90116403 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa22c108610 msgr2=0x7fa22c1a4930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:58.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.700+0000 7fa233800640 1 --2- 172.21.15.83:0/90116403 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa22c108610 0x7fa22c1a4930 secure :-1 s=READY pgs=2866 cs=0 l=1 rev1=1 crypto rx=0x7fa22000d770 tx=0x7fa22000dc30 comp rx=0 tx=0).stop 2024-04-24T09:17:58.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.700+0000 7fa233800640 1 -- 172.21.15.83:0/90116403 shutdown_connections 2024-04-24T09:17:58.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.700+0000 7fa233800640 1 --2- 172.21.15.83:0/90116403 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa2140a3da0 0x7fa2140a6260 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:58.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.700+0000 7fa233800640 1 --2- 172.21.15.83:0/90116403 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa22c10e660 0x7fa22c19eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:58.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.700+0000 7fa233800640 1 --2- 172.21.15.83:0/90116403 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa22c108fd0 0x7fa22c1a4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:58.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.700+0000 7fa233800640 1 --2- 172.21.15.83:0/90116403 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa22c108610 0x7fa22c1a4930 unknown :-1 s=CLOSED pgs=2866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:58.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.700+0000 7fa233800640 1 -- 172.21.15.83:0/90116403 >> 172.21.15.83:0/90116403 conn(0x7fa22c076540 msgr2=0x7fa22c077b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:58.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.700+0000 7fa233800640 1 -- 172.21.15.83:0/90116403 shutdown_connections 2024-04-24T09:17:58.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.700+0000 7fa233800640 1 -- 172.21.15.83:0/90116403 wait complete. 2024-04-24T09:17:58.704 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:17:58 smithi138 ceph-mon[31441]: pgmap v833: 129 pgs: 129 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-04-24T09:17:58.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:17:58 smithi189 ceph-mon[31791]: pgmap v833: 129 pgs: 129 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-04-24T09:17:58.714 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:17:58 smithi083 ceph-mon[27907]: pgmap v833: 129 pgs: 129 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-04-24T09:17:58.718 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-24T09:17:58.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.924+0000 7fb113e00640 1 -- 172.21.15.83:0/2198999035 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb1141071f0 msgr2=0x7fb11410edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:58.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.924+0000 7fb113e00640 1 --2- 172.21.15.83:0/2198999035 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb1141071f0 0x7fb11410edc0 secure :-1 s=READY pgs=2871 cs=0 l=1 rev1=1 crypto rx=0x7fb10c009ca0 tx=0x7fb10c02a8c0 comp rx=0 tx=0).stop 2024-04-24T09:17:58.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.925+0000 7fb113e00640 1 -- 172.21.15.83:0/2198999035 shutdown_connections 2024-04-24T09:17:58.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.925+0000 7fb113e00640 1 --2- 172.21.15.83:0/2198999035 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb1141071f0 0x7fb11410edc0 unknown :-1 s=CLOSED pgs=2871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:58.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.925+0000 7fb113e00640 1 --2- 172.21.15.83:0/2198999035 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb114102ad0 0x7fb114106b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:58.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.925+0000 7fb113e00640 1 --2- 172.21.15.83:0/2198999035 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb114102120 0x7fb114102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:58.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.925+0000 7fb113e00640 1 -- 172.21.15.83:0/2198999035 >> 172.21.15.83:0/2198999035 conn(0x7fb1140fd9a0 msgr2=0x7fb1140ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:58.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.925+0000 7fb113e00640 1 -- 172.21.15.83:0/2198999035 shutdown_connections 2024-04-24T09:17:58.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.925+0000 7fb113e00640 1 -- 172.21.15.83:0/2198999035 wait complete. 2024-04-24T09:17:58.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.926+0000 7fb113e00640 1 Processor -- start 2024-04-24T09:17:58.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.926+0000 7fb113e00640 1 -- start start 2024-04-24T09:17:58.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.926+0000 7fb113e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb114102120 0x7fb1141a0590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:58.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.926+0000 7fb113e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb1141071f0 0x7fb1141a0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:58.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.926+0000 7fb113e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb11419a6f0 0x7fb11419ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:58.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.926+0000 7fb113e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb114112190 con 0x7fb1141071f0 2024-04-24T09:17:58.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.926+0000 7fb113e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb114112010 con 0x7fb114102120 2024-04-24T09:17:58.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.926+0000 7fb113e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb114112310 con 0x7fb11419a6f0 2024-04-24T09:17:58.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.926+0000 7fb112000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb1141071f0 0x7fb1141a0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:58.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.926+0000 7fb112000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb1141071f0 0x7fb1141a0ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:32994/0 (socket says 172.21.15.83:32994) 2024-04-24T09:17:58.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.926+0000 7fb112000640 1 -- 172.21.15.83:0/1651989674 learned_addr learned my addr 172.21.15.83:0/1651989674 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:17:58.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.927+0000 7fb112a00640 1 --2- 172.21.15.83:0/1651989674 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb114102120 0x7fb1141a0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:58.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.927+0000 7fb113400640 1 --2- 172.21.15.83:0/1651989674 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb11419a6f0 0x7fb11419ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:58.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.927+0000 7fb112000640 1 -- 172.21.15.83:0/1651989674 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb11419a6f0 msgr2=0x7fb11419ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:58.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.927+0000 7fb112000640 1 --2- 172.21.15.83:0/1651989674 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb11419a6f0 0x7fb11419ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:58.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.927+0000 7fb112000640 1 -- 172.21.15.83:0/1651989674 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb114102120 msgr2=0x7fb1141a0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:58.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.927+0000 7fb112000640 1 --2- 172.21.15.83:0/1651989674 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb114102120 0x7fb1141a0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:58.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.927+0000 7fb112000640 1 -- 172.21.15.83:0/1651989674 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb11419b2f0 con 0x7fb1141071f0 2024-04-24T09:17:58.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.927+0000 7fb112a00640 1 --2- 172.21.15.83:0/1651989674 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb114102120 0x7fb1141a0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:17:58.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.927+0000 7fb112000640 1 --2- 172.21.15.83:0/1651989674 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb1141071f0 0x7fb1141a0ad0 secure :-1 s=READY pgs=2872 cs=0 l=1 rev1=1 crypto rx=0x7fb10000d770 tx=0x7fb10000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:58.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.927+0000 7fb118a00640 1 -- 172.21.15.83:0/1651989674 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb100017070 con 0x7fb1141071f0 2024-04-24T09:17:58.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.927+0000 7fb113e00640 1 -- 172.21.15.83:0/1651989674 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1141a7330 con 0x7fb1141071f0 2024-04-24T09:17:58.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.927+0000 7fb118a00640 1 -- 172.21.15.83:0/1651989674 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb100009c30 con 0x7fb1141071f0 2024-04-24T09:17:58.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.928+0000 7fb113e00640 1 -- 172.21.15.83:0/1651989674 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1141a7730 con 0x7fb1141071f0 2024-04-24T09:17:58.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.928+0000 7fb118a00640 1 -- 172.21.15.83:0/1651989674 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb100021060 con 0x7fb1141071f0 2024-04-24T09:17:58.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.931+0000 7fb11cc0d740 1 Processor -- start 2024-04-24T09:17:58.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.931+0000 7fb11cc0d740 1 -- start start 2024-04-24T09:17:58.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.931+0000 7fb11cc0d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b6b5f22b40 0x55b6b5f22f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:58.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.931+0000 7fb11cc0d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b6b5f23460 0x55b6b5f2d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:58.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.931+0000 7fb118a00640 1 -- 172.21.15.83:0/1651989674 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb10001a070 con 0x7fb1141071f0 2024-04-24T09:17:58.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.931+0000 7fb11cc0d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b6b5f2db30 0x55b6b5f2ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:58.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.931+0000 7fb11cc0d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55b6b5e881d0 con 0x55b6b5f2db30 2024-04-24T09:17:58.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.931+0000 7fb11cc0d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b6b5e2e450 con 0x55b6b5f22b40 2024-04-24T09:17:58.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.931+0000 7fb11cc0d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55b6b5e84190 con 0x55b6b5f23460 2024-04-24T09:17:58.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.932+0000 7fb112000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b6b5f2db30 0x55b6b5f2ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:58.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.932+0000 7fb112000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b6b5f2db30 0x55b6b5f2ff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33002/0 (socket says 172.21.15.83:33002) 2024-04-24T09:17:58.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.932+0000 7fb112000640 1 -- 172.21.15.83:0/3352837732 learned_addr learned my addr 172.21.15.83:0/3352837732 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:17:58.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.932+0000 7fb112a00640 1 --2- 172.21.15.83:0/3352837732 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b6b5f23460 0x55b6b5f2d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:58.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.932+0000 7fb113400640 1 --2- 172.21.15.83:0/3352837732 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b6b5f22b40 0x55b6b5f22f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:58.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.932+0000 7fb112000640 1 -- 172.21.15.83:0/3352837732 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b6b5f23460 msgr2=0x55b6b5f2d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:58.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.932+0000 7fb112000640 1 --2- 172.21.15.83:0/3352837732 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b6b5f23460 0x55b6b5f2d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:58.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.932+0000 7fb112000640 1 -- 172.21.15.83:0/3352837732 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b6b5f22b40 msgr2=0x55b6b5f22f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:58.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.932+0000 7fb112000640 1 --2- 172.21.15.83:0/3352837732 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b6b5f22b40 0x55b6b5f22f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:58.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.932+0000 7fb112000640 1 -- 172.21.15.83:0/3352837732 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6b5eeb310 con 0x55b6b5f2db30 2024-04-24T09:17:58.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.932+0000 7fb113400640 1 --2- 172.21.15.83:0/3352837732 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b6b5f22b40 0x55b6b5f22f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:17:58.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.933+0000 7fb112000640 1 --2- 172.21.15.83:0/3352837732 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b6b5f2db30 0x55b6b5f2ff20 secure :-1 s=READY pgs=2873 cs=0 l=1 rev1=1 crypto rx=0x7fb1000cd9d0 tx=0x7fb1000d8750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd13af707a7abb47 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:58.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.933+0000 7fb0ffe00640 1 -- 172.21.15.83:0/3352837732 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1000d0db0 con 0x55b6b5f2db30 2024-04-24T09:17:58.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.933+0000 7fb0ffe00640 1 -- 172.21.15.83:0/3352837732 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb1000dd070 con 0x55b6b5f2db30 2024-04-24T09:17:58.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.933+0000 7fb0ffe00640 1 -- 172.21.15.83:0/3352837732 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1000e27e0 con 0x55b6b5f2db30 2024-04-24T09:17:58.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.933+0000 7fb118a00640 1 --2- 172.21.15.83:0/1651989674 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb0f80a3d20 0x7fb0f80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:58.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.933+0000 7fb118a00640 1 -- 172.21.15.83:0/1651989674 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 9583+0+0 (secure 0 0 0) 0x7fb1000d07c0 con 0x7fb1141071f0 2024-04-24T09:17:58.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.933+0000 7fb113400640 1 --2- 172.21.15.83:0/1651989674 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb0f80a3d20 0x7fb0f80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:58.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.934+0000 7fb11cc0d740 1 -- 172.21.15.83:0/3352837732 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b6b5f2db30 msgr2=0x55b6b5f2ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:58.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.934+0000 7fb11cc0d740 1 --2- 172.21.15.83:0/3352837732 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b6b5f2db30 0x55b6b5f2ff20 secure :-1 s=READY pgs=2873 cs=0 l=1 rev1=1 crypto rx=0x7fb1000cd9d0 tx=0x7fb1000d8750 comp rx=0 tx=0).stop 2024-04-24T09:17:58.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.934+0000 7fb11cc0d740 1 -- 172.21.15.83:0/3352837732 shutdown_connections 2024-04-24T09:17:58.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.934+0000 7fb11cc0d740 1 --2- 172.21.15.83:0/3352837732 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b6b5f2db30 0x55b6b5f2ff20 unknown :-1 s=CLOSED pgs=2873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:58.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.934+0000 7fb11cc0d740 1 --2- 172.21.15.83:0/3352837732 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b6b5f23460 0x55b6b5f2d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:58.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.934+0000 7fb11cc0d740 1 --2- 172.21.15.83:0/3352837732 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b6b5f22b40 0x55b6b5f22f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:58.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.934+0000 7fb11cc0d740 1 -- 172.21.15.83:0/3352837732 >> 172.21.15.83:0/3352837732 conn(0x55b6b5f1ee60 msgr2=0x55b6b5f1f240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:58.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.934+0000 7fb113400640 1 --2- 172.21.15.83:0/1651989674 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb0f80a3d20 0x7fb0f80a61e0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fb10c02f3a0 tx=0x7fb10c02f310 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:58.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.934+0000 7fb11cc0d740 1 -- 172.21.15.83:0/3352837732 shutdown_connections 2024-04-24T09:17:58.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.934+0000 7fb11cc0d740 1 -- 172.21.15.83:0/3352837732 wait complete. 2024-04-24T09:17:58.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.935+0000 7fb11cc0d740 1 -- start start 2024-04-24T09:17:58.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.935+0000 7fb11cc0d740 1 Processor -- start 2024-04-24T09:17:58.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.935+0000 7fb11cc0d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b6b5f22b40 0x55b6b5f2d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:58.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.935+0000 7fb11cc0d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b6b5f23460 0x55b6b5f2a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:58.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.935+0000 7fb11cc0d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b6b5f2db30 0x55b6b5f2afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:58.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.935+0000 7fb11cc0d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55b6b5f309c0 con 0x55b6b5f2db30 2024-04-24T09:17:58.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.935+0000 7fb11cc0d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b6b5e41810 con 0x55b6b5f22b40 2024-04-24T09:17:58.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.935+0000 7fb11cc0d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55b6b5f30b40 con 0x55b6b5f23460 2024-04-24T09:17:58.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.935+0000 7fb11cc0d740 10 client.0 _ll_register_callbacks cb 0x55b6b5e0f140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:17:58.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.936+0000 7fb113400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b6b5f2db30 0x55b6b5f2afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:58.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.936+0000 7fb113400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b6b5f2db30 0x55b6b5f2afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33014/0 (socket says 172.21.15.83:33014) 2024-04-24T09:17:58.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.936+0000 7fb113400640 1 -- 172.21.15.83:0/514434727 learned_addr learned my addr 172.21.15.83:0/514434727 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:17:58.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.936+0000 7fb112000640 1 --2- 172.21.15.83:0/514434727 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b6b5f23460 0x55b6b5f2a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:58.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.936+0000 7fb112a00640 1 --2- 172.21.15.83:0/514434727 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b6b5f22b40 0x55b6b5f2d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:58.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.936+0000 7fb113400640 1 -- 172.21.15.83:0/514434727 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b6b5f23460 msgr2=0x55b6b5f2a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:58.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.936+0000 7fb113400640 1 --2- 172.21.15.83:0/514434727 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b6b5f23460 0x55b6b5f2a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:58.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.936+0000 7fb113400640 1 -- 172.21.15.83:0/514434727 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b6b5f22b40 msgr2=0x55b6b5f2d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:58.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.936+0000 7fb113400640 1 --2- 172.21.15.83:0/514434727 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b6b5f22b40 0x55b6b5f2d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:58.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.936+0000 7fb113400640 1 -- 172.21.15.83:0/514434727 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6b5e523f0 con 0x55b6b5f2db30 2024-04-24T09:17:58.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.936+0000 7fb113400640 1 --2- 172.21.15.83:0/514434727 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b6b5f2db30 0x55b6b5f2afd0 secure :-1 s=READY pgs=2874 cs=0 l=1 rev1=1 crypto rx=0x7fb10c031680 tx=0x7fb10c0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=767def78b1f3e5c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:58.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.936+0000 7fb0fcc00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:17:58.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.937+0000 7fb0fcc00640 1 -- 172.21.15.83:0/514434727 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb10c04e070 con 0x55b6b5f2db30 2024-04-24T09:17:58.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.937+0000 7fb0fcc00640 1 -- 172.21.15.83:0/514434727 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(134..134 src has 1..134) ==== 9583+0+0 (secure 0 0 0) 0x7fb10c004740 con 0x55b6b5f2db30 2024-04-24T09:17:58.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.937+0000 7fb11cc0d740 1 -- 172.21.15.83:0/514434727 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1000dd070 con 0x55b6b5f2db30 2024-04-24T09:17:58.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.937+0000 7fb0fcc00640 1 -- 172.21.15.83:0/514434727 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7fb10c05a050 con 0x55b6b5f2db30 2024-04-24T09:17:58.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.937+0000 7fb11cc0d740 10 client.24381 fetch_fsmap learned FSMap version 79 2024-04-24T09:17:58.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.937+0000 7fb11cc0d740 1 -- 172.21.15.83:0/514434727 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=79,monmap=4+}) -- 0x55b6b5b9a3f0 con 0x55b6b5f2db30 2024-04-24T09:17:58.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.937+0000 7fb0fcc00640 1 -- 172.21.15.83:0/514434727 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb1000dd070 con 0x55b6b5f2db30 2024-04-24T09:17:58.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.938+0000 7fb0fcc00640 1 -- 172.21.15.83:0/514434727 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 79) ==== 8398+0+0 (secure 0 0 0) 0x7fb10c043040 con 0x55b6b5f2db30 2024-04-24T09:17:58.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.938+0000 7fb11cc0d740 10 client.24381 fetch_fsmap finished waiting for FSMap version 79 2024-04-24T09:17:58.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.938+0000 7fb11cc0d740 10 client.24381 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-24T09:17:58.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.938+0000 7fb11cc0d740 20 client.24381 populate_metadata read hostname 'smithi083' 2024-04-24T09:17:58.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.938+0000 7fb11cc0d740 1 --2- 172.21.15.83:0/514434727 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55b6b5f3dd10 0x55b6b5f40190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:58.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.938+0000 7fb11cc0d740 4 client.24381 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:17:58.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.938+0000 7fb11cc0d740 1 -- 172.21.15.83:0/514434727 --> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b6b5e95bf0 con 0x55b6b5f3dd10 2024-04-24T09:17:58.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.938+0000 7fb112a00640 1 --2- 172.21.15.83:0/514434727 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55b6b5f3dd10 0x55b6b5f40190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:58.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.939+0000 7fb112a00640 1 --2- 172.21.15.83:0/514434727 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55b6b5f3dd10 0x55b6b5f40190 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=370f1928243dbafc server_cookie=211c91ecb0eed920 in_seq=0 out_seq=0 2024-04-24T09:17:58.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.939+0000 7fb0fcc00640 10 client.24381 ms_handle_connect on v2:172.21.15.138:6832/1157967658 2024-04-24T09:17:58.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.942+0000 7fb0fcc00640 1 -- 172.21.15.83:0/514434727 <== mds.3 v2:172.21.15.138:6832/1157967658 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b6b5e95bf0 con 0x55b6b5f3dd10 2024-04-24T09:17:58.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.942+0000 7fb0fcc00640 10 client.24381 handle_command_reply: tid=0 2024-04-24T09:17:58.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.942+0000 7fb11cc0d740 1 client.24381 shutdown 2024-04-24T09:17:58.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.942+0000 7fb11cc0d740 1 -- 172.21.15.83:0/514434727 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b6b5f2db30 msgr2=0x55b6b5f2afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:58.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.942+0000 7fb11cc0d740 1 --2- 172.21.15.83:0/514434727 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b6b5f2db30 0x55b6b5f2afd0 secure :-1 s=READY pgs=2874 cs=0 l=1 rev1=1 crypto rx=0x7fb10c031680 tx=0x7fb10c0095c0 comp rx=0 tx=0).stop 2024-04-24T09:17:58.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.942+0000 7fb11cc0d740 1 -- 172.21.15.83:0/514434727 shutdown_connections 2024-04-24T09:17:58.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.942+0000 7fb11cc0d740 1 --2- 172.21.15.83:0/514434727 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b6b5f2db30 0x55b6b5f2afd0 unknown :-1 s=CLOSED pgs=2874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:58.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.942+0000 7fb11cc0d740 1 --2- 172.21.15.83:0/514434727 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55b6b5f3dd10 0x55b6b5f40190 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:58.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.942+0000 7fb11cc0d740 1 --2- 172.21.15.83:0/514434727 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b6b5f23460 0x55b6b5f2a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:58.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.942+0000 7fb11cc0d740 1 --2- 172.21.15.83:0/514434727 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b6b5f22b40 0x55b6b5f2d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:58.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.942+0000 7fb11cc0d740 1 -- 172.21.15.83:0/514434727 >> 172.21.15.83:0/514434727 conn(0x55b6b5f1ee60 msgr2=0x55b6b5f24c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:58.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.942+0000 7fb0fcc00640 0 client.24381 ms_handle_reset on v2:172.21.15.138:6832/1157967658 2024-04-24T09:17:58.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.942+0000 7fb11cc0d740 1 -- 172.21.15.83:0/514434727 shutdown_connections 2024-04-24T09:17:58.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.942+0000 7fb11cc0d740 1 -- 172.21.15.83:0/514434727 wait complete. 2024-04-24T09:17:58.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.942+0000 7fb11cc0d740 20 client.24381 trim_cache size 0 max 16384 2024-04-24T09:17:58.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.957+0000 7fb11cc0d740 1 Processor -- start 2024-04-24T09:17:58.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.957+0000 7fb11cc0d740 1 -- start start 2024-04-24T09:17:58.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.957+0000 7fb11cc0d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b6b5f22b40 0x55b6b5f2d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:58.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.957+0000 7fb11cc0d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b6b5f23460 0x55b6b5f45aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:58.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.957+0000 7fb11cc0d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b6b5f2db30 0x55b6b5f45fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:58.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.957+0000 7fb11cc0d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb1141b15f0 con 0x55b6b5f22b40 2024-04-24T09:17:58.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.957+0000 7fb11cc0d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb1141b31f0 con 0x55b6b5f2db30 2024-04-24T09:17:58.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.957+0000 7fb11cc0d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb1141b1cc0 con 0x55b6b5f23460 2024-04-24T09:17:58.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.958+0000 7fb112a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b6b5f22b40 0x55b6b5f2d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:58.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.958+0000 7fb112a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b6b5f22b40 0x55b6b5f2d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33020/0 (socket says 172.21.15.83:33020) 2024-04-24T09:17:58.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.958+0000 7fb112a00640 1 -- 172.21.15.83:0/4053469108 learned_addr learned my addr 172.21.15.83:0/4053469108 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:17:58.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.958+0000 7fb113400640 1 --2- 172.21.15.83:0/4053469108 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b6b5f2db30 0x55b6b5f45fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:58.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.958+0000 7fb112000640 1 --2- 172.21.15.83:0/4053469108 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b6b5f23460 0x55b6b5f45aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:58.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.958+0000 7fb112a00640 1 -- 172.21.15.83:0/4053469108 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b6b5f23460 msgr2=0x55b6b5f45aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:58.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.958+0000 7fb112a00640 1 --2- 172.21.15.83:0/4053469108 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b6b5f23460 0x55b6b5f45aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:58.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.958+0000 7fb112a00640 1 -- 172.21.15.83:0/4053469108 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b6b5f2db30 msgr2=0x55b6b5f45fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:58.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.958+0000 7fb112a00640 1 --2- 172.21.15.83:0/4053469108 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b6b5f2db30 0x55b6b5f45fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:58.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.958+0000 7fb112a00640 1 -- 172.21.15.83:0/4053469108 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6b5c647d0 con 0x55b6b5f22b40 2024-04-24T09:17:58.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.959+0000 7fb112a00640 1 --2- 172.21.15.83:0/4053469108 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b6b5f22b40 0x55b6b5f2d0d0 secure :-1 s=READY pgs=2875 cs=0 l=1 rev1=1 crypto rx=0x55b6b5f40a40 tx=0x7fb104005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=415d96e249448cf9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:58.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.959+0000 7fb0ff400640 1 -- 172.21.15.83:0/4053469108 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb104013d70 con 0x55b6b5f22b40 2024-04-24T09:17:58.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.959+0000 7fb0ff400640 1 -- 172.21.15.83:0/4053469108 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb10401e030 con 0x55b6b5f22b40 2024-04-24T09:17:58.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.959+0000 7fb0ff400640 1 -- 172.21.15.83:0/4053469108 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb104004730 con 0x55b6b5f22b40 2024-04-24T09:17:58.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.960+0000 7fb11cc0d740 1 -- 172.21.15.83:0/4053469108 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b6b5f22b40 msgr2=0x55b6b5f2d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:58.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.960+0000 7fb11cc0d740 1 --2- 172.21.15.83:0/4053469108 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b6b5f22b40 0x55b6b5f2d0d0 secure :-1 s=READY pgs=2875 cs=0 l=1 rev1=1 crypto rx=0x55b6b5f40a40 tx=0x7fb104005850 comp rx=0 tx=0).stop 2024-04-24T09:17:58.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.960+0000 7fb11cc0d740 1 -- 172.21.15.83:0/4053469108 shutdown_connections 2024-04-24T09:17:58.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.960+0000 7fb11cc0d740 1 --2- 172.21.15.83:0/4053469108 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b6b5f2db30 0x55b6b5f45fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:58.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.960+0000 7fb11cc0d740 1 --2- 172.21.15.83:0/4053469108 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b6b5f23460 0x55b6b5f45aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:58.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.960+0000 7fb11cc0d740 1 --2- 172.21.15.83:0/4053469108 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b6b5f22b40 0x55b6b5f2d0d0 unknown :-1 s=CLOSED pgs=2875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:58.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.960+0000 7fb11cc0d740 1 -- 172.21.15.83:0/4053469108 >> 172.21.15.83:0/4053469108 conn(0x55b6b5f1ee60 msgr2=0x55b6b5f1df50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:58.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.960+0000 7fb11cc0d740 1 -- 172.21.15.83:0/4053469108 shutdown_connections 2024-04-24T09:17:58.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.960+0000 7fb11cc0d740 1 -- 172.21.15.83:0/4053469108 wait complete. 2024-04-24T09:17:58.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.960+0000 7fb11cc0d740 1 -- start start 2024-04-24T09:17:58.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.961+0000 7fb11cc0d740 1 Processor -- start 2024-04-24T09:17:58.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.961+0000 7fb11cc0d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b6b5f22b40 0x55b6b5f362e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:58.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.961+0000 7fb11cc0d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b6b5f23460 0x55b6b5f27ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:58.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.961+0000 7fb11cc0d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b6b5f2db30 0x55b6b5f2ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:58.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.961+0000 7fb11cc0d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb1141b2af0 con 0x55b6b5f2db30 2024-04-24T09:17:58.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.961+0000 7fb11cc0d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb1141b23c0 con 0x55b6b5f23460 2024-04-24T09:17:58.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.961+0000 7fb11cc0d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb1141af5f0 con 0x55b6b5f22b40 2024-04-24T09:17:58.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.961+0000 7fb11cc0d740 10 client.0 _ll_register_callbacks cb 0x7fb10c009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:17:58.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.961+0000 7fb113400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b6b5f2db30 0x55b6b5f2ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:58.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.961+0000 7fb113400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b6b5f2db30 0x55b6b5f2ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33034/0 (socket says 172.21.15.83:33034) 2024-04-24T09:17:58.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.961+0000 7fb113400640 1 -- 172.21.15.83:0/51269259 learned_addr learned my addr 172.21.15.83:0/51269259 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:17:58.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.962+0000 7fb112a00640 1 --2- 172.21.15.83:0/51269259 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b6b5f22b40 0x55b6b5f362e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:58.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.962+0000 7fb113400640 1 -- 172.21.15.83:0/51269259 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b6b5f22b40 msgr2=0x55b6b5f362e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:58.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.962+0000 7fb113400640 1 --2- 172.21.15.83:0/51269259 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b6b5f22b40 0x55b6b5f362e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:58.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.962+0000 7fb113400640 1 -- 172.21.15.83:0/51269259 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b6b5f23460 msgr2=0x55b6b5f27ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:58.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.962+0000 7fb113400640 1 --2- 172.21.15.83:0/51269259 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b6b5f23460 0x55b6b5f27ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:58.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.962+0000 7fb113400640 1 -- 172.21.15.83:0/51269259 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6b5f2f110 con 0x55b6b5f2db30 2024-04-24T09:17:58.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.962+0000 7fb112a00640 1 --2- 172.21.15.83:0/51269259 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b6b5f22b40 0x55b6b5f362e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:17:58.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.962+0000 7fb113400640 1 --2- 172.21.15.83:0/51269259 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b6b5f2db30 0x55b6b5f2ebd0 secure :-1 s=READY pgs=2876 cs=0 l=1 rev1=1 crypto rx=0x7fb10c02ade0 tx=0x7fb10c0459e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4929284bb58599f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:58.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.962+0000 7fb0fcc00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:17:58.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.962+0000 7fb0fcc00640 1 -- 172.21.15.83:0/51269259 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb10c057070 con 0x55b6b5f2db30 2024-04-24T09:17:58.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.962+0000 7fb11cc0d740 1 -- 172.21.15.83:0/51269259 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb10401e030 con 0x55b6b5f2db30 2024-04-24T09:17:58.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.962+0000 7fb0fcc00640 1 -- 172.21.15.83:0/51269259 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(134..134 src has 1..134) ==== 9583+0+0 (secure 0 0 0) 0x7fb10c004410 con 0x55b6b5f2db30 2024-04-24T09:17:58.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.963+0000 7fb0fcc00640 1 -- 172.21.15.83:0/51269259 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7fb10c04e040 con 0x55b6b5f2db30 2024-04-24T09:17:58.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.963+0000 7fb11cc0d740 10 client.24387 fetch_fsmap learned FSMap version 79 2024-04-24T09:17:58.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.963+0000 7fb11cc0d740 1 -- 172.21.15.83:0/51269259 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=79,monmap=4+}) -- 0x55b6b5fa4900 con 0x55b6b5f2db30 2024-04-24T09:17:58.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.963+0000 7fb0fcc00640 1 -- 172.21.15.83:0/51269259 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb10401e030 con 0x55b6b5f2db30 2024-04-24T09:17:58.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.963+0000 7fb0fcc00640 1 -- 172.21.15.83:0/51269259 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 79) ==== 8398+0+0 (secure 0 0 0) 0x7fb10c043040 con 0x55b6b5f2db30 2024-04-24T09:17:58.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.963+0000 7fb11cc0d740 10 client.24387 fetch_fsmap finished waiting for FSMap version 79 2024-04-24T09:17:58.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.963+0000 7fb11cc0d740 10 client.24387 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-24T09:17:58.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.963+0000 7fb11cc0d740 20 client.24387 populate_metadata read hostname 'smithi083' 2024-04-24T09:17:58.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.963+0000 7fb11cc0d740 1 --2- 172.21.15.83:0/51269259 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55b6b5fa5100 0x55b6b5fa7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:58.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.963+0000 7fb11cc0d740 4 client.24387 mds_command: new command op to 34427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:17:58.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.963+0000 7fb11cc0d740 1 -- 172.21.15.83:0/51269259 --> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b6b5e8fa60 con 0x55b6b5fa5100 2024-04-24T09:17:58.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.964+0000 7fb112a00640 1 --2- 172.21.15.83:0/51269259 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55b6b5fa5100 0x55b6b5fa7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:58.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.965+0000 7fb112a00640 1 --2- 172.21.15.83:0/51269259 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55b6b5fa5100 0x55b6b5fa7560 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d816147ac2e04ab2 server_cookie=52ec6b06f31ce0ae in_seq=0 out_seq=0 2024-04-24T09:17:58.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.965+0000 7fb0fcc00640 10 client.24387 ms_handle_connect on v2:172.21.15.138:6832/1157967658 2024-04-24T09:17:58.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.965+0000 7fb0fcc00640 1 -- 172.21.15.83:0/51269259 <== mds.3 v2:172.21.15.138:6832/1157967658 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b6b5e8fa60 con 0x55b6b5fa5100 2024-04-24T09:17:58.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.965+0000 7fb0fcc00640 10 client.24387 handle_command_reply: tid=0 2024-04-24T09:17:58.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.965+0000 7fb11cc0d740 1 client.24387 shutdown 2024-04-24T09:17:58.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.965+0000 7fb11cc0d740 1 -- 172.21.15.83:0/51269259 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b6b5f2db30 msgr2=0x55b6b5f2ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:58.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.965+0000 7fb11cc0d740 1 --2- 172.21.15.83:0/51269259 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b6b5f2db30 0x55b6b5f2ebd0 secure :-1 s=READY pgs=2876 cs=0 l=1 rev1=1 crypto rx=0x7fb10c02ade0 tx=0x7fb10c0459e0 comp rx=0 tx=0).stop 2024-04-24T09:17:58.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.965+0000 7fb11cc0d740 1 -- 172.21.15.83:0/51269259 shutdown_connections 2024-04-24T09:17:58.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.965+0000 7fb11cc0d740 1 --2- 172.21.15.83:0/51269259 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b6b5f2db30 0x55b6b5f2ebd0 unknown :-1 s=CLOSED pgs=2876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:58.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.965+0000 7fb11cc0d740 1 --2- 172.21.15.83:0/51269259 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b6b5f23460 0x55b6b5f27ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:58.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.965+0000 7fb11cc0d740 1 --2- 172.21.15.83:0/51269259 >> [v2:172.21.15.138:6832/1157967658,v1:172.21.15.138:6833/1157967658] conn(0x55b6b5fa5100 0x55b6b5fa7560 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:58.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.965+0000 7fb11cc0d740 1 --2- 172.21.15.83:0/51269259 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b6b5f22b40 0x55b6b5f362e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:58.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.965+0000 7fb0fcc00640 0 client.24387 ms_handle_reset on v2:172.21.15.138:6832/1157967658 2024-04-24T09:17:58.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.965+0000 7fb11cc0d740 1 -- 172.21.15.83:0/51269259 >> 172.21.15.83:0/51269259 conn(0x55b6b5f1ee60 msgr2=0x55b6b5f242d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:58.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.966+0000 7fb11cc0d740 1 -- 172.21.15.83:0/51269259 shutdown_connections 2024-04-24T09:17:58.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.966+0000 7fb11cc0d740 1 -- 172.21.15.83:0/51269259 wait complete. 2024-04-24T09:17:58.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.966+0000 7fb11cc0d740 20 client.24387 trim_cache size 0 max 16384 2024-04-24T09:17:58.968 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:17:58.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.966+0000 7fb113e00640 1 -- 172.21.15.83:0/1651989674 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb0f80a3d20 msgr2=0x7fb0f80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:58.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.966+0000 7fb113e00640 1 --2- 172.21.15.83:0/1651989674 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb0f80a3d20 0x7fb0f80a61e0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fb10c02f3a0 tx=0x7fb10c02f310 comp rx=0 tx=0).stop 2024-04-24T09:17:58.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.966+0000 7fb113e00640 1 -- 172.21.15.83:0/1651989674 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb1141071f0 msgr2=0x7fb1141a0ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:58.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.966+0000 7fb113e00640 1 --2- 172.21.15.83:0/1651989674 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb1141071f0 0x7fb1141a0ad0 secure :-1 s=READY pgs=2872 cs=0 l=1 rev1=1 crypto rx=0x7fb10000d770 tx=0x7fb10000dc30 comp rx=0 tx=0).stop 2024-04-24T09:17:58.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.966+0000 7fb113e00640 1 -- 172.21.15.83:0/1651989674 shutdown_connections 2024-04-24T09:17:58.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.966+0000 7fb113e00640 1 --2- 172.21.15.83:0/1651989674 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb0f80a3d20 0x7fb0f80a61e0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:58.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.966+0000 7fb113e00640 1 --2- 172.21.15.83:0/1651989674 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb11419a6f0 0x7fb11419ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:58.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.966+0000 7fb113e00640 1 --2- 172.21.15.83:0/1651989674 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb1141071f0 0x7fb1141a0ad0 unknown :-1 s=CLOSED pgs=2872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:58.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.966+0000 7fb113e00640 1 --2- 172.21.15.83:0/1651989674 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb114102120 0x7fb1141a0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:58.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.966+0000 7fb113e00640 1 -- 172.21.15.83:0/1651989674 >> 172.21.15.83:0/1651989674 conn(0x7fb1140fd9a0 msgr2=0x7fb1140fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:58.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.967+0000 7fb113e00640 1 -- 172.21.15.83:0/1651989674 shutdown_connections 2024-04-24T09:17:58.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:58.967+0000 7fb113e00640 1 -- 172.21.15.83:0/1651989674 wait complete. 2024-04-24T09:17:58.983 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T09:17:59.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.177+0000 7fda49e00640 1 -- 172.21.15.83:0/4031700291 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda44108580 msgr2=0x7fda44108a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:59.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.177+0000 7fda49e00640 1 --2- 172.21.15.83:0/4031700291 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda44108580 0x7fda44108a00 secure :-1 s=READY pgs=2877 cs=0 l=1 rev1=1 crypto rx=0x7fda30009930 tx=0x7fda3002a420 comp rx=0 tx=0).stop 2024-04-24T09:17:59.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.177+0000 7fda49e00640 1 -- 172.21.15.83:0/4031700291 shutdown_connections 2024-04-24T09:17:59.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.177+0000 7fda49e00640 1 --2- 172.21.15.83:0/4031700291 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fda44108f40 0x7fda441153b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:59.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.177+0000 7fda49e00640 1 --2- 172.21.15.83:0/4031700291 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda44108580 0x7fda44108a00 unknown :-1 s=CLOSED pgs=2877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:59.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.177+0000 7fda49e00640 1 --2- 172.21.15.83:0/4031700291 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fda4410d500 0x7fda4410d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:59.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.177+0000 7fda49e00640 1 -- 172.21.15.83:0/4031700291 >> 172.21.15.83:0/4031700291 conn(0x7fda44077e20 msgr2=0x7fda44106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:59.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.177+0000 7fda49e00640 1 -- 172.21.15.83:0/4031700291 shutdown_connections 2024-04-24T09:17:59.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.177+0000 7fda49e00640 1 -- 172.21.15.83:0/4031700291 wait complete. 2024-04-24T09:17:59.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.178+0000 7fda49e00640 1 Processor -- start 2024-04-24T09:17:59.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.178+0000 7fda49e00640 1 -- start start 2024-04-24T09:17:59.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.178+0000 7fda49e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fda44108580 0x7fda441a48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:59.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.178+0000 7fda49e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fda44108f40 0x7fda441a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:59.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.178+0000 7fda49e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda4410d500 0x7fda4419eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:59.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.178+0000 7fda49e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fda441187d0 con 0x7fda4410d500 2024-04-24T09:17:59.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.178+0000 7fda49e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fda44118650 con 0x7fda44108580 2024-04-24T09:17:59.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.178+0000 7fda49e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fda44118950 con 0x7fda44108f40 2024-04-24T09:17:59.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.179+0000 7fda49400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda4410d500 0x7fda4419eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:59.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.179+0000 7fda49400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda4410d500 0x7fda4419eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33052/0 (socket says 172.21.15.83:33052) 2024-04-24T09:17:59.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.179+0000 7fda49400640 1 -- 172.21.15.83:0/2856199736 learned_addr learned my addr 172.21.15.83:0/2856199736 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:17:59.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.179+0000 7fda43e00640 1 --2- 172.21.15.83:0/2856199736 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fda44108f40 0x7fda441a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:59.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.179+0000 7fda48a00640 1 --2- 172.21.15.83:0/2856199736 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fda44108580 0x7fda441a48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:59.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.179+0000 7fda49400640 1 -- 172.21.15.83:0/2856199736 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fda44108f40 msgr2=0x7fda441a4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:59.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.179+0000 7fda49400640 1 --2- 172.21.15.83:0/2856199736 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fda44108f40 0x7fda441a4e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:59.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.179+0000 7fda49400640 1 -- 172.21.15.83:0/2856199736 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fda44108580 msgr2=0x7fda441a48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:59.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.179+0000 7fda49400640 1 --2- 172.21.15.83:0/2856199736 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fda44108580 0x7fda441a48f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:59.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.179+0000 7fda49400640 1 -- 172.21.15.83:0/2856199736 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda4419f270 con 0x7fda4410d500 2024-04-24T09:17:59.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.179+0000 7fda49400640 1 --2- 172.21.15.83:0/2856199736 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda4410d500 0x7fda4419eaf0 secure :-1 s=READY pgs=2878 cs=0 l=1 rev1=1 crypto rx=0x7fda3c00e960 tx=0x7fda3c00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:59.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.180+0000 7fda41600640 1 -- 172.21.15.83:0/2856199736 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda3c004d20 con 0x7fda4410d500 2024-04-24T09:17:59.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.180+0000 7fda41600640 1 -- 172.21.15.83:0/2856199736 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fda3c005020 con 0x7fda4410d500 2024-04-24T09:17:59.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.180+0000 7fda41600640 1 -- 172.21.15.83:0/2856199736 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda3c0116d0 con 0x7fda4410d500 2024-04-24T09:17:59.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.180+0000 7fda49e00640 1 -- 172.21.15.83:0/2856199736 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda4419f560 con 0x7fda4410d500 2024-04-24T09:17:59.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.180+0000 7fda49e00640 1 -- 172.21.15.83:0/2856199736 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda441ab7b0 con 0x7fda4410d500 2024-04-24T09:17:59.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.184+0000 7fda4e224740 1 Processor -- start 2024-04-24T09:17:59.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.184+0000 7fda41600640 1 -- 172.21.15.83:0/2856199736 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fda3c0051d0 con 0x7fda4410d500 2024-04-24T09:17:59.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.184+0000 7fda4e224740 1 -- start start 2024-04-24T09:17:59.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.185+0000 7fda4e224740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564ec1ee1b40 0x564ec1ee1f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:59.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.185+0000 7fda4e224740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564ec1ee2460 0x564ec1eec5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:59.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.185+0000 7fda4e224740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ec1eecb30 0x564ec1eeef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:59.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.185+0000 7fda4e224740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x564ec1e471d0 con 0x564ec1eecb30 2024-04-24T09:17:59.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.185+0000 7fda4e224740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x564ec1ded450 con 0x564ec1ee2460 2024-04-24T09:17:59.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.185+0000 7fda4e224740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x564ec1e43190 con 0x564ec1ee1b40 2024-04-24T09:17:59.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.185+0000 7fda43e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ec1eecb30 0x564ec1eeef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:59.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.185+0000 7fda43e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ec1eecb30 0x564ec1eeef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33062/0 (socket says 172.21.15.83:33062) 2024-04-24T09:17:59.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.185+0000 7fda43e00640 1 -- 172.21.15.83:0/2905322593 learned_addr learned my addr 172.21.15.83:0/2905322593 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:17:59.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.185+0000 7fda48a00640 1 --2- 172.21.15.83:0/2905322593 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564ec1ee2460 0x564ec1eec5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:59.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.185+0000 7fda43e00640 1 --2- 172.21.15.83:0/2905322593 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564ec1ee1b40 0x564ec1ee1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:59.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.186+0000 7fda43e00640 1 -- 172.21.15.83:0/2905322593 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564ec1ee1b40 msgr2=0x564ec1ee1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:59.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.186+0000 7fda43e00640 1 --2- 172.21.15.83:0/2905322593 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564ec1ee1b40 0x564ec1ee1f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:59.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.186+0000 7fda43e00640 1 -- 172.21.15.83:0/2905322593 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564ec1ee2460 msgr2=0x564ec1eec5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:59.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.186+0000 7fda43e00640 1 --2- 172.21.15.83:0/2905322593 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564ec1ee2460 0x564ec1eec5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:59.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.186+0000 7fda43e00640 1 -- 172.21.15.83:0/2905322593 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ec1eaa310 con 0x564ec1eecb30 2024-04-24T09:17:59.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.186+0000 7fda41600640 1 --2- 172.21.15.83:0/2856199736 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fda2c0a3d20 0x7fda2c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:59.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.186+0000 7fda41600640 1 -- 172.21.15.83:0/2856199736 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 9583+0+0 (secure 0 0 0) 0x7fda3c0d0e60 con 0x7fda4410d500 2024-04-24T09:17:59.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.186+0000 7fda48a00640 1 --2- 172.21.15.83:0/2856199736 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fda2c0a3d20 0x7fda2c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:59.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.186+0000 7fda43e00640 1 --2- 172.21.15.83:0/2905322593 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ec1eecb30 0x564ec1eeef20 secure :-1 s=READY pgs=2879 cs=0 l=1 rev1=1 crypto rx=0x7fda30009930 tx=0x7fda30031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36f868b90165ac29 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:59.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.186+0000 7fda35c00640 1 -- 172.21.15.83:0/2905322593 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda30041070 con 0x564ec1eecb30 2024-04-24T09:17:59.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.187+0000 7fda35c00640 1 -- 172.21.15.83:0/2905322593 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fda30003aa0 con 0x564ec1eecb30 2024-04-24T09:17:59.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.187+0000 7fda35c00640 1 -- 172.21.15.83:0/2905322593 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda30035040 con 0x564ec1eecb30 2024-04-24T09:17:59.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.187+0000 7fda4e224740 1 -- 172.21.15.83:0/2905322593 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ec1eecb30 msgr2=0x564ec1eeef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:59.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.187+0000 7fda4e224740 1 --2- 172.21.15.83:0/2905322593 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ec1eecb30 0x564ec1eeef20 secure :-1 s=READY pgs=2879 cs=0 l=1 rev1=1 crypto rx=0x7fda30009930 tx=0x7fda30031400 comp rx=0 tx=0).stop 2024-04-24T09:17:59.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.187+0000 7fda48a00640 1 --2- 172.21.15.83:0/2856199736 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fda2c0a3d20 0x7fda2c0a61e0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x564ec1ee3100 tx=0x7fda38005f30 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:59.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.187+0000 7fda4e224740 1 -- 172.21.15.83:0/2905322593 shutdown_connections 2024-04-24T09:17:59.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.187+0000 7fda4e224740 1 --2- 172.21.15.83:0/2905322593 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ec1eecb30 0x564ec1eeef20 unknown :-1 s=CLOSED pgs=2879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:59.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.187+0000 7fda4e224740 1 --2- 172.21.15.83:0/2905322593 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564ec1ee2460 0x564ec1eec5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:59.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.187+0000 7fda4e224740 1 --2- 172.21.15.83:0/2905322593 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564ec1ee1b40 0x564ec1ee1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:59.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.187+0000 7fda4e224740 1 -- 172.21.15.83:0/2905322593 >> 172.21.15.83:0/2905322593 conn(0x564ec1edde60 msgr2=0x564ec1ede240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:59.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.187+0000 7fda4e224740 1 -- 172.21.15.83:0/2905322593 shutdown_connections 2024-04-24T09:17:59.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.188+0000 7fda4e224740 1 -- 172.21.15.83:0/2905322593 wait complete. 2024-04-24T09:17:59.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.188+0000 7fda4e224740 1 -- start start 2024-04-24T09:17:59.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.188+0000 7fda4e224740 1 Processor -- start 2024-04-24T09:17:59.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.188+0000 7fda4e224740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564ec1ee1b40 0x564ec1eec540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:59.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.189+0000 7fda4e224740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564ec1ee2460 0x564ec1ee9830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:59.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.189+0000 7fda4e224740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ec1eecb30 0x564ec1ee9e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:59.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.189+0000 7fda4e224740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x564ec1eef960 con 0x564ec1eecb30 2024-04-24T09:17:59.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.189+0000 7fda4e224740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x564ec1e00810 con 0x564ec1ee1b40 2024-04-24T09:17:59.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.189+0000 7fda4e224740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x564ec1eefae0 con 0x564ec1ee2460 2024-04-24T09:17:59.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.189+0000 7fda4e224740 10 client.0 _ll_register_callbacks cb 0x564ec1dce140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:17:59.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.189+0000 7fda49400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ec1eecb30 0x564ec1ee9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:59.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.189+0000 7fda49400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ec1eecb30 0x564ec1ee9e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33068/0 (socket says 172.21.15.83:33068) 2024-04-24T09:17:59.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.189+0000 7fda49400640 1 -- 172.21.15.83:0/1926315432 learned_addr learned my addr 172.21.15.83:0/1926315432 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:17:59.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.189+0000 7fda48a00640 1 --2- 172.21.15.83:0/1926315432 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564ec1ee1b40 0x564ec1eec540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:59.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.189+0000 7fda43e00640 1 --2- 172.21.15.83:0/1926315432 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564ec1ee2460 0x564ec1ee9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:59.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.189+0000 7fda49400640 1 -- 172.21.15.83:0/1926315432 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564ec1ee2460 msgr2=0x564ec1ee9830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:59.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.189+0000 7fda49400640 1 --2- 172.21.15.83:0/1926315432 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564ec1ee2460 0x564ec1ee9830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:59.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.189+0000 7fda49400640 1 -- 172.21.15.83:0/1926315432 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564ec1ee1b40 msgr2=0x564ec1eec540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:59.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.189+0000 7fda49400640 1 --2- 172.21.15.83:0/1926315432 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564ec1ee1b40 0x564ec1eec540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:59.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.189+0000 7fda49400640 1 -- 172.21.15.83:0/1926315432 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ec1e113f0 con 0x564ec1eecb30 2024-04-24T09:17:59.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.189+0000 7fda48a00640 1 --2- 172.21.15.83:0/1926315432 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564ec1ee1b40 0x564ec1eec540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:17:59.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.190+0000 7fda49400640 1 --2- 172.21.15.83:0/1926315432 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ec1eecb30 0x564ec1ee9e40 secure :-1 s=READY pgs=2880 cs=0 l=1 rev1=1 crypto rx=0x7fda3c098de0 tx=0x7fda3c09c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2955ac8eca38b6a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:59.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.190+0000 7fda2a000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:17:59.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.190+0000 7fda2a000640 1 -- 172.21.15.83:0/1926315432 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda3c01f040 con 0x564ec1eecb30 2024-04-24T09:17:59.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.190+0000 7fda2a000640 1 -- 172.21.15.83:0/1926315432 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(134..134 src has 1..134) ==== 9583+0+0 (secure 0 0 0) 0x7fda3c0870c0 con 0x564ec1eecb30 2024-04-24T09:17:59.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.190+0000 7fda4e224740 1 -- 172.21.15.83:0/1926315432 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda30003aa0 con 0x564ec1eecb30 2024-04-24T09:17:59.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.190+0000 7fda2a000640 1 -- 172.21.15.83:0/1926315432 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7fda3c0a55f0 con 0x564ec1eecb30 2024-04-24T09:17:59.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.190+0000 7fda4e224740 10 client.24399 fetch_fsmap learned FSMap version 79 2024-04-24T09:17:59.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.190+0000 7fda4e224740 1 -- 172.21.15.83:0/1926315432 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=79,monmap=4+}) -- 0x564ec1b593f0 con 0x564ec1eecb30 2024-04-24T09:17:59.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.190+0000 7fda2a000640 1 -- 172.21.15.83:0/1926315432 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fda30003aa0 con 0x564ec1eecb30 2024-04-24T09:17:59.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.190+0000 7fda2a000640 1 -- 172.21.15.83:0/1926315432 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 79) ==== 8398+0+0 (secure 0 0 0) 0x7fda3c00f040 con 0x564ec1eecb30 2024-04-24T09:17:59.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.191+0000 7fda4e224740 10 client.24399 fetch_fsmap finished waiting for FSMap version 79 2024-04-24T09:17:59.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.191+0000 7fda4e224740 10 client.24399 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-24T09:17:59.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.191+0000 7fda4e224740 20 client.24399 populate_metadata read hostname 'smithi083' 2024-04-24T09:17:59.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.191+0000 7fda4e224740 1 --2- 172.21.15.83:0/1926315432 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x564ec1efcd40 0x564ec1eff1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:59.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.191+0000 7fda4e224740 4 client.24399 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:17:59.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.191+0000 7fda4e224740 1 -- 172.21.15.83:0/1926315432 --> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564ec1e54bf0 con 0x564ec1efcd40 2024-04-24T09:17:59.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.191+0000 7fda48a00640 1 --2- 172.21.15.83:0/1926315432 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x564ec1efcd40 0x564ec1eff1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:59.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.192+0000 7fda48a00640 1 --2- 172.21.15.83:0/1926315432 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x564ec1efcd40 0x564ec1eff1b0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d862878477d7683f server_cookie=a37dc3b01d8055c4 in_seq=0 out_seq=0 2024-04-24T09:17:59.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.192+0000 7fda2a000640 10 client.24399 ms_handle_connect on v2:172.21.15.138:6834/1909219758 2024-04-24T09:17:59.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.194+0000 7fda2a000640 1 -- 172.21.15.83:0/1926315432 <== mds.2 v2:172.21.15.138:6834/1909219758 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564ec1e54bf0 con 0x564ec1efcd40 2024-04-24T09:17:59.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.194+0000 7fda2a000640 10 client.24399 handle_command_reply: tid=0 2024-04-24T09:17:59.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.194+0000 7fda4e224740 1 client.24399 shutdown 2024-04-24T09:17:59.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.195+0000 7fda4e224740 1 -- 172.21.15.83:0/1926315432 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ec1eecb30 msgr2=0x564ec1ee9e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:59.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.195+0000 7fda4e224740 1 --2- 172.21.15.83:0/1926315432 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ec1eecb30 0x564ec1ee9e40 secure :-1 s=READY pgs=2880 cs=0 l=1 rev1=1 crypto rx=0x7fda3c098de0 tx=0x7fda3c09c980 comp rx=0 tx=0).stop 2024-04-24T09:17:59.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.195+0000 7fda4e224740 1 -- 172.21.15.83:0/1926315432 shutdown_connections 2024-04-24T09:17:59.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.195+0000 7fda4e224740 1 --2- 172.21.15.83:0/1926315432 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x564ec1efcd40 0x564ec1eff1b0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:59.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.195+0000 7fda4e224740 1 --2- 172.21.15.83:0/1926315432 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ec1eecb30 0x564ec1ee9e40 unknown :-1 s=CLOSED pgs=2880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:59.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.195+0000 7fda4e224740 1 --2- 172.21.15.83:0/1926315432 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564ec1ee2460 0x564ec1ee9830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:59.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.195+0000 7fda4e224740 1 --2- 172.21.15.83:0/1926315432 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564ec1ee1b40 0x564ec1eec540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:59.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.195+0000 7fda2a000640 0 client.24399 ms_handle_reset on v2:172.21.15.138:6834/1909219758 2024-04-24T09:17:59.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.195+0000 7fda4e224740 1 -- 172.21.15.83:0/1926315432 >> 172.21.15.83:0/1926315432 conn(0x564ec1edde60 msgr2=0x564ec1ee3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:59.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.195+0000 7fda4e224740 1 -- 172.21.15.83:0/1926315432 shutdown_connections 2024-04-24T09:17:59.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.195+0000 7fda4e224740 1 -- 172.21.15.83:0/1926315432 wait complete. 2024-04-24T09:17:59.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.195+0000 7fda4e224740 20 client.24399 trim_cache size 0 max 16384 2024-04-24T09:17:59.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.209+0000 7fda4e224740 1 Processor -- start 2024-04-24T09:17:59.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.209+0000 7fda4e224740 1 -- start start 2024-04-24T09:17:59.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.210+0000 7fda4e224740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564ec1ee1b40 0x564ec1edcdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:59.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.210+0000 7fda4e224740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ec1ee2460 0x564ec1eec070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:59.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.210+0000 7fda4e224740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564ec1eecb30 0x564ec1f04a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:59.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.210+0000 7fda4e224740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fda441b5960 con 0x564ec1ee2460 2024-04-24T09:17:59.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.210+0000 7fda4e224740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fda441b7560 con 0x564ec1ee1b40 2024-04-24T09:17:59.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.210+0000 7fda4e224740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fda441b6030 con 0x564ec1eecb30 2024-04-24T09:17:59.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.210+0000 7fda43e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ec1ee2460 0x564ec1eec070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:59.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.210+0000 7fda43e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ec1ee2460 0x564ec1eec070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33074/0 (socket says 172.21.15.83:33074) 2024-04-24T09:17:59.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.210+0000 7fda43e00640 1 -- 172.21.15.83:0/776741954 learned_addr learned my addr 172.21.15.83:0/776741954 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:17:59.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.210+0000 7fda48a00640 1 --2- 172.21.15.83:0/776741954 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564ec1ee1b40 0x564ec1edcdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:59.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.211+0000 7fda49400640 1 --2- 172.21.15.83:0/776741954 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564ec1eecb30 0x564ec1f04a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:59.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.211+0000 7fda43e00640 1 -- 172.21.15.83:0/776741954 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564ec1eecb30 msgr2=0x564ec1f04a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:59.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.211+0000 7fda43e00640 1 --2- 172.21.15.83:0/776741954 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564ec1eecb30 0x564ec1f04a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:59.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.211+0000 7fda43e00640 1 -- 172.21.15.83:0/776741954 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564ec1ee1b40 msgr2=0x564ec1edcdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:59.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.211+0000 7fda43e00640 1 --2- 172.21.15.83:0/776741954 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564ec1ee1b40 0x564ec1edcdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:59.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.211+0000 7fda43e00640 1 -- 172.21.15.83:0/776741954 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ec1c237d0 con 0x564ec1ee2460 2024-04-24T09:17:59.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.211+0000 7fda43e00640 1 --2- 172.21.15.83:0/776741954 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ec1ee2460 0x564ec1eec070 secure :-1 s=READY pgs=2881 cs=0 l=1 rev1=1 crypto rx=0x7fda30009930 tx=0x7fda30005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55d9b59c38def366 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:59.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.211+0000 7fda35200640 1 -- 172.21.15.83:0/776741954 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda30041030 con 0x564ec1ee2460 2024-04-24T09:17:59.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.211+0000 7fda35200640 1 -- 172.21.15.83:0/776741954 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fda3004e070 con 0x564ec1ee2460 2024-04-24T09:17:59.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.211+0000 7fda35200640 1 -- 172.21.15.83:0/776741954 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda30031b70 con 0x564ec1ee2460 2024-04-24T09:17:59.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.212+0000 7fda4e224740 1 -- 172.21.15.83:0/776741954 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ec1ee2460 msgr2=0x564ec1eec070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:59.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.212+0000 7fda4e224740 1 --2- 172.21.15.83:0/776741954 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ec1ee2460 0x564ec1eec070 secure :-1 s=READY pgs=2881 cs=0 l=1 rev1=1 crypto rx=0x7fda30009930 tx=0x7fda30005780 comp rx=0 tx=0).stop 2024-04-24T09:17:59.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.212+0000 7fda4e224740 1 -- 172.21.15.83:0/776741954 shutdown_connections 2024-04-24T09:17:59.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.212+0000 7fda4e224740 1 --2- 172.21.15.83:0/776741954 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564ec1eecb30 0x564ec1f04a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:59.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.212+0000 7fda4e224740 1 --2- 172.21.15.83:0/776741954 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ec1ee2460 0x564ec1eec070 unknown :-1 s=CLOSED pgs=2881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:59.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.212+0000 7fda4e224740 1 --2- 172.21.15.83:0/776741954 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564ec1ee1b40 0x564ec1edcdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:59.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.212+0000 7fda4e224740 1 -- 172.21.15.83:0/776741954 >> 172.21.15.83:0/776741954 conn(0x564ec1edde60 msgr2=0x564ec1edc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:59.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.212+0000 7fda4e224740 1 -- 172.21.15.83:0/776741954 shutdown_connections 2024-04-24T09:17:59.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.212+0000 7fda4e224740 1 -- 172.21.15.83:0/776741954 wait complete. 2024-04-24T09:17:59.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.212+0000 7fda4e224740 1 -- start start 2024-04-24T09:17:59.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.213+0000 7fda4e224740 1 Processor -- start 2024-04-24T09:17:59.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.213+0000 7fda4e224740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564ec1ee1b40 0x564ec1ef4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:59.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.213+0000 7fda4e224740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ec1ee2460 0x564ec1ef52d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:59.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.213+0000 7fda4e224740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564ec1eecb30 0x564ec1ee5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:59.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.213+0000 7fda4e224740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fda441b6e60 con 0x564ec1ee2460 2024-04-24T09:17:59.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.213+0000 7fda4e224740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fda441b6730 con 0x564ec1eecb30 2024-04-24T09:17:59.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.213+0000 7fda4e224740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fda441b3960 con 0x564ec1ee1b40 2024-04-24T09:17:59.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.213+0000 7fda4e224740 10 client.0 _ll_register_callbacks cb 0x7fda3c098c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:17:59.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.213+0000 7fda43e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ec1ee2460 0x564ec1ef52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:59.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.213+0000 7fda43e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ec1ee2460 0x564ec1ef52d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33086/0 (socket says 172.21.15.83:33086) 2024-04-24T09:17:59.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.213+0000 7fda43e00640 1 -- 172.21.15.83:0/190559417 learned_addr learned my addr 172.21.15.83:0/190559417 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:17:59.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.214+0000 7fda48a00640 1 --2- 172.21.15.83:0/190559417 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564ec1ee1b40 0x564ec1ef4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:59.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.214+0000 7fda49400640 1 --2- 172.21.15.83:0/190559417 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564ec1eecb30 0x564ec1ee5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:59.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.214+0000 7fda43e00640 1 -- 172.21.15.83:0/190559417 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564ec1ee1b40 msgr2=0x564ec1ef4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:59.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.214+0000 7fda43e00640 1 --2- 172.21.15.83:0/190559417 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564ec1ee1b40 0x564ec1ef4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:59.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.214+0000 7fda43e00640 1 -- 172.21.15.83:0/190559417 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564ec1eecb30 msgr2=0x564ec1ee5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:59.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.214+0000 7fda43e00640 1 --2- 172.21.15.83:0/190559417 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564ec1eecb30 0x564ec1ee5aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:59.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.214+0000 7fda43e00640 1 -- 172.21.15.83:0/190559417 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ec1ee62a0 con 0x564ec1ee2460 2024-04-24T09:17:59.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.214+0000 7fda49400640 1 --2- 172.21.15.83:0/190559417 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564ec1eecb30 0x564ec1ee5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:17:59.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.214+0000 7fda43e00640 1 --2- 172.21.15.83:0/190559417 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ec1ee2460 0x564ec1ef52d0 secure :-1 s=READY pgs=2882 cs=0 l=1 rev1=1 crypto rx=0x7fda30031ed0 tx=0x7fda30031f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2142157f6e1ad91f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:59.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.214+0000 7fda2a000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:17:59.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.214+0000 7fda2a000640 1 -- 172.21.15.83:0/190559417 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda30035030 con 0x564ec1ee2460 2024-04-24T09:17:59.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.214+0000 7fda4e224740 1 -- 172.21.15.83:0/190559417 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda3004e070 con 0x564ec1ee2460 2024-04-24T09:17:59.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.214+0000 7fda2a000640 1 -- 172.21.15.83:0/190559417 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(134..134 src has 1..134) ==== 9583+0+0 (secure 0 0 0) 0x7fda30003a00 con 0x564ec1ee2460 2024-04-24T09:17:59.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.215+0000 7fda2a000640 1 -- 172.21.15.83:0/190559417 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7fda30056070 con 0x564ec1ee2460 2024-04-24T09:17:59.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.215+0000 7fda4e224740 10 client.24405 fetch_fsmap learned FSMap version 79 2024-04-24T09:17:59.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.215+0000 7fda4e224740 1 -- 172.21.15.83:0/190559417 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=79,monmap=4+}) -- 0x564ec1ee7560 con 0x564ec1ee2460 2024-04-24T09:17:59.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.215+0000 7fda2a000640 1 -- 172.21.15.83:0/190559417 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fda3004e070 con 0x564ec1ee2460 2024-04-24T09:17:59.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.215+0000 7fda2a000640 1 -- 172.21.15.83:0/190559417 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 79) ==== 8398+0+0 (secure 0 0 0) 0x7fda3003a070 con 0x564ec1ee2460 2024-04-24T09:17:59.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.215+0000 7fda4e224740 10 client.24405 fetch_fsmap finished waiting for FSMap version 79 2024-04-24T09:17:59.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.215+0000 7fda4e224740 10 client.24405 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-04-24T09:17:59.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.215+0000 7fda4e224740 20 client.24405 populate_metadata read hostname 'smithi083' 2024-04-24T09:17:59.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.215+0000 7fda4e224740 1 --2- 172.21.15.83:0/190559417 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x564ec1f64300 0x564ec1f666f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:59.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.215+0000 7fda4e224740 4 client.24405 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:17:59.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.215+0000 7fda4e224740 1 -- 172.21.15.83:0/190559417 --> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564ec1e4ea60 con 0x564ec1f64300 2024-04-24T09:17:59.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.216+0000 7fda48a00640 1 --2- 172.21.15.83:0/190559417 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x564ec1f64300 0x564ec1f666f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:59.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.217+0000 7fda48a00640 1 --2- 172.21.15.83:0/190559417 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x564ec1f64300 0x564ec1f666f0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=35602c45a47516a3 server_cookie=764cc0ec78ea8b74 in_seq=0 out_seq=0 2024-04-24T09:17:59.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.217+0000 7fda2a000640 10 client.24405 ms_handle_connect on v2:172.21.15.138:6834/1909219758 2024-04-24T09:17:59.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.217+0000 7fda2a000640 1 -- 172.21.15.83:0/190559417 <== mds.2 v2:172.21.15.138:6834/1909219758 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564ec1e4ea60 con 0x564ec1f64300 2024-04-24T09:17:59.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.217+0000 7fda2a000640 10 client.24405 handle_command_reply: tid=0 2024-04-24T09:17:59.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.217+0000 7fda4e224740 1 client.24405 shutdown 2024-04-24T09:17:59.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.218+0000 7fda4e224740 1 -- 172.21.15.83:0/190559417 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ec1ee2460 msgr2=0x564ec1ef52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:59.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.218+0000 7fda4e224740 1 --2- 172.21.15.83:0/190559417 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ec1ee2460 0x564ec1ef52d0 secure :-1 s=READY pgs=2882 cs=0 l=1 rev1=1 crypto rx=0x7fda30031ed0 tx=0x7fda30031f00 comp rx=0 tx=0).stop 2024-04-24T09:17:59.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.218+0000 7fda4e224740 1 -- 172.21.15.83:0/190559417 shutdown_connections 2024-04-24T09:17:59.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.218+0000 7fda4e224740 1 --2- 172.21.15.83:0/190559417 >> [v2:172.21.15.138:6834/1909219758,v1:172.21.15.138:6835/1909219758] conn(0x564ec1f64300 0x564ec1f666f0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:59.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.218+0000 7fda4e224740 1 --2- 172.21.15.83:0/190559417 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564ec1eecb30 0x564ec1ee5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:59.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.218+0000 7fda2a000640 0 client.24405 ms_handle_reset on v2:172.21.15.138:6834/1909219758 2024-04-24T09:17:59.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.218+0000 7fda4e224740 1 --2- 172.21.15.83:0/190559417 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564ec1ee2460 0x564ec1ef52d0 unknown :-1 s=CLOSED pgs=2882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:59.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.218+0000 7fda4e224740 1 --2- 172.21.15.83:0/190559417 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564ec1ee1b40 0x564ec1ef4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:59.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.218+0000 7fda4e224740 1 -- 172.21.15.83:0/190559417 >> 172.21.15.83:0/190559417 conn(0x564ec1edde60 msgr2=0x564ec1edcf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:59.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.218+0000 7fda4e224740 1 -- 172.21.15.83:0/190559417 shutdown_connections 2024-04-24T09:17:59.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.218+0000 7fda4e224740 1 -- 172.21.15.83:0/190559417 wait complete. 2024-04-24T09:17:59.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.218+0000 7fda4e224740 20 client.24405 trim_cache size 0 max 16384 2024-04-24T09:17:59.220 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:17:59.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.219+0000 7fda49e00640 1 -- 172.21.15.83:0/2856199736 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fda2c0a3d20 msgr2=0x7fda2c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:59.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.219+0000 7fda49e00640 1 --2- 172.21.15.83:0/2856199736 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fda2c0a3d20 0x7fda2c0a61e0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x564ec1ee3100 tx=0x7fda38005f30 comp rx=0 tx=0).stop 2024-04-24T09:17:59.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.219+0000 7fda49e00640 1 -- 172.21.15.83:0/2856199736 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda4410d500 msgr2=0x7fda4419eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:59.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.219+0000 7fda49e00640 1 --2- 172.21.15.83:0/2856199736 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda4410d500 0x7fda4419eaf0 secure :-1 s=READY pgs=2878 cs=0 l=1 rev1=1 crypto rx=0x7fda3c00e960 tx=0x7fda3c00ee20 comp rx=0 tx=0).stop 2024-04-24T09:17:59.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.219+0000 7fda49e00640 1 -- 172.21.15.83:0/2856199736 shutdown_connections 2024-04-24T09:17:59.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.219+0000 7fda49e00640 1 --2- 172.21.15.83:0/2856199736 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fda2c0a3d20 0x7fda2c0a61e0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:59.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.219+0000 7fda49e00640 1 --2- 172.21.15.83:0/2856199736 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda4410d500 0x7fda4419eaf0 unknown :-1 s=CLOSED pgs=2878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:59.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.219+0000 7fda49e00640 1 --2- 172.21.15.83:0/2856199736 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fda44108f40 0x7fda441a4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:59.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.219+0000 7fda49e00640 1 --2- 172.21.15.83:0/2856199736 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fda44108580 0x7fda441a48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:59.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.219+0000 7fda49e00640 1 -- 172.21.15.83:0/2856199736 >> 172.21.15.83:0/2856199736 conn(0x7fda44077e20 msgr2=0x7fda440783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:59.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.219+0000 7fda49e00640 1 -- 172.21.15.83:0/2856199736 shutdown_connections 2024-04-24T09:17:59.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.219+0000 7fda49e00640 1 -- 172.21.15.83:0/2856199736 wait complete. 2024-04-24T09:17:59.235 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T09:17:59.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.437+0000 7f0605e00640 1 -- 172.21.15.83:0/2649618302 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06001044b0 msgr2=0x7f0600104890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:59.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.437+0000 7f0605e00640 1 --2- 172.21.15.83:0/2649618302 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06001044b0 0x7f0600104890 secure :-1 s=READY pgs=2883 cs=0 l=1 rev1=1 crypto rx=0x7f05f0009ca0 tx=0x7f05f002a8c0 comp rx=0 tx=0).stop 2024-04-24T09:17:59.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.437+0000 7f0605e00640 1 -- 172.21.15.83:0/2649618302 shutdown_connections 2024-04-24T09:17:59.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.437+0000 7f0605e00640 1 --2- 172.21.15.83:0/2649618302 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0600109580 0x7f0600111150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:59.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.437+0000 7f0605e00640 1 --2- 172.21.15.83:0/2649618302 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0600104e60 0x7f0600108ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:59.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.437+0000 7f0605e00640 1 --2- 172.21.15.83:0/2649618302 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06001044b0 0x7f0600104890 unknown :-1 s=CLOSED pgs=2883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:59.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.438+0000 7f0605e00640 1 -- 172.21.15.83:0/2649618302 >> 172.21.15.83:0/2649618302 conn(0x7f06000ffd90 msgr2=0x7f06001021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:59.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.438+0000 7f0605e00640 1 -- 172.21.15.83:0/2649618302 shutdown_connections 2024-04-24T09:17:59.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.438+0000 7f0605e00640 1 -- 172.21.15.83:0/2649618302 wait complete. 2024-04-24T09:17:59.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.439+0000 7f0605e00640 1 Processor -- start 2024-04-24T09:17:59.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.439+0000 7f0605e00640 1 -- start start 2024-04-24T09:17:59.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.439+0000 7f0605e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06001044b0 0x7f06001a07c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:59.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.439+0000 7f0605e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0600109580 0x7f06001a0d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:59.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.439+0000 7f0605e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f060019a970 0x7f060019ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:59.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.439+0000 7f0605e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f06001144f0 con 0x7f06001044b0 2024-04-24T09:17:59.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.439+0000 7f0605e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0600114370 con 0x7f060019a970 2024-04-24T09:17:59.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.439+0000 7f0605e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0600114670 con 0x7f0600109580 2024-04-24T09:17:59.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.439+0000 7f05fea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06001044b0 0x7f06001a07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:59.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.439+0000 7f05fea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06001044b0 0x7f06001a07c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33110/0 (socket says 172.21.15.83:33110) 2024-04-24T09:17:59.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.439+0000 7f05fea00640 1 -- 172.21.15.83:0/3209460185 learned_addr learned my addr 172.21.15.83:0/3209460185 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:17:59.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.440+0000 7f05fe000640 1 --2- 172.21.15.83:0/3209460185 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0600109580 0x7f06001a0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:59.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.440+0000 7f05fea00640 1 -- 172.21.15.83:0/3209460185 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0600109580 msgr2=0x7f06001a0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:59.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.440+0000 7f05fea00640 1 --2- 172.21.15.83:0/3209460185 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0600109580 0x7f06001a0d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:59.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.440+0000 7f05fea00640 1 -- 172.21.15.83:0/3209460185 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f060019a970 msgr2=0x7f060019ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:59.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.440+0000 7f05fea00640 1 --2- 172.21.15.83:0/3209460185 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f060019a970 0x7f060019ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:59.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.440+0000 7f05fea00640 1 -- 172.21.15.83:0/3209460185 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f060019b490 con 0x7f06001044b0 2024-04-24T09:17:59.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.440+0000 7f05fea00640 1 --2- 172.21.15.83:0/3209460185 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06001044b0 0x7f06001a07c0 secure :-1 s=READY pgs=2884 cs=0 l=1 rev1=1 crypto rx=0x7f05f002ade0 tx=0x7f05f0032ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:59.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.440+0000 7f0604a00640 1 -- 172.21.15.83:0/3209460185 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f0003cf0 con 0x7f06001044b0 2024-04-24T09:17:59.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.440+0000 7f0604a00640 1 -- 172.21.15.83:0/3209460185 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f05f00044f0 con 0x7f06001044b0 2024-04-24T09:17:59.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.440+0000 7f0605e00640 1 -- 172.21.15.83:0/3209460185 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f060019b720 con 0x7f06001044b0 2024-04-24T09:17:59.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.440+0000 7f0604a00640 1 -- 172.21.15.83:0/3209460185 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f0030c40 con 0x7f06001044b0 2024-04-24T09:17:59.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.441+0000 7f0605e00640 1 -- 172.21.15.83:0/3209460185 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06001a7970 con 0x7f06001044b0 2024-04-24T09:17:59.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.444+0000 7f0608d84740 1 Processor -- start 2024-04-24T09:17:59.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.444+0000 7f0604a00640 1 -- 172.21.15.83:0/3209460185 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f05f0004040 con 0x7f06001044b0 2024-04-24T09:17:59.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.444+0000 7f0608d84740 1 -- start start 2024-04-24T09:17:59.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.444+0000 7f0608d84740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56130308e950 0x56130308ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:59.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.445+0000 7f0608d84740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56130308f270 0x561303099400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:59.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.445+0000 7f0608d84740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561303099940 0x56130309bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:59.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.445+0000 7f0608d84740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561302ff41d0 con 0x56130308f270 2024-04-24T09:17:59.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.445+0000 7f0608d84740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x561302f9a450 con 0x561303099940 2024-04-24T09:17:59.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.445+0000 7f0608d84740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x561302ff0190 con 0x56130308e950 2024-04-24T09:17:59.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.445+0000 7f05fea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56130308f270 0x561303099400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:59.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.445+0000 7f05fea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56130308f270 0x561303099400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33120/0 (socket says 172.21.15.83:33120) 2024-04-24T09:17:59.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.445+0000 7f05fea00640 1 -- 172.21.15.83:0/604998026 learned_addr learned my addr 172.21.15.83:0/604998026 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:17:59.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.445+0000 7f05fe000640 1 --2- 172.21.15.83:0/604998026 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561303099940 0x56130309bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:59.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.445+0000 7f05ff400640 1 --2- 172.21.15.83:0/604998026 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56130308e950 0x56130308ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:59.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.445+0000 7f05fea00640 1 -- 172.21.15.83:0/604998026 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56130308e950 msgr2=0x56130308ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:59.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.445+0000 7f05fea00640 1 --2- 172.21.15.83:0/604998026 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56130308e950 0x56130308ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:59.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.445+0000 7f05fea00640 1 -- 172.21.15.83:0/604998026 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561303099940 msgr2=0x56130309bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:59.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.445+0000 7f05fea00640 1 --2- 172.21.15.83:0/604998026 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561303099940 0x56130309bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:59.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.445+0000 7f05fea00640 1 -- 172.21.15.83:0/604998026 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561303057310 con 0x56130308f270 2024-04-24T09:17:59.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.445+0000 7f05ff400640 1 --2- 172.21.15.83:0/604998026 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56130308e950 0x56130308ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:17:59.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.446+0000 7f0604a00640 1 --2- 172.21.15.83:0/3209460185 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f05e40a3d20 0x7f05e40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:59.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.446+0000 7f05fea00640 1 --2- 172.21.15.83:0/604998026 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56130308f270 0x561303099400 secure :-1 s=READY pgs=2885 cs=0 l=1 rev1=1 crypto rx=0x7f05f00ef590 tx=0x7f05f00fa3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61c851e745d28e51 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:59.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.446+0000 7f0604a00640 1 -- 172.21.15.83:0/3209460185 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 9583+0+0 (secure 0 0 0) 0x7f05f00f21e0 con 0x7f06001044b0 2024-04-24T09:17:59.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.446+0000 7f05efe00640 1 -- 172.21.15.83:0/604998026 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f00ba110 con 0x56130308f270 2024-04-24T09:17:59.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.446+0000 7f05ff400640 1 --2- 172.21.15.83:0/3209460185 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f05e40a3d20 0x7f05e40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:59.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.446+0000 7f05efe00640 1 -- 172.21.15.83:0/604998026 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f05f0004330 con 0x56130308f270 2024-04-24T09:17:59.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.446+0000 7f05efe00640 1 -- 172.21.15.83:0/604998026 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f00bd7d0 con 0x56130308f270 2024-04-24T09:17:59.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.447+0000 7f0608d84740 1 -- 172.21.15.83:0/604998026 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56130308f270 msgr2=0x561303099400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:59.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.447+0000 7f0608d84740 1 --2- 172.21.15.83:0/604998026 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56130308f270 0x561303099400 secure :-1 s=READY pgs=2885 cs=0 l=1 rev1=1 crypto rx=0x7f05f00ef590 tx=0x7f05f00fa3b0 comp rx=0 tx=0).stop 2024-04-24T09:17:59.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.447+0000 7f05ff400640 1 --2- 172.21.15.83:0/3209460185 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f05e40a3d20 0x7f05e40a61e0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f05f80058c0 tx=0x7f05f8005e70 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:59.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.447+0000 7f0608d84740 1 -- 172.21.15.83:0/604998026 shutdown_connections 2024-04-24T09:17:59.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.447+0000 7f0608d84740 1 --2- 172.21.15.83:0/604998026 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561303099940 0x56130309bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:59.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.447+0000 7f0608d84740 1 --2- 172.21.15.83:0/604998026 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56130308f270 0x561303099400 unknown :-1 s=CLOSED pgs=2885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:59.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.447+0000 7f0608d84740 1 --2- 172.21.15.83:0/604998026 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56130308e950 0x56130308ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:59.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.447+0000 7f0608d84740 1 -- 172.21.15.83:0/604998026 >> 172.21.15.83:0/604998026 conn(0x56130308ac70 msgr2=0x56130308b050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:59.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.447+0000 7f0608d84740 1 -- 172.21.15.83:0/604998026 shutdown_connections 2024-04-24T09:17:59.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.447+0000 7f0608d84740 1 -- 172.21.15.83:0/604998026 wait complete. 2024-04-24T09:17:59.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.448+0000 7f0608d84740 1 -- start start 2024-04-24T09:17:59.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.448+0000 7f0608d84740 1 Processor -- start 2024-04-24T09:17:59.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.448+0000 7f0608d84740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56130308e950 0x5613030993d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:59.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.448+0000 7f0608d84740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56130308f270 0x5613030966c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:59.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.448+0000 7f0608d84740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561303099940 0x561303096db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:59.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.448+0000 7f0608d84740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561302ff41d0 con 0x56130308f270 2024-04-24T09:17:59.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.448+0000 7f0608d84740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x561302fad810 con 0x561303099940 2024-04-24T09:17:59.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.448+0000 7f0608d84740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56130309c4f0 con 0x56130308e950 2024-04-24T09:17:59.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.448+0000 7f0608d84740 10 client.0 _ll_register_callbacks cb 0x561302f7b140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:17:59.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.449+0000 7f05fe000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56130308f270 0x5613030966c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:59.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.449+0000 7f05fe000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56130308f270 0x5613030966c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33122/0 (socket says 172.21.15.83:33122) 2024-04-24T09:17:59.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.449+0000 7f05fe000640 1 -- 172.21.15.83:0/3976878692 learned_addr learned my addr 172.21.15.83:0/3976878692 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:17:59.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.449+0000 7f05fea00640 1 --2- 172.21.15.83:0/3976878692 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56130308e950 0x5613030993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:59.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.449+0000 7f05ff400640 1 --2- 172.21.15.83:0/3976878692 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561303099940 0x561303096db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:59.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.449+0000 7f05fe000640 1 -- 172.21.15.83:0/3976878692 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56130308e950 msgr2=0x5613030993d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:59.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.449+0000 7f05fe000640 1 --2- 172.21.15.83:0/3976878692 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56130308e950 0x5613030993d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:59.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.449+0000 7f05fe000640 1 -- 172.21.15.83:0/3976878692 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561303099940 msgr2=0x561303096db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:59.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.449+0000 7f05fe000640 1 --2- 172.21.15.83:0/3976878692 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561303099940 0x561303096db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:59.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.449+0000 7f05fe000640 1 -- 172.21.15.83:0/3976878692 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561302fbe3f0 con 0x56130308f270 2024-04-24T09:17:59.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.449+0000 7f05ff400640 1 --2- 172.21.15.83:0/3976878692 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561303099940 0x561303096db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:17:59.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.449+0000 7f05fe000640 1 --2- 172.21.15.83:0/3976878692 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56130308f270 0x5613030966c0 secure :-1 s=READY pgs=2886 cs=0 l=1 rev1=1 crypto rx=0x7f05e80103a0 tx=0x7f05e8010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d934da0ed4e024a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:59.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.449+0000 7f05ecc00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:17:59.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.450+0000 7f05ecc00640 1 -- 172.21.15.83:0/3976878692 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e8012920 con 0x56130308f270 2024-04-24T09:17:59.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.450+0000 7f05ecc00640 1 -- 172.21.15.83:0/3976878692 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(134..134 src has 1..134) ==== 9583+0+0 (secure 0 0 0) 0x7f05e8012ac0 con 0x56130308f270 2024-04-24T09:17:59.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.450+0000 7f0608d84740 1 -- 172.21.15.83:0/3976878692 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561303001bf0 con 0x56130308f270 2024-04-24T09:17:59.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.450+0000 7f05ecc00640 1 -- 172.21.15.83:0/3976878692 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7f05e8025050 con 0x56130308f270 2024-04-24T09:17:59.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.450+0000 7f0608d84740 10 client.24417 fetch_fsmap learned FSMap version 79 2024-04-24T09:17:59.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.450+0000 7f0608d84740 1 -- 172.21.15.83:0/3976878692 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=79,monmap=4+}) -- 0x561302d063f0 con 0x56130308f270 2024-04-24T09:17:59.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.450+0000 7f05ecc00640 1 -- 172.21.15.83:0/3976878692 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x561303001bf0 con 0x56130308f270 2024-04-24T09:17:59.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.450+0000 7f05ecc00640 1 -- 172.21.15.83:0/3976878692 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 79) ==== 8398+0+0 (secure 0 0 0) 0x7f05e800f040 con 0x56130308f270 2024-04-24T09:17:59.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.451+0000 7f0608d84740 10 client.24417 fetch_fsmap finished waiting for FSMap version 79 2024-04-24T09:17:59.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.451+0000 7f0608d84740 10 client.24417 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-24T09:17:59.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.451+0000 7f0608d84740 20 client.24417 populate_metadata read hostname 'smithi083' 2024-04-24T09:17:59.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.451+0000 7f0608d84740 1 --2- 172.21.15.83:0/3976878692 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x5613030a9710 0x5613030abb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:59.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.451+0000 7f0608d84740 4 client.24417 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:17:59.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.451+0000 7f0608d84740 1 -- 172.21.15.83:0/3976878692 --> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561302ffba60 con 0x5613030a9710 2024-04-24T09:17:59.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.451+0000 7f05fea00640 1 --2- 172.21.15.83:0/3976878692 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x5613030a9710 0x5613030abb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:59.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.452+0000 7f05fea00640 1 --2- 172.21.15.83:0/3976878692 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x5613030a9710 0x5613030abb70 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=61a35eca52bc70c9 server_cookie=e9eee10ddc3afa24 in_seq=0 out_seq=0 2024-04-24T09:17:59.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.452+0000 7f05ecc00640 10 client.24417 ms_handle_connect on v2:172.21.15.138:6836/900566754 2024-04-24T09:17:59.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.454+0000 7f05ecc00640 1 -- 172.21.15.83:0/3976878692 <== mds.1 v2:172.21.15.138:6836/900566754 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561302ffba60 con 0x5613030a9710 2024-04-24T09:17:59.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.454+0000 7f05ecc00640 10 client.24417 handle_command_reply: tid=0 2024-04-24T09:17:59.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.454+0000 7f0608d84740 1 client.24417 shutdown 2024-04-24T09:17:59.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.454+0000 7f0608d84740 1 -- 172.21.15.83:0/3976878692 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56130308f270 msgr2=0x5613030966c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:59.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.454+0000 7f0608d84740 1 --2- 172.21.15.83:0/3976878692 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56130308f270 0x5613030966c0 secure :-1 s=READY pgs=2886 cs=0 l=1 rev1=1 crypto rx=0x7f05e80103a0 tx=0x7f05e8010860 comp rx=0 tx=0).stop 2024-04-24T09:17:59.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.455+0000 7f0608d84740 1 -- 172.21.15.83:0/3976878692 shutdown_connections 2024-04-24T09:17:59.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.455+0000 7f0608d84740 1 --2- 172.21.15.83:0/3976878692 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561303099940 0x561303096db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:59.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.455+0000 7f0608d84740 1 --2- 172.21.15.83:0/3976878692 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56130308f270 0x5613030966c0 unknown :-1 s=CLOSED pgs=2886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:59.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.455+0000 7f0608d84740 1 --2- 172.21.15.83:0/3976878692 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x5613030a9710 0x5613030abb70 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:59.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.455+0000 7f0608d84740 1 --2- 172.21.15.83:0/3976878692 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56130308e950 0x5613030993d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:59.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.455+0000 7f05ecc00640 0 client.24417 ms_handle_reset on v2:172.21.15.138:6836/900566754 2024-04-24T09:17:59.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.455+0000 7f0608d84740 1 -- 172.21.15.83:0/3976878692 >> 172.21.15.83:0/3976878692 conn(0x56130308ac70 msgr2=0x561303090a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:59.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.455+0000 7f0608d84740 1 -- 172.21.15.83:0/3976878692 shutdown_connections 2024-04-24T09:17:59.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.455+0000 7f0608d84740 1 -- 172.21.15.83:0/3976878692 wait complete. 2024-04-24T09:17:59.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.455+0000 7f0608d84740 20 client.24417 trim_cache size 0 max 16384 2024-04-24T09:17:59.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.469+0000 7f0608d84740 1 Processor -- start 2024-04-24T09:17:59.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.470+0000 7f0608d84740 1 -- start start 2024-04-24T09:17:59.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.470+0000 7f0608d84740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56130308e950 0x56130308a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:59.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.470+0000 7f0608d84740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56130308f270 0x561303098f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:59.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.470+0000 7f0608d84740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561303099940 0x5613030b1480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:59.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.470+0000 7f0608d84740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f06001b18d0 con 0x56130308f270 2024-04-24T09:17:59.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.470+0000 7f0608d84740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f06001b34d0 con 0x56130308e950 2024-04-24T09:17:59.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.470+0000 7f0608d84740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f06001b1fa0 con 0x561303099940 2024-04-24T09:17:59.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.470+0000 7f05fe000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56130308f270 0x561303098f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:59.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.470+0000 7f05fe000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56130308f270 0x561303098f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33128/0 (socket says 172.21.15.83:33128) 2024-04-24T09:17:59.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.470+0000 7f05fe000640 1 -- 172.21.15.83:0/576926721 learned_addr learned my addr 172.21.15.83:0/576926721 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:17:59.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.470+0000 7f05ff400640 1 --2- 172.21.15.83:0/576926721 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561303099940 0x5613030b1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:59.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.471+0000 7f05fea00640 1 --2- 172.21.15.83:0/576926721 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56130308e950 0x56130308a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:59.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.471+0000 7f05fe000640 1 -- 172.21.15.83:0/576926721 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561303099940 msgr2=0x5613030b1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:59.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.471+0000 7f05fe000640 1 --2- 172.21.15.83:0/576926721 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561303099940 0x5613030b1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:59.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.471+0000 7f05fe000640 1 -- 172.21.15.83:0/576926721 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56130308e950 msgr2=0x56130308a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:59.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.471+0000 7f05fe000640 1 --2- 172.21.15.83:0/576926721 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56130308e950 0x56130308a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:59.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.471+0000 7f05fe000640 1 -- 172.21.15.83:0/576926721 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56130308f0a0 con 0x56130308f270 2024-04-24T09:17:59.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.471+0000 7f05fe000640 1 --2- 172.21.15.83:0/576926721 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56130308f270 0x561303098f00 secure :-1 s=READY pgs=2887 cs=0 l=1 rev1=1 crypto rx=0x7f05e800eca0 tx=0x7f05e800e410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f06990cf83b9ee3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:59.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.471+0000 7f05ef400640 1 -- 172.21.15.83:0/576926721 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e8022040 con 0x56130308f270 2024-04-24T09:17:59.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.471+0000 7f05ef400640 1 -- 172.21.15.83:0/576926721 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f05e80054f0 con 0x56130308f270 2024-04-24T09:17:59.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.471+0000 7f05ef400640 1 -- 172.21.15.83:0/576926721 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e8004620 con 0x56130308f270 2024-04-24T09:17:59.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.472+0000 7f0608d84740 1 -- 172.21.15.83:0/576926721 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56130308f270 msgr2=0x561303098f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:59.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.472+0000 7f0608d84740 1 --2- 172.21.15.83:0/576926721 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56130308f270 0x561303098f00 secure :-1 s=READY pgs=2887 cs=0 l=1 rev1=1 crypto rx=0x7f05e800eca0 tx=0x7f05e800e410 comp rx=0 tx=0).stop 2024-04-24T09:17:59.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.472+0000 7f0608d84740 1 -- 172.21.15.83:0/576926721 shutdown_connections 2024-04-24T09:17:59.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.472+0000 7f0608d84740 1 --2- 172.21.15.83:0/576926721 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561303099940 0x5613030b1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:59.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.472+0000 7f0608d84740 1 --2- 172.21.15.83:0/576926721 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56130308f270 0x561303098f00 unknown :-1 s=CLOSED pgs=2887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:59.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.472+0000 7f0608d84740 1 --2- 172.21.15.83:0/576926721 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56130308e950 0x56130308a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:59.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.472+0000 7f0608d84740 1 -- 172.21.15.83:0/576926721 >> 172.21.15.83:0/576926721 conn(0x56130308ac70 msgr2=0x56130308b050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:59.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.472+0000 7f0608d84740 1 -- 172.21.15.83:0/576926721 shutdown_connections 2024-04-24T09:17:59.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.472+0000 7f0608d84740 1 -- 172.21.15.83:0/576926721 wait complete. 2024-04-24T09:17:59.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.472+0000 7f0608d84740 1 -- start start 2024-04-24T09:17:59.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.473+0000 7f0608d84740 1 Processor -- start 2024-04-24T09:17:59.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.473+0000 7f0608d84740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56130308e950 0x56130309a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:59.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.473+0000 7f0608d84740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56130308f270 0x56130309ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:59.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.473+0000 7f0608d84740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561303099940 0x56130309b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:59.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.473+0000 7f0608d84740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f06001b2dd0 con 0x561303099940 2024-04-24T09:17:59.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.473+0000 7f0608d84740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f06001b26a0 con 0x56130308f270 2024-04-24T09:17:59.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.473+0000 7f0608d84740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f06001af8d0 con 0x56130308e950 2024-04-24T09:17:59.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.473+0000 7f0608d84740 10 client.0 _ll_register_callbacks cb 0x7f05e8005f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:17:59.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.473+0000 7f05ff400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561303099940 0x56130309b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:59.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.473+0000 7f05ff400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561303099940 0x56130309b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33140/0 (socket says 172.21.15.83:33140) 2024-04-24T09:17:59.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.473+0000 7f05ff400640 1 -- 172.21.15.83:0/2321987861 learned_addr learned my addr 172.21.15.83:0/2321987861 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:17:59.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.474+0000 7f05fea00640 1 --2- 172.21.15.83:0/2321987861 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56130308e950 0x56130309a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:59.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.474+0000 7f05fe000640 1 --2- 172.21.15.83:0/2321987861 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56130308f270 0x56130309ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:59.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.474+0000 7f05ff400640 1 -- 172.21.15.83:0/2321987861 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56130308e950 msgr2=0x56130309a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:59.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.474+0000 7f05ff400640 1 --2- 172.21.15.83:0/2321987861 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56130308e950 0x56130309a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:59.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.474+0000 7f05ff400640 1 -- 172.21.15.83:0/2321987861 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56130308f270 msgr2=0x56130309ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:59.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.474+0000 7f05ff400640 1 --2- 172.21.15.83:0/2321987861 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56130308f270 0x56130309ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:59.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.474+0000 7f05ff400640 1 -- 172.21.15.83:0/2321987861 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561302dd07d0 con 0x561303099940 2024-04-24T09:17:59.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.474+0000 7f05ff400640 1 --2- 172.21.15.83:0/2321987861 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561303099940 0x56130309b2d0 secure :-1 s=READY pgs=2888 cs=0 l=1 rev1=1 crypto rx=0x7f05f800e8e0 tx=0x7f05f8005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad0bb3d28269710a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:59.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.474+0000 7f05ecc00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:17:59.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.474+0000 7f05ecc00640 1 -- 172.21.15.83:0/2321987861 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f8005220 con 0x561303099940 2024-04-24T09:17:59.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.475+0000 7f0608d84740 1 -- 172.21.15.83:0/2321987861 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05e80054f0 con 0x561303099940 2024-04-24T09:17:59.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.475+0000 7f05ecc00640 1 -- 172.21.15.83:0/2321987861 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(134..134 src has 1..134) ==== 9583+0+0 (secure 0 0 0) 0x7f05f80053c0 con 0x561303099940 2024-04-24T09:17:59.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.475+0000 7f05ecc00640 1 -- 172.21.15.83:0/2321987861 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7f05f802f050 con 0x561303099940 2024-04-24T09:17:59.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.475+0000 7f0608d84740 10 client.24423 fetch_fsmap learned FSMap version 79 2024-04-24T09:17:59.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.475+0000 7f0608d84740 1 -- 172.21.15.83:0/2321987861 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=79,monmap=4+}) -- 0x561303118850 con 0x561303099940 2024-04-24T09:17:59.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.475+0000 7f05ecc00640 1 -- 172.21.15.83:0/2321987861 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f05e80054f0 con 0x561303099940 2024-04-24T09:17:59.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.475+0000 7f05ecc00640 1 -- 172.21.15.83:0/2321987861 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 79) ==== 8398+0+0 (secure 0 0 0) 0x7f05f800f040 con 0x561303099940 2024-04-24T09:17:59.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.475+0000 7f0608d84740 10 client.24423 fetch_fsmap finished waiting for FSMap version 79 2024-04-24T09:17:59.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.475+0000 7f0608d84740 10 client.24423 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-24T09:17:59.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.475+0000 7f0608d84740 20 client.24423 populate_metadata read hostname 'smithi083' 2024-04-24T09:17:59.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.475+0000 7f0608d84740 1 --2- 172.21.15.83:0/2321987861 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x561303118fa0 0x56130311b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:59.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.475+0000 7f0608d84740 4 client.24423 mds_command: new command op to 34493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:17:59.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.475+0000 7f0608d84740 1 -- 172.21.15.83:0/2321987861 --> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561302f774d0 con 0x561303118fa0 2024-04-24T09:17:59.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.476+0000 7f05fea00640 1 --2- 172.21.15.83:0/2321987861 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x561303118fa0 0x56130311b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:59.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.477+0000 7f05fea00640 1 --2- 172.21.15.83:0/2321987861 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x561303118fa0 0x56130311b3c0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=522e3dcd6affa28 server_cookie=70693bbf57ffff68 in_seq=0 out_seq=0 2024-04-24T09:17:59.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.477+0000 7f05ecc00640 10 client.24423 ms_handle_connect on v2:172.21.15.138:6836/900566754 2024-04-24T09:17:59.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.477+0000 7f05ecc00640 1 -- 172.21.15.83:0/2321987861 <== mds.1 v2:172.21.15.138:6836/900566754 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561302f774d0 con 0x561303118fa0 2024-04-24T09:17:59.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.477+0000 7f05ecc00640 10 client.24423 handle_command_reply: tid=0 2024-04-24T09:17:59.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.477+0000 7f0608d84740 1 client.24423 shutdown 2024-04-24T09:17:59.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.478+0000 7f0608d84740 1 -- 172.21.15.83:0/2321987861 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561303099940 msgr2=0x56130309b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:59.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.478+0000 7f0608d84740 1 --2- 172.21.15.83:0/2321987861 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561303099940 0x56130309b2d0 secure :-1 s=READY pgs=2888 cs=0 l=1 rev1=1 crypto rx=0x7f05f800e8e0 tx=0x7f05f8005850 comp rx=0 tx=0).stop 2024-04-24T09:17:59.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.478+0000 7f0608d84740 1 -- 172.21.15.83:0/2321987861 shutdown_connections 2024-04-24T09:17:59.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.478+0000 7f0608d84740 1 --2- 172.21.15.83:0/2321987861 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561303099940 0x56130309b2d0 unknown :-1 s=CLOSED pgs=2888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:59.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.478+0000 7f0608d84740 1 --2- 172.21.15.83:0/2321987861 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56130308f270 0x56130309ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:59.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.478+0000 7f0608d84740 1 --2- 172.21.15.83:0/2321987861 >> [v2:172.21.15.138:6836/900566754,v1:172.21.15.138:6837/900566754] conn(0x561303118fa0 0x56130311b3c0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:59.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.478+0000 7f0608d84740 1 --2- 172.21.15.83:0/2321987861 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56130308e950 0x56130309a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:59.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.478+0000 7f05ecc00640 0 client.24423 ms_handle_reset on v2:172.21.15.138:6836/900566754 2024-04-24T09:17:59.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.478+0000 7f0608d84740 1 -- 172.21.15.83:0/2321987861 >> 172.21.15.83:0/2321987861 conn(0x56130308ac70 msgr2=0x5613030901b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:59.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.478+0000 7f0608d84740 1 -- 172.21.15.83:0/2321987861 shutdown_connections 2024-04-24T09:17:59.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.478+0000 7f0608d84740 1 -- 172.21.15.83:0/2321987861 wait complete. 2024-04-24T09:17:59.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.478+0000 7f0608d84740 20 client.24423 trim_cache size 0 max 16384 2024-04-24T09:17:59.480 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:17:59.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.479+0000 7f0605e00640 1 -- 172.21.15.83:0/3209460185 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f05e40a3d20 msgr2=0x7f05e40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:59.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.479+0000 7f0605e00640 1 --2- 172.21.15.83:0/3209460185 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f05e40a3d20 0x7f05e40a61e0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f05f80058c0 tx=0x7f05f8005e70 comp rx=0 tx=0).stop 2024-04-24T09:17:59.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.479+0000 7f0605e00640 1 -- 172.21.15.83:0/3209460185 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06001044b0 msgr2=0x7f06001a07c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:59.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.479+0000 7f0605e00640 1 --2- 172.21.15.83:0/3209460185 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06001044b0 0x7f06001a07c0 secure :-1 s=READY pgs=2884 cs=0 l=1 rev1=1 crypto rx=0x7f05f002ade0 tx=0x7f05f0032ff0 comp rx=0 tx=0).stop 2024-04-24T09:17:59.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.479+0000 7f0605e00640 1 -- 172.21.15.83:0/3209460185 shutdown_connections 2024-04-24T09:17:59.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.479+0000 7f0605e00640 1 --2- 172.21.15.83:0/3209460185 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f05e40a3d20 0x7f05e40a61e0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:59.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.479+0000 7f0605e00640 1 --2- 172.21.15.83:0/3209460185 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f060019a970 0x7f060019ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:59.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.479+0000 7f0605e00640 1 --2- 172.21.15.83:0/3209460185 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0600109580 0x7f06001a0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:59.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.479+0000 7f0605e00640 1 --2- 172.21.15.83:0/3209460185 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06001044b0 0x7f06001a07c0 unknown :-1 s=CLOSED pgs=2884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:59.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.479+0000 7f0605e00640 1 -- 172.21.15.83:0/3209460185 >> 172.21.15.83:0/3209460185 conn(0x7f06000ffd90 msgr2=0x7f0600101230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:59.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.479+0000 7f0605e00640 1 -- 172.21.15.83:0/3209460185 shutdown_connections 2024-04-24T09:17:59.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.479+0000 7f0605e00640 1 -- 172.21.15.83:0/3209460185 wait complete. 2024-04-24T09:17:59.493 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:17:59 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/1046491100' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:17:59.496 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T09:17:59.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.698+0000 7efee8a00640 1 -- 172.21.15.83:0/1399852842 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efee4102ad0 msgr2=0x7efee4106b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:59.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.698+0000 7efee8a00640 1 --2- 172.21.15.83:0/1399852842 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efee4102ad0 0x7efee4106b10 secure :-1 s=READY pgs=2889 cs=0 l=1 rev1=1 crypto rx=0x7efecc009900 tx=0x7efecc02a3f0 comp rx=0 tx=0).stop 2024-04-24T09:17:59.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.698+0000 7efee8a00640 1 -- 172.21.15.83:0/1399852842 shutdown_connections 2024-04-24T09:17:59.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.698+0000 7efee8a00640 1 --2- 172.21.15.83:0/1399852842 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efee41071f0 0x7efee410edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:59.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.698+0000 7efee8a00640 1 --2- 172.21.15.83:0/1399852842 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efee4102ad0 0x7efee4106b10 unknown :-1 s=CLOSED pgs=2889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:59.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.698+0000 7efee8a00640 1 --2- 172.21.15.83:0/1399852842 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efee4102120 0x7efee4102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:59.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.698+0000 7efee8a00640 1 -- 172.21.15.83:0/1399852842 >> 172.21.15.83:0/1399852842 conn(0x7efee40fd9a0 msgr2=0x7efee40ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:59.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.698+0000 7efee8a00640 1 -- 172.21.15.83:0/1399852842 shutdown_connections 2024-04-24T09:17:59.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.699+0000 7efee8a00640 1 -- 172.21.15.83:0/1399852842 wait complete. 2024-04-24T09:17:59.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.699+0000 7efee8a00640 1 Processor -- start 2024-04-24T09:17:59.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.700+0000 7efee8a00640 1 -- start start 2024-04-24T09:17:59.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.700+0000 7efee8a00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efee4102120 0x7efee4198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:59.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.700+0000 7efee8a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efee41071f0 0x7efee4198bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:59.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.700+0000 7efee8a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efee419cf40 0x7efee419d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:59.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.700+0000 7efee8a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7efee4112160 con 0x7efee419cf40 2024-04-24T09:17:59.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.700+0000 7efee8a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efee4111fe0 con 0x7efee41071f0 2024-04-24T09:17:59.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.700+0000 7efee8a00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7efee41122e0 con 0x7efee4102120 2024-04-24T09:17:59.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.700+0000 7efee3200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efee419cf40 0x7efee419d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:59.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.700+0000 7efee3200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efee419cf40 0x7efee419d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33160/0 (socket says 172.21.15.83:33160) 2024-04-24T09:17:59.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.700+0000 7efee3200640 1 -- 172.21.15.83:0/319953000 learned_addr learned my addr 172.21.15.83:0/319953000 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:17:59.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.701+0000 7efee2800640 1 --2- 172.21.15.83:0/319953000 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efee4102120 0x7efee4198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:59.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.701+0000 7efed9e00640 1 --2- 172.21.15.83:0/319953000 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efee41071f0 0x7efee4198bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:59.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.701+0000 7efee3200640 1 -- 172.21.15.83:0/319953000 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efee4102120 msgr2=0x7efee4198680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:59.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.701+0000 7efee3200640 1 --2- 172.21.15.83:0/319953000 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efee4102120 0x7efee4198680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:59.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.701+0000 7efee3200640 1 -- 172.21.15.83:0/319953000 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efee41071f0 msgr2=0x7efee4198bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:59.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.701+0000 7efee3200640 1 --2- 172.21.15.83:0/319953000 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efee41071f0 0x7efee4198bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:59.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.701+0000 7efee3200640 1 -- 172.21.15.83:0/319953000 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efee419da30 con 0x7efee419cf40 2024-04-24T09:17:59.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.701+0000 7efee3200640 1 --2- 172.21.15.83:0/319953000 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efee419cf40 0x7efee419d3f0 secure :-1 s=READY pgs=2890 cs=0 l=1 rev1=1 crypto rx=0x7efedc00cad0 tx=0x7efedc00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:59.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.701+0000 7efedbe00640 1 -- 172.21.15.83:0/319953000 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efedc00ae20 con 0x7efee419cf40 2024-04-24T09:17:59.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.701+0000 7efedbe00640 1 -- 172.21.15.83:0/319953000 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efedc005020 con 0x7efee419cf40 2024-04-24T09:17:59.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.701+0000 7efedbe00640 1 -- 172.21.15.83:0/319953000 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efedc00f6b0 con 0x7efee419cf40 2024-04-24T09:17:59.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.701+0000 7efee8a00640 1 -- 172.21.15.83:0/319953000 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efee419dd20 con 0x7efee419cf40 2024-04-24T09:17:59.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.702+0000 7efee8a00640 1 -- 172.21.15.83:0/319953000 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efee41a54c0 con 0x7efee419cf40 2024-04-24T09:17:59.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.705+0000 7efeec22b740 1 Processor -- start 2024-04-24T09:17:59.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.705+0000 7efedbe00640 1 -- 172.21.15.83:0/319953000 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7efedc0051d0 con 0x7efee419cf40 2024-04-24T09:17:59.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.705+0000 7efeec22b740 1 -- start start 2024-04-24T09:17:59.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:17:59 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/1046491100' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:17:59.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.705+0000 7efeec22b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e23f2cb40 0x559e23f2cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:59.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.705+0000 7efeec22b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e23f2d460 0x559e23f375f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:59.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.706+0000 7efeec22b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e23f37b30 0x559e23f39f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:59.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.706+0000 7efeec22b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559e23e921d0 con 0x559e23f37b30 2024-04-24T09:17:59.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.706+0000 7efeec22b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559e23e38450 con 0x559e23f2cb40 2024-04-24T09:17:59.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.706+0000 7efeec22b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559e23e8e190 con 0x559e23f2d460 2024-04-24T09:17:59.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.706+0000 7efed9e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e23f37b30 0x559e23f39f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:59.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.706+0000 7efed9e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e23f37b30 0x559e23f39f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33174/0 (socket says 172.21.15.83:33174) 2024-04-24T09:17:59.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.706+0000 7efed9e00640 1 -- 172.21.15.83:0/981975573 learned_addr learned my addr 172.21.15.83:0/981975573 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:17:59.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.706+0000 7efee2800640 1 --2- 172.21.15.83:0/981975573 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e23f2d460 0x559e23f375f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:59.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.706+0000 7efed9e00640 1 --2- 172.21.15.83:0/981975573 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e23f2cb40 0x559e23f2cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:59.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.706+0000 7efed9e00640 1 -- 172.21.15.83:0/981975573 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e23f2d460 msgr2=0x559e23f375f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:59.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.706+0000 7efed9e00640 1 --2- 172.21.15.83:0/981975573 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e23f2d460 0x559e23f375f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:59.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.706+0000 7efed9e00640 1 -- 172.21.15.83:0/981975573 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e23f2cb40 msgr2=0x559e23f2cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:59.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.706+0000 7efed9e00640 1 --2- 172.21.15.83:0/981975573 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e23f2cb40 0x559e23f2cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:59.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.706+0000 7efed9e00640 1 -- 172.21.15.83:0/981975573 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e23ef5310 con 0x559e23f37b30 2024-04-24T09:17:59.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.707+0000 7efedbe00640 1 --2- 172.21.15.83:0/319953000 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7efec80a3da0 0x7efec80a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:59.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.707+0000 7efed9e00640 1 --2- 172.21.15.83:0/981975573 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e23f37b30 0x559e23f39f20 secure :-1 s=READY pgs=2891 cs=0 l=1 rev1=1 crypto rx=0x7efecc0044f0 tx=0x7efecc031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f4d9673406d06d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:59.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.707+0000 7efedbe00640 1 -- 172.21.15.83:0/319953000 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 9583+0+0 (secure 0 0 0) 0x7efedc0d0ce0 con 0x7efee419cf40 2024-04-24T09:17:59.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.707+0000 7efee2800640 1 --2- 172.21.15.83:0/319953000 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7efec80a3da0 0x7efec80a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:59.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.707+0000 7efed3400640 1 -- 172.21.15.83:0/981975573 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efecc045070 con 0x559e23f37b30 2024-04-24T09:17:59.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.707+0000 7efed3400640 1 -- 172.21.15.83:0/981975573 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efecc0046c0 con 0x559e23f37b30 2024-04-24T09:17:59.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.707+0000 7efed3400640 1 -- 172.21.15.83:0/981975573 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efecc035040 con 0x559e23f37b30 2024-04-24T09:17:59.709 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:17:59 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/1046491100' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:17:59.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.708+0000 7efeec22b740 1 -- 172.21.15.83:0/981975573 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e23f37b30 msgr2=0x559e23f39f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:59.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.708+0000 7efee2800640 1 --2- 172.21.15.83:0/319953000 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7efec80a3da0 0x7efec80a6260 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7efed40058c0 tx=0x7efed4005850 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:59.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.708+0000 7efeec22b740 1 --2- 172.21.15.83:0/981975573 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e23f37b30 0x559e23f39f20 secure :-1 s=READY pgs=2891 cs=0 l=1 rev1=1 crypto rx=0x7efecc0044f0 tx=0x7efecc031400 comp rx=0 tx=0).stop 2024-04-24T09:17:59.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.708+0000 7efeec22b740 1 -- 172.21.15.83:0/981975573 shutdown_connections 2024-04-24T09:17:59.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.708+0000 7efeec22b740 1 --2- 172.21.15.83:0/981975573 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e23f37b30 0x559e23f39f20 unknown :-1 s=CLOSED pgs=2891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:59.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.708+0000 7efeec22b740 1 --2- 172.21.15.83:0/981975573 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e23f2d460 0x559e23f375f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:59.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.708+0000 7efeec22b740 1 --2- 172.21.15.83:0/981975573 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e23f2cb40 0x559e23f2cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:59.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.708+0000 7efeec22b740 1 -- 172.21.15.83:0/981975573 >> 172.21.15.83:0/981975573 conn(0x559e23f28e60 msgr2=0x559e23f29240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:59.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.708+0000 7efeec22b740 1 -- 172.21.15.83:0/981975573 shutdown_connections 2024-04-24T09:17:59.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.708+0000 7efeec22b740 1 -- 172.21.15.83:0/981975573 wait complete. 2024-04-24T09:17:59.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.709+0000 7efeec22b740 1 -- start start 2024-04-24T09:17:59.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.709+0000 7efeec22b740 1 Processor -- start 2024-04-24T09:17:59.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.709+0000 7efeec22b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e23f2cb40 0x559e23f37590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:59.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.709+0000 7efeec22b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e23f2d460 0x559e23f34880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:59.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.709+0000 7efeec22b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e23f37b30 0x559e23f34fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:59.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.709+0000 7efeec22b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559e23f3a960 con 0x559e23f2cb40 2024-04-24T09:17:59.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.709+0000 7efeec22b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559e23e4b810 con 0x559e23f37b30 2024-04-24T09:17:59.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.709+0000 7efeec22b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559e23f3aae0 con 0x559e23f2d460 2024-04-24T09:17:59.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.709+0000 7efeec22b740 10 client.0 _ll_register_callbacks cb 0x559e23e19140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:17:59.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.710+0000 7efee2800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e23f2cb40 0x559e23f37590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:59.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.710+0000 7efee2800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e23f2cb40 0x559e23f37590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33182/0 (socket says 172.21.15.83:33182) 2024-04-24T09:17:59.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.710+0000 7efee2800640 1 -- 172.21.15.83:0/3534420640 learned_addr learned my addr 172.21.15.83:0/3534420640 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:17:59.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.710+0000 7efed9e00640 1 --2- 172.21.15.83:0/3534420640 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e23f2d460 0x559e23f34880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:59.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.710+0000 7efee3200640 1 --2- 172.21.15.83:0/3534420640 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e23f37b30 0x559e23f34fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:59.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.710+0000 7efee2800640 1 -- 172.21.15.83:0/3534420640 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e23f2d460 msgr2=0x559e23f34880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:59.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.710+0000 7efee2800640 1 --2- 172.21.15.83:0/3534420640 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e23f2d460 0x559e23f34880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:59.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.710+0000 7efee2800640 1 -- 172.21.15.83:0/3534420640 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e23f37b30 msgr2=0x559e23f34fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:59.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.710+0000 7efee2800640 1 --2- 172.21.15.83:0/3534420640 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e23f37b30 0x559e23f34fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:59.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.710+0000 7efee2800640 1 -- 172.21.15.83:0/3534420640 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e23e5c3f0 con 0x559e23f2cb40 2024-04-24T09:17:59.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.710+0000 7efee2800640 1 --2- 172.21.15.83:0/3534420640 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e23f2cb40 0x559e23f37590 secure :-1 s=READY pgs=2892 cs=0 l=1 rev1=1 crypto rx=0x7efed4011e10 tx=0x7efed40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=778caf858c7dcff6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:59.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.710+0000 7efec7e00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:17:59.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.711+0000 7efec7e00640 1 -- 172.21.15.83:0/3534420640 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed4004030 con 0x559e23f2cb40 2024-04-24T09:17:59.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.711+0000 7efec7e00640 1 -- 172.21.15.83:0/3534420640 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(134..134 src has 1..134) ==== 9583+0+0 (secure 0 0 0) 0x7efed40041d0 con 0x559e23f2cb40 2024-04-24T09:17:59.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.711+0000 7efeec22b740 1 -- 172.21.15.83:0/3534420640 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efecc0046c0 con 0x559e23f2cb40 2024-04-24T09:17:59.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.711+0000 7efec7e00640 1 -- 172.21.15.83:0/3534420640 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7efed402e050 con 0x559e23f2cb40 2024-04-24T09:17:59.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.711+0000 7efeec22b740 10 client.24435 fetch_fsmap learned FSMap version 79 2024-04-24T09:17:59.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.711+0000 7efeec22b740 1 -- 172.21.15.83:0/3534420640 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=79,monmap=4+}) -- 0x559e23ba43f0 con 0x559e23f2cb40 2024-04-24T09:17:59.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.711+0000 7efec7e00640 1 -- 172.21.15.83:0/3534420640 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efecc0046c0 con 0x559e23f2cb40 2024-04-24T09:17:59.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.711+0000 7efec7e00640 1 -- 172.21.15.83:0/3534420640 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 79) ==== 8398+0+0 (secure 0 0 0) 0x7efed401e420 con 0x559e23f2cb40 2024-04-24T09:17:59.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.712+0000 7efeec22b740 10 client.24435 fetch_fsmap finished waiting for FSMap version 79 2024-04-24T09:17:59.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.712+0000 7efeec22b740 10 client.24435 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:17:59.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.712+0000 7efeec22b740 20 client.24435 populate_metadata read hostname 'smithi083' 2024-04-24T09:17:59.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.712+0000 7efeec22b740 1 --2- 172.21.15.83:0/3534420640 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x559e23f47e40 0x559e23f4a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:59.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.712+0000 7efeec22b740 4 client.24435 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T09:17:59.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.712+0000 7efeec22b740 1 -- 172.21.15.83:0/3534420640 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559e23e9fbf0 con 0x559e23f47e40 2024-04-24T09:17:59.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.712+0000 7efed9e00640 1 --2- 172.21.15.83:0/3534420640 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x559e23f47e40 0x559e23f4a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:59.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.713+0000 7efed9e00640 1 --2- 172.21.15.83:0/3534420640 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x559e23f47e40 0x559e23f4a2c0 crc :-1 s=READY pgs=637 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=514c47c208c65e11 server_cookie=d47feffee4ce472c in_seq=0 out_seq=0 2024-04-24T09:17:59.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.713+0000 7efec7e00640 10 client.24435 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:17:59.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.715+0000 7efec7e00640 1 -- 172.21.15.83:0/3534420640 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559e23e9fbf0 con 0x559e23f47e40 2024-04-24T09:17:59.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.715+0000 7efec7e00640 10 client.24435 handle_command_reply: tid=0 2024-04-24T09:17:59.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.715+0000 7efeec22b740 1 client.24435 shutdown 2024-04-24T09:17:59.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.715+0000 7efeec22b740 1 -- 172.21.15.83:0/3534420640 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e23f2cb40 msgr2=0x559e23f37590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:59.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.715+0000 7efeec22b740 1 --2- 172.21.15.83:0/3534420640 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e23f2cb40 0x559e23f37590 secure :-1 s=READY pgs=2892 cs=0 l=1 rev1=1 crypto rx=0x7efed4011e10 tx=0x7efed40046c0 comp rx=0 tx=0).stop 2024-04-24T09:17:59.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.715+0000 7efeec22b740 1 -- 172.21.15.83:0/3534420640 shutdown_connections 2024-04-24T09:17:59.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.715+0000 7efeec22b740 1 --2- 172.21.15.83:0/3534420640 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x559e23f47e40 0x559e23f4a2c0 crc :-1 s=READY pgs=637 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:59.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.715+0000 7efeec22b740 1 --2- 172.21.15.83:0/3534420640 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e23f37b30 0x559e23f34fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:59.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.715+0000 7efeec22b740 1 --2- 172.21.15.83:0/3534420640 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e23f2d460 0x559e23f34880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:59.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.715+0000 7efeec22b740 1 --2- 172.21.15.83:0/3534420640 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e23f2cb40 0x559e23f37590 unknown :-1 s=CLOSED pgs=2892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:59.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.715+0000 7efeec22b740 1 -- 172.21.15.83:0/3534420640 >> 172.21.15.83:0/3534420640 conn(0x559e23f28e60 msgr2=0x559e23f2ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:59.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.715+0000 7efec7e00640 0 client.24435 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:17:59.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.716+0000 7efeec22b740 1 -- 172.21.15.83:0/3534420640 shutdown_connections 2024-04-24T09:17:59.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.716+0000 7efeec22b740 1 -- 172.21.15.83:0/3534420640 wait complete. 2024-04-24T09:17:59.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.716+0000 7efeec22b740 20 client.24435 trim_cache size 0 max 16384 2024-04-24T09:17:59.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.730+0000 7efeec22b740 1 Processor -- start 2024-04-24T09:17:59.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.730+0000 7efeec22b740 1 -- start start 2024-04-24T09:17:59.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.730+0000 7efeec22b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e23f2cb40 0x559e23f43850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:59.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.730+0000 7efeec22b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e23f2d460 0x559e23f2f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:59.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.731+0000 7efeec22b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e23f37b30 0x559e23f28190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:59.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.731+0000 7efeec22b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7efee41af330 con 0x559e23f37b30 2024-04-24T09:17:59.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.731+0000 7efeec22b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efee41b0f30 con 0x559e23f2cb40 2024-04-24T09:17:59.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.731+0000 7efeec22b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7efee41afa00 con 0x559e23f2d460 2024-04-24T09:17:59.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.731+0000 7efee3200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e23f37b30 0x559e23f28190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:59.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.731+0000 7efee3200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e23f37b30 0x559e23f28190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33198/0 (socket says 172.21.15.83:33198) 2024-04-24T09:17:59.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.731+0000 7efee3200640 1 -- 172.21.15.83:0/1787016132 learned_addr learned my addr 172.21.15.83:0/1787016132 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:17:59.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.731+0000 7efee3200640 1 -- 172.21.15.83:0/1787016132 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e23f2d460 msgr2=0x559e23f2f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:59.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.731+0000 7efee3200640 1 --2- 172.21.15.83:0/1787016132 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e23f2d460 0x559e23f2f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:59.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.731+0000 7efee2800640 1 --2- 172.21.15.83:0/1787016132 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e23f2cb40 0x559e23f43850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:59.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.731+0000 7efee3200640 1 -- 172.21.15.83:0/1787016132 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e23f2cb40 msgr2=0x559e23f43850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:59.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.731+0000 7efee3200640 1 --2- 172.21.15.83:0/1787016132 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e23f2cb40 0x559e23f43850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:59.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.731+0000 7efee3200640 1 -- 172.21.15.83:0/1787016132 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e23f2d290 con 0x559e23f37b30 2024-04-24T09:17:59.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.732+0000 7efee3200640 1 --2- 172.21.15.83:0/1787016132 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e23f37b30 0x559e23f28190 secure :-1 s=READY pgs=2893 cs=0 l=1 rev1=1 crypto rx=0x7efedc005cc0 tx=0x7efedc09c280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d24d0b41b70b6f49 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:59.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.732+0000 7efed2a00640 1 -- 172.21.15.83:0/1787016132 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efedc01d040 con 0x559e23f37b30 2024-04-24T09:17:59.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.732+0000 7efed2a00640 1 -- 172.21.15.83:0/1787016132 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efedc0a5470 con 0x559e23f37b30 2024-04-24T09:17:59.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.732+0000 7efed2a00640 1 -- 172.21.15.83:0/1787016132 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efedc098e70 con 0x559e23f37b30 2024-04-24T09:17:59.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.732+0000 7efeec22b740 1 -- 172.21.15.83:0/1787016132 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e23f37b30 msgr2=0x559e23f28190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:59.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.732+0000 7efeec22b740 1 --2- 172.21.15.83:0/1787016132 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e23f37b30 0x559e23f28190 secure :-1 s=READY pgs=2893 cs=0 l=1 rev1=1 crypto rx=0x7efedc005cc0 tx=0x7efedc09c280 comp rx=0 tx=0).stop 2024-04-24T09:17:59.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.732+0000 7efeec22b740 1 -- 172.21.15.83:0/1787016132 shutdown_connections 2024-04-24T09:17:59.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.732+0000 7efeec22b740 1 --2- 172.21.15.83:0/1787016132 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e23f37b30 0x559e23f28190 unknown :-1 s=CLOSED pgs=2893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:59.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.732+0000 7efeec22b740 1 --2- 172.21.15.83:0/1787016132 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e23f2d460 0x559e23f2f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:59.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.732+0000 7efeec22b740 1 --2- 172.21.15.83:0/1787016132 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e23f2cb40 0x559e23f43850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:59.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.732+0000 7efeec22b740 1 -- 172.21.15.83:0/1787016132 >> 172.21.15.83:0/1787016132 conn(0x559e23f28e60 msgr2=0x559e23f2eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:59.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.733+0000 7efeec22b740 1 -- 172.21.15.83:0/1787016132 shutdown_connections 2024-04-24T09:17:59.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.733+0000 7efeec22b740 1 -- 172.21.15.83:0/1787016132 wait complete. 2024-04-24T09:17:59.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.733+0000 7efeec22b740 1 -- start start 2024-04-24T09:17:59.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.733+0000 7efeec22b740 1 Processor -- start 2024-04-24T09:17:59.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.734+0000 7efeec22b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e23f2cb40 0x559e23f38c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:59.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.734+0000 7efeec22b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e23f2d460 0x559e23f39140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:59.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.734+0000 7efeec22b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e23f37b30 0x559e23f3e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:59.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.734+0000 7efeec22b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7efee41b0830 con 0x559e23f37b30 2024-04-24T09:17:59.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.734+0000 7efeec22b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efee41b0100 con 0x559e23f2d460 2024-04-24T09:17:59.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.734+0000 7efeec22b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7efee41ad330 con 0x559e23f2cb40 2024-04-24T09:17:59.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.734+0000 7efeec22b740 10 client.0 _ll_register_callbacks cb 0x7efed400c960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T09:17:59.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.734+0000 7efee3200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e23f37b30 0x559e23f3e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:59.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.734+0000 7efee3200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e23f37b30 0x559e23f3e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33202/0 (socket says 172.21.15.83:33202) 2024-04-24T09:17:59.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.734+0000 7efee3200640 1 -- 172.21.15.83:0/1977330060 learned_addr learned my addr 172.21.15.83:0/1977330060 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:17:59.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.734+0000 7efed9e00640 1 --2- 172.21.15.83:0/1977330060 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e23f2d460 0x559e23f39140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:59.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.734+0000 7efee2800640 1 --2- 172.21.15.83:0/1977330060 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e23f2cb40 0x559e23f38c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:59.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.734+0000 7efee3200640 1 -- 172.21.15.83:0/1977330060 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e23f2cb40 msgr2=0x559e23f38c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:59.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.734+0000 7efee3200640 1 --2- 172.21.15.83:0/1977330060 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e23f2cb40 0x559e23f38c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:59.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.734+0000 7efee3200640 1 -- 172.21.15.83:0/1977330060 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e23f2d460 msgr2=0x559e23f39140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T09:17:59.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.734+0000 7efee3200640 1 --2- 172.21.15.83:0/1977330060 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e23f2d460 0x559e23f39140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:59.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.734+0000 7efee3200640 1 -- 172.21.15.83:0/1977330060 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e23c6e7d0 con 0x559e23f37b30 2024-04-24T09:17:59.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.734+0000 7efed9e00640 1 --2- 172.21.15.83:0/1977330060 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e23f2d460 0x559e23f39140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:17:59.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.735+0000 7efee3200640 1 --2- 172.21.15.83:0/1977330060 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e23f37b30 0x559e23f3e290 secure :-1 s=READY pgs=2894 cs=0 l=1 rev1=1 crypto rx=0x7efedc09c500 tx=0x7efedc0871e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d8162fd3d0fbc4a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:17:59.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.735+0000 7efec7e00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-24T09:17:59.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.735+0000 7efec7e00640 1 -- 172.21.15.83:0/1977330060 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efedc01d040 con 0x559e23f37b30 2024-04-24T09:17:59.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.735+0000 7efec7e00640 1 -- 172.21.15.83:0/1977330060 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(134..134 src has 1..134) ==== 9583+0+0 (secure 0 0 0) 0x7efedc09db30 con 0x559e23f37b30 2024-04-24T09:17:59.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.735+0000 7efeec22b740 1 -- 172.21.15.83:0/1977330060 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efedc0a5470 con 0x559e23f37b30 2024-04-24T09:17:59.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.735+0000 7efec7e00640 1 -- 172.21.15.83:0/1977330060 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7efedc083050 con 0x559e23f37b30 2024-04-24T09:17:59.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.735+0000 7efeec22b740 10 client.24441 fetch_fsmap learned FSMap version 79 2024-04-24T09:17:59.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.735+0000 7efeec22b740 1 -- 172.21.15.83:0/1977330060 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=79,monmap=4+}) -- 0x559e23fae870 con 0x559e23f37b30 2024-04-24T09:17:59.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.735+0000 7efec7e00640 1 -- 172.21.15.83:0/1977330060 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efedc0a5470 con 0x559e23f37b30 2024-04-24T09:17:59.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.735+0000 7efec7e00640 1 -- 172.21.15.83:0/1977330060 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 79) ==== 8398+0+0 (secure 0 0 0) 0x7efedc0b3030 con 0x559e23f37b30 2024-04-24T09:17:59.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.736+0000 7efeec22b740 10 client.24441 fetch_fsmap finished waiting for FSMap version 79 2024-04-24T09:17:59.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.736+0000 7efeec22b740 10 client.24441 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T09:17:59.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.736+0000 7efeec22b740 20 client.24441 populate_metadata read hostname 'smithi083' 2024-04-24T09:17:59.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.736+0000 7efeec22b740 1 --2- 172.21.15.83:0/1977330060 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x559e23faef30 0x559e23fb1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:17:59.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.736+0000 7efeec22b740 4 client.24441 mds_command: new command op to 34520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T09:17:59.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.736+0000 7efeec22b740 1 -- 172.21.15.83:0/1977330060 --> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559e23e99a60 con 0x559e23faef30 2024-04-24T09:17:59.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.736+0000 7efee2800640 1 --2- 172.21.15.83:0/1977330060 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x559e23faef30 0x559e23fb1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:17:59.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.737+0000 7efee2800640 1 --2- 172.21.15.83:0/1977330060 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x559e23faef30 0x559e23fb1350 crc :-1 s=READY pgs=638 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7fa4878c96ee20ef server_cookie=92205450c881ff7 in_seq=0 out_seq=0 2024-04-24T09:17:59.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.737+0000 7efec7e00640 10 client.24441 ms_handle_connect on v2:172.21.15.138:6838/2236932393 2024-04-24T09:17:59.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.738+0000 7efec7e00640 1 -- 172.21.15.83:0/1977330060 <== mds.0 v2:172.21.15.138:6838/2236932393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559e23e99a60 con 0x559e23faef30 2024-04-24T09:17:59.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.738+0000 7efec7e00640 10 client.24441 handle_command_reply: tid=0 2024-04-24T09:17:59.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.738+0000 7efeec22b740 1 client.24441 shutdown 2024-04-24T09:17:59.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.738+0000 7efeec22b740 1 -- 172.21.15.83:0/1977330060 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e23f37b30 msgr2=0x559e23f3e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:59.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.738+0000 7efeec22b740 1 --2- 172.21.15.83:0/1977330060 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e23f37b30 0x559e23f3e290 secure :-1 s=READY pgs=2894 cs=0 l=1 rev1=1 crypto rx=0x7efedc09c500 tx=0x7efedc0871e0 comp rx=0 tx=0).stop 2024-04-24T09:17:59.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.738+0000 7efeec22b740 1 -- 172.21.15.83:0/1977330060 shutdown_connections 2024-04-24T09:17:59.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.738+0000 7efeec22b740 1 --2- 172.21.15.83:0/1977330060 >> [v2:172.21.15.138:6838/2236932393,v1:172.21.15.138:6839/2236932393] conn(0x559e23faef30 0x559e23fb1350 crc :-1 s=READY pgs=638 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:59.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.738+0000 7efeec22b740 1 --2- 172.21.15.83:0/1977330060 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559e23f37b30 0x559e23f3e290 unknown :-1 s=CLOSED pgs=2894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:59.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.738+0000 7efec7e00640 0 client.24441 ms_handle_reset on v2:172.21.15.138:6838/2236932393 2024-04-24T09:17:59.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.738+0000 7efeec22b740 1 --2- 172.21.15.83:0/1977330060 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559e23f2d460 0x559e23f39140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:59.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.738+0000 7efeec22b740 1 --2- 172.21.15.83:0/1977330060 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559e23f2cb40 0x559e23f38c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:59.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.738+0000 7efeec22b740 1 -- 172.21.15.83:0/1977330060 >> 172.21.15.83:0/1977330060 conn(0x559e23f28e60 msgr2=0x559e23f42840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:59.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.738+0000 7efeec22b740 1 -- 172.21.15.83:0/1977330060 shutdown_connections 2024-04-24T09:17:59.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.738+0000 7efeec22b740 1 -- 172.21.15.83:0/1977330060 wait complete. 2024-04-24T09:17:59.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.738+0000 7efeec22b740 20 client.24441 trim_cache size 0 max 16384 2024-04-24T09:17:59.741 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-24T09:17:59.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.739+0000 7efee8a00640 1 -- 172.21.15.83:0/319953000 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7efec80a3da0 msgr2=0x7efec80a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:59.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.739+0000 7efee8a00640 1 --2- 172.21.15.83:0/319953000 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7efec80a3da0 0x7efec80a6260 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7efed40058c0 tx=0x7efed4005850 comp rx=0 tx=0).stop 2024-04-24T09:17:59.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.739+0000 7efee8a00640 1 -- 172.21.15.83:0/319953000 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efee419cf40 msgr2=0x7efee419d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:17:59.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.739+0000 7efee8a00640 1 --2- 172.21.15.83:0/319953000 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efee419cf40 0x7efee419d3f0 secure :-1 s=READY pgs=2890 cs=0 l=1 rev1=1 crypto rx=0x7efedc00cad0 tx=0x7efedc00cf90 comp rx=0 tx=0).stop 2024-04-24T09:17:59.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.739+0000 7efee8a00640 1 -- 172.21.15.83:0/319953000 shutdown_connections 2024-04-24T09:17:59.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.739+0000 7efee8a00640 1 --2- 172.21.15.83:0/319953000 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7efec80a3da0 0x7efec80a6260 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:59.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.739+0000 7efee8a00640 1 --2- 172.21.15.83:0/319953000 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efee419cf40 0x7efee419d3f0 unknown :-1 s=CLOSED pgs=2890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:59.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.739+0000 7efee8a00640 1 --2- 172.21.15.83:0/319953000 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efee41071f0 0x7efee4198bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:59.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.739+0000 7efee8a00640 1 --2- 172.21.15.83:0/319953000 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efee4102120 0x7efee4198680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:17:59.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.739+0000 7efee8a00640 1 -- 172.21.15.83:0/319953000 >> 172.21.15.83:0/319953000 conn(0x7efee40fd9a0 msgr2=0x7efee40ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:17:59.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.739+0000 7efee8a00640 1 -- 172.21.15.83:0/319953000 shutdown_connections 2024-04-24T09:17:59.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:17:59.739+0000 7efee8a00640 1 -- 172.21.15.83:0/319953000 wait complete. 2024-04-24T09:18:00.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:18:00 smithi138 ceph-mon[31441]: pgmap v834: 129 pgs: 129 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-04-24T09:18:00.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:18:00 smithi189 ceph-mon[31791]: pgmap v834: 129 pgs: 129 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-04-24T09:18:00.758 INFO:tasks.fwd_scrub.fs.[cephfs]:end scrubbing fs: cephfs 2024-04-24T09:18:00.758 INFO:tasks.fwd_scrub:done joining 2024-04-24T09:18:00.758 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-04-24T09:18:00.770 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T09:18:00.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:18:00 smithi083 ceph-mon[27907]: pgmap v834: 129 pgs: 129 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-04-24T09:18:00.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:00.987+0000 7f750a600640 1 -- 172.21.15.83:0/1638251007 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7504109360 msgr2=0x7f7504110f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:00.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:00.987+0000 7f750a600640 1 --2- 172.21.15.83:0/1638251007 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7504109360 0x7f7504110f30 secure :-1 s=READY pgs=2895 cs=0 l=1 rev1=1 crypto rx=0x7f74fc009ca0 tx=0x7f74fc02a8c0 comp rx=0 tx=0).stop 2024-04-24T09:18:00.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:00.987+0000 7f750a600640 1 -- 172.21.15.83:0/1638251007 shutdown_connections 2024-04-24T09:18:00.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:00.987+0000 7f750a600640 1 --2- 172.21.15.83:0/1638251007 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7504109360 0x7f7504110f30 unknown :-1 s=CLOSED pgs=2895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:00.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:00.987+0000 7f750a600640 1 --2- 172.21.15.83:0/1638251007 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7504104c40 0x7f7504108c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:00.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:00.987+0000 7f750a600640 1 --2- 172.21.15.83:0/1638251007 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7504104290 0x7f7504104670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:00.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:00.987+0000 7f750a600640 1 -- 172.21.15.83:0/1638251007 >> 172.21.15.83:0/1638251007 conn(0x7f75040ffad0 msgr2=0x7f7504101ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:18:00.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:00.987+0000 7f750a600640 1 -- 172.21.15.83:0/1638251007 shutdown_connections 2024-04-24T09:18:00.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:00.987+0000 7f750a600640 1 -- 172.21.15.83:0/1638251007 wait complete. 2024-04-24T09:18:00.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:00.988+0000 7f750a600640 1 Processor -- start 2024-04-24T09:18:00.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:00.989+0000 7f750a600640 1 -- start start 2024-04-24T09:18:00.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:00.989+0000 7f750a600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7504104290 0x7f75041a05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:00.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:00.989+0000 7f750a600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7504109360 0x7f75041a0b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:00.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:00.989+0000 7f750a600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f750419a6a0 0x7f750419ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:00.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:00.989+0000 7f750a600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7504114300 con 0x7f7504109360 2024-04-24T09:18:00.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:00.989+0000 7f750a600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7504114180 con 0x7f7504104290 2024-04-24T09:18:00.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:00.989+0000 7f750a600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7504114480 con 0x7f750419a6a0 2024-04-24T09:18:00.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:00.989+0000 7f7503e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7504109360 0x7f75041a0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:00.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:00.989+0000 7f7503e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7504109360 0x7f75041a0b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33230/0 (socket says 172.21.15.83:33230) 2024-04-24T09:18:00.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:00.989+0000 7f7503e00640 1 -- 172.21.15.83:0/3107281503 learned_addr learned my addr 172.21.15.83:0/3107281503 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:18:00.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:00.990+0000 7f7509200640 1 --2- 172.21.15.83:0/3107281503 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7504104290 0x7f75041a05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:00.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:00.990+0000 7f7503e00640 1 -- 172.21.15.83:0/3107281503 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f750419a6a0 msgr2=0x7f750419ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:00.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:00.990+0000 7f7503e00640 1 --2- 172.21.15.83:0/3107281503 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f750419a6a0 0x7f750419ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:00.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:00.990+0000 7f7503e00640 1 -- 172.21.15.83:0/3107281503 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7504104290 msgr2=0x7f75041a05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:00.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:00.990+0000 7f7503e00640 1 --2- 172.21.15.83:0/3107281503 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7504104290 0x7f75041a05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:00.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:00.990+0000 7f7503e00640 1 -- 172.21.15.83:0/3107281503 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f750419b2a0 con 0x7f7504109360 2024-04-24T09:18:00.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:00.990+0000 7f7503e00640 1 --2- 172.21.15.83:0/3107281503 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7504109360 0x7f75041a0b20 secure :-1 s=READY pgs=2896 cs=0 l=1 rev1=1 crypto rx=0x7f74f400d770 tx=0x7f74f400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:18:00.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:00.990+0000 7f7501600640 1 -- 172.21.15.83:0/3107281503 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74f4017070 con 0x7f7504109360 2024-04-24T09:18:00.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:00.990+0000 7f7501600640 1 -- 172.21.15.83:0/3107281503 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f74f4009c30 con 0x7f7504109360 2024-04-24T09:18:00.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:00.990+0000 7f7501600640 1 -- 172.21.15.83:0/3107281503 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74f400f620 con 0x7f7504109360 2024-04-24T09:18:00.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:00.990+0000 7f750a600640 1 -- 172.21.15.83:0/3107281503 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75041a7380 con 0x7f7504109360 2024-04-24T09:18:00.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:00.991+0000 7f750a600640 1 -- 172.21.15.83:0/3107281503 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75041a7730 con 0x7f7504109360 2024-04-24T09:18:00.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:00.992+0000 7f750a600640 1 -- 172.21.15.83:0/3107281503 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7504105760 con 0x7f7504109360 2024-04-24T09:18:00.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:00.995+0000 7f7501600640 1 -- 172.21.15.83:0/3107281503 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f74f401a070 con 0x7f7504109360 2024-04-24T09:18:00.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:00.996+0000 7f7501600640 1 --2- 172.21.15.83:0/3107281503 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f74ec0a3da0 0x7f74ec0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:00.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:00.996+0000 7f7501600640 1 -- 172.21.15.83:0/3107281503 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 9583+0+0 (secure 0 0 0) 0x7f74f40d06e0 con 0x7f7504109360 2024-04-24T09:18:00.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:00.997+0000 7f7509200640 1 --2- 172.21.15.83:0/3107281503 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f74ec0a3da0 0x7f74ec0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:00.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:00.998+0000 7f7509200640 1 --2- 172.21.15.83:0/3107281503 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f74ec0a3da0 0x7f74ec0a6260 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f74f80058c0 tx=0x7f74f800a4c0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:18:01.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:01.002+0000 7f7501600640 1 -- 172.21.15.83:0/3107281503 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f74f4009de0 con 0x7f7504109360 2024-04-24T09:18:01.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:01.229+0000 7f750a600640 1 -- 172.21.15.83:0/3107281503 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f750410c590 con 0x7f7504109360 2024-04-24T09:18:01.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:01.231+0000 7f7501600640 1 -- 172.21.15.83:0/3107281503 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v134) ==== 74+0+25453 (secure 0 0 0) 0x7f74f409c960 con 0x7f7504109360 2024-04-24T09:18:01.231 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T09:18:01.231 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":134,"fsid":"bd3e039e-0215-11ef-bc93-c7b262605968","created":"2024-04-24T08:37:32.209944+0000","modified":"2024-04-24T09:17:09.817023+0000","last_up_change":"2024-04-24T08:47:17.966041+0000","last_in_change":"2024-04-24T08:46:53.835936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T08:41:56.834372+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T08:48:26.978311+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T08:48:28.352731+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"133","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":46,"snap_epoch":133,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T08:48:32.275167+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"133","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":46,"snap_epoch":133,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2cd77f3a-19dd-4c6e-a199-1dcbd5faa679","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2132675555},{"type":"v1","addr":"172.21.15.83:6803","nonce":2132675555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2132675555},{"type":"v1","addr":"172.21.15.83:6805","nonce":2132675555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2132675555},{"type":"v1","addr":"172.21.15.83:6809","nonce":2132675555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2132675555},{"type":"v1","addr":"172.21.15.83:6807","nonce":2132675555}]},"public_addr":"172.21.15.83:6803/2132675555","cluster_addr":"172.21.15.83:6805/2132675555","heartbeat_back_addr":"172.21.15.83:6809/2132675555","heartbeat_front_addr":"172.21.15.83:6807/2132675555","state":["exists","up"]},{"osd":1,"uuid":"aba7f5df-2a4c-4c23-b502-d193605a5b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":3589158880},{"type":"v1","addr":"172.21.15.138:6801","nonce":3589158880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":3589158880},{"type":"v1","addr":"172.21.15.138:6803","nonce":3589158880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":3589158880},{"type":"v1","addr":"172.21.15.138:6807","nonce":3589158880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":3589158880},{"type":"v1","addr":"172.21.15.138:6805","nonce":3589158880}]},"public_addr":"172.21.15.138:6801/3589158880","cluster_addr":"172.21.15.138:6803/3589158880","heartbeat_back_addr":"172.21.15.138:6807/3589158880","heartbeat_front_addr":"172.21.15.138:6805/3589158880","state":["exists","up"]},{"osd":2,"uuid":"071bc167-2333-4420-bb58-bffbc10d7641","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":3169072688},{"type":"v1","addr":"172.21.15.189:6801","nonce":3169072688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":3169072688},{"type":"v1","addr":"172.21.15.189:6803","nonce":3169072688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":3169072688},{"type":"v1","addr":"172.21.15.189:6807","nonce":3169072688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":3169072688},{"type":"v1","addr":"172.21.15.189:6805","nonce":3169072688}]},"public_addr":"172.21.15.189:6801/3169072688","cluster_addr":"172.21.15.189:6803/3169072688","heartbeat_back_addr":"172.21.15.189:6807/3169072688","heartbeat_front_addr":"172.21.15.189:6805/3169072688","state":["exists","up"]},{"osd":3,"uuid":"f5624d78-4455-4e07-8c1a-f47c94cddb23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3131092450},{"type":"v1","addr":"172.21.15.83:6811","nonce":3131092450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3131092450},{"type":"v1","addr":"172.21.15.83:6813","nonce":3131092450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3131092450},{"type":"v1","addr":"172.21.15.83:6817","nonce":3131092450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3131092450},{"type":"v1","addr":"172.21.15.83:6815","nonce":3131092450}]},"public_addr":"172.21.15.83:6811/3131092450","cluster_addr":"172.21.15.83:6813/3131092450","heartbeat_back_addr":"172.21.15.83:6817/3131092450","heartbeat_front_addr":"172.21.15.83:6815/3131092450","state":["exists","up"]},{"osd":4,"uuid":"7b50cd14-9385-4535-a7b0-1c1da93c8548","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":823775360},{"type":"v1","addr":"172.21.15.138:6809","nonce":823775360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":823775360},{"type":"v1","addr":"172.21.15.138:6811","nonce":823775360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":823775360},{"type":"v1","addr":"172.21.15.138:6815","nonce":823775360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":823775360},{"type":"v1","addr":"172.21.15.138:6813","nonce":823775360}]},"public_addr":"172.21.15.138:6809/823775360","cluster_addr":"172.21.15.138:6811/823775360","heartbeat_back_addr":"172.21.15.138:6815/823775360","heartbeat_front_addr":"172.21.15.138:6813/823775360","state":["exists","up"]},{"osd":5,"uuid":"9c647797-35bc-449f-be84-af97d2c69c2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":1399817431},{"type":"v1","addr":"172.21.15.189:6809","nonce":1399817431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":1399817431},{"type":"v1","addr":"172.21.15.189:6811","nonce":1399817431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":1399817431},{"type":"v1","addr":"172.21.15.189:6815","nonce":1399817431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":1399817431},{"type":"v1","addr":"172.21.15.189:6813","nonce":1399817431}]},"public_addr":"172.21.15.189:6809/1399817431","cluster_addr":"172.21.15.189:6811/1399817431","heartbeat_back_addr":"172.21.15.189:6815/1399817431","heartbeat_front_addr":"172.21.15.189:6813/1399817431","state":["exists","up"]},{"osd":6,"uuid":"8f056584-16c8-4454-b445-3f780f26c196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":1611358483},{"type":"v1","addr":"172.21.15.83:6819","nonce":1611358483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1611358483},{"type":"v1","addr":"172.21.15.83:6821","nonce":1611358483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1611358483},{"type":"v1","addr":"172.21.15.83:6825","nonce":1611358483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":1611358483},{"type":"v1","addr":"172.21.15.83:6823","nonce":1611358483}]},"public_addr":"172.21.15.83:6819/1611358483","cluster_addr":"172.21.15.83:6821/1611358483","heartbeat_back_addr":"172.21.15.83:6825/1611358483","heartbeat_front_addr":"172.21.15.83:6823/1611358483","state":["exists","up"]},{"osd":7,"uuid":"7603efe3-da77-4b7f-9d03-29c4f293b6d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":3557856706},{"type":"v1","addr":"172.21.15.138:6817","nonce":3557856706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":3557856706},{"type":"v1","addr":"172.21.15.138:6819","nonce":3557856706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":3557856706},{"type":"v1","addr":"172.21.15.138:6823","nonce":3557856706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":3557856706},{"type":"v1","addr":"172.21.15.138:6821","nonce":3557856706}]},"public_addr":"172.21.15.138:6817/3557856706","cluster_addr":"172.21.15.138:6819/3557856706","heartbeat_back_addr":"172.21.15.138:6823/3557856706","heartbeat_front_addr":"172.21.15.138:6821/3557856706","state":["exists","up"]},{"osd":8,"uuid":"27d7d1b8-d6a0-417e-b8aa-596325aa1a77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":207018554},{"type":"v1","addr":"172.21.15.189:6817","nonce":207018554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":207018554},{"type":"v1","addr":"172.21.15.189:6819","nonce":207018554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":207018554},{"type":"v1","addr":"172.21.15.189:6823","nonce":207018554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":207018554},{"type":"v1","addr":"172.21.15.189:6821","nonce":207018554}]},"public_addr":"172.21.15.189:6817/207018554","cluster_addr":"172.21.15.189:6819/207018554","heartbeat_back_addr":"172.21.15.189:6823/207018554","heartbeat_front_addr":"172.21.15.189:6821/207018554","state":["exists","up"]},{"osd":9,"uuid":"baf19b98-881b-4d3a-9f72-8aa2dd73cad6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":519636751},{"type":"v1","addr":"172.21.15.83:6827","nonce":519636751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":519636751},{"type":"v1","addr":"172.21.15.83:6829","nonce":519636751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":519636751},{"type":"v1","addr":"172.21.15.83:6833","nonce":519636751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":519636751},{"type":"v1","addr":"172.21.15.83:6831","nonce":519636751}]},"public_addr":"172.21.15.83:6827/519636751","cluster_addr":"172.21.15.83:6829/519636751","heartbeat_back_addr":"172.21.15.83:6833/519636751","heartbeat_front_addr":"172.21.15.83:6831/519636751","state":["exists","up"]},{"osd":10,"uuid":"eb346717-5c07-47c8-95be-0e68fe449fdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1130872098},{"type":"v1","addr":"172.21.15.138:6825","nonce":1130872098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1130872098},{"type":"v1","addr":"172.21.15.138:6827","nonce":1130872098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1130872098},{"type":"v1","addr":"172.21.15.138:6831","nonce":1130872098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1130872098},{"type":"v1","addr":"172.21.15.138:6829","nonce":1130872098}]},"public_addr":"172.21.15.138:6825/1130872098","cluster_addr":"172.21.15.138:6827/1130872098","heartbeat_back_addr":"172.21.15.138:6831/1130872098","heartbeat_front_addr":"172.21.15.138:6829/1130872098","state":["exists","up"]},{"osd":11,"uuid":"8e37568e-981c-45b8-b2af-2f1b73898542","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":2998782690},{"type":"v1","addr":"172.21.15.189:6825","nonce":2998782690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":2998782690},{"type":"v1","addr":"172.21.15.189:6827","nonce":2998782690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":2998782690},{"type":"v1","addr":"172.21.15.189:6831","nonce":2998782690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":2998782690},{"type":"v1","addr":"172.21.15.189:6829","nonce":2998782690}]},"public_addr":"172.21.15.189:6825/2998782690","cluster_addr":"172.21.15.189:6827/2998782690","heartbeat_back_addr":"172.21.15.189:6831/2998782690","heartbeat_front_addr":"172.21.15.189:6829/2998782690","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:40:47.980103+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:41:21.970795+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:41:55.160426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:42:30.400475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:43:05.705663+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:43:40.112312+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:44:16.781289+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:44:52.151187+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:45:28.055944+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:46:04.969301+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:46:40.918306+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:47:16.886874+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"2.1d","mappings":[{"from":2,"to":1}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.2a","mappings":[{"from":3,"to":8}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.a","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:0/1096290901":"2024-04-25T08:51:36.347763+0000","172.21.15.83:6800/2550826560":"2024-04-25T08:51:36.347763+0000","172.21.15.83:0/1743153945":"2024-04-25T08:51:36.347763+0000","172.21.15.83:0/3306681366":"2024-04-25T08:51:36.347763+0000","172.21.15.83:6801/820049883":"2024-04-25T08:38:40.661315+0000","172.21.15.83:6801/2550826560":"2024-04-25T08:51:36.347763+0000","172.21.15.83:0/4272874138":"2024-04-25T08:38:40.661315+0000","172.21.15.83:0/4212994335":"2024-04-25T08:38:40.661315+0000","172.21.15.83:6801/1996371091":"2024-04-25T08:38:05.800856+0000","172.21.15.83:0/1820663683":"2024-04-25T08:38:05.800856+0000","172.21.15.83:0/3802897262":"2024-04-25T08:38:05.800856+0000","172.21.15.83:6800/1996371091":"2024-04-25T08:38:05.800856+0000","172.21.15.83:6800/820049883":"2024-04-25T08:38:40.661315+0000","172.21.15.83:0/1366125049":"2024-04-25T08:38:05.800856+0000","172.21.15.83:0/4159820288":"2024-04-25T08:51:36.347763+0000","172.21.15.83:0/340452868":"2024-04-25T08:38:40.661315+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":33,"length":1},{"begin":46,"length":1}]},{"pool":4,"snaps":[{"begin":33,"length":1},{"begin":46,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T09:18:01.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:01.234+0000 7f750a600640 1 -- 172.21.15.83:0/3107281503 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f74ec0a3da0 msgr2=0x7f74ec0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:01.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:01.234+0000 7f750a600640 1 --2- 172.21.15.83:0/3107281503 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f74ec0a3da0 0x7f74ec0a6260 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f74f80058c0 tx=0x7f74f800a4c0 comp rx=0 tx=0).stop 2024-04-24T09:18:01.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:01.234+0000 7f750a600640 1 -- 172.21.15.83:0/3107281503 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7504109360 msgr2=0x7f75041a0b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:01.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:01.234+0000 7f750a600640 1 --2- 172.21.15.83:0/3107281503 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7504109360 0x7f75041a0b20 secure :-1 s=READY pgs=2896 cs=0 l=1 rev1=1 crypto rx=0x7f74f400d770 tx=0x7f74f400dc30 comp rx=0 tx=0).stop 2024-04-24T09:18:01.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:01.234+0000 7f750a600640 1 -- 172.21.15.83:0/3107281503 shutdown_connections 2024-04-24T09:18:01.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:01.234+0000 7f750a600640 1 --2- 172.21.15.83:0/3107281503 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f74ec0a3da0 0x7f74ec0a6260 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:01.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:01.234+0000 7f750a600640 1 --2- 172.21.15.83:0/3107281503 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f750419a6a0 0x7f750419ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:01.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:01.234+0000 7f750a600640 1 --2- 172.21.15.83:0/3107281503 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7504109360 0x7f75041a0b20 unknown :-1 s=CLOSED pgs=2896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:01.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:01.234+0000 7f750a600640 1 --2- 172.21.15.83:0/3107281503 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7504104290 0x7f75041a05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:01.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:01.234+0000 7f750a600640 1 -- 172.21.15.83:0/3107281503 >> 172.21.15.83:0/3107281503 conn(0x7f75040ffad0 msgr2=0x7f7504101d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:18:01.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:01.235+0000 7f750a600640 1 -- 172.21.15.83:0/3107281503 shutdown_connections 2024-04-24T09:18:01.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:01.235+0000 7f750a600640 1 -- 172.21.15.83:0/3107281503 wait complete. 2024-04-24T09:18:01.254 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T08:41:56.834372+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T08:48:26.978311+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T08:48:28.352731+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '133', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 46, 'snap_epoch': 133, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-24T08:48:32.275167+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '133', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 46, 'snap_epoch': 133, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-24T09:18:01.255 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T09:18:01.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:01.509+0000 7ffb19400640 1 -- 172.21.15.83:0/4234372003 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb140743b0 msgr2=0x7ffb14074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:01.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:01.509+0000 7ffb19400640 1 --2- 172.21.15.83:0/4234372003 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb140743b0 0x7ffb14074790 secure :-1 s=READY pgs=2897 cs=0 l=1 rev1=1 crypto rx=0x7ffb04009900 tx=0x7ffb0402a480 comp rx=0 tx=0).stop 2024-04-24T09:18:01.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:01.509+0000 7ffb19400640 1 -- 172.21.15.83:0/4234372003 shutdown_connections 2024-04-24T09:18:01.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:01.509+0000 7ffb19400640 1 --2- 172.21.15.83:0/4234372003 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffb1410e770 0x7ffb14110b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:01.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:01.509+0000 7ffb19400640 1 --2- 172.21.15.83:0/4234372003 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb14074cd0 0x7ffb1410e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:01.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:01.509+0000 7ffb19400640 1 --2- 172.21.15.83:0/4234372003 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb140743b0 0x7ffb14074790 unknown :-1 s=CLOSED pgs=2897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:01.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:01.509+0000 7ffb19400640 1 -- 172.21.15.83:0/4234372003 >> 172.21.15.83:0/4234372003 conn(0x7ffb140fd9a0 msgr2=0x7ffb140ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:18:01.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:01.509+0000 7ffb19400640 1 -- 172.21.15.83:0/4234372003 shutdown_connections 2024-04-24T09:18:01.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:01.509+0000 7ffb19400640 1 -- 172.21.15.83:0/4234372003 wait complete. 2024-04-24T09:18:01.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:01.510+0000 7ffb19400640 1 Processor -- start 2024-04-24T09:18:01.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:01.510+0000 7ffb19400640 1 -- start start 2024-04-24T09:18:01.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:01.510+0000 7ffb19400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb140743b0 0x7ffb141a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:01.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:01.510+0000 7ffb19400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb14074cd0 0x7ffb141a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:01.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:01.511+0000 7ffb19400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffb1410e770 0x7ffb1419a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:01.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:01.511+0000 7ffb19400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ffb14114370 con 0x7ffb14074cd0 2024-04-24T09:18:01.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:01.511+0000 7ffb19400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ffb141141f0 con 0x7ffb140743b0 2024-04-24T09:18:01.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:01.511+0000 7ffb19400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ffb141144f0 con 0x7ffb1410e770 2024-04-24T09:18:01.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:01.511+0000 7ffb13400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb14074cd0 0x7ffb141a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:01.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:01.511+0000 7ffb13400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb14074cd0 0x7ffb141a0b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33246/0 (socket says 172.21.15.83:33246) 2024-04-24T09:18:01.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:01.511+0000 7ffb13400640 1 -- 172.21.15.83:0/711259358 learned_addr learned my addr 172.21.15.83:0/711259358 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:18:01.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:01.511+0000 7ffb13e00640 1 --2- 172.21.15.83:0/711259358 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb140743b0 0x7ffb141a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:01.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:01.511+0000 7ffb18a00640 1 --2- 172.21.15.83:0/711259358 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffb1410e770 0x7ffb1419a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:01.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:01.511+0000 7ffb13400640 1 -- 172.21.15.83:0/711259358 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffb1410e770 msgr2=0x7ffb1419a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:01.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:01.511+0000 7ffb13400640 1 --2- 172.21.15.83:0/711259358 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffb1410e770 0x7ffb1419a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:01.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:01.511+0000 7ffb13400640 1 -- 172.21.15.83:0/711259358 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb140743b0 msgr2=0x7ffb141a0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:01.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:01.511+0000 7ffb13400640 1 --2- 172.21.15.83:0/711259358 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb140743b0 0x7ffb141a0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:01.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:01.511+0000 7ffb13400640 1 -- 172.21.15.83:0/711259358 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb1419afd0 con 0x7ffb14074cd0 2024-04-24T09:18:01.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:01.511+0000 7ffb18a00640 1 --2- 172.21.15.83:0/711259358 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffb1410e770 0x7ffb1419a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:18:01.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:01.512+0000 7ffb13400640 1 --2- 172.21.15.83:0/711259358 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb14074cd0 0x7ffb141a0b90 secure :-1 s=READY pgs=2898 cs=0 l=1 rev1=1 crypto rx=0x7ffb0000d950 tx=0x7ffb0000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:18:01.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:01.512+0000 7ffb10c00640 1 -- 172.21.15.83:0/711259358 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb00017070 con 0x7ffb14074cd0 2024-04-24T09:18:01.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:01.512+0000 7ffb19400640 1 -- 172.21.15.83:0/711259358 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb1419b2c0 con 0x7ffb14074cd0 2024-04-24T09:18:01.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:01.512+0000 7ffb10c00640 1 -- 172.21.15.83:0/711259358 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffb00009c30 con 0x7ffb14074cd0 2024-04-24T09:18:01.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:01.512+0000 7ffb10c00640 1 -- 172.21.15.83:0/711259358 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb0000f620 con 0x7ffb14074cd0 2024-04-24T09:18:01.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:01.512+0000 7ffb19400640 1 -- 172.21.15.83:0/711259358 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb141a75d0 con 0x7ffb14074cd0 2024-04-24T09:18:01.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:01.513+0000 7ffb19400640 1 -- 172.21.15.83:0/711259358 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb14075a60 con 0x7ffb14074cd0 2024-04-24T09:18:01.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:01.516+0000 7ffb10c00640 1 -- 172.21.15.83:0/711259358 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ffb000109a0 con 0x7ffb14074cd0 2024-04-24T09:18:01.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:01.517+0000 7ffb10c00640 1 --2- 172.21.15.83:0/711259358 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ffaf80a3da0 0x7ffaf80a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:01.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:01.517+0000 7ffb10c00640 1 -- 172.21.15.83:0/711259358 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 9583+0+0 (secure 0 0 0) 0x7ffb000d08e0 con 0x7ffb14074cd0 2024-04-24T09:18:01.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:01.518+0000 7ffb13e00640 1 --2- 172.21.15.83:0/711259358 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ffaf80a3da0 0x7ffaf80a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:01.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:01.518+0000 7ffb13e00640 1 --2- 172.21.15.83:0/711259358 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ffaf80a3da0 0x7ffaf80a6260 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7ffb04000c00 tx=0x7ffb04031400 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:18:01.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:01.522+0000 7ffb10c00640 1 -- 172.21.15.83:0/711259358 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ffb00009de0 con 0x7ffb14074cd0 2024-04-24T09:18:01.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:18:01 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/3107281503' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:18:01.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:18:01 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/3107281503' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:18:01.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:18:01 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/3107281503' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:18:01.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:01.726+0000 7ffb19400640 1 -- 172.21.15.83:0/711259358 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ffb141098f0 con 0x7ffb14074cd0 2024-04-24T09:18:01.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:01.726+0000 7ffb10c00640 1 -- 172.21.15.83:0/711259358 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v134) ==== 93+0+10 (secure 0 0 0) 0x7ffb0009cb00 con 0x7ffb14074cd0 2024-04-24T09:18:01.726 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-24T09:18:01.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:01.729+0000 7ffb19400640 1 -- 172.21.15.83:0/711259358 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ffaf80a3da0 msgr2=0x7ffaf80a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:01.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:01.729+0000 7ffb19400640 1 --2- 172.21.15.83:0/711259358 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ffaf80a3da0 0x7ffaf80a6260 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7ffb04000c00 tx=0x7ffb04031400 comp rx=0 tx=0).stop 2024-04-24T09:18:01.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:01.729+0000 7ffb19400640 1 -- 172.21.15.83:0/711259358 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb14074cd0 msgr2=0x7ffb141a0b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:01.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:01.729+0000 7ffb19400640 1 --2- 172.21.15.83:0/711259358 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb14074cd0 0x7ffb141a0b90 secure :-1 s=READY pgs=2898 cs=0 l=1 rev1=1 crypto rx=0x7ffb0000d950 tx=0x7ffb0000de10 comp rx=0 tx=0).stop 2024-04-24T09:18:01.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:01.729+0000 7ffb19400640 1 -- 172.21.15.83:0/711259358 shutdown_connections 2024-04-24T09:18:01.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:01.729+0000 7ffb19400640 1 --2- 172.21.15.83:0/711259358 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ffaf80a3da0 0x7ffaf80a6260 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:01.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:01.729+0000 7ffb19400640 1 --2- 172.21.15.83:0/711259358 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffb1410e770 0x7ffb1419a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:01.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:01.729+0000 7ffb19400640 1 --2- 172.21.15.83:0/711259358 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb14074cd0 0x7ffb141a0b90 unknown :-1 s=CLOSED pgs=2898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:01.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:01.729+0000 7ffb19400640 1 --2- 172.21.15.83:0/711259358 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb140743b0 0x7ffb141a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:01.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:01.729+0000 7ffb19400640 1 -- 172.21.15.83:0/711259358 >> 172.21.15.83:0/711259358 conn(0x7ffb140fd9a0 msgr2=0x7ffb140ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:18:01.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:01.730+0000 7ffb19400640 1 -- 172.21.15.83:0/711259358 shutdown_connections 2024-04-24T09:18:01.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:01.730+0000 7ffb19400640 1 -- 172.21.15.83:0/711259358 wait complete. 2024-04-24T09:18:01.746 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T09:18:01.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:01.949+0000 7fba1ac00640 1 -- 172.21.15.83:0/3926018587 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba141071f0 msgr2=0x7fba1410edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:01.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:01.949+0000 7fba1ac00640 1 --2- 172.21.15.83:0/3926018587 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba141071f0 0x7fba1410edc0 secure :-1 s=READY pgs=2899 cs=0 l=1 rev1=1 crypto rx=0x7fba0c009ca0 tx=0x7fba0c02a8b0 comp rx=0 tx=0).stop 2024-04-24T09:18:01.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:01.949+0000 7fba1ac00640 1 -- 172.21.15.83:0/3926018587 shutdown_connections 2024-04-24T09:18:01.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:01.949+0000 7fba1ac00640 1 --2- 172.21.15.83:0/3926018587 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba141071f0 0x7fba1410edc0 unknown :-1 s=CLOSED pgs=2899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:01.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:01.949+0000 7fba1ac00640 1 --2- 172.21.15.83:0/3926018587 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fba14102ad0 0x7fba14106b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:01.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:01.949+0000 7fba1ac00640 1 --2- 172.21.15.83:0/3926018587 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba14102120 0x7fba14102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:01.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:01.949+0000 7fba1ac00640 1 -- 172.21.15.83:0/3926018587 >> 172.21.15.83:0/3926018587 conn(0x7fba140fd9a0 msgr2=0x7fba140ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:18:01.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:01.949+0000 7fba1ac00640 1 -- 172.21.15.83:0/3926018587 shutdown_connections 2024-04-24T09:18:01.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:01.949+0000 7fba1ac00640 1 -- 172.21.15.83:0/3926018587 wait complete. 2024-04-24T09:18:01.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:01.950+0000 7fba1ac00640 1 Processor -- start 2024-04-24T09:18:01.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:01.950+0000 7fba1ac00640 1 -- start start 2024-04-24T09:18:01.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:01.950+0000 7fba1ac00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba14102120 0x7fba141a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:01.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:01.950+0000 7fba1ac00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fba141071f0 0x7fba141a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:01.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:01.950+0000 7fba1ac00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba1419a710 0x7fba1419ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:01.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:01.950+0000 7fba1ac00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fba141121b0 con 0x7fba14102120 2024-04-24T09:18:01.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:01.950+0000 7fba1ac00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fba14112030 con 0x7fba1419a710 2024-04-24T09:18:01.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:01.950+0000 7fba1ac00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fba14112330 con 0x7fba141071f0 2024-04-24T09:18:01.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:01.951+0000 7fba19800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba14102120 0x7fba141a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:01.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:01.951+0000 7fba19800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba14102120 0x7fba141a0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33258/0 (socket says 172.21.15.83:33258) 2024-04-24T09:18:01.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:01.951+0000 7fba19800640 1 -- 172.21.15.83:0/2232833794 learned_addr learned my addr 172.21.15.83:0/2232833794 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:18:01.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:01.951+0000 7fba1a200640 1 --2- 172.21.15.83:0/2232833794 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba1419a710 0x7fba1419ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:01.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:01.951+0000 7fba19800640 1 -- 172.21.15.83:0/2232833794 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fba141071f0 msgr2=0x7fba141a0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:01.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:01.951+0000 7fba19800640 1 --2- 172.21.15.83:0/2232833794 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fba141071f0 0x7fba141a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:01.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:01.951+0000 7fba19800640 1 -- 172.21.15.83:0/2232833794 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba1419a710 msgr2=0x7fba1419ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:01.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:01.951+0000 7fba19800640 1 --2- 172.21.15.83:0/2232833794 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba1419a710 0x7fba1419ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:01.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:01.951+0000 7fba19800640 1 -- 172.21.15.83:0/2232833794 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba1419b310 con 0x7fba14102120 2024-04-24T09:18:01.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:01.951+0000 7fba19800640 1 --2- 172.21.15.83:0/2232833794 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba14102120 0x7fba141a0650 secure :-1 s=READY pgs=2900 cs=0 l=1 rev1=1 crypto rx=0x7fba0800c890 tx=0x7fba0800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:18:01.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:01.952+0000 7fba12000640 1 -- 172.21.15.83:0/2232833794 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba08004220 con 0x7fba14102120 2024-04-24T09:18:01.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:01.952+0000 7fba12000640 1 -- 172.21.15.83:0/2232833794 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fba080043c0 con 0x7fba14102120 2024-04-24T09:18:01.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:01.952+0000 7fba12000640 1 -- 172.21.15.83:0/2232833794 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba08004a00 con 0x7fba14102120 2024-04-24T09:18:01.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:01.952+0000 7fba1ac00640 1 -- 172.21.15.83:0/2232833794 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba141a73f0 con 0x7fba14102120 2024-04-24T09:18:01.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:01.952+0000 7fba1ac00640 1 -- 172.21.15.83:0/2232833794 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba141a7850 con 0x7fba14102120 2024-04-24T09:18:01.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:01.953+0000 7fba1ac00640 1 -- 172.21.15.83:0/2232833794 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba141035f0 con 0x7fba14102120 2024-04-24T09:18:01.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:01.955+0000 7fba12000640 1 -- 172.21.15.83:0/2232833794 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fba08004ba0 con 0x7fba14102120 2024-04-24T09:18:01.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:01.956+0000 7fba12000640 1 --2- 172.21.15.83:0/2232833794 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb9fc0a3da0 0x7fb9fc0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:01.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:01.957+0000 7fba12000640 1 -- 172.21.15.83:0/2232833794 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 9583+0+0 (secure 0 0 0) 0x7fba080d0950 con 0x7fba14102120 2024-04-24T09:18:01.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:01.957+0000 7fba18e00640 1 --2- 172.21.15.83:0/2232833794 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb9fc0a3da0 0x7fb9fc0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:01.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:01.957+0000 7fba18e00640 1 --2- 172.21.15.83:0/2232833794 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb9fc0a3da0 0x7fb9fc0a6260 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fba1419bf60 tx=0x7fba04006c90 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:18:01.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:01.960+0000 7fba12000640 1 -- 172.21.15.83:0/2232833794 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fba08014030 con 0x7fba14102120 2024-04-24T09:18:02.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:02.163+0000 7fba1ac00640 1 -- 172.21.15.83:0/2232833794 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fba1419bc40 con 0x7fba14102120 2024-04-24T09:18:02.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:02.164+0000 7fba12000640 1 -- 172.21.15.83:0/2232833794 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v134) ==== 104+0+11 (secure 0 0 0) 0x7fba0809cb70 con 0x7fba14102120 2024-04-24T09:18:02.164 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-24T09:18:02.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:02.167+0000 7fba1ac00640 1 -- 172.21.15.83:0/2232833794 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb9fc0a3da0 msgr2=0x7fb9fc0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:02.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:02.167+0000 7fba1ac00640 1 --2- 172.21.15.83:0/2232833794 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb9fc0a3da0 0x7fb9fc0a6260 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fba1419bf60 tx=0x7fba04006c90 comp rx=0 tx=0).stop 2024-04-24T09:18:02.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:02.167+0000 7fba1ac00640 1 -- 172.21.15.83:0/2232833794 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba14102120 msgr2=0x7fba141a0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:02.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:02.167+0000 7fba1ac00640 1 --2- 172.21.15.83:0/2232833794 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba14102120 0x7fba141a0650 secure :-1 s=READY pgs=2900 cs=0 l=1 rev1=1 crypto rx=0x7fba0800c890 tx=0x7fba0800cd50 comp rx=0 tx=0).stop 2024-04-24T09:18:02.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:02.167+0000 7fba1ac00640 1 -- 172.21.15.83:0/2232833794 shutdown_connections 2024-04-24T09:18:02.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:02.167+0000 7fba1ac00640 1 --2- 172.21.15.83:0/2232833794 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb9fc0a3da0 0x7fb9fc0a6260 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:02.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:02.167+0000 7fba1ac00640 1 --2- 172.21.15.83:0/2232833794 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba1419a710 0x7fba1419ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:02.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:02.167+0000 7fba1ac00640 1 --2- 172.21.15.83:0/2232833794 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fba141071f0 0x7fba141a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:02.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:02.167+0000 7fba1ac00640 1 --2- 172.21.15.83:0/2232833794 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba14102120 0x7fba141a0650 unknown :-1 s=CLOSED pgs=2900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:02.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:02.167+0000 7fba1ac00640 1 -- 172.21.15.83:0/2232833794 >> 172.21.15.83:0/2232833794 conn(0x7fba140fd9a0 msgr2=0x7fba140fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:18:02.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:02.168+0000 7fba1ac00640 1 -- 172.21.15.83:0/2232833794 shutdown_connections 2024-04-24T09:18:02.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:02.168+0000 7fba1ac00640 1 -- 172.21.15.83:0/2232833794 wait complete. 2024-04-24T09:18:02.185 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T09:18:02.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:02.396+0000 7f2abea00640 1 -- 172.21.15.83:0/2601178209 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ab8109520 msgr2=0x7f2ab81110f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:02.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:02.396+0000 7f2abea00640 1 --2- 172.21.15.83:0/2601178209 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ab8109520 0x7f2ab81110f0 secure :-1 s=READY pgs=2901 cs=0 l=1 rev1=1 crypto rx=0x7f2ab0009ca0 tx=0x7f2ab002a8c0 comp rx=0 tx=0).stop 2024-04-24T09:18:02.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:02.396+0000 7f2abea00640 1 -- 172.21.15.83:0/2601178209 shutdown_connections 2024-04-24T09:18:02.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:02.396+0000 7f2abea00640 1 --2- 172.21.15.83:0/2601178209 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ab8109520 0x7f2ab81110f0 unknown :-1 s=CLOSED pgs=2901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:02.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:02.396+0000 7f2abea00640 1 --2- 172.21.15.83:0/2601178209 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ab8104e00 0x7f2ab8108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:02.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:02.396+0000 7f2abea00640 1 --2- 172.21.15.83:0/2601178209 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2ab8104450 0x7f2ab8104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:02.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:02.396+0000 7f2abea00640 1 -- 172.21.15.83:0/2601178209 >> 172.21.15.83:0/2601178209 conn(0x7f2ab80ffc70 msgr2=0x7f2ab8102090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:18:02.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:02.396+0000 7f2abea00640 1 -- 172.21.15.83:0/2601178209 shutdown_connections 2024-04-24T09:18:02.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:02.396+0000 7f2abea00640 1 -- 172.21.15.83:0/2601178209 wait complete. 2024-04-24T09:18:02.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:02.397+0000 7f2abea00640 1 Processor -- start 2024-04-24T09:18:02.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:02.397+0000 7f2abea00640 1 -- start start 2024-04-24T09:18:02.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:02.398+0000 7f2abea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ab8104450 0x7f2ab81a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:02.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:02.398+0000 7f2abea00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ab8109520 0x7f2ab81a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:02.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:02.398+0000 7f2abea00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2ab819a810 0x7f2ab819acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:02.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:02.398+0000 7f2abea00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2ab8114490 con 0x7f2ab8104450 2024-04-24T09:18:02.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:02.398+0000 7f2abea00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2ab8114310 con 0x7f2ab8109520 2024-04-24T09:18:02.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:02.398+0000 7f2abea00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2ab8114610 con 0x7f2ab819a810 2024-04-24T09:18:02.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:02.398+0000 7f2ab7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ab8104450 0x7f2ab81a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:02.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:02.398+0000 7f2ab7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ab8104450 0x7f2ab81a06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33264/0 (socket says 172.21.15.83:33264) 2024-04-24T09:18:02.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:02.398+0000 7f2ab7e00640 1 -- 172.21.15.83:0/3025605232 learned_addr learned my addr 172.21.15.83:0/3025605232 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:18:02.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:02.399+0000 7f2ab7400640 1 --2- 172.21.15.83:0/3025605232 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ab8109520 0x7f2ab81a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:02.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:02.399+0000 7f2ab7e00640 1 -- 172.21.15.83:0/3025605232 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2ab819a810 msgr2=0x7f2ab819acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:02.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:02.399+0000 7f2ab7e00640 1 --2- 172.21.15.83:0/3025605232 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2ab819a810 0x7f2ab819acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:02.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:02.399+0000 7f2ab7e00640 1 -- 172.21.15.83:0/3025605232 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ab8109520 msgr2=0x7f2ab81a0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:02.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:02.399+0000 7f2ab7e00640 1 --2- 172.21.15.83:0/3025605232 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ab8109520 0x7f2ab81a0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:02.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:02.399+0000 7f2ab7e00640 1 -- 172.21.15.83:0/3025605232 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ab819b3a0 con 0x7f2ab8104450 2024-04-24T09:18:02.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:02.399+0000 7f2ab7e00640 1 --2- 172.21.15.83:0/3025605232 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ab8104450 0x7f2ab81a06a0 secure :-1 s=READY pgs=2902 cs=0 l=1 rev1=1 crypto rx=0x7f2aa800d770 tx=0x7f2aa800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:18:02.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:02.399+0000 7f2ab4c00640 1 -- 172.21.15.83:0/3025605232 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aa8017070 con 0x7f2ab8104450 2024-04-24T09:18:02.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:02.399+0000 7f2ab4c00640 1 -- 172.21.15.83:0/3025605232 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2aa80054d0 con 0x7f2ab8104450 2024-04-24T09:18:02.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:02.399+0000 7f2abea00640 1 -- 172.21.15.83:0/3025605232 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ab819b690 con 0x7f2ab8104450 2024-04-24T09:18:02.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:02.399+0000 7f2ab4c00640 1 -- 172.21.15.83:0/3025605232 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aa8004cf0 con 0x7f2ab8104450 2024-04-24T09:18:02.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:02.399+0000 7f2abea00640 1 -- 172.21.15.83:0/3025605232 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ab81a7820 con 0x7f2ab8104450 2024-04-24T09:18:02.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:02.401+0000 7f2abea00640 1 -- 172.21.15.83:0/3025605232 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ab8105920 con 0x7f2ab8104450 2024-04-24T09:18:02.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:02.403+0000 7f2ab4c00640 1 -- 172.21.15.83:0/3025605232 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2aa8004e90 con 0x7f2ab8104450 2024-04-24T09:18:02.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:02.404+0000 7f2ab4c00640 1 --2- 172.21.15.83:0/3025605232 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f2a9c0a3d20 0x7f2a9c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:02.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:02.404+0000 7f2ab4c00640 1 -- 172.21.15.83:0/3025605232 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 9583+0+0 (secure 0 0 0) 0x7f2aa80d0750 con 0x7f2ab8104450 2024-04-24T09:18:02.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:02.404+0000 7f2ab7400640 1 --2- 172.21.15.83:0/3025605232 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f2a9c0a3d20 0x7f2a9c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:02.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:02.405+0000 7f2ab7400640 1 --2- 172.21.15.83:0/3025605232 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f2a9c0a3d20 0x7f2a9c0a61e0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f2ab810aad0 tx=0x7f2aa0005f90 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:18:02.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:02.408+0000 7f2ab4c00640 1 -- 172.21.15.83:0/3025605232 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2ab819b690 con 0x7f2ab8104450 2024-04-24T09:18:02.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:18:02 smithi083 ceph-mon[27907]: pgmap v835: 129 pgs: 129 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 3.6 KiB/s wr, 9 op/s 2024-04-24T09:18:02.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:18:02 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/711259358' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T09:18:02.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:18:02 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/2232833794' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T09:18:02.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:02.616+0000 7f2abea00640 1 -- 172.21.15.83:0/3025605232 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f2ab819bd40 con 0x7f2ab8104450 2024-04-24T09:18:02.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:02.617+0000 7f2ab4c00640 1 -- 172.21.15.83:0/3025605232 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v134) ==== 100+0+11 (secure 0 0 0) 0x7f2ab819bd40 con 0x7f2ab8104450 2024-04-24T09:18:02.616 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2024-04-24T09:18:02.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:02.619+0000 7f2abea00640 1 -- 172.21.15.83:0/3025605232 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f2a9c0a3d20 msgr2=0x7f2a9c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:02.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:02.619+0000 7f2abea00640 1 --2- 172.21.15.83:0/3025605232 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f2a9c0a3d20 0x7f2a9c0a61e0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f2ab810aad0 tx=0x7f2aa0005f90 comp rx=0 tx=0).stop 2024-04-24T09:18:02.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:02.620+0000 7f2abea00640 1 -- 172.21.15.83:0/3025605232 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ab8104450 msgr2=0x7f2ab81a06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:02.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:02.620+0000 7f2abea00640 1 --2- 172.21.15.83:0/3025605232 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ab8104450 0x7f2ab81a06a0 secure :-1 s=READY pgs=2902 cs=0 l=1 rev1=1 crypto rx=0x7f2aa800d770 tx=0x7f2aa800dc30 comp rx=0 tx=0).stop 2024-04-24T09:18:02.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:02.620+0000 7f2abea00640 1 -- 172.21.15.83:0/3025605232 shutdown_connections 2024-04-24T09:18:02.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:02.620+0000 7f2abea00640 1 --2- 172.21.15.83:0/3025605232 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f2a9c0a3d20 0x7f2a9c0a61e0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:02.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:02.620+0000 7f2abea00640 1 --2- 172.21.15.83:0/3025605232 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2ab819a810 0x7f2ab819acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:02.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:02.620+0000 7f2abea00640 1 --2- 172.21.15.83:0/3025605232 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ab8109520 0x7f2ab81a0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:02.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:02.620+0000 7f2abea00640 1 --2- 172.21.15.83:0/3025605232 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ab8104450 0x7f2ab81a06a0 unknown :-1 s=CLOSED pgs=2902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:02.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:02.620+0000 7f2abea00640 1 -- 172.21.15.83:0/3025605232 >> 172.21.15.83:0/3025605232 conn(0x7f2ab80ffc70 msgr2=0x7f2ab8100fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:18:02.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:02.620+0000 7f2abea00640 1 -- 172.21.15.83:0/3025605232 shutdown_connections 2024-04-24T09:18:02.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:02.620+0000 7f2abea00640 1 -- 172.21.15.83:0/3025605232 wait complete. 2024-04-24T09:18:02.637 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-24T09:18:02.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:18:02 smithi138 ceph-mon[31441]: pgmap v835: 129 pgs: 129 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 3.6 KiB/s wr, 9 op/s 2024-04-24T09:18:02.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:18:02 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/711259358' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T09:18:02.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:18:02 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/2232833794' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T09:18:02.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:18:02 smithi189 ceph-mon[31791]: pgmap v835: 129 pgs: 129 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 3.6 KiB/s wr, 9 op/s 2024-04-24T09:18:02.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:18:02 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/711259358' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T09:18:02.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:18:02 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/2232833794' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T09:18:02.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:02.845+0000 7f0d95800640 1 -- 172.21.15.83:0/1786267789 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d90101990 msgr2=0x7f0d90105e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:02.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:02.845+0000 7f0d95800640 1 --2- 172.21.15.83:0/1786267789 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d90101990 0x7f0d90105e50 secure :-1 s=READY pgs=2903 cs=0 l=1 rev1=1 crypto rx=0x7f0d78009ca0 tx=0x7f0d7802a9e0 comp rx=0 tx=0).stop 2024-04-24T09:18:02.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:02.845+0000 7f0d95800640 1 -- 172.21.15.83:0/1786267789 shutdown_connections 2024-04-24T09:18:02.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:02.845+0000 7f0d95800640 1 --2- 172.21.15.83:0/1786267789 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0d90106390 0x7f0d90106820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:02.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:02.845+0000 7f0d95800640 1 --2- 172.21.15.83:0/1786267789 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d90101990 0x7f0d90105e50 unknown :-1 s=CLOSED pgs=2903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:02.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:02.845+0000 7f0d95800640 1 --2- 172.21.15.83:0/1786267789 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0d90101070 0x7f0d90101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:02.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:02.845+0000 7f0d95800640 1 -- 172.21.15.83:0/1786267789 >> 172.21.15.83:0/1786267789 conn(0x7f0d90078000 msgr2=0x7f0d90078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:18:02.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:02.846+0000 7f0d95800640 1 -- 172.21.15.83:0/1786267789 shutdown_connections 2024-04-24T09:18:02.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:02.846+0000 7f0d95800640 1 -- 172.21.15.83:0/1786267789 wait complete. 2024-04-24T09:18:02.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:02.847+0000 7f0d95800640 1 Processor -- start 2024-04-24T09:18:02.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:02.847+0000 7f0d95800640 1 -- start start 2024-04-24T09:18:02.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:02.847+0000 7f0d95800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d90101070 0x7f0d9019edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:02.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:02.847+0000 7f0d95800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0d90101990 0x7f0d9019f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:02.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:02.847+0000 7f0d95800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0d90106390 0x7f0d901a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:02.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:02.847+0000 7f0d95800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0d90118870 con 0x7f0d90101070 2024-04-24T09:18:02.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:02.847+0000 7f0d95800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0d901186f0 con 0x7f0d90106390 2024-04-24T09:18:02.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:02.847+0000 7f0d95800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0d901189f0 con 0x7f0d90101990 2024-04-24T09:18:02.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:02.848+0000 7f0d8ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d90101070 0x7f0d9019edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:02.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:02.848+0000 7f0d8ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d90101070 0x7f0d9019edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33276/0 (socket says 172.21.15.83:33276) 2024-04-24T09:18:02.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:02.848+0000 7f0d8ea00640 1 -- 172.21.15.83:0/2028765119 learned_addr learned my addr 172.21.15.83:0/2028765119 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:18:02.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:02.848+0000 7f0d8e000640 1 --2- 172.21.15.83:0/2028765119 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0d90101990 0x7f0d9019f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:02.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:02.848+0000 7f0d8f400640 1 --2- 172.21.15.83:0/2028765119 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0d90106390 0x7f0d901a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:02.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:02.848+0000 7f0d8ea00640 1 -- 172.21.15.83:0/2028765119 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0d90101990 msgr2=0x7f0d9019f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:02.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:02.848+0000 7f0d8ea00640 1 --2- 172.21.15.83:0/2028765119 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0d90101990 0x7f0d9019f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:02.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:02.848+0000 7f0d8ea00640 1 -- 172.21.15.83:0/2028765119 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0d90106390 msgr2=0x7f0d901a3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:02.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:02.848+0000 7f0d8ea00640 1 --2- 172.21.15.83:0/2028765119 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0d90106390 0x7f0d901a3690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:02.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:02.848+0000 7f0d8ea00640 1 -- 172.21.15.83:0/2028765119 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d901a3dc0 con 0x7f0d90101070 2024-04-24T09:18:02.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:02.848+0000 7f0d8ea00640 1 --2- 172.21.15.83:0/2028765119 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d90101070 0x7f0d9019edd0 secure :-1 s=READY pgs=2904 cs=0 l=1 rev1=1 crypto rx=0x7f0d8000d770 tx=0x7f0d8000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:18:02.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:02.849+0000 7f0d87400640 1 -- 172.21.15.83:0/2028765119 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d80017070 con 0x7f0d90101070 2024-04-24T09:18:02.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:02.849+0000 7f0d87400640 1 -- 172.21.15.83:0/2028765119 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0d800054d0 con 0x7f0d90101070 2024-04-24T09:18:02.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:02.849+0000 7f0d95800640 1 -- 172.21.15.83:0/2028765119 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d901a40b0 con 0x7f0d90101070 2024-04-24T09:18:02.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:02.849+0000 7f0d87400640 1 -- 172.21.15.83:0/2028765119 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d80004cf0 con 0x7f0d90101070 2024-04-24T09:18:02.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:02.849+0000 7f0d95800640 1 -- 172.21.15.83:0/2028765119 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d901ab940 con 0x7f0d90101070 2024-04-24T09:18:02.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:02.850+0000 7f0d95800640 1 -- 172.21.15.83:0/2028765119 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d90068e20 con 0x7f0d90101070 2024-04-24T09:18:02.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:02.852+0000 7f0d87400640 1 -- 172.21.15.83:0/2028765119 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0d80004e90 con 0x7f0d90101070 2024-04-24T09:18:02.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:02.853+0000 7f0d87400640 1 --2- 172.21.15.83:0/2028765119 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f0d700a3d20 0x7f0d700a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:02.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:02.854+0000 7f0d87400640 1 -- 172.21.15.83:0/2028765119 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 9583+0+0 (secure 0 0 0) 0x7f0d800d0750 con 0x7f0d90101070 2024-04-24T09:18:02.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:02.854+0000 7f0d8e000640 1 --2- 172.21.15.83:0/2028765119 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f0d700a3d20 0x7f0d700a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:02.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:02.854+0000 7f0d8e000640 1 --2- 172.21.15.83:0/2028765119 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f0d700a3d20 0x7f0d700a61e0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f0d78009880 tx=0x7f0d78005ef0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:18:02.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:02.858+0000 7f0d87400640 1 -- 172.21.15.83:0/2028765119 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0d8009c9d0 con 0x7f0d90101070 2024-04-24T09:18:03.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:03.062+0000 7f0d95800640 1 -- 172.21.15.83:0/2028765119 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f0d901abc00 con 0x7f0d90101070 2024-04-24T09:18:03.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:03.063+0000 7f0d87400640 1 -- 172.21.15.83:0/2028765119 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v134) ==== 103+0+11 (secure 0 0 0) 0x7f0d8009c2d0 con 0x7f0d90101070 2024-04-24T09:18:03.062 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-24T09:18:03.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:03.065+0000 7f0d95800640 1 -- 172.21.15.83:0/2028765119 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f0d700a3d20 msgr2=0x7f0d700a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:03.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:03.066+0000 7f0d95800640 1 --2- 172.21.15.83:0/2028765119 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f0d700a3d20 0x7f0d700a61e0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f0d78009880 tx=0x7f0d78005ef0 comp rx=0 tx=0).stop 2024-04-24T09:18:03.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:03.066+0000 7f0d95800640 1 -- 172.21.15.83:0/2028765119 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d90101070 msgr2=0x7f0d9019edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:03.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:03.066+0000 7f0d95800640 1 --2- 172.21.15.83:0/2028765119 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d90101070 0x7f0d9019edd0 secure :-1 s=READY pgs=2904 cs=0 l=1 rev1=1 crypto rx=0x7f0d8000d770 tx=0x7f0d8000dc30 comp rx=0 tx=0).stop 2024-04-24T09:18:03.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:03.066+0000 7f0d95800640 1 -- 172.21.15.83:0/2028765119 shutdown_connections 2024-04-24T09:18:03.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:03.066+0000 7f0d95800640 1 --2- 172.21.15.83:0/2028765119 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f0d700a3d20 0x7f0d700a61e0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:03.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:03.066+0000 7f0d95800640 1 --2- 172.21.15.83:0/2028765119 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0d90106390 0x7f0d901a3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:03.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:03.066+0000 7f0d95800640 1 --2- 172.21.15.83:0/2028765119 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0d90101990 0x7f0d9019f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:03.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:03.066+0000 7f0d95800640 1 --2- 172.21.15.83:0/2028765119 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d90101070 0x7f0d9019edd0 unknown :-1 s=CLOSED pgs=2904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:03.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:03.066+0000 7f0d95800640 1 -- 172.21.15.83:0/2028765119 >> 172.21.15.83:0/2028765119 conn(0x7f0d90078000 msgr2=0x7f0d90103e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:18:03.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:03.066+0000 7f0d95800640 1 -- 172.21.15.83:0/2028765119 shutdown_connections 2024-04-24T09:18:03.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:03.066+0000 7f0d95800640 1 -- 172.21.15.83:0/2028765119 wait complete. 2024-04-24T09:18:03.084 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-04-24T09:18:03.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:03.283+0000 7f1db6c00640 1 -- 172.21.15.83:0/525018957 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1db010d3c0 msgr2=0x7f1db010d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:03.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:03.283+0000 7f1db6c00640 1 --2- 172.21.15.83:0/525018957 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1db010d3c0 0x7f1db010d7a0 secure :-1 s=READY pgs=2905 cs=0 l=1 rev1=1 crypto rx=0x7f1da4009900 tx=0x7f1da402a480 comp rx=0 tx=0).stop 2024-04-24T09:18:03.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:03.283+0000 7f1db6c00640 1 -- 172.21.15.83:0/525018957 shutdown_connections 2024-04-24T09:18:03.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:03.283+0000 7f1db6c00640 1 --2- 172.21.15.83:0/525018957 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1db0108e40 0x7f1db01152b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:03.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:03.283+0000 7f1db6c00640 1 --2- 172.21.15.83:0/525018957 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1db01084a0 0x7f1db0108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:03.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:03.283+0000 7f1db6c00640 1 --2- 172.21.15.83:0/525018957 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1db010d3c0 0x7f1db010d7a0 unknown :-1 s=CLOSED pgs=2905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:03.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:03.283+0000 7f1db6c00640 1 -- 172.21.15.83:0/525018957 >> 172.21.15.83:0/525018957 conn(0x7f1db0077b80 msgr2=0x7f1db0079fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:18:03.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:03.283+0000 7f1db6c00640 1 -- 172.21.15.83:0/525018957 shutdown_connections 2024-04-24T09:18:03.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:03.283+0000 7f1db6c00640 1 -- 172.21.15.83:0/525018957 wait complete. 2024-04-24T09:18:03.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:03.284+0000 7f1db6c00640 1 Processor -- start 2024-04-24T09:18:03.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:03.284+0000 7f1db6c00640 1 -- start start 2024-04-24T09:18:03.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:03.285+0000 7f1db6c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1db01084a0 0x7f1db01a4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:03.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:03.285+0000 7f1db6c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1db0108e40 0x7f1db01a4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:03.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:03.285+0000 7f1db6c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1db010d3c0 0x7f1db019ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:03.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:03.285+0000 7f1db6c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1db01186b0 con 0x7f1db01084a0 2024-04-24T09:18:03.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:03.285+0000 7f1db6c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1db0118530 con 0x7f1db0108e40 2024-04-24T09:18:03.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:03.285+0000 7f1db6c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1db0118830 con 0x7f1db010d3c0 2024-04-24T09:18:03.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:03.285+0000 7f1db5800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1db01084a0 0x7f1db01a4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:03.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:03.285+0000 7f1db5800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1db01084a0 0x7f1db01a4850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33302/0 (socket says 172.21.15.83:33302) 2024-04-24T09:18:03.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:03.285+0000 7f1db5800640 1 -- 172.21.15.83:0/3546085317 learned_addr learned my addr 172.21.15.83:0/3546085317 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:18:03.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:03.285+0000 7f1db4e00640 1 --2- 172.21.15.83:0/3546085317 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1db0108e40 0x7f1db01a4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:03.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:03.285+0000 7f1db6200640 1 --2- 172.21.15.83:0/3546085317 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1db010d3c0 0x7f1db019ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:03.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:03.285+0000 7f1db5800640 1 -- 172.21.15.83:0/3546085317 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1db010d3c0 msgr2=0x7f1db019ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:03.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:03.286+0000 7f1db5800640 1 --2- 172.21.15.83:0/3546085317 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1db010d3c0 0x7f1db019ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:03.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:03.286+0000 7f1db5800640 1 -- 172.21.15.83:0/3546085317 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1db0108e40 msgr2=0x7f1db01a4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:03.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:03.286+0000 7f1db5800640 1 --2- 172.21.15.83:0/3546085317 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1db0108e40 0x7f1db01a4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:03.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:03.286+0000 7f1db5800640 1 -- 172.21.15.83:0/3546085317 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1db019f1d0 con 0x7f1db01084a0 2024-04-24T09:18:03.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:03.286+0000 7f1db4e00640 1 --2- 172.21.15.83:0/3546085317 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1db0108e40 0x7f1db01a4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:18:03.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:03.286+0000 7f1db6200640 1 --2- 172.21.15.83:0/3546085317 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1db010d3c0 0x7f1db019ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:18:03.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:03.286+0000 7f1db5800640 1 --2- 172.21.15.83:0/3546085317 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1db01084a0 0x7f1db01a4850 secure :-1 s=READY pgs=2906 cs=0 l=1 rev1=1 crypto rx=0x7f1da4005940 tx=0x7f1da4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:18:03.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:03.286+0000 7f1dae000640 1 -- 172.21.15.83:0/3546085317 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da403f030 con 0x7f1db01084a0 2024-04-24T09:18:03.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:03.286+0000 7f1dae000640 1 -- 172.21.15.83:0/3546085317 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1da4003aa0 con 0x7f1db01084a0 2024-04-24T09:18:03.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:03.286+0000 7f1db6c00640 1 -- 172.21.15.83:0/3546085317 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1db019f460 con 0x7f1db01084a0 2024-04-24T09:18:03.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:03.286+0000 7f1dae000640 1 -- 172.21.15.83:0/3546085317 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da4035070 con 0x7f1db01084a0 2024-04-24T09:18:03.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:03.286+0000 7f1db6c00640 1 -- 172.21.15.83:0/3546085317 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1db01ab710 con 0x7f1db01084a0 2024-04-24T09:18:03.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:03.287+0000 7f1db6c00640 1 -- 172.21.15.83:0/3546085317 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1db0069190 con 0x7f1db01084a0 2024-04-24T09:18:03.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:03.289+0000 7f1dae000640 1 -- 172.21.15.83:0/3546085317 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1da4042030 con 0x7f1db01084a0 2024-04-24T09:18:03.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:03.291+0000 7f1dae000640 1 --2- 172.21.15.83:0/3546085317 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f1d980a3d20 0x7f1d980a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:03.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:03.291+0000 7f1dae000640 1 -- 172.21.15.83:0/3546085317 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 9583+0+0 (secure 0 0 0) 0x7f1da40f14b0 con 0x7f1db01084a0 2024-04-24T09:18:03.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:03.291+0000 7f1db4e00640 1 --2- 172.21.15.83:0/3546085317 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f1d980a3d20 0x7f1d980a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:03.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:03.292+0000 7f1db4e00640 1 --2- 172.21.15.83:0/3546085317 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f1d980a3d20 0x7f1d980a61e0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f1db007a370 tx=0x7f1d9c005e50 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:18:03.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:03.295+0000 7f1dae000640 1 -- 172.21.15.83:0/3546085317 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1da40bd6d0 con 0x7f1db01084a0 2024-04-24T09:18:03.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:03.551+0000 7f1db6c00640 1 -- 172.21.15.83:0/3546085317 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f1db0110960 con 0x7f1db01084a0 2024-04-24T09:18:03.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:03.569+0000 7f1dae000640 1 -- 172.21.15.83:0/3546085317 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v26) ==== 81+0+746061 (secure 0 0 0) 0x7f1da40bcfd0 con 0x7f1db01084a0 2024-04-24T09:18:03.580 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T09:18:03.580 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T09:18:03.580 INFO:teuthology.orchestra.run.smithi083.stdout: "epoch": 26, 2024-04-24T09:18:03.580 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.580 INFO:teuthology.orchestra.run.smithi083.stdout: "active_gid": 15579, 2024-04-24T09:18:03.580 INFO:teuthology.orchestra.run.smithi083.stdout: "active_name": "x", 2024-04-24T09:18:03.580 INFO:teuthology.orchestra.run.smithi083.stdout: "active_addrs": { 2024-04-24T09:18:03.580 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-04-24T09:18:03.581 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:03.581 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-04-24T09:18:03.581 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6800", 2024-04-24T09:18:03.581 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 3594769126 2024-04-24T09:18:03.581 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.581 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:03.581 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-04-24T09:18:03.581 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6801", 2024-04-24T09:18:03.581 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 3594769126 2024-04-24T09:18:03.581 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:03.581 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-04-24T09:18:03.581 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.581 INFO:teuthology.orchestra.run.smithi083.stdout: "active_addr": "172.21.15.83:6801/3594769126", 2024-04-24T09:18:03.581 INFO:teuthology.orchestra.run.smithi083.stdout: "active_change": "2024-04-24T08:51:36.348358+0000", 2024-04-24T09:18:03.582 INFO:teuthology.orchestra.run.smithi083.stdout: "active_mgr_features": 4540701547738038271, 2024-04-24T09:18:03.582 INFO:teuthology.orchestra.run.smithi083.stdout: "available": true, 2024-04-24T09:18:03.582 INFO:teuthology.orchestra.run.smithi083.stdout: "standbys": [ 2024-04-24T09:18:03.582 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:03.582 INFO:teuthology.orchestra.run.smithi083.stdout: "gid": 24991, 2024-04-24T09:18:03.582 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "z", 2024-04-24T09:18:03.582 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr_features": 4540701547738038271, 2024-04-24T09:18:03.582 INFO:teuthology.orchestra.run.smithi083.stdout: "available_modules": [ 2024-04-24T09:18:03.582 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:03.582 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "alerts", 2024-04-24T09:18:03.582 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-24T09:18:03.582 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-24T09:18:03.582 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-24T09:18:03.582 INFO:teuthology.orchestra.run.smithi083.stdout: "interval": { 2024-04-24T09:18:03.583 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "interval", 2024-04-24T09:18:03.583 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-24T09:18:03.583 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.583 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.583 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "60", 2024-04-24T09:18:03.583 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.583 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.583 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.583 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "How frequently to reexamine health status", 2024-04-24T09:18:03.583 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.583 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.583 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.583 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.583 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-24T09:18:03.583 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-24T09:18:03.583 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.584 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.584 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.584 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:03.584 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.584 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.584 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:03.584 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:03.584 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:03.584 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:03.584 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:03.584 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:03.584 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:03.584 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:03.585 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.585 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.585 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.585 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.585 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.585 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-24T09:18:03.585 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-24T09:18:03.585 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.585 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.585 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.585 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:03.585 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.585 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.585 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.586 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.586 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.586 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.586 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.586 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.586 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-24T09:18:03.586 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-24T09:18:03.586 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.586 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.586 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.586 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-24T09:18:03.586 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.586 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.586 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:03.587 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:03.587 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:03.587 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:03.587 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:03.587 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:03.587 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:03.587 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:03.587 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.587 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.587 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.587 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.587 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.587 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-24T09:18:03.587 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-24T09:18:03.587 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.587 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.588 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.588 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:03.588 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.588 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.588 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.588 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.588 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.588 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.588 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.588 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.588 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_destination": { 2024-04-24T09:18:03.588 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_destination", 2024-04-24T09:18:03.588 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.588 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.588 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.588 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:03.588 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.589 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.589 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.589 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Email address to send alerts to", 2024-04-24T09:18:03.589 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.589 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.589 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.589 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.589 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_from_name": { 2024-04-24T09:18:03.589 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_from_name", 2024-04-24T09:18:03.589 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.589 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.589 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.589 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "Ceph", 2024-04-24T09:18:03.589 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.589 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.590 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.590 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Email From: name", 2024-04-24T09:18:03.590 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.590 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.590 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.590 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.590 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_host": { 2024-04-24T09:18:03.590 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_host", 2024-04-24T09:18:03.590 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.590 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.590 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.590 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:03.590 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.590 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.591 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.591 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "SMTP server", 2024-04-24T09:18:03.591 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.591 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.591 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.591 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.591 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_password": { 2024-04-24T09:18:03.591 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_password", 2024-04-24T09:18:03.591 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.591 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.591 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.591 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:03.591 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.591 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.592 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.592 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Password to authenticate with", 2024-04-24T09:18:03.592 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.592 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.592 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.592 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.592 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_port": { 2024-04-24T09:18:03.592 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_port", 2024-04-24T09:18:03.592 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:03.592 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.592 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.592 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "465", 2024-04-24T09:18:03.592 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.592 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.593 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.593 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "SMTP port", 2024-04-24T09:18:03.593 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.593 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.593 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.593 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.593 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_sender": { 2024-04-24T09:18:03.593 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_sender", 2024-04-24T09:18:03.593 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.593 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.593 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.593 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:03.593 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.593 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.594 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.594 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "SMTP envelope sender", 2024-04-24T09:18:03.594 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.594 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.594 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.594 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.594 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_ssl": { 2024-04-24T09:18:03.594 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_ssl", 2024-04-24T09:18:03.594 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.594 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.594 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.594 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:03.594 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.595 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.595 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.595 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-24T09:18:03.595 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.595 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.595 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.595 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.595 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_user": { 2024-04-24T09:18:03.595 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_user", 2024-04-24T09:18:03.595 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.595 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.595 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.595 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:03.595 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.595 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.596 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.596 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "User to authenticate as", 2024-04-24T09:18:03.596 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.596 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.596 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.596 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:03.596 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:03.596 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.596 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:03.596 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "balancer", 2024-04-24T09:18:03.596 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-24T09:18:03.596 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-24T09:18:03.596 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-24T09:18:03.596 INFO:teuthology.orchestra.run.smithi083.stdout: "active": { 2024-04-24T09:18:03.596 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "active", 2024-04-24T09:18:03.596 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.597 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.597 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.597 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:03.597 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.597 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.597 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.597 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "automatically balance PGs across cluster", 2024-04-24T09:18:03.597 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.597 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.597 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.597 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.597 INFO:teuthology.orchestra.run.smithi083.stdout: "begin_time": { 2024-04-24T09:18:03.597 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "begin_time", 2024-04-24T09:18:03.597 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.597 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.598 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.598 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0000", 2024-04-24T09:18:03.598 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.598 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.598 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.598 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "beginning time of day to automatically balance", 2024-04-24T09:18:03.598 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-24T09:18:03.598 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.598 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.598 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.598 INFO:teuthology.orchestra.run.smithi083.stdout: "begin_weekday": { 2024-04-24T09:18:03.598 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "begin_weekday", 2024-04-24T09:18:03.599 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "uint", 2024-04-24T09:18:03.599 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.599 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.599 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-04-24T09:18:03.599 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "0", 2024-04-24T09:18:03.599 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "6", 2024-04-24T09:18:03.599 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.599 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-24T09:18:03.599 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-24T09:18:03.599 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.599 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.599 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.599 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_compat_max_iterations": { 2024-04-24T09:18:03.599 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "crush_compat_max_iterations", 2024-04-24T09:18:03.599 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "uint", 2024-04-24T09:18:03.600 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.600 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.600 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "25", 2024-04-24T09:18:03.600 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "1", 2024-04-24T09:18:03.600 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "250", 2024-04-24T09:18:03.600 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.600 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-24T09:18:03.600 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.600 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.600 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.600 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.600 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_compat_metrics": { 2024-04-24T09:18:03.600 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "crush_compat_metrics", 2024-04-24T09:18:03.601 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.601 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.601 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.601 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "pgs,objects,bytes", 2024-04-24T09:18:03.601 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.601 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.601 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.601 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-24T09:18:03.601 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-24T09:18:03.601 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.601 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.601 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.601 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_compat_step": { 2024-04-24T09:18:03.601 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "crush_compat_step", 2024-04-24T09:18:03.601 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-04-24T09:18:03.602 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.602 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.602 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0.5", 2024-04-24T09:18:03.602 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "0.001", 2024-04-24T09:18:03.602 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "0.999", 2024-04-24T09:18:03.602 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.602 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "aggressiveness of optimization", 2024-04-24T09:18:03.602 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-24T09:18:03.602 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.602 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.602 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.602 INFO:teuthology.orchestra.run.smithi083.stdout: "end_time": { 2024-04-24T09:18:03.602 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "end_time", 2024-04-24T09:18:03.602 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.603 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.603 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.603 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "2359", 2024-04-24T09:18:03.603 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.603 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.603 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.603 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "ending time of day to automatically balance", 2024-04-24T09:18:03.603 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-24T09:18:03.603 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.603 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.603 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.603 INFO:teuthology.orchestra.run.smithi083.stdout: "end_weekday": { 2024-04-24T09:18:03.603 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "end_weekday", 2024-04-24T09:18:03.603 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "uint", 2024-04-24T09:18:03.603 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.604 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.604 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-04-24T09:18:03.604 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "0", 2024-04-24T09:18:03.604 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "6", 2024-04-24T09:18:03.604 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.604 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-24T09:18:03.604 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-24T09:18:03.604 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.604 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.604 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.604 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-24T09:18:03.604 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-24T09:18:03.604 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.604 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.604 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.605 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:03.605 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.605 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.605 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:03.605 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:03.605 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:03.605 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:03.605 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:03.605 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:03.605 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:03.605 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:03.605 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.605 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.605 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.606 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.606 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.606 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-24T09:18:03.606 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-24T09:18:03.606 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.606 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.606 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.606 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:03.606 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.606 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.606 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.606 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.606 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.607 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.607 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.607 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.607 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-24T09:18:03.607 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-24T09:18:03.607 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.607 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.607 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.607 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-24T09:18:03.607 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.607 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.607 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:03.607 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:03.607 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:03.608 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:03.608 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:03.608 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:03.608 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:03.608 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:03.608 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.608 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.608 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.608 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.608 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.608 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-24T09:18:03.608 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-24T09:18:03.608 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.608 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.608 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.609 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:03.609 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.609 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.609 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.609 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.609 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.609 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.609 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.609 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.609 INFO:teuthology.orchestra.run.smithi083.stdout: "min_score": { 2024-04-24T09:18:03.609 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "min_score", 2024-04-24T09:18:03.609 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-04-24T09:18:03.609 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.609 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.609 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-04-24T09:18:03.609 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.610 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.610 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.610 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-24T09:18:03.610 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.610 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.610 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.610 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.610 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": { 2024-04-24T09:18:03.610 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mode", 2024-04-24T09:18:03.610 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.610 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.610 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.610 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "upmap", 2024-04-24T09:18:03.611 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.611 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.611 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:03.611 INFO:teuthology.orchestra.run.smithi083.stdout: "crush-compat", 2024-04-24T09:18:03.611 INFO:teuthology.orchestra.run.smithi083.stdout: "none", 2024-04-24T09:18:03.611 INFO:teuthology.orchestra.run.smithi083.stdout: "read", 2024-04-24T09:18:03.611 INFO:teuthology.orchestra.run.smithi083.stdout: "upmap", 2024-04-24T09:18:03.611 INFO:teuthology.orchestra.run.smithi083.stdout: "upmap-read" 2024-04-24T09:18:03.611 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:03.611 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Balancer mode", 2024-04-24T09:18:03.611 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.611 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.611 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.611 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.612 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_ids": { 2024-04-24T09:18:03.612 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "pool_ids", 2024-04-24T09:18:03.612 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.612 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.612 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.612 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:03.612 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.612 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.612 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.612 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-24T09:18:03.612 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.612 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.612 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.612 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.612 INFO:teuthology.orchestra.run.smithi083.stdout: "sleep_interval": { 2024-04-24T09:18:03.613 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sleep_interval", 2024-04-24T09:18:03.613 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-24T09:18:03.613 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.613 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.613 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "60", 2024-04-24T09:18:03.613 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.613 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.613 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.613 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-24T09:18:03.613 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.613 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.613 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.613 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.613 INFO:teuthology.orchestra.run.smithi083.stdout: "upmap_max_deviation": { 2024-04-24T09:18:03.613 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "upmap_max_deviation", 2024-04-24T09:18:03.614 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:03.614 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.614 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.614 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "5", 2024-04-24T09:18:03.614 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "1", 2024-04-24T09:18:03.614 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.614 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.614 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-24T09:18:03.614 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-24T09:18:03.614 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.614 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.614 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.614 INFO:teuthology.orchestra.run.smithi083.stdout: "upmap_max_optimizations": { 2024-04-24T09:18:03.614 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "upmap_max_optimizations", 2024-04-24T09:18:03.615 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "uint", 2024-04-24T09:18:03.615 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.615 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.615 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-04-24T09:18:03.615 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.615 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.615 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.615 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-24T09:18:03.615 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.615 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.615 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.615 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:03.615 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:03.615 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.615 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:03.616 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "cephadm", 2024-04-24T09:18:03.616 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-24T09:18:03.616 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-24T09:18:03.616 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-24T09:18:03.616 INFO:teuthology.orchestra.run.smithi083.stdout: "agent_down_multiplier": { 2024-04-24T09:18:03.616 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "agent_down_multiplier", 2024-04-24T09:18:03.616 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-04-24T09:18:03.616 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.616 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.616 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "3.0", 2024-04-24T09:18:03.616 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.616 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.616 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.616 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-24T09:18:03.617 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.617 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.617 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.617 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.617 INFO:teuthology.orchestra.run.smithi083.stdout: "agent_refresh_rate": { 2024-04-24T09:18:03.617 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "agent_refresh_rate", 2024-04-24T09:18:03.617 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-24T09:18:03.617 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.617 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.617 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "20", 2024-04-24T09:18:03.617 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.617 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.617 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.617 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-24T09:18:03.617 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.618 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.618 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.618 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.618 INFO:teuthology.orchestra.run.smithi083.stdout: "agent_starting_port": { 2024-04-24T09:18:03.618 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "agent_starting_port", 2024-04-24T09:18:03.618 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:03.618 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.618 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.618 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "4721", 2024-04-24T09:18:03.618 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.618 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.618 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.618 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-24T09:18:03.618 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.619 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.619 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.619 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.619 INFO:teuthology.orchestra.run.smithi083.stdout: "allow_ptrace": { 2024-04-24T09:18:03.619 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "allow_ptrace", 2024-04-24T09:18:03.619 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.619 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.619 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.619 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:03.619 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.619 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.619 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.619 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-24T09:18:03.619 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-24T09:18:03.620 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.620 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.620 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.620 INFO:teuthology.orchestra.run.smithi083.stdout: "autotune_interval": { 2024-04-24T09:18:03.620 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "autotune_interval", 2024-04-24T09:18:03.620 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-24T09:18:03.620 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.620 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.620 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "600", 2024-04-24T09:18:03.620 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.620 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.620 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.620 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-24T09:18:03.621 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.621 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.621 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.621 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.621 INFO:teuthology.orchestra.run.smithi083.stdout: "autotune_memory_target_ratio": { 2024-04-24T09:18:03.621 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "autotune_memory_target_ratio", 2024-04-24T09:18:03.621 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-04-24T09:18:03.621 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.621 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.621 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0.7", 2024-04-24T09:18:03.621 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.621 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.621 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.621 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-24T09:18:03.622 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.622 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.622 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.622 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.622 INFO:teuthology.orchestra.run.smithi083.stdout: "cephadm_log_destination": { 2024-04-24T09:18:03.622 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "cephadm_log_destination", 2024-04-24T09:18:03.622 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.622 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.622 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.622 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:03.622 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.622 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.622 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:03.622 INFO:teuthology.orchestra.run.smithi083.stdout: "file", 2024-04-24T09:18:03.623 INFO:teuthology.orchestra.run.smithi083.stdout: "file,syslog", 2024-04-24T09:18:03.623 INFO:teuthology.orchestra.run.smithi083.stdout: "syslog" 2024-04-24T09:18:03.623 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:03.623 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-24T09:18:03.623 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.623 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.623 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.623 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.623 INFO:teuthology.orchestra.run.smithi083.stdout: "cgroups_split": { 2024-04-24T09:18:03.623 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "cgroups_split", 2024-04-24T09:18:03.623 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.623 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.623 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.623 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:03.623 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.624 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.624 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.624 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-24T09:18:03.624 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.624 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.624 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.624 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.624 INFO:teuthology.orchestra.run.smithi083.stdout: "config_checks_enabled": { 2024-04-24T09:18:03.624 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "config_checks_enabled", 2024-04-24T09:18:03.624 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.624 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.624 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.624 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:03.624 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.624 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.624 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.625 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-24T09:18:03.625 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.625 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.625 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.625 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.625 INFO:teuthology.orchestra.run.smithi083.stdout: "config_dashboard": { 2024-04-24T09:18:03.625 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "config_dashboard", 2024-04-24T09:18:03.625 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.625 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.625 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.625 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:03.625 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.625 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.625 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.625 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-24T09:18:03.625 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.626 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.626 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.626 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.626 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_alertmanager": { 2024-04-24T09:18:03.626 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_alertmanager", 2024-04-24T09:18:03.626 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.626 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.626 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.626 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-24T09:18:03.626 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.626 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.626 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.626 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Prometheus container image", 2024-04-24T09:18:03.626 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.626 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.627 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.627 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.627 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_base": { 2024-04-24T09:18:03.627 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_base", 2024-04-24T09:18:03.627 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.627 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.627 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.627 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-24T09:18:03.627 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.627 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.627 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.627 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Container image name, without the tag", 2024-04-24T09:18:03.627 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.627 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.627 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.627 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.628 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_elasticsearch": { 2024-04-24T09:18:03.628 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_elasticsearch", 2024-04-24T09:18:03.628 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.628 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.628 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.628 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-24T09:18:03.628 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.628 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.628 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.628 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "elasticsearch container image", 2024-04-24T09:18:03.628 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.628 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.628 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.628 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.628 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_grafana": { 2024-04-24T09:18:03.628 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_grafana", 2024-04-24T09:18:03.629 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.629 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.629 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.629 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-24T09:18:03.629 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.629 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.629 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.629 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Prometheus container image", 2024-04-24T09:18:03.629 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.629 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.629 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.629 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.629 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_haproxy": { 2024-04-24T09:18:03.629 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_haproxy", 2024-04-24T09:18:03.629 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.629 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.629 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.630 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-24T09:18:03.630 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.630 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.630 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.630 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "HAproxy container image", 2024-04-24T09:18:03.630 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.630 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.630 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.630 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.630 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_jaeger_agent": { 2024-04-24T09:18:03.630 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_jaeger_agent", 2024-04-24T09:18:03.630 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.630 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.631 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.631 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-24T09:18:03.631 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.631 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.631 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.631 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Jaeger agent container image", 2024-04-24T09:18:03.631 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.631 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.631 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.631 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.631 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_jaeger_collector": { 2024-04-24T09:18:03.631 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_jaeger_collector", 2024-04-24T09:18:03.631 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.631 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.631 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.632 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-24T09:18:03.632 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.632 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.632 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.632 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Jaeger collector container image", 2024-04-24T09:18:03.632 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.632 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.632 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.632 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.632 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_jaeger_query": { 2024-04-24T09:18:03.632 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_jaeger_query", 2024-04-24T09:18:03.632 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.632 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.632 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.632 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-24T09:18:03.632 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.633 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.633 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.633 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Jaeger query container image", 2024-04-24T09:18:03.633 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.633 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.633 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.633 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.633 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_keepalived": { 2024-04-24T09:18:03.633 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_keepalived", 2024-04-24T09:18:03.633 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.633 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.633 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.633 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-24T09:18:03.633 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.633 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.634 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.634 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Keepalived container image", 2024-04-24T09:18:03.634 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.634 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.634 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.634 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.634 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_loki": { 2024-04-24T09:18:03.634 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_loki", 2024-04-24T09:18:03.634 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.634 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.634 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.634 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-24T09:18:03.634 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.634 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.634 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.635 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Loki container image", 2024-04-24T09:18:03.635 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.635 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.635 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.635 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.635 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_node_exporter": { 2024-04-24T09:18:03.635 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_node_exporter", 2024-04-24T09:18:03.635 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.635 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.635 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.635 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-24T09:18:03.635 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.635 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.635 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.636 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Prometheus container image", 2024-04-24T09:18:03.636 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.636 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.636 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.636 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.636 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_nvmeof": { 2024-04-24T09:18:03.636 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_nvmeof", 2024-04-24T09:18:03.636 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.636 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.636 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.636 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-24T09:18:03.636 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.636 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.636 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.636 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Nvme-of container image", 2024-04-24T09:18:03.637 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.637 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.637 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.637 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.637 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_prometheus": { 2024-04-24T09:18:03.637 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_prometheus", 2024-04-24T09:18:03.637 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.637 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.637 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.637 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-24T09:18:03.637 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.637 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.637 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.637 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Prometheus container image", 2024-04-24T09:18:03.637 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.637 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.638 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.638 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.638 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_promtail": { 2024-04-24T09:18:03.638 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_promtail", 2024-04-24T09:18:03.638 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.638 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.638 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.638 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-24T09:18:03.638 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.638 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.638 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.638 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Promtail container image", 2024-04-24T09:18:03.638 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.638 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.638 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.639 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.639 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_samba": { 2024-04-24T09:18:03.639 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_samba", 2024-04-24T09:18:03.639 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.639 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.639 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.639 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-24T09:18:03.639 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.639 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.639 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.639 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Samba/SMB container image", 2024-04-24T09:18:03.639 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.639 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.639 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.639 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.639 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_snmp_gateway": { 2024-04-24T09:18:03.640 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_snmp_gateway", 2024-04-24T09:18:03.640 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.640 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.640 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.640 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-24T09:18:03.640 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.640 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.640 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.640 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "SNMP Gateway container image", 2024-04-24T09:18:03.640 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.640 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.640 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.640 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.640 INFO:teuthology.orchestra.run.smithi083.stdout: "container_init": { 2024-04-24T09:18:03.641 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_init", 2024-04-24T09:18:03.641 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.641 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.641 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.641 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:03.641 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.641 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.641 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.641 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Run podman/docker with `--init`", 2024-04-24T09:18:03.641 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.641 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.641 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.641 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.641 INFO:teuthology.orchestra.run.smithi083.stdout: "daemon_cache_timeout": { 2024-04-24T09:18:03.641 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "daemon_cache_timeout", 2024-04-24T09:18:03.642 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-24T09:18:03.642 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.642 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.642 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "600", 2024-04-24T09:18:03.642 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.642 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.642 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.642 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-24T09:18:03.642 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.642 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.642 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.642 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.642 INFO:teuthology.orchestra.run.smithi083.stdout: "default_cephadm_command_timeout": { 2024-04-24T09:18:03.642 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "default_cephadm_command_timeout", 2024-04-24T09:18:03.643 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:03.643 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.643 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.643 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "900", 2024-04-24T09:18:03.643 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.643 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.643 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.643 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-24T09:18:03.643 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.643 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.643 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.643 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.643 INFO:teuthology.orchestra.run.smithi083.stdout: "default_registry": { 2024-04-24T09:18:03.643 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "default_registry", 2024-04-24T09:18:03.643 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.643 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.644 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.644 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "docker.io", 2024-04-24T09:18:03.644 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.644 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.644 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.644 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-24T09:18:03.644 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.644 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.644 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.644 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.644 INFO:teuthology.orchestra.run.smithi083.stdout: "device_cache_timeout": { 2024-04-24T09:18:03.644 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "device_cache_timeout", 2024-04-24T09:18:03.644 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-24T09:18:03.644 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.644 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.645 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "1800", 2024-04-24T09:18:03.645 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.645 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.645 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.645 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "seconds to cache device inventory", 2024-04-24T09:18:03.645 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.645 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.645 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.645 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.645 INFO:teuthology.orchestra.run.smithi083.stdout: "device_enhanced_scan": { 2024-04-24T09:18:03.645 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "device_enhanced_scan", 2024-04-24T09:18:03.645 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.645 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.645 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.645 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:03.645 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.646 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.646 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.646 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-24T09:18:03.646 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.646 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.646 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.646 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.646 INFO:teuthology.orchestra.run.smithi083.stdout: "facts_cache_timeout": { 2024-04-24T09:18:03.646 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "facts_cache_timeout", 2024-04-24T09:18:03.646 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-24T09:18:03.646 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.646 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.646 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "60", 2024-04-24T09:18:03.646 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.646 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.647 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.647 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "seconds to cache host facts data", 2024-04-24T09:18:03.647 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.647 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.647 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.647 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.647 INFO:teuthology.orchestra.run.smithi083.stdout: "grafana_dashboards_path": { 2024-04-24T09:18:03.647 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "grafana_dashboards_path", 2024-04-24T09:18:03.647 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.647 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.647 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.647 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-24T09:18:03.647 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.647 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.647 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.648 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-24T09:18:03.648 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.648 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.648 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.648 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.648 INFO:teuthology.orchestra.run.smithi083.stdout: "host_check_interval": { 2024-04-24T09:18:03.648 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "host_check_interval", 2024-04-24T09:18:03.648 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-24T09:18:03.648 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.648 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.648 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "600", 2024-04-24T09:18:03.648 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.648 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.648 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.648 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how frequently to perform a host check", 2024-04-24T09:18:03.648 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.649 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.649 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.649 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.649 INFO:teuthology.orchestra.run.smithi083.stdout: "hw_monitoring": { 2024-04-24T09:18:03.649 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "hw_monitoring", 2024-04-24T09:18:03.649 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.649 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.649 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.649 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:03.649 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.649 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.649 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.649 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-24T09:18:03.649 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.649 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.649 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.650 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.650 INFO:teuthology.orchestra.run.smithi083.stdout: "inventory_list_all": { 2024-04-24T09:18:03.650 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "inventory_list_all", 2024-04-24T09:18:03.650 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.650 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.650 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.650 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:03.650 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.650 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.650 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.650 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-24T09:18:03.650 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.650 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.650 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.651 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.651 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-24T09:18:03.651 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-24T09:18:03.651 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.651 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.651 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.651 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:03.651 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.651 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.651 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:03.651 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:03.651 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:03.651 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:03.651 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:03.651 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:03.651 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:03.651 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:03.652 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.652 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.652 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.652 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.652 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.652 INFO:teuthology.orchestra.run.smithi083.stdout: "log_refresh_metadata": { 2024-04-24T09:18:03.652 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_refresh_metadata", 2024-04-24T09:18:03.652 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.652 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.652 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.652 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:03.652 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.652 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.652 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.653 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-24T09:18:03.653 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.653 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.653 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.653 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.653 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-24T09:18:03.653 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-24T09:18:03.653 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.653 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.653 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.653 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:03.653 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.653 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.653 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.653 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-24T09:18:03.653 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.654 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.654 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.654 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.654 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-24T09:18:03.654 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-24T09:18:03.654 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.654 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.654 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.654 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-24T09:18:03.654 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.654 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.654 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:03.654 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:03.654 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:03.655 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:03.655 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:03.655 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:03.655 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:03.655 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:03.655 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.655 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.655 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.655 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.655 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.655 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-24T09:18:03.655 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-24T09:18:03.655 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.655 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.655 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.655 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:03.656 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.656 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.656 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.656 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.656 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.656 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.656 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.656 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.656 INFO:teuthology.orchestra.run.smithi083.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-24T09:18:03.656 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-24T09:18:03.656 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.656 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.656 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.656 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:03.656 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.656 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.656 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.657 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-24T09:18:03.657 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.657 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.657 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.657 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.657 INFO:teuthology.orchestra.run.smithi083.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-24T09:18:03.657 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-24T09:18:03.657 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.657 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.657 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.657 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "*", 2024-04-24T09:18:03.657 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.657 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.657 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.657 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-24T09:18:03.657 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.658 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.658 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.658 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.658 INFO:teuthology.orchestra.run.smithi083.stdout: "max_count_per_host": { 2024-04-24T09:18:03.658 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "max_count_per_host", 2024-04-24T09:18:03.658 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:03.658 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.658 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.658 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-04-24T09:18:03.658 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.658 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.658 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.658 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "max number of daemons per service per host", 2024-04-24T09:18:03.658 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.659 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.659 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.659 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.659 INFO:teuthology.orchestra.run.smithi083.stdout: "max_osd_draining_count": { 2024-04-24T09:18:03.659 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "max_osd_draining_count", 2024-04-24T09:18:03.659 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:03.659 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.659 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.659 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-04-24T09:18:03.659 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.659 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.659 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.659 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-24T09:18:03.659 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.659 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.659 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.660 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.660 INFO:teuthology.orchestra.run.smithi083.stdout: "migration_current": { 2024-04-24T09:18:03.660 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "migration_current", 2024-04-24T09:18:03.660 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:03.660 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.660 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.660 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:03.660 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.660 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.660 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.660 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "internal - do not modify", 2024-04-24T09:18:03.660 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.660 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.660 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.660 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.660 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": { 2024-04-24T09:18:03.660 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mode", 2024-04-24T09:18:03.661 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.661 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.661 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.661 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "root", 2024-04-24T09:18:03.661 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.661 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.661 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:03.661 INFO:teuthology.orchestra.run.smithi083.stdout: "cephadm-package", 2024-04-24T09:18:03.661 INFO:teuthology.orchestra.run.smithi083.stdout: "root" 2024-04-24T09:18:03.661 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:03.661 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "mode for remote execution of cephadm", 2024-04-24T09:18:03.661 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.661 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.661 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.661 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.661 INFO:teuthology.orchestra.run.smithi083.stdout: "oob_default_addr": { 2024-04-24T09:18:03.662 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "oob_default_addr", 2024-04-24T09:18:03.662 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.662 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.662 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.662 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "169.254.1.1", 2024-04-24T09:18:03.662 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.662 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.662 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.662 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-24T09:18:03.662 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.662 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.662 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.662 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.662 INFO:teuthology.orchestra.run.smithi083.stdout: "prometheus_alerts_path": { 2024-04-24T09:18:03.663 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "prometheus_alerts_path", 2024-04-24T09:18:03.663 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.663 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.663 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.663 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-24T09:18:03.663 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.663 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.663 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.663 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-24T09:18:03.663 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.663 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.663 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.663 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.663 INFO:teuthology.orchestra.run.smithi083.stdout: "registry_insecure": { 2024-04-24T09:18:03.663 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "registry_insecure", 2024-04-24T09:18:03.664 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.664 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.664 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.664 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:03.664 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.664 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.664 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.664 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-24T09:18:03.664 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.664 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.664 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.664 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.664 INFO:teuthology.orchestra.run.smithi083.stdout: "registry_password": { 2024-04-24T09:18:03.664 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "registry_password", 2024-04-24T09:18:03.664 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.664 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.665 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.665 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:03.665 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.665 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.665 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.665 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-24T09:18:03.665 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.665 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.665 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.665 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.665 INFO:teuthology.orchestra.run.smithi083.stdout: "registry_url": { 2024-04-24T09:18:03.665 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "registry_url", 2024-04-24T09:18:03.665 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.665 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.665 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.665 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:03.666 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.666 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.666 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.666 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-24T09:18:03.666 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.666 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.666 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.666 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.666 INFO:teuthology.orchestra.run.smithi083.stdout: "registry_username": { 2024-04-24T09:18:03.666 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "registry_username", 2024-04-24T09:18:03.666 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.666 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.666 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.666 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:03.666 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.667 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.667 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.667 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-24T09:18:03.667 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.667 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.667 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.667 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.667 INFO:teuthology.orchestra.run.smithi083.stdout: "secure_monitoring_stack": { 2024-04-24T09:18:03.667 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "secure_monitoring_stack", 2024-04-24T09:18:03.667 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.667 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.667 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.667 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:03.667 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.667 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.667 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.668 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-24T09:18:03.668 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.668 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.668 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.668 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.668 INFO:teuthology.orchestra.run.smithi083.stdout: "service_discovery_port": { 2024-04-24T09:18:03.668 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "service_discovery_port", 2024-04-24T09:18:03.668 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:03.668 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.668 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.668 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "8765", 2024-04-24T09:18:03.668 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.668 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.668 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.668 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "cephadm service discovery port", 2024-04-24T09:18:03.669 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.669 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.669 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.669 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.669 INFO:teuthology.orchestra.run.smithi083.stdout: "ssh_config_file": { 2024-04-24T09:18:03.669 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ssh_config_file", 2024-04-24T09:18:03.669 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.669 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.669 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.669 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:03.669 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.669 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.669 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.670 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-24T09:18:03.670 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.670 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.670 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.670 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.670 INFO:teuthology.orchestra.run.smithi083.stdout: "use_agent": { 2024-04-24T09:18:03.670 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "use_agent", 2024-04-24T09:18:03.670 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.670 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.670 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.670 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:03.670 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.670 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.670 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.671 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-24T09:18:03.671 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.671 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.671 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.671 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.671 INFO:teuthology.orchestra.run.smithi083.stdout: "use_repo_digest": { 2024-04-24T09:18:03.671 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "use_repo_digest", 2024-04-24T09:18:03.671 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.671 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.671 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.671 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:03.671 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.671 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.671 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.671 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-24T09:18:03.671 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.672 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.672 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.672 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.672 INFO:teuthology.orchestra.run.smithi083.stdout: "warn_on_failed_host_check": { 2024-04-24T09:18:03.672 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "warn_on_failed_host_check", 2024-04-24T09:18:03.672 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.672 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.672 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.672 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:03.672 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.672 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.672 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.672 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "raise a health warning if the host check fails", 2024-04-24T09:18:03.672 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.672 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.673 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.673 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.673 INFO:teuthology.orchestra.run.smithi083.stdout: "warn_on_stray_daemons": { 2024-04-24T09:18:03.673 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "warn_on_stray_daemons", 2024-04-24T09:18:03.673 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.673 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.673 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.673 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:03.673 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.673 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.673 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.673 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-24T09:18:03.673 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.673 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.673 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.673 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.674 INFO:teuthology.orchestra.run.smithi083.stdout: "warn_on_stray_hosts": { 2024-04-24T09:18:03.674 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "warn_on_stray_hosts", 2024-04-24T09:18:03.674 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.674 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.674 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.674 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:03.674 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.674 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.674 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.674 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-24T09:18:03.674 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.674 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.674 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.674 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:03.674 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:03.675 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.675 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:03.675 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "crash", 2024-04-24T09:18:03.675 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-24T09:18:03.675 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-24T09:18:03.675 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-24T09:18:03.675 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-24T09:18:03.675 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-24T09:18:03.675 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.675 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.675 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.675 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:03.675 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.675 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.675 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:03.676 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:03.676 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:03.676 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:03.676 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:03.676 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:03.676 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:03.676 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:03.676 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.676 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.676 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.676 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.676 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.676 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-24T09:18:03.676 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-24T09:18:03.676 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.676 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.677 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.677 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:03.677 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.677 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.677 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.677 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.677 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.677 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.677 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.677 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.677 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-24T09:18:03.677 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-24T09:18:03.677 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.677 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.677 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.678 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-24T09:18:03.678 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.678 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.678 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:03.678 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:03.678 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:03.678 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:03.678 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:03.678 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:03.678 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:03.678 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:03.678 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.678 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.678 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.678 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.679 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.679 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-24T09:18:03.679 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-24T09:18:03.679 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.679 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.679 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.679 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:03.679 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.679 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.679 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.679 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.679 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.679 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.679 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.679 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.679 INFO:teuthology.orchestra.run.smithi083.stdout: "retain_interval": { 2024-04-24T09:18:03.679 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "retain_interval", 2024-04-24T09:18:03.680 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-24T09:18:03.680 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.680 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.680 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "31536000", 2024-04-24T09:18:03.680 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.680 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.680 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.680 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-24T09:18:03.680 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.680 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.680 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.680 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.680 INFO:teuthology.orchestra.run.smithi083.stdout: "warn_recent_interval": { 2024-04-24T09:18:03.680 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "warn_recent_interval", 2024-04-24T09:18:03.680 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-24T09:18:03.681 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.681 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.681 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "1209600", 2024-04-24T09:18:03.681 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.681 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.681 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.681 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-24T09:18:03.681 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.681 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.681 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.681 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:03.681 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:03.681 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.681 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:03.681 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "dashboard", 2024-04-24T09:18:03.681 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-24T09:18:03.682 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-24T09:18:03.682 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-24T09:18:03.682 INFO:teuthology.orchestra.run.smithi083.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-24T09:18:03.682 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-24T09:18:03.682 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:03.682 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.682 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.682 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-04-24T09:18:03.682 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.682 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.682 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.682 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.682 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.682 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.682 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.682 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.683 INFO:teuthology.orchestra.run.smithi083.stdout: "ALERTMANAGER_API_HOST": { 2024-04-24T09:18:03.683 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-24T09:18:03.683 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.683 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.683 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.683 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:03.683 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.683 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.683 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.683 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.683 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.683 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.683 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.683 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.683 INFO:teuthology.orchestra.run.smithi083.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-24T09:18:03.683 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-24T09:18:03.684 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.684 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.684 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.684 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:03.684 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.684 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.684 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.684 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.684 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.684 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.684 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.684 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.684 INFO:teuthology.orchestra.run.smithi083.stdout: "AUDIT_API_ENABLED": { 2024-04-24T09:18:03.684 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "AUDIT_API_ENABLED", 2024-04-24T09:18:03.684 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.684 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.685 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.685 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:03.685 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.685 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.685 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.685 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.685 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.685 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.685 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.685 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.685 INFO:teuthology.orchestra.run.smithi083.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-24T09:18:03.685 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-24T09:18:03.685 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.685 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.685 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.685 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:03.686 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.686 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.686 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.686 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.686 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.686 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.686 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.686 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.686 INFO:teuthology.orchestra.run.smithi083.stdout: "ENABLE_BROWSABLE_API": { 2024-04-24T09:18:03.686 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-24T09:18:03.686 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.686 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.686 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.687 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:03.687 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.687 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.687 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.687 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.687 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.687 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.687 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.687 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.687 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-24T09:18:03.687 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-24T09:18:03.687 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.687 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.688 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.688 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:03.688 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.688 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.688 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.688 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.688 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.688 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.688 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.688 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.688 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-24T09:18:03.688 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-24T09:18:03.688 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.688 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.688 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.689 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:03.689 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.689 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.689 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.689 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.689 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.689 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.689 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.689 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.689 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-24T09:18:03.689 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-24T09:18:03.689 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.689 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.689 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.689 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:03.690 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.690 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.690 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.690 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.690 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.690 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.690 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.690 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.690 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-24T09:18:03.690 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-24T09:18:03.690 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.690 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.691 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.691 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:03.691 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.691 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.691 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.691 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.691 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.691 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.691 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.691 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.691 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-24T09:18:03.691 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-24T09:18:03.691 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.691 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.691 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.691 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:03.692 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.692 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.692 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.692 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.692 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.692 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.692 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.692 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.692 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-24T09:18:03.692 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-24T09:18:03.692 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.692 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.692 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.692 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:03.692 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.692 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.693 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.693 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.693 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.693 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.693 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.693 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.693 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-24T09:18:03.693 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-24T09:18:03.693 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.693 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.693 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.693 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:03.693 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.693 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.693 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.693 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.694 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.694 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.694 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.694 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.694 INFO:teuthology.orchestra.run.smithi083.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-24T09:18:03.694 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-24T09:18:03.694 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.694 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.694 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.694 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:03.694 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.694 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.694 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.694 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.695 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.695 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.695 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.695 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.695 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_API_PASSWORD": { 2024-04-24T09:18:03.695 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-24T09:18:03.695 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.695 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.695 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.695 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "admin", 2024-04-24T09:18:03.695 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.695 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.695 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.695 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.695 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.695 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.696 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.696 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.696 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-24T09:18:03.696 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-24T09:18:03.696 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.696 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.696 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.696 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:03.696 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.696 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.696 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.696 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.696 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.696 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.696 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.696 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.697 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_API_URL": { 2024-04-24T09:18:03.697 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_API_URL", 2024-04-24T09:18:03.697 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.697 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.697 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.697 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:03.697 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.697 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.697 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.697 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.697 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.697 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.697 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.697 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.697 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_API_USERNAME": { 2024-04-24T09:18:03.697 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-24T09:18:03.698 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.698 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.698 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.698 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "admin", 2024-04-24T09:18:03.698 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.698 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.698 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.698 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.698 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.698 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.698 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.698 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.698 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-24T09:18:03.699 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-24T09:18:03.699 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.699 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.699 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.699 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:03.699 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.699 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.699 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.699 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.699 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.699 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.699 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.699 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.699 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-24T09:18:03.699 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-24T09:18:03.699 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.700 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.700 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.700 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:03.700 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.700 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.700 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.700 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.700 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.700 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.700 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.700 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.700 INFO:teuthology.orchestra.run.smithi083.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-24T09:18:03.700 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-24T09:18:03.700 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.701 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.701 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.701 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:03.701 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.701 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.701 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.701 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.701 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.701 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.701 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.701 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.701 INFO:teuthology.orchestra.run.smithi083.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-24T09:18:03.701 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-24T09:18:03.701 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.701 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.701 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.702 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:03.702 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.702 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.702 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.702 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.702 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.702 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.702 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.702 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.702 INFO:teuthology.orchestra.run.smithi083.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-24T09:18:03.702 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-24T09:18:03.702 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.702 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.702 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.702 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "[]", 2024-04-24T09:18:03.703 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.703 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.703 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.703 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.703 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.703 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.703 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.703 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.703 INFO:teuthology.orchestra.run.smithi083.stdout: "MULTICLUSTER_CONFIG": { 2024-04-24T09:18:03.703 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-24T09:18:03.703 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.703 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.703 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.703 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "{}", 2024-04-24T09:18:03.703 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.703 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.704 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.704 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.704 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.704 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.704 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.704 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.704 INFO:teuthology.orchestra.run.smithi083.stdout: "PROMETHEUS_API_HOST": { 2024-04-24T09:18:03.704 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-24T09:18:03.704 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.704 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.704 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.704 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:03.704 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.704 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.704 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.705 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.705 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.705 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.705 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.705 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.705 INFO:teuthology.orchestra.run.smithi083.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-24T09:18:03.705 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-24T09:18:03.705 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.705 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.705 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.705 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:03.705 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.705 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.705 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.705 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.705 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.706 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.706 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.706 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.706 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-24T09:18:03.706 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-24T09:18:03.706 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.706 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.706 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.706 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:03.706 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.706 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.706 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.706 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.707 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.707 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.707 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.707 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.707 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-24T09:18:03.707 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-24T09:18:03.707 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.707 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.707 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.707 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:03.707 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.707 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.707 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.707 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.707 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.708 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.708 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.708 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.708 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-24T09:18:03.708 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-24T09:18:03.708 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.708 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.708 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.708 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:03.708 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.708 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.708 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.708 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.708 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.708 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.709 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.709 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.709 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-24T09:18:03.709 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-24T09:18:03.709 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.709 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.709 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.709 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:03.709 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.709 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.709 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.709 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.709 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.709 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.709 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.710 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.710 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-24T09:18:03.710 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-24T09:18:03.710 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.710 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.710 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.710 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:03.710 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.710 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.710 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.710 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.710 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.710 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.711 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.711 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.711 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-24T09:18:03.711 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-24T09:18:03.711 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.711 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.711 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.711 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:03.711 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.711 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.711 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.711 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.711 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.711 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.711 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.712 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.712 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-24T09:18:03.712 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-24T09:18:03.712 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.712 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.712 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.712 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:03.712 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.712 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.712 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.712 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.712 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.712 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.712 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.713 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.713 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_ENABLED": { 2024-04-24T09:18:03.713 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-24T09:18:03.713 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.713 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.713 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.713 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:03.713 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.713 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.713 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.713 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.713 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.713 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.713 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.713 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.714 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-24T09:18:03.714 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-24T09:18:03.714 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.714 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.714 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.714 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-24T09:18:03.714 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.714 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.714 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.714 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.714 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.714 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.714 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.715 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.715 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-24T09:18:03.715 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-24T09:18:03.715 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:03.715 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.715 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.715 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-04-24T09:18:03.715 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.715 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.715 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.715 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.715 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.715 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.716 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.716 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.716 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-24T09:18:03.716 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-24T09:18:03.716 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:03.716 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.716 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.716 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "8", 2024-04-24T09:18:03.716 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.716 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.716 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.716 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.716 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.716 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.716 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.717 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.717 INFO:teuthology.orchestra.run.smithi083.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-24T09:18:03.717 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-24T09:18:03.717 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:03.717 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.717 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.717 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "45", 2024-04-24T09:18:03.717 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.717 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.717 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.717 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.717 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.717 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.717 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.717 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.717 INFO:teuthology.orchestra.run.smithi083.stdout: "RGW_API_ACCESS_KEY": { 2024-04-24T09:18:03.718 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-24T09:18:03.718 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.718 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.718 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.718 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:03.718 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.718 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.718 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.718 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.718 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.718 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.718 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.718 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.718 INFO:teuthology.orchestra.run.smithi083.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-24T09:18:03.718 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-24T09:18:03.719 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.719 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.719 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.719 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "admin", 2024-04-24T09:18:03.719 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.719 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.719 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.719 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.719 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.719 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.719 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.719 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.719 INFO:teuthology.orchestra.run.smithi083.stdout: "RGW_API_SECRET_KEY": { 2024-04-24T09:18:03.719 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-24T09:18:03.719 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.719 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.720 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.720 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:03.720 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.720 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.720 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.720 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.720 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.720 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.720 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.720 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.720 INFO:teuthology.orchestra.run.smithi083.stdout: "RGW_API_SSL_VERIFY": { 2024-04-24T09:18:03.720 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-24T09:18:03.720 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.720 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.720 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.721 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:03.721 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.721 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.721 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.721 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.721 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.721 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.721 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.721 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.721 INFO:teuthology.orchestra.run.smithi083.stdout: "UNSAFE_TLS_v1_2": { 2024-04-24T09:18:03.721 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-24T09:18:03.721 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.721 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.721 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.722 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:03.722 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.722 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.722 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.722 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.722 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.722 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.722 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.722 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.722 INFO:teuthology.orchestra.run.smithi083.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-24T09:18:03.722 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-24T09:18:03.722 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:03.722 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.722 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.722 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-04-24T09:18:03.723 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.723 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.723 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.723 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.723 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.723 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.723 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.723 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.723 INFO:teuthology.orchestra.run.smithi083.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-24T09:18:03.723 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-24T09:18:03.723 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:03.723 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.723 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.723 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-04-24T09:18:03.723 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.723 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.724 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.724 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.724 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.724 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.724 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.724 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.724 INFO:teuthology.orchestra.run.smithi083.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-24T09:18:03.724 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-24T09:18:03.724 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:03.724 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.724 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.724 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "5", 2024-04-24T09:18:03.724 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.724 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.724 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.725 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.725 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.725 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.725 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.725 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.725 INFO:teuthology.orchestra.run.smithi083.stdout: "cross_origin_url": { 2024-04-24T09:18:03.725 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "cross_origin_url", 2024-04-24T09:18:03.725 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.725 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.725 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.725 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:03.725 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.725 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.725 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.725 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.725 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.726 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.726 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.726 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.726 INFO:teuthology.orchestra.run.smithi083.stdout: "crt_file": { 2024-04-24T09:18:03.726 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "crt_file", 2024-04-24T09:18:03.726 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.726 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.726 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.726 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:03.726 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.726 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.726 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.726 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.726 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.727 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.727 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.727 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.727 INFO:teuthology.orchestra.run.smithi083.stdout: "debug": { 2024-04-24T09:18:03.727 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "debug", 2024-04-24T09:18:03.727 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.727 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.727 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.727 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:03.727 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.727 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.727 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.727 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Enable/disable debug options", 2024-04-24T09:18:03.727 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.727 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.727 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.728 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.728 INFO:teuthology.orchestra.run.smithi083.stdout: "jwt_token_ttl": { 2024-04-24T09:18:03.728 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "jwt_token_ttl", 2024-04-24T09:18:03.728 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:03.728 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.728 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.728 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "28800", 2024-04-24T09:18:03.728 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.728 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.728 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.728 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.728 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.728 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.728 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.728 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.728 INFO:teuthology.orchestra.run.smithi083.stdout: "key_file": { 2024-04-24T09:18:03.729 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "key_file", 2024-04-24T09:18:03.729 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.729 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.729 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.729 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:03.729 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.729 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.729 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.729 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.729 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.729 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.729 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.729 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.729 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-24T09:18:03.729 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-24T09:18:03.729 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.730 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.730 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.730 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:03.730 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.730 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.730 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:03.730 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:03.730 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:03.730 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:03.730 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:03.730 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:03.730 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:03.730 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:03.730 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.730 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.731 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.731 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.731 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.731 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-24T09:18:03.731 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-24T09:18:03.731 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.731 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.731 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.731 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:03.731 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.731 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.731 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.731 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.731 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.731 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.731 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.732 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.732 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-24T09:18:03.732 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-24T09:18:03.732 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.732 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.732 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.732 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-24T09:18:03.732 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.732 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.732 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:03.732 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:03.732 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:03.732 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:03.732 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:03.732 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:03.733 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:03.733 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:03.733 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.733 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.733 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.733 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.733 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.733 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-24T09:18:03.733 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-24T09:18:03.733 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.733 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.733 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.733 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:03.733 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.733 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.733 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.734 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.734 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.734 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.734 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.734 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.734 INFO:teuthology.orchestra.run.smithi083.stdout: "motd": { 2024-04-24T09:18:03.734 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "motd", 2024-04-24T09:18:03.734 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.734 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.734 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.734 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:03.734 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.734 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.734 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.734 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "The message of the day", 2024-04-24T09:18:03.735 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.735 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.735 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.735 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.735 INFO:teuthology.orchestra.run.smithi083.stdout: "redirect_resolve_ip_addr": { 2024-04-24T09:18:03.735 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "redirect_resolve_ip_addr", 2024-04-24T09:18:03.735 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.735 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.735 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.735 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:03.735 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.735 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.735 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.735 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.736 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.736 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.736 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.736 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.736 INFO:teuthology.orchestra.run.smithi083.stdout: "server_addr": { 2024-04-24T09:18:03.736 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_addr", 2024-04-24T09:18:03.736 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.736 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.736 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.736 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "::", 2024-04-24T09:18:03.736 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.736 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.736 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.736 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.737 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.737 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.737 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.737 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.737 INFO:teuthology.orchestra.run.smithi083.stdout: "server_port": { 2024-04-24T09:18:03.737 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_port", 2024-04-24T09:18:03.737 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:03.737 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.737 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.737 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "8080", 2024-04-24T09:18:03.737 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.737 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.737 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.737 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.737 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.737 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.738 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.738 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.738 INFO:teuthology.orchestra.run.smithi083.stdout: "ssl": { 2024-04-24T09:18:03.738 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ssl", 2024-04-24T09:18:03.738 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.738 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.738 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.738 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:03.738 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.738 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.738 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.738 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.738 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.738 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.738 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.739 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.739 INFO:teuthology.orchestra.run.smithi083.stdout: "ssl_server_port": { 2024-04-24T09:18:03.739 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ssl_server_port", 2024-04-24T09:18:03.739 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:03.739 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.739 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.739 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "8443", 2024-04-24T09:18:03.739 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.739 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.739 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.739 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.739 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.739 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.739 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.740 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.740 INFO:teuthology.orchestra.run.smithi083.stdout: "standby_behaviour": { 2024-04-24T09:18:03.740 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "standby_behaviour", 2024-04-24T09:18:03.740 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.740 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.740 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.740 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "redirect", 2024-04-24T09:18:03.740 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.740 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.740 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:03.740 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:03.740 INFO:teuthology.orchestra.run.smithi083.stdout: "redirect" 2024-04-24T09:18:03.740 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:03.740 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.740 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.741 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.741 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.741 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.741 INFO:teuthology.orchestra.run.smithi083.stdout: "standby_error_status_code": { 2024-04-24T09:18:03.741 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "standby_error_status_code", 2024-04-24T09:18:03.741 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:03.741 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.741 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.741 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "500", 2024-04-24T09:18:03.741 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "400", 2024-04-24T09:18:03.741 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "599", 2024-04-24T09:18:03.741 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.741 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.741 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.741 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.741 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.742 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.742 INFO:teuthology.orchestra.run.smithi083.stdout: "url_prefix": { 2024-04-24T09:18:03.742 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "url_prefix", 2024-04-24T09:18:03.742 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.742 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.742 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.742 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:03.742 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.742 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.742 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.742 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.742 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.742 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.742 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.742 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:03.743 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:03.743 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.743 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:03.743 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "devicehealth", 2024-04-24T09:18:03.743 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-24T09:18:03.743 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-24T09:18:03.743 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-24T09:18:03.743 INFO:teuthology.orchestra.run.smithi083.stdout: "enable_monitoring": { 2024-04-24T09:18:03.743 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "enable_monitoring", 2024-04-24T09:18:03.743 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.743 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.743 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.743 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:03.743 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.743 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.743 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.743 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "monitor device health metrics", 2024-04-24T09:18:03.744 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.744 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.744 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.744 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.744 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-24T09:18:03.744 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-24T09:18:03.744 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.744 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.744 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.744 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:03.744 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.744 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.744 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:03.744 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:03.744 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:03.745 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:03.745 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:03.745 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:03.745 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:03.745 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:03.745 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.745 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.745 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.745 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.745 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.745 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-24T09:18:03.745 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-24T09:18:03.745 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.745 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.745 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.745 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:03.746 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.746 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.746 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.746 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.746 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.746 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.746 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.746 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.746 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-24T09:18:03.746 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-24T09:18:03.746 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.746 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.746 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.746 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-24T09:18:03.746 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.746 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.747 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:03.747 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:03.747 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:03.747 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:03.747 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:03.747 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:03.747 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:03.747 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:03.747 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.747 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.747 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.747 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.747 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.747 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-24T09:18:03.747 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-24T09:18:03.747 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.748 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.748 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.748 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:03.748 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.748 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.748 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.748 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.748 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.748 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.748 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.748 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.748 INFO:teuthology.orchestra.run.smithi083.stdout: "mark_out_threshold": { 2024-04-24T09:18:03.748 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mark_out_threshold", 2024-04-24T09:18:03.748 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-24T09:18:03.748 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.749 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.749 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "2419200", 2024-04-24T09:18:03.749 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.749 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.749 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.749 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-24T09:18:03.749 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.749 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.749 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.749 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.749 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_name": { 2024-04-24T09:18:03.749 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "pool_name", 2024-04-24T09:18:03.749 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.749 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.749 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.750 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "device_health_metrics", 2024-04-24T09:18:03.750 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.750 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.750 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.750 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-24T09:18:03.750 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.750 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.750 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.750 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.750 INFO:teuthology.orchestra.run.smithi083.stdout: "retention_period": { 2024-04-24T09:18:03.750 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "retention_period", 2024-04-24T09:18:03.750 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-24T09:18:03.750 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.750 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.751 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "15552000", 2024-04-24T09:18:03.751 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.751 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.751 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.751 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how long to retain device health metrics", 2024-04-24T09:18:03.751 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.751 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.751 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.751 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.751 INFO:teuthology.orchestra.run.smithi083.stdout: "scrape_frequency": { 2024-04-24T09:18:03.751 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "scrape_frequency", 2024-04-24T09:18:03.751 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-24T09:18:03.751 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.751 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.751 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "86400", 2024-04-24T09:18:03.751 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.752 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.752 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.752 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-24T09:18:03.752 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.752 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.752 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.752 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.752 INFO:teuthology.orchestra.run.smithi083.stdout: "self_heal": { 2024-04-24T09:18:03.752 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "self_heal", 2024-04-24T09:18:03.752 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.752 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.752 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.752 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:03.752 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.752 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.752 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.752 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-24T09:18:03.753 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.753 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.753 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.753 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.753 INFO:teuthology.orchestra.run.smithi083.stdout: "sleep_interval": { 2024-04-24T09:18:03.753 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sleep_interval", 2024-04-24T09:18:03.753 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-24T09:18:03.753 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.753 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.753 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "600", 2024-04-24T09:18:03.753 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.753 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.753 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.753 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how frequently to wake up and check device health", 2024-04-24T09:18:03.753 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.753 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.754 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.754 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.754 INFO:teuthology.orchestra.run.smithi083.stdout: "warn_threshold": { 2024-04-24T09:18:03.754 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "warn_threshold", 2024-04-24T09:18:03.754 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-24T09:18:03.754 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.754 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.754 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "7257600", 2024-04-24T09:18:03.754 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.754 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.754 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.754 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-24T09:18:03.754 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.754 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.754 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.754 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:03.755 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:03.755 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.755 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:03.755 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "diskprediction_local", 2024-04-24T09:18:03.755 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-24T09:18:03.755 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-24T09:18:03.755 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-24T09:18:03.755 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-24T09:18:03.755 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-24T09:18:03.755 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.755 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.755 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.755 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:03.755 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.755 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.755 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:03.756 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:03.756 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:03.756 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:03.756 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:03.756 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:03.756 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:03.756 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:03.756 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.756 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.756 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.756 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.756 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.756 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-24T09:18:03.756 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-24T09:18:03.756 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.756 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.757 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.757 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:03.757 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.757 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.757 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.757 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.757 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.757 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.757 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.757 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.757 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-24T09:18:03.757 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-24T09:18:03.757 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.757 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.757 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.757 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-24T09:18:03.758 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.758 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.758 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:03.758 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:03.758 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:03.758 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:03.758 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:03.758 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:03.758 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:03.758 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:03.758 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.758 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.758 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.758 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.758 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.759 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-24T09:18:03.759 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-24T09:18:03.759 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.759 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.759 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.759 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:03.759 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.759 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.759 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.759 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.759 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.759 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.759 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.759 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.759 INFO:teuthology.orchestra.run.smithi083.stdout: "predict_interval": { 2024-04-24T09:18:03.759 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "predict_interval", 2024-04-24T09:18:03.760 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.760 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.760 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.760 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "86400", 2024-04-24T09:18:03.760 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.760 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.760 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.760 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.760 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.760 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.760 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.760 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.760 INFO:teuthology.orchestra.run.smithi083.stdout: "predictor_model": { 2024-04-24T09:18:03.760 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "predictor_model", 2024-04-24T09:18:03.760 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.760 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.761 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.761 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "prophetstor", 2024-04-24T09:18:03.761 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.761 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.761 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.761 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.761 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.761 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.761 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.761 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.761 INFO:teuthology.orchestra.run.smithi083.stdout: "sleep_interval": { 2024-04-24T09:18:03.761 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sleep_interval", 2024-04-24T09:18:03.761 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.761 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.761 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.762 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "600", 2024-04-24T09:18:03.762 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.762 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.762 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.762 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.762 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.762 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.762 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.762 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:03.762 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:03.762 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.762 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:03.762 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "influx", 2024-04-24T09:18:03.762 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": false, 2024-04-24T09:18:03.762 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "influxdb python module not found", 2024-04-24T09:18:03.762 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-24T09:18:03.763 INFO:teuthology.orchestra.run.smithi083.stdout: "batch_size": { 2024-04-24T09:18:03.763 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "batch_size", 2024-04-24T09:18:03.763 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:03.763 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.763 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.763 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "5000", 2024-04-24T09:18:03.763 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.763 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.763 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.763 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-24T09:18:03.763 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.763 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.763 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.763 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.763 INFO:teuthology.orchestra.run.smithi083.stdout: "database": { 2024-04-24T09:18:03.763 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "database", 2024-04-24T09:18:03.764 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.764 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.764 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.764 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "ceph", 2024-04-24T09:18:03.764 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.764 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.764 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.764 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-24T09:18:03.764 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.764 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.764 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.764 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.764 INFO:teuthology.orchestra.run.smithi083.stdout: "hostname": { 2024-04-24T09:18:03.764 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "hostname", 2024-04-24T09:18:03.764 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.764 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.765 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.765 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:03.765 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.765 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.765 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.765 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "InfluxDB server hostname", 2024-04-24T09:18:03.765 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.765 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.765 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.765 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.765 INFO:teuthology.orchestra.run.smithi083.stdout: "interval": { 2024-04-24T09:18:03.765 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "interval", 2024-04-24T09:18:03.765 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-24T09:18:03.765 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.765 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.765 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "30", 2024-04-24T09:18:03.766 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "5", 2024-04-24T09:18:03.766 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.766 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.766 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-24T09:18:03.766 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.766 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.766 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.766 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.766 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-24T09:18:03.766 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-24T09:18:03.766 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.766 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.766 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.766 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:03.766 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.766 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.766 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:03.767 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:03.767 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:03.767 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:03.767 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:03.767 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:03.767 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:03.767 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:03.767 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.767 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.767 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.767 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.767 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.767 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-24T09:18:03.767 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-24T09:18:03.767 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.767 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.768 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.768 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:03.768 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.768 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.768 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.768 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.768 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.768 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.768 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.768 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.768 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-24T09:18:03.768 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-24T09:18:03.768 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.768 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.768 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.769 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-24T09:18:03.769 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.769 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.769 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:03.769 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:03.769 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:03.769 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:03.769 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:03.769 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:03.769 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:03.769 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:03.769 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.769 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.769 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.769 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.769 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.770 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-24T09:18:03.770 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-24T09:18:03.770 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.770 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.770 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.770 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:03.770 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.770 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.770 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.770 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.770 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.770 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.770 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.770 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.770 INFO:teuthology.orchestra.run.smithi083.stdout: "password": { 2024-04-24T09:18:03.771 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "password", 2024-04-24T09:18:03.771 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.771 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.771 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.771 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:03.771 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.771 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.771 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.771 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "password of InfluxDB server user", 2024-04-24T09:18:03.771 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.771 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.771 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.771 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.771 INFO:teuthology.orchestra.run.smithi083.stdout: "port": { 2024-04-24T09:18:03.771 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "port", 2024-04-24T09:18:03.772 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:03.772 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.772 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.772 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "8086", 2024-04-24T09:18:03.772 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.772 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.772 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.772 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "InfluxDB server port", 2024-04-24T09:18:03.772 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.772 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.772 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.772 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.772 INFO:teuthology.orchestra.run.smithi083.stdout: "ssl": { 2024-04-24T09:18:03.773 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ssl", 2024-04-24T09:18:03.773 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.773 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.773 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.773 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "false", 2024-04-24T09:18:03.773 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.773 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.773 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.773 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-24T09:18:03.773 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.773 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.773 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.773 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.773 INFO:teuthology.orchestra.run.smithi083.stdout: "threads": { 2024-04-24T09:18:03.773 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "threads", 2024-04-24T09:18:03.774 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:03.774 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.774 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.774 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "5", 2024-04-24T09:18:03.774 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "1", 2024-04-24T09:18:03.774 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "32", 2024-04-24T09:18:03.774 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.774 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-24T09:18:03.774 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.774 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.774 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.774 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.774 INFO:teuthology.orchestra.run.smithi083.stdout: "username": { 2024-04-24T09:18:03.774 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "username", 2024-04-24T09:18:03.774 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.774 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.775 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.775 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:03.775 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.775 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.775 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.775 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "username of InfluxDB server user", 2024-04-24T09:18:03.775 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.775 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.775 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.775 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.775 INFO:teuthology.orchestra.run.smithi083.stdout: "verify_ssl": { 2024-04-24T09:18:03.775 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "verify_ssl", 2024-04-24T09:18:03.775 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.775 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.775 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.776 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "true", 2024-04-24T09:18:03.776 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.776 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.776 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.776 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-24T09:18:03.776 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.776 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.776 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.776 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:03.776 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:03.776 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.776 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:03.776 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "insights", 2024-04-24T09:18:03.776 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-24T09:18:03.776 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-24T09:18:03.777 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-24T09:18:03.777 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-24T09:18:03.777 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-24T09:18:03.777 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.777 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.777 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.777 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:03.777 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.777 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.777 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:03.777 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:03.777 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:03.777 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:03.777 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:03.777 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:03.778 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:03.778 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:03.778 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.778 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.778 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.778 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.778 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.778 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-24T09:18:03.778 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-24T09:18:03.778 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.778 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.778 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.778 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:03.778 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.778 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.779 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.779 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.779 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.779 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.779 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.779 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.779 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-24T09:18:03.779 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-24T09:18:03.779 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.779 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.779 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.779 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-24T09:18:03.779 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.779 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.779 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:03.779 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:03.780 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:03.780 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:03.780 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:03.780 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:03.780 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:03.780 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:03.780 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.780 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.780 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.780 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.780 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.780 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-24T09:18:03.780 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-24T09:18:03.780 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.780 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.781 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.781 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:03.781 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.781 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.781 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.781 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.781 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.781 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.781 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.781 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:03.781 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:03.781 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.781 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:03.781 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "iostat", 2024-04-24T09:18:03.781 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-24T09:18:03.781 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-24T09:18:03.782 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-24T09:18:03.782 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-24T09:18:03.782 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-24T09:18:03.782 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.782 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.782 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.782 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:03.782 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.782 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.782 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:03.782 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:03.782 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:03.782 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:03.782 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:03.782 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:03.782 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:03.783 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:03.783 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.783 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.783 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.783 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.783 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.783 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-24T09:18:03.783 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-24T09:18:03.783 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.783 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.783 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.783 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:03.783 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.783 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.783 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.783 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.783 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.784 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.784 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.784 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.784 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-24T09:18:03.784 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-24T09:18:03.784 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.784 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.784 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.784 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-24T09:18:03.784 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.784 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.784 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:03.784 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:03.784 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:03.784 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:03.785 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:03.785 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:03.785 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:03.785 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:03.785 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.785 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.785 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.785 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.785 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.785 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-24T09:18:03.785 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-24T09:18:03.785 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.785 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.785 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.785 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:03.786 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.786 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.786 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.786 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.786 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.786 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.786 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.786 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:03.786 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:03.786 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.786 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:03.786 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "k8sevents", 2024-04-24T09:18:03.786 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-24T09:18:03.786 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-24T09:18:03.787 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-24T09:18:03.787 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_event_retention_days": { 2024-04-24T09:18:03.787 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ceph_event_retention_days", 2024-04-24T09:18:03.787 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:03.787 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.787 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.787 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "7", 2024-04-24T09:18:03.787 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.787 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.787 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.787 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-24T09:18:03.787 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.787 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.787 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.787 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.787 INFO:teuthology.orchestra.run.smithi083.stdout: "config_check_secs": { 2024-04-24T09:18:03.788 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "config_check_secs", 2024-04-24T09:18:03.788 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:03.788 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.788 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.788 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-04-24T09:18:03.788 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "10", 2024-04-24T09:18:03.788 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.788 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.788 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-24T09:18:03.788 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.788 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.788 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.788 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.788 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-24T09:18:03.788 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-24T09:18:03.789 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.789 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.789 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.789 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:03.789 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.789 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.789 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:03.789 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:03.789 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:03.789 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:03.789 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:03.789 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:03.789 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:03.789 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:03.789 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.790 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.790 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.790 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.790 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.790 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-24T09:18:03.790 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-24T09:18:03.790 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.790 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.790 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.790 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:03.790 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.790 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.790 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.790 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.791 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.791 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.791 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.791 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.791 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-24T09:18:03.791 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-24T09:18:03.791 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.791 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.791 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.791 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-24T09:18:03.791 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.791 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.791 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:03.791 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:03.791 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:03.791 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:03.791 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:03.792 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:03.792 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:03.792 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:03.792 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.792 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.792 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.792 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.792 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.792 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-24T09:18:03.792 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-24T09:18:03.792 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.792 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.792 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.792 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:03.792 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.792 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.792 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.793 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.793 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.793 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.793 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.793 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:03.793 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:03.793 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.793 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:03.793 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "localpool", 2024-04-24T09:18:03.793 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-24T09:18:03.793 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-24T09:18:03.793 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-24T09:18:03.793 INFO:teuthology.orchestra.run.smithi083.stdout: "failure_domain": { 2024-04-24T09:18:03.793 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "failure_domain", 2024-04-24T09:18:03.793 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.793 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.794 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.794 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "host", 2024-04-24T09:18:03.794 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.794 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.794 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.794 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "failure domain for any created local pool", 2024-04-24T09:18:03.794 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-24T09:18:03.794 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.794 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.794 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.794 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-24T09:18:03.794 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-24T09:18:03.794 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.794 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.794 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.794 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:03.795 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.795 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.795 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:03.795 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:03.795 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:03.795 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:03.795 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:03.795 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:03.795 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:03.795 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:03.795 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.795 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.795 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.795 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.795 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.795 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-24T09:18:03.796 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-24T09:18:03.796 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.796 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.796 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.796 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:03.796 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.796 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.796 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.796 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.796 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.796 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.796 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.796 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.796 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-24T09:18:03.796 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-24T09:18:03.796 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.796 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.797 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.797 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-24T09:18:03.797 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.797 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.797 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:03.797 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:03.797 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:03.797 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:03.797 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:03.797 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:03.797 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:03.797 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:03.797 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.797 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.797 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.798 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.798 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.798 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-24T09:18:03.798 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-24T09:18:03.798 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.798 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.798 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.798 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:03.798 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.798 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.798 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.798 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.798 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.798 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.799 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.799 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.799 INFO:teuthology.orchestra.run.smithi083.stdout: "min_size": { 2024-04-24T09:18:03.799 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "min_size", 2024-04-24T09:18:03.799 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:03.799 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.799 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.799 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:03.799 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.799 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.799 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.799 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "default min_size for any created local pool", 2024-04-24T09:18:03.799 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-24T09:18:03.799 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.799 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.800 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.800 INFO:teuthology.orchestra.run.smithi083.stdout: "num_rep": { 2024-04-24T09:18:03.800 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "num_rep", 2024-04-24T09:18:03.800 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:03.800 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.800 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.800 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "3", 2024-04-24T09:18:03.800 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.800 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.800 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.800 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "default replica count for any created local pool", 2024-04-24T09:18:03.800 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.800 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.800 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.801 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.801 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num": { 2024-04-24T09:18:03.801 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "pg_num", 2024-04-24T09:18:03.801 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:03.801 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.801 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.801 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "128", 2024-04-24T09:18:03.801 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.801 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.801 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.801 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "default pg_num for any created local pool", 2024-04-24T09:18:03.801 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.801 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.801 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.801 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.801 INFO:teuthology.orchestra.run.smithi083.stdout: "prefix": { 2024-04-24T09:18:03.802 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "prefix", 2024-04-24T09:18:03.802 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.802 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.802 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.802 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:03.802 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.802 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.802 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.802 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "name prefix for any created local pool", 2024-04-24T09:18:03.802 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.802 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.802 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.802 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.802 INFO:teuthology.orchestra.run.smithi083.stdout: "subtree": { 2024-04-24T09:18:03.802 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "subtree", 2024-04-24T09:18:03.803 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.803 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.803 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.803 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "rack", 2024-04-24T09:18:03.803 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.803 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.803 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.803 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-24T09:18:03.803 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-24T09:18:03.803 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.803 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.803 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:03.803 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:03.803 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.803 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:03.803 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mds_autoscaler", 2024-04-24T09:18:03.804 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-24T09:18:03.804 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-24T09:18:03.804 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-24T09:18:03.804 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-24T09:18:03.804 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-24T09:18:03.804 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.804 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.804 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.804 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:03.804 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.804 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.804 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:03.804 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:03.804 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:03.804 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:03.804 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:03.804 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:03.805 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:03.805 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:03.805 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.805 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.805 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.805 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.805 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.805 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-24T09:18:03.805 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-24T09:18:03.805 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.805 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.805 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.805 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:03.805 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.805 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.805 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.806 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.806 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.806 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.806 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.806 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.806 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-24T09:18:03.806 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-24T09:18:03.806 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.806 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.806 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.806 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-24T09:18:03.806 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.806 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.806 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:03.806 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:03.807 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:03.807 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:03.807 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:03.807 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:03.807 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:03.807 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:03.807 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.807 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.807 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.807 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.807 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.807 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-24T09:18:03.807 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-24T09:18:03.807 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.807 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.807 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.808 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:03.808 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.808 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.808 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.808 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.808 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.808 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.808 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.808 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:03.808 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:03.808 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.808 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:03.808 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mirroring", 2024-04-24T09:18:03.808 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-24T09:18:03.808 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-24T09:18:03.808 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-24T09:18:03.808 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-24T09:18:03.809 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-24T09:18:03.809 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.809 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.809 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.809 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:03.809 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.809 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.809 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:03.809 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:03.809 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:03.809 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:03.809 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:03.809 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:03.809 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:03.809 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:03.809 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.810 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.810 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.810 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.810 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.810 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-24T09:18:03.810 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-24T09:18:03.810 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.810 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.810 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.810 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:03.810 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.810 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.810 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.811 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.811 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.811 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.811 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.811 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.811 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-24T09:18:03.811 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-24T09:18:03.811 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.811 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.811 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.811 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-24T09:18:03.811 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.811 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.811 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:03.811 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:03.811 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:03.812 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:03.812 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:03.812 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:03.812 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:03.812 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:03.812 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.812 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.812 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.812 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.812 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.812 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-24T09:18:03.812 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-24T09:18:03.812 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.812 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.812 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.812 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:03.813 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.813 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.813 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.813 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.813 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.813 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.813 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.813 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:03.813 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:03.813 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.813 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:03.813 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "nfs", 2024-04-24T09:18:03.813 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-24T09:18:03.813 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-24T09:18:03.814 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-24T09:18:03.814 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-24T09:18:03.814 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-24T09:18:03.814 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.814 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.814 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.814 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:03.814 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.814 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.814 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:03.814 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:03.814 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:03.814 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:03.814 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:03.814 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:03.815 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:03.815 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:03.815 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.815 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.815 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.815 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.815 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.815 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-24T09:18:03.815 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-24T09:18:03.815 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.815 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.815 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.815 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:03.815 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.815 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.815 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.816 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.816 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.816 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.816 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.816 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.816 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-24T09:18:03.816 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-24T09:18:03.816 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.816 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.816 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.816 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-24T09:18:03.816 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.816 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.816 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:03.816 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:03.816 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:03.817 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:03.817 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:03.817 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:03.817 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:03.817 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:03.817 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.817 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.817 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.817 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.817 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.817 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-24T09:18:03.817 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-24T09:18:03.817 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.817 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.817 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.817 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:03.817 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.818 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.818 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.818 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.818 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.818 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.818 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.818 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:03.818 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:03.818 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.818 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:03.818 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "orchestrator", 2024-04-24T09:18:03.818 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-24T09:18:03.818 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-24T09:18:03.818 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-24T09:18:03.818 INFO:teuthology.orchestra.run.smithi083.stdout: "fail_fs": { 2024-04-24T09:18:03.819 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "fail_fs", 2024-04-24T09:18:03.819 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.819 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.819 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.819 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:03.819 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.819 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.819 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.819 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-24T09:18:03.819 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.819 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.819 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.819 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.819 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-24T09:18:03.819 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-24T09:18:03.819 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.820 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.820 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.820 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:03.820 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.820 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.820 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:03.820 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:03.820 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:03.820 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:03.820 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:03.820 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:03.820 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:03.820 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:03.820 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.820 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.821 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.821 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.821 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.821 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-24T09:18:03.821 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-24T09:18:03.821 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.821 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.821 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.821 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:03.821 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.821 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.821 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.821 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.821 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.821 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.821 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.822 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.822 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-24T09:18:03.822 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-24T09:18:03.823 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.823 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.823 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.823 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-24T09:18:03.823 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.824 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.824 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:03.824 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:03.824 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:03.824 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:03.824 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:03.824 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:03.824 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:03.824 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:03.824 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.824 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.824 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.824 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.824 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.824 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-24T09:18:03.825 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-24T09:18:03.825 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.825 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.825 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.825 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:03.825 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.825 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.825 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.825 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.825 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.825 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.825 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.825 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.825 INFO:teuthology.orchestra.run.smithi083.stdout: "orchestrator": { 2024-04-24T09:18:03.826 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "orchestrator", 2024-04-24T09:18:03.826 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.826 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.826 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.826 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:03.826 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.826 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.826 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:03.826 INFO:teuthology.orchestra.run.smithi083.stdout: "cephadm", 2024-04-24T09:18:03.826 INFO:teuthology.orchestra.run.smithi083.stdout: "rook", 2024-04-24T09:18:03.826 INFO:teuthology.orchestra.run.smithi083.stdout: "test_orchestrator" 2024-04-24T09:18:03.826 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:03.826 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Orchestrator backend", 2024-04-24T09:18:03.826 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.827 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.827 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.827 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:03.827 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:03.827 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.827 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:03.827 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "osd_perf_query", 2024-04-24T09:18:03.827 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-24T09:18:03.827 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-24T09:18:03.827 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-24T09:18:03.827 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-24T09:18:03.827 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-24T09:18:03.827 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.827 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.827 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.828 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:03.828 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.828 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.828 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:03.828 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:03.828 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:03.828 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:03.828 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:03.828 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:03.828 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:03.828 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:03.828 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.828 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.828 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.828 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.828 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.829 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-24T09:18:03.829 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-24T09:18:03.829 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.829 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.829 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.829 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:03.829 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.829 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.829 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.829 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.829 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.829 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.829 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.830 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.830 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-24T09:18:03.830 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-24T09:18:03.830 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.830 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.830 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.830 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-24T09:18:03.830 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.830 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.830 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:03.830 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:03.830 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:03.830 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:03.830 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:03.831 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:03.831 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:03.831 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:03.831 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.831 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.831 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.831 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.831 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.831 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-24T09:18:03.831 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-24T09:18:03.831 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.831 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.831 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.831 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:03.832 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.832 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.832 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.832 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.832 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.832 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.832 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.832 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:03.832 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:03.832 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.832 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:03.832 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "osd_support", 2024-04-24T09:18:03.832 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-24T09:18:03.832 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-24T09:18:03.832 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-24T09:18:03.832 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-24T09:18:03.833 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-24T09:18:03.833 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.833 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.833 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.833 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:03.833 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.833 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.833 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:03.833 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:03.833 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:03.833 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:03.833 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:03.833 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:03.833 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:03.833 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:03.833 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.834 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.834 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.840 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.840 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.840 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-24T09:18:03.840 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-24T09:18:03.840 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.840 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.840 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.840 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:03.841 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.841 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.841 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.841 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.841 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.841 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.841 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.841 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.841 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-24T09:18:03.841 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-24T09:18:03.841 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.841 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.841 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.841 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-24T09:18:03.842 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.842 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.842 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:03.842 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:03.842 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:03.842 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:03.842 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:03.842 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:03.842 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:03.842 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:03.842 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.842 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.842 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.843 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.843 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.843 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-24T09:18:03.843 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-24T09:18:03.843 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.843 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.843 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.843 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:03.843 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.843 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.843 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.843 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.843 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.843 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.844 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.844 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:03.844 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:03.844 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.844 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:03.844 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "pg_autoscaler", 2024-04-24T09:18:03.844 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-24T09:18:03.844 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-24T09:18:03.844 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-24T09:18:03.844 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-24T09:18:03.844 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-24T09:18:03.844 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.844 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.844 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.845 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:03.845 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.845 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.845 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:03.845 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:03.845 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:03.845 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:03.845 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:03.845 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:03.845 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:03.845 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:03.845 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.845 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.845 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.845 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.846 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.846 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-24T09:18:03.846 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-24T09:18:03.846 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.846 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.846 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.846 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:03.846 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.846 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.846 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.846 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.846 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.846 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.846 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.847 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.847 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-24T09:18:03.847 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-24T09:18:03.847 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.847 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.847 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.847 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-24T09:18:03.847 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.847 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.847 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:03.847 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:03.847 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:03.847 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:03.847 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:03.847 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:03.847 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:03.848 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:03.848 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.848 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.848 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.848 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.848 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.848 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-24T09:18:03.848 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-24T09:18:03.848 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.848 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.848 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.848 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:03.848 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.848 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.849 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.849 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.849 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.849 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.849 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.849 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.849 INFO:teuthology.orchestra.run.smithi083.stdout: "sleep_interval": { 2024-04-24T09:18:03.849 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sleep_interval", 2024-04-24T09:18:03.849 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-24T09:18:03.849 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.849 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.849 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "60", 2024-04-24T09:18:03.849 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.849 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.849 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.850 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.850 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.850 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.850 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.850 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.850 INFO:teuthology.orchestra.run.smithi083.stdout: "threshold": { 2024-04-24T09:18:03.850 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "threshold", 2024-04-24T09:18:03.850 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-04-24T09:18:03.850 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.850 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.850 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "3.0", 2024-04-24T09:18:03.850 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "1.0", 2024-04-24T09:18:03.850 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.850 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.851 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "scaling threshold", 2024-04-24T09:18:03.851 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-24T09:18:03.851 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.851 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.851 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:03.851 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:03.851 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.851 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:03.851 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "progress", 2024-04-24T09:18:03.851 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-24T09:18:03.851 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-24T09:18:03.851 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-24T09:18:03.851 INFO:teuthology.orchestra.run.smithi083.stdout: "allow_pg_recovery_event": { 2024-04-24T09:18:03.851 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "allow_pg_recovery_event", 2024-04-24T09:18:03.851 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.852 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.852 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.852 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:03.852 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.852 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.852 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.852 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-24T09:18:03.852 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.852 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.852 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.852 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.852 INFO:teuthology.orchestra.run.smithi083.stdout: "enabled": { 2024-04-24T09:18:03.852 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "enabled", 2024-04-24T09:18:03.852 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.852 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.852 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.853 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:03.853 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.853 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.853 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.853 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.853 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.853 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.853 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.853 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.853 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-24T09:18:03.853 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-24T09:18:03.853 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.853 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.853 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.853 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:03.854 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.854 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.854 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:03.854 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:03.854 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:03.854 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:03.854 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:03.854 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:03.854 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:03.854 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:03.854 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.854 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.854 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.854 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.854 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.855 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-24T09:18:03.855 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-24T09:18:03.855 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.855 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.855 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.855 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:03.855 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.855 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.855 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.855 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.855 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.855 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.855 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.855 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.855 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-24T09:18:03.856 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-24T09:18:03.856 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.856 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.856 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.856 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-24T09:18:03.856 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.856 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.856 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:03.856 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:03.856 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:03.856 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:03.856 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:03.856 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:03.856 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:03.856 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:03.857 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.857 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.857 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.857 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.857 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.857 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-24T09:18:03.857 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-24T09:18:03.857 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.857 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.857 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.857 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:03.857 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.857 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.857 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.857 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.858 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.858 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.858 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.858 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.858 INFO:teuthology.orchestra.run.smithi083.stdout: "max_completed_events": { 2024-04-24T09:18:03.858 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "max_completed_events", 2024-04-24T09:18:03.858 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:03.858 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.858 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.858 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "50", 2024-04-24T09:18:03.858 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.858 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.858 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.858 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "number of past completed events to remember", 2024-04-24T09:18:03.859 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.859 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.859 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.859 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.859 INFO:teuthology.orchestra.run.smithi083.stdout: "sleep_interval": { 2024-04-24T09:18:03.859 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sleep_interval", 2024-04-24T09:18:03.859 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-24T09:18:03.859 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.859 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.859 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "5", 2024-04-24T09:18:03.859 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.859 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.859 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.859 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how long the module is going to sleep", 2024-04-24T09:18:03.860 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.860 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.860 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.860 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:03.860 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:03.860 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.860 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:03.860 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "prometheus", 2024-04-24T09:18:03.860 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-24T09:18:03.860 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-24T09:18:03.860 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-24T09:18:03.860 INFO:teuthology.orchestra.run.smithi083.stdout: "cache": { 2024-04-24T09:18:03.860 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "cache", 2024-04-24T09:18:03.860 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.861 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.861 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.861 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:03.861 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.861 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.861 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.861 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.861 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.861 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.861 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.861 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.861 INFO:teuthology.orchestra.run.smithi083.stdout: "exclude_perf_counters": { 2024-04-24T09:18:03.861 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "exclude_perf_counters", 2024-04-24T09:18:03.861 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.861 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.862 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.862 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:03.862 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.862 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.862 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.862 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-24T09:18:03.862 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-24T09:18:03.862 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.862 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.862 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.862 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-24T09:18:03.862 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-24T09:18:03.862 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.862 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.863 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.863 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:03.863 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.863 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.863 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:03.863 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:03.863 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:03.863 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:03.863 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:03.863 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:03.863 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:03.863 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:03.863 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.863 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.863 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.864 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.864 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.864 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-24T09:18:03.864 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-24T09:18:03.864 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.864 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.864 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.864 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:03.864 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.864 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.864 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.864 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.864 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.864 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.864 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.864 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.865 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-24T09:18:03.865 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-24T09:18:03.865 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.865 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.865 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.865 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-24T09:18:03.865 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.865 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.865 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:03.865 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:03.865 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:03.865 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:03.865 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:03.865 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:03.865 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:03.866 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:03.866 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.866 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.866 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.866 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.866 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.866 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-24T09:18:03.866 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-24T09:18:03.866 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.866 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.866 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.866 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:03.866 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.866 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.867 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.867 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.867 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.867 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.867 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.867 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.867 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_stats_pools": { 2024-04-24T09:18:03.867 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rbd_stats_pools", 2024-04-24T09:18:03.867 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.867 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.867 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.867 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:03.867 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.867 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.868 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.868 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.868 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.868 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.868 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.868 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.868 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-24T09:18:03.868 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-24T09:18:03.868 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:03.868 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.868 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.868 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "300", 2024-04-24T09:18:03.868 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.868 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.869 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.869 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.869 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.869 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.869 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.869 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.869 INFO:teuthology.orchestra.run.smithi083.stdout: "scrape_interval": { 2024-04-24T09:18:03.869 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "scrape_interval", 2024-04-24T09:18:03.869 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-04-24T09:18:03.869 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.869 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.869 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "15.0", 2024-04-24T09:18:03.870 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.870 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.870 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.870 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.870 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.870 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.870 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.870 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.870 INFO:teuthology.orchestra.run.smithi083.stdout: "server_addr": { 2024-04-24T09:18:03.870 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_addr", 2024-04-24T09:18:03.870 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.870 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.870 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.871 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "::", 2024-04-24T09:18:03.871 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.871 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.871 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.871 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-24T09:18:03.871 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.871 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.871 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.871 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.871 INFO:teuthology.orchestra.run.smithi083.stdout: "server_port": { 2024-04-24T09:18:03.871 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_port", 2024-04-24T09:18:03.871 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:03.871 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.871 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.871 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "9283", 2024-04-24T09:18:03.872 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.872 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.872 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.872 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-24T09:18:03.872 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.872 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.872 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.872 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.872 INFO:teuthology.orchestra.run.smithi083.stdout: "stale_cache_strategy": { 2024-04-24T09:18:03.872 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "stale_cache_strategy", 2024-04-24T09:18:03.872 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.872 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.872 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.872 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "log", 2024-04-24T09:18:03.873 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.873 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.873 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.873 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.873 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.873 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.873 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.873 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.873 INFO:teuthology.orchestra.run.smithi083.stdout: "standby_behaviour": { 2024-04-24T09:18:03.873 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "standby_behaviour", 2024-04-24T09:18:03.873 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.873 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.873 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.873 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "default", 2024-04-24T09:18:03.873 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.874 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.874 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:03.874 INFO:teuthology.orchestra.run.smithi083.stdout: "default", 2024-04-24T09:18:03.874 INFO:teuthology.orchestra.run.smithi083.stdout: "error" 2024-04-24T09:18:03.874 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:03.874 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.874 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.874 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.874 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.874 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.874 INFO:teuthology.orchestra.run.smithi083.stdout: "standby_error_status_code": { 2024-04-24T09:18:03.874 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "standby_error_status_code", 2024-04-24T09:18:03.874 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:03.874 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.875 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.875 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "500", 2024-04-24T09:18:03.875 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "400", 2024-04-24T09:18:03.875 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "599", 2024-04-24T09:18:03.875 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.875 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.875 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.875 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.875 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.875 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:03.875 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:03.875 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.875 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:03.875 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rbd_support", 2024-04-24T09:18:03.875 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-24T09:18:03.876 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-24T09:18:03.876 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-24T09:18:03.876 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-24T09:18:03.876 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-24T09:18:03.876 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.876 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.876 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.876 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:03.876 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.876 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.876 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:03.876 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:03.876 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:03.876 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:03.876 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:03.877 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:03.877 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:03.877 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:03.877 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.877 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.877 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.877 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.877 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.877 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-24T09:18:03.877 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-24T09:18:03.877 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.877 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.877 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.877 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:03.877 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.878 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.878 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.878 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.878 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.878 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.878 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.878 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.878 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-24T09:18:03.878 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-24T09:18:03.878 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.878 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.878 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.878 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-24T09:18:03.878 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.879 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.879 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:03.879 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:03.879 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:03.879 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:03.879 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:03.879 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:03.879 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:03.879 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:03.879 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.879 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.879 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.879 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.879 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.880 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-24T09:18:03.880 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-24T09:18:03.880 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.880 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.880 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.880 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:03.880 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.880 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.880 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.880 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.880 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.880 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.880 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.880 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.880 INFO:teuthology.orchestra.run.smithi083.stdout: "max_concurrent_snap_create": { 2024-04-24T09:18:03.881 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "max_concurrent_snap_create", 2024-04-24T09:18:03.881 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:03.881 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.881 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.881 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-04-24T09:18:03.881 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.881 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.881 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.881 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.881 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.881 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.881 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.881 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.881 INFO:teuthology.orchestra.run.smithi083.stdout: "mirror_snapshot_schedule": { 2024-04-24T09:18:03.881 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mirror_snapshot_schedule", 2024-04-24T09:18:03.882 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.882 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.882 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.882 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:03.882 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.882 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.882 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.882 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.882 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.882 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.882 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.882 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.882 INFO:teuthology.orchestra.run.smithi083.stdout: "trash_purge_schedule": { 2024-04-24T09:18:03.883 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "trash_purge_schedule", 2024-04-24T09:18:03.883 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.883 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.883 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.883 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:03.883 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.883 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.883 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.883 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.883 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.883 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.883 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.883 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:03.883 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:03.884 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.884 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:03.884 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "restful", 2024-04-24T09:18:03.884 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-24T09:18:03.884 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-24T09:18:03.884 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-24T09:18:03.884 INFO:teuthology.orchestra.run.smithi083.stdout: "enable_auth": { 2024-04-24T09:18:03.884 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "enable_auth", 2024-04-24T09:18:03.884 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.884 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.884 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.884 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:03.884 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.884 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.884 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.885 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.885 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.885 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.885 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.885 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.885 INFO:teuthology.orchestra.run.smithi083.stdout: "key_file": { 2024-04-24T09:18:03.885 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "key_file", 2024-04-24T09:18:03.885 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.885 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.885 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.885 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:03.885 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.885 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.885 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.886 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.886 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.886 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.886 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.886 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.886 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-24T09:18:03.886 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-24T09:18:03.886 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.886 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.886 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.886 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:03.886 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.886 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.886 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:03.886 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:03.887 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:03.887 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:03.887 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:03.887 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:03.887 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:03.887 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:03.887 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.887 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.887 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.887 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.887 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.887 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-24T09:18:03.887 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-24T09:18:03.887 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.888 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.888 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.888 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:03.888 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.888 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.888 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.888 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.888 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.888 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.888 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.888 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.888 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-24T09:18:03.888 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-24T09:18:03.889 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.889 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.889 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.889 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-24T09:18:03.889 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.889 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.889 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:03.889 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:03.889 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:03.889 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:03.889 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:03.889 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:03.889 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:03.889 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:03.889 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.890 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.890 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.890 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.890 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.890 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-24T09:18:03.890 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-24T09:18:03.890 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.890 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.890 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.890 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:03.890 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.890 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.890 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.890 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.891 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.891 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.891 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.891 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.891 INFO:teuthology.orchestra.run.smithi083.stdout: "server_addr": { 2024-04-24T09:18:03.891 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_addr", 2024-04-24T09:18:03.891 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.891 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.891 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.891 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:03.891 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.891 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.891 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.891 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.891 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.891 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.892 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.892 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.892 INFO:teuthology.orchestra.run.smithi083.stdout: "server_port": { 2024-04-24T09:18:03.892 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_port", 2024-04-24T09:18:03.892 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.892 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.892 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.892 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:03.892 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.892 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.892 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.892 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.892 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.892 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.892 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.892 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:03.893 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:03.893 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.893 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:03.893 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rgw", 2024-04-24T09:18:03.893 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-24T09:18:03.893 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-24T09:18:03.893 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-24T09:18:03.893 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-24T09:18:03.893 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-24T09:18:03.893 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.893 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.893 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.893 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:03.893 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.893 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.893 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:03.894 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:03.894 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:03.894 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:03.894 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:03.894 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:03.894 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:03.894 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:03.894 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.894 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.894 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.894 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.894 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.894 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-24T09:18:03.894 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-24T09:18:03.894 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.894 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.895 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.895 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:03.895 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.895 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.895 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.895 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.895 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.895 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.895 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.895 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.895 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-24T09:18:03.895 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-24T09:18:03.895 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.895 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.895 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.895 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-24T09:18:03.896 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.896 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.896 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:03.896 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:03.896 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:03.896 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:03.896 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:03.896 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:03.896 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:03.896 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:03.896 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.896 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.896 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.896 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.896 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.896 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-24T09:18:03.897 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-24T09:18:03.897 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.897 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.897 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.897 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:03.897 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.897 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.897 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.897 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.897 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.897 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.897 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.897 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:03.897 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:03.897 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.897 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:03.898 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rook", 2024-04-24T09:18:03.898 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-24T09:18:03.898 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-24T09:18:03.898 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-24T09:18:03.898 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-24T09:18:03.898 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-24T09:18:03.898 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.898 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.898 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.898 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:03.898 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.898 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.898 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:03.898 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:03.898 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:03.899 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:03.899 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:03.899 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:03.899 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:03.899 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:03.899 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.899 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.899 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.899 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.899 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.899 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-24T09:18:03.899 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-24T09:18:03.899 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.899 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.899 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.900 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:03.900 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.900 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.900 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.900 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.900 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.900 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.900 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.900 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.900 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-24T09:18:03.900 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-24T09:18:03.900 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.900 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.900 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.900 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-24T09:18:03.901 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.901 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.901 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:03.901 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:03.901 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:03.901 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:03.901 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:03.901 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:03.901 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:03.901 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:03.901 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.901 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.901 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.901 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.901 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.901 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-24T09:18:03.902 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-24T09:18:03.902 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.902 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.902 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.902 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:03.902 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.902 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.902 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.902 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.902 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.902 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.902 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.902 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.902 INFO:teuthology.orchestra.run.smithi083.stdout: "storage_class": { 2024-04-24T09:18:03.902 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "storage_class", 2024-04-24T09:18:03.903 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.903 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.903 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.903 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "local", 2024-04-24T09:18:03.903 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.903 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.903 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.903 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-24T09:18:03.903 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.903 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.903 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.903 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:03.903 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:03.903 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.903 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:03.903 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "selftest", 2024-04-24T09:18:03.904 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-24T09:18:03.904 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-24T09:18:03.904 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-24T09:18:03.904 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-24T09:18:03.904 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-24T09:18:03.904 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.904 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.904 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.904 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:03.904 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.904 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.904 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:03.904 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:03.904 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:03.904 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:03.904 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:03.904 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:03.905 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:03.905 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:03.905 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.905 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.905 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.905 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.905 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.905 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-24T09:18:03.905 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-24T09:18:03.905 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.905 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.905 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.905 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:03.905 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.905 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.906 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.906 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.906 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.906 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.906 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.906 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.906 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-24T09:18:03.906 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-24T09:18:03.906 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.906 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.906 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.906 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-24T09:18:03.906 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.906 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.907 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:03.907 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:03.907 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:03.907 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:03.907 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:03.907 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:03.907 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:03.907 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:03.907 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.907 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.907 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.907 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.907 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.907 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-24T09:18:03.907 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-24T09:18:03.908 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.908 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.908 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.908 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:03.908 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.908 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.908 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.908 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.908 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.908 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.908 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.908 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.908 INFO:teuthology.orchestra.run.smithi083.stdout: "roption1": { 2024-04-24T09:18:03.908 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "roption1", 2024-04-24T09:18:03.908 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.909 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.909 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.909 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:03.909 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.909 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.909 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.909 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.909 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.909 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.909 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.909 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.909 INFO:teuthology.orchestra.run.smithi083.stdout: "roption2": { 2024-04-24T09:18:03.909 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "roption2", 2024-04-24T09:18:03.910 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.910 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.910 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.910 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "xyz", 2024-04-24T09:18:03.910 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.910 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.910 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.910 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.910 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.910 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.910 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.910 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.910 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption1": { 2024-04-24T09:18:03.911 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption1", 2024-04-24T09:18:03.911 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.911 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.911 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.911 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:03.911 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.911 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.911 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.911 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.911 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.911 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.911 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.911 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.911 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption2": { 2024-04-24T09:18:03.911 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption2", 2024-04-24T09:18:03.911 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:03.912 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.912 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.912 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:03.912 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.912 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.912 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.912 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.912 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.912 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.912 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.912 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.912 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption3": { 2024-04-24T09:18:03.912 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption3", 2024-04-24T09:18:03.912 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-04-24T09:18:03.912 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.913 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.913 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:03.913 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.913 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.913 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.913 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.913 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.913 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.913 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.913 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.913 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption4": { 2024-04-24T09:18:03.913 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption4", 2024-04-24T09:18:03.913 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.913 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.913 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.913 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:03.914 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.914 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.914 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.914 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.914 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.914 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.914 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.914 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.914 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption5": { 2024-04-24T09:18:03.914 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption5", 2024-04-24T09:18:03.914 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.914 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.914 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.914 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:03.915 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.915 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.915 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.915 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.915 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.915 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.915 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.915 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.915 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption6": { 2024-04-24T09:18:03.915 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption6", 2024-04-24T09:18:03.915 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.915 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.915 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.915 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:03.915 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.916 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.916 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.916 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.916 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.916 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.916 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.916 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.916 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption7": { 2024-04-24T09:18:03.916 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption7", 2024-04-24T09:18:03.916 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:03.916 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.916 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.916 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:03.916 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "1", 2024-04-24T09:18:03.916 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "42", 2024-04-24T09:18:03.916 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.917 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.917 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.917 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.917 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.917 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.917 INFO:teuthology.orchestra.run.smithi083.stdout: "testkey": { 2024-04-24T09:18:03.917 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "testkey", 2024-04-24T09:18:03.917 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.917 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.917 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.917 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:03.917 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.917 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.917 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.918 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.918 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.918 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.918 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.918 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.918 INFO:teuthology.orchestra.run.smithi083.stdout: "testlkey": { 2024-04-24T09:18:03.918 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "testlkey", 2024-04-24T09:18:03.918 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.918 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.918 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.918 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:03.918 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.918 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.918 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.918 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.918 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.919 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.919 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.919 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.919 INFO:teuthology.orchestra.run.smithi083.stdout: "testnewline": { 2024-04-24T09:18:03.919 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "testnewline", 2024-04-24T09:18:03.919 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.919 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.919 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.919 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:03.919 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.919 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.919 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.919 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.919 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.919 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.919 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.920 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:03.920 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:03.920 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.920 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:03.920 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "snap_schedule", 2024-04-24T09:18:03.920 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-24T09:18:03.920 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-24T09:18:03.920 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-24T09:18:03.920 INFO:teuthology.orchestra.run.smithi083.stdout: "allow_m_granularity": { 2024-04-24T09:18:03.920 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "allow_m_granularity", 2024-04-24T09:18:03.920 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.920 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.920 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.920 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:03.920 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.920 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.921 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.921 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "allow minute scheduled snapshots", 2024-04-24T09:18:03.921 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.921 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.921 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.921 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.921 INFO:teuthology.orchestra.run.smithi083.stdout: "dump_on_update": { 2024-04-24T09:18:03.921 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "dump_on_update", 2024-04-24T09:18:03.921 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.921 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.921 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.921 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:03.921 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.921 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.922 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.922 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "dump database to debug log on update", 2024-04-24T09:18:03.922 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.922 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.922 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.922 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.922 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-24T09:18:03.922 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-24T09:18:03.922 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.922 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.922 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.922 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:03.922 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.922 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.922 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:03.922 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:03.923 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:03.923 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:03.923 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:03.923 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:03.923 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:03.923 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:03.923 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.923 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.923 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.923 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.923 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.923 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-24T09:18:03.923 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-24T09:18:03.923 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.923 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.923 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.924 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:03.924 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.924 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.924 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.924 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.924 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.924 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.924 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.924 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.924 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-24T09:18:03.924 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-24T09:18:03.924 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.924 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.924 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.924 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-24T09:18:03.924 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.925 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.925 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:03.925 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:03.925 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:03.925 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:03.925 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:03.925 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:03.925 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:03.925 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:03.925 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.925 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.925 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.925 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.925 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.925 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-24T09:18:03.925 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-24T09:18:03.926 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.926 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.926 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.926 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:03.926 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.926 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.926 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.926 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.926 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.926 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.926 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.926 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:03.926 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:03.926 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.926 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:03.927 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "stats", 2024-04-24T09:18:03.927 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-24T09:18:03.927 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-24T09:18:03.927 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-24T09:18:03.927 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-24T09:18:03.927 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-24T09:18:03.927 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.927 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.927 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.927 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:03.927 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.927 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.927 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:03.927 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:03.927 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:03.927 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:03.927 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:03.928 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:03.928 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:03.928 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:03.928 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.928 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.928 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.928 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.928 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.928 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-24T09:18:03.928 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-24T09:18:03.928 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.928 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.928 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.928 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:03.928 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.928 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.929 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.929 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.929 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.929 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.929 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.929 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.929 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-24T09:18:03.929 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-24T09:18:03.929 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.929 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.929 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.929 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-24T09:18:03.929 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.929 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.929 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:03.929 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:03.930 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:03.930 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:03.930 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:03.930 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:03.930 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:03.930 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:03.930 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.930 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.930 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.930 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.930 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.930 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-24T09:18:03.930 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-24T09:18:03.930 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.930 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.931 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.931 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:03.931 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.931 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.931 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.931 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.931 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.931 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.931 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.931 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:03.931 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:03.931 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.931 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:03.931 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "status", 2024-04-24T09:18:03.931 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-24T09:18:03.931 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-24T09:18:03.932 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-24T09:18:03.932 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-24T09:18:03.932 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-24T09:18:03.932 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.932 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.932 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.932 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:03.932 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.932 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.932 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:03.932 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:03.932 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:03.932 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:03.932 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:03.932 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:03.933 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:03.933 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:03.933 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.933 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.933 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.933 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.933 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.933 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-24T09:18:03.933 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-24T09:18:03.933 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.933 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.933 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.933 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:03.933 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.933 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.934 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.934 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.934 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.934 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.934 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.934 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.934 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-24T09:18:03.934 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-24T09:18:03.934 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.934 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.934 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.934 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-24T09:18:03.934 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.934 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.935 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:03.935 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:03.935 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:03.935 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:03.935 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:03.935 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:03.935 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:03.935 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:03.935 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.935 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.935 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.935 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.935 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.935 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-24T09:18:03.935 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-24T09:18:03.935 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.936 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.936 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.936 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:03.936 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.936 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.936 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.936 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.936 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.936 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.936 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.936 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:03.936 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:03.936 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.936 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:03.936 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "telegraf", 2024-04-24T09:18:03.936 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-24T09:18:03.937 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-24T09:18:03.937 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-24T09:18:03.937 INFO:teuthology.orchestra.run.smithi083.stdout: "address": { 2024-04-24T09:18:03.937 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "address", 2024-04-24T09:18:03.937 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.937 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.937 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.937 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-24T09:18:03.937 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.937 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.937 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.937 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.937 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.937 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.937 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.938 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.938 INFO:teuthology.orchestra.run.smithi083.stdout: "interval": { 2024-04-24T09:18:03.938 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "interval", 2024-04-24T09:18:03.938 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-24T09:18:03.938 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.938 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.938 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "15", 2024-04-24T09:18:03.938 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.938 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.938 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.938 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.938 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.938 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.938 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.938 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.939 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-24T09:18:03.939 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-24T09:18:03.939 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.939 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.939 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.939 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:03.939 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.939 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.939 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:03.939 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:03.939 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:03.939 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:03.939 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:03.939 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:03.939 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:03.940 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:03.940 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.940 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.940 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.940 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.940 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.940 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-24T09:18:03.940 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-24T09:18:03.940 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.940 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.940 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.940 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:03.940 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.940 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.940 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.941 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.941 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.941 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.941 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.941 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.941 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-24T09:18:03.941 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-24T09:18:03.941 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.941 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.941 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.941 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-24T09:18:03.941 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.941 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.941 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:03.941 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:03.941 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:03.942 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:03.942 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:03.942 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:03.942 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:03.942 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:03.942 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.942 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.942 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.942 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.942 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.942 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-24T09:18:03.942 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-24T09:18:03.942 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.942 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.942 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.942 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:03.943 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.943 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.943 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.943 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.943 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.943 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.943 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.943 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:03.943 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:03.943 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.943 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:03.943 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "telemetry", 2024-04-24T09:18:03.943 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-24T09:18:03.943 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-24T09:18:03.943 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-24T09:18:03.943 INFO:teuthology.orchestra.run.smithi083.stdout: "channel_basic": { 2024-04-24T09:18:03.944 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "channel_basic", 2024-04-24T09:18:03.944 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.944 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.944 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.944 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:03.944 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.944 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.944 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.944 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-24T09:18:03.944 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.944 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.944 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.944 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.944 INFO:teuthology.orchestra.run.smithi083.stdout: "channel_crash": { 2024-04-24T09:18:03.944 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "channel_crash", 2024-04-24T09:18:03.945 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.945 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.945 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.945 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:03.945 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.945 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.945 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.945 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-24T09:18:03.945 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.945 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.945 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.945 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.945 INFO:teuthology.orchestra.run.smithi083.stdout: "channel_device": { 2024-04-24T09:18:03.945 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "channel_device", 2024-04-24T09:18:03.945 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.945 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.945 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.946 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:03.946 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.946 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.946 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.946 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-24T09:18:03.946 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.946 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.946 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.946 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.946 INFO:teuthology.orchestra.run.smithi083.stdout: "channel_ident": { 2024-04-24T09:18:03.946 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "channel_ident", 2024-04-24T09:18:03.946 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.946 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.946 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.947 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:03.947 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.947 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.947 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.947 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-24T09:18:03.947 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.947 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.947 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.947 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.947 INFO:teuthology.orchestra.run.smithi083.stdout: "channel_perf": { 2024-04-24T09:18:03.947 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "channel_perf", 2024-04-24T09:18:03.947 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.947 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.947 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.947 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:03.947 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.948 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.948 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.948 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-24T09:18:03.948 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.948 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.948 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.948 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.948 INFO:teuthology.orchestra.run.smithi083.stdout: "contact": { 2024-04-24T09:18:03.948 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "contact", 2024-04-24T09:18:03.948 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.948 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.948 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.948 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:03.948 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.948 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.948 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.949 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.949 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.949 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.949 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.949 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.949 INFO:teuthology.orchestra.run.smithi083.stdout: "description": { 2024-04-24T09:18:03.949 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "description", 2024-04-24T09:18:03.949 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.949 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.949 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.949 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:03.949 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.949 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.949 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.949 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.949 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.950 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.950 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.950 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.950 INFO:teuthology.orchestra.run.smithi083.stdout: "device_url": { 2024-04-24T09:18:03.950 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "device_url", 2024-04-24T09:18:03.950 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.950 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.950 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.950 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-24T09:18:03.950 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.950 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.950 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.950 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.950 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.951 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.951 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.951 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.951 INFO:teuthology.orchestra.run.smithi083.stdout: "enabled": { 2024-04-24T09:18:03.951 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "enabled", 2024-04-24T09:18:03.951 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.951 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.951 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.951 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:03.951 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.951 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.951 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.951 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.951 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.951 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.951 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.952 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.952 INFO:teuthology.orchestra.run.smithi083.stdout: "interval": { 2024-04-24T09:18:03.952 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "interval", 2024-04-24T09:18:03.952 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:03.952 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.952 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.952 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "24", 2024-04-24T09:18:03.952 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "8", 2024-04-24T09:18:03.952 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.952 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.952 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.952 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.952 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.952 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.952 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.952 INFO:teuthology.orchestra.run.smithi083.stdout: "last_opt_revision": { 2024-04-24T09:18:03.953 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "last_opt_revision", 2024-04-24T09:18:03.953 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:03.953 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.953 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.953 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "1", 2024-04-24T09:18:03.953 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.953 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.953 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.953 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.953 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.953 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.953 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.953 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.953 INFO:teuthology.orchestra.run.smithi083.stdout: "leaderboard": { 2024-04-24T09:18:03.953 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "leaderboard", 2024-04-24T09:18:03.953 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.953 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.954 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.954 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:03.954 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.954 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.954 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.954 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.954 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.954 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.954 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.954 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.954 INFO:teuthology.orchestra.run.smithi083.stdout: "leaderboard_description": { 2024-04-24T09:18:03.954 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "leaderboard_description", 2024-04-24T09:18:03.954 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.954 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.954 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.954 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:03.955 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.955 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.955 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.955 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.955 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.955 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.955 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.955 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.955 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-24T09:18:03.955 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-24T09:18:03.955 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.955 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.955 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.955 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:03.955 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.955 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.956 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:03.956 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:03.956 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:03.956 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:03.956 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:03.956 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:03.956 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:03.956 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:03.956 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.956 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.956 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.956 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.956 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.956 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-24T09:18:03.956 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-24T09:18:03.956 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.956 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.957 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.957 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:03.957 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.957 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.957 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.957 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.957 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.957 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.957 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.957 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.957 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-24T09:18:03.957 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-24T09:18:03.957 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.957 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.957 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.957 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-24T09:18:03.958 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.958 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.958 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:03.958 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:03.958 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:03.958 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:03.958 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:03.958 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:03.958 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:03.958 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:03.958 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.958 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.958 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.958 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.958 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.959 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-24T09:18:03.959 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-24T09:18:03.959 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.959 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.959 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.959 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:03.959 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.959 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.959 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.959 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.959 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.959 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.959 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.959 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.959 INFO:teuthology.orchestra.run.smithi083.stdout: "organization": { 2024-04-24T09:18:03.959 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "organization", 2024-04-24T09:18:03.960 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.960 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.960 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.960 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:03.960 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.960 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.960 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.960 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.960 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.960 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.960 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.960 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.960 INFO:teuthology.orchestra.run.smithi083.stdout: "proxy": { 2024-04-24T09:18:03.960 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "proxy", 2024-04-24T09:18:03.961 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.961 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.961 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.961 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:03.961 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.961 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.961 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.961 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.961 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.961 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.961 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.961 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.961 INFO:teuthology.orchestra.run.smithi083.stdout: "url": { 2024-04-24T09:18:03.961 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "url", 2024-04-24T09:18:03.961 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.961 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.962 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.962 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-24T09:18:03.962 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.962 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.962 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.962 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.962 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.962 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.962 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.962 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:03.962 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:03.962 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.962 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:03.962 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "test_orchestrator", 2024-04-24T09:18:03.962 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-24T09:18:03.963 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-24T09:18:03.963 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-24T09:18:03.963 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-24T09:18:03.963 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-24T09:18:03.963 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.963 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.963 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.963 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:03.963 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.963 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.963 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:03.963 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:03.963 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:03.963 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:03.963 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:03.964 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:03.964 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:03.964 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:03.964 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.964 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.964 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.964 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.964 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.964 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-24T09:18:03.964 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-24T09:18:03.964 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.964 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.964 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.964 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:03.964 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.964 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.964 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.965 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.965 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.965 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.965 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.965 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.965 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-24T09:18:03.965 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-24T09:18:03.965 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.965 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.965 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.965 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-24T09:18:03.965 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.965 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.965 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:03.965 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:03.965 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:03.966 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:03.966 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:03.966 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:03.966 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:03.966 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:03.966 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.966 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.966 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.966 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.966 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.966 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-24T09:18:03.966 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-24T09:18:03.966 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.966 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.966 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.966 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:03.967 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.967 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.967 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.967 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.967 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.967 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.967 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.967 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:03.967 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:03.967 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.967 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:03.967 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "volumes", 2024-04-24T09:18:03.967 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-24T09:18:03.967 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-24T09:18:03.967 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-24T09:18:03.967 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-24T09:18:03.968 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-24T09:18:03.968 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.968 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.968 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.968 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:03.968 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.968 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.968 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:03.968 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:03.968 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:03.968 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:03.968 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:03.968 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:03.968 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:03.968 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:03.968 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.969 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.969 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.969 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.969 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.969 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-24T09:18:03.969 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-24T09:18:03.969 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.969 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.969 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.969 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:03.969 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.969 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.969 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.969 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.970 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.970 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.970 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.970 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.970 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-24T09:18:03.970 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-24T09:18:03.970 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.970 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.970 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.970 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-24T09:18:03.970 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.970 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.970 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:03.970 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:03.970 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:03.970 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:03.971 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:03.971 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:03.971 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:03.971 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:03.971 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.971 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.971 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.971 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.971 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.971 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-24T09:18:03.971 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-24T09:18:03.971 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.971 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.971 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.971 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:03.972 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.972 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.972 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.972 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.972 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.972 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.972 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.972 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.972 INFO:teuthology.orchestra.run.smithi083.stdout: "max_concurrent_clones": { 2024-04-24T09:18:03.972 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "max_concurrent_clones", 2024-04-24T09:18:03.972 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:03.972 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.972 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.972 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "4", 2024-04-24T09:18:03.972 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.972 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.973 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.973 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-24T09:18:03.973 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.973 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.973 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.973 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.973 INFO:teuthology.orchestra.run.smithi083.stdout: "periodic_async_work": { 2024-04-24T09:18:03.973 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "periodic_async_work", 2024-04-24T09:18:03.973 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.973 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.973 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.973 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:03.973 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.973 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.973 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.973 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Periodically check for async work", 2024-04-24T09:18:03.974 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.974 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.974 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.974 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.974 INFO:teuthology.orchestra.run.smithi083.stdout: "snapshot_clone_delay": { 2024-04-24T09:18:03.974 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "snapshot_clone_delay", 2024-04-24T09:18:03.974 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:03.974 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.974 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.974 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-04-24T09:18:03.974 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.974 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.974 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.974 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-24T09:18:03.974 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.975 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.975 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.975 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.975 INFO:teuthology.orchestra.run.smithi083.stdout: "snapshot_clone_no_wait": { 2024-04-24T09:18:03.975 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "snapshot_clone_no_wait", 2024-04-24T09:18:03.975 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.975 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.975 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.975 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:03.975 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.975 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.975 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.975 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-24T09:18:03.975 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.975 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.976 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.976 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:03.976 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:03.976 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.976 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:03.976 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "zabbix", 2024-04-24T09:18:03.976 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-24T09:18:03.976 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-24T09:18:03.976 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-24T09:18:03.976 INFO:teuthology.orchestra.run.smithi083.stdout: "discovery_interval": { 2024-04-24T09:18:03.976 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "discovery_interval", 2024-04-24T09:18:03.976 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "uint", 2024-04-24T09:18:03.976 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.976 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.976 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "100", 2024-04-24T09:18:03.976 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.977 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.977 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.977 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.977 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.977 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.977 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.977 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.977 INFO:teuthology.orchestra.run.smithi083.stdout: "identifier": { 2024-04-24T09:18:03.977 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "identifier", 2024-04-24T09:18:03.977 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.977 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.977 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.977 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:03.977 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.977 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.977 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.978 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.978 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.978 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.978 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.978 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.978 INFO:teuthology.orchestra.run.smithi083.stdout: "interval": { 2024-04-24T09:18:03.978 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "interval", 2024-04-24T09:18:03.978 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-24T09:18:03.978 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.978 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.978 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "60", 2024-04-24T09:18:03.978 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.978 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.978 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.978 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.979 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.979 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.979 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.979 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.979 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-24T09:18:03.979 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-24T09:18:03.979 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.979 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.979 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.979 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:03.979 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.979 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.979 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:03.979 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:03.979 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:03.980 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:03.980 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:03.980 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:03.980 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:03.980 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:03.980 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.980 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.980 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.980 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.980 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.980 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-24T09:18:03.980 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-24T09:18:03.980 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.980 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.980 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.981 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:03.981 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.981 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.981 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.981 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.981 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.981 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.981 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.981 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.981 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-24T09:18:03.981 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-24T09:18:03.981 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.981 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.981 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.981 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-24T09:18:03.981 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.982 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.982 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:03.982 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:03.982 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:03.982 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:03.982 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:03.982 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:03.982 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:03.982 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:03.982 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.982 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.982 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.982 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.982 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.982 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-24T09:18:03.982 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-24T09:18:03.983 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.983 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.983 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.983 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:03.983 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.983 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.983 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.983 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.983 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.983 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.983 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.983 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.983 INFO:teuthology.orchestra.run.smithi083.stdout: "zabbix_host": { 2024-04-24T09:18:03.983 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "zabbix_host", 2024-04-24T09:18:03.983 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.984 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.984 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.984 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:03.984 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.984 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.984 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.984 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.984 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.984 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.984 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.984 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.984 INFO:teuthology.orchestra.run.smithi083.stdout: "zabbix_port": { 2024-04-24T09:18:03.984 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "zabbix_port", 2024-04-24T09:18:03.984 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:03.984 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.985 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.985 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10051", 2024-04-24T09:18:03.985 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.985 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.985 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.985 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.985 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.985 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.985 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.985 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.985 INFO:teuthology.orchestra.run.smithi083.stdout: "zabbix_sender": { 2024-04-24T09:18:03.985 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "zabbix_sender", 2024-04-24T09:18:03.985 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.985 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.985 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:03.985 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-24T09:18:03.986 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.986 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.986 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.986 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.986 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.986 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.986 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.986 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:03.986 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:03.986 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:03.986 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-04-24T09:18:03.986 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.986 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:03.987 INFO:teuthology.orchestra.run.smithi083.stdout: "gid": 35060, 2024-04-24T09:18:03.987 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "y", 2024-04-24T09:18:03.987 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr_features": 4540701547738038271, 2024-04-24T09:18:03.987 INFO:teuthology.orchestra.run.smithi083.stdout: "available_modules": [ 2024-04-24T09:18:03.987 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:03.987 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "alerts", 2024-04-24T09:18:03.987 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-24T09:18:03.987 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-24T09:18:03.987 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-24T09:18:03.987 INFO:teuthology.orchestra.run.smithi083.stdout: "interval": { 2024-04-24T09:18:03.987 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "interval", 2024-04-24T09:18:03.987 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-24T09:18:03.987 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.987 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.987 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "60", 2024-04-24T09:18:03.987 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.988 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.988 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.988 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "How frequently to reexamine health status", 2024-04-24T09:18:03.988 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.988 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.988 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.988 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.988 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-24T09:18:03.988 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-24T09:18:03.988 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.988 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.988 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.988 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:03.989 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.989 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.989 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:03.989 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:03.989 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:03.989 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:03.989 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:03.989 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:03.989 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:03.989 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:03.989 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.989 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.989 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.989 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.989 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.990 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-24T09:18:03.990 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-24T09:18:03.990 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.990 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.990 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.990 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:03.990 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.990 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.990 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.990 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.990 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.990 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.990 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.990 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.990 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-24T09:18:03.991 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-24T09:18:03.991 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.991 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.991 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.991 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-24T09:18:03.991 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.991 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.991 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:03.991 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:03.991 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:03.991 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:03.991 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:03.991 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:03.991 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:03.991 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:03.992 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.992 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.992 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.992 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.992 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.992 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-24T09:18:03.992 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-24T09:18:03.992 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.992 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.992 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.992 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:03.992 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.992 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.992 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.992 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:03.992 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.992 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.993 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.993 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.993 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_destination": { 2024-04-24T09:18:03.993 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_destination", 2024-04-24T09:18:03.993 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.993 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.993 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.993 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:03.993 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.993 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.993 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.993 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Email address to send alerts to", 2024-04-24T09:18:03.993 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.993 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.993 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.993 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.993 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_from_name": { 2024-04-24T09:18:03.994 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_from_name", 2024-04-24T09:18:03.994 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.994 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.994 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.994 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "Ceph", 2024-04-24T09:18:03.994 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.994 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.994 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.994 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Email From: name", 2024-04-24T09:18:03.994 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.994 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.994 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.994 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.994 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_host": { 2024-04-24T09:18:03.994 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_host", 2024-04-24T09:18:03.995 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.995 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.995 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.995 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:03.995 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.995 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.995 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.995 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "SMTP server", 2024-04-24T09:18:03.995 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.995 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.995 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.995 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.995 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_password": { 2024-04-24T09:18:03.995 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_password", 2024-04-24T09:18:03.995 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.995 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.996 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.996 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:03.996 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.996 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.996 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.996 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Password to authenticate with", 2024-04-24T09:18:03.996 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.996 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.996 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.996 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.996 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_port": { 2024-04-24T09:18:03.996 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_port", 2024-04-24T09:18:03.996 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:03.996 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.996 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.997 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "465", 2024-04-24T09:18:03.997 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.997 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.997 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.997 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "SMTP port", 2024-04-24T09:18:03.997 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.997 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.997 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.997 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.997 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_sender": { 2024-04-24T09:18:03.997 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_sender", 2024-04-24T09:18:03.997 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.997 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.997 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.997 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:03.997 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.998 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.998 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.998 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "SMTP envelope sender", 2024-04-24T09:18:03.998 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.998 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.998 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.998 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.998 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_ssl": { 2024-04-24T09:18:03.998 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_ssl", 2024-04-24T09:18:03.998 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:03.998 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.998 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.998 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:03.998 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.998 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.998 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:03.999 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-24T09:18:03.999 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:03.999 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:03.999 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:03.999 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:03.999 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_user": { 2024-04-24T09:18:03.999 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_user", 2024-04-24T09:18:03.999 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:03.999 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:03.999 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:03.999 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:03.999 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:03.999 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:03.999 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.000 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "User to authenticate as", 2024-04-24T09:18:04.000 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.000 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.000 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.000 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.000 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.000 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.000 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:04.000 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "balancer", 2024-04-24T09:18:04.000 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-24T09:18:04.000 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-24T09:18:04.000 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-24T09:18:04.000 INFO:teuthology.orchestra.run.smithi083.stdout: "active": { 2024-04-24T09:18:04.000 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "active", 2024-04-24T09:18:04.001 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.001 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.001 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.001 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:04.001 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.001 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.001 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.001 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "automatically balance PGs across cluster", 2024-04-24T09:18:04.001 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.001 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.001 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.001 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.001 INFO:teuthology.orchestra.run.smithi083.stdout: "begin_time": { 2024-04-24T09:18:04.001 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "begin_time", 2024-04-24T09:18:04.001 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.002 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.002 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.002 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0000", 2024-04-24T09:18:04.002 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.002 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.002 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.002 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "beginning time of day to automatically balance", 2024-04-24T09:18:04.002 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-24T09:18:04.002 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.002 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.002 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.002 INFO:teuthology.orchestra.run.smithi083.stdout: "begin_weekday": { 2024-04-24T09:18:04.002 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "begin_weekday", 2024-04-24T09:18:04.002 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "uint", 2024-04-24T09:18:04.002 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.002 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.003 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-04-24T09:18:04.003 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "0", 2024-04-24T09:18:04.003 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "6", 2024-04-24T09:18:04.003 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.003 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-24T09:18:04.003 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-24T09:18:04.003 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.003 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.003 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.003 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_compat_max_iterations": { 2024-04-24T09:18:04.003 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "crush_compat_max_iterations", 2024-04-24T09:18:04.003 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "uint", 2024-04-24T09:18:04.003 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.003 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.003 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "25", 2024-04-24T09:18:04.003 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "1", 2024-04-24T09:18:04.004 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "250", 2024-04-24T09:18:04.004 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.004 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-24T09:18:04.004 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.004 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.004 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.004 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.004 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_compat_metrics": { 2024-04-24T09:18:04.004 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "crush_compat_metrics", 2024-04-24T09:18:04.004 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.004 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.004 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.004 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "pgs,objects,bytes", 2024-04-24T09:18:04.004 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.004 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.004 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.005 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-24T09:18:04.005 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-24T09:18:04.005 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.005 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.005 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.005 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_compat_step": { 2024-04-24T09:18:04.005 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "crush_compat_step", 2024-04-24T09:18:04.005 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-04-24T09:18:04.005 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.005 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.005 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0.5", 2024-04-24T09:18:04.005 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "0.001", 2024-04-24T09:18:04.005 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "0.999", 2024-04-24T09:18:04.005 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.005 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "aggressiveness of optimization", 2024-04-24T09:18:04.006 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-24T09:18:04.006 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.006 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.006 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.006 INFO:teuthology.orchestra.run.smithi083.stdout: "end_time": { 2024-04-24T09:18:04.006 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "end_time", 2024-04-24T09:18:04.006 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.006 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.006 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.006 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "2359", 2024-04-24T09:18:04.006 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.006 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.006 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.006 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "ending time of day to automatically balance", 2024-04-24T09:18:04.006 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-24T09:18:04.007 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.007 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.007 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.007 INFO:teuthology.orchestra.run.smithi083.stdout: "end_weekday": { 2024-04-24T09:18:04.007 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "end_weekday", 2024-04-24T09:18:04.007 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "uint", 2024-04-24T09:18:04.007 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.007 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.007 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-04-24T09:18:04.007 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "0", 2024-04-24T09:18:04.007 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "6", 2024-04-24T09:18:04.007 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.007 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-24T09:18:04.007 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-24T09:18:04.007 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.007 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.008 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.008 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-24T09:18:04.008 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-24T09:18:04.008 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.008 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.008 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.008 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.008 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.008 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.008 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.008 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.008 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.008 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.008 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.008 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.008 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.009 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.009 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.009 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.009 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.009 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.009 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.009 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-24T09:18:04.009 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-24T09:18:04.009 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.009 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.009 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.009 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.009 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.009 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.009 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.009 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.010 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.010 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.010 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.010 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.010 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-24T09:18:04.010 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-24T09:18:04.010 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.010 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.010 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.010 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-24T09:18:04.010 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.010 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.010 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.011 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.011 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.011 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.011 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.011 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.011 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.011 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.011 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.011 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.011 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.011 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.011 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.011 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-24T09:18:04.011 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-24T09:18:04.011 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.011 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.012 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.012 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.012 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.012 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.012 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.012 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.012 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.012 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.012 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.012 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.012 INFO:teuthology.orchestra.run.smithi083.stdout: "min_score": { 2024-04-24T09:18:04.012 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "min_score", 2024-04-24T09:18:04.012 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-04-24T09:18:04.012 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.012 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.012 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-04-24T09:18:04.013 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.013 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.013 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.013 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-24T09:18:04.013 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.013 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.013 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.013 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.013 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": { 2024-04-24T09:18:04.013 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mode", 2024-04-24T09:18:04.013 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.013 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.013 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.013 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "upmap", 2024-04-24T09:18:04.013 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.013 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.014 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.014 INFO:teuthology.orchestra.run.smithi083.stdout: "crush-compat", 2024-04-24T09:18:04.014 INFO:teuthology.orchestra.run.smithi083.stdout: "none", 2024-04-24T09:18:04.014 INFO:teuthology.orchestra.run.smithi083.stdout: "read", 2024-04-24T09:18:04.014 INFO:teuthology.orchestra.run.smithi083.stdout: "upmap", 2024-04-24T09:18:04.014 INFO:teuthology.orchestra.run.smithi083.stdout: "upmap-read" 2024-04-24T09:18:04.014 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.014 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Balancer mode", 2024-04-24T09:18:04.014 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.014 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.014 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.014 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.014 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_ids": { 2024-04-24T09:18:04.014 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "pool_ids", 2024-04-24T09:18:04.014 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.015 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.015 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.015 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.015 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.015 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.015 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.015 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-24T09:18:04.015 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.015 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.015 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.015 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.015 INFO:teuthology.orchestra.run.smithi083.stdout: "sleep_interval": { 2024-04-24T09:18:04.015 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sleep_interval", 2024-04-24T09:18:04.015 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-24T09:18:04.015 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.015 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.016 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "60", 2024-04-24T09:18:04.016 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.016 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.016 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.016 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-24T09:18:04.016 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.016 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.016 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.016 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.016 INFO:teuthology.orchestra.run.smithi083.stdout: "upmap_max_deviation": { 2024-04-24T09:18:04.016 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "upmap_max_deviation", 2024-04-24T09:18:04.016 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:04.016 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.016 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.016 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "5", 2024-04-24T09:18:04.016 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "1", 2024-04-24T09:18:04.016 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.017 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.017 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-24T09:18:04.017 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-24T09:18:04.017 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.017 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.017 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.017 INFO:teuthology.orchestra.run.smithi083.stdout: "upmap_max_optimizations": { 2024-04-24T09:18:04.017 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "upmap_max_optimizations", 2024-04-24T09:18:04.017 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "uint", 2024-04-24T09:18:04.017 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.017 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.017 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-04-24T09:18:04.017 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.017 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.017 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.018 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-24T09:18:04.018 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.018 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.018 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.018 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.018 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.018 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.018 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:04.018 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "cephadm", 2024-04-24T09:18:04.018 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-24T09:18:04.018 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-24T09:18:04.018 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-24T09:18:04.018 INFO:teuthology.orchestra.run.smithi083.stdout: "agent_down_multiplier": { 2024-04-24T09:18:04.018 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "agent_down_multiplier", 2024-04-24T09:18:04.018 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-04-24T09:18:04.019 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.019 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.019 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "3.0", 2024-04-24T09:18:04.019 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.019 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.019 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.019 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-24T09:18:04.019 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.019 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.019 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.019 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.019 INFO:teuthology.orchestra.run.smithi083.stdout: "agent_refresh_rate": { 2024-04-24T09:18:04.019 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "agent_refresh_rate", 2024-04-24T09:18:04.019 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-24T09:18:04.020 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.020 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.020 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "20", 2024-04-24T09:18:04.020 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.020 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.020 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.020 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-24T09:18:04.020 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.020 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.020 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.020 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.020 INFO:teuthology.orchestra.run.smithi083.stdout: "agent_starting_port": { 2024-04-24T09:18:04.020 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "agent_starting_port", 2024-04-24T09:18:04.020 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:04.020 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.021 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.021 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "4721", 2024-04-24T09:18:04.021 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.021 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.021 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.021 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-24T09:18:04.021 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.021 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.021 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.021 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.021 INFO:teuthology.orchestra.run.smithi083.stdout: "allow_ptrace": { 2024-04-24T09:18:04.021 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "allow_ptrace", 2024-04-24T09:18:04.021 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.021 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.021 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.021 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.022 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.022 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.022 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.022 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-24T09:18:04.022 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-24T09:18:04.022 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.022 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.022 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.022 INFO:teuthology.orchestra.run.smithi083.stdout: "autotune_interval": { 2024-04-24T09:18:04.022 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "autotune_interval", 2024-04-24T09:18:04.022 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-24T09:18:04.022 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.022 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.022 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "600", 2024-04-24T09:18:04.022 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.023 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.023 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.023 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-24T09:18:04.023 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.023 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.023 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.023 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.023 INFO:teuthology.orchestra.run.smithi083.stdout: "autotune_memory_target_ratio": { 2024-04-24T09:18:04.023 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "autotune_memory_target_ratio", 2024-04-24T09:18:04.023 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-04-24T09:18:04.023 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.023 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.023 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0.7", 2024-04-24T09:18:04.023 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.023 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.023 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.024 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-24T09:18:04.024 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.024 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.024 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.024 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.024 INFO:teuthology.orchestra.run.smithi083.stdout: "cephadm_log_destination": { 2024-04-24T09:18:04.024 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "cephadm_log_destination", 2024-04-24T09:18:04.024 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.024 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.024 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.024 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.024 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.024 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.024 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.024 INFO:teuthology.orchestra.run.smithi083.stdout: "file", 2024-04-24T09:18:04.024 INFO:teuthology.orchestra.run.smithi083.stdout: "file,syslog", 2024-04-24T09:18:04.025 INFO:teuthology.orchestra.run.smithi083.stdout: "syslog" 2024-04-24T09:18:04.025 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.025 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-24T09:18:04.025 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.025 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.025 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.025 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.025 INFO:teuthology.orchestra.run.smithi083.stdout: "cgroups_split": { 2024-04-24T09:18:04.025 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "cgroups_split", 2024-04-24T09:18:04.025 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.025 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.025 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.025 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:04.025 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.025 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.025 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.026 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-24T09:18:04.026 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.026 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.026 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.026 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.026 INFO:teuthology.orchestra.run.smithi083.stdout: "config_checks_enabled": { 2024-04-24T09:18:04.026 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "config_checks_enabled", 2024-04-24T09:18:04.026 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.026 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.026 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.026 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.026 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.026 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.026 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.026 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-24T09:18:04.027 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.027 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.027 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.027 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.027 INFO:teuthology.orchestra.run.smithi083.stdout: "config_dashboard": { 2024-04-24T09:18:04.027 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "config_dashboard", 2024-04-24T09:18:04.027 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.027 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.027 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.027 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:04.027 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.027 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.027 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.027 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-24T09:18:04.027 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.027 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.028 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.028 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.028 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_alertmanager": { 2024-04-24T09:18:04.028 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_alertmanager", 2024-04-24T09:18:04.028 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.028 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.028 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.028 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-24T09:18:04.028 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.028 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.028 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.028 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Prometheus container image", 2024-04-24T09:18:04.028 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.028 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.028 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.028 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.028 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_base": { 2024-04-24T09:18:04.029 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_base", 2024-04-24T09:18:04.029 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.029 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.029 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.029 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-24T09:18:04.029 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.029 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.029 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.029 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Container image name, without the tag", 2024-04-24T09:18:04.029 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.029 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.029 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.029 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.029 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_elasticsearch": { 2024-04-24T09:18:04.029 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_elasticsearch", 2024-04-24T09:18:04.029 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.030 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.030 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.030 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-24T09:18:04.030 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.030 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.030 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.030 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "elasticsearch container image", 2024-04-24T09:18:04.030 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.030 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.030 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.030 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.030 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_grafana": { 2024-04-24T09:18:04.031 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_grafana", 2024-04-24T09:18:04.031 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.031 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.031 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.031 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-24T09:18:04.031 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.031 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.031 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.031 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Prometheus container image", 2024-04-24T09:18:04.031 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.031 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.031 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.031 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.031 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_haproxy": { 2024-04-24T09:18:04.031 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_haproxy", 2024-04-24T09:18:04.031 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.032 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.032 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.032 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-24T09:18:04.032 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.032 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.032 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.032 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "HAproxy container image", 2024-04-24T09:18:04.032 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.032 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.032 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.032 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.032 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_jaeger_agent": { 2024-04-24T09:18:04.032 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_jaeger_agent", 2024-04-24T09:18:04.032 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.033 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.033 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.033 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-24T09:18:04.033 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.033 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.033 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.033 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Jaeger agent container image", 2024-04-24T09:18:04.033 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.033 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.033 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.033 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.033 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_jaeger_collector": { 2024-04-24T09:18:04.033 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_jaeger_collector", 2024-04-24T09:18:04.033 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.033 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.034 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.034 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-24T09:18:04.034 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.034 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.034 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.034 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Jaeger collector container image", 2024-04-24T09:18:04.034 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.034 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.034 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.034 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.034 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_jaeger_query": { 2024-04-24T09:18:04.034 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_jaeger_query", 2024-04-24T09:18:04.034 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.035 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.035 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.035 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-24T09:18:04.035 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.035 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.035 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.035 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Jaeger query container image", 2024-04-24T09:18:04.035 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.035 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.035 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.035 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.035 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_keepalived": { 2024-04-24T09:18:04.035 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_keepalived", 2024-04-24T09:18:04.035 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.035 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.036 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.036 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-24T09:18:04.036 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.036 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.036 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.036 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Keepalived container image", 2024-04-24T09:18:04.036 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.036 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.036 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.036 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.036 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_loki": { 2024-04-24T09:18:04.036 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_loki", 2024-04-24T09:18:04.036 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.036 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.036 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.036 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-24T09:18:04.036 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.037 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.037 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.037 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Loki container image", 2024-04-24T09:18:04.037 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.037 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.037 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.037 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.037 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_node_exporter": { 2024-04-24T09:18:04.037 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_node_exporter", 2024-04-24T09:18:04.037 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.037 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.037 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.037 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-24T09:18:04.037 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.037 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.038 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.038 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Prometheus container image", 2024-04-24T09:18:04.038 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.038 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.038 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.038 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.038 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_nvmeof": { 2024-04-24T09:18:04.038 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_nvmeof", 2024-04-24T09:18:04.038 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.038 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.038 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.038 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-24T09:18:04.038 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.038 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.038 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.039 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Nvme-of container image", 2024-04-24T09:18:04.039 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.039 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.039 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.039 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.039 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_prometheus": { 2024-04-24T09:18:04.039 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_prometheus", 2024-04-24T09:18:04.039 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.039 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.039 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.039 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-24T09:18:04.039 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.039 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.039 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.039 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Prometheus container image", 2024-04-24T09:18:04.039 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.040 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.040 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.040 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.040 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_promtail": { 2024-04-24T09:18:04.040 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_promtail", 2024-04-24T09:18:04.040 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.040 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.040 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.040 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-24T09:18:04.040 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.040 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.040 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.040 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Promtail container image", 2024-04-24T09:18:04.040 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.041 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.041 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.041 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.041 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_samba": { 2024-04-24T09:18:04.041 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_samba", 2024-04-24T09:18:04.041 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.041 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.041 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.041 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-24T09:18:04.041 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.041 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.041 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.041 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Samba/SMB container image", 2024-04-24T09:18:04.041 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.041 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.042 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.042 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.042 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_snmp_gateway": { 2024-04-24T09:18:04.042 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_snmp_gateway", 2024-04-24T09:18:04.042 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.042 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.042 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.042 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-24T09:18:04.042 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.042 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.042 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.042 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "SNMP Gateway container image", 2024-04-24T09:18:04.042 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.042 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.042 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.043 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.043 INFO:teuthology.orchestra.run.smithi083.stdout: "container_init": { 2024-04-24T09:18:04.043 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_init", 2024-04-24T09:18:04.043 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.043 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.043 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.043 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:04.043 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.043 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.043 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.043 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Run podman/docker with `--init`", 2024-04-24T09:18:04.043 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.043 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.043 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.043 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.043 INFO:teuthology.orchestra.run.smithi083.stdout: "daemon_cache_timeout": { 2024-04-24T09:18:04.044 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "daemon_cache_timeout", 2024-04-24T09:18:04.044 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-24T09:18:04.044 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.044 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.044 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "600", 2024-04-24T09:18:04.044 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.044 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.044 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.044 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-24T09:18:04.044 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.044 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.044 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.044 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.044 INFO:teuthology.orchestra.run.smithi083.stdout: "default_cephadm_command_timeout": { 2024-04-24T09:18:04.044 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "default_cephadm_command_timeout", 2024-04-24T09:18:04.044 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:04.045 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.045 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.045 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "900", 2024-04-24T09:18:04.045 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.045 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.045 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.045 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-24T09:18:04.045 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.045 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.045 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.045 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.045 INFO:teuthology.orchestra.run.smithi083.stdout: "default_registry": { 2024-04-24T09:18:04.045 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "default_registry", 2024-04-24T09:18:04.045 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.045 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.046 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.046 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "docker.io", 2024-04-24T09:18:04.046 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.046 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.046 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.046 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-24T09:18:04.046 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.046 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.046 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.046 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.046 INFO:teuthology.orchestra.run.smithi083.stdout: "device_cache_timeout": { 2024-04-24T09:18:04.046 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "device_cache_timeout", 2024-04-24T09:18:04.046 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-24T09:18:04.046 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.046 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.047 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "1800", 2024-04-24T09:18:04.047 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.047 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.047 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.047 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "seconds to cache device inventory", 2024-04-24T09:18:04.047 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.047 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.047 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.047 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.047 INFO:teuthology.orchestra.run.smithi083.stdout: "device_enhanced_scan": { 2024-04-24T09:18:04.047 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "device_enhanced_scan", 2024-04-24T09:18:04.047 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.047 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.047 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.047 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.047 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.048 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.048 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.048 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-24T09:18:04.048 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.048 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.048 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.048 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.048 INFO:teuthology.orchestra.run.smithi083.stdout: "facts_cache_timeout": { 2024-04-24T09:18:04.048 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "facts_cache_timeout", 2024-04-24T09:18:04.048 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-24T09:18:04.048 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.048 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.048 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "60", 2024-04-24T09:18:04.048 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.048 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.048 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.048 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "seconds to cache host facts data", 2024-04-24T09:18:04.049 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.049 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.049 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.049 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.049 INFO:teuthology.orchestra.run.smithi083.stdout: "grafana_dashboards_path": { 2024-04-24T09:18:04.049 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "grafana_dashboards_path", 2024-04-24T09:18:04.049 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.049 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.049 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.049 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-24T09:18:04.049 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.049 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.049 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.049 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-24T09:18:04.049 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.049 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.050 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.050 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.050 INFO:teuthology.orchestra.run.smithi083.stdout: "host_check_interval": { 2024-04-24T09:18:04.050 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "host_check_interval", 2024-04-24T09:18:04.050 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-24T09:18:04.050 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.050 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.050 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "600", 2024-04-24T09:18:04.050 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.050 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.050 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.050 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how frequently to perform a host check", 2024-04-24T09:18:04.050 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.050 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.051 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.051 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.051 INFO:teuthology.orchestra.run.smithi083.stdout: "hw_monitoring": { 2024-04-24T09:18:04.051 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "hw_monitoring", 2024-04-24T09:18:04.051 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.051 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.051 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.051 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.051 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.051 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.051 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.051 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-24T09:18:04.051 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.051 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.051 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.051 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.052 INFO:teuthology.orchestra.run.smithi083.stdout: "inventory_list_all": { 2024-04-24T09:18:04.052 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "inventory_list_all", 2024-04-24T09:18:04.052 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.052 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.052 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.052 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.052 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.052 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.052 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.052 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-24T09:18:04.052 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.052 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.052 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.052 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.052 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-24T09:18:04.053 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-24T09:18:04.053 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.053 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.053 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.053 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.053 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.053 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.053 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.053 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.053 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.053 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.053 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.053 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.053 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.053 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.053 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.054 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.054 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.054 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.054 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.054 INFO:teuthology.orchestra.run.smithi083.stdout: "log_refresh_metadata": { 2024-04-24T09:18:04.054 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_refresh_metadata", 2024-04-24T09:18:04.054 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.054 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.054 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.054 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.054 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.054 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.054 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.054 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-24T09:18:04.054 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.055 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.055 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.055 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.055 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-24T09:18:04.055 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-24T09:18:04.055 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.055 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.055 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.055 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:04.055 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.055 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.055 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.055 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-24T09:18:04.055 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.055 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.055 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.055 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.056 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-24T09:18:04.056 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-24T09:18:04.056 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.056 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.056 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.056 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-24T09:18:04.056 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.056 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.056 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.056 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.056 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.056 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.056 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.056 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.056 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.056 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.057 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.057 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.057 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.057 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.057 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.057 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-24T09:18:04.057 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-24T09:18:04.057 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.057 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.057 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.057 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.057 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.057 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.057 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.057 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.057 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.058 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.058 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.058 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.058 INFO:teuthology.orchestra.run.smithi083.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-24T09:18:04.058 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-24T09:18:04.058 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.058 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.058 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.058 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.058 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.058 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.058 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.058 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-24T09:18:04.058 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.058 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.058 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.058 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.059 INFO:teuthology.orchestra.run.smithi083.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-24T09:18:04.059 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-24T09:18:04.059 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.059 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.059 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.059 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "*", 2024-04-24T09:18:04.059 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.059 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.059 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.059 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-24T09:18:04.059 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.059 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.059 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.059 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.059 INFO:teuthology.orchestra.run.smithi083.stdout: "max_count_per_host": { 2024-04-24T09:18:04.059 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "max_count_per_host", 2024-04-24T09:18:04.060 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:04.060 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.060 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.060 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-04-24T09:18:04.060 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.060 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.060 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.060 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "max number of daemons per service per host", 2024-04-24T09:18:04.060 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.060 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.060 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.060 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.060 INFO:teuthology.orchestra.run.smithi083.stdout: "max_osd_draining_count": { 2024-04-24T09:18:04.060 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "max_osd_draining_count", 2024-04-24T09:18:04.060 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:04.061 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.061 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.061 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-04-24T09:18:04.061 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.061 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.061 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.061 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-24T09:18:04.061 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.061 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.061 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.061 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.061 INFO:teuthology.orchestra.run.smithi083.stdout: "migration_current": { 2024-04-24T09:18:04.061 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "migration_current", 2024-04-24T09:18:04.061 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:04.061 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.061 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.062 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.062 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.062 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.062 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.062 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "internal - do not modify", 2024-04-24T09:18:04.062 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.062 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.062 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.062 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.062 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": { 2024-04-24T09:18:04.062 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mode", 2024-04-24T09:18:04.062 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.062 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.062 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.062 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "root", 2024-04-24T09:18:04.062 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.063 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.063 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.063 INFO:teuthology.orchestra.run.smithi083.stdout: "cephadm-package", 2024-04-24T09:18:04.063 INFO:teuthology.orchestra.run.smithi083.stdout: "root" 2024-04-24T09:18:04.063 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.063 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "mode for remote execution of cephadm", 2024-04-24T09:18:04.063 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.063 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.063 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.063 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.063 INFO:teuthology.orchestra.run.smithi083.stdout: "oob_default_addr": { 2024-04-24T09:18:04.063 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "oob_default_addr", 2024-04-24T09:18:04.063 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.063 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.063 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.064 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "169.254.1.1", 2024-04-24T09:18:04.064 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.064 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.064 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.064 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-24T09:18:04.064 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.064 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.064 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.064 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.064 INFO:teuthology.orchestra.run.smithi083.stdout: "prometheus_alerts_path": { 2024-04-24T09:18:04.064 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "prometheus_alerts_path", 2024-04-24T09:18:04.064 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.064 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.064 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.064 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-24T09:18:04.065 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.065 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.065 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.065 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-24T09:18:04.065 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.065 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.065 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.065 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.065 INFO:teuthology.orchestra.run.smithi083.stdout: "registry_insecure": { 2024-04-24T09:18:04.065 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "registry_insecure", 2024-04-24T09:18:04.065 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.065 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.065 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.065 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.065 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.065 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.066 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.066 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-24T09:18:04.066 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.066 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.066 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.066 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.066 INFO:teuthology.orchestra.run.smithi083.stdout: "registry_password": { 2024-04-24T09:18:04.066 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "registry_password", 2024-04-24T09:18:04.066 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.066 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.066 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.066 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.066 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.066 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.066 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.067 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-24T09:18:04.067 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.067 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.067 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.067 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.067 INFO:teuthology.orchestra.run.smithi083.stdout: "registry_url": { 2024-04-24T09:18:04.067 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "registry_url", 2024-04-24T09:18:04.067 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.067 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.067 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.067 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.067 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.067 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.067 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.067 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-24T09:18:04.067 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.068 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.068 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.068 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.068 INFO:teuthology.orchestra.run.smithi083.stdout: "registry_username": { 2024-04-24T09:18:04.068 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "registry_username", 2024-04-24T09:18:04.068 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.068 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.068 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.068 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.068 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.068 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.068 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.068 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-24T09:18:04.068 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.068 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.069 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.069 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.069 INFO:teuthology.orchestra.run.smithi083.stdout: "secure_monitoring_stack": { 2024-04-24T09:18:04.069 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "secure_monitoring_stack", 2024-04-24T09:18:04.069 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.069 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.069 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.069 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.069 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.069 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.069 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.069 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-24T09:18:04.069 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.069 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.069 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.070 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.070 INFO:teuthology.orchestra.run.smithi083.stdout: "service_discovery_port": { 2024-04-24T09:18:04.070 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "service_discovery_port", 2024-04-24T09:18:04.070 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:04.070 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.070 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.070 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "8765", 2024-04-24T09:18:04.070 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.070 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.070 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.070 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "cephadm service discovery port", 2024-04-24T09:18:04.070 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.070 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.070 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.070 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.071 INFO:teuthology.orchestra.run.smithi083.stdout: "ssh_config_file": { 2024-04-24T09:18:04.071 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ssh_config_file", 2024-04-24T09:18:04.071 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.071 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.071 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.071 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.071 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.071 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.071 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.071 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-24T09:18:04.071 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.071 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.071 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.071 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.071 INFO:teuthology.orchestra.run.smithi083.stdout: "use_agent": { 2024-04-24T09:18:04.071 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "use_agent", 2024-04-24T09:18:04.072 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.072 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.072 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.072 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.072 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.072 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.072 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.072 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-24T09:18:04.072 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.072 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.072 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.072 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.072 INFO:teuthology.orchestra.run.smithi083.stdout: "use_repo_digest": { 2024-04-24T09:18:04.072 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "use_repo_digest", 2024-04-24T09:18:04.072 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.072 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.073 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.073 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:04.073 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.073 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.073 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.073 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-24T09:18:04.073 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.073 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.073 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.073 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.073 INFO:teuthology.orchestra.run.smithi083.stdout: "warn_on_failed_host_check": { 2024-04-24T09:18:04.073 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "warn_on_failed_host_check", 2024-04-24T09:18:04.073 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.073 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.073 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.074 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:04.074 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.074 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.074 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.074 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "raise a health warning if the host check fails", 2024-04-24T09:18:04.074 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.074 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.074 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.074 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.074 INFO:teuthology.orchestra.run.smithi083.stdout: "warn_on_stray_daemons": { 2024-04-24T09:18:04.074 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "warn_on_stray_daemons", 2024-04-24T09:18:04.074 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.074 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.074 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.074 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:04.075 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.075 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.075 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.075 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-24T09:18:04.075 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.075 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.075 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.075 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.075 INFO:teuthology.orchestra.run.smithi083.stdout: "warn_on_stray_hosts": { 2024-04-24T09:18:04.075 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "warn_on_stray_hosts", 2024-04-24T09:18:04.075 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.075 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.075 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.075 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:04.075 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.075 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.076 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.076 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-24T09:18:04.076 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.076 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.076 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.076 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.076 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.076 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.076 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:04.076 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "crash", 2024-04-24T09:18:04.076 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-24T09:18:04.076 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-24T09:18:04.076 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-24T09:18:04.076 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-24T09:18:04.076 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-24T09:18:04.076 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.077 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.077 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.077 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.077 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.077 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.077 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.077 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.077 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.077 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.077 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.077 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.077 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.077 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.077 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.077 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.078 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.078 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.078 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.078 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-24T09:18:04.078 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-24T09:18:04.078 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.078 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.078 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.078 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.078 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.078 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.078 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.078 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.078 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.078 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.079 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.079 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.079 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-24T09:18:04.079 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-24T09:18:04.079 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.079 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.079 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.079 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-24T09:18:04.079 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.079 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.079 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.079 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.079 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.079 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.079 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.080 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.080 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.080 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.080 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.080 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.080 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.080 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.080 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.080 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-24T09:18:04.080 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-24T09:18:04.080 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.080 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.080 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.080 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.080 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.081 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.081 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.081 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.081 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.081 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.081 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.081 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.081 INFO:teuthology.orchestra.run.smithi083.stdout: "retain_interval": { 2024-04-24T09:18:04.081 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "retain_interval", 2024-04-24T09:18:04.081 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-24T09:18:04.081 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.081 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.081 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "31536000", 2024-04-24T09:18:04.081 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.081 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.082 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.082 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-24T09:18:04.082 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.082 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.082 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.082 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.082 INFO:teuthology.orchestra.run.smithi083.stdout: "warn_recent_interval": { 2024-04-24T09:18:04.082 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "warn_recent_interval", 2024-04-24T09:18:04.082 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-24T09:18:04.082 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.082 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.082 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "1209600", 2024-04-24T09:18:04.082 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.082 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.082 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.082 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-24T09:18:04.083 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.083 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.083 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.083 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.083 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.083 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.083 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:04.083 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "dashboard", 2024-04-24T09:18:04.083 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-24T09:18:04.083 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-24T09:18:04.083 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-24T09:18:04.083 INFO:teuthology.orchestra.run.smithi083.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-24T09:18:04.083 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-24T09:18:04.083 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:04.083 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.083 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.084 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-04-24T09:18:04.084 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.084 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.084 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.084 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.084 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.084 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.084 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.084 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.084 INFO:teuthology.orchestra.run.smithi083.stdout: "ALERTMANAGER_API_HOST": { 2024-04-24T09:18:04.084 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-24T09:18:04.084 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.084 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.084 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.084 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.084 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.085 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.085 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.085 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.085 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.085 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.085 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.085 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.085 INFO:teuthology.orchestra.run.smithi083.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-24T09:18:04.085 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-24T09:18:04.085 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.085 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.085 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.085 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:04.085 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.085 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.085 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.086 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.086 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.086 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.086 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.086 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.086 INFO:teuthology.orchestra.run.smithi083.stdout: "AUDIT_API_ENABLED": { 2024-04-24T09:18:04.086 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "AUDIT_API_ENABLED", 2024-04-24T09:18:04.086 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.086 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.086 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.086 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.086 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.086 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.086 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.086 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.087 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.087 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.087 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.087 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.087 INFO:teuthology.orchestra.run.smithi083.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-24T09:18:04.087 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-24T09:18:04.087 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.087 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.087 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.087 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:04.087 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.087 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.087 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.087 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.087 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.087 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.088 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.088 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.088 INFO:teuthology.orchestra.run.smithi083.stdout: "ENABLE_BROWSABLE_API": { 2024-04-24T09:18:04.088 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-24T09:18:04.088 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.088 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.088 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.088 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:04.088 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.088 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.088 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.088 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.088 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.088 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.089 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.089 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.089 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-24T09:18:04.089 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-24T09:18:04.089 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.089 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.089 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.089 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:04.089 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.089 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.089 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.089 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.089 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.089 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.089 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.089 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.090 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-24T09:18:04.090 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-24T09:18:04.090 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.090 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.090 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.090 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:04.090 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.090 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.090 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.090 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.090 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.090 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.090 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.090 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.091 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-24T09:18:04.091 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-24T09:18:04.091 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.091 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.091 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.091 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:04.091 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.091 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.091 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.091 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.091 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.091 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.091 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.091 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.092 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-24T09:18:04.092 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-24T09:18:04.092 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.092 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.092 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.092 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:04.092 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.092 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.092 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.092 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.092 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.092 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.092 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.092 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.092 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-24T09:18:04.092 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-24T09:18:04.093 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.093 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.093 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.093 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:04.093 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.093 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.093 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.093 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.093 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.093 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.093 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.093 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.093 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-24T09:18:04.093 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-24T09:18:04.093 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.093 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.094 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.094 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:04.094 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.094 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.094 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.094 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.094 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.094 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.094 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.094 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.094 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-24T09:18:04.094 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-24T09:18:04.094 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.094 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.094 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.095 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:04.095 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.095 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.095 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.095 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.095 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.095 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.095 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.095 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.095 INFO:teuthology.orchestra.run.smithi083.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-24T09:18:04.095 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-24T09:18:04.095 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.095 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.095 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.095 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.095 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.095 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.096 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.096 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.096 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.096 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.096 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.096 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.096 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_API_PASSWORD": { 2024-04-24T09:18:04.096 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-24T09:18:04.096 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.096 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.096 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.096 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "admin", 2024-04-24T09:18:04.096 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.096 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.096 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.096 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.097 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.097 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.097 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.097 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.097 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-24T09:18:04.097 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-24T09:18:04.097 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.097 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.097 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.097 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:04.097 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.097 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.097 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.097 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.097 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.097 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.097 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.098 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.098 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_API_URL": { 2024-04-24T09:18:04.098 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_API_URL", 2024-04-24T09:18:04.098 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.098 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.098 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.098 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.098 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.098 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.098 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.098 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.098 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.098 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.098 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.098 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.099 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_API_USERNAME": { 2024-04-24T09:18:04.099 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-24T09:18:04.099 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.099 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.099 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.099 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "admin", 2024-04-24T09:18:04.099 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.099 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.099 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.099 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.099 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.099 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.099 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.099 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.099 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-24T09:18:04.100 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-24T09:18:04.100 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.100 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.100 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.100 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.100 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.100 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.100 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.100 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.100 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.100 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.100 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.100 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.100 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-24T09:18:04.100 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-24T09:18:04.101 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.101 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.101 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.101 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.101 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.101 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.101 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.101 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.101 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.101 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.101 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.101 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.101 INFO:teuthology.orchestra.run.smithi083.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-24T09:18:04.101 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-24T09:18:04.101 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.102 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.102 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.102 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:04.102 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.102 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.102 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.102 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.102 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.102 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.102 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.102 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.102 INFO:teuthology.orchestra.run.smithi083.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-24T09:18:04.102 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-24T09:18:04.102 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.103 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.103 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.103 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.103 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.103 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.103 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.103 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.103 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.103 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.103 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.103 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.103 INFO:teuthology.orchestra.run.smithi083.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-24T09:18:04.103 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-24T09:18:04.103 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.103 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.103 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.103 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "[]", 2024-04-24T09:18:04.104 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.104 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.104 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.104 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.104 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.104 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.104 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.104 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.104 INFO:teuthology.orchestra.run.smithi083.stdout: "MULTICLUSTER_CONFIG": { 2024-04-24T09:18:04.104 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-24T09:18:04.104 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.104 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.104 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.104 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "{}", 2024-04-24T09:18:04.104 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.104 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.105 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.105 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.105 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.105 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.105 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.105 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.105 INFO:teuthology.orchestra.run.smithi083.stdout: "PROMETHEUS_API_HOST": { 2024-04-24T09:18:04.105 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-24T09:18:04.105 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.105 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.105 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.105 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.105 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.105 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.105 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.105 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.106 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.106 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.106 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.106 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.106 INFO:teuthology.orchestra.run.smithi083.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-24T09:18:04.106 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-24T09:18:04.106 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.106 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.106 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.106 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:04.106 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.106 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.106 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.106 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.106 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.107 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.107 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.107 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.107 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-24T09:18:04.107 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-24T09:18:04.107 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.107 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.107 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.107 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.107 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.107 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.107 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.107 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.107 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.107 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.107 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.108 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.108 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-24T09:18:04.108 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-24T09:18:04.108 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.108 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.108 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.108 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.108 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.108 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.108 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.108 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.108 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.108 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.108 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.108 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.108 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-24T09:18:04.108 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-24T09:18:04.109 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.109 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.109 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.109 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:04.109 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.109 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.109 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.109 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.109 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.109 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.109 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.109 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.109 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-24T09:18:04.109 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-24T09:18:04.109 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.109 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.110 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.110 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:04.110 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.110 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.110 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.110 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.110 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.110 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.110 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.110 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.110 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-24T09:18:04.110 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-24T09:18:04.110 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.110 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.111 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.111 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.111 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.111 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.111 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.111 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.111 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.111 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.111 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.111 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.111 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-24T09:18:04.111 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-24T09:18:04.111 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.111 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.112 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.112 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.112 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.112 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.112 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.112 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.112 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.112 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.112 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.112 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.112 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-24T09:18:04.112 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-24T09:18:04.112 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.112 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.112 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.112 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.112 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.113 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.113 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.113 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.113 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.113 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.113 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.113 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.113 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_ENABLED": { 2024-04-24T09:18:04.113 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-24T09:18:04.113 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.113 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.113 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.113 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:04.113 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.113 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.113 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.114 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.114 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.114 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.114 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.114 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.114 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-24T09:18:04.114 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-24T09:18:04.114 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.114 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.114 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.114 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-24T09:18:04.114 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.114 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.114 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.114 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.115 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.115 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.115 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.115 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.115 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-24T09:18:04.115 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-24T09:18:04.115 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:04.115 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.115 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.115 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-04-24T09:18:04.115 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.115 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.115 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.115 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.115 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.115 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.116 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.116 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.116 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-24T09:18:04.116 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-24T09:18:04.116 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:04.116 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.116 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.116 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "8", 2024-04-24T09:18:04.116 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.116 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.116 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.116 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.116 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.116 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.116 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.116 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.116 INFO:teuthology.orchestra.run.smithi083.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-24T09:18:04.117 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-24T09:18:04.117 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:04.117 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.117 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.117 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "45", 2024-04-24T09:18:04.117 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.117 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.117 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.117 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.117 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.117 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.117 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.117 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.117 INFO:teuthology.orchestra.run.smithi083.stdout: "RGW_API_ACCESS_KEY": { 2024-04-24T09:18:04.117 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-24T09:18:04.117 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.118 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.118 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.118 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.118 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.118 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.118 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.118 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.118 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.118 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.118 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.118 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.118 INFO:teuthology.orchestra.run.smithi083.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-24T09:18:04.118 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-24T09:18:04.118 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.118 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.119 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.119 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "admin", 2024-04-24T09:18:04.119 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.119 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.119 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.119 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.119 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.119 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.119 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.119 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.119 INFO:teuthology.orchestra.run.smithi083.stdout: "RGW_API_SECRET_KEY": { 2024-04-24T09:18:04.119 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-24T09:18:04.119 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.119 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.119 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.119 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.119 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.120 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.120 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.120 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.120 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.120 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.120 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.120 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.120 INFO:teuthology.orchestra.run.smithi083.stdout: "RGW_API_SSL_VERIFY": { 2024-04-24T09:18:04.120 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-24T09:18:04.120 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.120 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.120 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.120 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:04.120 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.120 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.121 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.121 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.121 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.121 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.121 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.121 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.121 INFO:teuthology.orchestra.run.smithi083.stdout: "UNSAFE_TLS_v1_2": { 2024-04-24T09:18:04.121 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-24T09:18:04.121 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.121 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.121 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.121 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.121 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.121 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.121 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.121 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.121 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.122 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.122 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.122 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.122 INFO:teuthology.orchestra.run.smithi083.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-24T09:18:04.122 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-24T09:18:04.122 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:04.122 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.122 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.122 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-04-24T09:18:04.122 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.122 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.122 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.122 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.122 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.122 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.123 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.123 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.123 INFO:teuthology.orchestra.run.smithi083.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-24T09:18:04.123 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-24T09:18:04.123 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:04.123 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.123 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.123 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-04-24T09:18:04.123 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.123 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.123 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.123 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.123 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.123 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.123 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.123 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.124 INFO:teuthology.orchestra.run.smithi083.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-24T09:18:04.124 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-24T09:18:04.124 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:04.124 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.124 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.124 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "5", 2024-04-24T09:18:04.124 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.124 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.124 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.124 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.124 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.124 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.124 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.124 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.124 INFO:teuthology.orchestra.run.smithi083.stdout: "cross_origin_url": { 2024-04-24T09:18:04.124 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "cross_origin_url", 2024-04-24T09:18:04.125 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.125 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.125 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.125 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.125 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.125 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.125 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.125 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.125 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.125 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.125 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.125 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.125 INFO:teuthology.orchestra.run.smithi083.stdout: "crt_file": { 2024-04-24T09:18:04.125 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "crt_file", 2024-04-24T09:18:04.126 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.126 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.126 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.126 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.126 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.126 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.126 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.126 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.126 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.126 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.126 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.126 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.126 INFO:teuthology.orchestra.run.smithi083.stdout: "debug": { 2024-04-24T09:18:04.126 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "debug", 2024-04-24T09:18:04.126 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.127 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.127 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.127 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.127 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.127 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.127 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.127 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Enable/disable debug options", 2024-04-24T09:18:04.127 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.127 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.127 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.127 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.127 INFO:teuthology.orchestra.run.smithi083.stdout: "jwt_token_ttl": { 2024-04-24T09:18:04.127 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "jwt_token_ttl", 2024-04-24T09:18:04.127 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:04.127 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.127 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.128 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "28800", 2024-04-24T09:18:04.128 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.128 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.128 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.128 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.128 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.128 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.128 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.128 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.128 INFO:teuthology.orchestra.run.smithi083.stdout: "key_file": { 2024-04-24T09:18:04.128 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "key_file", 2024-04-24T09:18:04.128 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.128 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.128 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.128 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.128 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.128 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.129 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.129 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.129 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.129 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.129 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.129 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.129 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-24T09:18:04.129 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-24T09:18:04.129 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.129 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.129 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.129 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.129 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.129 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.129 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.129 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.130 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.130 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.130 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.130 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.130 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.130 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.130 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.130 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.130 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.130 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.130 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.130 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-24T09:18:04.130 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-24T09:18:04.130 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.131 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.131 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.131 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.131 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.131 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.131 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.131 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.131 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.131 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.131 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.131 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.131 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-24T09:18:04.131 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-24T09:18:04.131 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.131 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.131 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.132 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-24T09:18:04.132 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.132 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.132 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.132 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.132 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.132 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.132 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.132 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.132 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.132 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.132 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.132 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.132 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.132 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.132 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.133 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-24T09:18:04.133 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-24T09:18:04.133 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.133 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.133 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.133 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.133 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.133 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.133 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.133 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.133 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.133 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.133 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.133 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.133 INFO:teuthology.orchestra.run.smithi083.stdout: "motd": { 2024-04-24T09:18:04.133 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "motd", 2024-04-24T09:18:04.133 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.134 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.134 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.134 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.134 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.134 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.134 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.134 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "The message of the day", 2024-04-24T09:18:04.134 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.134 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.134 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.134 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.134 INFO:teuthology.orchestra.run.smithi083.stdout: "redirect_resolve_ip_addr": { 2024-04-24T09:18:04.134 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "redirect_resolve_ip_addr", 2024-04-24T09:18:04.134 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.135 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.135 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.135 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.135 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.135 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.135 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.135 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.135 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.135 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.135 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.135 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.135 INFO:teuthology.orchestra.run.smithi083.stdout: "server_addr": { 2024-04-24T09:18:04.135 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_addr", 2024-04-24T09:18:04.135 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.136 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.136 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.136 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "::", 2024-04-24T09:18:04.136 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.136 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.136 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.136 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.136 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.136 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.136 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.136 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.136 INFO:teuthology.orchestra.run.smithi083.stdout: "server_port": { 2024-04-24T09:18:04.136 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_port", 2024-04-24T09:18:04.136 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:04.136 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.137 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.137 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "8080", 2024-04-24T09:18:04.137 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.137 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.137 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.137 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.137 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.137 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.137 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.137 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.137 INFO:teuthology.orchestra.run.smithi083.stdout: "ssl": { 2024-04-24T09:18:04.137 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ssl", 2024-04-24T09:18:04.137 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.138 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.138 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.138 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:04.138 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.138 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.138 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.138 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.138 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.138 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.138 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.138 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.138 INFO:teuthology.orchestra.run.smithi083.stdout: "ssl_server_port": { 2024-04-24T09:18:04.138 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ssl_server_port", 2024-04-24T09:18:04.138 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:04.138 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.138 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.139 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "8443", 2024-04-24T09:18:04.139 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.139 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.139 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.139 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.139 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.139 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.139 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.139 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.139 INFO:teuthology.orchestra.run.smithi083.stdout: "standby_behaviour": { 2024-04-24T09:18:04.139 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "standby_behaviour", 2024-04-24T09:18:04.139 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.139 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.139 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.139 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "redirect", 2024-04-24T09:18:04.140 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.140 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.140 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.140 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.140 INFO:teuthology.orchestra.run.smithi083.stdout: "redirect" 2024-04-24T09:18:04.140 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.140 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.140 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.140 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.140 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.140 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.140 INFO:teuthology.orchestra.run.smithi083.stdout: "standby_error_status_code": { 2024-04-24T09:18:04.140 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "standby_error_status_code", 2024-04-24T09:18:04.140 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:04.140 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.141 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.141 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "500", 2024-04-24T09:18:04.141 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "400", 2024-04-24T09:18:04.141 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "599", 2024-04-24T09:18:04.141 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.141 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.141 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.141 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.141 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.141 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.141 INFO:teuthology.orchestra.run.smithi083.stdout: "url_prefix": { 2024-04-24T09:18:04.141 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "url_prefix", 2024-04-24T09:18:04.141 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.141 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.142 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.142 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.142 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.142 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.142 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.142 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.142 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.142 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.142 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.142 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.142 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.142 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.142 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:04.142 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "devicehealth", 2024-04-24T09:18:04.142 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-24T09:18:04.143 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-24T09:18:04.143 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-24T09:18:04.143 INFO:teuthology.orchestra.run.smithi083.stdout: "enable_monitoring": { 2024-04-24T09:18:04.143 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "enable_monitoring", 2024-04-24T09:18:04.143 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.143 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.143 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.143 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:04.143 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.143 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.143 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.143 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "monitor device health metrics", 2024-04-24T09:18:04.143 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.143 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.144 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.144 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.144 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-24T09:18:04.144 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-24T09:18:04.144 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.144 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.144 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.144 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.144 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.144 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.144 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.144 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.144 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.144 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.144 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.145 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.145 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.145 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.145 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.145 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.145 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.145 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.145 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.145 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-24T09:18:04.145 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-24T09:18:04.145 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.145 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.145 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.145 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.145 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.145 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.146 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.146 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.146 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.146 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.146 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.146 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.146 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-24T09:18:04.146 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-24T09:18:04.146 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.146 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.146 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.146 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-24T09:18:04.146 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.146 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.146 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.147 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.147 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.147 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.147 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.147 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.147 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.147 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.147 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.147 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.147 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.147 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.147 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.147 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-24T09:18:04.147 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-24T09:18:04.147 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.148 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.148 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.148 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.148 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.148 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.148 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.148 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.148 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.148 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.148 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.148 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.148 INFO:teuthology.orchestra.run.smithi083.stdout: "mark_out_threshold": { 2024-04-24T09:18:04.148 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mark_out_threshold", 2024-04-24T09:18:04.148 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-24T09:18:04.148 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.148 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.149 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "2419200", 2024-04-24T09:18:04.149 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.149 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.149 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.149 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-24T09:18:04.149 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.149 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.149 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.149 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.149 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_name": { 2024-04-24T09:18:04.149 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "pool_name", 2024-04-24T09:18:04.149 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.149 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.149 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.149 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "device_health_metrics", 2024-04-24T09:18:04.149 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.150 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.150 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.150 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-24T09:18:04.150 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.150 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.150 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.150 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.150 INFO:teuthology.orchestra.run.smithi083.stdout: "retention_period": { 2024-04-24T09:18:04.150 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "retention_period", 2024-04-24T09:18:04.150 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-24T09:18:04.150 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.150 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.150 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "15552000", 2024-04-24T09:18:04.151 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.151 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.151 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.151 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how long to retain device health metrics", 2024-04-24T09:18:04.151 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.151 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.151 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.151 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.151 INFO:teuthology.orchestra.run.smithi083.stdout: "scrape_frequency": { 2024-04-24T09:18:04.151 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "scrape_frequency", 2024-04-24T09:18:04.151 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-24T09:18:04.151 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.151 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.151 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "86400", 2024-04-24T09:18:04.152 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.152 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.152 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.152 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-24T09:18:04.152 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.152 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.152 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.152 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.152 INFO:teuthology.orchestra.run.smithi083.stdout: "self_heal": { 2024-04-24T09:18:04.152 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "self_heal", 2024-04-24T09:18:04.152 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.152 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.152 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.152 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:04.152 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.152 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.152 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.153 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-24T09:18:04.153 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.153 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.153 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.153 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.153 INFO:teuthology.orchestra.run.smithi083.stdout: "sleep_interval": { 2024-04-24T09:18:04.153 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sleep_interval", 2024-04-24T09:18:04.153 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-24T09:18:04.153 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.153 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.153 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "600", 2024-04-24T09:18:04.153 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.153 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.153 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.153 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how frequently to wake up and check device health", 2024-04-24T09:18:04.153 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.154 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.154 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.154 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.154 INFO:teuthology.orchestra.run.smithi083.stdout: "warn_threshold": { 2024-04-24T09:18:04.154 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "warn_threshold", 2024-04-24T09:18:04.154 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-24T09:18:04.154 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.154 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.154 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "7257600", 2024-04-24T09:18:04.154 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.154 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.154 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.154 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-24T09:18:04.154 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.154 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.154 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.155 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.155 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.155 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.155 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:04.155 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "diskprediction_local", 2024-04-24T09:18:04.155 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-24T09:18:04.155 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-24T09:18:04.155 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-24T09:18:04.155 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-24T09:18:04.155 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-24T09:18:04.155 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.155 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.155 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.155 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.156 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.156 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.156 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.156 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.156 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.156 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.156 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.156 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.156 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.156 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.156 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.156 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.156 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.156 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.156 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.156 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-24T09:18:04.156 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-24T09:18:04.157 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.157 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.157 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.157 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.157 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.157 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.157 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.157 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.157 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.157 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.157 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.157 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.157 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-24T09:18:04.157 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-24T09:18:04.157 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.157 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.158 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.158 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-24T09:18:04.158 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.158 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.158 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.158 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.158 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.158 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.158 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.158 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.158 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.158 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.158 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.158 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.158 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.158 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.159 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.159 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-24T09:18:04.159 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-24T09:18:04.159 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.159 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.159 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.159 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.159 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.159 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.159 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.159 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.159 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.159 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.159 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.159 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.159 INFO:teuthology.orchestra.run.smithi083.stdout: "predict_interval": { 2024-04-24T09:18:04.160 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "predict_interval", 2024-04-24T09:18:04.160 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.160 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.160 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.160 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "86400", 2024-04-24T09:18:04.160 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.160 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.160 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.160 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.160 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.160 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.160 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.160 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.160 INFO:teuthology.orchestra.run.smithi083.stdout: "predictor_model": { 2024-04-24T09:18:04.160 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "predictor_model", 2024-04-24T09:18:04.161 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.161 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.161 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.161 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "prophetstor", 2024-04-24T09:18:04.161 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.161 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.161 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.161 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.161 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.161 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.161 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.161 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.161 INFO:teuthology.orchestra.run.smithi083.stdout: "sleep_interval": { 2024-04-24T09:18:04.161 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sleep_interval", 2024-04-24T09:18:04.161 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.161 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.162 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.162 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "600", 2024-04-24T09:18:04.162 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.162 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.162 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.162 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.162 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.162 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.162 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.162 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.162 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.162 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.162 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:04.162 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "influx", 2024-04-24T09:18:04.162 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": false, 2024-04-24T09:18:04.162 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "influxdb python module not found", 2024-04-24T09:18:04.163 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-24T09:18:04.163 INFO:teuthology.orchestra.run.smithi083.stdout: "batch_size": { 2024-04-24T09:18:04.163 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "batch_size", 2024-04-24T09:18:04.163 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:04.163 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.163 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.163 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "5000", 2024-04-24T09:18:04.163 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.163 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.163 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.163 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-24T09:18:04.163 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.163 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.163 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.163 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.164 INFO:teuthology.orchestra.run.smithi083.stdout: "database": { 2024-04-24T09:18:04.164 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "database", 2024-04-24T09:18:04.164 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.164 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.164 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.164 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "ceph", 2024-04-24T09:18:04.164 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.164 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.164 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.164 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-24T09:18:04.164 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.164 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.164 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.164 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.164 INFO:teuthology.orchestra.run.smithi083.stdout: "hostname": { 2024-04-24T09:18:04.164 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "hostname", 2024-04-24T09:18:04.165 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.165 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.165 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.165 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.165 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.165 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.165 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.165 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "InfluxDB server hostname", 2024-04-24T09:18:04.165 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.165 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.165 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.165 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.165 INFO:teuthology.orchestra.run.smithi083.stdout: "interval": { 2024-04-24T09:18:04.165 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "interval", 2024-04-24T09:18:04.165 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-24T09:18:04.165 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.165 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.166 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "30", 2024-04-24T09:18:04.166 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "5", 2024-04-24T09:18:04.166 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.166 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.166 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-24T09:18:04.166 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.166 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.166 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.166 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.166 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-24T09:18:04.166 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-24T09:18:04.166 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.166 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.166 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.166 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.167 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.167 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.167 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.167 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.167 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.167 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.167 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.167 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.167 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.167 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.167 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.167 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.167 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.167 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.167 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.167 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-24T09:18:04.168 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-24T09:18:04.168 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.168 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.168 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.168 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.168 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.168 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.168 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.168 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.168 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.168 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.168 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.168 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.168 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-24T09:18:04.168 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-24T09:18:04.168 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.168 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.169 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.169 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-24T09:18:04.169 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.169 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.169 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.169 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.169 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.169 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.169 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.169 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.169 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.169 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.169 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.169 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.170 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.170 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.170 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.170 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-24T09:18:04.170 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-24T09:18:04.170 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.170 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.170 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.170 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.170 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.170 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.170 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.170 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.170 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.170 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.171 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.171 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.171 INFO:teuthology.orchestra.run.smithi083.stdout: "password": { 2024-04-24T09:18:04.171 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "password", 2024-04-24T09:18:04.171 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.171 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.171 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.171 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.171 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.171 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.171 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.171 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "password of InfluxDB server user", 2024-04-24T09:18:04.171 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.171 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.171 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.172 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.172 INFO:teuthology.orchestra.run.smithi083.stdout: "port": { 2024-04-24T09:18:04.172 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "port", 2024-04-24T09:18:04.172 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:04.172 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.172 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.172 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "8086", 2024-04-24T09:18:04.172 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.172 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.172 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.172 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "InfluxDB server port", 2024-04-24T09:18:04.172 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.172 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.172 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.172 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.173 INFO:teuthology.orchestra.run.smithi083.stdout: "ssl": { 2024-04-24T09:18:04.173 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ssl", 2024-04-24T09:18:04.173 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.173 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.173 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.173 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "false", 2024-04-24T09:18:04.173 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.173 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.173 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.173 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-24T09:18:04.173 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.173 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.173 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.173 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.174 INFO:teuthology.orchestra.run.smithi083.stdout: "threads": { 2024-04-24T09:18:04.174 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "threads", 2024-04-24T09:18:04.174 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:04.174 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.174 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.174 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "5", 2024-04-24T09:18:04.174 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "1", 2024-04-24T09:18:04.174 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "32", 2024-04-24T09:18:04.174 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.174 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-24T09:18:04.174 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.174 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.174 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.174 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.174 INFO:teuthology.orchestra.run.smithi083.stdout: "username": { 2024-04-24T09:18:04.174 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "username", 2024-04-24T09:18:04.175 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.175 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.175 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.175 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.175 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.175 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.175 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.175 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "username of InfluxDB server user", 2024-04-24T09:18:04.175 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.175 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.175 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.175 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.175 INFO:teuthology.orchestra.run.smithi083.stdout: "verify_ssl": { 2024-04-24T09:18:04.175 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "verify_ssl", 2024-04-24T09:18:04.175 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.175 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.176 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.176 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "true", 2024-04-24T09:18:04.176 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.176 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.176 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.176 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-24T09:18:04.176 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.176 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.176 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.176 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.176 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.176 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.176 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:04.176 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "insights", 2024-04-24T09:18:04.176 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-24T09:18:04.177 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-24T09:18:04.177 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-24T09:18:04.177 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-24T09:18:04.177 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-24T09:18:04.177 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.177 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.177 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.177 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.177 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.177 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.177 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.177 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.177 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.177 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.177 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.177 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.178 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.178 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.178 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.178 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.178 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.178 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.178 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.178 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-24T09:18:04.178 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-24T09:18:04.178 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.178 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.178 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.178 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.178 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.179 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.179 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.179 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.179 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.179 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.179 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.179 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.179 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-24T09:18:04.179 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-24T09:18:04.179 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.179 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.179 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.179 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-24T09:18:04.179 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.179 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.179 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.180 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.180 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.180 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.180 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.180 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.180 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.180 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.180 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.180 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.180 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.180 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.180 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.180 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-24T09:18:04.180 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-24T09:18:04.180 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.181 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.181 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.181 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.181 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.181 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.181 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.181 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.181 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.181 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.181 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.181 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.181 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.181 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.181 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:04.181 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "iostat", 2024-04-24T09:18:04.181 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-24T09:18:04.182 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-24T09:18:04.182 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-24T09:18:04.182 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-24T09:18:04.182 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-24T09:18:04.182 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.182 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.182 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.182 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.182 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.182 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.182 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.182 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.182 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.182 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.182 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.182 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.183 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.183 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.183 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.183 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.183 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.183 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.183 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.183 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-24T09:18:04.183 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-24T09:18:04.183 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.183 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.183 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.183 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.183 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.183 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.184 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.184 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.184 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.184 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.184 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.184 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.184 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-24T09:18:04.184 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-24T09:18:04.184 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.184 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.184 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.184 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-24T09:18:04.184 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.184 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.184 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.184 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.185 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.185 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.185 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.185 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.185 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.185 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.185 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.185 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.185 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.185 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.185 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.185 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-24T09:18:04.185 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-24T09:18:04.185 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.185 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.186 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.186 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.186 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.186 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.186 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.186 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.186 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.186 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.186 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.186 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.186 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.186 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.186 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:04.186 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "k8sevents", 2024-04-24T09:18:04.186 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-24T09:18:04.186 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-24T09:18:04.187 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-24T09:18:04.187 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_event_retention_days": { 2024-04-24T09:18:04.187 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ceph_event_retention_days", 2024-04-24T09:18:04.187 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:04.187 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.187 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.187 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "7", 2024-04-24T09:18:04.187 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.187 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.187 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.187 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-24T09:18:04.187 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.187 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.187 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.187 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.187 INFO:teuthology.orchestra.run.smithi083.stdout: "config_check_secs": { 2024-04-24T09:18:04.188 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "config_check_secs", 2024-04-24T09:18:04.188 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:04.188 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.188 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.188 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-04-24T09:18:04.188 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "10", 2024-04-24T09:18:04.188 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.188 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.188 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-24T09:18:04.188 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.188 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.188 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.188 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.188 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-24T09:18:04.189 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-24T09:18:04.189 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.189 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.189 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.189 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.189 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.189 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.189 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.189 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.189 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.189 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.189 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.189 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.189 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.189 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.190 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.190 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.190 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.190 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.190 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.190 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-24T09:18:04.190 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-24T09:18:04.190 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.190 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.190 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.190 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.190 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.190 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.191 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.191 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.191 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.191 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.191 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.191 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.191 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-24T09:18:04.191 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-24T09:18:04.191 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.191 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.191 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.191 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-24T09:18:04.191 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.191 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.191 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.191 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.192 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.192 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.192 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.192 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.192 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.192 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.192 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.192 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.192 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.192 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.192 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.192 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-24T09:18:04.192 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-24T09:18:04.192 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.192 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.192 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.193 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.193 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.193 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.193 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.193 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.193 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.193 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.193 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.193 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.193 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.193 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.193 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:04.193 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "localpool", 2024-04-24T09:18:04.193 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-24T09:18:04.193 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-24T09:18:04.193 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-24T09:18:04.193 INFO:teuthology.orchestra.run.smithi083.stdout: "failure_domain": { 2024-04-24T09:18:04.194 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "failure_domain", 2024-04-24T09:18:04.194 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.194 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.194 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.194 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "host", 2024-04-24T09:18:04.194 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.194 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.194 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.194 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "failure domain for any created local pool", 2024-04-24T09:18:04.194 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-24T09:18:04.194 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.194 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.194 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.194 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-24T09:18:04.194 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-24T09:18:04.195 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.195 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.195 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.195 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.195 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.195 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.195 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.195 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.195 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.195 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.195 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.195 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.195 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.195 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.195 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.195 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.195 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.196 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.196 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.196 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-24T09:18:04.196 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-24T09:18:04.196 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.196 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.196 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.196 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.196 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.196 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.196 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.196 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.197 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.197 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.197 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.197 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.197 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-24T09:18:04.197 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-24T09:18:04.197 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.197 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.197 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.197 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-24T09:18:04.197 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.197 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.197 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.197 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.197 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.197 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.198 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.198 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.198 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.198 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.198 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.198 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.198 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.198 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.198 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.198 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-24T09:18:04.198 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-24T09:18:04.198 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.198 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.198 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.198 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.199 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.199 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.199 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.199 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.199 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.199 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.199 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.199 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.199 INFO:teuthology.orchestra.run.smithi083.stdout: "min_size": { 2024-04-24T09:18:04.199 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "min_size", 2024-04-24T09:18:04.199 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:04.199 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.199 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.199 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.199 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.199 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.200 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.200 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "default min_size for any created local pool", 2024-04-24T09:18:04.200 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-24T09:18:04.200 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.200 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.200 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.200 INFO:teuthology.orchestra.run.smithi083.stdout: "num_rep": { 2024-04-24T09:18:04.200 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "num_rep", 2024-04-24T09:18:04.200 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:04.200 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.200 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.200 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "3", 2024-04-24T09:18:04.200 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.200 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.201 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.201 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "default replica count for any created local pool", 2024-04-24T09:18:04.201 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.201 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.201 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.201 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.201 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num": { 2024-04-24T09:18:04.201 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "pg_num", 2024-04-24T09:18:04.201 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:04.201 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.201 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.201 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "128", 2024-04-24T09:18:04.201 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.201 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.201 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.202 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "default pg_num for any created local pool", 2024-04-24T09:18:04.202 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.202 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.202 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.202 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.202 INFO:teuthology.orchestra.run.smithi083.stdout: "prefix": { 2024-04-24T09:18:04.202 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "prefix", 2024-04-24T09:18:04.202 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.202 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.202 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.202 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.202 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.202 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.202 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.202 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "name prefix for any created local pool", 2024-04-24T09:18:04.203 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.203 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.203 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.203 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.203 INFO:teuthology.orchestra.run.smithi083.stdout: "subtree": { 2024-04-24T09:18:04.203 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "subtree", 2024-04-24T09:18:04.203 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.203 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.203 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.203 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "rack", 2024-04-24T09:18:04.203 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.203 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.203 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.203 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-24T09:18:04.203 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-24T09:18:04.203 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.203 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.204 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.204 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.204 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.204 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:04.204 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mds_autoscaler", 2024-04-24T09:18:04.204 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-24T09:18:04.204 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-24T09:18:04.204 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-24T09:18:04.204 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-24T09:18:04.204 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-24T09:18:04.204 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.204 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.204 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.204 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.204 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.204 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.205 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.205 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.205 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.205 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.205 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.205 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.205 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.205 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.205 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.205 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.205 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.205 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.205 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.205 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-24T09:18:04.205 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-24T09:18:04.205 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.206 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.206 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.206 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.206 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.206 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.206 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.206 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.206 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.206 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.206 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.206 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.206 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-24T09:18:04.206 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-24T09:18:04.206 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.206 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.207 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.207 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-24T09:18:04.207 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.207 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.207 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.207 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.207 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.207 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.207 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.207 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.207 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.207 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.207 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.207 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.207 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.207 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.207 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.208 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-24T09:18:04.208 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-24T09:18:04.208 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.208 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.208 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.208 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.208 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.208 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.208 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.208 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.208 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.208 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.208 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.208 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.208 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.209 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.209 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:04.209 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mirroring", 2024-04-24T09:18:04.209 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-24T09:18:04.209 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-24T09:18:04.209 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-24T09:18:04.209 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-24T09:18:04.209 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-24T09:18:04.209 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.209 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.209 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.209 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.209 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.209 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.209 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.209 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.210 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.210 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.210 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.210 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.210 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.210 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.210 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.210 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.210 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.210 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.210 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.210 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-24T09:18:04.210 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-24T09:18:04.210 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.211 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.211 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.211 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.211 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.211 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.211 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.211 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.211 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.211 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.211 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.211 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.211 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-24T09:18:04.211 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-24T09:18:04.211 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.211 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.212 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.212 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-24T09:18:04.212 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.212 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.212 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.212 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.212 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.212 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.212 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.212 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.212 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.212 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.212 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.212 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.212 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.212 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.213 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.213 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-24T09:18:04.213 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-24T09:18:04.213 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.213 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.213 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.213 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.213 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.213 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.213 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.213 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.213 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.213 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.213 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.213 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.213 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.214 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.214 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:04.214 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "nfs", 2024-04-24T09:18:04.214 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-24T09:18:04.214 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-24T09:18:04.214 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-24T09:18:04.214 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-24T09:18:04.214 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-24T09:18:04.214 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.214 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.214 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.214 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.214 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.214 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.214 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.215 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.215 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.215 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.215 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.215 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.215 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.215 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.215 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.215 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.215 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.215 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.215 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.215 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-24T09:18:04.215 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-24T09:18:04.215 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.216 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.216 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.216 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.216 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.216 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.216 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.216 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.216 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.216 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.216 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.216 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.216 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-24T09:18:04.216 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-24T09:18:04.216 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.216 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.216 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.217 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-24T09:18:04.217 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.217 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.217 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.217 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.217 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.217 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.217 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.217 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.217 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.217 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.217 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.217 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.217 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.217 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.217 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.218 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-24T09:18:04.218 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-24T09:18:04.218 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.218 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.218 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.218 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.218 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.218 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.218 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.218 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.218 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.218 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.218 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.218 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.218 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.218 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.219 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:04.219 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "orchestrator", 2024-04-24T09:18:04.219 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-24T09:18:04.219 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-24T09:18:04.219 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-24T09:18:04.219 INFO:teuthology.orchestra.run.smithi083.stdout: "fail_fs": { 2024-04-24T09:18:04.219 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "fail_fs", 2024-04-24T09:18:04.219 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.219 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.219 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.219 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.219 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.219 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.219 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.219 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-24T09:18:04.220 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.220 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.220 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.220 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.220 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-24T09:18:04.220 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-24T09:18:04.220 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.220 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.220 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.220 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.220 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.220 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.220 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.220 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.220 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.220 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.220 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.221 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.221 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.221 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.221 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.221 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.221 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.221 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.221 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.221 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-24T09:18:04.221 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-24T09:18:04.221 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.221 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.221 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.221 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.221 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.222 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.222 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.222 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.222 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.222 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.222 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.222 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.222 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-24T09:18:04.222 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-24T09:18:04.222 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.222 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.222 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.222 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-24T09:18:04.222 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.222 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.223 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.223 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.223 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.223 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.223 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.223 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.223 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.223 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.223 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.223 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.223 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.223 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.223 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.223 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-24T09:18:04.223 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-24T09:18:04.223 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.223 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.224 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.224 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.224 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.224 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.224 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.224 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.224 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.224 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.224 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.224 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.224 INFO:teuthology.orchestra.run.smithi083.stdout: "orchestrator": { 2024-04-24T09:18:04.224 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "orchestrator", 2024-04-24T09:18:04.224 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.224 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.224 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.224 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.225 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.225 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.225 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.225 INFO:teuthology.orchestra.run.smithi083.stdout: "cephadm", 2024-04-24T09:18:04.225 INFO:teuthology.orchestra.run.smithi083.stdout: "rook", 2024-04-24T09:18:04.225 INFO:teuthology.orchestra.run.smithi083.stdout: "test_orchestrator" 2024-04-24T09:18:04.225 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.225 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Orchestrator backend", 2024-04-24T09:18:04.225 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.225 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.225 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.225 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.225 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.225 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.225 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:04.225 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "osd_perf_query", 2024-04-24T09:18:04.226 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-24T09:18:04.226 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-24T09:18:04.226 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-24T09:18:04.226 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-24T09:18:04.226 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-24T09:18:04.226 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.226 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.226 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.226 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.226 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.226 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.226 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.226 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.226 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.227 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.227 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.227 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.227 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.227 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.227 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.227 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.227 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.227 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.227 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.227 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-24T09:18:04.227 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-24T09:18:04.227 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.227 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.227 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.228 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.228 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.228 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.228 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.228 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.228 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.228 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.228 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.228 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.228 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-24T09:18:04.228 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-24T09:18:04.228 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.228 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.228 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.228 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-24T09:18:04.228 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.229 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.229 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.229 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.229 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.229 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.229 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.229 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.229 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.229 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.229 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.229 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.229 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.229 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.229 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.229 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-24T09:18:04.229 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-24T09:18:04.230 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.230 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.230 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.230 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.230 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.230 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.230 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.230 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.230 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.230 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.230 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.230 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.230 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.230 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.230 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:04.231 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "osd_support", 2024-04-24T09:18:04.231 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-24T09:18:04.231 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-24T09:18:04.231 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-24T09:18:04.231 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-24T09:18:04.231 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-24T09:18:04.231 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.231 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.231 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.231 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.231 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.231 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.231 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.231 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.231 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.232 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.232 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.232 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.232 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.232 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.232 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.232 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.232 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.232 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.232 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.232 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-24T09:18:04.232 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-24T09:18:04.232 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.232 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.232 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.232 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.232 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.233 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.233 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.233 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.233 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.233 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.233 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.233 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.233 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-24T09:18:04.233 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-24T09:18:04.233 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.233 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.233 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.233 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-24T09:18:04.233 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.233 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.234 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.234 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.234 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.234 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.234 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.234 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.234 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.234 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.234 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.234 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.234 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.234 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.234 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.234 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-24T09:18:04.235 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-24T09:18:04.235 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.235 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.235 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.235 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.235 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.235 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.235 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.235 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.235 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.235 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.235 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.235 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.235 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.235 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.235 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:04.235 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "pg_autoscaler", 2024-04-24T09:18:04.236 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-24T09:18:04.236 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-24T09:18:04.236 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-24T09:18:04.236 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-24T09:18:04.236 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-24T09:18:04.236 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.236 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.236 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.236 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.236 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.236 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.236 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.236 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.236 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.236 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.237 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.237 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.237 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.237 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.237 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.237 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.237 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.237 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.237 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.237 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-24T09:18:04.237 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-24T09:18:04.237 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.237 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.237 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.237 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.238 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.238 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.238 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.238 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.238 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.238 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.238 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.238 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.238 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-24T09:18:04.238 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-24T09:18:04.238 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.238 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.238 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.238 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-24T09:18:04.238 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.239 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.239 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.239 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.239 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.239 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.239 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.239 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.239 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.239 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.239 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.239 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.239 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.239 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.239 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.240 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-24T09:18:04.240 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-24T09:18:04.240 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.240 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.240 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.240 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.240 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.240 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.240 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.240 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.240 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.240 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.240 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.240 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.240 INFO:teuthology.orchestra.run.smithi083.stdout: "sleep_interval": { 2024-04-24T09:18:04.241 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sleep_interval", 2024-04-24T09:18:04.241 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-24T09:18:04.241 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.241 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.241 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "60", 2024-04-24T09:18:04.241 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.241 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.241 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.241 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.241 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.241 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.241 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.241 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.241 INFO:teuthology.orchestra.run.smithi083.stdout: "threshold": { 2024-04-24T09:18:04.241 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "threshold", 2024-04-24T09:18:04.242 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-04-24T09:18:04.242 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.242 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.242 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "3.0", 2024-04-24T09:18:04.242 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "1.0", 2024-04-24T09:18:04.242 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.242 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.242 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "scaling threshold", 2024-04-24T09:18:04.242 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-24T09:18:04.242 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.242 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.242 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.242 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.242 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.242 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:04.242 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "progress", 2024-04-24T09:18:04.243 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-24T09:18:04.243 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-24T09:18:04.243 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-24T09:18:04.243 INFO:teuthology.orchestra.run.smithi083.stdout: "allow_pg_recovery_event": { 2024-04-24T09:18:04.243 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "allow_pg_recovery_event", 2024-04-24T09:18:04.243 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.243 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.243 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.243 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.243 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.243 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.243 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.243 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-24T09:18:04.243 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.243 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.243 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.244 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.244 INFO:teuthology.orchestra.run.smithi083.stdout: "enabled": { 2024-04-24T09:18:04.244 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "enabled", 2024-04-24T09:18:04.244 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.244 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.244 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.244 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:04.244 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.244 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.244 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.244 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.244 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.244 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.244 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.244 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.244 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-24T09:18:04.244 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-24T09:18:04.245 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.245 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.245 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.245 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.245 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.245 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.245 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.245 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.245 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.245 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.245 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.245 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.245 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.245 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.245 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.245 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.246 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.246 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.246 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.246 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-24T09:18:04.246 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-24T09:18:04.246 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.246 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.246 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.246 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.246 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.246 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.246 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.246 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.246 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.246 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.247 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.247 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.247 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-24T09:18:04.247 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-24T09:18:04.247 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.247 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.247 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.247 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-24T09:18:04.247 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.247 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.247 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.247 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.247 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.247 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.247 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.247 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.248 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.248 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.248 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.248 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.248 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.248 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.248 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.248 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-24T09:18:04.248 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-24T09:18:04.248 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.248 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.248 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.248 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.248 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.248 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.249 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.249 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.249 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.249 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.249 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.249 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.249 INFO:teuthology.orchestra.run.smithi083.stdout: "max_completed_events": { 2024-04-24T09:18:04.249 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "max_completed_events", 2024-04-24T09:18:04.249 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:04.249 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.249 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.249 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "50", 2024-04-24T09:18:04.249 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.249 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.249 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.250 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "number of past completed events to remember", 2024-04-24T09:18:04.250 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.250 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.250 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.250 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.250 INFO:teuthology.orchestra.run.smithi083.stdout: "sleep_interval": { 2024-04-24T09:18:04.250 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sleep_interval", 2024-04-24T09:18:04.250 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-24T09:18:04.250 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.250 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.250 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "5", 2024-04-24T09:18:04.250 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.250 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.250 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.251 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how long the module is going to sleep", 2024-04-24T09:18:04.251 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.251 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.251 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.251 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.251 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.251 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.251 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:04.251 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "prometheus", 2024-04-24T09:18:04.251 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-24T09:18:04.251 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-24T09:18:04.251 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-24T09:18:04.251 INFO:teuthology.orchestra.run.smithi083.stdout: "cache": { 2024-04-24T09:18:04.251 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "cache", 2024-04-24T09:18:04.252 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.252 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.252 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.252 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:04.252 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.252 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.252 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.252 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.252 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.252 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.252 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.252 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.252 INFO:teuthology.orchestra.run.smithi083.stdout: "exclude_perf_counters": { 2024-04-24T09:18:04.252 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "exclude_perf_counters", 2024-04-24T09:18:04.252 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.252 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.253 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.253 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:04.253 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.253 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.253 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.253 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-24T09:18:04.253 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-24T09:18:04.253 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.253 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.253 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.253 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-24T09:18:04.253 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-24T09:18:04.253 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.253 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.253 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.253 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.253 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.254 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.254 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.254 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.254 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.254 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.254 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.254 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.254 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.254 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.254 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.254 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.254 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.254 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.254 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.254 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-24T09:18:04.255 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-24T09:18:04.255 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.255 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.255 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.255 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.255 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.255 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.255 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.255 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.255 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.255 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.255 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.255 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.255 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-24T09:18:04.255 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-24T09:18:04.255 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.255 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.256 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.256 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-24T09:18:04.256 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.256 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.256 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.256 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.256 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.256 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.256 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.256 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.256 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.256 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.256 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.256 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.256 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.256 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.256 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.257 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-24T09:18:04.257 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-24T09:18:04.257 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.257 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.257 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.257 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.257 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.257 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.257 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.257 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.257 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.257 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.257 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.257 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.257 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_stats_pools": { 2024-04-24T09:18:04.257 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rbd_stats_pools", 2024-04-24T09:18:04.258 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.258 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.258 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.258 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.258 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.258 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.258 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.258 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.258 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.258 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.258 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.258 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.258 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-24T09:18:04.258 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-24T09:18:04.258 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:04.259 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.259 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.259 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "300", 2024-04-24T09:18:04.259 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.259 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.259 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.259 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.259 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.259 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.259 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.259 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.259 INFO:teuthology.orchestra.run.smithi083.stdout: "scrape_interval": { 2024-04-24T09:18:04.259 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "scrape_interval", 2024-04-24T09:18:04.259 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-04-24T09:18:04.259 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.259 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.260 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "15.0", 2024-04-24T09:18:04.260 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.260 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.260 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.260 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.260 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.260 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.260 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.260 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.260 INFO:teuthology.orchestra.run.smithi083.stdout: "server_addr": { 2024-04-24T09:18:04.260 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_addr", 2024-04-24T09:18:04.260 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.260 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.260 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.260 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "::", 2024-04-24T09:18:04.261 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.261 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.261 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.261 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-24T09:18:04.261 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.261 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.261 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.261 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.261 INFO:teuthology.orchestra.run.smithi083.stdout: "server_port": { 2024-04-24T09:18:04.261 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_port", 2024-04-24T09:18:04.261 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:04.261 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.261 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.261 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "9283", 2024-04-24T09:18:04.261 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.261 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.262 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.262 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-24T09:18:04.262 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.262 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.262 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.262 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.262 INFO:teuthology.orchestra.run.smithi083.stdout: "stale_cache_strategy": { 2024-04-24T09:18:04.262 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "stale_cache_strategy", 2024-04-24T09:18:04.262 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.262 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.262 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.262 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "log", 2024-04-24T09:18:04.262 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.262 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.263 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.263 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.263 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.263 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.263 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.263 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.263 INFO:teuthology.orchestra.run.smithi083.stdout: "standby_behaviour": { 2024-04-24T09:18:04.263 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "standby_behaviour", 2024-04-24T09:18:04.263 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.263 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.263 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.263 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "default", 2024-04-24T09:18:04.263 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.263 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.263 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.263 INFO:teuthology.orchestra.run.smithi083.stdout: "default", 2024-04-24T09:18:04.263 INFO:teuthology.orchestra.run.smithi083.stdout: "error" 2024-04-24T09:18:04.264 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.264 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.264 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.264 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.264 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.264 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.264 INFO:teuthology.orchestra.run.smithi083.stdout: "standby_error_status_code": { 2024-04-24T09:18:04.264 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "standby_error_status_code", 2024-04-24T09:18:04.264 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:04.264 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.264 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.264 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "500", 2024-04-24T09:18:04.264 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "400", 2024-04-24T09:18:04.264 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "599", 2024-04-24T09:18:04.264 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.265 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.265 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.265 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.265 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.265 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.265 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.265 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.265 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:04.265 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rbd_support", 2024-04-24T09:18:04.265 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-24T09:18:04.265 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-24T09:18:04.265 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-24T09:18:04.265 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-24T09:18:04.265 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-24T09:18:04.265 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.265 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.265 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.266 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.266 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.266 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.266 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.266 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.266 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.266 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.266 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.266 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.266 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.266 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.266 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.266 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.266 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.266 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.267 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.267 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-24T09:18:04.267 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-24T09:18:04.267 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.267 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.267 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.267 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.267 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.267 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.267 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.267 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.267 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.267 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.267 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.267 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.267 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-24T09:18:04.267 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-24T09:18:04.268 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.268 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.268 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.268 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-24T09:18:04.268 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.268 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.268 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.268 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.268 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.268 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.268 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.268 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.268 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.268 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.268 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.268 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.269 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.269 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.269 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.269 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-24T09:18:04.269 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-24T09:18:04.269 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.269 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.269 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.269 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.269 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.269 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.269 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.269 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.269 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.269 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.269 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.270 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.270 INFO:teuthology.orchestra.run.smithi083.stdout: "max_concurrent_snap_create": { 2024-04-24T09:18:04.270 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "max_concurrent_snap_create", 2024-04-24T09:18:04.270 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:04.270 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.270 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.270 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-04-24T09:18:04.270 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.270 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.270 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.270 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.270 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.270 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.270 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.271 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.271 INFO:teuthology.orchestra.run.smithi083.stdout: "mirror_snapshot_schedule": { 2024-04-24T09:18:04.271 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mirror_snapshot_schedule", 2024-04-24T09:18:04.271 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.271 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.271 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.271 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.271 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.271 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.271 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.271 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.271 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.271 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.271 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.271 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.272 INFO:teuthology.orchestra.run.smithi083.stdout: "trash_purge_schedule": { 2024-04-24T09:18:04.272 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "trash_purge_schedule", 2024-04-24T09:18:04.272 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.272 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.272 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.272 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.272 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.272 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.272 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.272 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.272 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.272 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.272 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.272 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.272 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.272 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.272 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:04.273 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "restful", 2024-04-24T09:18:04.273 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-24T09:18:04.273 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-24T09:18:04.273 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-24T09:18:04.273 INFO:teuthology.orchestra.run.smithi083.stdout: "enable_auth": { 2024-04-24T09:18:04.273 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "enable_auth", 2024-04-24T09:18:04.273 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.273 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.273 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.273 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:04.273 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.273 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.273 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.273 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.273 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.273 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.274 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.274 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.274 INFO:teuthology.orchestra.run.smithi083.stdout: "key_file": { 2024-04-24T09:18:04.274 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "key_file", 2024-04-24T09:18:04.274 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.274 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.274 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.274 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.274 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.274 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.274 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.274 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.274 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.274 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.274 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.274 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.275 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-24T09:18:04.275 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-24T09:18:04.275 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.275 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.275 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.275 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.275 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.275 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.275 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.275 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.275 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.275 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.275 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.275 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.275 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.275 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.276 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.276 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.276 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.276 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.276 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.276 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-24T09:18:04.276 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-24T09:18:04.276 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.276 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.276 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.276 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.276 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.276 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.276 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.276 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.276 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.277 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.277 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.277 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.277 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-24T09:18:04.277 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-24T09:18:04.277 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.277 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.277 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.277 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-24T09:18:04.277 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.277 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.277 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.277 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.277 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.277 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.277 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.277 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.278 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.278 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.278 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.278 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.278 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.278 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.278 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.278 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-24T09:18:04.278 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-24T09:18:04.278 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.278 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.278 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.278 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.278 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.279 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.279 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.279 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.279 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.279 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.279 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.279 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.279 INFO:teuthology.orchestra.run.smithi083.stdout: "server_addr": { 2024-04-24T09:18:04.279 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_addr", 2024-04-24T09:18:04.279 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.279 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.279 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.279 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.279 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.279 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.280 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.280 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.280 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.280 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.280 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.280 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.280 INFO:teuthology.orchestra.run.smithi083.stdout: "server_port": { 2024-04-24T09:18:04.280 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_port", 2024-04-24T09:18:04.280 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.280 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.280 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.280 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.280 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.280 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.280 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.281 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.281 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.281 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.281 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.281 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.281 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.281 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.281 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:04.281 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rgw", 2024-04-24T09:18:04.281 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-24T09:18:04.281 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-24T09:18:04.281 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-24T09:18:04.282 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-24T09:18:04.282 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-24T09:18:04.282 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.282 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.282 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.282 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.282 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.282 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.282 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.282 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.282 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.282 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.282 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.282 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.282 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.283 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.283 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.283 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.283 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.283 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.283 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.283 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-24T09:18:04.283 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-24T09:18:04.283 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.283 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.283 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.283 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.283 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.283 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.283 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.283 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.284 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.284 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.284 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.284 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.284 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-24T09:18:04.284 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-24T09:18:04.284 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.284 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.284 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.284 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-24T09:18:04.284 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.284 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.284 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.284 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.284 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.285 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.285 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.285 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.285 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.285 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.285 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.285 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.285 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.285 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.285 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.285 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-24T09:18:04.285 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-24T09:18:04.285 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.285 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.285 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.286 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.286 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.286 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.286 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.286 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.286 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.286 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.286 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.286 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.286 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.286 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.286 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:04.286 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rook", 2024-04-24T09:18:04.286 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-24T09:18:04.287 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-24T09:18:04.287 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-24T09:18:04.287 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-24T09:18:04.287 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-24T09:18:04.287 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.287 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.287 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.287 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.287 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.287 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.287 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.287 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.287 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.287 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.287 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.287 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.288 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.288 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.288 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.288 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.288 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.288 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.288 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.288 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-24T09:18:04.288 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-24T09:18:04.288 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.288 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.288 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.288 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.288 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.288 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.288 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.288 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.289 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.289 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.289 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.289 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.289 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-24T09:18:04.289 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-24T09:18:04.289 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.289 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.289 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.289 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-24T09:18:04.289 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.289 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.289 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.289 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.290 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.290 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.290 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.290 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.290 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.290 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.290 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.290 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.290 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.290 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.290 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.290 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-24T09:18:04.290 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-24T09:18:04.290 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.291 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.291 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.291 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.291 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.291 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.291 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.291 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.291 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.291 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.291 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.291 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.291 INFO:teuthology.orchestra.run.smithi083.stdout: "storage_class": { 2024-04-24T09:18:04.291 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "storage_class", 2024-04-24T09:18:04.291 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.292 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.292 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.292 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "local", 2024-04-24T09:18:04.292 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.292 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.292 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.292 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-24T09:18:04.292 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.292 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.292 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.292 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.292 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.292 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.292 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:04.292 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "selftest", 2024-04-24T09:18:04.292 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-24T09:18:04.293 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-24T09:18:04.293 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-24T09:18:04.293 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-24T09:18:04.293 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-24T09:18:04.293 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.293 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.293 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.293 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.293 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.293 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.293 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.293 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.293 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.293 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.293 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.293 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.294 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.294 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.294 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.294 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.294 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.294 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.294 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.294 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-24T09:18:04.294 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-24T09:18:04.294 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.294 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.294 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.294 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.294 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.294 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.294 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.294 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.295 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.295 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.295 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.295 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.295 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-24T09:18:04.295 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-24T09:18:04.295 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.295 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.295 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.295 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-24T09:18:04.295 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.295 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.295 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.295 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.295 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.296 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.296 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.296 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.296 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.296 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.296 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.296 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.296 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.296 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.296 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.296 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-24T09:18:04.296 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-24T09:18:04.296 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.296 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.296 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.296 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.297 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.297 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.297 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.297 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.297 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.297 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.297 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.297 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.297 INFO:teuthology.orchestra.run.smithi083.stdout: "roption1": { 2024-04-24T09:18:04.297 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "roption1", 2024-04-24T09:18:04.297 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.297 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.297 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.297 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.297 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.297 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.297 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.298 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.298 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.298 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.298 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.298 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.298 INFO:teuthology.orchestra.run.smithi083.stdout: "roption2": { 2024-04-24T09:18:04.298 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "roption2", 2024-04-24T09:18:04.298 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.298 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.298 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.298 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "xyz", 2024-04-24T09:18:04.298 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.298 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.298 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.298 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.299 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.299 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.299 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.299 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.299 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption1": { 2024-04-24T09:18:04.299 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption1", 2024-04-24T09:18:04.299 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.299 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.299 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.299 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.299 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.299 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.299 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.299 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.299 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.299 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.300 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.300 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.300 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption2": { 2024-04-24T09:18:04.300 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption2", 2024-04-24T09:18:04.300 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:04.300 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.300 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.300 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.300 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.300 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.300 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.300 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.300 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.300 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.300 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.301 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.301 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption3": { 2024-04-24T09:18:04.301 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption3", 2024-04-24T09:18:04.301 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-04-24T09:18:04.301 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.301 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.301 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.301 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.301 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.301 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.301 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.301 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.301 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.301 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.301 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.301 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption4": { 2024-04-24T09:18:04.302 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption4", 2024-04-24T09:18:04.302 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.302 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.302 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.302 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.302 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.302 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.302 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.302 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.302 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.302 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.302 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.302 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.302 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption5": { 2024-04-24T09:18:04.302 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption5", 2024-04-24T09:18:04.302 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.303 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.303 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.303 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.303 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.303 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.303 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.303 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.303 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.303 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.303 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.303 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.303 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption6": { 2024-04-24T09:18:04.303 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption6", 2024-04-24T09:18:04.303 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.303 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.303 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.304 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:04.304 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.304 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.304 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.304 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.304 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.304 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.304 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.304 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.304 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption7": { 2024-04-24T09:18:04.304 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption7", 2024-04-24T09:18:04.304 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:04.304 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.304 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.304 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.304 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "1", 2024-04-24T09:18:04.305 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "42", 2024-04-24T09:18:04.305 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.305 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.305 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.305 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.305 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.305 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.305 INFO:teuthology.orchestra.run.smithi083.stdout: "testkey": { 2024-04-24T09:18:04.305 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "testkey", 2024-04-24T09:18:04.305 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.305 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.305 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.305 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.305 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.305 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.305 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.306 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.306 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.306 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.306 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.306 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.306 INFO:teuthology.orchestra.run.smithi083.stdout: "testlkey": { 2024-04-24T09:18:04.306 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "testlkey", 2024-04-24T09:18:04.306 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.306 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.306 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.306 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.306 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.306 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.306 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.306 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.306 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.307 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.307 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.307 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.307 INFO:teuthology.orchestra.run.smithi083.stdout: "testnewline": { 2024-04-24T09:18:04.307 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "testnewline", 2024-04-24T09:18:04.307 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.307 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.307 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.307 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.307 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.307 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.307 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.307 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.307 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.307 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.308 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.308 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.308 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.308 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.308 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:04.308 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "snap_schedule", 2024-04-24T09:18:04.308 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-24T09:18:04.308 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-24T09:18:04.308 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-24T09:18:04.308 INFO:teuthology.orchestra.run.smithi083.stdout: "allow_m_granularity": { 2024-04-24T09:18:04.308 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "allow_m_granularity", 2024-04-24T09:18:04.308 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.308 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.308 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.308 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.308 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.309 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.309 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.309 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "allow minute scheduled snapshots", 2024-04-24T09:18:04.309 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.309 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.309 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.309 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.309 INFO:teuthology.orchestra.run.smithi083.stdout: "dump_on_update": { 2024-04-24T09:18:04.309 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "dump_on_update", 2024-04-24T09:18:04.309 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.309 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.309 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.309 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.309 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.309 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.310 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.310 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "dump database to debug log on update", 2024-04-24T09:18:04.310 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.310 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.310 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.310 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.310 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-24T09:18:04.310 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-24T09:18:04.310 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.310 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.310 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.310 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.310 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.311 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.311 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.311 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.311 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.311 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.311 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.311 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.311 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.311 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.311 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.311 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.311 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.311 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.311 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.311 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-24T09:18:04.312 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-24T09:18:04.312 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.312 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.312 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.312 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.312 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.312 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.312 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.312 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.312 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.312 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.312 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.312 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.312 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-24T09:18:04.312 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-24T09:18:04.312 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.313 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.313 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.313 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-24T09:18:04.313 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.313 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.313 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.313 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.313 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.313 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.313 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.313 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.313 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.313 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.313 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.313 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.313 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.314 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.314 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.314 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-24T09:18:04.314 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-24T09:18:04.314 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.314 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.314 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.314 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.314 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.314 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.314 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.314 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.314 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.314 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.314 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.315 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.315 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.315 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.315 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:04.315 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "stats", 2024-04-24T09:18:04.315 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-24T09:18:04.315 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-24T09:18:04.315 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-24T09:18:04.315 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-24T09:18:04.315 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-24T09:18:04.315 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.315 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.315 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.315 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.315 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.315 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.315 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.316 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.316 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.316 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.316 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.316 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.316 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.316 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.316 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.316 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.316 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.316 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.316 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.316 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-24T09:18:04.316 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-24T09:18:04.316 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.316 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.317 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.317 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.317 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.317 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.317 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.317 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.317 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.317 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.317 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.317 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.317 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-24T09:18:04.317 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-24T09:18:04.317 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.317 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.317 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.317 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-24T09:18:04.318 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.318 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.318 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.318 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.318 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.318 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.318 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.318 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.318 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.318 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.318 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.318 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.318 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.318 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.319 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.319 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-24T09:18:04.319 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-24T09:18:04.319 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.319 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.319 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.319 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.319 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.319 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.319 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.319 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.319 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.319 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.319 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.319 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.319 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.320 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.320 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:04.320 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "status", 2024-04-24T09:18:04.320 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-24T09:18:04.320 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-24T09:18:04.320 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-24T09:18:04.320 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-24T09:18:04.320 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-24T09:18:04.320 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.320 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.320 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.320 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.320 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.320 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.320 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.321 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.321 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.321 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.321 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.321 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.321 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.321 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.321 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.321 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.321 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.321 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.321 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.321 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-24T09:18:04.321 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-24T09:18:04.321 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.321 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.322 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.322 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.322 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.322 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.322 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.322 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.322 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.322 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.322 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.322 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.322 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-24T09:18:04.322 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-24T09:18:04.322 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.322 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.322 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.323 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-24T09:18:04.323 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.323 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.323 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.323 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.323 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.323 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.323 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.323 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.323 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.323 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.323 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.323 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.323 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.323 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.324 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.324 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-24T09:18:04.324 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-24T09:18:04.324 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.324 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.324 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.324 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.324 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.324 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.324 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.324 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.324 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.324 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.324 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.324 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.324 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.325 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.325 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:04.325 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "telegraf", 2024-04-24T09:18:04.325 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-24T09:18:04.325 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-24T09:18:04.325 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-24T09:18:04.325 INFO:teuthology.orchestra.run.smithi083.stdout: "address": { 2024-04-24T09:18:04.325 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "address", 2024-04-24T09:18:04.325 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.325 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.325 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.325 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-24T09:18:04.325 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.325 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.325 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.325 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.326 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.326 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.326 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.326 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.326 INFO:teuthology.orchestra.run.smithi083.stdout: "interval": { 2024-04-24T09:18:04.326 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "interval", 2024-04-24T09:18:04.326 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-24T09:18:04.326 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.326 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.326 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "15", 2024-04-24T09:18:04.326 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.326 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.326 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.326 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.326 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.326 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.327 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.327 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.327 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-24T09:18:04.327 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-24T09:18:04.327 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.327 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.327 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.327 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.327 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.327 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.327 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.327 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.327 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.327 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.327 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.328 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.328 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.328 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.328 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.328 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.328 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.328 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.328 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.328 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-24T09:18:04.328 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-24T09:18:04.328 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.328 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.328 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.328 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.328 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.328 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.329 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.329 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.329 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.329 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.329 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.329 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.329 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-24T09:18:04.329 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-24T09:18:04.329 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.329 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.329 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.329 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-24T09:18:04.329 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.329 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.329 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.329 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.329 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.330 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.330 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.330 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.330 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.330 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.330 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.330 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.330 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.330 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.330 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.330 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-24T09:18:04.330 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-24T09:18:04.330 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.330 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.330 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.331 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.331 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.331 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.331 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.331 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.331 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.331 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.331 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.331 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.331 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.331 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.331 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:04.331 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "telemetry", 2024-04-24T09:18:04.331 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-24T09:18:04.331 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-24T09:18:04.332 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-24T09:18:04.332 INFO:teuthology.orchestra.run.smithi083.stdout: "channel_basic": { 2024-04-24T09:18:04.332 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "channel_basic", 2024-04-24T09:18:04.332 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.332 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.332 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.332 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:04.332 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.332 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.332 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.332 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-24T09:18:04.332 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.332 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.332 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.332 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.332 INFO:teuthology.orchestra.run.smithi083.stdout: "channel_crash": { 2024-04-24T09:18:04.333 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "channel_crash", 2024-04-24T09:18:04.333 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.333 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.333 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.333 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:04.333 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.333 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.333 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.333 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-24T09:18:04.333 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.333 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.333 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.333 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.333 INFO:teuthology.orchestra.run.smithi083.stdout: "channel_device": { 2024-04-24T09:18:04.333 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "channel_device", 2024-04-24T09:18:04.333 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.334 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.334 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.334 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:04.334 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.334 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.334 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.334 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-24T09:18:04.334 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.334 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.334 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.334 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.334 INFO:teuthology.orchestra.run.smithi083.stdout: "channel_ident": { 2024-04-24T09:18:04.334 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "channel_ident", 2024-04-24T09:18:04.334 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.335 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.335 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.335 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.335 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.335 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.335 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.335 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-24T09:18:04.335 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.335 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.335 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.335 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.335 INFO:teuthology.orchestra.run.smithi083.stdout: "channel_perf": { 2024-04-24T09:18:04.335 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "channel_perf", 2024-04-24T09:18:04.335 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.335 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.335 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.336 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.336 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.336 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.336 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.336 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-24T09:18:04.336 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.336 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.336 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.336 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.336 INFO:teuthology.orchestra.run.smithi083.stdout: "contact": { 2024-04-24T09:18:04.336 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "contact", 2024-04-24T09:18:04.336 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.336 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.336 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.336 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.336 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.337 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.337 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.337 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.337 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.337 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.337 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.337 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.337 INFO:teuthology.orchestra.run.smithi083.stdout: "description": { 2024-04-24T09:18:04.337 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "description", 2024-04-24T09:18:04.337 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.337 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.337 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.337 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.337 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.337 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.337 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.338 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.338 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.338 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.338 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.338 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.338 INFO:teuthology.orchestra.run.smithi083.stdout: "device_url": { 2024-04-24T09:18:04.338 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "device_url", 2024-04-24T09:18:04.338 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.338 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.338 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.338 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-24T09:18:04.338 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.338 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.338 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.338 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.338 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.339 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.339 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.339 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.339 INFO:teuthology.orchestra.run.smithi083.stdout: "enabled": { 2024-04-24T09:18:04.339 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "enabled", 2024-04-24T09:18:04.339 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.339 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.339 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.339 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.339 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.339 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.339 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.339 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.339 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.339 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.340 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.340 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.340 INFO:teuthology.orchestra.run.smithi083.stdout: "interval": { 2024-04-24T09:18:04.340 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "interval", 2024-04-24T09:18:04.340 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:04.340 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.340 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.340 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "24", 2024-04-24T09:18:04.340 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "8", 2024-04-24T09:18:04.340 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.340 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.340 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.340 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.340 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.341 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.341 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.341 INFO:teuthology.orchestra.run.smithi083.stdout: "last_opt_revision": { 2024-04-24T09:18:04.341 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "last_opt_revision", 2024-04-24T09:18:04.341 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:04.341 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.341 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.341 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "1", 2024-04-24T09:18:04.341 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.341 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.341 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.341 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.341 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.341 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.341 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.342 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.342 INFO:teuthology.orchestra.run.smithi083.stdout: "leaderboard": { 2024-04-24T09:18:04.342 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "leaderboard", 2024-04-24T09:18:04.342 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.342 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.342 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.342 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.342 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.342 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.342 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.342 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.342 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.342 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.342 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.343 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.343 INFO:teuthology.orchestra.run.smithi083.stdout: "leaderboard_description": { 2024-04-24T09:18:04.343 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "leaderboard_description", 2024-04-24T09:18:04.343 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.343 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.343 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.343 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.343 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.343 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.343 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.343 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.343 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.343 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.343 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.343 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.343 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-24T09:18:04.344 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-24T09:18:04.344 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.344 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.344 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.344 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.344 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.344 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.344 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.344 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.344 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.344 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.344 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.344 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.344 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.344 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.344 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.344 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.345 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.345 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.345 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.345 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-24T09:18:04.345 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-24T09:18:04.345 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.345 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.345 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.345 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.345 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.345 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.345 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.345 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.345 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.345 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.345 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.346 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.346 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-24T09:18:04.346 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-24T09:18:04.346 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.346 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.346 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.346 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-24T09:18:04.346 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.346 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.346 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.346 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.346 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.346 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.346 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.346 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.347 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.347 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.347 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.347 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.347 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.347 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.347 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.347 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-24T09:18:04.347 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-24T09:18:04.347 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.347 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.347 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.347 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.347 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.347 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.347 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.348 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.348 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.348 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.348 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.348 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.348 INFO:teuthology.orchestra.run.smithi083.stdout: "organization": { 2024-04-24T09:18:04.348 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "organization", 2024-04-24T09:18:04.348 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.348 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.348 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.348 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.348 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.348 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.348 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.348 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.348 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.349 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.349 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.349 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.349 INFO:teuthology.orchestra.run.smithi083.stdout: "proxy": { 2024-04-24T09:18:04.349 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "proxy", 2024-04-24T09:18:04.349 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.349 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.349 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.349 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.349 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.349 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.349 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.349 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.349 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.349 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.350 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.350 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.350 INFO:teuthology.orchestra.run.smithi083.stdout: "url": { 2024-04-24T09:18:04.350 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "url", 2024-04-24T09:18:04.350 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.350 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.350 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.350 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-24T09:18:04.350 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.350 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.350 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.350 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.350 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.351 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.351 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.351 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.351 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.351 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.351 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:04.351 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "test_orchestrator", 2024-04-24T09:18:04.351 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-24T09:18:04.351 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-24T09:18:04.351 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-24T09:18:04.351 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-24T09:18:04.351 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-24T09:18:04.351 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.351 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.351 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.351 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.352 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.352 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.352 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.352 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.352 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.352 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.352 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.352 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.352 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.352 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.352 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.352 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.352 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.352 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.352 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.353 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-24T09:18:04.353 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-24T09:18:04.353 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.353 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.353 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.353 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.353 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.353 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.353 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.353 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.353 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.353 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.353 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.353 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.353 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-24T09:18:04.353 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-24T09:18:04.354 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.354 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.354 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.354 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-24T09:18:04.354 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.354 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.354 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.354 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.354 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.354 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.354 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.354 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.354 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.354 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.354 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.355 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.355 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.355 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.355 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.355 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-24T09:18:04.355 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-24T09:18:04.355 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.355 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.355 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.355 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.355 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.355 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.355 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.355 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.355 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.355 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.356 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.356 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.356 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.356 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.356 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:04.356 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "volumes", 2024-04-24T09:18:04.356 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-24T09:18:04.356 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-24T09:18:04.356 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-24T09:18:04.356 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-24T09:18:04.356 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-24T09:18:04.356 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.356 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.356 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.356 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.356 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.356 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.357 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.357 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.357 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.357 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.357 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.357 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.357 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.357 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.357 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.357 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.357 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.357 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.357 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.357 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-24T09:18:04.357 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-24T09:18:04.357 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.358 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.358 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.358 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.358 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.358 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.358 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.358 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.358 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.358 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.358 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.358 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.358 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-24T09:18:04.358 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-24T09:18:04.358 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.358 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.359 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.359 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-24T09:18:04.359 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.359 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.359 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.359 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.359 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.359 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.359 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.359 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.359 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.359 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.359 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.359 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.359 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.359 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.360 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.360 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-24T09:18:04.360 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-24T09:18:04.360 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.360 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.360 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.360 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.360 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.360 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.360 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.360 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.360 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.360 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.360 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.360 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.361 INFO:teuthology.orchestra.run.smithi083.stdout: "max_concurrent_clones": { 2024-04-24T09:18:04.361 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "max_concurrent_clones", 2024-04-24T09:18:04.361 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:04.361 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.361 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.361 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "4", 2024-04-24T09:18:04.361 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.361 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.361 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.361 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-24T09:18:04.361 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.361 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.361 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.361 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.361 INFO:teuthology.orchestra.run.smithi083.stdout: "periodic_async_work": { 2024-04-24T09:18:04.362 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "periodic_async_work", 2024-04-24T09:18:04.362 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.362 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.362 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.362 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.362 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.362 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.362 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.362 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Periodically check for async work", 2024-04-24T09:18:04.362 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.362 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.362 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.362 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.362 INFO:teuthology.orchestra.run.smithi083.stdout: "snapshot_clone_delay": { 2024-04-24T09:18:04.362 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "snapshot_clone_delay", 2024-04-24T09:18:04.363 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:04.363 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.363 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.363 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-04-24T09:18:04.363 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.363 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.363 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.363 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-24T09:18:04.363 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.363 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.363 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.363 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.363 INFO:teuthology.orchestra.run.smithi083.stdout: "snapshot_clone_no_wait": { 2024-04-24T09:18:04.363 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "snapshot_clone_no_wait", 2024-04-24T09:18:04.363 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.364 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.364 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.364 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:04.364 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.364 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.364 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.364 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-24T09:18:04.364 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.364 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.364 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.364 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.364 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.364 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.364 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:04.364 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "zabbix", 2024-04-24T09:18:04.364 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-24T09:18:04.364 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-24T09:18:04.365 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-24T09:18:04.365 INFO:teuthology.orchestra.run.smithi083.stdout: "discovery_interval": { 2024-04-24T09:18:04.365 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "discovery_interval", 2024-04-24T09:18:04.365 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "uint", 2024-04-24T09:18:04.365 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.365 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.365 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "100", 2024-04-24T09:18:04.365 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.365 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.365 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.365 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.365 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.365 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.365 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.365 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.365 INFO:teuthology.orchestra.run.smithi083.stdout: "identifier": { 2024-04-24T09:18:04.366 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "identifier", 2024-04-24T09:18:04.366 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.366 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.366 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.366 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.366 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.366 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.366 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.366 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.366 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.366 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.366 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.366 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.366 INFO:teuthology.orchestra.run.smithi083.stdout: "interval": { 2024-04-24T09:18:04.367 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "interval", 2024-04-24T09:18:04.367 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-24T09:18:04.367 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.367 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.367 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "60", 2024-04-24T09:18:04.367 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.367 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.367 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.367 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.367 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.367 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.367 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.367 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.367 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-24T09:18:04.367 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-24T09:18:04.368 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.368 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.368 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.368 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.368 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.368 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.368 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.368 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.368 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.368 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.368 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.368 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.368 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.368 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.368 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.368 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.368 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.369 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.369 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.369 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-24T09:18:04.369 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-24T09:18:04.369 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.369 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.369 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.369 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.369 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.369 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.369 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.369 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.369 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.369 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.370 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.370 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.370 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-24T09:18:04.370 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-24T09:18:04.370 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.370 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.370 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.370 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-24T09:18:04.370 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.370 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.370 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.370 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.370 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.370 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.371 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.371 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.371 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.371 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.371 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.371 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.371 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.371 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.371 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.371 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-24T09:18:04.371 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-24T09:18:04.371 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.371 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.371 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.372 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.372 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.372 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.372 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.372 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.372 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.372 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.372 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.372 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.372 INFO:teuthology.orchestra.run.smithi083.stdout: "zabbix_host": { 2024-04-24T09:18:04.372 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "zabbix_host", 2024-04-24T09:18:04.372 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.372 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.372 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.372 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.372 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.373 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.373 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.373 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.373 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.373 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.373 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.373 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.373 INFO:teuthology.orchestra.run.smithi083.stdout: "zabbix_port": { 2024-04-24T09:18:04.373 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "zabbix_port", 2024-04-24T09:18:04.373 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:04.373 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.373 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.373 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10051", 2024-04-24T09:18:04.373 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.374 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.374 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.374 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.374 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.374 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.374 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.374 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.374 INFO:teuthology.orchestra.run.smithi083.stdout: "zabbix_sender": { 2024-04-24T09:18:04.374 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "zabbix_sender", 2024-04-24T09:18:04.374 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.374 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.374 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.374 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-24T09:18:04.374 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.374 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.375 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.375 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.375 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.375 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.375 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.375 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.375 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.375 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.375 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-04-24T09:18:04.375 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.375 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.375 INFO:teuthology.orchestra.run.smithi083.stdout: "modules": [ 2024-04-24T09:18:04.375 INFO:teuthology.orchestra.run.smithi083.stdout: "cephadm", 2024-04-24T09:18:04.375 INFO:teuthology.orchestra.run.smithi083.stdout: "dashboard", 2024-04-24T09:18:04.375 INFO:teuthology.orchestra.run.smithi083.stdout: "iostat", 2024-04-24T09:18:04.375 INFO:teuthology.orchestra.run.smithi083.stdout: "nfs", 2024-04-24T09:18:04.376 INFO:teuthology.orchestra.run.smithi083.stdout: "restful", 2024-04-24T09:18:04.376 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_schedule" 2024-04-24T09:18:04.376 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.376 INFO:teuthology.orchestra.run.smithi083.stdout: "available_modules": [ 2024-04-24T09:18:04.376 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:04.376 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "alerts", 2024-04-24T09:18:04.376 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-24T09:18:04.376 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-24T09:18:04.376 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-24T09:18:04.376 INFO:teuthology.orchestra.run.smithi083.stdout: "interval": { 2024-04-24T09:18:04.376 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "interval", 2024-04-24T09:18:04.376 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-24T09:18:04.376 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.376 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.376 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "60", 2024-04-24T09:18:04.377 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.377 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.377 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.377 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "How frequently to reexamine health status", 2024-04-24T09:18:04.377 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.377 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.377 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.377 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.377 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-24T09:18:04.377 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-24T09:18:04.377 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.377 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.377 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.377 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.377 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.377 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.378 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.378 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.378 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.378 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.378 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.378 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.378 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.378 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.378 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.378 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.378 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.378 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.378 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.378 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-24T09:18:04.379 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-24T09:18:04.379 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.379 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.379 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.379 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.379 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.379 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.379 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.379 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.379 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.379 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.379 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.379 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.379 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-24T09:18:04.380 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-24T09:18:04.380 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.380 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.380 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.380 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-24T09:18:04.380 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.380 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.380 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.380 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.380 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.380 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.380 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.380 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.380 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.380 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.381 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.381 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.381 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.381 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.381 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.381 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-24T09:18:04.381 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-24T09:18:04.381 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.381 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.381 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.381 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.381 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.381 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.381 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.382 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.382 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.382 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.382 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.382 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.382 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_destination": { 2024-04-24T09:18:04.382 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_destination", 2024-04-24T09:18:04.382 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.382 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.382 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.382 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.382 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.382 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.382 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.383 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Email address to send alerts to", 2024-04-24T09:18:04.383 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.383 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.383 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.383 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.383 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_from_name": { 2024-04-24T09:18:04.383 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_from_name", 2024-04-24T09:18:04.383 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.383 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.383 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.383 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "Ceph", 2024-04-24T09:18:04.383 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.383 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.383 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.383 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Email From: name", 2024-04-24T09:18:04.383 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.384 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.384 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.384 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.384 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_host": { 2024-04-24T09:18:04.384 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_host", 2024-04-24T09:18:04.384 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.384 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.384 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.384 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.384 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.384 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.384 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.384 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "SMTP server", 2024-04-24T09:18:04.384 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.384 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.384 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.385 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.385 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_password": { 2024-04-24T09:18:04.385 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_password", 2024-04-24T09:18:04.385 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.385 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.385 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.385 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.385 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.385 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.385 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.385 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Password to authenticate with", 2024-04-24T09:18:04.385 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.385 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.385 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.386 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.386 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_port": { 2024-04-24T09:18:04.386 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_port", 2024-04-24T09:18:04.386 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:04.386 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.386 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.386 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "465", 2024-04-24T09:18:04.386 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.386 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.386 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.386 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "SMTP port", 2024-04-24T09:18:04.386 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.386 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.386 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.386 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.386 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_sender": { 2024-04-24T09:18:04.387 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_sender", 2024-04-24T09:18:04.387 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.387 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.387 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.387 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.387 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.387 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.387 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.387 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "SMTP envelope sender", 2024-04-24T09:18:04.387 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.387 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.387 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.387 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.387 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_ssl": { 2024-04-24T09:18:04.387 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_ssl", 2024-04-24T09:18:04.387 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.388 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.388 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.388 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:04.388 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.388 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.388 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.388 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-24T09:18:04.388 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.388 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.388 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.388 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.388 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_user": { 2024-04-24T09:18:04.388 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_user", 2024-04-24T09:18:04.388 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.388 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.388 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.389 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.389 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.389 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.389 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.389 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "User to authenticate as", 2024-04-24T09:18:04.389 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.389 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.389 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.389 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.389 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.389 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.389 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:04.389 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "balancer", 2024-04-24T09:18:04.389 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-24T09:18:04.389 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-24T09:18:04.390 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-24T09:18:04.390 INFO:teuthology.orchestra.run.smithi083.stdout: "active": { 2024-04-24T09:18:04.390 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "active", 2024-04-24T09:18:04.390 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.390 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.390 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.390 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:04.390 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.390 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.390 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.390 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "automatically balance PGs across cluster", 2024-04-24T09:18:04.390 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.390 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.390 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.390 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.391 INFO:teuthology.orchestra.run.smithi083.stdout: "begin_time": { 2024-04-24T09:18:04.391 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "begin_time", 2024-04-24T09:18:04.391 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.391 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.391 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.391 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0000", 2024-04-24T09:18:04.391 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.391 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.391 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.391 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "beginning time of day to automatically balance", 2024-04-24T09:18:04.391 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-24T09:18:04.391 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.391 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.391 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.391 INFO:teuthology.orchestra.run.smithi083.stdout: "begin_weekday": { 2024-04-24T09:18:04.391 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "begin_weekday", 2024-04-24T09:18:04.392 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "uint", 2024-04-24T09:18:04.392 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.392 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.392 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-04-24T09:18:04.392 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "0", 2024-04-24T09:18:04.392 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "6", 2024-04-24T09:18:04.392 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.392 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-24T09:18:04.392 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-24T09:18:04.392 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.392 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.392 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.392 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_compat_max_iterations": { 2024-04-24T09:18:04.392 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "crush_compat_max_iterations", 2024-04-24T09:18:04.392 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "uint", 2024-04-24T09:18:04.392 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.393 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.393 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "25", 2024-04-24T09:18:04.393 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "1", 2024-04-24T09:18:04.393 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "250", 2024-04-24T09:18:04.393 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.393 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-24T09:18:04.393 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.393 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.393 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.393 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.393 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_compat_metrics": { 2024-04-24T09:18:04.393 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "crush_compat_metrics", 2024-04-24T09:18:04.393 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.393 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.393 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.393 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "pgs,objects,bytes", 2024-04-24T09:18:04.394 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.394 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.394 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.394 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-24T09:18:04.394 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-24T09:18:04.394 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.394 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.394 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.394 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_compat_step": { 2024-04-24T09:18:04.394 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "crush_compat_step", 2024-04-24T09:18:04.394 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-04-24T09:18:04.394 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.394 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.394 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0.5", 2024-04-24T09:18:04.394 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "0.001", 2024-04-24T09:18:04.395 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "0.999", 2024-04-24T09:18:04.395 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.395 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "aggressiveness of optimization", 2024-04-24T09:18:04.395 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-24T09:18:04.395 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.395 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.395 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.395 INFO:teuthology.orchestra.run.smithi083.stdout: "end_time": { 2024-04-24T09:18:04.395 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "end_time", 2024-04-24T09:18:04.395 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.395 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.395 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.395 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "2359", 2024-04-24T09:18:04.395 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.395 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.395 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.396 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "ending time of day to automatically balance", 2024-04-24T09:18:04.396 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-24T09:18:04.396 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.396 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.396 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.396 INFO:teuthology.orchestra.run.smithi083.stdout: "end_weekday": { 2024-04-24T09:18:04.396 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "end_weekday", 2024-04-24T09:18:04.396 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "uint", 2024-04-24T09:18:04.396 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.396 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.396 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-04-24T09:18:04.396 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "0", 2024-04-24T09:18:04.396 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "6", 2024-04-24T09:18:04.396 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.396 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-24T09:18:04.396 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-24T09:18:04.396 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.397 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.397 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.397 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-24T09:18:04.397 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-24T09:18:04.397 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.397 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.397 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.397 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.397 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.397 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.397 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.397 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.397 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.397 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.397 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.397 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.397 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.398 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.398 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.398 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.398 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.398 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.398 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.398 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-24T09:18:04.398 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-24T09:18:04.398 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.398 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.398 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.398 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.398 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.398 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.398 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.399 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.399 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.399 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.399 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.399 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.399 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-24T09:18:04.399 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-24T09:18:04.399 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.399 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.399 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.399 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-24T09:18:04.399 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.399 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.399 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.399 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.399 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.400 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.400 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.400 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.400 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.400 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.400 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.400 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.400 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.400 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.400 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.400 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-24T09:18:04.400 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-24T09:18:04.400 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.400 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.400 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.401 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.401 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.401 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.401 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.401 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.401 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.401 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.401 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.401 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.401 INFO:teuthology.orchestra.run.smithi083.stdout: "min_score": { 2024-04-24T09:18:04.401 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "min_score", 2024-04-24T09:18:04.401 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-04-24T09:18:04.401 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.401 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.402 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-04-24T09:18:04.402 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.402 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.402 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.402 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-24T09:18:04.402 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.402 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.402 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.402 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.402 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": { 2024-04-24T09:18:04.402 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mode", 2024-04-24T09:18:04.402 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.402 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.402 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.403 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "upmap", 2024-04-24T09:18:04.403 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.403 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.403 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.403 INFO:teuthology.orchestra.run.smithi083.stdout: "crush-compat", 2024-04-24T09:18:04.403 INFO:teuthology.orchestra.run.smithi083.stdout: "none", 2024-04-24T09:18:04.403 INFO:teuthology.orchestra.run.smithi083.stdout: "read", 2024-04-24T09:18:04.403 INFO:teuthology.orchestra.run.smithi083.stdout: "upmap", 2024-04-24T09:18:04.403 INFO:teuthology.orchestra.run.smithi083.stdout: "upmap-read" 2024-04-24T09:18:04.403 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.403 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Balancer mode", 2024-04-24T09:18:04.403 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.403 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.403 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.403 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.403 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_ids": { 2024-04-24T09:18:04.404 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "pool_ids", 2024-04-24T09:18:04.404 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.404 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.404 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.404 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.404 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.404 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.404 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.404 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-24T09:18:04.404 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.404 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.404 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.404 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.404 INFO:teuthology.orchestra.run.smithi083.stdout: "sleep_interval": { 2024-04-24T09:18:04.404 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sleep_interval", 2024-04-24T09:18:04.404 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-24T09:18:04.405 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.405 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.405 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "60", 2024-04-24T09:18:04.405 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.405 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.405 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.405 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-24T09:18:04.405 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.405 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.405 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.405 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.405 INFO:teuthology.orchestra.run.smithi083.stdout: "upmap_max_deviation": { 2024-04-24T09:18:04.405 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "upmap_max_deviation", 2024-04-24T09:18:04.405 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:04.405 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.406 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.406 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "5", 2024-04-24T09:18:04.406 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "1", 2024-04-24T09:18:04.406 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.406 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.406 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-24T09:18:04.406 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-24T09:18:04.406 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.406 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.406 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.406 INFO:teuthology.orchestra.run.smithi083.stdout: "upmap_max_optimizations": { 2024-04-24T09:18:04.406 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "upmap_max_optimizations", 2024-04-24T09:18:04.406 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "uint", 2024-04-24T09:18:04.406 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.406 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.407 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-04-24T09:18:04.407 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.407 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.407 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.407 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-24T09:18:04.407 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.407 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.407 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.407 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.407 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.407 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.407 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:04.407 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "cephadm", 2024-04-24T09:18:04.407 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-24T09:18:04.407 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-24T09:18:04.408 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-24T09:18:04.408 INFO:teuthology.orchestra.run.smithi083.stdout: "agent_down_multiplier": { 2024-04-24T09:18:04.408 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "agent_down_multiplier", 2024-04-24T09:18:04.408 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-04-24T09:18:04.408 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.408 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.408 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "3.0", 2024-04-24T09:18:04.408 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.408 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.408 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.408 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-24T09:18:04.408 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.408 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.408 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.408 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.408 INFO:teuthology.orchestra.run.smithi083.stdout: "agent_refresh_rate": { 2024-04-24T09:18:04.409 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "agent_refresh_rate", 2024-04-24T09:18:04.409 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-24T09:18:04.409 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.409 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.409 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "20", 2024-04-24T09:18:04.409 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.409 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.409 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.409 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-24T09:18:04.409 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.409 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.409 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.409 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.409 INFO:teuthology.orchestra.run.smithi083.stdout: "agent_starting_port": { 2024-04-24T09:18:04.409 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "agent_starting_port", 2024-04-24T09:18:04.409 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:04.409 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.410 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.410 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "4721", 2024-04-24T09:18:04.410 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.410 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.410 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.410 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-24T09:18:04.410 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.410 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.410 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.410 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.410 INFO:teuthology.orchestra.run.smithi083.stdout: "allow_ptrace": { 2024-04-24T09:18:04.410 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "allow_ptrace", 2024-04-24T09:18:04.410 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.410 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.411 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.411 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.411 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.411 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.411 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.411 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-24T09:18:04.411 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-24T09:18:04.411 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.411 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.411 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.411 INFO:teuthology.orchestra.run.smithi083.stdout: "autotune_interval": { 2024-04-24T09:18:04.411 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "autotune_interval", 2024-04-24T09:18:04.411 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-24T09:18:04.411 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.411 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.412 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "600", 2024-04-24T09:18:04.412 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.412 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.412 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.412 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-24T09:18:04.412 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.412 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.412 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.412 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.412 INFO:teuthology.orchestra.run.smithi083.stdout: "autotune_memory_target_ratio": { 2024-04-24T09:18:04.412 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "autotune_memory_target_ratio", 2024-04-24T09:18:04.412 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-04-24T09:18:04.412 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.412 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.412 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0.7", 2024-04-24T09:18:04.412 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.413 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.413 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.413 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-24T09:18:04.413 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.413 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.413 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.413 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.413 INFO:teuthology.orchestra.run.smithi083.stdout: "cephadm_log_destination": { 2024-04-24T09:18:04.413 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "cephadm_log_destination", 2024-04-24T09:18:04.413 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.413 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.413 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.413 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.413 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.413 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.413 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.414 INFO:teuthology.orchestra.run.smithi083.stdout: "file", 2024-04-24T09:18:04.414 INFO:teuthology.orchestra.run.smithi083.stdout: "file,syslog", 2024-04-24T09:18:04.414 INFO:teuthology.orchestra.run.smithi083.stdout: "syslog" 2024-04-24T09:18:04.414 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.414 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-24T09:18:04.414 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.414 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.414 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.414 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.414 INFO:teuthology.orchestra.run.smithi083.stdout: "cgroups_split": { 2024-04-24T09:18:04.414 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "cgroups_split", 2024-04-24T09:18:04.414 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.414 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.414 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.415 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:04.415 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.415 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.415 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.415 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-24T09:18:04.415 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.415 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.415 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.415 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.415 INFO:teuthology.orchestra.run.smithi083.stdout: "config_checks_enabled": { 2024-04-24T09:18:04.415 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "config_checks_enabled", 2024-04-24T09:18:04.415 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.415 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.415 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.415 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.416 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.416 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.416 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.416 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-24T09:18:04.416 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.416 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.416 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.416 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.416 INFO:teuthology.orchestra.run.smithi083.stdout: "config_dashboard": { 2024-04-24T09:18:04.416 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "config_dashboard", 2024-04-24T09:18:04.416 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.416 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.416 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.416 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:04.416 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.417 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.417 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.417 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-24T09:18:04.417 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.417 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.417 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.417 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.417 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_alertmanager": { 2024-04-24T09:18:04.417 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_alertmanager", 2024-04-24T09:18:04.417 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.417 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.417 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.417 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-24T09:18:04.417 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.417 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.417 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.418 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Prometheus container image", 2024-04-24T09:18:04.418 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.418 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.418 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.418 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.418 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_base": { 2024-04-24T09:18:04.418 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_base", 2024-04-24T09:18:04.418 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.418 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.418 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.418 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-24T09:18:04.418 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.418 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.418 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.419 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Container image name, without the tag", 2024-04-24T09:18:04.419 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.419 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.419 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.419 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.419 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_elasticsearch": { 2024-04-24T09:18:04.419 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_elasticsearch", 2024-04-24T09:18:04.419 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.419 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.419 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.419 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-24T09:18:04.419 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.419 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.419 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.419 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "elasticsearch container image", 2024-04-24T09:18:04.420 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.420 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.420 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.420 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.420 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_grafana": { 2024-04-24T09:18:04.420 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_grafana", 2024-04-24T09:18:04.420 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.420 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.420 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.420 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-24T09:18:04.420 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.420 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.420 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.420 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Prometheus container image", 2024-04-24T09:18:04.420 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.421 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.421 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.421 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.421 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_haproxy": { 2024-04-24T09:18:04.421 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_haproxy", 2024-04-24T09:18:04.421 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.421 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.421 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.421 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-24T09:18:04.421 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.421 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.421 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.421 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "HAproxy container image", 2024-04-24T09:18:04.421 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.422 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.422 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.422 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.422 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_jaeger_agent": { 2024-04-24T09:18:04.422 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_jaeger_agent", 2024-04-24T09:18:04.422 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.422 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.422 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.422 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-24T09:18:04.422 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.422 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.422 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.422 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Jaeger agent container image", 2024-04-24T09:18:04.422 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.422 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.423 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.423 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.423 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_jaeger_collector": { 2024-04-24T09:18:04.423 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_jaeger_collector", 2024-04-24T09:18:04.423 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.423 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.423 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.423 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-24T09:18:04.423 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.423 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.423 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.423 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Jaeger collector container image", 2024-04-24T09:18:04.423 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.423 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.423 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.423 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.424 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_jaeger_query": { 2024-04-24T09:18:04.424 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_jaeger_query", 2024-04-24T09:18:04.424 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.424 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.424 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.424 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-24T09:18:04.424 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.424 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.424 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.424 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Jaeger query container image", 2024-04-24T09:18:04.424 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.424 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.424 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.424 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.424 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_keepalived": { 2024-04-24T09:18:04.424 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_keepalived", 2024-04-24T09:18:04.424 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.425 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.425 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.425 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-24T09:18:04.425 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.425 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.425 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.425 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Keepalived container image", 2024-04-24T09:18:04.425 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.425 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.425 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.425 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.425 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_loki": { 2024-04-24T09:18:04.425 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_loki", 2024-04-24T09:18:04.425 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.425 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.425 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.426 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-24T09:18:04.426 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.426 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.426 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.426 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Loki container image", 2024-04-24T09:18:04.426 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.426 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.426 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.426 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.426 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_node_exporter": { 2024-04-24T09:18:04.426 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_node_exporter", 2024-04-24T09:18:04.426 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.426 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.426 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.427 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-24T09:18:04.427 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.427 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.427 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.427 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Prometheus container image", 2024-04-24T09:18:04.427 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.427 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.427 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.427 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.427 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_nvmeof": { 2024-04-24T09:18:04.427 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_nvmeof", 2024-04-24T09:18:04.427 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.427 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.427 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.427 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-24T09:18:04.427 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.428 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.428 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.428 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Nvme-of container image", 2024-04-24T09:18:04.428 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.428 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.428 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.428 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.428 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_prometheus": { 2024-04-24T09:18:04.428 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_prometheus", 2024-04-24T09:18:04.428 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.428 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.428 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.428 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-24T09:18:04.428 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.428 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.429 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.429 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Prometheus container image", 2024-04-24T09:18:04.429 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.429 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.429 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.429 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.429 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_promtail": { 2024-04-24T09:18:04.429 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_promtail", 2024-04-24T09:18:04.429 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.429 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.429 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.429 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-24T09:18:04.429 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.429 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.430 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.430 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Promtail container image", 2024-04-24T09:18:04.430 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.430 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.430 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.430 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.430 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_samba": { 2024-04-24T09:18:04.430 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_samba", 2024-04-24T09:18:04.430 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.430 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.430 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.430 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-24T09:18:04.430 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.431 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.431 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.431 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Samba/SMB container image", 2024-04-24T09:18:04.431 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.431 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.431 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.431 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.431 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_snmp_gateway": { 2024-04-24T09:18:04.431 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_snmp_gateway", 2024-04-24T09:18:04.431 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.431 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.431 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.431 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-24T09:18:04.431 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.432 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.432 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.432 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "SNMP Gateway container image", 2024-04-24T09:18:04.432 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.432 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.432 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.432 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.432 INFO:teuthology.orchestra.run.smithi083.stdout: "container_init": { 2024-04-24T09:18:04.432 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_init", 2024-04-24T09:18:04.432 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.432 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.432 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.432 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:04.432 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.432 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.432 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.433 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Run podman/docker with `--init`", 2024-04-24T09:18:04.433 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.433 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.433 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.433 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.433 INFO:teuthology.orchestra.run.smithi083.stdout: "daemon_cache_timeout": { 2024-04-24T09:18:04.433 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "daemon_cache_timeout", 2024-04-24T09:18:04.433 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-24T09:18:04.433 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.433 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.433 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "600", 2024-04-24T09:18:04.433 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.433 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.433 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.433 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-24T09:18:04.434 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.434 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.434 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.434 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.434 INFO:teuthology.orchestra.run.smithi083.stdout: "default_cephadm_command_timeout": { 2024-04-24T09:18:04.434 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "default_cephadm_command_timeout", 2024-04-24T09:18:04.434 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:04.434 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.434 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.434 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "900", 2024-04-24T09:18:04.434 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.434 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.434 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.434 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-24T09:18:04.434 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.435 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.435 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.435 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.435 INFO:teuthology.orchestra.run.smithi083.stdout: "default_registry": { 2024-04-24T09:18:04.435 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "default_registry", 2024-04-24T09:18:04.435 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.435 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.435 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.435 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "docker.io", 2024-04-24T09:18:04.435 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.435 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.435 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.435 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-24T09:18:04.435 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.435 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.435 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.436 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.436 INFO:teuthology.orchestra.run.smithi083.stdout: "device_cache_timeout": { 2024-04-24T09:18:04.436 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "device_cache_timeout", 2024-04-24T09:18:04.436 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-24T09:18:04.436 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.436 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.436 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "1800", 2024-04-24T09:18:04.436 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.436 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.436 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.436 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "seconds to cache device inventory", 2024-04-24T09:18:04.436 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.436 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.436 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.436 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.436 INFO:teuthology.orchestra.run.smithi083.stdout: "device_enhanced_scan": { 2024-04-24T09:18:04.437 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "device_enhanced_scan", 2024-04-24T09:18:04.437 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.437 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.437 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.437 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.437 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.437 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.437 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.437 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-24T09:18:04.437 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.437 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.437 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.437 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.437 INFO:teuthology.orchestra.run.smithi083.stdout: "facts_cache_timeout": { 2024-04-24T09:18:04.437 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "facts_cache_timeout", 2024-04-24T09:18:04.437 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-24T09:18:04.438 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.438 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.438 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "60", 2024-04-24T09:18:04.438 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.438 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.438 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.438 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "seconds to cache host facts data", 2024-04-24T09:18:04.438 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.438 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.438 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.438 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.438 INFO:teuthology.orchestra.run.smithi083.stdout: "grafana_dashboards_path": { 2024-04-24T09:18:04.438 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "grafana_dashboards_path", 2024-04-24T09:18:04.438 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.438 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.439 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.439 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-24T09:18:04.439 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.439 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.439 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.439 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-24T09:18:04.439 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.439 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.439 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.439 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.439 INFO:teuthology.orchestra.run.smithi083.stdout: "host_check_interval": { 2024-04-24T09:18:04.439 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "host_check_interval", 2024-04-24T09:18:04.439 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-24T09:18:04.439 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.439 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.440 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "600", 2024-04-24T09:18:04.440 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.440 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.440 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.440 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how frequently to perform a host check", 2024-04-24T09:18:04.440 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.440 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.440 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.440 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.440 INFO:teuthology.orchestra.run.smithi083.stdout: "hw_monitoring": { 2024-04-24T09:18:04.440 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "hw_monitoring", 2024-04-24T09:18:04.440 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.440 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.440 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.441 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.441 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.441 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.441 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.441 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-24T09:18:04.441 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.441 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.441 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.441 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.441 INFO:teuthology.orchestra.run.smithi083.stdout: "inventory_list_all": { 2024-04-24T09:18:04.441 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "inventory_list_all", 2024-04-24T09:18:04.441 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.441 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.441 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.441 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.442 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.442 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.442 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.442 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-24T09:18:04.442 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.442 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.442 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.442 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.442 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-24T09:18:04.442 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-24T09:18:04.442 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.442 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.442 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.443 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.443 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.443 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.443 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.443 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.443 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.443 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.443 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.443 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.443 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.443 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.443 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.443 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.443 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.443 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.444 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.444 INFO:teuthology.orchestra.run.smithi083.stdout: "log_refresh_metadata": { 2024-04-24T09:18:04.444 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_refresh_metadata", 2024-04-24T09:18:04.444 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.444 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.444 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.444 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.444 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.444 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.444 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.444 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-24T09:18:04.444 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.444 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.444 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.444 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.444 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-24T09:18:04.445 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-24T09:18:04.445 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.445 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.445 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.445 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:04.445 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.445 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.445 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.445 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-24T09:18:04.445 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.445 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.445 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.445 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.445 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-24T09:18:04.445 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-24T09:18:04.445 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.446 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.446 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.446 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-24T09:18:04.446 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.446 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.446 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.446 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.446 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.446 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.446 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.446 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.446 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.446 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.446 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.447 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.447 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.447 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.447 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.447 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-24T09:18:04.447 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-24T09:18:04.447 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.447 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.447 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.447 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.447 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.447 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.447 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.447 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.447 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.447 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.448 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.448 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.448 INFO:teuthology.orchestra.run.smithi083.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-24T09:18:04.448 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-24T09:18:04.448 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.448 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.448 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.448 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.448 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.448 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.448 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.448 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-24T09:18:04.448 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.448 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.448 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.448 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.449 INFO:teuthology.orchestra.run.smithi083.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-24T09:18:04.449 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-24T09:18:04.449 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.449 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.449 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.449 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "*", 2024-04-24T09:18:04.449 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.449 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.449 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.449 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-24T09:18:04.449 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.449 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.449 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.449 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.449 INFO:teuthology.orchestra.run.smithi083.stdout: "max_count_per_host": { 2024-04-24T09:18:04.449 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "max_count_per_host", 2024-04-24T09:18:04.450 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:04.450 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.450 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.450 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-04-24T09:18:04.450 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.450 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.450 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.450 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "max number of daemons per service per host", 2024-04-24T09:18:04.450 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.450 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.450 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.450 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.450 INFO:teuthology.orchestra.run.smithi083.stdout: "max_osd_draining_count": { 2024-04-24T09:18:04.450 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "max_osd_draining_count", 2024-04-24T09:18:04.450 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:04.451 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.451 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.451 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-04-24T09:18:04.451 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.451 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.451 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.451 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-24T09:18:04.451 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.451 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.451 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.451 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.451 INFO:teuthology.orchestra.run.smithi083.stdout: "migration_current": { 2024-04-24T09:18:04.451 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "migration_current", 2024-04-24T09:18:04.451 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:04.451 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.451 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.452 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.452 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.452 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.452 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.452 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "internal - do not modify", 2024-04-24T09:18:04.452 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.452 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.452 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.452 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.452 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": { 2024-04-24T09:18:04.452 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mode", 2024-04-24T09:18:04.452 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.452 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.452 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.452 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "root", 2024-04-24T09:18:04.453 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.453 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.453 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.453 INFO:teuthology.orchestra.run.smithi083.stdout: "cephadm-package", 2024-04-24T09:18:04.453 INFO:teuthology.orchestra.run.smithi083.stdout: "root" 2024-04-24T09:18:04.453 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.453 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "mode for remote execution of cephadm", 2024-04-24T09:18:04.453 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.453 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.453 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.453 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.453 INFO:teuthology.orchestra.run.smithi083.stdout: "oob_default_addr": { 2024-04-24T09:18:04.453 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "oob_default_addr", 2024-04-24T09:18:04.453 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.453 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.453 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.454 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "169.254.1.1", 2024-04-24T09:18:04.454 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.454 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.454 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.454 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-24T09:18:04.454 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.454 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.454 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.454 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.454 INFO:teuthology.orchestra.run.smithi083.stdout: "prometheus_alerts_path": { 2024-04-24T09:18:04.454 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "prometheus_alerts_path", 2024-04-24T09:18:04.454 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.454 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.454 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.454 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-24T09:18:04.454 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.455 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.455 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.455 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-24T09:18:04.455 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.455 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.455 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.455 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.455 INFO:teuthology.orchestra.run.smithi083.stdout: "registry_insecure": { 2024-04-24T09:18:04.455 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "registry_insecure", 2024-04-24T09:18:04.455 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.455 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.455 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.455 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.455 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.455 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.455 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.456 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-24T09:18:04.456 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.456 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.456 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.456 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.456 INFO:teuthology.orchestra.run.smithi083.stdout: "registry_password": { 2024-04-24T09:18:04.456 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "registry_password", 2024-04-24T09:18:04.456 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.456 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.456 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.456 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.456 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.456 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.456 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.456 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-24T09:18:04.456 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.457 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.457 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.457 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.457 INFO:teuthology.orchestra.run.smithi083.stdout: "registry_url": { 2024-04-24T09:18:04.457 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "registry_url", 2024-04-24T09:18:04.457 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.457 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.457 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.457 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.457 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.457 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.457 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.457 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-24T09:18:04.457 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.457 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.457 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.458 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.458 INFO:teuthology.orchestra.run.smithi083.stdout: "registry_username": { 2024-04-24T09:18:04.458 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "registry_username", 2024-04-24T09:18:04.458 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.458 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.458 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.458 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.458 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.458 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.458 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.458 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-24T09:18:04.458 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.458 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.458 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.458 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.459 INFO:teuthology.orchestra.run.smithi083.stdout: "secure_monitoring_stack": { 2024-04-24T09:18:04.459 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "secure_monitoring_stack", 2024-04-24T09:18:04.459 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.459 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.459 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.459 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.459 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.459 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.459 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.459 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-24T09:18:04.459 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.459 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.459 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.459 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.459 INFO:teuthology.orchestra.run.smithi083.stdout: "service_discovery_port": { 2024-04-24T09:18:04.459 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "service_discovery_port", 2024-04-24T09:18:04.460 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:04.460 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.460 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.460 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "8765", 2024-04-24T09:18:04.460 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.460 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.460 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.460 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "cephadm service discovery port", 2024-04-24T09:18:04.460 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.460 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.460 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.460 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.460 INFO:teuthology.orchestra.run.smithi083.stdout: "ssh_config_file": { 2024-04-24T09:18:04.460 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ssh_config_file", 2024-04-24T09:18:04.460 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.460 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.461 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.461 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.461 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.461 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.461 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.461 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-24T09:18:04.461 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.461 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.461 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.461 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.461 INFO:teuthology.orchestra.run.smithi083.stdout: "use_agent": { 2024-04-24T09:18:04.461 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "use_agent", 2024-04-24T09:18:04.461 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.461 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.461 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.461 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.462 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.462 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.462 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.462 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-24T09:18:04.462 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.462 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.462 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.462 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.462 INFO:teuthology.orchestra.run.smithi083.stdout: "use_repo_digest": { 2024-04-24T09:18:04.462 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "use_repo_digest", 2024-04-24T09:18:04.462 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.462 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.462 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.462 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:04.462 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.463 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.463 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.463 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-24T09:18:04.463 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.463 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.463 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.463 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.463 INFO:teuthology.orchestra.run.smithi083.stdout: "warn_on_failed_host_check": { 2024-04-24T09:18:04.463 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "warn_on_failed_host_check", 2024-04-24T09:18:04.463 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.463 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.463 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.463 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:04.463 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.463 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.463 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.463 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "raise a health warning if the host check fails", 2024-04-24T09:18:04.464 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.464 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.464 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.464 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.464 INFO:teuthology.orchestra.run.smithi083.stdout: "warn_on_stray_daemons": { 2024-04-24T09:18:04.464 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "warn_on_stray_daemons", 2024-04-24T09:18:04.464 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.464 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.464 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.464 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:04.464 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.464 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.464 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.464 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-24T09:18:04.464 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.464 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.465 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.465 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.465 INFO:teuthology.orchestra.run.smithi083.stdout: "warn_on_stray_hosts": { 2024-04-24T09:18:04.465 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "warn_on_stray_hosts", 2024-04-24T09:18:04.465 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.465 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.465 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.465 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:04.465 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.465 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.465 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.465 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-24T09:18:04.465 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.465 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.465 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.465 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.465 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.466 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.466 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:04.466 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "crash", 2024-04-24T09:18:04.466 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-24T09:18:04.466 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-24T09:18:04.466 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-24T09:18:04.466 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-24T09:18:04.466 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-24T09:18:04.466 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.466 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.466 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.466 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.466 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.466 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.466 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.467 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.467 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.467 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.467 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.467 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.467 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.467 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.467 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.467 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.467 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.467 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.467 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.467 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-24T09:18:04.467 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-24T09:18:04.467 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.467 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.468 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.468 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.468 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.468 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.468 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.468 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.468 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.468 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.468 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.468 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.468 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-24T09:18:04.468 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-24T09:18:04.468 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.468 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.468 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.468 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-24T09:18:04.469 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.469 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.469 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.469 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.469 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.469 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.469 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.469 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.469 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.469 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.469 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.469 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.469 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.469 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.470 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.470 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-24T09:18:04.470 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-24T09:18:04.470 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.470 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.470 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.470 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.470 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.470 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.470 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.470 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.470 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.470 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.470 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.471 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.471 INFO:teuthology.orchestra.run.smithi083.stdout: "retain_interval": { 2024-04-24T09:18:04.471 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "retain_interval", 2024-04-24T09:18:04.471 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-24T09:18:04.471 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.471 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.471 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "31536000", 2024-04-24T09:18:04.471 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.471 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.471 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.471 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-24T09:18:04.471 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.471 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.471 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.471 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.472 INFO:teuthology.orchestra.run.smithi083.stdout: "warn_recent_interval": { 2024-04-24T09:18:04.472 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "warn_recent_interval", 2024-04-24T09:18:04.472 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-24T09:18:04.472 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.472 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.472 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "1209600", 2024-04-24T09:18:04.472 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.472 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.472 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.472 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-24T09:18:04.472 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.472 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.472 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.472 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.473 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.473 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.473 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:04.473 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "dashboard", 2024-04-24T09:18:04.473 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-24T09:18:04.473 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-24T09:18:04.473 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-24T09:18:04.473 INFO:teuthology.orchestra.run.smithi083.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-24T09:18:04.473 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-24T09:18:04.473 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:04.473 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.473 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.473 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-04-24T09:18:04.473 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.474 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.474 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.474 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.474 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.474 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.474 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.474 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.474 INFO:teuthology.orchestra.run.smithi083.stdout: "ALERTMANAGER_API_HOST": { 2024-04-24T09:18:04.474 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-24T09:18:04.474 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.474 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.474 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.474 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.474 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.475 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.475 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.475 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.475 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.475 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.475 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.475 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.475 INFO:teuthology.orchestra.run.smithi083.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-24T09:18:04.475 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-24T09:18:04.475 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.475 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.475 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.475 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:04.475 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.475 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.475 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.476 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.476 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.476 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.476 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.476 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.476 INFO:teuthology.orchestra.run.smithi083.stdout: "AUDIT_API_ENABLED": { 2024-04-24T09:18:04.476 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "AUDIT_API_ENABLED", 2024-04-24T09:18:04.476 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.476 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.476 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.476 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.476 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.476 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.476 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.476 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.477 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.477 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.477 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.477 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.477 INFO:teuthology.orchestra.run.smithi083.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-24T09:18:04.477 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-24T09:18:04.477 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.477 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.477 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.477 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:04.477 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.477 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.477 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.477 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.477 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.478 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.478 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.478 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.478 INFO:teuthology.orchestra.run.smithi083.stdout: "ENABLE_BROWSABLE_API": { 2024-04-24T09:18:04.478 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-24T09:18:04.478 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.478 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.478 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.478 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:04.478 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.478 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.478 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.478 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.478 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.479 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.479 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.479 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.479 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-24T09:18:04.479 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-24T09:18:04.479 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.479 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.479 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.479 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:04.479 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.479 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.479 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.479 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.480 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.480 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.480 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.480 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.480 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-24T09:18:04.480 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-24T09:18:04.480 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.480 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.480 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.480 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:04.480 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.480 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.480 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.480 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.480 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.481 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.481 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.481 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.481 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-24T09:18:04.481 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-24T09:18:04.481 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.481 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.481 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.481 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:04.481 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.481 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.481 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.481 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.481 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.481 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.482 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.482 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.482 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-24T09:18:04.482 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-24T09:18:04.482 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.482 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.482 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.482 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:04.482 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.482 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.482 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.482 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.482 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.483 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.483 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.483 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.483 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-24T09:18:04.483 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-24T09:18:04.483 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.485 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.485 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.485 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:04.486 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.486 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.486 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.486 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.486 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.486 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.486 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.486 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.486 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-24T09:18:04.486 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-24T09:18:04.486 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.486 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.486 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.487 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:04.487 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.487 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.487 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.487 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.487 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.487 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.487 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.487 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.487 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-24T09:18:04.487 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-24T09:18:04.487 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.487 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.487 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.487 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:04.488 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.488 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.488 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.488 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.488 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.488 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.488 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.488 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.488 INFO:teuthology.orchestra.run.smithi083.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-24T09:18:04.488 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-24T09:18:04.488 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.488 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.488 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.488 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.488 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.489 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.489 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.489 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.489 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.489 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.489 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.489 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.489 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_API_PASSWORD": { 2024-04-24T09:18:04.489 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-24T09:18:04.489 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.489 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.489 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.489 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "admin", 2024-04-24T09:18:04.489 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.489 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.490 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.490 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.490 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.490 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.490 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.490 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.490 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-24T09:18:04.490 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-24T09:18:04.490 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.490 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.490 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.490 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:04.490 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.490 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.491 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.491 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.491 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.491 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.491 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.491 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.491 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_API_URL": { 2024-04-24T09:18:04.491 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_API_URL", 2024-04-24T09:18:04.491 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.491 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.491 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.491 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.491 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.491 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.491 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.492 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.492 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.492 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.492 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.492 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.492 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_API_USERNAME": { 2024-04-24T09:18:04.492 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-24T09:18:04.492 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.492 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.492 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.492 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "admin", 2024-04-24T09:18:04.492 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.492 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.492 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.492 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.492 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.493 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.493 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.493 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.493 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-24T09:18:04.493 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-24T09:18:04.493 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.493 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.493 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.493 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.493 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.493 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.493 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.493 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.493 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.493 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.493 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.494 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.494 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-24T09:18:04.494 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-24T09:18:04.494 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.494 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.494 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.494 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.494 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.494 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.494 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.494 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.494 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.494 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.494 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.494 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.494 INFO:teuthology.orchestra.run.smithi083.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-24T09:18:04.495 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-24T09:18:04.495 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.495 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.495 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.495 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:04.495 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.495 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.495 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.495 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.495 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.495 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.495 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.495 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.495 INFO:teuthology.orchestra.run.smithi083.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-24T09:18:04.495 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-24T09:18:04.496 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.496 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.496 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.496 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.496 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.496 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.496 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.496 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.496 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.496 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.496 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.496 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.496 INFO:teuthology.orchestra.run.smithi083.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-24T09:18:04.496 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-24T09:18:04.496 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.496 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.497 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.497 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "[]", 2024-04-24T09:18:04.497 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.497 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.497 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.497 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.497 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.497 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.497 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.497 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.497 INFO:teuthology.orchestra.run.smithi083.stdout: "MULTICLUSTER_CONFIG": { 2024-04-24T09:18:04.497 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-24T09:18:04.497 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.497 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.497 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.497 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "{}", 2024-04-24T09:18:04.498 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.498 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.498 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.498 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.498 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.498 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.498 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.498 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.498 INFO:teuthology.orchestra.run.smithi083.stdout: "PROMETHEUS_API_HOST": { 2024-04-24T09:18:04.498 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-24T09:18:04.498 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.498 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.498 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.498 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.498 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.498 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.499 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.499 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.499 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.499 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.499 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.499 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.499 INFO:teuthology.orchestra.run.smithi083.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-24T09:18:04.499 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-24T09:18:04.499 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.499 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.499 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.499 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:04.499 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.499 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.499 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.500 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.500 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.500 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.500 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.500 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.500 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-24T09:18:04.500 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-24T09:18:04.500 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.500 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.500 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.500 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.500 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.500 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.500 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.501 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.501 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.501 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.501 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.501 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.501 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-24T09:18:04.501 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-24T09:18:04.501 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.501 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.501 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.501 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.501 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.501 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.501 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.501 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.502 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.502 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.502 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.502 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.502 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-24T09:18:04.502 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-24T09:18:04.502 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.502 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.502 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.502 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:04.502 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.502 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.502 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.502 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.503 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.503 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.503 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.503 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.503 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-24T09:18:04.503 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-24T09:18:04.503 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.503 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.503 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.503 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:04.503 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.503 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.503 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.503 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.503 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.503 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.504 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.504 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.504 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-24T09:18:04.504 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-24T09:18:04.504 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.504 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.504 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.504 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.504 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.504 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.504 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.504 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.504 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.504 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.504 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.505 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.505 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-24T09:18:04.505 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-24T09:18:04.505 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.505 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.505 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.505 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.505 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.505 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.505 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.505 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.505 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.505 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.505 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.505 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.505 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-24T09:18:04.506 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-24T09:18:04.506 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.506 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.506 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.506 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.506 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.506 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.506 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.506 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.506 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.506 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.506 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.506 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.506 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_ENABLED": { 2024-04-24T09:18:04.506 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-24T09:18:04.506 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.507 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.507 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.507 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:04.507 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.507 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.507 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.507 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.507 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.507 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.507 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.507 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.507 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-24T09:18:04.507 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-24T09:18:04.507 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.507 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.508 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.508 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-24T09:18:04.508 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.508 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.508 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.508 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.508 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.508 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.508 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.508 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.508 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-24T09:18:04.508 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-24T09:18:04.508 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:04.508 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.508 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.508 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-04-24T09:18:04.509 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.509 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.509 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.509 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.509 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.509 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.509 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.509 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.509 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-24T09:18:04.509 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-24T09:18:04.509 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:04.509 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.509 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.509 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "8", 2024-04-24T09:18:04.509 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.509 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.510 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.510 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.510 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.510 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.510 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.510 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.510 INFO:teuthology.orchestra.run.smithi083.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-24T09:18:04.510 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-24T09:18:04.510 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:04.510 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.510 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.510 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "45", 2024-04-24T09:18:04.510 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.510 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.511 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.511 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.511 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.511 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.511 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.511 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.511 INFO:teuthology.orchestra.run.smithi083.stdout: "RGW_API_ACCESS_KEY": { 2024-04-24T09:18:04.511 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-24T09:18:04.511 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.511 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.511 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.511 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.511 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.511 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.511 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.512 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.512 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.512 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.512 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.512 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.512 INFO:teuthology.orchestra.run.smithi083.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-24T09:18:04.512 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-24T09:18:04.512 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.512 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.512 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.512 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "admin", 2024-04-24T09:18:04.512 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.512 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.512 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.512 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.512 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.513 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.513 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.513 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.513 INFO:teuthology.orchestra.run.smithi083.stdout: "RGW_API_SECRET_KEY": { 2024-04-24T09:18:04.513 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-24T09:18:04.513 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.513 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.513 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.513 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.513 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.513 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.513 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.513 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.513 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.513 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.513 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.514 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.514 INFO:teuthology.orchestra.run.smithi083.stdout: "RGW_API_SSL_VERIFY": { 2024-04-24T09:18:04.514 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-24T09:18:04.514 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.514 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.514 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.514 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:04.514 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.514 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.514 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.514 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.514 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.514 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.514 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.514 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.515 INFO:teuthology.orchestra.run.smithi083.stdout: "UNSAFE_TLS_v1_2": { 2024-04-24T09:18:04.515 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-24T09:18:04.515 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.515 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.515 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.515 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.515 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.515 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.515 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.515 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.515 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.515 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.515 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.515 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.515 INFO:teuthology.orchestra.run.smithi083.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-24T09:18:04.516 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-24T09:18:04.516 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:04.516 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.516 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.516 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-04-24T09:18:04.516 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.516 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.516 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.516 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.516 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.516 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.516 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.516 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.516 INFO:teuthology.orchestra.run.smithi083.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-24T09:18:04.516 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-24T09:18:04.516 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:04.517 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.517 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.517 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-04-24T09:18:04.517 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.517 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.517 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.517 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.517 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.517 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.517 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.517 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.517 INFO:teuthology.orchestra.run.smithi083.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-24T09:18:04.517 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-24T09:18:04.517 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:04.517 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.517 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.518 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "5", 2024-04-24T09:18:04.518 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.518 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.518 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.518 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.518 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.518 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.518 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.518 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.518 INFO:teuthology.orchestra.run.smithi083.stdout: "cross_origin_url": { 2024-04-24T09:18:04.518 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "cross_origin_url", 2024-04-24T09:18:04.518 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.518 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.518 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.518 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.519 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.519 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.519 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.519 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.519 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.519 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.519 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.519 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.519 INFO:teuthology.orchestra.run.smithi083.stdout: "crt_file": { 2024-04-24T09:18:04.519 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "crt_file", 2024-04-24T09:18:04.519 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.519 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.519 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.519 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.519 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.519 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.520 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.520 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.520 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.520 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.520 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.520 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.520 INFO:teuthology.orchestra.run.smithi083.stdout: "debug": { 2024-04-24T09:18:04.520 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "debug", 2024-04-24T09:18:04.520 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.520 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.520 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.520 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.520 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.520 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.520 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.521 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Enable/disable debug options", 2024-04-24T09:18:04.521 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.521 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.521 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.521 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.521 INFO:teuthology.orchestra.run.smithi083.stdout: "jwt_token_ttl": { 2024-04-24T09:18:04.521 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "jwt_token_ttl", 2024-04-24T09:18:04.521 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:04.521 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.521 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.521 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "28800", 2024-04-24T09:18:04.521 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.521 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.521 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.521 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.521 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.522 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.522 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.522 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.522 INFO:teuthology.orchestra.run.smithi083.stdout: "key_file": { 2024-04-24T09:18:04.522 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "key_file", 2024-04-24T09:18:04.522 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.522 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.522 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.522 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.522 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.522 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.522 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.522 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.523 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.523 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.523 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.523 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.523 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-24T09:18:04.523 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-24T09:18:04.523 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.523 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.523 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.524 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.524 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.524 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.524 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.524 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.524 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.524 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.524 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.525 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.525 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.525 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.525 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.525 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.525 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.525 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.525 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.525 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-24T09:18:04.526 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-24T09:18:04.526 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.526 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.526 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.526 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.526 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.526 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.527 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.527 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.527 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.527 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.527 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.527 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.527 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-24T09:18:04.527 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-24T09:18:04.527 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.528 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.528 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.528 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-24T09:18:04.528 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.528 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.528 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.528 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.529 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.529 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.529 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.529 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.529 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.529 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.529 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.529 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.529 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.530 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.530 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.530 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-24T09:18:04.530 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-24T09:18:04.530 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.530 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.530 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.530 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.530 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.530 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.530 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.530 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.531 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.531 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.531 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.531 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.531 INFO:teuthology.orchestra.run.smithi083.stdout: "motd": { 2024-04-24T09:18:04.531 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "motd", 2024-04-24T09:18:04.531 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.531 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.531 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.531 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.531 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.531 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.531 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.531 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "The message of the day", 2024-04-24T09:18:04.531 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.531 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.532 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.532 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.532 INFO:teuthology.orchestra.run.smithi083.stdout: "redirect_resolve_ip_addr": { 2024-04-24T09:18:04.532 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "redirect_resolve_ip_addr", 2024-04-24T09:18:04.532 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.532 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.532 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.532 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.532 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.532 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.532 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.532 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.532 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.532 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.532 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.533 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.533 INFO:teuthology.orchestra.run.smithi083.stdout: "server_addr": { 2024-04-24T09:18:04.533 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_addr", 2024-04-24T09:18:04.533 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.533 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.533 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.533 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "::", 2024-04-24T09:18:04.533 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.533 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.533 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.533 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.533 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.533 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.533 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.533 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.534 INFO:teuthology.orchestra.run.smithi083.stdout: "server_port": { 2024-04-24T09:18:04.534 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_port", 2024-04-24T09:18:04.534 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:04.534 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.534 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.534 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "8080", 2024-04-24T09:18:04.534 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.534 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.534 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.534 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.534 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.534 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.534 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.534 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.534 INFO:teuthology.orchestra.run.smithi083.stdout: "ssl": { 2024-04-24T09:18:04.535 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ssl", 2024-04-24T09:18:04.535 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.535 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.535 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.535 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:04.535 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.535 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.535 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.535 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.535 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.535 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.535 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.535 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.535 INFO:teuthology.orchestra.run.smithi083.stdout: "ssl_server_port": { 2024-04-24T09:18:04.535 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ssl_server_port", 2024-04-24T09:18:04.536 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:04.536 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.536 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.536 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "8443", 2024-04-24T09:18:04.536 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.536 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.536 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.536 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.536 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.536 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.536 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.536 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.536 INFO:teuthology.orchestra.run.smithi083.stdout: "standby_behaviour": { 2024-04-24T09:18:04.536 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "standby_behaviour", 2024-04-24T09:18:04.536 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.536 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.537 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.537 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "redirect", 2024-04-24T09:18:04.537 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.537 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.537 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.537 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.537 INFO:teuthology.orchestra.run.smithi083.stdout: "redirect" 2024-04-24T09:18:04.537 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.537 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.537 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.537 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.537 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.537 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.537 INFO:teuthology.orchestra.run.smithi083.stdout: "standby_error_status_code": { 2024-04-24T09:18:04.537 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "standby_error_status_code", 2024-04-24T09:18:04.537 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:04.538 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.538 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.538 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "500", 2024-04-24T09:18:04.538 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "400", 2024-04-24T09:18:04.538 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "599", 2024-04-24T09:18:04.538 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.538 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.538 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.538 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.538 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.538 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.538 INFO:teuthology.orchestra.run.smithi083.stdout: "url_prefix": { 2024-04-24T09:18:04.538 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "url_prefix", 2024-04-24T09:18:04.538 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.539 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.539 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.539 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.539 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.539 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.539 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.539 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.539 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.539 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.539 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.539 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.539 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.539 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.539 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:04.539 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "devicehealth", 2024-04-24T09:18:04.539 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-24T09:18:04.540 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-24T09:18:04.540 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-24T09:18:04.540 INFO:teuthology.orchestra.run.smithi083.stdout: "enable_monitoring": { 2024-04-24T09:18:04.540 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "enable_monitoring", 2024-04-24T09:18:04.540 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.540 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.540 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.540 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:04.540 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.540 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.540 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.540 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "monitor device health metrics", 2024-04-24T09:18:04.540 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.541 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.541 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.541 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.541 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-24T09:18:04.541 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-24T09:18:04.541 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.541 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.541 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.541 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.541 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.541 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.541 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.541 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.541 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.541 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.542 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.542 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.542 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.542 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.542 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.542 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.542 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.542 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.542 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.542 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-24T09:18:04.542 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-24T09:18:04.542 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.542 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.542 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.542 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.543 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.543 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.543 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.543 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.543 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.543 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.543 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.543 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.543 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-24T09:18:04.543 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-24T09:18:04.543 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.543 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.543 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.543 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-24T09:18:04.544 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.544 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.544 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.544 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.544 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.544 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.544 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.544 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.544 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.544 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.544 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.544 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.544 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.544 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.544 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.544 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-24T09:18:04.544 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-24T09:18:04.545 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.545 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.545 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.545 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.545 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.545 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.545 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.545 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.545 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.545 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.545 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.545 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.545 INFO:teuthology.orchestra.run.smithi083.stdout: "mark_out_threshold": { 2024-04-24T09:18:04.545 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mark_out_threshold", 2024-04-24T09:18:04.545 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-24T09:18:04.546 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.546 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.546 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "2419200", 2024-04-24T09:18:04.546 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.546 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.546 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.546 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-24T09:18:04.546 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.546 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.546 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.546 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.546 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_name": { 2024-04-24T09:18:04.546 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "pool_name", 2024-04-24T09:18:04.546 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.546 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.546 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.547 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "device_health_metrics", 2024-04-24T09:18:04.547 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.547 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.547 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.547 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-24T09:18:04.547 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.547 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.547 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.547 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.547 INFO:teuthology.orchestra.run.smithi083.stdout: "retention_period": { 2024-04-24T09:18:04.547 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "retention_period", 2024-04-24T09:18:04.547 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-24T09:18:04.547 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.547 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.547 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "15552000", 2024-04-24T09:18:04.547 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.547 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.548 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.548 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how long to retain device health metrics", 2024-04-24T09:18:04.548 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.548 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.548 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.548 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.548 INFO:teuthology.orchestra.run.smithi083.stdout: "scrape_frequency": { 2024-04-24T09:18:04.548 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "scrape_frequency", 2024-04-24T09:18:04.548 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-24T09:18:04.548 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.548 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.548 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "86400", 2024-04-24T09:18:04.549 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.549 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.549 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.549 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-24T09:18:04.549 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.549 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.549 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.549 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.549 INFO:teuthology.orchestra.run.smithi083.stdout: "self_heal": { 2024-04-24T09:18:04.549 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "self_heal", 2024-04-24T09:18:04.549 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.549 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.549 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.549 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:04.549 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.550 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.550 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.550 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-24T09:18:04.550 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.550 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.550 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.550 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.550 INFO:teuthology.orchestra.run.smithi083.stdout: "sleep_interval": { 2024-04-24T09:18:04.550 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sleep_interval", 2024-04-24T09:18:04.550 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-24T09:18:04.550 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.550 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.550 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "600", 2024-04-24T09:18:04.550 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.550 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.551 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.551 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how frequently to wake up and check device health", 2024-04-24T09:18:04.551 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.551 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.551 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.551 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.551 INFO:teuthology.orchestra.run.smithi083.stdout: "warn_threshold": { 2024-04-24T09:18:04.551 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "warn_threshold", 2024-04-24T09:18:04.551 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-24T09:18:04.551 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.551 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.551 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "7257600", 2024-04-24T09:18:04.551 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.551 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.551 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.552 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-24T09:18:04.552 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.552 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.552 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.552 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.552 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.552 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.552 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:04.552 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "diskprediction_local", 2024-04-24T09:18:04.552 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-24T09:18:04.552 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-24T09:18:04.552 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-24T09:18:04.552 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-24T09:18:04.552 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-24T09:18:04.553 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.553 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.553 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.553 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.553 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.553 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.553 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.553 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.553 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.553 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.553 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.553 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.553 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.553 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.553 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.554 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.554 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.554 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.554 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.554 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-24T09:18:04.554 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-24T09:18:04.554 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.554 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.554 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.554 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.554 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.554 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.554 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.555 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.555 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.555 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.555 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.555 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.555 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-24T09:18:04.555 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-24T09:18:04.555 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.555 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.555 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.555 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-24T09:18:04.555 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.555 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.555 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.555 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.556 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.556 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.556 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.556 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.556 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.556 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.556 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.556 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.556 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.556 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.556 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.556 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-24T09:18:04.556 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-24T09:18:04.556 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.556 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.557 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.557 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.557 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.557 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.557 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.557 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.557 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.557 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.557 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.557 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.557 INFO:teuthology.orchestra.run.smithi083.stdout: "predict_interval": { 2024-04-24T09:18:04.557 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "predict_interval", 2024-04-24T09:18:04.557 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.557 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.557 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.557 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "86400", 2024-04-24T09:18:04.557 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.558 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.558 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.558 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.558 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.558 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.558 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.558 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.558 INFO:teuthology.orchestra.run.smithi083.stdout: "predictor_model": { 2024-04-24T09:18:04.558 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "predictor_model", 2024-04-24T09:18:04.558 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.558 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.558 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.558 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "prophetstor", 2024-04-24T09:18:04.558 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.559 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.559 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.559 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.559 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.559 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.559 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.559 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.559 INFO:teuthology.orchestra.run.smithi083.stdout: "sleep_interval": { 2024-04-24T09:18:04.559 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sleep_interval", 2024-04-24T09:18:04.559 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.559 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.559 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.559 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "600", 2024-04-24T09:18:04.559 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.559 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.560 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.560 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.560 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.560 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.560 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.560 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.560 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.560 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.560 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:04.560 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "influx", 2024-04-24T09:18:04.560 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": false, 2024-04-24T09:18:04.560 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "influxdb python module not found", 2024-04-24T09:18:04.560 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-24T09:18:04.560 INFO:teuthology.orchestra.run.smithi083.stdout: "batch_size": { 2024-04-24T09:18:04.561 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "batch_size", 2024-04-24T09:18:04.561 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:04.561 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.561 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.561 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "5000", 2024-04-24T09:18:04.561 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.561 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.561 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.561 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-24T09:18:04.561 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.561 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.561 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.561 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.561 INFO:teuthology.orchestra.run.smithi083.stdout: "database": { 2024-04-24T09:18:04.561 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "database", 2024-04-24T09:18:04.562 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.562 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.562 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.562 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "ceph", 2024-04-24T09:18:04.562 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.562 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.562 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.562 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-24T09:18:04.562 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.562 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.562 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.562 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.562 INFO:teuthology.orchestra.run.smithi083.stdout: "hostname": { 2024-04-24T09:18:04.562 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "hostname", 2024-04-24T09:18:04.562 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.562 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.563 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.563 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.563 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.563 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.563 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.563 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "InfluxDB server hostname", 2024-04-24T09:18:04.563 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.563 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.563 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.563 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.563 INFO:teuthology.orchestra.run.smithi083.stdout: "interval": { 2024-04-24T09:18:04.563 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "interval", 2024-04-24T09:18:04.563 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-24T09:18:04.563 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.563 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.563 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "30", 2024-04-24T09:18:04.564 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "5", 2024-04-24T09:18:04.564 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.564 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.564 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-24T09:18:04.564 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.564 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.564 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.564 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.564 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-24T09:18:04.564 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-24T09:18:04.564 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.564 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.564 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.564 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.564 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.564 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.565 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.565 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.565 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.565 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.565 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.565 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.565 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.565 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.565 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.565 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.565 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.565 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.565 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.565 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-24T09:18:04.565 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-24T09:18:04.565 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.566 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.566 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.566 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.566 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.566 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.566 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.566 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.566 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.566 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.566 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.566 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.566 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-24T09:18:04.566 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-24T09:18:04.566 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.566 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.566 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.567 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-24T09:18:04.567 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.567 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.567 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.567 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.567 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.567 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.567 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.567 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.567 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.567 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.567 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.567 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.567 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.567 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.568 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.568 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-24T09:18:04.568 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-24T09:18:04.568 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.568 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.568 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.568 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.568 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.568 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.568 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.568 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.568 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.568 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.568 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.568 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.568 INFO:teuthology.orchestra.run.smithi083.stdout: "password": { 2024-04-24T09:18:04.569 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "password", 2024-04-24T09:18:04.569 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.569 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.569 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.569 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.569 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.569 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.569 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.569 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "password of InfluxDB server user", 2024-04-24T09:18:04.569 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.569 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.569 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.569 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.569 INFO:teuthology.orchestra.run.smithi083.stdout: "port": { 2024-04-24T09:18:04.569 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "port", 2024-04-24T09:18:04.570 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:04.570 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.570 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.570 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "8086", 2024-04-24T09:18:04.570 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.570 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.570 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.570 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "InfluxDB server port", 2024-04-24T09:18:04.570 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.570 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.570 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.570 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.570 INFO:teuthology.orchestra.run.smithi083.stdout: "ssl": { 2024-04-24T09:18:04.570 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ssl", 2024-04-24T09:18:04.571 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.571 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.571 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.571 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "false", 2024-04-24T09:18:04.571 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.571 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.571 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.571 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-24T09:18:04.571 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.571 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.571 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.571 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.571 INFO:teuthology.orchestra.run.smithi083.stdout: "threads": { 2024-04-24T09:18:04.572 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "threads", 2024-04-24T09:18:04.572 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:04.572 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.572 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.572 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "5", 2024-04-24T09:18:04.572 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "1", 2024-04-24T09:18:04.572 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "32", 2024-04-24T09:18:04.572 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.572 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-24T09:18:04.572 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.572 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.572 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.572 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.572 INFO:teuthology.orchestra.run.smithi083.stdout: "username": { 2024-04-24T09:18:04.572 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "username", 2024-04-24T09:18:04.572 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.573 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.573 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.573 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.573 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.573 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.573 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.573 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "username of InfluxDB server user", 2024-04-24T09:18:04.573 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.573 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.573 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.573 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.573 INFO:teuthology.orchestra.run.smithi083.stdout: "verify_ssl": { 2024-04-24T09:18:04.573 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "verify_ssl", 2024-04-24T09:18:04.573 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.573 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.574 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.574 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "true", 2024-04-24T09:18:04.574 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.574 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.574 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.574 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-24T09:18:04.574 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.574 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.574 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.574 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.574 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.574 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.574 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:04.575 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "insights", 2024-04-24T09:18:04.575 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-24T09:18:04.575 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-24T09:18:04.575 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-24T09:18:04.575 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-24T09:18:04.575 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-24T09:18:04.575 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.575 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.575 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.575 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.575 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.575 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.575 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.575 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.575 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.575 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.575 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.576 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.576 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.576 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.576 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.576 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.576 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.576 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.576 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.576 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-24T09:18:04.576 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-24T09:18:04.576 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.576 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.576 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.576 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.576 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.576 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.577 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.577 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.577 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.577 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.577 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.577 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.577 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-24T09:18:04.577 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-24T09:18:04.577 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.577 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.577 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.577 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-24T09:18:04.577 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.577 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.577 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.577 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.577 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.578 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.578 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.578 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.578 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.578 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.578 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.578 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.578 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.578 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.578 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.578 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-24T09:18:04.578 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-24T09:18:04.578 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.578 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.578 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.579 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.579 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.579 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.579 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.579 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.579 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.579 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.579 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.579 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.579 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.579 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.579 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:04.579 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "iostat", 2024-04-24T09:18:04.579 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-24T09:18:04.579 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-24T09:18:04.580 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-24T09:18:04.580 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-24T09:18:04.580 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-24T09:18:04.580 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.580 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.580 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.580 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.580 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.580 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.580 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.580 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.580 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.580 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.580 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.581 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.581 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.581 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.581 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.581 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.581 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.581 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.581 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.581 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-24T09:18:04.581 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-24T09:18:04.581 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.581 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.581 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.581 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.581 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.581 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.582 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.582 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.582 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.582 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.582 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.582 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.582 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-24T09:18:04.582 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-24T09:18:04.582 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.582 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.582 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.582 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-24T09:18:04.582 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.582 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.582 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.582 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.583 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.583 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.583 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.583 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.583 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.583 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.583 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.583 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.583 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.583 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.583 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.583 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-24T09:18:04.583 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-24T09:18:04.583 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.583 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.583 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.584 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.584 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.584 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.584 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.584 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.584 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.584 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.584 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.584 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.584 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.584 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.584 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:04.584 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "k8sevents", 2024-04-24T09:18:04.584 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-24T09:18:04.584 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-24T09:18:04.584 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-24T09:18:04.585 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_event_retention_days": { 2024-04-24T09:18:04.585 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ceph_event_retention_days", 2024-04-24T09:18:04.585 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:04.585 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.585 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.585 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "7", 2024-04-24T09:18:04.585 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.585 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.585 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.585 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-24T09:18:04.585 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.585 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.585 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.585 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.585 INFO:teuthology.orchestra.run.smithi083.stdout: "config_check_secs": { 2024-04-24T09:18:04.585 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "config_check_secs", 2024-04-24T09:18:04.586 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:04.586 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.586 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.586 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-04-24T09:18:04.586 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "10", 2024-04-24T09:18:04.586 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.586 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.586 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-24T09:18:04.586 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.586 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.586 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.586 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.586 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-24T09:18:04.586 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-24T09:18:04.586 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.587 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.587 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.587 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.587 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.587 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.587 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.587 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.587 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.587 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.587 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.587 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.587 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.587 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.587 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.587 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.587 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.588 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.588 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.588 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-24T09:18:04.588 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-24T09:18:04.588 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.588 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.588 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.588 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.588 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.588 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.588 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.588 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.588 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.588 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.588 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.589 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.589 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-24T09:18:04.589 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-24T09:18:04.589 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.589 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.589 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.589 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-24T09:18:04.589 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.589 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.589 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.589 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.589 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.589 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.589 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.589 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.589 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.590 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.590 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.590 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.590 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.590 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.590 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.590 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-24T09:18:04.590 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-24T09:18:04.590 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.590 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.590 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.590 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.590 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.590 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.591 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.591 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.591 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.591 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.591 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.591 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.591 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.591 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.591 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:04.591 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "localpool", 2024-04-24T09:18:04.591 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-24T09:18:04.591 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-24T09:18:04.591 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-24T09:18:04.591 INFO:teuthology.orchestra.run.smithi083.stdout: "failure_domain": { 2024-04-24T09:18:04.591 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "failure_domain", 2024-04-24T09:18:04.591 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.592 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.592 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.592 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "host", 2024-04-24T09:18:04.592 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.592 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.592 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.592 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "failure domain for any created local pool", 2024-04-24T09:18:04.592 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-24T09:18:04.592 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.592 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.592 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.592 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-24T09:18:04.592 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-24T09:18:04.592 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.592 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.592 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.593 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.593 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.593 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.593 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.593 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.593 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.593 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.593 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.593 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.593 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.593 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.593 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.593 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.593 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.594 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.594 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.594 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-24T09:18:04.594 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-24T09:18:04.594 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.594 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.594 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.594 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.594 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.594 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.594 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.594 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.594 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.594 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.595 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.595 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.595 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-24T09:18:04.595 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-24T09:18:04.595 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.595 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.595 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.595 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-24T09:18:04.595 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.595 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.595 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.595 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.595 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.595 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.595 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.596 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.596 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.596 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.596 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.596 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.596 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.596 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.596 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.596 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-24T09:18:04.596 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-24T09:18:04.596 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.596 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.596 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.596 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.596 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.596 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.597 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.597 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.597 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.597 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.597 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.597 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.597 INFO:teuthology.orchestra.run.smithi083.stdout: "min_size": { 2024-04-24T09:18:04.597 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "min_size", 2024-04-24T09:18:04.597 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:04.597 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.597 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.597 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.597 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.597 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.597 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.597 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "default min_size for any created local pool", 2024-04-24T09:18:04.598 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-24T09:18:04.598 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.598 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.598 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.598 INFO:teuthology.orchestra.run.smithi083.stdout: "num_rep": { 2024-04-24T09:18:04.598 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "num_rep", 2024-04-24T09:18:04.598 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:04.598 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.598 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.598 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "3", 2024-04-24T09:18:04.598 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.598 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.598 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.598 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "default replica count for any created local pool", 2024-04-24T09:18:04.598 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.599 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.599 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.599 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.599 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num": { 2024-04-24T09:18:04.599 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "pg_num", 2024-04-24T09:18:04.599 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:04.599 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.599 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.599 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "128", 2024-04-24T09:18:04.599 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.599 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.599 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.599 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "default pg_num for any created local pool", 2024-04-24T09:18:04.599 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.599 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.600 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.600 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.600 INFO:teuthology.orchestra.run.smithi083.stdout: "prefix": { 2024-04-24T09:18:04.600 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "prefix", 2024-04-24T09:18:04.600 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.600 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.600 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.600 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.600 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.600 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.600 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.600 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "name prefix for any created local pool", 2024-04-24T09:18:04.600 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.601 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.601 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.601 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.601 INFO:teuthology.orchestra.run.smithi083.stdout: "subtree": { 2024-04-24T09:18:04.601 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "subtree", 2024-04-24T09:18:04.601 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.601 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.601 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.601 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "rack", 2024-04-24T09:18:04.601 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.601 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.601 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.601 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-24T09:18:04.601 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-24T09:18:04.601 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.602 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.602 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.602 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.602 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.602 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:04.602 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mds_autoscaler", 2024-04-24T09:18:04.602 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-24T09:18:04.602 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-24T09:18:04.602 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-24T09:18:04.602 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-24T09:18:04.602 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-24T09:18:04.602 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.602 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.602 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.602 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.603 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.603 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.603 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.603 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.603 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.603 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.603 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.603 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.603 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.603 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.603 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.603 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.603 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.603 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.603 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.603 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-24T09:18:04.604 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-24T09:18:04.604 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.604 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.604 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.604 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.604 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.604 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.604 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.604 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.604 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.604 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.604 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.604 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.604 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-24T09:18:04.604 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-24T09:18:04.604 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.605 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.605 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.605 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-24T09:18:04.605 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.605 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.605 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.605 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.605 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.605 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.605 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.605 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.605 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.605 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.605 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.605 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.605 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.606 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.606 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.606 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-24T09:18:04.606 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-24T09:18:04.606 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.606 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.606 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.606 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.606 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.606 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.606 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.606 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.606 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.606 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.606 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.607 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.607 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.607 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.607 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:04.607 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mirroring", 2024-04-24T09:18:04.607 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-24T09:18:04.607 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-24T09:18:04.607 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-24T09:18:04.607 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-24T09:18:04.607 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-24T09:18:04.607 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.607 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.607 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.607 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.608 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.608 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.608 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.608 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.608 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.608 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.608 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.608 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.608 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.608 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.608 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.608 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.608 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.608 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.608 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.609 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-24T09:18:04.609 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-24T09:18:04.609 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.609 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.609 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.609 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.609 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.609 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.609 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.609 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.609 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.609 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.609 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.610 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.610 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-24T09:18:04.610 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-24T09:18:04.610 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.610 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.610 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.610 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-24T09:18:04.610 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.610 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.610 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.610 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.610 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.610 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.611 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.611 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.611 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.611 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.611 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.611 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.611 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.611 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.611 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.611 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-24T09:18:04.611 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-24T09:18:04.611 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.611 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.611 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.612 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.612 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.612 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.612 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.612 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.612 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.612 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.612 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.612 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.612 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.612 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.612 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:04.612 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "nfs", 2024-04-24T09:18:04.613 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-24T09:18:04.613 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-24T09:18:04.613 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-24T09:18:04.613 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-24T09:18:04.613 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-24T09:18:04.613 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.613 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.613 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.613 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.613 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.613 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.613 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.613 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.613 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.614 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.614 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.614 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.614 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.614 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.614 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.614 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.614 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.614 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.614 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.614 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-24T09:18:04.614 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-24T09:18:04.614 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.614 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.615 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.615 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.615 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.615 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.615 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.615 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.615 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.615 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.615 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.615 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.615 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-24T09:18:04.615 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-24T09:18:04.615 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.615 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.615 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.616 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-24T09:18:04.616 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.616 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.616 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.616 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.616 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.616 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.616 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.616 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.616 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.616 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.616 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.616 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.616 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.616 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.617 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.617 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-24T09:18:04.617 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-24T09:18:04.617 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.617 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.617 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.617 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.617 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.617 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.617 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.617 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.617 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.617 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.617 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.617 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.618 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.618 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.618 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:04.618 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "orchestrator", 2024-04-24T09:18:04.618 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-24T09:18:04.618 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-24T09:18:04.618 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-24T09:18:04.618 INFO:teuthology.orchestra.run.smithi083.stdout: "fail_fs": { 2024-04-24T09:18:04.618 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "fail_fs", 2024-04-24T09:18:04.618 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.618 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.618 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.618 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.618 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.619 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.619 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.619 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-24T09:18:04.619 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.619 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.619 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.619 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.619 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-24T09:18:04.619 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-24T09:18:04.619 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.619 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.619 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.619 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.619 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.619 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.620 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.620 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.620 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.620 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.620 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.620 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.620 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.620 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.620 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.620 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.620 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.620 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.620 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.620 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-24T09:18:04.621 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-24T09:18:04.621 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.621 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.621 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.621 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.621 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.621 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.621 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.621 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.621 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.621 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.621 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.621 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.621 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-24T09:18:04.622 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-24T09:18:04.622 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.622 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.622 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.622 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-24T09:18:04.622 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.622 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.622 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.622 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.622 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.622 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.622 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.622 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.622 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.623 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.623 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.623 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.623 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.623 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.623 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.623 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-24T09:18:04.623 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-24T09:18:04.623 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.623 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.623 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.623 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.623 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.623 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.624 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.624 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.624 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.624 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.624 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.624 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.624 INFO:teuthology.orchestra.run.smithi083.stdout: "orchestrator": { 2024-04-24T09:18:04.624 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "orchestrator", 2024-04-24T09:18:04.624 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.624 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.624 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.624 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.624 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.624 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.625 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.625 INFO:teuthology.orchestra.run.smithi083.stdout: "cephadm", 2024-04-24T09:18:04.625 INFO:teuthology.orchestra.run.smithi083.stdout: "rook", 2024-04-24T09:18:04.625 INFO:teuthology.orchestra.run.smithi083.stdout: "test_orchestrator" 2024-04-24T09:18:04.625 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.625 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Orchestrator backend", 2024-04-24T09:18:04.625 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.625 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.625 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.625 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.625 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.625 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.625 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:04.625 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "osd_perf_query", 2024-04-24T09:18:04.625 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-24T09:18:04.626 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-24T09:18:04.626 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-24T09:18:04.626 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-24T09:18:04.626 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-24T09:18:04.626 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.626 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.626 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.626 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.626 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.626 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.626 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.626 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.626 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.627 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.627 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.627 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.627 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.627 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.627 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.627 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.627 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.627 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.627 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.627 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-24T09:18:04.627 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-24T09:18:04.627 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.627 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.628 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.628 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.628 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.628 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.628 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.628 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.628 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.628 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.628 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.628 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.628 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-24T09:18:04.628 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-24T09:18:04.628 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.628 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.629 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.629 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-24T09:18:04.629 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.629 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.629 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.629 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.629 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.629 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.629 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.629 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.629 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.629 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.629 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.629 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.630 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.630 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.630 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.630 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-24T09:18:04.630 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-24T09:18:04.630 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.630 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.630 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.630 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.630 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.630 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.630 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.630 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.631 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.631 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.631 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.631 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.631 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.631 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.631 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:04.631 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "osd_support", 2024-04-24T09:18:04.631 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-24T09:18:04.631 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-24T09:18:04.631 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-24T09:18:04.631 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-24T09:18:04.631 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-24T09:18:04.631 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.631 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.632 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.632 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.632 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.632 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.632 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.632 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.632 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.632 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.632 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.632 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.632 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.632 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.632 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.632 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.632 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.633 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.633 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.633 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-24T09:18:04.633 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-24T09:18:04.633 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.633 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.633 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.633 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.633 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.633 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.633 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.633 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.633 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.633 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.634 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.634 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.634 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-24T09:18:04.634 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-24T09:18:04.634 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.634 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.634 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.634 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-24T09:18:04.634 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.634 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.634 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.634 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.634 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.634 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.635 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.635 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.635 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.635 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.635 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.635 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.635 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.635 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.635 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.635 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-24T09:18:04.635 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-24T09:18:04.635 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.635 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.635 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.635 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.636 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.636 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.636 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.636 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.636 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.636 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.636 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.636 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.636 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.636 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.636 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:04.636 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "pg_autoscaler", 2024-04-24T09:18:04.636 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-24T09:18:04.636 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-24T09:18:04.636 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-24T09:18:04.636 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-24T09:18:04.637 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-24T09:18:04.637 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.637 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.637 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.637 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.637 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.637 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.637 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.637 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.637 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.637 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.637 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.637 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.637 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.637 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.638 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.638 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.638 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.638 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.638 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.638 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-24T09:18:04.638 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-24T09:18:04.638 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.638 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.638 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.638 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.638 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.638 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.638 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.639 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.639 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.639 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.639 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.639 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.639 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-24T09:18:04.639 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-24T09:18:04.639 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.639 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.639 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.639 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-24T09:18:04.639 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.639 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.639 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.640 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.640 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.640 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.640 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.640 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.640 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.640 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.640 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.640 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.640 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.640 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.640 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.640 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-24T09:18:04.640 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-24T09:18:04.641 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.641 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.641 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.641 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.641 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.641 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.641 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.641 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.641 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.641 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.641 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.641 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.641 INFO:teuthology.orchestra.run.smithi083.stdout: "sleep_interval": { 2024-04-24T09:18:04.641 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sleep_interval", 2024-04-24T09:18:04.641 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-24T09:18:04.641 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.641 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.642 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "60", 2024-04-24T09:18:04.642 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.642 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.642 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.642 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.642 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.642 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.642 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.642 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.642 INFO:teuthology.orchestra.run.smithi083.stdout: "threshold": { 2024-04-24T09:18:04.642 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "threshold", 2024-04-24T09:18:04.642 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-04-24T09:18:04.642 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.642 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.643 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "3.0", 2024-04-24T09:18:04.643 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "1.0", 2024-04-24T09:18:04.643 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.643 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.643 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "scaling threshold", 2024-04-24T09:18:04.643 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-24T09:18:04.643 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.643 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.643 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.643 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.643 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.643 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:04.643 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "progress", 2024-04-24T09:18:04.643 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-24T09:18:04.644 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-24T09:18:04.644 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-24T09:18:04.644 INFO:teuthology.orchestra.run.smithi083.stdout: "allow_pg_recovery_event": { 2024-04-24T09:18:04.644 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "allow_pg_recovery_event", 2024-04-24T09:18:04.644 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.644 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.644 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.644 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.644 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.644 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.644 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.644 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-24T09:18:04.644 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.644 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.644 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.644 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.645 INFO:teuthology.orchestra.run.smithi083.stdout: "enabled": { 2024-04-24T09:18:04.645 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "enabled", 2024-04-24T09:18:04.645 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.645 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.645 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.645 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:04.645 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.645 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.645 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.645 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.645 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.645 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.645 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.645 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.645 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-24T09:18:04.646 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-24T09:18:04.646 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.646 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.646 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.646 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.646 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.646 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.646 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.646 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.646 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.646 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.646 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.646 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.646 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.647 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.647 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.647 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.647 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.647 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.647 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.647 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-24T09:18:04.647 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-24T09:18:04.647 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.647 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.647 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.647 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.647 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.647 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.647 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.648 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.648 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.648 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.648 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.648 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.648 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-24T09:18:04.648 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-24T09:18:04.648 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.648 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.648 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.648 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-24T09:18:04.648 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.648 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.648 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.648 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.648 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.649 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.649 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.649 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.649 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.649 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.649 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.649 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.649 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.649 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.649 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.649 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-24T09:18:04.649 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-24T09:18:04.649 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.649 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.649 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.649 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.650 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.650 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.650 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.650 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.650 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.650 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.650 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.650 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.650 INFO:teuthology.orchestra.run.smithi083.stdout: "max_completed_events": { 2024-04-24T09:18:04.650 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "max_completed_events", 2024-04-24T09:18:04.650 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:04.650 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.650 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.651 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "50", 2024-04-24T09:18:04.651 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.651 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.651 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.651 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "number of past completed events to remember", 2024-04-24T09:18:04.651 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.651 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.651 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.651 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.651 INFO:teuthology.orchestra.run.smithi083.stdout: "sleep_interval": { 2024-04-24T09:18:04.651 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sleep_interval", 2024-04-24T09:18:04.651 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-24T09:18:04.651 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.651 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.651 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "5", 2024-04-24T09:18:04.651 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.652 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.652 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.652 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how long the module is going to sleep", 2024-04-24T09:18:04.652 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.652 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.652 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.652 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.652 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.652 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.652 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:04.652 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "prometheus", 2024-04-24T09:18:04.652 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-24T09:18:04.652 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-24T09:18:04.652 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-24T09:18:04.652 INFO:teuthology.orchestra.run.smithi083.stdout: "cache": { 2024-04-24T09:18:04.652 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "cache", 2024-04-24T09:18:04.652 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.653 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.653 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.653 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:04.653 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.653 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.653 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.653 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.653 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.653 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.653 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.653 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.653 INFO:teuthology.orchestra.run.smithi083.stdout: "exclude_perf_counters": { 2024-04-24T09:18:04.653 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "exclude_perf_counters", 2024-04-24T09:18:04.653 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.653 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.654 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.654 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:04.654 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.654 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.654 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.654 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-24T09:18:04.654 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-24T09:18:04.654 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.654 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.654 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.654 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-24T09:18:04.654 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-24T09:18:04.654 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.654 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.655 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.655 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.655 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.655 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.655 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.655 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.655 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.655 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.655 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.655 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.655 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.655 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.655 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.655 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.655 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.655 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.656 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.656 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-24T09:18:04.656 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-24T09:18:04.656 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.656 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.656 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.656 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.656 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.656 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.656 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.656 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.656 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.656 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.656 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.656 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.656 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-24T09:18:04.656 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-24T09:18:04.657 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.657 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.657 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.657 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-24T09:18:04.657 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.657 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.657 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.657 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.657 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.657 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.657 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.657 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.657 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.657 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.657 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.658 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.658 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.658 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.658 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.658 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-24T09:18:04.658 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-24T09:18:04.658 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.658 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.658 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.658 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.658 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.658 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.658 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.658 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.659 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.659 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.659 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.659 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.659 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_stats_pools": { 2024-04-24T09:18:04.659 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rbd_stats_pools", 2024-04-24T09:18:04.659 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.659 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.659 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.659 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.659 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.659 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.659 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.659 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.660 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.660 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.660 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.660 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.660 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-24T09:18:04.660 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-24T09:18:04.660 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:04.660 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.660 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.660 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "300", 2024-04-24T09:18:04.660 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.660 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.660 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.660 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.661 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.661 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.661 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.661 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.661 INFO:teuthology.orchestra.run.smithi083.stdout: "scrape_interval": { 2024-04-24T09:18:04.661 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "scrape_interval", 2024-04-24T09:18:04.661 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-04-24T09:18:04.661 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.661 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.661 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "15.0", 2024-04-24T09:18:04.661 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.661 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.661 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.661 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.661 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.661 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.662 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.662 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.662 INFO:teuthology.orchestra.run.smithi083.stdout: "server_addr": { 2024-04-24T09:18:04.662 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_addr", 2024-04-24T09:18:04.662 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.662 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.662 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.662 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "::", 2024-04-24T09:18:04.662 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.662 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.662 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.662 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-24T09:18:04.662 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.662 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.663 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.663 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.663 INFO:teuthology.orchestra.run.smithi083.stdout: "server_port": { 2024-04-24T09:18:04.663 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_port", 2024-04-24T09:18:04.663 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:04.663 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.663 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.663 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "9283", 2024-04-24T09:18:04.663 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.663 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.663 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.663 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-24T09:18:04.663 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.663 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.663 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.663 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.664 INFO:teuthology.orchestra.run.smithi083.stdout: "stale_cache_strategy": { 2024-04-24T09:18:04.664 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "stale_cache_strategy", 2024-04-24T09:18:04.664 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.664 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.664 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.664 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "log", 2024-04-24T09:18:04.664 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.664 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.664 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.664 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.664 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.664 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.664 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.664 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.664 INFO:teuthology.orchestra.run.smithi083.stdout: "standby_behaviour": { 2024-04-24T09:18:04.664 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "standby_behaviour", 2024-04-24T09:18:04.665 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.665 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.665 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.665 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "default", 2024-04-24T09:18:04.665 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.665 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.665 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.665 INFO:teuthology.orchestra.run.smithi083.stdout: "default", 2024-04-24T09:18:04.665 INFO:teuthology.orchestra.run.smithi083.stdout: "error" 2024-04-24T09:18:04.665 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.665 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.665 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.665 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.665 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.665 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.665 INFO:teuthology.orchestra.run.smithi083.stdout: "standby_error_status_code": { 2024-04-24T09:18:04.665 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "standby_error_status_code", 2024-04-24T09:18:04.666 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:04.666 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.666 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.666 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "500", 2024-04-24T09:18:04.666 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "400", 2024-04-24T09:18:04.666 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "599", 2024-04-24T09:18:04.666 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.666 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.666 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.666 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.666 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.666 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.666 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.666 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.667 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:04.667 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rbd_support", 2024-04-24T09:18:04.667 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-24T09:18:04.667 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-24T09:18:04.667 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-24T09:18:04.667 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-24T09:18:04.667 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-24T09:18:04.667 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.667 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.667 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.667 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.667 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.667 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.667 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.667 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.667 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.668 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.668 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.668 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.668 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.668 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.668 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.668 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.668 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.668 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.668 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.668 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-24T09:18:04.668 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-24T09:18:04.668 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.668 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.668 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.668 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.669 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.669 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.669 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.669 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.669 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.669 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.669 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.669 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.669 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-24T09:18:04.669 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-24T09:18:04.669 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.669 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.669 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.669 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-24T09:18:04.669 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.669 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.670 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.670 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.670 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.670 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.670 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.670 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.670 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.670 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.670 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.670 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.670 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.670 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.670 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.671 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-24T09:18:04.671 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-24T09:18:04.671 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.671 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.671 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.671 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.671 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.671 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.671 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.671 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.671 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.671 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.671 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.671 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.671 INFO:teuthology.orchestra.run.smithi083.stdout: "max_concurrent_snap_create": { 2024-04-24T09:18:04.672 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "max_concurrent_snap_create", 2024-04-24T09:18:04.672 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:04.672 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.672 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.672 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-04-24T09:18:04.672 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.672 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.672 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.672 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.672 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.672 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.672 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.672 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.672 INFO:teuthology.orchestra.run.smithi083.stdout: "mirror_snapshot_schedule": { 2024-04-24T09:18:04.672 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mirror_snapshot_schedule", 2024-04-24T09:18:04.673 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.673 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.673 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.673 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.673 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.673 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.673 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.673 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.673 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.673 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.673 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.673 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.673 INFO:teuthology.orchestra.run.smithi083.stdout: "trash_purge_schedule": { 2024-04-24T09:18:04.673 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "trash_purge_schedule", 2024-04-24T09:18:04.673 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.674 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.674 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.674 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.674 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.674 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.674 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.674 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.674 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.674 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.674 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.674 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.674 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.674 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.675 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:04.675 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "restful", 2024-04-24T09:18:04.675 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-24T09:18:04.675 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-24T09:18:04.675 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-24T09:18:04.675 INFO:teuthology.orchestra.run.smithi083.stdout: "enable_auth": { 2024-04-24T09:18:04.675 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "enable_auth", 2024-04-24T09:18:04.675 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.675 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.675 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.675 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:04.675 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.675 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.675 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.675 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.675 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.676 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.676 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.676 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.676 INFO:teuthology.orchestra.run.smithi083.stdout: "key_file": { 2024-04-24T09:18:04.676 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "key_file", 2024-04-24T09:18:04.676 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.676 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.676 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.676 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.676 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.676 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.676 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.676 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.676 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.676 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.676 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.677 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.677 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-24T09:18:04.677 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-24T09:18:04.677 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.677 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.677 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.677 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.677 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.677 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.677 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.677 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.677 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.677 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.677 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.677 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.678 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.678 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.678 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.678 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.678 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.678 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.678 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.678 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-24T09:18:04.678 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-24T09:18:04.678 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.678 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.678 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.678 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.678 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.678 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.679 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.679 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.679 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.679 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.679 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.679 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.679 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-24T09:18:04.679 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-24T09:18:04.679 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.679 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.679 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.679 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-24T09:18:04.679 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.679 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.679 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.680 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.680 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.680 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.680 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.680 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.680 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.680 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.680 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.680 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.680 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.680 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.680 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.680 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-24T09:18:04.680 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-24T09:18:04.680 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.681 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.681 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.681 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.681 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.681 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.681 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.681 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.681 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.681 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.681 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.681 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.681 INFO:teuthology.orchestra.run.smithi083.stdout: "server_addr": { 2024-04-24T09:18:04.681 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_addr", 2024-04-24T09:18:04.681 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.681 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.682 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.682 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.682 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.682 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.682 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.682 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.682 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.682 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.682 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.682 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.682 INFO:teuthology.orchestra.run.smithi083.stdout: "server_port": { 2024-04-24T09:18:04.682 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_port", 2024-04-24T09:18:04.682 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.683 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.683 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.683 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.683 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.683 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.683 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.683 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.683 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.683 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.683 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.683 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.683 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.683 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.683 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:04.683 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rgw", 2024-04-24T09:18:04.684 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-24T09:18:04.684 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-24T09:18:04.684 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-24T09:18:04.684 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-24T09:18:04.684 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-24T09:18:04.684 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.684 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.684 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.684 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.684 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.684 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.684 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.684 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.684 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.684 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.684 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.685 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.685 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.685 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.685 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.685 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.685 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.685 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.685 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.685 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-24T09:18:04.685 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-24T09:18:04.685 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.685 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.685 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.685 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.685 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.685 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.685 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.686 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.686 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.686 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.686 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.686 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.686 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-24T09:18:04.686 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-24T09:18:04.686 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.686 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.686 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.686 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-24T09:18:04.686 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.686 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.686 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.687 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.687 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.687 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.687 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.687 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.687 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.687 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.687 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.687 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.687 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.687 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.687 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.687 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-24T09:18:04.687 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-24T09:18:04.687 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.687 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.688 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.688 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.688 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.688 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.688 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.688 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.688 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.688 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.688 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.688 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.688 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.688 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.688 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:04.688 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rook", 2024-04-24T09:18:04.688 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-24T09:18:04.688 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-24T09:18:04.689 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-24T09:18:04.689 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-24T09:18:04.689 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-24T09:18:04.689 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.689 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.689 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.689 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.689 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.689 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.689 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.689 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.689 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.689 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.689 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.689 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.689 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.690 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.690 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.690 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.690 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.690 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.690 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.690 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-24T09:18:04.690 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-24T09:18:04.690 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.690 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.690 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.690 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.690 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.690 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.691 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.691 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.691 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.691 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.691 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.691 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.691 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-24T09:18:04.691 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-24T09:18:04.691 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.691 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.691 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.691 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-24T09:18:04.691 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.691 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.691 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.691 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.692 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.692 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.692 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.692 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.692 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.692 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.692 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.692 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.692 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.692 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.692 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.692 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-24T09:18:04.692 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-24T09:18:04.692 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.692 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.693 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.693 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.693 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.693 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.693 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.693 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.693 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.693 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.693 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.693 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.693 INFO:teuthology.orchestra.run.smithi083.stdout: "storage_class": { 2024-04-24T09:18:04.693 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "storage_class", 2024-04-24T09:18:04.693 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.693 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.693 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.694 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "local", 2024-04-24T09:18:04.694 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.694 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.694 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.694 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-24T09:18:04.694 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.694 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.694 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.694 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.694 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.694 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.694 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:04.694 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "selftest", 2024-04-24T09:18:04.694 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-24T09:18:04.695 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-24T09:18:04.695 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-24T09:18:04.695 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-24T09:18:04.695 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-24T09:18:04.695 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.695 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.695 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.695 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.695 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.695 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.695 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.695 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.695 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.695 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.696 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.696 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.696 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.696 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.696 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.696 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.696 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.696 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.696 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.696 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-24T09:18:04.696 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-24T09:18:04.696 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.696 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.696 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.697 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.697 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.697 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.697 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.697 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.697 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.697 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.697 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.697 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.697 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-24T09:18:04.697 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-24T09:18:04.697 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.697 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.697 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.697 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-24T09:18:04.698 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.698 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.698 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.698 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.698 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.698 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.698 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.698 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.698 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.698 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.698 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.698 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.698 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.698 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.698 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.698 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-24T09:18:04.699 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-24T09:18:04.699 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.699 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.699 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.699 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.699 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.699 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.699 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.699 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.699 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.699 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.699 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.699 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.699 INFO:teuthology.orchestra.run.smithi083.stdout: "roption1": { 2024-04-24T09:18:04.699 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "roption1", 2024-04-24T09:18:04.700 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.700 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.700 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.700 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.700 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.700 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.700 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.700 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.700 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.700 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.700 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.700 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.700 INFO:teuthology.orchestra.run.smithi083.stdout: "roption2": { 2024-04-24T09:18:04.700 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "roption2", 2024-04-24T09:18:04.701 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.701 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.701 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.701 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "xyz", 2024-04-24T09:18:04.701 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.701 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.701 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.701 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.701 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.701 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.701 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.701 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.701 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption1": { 2024-04-24T09:18:04.701 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption1", 2024-04-24T09:18:04.702 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.702 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.702 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.702 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.702 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.702 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.702 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.702 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.702 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.702 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.702 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.702 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.702 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption2": { 2024-04-24T09:18:04.702 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption2", 2024-04-24T09:18:04.702 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:04.703 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.703 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.703 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.703 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.703 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.703 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.703 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.703 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.703 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.703 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.703 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.703 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption3": { 2024-04-24T09:18:04.703 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption3", 2024-04-24T09:18:04.703 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-04-24T09:18:04.703 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.704 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.704 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.704 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.704 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.704 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.704 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.704 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.704 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.704 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.704 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.704 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption4": { 2024-04-24T09:18:04.704 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption4", 2024-04-24T09:18:04.704 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.704 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.704 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.705 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.705 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.705 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.705 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.705 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.705 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.705 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.705 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.705 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.705 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption5": { 2024-04-24T09:18:04.705 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption5", 2024-04-24T09:18:04.705 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.705 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.705 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.705 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.705 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.706 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.706 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.706 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.706 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.706 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.706 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.706 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.706 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption6": { 2024-04-24T09:18:04.706 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption6", 2024-04-24T09:18:04.706 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.706 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.706 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.706 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:04.706 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.706 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.706 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.707 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.707 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.707 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.707 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.707 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.707 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption7": { 2024-04-24T09:18:04.707 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption7", 2024-04-24T09:18:04.707 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:04.707 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.707 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.707 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.707 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "1", 2024-04-24T09:18:04.707 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "42", 2024-04-24T09:18:04.707 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.707 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.707 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.708 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.708 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.708 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.708 INFO:teuthology.orchestra.run.smithi083.stdout: "testkey": { 2024-04-24T09:18:04.708 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "testkey", 2024-04-24T09:18:04.708 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.708 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.708 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.708 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.708 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.708 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.708 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.708 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.708 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.708 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.708 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.709 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.709 INFO:teuthology.orchestra.run.smithi083.stdout: "testlkey": { 2024-04-24T09:18:04.709 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "testlkey", 2024-04-24T09:18:04.709 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.709 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.709 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.709 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.709 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.709 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.709 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.709 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.709 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.709 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.709 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.709 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.709 INFO:teuthology.orchestra.run.smithi083.stdout: "testnewline": { 2024-04-24T09:18:04.710 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "testnewline", 2024-04-24T09:18:04.710 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.710 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.710 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.710 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.710 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.710 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.710 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.710 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.710 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.710 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.710 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.710 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.711 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.711 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.711 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:04.711 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "snap_schedule", 2024-04-24T09:18:04.711 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-24T09:18:04.711 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-24T09:18:04.711 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-24T09:18:04.711 INFO:teuthology.orchestra.run.smithi083.stdout: "allow_m_granularity": { 2024-04-24T09:18:04.711 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "allow_m_granularity", 2024-04-24T09:18:04.711 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.711 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.711 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.711 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.711 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.711 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.712 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.712 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "allow minute scheduled snapshots", 2024-04-24T09:18:04.712 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.712 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.712 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.712 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.712 INFO:teuthology.orchestra.run.smithi083.stdout: "dump_on_update": { 2024-04-24T09:18:04.712 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "dump_on_update", 2024-04-24T09:18:04.712 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.712 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.712 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.712 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.712 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.712 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.712 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.712 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "dump database to debug log on update", 2024-04-24T09:18:04.713 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.713 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.713 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.713 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.713 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-24T09:18:04.713 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-24T09:18:04.713 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.713 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.713 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.713 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.713 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.713 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.713 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.713 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.713 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.714 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.714 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.714 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.714 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.714 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.714 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.714 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.714 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.714 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.714 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.714 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-24T09:18:04.714 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-24T09:18:04.714 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.714 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.715 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.715 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.715 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.715 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.715 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.715 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.715 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.715 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.715 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.715 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.715 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-24T09:18:04.715 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-24T09:18:04.715 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.715 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.715 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.715 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-24T09:18:04.716 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.716 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.716 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.716 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.716 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.716 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.716 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.716 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.716 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.716 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.716 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.716 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.716 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.716 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.717 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.717 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-24T09:18:04.717 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-24T09:18:04.717 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.717 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.717 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.717 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.717 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.717 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.717 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.717 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.717 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.717 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.717 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.717 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.717 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.718 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.718 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:04.718 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "stats", 2024-04-24T09:18:04.718 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-24T09:18:04.718 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-24T09:18:04.718 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-24T09:18:04.718 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-24T09:18:04.718 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-24T09:18:04.718 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.718 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.718 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.718 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.718 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.718 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.719 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.719 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.719 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.719 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.719 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.719 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.719 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.719 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.719 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.719 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.719 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.719 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.719 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.719 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-24T09:18:04.719 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-24T09:18:04.719 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.720 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.720 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.720 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.720 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.720 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.720 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.720 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.720 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.720 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.720 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.720 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.720 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-24T09:18:04.720 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-24T09:18:04.720 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.721 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.721 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.721 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-24T09:18:04.721 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.721 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.721 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.721 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.721 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.721 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.721 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.721 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.721 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.721 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.721 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.721 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.722 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.722 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.722 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.722 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-24T09:18:04.722 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-24T09:18:04.722 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.722 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.722 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.722 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.722 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.722 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.722 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.722 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.722 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.723 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.723 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.723 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.723 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.723 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.723 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:04.723 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "status", 2024-04-24T09:18:04.723 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-24T09:18:04.723 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-24T09:18:04.723 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-24T09:18:04.723 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-24T09:18:04.723 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-24T09:18:04.723 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.723 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.723 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.723 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.724 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.724 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.724 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.724 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.724 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.724 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.724 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.724 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.724 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.724 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.724 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.724 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.724 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.724 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.724 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.724 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-24T09:18:04.725 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-24T09:18:04.725 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.725 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.725 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.725 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.725 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.725 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.725 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.725 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.725 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.725 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.725 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.725 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.725 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-24T09:18:04.726 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-24T09:18:04.726 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.726 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.726 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.726 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-24T09:18:04.726 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.726 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.726 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.726 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.726 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.726 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.726 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.726 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.726 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.726 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.726 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.727 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.727 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.727 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.727 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.727 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-24T09:18:04.727 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-24T09:18:04.727 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.727 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.727 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.727 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.727 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.727 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.727 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.727 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.727 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.728 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.728 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.728 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.728 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.728 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.728 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:04.728 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "telegraf", 2024-04-24T09:18:04.728 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-24T09:18:04.728 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-24T09:18:04.728 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-24T09:18:04.728 INFO:teuthology.orchestra.run.smithi083.stdout: "address": { 2024-04-24T09:18:04.728 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "address", 2024-04-24T09:18:04.728 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.728 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.728 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.728 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-24T09:18:04.729 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.729 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.729 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.729 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.729 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.729 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.729 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.729 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.729 INFO:teuthology.orchestra.run.smithi083.stdout: "interval": { 2024-04-24T09:18:04.729 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "interval", 2024-04-24T09:18:04.729 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-24T09:18:04.729 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.729 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.729 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "15", 2024-04-24T09:18:04.729 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.729 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.730 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.730 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.730 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.730 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.730 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.730 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.730 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-24T09:18:04.730 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-24T09:18:04.730 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.730 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.730 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.730 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.730 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.730 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.731 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.731 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.731 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.731 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.731 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.731 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.731 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.731 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.731 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.731 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.731 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.731 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.731 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.731 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-24T09:18:04.731 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-24T09:18:04.732 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.732 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.732 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.732 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.732 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.732 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.732 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.732 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.732 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.732 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.732 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.732 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.732 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-24T09:18:04.732 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-24T09:18:04.732 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.732 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.733 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.733 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-24T09:18:04.733 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.733 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.733 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.733 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.733 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.733 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.733 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.733 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.733 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.733 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.733 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.733 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.733 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.733 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.734 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.734 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-24T09:18:04.734 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-24T09:18:04.734 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.734 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.734 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.734 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.734 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.734 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.734 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.734 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.734 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.734 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.734 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.734 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.735 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.735 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.735 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:04.735 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "telemetry", 2024-04-24T09:18:04.735 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-24T09:18:04.735 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-24T09:18:04.735 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-24T09:18:04.735 INFO:teuthology.orchestra.run.smithi083.stdout: "channel_basic": { 2024-04-24T09:18:04.735 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "channel_basic", 2024-04-24T09:18:04.735 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.735 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.735 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.735 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:04.735 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.735 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.735 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.736 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-24T09:18:04.736 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.736 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.736 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.736 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.736 INFO:teuthology.orchestra.run.smithi083.stdout: "channel_crash": { 2024-04-24T09:18:04.736 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "channel_crash", 2024-04-24T09:18:04.736 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.736 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.736 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.736 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:04.736 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.736 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.736 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.736 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-24T09:18:04.736 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.737 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.737 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.737 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.737 INFO:teuthology.orchestra.run.smithi083.stdout: "channel_device": { 2024-04-24T09:18:04.737 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "channel_device", 2024-04-24T09:18:04.737 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.737 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.737 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.737 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:04.737 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.737 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.737 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.737 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-24T09:18:04.737 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.737 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.738 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.738 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.738 INFO:teuthology.orchestra.run.smithi083.stdout: "channel_ident": { 2024-04-24T09:18:04.738 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "channel_ident", 2024-04-24T09:18:04.738 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.738 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.738 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.738 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.738 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.738 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.738 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.738 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-24T09:18:04.738 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.738 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.738 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.739 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.739 INFO:teuthology.orchestra.run.smithi083.stdout: "channel_perf": { 2024-04-24T09:18:04.739 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "channel_perf", 2024-04-24T09:18:04.739 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.739 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.739 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.739 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.739 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.739 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.739 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.739 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-24T09:18:04.739 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.739 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.739 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.739 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.740 INFO:teuthology.orchestra.run.smithi083.stdout: "contact": { 2024-04-24T09:18:04.740 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "contact", 2024-04-24T09:18:04.740 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.740 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.740 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.740 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.740 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.740 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.740 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.740 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.740 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.740 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.740 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.740 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.740 INFO:teuthology.orchestra.run.smithi083.stdout: "description": { 2024-04-24T09:18:04.741 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "description", 2024-04-24T09:18:04.741 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.741 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.741 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.741 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.741 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.741 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.741 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.741 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.741 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.741 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.741 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.741 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.741 INFO:teuthology.orchestra.run.smithi083.stdout: "device_url": { 2024-04-24T09:18:04.741 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "device_url", 2024-04-24T09:18:04.741 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.742 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.742 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.742 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-24T09:18:04.742 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.742 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.742 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.742 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.742 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.742 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.742 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.742 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.742 INFO:teuthology.orchestra.run.smithi083.stdout: "enabled": { 2024-04-24T09:18:04.742 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "enabled", 2024-04-24T09:18:04.742 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.743 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.743 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.743 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.743 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.743 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.743 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.743 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.743 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.743 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.743 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.743 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.743 INFO:teuthology.orchestra.run.smithi083.stdout: "interval": { 2024-04-24T09:18:04.743 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "interval", 2024-04-24T09:18:04.743 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:04.744 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.744 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.744 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "24", 2024-04-24T09:18:04.744 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "8", 2024-04-24T09:18:04.744 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.744 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.744 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.744 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.744 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.744 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.744 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.744 INFO:teuthology.orchestra.run.smithi083.stdout: "last_opt_revision": { 2024-04-24T09:18:04.744 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "last_opt_revision", 2024-04-24T09:18:04.744 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:04.744 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.745 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.745 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "1", 2024-04-24T09:18:04.745 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.745 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.745 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.745 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.745 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.745 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.745 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.745 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.745 INFO:teuthology.orchestra.run.smithi083.stdout: "leaderboard": { 2024-04-24T09:18:04.745 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "leaderboard", 2024-04-24T09:18:04.745 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.745 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.745 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.745 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.746 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.746 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.746 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.746 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.746 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.746 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.746 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.746 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.746 INFO:teuthology.orchestra.run.smithi083.stdout: "leaderboard_description": { 2024-04-24T09:18:04.746 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "leaderboard_description", 2024-04-24T09:18:04.746 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.746 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.746 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.746 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.746 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.747 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.747 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.747 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.747 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.747 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.747 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.747 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.747 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-24T09:18:04.747 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-24T09:18:04.747 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.747 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.747 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.747 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.747 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.747 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.747 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.748 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.748 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.748 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.748 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.748 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.748 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.748 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.748 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.748 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.748 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.748 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.748 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.748 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-24T09:18:04.748 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-24T09:18:04.748 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.748 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.748 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.749 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.749 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.749 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.749 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.749 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.749 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.749 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.749 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.749 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.749 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-24T09:18:04.749 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-24T09:18:04.749 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.749 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.749 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.749 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-24T09:18:04.750 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.750 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.750 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.750 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.750 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.750 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.750 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.750 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.750 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.750 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.750 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.750 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.750 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.751 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.751 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.751 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-24T09:18:04.751 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-24T09:18:04.751 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.751 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.751 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.751 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.751 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.751 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.751 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.751 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.751 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.751 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.751 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.751 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.752 INFO:teuthology.orchestra.run.smithi083.stdout: "organization": { 2024-04-24T09:18:04.752 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "organization", 2024-04-24T09:18:04.752 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.752 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.752 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.752 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.752 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.752 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.752 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.752 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.752 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.752 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.752 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.752 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.753 INFO:teuthology.orchestra.run.smithi083.stdout: "proxy": { 2024-04-24T09:18:04.753 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "proxy", 2024-04-24T09:18:04.753 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.753 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.753 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.753 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.753 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.753 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.753 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.753 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.753 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.753 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.753 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.753 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.754 INFO:teuthology.orchestra.run.smithi083.stdout: "url": { 2024-04-24T09:18:04.754 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "url", 2024-04-24T09:18:04.754 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.754 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.754 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.754 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-24T09:18:04.754 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.754 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.754 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.754 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.754 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.754 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.754 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.754 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.755 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.755 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.755 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:04.755 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "test_orchestrator", 2024-04-24T09:18:04.755 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-24T09:18:04.755 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-24T09:18:04.755 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-24T09:18:04.755 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-24T09:18:04.755 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-24T09:18:04.755 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.755 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.755 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.755 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.755 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.755 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.755 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.756 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.756 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.756 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.756 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.756 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.756 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.756 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.756 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.756 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.756 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.756 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.756 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.756 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-24T09:18:04.756 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-24T09:18:04.756 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.756 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.757 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.757 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.757 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.757 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.757 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.757 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.757 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.757 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.757 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.757 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.757 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-24T09:18:04.757 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-24T09:18:04.757 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.757 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.757 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.757 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-24T09:18:04.758 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.758 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.758 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.758 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.758 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.758 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.758 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.758 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.758 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.758 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.758 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.758 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.758 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.758 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.758 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.759 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-24T09:18:04.759 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-24T09:18:04.759 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.759 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.759 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.759 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.759 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.759 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.759 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.759 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.759 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.759 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.759 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.759 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.759 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.759 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.760 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:04.760 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "volumes", 2024-04-24T09:18:04.760 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-24T09:18:04.760 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-24T09:18:04.760 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-24T09:18:04.760 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-24T09:18:04.760 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-24T09:18:04.760 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.760 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.760 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.760 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.760 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.760 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.760 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.760 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.760 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.761 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.761 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.761 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.761 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.761 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.761 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.761 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.761 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.761 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.761 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.761 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-24T09:18:04.761 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-24T09:18:04.761 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.761 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.761 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.762 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.762 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.762 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.762 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.762 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.762 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.762 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.762 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.762 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.762 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-24T09:18:04.762 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-24T09:18:04.762 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.762 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.762 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.762 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-24T09:18:04.762 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.763 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.763 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.763 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.763 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.763 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.763 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.763 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.763 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.763 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.763 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.763 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.763 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.763 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.763 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.763 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-24T09:18:04.763 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-24T09:18:04.764 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.764 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.764 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.764 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.764 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.764 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.764 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.764 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.764 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.764 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.764 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.764 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.764 INFO:teuthology.orchestra.run.smithi083.stdout: "max_concurrent_clones": { 2024-04-24T09:18:04.764 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "max_concurrent_clones", 2024-04-24T09:18:04.764 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:04.764 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.765 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.765 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "4", 2024-04-24T09:18:04.765 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.765 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.765 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.765 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-24T09:18:04.765 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.765 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.765 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.765 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.765 INFO:teuthology.orchestra.run.smithi083.stdout: "periodic_async_work": { 2024-04-24T09:18:04.765 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "periodic_async_work", 2024-04-24T09:18:04.765 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.765 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.765 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.765 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.766 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.766 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.766 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.766 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Periodically check for async work", 2024-04-24T09:18:04.766 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.766 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.766 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.766 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.766 INFO:teuthology.orchestra.run.smithi083.stdout: "snapshot_clone_delay": { 2024-04-24T09:18:04.766 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "snapshot_clone_delay", 2024-04-24T09:18:04.766 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:04.766 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.766 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.766 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-04-24T09:18:04.766 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.766 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.766 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.767 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-24T09:18:04.767 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.767 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.767 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.767 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.767 INFO:teuthology.orchestra.run.smithi083.stdout: "snapshot_clone_no_wait": { 2024-04-24T09:18:04.767 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "snapshot_clone_no_wait", 2024-04-24T09:18:04.767 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.767 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.767 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.767 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-24T09:18:04.767 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.767 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.767 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.767 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-24T09:18:04.768 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.768 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.768 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.768 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.768 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.768 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.768 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:04.768 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "zabbix", 2024-04-24T09:18:04.768 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-24T09:18:04.768 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-24T09:18:04.768 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-24T09:18:04.768 INFO:teuthology.orchestra.run.smithi083.stdout: "discovery_interval": { 2024-04-24T09:18:04.768 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "discovery_interval", 2024-04-24T09:18:04.768 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "uint", 2024-04-24T09:18:04.768 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.768 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.769 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "100", 2024-04-24T09:18:04.769 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.769 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.769 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.769 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.769 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.769 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.769 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.769 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.769 INFO:teuthology.orchestra.run.smithi083.stdout: "identifier": { 2024-04-24T09:18:04.769 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "identifier", 2024-04-24T09:18:04.769 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.769 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.769 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.769 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.770 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.770 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.770 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.770 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.770 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.770 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.770 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.770 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.770 INFO:teuthology.orchestra.run.smithi083.stdout: "interval": { 2024-04-24T09:18:04.770 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "interval", 2024-04-24T09:18:04.770 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-24T09:18:04.770 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.770 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.771 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "60", 2024-04-24T09:18:04.771 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.771 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.771 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.771 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.771 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.771 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.771 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.771 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.771 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-24T09:18:04.771 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-24T09:18:04.771 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.771 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.771 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.771 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.771 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.772 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.772 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.772 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.772 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.772 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.772 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.772 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.772 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.772 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.772 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.772 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.772 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.772 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.772 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.772 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-24T09:18:04.772 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-24T09:18:04.773 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.773 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.773 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.773 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.773 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.773 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.773 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.773 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.773 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.773 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.773 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.773 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.773 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-24T09:18:04.773 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-24T09:18:04.774 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.774 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.774 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.774 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-24T09:18:04.774 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.774 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.774 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-24T09:18:04.774 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-24T09:18:04.774 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-24T09:18:04.774 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-24T09:18:04.774 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-24T09:18:04.774 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-24T09:18:04.774 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-24T09:18:04.774 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.775 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.775 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.775 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.775 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.775 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.775 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-24T09:18:04.775 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-24T09:18:04.775 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-24T09:18:04.775 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.775 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-24T09:18:04.775 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-24T09:18:04.775 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.775 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.775 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.776 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.776 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.776 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.776 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.776 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.776 INFO:teuthology.orchestra.run.smithi083.stdout: "zabbix_host": { 2024-04-24T09:18:04.776 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "zabbix_host", 2024-04-24T09:18:04.776 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.776 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.776 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.776 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-24T09:18:04.776 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.776 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.776 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.776 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.777 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.777 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.777 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.777 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.777 INFO:teuthology.orchestra.run.smithi083.stdout: "zabbix_port": { 2024-04-24T09:18:04.777 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "zabbix_port", 2024-04-24T09:18:04.777 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-24T09:18:04.777 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.777 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.777 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10051", 2024-04-24T09:18:04.777 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.777 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.777 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.777 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.777 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.778 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.778 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.778 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.778 INFO:teuthology.orchestra.run.smithi083.stdout: "zabbix_sender": { 2024-04-24T09:18:04.778 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "zabbix_sender", 2024-04-24T09:18:04.778 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-24T09:18:04.778 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-24T09:18:04.778 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-24T09:18:04.778 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-24T09:18:04.778 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-24T09:18:04.778 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-24T09:18:04.778 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-24T09:18:04.778 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-24T09:18:04.778 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-24T09:18:04.778 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-24T09:18:04.778 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-24T09:18:04.779 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.779 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.779 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.779 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.779 INFO:teuthology.orchestra.run.smithi083.stdout: "services": { 2024-04-24T09:18:04.779 INFO:teuthology.orchestra.run.smithi083.stdout: "dashboard": "https://172.21.15.83:8443/" 2024-04-24T09:18:04.779 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.779 INFO:teuthology.orchestra.run.smithi083.stdout: "always_on_modules": { 2024-04-24T09:18:04.779 INFO:teuthology.orchestra.run.smithi083.stdout: "octopus": [ 2024-04-24T09:18:04.779 INFO:teuthology.orchestra.run.smithi083.stdout: "balancer", 2024-04-24T09:18:04.779 INFO:teuthology.orchestra.run.smithi083.stdout: "crash", 2024-04-24T09:18:04.779 INFO:teuthology.orchestra.run.smithi083.stdout: "devicehealth", 2024-04-24T09:18:04.779 INFO:teuthology.orchestra.run.smithi083.stdout: "orchestrator", 2024-04-24T09:18:04.779 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_autoscaler", 2024-04-24T09:18:04.779 INFO:teuthology.orchestra.run.smithi083.stdout: "progress", 2024-04-24T09:18:04.780 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_support", 2024-04-24T09:18:04.780 INFO:teuthology.orchestra.run.smithi083.stdout: "status", 2024-04-24T09:18:04.780 INFO:teuthology.orchestra.run.smithi083.stdout: "telemetry", 2024-04-24T09:18:04.780 INFO:teuthology.orchestra.run.smithi083.stdout: "volumes" 2024-04-24T09:18:04.780 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.780 INFO:teuthology.orchestra.run.smithi083.stdout: "pacific": [ 2024-04-24T09:18:04.780 INFO:teuthology.orchestra.run.smithi083.stdout: "balancer", 2024-04-24T09:18:04.780 INFO:teuthology.orchestra.run.smithi083.stdout: "crash", 2024-04-24T09:18:04.780 INFO:teuthology.orchestra.run.smithi083.stdout: "devicehealth", 2024-04-24T09:18:04.780 INFO:teuthology.orchestra.run.smithi083.stdout: "orchestrator", 2024-04-24T09:18:04.780 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_autoscaler", 2024-04-24T09:18:04.780 INFO:teuthology.orchestra.run.smithi083.stdout: "progress", 2024-04-24T09:18:04.780 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_support", 2024-04-24T09:18:04.780 INFO:teuthology.orchestra.run.smithi083.stdout: "status", 2024-04-24T09:18:04.780 INFO:teuthology.orchestra.run.smithi083.stdout: "telemetry", 2024-04-24T09:18:04.781 INFO:teuthology.orchestra.run.smithi083.stdout: "volumes" 2024-04-24T09:18:04.781 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.781 INFO:teuthology.orchestra.run.smithi083.stdout: "quincy": [ 2024-04-24T09:18:04.781 INFO:teuthology.orchestra.run.smithi083.stdout: "balancer", 2024-04-24T09:18:04.781 INFO:teuthology.orchestra.run.smithi083.stdout: "crash", 2024-04-24T09:18:04.781 INFO:teuthology.orchestra.run.smithi083.stdout: "devicehealth", 2024-04-24T09:18:04.781 INFO:teuthology.orchestra.run.smithi083.stdout: "orchestrator", 2024-04-24T09:18:04.781 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_autoscaler", 2024-04-24T09:18:04.781 INFO:teuthology.orchestra.run.smithi083.stdout: "progress", 2024-04-24T09:18:04.781 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_support", 2024-04-24T09:18:04.781 INFO:teuthology.orchestra.run.smithi083.stdout: "status", 2024-04-24T09:18:04.781 INFO:teuthology.orchestra.run.smithi083.stdout: "telemetry", 2024-04-24T09:18:04.781 INFO:teuthology.orchestra.run.smithi083.stdout: "volumes" 2024-04-24T09:18:04.781 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.781 INFO:teuthology.orchestra.run.smithi083.stdout: "reef": [ 2024-04-24T09:18:04.782 INFO:teuthology.orchestra.run.smithi083.stdout: "balancer", 2024-04-24T09:18:04.782 INFO:teuthology.orchestra.run.smithi083.stdout: "crash", 2024-04-24T09:18:04.782 INFO:teuthology.orchestra.run.smithi083.stdout: "devicehealth", 2024-04-24T09:18:04.782 INFO:teuthology.orchestra.run.smithi083.stdout: "orchestrator", 2024-04-24T09:18:04.782 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_autoscaler", 2024-04-24T09:18:04.782 INFO:teuthology.orchestra.run.smithi083.stdout: "progress", 2024-04-24T09:18:04.782 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_support", 2024-04-24T09:18:04.782 INFO:teuthology.orchestra.run.smithi083.stdout: "status", 2024-04-24T09:18:04.782 INFO:teuthology.orchestra.run.smithi083.stdout: "telemetry", 2024-04-24T09:18:04.782 INFO:teuthology.orchestra.run.smithi083.stdout: "volumes" 2024-04-24T09:18:04.782 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-24T09:18:04.782 INFO:teuthology.orchestra.run.smithi083.stdout: "squid": [ 2024-04-24T09:18:04.782 INFO:teuthology.orchestra.run.smithi083.stdout: "balancer", 2024-04-24T09:18:04.782 INFO:teuthology.orchestra.run.smithi083.stdout: "crash", 2024-04-24T09:18:04.783 INFO:teuthology.orchestra.run.smithi083.stdout: "devicehealth", 2024-04-24T09:18:04.783 INFO:teuthology.orchestra.run.smithi083.stdout: "orchestrator", 2024-04-24T09:18:04.783 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_autoscaler", 2024-04-24T09:18:04.783 INFO:teuthology.orchestra.run.smithi083.stdout: "progress", 2024-04-24T09:18:04.783 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_support", 2024-04-24T09:18:04.783 INFO:teuthology.orchestra.run.smithi083.stdout: "status", 2024-04-24T09:18:04.783 INFO:teuthology.orchestra.run.smithi083.stdout: "telemetry", 2024-04-24T09:18:04.783 INFO:teuthology.orchestra.run.smithi083.stdout: "volumes" 2024-04-24T09:18:04.783 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-04-24T09:18:04.783 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.783 INFO:teuthology.orchestra.run.smithi083.stdout: "last_failure_osd_epoch": 94, 2024-04-24T09:18:04.783 INFO:teuthology.orchestra.run.smithi083.stdout: "active_clients": [ 2024-04-24T09:18:04.783 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:04.783 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "devicehealth", 2024-04-24T09:18:04.783 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-04-24T09:18:04.783 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:04.784 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-04-24T09:18:04.784 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:0", 2024-04-24T09:18:04.784 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 654596821 2024-04-24T09:18:04.784 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.784 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-04-24T09:18:04.784 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.784 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:04.784 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "libcephsqlite", 2024-04-24T09:18:04.784 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-04-24T09:18:04.784 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:04.784 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-04-24T09:18:04.784 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:0", 2024-04-24T09:18:04.784 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 779436740 2024-04-24T09:18:04.784 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.784 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-04-24T09:18:04.784 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.785 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:04.785 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rbd_support", 2024-04-24T09:18:04.785 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-04-24T09:18:04.785 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:04.785 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-04-24T09:18:04.785 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:0", 2024-04-24T09:18:04.785 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 3303151749 2024-04-24T09:18:04.785 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.785 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-04-24T09:18:04.785 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.785 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:04.785 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "snap_schedule", 2024-04-24T09:18:04.785 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-04-24T09:18:04.785 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:04.785 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-04-24T09:18:04.786 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:0", 2024-04-24T09:18:04.786 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 4283930798 2024-04-24T09:18:04.786 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.786 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-04-24T09:18:04.786 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.786 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:04.786 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "snap_schedule", 2024-04-24T09:18:04.786 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-04-24T09:18:04.786 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:04.786 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-04-24T09:18:04.786 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:0", 2024-04-24T09:18:04.786 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2020899242 2024-04-24T09:18:04.786 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.786 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-04-24T09:18:04.786 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:04.787 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:04.787 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "volumes", 2024-04-24T09:18:04.787 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-04-24T09:18:04.787 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-24T09:18:04.787 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-04-24T09:18:04.787 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:0", 2024-04-24T09:18:04.787 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 3787893223 2024-04-24T09:18:04.787 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.787 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-04-24T09:18:04.787 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:04.787 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-04-24T09:18:04.787 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T09:18:04.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:18:03 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/3025605232' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T09:18:04.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:18:03 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/2028765119' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T09:18:04.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:18:04 smithi189 ceph-mon[31791]: pgmap v836: 129 pgs: 129 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 48 KiB/s wr, 30 op/s 2024-04-24T09:18:04.789 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:18:04 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/3546085317' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-24T09:18:04.789 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:18:03 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/3025605232' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T09:18:04.789 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:18:03 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/2028765119' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T09:18:04.789 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:18:04 smithi138 ceph-mon[31441]: pgmap v836: 129 pgs: 129 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 48 KiB/s wr, 30 op/s 2024-04-24T09:18:04.789 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:18:04 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/3546085317' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-24T09:18:04.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:03.582+0000 7f1db6c00640 1 -- 172.21.15.83:0/3546085317 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f1d980a3d20 msgr2=0x7f1d980a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:04.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:03.582+0000 7f1db6c00640 1 --2- 172.21.15.83:0/3546085317 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f1d980a3d20 0x7f1d980a61e0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f1db007a370 tx=0x7f1d9c005e50 comp rx=0 tx=0).stop 2024-04-24T09:18:04.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:03.582+0000 7f1db6c00640 1 -- 172.21.15.83:0/3546085317 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1db01084a0 msgr2=0x7f1db01a4850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:04.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:03.582+0000 7f1db6c00640 1 --2- 172.21.15.83:0/3546085317 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1db01084a0 0x7f1db01a4850 secure :-1 s=READY pgs=2906 cs=0 l=1 rev1=1 crypto rx=0x7f1da4005940 tx=0x7f1da4004270 comp rx=0 tx=0).stop 2024-04-24T09:18:04.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:03.583+0000 7f1db6c00640 1 -- 172.21.15.83:0/3546085317 shutdown_connections 2024-04-24T09:18:04.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:03.583+0000 7f1db6c00640 1 --2- 172.21.15.83:0/3546085317 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f1d980a3d20 0x7f1d980a61e0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:04.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:03.583+0000 7f1db6c00640 1 --2- 172.21.15.83:0/3546085317 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1db010d3c0 0x7f1db019ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:04.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:03.583+0000 7f1db6c00640 1 --2- 172.21.15.83:0/3546085317 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1db0108e40 0x7f1db01a4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:04.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:03.583+0000 7f1db6c00640 1 --2- 172.21.15.83:0/3546085317 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1db01084a0 0x7f1db01a4850 unknown :-1 s=CLOSED pgs=2906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:04.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:03.583+0000 7f1db6c00640 1 -- 172.21.15.83:0/3546085317 >> 172.21.15.83:0/3546085317 conn(0x7f1db0077b80 msgr2=0x7f1db0078510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:18:04.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:03.583+0000 7f1db6c00640 1 -- 172.21.15.83:0/3546085317 shutdown_connections 2024-04-24T09:18:04.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:03.583+0000 7f1db6c00640 1 -- 172.21.15.83:0/3546085317 wait complete. 2024-04-24T09:18:04.797 DEBUG:tasks.check_counter:Getting stats from a 2024-04-24T09:18:04.798 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid bd3e039e-0215-11ef-bc93-c7b262605968 -- ceph daemon mds.a perf dump 2024-04-24T09:18:04.798 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:18:03 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/3025605232' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T09:18:04.798 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:18:03 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/2028765119' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T09:18:04.806 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:18:04 smithi083 ceph-mon[27907]: pgmap v836: 129 pgs: 129 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 48 KiB/s wr, 30 op/s 2024-04-24T09:18:04.806 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:18:04 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/3546085317' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-24T09:18:05.306 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:18:06.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:18:06 smithi138 ceph-mon[31441]: pgmap v837: 129 pgs: 129 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 48 KiB/s wr, 26 op/s 2024-04-24T09:18:06.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:18:06 smithi189 ceph-mon[31791]: pgmap v837: 129 pgs: 129 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 48 KiB/s wr, 26 op/s 2024-04-24T09:18:06.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:18:06 smithi083 ceph-mon[27907]: pgmap v837: 129 pgs: 129 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 48 KiB/s wr, 26 op/s 2024-04-24T09:18:07.547 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:18:07 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:18:07.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:18:07 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:18:07.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:18:07 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T09:18:08.132 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T09:18:08.132 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-0": { 2024-04-24T09:18:08.132 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 3218, 2024-04-24T09:18:08.132 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 3218, 2024-04-24T09:18:08.132 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 77780785, 2024-04-24T09:18:08.132 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 1022501, 2024-04-24T09:18:08.132 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 8, 2024-04-24T09:18:08.133 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 4, 2024-04-24T09:18:08.133 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.766882313, 2024-04-24T09:18:08.133 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.202730315, 2024-04-24T09:18:08.133 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.336459409, 2024-04-24T09:18:08.133 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.170893733, 2024-04-24T09:18:08.133 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T09:18:08.133 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 3217, 2024-04-24T09:18:08.133 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.255218557, 2024-04-24T09:18:08.133 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000079334 2024-04-24T09:18:08.133 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:08.133 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-04-24T09:18:08.133 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-24T09:18:08.133 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-24T09:18:08.133 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-24T09:18:08.133 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:08.134 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-04-24T09:18:08.134 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 448 2024-04-24T09:18:08.134 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:08.134 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-1": { 2024-04-24T09:18:08.134 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 2695, 2024-04-24T09:18:08.134 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 2632, 2024-04-24T09:18:08.134 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 124262941, 2024-04-24T09:18:08.134 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 972574, 2024-04-24T09:18:08.134 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 6, 2024-04-24T09:18:08.134 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 5, 2024-04-24T09:18:08.134 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.798535171, 2024-04-24T09:18:08.134 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.187076151, 2024-04-24T09:18:08.134 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.466508283, 2024-04-24T09:18:08.134 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.130081373, 2024-04-24T09:18:08.134 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T09:18:08.135 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 2631, 2024-04-24T09:18:08.135 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.213132241, 2024-04-24T09:18:08.135 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000081008 2024-04-24T09:18:08.135 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:08.135 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-04-24T09:18:08.135 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-24T09:18:08.135 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-24T09:18:08.135 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-24T09:18:08.135 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:08.135 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 2071648, 2024-04-24T09:18:08.135 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 302480 2024-04-24T09:18:08.135 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:08.135 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-2": { 2024-04-24T09:18:08.135 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 1001, 2024-04-24T09:18:08.135 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 1361, 2024-04-24T09:18:08.136 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 71566034, 2024-04-24T09:18:08.136 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 1204228, 2024-04-24T09:18:08.136 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 10, 2024-04-24T09:18:08.136 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 6, 2024-04-24T09:18:08.136 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.381647287, 2024-04-24T09:18:08.136 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.120299949, 2024-04-24T09:18:08.136 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.179870135, 2024-04-24T09:18:08.136 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.057999926, 2024-04-24T09:18:08.136 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T09:18:08.136 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 1358, 2024-04-24T09:18:08.136 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.120495335, 2024-04-24T09:18:08.136 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000088729 2024-04-24T09:18:08.136 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:08.136 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-04-24T09:18:08.136 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-24T09:18:08.136 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-24T09:18:08.137 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-24T09:18:08.137 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:08.137 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 480, 2024-04-24T09:18:08.137 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 886864 2024-04-24T09:18:08.137 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:08.137 INFO:teuthology.orchestra.run.smithi083.stdout: "cct": { 2024-04-24T09:18:08.137 INFO:teuthology.orchestra.run.smithi083.stdout: "total_workers": 1, 2024-04-24T09:18:08.137 INFO:teuthology.orchestra.run.smithi083.stdout: "unhealthy_workers": 0 2024-04-24T09:18:08.137 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:08.137 INFO:teuthology.orchestra.run.smithi083.stdout: "finisher-MDSRank": { 2024-04-24T09:18:08.137 INFO:teuthology.orchestra.run.smithi083.stdout: "queue_len": 0, 2024-04-24T09:18:08.137 INFO:teuthology.orchestra.run.smithi083.stdout: "complete_latency": { 2024-04-24T09:18:08.137 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 10710, 2024-04-24T09:18:08.137 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.480067117, 2024-04-24T09:18:08.137 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000044824 2024-04-24T09:18:08.138 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:08.138 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:08.138 INFO:teuthology.orchestra.run.smithi083.stdout: "finisher-PurgeQueue": { 2024-04-24T09:18:08.138 INFO:teuthology.orchestra.run.smithi083.stdout: "queue_len": 0, 2024-04-24T09:18:08.138 INFO:teuthology.orchestra.run.smithi083.stdout: "complete_latency": { 2024-04-24T09:18:08.138 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-24T09:18:08.138 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-24T09:18:08.138 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-24T09:18:08.138 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:08.138 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:08.138 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": { 2024-04-24T09:18:08.138 INFO:teuthology.orchestra.run.smithi083.stdout: "request": 0, 2024-04-24T09:18:08.138 INFO:teuthology.orchestra.run.smithi083.stdout: "reply": 0, 2024-04-24T09:18:08.138 INFO:teuthology.orchestra.run.smithi083.stdout: "reply_latency": { 2024-04-24T09:18:08.139 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-24T09:18:08.139 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-24T09:18:08.139 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-24T09:18:08.139 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:08.139 INFO:teuthology.orchestra.run.smithi083.stdout: "slow_reply": 0, 2024-04-24T09:18:08.139 INFO:teuthology.orchestra.run.smithi083.stdout: "forward": 0, 2024-04-24T09:18:08.139 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_fetch_complete": 0, 2024-04-24T09:18:08.139 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_fetch_keys": 0, 2024-04-24T09:18:08.139 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_commit": 0, 2024-04-24T09:18:08.139 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_split": 0, 2024-04-24T09:18:08.139 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_merge": 0, 2024-04-24T09:18:08.139 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes": 10072, 2024-04-24T09:18:08.139 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_top": 0, 2024-04-24T09:18:08.139 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_bottom": 10072, 2024-04-24T09:18:08.140 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_pin_tail": 0, 2024-04-24T09:18:08.140 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_pinned": 10072, 2024-04-24T09:18:08.140 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_expired": 0, 2024-04-24T09:18:08.140 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_with_caps": 0, 2024-04-24T09:18:08.140 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": 0, 2024-04-24T09:18:08.140 INFO:teuthology.orchestra.run.smithi083.stdout: "subtrees": 1, 2024-04-24T09:18:08.140 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse": 0, 2024-04-24T09:18:08.140 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_hit": 0, 2024-04-24T09:18:08.140 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_forward": 0, 2024-04-24T09:18:08.140 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_discover": 0, 2024-04-24T09:18:08.140 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_dir_fetch": 0, 2024-04-24T09:18:08.140 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_remote_ino": 0, 2024-04-24T09:18:08.140 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_lock": 0, 2024-04-24T09:18:08.140 INFO:teuthology.orchestra.run.smithi083.stdout: "load_cent": 0, 2024-04-24T09:18:08.140 INFO:teuthology.orchestra.run.smithi083.stdout: "q": 0, 2024-04-24T09:18:08.140 INFO:teuthology.orchestra.run.smithi083.stdout: "exported": 0, 2024-04-24T09:18:08.141 INFO:teuthology.orchestra.run.smithi083.stdout: "exported_inodes": 0, 2024-04-24T09:18:08.141 INFO:teuthology.orchestra.run.smithi083.stdout: "imported": 0, 2024-04-24T09:18:08.141 INFO:teuthology.orchestra.run.smithi083.stdout: "imported_inodes": 0, 2024-04-24T09:18:08.141 INFO:teuthology.orchestra.run.smithi083.stdout: "openino_dir_fetch": 0, 2024-04-24T09:18:08.141 INFO:teuthology.orchestra.run.smithi083.stdout: "openino_backtrace_fetch": 0, 2024-04-24T09:18:08.141 INFO:teuthology.orchestra.run.smithi083.stdout: "openino_peer_discover": 0, 2024-04-24T09:18:08.141 INFO:teuthology.orchestra.run.smithi083.stdout: "root_rfiles": 0, 2024-04-24T09:18:08.141 INFO:teuthology.orchestra.run.smithi083.stdout: "root_rbytes": 0, 2024-04-24T09:18:08.141 INFO:teuthology.orchestra.run.smithi083.stdout: "root_rsnaps": 0, 2024-04-24T09:18:08.141 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_backtrace_fetch": 0, 2024-04-24T09:18:08.141 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_set_tag": 0, 2024-04-24T09:18:08.141 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_backtrace_repaired": 0, 2024-04-24T09:18:08.141 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_inotable_repaired": 0, 2024-04-24T09:18:08.141 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_dir_inodes": 0, 2024-04-24T09:18:08.141 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_dir_base_inodes": 0, 2024-04-24T09:18:08.141 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_dirfrag_rstats": 0, 2024-04-24T09:18:08.141 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_file_inodes": 0, 2024-04-24T09:18:08.142 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_inode_file_caps": 0, 2024-04-24T09:18:08.142 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_revoke": 0, 2024-04-24T09:18:08.142 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_grant": 0, 2024-04-24T09:18:08.142 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_trunc": 0, 2024-04-24T09:18:08.142 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-24T09:18:08.142 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-24T09:18:08.142 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_caps": 0, 2024-04-24T09:18:08.142 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_caps_dirty": 0, 2024-04-24T09:18:08.142 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_cap_release": 0, 2024-04-24T09:18:08.142 INFO:teuthology.orchestra.run.smithi083.stdout: "process_request_cap_release": 0 2024-04-24T09:18:08.142 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:08.142 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_cache": { 2024-04-24T09:18:08.142 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_update": 0, 2024-04-24T09:18:08.142 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_update_receipt": 0, 2024-04-24T09:18:08.142 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_try_discover": 0, 2024-04-24T09:18:08.142 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_send_discover": 0, 2024-04-24T09:18:08.143 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_handle_discover": 0, 2024-04-24T09:18:08.143 INFO:teuthology.orchestra.run.smithi083.stdout: "num_strays": 0, 2024-04-24T09:18:08.143 INFO:teuthology.orchestra.run.smithi083.stdout: "num_strays_delayed": 0, 2024-04-24T09:18:08.143 INFO:teuthology.orchestra.run.smithi083.stdout: "num_strays_enqueuing": 0, 2024-04-24T09:18:08.143 INFO:teuthology.orchestra.run.smithi083.stdout: "strays_created": 0, 2024-04-24T09:18:08.143 INFO:teuthology.orchestra.run.smithi083.stdout: "strays_enqueued": 0, 2024-04-24T09:18:08.143 INFO:teuthology.orchestra.run.smithi083.stdout: "strays_reintegrated": 0, 2024-04-24T09:18:08.143 INFO:teuthology.orchestra.run.smithi083.stdout: "strays_migrated": 0, 2024-04-24T09:18:08.143 INFO:teuthology.orchestra.run.smithi083.stdout: "num_recovering_processing": 0, 2024-04-24T09:18:08.143 INFO:teuthology.orchestra.run.smithi083.stdout: "num_recovering_enqueued": 0, 2024-04-24T09:18:08.143 INFO:teuthology.orchestra.run.smithi083.stdout: "num_recovering_prioritized": 0, 2024-04-24T09:18:08.143 INFO:teuthology.orchestra.run.smithi083.stdout: "recovery_started": 0, 2024-04-24T09:18:08.143 INFO:teuthology.orchestra.run.smithi083.stdout: "recovery_completed": 0, 2024-04-24T09:18:08.143 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_quiesce_path": 0, 2024-04-24T09:18:08.143 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_quiesce_inode": 0, 2024-04-24T09:18:08.144 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_enqueue_scrub": 0, 2024-04-24T09:18:08.144 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_exportdir": 0, 2024-04-24T09:18:08.144 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_flush": 0, 2024-04-24T09:18:08.144 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_fragmentdir": 0, 2024-04-24T09:18:08.144 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_fragstats": 0, 2024-04-24T09:18:08.144 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_inodestats": 0 2024-04-24T09:18:08.144 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:08.144 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_log": { 2024-04-24T09:18:08.144 INFO:teuthology.orchestra.run.smithi083.stdout: "evlrg": 0, 2024-04-24T09:18:08.144 INFO:teuthology.orchestra.run.smithi083.stdout: "evadd": 0, 2024-04-24T09:18:08.144 INFO:teuthology.orchestra.run.smithi083.stdout: "evex": 0, 2024-04-24T09:18:08.144 INFO:teuthology.orchestra.run.smithi083.stdout: "evtrm": 0, 2024-04-24T09:18:08.144 INFO:teuthology.orchestra.run.smithi083.stdout: "ev": 53979, 2024-04-24T09:18:08.144 INFO:teuthology.orchestra.run.smithi083.stdout: "evexg": 0, 2024-04-24T09:18:08.144 INFO:teuthology.orchestra.run.smithi083.stdout: "evexd": 0, 2024-04-24T09:18:08.144 INFO:teuthology.orchestra.run.smithi083.stdout: "segadd": 0, 2024-04-24T09:18:08.145 INFO:teuthology.orchestra.run.smithi083.stdout: "segex": 0, 2024-04-24T09:18:08.145 INFO:teuthology.orchestra.run.smithi083.stdout: "segtrm": 0, 2024-04-24T09:18:08.145 INFO:teuthology.orchestra.run.smithi083.stdout: "seg": 88, 2024-04-24T09:18:08.145 INFO:teuthology.orchestra.run.smithi083.stdout: "segmjr": 6, 2024-04-24T09:18:08.145 INFO:teuthology.orchestra.run.smithi083.stdout: "segexg": 0, 2024-04-24T09:18:08.145 INFO:teuthology.orchestra.run.smithi083.stdout: "segexd": 0, 2024-04-24T09:18:08.145 INFO:teuthology.orchestra.run.smithi083.stdout: "expos": 4194304, 2024-04-24T09:18:08.145 INFO:teuthology.orchestra.run.smithi083.stdout: "wrpos": 273943710, 2024-04-24T09:18:08.145 INFO:teuthology.orchestra.run.smithi083.stdout: "rdpos": 273929797, 2024-04-24T09:18:08.145 INFO:teuthology.orchestra.run.smithi083.stdout: "jlat": { 2024-04-24T09:18:08.145 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-24T09:18:08.145 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-24T09:18:08.145 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-24T09:18:08.145 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:08.145 INFO:teuthology.orchestra.run.smithi083.stdout: "replayed": 53979 2024-04-24T09:18:08.146 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:08.146 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_mem": { 2024-04-24T09:18:08.146 INFO:teuthology.orchestra.run.smithi083.stdout: "ino": 1524, 2024-04-24T09:18:08.146 INFO:teuthology.orchestra.run.smithi083.stdout: "ino+": 10017, 2024-04-24T09:18:08.146 INFO:teuthology.orchestra.run.smithi083.stdout: "ino-": 8493, 2024-04-24T09:18:08.146 INFO:teuthology.orchestra.run.smithi083.stdout: "dir": 59, 2024-04-24T09:18:08.146 INFO:teuthology.orchestra.run.smithi083.stdout: "dir+": 92, 2024-04-24T09:18:08.146 INFO:teuthology.orchestra.run.smithi083.stdout: "dir-": 33, 2024-04-24T09:18:08.146 INFO:teuthology.orchestra.run.smithi083.stdout: "dn": 10072, 2024-04-24T09:18:08.146 INFO:teuthology.orchestra.run.smithi083.stdout: "dn+": 10376, 2024-04-24T09:18:08.146 INFO:teuthology.orchestra.run.smithi083.stdout: "dn-": 304, 2024-04-24T09:18:08.146 INFO:teuthology.orchestra.run.smithi083.stdout: "cap": 0, 2024-04-24T09:18:08.146 INFO:teuthology.orchestra.run.smithi083.stdout: "cap+": 0, 2024-04-24T09:18:08.146 INFO:teuthology.orchestra.run.smithi083.stdout: "cap-": 0, 2024-04-24T09:18:08.146 INFO:teuthology.orchestra.run.smithi083.stdout: "rss": 67488, 2024-04-24T09:18:08.146 INFO:teuthology.orchestra.run.smithi083.stdout: "heap": 215416 2024-04-24T09:18:08.146 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:08.147 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_server": { 2024-04-24T09:18:08.147 INFO:teuthology.orchestra.run.smithi083.stdout: "dispatch_client_request": 0, 2024-04-24T09:18:08.147 INFO:teuthology.orchestra.run.smithi083.stdout: "dispatch_server_request": 0, 2024-04-24T09:18:08.147 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_request": 0, 2024-04-24T09:18:08.147 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_session": 0, 2024-04-24T09:18:08.147 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_peer_request": 0, 2024-04-24T09:18:08.147 INFO:teuthology.orchestra.run.smithi083.stdout: "req_create_latency": { 2024-04-24T09:18:08.147 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-24T09:18:08.147 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-24T09:18:08.147 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-24T09:18:08.147 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:08.147 INFO:teuthology.orchestra.run.smithi083.stdout: "req_getattr_latency": { 2024-04-24T09:18:08.147 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-24T09:18:08.147 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-24T09:18:08.147 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-24T09:18:08.147 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:08.148 INFO:teuthology.orchestra.run.smithi083.stdout: "req_getfilelock_latency": { 2024-04-24T09:18:08.148 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-24T09:18:08.148 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-24T09:18:08.148 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-24T09:18:08.148 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:08.148 INFO:teuthology.orchestra.run.smithi083.stdout: "req_link_latency": { 2024-04-24T09:18:08.148 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-24T09:18:08.148 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-24T09:18:08.148 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-24T09:18:08.148 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:08.148 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookup_latency": { 2024-04-24T09:18:08.148 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-24T09:18:08.148 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-24T09:18:08.148 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-24T09:18:08.148 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:08.148 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookuphash_latency": { 2024-04-24T09:18:08.149 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-24T09:18:08.149 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-24T09:18:08.149 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-24T09:18:08.149 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:08.149 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookupino_latency": { 2024-04-24T09:18:08.149 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-24T09:18:08.149 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-24T09:18:08.149 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-24T09:18:08.149 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:08.149 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookupname_latency": { 2024-04-24T09:18:08.149 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-24T09:18:08.149 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-24T09:18:08.149 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-24T09:18:08.149 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:08.149 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookupparent_latency": { 2024-04-24T09:18:08.149 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-24T09:18:08.150 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-24T09:18:08.150 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-24T09:18:08.150 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:08.150 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookupsnap_latency": { 2024-04-24T09:18:08.150 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-24T09:18:08.150 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-24T09:18:08.150 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-24T09:18:08.150 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:08.150 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lssnap_latency": { 2024-04-24T09:18:08.150 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-24T09:18:08.150 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-24T09:18:08.150 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-24T09:18:08.150 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:08.151 INFO:teuthology.orchestra.run.smithi083.stdout: "req_mkdir_latency": { 2024-04-24T09:18:08.151 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-24T09:18:08.151 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-24T09:18:08.151 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-24T09:18:08.151 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:08.151 INFO:teuthology.orchestra.run.smithi083.stdout: "req_mknod_latency": { 2024-04-24T09:18:08.151 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-24T09:18:08.151 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-24T09:18:08.151 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-24T09:18:08.152 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:08.152 INFO:teuthology.orchestra.run.smithi083.stdout: "req_mksnap_latency": { 2024-04-24T09:18:08.152 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-24T09:18:08.152 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-24T09:18:08.152 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-24T09:18:08.152 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:08.152 INFO:teuthology.orchestra.run.smithi083.stdout: "req_open_latency": { 2024-04-24T09:18:08.152 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-24T09:18:08.152 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-24T09:18:08.152 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-24T09:18:08.152 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:08.152 INFO:teuthology.orchestra.run.smithi083.stdout: "req_readdir_latency": { 2024-04-24T09:18:08.152 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-24T09:18:08.152 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-24T09:18:08.153 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-24T09:18:08.153 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:08.153 INFO:teuthology.orchestra.run.smithi083.stdout: "req_rename_latency": { 2024-04-24T09:18:08.153 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-24T09:18:08.153 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-24T09:18:08.153 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-24T09:18:08.153 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:08.153 INFO:teuthology.orchestra.run.smithi083.stdout: "req_renamesnap_latency": { 2024-04-24T09:18:08.153 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-24T09:18:08.153 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-24T09:18:08.153 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-24T09:18:08.153 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:08.153 INFO:teuthology.orchestra.run.smithi083.stdout: "req_snapdiff_latency": { 2024-04-24T09:18:08.153 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-24T09:18:08.153 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-24T09:18:08.153 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-24T09:18:08.154 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:08.154 INFO:teuthology.orchestra.run.smithi083.stdout: "req_rmdir_latency": { 2024-04-24T09:18:08.154 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-24T09:18:08.154 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-24T09:18:08.154 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-24T09:18:08.154 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:08.154 INFO:teuthology.orchestra.run.smithi083.stdout: "req_rmsnap_latency": { 2024-04-24T09:18:08.154 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-24T09:18:08.154 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-24T09:18:08.154 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-24T09:18:08.154 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:08.154 INFO:teuthology.orchestra.run.smithi083.stdout: "req_rmxattr_latency": { 2024-04-24T09:18:08.154 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-24T09:18:08.154 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-24T09:18:08.154 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-24T09:18:08.155 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:08.155 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setattr_latency": { 2024-04-24T09:18:08.155 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-24T09:18:08.155 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-24T09:18:08.155 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-24T09:18:08.155 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:08.155 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setdirlayout_latency": { 2024-04-24T09:18:08.155 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-24T09:18:08.155 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-24T09:18:08.155 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-24T09:18:08.155 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:08.155 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setfilelock_latency": { 2024-04-24T09:18:08.155 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-24T09:18:08.155 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-24T09:18:08.155 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-24T09:18:08.155 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:08.156 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setlayout_latency": { 2024-04-24T09:18:08.156 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-24T09:18:08.156 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-24T09:18:08.156 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-24T09:18:08.156 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:08.156 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setxattr_latency": { 2024-04-24T09:18:08.156 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-24T09:18:08.156 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-24T09:18:08.156 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-24T09:18:08.156 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:08.156 INFO:teuthology.orchestra.run.smithi083.stdout: "req_symlink_latency": { 2024-04-24T09:18:08.156 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-24T09:18:08.156 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-24T09:18:08.156 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-24T09:18:08.156 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:08.157 INFO:teuthology.orchestra.run.smithi083.stdout: "req_unlink_latency": { 2024-04-24T09:18:08.157 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-24T09:18:08.157 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-24T09:18:08.157 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-24T09:18:08.157 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:08.157 INFO:teuthology.orchestra.run.smithi083.stdout: "cap_revoke_eviction": 0, 2024-04-24T09:18:08.157 INFO:teuthology.orchestra.run.smithi083.stdout: "cap_acquisition_throttle": 0, 2024-04-24T09:18:08.157 INFO:teuthology.orchestra.run.smithi083.stdout: "req_getvxattr_latency": { 2024-04-24T09:18:08.157 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-24T09:18:08.157 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-24T09:18:08.157 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-24T09:18:08.157 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:08.157 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:08.157 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_sessions": { 2024-04-24T09:18:08.157 INFO:teuthology.orchestra.run.smithi083.stdout: "session_count": 2, 2024-04-24T09:18:08.157 INFO:teuthology.orchestra.run.smithi083.stdout: "session_add": 4, 2024-04-24T09:18:08.157 INFO:teuthology.orchestra.run.smithi083.stdout: "session_remove": 2, 2024-04-24T09:18:08.158 INFO:teuthology.orchestra.run.smithi083.stdout: "sessions_open": 4, 2024-04-24T09:18:08.158 INFO:teuthology.orchestra.run.smithi083.stdout: "sessions_stale": 0, 2024-04-24T09:18:08.158 INFO:teuthology.orchestra.run.smithi083.stdout: "total_load": 0, 2024-04-24T09:18:08.158 INFO:teuthology.orchestra.run.smithi083.stdout: "average_load": 0, 2024-04-24T09:18:08.158 INFO:teuthology.orchestra.run.smithi083.stdout: "avg_session_uptime": 4433, 2024-04-24T09:18:08.158 INFO:teuthology.orchestra.run.smithi083.stdout: "mdthresh_evicted": 0 2024-04-24T09:18:08.158 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:08.158 INFO:teuthology.orchestra.run.smithi083.stdout: "mempool": { 2024-04-24T09:18:08.158 INFO:teuthology.orchestra.run.smithi083.stdout: "bloom_filter_bytes": 0, 2024-04-24T09:18:08.158 INFO:teuthology.orchestra.run.smithi083.stdout: "bloom_filter_items": 0, 2024-04-24T09:18:08.158 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_alloc_bytes": 0, 2024-04-24T09:18:08.158 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_alloc_items": 0, 2024-04-24T09:18:08.158 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_data_bytes": 0, 2024-04-24T09:18:08.158 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_data_items": 0, 2024-04-24T09:18:08.159 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-24T09:18:08.159 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_onode_items": 0, 2024-04-24T09:18:08.159 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-24T09:18:08.159 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_meta_items": 0, 2024-04-24T09:18:08.159 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_other_bytes": 0, 2024-04-24T09:18:08.159 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_other_items": 0, 2024-04-24T09:18:08.159 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-24T09:18:08.159 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_buffer_items": 0, 2024-04-24T09:18:08.159 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_extent_bytes": 0, 2024-04-24T09:18:08.159 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_extent_items": 0, 2024-04-24T09:18:08.159 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_blob_bytes": 0, 2024-04-24T09:18:08.159 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_blob_items": 0, 2024-04-24T09:18:08.159 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-24T09:18:08.159 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_shared_blob_items": 0, 2024-04-24T09:18:08.159 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-24T09:18:08.159 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_inline_bl_items": 0, 2024-04-24T09:18:08.160 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_bytes": 0, 2024-04-24T09:18:08.160 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_items": 0, 2024-04-24T09:18:08.160 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_txc_bytes": 0, 2024-04-24T09:18:08.160 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_txc_items": 0, 2024-04-24T09:18:08.160 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-24T09:18:08.160 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_deferred_items": 0, 2024-04-24T09:18:08.160 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_bytes": 0, 2024-04-24T09:18:08.160 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_items": 0, 2024-04-24T09:18:08.160 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_bytes": 0, 2024-04-24T09:18:08.160 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_items": 0, 2024-04-24T09:18:08.160 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_reader_bytes": 0, 2024-04-24T09:18:08.160 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_reader_items": 0, 2024-04-24T09:18:08.160 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_writer_bytes": 0, 2024-04-24T09:18:08.160 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_writer_items": 0, 2024-04-24T09:18:08.160 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_anon_bytes": 242577, 2024-04-24T09:18:08.160 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_anon_items": 87, 2024-04-24T09:18:08.160 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_meta_bytes": 0, 2024-04-24T09:18:08.161 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_meta_items": 0, 2024-04-24T09:18:08.161 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_bytes": 0, 2024-04-24T09:18:08.161 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_items": 0, 2024-04-24T09:18:08.161 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mapbl_bytes": 0, 2024-04-24T09:18:08.161 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mapbl_items": 0, 2024-04-24T09:18:08.161 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pglog_bytes": 0, 2024-04-24T09:18:08.161 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pglog_items": 0, 2024-04-24T09:18:08.161 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_bytes": 10040, 2024-04-24T09:18:08.161 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_items": 194, 2024-04-24T09:18:08.161 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_mapping_bytes": 0, 2024-04-24T09:18:08.161 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_mapping_items": 0, 2024-04-24T09:18:08.161 INFO:teuthology.orchestra.run.smithi083.stdout: "pgmap_bytes": 0, 2024-04-24T09:18:08.161 INFO:teuthology.orchestra.run.smithi083.stdout: "pgmap_items": 0, 2024-04-24T09:18:08.161 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_co_bytes": 10735654, 2024-04-24T09:18:08.161 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_co_items": 49318, 2024-04-24T09:18:08.161 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_1_bytes": 0, 2024-04-24T09:18:08.162 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_1_items": 0, 2024-04-24T09:18:08.162 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_2_bytes": 0, 2024-04-24T09:18:08.162 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_2_items": 0 2024-04-24T09:18:08.162 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:08.162 INFO:teuthology.orchestra.run.smithi083.stdout: "objecter": { 2024-04-24T09:18:08.162 INFO:teuthology.orchestra.run.smithi083.stdout: "op_active": 0, 2024-04-24T09:18:08.162 INFO:teuthology.orchestra.run.smithi083.stdout: "op_laggy": 0, 2024-04-24T09:18:08.162 INFO:teuthology.orchestra.run.smithi083.stdout: "op_send": 6319, 2024-04-24T09:18:08.162 INFO:teuthology.orchestra.run.smithi083.stdout: "op_send_bytes": 21, 2024-04-24T09:18:08.162 INFO:teuthology.orchestra.run.smithi083.stdout: "op_resend": 3, 2024-04-24T09:18:08.162 INFO:teuthology.orchestra.run.smithi083.stdout: "op_reply": 6319, 2024-04-24T09:18:08.162 INFO:teuthology.orchestra.run.smithi083.stdout: "op_latency": { 2024-04-24T09:18:08.162 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 6319, 2024-04-24T09:18:08.162 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 9.880767402, 2024-04-24T09:18:08.162 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.001563659 2024-04-24T09:18:08.162 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:08.163 INFO:teuthology.orchestra.run.smithi083.stdout: "op_inflight": 0, 2024-04-24T09:18:08.163 INFO:teuthology.orchestra.run.smithi083.stdout: "oplen_avg": { 2024-04-24T09:18:08.163 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 6319, 2024-04-24T09:18:08.163 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 6320 2024-04-24T09:18:08.163 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:08.163 INFO:teuthology.orchestra.run.smithi083.stdout: "op": 6319, 2024-04-24T09:18:08.163 INFO:teuthology.orchestra.run.smithi083.stdout: "op_r": 6319, 2024-04-24T09:18:08.163 INFO:teuthology.orchestra.run.smithi083.stdout: "op_w": 0, 2024-04-24T09:18:08.163 INFO:teuthology.orchestra.run.smithi083.stdout: "op_rmw": 0, 2024-04-24T09:18:08.163 INFO:teuthology.orchestra.run.smithi083.stdout: "op_pg": 0, 2024-04-24T09:18:08.163 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_stat": 3357, 2024-04-24T09:18:08.163 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_create": 0, 2024-04-24T09:18:08.163 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_read": 2960, 2024-04-24T09:18:08.163 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_write": 0, 2024-04-24T09:18:08.163 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_writefull": 0, 2024-04-24T09:18:08.163 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_writesame": 0, 2024-04-24T09:18:08.164 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_append": 0, 2024-04-24T09:18:08.164 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_zero": 0, 2024-04-24T09:18:08.164 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_truncate": 0, 2024-04-24T09:18:08.164 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_delete": 0, 2024-04-24T09:18:08.164 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_mapext": 0, 2024-04-24T09:18:08.164 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_sparse_read": 0, 2024-04-24T09:18:08.164 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_clonerange": 0, 2024-04-24T09:18:08.164 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_getxattr": 1, 2024-04-24T09:18:08.164 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_setxattr": 0, 2024-04-24T09:18:08.164 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_cmpxattr": 0, 2024-04-24T09:18:08.164 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_rmxattr": 0, 2024-04-24T09:18:08.164 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_resetxattrs": 0, 2024-04-24T09:18:08.164 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_call": 0, 2024-04-24T09:18:08.164 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_watch": 0, 2024-04-24T09:18:08.164 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_notify": 0, 2024-04-24T09:18:08.164 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_src_cmpxattr": 0, 2024-04-24T09:18:08.164 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_pgls": 0, 2024-04-24T09:18:08.165 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_pgls_filter": 0, 2024-04-24T09:18:08.165 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_other": 0, 2024-04-24T09:18:08.165 INFO:teuthology.orchestra.run.smithi083.stdout: "linger_active": 0, 2024-04-24T09:18:08.165 INFO:teuthology.orchestra.run.smithi083.stdout: "linger_send": 0, 2024-04-24T09:18:08.165 INFO:teuthology.orchestra.run.smithi083.stdout: "linger_resend": 0, 2024-04-24T09:18:08.165 INFO:teuthology.orchestra.run.smithi083.stdout: "linger_ping": 0, 2024-04-24T09:18:08.165 INFO:teuthology.orchestra.run.smithi083.stdout: "poolop_active": 0, 2024-04-24T09:18:08.165 INFO:teuthology.orchestra.run.smithi083.stdout: "poolop_send": 0, 2024-04-24T09:18:08.165 INFO:teuthology.orchestra.run.smithi083.stdout: "poolop_resend": 0, 2024-04-24T09:18:08.165 INFO:teuthology.orchestra.run.smithi083.stdout: "poolstat_active": 0, 2024-04-24T09:18:08.165 INFO:teuthology.orchestra.run.smithi083.stdout: "poolstat_send": 0, 2024-04-24T09:18:08.165 INFO:teuthology.orchestra.run.smithi083.stdout: "poolstat_resend": 0, 2024-04-24T09:18:08.165 INFO:teuthology.orchestra.run.smithi083.stdout: "statfs_active": 0, 2024-04-24T09:18:08.165 INFO:teuthology.orchestra.run.smithi083.stdout: "statfs_send": 0, 2024-04-24T09:18:08.165 INFO:teuthology.orchestra.run.smithi083.stdout: "statfs_resend": 0, 2024-04-24T09:18:08.165 INFO:teuthology.orchestra.run.smithi083.stdout: "command_active": 0, 2024-04-24T09:18:08.165 INFO:teuthology.orchestra.run.smithi083.stdout: "command_send": 0, 2024-04-24T09:18:08.165 INFO:teuthology.orchestra.run.smithi083.stdout: "command_resend": 0, 2024-04-24T09:18:08.166 INFO:teuthology.orchestra.run.smithi083.stdout: "map_epoch": 135, 2024-04-24T09:18:08.166 INFO:teuthology.orchestra.run.smithi083.stdout: "map_full": 0, 2024-04-24T09:18:08.166 INFO:teuthology.orchestra.run.smithi083.stdout: "map_inc": 46, 2024-04-24T09:18:08.166 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_sessions": 11, 2024-04-24T09:18:08.166 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_session_open": 12, 2024-04-24T09:18:08.166 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_session_close": 1, 2024-04-24T09:18:08.166 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_laggy": 0, 2024-04-24T09:18:08.166 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_wr": 0, 2024-04-24T09:18:08.166 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_rd": 2, 2024-04-24T09:18:08.166 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_del": 0 2024-04-24T09:18:08.166 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:08.166 INFO:teuthology.orchestra.run.smithi083.stdout: "oft": { 2024-04-24T09:18:08.166 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_total_objs": 0, 2024-04-24T09:18:08.166 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_total_kv_pairs": 0, 2024-04-24T09:18:08.166 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_total_updates": 0, 2024-04-24T09:18:08.167 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_total_removes": 0 2024-04-24T09:18:08.167 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:08.167 INFO:teuthology.orchestra.run.smithi083.stdout: "osd-slow-ops": { 2024-04-24T09:18:08.167 INFO:teuthology.orchestra.run.smithi083.stdout: "slow_ops_count": 0 2024-04-24T09:18:08.167 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:08.167 INFO:teuthology.orchestra.run.smithi083.stdout: "purge_queue": { 2024-04-24T09:18:08.167 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executing_ops": 0, 2024-04-24T09:18:08.167 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executing_ops_high_water": 0, 2024-04-24T09:18:08.167 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executing": 0, 2024-04-24T09:18:08.167 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executing_high_water": 0, 2024-04-24T09:18:08.167 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executed_ops": 0, 2024-04-24T09:18:08.167 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executed": 0, 2024-04-24T09:18:08.167 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_item_in_journal": 0 2024-04-24T09:18:08.167 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:08.167 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-24T09:18:08.168 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-04-24T09:18:08.168 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 104857600, 2024-04-24T09:18:08.168 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-04-24T09:18:08.168 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 6910, 2024-04-24T09:18:08.168 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 272954926, 2024-04-24T09:18:08.168 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-04-24T09:18:08.168 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 6910, 2024-04-24T09:18:08.168 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 0, 2024-04-24T09:18:08.168 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 0, 2024-04-24T09:18:08.168 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 6910, 2024-04-24T09:18:08.168 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 272954926, 2024-04-24T09:18:08.168 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-04-24T09:18:08.168 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-24T09:18:08.168 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-24T09:18:08.168 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-24T09:18:08.168 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:08.168 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:08.169 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-objecter_bytes": { 2024-04-24T09:18:08.169 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-04-24T09:18:08.169 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 104857600, 2024-04-24T09:18:08.169 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-04-24T09:18:08.169 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 0, 2024-04-24T09:18:08.169 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 0, 2024-04-24T09:18:08.169 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-04-24T09:18:08.169 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 0, 2024-04-24T09:18:08.169 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 6319, 2024-04-24T09:18:08.169 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 269788912, 2024-04-24T09:18:08.169 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 6319, 2024-04-24T09:18:08.169 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 269788912, 2024-04-24T09:18:08.169 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-04-24T09:18:08.169 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-24T09:18:08.169 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-24T09:18:08.169 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-24T09:18:08.170 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:08.170 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:08.170 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-objecter_ops": { 2024-04-24T09:18:08.170 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-04-24T09:18:08.170 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 1024, 2024-04-24T09:18:08.170 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-04-24T09:18:08.170 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 0, 2024-04-24T09:18:08.170 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 0, 2024-04-24T09:18:08.170 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-04-24T09:18:08.170 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 0, 2024-04-24T09:18:08.170 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 6319, 2024-04-24T09:18:08.170 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 6319, 2024-04-24T09:18:08.170 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 6319, 2024-04-24T09:18:08.170 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 6319, 2024-04-24T09:18:08.170 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-04-24T09:18:08.170 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-24T09:18:08.171 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-24T09:18:08.171 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-24T09:18:08.171 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:08.171 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:08.171 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-write_buf_throttle": { 2024-04-24T09:18:08.171 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-04-24T09:18:08.171 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 3758096384, 2024-04-24T09:18:08.171 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-04-24T09:18:08.171 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 0, 2024-04-24T09:18:08.171 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 0, 2024-04-24T09:18:08.171 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-04-24T09:18:08.171 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 0, 2024-04-24T09:18:08.171 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 0, 2024-04-24T09:18:08.171 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 0, 2024-04-24T09:18:08.171 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 0, 2024-04-24T09:18:08.171 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 0, 2024-04-24T09:18:08.172 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-04-24T09:18:08.172 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-24T09:18:08.172 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-24T09:18:08.172 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-24T09:18:08.172 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:08.172 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:08.172 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-write_buf_throttle-0x555dacc0a780": { 2024-04-24T09:18:08.172 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-04-24T09:18:08.172 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 3758096384, 2024-04-24T09:18:08.172 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-04-24T09:18:08.172 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 0, 2024-04-24T09:18:08.172 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 0, 2024-04-24T09:18:08.172 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-04-24T09:18:08.172 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 0, 2024-04-24T09:18:08.172 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 0, 2024-04-24T09:18:08.172 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 0, 2024-04-24T09:18:08.173 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 0, 2024-04-24T09:18:08.173 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 0, 2024-04-24T09:18:08.173 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-04-24T09:18:08.173 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-24T09:18:08.173 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-24T09:18:08.173 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-24T09:18:08.173 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:08.173 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:08.173 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T09:18:08.656 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:18:08 smithi083 ceph-mon[27907]: pgmap v838: 129 pgs: 129 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 48 KiB/s wr, 26 op/s 2024-04-24T09:18:08.656 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:18:08 smithi083 ceph-mon[27907]: osdmap e135: 12 total, 12 up, 12 in 2024-04-24T09:18:08.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:18:08 smithi138 ceph-mon[31441]: pgmap v838: 129 pgs: 129 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 48 KiB/s wr, 26 op/s 2024-04-24T09:18:08.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:18:08 smithi138 ceph-mon[31441]: osdmap e135: 12 total, 12 up, 12 in 2024-04-24T09:18:08.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:18:08 smithi189 ceph-mon[31791]: pgmap v838: 129 pgs: 129 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 48 KiB/s wr, 26 op/s 2024-04-24T09:18:08.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:18:08 smithi189 ceph-mon[31791]: osdmap e135: 12 total, 12 up, 12 in 2024-04-24T09:18:09.289 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-04-24T09:18:09.289 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-04-24T09:18:09.290 INFO:tasks.check_counter:Daemon mds.a mds.root_rsnaps=0 2024-04-24T09:18:09.290 INFO:tasks.check_counter:Daemon mds.a mds_server.req_mksnap_latency.avgcount=0 2024-04-24T09:18:09.290 INFO:tasks.check_counter:Daemon mds.a mds_server.req_rmsnap_latency.avgcount=0 2024-04-24T09:18:09.290 DEBUG:tasks.check_counter:Getting stats from d 2024-04-24T09:18:09.290 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid bd3e039e-0215-11ef-bc93-c7b262605968 -- ceph daemon mds.d perf dump 2024-04-24T09:18:09.735 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:18:10.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:18:10 smithi083 ceph-mon[27907]: pgmap v840: 129 pgs: 23 active+clean+snaptrim, 69 active+clean+snaptrim_wait, 37 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 56 KiB/s wr, 31 op/s 2024-04-24T09:18:10.695 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:18:10 smithi189 ceph-mon[31791]: pgmap v840: 129 pgs: 23 active+clean+snaptrim, 69 active+clean+snaptrim_wait, 37 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 56 KiB/s wr, 31 op/s 2024-04-24T09:18:10.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:18:10 smithi138 ceph-mon[31441]: pgmap v840: 129 pgs: 23 active+clean+snaptrim, 69 active+clean+snaptrim_wait, 37 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 56 KiB/s wr, 31 op/s 2024-04-24T09:18:12.433 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T09:18:12.434 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-0": { 2024-04-24T09:18:12.434 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 494, 2024-04-24T09:18:12.434 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 483, 2024-04-24T09:18:12.434 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 1667888, 2024-04-24T09:18:12.434 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 299008, 2024-04-24T09:18:12.434 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 4, 2024-04-24T09:18:12.434 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 0, 2024-04-24T09:18:12.434 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.169311383, 2024-04-24T09:18:12.434 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.065790932, 2024-04-24T09:18:12.434 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.098265560, 2024-04-24T09:18:12.434 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.020505294, 2024-04-24T09:18:12.434 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T09:18:12.434 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 482, 2024-04-24T09:18:12.435 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.043217448, 2024-04-24T09:18:12.435 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000089662 2024-04-24T09:18:12.435 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:12.435 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-04-24T09:18:12.435 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-24T09:18:12.435 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-24T09:18:12.435 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-24T09:18:12.435 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:12.435 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 1667888, 2024-04-24T09:18:12.435 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 299008 2024-04-24T09:18:12.435 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:12.435 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-1": { 2024-04-24T09:18:12.435 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 7, 2024-04-24T09:18:12.435 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 371, 2024-04-24T09:18:12.435 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 3424, 2024-04-24T09:18:12.435 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 299776, 2024-04-24T09:18:12.436 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 6, 2024-04-24T09:18:12.436 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 3, 2024-04-24T09:18:12.436 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.060861625, 2024-04-24T09:18:12.436 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.047301475, 2024-04-24T09:18:12.436 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.005393685, 2024-04-24T09:18:12.436 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-24T09:18:12.436 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T09:18:12.436 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 369, 2024-04-24T09:18:12.436 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.048488816, 2024-04-24T09:18:12.436 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000131406 2024-04-24T09:18:12.436 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:12.436 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-04-24T09:18:12.436 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-24T09:18:12.436 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-24T09:18:12.436 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-24T09:18:12.436 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:12.437 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 3424, 2024-04-24T09:18:12.437 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 299776 2024-04-24T09:18:12.437 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:12.437 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-2": { 2024-04-24T09:18:12.437 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 0, 2024-04-24T09:18:12.437 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 2, 2024-04-24T09:18:12.437 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 0, 2024-04-24T09:18:12.437 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 0, 2024-04-24T09:18:12.437 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 2, 2024-04-24T09:18:12.437 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 1, 2024-04-24T09:18:12.437 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.001030312, 2024-04-24T09:18:12.437 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.000000000, 2024-04-24T09:18:12.437 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.000824669, 2024-04-24T09:18:12.437 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-24T09:18:12.437 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T09:18:12.437 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-24T09:18:12.437 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-24T09:18:12.438 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-24T09:18:12.438 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:12.438 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-04-24T09:18:12.438 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-24T09:18:12.438 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-24T09:18:12.438 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-24T09:18:12.438 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:12.438 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-24T09:18:12.438 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 0 2024-04-24T09:18:12.438 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:12.438 INFO:teuthology.orchestra.run.smithi083.stdout: "cct": { 2024-04-24T09:18:12.438 INFO:teuthology.orchestra.run.smithi083.stdout: "total_workers": 0, 2024-04-24T09:18:12.438 INFO:teuthology.orchestra.run.smithi083.stdout: "unhealthy_workers": 0 2024-04-24T09:18:12.439 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:12.439 INFO:teuthology.orchestra.run.smithi083.stdout: "mempool": { 2024-04-24T09:18:12.439 INFO:teuthology.orchestra.run.smithi083.stdout: "bloom_filter_bytes": 0, 2024-04-24T09:18:12.439 INFO:teuthology.orchestra.run.smithi083.stdout: "bloom_filter_items": 0, 2024-04-24T09:18:12.439 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_alloc_bytes": 0, 2024-04-24T09:18:12.439 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_alloc_items": 0, 2024-04-24T09:18:12.439 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_data_bytes": 0, 2024-04-24T09:18:12.439 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_data_items": 0, 2024-04-24T09:18:12.439 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-24T09:18:12.439 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_onode_items": 0, 2024-04-24T09:18:12.439 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-24T09:18:12.439 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_meta_items": 0, 2024-04-24T09:18:12.439 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_other_bytes": 0, 2024-04-24T09:18:12.439 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_other_items": 0, 2024-04-24T09:18:12.439 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-24T09:18:12.439 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_buffer_items": 0, 2024-04-24T09:18:12.440 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_extent_bytes": 0, 2024-04-24T09:18:12.440 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_extent_items": 0, 2024-04-24T09:18:12.440 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_blob_bytes": 0, 2024-04-24T09:18:12.440 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_blob_items": 0, 2024-04-24T09:18:12.440 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-24T09:18:12.440 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_shared_blob_items": 0, 2024-04-24T09:18:12.440 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-24T09:18:12.440 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_inline_bl_items": 0, 2024-04-24T09:18:12.440 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_bytes": 0, 2024-04-24T09:18:12.440 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_items": 0, 2024-04-24T09:18:12.440 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_txc_bytes": 0, 2024-04-24T09:18:12.440 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_txc_items": 0, 2024-04-24T09:18:12.440 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-24T09:18:12.440 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_deferred_items": 0, 2024-04-24T09:18:12.440 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_bytes": 0, 2024-04-24T09:18:12.441 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_items": 0, 2024-04-24T09:18:12.441 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_bytes": 0, 2024-04-24T09:18:12.441 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_items": 0, 2024-04-24T09:18:12.441 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_reader_bytes": 0, 2024-04-24T09:18:12.441 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_reader_items": 0, 2024-04-24T09:18:12.441 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_writer_bytes": 0, 2024-04-24T09:18:12.441 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_writer_items": 0, 2024-04-24T09:18:12.441 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_anon_bytes": 148714, 2024-04-24T09:18:12.441 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_anon_items": 30, 2024-04-24T09:18:12.441 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_meta_bytes": 0, 2024-04-24T09:18:12.441 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_meta_items": 0, 2024-04-24T09:18:12.441 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_bytes": 0, 2024-04-24T09:18:12.441 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_items": 0, 2024-04-24T09:18:12.441 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mapbl_bytes": 0, 2024-04-24T09:18:12.441 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mapbl_items": 0, 2024-04-24T09:18:12.441 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pglog_bytes": 0, 2024-04-24T09:18:12.442 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pglog_items": 0, 2024-04-24T09:18:12.442 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_bytes": 0, 2024-04-24T09:18:12.442 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_items": 0, 2024-04-24T09:18:12.442 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_mapping_bytes": 0, 2024-04-24T09:18:12.442 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_mapping_items": 0, 2024-04-24T09:18:12.442 INFO:teuthology.orchestra.run.smithi083.stdout: "pgmap_bytes": 0, 2024-04-24T09:18:12.442 INFO:teuthology.orchestra.run.smithi083.stdout: "pgmap_items": 0, 2024-04-24T09:18:12.442 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_co_bytes": 928, 2024-04-24T09:18:12.442 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_co_items": 2, 2024-04-24T09:18:12.442 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_1_bytes": 0, 2024-04-24T09:18:12.442 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_1_items": 0, 2024-04-24T09:18:12.442 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_2_bytes": 0, 2024-04-24T09:18:12.442 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_2_items": 0 2024-04-24T09:18:12.442 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:12.442 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-24T09:18:12.442 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-04-24T09:18:12.443 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 104857600, 2024-04-24T09:18:12.443 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-04-24T09:18:12.443 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 497, 2024-04-24T09:18:12.443 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 1598096, 2024-04-24T09:18:12.443 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-04-24T09:18:12.443 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 497, 2024-04-24T09:18:12.443 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 0, 2024-04-24T09:18:12.443 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 0, 2024-04-24T09:18:12.443 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 497, 2024-04-24T09:18:12.443 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 1598096, 2024-04-24T09:18:12.443 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-04-24T09:18:12.443 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-24T09:18:12.443 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-24T09:18:12.443 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-24T09:18:12.443 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:12.443 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:12.443 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T09:18:12.677 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:18:12 smithi138 ceph-mon[31441]: pgmap v841: 129 pgs: 23 active+clean+snaptrim, 69 active+clean+snaptrim_wait, 37 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 56 KiB/s wr, 31 op/s 2024-04-24T09:18:12.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:18:12 smithi189 ceph-mon[31791]: pgmap v841: 129 pgs: 23 active+clean+snaptrim, 69 active+clean+snaptrim_wait, 37 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 56 KiB/s wr, 31 op/s 2024-04-24T09:18:12.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:18:12 smithi083 ceph-mon[27907]: pgmap v841: 129 pgs: 23 active+clean+snaptrim, 69 active+clean+snaptrim_wait, 37 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 56 KiB/s wr, 31 op/s 2024-04-24T09:18:13.357 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-04-24T09:18:13.357 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-04-24T09:18:13.357 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-04-24T09:18:13.357 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-04-24T09:18:13.357 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-04-24T09:18:13.357 DEBUG:tasks.check_counter:Getting stats from g 2024-04-24T09:18:13.358 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid bd3e039e-0215-11ef-bc93-c7b262605968 -- ceph daemon mds.g perf dump 2024-04-24T09:18:13.799 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:18:14.518 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:18:14 smithi083 ceph-mon[27907]: pgmap v842: 129 pgs: 129 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.1 KiB/s wr, 10 op/s 2024-04-24T09:18:14.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:18:14 smithi138 ceph-mon[31441]: pgmap v842: 129 pgs: 129 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.1 KiB/s wr, 10 op/s 2024-04-24T09:18:14.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:18:14 smithi189 ceph-mon[31791]: pgmap v842: 129 pgs: 129 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.1 KiB/s wr, 10 op/s 2024-04-24T09:18:15.703 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:18:15 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:18:15.703 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:18:15 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:18:15.703 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:18:15 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:18:15.703 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:18:15 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:18:15.704 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:18:15 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:18:15.704 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:18:15 smithi083 ceph-mon[27907]: osdmap e136: 12 total, 12 up, 12 in 2024-04-24T09:18:15.704 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:18:15 smithi083 ceph-mon[27907]: pgmap v844: 129 pgs: 129 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 2.6 KiB/s wr, 13 op/s 2024-04-24T09:18:15.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:18:15 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:18:15.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:18:15 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:18:15.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:18:15 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:18:15.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:18:15 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:18:15.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:18:15 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:18:15.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:18:15 smithi138 ceph-mon[31441]: osdmap e136: 12 total, 12 up, 12 in 2024-04-24T09:18:15.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:18:15 smithi138 ceph-mon[31441]: pgmap v844: 129 pgs: 129 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 2.6 KiB/s wr, 13 op/s 2024-04-24T09:18:15.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:18:15 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:18:15.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:18:15 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:18:15.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:18:15 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T09:18:15.957 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:18:15 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T09:18:15.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:18:15 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' 2024-04-24T09:18:15.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:18:15 smithi189 ceph-mon[31791]: osdmap e136: 12 total, 12 up, 12 in 2024-04-24T09:18:15.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:18:15 smithi189 ceph-mon[31791]: pgmap v844: 129 pgs: 129 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 2.6 KiB/s wr, 13 op/s 2024-04-24T09:18:16.399 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T09:18:16.400 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-0": { 2024-04-24T09:18:16.400 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 3041, 2024-04-24T09:18:16.400 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 2978, 2024-04-24T09:18:16.400 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 65752792, 2024-04-24T09:18:16.400 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 1083376, 2024-04-24T09:18:16.400 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 7, 2024-04-24T09:18:16.400 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 3, 2024-04-24T09:18:16.400 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.753838612, 2024-04-24T09:18:16.400 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.269673331, 2024-04-24T09:18:16.400 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.319909805, 2024-04-24T09:18:16.400 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.138539338, 2024-04-24T09:18:16.400 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T09:18:16.400 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 2977, 2024-04-24T09:18:16.401 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.252837404, 2024-04-24T09:18:16.401 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000084930 2024-04-24T09:18:16.401 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:16.401 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-04-24T09:18:16.401 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-24T09:18:16.401 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-24T09:18:16.401 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-24T09:18:16.401 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:16.401 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 2073872, 2024-04-24T09:18:16.401 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 304912 2024-04-24T09:18:16.401 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:16.401 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-1": { 2024-04-24T09:18:16.401 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 2542, 2024-04-24T09:18:16.402 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 2902, 2024-04-24T09:18:16.402 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 54996543, 2024-04-24T09:18:16.402 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 1608025, 2024-04-24T09:18:16.402 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 9, 2024-04-24T09:18:16.402 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 7, 2024-04-24T09:18:16.402 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.561485378, 2024-04-24T09:18:16.402 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.180992880, 2024-04-24T09:18:16.402 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.216381594, 2024-04-24T09:18:16.402 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.124701589, 2024-04-24T09:18:16.402 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T09:18:16.402 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 2900, 2024-04-24T09:18:16.402 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.257131734, 2024-04-24T09:18:16.402 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000088666 2024-04-24T09:18:16.403 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:16.403 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-04-24T09:18:16.403 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-24T09:18:16.403 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-24T09:18:16.403 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-24T09:18:16.403 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:16.403 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-04-24T09:18:16.403 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 801520 2024-04-24T09:18:16.403 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:16.403 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-2": { 2024-04-24T09:18:16.403 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 869, 2024-04-24T09:18:16.403 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 871, 2024-04-24T09:18:16.403 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 71524451, 2024-04-24T09:18:16.403 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 276354, 2024-04-24T09:18:16.404 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 8, 2024-04-24T09:18:16.404 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 5, 2024-04-24T09:18:16.404 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.264856985, 2024-04-24T09:18:16.404 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.031599401, 2024-04-24T09:18:16.404 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.190400232, 2024-04-24T09:18:16.404 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.033680161, 2024-04-24T09:18:16.404 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T09:18:16.404 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 869, 2024-04-24T09:18:16.404 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.063444188, 2024-04-24T09:18:16.404 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000073008 2024-04-24T09:18:16.404 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:16.404 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-04-24T09:18:16.404 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-24T09:18:16.404 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-24T09:18:16.404 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-24T09:18:16.405 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:16.405 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-24T09:18:16.405 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 0 2024-04-24T09:18:16.405 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:16.405 INFO:teuthology.orchestra.run.smithi083.stdout: "cct": { 2024-04-24T09:18:16.405 INFO:teuthology.orchestra.run.smithi083.stdout: "total_workers": 1, 2024-04-24T09:18:16.405 INFO:teuthology.orchestra.run.smithi083.stdout: "unhealthy_workers": 0 2024-04-24T09:18:16.405 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:16.405 INFO:teuthology.orchestra.run.smithi083.stdout: "finisher-MDSRank": { 2024-04-24T09:18:16.405 INFO:teuthology.orchestra.run.smithi083.stdout: "queue_len": 0, 2024-04-24T09:18:16.405 INFO:teuthology.orchestra.run.smithi083.stdout: "complete_latency": { 2024-04-24T09:18:16.405 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 9969, 2024-04-24T09:18:16.405 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.419228360, 2024-04-24T09:18:16.405 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000042053 2024-04-24T09:18:16.405 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:16.406 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:16.406 INFO:teuthology.orchestra.run.smithi083.stdout: "finisher-PurgeQueue": { 2024-04-24T09:18:16.406 INFO:teuthology.orchestra.run.smithi083.stdout: "queue_len": 0, 2024-04-24T09:18:16.406 INFO:teuthology.orchestra.run.smithi083.stdout: "complete_latency": { 2024-04-24T09:18:16.406 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-24T09:18:16.406 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-24T09:18:16.406 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-24T09:18:16.406 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:16.406 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:16.406 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": { 2024-04-24T09:18:16.406 INFO:teuthology.orchestra.run.smithi083.stdout: "request": 0, 2024-04-24T09:18:16.406 INFO:teuthology.orchestra.run.smithi083.stdout: "reply": 0, 2024-04-24T09:18:16.406 INFO:teuthology.orchestra.run.smithi083.stdout: "reply_latency": { 2024-04-24T09:18:16.406 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-24T09:18:16.407 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-24T09:18:16.407 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-24T09:18:16.407 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:16.407 INFO:teuthology.orchestra.run.smithi083.stdout: "slow_reply": 0, 2024-04-24T09:18:16.407 INFO:teuthology.orchestra.run.smithi083.stdout: "forward": 0, 2024-04-24T09:18:16.407 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_fetch_complete": 0, 2024-04-24T09:18:16.407 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_fetch_keys": 0, 2024-04-24T09:18:16.407 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_commit": 0, 2024-04-24T09:18:16.407 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_split": 0, 2024-04-24T09:18:16.407 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_merge": 0, 2024-04-24T09:18:16.407 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes": 3884, 2024-04-24T09:18:16.407 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_top": 1158, 2024-04-24T09:18:16.408 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_bottom": 2726, 2024-04-24T09:18:16.408 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_pin_tail": 0, 2024-04-24T09:18:16.408 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_pinned": 2229, 2024-04-24T09:18:16.408 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_expired": 0, 2024-04-24T09:18:16.408 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_with_caps": 0, 2024-04-24T09:18:16.408 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": 0, 2024-04-24T09:18:16.408 INFO:teuthology.orchestra.run.smithi083.stdout: "subtrees": 1, 2024-04-24T09:18:16.408 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse": 0, 2024-04-24T09:18:16.408 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_hit": 0, 2024-04-24T09:18:16.408 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_forward": 0, 2024-04-24T09:18:16.408 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_discover": 0, 2024-04-24T09:18:16.408 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_dir_fetch": 0, 2024-04-24T09:18:16.408 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_remote_ino": 0, 2024-04-24T09:18:16.409 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_lock": 0, 2024-04-24T09:18:16.409 INFO:teuthology.orchestra.run.smithi083.stdout: "load_cent": 0, 2024-04-24T09:18:16.409 INFO:teuthology.orchestra.run.smithi083.stdout: "q": 0, 2024-04-24T09:18:16.409 INFO:teuthology.orchestra.run.smithi083.stdout: "exported": 0, 2024-04-24T09:18:16.409 INFO:teuthology.orchestra.run.smithi083.stdout: "exported_inodes": 0, 2024-04-24T09:18:16.409 INFO:teuthology.orchestra.run.smithi083.stdout: "imported": 0, 2024-04-24T09:18:16.409 INFO:teuthology.orchestra.run.smithi083.stdout: "imported_inodes": 0, 2024-04-24T09:18:16.409 INFO:teuthology.orchestra.run.smithi083.stdout: "openino_dir_fetch": 0, 2024-04-24T09:18:16.409 INFO:teuthology.orchestra.run.smithi083.stdout: "openino_backtrace_fetch": 0, 2024-04-24T09:18:16.409 INFO:teuthology.orchestra.run.smithi083.stdout: "openino_peer_discover": 0, 2024-04-24T09:18:16.409 INFO:teuthology.orchestra.run.smithi083.stdout: "root_rfiles": 0, 2024-04-24T09:18:16.409 INFO:teuthology.orchestra.run.smithi083.stdout: "root_rbytes": 0, 2024-04-24T09:18:16.409 INFO:teuthology.orchestra.run.smithi083.stdout: "root_rsnaps": 0, 2024-04-24T09:18:16.409 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_backtrace_fetch": 0, 2024-04-24T09:18:16.410 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_set_tag": 0, 2024-04-24T09:18:16.410 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_backtrace_repaired": 0, 2024-04-24T09:18:16.410 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_inotable_repaired": 0, 2024-04-24T09:18:16.410 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_dir_inodes": 0, 2024-04-24T09:18:16.410 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_dir_base_inodes": 0, 2024-04-24T09:18:16.410 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_dirfrag_rstats": 0, 2024-04-24T09:18:16.410 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_file_inodes": 0, 2024-04-24T09:18:16.410 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_inode_file_caps": 0, 2024-04-24T09:18:16.410 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_revoke": 0, 2024-04-24T09:18:16.410 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_grant": 0, 2024-04-24T09:18:16.410 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_trunc": 0, 2024-04-24T09:18:16.410 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-24T09:18:16.411 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-24T09:18:16.411 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_caps": 0, 2024-04-24T09:18:16.411 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_caps_dirty": 0, 2024-04-24T09:18:16.411 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_cap_release": 0, 2024-04-24T09:18:16.411 INFO:teuthology.orchestra.run.smithi083.stdout: "process_request_cap_release": 0 2024-04-24T09:18:16.411 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:16.411 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_cache": { 2024-04-24T09:18:16.411 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_update": 0, 2024-04-24T09:18:16.411 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_update_receipt": 0, 2024-04-24T09:18:16.411 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_try_discover": 0, 2024-04-24T09:18:16.411 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_send_discover": 0, 2024-04-24T09:18:16.411 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_handle_discover": 0, 2024-04-24T09:18:16.411 INFO:teuthology.orchestra.run.smithi083.stdout: "num_strays": 0, 2024-04-24T09:18:16.411 INFO:teuthology.orchestra.run.smithi083.stdout: "num_strays_delayed": 0, 2024-04-24T09:18:16.412 INFO:teuthology.orchestra.run.smithi083.stdout: "num_strays_enqueuing": 0, 2024-04-24T09:18:16.412 INFO:teuthology.orchestra.run.smithi083.stdout: "strays_created": 0, 2024-04-24T09:18:16.412 INFO:teuthology.orchestra.run.smithi083.stdout: "strays_enqueued": 0, 2024-04-24T09:18:16.412 INFO:teuthology.orchestra.run.smithi083.stdout: "strays_reintegrated": 0, 2024-04-24T09:18:16.412 INFO:teuthology.orchestra.run.smithi083.stdout: "strays_migrated": 0, 2024-04-24T09:18:16.412 INFO:teuthology.orchestra.run.smithi083.stdout: "num_recovering_processing": 0, 2024-04-24T09:18:16.412 INFO:teuthology.orchestra.run.smithi083.stdout: "num_recovering_enqueued": 0, 2024-04-24T09:18:16.412 INFO:teuthology.orchestra.run.smithi083.stdout: "num_recovering_prioritized": 0, 2024-04-24T09:18:16.412 INFO:teuthology.orchestra.run.smithi083.stdout: "recovery_started": 0, 2024-04-24T09:18:16.412 INFO:teuthology.orchestra.run.smithi083.stdout: "recovery_completed": 0, 2024-04-24T09:18:16.412 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_quiesce_path": 0, 2024-04-24T09:18:16.412 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_quiesce_inode": 0, 2024-04-24T09:18:16.412 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_enqueue_scrub": 0, 2024-04-24T09:18:16.413 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_exportdir": 0, 2024-04-24T09:18:16.413 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_flush": 0, 2024-04-24T09:18:16.413 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_fragmentdir": 0, 2024-04-24T09:18:16.413 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_fragstats": 0, 2024-04-24T09:18:16.413 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_inodestats": 0 2024-04-24T09:18:16.413 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:16.413 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_log": { 2024-04-24T09:18:16.413 INFO:teuthology.orchestra.run.smithi083.stdout: "evlrg": 0, 2024-04-24T09:18:16.413 INFO:teuthology.orchestra.run.smithi083.stdout: "evadd": 0, 2024-04-24T09:18:16.413 INFO:teuthology.orchestra.run.smithi083.stdout: "evex": 0, 2024-04-24T09:18:16.413 INFO:teuthology.orchestra.run.smithi083.stdout: "evtrm": 0, 2024-04-24T09:18:16.413 INFO:teuthology.orchestra.run.smithi083.stdout: "ev": 23382, 2024-04-24T09:18:16.413 INFO:teuthology.orchestra.run.smithi083.stdout: "evexg": 0, 2024-04-24T09:18:16.413 INFO:teuthology.orchestra.run.smithi083.stdout: "evexd": 0, 2024-04-24T09:18:16.413 INFO:teuthology.orchestra.run.smithi083.stdout: "segadd": 0, 2024-04-24T09:18:16.413 INFO:teuthology.orchestra.run.smithi083.stdout: "segex": 0, 2024-04-24T09:18:16.413 INFO:teuthology.orchestra.run.smithi083.stdout: "segtrm": 0, 2024-04-24T09:18:16.414 INFO:teuthology.orchestra.run.smithi083.stdout: "seg": 50, 2024-04-24T09:18:16.414 INFO:teuthology.orchestra.run.smithi083.stdout: "segmjr": 3, 2024-04-24T09:18:16.414 INFO:teuthology.orchestra.run.smithi083.stdout: "segexg": 0, 2024-04-24T09:18:16.414 INFO:teuthology.orchestra.run.smithi083.stdout: "segexd": 0, 2024-04-24T09:18:16.414 INFO:teuthology.orchestra.run.smithi083.stdout: "expos": 4194304, 2024-04-24T09:18:16.414 INFO:teuthology.orchestra.run.smithi083.stdout: "wrpos": 192759999, 2024-04-24T09:18:16.414 INFO:teuthology.orchestra.run.smithi083.stdout: "rdpos": 192752191, 2024-04-24T09:18:16.414 INFO:teuthology.orchestra.run.smithi083.stdout: "jlat": { 2024-04-24T09:18:16.414 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-24T09:18:16.414 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-24T09:18:16.414 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-24T09:18:16.414 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:16.414 INFO:teuthology.orchestra.run.smithi083.stdout: "replayed": 23382 2024-04-24T09:18:16.414 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:16.414 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_mem": { 2024-04-24T09:18:16.414 INFO:teuthology.orchestra.run.smithi083.stdout: "ino": 2413, 2024-04-24T09:18:16.415 INFO:teuthology.orchestra.run.smithi083.stdout: "ino+": 4058, 2024-04-24T09:18:16.415 INFO:teuthology.orchestra.run.smithi083.stdout: "ino-": 1645, 2024-04-24T09:18:16.415 INFO:teuthology.orchestra.run.smithi083.stdout: "dir": 83, 2024-04-24T09:18:16.415 INFO:teuthology.orchestra.run.smithi083.stdout: "dir+": 105, 2024-04-24T09:18:16.415 INFO:teuthology.orchestra.run.smithi083.stdout: "dir-": 22, 2024-04-24T09:18:16.415 INFO:teuthology.orchestra.run.smithi083.stdout: "dn": 3884, 2024-04-24T09:18:16.415 INFO:teuthology.orchestra.run.smithi083.stdout: "dn+": 4645, 2024-04-24T09:18:16.415 INFO:teuthology.orchestra.run.smithi083.stdout: "dn-": 761, 2024-04-24T09:18:16.415 INFO:teuthology.orchestra.run.smithi083.stdout: "cap": 0, 2024-04-24T09:18:16.415 INFO:teuthology.orchestra.run.smithi083.stdout: "cap+": 0, 2024-04-24T09:18:16.415 INFO:teuthology.orchestra.run.smithi083.stdout: "cap-": 0, 2024-04-24T09:18:16.415 INFO:teuthology.orchestra.run.smithi083.stdout: "rss": 62392, 2024-04-24T09:18:16.415 INFO:teuthology.orchestra.run.smithi083.stdout: "heap": 215416 2024-04-24T09:18:16.415 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:16.415 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_server": { 2024-04-24T09:18:16.415 INFO:teuthology.orchestra.run.smithi083.stdout: "dispatch_client_request": 0, 2024-04-24T09:18:16.416 INFO:teuthology.orchestra.run.smithi083.stdout: "dispatch_server_request": 0, 2024-04-24T09:18:16.416 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_request": 0, 2024-04-24T09:18:16.416 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_session": 0, 2024-04-24T09:18:16.416 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_peer_request": 0, 2024-04-24T09:18:16.416 INFO:teuthology.orchestra.run.smithi083.stdout: "req_create_latency": { 2024-04-24T09:18:16.416 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-24T09:18:16.416 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-24T09:18:16.416 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-24T09:18:16.416 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:16.416 INFO:teuthology.orchestra.run.smithi083.stdout: "req_getattr_latency": { 2024-04-24T09:18:16.416 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-24T09:18:16.416 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-24T09:18:16.416 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-24T09:18:16.416 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:16.416 INFO:teuthology.orchestra.run.smithi083.stdout: "req_getfilelock_latency": { 2024-04-24T09:18:16.417 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-24T09:18:16.417 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-24T09:18:16.417 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-24T09:18:16.417 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:16.417 INFO:teuthology.orchestra.run.smithi083.stdout: "req_link_latency": { 2024-04-24T09:18:16.417 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-24T09:18:16.417 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-24T09:18:16.417 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-24T09:18:16.417 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:16.417 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookup_latency": { 2024-04-24T09:18:16.417 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-24T09:18:16.417 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-24T09:18:16.417 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-24T09:18:16.417 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:16.417 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookuphash_latency": { 2024-04-24T09:18:16.417 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-24T09:18:16.418 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-24T09:18:16.418 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-24T09:18:16.418 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:16.418 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookupino_latency": { 2024-04-24T09:18:16.418 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-24T09:18:16.418 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-24T09:18:16.418 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-24T09:18:16.418 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:16.418 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookupname_latency": { 2024-04-24T09:18:16.418 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-24T09:18:16.418 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-24T09:18:16.418 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-24T09:18:16.418 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:16.418 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookupparent_latency": { 2024-04-24T09:18:16.418 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-24T09:18:16.418 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-24T09:18:16.419 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-24T09:18:16.419 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:16.419 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookupsnap_latency": { 2024-04-24T09:18:16.419 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-24T09:18:16.419 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-24T09:18:16.419 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-24T09:18:16.419 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:16.419 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lssnap_latency": { 2024-04-24T09:18:16.419 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-24T09:18:16.419 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-24T09:18:16.419 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-24T09:18:16.419 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:16.420 INFO:teuthology.orchestra.run.smithi083.stdout: "req_mkdir_latency": { 2024-04-24T09:18:16.420 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-24T09:18:16.420 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-24T09:18:16.420 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-24T09:18:16.420 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:16.420 INFO:teuthology.orchestra.run.smithi083.stdout: "req_mknod_latency": { 2024-04-24T09:18:16.420 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-24T09:18:16.420 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-24T09:18:16.420 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-24T09:18:16.420 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:16.420 INFO:teuthology.orchestra.run.smithi083.stdout: "req_mksnap_latency": { 2024-04-24T09:18:16.421 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-24T09:18:16.421 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-24T09:18:16.421 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-24T09:18:16.421 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:16.421 INFO:teuthology.orchestra.run.smithi083.stdout: "req_open_latency": { 2024-04-24T09:18:16.421 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-24T09:18:16.421 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-24T09:18:16.421 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-24T09:18:16.421 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:16.421 INFO:teuthology.orchestra.run.smithi083.stdout: "req_readdir_latency": { 2024-04-24T09:18:16.421 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-24T09:18:16.421 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-24T09:18:16.421 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-24T09:18:16.421 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:16.421 INFO:teuthology.orchestra.run.smithi083.stdout: "req_rename_latency": { 2024-04-24T09:18:16.422 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-24T09:18:16.422 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-24T09:18:16.422 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-24T09:18:16.422 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:16.422 INFO:teuthology.orchestra.run.smithi083.stdout: "req_renamesnap_latency": { 2024-04-24T09:18:16.422 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-24T09:18:16.422 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-24T09:18:16.422 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-24T09:18:16.422 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:16.422 INFO:teuthology.orchestra.run.smithi083.stdout: "req_snapdiff_latency": { 2024-04-24T09:18:16.422 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-24T09:18:16.422 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-24T09:18:16.422 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-24T09:18:16.422 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:16.423 INFO:teuthology.orchestra.run.smithi083.stdout: "req_rmdir_latency": { 2024-04-24T09:18:16.423 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-24T09:18:16.423 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-24T09:18:16.423 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-24T09:18:16.423 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:16.423 INFO:teuthology.orchestra.run.smithi083.stdout: "req_rmsnap_latency": { 2024-04-24T09:18:16.423 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-24T09:18:16.423 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-24T09:18:16.423 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-24T09:18:16.423 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:16.423 INFO:teuthology.orchestra.run.smithi083.stdout: "req_rmxattr_latency": { 2024-04-24T09:18:16.423 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-24T09:18:16.423 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-24T09:18:16.423 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-24T09:18:16.423 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:16.424 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setattr_latency": { 2024-04-24T09:18:16.424 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-24T09:18:16.424 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-24T09:18:16.424 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-24T09:18:16.424 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:16.424 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setdirlayout_latency": { 2024-04-24T09:18:16.424 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-24T09:18:16.424 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-24T09:18:16.424 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-24T09:18:16.424 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:16.424 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setfilelock_latency": { 2024-04-24T09:18:16.424 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-24T09:18:16.424 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-24T09:18:16.424 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-24T09:18:16.424 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:16.425 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setlayout_latency": { 2024-04-24T09:18:16.425 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-24T09:18:16.425 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-24T09:18:16.425 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-24T09:18:16.425 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:16.425 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setxattr_latency": { 2024-04-24T09:18:16.425 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-24T09:18:16.425 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-24T09:18:16.425 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-24T09:18:16.425 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:16.425 INFO:teuthology.orchestra.run.smithi083.stdout: "req_symlink_latency": { 2024-04-24T09:18:16.425 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-24T09:18:16.425 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-24T09:18:16.425 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-24T09:18:16.425 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:16.426 INFO:teuthology.orchestra.run.smithi083.stdout: "req_unlink_latency": { 2024-04-24T09:18:16.426 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-24T09:18:16.426 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-24T09:18:16.426 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-24T09:18:16.426 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:16.426 INFO:teuthology.orchestra.run.smithi083.stdout: "cap_revoke_eviction": 0, 2024-04-24T09:18:16.426 INFO:teuthology.orchestra.run.smithi083.stdout: "cap_acquisition_throttle": 0, 2024-04-24T09:18:16.426 INFO:teuthology.orchestra.run.smithi083.stdout: "req_getvxattr_latency": { 2024-04-24T09:18:16.426 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-24T09:18:16.426 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-24T09:18:16.426 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-24T09:18:16.426 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:16.426 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:16.426 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_sessions": { 2024-04-24T09:18:16.427 INFO:teuthology.orchestra.run.smithi083.stdout: "session_count": 1, 2024-04-24T09:18:16.427 INFO:teuthology.orchestra.run.smithi083.stdout: "session_add": 2, 2024-04-24T09:18:16.427 INFO:teuthology.orchestra.run.smithi083.stdout: "session_remove": 1, 2024-04-24T09:18:16.427 INFO:teuthology.orchestra.run.smithi083.stdout: "sessions_open": 2, 2024-04-24T09:18:16.427 INFO:teuthology.orchestra.run.smithi083.stdout: "sessions_stale": 0, 2024-04-24T09:18:16.427 INFO:teuthology.orchestra.run.smithi083.stdout: "total_load": 0, 2024-04-24T09:18:16.427 INFO:teuthology.orchestra.run.smithi083.stdout: "average_load": 0, 2024-04-24T09:18:16.427 INFO:teuthology.orchestra.run.smithi083.stdout: "avg_session_uptime": 4439, 2024-04-24T09:18:16.427 INFO:teuthology.orchestra.run.smithi083.stdout: "mdthresh_evicted": 0 2024-04-24T09:18:16.427 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:16.427 INFO:teuthology.orchestra.run.smithi083.stdout: "mempool": { 2024-04-24T09:18:16.427 INFO:teuthology.orchestra.run.smithi083.stdout: "bloom_filter_bytes": 0, 2024-04-24T09:18:16.427 INFO:teuthology.orchestra.run.smithi083.stdout: "bloom_filter_items": 0, 2024-04-24T09:18:16.427 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_alloc_bytes": 0, 2024-04-24T09:18:16.427 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_alloc_items": 0, 2024-04-24T09:18:16.428 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_data_bytes": 0, 2024-04-24T09:18:16.428 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_data_items": 0, 2024-04-24T09:18:16.428 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-24T09:18:16.428 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_onode_items": 0, 2024-04-24T09:18:16.428 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-24T09:18:16.428 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_meta_items": 0, 2024-04-24T09:18:16.428 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_other_bytes": 0, 2024-04-24T09:18:16.428 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_other_items": 0, 2024-04-24T09:18:16.428 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-24T09:18:16.428 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_buffer_items": 0, 2024-04-24T09:18:16.428 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_extent_bytes": 0, 2024-04-24T09:18:16.428 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_extent_items": 0, 2024-04-24T09:18:16.428 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_blob_bytes": 0, 2024-04-24T09:18:16.428 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_blob_items": 0, 2024-04-24T09:18:16.429 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-24T09:18:16.429 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_shared_blob_items": 0, 2024-04-24T09:18:16.429 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-24T09:18:16.429 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_inline_bl_items": 0, 2024-04-24T09:18:16.429 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_bytes": 0, 2024-04-24T09:18:16.429 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_items": 0, 2024-04-24T09:18:16.429 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_txc_bytes": 0, 2024-04-24T09:18:16.429 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_txc_items": 0, 2024-04-24T09:18:16.429 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-24T09:18:16.429 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_deferred_items": 0, 2024-04-24T09:18:16.429 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_bytes": 0, 2024-04-24T09:18:16.429 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_items": 0, 2024-04-24T09:18:16.429 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_bytes": 0, 2024-04-24T09:18:16.429 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_items": 0, 2024-04-24T09:18:16.429 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_reader_bytes": 0, 2024-04-24T09:18:16.430 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_reader_items": 0, 2024-04-24T09:18:16.430 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_writer_bytes": 0, 2024-04-24T09:18:16.430 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_writer_items": 0, 2024-04-24T09:18:16.430 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_anon_bytes": 241073, 2024-04-24T09:18:16.430 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_anon_items": 84, 2024-04-24T09:18:16.430 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_meta_bytes": 0, 2024-04-24T09:18:16.430 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_meta_items": 0, 2024-04-24T09:18:16.430 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_bytes": 0, 2024-04-24T09:18:16.430 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_items": 0, 2024-04-24T09:18:16.430 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mapbl_bytes": 0, 2024-04-24T09:18:16.430 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mapbl_items": 0, 2024-04-24T09:18:16.430 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pglog_bytes": 0, 2024-04-24T09:18:16.430 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pglog_items": 0, 2024-04-24T09:18:16.430 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_bytes": 10040, 2024-04-24T09:18:16.430 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_items": 194, 2024-04-24T09:18:16.431 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_mapping_bytes": 0, 2024-04-24T09:18:16.431 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_mapping_items": 0, 2024-04-24T09:18:16.431 INFO:teuthology.orchestra.run.smithi083.stdout: "pgmap_bytes": 0, 2024-04-24T09:18:16.431 INFO:teuthology.orchestra.run.smithi083.stdout: "pgmap_items": 0, 2024-04-24T09:18:16.431 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_co_bytes": 8112401, 2024-04-24T09:18:16.431 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_co_items": 24107, 2024-04-24T09:18:16.431 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_1_bytes": 0, 2024-04-24T09:18:16.431 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_1_items": 0, 2024-04-24T09:18:16.431 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_2_bytes": 0, 2024-04-24T09:18:16.431 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_2_items": 0 2024-04-24T09:18:16.431 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:16.431 INFO:teuthology.orchestra.run.smithi083.stdout: "objecter": { 2024-04-24T09:18:16.431 INFO:teuthology.orchestra.run.smithi083.stdout: "op_active": 0, 2024-04-24T09:18:16.431 INFO:teuthology.orchestra.run.smithi083.stdout: "op_laggy": 0, 2024-04-24T09:18:16.432 INFO:teuthology.orchestra.run.smithi083.stdout: "op_send": 5853, 2024-04-24T09:18:16.432 INFO:teuthology.orchestra.run.smithi083.stdout: "op_send_bytes": 21, 2024-04-24T09:18:16.432 INFO:teuthology.orchestra.run.smithi083.stdout: "op_resend": 3, 2024-04-24T09:18:16.432 INFO:teuthology.orchestra.run.smithi083.stdout: "op_reply": 5853, 2024-04-24T09:18:16.432 INFO:teuthology.orchestra.run.smithi083.stdout: "op_latency": { 2024-04-24T09:18:16.432 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 5853, 2024-04-24T09:18:16.432 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 9.791769787, 2024-04-24T09:18:16.432 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.001672948 2024-04-24T09:18:16.432 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:16.432 INFO:teuthology.orchestra.run.smithi083.stdout: "op_inflight": 0, 2024-04-24T09:18:16.432 INFO:teuthology.orchestra.run.smithi083.stdout: "oplen_avg": { 2024-04-24T09:18:16.432 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 5853, 2024-04-24T09:18:16.432 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 5854 2024-04-24T09:18:16.432 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:16.432 INFO:teuthology.orchestra.run.smithi083.stdout: "op": 5853, 2024-04-24T09:18:16.433 INFO:teuthology.orchestra.run.smithi083.stdout: "op_r": 5853, 2024-04-24T09:18:16.433 INFO:teuthology.orchestra.run.smithi083.stdout: "op_w": 0, 2024-04-24T09:18:16.433 INFO:teuthology.orchestra.run.smithi083.stdout: "op_rmw": 0, 2024-04-24T09:18:16.433 INFO:teuthology.orchestra.run.smithi083.stdout: "op_pg": 0, 2024-04-24T09:18:16.433 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_stat": 3351, 2024-04-24T09:18:16.433 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_create": 0, 2024-04-24T09:18:16.433 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_read": 2500, 2024-04-24T09:18:16.433 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_write": 0, 2024-04-24T09:18:16.433 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_writefull": 0, 2024-04-24T09:18:16.433 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_writesame": 0, 2024-04-24T09:18:16.433 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_append": 0, 2024-04-24T09:18:16.433 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_zero": 0, 2024-04-24T09:18:16.433 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_truncate": 0, 2024-04-24T09:18:16.433 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_delete": 0, 2024-04-24T09:18:16.433 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_mapext": 0, 2024-04-24T09:18:16.434 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_sparse_read": 0, 2024-04-24T09:18:16.434 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_clonerange": 0, 2024-04-24T09:18:16.434 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_getxattr": 1, 2024-04-24T09:18:16.434 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_setxattr": 0, 2024-04-24T09:18:16.434 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_cmpxattr": 0, 2024-04-24T09:18:16.434 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_rmxattr": 0, 2024-04-24T09:18:16.434 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_resetxattrs": 0, 2024-04-24T09:18:16.434 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_call": 0, 2024-04-24T09:18:16.434 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_watch": 0, 2024-04-24T09:18:16.434 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_notify": 0, 2024-04-24T09:18:16.434 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_src_cmpxattr": 0, 2024-04-24T09:18:16.434 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_pgls": 0, 2024-04-24T09:18:16.434 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_pgls_filter": 0, 2024-04-24T09:18:16.434 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_other": 0, 2024-04-24T09:18:16.434 INFO:teuthology.orchestra.run.smithi083.stdout: "linger_active": 0, 2024-04-24T09:18:16.434 INFO:teuthology.orchestra.run.smithi083.stdout: "linger_send": 0, 2024-04-24T09:18:16.435 INFO:teuthology.orchestra.run.smithi083.stdout: "linger_resend": 0, 2024-04-24T09:18:16.435 INFO:teuthology.orchestra.run.smithi083.stdout: "linger_ping": 0, 2024-04-24T09:18:16.435 INFO:teuthology.orchestra.run.smithi083.stdout: "poolop_active": 0, 2024-04-24T09:18:16.435 INFO:teuthology.orchestra.run.smithi083.stdout: "poolop_send": 0, 2024-04-24T09:18:16.435 INFO:teuthology.orchestra.run.smithi083.stdout: "poolop_resend": 0, 2024-04-24T09:18:16.435 INFO:teuthology.orchestra.run.smithi083.stdout: "poolstat_active": 0, 2024-04-24T09:18:16.435 INFO:teuthology.orchestra.run.smithi083.stdout: "poolstat_send": 0, 2024-04-24T09:18:16.435 INFO:teuthology.orchestra.run.smithi083.stdout: "poolstat_resend": 0, 2024-04-24T09:18:16.435 INFO:teuthology.orchestra.run.smithi083.stdout: "statfs_active": 0, 2024-04-24T09:18:16.435 INFO:teuthology.orchestra.run.smithi083.stdout: "statfs_send": 0, 2024-04-24T09:18:16.435 INFO:teuthology.orchestra.run.smithi083.stdout: "statfs_resend": 0, 2024-04-24T09:18:16.435 INFO:teuthology.orchestra.run.smithi083.stdout: "command_active": 0, 2024-04-24T09:18:16.435 INFO:teuthology.orchestra.run.smithi083.stdout: "command_send": 0, 2024-04-24T09:18:16.435 INFO:teuthology.orchestra.run.smithi083.stdout: "command_resend": 0, 2024-04-24T09:18:16.435 INFO:teuthology.orchestra.run.smithi083.stdout: "map_epoch": 136, 2024-04-24T09:18:16.435 INFO:teuthology.orchestra.run.smithi083.stdout: "map_full": 0, 2024-04-24T09:18:16.436 INFO:teuthology.orchestra.run.smithi083.stdout: "map_inc": 47, 2024-04-24T09:18:16.436 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_sessions": 11, 2024-04-24T09:18:16.436 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_session_open": 13, 2024-04-24T09:18:16.436 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_session_close": 2, 2024-04-24T09:18:16.436 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_laggy": 0, 2024-04-24T09:18:16.436 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_wr": 0, 2024-04-24T09:18:16.436 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_rd": 2, 2024-04-24T09:18:16.436 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_del": 0 2024-04-24T09:18:16.436 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:16.436 INFO:teuthology.orchestra.run.smithi083.stdout: "oft": { 2024-04-24T09:18:16.436 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_total_objs": 0, 2024-04-24T09:18:16.436 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_total_kv_pairs": 0, 2024-04-24T09:18:16.436 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_total_updates": 0, 2024-04-24T09:18:16.436 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_total_removes": 0 2024-04-24T09:18:16.436 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:16.437 INFO:teuthology.orchestra.run.smithi083.stdout: "osd-slow-ops": { 2024-04-24T09:18:16.437 INFO:teuthology.orchestra.run.smithi083.stdout: "slow_ops_count": 0 2024-04-24T09:18:16.437 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:16.437 INFO:teuthology.orchestra.run.smithi083.stdout: "purge_queue": { 2024-04-24T09:18:16.437 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executing_ops": 0, 2024-04-24T09:18:16.437 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executing_ops_high_water": 0, 2024-04-24T09:18:16.437 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executing": 0, 2024-04-24T09:18:16.437 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executing_high_water": 0, 2024-04-24T09:18:16.437 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executed_ops": 0, 2024-04-24T09:18:16.437 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executed": 0, 2024-04-24T09:18:16.437 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_item_in_journal": 0 2024-04-24T09:18:16.437 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:16.437 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-24T09:18:16.437 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-04-24T09:18:16.437 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 104857600, 2024-04-24T09:18:16.437 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-04-24T09:18:16.438 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 6448, 2024-04-24T09:18:16.438 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 191660334, 2024-04-24T09:18:16.438 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-04-24T09:18:16.438 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 6448, 2024-04-24T09:18:16.438 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 0, 2024-04-24T09:18:16.438 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 0, 2024-04-24T09:18:16.438 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 6448, 2024-04-24T09:18:16.438 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 191660334, 2024-04-24T09:18:16.438 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-04-24T09:18:16.438 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-24T09:18:16.438 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-24T09:18:16.438 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-24T09:18:16.438 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:16.438 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:16.438 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-objecter_bytes": { 2024-04-24T09:18:16.438 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-04-24T09:18:16.439 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 104857600, 2024-04-24T09:18:16.439 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-04-24T09:18:16.439 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 0, 2024-04-24T09:18:16.439 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 0, 2024-04-24T09:18:16.439 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-04-24T09:18:16.439 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 0, 2024-04-24T09:18:16.439 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 5853, 2024-04-24T09:18:16.439 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 188565700, 2024-04-24T09:18:16.439 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 5853, 2024-04-24T09:18:16.439 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 188565700, 2024-04-24T09:18:16.439 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-04-24T09:18:16.439 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-24T09:18:16.439 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-24T09:18:16.439 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-24T09:18:16.440 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:16.440 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:16.440 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-objecter_ops": { 2024-04-24T09:18:16.440 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-04-24T09:18:16.440 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 1024, 2024-04-24T09:18:16.440 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-04-24T09:18:16.440 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 0, 2024-04-24T09:18:16.440 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 0, 2024-04-24T09:18:16.440 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-04-24T09:18:16.440 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 0, 2024-04-24T09:18:16.440 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 5853, 2024-04-24T09:18:16.440 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 5853, 2024-04-24T09:18:16.440 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 5853, 2024-04-24T09:18:16.440 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 5853, 2024-04-24T09:18:16.441 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-04-24T09:18:16.441 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-24T09:18:16.441 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-24T09:18:16.441 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-24T09:18:16.441 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:16.441 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:16.441 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-write_buf_throttle": { 2024-04-24T09:18:16.441 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-04-24T09:18:16.441 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 3758096384, 2024-04-24T09:18:16.441 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-04-24T09:18:16.441 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 0, 2024-04-24T09:18:16.442 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 0, 2024-04-24T09:18:16.442 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-04-24T09:18:16.442 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 0, 2024-04-24T09:18:16.442 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 0, 2024-04-24T09:18:16.442 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 0, 2024-04-24T09:18:16.442 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 0, 2024-04-24T09:18:16.442 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 0, 2024-04-24T09:18:16.442 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-04-24T09:18:16.442 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-24T09:18:16.442 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-24T09:18:16.442 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-24T09:18:16.442 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:16.442 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:16.442 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-write_buf_throttle-0x564496780780": { 2024-04-24T09:18:16.443 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-04-24T09:18:16.443 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 3758096384, 2024-04-24T09:18:16.443 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-04-24T09:18:16.443 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 0, 2024-04-24T09:18:16.443 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 0, 2024-04-24T09:18:16.443 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-04-24T09:18:16.443 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 0, 2024-04-24T09:18:16.443 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 0, 2024-04-24T09:18:16.443 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 0, 2024-04-24T09:18:16.443 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 0, 2024-04-24T09:18:16.443 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 0, 2024-04-24T09:18:16.443 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-04-24T09:18:16.443 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-24T09:18:16.443 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-24T09:18:16.443 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-24T09:18:16.443 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:16.444 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:16.444 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T09:18:17.407 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-04-24T09:18:17.407 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-04-24T09:18:17.407 INFO:tasks.check_counter:Daemon mds.g mds.root_rsnaps=0 2024-04-24T09:18:17.408 INFO:tasks.check_counter:Daemon mds.g mds_server.req_mksnap_latency.avgcount=0 2024-04-24T09:18:17.408 INFO:tasks.check_counter:Daemon mds.g mds_server.req_rmsnap_latency.avgcount=0 2024-04-24T09:18:17.408 DEBUG:tasks.check_counter:Getting stats from j 2024-04-24T09:18:17.408 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid bd3e039e-0215-11ef-bc93-c7b262605968 -- ceph daemon mds.j perf dump 2024-04-24T09:18:17.860 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T09:18:18.258 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:18:18 smithi083 ceph-mon[27907]: pgmap v845: 129 pgs: 129 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-04-24T09:18:18.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:18:18 smithi138 ceph-mon[31441]: pgmap v845: 129 pgs: 129 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-04-24T09:18:18.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:18:18 smithi189 ceph-mon[31791]: pgmap v845: 129 pgs: 129 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-04-24T09:18:20.202 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:18:20 smithi083 ceph-mon[27907]: pgmap v846: 129 pgs: 129 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-24T09:18:20.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:18:20 smithi138 ceph-mon[31441]: pgmap v846: 129 pgs: 129 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-24T09:18:20.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:18:20 smithi189 ceph-mon[31791]: pgmap v846: 129 pgs: 129 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-24T09:18:20.786 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-24T09:18:20.786 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-0": { 2024-04-24T09:18:20.786 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 0, 2024-04-24T09:18:20.786 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 2, 2024-04-24T09:18:20.786 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 0, 2024-04-24T09:18:20.786 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 0, 2024-04-24T09:18:20.786 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 4, 2024-04-24T09:18:20.786 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 0, 2024-04-24T09:18:20.786 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.003115004, 2024-04-24T09:18:20.786 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.000000000, 2024-04-24T09:18:20.786 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.000352747, 2024-04-24T09:18:20.787 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-24T09:18:20.787 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T09:18:20.787 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-24T09:18:20.787 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-24T09:18:20.787 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-24T09:18:20.787 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:20.787 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-04-24T09:18:20.787 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-24T09:18:20.787 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-24T09:18:20.787 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-24T09:18:20.787 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:20.787 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-24T09:18:20.787 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 0 2024-04-24T09:18:20.787 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:20.787 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-1": { 2024-04-24T09:18:20.787 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 501, 2024-04-24T09:18:20.788 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 488, 2024-04-24T09:18:20.788 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 1671648, 2024-04-24T09:18:20.788 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 301376, 2024-04-24T09:18:20.788 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 2, 2024-04-24T09:18:20.788 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 1, 2024-04-24T09:18:20.788 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.161890139, 2024-04-24T09:18:20.788 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.065964579, 2024-04-24T09:18:20.788 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.089539702, 2024-04-24T09:18:20.788 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.019978756, 2024-04-24T09:18:20.788 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T09:18:20.788 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 488, 2024-04-24T09:18:20.788 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.043707600, 2024-04-24T09:18:20.788 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000089564 2024-04-24T09:18:20.788 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:20.788 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-04-24T09:18:20.789 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-24T09:18:20.789 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-24T09:18:20.789 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-24T09:18:20.789 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:20.789 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 1671648, 2024-04-24T09:18:20.789 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 301376 2024-04-24T09:18:20.789 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:20.789 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-2": { 2024-04-24T09:18:20.789 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 3, 2024-04-24T09:18:20.789 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 372, 2024-04-24T09:18:20.789 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 480, 2024-04-24T09:18:20.789 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 299856, 2024-04-24T09:18:20.789 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 6, 2024-04-24T09:18:20.789 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 3, 2024-04-24T09:18:20.789 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.063292211, 2024-04-24T09:18:20.789 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.050094408, 2024-04-24T09:18:20.790 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.004984565, 2024-04-24T09:18:20.790 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-24T09:18:20.790 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T09:18:20.790 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 369, 2024-04-24T09:18:20.790 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.035697290, 2024-04-24T09:18:20.790 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000096740 2024-04-24T09:18:20.790 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:20.790 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-04-24T09:18:20.790 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-24T09:18:20.790 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-24T09:18:20.790 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-24T09:18:20.790 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:20.790 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 480, 2024-04-24T09:18:20.790 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 299856 2024-04-24T09:18:20.790 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:20.791 INFO:teuthology.orchestra.run.smithi083.stdout: "cct": { 2024-04-24T09:18:20.791 INFO:teuthology.orchestra.run.smithi083.stdout: "total_workers": 0, 2024-04-24T09:18:20.791 INFO:teuthology.orchestra.run.smithi083.stdout: "unhealthy_workers": 0 2024-04-24T09:18:20.791 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:20.791 INFO:teuthology.orchestra.run.smithi083.stdout: "mempool": { 2024-04-24T09:18:20.791 INFO:teuthology.orchestra.run.smithi083.stdout: "bloom_filter_bytes": 0, 2024-04-24T09:18:20.791 INFO:teuthology.orchestra.run.smithi083.stdout: "bloom_filter_items": 0, 2024-04-24T09:18:20.791 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_alloc_bytes": 0, 2024-04-24T09:18:20.791 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_alloc_items": 0, 2024-04-24T09:18:20.791 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_data_bytes": 0, 2024-04-24T09:18:20.791 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_data_items": 0, 2024-04-24T09:18:20.791 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-24T09:18:20.791 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_onode_items": 0, 2024-04-24T09:18:20.791 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-24T09:18:20.791 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_meta_items": 0, 2024-04-24T09:18:20.791 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_other_bytes": 0, 2024-04-24T09:18:20.792 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_other_items": 0, 2024-04-24T09:18:20.792 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-24T09:18:20.792 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_buffer_items": 0, 2024-04-24T09:18:20.792 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_extent_bytes": 0, 2024-04-24T09:18:20.792 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_extent_items": 0, 2024-04-24T09:18:20.792 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_blob_bytes": 0, 2024-04-24T09:18:20.792 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_blob_items": 0, 2024-04-24T09:18:20.792 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-24T09:18:20.792 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_shared_blob_items": 0, 2024-04-24T09:18:20.792 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-24T09:18:20.792 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_inline_bl_items": 0, 2024-04-24T09:18:20.792 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_bytes": 0, 2024-04-24T09:18:20.792 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_items": 0, 2024-04-24T09:18:20.792 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_txc_bytes": 0, 2024-04-24T09:18:20.792 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_txc_items": 0, 2024-04-24T09:18:20.792 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-24T09:18:20.792 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_deferred_items": 0, 2024-04-24T09:18:20.793 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_bytes": 0, 2024-04-24T09:18:20.793 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_items": 0, 2024-04-24T09:18:20.793 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_bytes": 0, 2024-04-24T09:18:20.793 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_items": 0, 2024-04-24T09:18:20.793 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_reader_bytes": 0, 2024-04-24T09:18:20.793 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_reader_items": 0, 2024-04-24T09:18:20.793 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_writer_bytes": 0, 2024-04-24T09:18:20.793 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_writer_items": 0, 2024-04-24T09:18:20.793 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_anon_bytes": 148732, 2024-04-24T09:18:20.793 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_anon_items": 31, 2024-04-24T09:18:20.793 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_meta_bytes": 0, 2024-04-24T09:18:20.793 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_meta_items": 0, 2024-04-24T09:18:20.793 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_bytes": 0, 2024-04-24T09:18:20.793 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_items": 0, 2024-04-24T09:18:20.793 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mapbl_bytes": 0, 2024-04-24T09:18:20.793 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mapbl_items": 0, 2024-04-24T09:18:20.793 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pglog_bytes": 0, 2024-04-24T09:18:20.794 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pglog_items": 0, 2024-04-24T09:18:20.794 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_bytes": 0, 2024-04-24T09:18:20.794 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_items": 0, 2024-04-24T09:18:20.794 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_mapping_bytes": 0, 2024-04-24T09:18:20.794 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_mapping_items": 0, 2024-04-24T09:18:20.794 INFO:teuthology.orchestra.run.smithi083.stdout: "pgmap_bytes": 0, 2024-04-24T09:18:20.794 INFO:teuthology.orchestra.run.smithi083.stdout: "pgmap_items": 0, 2024-04-24T09:18:20.794 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_co_bytes": 928, 2024-04-24T09:18:20.794 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_co_items": 2, 2024-04-24T09:18:20.794 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_1_bytes": 0, 2024-04-24T09:18:20.794 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_1_items": 0, 2024-04-24T09:18:20.794 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_2_bytes": 0, 2024-04-24T09:18:20.794 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_2_items": 0 2024-04-24T09:18:20.794 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-24T09:18:20.795 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-24T09:18:20.795 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-04-24T09:18:20.795 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 104857600, 2024-04-24T09:18:20.795 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-04-24T09:18:20.795 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 500, 2024-04-24T09:18:20.795 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 1598486, 2024-04-24T09:18:20.795 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-04-24T09:18:20.795 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 500, 2024-04-24T09:18:20.795 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 0, 2024-04-24T09:18:20.795 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 0, 2024-04-24T09:18:20.795 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 500, 2024-04-24T09:18:20.795 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 1598486, 2024-04-24T09:18:20.795 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-04-24T09:18:20.795 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-24T09:18:20.795 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-24T09:18:20.795 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-24T09:18:20.795 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:20.795 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-24T09:18:20.796 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-24T09:18:21.852 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-04-24T09:18:21.852 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-04-24T09:18:21.852 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-04-24T09:18:21.852 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-04-24T09:18:21.852 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-04-24T09:18:21.852 DEBUG:tasks.check_counter:Getting stats from b 2024-04-24T09:18:21.852 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid bd3e039e-0215-11ef-bc93-c7b262605968 -- ceph daemon mds.b perf dump 2024-04-24T09:18:22.442 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T09:18:22.454 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:18:22 smithi138 ceph-mon[31441]: pgmap v847: 129 pgs: 129 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-24T09:18:22.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:18:22 smithi189 ceph-mon[31791]: pgmap v847: 129 pgs: 129 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-24T09:18:22.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:18:22 smithi083 ceph-mon[27907]: pgmap v847: 129 pgs: 129 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-24T09:18:24.389 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:18:24 smithi138 ceph-mon[31441]: pgmap v848: 129 pgs: 129 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-04-24T09:18:24.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:18:24 smithi189 ceph-mon[31791]: pgmap v848: 129 pgs: 129 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-04-24T09:18:24.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:18:24 smithi083 ceph-mon[27907]: pgmap v848: 129 pgs: 129 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-04-24T09:18:25.475 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-04-24T09:18:25.475 INFO:teuthology.orchestra.run.smithi138.stdout: "AsyncMessenger::Worker-0": { 2024-04-24T09:18:25.475 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_messages": 117464, 2024-04-24T09:18:25.475 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages": 147671, 2024-04-24T09:18:25.475 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_bytes": 36791100, 2024-04-24T09:18:25.475 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_bytes": 150698036, 2024-04-24T09:18:25.476 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_created_connections": 13, 2024-04-24T09:18:25.476 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_active_connections": 4, 2024-04-24T09:18:25.476 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_total_time": 19.670400083, 2024-04-24T09:18:25.476 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_send_time": 7.618168408, 2024-04-24T09:18:25.476 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_recv_time": 8.909434817, 2024-04-24T09:18:25.476 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_fast_dispatch_time": 3.684791622, 2024-04-24T09:18:25.476 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T09:18:25.476 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 147669, 2024-04-24T09:18:25.476 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 20.931139012, 2024-04-24T09:18:25.476 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000141743 2024-04-24T09:18:25.476 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:25.476 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_handle_ack_lat": { 2024-04-24T09:18:25.476 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 205664, 2024-04-24T09:18:25.476 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.070125714, 2024-04-24T09:18:25.477 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000340 2024-04-24T09:18:25.477 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:25.477 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-24T09:18:25.477 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_encrypted_bytes": 641984 2024-04-24T09:18:25.477 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:25.477 INFO:teuthology.orchestra.run.smithi138.stdout: "AsyncMessenger::Worker-1": { 2024-04-24T09:18:25.477 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_messages": 546232, 2024-04-24T09:18:25.477 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages": 591880, 2024-04-24T09:18:25.477 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_bytes": 164235958, 2024-04-24T09:18:25.477 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_bytes": 487269731, 2024-04-24T09:18:25.477 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_created_connections": 25, 2024-04-24T09:18:25.477 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_active_connections": 5, 2024-04-24T09:18:25.477 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_total_time": 82.825721223, 2024-04-24T09:18:25.477 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_send_time": 32.421987265, 2024-04-24T09:18:25.478 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_recv_time": 51.301686636, 2024-04-24T09:18:25.478 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_fast_dispatch_time": 7.981413910, 2024-04-24T09:18:25.478 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T09:18:25.478 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 591878, 2024-04-24T09:18:25.478 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 50.779623325, 2024-04-24T09:18:25.478 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000085794 2024-04-24T09:18:25.478 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:25.478 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_handle_ack_lat": { 2024-04-24T09:18:25.478 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 1055436, 2024-04-24T09:18:25.478 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.359204261, 2024-04-24T09:18:25.478 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000340 2024-04-24T09:18:25.478 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:25.478 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_encrypted_bytes": 946352, 2024-04-24T09:18:25.479 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_encrypted_bytes": 446528 2024-04-24T09:18:25.479 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:25.479 INFO:teuthology.orchestra.run.smithi138.stdout: "AsyncMessenger::Worker-2": { 2024-04-24T09:18:25.479 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_messages": 108416, 2024-04-24T09:18:25.479 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages": 92767, 2024-04-24T09:18:25.479 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_bytes": 50821889, 2024-04-24T09:18:25.479 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_bytes": 82405467, 2024-04-24T09:18:25.479 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_created_connections": 603, 2024-04-24T09:18:25.479 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_active_connections": 7, 2024-04-24T09:18:25.479 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_total_time": 22.108636895, 2024-04-24T09:18:25.479 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_send_time": 10.678990367, 2024-04-24T09:18:25.479 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_recv_time": 11.145612478, 2024-04-24T09:18:25.479 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_fast_dispatch_time": 2.576154383, 2024-04-24T09:18:25.480 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T09:18:25.480 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 92766, 2024-04-24T09:18:25.480 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 14.033033456, 2024-04-24T09:18:25.480 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000151273 2024-04-24T09:18:25.480 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:25.480 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_handle_ack_lat": { 2024-04-24T09:18:25.480 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 164267, 2024-04-24T09:18:25.480 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.061314611, 2024-04-24T09:18:25.480 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000373 2024-04-24T09:18:25.480 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:25.480 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_encrypted_bytes": 2172032, 2024-04-24T09:18:25.480 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_encrypted_bytes": 389504 2024-04-24T09:18:25.481 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:25.481 INFO:teuthology.orchestra.run.smithi138.stdout: "cct": { 2024-04-24T09:18:25.481 INFO:teuthology.orchestra.run.smithi138.stdout: "total_workers": 1, 2024-04-24T09:18:25.481 INFO:teuthology.orchestra.run.smithi138.stdout: "unhealthy_workers": 0 2024-04-24T09:18:25.481 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:25.481 INFO:teuthology.orchestra.run.smithi138.stdout: "finisher-MDSRank": { 2024-04-24T09:18:25.481 INFO:teuthology.orchestra.run.smithi138.stdout: "queue_len": 0, 2024-04-24T09:18:25.481 INFO:teuthology.orchestra.run.smithi138.stdout: "complete_latency": { 2024-04-24T09:18:25.481 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 57769, 2024-04-24T09:18:25.481 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 73.657790104, 2024-04-24T09:18:25.481 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.001275040 2024-04-24T09:18:25.481 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-24T09:18:25.481 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:25.482 INFO:teuthology.orchestra.run.smithi138.stdout: "finisher-PurgeQueue": { 2024-04-24T09:18:25.482 INFO:teuthology.orchestra.run.smithi138.stdout: "queue_len": 0, 2024-04-24T09:18:25.482 INFO:teuthology.orchestra.run.smithi138.stdout: "complete_latency": { 2024-04-24T09:18:25.482 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 18896, 2024-04-24T09:18:25.482 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 2.810806638, 2024-04-24T09:18:25.482 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000148751 2024-04-24T09:18:25.482 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-24T09:18:25.482 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:25.482 INFO:teuthology.orchestra.run.smithi138.stdout: "mds": { 2024-04-24T09:18:25.482 INFO:teuthology.orchestra.run.smithi138.stdout: "request": 472589, 2024-04-24T09:18:25.482 INFO:teuthology.orchestra.run.smithi138.stdout: "reply": 457188, 2024-04-24T09:18:25.482 INFO:teuthology.orchestra.run.smithi138.stdout: "reply_latency": { 2024-04-24T09:18:25.482 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 457188, 2024-04-24T09:18:25.482 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 845.517282086, 2024-04-24T09:18:25.483 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.001849386 2024-04-24T09:18:25.483 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:25.483 INFO:teuthology.orchestra.run.smithi138.stdout: "slow_reply": 0, 2024-04-24T09:18:25.483 INFO:teuthology.orchestra.run.smithi138.stdout: "forward": 15401, 2024-04-24T09:18:25.483 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_fetch_complete": 24, 2024-04-24T09:18:25.483 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_fetch_keys": 2, 2024-04-24T09:18:25.483 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_commit": 3780, 2024-04-24T09:18:25.483 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_split": 6, 2024-04-24T09:18:25.483 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_merge": 0, 2024-04-24T09:18:25.483 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes": 928, 2024-04-24T09:18:25.483 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_top": 443, 2024-04-24T09:18:25.483 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_bottom": 459, 2024-04-24T09:18:25.483 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_pin_tail": 26, 2024-04-24T09:18:25.483 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_pinned": 289, 2024-04-24T09:18:25.483 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_expired": 3779, 2024-04-24T09:18:25.484 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_with_caps": 3, 2024-04-24T09:18:25.484 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": 3, 2024-04-24T09:18:25.484 INFO:teuthology.orchestra.run.smithi138.stdout: "subtrees": 7, 2024-04-24T09:18:25.484 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse": 917787, 2024-04-24T09:18:25.484 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_hit": 871338, 2024-04-24T09:18:25.484 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_forward": 8599, 2024-04-24T09:18:25.484 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_discover": 120, 2024-04-24T09:18:25.484 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_dir_fetch": 0, 2024-04-24T09:18:25.484 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_remote_ino": 0, 2024-04-24T09:18:25.484 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_lock": 1, 2024-04-24T09:18:25.484 INFO:teuthology.orchestra.run.smithi138.stdout: "load_cent": 110, 2024-04-24T09:18:25.484 INFO:teuthology.orchestra.run.smithi138.stdout: "q": 0, 2024-04-24T09:18:25.484 INFO:teuthology.orchestra.run.smithi138.stdout: "exported": 78, 2024-04-24T09:18:25.484 INFO:teuthology.orchestra.run.smithi138.stdout: "exported_inodes": 3277, 2024-04-24T09:18:25.485 INFO:teuthology.orchestra.run.smithi138.stdout: "imported": 62, 2024-04-24T09:18:25.485 INFO:teuthology.orchestra.run.smithi138.stdout: "imported_inodes": 1297, 2024-04-24T09:18:25.485 INFO:teuthology.orchestra.run.smithi138.stdout: "openino_dir_fetch": 0, 2024-04-24T09:18:25.485 INFO:teuthology.orchestra.run.smithi138.stdout: "openino_backtrace_fetch": 5465, 2024-04-24T09:18:25.485 INFO:teuthology.orchestra.run.smithi138.stdout: "openino_peer_discover": 0, 2024-04-24T09:18:25.485 INFO:teuthology.orchestra.run.smithi138.stdout: "root_rfiles": 4, 2024-04-24T09:18:25.485 INFO:teuthology.orchestra.run.smithi138.stdout: "root_rbytes": 234, 2024-04-24T09:18:25.485 INFO:teuthology.orchestra.run.smithi138.stdout: "root_rsnaps": 7, 2024-04-24T09:18:25.485 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_backtrace_fetch": 5465, 2024-04-24T09:18:25.485 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_set_tag": 0, 2024-04-24T09:18:25.485 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_backtrace_repaired": 0, 2024-04-24T09:18:25.485 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_inotable_repaired": 0, 2024-04-24T09:18:25.485 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_dir_inodes": 1099, 2024-04-24T09:18:25.485 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_dir_base_inodes": 46, 2024-04-24T09:18:25.485 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_dirfrag_rstats": 1053, 2024-04-24T09:18:25.486 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_file_inodes": 4366, 2024-04-24T09:18:25.486 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_inode_file_caps": 0, 2024-04-24T09:18:25.486 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_cap_op_revoke": 26806, 2024-04-24T09:18:25.486 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_cap_op_grant": 34652, 2024-04-24T09:18:25.486 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_cap_op_trunc": 0, 2024-04-24T09:18:25.486 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-24T09:18:25.486 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-24T09:18:25.486 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_client_caps": 42091, 2024-04-24T09:18:25.486 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_client_caps_dirty": 10682, 2024-04-24T09:18:25.486 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_client_cap_release": 776, 2024-04-24T09:18:25.486 INFO:teuthology.orchestra.run.smithi138.stdout: "process_request_cap_release": 30804 2024-04-24T09:18:25.486 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:25.486 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_cache": { 2024-04-24T09:18:25.486 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_update": 25, 2024-04-24T09:18:25.487 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_update_receipt": 15, 2024-04-24T09:18:25.487 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_try_discover": 10, 2024-04-24T09:18:25.487 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_send_discover": 125, 2024-04-24T09:18:25.487 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_handle_discover": 240, 2024-04-24T09:18:25.487 INFO:teuthology.orchestra.run.smithi138.stdout: "num_strays": 23, 2024-04-24T09:18:25.487 INFO:teuthology.orchestra.run.smithi138.stdout: "num_strays_delayed": 0, 2024-04-24T09:18:25.487 INFO:teuthology.orchestra.run.smithi138.stdout: "num_strays_enqueuing": 0, 2024-04-24T09:18:25.487 INFO:teuthology.orchestra.run.smithi138.stdout: "strays_created": 8452, 2024-04-24T09:18:25.487 INFO:teuthology.orchestra.run.smithi138.stdout: "strays_enqueued": 8429, 2024-04-24T09:18:25.487 INFO:teuthology.orchestra.run.smithi138.stdout: "strays_reintegrated": 0, 2024-04-24T09:18:25.487 INFO:teuthology.orchestra.run.smithi138.stdout: "strays_migrated": 0, 2024-04-24T09:18:25.487 INFO:teuthology.orchestra.run.smithi138.stdout: "num_recovering_processing": 0, 2024-04-24T09:18:25.487 INFO:teuthology.orchestra.run.smithi138.stdout: "num_recovering_enqueued": 0, 2024-04-24T09:18:25.487 INFO:teuthology.orchestra.run.smithi138.stdout: "num_recovering_prioritized": 0, 2024-04-24T09:18:25.487 INFO:teuthology.orchestra.run.smithi138.stdout: "recovery_started": 0, 2024-04-24T09:18:25.487 INFO:teuthology.orchestra.run.smithi138.stdout: "recovery_completed": 0, 2024-04-24T09:18:25.488 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_quiesce_path": 0, 2024-04-24T09:18:25.488 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_quiesce_inode": 0, 2024-04-24T09:18:25.488 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_enqueue_scrub": 46, 2024-04-24T09:18:25.488 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_exportdir": 94, 2024-04-24T09:18:25.488 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_flush": 0, 2024-04-24T09:18:25.488 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_fragmentdir": 6, 2024-04-24T09:18:25.488 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_fragstats": 0, 2024-04-24T09:18:25.488 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_inodestats": 0 2024-04-24T09:18:25.488 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:25.488 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_log": { 2024-04-24T09:18:25.488 INFO:teuthology.orchestra.run.smithi138.stdout: "evlrg": 0, 2024-04-24T09:18:25.488 INFO:teuthology.orchestra.run.smithi138.stdout: "evadd": 84949, 2024-04-24T09:18:25.488 INFO:teuthology.orchestra.run.smithi138.stdout: "evex": 83471, 2024-04-24T09:18:25.488 INFO:teuthology.orchestra.run.smithi138.stdout: "evtrm": 83471, 2024-04-24T09:18:25.488 INFO:teuthology.orchestra.run.smithi138.stdout: "ev": 1478, 2024-04-24T09:18:25.489 INFO:teuthology.orchestra.run.smithi138.stdout: "evexg": 0, 2024-04-24T09:18:25.489 INFO:teuthology.orchestra.run.smithi138.stdout: "evexd": 1627, 2024-04-24T09:18:25.489 INFO:teuthology.orchestra.run.smithi138.stdout: "segadd": 206, 2024-04-24T09:18:25.489 INFO:teuthology.orchestra.run.smithi138.stdout: "segex": 203, 2024-04-24T09:18:25.489 INFO:teuthology.orchestra.run.smithi138.stdout: "segtrm": 203, 2024-04-24T09:18:25.489 INFO:teuthology.orchestra.run.smithi138.stdout: "seg": 3, 2024-04-24T09:18:25.489 INFO:teuthology.orchestra.run.smithi138.stdout: "segmjr": 1, 2024-04-24T09:18:25.489 INFO:teuthology.orchestra.run.smithi138.stdout: "segexg": 0, 2024-04-24T09:18:25.489 INFO:teuthology.orchestra.run.smithi138.stdout: "segexd": 3, 2024-04-24T09:18:25.489 INFO:teuthology.orchestra.run.smithi138.stdout: "expos": 249955747, 2024-04-24T09:18:25.489 INFO:teuthology.orchestra.run.smithi138.stdout: "wrpos": 254679510, 2024-04-24T09:18:25.489 INFO:teuthology.orchestra.run.smithi138.stdout: "rdpos": 0, 2024-04-24T09:18:25.489 INFO:teuthology.orchestra.run.smithi138.stdout: "jlat": { 2024-04-24T09:18:25.489 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 27607, 2024-04-24T09:18:25.489 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 223.283326319, 2024-04-24T09:18:25.490 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.008087924 2024-04-24T09:18:25.490 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:25.490 INFO:teuthology.orchestra.run.smithi138.stdout: "replayed": 0 2024-04-24T09:18:25.490 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:25.490 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_mem": { 2024-04-24T09:18:25.490 INFO:teuthology.orchestra.run.smithi138.stdout: "ino": 734, 2024-04-24T09:18:25.490 INFO:teuthology.orchestra.run.smithi138.stdout: "ino+": 12394, 2024-04-24T09:18:25.490 INFO:teuthology.orchestra.run.smithi138.stdout: "ino-": 11660, 2024-04-24T09:18:25.490 INFO:teuthology.orchestra.run.smithi138.stdout: "dir": 70, 2024-04-24T09:18:25.490 INFO:teuthology.orchestra.run.smithi138.stdout: "dir+": 332, 2024-04-24T09:18:25.490 INFO:teuthology.orchestra.run.smithi138.stdout: "dir-": 262, 2024-04-24T09:18:25.490 INFO:teuthology.orchestra.run.smithi138.stdout: "dn": 928, 2024-04-24T09:18:25.490 INFO:teuthology.orchestra.run.smithi138.stdout: "dn+": 20940, 2024-04-24T09:18:25.490 INFO:teuthology.orchestra.run.smithi138.stdout: "dn-": 20012, 2024-04-24T09:18:25.491 INFO:teuthology.orchestra.run.smithi138.stdout: "cap": 3, 2024-04-24T09:18:25.491 INFO:teuthology.orchestra.run.smithi138.stdout: "cap+": 9276, 2024-04-24T09:18:25.491 INFO:teuthology.orchestra.run.smithi138.stdout: "cap-": 9273, 2024-04-24T09:18:25.491 INFO:teuthology.orchestra.run.smithi138.stdout: "rss": 81104, 2024-04-24T09:18:25.491 INFO:teuthology.orchestra.run.smithi138.stdout: "heap": 231800 2024-04-24T09:18:25.491 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:25.491 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_server": { 2024-04-24T09:18:25.491 INFO:teuthology.orchestra.run.smithi138.stdout: "dispatch_client_request": 554290, 2024-04-24T09:18:25.491 INFO:teuthology.orchestra.run.smithi138.stdout: "dispatch_server_request": 6683, 2024-04-24T09:18:25.491 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_client_request": 472589, 2024-04-24T09:18:25.491 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_client_session": 652, 2024-04-24T09:18:25.491 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_peer_request": 33856, 2024-04-24T09:18:25.491 INFO:teuthology.orchestra.run.smithi138.stdout: "req_create_latency": { 2024-04-24T09:18:25.491 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 8936, 2024-04-24T09:18:25.491 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 45.922100334, 2024-04-24T09:18:25.492 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.005138999 2024-04-24T09:18:25.492 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:25.492 INFO:teuthology.orchestra.run.smithi138.stdout: "req_getattr_latency": { 2024-04-24T09:18:25.492 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 386163, 2024-04-24T09:18:25.492 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 307.294941203, 2024-04-24T09:18:25.492 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000795764 2024-04-24T09:18:25.492 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:25.492 INFO:teuthology.orchestra.run.smithi138.stdout: "req_getfilelock_latency": { 2024-04-24T09:18:25.492 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-24T09:18:25.492 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-24T09:18:25.492 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-24T09:18:25.492 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:25.492 INFO:teuthology.orchestra.run.smithi138.stdout: "req_link_latency": { 2024-04-24T09:18:25.492 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-24T09:18:25.492 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-24T09:18:25.492 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-24T09:18:25.493 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:25.493 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookup_latency": { 2024-04-24T09:18:25.493 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 23187, 2024-04-24T09:18:25.493 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 309.002938498, 2024-04-24T09:18:25.493 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.013326559 2024-04-24T09:18:25.493 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:25.493 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookuphash_latency": { 2024-04-24T09:18:25.493 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-24T09:18:25.493 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-24T09:18:25.493 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-24T09:18:25.493 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:25.493 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookupino_latency": { 2024-04-24T09:18:25.493 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-24T09:18:25.493 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-24T09:18:25.493 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-24T09:18:25.493 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:25.494 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookupname_latency": { 2024-04-24T09:18:25.494 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-24T09:18:25.494 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-24T09:18:25.494 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-24T09:18:25.494 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:25.494 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookupparent_latency": { 2024-04-24T09:18:25.494 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-24T09:18:25.494 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-24T09:18:25.494 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-24T09:18:25.494 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:25.494 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookupsnap_latency": { 2024-04-24T09:18:25.495 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-24T09:18:25.495 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-24T09:18:25.495 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-24T09:18:25.495 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:25.495 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lssnap_latency": { 2024-04-24T09:18:25.495 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 27, 2024-04-24T09:18:25.495 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.012491061, 2024-04-24T09:18:25.495 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000462631 2024-04-24T09:18:25.495 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:25.495 INFO:teuthology.orchestra.run.smithi138.stdout: "req_mkdir_latency": { 2024-04-24T09:18:25.495 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 119, 2024-04-24T09:18:25.496 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.672858425, 2024-04-24T09:18:25.496 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.005654272 2024-04-24T09:18:25.496 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:25.496 INFO:teuthology.orchestra.run.smithi138.stdout: "req_mknod_latency": { 2024-04-24T09:18:25.496 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-24T09:18:25.496 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-24T09:18:25.496 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-24T09:18:25.496 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:25.496 INFO:teuthology.orchestra.run.smithi138.stdout: "req_mksnap_latency": { 2024-04-24T09:18:25.496 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 27, 2024-04-24T09:18:25.496 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 6.498469529, 2024-04-24T09:18:25.496 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.240684056 2024-04-24T09:18:25.496 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:25.497 INFO:teuthology.orchestra.run.smithi138.stdout: "req_open_latency": { 2024-04-24T09:18:25.497 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 5444, 2024-04-24T09:18:25.497 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 8.542131705, 2024-04-24T09:18:25.497 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.001569091 2024-04-24T09:18:25.497 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:25.497 INFO:teuthology.orchestra.run.smithi138.stdout: "req_readdir_latency": { 2024-04-24T09:18:25.497 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 22924, 2024-04-24T09:18:25.497 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 65.041240576, 2024-04-24T09:18:25.497 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.002837255 2024-04-24T09:18:25.497 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:25.497 INFO:teuthology.orchestra.run.smithi138.stdout: "req_rename_latency": { 2024-04-24T09:18:25.497 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 1616, 2024-04-24T09:18:25.497 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 49.551779608, 2024-04-24T09:18:25.497 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.030663229 2024-04-24T09:18:25.497 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:25.498 INFO:teuthology.orchestra.run.smithi138.stdout: "req_renamesnap_latency": { 2024-04-24T09:18:25.498 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-24T09:18:25.498 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-24T09:18:25.498 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-24T09:18:25.498 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:25.498 INFO:teuthology.orchestra.run.smithi138.stdout: "req_snapdiff_latency": { 2024-04-24T09:18:25.498 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-24T09:18:25.498 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-24T09:18:25.498 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-24T09:18:25.498 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:25.498 INFO:teuthology.orchestra.run.smithi138.stdout: "req_rmdir_latency": { 2024-04-24T09:18:25.498 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 34, 2024-04-24T09:18:25.498 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.551153724, 2024-04-24T09:18:25.498 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.016210403 2024-04-24T09:18:25.498 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:25.499 INFO:teuthology.orchestra.run.smithi138.stdout: "req_rmsnap_latency": { 2024-04-24T09:18:25.499 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 20, 2024-04-24T09:18:25.499 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 1.036785363, 2024-04-24T09:18:25.499 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.051839268 2024-04-24T09:18:25.499 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:25.499 INFO:teuthology.orchestra.run.smithi138.stdout: "req_rmxattr_latency": { 2024-04-24T09:18:25.499 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-24T09:18:25.499 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-24T09:18:25.499 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-24T09:18:25.499 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:25.499 INFO:teuthology.orchestra.run.smithi138.stdout: "req_setattr_latency": { 2024-04-24T09:18:25.500 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 2, 2024-04-24T09:18:25.500 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.003567847, 2024-04-24T09:18:25.500 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.001783923 2024-04-24T09:18:25.500 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:25.500 INFO:teuthology.orchestra.run.smithi138.stdout: "req_setdirlayout_latency": { 2024-04-24T09:18:25.500 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-24T09:18:25.500 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-24T09:18:25.500 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-24T09:18:25.500 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:25.500 INFO:teuthology.orchestra.run.smithi138.stdout: "req_setfilelock_latency": { 2024-04-24T09:18:25.500 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 252, 2024-04-24T09:18:25.500 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.228247270, 2024-04-24T09:18:25.500 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000905743 2024-04-24T09:18:25.501 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:25.501 INFO:teuthology.orchestra.run.smithi138.stdout: "req_setlayout_latency": { 2024-04-24T09:18:25.501 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-24T09:18:25.501 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-24T09:18:25.501 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-24T09:18:25.501 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:25.501 INFO:teuthology.orchestra.run.smithi138.stdout: "req_setxattr_latency": { 2024-04-24T09:18:25.501 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 8, 2024-04-24T09:18:25.501 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.047777322, 2024-04-24T09:18:25.501 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.005972165 2024-04-24T09:18:25.501 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:25.501 INFO:teuthology.orchestra.run.smithi138.stdout: "req_symlink_latency": { 2024-04-24T09:18:25.501 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-24T09:18:25.501 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-24T09:18:25.501 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-24T09:18:25.502 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:25.502 INFO:teuthology.orchestra.run.smithi138.stdout: "req_unlink_latency": { 2024-04-24T09:18:25.502 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 8429, 2024-04-24T09:18:25.502 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 51.110799621, 2024-04-24T09:18:25.502 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.006063684 2024-04-24T09:18:25.502 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:25.502 INFO:teuthology.orchestra.run.smithi138.stdout: "cap_revoke_eviction": 0, 2024-04-24T09:18:25.502 INFO:teuthology.orchestra.run.smithi138.stdout: "cap_acquisition_throttle": 0, 2024-04-24T09:18:25.502 INFO:teuthology.orchestra.run.smithi138.stdout: "req_getvxattr_latency": { 2024-04-24T09:18:25.502 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-24T09:18:25.502 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-24T09:18:25.502 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-24T09:18:25.502 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-24T09:18:25.502 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:25.502 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_sessions": { 2024-04-24T09:18:25.503 INFO:teuthology.orchestra.run.smithi138.stdout: "session_count": 2, 2024-04-24T09:18:25.503 INFO:teuthology.orchestra.run.smithi138.stdout: "session_add": 6, 2024-04-24T09:18:25.503 INFO:teuthology.orchestra.run.smithi138.stdout: "session_remove": 4, 2024-04-24T09:18:25.503 INFO:teuthology.orchestra.run.smithi138.stdout: "sessions_open": 2, 2024-04-24T09:18:25.503 INFO:teuthology.orchestra.run.smithi138.stdout: "sessions_stale": 0, 2024-04-24T09:18:25.503 INFO:teuthology.orchestra.run.smithi138.stdout: "total_load": 26041, 2024-04-24T09:18:25.503 INFO:teuthology.orchestra.run.smithi138.stdout: "average_load": 13020, 2024-04-24T09:18:25.503 INFO:teuthology.orchestra.run.smithi138.stdout: "avg_session_uptime": 1594, 2024-04-24T09:18:25.503 INFO:teuthology.orchestra.run.smithi138.stdout: "mdthresh_evicted": 0 2024-04-24T09:18:25.503 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:25.503 INFO:teuthology.orchestra.run.smithi138.stdout: "mempool": { 2024-04-24T09:18:25.503 INFO:teuthology.orchestra.run.smithi138.stdout: "bloom_filter_bytes": 240, 2024-04-24T09:18:25.503 INFO:teuthology.orchestra.run.smithi138.stdout: "bloom_filter_items": 240, 2024-04-24T09:18:25.503 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_alloc_bytes": 0, 2024-04-24T09:18:25.503 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_alloc_items": 0, 2024-04-24T09:18:25.505 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_data_bytes": 0, 2024-04-24T09:18:25.505 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_data_items": 0, 2024-04-24T09:18:25.505 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-24T09:18:25.505 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_onode_items": 0, 2024-04-24T09:18:25.505 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-24T09:18:25.505 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_meta_items": 0, 2024-04-24T09:18:25.505 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_other_bytes": 0, 2024-04-24T09:18:25.505 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_other_items": 0, 2024-04-24T09:18:25.506 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-24T09:18:25.506 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_buffer_items": 0, 2024-04-24T09:18:25.506 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_extent_bytes": 0, 2024-04-24T09:18:25.506 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_extent_items": 0, 2024-04-24T09:18:25.506 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_blob_bytes": 0, 2024-04-24T09:18:25.506 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_blob_items": 0, 2024-04-24T09:18:25.506 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-24T09:18:25.506 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_shared_blob_items": 0, 2024-04-24T09:18:25.506 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-24T09:18:25.506 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_inline_bl_items": 0, 2024-04-24T09:18:25.506 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_fsck_bytes": 0, 2024-04-24T09:18:25.506 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_fsck_items": 0, 2024-04-24T09:18:25.506 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_txc_bytes": 0, 2024-04-24T09:18:25.507 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_txc_items": 0, 2024-04-24T09:18:25.507 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-24T09:18:25.507 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_writing_deferred_items": 0, 2024-04-24T09:18:25.507 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_writing_bytes": 0, 2024-04-24T09:18:25.507 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_writing_items": 0, 2024-04-24T09:18:25.507 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_bytes": 0, 2024-04-24T09:18:25.507 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_items": 0, 2024-04-24T09:18:25.507 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_file_reader_bytes": 0, 2024-04-24T09:18:25.507 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_file_reader_items": 0, 2024-04-24T09:18:25.507 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_file_writer_bytes": 0, 2024-04-24T09:18:25.507 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_file_writer_items": 0, 2024-04-24T09:18:25.507 INFO:teuthology.orchestra.run.smithi138.stdout: "buffer_anon_bytes": 532439, 2024-04-24T09:18:25.507 INFO:teuthology.orchestra.run.smithi138.stdout: "buffer_anon_items": 168, 2024-04-24T09:18:25.507 INFO:teuthology.orchestra.run.smithi138.stdout: "buffer_meta_bytes": 88, 2024-04-24T09:18:25.507 INFO:teuthology.orchestra.run.smithi138.stdout: "buffer_meta_items": 1, 2024-04-24T09:18:25.507 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_bytes": 0, 2024-04-24T09:18:25.508 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_items": 0, 2024-04-24T09:18:25.508 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_mapbl_bytes": 0, 2024-04-24T09:18:25.508 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_mapbl_items": 0, 2024-04-24T09:18:25.508 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pglog_bytes": 0, 2024-04-24T09:18:25.508 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pglog_items": 0, 2024-04-24T09:18:25.508 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_bytes": 9800, 2024-04-24T09:18:25.508 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_items": 188, 2024-04-24T09:18:25.508 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_mapping_bytes": 0, 2024-04-24T09:18:25.508 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_mapping_items": 0, 2024-04-24T09:18:25.508 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap_bytes": 0, 2024-04-24T09:18:25.508 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap_items": 0, 2024-04-24T09:18:25.508 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_co_bytes": 2580195, 2024-04-24T09:18:25.508 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_co_items": 12527, 2024-04-24T09:18:25.508 INFO:teuthology.orchestra.run.smithi138.stdout: "unittest_1_bytes": 0, 2024-04-24T09:18:25.508 INFO:teuthology.orchestra.run.smithi138.stdout: "unittest_1_items": 0, 2024-04-24T09:18:25.508 INFO:teuthology.orchestra.run.smithi138.stdout: "unittest_2_bytes": 0, 2024-04-24T09:18:25.509 INFO:teuthology.orchestra.run.smithi138.stdout: "unittest_2_items": 0 2024-04-24T09:18:25.509 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:25.509 INFO:teuthology.orchestra.run.smithi138.stdout: "objecter": { 2024-04-24T09:18:25.509 INFO:teuthology.orchestra.run.smithi138.stdout: "op_active": 0, 2024-04-24T09:18:25.509 INFO:teuthology.orchestra.run.smithi138.stdout: "op_laggy": 0, 2024-04-24T09:18:25.509 INFO:teuthology.orchestra.run.smithi138.stdout: "op_send": 65399, 2024-04-24T09:18:25.509 INFO:teuthology.orchestra.run.smithi138.stdout: "op_send_bytes": 269259926, 2024-04-24T09:18:25.509 INFO:teuthology.orchestra.run.smithi138.stdout: "op_resend": 31, 2024-04-24T09:18:25.509 INFO:teuthology.orchestra.run.smithi138.stdout: "op_reply": 65399, 2024-04-24T09:18:25.509 INFO:teuthology.orchestra.run.smithi138.stdout: "op_latency": { 2024-04-24T09:18:25.509 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 65399, 2024-04-24T09:18:25.509 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 969.440373774, 2024-04-24T09:18:25.509 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.014823473 2024-04-24T09:18:25.509 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:25.509 INFO:teuthology.orchestra.run.smithi138.stdout: "op_inflight": 0, 2024-04-24T09:18:25.510 INFO:teuthology.orchestra.run.smithi138.stdout: "oplen_avg": { 2024-04-24T09:18:25.510 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 65399, 2024-04-24T09:18:25.510 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 84385 2024-04-24T09:18:25.510 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:25.510 INFO:teuthology.orchestra.run.smithi138.stdout: "op": 65399, 2024-04-24T09:18:25.510 INFO:teuthology.orchestra.run.smithi138.stdout: "op_r": 6983, 2024-04-24T09:18:25.510 INFO:teuthology.orchestra.run.smithi138.stdout: "op_w": 58416, 2024-04-24T09:18:25.510 INFO:teuthology.orchestra.run.smithi138.stdout: "op_rmw": 0, 2024-04-24T09:18:25.510 INFO:teuthology.orchestra.run.smithi138.stdout: "op_pg": 0, 2024-04-24T09:18:25.510 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_stat": 3622, 2024-04-24T09:18:25.510 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_create": 5984, 2024-04-24T09:18:25.511 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_read": 1413, 2024-04-24T09:18:25.511 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_write": 28974, 2024-04-24T09:18:25.511 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_writefull": 1170, 2024-04-24T09:18:25.511 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_writesame": 0, 2024-04-24T09:18:25.511 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_append": 0, 2024-04-24T09:18:25.511 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_zero": 0, 2024-04-24T09:18:25.511 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_truncate": 6, 2024-04-24T09:18:25.511 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_delete": 17487, 2024-04-24T09:18:25.511 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_mapext": 0, 2024-04-24T09:18:25.511 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_sparse_read": 0, 2024-04-24T09:18:25.511 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_clonerange": 0, 2024-04-24T09:18:25.511 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_getxattr": 5536, 2024-04-24T09:18:25.511 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_setxattr": 9097, 2024-04-24T09:18:25.511 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_cmpxattr": 0, 2024-04-24T09:18:25.511 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_rmxattr": 0, 2024-04-24T09:18:25.512 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_resetxattrs": 0, 2024-04-24T09:18:25.512 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_call": 0, 2024-04-24T09:18:25.512 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_watch": 0, 2024-04-24T09:18:25.512 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_notify": 0, 2024-04-24T09:18:25.512 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_src_cmpxattr": 0, 2024-04-24T09:18:25.512 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_pgls": 0, 2024-04-24T09:18:25.512 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_pgls_filter": 0, 2024-04-24T09:18:25.512 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_other": 0, 2024-04-24T09:18:25.512 INFO:teuthology.orchestra.run.smithi138.stdout: "linger_active": 0, 2024-04-24T09:18:25.512 INFO:teuthology.orchestra.run.smithi138.stdout: "linger_send": 0, 2024-04-24T09:18:25.512 INFO:teuthology.orchestra.run.smithi138.stdout: "linger_resend": 0, 2024-04-24T09:18:25.512 INFO:teuthology.orchestra.run.smithi138.stdout: "linger_ping": 0, 2024-04-24T09:18:25.512 INFO:teuthology.orchestra.run.smithi138.stdout: "poolop_active": 0, 2024-04-24T09:18:25.512 INFO:teuthology.orchestra.run.smithi138.stdout: "poolop_send": 0, 2024-04-24T09:18:25.512 INFO:teuthology.orchestra.run.smithi138.stdout: "poolop_resend": 0, 2024-04-24T09:18:25.512 INFO:teuthology.orchestra.run.smithi138.stdout: "poolstat_active": 0, 2024-04-24T09:18:25.513 INFO:teuthology.orchestra.run.smithi138.stdout: "poolstat_send": 0, 2024-04-24T09:18:25.513 INFO:teuthology.orchestra.run.smithi138.stdout: "poolstat_resend": 0, 2024-04-24T09:18:25.513 INFO:teuthology.orchestra.run.smithi138.stdout: "statfs_active": 0, 2024-04-24T09:18:25.513 INFO:teuthology.orchestra.run.smithi138.stdout: "statfs_send": 0, 2024-04-24T09:18:25.513 INFO:teuthology.orchestra.run.smithi138.stdout: "statfs_resend": 0, 2024-04-24T09:18:25.513 INFO:teuthology.orchestra.run.smithi138.stdout: "command_active": 0, 2024-04-24T09:18:25.513 INFO:teuthology.orchestra.run.smithi138.stdout: "command_send": 0, 2024-04-24T09:18:25.513 INFO:teuthology.orchestra.run.smithi138.stdout: "command_resend": 0, 2024-04-24T09:18:25.513 INFO:teuthology.orchestra.run.smithi138.stdout: "map_epoch": 136, 2024-04-24T09:18:25.513 INFO:teuthology.orchestra.run.smithi138.stdout: "map_full": 0, 2024-04-24T09:18:25.513 INFO:teuthology.orchestra.run.smithi138.stdout: "map_inc": 52, 2024-04-24T09:18:25.513 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_sessions": 12, 2024-04-24T09:18:25.513 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_session_open": 12, 2024-04-24T09:18:25.513 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_session_close": 0, 2024-04-24T09:18:25.513 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_laggy": 0, 2024-04-24T09:18:25.514 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_wr": 8872, 2024-04-24T09:18:25.514 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_rd": 85, 2024-04-24T09:18:25.514 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_del": 2139 2024-04-24T09:18:25.514 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:25.514 INFO:teuthology.orchestra.run.smithi138.stdout: "oft": { 2024-04-24T09:18:25.514 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_total_objs": 1, 2024-04-24T09:18:25.514 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_total_kv_pairs": 0, 2024-04-24T09:18:25.514 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_total_updates": 0, 2024-04-24T09:18:25.514 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_total_removes": 0 2024-04-24T09:18:25.514 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:25.514 INFO:teuthology.orchestra.run.smithi138.stdout: "osd-slow-ops": { 2024-04-24T09:18:25.514 INFO:teuthology.orchestra.run.smithi138.stdout: "slow_ops_count": 0 2024-04-24T09:18:25.514 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:25.515 INFO:teuthology.orchestra.run.smithi138.stdout: "purge_queue": { 2024-04-24T09:18:25.515 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executing_ops": 0, 2024-04-24T09:18:25.515 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executing_ops_high_water": 10, 2024-04-24T09:18:25.515 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executing": 0, 2024-04-24T09:18:25.515 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executing_high_water": 5, 2024-04-24T09:18:25.515 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executed_ops": 16858, 2024-04-24T09:18:25.515 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executed": 8429, 2024-04-24T09:18:25.515 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_item_in_journal": 0 2024-04-24T09:18:25.515 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:25.515 INFO:teuthology.orchestra.run.smithi138.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-24T09:18:25.515 INFO:teuthology.orchestra.run.smithi138.stdout: "val": 0, 2024-04-24T09:18:25.515 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 104857600, 2024-04-24T09:18:25.515 INFO:teuthology.orchestra.run.smithi138.stdout: "get_started": 0, 2024-04-24T09:18:25.515 INFO:teuthology.orchestra.run.smithi138.stdout: "get": 772108, 2024-04-24T09:18:25.515 INFO:teuthology.orchestra.run.smithi138.stdout: "get_sum": 182215113, 2024-04-24T09:18:25.516 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_fail": 0, 2024-04-24T09:18:25.516 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_success": 772108, 2024-04-24T09:18:25.516 INFO:teuthology.orchestra.run.smithi138.stdout: "take": 0, 2024-04-24T09:18:25.516 INFO:teuthology.orchestra.run.smithi138.stdout: "take_sum": 0, 2024-04-24T09:18:25.516 INFO:teuthology.orchestra.run.smithi138.stdout: "put": 772108, 2024-04-24T09:18:25.516 INFO:teuthology.orchestra.run.smithi138.stdout: "put_sum": 182215113, 2024-04-24T09:18:25.516 INFO:teuthology.orchestra.run.smithi138.stdout: "wait": { 2024-04-24T09:18:25.516 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-24T09:18:25.516 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-24T09:18:25.516 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-24T09:18:25.516 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-24T09:18:25.516 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:25.516 INFO:teuthology.orchestra.run.smithi138.stdout: "throttle-objecter_bytes": { 2024-04-24T09:18:25.516 INFO:teuthology.orchestra.run.smithi138.stdout: "val": 0, 2024-04-24T09:18:25.516 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 104857600, 2024-04-24T09:18:25.516 INFO:teuthology.orchestra.run.smithi138.stdout: "get_started": 0, 2024-04-24T09:18:25.517 INFO:teuthology.orchestra.run.smithi138.stdout: "get": 0, 2024-04-24T09:18:25.517 INFO:teuthology.orchestra.run.smithi138.stdout: "get_sum": 0, 2024-04-24T09:18:25.517 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_fail": 0, 2024-04-24T09:18:25.517 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_success": 0, 2024-04-24T09:18:25.517 INFO:teuthology.orchestra.run.smithi138.stdout: "take": 65399, 2024-04-24T09:18:25.517 INFO:teuthology.orchestra.run.smithi138.stdout: "take_sum": 270188460, 2024-04-24T09:18:25.517 INFO:teuthology.orchestra.run.smithi138.stdout: "put": 65399, 2024-04-24T09:18:25.517 INFO:teuthology.orchestra.run.smithi138.stdout: "put_sum": 270188460, 2024-04-24T09:18:25.517 INFO:teuthology.orchestra.run.smithi138.stdout: "wait": { 2024-04-24T09:18:25.517 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-24T09:18:25.517 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-24T09:18:25.517 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-24T09:18:25.517 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-24T09:18:25.517 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:25.517 INFO:teuthology.orchestra.run.smithi138.stdout: "throttle-objecter_ops": { 2024-04-24T09:18:25.517 INFO:teuthology.orchestra.run.smithi138.stdout: "val": 0, 2024-04-24T09:18:25.518 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 1024, 2024-04-24T09:18:25.518 INFO:teuthology.orchestra.run.smithi138.stdout: "get_started": 0, 2024-04-24T09:18:25.518 INFO:teuthology.orchestra.run.smithi138.stdout: "get": 0, 2024-04-24T09:18:25.518 INFO:teuthology.orchestra.run.smithi138.stdout: "get_sum": 0, 2024-04-24T09:18:25.518 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_fail": 0, 2024-04-24T09:18:25.518 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_success": 0, 2024-04-24T09:18:25.518 INFO:teuthology.orchestra.run.smithi138.stdout: "take": 65399, 2024-04-24T09:18:25.518 INFO:teuthology.orchestra.run.smithi138.stdout: "take_sum": 65399, 2024-04-24T09:18:25.518 INFO:teuthology.orchestra.run.smithi138.stdout: "put": 65399, 2024-04-24T09:18:25.518 INFO:teuthology.orchestra.run.smithi138.stdout: "put_sum": 65399, 2024-04-24T09:18:25.518 INFO:teuthology.orchestra.run.smithi138.stdout: "wait": { 2024-04-24T09:18:25.518 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-24T09:18:25.518 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-24T09:18:25.518 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-24T09:18:25.519 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-24T09:18:25.519 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:25.519 INFO:teuthology.orchestra.run.smithi138.stdout: "throttle-write_buf_throttle": { 2024-04-24T09:18:25.519 INFO:teuthology.orchestra.run.smithi138.stdout: "val": 0, 2024-04-24T09:18:25.519 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 3758096384, 2024-04-24T09:18:25.519 INFO:teuthology.orchestra.run.smithi138.stdout: "get_started": 0, 2024-04-24T09:18:25.519 INFO:teuthology.orchestra.run.smithi138.stdout: "get": 8429, 2024-04-24T09:18:25.519 INFO:teuthology.orchestra.run.smithi138.stdout: "get_sum": 929897, 2024-04-24T09:18:25.519 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_fail": 0, 2024-04-24T09:18:25.519 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_success": 8429, 2024-04-24T09:18:25.519 INFO:teuthology.orchestra.run.smithi138.stdout: "take": 0, 2024-04-24T09:18:25.519 INFO:teuthology.orchestra.run.smithi138.stdout: "take_sum": 0, 2024-04-24T09:18:25.519 INFO:teuthology.orchestra.run.smithi138.stdout: "put": 1367, 2024-04-24T09:18:25.519 INFO:teuthology.orchestra.run.smithi138.stdout: "put_sum": 929897, 2024-04-24T09:18:25.519 INFO:teuthology.orchestra.run.smithi138.stdout: "wait": { 2024-04-24T09:18:25.519 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-24T09:18:25.520 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-24T09:18:25.520 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-24T09:18:25.520 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-24T09:18:25.520 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:25.520 INFO:teuthology.orchestra.run.smithi138.stdout: "throttle-write_buf_throttle-0x55cf870e4500": { 2024-04-24T09:18:25.520 INFO:teuthology.orchestra.run.smithi138.stdout: "val": 0, 2024-04-24T09:18:25.520 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 3758096384, 2024-04-24T09:18:25.520 INFO:teuthology.orchestra.run.smithi138.stdout: "get_started": 0, 2024-04-24T09:18:25.520 INFO:teuthology.orchestra.run.smithi138.stdout: "get": 84949, 2024-04-24T09:18:25.520 INFO:teuthology.orchestra.run.smithi138.stdout: "get_sum": 250485206, 2024-04-24T09:18:25.520 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_fail": 0, 2024-04-24T09:18:25.520 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_success": 84949, 2024-04-24T09:18:25.520 INFO:teuthology.orchestra.run.smithi138.stdout: "take": 0, 2024-04-24T09:18:25.520 INFO:teuthology.orchestra.run.smithi138.stdout: "take_sum": 0, 2024-04-24T09:18:25.520 INFO:teuthology.orchestra.run.smithi138.stdout: "put": 27607, 2024-04-24T09:18:25.521 INFO:teuthology.orchestra.run.smithi138.stdout: "put_sum": 250485206, 2024-04-24T09:18:25.521 INFO:teuthology.orchestra.run.smithi138.stdout: "wait": { 2024-04-24T09:18:25.521 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-24T09:18:25.521 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-24T09:18:25.521 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-24T09:18:25.521 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-24T09:18:25.521 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-24T09:18:25.521 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-04-24T09:18:26.401 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:18:26 smithi138 ceph-mon[31441]: pgmap v849: 129 pgs: 129 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-24T09:18:26.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:18:26 smithi189 ceph-mon[31791]: pgmap v849: 129 pgs: 129 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-24T09:18:26.466 INFO:tasks.check_counter:Daemon mds.b mds.exported=78 2024-04-24T09:18:26.466 INFO:tasks.check_counter:Daemon mds.b mds.imported=62 2024-04-24T09:18:26.466 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=7 2024-04-24T09:18:26.466 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=27 2024-04-24T09:18:26.466 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=20 2024-04-24T09:18:26.466 DEBUG:tasks.check_counter:Getting stats from e 2024-04-24T09:18:26.466 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid bd3e039e-0215-11ef-bc93-c7b262605968 -- ceph daemon mds.e perf dump 2024-04-24T09:18:26.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:18:26 smithi083 ceph-mon[27907]: pgmap v849: 129 pgs: 129 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-24T09:18:26.914 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T09:18:28.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:18:28 smithi138 ceph-mon[31441]: pgmap v850: 129 pgs: 129 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-24T09:18:28.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:18:28 smithi189 ceph-mon[31791]: pgmap v850: 129 pgs: 129 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-24T09:18:28.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:18:28 smithi083 ceph-mon[27907]: pgmap v850: 129 pgs: 129 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-24T09:18:29.466 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-04-24T09:18:29.466 INFO:teuthology.orchestra.run.smithi138.stdout: "AsyncMessenger::Worker-0": { 2024-04-24T09:18:29.466 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_messages": 47733, 2024-04-24T09:18:29.466 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages": 39752, 2024-04-24T09:18:29.466 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_bytes": 17093638, 2024-04-24T09:18:29.466 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_bytes": 98023429, 2024-04-24T09:18:29.466 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_created_connections": 94, 2024-04-24T09:18:29.467 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_active_connections": 6, 2024-04-24T09:18:29.467 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_total_time": 8.413089710, 2024-04-24T09:18:29.467 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_send_time": 3.781425123, 2024-04-24T09:18:29.467 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_recv_time": 4.233674988, 2024-04-24T09:18:29.467 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_fast_dispatch_time": 1.178700272, 2024-04-24T09:18:29.467 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T09:18:29.467 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 39756, 2024-04-24T09:18:29.467 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 4.979664473, 2024-04-24T09:18:29.467 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000125255 2024-04-24T09:18:29.467 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:29.467 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_handle_ack_lat": { 2024-04-24T09:18:29.467 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 60673, 2024-04-24T09:18:29.467 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.026112000, 2024-04-24T09:18:29.467 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000430 2024-04-24T09:18:29.468 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:29.468 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-24T09:18:29.468 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_encrypted_bytes": 0 2024-04-24T09:18:29.468 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:29.468 INFO:teuthology.orchestra.run.smithi138.stdout: "AsyncMessenger::Worker-1": { 2024-04-24T09:18:29.468 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_messages": 35265, 2024-04-24T09:18:29.468 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages": 27490, 2024-04-24T09:18:29.468 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_bytes": 13696150, 2024-04-24T09:18:29.468 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_bytes": 108571030, 2024-04-24T09:18:29.468 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_created_connections": 12, 2024-04-24T09:18:29.468 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_active_connections": 6, 2024-04-24T09:18:29.468 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_total_time": 5.892917051, 2024-04-24T09:18:29.468 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_send_time": 1.984742398, 2024-04-24T09:18:29.468 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_recv_time": 1.831620563, 2024-04-24T09:18:29.469 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_fast_dispatch_time": 2.243384629, 2024-04-24T09:18:29.469 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T09:18:29.469 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 27488, 2024-04-24T09:18:29.469 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 3.221198612, 2024-04-24T09:18:29.469 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000117185 2024-04-24T09:18:29.469 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:29.469 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_handle_ack_lat": { 2024-04-24T09:18:29.469 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 29992, 2024-04-24T09:18:29.469 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.007850622, 2024-04-24T09:18:29.469 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000261 2024-04-24T09:18:29.469 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:29.469 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_encrypted_bytes": 6272, 2024-04-24T09:18:29.469 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_encrypted_bytes": 126704 2024-04-24T09:18:29.469 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:29.469 INFO:teuthology.orchestra.run.smithi138.stdout: "AsyncMessenger::Worker-2": { 2024-04-24T09:18:29.470 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_messages": 382117, 2024-04-24T09:18:29.470 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages": 433295, 2024-04-24T09:18:29.470 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_bytes": 117227995, 2024-04-24T09:18:29.470 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_bytes": 366891159, 2024-04-24T09:18:29.470 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_created_connections": 23, 2024-04-24T09:18:29.470 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_active_connections": 8, 2024-04-24T09:18:29.470 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_total_time": 42.634726550, 2024-04-24T09:18:29.470 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_send_time": 17.922710327, 2024-04-24T09:18:29.470 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_recv_time": 27.653652160, 2024-04-24T09:18:29.470 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_fast_dispatch_time": 1.642219498, 2024-04-24T09:18:29.470 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T09:18:29.470 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 433288, 2024-04-24T09:18:29.470 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 32.788631086, 2024-04-24T09:18:29.470 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000075673 2024-04-24T09:18:29.471 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:29.471 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_handle_ack_lat": { 2024-04-24T09:18:29.471 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 762231, 2024-04-24T09:18:29.471 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.235029844, 2024-04-24T09:18:29.471 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000308 2024-04-24T09:18:29.471 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:29.471 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_encrypted_bytes": 2110512, 2024-04-24T09:18:29.471 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_encrypted_bytes": 1006608 2024-04-24T09:18:29.471 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:29.471 INFO:teuthology.orchestra.run.smithi138.stdout: "cct": { 2024-04-24T09:18:29.471 INFO:teuthology.orchestra.run.smithi138.stdout: "total_workers": 1, 2024-04-24T09:18:29.471 INFO:teuthology.orchestra.run.smithi138.stdout: "unhealthy_workers": 0 2024-04-24T09:18:29.471 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:29.471 INFO:teuthology.orchestra.run.smithi138.stdout: "finisher-MDSRank": { 2024-04-24T09:18:29.471 INFO:teuthology.orchestra.run.smithi138.stdout: "queue_len": 0, 2024-04-24T09:18:29.472 INFO:teuthology.orchestra.run.smithi138.stdout: "complete_latency": { 2024-04-24T09:18:29.472 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 32357, 2024-04-24T09:18:29.472 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 42.705741323, 2024-04-24T09:18:29.472 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.001319830 2024-04-24T09:18:29.472 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-24T09:18:29.472 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:29.472 INFO:teuthology.orchestra.run.smithi138.stdout: "finisher-PurgeQueue": { 2024-04-24T09:18:29.472 INFO:teuthology.orchestra.run.smithi138.stdout: "queue_len": 0, 2024-04-24T09:18:29.472 INFO:teuthology.orchestra.run.smithi138.stdout: "complete_latency": { 2024-04-24T09:18:29.472 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 18704, 2024-04-24T09:18:29.472 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 2.810978483, 2024-04-24T09:18:29.472 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000150287 2024-04-24T09:18:29.472 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-24T09:18:29.472 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:29.473 INFO:teuthology.orchestra.run.smithi138.stdout: "mds": { 2024-04-24T09:18:29.473 INFO:teuthology.orchestra.run.smithi138.stdout: "request": 326023, 2024-04-24T09:18:29.473 INFO:teuthology.orchestra.run.smithi138.stdout: "reply": 325320, 2024-04-24T09:18:29.473 INFO:teuthology.orchestra.run.smithi138.stdout: "reply_latency": { 2024-04-24T09:18:29.473 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 325320, 2024-04-24T09:18:29.473 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 301.166837450, 2024-04-24T09:18:29.473 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000925755 2024-04-24T09:18:29.473 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:29.473 INFO:teuthology.orchestra.run.smithi138.stdout: "slow_reply": 0, 2024-04-24T09:18:29.473 INFO:teuthology.orchestra.run.smithi138.stdout: "forward": 703, 2024-04-24T09:18:29.473 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_fetch_complete": 19, 2024-04-24T09:18:29.473 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_fetch_keys": 0, 2024-04-24T09:18:29.473 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_commit": 68, 2024-04-24T09:18:29.473 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_split": 7, 2024-04-24T09:18:29.473 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_merge": 0, 2024-04-24T09:18:29.474 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes": 1883, 2024-04-24T09:18:29.474 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_top": 233, 2024-04-24T09:18:29.474 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_bottom": 1576, 2024-04-24T09:18:29.474 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_pin_tail": 74, 2024-04-24T09:18:29.474 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_pinned": 1548, 2024-04-24T09:18:29.474 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_expired": 1028, 2024-04-24T09:18:29.474 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_with_caps": 1, 2024-04-24T09:18:29.474 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": 1, 2024-04-24T09:18:29.474 INFO:teuthology.orchestra.run.smithi138.stdout: "subtrees": 7, 2024-04-24T09:18:29.474 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse": 617261, 2024-04-24T09:18:29.474 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_hit": 608089, 2024-04-24T09:18:29.474 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_forward": 599, 2024-04-24T09:18:29.474 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_discover": 66, 2024-04-24T09:18:29.474 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_dir_fetch": 0, 2024-04-24T09:18:29.475 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_remote_ino": 0, 2024-04-24T09:18:29.475 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_lock": 0, 2024-04-24T09:18:29.475 INFO:teuthology.orchestra.run.smithi138.stdout: "load_cent": 0, 2024-04-24T09:18:29.475 INFO:teuthology.orchestra.run.smithi138.stdout: "q": 0, 2024-04-24T09:18:29.475 INFO:teuthology.orchestra.run.smithi138.stdout: "exported": 33, 2024-04-24T09:18:29.475 INFO:teuthology.orchestra.run.smithi138.stdout: "exported_inodes": 843, 2024-04-24T09:18:29.475 INFO:teuthology.orchestra.run.smithi138.stdout: "imported": 13, 2024-04-24T09:18:29.475 INFO:teuthology.orchestra.run.smithi138.stdout: "imported_inodes": 435, 2024-04-24T09:18:29.475 INFO:teuthology.orchestra.run.smithi138.stdout: "openino_dir_fetch": 0, 2024-04-24T09:18:29.475 INFO:teuthology.orchestra.run.smithi138.stdout: "openino_backtrace_fetch": 4290, 2024-04-24T09:18:29.475 INFO:teuthology.orchestra.run.smithi138.stdout: "openino_peer_discover": 0, 2024-04-24T09:18:29.475 INFO:teuthology.orchestra.run.smithi138.stdout: "root_rfiles": 4, 2024-04-24T09:18:29.475 INFO:teuthology.orchestra.run.smithi138.stdout: "root_rbytes": 234, 2024-04-24T09:18:29.475 INFO:teuthology.orchestra.run.smithi138.stdout: "root_rsnaps": 7, 2024-04-24T09:18:29.475 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_backtrace_fetch": 4290, 2024-04-24T09:18:29.475 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_set_tag": 0, 2024-04-24T09:18:29.476 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_backtrace_repaired": 0, 2024-04-24T09:18:29.476 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_inotable_repaired": 0, 2024-04-24T09:18:29.476 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_dir_inodes": 451, 2024-04-24T09:18:29.476 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_dir_base_inodes": 0, 2024-04-24T09:18:29.476 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_dirfrag_rstats": 451, 2024-04-24T09:18:29.476 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_file_inodes": 3839, 2024-04-24T09:18:29.476 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_inode_file_caps": 0, 2024-04-24T09:18:29.476 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_cap_op_revoke": 26634, 2024-04-24T09:18:29.476 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_cap_op_grant": 39671, 2024-04-24T09:18:29.476 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_cap_op_trunc": 0, 2024-04-24T09:18:29.476 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-24T09:18:29.476 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-24T09:18:29.476 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_client_caps": 41160, 2024-04-24T09:18:29.476 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_client_caps_dirty": 9990, 2024-04-24T09:18:29.476 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_client_cap_release": 893, 2024-04-24T09:18:29.476 INFO:teuthology.orchestra.run.smithi138.stdout: "process_request_cap_release": 28009 2024-04-24T09:18:29.477 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:29.477 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_cache": { 2024-04-24T09:18:29.477 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_update": 18, 2024-04-24T09:18:29.477 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_update_receipt": 58, 2024-04-24T09:18:29.477 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_try_discover": 44, 2024-04-24T09:18:29.477 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_send_discover": 89, 2024-04-24T09:18:29.477 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_handle_discover": 99, 2024-04-24T09:18:29.477 INFO:teuthology.orchestra.run.smithi138.stdout: "num_strays": 20, 2024-04-24T09:18:29.477 INFO:teuthology.orchestra.run.smithi138.stdout: "num_strays_delayed": 0, 2024-04-24T09:18:29.477 INFO:teuthology.orchestra.run.smithi138.stdout: "num_strays_enqueuing": 0, 2024-04-24T09:18:29.477 INFO:teuthology.orchestra.run.smithi138.stdout: "strays_created": 7692, 2024-04-24T09:18:29.477 INFO:teuthology.orchestra.run.smithi138.stdout: "strays_enqueued": 7672, 2024-04-24T09:18:29.477 INFO:teuthology.orchestra.run.smithi138.stdout: "strays_reintegrated": 0, 2024-04-24T09:18:29.477 INFO:teuthology.orchestra.run.smithi138.stdout: "strays_migrated": 0, 2024-04-24T09:18:29.477 INFO:teuthology.orchestra.run.smithi138.stdout: "num_recovering_processing": 0, 2024-04-24T09:18:29.478 INFO:teuthology.orchestra.run.smithi138.stdout: "num_recovering_enqueued": 0, 2024-04-24T09:18:29.478 INFO:teuthology.orchestra.run.smithi138.stdout: "num_recovering_prioritized": 0, 2024-04-24T09:18:29.478 INFO:teuthology.orchestra.run.smithi138.stdout: "recovery_started": 0, 2024-04-24T09:18:29.478 INFO:teuthology.orchestra.run.smithi138.stdout: "recovery_completed": 0, 2024-04-24T09:18:29.478 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_quiesce_path": 0, 2024-04-24T09:18:29.478 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_quiesce_inode": 0, 2024-04-24T09:18:29.478 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_enqueue_scrub": 0, 2024-04-24T09:18:29.478 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_exportdir": 33, 2024-04-24T09:18:29.478 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_flush": 0, 2024-04-24T09:18:29.478 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_fragmentdir": 7, 2024-04-24T09:18:29.478 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_fragstats": 0, 2024-04-24T09:18:29.478 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_inodestats": 0 2024-04-24T09:18:29.478 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:29.478 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_log": { 2024-04-24T09:18:29.478 INFO:teuthology.orchestra.run.smithi138.stdout: "evlrg": 0, 2024-04-24T09:18:29.479 INFO:teuthology.orchestra.run.smithi138.stdout: "evadd": 56121, 2024-04-24T09:18:29.479 INFO:teuthology.orchestra.run.smithi138.stdout: "evex": 0, 2024-04-24T09:18:29.479 INFO:teuthology.orchestra.run.smithi138.stdout: "evtrm": 0, 2024-04-24T09:18:29.479 INFO:teuthology.orchestra.run.smithi138.stdout: "ev": 56121, 2024-04-24T09:18:29.479 INFO:teuthology.orchestra.run.smithi138.stdout: "evexg": 0, 2024-04-24T09:18:29.479 INFO:teuthology.orchestra.run.smithi138.stdout: "evexd": 0, 2024-04-24T09:18:29.479 INFO:teuthology.orchestra.run.smithi138.stdout: "segadd": 88, 2024-04-24T09:18:29.479 INFO:teuthology.orchestra.run.smithi138.stdout: "segex": 0, 2024-04-24T09:18:29.479 INFO:teuthology.orchestra.run.smithi138.stdout: "segtrm": 0, 2024-04-24T09:18:29.479 INFO:teuthology.orchestra.run.smithi138.stdout: "seg": 88, 2024-04-24T09:18:29.479 INFO:teuthology.orchestra.run.smithi138.stdout: "segmjr": 6, 2024-04-24T09:18:29.479 INFO:teuthology.orchestra.run.smithi138.stdout: "segexg": 0, 2024-04-24T09:18:29.479 INFO:teuthology.orchestra.run.smithi138.stdout: "segexd": 0, 2024-04-24T09:18:29.479 INFO:teuthology.orchestra.run.smithi138.stdout: "expos": 4194304, 2024-04-24T09:18:29.479 INFO:teuthology.orchestra.run.smithi138.stdout: "wrpos": 272469094, 2024-04-24T09:18:29.479 INFO:teuthology.orchestra.run.smithi138.stdout: "rdpos": 0, 2024-04-24T09:18:29.480 INFO:teuthology.orchestra.run.smithi138.stdout: "jlat": { 2024-04-24T09:18:29.480 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 14925, 2024-04-24T09:18:29.480 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 96.419312581, 2024-04-24T09:18:29.480 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.006460255 2024-04-24T09:18:29.480 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:29.480 INFO:teuthology.orchestra.run.smithi138.stdout: "replayed": 0 2024-04-24T09:18:29.480 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:29.480 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_mem": { 2024-04-24T09:18:29.480 INFO:teuthology.orchestra.run.smithi138.stdout: "ino": 1592, 2024-04-24T09:18:29.480 INFO:teuthology.orchestra.run.smithi138.stdout: "ino+": 10157, 2024-04-24T09:18:29.480 INFO:teuthology.orchestra.run.smithi138.stdout: "ino-": 8565, 2024-04-24T09:18:29.480 INFO:teuthology.orchestra.run.smithi138.stdout: "dir": 79, 2024-04-24T09:18:29.480 INFO:teuthology.orchestra.run.smithi138.stdout: "dir+": 164, 2024-04-24T09:18:29.480 INFO:teuthology.orchestra.run.smithi138.stdout: "dir-": 85, 2024-04-24T09:18:29.480 INFO:teuthology.orchestra.run.smithi138.stdout: "dn": 1883, 2024-04-24T09:18:29.480 INFO:teuthology.orchestra.run.smithi138.stdout: "dn+": 11316, 2024-04-24T09:18:29.481 INFO:teuthology.orchestra.run.smithi138.stdout: "dn-": 9433, 2024-04-24T09:18:29.481 INFO:teuthology.orchestra.run.smithi138.stdout: "cap": 1, 2024-04-24T09:18:29.481 INFO:teuthology.orchestra.run.smithi138.stdout: "cap+": 7797, 2024-04-24T09:18:29.481 INFO:teuthology.orchestra.run.smithi138.stdout: "cap-": 7796, 2024-04-24T09:18:29.481 INFO:teuthology.orchestra.run.smithi138.stdout: "rss": 72956, 2024-04-24T09:18:29.481 INFO:teuthology.orchestra.run.smithi138.stdout: "heap": 223608 2024-04-24T09:18:29.481 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:29.481 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_server": { 2024-04-24T09:18:29.481 INFO:teuthology.orchestra.run.smithi138.stdout: "dispatch_client_request": 352465, 2024-04-24T09:18:29.481 INFO:teuthology.orchestra.run.smithi138.stdout: "dispatch_server_request": 499, 2024-04-24T09:18:29.481 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_client_request": 326023, 2024-04-24T09:18:29.481 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_client_session": 777, 2024-04-24T09:18:29.481 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_peer_request": 3317, 2024-04-24T09:18:29.481 INFO:teuthology.orchestra.run.smithi138.stdout: "req_create_latency": { 2024-04-24T09:18:29.481 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 7534, 2024-04-24T09:18:29.481 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 18.499261593, 2024-04-24T09:18:29.482 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.002455436 2024-04-24T09:18:29.482 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:29.482 INFO:teuthology.orchestra.run.smithi138.stdout: "req_getattr_latency": { 2024-04-24T09:18:29.482 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 278290, 2024-04-24T09:18:29.482 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 143.502092974, 2024-04-24T09:18:29.482 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000515656 2024-04-24T09:18:29.482 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:29.482 INFO:teuthology.orchestra.run.smithi138.stdout: "req_getfilelock_latency": { 2024-04-24T09:18:29.482 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-24T09:18:29.482 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-24T09:18:29.482 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-24T09:18:29.482 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:29.482 INFO:teuthology.orchestra.run.smithi138.stdout: "req_link_latency": { 2024-04-24T09:18:29.482 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-24T09:18:29.483 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-24T09:18:29.483 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-24T09:18:29.483 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:29.483 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookup_latency": { 2024-04-24T09:18:29.483 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 2493, 2024-04-24T09:18:29.483 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 6.644826809, 2024-04-24T09:18:29.483 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.002665393 2024-04-24T09:18:29.483 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:29.483 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookuphash_latency": { 2024-04-24T09:18:29.483 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-24T09:18:29.483 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-24T09:18:29.483 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-24T09:18:29.483 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:29.483 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookupino_latency": { 2024-04-24T09:18:29.483 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-24T09:18:29.484 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-24T09:18:29.484 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-24T09:18:29.484 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:29.484 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookupname_latency": { 2024-04-24T09:18:29.484 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-24T09:18:29.484 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-24T09:18:29.484 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-24T09:18:29.484 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:29.484 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookupparent_latency": { 2024-04-24T09:18:29.484 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-24T09:18:29.484 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-24T09:18:29.484 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-24T09:18:29.484 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:29.484 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookupsnap_latency": { 2024-04-24T09:18:29.484 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-24T09:18:29.485 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-24T09:18:29.485 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-24T09:18:29.485 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:29.485 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lssnap_latency": { 2024-04-24T09:18:29.485 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-24T09:18:29.485 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-24T09:18:29.485 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-24T09:18:29.486 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:29.486 INFO:teuthology.orchestra.run.smithi138.stdout: "req_mkdir_latency": { 2024-04-24T09:18:29.486 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-24T09:18:29.486 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-24T09:18:29.486 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-24T09:18:29.486 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:29.486 INFO:teuthology.orchestra.run.smithi138.stdout: "req_mknod_latency": { 2024-04-24T09:18:29.486 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-24T09:18:29.486 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-24T09:18:29.486 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-24T09:18:29.486 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:29.486 INFO:teuthology.orchestra.run.smithi138.stdout: "req_mksnap_latency": { 2024-04-24T09:18:29.486 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-24T09:18:29.486 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-24T09:18:29.487 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-24T09:18:29.487 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:29.487 INFO:teuthology.orchestra.run.smithi138.stdout: "req_open_latency": { 2024-04-24T09:18:29.487 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 5706, 2024-04-24T09:18:29.487 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 6.857118250, 2024-04-24T09:18:29.487 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.001201738 2024-04-24T09:18:29.487 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:29.487 INFO:teuthology.orchestra.run.smithi138.stdout: "req_readdir_latency": { 2024-04-24T09:18:29.487 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 21692, 2024-04-24T09:18:29.487 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 88.879807616, 2024-04-24T09:18:29.487 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.004097354 2024-04-24T09:18:29.487 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:29.487 INFO:teuthology.orchestra.run.smithi138.stdout: "req_rename_latency": { 2024-04-24T09:18:29.487 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 1697, 2024-04-24T09:18:29.487 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 7.332228478, 2024-04-24T09:18:29.488 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.004320700 2024-04-24T09:18:29.488 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:29.488 INFO:teuthology.orchestra.run.smithi138.stdout: "req_renamesnap_latency": { 2024-04-24T09:18:29.488 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-24T09:18:29.488 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-24T09:18:29.488 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-24T09:18:29.488 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:29.488 INFO:teuthology.orchestra.run.smithi138.stdout: "req_snapdiff_latency": { 2024-04-24T09:18:29.488 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-24T09:18:29.488 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-24T09:18:29.488 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-24T09:18:29.488 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:29.488 INFO:teuthology.orchestra.run.smithi138.stdout: "req_rmdir_latency": { 2024-04-24T09:18:29.488 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 20, 2024-04-24T09:18:29.488 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.241852070, 2024-04-24T09:18:29.489 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.012092603 2024-04-24T09:18:29.489 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:29.489 INFO:teuthology.orchestra.run.smithi138.stdout: "req_rmsnap_latency": { 2024-04-24T09:18:29.489 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-24T09:18:29.489 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-24T09:18:29.489 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-24T09:18:29.489 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:29.489 INFO:teuthology.orchestra.run.smithi138.stdout: "req_rmxattr_latency": { 2024-04-24T09:18:29.489 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-24T09:18:29.489 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-24T09:18:29.489 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-24T09:18:29.489 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:29.489 INFO:teuthology.orchestra.run.smithi138.stdout: "req_setattr_latency": { 2024-04-24T09:18:29.489 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-24T09:18:29.489 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-24T09:18:29.490 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-24T09:18:29.490 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:29.490 INFO:teuthology.orchestra.run.smithi138.stdout: "req_setdirlayout_latency": { 2024-04-24T09:18:29.490 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-24T09:18:29.490 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-24T09:18:29.490 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-24T09:18:29.490 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:29.490 INFO:teuthology.orchestra.run.smithi138.stdout: "req_setfilelock_latency": { 2024-04-24T09:18:29.490 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 216, 2024-04-24T09:18:29.490 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.115947739, 2024-04-24T09:18:29.490 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000536795 2024-04-24T09:18:29.490 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:29.490 INFO:teuthology.orchestra.run.smithi138.stdout: "req_setlayout_latency": { 2024-04-24T09:18:29.490 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-24T09:18:29.491 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-24T09:18:29.491 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-24T09:18:29.491 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:29.491 INFO:teuthology.orchestra.run.smithi138.stdout: "req_setxattr_latency": { 2024-04-24T09:18:29.491 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-24T09:18:29.491 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-24T09:18:29.491 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-24T09:18:29.491 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:29.491 INFO:teuthology.orchestra.run.smithi138.stdout: "req_symlink_latency": { 2024-04-24T09:18:29.491 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-24T09:18:29.491 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-24T09:18:29.491 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-24T09:18:29.491 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:29.491 INFO:teuthology.orchestra.run.smithi138.stdout: "req_unlink_latency": { 2024-04-24T09:18:29.491 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 7672, 2024-04-24T09:18:29.492 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 29.093701921, 2024-04-24T09:18:29.492 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.003792192 2024-04-24T09:18:29.492 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:29.492 INFO:teuthology.orchestra.run.smithi138.stdout: "cap_revoke_eviction": 0, 2024-04-24T09:18:29.492 INFO:teuthology.orchestra.run.smithi138.stdout: "cap_acquisition_throttle": 0, 2024-04-24T09:18:29.492 INFO:teuthology.orchestra.run.smithi138.stdout: "req_getvxattr_latency": { 2024-04-24T09:18:29.492 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-24T09:18:29.492 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-24T09:18:29.492 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-24T09:18:29.492 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-24T09:18:29.492 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:29.492 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_sessions": { 2024-04-24T09:18:29.492 INFO:teuthology.orchestra.run.smithi138.stdout: "session_count": 2, 2024-04-24T09:18:29.492 INFO:teuthology.orchestra.run.smithi138.stdout: "session_add": 2, 2024-04-24T09:18:29.492 INFO:teuthology.orchestra.run.smithi138.stdout: "session_remove": 0, 2024-04-24T09:18:29.493 INFO:teuthology.orchestra.run.smithi138.stdout: "sessions_open": 2, 2024-04-24T09:18:29.493 INFO:teuthology.orchestra.run.smithi138.stdout: "sessions_stale": 0, 2024-04-24T09:18:29.493 INFO:teuthology.orchestra.run.smithi138.stdout: "total_load": 35242, 2024-04-24T09:18:29.493 INFO:teuthology.orchestra.run.smithi138.stdout: "average_load": 17621, 2024-04-24T09:18:29.493 INFO:teuthology.orchestra.run.smithi138.stdout: "avg_session_uptime": 1601, 2024-04-24T09:18:29.493 INFO:teuthology.orchestra.run.smithi138.stdout: "mdthresh_evicted": 0 2024-04-24T09:18:29.493 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:29.493 INFO:teuthology.orchestra.run.smithi138.stdout: "mempool": { 2024-04-24T09:18:29.493 INFO:teuthology.orchestra.run.smithi138.stdout: "bloom_filter_bytes": 0, 2024-04-24T09:18:29.493 INFO:teuthology.orchestra.run.smithi138.stdout: "bloom_filter_items": 0, 2024-04-24T09:18:29.493 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_alloc_bytes": 0, 2024-04-24T09:18:29.493 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_alloc_items": 0, 2024-04-24T09:18:29.493 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_data_bytes": 0, 2024-04-24T09:18:29.493 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_data_items": 0, 2024-04-24T09:18:29.493 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-24T09:18:29.494 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_onode_items": 0, 2024-04-24T09:18:29.494 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-24T09:18:29.494 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_meta_items": 0, 2024-04-24T09:18:29.494 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_other_bytes": 0, 2024-04-24T09:18:29.494 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_other_items": 0, 2024-04-24T09:18:29.494 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-24T09:18:29.494 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_buffer_items": 0, 2024-04-24T09:18:29.494 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_extent_bytes": 0, 2024-04-24T09:18:29.494 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_extent_items": 0, 2024-04-24T09:18:29.494 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_blob_bytes": 0, 2024-04-24T09:18:29.494 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_blob_items": 0, 2024-04-24T09:18:29.494 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-24T09:18:29.494 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_shared_blob_items": 0, 2024-04-24T09:18:29.494 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-24T09:18:29.495 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_inline_bl_items": 0, 2024-04-24T09:18:29.495 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_fsck_bytes": 0, 2024-04-24T09:18:29.495 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_fsck_items": 0, 2024-04-24T09:18:29.495 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_txc_bytes": 0, 2024-04-24T09:18:29.495 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_txc_items": 0, 2024-04-24T09:18:29.495 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-24T09:18:29.495 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_writing_deferred_items": 0, 2024-04-24T09:18:29.495 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_writing_bytes": 0, 2024-04-24T09:18:29.495 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_writing_items": 0, 2024-04-24T09:18:29.495 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_bytes": 0, 2024-04-24T09:18:29.495 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_items": 0, 2024-04-24T09:18:29.495 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_file_reader_bytes": 0, 2024-04-24T09:18:29.495 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_file_reader_items": 0, 2024-04-24T09:18:29.495 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_file_writer_bytes": 0, 2024-04-24T09:18:29.495 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_file_writer_items": 0, 2024-04-24T09:18:29.495 INFO:teuthology.orchestra.run.smithi138.stdout: "buffer_anon_bytes": 484536, 2024-04-24T09:18:29.496 INFO:teuthology.orchestra.run.smithi138.stdout: "buffer_anon_items": 170, 2024-04-24T09:18:29.496 INFO:teuthology.orchestra.run.smithi138.stdout: "buffer_meta_bytes": 0, 2024-04-24T09:18:29.496 INFO:teuthology.orchestra.run.smithi138.stdout: "buffer_meta_items": 0, 2024-04-24T09:18:29.496 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_bytes": 0, 2024-04-24T09:18:29.496 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_items": 0, 2024-04-24T09:18:29.496 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_mapbl_bytes": 0, 2024-04-24T09:18:29.496 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_mapbl_items": 0, 2024-04-24T09:18:29.496 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pglog_bytes": 0, 2024-04-24T09:18:29.496 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pglog_items": 0, 2024-04-24T09:18:29.496 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_bytes": 9800, 2024-04-24T09:18:29.496 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_items": 188, 2024-04-24T09:18:29.496 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_mapping_bytes": 0, 2024-04-24T09:18:29.496 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_mapping_items": 0, 2024-04-24T09:18:29.496 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap_bytes": 0, 2024-04-24T09:18:29.496 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap_items": 0, 2024-04-24T09:18:29.497 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_co_bytes": 5141344, 2024-04-24T09:18:29.497 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_co_items": 22671, 2024-04-24T09:18:29.497 INFO:teuthology.orchestra.run.smithi138.stdout: "unittest_1_bytes": 0, 2024-04-24T09:18:29.497 INFO:teuthology.orchestra.run.smithi138.stdout: "unittest_1_items": 0, 2024-04-24T09:18:29.497 INFO:teuthology.orchestra.run.smithi138.stdout: "unittest_2_bytes": 0, 2024-04-24T09:18:29.497 INFO:teuthology.orchestra.run.smithi138.stdout: "unittest_2_items": 0 2024-04-24T09:18:29.497 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:29.497 INFO:teuthology.orchestra.run.smithi138.stdout: "objecter": { 2024-04-24T09:18:29.497 INFO:teuthology.orchestra.run.smithi138.stdout: "op_active": 0, 2024-04-24T09:18:29.497 INFO:teuthology.orchestra.run.smithi138.stdout: "op_laggy": 0, 2024-04-24T09:18:29.497 INFO:teuthology.orchestra.run.smithi138.stdout: "op_send": 40027, 2024-04-24T09:18:29.497 INFO:teuthology.orchestra.run.smithi138.stdout: "op_send_bytes": 270377262, 2024-04-24T09:18:29.497 INFO:teuthology.orchestra.run.smithi138.stdout: "op_resend": 33, 2024-04-24T09:18:29.497 INFO:teuthology.orchestra.run.smithi138.stdout: "op_reply": 40027, 2024-04-24T09:18:29.497 INFO:teuthology.orchestra.run.smithi138.stdout: "op_latency": { 2024-04-24T09:18:29.497 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 40027, 2024-04-24T09:18:29.497 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 295.450037968, 2024-04-24T09:18:29.498 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.007381268 2024-04-24T09:18:29.498 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:29.498 INFO:teuthology.orchestra.run.smithi138.stdout: "op_inflight": 0, 2024-04-24T09:18:29.498 INFO:teuthology.orchestra.run.smithi138.stdout: "oplen_avg": { 2024-04-24T09:18:29.498 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 40027, 2024-04-24T09:18:29.498 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 41975 2024-04-24T09:18:29.498 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:29.498 INFO:teuthology.orchestra.run.smithi138.stdout: "op": 40027, 2024-04-24T09:18:29.498 INFO:teuthology.orchestra.run.smithi138.stdout: "op_r": 5942, 2024-04-24T09:18:29.498 INFO:teuthology.orchestra.run.smithi138.stdout: "op_w": 34085, 2024-04-24T09:18:29.498 INFO:teuthology.orchestra.run.smithi138.stdout: "op_rmw": 0, 2024-04-24T09:18:29.498 INFO:teuthology.orchestra.run.smithi138.stdout: "op_pg": 0, 2024-04-24T09:18:29.498 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_stat": 0, 2024-04-24T09:18:29.498 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_create": 10, 2024-04-24T09:18:29.498 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_read": 1624, 2024-04-24T09:18:29.499 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_write": 16549, 2024-04-24T09:18:29.499 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_writefull": 963, 2024-04-24T09:18:29.499 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_writesame": 0, 2024-04-24T09:18:29.499 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_append": 0, 2024-04-24T09:18:29.499 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_zero": 0, 2024-04-24T09:18:29.499 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_truncate": 7, 2024-04-24T09:18:29.499 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_delete": 15419, 2024-04-24T09:18:29.499 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_mapext": 0, 2024-04-24T09:18:29.499 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_sparse_read": 0, 2024-04-24T09:18:29.499 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_clonerange": 0, 2024-04-24T09:18:29.499 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_getxattr": 4293, 2024-04-24T09:18:29.499 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_setxattr": 20, 2024-04-24T09:18:29.499 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_cmpxattr": 0, 2024-04-24T09:18:29.499 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_rmxattr": 0, 2024-04-24T09:18:29.499 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_resetxattrs": 0, 2024-04-24T09:18:29.499 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_call": 0, 2024-04-24T09:18:29.500 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_watch": 0, 2024-04-24T09:18:29.500 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_notify": 0, 2024-04-24T09:18:29.500 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_src_cmpxattr": 0, 2024-04-24T09:18:29.500 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_pgls": 0, 2024-04-24T09:18:29.500 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_pgls_filter": 0, 2024-04-24T09:18:29.500 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_other": 0, 2024-04-24T09:18:29.500 INFO:teuthology.orchestra.run.smithi138.stdout: "linger_active": 0, 2024-04-24T09:18:29.500 INFO:teuthology.orchestra.run.smithi138.stdout: "linger_send": 0, 2024-04-24T09:18:29.500 INFO:teuthology.orchestra.run.smithi138.stdout: "linger_resend": 0, 2024-04-24T09:18:29.500 INFO:teuthology.orchestra.run.smithi138.stdout: "linger_ping": 0, 2024-04-24T09:18:29.500 INFO:teuthology.orchestra.run.smithi138.stdout: "poolop_active": 0, 2024-04-24T09:18:29.500 INFO:teuthology.orchestra.run.smithi138.stdout: "poolop_send": 0, 2024-04-24T09:18:29.500 INFO:teuthology.orchestra.run.smithi138.stdout: "poolop_resend": 0, 2024-04-24T09:18:29.500 INFO:teuthology.orchestra.run.smithi138.stdout: "poolstat_active": 0, 2024-04-24T09:18:29.500 INFO:teuthology.orchestra.run.smithi138.stdout: "poolstat_send": 0, 2024-04-24T09:18:29.501 INFO:teuthology.orchestra.run.smithi138.stdout: "poolstat_resend": 0, 2024-04-24T09:18:29.501 INFO:teuthology.orchestra.run.smithi138.stdout: "statfs_active": 0, 2024-04-24T09:18:29.501 INFO:teuthology.orchestra.run.smithi138.stdout: "statfs_send": 0, 2024-04-24T09:18:29.501 INFO:teuthology.orchestra.run.smithi138.stdout: "statfs_resend": 0, 2024-04-24T09:18:29.501 INFO:teuthology.orchestra.run.smithi138.stdout: "command_active": 0, 2024-04-24T09:18:29.501 INFO:teuthology.orchestra.run.smithi138.stdout: "command_send": 0, 2024-04-24T09:18:29.501 INFO:teuthology.orchestra.run.smithi138.stdout: "command_resend": 0, 2024-04-24T09:18:29.501 INFO:teuthology.orchestra.run.smithi138.stdout: "map_epoch": 136, 2024-04-24T09:18:29.501 INFO:teuthology.orchestra.run.smithi138.stdout: "map_full": 0, 2024-04-24T09:18:29.501 INFO:teuthology.orchestra.run.smithi138.stdout: "map_inc": 47, 2024-04-24T09:18:29.501 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_sessions": 12, 2024-04-24T09:18:29.501 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_session_open": 12, 2024-04-24T09:18:29.501 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_session_close": 0, 2024-04-24T09:18:29.501 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_laggy": 0, 2024-04-24T09:18:29.501 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_wr": 2082, 2024-04-24T09:18:29.502 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_rd": 47, 2024-04-24T09:18:29.502 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_del": 961 2024-04-24T09:18:29.502 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:29.502 INFO:teuthology.orchestra.run.smithi138.stdout: "oft": { 2024-04-24T09:18:29.502 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_total_objs": 1, 2024-04-24T09:18:29.502 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_total_kv_pairs": 0, 2024-04-24T09:18:29.502 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_total_updates": 0, 2024-04-24T09:18:29.502 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_total_removes": 0 2024-04-24T09:18:29.502 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:29.502 INFO:teuthology.orchestra.run.smithi138.stdout: "osd-slow-ops": { 2024-04-24T09:18:29.502 INFO:teuthology.orchestra.run.smithi138.stdout: "slow_ops_count": 0 2024-04-24T09:18:29.502 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:29.502 INFO:teuthology.orchestra.run.smithi138.stdout: "purge_queue": { 2024-04-24T09:18:29.502 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executing_ops": 0, 2024-04-24T09:18:29.502 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executing_ops_high_water": 10, 2024-04-24T09:18:29.503 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executing": 0, 2024-04-24T09:18:29.503 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executing_high_water": 5, 2024-04-24T09:18:29.503 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executed_ops": 15344, 2024-04-24T09:18:29.503 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executed": 7672, 2024-04-24T09:18:29.503 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_item_in_journal": 0 2024-04-24T09:18:29.503 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:29.503 INFO:teuthology.orchestra.run.smithi138.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-24T09:18:29.503 INFO:teuthology.orchestra.run.smithi138.stdout: "val": 0, 2024-04-24T09:18:29.503 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 104857600, 2024-04-24T09:18:29.503 INFO:teuthology.orchestra.run.smithi138.stdout: "get_started": 0, 2024-04-24T09:18:29.503 INFO:teuthology.orchestra.run.smithi138.stdout: "get": 465111, 2024-04-24T09:18:29.503 INFO:teuthology.orchestra.run.smithi138.stdout: "get_sum": 106118217, 2024-04-24T09:18:29.503 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_fail": 0, 2024-04-24T09:18:29.503 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_success": 465111, 2024-04-24T09:18:29.504 INFO:teuthology.orchestra.run.smithi138.stdout: "take": 0, 2024-04-24T09:18:29.504 INFO:teuthology.orchestra.run.smithi138.stdout: "take_sum": 0, 2024-04-24T09:18:29.504 INFO:teuthology.orchestra.run.smithi138.stdout: "put": 465111, 2024-04-24T09:18:29.504 INFO:teuthology.orchestra.run.smithi138.stdout: "put_sum": 106118217, 2024-04-24T09:18:29.504 INFO:teuthology.orchestra.run.smithi138.stdout: "wait": { 2024-04-24T09:18:29.504 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-24T09:18:29.504 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-24T09:18:29.504 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-24T09:18:29.504 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-24T09:18:29.504 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:29.504 INFO:teuthology.orchestra.run.smithi138.stdout: "throttle-objecter_bytes": { 2024-04-24T09:18:29.504 INFO:teuthology.orchestra.run.smithi138.stdout: "val": 0, 2024-04-24T09:18:29.504 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 104857600, 2024-04-24T09:18:29.504 INFO:teuthology.orchestra.run.smithi138.stdout: "get_started": 0, 2024-04-24T09:18:29.504 INFO:teuthology.orchestra.run.smithi138.stdout: "get": 0, 2024-04-24T09:18:29.504 INFO:teuthology.orchestra.run.smithi138.stdout: "get_sum": 0, 2024-04-24T09:18:29.505 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_fail": 0, 2024-04-24T09:18:29.505 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_success": 0, 2024-04-24T09:18:29.505 INFO:teuthology.orchestra.run.smithi138.stdout: "take": 40027, 2024-04-24T09:18:29.505 INFO:teuthology.orchestra.run.smithi138.stdout: "take_sum": 271222571, 2024-04-24T09:18:29.505 INFO:teuthology.orchestra.run.smithi138.stdout: "put": 40027, 2024-04-24T09:18:29.505 INFO:teuthology.orchestra.run.smithi138.stdout: "put_sum": 271222571, 2024-04-24T09:18:29.505 INFO:teuthology.orchestra.run.smithi138.stdout: "wait": { 2024-04-24T09:18:29.505 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-24T09:18:29.505 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-24T09:18:29.505 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-24T09:18:29.505 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-24T09:18:29.505 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:29.505 INFO:teuthology.orchestra.run.smithi138.stdout: "throttle-objecter_ops": { 2024-04-24T09:18:29.505 INFO:teuthology.orchestra.run.smithi138.stdout: "val": 0, 2024-04-24T09:18:29.505 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 1024, 2024-04-24T09:18:29.505 INFO:teuthology.orchestra.run.smithi138.stdout: "get_started": 0, 2024-04-24T09:18:29.506 INFO:teuthology.orchestra.run.smithi138.stdout: "get": 0, 2024-04-24T09:18:29.506 INFO:teuthology.orchestra.run.smithi138.stdout: "get_sum": 0, 2024-04-24T09:18:29.506 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_fail": 0, 2024-04-24T09:18:29.506 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_success": 0, 2024-04-24T09:18:29.506 INFO:teuthology.orchestra.run.smithi138.stdout: "take": 40027, 2024-04-24T09:18:29.506 INFO:teuthology.orchestra.run.smithi138.stdout: "take_sum": 40027, 2024-04-24T09:18:29.506 INFO:teuthology.orchestra.run.smithi138.stdout: "put": 40027, 2024-04-24T09:18:29.506 INFO:teuthology.orchestra.run.smithi138.stdout: "put_sum": 40027, 2024-04-24T09:18:29.506 INFO:teuthology.orchestra.run.smithi138.stdout: "wait": { 2024-04-24T09:18:29.506 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-24T09:18:29.506 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-24T09:18:29.506 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-24T09:18:29.506 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-24T09:18:29.506 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:29.507 INFO:teuthology.orchestra.run.smithi138.stdout: "throttle-write_buf_throttle": { 2024-04-24T09:18:29.507 INFO:teuthology.orchestra.run.smithi138.stdout: "val": 0, 2024-04-24T09:18:29.507 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 3758096384, 2024-04-24T09:18:29.507 INFO:teuthology.orchestra.run.smithi138.stdout: "get_started": 0, 2024-04-24T09:18:29.507 INFO:teuthology.orchestra.run.smithi138.stdout: "get": 7672, 2024-04-24T09:18:29.507 INFO:teuthology.orchestra.run.smithi138.stdout: "get_sum": 845888, 2024-04-24T09:18:29.507 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_fail": 0, 2024-04-24T09:18:29.507 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_success": 7672, 2024-04-24T09:18:29.507 INFO:teuthology.orchestra.run.smithi138.stdout: "take": 0, 2024-04-24T09:18:29.507 INFO:teuthology.orchestra.run.smithi138.stdout: "take_sum": 0, 2024-04-24T09:18:29.507 INFO:teuthology.orchestra.run.smithi138.stdout: "put": 1624, 2024-04-24T09:18:29.507 INFO:teuthology.orchestra.run.smithi138.stdout: "put_sum": 845888, 2024-04-24T09:18:29.507 INFO:teuthology.orchestra.run.smithi138.stdout: "wait": { 2024-04-24T09:18:29.507 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-24T09:18:29.507 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-24T09:18:29.507 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-24T09:18:29.508 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-24T09:18:29.508 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:29.508 INFO:teuthology.orchestra.run.smithi138.stdout: "throttle-write_buf_throttle-0x56222b2ea500": { 2024-04-24T09:18:29.508 INFO:teuthology.orchestra.run.smithi138.stdout: "val": 0, 2024-04-24T09:18:29.508 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 3758096384, 2024-04-24T09:18:29.508 INFO:teuthology.orchestra.run.smithi138.stdout: "get_started": 0, 2024-04-24T09:18:29.508 INFO:teuthology.orchestra.run.smithi138.stdout: "get": 56121, 2024-04-24T09:18:29.508 INFO:teuthology.orchestra.run.smithi138.stdout: "get_sum": 268274790, 2024-04-24T09:18:29.508 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_fail": 0, 2024-04-24T09:18:29.508 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_success": 56121, 2024-04-24T09:18:29.508 INFO:teuthology.orchestra.run.smithi138.stdout: "take": 0, 2024-04-24T09:18:29.508 INFO:teuthology.orchestra.run.smithi138.stdout: "take_sum": 0, 2024-04-24T09:18:29.508 INFO:teuthology.orchestra.run.smithi138.stdout: "put": 14925, 2024-04-24T09:18:29.508 INFO:teuthology.orchestra.run.smithi138.stdout: "put_sum": 268274790, 2024-04-24T09:18:29.508 INFO:teuthology.orchestra.run.smithi138.stdout: "wait": { 2024-04-24T09:18:29.508 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-24T09:18:29.508 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-24T09:18:29.509 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-24T09:18:29.509 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-24T09:18:29.509 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-24T09:18:29.509 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-04-24T09:18:30.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:18:30 smithi138 ceph-mon[31441]: pgmap v851: 129 pgs: 129 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 12 op/s 2024-04-24T09:18:30.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:18:30 smithi189 ceph-mon[31791]: pgmap v851: 129 pgs: 129 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 12 op/s 2024-04-24T09:18:30.492 INFO:tasks.check_counter:Daemon mds.e mds.exported=33 2024-04-24T09:18:30.492 INFO:tasks.check_counter:Daemon mds.e mds.imported=13 2024-04-24T09:18:30.492 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=7 2024-04-24T09:18:30.492 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-04-24T09:18:30.492 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-04-24T09:18:30.492 DEBUG:tasks.check_counter:Getting stats from h 2024-04-24T09:18:30.493 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid bd3e039e-0215-11ef-bc93-c7b262605968 -- ceph daemon mds.h perf dump 2024-04-24T09:18:30.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:18:30 smithi083 ceph-mon[27907]: pgmap v851: 129 pgs: 129 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 12 op/s 2024-04-24T09:18:30.975 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T09:18:32.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:18:32 smithi083 ceph-mon[27907]: pgmap v852: 129 pgs: 129 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-24T09:18:32.398 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:18:32 smithi138 ceph-mon[31441]: pgmap v852: 129 pgs: 129 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-24T09:18:32.458 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:18:32 smithi189 ceph-mon[31791]: pgmap v852: 129 pgs: 129 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-24T09:18:33.434 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-04-24T09:18:33.434 INFO:teuthology.orchestra.run.smithi138.stdout: "AsyncMessenger::Worker-0": { 2024-04-24T09:18:33.434 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_messages": 397291, 2024-04-24T09:18:33.434 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages": 450809, 2024-04-24T09:18:33.435 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_bytes": 118060253, 2024-04-24T09:18:33.435 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_bytes": 402970483, 2024-04-24T09:18:33.435 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_created_connections": 101, 2024-04-24T09:18:33.435 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_active_connections": 6, 2024-04-24T09:18:33.435 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_total_time": 45.273101042, 2024-04-24T09:18:33.435 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_send_time": 19.453368142, 2024-04-24T09:18:33.435 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_recv_time": 29.206295827, 2024-04-24T09:18:33.435 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_fast_dispatch_time": 1.429737565, 2024-04-24T09:18:33.435 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T09:18:33.435 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 450803, 2024-04-24T09:18:33.435 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 33.625433819, 2024-04-24T09:18:33.435 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000074590 2024-04-24T09:18:33.435 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:33.435 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_handle_ack_lat": { 2024-04-24T09:18:33.435 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 794037, 2024-04-24T09:18:33.436 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.237462126, 2024-04-24T09:18:33.436 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000299 2024-04-24T09:18:33.436 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:33.436 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_encrypted_bytes": 4752, 2024-04-24T09:18:33.436 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_encrypted_bytes": 17648 2024-04-24T09:18:33.436 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:33.436 INFO:teuthology.orchestra.run.smithi138.stdout: "AsyncMessenger::Worker-1": { 2024-04-24T09:18:33.436 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_messages": 72028, 2024-04-24T09:18:33.436 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages": 102110, 2024-04-24T09:18:33.436 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_bytes": 19496059, 2024-04-24T09:18:33.436 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_bytes": 157984420, 2024-04-24T09:18:33.436 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_created_connections": 13, 2024-04-24T09:18:33.436 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_active_connections": 6, 2024-04-24T09:18:33.436 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_total_time": 15.170706720, 2024-04-24T09:18:33.436 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_send_time": 8.120353628, 2024-04-24T09:18:33.436 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_recv_time": 7.053373842, 2024-04-24T09:18:33.437 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_fast_dispatch_time": 1.226838041, 2024-04-24T09:18:33.437 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T09:18:33.437 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 102113, 2024-04-24T09:18:33.437 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 10.189288879, 2024-04-24T09:18:33.437 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000099784 2024-04-24T09:18:33.437 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:33.437 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_handle_ack_lat": { 2024-04-24T09:18:33.437 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 145462, 2024-04-24T09:18:33.437 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.048502400, 2024-04-24T09:18:33.437 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000333 2024-04-24T09:18:33.437 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:33.437 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_encrypted_bytes": 2113952, 2024-04-24T09:18:33.437 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_encrypted_bytes": 364736 2024-04-24T09:18:33.437 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:33.438 INFO:teuthology.orchestra.run.smithi138.stdout: "AsyncMessenger::Worker-2": { 2024-04-24T09:18:33.438 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_messages": 35759, 2024-04-24T09:18:33.438 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages": 35752, 2024-04-24T09:18:33.438 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_bytes": 10965054, 2024-04-24T09:18:33.438 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_bytes": 127564174, 2024-04-24T09:18:33.438 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_created_connections": 17, 2024-04-24T09:18:33.438 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_active_connections": 7, 2024-04-24T09:18:33.438 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_total_time": 6.308387277, 2024-04-24T09:18:33.438 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_send_time": 1.708588210, 2024-04-24T09:18:33.438 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_recv_time": 1.665700612, 2024-04-24T09:18:33.438 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_fast_dispatch_time": 2.862564279, 2024-04-24T09:18:33.438 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T09:18:33.438 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 35750, 2024-04-24T09:18:33.438 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 4.250768582, 2024-04-24T09:18:33.438 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000118902 2024-04-24T09:18:33.439 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:33.439 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_handle_ack_lat": { 2024-04-24T09:18:33.439 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 27654, 2024-04-24T09:18:33.439 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.007714663, 2024-04-24T09:18:33.439 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000278 2024-04-24T09:18:33.439 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:33.439 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_encrypted_bytes": 5440, 2024-04-24T09:18:33.439 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_encrypted_bytes": 753984 2024-04-24T09:18:33.439 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:33.439 INFO:teuthology.orchestra.run.smithi138.stdout: "cct": { 2024-04-24T09:18:33.439 INFO:teuthology.orchestra.run.smithi138.stdout: "total_workers": 1, 2024-04-24T09:18:33.439 INFO:teuthology.orchestra.run.smithi138.stdout: "unhealthy_workers": 0 2024-04-24T09:18:33.439 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:33.439 INFO:teuthology.orchestra.run.smithi138.stdout: "finisher-MDSRank": { 2024-04-24T09:18:33.439 INFO:teuthology.orchestra.run.smithi138.stdout: "queue_len": 0, 2024-04-24T09:18:33.439 INFO:teuthology.orchestra.run.smithi138.stdout: "complete_latency": { 2024-04-24T09:18:33.440 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 38068, 2024-04-24T09:18:33.440 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 48.453178892, 2024-04-24T09:18:33.440 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.001272806 2024-04-24T09:18:33.440 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-24T09:18:33.440 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:33.440 INFO:teuthology.orchestra.run.smithi138.stdout: "finisher-PurgeQueue": { 2024-04-24T09:18:33.440 INFO:teuthology.orchestra.run.smithi138.stdout: "queue_len": 0, 2024-04-24T09:18:33.440 INFO:teuthology.orchestra.run.smithi138.stdout: "complete_latency": { 2024-04-24T09:18:33.440 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 20425, 2024-04-24T09:18:33.440 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 2.618900480, 2024-04-24T09:18:33.440 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000128220 2024-04-24T09:18:33.440 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-24T09:18:33.440 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:33.440 INFO:teuthology.orchestra.run.smithi138.stdout: "mds": { 2024-04-24T09:18:33.440 INFO:teuthology.orchestra.run.smithi138.stdout: "request": 330336, 2024-04-24T09:18:33.441 INFO:teuthology.orchestra.run.smithi138.stdout: "reply": 330282, 2024-04-24T09:18:33.441 INFO:teuthology.orchestra.run.smithi138.stdout: "reply_latency": { 2024-04-24T09:18:33.441 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 330282, 2024-04-24T09:18:33.441 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 327.072538157, 2024-04-24T09:18:33.441 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000990282 2024-04-24T09:18:33.441 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:33.441 INFO:teuthology.orchestra.run.smithi138.stdout: "slow_reply": 0, 2024-04-24T09:18:33.441 INFO:teuthology.orchestra.run.smithi138.stdout: "forward": 54, 2024-04-24T09:18:33.441 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_fetch_complete": 12, 2024-04-24T09:18:33.441 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_fetch_keys": 0, 2024-04-24T09:18:33.441 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_commit": 89, 2024-04-24T09:18:33.441 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_split": 9, 2024-04-24T09:18:33.441 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_merge": 5, 2024-04-24T09:18:33.441 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes": 2528, 2024-04-24T09:18:33.442 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_top": 673, 2024-04-24T09:18:33.442 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_bottom": 1825, 2024-04-24T09:18:33.442 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_pin_tail": 30, 2024-04-24T09:18:33.442 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_pinned": 1564, 2024-04-24T09:18:33.442 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_expired": 874, 2024-04-24T09:18:33.442 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_with_caps": 2, 2024-04-24T09:18:33.442 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": 2, 2024-04-24T09:18:33.442 INFO:teuthology.orchestra.run.smithi138.stdout: "subtrees": 6, 2024-04-24T09:18:33.442 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse": 625307, 2024-04-24T09:18:33.442 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_hit": 602102, 2024-04-24T09:18:33.442 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_forward": 44, 2024-04-24T09:18:33.442 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_discover": 36, 2024-04-24T09:18:33.442 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_dir_fetch": 1, 2024-04-24T09:18:33.442 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_remote_ino": 0, 2024-04-24T09:18:33.443 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_lock": 163, 2024-04-24T09:18:33.443 INFO:teuthology.orchestra.run.smithi138.stdout: "load_cent": 0, 2024-04-24T09:18:33.443 INFO:teuthology.orchestra.run.smithi138.stdout: "q": 0, 2024-04-24T09:18:33.443 INFO:teuthology.orchestra.run.smithi138.stdout: "exported": 29, 2024-04-24T09:18:33.443 INFO:teuthology.orchestra.run.smithi138.stdout: "exported_inodes": 754, 2024-04-24T09:18:33.443 INFO:teuthology.orchestra.run.smithi138.stdout: "imported": 23, 2024-04-24T09:18:33.443 INFO:teuthology.orchestra.run.smithi138.stdout: "imported_inodes": 1010, 2024-04-24T09:18:33.443 INFO:teuthology.orchestra.run.smithi138.stdout: "openino_dir_fetch": 0, 2024-04-24T09:18:33.443 INFO:teuthology.orchestra.run.smithi138.stdout: "openino_backtrace_fetch": 4532, 2024-04-24T09:18:33.443 INFO:teuthology.orchestra.run.smithi138.stdout: "openino_peer_discover": 0, 2024-04-24T09:18:33.443 INFO:teuthology.orchestra.run.smithi138.stdout: "root_rfiles": 4, 2024-04-24T09:18:33.443 INFO:teuthology.orchestra.run.smithi138.stdout: "root_rbytes": 234, 2024-04-24T09:18:33.443 INFO:teuthology.orchestra.run.smithi138.stdout: "root_rsnaps": 7, 2024-04-24T09:18:33.443 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_backtrace_fetch": 4532, 2024-04-24T09:18:33.444 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_set_tag": 0, 2024-04-24T09:18:33.444 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_backtrace_repaired": 0, 2024-04-24T09:18:33.444 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_inotable_repaired": 0, 2024-04-24T09:18:33.444 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_dir_inodes": 420, 2024-04-24T09:18:33.444 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_dir_base_inodes": 0, 2024-04-24T09:18:33.444 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_dirfrag_rstats": 420, 2024-04-24T09:18:33.444 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_file_inodes": 4112, 2024-04-24T09:18:33.444 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_inode_file_caps": 2400, 2024-04-24T09:18:33.444 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_cap_op_revoke": 23205, 2024-04-24T09:18:33.444 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_cap_op_grant": 33803, 2024-04-24T09:18:33.444 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_cap_op_trunc": 0, 2024-04-24T09:18:33.444 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_cap_op_flushsnap_ack": 1, 2024-04-24T09:18:33.444 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-24T09:18:33.444 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_client_caps": 40079, 2024-04-24T09:18:33.445 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_client_caps_dirty": 12036, 2024-04-24T09:18:33.445 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_client_cap_release": 1007, 2024-04-24T09:18:33.445 INFO:teuthology.orchestra.run.smithi138.stdout: "process_request_cap_release": 29850 2024-04-24T09:18:33.445 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:33.445 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_cache": { 2024-04-24T09:18:33.445 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_update": 12, 2024-04-24T09:18:33.445 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_update_receipt": 35, 2024-04-24T09:18:33.445 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_try_discover": 26, 2024-04-24T09:18:33.445 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_send_discover": 53, 2024-04-24T09:18:33.445 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_handle_discover": 95, 2024-04-24T09:18:33.445 INFO:teuthology.orchestra.run.smithi138.stdout: "num_strays": 20, 2024-04-24T09:18:33.445 INFO:teuthology.orchestra.run.smithi138.stdout: "num_strays_delayed": 0, 2024-04-24T09:18:33.445 INFO:teuthology.orchestra.run.smithi138.stdout: "num_strays_enqueuing": 0, 2024-04-24T09:18:33.445 INFO:teuthology.orchestra.run.smithi138.stdout: "strays_created": 8170, 2024-04-24T09:18:33.446 INFO:teuthology.orchestra.run.smithi138.stdout: "strays_enqueued": 8150, 2024-04-24T09:18:33.446 INFO:teuthology.orchestra.run.smithi138.stdout: "strays_reintegrated": 0, 2024-04-24T09:18:33.446 INFO:teuthology.orchestra.run.smithi138.stdout: "strays_migrated": 0, 2024-04-24T09:18:33.446 INFO:teuthology.orchestra.run.smithi138.stdout: "num_recovering_processing": 0, 2024-04-24T09:18:33.446 INFO:teuthology.orchestra.run.smithi138.stdout: "num_recovering_enqueued": 0, 2024-04-24T09:18:33.446 INFO:teuthology.orchestra.run.smithi138.stdout: "num_recovering_prioritized": 0, 2024-04-24T09:18:33.446 INFO:teuthology.orchestra.run.smithi138.stdout: "recovery_started": 0, 2024-04-24T09:18:33.446 INFO:teuthology.orchestra.run.smithi138.stdout: "recovery_completed": 0, 2024-04-24T09:18:33.446 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_quiesce_path": 0, 2024-04-24T09:18:33.446 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_quiesce_inode": 0, 2024-04-24T09:18:33.446 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_enqueue_scrub": 0, 2024-04-24T09:18:33.447 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_exportdir": 33, 2024-04-24T09:18:33.447 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_flush": 0, 2024-04-24T09:18:33.447 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_fragmentdir": 14, 2024-04-24T09:18:33.447 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_fragstats": 0, 2024-04-24T09:18:33.447 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_inodestats": 0 2024-04-24T09:18:33.447 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:33.447 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_log": { 2024-04-24T09:18:33.447 INFO:teuthology.orchestra.run.smithi138.stdout: "evlrg": 0, 2024-04-24T09:18:33.447 INFO:teuthology.orchestra.run.smithi138.stdout: "evadd": 63657, 2024-04-24T09:18:33.447 INFO:teuthology.orchestra.run.smithi138.stdout: "evex": 0, 2024-04-24T09:18:33.447 INFO:teuthology.orchestra.run.smithi138.stdout: "evtrm": 0, 2024-04-24T09:18:33.447 INFO:teuthology.orchestra.run.smithi138.stdout: "ev": 63657, 2024-04-24T09:18:33.447 INFO:teuthology.orchestra.run.smithi138.stdout: "evexg": 0, 2024-04-24T09:18:33.447 INFO:teuthology.orchestra.run.smithi138.stdout: "evexd": 0, 2024-04-24T09:18:33.447 INFO:teuthology.orchestra.run.smithi138.stdout: "segadd": 98, 2024-04-24T09:18:33.448 INFO:teuthology.orchestra.run.smithi138.stdout: "segex": 0, 2024-04-24T09:18:33.448 INFO:teuthology.orchestra.run.smithi138.stdout: "segtrm": 0, 2024-04-24T09:18:33.448 INFO:teuthology.orchestra.run.smithi138.stdout: "seg": 98, 2024-04-24T09:18:33.448 INFO:teuthology.orchestra.run.smithi138.stdout: "segmjr": 7, 2024-04-24T09:18:33.448 INFO:teuthology.orchestra.run.smithi138.stdout: "segexg": 0, 2024-04-24T09:18:33.448 INFO:teuthology.orchestra.run.smithi138.stdout: "segexd": 0, 2024-04-24T09:18:33.448 INFO:teuthology.orchestra.run.smithi138.stdout: "expos": 4194304, 2024-04-24T09:18:33.448 INFO:teuthology.orchestra.run.smithi138.stdout: "wrpos": 336811422, 2024-04-24T09:18:33.448 INFO:teuthology.orchestra.run.smithi138.stdout: "rdpos": 0, 2024-04-24T09:18:33.448 INFO:teuthology.orchestra.run.smithi138.stdout: "jlat": { 2024-04-24T09:18:33.448 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 17654, 2024-04-24T09:18:33.448 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 105.774507335, 2024-04-24T09:18:33.448 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.005991532 2024-04-24T09:18:33.448 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:33.448 INFO:teuthology.orchestra.run.smithi138.stdout: "replayed": 0 2024-04-24T09:18:33.449 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:33.449 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_mem": { 2024-04-24T09:18:33.449 INFO:teuthology.orchestra.run.smithi138.stdout: "ino": 2237, 2024-04-24T09:18:33.449 INFO:teuthology.orchestra.run.smithi138.stdout: "ino+": 11134, 2024-04-24T09:18:33.449 INFO:teuthology.orchestra.run.smithi138.stdout: "ino-": 8897, 2024-04-24T09:18:33.449 INFO:teuthology.orchestra.run.smithi138.stdout: "dir": 109, 2024-04-24T09:18:33.449 INFO:teuthology.orchestra.run.smithi138.stdout: "dir+": 160, 2024-04-24T09:18:33.449 INFO:teuthology.orchestra.run.smithi138.stdout: "dir-": 51, 2024-04-24T09:18:33.449 INFO:teuthology.orchestra.run.smithi138.stdout: "dn": 2528, 2024-04-24T09:18:33.449 INFO:teuthology.orchestra.run.smithi138.stdout: "dn+": 14294, 2024-04-24T09:18:33.449 INFO:teuthology.orchestra.run.smithi138.stdout: "dn-": 11766, 2024-04-24T09:18:33.449 INFO:teuthology.orchestra.run.smithi138.stdout: "cap": 2, 2024-04-24T09:18:33.449 INFO:teuthology.orchestra.run.smithi138.stdout: "cap+": 8187, 2024-04-24T09:18:33.450 INFO:teuthology.orchestra.run.smithi138.stdout: "cap-": 8185, 2024-04-24T09:18:33.450 INFO:teuthology.orchestra.run.smithi138.stdout: "rss": 75988, 2024-04-24T09:18:33.450 INFO:teuthology.orchestra.run.smithi138.stdout: "heap": 223608 2024-04-24T09:18:33.450 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:33.450 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_server": { 2024-04-24T09:18:33.450 INFO:teuthology.orchestra.run.smithi138.stdout: "dispatch_client_request": 368982, 2024-04-24T09:18:33.450 INFO:teuthology.orchestra.run.smithi138.stdout: "dispatch_server_request": 741, 2024-04-24T09:18:33.450 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_client_request": 330336, 2024-04-24T09:18:33.450 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_client_session": 794, 2024-04-24T09:18:33.450 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_peer_request": 5432, 2024-04-24T09:18:33.450 INFO:teuthology.orchestra.run.smithi138.stdout: "req_create_latency": { 2024-04-24T09:18:33.450 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 7984, 2024-04-24T09:18:33.450 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 25.938787195, 2024-04-24T09:18:33.450 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.003248846 2024-04-24T09:18:33.451 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:33.451 INFO:teuthology.orchestra.run.smithi138.stdout: "req_getattr_latency": { 2024-04-24T09:18:33.451 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 267544, 2024-04-24T09:18:33.451 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 145.017590476, 2024-04-24T09:18:33.451 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000542032 2024-04-24T09:18:33.451 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:33.451 INFO:teuthology.orchestra.run.smithi138.stdout: "req_getfilelock_latency": { 2024-04-24T09:18:33.451 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-24T09:18:33.451 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-24T09:18:33.451 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-24T09:18:33.451 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:33.451 INFO:teuthology.orchestra.run.smithi138.stdout: "req_link_latency": { 2024-04-24T09:18:33.451 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-24T09:18:33.451 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-24T09:18:33.451 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-24T09:18:33.452 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:33.452 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookup_latency": { 2024-04-24T09:18:33.452 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 7329, 2024-04-24T09:18:33.452 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 7.425084988, 2024-04-24T09:18:33.452 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.001013110 2024-04-24T09:18:33.452 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:33.452 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookuphash_latency": { 2024-04-24T09:18:33.452 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-24T09:18:33.452 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-24T09:18:33.452 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-24T09:18:33.452 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:33.452 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookupino_latency": { 2024-04-24T09:18:33.452 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-24T09:18:33.452 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-24T09:18:33.452 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-24T09:18:33.453 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:33.453 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookupname_latency": { 2024-04-24T09:18:33.453 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-24T09:18:33.453 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-24T09:18:33.453 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-24T09:18:33.453 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:33.453 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookupparent_latency": { 2024-04-24T09:18:33.453 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-24T09:18:33.453 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-24T09:18:33.453 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-24T09:18:33.453 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:33.453 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookupsnap_latency": { 2024-04-24T09:18:33.453 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-24T09:18:33.453 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-24T09:18:33.454 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-24T09:18:33.454 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:33.454 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lssnap_latency": { 2024-04-24T09:18:33.454 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-24T09:18:33.454 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-24T09:18:33.454 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-24T09:18:33.454 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:33.454 INFO:teuthology.orchestra.run.smithi138.stdout: "req_mkdir_latency": { 2024-04-24T09:18:33.454 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-24T09:18:33.455 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-24T09:18:33.455 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-24T09:18:33.455 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:33.455 INFO:teuthology.orchestra.run.smithi138.stdout: "req_mknod_latency": { 2024-04-24T09:18:33.455 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-24T09:18:33.455 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-24T09:18:33.455 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-24T09:18:33.455 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:33.455 INFO:teuthology.orchestra.run.smithi138.stdout: "req_mksnap_latency": { 2024-04-24T09:18:33.455 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-24T09:18:33.455 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-24T09:18:33.455 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-24T09:18:33.455 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:33.455 INFO:teuthology.orchestra.run.smithi138.stdout: "req_open_latency": { 2024-04-24T09:18:33.455 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 6103, 2024-04-24T09:18:33.455 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 8.835438504, 2024-04-24T09:18:33.456 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.001447720 2024-04-24T09:18:33.456 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:33.456 INFO:teuthology.orchestra.run.smithi138.stdout: "req_readdir_latency": { 2024-04-24T09:18:33.456 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 30985, 2024-04-24T09:18:33.456 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 92.542309570, 2024-04-24T09:18:33.456 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.002986680 2024-04-24T09:18:33.456 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:33.456 INFO:teuthology.orchestra.run.smithi138.stdout: "req_rename_latency": { 2024-04-24T09:18:33.456 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 1813, 2024-04-24T09:18:33.456 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 11.899396029, 2024-04-24T09:18:33.456 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.006563373 2024-04-24T09:18:33.456 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:33.456 INFO:teuthology.orchestra.run.smithi138.stdout: "req_renamesnap_latency": { 2024-04-24T09:18:33.456 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-24T09:18:33.456 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-24T09:18:33.457 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-24T09:18:33.457 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:33.457 INFO:teuthology.orchestra.run.smithi138.stdout: "req_snapdiff_latency": { 2024-04-24T09:18:33.457 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-24T09:18:33.457 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-24T09:18:33.457 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-24T09:18:33.457 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:33.457 INFO:teuthology.orchestra.run.smithi138.stdout: "req_rmdir_latency": { 2024-04-24T09:18:33.457 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 20, 2024-04-24T09:18:33.457 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.272259111, 2024-04-24T09:18:33.457 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.013612955 2024-04-24T09:18:33.457 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:33.457 INFO:teuthology.orchestra.run.smithi138.stdout: "req_rmsnap_latency": { 2024-04-24T09:18:33.457 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-24T09:18:33.457 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-24T09:18:33.458 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-24T09:18:33.458 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:33.458 INFO:teuthology.orchestra.run.smithi138.stdout: "req_rmxattr_latency": { 2024-04-24T09:18:33.458 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-24T09:18:33.458 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-24T09:18:33.458 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-24T09:18:33.458 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:33.458 INFO:teuthology.orchestra.run.smithi138.stdout: "req_setattr_latency": { 2024-04-24T09:18:33.458 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 78, 2024-04-24T09:18:33.458 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.396074453, 2024-04-24T09:18:33.458 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.005077877 2024-04-24T09:18:33.458 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:33.458 INFO:teuthology.orchestra.run.smithi138.stdout: "req_setdirlayout_latency": { 2024-04-24T09:18:33.458 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-24T09:18:33.459 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-24T09:18:33.459 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-24T09:18:33.459 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:33.459 INFO:teuthology.orchestra.run.smithi138.stdout: "req_setfilelock_latency": { 2024-04-24T09:18:33.459 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 276, 2024-04-24T09:18:33.459 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.253079830, 2024-04-24T09:18:33.459 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000916955 2024-04-24T09:18:33.459 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:33.459 INFO:teuthology.orchestra.run.smithi138.stdout: "req_setlayout_latency": { 2024-04-24T09:18:33.459 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-24T09:18:33.459 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-24T09:18:33.459 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-24T09:18:33.459 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:33.459 INFO:teuthology.orchestra.run.smithi138.stdout: "req_setxattr_latency": { 2024-04-24T09:18:33.460 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-24T09:18:33.460 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-24T09:18:33.460 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-24T09:18:33.460 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:33.460 INFO:teuthology.orchestra.run.smithi138.stdout: "req_symlink_latency": { 2024-04-24T09:18:33.460 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-24T09:18:33.460 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-24T09:18:33.460 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-24T09:18:33.460 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:33.460 INFO:teuthology.orchestra.run.smithi138.stdout: "req_unlink_latency": { 2024-04-24T09:18:33.460 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 8150, 2024-04-24T09:18:33.460 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 34.492518001, 2024-04-24T09:18:33.460 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.004232210 2024-04-24T09:18:33.460 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:33.460 INFO:teuthology.orchestra.run.smithi138.stdout: "cap_revoke_eviction": 0, 2024-04-24T09:18:33.461 INFO:teuthology.orchestra.run.smithi138.stdout: "cap_acquisition_throttle": 0, 2024-04-24T09:18:33.461 INFO:teuthology.orchestra.run.smithi138.stdout: "req_getvxattr_latency": { 2024-04-24T09:18:33.461 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-24T09:18:33.461 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-24T09:18:33.461 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-24T09:18:33.461 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-24T09:18:33.461 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:33.461 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_sessions": { 2024-04-24T09:18:33.461 INFO:teuthology.orchestra.run.smithi138.stdout: "session_count": 2, 2024-04-24T09:18:33.461 INFO:teuthology.orchestra.run.smithi138.stdout: "session_add": 4, 2024-04-24T09:18:33.461 INFO:teuthology.orchestra.run.smithi138.stdout: "session_remove": 2, 2024-04-24T09:18:33.461 INFO:teuthology.orchestra.run.smithi138.stdout: "sessions_open": 2, 2024-04-24T09:18:33.461 INFO:teuthology.orchestra.run.smithi138.stdout: "sessions_stale": 0, 2024-04-24T09:18:33.461 INFO:teuthology.orchestra.run.smithi138.stdout: "total_load": 35597, 2024-04-24T09:18:33.462 INFO:teuthology.orchestra.run.smithi138.stdout: "average_load": 17798, 2024-04-24T09:18:33.462 INFO:teuthology.orchestra.run.smithi138.stdout: "avg_session_uptime": 1603, 2024-04-24T09:18:33.462 INFO:teuthology.orchestra.run.smithi138.stdout: "mdthresh_evicted": 0 2024-04-24T09:18:33.462 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:33.462 INFO:teuthology.orchestra.run.smithi138.stdout: "mempool": { 2024-04-24T09:18:33.462 INFO:teuthology.orchestra.run.smithi138.stdout: "bloom_filter_bytes": 0, 2024-04-24T09:18:33.462 INFO:teuthology.orchestra.run.smithi138.stdout: "bloom_filter_items": 0, 2024-04-24T09:18:33.462 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_alloc_bytes": 0, 2024-04-24T09:18:33.462 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_alloc_items": 0, 2024-04-24T09:18:33.462 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_data_bytes": 0, 2024-04-24T09:18:33.462 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_data_items": 0, 2024-04-24T09:18:33.462 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-24T09:18:33.462 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_onode_items": 0, 2024-04-24T09:18:33.462 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-24T09:18:33.463 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_meta_items": 0, 2024-04-24T09:18:33.463 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_other_bytes": 0, 2024-04-24T09:18:33.463 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_other_items": 0, 2024-04-24T09:18:33.463 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-24T09:18:33.463 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_buffer_items": 0, 2024-04-24T09:18:33.463 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_extent_bytes": 0, 2024-04-24T09:18:33.463 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_extent_items": 0, 2024-04-24T09:18:33.463 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_blob_bytes": 0, 2024-04-24T09:18:33.463 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_blob_items": 0, 2024-04-24T09:18:33.463 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-24T09:18:33.463 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_shared_blob_items": 0, 2024-04-24T09:18:33.463 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-24T09:18:33.463 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_inline_bl_items": 0, 2024-04-24T09:18:33.463 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_fsck_bytes": 0, 2024-04-24T09:18:33.464 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_fsck_items": 0, 2024-04-24T09:18:33.464 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_txc_bytes": 0, 2024-04-24T09:18:33.464 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_txc_items": 0, 2024-04-24T09:18:33.464 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-24T09:18:33.464 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_writing_deferred_items": 0, 2024-04-24T09:18:33.464 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_writing_bytes": 0, 2024-04-24T09:18:33.464 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_writing_items": 0, 2024-04-24T09:18:33.464 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_bytes": 0, 2024-04-24T09:18:33.464 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_items": 0, 2024-04-24T09:18:33.464 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_file_reader_bytes": 0, 2024-04-24T09:18:33.464 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_file_reader_items": 0, 2024-04-24T09:18:33.464 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_file_writer_bytes": 0, 2024-04-24T09:18:33.464 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_file_writer_items": 0, 2024-04-24T09:18:33.464 INFO:teuthology.orchestra.run.smithi138.stdout: "buffer_anon_bytes": 451851, 2024-04-24T09:18:33.465 INFO:teuthology.orchestra.run.smithi138.stdout: "buffer_anon_items": 154, 2024-04-24T09:18:33.465 INFO:teuthology.orchestra.run.smithi138.stdout: "buffer_meta_bytes": 0, 2024-04-24T09:18:33.465 INFO:teuthology.orchestra.run.smithi138.stdout: "buffer_meta_items": 0, 2024-04-24T09:18:33.465 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_bytes": 0, 2024-04-24T09:18:33.465 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_items": 0, 2024-04-24T09:18:33.465 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_mapbl_bytes": 0, 2024-04-24T09:18:33.465 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_mapbl_items": 0, 2024-04-24T09:18:33.465 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pglog_bytes": 0, 2024-04-24T09:18:33.465 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pglog_items": 0, 2024-04-24T09:18:33.465 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_bytes": 9800, 2024-04-24T09:18:33.465 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_items": 188, 2024-04-24T09:18:33.465 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_mapping_bytes": 0, 2024-04-24T09:18:33.465 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_mapping_items": 0, 2024-04-24T09:18:33.465 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap_bytes": 0, 2024-04-24T09:18:33.465 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap_items": 0, 2024-04-24T09:18:33.465 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_co_bytes": 7000475, 2024-04-24T09:18:33.466 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_co_items": 29205, 2024-04-24T09:18:33.466 INFO:teuthology.orchestra.run.smithi138.stdout: "unittest_1_bytes": 0, 2024-04-24T09:18:33.466 INFO:teuthology.orchestra.run.smithi138.stdout: "unittest_1_items": 0, 2024-04-24T09:18:33.466 INFO:teuthology.orchestra.run.smithi138.stdout: "unittest_2_bytes": 0, 2024-04-24T09:18:33.466 INFO:teuthology.orchestra.run.smithi138.stdout: "unittest_2_items": 0 2024-04-24T09:18:33.466 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:33.466 INFO:teuthology.orchestra.run.smithi138.stdout: "objecter": { 2024-04-24T09:18:33.466 INFO:teuthology.orchestra.run.smithi138.stdout: "op_active": 0, 2024-04-24T09:18:33.466 INFO:teuthology.orchestra.run.smithi138.stdout: "op_laggy": 0, 2024-04-24T09:18:33.466 INFO:teuthology.orchestra.run.smithi138.stdout: "op_send": 44512, 2024-04-24T09:18:33.466 INFO:teuthology.orchestra.run.smithi138.stdout: "op_send_bytes": 334913638, 2024-04-24T09:18:33.466 INFO:teuthology.orchestra.run.smithi138.stdout: "op_resend": 33, 2024-04-24T09:18:33.466 INFO:teuthology.orchestra.run.smithi138.stdout: "op_reply": 44512, 2024-04-24T09:18:33.466 INFO:teuthology.orchestra.run.smithi138.stdout: "op_latency": { 2024-04-24T09:18:33.467 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 44512, 2024-04-24T09:18:33.467 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 317.001600713, 2024-04-24T09:18:33.467 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.007121711 2024-04-24T09:18:33.467 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:33.467 INFO:teuthology.orchestra.run.smithi138.stdout: "op_inflight": 0, 2024-04-24T09:18:33.467 INFO:teuthology.orchestra.run.smithi138.stdout: "oplen_avg": { 2024-04-24T09:18:33.467 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 44512, 2024-04-24T09:18:33.467 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 46597 2024-04-24T09:18:33.467 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:33.467 INFO:teuthology.orchestra.run.smithi138.stdout: "op": 44512, 2024-04-24T09:18:33.467 INFO:teuthology.orchestra.run.smithi138.stdout: "op_r": 6358, 2024-04-24T09:18:33.467 INFO:teuthology.orchestra.run.smithi138.stdout: "op_w": 38154, 2024-04-24T09:18:33.467 INFO:teuthology.orchestra.run.smithi138.stdout: "op_rmw": 0, 2024-04-24T09:18:33.467 INFO:teuthology.orchestra.run.smithi138.stdout: "op_pg": 0, 2024-04-24T09:18:33.468 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_stat": 0, 2024-04-24T09:18:33.468 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_create": 10, 2024-04-24T09:18:33.468 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_read": 1803, 2024-04-24T09:18:33.468 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_write": 19457, 2024-04-24T09:18:33.468 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_writefull": 1074, 2024-04-24T09:18:33.468 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_writesame": 0, 2024-04-24T09:18:33.468 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_append": 0, 2024-04-24T09:18:33.468 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_zero": 0, 2024-04-24T09:18:33.468 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_truncate": 7, 2024-04-24T09:18:33.468 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_delete": 16403, 2024-04-24T09:18:33.468 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_mapext": 0, 2024-04-24T09:18:33.468 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_sparse_read": 0, 2024-04-24T09:18:33.468 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_clonerange": 0, 2024-04-24T09:18:33.468 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_getxattr": 4532, 2024-04-24T09:18:33.468 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_setxattr": 20, 2024-04-24T09:18:33.469 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_cmpxattr": 0, 2024-04-24T09:18:33.469 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_rmxattr": 0, 2024-04-24T09:18:33.469 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_resetxattrs": 0, 2024-04-24T09:18:33.469 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_call": 0, 2024-04-24T09:18:33.469 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_watch": 0, 2024-04-24T09:18:33.469 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_notify": 0, 2024-04-24T09:18:33.469 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_src_cmpxattr": 0, 2024-04-24T09:18:33.469 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_pgls": 0, 2024-04-24T09:18:33.469 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_pgls_filter": 0, 2024-04-24T09:18:33.469 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_other": 0, 2024-04-24T09:18:33.469 INFO:teuthology.orchestra.run.smithi138.stdout: "linger_active": 0, 2024-04-24T09:18:33.469 INFO:teuthology.orchestra.run.smithi138.stdout: "linger_send": 0, 2024-04-24T09:18:33.469 INFO:teuthology.orchestra.run.smithi138.stdout: "linger_resend": 0, 2024-04-24T09:18:33.469 INFO:teuthology.orchestra.run.smithi138.stdout: "linger_ping": 0, 2024-04-24T09:18:33.469 INFO:teuthology.orchestra.run.smithi138.stdout: "poolop_active": 0, 2024-04-24T09:18:33.469 INFO:teuthology.orchestra.run.smithi138.stdout: "poolop_send": 0, 2024-04-24T09:18:33.470 INFO:teuthology.orchestra.run.smithi138.stdout: "poolop_resend": 0, 2024-04-24T09:18:33.470 INFO:teuthology.orchestra.run.smithi138.stdout: "poolstat_active": 0, 2024-04-24T09:18:33.470 INFO:teuthology.orchestra.run.smithi138.stdout: "poolstat_send": 0, 2024-04-24T09:18:33.470 INFO:teuthology.orchestra.run.smithi138.stdout: "poolstat_resend": 0, 2024-04-24T09:18:33.470 INFO:teuthology.orchestra.run.smithi138.stdout: "statfs_active": 0, 2024-04-24T09:18:33.470 INFO:teuthology.orchestra.run.smithi138.stdout: "statfs_send": 0, 2024-04-24T09:18:33.470 INFO:teuthology.orchestra.run.smithi138.stdout: "statfs_resend": 0, 2024-04-24T09:18:33.470 INFO:teuthology.orchestra.run.smithi138.stdout: "command_active": 0, 2024-04-24T09:18:33.470 INFO:teuthology.orchestra.run.smithi138.stdout: "command_send": 0, 2024-04-24T09:18:33.470 INFO:teuthology.orchestra.run.smithi138.stdout: "command_resend": 0, 2024-04-24T09:18:33.470 INFO:teuthology.orchestra.run.smithi138.stdout: "map_epoch": 136, 2024-04-24T09:18:33.470 INFO:teuthology.orchestra.run.smithi138.stdout: "map_full": 0, 2024-04-24T09:18:33.470 INFO:teuthology.orchestra.run.smithi138.stdout: "map_inc": 47, 2024-04-24T09:18:33.470 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_sessions": 12, 2024-04-24T09:18:33.470 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_session_open": 12, 2024-04-24T09:18:33.470 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_session_close": 0, 2024-04-24T09:18:33.471 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_laggy": 0, 2024-04-24T09:18:33.471 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_wr": 2247, 2024-04-24T09:18:33.471 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_rd": 35, 2024-04-24T09:18:33.471 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_del": 1009 2024-04-24T09:18:33.471 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:33.471 INFO:teuthology.orchestra.run.smithi138.stdout: "oft": { 2024-04-24T09:18:33.471 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_total_objs": 1, 2024-04-24T09:18:33.471 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_total_kv_pairs": 0, 2024-04-24T09:18:33.471 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_total_updates": 0, 2024-04-24T09:18:33.471 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_total_removes": 0 2024-04-24T09:18:33.471 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:33.471 INFO:teuthology.orchestra.run.smithi138.stdout: "osd-slow-ops": { 2024-04-24T09:18:33.471 INFO:teuthology.orchestra.run.smithi138.stdout: "slow_ops_count": 0 2024-04-24T09:18:33.471 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:33.472 INFO:teuthology.orchestra.run.smithi138.stdout: "purge_queue": { 2024-04-24T09:18:33.472 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executing_ops": 0, 2024-04-24T09:18:33.472 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executing_ops_high_water": 10, 2024-04-24T09:18:33.472 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executing": 0, 2024-04-24T09:18:33.472 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executing_high_water": 5, 2024-04-24T09:18:33.472 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executed_ops": 16300, 2024-04-24T09:18:33.472 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executed": 8150, 2024-04-24T09:18:33.472 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_item_in_journal": 0 2024-04-24T09:18:33.472 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:33.472 INFO:teuthology.orchestra.run.smithi138.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-24T09:18:33.472 INFO:teuthology.orchestra.run.smithi138.stdout: "val": 0, 2024-04-24T09:18:33.472 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 104857600, 2024-04-24T09:18:33.472 INFO:teuthology.orchestra.run.smithi138.stdout: "get_started": 0, 2024-04-24T09:18:33.472 INFO:teuthology.orchestra.run.smithi138.stdout: "get": 505074, 2024-04-24T09:18:33.472 INFO:teuthology.orchestra.run.smithi138.stdout: "get_sum": 103024478, 2024-04-24T09:18:33.473 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_fail": 0, 2024-04-24T09:18:33.473 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_success": 505074, 2024-04-24T09:18:33.473 INFO:teuthology.orchestra.run.smithi138.stdout: "take": 0, 2024-04-24T09:18:33.473 INFO:teuthology.orchestra.run.smithi138.stdout: "take_sum": 0, 2024-04-24T09:18:33.473 INFO:teuthology.orchestra.run.smithi138.stdout: "put": 505074, 2024-04-24T09:18:33.473 INFO:teuthology.orchestra.run.smithi138.stdout: "put_sum": 103024478, 2024-04-24T09:18:33.473 INFO:teuthology.orchestra.run.smithi138.stdout: "wait": { 2024-04-24T09:18:33.473 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-24T09:18:33.473 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-24T09:18:33.473 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-24T09:18:33.473 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-24T09:18:33.473 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:33.473 INFO:teuthology.orchestra.run.smithi138.stdout: "throttle-objecter_bytes": { 2024-04-24T09:18:33.473 INFO:teuthology.orchestra.run.smithi138.stdout: "val": 0, 2024-04-24T09:18:33.473 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 104857600, 2024-04-24T09:18:33.474 INFO:teuthology.orchestra.run.smithi138.stdout: "get_started": 0, 2024-04-24T09:18:33.474 INFO:teuthology.orchestra.run.smithi138.stdout: "get": 0, 2024-04-24T09:18:33.474 INFO:teuthology.orchestra.run.smithi138.stdout: "get_sum": 0, 2024-04-24T09:18:33.474 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_fail": 0, 2024-04-24T09:18:33.474 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_success": 0, 2024-04-24T09:18:33.474 INFO:teuthology.orchestra.run.smithi138.stdout: "take": 44512, 2024-04-24T09:18:33.474 INFO:teuthology.orchestra.run.smithi138.stdout: "take_sum": 335812933, 2024-04-24T09:18:33.474 INFO:teuthology.orchestra.run.smithi138.stdout: "put": 44512, 2024-04-24T09:18:33.474 INFO:teuthology.orchestra.run.smithi138.stdout: "put_sum": 335812933, 2024-04-24T09:18:33.474 INFO:teuthology.orchestra.run.smithi138.stdout: "wait": { 2024-04-24T09:18:33.474 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-24T09:18:33.474 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-24T09:18:33.474 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-24T09:18:33.474 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-24T09:18:33.474 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:33.474 INFO:teuthology.orchestra.run.smithi138.stdout: "throttle-objecter_ops": { 2024-04-24T09:18:33.475 INFO:teuthology.orchestra.run.smithi138.stdout: "val": 0, 2024-04-24T09:18:33.475 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 1024, 2024-04-24T09:18:33.475 INFO:teuthology.orchestra.run.smithi138.stdout: "get_started": 0, 2024-04-24T09:18:33.475 INFO:teuthology.orchestra.run.smithi138.stdout: "get": 0, 2024-04-24T09:18:33.475 INFO:teuthology.orchestra.run.smithi138.stdout: "get_sum": 0, 2024-04-24T09:18:33.475 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_fail": 0, 2024-04-24T09:18:33.475 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_success": 0, 2024-04-24T09:18:33.475 INFO:teuthology.orchestra.run.smithi138.stdout: "take": 44512, 2024-04-24T09:18:33.475 INFO:teuthology.orchestra.run.smithi138.stdout: "take_sum": 44512, 2024-04-24T09:18:33.475 INFO:teuthology.orchestra.run.smithi138.stdout: "put": 44512, 2024-04-24T09:18:33.475 INFO:teuthology.orchestra.run.smithi138.stdout: "put_sum": 44512, 2024-04-24T09:18:33.475 INFO:teuthology.orchestra.run.smithi138.stdout: "wait": { 2024-04-24T09:18:33.475 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-24T09:18:33.475 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-24T09:18:33.475 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-24T09:18:33.475 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-24T09:18:33.475 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:33.476 INFO:teuthology.orchestra.run.smithi138.stdout: "throttle-write_buf_throttle": { 2024-04-24T09:18:33.476 INFO:teuthology.orchestra.run.smithi138.stdout: "val": 0, 2024-04-24T09:18:33.476 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 3758096384, 2024-04-24T09:18:33.476 INFO:teuthology.orchestra.run.smithi138.stdout: "get_started": 0, 2024-04-24T09:18:33.476 INFO:teuthology.orchestra.run.smithi138.stdout: "get": 8150, 2024-04-24T09:18:33.476 INFO:teuthology.orchestra.run.smithi138.stdout: "get_sum": 899830, 2024-04-24T09:18:33.476 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_fail": 0, 2024-04-24T09:18:33.476 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_success": 8150, 2024-04-24T09:18:33.476 INFO:teuthology.orchestra.run.smithi138.stdout: "take": 0, 2024-04-24T09:18:33.476 INFO:teuthology.orchestra.run.smithi138.stdout: "take_sum": 0, 2024-04-24T09:18:33.476 INFO:teuthology.orchestra.run.smithi138.stdout: "put": 1803, 2024-04-24T09:18:33.476 INFO:teuthology.orchestra.run.smithi138.stdout: "put_sum": 899830, 2024-04-24T09:18:33.476 INFO:teuthology.orchestra.run.smithi138.stdout: "wait": { 2024-04-24T09:18:33.476 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-24T09:18:33.476 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-24T09:18:33.476 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-24T09:18:33.476 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-24T09:18:33.477 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:33.477 INFO:teuthology.orchestra.run.smithi138.stdout: "throttle-write_buf_throttle-0x55e3b9734500": { 2024-04-24T09:18:33.477 INFO:teuthology.orchestra.run.smithi138.stdout: "val": 0, 2024-04-24T09:18:33.477 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 3758096384, 2024-04-24T09:18:33.477 INFO:teuthology.orchestra.run.smithi138.stdout: "get_started": 0, 2024-04-24T09:18:33.477 INFO:teuthology.orchestra.run.smithi138.stdout: "get": 63657, 2024-04-24T09:18:33.477 INFO:teuthology.orchestra.run.smithi138.stdout: "get_sum": 332617118, 2024-04-24T09:18:33.477 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_fail": 0, 2024-04-24T09:18:33.477 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_success": 63657, 2024-04-24T09:18:33.477 INFO:teuthology.orchestra.run.smithi138.stdout: "take": 0, 2024-04-24T09:18:33.477 INFO:teuthology.orchestra.run.smithi138.stdout: "take_sum": 0, 2024-04-24T09:18:33.477 INFO:teuthology.orchestra.run.smithi138.stdout: "put": 17654, 2024-04-24T09:18:33.477 INFO:teuthology.orchestra.run.smithi138.stdout: "put_sum": 332617118, 2024-04-24T09:18:33.477 INFO:teuthology.orchestra.run.smithi138.stdout: "wait": { 2024-04-24T09:18:33.477 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-24T09:18:33.477 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-24T09:18:33.477 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-24T09:18:33.478 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-24T09:18:33.478 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-24T09:18:33.478 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-04-24T09:18:34.205 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:18:34 smithi138 ceph-mon[31441]: pgmap v853: 129 pgs: 129 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-24T09:18:34.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:18:34 smithi189 ceph-mon[31791]: pgmap v853: 129 pgs: 129 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-24T09:18:34.473 INFO:tasks.check_counter:Daemon mds.h mds.exported=29 2024-04-24T09:18:34.473 INFO:tasks.check_counter:Daemon mds.h mds.imported=23 2024-04-24T09:18:34.473 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=7 2024-04-24T09:18:34.473 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-04-24T09:18:34.473 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-04-24T09:18:34.473 DEBUG:tasks.check_counter:Getting stats from k 2024-04-24T09:18:34.473 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid bd3e039e-0215-11ef-bc93-c7b262605968 -- ceph daemon mds.k perf dump 2024-04-24T09:18:34.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:18:34 smithi083 ceph-mon[27907]: pgmap v853: 129 pgs: 129 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-24T09:18:34.909 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T09:18:36.328 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:18:36 smithi138 ceph-mon[31441]: pgmap v854: 129 pgs: 129 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-24T09:18:36.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:18:36 smithi189 ceph-mon[31791]: pgmap v854: 129 pgs: 129 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-24T09:18:36.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:18:36 smithi083 ceph-mon[27907]: pgmap v854: 129 pgs: 129 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-24T09:18:37.256 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-04-24T09:18:37.256 INFO:teuthology.orchestra.run.smithi138.stdout: "AsyncMessenger::Worker-0": { 2024-04-24T09:18:37.256 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_messages": 44830, 2024-04-24T09:18:37.256 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages": 39984, 2024-04-24T09:18:37.256 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_bytes": 13518985, 2024-04-24T09:18:37.256 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_bytes": 111133781, 2024-04-24T09:18:37.257 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_created_connections": 97, 2024-04-24T09:18:37.257 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_active_connections": 3, 2024-04-24T09:18:37.257 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_total_time": 11.619953240, 2024-04-24T09:18:37.257 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_send_time": 4.737995100, 2024-04-24T09:18:37.257 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_recv_time": 5.146084618, 2024-04-24T09:18:37.257 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_fast_dispatch_time": 2.610772004, 2024-04-24T09:18:37.257 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T09:18:37.257 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 39984, 2024-04-24T09:18:37.257 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 4.482900531, 2024-04-24T09:18:37.257 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000112117 2024-04-24T09:18:37.257 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:37.257 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_handle_ack_lat": { 2024-04-24T09:18:37.257 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 51935, 2024-04-24T09:18:37.257 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.029350131, 2024-04-24T09:18:37.257 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000565 2024-04-24T09:18:37.258 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:37.258 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_encrypted_bytes": 3168, 2024-04-24T09:18:37.258 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_encrypted_bytes": 16208 2024-04-24T09:18:37.258 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:37.258 INFO:teuthology.orchestra.run.smithi138.stdout: "AsyncMessenger::Worker-1": { 2024-04-24T09:18:37.258 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_messages": 16237, 2024-04-24T09:18:37.258 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages": 16310, 2024-04-24T09:18:37.258 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_bytes": 6573475, 2024-04-24T09:18:37.258 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_bytes": 118196522, 2024-04-24T09:18:37.258 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_created_connections": 16, 2024-04-24T09:18:37.258 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_active_connections": 6, 2024-04-24T09:18:37.258 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_total_time": 2.785387732, 2024-04-24T09:18:37.258 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_send_time": 1.023339262, 2024-04-24T09:18:37.258 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_recv_time": 0.684217634, 2024-04-24T09:18:37.259 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_fast_dispatch_time": 0.925363010, 2024-04-24T09:18:37.259 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T09:18:37.259 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 16310, 2024-04-24T09:18:37.259 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 2.501897973, 2024-04-24T09:18:37.259 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000153396 2024-04-24T09:18:37.259 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:37.259 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_handle_ack_lat": { 2024-04-24T09:18:37.259 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 1271, 2024-04-24T09:18:37.259 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000665522, 2024-04-24T09:18:37.259 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000523 2024-04-24T09:18:37.259 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:37.259 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_encrypted_bytes": 2109728, 2024-04-24T09:18:37.259 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_encrypted_bytes": 372672 2024-04-24T09:18:37.260 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:37.260 INFO:teuthology.orchestra.run.smithi138.stdout: "AsyncMessenger::Worker-2": { 2024-04-24T09:18:37.260 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_messages": 397324, 2024-04-24T09:18:37.260 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages": 440664, 2024-04-24T09:18:37.260 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_bytes": 120144639, 2024-04-24T09:18:37.260 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_bytes": 334159089, 2024-04-24T09:18:37.260 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_created_connections": 18, 2024-04-24T09:18:37.260 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_active_connections": 7, 2024-04-24T09:18:37.260 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_total_time": 44.261989102, 2024-04-24T09:18:37.260 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_send_time": 19.154602228, 2024-04-24T09:18:37.260 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_recv_time": 29.109837809, 2024-04-24T09:18:37.260 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_fast_dispatch_time": 0.878375521, 2024-04-24T09:18:37.261 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T09:18:37.261 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 440659, 2024-04-24T09:18:37.261 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 35.199230572, 2024-04-24T09:18:37.261 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000079878 2024-04-24T09:18:37.261 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:37.261 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_handle_ack_lat": { 2024-04-24T09:18:37.261 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 784675, 2024-04-24T09:18:37.261 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.256099135, 2024-04-24T09:18:37.261 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000326 2024-04-24T09:18:37.261 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:37.261 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_encrypted_bytes": 1248, 2024-04-24T09:18:37.261 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_encrypted_bytes": 750608 2024-04-24T09:18:37.261 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:37.262 INFO:teuthology.orchestra.run.smithi138.stdout: "cct": { 2024-04-24T09:18:37.262 INFO:teuthology.orchestra.run.smithi138.stdout: "total_workers": 1, 2024-04-24T09:18:37.262 INFO:teuthology.orchestra.run.smithi138.stdout: "unhealthy_workers": 0 2024-04-24T09:18:37.262 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:37.262 INFO:teuthology.orchestra.run.smithi138.stdout: "finisher-MDSRank": { 2024-04-24T09:18:37.262 INFO:teuthology.orchestra.run.smithi138.stdout: "queue_len": 0, 2024-04-24T09:18:37.262 INFO:teuthology.orchestra.run.smithi138.stdout: "complete_latency": { 2024-04-24T09:18:37.262 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 30408, 2024-04-24T09:18:37.262 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 43.096813696, 2024-04-24T09:18:37.262 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.001417285 2024-04-24T09:18:37.262 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-24T09:18:37.262 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:37.263 INFO:teuthology.orchestra.run.smithi138.stdout: "finisher-PurgeQueue": { 2024-04-24T09:18:37.263 INFO:teuthology.orchestra.run.smithi138.stdout: "queue_len": 0, 2024-04-24T09:18:37.263 INFO:teuthology.orchestra.run.smithi138.stdout: "complete_latency": { 2024-04-24T09:18:37.263 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 20912, 2024-04-24T09:18:37.263 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 2.923435254, 2024-04-24T09:18:37.263 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000139797 2024-04-24T09:18:37.263 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-24T09:18:37.263 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:37.263 INFO:teuthology.orchestra.run.smithi138.stdout: "mds": { 2024-04-24T09:18:37.263 INFO:teuthology.orchestra.run.smithi138.stdout: "request": 320455, 2024-04-24T09:18:37.263 INFO:teuthology.orchestra.run.smithi138.stdout: "reply": 319558, 2024-04-24T09:18:37.263 INFO:teuthology.orchestra.run.smithi138.stdout: "reply_latency": { 2024-04-24T09:18:37.263 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 319558, 2024-04-24T09:18:37.264 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 298.171226099, 2024-04-24T09:18:37.264 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000933073 2024-04-24T09:18:37.264 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:37.264 INFO:teuthology.orchestra.run.smithi138.stdout: "slow_reply": 0, 2024-04-24T09:18:37.264 INFO:teuthology.orchestra.run.smithi138.stdout: "forward": 897, 2024-04-24T09:18:37.264 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_fetch_complete": 0, 2024-04-24T09:18:37.264 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_fetch_keys": 0, 2024-04-24T09:18:37.264 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_commit": 45, 2024-04-24T09:18:37.264 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_split": 4, 2024-04-24T09:18:37.264 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_merge": 0, 2024-04-24T09:18:37.264 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes": 1818, 2024-04-24T09:18:37.264 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_top": 256, 2024-04-24T09:18:37.264 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_bottom": 1495, 2024-04-24T09:18:37.264 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_pin_tail": 67, 2024-04-24T09:18:37.264 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_pinned": 1451, 2024-04-24T09:18:37.264 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_expired": 390, 2024-04-24T09:18:37.265 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_with_caps": 1, 2024-04-24T09:18:37.265 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": 1, 2024-04-24T09:18:37.265 INFO:teuthology.orchestra.run.smithi138.stdout: "subtrees": 6, 2024-04-24T09:18:37.265 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse": 610727, 2024-04-24T09:18:37.265 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_hit": 597532, 2024-04-24T09:18:37.265 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_forward": 891, 2024-04-24T09:18:37.265 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_discover": 30, 2024-04-24T09:18:37.265 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_dir_fetch": 0, 2024-04-24T09:18:37.265 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_remote_ino": 0, 2024-04-24T09:18:37.265 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_lock": 0, 2024-04-24T09:18:37.265 INFO:teuthology.orchestra.run.smithi138.stdout: "load_cent": 0, 2024-04-24T09:18:37.266 INFO:teuthology.orchestra.run.smithi138.stdout: "q": 0, 2024-04-24T09:18:37.266 INFO:teuthology.orchestra.run.smithi138.stdout: "exported": 7, 2024-04-24T09:18:37.266 INFO:teuthology.orchestra.run.smithi138.stdout: "exported_inodes": 277, 2024-04-24T09:18:37.266 INFO:teuthology.orchestra.run.smithi138.stdout: "imported": 7, 2024-04-24T09:18:37.266 INFO:teuthology.orchestra.run.smithi138.stdout: "imported_inodes": 201, 2024-04-24T09:18:37.266 INFO:teuthology.orchestra.run.smithi138.stdout: "openino_dir_fetch": 0, 2024-04-24T09:18:37.266 INFO:teuthology.orchestra.run.smithi138.stdout: "openino_backtrace_fetch": 3785, 2024-04-24T09:18:37.266 INFO:teuthology.orchestra.run.smithi138.stdout: "openino_peer_discover": 0, 2024-04-24T09:18:37.266 INFO:teuthology.orchestra.run.smithi138.stdout: "root_rfiles": 4, 2024-04-24T09:18:37.266 INFO:teuthology.orchestra.run.smithi138.stdout: "root_rbytes": 234, 2024-04-24T09:18:37.266 INFO:teuthology.orchestra.run.smithi138.stdout: "root_rsnaps": 7, 2024-04-24T09:18:37.266 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_backtrace_fetch": 3785, 2024-04-24T09:18:37.267 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_set_tag": 0, 2024-04-24T09:18:37.267 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_backtrace_repaired": 0, 2024-04-24T09:18:37.267 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_inotable_repaired": 0, 2024-04-24T09:18:37.267 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_dir_inodes": 420, 2024-04-24T09:18:37.267 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_dir_base_inodes": 0, 2024-04-24T09:18:37.267 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_dirfrag_rstats": 420, 2024-04-24T09:18:37.267 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_file_inodes": 3365, 2024-04-24T09:18:37.267 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_inode_file_caps": 0, 2024-04-24T09:18:37.267 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_cap_op_revoke": 26964, 2024-04-24T09:18:37.267 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_cap_op_grant": 40344, 2024-04-24T09:18:37.267 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_cap_op_trunc": 0, 2024-04-24T09:18:37.267 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_cap_op_flushsnap_ack": 1, 2024-04-24T09:18:37.267 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-24T09:18:37.267 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_client_caps": 40551, 2024-04-24T09:18:37.267 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_client_caps_dirty": 9552, 2024-04-24T09:18:37.268 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_client_cap_release": 998, 2024-04-24T09:18:37.268 INFO:teuthology.orchestra.run.smithi138.stdout: "process_request_cap_release": 29444 2024-04-24T09:18:37.268 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:37.268 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_cache": { 2024-04-24T09:18:37.268 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_update": 18, 2024-04-24T09:18:37.268 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_update_receipt": 30, 2024-04-24T09:18:37.268 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_try_discover": 22, 2024-04-24T09:18:37.268 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_send_discover": 42, 2024-04-24T09:18:37.268 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_handle_discover": 52, 2024-04-24T09:18:37.268 INFO:teuthology.orchestra.run.smithi138.stdout: "num_strays": 20, 2024-04-24T09:18:37.268 INFO:teuthology.orchestra.run.smithi138.stdout: "num_strays_delayed": 0, 2024-04-24T09:18:37.268 INFO:teuthology.orchestra.run.smithi138.stdout: "num_strays_enqueuing": 0, 2024-04-24T09:18:37.268 INFO:teuthology.orchestra.run.smithi138.stdout: "strays_created": 8273, 2024-04-24T09:18:37.268 INFO:teuthology.orchestra.run.smithi138.stdout: "strays_enqueued": 8253, 2024-04-24T09:18:37.268 INFO:teuthology.orchestra.run.smithi138.stdout: "strays_reintegrated": 0, 2024-04-24T09:18:37.269 INFO:teuthology.orchestra.run.smithi138.stdout: "strays_migrated": 0, 2024-04-24T09:18:37.269 INFO:teuthology.orchestra.run.smithi138.stdout: "num_recovering_processing": 0, 2024-04-24T09:18:37.269 INFO:teuthology.orchestra.run.smithi138.stdout: "num_recovering_enqueued": 0, 2024-04-24T09:18:37.269 INFO:teuthology.orchestra.run.smithi138.stdout: "num_recovering_prioritized": 0, 2024-04-24T09:18:37.269 INFO:teuthology.orchestra.run.smithi138.stdout: "recovery_started": 0, 2024-04-24T09:18:37.269 INFO:teuthology.orchestra.run.smithi138.stdout: "recovery_completed": 0, 2024-04-24T09:18:37.269 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_quiesce_path": 0, 2024-04-24T09:18:37.269 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_quiesce_inode": 0, 2024-04-24T09:18:37.269 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_enqueue_scrub": 0, 2024-04-24T09:18:37.269 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_exportdir": 8, 2024-04-24T09:18:37.269 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_flush": 0, 2024-04-24T09:18:37.269 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_fragmentdir": 4, 2024-04-24T09:18:37.270 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_fragstats": 0, 2024-04-24T09:18:37.270 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_inodestats": 0 2024-04-24T09:18:37.270 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:37.270 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_log": { 2024-04-24T09:18:37.270 INFO:teuthology.orchestra.run.smithi138.stdout: "evlrg": 0, 2024-04-24T09:18:37.270 INFO:teuthology.orchestra.run.smithi138.stdout: "evadd": 53979, 2024-04-24T09:18:37.270 INFO:teuthology.orchestra.run.smithi138.stdout: "evex": 0, 2024-04-24T09:18:37.270 INFO:teuthology.orchestra.run.smithi138.stdout: "evtrm": 0, 2024-04-24T09:18:37.270 INFO:teuthology.orchestra.run.smithi138.stdout: "ev": 53979, 2024-04-24T09:18:37.271 INFO:teuthology.orchestra.run.smithi138.stdout: "evexg": 0, 2024-04-24T09:18:37.271 INFO:teuthology.orchestra.run.smithi138.stdout: "evexd": 0, 2024-04-24T09:18:37.271 INFO:teuthology.orchestra.run.smithi138.stdout: "segadd": 88, 2024-04-24T09:18:37.271 INFO:teuthology.orchestra.run.smithi138.stdout: "segex": 0, 2024-04-24T09:18:37.271 INFO:teuthology.orchestra.run.smithi138.stdout: "segtrm": 0, 2024-04-24T09:18:37.271 INFO:teuthology.orchestra.run.smithi138.stdout: "seg": 88, 2024-04-24T09:18:37.271 INFO:teuthology.orchestra.run.smithi138.stdout: "segmjr": 6, 2024-04-24T09:18:37.271 INFO:teuthology.orchestra.run.smithi138.stdout: "segexg": 0, 2024-04-24T09:18:37.271 INFO:teuthology.orchestra.run.smithi138.stdout: "segexd": 0, 2024-04-24T09:18:37.271 INFO:teuthology.orchestra.run.smithi138.stdout: "expos": 4194304, 2024-04-24T09:18:37.271 INFO:teuthology.orchestra.run.smithi138.stdout: "wrpos": 273943710, 2024-04-24T09:18:37.272 INFO:teuthology.orchestra.run.smithi138.stdout: "rdpos": 0, 2024-04-24T09:18:37.272 INFO:teuthology.orchestra.run.smithi138.stdout: "jlat": { 2024-04-24T09:18:37.272 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 14466, 2024-04-24T09:18:37.272 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 96.287972693, 2024-04-24T09:18:37.272 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.006656157 2024-04-24T09:18:37.272 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:37.272 INFO:teuthology.orchestra.run.smithi138.stdout: "replayed": 0 2024-04-24T09:18:37.272 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:37.272 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_mem": { 2024-04-24T09:18:37.272 INFO:teuthology.orchestra.run.smithi138.stdout: "ino": 1527, 2024-04-24T09:18:37.272 INFO:teuthology.orchestra.run.smithi138.stdout: "ino+": 10071, 2024-04-24T09:18:37.272 INFO:teuthology.orchestra.run.smithi138.stdout: "ino-": 8544, 2024-04-24T09:18:37.272 INFO:teuthology.orchestra.run.smithi138.stdout: "dir": 72, 2024-04-24T09:18:37.273 INFO:teuthology.orchestra.run.smithi138.stdout: "dir+": 122, 2024-04-24T09:18:37.273 INFO:teuthology.orchestra.run.smithi138.stdout: "dir-": 50, 2024-04-24T09:18:37.273 INFO:teuthology.orchestra.run.smithi138.stdout: "dn": 1818, 2024-04-24T09:18:37.273 INFO:teuthology.orchestra.run.smithi138.stdout: "dn+": 11358, 2024-04-24T09:18:37.273 INFO:teuthology.orchestra.run.smithi138.stdout: "dn-": 9540, 2024-04-24T09:18:37.273 INFO:teuthology.orchestra.run.smithi138.stdout: "cap": 1, 2024-04-24T09:18:37.273 INFO:teuthology.orchestra.run.smithi138.stdout: "cap+": 8297, 2024-04-24T09:18:37.273 INFO:teuthology.orchestra.run.smithi138.stdout: "cap-": 8296, 2024-04-24T09:18:37.273 INFO:teuthology.orchestra.run.smithi138.stdout: "rss": 75164, 2024-04-24T09:18:37.273 INFO:teuthology.orchestra.run.smithi138.stdout: "heap": 223608 2024-04-24T09:18:37.273 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:37.273 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_server": { 2024-04-24T09:18:37.274 INFO:teuthology.orchestra.run.smithi138.stdout: "dispatch_client_request": 349145, 2024-04-24T09:18:37.274 INFO:teuthology.orchestra.run.smithi138.stdout: "dispatch_server_request": 30, 2024-04-24T09:18:37.274 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_client_request": 320455, 2024-04-24T09:18:37.274 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_client_session": 901, 2024-04-24T09:18:37.274 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_peer_request": 3910, 2024-04-24T09:18:37.274 INFO:teuthology.orchestra.run.smithi138.stdout: "req_create_latency": { 2024-04-24T09:18:37.274 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 8113, 2024-04-24T09:18:37.274 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 20.067103906, 2024-04-24T09:18:37.274 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.002473450 2024-04-24T09:18:37.274 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:37.274 INFO:teuthology.orchestra.run.smithi138.stdout: "req_getattr_latency": { 2024-04-24T09:18:37.274 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 275125, 2024-04-24T09:18:37.274 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 130.717216646, 2024-04-24T09:18:37.274 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000475119 2024-04-24T09:18:37.275 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:37.275 INFO:teuthology.orchestra.run.smithi138.stdout: "req_getfilelock_latency": { 2024-04-24T09:18:37.275 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-24T09:18:37.275 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-24T09:18:37.275 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-24T09:18:37.275 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:37.275 INFO:teuthology.orchestra.run.smithi138.stdout: "req_link_latency": { 2024-04-24T09:18:37.275 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-24T09:18:37.275 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-24T09:18:37.275 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-24T09:18:37.275 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:37.275 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookup_latency": { 2024-04-24T09:18:37.275 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 2052, 2024-04-24T09:18:37.275 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 4.324435988, 2024-04-24T09:18:37.276 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.002107424 2024-04-24T09:18:37.276 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:37.276 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookuphash_latency": { 2024-04-24T09:18:37.276 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-24T09:18:37.276 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-24T09:18:37.276 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-24T09:18:37.276 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:37.276 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookupino_latency": { 2024-04-24T09:18:37.276 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-24T09:18:37.276 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-24T09:18:37.276 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-24T09:18:37.276 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:37.276 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookupname_latency": { 2024-04-24T09:18:37.277 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-24T09:18:37.277 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-24T09:18:37.277 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-24T09:18:37.277 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:37.277 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookupparent_latency": { 2024-04-24T09:18:37.277 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-24T09:18:37.277 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-24T09:18:37.277 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-24T09:18:37.277 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:37.277 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookupsnap_latency": { 2024-04-24T09:18:37.277 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-24T09:18:37.277 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-24T09:18:37.278 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-24T09:18:37.278 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:37.278 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lssnap_latency": { 2024-04-24T09:18:37.278 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-24T09:18:37.278 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-24T09:18:37.278 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-24T09:18:37.278 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:37.279 INFO:teuthology.orchestra.run.smithi138.stdout: "req_mkdir_latency": { 2024-04-24T09:18:37.279 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-24T09:18:37.279 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-24T09:18:37.279 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-24T09:18:37.279 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:37.279 INFO:teuthology.orchestra.run.smithi138.stdout: "req_mknod_latency": { 2024-04-24T09:18:37.279 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-24T09:18:37.279 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-24T09:18:37.279 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-24T09:18:37.279 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:37.279 INFO:teuthology.orchestra.run.smithi138.stdout: "req_mksnap_latency": { 2024-04-24T09:18:37.279 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-24T09:18:37.279 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-24T09:18:37.279 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-24T09:18:37.280 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:37.280 INFO:teuthology.orchestra.run.smithi138.stdout: "req_open_latency": { 2024-04-24T09:18:37.280 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 5087, 2024-04-24T09:18:37.280 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 6.666943077, 2024-04-24T09:18:37.280 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.001310584 2024-04-24T09:18:37.280 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:37.280 INFO:teuthology.orchestra.run.smithi138.stdout: "req_readdir_latency": { 2024-04-24T09:18:37.280 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 19019, 2024-04-24T09:18:37.280 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 91.770823014, 2024-04-24T09:18:37.280 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.004825218 2024-04-24T09:18:37.280 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:37.280 INFO:teuthology.orchestra.run.smithi138.stdout: "req_rename_latency": { 2024-04-24T09:18:37.280 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 1593, 2024-04-24T09:18:37.280 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 5.011322141, 2024-04-24T09:18:37.281 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.003145839 2024-04-24T09:18:37.281 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:37.281 INFO:teuthology.orchestra.run.smithi138.stdout: "req_renamesnap_latency": { 2024-04-24T09:18:37.281 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-24T09:18:37.281 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-24T09:18:37.281 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-24T09:18:37.281 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:37.281 INFO:teuthology.orchestra.run.smithi138.stdout: "req_snapdiff_latency": { 2024-04-24T09:18:37.281 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-24T09:18:37.281 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-24T09:18:37.281 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-24T09:18:37.281 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:37.281 INFO:teuthology.orchestra.run.smithi138.stdout: "req_rmdir_latency": { 2024-04-24T09:18:37.281 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 20, 2024-04-24T09:18:37.281 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.271797604, 2024-04-24T09:18:37.281 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.013589880 2024-04-24T09:18:37.282 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:37.282 INFO:teuthology.orchestra.run.smithi138.stdout: "req_rmsnap_latency": { 2024-04-24T09:18:37.282 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-24T09:18:37.282 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-24T09:18:37.282 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-24T09:18:37.282 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:37.282 INFO:teuthology.orchestra.run.smithi138.stdout: "req_rmxattr_latency": { 2024-04-24T09:18:37.282 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-24T09:18:37.282 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-24T09:18:37.282 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-24T09:18:37.282 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:37.282 INFO:teuthology.orchestra.run.smithi138.stdout: "req_setattr_latency": { 2024-04-24T09:18:37.282 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-24T09:18:37.282 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-24T09:18:37.282 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-24T09:18:37.283 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:37.283 INFO:teuthology.orchestra.run.smithi138.stdout: "req_setdirlayout_latency": { 2024-04-24T09:18:37.283 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-24T09:18:37.283 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-24T09:18:37.283 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-24T09:18:37.283 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:37.283 INFO:teuthology.orchestra.run.smithi138.stdout: "req_setfilelock_latency": { 2024-04-24T09:18:37.283 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 296, 2024-04-24T09:18:37.283 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.171247163, 2024-04-24T09:18:37.283 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000578537 2024-04-24T09:18:37.283 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:37.283 INFO:teuthology.orchestra.run.smithi138.stdout: "req_setlayout_latency": { 2024-04-24T09:18:37.283 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-24T09:18:37.283 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-24T09:18:37.283 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-24T09:18:37.283 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:37.284 INFO:teuthology.orchestra.run.smithi138.stdout: "req_setxattr_latency": { 2024-04-24T09:18:37.284 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-24T09:18:37.284 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-24T09:18:37.284 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-24T09:18:37.284 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:37.284 INFO:teuthology.orchestra.run.smithi138.stdout: "req_symlink_latency": { 2024-04-24T09:18:37.284 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-24T09:18:37.284 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-24T09:18:37.284 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-24T09:18:37.284 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:37.284 INFO:teuthology.orchestra.run.smithi138.stdout: "req_unlink_latency": { 2024-04-24T09:18:37.284 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 8253, 2024-04-24T09:18:37.284 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 39.170336560, 2024-04-24T09:18:37.284 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.004746193 2024-04-24T09:18:37.284 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:37.285 INFO:teuthology.orchestra.run.smithi138.stdout: "cap_revoke_eviction": 0, 2024-04-24T09:18:37.285 INFO:teuthology.orchestra.run.smithi138.stdout: "cap_acquisition_throttle": 0, 2024-04-24T09:18:37.285 INFO:teuthology.orchestra.run.smithi138.stdout: "req_getvxattr_latency": { 2024-04-24T09:18:37.285 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-24T09:18:37.285 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-24T09:18:37.285 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-24T09:18:37.285 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-24T09:18:37.285 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:37.285 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_sessions": { 2024-04-24T09:18:37.285 INFO:teuthology.orchestra.run.smithi138.stdout: "session_count": 2, 2024-04-24T09:18:37.285 INFO:teuthology.orchestra.run.smithi138.stdout: "session_add": 4, 2024-04-24T09:18:37.285 INFO:teuthology.orchestra.run.smithi138.stdout: "session_remove": 2, 2024-04-24T09:18:37.285 INFO:teuthology.orchestra.run.smithi138.stdout: "sessions_open": 2, 2024-04-24T09:18:37.285 INFO:teuthology.orchestra.run.smithi138.stdout: "sessions_stale": 0, 2024-04-24T09:18:37.285 INFO:teuthology.orchestra.run.smithi138.stdout: "total_load": 33669, 2024-04-24T09:18:37.285 INFO:teuthology.orchestra.run.smithi138.stdout: "average_load": 16834, 2024-04-24T09:18:37.286 INFO:teuthology.orchestra.run.smithi138.stdout: "avg_session_uptime": 1609, 2024-04-24T09:18:37.286 INFO:teuthology.orchestra.run.smithi138.stdout: "mdthresh_evicted": 0 2024-04-24T09:18:37.286 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:37.286 INFO:teuthology.orchestra.run.smithi138.stdout: "mempool": { 2024-04-24T09:18:37.286 INFO:teuthology.orchestra.run.smithi138.stdout: "bloom_filter_bytes": 0, 2024-04-24T09:18:37.286 INFO:teuthology.orchestra.run.smithi138.stdout: "bloom_filter_items": 0, 2024-04-24T09:18:37.286 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_alloc_bytes": 0, 2024-04-24T09:18:37.286 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_alloc_items": 0, 2024-04-24T09:18:37.286 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_data_bytes": 0, 2024-04-24T09:18:37.286 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_data_items": 0, 2024-04-24T09:18:37.286 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-24T09:18:37.286 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_onode_items": 0, 2024-04-24T09:18:37.286 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-24T09:18:37.286 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_meta_items": 0, 2024-04-24T09:18:37.286 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_other_bytes": 0, 2024-04-24T09:18:37.287 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_other_items": 0, 2024-04-24T09:18:37.287 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-24T09:18:37.287 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_buffer_items": 0, 2024-04-24T09:18:37.287 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_extent_bytes": 0, 2024-04-24T09:18:37.287 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_extent_items": 0, 2024-04-24T09:18:37.287 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_blob_bytes": 0, 2024-04-24T09:18:37.287 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_blob_items": 0, 2024-04-24T09:18:37.287 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-24T09:18:37.287 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_shared_blob_items": 0, 2024-04-24T09:18:37.287 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-24T09:18:37.287 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_inline_bl_items": 0, 2024-04-24T09:18:37.287 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_fsck_bytes": 0, 2024-04-24T09:18:37.287 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_fsck_items": 0, 2024-04-24T09:18:37.287 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_txc_bytes": 0, 2024-04-24T09:18:37.287 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_txc_items": 0, 2024-04-24T09:18:37.288 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-24T09:18:37.288 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_writing_deferred_items": 0, 2024-04-24T09:18:37.288 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_writing_bytes": 0, 2024-04-24T09:18:37.288 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_writing_items": 0, 2024-04-24T09:18:37.288 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_bytes": 0, 2024-04-24T09:18:37.288 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_items": 0, 2024-04-24T09:18:37.288 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_file_reader_bytes": 0, 2024-04-24T09:18:37.288 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_file_reader_items": 0, 2024-04-24T09:18:37.288 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_file_writer_bytes": 0, 2024-04-24T09:18:37.288 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_file_writer_items": 0, 2024-04-24T09:18:37.288 INFO:teuthology.orchestra.run.smithi138.stdout: "buffer_anon_bytes": 478126, 2024-04-24T09:18:37.288 INFO:teuthology.orchestra.run.smithi138.stdout: "buffer_anon_items": 168, 2024-04-24T09:18:37.289 INFO:teuthology.orchestra.run.smithi138.stdout: "buffer_meta_bytes": 0, 2024-04-24T09:18:37.289 INFO:teuthology.orchestra.run.smithi138.stdout: "buffer_meta_items": 0, 2024-04-24T09:18:37.289 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_bytes": 0, 2024-04-24T09:18:37.289 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_items": 0, 2024-04-24T09:18:37.289 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_mapbl_bytes": 0, 2024-04-24T09:18:37.289 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_mapbl_items": 0, 2024-04-24T09:18:37.289 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pglog_bytes": 0, 2024-04-24T09:18:37.289 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pglog_items": 0, 2024-04-24T09:18:37.289 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_bytes": 9800, 2024-04-24T09:18:37.289 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_items": 188, 2024-04-24T09:18:37.289 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_mapping_bytes": 0, 2024-04-24T09:18:37.290 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_mapping_items": 0, 2024-04-24T09:18:37.290 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap_bytes": 0, 2024-04-24T09:18:37.290 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap_items": 0, 2024-04-24T09:18:37.290 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_co_bytes": 4926171, 2024-04-24T09:18:37.290 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_co_items": 22075, 2024-04-24T09:18:37.290 INFO:teuthology.orchestra.run.smithi138.stdout: "unittest_1_bytes": 0, 2024-04-24T09:18:37.290 INFO:teuthology.orchestra.run.smithi138.stdout: "unittest_1_items": 0, 2024-04-24T09:18:37.290 INFO:teuthology.orchestra.run.smithi138.stdout: "unittest_2_bytes": 0, 2024-04-24T09:18:37.290 INFO:teuthology.orchestra.run.smithi138.stdout: "unittest_2_items": 0 2024-04-24T09:18:37.290 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:37.290 INFO:teuthology.orchestra.run.smithi138.stdout: "objecter": { 2024-04-24T09:18:37.290 INFO:teuthology.orchestra.run.smithi138.stdout: "op_active": 0, 2024-04-24T09:18:37.291 INFO:teuthology.orchestra.run.smithi138.stdout: "op_laggy": 0, 2024-04-24T09:18:37.291 INFO:teuthology.orchestra.run.smithi138.stdout: "op_send": 40921, 2024-04-24T09:18:37.291 INFO:teuthology.orchestra.run.smithi138.stdout: "op_send_bytes": 271785133, 2024-04-24T09:18:37.291 INFO:teuthology.orchestra.run.smithi138.stdout: "op_resend": 30, 2024-04-24T09:18:37.291 INFO:teuthology.orchestra.run.smithi138.stdout: "op_reply": 40921, 2024-04-24T09:18:37.291 INFO:teuthology.orchestra.run.smithi138.stdout: "op_latency": { 2024-04-24T09:18:37.291 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 40921, 2024-04-24T09:18:37.291 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 303.162842702, 2024-04-24T09:18:37.291 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.007408490 2024-04-24T09:18:37.291 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:37.291 INFO:teuthology.orchestra.run.smithi138.stdout: "op_inflight": 0, 2024-04-24T09:18:37.291 INFO:teuthology.orchestra.run.smithi138.stdout: "oplen_avg": { 2024-04-24T09:18:37.291 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 40921, 2024-04-24T09:18:37.292 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 43006 2024-04-24T09:18:37.292 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:37.292 INFO:teuthology.orchestra.run.smithi138.stdout: "op": 40921, 2024-04-24T09:18:37.292 INFO:teuthology.orchestra.run.smithi138.stdout: "op_r": 5659, 2024-04-24T09:18:37.292 INFO:teuthology.orchestra.run.smithi138.stdout: "op_w": 35262, 2024-04-24T09:18:37.292 INFO:teuthology.orchestra.run.smithi138.stdout: "op_rmw": 0, 2024-04-24T09:18:37.292 INFO:teuthology.orchestra.run.smithi138.stdout: "op_pg": 0, 2024-04-24T09:18:37.292 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_stat": 0, 2024-04-24T09:18:37.292 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_create": 10, 2024-04-24T09:18:37.292 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_read": 1874, 2024-04-24T09:18:37.292 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_write": 16340, 2024-04-24T09:18:37.292 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_writefull": 1068, 2024-04-24T09:18:37.292 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_writesame": 0, 2024-04-24T09:18:37.292 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_append": 0, 2024-04-24T09:18:37.292 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_zero": 0, 2024-04-24T09:18:37.293 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_truncate": 4, 2024-04-24T09:18:37.293 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_delete": 16582, 2024-04-24T09:18:37.293 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_mapext": 0, 2024-04-24T09:18:37.293 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_sparse_read": 0, 2024-04-24T09:18:37.293 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_clonerange": 0, 2024-04-24T09:18:37.293 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_getxattr": 3785, 2024-04-24T09:18:37.293 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_setxattr": 20, 2024-04-24T09:18:37.293 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_cmpxattr": 0, 2024-04-24T09:18:37.293 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_rmxattr": 0, 2024-04-24T09:18:37.293 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_resetxattrs": 0, 2024-04-24T09:18:37.293 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_call": 0, 2024-04-24T09:18:37.294 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_watch": 0, 2024-04-24T09:18:37.294 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_notify": 0, 2024-04-24T09:18:37.294 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_src_cmpxattr": 0, 2024-04-24T09:18:37.294 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_pgls": 0, 2024-04-24T09:18:37.294 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_pgls_filter": 0, 2024-04-24T09:18:37.294 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_other": 0, 2024-04-24T09:18:37.294 INFO:teuthology.orchestra.run.smithi138.stdout: "linger_active": 0, 2024-04-24T09:18:37.294 INFO:teuthology.orchestra.run.smithi138.stdout: "linger_send": 0, 2024-04-24T09:18:37.294 INFO:teuthology.orchestra.run.smithi138.stdout: "linger_resend": 0, 2024-04-24T09:18:37.294 INFO:teuthology.orchestra.run.smithi138.stdout: "linger_ping": 0, 2024-04-24T09:18:37.294 INFO:teuthology.orchestra.run.smithi138.stdout: "poolop_active": 0, 2024-04-24T09:18:37.294 INFO:teuthology.orchestra.run.smithi138.stdout: "poolop_send": 0, 2024-04-24T09:18:37.294 INFO:teuthology.orchestra.run.smithi138.stdout: "poolop_resend": 0, 2024-04-24T09:18:37.295 INFO:teuthology.orchestra.run.smithi138.stdout: "poolstat_active": 0, 2024-04-24T09:18:37.295 INFO:teuthology.orchestra.run.smithi138.stdout: "poolstat_send": 0, 2024-04-24T09:18:37.295 INFO:teuthology.orchestra.run.smithi138.stdout: "poolstat_resend": 0, 2024-04-24T09:18:37.295 INFO:teuthology.orchestra.run.smithi138.stdout: "statfs_active": 0, 2024-04-24T09:18:37.295 INFO:teuthology.orchestra.run.smithi138.stdout: "statfs_send": 0, 2024-04-24T09:18:37.295 INFO:teuthology.orchestra.run.smithi138.stdout: "statfs_resend": 0, 2024-04-24T09:18:37.295 INFO:teuthology.orchestra.run.smithi138.stdout: "command_active": 0, 2024-04-24T09:18:37.295 INFO:teuthology.orchestra.run.smithi138.stdout: "command_send": 0, 2024-04-24T09:18:37.295 INFO:teuthology.orchestra.run.smithi138.stdout: "command_resend": 0, 2024-04-24T09:18:37.295 INFO:teuthology.orchestra.run.smithi138.stdout: "map_epoch": 136, 2024-04-24T09:18:37.295 INFO:teuthology.orchestra.run.smithi138.stdout: "map_full": 0, 2024-04-24T09:18:37.295 INFO:teuthology.orchestra.run.smithi138.stdout: "map_inc": 47, 2024-04-24T09:18:37.295 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_sessions": 12, 2024-04-24T09:18:37.296 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_session_open": 12, 2024-04-24T09:18:37.296 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_session_close": 0, 2024-04-24T09:18:37.296 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_laggy": 0, 2024-04-24T09:18:37.296 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_wr": 2289, 2024-04-24T09:18:37.296 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_rd": 0, 2024-04-24T09:18:37.296 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_del": 1034 2024-04-24T09:18:37.296 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:37.296 INFO:teuthology.orchestra.run.smithi138.stdout: "oft": { 2024-04-24T09:18:37.296 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_total_objs": 1, 2024-04-24T09:18:37.296 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_total_kv_pairs": 0, 2024-04-24T09:18:37.296 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_total_updates": 0, 2024-04-24T09:18:37.296 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_total_removes": 0 2024-04-24T09:18:37.296 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:37.297 INFO:teuthology.orchestra.run.smithi138.stdout: "osd-slow-ops": { 2024-04-24T09:18:37.297 INFO:teuthology.orchestra.run.smithi138.stdout: "slow_ops_count": 0 2024-04-24T09:18:37.297 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:37.297 INFO:teuthology.orchestra.run.smithi138.stdout: "purge_queue": { 2024-04-24T09:18:37.297 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executing_ops": 0, 2024-04-24T09:18:37.297 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executing_ops_high_water": 10, 2024-04-24T09:18:37.297 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executing": 0, 2024-04-24T09:18:37.297 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executing_high_water": 5, 2024-04-24T09:18:37.297 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executed_ops": 16506, 2024-04-24T09:18:37.297 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executed": 8253, 2024-04-24T09:18:37.297 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_item_in_journal": 0 2024-04-24T09:18:37.297 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:37.297 INFO:teuthology.orchestra.run.smithi138.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-24T09:18:37.298 INFO:teuthology.orchestra.run.smithi138.stdout: "val": 0, 2024-04-24T09:18:37.298 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 104857600, 2024-04-24T09:18:37.298 INFO:teuthology.orchestra.run.smithi138.stdout: "get_started": 0, 2024-04-24T09:18:37.298 INFO:teuthology.orchestra.run.smithi138.stdout: "get": 458387, 2024-04-24T09:18:37.298 INFO:teuthology.orchestra.run.smithi138.stdout: "get_sum": 98942365, 2024-04-24T09:18:37.298 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_fail": 0, 2024-04-24T09:18:37.298 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_success": 458387, 2024-04-24T09:18:37.298 INFO:teuthology.orchestra.run.smithi138.stdout: "take": 0, 2024-04-24T09:18:37.298 INFO:teuthology.orchestra.run.smithi138.stdout: "take_sum": 0, 2024-04-24T09:18:37.298 INFO:teuthology.orchestra.run.smithi138.stdout: "put": 458387, 2024-04-24T09:18:37.298 INFO:teuthology.orchestra.run.smithi138.stdout: "put_sum": 98942365, 2024-04-24T09:18:37.298 INFO:teuthology.orchestra.run.smithi138.stdout: "wait": { 2024-04-24T09:18:37.298 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-24T09:18:37.299 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-24T09:18:37.299 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-24T09:18:37.299 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-24T09:18:37.299 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:37.299 INFO:teuthology.orchestra.run.smithi138.stdout: "throttle-objecter_bytes": { 2024-04-24T09:18:37.299 INFO:teuthology.orchestra.run.smithi138.stdout: "val": 0, 2024-04-24T09:18:37.299 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 104857600, 2024-04-24T09:18:37.299 INFO:teuthology.orchestra.run.smithi138.stdout: "get_started": 0, 2024-04-24T09:18:37.299 INFO:teuthology.orchestra.run.smithi138.stdout: "get": 0, 2024-04-24T09:18:37.299 INFO:teuthology.orchestra.run.smithi138.stdout: "get_sum": 0, 2024-04-24T09:18:37.299 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_fail": 0, 2024-04-24T09:18:37.300 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_success": 0, 2024-04-24T09:18:37.300 INFO:teuthology.orchestra.run.smithi138.stdout: "take": 40921, 2024-04-24T09:18:37.300 INFO:teuthology.orchestra.run.smithi138.stdout: "take_sum": 272693350, 2024-04-24T09:18:37.300 INFO:teuthology.orchestra.run.smithi138.stdout: "put": 40921, 2024-04-24T09:18:37.300 INFO:teuthology.orchestra.run.smithi138.stdout: "put_sum": 272693350, 2024-04-24T09:18:37.300 INFO:teuthology.orchestra.run.smithi138.stdout: "wait": { 2024-04-24T09:18:37.300 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-24T09:18:37.300 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-24T09:18:37.300 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-24T09:18:37.300 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-24T09:18:37.300 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:37.300 INFO:teuthology.orchestra.run.smithi138.stdout: "throttle-objecter_ops": { 2024-04-24T09:18:37.300 INFO:teuthology.orchestra.run.smithi138.stdout: "val": 0, 2024-04-24T09:18:37.301 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 1024, 2024-04-24T09:18:37.301 INFO:teuthology.orchestra.run.smithi138.stdout: "get_started": 0, 2024-04-24T09:18:37.301 INFO:teuthology.orchestra.run.smithi138.stdout: "get": 0, 2024-04-24T09:18:37.301 INFO:teuthology.orchestra.run.smithi138.stdout: "get_sum": 0, 2024-04-24T09:18:37.301 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_fail": 0, 2024-04-24T09:18:37.301 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_success": 0, 2024-04-24T09:18:37.301 INFO:teuthology.orchestra.run.smithi138.stdout: "take": 40921, 2024-04-24T09:18:37.301 INFO:teuthology.orchestra.run.smithi138.stdout: "take_sum": 40921, 2024-04-24T09:18:37.301 INFO:teuthology.orchestra.run.smithi138.stdout: "put": 40921, 2024-04-24T09:18:37.301 INFO:teuthology.orchestra.run.smithi138.stdout: "put_sum": 40921, 2024-04-24T09:18:37.301 INFO:teuthology.orchestra.run.smithi138.stdout: "wait": { 2024-04-24T09:18:37.301 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-24T09:18:37.301 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-24T09:18:37.301 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-24T09:18:37.301 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-24T09:18:37.301 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:37.302 INFO:teuthology.orchestra.run.smithi138.stdout: "throttle-write_buf_throttle": { 2024-04-24T09:18:37.302 INFO:teuthology.orchestra.run.smithi138.stdout: "val": 0, 2024-04-24T09:18:37.302 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 3758096384, 2024-04-24T09:18:37.302 INFO:teuthology.orchestra.run.smithi138.stdout: "get_started": 0, 2024-04-24T09:18:37.302 INFO:teuthology.orchestra.run.smithi138.stdout: "get": 8253, 2024-04-24T09:18:37.302 INFO:teuthology.orchestra.run.smithi138.stdout: "get_sum": 908217, 2024-04-24T09:18:37.302 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_fail": 0, 2024-04-24T09:18:37.302 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_success": 8253, 2024-04-24T09:18:37.302 INFO:teuthology.orchestra.run.smithi138.stdout: "take": 0, 2024-04-24T09:18:37.302 INFO:teuthology.orchestra.run.smithi138.stdout: "take_sum": 0, 2024-04-24T09:18:37.303 INFO:teuthology.orchestra.run.smithi138.stdout: "put": 1874, 2024-04-24T09:18:37.303 INFO:teuthology.orchestra.run.smithi138.stdout: "put_sum": 908217, 2024-04-24T09:18:37.303 INFO:teuthology.orchestra.run.smithi138.stdout: "wait": { 2024-04-24T09:18:37.303 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-24T09:18:37.303 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-24T09:18:37.303 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-24T09:18:37.303 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-24T09:18:37.303 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-24T09:18:37.303 INFO:teuthology.orchestra.run.smithi138.stdout: "throttle-write_buf_throttle-0x55ffbefa8500": { 2024-04-24T09:18:37.303 INFO:teuthology.orchestra.run.smithi138.stdout: "val": 0, 2024-04-24T09:18:37.303 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 3758096384, 2024-04-24T09:18:37.303 INFO:teuthology.orchestra.run.smithi138.stdout: "get_started": 0, 2024-04-24T09:18:37.303 INFO:teuthology.orchestra.run.smithi138.stdout: "get": 53979, 2024-04-24T09:18:37.303 INFO:teuthology.orchestra.run.smithi138.stdout: "get_sum": 269749406, 2024-04-24T09:18:37.303 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_fail": 0, 2024-04-24T09:18:37.303 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_success": 53979, 2024-04-24T09:18:37.304 INFO:teuthology.orchestra.run.smithi138.stdout: "take": 0, 2024-04-24T09:18:37.304 INFO:teuthology.orchestra.run.smithi138.stdout: "take_sum": 0, 2024-04-24T09:18:37.304 INFO:teuthology.orchestra.run.smithi138.stdout: "put": 14466, 2024-04-24T09:18:37.304 INFO:teuthology.orchestra.run.smithi138.stdout: "put_sum": 269749406, 2024-04-24T09:18:37.304 INFO:teuthology.orchestra.run.smithi138.stdout: "wait": { 2024-04-24T09:18:37.304 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-24T09:18:37.304 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-24T09:18:37.304 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-24T09:18:37.304 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-24T09:18:37.304 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-24T09:18:37.304 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-04-24T09:18:38.374 INFO:tasks.check_counter:Daemon mds.k mds.exported=7 2024-04-24T09:18:38.374 INFO:tasks.check_counter:Daemon mds.k mds.imported=7 2024-04-24T09:18:38.374 INFO:tasks.check_counter:Daemon mds.k mds.root_rsnaps=7 2024-04-24T09:18:38.374 INFO:tasks.check_counter:Daemon mds.k mds_server.req_mksnap_latency.avgcount=0 2024-04-24T09:18:38.374 INFO:tasks.check_counter:Daemon mds.k mds_server.req_rmsnap_latency.avgcount=0 2024-04-24T09:18:38.374 DEBUG:tasks.check_counter:Getting stats from c 2024-04-24T09:18:38.375 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid bd3e039e-0215-11ef-bc93-c7b262605968 -- ceph daemon mds.c perf dump 2024-04-24T09:18:38.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:18:38 smithi138 ceph-mon[31441]: pgmap v855: 129 pgs: 129 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-24T09:18:38.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:18:38 smithi189 ceph-mon[31791]: pgmap v855: 129 pgs: 129 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-24T09:18:38.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:18:38 smithi083 ceph-mon[27907]: pgmap v855: 129 pgs: 129 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-24T09:18:38.840 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T09:18:40.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:18:40 smithi138 ceph-mon[31441]: pgmap v856: 129 pgs: 129 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-24T09:18:40.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:18:40 smithi189 ceph-mon[31791]: pgmap v856: 129 pgs: 129 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-24T09:18:40.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:18:40 smithi083 ceph-mon[27907]: pgmap v856: 129 pgs: 129 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-24T09:18:41.112 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-04-24T09:18:41.112 INFO:teuthology.orchestra.run.smithi189.stdout: "AsyncMessenger::Worker-0": { 2024-04-24T09:18:41.112 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_messages": 1533, 2024-04-24T09:18:41.112 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages": 1533, 2024-04-24T09:18:41.112 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_bytes": 99880936, 2024-04-24T09:18:41.112 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_bytes": 486677, 2024-04-24T09:18:41.112 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_created_connections": 8, 2024-04-24T09:18:41.112 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_active_connections": 4, 2024-04-24T09:18:41.112 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_total_time": 0.564761447, 2024-04-24T09:18:41.112 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_send_time": 0.124948182, 2024-04-24T09:18:41.112 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_recv_time": 0.320358708, 2024-04-24T09:18:41.112 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_fast_dispatch_time": 0.087392254, 2024-04-24T09:18:41.113 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T09:18:41.113 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 1532, 2024-04-24T09:18:41.113 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.154922392, 2024-04-24T09:18:41.113 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000101124 2024-04-24T09:18:41.113 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:41.113 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_handle_ack_lat": { 2024-04-24T09:18:41.113 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:41.113 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:41.113 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:41.113 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:41.113 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-04-24T09:18:41.113 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_encrypted_bytes": 448 2024-04-24T09:18:41.113 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:41.113 INFO:teuthology.orchestra.run.smithi189.stdout: "AsyncMessenger::Worker-1": { 2024-04-24T09:18:41.113 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_messages": 2882, 2024-04-24T09:18:41.113 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages": 2837, 2024-04-24T09:18:41.114 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_bytes": 89379923, 2024-04-24T09:18:41.114 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_bytes": 989723, 2024-04-24T09:18:41.114 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_created_connections": 6, 2024-04-24T09:18:41.114 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_active_connections": 5, 2024-04-24T09:18:41.114 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_total_time": 0.842015879, 2024-04-24T09:18:41.114 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_send_time": 0.224272691, 2024-04-24T09:18:41.114 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_recv_time": 0.398210246, 2024-04-24T09:18:41.114 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_fast_dispatch_time": 0.166430828, 2024-04-24T09:18:41.114 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T09:18:41.114 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 2836, 2024-04-24T09:18:41.114 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.277909659, 2024-04-24T09:18:41.114 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000097993 2024-04-24T09:18:41.114 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:41.114 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_handle_ack_lat": { 2024-04-24T09:18:41.114 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:41.115 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:41.115 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:41.115 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:41.115 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_encrypted_bytes": 578960, 2024-04-24T09:18:41.115 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_encrypted_bytes": 200704 2024-04-24T09:18:41.115 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:41.115 INFO:teuthology.orchestra.run.smithi189.stdout: "AsyncMessenger::Worker-2": { 2024-04-24T09:18:41.115 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_messages": 614, 2024-04-24T09:18:41.115 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages": 851, 2024-04-24T09:18:41.115 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_bytes": 37904504, 2024-04-24T09:18:41.115 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_bytes": 726977, 2024-04-24T09:18:41.115 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_created_connections": 7, 2024-04-24T09:18:41.115 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_active_connections": 5, 2024-04-24T09:18:41.116 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_total_time": 0.263307463, 2024-04-24T09:18:41.116 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_send_time": 0.071406278, 2024-04-24T09:18:41.116 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_recv_time": 0.137610967, 2024-04-24T09:18:41.116 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_fast_dispatch_time": 0.035978043, 2024-04-24T09:18:41.116 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T09:18:41.116 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 849, 2024-04-24T09:18:41.116 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.098717849, 2024-04-24T09:18:41.116 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000116275 2024-04-24T09:18:41.116 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:41.116 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_handle_ack_lat": { 2024-04-24T09:18:41.116 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:41.116 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:41.116 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:41.116 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:41.117 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-24T09:18:41.117 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_encrypted_bytes": 532352 2024-04-24T09:18:41.117 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:41.117 INFO:teuthology.orchestra.run.smithi189.stdout: "cct": { 2024-04-24T09:18:41.117 INFO:teuthology.orchestra.run.smithi189.stdout: "total_workers": 1, 2024-04-24T09:18:41.117 INFO:teuthology.orchestra.run.smithi189.stdout: "unhealthy_workers": 0 2024-04-24T09:18:41.117 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:41.117 INFO:teuthology.orchestra.run.smithi189.stdout: "finisher-MDSRank": { 2024-04-24T09:18:41.117 INFO:teuthology.orchestra.run.smithi189.stdout: "queue_len": 0, 2024-04-24T09:18:41.117 INFO:teuthology.orchestra.run.smithi189.stdout: "complete_latency": { 2024-04-24T09:18:41.117 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 7909, 2024-04-24T09:18:41.117 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.770498429, 2024-04-24T09:18:41.117 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000097420 2024-04-24T09:18:41.117 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-04-24T09:18:41.117 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:41.118 INFO:teuthology.orchestra.run.smithi189.stdout: "finisher-PurgeQueue": { 2024-04-24T09:18:41.118 INFO:teuthology.orchestra.run.smithi189.stdout: "queue_len": 0, 2024-04-24T09:18:41.118 INFO:teuthology.orchestra.run.smithi189.stdout: "complete_latency": { 2024-04-24T09:18:41.118 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:41.118 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:41.118 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:41.118 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-04-24T09:18:41.118 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:41.118 INFO:teuthology.orchestra.run.smithi189.stdout: "mds": { 2024-04-24T09:18:41.118 INFO:teuthology.orchestra.run.smithi189.stdout: "request": 0, 2024-04-24T09:18:41.118 INFO:teuthology.orchestra.run.smithi189.stdout: "reply": 0, 2024-04-24T09:18:41.119 INFO:teuthology.orchestra.run.smithi189.stdout: "reply_latency": { 2024-04-24T09:18:41.119 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:41.119 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:41.119 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:41.119 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:41.119 INFO:teuthology.orchestra.run.smithi189.stdout: "slow_reply": 0, 2024-04-24T09:18:41.119 INFO:teuthology.orchestra.run.smithi189.stdout: "forward": 0, 2024-04-24T09:18:41.119 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_fetch_complete": 0, 2024-04-24T09:18:41.119 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_fetch_keys": 0, 2024-04-24T09:18:41.119 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_commit": 0, 2024-04-24T09:18:41.119 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_split": 0, 2024-04-24T09:18:41.119 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_merge": 0, 2024-04-24T09:18:41.119 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes": 9512, 2024-04-24T09:18:41.119 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_top": 6297, 2024-04-24T09:18:41.119 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_bottom": 3215, 2024-04-24T09:18:41.120 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_pin_tail": 0, 2024-04-24T09:18:41.120 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_pinned": 516, 2024-04-24T09:18:41.120 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_expired": 0, 2024-04-24T09:18:41.120 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_with_caps": 0, 2024-04-24T09:18:41.120 INFO:teuthology.orchestra.run.smithi189.stdout: "caps": 0, 2024-04-24T09:18:41.120 INFO:teuthology.orchestra.run.smithi189.stdout: "subtrees": 3, 2024-04-24T09:18:41.120 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse": 0, 2024-04-24T09:18:41.120 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_hit": 0, 2024-04-24T09:18:41.120 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_forward": 0, 2024-04-24T09:18:41.120 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_discover": 0, 2024-04-24T09:18:41.120 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_dir_fetch": 0, 2024-04-24T09:18:41.120 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_remote_ino": 0, 2024-04-24T09:18:41.120 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_lock": 0, 2024-04-24T09:18:41.120 INFO:teuthology.orchestra.run.smithi189.stdout: "load_cent": 0, 2024-04-24T09:18:41.120 INFO:teuthology.orchestra.run.smithi189.stdout: "q": 0, 2024-04-24T09:18:41.121 INFO:teuthology.orchestra.run.smithi189.stdout: "exported": 0, 2024-04-24T09:18:41.121 INFO:teuthology.orchestra.run.smithi189.stdout: "exported_inodes": 0, 2024-04-24T09:18:41.121 INFO:teuthology.orchestra.run.smithi189.stdout: "imported": 0, 2024-04-24T09:18:41.121 INFO:teuthology.orchestra.run.smithi189.stdout: "imported_inodes": 0, 2024-04-24T09:18:41.121 INFO:teuthology.orchestra.run.smithi189.stdout: "openino_dir_fetch": 0, 2024-04-24T09:18:41.121 INFO:teuthology.orchestra.run.smithi189.stdout: "openino_backtrace_fetch": 0, 2024-04-24T09:18:41.121 INFO:teuthology.orchestra.run.smithi189.stdout: "openino_peer_discover": 0, 2024-04-24T09:18:41.121 INFO:teuthology.orchestra.run.smithi189.stdout: "root_rfiles": 4, 2024-04-24T09:18:41.121 INFO:teuthology.orchestra.run.smithi189.stdout: "root_rbytes": 234, 2024-04-24T09:18:41.121 INFO:teuthology.orchestra.run.smithi189.stdout: "root_rsnaps": 7, 2024-04-24T09:18:41.121 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_backtrace_fetch": 0, 2024-04-24T09:18:41.121 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_set_tag": 0, 2024-04-24T09:18:41.121 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_backtrace_repaired": 0, 2024-04-24T09:18:41.121 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_inotable_repaired": 0, 2024-04-24T09:18:41.121 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_dir_inodes": 0, 2024-04-24T09:18:41.122 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_dir_base_inodes": 0, 2024-04-24T09:18:41.122 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_dirfrag_rstats": 0, 2024-04-24T09:18:41.122 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_file_inodes": 0, 2024-04-24T09:18:41.122 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_inode_file_caps": 0, 2024-04-24T09:18:41.122 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_revoke": 0, 2024-04-24T09:18:41.122 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_grant": 0, 2024-04-24T09:18:41.122 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_trunc": 0, 2024-04-24T09:18:41.122 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-24T09:18:41.122 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-24T09:18:41.122 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_caps": 0, 2024-04-24T09:18:41.122 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_caps_dirty": 0, 2024-04-24T09:18:41.122 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_cap_release": 0, 2024-04-24T09:18:41.122 INFO:teuthology.orchestra.run.smithi189.stdout: "process_request_cap_release": 0 2024-04-24T09:18:41.122 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:41.123 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_cache": { 2024-04-24T09:18:41.123 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_update": 0, 2024-04-24T09:18:41.123 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_update_receipt": 0, 2024-04-24T09:18:41.123 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_try_discover": 0, 2024-04-24T09:18:41.123 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_send_discover": 0, 2024-04-24T09:18:41.123 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_handle_discover": 0, 2024-04-24T09:18:41.123 INFO:teuthology.orchestra.run.smithi189.stdout: "num_strays": 0, 2024-04-24T09:18:41.123 INFO:teuthology.orchestra.run.smithi189.stdout: "num_strays_delayed": 0, 2024-04-24T09:18:41.123 INFO:teuthology.orchestra.run.smithi189.stdout: "num_strays_enqueuing": 0, 2024-04-24T09:18:41.123 INFO:teuthology.orchestra.run.smithi189.stdout: "strays_created": 0, 2024-04-24T09:18:41.123 INFO:teuthology.orchestra.run.smithi189.stdout: "strays_enqueued": 0, 2024-04-24T09:18:41.123 INFO:teuthology.orchestra.run.smithi189.stdout: "strays_reintegrated": 0, 2024-04-24T09:18:41.123 INFO:teuthology.orchestra.run.smithi189.stdout: "strays_migrated": 0, 2024-04-24T09:18:41.123 INFO:teuthology.orchestra.run.smithi189.stdout: "num_recovering_processing": 0, 2024-04-24T09:18:41.123 INFO:teuthology.orchestra.run.smithi189.stdout: "num_recovering_enqueued": 0, 2024-04-24T09:18:41.123 INFO:teuthology.orchestra.run.smithi189.stdout: "num_recovering_prioritized": 0, 2024-04-24T09:18:41.124 INFO:teuthology.orchestra.run.smithi189.stdout: "recovery_started": 0, 2024-04-24T09:18:41.124 INFO:teuthology.orchestra.run.smithi189.stdout: "recovery_completed": 0, 2024-04-24T09:18:41.124 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_quiesce_path": 0, 2024-04-24T09:18:41.124 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_quiesce_inode": 0, 2024-04-24T09:18:41.124 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_enqueue_scrub": 0, 2024-04-24T09:18:41.124 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_exportdir": 0, 2024-04-24T09:18:41.124 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_flush": 0, 2024-04-24T09:18:41.124 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_fragmentdir": 0, 2024-04-24T09:18:41.124 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_fragstats": 0, 2024-04-24T09:18:41.124 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_inodestats": 0 2024-04-24T09:18:41.124 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:41.124 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_log": { 2024-04-24T09:18:41.124 INFO:teuthology.orchestra.run.smithi189.stdout: "evlrg": 0, 2024-04-24T09:18:41.124 INFO:teuthology.orchestra.run.smithi189.stdout: "evadd": 0, 2024-04-24T09:18:41.124 INFO:teuthology.orchestra.run.smithi189.stdout: "evex": 0, 2024-04-24T09:18:41.124 INFO:teuthology.orchestra.run.smithi189.stdout: "evtrm": 0, 2024-04-24T09:18:41.125 INFO:teuthology.orchestra.run.smithi189.stdout: "ev": 80734, 2024-04-24T09:18:41.125 INFO:teuthology.orchestra.run.smithi189.stdout: "evexg": 0, 2024-04-24T09:18:41.125 INFO:teuthology.orchestra.run.smithi189.stdout: "evexd": 0, 2024-04-24T09:18:41.125 INFO:teuthology.orchestra.run.smithi189.stdout: "segadd": 0, 2024-04-24T09:18:41.125 INFO:teuthology.orchestra.run.smithi189.stdout: "segex": 0, 2024-04-24T09:18:41.125 INFO:teuthology.orchestra.run.smithi189.stdout: "segtrm": 0, 2024-04-24T09:18:41.125 INFO:teuthology.orchestra.run.smithi189.stdout: "seg": 3, 2024-04-24T09:18:41.125 INFO:teuthology.orchestra.run.smithi189.stdout: "segmjr": 83, 2024-04-24T09:18:41.125 INFO:teuthology.orchestra.run.smithi189.stdout: "segexg": 0, 2024-04-24T09:18:41.125 INFO:teuthology.orchestra.run.smithi189.stdout: "segexd": 0, 2024-04-24T09:18:41.125 INFO:teuthology.orchestra.run.smithi189.stdout: "expos": 249955747, 2024-04-24T09:18:41.125 INFO:teuthology.orchestra.run.smithi189.stdout: "wrpos": 254679510, 2024-04-24T09:18:41.125 INFO:teuthology.orchestra.run.smithi189.stdout: "rdpos": 254679374, 2024-04-24T09:18:41.126 INFO:teuthology.orchestra.run.smithi189.stdout: "jlat": { 2024-04-24T09:18:41.126 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:41.126 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:41.126 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:41.126 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:41.126 INFO:teuthology.orchestra.run.smithi189.stdout: "replayed": 80734 2024-04-24T09:18:41.126 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:41.126 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_mem": { 2024-04-24T09:18:41.126 INFO:teuthology.orchestra.run.smithi189.stdout: "ino": 879, 2024-04-24T09:18:41.126 INFO:teuthology.orchestra.run.smithi189.stdout: "ino+": 12139, 2024-04-24T09:18:41.126 INFO:teuthology.orchestra.run.smithi189.stdout: "ino-": 11260, 2024-04-24T09:18:41.126 INFO:teuthology.orchestra.run.smithi189.stdout: "dir": 54, 2024-04-24T09:18:41.126 INFO:teuthology.orchestra.run.smithi189.stdout: "dir+": 212, 2024-04-24T09:18:41.126 INFO:teuthology.orchestra.run.smithi189.stdout: "dir-": 158, 2024-04-24T09:18:41.127 INFO:teuthology.orchestra.run.smithi189.stdout: "dn": 9512, 2024-04-24T09:18:41.127 INFO:teuthology.orchestra.run.smithi189.stdout: "dn+": 12847, 2024-04-24T09:18:41.127 INFO:teuthology.orchestra.run.smithi189.stdout: "dn-": 3335, 2024-04-24T09:18:41.127 INFO:teuthology.orchestra.run.smithi189.stdout: "cap": 0, 2024-04-24T09:18:41.127 INFO:teuthology.orchestra.run.smithi189.stdout: "cap+": 0, 2024-04-24T09:18:41.127 INFO:teuthology.orchestra.run.smithi189.stdout: "cap-": 0, 2024-04-24T09:18:41.127 INFO:teuthology.orchestra.run.smithi189.stdout: "rss": 69032, 2024-04-24T09:18:41.127 INFO:teuthology.orchestra.run.smithi189.stdout: "heap": 215392 2024-04-24T09:18:41.127 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:41.127 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_server": { 2024-04-24T09:18:41.127 INFO:teuthology.orchestra.run.smithi189.stdout: "dispatch_client_request": 0, 2024-04-24T09:18:41.127 INFO:teuthology.orchestra.run.smithi189.stdout: "dispatch_server_request": 0, 2024-04-24T09:18:41.127 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_request": 0, 2024-04-24T09:18:41.127 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_session": 0, 2024-04-24T09:18:41.128 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_peer_request": 0, 2024-04-24T09:18:41.128 INFO:teuthology.orchestra.run.smithi189.stdout: "req_create_latency": { 2024-04-24T09:18:41.128 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:41.128 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:41.128 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:41.128 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:41.128 INFO:teuthology.orchestra.run.smithi189.stdout: "req_getattr_latency": { 2024-04-24T09:18:41.128 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:41.128 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:41.128 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:41.128 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:41.128 INFO:teuthology.orchestra.run.smithi189.stdout: "req_getfilelock_latency": { 2024-04-24T09:18:41.128 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:41.128 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:41.128 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:41.128 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:41.129 INFO:teuthology.orchestra.run.smithi189.stdout: "req_link_latency": { 2024-04-24T09:18:41.129 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:41.129 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:41.129 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:41.129 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:41.129 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookup_latency": { 2024-04-24T09:18:41.129 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:41.129 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:41.129 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:41.129 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:41.129 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookuphash_latency": { 2024-04-24T09:18:41.129 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:41.129 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:41.129 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:41.129 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:41.129 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookupino_latency": { 2024-04-24T09:18:41.129 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:41.130 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:41.130 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:41.130 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:41.130 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookupname_latency": { 2024-04-24T09:18:41.130 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:41.130 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:41.130 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:41.130 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:41.130 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookupparent_latency": { 2024-04-24T09:18:41.130 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:41.130 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:41.130 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:41.130 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:41.130 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookupsnap_latency": { 2024-04-24T09:18:41.130 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:41.131 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:41.131 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:41.131 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:41.131 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lssnap_latency": { 2024-04-24T09:18:41.131 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:41.131 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:41.131 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:41.131 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:41.131 INFO:teuthology.orchestra.run.smithi189.stdout: "req_mkdir_latency": { 2024-04-24T09:18:41.132 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:41.132 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:41.132 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:41.132 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:41.132 INFO:teuthology.orchestra.run.smithi189.stdout: "req_mknod_latency": { 2024-04-24T09:18:41.132 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:41.132 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:41.132 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:41.132 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:41.132 INFO:teuthology.orchestra.run.smithi189.stdout: "req_mksnap_latency": { 2024-04-24T09:18:41.132 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:41.132 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:41.132 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:41.132 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:41.132 INFO:teuthology.orchestra.run.smithi189.stdout: "req_open_latency": { 2024-04-24T09:18:41.133 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:41.133 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:41.133 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:41.133 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:41.133 INFO:teuthology.orchestra.run.smithi189.stdout: "req_readdir_latency": { 2024-04-24T09:18:41.133 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:41.133 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:41.133 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:41.133 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:41.133 INFO:teuthology.orchestra.run.smithi189.stdout: "req_rename_latency": { 2024-04-24T09:18:41.133 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:41.133 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:41.133 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:41.133 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:41.134 INFO:teuthology.orchestra.run.smithi189.stdout: "req_renamesnap_latency": { 2024-04-24T09:18:41.134 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:41.134 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:41.134 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:41.134 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:41.134 INFO:teuthology.orchestra.run.smithi189.stdout: "req_snapdiff_latency": { 2024-04-24T09:18:41.134 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:41.134 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:41.134 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:41.134 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:41.134 INFO:teuthology.orchestra.run.smithi189.stdout: "req_rmdir_latency": { 2024-04-24T09:18:41.134 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:41.134 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:41.134 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:41.135 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:41.135 INFO:teuthology.orchestra.run.smithi189.stdout: "req_rmsnap_latency": { 2024-04-24T09:18:41.135 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:41.135 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:41.135 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:41.135 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:41.135 INFO:teuthology.orchestra.run.smithi189.stdout: "req_rmxattr_latency": { 2024-04-24T09:18:41.135 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:41.135 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:41.135 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:41.135 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:41.135 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setattr_latency": { 2024-04-24T09:18:41.135 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:41.135 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:41.135 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:41.135 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:41.136 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setdirlayout_latency": { 2024-04-24T09:18:41.136 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:41.136 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:41.136 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:41.136 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:41.136 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setfilelock_latency": { 2024-04-24T09:18:41.136 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:41.136 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:41.136 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:41.136 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:41.136 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setlayout_latency": { 2024-04-24T09:18:41.136 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:41.136 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:41.136 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:41.136 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:41.136 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setxattr_latency": { 2024-04-24T09:18:41.137 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:41.137 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:41.137 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:41.137 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:41.137 INFO:teuthology.orchestra.run.smithi189.stdout: "req_symlink_latency": { 2024-04-24T09:18:41.137 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:41.137 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:41.137 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:41.137 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:41.137 INFO:teuthology.orchestra.run.smithi189.stdout: "req_unlink_latency": { 2024-04-24T09:18:41.137 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:41.137 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:41.137 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:41.137 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:41.138 INFO:teuthology.orchestra.run.smithi189.stdout: "cap_revoke_eviction": 0, 2024-04-24T09:18:41.138 INFO:teuthology.orchestra.run.smithi189.stdout: "cap_acquisition_throttle": 0, 2024-04-24T09:18:41.138 INFO:teuthology.orchestra.run.smithi189.stdout: "req_getvxattr_latency": { 2024-04-24T09:18:41.138 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:41.138 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:41.138 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:41.138 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-04-24T09:18:41.138 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:41.138 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_sessions": { 2024-04-24T09:18:41.138 INFO:teuthology.orchestra.run.smithi189.stdout: "session_count": 2, 2024-04-24T09:18:41.138 INFO:teuthology.orchestra.run.smithi189.stdout: "session_add": 2, 2024-04-24T09:18:41.138 INFO:teuthology.orchestra.run.smithi189.stdout: "session_remove": 0, 2024-04-24T09:18:41.138 INFO:teuthology.orchestra.run.smithi189.stdout: "sessions_open": 0, 2024-04-24T09:18:41.138 INFO:teuthology.orchestra.run.smithi189.stdout: "sessions_stale": 0, 2024-04-24T09:18:41.139 INFO:teuthology.orchestra.run.smithi189.stdout: "total_load": 0, 2024-04-24T09:18:41.139 INFO:teuthology.orchestra.run.smithi189.stdout: "average_load": 0, 2024-04-24T09:18:41.139 INFO:teuthology.orchestra.run.smithi189.stdout: "avg_session_uptime": 3008, 2024-04-24T09:18:41.139 INFO:teuthology.orchestra.run.smithi189.stdout: "mdthresh_evicted": 0 2024-04-24T09:18:41.139 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:41.139 INFO:teuthology.orchestra.run.smithi189.stdout: "mempool": { 2024-04-24T09:18:41.139 INFO:teuthology.orchestra.run.smithi189.stdout: "bloom_filter_bytes": 0, 2024-04-24T09:18:41.139 INFO:teuthology.orchestra.run.smithi189.stdout: "bloom_filter_items": 0, 2024-04-24T09:18:41.139 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_alloc_bytes": 0, 2024-04-24T09:18:41.139 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_alloc_items": 0, 2024-04-24T09:18:41.139 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_data_bytes": 0, 2024-04-24T09:18:41.139 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_data_items": 0, 2024-04-24T09:18:41.139 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-24T09:18:41.139 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_onode_items": 0, 2024-04-24T09:18:41.140 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-24T09:18:41.140 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_meta_items": 0, 2024-04-24T09:18:41.140 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_other_bytes": 0, 2024-04-24T09:18:41.140 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_other_items": 0, 2024-04-24T09:18:41.140 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-24T09:18:41.140 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_buffer_items": 0, 2024-04-24T09:18:41.140 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_extent_bytes": 0, 2024-04-24T09:18:41.140 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_extent_items": 0, 2024-04-24T09:18:41.140 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_blob_bytes": 0, 2024-04-24T09:18:41.140 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_blob_items": 0, 2024-04-24T09:18:41.140 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-24T09:18:41.140 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_shared_blob_items": 0, 2024-04-24T09:18:41.140 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-24T09:18:41.140 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_inline_bl_items": 0, 2024-04-24T09:18:41.140 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_fsck_bytes": 0, 2024-04-24T09:18:41.141 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_fsck_items": 0, 2024-04-24T09:18:41.141 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_txc_bytes": 0, 2024-04-24T09:18:41.141 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_txc_items": 0, 2024-04-24T09:18:41.141 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-24T09:18:41.141 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_deferred_items": 0, 2024-04-24T09:18:41.141 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_bytes": 0, 2024-04-24T09:18:41.141 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_items": 0, 2024-04-24T09:18:41.141 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_bytes": 0, 2024-04-24T09:18:41.141 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_items": 0, 2024-04-24T09:18:41.141 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_reader_bytes": 0, 2024-04-24T09:18:41.141 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_reader_items": 0, 2024-04-24T09:18:41.141 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_writer_bytes": 0, 2024-04-24T09:18:41.141 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_writer_items": 0, 2024-04-24T09:18:41.141 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_anon_bytes": 240983, 2024-04-24T09:18:41.142 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_anon_items": 81, 2024-04-24T09:18:41.142 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_meta_bytes": 0, 2024-04-24T09:18:41.142 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_meta_items": 0, 2024-04-24T09:18:41.142 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_bytes": 0, 2024-04-24T09:18:41.142 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_items": 0, 2024-04-24T09:18:41.142 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_mapbl_bytes": 0, 2024-04-24T09:18:41.142 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_mapbl_items": 0, 2024-04-24T09:18:41.142 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_pglog_bytes": 0, 2024-04-24T09:18:41.142 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_pglog_items": 0, 2024-04-24T09:18:41.142 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_bytes": 9800, 2024-04-24T09:18:41.142 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_items": 188, 2024-04-24T09:18:41.142 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_mapping_bytes": 0, 2024-04-24T09:18:41.142 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_mapping_items": 0, 2024-04-24T09:18:41.142 INFO:teuthology.orchestra.run.smithi189.stdout: "pgmap_bytes": 0, 2024-04-24T09:18:41.142 INFO:teuthology.orchestra.run.smithi189.stdout: "pgmap_items": 0, 2024-04-24T09:18:41.143 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_co_bytes": 8770931, 2024-04-24T09:18:41.143 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_co_items": 44151, 2024-04-24T09:18:41.143 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_1_bytes": 0, 2024-04-24T09:18:41.143 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_1_items": 0, 2024-04-24T09:18:41.143 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_2_bytes": 0, 2024-04-24T09:18:41.143 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_2_items": 0 2024-04-24T09:18:41.143 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:41.143 INFO:teuthology.orchestra.run.smithi189.stdout: "objecter": { 2024-04-24T09:18:41.143 INFO:teuthology.orchestra.run.smithi189.stdout: "op_active": 0, 2024-04-24T09:18:41.143 INFO:teuthology.orchestra.run.smithi189.stdout: "op_laggy": 0, 2024-04-24T09:18:41.143 INFO:teuthology.orchestra.run.smithi189.stdout: "op_send": 4622, 2024-04-24T09:18:41.143 INFO:teuthology.orchestra.run.smithi189.stdout: "op_send_bytes": 26, 2024-04-24T09:18:41.143 INFO:teuthology.orchestra.run.smithi189.stdout: "op_resend": 4, 2024-04-24T09:18:41.143 INFO:teuthology.orchestra.run.smithi189.stdout: "op_reply": 4622, 2024-04-24T09:18:41.143 INFO:teuthology.orchestra.run.smithi189.stdout: "op_latency": { 2024-04-24T09:18:41.144 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 4622, 2024-04-24T09:18:41.144 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 9.389817598, 2024-04-24T09:18:41.144 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.002031548 2024-04-24T09:18:41.144 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:41.144 INFO:teuthology.orchestra.run.smithi189.stdout: "op_inflight": 0, 2024-04-24T09:18:41.144 INFO:teuthology.orchestra.run.smithi189.stdout: "oplen_avg": { 2024-04-24T09:18:41.144 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 4622, 2024-04-24T09:18:41.144 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 4623 2024-04-24T09:18:41.144 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:41.144 INFO:teuthology.orchestra.run.smithi189.stdout: "op": 4622, 2024-04-24T09:18:41.144 INFO:teuthology.orchestra.run.smithi189.stdout: "op_r": 4622, 2024-04-24T09:18:41.144 INFO:teuthology.orchestra.run.smithi189.stdout: "op_w": 0, 2024-04-24T09:18:41.144 INFO:teuthology.orchestra.run.smithi189.stdout: "op_rmw": 0, 2024-04-24T09:18:41.144 INFO:teuthology.orchestra.run.smithi189.stdout: "op_pg": 0, 2024-04-24T09:18:41.144 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_stat": 2314, 2024-04-24T09:18:41.144 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_create": 0, 2024-04-24T09:18:41.145 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_read": 2305, 2024-04-24T09:18:41.145 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_write": 0, 2024-04-24T09:18:41.145 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_writefull": 0, 2024-04-24T09:18:41.145 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_writesame": 0, 2024-04-24T09:18:41.145 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_append": 0, 2024-04-24T09:18:41.145 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_zero": 0, 2024-04-24T09:18:41.145 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_truncate": 0, 2024-04-24T09:18:41.145 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_delete": 0, 2024-04-24T09:18:41.145 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_mapext": 0, 2024-04-24T09:18:41.145 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_sparse_read": 0, 2024-04-24T09:18:41.145 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_clonerange": 0, 2024-04-24T09:18:41.145 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_getxattr": 2, 2024-04-24T09:18:41.145 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_setxattr": 0, 2024-04-24T09:18:41.145 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_cmpxattr": 0, 2024-04-24T09:18:41.145 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_rmxattr": 0, 2024-04-24T09:18:41.145 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_resetxattrs": 0, 2024-04-24T09:18:41.145 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_call": 0, 2024-04-24T09:18:41.146 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_watch": 0, 2024-04-24T09:18:41.146 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_notify": 0, 2024-04-24T09:18:41.146 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_src_cmpxattr": 0, 2024-04-24T09:18:41.146 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_pgls": 0, 2024-04-24T09:18:41.146 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_pgls_filter": 0, 2024-04-24T09:18:41.146 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_other": 0, 2024-04-24T09:18:41.146 INFO:teuthology.orchestra.run.smithi189.stdout: "linger_active": 0, 2024-04-24T09:18:41.146 INFO:teuthology.orchestra.run.smithi189.stdout: "linger_send": 0, 2024-04-24T09:18:41.146 INFO:teuthology.orchestra.run.smithi189.stdout: "linger_resend": 0, 2024-04-24T09:18:41.146 INFO:teuthology.orchestra.run.smithi189.stdout: "linger_ping": 0, 2024-04-24T09:18:41.146 INFO:teuthology.orchestra.run.smithi189.stdout: "poolop_active": 0, 2024-04-24T09:18:41.146 INFO:teuthology.orchestra.run.smithi189.stdout: "poolop_send": 0, 2024-04-24T09:18:41.146 INFO:teuthology.orchestra.run.smithi189.stdout: "poolop_resend": 0, 2024-04-24T09:18:41.146 INFO:teuthology.orchestra.run.smithi189.stdout: "poolstat_active": 0, 2024-04-24T09:18:41.146 INFO:teuthology.orchestra.run.smithi189.stdout: "poolstat_send": 0, 2024-04-24T09:18:41.146 INFO:teuthology.orchestra.run.smithi189.stdout: "poolstat_resend": 0, 2024-04-24T09:18:41.147 INFO:teuthology.orchestra.run.smithi189.stdout: "statfs_active": 0, 2024-04-24T09:18:41.147 INFO:teuthology.orchestra.run.smithi189.stdout: "statfs_send": 0, 2024-04-24T09:18:41.147 INFO:teuthology.orchestra.run.smithi189.stdout: "statfs_resend": 0, 2024-04-24T09:18:41.147 INFO:teuthology.orchestra.run.smithi189.stdout: "command_active": 0, 2024-04-24T09:18:41.147 INFO:teuthology.orchestra.run.smithi189.stdout: "command_send": 0, 2024-04-24T09:18:41.147 INFO:teuthology.orchestra.run.smithi189.stdout: "command_resend": 0, 2024-04-24T09:18:41.147 INFO:teuthology.orchestra.run.smithi189.stdout: "map_epoch": 136, 2024-04-24T09:18:41.147 INFO:teuthology.orchestra.run.smithi189.stdout: "map_full": 0, 2024-04-24T09:18:41.147 INFO:teuthology.orchestra.run.smithi189.stdout: "map_inc": 37, 2024-04-24T09:18:41.147 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_sessions": 11, 2024-04-24T09:18:41.147 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_session_open": 11, 2024-04-24T09:18:41.147 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_session_close": 0, 2024-04-24T09:18:41.147 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_laggy": 0, 2024-04-24T09:18:41.147 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_wr": 0, 2024-04-24T09:18:41.147 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_rd": 2, 2024-04-24T09:18:41.148 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_del": 0 2024-04-24T09:18:41.148 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:41.148 INFO:teuthology.orchestra.run.smithi189.stdout: "oft": { 2024-04-24T09:18:41.148 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_total_objs": 0, 2024-04-24T09:18:41.148 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_total_kv_pairs": 0, 2024-04-24T09:18:41.148 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_total_updates": 0, 2024-04-24T09:18:41.148 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_total_removes": 0 2024-04-24T09:18:41.148 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:41.148 INFO:teuthology.orchestra.run.smithi189.stdout: "osd-slow-ops": { 2024-04-24T09:18:41.148 INFO:teuthology.orchestra.run.smithi189.stdout: "slow_ops_count": 0 2024-04-24T09:18:41.148 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:41.148 INFO:teuthology.orchestra.run.smithi189.stdout: "purge_queue": { 2024-04-24T09:18:41.148 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executing_ops": 0, 2024-04-24T09:18:41.148 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executing_ops_high_water": 0, 2024-04-24T09:18:41.148 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executing": 0, 2024-04-24T09:18:41.148 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executing_high_water": 0, 2024-04-24T09:18:41.149 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executed_ops": 0, 2024-04-24T09:18:41.149 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executed": 0, 2024-04-24T09:18:41.149 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_item_in_journal": 0 2024-04-24T09:18:41.149 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:41.149 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-24T09:18:41.149 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-04-24T09:18:41.149 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 104857600, 2024-04-24T09:18:41.149 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-04-24T09:18:41.149 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 5025, 2024-04-24T09:18:41.149 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 226689750, 2024-04-24T09:18:41.149 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-04-24T09:18:41.149 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 5025, 2024-04-24T09:18:41.149 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 0, 2024-04-24T09:18:41.149 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 0, 2024-04-24T09:18:41.149 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 5025, 2024-04-24T09:18:41.150 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 226689750, 2024-04-24T09:18:41.150 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-04-24T09:18:41.150 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:41.150 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:41.150 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:41.150 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-04-24T09:18:41.150 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:41.150 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-objecter_bytes": { 2024-04-24T09:18:41.150 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-04-24T09:18:41.150 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 104857600, 2024-04-24T09:18:41.150 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-04-24T09:18:41.150 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 0, 2024-04-24T09:18:41.150 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 0, 2024-04-24T09:18:41.150 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-04-24T09:18:41.150 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 0, 2024-04-24T09:18:41.151 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 4622, 2024-04-24T09:18:41.151 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 225312099, 2024-04-24T09:18:41.151 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 4622, 2024-04-24T09:18:41.151 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 225312099, 2024-04-24T09:18:41.151 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-04-24T09:18:41.151 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:41.151 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:41.151 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:41.151 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-04-24T09:18:41.151 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:41.151 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-objecter_ops": { 2024-04-24T09:18:41.151 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-04-24T09:18:41.151 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 1024, 2024-04-24T09:18:41.151 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-04-24T09:18:41.151 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 0, 2024-04-24T09:18:41.152 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 0, 2024-04-24T09:18:41.152 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-04-24T09:18:41.152 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 0, 2024-04-24T09:18:41.152 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 4622, 2024-04-24T09:18:41.152 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 4622, 2024-04-24T09:18:41.152 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 4622, 2024-04-24T09:18:41.152 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 4622, 2024-04-24T09:18:41.152 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-04-24T09:18:41.152 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:41.152 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:41.152 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:41.152 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-04-24T09:18:41.152 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:41.152 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-write_buf_throttle": { 2024-04-24T09:18:41.152 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-04-24T09:18:41.152 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 3758096384, 2024-04-24T09:18:41.153 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-04-24T09:18:41.153 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 0, 2024-04-24T09:18:41.153 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 0, 2024-04-24T09:18:41.153 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-04-24T09:18:41.153 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 0, 2024-04-24T09:18:41.153 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 0, 2024-04-24T09:18:41.153 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 0, 2024-04-24T09:18:41.153 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 0, 2024-04-24T09:18:41.153 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 0, 2024-04-24T09:18:41.153 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-04-24T09:18:41.153 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:41.153 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:41.153 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:41.153 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-04-24T09:18:41.153 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:41.153 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-write_buf_throttle-0x5607e16ee780": { 2024-04-24T09:18:41.154 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-04-24T09:18:41.154 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 3758096384, 2024-04-24T09:18:41.154 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-04-24T09:18:41.154 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 0, 2024-04-24T09:18:41.154 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 0, 2024-04-24T09:18:41.154 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-04-24T09:18:41.154 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 0, 2024-04-24T09:18:41.154 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 0, 2024-04-24T09:18:41.154 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 0, 2024-04-24T09:18:41.154 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 0, 2024-04-24T09:18:41.154 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 0, 2024-04-24T09:18:41.154 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-04-24T09:18:41.154 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:41.154 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:41.154 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:41.154 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-04-24T09:18:41.155 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-04-24T09:18:41.155 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-04-24T09:18:42.072 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-04-24T09:18:42.072 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-04-24T09:18:42.072 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=7 2024-04-24T09:18:42.072 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-04-24T09:18:42.072 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-04-24T09:18:42.072 DEBUG:tasks.check_counter:Getting stats from f 2024-04-24T09:18:42.073 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid bd3e039e-0215-11ef-bc93-c7b262605968 -- ceph daemon mds.f perf dump 2024-04-24T09:18:42.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:18:42 smithi138 ceph-mon[31441]: pgmap v857: 129 pgs: 129 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-24T09:18:42.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:18:42 smithi189 ceph-mon[31791]: pgmap v857: 129 pgs: 129 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-24T09:18:42.515 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T09:18:42.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:18:42 smithi083 ceph-mon[27907]: pgmap v857: 129 pgs: 129 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-24T09:18:44.373 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:18:44 smithi189 ceph-mon[31791]: pgmap v858: 129 pgs: 129 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-24T09:18:44.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:18:44 smithi138 ceph-mon[31441]: pgmap v858: 129 pgs: 129 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-24T09:18:44.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:18:44 smithi083 ceph-mon[27907]: pgmap v858: 129 pgs: 129 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-24T09:18:44.925 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-04-24T09:18:44.925 INFO:teuthology.orchestra.run.smithi189.stdout: "AsyncMessenger::Worker-0": { 2024-04-24T09:18:44.925 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_messages": 3184, 2024-04-24T09:18:44.925 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages": 3119, 2024-04-24T09:18:44.925 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_bytes": 111937660, 2024-04-24T09:18:44.925 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_bytes": 1131568, 2024-04-24T09:18:44.925 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_created_connections": 7, 2024-04-24T09:18:44.925 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_active_connections": 3, 2024-04-24T09:18:44.925 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_total_time": 1.065730555, 2024-04-24T09:18:44.925 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_send_time": 0.249383459, 2024-04-24T09:18:44.925 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_recv_time": 0.603678585, 2024-04-24T09:18:44.926 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_fast_dispatch_time": 0.179218774, 2024-04-24T09:18:44.926 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T09:18:44.926 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 3119, 2024-04-24T09:18:44.926 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.316266007, 2024-04-24T09:18:44.926 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000101399 2024-04-24T09:18:44.926 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:44.926 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_handle_ack_lat": { 2024-04-24T09:18:44.926 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:44.926 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:44.926 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:44.926 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:44.926 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_encrypted_bytes": 2079808, 2024-04-24T09:18:44.926 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_encrypted_bytes": 312400 2024-04-24T09:18:44.926 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:44.926 INFO:teuthology.orchestra.run.smithi189.stdout: "AsyncMessenger::Worker-1": { 2024-04-24T09:18:44.927 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_messages": 961, 2024-04-24T09:18:44.927 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages": 1332, 2024-04-24T09:18:44.927 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_bytes": 97986980, 2024-04-24T09:18:44.927 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_bytes": 1053088, 2024-04-24T09:18:44.927 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_created_connections": 9, 2024-04-24T09:18:44.927 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_active_connections": 7, 2024-04-24T09:18:44.927 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_total_time": 0.531946436, 2024-04-24T09:18:44.927 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_send_time": 0.099074568, 2024-04-24T09:18:44.927 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_recv_time": 0.347132127, 2024-04-24T09:18:44.927 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_fast_dispatch_time": 0.056240184, 2024-04-24T09:18:44.927 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T09:18:44.927 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 1329, 2024-04-24T09:18:44.927 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.141506136, 2024-04-24T09:18:44.927 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000106475 2024-04-24T09:18:44.927 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:44.927 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_handle_ack_lat": { 2024-04-24T09:18:44.928 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:44.928 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:44.928 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:44.928 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:44.928 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-24T09:18:44.928 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_encrypted_bytes": 748064 2024-04-24T09:18:44.928 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:44.928 INFO:teuthology.orchestra.run.smithi189.stdout: "AsyncMessenger::Worker-2": { 2024-04-24T09:18:44.928 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_messages": 2958, 2024-04-24T09:18:44.928 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages": 2960, 2024-04-24T09:18:44.928 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_bytes": 126581097, 2024-04-24T09:18:44.928 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_bytes": 940651, 2024-04-24T09:18:44.928 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_created_connections": 6, 2024-04-24T09:18:44.928 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_active_connections": 5, 2024-04-24T09:18:44.928 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_total_time": 0.893644364, 2024-04-24T09:18:44.928 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_send_time": 0.227273280, 2024-04-24T09:18:44.928 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_recv_time": 0.455112600, 2024-04-24T09:18:44.929 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_fast_dispatch_time": 0.165483041, 2024-04-24T09:18:44.929 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T09:18:44.929 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 2958, 2024-04-24T09:18:44.929 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.271776497, 2024-04-24T09:18:44.929 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000091878 2024-04-24T09:18:44.929 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:44.929 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_handle_ack_lat": { 2024-04-24T09:18:44.929 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:44.929 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:44.929 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:44.929 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:44.929 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-24T09:18:44.929 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_encrypted_bytes": 0 2024-04-24T09:18:44.929 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:44.929 INFO:teuthology.orchestra.run.smithi189.stdout: "cct": { 2024-04-24T09:18:44.929 INFO:teuthology.orchestra.run.smithi189.stdout: "total_workers": 1, 2024-04-24T09:18:44.930 INFO:teuthology.orchestra.run.smithi189.stdout: "unhealthy_workers": 0 2024-04-24T09:18:44.930 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:44.930 INFO:teuthology.orchestra.run.smithi189.stdout: "finisher-MDSRank": { 2024-04-24T09:18:44.930 INFO:teuthology.orchestra.run.smithi189.stdout: "queue_len": 0, 2024-04-24T09:18:44.930 INFO:teuthology.orchestra.run.smithi189.stdout: "complete_latency": { 2024-04-24T09:18:44.930 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 10973, 2024-04-24T09:18:44.930 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.585118630, 2024-04-24T09:18:44.930 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000053323 2024-04-24T09:18:44.930 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-04-24T09:18:44.930 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:44.930 INFO:teuthology.orchestra.run.smithi189.stdout: "finisher-PurgeQueue": { 2024-04-24T09:18:44.930 INFO:teuthology.orchestra.run.smithi189.stdout: "queue_len": 0, 2024-04-24T09:18:44.930 INFO:teuthology.orchestra.run.smithi189.stdout: "complete_latency": { 2024-04-24T09:18:44.930 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:44.930 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:44.930 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:44.931 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-04-24T09:18:44.931 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:44.931 INFO:teuthology.orchestra.run.smithi189.stdout: "mds": { 2024-04-24T09:18:44.931 INFO:teuthology.orchestra.run.smithi189.stdout: "request": 0, 2024-04-24T09:18:44.931 INFO:teuthology.orchestra.run.smithi189.stdout: "reply": 0, 2024-04-24T09:18:44.931 INFO:teuthology.orchestra.run.smithi189.stdout: "reply_latency": { 2024-04-24T09:18:44.931 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:44.931 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:44.931 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:44.931 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:44.931 INFO:teuthology.orchestra.run.smithi189.stdout: "slow_reply": 0, 2024-04-24T09:18:44.931 INFO:teuthology.orchestra.run.smithi189.stdout: "forward": 0, 2024-04-24T09:18:44.931 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_fetch_complete": 0, 2024-04-24T09:18:44.931 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_fetch_keys": 0, 2024-04-24T09:18:44.931 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_commit": 0, 2024-04-24T09:18:44.932 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_split": 0, 2024-04-24T09:18:44.932 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_merge": 0, 2024-04-24T09:18:44.932 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes": 10739, 2024-04-24T09:18:44.932 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_top": 311, 2024-04-24T09:18:44.932 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_bottom": 10428, 2024-04-24T09:18:44.932 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_pin_tail": 0, 2024-04-24T09:18:44.932 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_pinned": 10294, 2024-04-24T09:18:44.932 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_expired": 1, 2024-04-24T09:18:44.932 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_with_caps": 0, 2024-04-24T09:18:44.932 INFO:teuthology.orchestra.run.smithi189.stdout: "caps": 0, 2024-04-24T09:18:44.932 INFO:teuthology.orchestra.run.smithi189.stdout: "subtrees": 1, 2024-04-24T09:18:44.932 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse": 0, 2024-04-24T09:18:44.932 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_hit": 0, 2024-04-24T09:18:44.932 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_forward": 0, 2024-04-24T09:18:44.932 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_discover": 0, 2024-04-24T09:18:44.932 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_dir_fetch": 0, 2024-04-24T09:18:44.933 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_remote_ino": 0, 2024-04-24T09:18:44.933 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_lock": 0, 2024-04-24T09:18:44.933 INFO:teuthology.orchestra.run.smithi189.stdout: "load_cent": 0, 2024-04-24T09:18:44.933 INFO:teuthology.orchestra.run.smithi189.stdout: "q": 0, 2024-04-24T09:18:44.933 INFO:teuthology.orchestra.run.smithi189.stdout: "exported": 0, 2024-04-24T09:18:44.933 INFO:teuthology.orchestra.run.smithi189.stdout: "exported_inodes": 0, 2024-04-24T09:18:44.933 INFO:teuthology.orchestra.run.smithi189.stdout: "imported": 0, 2024-04-24T09:18:44.933 INFO:teuthology.orchestra.run.smithi189.stdout: "imported_inodes": 0, 2024-04-24T09:18:44.933 INFO:teuthology.orchestra.run.smithi189.stdout: "openino_dir_fetch": 0, 2024-04-24T09:18:44.933 INFO:teuthology.orchestra.run.smithi189.stdout: "openino_backtrace_fetch": 0, 2024-04-24T09:18:44.933 INFO:teuthology.orchestra.run.smithi189.stdout: "openino_peer_discover": 0, 2024-04-24T09:18:44.933 INFO:teuthology.orchestra.run.smithi189.stdout: "root_rfiles": 0, 2024-04-24T09:18:44.933 INFO:teuthology.orchestra.run.smithi189.stdout: "root_rbytes": 0, 2024-04-24T09:18:44.933 INFO:teuthology.orchestra.run.smithi189.stdout: "root_rsnaps": 0, 2024-04-24T09:18:44.933 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_backtrace_fetch": 0, 2024-04-24T09:18:44.933 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_set_tag": 0, 2024-04-24T09:18:44.933 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_backtrace_repaired": 0, 2024-04-24T09:18:44.934 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_inotable_repaired": 0, 2024-04-24T09:18:44.934 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_dir_inodes": 0, 2024-04-24T09:18:44.934 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_dir_base_inodes": 0, 2024-04-24T09:18:44.934 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_dirfrag_rstats": 0, 2024-04-24T09:18:44.934 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_file_inodes": 0, 2024-04-24T09:18:44.934 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_inode_file_caps": 0, 2024-04-24T09:18:44.934 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_revoke": 0, 2024-04-24T09:18:44.934 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_grant": 0, 2024-04-24T09:18:44.934 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_trunc": 0, 2024-04-24T09:18:44.934 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-24T09:18:44.934 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-24T09:18:44.934 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_caps": 0, 2024-04-24T09:18:44.934 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_caps_dirty": 0, 2024-04-24T09:18:44.934 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_cap_release": 0, 2024-04-24T09:18:44.934 INFO:teuthology.orchestra.run.smithi189.stdout: "process_request_cap_release": 0 2024-04-24T09:18:44.935 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:44.935 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_cache": { 2024-04-24T09:18:44.935 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_update": 0, 2024-04-24T09:18:44.935 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_update_receipt": 0, 2024-04-24T09:18:44.935 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_try_discover": 0, 2024-04-24T09:18:44.935 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_send_discover": 0, 2024-04-24T09:18:44.935 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_handle_discover": 0, 2024-04-24T09:18:44.935 INFO:teuthology.orchestra.run.smithi189.stdout: "num_strays": 0, 2024-04-24T09:18:44.935 INFO:teuthology.orchestra.run.smithi189.stdout: "num_strays_delayed": 0, 2024-04-24T09:18:44.935 INFO:teuthology.orchestra.run.smithi189.stdout: "num_strays_enqueuing": 0, 2024-04-24T09:18:44.935 INFO:teuthology.orchestra.run.smithi189.stdout: "strays_created": 0, 2024-04-24T09:18:44.935 INFO:teuthology.orchestra.run.smithi189.stdout: "strays_enqueued": 0, 2024-04-24T09:18:44.935 INFO:teuthology.orchestra.run.smithi189.stdout: "strays_reintegrated": 0, 2024-04-24T09:18:44.935 INFO:teuthology.orchestra.run.smithi189.stdout: "strays_migrated": 0, 2024-04-24T09:18:44.935 INFO:teuthology.orchestra.run.smithi189.stdout: "num_recovering_processing": 0, 2024-04-24T09:18:44.935 INFO:teuthology.orchestra.run.smithi189.stdout: "num_recovering_enqueued": 0, 2024-04-24T09:18:44.935 INFO:teuthology.orchestra.run.smithi189.stdout: "num_recovering_prioritized": 0, 2024-04-24T09:18:44.936 INFO:teuthology.orchestra.run.smithi189.stdout: "recovery_started": 0, 2024-04-24T09:18:44.936 INFO:teuthology.orchestra.run.smithi189.stdout: "recovery_completed": 0, 2024-04-24T09:18:44.936 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_quiesce_path": 0, 2024-04-24T09:18:44.936 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_quiesce_inode": 0, 2024-04-24T09:18:44.936 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_enqueue_scrub": 0, 2024-04-24T09:18:44.936 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_exportdir": 0, 2024-04-24T09:18:44.936 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_flush": 0, 2024-04-24T09:18:44.936 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_fragmentdir": 0, 2024-04-24T09:18:44.936 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_fragstats": 0, 2024-04-24T09:18:44.936 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_inodestats": 0 2024-04-24T09:18:44.936 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:44.936 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_log": { 2024-04-24T09:18:44.936 INFO:teuthology.orchestra.run.smithi189.stdout: "evlrg": 0, 2024-04-24T09:18:44.936 INFO:teuthology.orchestra.run.smithi189.stdout: "evadd": 0, 2024-04-24T09:18:44.936 INFO:teuthology.orchestra.run.smithi189.stdout: "evex": 0, 2024-04-24T09:18:44.936 INFO:teuthology.orchestra.run.smithi189.stdout: "evtrm": 0, 2024-04-24T09:18:44.936 INFO:teuthology.orchestra.run.smithi189.stdout: "ev": 63657, 2024-04-24T09:18:44.937 INFO:teuthology.orchestra.run.smithi189.stdout: "evexg": 0, 2024-04-24T09:18:44.937 INFO:teuthology.orchestra.run.smithi189.stdout: "evexd": 0, 2024-04-24T09:18:44.937 INFO:teuthology.orchestra.run.smithi189.stdout: "segadd": 0, 2024-04-24T09:18:44.937 INFO:teuthology.orchestra.run.smithi189.stdout: "segex": 0, 2024-04-24T09:18:44.937 INFO:teuthology.orchestra.run.smithi189.stdout: "segtrm": 0, 2024-04-24T09:18:44.937 INFO:teuthology.orchestra.run.smithi189.stdout: "seg": 98, 2024-04-24T09:18:44.937 INFO:teuthology.orchestra.run.smithi189.stdout: "segmjr": 7, 2024-04-24T09:18:44.937 INFO:teuthology.orchestra.run.smithi189.stdout: "segexg": 0, 2024-04-24T09:18:44.937 INFO:teuthology.orchestra.run.smithi189.stdout: "segexd": 0, 2024-04-24T09:18:44.937 INFO:teuthology.orchestra.run.smithi189.stdout: "expos": 4194304, 2024-04-24T09:18:44.937 INFO:teuthology.orchestra.run.smithi189.stdout: "wrpos": 336811422, 2024-04-24T09:18:44.937 INFO:teuthology.orchestra.run.smithi189.stdout: "rdpos": 336790034, 2024-04-24T09:18:44.937 INFO:teuthology.orchestra.run.smithi189.stdout: "jlat": { 2024-04-24T09:18:44.937 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:44.937 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:44.937 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:44.938 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:44.938 INFO:teuthology.orchestra.run.smithi189.stdout: "replayed": 63657 2024-04-24T09:18:44.938 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:44.938 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_mem": { 2024-04-24T09:18:44.938 INFO:teuthology.orchestra.run.smithi189.stdout: "ino": 2294, 2024-04-24T09:18:44.938 INFO:teuthology.orchestra.run.smithi189.stdout: "ino+": 11127, 2024-04-24T09:18:44.938 INFO:teuthology.orchestra.run.smithi189.stdout: "ino-": 8833, 2024-04-24T09:18:44.938 INFO:teuthology.orchestra.run.smithi189.stdout: "dir": 96, 2024-04-24T09:18:44.938 INFO:teuthology.orchestra.run.smithi189.stdout: "dir+": 160, 2024-04-24T09:18:44.938 INFO:teuthology.orchestra.run.smithi189.stdout: "dir-": 64, 2024-04-24T09:18:44.938 INFO:teuthology.orchestra.run.smithi189.stdout: "dn": 10739, 2024-04-24T09:18:44.938 INFO:teuthology.orchestra.run.smithi189.stdout: "dn+": 11543, 2024-04-24T09:18:44.938 INFO:teuthology.orchestra.run.smithi189.stdout: "dn-": 804, 2024-04-24T09:18:44.938 INFO:teuthology.orchestra.run.smithi189.stdout: "cap": 0, 2024-04-24T09:18:44.938 INFO:teuthology.orchestra.run.smithi189.stdout: "cap+": 0, 2024-04-24T09:18:44.938 INFO:teuthology.orchestra.run.smithi189.stdout: "cap-": 0, 2024-04-24T09:18:44.939 INFO:teuthology.orchestra.run.smithi189.stdout: "rss": 70832, 2024-04-24T09:18:44.939 INFO:teuthology.orchestra.run.smithi189.stdout: "heap": 215416 2024-04-24T09:18:44.939 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:44.939 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_server": { 2024-04-24T09:18:44.939 INFO:teuthology.orchestra.run.smithi189.stdout: "dispatch_client_request": 0, 2024-04-24T09:18:44.939 INFO:teuthology.orchestra.run.smithi189.stdout: "dispatch_server_request": 0, 2024-04-24T09:18:44.939 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_request": 0, 2024-04-24T09:18:44.939 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_session": 0, 2024-04-24T09:18:44.939 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_peer_request": 0, 2024-04-24T09:18:44.939 INFO:teuthology.orchestra.run.smithi189.stdout: "req_create_latency": { 2024-04-24T09:18:44.939 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:44.939 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:44.939 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:44.939 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:44.939 INFO:teuthology.orchestra.run.smithi189.stdout: "req_getattr_latency": { 2024-04-24T09:18:44.939 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:44.939 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:44.940 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:44.940 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:44.940 INFO:teuthology.orchestra.run.smithi189.stdout: "req_getfilelock_latency": { 2024-04-24T09:18:44.940 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:44.940 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:44.940 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:44.940 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:44.940 INFO:teuthology.orchestra.run.smithi189.stdout: "req_link_latency": { 2024-04-24T09:18:44.940 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:44.940 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:44.940 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:44.940 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:44.940 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookup_latency": { 2024-04-24T09:18:44.940 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:44.940 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:44.940 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:44.940 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:44.941 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookuphash_latency": { 2024-04-24T09:18:44.941 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:44.941 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:44.941 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:44.941 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:44.941 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookupino_latency": { 2024-04-24T09:18:44.941 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:44.941 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:44.941 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:44.941 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:44.941 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookupname_latency": { 2024-04-24T09:18:44.941 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:44.941 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:44.941 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:44.941 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:44.941 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookupparent_latency": { 2024-04-24T09:18:44.941 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:44.942 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:44.942 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:44.942 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:44.942 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookupsnap_latency": { 2024-04-24T09:18:44.942 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:44.942 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:44.942 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:44.942 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:44.942 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lssnap_latency": { 2024-04-24T09:18:44.942 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:44.942 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:44.942 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:44.942 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:44.943 INFO:teuthology.orchestra.run.smithi189.stdout: "req_mkdir_latency": { 2024-04-24T09:18:44.943 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:44.943 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:44.943 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:44.943 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:44.943 INFO:teuthology.orchestra.run.smithi189.stdout: "req_mknod_latency": { 2024-04-24T09:18:44.943 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:44.943 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:44.943 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:44.943 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:44.943 INFO:teuthology.orchestra.run.smithi189.stdout: "req_mksnap_latency": { 2024-04-24T09:18:44.943 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:44.943 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:44.944 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:44.944 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:44.944 INFO:teuthology.orchestra.run.smithi189.stdout: "req_open_latency": { 2024-04-24T09:18:44.944 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:44.944 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:44.944 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:44.944 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:44.944 INFO:teuthology.orchestra.run.smithi189.stdout: "req_readdir_latency": { 2024-04-24T09:18:44.944 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:44.944 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:44.944 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:44.944 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:44.944 INFO:teuthology.orchestra.run.smithi189.stdout: "req_rename_latency": { 2024-04-24T09:18:44.944 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:44.944 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:44.944 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:44.944 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:44.945 INFO:teuthology.orchestra.run.smithi189.stdout: "req_renamesnap_latency": { 2024-04-24T09:18:44.945 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:44.945 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:44.945 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:44.945 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:44.945 INFO:teuthology.orchestra.run.smithi189.stdout: "req_snapdiff_latency": { 2024-04-24T09:18:44.945 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:44.945 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:44.945 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:44.945 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:44.945 INFO:teuthology.orchestra.run.smithi189.stdout: "req_rmdir_latency": { 2024-04-24T09:18:44.945 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:44.945 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:44.945 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:44.945 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:44.945 INFO:teuthology.orchestra.run.smithi189.stdout: "req_rmsnap_latency": { 2024-04-24T09:18:44.945 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:44.946 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:44.946 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:44.946 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:44.946 INFO:teuthology.orchestra.run.smithi189.stdout: "req_rmxattr_latency": { 2024-04-24T09:18:44.946 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:44.946 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:44.946 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:44.946 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:44.946 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setattr_latency": { 2024-04-24T09:18:44.946 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:44.946 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:44.946 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:44.946 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:44.946 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setdirlayout_latency": { 2024-04-24T09:18:44.947 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:44.947 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:44.947 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:44.947 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:44.947 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setfilelock_latency": { 2024-04-24T09:18:44.947 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:44.947 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:44.947 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:44.947 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:44.947 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setlayout_latency": { 2024-04-24T09:18:44.947 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:44.947 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:44.947 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:44.947 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:44.947 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setxattr_latency": { 2024-04-24T09:18:44.947 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:44.948 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:44.948 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:44.948 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:44.948 INFO:teuthology.orchestra.run.smithi189.stdout: "req_symlink_latency": { 2024-04-24T09:18:44.948 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:44.948 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:44.948 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:44.948 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:44.948 INFO:teuthology.orchestra.run.smithi189.stdout: "req_unlink_latency": { 2024-04-24T09:18:44.948 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:44.948 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:44.948 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:44.948 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:44.948 INFO:teuthology.orchestra.run.smithi189.stdout: "cap_revoke_eviction": 0, 2024-04-24T09:18:44.948 INFO:teuthology.orchestra.run.smithi189.stdout: "cap_acquisition_throttle": 0, 2024-04-24T09:18:44.948 INFO:teuthology.orchestra.run.smithi189.stdout: "req_getvxattr_latency": { 2024-04-24T09:18:44.949 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:44.949 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:44.949 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:44.949 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-04-24T09:18:44.949 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:44.949 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_sessions": { 2024-04-24T09:18:44.949 INFO:teuthology.orchestra.run.smithi189.stdout: "session_count": 2, 2024-04-24T09:18:44.949 INFO:teuthology.orchestra.run.smithi189.stdout: "session_add": 4, 2024-04-24T09:18:44.949 INFO:teuthology.orchestra.run.smithi189.stdout: "session_remove": 2, 2024-04-24T09:18:44.949 INFO:teuthology.orchestra.run.smithi189.stdout: "sessions_open": 3, 2024-04-24T09:18:44.949 INFO:teuthology.orchestra.run.smithi189.stdout: "sessions_stale": 0, 2024-04-24T09:18:44.949 INFO:teuthology.orchestra.run.smithi189.stdout: "total_load": 0, 2024-04-24T09:18:44.949 INFO:teuthology.orchestra.run.smithi189.stdout: "average_load": 0, 2024-04-24T09:18:44.949 INFO:teuthology.orchestra.run.smithi189.stdout: "avg_session_uptime": 3709, 2024-04-24T09:18:44.949 INFO:teuthology.orchestra.run.smithi189.stdout: "mdthresh_evicted": 0 2024-04-24T09:18:44.949 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:44.950 INFO:teuthology.orchestra.run.smithi189.stdout: "mempool": { 2024-04-24T09:18:44.950 INFO:teuthology.orchestra.run.smithi189.stdout: "bloom_filter_bytes": 0, 2024-04-24T09:18:44.950 INFO:teuthology.orchestra.run.smithi189.stdout: "bloom_filter_items": 0, 2024-04-24T09:18:44.950 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_alloc_bytes": 0, 2024-04-24T09:18:44.950 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_alloc_items": 0, 2024-04-24T09:18:44.950 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_data_bytes": 0, 2024-04-24T09:18:44.950 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_data_items": 0, 2024-04-24T09:18:44.950 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-24T09:18:44.950 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_onode_items": 0, 2024-04-24T09:18:44.950 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-24T09:18:44.950 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_meta_items": 0, 2024-04-24T09:18:44.950 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_other_bytes": 0, 2024-04-24T09:18:44.950 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_other_items": 0, 2024-04-24T09:18:44.950 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-24T09:18:44.950 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_buffer_items": 0, 2024-04-24T09:18:44.951 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_extent_bytes": 0, 2024-04-24T09:18:44.951 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_extent_items": 0, 2024-04-24T09:18:44.951 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_blob_bytes": 0, 2024-04-24T09:18:44.951 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_blob_items": 0, 2024-04-24T09:18:44.951 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-24T09:18:44.951 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_shared_blob_items": 0, 2024-04-24T09:18:44.951 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-24T09:18:44.951 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_inline_bl_items": 0, 2024-04-24T09:18:44.951 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_fsck_bytes": 0, 2024-04-24T09:18:44.951 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_fsck_items": 0, 2024-04-24T09:18:44.951 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_txc_bytes": 0, 2024-04-24T09:18:44.951 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_txc_items": 0, 2024-04-24T09:18:44.951 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-24T09:18:44.951 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_deferred_items": 0, 2024-04-24T09:18:44.951 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_bytes": 0, 2024-04-24T09:18:44.951 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_items": 0, 2024-04-24T09:18:44.952 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_bytes": 0, 2024-04-24T09:18:44.952 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_items": 0, 2024-04-24T09:18:44.952 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_reader_bytes": 0, 2024-04-24T09:18:44.952 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_reader_items": 0, 2024-04-24T09:18:44.952 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_writer_bytes": 0, 2024-04-24T09:18:44.952 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_writer_items": 0, 2024-04-24T09:18:44.952 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_anon_bytes": 241017, 2024-04-24T09:18:44.952 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_anon_items": 83, 2024-04-24T09:18:44.952 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_meta_bytes": 0, 2024-04-24T09:18:44.952 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_meta_items": 0, 2024-04-24T09:18:44.952 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_bytes": 0, 2024-04-24T09:18:44.952 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_items": 0, 2024-04-24T09:18:44.952 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_mapbl_bytes": 0, 2024-04-24T09:18:44.952 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_mapbl_items": 0, 2024-04-24T09:18:44.952 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_pglog_bytes": 0, 2024-04-24T09:18:44.952 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_pglog_items": 0, 2024-04-24T09:18:44.953 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_bytes": 9800, 2024-04-24T09:18:44.953 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_items": 188, 2024-04-24T09:18:44.953 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_mapping_bytes": 0, 2024-04-24T09:18:44.953 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_mapping_items": 0, 2024-04-24T09:18:44.953 INFO:teuthology.orchestra.run.smithi189.stdout: "pgmap_bytes": 0, 2024-04-24T09:18:44.953 INFO:teuthology.orchestra.run.smithi189.stdout: "pgmap_items": 0, 2024-04-24T09:18:44.953 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_co_bytes": 12910582, 2024-04-24T09:18:44.953 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_co_items": 53993, 2024-04-24T09:18:44.953 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_1_bytes": 0, 2024-04-24T09:18:44.953 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_1_items": 0, 2024-04-24T09:18:44.953 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_2_bytes": 0, 2024-04-24T09:18:44.953 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_2_items": 0 2024-04-24T09:18:44.953 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:44.953 INFO:teuthology.orchestra.run.smithi189.stdout: "objecter": { 2024-04-24T09:18:44.953 INFO:teuthology.orchestra.run.smithi189.stdout: "op_active": 0, 2024-04-24T09:18:44.953 INFO:teuthology.orchestra.run.smithi189.stdout: "op_laggy": 0, 2024-04-24T09:18:44.953 INFO:teuthology.orchestra.run.smithi189.stdout: "op_send": 6493, 2024-04-24T09:18:44.954 INFO:teuthology.orchestra.run.smithi189.stdout: "op_send_bytes": 21, 2024-04-24T09:18:44.954 INFO:teuthology.orchestra.run.smithi189.stdout: "op_resend": 3, 2024-04-24T09:18:44.954 INFO:teuthology.orchestra.run.smithi189.stdout: "op_reply": 6493, 2024-04-24T09:18:44.954 INFO:teuthology.orchestra.run.smithi189.stdout: "op_latency": { 2024-04-24T09:18:44.954 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 6493, 2024-04-24T09:18:44.954 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 11.381778734, 2024-04-24T09:18:44.954 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.001752930 2024-04-24T09:18:44.954 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:44.954 INFO:teuthology.orchestra.run.smithi189.stdout: "op_inflight": 0, 2024-04-24T09:18:44.954 INFO:teuthology.orchestra.run.smithi189.stdout: "oplen_avg": { 2024-04-24T09:18:44.954 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 6493, 2024-04-24T09:18:44.954 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 6494 2024-04-24T09:18:44.954 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:44.954 INFO:teuthology.orchestra.run.smithi189.stdout: "op": 6493, 2024-04-24T09:18:44.954 INFO:teuthology.orchestra.run.smithi189.stdout: "op_r": 6493, 2024-04-24T09:18:44.955 INFO:teuthology.orchestra.run.smithi189.stdout: "op_w": 0, 2024-04-24T09:18:44.955 INFO:teuthology.orchestra.run.smithi189.stdout: "op_rmw": 0, 2024-04-24T09:18:44.955 INFO:teuthology.orchestra.run.smithi189.stdout: "op_pg": 0, 2024-04-24T09:18:44.955 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_stat": 3565, 2024-04-24T09:18:44.955 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_create": 0, 2024-04-24T09:18:44.955 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_read": 2926, 2024-04-24T09:18:44.955 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_write": 0, 2024-04-24T09:18:44.955 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_writefull": 0, 2024-04-24T09:18:44.955 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_writesame": 0, 2024-04-24T09:18:44.955 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_append": 0, 2024-04-24T09:18:44.955 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_zero": 0, 2024-04-24T09:18:44.955 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_truncate": 0, 2024-04-24T09:18:44.955 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_delete": 0, 2024-04-24T09:18:44.955 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_mapext": 0, 2024-04-24T09:18:44.955 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_sparse_read": 0, 2024-04-24T09:18:44.955 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_clonerange": 0, 2024-04-24T09:18:44.955 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_getxattr": 1, 2024-04-24T09:18:44.956 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_setxattr": 0, 2024-04-24T09:18:44.956 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_cmpxattr": 0, 2024-04-24T09:18:44.956 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_rmxattr": 0, 2024-04-24T09:18:44.956 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_resetxattrs": 0, 2024-04-24T09:18:44.956 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_call": 0, 2024-04-24T09:18:44.956 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_watch": 0, 2024-04-24T09:18:44.956 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_notify": 0, 2024-04-24T09:18:44.956 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_src_cmpxattr": 0, 2024-04-24T09:18:44.956 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_pgls": 0, 2024-04-24T09:18:44.956 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_pgls_filter": 0, 2024-04-24T09:18:44.956 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_other": 0, 2024-04-24T09:18:44.956 INFO:teuthology.orchestra.run.smithi189.stdout: "linger_active": 0, 2024-04-24T09:18:44.956 INFO:teuthology.orchestra.run.smithi189.stdout: "linger_send": 0, 2024-04-24T09:18:44.956 INFO:teuthology.orchestra.run.smithi189.stdout: "linger_resend": 0, 2024-04-24T09:18:44.956 INFO:teuthology.orchestra.run.smithi189.stdout: "linger_ping": 0, 2024-04-24T09:18:44.956 INFO:teuthology.orchestra.run.smithi189.stdout: "poolop_active": 0, 2024-04-24T09:18:44.957 INFO:teuthology.orchestra.run.smithi189.stdout: "poolop_send": 0, 2024-04-24T09:18:44.957 INFO:teuthology.orchestra.run.smithi189.stdout: "poolop_resend": 0, 2024-04-24T09:18:44.957 INFO:teuthology.orchestra.run.smithi189.stdout: "poolstat_active": 0, 2024-04-24T09:18:44.957 INFO:teuthology.orchestra.run.smithi189.stdout: "poolstat_send": 0, 2024-04-24T09:18:44.957 INFO:teuthology.orchestra.run.smithi189.stdout: "poolstat_resend": 0, 2024-04-24T09:18:44.957 INFO:teuthology.orchestra.run.smithi189.stdout: "statfs_active": 0, 2024-04-24T09:18:44.957 INFO:teuthology.orchestra.run.smithi189.stdout: "statfs_send": 0, 2024-04-24T09:18:44.957 INFO:teuthology.orchestra.run.smithi189.stdout: "statfs_resend": 0, 2024-04-24T09:18:44.957 INFO:teuthology.orchestra.run.smithi189.stdout: "command_active": 0, 2024-04-24T09:18:44.957 INFO:teuthology.orchestra.run.smithi189.stdout: "command_send": 0, 2024-04-24T09:18:44.957 INFO:teuthology.orchestra.run.smithi189.stdout: "command_resend": 0, 2024-04-24T09:18:44.957 INFO:teuthology.orchestra.run.smithi189.stdout: "map_epoch": 136, 2024-04-24T09:18:44.957 INFO:teuthology.orchestra.run.smithi189.stdout: "map_full": 0, 2024-04-24T09:18:44.957 INFO:teuthology.orchestra.run.smithi189.stdout: "map_inc": 47, 2024-04-24T09:18:44.957 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_sessions": 11, 2024-04-24T09:18:44.957 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_session_open": 11, 2024-04-24T09:18:44.957 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_session_close": 0, 2024-04-24T09:18:44.958 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_laggy": 0, 2024-04-24T09:18:44.958 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_wr": 0, 2024-04-24T09:18:44.958 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_rd": 2, 2024-04-24T09:18:44.958 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_del": 0 2024-04-24T09:18:44.958 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:44.958 INFO:teuthology.orchestra.run.smithi189.stdout: "oft": { 2024-04-24T09:18:44.958 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_total_objs": 0, 2024-04-24T09:18:44.958 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_total_kv_pairs": 0, 2024-04-24T09:18:44.958 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_total_updates": 0, 2024-04-24T09:18:44.958 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_total_removes": 0 2024-04-24T09:18:44.958 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:44.958 INFO:teuthology.orchestra.run.smithi189.stdout: "osd-slow-ops": { 2024-04-24T09:18:44.958 INFO:teuthology.orchestra.run.smithi189.stdout: "slow_ops_count": 0 2024-04-24T09:18:44.958 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:44.958 INFO:teuthology.orchestra.run.smithi189.stdout: "purge_queue": { 2024-04-24T09:18:44.958 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executing_ops": 0, 2024-04-24T09:18:44.959 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executing_ops_high_water": 0, 2024-04-24T09:18:44.959 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executing": 0, 2024-04-24T09:18:44.959 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executing_high_water": 0, 2024-04-24T09:18:44.959 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executed_ops": 0, 2024-04-24T09:18:44.959 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executed": 0, 2024-04-24T09:18:44.959 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_item_in_journal": 0 2024-04-24T09:18:44.959 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:44.959 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-24T09:18:44.959 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-04-24T09:18:44.959 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 104857600, 2024-04-24T09:18:44.959 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-04-24T09:18:44.959 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 7099, 2024-04-24T09:18:44.959 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 335833123, 2024-04-24T09:18:44.959 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-04-24T09:18:44.959 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 7099, 2024-04-24T09:18:44.959 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 0, 2024-04-24T09:18:44.960 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 0, 2024-04-24T09:18:44.960 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 7099, 2024-04-24T09:18:44.960 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 335833123, 2024-04-24T09:18:44.960 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-04-24T09:18:44.960 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:44.960 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:44.960 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:44.960 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-04-24T09:18:44.960 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:44.960 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-objecter_bytes": { 2024-04-24T09:18:44.960 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-04-24T09:18:44.960 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 104857600, 2024-04-24T09:18:44.960 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-04-24T09:18:44.960 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 0, 2024-04-24T09:18:44.960 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 0, 2024-04-24T09:18:44.960 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-04-24T09:18:44.960 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 0, 2024-04-24T09:18:44.961 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 6493, 2024-04-24T09:18:44.961 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 332631063, 2024-04-24T09:18:44.961 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 6493, 2024-04-24T09:18:44.961 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 332631063, 2024-04-24T09:18:44.961 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-04-24T09:18:44.961 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:44.961 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:44.961 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:44.961 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-04-24T09:18:44.961 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:44.961 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-objecter_ops": { 2024-04-24T09:18:44.961 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-04-24T09:18:44.961 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 1024, 2024-04-24T09:18:44.961 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-04-24T09:18:44.961 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 0, 2024-04-24T09:18:44.961 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 0, 2024-04-24T09:18:44.961 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-04-24T09:18:44.962 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 0, 2024-04-24T09:18:44.962 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 6493, 2024-04-24T09:18:44.962 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 6493, 2024-04-24T09:18:44.962 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 6493, 2024-04-24T09:18:44.962 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 6493, 2024-04-24T09:18:44.962 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-04-24T09:18:44.962 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:44.962 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:44.962 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:44.962 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-04-24T09:18:44.962 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:44.962 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-write_buf_throttle": { 2024-04-24T09:18:44.962 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-04-24T09:18:44.962 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 3758096384, 2024-04-24T09:18:44.962 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-04-24T09:18:44.962 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 0, 2024-04-24T09:18:44.962 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 0, 2024-04-24T09:18:44.963 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-04-24T09:18:44.963 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 0, 2024-04-24T09:18:44.963 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 0, 2024-04-24T09:18:44.963 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 0, 2024-04-24T09:18:44.963 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 0, 2024-04-24T09:18:44.963 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 0, 2024-04-24T09:18:44.963 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-04-24T09:18:44.963 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:44.963 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:44.963 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:44.963 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-04-24T09:18:44.963 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:44.963 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-write_buf_throttle-0x557e1d52a780": { 2024-04-24T09:18:44.963 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-04-24T09:18:44.963 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 3758096384, 2024-04-24T09:18:44.963 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-04-24T09:18:44.963 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 0, 2024-04-24T09:18:44.964 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 0, 2024-04-24T09:18:44.964 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-04-24T09:18:44.964 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 0, 2024-04-24T09:18:44.964 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 0, 2024-04-24T09:18:44.964 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 0, 2024-04-24T09:18:44.964 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 0, 2024-04-24T09:18:44.964 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 0, 2024-04-24T09:18:44.964 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-04-24T09:18:44.964 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:44.964 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:44.964 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:44.964 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-04-24T09:18:44.964 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-04-24T09:18:44.964 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-04-24T09:18:45.897 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-04-24T09:18:45.897 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-04-24T09:18:45.897 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=0 2024-04-24T09:18:45.898 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-04-24T09:18:45.898 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-04-24T09:18:45.898 DEBUG:tasks.check_counter:Getting stats from i 2024-04-24T09:18:45.898 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid bd3e039e-0215-11ef-bc93-c7b262605968 -- ceph daemon mds.i perf dump 2024-04-24T09:18:46.350 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T09:18:46.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:18:46 smithi138 ceph-mon[31441]: pgmap v859: 129 pgs: 129 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-24T09:18:46.457 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:18:46 smithi189 ceph-mon[31791]: pgmap v859: 129 pgs: 129 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-24T09:18:46.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:18:46 smithi083 ceph-mon[27907]: pgmap v859: 129 pgs: 129 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-24T09:18:48.398 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:18:48 smithi189 ceph-mon[31791]: pgmap v860: 129 pgs: 129 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-24T09:18:48.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:18:48 smithi083 ceph-mon[27907]: pgmap v860: 129 pgs: 129 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-24T09:18:48.612 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-04-24T09:18:48.613 INFO:teuthology.orchestra.run.smithi189.stdout: "AsyncMessenger::Worker-0": { 2024-04-24T09:18:48.613 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_messages": 6667, 2024-04-24T09:18:48.613 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages": 6485, 2024-04-24T09:18:48.613 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_bytes": 4076669, 2024-04-24T09:18:48.613 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_bytes": 40795471, 2024-04-24T09:18:48.613 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_created_connections": 60, 2024-04-24T09:18:48.613 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_active_connections": 6, 2024-04-24T09:18:48.613 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_total_time": 1.354524247, 2024-04-24T09:18:48.613 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_send_time": 0.490332233, 2024-04-24T09:18:48.613 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_recv_time": 0.486131549, 2024-04-24T09:18:48.613 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_fast_dispatch_time": 0.302461593, 2024-04-24T09:18:48.613 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T09:18:48.613 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 6479, 2024-04-24T09:18:48.614 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.739391057, 2024-04-24T09:18:48.614 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000114121 2024-04-24T09:18:48.614 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:48.614 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_handle_ack_lat": { 2024-04-24T09:18:48.614 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 1338, 2024-04-24T09:18:48.614 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000454606, 2024-04-24T09:18:48.614 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000339 2024-04-24T09:18:48.614 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:48.614 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_encrypted_bytes": 2094896, 2024-04-24T09:18:48.614 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_encrypted_bytes": 339456 2024-04-24T09:18:48.614 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:48.614 INFO:teuthology.orchestra.run.smithi189.stdout: "AsyncMessenger::Worker-1": { 2024-04-24T09:18:48.614 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_messages": 467328, 2024-04-24T09:18:48.615 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages": 449912, 2024-04-24T09:18:48.615 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_bytes": 133552720, 2024-04-24T09:18:48.615 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_bytes": 324689349, 2024-04-24T09:18:48.615 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_created_connections": 16, 2024-04-24T09:18:48.615 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_active_connections": 6, 2024-04-24T09:18:48.615 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_total_time": 77.177679259, 2024-04-24T09:18:48.615 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_send_time": 33.797829544, 2024-04-24T09:18:48.615 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_recv_time": 53.600076254, 2024-04-24T09:18:48.615 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_fast_dispatch_time": 0.453475198, 2024-04-24T09:18:48.615 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T09:18:48.615 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 449912, 2024-04-24T09:18:48.615 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 33.070487091, 2024-04-24T09:18:48.615 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000073504 2024-04-24T09:18:48.615 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:48.615 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_handle_ack_lat": { 2024-04-24T09:18:48.615 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 879782, 2024-04-24T09:18:48.616 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.362989932, 2024-04-24T09:18:48.616 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000412 2024-04-24T09:18:48.616 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:48.616 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-24T09:18:48.616 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_encrypted_bytes": 752624 2024-04-24T09:18:48.616 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:48.616 INFO:teuthology.orchestra.run.smithi189.stdout: "AsyncMessenger::Worker-2": { 2024-04-24T09:18:48.616 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_messages": 36889, 2024-04-24T09:18:48.616 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages": 39145, 2024-04-24T09:18:48.616 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_bytes": 10796796, 2024-04-24T09:18:48.616 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_bytes": 90072863, 2024-04-24T09:18:48.616 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_created_connections": 57, 2024-04-24T09:18:48.616 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_active_connections": 5, 2024-04-24T09:18:48.616 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_total_time": 7.055293042, 2024-04-24T09:18:48.616 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_send_time": 3.057337514, 2024-04-24T09:18:48.617 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_recv_time": 3.995662999, 2024-04-24T09:18:48.617 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_fast_dispatch_time": 0.549682911, 2024-04-24T09:18:48.617 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T09:18:48.617 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 39146, 2024-04-24T09:18:48.617 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 3.403011454, 2024-04-24T09:18:48.617 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000086931 2024-04-24T09:18:48.617 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:48.617 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_handle_ack_lat": { 2024-04-24T09:18:48.617 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 56857, 2024-04-24T09:18:48.617 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.022497713, 2024-04-24T09:18:48.617 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000395 2024-04-24T09:18:48.617 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:48.617 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-04-24T09:18:48.617 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_encrypted_bytes": 448 2024-04-24T09:18:48.617 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:48.617 INFO:teuthology.orchestra.run.smithi189.stdout: "cct": { 2024-04-24T09:18:48.618 INFO:teuthology.orchestra.run.smithi189.stdout: "total_workers": 1, 2024-04-24T09:18:48.618 INFO:teuthology.orchestra.run.smithi189.stdout: "unhealthy_workers": 0 2024-04-24T09:18:48.618 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:48.618 INFO:teuthology.orchestra.run.smithi189.stdout: "finisher-MDSRank": { 2024-04-24T09:18:48.618 INFO:teuthology.orchestra.run.smithi189.stdout: "queue_len": 0, 2024-04-24T09:18:48.618 INFO:teuthology.orchestra.run.smithi189.stdout: "complete_latency": { 2024-04-24T09:18:48.618 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 34317, 2024-04-24T09:18:48.618 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 27.918994644, 2024-04-24T09:18:48.618 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000813561 2024-04-24T09:18:48.618 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-04-24T09:18:48.618 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:48.618 INFO:teuthology.orchestra.run.smithi189.stdout: "finisher-PurgeQueue": { 2024-04-24T09:18:48.618 INFO:teuthology.orchestra.run.smithi189.stdout: "queue_len": 0, 2024-04-24T09:18:48.619 INFO:teuthology.orchestra.run.smithi189.stdout: "complete_latency": { 2024-04-24T09:18:48.619 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 2992, 2024-04-24T09:18:48.619 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.556284582, 2024-04-24T09:18:48.619 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000185923 2024-04-24T09:18:48.619 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-04-24T09:18:48.619 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:48.619 INFO:teuthology.orchestra.run.smithi189.stdout: "mds": { 2024-04-24T09:18:48.619 INFO:teuthology.orchestra.run.smithi189.stdout: "request": 340438, 2024-04-24T09:18:48.619 INFO:teuthology.orchestra.run.smithi189.stdout: "reply": 319046, 2024-04-24T09:18:48.619 INFO:teuthology.orchestra.run.smithi189.stdout: "reply_latency": { 2024-04-24T09:18:48.619 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 319046, 2024-04-24T09:18:48.619 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 335.274503153, 2024-04-24T09:18:48.619 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.001050865 2024-04-24T09:18:48.619 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:48.620 INFO:teuthology.orchestra.run.smithi189.stdout: "slow_reply": 0, 2024-04-24T09:18:48.620 INFO:teuthology.orchestra.run.smithi189.stdout: "forward": 21392, 2024-04-24T09:18:48.620 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_fetch_complete": 0, 2024-04-24T09:18:48.620 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_fetch_keys": 0, 2024-04-24T09:18:48.620 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_commit": 11, 2024-04-24T09:18:48.620 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_split": 0, 2024-04-24T09:18:48.620 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_merge": 0, 2024-04-24T09:18:48.620 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes": 2810, 2024-04-24T09:18:48.620 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_top": 1110, 2024-04-24T09:18:48.620 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_bottom": 1637, 2024-04-24T09:18:48.620 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_pin_tail": 63, 2024-04-24T09:18:48.620 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_pinned": 1163, 2024-04-24T09:18:48.620 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_expired": 1005, 2024-04-24T09:18:48.620 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_with_caps": 0, 2024-04-24T09:18:48.620 INFO:teuthology.orchestra.run.smithi189.stdout: "caps": 0, 2024-04-24T09:18:48.620 INFO:teuthology.orchestra.run.smithi189.stdout: "subtrees": 5, 2024-04-24T09:18:48.621 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse": 655848, 2024-04-24T09:18:48.621 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_hit": 649502, 2024-04-24T09:18:48.621 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_forward": 4323, 2024-04-24T09:18:48.621 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_discover": 179, 2024-04-24T09:18:48.621 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_dir_fetch": 0, 2024-04-24T09:18:48.621 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_remote_ino": 0, 2024-04-24T09:18:48.621 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_lock": 13, 2024-04-24T09:18:48.621 INFO:teuthology.orchestra.run.smithi189.stdout: "load_cent": 0, 2024-04-24T09:18:48.621 INFO:teuthology.orchestra.run.smithi189.stdout: "q": 0, 2024-04-24T09:18:48.621 INFO:teuthology.orchestra.run.smithi189.stdout: "exported": 31, 2024-04-24T09:18:48.621 INFO:teuthology.orchestra.run.smithi189.stdout: "exported_inodes": 744, 2024-04-24T09:18:48.621 INFO:teuthology.orchestra.run.smithi189.stdout: "imported": 73, 2024-04-24T09:18:48.621 INFO:teuthology.orchestra.run.smithi189.stdout: "imported_inodes": 2952, 2024-04-24T09:18:48.621 INFO:teuthology.orchestra.run.smithi189.stdout: "openino_dir_fetch": 0, 2024-04-24T09:18:48.621 INFO:teuthology.orchestra.run.smithi189.stdout: "openino_backtrace_fetch": 1411, 2024-04-24T09:18:48.621 INFO:teuthology.orchestra.run.smithi189.stdout: "openino_peer_discover": 0, 2024-04-24T09:18:48.621 INFO:teuthology.orchestra.run.smithi189.stdout: "root_rfiles": 4, 2024-04-24T09:18:48.622 INFO:teuthology.orchestra.run.smithi189.stdout: "root_rbytes": 234, 2024-04-24T09:18:48.622 INFO:teuthology.orchestra.run.smithi189.stdout: "root_rsnaps": 7, 2024-04-24T09:18:48.622 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_backtrace_fetch": 1411, 2024-04-24T09:18:48.622 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_set_tag": 0, 2024-04-24T09:18:48.622 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_backtrace_repaired": 0, 2024-04-24T09:18:48.622 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_inotable_repaired": 0, 2024-04-24T09:18:48.622 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_dir_inodes": 610, 2024-04-24T09:18:48.622 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_dir_base_inodes": 0, 2024-04-24T09:18:48.622 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_dirfrag_rstats": 610, 2024-04-24T09:18:48.622 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_file_inodes": 801, 2024-04-24T09:18:48.622 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_inode_file_caps": 2, 2024-04-24T09:18:48.622 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_revoke": 12205, 2024-04-24T09:18:48.622 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_grant": 3890, 2024-04-24T09:18:48.622 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_trunc": 0, 2024-04-24T09:18:48.622 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-24T09:18:48.623 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-24T09:18:48.623 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_caps": 14308, 2024-04-24T09:18:48.623 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_caps_dirty": 1503, 2024-04-24T09:18:48.623 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_cap_release": 199, 2024-04-24T09:18:48.623 INFO:teuthology.orchestra.run.smithi189.stdout: "process_request_cap_release": 3942 2024-04-24T09:18:48.623 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:48.623 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_cache": { 2024-04-24T09:18:48.623 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_update": 4, 2024-04-24T09:18:48.623 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_update_receipt": 180, 2024-04-24T09:18:48.623 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_try_discover": 139, 2024-04-24T09:18:48.623 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_send_discover": 253, 2024-04-24T09:18:48.623 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_handle_discover": 76, 2024-04-24T09:18:48.623 INFO:teuthology.orchestra.run.smithi189.stdout: "num_strays": 30, 2024-04-24T09:18:48.623 INFO:teuthology.orchestra.run.smithi189.stdout: "num_strays_delayed": 0, 2024-04-24T09:18:48.623 INFO:teuthology.orchestra.run.smithi189.stdout: "num_strays_enqueuing": 0, 2024-04-24T09:18:48.624 INFO:teuthology.orchestra.run.smithi189.stdout: "strays_created": 1099, 2024-04-24T09:18:48.624 INFO:teuthology.orchestra.run.smithi189.stdout: "strays_enqueued": 1069, 2024-04-24T09:18:48.624 INFO:teuthology.orchestra.run.smithi189.stdout: "strays_reintegrated": 0, 2024-04-24T09:18:48.624 INFO:teuthology.orchestra.run.smithi189.stdout: "strays_migrated": 0, 2024-04-24T09:18:48.624 INFO:teuthology.orchestra.run.smithi189.stdout: "num_recovering_processing": 0, 2024-04-24T09:18:48.624 INFO:teuthology.orchestra.run.smithi189.stdout: "num_recovering_enqueued": 0, 2024-04-24T09:18:48.624 INFO:teuthology.orchestra.run.smithi189.stdout: "num_recovering_prioritized": 0, 2024-04-24T09:18:48.624 INFO:teuthology.orchestra.run.smithi189.stdout: "recovery_started": 0, 2024-04-24T09:18:48.624 INFO:teuthology.orchestra.run.smithi189.stdout: "recovery_completed": 0, 2024-04-24T09:18:48.624 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_quiesce_path": 0, 2024-04-24T09:18:48.624 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_quiesce_inode": 0, 2024-04-24T09:18:48.624 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_enqueue_scrub": 0, 2024-04-24T09:18:48.624 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_exportdir": 32, 2024-04-24T09:18:48.624 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_flush": 0, 2024-04-24T09:18:48.625 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_fragmentdir": 0, 2024-04-24T09:18:48.625 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_fragstats": 0, 2024-04-24T09:18:48.625 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_inodestats": 0 2024-04-24T09:18:48.625 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:48.625 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_log": { 2024-04-24T09:18:48.625 INFO:teuthology.orchestra.run.smithi189.stdout: "evlrg": 0, 2024-04-24T09:18:48.625 INFO:teuthology.orchestra.run.smithi189.stdout: "evadd": 23382, 2024-04-24T09:18:48.625 INFO:teuthology.orchestra.run.smithi189.stdout: "evex": 0, 2024-04-24T09:18:48.625 INFO:teuthology.orchestra.run.smithi189.stdout: "evtrm": 0, 2024-04-24T09:18:48.625 INFO:teuthology.orchestra.run.smithi189.stdout: "ev": 23382, 2024-04-24T09:18:48.625 INFO:teuthology.orchestra.run.smithi189.stdout: "evexg": 0, 2024-04-24T09:18:48.625 INFO:teuthology.orchestra.run.smithi189.stdout: "evexd": 0, 2024-04-24T09:18:48.625 INFO:teuthology.orchestra.run.smithi189.stdout: "segadd": 50, 2024-04-24T09:18:48.625 INFO:teuthology.orchestra.run.smithi189.stdout: "segex": 0, 2024-04-24T09:18:48.625 INFO:teuthology.orchestra.run.smithi189.stdout: "segtrm": 0, 2024-04-24T09:18:48.625 INFO:teuthology.orchestra.run.smithi189.stdout: "seg": 50, 2024-04-24T09:18:48.626 INFO:teuthology.orchestra.run.smithi189.stdout: "segmjr": 3, 2024-04-24T09:18:48.626 INFO:teuthology.orchestra.run.smithi189.stdout: "segexg": 0, 2024-04-24T09:18:48.626 INFO:teuthology.orchestra.run.smithi189.stdout: "segexd": 0, 2024-04-24T09:18:48.626 INFO:teuthology.orchestra.run.smithi189.stdout: "expos": 4194304, 2024-04-24T09:18:48.626 INFO:teuthology.orchestra.run.smithi189.stdout: "wrpos": 192759999, 2024-04-24T09:18:48.626 INFO:teuthology.orchestra.run.smithi189.stdout: "rdpos": 0, 2024-04-24T09:18:48.626 INFO:teuthology.orchestra.run.smithi189.stdout: "jlat": { 2024-04-24T09:18:48.626 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 17915, 2024-04-24T09:18:48.626 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 152.176148625, 2024-04-24T09:18:48.626 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.008494342 2024-04-24T09:18:48.626 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:48.626 INFO:teuthology.orchestra.run.smithi189.stdout: "replayed": 0 2024-04-24T09:18:48.626 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:48.626 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_mem": { 2024-04-24T09:18:48.626 INFO:teuthology.orchestra.run.smithi189.stdout: "ino": 2411, 2024-04-24T09:18:48.626 INFO:teuthology.orchestra.run.smithi189.stdout: "ino+": 4174, 2024-04-24T09:18:48.627 INFO:teuthology.orchestra.run.smithi189.stdout: "ino-": 1763, 2024-04-24T09:18:48.627 INFO:teuthology.orchestra.run.smithi189.stdout: "dir": 97, 2024-04-24T09:18:48.627 INFO:teuthology.orchestra.run.smithi189.stdout: "dir+": 198, 2024-04-24T09:18:48.627 INFO:teuthology.orchestra.run.smithi189.stdout: "dir-": 101, 2024-04-24T09:18:48.627 INFO:teuthology.orchestra.run.smithi189.stdout: "dn": 2810, 2024-04-24T09:18:48.627 INFO:teuthology.orchestra.run.smithi189.stdout: "dn+": 5071, 2024-04-24T09:18:48.627 INFO:teuthology.orchestra.run.smithi189.stdout: "dn-": 2261, 2024-04-24T09:18:48.627 INFO:teuthology.orchestra.run.smithi189.stdout: "cap": 0, 2024-04-24T09:18:48.627 INFO:teuthology.orchestra.run.smithi189.stdout: "cap+": 1348, 2024-04-24T09:18:48.627 INFO:teuthology.orchestra.run.smithi189.stdout: "cap-": 1348, 2024-04-24T09:18:48.627 INFO:teuthology.orchestra.run.smithi189.stdout: "rss": 73432, 2024-04-24T09:18:48.627 INFO:teuthology.orchestra.run.smithi189.stdout: "heap": 223608 2024-04-24T09:18:48.627 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:48.627 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_server": { 2024-04-24T09:18:48.627 INFO:teuthology.orchestra.run.smithi189.stdout: "dispatch_client_request": 347637, 2024-04-24T09:18:48.628 INFO:teuthology.orchestra.run.smithi189.stdout: "dispatch_server_request": 25283, 2024-04-24T09:18:48.628 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_request": 340438, 2024-04-24T09:18:48.628 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_session": 162, 2024-04-24T09:18:48.628 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_peer_request": 50250, 2024-04-24T09:18:48.628 INFO:teuthology.orchestra.run.smithi189.stdout: "req_create_latency": { 2024-04-24T09:18:48.628 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 1010, 2024-04-24T09:18:48.628 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 3.337954835, 2024-04-24T09:18:48.628 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.003304905 2024-04-24T09:18:48.628 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:48.628 INFO:teuthology.orchestra.run.smithi189.stdout: "req_getattr_latency": { 2024-04-24T09:18:48.628 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 308996, 2024-04-24T09:18:48.628 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 242.142692879, 2024-04-24T09:18:48.628 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000783643 2024-04-24T09:18:48.628 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:48.628 INFO:teuthology.orchestra.run.smithi189.stdout: "req_getfilelock_latency": { 2024-04-24T09:18:48.629 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:48.629 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:48.629 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:48.629 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:48.629 INFO:teuthology.orchestra.run.smithi189.stdout: "req_link_latency": { 2024-04-24T09:18:48.629 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:48.629 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:48.629 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:48.629 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:48.629 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookup_latency": { 2024-04-24T09:18:48.629 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 4124, 2024-04-24T09:18:48.629 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 65.311086682, 2024-04-24T09:18:48.629 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.015836829 2024-04-24T09:18:48.629 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:48.629 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookuphash_latency": { 2024-04-24T09:18:48.629 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:48.629 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:48.630 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:48.630 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:48.630 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookupino_latency": { 2024-04-24T09:18:48.630 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:48.630 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:48.630 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:48.630 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:48.630 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookupname_latency": { 2024-04-24T09:18:48.630 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:48.630 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:48.630 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:48.630 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:48.630 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookupparent_latency": { 2024-04-24T09:18:48.630 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:48.630 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:48.630 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:48.631 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:48.631 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookupsnap_latency": { 2024-04-24T09:18:48.631 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:48.631 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:48.631 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:48.631 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:48.631 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lssnap_latency": { 2024-04-24T09:18:48.631 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:48.631 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:48.632 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:48.632 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:48.632 INFO:teuthology.orchestra.run.smithi189.stdout: "req_mkdir_latency": { 2024-04-24T09:18:48.632 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:48.632 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:48.632 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:48.632 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:48.632 INFO:teuthology.orchestra.run.smithi189.stdout: "req_mknod_latency": { 2024-04-24T09:18:48.632 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:48.632 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:48.632 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:48.632 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:48.632 INFO:teuthology.orchestra.run.smithi189.stdout: "req_mksnap_latency": { 2024-04-24T09:18:48.632 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:48.632 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:48.632 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:48.632 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:48.633 INFO:teuthology.orchestra.run.smithi189.stdout: "req_open_latency": { 2024-04-24T09:18:48.633 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 758, 2024-04-24T09:18:48.633 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 1.281765798, 2024-04-24T09:18:48.633 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.001690983 2024-04-24T09:18:48.633 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:48.633 INFO:teuthology.orchestra.run.smithi189.stdout: "req_readdir_latency": { 2024-04-24T09:18:48.633 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 2792, 2024-04-24T09:18:48.633 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 11.080824757, 2024-04-24T09:18:48.633 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.003968776 2024-04-24T09:18:48.633 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:48.633 INFO:teuthology.orchestra.run.smithi189.stdout: "req_rename_latency": { 2024-04-24T09:18:48.633 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 243, 2024-04-24T09:18:48.633 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 4.080156373, 2024-04-24T09:18:48.633 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.016790766 2024-04-24T09:18:48.633 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:48.634 INFO:teuthology.orchestra.run.smithi189.stdout: "req_renamesnap_latency": { 2024-04-24T09:18:48.634 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:48.634 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:48.634 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:48.634 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:48.634 INFO:teuthology.orchestra.run.smithi189.stdout: "req_snapdiff_latency": { 2024-04-24T09:18:48.634 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:48.634 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:48.634 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:48.634 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:48.634 INFO:teuthology.orchestra.run.smithi189.stdout: "req_rmdir_latency": { 2024-04-24T09:18:48.634 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 30, 2024-04-24T09:18:48.634 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.664305320, 2024-04-24T09:18:48.634 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.022143510 2024-04-24T09:18:48.634 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:48.635 INFO:teuthology.orchestra.run.smithi189.stdout: "req_rmsnap_latency": { 2024-04-24T09:18:48.635 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:48.635 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:48.635 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:48.635 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:48.635 INFO:teuthology.orchestra.run.smithi189.stdout: "req_rmxattr_latency": { 2024-04-24T09:18:48.635 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:48.635 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:48.635 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:48.635 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:48.635 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setattr_latency": { 2024-04-24T09:18:48.635 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:48.635 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:48.635 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:48.635 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:48.635 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setdirlayout_latency": { 2024-04-24T09:18:48.636 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:48.636 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:48.636 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:48.636 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:48.636 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setfilelock_latency": { 2024-04-24T09:18:48.636 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 24, 2024-04-24T09:18:48.636 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.037502384, 2024-04-24T09:18:48.636 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.001562599 2024-04-24T09:18:48.636 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:48.636 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setlayout_latency": { 2024-04-24T09:18:48.636 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:48.636 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:48.636 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:48.636 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:48.636 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setxattr_latency": { 2024-04-24T09:18:48.637 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:48.637 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:48.637 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:48.637 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:48.637 INFO:teuthology.orchestra.run.smithi189.stdout: "req_symlink_latency": { 2024-04-24T09:18:48.637 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:48.637 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:48.637 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:48.637 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:48.637 INFO:teuthology.orchestra.run.smithi189.stdout: "req_unlink_latency": { 2024-04-24T09:18:48.637 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 1069, 2024-04-24T09:18:48.637 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 7.338214125, 2024-04-24T09:18:48.637 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.006864559 2024-04-24T09:18:48.637 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:48.637 INFO:teuthology.orchestra.run.smithi189.stdout: "cap_revoke_eviction": 0, 2024-04-24T09:18:48.637 INFO:teuthology.orchestra.run.smithi189.stdout: "cap_acquisition_throttle": 0, 2024-04-24T09:18:48.637 INFO:teuthology.orchestra.run.smithi189.stdout: "req_getvxattr_latency": { 2024-04-24T09:18:48.638 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:48.638 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:48.638 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:48.638 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-04-24T09:18:48.638 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:48.638 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_sessions": { 2024-04-24T09:18:48.638 INFO:teuthology.orchestra.run.smithi189.stdout: "session_count": 1, 2024-04-24T09:18:48.638 INFO:teuthology.orchestra.run.smithi189.stdout: "session_add": 2, 2024-04-24T09:18:48.638 INFO:teuthology.orchestra.run.smithi189.stdout: "session_remove": 1, 2024-04-24T09:18:48.638 INFO:teuthology.orchestra.run.smithi189.stdout: "sessions_open": 1, 2024-04-24T09:18:48.638 INFO:teuthology.orchestra.run.smithi189.stdout: "sessions_stale": 0, 2024-04-24T09:18:48.638 INFO:teuthology.orchestra.run.smithi189.stdout: "total_load": 35217, 2024-04-24T09:18:48.638 INFO:teuthology.orchestra.run.smithi189.stdout: "average_load": 35217, 2024-04-24T09:18:48.638 INFO:teuthology.orchestra.run.smithi189.stdout: "avg_session_uptime": 1630, 2024-04-24T09:18:48.639 INFO:teuthology.orchestra.run.smithi189.stdout: "mdthresh_evicted": 0 2024-04-24T09:18:48.639 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:48.639 INFO:teuthology.orchestra.run.smithi189.stdout: "mempool": { 2024-04-24T09:18:48.639 INFO:teuthology.orchestra.run.smithi189.stdout: "bloom_filter_bytes": 0, 2024-04-24T09:18:48.639 INFO:teuthology.orchestra.run.smithi189.stdout: "bloom_filter_items": 0, 2024-04-24T09:18:48.639 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_alloc_bytes": 0, 2024-04-24T09:18:48.639 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_alloc_items": 0, 2024-04-24T09:18:48.639 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_data_bytes": 0, 2024-04-24T09:18:48.639 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_data_items": 0, 2024-04-24T09:18:48.639 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-24T09:18:48.639 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_onode_items": 0, 2024-04-24T09:18:48.639 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-24T09:18:48.639 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_meta_items": 0, 2024-04-24T09:18:48.639 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_other_bytes": 0, 2024-04-24T09:18:48.639 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_other_items": 0, 2024-04-24T09:18:48.640 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-24T09:18:48.640 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_buffer_items": 0, 2024-04-24T09:18:48.640 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_extent_bytes": 0, 2024-04-24T09:18:48.640 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_extent_items": 0, 2024-04-24T09:18:48.640 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_blob_bytes": 0, 2024-04-24T09:18:48.640 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_blob_items": 0, 2024-04-24T09:18:48.640 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-24T09:18:48.640 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_shared_blob_items": 0, 2024-04-24T09:18:48.640 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-24T09:18:48.640 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_inline_bl_items": 0, 2024-04-24T09:18:48.640 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_fsck_bytes": 0, 2024-04-24T09:18:48.640 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_fsck_items": 0, 2024-04-24T09:18:48.640 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_txc_bytes": 0, 2024-04-24T09:18:48.640 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_txc_items": 0, 2024-04-24T09:18:48.640 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-24T09:18:48.640 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_deferred_items": 0, 2024-04-24T09:18:48.641 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_bytes": 0, 2024-04-24T09:18:48.641 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_items": 0, 2024-04-24T09:18:48.641 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_bytes": 0, 2024-04-24T09:18:48.641 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_items": 0, 2024-04-24T09:18:48.641 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_reader_bytes": 0, 2024-04-24T09:18:48.641 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_reader_items": 0, 2024-04-24T09:18:48.641 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_writer_bytes": 0, 2024-04-24T09:18:48.641 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_writer_items": 0, 2024-04-24T09:18:48.641 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_anon_bytes": 538005, 2024-04-24T09:18:48.641 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_anon_items": 168, 2024-04-24T09:18:48.641 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_meta_bytes": 0, 2024-04-24T09:18:48.641 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_meta_items": 0, 2024-04-24T09:18:48.641 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_bytes": 0, 2024-04-24T09:18:48.641 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_items": 0, 2024-04-24T09:18:48.641 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_mapbl_bytes": 0, 2024-04-24T09:18:48.642 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_mapbl_items": 0, 2024-04-24T09:18:48.642 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_pglog_bytes": 0, 2024-04-24T09:18:48.642 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_pglog_items": 0, 2024-04-24T09:18:48.642 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_bytes": 9800, 2024-04-24T09:18:48.642 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_items": 188, 2024-04-24T09:18:48.642 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_mapping_bytes": 0, 2024-04-24T09:18:48.642 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_mapping_items": 0, 2024-04-24T09:18:48.642 INFO:teuthology.orchestra.run.smithi189.stdout: "pgmap_bytes": 0, 2024-04-24T09:18:48.642 INFO:teuthology.orchestra.run.smithi189.stdout: "pgmap_items": 0, 2024-04-24T09:18:48.642 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_co_bytes": 7455870, 2024-04-24T09:18:48.642 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_co_items": 28323, 2024-04-24T09:18:48.642 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_1_bytes": 0, 2024-04-24T09:18:48.642 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_1_items": 0, 2024-04-24T09:18:48.642 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_2_bytes": 0, 2024-04-24T09:18:48.642 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_2_items": 0 2024-04-24T09:18:48.642 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:48.643 INFO:teuthology.orchestra.run.smithi189.stdout: "objecter": { 2024-04-24T09:18:48.643 INFO:teuthology.orchestra.run.smithi189.stdout: "op_active": 0, 2024-04-24T09:18:48.643 INFO:teuthology.orchestra.run.smithi189.stdout: "op_laggy": 0, 2024-04-24T09:18:48.643 INFO:teuthology.orchestra.run.smithi189.stdout: "op_send": 22735, 2024-04-24T09:18:48.643 INFO:teuthology.orchestra.run.smithi189.stdout: "op_send_bytes": 188831495, 2024-04-24T09:18:48.643 INFO:teuthology.orchestra.run.smithi189.stdout: "op_resend": 33, 2024-04-24T09:18:48.643 INFO:teuthology.orchestra.run.smithi189.stdout: "op_reply": 22735, 2024-04-24T09:18:48.643 INFO:teuthology.orchestra.run.smithi189.stdout: "op_latency": { 2024-04-24T09:18:48.643 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 22735, 2024-04-24T09:18:48.643 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 175.048613969, 2024-04-24T09:18:48.643 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.007699521 2024-04-24T09:18:48.643 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:48.643 INFO:teuthology.orchestra.run.smithi189.stdout: "op_inflight": 0, 2024-04-24T09:18:48.643 INFO:teuthology.orchestra.run.smithi189.stdout: "oplen_avg": { 2024-04-24T09:18:48.643 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 22735, 2024-04-24T09:18:48.644 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 23153 2024-04-24T09:18:48.644 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:48.644 INFO:teuthology.orchestra.run.smithi189.stdout: "op": 22735, 2024-04-24T09:18:48.644 INFO:teuthology.orchestra.run.smithi189.stdout: "op_r": 1747, 2024-04-24T09:18:48.644 INFO:teuthology.orchestra.run.smithi189.stdout: "op_w": 20988, 2024-04-24T09:18:48.644 INFO:teuthology.orchestra.run.smithi189.stdout: "op_rmw": 0, 2024-04-24T09:18:48.644 INFO:teuthology.orchestra.run.smithi189.stdout: "op_pg": 0, 2024-04-24T09:18:48.644 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_stat": 0, 2024-04-24T09:18:48.644 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_create": 10, 2024-04-24T09:18:48.644 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_read": 336, 2024-04-24T09:18:48.644 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_write": 18251, 2024-04-24T09:18:48.644 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_writefull": 243, 2024-04-24T09:18:48.644 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_writesame": 0, 2024-04-24T09:18:48.644 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_append": 0, 2024-04-24T09:18:48.644 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_zero": 0, 2024-04-24T09:18:48.645 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_truncate": 0, 2024-04-24T09:18:48.645 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_delete": 2194, 2024-04-24T09:18:48.645 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_mapext": 0, 2024-04-24T09:18:48.645 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_sparse_read": 0, 2024-04-24T09:18:48.645 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_clonerange": 0, 2024-04-24T09:18:48.645 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_getxattr": 1411, 2024-04-24T09:18:48.645 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_setxattr": 20, 2024-04-24T09:18:48.645 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_cmpxattr": 0, 2024-04-24T09:18:48.645 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_rmxattr": 0, 2024-04-24T09:18:48.645 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_resetxattrs": 0, 2024-04-24T09:18:48.645 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_call": 0, 2024-04-24T09:18:48.645 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_watch": 0, 2024-04-24T09:18:48.645 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_notify": 0, 2024-04-24T09:18:48.645 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_src_cmpxattr": 0, 2024-04-24T09:18:48.645 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_pgls": 0, 2024-04-24T09:18:48.646 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_pgls_filter": 0, 2024-04-24T09:18:48.646 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_other": 0, 2024-04-24T09:18:48.646 INFO:teuthology.orchestra.run.smithi189.stdout: "linger_active": 0, 2024-04-24T09:18:48.646 INFO:teuthology.orchestra.run.smithi189.stdout: "linger_send": 0, 2024-04-24T09:18:48.646 INFO:teuthology.orchestra.run.smithi189.stdout: "linger_resend": 0, 2024-04-24T09:18:48.646 INFO:teuthology.orchestra.run.smithi189.stdout: "linger_ping": 0, 2024-04-24T09:18:48.646 INFO:teuthology.orchestra.run.smithi189.stdout: "poolop_active": 0, 2024-04-24T09:18:48.646 INFO:teuthology.orchestra.run.smithi189.stdout: "poolop_send": 0, 2024-04-24T09:18:48.646 INFO:teuthology.orchestra.run.smithi189.stdout: "poolop_resend": 0, 2024-04-24T09:18:48.646 INFO:teuthology.orchestra.run.smithi189.stdout: "poolstat_active": 0, 2024-04-24T09:18:48.646 INFO:teuthology.orchestra.run.smithi189.stdout: "poolstat_send": 0, 2024-04-24T09:18:48.646 INFO:teuthology.orchestra.run.smithi189.stdout: "poolstat_resend": 0, 2024-04-24T09:18:48.646 INFO:teuthology.orchestra.run.smithi189.stdout: "statfs_active": 0, 2024-04-24T09:18:48.646 INFO:teuthology.orchestra.run.smithi189.stdout: "statfs_send": 0, 2024-04-24T09:18:48.646 INFO:teuthology.orchestra.run.smithi189.stdout: "statfs_resend": 0, 2024-04-24T09:18:48.647 INFO:teuthology.orchestra.run.smithi189.stdout: "command_active": 0, 2024-04-24T09:18:48.647 INFO:teuthology.orchestra.run.smithi189.stdout: "command_send": 0, 2024-04-24T09:18:48.647 INFO:teuthology.orchestra.run.smithi189.stdout: "command_resend": 0, 2024-04-24T09:18:48.647 INFO:teuthology.orchestra.run.smithi189.stdout: "map_epoch": 136, 2024-04-24T09:18:48.647 INFO:teuthology.orchestra.run.smithi189.stdout: "map_full": 0, 2024-04-24T09:18:48.647 INFO:teuthology.orchestra.run.smithi189.stdout: "map_inc": 47, 2024-04-24T09:18:48.647 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_sessions": 12, 2024-04-24T09:18:48.647 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_session_open": 16, 2024-04-24T09:18:48.647 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_session_close": 4, 2024-04-24T09:18:48.647 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_laggy": 0, 2024-04-24T09:18:48.647 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_wr": 492, 2024-04-24T09:18:48.647 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_rd": 0, 2024-04-24T09:18:48.647 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_del": 196 2024-04-24T09:18:48.647 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:48.647 INFO:teuthology.orchestra.run.smithi189.stdout: "oft": { 2024-04-24T09:18:48.648 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_total_objs": 1, 2024-04-24T09:18:48.648 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_total_kv_pairs": 0, 2024-04-24T09:18:48.648 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_total_updates": 0, 2024-04-24T09:18:48.648 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_total_removes": 0 2024-04-24T09:18:48.648 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:48.648 INFO:teuthology.orchestra.run.smithi189.stdout: "osd-slow-ops": { 2024-04-24T09:18:48.648 INFO:teuthology.orchestra.run.smithi189.stdout: "slow_ops_count": 0 2024-04-24T09:18:48.648 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:48.648 INFO:teuthology.orchestra.run.smithi189.stdout: "purge_queue": { 2024-04-24T09:18:48.648 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executing_ops": 0, 2024-04-24T09:18:48.648 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executing_ops_high_water": 10, 2024-04-24T09:18:48.648 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executing": 0, 2024-04-24T09:18:48.648 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executing_high_water": 5, 2024-04-24T09:18:48.648 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executed_ops": 2138, 2024-04-24T09:18:48.648 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executed": 1069, 2024-04-24T09:18:48.648 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_item_in_journal": 0 2024-04-24T09:18:48.649 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:48.649 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-24T09:18:48.649 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-04-24T09:18:48.649 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 104857600, 2024-04-24T09:18:48.649 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-04-24T09:18:48.649 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 510880, 2024-04-24T09:18:48.649 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 102409556, 2024-04-24T09:18:48.649 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-04-24T09:18:48.649 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 510880, 2024-04-24T09:18:48.649 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 0, 2024-04-24T09:18:48.649 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 0, 2024-04-24T09:18:48.649 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 510880, 2024-04-24T09:18:48.649 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 102409556, 2024-04-24T09:18:48.649 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-04-24T09:18:48.649 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:48.649 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:48.650 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:48.650 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-04-24T09:18:48.650 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:48.650 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-objecter_bytes": { 2024-04-24T09:18:48.650 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-04-24T09:18:48.650 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 104857600, 2024-04-24T09:18:48.650 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-04-24T09:18:48.650 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 0, 2024-04-24T09:18:48.650 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 0, 2024-04-24T09:18:48.650 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-04-24T09:18:48.650 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 0, 2024-04-24T09:18:48.650 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 22735, 2024-04-24T09:18:48.650 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 188951616, 2024-04-24T09:18:48.650 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 22735, 2024-04-24T09:18:48.650 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 188951616, 2024-04-24T09:18:48.651 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-04-24T09:18:48.651 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:48.651 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:48.651 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:48.651 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-04-24T09:18:48.651 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:48.651 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-objecter_ops": { 2024-04-24T09:18:48.651 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-04-24T09:18:48.651 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 1024, 2024-04-24T09:18:48.651 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-04-24T09:18:48.651 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 0, 2024-04-24T09:18:48.651 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 0, 2024-04-24T09:18:48.651 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-04-24T09:18:48.651 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 0, 2024-04-24T09:18:48.651 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 22735, 2024-04-24T09:18:48.651 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 22735, 2024-04-24T09:18:48.652 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 22735, 2024-04-24T09:18:48.652 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 22735, 2024-04-24T09:18:48.652 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-04-24T09:18:48.652 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:48.652 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:48.652 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:48.652 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-04-24T09:18:48.652 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:48.652 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-write_buf_throttle": { 2024-04-24T09:18:48.652 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-04-24T09:18:48.652 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 3758096384, 2024-04-24T09:18:48.652 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-04-24T09:18:48.652 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 1069, 2024-04-24T09:18:48.652 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 120121, 2024-04-24T09:18:48.652 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-04-24T09:18:48.652 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 1069, 2024-04-24T09:18:48.653 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 0, 2024-04-24T09:18:48.653 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 0, 2024-04-24T09:18:48.653 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 336, 2024-04-24T09:18:48.653 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 120121, 2024-04-24T09:18:48.653 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-04-24T09:18:48.653 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:48.653 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:48.653 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:48.653 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-04-24T09:18:48.653 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:48.653 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-write_buf_throttle-0x5640d33f8500": { 2024-04-24T09:18:48.653 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-04-24T09:18:48.653 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 3758096384, 2024-04-24T09:18:48.653 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-04-24T09:18:48.653 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 23382, 2024-04-24T09:18:48.653 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 188565695, 2024-04-24T09:18:48.654 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-04-24T09:18:48.654 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 23382, 2024-04-24T09:18:48.654 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 0, 2024-04-24T09:18:48.654 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 0, 2024-04-24T09:18:48.654 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 17915, 2024-04-24T09:18:48.654 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 188565695, 2024-04-24T09:18:48.654 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-04-24T09:18:48.654 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:48.654 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:48.654 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:48.654 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-04-24T09:18:48.654 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-04-24T09:18:48.654 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-04-24T09:18:48.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:18:48 smithi138 ceph-mon[31441]: pgmap v860: 129 pgs: 129 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-24T09:18:49.674 INFO:tasks.check_counter:Daemon mds.i mds.exported=31 2024-04-24T09:18:49.674 INFO:tasks.check_counter:Daemon mds.i mds.imported=73 2024-04-24T09:18:49.674 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=7 2024-04-24T09:18:49.674 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-04-24T09:18:49.674 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-04-24T09:18:49.674 DEBUG:tasks.check_counter:Getting stats from l 2024-04-24T09:18:49.674 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0 shell --fsid bd3e039e-0215-11ef-bc93-c7b262605968 -- ceph daemon mds.l perf dump 2024-04-24T09:18:50.114 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T09:18:50.501 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:18:50 smithi189 ceph-mon[31791]: pgmap v861: 129 pgs: 129 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-24T09:18:50.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:18:50 smithi083 ceph-mon[27907]: pgmap v861: 129 pgs: 129 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-24T09:18:50.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:18:50 smithi138 ceph-mon[31441]: pgmap v861: 129 pgs: 129 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-24T09:18:52.322 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-04-24T09:18:52.323 INFO:teuthology.orchestra.run.smithi189.stdout: "AsyncMessenger::Worker-0": { 2024-04-24T09:18:52.323 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_messages": 1870, 2024-04-24T09:18:52.323 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages": 1870, 2024-04-24T09:18:52.323 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_bytes": 109535436, 2024-04-24T09:18:52.323 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_bytes": 593900, 2024-04-24T09:18:52.323 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_created_connections": 9, 2024-04-24T09:18:52.323 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_active_connections": 4, 2024-04-24T09:18:52.323 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_total_time": 0.680803721, 2024-04-24T09:18:52.323 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_send_time": 0.128313597, 2024-04-24T09:18:52.323 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_recv_time": 0.402793241, 2024-04-24T09:18:52.323 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_fast_dispatch_time": 0.107218440, 2024-04-24T09:18:52.323 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T09:18:52.323 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 1869, 2024-04-24T09:18:52.324 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.207307620, 2024-04-24T09:18:52.324 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000110919 2024-04-24T09:18:52.324 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:52.324 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_handle_ack_lat": { 2024-04-24T09:18:52.324 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:52.324 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:52.324 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:52.324 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:52.324 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-04-24T09:18:52.324 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_encrypted_bytes": 448 2024-04-24T09:18:52.324 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:52.324 INFO:teuthology.orchestra.run.smithi189.stdout: "AsyncMessenger::Worker-1": { 2024-04-24T09:18:52.324 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_messages": 1942, 2024-04-24T09:18:52.325 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages": 2319, 2024-04-24T09:18:52.325 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_bytes": 109387129, 2024-04-24T09:18:52.325 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_bytes": 1369277, 2024-04-24T09:18:52.325 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_created_connections": 8, 2024-04-24T09:18:52.325 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_active_connections": 6, 2024-04-24T09:18:52.325 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_total_time": 0.672505372, 2024-04-24T09:18:52.325 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_send_time": 0.174497292, 2024-04-24T09:18:52.325 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_recv_time": 0.330119748, 2024-04-24T09:18:52.325 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_fast_dispatch_time": 0.118194262, 2024-04-24T09:18:52.325 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T09:18:52.325 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 2316, 2024-04-24T09:18:52.325 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.230248909, 2024-04-24T09:18:52.325 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000099416 2024-04-24T09:18:52.325 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:52.325 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_handle_ack_lat": { 2024-04-24T09:18:52.325 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:52.326 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:52.326 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:52.326 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:52.326 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-24T09:18:52.326 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_encrypted_bytes": 752352 2024-04-24T09:18:52.326 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:52.326 INFO:teuthology.orchestra.run.smithi189.stdout: "AsyncMessenger::Worker-2": { 2024-04-24T09:18:52.326 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_messages": 3082, 2024-04-24T09:18:52.326 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages": 3019, 2024-04-24T09:18:52.326 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_bytes": 53181106, 2024-04-24T09:18:52.326 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_bytes": 1102210, 2024-04-24T09:18:52.326 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_created_connections": 5, 2024-04-24T09:18:52.326 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_active_connections": 4, 2024-04-24T09:18:52.327 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_total_time": 0.878802850, 2024-04-24T09:18:52.327 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_send_time": 0.247706038, 2024-04-24T09:18:52.327 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_recv_time": 0.415190794, 2024-04-24T09:18:52.327 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_fast_dispatch_time": 0.182699082, 2024-04-24T09:18:52.327 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T09:18:52.327 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 3018, 2024-04-24T09:18:52.327 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.311576773, 2024-04-24T09:18:52.327 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000103239 2024-04-24T09:18:52.327 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:52.327 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_handle_ack_lat": { 2024-04-24T09:18:52.327 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:52.327 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:52.327 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:52.327 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:52.327 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_encrypted_bytes": 2078768, 2024-04-24T09:18:52.328 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_encrypted_bytes": 316432 2024-04-24T09:18:52.328 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:52.328 INFO:teuthology.orchestra.run.smithi189.stdout: "cct": { 2024-04-24T09:18:52.328 INFO:teuthology.orchestra.run.smithi189.stdout: "total_workers": 1, 2024-04-24T09:18:52.328 INFO:teuthology.orchestra.run.smithi189.stdout: "unhealthy_workers": 0 2024-04-24T09:18:52.328 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:52.328 INFO:teuthology.orchestra.run.smithi189.stdout: "finisher-MDSRank": { 2024-04-24T09:18:52.328 INFO:teuthology.orchestra.run.smithi189.stdout: "queue_len": 0, 2024-04-24T09:18:52.328 INFO:teuthology.orchestra.run.smithi189.stdout: "complete_latency": { 2024-04-24T09:18:52.328 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 10556, 2024-04-24T09:18:52.328 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.580390317, 2024-04-24T09:18:52.328 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000054982 2024-04-24T09:18:52.328 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-04-24T09:18:52.328 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:52.328 INFO:teuthology.orchestra.run.smithi189.stdout: "finisher-PurgeQueue": { 2024-04-24T09:18:52.329 INFO:teuthology.orchestra.run.smithi189.stdout: "queue_len": 0, 2024-04-24T09:18:52.329 INFO:teuthology.orchestra.run.smithi189.stdout: "complete_latency": { 2024-04-24T09:18:52.329 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:52.329 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:52.329 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:52.329 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-04-24T09:18:52.329 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:52.329 INFO:teuthology.orchestra.run.smithi189.stdout: "mds": { 2024-04-24T09:18:52.329 INFO:teuthology.orchestra.run.smithi189.stdout: "request": 0, 2024-04-24T09:18:52.329 INFO:teuthology.orchestra.run.smithi189.stdout: "reply": 0, 2024-04-24T09:18:52.329 INFO:teuthology.orchestra.run.smithi189.stdout: "reply_latency": { 2024-04-24T09:18:52.329 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:52.329 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:52.329 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:52.329 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:52.329 INFO:teuthology.orchestra.run.smithi189.stdout: "slow_reply": 0, 2024-04-24T09:18:52.330 INFO:teuthology.orchestra.run.smithi189.stdout: "forward": 0, 2024-04-24T09:18:52.330 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_fetch_complete": 0, 2024-04-24T09:18:52.330 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_fetch_keys": 0, 2024-04-24T09:18:52.330 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_commit": 0, 2024-04-24T09:18:52.330 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_split": 0, 2024-04-24T09:18:52.330 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_merge": 0, 2024-04-24T09:18:52.330 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes": 9580, 2024-04-24T09:18:52.330 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_top": 0, 2024-04-24T09:18:52.330 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_bottom": 9580, 2024-04-24T09:18:52.330 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_pin_tail": 0, 2024-04-24T09:18:52.330 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_pinned": 9579, 2024-04-24T09:18:52.330 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_expired": 0, 2024-04-24T09:18:52.330 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_with_caps": 0, 2024-04-24T09:18:52.330 INFO:teuthology.orchestra.run.smithi189.stdout: "caps": 0, 2024-04-24T09:18:52.330 INFO:teuthology.orchestra.run.smithi189.stdout: "subtrees": 3, 2024-04-24T09:18:52.331 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse": 0, 2024-04-24T09:18:52.331 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_hit": 0, 2024-04-24T09:18:52.331 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_forward": 0, 2024-04-24T09:18:52.331 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_discover": 0, 2024-04-24T09:18:52.331 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_dir_fetch": 0, 2024-04-24T09:18:52.331 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_remote_ino": 0, 2024-04-24T09:18:52.331 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_lock": 0, 2024-04-24T09:18:52.331 INFO:teuthology.orchestra.run.smithi189.stdout: "load_cent": 0, 2024-04-24T09:18:52.331 INFO:teuthology.orchestra.run.smithi189.stdout: "q": 0, 2024-04-24T09:18:52.331 INFO:teuthology.orchestra.run.smithi189.stdout: "exported": 0, 2024-04-24T09:18:52.331 INFO:teuthology.orchestra.run.smithi189.stdout: "exported_inodes": 0, 2024-04-24T09:18:52.331 INFO:teuthology.orchestra.run.smithi189.stdout: "imported": 0, 2024-04-24T09:18:52.331 INFO:teuthology.orchestra.run.smithi189.stdout: "imported_inodes": 0, 2024-04-24T09:18:52.331 INFO:teuthology.orchestra.run.smithi189.stdout: "openino_dir_fetch": 0, 2024-04-24T09:18:52.331 INFO:teuthology.orchestra.run.smithi189.stdout: "openino_backtrace_fetch": 0, 2024-04-24T09:18:52.331 INFO:teuthology.orchestra.run.smithi189.stdout: "openino_peer_discover": 0, 2024-04-24T09:18:52.332 INFO:teuthology.orchestra.run.smithi189.stdout: "root_rfiles": 0, 2024-04-24T09:18:52.332 INFO:teuthology.orchestra.run.smithi189.stdout: "root_rbytes": 0, 2024-04-24T09:18:52.332 INFO:teuthology.orchestra.run.smithi189.stdout: "root_rsnaps": 0, 2024-04-24T09:18:52.332 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_backtrace_fetch": 0, 2024-04-24T09:18:52.332 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_set_tag": 0, 2024-04-24T09:18:52.332 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_backtrace_repaired": 0, 2024-04-24T09:18:52.332 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_inotable_repaired": 0, 2024-04-24T09:18:52.332 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_dir_inodes": 0, 2024-04-24T09:18:52.332 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_dir_base_inodes": 0, 2024-04-24T09:18:52.332 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_dirfrag_rstats": 0, 2024-04-24T09:18:52.332 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_file_inodes": 0, 2024-04-24T09:18:52.332 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_inode_file_caps": 0, 2024-04-24T09:18:52.332 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_revoke": 0, 2024-04-24T09:18:52.332 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_grant": 0, 2024-04-24T09:18:52.332 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_trunc": 0, 2024-04-24T09:18:52.333 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-24T09:18:52.333 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-24T09:18:52.333 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_caps": 0, 2024-04-24T09:18:52.333 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_caps_dirty": 0, 2024-04-24T09:18:52.333 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_cap_release": 0, 2024-04-24T09:18:52.333 INFO:teuthology.orchestra.run.smithi189.stdout: "process_request_cap_release": 0 2024-04-24T09:18:52.333 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:52.333 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_cache": { 2024-04-24T09:18:52.333 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_update": 0, 2024-04-24T09:18:52.333 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_update_receipt": 0, 2024-04-24T09:18:52.333 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_try_discover": 0, 2024-04-24T09:18:52.333 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_send_discover": 0, 2024-04-24T09:18:52.333 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_handle_discover": 0, 2024-04-24T09:18:52.333 INFO:teuthology.orchestra.run.smithi189.stdout: "num_strays": 0, 2024-04-24T09:18:52.333 INFO:teuthology.orchestra.run.smithi189.stdout: "num_strays_delayed": 0, 2024-04-24T09:18:52.333 INFO:teuthology.orchestra.run.smithi189.stdout: "num_strays_enqueuing": 0, 2024-04-24T09:18:52.334 INFO:teuthology.orchestra.run.smithi189.stdout: "strays_created": 0, 2024-04-24T09:18:52.334 INFO:teuthology.orchestra.run.smithi189.stdout: "strays_enqueued": 0, 2024-04-24T09:18:52.334 INFO:teuthology.orchestra.run.smithi189.stdout: "strays_reintegrated": 0, 2024-04-24T09:18:52.334 INFO:teuthology.orchestra.run.smithi189.stdout: "strays_migrated": 0, 2024-04-24T09:18:52.334 INFO:teuthology.orchestra.run.smithi189.stdout: "num_recovering_processing": 0, 2024-04-24T09:18:52.334 INFO:teuthology.orchestra.run.smithi189.stdout: "num_recovering_enqueued": 0, 2024-04-24T09:18:52.334 INFO:teuthology.orchestra.run.smithi189.stdout: "num_recovering_prioritized": 0, 2024-04-24T09:18:52.334 INFO:teuthology.orchestra.run.smithi189.stdout: "recovery_started": 0, 2024-04-24T09:18:52.334 INFO:teuthology.orchestra.run.smithi189.stdout: "recovery_completed": 0, 2024-04-24T09:18:52.334 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_quiesce_path": 0, 2024-04-24T09:18:52.334 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_quiesce_inode": 0, 2024-04-24T09:18:52.334 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_enqueue_scrub": 0, 2024-04-24T09:18:52.334 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_exportdir": 0, 2024-04-24T09:18:52.334 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_flush": 0, 2024-04-24T09:18:52.335 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_fragmentdir": 0, 2024-04-24T09:18:52.335 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_fragstats": 0, 2024-04-24T09:18:52.335 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_inodestats": 0 2024-04-24T09:18:52.335 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:52.335 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_log": { 2024-04-24T09:18:52.335 INFO:teuthology.orchestra.run.smithi189.stdout: "evlrg": 0, 2024-04-24T09:18:52.335 INFO:teuthology.orchestra.run.smithi189.stdout: "evadd": 0, 2024-04-24T09:18:52.335 INFO:teuthology.orchestra.run.smithi189.stdout: "evex": 0, 2024-04-24T09:18:52.335 INFO:teuthology.orchestra.run.smithi189.stdout: "evtrm": 0, 2024-04-24T09:18:52.335 INFO:teuthology.orchestra.run.smithi189.stdout: "ev": 56121, 2024-04-24T09:18:52.335 INFO:teuthology.orchestra.run.smithi189.stdout: "evexg": 0, 2024-04-24T09:18:52.335 INFO:teuthology.orchestra.run.smithi189.stdout: "evexd": 0, 2024-04-24T09:18:52.335 INFO:teuthology.orchestra.run.smithi189.stdout: "segadd": 0, 2024-04-24T09:18:52.335 INFO:teuthology.orchestra.run.smithi189.stdout: "segex": 0, 2024-04-24T09:18:52.335 INFO:teuthology.orchestra.run.smithi189.stdout: "segtrm": 0, 2024-04-24T09:18:52.335 INFO:teuthology.orchestra.run.smithi189.stdout: "seg": 88, 2024-04-24T09:18:52.336 INFO:teuthology.orchestra.run.smithi189.stdout: "segmjr": 6, 2024-04-24T09:18:52.336 INFO:teuthology.orchestra.run.smithi189.stdout: "segexg": 0, 2024-04-24T09:18:52.336 INFO:teuthology.orchestra.run.smithi189.stdout: "segexd": 0, 2024-04-24T09:18:52.336 INFO:teuthology.orchestra.run.smithi189.stdout: "expos": 4194304, 2024-04-24T09:18:52.336 INFO:teuthology.orchestra.run.smithi189.stdout: "wrpos": 272469094, 2024-04-24T09:18:52.336 INFO:teuthology.orchestra.run.smithi189.stdout: "rdpos": 272446153, 2024-04-24T09:18:52.336 INFO:teuthology.orchestra.run.smithi189.stdout: "jlat": { 2024-04-24T09:18:52.336 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:52.336 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:52.336 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:52.336 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:52.336 INFO:teuthology.orchestra.run.smithi189.stdout: "replayed": 56121 2024-04-24T09:18:52.336 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:52.336 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_mem": { 2024-04-24T09:18:52.336 INFO:teuthology.orchestra.run.smithi189.stdout: "ino": 1613, 2024-04-24T09:18:52.336 INFO:teuthology.orchestra.run.smithi189.stdout: "ino+": 10104, 2024-04-24T09:18:52.337 INFO:teuthology.orchestra.run.smithi189.stdout: "ino-": 8491, 2024-04-24T09:18:52.337 INFO:teuthology.orchestra.run.smithi189.stdout: "dir": 64, 2024-04-24T09:18:52.337 INFO:teuthology.orchestra.run.smithi189.stdout: "dir+": 121, 2024-04-24T09:18:52.337 INFO:teuthology.orchestra.run.smithi189.stdout: "dir-": 57, 2024-04-24T09:18:52.337 INFO:teuthology.orchestra.run.smithi189.stdout: "dn": 9580, 2024-04-24T09:18:52.337 INFO:teuthology.orchestra.run.smithi189.stdout: "dn+": 10452, 2024-04-24T09:18:52.337 INFO:teuthology.orchestra.run.smithi189.stdout: "dn-": 872, 2024-04-24T09:18:52.337 INFO:teuthology.orchestra.run.smithi189.stdout: "cap": 0, 2024-04-24T09:18:52.337 INFO:teuthology.orchestra.run.smithi189.stdout: "cap+": 0, 2024-04-24T09:18:52.337 INFO:teuthology.orchestra.run.smithi189.stdout: "cap-": 0, 2024-04-24T09:18:52.337 INFO:teuthology.orchestra.run.smithi189.stdout: "rss": 67416, 2024-04-24T09:18:52.337 INFO:teuthology.orchestra.run.smithi189.stdout: "heap": 215416 2024-04-24T09:18:52.337 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:52.337 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_server": { 2024-04-24T09:18:52.337 INFO:teuthology.orchestra.run.smithi189.stdout: "dispatch_client_request": 0, 2024-04-24T09:18:52.337 INFO:teuthology.orchestra.run.smithi189.stdout: "dispatch_server_request": 0, 2024-04-24T09:18:52.338 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_request": 0, 2024-04-24T09:18:52.338 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_session": 0, 2024-04-24T09:18:52.338 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_peer_request": 0, 2024-04-24T09:18:52.338 INFO:teuthology.orchestra.run.smithi189.stdout: "req_create_latency": { 2024-04-24T09:18:52.338 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:52.338 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:52.338 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:52.338 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:52.338 INFO:teuthology.orchestra.run.smithi189.stdout: "req_getattr_latency": { 2024-04-24T09:18:52.338 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:52.338 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:52.338 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:52.338 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:52.338 INFO:teuthology.orchestra.run.smithi189.stdout: "req_getfilelock_latency": { 2024-04-24T09:18:52.338 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:52.339 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:52.339 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:52.339 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:52.339 INFO:teuthology.orchestra.run.smithi189.stdout: "req_link_latency": { 2024-04-24T09:18:52.339 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:52.339 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:52.339 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:52.339 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:52.339 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookup_latency": { 2024-04-24T09:18:52.339 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:52.339 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:52.339 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:52.339 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:52.339 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookuphash_latency": { 2024-04-24T09:18:52.339 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:52.339 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:52.339 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:52.340 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:52.340 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookupino_latency": { 2024-04-24T09:18:52.340 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:52.340 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:52.340 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:52.340 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:52.340 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookupname_latency": { 2024-04-24T09:18:52.340 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:52.340 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:52.340 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:52.340 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:52.340 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookupparent_latency": { 2024-04-24T09:18:52.340 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:52.340 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:52.341 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:52.341 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:52.341 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookupsnap_latency": { 2024-04-24T09:18:52.341 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:52.341 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:52.341 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:52.341 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:52.341 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lssnap_latency": { 2024-04-24T09:18:52.341 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:52.341 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:52.341 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:52.341 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:52.342 INFO:teuthology.orchestra.run.smithi189.stdout: "req_mkdir_latency": { 2024-04-24T09:18:52.342 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:52.342 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:52.342 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:52.342 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:52.342 INFO:teuthology.orchestra.run.smithi189.stdout: "req_mknod_latency": { 2024-04-24T09:18:52.342 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:52.342 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:52.342 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:52.342 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:52.342 INFO:teuthology.orchestra.run.smithi189.stdout: "req_mksnap_latency": { 2024-04-24T09:18:52.342 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:52.343 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:52.343 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:52.343 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:52.343 INFO:teuthology.orchestra.run.smithi189.stdout: "req_open_latency": { 2024-04-24T09:18:52.343 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:52.343 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:52.343 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:52.343 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:52.343 INFO:teuthology.orchestra.run.smithi189.stdout: "req_readdir_latency": { 2024-04-24T09:18:52.343 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:52.343 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:52.343 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:52.343 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:52.343 INFO:teuthology.orchestra.run.smithi189.stdout: "req_rename_latency": { 2024-04-24T09:18:52.343 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:52.343 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:52.344 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:52.344 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:52.344 INFO:teuthology.orchestra.run.smithi189.stdout: "req_renamesnap_latency": { 2024-04-24T09:18:52.344 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:52.344 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:52.344 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:52.344 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:52.344 INFO:teuthology.orchestra.run.smithi189.stdout: "req_snapdiff_latency": { 2024-04-24T09:18:52.344 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:52.344 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:52.344 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:52.344 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:52.344 INFO:teuthology.orchestra.run.smithi189.stdout: "req_rmdir_latency": { 2024-04-24T09:18:52.344 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:52.345 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:52.345 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:52.345 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:52.345 INFO:teuthology.orchestra.run.smithi189.stdout: "req_rmsnap_latency": { 2024-04-24T09:18:52.345 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:52.345 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:52.345 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:52.345 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:52.345 INFO:teuthology.orchestra.run.smithi189.stdout: "req_rmxattr_latency": { 2024-04-24T09:18:52.345 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:52.345 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:52.345 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:52.345 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:52.345 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setattr_latency": { 2024-04-24T09:18:52.345 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:52.345 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:52.345 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:52.346 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:52.346 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setdirlayout_latency": { 2024-04-24T09:18:52.346 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:52.346 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:52.346 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:52.346 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:52.346 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setfilelock_latency": { 2024-04-24T09:18:52.346 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:52.346 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:52.346 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:52.346 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:52.346 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setlayout_latency": { 2024-04-24T09:18:52.346 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:52.346 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:52.346 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:52.347 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:52.347 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setxattr_latency": { 2024-04-24T09:18:52.347 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:52.347 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:52.347 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:52.347 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:52.347 INFO:teuthology.orchestra.run.smithi189.stdout: "req_symlink_latency": { 2024-04-24T09:18:52.347 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:52.347 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:52.347 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:52.347 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:52.347 INFO:teuthology.orchestra.run.smithi189.stdout: "req_unlink_latency": { 2024-04-24T09:18:52.347 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:52.347 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:52.347 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:52.348 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:52.348 INFO:teuthology.orchestra.run.smithi189.stdout: "cap_revoke_eviction": 0, 2024-04-24T09:18:52.348 INFO:teuthology.orchestra.run.smithi189.stdout: "cap_acquisition_throttle": 0, 2024-04-24T09:18:52.348 INFO:teuthology.orchestra.run.smithi189.stdout: "req_getvxattr_latency": { 2024-04-24T09:18:52.348 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:52.348 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:52.348 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:52.348 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-04-24T09:18:52.348 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:52.348 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_sessions": { 2024-04-24T09:18:52.348 INFO:teuthology.orchestra.run.smithi189.stdout: "session_count": 2, 2024-04-24T09:18:52.348 INFO:teuthology.orchestra.run.smithi189.stdout: "session_add": 2, 2024-04-24T09:18:52.348 INFO:teuthology.orchestra.run.smithi189.stdout: "session_remove": 0, 2024-04-24T09:18:52.348 INFO:teuthology.orchestra.run.smithi189.stdout: "sessions_open": 2, 2024-04-24T09:18:52.348 INFO:teuthology.orchestra.run.smithi189.stdout: "sessions_stale": 0, 2024-04-24T09:18:52.349 INFO:teuthology.orchestra.run.smithi189.stdout: "total_load": 0, 2024-04-24T09:18:52.349 INFO:teuthology.orchestra.run.smithi189.stdout: "average_load": 0, 2024-04-24T09:18:52.349 INFO:teuthology.orchestra.run.smithi189.stdout: "avg_session_uptime": 3023, 2024-04-24T09:18:52.349 INFO:teuthology.orchestra.run.smithi189.stdout: "mdthresh_evicted": 0 2024-04-24T09:18:52.349 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:52.349 INFO:teuthology.orchestra.run.smithi189.stdout: "mempool": { 2024-04-24T09:18:52.349 INFO:teuthology.orchestra.run.smithi189.stdout: "bloom_filter_bytes": 0, 2024-04-24T09:18:52.349 INFO:teuthology.orchestra.run.smithi189.stdout: "bloom_filter_items": 0, 2024-04-24T09:18:52.349 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_alloc_bytes": 0, 2024-04-24T09:18:52.349 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_alloc_items": 0, 2024-04-24T09:18:52.349 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_data_bytes": 0, 2024-04-24T09:18:52.349 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_data_items": 0, 2024-04-24T09:18:52.349 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-24T09:18:52.349 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_onode_items": 0, 2024-04-24T09:18:52.349 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-24T09:18:52.349 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_meta_items": 0, 2024-04-24T09:18:52.350 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_other_bytes": 0, 2024-04-24T09:18:52.350 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_other_items": 0, 2024-04-24T09:18:52.350 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-24T09:18:52.350 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_buffer_items": 0, 2024-04-24T09:18:52.350 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_extent_bytes": 0, 2024-04-24T09:18:52.350 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_extent_items": 0, 2024-04-24T09:18:52.350 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_blob_bytes": 0, 2024-04-24T09:18:52.350 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_blob_items": 0, 2024-04-24T09:18:52.350 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-24T09:18:52.350 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_shared_blob_items": 0, 2024-04-24T09:18:52.350 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-24T09:18:52.350 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_inline_bl_items": 0, 2024-04-24T09:18:52.350 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_fsck_bytes": 0, 2024-04-24T09:18:52.350 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_fsck_items": 0, 2024-04-24T09:18:52.350 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_txc_bytes": 0, 2024-04-24T09:18:52.351 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_txc_items": 0, 2024-04-24T09:18:52.351 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-24T09:18:52.351 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_deferred_items": 0, 2024-04-24T09:18:52.351 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_bytes": 0, 2024-04-24T09:18:52.351 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_items": 0, 2024-04-24T09:18:52.351 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_bytes": 0, 2024-04-24T09:18:52.351 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_items": 0, 2024-04-24T09:18:52.351 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_reader_bytes": 0, 2024-04-24T09:18:52.351 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_reader_items": 0, 2024-04-24T09:18:52.351 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_writer_bytes": 0, 2024-04-24T09:18:52.351 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_writer_items": 0, 2024-04-24T09:18:52.351 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_anon_bytes": 233703, 2024-04-24T09:18:52.351 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_anon_items": 80, 2024-04-24T09:18:52.351 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_meta_bytes": 0, 2024-04-24T09:18:52.351 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_meta_items": 0, 2024-04-24T09:18:52.351 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_bytes": 0, 2024-04-24T09:18:52.351 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_items": 0, 2024-04-24T09:18:52.352 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_mapbl_bytes": 0, 2024-04-24T09:18:52.352 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_mapbl_items": 0, 2024-04-24T09:18:52.352 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_pglog_bytes": 0, 2024-04-24T09:18:52.352 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_pglog_items": 0, 2024-04-24T09:18:52.352 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_bytes": 9800, 2024-04-24T09:18:52.352 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_items": 188, 2024-04-24T09:18:52.352 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_mapping_bytes": 0, 2024-04-24T09:18:52.352 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_mapping_items": 0, 2024-04-24T09:18:52.352 INFO:teuthology.orchestra.run.smithi189.stdout: "pgmap_bytes": 0, 2024-04-24T09:18:52.352 INFO:teuthology.orchestra.run.smithi189.stdout: "pgmap_items": 0, 2024-04-24T09:18:52.352 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_co_bytes": 10572275, 2024-04-24T09:18:52.352 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_co_items": 47738, 2024-04-24T09:18:52.352 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_1_bytes": 0, 2024-04-24T09:18:52.352 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_1_items": 0, 2024-04-24T09:18:52.352 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_2_bytes": 0, 2024-04-24T09:18:52.352 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_2_items": 0 2024-04-24T09:18:52.353 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:52.353 INFO:teuthology.orchestra.run.smithi189.stdout: "objecter": { 2024-04-24T09:18:52.353 INFO:teuthology.orchestra.run.smithi189.stdout: "op_active": 0, 2024-04-24T09:18:52.353 INFO:teuthology.orchestra.run.smithi189.stdout: "op_laggy": 0, 2024-04-24T09:18:52.353 INFO:teuthology.orchestra.run.smithi189.stdout: "op_send": 6277, 2024-04-24T09:18:52.353 INFO:teuthology.orchestra.run.smithi189.stdout: "op_send_bytes": 21, 2024-04-24T09:18:52.353 INFO:teuthology.orchestra.run.smithi189.stdout: "op_resend": 3, 2024-04-24T09:18:52.353 INFO:teuthology.orchestra.run.smithi189.stdout: "op_reply": 6277, 2024-04-24T09:18:52.353 INFO:teuthology.orchestra.run.smithi189.stdout: "op_latency": { 2024-04-24T09:18:52.353 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 6277, 2024-04-24T09:18:52.353 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 9.939806769, 2024-04-24T09:18:52.353 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.001583528 2024-04-24T09:18:52.353 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:52.353 INFO:teuthology.orchestra.run.smithi189.stdout: "op_inflight": 0, 2024-04-24T09:18:52.353 INFO:teuthology.orchestra.run.smithi189.stdout: "oplen_avg": { 2024-04-24T09:18:52.354 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 6277, 2024-04-24T09:18:52.354 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 6278 2024-04-24T09:18:52.354 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:52.354 INFO:teuthology.orchestra.run.smithi189.stdout: "op": 6277, 2024-04-24T09:18:52.354 INFO:teuthology.orchestra.run.smithi189.stdout: "op_r": 6277, 2024-04-24T09:18:52.354 INFO:teuthology.orchestra.run.smithi189.stdout: "op_w": 0, 2024-04-24T09:18:52.354 INFO:teuthology.orchestra.run.smithi189.stdout: "op_rmw": 0, 2024-04-24T09:18:52.354 INFO:teuthology.orchestra.run.smithi189.stdout: "op_pg": 0, 2024-04-24T09:18:52.354 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_stat": 3438, 2024-04-24T09:18:52.354 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_create": 0, 2024-04-24T09:18:52.354 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_read": 2837, 2024-04-24T09:18:52.354 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_write": 0, 2024-04-24T09:18:52.354 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_writefull": 0, 2024-04-24T09:18:52.354 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_writesame": 0, 2024-04-24T09:18:52.354 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_append": 0, 2024-04-24T09:18:52.354 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_zero": 0, 2024-04-24T09:18:52.355 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_truncate": 0, 2024-04-24T09:18:52.355 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_delete": 0, 2024-04-24T09:18:52.355 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_mapext": 0, 2024-04-24T09:18:52.355 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_sparse_read": 0, 2024-04-24T09:18:52.355 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_clonerange": 0, 2024-04-24T09:18:52.355 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_getxattr": 1, 2024-04-24T09:18:52.355 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_setxattr": 0, 2024-04-24T09:18:52.355 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_cmpxattr": 0, 2024-04-24T09:18:52.355 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_rmxattr": 0, 2024-04-24T09:18:52.355 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_resetxattrs": 0, 2024-04-24T09:18:52.355 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_call": 0, 2024-04-24T09:18:52.355 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_watch": 0, 2024-04-24T09:18:52.355 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_notify": 0, 2024-04-24T09:18:52.355 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_src_cmpxattr": 0, 2024-04-24T09:18:52.356 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_pgls": 0, 2024-04-24T09:18:52.356 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_pgls_filter": 0, 2024-04-24T09:18:52.356 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_other": 0, 2024-04-24T09:18:52.356 INFO:teuthology.orchestra.run.smithi189.stdout: "linger_active": 0, 2024-04-24T09:18:52.356 INFO:teuthology.orchestra.run.smithi189.stdout: "linger_send": 0, 2024-04-24T09:18:52.356 INFO:teuthology.orchestra.run.smithi189.stdout: "linger_resend": 0, 2024-04-24T09:18:52.356 INFO:teuthology.orchestra.run.smithi189.stdout: "linger_ping": 0, 2024-04-24T09:18:52.356 INFO:teuthology.orchestra.run.smithi189.stdout: "poolop_active": 0, 2024-04-24T09:18:52.356 INFO:teuthology.orchestra.run.smithi189.stdout: "poolop_send": 0, 2024-04-24T09:18:52.356 INFO:teuthology.orchestra.run.smithi189.stdout: "poolop_resend": 0, 2024-04-24T09:18:52.356 INFO:teuthology.orchestra.run.smithi189.stdout: "poolstat_active": 0, 2024-04-24T09:18:52.356 INFO:teuthology.orchestra.run.smithi189.stdout: "poolstat_send": 0, 2024-04-24T09:18:52.356 INFO:teuthology.orchestra.run.smithi189.stdout: "poolstat_resend": 0, 2024-04-24T09:18:52.356 INFO:teuthology.orchestra.run.smithi189.stdout: "statfs_active": 0, 2024-04-24T09:18:52.356 INFO:teuthology.orchestra.run.smithi189.stdout: "statfs_send": 0, 2024-04-24T09:18:52.357 INFO:teuthology.orchestra.run.smithi189.stdout: "statfs_resend": 0, 2024-04-24T09:18:52.357 INFO:teuthology.orchestra.run.smithi189.stdout: "command_active": 0, 2024-04-24T09:18:52.357 INFO:teuthology.orchestra.run.smithi189.stdout: "command_send": 0, 2024-04-24T09:18:52.357 INFO:teuthology.orchestra.run.smithi189.stdout: "command_resend": 0, 2024-04-24T09:18:52.357 INFO:teuthology.orchestra.run.smithi189.stdout: "map_epoch": 136, 2024-04-24T09:18:52.357 INFO:teuthology.orchestra.run.smithi189.stdout: "map_full": 0, 2024-04-24T09:18:52.357 INFO:teuthology.orchestra.run.smithi189.stdout: "map_inc": 47, 2024-04-24T09:18:52.357 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_sessions": 10, 2024-04-24T09:18:52.357 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_session_open": 11, 2024-04-24T09:18:52.357 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_session_close": 1, 2024-04-24T09:18:52.357 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_laggy": 0, 2024-04-24T09:18:52.357 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_wr": 0, 2024-04-24T09:18:52.357 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_rd": 2, 2024-04-24T09:18:52.357 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_del": 0 2024-04-24T09:18:52.357 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:52.358 INFO:teuthology.orchestra.run.smithi189.stdout: "oft": { 2024-04-24T09:18:52.358 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_total_objs": 0, 2024-04-24T09:18:52.358 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_total_kv_pairs": 0, 2024-04-24T09:18:52.358 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_total_updates": 0, 2024-04-24T09:18:52.358 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_total_removes": 0 2024-04-24T09:18:52.358 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:52.358 INFO:teuthology.orchestra.run.smithi189.stdout: "osd-slow-ops": { 2024-04-24T09:18:52.358 INFO:teuthology.orchestra.run.smithi189.stdout: "slow_ops_count": 0 2024-04-24T09:18:52.358 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:52.358 INFO:teuthology.orchestra.run.smithi189.stdout: "purge_queue": { 2024-04-24T09:18:52.358 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executing_ops": 0, 2024-04-24T09:18:52.358 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executing_ops_high_water": 0, 2024-04-24T09:18:52.358 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executing": 0, 2024-04-24T09:18:52.358 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executing_high_water": 0, 2024-04-24T09:18:52.359 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executed_ops": 0, 2024-04-24T09:18:52.359 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executed": 0, 2024-04-24T09:18:52.359 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_item_in_journal": 0 2024-04-24T09:18:52.359 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:52.359 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-24T09:18:52.359 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-04-24T09:18:52.359 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 104857600, 2024-04-24T09:18:52.359 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-04-24T09:18:52.359 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 6890, 2024-04-24T09:18:52.359 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 271449503, 2024-04-24T09:18:52.359 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-04-24T09:18:52.359 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 6890, 2024-04-24T09:18:52.359 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 0, 2024-04-24T09:18:52.359 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 0, 2024-04-24T09:18:52.359 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 6890, 2024-04-24T09:18:52.360 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 271449503, 2024-04-24T09:18:52.360 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-04-24T09:18:52.360 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:52.360 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:52.360 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:52.360 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-04-24T09:18:52.360 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:52.360 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-objecter_bytes": { 2024-04-24T09:18:52.360 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-04-24T09:18:52.360 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 104857600, 2024-04-24T09:18:52.360 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-04-24T09:18:52.360 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 0, 2024-04-24T09:18:52.360 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 0, 2024-04-24T09:18:52.360 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-04-24T09:18:52.361 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 0, 2024-04-24T09:18:52.361 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 6277, 2024-04-24T09:18:52.361 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 268282609, 2024-04-24T09:18:52.361 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 6277, 2024-04-24T09:18:52.361 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 268282609, 2024-04-24T09:18:52.361 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-04-24T09:18:52.361 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:52.361 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:52.361 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:52.361 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-04-24T09:18:52.361 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:52.361 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-objecter_ops": { 2024-04-24T09:18:52.361 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-04-24T09:18:52.361 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 1024, 2024-04-24T09:18:52.361 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-04-24T09:18:52.362 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 0, 2024-04-24T09:18:52.362 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 0, 2024-04-24T09:18:52.362 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-04-24T09:18:52.362 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 0, 2024-04-24T09:18:52.362 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 6277, 2024-04-24T09:18:52.362 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 6277, 2024-04-24T09:18:52.362 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 6277, 2024-04-24T09:18:52.362 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 6277, 2024-04-24T09:18:52.362 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-04-24T09:18:52.362 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:52.362 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:52.362 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:52.362 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-04-24T09:18:52.362 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:52.362 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-write_buf_throttle": { 2024-04-24T09:18:52.363 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-04-24T09:18:52.363 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 3758096384, 2024-04-24T09:18:52.363 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-04-24T09:18:52.363 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 0, 2024-04-24T09:18:52.363 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 0, 2024-04-24T09:18:52.363 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-04-24T09:18:52.363 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 0, 2024-04-24T09:18:52.363 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 0, 2024-04-24T09:18:52.363 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 0, 2024-04-24T09:18:52.363 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 0, 2024-04-24T09:18:52.363 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 0, 2024-04-24T09:18:52.363 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-04-24T09:18:52.363 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:52.363 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:52.363 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:52.363 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-04-24T09:18:52.364 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-04-24T09:18:52.364 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-write_buf_throttle-0x559cf7bc0780": { 2024-04-24T09:18:52.364 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-04-24T09:18:52.364 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 3758096384, 2024-04-24T09:18:52.364 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-04-24T09:18:52.364 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 0, 2024-04-24T09:18:52.364 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 0, 2024-04-24T09:18:52.364 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-04-24T09:18:52.364 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 0, 2024-04-24T09:18:52.364 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 0, 2024-04-24T09:18:52.364 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 0, 2024-04-24T09:18:52.364 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 0, 2024-04-24T09:18:52.364 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 0, 2024-04-24T09:18:52.364 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-04-24T09:18:52.364 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-04-24T09:18:52.364 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-04-24T09:18:52.365 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-04-24T09:18:52.365 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-04-24T09:18:52.365 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-04-24T09:18:52.365 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-04-24T09:18:52.365 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:18:52 smithi189 ceph-mon[31791]: pgmap v862: 129 pgs: 129 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-24T09:18:52.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:18:52 smithi138 ceph-mon[31441]: pgmap v862: 129 pgs: 129 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-24T09:18:52.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:18:52 smithi083 ceph-mon[27907]: pgmap v862: 129 pgs: 129 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-24T09:18:53.302 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-04-24T09:18:53.302 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-04-24T09:18:53.302 INFO:tasks.check_counter:Daemon mds.l mds.root_rsnaps=0 2024-04-24T09:18:53.302 INFO:tasks.check_counter:Daemon mds.l mds_server.req_mksnap_latency.avgcount=0 2024-04-24T09:18:53.302 INFO:tasks.check_counter:Daemon mds.l mds_server.req_rmsnap_latency.avgcount=0 2024-04-24T09:18:53.302 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-04-24T09:18:53.313 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-04-24T09:18:53.313 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-24T09:18:53.313 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T09:18:53.387 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-24T09:18:53.387 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T09:18:53.442 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T09:18:53.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:53.649+0000 7fd30e200640 1 -- 172.21.15.83:0/3678569969 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd308108e40 msgr2=0x7fd3081152b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:53.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:53.649+0000 7fd30e200640 1 --2- 172.21.15.83:0/3678569969 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd308108e40 0x7fd3081152b0 secure :-1 s=READY pgs=2907 cs=0 l=1 rev1=1 crypto rx=0x7fd300009ca0 tx=0x7fd30002a950 comp rx=0 tx=0).stop 2024-04-24T09:18:53.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:53.649+0000 7fd30e200640 1 -- 172.21.15.83:0/3678569969 shutdown_connections 2024-04-24T09:18:53.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:53.649+0000 7fd30e200640 1 --2- 172.21.15.83:0/3678569969 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd308108e40 0x7fd3081152b0 unknown :-1 s=CLOSED pgs=2907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:53.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:53.649+0000 7fd30e200640 1 --2- 172.21.15.83:0/3678569969 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd3081084a0 0x7fd308108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:53.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:53.649+0000 7fd30e200640 1 --2- 172.21.15.83:0/3678569969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd30810d3c0 0x7fd30810d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:53.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:53.649+0000 7fd30e200640 1 -- 172.21.15.83:0/3678569969 >> 172.21.15.83:0/3678569969 conn(0x7fd308077b80 msgr2=0x7fd308079fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:18:53.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:53.649+0000 7fd30e200640 1 -- 172.21.15.83:0/3678569969 shutdown_connections 2024-04-24T09:18:53.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:53.649+0000 7fd30e200640 1 -- 172.21.15.83:0/3678569969 wait complete. 2024-04-24T09:18:53.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:53.650+0000 7fd30e200640 1 Processor -- start 2024-04-24T09:18:53.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:53.650+0000 7fd30e200640 1 -- start start 2024-04-24T09:18:53.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:53.651+0000 7fd30e200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3081084a0 0x7fd3081a4800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:53.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:53.651+0000 7fd30e200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd308108e40 0x7fd3081a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:53.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:53.651+0000 7fd30e200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd30810d3c0 0x7fd30819e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:53.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:53.651+0000 7fd30e200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd3081186b0 con 0x7fd3081084a0 2024-04-24T09:18:53.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:53.651+0000 7fd30e200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd308118530 con 0x7fd30810d3c0 2024-04-24T09:18:53.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:53.651+0000 7fd30e200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd308118830 con 0x7fd308108e40 2024-04-24T09:18:53.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:53.651+0000 7fd30ce00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3081084a0 0x7fd3081a4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:53.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:53.651+0000 7fd30ce00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3081084a0 0x7fd3081a4800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38130/0 (socket says 172.21.15.83:38130) 2024-04-24T09:18:53.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:53.651+0000 7fd30ce00640 1 -- 172.21.15.83:0/2334669435 learned_addr learned my addr 172.21.15.83:0/2334669435 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:18:53.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:53.651+0000 7fd2ffe00640 1 --2- 172.21.15.83:0/2334669435 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd308108e40 0x7fd3081a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:53.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:53.651+0000 7fd30d800640 1 --2- 172.21.15.83:0/2334669435 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd30810d3c0 0x7fd30819e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:53.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:53.652+0000 7fd30ce00640 1 -- 172.21.15.83:0/2334669435 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd308108e40 msgr2=0x7fd3081a4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:53.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:53.652+0000 7fd30ce00640 1 --2- 172.21.15.83:0/2334669435 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd308108e40 0x7fd3081a4d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:53.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:53.652+0000 7fd30ce00640 1 -- 172.21.15.83:0/2334669435 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd30810d3c0 msgr2=0x7fd30819e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:53.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:53.652+0000 7fd30ce00640 1 --2- 172.21.15.83:0/2334669435 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd30810d3c0 0x7fd30819e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:53.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:53.652+0000 7fd30ce00640 1 -- 172.21.15.83:0/2334669435 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd30819f130 con 0x7fd3081084a0 2024-04-24T09:18:53.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:53.652+0000 7fd30d800640 1 --2- 172.21.15.83:0/2334669435 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd30810d3c0 0x7fd30819e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:18:53.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:53.652+0000 7fd30ce00640 1 --2- 172.21.15.83:0/2334669435 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3081084a0 0x7fd3081a4800 secure :-1 s=READY pgs=2908 cs=0 l=1 rev1=1 crypto rx=0x7fd2f800c890 tx=0x7fd2f800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:18:53.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:53.652+0000 7fd306000640 1 -- 172.21.15.83:0/2334669435 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2f8004220 con 0x7fd3081084a0 2024-04-24T09:18:53.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:53.652+0000 7fd306000640 1 -- 172.21.15.83:0/2334669435 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd2f80043c0 con 0x7fd3081084a0 2024-04-24T09:18:53.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:53.652+0000 7fd306000640 1 -- 172.21.15.83:0/2334669435 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2f8004a00 con 0x7fd3081084a0 2024-04-24T09:18:53.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:53.652+0000 7fd30e200640 1 -- 172.21.15.83:0/2334669435 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd30819f420 con 0x7fd3081084a0 2024-04-24T09:18:53.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:53.652+0000 7fd30e200640 1 -- 172.21.15.83:0/2334669435 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3081ab6c0 con 0x7fd3081084a0 2024-04-24T09:18:53.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:53.654+0000 7fd30e200640 1 -- 172.21.15.83:0/2334669435 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd308069190 con 0x7fd3081084a0 2024-04-24T09:18:53.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:53.656+0000 7fd306000640 1 -- 172.21.15.83:0/2334669435 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd2f8004ba0 con 0x7fd3081084a0 2024-04-24T09:18:53.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:53.657+0000 7fd306000640 1 --2- 172.21.15.83:0/2334669435 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fd2ec0a3d20 0x7fd2ec0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:53.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:53.657+0000 7fd306000640 1 -- 172.21.15.83:0/2334669435 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 9583+0+0 (secure 0 0 0) 0x7fd2f80d09c0 con 0x7fd3081084a0 2024-04-24T09:18:53.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:53.657+0000 7fd2ffe00640 1 --2- 172.21.15.83:0/2334669435 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fd2ec0a3d20 0x7fd2ec0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:53.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:53.658+0000 7fd2ffe00640 1 --2- 172.21.15.83:0/2334669435 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fd2ec0a3d20 0x7fd2ec0a61e0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fd2f0005fd0 tx=0x7fd2f0005850 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:18:53.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:53.662+0000 7fd306000640 1 -- 172.21.15.83:0/2334669435 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd2f8014030 con 0x7fd3081084a0 2024-04-24T09:18:53.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:53.864+0000 7fd30e200640 1 -- 172.21.15.83:0/2334669435 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd30819fee0 con 0x7fd3081084a0 2024-04-24T09:18:53.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:53.869+0000 7fd306000640 1 -- 172.21.15.83:0/2334669435 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v136) ==== 74+0+25453 (secure 0 0 0) 0x7fd2f809cbe0 con 0x7fd3081084a0 2024-04-24T09:18:53.870 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T09:18:53.870 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":136,"fsid":"bd3e039e-0215-11ef-bc93-c7b262605968","created":"2024-04-24T08:37:32.209944+0000","modified":"2024-04-24T09:18:14.841781+0000","last_up_change":"2024-04-24T08:47:17.966041+0000","last_in_change":"2024-04-24T08:46:53.835936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T08:41:56.834372+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T08:48:26.978311+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T08:48:28.352731+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"135","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":48,"snap_epoch":135,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T08:48:32.275167+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"135","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":48,"snap_epoch":135,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2cd77f3a-19dd-4c6e-a199-1dcbd5faa679","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2132675555},{"type":"v1","addr":"172.21.15.83:6803","nonce":2132675555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2132675555},{"type":"v1","addr":"172.21.15.83:6805","nonce":2132675555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2132675555},{"type":"v1","addr":"172.21.15.83:6809","nonce":2132675555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2132675555},{"type":"v1","addr":"172.21.15.83:6807","nonce":2132675555}]},"public_addr":"172.21.15.83:6803/2132675555","cluster_addr":"172.21.15.83:6805/2132675555","heartbeat_back_addr":"172.21.15.83:6809/2132675555","heartbeat_front_addr":"172.21.15.83:6807/2132675555","state":["exists","up"]},{"osd":1,"uuid":"aba7f5df-2a4c-4c23-b502-d193605a5b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":3589158880},{"type":"v1","addr":"172.21.15.138:6801","nonce":3589158880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":3589158880},{"type":"v1","addr":"172.21.15.138:6803","nonce":3589158880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":3589158880},{"type":"v1","addr":"172.21.15.138:6807","nonce":3589158880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":3589158880},{"type":"v1","addr":"172.21.15.138:6805","nonce":3589158880}]},"public_addr":"172.21.15.138:6801/3589158880","cluster_addr":"172.21.15.138:6803/3589158880","heartbeat_back_addr":"172.21.15.138:6807/3589158880","heartbeat_front_addr":"172.21.15.138:6805/3589158880","state":["exists","up"]},{"osd":2,"uuid":"071bc167-2333-4420-bb58-bffbc10d7641","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":3169072688},{"type":"v1","addr":"172.21.15.189:6801","nonce":3169072688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":3169072688},{"type":"v1","addr":"172.21.15.189:6803","nonce":3169072688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":3169072688},{"type":"v1","addr":"172.21.15.189:6807","nonce":3169072688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":3169072688},{"type":"v1","addr":"172.21.15.189:6805","nonce":3169072688}]},"public_addr":"172.21.15.189:6801/3169072688","cluster_addr":"172.21.15.189:6803/3169072688","heartbeat_back_addr":"172.21.15.189:6807/3169072688","heartbeat_front_addr":"172.21.15.189:6805/3169072688","state":["exists","up"]},{"osd":3,"uuid":"f5624d78-4455-4e07-8c1a-f47c94cddb23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3131092450},{"type":"v1","addr":"172.21.15.83:6811","nonce":3131092450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3131092450},{"type":"v1","addr":"172.21.15.83:6813","nonce":3131092450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3131092450},{"type":"v1","addr":"172.21.15.83:6817","nonce":3131092450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3131092450},{"type":"v1","addr":"172.21.15.83:6815","nonce":3131092450}]},"public_addr":"172.21.15.83:6811/3131092450","cluster_addr":"172.21.15.83:6813/3131092450","heartbeat_back_addr":"172.21.15.83:6817/3131092450","heartbeat_front_addr":"172.21.15.83:6815/3131092450","state":["exists","up"]},{"osd":4,"uuid":"7b50cd14-9385-4535-a7b0-1c1da93c8548","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":823775360},{"type":"v1","addr":"172.21.15.138:6809","nonce":823775360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":823775360},{"type":"v1","addr":"172.21.15.138:6811","nonce":823775360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":823775360},{"type":"v1","addr":"172.21.15.138:6815","nonce":823775360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":823775360},{"type":"v1","addr":"172.21.15.138:6813","nonce":823775360}]},"public_addr":"172.21.15.138:6809/823775360","cluster_addr":"172.21.15.138:6811/823775360","heartbeat_back_addr":"172.21.15.138:6815/823775360","heartbeat_front_addr":"172.21.15.138:6813/823775360","state":["exists","up"]},{"osd":5,"uuid":"9c647797-35bc-449f-be84-af97d2c69c2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":1399817431},{"type":"v1","addr":"172.21.15.189:6809","nonce":1399817431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":1399817431},{"type":"v1","addr":"172.21.15.189:6811","nonce":1399817431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":1399817431},{"type":"v1","addr":"172.21.15.189:6815","nonce":1399817431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":1399817431},{"type":"v1","addr":"172.21.15.189:6813","nonce":1399817431}]},"public_addr":"172.21.15.189:6809/1399817431","cluster_addr":"172.21.15.189:6811/1399817431","heartbeat_back_addr":"172.21.15.189:6815/1399817431","heartbeat_front_addr":"172.21.15.189:6813/1399817431","state":["exists","up"]},{"osd":6,"uuid":"8f056584-16c8-4454-b445-3f780f26c196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":1611358483},{"type":"v1","addr":"172.21.15.83:6819","nonce":1611358483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1611358483},{"type":"v1","addr":"172.21.15.83:6821","nonce":1611358483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1611358483},{"type":"v1","addr":"172.21.15.83:6825","nonce":1611358483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":1611358483},{"type":"v1","addr":"172.21.15.83:6823","nonce":1611358483}]},"public_addr":"172.21.15.83:6819/1611358483","cluster_addr":"172.21.15.83:6821/1611358483","heartbeat_back_addr":"172.21.15.83:6825/1611358483","heartbeat_front_addr":"172.21.15.83:6823/1611358483","state":["exists","up"]},{"osd":7,"uuid":"7603efe3-da77-4b7f-9d03-29c4f293b6d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":3557856706},{"type":"v1","addr":"172.21.15.138:6817","nonce":3557856706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":3557856706},{"type":"v1","addr":"172.21.15.138:6819","nonce":3557856706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":3557856706},{"type":"v1","addr":"172.21.15.138:6823","nonce":3557856706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":3557856706},{"type":"v1","addr":"172.21.15.138:6821","nonce":3557856706}]},"public_addr":"172.21.15.138:6817/3557856706","cluster_addr":"172.21.15.138:6819/3557856706","heartbeat_back_addr":"172.21.15.138:6823/3557856706","heartbeat_front_addr":"172.21.15.138:6821/3557856706","state":["exists","up"]},{"osd":8,"uuid":"27d7d1b8-d6a0-417e-b8aa-596325aa1a77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":207018554},{"type":"v1","addr":"172.21.15.189:6817","nonce":207018554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":207018554},{"type":"v1","addr":"172.21.15.189:6819","nonce":207018554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":207018554},{"type":"v1","addr":"172.21.15.189:6823","nonce":207018554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":207018554},{"type":"v1","addr":"172.21.15.189:6821","nonce":207018554}]},"public_addr":"172.21.15.189:6817/207018554","cluster_addr":"172.21.15.189:6819/207018554","heartbeat_back_addr":"172.21.15.189:6823/207018554","heartbeat_front_addr":"172.21.15.189:6821/207018554","state":["exists","up"]},{"osd":9,"uuid":"baf19b98-881b-4d3a-9f72-8aa2dd73cad6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":519636751},{"type":"v1","addr":"172.21.15.83:6827","nonce":519636751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":519636751},{"type":"v1","addr":"172.21.15.83:6829","nonce":519636751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":519636751},{"type":"v1","addr":"172.21.15.83:6833","nonce":519636751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":519636751},{"type":"v1","addr":"172.21.15.83:6831","nonce":519636751}]},"public_addr":"172.21.15.83:6827/519636751","cluster_addr":"172.21.15.83:6829/519636751","heartbeat_back_addr":"172.21.15.83:6833/519636751","heartbeat_front_addr":"172.21.15.83:6831/519636751","state":["exists","up"]},{"osd":10,"uuid":"eb346717-5c07-47c8-95be-0e68fe449fdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1130872098},{"type":"v1","addr":"172.21.15.138:6825","nonce":1130872098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1130872098},{"type":"v1","addr":"172.21.15.138:6827","nonce":1130872098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1130872098},{"type":"v1","addr":"172.21.15.138:6831","nonce":1130872098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1130872098},{"type":"v1","addr":"172.21.15.138:6829","nonce":1130872098}]},"public_addr":"172.21.15.138:6825/1130872098","cluster_addr":"172.21.15.138:6827/1130872098","heartbeat_back_addr":"172.21.15.138:6831/1130872098","heartbeat_front_addr":"172.21.15.138:6829/1130872098","state":["exists","up"]},{"osd":11,"uuid":"8e37568e-981c-45b8-b2af-2f1b73898542","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":2998782690},{"type":"v1","addr":"172.21.15.189:6825","nonce":2998782690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":2998782690},{"type":"v1","addr":"172.21.15.189:6827","nonce":2998782690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":2998782690},{"type":"v1","addr":"172.21.15.189:6831","nonce":2998782690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":2998782690},{"type":"v1","addr":"172.21.15.189:6829","nonce":2998782690}]},"public_addr":"172.21.15.189:6825/2998782690","cluster_addr":"172.21.15.189:6827/2998782690","heartbeat_back_addr":"172.21.15.189:6831/2998782690","heartbeat_front_addr":"172.21.15.189:6829/2998782690","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:40:47.980103+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:41:21.970795+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:41:55.160426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:42:30.400475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:43:05.705663+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:43:40.112312+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:44:16.781289+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:44:52.151187+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:45:28.055944+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:46:04.969301+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:46:40.918306+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:47:16.886874+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"2.1d","mappings":[{"from":2,"to":1}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.2a","mappings":[{"from":3,"to":8}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.a","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:0/1096290901":"2024-04-25T08:51:36.347763+0000","172.21.15.83:6800/2550826560":"2024-04-25T08:51:36.347763+0000","172.21.15.83:0/1743153945":"2024-04-25T08:51:36.347763+0000","172.21.15.83:0/3306681366":"2024-04-25T08:51:36.347763+0000","172.21.15.83:6801/820049883":"2024-04-25T08:38:40.661315+0000","172.21.15.83:6801/2550826560":"2024-04-25T08:51:36.347763+0000","172.21.15.83:0/4272874138":"2024-04-25T08:38:40.661315+0000","172.21.15.83:0/4212994335":"2024-04-25T08:38:40.661315+0000","172.21.15.83:6801/1996371091":"2024-04-25T08:38:05.800856+0000","172.21.15.83:0/1820663683":"2024-04-25T08:38:05.800856+0000","172.21.15.83:0/3802897262":"2024-04-25T08:38:05.800856+0000","172.21.15.83:6800/1996371091":"2024-04-25T08:38:05.800856+0000","172.21.15.83:6800/820049883":"2024-04-25T08:38:40.661315+0000","172.21.15.83:0/1366125049":"2024-04-25T08:38:05.800856+0000","172.21.15.83:0/4159820288":"2024-04-25T08:51:36.347763+0000","172.21.15.83:0/340452868":"2024-04-25T08:38:40.661315+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":35,"length":1},{"begin":48,"length":1}]},{"pool":4,"snaps":[{"begin":35,"length":1},{"begin":48,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T09:18:53.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:53.872+0000 7fd30e200640 1 -- 172.21.15.83:0/2334669435 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fd2ec0a3d20 msgr2=0x7fd2ec0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:53.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:53.872+0000 7fd30e200640 1 --2- 172.21.15.83:0/2334669435 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fd2ec0a3d20 0x7fd2ec0a61e0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fd2f0005fd0 tx=0x7fd2f0005850 comp rx=0 tx=0).stop 2024-04-24T09:18:53.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:53.872+0000 7fd30e200640 1 -- 172.21.15.83:0/2334669435 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3081084a0 msgr2=0x7fd3081a4800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:53.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:53.872+0000 7fd30e200640 1 --2- 172.21.15.83:0/2334669435 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3081084a0 0x7fd3081a4800 secure :-1 s=READY pgs=2908 cs=0 l=1 rev1=1 crypto rx=0x7fd2f800c890 tx=0x7fd2f800cd50 comp rx=0 tx=0).stop 2024-04-24T09:18:53.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:53.872+0000 7fd30e200640 1 -- 172.21.15.83:0/2334669435 shutdown_connections 2024-04-24T09:18:53.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:53.872+0000 7fd30e200640 1 --2- 172.21.15.83:0/2334669435 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fd2ec0a3d20 0x7fd2ec0a61e0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:53.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:53.872+0000 7fd30e200640 1 --2- 172.21.15.83:0/2334669435 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd30810d3c0 0x7fd30819e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:53.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:53.872+0000 7fd30e200640 1 --2- 172.21.15.83:0/2334669435 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd308108e40 0x7fd3081a4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:53.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:53.872+0000 7fd30e200640 1 --2- 172.21.15.83:0/2334669435 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3081084a0 0x7fd3081a4800 unknown :-1 s=CLOSED pgs=2908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:53.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:53.872+0000 7fd30e200640 1 -- 172.21.15.83:0/2334669435 >> 172.21.15.83:0/2334669435 conn(0x7fd308077b80 msgr2=0x7fd308078390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:18:53.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:53.872+0000 7fd30e200640 1 -- 172.21.15.83:0/2334669435 shutdown_connections 2024-04-24T09:18:53.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:53.872+0000 7fd30e200640 1 -- 172.21.15.83:0/2334669435 wait complete. 2024-04-24T09:18:53.892 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T08:41:56.834372+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T08:48:26.978311+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T08:48:28.352731+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '135', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 48, 'snap_epoch': 135, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-24T08:48:32.275167+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '135', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 48, 'snap_epoch': 135, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-24T09:18:53.892 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T09:18:54.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.107+0000 7fb2b6e00640 1 -- 172.21.15.83:0/4172027465 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb2b0104e60 msgr2=0x7fb2b0108ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:54.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.107+0000 7fb2b6e00640 1 --2- 172.21.15.83:0/4172027465 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb2b0104e60 0x7fb2b0108ea0 secure :-1 s=READY pgs=2909 cs=0 l=1 rev1=1 crypto rx=0x7fb29c009900 tx=0x7fb29c02a410 comp rx=0 tx=0).stop 2024-04-24T09:18:54.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.107+0000 7fb2b6e00640 1 -- 172.21.15.83:0/4172027465 shutdown_connections 2024-04-24T09:18:54.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.107+0000 7fb2b6e00640 1 --2- 172.21.15.83:0/4172027465 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2b0109580 0x7fb2b0111150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:54.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.107+0000 7fb2b6e00640 1 --2- 172.21.15.83:0/4172027465 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb2b0104e60 0x7fb2b0108ea0 unknown :-1 s=CLOSED pgs=2909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:54.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.107+0000 7fb2b6e00640 1 --2- 172.21.15.83:0/4172027465 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb2b01044b0 0x7fb2b0104890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:54.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.107+0000 7fb2b6e00640 1 -- 172.21.15.83:0/4172027465 >> 172.21.15.83:0/4172027465 conn(0x7fb2b00ffd90 msgr2=0x7fb2b01021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:18:54.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.107+0000 7fb2b6e00640 1 -- 172.21.15.83:0/4172027465 shutdown_connections 2024-04-24T09:18:54.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.107+0000 7fb2b6e00640 1 -- 172.21.15.83:0/4172027465 wait complete. 2024-04-24T09:18:54.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.108+0000 7fb2b6e00640 1 Processor -- start 2024-04-24T09:18:54.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.108+0000 7fb2b6e00640 1 -- start start 2024-04-24T09:18:54.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.108+0000 7fb2b6e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2b01044b0 0x7fb2b01a0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:54.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.109+0000 7fb2b6e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb2b0109580 0x7fb2b01a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:54.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.109+0000 7fb2b6e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb2b019a8e0 0x7fb2b019ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:54.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.109+0000 7fb2b6e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b01144a0 con 0x7fb2b0109580 2024-04-24T09:18:54.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.109+0000 7fb2b6e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b0114320 con 0x7fb2b01044b0 2024-04-24T09:18:54.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.109+0000 7fb2b6e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b0114620 con 0x7fb2b019a8e0 2024-04-24T09:18:54.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.109+0000 7fb2af400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb2b0109580 0x7fb2b01a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:54.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.109+0000 7fb2af400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb2b0109580 0x7fb2b01a0c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38142/0 (socket says 172.21.15.83:38142) 2024-04-24T09:18:54.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.109+0000 7fb2af400640 1 -- 172.21.15.83:0/1210693049 learned_addr learned my addr 172.21.15.83:0/1210693049 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:18:54.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.109+0000 7fb2afe00640 1 --2- 172.21.15.83:0/1210693049 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2b01044b0 0x7fb2b01a0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:54.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.109+0000 7fb2b4e00640 1 --2- 172.21.15.83:0/1210693049 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb2b019a8e0 0x7fb2b019ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:54.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.109+0000 7fb2af400640 1 -- 172.21.15.83:0/1210693049 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb2b019a8e0 msgr2=0x7fb2b019ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:54.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.109+0000 7fb2af400640 1 --2- 172.21.15.83:0/1210693049 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb2b019a8e0 0x7fb2b019ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:54.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.109+0000 7fb2af400640 1 -- 172.21.15.83:0/1210693049 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2b01044b0 msgr2=0x7fb2b01a0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:54.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.109+0000 7fb2af400640 1 --2- 172.21.15.83:0/1210693049 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2b01044b0 0x7fb2b01a0730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:54.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.109+0000 7fb2af400640 1 -- 172.21.15.83:0/1210693049 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2b019b400 con 0x7fb2b0109580 2024-04-24T09:18:54.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.110+0000 7fb2af400640 1 --2- 172.21.15.83:0/1210693049 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb2b0109580 0x7fb2b01a0c70 secure :-1 s=READY pgs=2910 cs=0 l=1 rev1=1 crypto rx=0x7fb29c004270 tx=0x7fb29c0042a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:18:54.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.110+0000 7fb2acc00640 1 -- 172.21.15.83:0/1210693049 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb29c03f030 con 0x7fb2b0109580 2024-04-24T09:18:54.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.110+0000 7fb2acc00640 1 -- 172.21.15.83:0/1210693049 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb29c0044c0 con 0x7fb2b0109580 2024-04-24T09:18:54.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.110+0000 7fb2b6e00640 1 -- 172.21.15.83:0/1210693049 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2b019b690 con 0x7fb2b0109580 2024-04-24T09:18:54.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.110+0000 7fb2acc00640 1 -- 172.21.15.83:0/1210693049 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb29c035070 con 0x7fb2b0109580 2024-04-24T09:18:54.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.110+0000 7fb2b6e00640 1 -- 172.21.15.83:0/1210693049 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2b01a7880 con 0x7fb2b0109580 2024-04-24T09:18:54.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.111+0000 7fb2b6e00640 1 -- 172.21.15.83:0/1210693049 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2b0069270 con 0x7fb2b0109580 2024-04-24T09:18:54.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.113+0000 7fb2acc00640 1 -- 172.21.15.83:0/1210693049 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb29c042030 con 0x7fb2b0109580 2024-04-24T09:18:54.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.114+0000 7fb2acc00640 1 --2- 172.21.15.83:0/1210693049 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb2940a3d20 0x7fb2940a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:54.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.115+0000 7fb2acc00640 1 -- 172.21.15.83:0/1210693049 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 9583+0+0 (secure 0 0 0) 0x7fb29c0f1210 con 0x7fb2b0109580 2024-04-24T09:18:54.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.115+0000 7fb2afe00640 1 --2- 172.21.15.83:0/1210693049 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb2940a3d20 0x7fb2940a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:54.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.116+0000 7fb2afe00640 1 --2- 172.21.15.83:0/1210693049 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb2940a3d20 0x7fb2940a61e0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fb2a0005fd0 tx=0x7fb2a0005850 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:18:54.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.119+0000 7fb2acc00640 1 -- 172.21.15.83:0/1210693049 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb29c0bd430 con 0x7fb2b0109580 2024-04-24T09:18:54.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.326+0000 7fb2b6e00640 1 -- 172.21.15.83:0/1210693049 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb2b0104e60 con 0x7fb2b0109580 2024-04-24T09:18:54.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.328+0000 7fb2acc00640 1 -- 172.21.15.83:0/1210693049 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v136) ==== 93+0+10 (secure 0 0 0) 0x7fb29c0bcd30 con 0x7fb2b0109580 2024-04-24T09:18:54.328 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-24T09:18:54.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.330+0000 7fb2b6e00640 1 -- 172.21.15.83:0/1210693049 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb2940a3d20 msgr2=0x7fb2940a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:54.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.330+0000 7fb2b6e00640 1 --2- 172.21.15.83:0/1210693049 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb2940a3d20 0x7fb2940a61e0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fb2a0005fd0 tx=0x7fb2a0005850 comp rx=0 tx=0).stop 2024-04-24T09:18:54.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.330+0000 7fb2b6e00640 1 -- 172.21.15.83:0/1210693049 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb2b0109580 msgr2=0x7fb2b01a0c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:54.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.330+0000 7fb2b6e00640 1 --2- 172.21.15.83:0/1210693049 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb2b0109580 0x7fb2b01a0c70 secure :-1 s=READY pgs=2910 cs=0 l=1 rev1=1 crypto rx=0x7fb29c004270 tx=0x7fb29c0042a0 comp rx=0 tx=0).stop 2024-04-24T09:18:54.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.330+0000 7fb2b6e00640 1 -- 172.21.15.83:0/1210693049 shutdown_connections 2024-04-24T09:18:54.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.330+0000 7fb2b6e00640 1 --2- 172.21.15.83:0/1210693049 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb2940a3d20 0x7fb2940a61e0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:54.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.330+0000 7fb2b6e00640 1 --2- 172.21.15.83:0/1210693049 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb2b019a8e0 0x7fb2b019ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:54.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.330+0000 7fb2b6e00640 1 --2- 172.21.15.83:0/1210693049 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb2b0109580 0x7fb2b01a0c70 unknown :-1 s=CLOSED pgs=2910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:54.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.330+0000 7fb2b6e00640 1 --2- 172.21.15.83:0/1210693049 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2b01044b0 0x7fb2b01a0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:54.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.330+0000 7fb2b6e00640 1 -- 172.21.15.83:0/1210693049 >> 172.21.15.83:0/1210693049 conn(0x7fb2b00ffd90 msgr2=0x7fb2b0101230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:18:54.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.330+0000 7fb2b6e00640 1 -- 172.21.15.83:0/1210693049 shutdown_connections 2024-04-24T09:18:54.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.331+0000 7fb2b6e00640 1 -- 172.21.15.83:0/1210693049 wait complete. 2024-04-24T09:18:54.350 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T09:18:54.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:18:54 smithi083 ceph-mon[27907]: pgmap v863: 129 pgs: 129 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-24T09:18:54.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:18:54 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/2334669435' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:18:54.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.555+0000 7fe0ece00640 1 -- 172.21.15.83:0/3078953685 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0e8108fd0 msgr2=0x7fe0e8115490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:54.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.555+0000 7fe0ece00640 1 --2- 172.21.15.83:0/3078953685 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0e8108fd0 0x7fe0e8115490 secure :-1 s=READY pgs=2911 cs=0 l=1 rev1=1 crypto rx=0x7fe0e0009930 tx=0x7fe0e002a420 comp rx=0 tx=0).stop 2024-04-24T09:18:54.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.555+0000 7fe0ece00640 1 -- 172.21.15.83:0/3078953685 shutdown_connections 2024-04-24T09:18:54.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.555+0000 7fe0ece00640 1 --2- 172.21.15.83:0/3078953685 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0e8108fd0 0x7fe0e8115490 unknown :-1 s=CLOSED pgs=2911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:54.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.555+0000 7fe0ece00640 1 --2- 172.21.15.83:0/3078953685 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe0e8108610 0x7fe0e8108a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:54.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.555+0000 7fe0ece00640 1 --2- 172.21.15.83:0/3078953685 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0e810e660 0x7fe0e810ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:54.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.555+0000 7fe0ece00640 1 -- 172.21.15.83:0/3078953685 >> 172.21.15.83:0/3078953685 conn(0x7fe0e8076540 msgr2=0x7fe0e8078960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:18:54.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.555+0000 7fe0ece00640 1 -- 172.21.15.83:0/3078953685 shutdown_connections 2024-04-24T09:18:54.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.555+0000 7fe0ece00640 1 -- 172.21.15.83:0/3078953685 wait complete. 2024-04-24T09:18:54.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.556+0000 7fe0ece00640 1 Processor -- start 2024-04-24T09:18:54.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.556+0000 7fe0ece00640 1 -- start start 2024-04-24T09:18:54.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.556+0000 7fe0ece00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe0e8108610 0x7fe0e819edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:54.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.556+0000 7fe0ece00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0e8108fd0 0x7fe0e819f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:54.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.556+0000 7fe0ece00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0e810e660 0x7fe0e81a3640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:54.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.556+0000 7fe0ece00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe0e8118890 con 0x7fe0e810e660 2024-04-24T09:18:54.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.556+0000 7fe0ece00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe0e8118710 con 0x7fe0e8108fd0 2024-04-24T09:18:54.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.556+0000 7fe0ece00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe0e8118a10 con 0x7fe0e8108610 2024-04-24T09:18:54.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.557+0000 7fe0e6800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0e810e660 0x7fe0e81a3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:54.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.557+0000 7fe0e6800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0e810e660 0x7fe0e81a3640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38156/0 (socket says 172.21.15.83:38156) 2024-04-24T09:18:54.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.557+0000 7fe0e6800640 1 -- 172.21.15.83:0/4056108253 learned_addr learned my addr 172.21.15.83:0/4056108253 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:18:54.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.557+0000 7fe0e5e00640 1 --2- 172.21.15.83:0/4056108253 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe0e8108610 0x7fe0e819edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:54.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.557+0000 7fe0dd400640 1 --2- 172.21.15.83:0/4056108253 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0e8108fd0 0x7fe0e819f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:54.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.557+0000 7fe0e6800640 1 -- 172.21.15.83:0/4056108253 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe0e8108610 msgr2=0x7fe0e819edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:54.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.557+0000 7fe0e6800640 1 --2- 172.21.15.83:0/4056108253 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe0e8108610 0x7fe0e819edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:54.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.557+0000 7fe0e6800640 1 -- 172.21.15.83:0/4056108253 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0e8108fd0 msgr2=0x7fe0e819f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:54.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.557+0000 7fe0e6800640 1 --2- 172.21.15.83:0/4056108253 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0e8108fd0 0x7fe0e819f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:54.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.557+0000 7fe0e6800640 1 -- 172.21.15.83:0/4056108253 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0e81a3d70 con 0x7fe0e810e660 2024-04-24T09:18:54.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.557+0000 7fe0dd400640 1 --2- 172.21.15.83:0/4056108253 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0e8108fd0 0x7fe0e819f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:18:54.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.557+0000 7fe0e5e00640 1 --2- 172.21.15.83:0/4056108253 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe0e8108610 0x7fe0e819edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:18:54.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.558+0000 7fe0e6800640 1 --2- 172.21.15.83:0/4056108253 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0e810e660 0x7fe0e81a3640 secure :-1 s=READY pgs=2912 cs=0 l=1 rev1=1 crypto rx=0x7fe0e0005940 tx=0x7fe0e0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:18:54.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.558+0000 7fe0df400640 1 -- 172.21.15.83:0/4056108253 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e003f030 con 0x7fe0e810e660 2024-04-24T09:18:54.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.558+0000 7fe0df400640 1 -- 172.21.15.83:0/4056108253 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe0e0003aa0 con 0x7fe0e810e660 2024-04-24T09:18:54.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.558+0000 7fe0ece00640 1 -- 172.21.15.83:0/4056108253 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0e81a4000 con 0x7fe0e810e660 2024-04-24T09:18:54.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.558+0000 7fe0df400640 1 -- 172.21.15.83:0/4056108253 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e0035070 con 0x7fe0e810e660 2024-04-24T09:18:54.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.558+0000 7fe0ece00640 1 -- 172.21.15.83:0/4056108253 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0e81ab8f0 con 0x7fe0e810e660 2024-04-24T09:18:54.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.559+0000 7fe0ece00640 1 -- 172.21.15.83:0/4056108253 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0e8069270 con 0x7fe0e810e660 2024-04-24T09:18:54.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.561+0000 7fe0df400640 1 -- 172.21.15.83:0/4056108253 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe0e0042030 con 0x7fe0e810e660 2024-04-24T09:18:54.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.563+0000 7fe0df400640 1 --2- 172.21.15.83:0/4056108253 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe0cc0a3da0 0x7fe0cc0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:54.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.563+0000 7fe0df400640 1 -- 172.21.15.83:0/4056108253 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 9583+0+0 (secure 0 0 0) 0x7fe0e00f13f0 con 0x7fe0e810e660 2024-04-24T09:18:54.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.563+0000 7fe0e5e00640 1 --2- 172.21.15.83:0/4056108253 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe0cc0a3da0 0x7fe0cc0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:54.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.564+0000 7fe0e5e00640 1 --2- 172.21.15.83:0/4056108253 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe0cc0a3da0 0x7fe0cc0a6260 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fe0e80792c0 tx=0x7fe0d80094f0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:18:54.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.567+0000 7fe0df400640 1 -- 172.21.15.83:0/4056108253 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe0e00bd610 con 0x7fe0e810e660 2024-04-24T09:18:54.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:18:54 smithi138 ceph-mon[31441]: pgmap v863: 129 pgs: 129 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-24T09:18:54.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:18:54 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/2334669435' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:18:54.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:18:54 smithi189 ceph-mon[31791]: pgmap v863: 129 pgs: 129 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-24T09:18:54.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:18:54 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/2334669435' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:18:54.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.770+0000 7fe0ece00640 1 -- 172.21.15.83:0/4056108253 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe0e8110bb0 con 0x7fe0e810e660 2024-04-24T09:18:54.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.771+0000 7fe0df400640 1 -- 172.21.15.83:0/4056108253 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v136) ==== 104+0+11 (secure 0 0 0) 0x7fe0e00bcf10 con 0x7fe0e810e660 2024-04-24T09:18:54.772 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-24T09:18:54.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.774+0000 7fe0ece00640 1 -- 172.21.15.83:0/4056108253 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe0cc0a3da0 msgr2=0x7fe0cc0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:54.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.774+0000 7fe0ece00640 1 --2- 172.21.15.83:0/4056108253 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe0cc0a3da0 0x7fe0cc0a6260 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fe0e80792c0 tx=0x7fe0d80094f0 comp rx=0 tx=0).stop 2024-04-24T09:18:54.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.774+0000 7fe0ece00640 1 -- 172.21.15.83:0/4056108253 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0e810e660 msgr2=0x7fe0e81a3640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:54.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.774+0000 7fe0ece00640 1 --2- 172.21.15.83:0/4056108253 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0e810e660 0x7fe0e81a3640 secure :-1 s=READY pgs=2912 cs=0 l=1 rev1=1 crypto rx=0x7fe0e0005940 tx=0x7fe0e0004270 comp rx=0 tx=0).stop 2024-04-24T09:18:54.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.774+0000 7fe0ece00640 1 -- 172.21.15.83:0/4056108253 shutdown_connections 2024-04-24T09:18:54.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.774+0000 7fe0ece00640 1 --2- 172.21.15.83:0/4056108253 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe0cc0a3da0 0x7fe0cc0a6260 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:54.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.774+0000 7fe0ece00640 1 --2- 172.21.15.83:0/4056108253 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0e810e660 0x7fe0e81a3640 unknown :-1 s=CLOSED pgs=2912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:54.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.774+0000 7fe0ece00640 1 --2- 172.21.15.83:0/4056108253 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0e8108fd0 0x7fe0e819f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:54.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.774+0000 7fe0ece00640 1 --2- 172.21.15.83:0/4056108253 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe0e8108610 0x7fe0e819edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:54.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.774+0000 7fe0ece00640 1 -- 172.21.15.83:0/4056108253 >> 172.21.15.83:0/4056108253 conn(0x7fe0e8076540 msgr2=0x7fe0e8077d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:18:54.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.774+0000 7fe0ece00640 1 -- 172.21.15.83:0/4056108253 shutdown_connections 2024-04-24T09:18:54.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.774+0000 7fe0ece00640 1 -- 172.21.15.83:0/4056108253 wait complete. 2024-04-24T09:18:54.794 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T09:18:54.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.998+0000 7fca56400640 1 -- 172.21.15.83:0/147507688 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fca50074430 msgr2=0x7fca50074810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:54.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.998+0000 7fca56400640 1 --2- 172.21.15.83:0/147507688 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fca50074430 0x7fca50074810 secure :-1 s=READY pgs=2913 cs=0 l=1 rev1=1 crypto rx=0x7fca40009900 tx=0x7fca4002a480 comp rx=0 tx=0).stop 2024-04-24T09:18:54.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.998+0000 7fca56400640 1 -- 172.21.15.83:0/147507688 shutdown_connections 2024-04-24T09:18:54.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.998+0000 7fca56400640 1 --2- 172.21.15.83:0/147507688 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fca5010e6e0 0x7fca50110b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:54.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.998+0000 7fca56400640 1 --2- 172.21.15.83:0/147507688 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fca50074d50 0x7fca5010e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:54.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.998+0000 7fca56400640 1 --2- 172.21.15.83:0/147507688 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fca50074430 0x7fca50074810 unknown :-1 s=CLOSED pgs=2913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:54.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.998+0000 7fca56400640 1 -- 172.21.15.83:0/147507688 >> 172.21.15.83:0/147507688 conn(0x7fca500fd9d0 msgr2=0x7fca500ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:18:54.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.998+0000 7fca56400640 1 -- 172.21.15.83:0/147507688 shutdown_connections 2024-04-24T09:18:54.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.998+0000 7fca56400640 1 -- 172.21.15.83:0/147507688 wait complete. 2024-04-24T09:18:55.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.999+0000 7fca56400640 1 Processor -- start 2024-04-24T09:18:55.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.999+0000 7fca56400640 1 -- start start 2024-04-24T09:18:55.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.999+0000 7fca56400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fca50074430 0x7fca501a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:55.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.999+0000 7fca56400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fca50074d50 0x7fca501a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:55.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.999+0000 7fca56400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fca5010e6e0 0x7fca5019a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:55.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.999+0000 7fca56400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fca50114370 con 0x7fca50074d50 2024-04-24T09:18:55.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.999+0000 7fca56400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fca501141f0 con 0x7fca5010e6e0 2024-04-24T09:18:55.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:54.999+0000 7fca56400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fca501144f0 con 0x7fca50074430 2024-04-24T09:18:55.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.000+0000 7fca4fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fca50074d50 0x7fca501a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:55.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.000+0000 7fca4fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fca50074d50 0x7fca501a0b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38164/0 (socket says 172.21.15.83:38164) 2024-04-24T09:18:55.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.000+0000 7fca4fe00640 1 -- 172.21.15.83:0/3304998655 learned_addr learned my addr 172.21.15.83:0/3304998655 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:18:55.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.000+0000 7fca55000640 1 --2- 172.21.15.83:0/3304998655 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fca50074430 0x7fca501a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:55.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.000+0000 7fca55a00640 1 --2- 172.21.15.83:0/3304998655 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fca5010e6e0 0x7fca5019a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:55.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.000+0000 7fca4fe00640 1 -- 172.21.15.83:0/3304998655 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fca50074430 msgr2=0x7fca501a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:55.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.000+0000 7fca4fe00640 1 --2- 172.21.15.83:0/3304998655 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fca50074430 0x7fca501a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:55.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.000+0000 7fca4fe00640 1 -- 172.21.15.83:0/3304998655 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fca5010e6e0 msgr2=0x7fca5019a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:55.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.000+0000 7fca4fe00640 1 --2- 172.21.15.83:0/3304998655 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fca5010e6e0 0x7fca5019a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:55.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.000+0000 7fca4fe00640 1 -- 172.21.15.83:0/3304998655 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca5019af60 con 0x7fca50074d50 2024-04-24T09:18:55.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.000+0000 7fca55a00640 1 --2- 172.21.15.83:0/3304998655 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fca5010e6e0 0x7fca5019a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:18:55.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.001+0000 7fca4fe00640 1 --2- 172.21.15.83:0/3304998655 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fca50074d50 0x7fca501a0b60 secure :-1 s=READY pgs=2914 cs=0 l=1 rev1=1 crypto rx=0x7fca3800d770 tx=0x7fca3800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:18:55.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.001+0000 7fca4d600640 1 -- 172.21.15.83:0/3304998655 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca38017070 con 0x7fca50074d50 2024-04-24T09:18:55.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.001+0000 7fca4d600640 1 -- 172.21.15.83:0/3304998655 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fca380054d0 con 0x7fca50074d50 2024-04-24T09:18:55.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.001+0000 7fca4d600640 1 -- 172.21.15.83:0/3304998655 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca38004cf0 con 0x7fca50074d50 2024-04-24T09:18:55.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.001+0000 7fca56400640 1 -- 172.21.15.83:0/3304998655 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca5019b250 con 0x7fca50074d50 2024-04-24T09:18:55.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.001+0000 7fca56400640 1 -- 172.21.15.83:0/3304998655 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca501a7470 con 0x7fca50074d50 2024-04-24T09:18:55.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.002+0000 7fca56400640 1 -- 172.21.15.83:0/3304998655 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca50075ae0 con 0x7fca50074d50 2024-04-24T09:18:55.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.004+0000 7fca4d600640 1 -- 172.21.15.83:0/3304998655 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fca38004e90 con 0x7fca50074d50 2024-04-24T09:18:55.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.006+0000 7fca4d600640 1 --2- 172.21.15.83:0/3304998655 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fca340a3da0 0x7fca340a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:55.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.006+0000 7fca4d600640 1 -- 172.21.15.83:0/3304998655 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 9583+0+0 (secure 0 0 0) 0x7fca380d0730 con 0x7fca50074d50 2024-04-24T09:18:55.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.006+0000 7fca55000640 1 --2- 172.21.15.83:0/3304998655 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fca340a3da0 0x7fca340a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:55.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.007+0000 7fca55000640 1 --2- 172.21.15.83:0/3304998655 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fca340a3da0 0x7fca340a6260 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fca40000c00 tx=0x7fca40031400 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:18:55.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.011+0000 7fca4d600640 1 -- 172.21.15.83:0/3304998655 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fca3809c950 con 0x7fca50074d50 2024-04-24T09:18:55.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.212+0000 7fca56400640 1 -- 172.21.15.83:0/3304998655 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fca501098b0 con 0x7fca50074d50 2024-04-24T09:18:55.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.213+0000 7fca4d600640 1 -- 172.21.15.83:0/3304998655 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v136) ==== 100+0+11 (secure 0 0 0) 0x7fca501098b0 con 0x7fca50074d50 2024-04-24T09:18:55.213 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2024-04-24T09:18:55.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.215+0000 7fca56400640 1 -- 172.21.15.83:0/3304998655 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fca340a3da0 msgr2=0x7fca340a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:55.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.215+0000 7fca56400640 1 --2- 172.21.15.83:0/3304998655 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fca340a3da0 0x7fca340a6260 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fca40000c00 tx=0x7fca40031400 comp rx=0 tx=0).stop 2024-04-24T09:18:55.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.215+0000 7fca56400640 1 -- 172.21.15.83:0/3304998655 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fca50074d50 msgr2=0x7fca501a0b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:55.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.215+0000 7fca56400640 1 --2- 172.21.15.83:0/3304998655 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fca50074d50 0x7fca501a0b60 secure :-1 s=READY pgs=2914 cs=0 l=1 rev1=1 crypto rx=0x7fca3800d770 tx=0x7fca3800dc30 comp rx=0 tx=0).stop 2024-04-24T09:18:55.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.215+0000 7fca56400640 1 -- 172.21.15.83:0/3304998655 shutdown_connections 2024-04-24T09:18:55.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.215+0000 7fca56400640 1 --2- 172.21.15.83:0/3304998655 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fca340a3da0 0x7fca340a6260 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:55.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.215+0000 7fca56400640 1 --2- 172.21.15.83:0/3304998655 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fca5010e6e0 0x7fca5019a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:55.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.215+0000 7fca56400640 1 --2- 172.21.15.83:0/3304998655 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fca50074d50 0x7fca501a0b60 unknown :-1 s=CLOSED pgs=2914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:55.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.215+0000 7fca56400640 1 --2- 172.21.15.83:0/3304998655 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fca50074430 0x7fca501a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:55.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.215+0000 7fca56400640 1 -- 172.21.15.83:0/3304998655 >> 172.21.15.83:0/3304998655 conn(0x7fca500fd9d0 msgr2=0x7fca500fe880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:18:55.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.216+0000 7fca56400640 1 -- 172.21.15.83:0/3304998655 shutdown_connections 2024-04-24T09:18:55.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.216+0000 7fca56400640 1 -- 172.21.15.83:0/3304998655 wait complete. 2024-04-24T09:18:55.235 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-24T09:18:55.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.432+0000 7f9e15c00640 1 -- 172.21.15.83:0/3176886673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9e10104e00 msgr2=0x7f9e10108e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:55.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.432+0000 7f9e15c00640 1 --2- 172.21.15.83:0/3176886673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9e10104e00 0x7f9e10108e40 secure :-1 s=READY pgs=2915 cs=0 l=1 rev1=1 crypto rx=0x7f9df8009900 tx=0x7f9df802a400 comp rx=0 tx=0).stop 2024-04-24T09:18:55.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.432+0000 7f9e15c00640 1 -- 172.21.15.83:0/3176886673 shutdown_connections 2024-04-24T09:18:55.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.432+0000 7f9e15c00640 1 --2- 172.21.15.83:0/3176886673 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9e10109520 0x7f9e101110f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:55.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.432+0000 7f9e15c00640 1 --2- 172.21.15.83:0/3176886673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9e10104e00 0x7f9e10108e40 unknown :-1 s=CLOSED pgs=2915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:55.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.432+0000 7f9e15c00640 1 --2- 172.21.15.83:0/3176886673 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9e10104450 0x7f9e10104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:55.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.432+0000 7f9e15c00640 1 -- 172.21.15.83:0/3176886673 >> 172.21.15.83:0/3176886673 conn(0x7f9e100ffc70 msgr2=0x7f9e10102090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:18:55.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.432+0000 7f9e15c00640 1 -- 172.21.15.83:0/3176886673 shutdown_connections 2024-04-24T09:18:55.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.433+0000 7f9e15c00640 1 -- 172.21.15.83:0/3176886673 wait complete. 2024-04-24T09:18:55.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.433+0000 7f9e15c00640 1 Processor -- start 2024-04-24T09:18:55.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.434+0000 7f9e15c00640 1 -- start start 2024-04-24T09:18:55.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.434+0000 7f9e15c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9e10104450 0x7f9e101a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:55.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.434+0000 7f9e15c00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9e10109520 0x7f9e101a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:55.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.434+0000 7f9e15c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9e1019a7e0 0x7f9e1019ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:55.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.434+0000 7f9e15c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9e10114460 con 0x7f9e1019a7e0 2024-04-24T09:18:55.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.434+0000 7f9e15c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9e101142e0 con 0x7f9e10104450 2024-04-24T09:18:55.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.434+0000 7f9e15c00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9e101145e0 con 0x7f9e10109520 2024-04-24T09:18:55.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.434+0000 7f9e0f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9e1019a7e0 0x7f9e1019ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:55.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.434+0000 7f9e0f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9e1019a7e0 0x7f9e1019ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38190/0 (socket says 172.21.15.83:38190) 2024-04-24T09:18:55.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.434+0000 7f9e0f400640 1 -- 172.21.15.83:0/1707662534 learned_addr learned my addr 172.21.15.83:0/1707662534 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:18:55.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.435+0000 7f9e0ea00640 1 --2- 172.21.15.83:0/1707662534 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9e10104450 0x7f9e101a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:55.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.435+0000 7f9e06000640 1 --2- 172.21.15.83:0/1707662534 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9e10109520 0x7f9e101a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:55.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.435+0000 7f9e0f400640 1 -- 172.21.15.83:0/1707662534 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9e10109520 msgr2=0x7f9e101a0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:55.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.435+0000 7f9e0f400640 1 --2- 172.21.15.83:0/1707662534 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9e10109520 0x7f9e101a0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:55.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.435+0000 7f9e0f400640 1 -- 172.21.15.83:0/1707662534 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9e10104450 msgr2=0x7f9e101a0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:55.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.435+0000 7f9e0f400640 1 --2- 172.21.15.83:0/1707662534 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9e10104450 0x7f9e101a0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:55.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.435+0000 7f9e0f400640 1 -- 172.21.15.83:0/1707662534 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e1019b370 con 0x7f9e1019a7e0 2024-04-24T09:18:55.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.435+0000 7f9e0ea00640 1 --2- 172.21.15.83:0/1707662534 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9e10104450 0x7f9e101a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:18:55.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.435+0000 7f9e0f400640 1 --2- 172.21.15.83:0/1707662534 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9e1019a7e0 0x7f9e1019ac90 secure :-1 s=READY pgs=2916 cs=0 l=1 rev1=1 crypto rx=0x7f9e0800cad0 tx=0x7f9e0800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:18:55.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.435+0000 7f9e07e00640 1 -- 172.21.15.83:0/1707662534 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e0800ae20 con 0x7f9e1019a7e0 2024-04-24T09:18:55.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.436+0000 7f9e15c00640 1 -- 172.21.15.83:0/1707662534 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e1019b660 con 0x7f9e1019a7e0 2024-04-24T09:18:55.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.436+0000 7f9e07e00640 1 -- 172.21.15.83:0/1707662534 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9e08005020 con 0x7f9e1019a7e0 2024-04-24T09:18:55.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.436+0000 7f9e07e00640 1 -- 172.21.15.83:0/1707662534 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e0800f6b0 con 0x7f9e1019a7e0 2024-04-24T09:18:55.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.436+0000 7f9e15c00640 1 -- 172.21.15.83:0/1707662534 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e101a7810 con 0x7f9e1019a7e0 2024-04-24T09:18:55.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.437+0000 7f9e15c00640 1 -- 172.21.15.83:0/1707662534 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e10105920 con 0x7f9e1019a7e0 2024-04-24T09:18:55.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.439+0000 7f9e07e00640 1 -- 172.21.15.83:0/1707662534 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9e080051d0 con 0x7f9e1019a7e0 2024-04-24T09:18:55.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.440+0000 7f9e07e00640 1 --2- 172.21.15.83:0/1707662534 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f9df40a3d20 0x7f9df40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:55.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.440+0000 7f9e07e00640 1 -- 172.21.15.83:0/1707662534 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 9583+0+0 (secure 0 0 0) 0x7f9e080d0d00 con 0x7f9e1019a7e0 2024-04-24T09:18:55.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.441+0000 7f9e0ea00640 1 --2- 172.21.15.83:0/1707662534 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f9df40a3d20 0x7f9df40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:55.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.441+0000 7f9e0ea00640 1 --2- 172.21.15.83:0/1707662534 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f9df40a3d20 0x7f9df40a61e0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f9e0000abe0 tx=0x7f9e0000a5c0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:18:55.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.445+0000 7f9e07e00640 1 -- 172.21.15.83:0/1707662534 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9e1019b660 con 0x7f9e1019a7e0 2024-04-24T09:18:55.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:18:55 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/1210693049' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T09:18:55.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:18:55 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/4056108253' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T09:18:55.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:18:55 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/3304998655' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T09:18:55.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.641+0000 7f9e15c00640 1 -- 172.21.15.83:0/1707662534 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f9e10104e00 con 0x7f9e1019a7e0 2024-04-24T09:18:55.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.643+0000 7f9e07e00640 1 -- 172.21.15.83:0/1707662534 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v136) ==== 103+0+11 (secure 0 0 0) 0x7f9e08014030 con 0x7f9e1019a7e0 2024-04-24T09:18:55.643 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-24T09:18:55.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.645+0000 7f9e15c00640 1 -- 172.21.15.83:0/1707662534 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f9df40a3d20 msgr2=0x7f9df40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:55.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.645+0000 7f9e15c00640 1 --2- 172.21.15.83:0/1707662534 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f9df40a3d20 0x7f9df40a61e0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f9e0000abe0 tx=0x7f9e0000a5c0 comp rx=0 tx=0).stop 2024-04-24T09:18:55.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.645+0000 7f9e15c00640 1 -- 172.21.15.83:0/1707662534 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9e1019a7e0 msgr2=0x7f9e1019ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:55.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.645+0000 7f9e15c00640 1 --2- 172.21.15.83:0/1707662534 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9e1019a7e0 0x7f9e1019ac90 secure :-1 s=READY pgs=2916 cs=0 l=1 rev1=1 crypto rx=0x7f9e0800cad0 tx=0x7f9e0800cf90 comp rx=0 tx=0).stop 2024-04-24T09:18:55.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.645+0000 7f9e15c00640 1 -- 172.21.15.83:0/1707662534 shutdown_connections 2024-04-24T09:18:55.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.645+0000 7f9e15c00640 1 --2- 172.21.15.83:0/1707662534 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f9df40a3d20 0x7f9df40a61e0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:55.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.645+0000 7f9e15c00640 1 --2- 172.21.15.83:0/1707662534 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9e1019a7e0 0x7f9e1019ac90 unknown :-1 s=CLOSED pgs=2916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:55.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.646+0000 7f9e15c00640 1 --2- 172.21.15.83:0/1707662534 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9e10109520 0x7f9e101a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:55.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.646+0000 7f9e15c00640 1 --2- 172.21.15.83:0/1707662534 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9e10104450 0x7f9e101a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:55.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.646+0000 7f9e15c00640 1 -- 172.21.15.83:0/1707662534 >> 172.21.15.83:0/1707662534 conn(0x7f9e100ffc70 msgr2=0x7f9e10100fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:18:55.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.646+0000 7f9e15c00640 1 -- 172.21.15.83:0/1707662534 shutdown_connections 2024-04-24T09:18:55.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.646+0000 7f9e15c00640 1 -- 172.21.15.83:0/1707662534 wait complete. 2024-04-24T09:18:55.664 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-24T09:18:55.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:18:55 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/1210693049' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T09:18:55.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:18:55 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/4056108253' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T09:18:55.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:18:55 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/3304998655' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T09:18:55.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:18:55 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/1210693049' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T09:18:55.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:18:55 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/4056108253' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T09:18:55.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:18:55 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/3304998655' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T09:18:55.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.860+0000 7fbd2ca00640 1 -- 172.21.15.83:0/2713952629 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd28104e60 msgr2=0x7fbd28108ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:55.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.860+0000 7fbd2ca00640 1 --2- 172.21.15.83:0/2713952629 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd28104e60 0x7fbd28108ea0 secure :-1 s=READY pgs=2917 cs=0 l=1 rev1=1 crypto rx=0x7fbd10009900 tx=0x7fbd1002a3f0 comp rx=0 tx=0).stop 2024-04-24T09:18:55.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.860+0000 7fbd2ca00640 1 -- 172.21.15.83:0/2713952629 shutdown_connections 2024-04-24T09:18:55.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.860+0000 7fbd2ca00640 1 --2- 172.21.15.83:0/2713952629 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd28109580 0x7fbd28111150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:55.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.860+0000 7fbd2ca00640 1 --2- 172.21.15.83:0/2713952629 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd28104e60 0x7fbd28108ea0 unknown :-1 s=CLOSED pgs=2917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:55.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.860+0000 7fbd2ca00640 1 --2- 172.21.15.83:0/2713952629 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbd281044b0 0x7fbd28104890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:55.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.860+0000 7fbd2ca00640 1 -- 172.21.15.83:0/2713952629 >> 172.21.15.83:0/2713952629 conn(0x7fbd280ffd90 msgr2=0x7fbd281021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:18:55.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.860+0000 7fbd2ca00640 1 -- 172.21.15.83:0/2713952629 shutdown_connections 2024-04-24T09:18:55.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.860+0000 7fbd2ca00640 1 -- 172.21.15.83:0/2713952629 wait complete. 2024-04-24T09:18:55.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.861+0000 7fbd2ca00640 1 Processor -- start 2024-04-24T09:18:55.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.861+0000 7fbd2ca00640 1 -- start start 2024-04-24T09:18:55.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.861+0000 7fbd2ca00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd281044b0 0x7fbd281a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:55.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.861+0000 7fbd2ca00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd28109580 0x7fbd281a0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:55.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.861+0000 7fbd2ca00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbd2819a870 0x7fbd2819ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:55.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.861+0000 7fbd2ca00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbd281144f0 con 0x7fbd281044b0 2024-04-24T09:18:55.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.861+0000 7fbd2ca00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbd28114370 con 0x7fbd28109580 2024-04-24T09:18:55.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.861+0000 7fbd2ca00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fbd28114670 con 0x7fbd2819a870 2024-04-24T09:18:55.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.862+0000 7fbd25e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd281044b0 0x7fbd281a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:55.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.862+0000 7fbd25e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd281044b0 0x7fbd281a06c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38208/0 (socket says 172.21.15.83:38208) 2024-04-24T09:18:55.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.862+0000 7fbd25e00640 1 -- 172.21.15.83:0/1190327304 learned_addr learned my addr 172.21.15.83:0/1190327304 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:18:55.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.862+0000 7fbd25400640 1 --2- 172.21.15.83:0/1190327304 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd28109580 0x7fbd281a0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:55.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.862+0000 7fbd26800640 1 --2- 172.21.15.83:0/1190327304 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbd2819a870 0x7fbd2819ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:55.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.862+0000 7fbd25e00640 1 -- 172.21.15.83:0/1190327304 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbd2819a870 msgr2=0x7fbd2819ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:55.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.862+0000 7fbd25e00640 1 --2- 172.21.15.83:0/1190327304 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbd2819a870 0x7fbd2819ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:55.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.862+0000 7fbd25e00640 1 -- 172.21.15.83:0/1190327304 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd28109580 msgr2=0x7fbd281a0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:55.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.862+0000 7fbd25e00640 1 --2- 172.21.15.83:0/1190327304 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd28109580 0x7fbd281a0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:55.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.862+0000 7fbd25e00640 1 -- 172.21.15.83:0/1190327304 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd2819b390 con 0x7fbd281044b0 2024-04-24T09:18:55.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.862+0000 7fbd25e00640 1 --2- 172.21.15.83:0/1190327304 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd281044b0 0x7fbd281a06c0 secure :-1 s=READY pgs=2918 cs=0 l=1 rev1=1 crypto rx=0x7fbd1800c920 tx=0x7fbd1800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:18:55.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.863+0000 7fbd1ea00640 1 -- 172.21.15.83:0/1190327304 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd18007c70 con 0x7fbd281044b0 2024-04-24T09:18:55.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.863+0000 7fbd1ea00640 1 -- 172.21.15.83:0/1190327304 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbd18007e10 con 0x7fbd281044b0 2024-04-24T09:18:55.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.863+0000 7fbd1ea00640 1 -- 172.21.15.83:0/1190327304 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd18018820 con 0x7fbd281044b0 2024-04-24T09:18:55.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.863+0000 7fbd2ca00640 1 -- 172.21.15.83:0/1190327304 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd2819b680 con 0x7fbd281044b0 2024-04-24T09:18:55.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.863+0000 7fbd2ca00640 1 -- 172.21.15.83:0/1190327304 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd281a7920 con 0x7fbd281044b0 2024-04-24T09:18:55.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.864+0000 7fbd2ca00640 1 -- 172.21.15.83:0/1190327304 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd28069270 con 0x7fbd281044b0 2024-04-24T09:18:55.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.866+0000 7fbd1ea00640 1 -- 172.21.15.83:0/1190327304 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fbd180189c0 con 0x7fbd281044b0 2024-04-24T09:18:55.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.867+0000 7fbd1ea00640 1 --2- 172.21.15.83:0/1190327304 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fbd0c0a3da0 0x7fbd0c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:55.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.868+0000 7fbd1ea00640 1 -- 172.21.15.83:0/1190327304 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 9583+0+0 (secure 0 0 0) 0x7fbd180d0e30 con 0x7fbd281044b0 2024-04-24T09:18:55.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.868+0000 7fbd25400640 1 --2- 172.21.15.83:0/1190327304 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fbd0c0a3da0 0x7fbd0c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:55.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.868+0000 7fbd25400640 1 --2- 172.21.15.83:0/1190327304 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fbd0c0a3da0 0x7fbd0c0a6260 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fbd10006010 tx=0x7fbd10000bc0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:18:55.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:55.871+0000 7fbd1ea00640 1 -- 172.21.15.83:0/1190327304 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbd18014030 con 0x7fbd281044b0 2024-04-24T09:18:56.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:56.087+0000 7fbd2ca00640 1 -- 172.21.15.83:0/1190327304 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fbd2810c810 con 0x7fbd281044b0 2024-04-24T09:18:56.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:56.088+0000 7fbd1ea00640 1 -- 172.21.15.83:0/1190327304 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v136) ==== 81+0+943 (secure 0 0 0) 0x7fbd1809d050 con 0x7fbd281044b0 2024-04-24T09:18:56.089 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:0/1096290901 2024-04-25T08:51:36.347763+0000 2024-04-24T09:18:56.089 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:6800/2550826560 2024-04-25T08:51:36.347763+0000 2024-04-24T09:18:56.089 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:0/1743153945 2024-04-25T08:51:36.347763+0000 2024-04-24T09:18:56.089 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:0/3306681366 2024-04-25T08:51:36.347763+0000 2024-04-24T09:18:56.089 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:6801/820049883 2024-04-25T08:38:40.661315+0000 2024-04-24T09:18:56.090 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:6801/2550826560 2024-04-25T08:51:36.347763+0000 2024-04-24T09:18:56.090 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:0/4272874138 2024-04-25T08:38:40.661315+0000 2024-04-24T09:18:56.090 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:0/4212994335 2024-04-25T08:38:40.661315+0000 2024-04-24T09:18:56.090 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:6801/1996371091 2024-04-25T08:38:05.800856+0000 2024-04-24T09:18:56.090 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:0/1820663683 2024-04-25T08:38:05.800856+0000 2024-04-24T09:18:56.090 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:0/3802897262 2024-04-25T08:38:05.800856+0000 2024-04-24T09:18:56.090 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:6800/1996371091 2024-04-25T08:38:05.800856+0000 2024-04-24T09:18:56.090 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:6800/820049883 2024-04-25T08:38:40.661315+0000 2024-04-24T09:18:56.090 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:0/1366125049 2024-04-25T08:38:05.800856+0000 2024-04-24T09:18:56.090 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:0/4159820288 2024-04-25T08:51:36.347763+0000 2024-04-24T09:18:56.090 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:0/340452868 2024-04-25T08:38:40.661315+0000 2024-04-24T09:18:56.090 INFO:teuthology.orchestra.run.smithi083.stderr:listed 16 entries 2024-04-24T09:18:56.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:56.091+0000 7fbd2ca00640 1 -- 172.21.15.83:0/1190327304 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fbd0c0a3da0 msgr2=0x7fbd0c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:56.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:56.092+0000 7fbd2ca00640 1 --2- 172.21.15.83:0/1190327304 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fbd0c0a3da0 0x7fbd0c0a6260 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fbd10006010 tx=0x7fbd10000bc0 comp rx=0 tx=0).stop 2024-04-24T09:18:56.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:56.092+0000 7fbd2ca00640 1 -- 172.21.15.83:0/1190327304 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd281044b0 msgr2=0x7fbd281a06c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:56.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:56.092+0000 7fbd2ca00640 1 --2- 172.21.15.83:0/1190327304 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd281044b0 0x7fbd281a06c0 secure :-1 s=READY pgs=2918 cs=0 l=1 rev1=1 crypto rx=0x7fbd1800c920 tx=0x7fbd1800cde0 comp rx=0 tx=0).stop 2024-04-24T09:18:56.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:56.092+0000 7fbd2ca00640 1 -- 172.21.15.83:0/1190327304 shutdown_connections 2024-04-24T09:18:56.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:56.092+0000 7fbd2ca00640 1 --2- 172.21.15.83:0/1190327304 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fbd0c0a3da0 0x7fbd0c0a6260 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:56.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:56.092+0000 7fbd2ca00640 1 --2- 172.21.15.83:0/1190327304 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbd2819a870 0x7fbd2819ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:56.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:56.092+0000 7fbd2ca00640 1 --2- 172.21.15.83:0/1190327304 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd28109580 0x7fbd281a0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:56.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:56.092+0000 7fbd2ca00640 1 --2- 172.21.15.83:0/1190327304 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd281044b0 0x7fbd281a06c0 unknown :-1 s=CLOSED pgs=2918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:56.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:56.092+0000 7fbd2ca00640 1 -- 172.21.15.83:0/1190327304 >> 172.21.15.83:0/1190327304 conn(0x7fbd280ffd90 msgr2=0x7fbd281010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:18:56.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:56.092+0000 7fbd2ca00640 1 -- 172.21.15.83:0/1190327304 shutdown_connections 2024-04-24T09:18:56.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:56.092+0000 7fbd2ca00640 1 -- 172.21.15.83:0/1190327304 wait complete. 2024-04-24T09:18:56.110 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-24T09:18:56.111 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T09:18:56.161 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T09:18:56.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:56.362+0000 7f9185e00640 1 -- 172.21.15.83:0/3179196507 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9180076530 msgr2=0x7f9180076910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:56.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:56.362+0000 7f9185e00640 1 --2- 172.21.15.83:0/3179196507 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9180076530 0x7f9180076910 secure :-1 s=READY pgs=2919 cs=0 l=1 rev1=1 crypto rx=0x7f9174009900 tx=0x7f917402a410 comp rx=0 tx=0).stop 2024-04-24T09:18:56.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:56.362+0000 7f9185e00640 1 -- 172.21.15.83:0/3179196507 shutdown_connections 2024-04-24T09:18:56.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:56.362+0000 7f9185e00640 1 --2- 172.21.15.83:0/3179196507 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9180112ca0 0x7f9180115090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:56.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:56.362+0000 7f9185e00640 1 --2- 172.21.15.83:0/3179196507 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9180076e50 0x7f9180112760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:56.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:56.362+0000 7f9185e00640 1 --2- 172.21.15.83:0/3179196507 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9180076530 0x7f9180076910 unknown :-1 s=CLOSED pgs=2919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:56.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:56.362+0000 7f9185e00640 1 -- 172.21.15.83:0/3179196507 >> 172.21.15.83:0/3179196507 conn(0x7f91800ffad0 msgr2=0x7f9180101ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:18:56.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:56.362+0000 7f9185e00640 1 -- 172.21.15.83:0/3179196507 shutdown_connections 2024-04-24T09:18:56.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:56.362+0000 7f9185e00640 1 -- 172.21.15.83:0/3179196507 wait complete. 2024-04-24T09:18:56.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:56.363+0000 7f9185e00640 1 Processor -- start 2024-04-24T09:18:56.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:56.363+0000 7f9185e00640 1 -- start start 2024-04-24T09:18:56.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:56.363+0000 7f9185e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9180076e50 0x7f918019ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:56.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:56.363+0000 7f9185e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9180112ca0 0x7f918019f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:56.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:56.363+0000 7f9185e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91801a36c0 0x7f91801a3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:56.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:56.363+0000 7f9185e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9180118710 con 0x7f9180112ca0 2024-04-24T09:18:56.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:56.363+0000 7f9185e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9180118590 con 0x7f91801a36c0 2024-04-24T09:18:56.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:56.363+0000 7f9185e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9180118890 con 0x7f9180076e50 2024-04-24T09:18:56.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:56.364+0000 7f917fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9180112ca0 0x7f918019f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:56.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:56.364+0000 7f917fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9180112ca0 0x7f918019f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38222/0 (socket says 172.21.15.83:38222) 2024-04-24T09:18:56.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:56.364+0000 7f917fe00640 1 -- 172.21.15.83:0/3134096215 learned_addr learned my addr 172.21.15.83:0/3134096215 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:18:56.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:56.364+0000 7f917fe00640 1 -- 172.21.15.83:0/3134096215 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9180076e50 msgr2=0x7f918019ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:56.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:56.364+0000 7f917fe00640 1 --2- 172.21.15.83:0/3134096215 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9180076e50 0x7f918019ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:56.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:56.364+0000 7f9185400640 1 --2- 172.21.15.83:0/3134096215 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91801a36c0 0x7f91801a3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:56.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:56.364+0000 7f917fe00640 1 -- 172.21.15.83:0/3134096215 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91801a36c0 msgr2=0x7f91801a3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:56.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:56.364+0000 7f917fe00640 1 --2- 172.21.15.83:0/3134096215 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91801a36c0 0x7f91801a3b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:56.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:56.364+0000 7f917fe00640 1 -- 172.21.15.83:0/3134096215 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91801a41b0 con 0x7f9180112ca0 2024-04-24T09:18:56.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:56.364+0000 7f917fe00640 1 --2- 172.21.15.83:0/3134096215 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9180112ca0 0x7f918019f390 secure :-1 s=READY pgs=2920 cs=0 l=1 rev1=1 crypto rx=0x7f916c00d950 tx=0x7f916c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:18:56.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:56.365+0000 7f917d600640 1 -- 172.21.15.83:0/3134096215 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f916c017030 con 0x7f9180112ca0 2024-04-24T09:18:56.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:56.365+0000 7f917d600640 1 -- 172.21.15.83:0/3134096215 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f916c004020 con 0x7f9180112ca0 2024-04-24T09:18:56.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:56.365+0000 7f9185e00640 1 -- 172.21.15.83:0/3134096215 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91801a44a0 con 0x7f9180112ca0 2024-04-24T09:18:56.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:56.365+0000 7f917d600640 1 -- 172.21.15.83:0/3134096215 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f916c0047a0 con 0x7f9180112ca0 2024-04-24T09:18:56.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:56.365+0000 7f9185e00640 1 -- 172.21.15.83:0/3134096215 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91801abc90 con 0x7f9180112ca0 2024-04-24T09:18:56.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:56.366+0000 7f9185e00640 1 -- 172.21.15.83:0/3134096215 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9180069190 con 0x7f9180112ca0 2024-04-24T09:18:56.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:56.368+0000 7f917d600640 1 -- 172.21.15.83:0/3134096215 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f916c0041d0 con 0x7f9180112ca0 2024-04-24T09:18:56.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:56.370+0000 7f917d600640 1 --2- 172.21.15.83:0/3134096215 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f91680a3d20 0x7f91680a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:56.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:56.370+0000 7f917d600640 1 -- 172.21.15.83:0/3134096215 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 9583+0+0 (secure 0 0 0) 0x7f916c0d07e0 con 0x7f9180112ca0 2024-04-24T09:18:56.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:56.370+0000 7f9184a00640 1 --2- 172.21.15.83:0/3134096215 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f91680a3d20 0x7f91680a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:56.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:56.371+0000 7f9184a00640 1 --2- 172.21.15.83:0/3134096215 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f91680a3d20 0x7f91680a61e0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f9174009230 tx=0x7f9174000bc0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:18:56.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:56.374+0000 7f917d600640 1 -- 172.21.15.83:0/3134096215 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f916c09ca00 con 0x7f9180112ca0 2024-04-24T09:18:56.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:18:56 smithi083 ceph-mon[27907]: pgmap v864: 129 pgs: 129 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-24T09:18:56.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:18:56 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/1707662534' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T09:18:56.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:18:56 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/1190327304' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-24T09:18:56.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:56.564+0000 7f9185e00640 1 -- 172.21.15.83:0/3134096215 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f918002d620 con 0x7f9180112ca0 2024-04-24T09:18:56.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:56.568+0000 7f917d600640 1 -- 172.21.15.83:0/3134096215 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v136) ==== 74+0+25453 (secure 0 0 0) 0x7f916c09c300 con 0x7f9180112ca0 2024-04-24T09:18:56.569 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T09:18:56.570 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":136,"fsid":"bd3e039e-0215-11ef-bc93-c7b262605968","created":"2024-04-24T08:37:32.209944+0000","modified":"2024-04-24T09:18:14.841781+0000","last_up_change":"2024-04-24T08:47:17.966041+0000","last_in_change":"2024-04-24T08:46:53.835936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T08:41:56.834372+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T08:48:26.978311+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T08:48:28.352731+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"135","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":48,"snap_epoch":135,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T08:48:32.275167+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"135","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":48,"snap_epoch":135,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2cd77f3a-19dd-4c6e-a199-1dcbd5faa679","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2132675555},{"type":"v1","addr":"172.21.15.83:6803","nonce":2132675555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2132675555},{"type":"v1","addr":"172.21.15.83:6805","nonce":2132675555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2132675555},{"type":"v1","addr":"172.21.15.83:6809","nonce":2132675555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2132675555},{"type":"v1","addr":"172.21.15.83:6807","nonce":2132675555}]},"public_addr":"172.21.15.83:6803/2132675555","cluster_addr":"172.21.15.83:6805/2132675555","heartbeat_back_addr":"172.21.15.83:6809/2132675555","heartbeat_front_addr":"172.21.15.83:6807/2132675555","state":["exists","up"]},{"osd":1,"uuid":"aba7f5df-2a4c-4c23-b502-d193605a5b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":3589158880},{"type":"v1","addr":"172.21.15.138:6801","nonce":3589158880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":3589158880},{"type":"v1","addr":"172.21.15.138:6803","nonce":3589158880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":3589158880},{"type":"v1","addr":"172.21.15.138:6807","nonce":3589158880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":3589158880},{"type":"v1","addr":"172.21.15.138:6805","nonce":3589158880}]},"public_addr":"172.21.15.138:6801/3589158880","cluster_addr":"172.21.15.138:6803/3589158880","heartbeat_back_addr":"172.21.15.138:6807/3589158880","heartbeat_front_addr":"172.21.15.138:6805/3589158880","state":["exists","up"]},{"osd":2,"uuid":"071bc167-2333-4420-bb58-bffbc10d7641","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":3169072688},{"type":"v1","addr":"172.21.15.189:6801","nonce":3169072688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":3169072688},{"type":"v1","addr":"172.21.15.189:6803","nonce":3169072688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":3169072688},{"type":"v1","addr":"172.21.15.189:6807","nonce":3169072688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":3169072688},{"type":"v1","addr":"172.21.15.189:6805","nonce":3169072688}]},"public_addr":"172.21.15.189:6801/3169072688","cluster_addr":"172.21.15.189:6803/3169072688","heartbeat_back_addr":"172.21.15.189:6807/3169072688","heartbeat_front_addr":"172.21.15.189:6805/3169072688","state":["exists","up"]},{"osd":3,"uuid":"f5624d78-4455-4e07-8c1a-f47c94cddb23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3131092450},{"type":"v1","addr":"172.21.15.83:6811","nonce":3131092450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3131092450},{"type":"v1","addr":"172.21.15.83:6813","nonce":3131092450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3131092450},{"type":"v1","addr":"172.21.15.83:6817","nonce":3131092450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3131092450},{"type":"v1","addr":"172.21.15.83:6815","nonce":3131092450}]},"public_addr":"172.21.15.83:6811/3131092450","cluster_addr":"172.21.15.83:6813/3131092450","heartbeat_back_addr":"172.21.15.83:6817/3131092450","heartbeat_front_addr":"172.21.15.83:6815/3131092450","state":["exists","up"]},{"osd":4,"uuid":"7b50cd14-9385-4535-a7b0-1c1da93c8548","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":823775360},{"type":"v1","addr":"172.21.15.138:6809","nonce":823775360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":823775360},{"type":"v1","addr":"172.21.15.138:6811","nonce":823775360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":823775360},{"type":"v1","addr":"172.21.15.138:6815","nonce":823775360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":823775360},{"type":"v1","addr":"172.21.15.138:6813","nonce":823775360}]},"public_addr":"172.21.15.138:6809/823775360","cluster_addr":"172.21.15.138:6811/823775360","heartbeat_back_addr":"172.21.15.138:6815/823775360","heartbeat_front_addr":"172.21.15.138:6813/823775360","state":["exists","up"]},{"osd":5,"uuid":"9c647797-35bc-449f-be84-af97d2c69c2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":1399817431},{"type":"v1","addr":"172.21.15.189:6809","nonce":1399817431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":1399817431},{"type":"v1","addr":"172.21.15.189:6811","nonce":1399817431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":1399817431},{"type":"v1","addr":"172.21.15.189:6815","nonce":1399817431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":1399817431},{"type":"v1","addr":"172.21.15.189:6813","nonce":1399817431}]},"public_addr":"172.21.15.189:6809/1399817431","cluster_addr":"172.21.15.189:6811/1399817431","heartbeat_back_addr":"172.21.15.189:6815/1399817431","heartbeat_front_addr":"172.21.15.189:6813/1399817431","state":["exists","up"]},{"osd":6,"uuid":"8f056584-16c8-4454-b445-3f780f26c196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":1611358483},{"type":"v1","addr":"172.21.15.83:6819","nonce":1611358483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1611358483},{"type":"v1","addr":"172.21.15.83:6821","nonce":1611358483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1611358483},{"type":"v1","addr":"172.21.15.83:6825","nonce":1611358483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":1611358483},{"type":"v1","addr":"172.21.15.83:6823","nonce":1611358483}]},"public_addr":"172.21.15.83:6819/1611358483","cluster_addr":"172.21.15.83:6821/1611358483","heartbeat_back_addr":"172.21.15.83:6825/1611358483","heartbeat_front_addr":"172.21.15.83:6823/1611358483","state":["exists","up"]},{"osd":7,"uuid":"7603efe3-da77-4b7f-9d03-29c4f293b6d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":3557856706},{"type":"v1","addr":"172.21.15.138:6817","nonce":3557856706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":3557856706},{"type":"v1","addr":"172.21.15.138:6819","nonce":3557856706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":3557856706},{"type":"v1","addr":"172.21.15.138:6823","nonce":3557856706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":3557856706},{"type":"v1","addr":"172.21.15.138:6821","nonce":3557856706}]},"public_addr":"172.21.15.138:6817/3557856706","cluster_addr":"172.21.15.138:6819/3557856706","heartbeat_back_addr":"172.21.15.138:6823/3557856706","heartbeat_front_addr":"172.21.15.138:6821/3557856706","state":["exists","up"]},{"osd":8,"uuid":"27d7d1b8-d6a0-417e-b8aa-596325aa1a77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":207018554},{"type":"v1","addr":"172.21.15.189:6817","nonce":207018554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":207018554},{"type":"v1","addr":"172.21.15.189:6819","nonce":207018554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":207018554},{"type":"v1","addr":"172.21.15.189:6823","nonce":207018554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":207018554},{"type":"v1","addr":"172.21.15.189:6821","nonce":207018554}]},"public_addr":"172.21.15.189:6817/207018554","cluster_addr":"172.21.15.189:6819/207018554","heartbeat_back_addr":"172.21.15.189:6823/207018554","heartbeat_front_addr":"172.21.15.189:6821/207018554","state":["exists","up"]},{"osd":9,"uuid":"baf19b98-881b-4d3a-9f72-8aa2dd73cad6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":519636751},{"type":"v1","addr":"172.21.15.83:6827","nonce":519636751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":519636751},{"type":"v1","addr":"172.21.15.83:6829","nonce":519636751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":519636751},{"type":"v1","addr":"172.21.15.83:6833","nonce":519636751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":519636751},{"type":"v1","addr":"172.21.15.83:6831","nonce":519636751}]},"public_addr":"172.21.15.83:6827/519636751","cluster_addr":"172.21.15.83:6829/519636751","heartbeat_back_addr":"172.21.15.83:6833/519636751","heartbeat_front_addr":"172.21.15.83:6831/519636751","state":["exists","up"]},{"osd":10,"uuid":"eb346717-5c07-47c8-95be-0e68fe449fdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1130872098},{"type":"v1","addr":"172.21.15.138:6825","nonce":1130872098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1130872098},{"type":"v1","addr":"172.21.15.138:6827","nonce":1130872098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1130872098},{"type":"v1","addr":"172.21.15.138:6831","nonce":1130872098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1130872098},{"type":"v1","addr":"172.21.15.138:6829","nonce":1130872098}]},"public_addr":"172.21.15.138:6825/1130872098","cluster_addr":"172.21.15.138:6827/1130872098","heartbeat_back_addr":"172.21.15.138:6831/1130872098","heartbeat_front_addr":"172.21.15.138:6829/1130872098","state":["exists","up"]},{"osd":11,"uuid":"8e37568e-981c-45b8-b2af-2f1b73898542","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":2998782690},{"type":"v1","addr":"172.21.15.189:6825","nonce":2998782690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":2998782690},{"type":"v1","addr":"172.21.15.189:6827","nonce":2998782690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":2998782690},{"type":"v1","addr":"172.21.15.189:6831","nonce":2998782690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":2998782690},{"type":"v1","addr":"172.21.15.189:6829","nonce":2998782690}]},"public_addr":"172.21.15.189:6825/2998782690","cluster_addr":"172.21.15.189:6827/2998782690","heartbeat_back_addr":"172.21.15.189:6831/2998782690","heartbeat_front_addr":"172.21.15.189:6829/2998782690","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:40:47.980103+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:41:21.970795+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:41:55.160426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:42:30.400475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:43:05.705663+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:43:40.112312+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:44:16.781289+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:44:52.151187+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:45:28.055944+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:46:04.969301+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:46:40.918306+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:47:16.886874+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"2.1d","mappings":[{"from":2,"to":1}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.2a","mappings":[{"from":3,"to":8}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.a","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:0/1096290901":"2024-04-25T08:51:36.347763+0000","172.21.15.83:6800/2550826560":"2024-04-25T08:51:36.347763+0000","172.21.15.83:0/1743153945":"2024-04-25T08:51:36.347763+0000","172.21.15.83:0/3306681366":"2024-04-25T08:51:36.347763+0000","172.21.15.83:6801/820049883":"2024-04-25T08:38:40.661315+0000","172.21.15.83:6801/2550826560":"2024-04-25T08:51:36.347763+0000","172.21.15.83:0/4272874138":"2024-04-25T08:38:40.661315+0000","172.21.15.83:0/4212994335":"2024-04-25T08:38:40.661315+0000","172.21.15.83:6801/1996371091":"2024-04-25T08:38:05.800856+0000","172.21.15.83:0/1820663683":"2024-04-25T08:38:05.800856+0000","172.21.15.83:0/3802897262":"2024-04-25T08:38:05.800856+0000","172.21.15.83:6800/1996371091":"2024-04-25T08:38:05.800856+0000","172.21.15.83:6800/820049883":"2024-04-25T08:38:40.661315+0000","172.21.15.83:0/1366125049":"2024-04-25T08:38:05.800856+0000","172.21.15.83:0/4159820288":"2024-04-25T08:51:36.347763+0000","172.21.15.83:0/340452868":"2024-04-25T08:38:40.661315+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":35,"length":1},{"begin":48,"length":1}]},{"pool":4,"snaps":[{"begin":35,"length":1},{"begin":48,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T09:18:56.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:56.571+0000 7f9185e00640 1 -- 172.21.15.83:0/3134096215 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f91680a3d20 msgr2=0x7f91680a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:56.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:56.571+0000 7f9185e00640 1 --2- 172.21.15.83:0/3134096215 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f91680a3d20 0x7f91680a61e0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f9174009230 tx=0x7f9174000bc0 comp rx=0 tx=0).stop 2024-04-24T09:18:56.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:56.571+0000 7f9185e00640 1 -- 172.21.15.83:0/3134096215 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9180112ca0 msgr2=0x7f918019f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:56.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:56.571+0000 7f9185e00640 1 --2- 172.21.15.83:0/3134096215 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9180112ca0 0x7f918019f390 secure :-1 s=READY pgs=2920 cs=0 l=1 rev1=1 crypto rx=0x7f916c00d950 tx=0x7f916c00de10 comp rx=0 tx=0).stop 2024-04-24T09:18:56.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:56.572+0000 7f9185e00640 1 -- 172.21.15.83:0/3134096215 shutdown_connections 2024-04-24T09:18:56.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:56.572+0000 7f9185e00640 1 --2- 172.21.15.83:0/3134096215 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f91680a3d20 0x7f91680a61e0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:56.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:56.572+0000 7f9185e00640 1 --2- 172.21.15.83:0/3134096215 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91801a36c0 0x7f91801a3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:56.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:56.572+0000 7f9185e00640 1 --2- 172.21.15.83:0/3134096215 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9180112ca0 0x7f918019f390 unknown :-1 s=CLOSED pgs=2920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:56.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:56.572+0000 7f9185e00640 1 --2- 172.21.15.83:0/3134096215 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9180076e50 0x7f918019ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:56.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:56.572+0000 7f9185e00640 1 -- 172.21.15.83:0/3134096215 >> 172.21.15.83:0/3134096215 conn(0x7f91800ffad0 msgr2=0x7f91801143c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:18:56.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:56.572+0000 7f9185e00640 1 -- 172.21.15.83:0/3134096215 shutdown_connections 2024-04-24T09:18:56.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:56.572+0000 7f9185e00640 1 -- 172.21.15.83:0/3134096215 wait complete. 2024-04-24T09:18:56.591 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T08:41:56.834372+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T08:48:26.978311+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T08:48:28.352731+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '135', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 48, 'snap_epoch': 135, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-24T08:48:32.275167+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '135', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 48, 'snap_epoch': 135, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-24T09:18:56.591 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T09:18:56.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:18:56 smithi138 ceph-mon[31441]: pgmap v864: 129 pgs: 129 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-24T09:18:56.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:18:56 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/1707662534' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T09:18:56.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:18:56 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/1190327304' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-24T09:18:56.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:18:56 smithi189 ceph-mon[31791]: pgmap v864: 129 pgs: 129 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-24T09:18:56.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:18:56 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/1707662534' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T09:18:56.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:18:56 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/1190327304' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-24T09:18:56.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:56.791+0000 7f236e200640 1 -- 172.21.15.83:0/2133932634 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2368102120 msgr2=0x7f2368102500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:56.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:56.791+0000 7f236e200640 1 --2- 172.21.15.83:0/2133932634 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2368102120 0x7f2368102500 secure :-1 s=READY pgs=2921 cs=0 l=1 rev1=1 crypto rx=0x7f2358009900 tx=0x7f235802a410 comp rx=0 tx=0).stop 2024-04-24T09:18:56.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:56.792+0000 7f236e200640 1 -- 172.21.15.83:0/2133932634 shutdown_connections 2024-04-24T09:18:56.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:56.792+0000 7f236e200640 1 --2- 172.21.15.83:0/2133932634 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f23681071f0 0x7f236810edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:56.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:56.792+0000 7f236e200640 1 --2- 172.21.15.83:0/2133932634 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2368102ad0 0x7f2368106b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:56.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:56.792+0000 7f236e200640 1 --2- 172.21.15.83:0/2133932634 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2368102120 0x7f2368102500 unknown :-1 s=CLOSED pgs=2921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:56.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:56.792+0000 7f236e200640 1 -- 172.21.15.83:0/2133932634 >> 172.21.15.83:0/2133932634 conn(0x7f23680fd9a0 msgr2=0x7f23680ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:18:56.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:56.792+0000 7f236e200640 1 -- 172.21.15.83:0/2133932634 shutdown_connections 2024-04-24T09:18:56.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:56.792+0000 7f236e200640 1 -- 172.21.15.83:0/2133932634 wait complete. 2024-04-24T09:18:56.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:56.793+0000 7f236e200640 1 Processor -- start 2024-04-24T09:18:56.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:56.793+0000 7f236e200640 1 -- start start 2024-04-24T09:18:56.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:56.793+0000 7f236e200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2368102120 0x7f23681a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:56.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:56.793+0000 7f236e200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23681071f0 0x7f23681a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:56.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:56.793+0000 7f236e200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f236819a710 0x7f236819ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:56.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:56.793+0000 7f236e200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2368112130 con 0x7f236819a710 2024-04-24T09:18:56.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:56.793+0000 7f236e200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2368111fb0 con 0x7f23681071f0 2024-04-24T09:18:56.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:56.793+0000 7f236e200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f23681122b0 con 0x7f2368102120 2024-04-24T09:18:56.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:56.793+0000 7f236d800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f236819a710 0x7f236819ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:56.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:56.793+0000 7f236d800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f236819a710 0x7f236819ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38232/0 (socket says 172.21.15.83:38232) 2024-04-24T09:18:56.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:56.793+0000 7f236d800640 1 -- 172.21.15.83:0/2553894829 learned_addr learned my addr 172.21.15.83:0/2553894829 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:18:56.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:56.794+0000 7f236ce00640 1 --2- 172.21.15.83:0/2553894829 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2368102120 0x7f23681a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:56.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:56.794+0000 7f2367e00640 1 --2- 172.21.15.83:0/2553894829 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23681071f0 0x7f23681a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:56.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:56.794+0000 7f236d800640 1 -- 172.21.15.83:0/2553894829 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2368102120 msgr2=0x7f23681a0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:56.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:56.794+0000 7f236d800640 1 --2- 172.21.15.83:0/2553894829 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2368102120 0x7f23681a0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:56.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:56.794+0000 7f236d800640 1 -- 172.21.15.83:0/2553894829 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23681071f0 msgr2=0x7f23681a0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:56.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:56.794+0000 7f236d800640 1 --2- 172.21.15.83:0/2553894829 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23681071f0 0x7f23681a0b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:56.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:56.794+0000 7f236d800640 1 -- 172.21.15.83:0/2553894829 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f236819b310 con 0x7f236819a710 2024-04-24T09:18:56.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:56.794+0000 7f236d800640 1 --2- 172.21.15.83:0/2553894829 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f236819a710 0x7f236819ab90 secure :-1 s=READY pgs=2922 cs=0 l=1 rev1=1 crypto rx=0x7f236000cad0 tx=0x7f236000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:18:56.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:56.794+0000 7f2365600640 1 -- 172.21.15.83:0/2553894829 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f236000ae20 con 0x7f236819a710 2024-04-24T09:18:56.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:56.794+0000 7f2365600640 1 -- 172.21.15.83:0/2553894829 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2360005020 con 0x7f236819a710 2024-04-24T09:18:56.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:56.794+0000 7f236e200640 1 -- 172.21.15.83:0/2553894829 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23681a73f0 con 0x7f236819a710 2024-04-24T09:18:56.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:56.795+0000 7f2365600640 1 -- 172.21.15.83:0/2553894829 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f236000f6b0 con 0x7f236819a710 2024-04-24T09:18:56.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:56.795+0000 7f236e200640 1 -- 172.21.15.83:0/2553894829 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23681a77a0 con 0x7f236819a710 2024-04-24T09:18:56.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:56.796+0000 7f236e200640 1 -- 172.21.15.83:0/2553894829 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23681035f0 con 0x7f236819a710 2024-04-24T09:18:56.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:56.798+0000 7f2365600640 1 -- 172.21.15.83:0/2553894829 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f23600051d0 con 0x7f236819a710 2024-04-24T09:18:56.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:56.799+0000 7f2365600640 1 --2- 172.21.15.83:0/2553894829 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f234c0a3d20 0x7f234c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:56.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:56.800+0000 7f2365600640 1 -- 172.21.15.83:0/2553894829 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 9583+0+0 (secure 0 0 0) 0x7f23600d10d0 con 0x7f236819a710 2024-04-24T09:18:56.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:56.800+0000 7f236ce00640 1 --2- 172.21.15.83:0/2553894829 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f234c0a3d20 0x7f234c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:56.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:56.800+0000 7f236ce00640 1 --2- 172.21.15.83:0/2553894829 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f234c0a3d20 0x7f234c0a61e0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f2358031440 tx=0x7f23580095f0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:18:56.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:56.802+0000 7f2365600640 1 -- 172.21.15.83:0/2553894829 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2360014030 con 0x7f236819a710 2024-04-24T09:18:57.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.004+0000 7f236e200640 1 -- 172.21.15.83:0/2553894829 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2368069190 con 0x7f236819a710 2024-04-24T09:18:57.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.005+0000 7f2365600640 1 -- 172.21.15.83:0/2553894829 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v136) ==== 93+0+10 (secure 0 0 0) 0x7f236009d2f0 con 0x7f236819a710 2024-04-24T09:18:57.005 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-24T09:18:57.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.007+0000 7f236e200640 1 -- 172.21.15.83:0/2553894829 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f234c0a3d20 msgr2=0x7f234c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:57.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.007+0000 7f236e200640 1 --2- 172.21.15.83:0/2553894829 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f234c0a3d20 0x7f234c0a61e0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f2358031440 tx=0x7f23580095f0 comp rx=0 tx=0).stop 2024-04-24T09:18:57.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.007+0000 7f236e200640 1 -- 172.21.15.83:0/2553894829 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f236819a710 msgr2=0x7f236819ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:57.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.007+0000 7f236e200640 1 --2- 172.21.15.83:0/2553894829 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f236819a710 0x7f236819ab90 secure :-1 s=READY pgs=2922 cs=0 l=1 rev1=1 crypto rx=0x7f236000cad0 tx=0x7f236000cf90 comp rx=0 tx=0).stop 2024-04-24T09:18:57.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.008+0000 7f236e200640 1 -- 172.21.15.83:0/2553894829 shutdown_connections 2024-04-24T09:18:57.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.008+0000 7f236e200640 1 --2- 172.21.15.83:0/2553894829 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f234c0a3d20 0x7f234c0a61e0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:57.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.008+0000 7f236e200640 1 --2- 172.21.15.83:0/2553894829 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f236819a710 0x7f236819ab90 unknown :-1 s=CLOSED pgs=2922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:57.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.008+0000 7f236e200640 1 --2- 172.21.15.83:0/2553894829 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23681071f0 0x7f23681a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:57.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.008+0000 7f236e200640 1 --2- 172.21.15.83:0/2553894829 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2368102120 0x7f23681a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:57.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.008+0000 7f236e200640 1 -- 172.21.15.83:0/2553894829 >> 172.21.15.83:0/2553894829 conn(0x7f23680fd9a0 msgr2=0x7f23680ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:18:57.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.008+0000 7f236e200640 1 -- 172.21.15.83:0/2553894829 shutdown_connections 2024-04-24T09:18:57.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.008+0000 7f236e200640 1 -- 172.21.15.83:0/2553894829 wait complete. 2024-04-24T09:18:57.027 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T09:18:57.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.220+0000 7ff4eac00640 1 -- 172.21.15.83:0/3215984828 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4e4109580 msgr2=0x7ff4e4111150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:57.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.221+0000 7ff4eac00640 1 --2- 172.21.15.83:0/3215984828 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4e4109580 0x7ff4e4111150 secure :-1 s=READY pgs=2923 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc009860 tx=0x7ff4dc02a340 comp rx=0 tx=0).stop 2024-04-24T09:18:57.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.221+0000 7ff4eac00640 1 -- 172.21.15.83:0/3215984828 shutdown_connections 2024-04-24T09:18:57.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.221+0000 7ff4eac00640 1 --2- 172.21.15.83:0/3215984828 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4e4109580 0x7ff4e4111150 unknown :-1 s=CLOSED pgs=2923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:57.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.221+0000 7ff4eac00640 1 --2- 172.21.15.83:0/3215984828 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff4e4104e60 0x7ff4e4108ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:57.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.221+0000 7ff4eac00640 1 --2- 172.21.15.83:0/3215984828 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff4e41044b0 0x7ff4e4104890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:57.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.221+0000 7ff4eac00640 1 -- 172.21.15.83:0/3215984828 >> 172.21.15.83:0/3215984828 conn(0x7ff4e40ffd90 msgr2=0x7ff4e41021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:18:57.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.221+0000 7ff4eac00640 1 -- 172.21.15.83:0/3215984828 shutdown_connections 2024-04-24T09:18:57.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.221+0000 7ff4eac00640 1 -- 172.21.15.83:0/3215984828 wait complete. 2024-04-24T09:18:57.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.222+0000 7ff4eac00640 1 Processor -- start 2024-04-24T09:18:57.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.222+0000 7ff4eac00640 1 -- start start 2024-04-24T09:18:57.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.222+0000 7ff4eac00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff4e41044b0 0x7ff4e41a06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:57.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.222+0000 7ff4eac00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff4e4109580 0x7ff4e41a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:57.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.222+0000 7ff4eac00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4e419a890 0x7ff4e419ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:57.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.222+0000 7ff4eac00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e41144f0 con 0x7ff4e419a890 2024-04-24T09:18:57.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.222+0000 7ff4eac00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e4114370 con 0x7ff4e41044b0 2024-04-24T09:18:57.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.222+0000 7ff4eac00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e4114670 con 0x7ff4e4109580 2024-04-24T09:18:57.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.222+0000 7ff4e8c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4e419a890 0x7ff4e419ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:57.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.223+0000 7ff4e8c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4e419a890 0x7ff4e419ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38256/0 (socket says 172.21.15.83:38256) 2024-04-24T09:18:57.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.223+0000 7ff4e8c00640 1 -- 172.21.15.83:0/4012492785 learned_addr learned my addr 172.21.15.83:0/4012492785 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:18:57.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.223+0000 7ff4e3400640 1 --2- 172.21.15.83:0/4012492785 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff4e4109580 0x7ff4e41a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:57.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.223+0000 7ff4e3e00640 1 --2- 172.21.15.83:0/4012492785 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff4e41044b0 0x7ff4e41a06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:57.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.223+0000 7ff4e8c00640 1 -- 172.21.15.83:0/4012492785 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff4e4109580 msgr2=0x7ff4e41a0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:57.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.223+0000 7ff4e8c00640 1 --2- 172.21.15.83:0/4012492785 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff4e4109580 0x7ff4e41a0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:57.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.223+0000 7ff4e8c00640 1 -- 172.21.15.83:0/4012492785 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff4e41044b0 msgr2=0x7ff4e41a06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:57.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.223+0000 7ff4e8c00640 1 --2- 172.21.15.83:0/4012492785 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff4e41044b0 0x7ff4e41a06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:57.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.223+0000 7ff4e8c00640 1 -- 172.21.15.83:0/4012492785 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4e419b3b0 con 0x7ff4e419a890 2024-04-24T09:18:57.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.223+0000 7ff4e8c00640 1 --2- 172.21.15.83:0/4012492785 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4e419a890 0x7ff4e419ad40 secure :-1 s=READY pgs=2924 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc02a930 tx=0x7ff4dc030bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:18:57.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.224+0000 7ff4e0c00640 1 -- 172.21.15.83:0/4012492785 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4dc037050 con 0x7ff4e419a890 2024-04-24T09:18:57.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.224+0000 7ff4e0c00640 1 -- 172.21.15.83:0/4012492785 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff4dc043070 con 0x7ff4e419a890 2024-04-24T09:18:57.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.224+0000 7ff4e0c00640 1 -- 172.21.15.83:0/4012492785 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4dc003b80 con 0x7ff4e419a890 2024-04-24T09:18:57.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.224+0000 7ff4eac00640 1 -- 172.21.15.83:0/4012492785 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4e419b640 con 0x7ff4e419a890 2024-04-24T09:18:57.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.224+0000 7ff4eac00640 1 -- 172.21.15.83:0/4012492785 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4e41a7890 con 0x7ff4e419a890 2024-04-24T09:18:57.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.225+0000 7ff4eac00640 1 -- 172.21.15.83:0/4012492785 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4e4069270 con 0x7ff4e419a890 2024-04-24T09:18:57.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.227+0000 7ff4e0c00640 1 -- 172.21.15.83:0/4012492785 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff4dc030450 con 0x7ff4e419a890 2024-04-24T09:18:57.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.228+0000 7ff4e0c00640 1 --2- 172.21.15.83:0/4012492785 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff4c40a3da0 0x7ff4c40a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:57.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.228+0000 7ff4e0c00640 1 -- 172.21.15.83:0/4012492785 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 9583+0+0 (secure 0 0 0) 0x7ff4dc0f0e10 con 0x7ff4e419a890 2024-04-24T09:18:57.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.229+0000 7ff4e3e00640 1 --2- 172.21.15.83:0/4012492785 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff4c40a3da0 0x7ff4c40a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:57.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.229+0000 7ff4e3e00640 1 --2- 172.21.15.83:0/4012492785 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff4c40a3da0 0x7ff4c40a6260 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7ff4e419b9d0 tx=0x7ff4d4005f90 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:18:57.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.233+0000 7ff4e0c00640 1 -- 172.21.15.83:0/4012492785 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff4dc0f4030 con 0x7ff4e419a890 2024-04-24T09:18:57.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:18:57 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/3134096215' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:18:57.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:18:57 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/2553894829' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T09:18:57.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.439+0000 7ff4eac00640 1 -- 172.21.15.83:0/4012492785 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff4e4104e60 con 0x7ff4e419a890 2024-04-24T09:18:57.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.440+0000 7ff4e0c00640 1 -- 172.21.15.83:0/4012492785 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v136) ==== 104+0+11 (secure 0 0 0) 0x7ff4dc0bd140 con 0x7ff4e419a890 2024-04-24T09:18:57.441 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-24T09:18:57.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.443+0000 7ff4eac00640 1 -- 172.21.15.83:0/4012492785 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff4c40a3da0 msgr2=0x7ff4c40a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:57.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.443+0000 7ff4eac00640 1 --2- 172.21.15.83:0/4012492785 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff4c40a3da0 0x7ff4c40a6260 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7ff4e419b9d0 tx=0x7ff4d4005f90 comp rx=0 tx=0).stop 2024-04-24T09:18:57.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.443+0000 7ff4eac00640 1 -- 172.21.15.83:0/4012492785 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4e419a890 msgr2=0x7ff4e419ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:57.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.443+0000 7ff4eac00640 1 --2- 172.21.15.83:0/4012492785 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4e419a890 0x7ff4e419ad40 secure :-1 s=READY pgs=2924 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc02a930 tx=0x7ff4dc030bd0 comp rx=0 tx=0).stop 2024-04-24T09:18:57.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.443+0000 7ff4eac00640 1 -- 172.21.15.83:0/4012492785 shutdown_connections 2024-04-24T09:18:57.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.443+0000 7ff4eac00640 1 --2- 172.21.15.83:0/4012492785 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff4c40a3da0 0x7ff4c40a6260 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:57.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.443+0000 7ff4eac00640 1 --2- 172.21.15.83:0/4012492785 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4e419a890 0x7ff4e419ad40 unknown :-1 s=CLOSED pgs=2924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:57.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.443+0000 7ff4eac00640 1 --2- 172.21.15.83:0/4012492785 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff4e4109580 0x7ff4e41a0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:57.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.443+0000 7ff4eac00640 1 --2- 172.21.15.83:0/4012492785 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff4e41044b0 0x7ff4e41a06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:57.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.443+0000 7ff4eac00640 1 -- 172.21.15.83:0/4012492785 >> 172.21.15.83:0/4012492785 conn(0x7ff4e40ffd90 msgr2=0x7ff4e41011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:18:57.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.443+0000 7ff4eac00640 1 -- 172.21.15.83:0/4012492785 shutdown_connections 2024-04-24T09:18:57.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.443+0000 7ff4eac00640 1 -- 172.21.15.83:0/4012492785 wait complete. 2024-04-24T09:18:57.461 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T09:18:57.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.648+0000 7f21f6400640 1 -- 172.21.15.83:0/1345169521 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f21f010d500 msgr2=0x7f21f010d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:57.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.648+0000 7f21f6400640 1 --2- 172.21.15.83:0/1345169521 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f21f010d500 0x7f21f010d8e0 secure :-1 s=READY pgs=2925 cs=0 l=1 rev1=1 crypto rx=0x7f21e4009930 tx=0x7f21e402a420 comp rx=0 tx=0).stop 2024-04-24T09:18:57.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.649+0000 7f21f6400640 1 -- 172.21.15.83:0/1345169521 shutdown_connections 2024-04-24T09:18:57.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.649+0000 7f21f6400640 1 --2- 172.21.15.83:0/1345169521 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f21f0108f40 0x7f21f01153b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:57.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.649+0000 7f21f6400640 1 --2- 172.21.15.83:0/1345169521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f21f0108580 0x7f21f0108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:57.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.649+0000 7f21f6400640 1 --2- 172.21.15.83:0/1345169521 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f21f010d500 0x7f21f010d8e0 unknown :-1 s=CLOSED pgs=2925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:57.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.649+0000 7f21f6400640 1 -- 172.21.15.83:0/1345169521 >> 172.21.15.83:0/1345169521 conn(0x7f21f0077e20 msgr2=0x7f21f0106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:18:57.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.649+0000 7f21f6400640 1 -- 172.21.15.83:0/1345169521 shutdown_connections 2024-04-24T09:18:57.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.649+0000 7f21f6400640 1 -- 172.21.15.83:0/1345169521 wait complete. 2024-04-24T09:18:57.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.650+0000 7f21f6400640 1 Processor -- start 2024-04-24T09:18:57.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.650+0000 7f21f6400640 1 -- start start 2024-04-24T09:18:57.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.650+0000 7f21f6400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f21f0108580 0x7f21f01a48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:57.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.650+0000 7f21f6400640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f21f0108f40 0x7f21f01a4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:57.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.650+0000 7f21f6400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f21f010d500 0x7f21f019eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:57.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.650+0000 7f21f6400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f21f01187c0 con 0x7f21f0108580 2024-04-24T09:18:57.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.650+0000 7f21f6400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f21f0118640 con 0x7f21f010d500 2024-04-24T09:18:57.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.650+0000 7f21f6400640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f21f0118940 con 0x7f21f0108f40 2024-04-24T09:18:57.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.650+0000 7f21f5000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f21f0108580 0x7f21f01a48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:57.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.650+0000 7f21f5000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f21f0108580 0x7f21f01a48e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38276/0 (socket says 172.21.15.83:38276) 2024-04-24T09:18:57.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.650+0000 7f21f5000640 1 -- 172.21.15.83:0/2915461451 learned_addr learned my addr 172.21.15.83:0/2915461451 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:18:57.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.651+0000 7f21efe00640 1 --2- 172.21.15.83:0/2915461451 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f21f0108f40 0x7f21f01a4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:57.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.651+0000 7f21f5a00640 1 --2- 172.21.15.83:0/2915461451 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f21f010d500 0x7f21f019eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:57.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.651+0000 7f21f5000640 1 -- 172.21.15.83:0/2915461451 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f21f0108f40 msgr2=0x7f21f01a4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:57.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.651+0000 7f21f5000640 1 --2- 172.21.15.83:0/2915461451 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f21f0108f40 0x7f21f01a4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:57.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.651+0000 7f21f5000640 1 -- 172.21.15.83:0/2915461451 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f21f010d500 msgr2=0x7f21f019eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:57.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.651+0000 7f21f5000640 1 --2- 172.21.15.83:0/2915461451 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f21f010d500 0x7f21f019eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:57.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.651+0000 7f21f5000640 1 -- 172.21.15.83:0/2915461451 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21f019f260 con 0x7f21f0108580 2024-04-24T09:18:57.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.651+0000 7f21efe00640 1 --2- 172.21.15.83:0/2915461451 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f21f0108f40 0x7f21f01a4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:18:57.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.651+0000 7f21f5000640 1 --2- 172.21.15.83:0/2915461451 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f21f0108580 0x7f21f01a48e0 secure :-1 s=READY pgs=2926 cs=0 l=1 rev1=1 crypto rx=0x7f21e4005ab0 tx=0x7f21e4003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:18:57.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.652+0000 7f21ed600640 1 -- 172.21.15.83:0/2915461451 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e403f070 con 0x7f21f0108580 2024-04-24T09:18:57.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.652+0000 7f21ed600640 1 -- 172.21.15.83:0/2915461451 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f21e4031d60 con 0x7f21f0108580 2024-04-24T09:18:57.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.652+0000 7f21f6400640 1 -- 172.21.15.83:0/2915461451 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21f019f4f0 con 0x7f21f0108580 2024-04-24T09:18:57.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.652+0000 7f21ed600640 1 -- 172.21.15.83:0/2915461451 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e4035030 con 0x7f21f0108580 2024-04-24T09:18:57.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.652+0000 7f21f6400640 1 -- 172.21.15.83:0/2915461451 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21f01ab7a0 con 0x7f21f0108580 2024-04-24T09:18:57.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.653+0000 7f21f6400640 1 -- 172.21.15.83:0/2915461451 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21f0069190 con 0x7f21f0108580 2024-04-24T09:18:57.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.655+0000 7f21ed600640 1 -- 172.21.15.83:0/2915461451 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f21e4042070 con 0x7f21f0108580 2024-04-24T09:18:57.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.656+0000 7f21ed600640 1 --2- 172.21.15.83:0/2915461451 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f21d80a3d20 0x7f21d80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:57.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.657+0000 7f21ed600640 1 -- 172.21.15.83:0/2915461451 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 9583+0+0 (secure 0 0 0) 0x7f21e40f1490 con 0x7f21f0108580 2024-04-24T09:18:57.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.657+0000 7f21efe00640 1 --2- 172.21.15.83:0/2915461451 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f21d80a3d20 0x7f21d80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:57.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.657+0000 7f21efe00640 1 --2- 172.21.15.83:0/2915461451 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f21d80a3d20 0x7f21d80a61e0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f21dc005d00 tx=0x7f21dc005c70 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:18:57.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.661+0000 7f21ed600640 1 -- 172.21.15.83:0/2915461451 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f21e40bd6b0 con 0x7f21f0108580 2024-04-24T09:18:57.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:18:57 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/3134096215' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:18:57.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:18:57 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/2553894829' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T09:18:57.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:18:57 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/3134096215' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:18:57.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:18:57 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/2553894829' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T09:18:57.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.861+0000 7f21f6400640 1 -- 172.21.15.83:0/2915461451 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f21f0110a50 con 0x7f21f0108580 2024-04-24T09:18:57.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.862+0000 7f21ed600640 1 -- 172.21.15.83:0/2915461451 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v136) ==== 100+0+11 (secure 0 0 0) 0x7f21e40bcfb0 con 0x7f21f0108580 2024-04-24T09:18:57.863 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2024-04-24T09:18:57.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.864+0000 7f21f6400640 1 -- 172.21.15.83:0/2915461451 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f21d80a3d20 msgr2=0x7f21d80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:57.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.864+0000 7f21f6400640 1 --2- 172.21.15.83:0/2915461451 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f21d80a3d20 0x7f21d80a61e0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f21dc005d00 tx=0x7f21dc005c70 comp rx=0 tx=0).stop 2024-04-24T09:18:57.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.865+0000 7f21f6400640 1 -- 172.21.15.83:0/2915461451 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f21f0108580 msgr2=0x7f21f01a48e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:57.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.865+0000 7f21f6400640 1 --2- 172.21.15.83:0/2915461451 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f21f0108580 0x7f21f01a48e0 secure :-1 s=READY pgs=2926 cs=0 l=1 rev1=1 crypto rx=0x7f21e4005ab0 tx=0x7f21e4003cd0 comp rx=0 tx=0).stop 2024-04-24T09:18:57.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.865+0000 7f21f6400640 1 -- 172.21.15.83:0/2915461451 shutdown_connections 2024-04-24T09:18:57.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.865+0000 7f21f6400640 1 --2- 172.21.15.83:0/2915461451 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f21d80a3d20 0x7f21d80a61e0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:57.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.865+0000 7f21f6400640 1 --2- 172.21.15.83:0/2915461451 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f21f010d500 0x7f21f019eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:57.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.865+0000 7f21f6400640 1 --2- 172.21.15.83:0/2915461451 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f21f0108f40 0x7f21f01a4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:57.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.865+0000 7f21f6400640 1 --2- 172.21.15.83:0/2915461451 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f21f0108580 0x7f21f01a48e0 unknown :-1 s=CLOSED pgs=2926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:57.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.865+0000 7f21f6400640 1 -- 172.21.15.83:0/2915461451 >> 172.21.15.83:0/2915461451 conn(0x7f21f0077e20 msgr2=0x7f21f00783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:18:57.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.865+0000 7f21f6400640 1 -- 172.21.15.83:0/2915461451 shutdown_connections 2024-04-24T09:18:57.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:57.865+0000 7f21f6400640 1 -- 172.21.15.83:0/2915461451 wait complete. 2024-04-24T09:18:57.884 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-24T09:18:58.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:58.088+0000 7f36e6600640 1 -- 172.21.15.83:0/2456545233 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f36e010d500 msgr2=0x7f36e010d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:58.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:58.088+0000 7f36e6600640 1 --2- 172.21.15.83:0/2456545233 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f36e010d500 0x7f36e010d8e0 secure :-1 s=READY pgs=2927 cs=0 l=1 rev1=1 crypto rx=0x7f36d0009cd0 tx=0x7f36d002a8e0 comp rx=0 tx=0).stop 2024-04-24T09:18:58.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:58.088+0000 7f36e6600640 1 -- 172.21.15.83:0/2456545233 shutdown_connections 2024-04-24T09:18:58.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:58.088+0000 7f36e6600640 1 --2- 172.21.15.83:0/2456545233 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f36e0108f40 0x7f36e01153b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:58.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:58.088+0000 7f36e6600640 1 --2- 172.21.15.83:0/2456545233 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f36e0108580 0x7f36e0108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:58.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:58.088+0000 7f36e6600640 1 --2- 172.21.15.83:0/2456545233 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f36e010d500 0x7f36e010d8e0 unknown :-1 s=CLOSED pgs=2927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:58.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:58.088+0000 7f36e6600640 1 -- 172.21.15.83:0/2456545233 >> 172.21.15.83:0/2456545233 conn(0x7f36e0077e20 msgr2=0x7f36e0106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:18:58.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:58.088+0000 7f36e6600640 1 -- 172.21.15.83:0/2456545233 shutdown_connections 2024-04-24T09:18:58.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:58.088+0000 7f36e6600640 1 -- 172.21.15.83:0/2456545233 wait complete. 2024-04-24T09:18:58.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:58.089+0000 7f36e6600640 1 Processor -- start 2024-04-24T09:18:58.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:58.089+0000 7f36e6600640 1 -- start start 2024-04-24T09:18:58.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:58.089+0000 7f36e6600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f36e0108580 0x7f36e01a4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:58.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:58.089+0000 7f36e6600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f36e0108f40 0x7f36e01a4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:58.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:58.090+0000 7f36e6600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f36e010d500 0x7f36e019eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:58.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:58.090+0000 7f36e6600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f36e01187e0 con 0x7f36e010d500 2024-04-24T09:18:58.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:58.090+0000 7f36e6600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f36e0118660 con 0x7f36e0108f40 2024-04-24T09:18:58.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:58.090+0000 7f36e6600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f36e0118960 con 0x7f36e0108580 2024-04-24T09:18:58.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:58.090+0000 7f36e5c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f36e010d500 0x7f36e019eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:58.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:58.090+0000 7f36e5c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f36e010d500 0x7f36e019eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38290/0 (socket says 172.21.15.83:38290) 2024-04-24T09:18:58.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:58.090+0000 7f36e5c00640 1 -- 172.21.15.83:0/1624415320 learned_addr learned my addr 172.21.15.83:0/1624415320 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:18:58.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:58.090+0000 7f36d7e00640 1 --2- 172.21.15.83:0/1624415320 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f36e0108f40 0x7f36e01a4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:58.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:58.090+0000 7f36e5200640 1 --2- 172.21.15.83:0/1624415320 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f36e0108580 0x7f36e01a4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:58.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:58.090+0000 7f36e5c00640 1 -- 172.21.15.83:0/1624415320 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f36e0108580 msgr2=0x7f36e01a4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:58.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:58.090+0000 7f36e5c00640 1 --2- 172.21.15.83:0/1624415320 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f36e0108580 0x7f36e01a4900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:58.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:58.090+0000 7f36e5c00640 1 -- 172.21.15.83:0/1624415320 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f36e0108f40 msgr2=0x7f36e01a4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:58.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:58.090+0000 7f36e5c00640 1 --2- 172.21.15.83:0/1624415320 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f36e0108f40 0x7f36e01a4e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:58.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:58.090+0000 7f36e5c00640 1 -- 172.21.15.83:0/1624415320 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36e019f280 con 0x7f36e010d500 2024-04-24T09:18:58.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:58.090+0000 7f36d7e00640 1 --2- 172.21.15.83:0/1624415320 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f36e0108f40 0x7f36e01a4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:18:58.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:58.091+0000 7f36e5c00640 1 --2- 172.21.15.83:0/1624415320 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f36e010d500 0x7f36e019eb00 secure :-1 s=READY pgs=2928 cs=0 l=1 rev1=1 crypto rx=0x7f36d800c720 tx=0x7f36d800cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:18:58.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:58.091+0000 7f36de000640 1 -- 172.21.15.83:0/1624415320 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d8007bb0 con 0x7f36e010d500 2024-04-24T09:18:58.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:58.091+0000 7f36de000640 1 -- 172.21.15.83:0/1624415320 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f36d800ce30 con 0x7f36e010d500 2024-04-24T09:18:58.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:58.091+0000 7f36de000640 1 -- 172.21.15.83:0/1624415320 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d8004d60 con 0x7f36e010d500 2024-04-24T09:18:58.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:58.091+0000 7f36e6600640 1 -- 172.21.15.83:0/1624415320 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36e019f570 con 0x7f36e010d500 2024-04-24T09:18:58.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:58.091+0000 7f36e6600640 1 -- 172.21.15.83:0/1624415320 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36e01ab7c0 con 0x7f36e010d500 2024-04-24T09:18:58.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:58.092+0000 7f36e6600640 1 -- 172.21.15.83:0/1624415320 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36e0069190 con 0x7f36e010d500 2024-04-24T09:18:58.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:58.094+0000 7f36de000640 1 -- 172.21.15.83:0/1624415320 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f36d8004f40 con 0x7f36e010d500 2024-04-24T09:18:58.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:58.095+0000 7f36de000640 1 --2- 172.21.15.83:0/1624415320 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f36c40a3d20 0x7f36c40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:58.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:58.095+0000 7f36de000640 1 -- 172.21.15.83:0/1624415320 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 9583+0+0 (secure 0 0 0) 0x7f36d80d0870 con 0x7f36e010d500 2024-04-24T09:18:58.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:58.096+0000 7f36e5200640 1 --2- 172.21.15.83:0/1624415320 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f36c40a3d20 0x7f36c40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:58.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:58.096+0000 7f36e5200640 1 --2- 172.21.15.83:0/1624415320 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f36c40a3d20 0x7f36c40a61e0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f36d002aed0 tx=0x7f36d0000f40 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:18:58.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:58.099+0000 7f36de000640 1 -- 172.21.15.83:0/1624415320 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f36d8014030 con 0x7f36e010d500 2024-04-24T09:18:58.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:58.303+0000 7f36e6600640 1 -- 172.21.15.83:0/1624415320 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f36e0110a50 con 0x7f36e010d500 2024-04-24T09:18:58.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:58.303+0000 7f36de000640 1 -- 172.21.15.83:0/1624415320 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v136) ==== 103+0+11 (secure 0 0 0) 0x7f36d809caf0 con 0x7f36e010d500 2024-04-24T09:18:58.304 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-24T09:18:58.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:58.306+0000 7f36e6600640 1 -- 172.21.15.83:0/1624415320 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f36c40a3d20 msgr2=0x7f36c40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:58.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:58.306+0000 7f36e6600640 1 --2- 172.21.15.83:0/1624415320 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f36c40a3d20 0x7f36c40a61e0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f36d002aed0 tx=0x7f36d0000f40 comp rx=0 tx=0).stop 2024-04-24T09:18:58.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:58.306+0000 7f36e6600640 1 -- 172.21.15.83:0/1624415320 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f36e010d500 msgr2=0x7f36e019eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:58.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:58.306+0000 7f36e6600640 1 --2- 172.21.15.83:0/1624415320 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f36e010d500 0x7f36e019eb00 secure :-1 s=READY pgs=2928 cs=0 l=1 rev1=1 crypto rx=0x7f36d800c720 tx=0x7f36d800cbe0 comp rx=0 tx=0).stop 2024-04-24T09:18:58.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:58.306+0000 7f36e6600640 1 -- 172.21.15.83:0/1624415320 shutdown_connections 2024-04-24T09:18:58.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:58.306+0000 7f36e6600640 1 --2- 172.21.15.83:0/1624415320 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f36c40a3d20 0x7f36c40a61e0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:58.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:58.306+0000 7f36e6600640 1 --2- 172.21.15.83:0/1624415320 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f36e010d500 0x7f36e019eb00 unknown :-1 s=CLOSED pgs=2928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:58.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:58.306+0000 7f36e6600640 1 --2- 172.21.15.83:0/1624415320 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f36e0108f40 0x7f36e01a4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:58.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:58.306+0000 7f36e6600640 1 --2- 172.21.15.83:0/1624415320 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f36e0108580 0x7f36e01a4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:58.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:58.306+0000 7f36e6600640 1 -- 172.21.15.83:0/1624415320 >> 172.21.15.83:0/1624415320 conn(0x7f36e0077e20 msgr2=0x7f36e00783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:18:58.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:58.307+0000 7f36e6600640 1 -- 172.21.15.83:0/1624415320 shutdown_connections 2024-04-24T09:18:58.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:58.307+0000 7f36e6600640 1 -- 172.21.15.83:0/1624415320 wait complete. 2024-04-24T09:18:58.326 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-24T09:18:58.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:58.532+0000 7fa901800640 1 -- 172.21.15.83:0/2896892976 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8fc108580 msgr2=0x7fa8fc108a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:58.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:58.532+0000 7fa901800640 1 --2- 172.21.15.83:0/2896892976 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8fc108580 0x7fa8fc108a00 secure :-1 s=READY pgs=2929 cs=0 l=1 rev1=1 crypto rx=0x7fa8e8009930 tx=0x7fa8e802a420 comp rx=0 tx=0).stop 2024-04-24T09:18:58.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:58.532+0000 7fa901800640 1 -- 172.21.15.83:0/2896892976 shutdown_connections 2024-04-24T09:18:58.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:58.532+0000 7fa901800640 1 --2- 172.21.15.83:0/2896892976 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa8fc108f40 0x7fa8fc1153b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:58.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:58.532+0000 7fa901800640 1 --2- 172.21.15.83:0/2896892976 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8fc108580 0x7fa8fc108a00 unknown :-1 s=CLOSED pgs=2929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:58.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:58.532+0000 7fa901800640 1 --2- 172.21.15.83:0/2896892976 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa8fc10d500 0x7fa8fc10d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:58.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:58.532+0000 7fa901800640 1 -- 172.21.15.83:0/2896892976 >> 172.21.15.83:0/2896892976 conn(0x7fa8fc077e20 msgr2=0x7fa8fc106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:18:58.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:58.532+0000 7fa901800640 1 -- 172.21.15.83:0/2896892976 shutdown_connections 2024-04-24T09:18:58.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:58.532+0000 7fa901800640 1 -- 172.21.15.83:0/2896892976 wait complete. 2024-04-24T09:18:58.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:58.533+0000 7fa901800640 1 Processor -- start 2024-04-24T09:18:58.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:58.533+0000 7fa901800640 1 -- start start 2024-04-24T09:18:58.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:58.533+0000 7fa901800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8fc108580 0x7fa8fc1a48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:58.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:58.533+0000 7fa901800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa8fc108f40 0x7fa8fc1a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:58.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:58.533+0000 7fa901800640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa8fc10d500 0x7fa8fc19eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:58.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:58.533+0000 7fa901800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa8fc1187d0 con 0x7fa8fc108580 2024-04-24T09:18:58.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:58.533+0000 7fa901800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa8fc118650 con 0x7fa8fc108f40 2024-04-24T09:18:58.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:58.533+0000 7fa901800640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa8fc118950 con 0x7fa8fc10d500 2024-04-24T09:18:58.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:58.534+0000 7fa8fbe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8fc108580 0x7fa8fc1a48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:58.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:58.534+0000 7fa8fbe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8fc108580 0x7fa8fc1a48f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38310/0 (socket says 172.21.15.83:38310) 2024-04-24T09:18:58.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:58.534+0000 7fa8fbe00640 1 -- 172.21.15.83:0/1955775358 learned_addr learned my addr 172.21.15.83:0/1955775358 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:18:58.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:58.534+0000 7fa900e00640 1 --2- 172.21.15.83:0/1955775358 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa8fc10d500 0x7fa8fc19eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:58.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:58.534+0000 7fa8fbe00640 1 -- 172.21.15.83:0/1955775358 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa8fc10d500 msgr2=0x7fa8fc19eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:58.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:58.534+0000 7fa8fb400640 1 --2- 172.21.15.83:0/1955775358 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa8fc108f40 0x7fa8fc1a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:58.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:58.534+0000 7fa8fbe00640 1 --2- 172.21.15.83:0/1955775358 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa8fc10d500 0x7fa8fc19eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:58.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:58.534+0000 7fa8fbe00640 1 -- 172.21.15.83:0/1955775358 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa8fc108f40 msgr2=0x7fa8fc1a4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:58.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:58.534+0000 7fa8fbe00640 1 --2- 172.21.15.83:0/1955775358 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa8fc108f40 0x7fa8fc1a4e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:58.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:58.534+0000 7fa8fbe00640 1 -- 172.21.15.83:0/1955775358 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8fc19f270 con 0x7fa8fc108580 2024-04-24T09:18:58.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:58.535+0000 7fa8fbe00640 1 --2- 172.21.15.83:0/1955775358 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8fc108580 0x7fa8fc1a48f0 secure :-1 s=READY pgs=2930 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec00d950 tx=0x7fa8ec00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:18:58.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:58.535+0000 7fa8f8c00640 1 -- 172.21.15.83:0/1955775358 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8ec017070 con 0x7fa8fc108580 2024-04-24T09:18:58.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:58.535+0000 7fa8f8c00640 1 -- 172.21.15.83:0/1955775358 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa8ec009c30 con 0x7fa8fc108580 2024-04-24T09:18:58.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:58.535+0000 7fa901800640 1 -- 172.21.15.83:0/1955775358 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8fc19f560 con 0x7fa8fc108580 2024-04-24T09:18:58.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:58.535+0000 7fa8f8c00640 1 -- 172.21.15.83:0/1955775358 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8ec00f620 con 0x7fa8fc108580 2024-04-24T09:18:58.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:58.535+0000 7fa901800640 1 -- 172.21.15.83:0/1955775358 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8fc1ab7b0 con 0x7fa8fc108580 2024-04-24T09:18:58.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:58.536+0000 7fa901800640 1 -- 172.21.15.83:0/1955775358 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8fc19ffd0 con 0x7fa8fc108580 2024-04-24T09:18:58.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:58.538+0000 7fa8f8c00640 1 -- 172.21.15.83:0/1955775358 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa8ec01a070 con 0x7fa8fc108580 2024-04-24T09:18:58.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:58.539+0000 7fa8f8c00640 1 --2- 172.21.15.83:0/1955775358 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa8e00a3d20 0x7fa8e00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:18:58.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:58.540+0000 7fa8f8c00640 1 -- 172.21.15.83:0/1955775358 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 9583+0+0 (secure 0 0 0) 0x7fa8ec0d0750 con 0x7fa8fc108580 2024-04-24T09:18:58.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:58.540+0000 7fa8fb400640 1 --2- 172.21.15.83:0/1955775358 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa8e00a3d20 0x7fa8e00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:18:58.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:58.541+0000 7fa8fb400640 1 --2- 172.21.15.83:0/1955775358 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa8e00a3d20 0x7fa8e00a61e0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fa8e8009230 tx=0x7fa8e8005fd0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:18:58.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:58.544+0000 7fa8f8c00640 1 -- 172.21.15.83:0/1955775358 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa8ec009de0 con 0x7fa8fc108580 2024-04-24T09:18:58.552 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:18:58 smithi083 ceph-mon[27907]: pgmap v865: 129 pgs: 129 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-24T09:18:58.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:18:58 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/4012492785' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T09:18:58.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:18:58 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/2915461451' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T09:18:58.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:18:58 smithi138 ceph-mon[31441]: pgmap v865: 129 pgs: 129 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-24T09:18:58.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:18:58 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/4012492785' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T09:18:58.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:18:58 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/2915461451' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T09:18:58.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:18:58 smithi189 ceph-mon[31791]: pgmap v865: 129 pgs: 129 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-24T09:18:58.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:18:58 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/4012492785' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T09:18:58.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:18:58 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/2915461451' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T09:18:58.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:58.748+0000 7fa901800640 1 -- 172.21.15.83:0/1955775358 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fa8fc069170 con 0x7fa8fc108580 2024-04-24T09:18:58.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:58.748+0000 7fa8f8c00640 1 -- 172.21.15.83:0/1955775358 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v136) ==== 81+0+943 (secure 0 0 0) 0x7fa8ec09c9d0 con 0x7fa8fc108580 2024-04-24T09:18:58.749 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:0/1096290901 2024-04-25T08:51:36.347763+0000 2024-04-24T09:18:58.749 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:6800/2550826560 2024-04-25T08:51:36.347763+0000 2024-04-24T09:18:58.749 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:0/1743153945 2024-04-25T08:51:36.347763+0000 2024-04-24T09:18:58.749 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:0/3306681366 2024-04-25T08:51:36.347763+0000 2024-04-24T09:18:58.749 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:6801/820049883 2024-04-25T08:38:40.661315+0000 2024-04-24T09:18:58.749 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:6801/2550826560 2024-04-25T08:51:36.347763+0000 2024-04-24T09:18:58.750 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:0/4272874138 2024-04-25T08:38:40.661315+0000 2024-04-24T09:18:58.750 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:0/4212994335 2024-04-25T08:38:40.661315+0000 2024-04-24T09:18:58.750 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:6801/1996371091 2024-04-25T08:38:05.800856+0000 2024-04-24T09:18:58.750 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:0/1820663683 2024-04-25T08:38:05.800856+0000 2024-04-24T09:18:58.750 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:0/3802897262 2024-04-25T08:38:05.800856+0000 2024-04-24T09:18:58.750 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:6800/1996371091 2024-04-25T08:38:05.800856+0000 2024-04-24T09:18:58.750 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:6800/820049883 2024-04-25T08:38:40.661315+0000 2024-04-24T09:18:58.750 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:0/1366125049 2024-04-25T08:38:05.800856+0000 2024-04-24T09:18:58.750 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:0/4159820288 2024-04-25T08:51:36.347763+0000 2024-04-24T09:18:58.750 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:0/340452868 2024-04-25T08:38:40.661315+0000 2024-04-24T09:18:58.750 INFO:teuthology.orchestra.run.smithi083.stderr:listed 16 entries 2024-04-24T09:18:58.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:58.751+0000 7fa901800640 1 -- 172.21.15.83:0/1955775358 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa8e00a3d20 msgr2=0x7fa8e00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:58.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:58.751+0000 7fa901800640 1 --2- 172.21.15.83:0/1955775358 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa8e00a3d20 0x7fa8e00a61e0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fa8e8009230 tx=0x7fa8e8005fd0 comp rx=0 tx=0).stop 2024-04-24T09:18:58.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:58.751+0000 7fa901800640 1 -- 172.21.15.83:0/1955775358 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8fc108580 msgr2=0x7fa8fc1a48f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:18:58.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:58.751+0000 7fa901800640 1 --2- 172.21.15.83:0/1955775358 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8fc108580 0x7fa8fc1a48f0 secure :-1 s=READY pgs=2930 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec00d950 tx=0x7fa8ec00de10 comp rx=0 tx=0).stop 2024-04-24T09:18:58.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:58.751+0000 7fa901800640 1 -- 172.21.15.83:0/1955775358 shutdown_connections 2024-04-24T09:18:58.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:58.751+0000 7fa901800640 1 --2- 172.21.15.83:0/1955775358 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fa8e00a3d20 0x7fa8e00a61e0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:58.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:58.751+0000 7fa901800640 1 --2- 172.21.15.83:0/1955775358 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa8fc10d500 0x7fa8fc19eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:58.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:58.751+0000 7fa901800640 1 --2- 172.21.15.83:0/1955775358 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa8fc108f40 0x7fa8fc1a4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:58.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:58.751+0000 7fa901800640 1 --2- 172.21.15.83:0/1955775358 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8fc108580 0x7fa8fc1a48f0 unknown :-1 s=CLOSED pgs=2930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:18:58.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:58.751+0000 7fa901800640 1 -- 172.21.15.83:0/1955775358 >> 172.21.15.83:0/1955775358 conn(0x7fa8fc077e20 msgr2=0x7fa8fc0783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:18:58.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:58.752+0000 7fa901800640 1 -- 172.21.15.83:0/1955775358 shutdown_connections 2024-04-24T09:18:58.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:18:58.752+0000 7fa901800640 1 -- 172.21.15.83:0/1955775358 wait complete. 2024-04-24T09:18:58.770 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi083.front.sepia.ceph.com... 2024-04-24T09:18:58.771 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T09:18:58.771 DEBUG:teuthology.orchestra.run.smithi083:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-24T09:18:58.867 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi083.stderr:ceph-fuse[82517]: fuse finished with error 0 and tester_r 0 2024-04-24T09:18:58.868 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T09:18:59.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:18:59 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/1624415320' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T09:18:59.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:18:59 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/1955775358' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-24T09:18:59.704 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:18:59 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/1624415320' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T09:18:59.704 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:18:59 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/1955775358' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-24T09:18:59.706 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:18:59 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/1624415320' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T09:18:59.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:18:59 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/1955775358' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-24T09:19:00.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:19:00 smithi083 ceph-mon[27907]: pgmap v866: 129 pgs: 129 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-24T09:19:00.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:19:00 smithi138 ceph-mon[31441]: pgmap v866: 129 pgs: 129 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-24T09:19:00.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:19:00 smithi189 ceph-mon[31791]: pgmap v866: 129 pgs: 129 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-24T09:19:02.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:19:02 smithi083 ceph-mon[27907]: pgmap v867: 129 pgs: 129 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-24T09:19:02.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:19:02 smithi138 ceph-mon[31441]: pgmap v867: 129 pgs: 129 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-24T09:19:02.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:19:02 smithi189 ceph-mon[31791]: pgmap v867: 129 pgs: 129 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-24T09:19:04.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:19:04 smithi083 ceph-mon[27907]: pgmap v868: 129 pgs: 129 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 46 KiB/s wr, 32 op/s 2024-04-24T09:19:04.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:19:04 smithi138 ceph-mon[31441]: pgmap v868: 129 pgs: 129 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 46 KiB/s wr, 32 op/s 2024-04-24T09:19:04.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:19:04 smithi189 ceph-mon[31791]: pgmap v868: 129 pgs: 129 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 46 KiB/s wr, 32 op/s 2024-04-24T09:19:04.970 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi083.front.sepia.ceph.com 2024-04-24T09:19:04.970 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T09:19:04.970 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-24T09:19:05.093 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T09:19:05.093 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list 2024-04-24T09:19:05.247 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-24T09:19:05.249 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T09:19:05.249 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T09:19:05.343 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T09:19:05.343 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link delete ceph-brx 2024-04-24T09:19:05.443 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-04-24T09:19:05.454 INFO:tasks.cephadm:Teardown begin 2024-04-24T09:19:05.455 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-24T09:19:05.455 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T09:19:05.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:05.639+0000 7fabb9600640 1 -- 172.21.15.83:0/870308636 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fabb4109510 msgr2=0x7fabb41110e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:05.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:05.639+0000 7fabb9600640 1 --2- 172.21.15.83:0/870308636 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fabb4109510 0x7fabb41110e0 secure :-1 s=READY pgs=2931 cs=0 l=1 rev1=1 crypto rx=0x7fabac009ca0 tx=0x7fabac02a8c0 comp rx=0 tx=0).stop 2024-04-24T09:19:05.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:05.639+0000 7fabb9600640 1 -- 172.21.15.83:0/870308636 shutdown_connections 2024-04-24T09:19:05.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:05.639+0000 7fabb9600640 1 --2- 172.21.15.83:0/870308636 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fabb4109510 0x7fabb41110e0 unknown :-1 s=CLOSED pgs=2931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:05.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:05.639+0000 7fabb9600640 1 --2- 172.21.15.83:0/870308636 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fabb4104df0 0x7fabb4108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:05.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:05.639+0000 7fabb9600640 1 --2- 172.21.15.83:0/870308636 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fabb4104440 0x7fabb4104820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:05.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:05.639+0000 7fabb9600640 1 -- 172.21.15.83:0/870308636 >> 172.21.15.83:0/870308636 conn(0x7fabb40ffd00 msgr2=0x7fabb4102120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:19:05.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:05.639+0000 7fabb9600640 1 -- 172.21.15.83:0/870308636 shutdown_connections 2024-04-24T09:19:05.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:05.639+0000 7fabb9600640 1 -- 172.21.15.83:0/870308636 wait complete. 2024-04-24T09:19:05.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:05.640+0000 7fabb9600640 1 Processor -- start 2024-04-24T09:19:05.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:05.640+0000 7fabb9600640 1 -- start start 2024-04-24T09:19:05.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:05.641+0000 7fabb9600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fabb4104440 0x7fabb41a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:05.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:05.641+0000 7fabb9600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fabb4109510 0x7fabb41a0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:05.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:05.641+0000 7fabb9600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fabb419a880 0x7fabb419ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:05.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:05.641+0000 7fabb9600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fabb4114520 con 0x7fabb4109510 2024-04-24T09:19:05.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:05.641+0000 7fabb9600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fabb41143a0 con 0x7fabb4104440 2024-04-24T09:19:05.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:05.641+0000 7fabb9600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fabb41146a0 con 0x7fabb419a880 2024-04-24T09:19:05.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:05.641+0000 7fababe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fabb4109510 0x7fabb41a0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:05.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:05.641+0000 7fababe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fabb4109510 0x7fabb41a0c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47554/0 (socket says 172.21.15.83:47554) 2024-04-24T09:19:05.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:05.641+0000 7fababe00640 1 -- 172.21.15.83:0/1579313991 learned_addr learned my addr 172.21.15.83:0/1579313991 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:19:05.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:05.641+0000 7fabb2a00640 1 --2- 172.21.15.83:0/1579313991 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fabb4104440 0x7fabb41a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:05.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:05.641+0000 7fabb3400640 1 --2- 172.21.15.83:0/1579313991 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fabb419a880 0x7fabb419ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:05.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:05.642+0000 7fababe00640 1 -- 172.21.15.83:0/1579313991 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fabb419a880 msgr2=0x7fabb419ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:05.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:05.642+0000 7fababe00640 1 --2- 172.21.15.83:0/1579313991 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fabb419a880 0x7fabb419ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:05.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:05.642+0000 7fababe00640 1 -- 172.21.15.83:0/1579313991 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fabb4104440 msgr2=0x7fabb41a06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:05.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:05.642+0000 7fababe00640 1 --2- 172.21.15.83:0/1579313991 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fabb4104440 0x7fabb41a06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:05.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:05.642+0000 7fababe00640 1 -- 172.21.15.83:0/1579313991 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabb419b410 con 0x7fabb4109510 2024-04-24T09:19:05.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:05.642+0000 7fababe00640 1 --2- 172.21.15.83:0/1579313991 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fabb4109510 0x7fabb41a0c00 secure :-1 s=READY pgs=2932 cs=0 l=1 rev1=1 crypto rx=0x7fab9c00b450 tx=0x7fab9c00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:19:05.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:05.642+0000 7fabab400640 1 -- 172.21.15.83:0/1579313991 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab9c015020 con 0x7fabb4109510 2024-04-24T09:19:05.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:05.642+0000 7fabab400640 1 -- 172.21.15.83:0/1579313991 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fab9c011d40 con 0x7fabb4109510 2024-04-24T09:19:05.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:05.642+0000 7fabab400640 1 -- 172.21.15.83:0/1579313991 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab9c00f340 con 0x7fabb4109510 2024-04-24T09:19:05.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:05.642+0000 7fabb9600640 1 -- 172.21.15.83:0/1579313991 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabb419b700 con 0x7fabb4109510 2024-04-24T09:19:05.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:05.642+0000 7fabb9600640 1 -- 172.21.15.83:0/1579313991 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabb41a7840 con 0x7fabb4109510 2024-04-24T09:19:05.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:05.643+0000 7fabb9600640 1 -- 172.21.15.83:0/1579313991 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabb4105910 con 0x7fabb4109510 2024-04-24T09:19:05.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:05.645+0000 7fabab400640 1 -- 172.21.15.83:0/1579313991 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fab9c018020 con 0x7fabb4109510 2024-04-24T09:19:05.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:05.647+0000 7fabab400640 1 --2- 172.21.15.83:0/1579313991 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fab980a3da0 0x7fab980a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:05.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:05.647+0000 7fabab400640 1 -- 172.21.15.83:0/1579313991 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 9583+0+0 (secure 0 0 0) 0x7fab9c0d0640 con 0x7fabb4109510 2024-04-24T09:19:05.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:05.647+0000 7fabb2a00640 1 --2- 172.21.15.83:0/1579313991 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fab980a3da0 0x7fab980a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:05.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:05.648+0000 7fabb2a00640 1 --2- 172.21.15.83:0/1579313991 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fab980a3da0 0x7fab980a6260 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7faba40058c0 tx=0x7faba400a4c0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:19:05.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:05.652+0000 7fabab400640 1 -- 172.21.15.83:0/1579313991 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fab9c01f020 con 0x7fabb4109510 2024-04-24T09:19:05.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:05.898+0000 7fabb9600640 1 -- 172.21.15.83:0/1579313991 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fabb4104df0 con 0x7fabb4109510 2024-04-24T09:19:05.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:05.899+0000 7fabab400640 1 -- 172.21.15.83:0/1579313991 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) ==== 94+0+11957 (secure 0 0 0) 0x7fabb419b700 con 0x7fabb4109510 2024-04-24T09:19:05.900 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T09:19:05.900 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":79,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/358872257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":358872257},{"type":"v1","addr":"172.21.15.83:6835","nonce":358872257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3157070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3157070851},{"type":"v1","addr":"172.21.15.83:6837","nonce":3157070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":79,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T08:48:29.422576+0000","modified":"2024-04-24T09:17:48.323281+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34520,"mds_1":34493,"mds_2":34451,"mds_3":34427,"mds_4":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14658":{"gid":14658,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6839/4085470844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":4085470844},{"type":"v1","addr":"172.21.15.83:6839","nonce":4085470844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14676":{"gid":14676,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6841/2634754210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":2634754210},{"type":"v1","addr":"172.21.15.83:6841","nonce":2634754210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24382":{"gid":24382,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/1859921310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1859921310},{"type":"v1","addr":"172.21.15.189:6833","nonce":1859921310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/2241923197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2241923197},{"type":"v1","addr":"172.21.15.189:6835","nonce":2241923197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.189:6839/2548965336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":2548965336},{"type":"v1","addr":"172.21.15.189:6839","nonce":2548965336}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26686":{"gid":26686,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6837/1678817161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":1678817161},{"type":"v1","addr":"172.21.15.189:6837","nonce":1678817161}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.138:6833/1157967658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1157967658},{"type":"v1","addr":"172.21.15.138:6833","nonce":1157967658}]},"join_fscid":-1,"export_targets":[0,1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.138:6835/1909219758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1909219758},{"type":"v1","addr":"172.21.15.138:6835","nonce":1909219758}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34493":{"gid":34493,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.138:6837/900566754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":900566754},{"type":"v1","addr":"172.21.15.138:6837","nonce":900566754}]},"join_fscid":-1,"export_targets":[0,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34520":{"gid":34520,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.138:6839/2236932393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2236932393},{"type":"v1","addr":"172.21.15.138:6839","nonce":2236932393}]},"join_fscid":-1,"export_targets":[4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34520,"qdb_cluster":[34520,24454,34427,34493,34451]},"id":1}]} 2024-04-24T09:19:05.901 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 79 2024-04-24T09:19:05.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:05.902+0000 7fabb9600640 1 -- 172.21.15.83:0/1579313991 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fab980a3da0 msgr2=0x7fab980a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:05.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:05.902+0000 7fabb9600640 1 --2- 172.21.15.83:0/1579313991 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fab980a3da0 0x7fab980a6260 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7faba40058c0 tx=0x7faba400a4c0 comp rx=0 tx=0).stop 2024-04-24T09:19:05.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:05.902+0000 7fabb9600640 1 -- 172.21.15.83:0/1579313991 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fabb4109510 msgr2=0x7fabb41a0c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:05.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:05.902+0000 7fabb9600640 1 --2- 172.21.15.83:0/1579313991 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fabb4109510 0x7fabb41a0c00 secure :-1 s=READY pgs=2932 cs=0 l=1 rev1=1 crypto rx=0x7fab9c00b450 tx=0x7fab9c00b910 comp rx=0 tx=0).stop 2024-04-24T09:19:05.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:05.902+0000 7fabb9600640 1 -- 172.21.15.83:0/1579313991 shutdown_connections 2024-04-24T09:19:05.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:05.902+0000 7fabb9600640 1 --2- 172.21.15.83:0/1579313991 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fab980a3da0 0x7fab980a6260 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:05.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:05.902+0000 7fabb9600640 1 --2- 172.21.15.83:0/1579313991 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fabb419a880 0x7fabb419ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:05.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:05.902+0000 7fabb9600640 1 --2- 172.21.15.83:0/1579313991 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fabb4109510 0x7fabb41a0c00 unknown :-1 s=CLOSED pgs=2932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:05.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:05.902+0000 7fabb9600640 1 --2- 172.21.15.83:0/1579313991 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fabb4104440 0x7fabb41a06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:05.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:05.902+0000 7fabb9600640 1 -- 172.21.15.83:0/1579313991 >> 172.21.15.83:0/1579313991 conn(0x7fabb40ffd00 msgr2=0x7fabb41020a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:19:05.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:05.903+0000 7fabb9600640 1 -- 172.21.15.83:0/1579313991 shutdown_connections 2024-04-24T09:19:05.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:05.903+0000 7fabb9600640 1 -- 172.21.15.83:0/1579313991 wait complete. 2024-04-24T09:19:05.922 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T09:19:06.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:06.113+0000 7feb63e00640 1 -- 172.21.15.83:0/2098364558 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb64104c20 msgr2=0x7feb64108c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:06.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:06.113+0000 7feb63e00640 1 --2- 172.21.15.83:0/2098364558 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb64104c20 0x7feb64108c60 secure :-1 s=READY pgs=2933 cs=0 l=1 rev1=1 crypto rx=0x7feb4c009900 tx=0x7feb4c02a410 comp rx=0 tx=0).stop 2024-04-24T09:19:06.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:06.113+0000 7feb63e00640 1 -- 172.21.15.83:0/2098364558 shutdown_connections 2024-04-24T09:19:06.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:06.113+0000 7feb63e00640 1 --2- 172.21.15.83:0/2098364558 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb64109340 0x7feb64110f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:06.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:06.113+0000 7feb63e00640 1 --2- 172.21.15.83:0/2098364558 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb64104c20 0x7feb64108c60 unknown :-1 s=CLOSED pgs=2933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:06.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:06.113+0000 7feb63e00640 1 --2- 172.21.15.83:0/2098364558 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feb64104270 0x7feb64104650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:06.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:06.113+0000 7feb63e00640 1 -- 172.21.15.83:0/2098364558 >> 172.21.15.83:0/2098364558 conn(0x7feb640ffad0 msgr2=0x7feb64101ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:19:06.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:06.113+0000 7feb63e00640 1 -- 172.21.15.83:0/2098364558 shutdown_connections 2024-04-24T09:19:06.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:06.113+0000 7feb63e00640 1 -- 172.21.15.83:0/2098364558 wait complete. 2024-04-24T09:19:06.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:06.114+0000 7feb63e00640 1 Processor -- start 2024-04-24T09:19:06.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:06.114+0000 7feb63e00640 1 -- start start 2024-04-24T09:19:06.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:06.115+0000 7feb63e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb64104270 0x7feb641a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:06.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:06.115+0000 7feb63e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feb64109340 0x7feb641a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:06.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:06.115+0000 7feb63e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb6419a770 0x7feb6419ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:06.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:06.115+0000 7feb63e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7feb64114280 con 0x7feb6419a770 2024-04-24T09:19:06.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:06.115+0000 7feb63e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7feb64114100 con 0x7feb64104270 2024-04-24T09:19:06.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:06.115+0000 7feb63e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7feb64114400 con 0x7feb64109340 2024-04-24T09:19:06.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:06.115+0000 7feb63400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb6419a770 0x7feb6419ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:06.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:06.115+0000 7feb63400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb6419a770 0x7feb6419ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47560/0 (socket says 172.21.15.83:47560) 2024-04-24T09:19:06.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:06.115+0000 7feb63400640 1 -- 172.21.15.83:0/2999168277 learned_addr learned my addr 172.21.15.83:0/2999168277 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:19:06.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:06.115+0000 7feb62a00640 1 --2- 172.21.15.83:0/2999168277 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb64104270 0x7feb641a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:06.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:06.115+0000 7feb5a000640 1 --2- 172.21.15.83:0/2999168277 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feb64109340 0x7feb641a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:06.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:06.116+0000 7feb63400640 1 -- 172.21.15.83:0/2999168277 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feb64109340 msgr2=0x7feb641a0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:06.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:06.116+0000 7feb63400640 1 --2- 172.21.15.83:0/2999168277 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feb64109340 0x7feb641a0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:06.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:06.116+0000 7feb63400640 1 -- 172.21.15.83:0/2999168277 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb64104270 msgr2=0x7feb641a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:06.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:06.116+0000 7feb63400640 1 --2- 172.21.15.83:0/2999168277 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb64104270 0x7feb641a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:06.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:06.116+0000 7feb63400640 1 -- 172.21.15.83:0/2999168277 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb6419b350 con 0x7feb6419a770 2024-04-24T09:19:06.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:06.116+0000 7feb5a000640 1 --2- 172.21.15.83:0/2999168277 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feb64109340 0x7feb641a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:19:06.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:06.116+0000 7feb63400640 1 --2- 172.21.15.83:0/2999168277 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb6419a770 0x7feb6419ac20 secure :-1 s=READY pgs=2934 cs=0 l=1 rev1=1 crypto rx=0x7feb5c00cad0 tx=0x7feb5c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:19:06.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:06.116+0000 7feb5be00640 1 -- 172.21.15.83:0/2999168277 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb5c00ae20 con 0x7feb6419a770 2024-04-24T09:19:06.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:06.116+0000 7feb5be00640 1 -- 172.21.15.83:0/2999168277 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feb5c005020 con 0x7feb6419a770 2024-04-24T09:19:06.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:06.116+0000 7feb63e00640 1 -- 172.21.15.83:0/2999168277 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb641a7430 con 0x7feb6419a770 2024-04-24T09:19:06.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:06.116+0000 7feb5be00640 1 -- 172.21.15.83:0/2999168277 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb5c00f6b0 con 0x7feb6419a770 2024-04-24T09:19:06.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:06.116+0000 7feb63e00640 1 -- 172.21.15.83:0/2999168277 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb641a77e0 con 0x7feb6419a770 2024-04-24T09:19:06.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:06.118+0000 7feb63e00640 1 -- 172.21.15.83:0/2999168277 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb64069190 con 0x7feb6419a770 2024-04-24T09:19:06.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:06.120+0000 7feb5be00640 1 -- 172.21.15.83:0/2999168277 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7feb5c0051d0 con 0x7feb6419a770 2024-04-24T09:19:06.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:06.121+0000 7feb5be00640 1 --2- 172.21.15.83:0/2999168277 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7feb480a3d20 0x7feb480a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:06.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:06.121+0000 7feb5be00640 1 -- 172.21.15.83:0/2999168277 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 9583+0+0 (secure 0 0 0) 0x7feb5c0d0ca0 con 0x7feb6419a770 2024-04-24T09:19:06.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:06.121+0000 7feb62a00640 1 --2- 172.21.15.83:0/2999168277 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7feb480a3d20 0x7feb480a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:06.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:06.122+0000 7feb62a00640 1 --2- 172.21.15.83:0/2999168277 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7feb480a3d20 0x7feb480a61e0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7feb54005fd0 tx=0x7feb54005850 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:19:06.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:06.126+0000 7feb5be00640 1 -- 172.21.15.83:0/2999168277 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7feb5c014030 con 0x7feb6419a770 2024-04-24T09:19:06.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:06.373+0000 7feb63e00640 1 -- 172.21.15.83:0/2999168277 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feb6410c5d0 con 0x7feb6419a770 2024-04-24T09:19:06.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:06.374+0000 7feb5be00640 1 -- 172.21.15.83:0/2999168277 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) ==== 94+0+11957 (secure 0 0 0) 0x7feb5c09cec0 con 0x7feb6419a770 2024-04-24T09:19:06.375 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T09:19:06.375 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":79,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/358872257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":358872257},{"type":"v1","addr":"172.21.15.83:6835","nonce":358872257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3157070851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3157070851},{"type":"v1","addr":"172.21.15.83:6837","nonce":3157070851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":79,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T08:48:29.422576+0000","modified":"2024-04-24T09:17:48.323281+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34520,"mds_1":34493,"mds_2":34451,"mds_3":34427,"mds_4":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14658":{"gid":14658,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6839/4085470844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":4085470844},{"type":"v1","addr":"172.21.15.83:6839","nonce":4085470844}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14676":{"gid":14676,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6841/2634754210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":2634754210},{"type":"v1","addr":"172.21.15.83:6841","nonce":2634754210}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24382":{"gid":24382,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/1859921310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1859921310},{"type":"v1","addr":"172.21.15.189:6833","nonce":1859921310}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/2241923197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2241923197},{"type":"v1","addr":"172.21.15.189:6835","nonce":2241923197}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.189:6839/2548965336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":2548965336},{"type":"v1","addr":"172.21.15.189:6839","nonce":2548965336}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26686":{"gid":26686,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6837/1678817161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":1678817161},{"type":"v1","addr":"172.21.15.189:6837","nonce":1678817161}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34427":{"gid":34427,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.138:6833/1157967658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1157967658},{"type":"v1","addr":"172.21.15.138:6833","nonce":1157967658}]},"join_fscid":-1,"export_targets":[0,1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34451":{"gid":34451,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.138:6835/1909219758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1909219758},{"type":"v1","addr":"172.21.15.138:6835","nonce":1909219758}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34493":{"gid":34493,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":15,"addr":"172.21.15.138:6837/900566754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":900566754},{"type":"v1","addr":"172.21.15.138:6837","nonce":900566754}]},"join_fscid":-1,"export_targets":[0,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34520":{"gid":34520,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.138:6839/2236932393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":2236932393},{"type":"v1","addr":"172.21.15.138:6839","nonce":2236932393}]},"join_fscid":-1,"export_targets":[4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34520,"qdb_cluster":[34520,24454,34427,34493,34451]},"id":1}]} 2024-04-24T09:19:06.375 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 79 2024-04-24T09:19:06.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:06.377+0000 7feb63e00640 1 -- 172.21.15.83:0/2999168277 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7feb480a3d20 msgr2=0x7feb480a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:06.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:06.377+0000 7feb63e00640 1 --2- 172.21.15.83:0/2999168277 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7feb480a3d20 0x7feb480a61e0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7feb54005fd0 tx=0x7feb54005850 comp rx=0 tx=0).stop 2024-04-24T09:19:06.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:06.377+0000 7feb63e00640 1 -- 172.21.15.83:0/2999168277 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb6419a770 msgr2=0x7feb6419ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:06.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:06.377+0000 7feb63e00640 1 --2- 172.21.15.83:0/2999168277 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb6419a770 0x7feb6419ac20 secure :-1 s=READY pgs=2934 cs=0 l=1 rev1=1 crypto rx=0x7feb5c00cad0 tx=0x7feb5c00cf90 comp rx=0 tx=0).stop 2024-04-24T09:19:06.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:06.377+0000 7feb63e00640 1 -- 172.21.15.83:0/2999168277 shutdown_connections 2024-04-24T09:19:06.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:06.377+0000 7feb63e00640 1 --2- 172.21.15.83:0/2999168277 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7feb480a3d20 0x7feb480a61e0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:06.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:06.377+0000 7feb63e00640 1 --2- 172.21.15.83:0/2999168277 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb6419a770 0x7feb6419ac20 unknown :-1 s=CLOSED pgs=2934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:06.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:06.377+0000 7feb63e00640 1 --2- 172.21.15.83:0/2999168277 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feb64109340 0x7feb641a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:06.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:06.377+0000 7feb63e00640 1 --2- 172.21.15.83:0/2999168277 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb64104270 0x7feb641a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:06.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:06.377+0000 7feb63e00640 1 -- 172.21.15.83:0/2999168277 >> 172.21.15.83:0/2999168277 conn(0x7feb640ffad0 msgr2=0x7feb64100e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:19:06.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:06.378+0000 7feb63e00640 1 -- 172.21.15.83:0/2999168277 shutdown_connections 2024-04-24T09:19:06.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:06.378+0000 7feb63e00640 1 -- 172.21.15.83:0/2999168277 wait complete. 2024-04-24T09:19:06.396 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T09:19:06.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:19:06 smithi083 ceph-mon[27907]: pgmap v869: 129 pgs: 129 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 46 KiB/s wr, 28 op/s 2024-04-24T09:19:06.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:19:06 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/1579313991' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:19:06.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:06.590+0000 7fb7ff000640 1 -- 172.21.15.83:0/2196873850 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7f8108e40 msgr2=0x7fb7f81152b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:06.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:06.590+0000 7fb7ff000640 1 --2- 172.21.15.83:0/2196873850 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7f8108e40 0x7fb7f81152b0 secure :-1 s=READY pgs=2935 cs=0 l=1 rev1=1 crypto rx=0x7fb7f0009ca0 tx=0x7fb7f002a950 comp rx=0 tx=0).stop 2024-04-24T09:19:06.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:06.591+0000 7fb7ff000640 1 -- 172.21.15.83:0/2196873850 shutdown_connections 2024-04-24T09:19:06.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:06.591+0000 7fb7ff000640 1 --2- 172.21.15.83:0/2196873850 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7f8108e40 0x7fb7f81152b0 unknown :-1 s=CLOSED pgs=2935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:06.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:06.591+0000 7fb7ff000640 1 --2- 172.21.15.83:0/2196873850 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb7f81084a0 0x7fb7f8108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:06.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:06.591+0000 7fb7ff000640 1 --2- 172.21.15.83:0/2196873850 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb7f810d3c0 0x7fb7f810d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:06.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:06.591+0000 7fb7ff000640 1 -- 172.21.15.83:0/2196873850 >> 172.21.15.83:0/2196873850 conn(0x7fb7f8077b80 msgr2=0x7fb7f8079fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:19:06.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:06.591+0000 7fb7ff000640 1 -- 172.21.15.83:0/2196873850 shutdown_connections 2024-04-24T09:19:06.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:06.591+0000 7fb7ff000640 1 -- 172.21.15.83:0/2196873850 wait complete. 2024-04-24T09:19:06.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:06.592+0000 7fb7ff000640 1 Processor -- start 2024-04-24T09:19:06.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:06.592+0000 7fb7ff000640 1 -- start start 2024-04-24T09:19:06.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:06.592+0000 7fb7ff000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7f81084a0 0x7fb7f81a47b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:06.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:06.592+0000 7fb7ff000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb7f8108e40 0x7fb7f81a4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:06.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:06.592+0000 7fb7ff000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb7f810d3c0 0x7fb7f819e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:06.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:06.592+0000 7fb7ff000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f8118700 con 0x7fb7f81084a0 2024-04-24T09:19:06.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:06.592+0000 7fb7ff000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f8118580 con 0x7fb7f810d3c0 2024-04-24T09:19:06.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:06.592+0000 7fb7ff000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f8118880 con 0x7fb7f8108e40 2024-04-24T09:19:06.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:06.592+0000 7fb7fdc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7f81084a0 0x7fb7f81a47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:06.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:06.592+0000 7fb7fdc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7f81084a0 0x7fb7f81a47b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47576/0 (socket says 172.21.15.83:47576) 2024-04-24T09:19:06.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:06.592+0000 7fb7fdc00640 1 -- 172.21.15.83:0/2744839667 learned_addr learned my addr 172.21.15.83:0/2744839667 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:19:06.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:06.593+0000 7fb7fd200640 1 --2- 172.21.15.83:0/2744839667 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb7f8108e40 0x7fb7f81a4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:06.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:06.593+0000 7fb7fe600640 1 --2- 172.21.15.83:0/2744839667 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb7f810d3c0 0x7fb7f819e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:06.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:06.593+0000 7fb7fdc00640 1 -- 172.21.15.83:0/2744839667 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb7f8108e40 msgr2=0x7fb7f81a4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:06.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:06.593+0000 7fb7fdc00640 1 --2- 172.21.15.83:0/2744839667 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb7f8108e40 0x7fb7f81a4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:06.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:06.593+0000 7fb7fdc00640 1 -- 172.21.15.83:0/2744839667 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb7f810d3c0 msgr2=0x7fb7f819e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:06.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:06.593+0000 7fb7fdc00640 1 --2- 172.21.15.83:0/2744839667 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb7f810d3c0 0x7fb7f819e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:06.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:06.593+0000 7fb7fdc00640 1 -- 172.21.15.83:0/2744839667 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7f819f0e0 con 0x7fb7f81084a0 2024-04-24T09:19:06.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:06.593+0000 7fb7fd200640 1 --2- 172.21.15.83:0/2744839667 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb7f8108e40 0x7fb7f81a4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:19:06.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:06.593+0000 7fb7fe600640 1 --2- 172.21.15.83:0/2744839667 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb7f810d3c0 0x7fb7f819e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:19:06.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:06.593+0000 7fb7fdc00640 1 --2- 172.21.15.83:0/2744839667 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7f81084a0 0x7fb7f81a47b0 secure :-1 s=READY pgs=2936 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec00c970 tx=0x7fb7ec00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:19:06.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:06.593+0000 7fb7f6000640 1 -- 172.21.15.83:0/2744839667 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7ec007bb0 con 0x7fb7f81084a0 2024-04-24T09:19:06.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:06.593+0000 7fb7f6000640 1 -- 172.21.15.83:0/2744839667 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb7ec007d50 con 0x7fb7f81084a0 2024-04-24T09:19:06.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:06.593+0000 7fb7ff000640 1 -- 172.21.15.83:0/2744839667 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7f819f3d0 con 0x7fb7f81084a0 2024-04-24T09:19:06.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:06.593+0000 7fb7f6000640 1 -- 172.21.15.83:0/2744839667 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7ec00f450 con 0x7fb7f81084a0 2024-04-24T09:19:06.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:06.593+0000 7fb7ff000640 1 -- 172.21.15.83:0/2744839667 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7f81ab670 con 0x7fb7f81084a0 2024-04-24T09:19:06.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:06.595+0000 7fb7ff000640 1 -- 172.21.15.83:0/2744839667 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7f8069190 con 0x7fb7f81084a0 2024-04-24T09:19:06.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:06.596+0000 7fb7f6000640 1 -- 172.21.15.83:0/2744839667 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb7ec004f20 con 0x7fb7f81084a0 2024-04-24T09:19:06.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:06.598+0000 7fb7f6000640 1 --2- 172.21.15.83:0/2744839667 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb7e00a3da0 0x7fb7e00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:06.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:06.598+0000 7fb7f6000640 1 -- 172.21.15.83:0/2744839667 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 9583+0+0 (secure 0 0 0) 0x7fb7ec0d09b0 con 0x7fb7f81084a0 2024-04-24T09:19:06.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:06.598+0000 7fb7fd200640 1 --2- 172.21.15.83:0/2744839667 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb7e00a3da0 0x7fb7e00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:06.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:06.599+0000 7fb7fd200640 1 --2- 172.21.15.83:0/2744839667 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb7e00a3da0 0x7fb7e00a6260 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fb7f807a2e0 tx=0x7fb7e4005d90 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:19:06.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:06.602+0000 7fb7f6000640 1 -- 172.21.15.83:0/2744839667 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb7ec014030 con 0x7fb7f81084a0 2024-04-24T09:19:06.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:19:06 smithi138 ceph-mon[31441]: pgmap v869: 129 pgs: 129 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 46 KiB/s wr, 28 op/s 2024-04-24T09:19:06.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:19:06 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/1579313991' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:19:06.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:19:06 smithi189 ceph-mon[31791]: pgmap v869: 129 pgs: 129 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 46 KiB/s wr, 28 op/s 2024-04-24T09:19:06.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:19:06 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/1579313991' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:19:06.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:06.807+0000 7fb7ff000640 1 -- 172.21.15.83:0/2744839667 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb7f8110370 con 0x7fb7f81084a0 2024-04-24T09:19:06.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:06.810+0000 7fb7f6000640 1 -- 172.21.15.83:0/2744839667 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v136) ==== 74+0+25453 (secure 0 0 0) 0x7fb7ec09cbd0 con 0x7fb7f81084a0 2024-04-24T09:19:06.811 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-24T09:19:06.811 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":136,"fsid":"bd3e039e-0215-11ef-bc93-c7b262605968","created":"2024-04-24T08:37:32.209944+0000","modified":"2024-04-24T09:18:14.841781+0000","last_up_change":"2024-04-24T08:47:17.966041+0000","last_in_change":"2024-04-24T08:46:53.835936+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T08:41:56.834372+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T08:48:26.978311+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T08:48:28.352731+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"135","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":48,"snap_epoch":135,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T08:48:32.275167+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"135","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":48,"snap_epoch":135,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2cd77f3a-19dd-4c6e-a199-1dcbd5faa679","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2132675555},{"type":"v1","addr":"172.21.15.83:6803","nonce":2132675555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2132675555},{"type":"v1","addr":"172.21.15.83:6805","nonce":2132675555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2132675555},{"type":"v1","addr":"172.21.15.83:6809","nonce":2132675555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2132675555},{"type":"v1","addr":"172.21.15.83:6807","nonce":2132675555}]},"public_addr":"172.21.15.83:6803/2132675555","cluster_addr":"172.21.15.83:6805/2132675555","heartbeat_back_addr":"172.21.15.83:6809/2132675555","heartbeat_front_addr":"172.21.15.83:6807/2132675555","state":["exists","up"]},{"osd":1,"uuid":"aba7f5df-2a4c-4c23-b502-d193605a5b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":3589158880},{"type":"v1","addr":"172.21.15.138:6801","nonce":3589158880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":3589158880},{"type":"v1","addr":"172.21.15.138:6803","nonce":3589158880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":3589158880},{"type":"v1","addr":"172.21.15.138:6807","nonce":3589158880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":3589158880},{"type":"v1","addr":"172.21.15.138:6805","nonce":3589158880}]},"public_addr":"172.21.15.138:6801/3589158880","cluster_addr":"172.21.15.138:6803/3589158880","heartbeat_back_addr":"172.21.15.138:6807/3589158880","heartbeat_front_addr":"172.21.15.138:6805/3589158880","state":["exists","up"]},{"osd":2,"uuid":"071bc167-2333-4420-bb58-bffbc10d7641","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":3169072688},{"type":"v1","addr":"172.21.15.189:6801","nonce":3169072688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":3169072688},{"type":"v1","addr":"172.21.15.189:6803","nonce":3169072688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":3169072688},{"type":"v1","addr":"172.21.15.189:6807","nonce":3169072688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":3169072688},{"type":"v1","addr":"172.21.15.189:6805","nonce":3169072688}]},"public_addr":"172.21.15.189:6801/3169072688","cluster_addr":"172.21.15.189:6803/3169072688","heartbeat_back_addr":"172.21.15.189:6807/3169072688","heartbeat_front_addr":"172.21.15.189:6805/3169072688","state":["exists","up"]},{"osd":3,"uuid":"f5624d78-4455-4e07-8c1a-f47c94cddb23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3131092450},{"type":"v1","addr":"172.21.15.83:6811","nonce":3131092450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3131092450},{"type":"v1","addr":"172.21.15.83:6813","nonce":3131092450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3131092450},{"type":"v1","addr":"172.21.15.83:6817","nonce":3131092450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3131092450},{"type":"v1","addr":"172.21.15.83:6815","nonce":3131092450}]},"public_addr":"172.21.15.83:6811/3131092450","cluster_addr":"172.21.15.83:6813/3131092450","heartbeat_back_addr":"172.21.15.83:6817/3131092450","heartbeat_front_addr":"172.21.15.83:6815/3131092450","state":["exists","up"]},{"osd":4,"uuid":"7b50cd14-9385-4535-a7b0-1c1da93c8548","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":823775360},{"type":"v1","addr":"172.21.15.138:6809","nonce":823775360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":823775360},{"type":"v1","addr":"172.21.15.138:6811","nonce":823775360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":823775360},{"type":"v1","addr":"172.21.15.138:6815","nonce":823775360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":823775360},{"type":"v1","addr":"172.21.15.138:6813","nonce":823775360}]},"public_addr":"172.21.15.138:6809/823775360","cluster_addr":"172.21.15.138:6811/823775360","heartbeat_back_addr":"172.21.15.138:6815/823775360","heartbeat_front_addr":"172.21.15.138:6813/823775360","state":["exists","up"]},{"osd":5,"uuid":"9c647797-35bc-449f-be84-af97d2c69c2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":1399817431},{"type":"v1","addr":"172.21.15.189:6809","nonce":1399817431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":1399817431},{"type":"v1","addr":"172.21.15.189:6811","nonce":1399817431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":1399817431},{"type":"v1","addr":"172.21.15.189:6815","nonce":1399817431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":1399817431},{"type":"v1","addr":"172.21.15.189:6813","nonce":1399817431}]},"public_addr":"172.21.15.189:6809/1399817431","cluster_addr":"172.21.15.189:6811/1399817431","heartbeat_back_addr":"172.21.15.189:6815/1399817431","heartbeat_front_addr":"172.21.15.189:6813/1399817431","state":["exists","up"]},{"osd":6,"uuid":"8f056584-16c8-4454-b445-3f780f26c196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":1611358483},{"type":"v1","addr":"172.21.15.83:6819","nonce":1611358483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1611358483},{"type":"v1","addr":"172.21.15.83:6821","nonce":1611358483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":1611358483},{"type":"v1","addr":"172.21.15.83:6825","nonce":1611358483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":1611358483},{"type":"v1","addr":"172.21.15.83:6823","nonce":1611358483}]},"public_addr":"172.21.15.83:6819/1611358483","cluster_addr":"172.21.15.83:6821/1611358483","heartbeat_back_addr":"172.21.15.83:6825/1611358483","heartbeat_front_addr":"172.21.15.83:6823/1611358483","state":["exists","up"]},{"osd":7,"uuid":"7603efe3-da77-4b7f-9d03-29c4f293b6d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":3557856706},{"type":"v1","addr":"172.21.15.138:6817","nonce":3557856706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":3557856706},{"type":"v1","addr":"172.21.15.138:6819","nonce":3557856706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":3557856706},{"type":"v1","addr":"172.21.15.138:6823","nonce":3557856706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":3557856706},{"type":"v1","addr":"172.21.15.138:6821","nonce":3557856706}]},"public_addr":"172.21.15.138:6817/3557856706","cluster_addr":"172.21.15.138:6819/3557856706","heartbeat_back_addr":"172.21.15.138:6823/3557856706","heartbeat_front_addr":"172.21.15.138:6821/3557856706","state":["exists","up"]},{"osd":8,"uuid":"27d7d1b8-d6a0-417e-b8aa-596325aa1a77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":207018554},{"type":"v1","addr":"172.21.15.189:6817","nonce":207018554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":207018554},{"type":"v1","addr":"172.21.15.189:6819","nonce":207018554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":207018554},{"type":"v1","addr":"172.21.15.189:6823","nonce":207018554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":207018554},{"type":"v1","addr":"172.21.15.189:6821","nonce":207018554}]},"public_addr":"172.21.15.189:6817/207018554","cluster_addr":"172.21.15.189:6819/207018554","heartbeat_back_addr":"172.21.15.189:6823/207018554","heartbeat_front_addr":"172.21.15.189:6821/207018554","state":["exists","up"]},{"osd":9,"uuid":"baf19b98-881b-4d3a-9f72-8aa2dd73cad6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":519636751},{"type":"v1","addr":"172.21.15.83:6827","nonce":519636751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":519636751},{"type":"v1","addr":"172.21.15.83:6829","nonce":519636751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":519636751},{"type":"v1","addr":"172.21.15.83:6833","nonce":519636751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":519636751},{"type":"v1","addr":"172.21.15.83:6831","nonce":519636751}]},"public_addr":"172.21.15.83:6827/519636751","cluster_addr":"172.21.15.83:6829/519636751","heartbeat_back_addr":"172.21.15.83:6833/519636751","heartbeat_front_addr":"172.21.15.83:6831/519636751","state":["exists","up"]},{"osd":10,"uuid":"eb346717-5c07-47c8-95be-0e68fe449fdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1130872098},{"type":"v1","addr":"172.21.15.138:6825","nonce":1130872098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1130872098},{"type":"v1","addr":"172.21.15.138:6827","nonce":1130872098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1130872098},{"type":"v1","addr":"172.21.15.138:6831","nonce":1130872098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1130872098},{"type":"v1","addr":"172.21.15.138:6829","nonce":1130872098}]},"public_addr":"172.21.15.138:6825/1130872098","cluster_addr":"172.21.15.138:6827/1130872098","heartbeat_back_addr":"172.21.15.138:6831/1130872098","heartbeat_front_addr":"172.21.15.138:6829/1130872098","state":["exists","up"]},{"osd":11,"uuid":"8e37568e-981c-45b8-b2af-2f1b73898542","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":2998782690},{"type":"v1","addr":"172.21.15.189:6825","nonce":2998782690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":2998782690},{"type":"v1","addr":"172.21.15.189:6827","nonce":2998782690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":2998782690},{"type":"v1","addr":"172.21.15.189:6831","nonce":2998782690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":2998782690},{"type":"v1","addr":"172.21.15.189:6829","nonce":2998782690}]},"public_addr":"172.21.15.189:6825/2998782690","cluster_addr":"172.21.15.189:6827/2998782690","heartbeat_back_addr":"172.21.15.189:6831/2998782690","heartbeat_front_addr":"172.21.15.189:6829/2998782690","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:40:47.980103+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:41:21.970795+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:41:55.160426+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:42:30.400475+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:43:05.705663+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:43:40.112312+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:44:16.781289+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:44:52.151187+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:45:28.055944+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:46:04.969301+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:46:40.918306+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T08:47:16.886874+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"2.1d","mappings":[{"from":2,"to":1}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.21","mappings":[{"from":4,"to":8}]},{"pgid":"3.2a","mappings":[{"from":3,"to":8}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.a","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:0/1096290901":"2024-04-25T08:51:36.347763+0000","172.21.15.83:6800/2550826560":"2024-04-25T08:51:36.347763+0000","172.21.15.83:0/1743153945":"2024-04-25T08:51:36.347763+0000","172.21.15.83:0/3306681366":"2024-04-25T08:51:36.347763+0000","172.21.15.83:6801/820049883":"2024-04-25T08:38:40.661315+0000","172.21.15.83:6801/2550826560":"2024-04-25T08:51:36.347763+0000","172.21.15.83:0/4272874138":"2024-04-25T08:38:40.661315+0000","172.21.15.83:0/4212994335":"2024-04-25T08:38:40.661315+0000","172.21.15.83:6801/1996371091":"2024-04-25T08:38:05.800856+0000","172.21.15.83:0/1820663683":"2024-04-25T08:38:05.800856+0000","172.21.15.83:0/3802897262":"2024-04-25T08:38:05.800856+0000","172.21.15.83:6800/1996371091":"2024-04-25T08:38:05.800856+0000","172.21.15.83:6800/820049883":"2024-04-25T08:38:40.661315+0000","172.21.15.83:0/1366125049":"2024-04-25T08:38:05.800856+0000","172.21.15.83:0/4159820288":"2024-04-25T08:51:36.347763+0000","172.21.15.83:0/340452868":"2024-04-25T08:38:40.661315+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":35,"length":1},{"begin":48,"length":1}]},{"pool":4,"snaps":[{"begin":35,"length":1},{"begin":48,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T09:19:06.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:06.814+0000 7fb7ff000640 1 -- 172.21.15.83:0/2744839667 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb7e00a3da0 msgr2=0x7fb7e00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:06.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:06.814+0000 7fb7ff000640 1 --2- 172.21.15.83:0/2744839667 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb7e00a3da0 0x7fb7e00a6260 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fb7f807a2e0 tx=0x7fb7e4005d90 comp rx=0 tx=0).stop 2024-04-24T09:19:06.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:06.814+0000 7fb7ff000640 1 -- 172.21.15.83:0/2744839667 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7f81084a0 msgr2=0x7fb7f81a47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:06.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:06.814+0000 7fb7ff000640 1 --2- 172.21.15.83:0/2744839667 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7f81084a0 0x7fb7f81a47b0 secure :-1 s=READY pgs=2936 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec00c970 tx=0x7fb7ec00ce30 comp rx=0 tx=0).stop 2024-04-24T09:19:06.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:06.814+0000 7fb7ff000640 1 -- 172.21.15.83:0/2744839667 shutdown_connections 2024-04-24T09:19:06.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:06.814+0000 7fb7ff000640 1 --2- 172.21.15.83:0/2744839667 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fb7e00a3da0 0x7fb7e00a6260 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:06.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:06.814+0000 7fb7ff000640 1 --2- 172.21.15.83:0/2744839667 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb7f810d3c0 0x7fb7f819e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:06.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:06.814+0000 7fb7ff000640 1 --2- 172.21.15.83:0/2744839667 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb7f8108e40 0x7fb7f81a4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:06.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:06.814+0000 7fb7ff000640 1 --2- 172.21.15.83:0/2744839667 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7f81084a0 0x7fb7f81a47b0 unknown :-1 s=CLOSED pgs=2936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:06.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:06.814+0000 7fb7ff000640 1 -- 172.21.15.83:0/2744839667 >> 172.21.15.83:0/2744839667 conn(0x7fb7f8077b80 msgr2=0x7fb7f8078320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:19:06.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:06.814+0000 7fb7ff000640 1 -- 172.21.15.83:0/2744839667 shutdown_connections 2024-04-24T09:19:06.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:06.814+0000 7fb7ff000640 1 -- 172.21.15.83:0/2744839667 wait complete. 2024-04-24T09:19:06.835 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-24T09:19:07.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:07.034+0000 7f3a76200640 1 -- 172.21.15.83:0/734263865 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a701093f0 msgr2=0x7f3a70110fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:07.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:07.034+0000 7f3a76200640 1 --2- 172.21.15.83:0/734263865 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a701093f0 0x7f3a70110fc0 secure :-1 s=READY pgs=2937 cs=0 l=1 rev1=1 crypto rx=0x7f3a68009ca0 tx=0x7f3a6802a8c0 comp rx=0 tx=0).stop 2024-04-24T09:19:07.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:07.034+0000 7f3a76200640 1 -- 172.21.15.83:0/734263865 shutdown_connections 2024-04-24T09:19:07.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:07.034+0000 7f3a76200640 1 --2- 172.21.15.83:0/734263865 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a701093f0 0x7f3a70110fc0 unknown :-1 s=CLOSED pgs=2937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:07.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:07.034+0000 7f3a76200640 1 --2- 172.21.15.83:0/734263865 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3a70104cd0 0x7f3a70108d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:07.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:07.035+0000 7f3a76200640 1 --2- 172.21.15.83:0/734263865 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a70104320 0x7f3a70104700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:07.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:07.035+0000 7f3a76200640 1 -- 172.21.15.83:0/734263865 >> 172.21.15.83:0/734263865 conn(0x7f3a700ffb40 msgr2=0x7f3a70101f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:19:07.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:07.035+0000 7f3a76200640 1 -- 172.21.15.83:0/734263865 shutdown_connections 2024-04-24T09:19:07.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:07.035+0000 7f3a76200640 1 -- 172.21.15.83:0/734263865 wait complete. 2024-04-24T09:19:07.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:07.036+0000 7f3a76200640 1 Processor -- start 2024-04-24T09:19:07.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:07.036+0000 7f3a76200640 1 -- start start 2024-04-24T09:19:07.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:07.036+0000 7f3a76200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a70104320 0x7f3a701a0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:07.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:07.036+0000 7f3a76200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3a701093f0 0x7f3a701a0970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:07.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:07.036+0000 7f3a76200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a7019a630 0x7f3a7019aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:07.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:07.036+0000 7f3a76200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3a70114390 con 0x7f3a7019a630 2024-04-24T09:19:07.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:07.036+0000 7f3a76200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3a70114210 con 0x7f3a70104320 2024-04-24T09:19:07.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:07.036+0000 7f3a76200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3a70114510 con 0x7f3a701093f0 2024-04-24T09:19:07.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:07.036+0000 7f3a75800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a7019a630 0x7f3a7019aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:07.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:07.036+0000 7f3a75800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a7019a630 0x7f3a7019aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47588/0 (socket says 172.21.15.83:47588) 2024-04-24T09:19:07.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:07.036+0000 7f3a75800640 1 -- 172.21.15.83:0/1623204405 learned_addr learned my addr 172.21.15.83:0/1623204405 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:19:07.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:07.037+0000 7f3a67e00640 1 --2- 172.21.15.83:0/1623204405 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3a701093f0 0x7f3a701a0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:07.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:07.037+0000 7f3a74e00640 1 --2- 172.21.15.83:0/1623204405 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a70104320 0x7f3a701a0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:07.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:07.037+0000 7f3a75800640 1 -- 172.21.15.83:0/1623204405 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3a701093f0 msgr2=0x7f3a701a0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:07.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:07.037+0000 7f3a75800640 1 --2- 172.21.15.83:0/1623204405 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3a701093f0 0x7f3a701a0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:07.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:07.037+0000 7f3a75800640 1 -- 172.21.15.83:0/1623204405 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a70104320 msgr2=0x7f3a701a0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:07.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:07.037+0000 7f3a75800640 1 --2- 172.21.15.83:0/1623204405 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a70104320 0x7f3a701a0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:07.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:07.037+0000 7f3a75800640 1 -- 172.21.15.83:0/1623204405 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a7019b150 con 0x7f3a7019a630 2024-04-24T09:19:07.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:07.037+0000 7f3a74e00640 1 --2- 172.21.15.83:0/1623204405 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a70104320 0x7f3a701a0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T09:19:07.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:07.037+0000 7f3a75800640 1 --2- 172.21.15.83:0/1623204405 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a7019a630 0x7f3a7019aae0 secure :-1 s=READY pgs=2938 cs=0 l=1 rev1=1 crypto rx=0x7f3a68031040 tx=0x7f3a68003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:19:07.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:07.037+0000 7f3a6e000640 1 -- 172.21.15.83:0/1623204405 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a68003ce0 con 0x7f3a7019a630 2024-04-24T09:19:07.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:07.037+0000 7f3a6e000640 1 -- 172.21.15.83:0/1623204405 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3a680044f0 con 0x7f3a7019a630 2024-04-24T09:19:07.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:07.038+0000 7f3a76200640 1 -- 172.21.15.83:0/1623204405 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a7019b3e0 con 0x7f3a7019a630 2024-04-24T09:19:07.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:07.037+0000 7f3a6e000640 1 -- 172.21.15.83:0/1623204405 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a68030c90 con 0x7f3a7019a630 2024-04-24T09:19:07.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:07.038+0000 7f3a76200640 1 -- 172.21.15.83:0/1623204405 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a701a75e0 con 0x7f3a7019a630 2024-04-24T09:19:07.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:07.039+0000 7f3a76200640 1 -- 172.21.15.83:0/1623204405 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a70069190 con 0x7f3a7019a630 2024-04-24T09:19:07.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:07.041+0000 7f3a6e000640 1 -- 172.21.15.83:0/1623204405 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3a680046a0 con 0x7f3a7019a630 2024-04-24T09:19:07.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:07.042+0000 7f3a6e000640 1 --2- 172.21.15.83:0/1623204405 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f3a540a3d20 0x7f3a540a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:07.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:07.042+0000 7f3a6e000640 1 -- 172.21.15.83:0/1623204405 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 9583+0+0 (secure 0 0 0) 0x7f3a680f2250 con 0x7f3a7019a630 2024-04-24T09:19:07.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:07.042+0000 7f3a74e00640 1 --2- 172.21.15.83:0/1623204405 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f3a540a3d20 0x7f3a540a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:07.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:07.043+0000 7f3a74e00640 1 --2- 172.21.15.83:0/1623204405 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f3a540a3d20 0x7f3a540a61e0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f3a6000baa0 tx=0x7f3a60005d90 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:19:07.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:07.046+0000 7f3a6e000640 1 -- 172.21.15.83:0/1623204405 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3a680be6a0 con 0x7f3a7019a630 2024-04-24T09:19:07.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:07.289+0000 7f3a76200640 1 -- 172.21.15.83:0/1623204405 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f3a7010c620 con 0x7f3a7019a630 2024-04-24T09:19:07.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:07.325+0000 7f3a6e000640 1 -- 172.21.15.83:0/1623204405 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v80) ==== 161+0+0 (secure 0 0 0) 0x7f3a680098b0 con 0x7f3a7019a630 2024-04-24T09:19:07.326 INFO:teuthology.orchestra.run.smithi083.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-24T09:19:07.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:07.330+0000 7f3a76200640 1 -- 172.21.15.83:0/1623204405 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f3a540a3d20 msgr2=0x7f3a540a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:07.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:07.330+0000 7f3a76200640 1 --2- 172.21.15.83:0/1623204405 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f3a540a3d20 0x7f3a540a61e0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f3a6000baa0 tx=0x7f3a60005d90 comp rx=0 tx=0).stop 2024-04-24T09:19:07.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:07.330+0000 7f3a76200640 1 -- 172.21.15.83:0/1623204405 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a7019a630 msgr2=0x7f3a7019aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:07.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:07.330+0000 7f3a76200640 1 --2- 172.21.15.83:0/1623204405 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a7019a630 0x7f3a7019aae0 secure :-1 s=READY pgs=2938 cs=0 l=1 rev1=1 crypto rx=0x7f3a68031040 tx=0x7f3a68003970 comp rx=0 tx=0).stop 2024-04-24T09:19:07.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:07.331+0000 7f3a76200640 1 -- 172.21.15.83:0/1623204405 shutdown_connections 2024-04-24T09:19:07.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:07.331+0000 7f3a76200640 1 --2- 172.21.15.83:0/1623204405 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f3a540a3d20 0x7f3a540a61e0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:07.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:07.331+0000 7f3a76200640 1 --2- 172.21.15.83:0/1623204405 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a7019a630 0x7f3a7019aae0 unknown :-1 s=CLOSED pgs=2938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:07.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:07.331+0000 7f3a76200640 1 --2- 172.21.15.83:0/1623204405 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3a701093f0 0x7f3a701a0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:07.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:07.331+0000 7f3a76200640 1 --2- 172.21.15.83:0/1623204405 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a70104320 0x7f3a701a0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:07.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:07.331+0000 7f3a76200640 1 -- 172.21.15.83:0/1623204405 >> 172.21.15.83:0/1623204405 conn(0x7f3a700ffb40 msgr2=0x7f3a70100690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:19:07.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:07.331+0000 7f3a76200640 1 -- 172.21.15.83:0/1623204405 shutdown_connections 2024-04-24T09:19:07.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:07.331+0000 7f3a76200640 1 -- 172.21.15.83:0/1623204405 wait complete. 2024-04-24T09:19:07.367 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-24T09:19:07.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:19:07 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/2999168277' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:19:07.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:19:07 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/2744839667' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:19:07.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:19:07 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mon-a[27903]: 2024-04-24T09:19:07.314+0000 7fbf7b200700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-24T09:19:07.554 INFO:journalctl@ceph.mds.a.smithi083.stdout:Apr 24 09:19:07 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-a[69821]: ignoring --setuser ceph since I am not root 2024-04-24T09:19:07.554 INFO:journalctl@ceph.mds.a.smithi083.stdout:Apr 24 09:19:07 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-a[69821]: ignoring --setgroup ceph since I am not root 2024-04-24T09:19:07.554 INFO:journalctl@ceph.mds.a.smithi083.stdout:Apr 24 09:19:07 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-a[69821]: starting mds.a at 2024-04-24T09:19:07.554 INFO:journalctl@ceph.mds.g.smithi083.stdout:Apr 24 09:19:07 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-g[69436]: ignoring --setuser ceph since I am not root 2024-04-24T09:19:07.554 INFO:journalctl@ceph.mds.g.smithi083.stdout:Apr 24 09:19:07 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-g[69436]: ignoring --setgroup ceph since I am not root 2024-04-24T09:19:07.555 INFO:journalctl@ceph.mds.g.smithi083.stdout:Apr 24 09:19:07 smithi083 ceph-mds[69440]: -- 172.21.15.83:0/2481288836 <== mon.1 v2:172.21.15.189:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558db6779a40 con 0x558db674d000 2024-04-24T09:19:07.555 INFO:journalctl@ceph.mds.g.smithi083.stdout:Apr 24 09:19:07 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-g[69436]: starting mds.g at 2024-04-24T09:19:07.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:07.682+0000 7ff40aa00640 1 -- 172.21.15.83:0/3612504927 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3f8035020 con 0x7ff4040a1620 2024-04-24T09:19:07.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:07.683+0000 7ff413000640 1 -- 172.21.15.83:0/3612504927 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4040a1620 msgr2=0x7ff4040a91f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:07.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:07.683+0000 7ff413000640 1 --2- 172.21.15.83:0/3612504927 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4040a1620 0x7ff4040a91f0 secure :-1 s=READY pgs=2944 cs=0 l=1 rev1=1 crypto rx=0x7ff3f8009900 tx=0x7ff3f802a440 comp rx=0 tx=0).stop 2024-04-24T09:19:07.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:07.684+0000 7ff413000640 1 -- 172.21.15.83:0/3612504927 shutdown_connections 2024-04-24T09:19:07.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:07.684+0000 7ff413000640 1 --2- 172.21.15.83:0/3612504927 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4040a1620 0x7ff4040a91f0 unknown :-1 s=CLOSED pgs=2944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:07.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:07.684+0000 7ff413000640 1 --2- 172.21.15.83:0/3612504927 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff40409cf00 0x7ff4040a0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:07.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:07.684+0000 7ff413000640 1 --2- 172.21.15.83:0/3612504927 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff40409c550 0x7ff40409c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:07.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:07.684+0000 7ff413000640 1 -- 172.21.15.83:0/3612504927 >> 172.21.15.83:0/3612504927 conn(0x7ff40400c810 msgr2=0x7ff40409a590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:19:07.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:07.684+0000 7ff413000640 1 -- 172.21.15.83:0/3612504927 shutdown_connections 2024-04-24T09:19:07.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:07.684+0000 7ff413000640 1 -- 172.21.15.83:0/3612504927 wait complete. 2024-04-24T09:19:07.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:07.689+0000 7ff413000640 1 Processor -- start 2024-04-24T09:19:07.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:07.689+0000 7ff413000640 1 -- start start 2024-04-24T09:19:07.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:07.690+0000 7ff413000640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff40409c550 0x7ff4041388a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:07.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:07.690+0000 7ff413000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff4040a1620 0x7ff404138de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:07.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:07.690+0000 7ff413000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff404132aa0 0x7ff404132f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:07.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:07.691+0000 7ff413000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff4040ac680 con 0x7ff404132aa0 2024-04-24T09:19:07.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:07.691+0000 7ff413000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff4040ac500 con 0x7ff4040a1620 2024-04-24T09:19:07.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:07.691+0000 7ff413000640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff4040ac800 con 0x7ff40409c550 2024-04-24T09:19:07.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:07.691+0000 7ff40b400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff4040a1620 0x7ff404138de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:07.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:07.691+0000 7ff40b400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff4040a1620 0x7ff404138de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.83:54454/0 (socket says 172.21.15.83:54454) 2024-04-24T09:19:07.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:07.691+0000 7ff40b400640 1 -- 172.21.15.83:0/2164111053 learned_addr learned my addr 172.21.15.83:0/2164111053 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:19:07.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:07.691+0000 7ff411000640 1 --2- 172.21.15.83:0/2164111053 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff404132aa0 0x7ff404132f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:07.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:07.691+0000 7ff40be00640 1 --2- 172.21.15.83:0/2164111053 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff40409c550 0x7ff4041388a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:07.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:07.692+0000 7ff40be00640 1 -- 172.21.15.83:0/2164111053 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff4040a1620 msgr2=0x7ff404138de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:07.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:07.692+0000 7ff40be00640 1 --2- 172.21.15.83:0/2164111053 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff4040a1620 0x7ff404138de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:07.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:07.692+0000 7ff40be00640 1 -- 172.21.15.83:0/2164111053 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff404132aa0 msgr2=0x7ff404132f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:07.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:07.692+0000 7ff40be00640 1 --2- 172.21.15.83:0/2164111053 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff404132aa0 0x7ff404132f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:07.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:07.692+0000 7ff40be00640 1 -- 172.21.15.83:0/2164111053 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4041335c0 con 0x7ff40409c550 2024-04-24T09:19:07.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:07.692+0000 7ff411000640 1 --2- 172.21.15.83:0/2164111053 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff404132aa0 0x7ff404132f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:19:07.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:07.694+0000 7ff40be00640 1 --2- 172.21.15.83:0/2164111053 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff40409c550 0x7ff4041388a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff40c050b40 tx=0x7ff40c077690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:19:07.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:07.694+0000 7ff408c00640 1 -- 172.21.15.83:0/2164111053 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff40c083620 con 0x7ff40409c550 2024-04-24T09:19:07.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:07.695+0000 7ff413000640 1 -- 172.21.15.83:0/2164111053 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4041338b0 con 0x7ff40409c550 2024-04-24T09:19:07.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:07.695+0000 7ff408c00640 1 -- 172.21.15.83:0/2164111053 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff40c079e50 con 0x7ff40409c550 2024-04-24T09:19:07.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:07.695+0000 7ff408c00640 1 -- 172.21.15.83:0/2164111053 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff40c082740 con 0x7ff40409c550 2024-04-24T09:19:07.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:07.695+0000 7ff413000640 1 -- 172.21.15.83:0/2164111053 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff40413fa60 con 0x7ff40409c550 2024-04-24T09:19:07.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:07.697+0000 7ff413000640 1 -- 172.21.15.83:0/2164111053 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4040054c0 con 0x7ff40409c550 2024-04-24T09:19:07.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:07.703+0000 7ff408c00640 1 -- 172.21.15.83:0/2164111053 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff40c08a060 con 0x7ff40409c550 2024-04-24T09:19:07.704 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:19:07 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/2999168277' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:19:07.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:19:07 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/2744839667' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:19:07.705 INFO:journalctl@ceph.mds.h.smithi138.stdout:Apr 24 09:19:07 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-h[61199]: ignoring --setuser ceph since I am not root 2024-04-24T09:19:07.705 INFO:journalctl@ceph.mds.h.smithi138.stdout:Apr 24 09:19:07 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-h[61199]: ignoring --setgroup ceph since I am not root 2024-04-24T09:19:07.705 INFO:journalctl@ceph.mds.h.smithi138.stdout:Apr 24 09:19:07 smithi138 ceph-mds[61203]: -- 172.21.15.138:0/1290738210 <== mon.1 v2:172.21.15.189:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557a9f94da40 con 0x557a9f921000 2024-04-24T09:19:07.705 INFO:journalctl@ceph.mds.h.smithi138.stdout:Apr 24 09:19:07 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-h[61199]: starting mds.h at 2024-04-24T09:19:07.705 INFO:journalctl@ceph.mds.k.smithi138.stdout:Apr 24 09:19:07 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-k[60581]: ignoring --setuser ceph since I am not root 2024-04-24T09:19:07.706 INFO:journalctl@ceph.mds.k.smithi138.stdout:Apr 24 09:19:07 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-k[60581]: ignoring --setgroup ceph since I am not root 2024-04-24T09:19:07.706 INFO:journalctl@ceph.mds.k.smithi138.stdout:Apr 24 09:19:07 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-k[60581]: starting mds.k at 2024-04-24T09:19:07.706 INFO:journalctl@ceph.mds.b.smithi138.stdout:Apr 24 09:19:07 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-b[61515]: ignoring --setuser ceph since I am not root 2024-04-24T09:19:07.706 INFO:journalctl@ceph.mds.b.smithi138.stdout:Apr 24 09:19:07 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-b[61515]: ignoring --setgroup ceph since I am not root 2024-04-24T09:19:07.706 INFO:journalctl@ceph.mds.b.smithi138.stdout:Apr 24 09:19:07 smithi138 ceph-mds[61519]: -- 172.21.15.138:0/1088312371 <== mon.1 v2:172.21.15.189:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a3b0501a40 con 0x55a3b04d4800 2024-04-24T09:19:07.706 INFO:journalctl@ceph.mds.b.smithi138.stdout:Apr 24 09:19:07 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-b[61515]: starting mds.b at 2024-04-24T09:19:07.706 INFO:journalctl@ceph.mds.e.smithi138.stdout:Apr 24 09:19:07 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-e[60896]: ignoring --setuser ceph since I am not root 2024-04-24T09:19:07.706 INFO:journalctl@ceph.mds.e.smithi138.stdout:Apr 24 09:19:07 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-e[60896]: ignoring --setgroup ceph since I am not root 2024-04-24T09:19:07.707 INFO:journalctl@ceph.mds.e.smithi138.stdout:Apr 24 09:19:07 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-e[60896]: starting mds.e at 2024-04-24T09:19:07.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:07.705+0000 7ff408c00640 1 --2- 172.21.15.83:0/2164111053 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff3f40a3e20 0x7ff3f40a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:07.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:07.705+0000 7ff408c00640 1 -- 172.21.15.83:0/2164111053 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 10013+0+0 (secure 0 0 0) 0x7ff40c13cf90 con 0x7ff40409c550 2024-04-24T09:19:07.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:07.706+0000 7ff40b400640 1 --2- 172.21.15.83:0/2164111053 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff3f40a3e20 0x7ff3f40a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:07.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:07.707+0000 7ff40b400640 1 --2- 172.21.15.83:0/2164111053 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff3f40a3e20 0x7ff3f40a62e0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7ff4040a28e0 tx=0x7ff400009190 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:19:07.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:19:07 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/2999168277' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T09:19:07.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:19:07 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/2744839667' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T09:19:07.708 INFO:journalctl@ceph.mds.c.smithi189.stdout:Apr 24 09:19:07 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-c[61724]: ignoring --setuser ceph since I am not root 2024-04-24T09:19:07.709 INFO:journalctl@ceph.mds.c.smithi189.stdout:Apr 24 09:19:07 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-c[61724]: ignoring --setgroup ceph since I am not root 2024-04-24T09:19:07.709 INFO:journalctl@ceph.mds.c.smithi189.stdout:Apr 24 09:19:07 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-c[61724]: starting mds.c at 2024-04-24T09:19:07.709 INFO:journalctl@ceph.mds.f.smithi189.stdout:Apr 24 09:19:07 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-f[61409]: ignoring --setuser ceph since I am not root 2024-04-24T09:19:07.709 INFO:journalctl@ceph.mds.f.smithi189.stdout:Apr 24 09:19:07 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-f[61409]: ignoring --setgroup ceph since I am not root 2024-04-24T09:19:07.709 INFO:journalctl@ceph.mds.f.smithi189.stdout:Apr 24 09:19:07 smithi189 ceph-mds[61418]: -- 172.21.15.189:0/72581642 <== mon.1 v2:172.21.15.189:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556292dc3a40 con 0x556292d97000 2024-04-24T09:19:07.709 INFO:journalctl@ceph.mds.f.smithi189.stdout:Apr 24 09:19:07 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-f[61409]: starting mds.f at 2024-04-24T09:19:07.709 INFO:journalctl@ceph.mds.i.smithi189.stdout:Apr 24 09:19:07 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-i[62031]: ignoring --setuser ceph since I am not root 2024-04-24T09:19:07.709 INFO:journalctl@ceph.mds.i.smithi189.stdout:Apr 24 09:19:07 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-i[62031]: ignoring --setgroup ceph since I am not root 2024-04-24T09:19:07.710 INFO:journalctl@ceph.mds.i.smithi189.stdout:Apr 24 09:19:07 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-i[62031]: starting mds.i at 2024-04-24T09:19:07.710 INFO:journalctl@ceph.mds.l.smithi189.stdout:Apr 24 09:19:07 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-l[61103]: ignoring --setuser ceph since I am not root 2024-04-24T09:19:07.710 INFO:journalctl@ceph.mds.l.smithi189.stdout:Apr 24 09:19:07 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-l[61103]: ignoring --setgroup ceph since I am not root 2024-04-24T09:19:07.710 INFO:journalctl@ceph.mds.l.smithi189.stdout:Apr 24 09:19:07 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-l[61103]: starting mds.l at 2024-04-24T09:19:07.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:07.711+0000 7ff408c00640 1 -- 172.21.15.83:0/2164111053 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff40c109000 con 0x7ff40409c550 2024-04-24T09:19:08.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:08.177+0000 7ff413000640 1 -- 172.21.15.83:0/2164111053 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7ff40409cf00 con 0x7ff40409c550 2024-04-24T09:19:08.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:08.333+0000 7ff408c00640 1 -- 172.21.15.83:0/2164111053 <== mon.1 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v81) ==== 104+0+0 (secure 0 0 0) 0x7ff40c108900 con 0x7ff40409c550 2024-04-24T09:19:08.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:08.337+0000 7ff413000640 1 -- 172.21.15.83:0/2164111053 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff3f40a3e20 msgr2=0x7ff3f40a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:08.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:08.337+0000 7ff413000640 1 --2- 172.21.15.83:0/2164111053 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff3f40a3e20 0x7ff3f40a62e0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7ff4040a28e0 tx=0x7ff400009190 comp rx=0 tx=0).stop 2024-04-24T09:19:08.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:08.337+0000 7ff413000640 1 -- 172.21.15.83:0/2164111053 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff40409c550 msgr2=0x7ff4041388a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:08.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:08.337+0000 7ff413000640 1 --2- 172.21.15.83:0/2164111053 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff40409c550 0x7ff4041388a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff40c050b40 tx=0x7ff40c077690 comp rx=0 tx=0).stop 2024-04-24T09:19:08.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:08.337+0000 7ff413000640 1 -- 172.21.15.83:0/2164111053 shutdown_connections 2024-04-24T09:19:08.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:08.337+0000 7ff413000640 1 --2- 172.21.15.83:0/2164111053 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7ff3f40a3e20 0x7ff3f40a62e0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:08.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:08.337+0000 7ff413000640 1 --2- 172.21.15.83:0/2164111053 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff404132aa0 0x7ff404132f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:08.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:08.337+0000 7ff413000640 1 --2- 172.21.15.83:0/2164111053 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff4040a1620 0x7ff404138de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:08.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:08.337+0000 7ff413000640 1 --2- 172.21.15.83:0/2164111053 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff40409c550 0x7ff4041388a0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:08.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:08.338+0000 7ff413000640 1 -- 172.21.15.83:0/2164111053 >> 172.21.15.83:0/2164111053 conn(0x7ff40400c810 msgr2=0x7ff40400a510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:19:08.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:08.338+0000 7ff413000640 1 -- 172.21.15.83:0/2164111053 shutdown_connections 2024-04-24T09:19:08.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:08.338+0000 7ff413000640 1 -- 172.21.15.83:0/2164111053 wait complete. 2024-04-24T09:19:08.369 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-24T09:19:08.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:08.548+0000 7fe506200640 1 -- 172.21.15.83:0/2103521264 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe500107320 msgr2=0x7fe50010eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:08.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:08.548+0000 7fe506200640 1 --2- 172.21.15.83:0/2103521264 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe500107320 0x7fe50010eef0 secure :-1 s=READY pgs=2945 cs=0 l=1 rev1=1 crypto rx=0x7fe4f8009ca0 tx=0x7fe4f802a8c0 comp rx=0 tx=0).stop 2024-04-24T09:19:08.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:08.548+0000 7fe506200640 1 -- 172.21.15.83:0/2103521264 shutdown_connections 2024-04-24T09:19:08.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:08.548+0000 7fe506200640 1 --2- 172.21.15.83:0/2103521264 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe500107320 0x7fe50010eef0 unknown :-1 s=CLOSED pgs=2945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:08.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:08.548+0000 7fe506200640 1 --2- 172.21.15.83:0/2103521264 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe500102c00 0x7fe500106c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:08.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:08.548+0000 7fe506200640 1 --2- 172.21.15.83:0/2103521264 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe500102250 0x7fe500102630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:08.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:08.548+0000 7fe506200640 1 -- 172.21.15.83:0/2103521264 >> 172.21.15.83:0/2103521264 conn(0x7fe500079490 msgr2=0x7fe5000798a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:19:08.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:08.548+0000 7fe506200640 1 -- 172.21.15.83:0/2103521264 shutdown_connections 2024-04-24T09:19:08.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:08.548+0000 7fe506200640 1 -- 172.21.15.83:0/2103521264 wait complete. 2024-04-24T09:19:08.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:08.549+0000 7fe506200640 1 Processor -- start 2024-04-24T09:19:08.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:08.549+0000 7fe506200640 1 -- start start 2024-04-24T09:19:08.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:08.550+0000 7fe506200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe500102250 0x7fe50019e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:08.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:08.550+0000 7fe506200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe500107320 0x7fe50019e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:08.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:08.550+0000 7fe506200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe500198600 0x7fe500198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:08.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:08.550+0000 7fe506200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe5001122e0 con 0x7fe500102250 2024-04-24T09:19:08.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:08.550+0000 7fe506200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe500112160 con 0x7fe500107320 2024-04-24T09:19:08.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:08.550+0000 7fe506200640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe500112460 con 0x7fe500198600 2024-04-24T09:19:08.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:08.550+0000 7fe4ff400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe500102250 0x7fe50019e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:08.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:08.550+0000 7fe4ff400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe500102250 0x7fe50019e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47636/0 (socket says 172.21.15.83:47636) 2024-04-24T09:19:08.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:08.550+0000 7fe4ff400640 1 -- 172.21.15.83:0/4021743667 learned_addr learned my addr 172.21.15.83:0/4021743667 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:19:08.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:08.551+0000 7fe4ffe00640 1 --2- 172.21.15.83:0/4021743667 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe500198600 0x7fe500198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:08.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:08.551+0000 7fe4ff400640 1 -- 172.21.15.83:0/4021743667 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe500198600 msgr2=0x7fe500198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:08.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:08.551+0000 7fe4ff400640 1 --2- 172.21.15.83:0/4021743667 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe500198600 0x7fe500198ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:08.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:08.551+0000 7fe4ff400640 1 -- 172.21.15.83:0/4021743667 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe500107320 msgr2=0x7fe50019e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:08.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:08.551+0000 7fe4f7e00640 1 --2- 172.21.15.83:0/4021743667 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe500107320 0x7fe50019e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:08.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:08.551+0000 7fe4ff400640 1 --2- 172.21.15.83:0/4021743667 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe500107320 0x7fe50019e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:08.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:08.551+0000 7fe4ff400640 1 -- 172.21.15.83:0/4021743667 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe500199120 con 0x7fe500102250 2024-04-24T09:19:08.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:08.551+0000 7fe4ff400640 1 --2- 172.21.15.83:0/4021743667 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe500102250 0x7fe50019e450 secure :-1 s=READY pgs=2946 cs=0 l=1 rev1=1 crypto rx=0x7fe4f000b4a0 tx=0x7fe4f000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:19:08.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:08.551+0000 7fe4fcc00640 1 -- 172.21.15.83:0/4021743667 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f0014070 con 0x7fe500102250 2024-04-24T09:19:08.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:08.551+0000 7fe4fcc00640 1 -- 172.21.15.83:0/4021743667 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe4f0010ac0 con 0x7fe500102250 2024-04-24T09:19:08.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:08.551+0000 7fe4fcc00640 1 -- 172.21.15.83:0/4021743667 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f0017c30 con 0x7fe500102250 2024-04-24T09:19:08.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:08.551+0000 7fe506200640 1 -- 172.21.15.83:0/4021743667 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe500199410 con 0x7fe500102250 2024-04-24T09:19:08.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:08.551+0000 7fe506200640 1 -- 172.21.15.83:0/4021743667 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5001a5660 con 0x7fe500102250 2024-04-24T09:19:08.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:08.552+0000 7fe506200640 1 -- 172.21.15.83:0/4021743667 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe500069270 con 0x7fe500102250 2024-04-24T09:19:08.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:08.554+0000 7fe4fcc00640 1 -- 172.21.15.83:0/4021743667 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe4f0017070 con 0x7fe500102250 2024-04-24T09:19:08.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:08.556+0000 7fe4fcc00640 1 --2- 172.21.15.83:0/4021743667 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe4e40a3d20 0x7fe4e40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:08.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:08.556+0000 7fe4fcc00640 1 -- 172.21.15.83:0/4021743667 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 10013+0+0 (secure 0 0 0) 0x7fe4f00d04b0 con 0x7fe500102250 2024-04-24T09:19:08.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:08.556+0000 7fe4f7e00640 1 --2- 172.21.15.83:0/4021743667 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe4e40a3d20 0x7fe4e40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:08.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:08.557+0000 7fe4f7e00640 1 --2- 172.21.15.83:0/4021743667 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe4e40a3d20 0x7fe4e40a61e0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fe500101900 tx=0x7fe4e8005c60 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:19:08.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:08.560+0000 7fe4fcc00640 1 -- 172.21.15.83:0/4021743667 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe4f009c630 con 0x7fe500102250 2024-04-24T09:19:08.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:19:08 smithi138 ceph-mon[31441]: pgmap v870: 129 pgs: 129 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 46 KiB/s wr, 28 op/s 2024-04-24T09:19:08.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:19:08 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/1623204405' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-24T09:19:08.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:19:08 smithi138 ceph-mon[31441]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-24T09:19:08.706 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:19:08 smithi138 ceph-mon[31441]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-24T09:19:08.706 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:19:08 smithi138 ceph-mon[31441]: osdmap e137: 12 total, 12 up, 12 in 2024-04-24T09:19:08.706 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:19:08 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/1623204405' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-24T09:19:08.706 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:19:08 smithi138 ceph-mon[31441]: osdmap e138: 12 total, 12 up, 12 in 2024-04-24T09:19:08.706 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:19:08 smithi138 ceph-mon[31441]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-04-24T09:19:08.706 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:19:08 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/2164111053' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-24T09:19:08.706 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:19:08 smithi138 ceph-mon[31441]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-24T09:19:08.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:19:08 smithi189 ceph-mon[31791]: pgmap v870: 129 pgs: 129 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 46 KiB/s wr, 28 op/s 2024-04-24T09:19:08.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:19:08 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/1623204405' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-24T09:19:08.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:19:08 smithi189 ceph-mon[31791]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-24T09:19:08.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:19:08 smithi189 ceph-mon[31791]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-24T09:19:08.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:19:08 smithi189 ceph-mon[31791]: osdmap e137: 12 total, 12 up, 12 in 2024-04-24T09:19:08.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:19:08 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/1623204405' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-24T09:19:08.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:19:08 smithi189 ceph-mon[31791]: osdmap e138: 12 total, 12 up, 12 in 2024-04-24T09:19:08.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:19:08 smithi189 ceph-mon[31791]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-04-24T09:19:08.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:19:08 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/2164111053' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-24T09:19:08.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:19:08 smithi189 ceph-mon[31791]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-24T09:19:08.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:08.752+0000 7fe506200640 1 -- 172.21.15.83:0/4021743667 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fe500199b30 con 0x7fe500102250 2024-04-24T09:19:08.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:19:08 smithi083 ceph-mon[27907]: pgmap v870: 129 pgs: 129 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 46 KiB/s wr, 28 op/s 2024-04-24T09:19:08.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:19:08 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/1623204405' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-24T09:19:08.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:19:08 smithi083 ceph-mon[27907]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-24T09:19:08.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:19:08 smithi083 ceph-mon[27907]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-24T09:19:08.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:19:08 smithi083 ceph-mon[27907]: osdmap e137: 12 total, 12 up, 12 in 2024-04-24T09:19:08.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:19:08 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/1623204405' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-24T09:19:08.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:19:08 smithi083 ceph-mon[27907]: osdmap e138: 12 total, 12 up, 12 in 2024-04-24T09:19:08.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:19:08 smithi083 ceph-mon[27907]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-04-24T09:19:08.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:19:08 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/2164111053' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-24T09:19:08.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:19:08 smithi083 ceph-mon[27907]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-24T09:19:09.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:09.332+0000 7fe4fcc00640 1 -- 172.21.15.83:0/4021743667 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v139) ==== 181+0+0 (secure 0 0 0) 0x7fe4f009bf30 con 0x7fe500102250 2024-04-24T09:19:09.333 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_metadata' removed 2024-04-24T09:19:09.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:09.336+0000 7fe506200640 1 -- 172.21.15.83:0/4021743667 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe4e40a3d20 msgr2=0x7fe4e40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:09.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:09.336+0000 7fe506200640 1 --2- 172.21.15.83:0/4021743667 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe4e40a3d20 0x7fe4e40a61e0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fe500101900 tx=0x7fe4e8005c60 comp rx=0 tx=0).stop 2024-04-24T09:19:09.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:09.336+0000 7fe506200640 1 -- 172.21.15.83:0/4021743667 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe500102250 msgr2=0x7fe50019e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:09.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:09.336+0000 7fe506200640 1 --2- 172.21.15.83:0/4021743667 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe500102250 0x7fe50019e450 secure :-1 s=READY pgs=2946 cs=0 l=1 rev1=1 crypto rx=0x7fe4f000b4a0 tx=0x7fe4f000b960 comp rx=0 tx=0).stop 2024-04-24T09:19:09.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:09.337+0000 7fe506200640 1 -- 172.21.15.83:0/4021743667 shutdown_connections 2024-04-24T09:19:09.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:09.337+0000 7fe506200640 1 --2- 172.21.15.83:0/4021743667 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7fe4e40a3d20 0x7fe4e40a61e0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:09.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:09.337+0000 7fe506200640 1 --2- 172.21.15.83:0/4021743667 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe500198600 0x7fe500198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:09.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:09.337+0000 7fe506200640 1 --2- 172.21.15.83:0/4021743667 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe500107320 0x7fe50019e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:09.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:09.337+0000 7fe506200640 1 --2- 172.21.15.83:0/4021743667 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe500102250 0x7fe50019e450 unknown :-1 s=CLOSED pgs=2946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:09.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:09.337+0000 7fe506200640 1 -- 172.21.15.83:0/4021743667 >> 172.21.15.83:0/4021743667 conn(0x7fe500079490 msgr2=0x7fe5000fffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:19:09.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:09.338+0000 7fe506200640 1 -- 172.21.15.83:0/4021743667 shutdown_connections 2024-04-24T09:19:09.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:09.338+0000 7fe506200640 1 -- 172.21.15.83:0/4021743667 wait complete. 2024-04-24T09:19:09.365 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-24T09:19:09.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:19:09 smithi083 ceph-mon[27907]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-24T09:19:09.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:19:09 smithi083 ceph-mon[27907]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-24T09:19:09.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:19:09 smithi083 ceph-mon[27907]: Cluster is now healthy 2024-04-24T09:19:09.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:19:09 smithi083 ceph-mon[27907]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-24T09:19:09.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:19:09 smithi083 ceph-mon[27907]: mds.? [v2:172.21.15.189:6832/3982017609,v1:172.21.15.189:6833/3982017609] up:boot 2024-04-24T09:19:09.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:19:09 smithi083 ceph-mon[27907]: mds.? [v2:172.21.15.189:6834/2508224750,v1:172.21.15.189:6836/2508224750] up:boot 2024-04-24T09:19:09.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:19:09 smithi083 ceph-mon[27907]: mds.? [v2:172.21.15.189:6835/2600233222,v1:172.21.15.189:6838/2600233222] up:boot 2024-04-24T09:19:09.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:19:09 smithi083 ceph-mon[27907]: mds.? [v2:172.21.15.189:6837/2204997973,v1:172.21.15.189:6839/2204997973] up:boot 2024-04-24T09:19:09.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:19:09 smithi083 ceph-mon[27907]: mds.? [v2:172.21.15.83:6838/2304379712,v1:172.21.15.83:6839/2304379712] up:boot 2024-04-24T09:19:09.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:19:09 smithi083 ceph-mon[27907]: mds.? [v2:172.21.15.138:6832/818800951,v1:172.21.15.138:6833/818800951] up:boot 2024-04-24T09:19:09.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:19:09 smithi083 ceph-mon[27907]: mds.? [v2:172.21.15.138:6836/3159978314,v1:172.21.15.138:6837/3159978314] up:boot 2024-04-24T09:19:09.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:19:09 smithi083 ceph-mon[27907]: mds.? [v2:172.21.15.138:6834/3963488485,v1:172.21.15.138:6835/3963488485] up:boot 2024-04-24T09:19:09.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:19:09 smithi083 ceph-mon[27907]: mds.? [v2:172.21.15.83:6840/3231655400,v1:172.21.15.83:6841/3231655400] up:boot 2024-04-24T09:19:09.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:19:09 smithi083 ceph-mon[27907]: mds.? [v2:172.21.15.138:6838/3752197684,v1:172.21.15.138:6839/3752197684] up:boot 2024-04-24T09:19:09.554 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:19:09 smithi083 ceph-mon[27907]: fsmap 12 up:standby 2024-04-24T09:19:09.555 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:19:09 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-24T09:19:09.555 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:19:09 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-24T09:19:09.555 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:19:09 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-24T09:19:09.555 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:19:09 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-24T09:19:09.555 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:19:09 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-24T09:19:09.555 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:19:09 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-24T09:19:09.555 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:19:09 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-24T09:19:09.555 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:19:09 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-24T09:19:09.555 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:19:09 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-24T09:19:09.555 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:19:09 smithi083 ceph-mon[27907]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-24T09:19:09.555 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:19:09 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/4021743667' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-24T09:19:09.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:09.577+0000 7f197da00640 1 -- 172.21.15.83:0/2457681175 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19781000d0 msgr2=0x7f1978100530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:09.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:09.577+0000 7f197da00640 1 --2- 172.21.15.83:0/2457681175 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19781000d0 0x7f1978100530 secure :-1 s=READY pgs=2947 cs=0 l=1 rev1=1 crypto rx=0x7f1960009900 tx=0x7f196002a480 comp rx=0 tx=0).stop 2024-04-24T09:19:09.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:09.577+0000 7f197da00640 1 -- 172.21.15.83:0/2457681175 shutdown_connections 2024-04-24T09:19:09.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:09.577+0000 7f197da00640 1 --2- 172.21.15.83:0/2457681175 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1978100a70 0x7f197810eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:09.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:09.577+0000 7f197da00640 1 --2- 172.21.15.83:0/2457681175 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19781000d0 0x7f1978100530 unknown :-1 s=CLOSED pgs=2947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:09.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:09.577+0000 7f197da00640 1 --2- 172.21.15.83:0/2457681175 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1978107060 0x7f1978107440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:09.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:09.577+0000 7f197da00640 1 -- 172.21.15.83:0/2457681175 >> 172.21.15.83:0/2457681175 conn(0x7f1978077050 msgr2=0x7f19780fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:19:09.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:09.578+0000 7f197da00640 1 -- 172.21.15.83:0/2457681175 shutdown_connections 2024-04-24T09:19:09.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:09.578+0000 7f197da00640 1 -- 172.21.15.83:0/2457681175 wait complete. 2024-04-24T09:19:09.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:09.579+0000 7f197da00640 1 Processor -- start 2024-04-24T09:19:09.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:09.579+0000 7f197da00640 1 -- start start 2024-04-24T09:19:09.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:09.579+0000 7f197da00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f19781000d0 0x7f1978101b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:09.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:09.579+0000 7f197da00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1978100a70 0x7f1978102080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:09.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:09.579+0000 7f197da00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1978107060 0x7f19781025c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:09.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:09.579+0000 7f197da00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1978112220 con 0x7f1978107060 2024-04-24T09:19:09.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:09.579+0000 7f197da00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f19781120a0 con 0x7f1978100a70 2024-04-24T09:19:09.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:09.579+0000 7f197da00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f19781123a0 con 0x7f19781000d0 2024-04-24T09:19:09.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:09.580+0000 7f1977400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1978107060 0x7f19781025c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:09.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:09.580+0000 7f1977400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1978107060 0x7f19781025c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34602/0 (socket says 172.21.15.83:34602) 2024-04-24T09:19:09.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:09.580+0000 7f1977400640 1 -- 172.21.15.83:0/1398950655 learned_addr learned my addr 172.21.15.83:0/1398950655 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:19:09.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:09.580+0000 7f196fe00640 1 --2- 172.21.15.83:0/1398950655 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1978100a70 0x7f1978102080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:09.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:09.580+0000 7f1976a00640 1 --2- 172.21.15.83:0/1398950655 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f19781000d0 0x7f1978101b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:09.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:09.580+0000 7f1977400640 1 -- 172.21.15.83:0/1398950655 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f19781000d0 msgr2=0x7f1978101b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:09.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:09.580+0000 7f1977400640 1 --2- 172.21.15.83:0/1398950655 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f19781000d0 0x7f1978101b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:09.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:09.580+0000 7f1977400640 1 -- 172.21.15.83:0/1398950655 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1978100a70 msgr2=0x7f1978102080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:09.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:09.580+0000 7f1977400640 1 --2- 172.21.15.83:0/1398950655 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1978100a70 0x7f1978102080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:09.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:09.580+0000 7f1977400640 1 -- 172.21.15.83:0/1398950655 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19781a6c80 con 0x7f1978107060 2024-04-24T09:19:09.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:09.580+0000 7f1976a00640 1 --2- 172.21.15.83:0/1398950655 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f19781000d0 0x7f1978101b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:19:09.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:09.580+0000 7f196fe00640 1 --2- 172.21.15.83:0/1398950655 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1978100a70 0x7f1978102080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:19:09.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:09.581+0000 7f1977400640 1 --2- 172.21.15.83:0/1398950655 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1978107060 0x7f19781025c0 secure :-1 s=READY pgs=2948 cs=0 l=1 rev1=1 crypto rx=0x7f197000eaa0 tx=0x7f197000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:19:09.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:09.581+0000 7f196f400640 1 -- 172.21.15.83:0/1398950655 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f197000cdb0 con 0x7f1978107060 2024-04-24T09:19:09.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:09.581+0000 7f196f400640 1 -- 172.21.15.83:0/1398950655 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1970005020 con 0x7f1978107060 2024-04-24T09:19:09.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:09.581+0000 7f196f400640 1 -- 172.21.15.83:0/1398950655 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19700116d0 con 0x7f1978107060 2024-04-24T09:19:09.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:09.581+0000 7f197da00640 1 -- 172.21.15.83:0/1398950655 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19781a6f70 con 0x7f1978107060 2024-04-24T09:19:09.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:09.581+0000 7f197da00640 1 -- 172.21.15.83:0/1398950655 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19781a73e0 con 0x7f1978107060 2024-04-24T09:19:09.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:09.583+0000 7f197da00640 1 -- 172.21.15.83:0/1398950655 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1978069270 con 0x7f1978107060 2024-04-24T09:19:09.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:09.585+0000 7f196f400640 1 -- 172.21.15.83:0/1398950655 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f19700051d0 con 0x7f1978107060 2024-04-24T09:19:09.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:09.587+0000 7f196f400640 1 --2- 172.21.15.83:0/1398950655 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f195c0a3ea0 0x7f195c0a6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:09.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:09.587+0000 7f196f400640 1 -- 172.21.15.83:0/1398950655 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(139..139 src has 1..139) ==== 9430+0+0 (secure 0 0 0) 0x7f19700d0d00 con 0x7f1978107060 2024-04-24T09:19:09.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:09.587+0000 7f1976a00640 1 --2- 172.21.15.83:0/1398950655 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f195c0a3ea0 0x7f195c0a6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:09.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:09.588+0000 7f1976a00640 1 --2- 172.21.15.83:0/1398950655 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f195c0a3ea0 0x7f195c0a6360 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f1968005fd0 tx=0x7f196800b5d0 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:19:09.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:09.592+0000 7f196f400640 1 -- 172.21.15.83:0/1398950655 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1970016030 con 0x7f1978107060 2024-04-24T09:19:09.706 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:19:09 smithi138 ceph-mon[31441]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-24T09:19:09.706 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:19:09 smithi138 ceph-mon[31441]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-24T09:19:09.706 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:19:09 smithi138 ceph-mon[31441]: Cluster is now healthy 2024-04-24T09:19:09.706 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:19:09 smithi138 ceph-mon[31441]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-24T09:19:09.706 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:19:09 smithi138 ceph-mon[31441]: mds.? [v2:172.21.15.189:6832/3982017609,v1:172.21.15.189:6833/3982017609] up:boot 2024-04-24T09:19:09.706 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:19:09 smithi138 ceph-mon[31441]: mds.? [v2:172.21.15.189:6834/2508224750,v1:172.21.15.189:6836/2508224750] up:boot 2024-04-24T09:19:09.707 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:19:09 smithi138 ceph-mon[31441]: mds.? [v2:172.21.15.189:6835/2600233222,v1:172.21.15.189:6838/2600233222] up:boot 2024-04-24T09:19:09.707 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:19:09 smithi138 ceph-mon[31441]: mds.? [v2:172.21.15.189:6837/2204997973,v1:172.21.15.189:6839/2204997973] up:boot 2024-04-24T09:19:09.707 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:19:09 smithi138 ceph-mon[31441]: mds.? [v2:172.21.15.83:6838/2304379712,v1:172.21.15.83:6839/2304379712] up:boot 2024-04-24T09:19:09.707 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:19:09 smithi138 ceph-mon[31441]: mds.? [v2:172.21.15.138:6832/818800951,v1:172.21.15.138:6833/818800951] up:boot 2024-04-24T09:19:09.707 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:19:09 smithi138 ceph-mon[31441]: mds.? [v2:172.21.15.138:6836/3159978314,v1:172.21.15.138:6837/3159978314] up:boot 2024-04-24T09:19:09.707 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:19:09 smithi138 ceph-mon[31441]: mds.? [v2:172.21.15.138:6834/3963488485,v1:172.21.15.138:6835/3963488485] up:boot 2024-04-24T09:19:09.707 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:19:09 smithi138 ceph-mon[31441]: mds.? [v2:172.21.15.83:6840/3231655400,v1:172.21.15.83:6841/3231655400] up:boot 2024-04-24T09:19:09.707 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:19:09 smithi138 ceph-mon[31441]: mds.? [v2:172.21.15.138:6838/3752197684,v1:172.21.15.138:6839/3752197684] up:boot 2024-04-24T09:19:09.707 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:19:09 smithi138 ceph-mon[31441]: fsmap 12 up:standby 2024-04-24T09:19:09.707 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:19:09 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-24T09:19:09.707 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:19:09 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-24T09:19:09.707 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:19:09 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-24T09:19:09.707 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:19:09 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-24T09:19:09.707 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:19:09 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-24T09:19:09.707 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:19:09 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-24T09:19:09.708 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:19:09 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-24T09:19:09.708 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:19:09 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-24T09:19:09.708 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:19:09 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-24T09:19:09.708 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:19:09 smithi138 ceph-mon[31441]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-24T09:19:09.708 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:19:09 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/4021743667' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-24T09:19:09.708 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:19:09 smithi189 ceph-mon[31791]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-24T09:19:09.709 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:19:09 smithi189 ceph-mon[31791]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-24T09:19:09.709 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:19:09 smithi189 ceph-mon[31791]: Cluster is now healthy 2024-04-24T09:19:09.709 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:19:09 smithi189 ceph-mon[31791]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-24T09:19:09.709 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:19:09 smithi189 ceph-mon[31791]: mds.? [v2:172.21.15.189:6832/3982017609,v1:172.21.15.189:6833/3982017609] up:boot 2024-04-24T09:19:09.709 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:19:09 smithi189 ceph-mon[31791]: mds.? [v2:172.21.15.189:6834/2508224750,v1:172.21.15.189:6836/2508224750] up:boot 2024-04-24T09:19:09.709 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:19:09 smithi189 ceph-mon[31791]: mds.? [v2:172.21.15.189:6835/2600233222,v1:172.21.15.189:6838/2600233222] up:boot 2024-04-24T09:19:09.709 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:19:09 smithi189 ceph-mon[31791]: mds.? [v2:172.21.15.189:6837/2204997973,v1:172.21.15.189:6839/2204997973] up:boot 2024-04-24T09:19:09.709 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:19:09 smithi189 ceph-mon[31791]: mds.? [v2:172.21.15.83:6838/2304379712,v1:172.21.15.83:6839/2304379712] up:boot 2024-04-24T09:19:09.709 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:19:09 smithi189 ceph-mon[31791]: mds.? [v2:172.21.15.138:6832/818800951,v1:172.21.15.138:6833/818800951] up:boot 2024-04-24T09:19:09.709 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:19:09 smithi189 ceph-mon[31791]: mds.? [v2:172.21.15.138:6836/3159978314,v1:172.21.15.138:6837/3159978314] up:boot 2024-04-24T09:19:09.709 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:19:09 smithi189 ceph-mon[31791]: mds.? [v2:172.21.15.138:6834/3963488485,v1:172.21.15.138:6835/3963488485] up:boot 2024-04-24T09:19:09.709 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:19:09 smithi189 ceph-mon[31791]: mds.? [v2:172.21.15.83:6840/3231655400,v1:172.21.15.83:6841/3231655400] up:boot 2024-04-24T09:19:09.709 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:19:09 smithi189 ceph-mon[31791]: mds.? [v2:172.21.15.138:6838/3752197684,v1:172.21.15.138:6839/3752197684] up:boot 2024-04-24T09:19:09.709 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:19:09 smithi189 ceph-mon[31791]: fsmap 12 up:standby 2024-04-24T09:19:09.709 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:19:09 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-24T09:19:09.709 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:19:09 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-24T09:19:09.710 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:19:09 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-24T09:19:09.710 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:19:09 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-24T09:19:09.710 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:19:09 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-24T09:19:09.710 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:19:09 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-24T09:19:09.710 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:19:09 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-24T09:19:09.710 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:19:09 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-24T09:19:09.710 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:19:09 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-24T09:19:09.710 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:19:09 smithi189 ceph-mon[31791]: from='mgr.15579 172.21.15.83:0/3899613094' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-24T09:19:09.710 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:19:09 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/4021743667' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-24T09:19:09.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:09.867+0000 7f197da00640 1 -- 172.21.15.83:0/1398950655 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f1978109fa0 con 0x7f1978107060 2024-04-24T09:19:10.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:10.348+0000 7f196f400640 1 -- 172.21.15.83:0/1398950655 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v140) ==== 169+0+0 (secure 0 0 0) 0x7f197009cfb0 con 0x7f1978107060 2024-04-24T09:19:10.349 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_data' removed 2024-04-24T09:19:10.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:10.351+0000 7f197da00640 1 -- 172.21.15.83:0/1398950655 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f195c0a3ea0 msgr2=0x7f195c0a6360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:10.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:10.351+0000 7f197da00640 1 --2- 172.21.15.83:0/1398950655 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f195c0a3ea0 0x7f195c0a6360 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f1968005fd0 tx=0x7f196800b5d0 comp rx=0 tx=0).stop 2024-04-24T09:19:10.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:10.352+0000 7f197da00640 1 -- 172.21.15.83:0/1398950655 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1978107060 msgr2=0x7f19781025c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:10.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:10.352+0000 7f197da00640 1 --2- 172.21.15.83:0/1398950655 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1978107060 0x7f19781025c0 secure :-1 s=READY pgs=2948 cs=0 l=1 rev1=1 crypto rx=0x7f197000eaa0 tx=0x7f197000ef60 comp rx=0 tx=0).stop 2024-04-24T09:19:10.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:10.352+0000 7f197da00640 1 -- 172.21.15.83:0/1398950655 shutdown_connections 2024-04-24T09:19:10.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:10.352+0000 7f197da00640 1 --2- 172.21.15.83:0/1398950655 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f195c0a3ea0 0x7f195c0a6360 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:10.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:10.352+0000 7f197da00640 1 --2- 172.21.15.83:0/1398950655 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1978107060 0x7f19781025c0 unknown :-1 s=CLOSED pgs=2948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:10.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:10.352+0000 7f197da00640 1 --2- 172.21.15.83:0/1398950655 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1978100a70 0x7f1978102080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:10.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:10.352+0000 7f197da00640 1 --2- 172.21.15.83:0/1398950655 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f19781000d0 0x7f1978101b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:10.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:10.352+0000 7f197da00640 1 -- 172.21.15.83:0/1398950655 >> 172.21.15.83:0/1398950655 conn(0x7f1978077050 msgr2=0x7f1978105150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:19:10.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:10.354+0000 7f197da00640 1 -- 172.21.15.83:0/1398950655 shutdown_connections 2024-04-24T09:19:10.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:10.355+0000 7f197da00640 1 -- 172.21.15.83:0/1398950655 wait complete. 2024-04-24T09:19:10.384 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-04-24T09:19:10.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:10.584+0000 7f1995e00640 1 -- 172.21.15.83:0/2355751832 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f199010e690 msgr2=0x7f199010ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:10.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:10.584+0000 7f1995e00640 1 --2- 172.21.15.83:0/2355751832 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f199010e690 0x7f199010ea70 secure :-1 s=READY pgs=2949 cs=0 l=1 rev1=1 crypto rx=0x7f1980009930 tx=0x7f198002a440 comp rx=0 tx=0).stop 2024-04-24T09:19:10.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:10.584+0000 7f1995e00640 1 -- 172.21.15.83:0/2355751832 shutdown_connections 2024-04-24T09:19:10.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:10.584+0000 7f1995e00640 1 --2- 172.21.15.83:0/2355751832 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1990108fb0 0x7f19901154c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:10.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:10.584+0000 7f1995e00640 1 --2- 172.21.15.83:0/2355751832 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f19901085f0 0x7f1990108a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:10.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:10.584+0000 7f1995e00640 1 --2- 172.21.15.83:0/2355751832 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f199010e690 0x7f199010ea70 unknown :-1 s=CLOSED pgs=2949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:10.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:10.584+0000 7f1995e00640 1 -- 172.21.15.83:0/2355751832 >> 172.21.15.83:0/2355751832 conn(0x7f1990076570 msgr2=0x7f1990078990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:19:10.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:10.584+0000 7f1995e00640 1 -- 172.21.15.83:0/2355751832 shutdown_connections 2024-04-24T09:19:10.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:10.584+0000 7f1995e00640 1 -- 172.21.15.83:0/2355751832 wait complete. 2024-04-24T09:19:10.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:10.586+0000 7f1995e00640 1 Processor -- start 2024-04-24T09:19:10.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:10.586+0000 7f1995e00640 1 -- start start 2024-04-24T09:19:10.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:10.586+0000 7f1995e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19901085f0 0x7f199019edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:10.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:10.586+0000 7f1995e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1990108fb0 0x7f199019f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:10.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:10.586+0000 7f1995e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f199010e690 0x7f19901a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:10.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:10.587+0000 7f1995e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1990118910 con 0x7f19901085f0 2024-04-24T09:19:10.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:10.587+0000 7f1995e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1990118790 con 0x7f199010e690 2024-04-24T09:19:10.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:10.587+0000 7f1995e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1990118a90 con 0x7f1990108fb0 2024-04-24T09:19:10.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:10.587+0000 7f198ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19901085f0 0x7f199019edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:10.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:10.587+0000 7f198ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19901085f0 0x7f199019edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:34618/0 (socket says 172.21.15.83:34618) 2024-04-24T09:19:10.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:10.587+0000 7f198ea00640 1 -- 172.21.15.83:0/3116912050 learned_addr learned my addr 172.21.15.83:0/3116912050 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-24T09:19:10.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:10.587+0000 7f198e000640 1 --2- 172.21.15.83:0/3116912050 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1990108fb0 0x7f199019f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:10.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:10.587+0000 7f198f400640 1 --2- 172.21.15.83:0/3116912050 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f199010e690 0x7f19901a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:10.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:10.588+0000 7f198ea00640 1 -- 172.21.15.83:0/3116912050 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1990108fb0 msgr2=0x7f199019f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:10.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:10.588+0000 7f198ea00640 1 --2- 172.21.15.83:0/3116912050 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1990108fb0 0x7f199019f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:10.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:10.588+0000 7f198ea00640 1 -- 172.21.15.83:0/3116912050 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f199010e690 msgr2=0x7f19901a3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:10.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:10.588+0000 7f198ea00640 1 --2- 172.21.15.83:0/3116912050 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f199010e690 0x7f19901a3670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:10.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:10.588+0000 7f198ea00640 1 -- 172.21.15.83:0/3116912050 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19901a3da0 con 0x7f19901085f0 2024-04-24T09:19:10.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:10.588+0000 7f198f400640 1 --2- 172.21.15.83:0/3116912050 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f199010e690 0x7f19901a3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T09:19:10.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:10.588+0000 7f198ea00640 1 --2- 172.21.15.83:0/3116912050 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19901085f0 0x7f199019edb0 secure :-1 s=READY pgs=2950 cs=0 l=1 rev1=1 crypto rx=0x7f1980031440 tx=0x7f1980003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:19:10.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:10.588+0000 7f1994a00640 1 -- 172.21.15.83:0/3116912050 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f198003f070 con 0x7f19901085f0 2024-04-24T09:19:10.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:10.588+0000 7f1994a00640 1 -- 172.21.15.83:0/3116912050 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1980003e40 con 0x7f19901085f0 2024-04-24T09:19:10.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:10.588+0000 7f1995e00640 1 -- 172.21.15.83:0/3116912050 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19901a4030 con 0x7f19901085f0 2024-04-24T09:19:10.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:10.588+0000 7f1994a00640 1 -- 172.21.15.83:0/3116912050 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1980035030 con 0x7f19901085f0 2024-04-24T09:19:10.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:10.588+0000 7f1995e00640 1 -- 172.21.15.83:0/3116912050 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19901ab920 con 0x7f19901085f0 2024-04-24T09:19:10.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:10.590+0000 7f1995e00640 1 -- 172.21.15.83:0/3116912050 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1990069270 con 0x7f19901085f0 2024-04-24T09:19:10.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:10.592+0000 7f1994a00640 1 -- 172.21.15.83:0/3116912050 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1980042070 con 0x7f19901085f0 2024-04-24T09:19:10.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:10.593+0000 7f1994a00640 1 --2- 172.21.15.83:0/3116912050 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f19740a3d60 0x7f19740a6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T09:19:10.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:10.593+0000 7f1994a00640 1 -- 172.21.15.83:0/3116912050 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 8842+0+0 (secure 0 0 0) 0x7f19800f11e0 con 0x7f19901085f0 2024-04-24T09:19:10.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:10.593+0000 7f198e000640 1 --2- 172.21.15.83:0/3116912050 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f19740a3d60 0x7f19740a6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T09:19:10.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:10.594+0000 7f198e000640 1 --2- 172.21.15.83:0/3116912050 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f19740a3d60 0x7f19740a6220 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f197c005fd0 tx=0x7f197c005850 comp rx=0 tx=0).ready entity=mgr.15579 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T09:19:10.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:10.598+0000 7f1994a00640 1 -- 172.21.15.83:0/3116912050 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f19800bd740 con 0x7f19901085f0 2024-04-24T09:19:10.607 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:19:10 smithi138 ceph-mon[31441]: pgmap v873: 129 pgs: 20 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 42 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 69 KiB/s wr, 41 op/s 2024-04-24T09:19:10.607 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:19:10 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/4021743667' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-24T09:19:10.607 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:19:10 smithi138 ceph-mon[31441]: osdmap e139: 12 total, 12 up, 12 in 2024-04-24T09:19:10.607 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:19:10 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/1398950655' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-24T09:19:10.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:19:10 smithi189 ceph-mon[31791]: pgmap v873: 129 pgs: 20 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 42 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 69 KiB/s wr, 41 op/s 2024-04-24T09:19:10.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:19:10 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/4021743667' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-24T09:19:10.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:19:10 smithi189 ceph-mon[31791]: osdmap e139: 12 total, 12 up, 12 in 2024-04-24T09:19:10.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:19:10 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/1398950655' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-24T09:19:10.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:19:10 smithi083 ceph-mon[27907]: pgmap v873: 129 pgs: 20 active+clean+snaptrim, 67 active+clean+snaptrim_wait, 42 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 69 KiB/s wr, 41 op/s 2024-04-24T09:19:10.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:19:10 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/4021743667' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-24T09:19:10.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:19:10 smithi083 ceph-mon[27907]: osdmap e139: 12 total, 12 up, 12 in 2024-04-24T09:19:10.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:19:10 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/1398950655' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-24T09:19:10.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:10.808+0000 7f1995e00640 1 -- 172.21.15.83:0/3116912050 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7f1990110b60 con 0x7f19901085f0 2024-04-24T09:19:11.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:11.362+0000 7f1994a00640 1 -- 172.21.15.83:0/3116912050 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v141) ==== 178+0+0 (secure 0 0 0) 0x7f19800bd040 con 0x7f19901085f0 2024-04-24T09:19:11.363 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_data_ec' removed 2024-04-24T09:19:11.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:11.365+0000 7f1995e00640 1 -- 172.21.15.83:0/3116912050 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f19740a3d60 msgr2=0x7f19740a6220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:11.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:11.365+0000 7f1995e00640 1 --2- 172.21.15.83:0/3116912050 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f19740a3d60 0x7f19740a6220 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f197c005fd0 tx=0x7f197c005850 comp rx=0 tx=0).stop 2024-04-24T09:19:11.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:11.365+0000 7f1995e00640 1 -- 172.21.15.83:0/3116912050 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19901085f0 msgr2=0x7f199019edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T09:19:11.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:11.365+0000 7f1995e00640 1 --2- 172.21.15.83:0/3116912050 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19901085f0 0x7f199019edb0 secure :-1 s=READY pgs=2950 cs=0 l=1 rev1=1 crypto rx=0x7f1980031440 tx=0x7f1980003cd0 comp rx=0 tx=0).stop 2024-04-24T09:19:11.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:11.366+0000 7f1995e00640 1 -- 172.21.15.83:0/3116912050 shutdown_connections 2024-04-24T09:19:11.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:11.366+0000 7f1995e00640 1 --2- 172.21.15.83:0/3116912050 >> [v2:172.21.15.83:6800/3594769126,v1:172.21.15.83:6801/3594769126] conn(0x7f19740a3d60 0x7f19740a6220 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:11.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:11.366+0000 7f1995e00640 1 --2- 172.21.15.83:0/3116912050 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f199010e690 0x7f19901a3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:11.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:11.366+0000 7f1995e00640 1 --2- 172.21.15.83:0/3116912050 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1990108fb0 0x7f199019f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:11.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:11.366+0000 7f1995e00640 1 --2- 172.21.15.83:0/3116912050 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19901085f0 0x7f199019edb0 unknown :-1 s=CLOSED pgs=2950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T09:19:11.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:11.366+0000 7f1995e00640 1 -- 172.21.15.83:0/3116912050 >> 172.21.15.83:0/3116912050 conn(0x7f1990076570 msgr2=0x7f1990077d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T09:19:11.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:11.367+0000 7f1995e00640 1 -- 172.21.15.83:0/3116912050 shutdown_connections 2024-04-24T09:19:11.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-24T09:19:11.367+0000 7f1995e00640 1 -- 172.21.15.83:0/3116912050 wait complete. 2024-04-24T09:19:11.392 DEBUG:teuthology.orchestra.run.smithi083:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-24T09:19:11.482 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-24T09:19:11.560 DEBUG:teuthology.orchestra.run.smithi189:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-24T09:19:11.636 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:19:11 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/1398950655' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-24T09:19:11.636 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:19:11 smithi189 ceph-mon[31791]: osdmap e140: 12 total, 12 up, 12 in 2024-04-24T09:19:11.636 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:19:11 smithi189 ceph-mon[31791]: from='client.? 172.21.15.83:0/3116912050' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-24T09:19:11.650 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-04-24T09:19:11.650 DEBUG:teuthology.orchestra.run.smithi083:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-24T09:19:11.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:19:11 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/1398950655' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-24T09:19:11.705 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:19:11 smithi138 ceph-mon[31441]: osdmap e140: 12 total, 12 up, 12 in 2024-04-24T09:19:11.706 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:19:11 smithi138 ceph-mon[31441]: from='client.? 172.21.15.83:0/3116912050' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-24T09:19:11.737 DEBUG:teuthology.orchestra.run.smithi138:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-24T09:19:11.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:19:11 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/1398950655' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-24T09:19:11.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:19:11 smithi083 ceph-mon[27907]: osdmap e140: 12 total, 12 up, 12 in 2024-04-24T09:19:11.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:19:11 smithi083 ceph-mon[27907]: from='client.? 172.21.15.83:0/3116912050' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-24T09:19:11.820 DEBUG:teuthology.orchestra.run.smithi189:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-24T09:19:11.860 INFO:tasks.cephadm:Stopping all daemons... 2024-04-24T09:19:11.860 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-04-24T09:19:11.860 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-bd3e039e-0215-11ef-bc93-c7b262605968@mon.a 2024-04-24T09:19:12.303 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:19:11 smithi083 systemd[1]: Stopping Ceph mon.a for bd3e039e-0215-11ef-bc93-c7b262605968... 2024-04-24T09:19:12.803 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:19:12 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mon-a[27903]: 2024-04-24T09:19:12.356+0000 7fbf85200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T09:19:12.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:19:12 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mon-a[27903]: 2024-04-24T09:19:12.356+0000 7fbf85200700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-04-24T09:19:12.804 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:19:12 smithi083 podman[131135]: 2024-04-24 09:19:12.551732295 +0000 UTC m=+0.340763189 container died 2eee14394f4c6bafb289fa6aabfe4f3e96dc4d11d2c3e85ddc981d229db7d05f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mon-a, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-24T09:19:13.274 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:19:13 smithi083 podman[131135]: 2024-04-24 09:19:13.022284173 +0000 UTC m=+0.811315074 container cleanup 2eee14394f4c6bafb289fa6aabfe4f3e96dc4d11d2c3e85ddc981d229db7d05f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mon-a, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=rishabh-main-apr17-a654945-1f81fb3, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-04-24T09:19:13.274 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:19:13 smithi083 bash[131135]: ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mon-a 2024-04-24T09:19:13.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 24 09:19:13 smithi083 podman[131199]: 2024-04-24 09:19:13.274000762 +0000 UTC m=+0.712705850 container remove 2eee14394f4c6bafb289fa6aabfe4f3e96dc4d11d2c3e85ddc981d229db7d05f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mon-a, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0) 2024-04-24T09:19:13.819 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bd3e039e-0215-11ef-bc93-c7b262605968@mon.a.service' 2024-04-24T09:19:13.946 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T09:19:13.946 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-04-24T09:19:13.946 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-04-24T09:19:13.946 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ceph-bd3e039e-0215-11ef-bc93-c7b262605968@mon.b 2024-04-24T09:19:14.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:19:14 smithi138 systemd[1]: Stopping Ceph mon.b for bd3e039e-0215-11ef-bc93-c7b262605968... 2024-04-24T09:19:14.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:19:14 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mon-b[31437]: 2024-04-24T09:19:14.670+0000 7f132b800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T09:19:14.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:19:14 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mon-b[31437]: 2024-04-24T09:19:14.670+0000 7f132b800700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-04-24T09:19:14.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:19:14 smithi138 podman[72246]: 2024-04-24 09:19:14.774838301 +0000 UTC m=+0.275983353 container died b10d504df99335c62eca80b46119fa2e150d7165d35ad2fe83a232c680edaf09 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mon-b, RELEASE=rishabh-main-apr17-a654945-1f81fb3, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-04-24T09:19:15.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:19:15 smithi138 podman[72246]: 2024-04-24 09:19:15.249162429 +0000 UTC m=+0.750307475 container cleanup b10d504df99335c62eca80b46119fa2e150d7165d35ad2fe83a232c680edaf09 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mon-b, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T09:19:15.455 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:19:15 smithi138 bash[72246]: ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mon-b 2024-04-24T09:19:15.955 INFO:journalctl@ceph.mon.b.smithi138.stdout:Apr 24 09:19:15 smithi138 podman[72261]: 2024-04-24 09:19:15.526182263 +0000 UTC m=+0.731966061 container remove b10d504df99335c62eca80b46119fa2e150d7165d35ad2fe83a232c680edaf09 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mon-b, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=rishabh-main-apr17-a654945-1f81fb3, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS) 2024-04-24T09:19:16.058 DEBUG:teuthology.orchestra.run.smithi138:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bd3e039e-0215-11ef-bc93-c7b262605968@mon.b.service' 2024-04-24T09:19:16.184 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T09:19:16.184 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-04-24T09:19:16.185 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-04-24T09:19:16.185 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ceph-bd3e039e-0215-11ef-bc93-c7b262605968@mon.c 2024-04-24T09:19:16.707 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:19:16 smithi189 systemd[1]: Stopping Ceph mon.c for bd3e039e-0215-11ef-bc93-c7b262605968... 2024-04-24T09:19:17.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:19:16 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mon-c[31783]: 2024-04-24T09:19:16.775+0000 7f43b6400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T09:19:17.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:19:16 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mon-c[31783]: 2024-04-24T09:19:16.775+0000 7f43b6400700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-04-24T09:19:17.208 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:19:16 smithi189 podman[76451]: 2024-04-24 09:19:16.90026962 +0000 UTC m=+0.268566614 container died 26c5d9f35126a527a600e689ef7420bdf2016000a6fe6bd7cc23b3fef2da291a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mon-c, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=rishabh-main-apr17-a654945-1f81fb3, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T09:19:17.822 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:19:17 smithi189 podman[76451]: 2024-04-24 09:19:17.594983952 +0000 UTC m=+0.963280973 container cleanup 26c5d9f35126a527a600e689ef7420bdf2016000a6fe6bd7cc23b3fef2da291a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mon-c, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=rishabh-main-apr17-a654945-1f81fb3, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-04-24T09:19:17.822 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:19:17 smithi189 bash[76451]: ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mon-c 2024-04-24T09:19:18.207 INFO:journalctl@ceph.mon.c.smithi189.stdout:Apr 24 09:19:17 smithi189 podman[76466]: 2024-04-24 09:19:17.821665125 +0000 UTC m=+0.902456279 container remove 26c5d9f35126a527a600e689ef7420bdf2016000a6fe6bd7cc23b3fef2da291a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mon-c, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD) 2024-04-24T09:19:18.343 DEBUG:teuthology.orchestra.run.smithi189:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bd3e039e-0215-11ef-bc93-c7b262605968@mon.c.service' 2024-04-24T09:19:18.473 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T09:19:18.473 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-04-24T09:19:18.473 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-04-24T09:19:18.473 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-bd3e039e-0215-11ef-bc93-c7b262605968@mgr.x 2024-04-24T09:19:18.804 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 09:19:18 smithi083 systemd[1]: Stopping Ceph mgr.x for bd3e039e-0215-11ef-bc93-c7b262605968... 2024-04-24T09:19:19.303 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 09:19:19 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T09:19:19.043+0000 7f3b8ac00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T09:19:19.303 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 09:19:19 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x[28137]: 2024-04-24T09:19:19.043+0000 7f3b8ac00700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-04-24T09:19:19.304 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 09:19:19 smithi083 podman[131349]: 2024-04-24 09:19:19.135894292 +0000 UTC m=+0.222220568 container died c4a5d09d7dac09d1f202ebaf3029d3f218e0b0b53d61de96dd3dd22217a8a1cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True) 2024-04-24T09:19:19.803 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 09:19:19 smithi083 podman[131349]: 2024-04-24 09:19:19.645082241 +0000 UTC m=+0.731408539 container cleanup c4a5d09d7dac09d1f202ebaf3029d3f218e0b0b53d61de96dd3dd22217a8a1cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True) 2024-04-24T09:19:19.803 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 09:19:19 smithi083 bash[131349]: ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x 2024-04-24T09:19:20.291 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 24 09:19:19 smithi083 podman[131363]: 2024-04-24 09:19:19.871822932 +0000 UTC m=+0.719526562 container remove c4a5d09d7dac09d1f202ebaf3029d3f218e0b0b53d61de96dd3dd22217a8a1cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1f81fb397ae98da7563d451a78c61574c8f4e6e0, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-x, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, RELEASE=rishabh-main-apr17-a654945-1f81fb3, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-24T09:19:20.337 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bd3e039e-0215-11ef-bc93-c7b262605968@mgr.x.service' 2024-04-24T09:19:20.451 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T09:19:20.451 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-04-24T09:19:20.451 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-04-24T09:19:20.452 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ceph-bd3e039e-0215-11ef-bc93-c7b262605968@mgr.y 2024-04-24T09:19:20.955 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Apr 24 09:19:20 smithi138 systemd[1]: Stopping Ceph mgr.y for bd3e039e-0215-11ef-bc93-c7b262605968... 2024-04-24T09:19:21.427 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Apr 24 09:19:21 smithi138 podman[72369]: 2024-04-24 09:19:21.093079218 +0000 UTC m=+0.242838572 container died 19f774befef758b542cb0f7f8f0d8b0a74c3520a2c0509a9341cf45580dd10ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-y, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=rishabh-main-apr17-a654945-1f81fb3, io.buildah.version=1.33.5) 2024-04-24T09:19:21.704 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Apr 24 09:19:21 smithi138 podman[72369]: 2024-04-24 09:19:21.614538418 +0000 UTC m=+0.764297797 container cleanup 19f774befef758b542cb0f7f8f0d8b0a74c3520a2c0509a9341cf45580dd10ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-y, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=rishabh-main-apr17-a654945-1f81fb3, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T09:19:21.705 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Apr 24 09:19:21 smithi138 bash[72369]: ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-y 2024-04-24T09:19:22.205 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Apr 24 09:19:21 smithi138 podman[72382]: 2024-04-24 09:19:21.891814806 +0000 UTC m=+0.780539443 container remove 19f774befef758b542cb0f7f8f0d8b0a74c3520a2c0509a9341cf45580dd10ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-y, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-04-24T09:19:22.205 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Apr 24 09:19:21 smithi138 systemd[1]: ceph-bd3e039e-0215-11ef-bc93-c7b262605968@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-04-24T09:19:22.468 DEBUG:teuthology.orchestra.run.smithi138:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bd3e039e-0215-11ef-bc93-c7b262605968@mgr.y.service' 2024-04-24T09:19:22.583 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T09:19:22.583 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-04-24T09:19:22.583 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-04-24T09:19:22.584 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ceph-bd3e039e-0215-11ef-bc93-c7b262605968@mgr.z 2024-04-24T09:19:22.957 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 24 09:19:22 smithi189 systemd[1]: Stopping Ceph mgr.z for bd3e039e-0215-11ef-bc93-c7b262605968... 2024-04-24T09:19:23.582 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 24 09:19:23 smithi189 podman[76583]: 2024-04-24 09:19:23.254222211 +0000 UTC m=+0.221396110 container died 40d25381f5bdc0fcd785d74294ca01f0be90dbe445041a6f67099ce178449e6c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-z, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-24T09:19:23.957 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 24 09:19:23 smithi189 podman[76583]: 2024-04-24 09:19:23.800581377 +0000 UTC m=+0.767755316 container cleanup 40d25381f5bdc0fcd785d74294ca01f0be90dbe445041a6f67099ce178449e6c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-z, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=rishabh-main-apr17-a654945-1f81fb3, io.buildah.version=1.33.5) 2024-04-24T09:19:23.957 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 24 09:19:23 smithi189 bash[76583]: ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-z 2024-04-24T09:19:24.457 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 24 09:19:24 smithi189 podman[76597]: 2024-04-24 09:19:24.0353214 +0000 UTC m=+0.765727637 container remove 40d25381f5bdc0fcd785d74294ca01f0be90dbe445041a6f67099ce178449e6c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mgr-z, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-04-24T09:19:24.457 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Apr 24 09:19:24 smithi189 systemd[1]: ceph-bd3e039e-0215-11ef-bc93-c7b262605968@mgr.z.service: Main process exited, code=exited, status=143/n/a 2024-04-24T09:19:24.551 DEBUG:teuthology.orchestra.run.smithi189:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bd3e039e-0215-11ef-bc93-c7b262605968@mgr.z.service' 2024-04-24T09:19:24.658 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T09:19:24.658 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-04-24T09:19:24.658 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-04-24T09:19:24.658 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-bd3e039e-0215-11ef-bc93-c7b262605968@osd.0 2024-04-24T09:19:25.053 INFO:journalctl@ceph.osd.0.smithi083.stdout:Apr 24 09:19:24 smithi083 systemd[1]: Stopping Ceph osd.0 for bd3e039e-0215-11ef-bc93-c7b262605968... 2024-04-24T09:19:25.554 INFO:journalctl@ceph.osd.0.smithi083.stdout:Apr 24 09:19:25 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-0[42292]: 2024-04-24T09:19:25.265+0000 7f2f97c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T09:19:25.554 INFO:journalctl@ceph.osd.0.smithi083.stdout:Apr 24 09:19:25 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-0[42292]: 2024-04-24T09:19:25.265+0000 7f2f97c00700 -1 osd.0 141 *** Got signal Terminated *** 2024-04-24T09:19:25.554 INFO:journalctl@ceph.osd.0.smithi083.stdout:Apr 24 09:19:25 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-0[42292]: 2024-04-24T09:19:25.265+0000 7f2f97c00700 -1 osd.0 141 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T09:19:31.140 INFO:journalctl@ceph.osd.0.smithi083.stdout:Apr 24 09:19:30 smithi083 podman[131472]: 2024-04-24 09:19:30.882410863 +0000 UTC m=+5.767328293 container died 78641b700829a57f0007fb24c80ade849d9c06116c1318efd36675070b3b9256 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=rishabh-main-apr17-a654945-1f81fb3, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-04-24T09:19:31.510 INFO:journalctl@ceph.osd.0.smithi083.stdout:Apr 24 09:19:31 smithi083 podman[131472]: 2024-04-24 09:19:31.275466905 +0000 UTC m=+6.160384373 container cleanup 78641b700829a57f0007fb24c80ade849d9c06116c1318efd36675070b3b9256 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-24T09:19:31.510 INFO:journalctl@ceph.osd.0.smithi083.stdout:Apr 24 09:19:31 smithi083 bash[131472]: ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-0 2024-04-24T09:19:31.803 INFO:journalctl@ceph.osd.0.smithi083.stdout:Apr 24 09:19:31 smithi083 podman[131495]: 2024-04-24 09:19:31.510133997 +0000 UTC m=+0.610374801 container remove 78641b700829a57f0007fb24c80ade849d9c06116c1318efd36675070b3b9256 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-0, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T09:19:32.632 INFO:journalctl@ceph.osd.0.smithi083.stdout:Apr 24 09:19:32 smithi083 podman[131580]: 2024-04-24 09:19:32.212010101 +0000 UTC m=+0.035848692 image pull b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644 2024-04-24T09:19:32.632 INFO:journalctl@ceph.osd.0.smithi083.stdout:Apr 24 09:19:32 smithi083 podman[131580]: 2024-04-24 09:19:32.336321419 +0000 UTC m=+0.160160037 container create 891f3c41fcf26b6a02e862da736a7482caf02351ce9411d539d79aa1c327ce75 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux ) 2024-04-24T09:19:32.632 INFO:journalctl@ceph.osd.0.smithi083.stdout:Apr 24 09:19:32 smithi083 podman[131580]: 2024-04-24 09:19:32.631598345 +0000 UTC m=+0.455436970 container init 891f3c41fcf26b6a02e862da736a7482caf02351ce9411d539d79aa1c327ce75 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-0-deactivate, maintainer=Guillaume Abrioux , RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-04-24T09:19:33.053 INFO:journalctl@ceph.osd.0.smithi083.stdout:Apr 24 09:19:32 smithi083 podman[131580]: 2024-04-24 09:19:32.644048843 +0000 UTC m=+0.467887468 container start 891f3c41fcf26b6a02e862da736a7482caf02351ce9411d539d79aa1c327ce75 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-24T09:19:33.053 INFO:journalctl@ceph.osd.0.smithi083.stdout:Apr 24 09:19:32 smithi083 podman[131580]: 2024-04-24 09:19:32.747437601 +0000 UTC m=+0.571276266 container attach 891f3c41fcf26b6a02e862da736a7482caf02351ce9411d539d79aa1c327ce75 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T09:19:33.803 INFO:journalctl@ceph.osd.0.smithi083.stdout:Apr 24 09:19:33 smithi083 podman[131580]: 2024-04-24 09:19:33.472589366 +0000 UTC m=+1.296427991 container died 891f3c41fcf26b6a02e862da736a7482caf02351ce9411d539d79aa1c327ce75 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-0-deactivate, maintainer=Guillaume Abrioux , RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=) 2024-04-24T09:19:34.235 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bd3e039e-0215-11ef-bc93-c7b262605968@osd.0.service' 2024-04-24T09:19:34.339 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T09:19:34.340 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-04-24T09:19:34.340 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-04-24T09:19:34.340 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ceph-bd3e039e-0215-11ef-bc93-c7b262605968@osd.1 2024-04-24T09:19:34.705 INFO:journalctl@ceph.osd.1.smithi138.stdout:Apr 24 09:19:34 smithi138 systemd[1]: Stopping Ceph osd.1 for bd3e039e-0215-11ef-bc93-c7b262605968... 2024-04-24T09:19:35.205 INFO:journalctl@ceph.osd.1.smithi138.stdout:Apr 24 09:19:34 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-1[36048]: 2024-04-24T09:19:34.935+0000 7f09d0e00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T09:19:35.205 INFO:journalctl@ceph.osd.1.smithi138.stdout:Apr 24 09:19:34 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-1[36048]: 2024-04-24T09:19:34.935+0000 7f09d0e00700 -1 osd.1 141 *** Got signal Terminated *** 2024-04-24T09:19:35.206 INFO:journalctl@ceph.osd.1.smithi138.stdout:Apr 24 09:19:34 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-1[36048]: 2024-04-24T09:19:34.935+0000 7f09d0e00700 -1 osd.1 141 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T09:19:40.925 INFO:journalctl@ceph.osd.1.smithi138.stdout:Apr 24 09:19:40 smithi138 podman[72489]: 2024-04-24 09:19:40.587928714 +0000 UTC m=+5.801652774 container died 0371890d4f5a3680596fddb0d099c70db094dad7e1cd9bcd1c1c2761af348191 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-1, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240424, RELEASE=rishabh-main-apr17-a654945-1f81fb3, maintainer=Guillaume Abrioux ) 2024-04-24T09:19:41.204 INFO:journalctl@ceph.osd.1.smithi138.stdout:Apr 24 09:19:41 smithi138 podman[72489]: 2024-04-24 09:19:41.046330653 +0000 UTC m=+6.260054723 container cleanup 0371890d4f5a3680596fddb0d099c70db094dad7e1cd9bcd1c1c2761af348191 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-1, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=rishabh-main-apr17-a654945-1f81fb3, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T09:19:41.205 INFO:journalctl@ceph.osd.1.smithi138.stdout:Apr 24 09:19:41 smithi138 bash[72489]: ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-1 2024-04-24T09:19:41.705 INFO:journalctl@ceph.osd.1.smithi138.stdout:Apr 24 09:19:41 smithi138 podman[72508]: 2024-04-24 09:19:41.314903227 +0000 UTC m=+0.717232074 container remove 0371890d4f5a3680596fddb0d099c70db094dad7e1cd9bcd1c1c2761af348191 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-1, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T09:19:42.455 INFO:journalctl@ceph.osd.1.smithi138.stdout:Apr 24 09:19:42 smithi138 podman[72602]: 2024-04-24 09:19:42.012970029 +0000 UTC m=+0.047458142 image pull b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644 2024-04-24T09:19:42.455 INFO:journalctl@ceph.osd.1.smithi138.stdout:Apr 24 09:19:42 smithi138 podman[72602]: 2024-04-24 09:19:42.149700912 +0000 UTC m=+0.184189065 container create b4bc26e5273fdc686dba65cb0c95b465b91f1b2f1c4934ee4651d1dcf6860fe0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-1-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424) 2024-04-24T09:19:42.955 INFO:journalctl@ceph.osd.1.smithi138.stdout:Apr 24 09:19:42 smithi138 podman[72602]: 2024-04-24 09:19:42.479439615 +0000 UTC m=+0.513927789 container init b4bc26e5273fdc686dba65cb0c95b465b91f1b2f1c4934ee4651d1dcf6860fe0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=rishabh-main-apr17-a654945-1f81fb3, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-04-24T09:19:42.955 INFO:journalctl@ceph.osd.1.smithi138.stdout:Apr 24 09:19:42 smithi138 podman[72602]: 2024-04-24 09:19:42.490737915 +0000 UTC m=+0.525226072 container start b4bc26e5273fdc686dba65cb0c95b465b91f1b2f1c4934ee4651d1dcf6860fe0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=rishabh-main-apr17-a654945-1f81fb3, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux ) 2024-04-24T09:19:42.955 INFO:journalctl@ceph.osd.1.smithi138.stdout:Apr 24 09:19:42 smithi138 podman[72602]: 2024-04-24 09:19:42.6283659 +0000 UTC m=+0.662854028 container attach b4bc26e5273fdc686dba65cb0c95b465b91f1b2f1c4934ee4651d1dcf6860fe0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T09:19:43.705 INFO:journalctl@ceph.osd.1.smithi138.stdout:Apr 24 09:19:43 smithi138 podman[72602]: 2024-04-24 09:19:43.412458105 +0000 UTC m=+1.446946262 container died b4bc26e5273fdc686dba65cb0c95b465b91f1b2f1c4934ee4651d1dcf6860fe0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-24T09:19:44.199 DEBUG:teuthology.orchestra.run.smithi138:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bd3e039e-0215-11ef-bc93-c7b262605968@osd.1.service' 2024-04-24T09:19:44.320 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T09:19:44.320 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-04-24T09:19:44.320 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-04-24T09:19:44.320 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ceph-bd3e039e-0215-11ef-bc93-c7b262605968@osd.2 2024-04-24T09:19:44.707 INFO:journalctl@ceph.osd.2.smithi189.stdout:Apr 24 09:19:44 smithi189 systemd[1]: Stopping Ceph osd.2 for bd3e039e-0215-11ef-bc93-c7b262605968... 2024-04-24T09:19:45.207 INFO:journalctl@ceph.osd.2.smithi189.stdout:Apr 24 09:19:44 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-2[36951]: 2024-04-24T09:19:44.885+0000 7fd8bc600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T09:19:45.207 INFO:journalctl@ceph.osd.2.smithi189.stdout:Apr 24 09:19:44 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-2[36951]: 2024-04-24T09:19:44.885+0000 7fd8bc600700 -1 osd.2 141 *** Got signal Terminated *** 2024-04-24T09:19:45.208 INFO:journalctl@ceph.osd.2.smithi189.stdout:Apr 24 09:19:44 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-2[36951]: 2024-04-24T09:19:44.885+0000 7fd8bc600700 -1 osd.2 141 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T09:19:50.957 INFO:journalctl@ceph.osd.2.smithi189.stdout:Apr 24 09:19:50 smithi189 podman[76708]: 2024-04-24 09:19:50.686192071 +0000 UTC m=+5.935183981 container died b3aabdbe9519ab5d061ab91ca1a3f52ec13e090a4dbe9f6a6526ad0af5afb1c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-24T09:19:51.372 INFO:journalctl@ceph.osd.2.smithi189.stdout:Apr 24 09:19:51 smithi189 podman[76708]: 2024-04-24 09:19:51.112361406 +0000 UTC m=+6.361353359 container cleanup b3aabdbe9519ab5d061ab91ca1a3f52ec13e090a4dbe9f6a6526ad0af5afb1c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-2, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-04-24T09:19:51.372 INFO:journalctl@ceph.osd.2.smithi189.stdout:Apr 24 09:19:51 smithi189 bash[76708]: ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-2 2024-04-24T09:19:51.707 INFO:journalctl@ceph.osd.2.smithi189.stdout:Apr 24 09:19:51 smithi189 podman[76738]: 2024-04-24 09:19:51.371831139 +0000 UTC m=+0.669462930 container remove b3aabdbe9519ab5d061ab91ca1a3f52ec13e090a4dbe9f6a6526ad0af5afb1c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=rishabh-main-apr17-a654945-1f81fb3, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-04-24T09:19:52.458 INFO:journalctl@ceph.osd.2.smithi189.stdout:Apr 24 09:19:52 smithi189 podman[76822]: 2024-04-24 09:19:52.107355887 +0000 UTC m=+0.039117683 image pull b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644 2024-04-24T09:19:52.458 INFO:journalctl@ceph.osd.2.smithi189.stdout:Apr 24 09:19:52 smithi189 podman[76822]: 2024-04-24 09:19:52.231243883 +0000 UTC m=+0.163005707 container create 2a453c422a406cbec65a6eaecbeb2cf3d462bb5b76fd39a3e5cdfa7ad2d6cfce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-2-deactivate, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS) 2024-04-24T09:19:52.957 INFO:journalctl@ceph.osd.2.smithi189.stdout:Apr 24 09:19:52 smithi189 podman[76822]: 2024-04-24 09:19:52.577053137 +0000 UTC m=+0.508814968 container init 2a453c422a406cbec65a6eaecbeb2cf3d462bb5b76fd39a3e5cdfa7ad2d6cfce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424) 2024-04-24T09:19:52.957 INFO:journalctl@ceph.osd.2.smithi189.stdout:Apr 24 09:19:52 smithi189 podman[76822]: 2024-04-24 09:19:52.588717405 +0000 UTC m=+0.520479254 container start 2a453c422a406cbec65a6eaecbeb2cf3d462bb5b76fd39a3e5cdfa7ad2d6cfce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-2-deactivate, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-04-24T09:19:52.957 INFO:journalctl@ceph.osd.2.smithi189.stdout:Apr 24 09:19:52 smithi189 podman[76822]: 2024-04-24 09:19:52.684364141 +0000 UTC m=+0.616125999 container attach 2a453c422a406cbec65a6eaecbeb2cf3d462bb5b76fd39a3e5cdfa7ad2d6cfce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=rishabh-main-apr17-a654945-1f81fb3, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240424) 2024-04-24T09:19:52.957 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:19:52 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:19:52.676+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:28.209530+0000 front 2024-04-24T09:19:28.209142+0000 (oldest deadline 2024-04-24T09:19:51.709380+0000) 2024-04-24T09:19:53.361 INFO:journalctl@ceph.osd.2.smithi189.stdout:Apr 24 09:19:53 smithi189 podman[76822]: 2024-04-24 09:19:53.041347679 +0000 UTC m=+0.973109511 container died 2a453c422a406cbec65a6eaecbeb2cf3d462bb5b76fd39a3e5cdfa7ad2d6cfce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-2-deactivate, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD) 2024-04-24T09:19:53.361 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 24 09:19:53 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-5[43532]: 2024-04-24T09:19:53.092+0000 7f3188c00700 -1 osd.5 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:28.938580+0000 front 2024-04-24T09:19:28.938712+0000 (oldest deadline 2024-04-24T09:19:52.438846+0000) 2024-04-24T09:19:53.476 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:19:53 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:19:53.217+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:27.386588+0000 front 2024-04-24T09:19:27.386891+0000 (oldest deadline 2024-04-24T09:19:52.686559+0000) 2024-04-24T09:19:53.791 DEBUG:teuthology.orchestra.run.smithi189:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bd3e039e-0215-11ef-bc93-c7b262605968@osd.2.service' 2024-04-24T09:19:53.903 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T09:19:53.903 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-04-24T09:19:53.903 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-04-24T09:19:53.903 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-bd3e039e-0215-11ef-bc93-c7b262605968@osd.3 2024-04-24T09:19:53.957 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:19:53 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:19:53.686+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:28.209530+0000 front 2024-04-24T09:19:28.209142+0000 (oldest deadline 2024-04-24T09:19:51.709380+0000) 2024-04-24T09:19:54.233 INFO:journalctl@ceph.osd.3.smithi083.stdout:Apr 24 09:19:54 smithi083 systemd[1]: Stopping Ceph osd.3 for bd3e039e-0215-11ef-bc93-c7b262605968... 2024-04-24T09:19:54.457 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 24 09:19:54 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-5[43532]: 2024-04-24T09:19:54.101+0000 7f3188c00700 -1 osd.5 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:28.938580+0000 front 2024-04-24T09:19:28.938712+0000 (oldest deadline 2024-04-24T09:19:52.438846+0000) 2024-04-24T09:19:54.553 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:19:54 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:19:54.231+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:27.386588+0000 front 2024-04-24T09:19:27.386891+0000 (oldest deadline 2024-04-24T09:19:52.686559+0000) 2024-04-24T09:19:54.553 INFO:journalctl@ceph.osd.3.smithi083.stdout:Apr 24 09:19:54 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-3[49218]: 2024-04-24T09:19:54.432+0000 7fbc9ac00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T09:19:54.553 INFO:journalctl@ceph.osd.3.smithi083.stdout:Apr 24 09:19:54 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-3[49218]: 2024-04-24T09:19:54.432+0000 7fbc9ac00700 -1 osd.3 141 *** Got signal Terminated *** 2024-04-24T09:19:54.554 INFO:journalctl@ceph.osd.3.smithi083.stdout:Apr 24 09:19:54 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-3[49218]: 2024-04-24T09:19:54.432+0000 7fbc9ac00700 -1 osd.3 141 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T09:19:54.957 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:19:54 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:19:54.650+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:28.209530+0000 front 2024-04-24T09:19:28.209142+0000 (oldest deadline 2024-04-24T09:19:51.709380+0000) 2024-04-24T09:19:55.457 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 24 09:19:55 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-5[43532]: 2024-04-24T09:19:55.125+0000 7f3188c00700 -1 osd.5 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:28.938580+0000 front 2024-04-24T09:19:28.938712+0000 (oldest deadline 2024-04-24T09:19:52.438846+0000) 2024-04-24T09:19:55.528 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:19:55 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:19:55.246+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:27.386588+0000 front 2024-04-24T09:19:27.386891+0000 (oldest deadline 2024-04-24T09:19:52.686559+0000) 2024-04-24T09:19:55.803 INFO:journalctl@ceph.osd.3.smithi083.stdout:Apr 24 09:19:55 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-3[49218]: 2024-04-24T09:19:55.673+0000 7fbc91400700 -1 osd.3 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:27.454163+0000 front 2024-04-24T09:19:27.454266+0000 (oldest deadline 2024-04-24T09:19:55.055616+0000) 2024-04-24T09:19:55.956 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:19:55 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:19:55.629+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:29.414806+0000 front 2024-04-24T09:19:29.414861+0000 (oldest deadline 2024-04-24T09:19:55.314800+0000) 2024-04-24T09:19:55.963 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:19:55 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:19:55.692+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:28.209530+0000 front 2024-04-24T09:19:28.209142+0000 (oldest deadline 2024-04-24T09:19:51.709380+0000) 2024-04-24T09:19:56.457 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 24 09:19:56 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-5[43532]: 2024-04-24T09:19:56.086+0000 7f3188c00700 -1 osd.5 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:28.938580+0000 front 2024-04-24T09:19:28.938712+0000 (oldest deadline 2024-04-24T09:19:52.438846+0000) 2024-04-24T09:19:56.553 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:19:56 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:19:56.254+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:27.386588+0000 front 2024-04-24T09:19:27.386891+0000 (oldest deadline 2024-04-24T09:19:52.686559+0000) 2024-04-24T09:19:56.955 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:19:56 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:19:56.592+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:29.414806+0000 front 2024-04-24T09:19:29.414861+0000 (oldest deadline 2024-04-24T09:19:55.314800+0000) 2024-04-24T09:19:57.053 INFO:journalctl@ceph.osd.3.smithi083.stdout:Apr 24 09:19:56 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-3[49218]: 2024-04-24T09:19:56.678+0000 7fbc91400700 -1 osd.3 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:27.454163+0000 front 2024-04-24T09:19:27.454266+0000 (oldest deadline 2024-04-24T09:19:55.055616+0000) 2024-04-24T09:19:57.098 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:19:56 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:19:56.720+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:28.209530+0000 front 2024-04-24T09:19:28.209142+0000 (oldest deadline 2024-04-24T09:19:51.709380+0000) 2024-04-24T09:19:57.457 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 24 09:19:57 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-5[43532]: 2024-04-24T09:19:57.098+0000 7f3188c00700 -1 osd.5 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:28.938580+0000 front 2024-04-24T09:19:28.938712+0000 (oldest deadline 2024-04-24T09:19:52.438846+0000) 2024-04-24T09:19:57.553 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:19:57 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:19:57.211+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:27.386588+0000 front 2024-04-24T09:19:27.386891+0000 (oldest deadline 2024-04-24T09:19:52.686559+0000) 2024-04-24T09:19:57.955 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:19:57 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:19:57.618+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:29.414806+0000 front 2024-04-24T09:19:29.414861+0000 (oldest deadline 2024-04-24T09:19:55.314800+0000) 2024-04-24T09:19:57.957 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:19:57 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:19:57.683+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:28.209530+0000 front 2024-04-24T09:19:28.209142+0000 (oldest deadline 2024-04-24T09:19:51.709380+0000) 2024-04-24T09:19:58.053 INFO:journalctl@ceph.osd.3.smithi083.stdout:Apr 24 09:19:57 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-3[49218]: 2024-04-24T09:19:57.703+0000 7fbc91400700 -1 osd.3 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:27.454163+0000 front 2024-04-24T09:19:27.454266+0000 (oldest deadline 2024-04-24T09:19:55.055616+0000) 2024-04-24T09:19:58.457 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 24 09:19:58 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-5[43532]: 2024-04-24T09:19:58.130+0000 7f3188c00700 -1 osd.5 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:28.938580+0000 front 2024-04-24T09:19:28.938712+0000 (oldest deadline 2024-04-24T09:19:52.438846+0000) 2024-04-24T09:19:58.553 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:19:58 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:19:58.194+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:27.386588+0000 front 2024-04-24T09:19:27.386891+0000 (oldest deadline 2024-04-24T09:19:52.686559+0000) 2024-04-24T09:19:58.955 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:19:58 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:19:58.650+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:29.414806+0000 front 2024-04-24T09:19:29.414861+0000 (oldest deadline 2024-04-24T09:19:55.314800+0000) 2024-04-24T09:19:59.053 INFO:journalctl@ceph.osd.3.smithi083.stdout:Apr 24 09:19:58 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-3[49218]: 2024-04-24T09:19:58.680+0000 7fbc91400700 -1 osd.3 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:27.454163+0000 front 2024-04-24T09:19:27.454266+0000 (oldest deadline 2024-04-24T09:19:55.055616+0000) 2024-04-24T09:19:59.124 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:19:58 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:19:58.708+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:28.209530+0000 front 2024-04-24T09:19:28.209142+0000 (oldest deadline 2024-04-24T09:19:51.709380+0000) 2024-04-24T09:19:59.457 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 24 09:19:59 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-5[43532]: 2024-04-24T09:19:59.124+0000 7f3188c00700 -1 osd.5 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:28.938580+0000 front 2024-04-24T09:19:28.938712+0000 (oldest deadline 2024-04-24T09:19:52.438846+0000) 2024-04-24T09:19:59.553 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:19:59 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:19:59.164+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:27.386588+0000 front 2024-04-24T09:19:27.386891+0000 (oldest deadline 2024-04-24T09:19:52.686559+0000) 2024-04-24T09:19:59.955 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:19:59 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:19:59.645+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:29.414806+0000 front 2024-04-24T09:19:29.414861+0000 (oldest deadline 2024-04-24T09:19:55.314800+0000) 2024-04-24T09:19:59.957 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:19:59 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:19:59.680+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:28.209530+0000 front 2024-04-24T09:19:28.209142+0000 (oldest deadline 2024-04-24T09:19:51.709380+0000) 2024-04-24T09:20:00.457 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 24 09:20:00 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-5[43532]: 2024-04-24T09:20:00.149+0000 7f3188c00700 -1 osd.5 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:28.938580+0000 front 2024-04-24T09:19:28.938712+0000 (oldest deadline 2024-04-24T09:19:52.438846+0000) 2024-04-24T09:20:00.501 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:00 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:00.204+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:27.386588+0000 front 2024-04-24T09:19:27.386891+0000 (oldest deadline 2024-04-24T09:19:52.686559+0000) 2024-04-24T09:20:00.501 INFO:journalctl@ceph.osd.3.smithi083.stdout:Apr 24 09:20:00 smithi083 podman[131686]: 2024-04-24 09:20:00.199065645 +0000 UTC m=+5.903835100 container died 045e0dfeee7ad90931a202343dd481bc3ea6a5364287862a422c890bcc32d0d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-3, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.vendor=CentOS) 2024-04-24T09:20:00.803 INFO:journalctl@ceph.osd.3.smithi083.stdout:Apr 24 09:20:00 smithi083 podman[131686]: 2024-04-24 09:20:00.629151588 +0000 UTC m=+6.333921093 container cleanup 045e0dfeee7ad90931a202343dd481bc3ea6a5364287862a422c890bcc32d0d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=rishabh-main-apr17-a654945-1f81fb3, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-24T09:20:00.803 INFO:journalctl@ceph.osd.3.smithi083.stdout:Apr 24 09:20:00 smithi083 bash[131686]: ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-3 2024-04-24T09:20:00.929 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:00 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:00.676+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:29.414806+0000 front 2024-04-24T09:19:29.414861+0000 (oldest deadline 2024-04-24T09:19:55.314800+0000) 2024-04-24T09:20:00.957 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:00 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:00.644+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:28.209530+0000 front 2024-04-24T09:19:28.209142+0000 (oldest deadline 2024-04-24T09:19:51.709380+0000) 2024-04-24T09:20:01.173 INFO:journalctl@ceph.osd.3.smithi083.stdout:Apr 24 09:20:00 smithi083 podman[131716]: 2024-04-24 09:20:00.957886529 +0000 UTC m=+0.740062914 container remove 045e0dfeee7ad90931a202343dd481bc3ea6a5364287862a422c890bcc32d0d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-3, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, RELEASE=rishabh-main-apr17-a654945-1f81fb3, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T09:20:01.205 INFO:journalctl@ceph.osd.4.smithi138.stdout:Apr 24 09:20:00 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-4[42520]: 2024-04-24T09:20:00.928+0000 7fcab6e00700 -1 osd.4 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:35.315079+0000 front 2024-04-24T09:19:35.315310+0000 (oldest deadline 2024-04-24T09:20:00.614545+0000) 2024-04-24T09:20:01.457 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 24 09:20:01 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-5[43532]: 2024-04-24T09:20:01.174+0000 7f3188c00700 -1 osd.5 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:28.938580+0000 front 2024-04-24T09:19:28.938712+0000 (oldest deadline 2024-04-24T09:19:52.438846+0000) 2024-04-24T09:20:01.457 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 24 09:20:01 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-5[43532]: 2024-04-24T09:20:01.174+0000 7f3188c00700 -1 osd.5 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.340865+0000 front 2024-04-24T09:19:38.340985+0000 (oldest deadline 2024-04-24T09:20:00.640742+0000) 2024-04-24T09:20:01.553 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:01 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:01.173+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:27.386588+0000 front 2024-04-24T09:19:27.386891+0000 (oldest deadline 2024-04-24T09:19:52.686559+0000) 2024-04-24T09:20:01.957 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:01 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:01.678+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:28.209530+0000 front 2024-04-24T09:19:28.209142+0000 (oldest deadline 2024-04-24T09:19:51.709380+0000) 2024-04-24T09:20:01.957 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:01 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:01.678+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.613982+0000 front 2024-04-24T09:19:38.614541+0000 (oldest deadline 2024-04-24T09:20:00.913987+0000) 2024-04-24T09:20:02.053 INFO:journalctl@ceph.osd.3.smithi083.stdout:Apr 24 09:20:01 smithi083 podman[131805]: 2024-04-24 09:20:01.664746918 +0000 UTC m=+0.032392445 image pull b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644 2024-04-24T09:20:02.053 INFO:journalctl@ceph.osd.3.smithi083.stdout:Apr 24 09:20:01 smithi083 podman[131805]: 2024-04-24 09:20:01.792144524 +0000 UTC m=+0.159790115 container create f3b56b30f87bc2855d1cbfd624f35d9c0605cf7be44f5892287f3d0ab2d1232d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T09:20:02.205 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:01 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:01.719+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:29.414806+0000 front 2024-04-24T09:19:29.414861+0000 (oldest deadline 2024-04-24T09:19:55.314800+0000) 2024-04-24T09:20:02.205 INFO:journalctl@ceph.osd.4.smithi138.stdout:Apr 24 09:20:01 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-4[42520]: 2024-04-24T09:20:01.965+0000 7fcab6e00700 -1 osd.4 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:35.315079+0000 front 2024-04-24T09:19:35.315310+0000 (oldest deadline 2024-04-24T09:20:00.614545+0000) 2024-04-24T09:20:02.457 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 24 09:20:02 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-5[43532]: 2024-04-24T09:20:02.194+0000 7f3188c00700 -1 osd.5 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:28.938580+0000 front 2024-04-24T09:19:28.938712+0000 (oldest deadline 2024-04-24T09:19:52.438846+0000) 2024-04-24T09:20:02.457 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 24 09:20:02 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-5[43532]: 2024-04-24T09:20:02.194+0000 7f3188c00700 -1 osd.5 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.340865+0000 front 2024-04-24T09:19:38.340985+0000 (oldest deadline 2024-04-24T09:20:00.640742+0000) 2024-04-24T09:20:02.553 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:02 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:02.193+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:27.386588+0000 front 2024-04-24T09:19:27.386891+0000 (oldest deadline 2024-04-24T09:19:52.686559+0000) 2024-04-24T09:20:02.553 INFO:journalctl@ceph.osd.3.smithi083.stdout:Apr 24 09:20:02 smithi083 podman[131805]: 2024-04-24 09:20:02.095748789 +0000 UTC m=+0.463394359 container init f3b56b30f87bc2855d1cbfd624f35d9c0605cf7be44f5892287f3d0ab2d1232d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=rishabh-main-apr17-a654945-1f81fb3, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2) 2024-04-24T09:20:02.553 INFO:journalctl@ceph.osd.3.smithi083.stdout:Apr 24 09:20:02 smithi083 podman[131805]: 2024-04-24 09:20:02.109167568 +0000 UTC m=+0.476813162 container start f3b56b30f87bc2855d1cbfd624f35d9c0605cf7be44f5892287f3d0ab2d1232d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-04-24T09:20:02.553 INFO:journalctl@ceph.osd.3.smithi083.stdout:Apr 24 09:20:02 smithi083 podman[131805]: 2024-04-24 09:20:02.203047547 +0000 UTC m=+0.570693125 container attach f3b56b30f87bc2855d1cbfd624f35d9c0605cf7be44f5892287f3d0ab2d1232d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=rishabh-main-apr17-a654945-1f81fb3, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_CLEAN=True) 2024-04-24T09:20:02.926 INFO:journalctl@ceph.osd.3.smithi083.stdout:Apr 24 09:20:02 smithi083 podman[131805]: 2024-04-24 09:20:02.559584832 +0000 UTC m=+0.927230430 container died f3b56b30f87bc2855d1cbfd624f35d9c0605cf7be44f5892287f3d0ab2d1232d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, RELEASE=rishabh-main-apr17-a654945-1f81fb3) 2024-04-24T09:20:02.957 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:02 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:02.688+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:28.209530+0000 front 2024-04-24T09:19:28.209142+0000 (oldest deadline 2024-04-24T09:19:51.709380+0000) 2024-04-24T09:20:02.957 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:02 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:02.688+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.613982+0000 front 2024-04-24T09:19:38.614541+0000 (oldest deadline 2024-04-24T09:20:00.913987+0000) 2024-04-24T09:20:03.205 INFO:journalctl@ceph.osd.4.smithi138.stdout:Apr 24 09:20:02 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-4[42520]: 2024-04-24T09:20:02.955+0000 7fcab6e00700 -1 osd.4 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:35.315079+0000 front 2024-04-24T09:19:35.315310+0000 (oldest deadline 2024-04-24T09:20:00.614545+0000) 2024-04-24T09:20:03.205 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:02 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:02.727+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:29.414806+0000 front 2024-04-24T09:19:29.414861+0000 (oldest deadline 2024-04-24T09:19:55.314800+0000) 2024-04-24T09:20:03.303 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bd3e039e-0215-11ef-bc93-c7b262605968@osd.3.service' 2024-04-24T09:20:03.406 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T09:20:03.407 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-04-24T09:20:03.407 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-04-24T09:20:03.407 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ceph-bd3e039e-0215-11ef-bc93-c7b262605968@osd.4 2024-04-24T09:20:03.457 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 24 09:20:03 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-5[43532]: 2024-04-24T09:20:03.157+0000 7f3188c00700 -1 osd.5 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:28.938580+0000 front 2024-04-24T09:19:28.938712+0000 (oldest deadline 2024-04-24T09:19:52.438846+0000) 2024-04-24T09:20:03.457 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 24 09:20:03 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-5[43532]: 2024-04-24T09:20:03.157+0000 7f3188c00700 -1 osd.5 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.340865+0000 front 2024-04-24T09:19:38.340985+0000 (oldest deadline 2024-04-24T09:20:00.640742+0000) 2024-04-24T09:20:03.553 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:03 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:03.199+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:27.386588+0000 front 2024-04-24T09:19:27.386891+0000 (oldest deadline 2024-04-24T09:19:52.686559+0000) 2024-04-24T09:20:03.955 INFO:journalctl@ceph.osd.4.smithi138.stdout:Apr 24 09:20:03 smithi138 systemd[1]: Stopping Ceph osd.4 for bd3e039e-0215-11ef-bc93-c7b262605968... 2024-04-24T09:20:03.955 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:03 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:03.680+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:29.414806+0000 front 2024-04-24T09:19:29.414861+0000 (oldest deadline 2024-04-24T09:19:55.314800+0000) 2024-04-24T09:20:03.956 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:03 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:03.680+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.119965+0000 front 2024-04-24T09:19:38.120034+0000 (oldest deadline 2024-04-24T09:20:02.820181+0000) 2024-04-24T09:20:04.177 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:03 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:03.725+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:28.209530+0000 front 2024-04-24T09:19:28.209142+0000 (oldest deadline 2024-04-24T09:19:51.709380+0000) 2024-04-24T09:20:04.177 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:03 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:03.725+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.613982+0000 front 2024-04-24T09:19:38.614541+0000 (oldest deadline 2024-04-24T09:20:00.913987+0000) 2024-04-24T09:20:04.455 INFO:journalctl@ceph.osd.4.smithi138.stdout:Apr 24 09:20:03 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-4[42520]: 2024-04-24T09:20:03.973+0000 7fcab6e00700 -1 osd.4 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:35.315079+0000 front 2024-04-24T09:19:35.315310+0000 (oldest deadline 2024-04-24T09:20:00.614545+0000) 2024-04-24T09:20:04.455 INFO:journalctl@ceph.osd.4.smithi138.stdout:Apr 24 09:20:04 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-4[42520]: 2024-04-24T09:20:04.017+0000 7fcac0600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T09:20:04.455 INFO:journalctl@ceph.osd.4.smithi138.stdout:Apr 24 09:20:04 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-4[42520]: 2024-04-24T09:20:04.017+0000 7fcac0600700 -1 osd.4 141 *** Got signal Terminated *** 2024-04-24T09:20:04.455 INFO:journalctl@ceph.osd.4.smithi138.stdout:Apr 24 09:20:04 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-4[42520]: 2024-04-24T09:20:04.017+0000 7fcac0600700 -1 osd.4 141 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T09:20:04.457 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 24 09:20:04 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-5[43532]: 2024-04-24T09:20:04.177+0000 7f3188c00700 -1 osd.5 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:28.938580+0000 front 2024-04-24T09:19:28.938712+0000 (oldest deadline 2024-04-24T09:19:52.438846+0000) 2024-04-24T09:20:04.457 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 24 09:20:04 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-5[43532]: 2024-04-24T09:20:04.177+0000 7f3188c00700 -1 osd.5 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.340865+0000 front 2024-04-24T09:19:38.340985+0000 (oldest deadline 2024-04-24T09:20:00.640742+0000) 2024-04-24T09:20:04.553 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:04 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:04.236+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:27.386588+0000 front 2024-04-24T09:19:27.386891+0000 (oldest deadline 2024-04-24T09:19:52.686559+0000) 2024-04-24T09:20:04.955 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:04 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:04.707+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:29.414806+0000 front 2024-04-24T09:19:29.414861+0000 (oldest deadline 2024-04-24T09:19:55.314800+0000) 2024-04-24T09:20:04.955 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:04 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:04.707+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.119965+0000 front 2024-04-24T09:19:38.120034+0000 (oldest deadline 2024-04-24T09:20:02.820181+0000) 2024-04-24T09:20:04.955 INFO:journalctl@ceph.osd.7.smithi138.stdout:Apr 24 09:20:04 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-7[49166]: 2024-04-24T09:20:04.586+0000 7fb49b200700 -1 osd.7 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:37.463919+0000 front 2024-04-24T09:19:37.464057+0000 (oldest deadline 2024-04-24T09:20:03.865663+0000) 2024-04-24T09:20:04.957 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:04 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:04.510+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:37.707135+0000 front 2024-04-24T09:19:37.707028+0000 (oldest deadline 2024-04-24T09:20:03.607224+0000) 2024-04-24T09:20:04.957 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:04 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:04.712+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:28.209530+0000 front 2024-04-24T09:19:28.209142+0000 (oldest deadline 2024-04-24T09:19:51.709380+0000) 2024-04-24T09:20:04.957 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:04 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:04.712+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.613982+0000 front 2024-04-24T09:19:38.614541+0000 (oldest deadline 2024-04-24T09:20:00.913987+0000) 2024-04-24T09:20:05.455 INFO:journalctl@ceph.osd.4.smithi138.stdout:Apr 24 09:20:05 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-4[42520]: 2024-04-24T09:20:05.001+0000 7fcab6e00700 -1 osd.4 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:35.315079+0000 front 2024-04-24T09:19:35.315310+0000 (oldest deadline 2024-04-24T09:20:00.614545+0000) 2024-04-24T09:20:05.457 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 24 09:20:05 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-5[43532]: 2024-04-24T09:20:05.162+0000 7f3188c00700 -1 osd.5 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:28.938580+0000 front 2024-04-24T09:19:28.938712+0000 (oldest deadline 2024-04-24T09:19:52.438846+0000) 2024-04-24T09:20:05.457 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 24 09:20:05 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-5[43532]: 2024-04-24T09:20:05.162+0000 7f3188c00700 -1 osd.5 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.340865+0000 front 2024-04-24T09:19:38.340985+0000 (oldest deadline 2024-04-24T09:20:00.640742+0000) 2024-04-24T09:20:05.553 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:05 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:05.257+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:27.386588+0000 front 2024-04-24T09:19:27.386891+0000 (oldest deadline 2024-04-24T09:19:52.686559+0000) 2024-04-24T09:20:05.553 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:05 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:05.257+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.891162+0000 front 2024-04-24T09:19:38.891270+0000 (oldest deadline 2024-04-24T09:20:04.790877+0000) 2024-04-24T09:20:05.553 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 24 09:20:05 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-6[55860]: 2024-04-24T09:20:05.368+0000 7f90fd800700 -1 osd.6 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:39.375781+0000 front 2024-04-24T09:19:39.376197+0000 (oldest deadline 2024-04-24T09:20:04.675612+0000) 2024-04-24T09:20:05.955 INFO:journalctl@ceph.osd.7.smithi138.stdout:Apr 24 09:20:05 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-7[49166]: 2024-04-24T09:20:05.609+0000 7fb49b200700 -1 osd.7 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:37.463919+0000 front 2024-04-24T09:19:37.464057+0000 (oldest deadline 2024-04-24T09:20:03.865663+0000) 2024-04-24T09:20:05.955 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:05 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:05.708+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:29.414806+0000 front 2024-04-24T09:19:29.414861+0000 (oldest deadline 2024-04-24T09:19:55.314800+0000) 2024-04-24T09:20:05.955 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:05 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:05.708+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.119965+0000 front 2024-04-24T09:19:38.120034+0000 (oldest deadline 2024-04-24T09:20:02.820181+0000) 2024-04-24T09:20:05.957 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:05 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:05.560+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:37.707135+0000 front 2024-04-24T09:19:37.707028+0000 (oldest deadline 2024-04-24T09:20:03.607224+0000) 2024-04-24T09:20:05.957 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:05 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:05.755+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:28.209530+0000 front 2024-04-24T09:19:28.209142+0000 (oldest deadline 2024-04-24T09:19:51.709380+0000) 2024-04-24T09:20:05.957 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:05 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:05.755+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.613982+0000 front 2024-04-24T09:19:38.614541+0000 (oldest deadline 2024-04-24T09:20:00.913987+0000) 2024-04-24T09:20:06.455 INFO:journalctl@ceph.osd.4.smithi138.stdout:Apr 24 09:20:06 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-4[42520]: 2024-04-24T09:20:06.047+0000 7fcab6e00700 -1 osd.4 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:35.315079+0000 front 2024-04-24T09:19:35.315310+0000 (oldest deadline 2024-04-24T09:20:00.614545+0000) 2024-04-24T09:20:06.549 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 24 09:20:06 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-5[43532]: 2024-04-24T09:20:06.212+0000 7f3188c00700 -1 osd.5 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:28.938580+0000 front 2024-04-24T09:19:28.938712+0000 (oldest deadline 2024-04-24T09:19:52.438846+0000) 2024-04-24T09:20:06.549 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 24 09:20:06 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-5[43532]: 2024-04-24T09:20:06.212+0000 7f3188c00700 -1 osd.5 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.340865+0000 front 2024-04-24T09:19:38.340985+0000 (oldest deadline 2024-04-24T09:20:00.640742+0000) 2024-04-24T09:20:06.553 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 24 09:20:06 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-6[55860]: 2024-04-24T09:20:06.331+0000 7f90fd800700 -1 osd.6 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:39.375781+0000 front 2024-04-24T09:19:39.376197+0000 (oldest deadline 2024-04-24T09:20:04.675612+0000) 2024-04-24T09:20:06.553 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:06 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:06.297+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:27.386588+0000 front 2024-04-24T09:19:27.386891+0000 (oldest deadline 2024-04-24T09:19:52.686559+0000) 2024-04-24T09:20:06.553 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:06 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:06.297+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.891162+0000 front 2024-04-24T09:19:38.891270+0000 (oldest deadline 2024-04-24T09:20:04.790877+0000) 2024-04-24T09:20:06.955 INFO:journalctl@ceph.osd.7.smithi138.stdout:Apr 24 09:20:06 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-7[49166]: 2024-04-24T09:20:06.565+0000 7fb49b200700 -1 osd.7 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:37.463919+0000 front 2024-04-24T09:19:37.464057+0000 (oldest deadline 2024-04-24T09:20:03.865663+0000) 2024-04-24T09:20:06.955 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:06 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:06.676+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:29.414806+0000 front 2024-04-24T09:19:29.414861+0000 (oldest deadline 2024-04-24T09:19:55.314800+0000) 2024-04-24T09:20:06.955 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:06 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:06.676+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.119965+0000 front 2024-04-24T09:19:38.120034+0000 (oldest deadline 2024-04-24T09:20:02.820181+0000) 2024-04-24T09:20:06.957 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:06 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:06.549+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:37.707135+0000 front 2024-04-24T09:19:37.707028+0000 (oldest deadline 2024-04-24T09:20:03.607224+0000) 2024-04-24T09:20:06.957 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:06 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:06.759+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:28.209530+0000 front 2024-04-24T09:19:28.209142+0000 (oldest deadline 2024-04-24T09:19:51.709380+0000) 2024-04-24T09:20:06.957 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:06 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:06.759+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.613982+0000 front 2024-04-24T09:19:38.614541+0000 (oldest deadline 2024-04-24T09:20:00.913987+0000) 2024-04-24T09:20:07.455 INFO:journalctl@ceph.osd.4.smithi138.stdout:Apr 24 09:20:07 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-4[42520]: 2024-04-24T09:20:07.064+0000 7fcab6e00700 -1 osd.4 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:35.315079+0000 front 2024-04-24T09:19:35.315310+0000 (oldest deadline 2024-04-24T09:20:00.614545+0000) 2024-04-24T09:20:07.529 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 24 09:20:07 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-5[43532]: 2024-04-24T09:20:07.221+0000 7f3188c00700 -1 osd.5 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:28.938580+0000 front 2024-04-24T09:19:28.938712+0000 (oldest deadline 2024-04-24T09:19:52.438846+0000) 2024-04-24T09:20:07.529 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 24 09:20:07 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-5[43532]: 2024-04-24T09:20:07.221+0000 7f3188c00700 -1 osd.5 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.340865+0000 front 2024-04-24T09:19:38.340985+0000 (oldest deadline 2024-04-24T09:20:00.640742+0000) 2024-04-24T09:20:07.803 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 24 09:20:07 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-6[55860]: 2024-04-24T09:20:07.341+0000 7f90fd800700 -1 osd.6 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:39.375781+0000 front 2024-04-24T09:19:39.376197+0000 (oldest deadline 2024-04-24T09:20:04.675612+0000) 2024-04-24T09:20:07.803 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:07 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:07.334+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:27.386588+0000 front 2024-04-24T09:19:27.386891+0000 (oldest deadline 2024-04-24T09:19:52.686559+0000) 2024-04-24T09:20:07.804 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:07 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:07.334+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.891162+0000 front 2024-04-24T09:19:38.891270+0000 (oldest deadline 2024-04-24T09:20:04.790877+0000) 2024-04-24T09:20:07.955 INFO:journalctl@ceph.osd.7.smithi138.stdout:Apr 24 09:20:07 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-7[49166]: 2024-04-24T09:20:07.568+0000 7fb49b200700 -1 osd.7 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:37.463919+0000 front 2024-04-24T09:19:37.464057+0000 (oldest deadline 2024-04-24T09:20:03.865663+0000) 2024-04-24T09:20:07.955 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:07 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:07.671+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:29.414806+0000 front 2024-04-24T09:19:29.414861+0000 (oldest deadline 2024-04-24T09:19:55.314800+0000) 2024-04-24T09:20:07.955 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:07 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:07.671+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.119965+0000 front 2024-04-24T09:19:38.120034+0000 (oldest deadline 2024-04-24T09:20:02.820181+0000) 2024-04-24T09:20:07.957 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:07 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:07.529+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:37.707135+0000 front 2024-04-24T09:19:37.707028+0000 (oldest deadline 2024-04-24T09:20:03.607224+0000) 2024-04-24T09:20:07.957 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:07 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:07.758+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:28.209530+0000 front 2024-04-24T09:19:28.209142+0000 (oldest deadline 2024-04-24T09:19:51.709380+0000) 2024-04-24T09:20:07.957 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:07 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:07.758+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.613982+0000 front 2024-04-24T09:19:38.614541+0000 (oldest deadline 2024-04-24T09:20:00.913987+0000) 2024-04-24T09:20:08.455 INFO:journalctl@ceph.osd.4.smithi138.stdout:Apr 24 09:20:08 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-4[42520]: 2024-04-24T09:20:08.015+0000 7fcab6e00700 -1 osd.4 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:35.315079+0000 front 2024-04-24T09:19:35.315310+0000 (oldest deadline 2024-04-24T09:20:00.614545+0000) 2024-04-24T09:20:08.457 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 24 09:20:08 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-5[43532]: 2024-04-24T09:20:08.195+0000 7f3188c00700 -1 osd.5 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:28.938580+0000 front 2024-04-24T09:19:28.938712+0000 (oldest deadline 2024-04-24T09:19:52.438846+0000) 2024-04-24T09:20:08.457 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 24 09:20:08 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-5[43532]: 2024-04-24T09:20:08.195+0000 7f3188c00700 -1 osd.5 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.340865+0000 front 2024-04-24T09:19:38.340985+0000 (oldest deadline 2024-04-24T09:20:00.640742+0000) 2024-04-24T09:20:08.553 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:08 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:08.381+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:27.386588+0000 front 2024-04-24T09:19:27.386891+0000 (oldest deadline 2024-04-24T09:19:52.686559+0000) 2024-04-24T09:20:08.553 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:08 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:08.381+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.891162+0000 front 2024-04-24T09:19:38.891270+0000 (oldest deadline 2024-04-24T09:20:04.790877+0000) 2024-04-24T09:20:08.553 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 24 09:20:08 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-6[55860]: 2024-04-24T09:20:08.301+0000 7f90fd800700 -1 osd.6 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:39.375781+0000 front 2024-04-24T09:19:39.376197+0000 (oldest deadline 2024-04-24T09:20:04.675612+0000) 2024-04-24T09:20:08.955 INFO:journalctl@ceph.osd.7.smithi138.stdout:Apr 24 09:20:08 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-7[49166]: 2024-04-24T09:20:08.595+0000 7fb49b200700 -1 osd.7 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:37.463919+0000 front 2024-04-24T09:19:37.464057+0000 (oldest deadline 2024-04-24T09:20:03.865663+0000) 2024-04-24T09:20:08.955 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:08 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:08.636+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:29.414806+0000 front 2024-04-24T09:19:29.414861+0000 (oldest deadline 2024-04-24T09:19:55.314800+0000) 2024-04-24T09:20:08.955 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:08 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:08.636+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.119965+0000 front 2024-04-24T09:19:38.120034+0000 (oldest deadline 2024-04-24T09:20:02.820181+0000) 2024-04-24T09:20:08.957 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:08 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:08.554+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:37.707135+0000 front 2024-04-24T09:19:37.707028+0000 (oldest deadline 2024-04-24T09:20:03.607224+0000) 2024-04-24T09:20:08.957 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:08 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:08.758+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:28.209530+0000 front 2024-04-24T09:19:28.209142+0000 (oldest deadline 2024-04-24T09:19:51.709380+0000) 2024-04-24T09:20:08.957 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:08 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:08.758+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.613982+0000 front 2024-04-24T09:19:38.614541+0000 (oldest deadline 2024-04-24T09:20:00.913987+0000) 2024-04-24T09:20:09.526 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 24 09:20:09 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-5[43532]: 2024-04-24T09:20:09.208+0000 7f3188c00700 -1 osd.5 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:28.938580+0000 front 2024-04-24T09:19:28.938712+0000 (oldest deadline 2024-04-24T09:19:52.438846+0000) 2024-04-24T09:20:09.526 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 24 09:20:09 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-5[43532]: 2024-04-24T09:20:09.208+0000 7f3188c00700 -1 osd.5 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.340865+0000 front 2024-04-24T09:19:38.340985+0000 (oldest deadline 2024-04-24T09:20:00.640742+0000) 2024-04-24T09:20:09.553 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:09 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:09.343+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:27.386588+0000 front 2024-04-24T09:19:27.386891+0000 (oldest deadline 2024-04-24T09:19:52.686559+0000) 2024-04-24T09:20:09.553 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:09 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:09.343+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.891162+0000 front 2024-04-24T09:19:38.891270+0000 (oldest deadline 2024-04-24T09:20:04.790877+0000) 2024-04-24T09:20:09.553 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 24 09:20:09 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-6[55860]: 2024-04-24T09:20:09.281+0000 7f90fd800700 -1 osd.6 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:39.375781+0000 front 2024-04-24T09:19:39.376197+0000 (oldest deadline 2024-04-24T09:20:04.675612+0000) 2024-04-24T09:20:09.955 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:09 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:09.645+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:29.414806+0000 front 2024-04-24T09:19:29.414861+0000 (oldest deadline 2024-04-24T09:19:55.314800+0000) 2024-04-24T09:20:09.955 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:09 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:09.645+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.119965+0000 front 2024-04-24T09:19:38.120034+0000 (oldest deadline 2024-04-24T09:20:02.820181+0000) 2024-04-24T09:20:09.955 INFO:journalctl@ceph.osd.4.smithi138.stdout:Apr 24 09:20:09 smithi138 podman[72711]: 2024-04-24 09:20:09.681423057 +0000 UTC m=+5.802462275 container died d786b86efb78a2f3d61013e4772e6265f45bf115dfa17b6bc399e0a7ed61fcb8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-4, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T09:20:09.956 INFO:journalctl@ceph.osd.7.smithi138.stdout:Apr 24 09:20:09 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-7[49166]: 2024-04-24T09:20:09.634+0000 7fb49b200700 -1 osd.7 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:37.463919+0000 front 2024-04-24T09:19:37.464057+0000 (oldest deadline 2024-04-24T09:20:03.865663+0000) 2024-04-24T09:20:09.957 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:09 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:09.526+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:37.707135+0000 front 2024-04-24T09:19:37.707028+0000 (oldest deadline 2024-04-24T09:20:03.607224+0000) 2024-04-24T09:20:09.957 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:09 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:09.751+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:28.209530+0000 front 2024-04-24T09:19:28.209142+0000 (oldest deadline 2024-04-24T09:19:51.709380+0000) 2024-04-24T09:20:09.957 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:09 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:09.751+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.613982+0000 front 2024-04-24T09:19:38.614541+0000 (oldest deadline 2024-04-24T09:20:00.913987+0000) 2024-04-24T09:20:10.448 INFO:journalctl@ceph.osd.4.smithi138.stdout:Apr 24 09:20:10 smithi138 podman[72711]: 2024-04-24 09:20:10.188232993 +0000 UTC m=+6.309272238 container cleanup d786b86efb78a2f3d61013e4772e6265f45bf115dfa17b6bc399e0a7ed61fcb8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-4, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=rishabh-main-apr17-a654945-1f81fb3, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-04-24T09:20:10.448 INFO:journalctl@ceph.osd.4.smithi138.stdout:Apr 24 09:20:10 smithi138 bash[72711]: ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-4 2024-04-24T09:20:10.493 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 24 09:20:10 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-5[43532]: 2024-04-24T09:20:10.240+0000 7f3188c00700 -1 osd.5 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:28.938580+0000 front 2024-04-24T09:19:28.938712+0000 (oldest deadline 2024-04-24T09:19:52.438846+0000) 2024-04-24T09:20:10.493 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 24 09:20:10 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-5[43532]: 2024-04-24T09:20:10.240+0000 7f3188c00700 -1 osd.5 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.340865+0000 front 2024-04-24T09:19:38.340985+0000 (oldest deadline 2024-04-24T09:20:00.640742+0000) 2024-04-24T09:20:10.553 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 24 09:20:10 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-6[55860]: 2024-04-24T09:20:10.279+0000 7f90fd800700 -1 osd.6 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:39.375781+0000 front 2024-04-24T09:19:39.376197+0000 (oldest deadline 2024-04-24T09:20:04.675612+0000) 2024-04-24T09:20:10.553 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:10 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:10.305+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:27.386588+0000 front 2024-04-24T09:19:27.386891+0000 (oldest deadline 2024-04-24T09:19:52.686559+0000) 2024-04-24T09:20:10.553 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:10 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:10.305+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.891162+0000 front 2024-04-24T09:19:38.891270+0000 (oldest deadline 2024-04-24T09:20:04.790877+0000) 2024-04-24T09:20:10.704 INFO:journalctl@ceph.osd.4.smithi138.stdout:Apr 24 09:20:10 smithi138 podman[72729]: 2024-04-24 09:20:10.448006962 +0000 UTC m=+0.752128751 container remove d786b86efb78a2f3d61013e4772e6265f45bf115dfa17b6bc399e0a7ed61fcb8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-4, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=rishabh-main-apr17-a654945-1f81fb3, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-04-24T09:20:10.705 INFO:journalctl@ceph.osd.7.smithi138.stdout:Apr 24 09:20:10 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-7[49166]: 2024-04-24T09:20:10.597+0000 7fb49b200700 -1 osd.7 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:37.463919+0000 front 2024-04-24T09:19:37.464057+0000 (oldest deadline 2024-04-24T09:20:03.865663+0000) 2024-04-24T09:20:10.705 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:10 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:10.651+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:29.414806+0000 front 2024-04-24T09:19:29.414861+0000 (oldest deadline 2024-04-24T09:19:55.314800+0000) 2024-04-24T09:20:10.705 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:10 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:10.651+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.119965+0000 front 2024-04-24T09:19:38.120034+0000 (oldest deadline 2024-04-24T09:20:02.820181+0000) 2024-04-24T09:20:10.767 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:10 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:10.493+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:37.707135+0000 front 2024-04-24T09:19:37.707028+0000 (oldest deadline 2024-04-24T09:20:03.607224+0000) 2024-04-24T09:20:11.193 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:10 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:10.768+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:28.209530+0000 front 2024-04-24T09:19:28.209142+0000 (oldest deadline 2024-04-24T09:19:51.709380+0000) 2024-04-24T09:20:11.194 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:10 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:10.768+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.613982+0000 front 2024-04-24T09:19:38.614541+0000 (oldest deadline 2024-04-24T09:20:00.913987+0000) 2024-04-24T09:20:11.445 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 24 09:20:11 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-5[43532]: 2024-04-24T09:20:11.193+0000 7f3188c00700 -1 osd.5 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:28.938580+0000 front 2024-04-24T09:19:28.938712+0000 (oldest deadline 2024-04-24T09:19:52.438846+0000) 2024-04-24T09:20:11.446 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 24 09:20:11 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-5[43532]: 2024-04-24T09:20:11.193+0000 7f3188c00700 -1 osd.5 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.340865+0000 front 2024-04-24T09:19:38.340985+0000 (oldest deadline 2024-04-24T09:20:00.640742+0000) 2024-04-24T09:20:11.553 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 24 09:20:11 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-6[55860]: 2024-04-24T09:20:11.327+0000 7f90fd800700 -1 osd.6 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:39.375781+0000 front 2024-04-24T09:19:39.376197+0000 (oldest deadline 2024-04-24T09:20:04.675612+0000) 2024-04-24T09:20:11.553 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:11 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:11.286+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:27.386588+0000 front 2024-04-24T09:19:27.386891+0000 (oldest deadline 2024-04-24T09:19:52.686559+0000) 2024-04-24T09:20:11.554 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:11 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:11.286+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.891162+0000 front 2024-04-24T09:19:38.891270+0000 (oldest deadline 2024-04-24T09:20:04.790877+0000) 2024-04-24T09:20:11.571 INFO:journalctl@ceph.osd.4.smithi138.stdout:Apr 24 09:20:11 smithi138 podman[72819]: 2024-04-24 09:20:11.138680476 +0000 UTC m=+0.036396160 image pull b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644 2024-04-24T09:20:11.571 INFO:journalctl@ceph.osd.4.smithi138.stdout:Apr 24 09:20:11 smithi138 podman[72819]: 2024-04-24 09:20:11.257690198 +0000 UTC m=+0.155405907 container create 359c4325debc121de75095415cd729db958724f9e7febd0e039b457523a08853 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=rishabh-main-apr17-a654945-1f81fb3, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS) 2024-04-24T09:20:11.571 INFO:journalctl@ceph.osd.4.smithi138.stdout:Apr 24 09:20:11 smithi138 podman[72819]: 2024-04-24 09:20:11.570503877 +0000 UTC m=+0.468219602 container init 359c4325debc121de75095415cd729db958724f9e7febd0e039b457523a08853 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-4-deactivate, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-24T09:20:11.707 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:11 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:11.446+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:37.707135+0000 front 2024-04-24T09:19:37.707028+0000 (oldest deadline 2024-04-24T09:20:03.607224+0000) 2024-04-24T09:20:11.955 INFO:journalctl@ceph.osd.7.smithi138.stdout:Apr 24 09:20:11 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-7[49166]: 2024-04-24T09:20:11.630+0000 7fb49b200700 -1 osd.7 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:37.463919+0000 front 2024-04-24T09:19:37.464057+0000 (oldest deadline 2024-04-24T09:20:03.865663+0000) 2024-04-24T09:20:11.955 INFO:journalctl@ceph.osd.4.smithi138.stdout:Apr 24 09:20:11 smithi138 podman[72819]: 2024-04-24 09:20:11.582867994 +0000 UTC m=+0.480583715 container start 359c4325debc121de75095415cd729db958724f9e7febd0e039b457523a08853 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240424, RELEASE=rishabh-main-apr17-a654945-1f81fb3, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-24T09:20:11.955 INFO:journalctl@ceph.osd.4.smithi138.stdout:Apr 24 09:20:11 smithi138 podman[72819]: 2024-04-24 09:20:11.677830171 +0000 UTC m=+0.575545919 container attach 359c4325debc121de75095415cd729db958724f9e7febd0e039b457523a08853 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T09:20:11.955 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:11 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:11.604+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:29.414806+0000 front 2024-04-24T09:19:29.414861+0000 (oldest deadline 2024-04-24T09:19:55.314800+0000) 2024-04-24T09:20:11.956 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:11 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:11.604+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.119965+0000 front 2024-04-24T09:19:38.120034+0000 (oldest deadline 2024-04-24T09:20:02.820181+0000) 2024-04-24T09:20:11.956 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:11 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:11.604+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:47.326279+0000 front 2024-04-24T09:19:47.326338+0000 (oldest deadline 2024-04-24T09:20:10.728055+0000) 2024-04-24T09:20:12.207 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:11 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:11.733+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:28.209530+0000 front 2024-04-24T09:19:28.209142+0000 (oldest deadline 2024-04-24T09:19:51.709380+0000) 2024-04-24T09:20:12.207 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:11 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:11.733+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.613982+0000 front 2024-04-24T09:19:38.614541+0000 (oldest deadline 2024-04-24T09:20:00.913987+0000) 2024-04-24T09:20:12.393 INFO:journalctl@ceph.osd.4.smithi138.stdout:Apr 24 09:20:12 smithi138 podman[72819]: 2024-04-24 09:20:12.033324045 +0000 UTC m=+0.931039730 container died 359c4325debc121de75095415cd729db958724f9e7febd0e039b457523a08853 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-4-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-04-24T09:20:12.553 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:12 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:12.244+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:27.386588+0000 front 2024-04-24T09:19:27.386891+0000 (oldest deadline 2024-04-24T09:19:52.686559+0000) 2024-04-24T09:20:12.553 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:12 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:12.244+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.891162+0000 front 2024-04-24T09:19:38.891270+0000 (oldest deadline 2024-04-24T09:20:04.790877+0000) 2024-04-24T09:20:12.553 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 24 09:20:12 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-6[55860]: 2024-04-24T09:20:12.342+0000 7f90fd800700 -1 osd.6 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:39.375781+0000 front 2024-04-24T09:19:39.376197+0000 (oldest deadline 2024-04-24T09:20:04.675612+0000) 2024-04-24T09:20:12.553 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 24 09:20:12 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-6[55860]: 2024-04-24T09:20:12.342+0000 7f90fd800700 -1 osd.6 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:46.978139+0000 front 2024-04-24T09:19:46.978058+0000 (oldest deadline 2024-04-24T09:20:12.278124+0000) 2024-04-24T09:20:12.705 INFO:journalctl@ceph.osd.7.smithi138.stdout:Apr 24 09:20:12 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-7[49166]: 2024-04-24T09:20:12.598+0000 7fb49b200700 -1 osd.7 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:37.463919+0000 front 2024-04-24T09:19:37.464057+0000 (oldest deadline 2024-04-24T09:20:03.865663+0000) 2024-04-24T09:20:12.705 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:12 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:12.630+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:29.414806+0000 front 2024-04-24T09:19:29.414861+0000 (oldest deadline 2024-04-24T09:19:55.314800+0000) 2024-04-24T09:20:12.705 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:12 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:12.630+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.119965+0000 front 2024-04-24T09:19:38.120034+0000 (oldest deadline 2024-04-24T09:20:02.820181+0000) 2024-04-24T09:20:12.705 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:12 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:12.630+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:47.326279+0000 front 2024-04-24T09:19:47.326338+0000 (oldest deadline 2024-04-24T09:20:10.728055+0000) 2024-04-24T09:20:12.707 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:12 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:12.397+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:37.707135+0000 front 2024-04-24T09:19:37.707028+0000 (oldest deadline 2024-04-24T09:20:03.607224+0000) 2024-04-24T09:20:12.707 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 24 09:20:12 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-5[43532]: 2024-04-24T09:20:12.208+0000 7f3188c00700 -1 osd.5 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:28.938580+0000 front 2024-04-24T09:19:28.938712+0000 (oldest deadline 2024-04-24T09:19:52.438846+0000) 2024-04-24T09:20:12.707 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 24 09:20:12 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-5[43532]: 2024-04-24T09:20:12.208+0000 7f3188c00700 -1 osd.5 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.340865+0000 front 2024-04-24T09:19:38.340985+0000 (oldest deadline 2024-04-24T09:20:00.640742+0000) 2024-04-24T09:20:12.890 DEBUG:teuthology.orchestra.run.smithi138:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bd3e039e-0215-11ef-bc93-c7b262605968@osd.4.service' 2024-04-24T09:20:12.988 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T09:20:12.988 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-04-24T09:20:12.989 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-04-24T09:20:12.989 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ceph-bd3e039e-0215-11ef-bc93-c7b262605968@osd.5 2024-04-24T09:20:13.064 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:12 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:12.759+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:28.209530+0000 front 2024-04-24T09:19:28.209142+0000 (oldest deadline 2024-04-24T09:19:51.709380+0000) 2024-04-24T09:20:13.064 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:12 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:12.759+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.613982+0000 front 2024-04-24T09:19:38.614541+0000 (oldest deadline 2024-04-24T09:20:00.913987+0000) 2024-04-24T09:20:13.403 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 24 09:20:13 smithi189 systemd[1]: Stopping Ceph osd.5 for bd3e039e-0215-11ef-bc93-c7b262605968... 2024-04-24T09:20:13.403 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 24 09:20:13 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-5[43532]: 2024-04-24T09:20:13.204+0000 7f3188c00700 -1 osd.5 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:28.938580+0000 front 2024-04-24T09:19:28.938712+0000 (oldest deadline 2024-04-24T09:19:52.438846+0000) 2024-04-24T09:20:13.403 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 24 09:20:13 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-5[43532]: 2024-04-24T09:20:13.204+0000 7f3188c00700 -1 osd.5 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.340865+0000 front 2024-04-24T09:19:38.340985+0000 (oldest deadline 2024-04-24T09:20:00.640742+0000) 2024-04-24T09:20:13.403 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 24 09:20:13 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-5[43532]: 2024-04-24T09:20:13.204+0000 7f3188c00700 -1 osd.5 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:47.043658+0000 front 2024-04-24T09:19:47.043834+0000 (oldest deadline 2024-04-24T09:20:12.844365+0000) 2024-04-24T09:20:13.553 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 24 09:20:13 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-6[55860]: 2024-04-24T09:20:13.312+0000 7f90fd800700 -1 osd.6 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:39.375781+0000 front 2024-04-24T09:19:39.376197+0000 (oldest deadline 2024-04-24T09:20:04.675612+0000) 2024-04-24T09:20:13.553 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 24 09:20:13 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-6[55860]: 2024-04-24T09:20:13.312+0000 7f90fd800700 -1 osd.6 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:46.978139+0000 front 2024-04-24T09:19:46.978058+0000 (oldest deadline 2024-04-24T09:20:12.278124+0000) 2024-04-24T09:20:13.553 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:13 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:13.250+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:27.386588+0000 front 2024-04-24T09:19:27.386891+0000 (oldest deadline 2024-04-24T09:19:52.686559+0000) 2024-04-24T09:20:13.554 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:13 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:13.250+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.891162+0000 front 2024-04-24T09:19:38.891270+0000 (oldest deadline 2024-04-24T09:20:04.790877+0000) 2024-04-24T09:20:13.707 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:13 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:13.402+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:37.707135+0000 front 2024-04-24T09:19:37.707028+0000 (oldest deadline 2024-04-24T09:20:03.607224+0000) 2024-04-24T09:20:13.707 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 24 09:20:13 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-5[43532]: 2024-04-24T09:20:13.571+0000 7f3192400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T09:20:13.707 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 24 09:20:13 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-5[43532]: 2024-04-24T09:20:13.571+0000 7f3192400700 -1 osd.5 141 *** Got signal Terminated *** 2024-04-24T09:20:13.707 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 24 09:20:13 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-5[43532]: 2024-04-24T09:20:13.571+0000 7f3192400700 -1 osd.5 141 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T09:20:13.955 INFO:journalctl@ceph.osd.7.smithi138.stdout:Apr 24 09:20:13 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-7[49166]: 2024-04-24T09:20:13.618+0000 7fb49b200700 -1 osd.7 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:37.463919+0000 front 2024-04-24T09:19:37.464057+0000 (oldest deadline 2024-04-24T09:20:03.865663+0000) 2024-04-24T09:20:13.955 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:13 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:13.594+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:29.414806+0000 front 2024-04-24T09:19:29.414861+0000 (oldest deadline 2024-04-24T09:19:55.314800+0000) 2024-04-24T09:20:13.955 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:13 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:13.594+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.119965+0000 front 2024-04-24T09:19:38.120034+0000 (oldest deadline 2024-04-24T09:20:02.820181+0000) 2024-04-24T09:20:13.955 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:13 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:13.594+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:47.326279+0000 front 2024-04-24T09:19:47.326338+0000 (oldest deadline 2024-04-24T09:20:10.728055+0000) 2024-04-24T09:20:14.207 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:13 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:13.762+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:28.209530+0000 front 2024-04-24T09:19:28.209142+0000 (oldest deadline 2024-04-24T09:19:51.709380+0000) 2024-04-24T09:20:14.207 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:13 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:13.762+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.613982+0000 front 2024-04-24T09:19:38.614541+0000 (oldest deadline 2024-04-24T09:20:00.913987+0000) 2024-04-24T09:20:14.553 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 24 09:20:14 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-6[55860]: 2024-04-24T09:20:14.310+0000 7f90fd800700 -1 osd.6 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:39.375781+0000 front 2024-04-24T09:19:39.376197+0000 (oldest deadline 2024-04-24T09:20:04.675612+0000) 2024-04-24T09:20:14.553 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 24 09:20:14 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-6[55860]: 2024-04-24T09:20:14.310+0000 7f90fd800700 -1 osd.6 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:46.978139+0000 front 2024-04-24T09:19:46.978058+0000 (oldest deadline 2024-04-24T09:20:12.278124+0000) 2024-04-24T09:20:14.553 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:14 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:14.262+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:27.386588+0000 front 2024-04-24T09:19:27.386891+0000 (oldest deadline 2024-04-24T09:19:52.686559+0000) 2024-04-24T09:20:14.553 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:14 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:14.262+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.891162+0000 front 2024-04-24T09:19:38.891270+0000 (oldest deadline 2024-04-24T09:20:04.790877+0000) 2024-04-24T09:20:14.553 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:14 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:14.262+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:48.794838+0000 front 2024-04-24T09:19:48.794972+0000 (oldest deadline 2024-04-24T09:20:14.095077+0000) 2024-04-24T09:20:14.707 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:14 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:14.437+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:37.707135+0000 front 2024-04-24T09:19:37.707028+0000 (oldest deadline 2024-04-24T09:20:03.607224+0000) 2024-04-24T09:20:14.707 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:14 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:14.437+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:48.713889+0000 front 2024-04-24T09:19:48.713979+0000 (oldest deadline 2024-04-24T09:20:13.414134+0000) 2024-04-24T09:20:14.707 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 24 09:20:14 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-5[43532]: 2024-04-24T09:20:14.224+0000 7f3188c00700 -1 osd.5 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:28.938580+0000 front 2024-04-24T09:19:28.938712+0000 (oldest deadline 2024-04-24T09:19:52.438846+0000) 2024-04-24T09:20:14.707 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 24 09:20:14 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-5[43532]: 2024-04-24T09:20:14.224+0000 7f3188c00700 -1 osd.5 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.340865+0000 front 2024-04-24T09:19:38.340985+0000 (oldest deadline 2024-04-24T09:20:00.640742+0000) 2024-04-24T09:20:14.708 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 24 09:20:14 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-5[43532]: 2024-04-24T09:20:14.224+0000 7f3188c00700 -1 osd.5 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:47.043658+0000 front 2024-04-24T09:19:47.043834+0000 (oldest deadline 2024-04-24T09:20:12.844365+0000) 2024-04-24T09:20:14.955 INFO:journalctl@ceph.osd.7.smithi138.stdout:Apr 24 09:20:14 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-7[49166]: 2024-04-24T09:20:14.646+0000 7fb49b200700 -1 osd.7 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:37.463919+0000 front 2024-04-24T09:19:37.464057+0000 (oldest deadline 2024-04-24T09:20:03.865663+0000) 2024-04-24T09:20:14.955 INFO:journalctl@ceph.osd.7.smithi138.stdout:Apr 24 09:20:14 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-7[49166]: 2024-04-24T09:20:14.646+0000 7fb49b200700 -1 osd.7 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:47.968238+0000 front 2024-04-24T09:19:47.968624+0000 (oldest deadline 2024-04-24T09:20:13.868496+0000) 2024-04-24T09:20:14.955 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:14 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:14.574+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:29.414806+0000 front 2024-04-24T09:19:29.414861+0000 (oldest deadline 2024-04-24T09:19:55.314800+0000) 2024-04-24T09:20:14.955 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:14 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:14.574+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.119965+0000 front 2024-04-24T09:19:38.120034+0000 (oldest deadline 2024-04-24T09:20:02.820181+0000) 2024-04-24T09:20:14.955 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:14 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:14.574+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:47.326279+0000 front 2024-04-24T09:19:47.326338+0000 (oldest deadline 2024-04-24T09:20:10.728055+0000) 2024-04-24T09:20:15.207 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:14 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:14.789+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:28.209530+0000 front 2024-04-24T09:19:28.209142+0000 (oldest deadline 2024-04-24T09:19:51.709380+0000) 2024-04-24T09:20:15.207 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:14 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:14.789+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.613982+0000 front 2024-04-24T09:19:38.614541+0000 (oldest deadline 2024-04-24T09:20:00.913987+0000) 2024-04-24T09:20:15.553 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 24 09:20:15 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-6[55860]: 2024-04-24T09:20:15.320+0000 7f90fd800700 -1 osd.6 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:39.375781+0000 front 2024-04-24T09:19:39.376197+0000 (oldest deadline 2024-04-24T09:20:04.675612+0000) 2024-04-24T09:20:15.553 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 24 09:20:15 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-6[55860]: 2024-04-24T09:20:15.320+0000 7f90fd800700 -1 osd.6 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:46.978139+0000 front 2024-04-24T09:19:46.978058+0000 (oldest deadline 2024-04-24T09:20:12.278124+0000) 2024-04-24T09:20:15.553 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:15 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:15.274+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:27.386588+0000 front 2024-04-24T09:19:27.386891+0000 (oldest deadline 2024-04-24T09:19:52.686559+0000) 2024-04-24T09:20:15.554 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:15 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:15.274+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.891162+0000 front 2024-04-24T09:19:38.891270+0000 (oldest deadline 2024-04-24T09:20:04.790877+0000) 2024-04-24T09:20:15.554 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:15 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:15.274+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:48.794838+0000 front 2024-04-24T09:19:48.794972+0000 (oldest deadline 2024-04-24T09:20:14.095077+0000) 2024-04-24T09:20:15.584 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:15 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:15.396+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:37.707135+0000 front 2024-04-24T09:19:37.707028+0000 (oldest deadline 2024-04-24T09:20:03.607224+0000) 2024-04-24T09:20:15.584 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:15 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:15.396+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:48.713889+0000 front 2024-04-24T09:19:48.713979+0000 (oldest deadline 2024-04-24T09:20:13.414134+0000) 2024-04-24T09:20:15.584 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 24 09:20:15 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-5[43532]: 2024-04-24T09:20:15.214+0000 7f3188c00700 -1 osd.5 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:28.938580+0000 front 2024-04-24T09:19:28.938712+0000 (oldest deadline 2024-04-24T09:19:52.438846+0000) 2024-04-24T09:20:15.584 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 24 09:20:15 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-5[43532]: 2024-04-24T09:20:15.214+0000 7f3188c00700 -1 osd.5 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.340865+0000 front 2024-04-24T09:19:38.340985+0000 (oldest deadline 2024-04-24T09:20:00.640742+0000) 2024-04-24T09:20:15.584 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 24 09:20:15 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-5[43532]: 2024-04-24T09:20:15.214+0000 7f3188c00700 -1 osd.5 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:47.043658+0000 front 2024-04-24T09:19:47.043834+0000 (oldest deadline 2024-04-24T09:20:12.844365+0000) 2024-04-24T09:20:15.908 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:15 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:15.825+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:28.209530+0000 front 2024-04-24T09:19:28.209142+0000 (oldest deadline 2024-04-24T09:19:51.709380+0000) 2024-04-24T09:20:15.909 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:15 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:15.825+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.613982+0000 front 2024-04-24T09:19:38.614541+0000 (oldest deadline 2024-04-24T09:20:00.913987+0000) 2024-04-24T09:20:15.955 INFO:journalctl@ceph.osd.7.smithi138.stdout:Apr 24 09:20:15 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-7[49166]: 2024-04-24T09:20:15.612+0000 7fb49b200700 -1 osd.7 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:37.463919+0000 front 2024-04-24T09:19:37.464057+0000 (oldest deadline 2024-04-24T09:20:03.865663+0000) 2024-04-24T09:20:15.955 INFO:journalctl@ceph.osd.7.smithi138.stdout:Apr 24 09:20:15 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-7[49166]: 2024-04-24T09:20:15.612+0000 7fb49b200700 -1 osd.7 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:47.968238+0000 front 2024-04-24T09:19:47.968624+0000 (oldest deadline 2024-04-24T09:20:13.868496+0000) 2024-04-24T09:20:15.955 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:15 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:15.526+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:29.414806+0000 front 2024-04-24T09:19:29.414861+0000 (oldest deadline 2024-04-24T09:19:55.314800+0000) 2024-04-24T09:20:15.955 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:15 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:15.526+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.119965+0000 front 2024-04-24T09:19:38.120034+0000 (oldest deadline 2024-04-24T09:20:02.820181+0000) 2024-04-24T09:20:15.955 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:15 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:15.526+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:47.326279+0000 front 2024-04-24T09:19:47.326338+0000 (oldest deadline 2024-04-24T09:20:10.728055+0000) 2024-04-24T09:20:16.457 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:16 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:16.362+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:37.707135+0000 front 2024-04-24T09:19:37.707028+0000 (oldest deadline 2024-04-24T09:20:03.607224+0000) 2024-04-24T09:20:16.457 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:16 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:16.362+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:48.713889+0000 front 2024-04-24T09:19:48.713979+0000 (oldest deadline 2024-04-24T09:20:13.414134+0000) 2024-04-24T09:20:16.457 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 24 09:20:16 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-5[43532]: 2024-04-24T09:20:16.203+0000 7f3188c00700 -1 osd.5 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:28.938580+0000 front 2024-04-24T09:19:28.938712+0000 (oldest deadline 2024-04-24T09:19:52.438846+0000) 2024-04-24T09:20:16.457 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 24 09:20:16 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-5[43532]: 2024-04-24T09:20:16.204+0000 7f3188c00700 -1 osd.5 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.340865+0000 front 2024-04-24T09:19:38.340985+0000 (oldest deadline 2024-04-24T09:20:00.640742+0000) 2024-04-24T09:20:16.458 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 24 09:20:16 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-5[43532]: 2024-04-24T09:20:16.204+0000 7f3188c00700 -1 osd.5 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:47.043658+0000 front 2024-04-24T09:19:47.043834+0000 (oldest deadline 2024-04-24T09:20:12.844365+0000) 2024-04-24T09:20:16.553 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 24 09:20:16 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-6[55860]: 2024-04-24T09:20:16.295+0000 7f90fd800700 -1 osd.6 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:39.375781+0000 front 2024-04-24T09:19:39.376197+0000 (oldest deadline 2024-04-24T09:20:04.675612+0000) 2024-04-24T09:20:16.553 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 24 09:20:16 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-6[55860]: 2024-04-24T09:20:16.295+0000 7f90fd800700 -1 osd.6 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:46.978139+0000 front 2024-04-24T09:19:46.978058+0000 (oldest deadline 2024-04-24T09:20:12.278124+0000) 2024-04-24T09:20:16.553 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:16 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:16.225+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:27.386588+0000 front 2024-04-24T09:19:27.386891+0000 (oldest deadline 2024-04-24T09:19:52.686559+0000) 2024-04-24T09:20:16.553 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:16 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:16.225+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.891162+0000 front 2024-04-24T09:19:38.891270+0000 (oldest deadline 2024-04-24T09:20:04.790877+0000) 2024-04-24T09:20:16.553 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:16 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:16.225+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:48.794838+0000 front 2024-04-24T09:19:48.794972+0000 (oldest deadline 2024-04-24T09:20:14.095077+0000) 2024-04-24T09:20:16.955 INFO:journalctl@ceph.osd.7.smithi138.stdout:Apr 24 09:20:16 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-7[49166]: 2024-04-24T09:20:16.619+0000 7fb49b200700 -1 osd.7 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:37.463919+0000 front 2024-04-24T09:19:37.464057+0000 (oldest deadline 2024-04-24T09:20:03.865663+0000) 2024-04-24T09:20:16.955 INFO:journalctl@ceph.osd.7.smithi138.stdout:Apr 24 09:20:16 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-7[49166]: 2024-04-24T09:20:16.619+0000 7fb49b200700 -1 osd.7 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:47.968238+0000 front 2024-04-24T09:19:47.968624+0000 (oldest deadline 2024-04-24T09:20:13.868496+0000) 2024-04-24T09:20:16.955 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:16 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:16.510+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:29.414806+0000 front 2024-04-24T09:19:29.414861+0000 (oldest deadline 2024-04-24T09:19:55.314800+0000) 2024-04-24T09:20:16.955 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:16 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:16.510+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.119965+0000 front 2024-04-24T09:19:38.120034+0000 (oldest deadline 2024-04-24T09:20:02.820181+0000) 2024-04-24T09:20:16.955 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:16 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:16.510+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:47.326279+0000 front 2024-04-24T09:19:47.326338+0000 (oldest deadline 2024-04-24T09:20:10.728055+0000) 2024-04-24T09:20:17.207 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:16 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:16.795+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:28.209530+0000 front 2024-04-24T09:19:28.209142+0000 (oldest deadline 2024-04-24T09:19:51.709380+0000) 2024-04-24T09:20:17.207 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:16 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:16.795+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.613982+0000 front 2024-04-24T09:19:38.614541+0000 (oldest deadline 2024-04-24T09:20:00.913987+0000) 2024-04-24T09:20:17.207 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:16 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:16.795+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:49.018457+0000 front 2024-04-24T09:19:49.018520+0000 (oldest deadline 2024-04-24T09:20:16.019822+0000) 2024-04-24T09:20:17.553 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 24 09:20:17 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-6[55860]: 2024-04-24T09:20:17.309+0000 7f90fd800700 -1 osd.6 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:39.375781+0000 front 2024-04-24T09:19:39.376197+0000 (oldest deadline 2024-04-24T09:20:04.675612+0000) 2024-04-24T09:20:17.553 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 24 09:20:17 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-6[55860]: 2024-04-24T09:20:17.309+0000 7f90fd800700 -1 osd.6 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:46.978139+0000 front 2024-04-24T09:19:46.978058+0000 (oldest deadline 2024-04-24T09:20:12.278124+0000) 2024-04-24T09:20:17.553 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:17 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:17.249+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:27.386588+0000 front 2024-04-24T09:19:27.386891+0000 (oldest deadline 2024-04-24T09:19:52.686559+0000) 2024-04-24T09:20:17.554 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:17 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:17.249+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.891162+0000 front 2024-04-24T09:19:38.891270+0000 (oldest deadline 2024-04-24T09:20:04.790877+0000) 2024-04-24T09:20:17.554 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:17 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:17.249+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:48.794838+0000 front 2024-04-24T09:19:48.794972+0000 (oldest deadline 2024-04-24T09:20:14.095077+0000) 2024-04-24T09:20:17.707 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:17 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:17.315+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:37.707135+0000 front 2024-04-24T09:19:37.707028+0000 (oldest deadline 2024-04-24T09:20:03.607224+0000) 2024-04-24T09:20:17.707 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:17 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:17.315+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:48.713889+0000 front 2024-04-24T09:19:48.713979+0000 (oldest deadline 2024-04-24T09:20:13.414134+0000) 2024-04-24T09:20:17.707 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 24 09:20:17 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-5[43532]: 2024-04-24T09:20:17.223+0000 7f3188c00700 -1 osd.5 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:28.938580+0000 front 2024-04-24T09:19:28.938712+0000 (oldest deadline 2024-04-24T09:19:52.438846+0000) 2024-04-24T09:20:17.708 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 24 09:20:17 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-5[43532]: 2024-04-24T09:20:17.223+0000 7f3188c00700 -1 osd.5 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.340865+0000 front 2024-04-24T09:19:38.340985+0000 (oldest deadline 2024-04-24T09:20:00.640742+0000) 2024-04-24T09:20:17.708 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 24 09:20:17 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-5[43532]: 2024-04-24T09:20:17.223+0000 7f3188c00700 -1 osd.5 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:47.043658+0000 front 2024-04-24T09:19:47.043834+0000 (oldest deadline 2024-04-24T09:20:12.844365+0000) 2024-04-24T09:20:17.955 INFO:journalctl@ceph.osd.7.smithi138.stdout:Apr 24 09:20:17 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-7[49166]: 2024-04-24T09:20:17.657+0000 7fb49b200700 -1 osd.7 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:37.463919+0000 front 2024-04-24T09:19:37.464057+0000 (oldest deadline 2024-04-24T09:20:03.865663+0000) 2024-04-24T09:20:17.955 INFO:journalctl@ceph.osd.7.smithi138.stdout:Apr 24 09:20:17 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-7[49166]: 2024-04-24T09:20:17.657+0000 7fb49b200700 -1 osd.7 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:47.968238+0000 front 2024-04-24T09:19:47.968624+0000 (oldest deadline 2024-04-24T09:20:13.868496+0000) 2024-04-24T09:20:17.955 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:17 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:17.484+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:29.414806+0000 front 2024-04-24T09:19:29.414861+0000 (oldest deadline 2024-04-24T09:19:55.314800+0000) 2024-04-24T09:20:17.955 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:17 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:17.484+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.119965+0000 front 2024-04-24T09:19:38.120034+0000 (oldest deadline 2024-04-24T09:20:02.820181+0000) 2024-04-24T09:20:17.955 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:17 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:17.484+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:47.326279+0000 front 2024-04-24T09:19:47.326338+0000 (oldest deadline 2024-04-24T09:20:10.728055+0000) 2024-04-24T09:20:18.207 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:17 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:17.801+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:28.209530+0000 front 2024-04-24T09:19:28.209142+0000 (oldest deadline 2024-04-24T09:19:51.709380+0000) 2024-04-24T09:20:18.207 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:17 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:17.801+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.613982+0000 front 2024-04-24T09:19:38.614541+0000 (oldest deadline 2024-04-24T09:20:00.913987+0000) 2024-04-24T09:20:18.207 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:17 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:17.801+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:49.018457+0000 front 2024-04-24T09:19:49.018520+0000 (oldest deadline 2024-04-24T09:20:16.019822+0000) 2024-04-24T09:20:18.553 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 24 09:20:18 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-6[55860]: 2024-04-24T09:20:18.268+0000 7f90fd800700 -1 osd.6 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:39.375781+0000 front 2024-04-24T09:19:39.376197+0000 (oldest deadline 2024-04-24T09:20:04.675612+0000) 2024-04-24T09:20:18.553 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 24 09:20:18 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-6[55860]: 2024-04-24T09:20:18.268+0000 7f90fd800700 -1 osd.6 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:46.978139+0000 front 2024-04-24T09:19:46.978058+0000 (oldest deadline 2024-04-24T09:20:12.278124+0000) 2024-04-24T09:20:18.553 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:18 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:18.296+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:27.386588+0000 front 2024-04-24T09:19:27.386891+0000 (oldest deadline 2024-04-24T09:19:52.686559+0000) 2024-04-24T09:20:18.554 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:18 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:18.296+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.891162+0000 front 2024-04-24T09:19:38.891270+0000 (oldest deadline 2024-04-24T09:20:04.790877+0000) 2024-04-24T09:20:18.554 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:18 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:18.296+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:48.794838+0000 front 2024-04-24T09:19:48.794972+0000 (oldest deadline 2024-04-24T09:20:14.095077+0000) 2024-04-24T09:20:18.707 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:18 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:18.339+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:37.707135+0000 front 2024-04-24T09:19:37.707028+0000 (oldest deadline 2024-04-24T09:20:03.607224+0000) 2024-04-24T09:20:18.707 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:18 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:18.339+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:48.713889+0000 front 2024-04-24T09:19:48.713979+0000 (oldest deadline 2024-04-24T09:20:13.414134+0000) 2024-04-24T09:20:18.707 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 24 09:20:18 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-5[43532]: 2024-04-24T09:20:18.239+0000 7f3188c00700 -1 osd.5 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:28.938580+0000 front 2024-04-24T09:19:28.938712+0000 (oldest deadline 2024-04-24T09:19:52.438846+0000) 2024-04-24T09:20:18.707 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 24 09:20:18 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-5[43532]: 2024-04-24T09:20:18.239+0000 7f3188c00700 -1 osd.5 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.340865+0000 front 2024-04-24T09:19:38.340985+0000 (oldest deadline 2024-04-24T09:20:00.640742+0000) 2024-04-24T09:20:18.707 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 24 09:20:18 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-5[43532]: 2024-04-24T09:20:18.239+0000 7f3188c00700 -1 osd.5 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:47.043658+0000 front 2024-04-24T09:19:47.043834+0000 (oldest deadline 2024-04-24T09:20:12.844365+0000) 2024-04-24T09:20:18.955 INFO:journalctl@ceph.osd.7.smithi138.stdout:Apr 24 09:20:18 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-7[49166]: 2024-04-24T09:20:18.661+0000 7fb49b200700 -1 osd.7 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:37.463919+0000 front 2024-04-24T09:19:37.464057+0000 (oldest deadline 2024-04-24T09:20:03.865663+0000) 2024-04-24T09:20:18.955 INFO:journalctl@ceph.osd.7.smithi138.stdout:Apr 24 09:20:18 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-7[49166]: 2024-04-24T09:20:18.661+0000 7fb49b200700 -1 osd.7 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:47.968238+0000 front 2024-04-24T09:19:47.968624+0000 (oldest deadline 2024-04-24T09:20:13.868496+0000) 2024-04-24T09:20:18.955 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:18 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:18.532+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:29.414806+0000 front 2024-04-24T09:19:29.414861+0000 (oldest deadline 2024-04-24T09:19:55.314800+0000) 2024-04-24T09:20:18.955 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:18 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:18.532+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.119965+0000 front 2024-04-24T09:19:38.120034+0000 (oldest deadline 2024-04-24T09:20:02.820181+0000) 2024-04-24T09:20:18.955 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:18 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:18.532+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:47.326279+0000 front 2024-04-24T09:19:47.326338+0000 (oldest deadline 2024-04-24T09:20:10.728055+0000) 2024-04-24T09:20:19.207 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:18 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:18.778+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:28.209530+0000 front 2024-04-24T09:19:28.209142+0000 (oldest deadline 2024-04-24T09:19:51.709380+0000) 2024-04-24T09:20:19.207 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:18 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:18.778+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.613982+0000 front 2024-04-24T09:19:38.614541+0000 (oldest deadline 2024-04-24T09:20:00.913987+0000) 2024-04-24T09:20:19.207 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:18 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:18.778+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:49.018457+0000 front 2024-04-24T09:19:49.018520+0000 (oldest deadline 2024-04-24T09:20:16.019822+0000) 2024-04-24T09:20:19.638 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:19 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:19.358+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:37.707135+0000 front 2024-04-24T09:19:37.707028+0000 (oldest deadline 2024-04-24T09:20:03.607224+0000) 2024-04-24T09:20:19.638 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:19 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:19.359+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:48.713889+0000 front 2024-04-24T09:19:48.713979+0000 (oldest deadline 2024-04-24T09:20:13.414134+0000) 2024-04-24T09:20:19.638 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 24 09:20:19 smithi189 podman[76931]: 2024-04-24 09:20:19.29056943 +0000 UTC m=+5.872101805 container died 208bb941707e34e6a92466bdafe8d5062044d5cae8f1e596d6a159bd0cd9dffa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True) 2024-04-24T09:20:19.806 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 24 09:20:19 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-6[55860]: 2024-04-24T09:20:19.306+0000 7f90fd800700 -1 osd.6 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:39.375781+0000 front 2024-04-24T09:19:39.376197+0000 (oldest deadline 2024-04-24T09:20:04.675612+0000) 2024-04-24T09:20:19.806 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 24 09:20:19 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-6[55860]: 2024-04-24T09:20:19.306+0000 7f90fd800700 -1 osd.6 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:46.978139+0000 front 2024-04-24T09:19:46.978058+0000 (oldest deadline 2024-04-24T09:20:12.278124+0000) 2024-04-24T09:20:19.806 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:19 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:19.341+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:27.386588+0000 front 2024-04-24T09:19:27.386891+0000 (oldest deadline 2024-04-24T09:19:52.686559+0000) 2024-04-24T09:20:19.807 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:19 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:19.341+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.891162+0000 front 2024-04-24T09:19:38.891270+0000 (oldest deadline 2024-04-24T09:20:04.790877+0000) 2024-04-24T09:20:19.807 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:19 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:19.341+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:48.794838+0000 front 2024-04-24T09:19:48.794972+0000 (oldest deadline 2024-04-24T09:20:14.095077+0000) 2024-04-24T09:20:19.955 INFO:journalctl@ceph.osd.7.smithi138.stdout:Apr 24 09:20:19 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-7[49166]: 2024-04-24T09:20:19.680+0000 7fb49b200700 -1 osd.7 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:37.463919+0000 front 2024-04-24T09:19:37.464057+0000 (oldest deadline 2024-04-24T09:20:03.865663+0000) 2024-04-24T09:20:19.955 INFO:journalctl@ceph.osd.7.smithi138.stdout:Apr 24 09:20:19 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-7[49166]: 2024-04-24T09:20:19.680+0000 7fb49b200700 -1 osd.7 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:47.968238+0000 front 2024-04-24T09:19:47.968624+0000 (oldest deadline 2024-04-24T09:20:13.868496+0000) 2024-04-24T09:20:19.955 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:19 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:19.552+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:29.414806+0000 front 2024-04-24T09:19:29.414861+0000 (oldest deadline 2024-04-24T09:19:55.314800+0000) 2024-04-24T09:20:19.955 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:19 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:19.552+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.119965+0000 front 2024-04-24T09:19:38.120034+0000 (oldest deadline 2024-04-24T09:20:02.820181+0000) 2024-04-24T09:20:19.955 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:19 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:19.552+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:47.326279+0000 front 2024-04-24T09:19:47.326338+0000 (oldest deadline 2024-04-24T09:20:10.728055+0000) 2024-04-24T09:20:19.957 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 24 09:20:19 smithi189 podman[76931]: 2024-04-24 09:20:19.757211185 +0000 UTC m=+6.338743556 container cleanup 208bb941707e34e6a92466bdafe8d5062044d5cae8f1e596d6a159bd0cd9dffa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-5, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-24T09:20:19.957 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 24 09:20:19 smithi189 bash[76931]: ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-5 2024-04-24T09:20:19.958 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:19 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:19.764+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:28.209530+0000 front 2024-04-24T09:19:28.209142+0000 (oldest deadline 2024-04-24T09:19:51.709380+0000) 2024-04-24T09:20:19.958 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:19 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:19.764+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.613982+0000 front 2024-04-24T09:19:38.614541+0000 (oldest deadline 2024-04-24T09:20:00.913987+0000) 2024-04-24T09:20:19.958 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:19 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:19.764+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:49.018457+0000 front 2024-04-24T09:19:49.018520+0000 (oldest deadline 2024-04-24T09:20:16.019822+0000) 2024-04-24T09:20:20.354 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 24 09:20:20 smithi189 podman[76962]: 2024-04-24 09:20:20.008486008 +0000 UTC m=+0.699015229 container remove 208bb941707e34e6a92466bdafe8d5062044d5cae8f1e596d6a159bd0cd9dffa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=rishabh-main-apr17-a654945-1f81fb3, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-24T09:20:20.707 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:20 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:20.353+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:37.707135+0000 front 2024-04-24T09:19:37.707028+0000 (oldest deadline 2024-04-24T09:20:03.607224+0000) 2024-04-24T09:20:20.707 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:20 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:20.353+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:48.713889+0000 front 2024-04-24T09:19:48.713979+0000 (oldest deadline 2024-04-24T09:20:13.414134+0000) 2024-04-24T09:20:20.803 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 24 09:20:20 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-6[55860]: 2024-04-24T09:20:20.354+0000 7f90fd800700 -1 osd.6 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:39.375781+0000 front 2024-04-24T09:19:39.376197+0000 (oldest deadline 2024-04-24T09:20:04.675612+0000) 2024-04-24T09:20:20.803 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 24 09:20:20 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-6[55860]: 2024-04-24T09:20:20.354+0000 7f90fd800700 -1 osd.6 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:46.978139+0000 front 2024-04-24T09:19:46.978058+0000 (oldest deadline 2024-04-24T09:20:12.278124+0000) 2024-04-24T09:20:20.803 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:20 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:20.312+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:27.386588+0000 front 2024-04-24T09:19:27.386891+0000 (oldest deadline 2024-04-24T09:19:52.686559+0000) 2024-04-24T09:20:20.803 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:20 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:20.312+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.891162+0000 front 2024-04-24T09:19:38.891270+0000 (oldest deadline 2024-04-24T09:20:04.790877+0000) 2024-04-24T09:20:20.803 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:20 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:20.312+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:48.794838+0000 front 2024-04-24T09:19:48.794972+0000 (oldest deadline 2024-04-24T09:20:14.095077+0000) 2024-04-24T09:20:20.955 INFO:journalctl@ceph.osd.7.smithi138.stdout:Apr 24 09:20:20 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-7[49166]: 2024-04-24T09:20:20.726+0000 7fb49b200700 -1 osd.7 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:37.463919+0000 front 2024-04-24T09:19:37.464057+0000 (oldest deadline 2024-04-24T09:20:03.865663+0000) 2024-04-24T09:20:20.955 INFO:journalctl@ceph.osd.7.smithi138.stdout:Apr 24 09:20:20 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-7[49166]: 2024-04-24T09:20:20.726+0000 7fb49b200700 -1 osd.7 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:47.968238+0000 front 2024-04-24T09:19:47.968624+0000 (oldest deadline 2024-04-24T09:20:13.868496+0000) 2024-04-24T09:20:20.955 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:20 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:20.596+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:29.414806+0000 front 2024-04-24T09:19:29.414861+0000 (oldest deadline 2024-04-24T09:19:55.314800+0000) 2024-04-24T09:20:20.955 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:20 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:20.596+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.119965+0000 front 2024-04-24T09:19:38.120034+0000 (oldest deadline 2024-04-24T09:20:02.820181+0000) 2024-04-24T09:20:20.955 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:20 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:20.596+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:47.326279+0000 front 2024-04-24T09:19:47.326338+0000 (oldest deadline 2024-04-24T09:20:10.728055+0000) 2024-04-24T09:20:21.096 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:20 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:20.762+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:28.209530+0000 front 2024-04-24T09:19:28.209142+0000 (oldest deadline 2024-04-24T09:19:51.709380+0000) 2024-04-24T09:20:21.096 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:20 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:20.763+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.613982+0000 front 2024-04-24T09:19:38.614541+0000 (oldest deadline 2024-04-24T09:20:00.913987+0000) 2024-04-24T09:20:21.096 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:20 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:20.763+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:49.018457+0000 front 2024-04-24T09:19:49.018520+0000 (oldest deadline 2024-04-24T09:20:16.019822+0000) 2024-04-24T09:20:21.096 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 24 09:20:20 smithi189 podman[77053]: 2024-04-24 09:20:20.65672339 +0000 UTC m=+0.037265292 image pull b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644 2024-04-24T09:20:21.096 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 24 09:20:20 smithi189 podman[77053]: 2024-04-24 09:20:20.792763607 +0000 UTC m=+0.173305530 container create ccd2bd771fd0f206c712345309cc6e951438d0f6e4b1f11711fb4a7adf087d0d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=rishabh-main-apr17-a654945-1f81fb3) 2024-04-24T09:20:21.096 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 24 09:20:21 smithi189 podman[77053]: 2024-04-24 09:20:21.096258856 +0000 UTC m=+0.476800786 container init ccd2bd771fd0f206c712345309cc6e951438d0f6e4b1f11711fb4a7adf087d0d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-5-deactivate, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=rishabh-main-apr17-a654945-1f81fb3, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-24T09:20:21.457 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 24 09:20:21 smithi189 podman[77053]: 2024-04-24 09:20:21.107390659 +0000 UTC m=+0.487932577 container start ccd2bd771fd0f206c712345309cc6e951438d0f6e4b1f11711fb4a7adf087d0d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=rishabh-main-apr17-a654945-1f81fb3, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-24T09:20:21.457 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 24 09:20:21 smithi189 podman[77053]: 2024-04-24 09:20:21.212219282 +0000 UTC m=+0.592761235 container attach ccd2bd771fd0f206c712345309cc6e951438d0f6e4b1f11711fb4a7adf087d0d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-04-24T09:20:21.457 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:21 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:21.335+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:37.707135+0000 front 2024-04-24T09:19:37.707028+0000 (oldest deadline 2024-04-24T09:20:03.607224+0000) 2024-04-24T09:20:21.457 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:21 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:21.335+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:48.713889+0000 front 2024-04-24T09:19:48.713979+0000 (oldest deadline 2024-04-24T09:20:13.414134+0000) 2024-04-24T09:20:21.803 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:21 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:21.306+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:27.386588+0000 front 2024-04-24T09:19:27.386891+0000 (oldest deadline 2024-04-24T09:19:52.686559+0000) 2024-04-24T09:20:21.803 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:21 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:21.306+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.891162+0000 front 2024-04-24T09:19:38.891270+0000 (oldest deadline 2024-04-24T09:20:04.790877+0000) 2024-04-24T09:20:21.803 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:21 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:21.306+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:48.794838+0000 front 2024-04-24T09:19:48.794972+0000 (oldest deadline 2024-04-24T09:20:14.095077+0000) 2024-04-24T09:20:21.803 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 24 09:20:21 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-6[55860]: 2024-04-24T09:20:21.393+0000 7f90fd800700 -1 osd.6 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:39.375781+0000 front 2024-04-24T09:19:39.376197+0000 (oldest deadline 2024-04-24T09:20:04.675612+0000) 2024-04-24T09:20:21.803 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 24 09:20:21 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-6[55860]: 2024-04-24T09:20:21.393+0000 7f90fd800700 -1 osd.6 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:46.978139+0000 front 2024-04-24T09:19:46.978058+0000 (oldest deadline 2024-04-24T09:20:12.278124+0000) 2024-04-24T09:20:21.804 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 24 09:20:21 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-6[55860]: 2024-04-24T09:20:21.393+0000 7f90fd800700 -1 osd.6 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:57.580254+0000 front 2024-04-24T09:19:57.580200+0000 (oldest deadline 2024-04-24T09:20:20.480599+0000) 2024-04-24T09:20:21.863 INFO:journalctl@ceph.osd.5.smithi189.stdout:Apr 24 09:20:21 smithi189 podman[77053]: 2024-04-24 09:20:21.556617818 +0000 UTC m=+0.937159778 container died ccd2bd771fd0f206c712345309cc6e951438d0f6e4b1f11711fb4a7adf087d0d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-04-24T09:20:21.864 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:21 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:21.799+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:28.209530+0000 front 2024-04-24T09:19:28.209142+0000 (oldest deadline 2024-04-24T09:19:51.709380+0000) 2024-04-24T09:20:21.864 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:21 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:21.799+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.613982+0000 front 2024-04-24T09:19:38.614541+0000 (oldest deadline 2024-04-24T09:20:00.913987+0000) 2024-04-24T09:20:21.864 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:21 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:21.799+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:49.018457+0000 front 2024-04-24T09:19:49.018520+0000 (oldest deadline 2024-04-24T09:20:16.019822+0000) 2024-04-24T09:20:21.955 INFO:journalctl@ceph.osd.7.smithi138.stdout:Apr 24 09:20:21 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-7[49166]: 2024-04-24T09:20:21.720+0000 7fb49b200700 -1 osd.7 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:37.463919+0000 front 2024-04-24T09:19:37.464057+0000 (oldest deadline 2024-04-24T09:20:03.865663+0000) 2024-04-24T09:20:21.955 INFO:journalctl@ceph.osd.7.smithi138.stdout:Apr 24 09:20:21 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-7[49166]: 2024-04-24T09:20:21.720+0000 7fb49b200700 -1 osd.7 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:47.968238+0000 front 2024-04-24T09:19:47.968624+0000 (oldest deadline 2024-04-24T09:20:13.868496+0000) 2024-04-24T09:20:21.955 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:21 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:21.557+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:29.414806+0000 front 2024-04-24T09:19:29.414861+0000 (oldest deadline 2024-04-24T09:19:55.314800+0000) 2024-04-24T09:20:21.955 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:21 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:21.557+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.119965+0000 front 2024-04-24T09:19:38.120034+0000 (oldest deadline 2024-04-24T09:20:02.820181+0000) 2024-04-24T09:20:21.955 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:21 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:21.557+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:47.326279+0000 front 2024-04-24T09:19:47.326338+0000 (oldest deadline 2024-04-24T09:20:10.728055+0000) 2024-04-24T09:20:22.328 DEBUG:teuthology.orchestra.run.smithi189:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bd3e039e-0215-11ef-bc93-c7b262605968@osd.5.service' 2024-04-24T09:20:22.442 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T09:20:22.442 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-04-24T09:20:22.442 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-04-24T09:20:22.443 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-bd3e039e-0215-11ef-bc93-c7b262605968@osd.6 2024-04-24T09:20:22.707 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:22 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:22.304+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:37.707135+0000 front 2024-04-24T09:19:37.707028+0000 (oldest deadline 2024-04-24T09:20:03.607224+0000) 2024-04-24T09:20:22.707 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:22 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:22.304+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:48.713889+0000 front 2024-04-24T09:19:48.713979+0000 (oldest deadline 2024-04-24T09:20:13.414134+0000) 2024-04-24T09:20:22.707 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:22 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:22.304+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:58.715994+0000 front 2024-04-24T09:19:58.716101+0000 (oldest deadline 2024-04-24T09:20:22.216080+0000) 2024-04-24T09:20:22.804 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:22 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:22.326+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:27.386588+0000 front 2024-04-24T09:19:27.386891+0000 (oldest deadline 2024-04-24T09:19:52.686559+0000) 2024-04-24T09:20:22.804 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:22 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:22.326+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.891162+0000 front 2024-04-24T09:19:38.891270+0000 (oldest deadline 2024-04-24T09:20:04.790877+0000) 2024-04-24T09:20:22.804 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:22 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:22.326+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:48.794838+0000 front 2024-04-24T09:19:48.794972+0000 (oldest deadline 2024-04-24T09:20:14.095077+0000) 2024-04-24T09:20:22.804 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 24 09:20:22 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-6[55860]: 2024-04-24T09:20:22.375+0000 7f90fd800700 -1 osd.6 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:39.375781+0000 front 2024-04-24T09:19:39.376197+0000 (oldest deadline 2024-04-24T09:20:04.675612+0000) 2024-04-24T09:20:22.804 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 24 09:20:22 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-6[55860]: 2024-04-24T09:20:22.375+0000 7f90fd800700 -1 osd.6 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:46.978139+0000 front 2024-04-24T09:19:46.978058+0000 (oldest deadline 2024-04-24T09:20:12.278124+0000) 2024-04-24T09:20:22.804 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 24 09:20:22 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-6[55860]: 2024-04-24T09:20:22.375+0000 7f90fd800700 -1 osd.6 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:57.580254+0000 front 2024-04-24T09:19:57.580200+0000 (oldest deadline 2024-04-24T09:20:20.480599+0000) 2024-04-24T09:20:22.804 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 24 09:20:22 smithi083 systemd[1]: Stopping Ceph osd.6 for bd3e039e-0215-11ef-bc93-c7b262605968... 2024-04-24T09:20:22.955 INFO:journalctl@ceph.osd.7.smithi138.stdout:Apr 24 09:20:22 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-7[49166]: 2024-04-24T09:20:22.715+0000 7fb49b200700 -1 osd.7 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:37.463919+0000 front 2024-04-24T09:19:37.464057+0000 (oldest deadline 2024-04-24T09:20:03.865663+0000) 2024-04-24T09:20:22.955 INFO:journalctl@ceph.osd.7.smithi138.stdout:Apr 24 09:20:22 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-7[49166]: 2024-04-24T09:20:22.715+0000 7fb49b200700 -1 osd.7 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:47.968238+0000 front 2024-04-24T09:19:47.968624+0000 (oldest deadline 2024-04-24T09:20:13.868496+0000) 2024-04-24T09:20:22.955 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:22 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:22.524+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:29.414806+0000 front 2024-04-24T09:19:29.414861+0000 (oldest deadline 2024-04-24T09:19:55.314800+0000) 2024-04-24T09:20:22.955 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:22 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:22.524+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.119965+0000 front 2024-04-24T09:19:38.120034+0000 (oldest deadline 2024-04-24T09:20:02.820181+0000) 2024-04-24T09:20:22.955 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:22 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:22.524+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:47.326279+0000 front 2024-04-24T09:19:47.326338+0000 (oldest deadline 2024-04-24T09:20:10.728055+0000) 2024-04-24T09:20:22.955 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:22 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:22.524+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:55.832608+0000 front 2024-04-24T09:19:55.832510+0000 (oldest deadline 2024-04-24T09:20:21.732750+0000) 2024-04-24T09:20:23.207 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:22 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:22.757+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:28.209530+0000 front 2024-04-24T09:19:28.209142+0000 (oldest deadline 2024-04-24T09:19:51.709380+0000) 2024-04-24T09:20:23.207 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:22 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:22.757+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.613982+0000 front 2024-04-24T09:19:38.614541+0000 (oldest deadline 2024-04-24T09:20:00.913987+0000) 2024-04-24T09:20:23.207 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:22 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:22.757+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:49.018457+0000 front 2024-04-24T09:19:49.018520+0000 (oldest deadline 2024-04-24T09:20:16.019822+0000) 2024-04-24T09:20:23.410 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:23 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:23.277+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:27.386588+0000 front 2024-04-24T09:19:27.386891+0000 (oldest deadline 2024-04-24T09:19:52.686559+0000) 2024-04-24T09:20:23.410 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:23 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:23.278+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.891162+0000 front 2024-04-24T09:19:38.891270+0000 (oldest deadline 2024-04-24T09:20:04.790877+0000) 2024-04-24T09:20:23.410 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:23 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:23.278+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:48.794838+0000 front 2024-04-24T09:19:48.794972+0000 (oldest deadline 2024-04-24T09:20:14.095077+0000) 2024-04-24T09:20:23.411 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:23 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:23.278+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:56.997332+0000 front 2024-04-24T09:19:56.997477+0000 (oldest deadline 2024-04-24T09:20:22.897608+0000) 2024-04-24T09:20:23.411 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 24 09:20:23 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-6[55860]: 2024-04-24T09:20:23.081+0000 7f9107000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T09:20:23.411 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 24 09:20:23 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-6[55860]: 2024-04-24T09:20:23.081+0000 7f9107000700 -1 osd.6 141 *** Got signal Terminated *** 2024-04-24T09:20:23.411 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 24 09:20:23 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-6[55860]: 2024-04-24T09:20:23.081+0000 7f9107000700 -1 osd.6 141 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T09:20:23.411 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 24 09:20:23 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-6[55860]: 2024-04-24T09:20:23.409+0000 7f90fd800700 -1 osd.6 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:39.375781+0000 front 2024-04-24T09:19:39.376197+0000 (oldest deadline 2024-04-24T09:20:04.675612+0000) 2024-04-24T09:20:23.411 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 24 09:20:23 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-6[55860]: 2024-04-24T09:20:23.409+0000 7f90fd800700 -1 osd.6 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:46.978139+0000 front 2024-04-24T09:19:46.978058+0000 (oldest deadline 2024-04-24T09:20:12.278124+0000) 2024-04-24T09:20:23.411 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 24 09:20:23 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-6[55860]: 2024-04-24T09:20:23.409+0000 7f90fd800700 -1 osd.6 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:57.580254+0000 front 2024-04-24T09:19:57.580200+0000 (oldest deadline 2024-04-24T09:20:20.480599+0000) 2024-04-24T09:20:23.707 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:23 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:23.300+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:37.707135+0000 front 2024-04-24T09:19:37.707028+0000 (oldest deadline 2024-04-24T09:20:03.607224+0000) 2024-04-24T09:20:23.707 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:23 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:23.300+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:48.713889+0000 front 2024-04-24T09:19:48.713979+0000 (oldest deadline 2024-04-24T09:20:13.414134+0000) 2024-04-24T09:20:23.707 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:23 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:23.300+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:58.715994+0000 front 2024-04-24T09:19:58.716101+0000 (oldest deadline 2024-04-24T09:20:22.216080+0000) 2024-04-24T09:20:23.955 INFO:journalctl@ceph.osd.7.smithi138.stdout:Apr 24 09:20:23 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-7[49166]: 2024-04-24T09:20:23.701+0000 7fb49b200700 -1 osd.7 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:37.463919+0000 front 2024-04-24T09:19:37.464057+0000 (oldest deadline 2024-04-24T09:20:03.865663+0000) 2024-04-24T09:20:23.955 INFO:journalctl@ceph.osd.7.smithi138.stdout:Apr 24 09:20:23 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-7[49166]: 2024-04-24T09:20:23.701+0000 7fb49b200700 -1 osd.7 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:47.968238+0000 front 2024-04-24T09:19:47.968624+0000 (oldest deadline 2024-04-24T09:20:13.868496+0000) 2024-04-24T09:20:23.955 INFO:journalctl@ceph.osd.7.smithi138.stdout:Apr 24 09:20:23 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-7[49166]: 2024-04-24T09:20:23.701+0000 7fb49b200700 -1 osd.7 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:58.373589+0000 front 2024-04-24T09:19:58.373837+0000 (oldest deadline 2024-04-24T09:20:23.073323+0000) 2024-04-24T09:20:23.955 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:23 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:23.557+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:29.414806+0000 front 2024-04-24T09:19:29.414861+0000 (oldest deadline 2024-04-24T09:19:55.314800+0000) 2024-04-24T09:20:23.955 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:23 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:23.557+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.119965+0000 front 2024-04-24T09:19:38.120034+0000 (oldest deadline 2024-04-24T09:20:02.820181+0000) 2024-04-24T09:20:23.956 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:23 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:23.557+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:47.326279+0000 front 2024-04-24T09:19:47.326338+0000 (oldest deadline 2024-04-24T09:20:10.728055+0000) 2024-04-24T09:20:23.956 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:23 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:23.557+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:55.832608+0000 front 2024-04-24T09:19:55.832510+0000 (oldest deadline 2024-04-24T09:20:21.732750+0000) 2024-04-24T09:20:24.207 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:23 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:23.756+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:28.209530+0000 front 2024-04-24T09:19:28.209142+0000 (oldest deadline 2024-04-24T09:19:51.709380+0000) 2024-04-24T09:20:24.207 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:23 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:23.756+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.613982+0000 front 2024-04-24T09:19:38.614541+0000 (oldest deadline 2024-04-24T09:20:00.913987+0000) 2024-04-24T09:20:24.207 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:23 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:23.756+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:49.018457+0000 front 2024-04-24T09:19:49.018520+0000 (oldest deadline 2024-04-24T09:20:16.019822+0000) 2024-04-24T09:20:24.438 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:24 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:24.287+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:27.386588+0000 front 2024-04-24T09:19:27.386891+0000 (oldest deadline 2024-04-24T09:19:52.686559+0000) 2024-04-24T09:20:24.438 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:24 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:24.287+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.891162+0000 front 2024-04-24T09:19:38.891270+0000 (oldest deadline 2024-04-24T09:20:04.790877+0000) 2024-04-24T09:20:24.438 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:24 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:24.287+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:48.794838+0000 front 2024-04-24T09:19:48.794972+0000 (oldest deadline 2024-04-24T09:20:14.095077+0000) 2024-04-24T09:20:24.438 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:24 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:24.287+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:56.997332+0000 front 2024-04-24T09:19:56.997477+0000 (oldest deadline 2024-04-24T09:20:22.897608+0000) 2024-04-24T09:20:24.707 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:24 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:24.287+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:37.707135+0000 front 2024-04-24T09:19:37.707028+0000 (oldest deadline 2024-04-24T09:20:03.607224+0000) 2024-04-24T09:20:24.707 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:24 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:24.287+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:48.713889+0000 front 2024-04-24T09:19:48.713979+0000 (oldest deadline 2024-04-24T09:20:13.414134+0000) 2024-04-24T09:20:24.707 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:24 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:24.287+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:58.715994+0000 front 2024-04-24T09:19:58.716101+0000 (oldest deadline 2024-04-24T09:20:22.216080+0000) 2024-04-24T09:20:24.803 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 24 09:20:24 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-6[55860]: 2024-04-24T09:20:24.437+0000 7f90fd800700 -1 osd.6 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:39.375781+0000 front 2024-04-24T09:19:39.376197+0000 (oldest deadline 2024-04-24T09:20:04.675612+0000) 2024-04-24T09:20:24.803 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 24 09:20:24 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-6[55860]: 2024-04-24T09:20:24.437+0000 7f90fd800700 -1 osd.6 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:46.978139+0000 front 2024-04-24T09:19:46.978058+0000 (oldest deadline 2024-04-24T09:20:12.278124+0000) 2024-04-24T09:20:24.803 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 24 09:20:24 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-6[55860]: 2024-04-24T09:20:24.437+0000 7f90fd800700 -1 osd.6 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:57.580254+0000 front 2024-04-24T09:19:57.580200+0000 (oldest deadline 2024-04-24T09:20:20.480599+0000) 2024-04-24T09:20:24.955 INFO:journalctl@ceph.osd.7.smithi138.stdout:Apr 24 09:20:24 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-7[49166]: 2024-04-24T09:20:24.697+0000 7fb49b200700 -1 osd.7 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:37.463919+0000 front 2024-04-24T09:19:37.464057+0000 (oldest deadline 2024-04-24T09:20:03.865663+0000) 2024-04-24T09:20:24.955 INFO:journalctl@ceph.osd.7.smithi138.stdout:Apr 24 09:20:24 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-7[49166]: 2024-04-24T09:20:24.697+0000 7fb49b200700 -1 osd.7 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:47.968238+0000 front 2024-04-24T09:19:47.968624+0000 (oldest deadline 2024-04-24T09:20:13.868496+0000) 2024-04-24T09:20:24.955 INFO:journalctl@ceph.osd.7.smithi138.stdout:Apr 24 09:20:24 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-7[49166]: 2024-04-24T09:20:24.697+0000 7fb49b200700 -1 osd.7 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:58.373589+0000 front 2024-04-24T09:19:58.373837+0000 (oldest deadline 2024-04-24T09:20:23.073323+0000) 2024-04-24T09:20:24.955 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:24 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:24.542+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:29.414806+0000 front 2024-04-24T09:19:29.414861+0000 (oldest deadline 2024-04-24T09:19:55.314800+0000) 2024-04-24T09:20:24.955 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:24 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:24.542+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.119965+0000 front 2024-04-24T09:19:38.120034+0000 (oldest deadline 2024-04-24T09:20:02.820181+0000) 2024-04-24T09:20:24.955 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:24 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:24.542+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:47.326279+0000 front 2024-04-24T09:19:47.326338+0000 (oldest deadline 2024-04-24T09:20:10.728055+0000) 2024-04-24T09:20:24.955 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:24 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:24.542+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:55.832608+0000 front 2024-04-24T09:19:55.832510+0000 (oldest deadline 2024-04-24T09:20:21.732750+0000) 2024-04-24T09:20:25.207 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:24 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:24.773+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:28.209530+0000 front 2024-04-24T09:19:28.209142+0000 (oldest deadline 2024-04-24T09:19:51.709380+0000) 2024-04-24T09:20:25.207 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:24 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:24.773+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.613982+0000 front 2024-04-24T09:19:38.614541+0000 (oldest deadline 2024-04-24T09:20:00.913987+0000) 2024-04-24T09:20:25.207 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:24 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:24.773+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:49.018457+0000 front 2024-04-24T09:19:49.018520+0000 (oldest deadline 2024-04-24T09:20:16.019822+0000) 2024-04-24T09:20:25.303 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:25 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:25.260+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:27.386588+0000 front 2024-04-24T09:19:27.386891+0000 (oldest deadline 2024-04-24T09:19:52.686559+0000) 2024-04-24T09:20:25.303 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:25 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:25.260+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.891162+0000 front 2024-04-24T09:19:38.891270+0000 (oldest deadline 2024-04-24T09:20:04.790877+0000) 2024-04-24T09:20:25.303 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:25 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:25.260+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:48.794838+0000 front 2024-04-24T09:19:48.794972+0000 (oldest deadline 2024-04-24T09:20:14.095077+0000) 2024-04-24T09:20:25.303 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:25 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:25.260+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:56.997332+0000 front 2024-04-24T09:19:56.997477+0000 (oldest deadline 2024-04-24T09:20:22.897608+0000) 2024-04-24T09:20:25.707 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:25 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:25.239+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:37.707135+0000 front 2024-04-24T09:19:37.707028+0000 (oldest deadline 2024-04-24T09:20:03.607224+0000) 2024-04-24T09:20:25.707 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:25 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:25.239+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:48.713889+0000 front 2024-04-24T09:19:48.713979+0000 (oldest deadline 2024-04-24T09:20:13.414134+0000) 2024-04-24T09:20:25.707 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:25 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:25.239+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:58.715994+0000 front 2024-04-24T09:19:58.716101+0000 (oldest deadline 2024-04-24T09:20:22.216080+0000) 2024-04-24T09:20:25.803 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 24 09:20:25 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-6[55860]: 2024-04-24T09:20:25.396+0000 7f90fd800700 -1 osd.6 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:39.375781+0000 front 2024-04-24T09:19:39.376197+0000 (oldest deadline 2024-04-24T09:20:04.675612+0000) 2024-04-24T09:20:25.803 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 24 09:20:25 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-6[55860]: 2024-04-24T09:20:25.396+0000 7f90fd800700 -1 osd.6 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:46.978139+0000 front 2024-04-24T09:19:46.978058+0000 (oldest deadline 2024-04-24T09:20:12.278124+0000) 2024-04-24T09:20:25.803 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 24 09:20:25 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-6[55860]: 2024-04-24T09:20:25.396+0000 7f90fd800700 -1 osd.6 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:57.580254+0000 front 2024-04-24T09:19:57.580200+0000 (oldest deadline 2024-04-24T09:20:20.480599+0000) 2024-04-24T09:20:25.955 INFO:journalctl@ceph.osd.7.smithi138.stdout:Apr 24 09:20:25 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-7[49166]: 2024-04-24T09:20:25.716+0000 7fb49b200700 -1 osd.7 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:37.463919+0000 front 2024-04-24T09:19:37.464057+0000 (oldest deadline 2024-04-24T09:20:03.865663+0000) 2024-04-24T09:20:25.955 INFO:journalctl@ceph.osd.7.smithi138.stdout:Apr 24 09:20:25 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-7[49166]: 2024-04-24T09:20:25.716+0000 7fb49b200700 -1 osd.7 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:47.968238+0000 front 2024-04-24T09:19:47.968624+0000 (oldest deadline 2024-04-24T09:20:13.868496+0000) 2024-04-24T09:20:25.955 INFO:journalctl@ceph.osd.7.smithi138.stdout:Apr 24 09:20:25 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-7[49166]: 2024-04-24T09:20:25.716+0000 7fb49b200700 -1 osd.7 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:58.373589+0000 front 2024-04-24T09:19:58.373837+0000 (oldest deadline 2024-04-24T09:20:23.073323+0000) 2024-04-24T09:20:25.955 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:25 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:25.558+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:29.414806+0000 front 2024-04-24T09:19:29.414861+0000 (oldest deadline 2024-04-24T09:19:55.314800+0000) 2024-04-24T09:20:25.955 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:25 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:25.558+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.119965+0000 front 2024-04-24T09:19:38.120034+0000 (oldest deadline 2024-04-24T09:20:02.820181+0000) 2024-04-24T09:20:25.955 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:25 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:25.558+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:47.326279+0000 front 2024-04-24T09:19:47.326338+0000 (oldest deadline 2024-04-24T09:20:10.728055+0000) 2024-04-24T09:20:25.955 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:25 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:25.558+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:55.832608+0000 front 2024-04-24T09:19:55.832510+0000 (oldest deadline 2024-04-24T09:20:21.732750+0000) 2024-04-24T09:20:26.207 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:25 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:25.803+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:28.209530+0000 front 2024-04-24T09:19:28.209142+0000 (oldest deadline 2024-04-24T09:19:51.709380+0000) 2024-04-24T09:20:26.207 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:25 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:25.803+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.613982+0000 front 2024-04-24T09:19:38.614541+0000 (oldest deadline 2024-04-24T09:20:00.913987+0000) 2024-04-24T09:20:26.207 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:25 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:25.804+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:49.018457+0000 front 2024-04-24T09:19:49.018520+0000 (oldest deadline 2024-04-24T09:20:16.019822+0000) 2024-04-24T09:20:26.553 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 24 09:20:26 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-6[55860]: 2024-04-24T09:20:26.444+0000 7f90fd800700 -1 osd.6 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:39.375781+0000 front 2024-04-24T09:19:39.376197+0000 (oldest deadline 2024-04-24T09:20:04.675612+0000) 2024-04-24T09:20:26.553 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 24 09:20:26 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-6[55860]: 2024-04-24T09:20:26.444+0000 7f90fd800700 -1 osd.6 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:46.978139+0000 front 2024-04-24T09:19:46.978058+0000 (oldest deadline 2024-04-24T09:20:12.278124+0000) 2024-04-24T09:20:26.554 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 24 09:20:26 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-6[55860]: 2024-04-24T09:20:26.444+0000 7f90fd800700 -1 osd.6 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:57.580254+0000 front 2024-04-24T09:19:57.580200+0000 (oldest deadline 2024-04-24T09:20:20.480599+0000) 2024-04-24T09:20:26.554 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:26 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:26.262+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:27.386588+0000 front 2024-04-24T09:19:27.386891+0000 (oldest deadline 2024-04-24T09:19:52.686559+0000) 2024-04-24T09:20:26.554 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:26 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:26.262+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.891162+0000 front 2024-04-24T09:19:38.891270+0000 (oldest deadline 2024-04-24T09:20:04.790877+0000) 2024-04-24T09:20:26.554 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:26 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:26.262+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:48.794838+0000 front 2024-04-24T09:19:48.794972+0000 (oldest deadline 2024-04-24T09:20:14.095077+0000) 2024-04-24T09:20:26.554 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:26 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:26.262+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:56.997332+0000 front 2024-04-24T09:19:56.997477+0000 (oldest deadline 2024-04-24T09:20:22.897608+0000) 2024-04-24T09:20:26.707 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:26 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:26.248+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:37.707135+0000 front 2024-04-24T09:19:37.707028+0000 (oldest deadline 2024-04-24T09:20:03.607224+0000) 2024-04-24T09:20:26.707 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:26 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:26.248+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:48.713889+0000 front 2024-04-24T09:19:48.713979+0000 (oldest deadline 2024-04-24T09:20:13.414134+0000) 2024-04-24T09:20:26.707 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:26 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:26.248+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:58.715994+0000 front 2024-04-24T09:19:58.716101+0000 (oldest deadline 2024-04-24T09:20:22.216080+0000) 2024-04-24T09:20:26.955 INFO:journalctl@ceph.osd.7.smithi138.stdout:Apr 24 09:20:26 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-7[49166]: 2024-04-24T09:20:26.749+0000 7fb49b200700 -1 osd.7 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:37.463919+0000 front 2024-04-24T09:19:37.464057+0000 (oldest deadline 2024-04-24T09:20:03.865663+0000) 2024-04-24T09:20:26.955 INFO:journalctl@ceph.osd.7.smithi138.stdout:Apr 24 09:20:26 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-7[49166]: 2024-04-24T09:20:26.749+0000 7fb49b200700 -1 osd.7 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:47.968238+0000 front 2024-04-24T09:19:47.968624+0000 (oldest deadline 2024-04-24T09:20:13.868496+0000) 2024-04-24T09:20:26.955 INFO:journalctl@ceph.osd.7.smithi138.stdout:Apr 24 09:20:26 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-7[49166]: 2024-04-24T09:20:26.749+0000 7fb49b200700 -1 osd.7 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:58.373589+0000 front 2024-04-24T09:19:58.373837+0000 (oldest deadline 2024-04-24T09:20:23.073323+0000) 2024-04-24T09:20:26.955 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:26 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:26.529+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:29.414806+0000 front 2024-04-24T09:19:29.414861+0000 (oldest deadline 2024-04-24T09:19:55.314800+0000) 2024-04-24T09:20:26.955 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:26 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:26.529+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.119965+0000 front 2024-04-24T09:19:38.120034+0000 (oldest deadline 2024-04-24T09:20:02.820181+0000) 2024-04-24T09:20:26.955 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:26 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:26.529+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:47.326279+0000 front 2024-04-24T09:19:47.326338+0000 (oldest deadline 2024-04-24T09:20:10.728055+0000) 2024-04-24T09:20:26.955 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:26 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:26.529+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:55.832608+0000 front 2024-04-24T09:19:55.832510+0000 (oldest deadline 2024-04-24T09:20:21.732750+0000) 2024-04-24T09:20:27.207 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:26 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:26.755+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:28.209530+0000 front 2024-04-24T09:19:28.209142+0000 (oldest deadline 2024-04-24T09:19:51.709380+0000) 2024-04-24T09:20:27.207 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:26 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:26.755+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.613982+0000 front 2024-04-24T09:19:38.614541+0000 (oldest deadline 2024-04-24T09:20:00.913987+0000) 2024-04-24T09:20:27.207 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:26 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:26.755+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:49.018457+0000 front 2024-04-24T09:19:49.018520+0000 (oldest deadline 2024-04-24T09:20:16.019822+0000) 2024-04-24T09:20:27.553 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 24 09:20:27 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-6[55860]: 2024-04-24T09:20:27.488+0000 7f90fd800700 -1 osd.6 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:39.375781+0000 front 2024-04-24T09:19:39.376197+0000 (oldest deadline 2024-04-24T09:20:04.675612+0000) 2024-04-24T09:20:27.553 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 24 09:20:27 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-6[55860]: 2024-04-24T09:20:27.488+0000 7f90fd800700 -1 osd.6 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:46.978139+0000 front 2024-04-24T09:19:46.978058+0000 (oldest deadline 2024-04-24T09:20:12.278124+0000) 2024-04-24T09:20:27.553 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 24 09:20:27 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-6[55860]: 2024-04-24T09:20:27.488+0000 7f90fd800700 -1 osd.6 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:57.580254+0000 front 2024-04-24T09:19:57.580200+0000 (oldest deadline 2024-04-24T09:20:20.480599+0000) 2024-04-24T09:20:27.554 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:27 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:27.274+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:27.386588+0000 front 2024-04-24T09:19:27.386891+0000 (oldest deadline 2024-04-24T09:19:52.686559+0000) 2024-04-24T09:20:27.554 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:27 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:27.274+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.891162+0000 front 2024-04-24T09:19:38.891270+0000 (oldest deadline 2024-04-24T09:20:04.790877+0000) 2024-04-24T09:20:27.554 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:27 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:27.275+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:48.794838+0000 front 2024-04-24T09:19:48.794972+0000 (oldest deadline 2024-04-24T09:20:14.095077+0000) 2024-04-24T09:20:27.554 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:27 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:27.275+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:56.997332+0000 front 2024-04-24T09:19:56.997477+0000 (oldest deadline 2024-04-24T09:20:22.897608+0000) 2024-04-24T09:20:27.707 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:27 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:27.226+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:37.707135+0000 front 2024-04-24T09:19:37.707028+0000 (oldest deadline 2024-04-24T09:20:03.607224+0000) 2024-04-24T09:20:27.707 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:27 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:27.226+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:48.713889+0000 front 2024-04-24T09:19:48.713979+0000 (oldest deadline 2024-04-24T09:20:13.414134+0000) 2024-04-24T09:20:27.707 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:27 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:27.226+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:58.715994+0000 front 2024-04-24T09:19:58.716101+0000 (oldest deadline 2024-04-24T09:20:22.216080+0000) 2024-04-24T09:20:27.955 INFO:journalctl@ceph.osd.7.smithi138.stdout:Apr 24 09:20:27 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-7[49166]: 2024-04-24T09:20:27.736+0000 7fb49b200700 -1 osd.7 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:37.463919+0000 front 2024-04-24T09:19:37.464057+0000 (oldest deadline 2024-04-24T09:20:03.865663+0000) 2024-04-24T09:20:27.955 INFO:journalctl@ceph.osd.7.smithi138.stdout:Apr 24 09:20:27 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-7[49166]: 2024-04-24T09:20:27.736+0000 7fb49b200700 -1 osd.7 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:47.968238+0000 front 2024-04-24T09:19:47.968624+0000 (oldest deadline 2024-04-24T09:20:13.868496+0000) 2024-04-24T09:20:27.955 INFO:journalctl@ceph.osd.7.smithi138.stdout:Apr 24 09:20:27 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-7[49166]: 2024-04-24T09:20:27.736+0000 7fb49b200700 -1 osd.7 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:58.373589+0000 front 2024-04-24T09:19:58.373837+0000 (oldest deadline 2024-04-24T09:20:23.073323+0000) 2024-04-24T09:20:27.955 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:27 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:27.576+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:29.414806+0000 front 2024-04-24T09:19:29.414861+0000 (oldest deadline 2024-04-24T09:19:55.314800+0000) 2024-04-24T09:20:27.955 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:27 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:27.576+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.119965+0000 front 2024-04-24T09:19:38.120034+0000 (oldest deadline 2024-04-24T09:20:02.820181+0000) 2024-04-24T09:20:27.956 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:27 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:27.576+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:47.326279+0000 front 2024-04-24T09:19:47.326338+0000 (oldest deadline 2024-04-24T09:20:10.728055+0000) 2024-04-24T09:20:27.956 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:27 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:27.576+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:55.832608+0000 front 2024-04-24T09:19:55.832510+0000 (oldest deadline 2024-04-24T09:20:21.732750+0000) 2024-04-24T09:20:28.207 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:27 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:27.784+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:28.209530+0000 front 2024-04-24T09:19:28.209142+0000 (oldest deadline 2024-04-24T09:19:51.709380+0000) 2024-04-24T09:20:28.207 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:27 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:27.784+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.613982+0000 front 2024-04-24T09:19:38.614541+0000 (oldest deadline 2024-04-24T09:20:00.913987+0000) 2024-04-24T09:20:28.207 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:27 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:27.784+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:49.018457+0000 front 2024-04-24T09:19:49.018520+0000 (oldest deadline 2024-04-24T09:20:16.019822+0000) 2024-04-24T09:20:28.553 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:28 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:28.269+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:27.386588+0000 front 2024-04-24T09:19:27.386891+0000 (oldest deadline 2024-04-24T09:19:52.686559+0000) 2024-04-24T09:20:28.553 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:28 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:28.269+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.891162+0000 front 2024-04-24T09:19:38.891270+0000 (oldest deadline 2024-04-24T09:20:04.790877+0000) 2024-04-24T09:20:28.553 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:28 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:28.269+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:48.794838+0000 front 2024-04-24T09:19:48.794972+0000 (oldest deadline 2024-04-24T09:20:14.095077+0000) 2024-04-24T09:20:28.554 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:28 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:28.269+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:56.997332+0000 front 2024-04-24T09:19:56.997477+0000 (oldest deadline 2024-04-24T09:20:22.897608+0000) 2024-04-24T09:20:28.707 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:28 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:28.211+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:37.707135+0000 front 2024-04-24T09:19:37.707028+0000 (oldest deadline 2024-04-24T09:20:03.607224+0000) 2024-04-24T09:20:28.707 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:28 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:28.211+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:48.713889+0000 front 2024-04-24T09:19:48.713979+0000 (oldest deadline 2024-04-24T09:20:13.414134+0000) 2024-04-24T09:20:28.707 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:28 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:28.211+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:58.715994+0000 front 2024-04-24T09:19:58.716101+0000 (oldest deadline 2024-04-24T09:20:22.216080+0000) 2024-04-24T09:20:28.955 INFO:journalctl@ceph.osd.7.smithi138.stdout:Apr 24 09:20:28 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-7[49166]: 2024-04-24T09:20:28.740+0000 7fb49b200700 -1 osd.7 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:37.463919+0000 front 2024-04-24T09:19:37.464057+0000 (oldest deadline 2024-04-24T09:20:03.865663+0000) 2024-04-24T09:20:28.955 INFO:journalctl@ceph.osd.7.smithi138.stdout:Apr 24 09:20:28 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-7[49166]: 2024-04-24T09:20:28.740+0000 7fb49b200700 -1 osd.7 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:47.968238+0000 front 2024-04-24T09:19:47.968624+0000 (oldest deadline 2024-04-24T09:20:13.868496+0000) 2024-04-24T09:20:28.955 INFO:journalctl@ceph.osd.7.smithi138.stdout:Apr 24 09:20:28 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-7[49166]: 2024-04-24T09:20:28.740+0000 7fb49b200700 -1 osd.7 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:58.373589+0000 front 2024-04-24T09:19:58.373837+0000 (oldest deadline 2024-04-24T09:20:23.073323+0000) 2024-04-24T09:20:28.955 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:28 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:28.609+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:29.414806+0000 front 2024-04-24T09:19:29.414861+0000 (oldest deadline 2024-04-24T09:19:55.314800+0000) 2024-04-24T09:20:28.955 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:28 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:28.609+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.119965+0000 front 2024-04-24T09:19:38.120034+0000 (oldest deadline 2024-04-24T09:20:02.820181+0000) 2024-04-24T09:20:28.955 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:28 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:28.609+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:47.326279+0000 front 2024-04-24T09:19:47.326338+0000 (oldest deadline 2024-04-24T09:20:10.728055+0000) 2024-04-24T09:20:28.955 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:28 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:28.609+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:55.832608+0000 front 2024-04-24T09:19:55.832510+0000 (oldest deadline 2024-04-24T09:20:21.732750+0000) 2024-04-24T09:20:28.980 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 24 09:20:28 smithi083 podman[131912]: 2024-04-24 09:20:28.655765029 +0000 UTC m=+5.748407001 container died 55e96ecb5117728cb4a5672be281e0a7b85cd4614bc8b7506c1d06a7fa543712 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-6, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=rishabh-main-apr17-a654945-1f81fb3) 2024-04-24T09:20:29.193 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:28 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:28.792+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:28.209530+0000 front 2024-04-24T09:19:28.209142+0000 (oldest deadline 2024-04-24T09:19:51.709380+0000) 2024-04-24T09:20:29.193 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:28 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:28.792+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.613982+0000 front 2024-04-24T09:19:38.614541+0000 (oldest deadline 2024-04-24T09:20:00.913987+0000) 2024-04-24T09:20:29.193 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:28 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:28.792+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:49.018457+0000 front 2024-04-24T09:19:49.018520+0000 (oldest deadline 2024-04-24T09:20:16.019822+0000) 2024-04-24T09:20:29.293 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 24 09:20:29 smithi083 podman[131912]: 2024-04-24 09:20:29.126072255 +0000 UTC m=+6.218714268 container cleanup 55e96ecb5117728cb4a5672be281e0a7b85cd4614bc8b7506c1d06a7fa543712 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-6, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2) 2024-04-24T09:20:29.293 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 24 09:20:29 smithi083 bash[131912]: ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-6 2024-04-24T09:20:29.457 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:29 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:29.190+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:37.707135+0000 front 2024-04-24T09:19:37.707028+0000 (oldest deadline 2024-04-24T09:20:03.607224+0000) 2024-04-24T09:20:29.457 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:29 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:29.190+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:48.713889+0000 front 2024-04-24T09:19:48.713979+0000 (oldest deadline 2024-04-24T09:20:13.414134+0000) 2024-04-24T09:20:29.457 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:29 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:29.190+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:58.715994+0000 front 2024-04-24T09:19:58.716101+0000 (oldest deadline 2024-04-24T09:20:22.216080+0000) 2024-04-24T09:20:29.553 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 24 09:20:29 smithi083 podman[131944]: 2024-04-24 09:20:29.402179252 +0000 UTC m=+0.735591261 container remove 55e96ecb5117728cb4a5672be281e0a7b85cd4614bc8b7506c1d06a7fa543712 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-6, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240424) 2024-04-24T09:20:29.553 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:29 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:29.292+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:27.386588+0000 front 2024-04-24T09:19:27.386891+0000 (oldest deadline 2024-04-24T09:19:52.686559+0000) 2024-04-24T09:20:29.553 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:29 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:29.292+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.891162+0000 front 2024-04-24T09:19:38.891270+0000 (oldest deadline 2024-04-24T09:20:04.790877+0000) 2024-04-24T09:20:29.553 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:29 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:29.292+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:48.794838+0000 front 2024-04-24T09:19:48.794972+0000 (oldest deadline 2024-04-24T09:20:14.095077+0000) 2024-04-24T09:20:29.553 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:29 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:29.292+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:56.997332+0000 front 2024-04-24T09:19:56.997477+0000 (oldest deadline 2024-04-24T09:20:22.897608+0000) 2024-04-24T09:20:29.955 INFO:journalctl@ceph.osd.7.smithi138.stdout:Apr 24 09:20:29 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-7[49166]: 2024-04-24T09:20:29.780+0000 7fb49b200700 -1 osd.7 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:37.463919+0000 front 2024-04-24T09:19:37.464057+0000 (oldest deadline 2024-04-24T09:20:03.865663+0000) 2024-04-24T09:20:29.955 INFO:journalctl@ceph.osd.7.smithi138.stdout:Apr 24 09:20:29 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-7[49166]: 2024-04-24T09:20:29.780+0000 7fb49b200700 -1 osd.7 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:47.968238+0000 front 2024-04-24T09:19:47.968624+0000 (oldest deadline 2024-04-24T09:20:13.868496+0000) 2024-04-24T09:20:29.955 INFO:journalctl@ceph.osd.7.smithi138.stdout:Apr 24 09:20:29 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-7[49166]: 2024-04-24T09:20:29.780+0000 7fb49b200700 -1 osd.7 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:58.373589+0000 front 2024-04-24T09:19:58.373837+0000 (oldest deadline 2024-04-24T09:20:23.073323+0000) 2024-04-24T09:20:29.955 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:29 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:29.640+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:29.414806+0000 front 2024-04-24T09:19:29.414861+0000 (oldest deadline 2024-04-24T09:19:55.314800+0000) 2024-04-24T09:20:29.955 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:29 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:29.640+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.119965+0000 front 2024-04-24T09:19:38.120034+0000 (oldest deadline 2024-04-24T09:20:02.820181+0000) 2024-04-24T09:20:29.956 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:29 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:29.640+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:47.326279+0000 front 2024-04-24T09:19:47.326338+0000 (oldest deadline 2024-04-24T09:20:10.728055+0000) 2024-04-24T09:20:29.956 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:29 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:29.640+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:55.832608+0000 front 2024-04-24T09:19:55.832510+0000 (oldest deadline 2024-04-24T09:20:21.732750+0000) 2024-04-24T09:20:30.207 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:29 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:29.793+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:28.209530+0000 front 2024-04-24T09:19:28.209142+0000 (oldest deadline 2024-04-24T09:19:51.709380+0000) 2024-04-24T09:20:30.208 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:29 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:29.793+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.613982+0000 front 2024-04-24T09:19:38.614541+0000 (oldest deadline 2024-04-24T09:20:00.913987+0000) 2024-04-24T09:20:30.208 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:29 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:29.793+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:49.018457+0000 front 2024-04-24T09:19:49.018520+0000 (oldest deadline 2024-04-24T09:20:16.019822+0000) 2024-04-24T09:20:30.465 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 24 09:20:30 smithi083 podman[132032]: 2024-04-24 09:20:30.105091891 +0000 UTC m=+0.032268010 image pull b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644 2024-04-24T09:20:30.465 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 24 09:20:30 smithi083 podman[132032]: 2024-04-24 09:20:30.203176656 +0000 UTC m=+0.130352773 container create 244ccd4ed78691282eccd6f1bf83a48b554c9eeb60946d35bea9fad7f8d01f4f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, RELEASE=rishabh-main-apr17-a654945-1f81fb3, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-04-24T09:20:30.465 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 24 09:20:30 smithi083 podman[132032]: 2024-04-24 09:20:30.465466558 +0000 UTC m=+0.392642724 container init 244ccd4ed78691282eccd6f1bf83a48b554c9eeb60946d35bea9fad7f8d01f4f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-6-deactivate, GIT_CLEAN=True, RELEASE=rishabh-main-apr17-a654945-1f81fb3, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-04-24T09:20:30.466 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:30 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:30.290+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:27.386588+0000 front 2024-04-24T09:19:27.386891+0000 (oldest deadline 2024-04-24T09:19:52.686559+0000) 2024-04-24T09:20:30.466 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:30 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:30.290+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.891162+0000 front 2024-04-24T09:19:38.891270+0000 (oldest deadline 2024-04-24T09:20:04.790877+0000) 2024-04-24T09:20:30.466 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:30 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:30.290+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:48.794838+0000 front 2024-04-24T09:19:48.794972+0000 (oldest deadline 2024-04-24T09:20:14.095077+0000) 2024-04-24T09:20:30.466 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:30 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:30.290+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:56.997332+0000 front 2024-04-24T09:19:56.997477+0000 (oldest deadline 2024-04-24T09:20:22.897608+0000) 2024-04-24T09:20:30.707 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:30 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:30.219+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:37.707135+0000 front 2024-04-24T09:19:37.707028+0000 (oldest deadline 2024-04-24T09:20:03.607224+0000) 2024-04-24T09:20:30.707 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:30 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:30.219+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:48.713889+0000 front 2024-04-24T09:19:48.713979+0000 (oldest deadline 2024-04-24T09:20:13.414134+0000) 2024-04-24T09:20:30.707 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:30 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:30.219+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:58.715994+0000 front 2024-04-24T09:19:58.716101+0000 (oldest deadline 2024-04-24T09:20:22.216080+0000) 2024-04-24T09:20:30.803 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 24 09:20:30 smithi083 podman[132032]: 2024-04-24 09:20:30.477334075 +0000 UTC m=+0.404510261 container start 244ccd4ed78691282eccd6f1bf83a48b554c9eeb60946d35bea9fad7f8d01f4f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=rishabh-main-apr17-a654945-1f81fb3, ceph=True) 2024-04-24T09:20:30.803 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 24 09:20:30 smithi083 podman[132032]: 2024-04-24 09:20:30.564321547 +0000 UTC m=+0.491497750 container attach 244ccd4ed78691282eccd6f1bf83a48b554c9eeb60946d35bea9fad7f8d01f4f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-6-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-04-24T09:20:30.955 INFO:journalctl@ceph.osd.7.smithi138.stdout:Apr 24 09:20:30 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-7[49166]: 2024-04-24T09:20:30.781+0000 7fb49b200700 -1 osd.7 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:37.463919+0000 front 2024-04-24T09:19:37.464057+0000 (oldest deadline 2024-04-24T09:20:03.865663+0000) 2024-04-24T09:20:30.955 INFO:journalctl@ceph.osd.7.smithi138.stdout:Apr 24 09:20:30 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-7[49166]: 2024-04-24T09:20:30.781+0000 7fb49b200700 -1 osd.7 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:47.968238+0000 front 2024-04-24T09:19:47.968624+0000 (oldest deadline 2024-04-24T09:20:13.868496+0000) 2024-04-24T09:20:30.955 INFO:journalctl@ceph.osd.7.smithi138.stdout:Apr 24 09:20:30 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-7[49166]: 2024-04-24T09:20:30.781+0000 7fb49b200700 -1 osd.7 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:58.373589+0000 front 2024-04-24T09:19:58.373837+0000 (oldest deadline 2024-04-24T09:20:23.073323+0000) 2024-04-24T09:20:30.955 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:30 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:30.652+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:29.414806+0000 front 2024-04-24T09:19:29.414861+0000 (oldest deadline 2024-04-24T09:19:55.314800+0000) 2024-04-24T09:20:30.955 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:30 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:30.652+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.119965+0000 front 2024-04-24T09:19:38.120034+0000 (oldest deadline 2024-04-24T09:20:02.820181+0000) 2024-04-24T09:20:30.955 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:30 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:30.652+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:47.326279+0000 front 2024-04-24T09:19:47.326338+0000 (oldest deadline 2024-04-24T09:20:10.728055+0000) 2024-04-24T09:20:30.955 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:30 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:30.652+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:55.832608+0000 front 2024-04-24T09:19:55.832510+0000 (oldest deadline 2024-04-24T09:20:21.732750+0000) 2024-04-24T09:20:31.181 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:30 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:30.771+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:28.209530+0000 front 2024-04-24T09:19:28.209142+0000 (oldest deadline 2024-04-24T09:19:51.709380+0000) 2024-04-24T09:20:31.182 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:30 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:30.771+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.613982+0000 front 2024-04-24T09:19:38.614541+0000 (oldest deadline 2024-04-24T09:20:00.913987+0000) 2024-04-24T09:20:31.182 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:30 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:30.771+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:49.018457+0000 front 2024-04-24T09:19:49.018520+0000 (oldest deadline 2024-04-24T09:20:16.019822+0000) 2024-04-24T09:20:31.276 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 24 09:20:30 smithi083 podman[132032]: 2024-04-24 09:20:30.921833878 +0000 UTC m=+0.849010060 container died 244ccd4ed78691282eccd6f1bf83a48b554c9eeb60946d35bea9fad7f8d01f4f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.build-date=20240424, ceph=True, RELEASE=rishabh-main-apr17-a654945-1f81fb3, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-04-24T09:20:31.457 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:31 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:31.181+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:37.707135+0000 front 2024-04-24T09:19:37.707028+0000 (oldest deadline 2024-04-24T09:20:03.607224+0000) 2024-04-24T09:20:31.457 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:31 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:31.181+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:48.713889+0000 front 2024-04-24T09:19:48.713979+0000 (oldest deadline 2024-04-24T09:20:13.414134+0000) 2024-04-24T09:20:31.457 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:31 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:31.181+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:58.715994+0000 front 2024-04-24T09:19:58.716101+0000 (oldest deadline 2024-04-24T09:20:22.216080+0000) 2024-04-24T09:20:31.457 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:31 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:31.181+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:05.020499+0000 front 2024-04-24T09:20:05.020565+0000 (oldest deadline 2024-04-24T09:20:30.822569+0000) 2024-04-24T09:20:31.553 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:31 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:31.275+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:27.386588+0000 front 2024-04-24T09:19:27.386891+0000 (oldest deadline 2024-04-24T09:19:52.686559+0000) 2024-04-24T09:20:31.553 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:31 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:31.275+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.891162+0000 front 2024-04-24T09:19:38.891270+0000 (oldest deadline 2024-04-24T09:20:04.790877+0000) 2024-04-24T09:20:31.553 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:31 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:31.275+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:48.794838+0000 front 2024-04-24T09:19:48.794972+0000 (oldest deadline 2024-04-24T09:20:14.095077+0000) 2024-04-24T09:20:31.554 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:31 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:31.275+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:56.997332+0000 front 2024-04-24T09:19:56.997477+0000 (oldest deadline 2024-04-24T09:20:22.897608+0000) 2024-04-24T09:20:31.554 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:31 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:31.275+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:06.400442+0000 front 2024-04-24T09:20:06.400694+0000 (oldest deadline 2024-04-24T09:20:31.099904+0000) 2024-04-24T09:20:31.806 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 24 09:20:31 smithi083 podman[132058]: 2024-04-24 09:20:31.767906067 +0000 UTC m=+0.827655356 container remove 244ccd4ed78691282eccd6f1bf83a48b554c9eeb60946d35bea9fad7f8d01f4f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-6-deactivate, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-24T09:20:31.806 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 24 09:20:31 smithi083 systemd[1]: ceph-bd3e039e-0215-11ef-bc93-c7b262605968@osd.6.service: Deactivated successfully. 2024-04-24T09:20:31.819 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bd3e039e-0215-11ef-bc93-c7b262605968@osd.6.service' 2024-04-24T09:20:31.931 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T09:20:31.931 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-04-24T09:20:31.931 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-04-24T09:20:31.932 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ceph-bd3e039e-0215-11ef-bc93-c7b262605968@osd.7 2024-04-24T09:20:31.954 INFO:journalctl@ceph.osd.7.smithi138.stdout:Apr 24 09:20:31 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-7[49166]: 2024-04-24T09:20:31.800+0000 7fb49b200700 -1 osd.7 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:37.463919+0000 front 2024-04-24T09:19:37.464057+0000 (oldest deadline 2024-04-24T09:20:03.865663+0000) 2024-04-24T09:20:31.955 INFO:journalctl@ceph.osd.7.smithi138.stdout:Apr 24 09:20:31 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-7[49166]: 2024-04-24T09:20:31.800+0000 7fb49b200700 -1 osd.7 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:47.968238+0000 front 2024-04-24T09:19:47.968624+0000 (oldest deadline 2024-04-24T09:20:13.868496+0000) 2024-04-24T09:20:31.955 INFO:journalctl@ceph.osd.7.smithi138.stdout:Apr 24 09:20:31 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-7[49166]: 2024-04-24T09:20:31.800+0000 7fb49b200700 -1 osd.7 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:58.373589+0000 front 2024-04-24T09:19:58.373837+0000 (oldest deadline 2024-04-24T09:20:23.073323+0000) 2024-04-24T09:20:31.955 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:31 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:31.661+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:29.414806+0000 front 2024-04-24T09:19:29.414861+0000 (oldest deadline 2024-04-24T09:19:55.314800+0000) 2024-04-24T09:20:31.955 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:31 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:31.661+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.119965+0000 front 2024-04-24T09:19:38.120034+0000 (oldest deadline 2024-04-24T09:20:02.820181+0000) 2024-04-24T09:20:31.955 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:31 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:31.661+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:47.326279+0000 front 2024-04-24T09:19:47.326338+0000 (oldest deadline 2024-04-24T09:20:10.728055+0000) 2024-04-24T09:20:31.955 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:31 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:31.661+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:55.832608+0000 front 2024-04-24T09:19:55.832510+0000 (oldest deadline 2024-04-24T09:20:21.732750+0000) 2024-04-24T09:20:31.955 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:31 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:31.661+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:08.736362+0000 front 2024-04-24T09:20:08.736680+0000 (oldest deadline 2024-04-24T09:20:31.537874+0000) 2024-04-24T09:20:32.199 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:31 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:31.806+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:28.209530+0000 front 2024-04-24T09:19:28.209142+0000 (oldest deadline 2024-04-24T09:19:51.709380+0000) 2024-04-24T09:20:32.199 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:31 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:31.806+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.613982+0000 front 2024-04-24T09:19:38.614541+0000 (oldest deadline 2024-04-24T09:20:00.913987+0000) 2024-04-24T09:20:32.199 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:31 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:31.806+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:49.018457+0000 front 2024-04-24T09:19:49.018520+0000 (oldest deadline 2024-04-24T09:20:16.019822+0000) 2024-04-24T09:20:32.199 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:31 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:31.806+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:07.025175+0000 front 2024-04-24T09:20:07.025338+0000 (oldest deadline 2024-04-24T09:20:31.125415+0000) 2024-04-24T09:20:32.274 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:32 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:32.273+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:27.386588+0000 front 2024-04-24T09:19:27.386891+0000 (oldest deadline 2024-04-24T09:19:52.686559+0000) 2024-04-24T09:20:32.274 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:32 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:32.273+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.891162+0000 front 2024-04-24T09:19:38.891270+0000 (oldest deadline 2024-04-24T09:20:04.790877+0000) 2024-04-24T09:20:32.274 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:32 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:32.273+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:48.794838+0000 front 2024-04-24T09:19:48.794972+0000 (oldest deadline 2024-04-24T09:20:14.095077+0000) 2024-04-24T09:20:32.274 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:32 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:32.273+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:56.997332+0000 front 2024-04-24T09:19:56.997477+0000 (oldest deadline 2024-04-24T09:20:22.897608+0000) 2024-04-24T09:20:32.274 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:32 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:32.273+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:06.400442+0000 front 2024-04-24T09:20:06.400694+0000 (oldest deadline 2024-04-24T09:20:31.099904+0000) 2024-04-24T09:20:32.455 INFO:journalctl@ceph.osd.7.smithi138.stdout:Apr 24 09:20:32 smithi138 systemd[1]: Stopping Ceph osd.7 for bd3e039e-0215-11ef-bc93-c7b262605968... 2024-04-24T09:20:32.457 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:32 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:32.198+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:37.707135+0000 front 2024-04-24T09:19:37.707028+0000 (oldest deadline 2024-04-24T09:20:03.607224+0000) 2024-04-24T09:20:32.457 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:32 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:32.198+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:48.713889+0000 front 2024-04-24T09:19:48.713979+0000 (oldest deadline 2024-04-24T09:20:13.414134+0000) 2024-04-24T09:20:32.457 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:32 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:32.198+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:58.715994+0000 front 2024-04-24T09:19:58.716101+0000 (oldest deadline 2024-04-24T09:20:22.216080+0000) 2024-04-24T09:20:32.457 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:32 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:32.198+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:05.020499+0000 front 2024-04-24T09:20:05.020565+0000 (oldest deadline 2024-04-24T09:20:30.822569+0000) 2024-04-24T09:20:32.836 INFO:journalctl@ceph.osd.7.smithi138.stdout:Apr 24 09:20:32 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-7[49166]: 2024-04-24T09:20:32.493+0000 7fb4a4a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T09:20:32.836 INFO:journalctl@ceph.osd.7.smithi138.stdout:Apr 24 09:20:32 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-7[49166]: 2024-04-24T09:20:32.493+0000 7fb4a4a00700 -1 osd.7 141 *** Got signal Terminated *** 2024-04-24T09:20:32.836 INFO:journalctl@ceph.osd.7.smithi138.stdout:Apr 24 09:20:32 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-7[49166]: 2024-04-24T09:20:32.493+0000 7fb4a4a00700 -1 osd.7 141 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T09:20:32.837 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:32 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:32.693+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:29.414806+0000 front 2024-04-24T09:19:29.414861+0000 (oldest deadline 2024-04-24T09:19:55.314800+0000) 2024-04-24T09:20:32.837 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:32 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:32.693+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.119965+0000 front 2024-04-24T09:19:38.120034+0000 (oldest deadline 2024-04-24T09:20:02.820181+0000) 2024-04-24T09:20:32.837 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:32 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:32.693+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:47.326279+0000 front 2024-04-24T09:19:47.326338+0000 (oldest deadline 2024-04-24T09:20:10.728055+0000) 2024-04-24T09:20:32.837 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:32 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:32.693+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:55.832608+0000 front 2024-04-24T09:19:55.832510+0000 (oldest deadline 2024-04-24T09:20:21.732750+0000) 2024-04-24T09:20:32.837 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:32 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:32.693+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:08.736362+0000 front 2024-04-24T09:20:08.736680+0000 (oldest deadline 2024-04-24T09:20:31.537874+0000) 2024-04-24T09:20:33.205 INFO:journalctl@ceph.osd.7.smithi138.stdout:Apr 24 09:20:32 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-7[49166]: 2024-04-24T09:20:32.836+0000 7fb49b200700 -1 osd.7 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:37.463919+0000 front 2024-04-24T09:19:37.464057+0000 (oldest deadline 2024-04-24T09:20:03.865663+0000) 2024-04-24T09:20:33.205 INFO:journalctl@ceph.osd.7.smithi138.stdout:Apr 24 09:20:32 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-7[49166]: 2024-04-24T09:20:32.836+0000 7fb49b200700 -1 osd.7 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:47.968238+0000 front 2024-04-24T09:19:47.968624+0000 (oldest deadline 2024-04-24T09:20:13.868496+0000) 2024-04-24T09:20:33.205 INFO:journalctl@ceph.osd.7.smithi138.stdout:Apr 24 09:20:32 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-7[49166]: 2024-04-24T09:20:32.836+0000 7fb49b200700 -1 osd.7 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:58.373589+0000 front 2024-04-24T09:19:58.373837+0000 (oldest deadline 2024-04-24T09:20:23.073323+0000) 2024-04-24T09:20:33.205 INFO:journalctl@ceph.osd.7.smithi138.stdout:Apr 24 09:20:32 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-7[49166]: 2024-04-24T09:20:32.836+0000 7fb49b200700 -1 osd.7 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:07.174896+0000 front 2024-04-24T09:20:07.175063+0000 (oldest deadline 2024-04-24T09:20:31.874738+0000) 2024-04-24T09:20:33.207 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:32 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:32.777+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:28.209530+0000 front 2024-04-24T09:19:28.209142+0000 (oldest deadline 2024-04-24T09:19:51.709380+0000) 2024-04-24T09:20:33.207 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:32 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:32.777+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.613982+0000 front 2024-04-24T09:19:38.614541+0000 (oldest deadline 2024-04-24T09:20:00.913987+0000) 2024-04-24T09:20:33.207 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:32 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:32.777+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:49.018457+0000 front 2024-04-24T09:19:49.018520+0000 (oldest deadline 2024-04-24T09:20:16.019822+0000) 2024-04-24T09:20:33.207 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:32 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:32.777+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:07.025175+0000 front 2024-04-24T09:20:07.025338+0000 (oldest deadline 2024-04-24T09:20:31.125415+0000) 2024-04-24T09:20:33.520 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:33 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:33.243+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:37.707135+0000 front 2024-04-24T09:19:37.707028+0000 (oldest deadline 2024-04-24T09:20:03.607224+0000) 2024-04-24T09:20:33.520 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:33 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:33.243+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:48.713889+0000 front 2024-04-24T09:19:48.713979+0000 (oldest deadline 2024-04-24T09:20:13.414134+0000) 2024-04-24T09:20:33.520 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:33 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:33.243+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:58.715994+0000 front 2024-04-24T09:19:58.716101+0000 (oldest deadline 2024-04-24T09:20:22.216080+0000) 2024-04-24T09:20:33.520 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:33 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:33.243+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:05.020499+0000 front 2024-04-24T09:20:05.020565+0000 (oldest deadline 2024-04-24T09:20:30.822569+0000) 2024-04-24T09:20:33.553 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:33 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:33.266+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:27.386588+0000 front 2024-04-24T09:19:27.386891+0000 (oldest deadline 2024-04-24T09:19:52.686559+0000) 2024-04-24T09:20:33.553 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:33 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:33.266+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.891162+0000 front 2024-04-24T09:19:38.891270+0000 (oldest deadline 2024-04-24T09:20:04.790877+0000) 2024-04-24T09:20:33.553 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:33 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:33.266+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:48.794838+0000 front 2024-04-24T09:19:48.794972+0000 (oldest deadline 2024-04-24T09:20:14.095077+0000) 2024-04-24T09:20:33.553 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:33 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:33.266+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:56.997332+0000 front 2024-04-24T09:19:56.997477+0000 (oldest deadline 2024-04-24T09:20:22.897608+0000) 2024-04-24T09:20:33.553 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:33 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:33.266+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:06.400442+0000 front 2024-04-24T09:20:06.400694+0000 (oldest deadline 2024-04-24T09:20:31.099904+0000) 2024-04-24T09:20:33.955 INFO:journalctl@ceph.osd.7.smithi138.stdout:Apr 24 09:20:33 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-7[49166]: 2024-04-24T09:20:33.811+0000 7fb49b200700 -1 osd.7 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:37.463919+0000 front 2024-04-24T09:19:37.464057+0000 (oldest deadline 2024-04-24T09:20:03.865663+0000) 2024-04-24T09:20:33.955 INFO:journalctl@ceph.osd.7.smithi138.stdout:Apr 24 09:20:33 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-7[49166]: 2024-04-24T09:20:33.811+0000 7fb49b200700 -1 osd.7 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:47.968238+0000 front 2024-04-24T09:19:47.968624+0000 (oldest deadline 2024-04-24T09:20:13.868496+0000) 2024-04-24T09:20:33.955 INFO:journalctl@ceph.osd.7.smithi138.stdout:Apr 24 09:20:33 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-7[49166]: 2024-04-24T09:20:33.811+0000 7fb49b200700 -1 osd.7 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:58.373589+0000 front 2024-04-24T09:19:58.373837+0000 (oldest deadline 2024-04-24T09:20:23.073323+0000) 2024-04-24T09:20:33.955 INFO:journalctl@ceph.osd.7.smithi138.stdout:Apr 24 09:20:33 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-7[49166]: 2024-04-24T09:20:33.811+0000 7fb49b200700 -1 osd.7 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:07.174896+0000 front 2024-04-24T09:20:07.175063+0000 (oldest deadline 2024-04-24T09:20:31.874738+0000) 2024-04-24T09:20:33.956 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:33 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:33.672+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:29.414806+0000 front 2024-04-24T09:19:29.414861+0000 (oldest deadline 2024-04-24T09:19:55.314800+0000) 2024-04-24T09:20:33.956 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:33 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:33.672+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.119965+0000 front 2024-04-24T09:19:38.120034+0000 (oldest deadline 2024-04-24T09:20:02.820181+0000) 2024-04-24T09:20:33.956 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:33 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:33.672+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:47.326279+0000 front 2024-04-24T09:19:47.326338+0000 (oldest deadline 2024-04-24T09:20:10.728055+0000) 2024-04-24T09:20:33.956 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:33 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:33.672+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:55.832608+0000 front 2024-04-24T09:19:55.832510+0000 (oldest deadline 2024-04-24T09:20:21.732750+0000) 2024-04-24T09:20:33.956 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:33 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:33.672+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:08.736362+0000 front 2024-04-24T09:20:08.736680+0000 (oldest deadline 2024-04-24T09:20:31.537874+0000) 2024-04-24T09:20:34.207 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:33 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:33.810+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:28.209530+0000 front 2024-04-24T09:19:28.209142+0000 (oldest deadline 2024-04-24T09:19:51.709380+0000) 2024-04-24T09:20:34.207 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:33 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:33.810+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.613982+0000 front 2024-04-24T09:19:38.614541+0000 (oldest deadline 2024-04-24T09:20:00.913987+0000) 2024-04-24T09:20:34.207 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:33 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:33.810+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:49.018457+0000 front 2024-04-24T09:19:49.018520+0000 (oldest deadline 2024-04-24T09:20:16.019822+0000) 2024-04-24T09:20:34.207 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:33 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:33.810+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:07.025175+0000 front 2024-04-24T09:20:07.025338+0000 (oldest deadline 2024-04-24T09:20:31.125415+0000) 2024-04-24T09:20:34.553 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:34 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:34.299+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:27.386588+0000 front 2024-04-24T09:19:27.386891+0000 (oldest deadline 2024-04-24T09:19:52.686559+0000) 2024-04-24T09:20:34.553 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:34 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:34.299+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.891162+0000 front 2024-04-24T09:19:38.891270+0000 (oldest deadline 2024-04-24T09:20:04.790877+0000) 2024-04-24T09:20:34.554 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:34 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:34.299+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:48.794838+0000 front 2024-04-24T09:19:48.794972+0000 (oldest deadline 2024-04-24T09:20:14.095077+0000) 2024-04-24T09:20:34.554 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:34 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:34.299+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:56.997332+0000 front 2024-04-24T09:19:56.997477+0000 (oldest deadline 2024-04-24T09:20:22.897608+0000) 2024-04-24T09:20:34.554 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:34 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:34.299+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:06.400442+0000 front 2024-04-24T09:20:06.400694+0000 (oldest deadline 2024-04-24T09:20:31.099904+0000) 2024-04-24T09:20:34.707 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:34 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:34.292+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:37.707135+0000 front 2024-04-24T09:19:37.707028+0000 (oldest deadline 2024-04-24T09:20:03.607224+0000) 2024-04-24T09:20:34.707 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:34 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:34.292+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:48.713889+0000 front 2024-04-24T09:19:48.713979+0000 (oldest deadline 2024-04-24T09:20:13.414134+0000) 2024-04-24T09:20:34.707 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:34 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:34.292+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:58.715994+0000 front 2024-04-24T09:19:58.716101+0000 (oldest deadline 2024-04-24T09:20:22.216080+0000) 2024-04-24T09:20:34.707 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:34 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:34.292+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:05.020499+0000 front 2024-04-24T09:20:05.020565+0000 (oldest deadline 2024-04-24T09:20:30.822569+0000) 2024-04-24T09:20:34.955 INFO:journalctl@ceph.osd.7.smithi138.stdout:Apr 24 09:20:34 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-7[49166]: 2024-04-24T09:20:34.769+0000 7fb49b200700 -1 osd.7 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:37.463919+0000 front 2024-04-24T09:19:37.464057+0000 (oldest deadline 2024-04-24T09:20:03.865663+0000) 2024-04-24T09:20:34.955 INFO:journalctl@ceph.osd.7.smithi138.stdout:Apr 24 09:20:34 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-7[49166]: 2024-04-24T09:20:34.769+0000 7fb49b200700 -1 osd.7 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:47.968238+0000 front 2024-04-24T09:19:47.968624+0000 (oldest deadline 2024-04-24T09:20:13.868496+0000) 2024-04-24T09:20:34.955 INFO:journalctl@ceph.osd.7.smithi138.stdout:Apr 24 09:20:34 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-7[49166]: 2024-04-24T09:20:34.769+0000 7fb49b200700 -1 osd.7 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:58.373589+0000 front 2024-04-24T09:19:58.373837+0000 (oldest deadline 2024-04-24T09:20:23.073323+0000) 2024-04-24T09:20:34.955 INFO:journalctl@ceph.osd.7.smithi138.stdout:Apr 24 09:20:34 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-7[49166]: 2024-04-24T09:20:34.769+0000 7fb49b200700 -1 osd.7 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:07.174896+0000 front 2024-04-24T09:20:07.175063+0000 (oldest deadline 2024-04-24T09:20:31.874738+0000) 2024-04-24T09:20:34.956 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:34 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:34.666+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:29.414806+0000 front 2024-04-24T09:19:29.414861+0000 (oldest deadline 2024-04-24T09:19:55.314800+0000) 2024-04-24T09:20:34.956 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:34 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:34.666+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.119965+0000 front 2024-04-24T09:19:38.120034+0000 (oldest deadline 2024-04-24T09:20:02.820181+0000) 2024-04-24T09:20:34.956 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:34 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:34.666+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:47.326279+0000 front 2024-04-24T09:19:47.326338+0000 (oldest deadline 2024-04-24T09:20:10.728055+0000) 2024-04-24T09:20:34.956 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:34 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:34.666+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:55.832608+0000 front 2024-04-24T09:19:55.832510+0000 (oldest deadline 2024-04-24T09:20:21.732750+0000) 2024-04-24T09:20:34.956 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:34 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:34.666+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:08.736362+0000 front 2024-04-24T09:20:08.736680+0000 (oldest deadline 2024-04-24T09:20:31.537874+0000) 2024-04-24T09:20:35.207 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:34 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:34.808+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:28.209530+0000 front 2024-04-24T09:19:28.209142+0000 (oldest deadline 2024-04-24T09:19:51.709380+0000) 2024-04-24T09:20:35.207 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:34 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:34.808+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.613982+0000 front 2024-04-24T09:19:38.614541+0000 (oldest deadline 2024-04-24T09:20:00.913987+0000) 2024-04-24T09:20:35.207 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:34 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:34.808+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:49.018457+0000 front 2024-04-24T09:19:49.018520+0000 (oldest deadline 2024-04-24T09:20:16.019822+0000) 2024-04-24T09:20:35.208 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:34 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:34.808+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:07.025175+0000 front 2024-04-24T09:20:07.025338+0000 (oldest deadline 2024-04-24T09:20:31.125415+0000) 2024-04-24T09:20:35.553 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:35 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:35.257+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:27.386588+0000 front 2024-04-24T09:19:27.386891+0000 (oldest deadline 2024-04-24T09:19:52.686559+0000) 2024-04-24T09:20:35.553 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:35 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:35.257+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.891162+0000 front 2024-04-24T09:19:38.891270+0000 (oldest deadline 2024-04-24T09:20:04.790877+0000) 2024-04-24T09:20:35.553 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:35 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:35.257+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:48.794838+0000 front 2024-04-24T09:19:48.794972+0000 (oldest deadline 2024-04-24T09:20:14.095077+0000) 2024-04-24T09:20:35.554 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:35 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:35.257+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:56.997332+0000 front 2024-04-24T09:19:56.997477+0000 (oldest deadline 2024-04-24T09:20:22.897608+0000) 2024-04-24T09:20:35.554 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:35 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:35.257+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:06.400442+0000 front 2024-04-24T09:20:06.400694+0000 (oldest deadline 2024-04-24T09:20:31.099904+0000) 2024-04-24T09:20:35.707 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:35 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:35.286+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:37.707135+0000 front 2024-04-24T09:19:37.707028+0000 (oldest deadline 2024-04-24T09:20:03.607224+0000) 2024-04-24T09:20:35.707 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:35 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:35.286+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:48.713889+0000 front 2024-04-24T09:19:48.713979+0000 (oldest deadline 2024-04-24T09:20:13.414134+0000) 2024-04-24T09:20:35.708 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:35 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:35.286+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:58.715994+0000 front 2024-04-24T09:19:58.716101+0000 (oldest deadline 2024-04-24T09:20:22.216080+0000) 2024-04-24T09:20:35.708 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:35 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:35.286+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:05.020499+0000 front 2024-04-24T09:20:05.020565+0000 (oldest deadline 2024-04-24T09:20:30.822569+0000) 2024-04-24T09:20:35.955 INFO:journalctl@ceph.osd.7.smithi138.stdout:Apr 24 09:20:35 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-7[49166]: 2024-04-24T09:20:35.754+0000 7fb49b200700 -1 osd.7 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:37.463919+0000 front 2024-04-24T09:19:37.464057+0000 (oldest deadline 2024-04-24T09:20:03.865663+0000) 2024-04-24T09:20:35.955 INFO:journalctl@ceph.osd.7.smithi138.stdout:Apr 24 09:20:35 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-7[49166]: 2024-04-24T09:20:35.754+0000 7fb49b200700 -1 osd.7 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:47.968238+0000 front 2024-04-24T09:19:47.968624+0000 (oldest deadline 2024-04-24T09:20:13.868496+0000) 2024-04-24T09:20:35.955 INFO:journalctl@ceph.osd.7.smithi138.stdout:Apr 24 09:20:35 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-7[49166]: 2024-04-24T09:20:35.754+0000 7fb49b200700 -1 osd.7 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:58.373589+0000 front 2024-04-24T09:19:58.373837+0000 (oldest deadline 2024-04-24T09:20:23.073323+0000) 2024-04-24T09:20:35.955 INFO:journalctl@ceph.osd.7.smithi138.stdout:Apr 24 09:20:35 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-7[49166]: 2024-04-24T09:20:35.754+0000 7fb49b200700 -1 osd.7 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:07.174896+0000 front 2024-04-24T09:20:07.175063+0000 (oldest deadline 2024-04-24T09:20:31.874738+0000) 2024-04-24T09:20:35.955 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:35 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:35.679+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:29.414806+0000 front 2024-04-24T09:19:29.414861+0000 (oldest deadline 2024-04-24T09:19:55.314800+0000) 2024-04-24T09:20:35.956 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:35 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:35.679+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.119965+0000 front 2024-04-24T09:19:38.120034+0000 (oldest deadline 2024-04-24T09:20:02.820181+0000) 2024-04-24T09:20:35.956 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:35 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:35.679+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:47.326279+0000 front 2024-04-24T09:19:47.326338+0000 (oldest deadline 2024-04-24T09:20:10.728055+0000) 2024-04-24T09:20:35.956 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:35 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:35.679+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:55.832608+0000 front 2024-04-24T09:19:55.832510+0000 (oldest deadline 2024-04-24T09:20:21.732750+0000) 2024-04-24T09:20:35.956 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:35 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:35.679+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:08.736362+0000 front 2024-04-24T09:20:08.736680+0000 (oldest deadline 2024-04-24T09:20:31.537874+0000) 2024-04-24T09:20:36.207 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:35 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:35.832+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:28.209530+0000 front 2024-04-24T09:19:28.209142+0000 (oldest deadline 2024-04-24T09:19:51.709380+0000) 2024-04-24T09:20:36.207 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:35 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:35.832+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.613982+0000 front 2024-04-24T09:19:38.614541+0000 (oldest deadline 2024-04-24T09:20:00.913987+0000) 2024-04-24T09:20:36.207 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:35 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:35.832+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:49.018457+0000 front 2024-04-24T09:19:49.018520+0000 (oldest deadline 2024-04-24T09:20:16.019822+0000) 2024-04-24T09:20:36.207 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:35 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:35.832+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:07.025175+0000 front 2024-04-24T09:20:07.025338+0000 (oldest deadline 2024-04-24T09:20:31.125415+0000) 2024-04-24T09:20:36.553 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:36 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:36.274+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:27.386588+0000 front 2024-04-24T09:19:27.386891+0000 (oldest deadline 2024-04-24T09:19:52.686559+0000) 2024-04-24T09:20:36.553 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:36 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:36.274+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.891162+0000 front 2024-04-24T09:19:38.891270+0000 (oldest deadline 2024-04-24T09:20:04.790877+0000) 2024-04-24T09:20:36.553 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:36 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:36.274+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:48.794838+0000 front 2024-04-24T09:19:48.794972+0000 (oldest deadline 2024-04-24T09:20:14.095077+0000) 2024-04-24T09:20:36.554 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:36 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:36.274+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:56.997332+0000 front 2024-04-24T09:19:56.997477+0000 (oldest deadline 2024-04-24T09:20:22.897608+0000) 2024-04-24T09:20:36.554 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:36 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:36.274+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:06.400442+0000 front 2024-04-24T09:20:06.400694+0000 (oldest deadline 2024-04-24T09:20:31.099904+0000) 2024-04-24T09:20:36.707 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:36 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:36.326+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:37.707135+0000 front 2024-04-24T09:19:37.707028+0000 (oldest deadline 2024-04-24T09:20:03.607224+0000) 2024-04-24T09:20:36.707 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:36 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:36.326+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:48.713889+0000 front 2024-04-24T09:19:48.713979+0000 (oldest deadline 2024-04-24T09:20:13.414134+0000) 2024-04-24T09:20:36.707 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:36 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:36.326+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:58.715994+0000 front 2024-04-24T09:19:58.716101+0000 (oldest deadline 2024-04-24T09:20:22.216080+0000) 2024-04-24T09:20:36.708 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:36 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:36.326+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:05.020499+0000 front 2024-04-24T09:20:05.020565+0000 (oldest deadline 2024-04-24T09:20:30.822569+0000) 2024-04-24T09:20:36.955 INFO:journalctl@ceph.osd.7.smithi138.stdout:Apr 24 09:20:36 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-7[49166]: 2024-04-24T09:20:36.746+0000 7fb49b200700 -1 osd.7 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:37.463919+0000 front 2024-04-24T09:19:37.464057+0000 (oldest deadline 2024-04-24T09:20:03.865663+0000) 2024-04-24T09:20:36.955 INFO:journalctl@ceph.osd.7.smithi138.stdout:Apr 24 09:20:36 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-7[49166]: 2024-04-24T09:20:36.746+0000 7fb49b200700 -1 osd.7 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:47.968238+0000 front 2024-04-24T09:19:47.968624+0000 (oldest deadline 2024-04-24T09:20:13.868496+0000) 2024-04-24T09:20:36.955 INFO:journalctl@ceph.osd.7.smithi138.stdout:Apr 24 09:20:36 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-7[49166]: 2024-04-24T09:20:36.746+0000 7fb49b200700 -1 osd.7 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:58.373589+0000 front 2024-04-24T09:19:58.373837+0000 (oldest deadline 2024-04-24T09:20:23.073323+0000) 2024-04-24T09:20:36.955 INFO:journalctl@ceph.osd.7.smithi138.stdout:Apr 24 09:20:36 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-7[49166]: 2024-04-24T09:20:36.746+0000 7fb49b200700 -1 osd.7 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:07.174896+0000 front 2024-04-24T09:20:07.175063+0000 (oldest deadline 2024-04-24T09:20:31.874738+0000) 2024-04-24T09:20:36.955 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:36 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:36.698+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:29.414806+0000 front 2024-04-24T09:19:29.414861+0000 (oldest deadline 2024-04-24T09:19:55.314800+0000) 2024-04-24T09:20:36.956 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:36 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:36.698+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.119965+0000 front 2024-04-24T09:19:38.120034+0000 (oldest deadline 2024-04-24T09:20:02.820181+0000) 2024-04-24T09:20:36.956 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:36 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:36.698+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:47.326279+0000 front 2024-04-24T09:19:47.326338+0000 (oldest deadline 2024-04-24T09:20:10.728055+0000) 2024-04-24T09:20:36.956 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:36 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:36.698+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:55.832608+0000 front 2024-04-24T09:19:55.832510+0000 (oldest deadline 2024-04-24T09:20:21.732750+0000) 2024-04-24T09:20:36.956 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:36 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:36.698+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:08.736362+0000 front 2024-04-24T09:20:08.736680+0000 (oldest deadline 2024-04-24T09:20:31.537874+0000) 2024-04-24T09:20:37.207 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:36 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:36.874+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:28.209530+0000 front 2024-04-24T09:19:28.209142+0000 (oldest deadline 2024-04-24T09:19:51.709380+0000) 2024-04-24T09:20:37.207 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:36 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:36.874+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.613982+0000 front 2024-04-24T09:19:38.614541+0000 (oldest deadline 2024-04-24T09:20:00.913987+0000) 2024-04-24T09:20:37.207 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:36 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:36.874+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:49.018457+0000 front 2024-04-24T09:19:49.018520+0000 (oldest deadline 2024-04-24T09:20:16.019822+0000) 2024-04-24T09:20:37.207 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:36 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:36.874+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:07.025175+0000 front 2024-04-24T09:20:07.025338+0000 (oldest deadline 2024-04-24T09:20:31.125415+0000) 2024-04-24T09:20:37.553 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:37 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:37.255+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:27.386588+0000 front 2024-04-24T09:19:27.386891+0000 (oldest deadline 2024-04-24T09:19:52.686559+0000) 2024-04-24T09:20:37.553 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:37 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:37.255+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.891162+0000 front 2024-04-24T09:19:38.891270+0000 (oldest deadline 2024-04-24T09:20:04.790877+0000) 2024-04-24T09:20:37.553 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:37 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:37.255+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:48.794838+0000 front 2024-04-24T09:19:48.794972+0000 (oldest deadline 2024-04-24T09:20:14.095077+0000) 2024-04-24T09:20:37.554 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:37 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:37.255+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:56.997332+0000 front 2024-04-24T09:19:56.997477+0000 (oldest deadline 2024-04-24T09:20:22.897608+0000) 2024-04-24T09:20:37.554 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:37 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:37.255+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:06.400442+0000 front 2024-04-24T09:20:06.400694+0000 (oldest deadline 2024-04-24T09:20:31.099904+0000) 2024-04-24T09:20:37.707 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:37 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:37.376+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:37.707135+0000 front 2024-04-24T09:19:37.707028+0000 (oldest deadline 2024-04-24T09:20:03.607224+0000) 2024-04-24T09:20:37.707 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:37 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:37.376+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:48.713889+0000 front 2024-04-24T09:19:48.713979+0000 (oldest deadline 2024-04-24T09:20:13.414134+0000) 2024-04-24T09:20:37.707 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:37 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:37.376+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:58.715994+0000 front 2024-04-24T09:19:58.716101+0000 (oldest deadline 2024-04-24T09:20:22.216080+0000) 2024-04-24T09:20:37.707 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:37 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:37.376+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:05.020499+0000 front 2024-04-24T09:20:05.020565+0000 (oldest deadline 2024-04-24T09:20:30.822569+0000) 2024-04-24T09:20:37.954 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:37 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:37.658+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:29.414806+0000 front 2024-04-24T09:19:29.414861+0000 (oldest deadline 2024-04-24T09:19:55.314800+0000) 2024-04-24T09:20:37.955 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:37 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:37.658+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.119965+0000 front 2024-04-24T09:19:38.120034+0000 (oldest deadline 2024-04-24T09:20:02.820181+0000) 2024-04-24T09:20:37.955 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:37 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:37.658+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:47.326279+0000 front 2024-04-24T09:19:47.326338+0000 (oldest deadline 2024-04-24T09:20:10.728055+0000) 2024-04-24T09:20:37.955 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:37 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:37.658+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:55.832608+0000 front 2024-04-24T09:19:55.832510+0000 (oldest deadline 2024-04-24T09:20:21.732750+0000) 2024-04-24T09:20:37.955 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:37 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:37.658+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:08.736362+0000 front 2024-04-24T09:20:08.736680+0000 (oldest deadline 2024-04-24T09:20:31.537874+0000) 2024-04-24T09:20:38.207 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:37 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:37.901+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:28.209530+0000 front 2024-04-24T09:19:28.209142+0000 (oldest deadline 2024-04-24T09:19:51.709380+0000) 2024-04-24T09:20:38.207 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:37 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:37.901+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.613982+0000 front 2024-04-24T09:19:38.614541+0000 (oldest deadline 2024-04-24T09:20:00.913987+0000) 2024-04-24T09:20:38.207 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:37 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:37.901+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:49.018457+0000 front 2024-04-24T09:19:49.018520+0000 (oldest deadline 2024-04-24T09:20:16.019822+0000) 2024-04-24T09:20:38.208 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:37 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:37.901+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:07.025175+0000 front 2024-04-24T09:20:07.025338+0000 (oldest deadline 2024-04-24T09:20:31.125415+0000) 2024-04-24T09:20:38.342 INFO:journalctl@ceph.osd.7.smithi138.stdout:Apr 24 09:20:38 smithi138 podman[72936]: 2024-04-24 09:20:38.01908059 +0000 UTC m=+5.671580155 container died 0de391613a5a74455c92fec93f03a36a9a19352e7f0dd711f3bd4c679778e15f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-7, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T09:20:38.553 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:38 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:38.256+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:27.386588+0000 front 2024-04-24T09:19:27.386891+0000 (oldest deadline 2024-04-24T09:19:52.686559+0000) 2024-04-24T09:20:38.553 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:38 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:38.256+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.891162+0000 front 2024-04-24T09:19:38.891270+0000 (oldest deadline 2024-04-24T09:20:04.790877+0000) 2024-04-24T09:20:38.553 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:38 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:38.256+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:48.794838+0000 front 2024-04-24T09:19:48.794972+0000 (oldest deadline 2024-04-24T09:20:14.095077+0000) 2024-04-24T09:20:38.553 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:38 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:38.256+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:56.997332+0000 front 2024-04-24T09:19:56.997477+0000 (oldest deadline 2024-04-24T09:20:22.897608+0000) 2024-04-24T09:20:38.554 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:38 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:38.256+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:06.400442+0000 front 2024-04-24T09:20:06.400694+0000 (oldest deadline 2024-04-24T09:20:31.099904+0000) 2024-04-24T09:20:38.703 INFO:journalctl@ceph.osd.7.smithi138.stdout:Apr 24 09:20:38 smithi138 podman[72936]: 2024-04-24 09:20:38.454157649 +0000 UTC m=+6.106657224 container cleanup 0de391613a5a74455c92fec93f03a36a9a19352e7f0dd711f3bd4c679778e15f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-7, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424) 2024-04-24T09:20:38.703 INFO:journalctl@ceph.osd.7.smithi138.stdout:Apr 24 09:20:38 smithi138 bash[72936]: ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-7 2024-04-24T09:20:38.707 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:38 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:38.414+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:37.707135+0000 front 2024-04-24T09:19:37.707028+0000 (oldest deadline 2024-04-24T09:20:03.607224+0000) 2024-04-24T09:20:38.707 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:38 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:38.414+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:48.713889+0000 front 2024-04-24T09:19:48.713979+0000 (oldest deadline 2024-04-24T09:20:13.414134+0000) 2024-04-24T09:20:38.707 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:38 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:38.414+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:58.715994+0000 front 2024-04-24T09:19:58.716101+0000 (oldest deadline 2024-04-24T09:20:22.216080+0000) 2024-04-24T09:20:38.707 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:38 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:38.414+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:05.020499+0000 front 2024-04-24T09:20:05.020565+0000 (oldest deadline 2024-04-24T09:20:30.822569+0000) 2024-04-24T09:20:38.955 INFO:journalctl@ceph.osd.7.smithi138.stdout:Apr 24 09:20:38 smithi138 podman[72955]: 2024-04-24 09:20:38.732406508 +0000 UTC m=+0.701335059 container remove 0de391613a5a74455c92fec93f03a36a9a19352e7f0dd711f3bd4c679778e15f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_BRANCH=HEAD) 2024-04-24T09:20:38.955 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:38 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:38.702+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:29.414806+0000 front 2024-04-24T09:19:29.414861+0000 (oldest deadline 2024-04-24T09:19:55.314800+0000) 2024-04-24T09:20:38.955 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:38 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:38.702+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.119965+0000 front 2024-04-24T09:19:38.120034+0000 (oldest deadline 2024-04-24T09:20:02.820181+0000) 2024-04-24T09:20:38.955 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:38 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:38.702+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:47.326279+0000 front 2024-04-24T09:19:47.326338+0000 (oldest deadline 2024-04-24T09:20:10.728055+0000) 2024-04-24T09:20:38.955 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:38 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:38.702+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:55.832608+0000 front 2024-04-24T09:19:55.832510+0000 (oldest deadline 2024-04-24T09:20:21.732750+0000) 2024-04-24T09:20:38.955 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:38 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:38.702+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:08.736362+0000 front 2024-04-24T09:20:08.736680+0000 (oldest deadline 2024-04-24T09:20:31.537874+0000) 2024-04-24T09:20:39.207 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:38 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:38.891+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:28.209530+0000 front 2024-04-24T09:19:28.209142+0000 (oldest deadline 2024-04-24T09:19:51.709380+0000) 2024-04-24T09:20:39.207 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:38 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:38.891+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.613982+0000 front 2024-04-24T09:19:38.614541+0000 (oldest deadline 2024-04-24T09:20:00.913987+0000) 2024-04-24T09:20:39.207 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:38 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:38.891+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:49.018457+0000 front 2024-04-24T09:19:49.018520+0000 (oldest deadline 2024-04-24T09:20:16.019822+0000) 2024-04-24T09:20:39.207 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:38 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:38.891+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:07.025175+0000 front 2024-04-24T09:20:07.025338+0000 (oldest deadline 2024-04-24T09:20:31.125415+0000) 2024-04-24T09:20:39.553 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:39 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:39.235+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:27.386588+0000 front 2024-04-24T09:19:27.386891+0000 (oldest deadline 2024-04-24T09:19:52.686559+0000) 2024-04-24T09:20:39.553 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:39 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:39.235+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.891162+0000 front 2024-04-24T09:19:38.891270+0000 (oldest deadline 2024-04-24T09:20:04.790877+0000) 2024-04-24T09:20:39.554 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:39 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:39.235+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:48.794838+0000 front 2024-04-24T09:19:48.794972+0000 (oldest deadline 2024-04-24T09:20:14.095077+0000) 2024-04-24T09:20:39.554 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:39 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:39.235+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:56.997332+0000 front 2024-04-24T09:19:56.997477+0000 (oldest deadline 2024-04-24T09:20:22.897608+0000) 2024-04-24T09:20:39.554 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:39 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:39.235+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:06.400442+0000 front 2024-04-24T09:20:06.400694+0000 (oldest deadline 2024-04-24T09:20:31.099904+0000) 2024-04-24T09:20:39.707 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:39 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:39.417+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:37.707135+0000 front 2024-04-24T09:19:37.707028+0000 (oldest deadline 2024-04-24T09:20:03.607224+0000) 2024-04-24T09:20:39.707 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:39 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:39.417+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:48.713889+0000 front 2024-04-24T09:19:48.713979+0000 (oldest deadline 2024-04-24T09:20:13.414134+0000) 2024-04-24T09:20:39.707 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:39 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:39.417+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:58.715994+0000 front 2024-04-24T09:19:58.716101+0000 (oldest deadline 2024-04-24T09:20:22.216080+0000) 2024-04-24T09:20:39.707 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:39 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:39.417+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:05.020499+0000 front 2024-04-24T09:20:05.020565+0000 (oldest deadline 2024-04-24T09:20:30.822569+0000) 2024-04-24T09:20:39.818 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:39 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:39.729+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:29.414806+0000 front 2024-04-24T09:19:29.414861+0000 (oldest deadline 2024-04-24T09:19:55.314800+0000) 2024-04-24T09:20:39.818 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:39 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:39.729+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.119965+0000 front 2024-04-24T09:19:38.120034+0000 (oldest deadline 2024-04-24T09:20:02.820181+0000) 2024-04-24T09:20:39.818 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:39 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:39.729+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:47.326279+0000 front 2024-04-24T09:19:47.326338+0000 (oldest deadline 2024-04-24T09:20:10.728055+0000) 2024-04-24T09:20:39.818 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:39 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:39.729+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:55.832608+0000 front 2024-04-24T09:19:55.832510+0000 (oldest deadline 2024-04-24T09:20:21.732750+0000) 2024-04-24T09:20:39.818 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:39 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:39.729+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:08.736362+0000 front 2024-04-24T09:20:08.736680+0000 (oldest deadline 2024-04-24T09:20:31.537874+0000) 2024-04-24T09:20:39.818 INFO:journalctl@ceph.osd.7.smithi138.stdout:Apr 24 09:20:39 smithi138 podman[73044]: 2024-04-24 09:20:39.473390557 +0000 UTC m=+0.035152479 image pull b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644 2024-04-24T09:20:39.818 INFO:journalctl@ceph.osd.7.smithi138.stdout:Apr 24 09:20:39 smithi138 podman[73044]: 2024-04-24 09:20:39.607203827 +0000 UTC m=+0.168965759 container create fefe662ba88f706c9546d3b2575c3b815a315a9f132d06e549cac12da9c0d9cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-7-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.build-date=20240424, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-04-24T09:20:40.205 INFO:journalctl@ceph.osd.7.smithi138.stdout:Apr 24 09:20:39 smithi138 podman[73044]: 2024-04-24 09:20:39.939402555 +0000 UTC m=+0.501164538 container init fefe662ba88f706c9546d3b2575c3b815a315a9f132d06e549cac12da9c0d9cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-24T09:20:40.205 INFO:journalctl@ceph.osd.7.smithi138.stdout:Apr 24 09:20:39 smithi138 podman[73044]: 2024-04-24 09:20:39.951698115 +0000 UTC m=+0.513460042 container start fefe662ba88f706c9546d3b2575c3b815a315a9f132d06e549cac12da9c0d9cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T09:20:40.205 INFO:journalctl@ceph.osd.7.smithi138.stdout:Apr 24 09:20:40 smithi138 podman[73044]: 2024-04-24 09:20:40.052035142 +0000 UTC m=+0.613797113 container attach fefe662ba88f706c9546d3b2575c3b815a315a9f132d06e549cac12da9c0d9cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-7-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-24T09:20:40.207 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:39 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:39.886+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:28.209530+0000 front 2024-04-24T09:19:28.209142+0000 (oldest deadline 2024-04-24T09:19:51.709380+0000) 2024-04-24T09:20:40.207 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:39 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:39.886+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.613982+0000 front 2024-04-24T09:19:38.614541+0000 (oldest deadline 2024-04-24T09:20:00.913987+0000) 2024-04-24T09:20:40.207 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:39 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:39.886+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:49.018457+0000 front 2024-04-24T09:19:49.018520+0000 (oldest deadline 2024-04-24T09:20:16.019822+0000) 2024-04-24T09:20:40.207 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:39 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:39.886+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:07.025175+0000 front 2024-04-24T09:20:07.025338+0000 (oldest deadline 2024-04-24T09:20:31.125415+0000) 2024-04-24T09:20:40.553 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:40 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:40.246+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:27.386588+0000 front 2024-04-24T09:19:27.386891+0000 (oldest deadline 2024-04-24T09:19:52.686559+0000) 2024-04-24T09:20:40.553 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:40 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:40.246+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.891162+0000 front 2024-04-24T09:19:38.891270+0000 (oldest deadline 2024-04-24T09:20:04.790877+0000) 2024-04-24T09:20:40.553 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:40 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:40.246+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:48.794838+0000 front 2024-04-24T09:19:48.794972+0000 (oldest deadline 2024-04-24T09:20:14.095077+0000) 2024-04-24T09:20:40.554 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:40 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:40.246+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:56.997332+0000 front 2024-04-24T09:19:56.997477+0000 (oldest deadline 2024-04-24T09:20:22.897608+0000) 2024-04-24T09:20:40.554 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:40 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:40.246+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:06.400442+0000 front 2024-04-24T09:20:06.400694+0000 (oldest deadline 2024-04-24T09:20:31.099904+0000) 2024-04-24T09:20:40.554 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:40 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:40.246+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-24T09:20:15.202727+0000 front 2024-04-24T09:20:15.202777+0000 (oldest deadline 2024-04-24T09:20:39.902935+0000) 2024-04-24T09:20:40.704 INFO:journalctl@ceph.osd.7.smithi138.stdout:Apr 24 09:20:40 smithi138 podman[73044]: 2024-04-24 09:20:40.403057557 +0000 UTC m=+0.964819457 container died fefe662ba88f706c9546d3b2575c3b815a315a9f132d06e549cac12da9c0d9cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.schema-version=1.0) 2024-04-24T09:20:40.707 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:40 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:40.446+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:37.707135+0000 front 2024-04-24T09:19:37.707028+0000 (oldest deadline 2024-04-24T09:20:03.607224+0000) 2024-04-24T09:20:40.707 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:40 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:40.446+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:48.713889+0000 front 2024-04-24T09:19:48.713979+0000 (oldest deadline 2024-04-24T09:20:13.414134+0000) 2024-04-24T09:20:40.707 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:40 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:40.446+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:58.715994+0000 front 2024-04-24T09:19:58.716101+0000 (oldest deadline 2024-04-24T09:20:22.216080+0000) 2024-04-24T09:20:40.707 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:40 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:40.446+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:05.020499+0000 front 2024-04-24T09:20:05.020565+0000 (oldest deadline 2024-04-24T09:20:30.822569+0000) 2024-04-24T09:20:41.090 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:40 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:40.752+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:29.414806+0000 front 2024-04-24T09:19:29.414861+0000 (oldest deadline 2024-04-24T09:19:55.314800+0000) 2024-04-24T09:20:41.090 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:40 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:40.752+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.119965+0000 front 2024-04-24T09:19:38.120034+0000 (oldest deadline 2024-04-24T09:20:02.820181+0000) 2024-04-24T09:20:41.090 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:40 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:40.752+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:47.326279+0000 front 2024-04-24T09:19:47.326338+0000 (oldest deadline 2024-04-24T09:20:10.728055+0000) 2024-04-24T09:20:41.090 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:40 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:40.752+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:55.832608+0000 front 2024-04-24T09:19:55.832510+0000 (oldest deadline 2024-04-24T09:20:21.732750+0000) 2024-04-24T09:20:41.090 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:40 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:40.752+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:08.736362+0000 front 2024-04-24T09:20:08.736680+0000 (oldest deadline 2024-04-24T09:20:31.537874+0000) 2024-04-24T09:20:41.207 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:40 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:40.922+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:28.209530+0000 front 2024-04-24T09:19:28.209142+0000 (oldest deadline 2024-04-24T09:19:51.709380+0000) 2024-04-24T09:20:41.207 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:40 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:40.922+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.613982+0000 front 2024-04-24T09:19:38.614541+0000 (oldest deadline 2024-04-24T09:20:00.913987+0000) 2024-04-24T09:20:41.207 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:40 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:40.922+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:49.018457+0000 front 2024-04-24T09:19:49.018520+0000 (oldest deadline 2024-04-24T09:20:16.019822+0000) 2024-04-24T09:20:41.207 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:40 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:40.922+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:07.025175+0000 front 2024-04-24T09:20:07.025338+0000 (oldest deadline 2024-04-24T09:20:31.125415+0000) 2024-04-24T09:20:41.251 DEBUG:teuthology.orchestra.run.smithi138:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bd3e039e-0215-11ef-bc93-c7b262605968@osd.7.service' 2024-04-24T09:20:41.356 INFO:journalctl@ceph.osd.7.smithi138.stdout:Apr 24 09:20:41 smithi138 podman[73070]: 2024-04-24 09:20:41.215339427 +0000 UTC m=+0.793430568 container remove fefe662ba88f706c9546d3b2575c3b815a315a9f132d06e549cac12da9c0d9cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-7-deactivate, ceph=True, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-24T09:20:41.357 INFO:journalctl@ceph.osd.7.smithi138.stdout:Apr 24 09:20:41 smithi138 systemd[1]: ceph-bd3e039e-0215-11ef-bc93-c7b262605968@osd.7.service: Deactivated successfully. 2024-04-24T09:20:41.357 INFO:journalctl@ceph.osd.7.smithi138.stdout:Apr 24 09:20:41 smithi138 systemd[1]: Stopped Ceph osd.7 for bd3e039e-0215-11ef-bc93-c7b262605968. 2024-04-24T09:20:41.357 INFO:journalctl@ceph.osd.7.smithi138.stdout:Apr 24 09:20:41 smithi138 systemd[1]: ceph-bd3e039e-0215-11ef-bc93-c7b262605968@osd.7.service: Consumed 5min 18.625s CPU time. 2024-04-24T09:20:41.390 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T09:20:41.390 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-04-24T09:20:41.390 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-04-24T09:20:41.391 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ceph-bd3e039e-0215-11ef-bc93-c7b262605968@osd.8 2024-04-24T09:20:41.553 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:41 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:41.290+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:27.386588+0000 front 2024-04-24T09:19:27.386891+0000 (oldest deadline 2024-04-24T09:19:52.686559+0000) 2024-04-24T09:20:41.553 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:41 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:41.290+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.891162+0000 front 2024-04-24T09:19:38.891270+0000 (oldest deadline 2024-04-24T09:20:04.790877+0000) 2024-04-24T09:20:41.553 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:41 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:41.291+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:48.794838+0000 front 2024-04-24T09:19:48.794972+0000 (oldest deadline 2024-04-24T09:20:14.095077+0000) 2024-04-24T09:20:41.553 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:41 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:41.291+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:56.997332+0000 front 2024-04-24T09:19:56.997477+0000 (oldest deadline 2024-04-24T09:20:22.897608+0000) 2024-04-24T09:20:41.554 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:41 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:41.291+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:06.400442+0000 front 2024-04-24T09:20:06.400694+0000 (oldest deadline 2024-04-24T09:20:31.099904+0000) 2024-04-24T09:20:41.554 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:41 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:41.291+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-24T09:20:15.202727+0000 front 2024-04-24T09:20:15.202777+0000 (oldest deadline 2024-04-24T09:20:39.902935+0000) 2024-04-24T09:20:41.707 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:41 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:41.417+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:37.707135+0000 front 2024-04-24T09:19:37.707028+0000 (oldest deadline 2024-04-24T09:20:03.607224+0000) 2024-04-24T09:20:41.708 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:41 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:41.417+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:48.713889+0000 front 2024-04-24T09:19:48.713979+0000 (oldest deadline 2024-04-24T09:20:13.414134+0000) 2024-04-24T09:20:41.708 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:41 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:41.417+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:58.715994+0000 front 2024-04-24T09:19:58.716101+0000 (oldest deadline 2024-04-24T09:20:22.216080+0000) 2024-04-24T09:20:41.708 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:41 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:41.417+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:05.020499+0000 front 2024-04-24T09:20:05.020565+0000 (oldest deadline 2024-04-24T09:20:30.822569+0000) 2024-04-24T09:20:41.708 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:41 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:41.417+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-24T09:20:18.426278+0000 front 2024-04-24T09:20:18.426341+0000 (oldest deadline 2024-04-24T09:20:41.326431+0000) 2024-04-24T09:20:41.708 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:41 smithi189 systemd[1]: Stopping Ceph osd.8 for bd3e039e-0215-11ef-bc93-c7b262605968... 2024-04-24T09:20:42.019 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:41 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:41.745+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:29.414806+0000 front 2024-04-24T09:19:29.414861+0000 (oldest deadline 2024-04-24T09:19:55.314800+0000) 2024-04-24T09:20:42.033 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:41 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:41.745+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.119965+0000 front 2024-04-24T09:19:38.120034+0000 (oldest deadline 2024-04-24T09:20:02.820181+0000) 2024-04-24T09:20:42.033 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:41 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:41.745+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:47.326279+0000 front 2024-04-24T09:19:47.326338+0000 (oldest deadline 2024-04-24T09:20:10.728055+0000) 2024-04-24T09:20:42.034 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:41 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:41.745+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:55.832608+0000 front 2024-04-24T09:19:55.832510+0000 (oldest deadline 2024-04-24T09:20:21.732750+0000) 2024-04-24T09:20:42.034 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:41 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:41.745+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:08.736362+0000 front 2024-04-24T09:20:08.736680+0000 (oldest deadline 2024-04-24T09:20:31.537874+0000) 2024-04-24T09:20:42.034 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:41 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:41.745+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-24T09:20:16.240265+0000 front 2024-04-24T09:20:16.240144+0000 (oldest deadline 2024-04-24T09:20:41.540224+0000) 2024-04-24T09:20:42.207 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:41 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:41.973+0000 7efd4b400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T09:20:42.207 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:41 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:41.973+0000 7efd4b400700 -1 osd.8 141 *** Got signal Terminated *** 2024-04-24T09:20:42.207 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:41 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:41.973+0000 7efd4b400700 -1 osd.8 141 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T09:20:42.208 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:41 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:41.924+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:28.209530+0000 front 2024-04-24T09:19:28.209142+0000 (oldest deadline 2024-04-24T09:19:51.709380+0000) 2024-04-24T09:20:42.208 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:41 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:41.924+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.613982+0000 front 2024-04-24T09:19:38.614541+0000 (oldest deadline 2024-04-24T09:20:00.913987+0000) 2024-04-24T09:20:42.208 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:41 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:41.924+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:49.018457+0000 front 2024-04-24T09:19:49.018520+0000 (oldest deadline 2024-04-24T09:20:16.019822+0000) 2024-04-24T09:20:42.208 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:41 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:41.924+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:07.025175+0000 front 2024-04-24T09:20:07.025338+0000 (oldest deadline 2024-04-24T09:20:31.125415+0000) 2024-04-24T09:20:42.553 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:42 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:42.287+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:27.386588+0000 front 2024-04-24T09:19:27.386891+0000 (oldest deadline 2024-04-24T09:19:52.686559+0000) 2024-04-24T09:20:42.553 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:42 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:42.287+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.891162+0000 front 2024-04-24T09:19:38.891270+0000 (oldest deadline 2024-04-24T09:20:04.790877+0000) 2024-04-24T09:20:42.553 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:42 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:42.287+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:48.794838+0000 front 2024-04-24T09:19:48.794972+0000 (oldest deadline 2024-04-24T09:20:14.095077+0000) 2024-04-24T09:20:42.554 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:42 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:42.287+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:56.997332+0000 front 2024-04-24T09:19:56.997477+0000 (oldest deadline 2024-04-24T09:20:22.897608+0000) 2024-04-24T09:20:42.554 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:42 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:42.287+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:06.400442+0000 front 2024-04-24T09:20:06.400694+0000 (oldest deadline 2024-04-24T09:20:31.099904+0000) 2024-04-24T09:20:42.554 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:42 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:42.287+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-24T09:20:15.202727+0000 front 2024-04-24T09:20:15.202777+0000 (oldest deadline 2024-04-24T09:20:39.902935+0000) 2024-04-24T09:20:42.707 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:42 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:42.393+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:37.707135+0000 front 2024-04-24T09:19:37.707028+0000 (oldest deadline 2024-04-24T09:20:03.607224+0000) 2024-04-24T09:20:42.707 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:42 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:42.393+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:48.713889+0000 front 2024-04-24T09:19:48.713979+0000 (oldest deadline 2024-04-24T09:20:13.414134+0000) 2024-04-24T09:20:42.707 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:42 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:42.393+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:58.715994+0000 front 2024-04-24T09:19:58.716101+0000 (oldest deadline 2024-04-24T09:20:22.216080+0000) 2024-04-24T09:20:42.707 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:42 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:42.393+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:05.020499+0000 front 2024-04-24T09:20:05.020565+0000 (oldest deadline 2024-04-24T09:20:30.822569+0000) 2024-04-24T09:20:42.707 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:42 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:42.393+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-24T09:20:18.426278+0000 front 2024-04-24T09:20:18.426341+0000 (oldest deadline 2024-04-24T09:20:41.326431+0000) 2024-04-24T09:20:43.205 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:42 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:42.716+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:29.414806+0000 front 2024-04-24T09:19:29.414861+0000 (oldest deadline 2024-04-24T09:19:55.314800+0000) 2024-04-24T09:20:43.205 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:42 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:42.716+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.119965+0000 front 2024-04-24T09:19:38.120034+0000 (oldest deadline 2024-04-24T09:20:02.820181+0000) 2024-04-24T09:20:43.205 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:42 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:42.716+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:47.326279+0000 front 2024-04-24T09:19:47.326338+0000 (oldest deadline 2024-04-24T09:20:10.728055+0000) 2024-04-24T09:20:43.205 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:42 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:42.716+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:55.832608+0000 front 2024-04-24T09:19:55.832510+0000 (oldest deadline 2024-04-24T09:20:21.732750+0000) 2024-04-24T09:20:43.205 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:42 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:42.716+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:08.736362+0000 front 2024-04-24T09:20:08.736680+0000 (oldest deadline 2024-04-24T09:20:31.537874+0000) 2024-04-24T09:20:43.206 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:42 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:42.716+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-24T09:20:16.240265+0000 front 2024-04-24T09:20:16.240144+0000 (oldest deadline 2024-04-24T09:20:41.540224+0000) 2024-04-24T09:20:43.207 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:42 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:42.952+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:28.209530+0000 front 2024-04-24T09:19:28.209142+0000 (oldest deadline 2024-04-24T09:19:51.709380+0000) 2024-04-24T09:20:43.207 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:42 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:42.952+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.613982+0000 front 2024-04-24T09:19:38.614541+0000 (oldest deadline 2024-04-24T09:20:00.913987+0000) 2024-04-24T09:20:43.207 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:42 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:42.952+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:49.018457+0000 front 2024-04-24T09:19:49.018520+0000 (oldest deadline 2024-04-24T09:20:16.019822+0000) 2024-04-24T09:20:43.207 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:42 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:42.952+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:07.025175+0000 front 2024-04-24T09:20:07.025338+0000 (oldest deadline 2024-04-24T09:20:31.125415+0000) 2024-04-24T09:20:43.207 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:42 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:42.952+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-24T09:20:17.027505+0000 front 2024-04-24T09:20:17.028269+0000 (oldest deadline 2024-04-24T09:20:42.327601+0000) 2024-04-24T09:20:43.553 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:43 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:43.239+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:27.386588+0000 front 2024-04-24T09:19:27.386891+0000 (oldest deadline 2024-04-24T09:19:52.686559+0000) 2024-04-24T09:20:43.553 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:43 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:43.239+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.891162+0000 front 2024-04-24T09:19:38.891270+0000 (oldest deadline 2024-04-24T09:20:04.790877+0000) 2024-04-24T09:20:43.554 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:43 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:43.239+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:48.794838+0000 front 2024-04-24T09:19:48.794972+0000 (oldest deadline 2024-04-24T09:20:14.095077+0000) 2024-04-24T09:20:43.554 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:43 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:43.239+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:56.997332+0000 front 2024-04-24T09:19:56.997477+0000 (oldest deadline 2024-04-24T09:20:22.897608+0000) 2024-04-24T09:20:43.554 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:43 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:43.239+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:06.400442+0000 front 2024-04-24T09:20:06.400694+0000 (oldest deadline 2024-04-24T09:20:31.099904+0000) 2024-04-24T09:20:43.554 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:43 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:43.239+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-24T09:20:15.202727+0000 front 2024-04-24T09:20:15.202777+0000 (oldest deadline 2024-04-24T09:20:39.902935+0000) 2024-04-24T09:20:43.707 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:43 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:43.387+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:37.707135+0000 front 2024-04-24T09:19:37.707028+0000 (oldest deadline 2024-04-24T09:20:03.607224+0000) 2024-04-24T09:20:43.707 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:43 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:43.387+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:48.713889+0000 front 2024-04-24T09:19:48.713979+0000 (oldest deadline 2024-04-24T09:20:13.414134+0000) 2024-04-24T09:20:43.707 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:43 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:43.387+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:58.715994+0000 front 2024-04-24T09:19:58.716101+0000 (oldest deadline 2024-04-24T09:20:22.216080+0000) 2024-04-24T09:20:43.707 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:43 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:43.387+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:05.020499+0000 front 2024-04-24T09:20:05.020565+0000 (oldest deadline 2024-04-24T09:20:30.822569+0000) 2024-04-24T09:20:43.707 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:43 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:43.387+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-24T09:20:18.426278+0000 front 2024-04-24T09:20:18.426341+0000 (oldest deadline 2024-04-24T09:20:41.326431+0000) 2024-04-24T09:20:44.205 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:43 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:43.734+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:29.414806+0000 front 2024-04-24T09:19:29.414861+0000 (oldest deadline 2024-04-24T09:19:55.314800+0000) 2024-04-24T09:20:44.205 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:43 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:43.734+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.119965+0000 front 2024-04-24T09:19:38.120034+0000 (oldest deadline 2024-04-24T09:20:02.820181+0000) 2024-04-24T09:20:44.205 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:43 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:43.734+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:47.326279+0000 front 2024-04-24T09:19:47.326338+0000 (oldest deadline 2024-04-24T09:20:10.728055+0000) 2024-04-24T09:20:44.205 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:43 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:43.734+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:55.832608+0000 front 2024-04-24T09:19:55.832510+0000 (oldest deadline 2024-04-24T09:20:21.732750+0000) 2024-04-24T09:20:44.205 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:43 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:43.734+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:08.736362+0000 front 2024-04-24T09:20:08.736680+0000 (oldest deadline 2024-04-24T09:20:31.537874+0000) 2024-04-24T09:20:44.206 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:43 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:43.734+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-24T09:20:16.240265+0000 front 2024-04-24T09:20:16.240144+0000 (oldest deadline 2024-04-24T09:20:41.540224+0000) 2024-04-24T09:20:44.207 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:43 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:43.933+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:28.209530+0000 front 2024-04-24T09:19:28.209142+0000 (oldest deadline 2024-04-24T09:19:51.709380+0000) 2024-04-24T09:20:44.207 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:43 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:43.934+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.613982+0000 front 2024-04-24T09:19:38.614541+0000 (oldest deadline 2024-04-24T09:20:00.913987+0000) 2024-04-24T09:20:44.207 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:43 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:43.934+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:49.018457+0000 front 2024-04-24T09:19:49.018520+0000 (oldest deadline 2024-04-24T09:20:16.019822+0000) 2024-04-24T09:20:44.207 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:43 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:43.934+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:07.025175+0000 front 2024-04-24T09:20:07.025338+0000 (oldest deadline 2024-04-24T09:20:31.125415+0000) 2024-04-24T09:20:44.207 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:43 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:43.934+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-24T09:20:17.027505+0000 front 2024-04-24T09:20:17.028269+0000 (oldest deadline 2024-04-24T09:20:42.327601+0000) 2024-04-24T09:20:44.553 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:44 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:44.202+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:27.386588+0000 front 2024-04-24T09:19:27.386891+0000 (oldest deadline 2024-04-24T09:19:52.686559+0000) 2024-04-24T09:20:44.553 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:44 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:44.202+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.891162+0000 front 2024-04-24T09:19:38.891270+0000 (oldest deadline 2024-04-24T09:20:04.790877+0000) 2024-04-24T09:20:44.554 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:44 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:44.202+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:48.794838+0000 front 2024-04-24T09:19:48.794972+0000 (oldest deadline 2024-04-24T09:20:14.095077+0000) 2024-04-24T09:20:44.554 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:44 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:44.202+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:56.997332+0000 front 2024-04-24T09:19:56.997477+0000 (oldest deadline 2024-04-24T09:20:22.897608+0000) 2024-04-24T09:20:44.554 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:44 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:44.202+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:06.400442+0000 front 2024-04-24T09:20:06.400694+0000 (oldest deadline 2024-04-24T09:20:31.099904+0000) 2024-04-24T09:20:44.554 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:44 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:44.202+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-24T09:20:15.202727+0000 front 2024-04-24T09:20:15.202777+0000 (oldest deadline 2024-04-24T09:20:39.902935+0000) 2024-04-24T09:20:44.707 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:44 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:44.378+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:37.707135+0000 front 2024-04-24T09:19:37.707028+0000 (oldest deadline 2024-04-24T09:20:03.607224+0000) 2024-04-24T09:20:44.707 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:44 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:44.378+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:48.713889+0000 front 2024-04-24T09:19:48.713979+0000 (oldest deadline 2024-04-24T09:20:13.414134+0000) 2024-04-24T09:20:44.707 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:44 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:44.378+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:58.715994+0000 front 2024-04-24T09:19:58.716101+0000 (oldest deadline 2024-04-24T09:20:22.216080+0000) 2024-04-24T09:20:44.707 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:44 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:44.378+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:05.020499+0000 front 2024-04-24T09:20:05.020565+0000 (oldest deadline 2024-04-24T09:20:30.822569+0000) 2024-04-24T09:20:44.708 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:44 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:44.378+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-24T09:20:18.426278+0000 front 2024-04-24T09:20:18.426341+0000 (oldest deadline 2024-04-24T09:20:41.326431+0000) 2024-04-24T09:20:45.205 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:44 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:44.723+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:29.414806+0000 front 2024-04-24T09:19:29.414861+0000 (oldest deadline 2024-04-24T09:19:55.314800+0000) 2024-04-24T09:20:45.206 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:44 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:44.723+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.119965+0000 front 2024-04-24T09:19:38.120034+0000 (oldest deadline 2024-04-24T09:20:02.820181+0000) 2024-04-24T09:20:45.206 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:44 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:44.723+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:47.326279+0000 front 2024-04-24T09:19:47.326338+0000 (oldest deadline 2024-04-24T09:20:10.728055+0000) 2024-04-24T09:20:45.206 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:44 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:44.723+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:55.832608+0000 front 2024-04-24T09:19:55.832510+0000 (oldest deadline 2024-04-24T09:20:21.732750+0000) 2024-04-24T09:20:45.206 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:44 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:44.723+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:08.736362+0000 front 2024-04-24T09:20:08.736680+0000 (oldest deadline 2024-04-24T09:20:31.537874+0000) 2024-04-24T09:20:45.206 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:44 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:44.723+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-24T09:20:16.240265+0000 front 2024-04-24T09:20:16.240144+0000 (oldest deadline 2024-04-24T09:20:41.540224+0000) 2024-04-24T09:20:45.367 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:44 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:44.971+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:28.209530+0000 front 2024-04-24T09:19:28.209142+0000 (oldest deadline 2024-04-24T09:19:51.709380+0000) 2024-04-24T09:20:45.367 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:44 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:44.971+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.613982+0000 front 2024-04-24T09:19:38.614541+0000 (oldest deadline 2024-04-24T09:20:00.913987+0000) 2024-04-24T09:20:45.367 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:44 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:44.971+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:49.018457+0000 front 2024-04-24T09:19:49.018520+0000 (oldest deadline 2024-04-24T09:20:16.019822+0000) 2024-04-24T09:20:45.367 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:44 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:44.971+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:07.025175+0000 front 2024-04-24T09:20:07.025338+0000 (oldest deadline 2024-04-24T09:20:31.125415+0000) 2024-04-24T09:20:45.367 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:44 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:44.971+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-24T09:20:17.027505+0000 front 2024-04-24T09:20:17.028269+0000 (oldest deadline 2024-04-24T09:20:42.327601+0000) 2024-04-24T09:20:45.553 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:45 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:45.232+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:27.386588+0000 front 2024-04-24T09:19:27.386891+0000 (oldest deadline 2024-04-24T09:19:52.686559+0000) 2024-04-24T09:20:45.553 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:45 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:45.232+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.891162+0000 front 2024-04-24T09:19:38.891270+0000 (oldest deadline 2024-04-24T09:20:04.790877+0000) 2024-04-24T09:20:45.554 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:45 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:45.232+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:48.794838+0000 front 2024-04-24T09:19:48.794972+0000 (oldest deadline 2024-04-24T09:20:14.095077+0000) 2024-04-24T09:20:45.554 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:45 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:45.232+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:56.997332+0000 front 2024-04-24T09:19:56.997477+0000 (oldest deadline 2024-04-24T09:20:22.897608+0000) 2024-04-24T09:20:45.554 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:45 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:45.232+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:06.400442+0000 front 2024-04-24T09:20:06.400694+0000 (oldest deadline 2024-04-24T09:20:31.099904+0000) 2024-04-24T09:20:45.554 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:45 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:45.232+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-24T09:20:15.202727+0000 front 2024-04-24T09:20:15.202777+0000 (oldest deadline 2024-04-24T09:20:39.902935+0000) 2024-04-24T09:20:45.707 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:45 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:45.366+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:37.707135+0000 front 2024-04-24T09:19:37.707028+0000 (oldest deadline 2024-04-24T09:20:03.607224+0000) 2024-04-24T09:20:45.707 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:45 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:45.366+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:48.713889+0000 front 2024-04-24T09:19:48.713979+0000 (oldest deadline 2024-04-24T09:20:13.414134+0000) 2024-04-24T09:20:45.708 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:45 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:45.366+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:58.715994+0000 front 2024-04-24T09:19:58.716101+0000 (oldest deadline 2024-04-24T09:20:22.216080+0000) 2024-04-24T09:20:45.708 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:45 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:45.366+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:05.020499+0000 front 2024-04-24T09:20:05.020565+0000 (oldest deadline 2024-04-24T09:20:30.822569+0000) 2024-04-24T09:20:45.708 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:45 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:45.366+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-24T09:20:18.426278+0000 front 2024-04-24T09:20:18.426341+0000 (oldest deadline 2024-04-24T09:20:41.326431+0000) 2024-04-24T09:20:46.205 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:45 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:45.734+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:29.414806+0000 front 2024-04-24T09:19:29.414861+0000 (oldest deadline 2024-04-24T09:19:55.314800+0000) 2024-04-24T09:20:46.205 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:45 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:45.734+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.119965+0000 front 2024-04-24T09:19:38.120034+0000 (oldest deadline 2024-04-24T09:20:02.820181+0000) 2024-04-24T09:20:46.205 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:45 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:45.734+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:47.326279+0000 front 2024-04-24T09:19:47.326338+0000 (oldest deadline 2024-04-24T09:20:10.728055+0000) 2024-04-24T09:20:46.205 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:45 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:45.734+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:55.832608+0000 front 2024-04-24T09:19:55.832510+0000 (oldest deadline 2024-04-24T09:20:21.732750+0000) 2024-04-24T09:20:46.205 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:45 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:45.734+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:08.736362+0000 front 2024-04-24T09:20:08.736680+0000 (oldest deadline 2024-04-24T09:20:31.537874+0000) 2024-04-24T09:20:46.206 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:45 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:45.734+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-24T09:20:16.240265+0000 front 2024-04-24T09:20:16.240144+0000 (oldest deadline 2024-04-24T09:20:41.540224+0000) 2024-04-24T09:20:46.417 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:45 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:45.984+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:28.209530+0000 front 2024-04-24T09:19:28.209142+0000 (oldest deadline 2024-04-24T09:19:51.709380+0000) 2024-04-24T09:20:46.417 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:45 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:45.984+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.613982+0000 front 2024-04-24T09:19:38.614541+0000 (oldest deadline 2024-04-24T09:20:00.913987+0000) 2024-04-24T09:20:46.417 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:45 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:45.984+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:49.018457+0000 front 2024-04-24T09:19:49.018520+0000 (oldest deadline 2024-04-24T09:20:16.019822+0000) 2024-04-24T09:20:46.417 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:45 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:45.984+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:07.025175+0000 front 2024-04-24T09:20:07.025338+0000 (oldest deadline 2024-04-24T09:20:31.125415+0000) 2024-04-24T09:20:46.417 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:45 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:45.984+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-24T09:20:17.027505+0000 front 2024-04-24T09:20:17.028269+0000 (oldest deadline 2024-04-24T09:20:42.327601+0000) 2024-04-24T09:20:46.553 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:46 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:46.238+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:27.386588+0000 front 2024-04-24T09:19:27.386891+0000 (oldest deadline 2024-04-24T09:19:52.686559+0000) 2024-04-24T09:20:46.554 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:46 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:46.238+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.891162+0000 front 2024-04-24T09:19:38.891270+0000 (oldest deadline 2024-04-24T09:20:04.790877+0000) 2024-04-24T09:20:46.554 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:46 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:46.238+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:48.794838+0000 front 2024-04-24T09:19:48.794972+0000 (oldest deadline 2024-04-24T09:20:14.095077+0000) 2024-04-24T09:20:46.554 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:46 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:46.238+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:56.997332+0000 front 2024-04-24T09:19:56.997477+0000 (oldest deadline 2024-04-24T09:20:22.897608+0000) 2024-04-24T09:20:46.554 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:46 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:46.238+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:06.400442+0000 front 2024-04-24T09:20:06.400694+0000 (oldest deadline 2024-04-24T09:20:31.099904+0000) 2024-04-24T09:20:46.554 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:46 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:46.238+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-24T09:20:15.202727+0000 front 2024-04-24T09:20:15.202777+0000 (oldest deadline 2024-04-24T09:20:39.902935+0000) 2024-04-24T09:20:46.707 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:46 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:46.416+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:37.707135+0000 front 2024-04-24T09:19:37.707028+0000 (oldest deadline 2024-04-24T09:20:03.607224+0000) 2024-04-24T09:20:46.707 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:46 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:46.416+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:48.713889+0000 front 2024-04-24T09:19:48.713979+0000 (oldest deadline 2024-04-24T09:20:13.414134+0000) 2024-04-24T09:20:46.707 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:46 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:46.416+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:58.715994+0000 front 2024-04-24T09:19:58.716101+0000 (oldest deadline 2024-04-24T09:20:22.216080+0000) 2024-04-24T09:20:46.707 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:46 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:46.416+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:05.020499+0000 front 2024-04-24T09:20:05.020565+0000 (oldest deadline 2024-04-24T09:20:30.822569+0000) 2024-04-24T09:20:46.707 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:46 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8[50065]: 2024-04-24T09:20:46.416+0000 7efd46c00700 -1 osd.8 141 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-24T09:20:18.426278+0000 front 2024-04-24T09:20:18.426341+0000 (oldest deadline 2024-04-24T09:20:41.326431+0000) 2024-04-24T09:20:47.205 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:46 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:46.778+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:29.414806+0000 front 2024-04-24T09:19:29.414861+0000 (oldest deadline 2024-04-24T09:19:55.314800+0000) 2024-04-24T09:20:47.205 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:46 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:46.778+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.119965+0000 front 2024-04-24T09:19:38.120034+0000 (oldest deadline 2024-04-24T09:20:02.820181+0000) 2024-04-24T09:20:47.206 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:46 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:46.778+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:47.326279+0000 front 2024-04-24T09:19:47.326338+0000 (oldest deadline 2024-04-24T09:20:10.728055+0000) 2024-04-24T09:20:47.206 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:46 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:46.778+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:55.832608+0000 front 2024-04-24T09:19:55.832510+0000 (oldest deadline 2024-04-24T09:20:21.732750+0000) 2024-04-24T09:20:47.206 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:46 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:46.778+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:08.736362+0000 front 2024-04-24T09:20:08.736680+0000 (oldest deadline 2024-04-24T09:20:31.537874+0000) 2024-04-24T09:20:47.206 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:46 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:46.778+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-24T09:20:16.240265+0000 front 2024-04-24T09:20:16.240144+0000 (oldest deadline 2024-04-24T09:20:41.540224+0000) 2024-04-24T09:20:47.457 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:47 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:47.032+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:28.209530+0000 front 2024-04-24T09:19:28.209142+0000 (oldest deadline 2024-04-24T09:19:51.709380+0000) 2024-04-24T09:20:47.457 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:47 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:47.032+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.613982+0000 front 2024-04-24T09:19:38.614541+0000 (oldest deadline 2024-04-24T09:20:00.913987+0000) 2024-04-24T09:20:47.457 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:47 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:47.032+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:49.018457+0000 front 2024-04-24T09:19:49.018520+0000 (oldest deadline 2024-04-24T09:20:16.019822+0000) 2024-04-24T09:20:47.457 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:47 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:47.032+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:07.025175+0000 front 2024-04-24T09:20:07.025338+0000 (oldest deadline 2024-04-24T09:20:31.125415+0000) 2024-04-24T09:20:47.458 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:47 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:47.032+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-24T09:20:17.027505+0000 front 2024-04-24T09:20:17.028269+0000 (oldest deadline 2024-04-24T09:20:42.327601+0000) 2024-04-24T09:20:47.553 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:47 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:47.275+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:27.386588+0000 front 2024-04-24T09:19:27.386891+0000 (oldest deadline 2024-04-24T09:19:52.686559+0000) 2024-04-24T09:20:47.553 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:47 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:47.275+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.891162+0000 front 2024-04-24T09:19:38.891270+0000 (oldest deadline 2024-04-24T09:20:04.790877+0000) 2024-04-24T09:20:47.553 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:47 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:47.275+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:48.794838+0000 front 2024-04-24T09:19:48.794972+0000 (oldest deadline 2024-04-24T09:20:14.095077+0000) 2024-04-24T09:20:47.553 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:47 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:47.275+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:56.997332+0000 front 2024-04-24T09:19:56.997477+0000 (oldest deadline 2024-04-24T09:20:22.897608+0000) 2024-04-24T09:20:47.554 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:47 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:47.275+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:06.400442+0000 front 2024-04-24T09:20:06.400694+0000 (oldest deadline 2024-04-24T09:20:31.099904+0000) 2024-04-24T09:20:47.554 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:47 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:47.275+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-24T09:20:15.202727+0000 front 2024-04-24T09:20:15.202777+0000 (oldest deadline 2024-04-24T09:20:39.902935+0000) 2024-04-24T09:20:47.839 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:47 smithi189 podman[77157]: 2024-04-24 09:20:47.543710396 +0000 UTC m=+5.711818273 container died 544e3f92c74d92e663107fc21abd86f49aa0951b9b0566adfcf3816e3266ba3a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=) 2024-04-24T09:20:48.205 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:47 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:47.787+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:29.414806+0000 front 2024-04-24T09:19:29.414861+0000 (oldest deadline 2024-04-24T09:19:55.314800+0000) 2024-04-24T09:20:48.205 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:47 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:47.787+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.119965+0000 front 2024-04-24T09:19:38.120034+0000 (oldest deadline 2024-04-24T09:20:02.820181+0000) 2024-04-24T09:20:48.205 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:47 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:47.787+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:47.326279+0000 front 2024-04-24T09:19:47.326338+0000 (oldest deadline 2024-04-24T09:20:10.728055+0000) 2024-04-24T09:20:48.205 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:47 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:47.787+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:55.832608+0000 front 2024-04-24T09:19:55.832510+0000 (oldest deadline 2024-04-24T09:20:21.732750+0000) 2024-04-24T09:20:48.205 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:47 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:47.787+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:08.736362+0000 front 2024-04-24T09:20:08.736680+0000 (oldest deadline 2024-04-24T09:20:31.537874+0000) 2024-04-24T09:20:48.206 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:47 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:47.787+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-24T09:20:16.240265+0000 front 2024-04-24T09:20:16.240144+0000 (oldest deadline 2024-04-24T09:20:41.540224+0000) 2024-04-24T09:20:48.207 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:47 smithi189 podman[77157]: 2024-04-24 09:20:47.950804577 +0000 UTC m=+6.118912467 container cleanup 544e3f92c74d92e663107fc21abd86f49aa0951b9b0566adfcf3816e3266ba3a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=rishabh-main-apr17-a654945-1f81fb3) 2024-04-24T09:20:48.207 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:47 smithi189 bash[77157]: ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8 2024-04-24T09:20:48.207 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:48 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:48.003+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:28.209530+0000 front 2024-04-24T09:19:28.209142+0000 (oldest deadline 2024-04-24T09:19:51.709380+0000) 2024-04-24T09:20:48.207 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:48 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:48.003+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.613982+0000 front 2024-04-24T09:19:38.614541+0000 (oldest deadline 2024-04-24T09:20:00.913987+0000) 2024-04-24T09:20:48.207 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:48 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:48.003+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:49.018457+0000 front 2024-04-24T09:19:49.018520+0000 (oldest deadline 2024-04-24T09:20:16.019822+0000) 2024-04-24T09:20:48.207 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:48 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:48.003+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:07.025175+0000 front 2024-04-24T09:20:07.025338+0000 (oldest deadline 2024-04-24T09:20:31.125415+0000) 2024-04-24T09:20:48.207 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:48 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:48.003+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-24T09:20:17.027505+0000 front 2024-04-24T09:20:17.028269+0000 (oldest deadline 2024-04-24T09:20:42.327601+0000) 2024-04-24T09:20:48.707 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:48 smithi189 podman[77175]: 2024-04-24 09:20:48.24360303 +0000 UTC m=+0.682632903 container remove 544e3f92c74d92e663107fc21abd86f49aa0951b9b0566adfcf3816e3266ba3a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-04-24T09:20:48.803 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:48 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:48.315+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:27.386588+0000 front 2024-04-24T09:19:27.386891+0000 (oldest deadline 2024-04-24T09:19:52.686559+0000) 2024-04-24T09:20:48.803 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:48 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:48.315+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.891162+0000 front 2024-04-24T09:19:38.891270+0000 (oldest deadline 2024-04-24T09:20:04.790877+0000) 2024-04-24T09:20:48.803 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:48 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:48.315+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:48.794838+0000 front 2024-04-24T09:19:48.794972+0000 (oldest deadline 2024-04-24T09:20:14.095077+0000) 2024-04-24T09:20:48.803 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:48 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:48.315+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:56.997332+0000 front 2024-04-24T09:19:56.997477+0000 (oldest deadline 2024-04-24T09:20:22.897608+0000) 2024-04-24T09:20:48.804 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:48 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:48.315+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:06.400442+0000 front 2024-04-24T09:20:06.400694+0000 (oldest deadline 2024-04-24T09:20:31.099904+0000) 2024-04-24T09:20:48.804 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:48 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:48.315+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-24T09:20:15.202727+0000 front 2024-04-24T09:20:15.202777+0000 (oldest deadline 2024-04-24T09:20:39.902935+0000) 2024-04-24T09:20:49.206 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:48 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:48.792+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:29.414806+0000 front 2024-04-24T09:19:29.414861+0000 (oldest deadline 2024-04-24T09:19:55.314800+0000) 2024-04-24T09:20:49.206 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:48 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:48.792+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.119965+0000 front 2024-04-24T09:19:38.120034+0000 (oldest deadline 2024-04-24T09:20:02.820181+0000) 2024-04-24T09:20:49.206 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:48 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:48.792+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:47.326279+0000 front 2024-04-24T09:19:47.326338+0000 (oldest deadline 2024-04-24T09:20:10.728055+0000) 2024-04-24T09:20:49.206 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:48 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:48.792+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:55.832608+0000 front 2024-04-24T09:19:55.832510+0000 (oldest deadline 2024-04-24T09:20:21.732750+0000) 2024-04-24T09:20:49.206 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:48 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:48.792+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:08.736362+0000 front 2024-04-24T09:20:08.736680+0000 (oldest deadline 2024-04-24T09:20:31.537874+0000) 2024-04-24T09:20:49.206 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:48 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:48.792+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-24T09:20:16.240265+0000 front 2024-04-24T09:20:16.240144+0000 (oldest deadline 2024-04-24T09:20:41.540224+0000) 2024-04-24T09:20:49.280 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:49 smithi189 podman[77262]: 2024-04-24 09:20:48.954970791 +0000 UTC m=+0.032428798 image pull b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644 2024-04-24T09:20:49.280 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:49 smithi189 podman[77262]: 2024-04-24 09:20:49.115789281 +0000 UTC m=+0.193247322 container create 26daafad2ddc135300236060403c5edddebd241307adef94a8cb28e7392b0694 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, CEPH_POINT_RELEASE=, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-24T09:20:49.280 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:48 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:48.996+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:28.209530+0000 front 2024-04-24T09:19:28.209142+0000 (oldest deadline 2024-04-24T09:19:51.709380+0000) 2024-04-24T09:20:49.280 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:48 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:48.996+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.613982+0000 front 2024-04-24T09:19:38.614541+0000 (oldest deadline 2024-04-24T09:20:00.913987+0000) 2024-04-24T09:20:49.280 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:48 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:48.996+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:49.018457+0000 front 2024-04-24T09:19:49.018520+0000 (oldest deadline 2024-04-24T09:20:16.019822+0000) 2024-04-24T09:20:49.281 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:48 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:48.996+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:07.025175+0000 front 2024-04-24T09:20:07.025338+0000 (oldest deadline 2024-04-24T09:20:31.125415+0000) 2024-04-24T09:20:49.281 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:48 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:48.996+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-24T09:20:17.027505+0000 front 2024-04-24T09:20:17.028269+0000 (oldest deadline 2024-04-24T09:20:42.327601+0000) 2024-04-24T09:20:49.707 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:49 smithi189 podman[77262]: 2024-04-24 09:20:49.414984021 +0000 UTC m=+0.492442069 container init 26daafad2ddc135300236060403c5edddebd241307adef94a8cb28e7392b0694 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=rishabh-main-apr17-a654945-1f81fb3, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-04-24T09:20:49.707 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:49 smithi189 podman[77262]: 2024-04-24 09:20:49.427512584 +0000 UTC m=+0.504970652 container start 26daafad2ddc135300236060403c5edddebd241307adef94a8cb28e7392b0694 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T09:20:49.707 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:49 smithi189 podman[77262]: 2024-04-24 09:20:49.513818907 +0000 UTC m=+0.591276955 container attach 26daafad2ddc135300236060403c5edddebd241307adef94a8cb28e7392b0694 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8-deactivate, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=) 2024-04-24T09:20:49.803 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:49 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:49.306+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:27.386588+0000 front 2024-04-24T09:19:27.386891+0000 (oldest deadline 2024-04-24T09:19:52.686559+0000) 2024-04-24T09:20:49.803 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:49 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:49.306+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.891162+0000 front 2024-04-24T09:19:38.891270+0000 (oldest deadline 2024-04-24T09:20:04.790877+0000) 2024-04-24T09:20:49.803 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:49 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:49.306+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:48.794838+0000 front 2024-04-24T09:19:48.794972+0000 (oldest deadline 2024-04-24T09:20:14.095077+0000) 2024-04-24T09:20:49.803 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:49 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:49.306+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:56.997332+0000 front 2024-04-24T09:19:56.997477+0000 (oldest deadline 2024-04-24T09:20:22.897608+0000) 2024-04-24T09:20:49.803 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:49 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:49.306+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:06.400442+0000 front 2024-04-24T09:20:06.400694+0000 (oldest deadline 2024-04-24T09:20:31.099904+0000) 2024-04-24T09:20:49.804 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:49 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:49.306+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-24T09:20:15.202727+0000 front 2024-04-24T09:20:15.202777+0000 (oldest deadline 2024-04-24T09:20:39.902935+0000) 2024-04-24T09:20:50.205 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:49 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:49.834+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:29.414806+0000 front 2024-04-24T09:19:29.414861+0000 (oldest deadline 2024-04-24T09:19:55.314800+0000) 2024-04-24T09:20:50.205 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:49 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:49.834+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.119965+0000 front 2024-04-24T09:19:38.120034+0000 (oldest deadline 2024-04-24T09:20:02.820181+0000) 2024-04-24T09:20:50.205 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:49 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:49.834+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:47.326279+0000 front 2024-04-24T09:19:47.326338+0000 (oldest deadline 2024-04-24T09:20:10.728055+0000) 2024-04-24T09:20:50.205 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:49 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:49.834+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:55.832608+0000 front 2024-04-24T09:19:55.832510+0000 (oldest deadline 2024-04-24T09:20:21.732750+0000) 2024-04-24T09:20:50.205 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:49 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:49.834+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:08.736362+0000 front 2024-04-24T09:20:08.736680+0000 (oldest deadline 2024-04-24T09:20:31.537874+0000) 2024-04-24T09:20:50.205 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:49 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:49.834+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-24T09:20:16.240265+0000 front 2024-04-24T09:20:16.240144+0000 (oldest deadline 2024-04-24T09:20:41.540224+0000) 2024-04-24T09:20:50.207 INFO:journalctl@ceph.osd.8.smithi189.stdout:Apr 24 09:20:49 smithi189 podman[77262]: 2024-04-24 09:20:49.896546464 +0000 UTC m=+0.974004512 container died 26daafad2ddc135300236060403c5edddebd241307adef94a8cb28e7392b0694 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-8-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=rishabh-main-apr17-a654945-1f81fb3, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-04-24T09:20:50.208 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:50 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:50.024+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:28.209530+0000 front 2024-04-24T09:19:28.209142+0000 (oldest deadline 2024-04-24T09:19:51.709380+0000) 2024-04-24T09:20:50.208 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:50 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:50.024+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.613982+0000 front 2024-04-24T09:19:38.614541+0000 (oldest deadline 2024-04-24T09:20:00.913987+0000) 2024-04-24T09:20:50.208 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:50 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:50.024+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:49.018457+0000 front 2024-04-24T09:19:49.018520+0000 (oldest deadline 2024-04-24T09:20:16.019822+0000) 2024-04-24T09:20:50.208 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:50 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:50.024+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:07.025175+0000 front 2024-04-24T09:20:07.025338+0000 (oldest deadline 2024-04-24T09:20:31.125415+0000) 2024-04-24T09:20:50.208 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:50 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:50.024+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-24T09:20:17.027505+0000 front 2024-04-24T09:20:17.028269+0000 (oldest deadline 2024-04-24T09:20:42.327601+0000) 2024-04-24T09:20:50.554 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:50 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:50.294+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:27.386588+0000 front 2024-04-24T09:19:27.386891+0000 (oldest deadline 2024-04-24T09:19:52.686559+0000) 2024-04-24T09:20:50.554 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:50 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:50.294+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.891162+0000 front 2024-04-24T09:19:38.891270+0000 (oldest deadline 2024-04-24T09:20:04.790877+0000) 2024-04-24T09:20:50.554 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:50 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:50.294+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:48.794838+0000 front 2024-04-24T09:19:48.794972+0000 (oldest deadline 2024-04-24T09:20:14.095077+0000) 2024-04-24T09:20:50.554 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:50 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:50.294+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:56.997332+0000 front 2024-04-24T09:19:56.997477+0000 (oldest deadline 2024-04-24T09:20:22.897608+0000) 2024-04-24T09:20:50.554 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:50 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:50.294+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:06.400442+0000 front 2024-04-24T09:20:06.400694+0000 (oldest deadline 2024-04-24T09:20:31.099904+0000) 2024-04-24T09:20:50.554 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:50 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:50.294+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-24T09:20:15.202727+0000 front 2024-04-24T09:20:15.202777+0000 (oldest deadline 2024-04-24T09:20:39.902935+0000) 2024-04-24T09:20:50.655 DEBUG:teuthology.orchestra.run.smithi189:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bd3e039e-0215-11ef-bc93-c7b262605968@osd.8.service' 2024-04-24T09:20:50.762 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T09:20:50.763 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-04-24T09:20:50.763 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-04-24T09:20:50.763 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-bd3e039e-0215-11ef-bc93-c7b262605968@osd.9 2024-04-24T09:20:51.205 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:50 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:50.792+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:29.414806+0000 front 2024-04-24T09:19:29.414861+0000 (oldest deadline 2024-04-24T09:19:55.314800+0000) 2024-04-24T09:20:51.205 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:50 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:50.792+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.119965+0000 front 2024-04-24T09:19:38.120034+0000 (oldest deadline 2024-04-24T09:20:02.820181+0000) 2024-04-24T09:20:51.205 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:50 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:50.792+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:47.326279+0000 front 2024-04-24T09:19:47.326338+0000 (oldest deadline 2024-04-24T09:20:10.728055+0000) 2024-04-24T09:20:51.205 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:50 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:50.792+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:55.832608+0000 front 2024-04-24T09:19:55.832510+0000 (oldest deadline 2024-04-24T09:20:21.732750+0000) 2024-04-24T09:20:51.206 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:50 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:50.792+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:08.736362+0000 front 2024-04-24T09:20:08.736680+0000 (oldest deadline 2024-04-24T09:20:31.537874+0000) 2024-04-24T09:20:51.206 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:50 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:50.792+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-24T09:20:16.240265+0000 front 2024-04-24T09:20:16.240144+0000 (oldest deadline 2024-04-24T09:20:41.540224+0000) 2024-04-24T09:20:51.246 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:50 smithi083 systemd[1]: Stopping Ceph osd.9 for bd3e039e-0215-11ef-bc93-c7b262605968... 2024-04-24T09:20:51.246 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:51 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:51.246+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:27.386588+0000 front 2024-04-24T09:19:27.386891+0000 (oldest deadline 2024-04-24T09:19:52.686559+0000) 2024-04-24T09:20:51.247 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:51 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:51.246+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.891162+0000 front 2024-04-24T09:19:38.891270+0000 (oldest deadline 2024-04-24T09:20:04.790877+0000) 2024-04-24T09:20:51.247 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:51 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:51.246+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:48.794838+0000 front 2024-04-24T09:19:48.794972+0000 (oldest deadline 2024-04-24T09:20:14.095077+0000) 2024-04-24T09:20:51.247 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:51 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:51.246+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:56.997332+0000 front 2024-04-24T09:19:56.997477+0000 (oldest deadline 2024-04-24T09:20:22.897608+0000) 2024-04-24T09:20:51.247 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:51 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:51.246+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:06.400442+0000 front 2024-04-24T09:20:06.400694+0000 (oldest deadline 2024-04-24T09:20:31.099904+0000) 2024-04-24T09:20:51.247 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:51 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:51.246+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-24T09:20:15.202727+0000 front 2024-04-24T09:20:15.202777+0000 (oldest deadline 2024-04-24T09:20:39.902935+0000) 2024-04-24T09:20:51.457 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:50 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:50.987+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:28.209530+0000 front 2024-04-24T09:19:28.209142+0000 (oldest deadline 2024-04-24T09:19:51.709380+0000) 2024-04-24T09:20:51.457 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:50 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:50.987+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.613982+0000 front 2024-04-24T09:19:38.614541+0000 (oldest deadline 2024-04-24T09:20:00.913987+0000) 2024-04-24T09:20:51.457 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:50 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:50.987+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:49.018457+0000 front 2024-04-24T09:19:49.018520+0000 (oldest deadline 2024-04-24T09:20:16.019822+0000) 2024-04-24T09:20:51.458 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:50 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:50.987+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:07.025175+0000 front 2024-04-24T09:20:07.025338+0000 (oldest deadline 2024-04-24T09:20:31.125415+0000) 2024-04-24T09:20:51.458 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:50 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:50.987+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-24T09:20:17.027505+0000 front 2024-04-24T09:20:17.028269+0000 (oldest deadline 2024-04-24T09:20:42.327601+0000) 2024-04-24T09:20:51.458 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:50 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:50.987+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-04-24T09:20:27.431096+0000 front 2024-04-24T09:20:27.431264+0000 (oldest deadline 2024-04-24T09:20:50.930971+0000) 2024-04-24T09:20:51.553 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:51 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:51.353+0000 7ff3e7000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T09:20:51.553 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:51 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:51.353+0000 7ff3e7000700 -1 osd.9 141 *** Got signal Terminated *** 2024-04-24T09:20:51.553 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:51 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:51.353+0000 7ff3e7000700 -1 osd.9 141 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T09:20:52.134 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:51 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:51.781+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:29.414806+0000 front 2024-04-24T09:19:29.414861+0000 (oldest deadline 2024-04-24T09:19:55.314800+0000) 2024-04-24T09:20:52.134 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:51 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:51.781+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.119965+0000 front 2024-04-24T09:19:38.120034+0000 (oldest deadline 2024-04-24T09:20:02.820181+0000) 2024-04-24T09:20:52.134 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:51 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:51.781+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:47.326279+0000 front 2024-04-24T09:19:47.326338+0000 (oldest deadline 2024-04-24T09:20:10.728055+0000) 2024-04-24T09:20:52.134 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:51 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:51.781+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:55.832608+0000 front 2024-04-24T09:19:55.832510+0000 (oldest deadline 2024-04-24T09:20:21.732750+0000) 2024-04-24T09:20:52.134 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:51 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:51.781+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:08.736362+0000 front 2024-04-24T09:20:08.736680+0000 (oldest deadline 2024-04-24T09:20:31.537874+0000) 2024-04-24T09:20:52.134 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:51 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:51.781+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-24T09:20:16.240265+0000 front 2024-04-24T09:20:16.240144+0000 (oldest deadline 2024-04-24T09:20:41.540224+0000) 2024-04-24T09:20:52.457 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:52 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:52.031+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:28.209530+0000 front 2024-04-24T09:19:28.209142+0000 (oldest deadline 2024-04-24T09:19:51.709380+0000) 2024-04-24T09:20:52.457 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:52 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:52.031+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.613982+0000 front 2024-04-24T09:19:38.614541+0000 (oldest deadline 2024-04-24T09:20:00.913987+0000) 2024-04-24T09:20:52.457 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:52 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:52.031+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:49.018457+0000 front 2024-04-24T09:19:49.018520+0000 (oldest deadline 2024-04-24T09:20:16.019822+0000) 2024-04-24T09:20:52.457 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:52 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:52.031+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:07.025175+0000 front 2024-04-24T09:20:07.025338+0000 (oldest deadline 2024-04-24T09:20:31.125415+0000) 2024-04-24T09:20:52.457 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:52 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:52.031+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-24T09:20:17.027505+0000 front 2024-04-24T09:20:17.028269+0000 (oldest deadline 2024-04-24T09:20:42.327601+0000) 2024-04-24T09:20:52.458 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:52 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:52.031+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-04-24T09:20:27.431096+0000 front 2024-04-24T09:20:27.431264+0000 (oldest deadline 2024-04-24T09:20:50.930971+0000) 2024-04-24T09:20:52.553 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:52 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:52.228+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:27.386588+0000 front 2024-04-24T09:19:27.386891+0000 (oldest deadline 2024-04-24T09:19:52.686559+0000) 2024-04-24T09:20:52.553 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:52 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:52.228+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.891162+0000 front 2024-04-24T09:19:38.891270+0000 (oldest deadline 2024-04-24T09:20:04.790877+0000) 2024-04-24T09:20:52.553 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:52 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:52.228+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:48.794838+0000 front 2024-04-24T09:19:48.794972+0000 (oldest deadline 2024-04-24T09:20:14.095077+0000) 2024-04-24T09:20:52.553 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:52 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:52.228+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:56.997332+0000 front 2024-04-24T09:19:56.997477+0000 (oldest deadline 2024-04-24T09:20:22.897608+0000) 2024-04-24T09:20:52.553 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:52 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:52.228+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:06.400442+0000 front 2024-04-24T09:20:06.400694+0000 (oldest deadline 2024-04-24T09:20:31.099904+0000) 2024-04-24T09:20:52.554 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:52 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:52.228+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-24T09:20:15.202727+0000 front 2024-04-24T09:20:15.202777+0000 (oldest deadline 2024-04-24T09:20:39.902935+0000) 2024-04-24T09:20:53.205 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:52 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:52.829+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:29.414806+0000 front 2024-04-24T09:19:29.414861+0000 (oldest deadline 2024-04-24T09:19:55.314800+0000) 2024-04-24T09:20:53.205 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:52 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:52.829+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.119965+0000 front 2024-04-24T09:19:38.120034+0000 (oldest deadline 2024-04-24T09:20:02.820181+0000) 2024-04-24T09:20:53.205 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:52 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:52.829+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:47.326279+0000 front 2024-04-24T09:19:47.326338+0000 (oldest deadline 2024-04-24T09:20:10.728055+0000) 2024-04-24T09:20:53.205 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:52 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:52.829+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:55.832608+0000 front 2024-04-24T09:19:55.832510+0000 (oldest deadline 2024-04-24T09:20:21.732750+0000) 2024-04-24T09:20:53.205 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:52 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:52.829+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:08.736362+0000 front 2024-04-24T09:20:08.736680+0000 (oldest deadline 2024-04-24T09:20:31.537874+0000) 2024-04-24T09:20:53.205 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:52 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:52.829+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-24T09:20:16.240265+0000 front 2024-04-24T09:20:16.240144+0000 (oldest deadline 2024-04-24T09:20:41.540224+0000) 2024-04-24T09:20:53.457 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:52 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:52.987+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:28.209530+0000 front 2024-04-24T09:19:28.209142+0000 (oldest deadline 2024-04-24T09:19:51.709380+0000) 2024-04-24T09:20:53.457 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:52 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:52.987+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.613982+0000 front 2024-04-24T09:19:38.614541+0000 (oldest deadline 2024-04-24T09:20:00.913987+0000) 2024-04-24T09:20:53.458 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:52 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:52.987+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:49.018457+0000 front 2024-04-24T09:19:49.018520+0000 (oldest deadline 2024-04-24T09:20:16.019822+0000) 2024-04-24T09:20:53.458 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:52 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:52.987+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:07.025175+0000 front 2024-04-24T09:20:07.025338+0000 (oldest deadline 2024-04-24T09:20:31.125415+0000) 2024-04-24T09:20:53.458 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:52 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:52.987+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-24T09:20:17.027505+0000 front 2024-04-24T09:20:17.028269+0000 (oldest deadline 2024-04-24T09:20:42.327601+0000) 2024-04-24T09:20:53.458 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:52 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:52.987+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-04-24T09:20:27.431096+0000 front 2024-04-24T09:20:27.431264+0000 (oldest deadline 2024-04-24T09:20:50.930971+0000) 2024-04-24T09:20:53.554 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:53 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:53.229+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:27.386588+0000 front 2024-04-24T09:19:27.386891+0000 (oldest deadline 2024-04-24T09:19:52.686559+0000) 2024-04-24T09:20:53.554 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:53 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:53.229+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.891162+0000 front 2024-04-24T09:19:38.891270+0000 (oldest deadline 2024-04-24T09:20:04.790877+0000) 2024-04-24T09:20:53.554 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:53 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:53.229+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:48.794838+0000 front 2024-04-24T09:19:48.794972+0000 (oldest deadline 2024-04-24T09:20:14.095077+0000) 2024-04-24T09:20:53.554 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:53 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:53.229+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:56.997332+0000 front 2024-04-24T09:19:56.997477+0000 (oldest deadline 2024-04-24T09:20:22.897608+0000) 2024-04-24T09:20:53.554 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:53 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:53.229+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:06.400442+0000 front 2024-04-24T09:20:06.400694+0000 (oldest deadline 2024-04-24T09:20:31.099904+0000) 2024-04-24T09:20:53.554 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:53 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:53.229+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-24T09:20:15.202727+0000 front 2024-04-24T09:20:15.202777+0000 (oldest deadline 2024-04-24T09:20:39.902935+0000) 2024-04-24T09:20:54.205 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:53 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:53.875+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:29.414806+0000 front 2024-04-24T09:19:29.414861+0000 (oldest deadline 2024-04-24T09:19:55.314800+0000) 2024-04-24T09:20:54.205 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:53 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:53.875+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.119965+0000 front 2024-04-24T09:19:38.120034+0000 (oldest deadline 2024-04-24T09:20:02.820181+0000) 2024-04-24T09:20:54.205 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:53 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:53.875+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:47.326279+0000 front 2024-04-24T09:19:47.326338+0000 (oldest deadline 2024-04-24T09:20:10.728055+0000) 2024-04-24T09:20:54.205 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:53 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:53.875+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:55.832608+0000 front 2024-04-24T09:19:55.832510+0000 (oldest deadline 2024-04-24T09:20:21.732750+0000) 2024-04-24T09:20:54.206 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:53 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:53.875+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:08.736362+0000 front 2024-04-24T09:20:08.736680+0000 (oldest deadline 2024-04-24T09:20:31.537874+0000) 2024-04-24T09:20:54.206 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:53 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:53.875+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-24T09:20:16.240265+0000 front 2024-04-24T09:20:16.240144+0000 (oldest deadline 2024-04-24T09:20:41.540224+0000) 2024-04-24T09:20:54.206 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:53 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:53.875+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-04-24T09:20:27.442280+0000 front 2024-04-24T09:20:27.441889+0000 (oldest deadline 2024-04-24T09:20:53.341732+0000) 2024-04-24T09:20:54.457 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:54 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:54.035+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:28.209530+0000 front 2024-04-24T09:19:28.209142+0000 (oldest deadline 2024-04-24T09:19:51.709380+0000) 2024-04-24T09:20:54.457 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:54 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:54.035+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.613982+0000 front 2024-04-24T09:19:38.614541+0000 (oldest deadline 2024-04-24T09:20:00.913987+0000) 2024-04-24T09:20:54.458 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:54 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:54.035+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:49.018457+0000 front 2024-04-24T09:19:49.018520+0000 (oldest deadline 2024-04-24T09:20:16.019822+0000) 2024-04-24T09:20:54.458 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:54 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:54.035+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:07.025175+0000 front 2024-04-24T09:20:07.025338+0000 (oldest deadline 2024-04-24T09:20:31.125415+0000) 2024-04-24T09:20:54.458 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:54 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:54.035+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-24T09:20:17.027505+0000 front 2024-04-24T09:20:17.028269+0000 (oldest deadline 2024-04-24T09:20:42.327601+0000) 2024-04-24T09:20:54.458 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:54 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:54.035+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-04-24T09:20:27.431096+0000 front 2024-04-24T09:20:27.431264+0000 (oldest deadline 2024-04-24T09:20:50.930971+0000) 2024-04-24T09:20:54.554 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:54 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:54.275+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:27.386588+0000 front 2024-04-24T09:19:27.386891+0000 (oldest deadline 2024-04-24T09:19:52.686559+0000) 2024-04-24T09:20:54.554 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:54 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:54.275+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.891162+0000 front 2024-04-24T09:19:38.891270+0000 (oldest deadline 2024-04-24T09:20:04.790877+0000) 2024-04-24T09:20:54.554 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:54 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:54.275+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:48.794838+0000 front 2024-04-24T09:19:48.794972+0000 (oldest deadline 2024-04-24T09:20:14.095077+0000) 2024-04-24T09:20:54.554 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:54 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:54.275+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:56.997332+0000 front 2024-04-24T09:19:56.997477+0000 (oldest deadline 2024-04-24T09:20:22.897608+0000) 2024-04-24T09:20:54.554 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:54 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:54.275+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:06.400442+0000 front 2024-04-24T09:20:06.400694+0000 (oldest deadline 2024-04-24T09:20:31.099904+0000) 2024-04-24T09:20:54.554 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:54 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:54.275+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-24T09:20:15.202727+0000 front 2024-04-24T09:20:15.202777+0000 (oldest deadline 2024-04-24T09:20:39.902935+0000) 2024-04-24T09:20:55.205 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:54 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:54.855+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:29.414806+0000 front 2024-04-24T09:19:29.414861+0000 (oldest deadline 2024-04-24T09:19:55.314800+0000) 2024-04-24T09:20:55.205 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:54 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:54.855+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.119965+0000 front 2024-04-24T09:19:38.120034+0000 (oldest deadline 2024-04-24T09:20:02.820181+0000) 2024-04-24T09:20:55.205 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:54 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:54.855+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:47.326279+0000 front 2024-04-24T09:19:47.326338+0000 (oldest deadline 2024-04-24T09:20:10.728055+0000) 2024-04-24T09:20:55.206 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:54 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:54.855+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:55.832608+0000 front 2024-04-24T09:19:55.832510+0000 (oldest deadline 2024-04-24T09:20:21.732750+0000) 2024-04-24T09:20:55.206 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:54 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:54.855+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:08.736362+0000 front 2024-04-24T09:20:08.736680+0000 (oldest deadline 2024-04-24T09:20:31.537874+0000) 2024-04-24T09:20:55.206 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:54 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:54.855+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-24T09:20:16.240265+0000 front 2024-04-24T09:20:16.240144+0000 (oldest deadline 2024-04-24T09:20:41.540224+0000) 2024-04-24T09:20:55.206 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:54 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:54.855+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-04-24T09:20:27.442280+0000 front 2024-04-24T09:20:27.441889+0000 (oldest deadline 2024-04-24T09:20:53.341732+0000) 2024-04-24T09:20:55.457 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:55 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:55.082+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:28.209530+0000 front 2024-04-24T09:19:28.209142+0000 (oldest deadline 2024-04-24T09:19:51.709380+0000) 2024-04-24T09:20:55.457 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:55 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:55.082+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.613982+0000 front 2024-04-24T09:19:38.614541+0000 (oldest deadline 2024-04-24T09:20:00.913987+0000) 2024-04-24T09:20:55.457 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:55 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:55.082+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:49.018457+0000 front 2024-04-24T09:19:49.018520+0000 (oldest deadline 2024-04-24T09:20:16.019822+0000) 2024-04-24T09:20:55.457 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:55 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:55.082+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:07.025175+0000 front 2024-04-24T09:20:07.025338+0000 (oldest deadline 2024-04-24T09:20:31.125415+0000) 2024-04-24T09:20:55.458 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:55 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:55.082+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-24T09:20:17.027505+0000 front 2024-04-24T09:20:17.028269+0000 (oldest deadline 2024-04-24T09:20:42.327601+0000) 2024-04-24T09:20:55.458 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:55 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:55.082+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-04-24T09:20:27.431096+0000 front 2024-04-24T09:20:27.431264+0000 (oldest deadline 2024-04-24T09:20:50.930971+0000) 2024-04-24T09:20:55.554 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:55 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:55.266+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:27.386588+0000 front 2024-04-24T09:19:27.386891+0000 (oldest deadline 2024-04-24T09:19:52.686559+0000) 2024-04-24T09:20:55.554 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:55 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:55.266+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.891162+0000 front 2024-04-24T09:19:38.891270+0000 (oldest deadline 2024-04-24T09:20:04.790877+0000) 2024-04-24T09:20:55.554 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:55 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:55.266+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:48.794838+0000 front 2024-04-24T09:19:48.794972+0000 (oldest deadline 2024-04-24T09:20:14.095077+0000) 2024-04-24T09:20:55.554 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:55 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:55.266+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:56.997332+0000 front 2024-04-24T09:19:56.997477+0000 (oldest deadline 2024-04-24T09:20:22.897608+0000) 2024-04-24T09:20:55.554 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:55 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:55.266+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:06.400442+0000 front 2024-04-24T09:20:06.400694+0000 (oldest deadline 2024-04-24T09:20:31.099904+0000) 2024-04-24T09:20:55.554 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:55 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:55.266+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-24T09:20:15.202727+0000 front 2024-04-24T09:20:15.202777+0000 (oldest deadline 2024-04-24T09:20:39.902935+0000) 2024-04-24T09:20:56.205 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:55 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:55.901+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:29.414806+0000 front 2024-04-24T09:19:29.414861+0000 (oldest deadline 2024-04-24T09:19:55.314800+0000) 2024-04-24T09:20:56.205 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:55 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:55.901+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.119965+0000 front 2024-04-24T09:19:38.120034+0000 (oldest deadline 2024-04-24T09:20:02.820181+0000) 2024-04-24T09:20:56.205 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:55 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:55.901+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:47.326279+0000 front 2024-04-24T09:19:47.326338+0000 (oldest deadline 2024-04-24T09:20:10.728055+0000) 2024-04-24T09:20:56.205 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:55 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:55.901+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:55.832608+0000 front 2024-04-24T09:19:55.832510+0000 (oldest deadline 2024-04-24T09:20:21.732750+0000) 2024-04-24T09:20:56.206 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:55 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:55.901+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:08.736362+0000 front 2024-04-24T09:20:08.736680+0000 (oldest deadline 2024-04-24T09:20:31.537874+0000) 2024-04-24T09:20:56.206 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:55 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:55.901+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-24T09:20:16.240265+0000 front 2024-04-24T09:20:16.240144+0000 (oldest deadline 2024-04-24T09:20:41.540224+0000) 2024-04-24T09:20:56.206 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:55 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:55.901+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-04-24T09:20:27.442280+0000 front 2024-04-24T09:20:27.441889+0000 (oldest deadline 2024-04-24T09:20:53.341732+0000) 2024-04-24T09:20:56.457 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:56 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:56.110+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:28.209530+0000 front 2024-04-24T09:19:28.209142+0000 (oldest deadline 2024-04-24T09:19:51.709380+0000) 2024-04-24T09:20:56.457 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:56 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:56.110+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.613982+0000 front 2024-04-24T09:19:38.614541+0000 (oldest deadline 2024-04-24T09:20:00.913987+0000) 2024-04-24T09:20:56.457 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:56 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:56.110+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:49.018457+0000 front 2024-04-24T09:19:49.018520+0000 (oldest deadline 2024-04-24T09:20:16.019822+0000) 2024-04-24T09:20:56.457 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:56 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:56.110+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:07.025175+0000 front 2024-04-24T09:20:07.025338+0000 (oldest deadline 2024-04-24T09:20:31.125415+0000) 2024-04-24T09:20:56.457 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:56 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:56.110+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-24T09:20:17.027505+0000 front 2024-04-24T09:20:17.028269+0000 (oldest deadline 2024-04-24T09:20:42.327601+0000) 2024-04-24T09:20:56.458 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:56 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:56.110+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-04-24T09:20:27.431096+0000 front 2024-04-24T09:20:27.431264+0000 (oldest deadline 2024-04-24T09:20:50.930971+0000) 2024-04-24T09:20:56.553 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:56 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:56.303+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:27.386588+0000 front 2024-04-24T09:19:27.386891+0000 (oldest deadline 2024-04-24T09:19:52.686559+0000) 2024-04-24T09:20:56.553 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:56 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:56.303+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.891162+0000 front 2024-04-24T09:19:38.891270+0000 (oldest deadline 2024-04-24T09:20:04.790877+0000) 2024-04-24T09:20:56.553 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:56 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:56.303+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:48.794838+0000 front 2024-04-24T09:19:48.794972+0000 (oldest deadline 2024-04-24T09:20:14.095077+0000) 2024-04-24T09:20:56.553 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:56 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:56.303+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:56.997332+0000 front 2024-04-24T09:19:56.997477+0000 (oldest deadline 2024-04-24T09:20:22.897608+0000) 2024-04-24T09:20:56.554 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:56 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:56.303+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:06.400442+0000 front 2024-04-24T09:20:06.400694+0000 (oldest deadline 2024-04-24T09:20:31.099904+0000) 2024-04-24T09:20:56.554 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:56 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9[62867]: 2024-04-24T09:20:56.303+0000 7ff3dd800700 -1 osd.9 141 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-24T09:20:15.202727+0000 front 2024-04-24T09:20:15.202777+0000 (oldest deadline 2024-04-24T09:20:39.902935+0000) 2024-04-24T09:20:57.205 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:56 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:56.931+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:29.414806+0000 front 2024-04-24T09:19:29.414861+0000 (oldest deadline 2024-04-24T09:19:55.314800+0000) 2024-04-24T09:20:57.205 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:56 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:56.931+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.119965+0000 front 2024-04-24T09:19:38.120034+0000 (oldest deadline 2024-04-24T09:20:02.820181+0000) 2024-04-24T09:20:57.205 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:56 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:56.931+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:47.326279+0000 front 2024-04-24T09:19:47.326338+0000 (oldest deadline 2024-04-24T09:20:10.728055+0000) 2024-04-24T09:20:57.206 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:56 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:56.931+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:55.832608+0000 front 2024-04-24T09:19:55.832510+0000 (oldest deadline 2024-04-24T09:20:21.732750+0000) 2024-04-24T09:20:57.206 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:56 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:56.931+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:08.736362+0000 front 2024-04-24T09:20:08.736680+0000 (oldest deadline 2024-04-24T09:20:31.537874+0000) 2024-04-24T09:20:57.206 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:56 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:56.931+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-24T09:20:16.240265+0000 front 2024-04-24T09:20:16.240144+0000 (oldest deadline 2024-04-24T09:20:41.540224+0000) 2024-04-24T09:20:57.206 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:56 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:56.931+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-04-24T09:20:27.442280+0000 front 2024-04-24T09:20:27.441889+0000 (oldest deadline 2024-04-24T09:20:53.341732+0000) 2024-04-24T09:20:57.375 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:57 smithi083 podman[132136]: 2024-04-24 09:20:57.122624557 +0000 UTC m=+5.933365185 container died e394d0129690b101e52ae3065e0dc259d764b7794c033e04ca954c8222b6cf27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, RELEASE=rishabh-main-apr17-a654945-1f81fb3, io.buildah.version=1.33.5) 2024-04-24T09:20:57.457 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:57 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:57.147+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:28.209530+0000 front 2024-04-24T09:19:28.209142+0000 (oldest deadline 2024-04-24T09:19:51.709380+0000) 2024-04-24T09:20:57.457 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:57 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:57.147+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.613982+0000 front 2024-04-24T09:19:38.614541+0000 (oldest deadline 2024-04-24T09:20:00.913987+0000) 2024-04-24T09:20:57.457 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:57 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:57.147+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:49.018457+0000 front 2024-04-24T09:19:49.018520+0000 (oldest deadline 2024-04-24T09:20:16.019822+0000) 2024-04-24T09:20:57.457 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:57 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:57.147+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:07.025175+0000 front 2024-04-24T09:20:07.025338+0000 (oldest deadline 2024-04-24T09:20:31.125415+0000) 2024-04-24T09:20:57.457 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:57 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:57.147+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-24T09:20:17.027505+0000 front 2024-04-24T09:20:17.028269+0000 (oldest deadline 2024-04-24T09:20:42.327601+0000) 2024-04-24T09:20:57.458 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:57 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:57.147+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-04-24T09:20:27.431096+0000 front 2024-04-24T09:20:27.431264+0000 (oldest deadline 2024-04-24T09:20:50.930971+0000) 2024-04-24T09:20:57.654 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:57 smithi083 podman[132136]: 2024-04-24 09:20:57.478149579 +0000 UTC m=+6.288890202 container cleanup e394d0129690b101e52ae3065e0dc259d764b7794c033e04ca954c8222b6cf27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9, RELEASE=rishabh-main-apr17-a654945-1f81fb3, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-04-24T09:20:57.654 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:57 smithi083 bash[132136]: ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9 2024-04-24T09:20:58.053 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:57 smithi083 podman[132164]: 2024-04-24 09:20:57.654037353 +0000 UTC m=+0.521942275 container remove e394d0129690b101e52ae3065e0dc259d764b7794c033e04ca954c8222b6cf27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=rishabh-main-apr17-a654945-1f81fb3, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240424) 2024-04-24T09:20:58.205 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:57 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:57.936+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:29.414806+0000 front 2024-04-24T09:19:29.414861+0000 (oldest deadline 2024-04-24T09:19:55.314800+0000) 2024-04-24T09:20:58.205 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:57 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:57.936+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.119965+0000 front 2024-04-24T09:19:38.120034+0000 (oldest deadline 2024-04-24T09:20:02.820181+0000) 2024-04-24T09:20:58.205 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:57 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:57.936+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:47.326279+0000 front 2024-04-24T09:19:47.326338+0000 (oldest deadline 2024-04-24T09:20:10.728055+0000) 2024-04-24T09:20:58.205 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:57 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:57.936+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:55.832608+0000 front 2024-04-24T09:19:55.832510+0000 (oldest deadline 2024-04-24T09:20:21.732750+0000) 2024-04-24T09:20:58.205 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:57 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:57.936+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:08.736362+0000 front 2024-04-24T09:20:08.736680+0000 (oldest deadline 2024-04-24T09:20:31.537874+0000) 2024-04-24T09:20:58.206 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:57 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:57.936+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-24T09:20:16.240265+0000 front 2024-04-24T09:20:16.240144+0000 (oldest deadline 2024-04-24T09:20:41.540224+0000) 2024-04-24T09:20:58.206 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:57 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:57.936+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-04-24T09:20:27.442280+0000 front 2024-04-24T09:20:27.441889+0000 (oldest deadline 2024-04-24T09:20:53.341732+0000) 2024-04-24T09:20:58.457 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:58 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:58.138+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:28.209530+0000 front 2024-04-24T09:19:28.209142+0000 (oldest deadline 2024-04-24T09:19:51.709380+0000) 2024-04-24T09:20:58.457 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:58 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:58.138+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.613982+0000 front 2024-04-24T09:19:38.614541+0000 (oldest deadline 2024-04-24T09:20:00.913987+0000) 2024-04-24T09:20:58.457 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:58 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:58.138+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:49.018457+0000 front 2024-04-24T09:19:49.018520+0000 (oldest deadline 2024-04-24T09:20:16.019822+0000) 2024-04-24T09:20:58.457 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:58 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:58.138+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:07.025175+0000 front 2024-04-24T09:20:07.025338+0000 (oldest deadline 2024-04-24T09:20:31.125415+0000) 2024-04-24T09:20:58.457 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:58 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:58.138+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-24T09:20:17.027505+0000 front 2024-04-24T09:20:17.028269+0000 (oldest deadline 2024-04-24T09:20:42.327601+0000) 2024-04-24T09:20:58.457 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:58 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:58.138+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-04-24T09:20:27.431096+0000 front 2024-04-24T09:20:27.431264+0000 (oldest deadline 2024-04-24T09:20:50.930971+0000) 2024-04-24T09:20:58.800 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:58 smithi083 podman[132252]: 2024-04-24 09:20:58.400970444 +0000 UTC m=+0.035551395 image pull b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644 2024-04-24T09:20:58.800 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:58 smithi083 podman[132252]: 2024-04-24 09:20:58.513102393 +0000 UTC m=+0.147683369 container create d5317394c43969bf42485e271bea2071431cf7634389e3eb2202cb64effe1656 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T09:20:58.800 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:58 smithi083 podman[132252]: 2024-04-24 09:20:58.800188961 +0000 UTC m=+0.434769947 container init d5317394c43969bf42485e271bea2071431cf7634389e3eb2202cb64effe1656 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=rishabh-main-apr17-a654945-1f81fb3, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T09:20:59.053 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:58 smithi083 podman[132252]: 2024-04-24 09:20:58.812182972 +0000 UTC m=+0.446763918 container start d5317394c43969bf42485e271bea2071431cf7634389e3eb2202cb64effe1656 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=rishabh-main-apr17-a654945-1f81fb3, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-04-24T09:20:59.053 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:58 smithi083 podman[132252]: 2024-04-24 09:20:58.899053702 +0000 UTC m=+0.533634728 container attach d5317394c43969bf42485e271bea2071431cf7634389e3eb2202cb64effe1656 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T09:20:59.205 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:58 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:58.900+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:29.414806+0000 front 2024-04-24T09:19:29.414861+0000 (oldest deadline 2024-04-24T09:19:55.314800+0000) 2024-04-24T09:20:59.205 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:58 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:58.900+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.119965+0000 front 2024-04-24T09:19:38.120034+0000 (oldest deadline 2024-04-24T09:20:02.820181+0000) 2024-04-24T09:20:59.205 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:58 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:58.900+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:47.326279+0000 front 2024-04-24T09:19:47.326338+0000 (oldest deadline 2024-04-24T09:20:10.728055+0000) 2024-04-24T09:20:59.205 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:58 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:58.900+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:55.832608+0000 front 2024-04-24T09:19:55.832510+0000 (oldest deadline 2024-04-24T09:20:21.732750+0000) 2024-04-24T09:20:59.205 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:58 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:58.900+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:08.736362+0000 front 2024-04-24T09:20:08.736680+0000 (oldest deadline 2024-04-24T09:20:31.537874+0000) 2024-04-24T09:20:59.205 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:58 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:58.900+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-24T09:20:16.240265+0000 front 2024-04-24T09:20:16.240144+0000 (oldest deadline 2024-04-24T09:20:41.540224+0000) 2024-04-24T09:20:59.205 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:58 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:58.900+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-04-24T09:20:27.442280+0000 front 2024-04-24T09:20:27.441889+0000 (oldest deadline 2024-04-24T09:20:53.341732+0000) 2024-04-24T09:20:59.457 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:59 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:59.177+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:28.209530+0000 front 2024-04-24T09:19:28.209142+0000 (oldest deadline 2024-04-24T09:19:51.709380+0000) 2024-04-24T09:20:59.457 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:59 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:59.177+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.613982+0000 front 2024-04-24T09:19:38.614541+0000 (oldest deadline 2024-04-24T09:20:00.913987+0000) 2024-04-24T09:20:59.457 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:59 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:59.177+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:49.018457+0000 front 2024-04-24T09:19:49.018520+0000 (oldest deadline 2024-04-24T09:20:16.019822+0000) 2024-04-24T09:20:59.457 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:59 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:59.177+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:07.025175+0000 front 2024-04-24T09:20:07.025338+0000 (oldest deadline 2024-04-24T09:20:31.125415+0000) 2024-04-24T09:20:59.458 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:59 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:59.177+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-24T09:20:17.027505+0000 front 2024-04-24T09:20:17.028269+0000 (oldest deadline 2024-04-24T09:20:42.327601+0000) 2024-04-24T09:20:59.458 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:20:59 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:20:59.177+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-04-24T09:20:27.431096+0000 front 2024-04-24T09:20:27.431264+0000 (oldest deadline 2024-04-24T09:20:50.930971+0000) 2024-04-24T09:20:59.553 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 24 09:20:59 smithi083 podman[132252]: 2024-04-24 09:20:59.267305916 +0000 UTC m=+0.901886906 container died d5317394c43969bf42485e271bea2071431cf7634389e3eb2202cb64effe1656 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-24T09:20:59.953 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bd3e039e-0215-11ef-bc93-c7b262605968@osd.9.service' 2024-04-24T09:21:00.067 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T09:21:00.068 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-04-24T09:21:00.068 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-04-24T09:21:00.068 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ceph-bd3e039e-0215-11ef-bc93-c7b262605968@osd.10 2024-04-24T09:21:00.205 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:59 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:59.937+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:29.414806+0000 front 2024-04-24T09:19:29.414861+0000 (oldest deadline 2024-04-24T09:19:55.314800+0000) 2024-04-24T09:21:00.205 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:59 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:59.937+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.119965+0000 front 2024-04-24T09:19:38.120034+0000 (oldest deadline 2024-04-24T09:20:02.820181+0000) 2024-04-24T09:21:00.205 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:59 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:59.937+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:47.326279+0000 front 2024-04-24T09:19:47.326338+0000 (oldest deadline 2024-04-24T09:20:10.728055+0000) 2024-04-24T09:21:00.205 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:59 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:59.937+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:55.832608+0000 front 2024-04-24T09:19:55.832510+0000 (oldest deadline 2024-04-24T09:20:21.732750+0000) 2024-04-24T09:21:00.205 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:59 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:59.937+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:08.736362+0000 front 2024-04-24T09:20:08.736680+0000 (oldest deadline 2024-04-24T09:20:31.537874+0000) 2024-04-24T09:21:00.206 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:59 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:59.937+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-24T09:20:16.240265+0000 front 2024-04-24T09:20:16.240144+0000 (oldest deadline 2024-04-24T09:20:41.540224+0000) 2024-04-24T09:21:00.206 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:20:59 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:20:59.937+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-04-24T09:20:27.442280+0000 front 2024-04-24T09:20:27.441889+0000 (oldest deadline 2024-04-24T09:20:53.341732+0000) 2024-04-24T09:21:00.206 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:21:00 smithi138 systemd[1]: Stopping Ceph osd.10 for bd3e039e-0215-11ef-bc93-c7b262605968... 2024-04-24T09:21:00.457 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:00 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:21:00.137+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:28.209530+0000 front 2024-04-24T09:19:28.209142+0000 (oldest deadline 2024-04-24T09:19:51.709380+0000) 2024-04-24T09:21:00.457 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:00 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:21:00.137+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.613982+0000 front 2024-04-24T09:19:38.614541+0000 (oldest deadline 2024-04-24T09:20:00.913987+0000) 2024-04-24T09:21:00.457 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:00 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:21:00.137+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:49.018457+0000 front 2024-04-24T09:19:49.018520+0000 (oldest deadline 2024-04-24T09:20:16.019822+0000) 2024-04-24T09:21:00.457 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:00 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:21:00.137+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:07.025175+0000 front 2024-04-24T09:20:07.025338+0000 (oldest deadline 2024-04-24T09:20:31.125415+0000) 2024-04-24T09:21:00.457 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:00 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:21:00.137+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-24T09:20:17.027505+0000 front 2024-04-24T09:20:17.028269+0000 (oldest deadline 2024-04-24T09:20:42.327601+0000) 2024-04-24T09:21:00.458 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:00 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:21:00.137+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-04-24T09:20:27.431096+0000 front 2024-04-24T09:20:27.431264+0000 (oldest deadline 2024-04-24T09:20:50.930971+0000) 2024-04-24T09:21:00.955 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:21:00 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:21:00.656+0000 7f3302800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T09:21:00.955 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:21:00 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:21:00.656+0000 7f3302800700 -1 osd.10 141 *** Got signal Terminated *** 2024-04-24T09:21:00.955 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:21:00 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:21:00.656+0000 7f3302800700 -1 osd.10 141 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T09:21:01.455 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:21:00 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:21:00.957+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:29.414806+0000 front 2024-04-24T09:19:29.414861+0000 (oldest deadline 2024-04-24T09:19:55.314800+0000) 2024-04-24T09:21:01.456 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:21:00 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:21:00.957+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.119965+0000 front 2024-04-24T09:19:38.120034+0000 (oldest deadline 2024-04-24T09:20:02.820181+0000) 2024-04-24T09:21:01.456 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:21:00 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:21:00.957+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:47.326279+0000 front 2024-04-24T09:19:47.326338+0000 (oldest deadline 2024-04-24T09:20:10.728055+0000) 2024-04-24T09:21:01.456 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:21:00 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:21:00.957+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:55.832608+0000 front 2024-04-24T09:19:55.832510+0000 (oldest deadline 2024-04-24T09:20:21.732750+0000) 2024-04-24T09:21:01.456 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:21:00 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:21:00.957+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:08.736362+0000 front 2024-04-24T09:20:08.736680+0000 (oldest deadline 2024-04-24T09:20:31.537874+0000) 2024-04-24T09:21:01.456 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:21:00 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:21:00.957+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-24T09:20:16.240265+0000 front 2024-04-24T09:20:16.240144+0000 (oldest deadline 2024-04-24T09:20:41.540224+0000) 2024-04-24T09:21:01.456 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:21:00 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:21:00.957+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-04-24T09:20:27.442280+0000 front 2024-04-24T09:20:27.441889+0000 (oldest deadline 2024-04-24T09:20:53.341732+0000) 2024-04-24T09:21:01.457 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:01 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:21:01.162+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:28.209530+0000 front 2024-04-24T09:19:28.209142+0000 (oldest deadline 2024-04-24T09:19:51.709380+0000) 2024-04-24T09:21:01.457 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:01 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:21:01.162+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.613982+0000 front 2024-04-24T09:19:38.614541+0000 (oldest deadline 2024-04-24T09:20:00.913987+0000) 2024-04-24T09:21:01.457 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:01 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:21:01.162+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:49.018457+0000 front 2024-04-24T09:19:49.018520+0000 (oldest deadline 2024-04-24T09:20:16.019822+0000) 2024-04-24T09:21:01.458 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:01 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:21:01.162+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:07.025175+0000 front 2024-04-24T09:20:07.025338+0000 (oldest deadline 2024-04-24T09:20:31.125415+0000) 2024-04-24T09:21:01.458 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:01 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:21:01.162+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-24T09:20:17.027505+0000 front 2024-04-24T09:20:17.028269+0000 (oldest deadline 2024-04-24T09:20:42.327601+0000) 2024-04-24T09:21:01.458 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:01 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:21:01.162+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-04-24T09:20:27.431096+0000 front 2024-04-24T09:20:27.431264+0000 (oldest deadline 2024-04-24T09:20:50.930971+0000) 2024-04-24T09:21:02.455 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:21:01 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:21:01.996+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:29.414806+0000 front 2024-04-24T09:19:29.414861+0000 (oldest deadline 2024-04-24T09:19:55.314800+0000) 2024-04-24T09:21:02.455 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:21:01 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:21:01.996+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.119965+0000 front 2024-04-24T09:19:38.120034+0000 (oldest deadline 2024-04-24T09:20:02.820181+0000) 2024-04-24T09:21:02.456 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:21:01 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:21:01.996+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:47.326279+0000 front 2024-04-24T09:19:47.326338+0000 (oldest deadline 2024-04-24T09:20:10.728055+0000) 2024-04-24T09:21:02.456 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:21:01 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:21:01.996+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:55.832608+0000 front 2024-04-24T09:19:55.832510+0000 (oldest deadline 2024-04-24T09:20:21.732750+0000) 2024-04-24T09:21:02.456 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:21:01 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:21:01.996+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:08.736362+0000 front 2024-04-24T09:20:08.736680+0000 (oldest deadline 2024-04-24T09:20:31.537874+0000) 2024-04-24T09:21:02.456 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:21:01 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:21:01.996+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-24T09:20:16.240265+0000 front 2024-04-24T09:20:16.240144+0000 (oldest deadline 2024-04-24T09:20:41.540224+0000) 2024-04-24T09:21:02.456 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:21:01 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:21:01.996+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-04-24T09:20:27.442280+0000 front 2024-04-24T09:20:27.441889+0000 (oldest deadline 2024-04-24T09:20:53.341732+0000) 2024-04-24T09:21:02.457 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:02 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:21:02.159+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:28.209530+0000 front 2024-04-24T09:19:28.209142+0000 (oldest deadline 2024-04-24T09:19:51.709380+0000) 2024-04-24T09:21:02.457 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:02 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:21:02.159+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.613982+0000 front 2024-04-24T09:19:38.614541+0000 (oldest deadline 2024-04-24T09:20:00.913987+0000) 2024-04-24T09:21:02.457 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:02 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:21:02.159+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:49.018457+0000 front 2024-04-24T09:19:49.018520+0000 (oldest deadline 2024-04-24T09:20:16.019822+0000) 2024-04-24T09:21:02.457 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:02 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:21:02.159+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:07.025175+0000 front 2024-04-24T09:20:07.025338+0000 (oldest deadline 2024-04-24T09:20:31.125415+0000) 2024-04-24T09:21:02.457 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:02 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:21:02.159+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-24T09:20:17.027505+0000 front 2024-04-24T09:20:17.028269+0000 (oldest deadline 2024-04-24T09:20:42.327601+0000) 2024-04-24T09:21:02.457 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:02 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:21:02.159+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-04-24T09:20:27.431096+0000 front 2024-04-24T09:20:27.431264+0000 (oldest deadline 2024-04-24T09:20:50.930971+0000) 2024-04-24T09:21:03.455 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:21:02 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:21:02.962+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:29.414806+0000 front 2024-04-24T09:19:29.414861+0000 (oldest deadline 2024-04-24T09:19:55.314800+0000) 2024-04-24T09:21:03.455 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:21:02 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:21:02.962+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.119965+0000 front 2024-04-24T09:19:38.120034+0000 (oldest deadline 2024-04-24T09:20:02.820181+0000) 2024-04-24T09:21:03.455 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:21:02 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:21:02.962+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:47.326279+0000 front 2024-04-24T09:19:47.326338+0000 (oldest deadline 2024-04-24T09:20:10.728055+0000) 2024-04-24T09:21:03.455 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:21:02 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:21:02.962+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:55.832608+0000 front 2024-04-24T09:19:55.832510+0000 (oldest deadline 2024-04-24T09:20:21.732750+0000) 2024-04-24T09:21:03.456 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:21:02 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:21:02.962+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:08.736362+0000 front 2024-04-24T09:20:08.736680+0000 (oldest deadline 2024-04-24T09:20:31.537874+0000) 2024-04-24T09:21:03.456 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:21:02 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:21:02.962+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-24T09:20:16.240265+0000 front 2024-04-24T09:20:16.240144+0000 (oldest deadline 2024-04-24T09:20:41.540224+0000) 2024-04-24T09:21:03.456 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:21:02 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:21:02.962+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-04-24T09:20:27.442280+0000 front 2024-04-24T09:20:27.441889+0000 (oldest deadline 2024-04-24T09:20:53.341732+0000) 2024-04-24T09:21:03.457 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:03 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:21:03.141+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:28.209530+0000 front 2024-04-24T09:19:28.209142+0000 (oldest deadline 2024-04-24T09:19:51.709380+0000) 2024-04-24T09:21:03.457 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:03 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:21:03.141+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.613982+0000 front 2024-04-24T09:19:38.614541+0000 (oldest deadline 2024-04-24T09:20:00.913987+0000) 2024-04-24T09:21:03.457 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:03 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:21:03.141+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:49.018457+0000 front 2024-04-24T09:19:49.018520+0000 (oldest deadline 2024-04-24T09:20:16.019822+0000) 2024-04-24T09:21:03.457 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:03 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:21:03.141+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:07.025175+0000 front 2024-04-24T09:20:07.025338+0000 (oldest deadline 2024-04-24T09:20:31.125415+0000) 2024-04-24T09:21:03.457 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:03 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:21:03.141+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-24T09:20:17.027505+0000 front 2024-04-24T09:20:17.028269+0000 (oldest deadline 2024-04-24T09:20:42.327601+0000) 2024-04-24T09:21:03.457 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:03 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:21:03.141+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-04-24T09:20:27.431096+0000 front 2024-04-24T09:20:27.431264+0000 (oldest deadline 2024-04-24T09:20:50.930971+0000) 2024-04-24T09:21:04.205 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:21:03 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:21:03.952+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:29.414806+0000 front 2024-04-24T09:19:29.414861+0000 (oldest deadline 2024-04-24T09:19:55.314800+0000) 2024-04-24T09:21:04.205 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:21:03 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:21:03.952+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.119965+0000 front 2024-04-24T09:19:38.120034+0000 (oldest deadline 2024-04-24T09:20:02.820181+0000) 2024-04-24T09:21:04.206 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:21:03 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:21:03.952+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:47.326279+0000 front 2024-04-24T09:19:47.326338+0000 (oldest deadline 2024-04-24T09:20:10.728055+0000) 2024-04-24T09:21:04.206 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:21:03 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:21:03.952+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:55.832608+0000 front 2024-04-24T09:19:55.832510+0000 (oldest deadline 2024-04-24T09:20:21.732750+0000) 2024-04-24T09:21:04.206 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:21:03 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:21:03.952+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:08.736362+0000 front 2024-04-24T09:20:08.736680+0000 (oldest deadline 2024-04-24T09:20:31.537874+0000) 2024-04-24T09:21:04.206 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:21:03 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:21:03.952+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-24T09:20:16.240265+0000 front 2024-04-24T09:20:16.240144+0000 (oldest deadline 2024-04-24T09:20:41.540224+0000) 2024-04-24T09:21:04.206 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:21:03 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:21:03.952+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-04-24T09:20:27.442280+0000 front 2024-04-24T09:20:27.441889+0000 (oldest deadline 2024-04-24T09:20:53.341732+0000) 2024-04-24T09:21:04.457 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:04 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:21:04.171+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:28.209530+0000 front 2024-04-24T09:19:28.209142+0000 (oldest deadline 2024-04-24T09:19:51.709380+0000) 2024-04-24T09:21:04.458 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:04 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:21:04.171+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.613982+0000 front 2024-04-24T09:19:38.614541+0000 (oldest deadline 2024-04-24T09:20:00.913987+0000) 2024-04-24T09:21:04.458 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:04 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:21:04.171+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:49.018457+0000 front 2024-04-24T09:19:49.018520+0000 (oldest deadline 2024-04-24T09:20:16.019822+0000) 2024-04-24T09:21:04.458 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:04 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:21:04.171+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:07.025175+0000 front 2024-04-24T09:20:07.025338+0000 (oldest deadline 2024-04-24T09:20:31.125415+0000) 2024-04-24T09:21:04.458 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:04 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:21:04.171+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-24T09:20:17.027505+0000 front 2024-04-24T09:20:17.028269+0000 (oldest deadline 2024-04-24T09:20:42.327601+0000) 2024-04-24T09:21:04.458 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:04 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:21:04.171+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-04-24T09:20:27.431096+0000 front 2024-04-24T09:20:27.431264+0000 (oldest deadline 2024-04-24T09:20:50.930971+0000) 2024-04-24T09:21:04.458 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:04 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:21:04.171+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6820 osd.7 since back 2024-04-24T09:20:37.333470+0000 front 2024-04-24T09:20:37.333503+0000 (oldest deadline 2024-04-24T09:21:03.233337+0000) 2024-04-24T09:21:05.455 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:21:04 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:21:04.955+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:29.414806+0000 front 2024-04-24T09:19:29.414861+0000 (oldest deadline 2024-04-24T09:19:55.314800+0000) 2024-04-24T09:21:05.455 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:21:04 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:21:04.955+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.119965+0000 front 2024-04-24T09:19:38.120034+0000 (oldest deadline 2024-04-24T09:20:02.820181+0000) 2024-04-24T09:21:05.456 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:21:04 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:21:04.955+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:47.326279+0000 front 2024-04-24T09:19:47.326338+0000 (oldest deadline 2024-04-24T09:20:10.728055+0000) 2024-04-24T09:21:05.456 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:21:04 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:21:04.955+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-24T09:19:55.832608+0000 front 2024-04-24T09:19:55.832510+0000 (oldest deadline 2024-04-24T09:20:21.732750+0000) 2024-04-24T09:21:05.456 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:21:04 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:21:04.955+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:08.736362+0000 front 2024-04-24T09:20:08.736680+0000 (oldest deadline 2024-04-24T09:20:31.537874+0000) 2024-04-24T09:21:05.456 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:21:04 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:21:04.955+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-24T09:20:16.240265+0000 front 2024-04-24T09:20:16.240144+0000 (oldest deadline 2024-04-24T09:20:41.540224+0000) 2024-04-24T09:21:05.456 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:21:04 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10[56044]: 2024-04-24T09:21:04.955+0000 7f32f9000700 -1 osd.10 141 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-04-24T09:20:27.442280+0000 front 2024-04-24T09:20:27.441889+0000 (oldest deadline 2024-04-24T09:20:53.341732+0000) 2024-04-24T09:21:05.457 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:05 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:21:05.178+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:28.209530+0000 front 2024-04-24T09:19:28.209142+0000 (oldest deadline 2024-04-24T09:19:51.709380+0000) 2024-04-24T09:21:05.457 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:05 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:21:05.178+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.613982+0000 front 2024-04-24T09:19:38.614541+0000 (oldest deadline 2024-04-24T09:20:00.913987+0000) 2024-04-24T09:21:05.457 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:05 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:21:05.178+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:49.018457+0000 front 2024-04-24T09:19:49.018520+0000 (oldest deadline 2024-04-24T09:20:16.019822+0000) 2024-04-24T09:21:05.457 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:05 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:21:05.178+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:07.025175+0000 front 2024-04-24T09:20:07.025338+0000 (oldest deadline 2024-04-24T09:20:31.125415+0000) 2024-04-24T09:21:05.458 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:05 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:21:05.178+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-24T09:20:17.027505+0000 front 2024-04-24T09:20:17.028269+0000 (oldest deadline 2024-04-24T09:20:42.327601+0000) 2024-04-24T09:21:05.458 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:05 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:21:05.178+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-04-24T09:20:27.431096+0000 front 2024-04-24T09:20:27.431264+0000 (oldest deadline 2024-04-24T09:20:50.930971+0000) 2024-04-24T09:21:05.458 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:05 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:21:05.178+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6820 osd.7 since back 2024-04-24T09:20:37.333470+0000 front 2024-04-24T09:20:37.333503+0000 (oldest deadline 2024-04-24T09:21:03.233337+0000) 2024-04-24T09:21:06.433 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:21:06 smithi138 podman[73161]: 2024-04-24 09:21:06.159479974 +0000 UTC m=+5.633613248 container died bb82991e68c94a214c54a16dd5e8f84f02caba7f7050a05abf5e2a612c2d1776 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10, ceph=True, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424) 2024-04-24T09:21:06.457 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:06 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:21:06.144+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:28.209530+0000 front 2024-04-24T09:19:28.209142+0000 (oldest deadline 2024-04-24T09:19:51.709380+0000) 2024-04-24T09:21:06.457 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:06 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:21:06.144+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.613982+0000 front 2024-04-24T09:19:38.614541+0000 (oldest deadline 2024-04-24T09:20:00.913987+0000) 2024-04-24T09:21:06.457 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:06 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:21:06.144+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:49.018457+0000 front 2024-04-24T09:19:49.018520+0000 (oldest deadline 2024-04-24T09:20:16.019822+0000) 2024-04-24T09:21:06.457 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:06 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:21:06.144+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:07.025175+0000 front 2024-04-24T09:20:07.025338+0000 (oldest deadline 2024-04-24T09:20:31.125415+0000) 2024-04-24T09:21:06.457 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:06 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:21:06.144+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-24T09:20:17.027505+0000 front 2024-04-24T09:20:17.028269+0000 (oldest deadline 2024-04-24T09:20:42.327601+0000) 2024-04-24T09:21:06.457 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:06 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:21:06.144+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-04-24T09:20:27.431096+0000 front 2024-04-24T09:20:27.431264+0000 (oldest deadline 2024-04-24T09:20:50.930971+0000) 2024-04-24T09:21:06.458 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:06 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:21:06.144+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6820 osd.7 since back 2024-04-24T09:20:37.333470+0000 front 2024-04-24T09:20:37.333503+0000 (oldest deadline 2024-04-24T09:21:03.233337+0000) 2024-04-24T09:21:06.704 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:21:06 smithi138 podman[73161]: 2024-04-24 09:21:06.579735872 +0000 UTC m=+6.053869139 container cleanup bb82991e68c94a214c54a16dd5e8f84f02caba7f7050a05abf5e2a612c2d1776 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-04-24T09:21:06.705 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:21:06 smithi138 bash[73161]: ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10 2024-04-24T09:21:07.205 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:21:06 smithi138 podman[73181]: 2024-04-24 09:21:06.828012692 +0000 UTC m=+0.655301455 container remove bb82991e68c94a214c54a16dd5e8f84f02caba7f7050a05abf5e2a612c2d1776 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=rishabh-main-apr17-a654945-1f81fb3, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=) 2024-04-24T09:21:07.457 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:07 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:21:07.177+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:28.209530+0000 front 2024-04-24T09:19:28.209142+0000 (oldest deadline 2024-04-24T09:19:51.709380+0000) 2024-04-24T09:21:07.457 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:07 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:21:07.177+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.613982+0000 front 2024-04-24T09:19:38.614541+0000 (oldest deadline 2024-04-24T09:20:00.913987+0000) 2024-04-24T09:21:07.458 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:07 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:21:07.177+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:49.018457+0000 front 2024-04-24T09:19:49.018520+0000 (oldest deadline 2024-04-24T09:20:16.019822+0000) 2024-04-24T09:21:07.458 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:07 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:21:07.177+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:07.025175+0000 front 2024-04-24T09:20:07.025338+0000 (oldest deadline 2024-04-24T09:20:31.125415+0000) 2024-04-24T09:21:07.458 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:07 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:21:07.177+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-24T09:20:17.027505+0000 front 2024-04-24T09:20:17.028269+0000 (oldest deadline 2024-04-24T09:20:42.327601+0000) 2024-04-24T09:21:07.458 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:07 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:21:07.177+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-04-24T09:20:27.431096+0000 front 2024-04-24T09:20:27.431264+0000 (oldest deadline 2024-04-24T09:20:50.930971+0000) 2024-04-24T09:21:07.458 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:07 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:21:07.177+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6820 osd.7 since back 2024-04-24T09:20:37.333470+0000 front 2024-04-24T09:20:37.333503+0000 (oldest deadline 2024-04-24T09:21:03.233337+0000) 2024-04-24T09:21:07.955 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:21:07 smithi138 podman[73267]: 2024-04-24 09:21:07.598724847 +0000 UTC m=+0.032934577 image pull b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644 2024-04-24T09:21:07.955 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:21:07 smithi138 podman[73267]: 2024-04-24 09:21:07.72894898 +0000 UTC m=+0.163158706 container create 98099cee9af20b1ccea478b6c6c191dd5534f8d7235af8d456fae4eecd85ee7f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS) 2024-04-24T09:21:08.455 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:21:08 smithi138 podman[73267]: 2024-04-24 09:21:08.026138924 +0000 UTC m=+0.460348716 container init 98099cee9af20b1ccea478b6c6c191dd5534f8d7235af8d456fae4eecd85ee7f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True) 2024-04-24T09:21:08.455 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:21:08 smithi138 podman[73267]: 2024-04-24 09:21:08.039159131 +0000 UTC m=+0.473368893 container start 98099cee9af20b1ccea478b6c6c191dd5534f8d7235af8d456fae4eecd85ee7f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10-deactivate, ceph=True, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-04-24T09:21:08.455 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:21:08 smithi138 podman[73267]: 2024-04-24 09:21:08.123940625 +0000 UTC m=+0.558150382 container attach 98099cee9af20b1ccea478b6c6c191dd5534f8d7235af8d456fae4eecd85ee7f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=rishabh-main-apr17-a654945-1f81fb3, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T09:21:08.457 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:08 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:21:08.203+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:28.209530+0000 front 2024-04-24T09:19:28.209142+0000 (oldest deadline 2024-04-24T09:19:51.709380+0000) 2024-04-24T09:21:08.457 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:08 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:21:08.203+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.613982+0000 front 2024-04-24T09:19:38.614541+0000 (oldest deadline 2024-04-24T09:20:00.913987+0000) 2024-04-24T09:21:08.457 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:08 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:21:08.203+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:49.018457+0000 front 2024-04-24T09:19:49.018520+0000 (oldest deadline 2024-04-24T09:20:16.019822+0000) 2024-04-24T09:21:08.457 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:08 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:21:08.203+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:07.025175+0000 front 2024-04-24T09:20:07.025338+0000 (oldest deadline 2024-04-24T09:20:31.125415+0000) 2024-04-24T09:21:08.457 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:08 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:21:08.203+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-24T09:20:17.027505+0000 front 2024-04-24T09:20:17.028269+0000 (oldest deadline 2024-04-24T09:20:42.327601+0000) 2024-04-24T09:21:08.457 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:08 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:21:08.203+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-04-24T09:20:27.431096+0000 front 2024-04-24T09:20:27.431264+0000 (oldest deadline 2024-04-24T09:20:50.930971+0000) 2024-04-24T09:21:08.458 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:08 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:21:08.203+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6820 osd.7 since back 2024-04-24T09:20:37.333470+0000 front 2024-04-24T09:20:37.333503+0000 (oldest deadline 2024-04-24T09:21:03.233337+0000) 2024-04-24T09:21:08.833 INFO:journalctl@ceph.osd.10.smithi138.stdout:Apr 24 09:21:08 smithi138 podman[73293]: 2024-04-24 09:21:08.546916798 +0000 UTC m=+0.036832549 container died 98099cee9af20b1ccea478b6c6c191dd5534f8d7235af8d456fae4eecd85ee7f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-10-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424) 2024-04-24T09:21:09.225 DEBUG:teuthology.orchestra.run.smithi138:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bd3e039e-0215-11ef-bc93-c7b262605968@osd.10.service' 2024-04-24T09:21:09.329 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T09:21:09.330 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-04-24T09:21:09.330 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-04-24T09:21:09.330 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ceph-bd3e039e-0215-11ef-bc93-c7b262605968@osd.11 2024-04-24T09:21:09.443 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:09 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:21:09.190+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:28.209530+0000 front 2024-04-24T09:19:28.209142+0000 (oldest deadline 2024-04-24T09:19:51.709380+0000) 2024-04-24T09:21:09.443 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:09 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:21:09.190+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.613982+0000 front 2024-04-24T09:19:38.614541+0000 (oldest deadline 2024-04-24T09:20:00.913987+0000) 2024-04-24T09:21:09.443 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:09 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:21:09.190+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:49.018457+0000 front 2024-04-24T09:19:49.018520+0000 (oldest deadline 2024-04-24T09:20:16.019822+0000) 2024-04-24T09:21:09.443 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:09 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:21:09.190+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:07.025175+0000 front 2024-04-24T09:20:07.025338+0000 (oldest deadline 2024-04-24T09:20:31.125415+0000) 2024-04-24T09:21:09.443 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:09 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:21:09.190+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-24T09:20:17.027505+0000 front 2024-04-24T09:20:17.028269+0000 (oldest deadline 2024-04-24T09:20:42.327601+0000) 2024-04-24T09:21:09.444 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:09 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:21:09.190+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-04-24T09:20:27.431096+0000 front 2024-04-24T09:20:27.431264+0000 (oldest deadline 2024-04-24T09:20:50.930971+0000) 2024-04-24T09:21:09.444 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:09 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:21:09.190+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6820 osd.7 since back 2024-04-24T09:20:37.333470+0000 front 2024-04-24T09:20:37.333503+0000 (oldest deadline 2024-04-24T09:21:03.233337+0000) 2024-04-24T09:21:09.707 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:09 smithi189 systemd[1]: Stopping Ceph osd.11 for bd3e039e-0215-11ef-bc93-c7b262605968... 2024-04-24T09:21:10.150 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:09 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:21:09.898+0000 7f0f74800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T09:21:10.151 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:09 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:21:09.898+0000 7f0f74800700 -1 osd.11 141 *** Got signal Terminated *** 2024-04-24T09:21:10.151 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:09 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:21:09.898+0000 7f0f74800700 -1 osd.11 141 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T09:21:10.151 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:10 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:21:10.148+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:28.209530+0000 front 2024-04-24T09:19:28.209142+0000 (oldest deadline 2024-04-24T09:19:51.709380+0000) 2024-04-24T09:21:10.151 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:10 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:21:10.148+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.613982+0000 front 2024-04-24T09:19:38.614541+0000 (oldest deadline 2024-04-24T09:20:00.913987+0000) 2024-04-24T09:21:10.151 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:10 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:21:10.148+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:49.018457+0000 front 2024-04-24T09:19:49.018520+0000 (oldest deadline 2024-04-24T09:20:16.019822+0000) 2024-04-24T09:21:10.151 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:10 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:21:10.148+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:07.025175+0000 front 2024-04-24T09:20:07.025338+0000 (oldest deadline 2024-04-24T09:20:31.125415+0000) 2024-04-24T09:21:10.151 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:10 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:21:10.148+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-24T09:20:17.027505+0000 front 2024-04-24T09:20:17.028269+0000 (oldest deadline 2024-04-24T09:20:42.327601+0000) 2024-04-24T09:21:10.151 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:10 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:21:10.148+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-04-24T09:20:27.431096+0000 front 2024-04-24T09:20:27.431264+0000 (oldest deadline 2024-04-24T09:20:50.930971+0000) 2024-04-24T09:21:10.151 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:10 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:21:10.148+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6820 osd.7 since back 2024-04-24T09:20:37.333470+0000 front 2024-04-24T09:20:37.333503+0000 (oldest deadline 2024-04-24T09:21:03.233337+0000) 2024-04-24T09:21:10.151 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:10 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:21:10.148+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.189:6820 osd.8 since back 2024-04-24T09:20:45.937064+0000 front 2024-04-24T09:20:45.937174+0000 (oldest deadline 2024-04-24T09:21:10.037071+0000) 2024-04-24T09:21:11.457 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:11 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:21:11.133+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:28.209530+0000 front 2024-04-24T09:19:28.209142+0000 (oldest deadline 2024-04-24T09:19:51.709380+0000) 2024-04-24T09:21:11.458 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:11 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:21:11.133+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.613982+0000 front 2024-04-24T09:19:38.614541+0000 (oldest deadline 2024-04-24T09:20:00.913987+0000) 2024-04-24T09:21:11.458 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:11 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:21:11.133+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:49.018457+0000 front 2024-04-24T09:19:49.018520+0000 (oldest deadline 2024-04-24T09:20:16.019822+0000) 2024-04-24T09:21:11.458 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:11 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:21:11.133+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:07.025175+0000 front 2024-04-24T09:20:07.025338+0000 (oldest deadline 2024-04-24T09:20:31.125415+0000) 2024-04-24T09:21:11.458 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:11 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:21:11.133+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-24T09:20:17.027505+0000 front 2024-04-24T09:20:17.028269+0000 (oldest deadline 2024-04-24T09:20:42.327601+0000) 2024-04-24T09:21:11.458 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:11 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:21:11.133+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-04-24T09:20:27.431096+0000 front 2024-04-24T09:20:27.431264+0000 (oldest deadline 2024-04-24T09:20:50.930971+0000) 2024-04-24T09:21:11.458 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:11 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:21:11.133+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6820 osd.7 since back 2024-04-24T09:20:37.333470+0000 front 2024-04-24T09:20:37.333503+0000 (oldest deadline 2024-04-24T09:21:03.233337+0000) 2024-04-24T09:21:11.458 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:11 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:21:11.133+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.189:6820 osd.8 since back 2024-04-24T09:20:45.937064+0000 front 2024-04-24T09:20:45.937174+0000 (oldest deadline 2024-04-24T09:21:10.037071+0000) 2024-04-24T09:21:12.457 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:12 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:21:12.172+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:28.209530+0000 front 2024-04-24T09:19:28.209142+0000 (oldest deadline 2024-04-24T09:19:51.709380+0000) 2024-04-24T09:21:12.457 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:12 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:21:12.172+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.613982+0000 front 2024-04-24T09:19:38.614541+0000 (oldest deadline 2024-04-24T09:20:00.913987+0000) 2024-04-24T09:21:12.457 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:12 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:21:12.172+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:49.018457+0000 front 2024-04-24T09:19:49.018520+0000 (oldest deadline 2024-04-24T09:20:16.019822+0000) 2024-04-24T09:21:12.458 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:12 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:21:12.172+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:07.025175+0000 front 2024-04-24T09:20:07.025338+0000 (oldest deadline 2024-04-24T09:20:31.125415+0000) 2024-04-24T09:21:12.458 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:12 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:21:12.172+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-24T09:20:17.027505+0000 front 2024-04-24T09:20:17.028269+0000 (oldest deadline 2024-04-24T09:20:42.327601+0000) 2024-04-24T09:21:12.458 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:12 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:21:12.172+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-04-24T09:20:27.431096+0000 front 2024-04-24T09:20:27.431264+0000 (oldest deadline 2024-04-24T09:20:50.930971+0000) 2024-04-24T09:21:12.458 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:12 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:21:12.172+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6820 osd.7 since back 2024-04-24T09:20:37.333470+0000 front 2024-04-24T09:20:37.333503+0000 (oldest deadline 2024-04-24T09:21:03.233337+0000) 2024-04-24T09:21:12.458 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:12 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:21:12.172+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.189:6820 osd.8 since back 2024-04-24T09:20:45.937064+0000 front 2024-04-24T09:20:45.937174+0000 (oldest deadline 2024-04-24T09:21:10.037071+0000) 2024-04-24T09:21:13.458 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:13 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:21:13.201+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:28.209530+0000 front 2024-04-24T09:19:28.209142+0000 (oldest deadline 2024-04-24T09:19:51.709380+0000) 2024-04-24T09:21:13.458 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:13 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:21:13.201+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.613982+0000 front 2024-04-24T09:19:38.614541+0000 (oldest deadline 2024-04-24T09:20:00.913987+0000) 2024-04-24T09:21:13.458 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:13 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:21:13.201+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:49.018457+0000 front 2024-04-24T09:19:49.018520+0000 (oldest deadline 2024-04-24T09:20:16.019822+0000) 2024-04-24T09:21:13.458 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:13 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:21:13.201+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:07.025175+0000 front 2024-04-24T09:20:07.025338+0000 (oldest deadline 2024-04-24T09:20:31.125415+0000) 2024-04-24T09:21:13.458 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:13 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:21:13.201+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-24T09:20:17.027505+0000 front 2024-04-24T09:20:17.028269+0000 (oldest deadline 2024-04-24T09:20:42.327601+0000) 2024-04-24T09:21:13.458 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:13 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:21:13.201+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-04-24T09:20:27.431096+0000 front 2024-04-24T09:20:27.431264+0000 (oldest deadline 2024-04-24T09:20:50.930971+0000) 2024-04-24T09:21:13.458 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:13 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:21:13.201+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6820 osd.7 since back 2024-04-24T09:20:37.333470+0000 front 2024-04-24T09:20:37.333503+0000 (oldest deadline 2024-04-24T09:21:03.233337+0000) 2024-04-24T09:21:13.458 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:13 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:21:13.201+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.189:6820 osd.8 since back 2024-04-24T09:20:45.937064+0000 front 2024-04-24T09:20:45.937174+0000 (oldest deadline 2024-04-24T09:21:10.037071+0000) 2024-04-24T09:21:14.457 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:14 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:21:14.174+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-24T09:19:28.209530+0000 front 2024-04-24T09:19:28.209142+0000 (oldest deadline 2024-04-24T09:19:51.709380+0000) 2024-04-24T09:21:14.458 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:14 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:21:14.174+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-04-24T09:19:38.613982+0000 front 2024-04-24T09:19:38.614541+0000 (oldest deadline 2024-04-24T09:20:00.913987+0000) 2024-04-24T09:21:14.458 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:14 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:21:14.174+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-04-24T09:19:49.018457+0000 front 2024-04-24T09:19:49.018520+0000 (oldest deadline 2024-04-24T09:20:16.019822+0000) 2024-04-24T09:21:14.458 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:14 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:21:14.174+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-04-24T09:20:07.025175+0000 front 2024-04-24T09:20:07.025338+0000 (oldest deadline 2024-04-24T09:20:31.125415+0000) 2024-04-24T09:21:14.458 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:14 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:21:14.174+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-04-24T09:20:17.027505+0000 front 2024-04-24T09:20:17.028269+0000 (oldest deadline 2024-04-24T09:20:42.327601+0000) 2024-04-24T09:21:14.458 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:14 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:21:14.174+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-04-24T09:20:27.431096+0000 front 2024-04-24T09:20:27.431264+0000 (oldest deadline 2024-04-24T09:20:50.930971+0000) 2024-04-24T09:21:14.458 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:14 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:21:14.174+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.138:6820 osd.7 since back 2024-04-24T09:20:37.333470+0000 front 2024-04-24T09:20:37.333503+0000 (oldest deadline 2024-04-24T09:21:03.233337+0000) 2024-04-24T09:21:14.458 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:14 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11[56728]: 2024-04-24T09:21:14.174+0000 7f0f70000700 -1 osd.11 141 heartbeat_check: no reply from 172.21.15.189:6820 osd.8 since back 2024-04-24T09:20:45.937064+0000 front 2024-04-24T09:20:45.937174+0000 (oldest deadline 2024-04-24T09:21:10.037071+0000) 2024-04-24T09:21:15.800 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:15 smithi189 podman[77367]: 2024-04-24 09:21:15.507510313 +0000 UTC m=+5.756614630 container died 82b9ae1f76d1c84f8ca2ca3ee0531e3498937cad943e15fe214dcd5a64a39999 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11, RELEASE=rishabh-main-apr17-a654945-1f81fb3, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True) 2024-04-24T09:21:16.116 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:15 smithi189 podman[77367]: 2024-04-24 09:21:15.943239 +0000 UTC m=+6.192343292 container cleanup 82b9ae1f76d1c84f8ca2ca3ee0531e3498937cad943e15fe214dcd5a64a39999 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-04-24T09:21:16.116 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:15 smithi189 bash[77367]: ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11 2024-04-24T09:21:16.457 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:16 smithi189 podman[77386]: 2024-04-24 09:21:16.17743212 +0000 UTC m=+0.659118892 container remove 82b9ae1f76d1c84f8ca2ca3ee0531e3498937cad943e15fe214dcd5a64a39999 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11, org.label-schema.build-date=20240424, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=rishabh-main-apr17-a654945-1f81fb3, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-24T09:21:17.155 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:16 smithi189 podman[77480]: 2024-04-24 09:21:16.775257805 +0000 UTC m=+0.033040688 image pull b98aea5aa06f9f14aacd874e6fe7f4a9ab599f4e4a4707fadae99d37abf77a02 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644 2024-04-24T09:21:17.155 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:16 smithi189 podman[77480]: 2024-04-24 09:21:16.882633741 +0000 UTC m=+0.140416661 container create 4959bfec703d377213d9e55376e7a2d0753e9342b12aea2cede31c1e9dc3c9e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11-deactivate, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux ) 2024-04-24T09:21:17.457 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:17 smithi189 podman[77480]: 2024-04-24 09:21:17.240250055 +0000 UTC m=+0.498032985 container init 4959bfec703d377213d9e55376e7a2d0753e9342b12aea2cede31c1e9dc3c9e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11-deactivate, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T09:21:17.458 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:17 smithi189 podman[77480]: 2024-04-24 09:21:17.250168862 +0000 UTC m=+0.507951788 container start 4959bfec703d377213d9e55376e7a2d0753e9342b12aea2cede31c1e9dc3c9e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240424, RELEASE=rishabh-main-apr17-a654945-1f81fb3, ceph=True) 2024-04-24T09:21:17.458 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:17 smithi189 podman[77480]: 2024-04-24 09:21:17.331022691 +0000 UTC m=+0.588805591 container attach 4959bfec703d377213d9e55376e7a2d0753e9342b12aea2cede31c1e9dc3c9e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11-deactivate, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, RELEASE=rishabh-main-apr17-a654945-1f81fb3, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-04-24T09:21:17.957 INFO:journalctl@ceph.osd.11.smithi189.stdout:Apr 24 09:21:17 smithi189 podman[77480]: 2024-04-24 09:21:17.701165132 +0000 UTC m=+0.958948015 container died 4959bfec703d377213d9e55376e7a2d0753e9342b12aea2cede31c1e9dc3c9e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-osd-11-deactivate, GIT_BRANCH=HEAD, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-24T09:21:18.468 DEBUG:teuthology.orchestra.run.smithi189:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bd3e039e-0215-11ef-bc93-c7b262605968@osd.11.service' 2024-04-24T09:21:18.584 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T09:21:18.584 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-04-24T09:21:18.584 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-04-24T09:21:18.584 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-bd3e039e-0215-11ef-bc93-c7b262605968@mds.a 2024-04-24T09:21:19.053 INFO:journalctl@ceph.mds.a.smithi083.stdout:Apr 24 09:21:18 smithi083 systemd[1]: Stopping Ceph mds.a for bd3e039e-0215-11ef-bc93-c7b262605968... 2024-04-24T09:21:19.553 INFO:journalctl@ceph.mds.a.smithi083.stdout:Apr 24 09:21:19 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-a[69821]: 2024-04-24T09:21:19.101+0000 7f6c8e000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T09:21:19.553 INFO:journalctl@ceph.mds.a.smithi083.stdout:Apr 24 09:21:19 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-a[69821]: 2024-04-24T09:21:19.101+0000 7f6c8e000700 -1 mds.a *** got signal Terminated *** 2024-04-24T09:21:23.425 INFO:journalctl@ceph.mds.a.smithi083.stdout:Apr 24 09:21:23 smithi083 podman[132357]: 2024-04-24 09:21:23.115643894 +0000 UTC m=+4.126917744 container died 0f9ba266ef2dc6cf0a25f4b10ed3f8bbcbb7c4a3b691281151f641af9b3b4d87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-a, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-04-24T09:21:23.803 INFO:journalctl@ceph.mds.a.smithi083.stdout:Apr 24 09:21:23 smithi083 podman[132357]: 2024-04-24 09:21:23.545614457 +0000 UTC m=+4.556888340 container cleanup 0f9ba266ef2dc6cf0a25f4b10ed3f8bbcbb7c4a3b691281151f641af9b3b4d87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-a, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS) 2024-04-24T09:21:23.803 INFO:journalctl@ceph.mds.a.smithi083.stdout:Apr 24 09:21:23 smithi083 bash[132357]: ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-a 2024-04-24T09:21:24.303 INFO:journalctl@ceph.mds.a.smithi083.stdout:Apr 24 09:21:23 smithi083 podman[132372]: 2024-04-24 09:21:23.863313794 +0000 UTC m=+0.726535367 container remove 0f9ba266ef2dc6cf0a25f4b10ed3f8bbcbb7c4a3b691281151f641af9b3b4d87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-a, RELEASE=rishabh-main-apr17-a654945-1f81fb3, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T09:21:24.348 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bd3e039e-0215-11ef-bc93-c7b262605968@mds.a.service' 2024-04-24T09:21:24.444 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T09:21:24.445 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-04-24T09:21:24.445 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-04-24T09:21:24.445 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-bd3e039e-0215-11ef-bc93-c7b262605968@mds.d 2024-04-24T09:21:24.782 INFO:journalctl@ceph.mds.d.smithi083.stdout:Apr 24 09:21:24 smithi083 systemd[1]: Stopping Ceph mds.d for bd3e039e-0215-11ef-bc93-c7b262605968... 2024-04-24T09:21:25.053 INFO:journalctl@ceph.mds.d.smithi083.stdout:Apr 24 09:21:25 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-d[68716]: 2024-04-24T09:21:25.015+0000 7fb0b4e00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T09:21:25.053 INFO:journalctl@ceph.mds.d.smithi083.stdout:Apr 24 09:21:25 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-d[68716]: 2024-04-24T09:21:25.015+0000 7fb0b4e00700 -1 mds.d *** got signal Terminated *** 2024-04-24T09:21:29.304 INFO:journalctl@ceph.mds.d.smithi083.stdout:Apr 24 09:21:29 smithi083 podman[132489]: 2024-04-24 09:21:29.032717282 +0000 UTC m=+4.147329416 container died 89397ba2650e92a5175eb4fd990e102146b7f8778ba7500da76ed049fe092f0c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-d, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240424, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-04-24T09:21:29.658 INFO:journalctl@ceph.mds.d.smithi083.stdout:Apr 24 09:21:29 smithi083 podman[132489]: 2024-04-24 09:21:29.441321067 +0000 UTC m=+4.555933201 container cleanup 89397ba2650e92a5175eb4fd990e102146b7f8778ba7500da76ed049fe092f0c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-d, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True) 2024-04-24T09:21:29.659 INFO:journalctl@ceph.mds.d.smithi083.stdout:Apr 24 09:21:29 smithi083 bash[132489]: ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-d 2024-04-24T09:21:30.053 INFO:journalctl@ceph.mds.d.smithi083.stdout:Apr 24 09:21:29 smithi083 podman[132503]: 2024-04-24 09:21:29.658831906 +0000 UTC m=+0.600284787 container remove 89397ba2650e92a5175eb4fd990e102146b7f8778ba7500da76ed049fe092f0c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-d, RELEASE=rishabh-main-apr17-a654945-1f81fb3, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T09:21:30.174 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bd3e039e-0215-11ef-bc93-c7b262605968@mds.d.service' 2024-04-24T09:21:30.278 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T09:21:30.278 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-04-24T09:21:30.279 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-04-24T09:21:30.279 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-bd3e039e-0215-11ef-bc93-c7b262605968@mds.g 2024-04-24T09:21:30.553 INFO:journalctl@ceph.mds.g.smithi083.stdout:Apr 24 09:21:30 smithi083 systemd[1]: Stopping Ceph mds.g for bd3e039e-0215-11ef-bc93-c7b262605968... 2024-04-24T09:21:31.053 INFO:journalctl@ceph.mds.g.smithi083.stdout:Apr 24 09:21:30 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-g[69436]: 2024-04-24T09:21:30.793+0000 7f8bb0e00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T09:21:31.053 INFO:journalctl@ceph.mds.g.smithi083.stdout:Apr 24 09:21:30 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-g[69436]: 2024-04-24T09:21:30.793+0000 7f8bb0e00700 -1 mds.g *** got signal Terminated *** 2024-04-24T09:21:35.107 INFO:journalctl@ceph.mds.g.smithi083.stdout:Apr 24 09:21:34 smithi083 podman[132606]: 2024-04-24 09:21:34.808033367 +0000 UTC m=+4.147704574 container died 4cd70b5367890a5992e77e603132ad8de365154ac5953b8e9c442b164a921f42 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-g, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS) 2024-04-24T09:21:35.504 INFO:journalctl@ceph.mds.g.smithi083.stdout:Apr 24 09:21:35 smithi083 podman[132606]: 2024-04-24 09:21:35.2278734 +0000 UTC m=+4.567544641 container cleanup 4cd70b5367890a5992e77e603132ad8de365154ac5953b8e9c442b164a921f42 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=rishabh-main-apr17-a654945-1f81fb3, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424) 2024-04-24T09:21:35.504 INFO:journalctl@ceph.mds.g.smithi083.stdout:Apr 24 09:21:35 smithi083 bash[132606]: ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-g 2024-04-24T09:21:35.803 INFO:journalctl@ceph.mds.g.smithi083.stdout:Apr 24 09:21:35 smithi083 podman[132620]: 2024-04-24 09:21:35.503757311 +0000 UTC m=+0.683177338 container remove 4cd70b5367890a5992e77e603132ad8de365154ac5953b8e9c442b164a921f42 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-g, GIT_CLEAN=True, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424) 2024-04-24T09:21:36.010 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bd3e039e-0215-11ef-bc93-c7b262605968@mds.g.service' 2024-04-24T09:21:36.116 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T09:21:36.116 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-04-24T09:21:36.116 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-04-24T09:21:36.116 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-bd3e039e-0215-11ef-bc93-c7b262605968@mds.j 2024-04-24T09:21:36.553 INFO:journalctl@ceph.mds.j.smithi083.stdout:Apr 24 09:21:36 smithi083 systemd[1]: Stopping Ceph mds.j for bd3e039e-0215-11ef-bc93-c7b262605968... 2024-04-24T09:21:37.053 INFO:journalctl@ceph.mds.j.smithi083.stdout:Apr 24 09:21:36 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-j[68338]: 2024-04-24T09:21:36.704+0000 7f06c1000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T09:21:37.053 INFO:journalctl@ceph.mds.j.smithi083.stdout:Apr 24 09:21:36 smithi083 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-j[68338]: 2024-04-24T09:21:36.704+0000 7f06c1000700 -1 mds.j *** got signal Terminated *** 2024-04-24T09:21:40.985 INFO:journalctl@ceph.mds.j.smithi083.stdout:Apr 24 09:21:40 smithi083 podman[132727]: 2024-04-24 09:21:40.727261184 +0000 UTC m=+4.154829626 container died 6c5b0557d05b558e79ffd54d24031bd12589a30f2caccbc1d8d8e12c9a164e1b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-j, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-04-24T09:21:41.303 INFO:journalctl@ceph.mds.j.smithi083.stdout:Apr 24 09:21:41 smithi083 podman[132727]: 2024-04-24 09:21:41.09784079 +0000 UTC m=+4.525409238 container cleanup 6c5b0557d05b558e79ffd54d24031bd12589a30f2caccbc1d8d8e12c9a164e1b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-j, org.label-schema.schema-version=1.0, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424) 2024-04-24T09:21:41.303 INFO:journalctl@ceph.mds.j.smithi083.stdout:Apr 24 09:21:41 smithi083 bash[132727]: ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-j 2024-04-24T09:21:41.579 INFO:journalctl@ceph.mds.j.smithi083.stdout:Apr 24 09:21:41 smithi083 podman[132741]: 2024-04-24 09:21:41.324667002 +0000 UTC m=+0.582819164 container remove 6c5b0557d05b558e79ffd54d24031bd12589a30f2caccbc1d8d8e12c9a164e1b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-j, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T09:21:41.614 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bd3e039e-0215-11ef-bc93-c7b262605968@mds.j.service' 2024-04-24T09:21:41.721 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T09:21:41.721 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-04-24T09:21:41.721 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-04-24T09:21:41.721 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ceph-bd3e039e-0215-11ef-bc93-c7b262605968@mds.b 2024-04-24T09:21:42.204 INFO:journalctl@ceph.mds.b.smithi138.stdout:Apr 24 09:21:41 smithi138 systemd[1]: Stopping Ceph mds.b for bd3e039e-0215-11ef-bc93-c7b262605968... 2024-04-24T09:21:42.568 INFO:journalctl@ceph.mds.b.smithi138.stdout:Apr 24 09:21:42 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-b[61515]: 2024-04-24T09:21:42.267+0000 7fb82a800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T09:21:42.568 INFO:journalctl@ceph.mds.b.smithi138.stdout:Apr 24 09:21:42 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-b[61515]: 2024-04-24T09:21:42.267+0000 7fb82a800700 -1 mds.b *** got signal Terminated *** 2024-04-24T09:21:43.705 INFO:journalctl@ceph.mds.b.smithi138.stdout:Apr 24 09:21:43 smithi138 podman[73372]: 2024-04-24 09:21:43.573501221 +0000 UTC m=+1.456528912 container died 23d05a95c3bd84b3eba4a1e7ea52485bbb3573a70da483c32a4a3abdec7aae74 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-b, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=rishabh-main-apr17-a654945-1f81fb3, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-24T09:21:44.205 INFO:journalctl@ceph.mds.b.smithi138.stdout:Apr 24 09:21:44 smithi138 podman[73372]: 2024-04-24 09:21:44.029043726 +0000 UTC m=+1.912071431 container cleanup 23d05a95c3bd84b3eba4a1e7ea52485bbb3573a70da483c32a4a3abdec7aae74 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-b, org.label-schema.vendor=CentOS, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-04-24T09:21:44.205 INFO:journalctl@ceph.mds.b.smithi138.stdout:Apr 24 09:21:44 smithi138 bash[73372]: ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-b 2024-04-24T09:21:44.704 INFO:journalctl@ceph.mds.b.smithi138.stdout:Apr 24 09:21:44 smithi138 podman[73397]: 2024-04-24 09:21:44.355508803 +0000 UTC m=+0.756109145 container remove 23d05a95c3bd84b3eba4a1e7ea52485bbb3573a70da483c32a4a3abdec7aae74 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-b, CEPH_POINT_RELEASE=, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-04-24T09:21:44.838 DEBUG:teuthology.orchestra.run.smithi138:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bd3e039e-0215-11ef-bc93-c7b262605968@mds.b.service' 2024-04-24T09:21:44.948 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T09:21:44.948 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-04-24T09:21:44.948 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-04-24T09:21:44.948 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ceph-bd3e039e-0215-11ef-bc93-c7b262605968@mds.e 2024-04-24T09:21:45.454 INFO:journalctl@ceph.mds.e.smithi138.stdout:Apr 24 09:21:45 smithi138 systemd[1]: Stopping Ceph mds.e for bd3e039e-0215-11ef-bc93-c7b262605968... 2024-04-24T09:21:45.955 INFO:journalctl@ceph.mds.e.smithi138.stdout:Apr 24 09:21:45 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-e[60896]: 2024-04-24T09:21:45.555+0000 7f8fa4a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T09:21:45.955 INFO:journalctl@ceph.mds.e.smithi138.stdout:Apr 24 09:21:45 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-e[60896]: 2024-04-24T09:21:45.555+0000 7f8fa4a00700 -1 mds.e *** got signal Terminated *** 2024-04-24T09:21:47.917 INFO:journalctl@ceph.mds.e.smithi138.stdout:Apr 24 09:21:47 smithi138 podman[73503]: 2024-04-24 09:21:47.578604804 +0000 UTC m=+2.164693782 container died f58a43afe68a010e282498166c3bf67976ad8fd7fbf4715e3df58c277d36d285 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-e, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=rishabh-main-apr17-a654945-1f81fb3) 2024-04-24T09:21:48.205 INFO:journalctl@ceph.mds.e.smithi138.stdout:Apr 24 09:21:48 smithi138 podman[73503]: 2024-04-24 09:21:48.050810215 +0000 UTC m=+2.636899186 container cleanup f58a43afe68a010e282498166c3bf67976ad8fd7fbf4715e3df58c277d36d285 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, RELEASE=rishabh-main-apr17-a654945-1f81fb3) 2024-04-24T09:21:48.205 INFO:journalctl@ceph.mds.e.smithi138.stdout:Apr 24 09:21:48 smithi138 bash[73503]: ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-e 2024-04-24T09:21:48.705 INFO:journalctl@ceph.mds.e.smithi138.stdout:Apr 24 09:21:48 smithi138 podman[73517]: 2024-04-24 09:21:48.285645661 +0000 UTC m=+0.693024770 container remove f58a43afe68a010e282498166c3bf67976ad8fd7fbf4715e3df58c277d36d285 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-e, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T09:21:48.829 DEBUG:teuthology.orchestra.run.smithi138:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bd3e039e-0215-11ef-bc93-c7b262605968@mds.e.service' 2024-04-24T09:21:48.932 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T09:21:48.933 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-04-24T09:21:48.933 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-04-24T09:21:48.933 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ceph-bd3e039e-0215-11ef-bc93-c7b262605968@mds.h 2024-04-24T09:21:49.455 INFO:journalctl@ceph.mds.h.smithi138.stdout:Apr 24 09:21:49 smithi138 systemd[1]: Stopping Ceph mds.h for bd3e039e-0215-11ef-bc93-c7b262605968... 2024-04-24T09:21:49.955 INFO:journalctl@ceph.mds.h.smithi138.stdout:Apr 24 09:21:49 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-h[61199]: 2024-04-24T09:21:49.510+0000 7f35bdc00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T09:21:49.955 INFO:journalctl@ceph.mds.h.smithi138.stdout:Apr 24 09:21:49 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-h[61199]: 2024-04-24T09:21:49.510+0000 7f35bdc00700 -1 mds.h *** got signal Terminated *** 2024-04-24T09:21:51.895 INFO:journalctl@ceph.mds.h.smithi138.stdout:Apr 24 09:21:51 smithi138 podman[73624]: 2024-04-24 09:21:51.594383845 +0000 UTC m=+2.212759146 container died 55769a4930c49093995c7798b7554eec429db4162c43c3f845aa00963123d54d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-h, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=rishabh-main-apr17-a654945-1f81fb3, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T09:21:52.205 INFO:journalctl@ceph.mds.h.smithi138.stdout:Apr 24 09:21:52 smithi138 podman[73624]: 2024-04-24 09:21:52.013915742 +0000 UTC m=+2.632291052 container cleanup 55769a4930c49093995c7798b7554eec429db4162c43c3f845aa00963123d54d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-h, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240424, RELEASE=rishabh-main-apr17-a654945-1f81fb3) 2024-04-24T09:21:52.205 INFO:journalctl@ceph.mds.h.smithi138.stdout:Apr 24 09:21:52 smithi138 bash[73624]: ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-h 2024-04-24T09:21:52.704 INFO:journalctl@ceph.mds.h.smithi138.stdout:Apr 24 09:21:52 smithi138 podman[73638]: 2024-04-24 09:21:52.35699501 +0000 UTC m=+0.750035006 container remove 55769a4930c49093995c7798b7554eec429db4162c43c3f845aa00963123d54d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-h, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True) 2024-04-24T09:21:52.960 DEBUG:teuthology.orchestra.run.smithi138:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bd3e039e-0215-11ef-bc93-c7b262605968@mds.h.service' 2024-04-24T09:21:53.067 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T09:21:53.067 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-04-24T09:21:53.067 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-04-24T09:21:53.067 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ceph-bd3e039e-0215-11ef-bc93-c7b262605968@mds.k 2024-04-24T09:21:53.673 INFO:journalctl@ceph.mds.k.smithi138.stdout:Apr 24 09:21:53 smithi138 systemd[1]: Stopping Ceph mds.k for bd3e039e-0215-11ef-bc93-c7b262605968... 2024-04-24T09:21:53.954 INFO:journalctl@ceph.mds.k.smithi138.stdout:Apr 24 09:21:53 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-k[60581]: 2024-04-24T09:21:53.672+0000 7f5d38a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T09:21:53.955 INFO:journalctl@ceph.mds.k.smithi138.stdout:Apr 24 09:21:53 smithi138 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-k[60581]: 2024-04-24T09:21:53.672+0000 7f5d38a00700 -1 mds.k *** got signal Terminated *** 2024-04-24T09:21:55.878 INFO:journalctl@ceph.mds.k.smithi138.stdout:Apr 24 09:21:55 smithi138 podman[73746]: 2024-04-24 09:21:55.565931216 +0000 UTC m=+2.025600763 container died 669f8f50454579bf817dd3613dc9a816ee2c9b7037ae95669a22f9d6583024a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-k, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T09:21:56.204 INFO:journalctl@ceph.mds.k.smithi138.stdout:Apr 24 09:21:56 smithi138 podman[73746]: 2024-04-24 09:21:56.086074089 +0000 UTC m=+2.545743630 container cleanup 669f8f50454579bf817dd3613dc9a816ee2c9b7037ae95669a22f9d6583024a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-k, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, RELEASE=rishabh-main-apr17-a654945-1f81fb3) 2024-04-24T09:21:56.205 INFO:journalctl@ceph.mds.k.smithi138.stdout:Apr 24 09:21:56 smithi138 bash[73746]: ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-k 2024-04-24T09:21:56.705 INFO:journalctl@ceph.mds.k.smithi138.stdout:Apr 24 09:21:56 smithi138 podman[73760]: 2024-04-24 09:21:56.345661918 +0000 UTC m=+0.762278931 container remove 669f8f50454579bf817dd3613dc9a816ee2c9b7037ae95669a22f9d6583024a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-k, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-04-24T09:21:56.786 DEBUG:teuthology.orchestra.run.smithi138:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bd3e039e-0215-11ef-bc93-c7b262605968@mds.k.service' 2024-04-24T09:21:56.892 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T09:21:56.892 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-04-24T09:21:56.893 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-04-24T09:21:56.893 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ceph-bd3e039e-0215-11ef-bc93-c7b262605968@mds.c 2024-04-24T09:21:57.441 INFO:journalctl@ceph.mds.c.smithi189.stdout:Apr 24 09:21:57 smithi189 systemd[1]: Stopping Ceph mds.c for bd3e039e-0215-11ef-bc93-c7b262605968... 2024-04-24T09:21:57.707 INFO:journalctl@ceph.mds.c.smithi189.stdout:Apr 24 09:21:57 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-c[61724]: 2024-04-24T09:21:57.472+0000 7fae44e00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T09:21:57.707 INFO:journalctl@ceph.mds.c.smithi189.stdout:Apr 24 09:21:57 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-c[61724]: 2024-04-24T09:21:57.472+0000 7fae44e00700 -1 mds.c *** got signal Terminated *** 2024-04-24T09:21:59.828 INFO:journalctl@ceph.mds.c.smithi189.stdout:Apr 24 09:21:59 smithi189 podman[77584]: 2024-04-24 09:21:59.536160972 +0000 UTC m=+2.198382591 container died f191a26fb2d2697eef67b2a1c3c0fd9d0cde7094e36f48c8133c50e7443ed4cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-c, maintainer=Guillaume Abrioux , RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=) 2024-04-24T09:22:00.181 INFO:journalctl@ceph.mds.c.smithi189.stdout:Apr 24 09:21:59 smithi189 podman[77584]: 2024-04-24 09:21:59.949001921 +0000 UTC m=+2.611223561 container cleanup f191a26fb2d2697eef67b2a1c3c0fd9d0cde7094e36f48c8133c50e7443ed4cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-c, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, RELEASE=rishabh-main-apr17-a654945-1f81fb3, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424) 2024-04-24T09:22:00.181 INFO:journalctl@ceph.mds.c.smithi189.stdout:Apr 24 09:21:59 smithi189 bash[77584]: ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-c 2024-04-24T09:22:00.457 INFO:journalctl@ceph.mds.c.smithi189.stdout:Apr 24 09:22:00 smithi189 podman[77598]: 2024-04-24 09:22:00.181268359 +0000 UTC m=+0.629051364 container remove f191a26fb2d2697eef67b2a1c3c0fd9d0cde7094e36f48c8133c50e7443ed4cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-c, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=rishabh-main-apr17-a654945-1f81fb3, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux ) 2024-04-24T09:22:00.726 DEBUG:teuthology.orchestra.run.smithi189:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bd3e039e-0215-11ef-bc93-c7b262605968@mds.c.service' 2024-04-24T09:22:00.836 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T09:22:00.836 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-04-24T09:22:00.837 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-04-24T09:22:00.837 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ceph-bd3e039e-0215-11ef-bc93-c7b262605968@mds.f 2024-04-24T09:22:01.451 INFO:journalctl@ceph.mds.f.smithi189.stdout:Apr 24 09:22:00 smithi189 systemd[1]: Stopping Ceph mds.f for bd3e039e-0215-11ef-bc93-c7b262605968... 2024-04-24T09:22:01.707 INFO:journalctl@ceph.mds.f.smithi189.stdout:Apr 24 09:22:01 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-f[61409]: 2024-04-24T09:22:01.449+0000 7fc307c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T09:22:01.707 INFO:journalctl@ceph.mds.f.smithi189.stdout:Apr 24 09:22:01 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-f[61409]: 2024-04-24T09:22:01.449+0000 7fc307c00700 -1 mds.f *** got signal Terminated *** 2024-04-24T09:22:03.806 INFO:journalctl@ceph.mds.f.smithi189.stdout:Apr 24 09:22:03 smithi189 podman[77705]: 2024-04-24 09:22:03.533487699 +0000 UTC m=+2.209059048 container died 2bceb0b43cf6ee27936a72e68367666c978f2be27e8e484ca20f5fc2fdbd84c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-f, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True) 2024-04-24T09:22:04.153 INFO:journalctl@ceph.mds.f.smithi189.stdout:Apr 24 09:22:03 smithi189 podman[77705]: 2024-04-24 09:22:03.927221822 +0000 UTC m=+2.602793208 container cleanup 2bceb0b43cf6ee27936a72e68367666c978f2be27e8e484ca20f5fc2fdbd84c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-f, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T09:22:04.153 INFO:journalctl@ceph.mds.f.smithi189.stdout:Apr 24 09:22:03 smithi189 bash[77705]: ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-f 2024-04-24T09:22:04.457 INFO:journalctl@ceph.mds.f.smithi189.stdout:Apr 24 09:22:04 smithi189 podman[77719]: 2024-04-24 09:22:04.153152038 +0000 UTC m=+0.603124190 container remove 2bceb0b43cf6ee27936a72e68367666c978f2be27e8e484ca20f5fc2fdbd84c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-24T09:22:04.644 DEBUG:teuthology.orchestra.run.smithi189:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bd3e039e-0215-11ef-bc93-c7b262605968@mds.f.service' 2024-04-24T09:22:04.749 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T09:22:04.749 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-04-24T09:22:04.749 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-04-24T09:22:04.749 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ceph-bd3e039e-0215-11ef-bc93-c7b262605968@mds.i 2024-04-24T09:22:05.207 INFO:journalctl@ceph.mds.i.smithi189.stdout:Apr 24 09:22:04 smithi189 systemd[1]: Stopping Ceph mds.i for bd3e039e-0215-11ef-bc93-c7b262605968... 2024-04-24T09:22:05.707 INFO:journalctl@ceph.mds.i.smithi189.stdout:Apr 24 09:22:05 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-i[62031]: 2024-04-24T09:22:05.378+0000 7f7b2f000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T09:22:05.707 INFO:journalctl@ceph.mds.i.smithi189.stdout:Apr 24 09:22:05 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-i[62031]: 2024-04-24T09:22:05.378+0000 7f7b2f000700 -1 mds.i *** got signal Terminated *** 2024-04-24T09:22:07.801 INFO:journalctl@ceph.mds.i.smithi189.stdout:Apr 24 09:22:07 smithi189 podman[77824]: 2024-04-24 09:22:07.523719887 +0000 UTC m=+2.273776775 container died 884f47db48cd847ef4ec1f67d55dd77d43a622e4da92dd7a8da359a9e6bec9d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-i, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T09:22:08.098 INFO:journalctl@ceph.mds.i.smithi189.stdout:Apr 24 09:22:07 smithi189 podman[77824]: 2024-04-24 09:22:07.922352074 +0000 UTC m=+2.672408967 container cleanup 884f47db48cd847ef4ec1f67d55dd77d43a622e4da92dd7a8da359a9e6bec9d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-i, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T09:22:08.098 INFO:journalctl@ceph.mds.i.smithi189.stdout:Apr 24 09:22:07 smithi189 bash[77824]: ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-i 2024-04-24T09:22:08.457 INFO:journalctl@ceph.mds.i.smithi189.stdout:Apr 24 09:22:08 smithi189 podman[77838]: 2024-04-24 09:22:08.098259089 +0000 UTC m=+0.556134210 container remove 884f47db48cd847ef4ec1f67d55dd77d43a622e4da92dd7a8da359a9e6bec9d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-i, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=rishabh-main-apr17-a654945-1f81fb3, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-04-24T09:22:08.602 DEBUG:teuthology.orchestra.run.smithi189:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bd3e039e-0215-11ef-bc93-c7b262605968@mds.i.service' 2024-04-24T09:22:08.703 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T09:22:08.704 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-04-24T09:22:08.704 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-04-24T09:22:08.704 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ceph-bd3e039e-0215-11ef-bc93-c7b262605968@mds.l 2024-04-24T09:22:09.206 INFO:journalctl@ceph.mds.l.smithi189.stdout:Apr 24 09:22:08 smithi189 systemd[1]: Stopping Ceph mds.l for bd3e039e-0215-11ef-bc93-c7b262605968... 2024-04-24T09:22:09.707 INFO:journalctl@ceph.mds.l.smithi189.stdout:Apr 24 09:22:09 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-l[61103]: 2024-04-24T09:22:09.282+0000 7f7eb4c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T09:22:09.707 INFO:journalctl@ceph.mds.l.smithi189.stdout:Apr 24 09:22:09 smithi189 ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-l[61103]: 2024-04-24T09:22:09.282+0000 7f7eb4c00700 -1 mds.l *** got signal Terminated *** 2024-04-24T09:22:11.864 INFO:journalctl@ceph.mds.l.smithi189.stdout:Apr 24 09:22:11 smithi189 podman[77944]: 2024-04-24 09:22:11.534825873 +0000 UTC m=+2.370015249 container died 4160a48691a987f6590b5634e2e8bdf86df71d71d9351109017c9cdb1a4738ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-l, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=rishabh-main-apr17-a654945-1f81fb3, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-04-24T09:22:12.167 INFO:journalctl@ceph.mds.l.smithi189.stdout:Apr 24 09:22:11 smithi189 podman[77944]: 2024-04-24 09:22:11.975365385 +0000 UTC m=+2.810554761 container cleanup 4160a48691a987f6590b5634e2e8bdf86df71d71d9351109017c9cdb1a4738ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-l, RELEASE=rishabh-main-apr17-a654945-1f81fb3, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-04-24T09:22:12.167 INFO:journalctl@ceph.mds.l.smithi189.stdout:Apr 24 09:22:11 smithi189 bash[77944]: ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-l 2024-04-24T09:22:12.457 INFO:journalctl@ceph.mds.l.smithi189.stdout:Apr 24 09:22:12 smithi189 podman[77959]: 2024-04-24 09:22:12.167673754 +0000 UTC m=+0.609679743 container remove 4160a48691a987f6590b5634e2e8bdf86df71d71d9351109017c9cdb1a4738ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:efd0727610781e3b0f3fd91053d49c16375fcec8c20c19c323264ec873568644, name=ceph-bd3e039e-0215-11ef-bc93-c7b262605968-mds-l, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=rishabh-main-apr17-a654945-1f81fb3, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424) 2024-04-24T09:22:12.641 DEBUG:teuthology.orchestra.run.smithi189:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bd3e039e-0215-11ef-bc93-c7b262605968@mds.l.service' 2024-04-24T09:22:12.741 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T09:22:12.741 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-04-24T09:22:12.741 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid bd3e039e-0215-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-24T09:22:13.042 INFO:teuthology.orchestra.run.smithi083.stdout:Deleting cluster with fsid: bd3e039e-0215-11ef-bc93-c7b262605968 2024-04-24T09:22:17.993 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid bd3e039e-0215-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-24T09:22:18.307 INFO:teuthology.orchestra.run.smithi138.stdout:Deleting cluster with fsid: bd3e039e-0215-11ef-bc93-c7b262605968 2024-04-24T09:22:23.095 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid bd3e039e-0215-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-24T09:22:23.411 INFO:teuthology.orchestra.run.smithi189.stdout:Deleting cluster with fsid: bd3e039e-0215-11ef-bc93-c7b262605968 2024-04-24T09:22:28.299 DEBUG:teuthology.orchestra.run.smithi083:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-24T09:22:28.391 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-24T09:22:28.486 DEBUG:teuthology.orchestra.run.smithi189:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-24T09:22:28.575 INFO:tasks.cephadm:Archiving crash dumps... 2024-04-24T09:22:28.577 DEBUG:teuthology.misc:Transferring archived files from smithi083:/var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-04-24_07:32:23-fs-rishabh-main-apr17-a654945-testing-default-smithi/7671216/remote/smithi083/crash 2024-04-24T09:22:28.578 DEBUG:teuthology.orchestra.run.smithi083:> sudo tar c -f - -C /var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/crash -- . 2024-04-24T09:22:28.681 INFO:teuthology.orchestra.run.smithi083.stderr:tar: /var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-24T09:22:28.681 INFO:teuthology.orchestra.run.smithi083.stderr:tar: Error is not recoverable: exiting now 2024-04-24T09:22:28.685 DEBUG:teuthology.misc:Transferring archived files from smithi138:/var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-04-24_07:32:23-fs-rishabh-main-apr17-a654945-testing-default-smithi/7671216/remote/smithi138/crash 2024-04-24T09:22:28.686 DEBUG:teuthology.orchestra.run.smithi138:> sudo tar c -f - -C /var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/crash -- . 2024-04-24T09:22:28.787 INFO:teuthology.orchestra.run.smithi138.stderr:tar: /var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-24T09:22:28.787 INFO:teuthology.orchestra.run.smithi138.stderr:tar: Error is not recoverable: exiting now 2024-04-24T09:22:28.792 DEBUG:teuthology.misc:Transferring archived files from smithi189:/var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-04-24_07:32:23-fs-rishabh-main-apr17-a654945-testing-default-smithi/7671216/remote/smithi189/crash 2024-04-24T09:22:28.793 DEBUG:teuthology.orchestra.run.smithi189:> sudo tar c -f - -C /var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/crash -- . 2024-04-24T09:22:28.886 INFO:teuthology.orchestra.run.smithi189.stderr:tar: /var/lib/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-24T09:22:28.886 INFO:teuthology.orchestra.run.smithi189.stderr:tar: Error is not recoverable: exiting now 2024-04-24T09:22:28.890 INFO:tasks.cephadm:Checking cluster log for badness... 2024-04-24T09:22:28.890 DEBUG:teuthology.orchestra.run.smithi083:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-04-24T09:22:29.058 INFO:teuthology.orchestra.run.smithi083.stdout:2024-04-24T08:55:08.800397+0000 mds.b (mds.0) 34 : cluster [WRN] Scrub error on inode 0x10000000215 (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/COREL) see mds.b log and `damage ls` output for details 2024-04-24T09:22:29.059 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-04-24T09:22:29.060 DEBUG:teuthology.orchestra.run.smithi083:> sudo egrep '\[SEC\]' /var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-04-24T09:22:29.161 DEBUG:teuthology.orchestra.run.smithi083:> sudo egrep '\[ERR\]' /var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-04-24T09:22:29.244 DEBUG:teuthology.orchestra.run.smithi083:> sudo egrep '\[WRN\]' /var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-04-24T09:22:29.326 INFO:teuthology.orchestra.run.smithi083.stdout:2024-04-24T08:55:08.800397+0000 mds.b (mds.0) 34 : cluster [WRN] Scrub error on inode 0x10000000215 (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/COREL) see mds.b log and `damage ls` output for details 2024-04-24T09:22:29.327 INFO:tasks.cephadm:Compressing logs... 2024-04-24T09:22:29.327 DEBUG:teuthology.orchestra.run.smithi083:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-24T09:22:29.332 DEBUG:teuthology.orchestra.run.smithi138:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-24T09:22:29.339 DEBUG:teuthology.orchestra.run.smithi189:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-24T09:22:29.391 INFO:teuthology.orchestra.run.smithi083.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/ceph-client.0.log 2024-04-24T09:22:29.391 INFO:teuthology.orchestra.run.smithi083.stderr:: No such file or directory 2024-04-24T09:22:29.392 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-24T09:22:29.393 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-04-24T09:22:29.394 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph.log 2024-04-24T09:22:29.395 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.log: gzip -5 --verbose -- /var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-24T09:22:29.396 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph-mds.g.log 2024-04-24T09:22:29.397 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph-mds.j.log 2024-04-24T09:22:29.398 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph-osd.3.log 2024-04-24T09:22:29.399 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph-osd.0.log 2024-04-24T09:22:29.400 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph-mgr.x.log 2024-04-24T09:22:29.401 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph-mon.a.log 2024-04-24T09:22:29.401 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph-mgr.x.log: 87.9% -- replaced with /var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-24T09:22:29.402 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph-osd.6.log 2024-04-24T09:22:29.403 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-24T09:22:29.404 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph-mds.a.log 2024-04-24T09:22:29.405 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-24T09:22:29.406 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph-mds.d.log 2024-04-24T09:22:29.406 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph-volume.log: 94.5% -- replaced with /var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph-mds.j.log.gz 2024-04-24T09:22:29.407 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph-osd.9.log 2024-04-24T09:22:29.413 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph-mds.d.log: /var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph-osd.9.log: 94.5% -- replaced with /var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph-mds.d.log.gz 2024-04-24T09:22:29.419 INFO:teuthology.orchestra.run.smithi189.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-04-24T09:22:29.421 INFO:teuthology.orchestra.run.smithi189.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-24T09:22:29.422 INFO:teuthology.orchestra.run.smithi189.stderr:gzip -5 --verbose -- /var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph.log 2024-04-24T09:22:29.423 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph-osd.2.log 2024-04-24T09:22:29.424 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph-mds.f.log 2024-04-24T09:22:29.424 INFO:teuthology.orchestra.run.smithi189.stderr: 92.3% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-24T09:22:29.425 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-24T09:22:29.426 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph-osd.5.log 2024-04-24T09:22:29.428 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph-mds.c.log 2024-04-24T09:22:29.429 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph-mon.c.log 2024-04-24T09:22:29.431 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph-mgr.z.log 2024-04-24T09:22:29.431 INFO:teuthology.orchestra.run.smithi189.stderr: 90.3% -- replaced with /var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-24T09:22:29.432 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph-mon.c.log: 87.9% -- replaced with /var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-24T09:22:29.432 INFO:teuthology.orchestra.run.smithi189.stderr:gzip -5 --verbose -- /var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph-mds.i.log 2024-04-24T09:22:29.434 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-24T09:22:29.440 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph-mds.i.log: /var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph.cephadm.log: 84.1% -- replaced with /var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-24T09:22:29.440 INFO:teuthology.orchestra.run.smithi189.stderr:gzip -5 --verbose -- /var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph-osd.11.log 2024-04-24T09:22:29.441 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph-osd.8.log 2024-04-24T09:22:29.442 INFO:teuthology.orchestra.run.smithi189.stderr:gzip -5 --verbose -- /var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-24T09:22:29.447 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph-mds.l.log 2024-04-24T09:22:29.448 INFO:teuthology.orchestra.run.smithi138.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-24T09:22:29.448 INFO:teuthology.orchestra.run.smithi138.stderr:: No such file or directory 2024-04-24T09:22:29.449 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph-volume.log: 92.7% -- replaced with /var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph-mgr.z.log.gz 2024-04-24T09:22:29.449 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph.log 2024-04-24T09:22:29.451 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-24T09:22:29.452 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph-osd.7.log 2024-04-24T09:22:29.453 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph-mds.b.log 2024-04-24T09:22:29.454 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph-osd.4.log 2024-04-24T09:22:29.456 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph-osd.1.log 2024-04-24T09:22:29.457 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph-mgr.y.log 2024-04-24T09:22:29.458 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph-mds.h.log 2024-04-24T09:22:29.458 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph-osd.10.log 2024-04-24T09:22:29.459 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-24T09:22:29.460 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph-mds.e.log 2024-04-24T09:22:29.461 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-24T09:22:29.462 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph-mon.b.log 2024-04-24T09:22:29.462 INFO:teuthology.orchestra.run.smithi083.stderr: 92.2% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-04-24T09:22:29.463 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph-mds.k.log 2024-04-24T09:22:29.469 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph-mon.b.log: /var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph-mds.k.log: 92.7% -- replaced with /var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph-mgr.y.log.gz 2024-04-24T09:22:29.474 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph-mds.l.log: 90.9% -- replaced with /var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-24T09:22:29.498 INFO:teuthology.orchestra.run.smithi083.stderr: 91.0% -- replaced with /var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-24T09:22:29.525 INFO:teuthology.orchestra.run.smithi083.stderr: 84.4% -- replaced with /var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-24T09:22:29.527 INFO:teuthology.orchestra.run.smithi083.stderr: 90.2% -- replaced with /var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-24T09:22:29.541 INFO:teuthology.orchestra.run.smithi138.stderr: 84.2% -- replaced with /var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-24T09:22:29.557 INFO:teuthology.orchestra.run.smithi138.stderr: 92.0% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-24T09:22:29.586 INFO:teuthology.orchestra.run.smithi138.stderr: 87.9% -- replaced with /var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-24T09:22:29.756 INFO:teuthology.orchestra.run.smithi138.stderr: 90.3% -- replaced with /var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-24T09:22:29.761 INFO:teuthology.orchestra.run.smithi138.stderr: 90.9% -- replaced with /var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-24T09:22:30.081 INFO:teuthology.orchestra.run.smithi083.stderr: 93.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-24T09:22:30.395 INFO:teuthology.orchestra.run.smithi189.stderr: 91.9% -- replaced with /var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph-mon.c.log.gz 2024-04-24T09:22:32.409 INFO:teuthology.orchestra.run.smithi083.stderr: 90.2% -- replaced with /var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph-mgr.x.log.gz 2024-04-24T09:22:33.760 INFO:teuthology.orchestra.run.smithi189.stderr: 94.5% -- replaced with /var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph-mds.l.log.gz 2024-04-24T09:22:34.094 INFO:teuthology.orchestra.run.smithi189.stderr: 94.5% -- replaced with /var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph-mds.f.log.gz 2024-04-24T09:22:35.171 INFO:teuthology.orchestra.run.smithi189.stderr: 94.2% -- replaced with /var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph-mds.c.log.gz 2024-04-24T09:22:45.136 INFO:teuthology.orchestra.run.smithi138.stderr: 91.9% -- replaced with /var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph-mon.b.log.gz 2024-04-24T09:22:45.765 INFO:teuthology.orchestra.run.smithi083.stderr: 90.8% -- replaced with /var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph-mon.a.log.gz 2024-04-24T09:22:50.005 INFO:teuthology.orchestra.run.smithi083.stderr: 95.1% -- replaced with /var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph-mds.g.log.gz 2024-04-24T09:22:52.515 INFO:teuthology.orchestra.run.smithi189.stderr: 91.1% -- replaced with /var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph-osd.11.log.gz 2024-04-24T09:22:58.363 INFO:teuthology.orchestra.run.smithi189.stderr: 90.8% -- replaced with /var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph-osd.8.log.gz 2024-04-24T09:23:00.814 INFO:teuthology.orchestra.run.smithi083.stderr: 94.8% -- replaced with /var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph-mds.a.log.gz 2024-04-24T09:23:02.830 INFO:teuthology.orchestra.run.smithi189.stderr: 91.0% -- replaced with /var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph-osd.2.log.gz 2024-04-24T09:23:16.141 INFO:teuthology.orchestra.run.smithi189.stderr: 90.9% -- replaced with /var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph-osd.5.log.gz 2024-04-24T09:23:53.335 INFO:teuthology.orchestra.run.smithi189.stderr: 94.7% -- replaced with /var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph-mds.i.log.gz 2024-04-24T09:23:53.339 INFO:teuthology.orchestra.run.smithi189.stderr: 2024-04-24T09:23:53.339 INFO:teuthology.orchestra.run.smithi189.stderr:real 1m23.947s 2024-04-24T09:23:53.339 INFO:teuthology.orchestra.run.smithi189.stderr:user 3m34.468s 2024-04-24T09:23:53.339 INFO:teuthology.orchestra.run.smithi189.stderr:sys 0m15.879s 2024-04-24T09:24:31.425 INFO:teuthology.orchestra.run.smithi083.stderr: 91.5% -- replaced with /var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph-osd.0.log.gz 2024-04-24T09:24:38.747 INFO:teuthology.orchestra.run.smithi083.stderr: 91.1% -- replaced with /var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph-osd.6.log.gz 2024-04-24T09:24:41.882 INFO:teuthology.orchestra.run.smithi083.stderr: 91.1% -- replaced with /var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph-osd.3.log.gz 2024-04-24T09:24:47.248 INFO:teuthology.orchestra.run.smithi083.stderr: 91.0% -- replaced with /var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph-osd.9.log.gz 2024-04-24T09:27:34.179 INFO:teuthology.orchestra.run.smithi138.stderr: 91.4% -- replaced with /var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph-osd.1.log.gz 2024-04-24T09:28:06.095 INFO:teuthology.orchestra.run.smithi138.stderr: 91.0% -- replaced with /var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph-osd.10.log.gz 2024-04-24T09:28:51.546 INFO:teuthology.orchestra.run.smithi138.stderr: 91.2% -- replaced with /var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph-osd.7.log.gz 2024-04-24T09:29:35.454 INFO:teuthology.orchestra.run.smithi083.stderr: 92.1% -- replaced with /var/log/ceph/ceph-client.0.log.gz 2024-04-24T09:29:35.457 INFO:teuthology.orchestra.run.smithi083.stderr: 2024-04-24T09:29:35.457 INFO:teuthology.orchestra.run.smithi083.stderr:real 7m6.094s 2024-04-24T09:29:35.457 INFO:teuthology.orchestra.run.smithi083.stderr:user 8m1.405s 2024-04-24T09:29:35.457 INFO:teuthology.orchestra.run.smithi083.stderr:sys 1m3.873s 2024-04-24T09:29:50.174 INFO:teuthology.orchestra.run.smithi138.stderr: 91.2% -- replaced with /var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph-osd.4.log.gz 2024-04-24T09:43:39.194 INFO:teuthology.orchestra.run.smithi138.stderr: 94.2% -- replaced with /var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph-mds.e.log.gz 2024-04-24T09:44:08.202 INFO:teuthology.orchestra.run.smithi138.stderr: 94.5% -- replaced with /var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph-mds.k.log.gz 2024-04-24T09:44:14.874 INFO:teuthology.orchestra.run.smithi138.stderr: 94.0% -- replaced with /var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph-mds.h.log.gz 2024-04-24T09:45:39.098 INFO:teuthology.orchestra.run.smithi138.stderr: 94.5% -- replaced with /var/log/ceph/bd3e039e-0215-11ef-bc93-c7b262605968/ceph-mds.b.log.gz 2024-04-24T09:45:39.101 INFO:teuthology.orchestra.run.smithi138.stderr: 2024-04-24T09:45:39.101 INFO:teuthology.orchestra.run.smithi138.stderr:real 23m9.717s 2024-04-24T09:45:39.102 INFO:teuthology.orchestra.run.smithi138.stderr:user 22m15.278s 2024-04-24T09:45:39.102 INFO:teuthology.orchestra.run.smithi138.stderr:sys 3m4.936s 2024-04-24T09:45:39.103 INFO:tasks.cephadm:Archiving logs... 2024-04-24T09:45:39.103 DEBUG:teuthology.misc:Transferring archived files from smithi083:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-24_07:32:23-fs-rishabh-main-apr17-a654945-testing-default-smithi/7671216/remote/smithi083/log 2024-04-24T09:45:39.105 DEBUG:teuthology.orchestra.run.smithi083:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-24T09:46:43.040 DEBUG:teuthology.misc:Transferring archived files from smithi138:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-24_07:32:23-fs-rishabh-main-apr17-a654945-testing-default-smithi/7671216/remote/smithi138/log 2024-04-24T09:46:43.041 DEBUG:teuthology.orchestra.run.smithi138:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-24T09:48:06.752 DEBUG:teuthology.misc:Transferring archived files from smithi189:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-24_07:32:23-fs-rishabh-main-apr17-a654945-testing-default-smithi/7671216/remote/smithi189/log 2024-04-24T09:48:06.753 DEBUG:teuthology.orchestra.run.smithi189:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-24T09:48:39.626 INFO:tasks.cephadm:Removing cluster... 2024-04-24T09:48:39.641 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid bd3e039e-0215-11ef-bc93-c7b262605968 --force 2024-04-24T09:48:40.176 INFO:teuthology.orchestra.run.smithi083.stdout:Deleting cluster with fsid: bd3e039e-0215-11ef-bc93-c7b262605968 2024-04-24T09:48:41.098 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid bd3e039e-0215-11ef-bc93-c7b262605968 --force 2024-04-24T09:48:41.951 INFO:teuthology.orchestra.run.smithi138.stdout:Deleting cluster with fsid: bd3e039e-0215-11ef-bc93-c7b262605968 2024-04-24T09:48:43.779 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid bd3e039e-0215-11ef-bc93-c7b262605968 --force 2024-04-24T09:48:44.096 INFO:teuthology.orchestra.run.smithi189.stdout:Deleting cluster with fsid: bd3e039e-0215-11ef-bc93-c7b262605968 2024-04-24T09:48:45.374 INFO:tasks.cephadm:Removing cephadm ... 2024-04-24T09:48:45.374 DEBUG:teuthology.orchestra.run.smithi083:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-24T09:48:45.438 DEBUG:teuthology.orchestra.run.smithi138:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-24T09:48:45.508 DEBUG:teuthology.orchestra.run.smithi189:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-24T09:48:45.563 INFO:tasks.cephadm:Teardown complete 2024-04-24T09:48:45.563 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-24T09:48:45.588 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-24T09:48:45.589 DEBUG:teuthology.orchestra.run.smithi083:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-24T09:48:45.606 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-24T09:48:45.618 DEBUG:teuthology.orchestra.run.smithi189:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-24T09:48:45.808 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-24T09:48:45.826 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-24T09:48:45.826 DEBUG:teuthology.orchestra.run.smithi083:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-24T09:48:45.836 DEBUG:teuthology.orchestra.run.smithi138:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-24T09:48:45.845 DEBUG:teuthology.orchestra.run.smithi189:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-24T09:48:45.882 INFO:teuthology.orchestra.run.smithi083.stderr:bash: line 1: ntpq: command not found 2024-04-24T09:48:45.892 INFO:teuthology.orchestra.run.smithi138.stderr:bash: line 1: ntpq: command not found 2024-04-24T09:48:45.905 INFO:teuthology.orchestra.run.smithi189.stderr:bash: line 1: ntpq: command not found 2024-04-24T09:48:45.916 INFO:teuthology.orchestra.run.smithi189.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-24T09:48:45.916 INFO:teuthology.orchestra.run.smithi189.stdout:=============================================================================== 2024-04-24T09:48:45.916 INFO:teuthology.orchestra.run.smithi189.stdout:^- hv01.front.sepia.ceph.com 2 6 377 0 +1676us[+1676us] +/- 591ms 2024-04-24T09:48:45.916 INFO:teuthology.orchestra.run.smithi189.stdout:^* hv02.front.sepia.ceph.com 3 6 377 0 -254us[ -253us] +/- 55ms 2024-04-24T09:48:45.916 INFO:teuthology.orchestra.run.smithi189.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 57 +396us[ +397us] +/- 84ms 2024-04-24T09:48:45.916 INFO:teuthology.orchestra.run.smithi189.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T09:48:45.969 INFO:teuthology.orchestra.run.smithi083.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-24T09:48:45.970 INFO:teuthology.orchestra.run.smithi083.stdout:=============================================================================== 2024-04-24T09:48:45.970 INFO:teuthology.orchestra.run.smithi083.stdout:^- hv01.front.sepia.ceph.com 2 6 377 61 +1692us[+1695us] +/- 590ms 2024-04-24T09:48:45.970 INFO:teuthology.orchestra.run.smithi083.stdout:^* hv02.front.sepia.ceph.com 3 6 377 56 -259us[ -256us] +/- 54ms 2024-04-24T09:48:45.970 INFO:teuthology.orchestra.run.smithi083.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 61 +413us[ +416us] +/- 84ms 2024-04-24T09:48:45.970 INFO:teuthology.orchestra.run.smithi083.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T09:48:46.014 INFO:teuthology.orchestra.run.smithi138.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-24T09:48:46.014 INFO:teuthology.orchestra.run.smithi138.stdout:=============================================================================== 2024-04-24T09:48:46.014 INFO:teuthology.orchestra.run.smithi138.stdout:^- hv01.front.sepia.ceph.com 2 6 377 62 +1690us[+1664us] +/- 590ms 2024-04-24T09:48:46.014 INFO:teuthology.orchestra.run.smithi138.stdout:^* hv02.front.sepia.ceph.com 3 6 377 62 -252us[ -277us] +/- 54ms 2024-04-24T09:48:46.014 INFO:teuthology.orchestra.run.smithi138.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 63 +407us[ +382us] +/- 84ms 2024-04-24T09:48:46.014 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T09:48:46.018 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-24T09:48:46.034 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-24T09:48:46.038 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-24T09:48:46.049 DEBUG:teuthology.orchestra.run.smithi083:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-24T09:48:46.341 DEBUG:teuthology.orchestra.run.smithi138:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-24T09:48:46.875 DEBUG:teuthology.orchestra.run.smithi189:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-24T09:48:47.091 DEBUG:teuthology.orchestra.run.smithi083:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-24T09:48:47.185 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T09:48:47.186 DEBUG:teuthology.orchestra.run.smithi138:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-24T09:48:47.286 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T09:48:47.286 DEBUG:teuthology.orchestra.run.smithi189:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-24T09:48:47.381 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T09:48:47.382 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-24T09:48:48.148 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-24T09:48:48.176 INFO:teuthology.task.internal:Duration was 4783.606718 seconds 2024-04-24T09:48:48.176 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-24T09:48:48.194 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-24T09:48:48.194 DEBUG:teuthology.orchestra.run.smithi083:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-24T09:48:48.203 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-24T09:48:48.212 DEBUG:teuthology.orchestra.run.smithi189:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-24T09:48:48.357 INFO:teuthology.orchestra.run.smithi083.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-24T09:48:48.360 INFO:teuthology.orchestra.run.smithi189.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-24T09:48:48.371 INFO:teuthology.orchestra.run.smithi138.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-24T09:48:48.823 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-24T09:48:48.823 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi083.front.sepia.ceph.com 2024-04-24T09:48:48.823 DEBUG:teuthology.orchestra.run.smithi083:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-24T09:48:48.902 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi138.front.sepia.ceph.com 2024-04-24T09:48:48.903 DEBUG:teuthology.orchestra.run.smithi138:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-24T09:48:48.992 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi189.front.sepia.ceph.com 2024-04-24T09:48:48.993 DEBUG:teuthology.orchestra.run.smithi189:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-24T09:48:49.077 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-24T09:48:49.077 DEBUG:teuthology.orchestra.run.smithi083:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-24T09:48:49.085 DEBUG:teuthology.orchestra.run.smithi138:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-24T09:48:49.096 DEBUG:teuthology.orchestra.run.smithi189:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-24T09:48:49.184 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-24T09:48:49.184 DEBUG:teuthology.orchestra.run.smithi083:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-24T09:48:49.193 DEBUG:teuthology.orchestra.run.smithi138:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-24T09:48:49.199 DEBUG:teuthology.orchestra.run.smithi189:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-24T09:48:49.757 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-24T09:48:49.773 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-24T09:48:49.773 DEBUG:teuthology.orchestra.run.smithi083:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-24T09:48:49.862 DEBUG:teuthology.orchestra.run.smithi138:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-24T09:48:49.955 DEBUG:teuthology.orchestra.run.smithi189:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-24T09:48:50.047 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-24T09:48:50.059 DEBUG:teuthology.orchestra.run.smithi083:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-24T09:48:50.067 DEBUG:teuthology.orchestra.run.smithi138:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-24T09:48:50.075 DEBUG:teuthology.orchestra.run.smithi189:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-24T09:48:50.175 INFO:teuthology.orchestra.run.smithi189.stdout:kernel.core_pattern = core 2024-04-24T09:48:50.177 INFO:teuthology.orchestra.run.smithi083.stdout:kernel.core_pattern = core 2024-04-24T09:48:50.192 INFO:teuthology.orchestra.run.smithi138.stdout:kernel.core_pattern = core 2024-04-24T09:48:50.254 DEBUG:teuthology.orchestra.run.smithi083:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-24T09:48:50.311 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T09:48:50.312 DEBUG:teuthology.orchestra.run.smithi138:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-24T09:48:50.368 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T09:48:50.368 DEBUG:teuthology.orchestra.run.smithi189:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-24T09:48:50.427 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T09:48:50.427 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-24T09:48:50.446 INFO:teuthology.task.internal:Transferring archived files... 2024-04-24T09:48:50.447 DEBUG:teuthology.misc:Transferring archived files from smithi083:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-24_07:32:23-fs-rishabh-main-apr17-a654945-testing-default-smithi/7671216/remote/smithi083 2024-04-24T09:48:50.448 DEBUG:teuthology.orchestra.run.smithi083:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-24T09:48:50.553 DEBUG:teuthology.misc:Transferring archived files from smithi138:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-24_07:32:23-fs-rishabh-main-apr17-a654945-testing-default-smithi/7671216/remote/smithi138 2024-04-24T09:48:50.554 DEBUG:teuthology.orchestra.run.smithi138:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-24T09:48:50.655 DEBUG:teuthology.misc:Transferring archived files from smithi189:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-24_07:32:23-fs-rishabh-main-apr17-a654945-testing-default-smithi/7671216/remote/smithi189 2024-04-24T09:48:50.656 DEBUG:teuthology.orchestra.run.smithi189:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-24T09:48:50.758 INFO:teuthology.task.internal:Removing archive directory... 2024-04-24T09:48:50.758 DEBUG:teuthology.orchestra.run.smithi083:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-24T09:48:50.766 DEBUG:teuthology.orchestra.run.smithi138:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-24T09:48:50.775 DEBUG:teuthology.orchestra.run.smithi189:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-24T09:48:50.834 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-24T09:48:50.847 INFO:teuthology.task.internal:Not uploading archives. 2024-04-24T09:48:50.847 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-24T09:48:50.885 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-24T09:48:50.886 DEBUG:teuthology.orchestra.run.smithi083:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-24T09:48:50.894 DEBUG:teuthology.orchestra.run.smithi138:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-24T09:48:50.902 DEBUG:teuthology.orchestra.run.smithi189:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-24T09:48:50.945 INFO:teuthology.orchestra.run.smithi083.stdout: 393229 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 24 09:48 /home/ubuntu/cephtest 2024-04-24T09:48:50.951 INFO:teuthology.orchestra.run.smithi138.stdout: 393249 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 24 09:48 /home/ubuntu/cephtest 2024-04-24T09:48:50.958 INFO:teuthology.orchestra.run.smithi189.stdout: 393940 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 24 09:48 /home/ubuntu/cephtest 2024-04-24T09:48:50.961 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-24T09:48:50.978 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-24T09:48:51.020 INFO:teuthology.nuke:Checking targets against current locks 2024-04-24T09:48:51.047 DEBUG:teuthology.nuke:shortname: smithi083 2024-04-24T09:48:51.047 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-24T09:48:51.073 DEBUG:teuthology.nuke:shortname: smithi138 2024-04-24T09:48:51.074 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-24T09:48:51.079 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi083.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-24_07:32:23-fs-rishabh-main-apr17-a654945-testing-default-smithi/7671216', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-24 08:19:54.065672', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBN0CclY6xwBKkTDnYoCHmRU6US1iwRUaHtWxWScJwFdW6vbqYpL5iTPgY/IStOEZqEQytMfRDSrD6ly+IGqpoLw='} 2024-04-24T09:48:51.101 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi138.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-24_07:32:23-fs-rishabh-main-apr17-a654945-testing-default-smithi/7671216', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-24 08:19:54.064423', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMbippNFPXqvRGlEyFz9OxiHoGjV3HYInY/bEfMEvHclwS46aMECzZQlVddaVAGWvk3UTQxHoTotU7jYrRybgZ8='} 2024-04-24T09:48:51.109 DEBUG:teuthology.nuke:shortname: smithi189 2024-04-24T09:48:51.109 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-24T09:48:51.141 INFO:teuthology.orchestra.console.smithi083:Power off 2024-04-24T09:48:51.142 DEBUG:teuthology.orchestra.console.smithi083:pexpect command: ipmitool -H smithi083.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-24T09:48:51.157 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi189.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-24_07:32:23-fs-rishabh-main-apr17-a654945-testing-default-smithi/7671216', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-24 08:19:54.062842', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLDz1dff8UPWDDhhdt+mKp29SBFeAd++C1OzFH+0BSbsAuDzBolYH2YUB2p2MYiBvDqNYdtNgjrSJNL7Hj7knNk='} 2024-04-24T09:48:51.167 INFO:teuthology.orchestra.console.smithi138:Power off 2024-04-24T09:48:51.167 DEBUG:teuthology.orchestra.console.smithi138:pexpect command: ipmitool -H smithi138.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-24T09:48:51.177 DEBUG:teuthology.orchestra.console.smithi083:power off output: Chassis Power Control: Down/Off 2024-04-24T09:48:51.177 DEBUG:teuthology.orchestra.console.smithi083:pexpect command: ipmitool -H smithi083.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-24T09:48:51.194 DEBUG:teuthology.orchestra.console.smithi138:power off output: Chassis Power Control: Down/Off 2024-04-24T09:48:51.194 DEBUG:teuthology.orchestra.console.smithi138:pexpect command: ipmitool -H smithi138.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-24T09:48:51.209 DEBUG:teuthology.orchestra.console.smithi083:check power output: Chassis Power is on 2024-04-24T09:48:51.219 DEBUG:teuthology.orchestra.console.smithi138:check power output: Chassis Power is on 2024-04-24T09:48:51.237 INFO:teuthology.orchestra.console.smithi189:Power off 2024-04-24T09:48:51.237 DEBUG:teuthology.orchestra.console.smithi189:pexpect command: ipmitool -H smithi189.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-24T09:48:51.263 DEBUG:teuthology.orchestra.console.smithi189:power off output: Chassis Power Control: Down/Off 2024-04-24T09:48:51.263 DEBUG:teuthology.orchestra.console.smithi189:pexpect command: ipmitool -H smithi189.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-24T09:48:51.287 DEBUG:teuthology.orchestra.console.smithi189:check power output: Chassis Power is on 2024-04-24T09:48:55.210 DEBUG:teuthology.orchestra.console.smithi083:pexpect command: ipmitool -H smithi083.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-24T09:48:55.221 DEBUG:teuthology.orchestra.console.smithi138:pexpect command: ipmitool -H smithi138.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-24T09:48:55.289 DEBUG:teuthology.orchestra.console.smithi189:pexpect command: ipmitool -H smithi189.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-24T09:48:55.322 DEBUG:teuthology.orchestra.console.smithi083:check power output: Chassis Power is on 2024-04-24T09:48:55.332 DEBUG:teuthology.orchestra.console.smithi138:check power output: Chassis Power is on 2024-04-24T09:48:55.400 DEBUG:teuthology.orchestra.console.smithi189:check power output: Chassis Power is on 2024-04-24T09:48:59.326 DEBUG:teuthology.orchestra.console.smithi083:pexpect command: ipmitool -H smithi083.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-24T09:48:59.341 DEBUG:teuthology.orchestra.console.smithi138:pexpect command: ipmitool -H smithi138.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-24T09:48:59.402 DEBUG:teuthology.orchestra.console.smithi189:pexpect command: ipmitool -H smithi189.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-24T09:48:59.443 DEBUG:teuthology.orchestra.console.smithi083:check power output: Chassis Power is off 2024-04-24T09:48:59.452 DEBUG:teuthology.orchestra.console.smithi138:check power output: Chassis Power is off 2024-04-24T09:48:59.513 DEBUG:teuthology.orchestra.console.smithi189:check power output: Chassis Power is on 2024-04-24T09:48:59.545 INFO:teuthology.orchestra.console.smithi083:Power off completed 2024-04-24T09:48:59.554 INFO:teuthology.orchestra.console.smithi138:Power off completed 2024-04-24T09:49:03.518 DEBUG:teuthology.orchestra.console.smithi189:pexpect command: ipmitool -H smithi189.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-24T09:49:03.629 DEBUG:teuthology.orchestra.console.smithi189:check power output: Chassis Power is off 2024-04-24T09:49:03.730 INFO:teuthology.orchestra.console.smithi189:Power off completed 2024-04-24T09:49:04.609 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/suites/dbench}} duration: 4783.606717586517 failure_reason: '"2024-04-24T08:55:08.800397+0000 mds.b (mds.0) 34 : cluster [WRN] Scrub error on inode 0x10000000215 (/volumes/qa/sv_0/e395bed6-5bba-41b8-a256-3505e4afcca2/client.0/tmp/clients/client0/~dmtmp/COREL) see mds.b log and `damage ls` output for details" in cluster log' flavor: default mon.a-kernel-sha1: 806ec36b21129f113ce356c007dd59694d340b37 mon.b-kernel-sha1: 806ec36b21129f113ce356c007dd59694d340b37 mon.c-kernel-sha1: 806ec36b21129f113ce356c007dd59694d340b37 owner: scheduled_rishabh@teuthology success: false 2024-04-24T09:49:04.609 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-24T09:49:04.704 INFO:teuthology.run:FAIL